2024-06-10T00:42:31.798 INFO:root:teuthology version: 0.0.1.dev303+g861a8dc 2024-06-10T00:42:31.798 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} --owner scheduled_teuthology@teuthology --name teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi --archive /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747357 -- /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747357/orig.config.yaml 2024-06-10T00:42:31.850 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-10T00:42:31.937 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747357 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7747357' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 7b7a5d375cf4e4afab96b9d4d229b699ae8304bb owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 178 sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba sleep_before_teardown: 0 subset: 130/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 7b7a5d375cf4e4afab96b9d4d229b699ae8304bb syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi007.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN51Ks1krYNoPJnG5l7Os8Z7u4ac3Dtmq1t5T3U5KTdgHTu9yl5RmUJrhW0wICd0fjaoXhElsBKWW43N6gJWKoA= smithi078.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIPLlo0JcZVXEs1CkptrjY+uwonHqUXybRnfRucZtVjszclXSoOHVx/PS+GaFxfGMvnf/iNoSNO0jd2MFs4BNY0= smithi175.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLA0jJ6WO/JTt7HxfZikvs+lAG8KGVQOfwdy2F77856B2BxLbMWmJgU2iUI+CTKERxY1Q4Vo6tgkuebH3ky/uNE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 861a8dcf7aa816a26e13f039336f7ed0a9aec0fa timestamp: 2024-06-09_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-06-10T00:42:31.938 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa; will attempt to use it 2024-06-10T00:42:31.938 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa/tasks 2024-06-10T00:42:31.939 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-06-10T00:42:31.940 INFO:teuthology.task.internal:Checking packages... 2024-06-10T00:42:31.961 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3d89accfe3201286f000e065a1faea2a88c7a6ba' 2024-06-10T00:42:31.961 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-06-10T00:42:31.961 INFO:teuthology.packaging:ref: None 2024-06-10T00:42:31.961 INFO:teuthology.packaging:tag: None 2024-06-10T00:42:31.961 INFO:teuthology.packaging:branch: squid 2024-06-10T00:42:31.961 INFO:teuthology.packaging:sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:42:31.961 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-06-10T00:42:32.200 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2970.g7b7a5d37 2024-06-10T00:42:32.202 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-06-10T00:42:32.213 INFO:teuthology.task.internal:no buildpackages task found 2024-06-10T00:42:32.213 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-06-10T00:42:32.221 INFO:teuthology.task.internal:Saving configuration 2024-06-10T00:42:32.240 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-06-10T00:42:32.248 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-06-10T00:42:32.274 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747357', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-10 00:37:42.857333', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN51Ks1krYNoPJnG5l7Os8Z7u4ac3Dtmq1t5T3U5KTdgHTu9yl5RmUJrhW0wICd0fjaoXhElsBKWW43N6gJWKoA='} 2024-06-10T00:42:32.298 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi078.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747357', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-10 00:37:42.860253', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIPLlo0JcZVXEs1CkptrjY+uwonHqUXybRnfRucZtVjszclXSoOHVx/PS+GaFxfGMvnf/iNoSNO0jd2MFs4BNY0='} 2024-06-10T00:42:32.328 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi175.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747357', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-10 00:37:42.858954', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLA0jJ6WO/JTt7HxfZikvs+lAG8KGVQOfwdy2F77856B2BxLbMWmJgU2iUI+CTKERxY1Q4Vo6tgkuebH3ky/uNE='} 2024-06-10T00:42:32.328 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-06-10T00:42:32.336 INFO:teuthology.task.internal:roles: ubuntu@smithi007.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-06-10T00:42:32.336 INFO:teuthology.task.internal:roles: ubuntu@smithi078.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-06-10T00:42:32.336 INFO:teuthology.task.internal:roles: ubuntu@smithi175.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-06-10T00:42:32.336 INFO:teuthology.run_tasks:Running task console_log... 2024-06-10T00:42:32.426 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff938aefbe0>, signals=[15]) 2024-06-10T00:42:32.426 INFO:teuthology.run_tasks:Running task internal.connect... 2024-06-10T00:42:32.435 INFO:teuthology.task.internal:Opening connections... 2024-06-10T00:42:32.435 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi007.front.sepia.ceph.com 2024-06-10T00:42:32.436 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T00:42:32.513 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi078.front.sepia.ceph.com 2024-06-10T00:42:32.514 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T00:42:32.588 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi175.front.sepia.ceph.com 2024-06-10T00:42:32.589 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T00:42:32.663 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-06-10T00:42:32.671 DEBUG:teuthology.orchestra.run.smithi007:> uname -m 2024-06-10T00:42:32.689 INFO:teuthology.orchestra.run.smithi007.stdout:x86_64 2024-06-10T00:42:32.689 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release 2024-06-10T00:42:32.747 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="CentOS Stream" 2024-06-10T00:42:32.747 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="9" 2024-06-10T00:42:32.747 INFO:teuthology.orchestra.run.smithi007.stdout:ID="centos" 2024-06-10T00:42:32.747 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE="rhel fedora" 2024-06-10T00:42:32.747 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="9" 2024-06-10T00:42:32.747 INFO:teuthology.orchestra.run.smithi007.stdout:PLATFORM_ID="platform:el9" 2024-06-10T00:42:32.747 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-10T00:42:32.747 INFO:teuthology.orchestra.run.smithi007.stdout:ANSI_COLOR="0;31" 2024-06-10T00:42:32.747 INFO:teuthology.orchestra.run.smithi007.stdout:LOGO="fedora-logo-icon" 2024-06-10T00:42:32.748 INFO:teuthology.orchestra.run.smithi007.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-10T00:42:32.748 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://centos.org/" 2024-06-10T00:42:32.748 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-10T00:42:32.748 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-10T00:42:32.748 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-10T00:42:32.749 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2024-06-10T00:42:32.766 DEBUG:teuthology.orchestra.run.smithi078:> uname -m 2024-06-10T00:42:32.783 INFO:teuthology.orchestra.run.smithi078.stdout:x86_64 2024-06-10T00:42:32.783 DEBUG:teuthology.orchestra.run.smithi078:> cat /etc/os-release 2024-06-10T00:42:32.839 INFO:teuthology.orchestra.run.smithi078.stdout:NAME="CentOS Stream" 2024-06-10T00:42:32.839 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION="9" 2024-06-10T00:42:32.839 INFO:teuthology.orchestra.run.smithi078.stdout:ID="centos" 2024-06-10T00:42:32.840 INFO:teuthology.orchestra.run.smithi078.stdout:ID_LIKE="rhel fedora" 2024-06-10T00:42:32.840 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION_ID="9" 2024-06-10T00:42:32.840 INFO:teuthology.orchestra.run.smithi078.stdout:PLATFORM_ID="platform:el9" 2024-06-10T00:42:32.840 INFO:teuthology.orchestra.run.smithi078.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-10T00:42:32.840 INFO:teuthology.orchestra.run.smithi078.stdout:ANSI_COLOR="0;31" 2024-06-10T00:42:32.840 INFO:teuthology.orchestra.run.smithi078.stdout:LOGO="fedora-logo-icon" 2024-06-10T00:42:32.840 INFO:teuthology.orchestra.run.smithi078.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-10T00:42:32.840 INFO:teuthology.orchestra.run.smithi078.stdout:HOME_URL="https://centos.org/" 2024-06-10T00:42:32.840 INFO:teuthology.orchestra.run.smithi078.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-10T00:42:32.840 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-10T00:42:32.840 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-10T00:42:32.840 INFO:teuthology.lock.ops:Updating smithi078.front.sepia.ceph.com on lock server 2024-06-10T00:42:32.858 DEBUG:teuthology.orchestra.run.smithi175:> uname -m 2024-06-10T00:42:32.876 INFO:teuthology.orchestra.run.smithi175.stdout:x86_64 2024-06-10T00:42:32.876 DEBUG:teuthology.orchestra.run.smithi175:> cat /etc/os-release 2024-06-10T00:42:32.933 INFO:teuthology.orchestra.run.smithi175.stdout:NAME="CentOS Stream" 2024-06-10T00:42:32.933 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION="9" 2024-06-10T00:42:32.933 INFO:teuthology.orchestra.run.smithi175.stdout:ID="centos" 2024-06-10T00:42:32.933 INFO:teuthology.orchestra.run.smithi175.stdout:ID_LIKE="rhel fedora" 2024-06-10T00:42:32.933 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_ID="9" 2024-06-10T00:42:32.933 INFO:teuthology.orchestra.run.smithi175.stdout:PLATFORM_ID="platform:el9" 2024-06-10T00:42:32.933 INFO:teuthology.orchestra.run.smithi175.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-10T00:42:32.933 INFO:teuthology.orchestra.run.smithi175.stdout:ANSI_COLOR="0;31" 2024-06-10T00:42:32.933 INFO:teuthology.orchestra.run.smithi175.stdout:LOGO="fedora-logo-icon" 2024-06-10T00:42:32.933 INFO:teuthology.orchestra.run.smithi175.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-10T00:42:32.934 INFO:teuthology.orchestra.run.smithi175.stdout:HOME_URL="https://centos.org/" 2024-06-10T00:42:32.934 INFO:teuthology.orchestra.run.smithi175.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-10T00:42:32.934 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-10T00:42:32.934 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-10T00:42:32.934 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server 2024-06-10T00:42:32.958 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-06-10T00:42:32.969 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-06-10T00:42:32.975 INFO:teuthology.task.internal:Checking for old test directory... 2024-06-10T00:42:32.975 DEBUG:teuthology.orchestra.run.smithi007:> test '!' -e /home/ubuntu/cephtest 2024-06-10T00:42:32.978 DEBUG:teuthology.orchestra.run.smithi078:> test '!' -e /home/ubuntu/cephtest 2024-06-10T00:42:32.980 DEBUG:teuthology.orchestra.run.smithi175:> test '!' -e /home/ubuntu/cephtest 2024-06-10T00:42:32.997 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-06-10T00:42:33.005 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-06-10T00:42:33.006 DEBUG:teuthology.orchestra.run.smithi007:> test -z $(ls -A /var/lib/ceph) 2024-06-10T00:42:33.036 DEBUG:teuthology.orchestra.run.smithi078:> test -z $(ls -A /var/lib/ceph) 2024-06-10T00:42:33.038 DEBUG:teuthology.orchestra.run.smithi175:> test -z $(ls -A /var/lib/ceph) 2024-06-10T00:42:33.153 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-06-10T00:42:33.233 INFO:teuthology.run_tasks:Running task kernel... 2024-06-10T00:42:33.247 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-06-10T00:42:33.247 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-06-10T00:42:33.247 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-06-10T00:42:33.248 DEBUG:teuthology.orchestra.run.smithi007:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-10T00:42:33.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T00:42:33.265 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-06-10T00:42:33.324 INFO:teuthology.orchestra.run.smithi007.stdout:5.14.0-437.el9.x86_64 2024-06-10T00:42:33.324 INFO:teuthology.task.kernel:Running kernel on smithi007: 5.14.0-437.el9.x86_64 2024-06-10T00:42:33.324 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum install -y kernel 2024-06-10T00:42:35.584 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.1 MB 00:00 2024-06-10T00:42:38.986 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 8.5 MB/s | 19 MB 00:02 2024-06-10T00:42:44.180 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 4.1 MB/s | 6.2 MB 00:01 2024-06-10T00:42:46.086 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 17 kB 00:00 2024-06-10T00:42:47.072 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 25 MB/s | 21 MB 00:00 2024-06-10T00:42:52.594 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-06-10T00:42:54.676 INFO:teuthology.orchestra.run.smithi007.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-06-10T00:42:54.676 INFO:teuthology.orchestra.run.smithi007.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-06-10T00:42:54.770 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-06-10T00:42:54.781 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-06-10T00:42:54.782 INFO:teuthology.orchestra.run.smithi007.stdout: Package Architecture Version Repository Size 2024-06-10T00:42:54.782 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-06-10T00:42:54.782 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-06-10T00:42:54.782 INFO:teuthology.orchestra.run.smithi007.stdout: kernel x86_64 5.14.0-452.el9 baseos 893 k 2024-06-10T00:42:54.782 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-06-10T00:42:54.782 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-core x86_64 5.14.0-452.el9 baseos 16 M 2024-06-10T00:42:54.782 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules x86_64 5.14.0-452.el9 baseos 35 M 2024-06-10T00:42:54.782 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-core x86_64 5.14.0-452.el9 baseos 29 M 2024-06-10T00:42:54.782 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:42:54.782 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-06-10T00:42:54.782 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-06-10T00:42:54.783 INFO:teuthology.orchestra.run.smithi007.stdout:Install 4 Packages 2024-06-10T00:42:54.783 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:42:54.783 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 80 M 2024-06-10T00:42:54.783 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 124 M 2024-06-10T00:42:54.783 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-06-10T00:42:54.990 INFO:teuthology.orchestra.run.smithi007.stdout:[MIRROR] kernel-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:42:54.990 INFO:teuthology.orchestra.run.smithi007.stdout:[MIRROR] kernel-core-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:42:54.994 INFO:teuthology.orchestra.run.smithi007.stdout:[MIRROR] kernel-modules-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:42:55.146 INFO:teuthology.orchestra.run.smithi007.stdout:(1/4): kernel-5.14.0-452.el9.x86_64.rpm 3.6 MB/s | 893 kB 00:00 2024-06-10T00:42:55.421 INFO:teuthology.orchestra.run.smithi007.stdout:(2/4): kernel-core-5.14.0-452.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-06-10T00:42:55.797 INFO:teuthology.orchestra.run.smithi007.stdout:(3/4): kernel-modules-core-5.14.0-452.el9.x86_6 44 MB/s | 29 MB 00:00 2024-06-10T00:42:56.122 INFO:teuthology.orchestra.run.smithi007.stdout:(4/4): kernel-modules-5.14.0-452.el9.x86_64.rpm 28 MB/s | 35 MB 00:01 2024-06-10T00:42:56.125 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-06-10T00:42:56.125 INFO:teuthology.orchestra.run.smithi007.stdout:Total 60 MB/s | 80 MB 00:01 2024-06-10T00:42:56.855 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-06-10T00:42:57.271 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-06-10T00:42:57.271 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-06-10T00:42:58.281 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-06-10T00:42:58.281 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-06-10T00:43:00.830 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-06-10T00:43:01.476 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-modules-core-5.14.0-452.el9.x86_64 1/4 2024-06-10T00:43:01.565 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-10T00:43:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-10T00:43:04.480 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-10T00:43:09.570 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-10T00:43:09.619 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-5.14.0-452.el9.x86_64 4/4 2024-06-10T00:43:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-10T00:43:45.710 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 4/4 2024-06-10T00:43:45.752 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 4/4 2024-06-10T00:43:46.058 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-5.14.0-452.el9.x86_64 4/4 2024-06-10T00:43:46.058 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-5.14.0-452.el9.x86_64 1/4 2024-06-10T00:43:46.058 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-10T00:43:46.058 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-10T00:43:46.824 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-10T00:43:46.824 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:43:46.824 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-06-10T00:43:46.825 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-5.14.0-452.el9.x86_64 2024-06-10T00:43:46.825 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-core-5.14.0-452.el9.x86_64 2024-06-10T00:43:46.825 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-5.14.0-452.el9.x86_64 2024-06-10T00:43:46.825 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-core-5.14.0-452.el9.x86_64 2024-06-10T00:43:46.825 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:43:46.825 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-06-10T00:43:47.181 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q kernel | sort -rV | head -n 1 2024-06-10T00:43:47.271 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-10T00:43:47.271 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-10T00:43:47.271 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-452.el9.x86_64 2024-06-10T00:43:47.271 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-10T00:43:47.272 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-10T00:43:47.272 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-06-10T00:43:47.272 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi007.front.sepia.ceph.com, path=None, version=distro) 2024-06-10T00:43:47.272 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q kernel | sort -rV | head -n 1 2024-06-10T00:43:47.325 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-10T00:43:47.325 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-10T00:43:47.326 DEBUG:teuthology.orchestra.run.smithi007:> sudo rpm -qi grub2-tools 2024-06-10T00:43:47.356 INFO:teuthology.orchestra.run.smithi007.stdout:Name : grub2-tools 2024-06-10T00:43:47.356 INFO:teuthology.orchestra.run.smithi007.stdout:Epoch : 1 2024-06-10T00:43:47.356 INFO:teuthology.orchestra.run.smithi007.stdout:Version : 2.06 2024-06-10T00:43:47.356 INFO:teuthology.orchestra.run.smithi007.stdout:Release : 68.el9 2024-06-10T00:43:47.357 INFO:teuthology.orchestra.run.smithi007.stdout:Architecture: x86_64 2024-06-10T00:43:47.357 INFO:teuthology.orchestra.run.smithi007.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-10T00:43:47.357 INFO:teuthology.orchestra.run.smithi007.stdout:Group : Unspecified 2024-06-10T00:43:47.357 INFO:teuthology.orchestra.run.smithi007.stdout:Size : 8264254 2024-06-10T00:43:47.357 INFO:teuthology.orchestra.run.smithi007.stdout:License : GPLv3+ 2024-06-10T00:43:47.357 INFO:teuthology.orchestra.run.smithi007.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-10T00:43:47.357 INFO:teuthology.orchestra.run.smithi007.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-10T00:43:47.357 INFO:teuthology.orchestra.run.smithi007.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-10T00:43:47.357 INFO:teuthology.orchestra.run.smithi007.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-10T00:43:47.357 INFO:teuthology.orchestra.run.smithi007.stdout:Packager : builder@centos.org 2024-06-10T00:43:47.357 INFO:teuthology.orchestra.run.smithi007.stdout:Vendor : CentOS 2024-06-10T00:43:47.357 INFO:teuthology.orchestra.run.smithi007.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-10T00:43:47.357 INFO:teuthology.orchestra.run.smithi007.stdout:Summary : Support tools for GRUB. 2024-06-10T00:43:47.357 INFO:teuthology.orchestra.run.smithi007.stdout:Description : 2024-06-10T00:43:47.357 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:43:47.358 INFO:teuthology.orchestra.run.smithi007.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-10T00:43:47.358 INFO:teuthology.orchestra.run.smithi007.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-10T00:43:47.358 INFO:teuthology.orchestra.run.smithi007.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-10T00:43:47.358 INFO:teuthology.orchestra.run.smithi007.stdout:hardware devices. 2024-06-10T00:43:47.358 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:43:47.358 INFO:teuthology.orchestra.run.smithi007.stdout:This subpackage provides tools for support of all platforms. 2024-06-10T00:43:47.359 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-10T00:43:47.359 INFO:teuthology.task.kernel:Updating grub on smithi007 to boot 5.14.0-452.el9.x86_64 2024-06-10T00:43:47.359 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-10T00:43:48.004 INFO:teuthology.orchestra.run.smithi007.stderr:Generating grub configuration file ... 2024-06-10T00:43:49.366 INFO:teuthology.orchestra.run.smithi007.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-10T00:43:49.414 INFO:teuthology.orchestra.run.smithi007.stderr:done 2024-06-10T00:43:49.416 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-06-10T00:43:49.431 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.8RW3zW3owz 2024-06-10T00:43:49.431 DEBUG:teuthology.orchestra.run.smithi007:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.8RW3zW3owz 2024-06-10T00:43:49.537 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.8RW3zW3owz 2024-06-10T00:43:49.636 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.8RW3zW3owz is 6KB 2024-06-10T00:43:49.684 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.8RW3zW3owz 2024-06-10T00:43:49.699 DEBUG:teuthology.orchestra.run.smithi007:> sudo /bin/ls /boot/loader/entries || true 2024-06-10T00:43:49.769 INFO:teuthology.orchestra.run.smithi007.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-10T00:43:49.769 INFO:teuthology.orchestra.run.smithi007.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-10T00:43:49.769 INFO:teuthology.orchestra.run.smithi007.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-10T00:43:49.769 INFO:teuthology.orchestra.run.smithi007.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64.conf 2024-06-10T00:43:49.771 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64 2024-06-10T00:43:49.921 DEBUG:teuthology.orchestra.run.smithi007:> sudo shutdown -r now 2024-06-10T00:44:19.924 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-06-10T00:44:19.924 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-06-10T00:44:19.925 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T00:45:19.928 DEBUG:teuthology.orchestra.remote:timed out 2024-06-10T00:45:28.929 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-06-10T00:45:28.930 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T00:46:00.747 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-06-10T00:46:01.131 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-06-10T00:46:01.131 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-452.el9.x86_64"... 2024-06-10T00:46:01.131 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-06-10T00:46:01.146 INFO:teuthology.orchestra.run.smithi007.stdout:5.14.0-452.el9.x86_64 2024-06-10T00:46:01.146 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-452.el9.x86_64 vs 5.14.0-452.el9.x86_64 2024-06-10T00:46:01.146 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-10T00:46:01.146 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-10T00:46:02.148 DEBUG:teuthology.parallel:result is None 2024-06-10T00:46:02.148 INFO:teuthology.run_tasks:Running task internal.base... 2024-06-10T00:46:02.156 INFO:teuthology.task.internal:Creating test directory... 2024-06-10T00:46:02.156 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-10T00:46:02.159 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-10T00:46:02.161 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-10T00:46:02.179 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-06-10T00:46:02.227 INFO:teuthology.run_tasks:Running task internal.archive... 2024-06-10T00:46:02.238 INFO:teuthology.task.internal:Creating archive directory... 2024-06-10T00:46:02.238 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-10T00:46:02.240 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-10T00:46:02.242 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-10T00:46:02.430 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-06-10T00:46:02.436 INFO:teuthology.task.internal:Enabling coredump saving... 2024-06-10T00:46:02.437 DEBUG:teuthology.orchestra.run.smithi078:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-10T00:46:02.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T00:46:02.453 DEBUG:teuthology.orchestra.run.smithi175:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-10T00:46:02.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T00:46:02.469 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-10T00:46:02.472 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-10T00:46:02.495 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-10T00:46:02.524 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-10T00:46:02.544 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-10T00:46:02.545 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-10T00:46:02.551 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-10T00:46:02.568 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-10T00:46:02.588 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-10T00:46:02.590 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-06-10T00:46:02.597 INFO:teuthology.task.internal:Configuring sudo... 2024-06-10T00:46:02.597 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-10T00:46:02.600 DEBUG:teuthology.orchestra.run.smithi078:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-10T00:46:02.602 DEBUG:teuthology.orchestra.run.smithi175:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-10T00:46:02.656 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-06-10T00:46:02.665 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-06-10T00:46:02.665 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-10T00:46:02.670 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-10T00:46:02.673 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-10T00:46:02.713 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-10T00:46:02.773 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-10T00:46:02.826 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-10T00:46:02.912 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-10T00:46:02.956 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-10T00:46:02.957 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-10T00:46:03.027 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-10T00:46:03.071 DEBUG:teuthology.orchestra.run.smithi078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-10T00:46:03.122 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-10T00:46:03.206 DEBUG:teuthology.orchestra.run.smithi078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-10T00:46:03.248 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-06-10T00:46:03.248 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-10T00:46:03.314 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-10T00:46:03.355 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-10T00:46:03.413 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-10T00:46:03.489 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-10T00:46:03.537 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-06-10T00:46:03.537 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-10T00:46:03.602 DEBUG:teuthology.orchestra.run.smithi007:> sudo service rsyslog restart 2024-06-10T00:46:03.605 DEBUG:teuthology.orchestra.run.smithi078:> sudo service rsyslog restart 2024-06-10T00:46:03.607 DEBUG:teuthology.orchestra.run.smithi175:> sudo service rsyslog restart 2024-06-10T00:46:03.641 INFO:teuthology.orchestra.run.smithi078.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-10T00:46:03.648 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-10T00:46:03.685 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-10T00:46:04.126 INFO:teuthology.run_tasks:Running task internal.timer... 2024-06-10T00:46:04.133 INFO:teuthology.task.internal:Starting timer... 2024-06-10T00:46:04.133 INFO:teuthology.run_tasks:Running task pcp... 2024-06-10T00:46:04.144 INFO:teuthology.run_tasks:Running task selinux... 2024-06-10T00:46:04.151 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-06-10T00:46:04.152 DEBUG:teuthology.orchestra.run.smithi007:> sudo service auditd rotate 2024-06-10T00:46:04.242 INFO:teuthology.orchestra.run.smithi007.stdout:Rotating logs: 2024-06-10T00:46:04.244 DEBUG:teuthology.orchestra.run.smithi078:> sudo service auditd rotate 2024-06-10T00:46:04.296 INFO:teuthology.orchestra.run.smithi078.stdout:Rotating logs: 2024-06-10T00:46:04.297 DEBUG:teuthology.orchestra.run.smithi175:> sudo service auditd rotate 2024-06-10T00:46:04.352 INFO:teuthology.orchestra.run.smithi175.stdout:Rotating logs: 2024-06-10T00:46:04.354 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-06-10T00:46:04.355 DEBUG:teuthology.orchestra.run.smithi007:> /usr/sbin/getenforce 2024-06-10T00:46:04.374 INFO:teuthology.orchestra.run.smithi007.stdout:Permissive 2024-06-10T00:46:04.374 DEBUG:teuthology.orchestra.run.smithi078:> /usr/sbin/getenforce 2024-06-10T00:46:04.411 INFO:teuthology.orchestra.run.smithi078.stdout:Permissive 2024-06-10T00:46:04.411 DEBUG:teuthology.orchestra.run.smithi175:> /usr/sbin/getenforce 2024-06-10T00:46:04.435 INFO:teuthology.orchestra.run.smithi175.stdout:Permissive 2024-06-10T00:46:04.435 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi007.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi078.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi175.front.sepia.ceph.com': 'permissive'} 2024-06-10T00:46:04.435 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-10T00:46:04.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T00:46:04.462 DEBUG:teuthology.orchestra.run.smithi078:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-10T00:46:04.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T00:46:04.490 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-10T00:46:04.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T00:46:04.516 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-06-10T00:46:04.516 DEBUG:teuthology.orchestra.run.smithi007:> sudo /usr/sbin/setenforce permissive 2024-06-10T00:46:04.569 DEBUG:teuthology.orchestra.run.smithi078:> sudo /usr/sbin/setenforce permissive 2024-06-10T00:46:04.595 DEBUG:teuthology.orchestra.run.smithi175:> sudo /usr/sbin/setenforce permissive 2024-06-10T00:46:04.620 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-06-10T00:46:04.629 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-06-10T00:46:04.632 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-06-10T00:46:04.768 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-06-10T00:46:04.778 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-06-10T00:46:04.780 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi007.front.sepia.ceph.com,smithi078.front.sepia.ceph.com,smithi175.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-06-10T00:50:39.972 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi007.front.sepia.ceph.com'), Remote(name='ubuntu@smithi078.front.sepia.ceph.com'), Remote(name='ubuntu@smithi175.front.sepia.ceph.com')] 2024-06-10T00:50:39.973 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-06-10T00:50:39.975 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T00:50:40.054 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-06-10T00:50:40.131 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-06-10T00:50:40.131 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-06-10T00:50:40.132 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T00:50:40.208 DEBUG:teuthology.orchestra.run.smithi078:> true 2024-06-10T00:50:40.289 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-06-10T00:50:40.289 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-06-10T00:50:40.290 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T00:50:40.363 DEBUG:teuthology.orchestra.run.smithi175:> true 2024-06-10T00:50:40.397 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-06-10T00:50:40.398 INFO:teuthology.run_tasks:Running task clock... 2024-06-10T00:50:40.412 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-06-10T00:50:40.413 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-10T00:50:40.413 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-10T00:50:40.415 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-10T00:50:40.416 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-10T00:50:40.418 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-10T00:50:40.418 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-10T00:50:40.443 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-10T00:50:40.445 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-10T00:50:40.457 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-10T00:50:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-10T00:50:40.467 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-10T00:50:40.479 INFO:teuthology.orchestra.run.smithi078.stderr:sudo: ntpd: command not found 2024-06-10T00:50:40.481 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-10T00:50:40.487 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: ntpd: command not found 2024-06-10T00:50:40.489 INFO:teuthology.orchestra.run.smithi078.stdout:506 Cannot talk to daemon 2024-06-10T00:50:40.499 INFO:teuthology.orchestra.run.smithi007.stdout:506 Cannot talk to daemon 2024-06-10T00:50:40.502 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-10T00:50:40.504 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: ntpd: command not found 2024-06-10T00:50:40.514 INFO:teuthology.orchestra.run.smithi175.stdout:506 Cannot talk to daemon 2024-06-10T00:50:40.515 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-10T00:50:40.515 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-10T00:50:40.527 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-10T00:50:40.529 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-10T00:50:40.540 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-10T00:50:40.567 INFO:teuthology.orchestra.run.smithi078.stderr:bash: line 1: ntpq: command not found 2024-06-10T00:50:40.571 INFO:teuthology.orchestra.run.smithi078.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-10T00:50:40.571 INFO:teuthology.orchestra.run.smithi078.stdout:=============================================================================== 2024-06-10T00:50:40.571 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:50:40.572 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:50:40.572 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:50:40.572 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:50:40.584 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-06-10T00:50:40.587 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-10T00:50:40.588 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-06-10T00:50:40.588 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:50:40.588 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:50:40.588 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:50:40.588 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:50:40.594 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2024-06-10T00:50:40.598 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-10T00:50:40.598 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2024-06-10T00:50:40.598 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:50:40.598 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:50:40.598 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:50:40.598 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:50:40.598 INFO:teuthology.run_tasks:Running task install... 2024-06-10T00:50:40.609 DEBUG:teuthology.task.install:project ceph 2024-06-10T00:50:40.609 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'}} 2024-06-10T00:50:40.609 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'} 2024-06-10T00:50:40.610 INFO:teuthology.task.install:Using flavor: default 2024-06-10T00:50:40.617 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-06-10T00:50:40.617 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-06-10T00:50:40.618 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-10T00:50:40.618 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:50:40.620 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-10T00:50:40.620 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:50:40.621 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-10T00:50:40.621 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:50:40.765 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-10T00:50:40.766 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-10T00:50:40.778 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-10T00:50:40.778 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-10T00:50:40.838 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-10T00:50:40.839 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-10T00:50:40.896 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-10T00:50:40.897 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-06-10T00:50:40.897 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-10T00:50:40.922 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-10T00:50:40.922 DEBUG:teuthology.orchestra.run.smithi175:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-10T00:50:40.943 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-10T00:50:40.943 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-06-10T00:50:40.943 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-10T00:50:40.960 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-10T00:50:40.960 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-10T00:50:40.960 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-10T00:50:40.967 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-10T00:50:40.967 DEBUG:teuthology.orchestra.run.smithi078:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-10T00:50:40.987 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-10T00:50:40.987 DEBUG:teuthology.orchestra.run.smithi007:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-10T00:50:40.988 DEBUG:teuthology.orchestra.run.smithi175:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-10T00:50:41.033 DEBUG:teuthology.orchestra.run.smithi078:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-10T00:50:41.054 DEBUG:teuthology.orchestra.run.smithi007:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-10T00:50:41.060 DEBUG:teuthology.orchestra.run.smithi175:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-10T00:50:41.105 DEBUG:teuthology.orchestra.run.smithi078:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-10T00:50:41.122 INFO:teuthology.orchestra.run.smithi175.stdout:check_obsoletes = 1 2024-06-10T00:50:41.124 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum clean all 2024-06-10T00:50:41.130 DEBUG:teuthology.orchestra.run.smithi007:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-10T00:50:41.168 INFO:teuthology.orchestra.run.smithi078.stdout:check_obsoletes = 1 2024-06-10T00:50:41.170 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum clean all 2024-06-10T00:50:41.196 INFO:teuthology.orchestra.run.smithi007.stdout:check_obsoletes = 1 2024-06-10T00:50:41.197 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum clean all 2024-06-10T00:50:41.346 INFO:teuthology.orchestra.run.smithi175.stdout:45 files removed 2024-06-10T00:50:41.373 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-10T00:50:41.393 INFO:teuthology.orchestra.run.smithi078.stdout:45 files removed 2024-06-10T00:50:41.420 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-10T00:50:41.431 INFO:teuthology.orchestra.run.smithi007.stdout:45 files removed 2024-06-10T00:50:41.459 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-10T00:50:41.964 INFO:teuthology.orchestra.run.smithi175.stdout:ceph packages for x86_64 251 kB/s | 83 kB 00:00 2024-06-10T00:50:42.052 INFO:teuthology.orchestra.run.smithi078.stdout:ceph packages for x86_64 224 kB/s | 83 kB 00:00 2024-06-10T00:50:42.068 INFO:teuthology.orchestra.run.smithi007.stdout:ceph packages for x86_64 261 kB/s | 83 kB 00:00 2024-06-10T00:50:42.289 INFO:teuthology.orchestra.run.smithi175.stdout:ceph noarch packages 41 kB/s | 11 kB 00:00 2024-06-10T00:50:42.373 INFO:teuthology.orchestra.run.smithi078.stdout:ceph noarch packages 42 kB/s | 11 kB 00:00 2024-06-10T00:50:42.444 INFO:teuthology.orchestra.run.smithi007.stdout:ceph noarch packages 36 kB/s | 11 kB 00:00 2024-06-10T00:50:42.698 INFO:teuthology.orchestra.run.smithi078.stdout:ceph source packages 6.5 kB/s | 1.9 kB 00:00 2024-06-10T00:50:42.715 INFO:teuthology.orchestra.run.smithi175.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2024-06-10T00:50:42.803 INFO:teuthology.orchestra.run.smithi007.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-06-10T00:50:44.309 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - BaseOS 5.2 MB/s | 8.1 MB 00:01 2024-06-10T00:50:44.486 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.1 MB 00:01 2024-06-10T00:50:45.416 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 3.1 MB/s | 8.1 MB 00:02 2024-06-10T00:50:47.959 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - AppStream 7.5 MB/s | 19 MB 00:02 2024-06-10T00:50:54.206 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 2.5 MB/s | 19 MB 00:07 2024-06-10T00:50:56.169 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - CRB 1.3 MB/s | 6.2 MB 00:04 2024-06-10T00:50:57.793 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 17 kB 00:00 2024-06-10T00:50:58.827 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - AppStream 1.4 MB/s | 19 MB 00:13 2024-06-10T00:50:59.012 INFO:teuthology.orchestra.run.smithi175.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-06-10T00:50:59.408 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 4.1 MB/s | 6.2 MB 00:01 2024-06-10T00:51:01.081 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 17 kB 00:00 2024-06-10T00:51:02.010 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 27 MB/s | 21 MB 00:00 2024-06-10T00:51:04.201 INFO:teuthology.orchestra.run.smithi175.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-06-10T00:51:04.260 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.2 MB 00:01 2024-06-10T00:51:05.826 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 17 kB 00:00 2024-06-10T00:51:06.494 INFO:teuthology.orchestra.run.smithi175.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-10T00:51:06.495 INFO:teuthology.orchestra.run.smithi175.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-10T00:51:06.506 INFO:teuthology.orchestra.run.smithi175.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-10T00:51:06.507 INFO:teuthology.orchestra.run.smithi175.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-10T00:51:06.508 INFO:teuthology.orchestra.run.smithi175.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-10T00:51:06.509 INFO:teuthology.orchestra.run.smithi175.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-10T00:51:06.510 INFO:teuthology.orchestra.run.smithi175.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-10T00:51:06.513 INFO:teuthology.orchestra.run.smithi175.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-10T00:51:06.513 INFO:teuthology.orchestra.run.smithi175.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-10T00:51:06.576 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-06-10T00:51:06.601 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-06-10T00:51:06.602 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-06-10T00:51:06.602 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-06-10T00:51:06.602 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-06-10T00:51:06.602 INFO:teuthology.orchestra.run.smithi175.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-10T00:51:06.602 INFO:teuthology.orchestra.run.smithi175.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-10T00:51:06.602 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-10T00:51:06.602 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-10T00:51:06.602 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-10T00:51:06.603 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-10T00:51:06.603 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-10T00:51:06.603 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-10T00:51:06.603 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-10T00:51:06.603 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-10T00:51:06.603 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-10T00:51:06.603 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-10T00:51:06.603 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-10T00:51:06.603 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-10T00:51:06.603 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-10T00:51:06.603 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-10T00:51:06.603 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-10T00:51:06.603 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-10T00:51:06.603 INFO:teuthology.orchestra.run.smithi175.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-10T00:51:06.604 INFO:teuthology.orchestra.run.smithi175.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-10T00:51:06.604 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-10T00:51:06.604 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-10T00:51:06.604 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-10T00:51:06.604 INFO:teuthology.orchestra.run.smithi175.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-10T00:51:06.604 INFO:teuthology.orchestra.run.smithi175.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-10T00:51:06.604 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-10T00:51:06.604 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-10T00:51:06.604 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-10T00:51:06.604 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-10T00:51:06.604 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-10T00:51:06.604 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-10T00:51:06.604 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-10T00:51:06.604 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-10T00:51:06.605 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-10T00:51:06.605 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-10T00:51:06.605 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-10T00:51:06.605 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-10T00:51:06.605 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrading: 2024-06-10T00:51:06.605 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-10T00:51:06.605 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-10T00:51:06.605 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-10T00:51:06.605 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-10T00:51:06.605 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-10T00:51:06.605 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-10T00:51:06.605 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-10T00:51:06.605 INFO:teuthology.orchestra.run.smithi175.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-10T00:51:06.605 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-10T00:51:06.606 INFO:teuthology.orchestra.run.smithi175.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-10T00:51:06.606 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-10T00:51:06.606 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-06-10T00:51:06.606 INFO:teuthology.orchestra.run.smithi175.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-10T00:51:06.606 INFO:teuthology.orchestra.run.smithi175.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-10T00:51:06.606 INFO:teuthology.orchestra.run.smithi175.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-10T00:51:06.606 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-10T00:51:06.606 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-10T00:51:06.606 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-10T00:51:06.606 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-10T00:51:06.606 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-10T00:51:06.606 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-10T00:51:06.606 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-10T00:51:06.607 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-10T00:51:06.607 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-10T00:51:06.607 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-10T00:51:06.607 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-10T00:51:06.607 INFO:teuthology.orchestra.run.smithi175.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-10T00:51:06.607 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-10T00:51:06.607 INFO:teuthology.orchestra.run.smithi175.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-10T00:51:06.607 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-10T00:51:06.607 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-10T00:51:06.607 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-10T00:51:06.607 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-10T00:51:06.607 INFO:teuthology.orchestra.run.smithi175.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-10T00:51:06.607 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-10T00:51:06.608 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-10T00:51:06.608 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-10T00:51:06.608 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-10T00:51:06.608 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-10T00:51:06.608 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-10T00:51:06.608 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-10T00:51:06.608 INFO:teuthology.orchestra.run.smithi175.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-10T00:51:06.608 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-10T00:51:06.608 INFO:teuthology.orchestra.run.smithi175.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-10T00:51:06.608 INFO:teuthology.orchestra.run.smithi175.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-10T00:51:06.608 INFO:teuthology.orchestra.run.smithi175.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-10T00:51:06.608 INFO:teuthology.orchestra.run.smithi175.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-10T00:51:06.608 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-10T00:51:06.608 INFO:teuthology.orchestra.run.smithi175.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-10T00:51:06.609 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-10T00:51:06.609 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-10T00:51:06.609 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-10T00:51:06.609 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-10T00:51:06.609 INFO:teuthology.orchestra.run.smithi175.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-10T00:51:06.609 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-10T00:51:06.609 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-10T00:51:06.609 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-10T00:51:06.609 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-10T00:51:06.609 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-10T00:51:06.609 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-10T00:51:06.609 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-10T00:51:06.610 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-10T00:51:06.610 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-10T00:51:06.610 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-10T00:51:06.610 INFO:teuthology.orchestra.run.smithi175.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-10T00:51:06.610 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-10T00:51:06.610 INFO:teuthology.orchestra.run.smithi175.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-10T00:51:06.610 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth noarch 1:2.29.0-2.el9 epel 205 k 2024-06-10T00:51:06.610 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-10T00:51:06.610 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-10T00:51:06.610 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-10T00:51:06.610 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-10T00:51:06.611 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-10T00:51:06.611 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-10T00:51:06.611 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-10T00:51:06.611 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-10T00:51:06.611 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-10T00:51:06.611 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-10T00:51:06.611 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-10T00:51:06.611 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-10T00:51:06.611 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-10T00:51:06.611 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-10T00:51:06.611 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-10T00:51:06.611 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-10T00:51:06.611 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-10T00:51:06.611 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-10T00:51:06.612 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-10T00:51:06.612 INFO:teuthology.orchestra.run.smithi175.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-10T00:51:06.612 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-10T00:51:06.612 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-10T00:51:06.612 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-10T00:51:06.612 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-10T00:51:06.612 INFO:teuthology.orchestra.run.smithi175.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-10T00:51:06.612 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-10T00:51:06.612 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-10T00:51:06.612 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-10T00:51:06.612 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-10T00:51:06.612 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-10T00:51:06.612 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-10T00:51:06.613 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-10T00:51:06.613 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-10T00:51:06.613 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-10T00:51:06.613 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-10T00:51:06.613 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-10T00:51:06.613 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-10T00:51:06.613 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-10T00:51:06.613 INFO:teuthology.orchestra.run.smithi175.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-10T00:51:06.613 INFO:teuthology.orchestra.run.smithi175.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-10T00:51:06.613 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-10T00:51:06.613 INFO:teuthology.orchestra.run.smithi175.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-10T00:51:06.613 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-10T00:51:06.614 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-10T00:51:06.614 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-10T00:51:06.614 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-10T00:51:06.614 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-10T00:51:06.614 INFO:teuthology.orchestra.run.smithi175.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-10T00:51:06.614 INFO:teuthology.orchestra.run.smithi175.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-10T00:51:06.614 INFO:teuthology.orchestra.run.smithi175.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-10T00:51:06.614 INFO:teuthology.orchestra.run.smithi175.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-10T00:51:06.614 INFO:teuthology.orchestra.run.smithi175.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-10T00:51:06.614 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-10T00:51:06.614 INFO:teuthology.orchestra.run.smithi175.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-10T00:51:06.614 INFO:teuthology.orchestra.run.smithi175.stdout:Installing weak dependencies: 2024-06-10T00:51:06.614 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-10T00:51:06.614 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-10T00:51:06.615 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-06-10T00:51:06.615 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-06-10T00:51:06.615 INFO:teuthology.orchestra.run.smithi175.stdout:Install 148 Packages 2024-06-10T00:51:06.615 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrade 11 Packages 2024-06-10T00:51:06.615 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-10T00:51:06.615 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 225 M 2024-06-10T00:51:06.615 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-06-10T00:51:06.711 INFO:teuthology.orchestra.run.smithi078.stdout:Extra Packages for Enterprise Linux 28 MB/s | 21 MB 00:00 2024-06-10T00:51:07.399 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-06-10T00:51:07.610 INFO:teuthology.orchestra.run.smithi175.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 31 kB/s | 6.6 kB 00:00 2024-06-10T00:51:08.244 INFO:teuthology.orchestra.run.smithi175.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 1.4 MB/s | 890 kB 00:00 2024-06-10T00:51:08.336 INFO:teuthology.orchestra.run.smithi175.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 1.6 MB/s | 145 kB 00:00 2024-06-10T00:51:08.703 INFO:teuthology.orchestra.run.smithi175.stdout:(4/159): ceph-common-19.0.0-2946.g3d89accf.el9. 16 MB/s | 21 MB 00:01 2024-06-10T00:51:08.820 INFO:teuthology.orchestra.run.smithi175.stdout:(5/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 13 MB/s | 1.5 MB 00:00 2024-06-10T00:51:09.120 INFO:teuthology.orchestra.run.smithi175.stdout:(6/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 16 MB/s | 4.7 MB 00:00 2024-06-10T00:51:09.170 INFO:teuthology.orchestra.run.smithi175.stdout:(7/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 2.8 MB/s | 2.4 MB 00:00 2024-06-10T00:51:09.346 INFO:teuthology.orchestra.run.smithi175.stdout:(8/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 2.7 MB/s | 5.4 MB 00:01 2024-06-10T00:51:09.438 INFO:teuthology.orchestra.run.smithi175.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 275 kB/s | 25 kB 00:00 2024-06-10T00:51:09.817 INFO:teuthology.orchestra.run.smithi007.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-10T00:51:09.818 INFO:teuthology.orchestra.run.smithi007.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-10T00:51:09.829 INFO:teuthology.orchestra.run.smithi007.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-10T00:51:09.830 INFO:teuthology.orchestra.run.smithi007.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-10T00:51:09.831 INFO:teuthology.orchestra.run.smithi007.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-10T00:51:09.831 INFO:teuthology.orchestra.run.smithi007.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-10T00:51:09.833 INFO:teuthology.orchestra.run.smithi007.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-10T00:51:09.835 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-10T00:51:09.836 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-10T00:51:09.904 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-06-10T00:51:09.920 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-06-10T00:51:09.920 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-06-10T00:51:09.920 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-06-10T00:51:09.920 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-06-10T00:51:09.920 INFO:teuthology.orchestra.run.smithi007.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-10T00:51:09.920 INFO:teuthology.orchestra.run.smithi007.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-10T00:51:09.920 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-10T00:51:09.920 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-10T00:51:09.920 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-10T00:51:09.920 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-10T00:51:09.920 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-10T00:51:09.920 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-10T00:51:09.921 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-10T00:51:09.921 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-10T00:51:09.921 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-10T00:51:09.921 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-10T00:51:09.921 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-10T00:51:09.921 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-10T00:51:09.921 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-10T00:51:09.921 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-10T00:51:09.921 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-10T00:51:09.921 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-10T00:51:09.921 INFO:teuthology.orchestra.run.smithi007.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-10T00:51:09.921 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-10T00:51:09.922 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-10T00:51:09.922 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-10T00:51:09.922 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-10T00:51:09.922 INFO:teuthology.orchestra.run.smithi007.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-10T00:51:09.922 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-10T00:51:09.922 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-10T00:51:09.922 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-10T00:51:09.922 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-10T00:51:09.922 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-10T00:51:09.922 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-10T00:51:09.922 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-10T00:51:09.922 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-10T00:51:09.923 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-10T00:51:09.923 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-10T00:51:09.923 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-10T00:51:09.923 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-10T00:51:09.923 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-10T00:51:09.923 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrading: 2024-06-10T00:51:09.924 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-10T00:51:09.924 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-10T00:51:09.924 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-10T00:51:09.924 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-10T00:51:09.924 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-10T00:51:09.924 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-10T00:51:09.924 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-10T00:51:09.924 INFO:teuthology.orchestra.run.smithi007.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-10T00:51:09.924 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-10T00:51:09.925 INFO:teuthology.orchestra.run.smithi007.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-10T00:51:09.925 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-10T00:51:09.925 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-06-10T00:51:09.925 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-10T00:51:09.925 INFO:teuthology.orchestra.run.smithi007.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-10T00:51:09.925 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-10T00:51:09.925 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-10T00:51:09.925 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-10T00:51:09.925 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-10T00:51:09.925 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-10T00:51:09.925 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-10T00:51:09.925 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-10T00:51:09.925 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-10T00:51:09.926 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-10T00:51:09.926 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-10T00:51:09.926 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-10T00:51:09.926 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-10T00:51:09.926 INFO:teuthology.orchestra.run.smithi007.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-10T00:51:09.926 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-10T00:51:09.926 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-10T00:51:09.926 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-10T00:51:09.926 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-10T00:51:09.926 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-10T00:51:09.926 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-10T00:51:09.926 INFO:teuthology.orchestra.run.smithi007.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-10T00:51:09.927 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-10T00:51:09.927 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-10T00:51:09.927 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-10T00:51:09.927 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-10T00:51:09.927 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-10T00:51:09.927 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-10T00:51:09.927 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-10T00:51:09.927 INFO:teuthology.orchestra.run.smithi007.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-10T00:51:09.927 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-10T00:51:09.927 INFO:teuthology.orchestra.run.smithi007.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-10T00:51:09.927 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-10T00:51:09.927 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-10T00:51:09.928 INFO:teuthology.orchestra.run.smithi007.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-10T00:51:09.928 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-10T00:51:09.928 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-10T00:51:09.928 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-10T00:51:09.928 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-10T00:51:09.928 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-10T00:51:09.928 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-10T00:51:09.928 INFO:teuthology.orchestra.run.smithi007.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-10T00:51:09.928 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-10T00:51:09.928 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-10T00:51:09.928 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-10T00:51:09.928 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-10T00:51:09.928 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-10T00:51:09.929 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-10T00:51:09.929 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-10T00:51:09.929 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-10T00:51:09.929 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-10T00:51:09.929 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-10T00:51:09.929 INFO:teuthology.orchestra.run.smithi007.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-10T00:51:09.929 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-10T00:51:09.929 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-10T00:51:09.929 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth noarch 1:2.29.0-2.el9 epel 205 k 2024-06-10T00:51:09.929 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-10T00:51:09.929 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-10T00:51:09.929 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-10T00:51:09.929 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-10T00:51:09.930 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-10T00:51:09.930 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-10T00:51:09.930 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-10T00:51:09.930 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-10T00:51:09.930 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-10T00:51:09.930 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-10T00:51:09.930 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-10T00:51:09.930 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-10T00:51:09.930 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-10T00:51:09.930 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-10T00:51:09.930 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-10T00:51:09.930 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-10T00:51:09.930 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-10T00:51:09.931 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-10T00:51:09.931 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-10T00:51:09.931 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-10T00:51:09.931 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-10T00:51:09.931 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-10T00:51:09.931 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-10T00:51:09.931 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-10T00:51:09.931 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-10T00:51:09.931 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-10T00:51:09.931 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-10T00:51:09.931 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-10T00:51:09.931 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-10T00:51:09.932 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-10T00:51:09.932 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-10T00:51:09.932 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-10T00:51:09.932 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-10T00:51:09.932 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-10T00:51:09.932 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-10T00:51:09.932 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-10T00:51:09.932 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-10T00:51:09.932 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-10T00:51:09.932 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-10T00:51:09.932 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-10T00:51:09.932 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-10T00:51:09.932 INFO:teuthology.orchestra.run.smithi007.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-10T00:51:09.933 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-10T00:51:09.933 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-10T00:51:09.933 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-10T00:51:09.933 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-10T00:51:09.933 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-10T00:51:09.933 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-10T00:51:09.933 INFO:teuthology.orchestra.run.smithi007.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-10T00:51:09.933 INFO:teuthology.orchestra.run.smithi007.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-10T00:51:09.933 INFO:teuthology.orchestra.run.smithi007.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-10T00:51:09.933 INFO:teuthology.orchestra.run.smithi007.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-10T00:51:09.933 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-10T00:51:09.933 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-10T00:51:09.933 INFO:teuthology.orchestra.run.smithi007.stdout:Installing weak dependencies: 2024-06-10T00:51:09.933 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-10T00:51:09.934 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:51:09.934 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-06-10T00:51:09.934 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-06-10T00:51:09.934 INFO:teuthology.orchestra.run.smithi007.stdout:Install 148 Packages 2024-06-10T00:51:09.934 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrade 11 Packages 2024-06-10T00:51:09.934 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:51:09.934 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 225 M 2024-06-10T00:51:09.935 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-06-10T00:51:10.030 INFO:teuthology.orchestra.run.smithi175.stdout:(10/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 18 MB/s | 17 MB 00:00 2024-06-10T00:51:10.088 INFO:teuthology.orchestra.run.smithi175.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 3.7 MB/s | 221 kB 00:00 2024-06-10T00:51:10.138 INFO:teuthology.orchestra.run.smithi175.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 542 kB/s | 27 kB 00:00 2024-06-10T00:51:10.189 INFO:teuthology.orchestra.run.smithi175.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 633 kB/s | 32 kB 00:00 2024-06-10T00:51:10.272 INFO:teuthology.orchestra.run.smithi175.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 8.8 MB/s | 747 kB 00:00 2024-06-10T00:51:10.331 INFO:teuthology.orchestra.run.smithi175.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 2.7 MB/s | 163 kB 00:00 2024-06-10T00:51:10.398 INFO:teuthology.orchestra.run.smithi175.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 1.9 MB/s | 127 kB 00:00 2024-06-10T00:51:10.464 INFO:teuthology.orchestra.run.smithi175.stdout:(17/159): libradosstriper1-19.0.0-2946.g3d89acc 7.4 MB/s | 501 kB 00:00 2024-06-10T00:51:10.698 INFO:teuthology.orchestra.run.smithi175.stdout:(18/159): ceph-radosgw-19.0.0-2946.g3d89accf.el 6.7 MB/s | 10 MB 00:01 2024-06-10T00:51:10.799 INFO:teuthology.orchestra.run.smithi175.stdout:(19/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 16 MB/s | 5.3 MB 00:00 2024-06-10T00:51:10.824 INFO:teuthology.orchestra.run.smithi175.stdout:(20/159): python3-ceph-argparse-19.0.0-2946.g3d 361 kB/s | 45 kB 00:00 2024-06-10T00:51:10.849 INFO:teuthology.orchestra.run.smithi175.stdout:(21/159): python3-ceph-common-19.0.0-2946.g3d89 2.6 MB/s | 135 kB 00:00 2024-06-10T00:51:10.883 INFO:teuthology.orchestra.run.smithi175.stdout:(22/159): python3-cephfs-19.0.0-2946.g3d89accf. 2.7 MB/s | 162 kB 00:00 2024-06-10T00:51:10.908 INFO:teuthology.orchestra.run.smithi175.stdout:(23/159): python3-rados-19.0.0-2946.g3d89accf.e 5.4 MB/s | 321 kB 00:00 2024-06-10T00:51:10.950 INFO:teuthology.orchestra.run.smithi175.stdout:(24/159): python3-rbd-19.0.0-2946.g3d89accf.el9 4.4 MB/s | 300 kB 00:00 2024-06-10T00:51:10.975 INFO:teuthology.orchestra.run.smithi175.stdout:(25/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.5 MB/s | 99 kB 00:00 2024-06-10T00:51:11.017 INFO:teuthology.orchestra.run.smithi175.stdout:(26/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.2 MB/s | 85 kB 00:00 2024-06-10T00:51:11.025 INFO:teuthology.orchestra.run.smithi007.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 30 kB/s | 6.6 kB 00:00 2024-06-10T00:51:11.126 INFO:teuthology.orchestra.run.smithi175.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 1.5 MB/s | 171 kB 00:00 2024-06-10T00:51:11.184 INFO:teuthology.orchestra.run.smithi175.stdout:(28/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 15 MB/s | 3.1 MB 00:00 2024-06-10T00:51:11.210 INFO:teuthology.orchestra.run.smithi175.stdout:(29/159): ceph-grafana-dashboards-19.0.0-2946.g 380 kB/s | 32 kB 00:00 2024-06-10T00:51:11.235 INFO:teuthology.orchestra.run.smithi175.stdout:(30/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 2.8 MB/s | 142 kB 00:00 2024-06-10T00:51:11.725 INFO:teuthology.orchestra.run.smithi007.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 1.2 MB/s | 890 kB 00:00 2024-06-10T00:51:11.777 INFO:teuthology.orchestra.run.smithi175.stdout:(31/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 6.2 MB/s | 3.5 MB 00:00 2024-06-10T00:51:11.876 INFO:teuthology.orchestra.run.smithi007.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 966 kB/s | 145 kB 00:00 2024-06-10T00:51:11.903 INFO:teuthology.orchestra.run.smithi078.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-06-10T00:51:11.918 INFO:teuthology.orchestra.run.smithi175.stdout:(32/159): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-06-10T00:51:11.944 INFO:teuthology.orchestra.run.smithi175.stdout:(33/159): ceph-mgr-modules-core-19.0.0-2946.g3d 1.5 MB/s | 249 kB 00:00 2024-06-10T00:51:11.969 INFO:teuthology.orchestra.run.smithi175.stdout:(34/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 986 kB/s | 49 kB 00:00 2024-06-10T00:51:12.003 INFO:teuthology.orchestra.run.smithi175.stdout:(35/159): ceph-prometheus-alerts-19.0.0-2946.g3 281 kB/s | 16 kB 00:00 2024-06-10T00:51:12.028 INFO:teuthology.orchestra.run.smithi175.stdout:(36/159): ceph-volume-19.0.0-2946.g3d89accf.el9 4.4 MB/s | 264 kB 00:00 2024-06-10T00:51:12.087 INFO:teuthology.orchestra.run.smithi175.stdout:(37/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 312 kB/s | 18 kB 00:00 2024-06-10T00:51:12.154 INFO:teuthology.orchestra.run.smithi175.stdout:(38/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 4.9 MB/s | 763 kB 00:00 2024-06-10T00:51:12.157 INFO:teuthology.orchestra.run.smithi175.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:51:12.159 INFO:teuthology.orchestra.run.smithi007.stdout:(4/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 4.0 MB/s | 5.4 MB 00:01 2024-06-10T00:51:12.192 INFO:teuthology.orchestra.run.smithi175.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:51:12.329 INFO:teuthology.orchestra.run.smithi175.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 332 kB/s | 80 kB 00:00 2024-06-10T00:51:12.330 INFO:teuthology.orchestra.run.smithi175.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:51:12.351 INFO:teuthology.orchestra.run.smithi007.stdout:(5/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 7.8 MB/s | 1.5 MB 00:00 2024-06-10T00:51:12.413 INFO:teuthology.orchestra.run.smithi175.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-06-10T00:51:12.446 INFO:teuthology.orchestra.run.smithi175.stdout:(41/159): libgfortran-11.4.1-3.el9.x86_64.rpm 2.7 MB/s | 803 kB 00:00 2024-06-10T00:51:12.497 INFO:teuthology.orchestra.run.smithi175.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-06-10T00:51:12.547 INFO:teuthology.orchestra.run.smithi175.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-06-10T00:51:12.668 INFO:teuthology.orchestra.run.smithi007.stdout:(6/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 3.0 MB/s | 2.4 MB 00:00 2024-06-10T00:51:12.739 INFO:teuthology.orchestra.run.smithi175.stdout:(44/159): python3-pycparser-2.20-6.el9.noarch.r 705 kB/s | 135 kB 00:00 2024-06-10T00:51:12.789 INFO:teuthology.orchestra.run.smithi175.stdout:(45/159): python3-cryptography-36.0.1-4.el9.x86 3.6 MB/s | 1.2 MB 00:00 2024-06-10T00:51:12.814 INFO:teuthology.orchestra.run.smithi175.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-06-10T00:51:12.815 INFO:teuthology.orchestra.run.smithi175.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-10T00:51:12.898 INFO:teuthology.orchestra.run.smithi175.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-06-10T00:51:12.899 INFO:teuthology.orchestra.run.smithi175.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:51:12.918 INFO:teuthology.orchestra.run.smithi007.stdout:(7/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 8.3 MB/s | 4.7 MB 00:00 2024-06-10T00:51:13.657 INFO:teuthology.orchestra.run.smithi175.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 818 kB/s | 689 kB 00:00 2024-06-10T00:51:13.657 INFO:teuthology.orchestra.run.smithi175.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:51:13.732 INFO:teuthology.orchestra.run.smithi175.stdout:(49/159): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-06-10T00:51:13.757 INFO:teuthology.orchestra.run.smithi175.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 107 kB 00:00 2024-06-10T00:51:13.841 INFO:teuthology.orchestra.run.smithi175.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 200 kB/s | 22 kB 00:00 2024-06-10T00:51:13.866 INFO:teuthology.orchestra.run.smithi175.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-06-10T00:51:13.941 INFO:teuthology.orchestra.run.smithi175.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 327 kB/s | 33 kB 00:00 2024-06-10T00:51:13.968 INFO:teuthology.orchestra.run.smithi007.stdout:(8/159): ceph-radosgw-19.0.0-2946.g3d89accf.el9 9.8 MB/s | 10 MB 00:01 2024-06-10T00:51:14.027 INFO:teuthology.orchestra.run.smithi007.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 433 kB/s | 25 kB 00:00 2024-06-10T00:51:14.041 INFO:teuthology.orchestra.run.smithi175.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 182 kB/s | 18 kB 00:00 2024-06-10T00:51:14.142 INFO:teuthology.orchestra.run.smithi175.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-06-10T00:51:14.161 INFO:teuthology.orchestra.run.smithi078.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-10T00:51:14.162 INFO:teuthology.orchestra.run.smithi078.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-10T00:51:14.173 INFO:teuthology.orchestra.run.smithi078.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-10T00:51:14.174 INFO:teuthology.orchestra.run.smithi078.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-10T00:51:14.174 INFO:teuthology.orchestra.run.smithi078.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-10T00:51:14.175 INFO:teuthology.orchestra.run.smithi078.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-10T00:51:14.177 INFO:teuthology.orchestra.run.smithi078.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-10T00:51:14.179 INFO:teuthology.orchestra.run.smithi078.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-10T00:51:14.180 INFO:teuthology.orchestra.run.smithi078.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-10T00:51:14.243 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-06-10T00:51:14.250 INFO:teuthology.orchestra.run.smithi175.stdout:(56/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-06-10T00:51:14.262 INFO:teuthology.orchestra.run.smithi078.stdout:======================================================================================= 2024-06-10T00:51:14.263 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2024-06-10T00:51:14.263 INFO:teuthology.orchestra.run.smithi078.stdout:======================================================================================= 2024-06-10T00:51:14.263 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-06-10T00:51:14.263 INFO:teuthology.orchestra.run.smithi078.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-10T00:51:14.264 INFO:teuthology.orchestra.run.smithi078.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-10T00:51:14.264 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-10T00:51:14.264 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-10T00:51:14.264 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-10T00:51:14.264 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-10T00:51:14.264 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-10T00:51:14.264 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-10T00:51:14.264 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-10T00:51:14.264 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-10T00:51:14.264 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-10T00:51:14.264 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-10T00:51:14.264 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-10T00:51:14.264 INFO:teuthology.orchestra.run.smithi078.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-10T00:51:14.265 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-10T00:51:14.265 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-10T00:51:14.265 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-10T00:51:14.265 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-10T00:51:14.265 INFO:teuthology.orchestra.run.smithi078.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-10T00:51:14.265 INFO:teuthology.orchestra.run.smithi078.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-10T00:51:14.265 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-10T00:51:14.265 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-10T00:51:14.265 INFO:teuthology.orchestra.run.smithi078.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-10T00:51:14.265 INFO:teuthology.orchestra.run.smithi078.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-10T00:51:14.265 INFO:teuthology.orchestra.run.smithi078.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-10T00:51:14.265 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-10T00:51:14.266 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-10T00:51:14.266 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-10T00:51:14.266 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-10T00:51:14.266 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-10T00:51:14.266 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-10T00:51:14.266 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-10T00:51:14.266 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-10T00:51:14.266 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-10T00:51:14.266 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-10T00:51:14.266 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-10T00:51:14.266 INFO:teuthology.orchestra.run.smithi078.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-10T00:51:14.266 INFO:teuthology.orchestra.run.smithi078.stdout:Upgrading: 2024-06-10T00:51:14.267 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-10T00:51:14.267 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-10T00:51:14.267 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-10T00:51:14.267 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-10T00:51:14.267 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-10T00:51:14.267 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-10T00:51:14.267 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-10T00:51:14.267 INFO:teuthology.orchestra.run.smithi078.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-10T00:51:14.267 INFO:teuthology.orchestra.run.smithi078.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-10T00:51:14.267 INFO:teuthology.orchestra.run.smithi078.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-10T00:51:14.267 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-10T00:51:14.267 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2024-06-10T00:51:14.267 INFO:teuthology.orchestra.run.smithi078.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-10T00:51:14.268 INFO:teuthology.orchestra.run.smithi078.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-10T00:51:14.268 INFO:teuthology.orchestra.run.smithi078.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-10T00:51:14.268 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-10T00:51:14.268 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-10T00:51:14.268 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-10T00:51:14.268 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-10T00:51:14.268 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-10T00:51:14.268 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-10T00:51:14.268 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-10T00:51:14.268 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-10T00:51:14.268 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-10T00:51:14.268 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-10T00:51:14.268 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-10T00:51:14.269 INFO:teuthology.orchestra.run.smithi078.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-10T00:51:14.269 INFO:teuthology.orchestra.run.smithi078.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-10T00:51:14.269 INFO:teuthology.orchestra.run.smithi078.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-10T00:51:14.269 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-10T00:51:14.269 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-10T00:51:14.269 INFO:teuthology.orchestra.run.smithi078.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-10T00:51:14.269 INFO:teuthology.orchestra.run.smithi078.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-10T00:51:14.269 INFO:teuthology.orchestra.run.smithi078.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-10T00:51:14.269 INFO:teuthology.orchestra.run.smithi078.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-10T00:51:14.269 INFO:teuthology.orchestra.run.smithi078.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-10T00:51:14.269 INFO:teuthology.orchestra.run.smithi078.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-10T00:51:14.269 INFO:teuthology.orchestra.run.smithi078.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-10T00:51:14.269 INFO:teuthology.orchestra.run.smithi078.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-10T00:51:14.270 INFO:teuthology.orchestra.run.smithi078.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-10T00:51:14.270 INFO:teuthology.orchestra.run.smithi078.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-10T00:51:14.270 INFO:teuthology.orchestra.run.smithi078.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-10T00:51:14.270 INFO:teuthology.orchestra.run.smithi078.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-10T00:51:14.270 INFO:teuthology.orchestra.run.smithi078.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-10T00:51:14.270 INFO:teuthology.orchestra.run.smithi078.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-10T00:51:14.270 INFO:teuthology.orchestra.run.smithi078.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-10T00:51:14.270 INFO:teuthology.orchestra.run.smithi078.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-10T00:51:14.270 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-10T00:51:14.270 INFO:teuthology.orchestra.run.smithi078.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-10T00:51:14.270 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-10T00:51:14.270 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-10T00:51:14.271 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-10T00:51:14.271 INFO:teuthology.orchestra.run.smithi078.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-10T00:51:14.271 INFO:teuthology.orchestra.run.smithi078.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-10T00:51:14.271 INFO:teuthology.orchestra.run.smithi078.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-10T00:51:14.271 INFO:teuthology.orchestra.run.smithi078.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-10T00:51:14.271 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-10T00:51:14.271 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-10T00:51:14.271 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-10T00:51:14.271 INFO:teuthology.orchestra.run.smithi078.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-10T00:51:14.271 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-10T00:51:14.271 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-10T00:51:14.271 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-10T00:51:14.271 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-10T00:51:14.272 INFO:teuthology.orchestra.run.smithi078.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-10T00:51:14.272 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-10T00:51:14.272 INFO:teuthology.orchestra.run.smithi078.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-10T00:51:14.272 INFO:teuthology.orchestra.run.smithi078.stdout: python3-google-auth noarch 1:2.29.0-2.el9 epel 205 k 2024-06-10T00:51:14.272 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-10T00:51:14.272 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-10T00:51:14.272 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-10T00:51:14.272 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-10T00:51:14.272 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-10T00:51:14.272 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-10T00:51:14.272 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-10T00:51:14.272 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-10T00:51:14.272 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-10T00:51:14.273 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-10T00:51:14.273 INFO:teuthology.orchestra.run.smithi078.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-10T00:51:14.273 INFO:teuthology.orchestra.run.smithi078.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-10T00:51:14.273 INFO:teuthology.orchestra.run.smithi078.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-10T00:51:14.273 INFO:teuthology.orchestra.run.smithi078.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-10T00:51:14.273 INFO:teuthology.orchestra.run.smithi078.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-10T00:51:14.273 INFO:teuthology.orchestra.run.smithi078.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-10T00:51:14.273 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-10T00:51:14.273 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-10T00:51:14.273 INFO:teuthology.orchestra.run.smithi078.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-10T00:51:14.273 INFO:teuthology.orchestra.run.smithi078.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-10T00:51:14.273 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-10T00:51:14.274 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-10T00:51:14.274 INFO:teuthology.orchestra.run.smithi078.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-10T00:51:14.274 INFO:teuthology.orchestra.run.smithi078.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-10T00:51:14.274 INFO:teuthology.orchestra.run.smithi078.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-10T00:51:14.274 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-10T00:51:14.274 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-10T00:51:14.274 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-10T00:51:14.274 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-10T00:51:14.274 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-10T00:51:14.274 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-10T00:51:14.274 INFO:teuthology.orchestra.run.smithi078.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-10T00:51:14.274 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-10T00:51:14.275 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-10T00:51:14.275 INFO:teuthology.orchestra.run.smithi078.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-10T00:51:14.275 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-10T00:51:14.275 INFO:teuthology.orchestra.run.smithi078.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-10T00:51:14.275 INFO:teuthology.orchestra.run.smithi078.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-10T00:51:14.275 INFO:teuthology.orchestra.run.smithi078.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-10T00:51:14.275 INFO:teuthology.orchestra.run.smithi078.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-10T00:51:14.275 INFO:teuthology.orchestra.run.smithi078.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-10T00:51:14.275 INFO:teuthology.orchestra.run.smithi078.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-10T00:51:14.275 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-10T00:51:14.275 INFO:teuthology.orchestra.run.smithi078.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-10T00:51:14.275 INFO:teuthology.orchestra.run.smithi078.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-10T00:51:14.275 INFO:teuthology.orchestra.run.smithi078.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-10T00:51:14.276 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-10T00:51:14.276 INFO:teuthology.orchestra.run.smithi078.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-10T00:51:14.276 INFO:teuthology.orchestra.run.smithi078.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-10T00:51:14.276 INFO:teuthology.orchestra.run.smithi078.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-10T00:51:14.276 INFO:teuthology.orchestra.run.smithi078.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-10T00:51:14.276 INFO:teuthology.orchestra.run.smithi078.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-10T00:51:14.276 INFO:teuthology.orchestra.run.smithi078.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-10T00:51:14.276 INFO:teuthology.orchestra.run.smithi078.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-10T00:51:14.276 INFO:teuthology.orchestra.run.smithi078.stdout:Installing weak dependencies: 2024-06-10T00:51:14.276 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-10T00:51:14.276 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-06-10T00:51:14.276 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-06-10T00:51:14.276 INFO:teuthology.orchestra.run.smithi078.stdout:======================================================================================= 2024-06-10T00:51:14.276 INFO:teuthology.orchestra.run.smithi078.stdout:Install 148 Packages 2024-06-10T00:51:14.276 INFO:teuthology.orchestra.run.smithi078.stdout:Upgrade 11 Packages 2024-06-10T00:51:14.277 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-06-10T00:51:14.277 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 225 M 2024-06-10T00:51:14.277 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-06-10T00:51:14.350 INFO:teuthology.orchestra.run.smithi175.stdout:(57/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 453 kB/s | 45 kB 00:00 2024-06-10T00:51:14.543 INFO:teuthology.orchestra.run.smithi175.stdout:(58/159): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 662 kB 00:00 2024-06-10T00:51:14.660 INFO:teuthology.orchestra.run.smithi175.stdout:(59/159): libtool-2.4.6-45.el9.x86_64.rpm 4.9 MB/s | 582 kB 00:00 2024-06-10T00:51:14.732 INFO:teuthology.orchestra.run.smithi078.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 28 kB/s | 6.6 kB 00:00 2024-06-10T00:51:14.760 INFO:teuthology.orchestra.run.smithi175.stdout:(60/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.3 MB/s | 3.0 MB 00:00 2024-06-10T00:51:14.787 INFO:teuthology.orchestra.run.smithi175.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 203 kB/s | 25 kB 00:00 2024-06-10T00:51:14.869 INFO:teuthology.orchestra.run.smithi175.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 437 kB/s | 47 kB 00:00 2024-06-10T00:51:14.894 INFO:teuthology.orchestra.run.smithi175.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-06-10T00:51:14.936 INFO:teuthology.orchestra.run.smithi007.stdout:(10/159): ceph-common-19.0.0-2946.g3d89accf.el9 5.1 MB/s | 21 MB 00:04 2024-06-10T00:51:14.995 INFO:teuthology.orchestra.run.smithi175.stdout:(64/159): openblas-0.3.21-2.el9.x86_64.rpm 352 kB/s | 35 kB 00:00 2024-06-10T00:51:15.094 INFO:teuthology.orchestra.run.smithi007.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 1.4 MB/s | 221 kB 00:00 2024-06-10T00:51:15.136 INFO:teuthology.orchestra.run.smithi175.stdout:(65/159): lua-5.4.4-4.el9.x86_64.rpm 706 kB/s | 188 kB 00:00 2024-06-10T00:51:15.186 INFO:teuthology.orchestra.run.smithi007.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 296 kB/s | 27 kB 00:00 2024-06-10T00:51:15.278 INFO:teuthology.orchestra.run.smithi007.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 346 kB/s | 32 kB 00:00 2024-06-10T00:51:15.363 INFO:teuthology.orchestra.run.smithi078.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 1.4 MB/s | 890 kB 00:00 2024-06-10T00:51:15.571 INFO:teuthology.orchestra.run.smithi078.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 695 kB/s | 145 kB 00:00 2024-06-10T00:51:15.628 INFO:teuthology.orchestra.run.smithi007.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 2.1 MB/s | 747 kB 00:00 2024-06-10T00:51:15.795 INFO:teuthology.orchestra.run.smithi007.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 981 kB/s | 163 kB 00:00 2024-06-10T00:51:15.887 INFO:teuthology.orchestra.run.smithi007.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 1.3 MB/s | 127 kB 00:00 2024-06-10T00:51:16.187 INFO:teuthology.orchestra.run.smithi007.stdout:(17/159): libradosstriper1-19.0.0-2946.g3d89acc 1.6 MB/s | 501 kB 00:00 2024-06-10T00:51:16.547 INFO:teuthology.orchestra.run.smithi078.stdout:(4/159): ceph-common-19.0.0-2946.g3d89accf.el9. 10 MB/s | 21 MB 00:02 2024-06-10T00:51:16.664 INFO:teuthology.orchestra.run.smithi078.stdout:(5/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 2.2 MB/s | 2.4 MB 00:01 2024-06-10T00:51:16.747 INFO:teuthology.orchestra.run.smithi078.stdout:(6/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 7.5 MB/s | 1.5 MB 00:00 2024-06-10T00:51:17.062 INFO:teuthology.orchestra.run.smithi175.stdout:(66/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 2.3 MB/s | 4.4 MB 00:01 2024-06-10T00:51:17.195 INFO:teuthology.orchestra.run.smithi175.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-06-10T00:51:17.239 INFO:teuthology.orchestra.run.smithi078.stdout:(7/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 1.9 MB/s | 5.4 MB 00:02 2024-06-10T00:51:17.320 INFO:teuthology.orchestra.run.smithi175.stdout:(68/159): postgresql-contrib-13.14-1.el9.x86_64 7.1 MB/s | 911 kB 00:00 2024-06-10T00:51:17.387 INFO:teuthology.orchestra.run.smithi175.stdout:(69/159): openblas-openmp-0.3.21-2.el9.x86_64.r 2.0 MB/s | 4.7 MB 00:02 2024-06-10T00:51:17.429 INFO:teuthology.orchestra.run.smithi175.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-06-10T00:51:17.555 INFO:teuthology.orchestra.run.smithi175.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 8.1 MB/s | 1.0 MB 00:00 2024-06-10T00:51:17.663 INFO:teuthology.orchestra.run.smithi175.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 809 kB/s | 87 kB 00:00 2024-06-10T00:51:17.922 INFO:teuthology.orchestra.run.smithi175.stdout:(73/159): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 6.0 MB 00:00 2024-06-10T00:51:18.030 INFO:teuthology.orchestra.run.smithi175.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 566 kB/s | 61 kB 00:00 2024-06-10T00:51:18.113 INFO:teuthology.orchestra.run.smithi007.stdout:(18/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 3.1 MB/s | 17 MB 00:05 2024-06-10T00:51:18.139 INFO:teuthology.orchestra.run.smithi175.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-06-10T00:51:18.188 INFO:teuthology.orchestra.run.smithi007.stdout:(19/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 2.7 MB/s | 5.3 MB 00:02 2024-06-10T00:51:18.213 INFO:teuthology.orchestra.run.smithi007.stdout:(20/159): python3-ceph-argparse-19.0.0-2946.g3d 451 kB/s | 45 kB 00:00 2024-06-10T00:51:18.247 INFO:teuthology.orchestra.run.smithi175.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 2.3 MB/s | 248 kB 00:00 2024-06-10T00:51:18.272 INFO:teuthology.orchestra.run.smithi007.stdout:(21/159): python3-ceph-common-19.0.0-2946.g3d89 1.6 MB/s | 135 kB 00:00 2024-06-10T00:51:18.281 INFO:teuthology.orchestra.run.smithi078.stdout:(8/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 2.9 MB/s | 4.7 MB 00:01 2024-06-10T00:51:18.314 INFO:teuthology.orchestra.run.smithi007.stdout:(22/159): python3-cephfs-19.0.0-2946.g3d89accf. 1.6 MB/s | 162 kB 00:00 2024-06-10T00:51:18.373 INFO:teuthology.orchestra.run.smithi175.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 205 kB/s | 26 kB 00:00 2024-06-10T00:51:18.381 INFO:teuthology.orchestra.run.smithi007.stdout:(23/159): python3-rados-19.0.0-2946.g3d89accf.e 2.9 MB/s | 321 kB 00:00 2024-06-10T00:51:18.448 INFO:teuthology.orchestra.run.smithi007.stdout:(24/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.5 MB/s | 99 kB 00:00 2024-06-10T00:51:18.473 INFO:teuthology.orchestra.run.smithi175.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 192 kB/s | 19 kB 00:00 2024-06-10T00:51:18.473 INFO:teuthology.orchestra.run.smithi007.stdout:(25/159): python3-rbd-19.0.0-2946.g3d89accf.el9 1.8 MB/s | 300 kB 00:00 2024-06-10T00:51:18.490 INFO:teuthology.orchestra.run.smithi078.stdout:(9/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x86 9.6 MB/s | 17 MB 00:01 2024-06-10T00:51:18.507 INFO:teuthology.orchestra.run.smithi078.stdout:(10/159): ceph-selinux-19.0.0-2946.g3d89accf.el 112 kB/s | 25 kB 00:00 2024-06-10T00:51:18.507 INFO:teuthology.orchestra.run.smithi007.stdout:(26/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.4 MB/s | 85 kB 00:00 2024-06-10T00:51:18.582 INFO:teuthology.orchestra.run.smithi175.stdout:(79/159): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-06-10T00:51:18.582 INFO:teuthology.orchestra.run.smithi007.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 2.2 MB/s | 171 kB 00:00 2024-06-10T00:51:18.598 INFO:teuthology.orchestra.run.smithi078.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 2.4 MB/s | 221 kB 00:00 2024-06-10T00:51:18.658 INFO:teuthology.orchestra.run.smithi007.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 422 kB/s | 32 kB 00:00 2024-06-10T00:51:18.690 INFO:teuthology.orchestra.run.smithi175.stdout:(80/159): python3-markupsafe-1.1.1-12.el9.x86_6 321 kB/s | 35 kB 00:00 2024-06-10T00:51:18.699 INFO:teuthology.orchestra.run.smithi078.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 271 kB/s | 27 kB 00:00 2024-06-10T00:51:18.758 INFO:teuthology.orchestra.run.smithi007.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 1.4 MB/s | 142 kB 00:00 2024-06-10T00:51:18.824 INFO:teuthology.orchestra.run.smithi078.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 254 kB/s | 32 kB 00:00 2024-06-10T00:51:18.932 INFO:teuthology.orchestra.run.smithi175.stdout:(81/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 5.1 MB 00:00 2024-06-10T00:51:19.041 INFO:teuthology.orchestra.run.smithi175.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-06-10T00:51:19.116 INFO:teuthology.orchestra.run.smithi078.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 2.5 MB/s | 747 kB 00:00 2024-06-10T00:51:19.157 INFO:teuthology.orchestra.run.smithi175.stdout:(83/159): python3-oauthlib-3.1.1-5.el9.noarch.r 1.9 MB/s | 222 kB 00:00 2024-06-10T00:51:19.233 INFO:teuthology.orchestra.run.smithi078.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 1.4 MB/s | 163 kB 00:00 2024-06-10T00:51:19.266 INFO:teuthology.orchestra.run.smithi175.stdout:(84/159): python3-packaging-20.9-5.el9.noarch.r 717 kB/s | 77 kB 00:00 2024-06-10T00:51:19.342 INFO:teuthology.orchestra.run.smithi078.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 1.1 MB/s | 127 kB 00:00 2024-06-10T00:51:19.366 INFO:teuthology.orchestra.run.smithi175.stdout:(85/159): python3-prettytable-0.7.2-27.el9.noar 423 kB/s | 42 kB 00:00 2024-06-10T00:51:19.475 INFO:teuthology.orchestra.run.smithi175.stdout:(86/159): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-06-10T00:51:19.542 INFO:teuthology.orchestra.run.smithi078.stdout:(17/159): libradosstriper1-19.0.0-2946.g3d89acc 2.4 MB/s | 501 kB 00:00 2024-06-10T00:51:19.584 INFO:teuthology.orchestra.run.smithi175.stdout:(87/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-06-10T00:51:19.692 INFO:teuthology.orchestra.run.smithi175.stdout:(88/159): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-06-10T00:51:19.716 INFO:teuthology.orchestra.run.smithi007.stdout:(30/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 2.5 MB/s | 3.1 MB 00:01 2024-06-10T00:51:19.801 INFO:teuthology.orchestra.run.smithi175.stdout:(89/159): python3-pytz-2021.1-5.el9.noarch.rpm 471 kB/s | 51 kB 00:00 2024-06-10T00:51:19.910 INFO:teuthology.orchestra.run.smithi175.stdout:(90/159): python3-requests-oauthlib-1.3.0-12.el 495 kB/s | 54 kB 00:00 2024-06-10T00:51:20.042 INFO:teuthology.orchestra.run.smithi007.stdout:(31/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 2.7 MB/s | 3.5 MB 00:01 2024-06-10T00:51:20.192 INFO:teuthology.orchestra.run.smithi007.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 1.6 MB/s | 249 kB 00:00 2024-06-10T00:51:20.284 INFO:teuthology.orchestra.run.smithi007.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 537 kB/s | 49 kB 00:00 2024-06-10T00:51:20.376 INFO:teuthology.orchestra.run.smithi007.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2946.g3 180 kB/s | 16 kB 00:00 2024-06-10T00:51:20.477 INFO:teuthology.orchestra.run.smithi175.stdout:(91/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-06-10T00:51:20.535 INFO:teuthology.orchestra.run.smithi007.stdout:(35/159): ceph-volume-19.0.0-2946.g3d89accf.el9 1.6 MB/s | 264 kB 00:00 2024-06-10T00:51:20.585 INFO:teuthology.orchestra.run.smithi175.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 386 kB/s | 42 kB 00:00 2024-06-10T00:51:20.693 INFO:teuthology.orchestra.run.smithi175.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 408 kB/s | 44 kB 00:00 2024-06-10T00:51:20.811 INFO:teuthology.orchestra.run.smithi175.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 1.8 MB/s | 221 kB 00:00 2024-06-10T00:51:20.843 INFO:teuthology.orchestra.run.smithi007.stdout:(36/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 2.4 MB/s | 763 kB 00:00 2024-06-10T00:51:20.911 INFO:teuthology.orchestra.run.smithi175.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 359 kB/s | 36 kB 00:00 2024-06-10T00:51:20.935 INFO:teuthology.orchestra.run.smithi007.stdout:(37/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 199 kB/s | 18 kB 00:00 2024-06-10T00:51:20.975 INFO:teuthology.orchestra.run.smithi007.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:51:21.020 INFO:teuthology.orchestra.run.smithi175.stdout:(96/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 437 kB/s | 47 kB 00:00 2024-06-10T00:51:21.128 INFO:teuthology.orchestra.run.smithi175.stdout:(97/159): socat-1.7.4.1-5.el9.x86_64.rpm 2.8 MB/s | 305 kB 00:00 2024-06-10T00:51:21.159 INFO:teuthology.orchestra.run.smithi078.stdout:(18/159): ceph-radosgw-19.0.0-2946.g3d89accf.el 2.6 MB/s | 10 MB 00:03 2024-06-10T00:51:21.234 INFO:teuthology.orchestra.run.smithi078.stdout:(19/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 3.1 MB/s | 5.3 MB 00:01 2024-06-10T00:51:21.237 INFO:teuthology.orchestra.run.smithi175.stdout:(98/159): uuid-1.6.2-55.el9.x86_64.rpm 533 kB/s | 58 kB 00:00 2024-06-10T00:51:21.259 INFO:teuthology.orchestra.run.smithi078.stdout:(20/159): python3-ceph-argparse-19.0.0-2946.g3d 451 kB/s | 45 kB 00:00 2024-06-10T00:51:21.293 INFO:teuthology.orchestra.run.smithi078.stdout:(21/159): python3-ceph-common-19.0.0-2946.g3d89 2.3 MB/s | 135 kB 00:00 2024-06-10T00:51:21.346 INFO:teuthology.orchestra.run.smithi175.stdout:(99/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 865 kB/s | 94 kB 00:00 2024-06-10T00:51:21.368 INFO:teuthology.orchestra.run.smithi078.stdout:(22/159): python3-cephfs-19.0.0-2946.g3d89accf. 1.5 MB/s | 162 kB 00:00 2024-06-10T00:51:21.427 INFO:teuthology.orchestra.run.smithi007.stdout:(38/159): ceph-test-19.0.0-2946.g3d89accf.el9.x 6.5 MB/s | 48 MB 00:07 2024-06-10T00:51:21.428 INFO:teuthology.orchestra.run.smithi007.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:51:21.435 INFO:teuthology.orchestra.run.smithi078.stdout:(23/159): python3-rados-19.0.0-2946.g3d89accf.e 2.2 MB/s | 321 kB 00:00 2024-06-10T00:51:21.454 INFO:teuthology.orchestra.run.smithi175.stdout:(100/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 588 kB/s | 64 kB 00:00 2024-06-10T00:51:21.519 INFO:teuthology.orchestra.run.smithi078.stdout:(24/159): python3-rbd-19.0.0-2946.g3d89accf.el9 1.9 MB/s | 300 kB 00:00 2024-06-10T00:51:21.536 INFO:teuthology.orchestra.run.smithi007.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 134 kB/s | 80 kB 00:00 2024-06-10T00:51:21.537 INFO:teuthology.orchestra.run.smithi007.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:51:21.544 INFO:teuthology.orchestra.run.smithi078.stdout:(25/159): python3-rgw-19.0.0-2946.g3d89accf.el9 914 kB/s | 99 kB 00:00 2024-06-10T00:51:21.563 INFO:teuthology.orchestra.run.smithi175.stdout:(101/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 414 kB/s | 45 kB 00:00 2024-06-10T00:51:21.595 INFO:teuthology.orchestra.run.smithi078.stdout:(26/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.1 MB/s | 85 kB 00:00 2024-06-10T00:51:21.598 INFO:teuthology.orchestra.run.smithi175.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:51:21.603 INFO:teuthology.orchestra.run.smithi007.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-06-10T00:51:21.637 INFO:teuthology.orchestra.run.smithi007.stdout:(41/159): libgfortran-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 803 kB 00:00 2024-06-10T00:51:21.662 INFO:teuthology.orchestra.run.smithi007.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-06-10T00:51:21.669 INFO:teuthology.orchestra.run.smithi078.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 2.2 MB/s | 171 kB 00:00 2024-06-10T00:51:21.729 INFO:teuthology.orchestra.run.smithi007.stdout:(43/159): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.2 MB 00:00 2024-06-10T00:51:21.729 INFO:teuthology.orchestra.run.smithi175.stdout:(102/159): lua-devel-5.4.4-4.el9.x86_64.rpm 134 kB/s | 22 kB 00:00 2024-06-10T00:51:21.733 INFO:teuthology.orchestra.run.smithi175.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:51:21.754 INFO:teuthology.orchestra.run.smithi007.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-06-10T00:51:21.770 INFO:teuthology.orchestra.run.smithi078.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 316 kB/s | 32 kB 00:00 2024-06-10T00:51:21.779 INFO:teuthology.orchestra.run.smithi007.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-06-10T00:51:21.804 INFO:teuthology.orchestra.run.smithi007.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-06-10T00:51:21.806 INFO:teuthology.orchestra.run.smithi007.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-10T00:51:21.839 INFO:teuthology.orchestra.run.smithi007.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-06-10T00:51:21.840 INFO:teuthology.orchestra.run.smithi007.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:51:21.854 INFO:teuthology.orchestra.run.smithi078.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 1.7 MB/s | 142 kB 00:00 2024-06-10T00:51:21.863 INFO:teuthology.orchestra.run.smithi175.stdout:(103/159): protobuf-compiler-3.14.0-13.el9.x86_ 6.3 MB/s | 863 kB 00:00 2024-06-10T00:51:21.897 INFO:teuthology.orchestra.run.smithi175.stdout:(104/159): abseil-cpp-20211102.0-3.el9.x86_64.r 16 MB/s | 548 kB 00:00 2024-06-10T00:51:21.922 INFO:teuthology.orchestra.run.smithi175.stdout:(105/159): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-06-10T00:51:21.947 INFO:teuthology.orchestra.run.smithi175.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 774 kB/s | 19 kB 00:00 2024-06-10T00:51:22.006 INFO:teuthology.orchestra.run.smithi007.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 689 kB 00:00 2024-06-10T00:51:22.007 INFO:teuthology.orchestra.run.smithi007.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:51:22.056 INFO:teuthology.orchestra.run.smithi007.stdout:(49/159): bison-3.7.4-5.el9.x86_64.rpm 4.2 MB/s | 943 kB 00:00 2024-06-10T00:51:22.090 INFO:teuthology.orchestra.run.smithi007.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 1.2 MB/s | 107 kB 00:00 2024-06-10T00:51:22.115 INFO:teuthology.orchestra.run.smithi007.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 370 kB/s | 22 kB 00:00 2024-06-10T00:51:22.140 INFO:teuthology.orchestra.run.smithi007.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-06-10T00:51:22.148 INFO:teuthology.orchestra.run.smithi175.stdout:(107/159): libarrow-9.0.0-11.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-06-10T00:51:22.166 INFO:teuthology.orchestra.run.smithi007.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 649 kB/s | 33 kB 00:00 2024-06-10T00:51:22.173 INFO:teuthology.orchestra.run.smithi175.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-06-10T00:51:22.199 INFO:teuthology.orchestra.run.smithi175.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-06-10T00:51:22.224 INFO:teuthology.orchestra.run.smithi175.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-06-10T00:51:22.229 INFO:teuthology.orchestra.run.smithi078.stdout:(30/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 4.5 MB/s | 3.1 MB 00:00 2024-06-10T00:51:22.249 INFO:teuthology.orchestra.run.smithi175.stdout:(111/159): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-06-10T00:51:22.250 INFO:teuthology.orchestra.run.smithi007.stdout:(54/159): ceph-mgr-diskprediction-local-19.0.0- 2.9 MB/s | 7.4 MB 00:02 2024-06-10T00:51:22.283 INFO:teuthology.orchestra.run.smithi175.stdout:(112/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 25 MB/s | 839 kB 00:00 2024-06-10T00:51:22.283 INFO:teuthology.orchestra.run.smithi007.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 155 kB/s | 18 kB 00:00 2024-06-10T00:51:22.316 INFO:teuthology.orchestra.run.smithi175.stdout:(113/159): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-06-10T00:51:22.334 INFO:teuthology.orchestra.run.smithi007.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-06-10T00:51:22.341 INFO:teuthology.orchestra.run.smithi175.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-06-10T00:51:22.367 INFO:teuthology.orchestra.run.smithi175.stdout:(115/159): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-06-10T00:51:22.367 INFO:teuthology.orchestra.run.smithi007.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-06-10T00:51:22.392 INFO:teuthology.orchestra.run.smithi175.stdout:(116/159): python3-certifi-2021.10.8-2.el9.noar 599 kB/s | 15 kB 00:00 2024-06-10T00:51:22.393 INFO:teuthology.orchestra.run.smithi007.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 773 kB/s | 45 kB 00:00 2024-06-10T00:51:22.409 INFO:teuthology.orchestra.run.smithi175.stdout:(117/159): python3-cheroot-8.6.0-4.el9.noarch.r 10 MB/s | 172 kB 00:00 2024-06-10T00:51:22.443 INFO:teuthology.orchestra.run.smithi175.stdout:(118/159): python3-cherrypy-18.6.1-2.el9.noarch 11 MB/s | 358 kB 00:00 2024-06-10T00:51:22.510 INFO:teuthology.orchestra.run.smithi007.stdout:(59/159): libtool-2.4.6-45.el9.x86_64.rpm 4.9 MB/s | 582 kB 00:00 2024-06-10T00:51:22.560 INFO:teuthology.orchestra.run.smithi007.stdout:(60/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 509 kB/s | 25 kB 00:00 2024-06-10T00:51:22.611 INFO:teuthology.orchestra.run.smithi007.stdout:(61/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-06-10T00:51:22.636 INFO:teuthology.orchestra.run.smithi007.stdout:(62/159): librdkafka-1.6.1-102.el9.x86_64.rpm 2.4 MB/s | 662 kB 00:00 2024-06-10T00:51:22.721 INFO:teuthology.orchestra.run.smithi007.stdout:(63/159): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-06-10T00:51:22.746 INFO:teuthology.orchestra.run.smithi007.stdout:(64/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-06-10T00:51:22.772 INFO:teuthology.orchestra.run.smithi007.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 691 kB/s | 35 kB 00:00 2024-06-10T00:51:22.868 INFO:teuthology.orchestra.run.smithi175.stdout:(119/159): ceph-test-19.0.0-2946.g3d89accf.el9. 3.6 MB/s | 48 MB 00:13 2024-06-10T00:51:22.927 INFO:teuthology.orchestra.run.smithi175.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 627 kB/s | 303 kB 00:00 2024-06-10T00:51:22.968 INFO:teuthology.orchestra.run.smithi175.stdout:(121/159): python3-google-auth-2.29.0-2.el9.noa 2.0 MB/s | 205 kB 00:00 2024-06-10T00:51:22.988 INFO:teuthology.orchestra.run.smithi078.stdout:(31/159): ceph-test-19.0.0-2946.g3d89accf.el9.x 11 MB/s | 48 MB 00:04 2024-06-10T00:51:22.998 INFO:teuthology.orchestra.run.smithi007.stdout:(66/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.5 MB/s | 3.0 MB 00:00 2024-06-10T00:51:23.002 INFO:teuthology.orchestra.run.smithi175.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-06-10T00:51:23.036 INFO:teuthology.orchestra.run.smithi175.stdout:(123/159): python3-jaraco-8.2.1-3.el9.noarch.rp 320 kB/s | 11 kB 00:00 2024-06-10T00:51:23.046 INFO:teuthology.orchestra.run.smithi078.stdout:(32/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 2.9 MB/s | 3.5 MB 00:01 2024-06-10T00:51:23.070 INFO:teuthology.orchestra.run.smithi175.stdout:(124/159): python3-grpcio-1.46.7-10.el9.x86_64. 15 MB/s | 2.0 MB 00:00 2024-06-10T00:51:23.072 INFO:teuthology.orchestra.run.smithi078.stdout:(33/159): ceph-mgr-modules-core-19.0.0-2946.g3d 2.9 MB/s | 249 kB 00:00 2024-06-10T00:51:23.095 INFO:teuthology.orchestra.run.smithi175.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 302 kB/s | 18 kB 00:00 2024-06-10T00:51:23.105 INFO:teuthology.orchestra.run.smithi078.stdout:(34/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 844 kB/s | 49 kB 00:00 2024-06-10T00:51:23.120 INFO:teuthology.orchestra.run.smithi175.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-06-10T00:51:23.131 INFO:teuthology.orchestra.run.smithi078.stdout:(35/159): ceph-prometheus-alerts-19.0.0-2946.g3 281 kB/s | 16 kB 00:00 2024-06-10T00:51:23.145 INFO:teuthology.orchestra.run.smithi175.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-06-10T00:51:23.170 INFO:teuthology.orchestra.run.smithi175.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-06-10T00:51:23.196 INFO:teuthology.orchestra.run.smithi175.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-06-10T00:51:23.214 INFO:teuthology.orchestra.run.smithi078.stdout:(36/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 9.0 MB/s | 763 kB 00:00 2024-06-10T00:51:23.221 INFO:teuthology.orchestra.run.smithi175.stdout:(130/159): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-06-10T00:51:23.240 INFO:teuthology.orchestra.run.smithi078.stdout:(37/159): ceph-volume-19.0.0-2946.g3d89accf.el9 1.9 MB/s | 264 kB 00:00 2024-06-10T00:51:23.246 INFO:teuthology.orchestra.run.smithi175.stdout:(131/159): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-06-10T00:51:23.265 INFO:teuthology.orchestra.run.smithi078.stdout:(38/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 363 kB/s | 18 kB 00:00 2024-06-10T00:51:23.280 INFO:teuthology.orchestra.run.smithi175.stdout:(132/159): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-06-10T00:51:23.305 INFO:teuthology.orchestra.run.smithi175.stdout:(133/159): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-06-10T00:51:23.321 INFO:teuthology.orchestra.run.smithi078.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:51:23.327 INFO:teuthology.orchestra.run.smithi078.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:51:23.339 INFO:teuthology.orchestra.run.smithi175.stdout:(134/159): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2024-06-10T00:51:23.364 INFO:teuthology.orchestra.run.smithi175.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-06-10T00:51:23.389 INFO:teuthology.orchestra.run.smithi175.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 564 kB/s | 28 kB 00:00 2024-06-10T00:51:23.407 INFO:teuthology.orchestra.run.smithi078.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 481 kB/s | 80 kB 00:00 2024-06-10T00:51:23.408 INFO:teuthology.orchestra.run.smithi078.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:51:23.414 INFO:teuthology.orchestra.run.smithi175.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-06-10T00:51:23.440 INFO:teuthology.orchestra.run.smithi175.stdout:(138/159): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-06-10T00:51:23.457 INFO:teuthology.orchestra.run.smithi078.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-06-10T00:51:23.465 INFO:teuthology.orchestra.run.smithi175.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-06-10T00:51:23.474 INFO:teuthology.orchestra.run.smithi007.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 3.3 MB/s | 1.6 MB 00:00 2024-06-10T00:51:23.482 INFO:teuthology.orchestra.run.smithi078.stdout:(41/159): libgfortran-11.4.1-3.el9.x86_64.rpm 3.6 MB/s | 803 kB 00:00 2024-06-10T00:51:23.490 INFO:teuthology.orchestra.run.smithi175.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-06-10T00:51:23.516 INFO:teuthology.orchestra.run.smithi175.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-06-10T00:51:23.541 INFO:teuthology.orchestra.run.smithi175.stdout:(142/159): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-06-10T00:51:23.541 INFO:teuthology.orchestra.run.smithi078.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-06-10T00:51:23.566 INFO:teuthology.orchestra.run.smithi175.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-06-10T00:51:23.583 INFO:teuthology.orchestra.run.smithi078.stdout:(43/159): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-06-10T00:51:23.591 INFO:teuthology.orchestra.run.smithi175.stdout:(144/159): python3-zc-lockfile-2.0-10.el9.noarc 802 kB/s | 20 kB 00:00 2024-06-10T00:51:23.600 INFO:teuthology.orchestra.run.smithi078.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-06-10T00:51:23.616 INFO:teuthology.orchestra.run.smithi175.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-06-10T00:51:23.634 INFO:teuthology.orchestra.run.smithi175.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 4.5 MB/s | 192 kB 00:00 2024-06-10T00:51:23.684 INFO:teuthology.orchestra.run.smithi078.stdout:(45/159): ceph-mgr-diskprediction-local-19.0.0- 5.1 MB/s | 7.4 MB 00:01 2024-06-10T00:51:23.692 INFO:teuthology.orchestra.run.smithi175.stdout:(147/159): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-06-10T00:51:23.709 INFO:teuthology.orchestra.run.smithi078.stdout:(46/159): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-06-10T00:51:23.734 INFO:teuthology.orchestra.run.smithi078.stdout:(47/159): python3-requests-2.25.1-8.el9.noarch. 932 kB/s | 125 kB 00:00 2024-06-10T00:51:23.735 INFO:teuthology.orchestra.run.smithi078.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-10T00:51:23.736 INFO:teuthology.orchestra.run.smithi078.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:51:23.757 INFO:teuthology.orchestra.run.smithi007.stdout:(68/159): postgresql-contrib-13.14-1.el9.x86_64 3.1 MB/s | 911 kB 00:00 2024-06-10T00:51:23.801 INFO:teuthology.orchestra.run.smithi078.stdout:(48/159): python3-urllib3-1.26.5-5.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-06-10T00:51:23.802 INFO:teuthology.orchestra.run.smithi078.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:51:23.841 INFO:teuthology.orchestra.run.smithi007.stdout:(69/159): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-06-10T00:51:24.058 INFO:teuthology.orchestra.run.smithi007.stdout:(70/159): openblas-openmp-0.3.21-2.el9.x86_64.r 3.6 MB/s | 4.7 MB 00:01 2024-06-10T00:51:24.101 INFO:teuthology.orchestra.run.smithi175.stdout:(148/159): librados2-19.0.0-2946.g3d89accf.el9. 7.3 MB/s | 3.4 MB 00:00 2024-06-10T00:51:24.200 INFO:teuthology.orchestra.run.smithi007.stdout:(71/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.1 MB/s | 4.4 MB 00:01 2024-06-10T00:51:24.267 INFO:teuthology.orchestra.run.smithi007.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-06-10T00:51:24.384 INFO:teuthology.orchestra.run.smithi175.stdout:(149/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 4.5 MB/s | 3.1 MB 00:00 2024-06-10T00:51:24.426 INFO:teuthology.orchestra.run.smithi007.stdout:(73/159): protobuf-3.14.0-13.el9.x86_64.rpm 2.7 MB/s | 1.0 MB 00:00 2024-06-10T00:51:24.476 INFO:teuthology.orchestra.run.smithi007.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-06-10T00:51:24.484 INFO:teuthology.orchestra.run.smithi175.stdout:(150/159): NetworkManager-1.47.91-1.el9.x86_64. 6.0 MB/s | 2.3 MB 00:00 2024-06-10T00:51:24.510 INFO:teuthology.orchestra.run.smithi078.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 151 kB/s | 107 kB 00:00 2024-06-10T00:51:24.535 INFO:teuthology.orchestra.run.smithi175.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 829 kB/s | 41 kB 00:00 2024-06-10T00:51:24.560 INFO:teuthology.orchestra.run.smithi007.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-06-10T00:51:24.593 INFO:teuthology.orchestra.run.smithi175.stdout:(152/159): NetworkManager-tui-1.47.91-1.el9.x86 4.2 MB/s | 248 kB 00:00 2024-06-10T00:51:24.644 INFO:teuthology.orchestra.run.smithi175.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 738 kB/s | 37 kB 00:00 2024-06-10T00:51:24.660 INFO:teuthology.orchestra.run.smithi007.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-06-10T00:51:24.660 INFO:teuthology.orchestra.run.smithi078.stdout:(50/159): automake-1.16.2-8.el9.noarch.rpm 724 kB/s | 689 kB 00:00 2024-06-10T00:51:24.694 INFO:teuthology.orchestra.run.smithi175.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 4.0 MB/s | 207 kB 00:00 2024-06-10T00:51:24.702 INFO:teuthology.orchestra.run.smithi007.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 616 kB/s | 26 kB 00:00 2024-06-10T00:51:24.727 INFO:teuthology.orchestra.run.smithi078.stdout:(51/159): bison-3.7.4-5.el9.x86_64.rpm 950 kB/s | 943 kB 00:00 2024-06-10T00:51:24.737 INFO:teuthology.orchestra.run.smithi175.stdout:(155/159): NetworkManager-libnm-1.47.91-1.el9.x 5.1 MB/s | 1.8 MB 00:00 2024-06-10T00:51:24.744 INFO:teuthology.orchestra.run.smithi007.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 460 kB/s | 19 kB 00:00 2024-06-10T00:51:24.762 INFO:teuthology.orchestra.run.smithi175.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 3.8 MB/s | 260 kB 00:00 2024-06-10T00:51:24.811 INFO:teuthology.orchestra.run.smithi007.stdout:(79/159): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-06-10T00:51:24.853 INFO:teuthology.orchestra.run.smithi007.stdout:(80/159): python3-markupsafe-1.1.1-12.el9.x86_6 837 kB/s | 35 kB 00:00 2024-06-10T00:51:24.912 INFO:teuthology.orchestra.run.smithi175.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 7.7 MB/s | 1.3 MB 00:00 2024-06-10T00:51:24.987 INFO:teuthology.orchestra.run.smithi175.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-06-10T00:51:25.119 INFO:teuthology.orchestra.run.smithi078.stdout:(52/159): elfutils-libelf-devel-0.191-4.el9.x86 36 kB/s | 22 kB 00:00 2024-06-10T00:51:25.311 INFO:teuthology.orchestra.run.smithi078.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 56 kB/s | 33 kB 00:00 2024-06-10T00:51:25.529 INFO:teuthology.orchestra.run.smithi078.stdout:(54/159): flex-2.6.4-9.el9.x86_64.rpm 361 kB/s | 313 kB 00:00 2024-06-10T00:51:25.629 INFO:teuthology.orchestra.run.smithi175.stdout:(159/159): postgresql-server-13.14-1.el9.x86_64 718 kB/s | 5.8 MB 00:08 2024-06-10T00:51:25.635 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-06-10T00:51:25.635 INFO:teuthology.orchestra.run.smithi175.stdout:Total 12 MB/s | 225 MB 00:19 2024-06-10T00:51:25.661 INFO:teuthology.orchestra.run.smithi007.stdout:(81/159): postgresql-server-13.14-1.el9.x86_64. 3.2 MB/s | 5.8 MB 00:01 2024-06-10T00:51:25.762 INFO:teuthology.orchestra.run.smithi007.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.5 MB/s | 250 kB 00:00 2024-06-10T00:51:25.829 INFO:teuthology.orchestra.run.smithi078.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 35 kB/s | 18 kB 00:00 2024-06-10T00:51:25.862 INFO:teuthology.orchestra.run.smithi007.stdout:(83/159): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-06-10T00:51:25.922 INFO:teuthology.orchestra.run.smithi007.stdout:(84/159): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-06-10T00:51:25.962 INFO:teuthology.orchestra.run.smithi007.stdout:(85/159): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-06-10T00:51:26.029 INFO:teuthology.orchestra.run.smithi078.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 22 kB/s | 11 kB 00:00 2024-06-10T00:51:26.079 INFO:teuthology.orchestra.run.smithi007.stdout:(86/159): python3-protobuf-3.14.0-13.el9.noarch 2.3 MB/s | 269 kB 00:00 2024-06-10T00:51:26.172 INFO:teuthology.orchestra.run.smithi007.stdout:(87/159): python3-babel-2.9.1-2.el9.noarch.rpm 3.1 MB/s | 6.0 MB 00:01 2024-06-10T00:51:26.214 INFO:teuthology.orchestra.run.smithi007.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-06-10T00:51:26.229 INFO:teuthology.orchestra.run.smithi078.stdout:(57/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.7 MB/s | 3.0 MB 00:01 2024-06-10T00:51:26.247 INFO:teuthology.orchestra.run.smithi007.stdout:(89/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.5 MB/s | 51 kB 00:00 2024-06-10T00:51:26.281 INFO:teuthology.orchestra.run.smithi007.stdout:(90/159): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-06-10T00:51:26.306 INFO:teuthology.orchestra.run.smithi007.stdout:(91/159): python3-pyasn1-modules-0.4.8-6.el9.no 2.0 MB/s | 279 kB 00:00 2024-06-10T00:51:26.348 INFO:teuthology.orchestra.run.smithi007.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-06-10T00:51:26.390 INFO:teuthology.orchestra.run.smithi007.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-06-10T00:51:26.507 INFO:teuthology.orchestra.run.smithi007.stdout:(94/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.1 MB/s | 5.1 MB 00:01 2024-06-10T00:51:26.532 INFO:teuthology.orchestra.run.smithi007.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 1.5 MB/s | 221 kB 00:00 2024-06-10T00:51:26.558 INFO:teuthology.orchestra.run.smithi007.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 715 kB/s | 36 kB 00:00 2024-06-10T00:51:26.583 INFO:teuthology.orchestra.run.smithi007.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-06-10T00:51:26.588 INFO:teuthology.orchestra.run.smithi078.stdout:(58/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 211 kB/s | 160 kB 00:00 2024-06-10T00:51:26.634 INFO:teuthology.orchestra.run.smithi007.stdout:(98/159): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-06-10T00:51:26.663 INFO:teuthology.orchestra.run.smithi078.stdout:(59/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 71 kB/s | 45 kB 00:00 2024-06-10T00:51:26.692 INFO:teuthology.orchestra.run.smithi007.stdout:(99/159): socat-1.7.4.1-5.el9.x86_64.rpm 2.2 MB/s | 305 kB 00:00 2024-06-10T00:51:26.717 INFO:teuthology.orchestra.run.smithi007.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.1 MB/s | 94 kB 00:00 2024-06-10T00:51:26.759 INFO:teuthology.orchestra.run.smithi007.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 951 kB/s | 64 kB 00:00 2024-06-10T00:51:26.785 INFO:teuthology.orchestra.run.smithi007.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 669 kB/s | 45 kB 00:00 2024-06-10T00:51:26.785 INFO:teuthology.orchestra.run.smithi007.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:51:26.786 INFO:teuthology.orchestra.run.smithi007.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:51:26.814 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-06-10T00:51:26.973 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-06-10T00:51:26.973 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-06-10T00:51:27.138 INFO:teuthology.orchestra.run.smithi078.stdout:(60/159): librdkafka-1.6.1-102.el9.x86_64.rpm 729 kB/s | 662 kB 00:00 2024-06-10T00:51:27.160 INFO:teuthology.orchestra.run.smithi007.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 56 kB/s | 22 kB 00:00 2024-06-10T00:51:27.194 INFO:teuthology.orchestra.run.smithi007.stdout:(104/159): abseil-cpp-20211102.0-3.el9.x86_64.r 16 MB/s | 548 kB 00:00 2024-06-10T00:51:27.227 INFO:teuthology.orchestra.run.smithi007.stdout:(105/159): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-06-10T00:51:27.252 INFO:teuthology.orchestra.run.smithi007.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 781 kB/s | 19 kB 00:00 2024-06-10T00:51:27.263 INFO:teuthology.orchestra.run.smithi078.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 42 kB/s | 25 kB 00:00 2024-06-10T00:51:27.361 INFO:teuthology.orchestra.run.smithi007.stdout:(107/159): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-06-10T00:51:27.386 INFO:teuthology.orchestra.run.smithi007.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-06-10T00:51:27.412 INFO:teuthology.orchestra.run.smithi007.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-06-10T00:51:27.521 INFO:teuthology.orchestra.run.smithi007.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 623 kB/s | 67 kB 00:00 2024-06-10T00:51:27.538 INFO:teuthology.orchestra.run.smithi078.stdout:(62/159): libtool-2.4.6-45.el9.x86_64.rpm 613 kB/s | 582 kB 00:00 2024-06-10T00:51:27.613 INFO:teuthology.orchestra.run.smithi007.stdout:(111/159): luarocks-3.9.2-5.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-06-10T00:51:27.655 INFO:teuthology.orchestra.run.smithi007.stdout:(112/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-06-10T00:51:27.697 INFO:teuthology.orchestra.run.smithi007.stdout:(113/159): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-06-10T00:51:27.730 INFO:teuthology.orchestra.run.smithi007.stdout:(114/159): protobuf-compiler-3.14.0-13.el9.x86_ 913 kB/s | 863 kB 00:00 2024-06-10T00:51:27.756 INFO:teuthology.orchestra.run.smithi007.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 744 kB/s | 43 kB 00:00 2024-06-10T00:51:27.772 INFO:teuthology.orchestra.run.smithi078.stdout:(63/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 75 kB/s | 47 kB 00:00 2024-06-10T00:51:27.806 INFO:teuthology.orchestra.run.smithi007.stdout:(116/159): python3-certifi-2021.10.8-2.el9.noar 689 kB/s | 15 kB 00:00 2024-06-10T00:51:27.832 INFO:teuthology.orchestra.run.smithi007.stdout:(117/159): python3-cachetools-4.2.4-1.el9.noarc 320 kB/s | 32 kB 00:00 2024-06-10T00:51:27.857 INFO:teuthology.orchestra.run.smithi007.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 3.5 MB/s | 172 kB 00:00 2024-06-10T00:51:27.891 INFO:teuthology.orchestra.run.smithi007.stdout:(119/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-06-10T00:51:27.916 INFO:teuthology.orchestra.run.smithi007.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-06-10T00:51:27.949 INFO:teuthology.orchestra.run.smithi007.stdout:(121/159): python3-google-auth-2.29.0-2.el9.noa 3.4 MB/s | 205 kB 00:00 2024-06-10T00:51:27.966 INFO:teuthology.orchestra.run.smithi007.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 8.5 MB/s | 144 kB 00:00 2024-06-10T00:51:27.992 INFO:teuthology.orchestra.run.smithi007.stdout:(123/159): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-06-10T00:51:28.008 INFO:teuthology.orchestra.run.smithi007.stdout:(124/159): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-06-10T00:51:28.026 INFO:teuthology.orchestra.run.smithi007.stdout:(125/159): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-06-10T00:51:28.052 INFO:teuthology.orchestra.run.smithi007.stdout:(126/159): python3-jaraco-functools-3.5.0-2.el9 737 kB/s | 19 kB 00:00 2024-06-10T00:51:28.068 INFO:teuthology.orchestra.run.smithi007.stdout:(127/159): python3-jaraco-text-3.2.0-6.el9.noar 1.3 MB/s | 20 kB 00:00 2024-06-10T00:51:28.089 INFO:teuthology.orchestra.run.smithi078.stdout:(64/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 354 kB/s | 292 kB 00:00 2024-06-10T00:51:28.110 INFO:teuthology.orchestra.run.smithi007.stdout:(128/159): python3-kubernetes-26.1.0-2.el9.noar 25 MB/s | 1.0 MB 00:00 2024-06-10T00:51:28.135 INFO:teuthology.orchestra.run.smithi007.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-06-10T00:51:28.160 INFO:teuthology.orchestra.run.smithi007.stdout:(130/159): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-06-10T00:51:28.186 INFO:teuthology.orchestra.run.smithi007.stdout:(131/159): python3-natsort-7.1.1-5.el9.noarch.r 2.2 MB/s | 58 kB 00:00 2024-06-10T00:51:28.236 INFO:teuthology.orchestra.run.smithi007.stdout:(132/159): python3-grpcio-1.46.7-10.el9.x86_64. 6.4 MB/s | 2.0 MB 00:00 2024-06-10T00:51:28.262 INFO:teuthology.orchestra.run.smithi007.stdout:(133/159): python3-portend-3.1.0-2.el9.noarch.r 665 kB/s | 16 kB 00:00 2024-06-10T00:51:28.287 INFO:teuthology.orchestra.run.smithi007.stdout:(134/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-06-10T00:51:28.312 INFO:teuthology.orchestra.run.smithi007.stdout:(135/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-06-10T00:51:28.337 INFO:teuthology.orchestra.run.smithi007.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-06-10T00:51:28.363 INFO:teuthology.orchestra.run.smithi007.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 610 kB/s | 31 kB 00:00 2024-06-10T00:51:28.364 INFO:teuthology.orchestra.run.smithi078.stdout:(65/159): lua-5.4.4-4.el9.x86_64.rpm 228 kB/s | 188 kB 00:00 2024-06-10T00:51:28.397 INFO:teuthology.orchestra.run.smithi007.stdout:(138/159): python3-rsa-4.9-2.el9.noarch.rpm 1.7 MB/s | 59 kB 00:00 2024-06-10T00:51:28.398 INFO:teuthology.orchestra.run.smithi078.stdout:(66/159): openblas-0.3.21-2.el9.x86_64.rpm 59 kB/s | 35 kB 00:00 2024-06-10T00:51:28.422 INFO:teuthology.orchestra.run.smithi007.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-06-10T00:51:28.447 INFO:teuthology.orchestra.run.smithi007.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-06-10T00:51:28.473 INFO:teuthology.orchestra.run.smithi007.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-06-10T00:51:28.498 INFO:teuthology.orchestra.run.smithi007.stdout:(142/159): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-06-10T00:51:28.523 INFO:teuthology.orchestra.run.smithi007.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-06-10T00:51:28.548 INFO:teuthology.orchestra.run.smithi007.stdout:(144/159): python3-zc-lockfile-2.0-10.el9.noarc 805 kB/s | 20 kB 00:00 2024-06-10T00:51:28.574 INFO:teuthology.orchestra.run.smithi007.stdout:(145/159): re2-20211101-3.el9.x86_64.rpm 7.5 MB/s | 192 kB 00:00 2024-06-10T00:51:28.607 INFO:teuthology.orchestra.run.smithi007.stdout:(146/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-06-10T00:51:28.666 INFO:teuthology.orchestra.run.smithi007.stdout:(147/159): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-06-10T00:51:28.771 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-06-10T00:51:28.771 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-06-10T00:51:29.108 INFO:teuthology.orchestra.run.smithi007.stdout:(148/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 7.1 MB/s | 3.1 MB 00:00 2024-06-10T00:51:29.248 INFO:teuthology.orchestra.run.smithi078.stdout:(67/159): openblas-openmp-0.3.21-2.el9.x86_64.r 4.1 MB/s | 4.7 MB 00:01 2024-06-10T00:51:29.283 INFO:teuthology.orchestra.run.smithi007.stdout:(149/159): NetworkManager-1.47.91-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-06-10T00:51:29.342 INFO:teuthology.orchestra.run.smithi007.stdout:(150/159): NetworkManager-libnm-1.47.91-1.el9.x 31 MB/s | 1.8 MB 00:00 2024-06-10T00:51:29.401 INFO:teuthology.orchestra.run.smithi007.stdout:(151/159): librados2-19.0.0-2946.g3d89accf.el9. 4.3 MB/s | 3.4 MB 00:00 2024-06-10T00:51:29.415 INFO:teuthology.orchestra.run.smithi078.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 1.6 MB/s | 1.6 MB 00:01 2024-06-10T00:51:29.426 INFO:teuthology.orchestra.run.smithi007.stdout:(152/159): NetworkManager-team-1.47.91-1.el9.x8 495 kB/s | 41 kB 00:00 2024-06-10T00:51:29.468 INFO:teuthology.orchestra.run.smithi007.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 888 kB/s | 37 kB 00:00 2024-06-10T00:51:29.510 INFO:teuthology.orchestra.run.smithi007.stdout:(154/159): NetworkManager-tui-1.47.91-1.el9.x86 2.2 MB/s | 248 kB 00:00 2024-06-10T00:51:29.535 INFO:teuthology.orchestra.run.smithi007.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 3.0 MB/s | 207 kB 00:00 2024-06-10T00:51:29.540 INFO:teuthology.orchestra.run.smithi078.stdout:(69/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.8 MB/s | 4.4 MB 00:01 2024-06-10T00:51:29.569 INFO:teuthology.orchestra.run.smithi007.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 4.3 MB/s | 260 kB 00:00 2024-06-10T00:51:29.611 INFO:teuthology.orchestra.run.smithi007.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 18 MB/s | 1.3 MB 00:00 2024-06-10T00:51:29.694 INFO:teuthology.orchestra.run.smithi007.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-06-10T00:51:30.224 INFO:teuthology.orchestra.run.smithi078.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 165 kB/s | 133 kB 00:00 2024-06-10T00:51:30.299 INFO:teuthology.orchestra.run.smithi078.stdout:(71/159): postgresql-contrib-13.14-1.el9.x86_64 869 kB/s | 911 kB 00:01 2024-06-10T00:51:30.800 INFO:teuthology.orchestra.run.smithi078.stdout:(72/159): postgresql-server-13.14-1.el9.x86_64. 4.7 MB/s | 5.8 MB 00:01 2024-06-10T00:51:31.016 INFO:teuthology.orchestra.run.smithi078.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 122 kB/s | 87 kB 00:00 2024-06-10T00:51:31.300 INFO:teuthology.orchestra.run.smithi078.stdout:(74/159): protobuf-3.14.0-13.el9.x86_64.rpm 961 kB/s | 1.0 MB 00:01 2024-06-10T00:51:31.667 INFO:teuthology.orchestra.run.smithi078.stdout:(75/159): python3-colorama-0.4.6-3.el9.noarch.r 94 kB/s | 61 kB 00:00 2024-06-10T00:51:31.944 INFO:teuthology.orchestra.run.smithi007.stdout:(159/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.9 MB/s | 16 MB 00:05 2024-06-10T00:51:31.951 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-06-10T00:51:31.951 INFO:teuthology.orchestra.run.smithi007.stdout:Total 10 MB/s | 225 MB 00:22 2024-06-10T00:51:32.009 INFO:teuthology.orchestra.run.smithi078.stdout:(76/159): python3-babel-2.9.1-2.el9.noarch.rpm 4.9 MB/s | 6.0 MB 00:01 2024-06-10T00:51:32.118 INFO:teuthology.orchestra.run.smithi078.stdout:(77/159): python3-devel-3.9.18-3.el9.x86_64.rpm 299 kB/s | 244 kB 00:00 2024-06-10T00:51:32.493 INFO:teuthology.orchestra.run.smithi078.stdout:(78/159): python3-jinja2-2.11.3-6.el9.noarch.rp 301 kB/s | 248 kB 00:00 2024-06-10T00:51:32.585 INFO:teuthology.orchestra.run.smithi078.stdout:(79/159): python3-jsonpatch-1.21-16.el9.noarch. 45 kB/s | 26 kB 00:00 2024-06-10T00:51:32.635 INFO:teuthology.orchestra.run.smithi078.stdout:(80/159): python3-jsonpointer-2.0-4.el9.noarch. 37 kB/s | 19 kB 00:00 2024-06-10T00:51:33.177 INFO:teuthology.orchestra.run.smithi078.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 59 kB/s | 35 kB 00:00 2024-06-10T00:51:33.193 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-06-10T00:51:33.277 INFO:teuthology.orchestra.run.smithi078.stdout:(82/159): python3-mako-1.1.4-6.el9.noarch.rpm 220 kB/s | 172 kB 00:00 2024-06-10T00:51:33.369 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-06-10T00:51:33.369 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-06-10T00:51:33.828 INFO:teuthology.orchestra.run.smithi078.stdout:(83/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.3 MB/s | 5.1 MB 00:01 2024-06-10T00:51:34.003 INFO:teuthology.orchestra.run.smithi078.stdout:(84/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 303 kB/s | 250 kB 00:00 2024-06-10T00:51:34.137 INFO:teuthology.orchestra.run.smithi078.stdout:(85/159): python3-oauthlib-3.1.1-5.el9.noarch.r 259 kB/s | 222 kB 00:00 2024-06-10T00:51:34.520 INFO:teuthology.orchestra.run.smithi078.stdout:(86/159): python3-packaging-20.9-5.el9.noarch.r 111 kB/s | 77 kB 00:00 2024-06-10T00:51:34.620 INFO:teuthology.orchestra.run.smithi078.stdout:(87/159): python3-prettytable-0.7.2-27.el9.noar 68 kB/s | 42 kB 00:00 2024-06-10T00:51:34.962 INFO:teuthology.orchestra.run.smithi078.stdout:(88/159): python3-protobuf-3.14.0-13.el9.noarch 326 kB/s | 269 kB 00:00 2024-06-10T00:51:35.116 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-06-10T00:51:35.199 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-10T00:51:35.237 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-10T00:51:35.292 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-10T00:51:35.321 INFO:teuthology.orchestra.run.smithi078.stdout:(89/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 198 kB/s | 159 kB 00:00 2024-06-10T00:51:35.324 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-10T00:51:35.350 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-10T00:51:35.393 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-10T00:51:35.471 INFO:teuthology.orchestra.run.smithi078.stdout:(90/159): python3-pyasn1-modules-0.4.8-6.el9.no 328 kB/s | 279 kB 00:00 2024-06-10T00:51:35.584 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-06-10T00:51:35.585 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-06-10T00:51:35.621 INFO:teuthology.orchestra.run.smithi078.stdout:(91/159): python3-pytz-2021.1-5.el9.noarch.rpm 77 kB/s | 51 kB 00:00 2024-06-10T00:51:35.724 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-10T00:51:35.834 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T00:51:35.956 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T00:51:35.985 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T00:51:35.988 INFO:teuthology.orchestra.run.smithi078.stdout:(92/159): python3-requests-oauthlib-1.3.0-12.el 80 kB/s | 54 kB 00:00 2024-06-10T00:51:36.076 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T00:51:36.106 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-10T00:51:36.139 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-10T00:51:36.230 INFO:teuthology.orchestra.run.smithi078.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 68 kB/s | 42 kB 00:00 2024-06-10T00:51:36.317 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-10T00:51:36.379 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-10T00:51:36.407 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T00:51:36.509 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T00:51:36.533 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-10T00:51:36.595 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-10T00:51:36.617 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-10T00:51:36.644 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-10T00:51:36.647 INFO:teuthology.orchestra.run.smithi078.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 67 kB/s | 44 kB 00:00 2024-06-10T00:51:36.852 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-10T00:51:36.973 INFO:teuthology.orchestra.run.smithi078.stdout:(95/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-06-10T00:51:36.977 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-10T00:51:37.098 INFO:teuthology.orchestra.run.smithi078.stdout:(96/159): qatlib-24.02.0-1.el9.x86_64.rpm 262 kB/s | 221 kB 00:00 2024-06-10T00:51:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-10T00:51:37.216 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-10T00:51:37.311 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T00:51:37.384 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T00:51:37.507 INFO:teuthology.orchestra.run.smithi078.stdout:(97/159): qatlib-service-24.02.0-1.el9.x86_64.r 42 kB/s | 36 kB 00:00 2024-06-10T00:51:37.588 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T00:51:37.694 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-10T00:51:37.732 INFO:teuthology.orchestra.run.smithi078.stdout:(98/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 65 kB/s | 47 kB 00:00 2024-06-10T00:51:37.786 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-10T00:51:37.823 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-10T00:51:37.858 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-10T00:51:37.923 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-10T00:51:37.957 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-10T00:51:38.001 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-10T00:51:38.033 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-10T00:51:38.040 INFO:teuthology.orchestra.run.smithi078.stdout:(99/159): socat-1.7.4.1-5.el9.x86_64.rpm 324 kB/s | 305 kB 00:00 2024-06-10T00:51:38.061 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-10T00:51:38.130 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-10T00:51:38.166 INFO:teuthology.orchestra.run.smithi078.stdout:(100/159): uuid-1.6.2-55.el9.x86_64.rpm 88 kB/s | 58 kB 00:00 2024-06-10T00:51:38.183 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-10T00:51:38.214 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-10T00:51:38.250 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-10T00:51:38.325 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-10T00:51:38.360 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-10T00:51:38.395 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-10T00:51:38.423 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-10T00:51:38.474 INFO:teuthology.orchestra.run.smithi078.stdout:(101/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 131 kB/s | 94 kB 00:00 2024-06-10T00:51:38.493 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-10T00:51:38.511 INFO:teuthology.orchestra.run.smithi078.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:51:38.537 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-10T00:51:38.591 INFO:teuthology.orchestra.run.smithi078.stdout:(102/159): lua-devel-5.4.4-4.el9.x86_64.rpm 191 kB/s | 22 kB 00:00 2024-06-10T00:51:38.592 INFO:teuthology.orchestra.run.smithi078.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:51:38.612 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-10T00:51:38.717 INFO:teuthology.orchestra.run.smithi078.stdout:(103/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 94 kB/s | 64 kB 00:00 2024-06-10T00:51:38.747 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-10T00:51:38.758 INFO:teuthology.orchestra.run.smithi078.stdout:(104/159): abseil-cpp-20211102.0-3.el9.x86_64.r 13 MB/s | 548 kB 00:00 2024-06-10T00:51:38.783 INFO:teuthology.orchestra.run.smithi078.stdout:(105/159): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-06-10T00:51:38.799 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-10T00:51:38.809 INFO:teuthology.orchestra.run.smithi078.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 777 kB/s | 19 kB 00:00 2024-06-10T00:51:38.834 INFO:teuthology.orchestra.run.smithi078.stdout:(107/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 67 kB/s | 45 kB 00:00 2024-06-10T00:51:38.868 INFO:teuthology.orchestra.run.smithi078.stdout:(108/159): protobuf-compiler-3.14.0-13.el9.x86_ 3.0 MB/s | 863 kB 00:00 2024-06-10T00:51:38.951 INFO:teuthology.orchestra.run.smithi078.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 222 kB/s | 26 kB 00:00 2024-06-10T00:51:38.994 INFO:teuthology.orchestra.run.smithi078.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 1.7 MB/s | 67 kB 00:00 2024-06-10T00:51:39.019 INFO:teuthology.orchestra.run.smithi078.stdout:(111/159): liboath-2.6.7-2.el9.x86_64.rpm 389 kB/s | 49 kB 00:00 2024-06-10T00:51:39.044 INFO:teuthology.orchestra.run.smithi078.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-06-10T00:51:39.128 INFO:teuthology.orchestra.run.smithi078.stdout:(113/159): libarrow-9.0.0-11.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-06-10T00:51:39.162 INFO:teuthology.orchestra.run.smithi078.stdout:(114/159): python3-asyncssh-2.13.2-5.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-06-10T00:51:39.187 INFO:teuthology.orchestra.run.smithi078.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 740 kB/s | 43 kB 00:00 2024-06-10T00:51:39.212 INFO:teuthology.orchestra.run.smithi078.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-06-10T00:51:39.229 INFO:teuthology.orchestra.run.smithi078.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 358 kB/s | 15 kB 00:00 2024-06-10T00:51:39.263 INFO:teuthology.orchestra.run.smithi078.stdout:(118/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.4 MB/s | 839 kB 00:00 2024-06-10T00:51:39.288 INFO:teuthology.orchestra.run.smithi078.stdout:(119/159): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-06-10T00:51:39.322 INFO:teuthology.orchestra.run.smithi078.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-06-10T00:51:39.329 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-10T00:51:39.347 INFO:teuthology.orchestra.run.smithi078.stdout:(121/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-06-10T00:51:39.372 INFO:teuthology.orchestra.run.smithi078.stdout:(122/159): python3-google-auth-2.29.0-2.el9.noa 2.4 MB/s | 205 kB 00:00 2024-06-10T00:51:39.398 INFO:teuthology.orchestra.run.smithi078.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-06-10T00:51:39.413 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-10T00:51:39.422 INFO:teuthology.orchestra.run.smithi078.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-06-10T00:51:39.442 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-10T00:51:39.449 INFO:teuthology.orchestra.run.smithi078.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 708 kB/s | 18 kB 00:00 2024-06-10T00:51:39.473 INFO:teuthology.orchestra.run.smithi078.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-06-10T00:51:39.478 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-10T00:51:39.499 INFO:teuthology.orchestra.run.smithi078.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-06-10T00:51:39.524 INFO:teuthology.orchestra.run.smithi078.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 390 kB/s | 20 kB 00:00 2024-06-10T00:51:39.527 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-10T00:51:39.549 INFO:teuthology.orchestra.run.smithi078.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-06-10T00:51:39.566 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-10T00:51:39.574 INFO:teuthology.orchestra.run.smithi078.stdout:(130/159): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-06-10T00:51:39.600 INFO:teuthology.orchestra.run.smithi078.stdout:(131/159): python3-natsort-7.1.1-5.el9.noarch.r 2.2 MB/s | 58 kB 00:00 2024-06-10T00:51:39.607 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-10T00:51:39.633 INFO:teuthology.orchestra.run.smithi078.stdout:(132/159): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-06-10T00:51:39.659 INFO:teuthology.orchestra.run.smithi078.stdout:(133/159): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-06-10T00:51:39.684 INFO:teuthology.orchestra.run.smithi078.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 326 kB/s | 16 kB 00:00 2024-06-10T00:51:39.709 INFO:teuthology.orchestra.run.smithi078.stdout:(135/159): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-06-10T00:51:39.735 INFO:teuthology.orchestra.run.smithi078.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-06-10T00:51:39.760 INFO:teuthology.orchestra.run.smithi078.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 606 kB/s | 31 kB 00:00 2024-06-10T00:51:39.762 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-10T00:51:39.786 INFO:teuthology.orchestra.run.smithi078.stdout:(138/159): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-06-10T00:51:39.835 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-10T00:51:39.836 INFO:teuthology.orchestra.run.smithi078.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-06-10T00:51:39.891 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-10T00:51:39.895 INFO:teuthology.orchestra.run.smithi078.stdout:(140/159): python3-grpcio-1.46.7-10.el9.x86_64. 3.6 MB/s | 2.0 MB 00:00 2024-06-10T00:51:39.920 INFO:teuthology.orchestra.run.smithi078.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 266 kB/s | 36 kB 00:00 2024-06-10T00:51:39.945 INFO:teuthology.orchestra.run.smithi078.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 467 kB/s | 51 kB 00:00 2024-06-10T00:51:39.970 INFO:teuthology.orchestra.run.smithi078.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-06-10T00:51:39.988 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-10T00:51:39.996 INFO:teuthology.orchestra.run.smithi078.stdout:(144/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-06-10T00:51:40.019 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-10T00:51:40.021 INFO:teuthology.orchestra.run.smithi078.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 396 kB/s | 20 kB 00:00 2024-06-10T00:51:40.047 INFO:teuthology.orchestra.run.smithi078.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-06-10T00:51:40.056 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-10T00:51:40.072 INFO:teuthology.orchestra.run.smithi078.stdout:(147/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-06-10T00:51:40.103 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-10T00:51:40.122 INFO:teuthology.orchestra.run.smithi078.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-06-10T00:51:40.139 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-10T00:51:40.182 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-10T00:51:40.224 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-10T00:51:40.281 INFO:teuthology.orchestra.run.smithi078.stdout:(149/159): NetworkManager-1.47.91-1.el9.x86_64. 15 MB/s | 2.3 MB 00:00 2024-06-10T00:51:40.348 INFO:teuthology.orchestra.run.smithi078.stdout:(150/159): NetworkManager-libnm-1.47.91-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-06-10T00:51:40.359 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-10T00:51:40.381 INFO:teuthology.orchestra.run.smithi078.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 1.2 MB/s | 41 kB 00:00 2024-06-10T00:51:40.401 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-10T00:51:40.423 INFO:teuthology.orchestra.run.smithi078.stdout:(152/159): NetworkManager-tui-1.47.91-1.el9.x86 5.8 MB/s | 248 kB 00:00 2024-06-10T00:51:40.442 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-10T00:51:40.457 INFO:teuthology.orchestra.run.smithi078.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 1.1 MB/s | 37 kB 00:00 2024-06-10T00:51:40.499 INFO:teuthology.orchestra.run.smithi078.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 4.9 MB/s | 207 kB 00:00 2024-06-10T00:51:40.537 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-10T00:51:40.541 INFO:teuthology.orchestra.run.smithi078.stdout:(155/159): elfutils-libs-0.191-4.el9.x86_64.rpm 6.1 MB/s | 260 kB 00:00 2024-06-10T00:51:40.567 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-google-auth-1:2.29.0-2.el9.noarch 63/170 2024-06-10T00:51:40.599 INFO:teuthology.orchestra.run.smithi078.stdout:(156/159): openssl-3.2.1-1.el9.x86_64.rpm 23 MB/s | 1.3 MB 00:00 2024-06-10T00:51:40.607 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-10T00:51:40.633 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-10T00:51:40.658 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-10T00:51:40.675 INFO:teuthology.orchestra.run.smithi078.stdout:(157/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 32 MB/s | 2.4 MB 00:00 2024-06-10T00:51:40.732 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-10T00:51:40.742 INFO:teuthology.orchestra.run.smithi078.stdout:(158/159): librados2-19.0.0-2946.g3d89accf.el9. 4.9 MB/s | 3.4 MB 00:00 2024-06-10T00:51:40.861 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-10T00:51:40.899 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-10T00:51:40.942 INFO:teuthology.orchestra.run.smithi078.stdout:(159/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 3.6 MB/s | 3.1 MB 00:00 2024-06-10T00:51:40.947 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-06-10T00:51:40.947 INFO:teuthology.orchestra.run.smithi078.stdout:Total 8.4 MB/s | 225 MB 00:26 2024-06-10T00:51:41.002 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-10T00:51:41.061 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-10T00:51:41.100 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-10T00:51:41.132 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-10T00:51:41.162 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-10T00:51:41.196 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-10T00:51:41.230 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-10T00:51:41.271 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-10T00:51:41.306 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-10T00:51:41.394 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-10T00:51:41.436 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-10T00:51:41.467 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-10T00:51:41.514 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-10T00:51:41.613 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-10T00:51:41.669 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-10T00:51:42.184 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-06-10T00:51:42.311 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-06-10T00:51:42.343 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-06-10T00:51:42.343 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-06-10T00:51:42.399 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-10T00:51:42.416 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-10T00:51:42.441 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-10T00:51:42.455 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-10T00:51:42.483 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-10T00:51:42.524 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-10T00:51:42.558 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-10T00:51:42.583 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-10T00:51:42.583 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-10T00:51:42.629 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-10T00:51:42.638 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-10T00:51:42.684 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-10T00:51:42.772 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-10T00:51:42.811 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-10T00:51:42.997 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-10T00:51:43.033 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-10T00:51:43.048 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-10T00:51:43.109 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T00:51:43.111 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-10T00:51:43.234 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T00:51:43.260 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T00:51:43.374 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T00:51:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-10T00:51:43.587 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-10T00:51:43.759 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-10T00:51:43.793 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-10T00:51:43.847 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-10T00:51:43.866 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-10T00:51:43.875 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T00:51:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T00:51:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-10T00:51:44.072 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-10T00:51:44.085 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-10T00:51:44.127 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-10T00:51:44.286 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-06-10T00:51:44.287 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-06-10T00:51:44.330 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-10T00:51:44.507 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-10T00:51:44.675 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-10T00:51:44.735 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-10T00:51:44.850 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T00:51:44.936 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T00:51:45.102 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-10T00:51:45.145 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T00:51:45.188 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-10T00:51:45.216 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-10T00:51:45.249 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-10T00:51:45.260 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-10T00:51:45.368 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-10T00:51:45.418 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-10T00:51:45.453 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-10T00:51:45.529 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-10T00:51:45.562 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-10T00:51:45.568 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-10T00:51:45.632 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-10T00:51:45.633 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-10T00:51:45.645 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-10T00:51:45.670 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-10T00:51:45.672 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-10T00:51:45.700 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-10T00:51:45.736 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-10T00:51:45.744 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-10T00:51:45.807 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-10T00:51:45.836 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-10T00:51:45.879 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-10T00:51:45.958 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-10T00:51:45.998 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-10T00:51:46.043 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-10T00:51:46.070 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-10T00:51:46.145 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-10T00:51:46.194 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-10T00:51:46.210 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-10T00:51:46.240 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-10T00:51:46.280 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-10T00:51:46.311 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-10T00:51:46.333 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-10T00:51:46.428 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-10T00:51:46.490 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-10T00:51:47.055 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-10T00:51:47.152 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-10T00:51:47.187 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-10T00:51:47.224 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-10T00:51:47.273 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-10T00:51:47.311 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-10T00:51:47.353 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-10T00:51:47.509 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-10T00:51:47.564 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-10T00:51:47.620 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-10T00:51:47.702 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-10T00:51:47.739 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-10T00:51:47.776 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-10T00:51:47.824 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-10T00:51:47.858 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-10T00:51:47.902 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-10T00:51:47.953 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-10T00:51:48.098 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-10T00:51:48.137 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-10T00:51:48.179 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-10T00:51:48.277 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-10T00:51:48.312 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-google-auth-1:2.29.0-2.el9.noarch 63/170 2024-06-10T00:51:48.356 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-10T00:51:48.394 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-10T00:51:48.418 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-10T00:51:48.517 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-10T00:51:48.560 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T00:51:48.597 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T00:51:48.653 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-10T00:51:48.684 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-10T00:51:48.712 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-10T00:51:48.747 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-10T00:51:48.786 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-10T00:51:48.826 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-10T00:51:48.858 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-10T00:51:48.890 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-10T00:51:48.924 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-10T00:51:48.966 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-10T00:51:49.000 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-10T00:51:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-10T00:51:49.137 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T00:51:49.140 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-10T00:51:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-10T00:51:49.175 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-10T00:51:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-10T00:51:49.268 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-10T00:51:49.268 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-10T00:51:49.268 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-10T00:51:49.268 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-10T00:51:49.301 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-10T00:51:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-10T00:51:49.398 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-10T00:51:50.179 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-10T00:51:50.235 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-10T00:51:50.269 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-10T00:51:50.376 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-10T00:51:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-10T00:51:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-10T00:51:50.547 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-06-10T00:51:50.570 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-10T00:51:50.613 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-10T00:51:50.622 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-10T00:51:50.660 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-10T00:51:50.713 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-10T00:51:50.748 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-10T00:51:50.773 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-10T00:51:50.798 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-10T00:51:50.816 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-10T00:51:50.827 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-10T00:51:50.909 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-10T00:51:51.207 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-10T00:51:51.266 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T00:51:51.388 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T00:51:51.409 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T00:51:51.501 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T00:51:51.522 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-10T00:51:51.554 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-10T00:51:51.614 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-10T00:51:51.730 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-10T00:51:51.763 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-10T00:51:51.811 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-10T00:51:51.848 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T00:51:51.952 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T00:51:51.974 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-10T00:51:52.036 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-10T00:51:52.059 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-10T00:51:52.084 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-10T00:51:52.284 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-10T00:51:52.409 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-10T00:51:52.556 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-10T00:51:52.608 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-10T00:51:52.699 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T00:51:52.784 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T00:51:52.973 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T00:51:53.083 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-10T00:51:53.098 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-10T00:51:53.179 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-10T00:51:53.197 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-10T00:51:53.208 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-10T00:51:53.225 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-10T00:51:53.243 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-10T00:51:53.258 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-10T00:51:53.307 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-10T00:51:53.342 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-10T00:51:53.386 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-10T00:51:53.418 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-10T00:51:53.446 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-10T00:51:53.514 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-10T00:51:53.567 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-10T00:51:53.586 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-10T00:51:53.598 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-10T00:51:53.634 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-10T00:51:53.637 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-10T00:51:53.710 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-10T00:51:53.729 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-10T00:51:53.745 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-10T00:51:53.759 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-10T00:51:53.787 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-10T00:51:53.796 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-10T00:51:53.817 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-10T00:51:53.887 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-10T00:51:53.932 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-10T00:51:54.005 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-10T00:51:54.149 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-10T00:51:54.201 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-10T00:51:54.294 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-10T00:51:54.324 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-10T00:51:54.395 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-10T00:51:54.425 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-10T00:51:54.723 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-10T00:51:54.804 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-10T00:51:54.833 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-10T00:51:54.869 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-10T00:51:54.910 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-10T00:51:54.948 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-10T00:51:54.990 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-10T00:51:55.150 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-10T00:51:55.209 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-10T00:51:55.264 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-10T00:51:55.337 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-10T00:51:55.368 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-10T00:51:55.404 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-10T00:51:55.452 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-10T00:51:55.487 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-10T00:51:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-10T00:51:55.573 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-10T00:51:55.710 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-10T00:51:55.791 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-10T00:51:55.832 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-10T00:51:55.928 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-10T00:51:55.965 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-google-auth-1:2.29.0-2.el9.noarch 63/170 2024-06-10T00:51:56.005 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-10T00:51:56.031 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-10T00:51:56.056 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-10T00:51:56.130 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-10T00:51:56.257 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-10T00:51:56.296 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-10T00:51:56.324 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-10T00:51:56.358 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-10T00:51:56.397 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-10T00:51:56.429 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-10T00:51:56.459 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-10T00:51:56.494 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-10T00:51:56.527 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-10T00:51:56.577 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-10T00:51:56.611 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-10T00:51:56.698 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-10T00:51:56.741 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-10T00:51:56.773 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-10T00:51:56.802 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T00:51:56.820 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-10T00:51:56.840 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T00:51:56.919 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-10T00:51:56.966 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-10T00:51:57.413 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T00:51:57.451 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-10T00:51:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-10T00:51:57.526 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-10T00:51:57.526 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-10T00:51:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:51:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-10T00:51:57.687 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-10T00:51:57.738 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-10T00:51:57.771 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-10T00:51:57.870 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-10T00:51:57.918 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-10T00:51:57.963 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-10T00:51:58.050 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-10T00:51:58.082 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-10T00:51:58.269 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-10T00:51:58.296 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-10T00:51:58.366 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-10T00:51:59.009 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-10T00:51:59.136 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-10T00:52:00.358 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-10T00:52:00.441 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-10T00:52:00.478 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-10T00:52:00.511 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-10T00:52:00.728 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-10T00:52:00.728 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /sys 2024-06-10T00:52:00.728 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /proc 2024-06-10T00:52:00.728 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /mnt 2024-06-10T00:52:00.728 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /var/tmp 2024-06-10T00:52:00.728 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /home 2024-06-10T00:52:00.728 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /root 2024-06-10T00:52:00.729 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /tmp 2024-06-10T00:52:00.729 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-10T00:52:00.823 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-10T00:52:00.874 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-10T00:52:00.897 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-10T00:52:00.913 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-10T00:52:00.913 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:52:00.914 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-10T00:52:00.914 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-10T00:52:00.965 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-10T00:52:00.995 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-10T00:52:01.031 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-10T00:52:01.504 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-10T00:52:01.527 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-10T00:52:01.597 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-10T00:52:01.611 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-10T00:52:02.408 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-10T00:52:02.434 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-10T00:52:02.434 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:52:02.434 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-10T00:52:02.434 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-10T00:52:02.434 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-10T00:52:02.434 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-10T00:52:02.658 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-10T00:52:02.681 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-10T00:52:02.681 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:52:02.681 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-10T00:52:02.681 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-10T00:52:02.681 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-10T00:52:02.681 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-10T00:52:03.117 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-10T00:52:03.140 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-10T00:52:03.140 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:52:03.140 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-10T00:52:03.140 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-10T00:52:03.140 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-10T00:52:03.140 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-10T00:52:03.255 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-10T00:52:03.313 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-10T00:52:03.368 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-10T00:52:03.553 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-10T00:52:03.601 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-10T00:52:03.679 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-10T00:52:03.767 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-10T00:52:03.854 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T00:52:03.883 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T00:52:03.907 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-10T00:52:03.959 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-10T00:52:04.000 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-10T00:52:04.035 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-10T00:52:04.424 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T00:52:04.428 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-10T00:52:04.459 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-10T00:52:04.469 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-10T00:52:04.490 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-10T00:52:04.553 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-10T00:52:04.560 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-10T00:52:04.560 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-10T00:52:04.560 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-10T00:52:04.560 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-06-10T00:52:04.578 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-10T00:52:04.582 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-10T00:52:05.622 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-10T00:52:05.659 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-10T00:52:06.627 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-10T00:52:06.659 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-10T00:52:06.762 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-10T00:52:06.923 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-10T00:52:06.953 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-10T00:52:06.976 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-10T00:52:06.976 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:52:06.976 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-10T00:52:06.976 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-10T00:52:06.977 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-10T00:52:06.977 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-10T00:52:07.020 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-10T00:52:07.031 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-10T00:52:07.129 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-10T00:52:07.178 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-10T00:52:08.085 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-10T00:52:08.131 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-10T00:52:08.153 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-10T00:52:08.153 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:52:08.154 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-10T00:52:08.154 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-10T00:52:08.154 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-10T00:52:08.154 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-10T00:52:08.207 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-10T00:52:08.228 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-10T00:52:08.229 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:52:08.229 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-10T00:52:08.229 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-10T00:52:08.274 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-10T00:52:08.296 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-10T00:52:08.296 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:52:08.296 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-10T00:52:08.297 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-10T00:52:08.297 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-10T00:52:08.297 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-10T00:52:08.600 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-10T00:52:08.622 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-10T00:52:08.622 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:52:08.622 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-10T00:52:08.622 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-10T00:52:08.622 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-10T00:52:08.622 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-10T00:52:09.719 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-10T00:52:09.719 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /sys 2024-06-10T00:52:09.719 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /proc 2024-06-10T00:52:09.719 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /mnt 2024-06-10T00:52:09.719 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /var/tmp 2024-06-10T00:52:09.719 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /home 2024-06-10T00:52:09.719 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /root 2024-06-10T00:52:09.719 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /tmp 2024-06-10T00:52:09.720 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:52:09.882 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-10T00:52:09.898 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-10T00:52:09.898 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:52:09.898 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-10T00:52:09.898 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:52:11.424 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-10T00:52:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-10T00:52:11.454 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:52:11.454 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-10T00:52:11.454 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-10T00:52:11.454 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-10T00:52:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:52:11.692 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-10T00:52:11.717 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-10T00:52:11.717 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:52:11.717 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-10T00:52:11.717 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-10T00:52:11.717 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-10T00:52:11.718 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:52:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-10T00:52:12.211 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-10T00:52:12.211 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:52:12.211 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-10T00:52:12.211 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-10T00:52:12.211 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-10T00:52:12.211 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:52:12.318 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-10T00:52:12.375 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-10T00:52:12.439 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-10T00:52:12.633 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-10T00:52:12.679 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-10T00:52:12.759 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-10T00:52:12.849 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-10T00:52:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-10T00:52:13.029 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-10T00:52:13.069 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-10T00:52:13.096 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-10T00:52:13.144 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-10T00:52:13.186 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-10T00:52:13.227 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-10T00:52:13.259 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-10T00:52:13.286 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-10T00:52:13.468 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-10T00:52:13.515 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-10T00:52:13.525 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-10T00:52:13.547 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-10T00:52:13.556 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-10T00:52:13.588 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-10T00:52:13.596 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-10T00:52:13.626 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-10T00:52:13.652 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-10T00:52:13.672 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-10T00:52:13.679 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-10T00:52:14.247 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T00:52:14.286 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T00:52:14.306 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T00:52:14.772 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-10T00:52:14.806 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-10T00:52:15.141 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-10T00:52:15.240 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-10T00:52:15.282 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-10T00:52:15.373 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-10T00:52:15.410 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-10T00:52:15.435 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-10T00:52:15.460 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-10T00:52:15.487 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-10T00:52:15.487 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-10T00:52:15.572 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-10T00:52:15.604 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-10T00:52:15.605 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-10T00:52:15.697 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-10T00:52:15.730 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-10T00:52:15.735 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T00:52:15.735 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T00:52:15.825 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-10T00:52:15.848 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-10T00:52:15.924 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T00:52:15.924 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-10T00:52:15.956 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-10T00:52:15.997 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-10T00:52:15.997 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /sys 2024-06-10T00:52:15.997 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /proc 2024-06-10T00:52:15.997 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /mnt 2024-06-10T00:52:15.998 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /var/tmp 2024-06-10T00:52:15.998 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /home 2024-06-10T00:52:15.998 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /root 2024-06-10T00:52:15.998 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /tmp 2024-06-10T00:52:15.998 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-06-10T00:52:16.016 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-10T00:52:16.041 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-10T00:52:16.106 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-10T00:52:16.127 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-10T00:52:16.158 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-10T00:52:16.169 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-10T00:52:16.183 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-10T00:52:16.183 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:52:16.183 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-10T00:52:16.183 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-10T00:52:16.184 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-10T00:52:16.184 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:52:16.185 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-10T00:52:16.185 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:52:16.185 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-10T00:52:16.185 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-06-10T00:52:16.234 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-10T00:52:16.246 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-10T00:52:16.347 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-10T00:52:16.406 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-10T00:52:17.356 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-10T00:52:17.387 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-10T00:52:17.412 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-10T00:52:17.412 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:52:17.412 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-10T00:52:17.412 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-10T00:52:17.412 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-10T00:52:17.412 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:52:17.471 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-10T00:52:17.495 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-10T00:52:17.495 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:52:17.495 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-10T00:52:17.495 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:52:17.555 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-10T00:52:17.579 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-10T00:52:17.579 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:52:17.579 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-10T00:52:17.580 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-10T00:52:17.580 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-10T00:52:17.580 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:52:17.673 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-10T00:52:17.698 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-10T00:52:17.699 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:52:17.699 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-10T00:52:17.699 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-10T00:52:17.699 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-10T00:52:17.699 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-06-10T00:52:17.905 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-10T00:52:17.924 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-10T00:52:17.930 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-10T00:52:17.930 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:52:17.930 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-10T00:52:17.930 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-10T00:52:17.930 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-10T00:52:17.930 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:52:17.946 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-10T00:52:17.946 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:52:17.946 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-10T00:52:17.947 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-10T00:52:17.947 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-10T00:52:17.947 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-06-10T00:52:18.391 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-10T00:52:18.413 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-10T00:52:18.414 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:52:18.414 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-10T00:52:18.414 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-10T00:52:18.414 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-10T00:52:18.414 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-06-10T00:52:18.512 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-10T00:52:18.569 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-10T00:52:18.625 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-10T00:52:18.808 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-10T00:52:18.859 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-10T00:52:18.935 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-10T00:52:19.024 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-10T00:52:19.162 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-10T00:52:19.208 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-10T00:52:19.240 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-10T00:52:19.267 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-10T00:52:19.728 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-10T00:52:19.766 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-10T00:52:19.798 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-10T00:52:19.859 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-10T00:52:19.889 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-10T00:52:20.916 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-10T00:52:20.949 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-10T00:52:21.526 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-10T00:52:21.526 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-10T00:52:21.526 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-10T00:52:21.526 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-10T00:52:21.526 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-10T00:52:21.526 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-10T00:52:21.526 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-10T00:52:21.526 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-10T00:52:21.526 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T00:52:21.526 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T00:52:21.527 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-10T00:52:21.527 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-10T00:52:21.527 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-10T00:52:21.527 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-10T00:52:21.527 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T00:52:21.527 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-10T00:52:21.527 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-10T00:52:21.527 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-10T00:52:21.527 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-10T00:52:21.527 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-10T00:52:21.527 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-10T00:52:21.527 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-10T00:52:21.528 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-10T00:52:21.528 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-10T00:52:21.528 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-10T00:52:21.528 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-10T00:52:21.528 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-10T00:52:21.528 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-10T00:52:21.528 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-10T00:52:21.528 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-10T00:52:21.528 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-10T00:52:21.528 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-10T00:52:21.528 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-10T00:52:21.528 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-10T00:52:21.528 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-10T00:52:21.529 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-10T00:52:21.529 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-10T00:52:21.529 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-10T00:52:21.529 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-10T00:52:21.529 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-10T00:52:21.529 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-10T00:52:21.529 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-10T00:52:21.529 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-10T00:52:21.529 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-10T00:52:21.529 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-10T00:52:21.529 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-10T00:52:21.529 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-10T00:52:21.530 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-10T00:52:21.530 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-10T00:52:21.530 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-10T00:52:21.530 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-10T00:52:21.530 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-10T00:52:21.530 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-10T00:52:21.530 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-10T00:52:21.530 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-10T00:52:21.530 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-10T00:52:21.530 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-10T00:52:21.530 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-10T00:52:21.530 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-10T00:52:21.530 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-10T00:52:21.531 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-10T00:52:21.531 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-10T00:52:21.531 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-10T00:52:21.531 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-10T00:52:21.531 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-10T00:52:21.531 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-10T00:52:21.531 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-10T00:52:21.531 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-10T00:52:21.531 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-10T00:52:21.531 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-10T00:52:21.531 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-10T00:52:21.531 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-10T00:52:21.531 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-10T00:52:21.532 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-10T00:52:21.532 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-10T00:52:21.532 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-10T00:52:21.532 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-10T00:52:21.532 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-10T00:52:21.532 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-10T00:52:21.532 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-10T00:52:21.532 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-10T00:52:21.532 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-10T00:52:21.532 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-10T00:52:21.532 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-10T00:52:21.532 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-10T00:52:21.532 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-10T00:52:21.532 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-10T00:52:21.533 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-10T00:52:21.533 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-10T00:52:21.533 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-10T00:52:21.533 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-10T00:52:21.533 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-10T00:52:21.533 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-10T00:52:21.533 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-10T00:52:21.533 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-10T00:52:21.533 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-10T00:52:21.533 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-10T00:52:21.533 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-10T00:52:21.533 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-10T00:52:21.533 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-10T00:52:21.533 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-10T00:52:21.534 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-10T00:52:21.534 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-10T00:52:21.534 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-10T00:52:21.534 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-10T00:52:21.534 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-10T00:52:21.534 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-10T00:52:21.534 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-10T00:52:21.534 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-10T00:52:21.534 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-10T00:52:21.534 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-10T00:52:21.534 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-10T00:52:21.534 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-10T00:52:21.534 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-10T00:52:21.535 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-10T00:52:21.535 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-10T00:52:21.535 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-10T00:52:21.535 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-10T00:52:21.535 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-10T00:52:21.535 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-10T00:52:21.535 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-10T00:52:21.535 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-10T00:52:21.535 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-google-auth-1:2.29.0-2.el9.noarch 122/170 2024-06-10T00:52:21.535 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-10T00:52:21.535 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-10T00:52:21.535 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-10T00:52:21.535 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-10T00:52:21.536 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-10T00:52:21.536 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-10T00:52:21.536 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-10T00:52:21.536 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-10T00:52:21.536 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-10T00:52:21.536 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-10T00:52:21.536 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-10T00:52:21.536 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-10T00:52:21.536 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-10T00:52:21.536 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-10T00:52:21.536 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-10T00:52:21.536 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-10T00:52:21.537 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-10T00:52:21.537 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-10T00:52:21.537 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-10T00:52:21.537 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-10T00:52:21.537 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-10T00:52:21.537 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-10T00:52:21.537 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-10T00:52:21.537 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-10T00:52:21.537 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-10T00:52:21.537 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-10T00:52:21.537 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-10T00:52:21.537 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-10T00:52:21.537 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-10T00:52:21.537 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-10T00:52:21.538 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-10T00:52:21.538 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-10T00:52:21.538 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-10T00:52:21.538 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-10T00:52:21.538 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-10T00:52:21.538 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-10T00:52:21.538 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-10T00:52:21.538 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-10T00:52:21.538 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-10T00:52:21.538 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-10T00:52:21.538 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-10T00:52:21.538 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-10T00:52:21.539 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-10T00:52:21.539 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-10T00:52:21.539 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-10T00:52:21.539 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-10T00:52:21.539 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-10T00:52:21.918 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-10T00:52:21.950 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-10T00:52:22.050 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-10T00:52:22.206 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-10T00:52:22.243 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-10T00:52:22.266 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-10T00:52:22.266 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:52:22.266 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-10T00:52:22.267 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-10T00:52:22.267 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-10T00:52:22.267 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-06-10T00:52:22.311 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-10T00:52:22.322 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-10T00:52:22.420 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-10T00:52:22.467 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-10T00:52:22.709 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-10T00:52:22.751 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-10T00:52:22.792 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-10T00:52:22.824 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-10T00:52:22.851 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-10T00:52:23.051 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-10T00:52:23.097 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-10T00:52:23.122 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-10T00:52:23.153 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-10T00:52:23.183 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-10T00:52:23.229 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-10T00:52:23.364 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-10T00:52:23.412 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-10T00:52:23.435 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-10T00:52:23.435 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:52:23.436 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-10T00:52:23.436 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-10T00:52:23.436 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-10T00:52:23.436 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-06-10T00:52:23.481 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-10T00:52:23.503 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-10T00:52:23.503 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:52:23.503 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-10T00:52:23.503 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-06-10T00:52:23.548 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-10T00:52:23.571 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-10T00:52:23.571 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:52:23.571 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-10T00:52:23.571 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-10T00:52:23.571 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-10T00:52:23.571 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-06-10T00:52:23.787 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T00:52:23.827 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T00:52:23.848 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T00:52:23.882 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-10T00:52:23.905 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-10T00:52:23.905 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:52:23.905 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-10T00:52:23.905 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-10T00:52:23.905 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-10T00:52:23.905 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-06-10T00:52:24.298 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-10T00:52:24.299 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-10T00:52:24.299 INFO:teuthology.orchestra.run.smithi175.stdout:Upgraded: 2024-06-10T00:52:24.299 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-10T00:52:24.299 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-10T00:52:24.299 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-10T00:52:24.299 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-10T00:52:24.299 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-10T00:52:24.299 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-10T00:52:24.299 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-10T00:52:24.299 INFO:teuthology.orchestra.run.smithi175.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:24.299 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:24.300 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-10T00:52:24.300 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-10T00:52:24.300 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-06-10T00:52:24.300 INFO:teuthology.orchestra.run.smithi175.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-10T00:52:24.300 INFO:teuthology.orchestra.run.smithi175.stdout: automake-1.16.2-8.el9.noarch 2024-06-10T00:52:24.300 INFO:teuthology.orchestra.run.smithi175.stdout: bison-3.7.4-5.el9.x86_64 2024-06-10T00:52:24.300 INFO:teuthology.orchestra.run.smithi175.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-10T00:52:24.300 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:24.300 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:24.300 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:24.300 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:24.300 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:52:24.300 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:24.301 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:24.301 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:24.301 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:52:24.301 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:52:24.301 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:52:24.301 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:52:24.301 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:52:24.301 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:24.301 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:24.301 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:52:24.301 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:24.301 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:24.302 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:24.302 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:52:24.302 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:52:24.302 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:24.302 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:24.302 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:52:24.302 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-10T00:52:24.302 INFO:teuthology.orchestra.run.smithi175.stdout: flex-2.6.4-9.el9.x86_64 2024-06-10T00:52:24.303 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-10T00:52:24.303 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-10T00:52:24.303 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-10T00:52:24.303 INFO:teuthology.orchestra.run.smithi175.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-10T00:52:24.303 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-10T00:52:24.303 INFO:teuthology.orchestra.run.smithi175.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-10T00:52:24.304 INFO:teuthology.orchestra.run.smithi175.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-10T00:52:24.304 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-10T00:52:24.304 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-10T00:52:24.304 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:24.304 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:24.304 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:24.304 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-10T00:52:24.305 INFO:teuthology.orchestra.run.smithi175.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-10T00:52:24.305 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-10T00:52:24.305 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-10T00:52:24.305 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-10T00:52:24.305 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:24.305 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:24.305 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-10T00:52:24.305 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:24.305 INFO:teuthology.orchestra.run.smithi175.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-10T00:52:24.305 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-10T00:52:24.305 INFO:teuthology.orchestra.run.smithi175.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-10T00:52:24.305 INFO:teuthology.orchestra.run.smithi175.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-10T00:52:24.305 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-10T00:52:24.305 INFO:teuthology.orchestra.run.smithi175.stdout: lua-5.4.4-4.el9.x86_64 2024-06-10T00:52:24.306 INFO:teuthology.orchestra.run.smithi175.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-10T00:52:24.306 INFO:teuthology.orchestra.run.smithi175.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-10T00:52:24.306 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-10T00:52:24.306 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-10T00:52:24.306 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-10T00:52:24.306 INFO:teuthology.orchestra.run.smithi175.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-10T00:52:24.306 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-13.14-1.el9.x86_64 2024-06-10T00:52:24.307 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-10T00:52:24.307 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-10T00:52:24.307 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-10T00:52:24.307 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-10T00:52:24.307 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-10T00:52:24.307 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-10T00:52:24.307 INFO:teuthology.orchestra.run.smithi175.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-10T00:52:24.307 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-10T00:52:24.307 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-10T00:52:24.307 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-10T00:52:24.308 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:24.308 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:24.308 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:24.308 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-10T00:52:24.308 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-10T00:52:24.308 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-10T00:52:24.308 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-10T00:52:24.308 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-10T00:52:24.308 INFO:teuthology.orchestra.run.smithi175.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-10T00:52:24.308 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-10T00:52:24.308 INFO:teuthology.orchestra.run.smithi175.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-10T00:52:24.308 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth-1:2.29.0-2.el9.noarch 2024-06-10T00:52:24.308 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-10T00:52:24.308 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-10T00:52:24.309 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-10T00:52:24.309 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-10T00:52:24.309 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-10T00:52:24.309 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-10T00:52:24.309 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-10T00:52:24.309 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-10T00:52:24.309 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-10T00:52:24.309 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-10T00:52:24.309 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-10T00:52:24.309 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-10T00:52:24.309 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-10T00:52:24.309 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-10T00:52:24.309 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-10T00:52:24.310 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-10T00:52:24.310 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-10T00:52:24.310 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-10T00:52:24.310 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-10T00:52:24.310 INFO:teuthology.orchestra.run.smithi175.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-10T00:52:24.310 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-10T00:52:24.310 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ply-3.11-14.el9.noarch 2024-06-10T00:52:24.310 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-10T00:52:24.310 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-10T00:52:24.310 INFO:teuthology.orchestra.run.smithi175.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-10T00:52:24.310 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-10T00:52:24.310 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-10T00:52:24.310 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-10T00:52:24.311 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-10T00:52:24.311 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-10T00:52:24.311 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-10T00:52:24.311 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:24.311 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:24.311 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-10T00:52:24.311 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-10T00:52:24.311 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-10T00:52:24.311 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:24.311 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-10T00:52:24.311 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-10T00:52:24.312 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-10T00:52:24.312 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-10T00:52:24.312 INFO:teuthology.orchestra.run.smithi175.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-10T00:52:24.312 INFO:teuthology.orchestra.run.smithi175.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-10T00:52:24.312 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-10T00:52:24.312 INFO:teuthology.orchestra.run.smithi175.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-10T00:52:24.312 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-10T00:52:24.312 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-10T00:52:24.312 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-10T00:52:24.312 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-10T00:52:24.312 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-10T00:52:24.312 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-10T00:52:24.312 INFO:teuthology.orchestra.run.smithi175.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-10T00:52:24.312 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:24.313 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:24.313 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:24.313 INFO:teuthology.orchestra.run.smithi175.stdout: re2-1:20211101-3.el9.x86_64 2024-06-10T00:52:24.313 INFO:teuthology.orchestra.run.smithi175.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-10T00:52:24.313 INFO:teuthology.orchestra.run.smithi175.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-10T00:52:24.313 INFO:teuthology.orchestra.run.smithi175.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-10T00:52:24.313 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-10T00:52:24.313 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-10T00:52:24.314 INFO:teuthology.orchestra.run.smithi175.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-10T00:52:24.314 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-10T00:52:24.314 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-06-10T00:52:24.760 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-10T00:52:24.765 DEBUG:teuthology.parallel:result is None 2024-06-10T00:52:24.858 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-10T00:52:24.900 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-10T00:52:24.990 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-10T00:52:25.019 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-10T00:52:25.052 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-10T00:52:25.078 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-10T00:52:25.104 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-10T00:52:25.105 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-10T00:52:25.181 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-10T00:52:25.214 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-10T00:52:25.214 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-10T00:52:25.299 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-10T00:52:25.323 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-10T00:52:25.329 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T00:52:25.329 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T00:52:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T00:52:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-10T00:52:25.601 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-10T00:52:25.626 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-10T00:52:25.694 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-10T00:52:28.435 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-10T00:52:28.477 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-10T00:52:28.517 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-10T00:52:28.558 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-10T00:52:28.585 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-10T00:52:28.766 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-10T00:52:28.814 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-10T00:52:28.847 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-10T00:52:28.887 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-10T00:52:28.917 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-10T00:52:28.955 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-10T00:52:29.493 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T00:52:29.536 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T00:52:29.556 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T00:52:30.386 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-10T00:52:30.466 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-10T00:52:30.507 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-10T00:52:30.588 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-10T00:52:30.619 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-10T00:52:30.644 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-10T00:52:30.669 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-10T00:52:30.696 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-10T00:52:30.696 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-10T00:52:30.790 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-10T00:52:30.830 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-10T00:52:30.830 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-10T00:52:30.923 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-10T00:52:30.948 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-10T00:52:30.953 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T00:52:30.953 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T00:52:31.133 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T00:52:31.133 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-10T00:52:31.201 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-10T00:52:31.225 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-10T00:52:31.298 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-10T00:52:31.396 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-10T00:52:31.396 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-10T00:52:31.396 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-10T00:52:31.396 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-10T00:52:31.396 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-10T00:52:31.397 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-10T00:52:31.397 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-10T00:52:31.397 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-10T00:52:31.397 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T00:52:31.397 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T00:52:31.397 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-10T00:52:31.397 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-10T00:52:31.397 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-10T00:52:31.397 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-10T00:52:31.397 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T00:52:31.397 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-10T00:52:31.397 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-10T00:52:31.398 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-10T00:52:31.398 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-10T00:52:31.398 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-10T00:52:31.398 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-10T00:52:31.398 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-10T00:52:31.398 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-10T00:52:31.398 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-10T00:52:31.398 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-10T00:52:31.398 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-10T00:52:31.399 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-10T00:52:31.399 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-10T00:52:31.399 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-10T00:52:31.399 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-10T00:52:31.399 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-10T00:52:31.399 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-10T00:52:31.399 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-10T00:52:31.399 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-10T00:52:31.399 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-10T00:52:31.399 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-10T00:52:31.399 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-10T00:52:31.399 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-10T00:52:31.399 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-10T00:52:31.399 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-10T00:52:31.400 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-10T00:52:31.400 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-10T00:52:31.400 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-10T00:52:31.400 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-10T00:52:31.400 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-10T00:52:31.400 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-10T00:52:31.400 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-10T00:52:31.400 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-10T00:52:31.400 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-10T00:52:31.400 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-10T00:52:31.400 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-10T00:52:31.400 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-10T00:52:31.400 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-10T00:52:31.401 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-10T00:52:31.401 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-10T00:52:31.401 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-10T00:52:31.401 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-10T00:52:31.401 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-10T00:52:31.401 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-10T00:52:31.401 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-10T00:52:31.401 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-10T00:52:31.401 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-10T00:52:31.401 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-10T00:52:31.401 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-10T00:52:31.401 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-10T00:52:31.401 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-10T00:52:31.402 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-10T00:52:31.402 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-10T00:52:31.402 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-10T00:52:31.402 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-10T00:52:31.402 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-10T00:52:31.402 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-10T00:52:31.402 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-10T00:52:31.402 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-10T00:52:31.402 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-10T00:52:31.402 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-10T00:52:31.403 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-10T00:52:31.403 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-10T00:52:31.403 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-10T00:52:31.403 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-10T00:52:31.403 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-10T00:52:31.403 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-10T00:52:31.403 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-10T00:52:31.403 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-10T00:52:31.403 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-10T00:52:31.403 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-10T00:52:31.403 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-10T00:52:31.403 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-10T00:52:31.403 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-10T00:52:31.404 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-10T00:52:31.404 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-10T00:52:31.404 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-10T00:52:31.404 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-10T00:52:31.404 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-10T00:52:31.404 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-10T00:52:31.404 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-10T00:52:31.404 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-10T00:52:31.404 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-10T00:52:31.404 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-10T00:52:31.404 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-10T00:52:31.404 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-10T00:52:31.404 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-10T00:52:31.404 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-10T00:52:31.405 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-10T00:52:31.405 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-10T00:52:31.405 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-10T00:52:31.405 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-10T00:52:31.405 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-10T00:52:31.405 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-10T00:52:31.405 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-10T00:52:31.405 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-10T00:52:31.405 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-10T00:52:31.405 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-10T00:52:31.405 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-10T00:52:31.405 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-10T00:52:31.405 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-10T00:52:31.406 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-10T00:52:31.406 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-10T00:52:31.406 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-10T00:52:31.406 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-10T00:52:31.406 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-10T00:52:31.406 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-10T00:52:31.406 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-google-auth-1:2.29.0-2.el9.noarch 122/170 2024-06-10T00:52:31.406 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-10T00:52:31.406 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-10T00:52:31.406 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-10T00:52:31.406 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-10T00:52:31.406 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-10T00:52:31.406 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-10T00:52:31.407 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-10T00:52:31.407 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-10T00:52:31.407 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-10T00:52:31.407 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-10T00:52:31.407 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-10T00:52:31.407 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-10T00:52:31.407 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-10T00:52:31.407 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-10T00:52:31.407 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-10T00:52:31.407 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-10T00:52:31.407 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-10T00:52:31.407 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-10T00:52:31.407 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-10T00:52:31.408 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-10T00:52:31.408 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-10T00:52:31.408 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-10T00:52:31.408 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-10T00:52:31.408 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-10T00:52:31.408 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-10T00:52:31.408 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-10T00:52:31.408 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-10T00:52:31.408 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-10T00:52:31.408 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-10T00:52:31.408 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-10T00:52:31.408 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-10T00:52:31.408 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-10T00:52:31.408 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-10T00:52:31.409 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-10T00:52:31.409 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-10T00:52:31.409 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-10T00:52:31.409 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-10T00:52:31.409 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-10T00:52:31.409 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-10T00:52:31.409 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-10T00:52:31.409 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-10T00:52:31.409 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-10T00:52:31.409 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-10T00:52:31.409 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-10T00:52:31.409 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-10T00:52:31.409 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-10T00:52:31.409 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-10T00:52:34.003 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-10T00:52:34.004 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:52:34.004 INFO:teuthology.orchestra.run.smithi007.stdout:Upgraded: 2024-06-10T00:52:34.004 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-10T00:52:34.004 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-10T00:52:34.004 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-10T00:52:34.004 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-10T00:52:34.004 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-10T00:52:34.004 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-10T00:52:34.004 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-10T00:52:34.004 INFO:teuthology.orchestra.run.smithi007.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:34.004 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:34.004 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-10T00:52:34.004 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-10T00:52:34.005 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-06-10T00:52:34.005 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-10T00:52:34.005 INFO:teuthology.orchestra.run.smithi007.stdout: automake-1.16.2-8.el9.noarch 2024-06-10T00:52:34.005 INFO:teuthology.orchestra.run.smithi007.stdout: bison-3.7.4-5.el9.x86_64 2024-06-10T00:52:34.005 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-10T00:52:34.005 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:34.005 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:34.005 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:34.005 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:34.005 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:52:34.005 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:34.005 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:34.005 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:34.006 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:52:34.006 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:52:34.006 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:52:34.006 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:52:34.006 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:52:34.006 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:34.006 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:34.006 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:52:34.006 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:34.006 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:34.007 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:34.007 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:52:34.007 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:52:34.007 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:34.007 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:34.007 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:52:34.007 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-10T00:52:34.007 INFO:teuthology.orchestra.run.smithi007.stdout: flex-2.6.4-9.el9.x86_64 2024-06-10T00:52:34.007 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-10T00:52:34.007 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-10T00:52:34.007 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-10T00:52:34.007 INFO:teuthology.orchestra.run.smithi007.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-10T00:52:34.007 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-10T00:52:34.008 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-10T00:52:34.008 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-10T00:52:34.008 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-10T00:52:34.008 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-10T00:52:34.008 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:34.008 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:34.008 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:34.008 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-10T00:52:34.008 INFO:teuthology.orchestra.run.smithi007.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-10T00:52:34.008 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-10T00:52:34.008 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-10T00:52:34.008 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-10T00:52:34.008 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:34.009 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:34.009 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-10T00:52:34.009 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:34.009 INFO:teuthology.orchestra.run.smithi007.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-10T00:52:34.009 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-10T00:52:34.009 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-10T00:52:34.009 INFO:teuthology.orchestra.run.smithi007.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-10T00:52:34.009 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-10T00:52:34.009 INFO:teuthology.orchestra.run.smithi007.stdout: lua-5.4.4-4.el9.x86_64 2024-06-10T00:52:34.009 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-10T00:52:34.009 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-10T00:52:34.009 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-10T00:52:34.009 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-10T00:52:34.010 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-10T00:52:34.010 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-10T00:52:34.010 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-13.14-1.el9.x86_64 2024-06-10T00:52:34.010 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-10T00:52:34.010 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-10T00:52:34.010 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-10T00:52:34.010 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-10T00:52:34.010 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-10T00:52:34.010 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-10T00:52:34.010 INFO:teuthology.orchestra.run.smithi007.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-10T00:52:34.011 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-10T00:52:34.011 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-10T00:52:34.011 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-10T00:52:34.011 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:34.011 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:34.011 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:34.011 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-10T00:52:34.011 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-10T00:52:34.011 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-10T00:52:34.011 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-10T00:52:34.011 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-10T00:52:34.011 INFO:teuthology.orchestra.run.smithi007.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-10T00:52:34.011 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-10T00:52:34.012 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-10T00:52:34.012 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth-1:2.29.0-2.el9.noarch 2024-06-10T00:52:34.012 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-10T00:52:34.012 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-10T00:52:34.012 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-10T00:52:34.012 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-10T00:52:34.012 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-10T00:52:34.012 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-10T00:52:34.012 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-10T00:52:34.012 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-10T00:52:34.012 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-10T00:52:34.012 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-10T00:52:34.012 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-10T00:52:34.012 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-10T00:52:34.013 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-10T00:52:34.013 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-10T00:52:34.013 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-10T00:52:34.013 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-10T00:52:34.013 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-10T00:52:34.013 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-10T00:52:34.013 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-10T00:52:34.013 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-10T00:52:34.013 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-10T00:52:34.013 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply-3.11-14.el9.noarch 2024-06-10T00:52:34.013 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-10T00:52:34.013 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-10T00:52:34.013 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-10T00:52:34.013 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-10T00:52:34.014 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-10T00:52:34.014 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-10T00:52:34.014 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-10T00:52:34.014 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-10T00:52:34.014 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-10T00:52:34.014 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:34.014 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:34.014 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-10T00:52:34.014 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-10T00:52:34.014 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-10T00:52:34.014 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:34.014 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-10T00:52:34.014 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-10T00:52:34.014 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-10T00:52:34.015 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-10T00:52:34.015 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-10T00:52:34.015 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-10T00:52:34.015 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-10T00:52:34.015 INFO:teuthology.orchestra.run.smithi007.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-10T00:52:34.015 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-10T00:52:34.015 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-10T00:52:34.015 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-10T00:52:34.015 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-10T00:52:34.015 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-10T00:52:34.015 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-10T00:52:34.015 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-10T00:52:34.015 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:34.015 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:34.016 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:34.016 INFO:teuthology.orchestra.run.smithi007.stdout: re2-1:20211101-3.el9.x86_64 2024-06-10T00:52:34.016 INFO:teuthology.orchestra.run.smithi007.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-10T00:52:34.016 INFO:teuthology.orchestra.run.smithi007.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-10T00:52:34.016 INFO:teuthology.orchestra.run.smithi007.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-10T00:52:34.016 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-10T00:52:34.016 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-10T00:52:34.016 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-10T00:52:34.016 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:52:34.017 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-06-10T00:52:34.397 DEBUG:teuthology.parallel:result is None 2024-06-10T00:52:36.850 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-10T00:52:36.856 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-10T00:52:36.856 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-10T00:52:36.856 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-10T00:52:36.856 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-10T00:52:36.857 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-10T00:52:36.857 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-10T00:52:36.857 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-10T00:52:36.857 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T00:52:36.857 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T00:52:36.857 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-10T00:52:36.857 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-10T00:52:36.857 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-10T00:52:36.857 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-10T00:52:36.857 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T00:52:36.857 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-10T00:52:36.857 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-10T00:52:36.857 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-10T00:52:36.858 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-10T00:52:36.858 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-10T00:52:36.858 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-10T00:52:36.858 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-10T00:52:36.858 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-10T00:52:36.858 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-10T00:52:36.858 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-10T00:52:36.858 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-10T00:52:36.858 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-10T00:52:36.858 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-10T00:52:36.858 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-10T00:52:36.858 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-10T00:52:36.858 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-10T00:52:36.859 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-10T00:52:36.859 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-10T00:52:36.859 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-10T00:52:36.859 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-10T00:52:36.859 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-10T00:52:36.859 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-10T00:52:36.859 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-10T00:52:36.859 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-10T00:52:36.859 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-10T00:52:36.859 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-10T00:52:36.859 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-10T00:52:36.859 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-10T00:52:36.859 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-10T00:52:36.859 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-10T00:52:36.860 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-10T00:52:36.860 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-10T00:52:36.860 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-10T00:52:36.860 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-10T00:52:36.860 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-10T00:52:36.860 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-10T00:52:36.860 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-10T00:52:36.860 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-10T00:52:36.860 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-10T00:52:36.860 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-10T00:52:36.860 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-10T00:52:36.860 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-10T00:52:36.860 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-10T00:52:36.860 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-10T00:52:36.861 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-10T00:52:36.861 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-10T00:52:36.861 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-10T00:52:36.861 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-10T00:52:36.861 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-10T00:52:36.861 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-10T00:52:36.861 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-10T00:52:36.861 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-10T00:52:36.861 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-10T00:52:36.861 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-10T00:52:36.861 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-10T00:52:36.861 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-10T00:52:36.861 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-10T00:52:36.862 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-10T00:52:36.862 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-10T00:52:36.862 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-10T00:52:36.862 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-10T00:52:36.862 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-10T00:52:36.862 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-10T00:52:36.862 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-10T00:52:36.862 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-10T00:52:36.862 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-10T00:52:36.862 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-10T00:52:36.862 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-10T00:52:36.862 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-10T00:52:36.862 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-10T00:52:36.862 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-10T00:52:36.863 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-10T00:52:36.863 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-10T00:52:36.863 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-10T00:52:36.863 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-10T00:52:36.863 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-10T00:52:36.863 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-10T00:52:36.863 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-10T00:52:36.863 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-10T00:52:36.863 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-10T00:52:36.863 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-10T00:52:36.863 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-10T00:52:36.863 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-10T00:52:36.863 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-10T00:52:36.863 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-10T00:52:36.864 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-10T00:52:36.864 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-10T00:52:36.864 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-10T00:52:36.864 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-10T00:52:36.864 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-10T00:52:36.864 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-10T00:52:36.864 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-10T00:52:36.864 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-10T00:52:36.864 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-10T00:52:36.864 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-10T00:52:36.864 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-10T00:52:36.864 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-10T00:52:36.864 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-10T00:52:36.865 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-10T00:52:36.865 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-10T00:52:36.865 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-10T00:52:36.865 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-10T00:52:36.865 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-10T00:52:36.865 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-10T00:52:36.865 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-10T00:52:36.865 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-10T00:52:36.865 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-10T00:52:36.865 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-google-auth-1:2.29.0-2.el9.noarch 122/170 2024-06-10T00:52:36.865 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-10T00:52:36.865 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-10T00:52:36.865 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-10T00:52:36.865 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-10T00:52:36.866 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-10T00:52:36.866 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-10T00:52:36.866 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-10T00:52:36.866 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-10T00:52:36.866 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-10T00:52:36.866 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-10T00:52:36.866 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-10T00:52:36.866 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-10T00:52:36.866 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-10T00:52:36.866 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-10T00:52:36.866 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-10T00:52:36.866 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-10T00:52:36.866 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-10T00:52:36.866 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-10T00:52:36.867 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-10T00:52:36.867 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-10T00:52:36.867 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-10T00:52:36.867 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-10T00:52:36.867 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-10T00:52:36.867 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-10T00:52:36.867 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-10T00:52:36.867 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-10T00:52:36.867 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-10T00:52:36.867 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-10T00:52:36.867 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-10T00:52:36.867 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-10T00:52:36.867 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-10T00:52:36.867 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-10T00:52:36.868 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-10T00:52:36.868 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-10T00:52:36.868 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-10T00:52:36.868 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-10T00:52:36.868 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-10T00:52:36.868 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-10T00:52:36.868 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-10T00:52:36.868 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-10T00:52:36.868 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-10T00:52:36.868 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-10T00:52:36.868 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-10T00:52:36.868 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-10T00:52:36.868 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-10T00:52:36.868 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-10T00:52:36.869 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-10T00:52:39.733 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-10T00:52:39.734 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-06-10T00:52:39.734 INFO:teuthology.orchestra.run.smithi078.stdout:Upgraded: 2024-06-10T00:52:39.734 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-10T00:52:39.734 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-10T00:52:39.734 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-10T00:52:39.734 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-10T00:52:39.735 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-10T00:52:39.735 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-10T00:52:39.735 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-10T00:52:39.735 INFO:teuthology.orchestra.run.smithi078.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:39.735 INFO:teuthology.orchestra.run.smithi078.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:39.735 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-10T00:52:39.735 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-10T00:52:39.735 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-06-10T00:52:39.735 INFO:teuthology.orchestra.run.smithi078.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-10T00:52:39.735 INFO:teuthology.orchestra.run.smithi078.stdout: automake-1.16.2-8.el9.noarch 2024-06-10T00:52:39.735 INFO:teuthology.orchestra.run.smithi078.stdout: bison-3.7.4-5.el9.x86_64 2024-06-10T00:52:39.736 INFO:teuthology.orchestra.run.smithi078.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-10T00:52:39.736 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:39.736 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:39.736 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:39.736 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:39.736 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:52:39.736 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:39.736 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:39.736 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:39.736 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:52:39.736 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:52:39.736 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:52:39.737 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:52:39.737 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:52:39.737 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:39.737 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:39.737 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:52:39.737 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:39.737 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:39.737 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:39.737 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:52:39.737 INFO:teuthology.orchestra.run.smithi078.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:52:39.737 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:39.737 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:39.738 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:52:39.738 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-10T00:52:39.738 INFO:teuthology.orchestra.run.smithi078.stdout: flex-2.6.4-9.el9.x86_64 2024-06-10T00:52:39.738 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-10T00:52:39.738 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-10T00:52:39.738 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-10T00:52:39.738 INFO:teuthology.orchestra.run.smithi078.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-10T00:52:39.738 INFO:teuthology.orchestra.run.smithi078.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-10T00:52:39.738 INFO:teuthology.orchestra.run.smithi078.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-10T00:52:39.738 INFO:teuthology.orchestra.run.smithi078.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-10T00:52:39.738 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-10T00:52:39.738 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-10T00:52:39.738 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:39.739 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:39.739 INFO:teuthology.orchestra.run.smithi078.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:39.739 INFO:teuthology.orchestra.run.smithi078.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-10T00:52:39.739 INFO:teuthology.orchestra.run.smithi078.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-10T00:52:39.739 INFO:teuthology.orchestra.run.smithi078.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-10T00:52:39.739 INFO:teuthology.orchestra.run.smithi078.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-10T00:52:39.739 INFO:teuthology.orchestra.run.smithi078.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-10T00:52:39.739 INFO:teuthology.orchestra.run.smithi078.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:39.739 INFO:teuthology.orchestra.run.smithi078.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:39.740 INFO:teuthology.orchestra.run.smithi078.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-10T00:52:39.740 INFO:teuthology.orchestra.run.smithi078.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:39.740 INFO:teuthology.orchestra.run.smithi078.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-10T00:52:39.740 INFO:teuthology.orchestra.run.smithi078.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-10T00:52:39.740 INFO:teuthology.orchestra.run.smithi078.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-10T00:52:39.740 INFO:teuthology.orchestra.run.smithi078.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-10T00:52:39.740 INFO:teuthology.orchestra.run.smithi078.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-10T00:52:39.740 INFO:teuthology.orchestra.run.smithi078.stdout: lua-5.4.4-4.el9.x86_64 2024-06-10T00:52:39.740 INFO:teuthology.orchestra.run.smithi078.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-10T00:52:39.740 INFO:teuthology.orchestra.run.smithi078.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-10T00:52:39.740 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-10T00:52:39.740 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-10T00:52:39.740 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-10T00:52:39.740 INFO:teuthology.orchestra.run.smithi078.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-10T00:52:39.741 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-13.14-1.el9.x86_64 2024-06-10T00:52:39.741 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-10T00:52:39.741 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-10T00:52:39.741 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-10T00:52:39.741 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-10T00:52:39.741 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-10T00:52:39.741 INFO:teuthology.orchestra.run.smithi078.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-10T00:52:39.741 INFO:teuthology.orchestra.run.smithi078.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-10T00:52:39.741 INFO:teuthology.orchestra.run.smithi078.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-10T00:52:39.742 INFO:teuthology.orchestra.run.smithi078.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-10T00:52:39.742 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-10T00:52:39.742 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:39.742 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:39.742 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:39.742 INFO:teuthology.orchestra.run.smithi078.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-10T00:52:39.742 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-10T00:52:39.742 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-10T00:52:39.742 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-10T00:52:39.742 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-10T00:52:39.743 INFO:teuthology.orchestra.run.smithi078.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-10T00:52:39.743 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-10T00:52:39.743 INFO:teuthology.orchestra.run.smithi078.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-10T00:52:39.743 INFO:teuthology.orchestra.run.smithi078.stdout: python3-google-auth-1:2.29.0-2.el9.noarch 2024-06-10T00:52:39.743 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-10T00:52:39.743 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-10T00:52:39.743 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-10T00:52:39.743 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-10T00:52:39.743 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-10T00:52:39.743 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-10T00:52:39.743 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-10T00:52:39.743 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-10T00:52:39.743 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-10T00:52:39.743 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-10T00:52:39.744 INFO:teuthology.orchestra.run.smithi078.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-10T00:52:39.744 INFO:teuthology.orchestra.run.smithi078.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-10T00:52:39.744 INFO:teuthology.orchestra.run.smithi078.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-10T00:52:39.744 INFO:teuthology.orchestra.run.smithi078.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-10T00:52:39.744 INFO:teuthology.orchestra.run.smithi078.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-10T00:52:39.744 INFO:teuthology.orchestra.run.smithi078.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-10T00:52:39.744 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-10T00:52:39.744 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-10T00:52:39.744 INFO:teuthology.orchestra.run.smithi078.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-10T00:52:39.744 INFO:teuthology.orchestra.run.smithi078.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-10T00:52:39.744 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-10T00:52:39.744 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ply-3.11-14.el9.noarch 2024-06-10T00:52:39.744 INFO:teuthology.orchestra.run.smithi078.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-10T00:52:39.744 INFO:teuthology.orchestra.run.smithi078.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-10T00:52:39.745 INFO:teuthology.orchestra.run.smithi078.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-10T00:52:39.745 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-10T00:52:39.745 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-10T00:52:39.745 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-10T00:52:39.745 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-10T00:52:39.745 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-10T00:52:39.745 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-10T00:52:39.745 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:39.745 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:39.745 INFO:teuthology.orchestra.run.smithi078.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-10T00:52:39.745 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-10T00:52:39.745 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-10T00:52:39.745 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:39.745 INFO:teuthology.orchestra.run.smithi078.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-10T00:52:39.746 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-10T00:52:39.746 INFO:teuthology.orchestra.run.smithi078.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-10T00:52:39.746 INFO:teuthology.orchestra.run.smithi078.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-10T00:52:39.746 INFO:teuthology.orchestra.run.smithi078.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-10T00:52:39.746 INFO:teuthology.orchestra.run.smithi078.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-10T00:52:39.746 INFO:teuthology.orchestra.run.smithi078.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-10T00:52:39.746 INFO:teuthology.orchestra.run.smithi078.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-10T00:52:39.746 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-10T00:52:39.746 INFO:teuthology.orchestra.run.smithi078.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-10T00:52:39.746 INFO:teuthology.orchestra.run.smithi078.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-10T00:52:39.746 INFO:teuthology.orchestra.run.smithi078.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-10T00:52:39.746 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-10T00:52:39.746 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-10T00:52:39.746 INFO:teuthology.orchestra.run.smithi078.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-10T00:52:39.747 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:39.747 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:39.747 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:52:39.747 INFO:teuthology.orchestra.run.smithi078.stdout: re2-1:20211101-3.el9.x86_64 2024-06-10T00:52:39.747 INFO:teuthology.orchestra.run.smithi078.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-10T00:52:39.747 INFO:teuthology.orchestra.run.smithi078.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-10T00:52:39.747 INFO:teuthology.orchestra.run.smithi078.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-10T00:52:39.747 INFO:teuthology.orchestra.run.smithi078.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-10T00:52:39.747 INFO:teuthology.orchestra.run.smithi078.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-10T00:52:39.747 INFO:teuthology.orchestra.run.smithi078.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-10T00:52:39.748 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-06-10T00:52:39.748 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-06-10T00:52:40.193 DEBUG:teuthology.parallel:result is None 2024-06-10T00:52:40.194 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:52:40.416 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-10T00:52:40.443 INFO:teuthology.orchestra.run.smithi007.stdout:19.0.0-2946.g3d89accf.el9 2024-06-10T00:52:40.443 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-10T00:52:40.443 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-10T00:52:40.445 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:52:40.661 DEBUG:teuthology.orchestra.run.smithi078:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-10T00:52:40.682 INFO:teuthology.orchestra.run.smithi078.stdout:19.0.0-2946.g3d89accf.el9 2024-06-10T00:52:40.683 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-10T00:52:40.683 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-10T00:52:40.684 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:52:40.823 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-10T00:52:40.848 INFO:teuthology.orchestra.run.smithi175.stdout:19.0.0-2946.g3d89accf.el9 2024-06-10T00:52:40.848 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-10T00:52:40.848 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-10T00:52:40.850 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-06-10T00:52:40.850 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-10T00:52:40.850 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-10T00:52:40.884 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-06-10T00:52:40.884 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-10T00:52:40.911 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-06-10T00:52:40.911 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-10T00:52:40.940 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-06-10T00:52:40.941 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-10T00:52:40.941 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/daemon-helper 2024-06-10T00:52:40.970 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-10T00:52:41.040 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-06-10T00:52:41.040 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/daemon-helper 2024-06-10T00:52:41.065 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-10T00:52:41.129 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-06-10T00:52:41.129 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/daemon-helper 2024-06-10T00:52:41.157 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-10T00:52:41.226 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-06-10T00:52:41.226 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-10T00:52:41.226 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-10T00:52:41.257 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-10T00:52:41.327 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-06-10T00:52:41.327 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-10T00:52:41.351 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-10T00:52:41.414 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-06-10T00:52:41.414 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-10T00:52:41.438 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-10T00:52:41.502 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-06-10T00:52:41.503 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-10T00:52:41.503 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/stdin-killer 2024-06-10T00:52:41.532 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-10T00:52:41.602 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-06-10T00:52:41.602 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/stdin-killer 2024-06-10T00:52:41.625 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-10T00:52:41.688 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-06-10T00:52:41.688 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/stdin-killer 2024-06-10T00:52:41.712 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-10T00:52:41.776 INFO:teuthology.run_tasks:Running task cephadm... 2024-06-10T00:52:41.869 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-06-10T00:52:41.869 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:52:41.870 INFO:tasks.cephadm:Cluster fsid is bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T00:52:41.870 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-06-10T00:52:41.870 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.7', 'mon.b': '172.21.15.78', 'mon.c': '172.21.15.175'} 2024-06-10T00:52:41.870 INFO:tasks.cephadm:First mon is mon.a on smithi007 2024-06-10T00:52:41.870 INFO:tasks.cephadm:First mgr is x 2024-06-10T00:52:41.870 INFO:tasks.cephadm:Normalizing hostnames... 2024-06-10T00:52:41.871 DEBUG:teuthology.orchestra.run.smithi007:> sudo hostname $(hostname -s) 2024-06-10T00:52:41.902 DEBUG:teuthology.orchestra.run.smithi078:> sudo hostname $(hostname -s) 2024-06-10T00:52:41.926 DEBUG:teuthology.orchestra.run.smithi175:> sudo hostname $(hostname -s) 2024-06-10T00:52:41.953 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-06-10T00:52:41.953 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:52:42.093 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55273/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2946-g3d89accf', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2946.g3d89accf'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-30 17:04:17.570799', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-06-10T00:52:42.228 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 3d89accfe3201286f000e065a1faea2a88c7a6ba from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:52:42.230 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm 2024-06-10T00:52:42.230 DEBUG:teuthology.orchestra.run.smithi007:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-10T00:52:42.769 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 10 00:52 /home/ubuntu/cephtest/cephadm 2024-06-10T00:52:42.769 DEBUG:teuthology.orchestra.run.smithi078:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-10T00:52:43.294 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 10 00:52 /home/ubuntu/cephtest/cephadm 2024-06-10T00:52:43.294 DEBUG:teuthology.orchestra.run.smithi175:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-10T00:52:43.713 INFO:teuthology.orchestra.run.smithi175.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 10 00:52 /home/ubuntu/cephtest/cephadm 2024-06-10T00:52:43.714 DEBUG:teuthology.orchestra.run.smithi007:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-10T00:52:43.734 DEBUG:teuthology.orchestra.run.smithi078:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-10T00:52:43.753 DEBUG:teuthology.orchestra.run.smithi175:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-10T00:52:43.786 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba on all hosts... 2024-06-10T00:52:43.787 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-10T00:52:43.789 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-10T00:52:43.795 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-10T00:52:44.596 INFO:teuthology.orchestra.run.smithi078.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-10T00:52:44.730 INFO:teuthology.orchestra.run.smithi007.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-10T00:52:44.738 INFO:teuthology.orchestra.run.smithi175.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-10T00:53:12.777 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T00:53:12.777 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-10T00:53:12.778 INFO:teuthology.orchestra.run.smithi007.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-10T00:53:12.778 INFO:teuthology.orchestra.run.smithi007.stdout: "repo_digests": [ 2024-06-10T00:53:12.778 INFO:teuthology.orchestra.run.smithi007.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-10T00:53:12.778 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-06-10T00:53:12.778 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T00:53:17.865 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-06-10T00:53:17.865 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-10T00:53:17.865 INFO:teuthology.orchestra.run.smithi175.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-10T00:53:17.865 INFO:teuthology.orchestra.run.smithi175.stdout: "repo_digests": [ 2024-06-10T00:53:17.866 INFO:teuthology.orchestra.run.smithi175.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-10T00:53:17.866 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-06-10T00:53:17.866 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-06-10T00:53:19.621 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-06-10T00:53:19.622 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-10T00:53:19.622 INFO:teuthology.orchestra.run.smithi078.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-10T00:53:19.622 INFO:teuthology.orchestra.run.smithi078.stdout: "repo_digests": [ 2024-06-10T00:53:19.622 INFO:teuthology.orchestra.run.smithi078.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-10T00:53:19.622 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-06-10T00:53:19.622 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-06-10T00:53:19.646 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph 2024-06-10T00:53:19.680 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /etc/ceph 2024-06-10T00:53:19.706 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /etc/ceph 2024-06-10T00:53:19.737 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 777 /etc/ceph 2024-06-10T00:53:19.768 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 777 /etc/ceph 2024-06-10T00:53:19.791 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 777 /etc/ceph 2024-06-10T00:53:19.818 INFO:tasks.cephadm:Writing seed config... 2024-06-10T00:53:19.819 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-06-10T00:53:19.819 INFO:tasks.cephadm: override: [client] debug client = 20 2024-06-10T00:53:19.819 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-06-10T00:53:19.819 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-06-10T00:53:19.819 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-06-10T00:53:19.820 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-06-10T00:53:19.820 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-06-10T00:53:19.820 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-06-10T00:53:19.820 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-06-10T00:53:19.820 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-06-10T00:53:19.820 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-06-10T00:53:19.820 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-06-10T00:53:19.820 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-06-10T00:53:19.820 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-06-10T00:53:19.820 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-06-10T00:53:19.820 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-06-10T00:53:19.820 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-06-10T00:53:19.820 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-06-10T00:53:19.820 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-06-10T00:53:19.821 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-06-10T00:53:19.821 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-06-10T00:53:19.821 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-06-10T00:53:19.821 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-06-10T00:53:19.821 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-06-10T00:53:19.821 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-06-10T00:53:19.821 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-06-10T00:53:19.821 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-06-10T00:53:19.821 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-06-10T00:53:19.821 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-06-10T00:53:19.821 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-06-10T00:53:19.821 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-06-10T00:53:19.821 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-06-10T00:53:19.821 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-06-10T00:53:19.821 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-06-10T00:53:19.822 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-06-10T00:53:19.822 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-06-10T00:53:19.822 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-06-10T00:53:19.822 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-06-10T00:53:19.822 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-06-10T00:53:19.822 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-06-10T00:53:19.822 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-06-10T00:53:19.822 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-06-10T00:53:19.822 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-06-10T00:53:19.822 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-06-10T00:53:19.822 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-06-10T00:53:19.823 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-10T00:53:19.823 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-06-10T00:53:19.843 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = bded58f8-26c3-11ef-bc9b-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-06-10T00:53:19.843 DEBUG:teuthology.orchestra.run.smithi007:mon.a> sudo journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mon.a.service 2024-06-10T00:53:19.885 DEBUG:teuthology.orchestra.run.smithi007:mgr.x> sudo journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mgr.x.service 2024-06-10T00:53:19.928 INFO:tasks.cephadm:Bootstrapping... 2024-06-10T00:53:19.928 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba -v bootstrap --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.7 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-06-10T00:53:20.134 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-06-10T00:53:20.134 INFO:teuthology.orchestra.run.smithi007.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba', '-v', 'bootstrap', '--fsid', 'bded58f8-26c3-11ef-bc9b-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.7', '--skip-admin-label'] 2024-06-10T00:53:20.135 INFO:teuthology.orchestra.run.smithi007.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-06-10T00:53:20.135 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying podman|docker is present... 2024-06-10T00:53:20.161 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout 5.0.2 2024-06-10T00:53:20.161 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying lvm2 is present... 2024-06-10T00:53:20.161 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying time synchronization is in place... 2024-06-10T00:53:20.169 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-10T00:53:20.169 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-10T00:53:20.175 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-10T00:53:20.175 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-06-10T00:53:20.182 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout enabled 2024-06-10T00:53:20.189 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout active 2024-06-10T00:53:20.189 INFO:teuthology.orchestra.run.smithi007.stdout:Unit chronyd.service is enabled and running 2024-06-10T00:53:20.189 INFO:teuthology.orchestra.run.smithi007.stdout:Repeating the final host check... 2024-06-10T00:53:20.214 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout 5.0.2 2024-06-10T00:53:20.214 INFO:teuthology.orchestra.run.smithi007.stdout:podman (/bin/podman) version 5.0.2 is present 2024-06-10T00:53:20.214 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl is present 2024-06-10T00:53:20.214 INFO:teuthology.orchestra.run.smithi007.stdout:lvcreate is present 2024-06-10T00:53:20.221 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-10T00:53:20.221 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-10T00:53:20.227 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-10T00:53:20.227 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-06-10T00:53:20.234 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout enabled 2024-06-10T00:53:20.241 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout active 2024-06-10T00:53:20.241 INFO:teuthology.orchestra.run.smithi007.stdout:Unit chronyd.service is enabled and running 2024-06-10T00:53:20.241 INFO:teuthology.orchestra.run.smithi007.stdout:Host looks OK 2024-06-10T00:53:20.241 INFO:teuthology.orchestra.run.smithi007.stdout:Cluster fsid: bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T00:53:20.241 INFO:teuthology.orchestra.run.smithi007.stdout:Acquiring lock 140421382370688 on /run/cephadm/bded58f8-26c3-11ef-bc9b-c7b262605968.lock 2024-06-10T00:53:20.242 INFO:teuthology.orchestra.run.smithi007.stdout:Lock 140421382370688 acquired on /run/cephadm/bded58f8-26c3-11ef-bc9b-c7b262605968.lock 2024-06-10T00:53:20.242 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying IP 172.21.15.7 port 3300 ... 2024-06-10T00:53:20.242 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying IP 172.21.15.7 port 6789 ... 2024-06-10T00:53:20.242 INFO:teuthology.orchestra.run.smithi007.stdout:Base mon IP(s) is [172.21.15.7:3300, 172.21.15.7:6789], mon addrv is [v2:172.21.15.7:3300,v1:172.21.15.7:6789] 2024-06-10T00:53:20.246 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.7 metric 100 2024-06-10T00:53:20.246 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.7 metric 100 2024-06-10T00:53:20.248 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-06-10T00:53:20.248 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-06-10T00:53:20.249 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-06-10T00:53:20.251 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-06-10T00:53:20.251 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-06-10T00:53:20.251 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-10T00:53:20.251 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-06-10T00:53:20.251 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15ea/64 scope link noprefixroute 2024-06-10T00:53:20.251 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-10T00:53:20.252 INFO:teuthology.orchestra.run.smithi007.stdout:Mon IP `172.21.15.7` is in CIDR network `172.21.0.0/20` 2024-06-10T00:53:20.252 INFO:teuthology.orchestra.run.smithi007.stdout:Mon IP `172.21.15.7` is in CIDR network `172.21.0.0/20` 2024-06-10T00:53:20.252 INFO:teuthology.orchestra.run.smithi007.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-06-10T00:53:20.253 INFO:teuthology.orchestra.run.smithi007.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-06-10T00:53:20.254 INFO:teuthology.orchestra.run.smithi007.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-10T00:53:20.737 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 2024-06-10T00:53:20.738 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-10T00:53:20.738 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Getting image source signatures 2024-06-10T00:53:20.738 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Copying blob sha256:53b2d4191269e9d20972bfa652565296fad6514cf6a3d99f19db644875f67876 2024-06-10T00:53:20.738 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Copying blob sha256:3ca969c7b6d79946c6f10490fffc00028babe7df67f50ed53c7bc209325e0612 2024-06-10T00:53:20.738 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Copying config sha256:b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 2024-06-10T00:53:20.738 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Writing manifest to image destination 2024-06-10T00:53:21.718 INFO:teuthology.orchestra.run.smithi007.stdout:ceph: stdout ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-10T00:53:21.718 INFO:teuthology.orchestra.run.smithi007.stdout:Ceph version: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-10T00:53:21.718 INFO:teuthology.orchestra.run.smithi007.stdout:Extracting ceph user uid/gid from container image... 2024-06-10T00:53:22.665 INFO:teuthology.orchestra.run.smithi007.stdout:stat: stdout 167 167 2024-06-10T00:53:22.665 INFO:teuthology.orchestra.run.smithi007.stdout:Creating initial keys... 2024-06-10T00:53:23.688 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph-authtool: stdout AQCDTmZm40CmABAAM8Po/BAItvo6xJpHskiHVA== 2024-06-10T00:53:24.735 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph-authtool: stdout AQCETmZmxMr+ARAAQu+UnHU4RfNwsHOXKsc02Q== 2024-06-10T00:53:25.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph-authtool: stdout AQCFTmZmxBrhAxAAaukcnk+jqpg600ArWr4/yw== 2024-06-10T00:53:25.727 INFO:teuthology.orchestra.run.smithi007.stdout:Creating initial monmap... 2024-06-10T00:53:26.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-10T00:53:26.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-06-10T00:53:26.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T00:53:26.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-10T00:53:26.640 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool for a [v2:172.21.15.7:3300,v1:172.21.15.7:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-10T00:53:26.640 INFO:teuthology.orchestra.run.smithi007.stdout:setting min_mon_release = quincy 2024-06-10T00:53:26.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: set fsid to bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T00:53:26.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-10T00:53:26.640 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:53:26.640 INFO:teuthology.orchestra.run.smithi007.stdout:Creating mon... 2024-06-10T00:53:27.605 INFO:teuthology.orchestra.run.smithi007.stdout:create mon.a on 2024-06-10T00:53:27.837 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-06-10T00:53:27.988 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-06-10T00:53:28.178 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-bded58f8-26c3-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-bded58f8-26c3-11ef-bc9b-c7b262605968.target. 2024-06-10T00:53:28.178 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-bded58f8-26c3-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-bded58f8-26c3-11ef-bc9b-c7b262605968.target. 2024-06-10T00:53:28.492 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mon.a 2024-06-10T00:53:28.493 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to reset failed state of unit ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mon.a.service: Unit ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mon.a.service not loaded. 2024-06-10T00:53:28.902 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-bded58f8-26c3-11ef-bc9b-c7b262605968.target.wants/ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mon.a.service → /etc/systemd/system/ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@.service. 2024-06-10T00:53:29.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:29 smithi007 podman[32004]: 2024-06-10 00:53:29.045737247 +0000 UTC m=+0.018089050 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:53:30.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:29 smithi007 podman[32004]: 2024-06-10 00:53:29.969397606 +0000 UTC m=+0.941749402 container create 55dc616f81f28658efd7740ece427862f71002d976a9bb3c88a4773fa3b67eab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-a, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-3d89acc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-06-10T00:53:32.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:31 smithi007 podman[32004]: 2024-06-10 00:53:31.913717521 +0000 UTC m=+2.886069325 container init 55dc616f81f28658efd7740ece427862f71002d976a9bb3c88a4773fa3b67eab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T00:53:32.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:31 smithi007 podman[32004]: 2024-06-10 00:53:31.918189404 +0000 UTC m=+2.890541216 container start 55dc616f81f28658efd7740ece427862f71002d976a9bb3c88a4773fa3b67eab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-06-10T00:53:32.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:31 smithi007 ceph-mon[32027]: mkfs bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T00:53:32.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:31 smithi007 ceph-mon[32027]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-10T00:53:32.999 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-10T00:53:32.999 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-06-10T00:53:33.007 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-10T00:53:33.007 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-06-10T00:53:33.008 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-06-10T00:53:33.008 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to enable service . firewalld.service is not available 2024-06-10T00:53:33.008 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mon to start... 2024-06-10T00:53:33.008 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mon... 2024-06-10T00:53:33.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:32 smithi007 ceph-mon[32027]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-10T00:53:33.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:32 smithi007 ceph-mon[32027]: monmap epoch 1 2024-06-10T00:53:33.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:32 smithi007 ceph-mon[32027]: fsid bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T00:53:33.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:32 smithi007 ceph-mon[32027]: last_changed 2024-06-10T00:53:26.101611+0000 2024-06-10T00:53:33.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:32 smithi007 ceph-mon[32027]: created 2024-06-10T00:53:26.101611+0000 2024-06-10T00:53:33.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:32 smithi007 ceph-mon[32027]: min_mon_release 19 (squid) 2024-06-10T00:53:33.270 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:32 smithi007 ceph-mon[32027]: election_strategy: 1 2024-06-10T00:53:33.270 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:32 smithi007 ceph-mon[32027]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-06-10T00:53:33.270 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:32 smithi007 ceph-mon[32027]: fsmap 2024-06-10T00:53:33.270 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:32 smithi007 ceph-mon[32027]: osdmap e1: 0 total, 0 up, 0 in 2024-06-10T00:53:33.270 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:32 smithi007 ceph-mon[32027]: mgrmap e1: no daemons active 2024-06-10T00:53:33.270 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:32 smithi007 bash[32004]: 55dc616f81f28658efd7740ece427862f71002d976a9bb3c88a4773fa3b67eab 2024-06-10T00:53:33.270 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:32 smithi007 systemd[1]: Started Ceph mon.a for bded58f8-26c3-11ef-bc9b-c7b262605968. 2024-06-10T00:53:36.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:35 smithi007 ceph-mon[32027]: from='client.? 172.21.15.7:0/3103900385' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-06-10T00:53:42.032 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout cluster: 2024-06-10T00:53:42.038 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout id: bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T00:53:42.038 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-06-10T00:53:42.038 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-06-10T00:53:42.039 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout services: 2024-06-10T00:53:42.039 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 3s) 2024-06-10T00:53:42.039 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-06-10T00:53:42.039 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-06-10T00:53:42.039 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-06-10T00:53:42.039 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout data: 2024-06-10T00:53:42.039 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-06-10T00:53:42.039 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-06-10T00:53:42.039 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-06-10T00:53:42.039 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout pgs: 2024-06-10T00:53:42.039 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-06-10T00:53:42.040 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.748+0000 7f783f4fe700 1 Processor -- start 2024-06-10T00:53:42.040 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.748+0000 7f783f4fe700 1 -- start start 2024-06-10T00:53:42.040 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.748+0000 7f783f4fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7838107ed0 0x7f78381082f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:42.040 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.748+0000 7f783f4fe700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7838108830 con 0x7f7838107ed0 2024-06-10T00:53:42.040 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.748+0000 7f783d29a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7838107ed0 0x7f78381082f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:42.040 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.748+0000 7f783d29a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7838107ed0 0x7f78381082f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54076/0 (socket says 172.21.15.7:54076) 2024-06-10T00:53:42.040 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.748+0000 7f783d29a700 1 -- 172.21.15.7:0/219777327 learned_addr learned my addr 172.21.15.7:0/219777327 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:53:42.040 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.749+0000 7f783d29a700 1 -- 172.21.15.7:0/219777327 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7838108fe0 con 0x7f7838107ed0 2024-06-10T00:53:42.040 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.749+0000 7f783d29a700 1 --2- 172.21.15.7:0/219777327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7838107ed0 0x7f78381082f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f78340098a0 tx=0x7f7834009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e1d1afd595c227 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:42.040 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.749+0000 7f782ffff700 1 -- 172.21.15.7:0/219777327 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7834004160 con 0x7f7838107ed0 2024-06-10T00:53:42.040 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.749+0000 7f782ffff700 1 -- 172.21.15.7:0/219777327 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7834004300 con 0x7f7838107ed0 2024-06-10T00:53:42.040 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.749+0000 7f782ffff700 1 -- 172.21.15.7:0/219777327 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78340036a0 con 0x7f7838107ed0 2024-06-10T00:53:42.040 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.750+0000 7f783f4fe700 1 -- 172.21.15.7:0/219777327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7838107ed0 msgr2=0x7f78381082f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:42.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.750+0000 7f783f4fe700 1 --2- 172.21.15.7:0/219777327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7838107ed0 0x7f78381082f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f78340098a0 tx=0x7f7834009bb0 comp rx=0 tx=0).stop 2024-06-10T00:53:42.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.750+0000 7f783f4fe700 1 -- 172.21.15.7:0/219777327 shutdown_connections 2024-06-10T00:53:42.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.750+0000 7f783f4fe700 1 --2- 172.21.15.7:0/219777327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7838107ed0 0x7f78381082f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:42.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.750+0000 7f783f4fe700 1 -- 172.21.15.7:0/219777327 >> 172.21.15.7:0/219777327 conn(0x7f7838103470 msgr2=0x7f78381058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:42.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.750+0000 7f783f4fe700 1 -- 172.21.15.7:0/219777327 shutdown_connections 2024-06-10T00:53:42.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.750+0000 7f783f4fe700 1 -- 172.21.15.7:0/219777327 wait complete. 2024-06-10T00:53:42.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.750+0000 7f783f4fe700 1 Processor -- start 2024-06-10T00:53:42.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.750+0000 7f783f4fe700 1 -- start start 2024-06-10T00:53:42.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.751+0000 7f783f4fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7838107ed0 0x7f783819d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:42.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.751+0000 7f783f4fe700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f78381094b0 con 0x7f7838107ed0 2024-06-10T00:53:42.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.751+0000 7f783d29a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7838107ed0 0x7f783819d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:42.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.751+0000 7f783d29a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7838107ed0 0x7f783819d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54084/0 (socket says 172.21.15.7:54084) 2024-06-10T00:53:42.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.751+0000 7f783d29a700 1 -- 172.21.15.7:0/3103900385 learned_addr learned my addr 172.21.15.7:0/3103900385 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:53:42.042 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.751+0000 7f783d29a700 1 -- 172.21.15.7:0/3103900385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783819dc20 con 0x7f7838107ed0 2024-06-10T00:53:42.042 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.751+0000 7f783d29a700 1 --2- 172.21.15.7:0/3103900385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7838107ed0 0x7f783819d6e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7834000c00 tx=0x7f7834004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:42.042 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.751+0000 7f782e7fc700 1 -- 172.21.15.7:0/3103900385 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7834003db0 con 0x7f7838107ed0 2024-06-10T00:53:42.042 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.751+0000 7f782e7fc700 1 -- 172.21.15.7:0/3103900385 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7834011630 con 0x7f7838107ed0 2024-06-10T00:53:42.042 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.751+0000 7f782e7fc700 1 -- 172.21.15.7:0/3103900385 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7834011950 con 0x7f7838107ed0 2024-06-10T00:53:42.042 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.751+0000 7f783f4fe700 1 -- 172.21.15.7:0/3103900385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f783819deb0 con 0x7f7838107ed0 2024-06-10T00:53:42.042 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.752+0000 7f783f4fe700 1 -- 172.21.15.7:0/3103900385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78381a0ba0 con 0x7f7838107ed0 2024-06-10T00:53:42.042 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.752+0000 7f782e7fc700 1 -- 172.21.15.7:0/3103900385 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7834011af0 con 0x7f7838107ed0 2024-06-10T00:53:42.042 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.752+0000 7f782e7fc700 1 -- 172.21.15.7:0/3103900385 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f783401c450 con 0x7f7838107ed0 2024-06-10T00:53:42.042 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.752+0000 7f783f4fe700 1 -- 172.21.15.7:0/3103900385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7838063af0 con 0x7f7838107ed0 2024-06-10T00:53:42.042 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.755+0000 7f782e7fc700 1 -- 172.21.15.7:0/3103900385 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7834032020 con 0x7f7838107ed0 2024-06-10T00:53:42.042 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.829+0000 7f783f4fe700 1 -- 172.21.15.7:0/3103900385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f78381a1090 con 0x7f7838107ed0 2024-06-10T00:53:42.043 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.829+0000 7f782e7fc700 1 -- 172.21.15.7:0/3103900385 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+310 (secure 0 0 0) 0x7f7834004260 con 0x7f7838107ed0 2024-06-10T00:53:42.043 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.830+0000 7f783f4fe700 1 -- 172.21.15.7:0/3103900385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7838107ed0 msgr2=0x7f783819d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:42.043 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.830+0000 7f783f4fe700 1 --2- 172.21.15.7:0/3103900385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7838107ed0 0x7f783819d6e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7834000c00 tx=0x7f7834004750 comp rx=0 tx=0).stop 2024-06-10T00:53:42.043 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.830+0000 7f783f4fe700 1 -- 172.21.15.7:0/3103900385 shutdown_connections 2024-06-10T00:53:42.043 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.830+0000 7f783f4fe700 1 --2- 172.21.15.7:0/3103900385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7838107ed0 0x7f783819d6e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:42.043 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.830+0000 7f783f4fe700 1 -- 172.21.15.7:0/3103900385 >> 172.21.15.7:0/3103900385 conn(0x7f7838103470 msgr2=0x7f783807d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:42.043 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.830+0000 7f783f4fe700 1 -- 172.21.15.7:0/3103900385 shutdown_connections 2024-06-10T00:53:42.043 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:35.830+0000 7f783f4fe700 1 -- 172.21.15.7:0/3103900385 wait complete. 2024-06-10T00:53:42.043 INFO:teuthology.orchestra.run.smithi007.stdout:mon is available 2024-06-10T00:53:42.043 INFO:teuthology.orchestra.run.smithi007.stdout:Assimilating anything we can from ceph.conf... 2024-06-10T00:53:42.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:42 smithi007 ceph-mon[32027]: from='client.? 172.21.15.7:0/4044904476' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-10T00:53:42.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:42 smithi007 ceph-mon[32027]: from='client.? 172.21.15.7:0/4044904476' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-06-10T00:53:43.033 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-06-10T00:53:43.034 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [global] 2024-06-10T00:53:43.034 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout fsid = bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T00:53:43.034 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.7:3300,v1:172.21.15.7:6789] 2024-06-10T00:53:43.034 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-10T00:53:43.034 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-10T00:53:43.034 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-10T00:53:43.035 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-10T00:53:43.035 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-06-10T00:53:43.035 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-10T00:53:43.035 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-10T00:53:43.035 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-06-10T00:53:43.035 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [osd] 2024-06-10T00:53:43.035 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-10T00:53:43.035 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-10T00:53:43.036 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-10T00:53:43.036 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-10T00:53:43.036 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-10T00:53:43.036 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.446+0000 7f1c7c755700 1 Processor -- start 2024-06-10T00:53:43.036 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.446+0000 7f1c7c755700 1 -- start start 2024-06-10T00:53:43.036 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.447+0000 7f1c7c755700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c741089e0 0x7f1c7410add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:43.036 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.447+0000 7f1c7c755700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7407bde0 con 0x7f1c741089e0 2024-06-10T00:53:43.036 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.447+0000 7f1c7a4f1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c741089e0 0x7f1c7410add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:43.037 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.447+0000 7f1c7a4f1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c741089e0 0x7f1c7410add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38198/0 (socket says 172.21.15.7:38198) 2024-06-10T00:53:43.037 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.447+0000 7f1c7a4f1700 1 -- 172.21.15.7:0/2543739620 learned_addr learned my addr 172.21.15.7:0/2543739620 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:53:43.037 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.447+0000 7f1c7a4f1700 1 -- 172.21.15.7:0/2543739620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c7410b310 con 0x7f1c741089e0 2024-06-10T00:53:43.037 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.447+0000 7f1c7a4f1700 1 --2- 172.21.15.7:0/2543739620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c741089e0 0x7f1c7410add0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1c700098a0 tx=0x7f1c70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e13c4e7ef8455f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:43.037 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.448+0000 7f1c794ef700 1 -- 172.21.15.7:0/2543739620 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c70004160 con 0x7f1c741089e0 2024-06-10T00:53:43.037 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.448+0000 7f1c794ef700 1 -- 172.21.15.7:0/2543739620 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c70004300 con 0x7f1c741089e0 2024-06-10T00:53:43.037 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.448+0000 7f1c794ef700 1 -- 172.21.15.7:0/2543739620 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c700036a0 con 0x7f1c741089e0 2024-06-10T00:53:43.037 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.448+0000 7f1c7c755700 1 -- 172.21.15.7:0/2543739620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c741089e0 msgr2=0x7f1c7410add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:43.038 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.448+0000 7f1c7c755700 1 --2- 172.21.15.7:0/2543739620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c741089e0 0x7f1c7410add0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1c700098a0 tx=0x7f1c70009bb0 comp rx=0 tx=0).stop 2024-06-10T00:53:43.038 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.448+0000 7f1c7c755700 1 -- 172.21.15.7:0/2543739620 shutdown_connections 2024-06-10T00:53:43.038 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.448+0000 7f1c7c755700 1 --2- 172.21.15.7:0/2543739620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c741089e0 0x7f1c7410add0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:43.038 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.448+0000 7f1c7c755700 1 -- 172.21.15.7:0/2543739620 >> 172.21.15.7:0/2543739620 conn(0x7f1c74102310 msgr2=0x7f1c74104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:43.038 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.448+0000 7f1c7c755700 1 -- 172.21.15.7:0/2543739620 shutdown_connections 2024-06-10T00:53:43.038 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.448+0000 7f1c7c755700 1 -- 172.21.15.7:0/2543739620 wait complete. 2024-06-10T00:53:43.038 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.449+0000 7f1c7c755700 1 Processor -- start 2024-06-10T00:53:43.038 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.449+0000 7f1c7c755700 1 -- start start 2024-06-10T00:53:43.039 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.449+0000 7f1c7c755700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c741089e0 0x7f1c7419b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:43.039 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.449+0000 7f1c7c755700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7410b720 con 0x7f1c741089e0 2024-06-10T00:53:43.039 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.449+0000 7f1c7a4f1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c741089e0 0x7f1c7419b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:43.039 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.449+0000 7f1c7a4f1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c741089e0 0x7f1c7419b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38200/0 (socket says 172.21.15.7:38200) 2024-06-10T00:53:43.039 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.449+0000 7f1c7a4f1700 1 -- 172.21.15.7:0/4044904476 learned_addr learned my addr 172.21.15.7:0/4044904476 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:53:43.039 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.450+0000 7f1c7a4f1700 1 -- 172.21.15.7:0/4044904476 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c7419ba60 con 0x7f1c741089e0 2024-06-10T00:53:43.039 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.450+0000 7f1c7a4f1700 1 --2- 172.21.15.7:0/4044904476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c741089e0 0x7f1c7419b520 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1c70000c00 tx=0x7f1c70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:43.039 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.450+0000 7f1c677fe700 1 -- 172.21.15.7:0/4044904476 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c70003db0 con 0x7f1c741089e0 2024-06-10T00:53:43.039 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.450+0000 7f1c677fe700 1 -- 172.21.15.7:0/4044904476 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c70011630 con 0x7f1c741089e0 2024-06-10T00:53:43.040 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.450+0000 7f1c7c755700 1 -- 172.21.15.7:0/4044904476 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c7419bcf0 con 0x7f1c741089e0 2024-06-10T00:53:43.040 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.450+0000 7f1c677fe700 1 -- 172.21.15.7:0/4044904476 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c70011950 con 0x7f1c741089e0 2024-06-10T00:53:43.040 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.450+0000 7f1c7c755700 1 -- 172.21.15.7:0/4044904476 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c7419e990 con 0x7f1c741089e0 2024-06-10T00:53:43.040 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.450+0000 7f1c677fe700 1 -- 172.21.15.7:0/4044904476 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1c70011af0 con 0x7f1c741089e0 2024-06-10T00:53:43.040 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.450+0000 7f1c677fe700 1 -- 172.21.15.7:0/4044904476 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1c7001c450 con 0x7f1c741089e0 2024-06-10T00:53:43.040 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.451+0000 7f1c7c755700 1 -- 172.21.15.7:0/4044904476 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c74063af0 con 0x7f1c741089e0 2024-06-10T00:53:43.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.453+0000 7f1c677fe700 1 -- 172.21.15.7:0/4044904476 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f1c70032020 con 0x7f1c741089e0 2024-06-10T00:53:43.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.525+0000 7f1c7c755700 1 -- 172.21.15.7:0/4044904476 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f1c7402deb0 con 0x7f1c741089e0 2024-06-10T00:53:43.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.528+0000 7f1c677fe700 1 -- 172.21.15.7:0/4044904476 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+427 (secure 0 0 0) 0x7f1c70004260 con 0x7f1c741089e0 2024-06-10T00:53:43.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.528+0000 7f1c677fe700 1 -- 172.21.15.7:0/4044904476 <== mon.0 v2:172.21.15.7:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1c7001b450 con 0x7f1c741089e0 2024-06-10T00:53:43.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.529+0000 7f1c7c755700 1 -- 172.21.15.7:0/4044904476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c741089e0 msgr2=0x7f1c7419b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:43.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.529+0000 7f1c7c755700 1 --2- 172.21.15.7:0/4044904476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c741089e0 0x7f1c7419b520 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1c70000c00 tx=0x7f1c70004750 comp rx=0 tx=0).stop 2024-06-10T00:53:43.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.529+0000 7f1c7c755700 1 -- 172.21.15.7:0/4044904476 shutdown_connections 2024-06-10T00:53:43.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.529+0000 7f1c7c755700 1 --2- 172.21.15.7:0/4044904476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c741089e0 0x7f1c7419b520 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:43.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.529+0000 7f1c7c755700 1 -- 172.21.15.7:0/4044904476 >> 172.21.15.7:0/4044904476 conn(0x7f1c74102310 msgr2=0x7f1c741046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:43.042 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.529+0000 7f1c7c755700 1 -- 172.21.15.7:0/4044904476 shutdown_connections 2024-06-10T00:53:43.042 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:42.529+0000 7f1c7c755700 1 -- 172.21.15.7:0/4044904476 wait complete. 2024-06-10T00:53:43.042 INFO:teuthology.orchestra.run.smithi007.stdout:Generating new minimal ceph.conf... 2024-06-10T00:53:43.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:43 smithi007 ceph-mon[32027]: from='client.? 172.21.15.7:0/2462411635' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:53:44.144 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.469+0000 7f0842335700 1 Processor -- start 2024-06-10T00:53:44.144 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.469+0000 7f0842335700 1 -- start start 2024-06-10T00:53:44.144 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.469+0000 7f0842335700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f083c107ed0 0x7f083c1082f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:44.144 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.469+0000 7f0842335700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f083c108830 con 0x7f083c107ed0 2024-06-10T00:53:44.144 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.469+0000 7f083bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f083c107ed0 0x7f083c1082f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:44.145 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.469+0000 7f083bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f083c107ed0 0x7f083c1082f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38212/0 (socket says 172.21.15.7:38212) 2024-06-10T00:53:44.145 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.469+0000 7f083bfff700 1 -- 172.21.15.7:0/1284117518 learned_addr learned my addr 172.21.15.7:0/1284117518 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:53:44.145 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.470+0000 7f083bfff700 1 -- 172.21.15.7:0/1284117518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f083c108fe0 con 0x7f083c107ed0 2024-06-10T00:53:44.145 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.470+0000 7f083bfff700 1 --2- 172.21.15.7:0/1284117518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f083c107ed0 0x7f083c1082f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f082c0098a0 tx=0x7f082c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a948b8f325d3016 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:44.145 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.470+0000 7f083affd700 1 -- 172.21.15.7:0/1284117518 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f082c004160 con 0x7f083c107ed0 2024-06-10T00:53:44.145 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.470+0000 7f083affd700 1 -- 172.21.15.7:0/1284117518 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f082c004300 con 0x7f083c107ed0 2024-06-10T00:53:44.145 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.470+0000 7f083affd700 1 -- 172.21.15.7:0/1284117518 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f082c011b50 con 0x7f083c107ed0 2024-06-10T00:53:44.145 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.470+0000 7f0842335700 1 -- 172.21.15.7:0/1284117518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f083c107ed0 msgr2=0x7f083c1082f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:44.145 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.470+0000 7f0842335700 1 --2- 172.21.15.7:0/1284117518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f083c107ed0 0x7f083c1082f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f082c0098a0 tx=0x7f082c009bb0 comp rx=0 tx=0).stop 2024-06-10T00:53:44.145 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.471+0000 7f0842335700 1 -- 172.21.15.7:0/1284117518 shutdown_connections 2024-06-10T00:53:44.145 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.471+0000 7f0842335700 1 --2- 172.21.15.7:0/1284117518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f083c107ed0 0x7f083c1082f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:44.146 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.471+0000 7f0842335700 1 -- 172.21.15.7:0/1284117518 >> 172.21.15.7:0/1284117518 conn(0x7f083c103470 msgr2=0x7f083c1058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:44.146 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.471+0000 7f0842335700 1 -- 172.21.15.7:0/1284117518 shutdown_connections 2024-06-10T00:53:44.146 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.471+0000 7f0842335700 1 -- 172.21.15.7:0/1284117518 wait complete. 2024-06-10T00:53:44.146 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.471+0000 7f0842335700 1 Processor -- start 2024-06-10T00:53:44.146 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.471+0000 7f0842335700 1 -- start start 2024-06-10T00:53:44.146 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.472+0000 7f0842335700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f083c107ed0 0x7f083c19bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:44.146 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.472+0000 7f0842335700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f083c10c2a0 con 0x7f083c107ed0 2024-06-10T00:53:44.146 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.472+0000 7f083bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f083c107ed0 0x7f083c19bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:44.146 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.472+0000 7f083bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f083c107ed0 0x7f083c19bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38222/0 (socket says 172.21.15.7:38222) 2024-06-10T00:53:44.146 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.472+0000 7f083bfff700 1 -- 172.21.15.7:0/2462411635 learned_addr learned my addr 172.21.15.7:0/2462411635 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:53:44.146 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.472+0000 7f083bfff700 1 -- 172.21.15.7:0/2462411635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f083c19c2f0 con 0x7f083c107ed0 2024-06-10T00:53:44.146 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.472+0000 7f083bfff700 1 --2- 172.21.15.7:0/2462411635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f083c107ed0 0x7f083c19bdb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f082c011670 tx=0x7f082c011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:44.147 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.472+0000 7f08397fa700 1 -- 172.21.15.7:0/2462411635 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f082c0039c0 con 0x7f083c107ed0 2024-06-10T00:53:44.147 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.472+0000 7f08397fa700 1 -- 172.21.15.7:0/2462411635 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f082c011e50 con 0x7f083c107ed0 2024-06-10T00:53:44.147 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.472+0000 7f0842335700 1 -- 172.21.15.7:0/2462411635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f083c19c580 con 0x7f083c107ed0 2024-06-10T00:53:44.147 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.472+0000 7f08397fa700 1 -- 172.21.15.7:0/2462411635 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f082c01a5c0 con 0x7f083c107ed0 2024-06-10T00:53:44.147 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.472+0000 7f0842335700 1 -- 172.21.15.7:0/2462411635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f083c19f230 con 0x7f083c107ed0 2024-06-10T00:53:44.147 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.473+0000 7f08397fa700 1 -- 172.21.15.7:0/2462411635 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f082c01ac00 con 0x7f083c107ed0 2024-06-10T00:53:44.147 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.473+0000 7f08397fa700 1 -- 172.21.15.7:0/2462411635 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f082c019700 con 0x7f083c107ed0 2024-06-10T00:53:44.147 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.473+0000 7f0842335700 1 -- 172.21.15.7:0/2462411635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f083c10c420 con 0x7f083c107ed0 2024-06-10T00:53:44.147 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.476+0000 7f08397fa700 1 -- 172.21.15.7:0/2462411635 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f082c003b60 con 0x7f083c107ed0 2024-06-10T00:53:44.147 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.546+0000 7f0842335700 1 -- 172.21.15.7:0/2462411635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f083c19f860 con 0x7f083c107ed0 2024-06-10T00:53:44.147 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.547+0000 7f08397fa700 1 -- 172.21.15.7:0/2462411635 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+173 (secure 0 0 0) 0x7f082c0199a0 con 0x7f083c107ed0 2024-06-10T00:53:44.147 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.547+0000 7f0842335700 1 -- 172.21.15.7:0/2462411635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f083c107ed0 msgr2=0x7f083c19bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:44.147 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.547+0000 7f0842335700 1 --2- 172.21.15.7:0/2462411635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f083c107ed0 0x7f083c19bdb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f082c011670 tx=0x7f082c011b60 comp rx=0 tx=0).stop 2024-06-10T00:53:44.148 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.548+0000 7f0842335700 1 -- 172.21.15.7:0/2462411635 shutdown_connections 2024-06-10T00:53:44.148 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.548+0000 7f0842335700 1 --2- 172.21.15.7:0/2462411635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f083c107ed0 0x7f083c19bdb0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:44.148 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.548+0000 7f0842335700 1 -- 172.21.15.7:0/2462411635 >> 172.21.15.7:0/2462411635 conn(0x7f083c103470 msgr2=0x7f083c103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:44.148 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.548+0000 7f0842335700 1 -- 172.21.15.7:0/2462411635 shutdown_connections 2024-06-10T00:53:44.148 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:43.548+0000 7f0842335700 1 -- 172.21.15.7:0/2462411635 wait complete. 2024-06-10T00:53:44.148 INFO:teuthology.orchestra.run.smithi007.stdout:Restarting the monitor... 2024-06-10T00:53:44.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:44 smithi007 systemd[1]: Stopping Ceph mon.a for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T00:53:44.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:44 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-a[32023]: 2024-06-10T00:53:44.415+0000 7fedc14f7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T00:53:44.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:44 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-a[32023]: 2024-06-10T00:53:44.415+0000 7fedc14f7700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-06-10T00:53:44.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:44 smithi007 podman[32271]: 2024-06-10 00:53:44.559010103 +0000 UTC m=+0.243725002 container died 55dc616f81f28658efd7740ece427862f71002d976a9bb3c88a4773fa3b67eab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-06-10T00:53:45.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:44 smithi007 podman[32271]: 2024-06-10 00:53:44.898006519 +0000 UTC m=+0.582721409 container cleanup 55dc616f81f28658efd7740ece427862f71002d976a9bb3c88a4773fa3b67eab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-06-10T00:53:45.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:44 smithi007 bash[32271]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-a 2024-06-10T00:53:45.491 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 podman[32286]: 2024-06-10 00:53:45.07402183 +0000 UTC m=+0.515154422 container remove 55dc616f81f28658efd7740ece427862f71002d976a9bb3c88a4773fa3b67eab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-06-10T00:53:45.491 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 systemd[1]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mon.a.service: Deactivated successfully. 2024-06-10T00:53:45.491 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 systemd[1]: Stopped Ceph mon.a for bded58f8-26c3-11ef-bc9b-c7b262605968. 2024-06-10T00:53:45.491 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 systemd[1]: Starting Ceph mon.a for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T00:53:45.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 podman[32379]: 2024-06-10 00:53:45.491402385 +0000 UTC m=+0.102418814 container create afb11117c1a1cf56b620e531af50ff3ae10ee0d8f9930b6499359c181188a62d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-a, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-06-10T00:53:45.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 podman[32379]: 2024-06-10 00:53:45.406219728 +0000 UTC m=+0.017236169 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:53:45.770 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 podman[32379]: 2024-06-10 00:53:45.702374645 +0000 UTC m=+0.313391084 container init afb11117c1a1cf56b620e531af50ff3ae10ee0d8f9930b6499359c181188a62d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-a, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-06-10T00:53:45.770 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 podman[32379]: 2024-06-10 00:53:45.705566073 +0000 UTC m=+0.316582512 container start afb11117c1a1cf56b620e531af50ff3ae10ee0d8f9930b6499359c181188a62d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-3d89acc) 2024-06-10T00:53:45.770 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: set uid:gid to 167:167 (ceph:ceph) 2024-06-10T00:53:45.770 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc), process ceph-mon, pid 2 2024-06-10T00:53:45.770 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: pidfile_write: ignore empty --pid-file 2024-06-10T00:53:45.770 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: load: jerasure load: lrc 2024-06-10T00:53:45.770 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: RocksDB version: 7.9.2 2024-06-10T00:53:45.770 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Git sha 0 2024-06-10T00:53:45.770 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Compile date 2024-05-30 15:42:47 2024-06-10T00:53:45.771 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: DB SUMMARY 2024-06-10T00:53:45.771 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: DB Session ID: V9SOUNHQJE5T8WR94GMY 2024-06-10T00:53:45.771 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: CURRENT file: CURRENT 2024-06-10T00:53:45.771 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: IDENTITY file: IDENTITY 2024-06-10T00:53:45.771 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-06-10T00:53:45.771 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-06-10T00:53:45.771 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104571 ; 2024-06-10T00:53:45.771 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.error_if_exists: 0 2024-06-10T00:53:45.771 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.create_if_missing: 0 2024-06-10T00:53:45.771 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.paranoid_checks: 1 2024-06-10T00:53:45.771 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-10T00:53:45.771 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-10T00:53:45.771 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-10T00:53:45.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.env: 0x5636bb66ec60 2024-06-10T00:53:45.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.fs: PosixFileSystem 2024-06-10T00:53:45.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.info_log: 0x5636bc435b20 2024-06-10T00:53:45.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.max_file_opening_threads: 16 2024-06-10T00:53:45.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.statistics: (nil) 2024-06-10T00:53:45.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.use_fsync: 0 2024-06-10T00:53:45.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.max_log_file_size: 0 2024-06-10T00:53:45.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-10T00:53:45.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-10T00:53:45.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.keep_log_file_num: 1000 2024-06-10T00:53:45.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.recycle_log_file_num: 0 2024-06-10T00:53:45.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.allow_fallocate: 1 2024-06-10T00:53:45.773 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.allow_mmap_reads: 0 2024-06-10T00:53:45.773 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.allow_mmap_writes: 0 2024-06-10T00:53:45.773 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.use_direct_reads: 0 2024-06-10T00:53:45.773 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-10T00:53:45.773 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.create_missing_column_families: 0 2024-06-10T00:53:45.773 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.db_log_dir: 2024-06-10T00:53:45.773 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.wal_dir: 2024-06-10T00:53:45.773 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-10T00:53:45.773 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-10T00:53:45.773 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-10T00:53:45.773 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-10T00:53:45.773 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-10T00:53:45.774 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-10T00:53:45.774 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.advise_random_on_open: 1 2024-06-10T00:53:45.774 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.db_write_buffer_size: 0 2024-06-10T00:53:45.774 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.write_buffer_manager: 0x5636bc4061e0 2024-06-10T00:53:45.774 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-10T00:53:45.774 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-10T00:53:45.774 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-10T00:53:45.774 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.rate_limiter: (nil) 2024-06-10T00:53:45.774 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-10T00:53:45.774 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.wal_recovery_mode: 2 2024-06-10T00:53:45.775 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.enable_thread_tracking: 0 2024-06-10T00:53:45.775 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.enable_pipelined_write: 0 2024-06-10T00:53:45.776 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.unordered_write: 0 2024-06-10T00:53:45.776 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-10T00:53:45.776 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-10T00:53:45.776 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-10T00:53:45.776 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-10T00:53:45.776 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.row_cache: None 2024-06-10T00:53:45.776 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.wal_filter: None 2024-06-10T00:53:45.776 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-10T00:53:45.776 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.allow_ingest_behind: 0 2024-06-10T00:53:45.776 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.two_write_queues: 0 2024-06-10T00:53:45.776 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.manual_wal_flush: 0 2024-06-10T00:53:45.776 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.wal_compression: 0 2024-06-10T00:53:45.776 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.atomic_flush: 0 2024-06-10T00:53:45.777 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-10T00:53:45.777 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-10T00:53:45.777 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-10T00:53:45.777 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.log_readahead_size: 0 2024-06-10T00:53:45.777 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-10T00:53:45.777 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.best_efforts_recovery: 0 2024-06-10T00:53:45.777 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-10T00:53:45.777 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-10T00:53:45.777 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.allow_data_in_errors: 0 2024-06-10T00:53:45.777 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.db_host_id: __hostname__ 2024-06-10T00:53:45.777 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-10T00:53:45.777 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.max_background_jobs: 2 2024-06-10T00:53:45.777 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.max_background_compactions: -1 2024-06-10T00:53:45.777 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.max_subcompactions: 1 2024-06-10T00:53:45.778 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-10T00:53:45.778 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-10T00:53:45.778 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-10T00:53:45.778 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.max_total_wal_size: 0 2024-06-10T00:53:45.778 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-10T00:53:45.778 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-10T00:53:45.778 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-10T00:53:45.778 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-10T00:53:45.778 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.max_open_files: -1 2024-06-10T00:53:45.778 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.bytes_per_sync: 0 2024-06-10T00:53:45.778 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-10T00:53:45.779 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-10T00:53:45.779 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.compaction_readahead_size: 0 2024-06-10T00:53:45.779 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.max_background_flushes: -1 2024-06-10T00:53:45.779 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Compression algorithms supported: 2024-06-10T00:53:45.779 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-10T00:53:45.779 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: kZSTD supported: 0 2024-06-10T00:53:45.779 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: kXpressCompression supported: 0 2024-06-10T00:53:45.779 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: kLZ4HCCompression supported: 1 2024-06-10T00:53:45.779 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: kZlibCompression supported: 1 2024-06-10T00:53:45.779 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: kSnappyCompression supported: 1 2024-06-10T00:53:45.779 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: kLZ4Compression supported: 1 2024-06-10T00:53:45.779 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: kBZip2Compression supported: 0 2024-06-10T00:53:45.779 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-10T00:53:45.779 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-10T00:53:45.780 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-06-10T00:53:45.780 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-10T00:53:45.780 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-10T00:53:45.780 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.merge_operator: 2024-06-10T00:53:45.780 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.compaction_filter: None 2024-06-10T00:53:45.780 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.compaction_filter_factory: None 2024-06-10T00:53:45.780 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.sst_partitioner_factory: None 2024-06-10T00:53:45.780 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-10T00:53:45.780 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-10T00:53:45.780 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5636bc435c40) 2024-06-10T00:53:45.780 INFO:journalctl@ceph.mon.a.smithi007.stdout: cache_index_and_filter_blocks: 1 2024-06-10T00:53:45.780 INFO:journalctl@ceph.mon.a.smithi007.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-10T00:53:45.780 INFO:journalctl@ceph.mon.a.smithi007.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-10T00:53:45.781 INFO:journalctl@ceph.mon.a.smithi007.stdout: pin_top_level_index_and_filter: 1 2024-06-10T00:53:45.781 INFO:journalctl@ceph.mon.a.smithi007.stdout: index_type: 0 2024-06-10T00:53:45.781 INFO:journalctl@ceph.mon.a.smithi007.stdout: data_block_index_type: 0 2024-06-10T00:53:45.781 INFO:journalctl@ceph.mon.a.smithi007.stdout: index_shortening: 1 2024-06-10T00:53:45.781 INFO:journalctl@ceph.mon.a.smithi007.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-10T00:53:45.781 INFO:journalctl@ceph.mon.a.smithi007.stdout: checksum: 4 2024-06-10T00:53:45.781 INFO:journalctl@ceph.mon.a.smithi007.stdout: no_block_cache: 0 2024-06-10T00:53:45.781 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache: 0x5636bc437090 2024-06-10T00:53:45.781 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache_name: BinnedLRUCache 2024-06-10T00:53:45.781 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache_options: 2024-06-10T00:53:45.781 INFO:journalctl@ceph.mon.a.smithi007.stdout: capacity : 536870912 2024-06-10T00:53:45.781 INFO:journalctl@ceph.mon.a.smithi007.stdout: num_shard_bits : 4 2024-06-10T00:53:45.781 INFO:journalctl@ceph.mon.a.smithi007.stdout: strict_capacity_limit : 0 2024-06-10T00:53:45.781 INFO:journalctl@ceph.mon.a.smithi007.stdout: high_pri_pool_ratio: 0.000 2024-06-10T00:53:45.782 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache_compressed: (nil) 2024-06-10T00:53:45.782 INFO:journalctl@ceph.mon.a.smithi007.stdout: persistent_cache: (nil) 2024-06-10T00:53:45.782 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_size: 4096 2024-06-10T00:53:45.782 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_size_deviation: 10 2024-06-10T00:53:45.782 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_restart_interval: 16 2024-06-10T00:53:45.782 INFO:journalctl@ceph.mon.a.smithi007.stdout: index_block_restart_interval: 1 2024-06-10T00:53:45.782 INFO:journalctl@ceph.mon.a.smithi007.stdout: metadata_block_size: 4096 2024-06-10T00:53:45.782 INFO:journalctl@ceph.mon.a.smithi007.stdout: partition_filters: 0 2024-06-10T00:53:45.782 INFO:journalctl@ceph.mon.a.smithi007.stdout: use_delta_encoding: 1 2024-06-10T00:53:45.782 INFO:journalctl@ceph.mon.a.smithi007.stdout: filter_policy: bloomfilter 2024-06-10T00:53:45.782 INFO:journalctl@ceph.mon.a.smithi007.stdout: whole_key_filtering: 1 2024-06-10T00:53:45.782 INFO:journalctl@ceph.mon.a.smithi007.stdout: verify_compression: 0 2024-06-10T00:53:45.782 INFO:journalctl@ceph.mon.a.smithi007.stdout: read_amp_bytes_per_bit: 0 2024-06-10T00:53:45.783 INFO:journalctl@ceph.mon.a.smithi007.stdout: format_version: 5 2024-06-10T00:53:45.783 INFO:journalctl@ceph.mon.a.smithi007.stdout: enable_index_compression: 1 2024-06-10T00:53:45.783 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_align: 0 2024-06-10T00:53:45.783 INFO:journalctl@ceph.mon.a.smithi007.stdout: max_auto_readahead_size: 262144 2024-06-10T00:53:45.783 INFO:journalctl@ceph.mon.a.smithi007.stdout: prepopulate_block_cache: 0 2024-06-10T00:53:45.783 INFO:journalctl@ceph.mon.a.smithi007.stdout: initial_auto_readahead_size: 8192 2024-06-10T00:53:45.783 INFO:journalctl@ceph.mon.a.smithi007.stdout: num_file_reads_for_auto_readahead: 2 2024-06-10T00:53:45.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.write_buffer_size: 33554432 2024-06-10T00:53:45.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.max_write_buffer_number: 2 2024-06-10T00:53:45.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.compression: NoCompression 2024-06-10T00:53:45.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.bottommost_compression: Disabled 2024-06-10T00:53:45.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.prefix_extractor: nullptr 2024-06-10T00:53:45.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-10T00:53:45.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.num_levels: 7 2024-06-10T00:53:45.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-10T00:53:45.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-10T00:53:45.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-10T00:53:45.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-10T00:53:45.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-10T00:53:45.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-10T00:53:45.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-10T00:53:45.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-10T00:53:45.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-10T00:53:45.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-10T00:53:45.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-10T00:53:45.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-10T00:53:45.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-10T00:53:45.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.compression_opts.level: 32767 2024-06-10T00:53:45.785 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.compression_opts.strategy: 0 2024-06-10T00:53:45.785 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-10T00:53:45.785 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-10T00:53:45.785 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-10T00:53:45.785 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-10T00:53:45.785 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.compression_opts.enabled: false 2024-06-10T00:53:45.785 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-10T00:53:45.785 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-10T00:53:45.785 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-10T00:53:45.785 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-10T00:53:45.785 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.target_file_size_base: 67108864 2024-06-10T00:53:45.786 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-10T00:53:45.786 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-10T00:53:45.786 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-10T00:53:45.786 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-10T00:53:45.786 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-10T00:53:45.786 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-10T00:53:45.786 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-10T00:53:45.786 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-10T00:53:45.786 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-10T00:53:45.786 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-10T00:53:45.786 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-10T00:53:45.786 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-10T00:53:45.787 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-10T00:53:45.787 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-10T00:53:45.787 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.arena_block_size: 1048576 2024-06-10T00:53:45.787 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-10T00:53:45.787 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-10T00:53:45.787 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.disable_auto_compactions: 0 2024-06-10T00:53:45.787 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-10T00:53:45.787 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-10T00:53:45.787 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-10T00:53:45.787 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-10T00:53:45.787 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-10T00:53:45.787 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-10T00:53:45.787 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-10T00:53:45.788 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-10T00:53:45.788 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-10T00:53:45.788 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-10T00:53:45.788 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.table_properties_collectors: 2024-06-10T00:53:45.788 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.inplace_update_support: 0 2024-06-10T00:53:45.788 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-10T00:53:45.788 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-10T00:53:45.788 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-10T00:53:45.788 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-10T00:53:45.788 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.bloom_locality: 0 2024-06-10T00:53:45.788 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.max_successive_merges: 0 2024-06-10T00:53:45.788 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-10T00:53:45.788 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.paranoid_file_checks: 0 2024-06-10T00:53:45.788 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.force_consistency_checks: 1 2024-06-10T00:53:45.789 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.report_bg_io_stats: 0 2024-06-10T00:53:45.789 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.ttl: 2592000 2024-06-10T00:53:45.789 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-10T00:53:45.789 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-10T00:53:45.789 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-10T00:53:45.789 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.enable_blob_files: false 2024-06-10T00:53:45.789 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.min_blob_size: 0 2024-06-10T00:53:45.789 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.blob_file_size: 268435456 2024-06-10T00:53:45.789 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-10T00:53:45.789 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-10T00:53:45.789 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-10T00:53:45.789 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-10T00:53:45.789 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-10T00:53:45.789 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.blob_file_starting_level: 0 2024-06-10T00:53:45.790 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-10T00:53:45.790 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-06-10T00:53:45.790 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-06-10T00:53:45.790 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 8eeafa35-9481-4d9b-82e4-1a4711075f18 2024-06-10T00:53:45.790 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717980825735599, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-06-10T00:53:45.790 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-06-10T00:53:45.790 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717980825736632, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99582, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 97490, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14084, "raw_average_key_size": 49, "raw_value_size": 89838, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1717980825, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "8eeafa35-9481-4d9b-82e4-1a4711075f18", "db_session_id": "V9SOUNHQJE5T8WR94GMY", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-06-10T00:53:45.790 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717980825736717, "job": 1, "event": "recovery_finished"} 2024-06-10T00:53:45.790 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-06-10T00:53:45.790 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-10T00:53:45.790 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5636bc526000 2024-06-10T00:53:45.790 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: DB pointer 0x5636bc510000 2024-06-10T00:53:45.790 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-10T00:53:45.791 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-10T00:53:45.791 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** DB Stats ** 2024-06-10T00:53:45.791 INFO:journalctl@ceph.mon.a.smithi007.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-10T00:53:45.791 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-10T00:53:45.791 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-10T00:53:45.791 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-10T00:53:45.791 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-10T00:53:45.791 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-10T00:53:45.791 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-10T00:53:45.791 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-06-10T00:53:45.791 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** Compaction Stats [default] ** 2024-06-10T00:53:45.791 INFO:journalctl@ceph.mon.a.smithi007.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-10T00:53:45.791 INFO:journalctl@ceph.mon.a.smithi007.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-10T00:53:45.791 INFO:journalctl@ceph.mon.a.smithi007.stdout: L0 2/0 99.08 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 156.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-10T00:53:45.792 INFO:journalctl@ceph.mon.a.smithi007.stdout: Sum 2/0 99.08 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 156.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-10T00:53:45.792 INFO:journalctl@ceph.mon.a.smithi007.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 156.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-10T00:53:45.792 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-06-10T00:53:45.792 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** Compaction Stats [default] ** 2024-06-10T00:53:45.792 INFO:journalctl@ceph.mon.a.smithi007.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-10T00:53:45.792 INFO:journalctl@ceph.mon.a.smithi007.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-10T00:53:45.792 INFO:journalctl@ceph.mon.a.smithi007.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 156.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-10T00:53:45.792 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-06-10T00:53:45.792 INFO:journalctl@ceph.mon.a.smithi007.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-10T00:53:45.792 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-06-10T00:53:45.792 INFO:journalctl@ceph.mon.a.smithi007.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-10T00:53:45.792 INFO:journalctl@ceph.mon.a.smithi007.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-10T00:53:45.792 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-10T00:53:45.793 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-10T00:53:45.793 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-10T00:53:45.793 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-10T00:53:45.793 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative compaction: 0.00 GB write, 30.35 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-10T00:53:45.793 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval compaction: 0.00 GB write, 30.35 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-10T00:53:45.793 INFO:journalctl@ceph.mon.a.smithi007.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-10T00:53:45.793 INFO:journalctl@ceph.mon.a.smithi007.stdout: Block cache BinnedLRUCache@0x5636bc437090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-06-10T00:53:45.793 INFO:journalctl@ceph.mon.a.smithi007.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-06-10T00:53:45.793 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-06-10T00:53:45.793 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-10T00:53:45.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: starting mon.a rank 0 at public addrs [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] at bind addrs [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T00:53:45.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: mon.a@-1(???) e1 preinit fsid bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T00:53:45.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: mon.a@-1(???).mds e1 new map 2024-06-10T00:53:45.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: mon.a@-1(???).mds e1 print_map 2024-06-10T00:53:45.794 INFO:journalctl@ceph.mon.a.smithi007.stdout: e1 2024-06-10T00:53:45.794 INFO:journalctl@ceph.mon.a.smithi007.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-10T00:53:45.794 INFO:journalctl@ceph.mon.a.smithi007.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-10T00:53:45.794 INFO:journalctl@ceph.mon.a.smithi007.stdout: legacy client fscid: -1 2024-06-10T00:53:45.794 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-06-10T00:53:45.794 INFO:journalctl@ceph.mon.a.smithi007.stdout: No filesystems configured 2024-06-10T00:53:45.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-10T00:53:45.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-10T00:53:45.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-10T00:53:45.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-10T00:53:45.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-06-10T00:53:45.795 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 bash[32379]: afb11117c1a1cf56b620e531af50ff3ae10ee0d8f9930b6499359c181188a62d 2024-06-10T00:53:45.795 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 systemd[1]: Started Ceph mon.a for bded58f8-26c3-11ef-bc9b-c7b262605968. 2024-06-10T00:53:45.795 INFO:teuthology.orchestra.run.smithi007.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-06-10T00:53:46.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-10T00:53:46.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: monmap epoch 1 2024-06-10T00:53:46.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: fsid bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T00:53:46.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: last_changed 2024-06-10T00:53:26.101611+0000 2024-06-10T00:53:46.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: created 2024-06-10T00:53:26.101611+0000 2024-06-10T00:53:46.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: min_mon_release 19 (squid) 2024-06-10T00:53:46.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: election_strategy: 1 2024-06-10T00:53:46.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-06-10T00:53:46.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: fsmap 2024-06-10T00:53:46.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: osdmap e1: 0 total, 0 up, 0 in 2024-06-10T00:53:46.020 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:45 smithi007 ceph-mon[32398]: mgrmap e1: no daemons active 2024-06-10T00:53:46.790 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.161+0000 7f26bf9e5700 1 Processor -- start 2024-06-10T00:53:46.791 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.161+0000 7f26bf9e5700 1 -- start start 2024-06-10T00:53:46.791 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.161+0000 7f26bf9e5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26b8107ee0 0x7f26b8108300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:46.791 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.161+0000 7f26bf9e5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f26b8108840 con 0x7f26b8107ee0 2024-06-10T00:53:46.791 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.161+0000 7f26bd781700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26b8107ee0 0x7f26b8108300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:46.791 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.161+0000 7f26bd781700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26b8107ee0 0x7f26b8108300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38230/0 (socket says 172.21.15.7:38230) 2024-06-10T00:53:46.791 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.161+0000 7f26bd781700 1 -- 172.21.15.7:0/380522355 learned_addr learned my addr 172.21.15.7:0/380522355 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:53:46.791 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.162+0000 7f26bd781700 1 -- 172.21.15.7:0/380522355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b8108ff0 con 0x7f26b8107ee0 2024-06-10T00:53:46.791 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.162+0000 7f26bd781700 1 --2- 172.21.15.7:0/380522355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26b8107ee0 0x7f26b8108300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f26b40098a0 tx=0x7f26b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1308dd7689e3fcee server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:46.791 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.162+0000 7f26affff700 1 -- 172.21.15.7:0/380522355 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26b4004160 con 0x7f26b8107ee0 2024-06-10T00:53:46.792 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.162+0000 7f26affff700 1 -- 172.21.15.7:0/380522355 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f26b4004300 con 0x7f26b8107ee0 2024-06-10T00:53:46.792 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.162+0000 7f26affff700 1 -- 172.21.15.7:0/380522355 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26b4011b50 con 0x7f26b8107ee0 2024-06-10T00:53:46.792 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.163+0000 7f26bf9e5700 1 -- 172.21.15.7:0/380522355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26b8107ee0 msgr2=0x7f26b8108300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:46.792 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.163+0000 7f26bf9e5700 1 --2- 172.21.15.7:0/380522355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26b8107ee0 0x7f26b8108300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f26b40098a0 tx=0x7f26b4009bb0 comp rx=0 tx=0).stop 2024-06-10T00:53:46.792 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.163+0000 7f26bf9e5700 1 -- 172.21.15.7:0/380522355 shutdown_connections 2024-06-10T00:53:46.792 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.163+0000 7f26bf9e5700 1 --2- 172.21.15.7:0/380522355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26b8107ee0 0x7f26b8108300 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:46.792 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.163+0000 7f26bf9e5700 1 -- 172.21.15.7:0/380522355 >> 172.21.15.7:0/380522355 conn(0x7f26b81034a0 msgr2=0x7f26b81058c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:46.792 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.163+0000 7f26bf9e5700 1 -- 172.21.15.7:0/380522355 shutdown_connections 2024-06-10T00:53:46.792 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.163+0000 7f26bf9e5700 1 -- 172.21.15.7:0/380522355 wait complete. 2024-06-10T00:53:46.792 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.163+0000 7f26bf9e5700 1 Processor -- start 2024-06-10T00:53:46.792 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.164+0000 7f26bf9e5700 1 -- start start 2024-06-10T00:53:46.793 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.164+0000 7f26bf9e5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26b8107ee0 0x7f26b819be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:46.793 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.164+0000 7f26bf9e5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f26b810c2b0 con 0x7f26b8107ee0 2024-06-10T00:53:46.793 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.164+0000 7f26bd781700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26b8107ee0 0x7f26b819be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:46.793 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.164+0000 7f26bd781700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26b8107ee0 0x7f26b819be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38240/0 (socket says 172.21.15.7:38240) 2024-06-10T00:53:46.793 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.164+0000 7f26bd781700 1 -- 172.21.15.7:0/302746174 learned_addr learned my addr 172.21.15.7:0/302746174 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:53:46.793 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.164+0000 7f26bd781700 1 -- 172.21.15.7:0/302746174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b819c390 con 0x7f26b8107ee0 2024-06-10T00:53:46.793 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.164+0000 7f26bd781700 1 --2- 172.21.15.7:0/302746174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26b8107ee0 0x7f26b819be50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f26b4009fb0 tx=0x7f26b4011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:46.793 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.164+0000 7f26ae7fc700 1 -- 172.21.15.7:0/302746174 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26b40038b0 con 0x7f26b8107ee0 2024-06-10T00:53:46.793 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.164+0000 7f26ae7fc700 1 -- 172.21.15.7:0/302746174 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f26b4011e50 con 0x7f26b8107ee0 2024-06-10T00:53:46.793 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.164+0000 7f26ae7fc700 1 -- 172.21.15.7:0/302746174 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26b4019420 con 0x7f26b8107ee0 2024-06-10T00:53:46.793 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.165+0000 7f26bf9e5700 1 -- 172.21.15.7:0/302746174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b819c620 con 0x7f26b8107ee0 2024-06-10T00:53:46.793 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.165+0000 7f26bf9e5700 1 -- 172.21.15.7:0/302746174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b819f310 con 0x7f26b8107ee0 2024-06-10T00:53:46.794 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.165+0000 7f26ae7fc700 1 -- 172.21.15.7:0/302746174 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f26b4019a60 con 0x7f26b8107ee0 2024-06-10T00:53:46.794 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.165+0000 7f26ae7fc700 1 -- 172.21.15.7:0/302746174 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f26b4024730 con 0x7f26b8107ee0 2024-06-10T00:53:46.794 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.165+0000 7f26bf9e5700 1 -- 172.21.15.7:0/302746174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26b810c430 con 0x7f26b8107ee0 2024-06-10T00:53:46.794 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.168+0000 7f26ae7fc700 1 -- 172.21.15.7:0/302746174 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f26b4043dd0 con 0x7f26b8107ee0 2024-06-10T00:53:46.794 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.237+0000 7f26bf9e5700 1 -- 172.21.15.7:0/302746174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f26b819f9a0 con 0x7f26b8107ee0 2024-06-10T00:53:46.794 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.239+0000 7f26ae7fc700 1 -- 172.21.15.7:0/302746174 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f26b4032030 con 0x7f26b8107ee0 2024-06-10T00:53:46.794 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.240+0000 7f26bf9e5700 1 -- 172.21.15.7:0/302746174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26b8107ee0 msgr2=0x7f26b819be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:46.794 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.240+0000 7f26bf9e5700 1 --2- 172.21.15.7:0/302746174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26b8107ee0 0x7f26b819be50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f26b4009fb0 tx=0x7f26b4011bf0 comp rx=0 tx=0).stop 2024-06-10T00:53:46.794 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.240+0000 7f26bf9e5700 1 -- 172.21.15.7:0/302746174 shutdown_connections 2024-06-10T00:53:46.794 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.240+0000 7f26bf9e5700 1 --2- 172.21.15.7:0/302746174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26b8107ee0 0x7f26b819be50 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:46.795 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.240+0000 7f26bf9e5700 1 -- 172.21.15.7:0/302746174 >> 172.21.15.7:0/302746174 conn(0x7f26b81034a0 msgr2=0x7f26b8104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:46.795 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.240+0000 7f26bf9e5700 1 -- 172.21.15.7:0/302746174 shutdown_connections 2024-06-10T00:53:46.795 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:46.240+0000 7f26bf9e5700 1 -- 172.21.15.7:0/302746174 wait complete. 2024-06-10T00:53:46.795 INFO:teuthology.orchestra.run.smithi007.stdout:Wrote config to /etc/ceph/ceph.conf 2024-06-10T00:53:46.816 INFO:teuthology.orchestra.run.smithi007.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-06-10T00:53:46.816 INFO:teuthology.orchestra.run.smithi007.stdout:Creating mgr... 2024-06-10T00:53:46.816 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying port 0.0.0.0:9283 ... 2024-06-10T00:53:46.817 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying port 0.0.0.0:8765 ... 2024-06-10T00:53:47.083 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mgr.x 2024-06-10T00:53:47.083 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to reset failed state of unit ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mgr.x.service: Unit ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mgr.x.service not loaded. 2024-06-10T00:53:47.238 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-bded58f8-26c3-11ef-bc9b-c7b262605968.target.wants/ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@.service. 2024-06-10T00:53:47.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:47 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/302746174' entity='client.admin' 2024-06-10T00:53:47.519 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:47 smithi007 systemd[1]: Starting Ceph mgr.x for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T00:53:48.019 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:47 smithi007 podman[32608]: 2024-06-10 00:53:47.488017949 +0000 UTC m=+0.017716971 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:53:48.399 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:48 smithi007 podman[32608]: 2024-06-10 00:53:48.105900939 +0000 UTC m=+0.635599936 container create 4bce828fe08d754fbbe86fcfe6c35c5a28ec7e5a062d672d35f59f4a2e6b4daf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T00:53:48.399 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:48 smithi007 podman[32608]: 2024-06-10 00:53:48.325109371 +0000 UTC m=+0.854808382 container init 4bce828fe08d754fbbe86fcfe6c35c5a28ec7e5a062d672d35f59f4a2e6b4daf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=) 2024-06-10T00:53:48.399 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:48 smithi007 podman[32608]: 2024-06-10 00:53:48.328560927 +0000 UTC m=+0.858259928 container start 4bce828fe08d754fbbe86fcfe6c35c5a28ec7e5a062d672d35f59f4a2e6b4daf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-06-10T00:53:48.400 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:48 smithi007 bash[32608]: 4bce828fe08d754fbbe86fcfe6c35c5a28ec7e5a062d672d35f59f4a2e6b4daf 2024-06-10T00:53:48.414 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-10T00:53:48.414 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-06-10T00:53:48.423 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-10T00:53:48.423 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-06-10T00:53:48.423 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-06-10T00:53:48.423 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to enable service . firewalld.service is not available 2024-06-10T00:53:48.433 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-10T00:53:48.433 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-06-10T00:53:48.441 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-10T00:53:48.441 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-06-10T00:53:48.441 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-06-10T00:53:48.442 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-06-10T00:53:48.442 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr to start... 2024-06-10T00:53:48.442 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr... 2024-06-10T00:53:48.652 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:48 smithi007 systemd[1]: Started Ceph mgr.x for bded58f8-26c3-11ef-bc9b-c7b262605968. 2024-06-10T00:53:48.652 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:48 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:53:48.501+0000 7f4b1d582200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-10T00:53:48.914 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:48 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:53:48.699+0000 7f4b1d582200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:53:48.914 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:48 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:53:48.809+0000 7f4b1d582200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-10T00:53:49.240 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:48 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3583106266' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-10T00:53:49.240 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:49 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:53:49.144+0000 7f4b1d582200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-10T00:53:49.240 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:49 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:53:49.239+0000 7f4b1d582200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-10T00:53:49.474 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-06-10T00:53:49.477 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-06-10T00:53:49.477 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "bded58f8-26c3-11ef-bc9b-c7b262605968", 2024-06-10T00:53:49.477 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-06-10T00:53:49.477 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-10T00:53:49.477 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-10T00:53:49.477 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-10T00:53:49.477 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:53:49.477 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-10T00:53:49.477 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-10T00:53:49.477 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-06-10T00:53:49.478 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-06-10T00:53:49.478 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-10T00:53:49.478 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "a" 2024-06-10T00:53:49.478 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-06-10T00:53:49.478 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-06-10T00:53:49.478 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-10T00:53:49.478 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:53:49.478 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-10T00:53:49.478 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-10T00:53:49.478 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:53:49.478 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-10T00:53:49.478 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:53:49.478 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-10T00:53:49.478 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-10T00:53:49.479 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-10T00:53:49.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-10T00:53:49.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-10T00:53:49.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-10T00:53:49.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:53:49.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-10T00:53:49.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-10T00:53:49.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-10T00:53:49.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-10T00:53:49.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-10T00:53:49.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-10T00:53:49.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-10T00:53:49.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-10T00:53:49.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-10T00:53:49.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:53:49.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-10T00:53:49.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:53:49.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-10T00:53:49.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-10T00:53:49.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:53:49.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-10T00:53:49.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-10T00:53:49.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-10T00:53:49.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-10T00:53:49.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-10T00:53:49.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-10T00:53:49.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-06-10T00:53:49.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-06-10T00:53:49.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-10T00:53:49.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:53:49.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-10T00:53:49.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:53:49.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-06-10T00:53:31.966769+0000", 2024-06-10T00:53:49.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-10T00:53:49.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:53:49.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-10T00:53:49.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-06-10T00:53:49.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.832+0000 7f575ccf2700 1 Processor -- start 2024-06-10T00:53:49.489 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.832+0000 7f575ccf2700 1 -- start start 2024-06-10T00:53:49.489 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.832+0000 7f575ccf2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57581088f0 0x7f575810ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:49.489 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.832+0000 7f575ccf2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f575807c730 con 0x7f57581088f0 2024-06-10T00:53:49.489 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.833+0000 7f575659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57581088f0 0x7f575810ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:49.489 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.833+0000 7f575659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57581088f0 0x7f575810ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38258/0 (socket says 172.21.15.7:38258) 2024-06-10T00:53:49.489 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.833+0000 7f575659c700 1 -- 172.21.15.7:0/3010220049 learned_addr learned my addr 172.21.15.7:0/3010220049 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:53:49.489 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.833+0000 7f575659c700 1 -- 172.21.15.7:0/3010220049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f575810b220 con 0x7f57581088f0 2024-06-10T00:53:49.489 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.833+0000 7f575659c700 1 --2- 172.21.15.7:0/3010220049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57581088f0 0x7f575810ace0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f57400098a0 tx=0x7f5740009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58cc8b4fd66b96cf server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:49.489 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.834+0000 7f575559a700 1 -- 172.21.15.7:0/3010220049 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5740004160 con 0x7f57581088f0 2024-06-10T00:53:49.489 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.834+0000 7f575559a700 1 -- 172.21.15.7:0/3010220049 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5740004300 con 0x7f57581088f0 2024-06-10T00:53:49.489 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.834+0000 7f575559a700 1 -- 172.21.15.7:0/3010220049 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5740011b50 con 0x7f57581088f0 2024-06-10T00:53:49.489 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.834+0000 7f575ccf2700 1 -- 172.21.15.7:0/3010220049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57581088f0 msgr2=0x7f575810ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:49.489 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.834+0000 7f575ccf2700 1 --2- 172.21.15.7:0/3010220049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57581088f0 0x7f575810ace0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f57400098a0 tx=0x7f5740009bb0 comp rx=0 tx=0).stop 2024-06-10T00:53:49.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.834+0000 7f575ccf2700 1 -- 172.21.15.7:0/3010220049 shutdown_connections 2024-06-10T00:53:49.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.834+0000 7f575ccf2700 1 --2- 172.21.15.7:0/3010220049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57581088f0 0x7f575810ace0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:49.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.834+0000 7f575ccf2700 1 -- 172.21.15.7:0/3010220049 >> 172.21.15.7:0/3010220049 conn(0x7f5758102380 msgr2=0x7f57581047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:49.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.834+0000 7f575ccf2700 1 -- 172.21.15.7:0/3010220049 shutdown_connections 2024-06-10T00:53:49.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.834+0000 7f575ccf2700 1 -- 172.21.15.7:0/3010220049 wait complete. 2024-06-10T00:53:49.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.835+0000 7f575ccf2700 1 Processor -- start 2024-06-10T00:53:49.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.835+0000 7f575ccf2700 1 -- start start 2024-06-10T00:53:49.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.835+0000 7f575ccf2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57581088f0 0x7f5758108170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:49.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.835+0000 7f575ccf2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f575810c160 con 0x7f57581088f0 2024-06-10T00:53:49.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.836+0000 7f575659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57581088f0 0x7f5758108170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:49.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.836+0000 7f575659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57581088f0 0x7f5758108170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38266/0 (socket says 172.21.15.7:38266) 2024-06-10T00:53:49.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.836+0000 7f575659c700 1 -- 172.21.15.7:0/3583106266 learned_addr learned my addr 172.21.15.7:0/3583106266 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:53:49.491 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.836+0000 7f575659c700 1 -- 172.21.15.7:0/3583106266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57581086b0 con 0x7f57581088f0 2024-06-10T00:53:49.491 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.836+0000 7f575659c700 1 --2- 172.21.15.7:0/3583106266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57581088f0 0x7f5758108170 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5740011b60 tx=0x7f5740011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:49.491 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.836+0000 7f574f7fe700 1 -- 172.21.15.7:0/3583106266 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5740003970 con 0x7f57581088f0 2024-06-10T00:53:49.491 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.836+0000 7f574f7fe700 1 -- 172.21.15.7:0/3583106266 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5740003b10 con 0x7f57581088f0 2024-06-10T00:53:49.491 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.836+0000 7f575ccf2700 1 -- 172.21.15.7:0/3583106266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57581068a0 con 0x7f57581088f0 2024-06-10T00:53:49.491 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.836+0000 7f574f7fe700 1 -- 172.21.15.7:0/3583106266 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5740024610 con 0x7f57581088f0 2024-06-10T00:53:49.491 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.836+0000 7f575ccf2700 1 -- 172.21.15.7:0/3583106266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5758106de0 con 0x7f57581088f0 2024-06-10T00:53:49.491 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.837+0000 7f574f7fe700 1 -- 172.21.15.7:0/3583106266 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5740024c50 con 0x7f57581088f0 2024-06-10T00:53:49.491 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.837+0000 7f574f7fe700 1 -- 172.21.15.7:0/3583106266 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5740011e00 con 0x7f57581088f0 2024-06-10T00:53:49.491 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.837+0000 7f575ccf2700 1 -- 172.21.15.7:0/3583106266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5758063af0 con 0x7f57581088f0 2024-06-10T00:53:49.491 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.840+0000 7f574f7fe700 1 -- 172.21.15.7:0/3583106266 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f574001a9f0 con 0x7f57581088f0 2024-06-10T00:53:49.491 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.913+0000 7f575ccf2700 1 -- 172.21.15.7:0/3583106266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f57581070f0 con 0x7f57581088f0 2024-06-10T00:53:49.491 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.914+0000 7f574f7fe700 1 -- 172.21.15.7:0/3583106266 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f574001a430 con 0x7f57581088f0 2024-06-10T00:53:49.492 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.915+0000 7f575ccf2700 1 -- 172.21.15.7:0/3583106266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57581088f0 msgr2=0x7f5758108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:49.492 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.915+0000 7f575ccf2700 1 --2- 172.21.15.7:0/3583106266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57581088f0 0x7f5758108170 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5740011b60 tx=0x7f5740011c40 comp rx=0 tx=0).stop 2024-06-10T00:53:49.492 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.915+0000 7f575ccf2700 1 -- 172.21.15.7:0/3583106266 shutdown_connections 2024-06-10T00:53:49.492 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.915+0000 7f575ccf2700 1 --2- 172.21.15.7:0/3583106266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57581088f0 0x7f5758108170 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:49.492 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.915+0000 7f575ccf2700 1 -- 172.21.15.7:0/3583106266 >> 172.21.15.7:0/3583106266 conn(0x7f5758102380 msgr2=0x7f5758102eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:49.492 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.916+0000 7f575ccf2700 1 -- 172.21.15.7:0/3583106266 shutdown_connections 2024-06-10T00:53:49.492 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:48.916+0000 7f575ccf2700 1 -- 172.21.15.7:0/3583106266 wait complete. 2024-06-10T00:53:49.492 INFO:teuthology.orchestra.run.smithi007.stdout:mgr not available, waiting (1/15)... 2024-06-10T00:53:50.769 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:50 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:53:50.514+0000 7f4b1d582200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-10T00:53:50.769 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:50 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:53:50.690+0000 7f4b1d582200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-10T00:53:51.168 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:50 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:53:50.867+0000 7f4b1d582200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-10T00:53:51.168 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:51 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:53:51.168+0000 7f4b1d582200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-10T00:53:51.475 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:51 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:53:51.264+0000 7f4b1d582200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-10T00:53:51.475 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:51 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:53:51.474+0000 7f4b1d582200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-10T00:53:51.768 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:51 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:53:51.678+0000 7f4b1d582200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:53:52.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:52 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/111543301' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-10T00:53:52.611 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:52 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:53:52.342+0000 7f4b1d582200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-10T00:53:52.616 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-06-10T00:53:52.616 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-06-10T00:53:52.616 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "bded58f8-26c3-11ef-bc9b-c7b262605968", 2024-06-10T00:53:52.616 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-06-10T00:53:52.616 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-10T00:53:52.616 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-10T00:53:52.616 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-10T00:53:52.617 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:53:52.617 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-10T00:53:52.617 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-10T00:53:52.617 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-06-10T00:53:52.617 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-06-10T00:53:52.617 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-10T00:53:52.617 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "a" 2024-06-10T00:53:52.617 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-06-10T00:53:52.617 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-06-10T00:53:52.618 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-10T00:53:52.618 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:53:52.618 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-10T00:53:52.618 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-10T00:53:52.618 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:53:52.618 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-10T00:53:52.619 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:53:52.619 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-10T00:53:52.619 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-10T00:53:52.619 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-10T00:53:52.634 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-10T00:53:52.634 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-10T00:53:52.634 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-10T00:53:52.635 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:53:52.635 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-10T00:53:52.635 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-10T00:53:52.635 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-10T00:53:52.635 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-10T00:53:52.635 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-10T00:53:52.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-10T00:53:52.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-10T00:53:52.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-10T00:53:52.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-10T00:53:52.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:53:52.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-10T00:53:52.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:53:52.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-10T00:53:52.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-10T00:53:52.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:53:52.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-10T00:53:52.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-10T00:53:52.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-10T00:53:52.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-10T00:53:52.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-10T00:53:52.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-10T00:53:52.639 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-06-10T00:53:52.639 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-06-10T00:53:52.639 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-10T00:53:52.639 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:53:52.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-10T00:53:52.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:53:52.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-06-10T00:53:31.966769+0000", 2024-06-10T00:53:52.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-10T00:53:52.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:53:52.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-10T00:53:52.641 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-06-10T00:53:52.641 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.917+0000 7f32f316a700 1 Processor -- start 2024-06-10T00:53:52.641 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.917+0000 7f32f316a700 1 -- start start 2024-06-10T00:53:52.641 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.918+0000 7f32f316a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32ec1089a0 0x7f32ec10ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:52.641 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.918+0000 7f32f316a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec07c6a0 con 0x7f32ec1089a0 2024-06-10T00:53:52.642 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.918+0000 7f32f0f06700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32ec1089a0 0x7f32ec10ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:52.642 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.918+0000 7f32f0f06700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32ec1089a0 0x7f32ec10ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59544/0 (socket says 172.21.15.7:59544) 2024-06-10T00:53:52.642 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.918+0000 7f32f0f06700 1 -- 172.21.15.7:0/3665555636 learned_addr learned my addr 172.21.15.7:0/3665555636 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:53:52.642 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.918+0000 7f32f0f06700 1 -- 172.21.15.7:0/3665555636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ec10b2d0 con 0x7f32ec1089a0 2024-06-10T00:53:52.642 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.918+0000 7f32f0f06700 1 --2- 172.21.15.7:0/3665555636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32ec1089a0 0x7f32ec10ad90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f32e00098a0 tx=0x7f32e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e58c903f49a1f8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:52.643 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.918+0000 7f32eb7fe700 1 -- 172.21.15.7:0/3665555636 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32e0004160 con 0x7f32ec1089a0 2024-06-10T00:53:52.643 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.919+0000 7f32eb7fe700 1 -- 172.21.15.7:0/3665555636 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f32e0004300 con 0x7f32ec1089a0 2024-06-10T00:53:52.643 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.919+0000 7f32eb7fe700 1 -- 172.21.15.7:0/3665555636 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32e0011b50 con 0x7f32ec1089a0 2024-06-10T00:53:52.643 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.919+0000 7f32f316a700 1 -- 172.21.15.7:0/3665555636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32ec1089a0 msgr2=0x7f32ec10ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:52.643 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.919+0000 7f32f316a700 1 --2- 172.21.15.7:0/3665555636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32ec1089a0 0x7f32ec10ad90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f32e00098a0 tx=0x7f32e0009bb0 comp rx=0 tx=0).stop 2024-06-10T00:53:52.643 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.919+0000 7f32f316a700 1 -- 172.21.15.7:0/3665555636 shutdown_connections 2024-06-10T00:53:52.644 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.919+0000 7f32f316a700 1 --2- 172.21.15.7:0/3665555636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32ec1089a0 0x7f32ec10ad90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:52.644 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.919+0000 7f32f316a700 1 -- 172.21.15.7:0/3665555636 >> 172.21.15.7:0/3665555636 conn(0x7f32ec102370 msgr2=0x7f32ec1047b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:52.644 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.919+0000 7f32f316a700 1 -- 172.21.15.7:0/3665555636 shutdown_connections 2024-06-10T00:53:52.644 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.919+0000 7f32f316a700 1 -- 172.21.15.7:0/3665555636 wait complete. 2024-06-10T00:53:52.644 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.920+0000 7f32f316a700 1 Processor -- start 2024-06-10T00:53:52.645 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.920+0000 7f32f316a700 1 -- start start 2024-06-10T00:53:52.645 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.920+0000 7f32f316a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32ec1089a0 0x7f32ec19bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:52.645 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.920+0000 7f32f316a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec10c220 con 0x7f32ec1089a0 2024-06-10T00:53:52.645 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.920+0000 7f32f0f06700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32ec1089a0 0x7f32ec19bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:52.645 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.920+0000 7f32f0f06700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32ec1089a0 0x7f32ec19bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59556/0 (socket says 172.21.15.7:59556) 2024-06-10T00:53:52.645 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.920+0000 7f32f0f06700 1 -- 172.21.15.7:0/111543301 learned_addr learned my addr 172.21.15.7:0/111543301 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:53:52.646 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.920+0000 7f32f0f06700 1 -- 172.21.15.7:0/111543301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ec19c290 con 0x7f32ec1089a0 2024-06-10T00:53:52.646 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.921+0000 7f32f0f06700 1 --2- 172.21.15.7:0/111543301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32ec1089a0 0x7f32ec19bd50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f32e0011670 tx=0x7f32e0011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:52.646 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.921+0000 7f32e9ffb700 1 -- 172.21.15.7:0/111543301 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32e00039c0 con 0x7f32ec1089a0 2024-06-10T00:53:52.646 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.921+0000 7f32e9ffb700 1 -- 172.21.15.7:0/111543301 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f32e0011e50 con 0x7f32ec1089a0 2024-06-10T00:53:52.646 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.921+0000 7f32e9ffb700 1 -- 172.21.15.7:0/111543301 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32e001a5c0 con 0x7f32ec1089a0 2024-06-10T00:53:52.647 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.921+0000 7f32f316a700 1 -- 172.21.15.7:0/111543301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32ec19c520 con 0x7f32ec1089a0 2024-06-10T00:53:52.647 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.921+0000 7f32f316a700 1 -- 172.21.15.7:0/111543301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32ec19f210 con 0x7f32ec1089a0 2024-06-10T00:53:52.647 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.921+0000 7f32e9ffb700 1 -- 172.21.15.7:0/111543301 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f32e001ac00 con 0x7f32ec1089a0 2024-06-10T00:53:52.647 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.921+0000 7f32e9ffb700 1 -- 172.21.15.7:0/111543301 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f32e0019a90 con 0x7f32ec1089a0 2024-06-10T00:53:52.647 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.922+0000 7f32f316a700 1 -- 172.21.15.7:0/111543301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32ec063af0 con 0x7f32ec1089a0 2024-06-10T00:53:52.647 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.924+0000 7f32e9ffb700 1 -- 172.21.15.7:0/111543301 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f32e0003b60 con 0x7f32ec1089a0 2024-06-10T00:53:52.648 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.999+0000 7f32f316a700 1 -- 172.21.15.7:0/111543301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f32ec19f7c0 con 0x7f32ec1089a0 2024-06-10T00:53:52.648 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:51.999+0000 7f32e9ffb700 1 -- 172.21.15.7:0/111543301 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f32e0019d30 con 0x7f32ec1089a0 2024-06-10T00:53:52.648 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:52.000+0000 7f32f316a700 1 -- 172.21.15.7:0/111543301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32ec1089a0 msgr2=0x7f32ec19bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:52.648 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:52.000+0000 7f32f316a700 1 --2- 172.21.15.7:0/111543301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32ec1089a0 0x7f32ec19bd50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f32e0011670 tx=0x7f32e0011b60 comp rx=0 tx=0).stop 2024-06-10T00:53:52.648 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:52.000+0000 7f32f316a700 1 -- 172.21.15.7:0/111543301 shutdown_connections 2024-06-10T00:53:52.648 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:52.000+0000 7f32f316a700 1 --2- 172.21.15.7:0/111543301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32ec1089a0 0x7f32ec19bd50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:52.649 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:52.000+0000 7f32f316a700 1 -- 172.21.15.7:0/111543301 >> 172.21.15.7:0/111543301 conn(0x7f32ec102370 msgr2=0x7f32ec102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:52.649 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:52.001+0000 7f32f316a700 1 -- 172.21.15.7:0/111543301 shutdown_connections 2024-06-10T00:53:52.649 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:52.001+0000 7f32f316a700 1 -- 172.21.15.7:0/111543301 wait complete. 2024-06-10T00:53:52.649 INFO:teuthology.orchestra.run.smithi007.stdout:mgr not available, waiting (2/15)... 2024-06-10T00:53:52.884 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:52 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:53:52.783+0000 7f4b1d582200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-10T00:53:52.884 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:52 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:53:52.884+0000 7f4b1d582200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-10T00:53:53.268 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:53 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:53:53.093+0000 7f4b1d582200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-10T00:53:53.992 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:53 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:53:53.640+0000 7f4b1d582200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-10T00:53:53.992 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:53 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:53:53.735+0000 7f4b1d582200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-10T00:53:54.268 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:53 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:53:53.991+0000 7f4b1d582200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-10T00:53:54.269 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:54 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:53:54.131+0000 7f4b1d582200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-10T00:53:54.269 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:54 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:53:54.230+0000 7f4b1d582200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-10T00:53:54.674 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:54 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:53:54.351+0000 7f4b1d582200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-10T00:53:54.674 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:54 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:53:54.673+0000 7f4b1d582200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-10T00:53:55.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:55 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/185744420' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-10T00:53:55.751 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:55 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:53:55.319+0000 7f4b1d582200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-10T00:53:55.751 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:53:55 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:53:55.550+0000 7f4b1d582200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-10T00:53:55.756 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-06-10T00:53:55.756 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-06-10T00:53:55.756 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "bded58f8-26c3-11ef-bc9b-c7b262605968", 2024-06-10T00:53:55.756 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-06-10T00:53:55.756 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-10T00:53:55.756 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-10T00:53:55.756 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-10T00:53:55.756 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:53:55.756 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-10T00:53:55.756 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-10T00:53:55.757 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-06-10T00:53:55.757 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-06-10T00:53:55.757 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-10T00:53:55.757 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "a" 2024-06-10T00:53:55.757 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-06-10T00:53:55.757 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-06-10T00:53:55.757 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-10T00:53:55.757 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:53:55.757 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-10T00:53:55.757 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-10T00:53:55.757 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:53:55.757 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-10T00:53:55.757 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:53:55.757 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-10T00:53:55.758 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-10T00:53:55.758 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-10T00:53:55.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-10T00:53:55.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-10T00:53:55.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-10T00:53:55.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:53:55.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-10T00:53:55.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-10T00:53:55.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-10T00:53:55.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-10T00:53:55.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-10T00:53:55.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-10T00:53:55.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-10T00:53:55.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-10T00:53:55.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-10T00:53:55.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:53:55.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-10T00:53:55.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:53:55.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-10T00:53:55.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-10T00:53:55.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:53:55.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-10T00:53:55.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-10T00:53:55.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-10T00:53:55.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-10T00:53:55.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-10T00:53:55.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-10T00:53:55.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-06-10T00:53:55.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-06-10T00:53:55.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-10T00:53:55.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:53:55.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-10T00:53:55.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:53:55.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-06-10T00:53:31.966769+0000", 2024-06-10T00:53:55.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-10T00:53:55.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:53:55.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-10T00:53:55.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-06-10T00:53:55.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.058+0000 7fd5cdaec700 1 Processor -- start 2024-06-10T00:53:55.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.058+0000 7fd5cdaec700 1 -- start start 2024-06-10T00:53:55.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.058+0000 7fd5cdaec700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5c8106dd0 0x7fd5c81071f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:55.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.058+0000 7fd5cdaec700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c81077c0 con 0x7fd5c8106dd0 2024-06-10T00:53:55.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.058+0000 7fd5c77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5c8106dd0 0x7fd5c81071f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:55.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.058+0000 7fd5c77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5c8106dd0 0x7fd5c81071f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59562/0 (socket says 172.21.15.7:59562) 2024-06-10T00:53:55.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.058+0000 7fd5c77fe700 1 -- 172.21.15.7:0/1646492484 learned_addr learned my addr 172.21.15.7:0/1646492484 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:53:55.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.059+0000 7fd5c77fe700 1 -- 172.21.15.7:0/1646492484 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c8107f80 con 0x7fd5c8106dd0 2024-06-10T00:53:55.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.059+0000 7fd5c77fe700 1 --2- 172.21.15.7:0/1646492484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5c8106dd0 0x7fd5c81071f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd5b00098a0 tx=0x7fd5b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaee53ec5dd90463 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:55.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.059+0000 7fd5c67fc700 1 -- 172.21.15.7:0/1646492484 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5b0004160 con 0x7fd5c8106dd0 2024-06-10T00:53:55.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.059+0000 7fd5c67fc700 1 -- 172.21.15.7:0/1646492484 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd5b0004300 con 0x7fd5c8106dd0 2024-06-10T00:53:55.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.059+0000 7fd5c67fc700 1 -- 172.21.15.7:0/1646492484 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5b0011b50 con 0x7fd5c8106dd0 2024-06-10T00:53:55.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.060+0000 7fd5cdaec700 1 -- 172.21.15.7:0/1646492484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5c8106dd0 msgr2=0x7fd5c81071f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:55.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.060+0000 7fd5cdaec700 1 --2- 172.21.15.7:0/1646492484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5c8106dd0 0x7fd5c81071f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd5b00098a0 tx=0x7fd5b0009bb0 comp rx=0 tx=0).stop 2024-06-10T00:53:55.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.060+0000 7fd5cdaec700 1 -- 172.21.15.7:0/1646492484 shutdown_connections 2024-06-10T00:53:55.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.060+0000 7fd5cdaec700 1 --2- 172.21.15.7:0/1646492484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5c8106dd0 0x7fd5c81071f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:55.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.060+0000 7fd5cdaec700 1 -- 172.21.15.7:0/1646492484 >> 172.21.15.7:0/1646492484 conn(0x7fd5c8102370 msgr2=0x7fd5c81047b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:55.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.060+0000 7fd5cdaec700 1 -- 172.21.15.7:0/1646492484 shutdown_connections 2024-06-10T00:53:55.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.060+0000 7fd5cdaec700 1 -- 172.21.15.7:0/1646492484 wait complete. 2024-06-10T00:53:55.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.060+0000 7fd5cdaec700 1 Processor -- start 2024-06-10T00:53:55.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.061+0000 7fd5cdaec700 1 -- start start 2024-06-10T00:53:55.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.061+0000 7fd5cdaec700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5c8106dd0 0x7fd5c819bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:55.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.061+0000 7fd5cdaec700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c810b330 con 0x7fd5c8106dd0 2024-06-10T00:53:55.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.061+0000 7fd5c77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5c8106dd0 0x7fd5c819bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:55.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.061+0000 7fd5c77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5c8106dd0 0x7fd5c819bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59578/0 (socket says 172.21.15.7:59578) 2024-06-10T00:53:55.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.061+0000 7fd5c77fe700 1 -- 172.21.15.7:0/185744420 learned_addr learned my addr 172.21.15.7:0/185744420 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:53:55.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.061+0000 7fd5c77fe700 1 -- 172.21.15.7:0/185744420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c819c320 con 0x7fd5c8106dd0 2024-06-10T00:53:55.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.061+0000 7fd5c77fe700 1 --2- 172.21.15.7:0/185744420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5c8106dd0 0x7fd5c819bde0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0011670 tx=0x7fd5b0011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:55.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.061+0000 7fd5c4ff9700 1 -- 172.21.15.7:0/185744420 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5b00039c0 con 0x7fd5c8106dd0 2024-06-10T00:53:55.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.062+0000 7fd5c4ff9700 1 -- 172.21.15.7:0/185744420 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd5b0011e50 con 0x7fd5c8106dd0 2024-06-10T00:53:55.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.062+0000 7fd5c4ff9700 1 -- 172.21.15.7:0/185744420 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5b001a5c0 con 0x7fd5c8106dd0 2024-06-10T00:53:55.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.062+0000 7fd5cdaec700 1 -- 172.21.15.7:0/185744420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c819c5b0 con 0x7fd5c8106dd0 2024-06-10T00:53:55.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.062+0000 7fd5cdaec700 1 -- 172.21.15.7:0/185744420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c819f2a0 con 0x7fd5c8106dd0 2024-06-10T00:53:55.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.062+0000 7fd5c4ff9700 1 -- 172.21.15.7:0/185744420 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd5b001ac00 con 0x7fd5c8106dd0 2024-06-10T00:53:55.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.062+0000 7fd5c4ff9700 1 -- 172.21.15.7:0/185744420 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd5b0019a90 con 0x7fd5c8106dd0 2024-06-10T00:53:55.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.063+0000 7fd5cdaec700 1 -- 172.21.15.7:0/185744420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5c8063af0 con 0x7fd5c8106dd0 2024-06-10T00:53:55.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.065+0000 7fd5c4ff9700 1 -- 172.21.15.7:0/185744420 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fd5b0003b60 con 0x7fd5c8106dd0 2024-06-10T00:53:55.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.138+0000 7fd5cdaec700 1 -- 172.21.15.7:0/185744420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd5c80008d0 con 0x7fd5c8106dd0 2024-06-10T00:53:55.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.139+0000 7fd5c4ff9700 1 -- 172.21.15.7:0/185744420 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fd5b0019d30 con 0x7fd5c8106dd0 2024-06-10T00:53:55.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.139+0000 7fd5cdaec700 1 -- 172.21.15.7:0/185744420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5c8106dd0 msgr2=0x7fd5c819bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:55.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.139+0000 7fd5cdaec700 1 --2- 172.21.15.7:0/185744420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5c8106dd0 0x7fd5c819bde0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0011670 tx=0x7fd5b0011b60 comp rx=0 tx=0).stop 2024-06-10T00:53:55.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.139+0000 7fd5cdaec700 1 -- 172.21.15.7:0/185744420 shutdown_connections 2024-06-10T00:53:55.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.139+0000 7fd5cdaec700 1 --2- 172.21.15.7:0/185744420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5c8106dd0 0x7fd5c819bde0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:55.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.139+0000 7fd5cdaec700 1 -- 172.21.15.7:0/185744420 >> 172.21.15.7:0/185744420 conn(0x7fd5c8102370 msgr2=0x7fd5c8103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:55.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.140+0000 7fd5cdaec700 1 -- 172.21.15.7:0/185744420 shutdown_connections 2024-06-10T00:53:55.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:55.140+0000 7fd5cdaec700 1 -- 172.21.15.7:0/185744420 wait complete. 2024-06-10T00:53:55.771 INFO:teuthology.orchestra.run.smithi007.stdout:mgr not available, waiting (3/15)... 2024-06-10T00:53:56.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:56 smithi007 ceph-mon[32398]: Activating manager daemon x 2024-06-10T00:53:56.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:56 smithi007 ceph-mon[32398]: mgrmap e2: x(active, starting, since 0.00316506s) 2024-06-10T00:53:56.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:56 smithi007 ceph-mon[32398]: from='mgr.14100 172.21.15.7:0/3642826195' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-10T00:53:56.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:56 smithi007 ceph-mon[32398]: from='mgr.14100 172.21.15.7:0/3642826195' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-10T00:53:56.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:56 smithi007 ceph-mon[32398]: from='mgr.14100 172.21.15.7:0/3642826195' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-10T00:53:56.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:56 smithi007 ceph-mon[32398]: from='mgr.14100 172.21.15.7:0/3642826195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:53:56.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:56 smithi007 ceph-mon[32398]: from='mgr.14100 172.21.15.7:0/3642826195' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-10T00:53:56.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:56 smithi007 ceph-mon[32398]: Manager daemon x is now available 2024-06-10T00:53:56.520 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:56 smithi007 ceph-mon[32398]: from='mgr.14100 172.21.15.7:0/3642826195' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-10T00:53:56.520 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:56 smithi007 ceph-mon[32398]: from='mgr.14100 172.21.15.7:0/3642826195' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-10T00:53:56.520 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:56 smithi007 ceph-mon[32398]: from='mgr.14100 172.21.15.7:0/3642826195' entity='mgr.x' 2024-06-10T00:53:56.520 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:56 smithi007 ceph-mon[32398]: from='mgr.14100 172.21.15.7:0/3642826195' entity='mgr.x' 2024-06-10T00:53:56.520 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:56 smithi007 ceph-mon[32398]: from='mgr.14100 172.21.15.7:0/3642826195' entity='mgr.x' 2024-06-10T00:53:57.888 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:57 smithi007 ceph-mon[32398]: mgrmap e3: x(active, since 1.00703s) 2024-06-10T00:53:58.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:58 smithi007 ceph-mon[32398]: mgrmap e4: x(active, since 2s) 2024-06-10T00:53:58.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:58 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3096445814' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-10T00:53:59.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-06-10T00:53:59.003 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-06-10T00:53:59.003 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "bded58f8-26c3-11ef-bc9b-c7b262605968", 2024-06-10T00:53:59.003 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-06-10T00:53:59.004 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-10T00:53:59.004 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-10T00:53:59.004 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-10T00:53:59.004 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:53:59.004 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-10T00:53:59.004 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-10T00:53:59.004 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-06-10T00:53:59.004 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-06-10T00:53:59.004 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-10T00:53:59.004 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "a" 2024-06-10T00:53:59.004 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-06-10T00:53:59.004 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-06-10T00:53:59.004 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-10T00:53:59.005 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:53:59.005 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-10T00:53:59.005 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-10T00:53:59.005 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:53:59.005 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-10T00:53:59.005 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:53:59.005 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-10T00:53:59.005 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-10T00:53:59.005 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-10T00:53:59.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-10T00:53:59.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-10T00:53:59.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-10T00:53:59.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:53:59.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-10T00:53:59.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-10T00:53:59.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-10T00:53:59.014 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-10T00:53:59.014 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-10T00:53:59.014 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-10T00:53:59.014 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-10T00:53:59.014 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-10T00:53:59.014 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-10T00:53:59.014 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:53:59.014 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-10T00:53:59.014 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:53:59.014 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-10T00:53:59.014 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-10T00:53:59.014 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:53:59.014 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-10T00:53:59.014 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-10T00:53:59.015 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-10T00:53:59.015 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-10T00:53:59.015 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-10T00:53:59.015 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-10T00:53:59.015 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-06-10T00:53:59.015 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-06-10T00:53:59.015 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-10T00:53:59.015 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:53:59.015 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-10T00:53:59.015 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:53:59.015 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-06-10T00:53:31.966769+0000", 2024-06-10T00:53:59.015 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-10T00:53:59.015 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:53:59.015 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-10T00:53:59.016 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-06-10T00:53:59.016 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.156+0000 7fd573b28700 1 Processor -- start 2024-06-10T00:53:59.016 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.156+0000 7fd573b28700 1 -- start start 2024-06-10T00:53:59.016 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.156+0000 7fd573b28700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd56c07def0 0x7fd56c07c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:59.016 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.156+0000 7fd573b28700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c07c920 con 0x7fd56c07def0 2024-06-10T00:53:59.016 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.156+0000 7fd5718c4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd56c07def0 0x7fd56c07c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:59.016 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.156+0000 7fd5718c4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd56c07def0 0x7fd56c07c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59620/0 (socket says 172.21.15.7:59620) 2024-06-10T00:53:59.016 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.156+0000 7fd5718c4700 1 -- 172.21.15.7:0/1832625644 learned_addr learned my addr 172.21.15.7:0/1832625644 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:53:59.016 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.157+0000 7fd5718c4700 1 -- 172.21.15.7:0/1832625644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd56c07caa0 con 0x7fd56c07def0 2024-06-10T00:53:59.016 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.157+0000 7fd5718c4700 1 --2- 172.21.15.7:0/1832625644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd56c07def0 0x7fd56c07c350 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd568009e80 tx=0x7fd56800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223160c4a6cb2cac server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:59.016 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.157+0000 7fd5708c2700 1 -- 172.21.15.7:0/1832625644 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd568017070 con 0x7fd56c07def0 2024-06-10T00:53:59.016 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.157+0000 7fd5708c2700 1 -- 172.21.15.7:0/1832625644 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd56800b8c0 con 0x7fd56c07def0 2024-06-10T00:53:59.017 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.157+0000 7fd5708c2700 1 -- 172.21.15.7:0/1832625644 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5680045c0 con 0x7fd56c07def0 2024-06-10T00:53:59.017 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.158+0000 7fd573b28700 1 -- 172.21.15.7:0/1832625644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd56c07def0 msgr2=0x7fd56c07c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:59.017 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.158+0000 7fd573b28700 1 --2- 172.21.15.7:0/1832625644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd56c07def0 0x7fd56c07c350 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd568009e80 tx=0x7fd56800b330 comp rx=0 tx=0).stop 2024-06-10T00:53:59.017 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.158+0000 7fd573b28700 1 -- 172.21.15.7:0/1832625644 shutdown_connections 2024-06-10T00:53:59.017 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.158+0000 7fd573b28700 1 --2- 172.21.15.7:0/1832625644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd56c07def0 0x7fd56c07c350 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:59.017 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.158+0000 7fd573b28700 1 -- 172.21.15.7:0/1832625644 >> 172.21.15.7:0/1832625644 conn(0x7fd56c103480 msgr2=0x7fd56c1058a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:59.017 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.158+0000 7fd573b28700 1 -- 172.21.15.7:0/1832625644 shutdown_connections 2024-06-10T00:53:59.017 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.158+0000 7fd573b28700 1 -- 172.21.15.7:0/1832625644 wait complete. 2024-06-10T00:53:59.017 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.159+0000 7fd573b28700 1 Processor -- start 2024-06-10T00:53:59.017 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.159+0000 7fd573b28700 1 -- start start 2024-06-10T00:53:59.017 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.159+0000 7fd573b28700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd56c07def0 0x7fd56c1a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:59.017 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.159+0000 7fd573b28700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c10a070 con 0x7fd56c07def0 2024-06-10T00:53:59.017 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.160+0000 7fd5718c4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd56c07def0 0x7fd56c1a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:59.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.160+0000 7fd5718c4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd56c07def0 0x7fd56c1a23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59630/0 (socket says 172.21.15.7:59630) 2024-06-10T00:53:59.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.160+0000 7fd5718c4700 1 -- 172.21.15.7:0/3096445814 learned_addr learned my addr 172.21.15.7:0/3096445814 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:53:59.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.160+0000 7fd5718c4700 1 -- 172.21.15.7:0/3096445814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd56c1a28f0 con 0x7fd56c07def0 2024-06-10T00:53:59.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.160+0000 7fd5718c4700 1 --2- 172.21.15.7:0/3096445814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd56c07def0 0x7fd56c1a23b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd568009530 tx=0x7fd56800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:59.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.160+0000 7fd562ffd700 1 -- 172.21.15.7:0/3096445814 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd568020020 con 0x7fd56c07def0 2024-06-10T00:53:59.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.160+0000 7fd573b28700 1 -- 172.21.15.7:0/3096445814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd56c1a2b80 con 0x7fd56c07def0 2024-06-10T00:53:59.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.160+0000 7fd573b28700 1 -- 172.21.15.7:0/3096445814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd56c1a5870 con 0x7fd56c07def0 2024-06-10T00:53:59.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.160+0000 7fd562ffd700 1 -- 172.21.15.7:0/3096445814 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd568003b50 con 0x7fd56c07def0 2024-06-10T00:53:59.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.160+0000 7fd562ffd700 1 -- 172.21.15.7:0/3096445814 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd568017040 con 0x7fd56c07def0 2024-06-10T00:53:59.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.161+0000 7fd562ffd700 1 -- 172.21.15.7:0/3096445814 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fd568023020 con 0x7fd56c07def0 2024-06-10T00:53:59.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.161+0000 7fd573b28700 1 -- 172.21.15.7:0/3096445814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd550005360 con 0x7fd56c07def0 2024-06-10T00:53:59.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.162+0000 7fd562ffd700 1 --2- 172.21.15.7:0/3096445814 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7fd558039050 0x7fd55803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:59.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.162+0000 7fd562ffd700 1 -- 172.21.15.7:0/3096445814 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd56800f050 con 0x7fd56c07def0 2024-06-10T00:53:59.019 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.162+0000 7fd5710c3700 1 --2- 172.21.15.7:0/3096445814 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7fd558039050 0x7fd55803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:59.019 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.162+0000 7fd5710c3700 1 --2- 172.21.15.7:0/3096445814 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7fd558039050 0x7fd55803b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd55c006fd0 tx=0x7fd55c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:59.019 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.167+0000 7fd562ffd700 1 -- 172.21.15.7:0/3096445814 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd56802a020 con 0x7fd56c07def0 2024-06-10T00:53:59.019 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.459+0000 7fd573b28700 1 -- 172.21.15.7:0/3096445814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd550005650 con 0x7fd56c07def0 2024-06-10T00:53:59.019 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.460+0000 7fd562ffd700 1 -- 172.21.15.7:0/3096445814 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fd56804c940 con 0x7fd56c07def0 2024-06-10T00:53:59.019 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.462+0000 7fd573b28700 1 -- 172.21.15.7:0/3096445814 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7fd558039050 msgr2=0x7fd55803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:59.019 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.462+0000 7fd573b28700 1 --2- 172.21.15.7:0/3096445814 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7fd558039050 0x7fd55803b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd55c006fd0 tx=0x7fd55c006e40 comp rx=0 tx=0).stop 2024-06-10T00:53:59.019 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.462+0000 7fd573b28700 1 -- 172.21.15.7:0/3096445814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd56c07def0 msgr2=0x7fd56c1a23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:59.019 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.462+0000 7fd573b28700 1 --2- 172.21.15.7:0/3096445814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd56c07def0 0x7fd56c1a23b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd568009530 tx=0x7fd56800bd50 comp rx=0 tx=0).stop 2024-06-10T00:53:59.019 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.462+0000 7fd573b28700 1 -- 172.21.15.7:0/3096445814 shutdown_connections 2024-06-10T00:53:59.019 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.462+0000 7fd573b28700 1 --2- 172.21.15.7:0/3096445814 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7fd558039050 0x7fd55803b510 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:59.019 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.463+0000 7fd573b28700 1 --2- 172.21.15.7:0/3096445814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd56c07def0 0x7fd56c1a23b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:59.019 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.463+0000 7fd573b28700 1 -- 172.21.15.7:0/3096445814 >> 172.21.15.7:0/3096445814 conn(0x7fd56c103480 msgr2=0x7fd56c105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:59.020 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.463+0000 7fd573b28700 1 -- 172.21.15.7:0/3096445814 shutdown_connections 2024-06-10T00:53:59.020 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:58.463+0000 7fd573b28700 1 -- 172.21.15.7:0/3096445814 wait complete. 2024-06-10T00:53:59.020 INFO:teuthology.orchestra.run.smithi007.stdout:mgr is available 2024-06-10T00:54:00.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:53:59 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3916743161' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-10T00:54:00.205 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-06-10T00:54:00.205 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [global] 2024-06-10T00:54:00.205 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout fsid = bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T00:54:00.205 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-10T00:54:00.206 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-10T00:54:00.206 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-10T00:54:00.206 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-10T00:54:00.206 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-06-10T00:54:00.206 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-10T00:54:00.206 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-10T00:54:00.206 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-06-10T00:54:00.206 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [osd] 2024-06-10T00:54:00.206 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-10T00:54:00.206 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-10T00:54:00.206 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-10T00:54:00.206 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-10T00:54:00.206 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-10T00:54:00.206 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.430+0000 7f37adf22700 1 Processor -- start 2024-06-10T00:54:00.206 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.430+0000 7f37adf22700 1 -- start start 2024-06-10T00:54:00.207 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.430+0000 7f37adf22700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37a8108950 0x7f37a810ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:00.207 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.430+0000 7f37adf22700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f37a807bea0 con 0x7f37a8108950 2024-06-10T00:54:00.207 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.430+0000 7f37a77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37a8108950 0x7f37a810ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:00.207 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.430+0000 7f37a77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37a8108950 0x7f37a810ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59632/0 (socket says 172.21.15.7:59632) 2024-06-10T00:54:00.207 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.430+0000 7f37a77fe700 1 -- 172.21.15.7:0/4115337409 learned_addr learned my addr 172.21.15.7:0/4115337409 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:00.207 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.430+0000 7f37a77fe700 1 -- 172.21.15.7:0/4115337409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a810b280 con 0x7f37a8108950 2024-06-10T00:54:00.207 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.431+0000 7f37a77fe700 1 --2- 172.21.15.7:0/4115337409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37a8108950 0x7f37a810ad40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f37980098a0 tx=0x7f3798009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=618f275f7488aac6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:00.207 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.431+0000 7f37a67fc700 1 -- 172.21.15.7:0/4115337409 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3798004160 con 0x7f37a8108950 2024-06-10T00:54:00.207 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.431+0000 7f37a67fc700 1 -- 172.21.15.7:0/4115337409 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3798004300 con 0x7f37a8108950 2024-06-10T00:54:00.207 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.431+0000 7f37a67fc700 1 -- 172.21.15.7:0/4115337409 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3798011b50 con 0x7f37a8108950 2024-06-10T00:54:00.207 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.431+0000 7f37adf22700 1 -- 172.21.15.7:0/4115337409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37a8108950 msgr2=0x7f37a810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:00.207 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.431+0000 7f37adf22700 1 --2- 172.21.15.7:0/4115337409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37a8108950 0x7f37a810ad40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f37980098a0 tx=0x7f3798009bb0 comp rx=0 tx=0).stop 2024-06-10T00:54:00.207 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.431+0000 7f37adf22700 1 -- 172.21.15.7:0/4115337409 shutdown_connections 2024-06-10T00:54:00.207 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.431+0000 7f37adf22700 1 --2- 172.21.15.7:0/4115337409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37a8108950 0x7f37a810ad40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:00.208 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.431+0000 7f37adf22700 1 -- 172.21.15.7:0/4115337409 >> 172.21.15.7:0/4115337409 conn(0x7f37a8102340 msgr2=0x7f37a81047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:00.208 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.431+0000 7f37adf22700 1 -- 172.21.15.7:0/4115337409 shutdown_connections 2024-06-10T00:54:00.208 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.431+0000 7f37adf22700 1 -- 172.21.15.7:0/4115337409 wait complete. 2024-06-10T00:54:00.208 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.432+0000 7f37adf22700 1 Processor -- start 2024-06-10T00:54:00.208 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.432+0000 7f37adf22700 1 -- start start 2024-06-10T00:54:00.208 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.432+0000 7f37adf22700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37a8108950 0x7f37a819bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:00.208 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.432+0000 7f37adf22700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f37a810c1c0 con 0x7f37a8108950 2024-06-10T00:54:00.208 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.433+0000 7f37a77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37a8108950 0x7f37a819bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:00.208 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.433+0000 7f37a77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37a8108950 0x7f37a819bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59648/0 (socket says 172.21.15.7:59648) 2024-06-10T00:54:00.208 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.433+0000 7f37a77fe700 1 -- 172.21.15.7:0/3916743161 learned_addr learned my addr 172.21.15.7:0/3916743161 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:00.208 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.433+0000 7f37a77fe700 1 -- 172.21.15.7:0/3916743161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a819c240 con 0x7f37a8108950 2024-06-10T00:54:00.208 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.433+0000 7f37a77fe700 1 --2- 172.21.15.7:0/3916743161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37a8108950 0x7f37a819bd00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3798011670 tx=0x7f3798011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:00.208 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.433+0000 7f37a4ff9700 1 -- 172.21.15.7:0/3916743161 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37980039c0 con 0x7f37a8108950 2024-06-10T00:54:00.208 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.433+0000 7f37a4ff9700 1 -- 172.21.15.7:0/3916743161 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3798011e50 con 0x7f37a8108950 2024-06-10T00:54:00.208 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.433+0000 7f37adf22700 1 -- 172.21.15.7:0/3916743161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37a819c530 con 0x7f37a8108950 2024-06-10T00:54:00.209 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.433+0000 7f37a4ff9700 1 -- 172.21.15.7:0/3916743161 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f379801a5c0 con 0x7f37a8108950 2024-06-10T00:54:00.209 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.433+0000 7f37adf22700 1 -- 172.21.15.7:0/3916743161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37a819f1b0 con 0x7f37a8108950 2024-06-10T00:54:00.209 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.434+0000 7f37a4ff9700 1 -- 172.21.15.7:0/3916743161 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f379801a870 con 0x7f37a8108950 2024-06-10T00:54:00.209 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.434+0000 7f37a4ff9700 1 --2- 172.21.15.7:0/3916743161 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7f3790039000 0x7f379003b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:00.209 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.434+0000 7f37adf22700 1 -- 172.21.15.7:0/3916743161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37a8063af0 con 0x7f37a8108950 2024-06-10T00:54:00.209 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.434+0000 7f37a4ff9700 1 -- 172.21.15.7:0/3916743161 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f379804d2a0 con 0x7f37a8108950 2024-06-10T00:54:00.209 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.434+0000 7f37a6ffd700 1 --2- 172.21.15.7:0/3916743161 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7f3790039000 0x7f379003b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:00.209 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.435+0000 7f37a6ffd700 1 --2- 172.21.15.7:0/3916743161 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7f3790039000 0x7f379003b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f379c006fd0 tx=0x7f379c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:00.209 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.440+0000 7f37a4ff9700 1 -- 172.21.15.7:0/3916743161 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3798019c70 con 0x7f37a8108950 2024-06-10T00:54:00.209 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.647+0000 7f37adf22700 1 -- 172.21.15.7:0/3916743161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f37a819f380 con 0x7f37a8108950 2024-06-10T00:54:00.209 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.648+0000 7f37a4ff9700 1 -- 172.21.15.7:0/3916743161 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7f379802d910 con 0x7f37a8108950 2024-06-10T00:54:00.209 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.650+0000 7f37adf22700 1 -- 172.21.15.7:0/3916743161 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7f3790039000 msgr2=0x7f379003b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:00.209 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.650+0000 7f37adf22700 1 --2- 172.21.15.7:0/3916743161 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7f3790039000 0x7f379003b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f379c006fd0 tx=0x7f379c006e40 comp rx=0 tx=0).stop 2024-06-10T00:54:00.209 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.650+0000 7f37adf22700 1 -- 172.21.15.7:0/3916743161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37a8108950 msgr2=0x7f37a819bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:00.209 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.650+0000 7f37adf22700 1 --2- 172.21.15.7:0/3916743161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37a8108950 0x7f37a819bd00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3798011670 tx=0x7f3798011b60 comp rx=0 tx=0).stop 2024-06-10T00:54:00.210 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.651+0000 7f37adf22700 1 -- 172.21.15.7:0/3916743161 shutdown_connections 2024-06-10T00:54:00.210 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.651+0000 7f37adf22700 1 --2- 172.21.15.7:0/3916743161 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7f3790039000 0x7f379003b4c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:00.210 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.651+0000 7f37adf22700 1 --2- 172.21.15.7:0/3916743161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37a8108950 0x7f37a819bd00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:00.210 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.651+0000 7f37adf22700 1 -- 172.21.15.7:0/3916743161 >> 172.21.15.7:0/3916743161 conn(0x7f37a8102340 msgr2=0x7f37a8102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:00.210 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.651+0000 7f37adf22700 1 -- 172.21.15.7:0/3916743161 shutdown_connections 2024-06-10T00:54:00.210 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:53:59.651+0000 7f37adf22700 1 -- 172.21.15.7:0/3916743161 wait complete. 2024-06-10T00:54:00.210 INFO:teuthology.orchestra.run.smithi007.stdout:Enabling cephadm module... 2024-06-10T00:54:01.236 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:00 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3254678527' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-06-10T00:54:01.236 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:00 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: ignoring --setuser ceph since I am not root 2024-06-10T00:54:01.236 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:00 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: ignoring --setgroup ceph since I am not root 2024-06-10T00:54:01.236 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:01 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:01.114+0000 7ff47ae77200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-10T00:54:01.518 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:01 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:01.311+0000 7ff47ae77200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:54:01.519 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:01 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:01.418+0000 7ff47ae77200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-10T00:54:01.527 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.613+0000 7fed5c521700 1 Processor -- start 2024-06-10T00:54:01.527 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.613+0000 7fed5c521700 1 -- start start 2024-06-10T00:54:01.527 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.613+0000 7fed5c521700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5407e6f0 0x7fed5407cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:01.527 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.613+0000 7fed5c521700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fed5407d170 con 0x7fed5407e6f0 2024-06-10T00:54:01.527 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.613+0000 7fed5a2bd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5407e6f0 0x7fed5407cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:01.528 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.614+0000 7fed5a2bd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5407e6f0 0x7fed5407cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59654/0 (socket says 172.21.15.7:59654) 2024-06-10T00:54:01.528 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.614+0000 7fed5a2bd700 1 -- 172.21.15.7:0/1420517501 learned_addr learned my addr 172.21.15.7:0/1420517501 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:01.528 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.614+0000 7fed5a2bd700 1 -- 172.21.15.7:0/1420517501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed5407d2f0 con 0x7fed5407e6f0 2024-06-10T00:54:01.528 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.614+0000 7fed5a2bd700 1 --2- 172.21.15.7:0/1420517501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5407e6f0 0x7fed5407cba0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fed500098a0 tx=0x7fed50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593b89b6c2afd607 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:01.528 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.614+0000 7fed592bb700 1 -- 172.21.15.7:0/1420517501 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed50004160 con 0x7fed5407e6f0 2024-06-10T00:54:01.528 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.614+0000 7fed592bb700 1 -- 172.21.15.7:0/1420517501 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fed50004300 con 0x7fed5407e6f0 2024-06-10T00:54:01.528 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.614+0000 7fed592bb700 1 -- 172.21.15.7:0/1420517501 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed50011b50 con 0x7fed5407e6f0 2024-06-10T00:54:01.528 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.615+0000 7fed5c521700 1 -- 172.21.15.7:0/1420517501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5407e6f0 msgr2=0x7fed5407cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:01.528 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.615+0000 7fed5c521700 1 --2- 172.21.15.7:0/1420517501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5407e6f0 0x7fed5407cba0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fed500098a0 tx=0x7fed50009bb0 comp rx=0 tx=0).stop 2024-06-10T00:54:01.528 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.615+0000 7fed5c521700 1 -- 172.21.15.7:0/1420517501 shutdown_connections 2024-06-10T00:54:01.528 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.615+0000 7fed5c521700 1 --2- 172.21.15.7:0/1420517501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5407e6f0 0x7fed5407cba0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:01.528 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.615+0000 7fed5c521700 1 -- 172.21.15.7:0/1420517501 >> 172.21.15.7:0/1420517501 conn(0x7fed54103470 msgr2=0x7fed54105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:01.529 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.615+0000 7fed5c521700 1 -- 172.21.15.7:0/1420517501 shutdown_connections 2024-06-10T00:54:01.529 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.615+0000 7fed5c521700 1 -- 172.21.15.7:0/1420517501 wait complete. 2024-06-10T00:54:01.529 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.616+0000 7fed5c521700 1 Processor -- start 2024-06-10T00:54:01.529 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.616+0000 7fed5c521700 1 -- start start 2024-06-10T00:54:01.529 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.616+0000 7fed5c521700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5407e6f0 0x7fed5419df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:01.529 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.616+0000 7fed5c521700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fed5410e400 con 0x7fed5407e6f0 2024-06-10T00:54:01.529 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.616+0000 7fed5a2bd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5407e6f0 0x7fed5419df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:01.529 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.616+0000 7fed5a2bd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5407e6f0 0x7fed5419df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59660/0 (socket says 172.21.15.7:59660) 2024-06-10T00:54:01.529 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.616+0000 7fed5a2bd700 1 -- 172.21.15.7:0/3254678527 learned_addr learned my addr 172.21.15.7:0/3254678527 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:01.529 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.616+0000 7fed5a2bd700 1 -- 172.21.15.7:0/3254678527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed5419e4b0 con 0x7fed5407e6f0 2024-06-10T00:54:01.529 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.616+0000 7fed5a2bd700 1 --2- 172.21.15.7:0/3254678527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5407e6f0 0x7fed5419df70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fed50011b60 tx=0x7fed50011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:01.529 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.617+0000 7fed4b7fe700 1 -- 172.21.15.7:0/3254678527 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed50003b20 con 0x7fed5407e6f0 2024-06-10T00:54:01.529 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.617+0000 7fed5c521700 1 -- 172.21.15.7:0/3254678527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed5419e740 con 0x7fed5407e6f0 2024-06-10T00:54:01.530 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.617+0000 7fed4b7fe700 1 -- 172.21.15.7:0/3254678527 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fed50003cc0 con 0x7fed5407e6f0 2024-06-10T00:54:01.530 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.617+0000 7fed4b7fe700 1 -- 172.21.15.7:0/3254678527 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed50024840 con 0x7fed5407e6f0 2024-06-10T00:54:01.530 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.617+0000 7fed5c521700 1 -- 172.21.15.7:0/3254678527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed541a1430 con 0x7fed5407e6f0 2024-06-10T00:54:01.530 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.617+0000 7fed4b7fe700 1 -- 172.21.15.7:0/3254678527 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fed50022020 con 0x7fed5407e6f0 2024-06-10T00:54:01.530 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.617+0000 7fed5c521700 1 -- 172.21.15.7:0/3254678527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed54063af0 con 0x7fed5407e6f0 2024-06-10T00:54:01.530 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.618+0000 7fed4b7fe700 1 --2- 172.21.15.7:0/3254678527 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7fed40039000 0x7fed4003b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:01.530 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.618+0000 7fed4b7fe700 1 -- 172.21.15.7:0/3254678527 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fed50011e00 con 0x7fed5407e6f0 2024-06-10T00:54:01.530 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.618+0000 7fed59abc700 1 --2- 172.21.15.7:0/3254678527 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7fed40039000 0x7fed4003b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:01.530 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.618+0000 7fed59abc700 1 --2- 172.21.15.7:0/3254678527 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7fed40039000 0x7fed4003b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fed44006fd0 tx=0x7fed44006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:01.530 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.623+0000 7fed4b7fe700 1 -- 172.21.15.7:0/3254678527 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fed50024e20 con 0x7fed5407e6f0 2024-06-10T00:54:01.530 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.888+0000 7fed5c521700 1 -- 172.21.15.7:0/3254678527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fed541a1600 con 0x7fed5407e6f0 2024-06-10T00:54:01.530 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.939+0000 7fed4b7fe700 1 -- 172.21.15.7:0/3254678527 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fed541a1600 con 0x7fed5407e6f0 2024-06-10T00:54:01.531 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.941+0000 7fed4b7fe700 1 -- 172.21.15.7:0/3254678527 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fed50024a60 con 0x7fed5407e6f0 2024-06-10T00:54:01.531 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.943+0000 7fed5c521700 1 -- 172.21.15.7:0/3254678527 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7fed40039000 msgr2=0x7fed4003b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:01.531 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.943+0000 7fed5c521700 1 --2- 172.21.15.7:0/3254678527 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7fed40039000 0x7fed4003b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fed44006fd0 tx=0x7fed44006e40 comp rx=0 tx=0).stop 2024-06-10T00:54:01.531 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.943+0000 7fed5c521700 1 -- 172.21.15.7:0/3254678527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5407e6f0 msgr2=0x7fed5419df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:01.531 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.943+0000 7fed5c521700 1 --2- 172.21.15.7:0/3254678527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5407e6f0 0x7fed5419df70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fed50011b60 tx=0x7fed50011c40 comp rx=0 tx=0).stop 2024-06-10T00:54:01.531 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.943+0000 7fed5c521700 1 -- 172.21.15.7:0/3254678527 shutdown_connections 2024-06-10T00:54:01.531 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.943+0000 7fed5c521700 1 --2- 172.21.15.7:0/3254678527 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7fed40039000 0x7fed4003b4c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:01.531 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.943+0000 7fed5c521700 1 --2- 172.21.15.7:0/3254678527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5407e6f0 0x7fed5419df70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:01.531 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.943+0000 7fed5c521700 1 -- 172.21.15.7:0/3254678527 >> 172.21.15.7:0/3254678527 conn(0x7fed54103470 msgr2=0x7fed54103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:01.531 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.943+0000 7fed5c521700 1 -- 172.21.15.7:0/3254678527 shutdown_connections 2024-06-10T00:54:01.531 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:00.943+0000 7fed5c521700 1 -- 172.21.15.7:0/3254678527 wait complete. 2024-06-10T00:54:01.776 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:01 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:01.756+0000 7ff47ae77200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-10T00:54:02.173 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:01 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:01.848+0000 7ff47ae77200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-10T00:54:02.173 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:01 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3254678527' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-06-10T00:54:02.173 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:01 smithi007 ceph-mon[32398]: mgrmap e5: x(active, since 5s) 2024-06-10T00:54:02.859 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-06-10T00:54:02.859 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-06-10T00:54:02.859 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-10T00:54:02.859 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-10T00:54:02.859 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-10T00:54:02.859 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-06-10T00:54:02.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.966+0000 7f699b333700 1 Processor -- start 2024-06-10T00:54:02.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.966+0000 7f699b333700 1 -- start start 2024-06-10T00:54:02.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.967+0000 7f699b333700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6994107e60 0x7f6994108280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:02.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.967+0000 7f699b333700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f69941087c0 con 0x7f6994107e60 2024-06-10T00:54:02.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.967+0000 7f69990cf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6994107e60 0x7f6994108280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:02.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.967+0000 7f69990cf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6994107e60 0x7f6994108280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33300/0 (socket says 172.21.15.7:33300) 2024-06-10T00:54:02.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.967+0000 7f69990cf700 1 -- 172.21.15.7:0/1099971651 learned_addr learned my addr 172.21.15.7:0/1099971651 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:02.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.967+0000 7f69990cf700 1 -- 172.21.15.7:0/1099971651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6994108fc0 con 0x7f6994107e60 2024-06-10T00:54:02.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.967+0000 7f69990cf700 1 --2- 172.21.15.7:0/1099971651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6994107e60 0x7f6994108280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f69900098a0 tx=0x7f6990009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6a3fb69f56b19 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:02.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.967+0000 7f698bfff700 1 -- 172.21.15.7:0/1099971651 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6990004160 con 0x7f6994107e60 2024-06-10T00:54:02.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.968+0000 7f698bfff700 1 -- 172.21.15.7:0/1099971651 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6990004300 con 0x7f6994107e60 2024-06-10T00:54:02.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.968+0000 7f698bfff700 1 -- 172.21.15.7:0/1099971651 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6990011b50 con 0x7f6994107e60 2024-06-10T00:54:02.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.968+0000 7f699b333700 1 -- 172.21.15.7:0/1099971651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6994107e60 msgr2=0x7f6994108280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:02.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.968+0000 7f699b333700 1 --2- 172.21.15.7:0/1099971651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6994107e60 0x7f6994108280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f69900098a0 tx=0x7f6990009bb0 comp rx=0 tx=0).stop 2024-06-10T00:54:02.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.968+0000 7f699b333700 1 -- 172.21.15.7:0/1099971651 shutdown_connections 2024-06-10T00:54:02.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.968+0000 7f699b333700 1 --2- 172.21.15.7:0/1099971651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6994107e60 0x7f6994108280 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:02.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.969+0000 7f699b333700 1 -- 172.21.15.7:0/1099971651 >> 172.21.15.7:0/1099971651 conn(0x7f69941033e0 msgr2=0x7f6994105840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:02.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.969+0000 7f699b333700 1 -- 172.21.15.7:0/1099971651 shutdown_connections 2024-06-10T00:54:02.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.969+0000 7f699b333700 1 -- 172.21.15.7:0/1099971651 wait complete. 2024-06-10T00:54:02.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.969+0000 7f699b333700 1 Processor -- start 2024-06-10T00:54:02.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.969+0000 7f699b333700 1 -- start start 2024-06-10T00:54:02.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.969+0000 7f699b333700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6994107e60 0x7f699407db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:02.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.969+0000 7f699b333700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f699410c280 con 0x7f6994107e60 2024-06-10T00:54:02.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.970+0000 7f69990cf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6994107e60 0x7f699407db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:02.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.970+0000 7f69990cf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6994107e60 0x7f699407db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33308/0 (socket says 172.21.15.7:33308) 2024-06-10T00:54:02.862 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.970+0000 7f69990cf700 1 -- 172.21.15.7:0/1009361110 learned_addr learned my addr 172.21.15.7:0/1009361110 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:02.862 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.970+0000 7f69990cf700 1 -- 172.21.15.7:0/1009361110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699407e0c0 con 0x7f6994107e60 2024-06-10T00:54:02.862 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.970+0000 7f69990cf700 1 --2- 172.21.15.7:0/1009361110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6994107e60 0x7f699407db80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6990011b60 tx=0x7f6990011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:02.862 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.970+0000 7f698a7fc700 1 -- 172.21.15.7:0/1009361110 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6990003b20 con 0x7f6994107e60 2024-06-10T00:54:02.862 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.970+0000 7f698a7fc700 1 -- 172.21.15.7:0/1009361110 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6990003cc0 con 0x7f6994107e60 2024-06-10T00:54:02.862 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.970+0000 7f699b333700 1 -- 172.21.15.7:0/1009361110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f699407c2b0 con 0x7f6994107e60 2024-06-10T00:54:02.862 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.970+0000 7f698a7fc700 1 -- 172.21.15.7:0/1009361110 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6990024840 con 0x7f6994107e60 2024-06-10T00:54:02.862 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.970+0000 7f699b333700 1 -- 172.21.15.7:0/1009361110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f699407c7f0 con 0x7f6994107e60 2024-06-10T00:54:02.862 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.971+0000 7f698a7fc700 1 -- 172.21.15.7:0/1009361110 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f6990022020 con 0x7f6994107e60 2024-06-10T00:54:02.862 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.971+0000 7f698a7fc700 1 --2- 172.21.15.7:0/1009361110 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7f69800390a0 0x7f698003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:02.862 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.971+0000 7f698a7fc700 1 -- 172.21.15.7:0/1009361110 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6990011e00 con 0x7f6994107e60 2024-06-10T00:54:02.862 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.971+0000 7f69988ce700 1 -- 172.21.15.7:0/1009361110 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7f69800390a0 msgr2=0x7f698003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/1100766262 2024-06-10T00:54:02.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.971+0000 7f69988ce700 1 --2- 172.21.15.7:0/1009361110 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7f69800390a0 0x7f698003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T00:54:02.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.971+0000 7f699b333700 1 -- 172.21.15.7:0/1009361110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6994063af0 con 0x7f6994107e60 2024-06-10T00:54:02.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:01.977+0000 7f698a7fc700 1 -- 172.21.15.7:0/1009361110 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6990019c70 con 0x7f6994107e60 2024-06-10T00:54:02.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:02.172+0000 7f69988ce700 1 -- 172.21.15.7:0/1009361110 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7f69800390a0 msgr2=0x7f698003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/1100766262 2024-06-10T00:54:02.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:02.172+0000 7f69988ce700 1 --2- 172.21.15.7:0/1009361110 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7f69800390a0 0x7f698003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-10T00:54:02.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:02.285+0000 7f699b333700 1 -- 172.21.15.7:0/1009361110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f699407d0d0 con 0x7f6994107e60 2024-06-10T00:54:02.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:02.286+0000 7f698a7fc700 1 -- 172.21.15.7:0/1009361110 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f69900172b0 con 0x7f6994107e60 2024-06-10T00:54:02.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:02.290+0000 7f699b333700 1 -- 172.21.15.7:0/1009361110 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7f69800390a0 msgr2=0x7f698003b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T00:54:02.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:02.290+0000 7f699b333700 1 --2- 172.21.15.7:0/1009361110 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7f69800390a0 0x7f698003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:02.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:02.290+0000 7f699b333700 1 -- 172.21.15.7:0/1009361110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6994107e60 msgr2=0x7f699407db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:02.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:02.290+0000 7f699b333700 1 --2- 172.21.15.7:0/1009361110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6994107e60 0x7f699407db80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6990011b60 tx=0x7f6990011c40 comp rx=0 tx=0).stop 2024-06-10T00:54:02.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:02.291+0000 7f699b333700 1 -- 172.21.15.7:0/1009361110 shutdown_connections 2024-06-10T00:54:02.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:02.291+0000 7f699b333700 1 --2- 172.21.15.7:0/1009361110 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7f69800390a0 0x7f698003b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:02.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:02.291+0000 7f699b333700 1 --2- 172.21.15.7:0/1009361110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6994107e60 0x7f699407db80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:02.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:02.291+0000 7f699b333700 1 -- 172.21.15.7:0/1009361110 >> 172.21.15.7:0/1009361110 conn(0x7f69941033e0 msgr2=0x7f6994103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:02.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:02.292+0000 7f699b333700 1 -- 172.21.15.7:0/1009361110 shutdown_connections 2024-06-10T00:54:02.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:02.292+0000 7f699b333700 1 -- 172.21.15.7:0/1009361110 wait complete. 2024-06-10T00:54:02.864 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for the mgr to restart... 2024-06-10T00:54:02.864 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr epoch 5... 2024-06-10T00:54:03.236 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:02 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1009361110' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-10T00:54:03.236 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:03 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:03.132+0000 7ff47ae77200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-10T00:54:03.488 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:03 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:03.313+0000 7ff47ae77200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-10T00:54:03.769 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:03 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:03.490+0000 7ff47ae77200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-10T00:54:04.099 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:03 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:03.792+0000 7ff47ae77200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-10T00:54:04.099 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:03 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:03.888+0000 7ff47ae77200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-10T00:54:04.099 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:04 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:04.099+0000 7ff47ae77200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-10T00:54:04.518 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:04 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:04.299+0000 7ff47ae77200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:54:05.018 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:04 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:04.932+0000 7ff47ae77200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-10T00:54:05.683 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:05 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:05.373+0000 7ff47ae77200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-10T00:54:05.684 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:05 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:05.474+0000 7ff47ae77200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-10T00:54:05.684 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:05 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:05.683+0000 7ff47ae77200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-10T00:54:06.518 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:06 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:06.230+0000 7ff47ae77200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-10T00:54:06.518 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:06 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:06.326+0000 7ff47ae77200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-10T00:54:06.938 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:06 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:06.579+0000 7ff47ae77200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-10T00:54:06.938 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:06 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:06.718+0000 7ff47ae77200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-10T00:54:06.938 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:06 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:06.817+0000 7ff47ae77200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-10T00:54:06.938 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:06 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:06.937+0000 7ff47ae77200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-10T00:54:07.519 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:07 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:07.249+0000 7ff47ae77200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-10T00:54:08.153 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:07 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:07.876+0000 7ff47ae77200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-10T00:54:08.153 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:08 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:08.100+0000 7ff47ae77200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-10T00:54:08.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:08 smithi007 ceph-mon[32398]: Active manager daemon x restarted 2024-06-10T00:54:08.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:08 smithi007 ceph-mon[32398]: Activating manager daemon x 2024-06-10T00:54:08.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:08 smithi007 ceph-mon[32398]: osdmap e2: 0 total, 0 up, 0 in 2024-06-10T00:54:08.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:08 smithi007 ceph-mon[32398]: mgrmap e6: x(active, starting, since 0.00346514s) 2024-06-10T00:54:08.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:08 smithi007 ceph-mon[32398]: from='mgr.14120 172.21.15.7:0/675400875' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:54:08.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:08 smithi007 ceph-mon[32398]: from='mgr.14120 172.21.15.7:0/675400875' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-10T00:54:08.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:08 smithi007 ceph-mon[32398]: from='mgr.14120 172.21.15.7:0/675400875' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-10T00:54:08.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:08 smithi007 ceph-mon[32398]: from='mgr.14120 172.21.15.7:0/675400875' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-10T00:54:08.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:08 smithi007 ceph-mon[32398]: from='mgr.14120 172.21.15.7:0/675400875' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-10T00:54:08.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:08 smithi007 ceph-mon[32398]: Manager daemon x is now available 2024-06-10T00:54:08.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:08 smithi007 ceph-mon[32398]: from='mgr.14120 172.21.15.7:0/675400875' entity='mgr.x' 2024-06-10T00:54:08.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:08 smithi007 ceph-mon[32398]: from='mgr.14120 172.21.15.7:0/675400875' entity='mgr.x' 2024-06-10T00:54:08.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:08 smithi007 ceph-mon[32398]: from='mgr.14120 172.21.15.7:0/675400875' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:54:08.520 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:08 smithi007 ceph-mon[32398]: from='mgr.14120 172.21.15.7:0/675400875' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:54:09.473 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:09 smithi007 ceph-mon[32398]: Found migration_current of "None". Setting to last migration. 2024-06-10T00:54:09.473 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:09 smithi007 ceph-mon[32398]: from='mgr.14120 172.21.15.7:0/675400875' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-10T00:54:09.473 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:09 smithi007 ceph-mon[32398]: from='mgr.14120 172.21.15.7:0/675400875' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-10T00:54:09.473 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:09 smithi007 ceph-mon[32398]: from='mgr.14120 172.21.15.7:0/675400875' entity='mgr.x' 2024-06-10T00:54:09.474 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:09 smithi007 ceph-mon[32398]: from='mgr.14120 172.21.15.7:0/675400875' entity='mgr.x' 2024-06-10T00:54:09.474 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:09 smithi007 ceph-mon[32398]: mgrmap e7: x(active, since 1.00568s) 2024-06-10T00:54:09.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-06-10T00:54:09.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-06-10T00:54:09.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-10T00:54:09.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-06-10T00:54:09.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.282+0000 7f7bc68cb700 1 Processor -- start 2024-06-10T00:54:09.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.282+0000 7f7bc68cb700 1 -- start start 2024-06-10T00:54:09.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.282+0000 7f7bc68cb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bc0106d80 0x7f7bc01071a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:09.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.282+0000 7f7bc68cb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc0107770 con 0x7f7bc0106d80 2024-06-10T00:54:09.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.282+0000 7f7bbffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bc0106d80 0x7f7bc01071a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:09.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.282+0000 7f7bbffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bc0106d80 0x7f7bc01071a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33310/0 (socket says 172.21.15.7:33310) 2024-06-10T00:54:09.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.282+0000 7f7bbffff700 1 -- 172.21.15.7:0/2032017613 learned_addr learned my addr 172.21.15.7:0/2032017613 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:09.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.282+0000 7f7bbffff700 1 -- 172.21.15.7:0/2032017613 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc0107f80 con 0x7f7bc0106d80 2024-06-10T00:54:09.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.283+0000 7f7bbffff700 1 --2- 172.21.15.7:0/2032017613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bc0106d80 0x7f7bc01071a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7bb000bb40 tx=0x7f7bb000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c449d06702fe4c9b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:09.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.283+0000 7f7bbeffd700 1 -- 172.21.15.7:0/2032017613 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bb000dbe0 con 0x7f7bc0106d80 2024-06-10T00:54:09.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.283+0000 7f7bbeffd700 1 -- 172.21.15.7:0/2032017613 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7bb00044f0 con 0x7f7bc0106d80 2024-06-10T00:54:09.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.283+0000 7f7bbeffd700 1 -- 172.21.15.7:0/2032017613 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bb0010470 con 0x7f7bc0106d80 2024-06-10T00:54:09.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.283+0000 7f7bc68cb700 1 -- 172.21.15.7:0/2032017613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bc0106d80 msgr2=0x7f7bc01071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:09.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.283+0000 7f7bc68cb700 1 --2- 172.21.15.7:0/2032017613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bc0106d80 0x7f7bc01071a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7bb000bb40 tx=0x7f7bb000be50 comp rx=0 tx=0).stop 2024-06-10T00:54:09.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.284+0000 7f7bc68cb700 1 -- 172.21.15.7:0/2032017613 shutdown_connections 2024-06-10T00:54:09.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.284+0000 7f7bc68cb700 1 --2- 172.21.15.7:0/2032017613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bc0106d80 0x7f7bc01071a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:09.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.284+0000 7f7bc68cb700 1 -- 172.21.15.7:0/2032017613 >> 172.21.15.7:0/2032017613 conn(0x7f7bc01022e0 msgr2=0x7f7bc0104760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:09.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.284+0000 7f7bc68cb700 1 -- 172.21.15.7:0/2032017613 shutdown_connections 2024-06-10T00:54:09.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.284+0000 7f7bc68cb700 1 -- 172.21.15.7:0/2032017613 wait complete. 2024-06-10T00:54:09.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.284+0000 7f7bc68cb700 1 Processor -- start 2024-06-10T00:54:09.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.284+0000 7f7bc68cb700 1 -- start start 2024-06-10T00:54:09.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.285+0000 7f7bc68cb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bc0106d80 0x7f7bc019dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:09.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.285+0000 7f7bc68cb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc007b870 con 0x7f7bc0106d80 2024-06-10T00:54:09.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.285+0000 7f7bbffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bc0106d80 0x7f7bc019dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:09.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.285+0000 7f7bbffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bc0106d80 0x7f7bc019dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33312/0 (socket says 172.21.15.7:33312) 2024-06-10T00:54:09.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.285+0000 7f7bbffff700 1 -- 172.21.15.7:0/1333145216 learned_addr learned my addr 172.21.15.7:0/1333145216 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:09.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.285+0000 7f7bbffff700 1 -- 172.21.15.7:0/1333145216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc019e4f0 con 0x7f7bc0106d80 2024-06-10T00:54:09.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.285+0000 7f7bbffff700 1 --2- 172.21.15.7:0/1333145216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bc0106d80 0x7f7bc019dfb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7bb000b340 tx=0x7f7bb000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:09.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.285+0000 7f7bbd7fa700 1 -- 172.21.15.7:0/1333145216 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bb0010680 con 0x7f7bc0106d80 2024-06-10T00:54:09.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.285+0000 7f7bbd7fa700 1 -- 172.21.15.7:0/1333145216 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7bb001d070 con 0x7f7bc0106d80 2024-06-10T00:54:09.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.286+0000 7f7bc68cb700 1 -- 172.21.15.7:0/1333145216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bc019e780 con 0x7f7bc0106d80 2024-06-10T00:54:09.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.286+0000 7f7bbd7fa700 1 -- 172.21.15.7:0/1333145216 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bb0021b10 con 0x7f7bc0106d80 2024-06-10T00:54:09.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.286+0000 7f7bc68cb700 1 -- 172.21.15.7:0/1333145216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bc01a1470 con 0x7f7bc0106d80 2024-06-10T00:54:09.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.286+0000 7f7bbd7fa700 1 -- 172.21.15.7:0/1333145216 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f7bb001f030 con 0x7f7bc0106d80 2024-06-10T00:54:09.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.287+0000 7f7bbd7fa700 1 --2- 172.21.15.7:0/1333145216 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7f7ba80390a0 0x7f7ba803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:09.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.287+0000 7f7bc68cb700 1 -- 172.21.15.7:0/1333145216 --> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7bc0063af0 con 0x7f7ba80390a0 2024-06-10T00:54:09.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.287+0000 7f7bbd7fa700 1 -- 172.21.15.7:0/1333145216 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7bb004dec0 con 0x7f7bc0106d80 2024-06-10T00:54:09.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.287+0000 7f7bbf7fe700 1 -- 172.21.15.7:0/1333145216 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7f7ba80390a0 msgr2=0x7f7ba803b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/1100766262 2024-06-10T00:54:09.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.287+0000 7f7bbf7fe700 1 --2- 172.21.15.7:0/1333145216 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7f7ba80390a0 0x7f7ba803b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T00:54:09.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.488+0000 7f7bbf7fe700 1 -- 172.21.15.7:0/1333145216 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7f7ba80390a0 msgr2=0x7f7ba803b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/1100766262 2024-06-10T00:54:09.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.488+0000 7f7bbf7fe700 1 --2- 172.21.15.7:0/1333145216 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7f7ba80390a0 0x7f7ba803b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-10T00:54:09.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.889+0000 7f7bbf7fe700 1 -- 172.21.15.7:0/1333145216 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7f7ba80390a0 msgr2=0x7f7ba803b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/1100766262 2024-06-10T00:54:09.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:03.889+0000 7f7bbf7fe700 1 --2- 172.21.15.7:0/1333145216 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7f7ba80390a0 0x7f7ba803b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-10T00:54:09.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:04.690+0000 7f7bbf7fe700 1 -- 172.21.15.7:0/1333145216 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7f7ba80390a0 msgr2=0x7f7ba803b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/1100766262 2024-06-10T00:54:09.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:04.690+0000 7f7bbf7fe700 1 --2- 172.21.15.7:0/1333145216 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7f7ba80390a0 0x7f7ba803b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-10T00:54:09.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:06.291+0000 7f7bbf7fe700 1 -- 172.21.15.7:0/1333145216 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7f7ba80390a0 msgr2=0x7f7ba803b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/1100766262 2024-06-10T00:54:09.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:06.291+0000 7f7bbf7fe700 1 --2- 172.21.15.7:0/1333145216 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7f7ba80390a0 0x7f7ba803b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-10T00:54:09.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:08.103+0000 7f7bbd7fa700 1 -- 172.21.15.7:0/1333145216 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f7bb0021d00 con 0x7f7bc0106d80 2024-06-10T00:54:09.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:08.103+0000 7f7bbd7fa700 1 -- 172.21.15.7:0/1333145216 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7f7ba80390a0 msgr2=0x7f7ba803b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T00:54:09.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:08.103+0000 7f7bbd7fa700 1 --2- 172.21.15.7:0/1333145216 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7f7ba80390a0 0x7f7ba803b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:09.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:09.105+0000 7f7bbd7fa700 1 -- 172.21.15.7:0/1333145216 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f7bb0019770 con 0x7f7bc0106d80 2024-06-10T00:54:09.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:09.105+0000 7f7bbd7fa700 1 --2- 172.21.15.7:0/1333145216 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f7ba803c830 0x7f7ba803ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:09.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:09.105+0000 7f7bbd7fa700 1 -- 172.21.15.7:0/1333145216 --> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7bb000ee90 con 0x7f7ba803c830 2024-06-10T00:54:09.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:09.105+0000 7f7bbf7fe700 1 --2- 172.21.15.7:0/1333145216 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f7ba803c830 0x7f7ba803ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:09.730 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:09.106+0000 7f7bbf7fe700 1 --2- 172.21.15.7:0/1333145216 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f7ba803c830 0x7f7ba803ec20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4003a10 tx=0x7f7bb40092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:09.730 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:09.106+0000 7f7bbd7fa700 1 -- 172.21.15.7:0/1333145216 <== mgr.14120 v2:172.21.15.7:6800/3298829389 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f7bb000ee90 con 0x7f7ba803c830 2024-06-10T00:54:09.730 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:09.114+0000 7f7bc68cb700 1 -- 172.21.15.7:0/1333145216 --> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f7bc0195d70 con 0x7f7ba803c830 2024-06-10T00:54:09.730 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:09.114+0000 7f7bbd7fa700 1 -- 172.21.15.7:0/1333145216 <== mgr.14120 v2:172.21.15.7:6800/3298829389 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f7bc0195d70 con 0x7f7ba803c830 2024-06-10T00:54:09.730 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:09.115+0000 7f7bc68cb700 1 -- 172.21.15.7:0/1333145216 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f7ba803c830 msgr2=0x7f7ba803ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:09.730 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:09.115+0000 7f7bc68cb700 1 --2- 172.21.15.7:0/1333145216 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f7ba803c830 0x7f7ba803ec20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4003a10 tx=0x7f7bb40092b0 comp rx=0 tx=0).stop 2024-06-10T00:54:09.730 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:09.115+0000 7f7bc68cb700 1 -- 172.21.15.7:0/1333145216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bc0106d80 msgr2=0x7f7bc019dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:09.730 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:09.115+0000 7f7bc68cb700 1 --2- 172.21.15.7:0/1333145216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bc0106d80 0x7f7bc019dfb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7bb000b340 tx=0x7f7bb000dd90 comp rx=0 tx=0).stop 2024-06-10T00:54:09.730 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:09.115+0000 7f7bc68cb700 1 -- 172.21.15.7:0/1333145216 shutdown_connections 2024-06-10T00:54:09.730 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:09.115+0000 7f7bc68cb700 1 --2- 172.21.15.7:0/1333145216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bc0106d80 0x7f7bc019dfb0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:09.730 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:09.115+0000 7f7bc68cb700 1 --2- 172.21.15.7:0/1333145216 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f7ba803c830 0x7f7ba803ec20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:09.730 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:09.115+0000 7f7bc68cb700 1 --2- 172.21.15.7:0/1333145216 >> [v2:172.21.15.7:6800/1100766262,v1:172.21.15.7:6801/1100766262] conn(0x7f7ba80390a0 0x7f7ba803b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:09.731 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:09.115+0000 7f7bc68cb700 1 -- 172.21.15.7:0/1333145216 >> 172.21.15.7:0/1333145216 conn(0x7f7bc01022e0 msgr2=0x7f7bc01945d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:09.731 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:09.115+0000 7f7bc68cb700 1 -- 172.21.15.7:0/1333145216 shutdown_connections 2024-06-10T00:54:09.731 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:09.115+0000 7f7bc68cb700 1 -- 172.21.15.7:0/1333145216 wait complete. 2024-06-10T00:54:09.731 INFO:teuthology.orchestra.run.smithi007.stdout:mgr epoch 5 is available 2024-06-10T00:54:09.731 INFO:teuthology.orchestra.run.smithi007.stdout:Setting orchestrator backend to cephadm... 2024-06-10T00:54:10.430 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:10 smithi007 ceph-mon[32398]: [10/Jun/2024:00:54:09] ENGINE Bus STARTING 2024-06-10T00:54:10.431 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:10 smithi007 ceph-mon[32398]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-10T00:54:10.431 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:10 smithi007 ceph-mon[32398]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-10T00:54:10.431 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:10 smithi007 ceph-mon[32398]: [10/Jun/2024:00:54:09] ENGINE Serving on https://172.21.15.7:7150 2024-06-10T00:54:10.431 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:10 smithi007 ceph-mon[32398]: [10/Jun/2024:00:54:09] ENGINE Serving on http://172.21.15.7:8765 2024-06-10T00:54:10.431 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:10 smithi007 ceph-mon[32398]: [10/Jun/2024:00:54:09] ENGINE Bus STARTED 2024-06-10T00:54:10.431 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:10 smithi007 ceph-mon[32398]: from='mgr.14120 172.21.15.7:0/675400875' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:54:11.020 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.197+0000 7f8d33e8b700 1 Processor -- start 2024-06-10T00:54:11.020 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.197+0000 7f8d33e8b700 1 -- start start 2024-06-10T00:54:11.020 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.198+0000 7f8d33e8b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d2c107e50 0x7f8d2c108270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:11.020 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.198+0000 7f8d33e8b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c1087b0 con 0x7f8d2c107e50 2024-06-10T00:54:11.021 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.198+0000 7f8d31c27700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d2c107e50 0x7f8d2c108270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:11.021 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.198+0000 7f8d31c27700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d2c107e50 0x7f8d2c108270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33374/0 (socket says 172.21.15.7:33374) 2024-06-10T00:54:11.021 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.198+0000 7f8d31c27700 1 -- 172.21.15.7:0/3824716218 learned_addr learned my addr 172.21.15.7:0/3824716218 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:11.021 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.198+0000 7f8d31c27700 1 -- 172.21.15.7:0/3824716218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d2c109040 con 0x7f8d2c107e50 2024-06-10T00:54:11.021 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.198+0000 7f8d31c27700 1 --2- 172.21.15.7:0/3824716218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d2c107e50 0x7f8d2c108270 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8d280098a0 tx=0x7f8d28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed007b8bc1802dd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:11.021 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.198+0000 7f8d30c25700 1 -- 172.21.15.7:0/3824716218 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d28004160 con 0x7f8d2c107e50 2024-06-10T00:54:11.021 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.198+0000 7f8d30c25700 1 -- 172.21.15.7:0/3824716218 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8d28004300 con 0x7f8d2c107e50 2024-06-10T00:54:11.021 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.198+0000 7f8d30c25700 1 -- 172.21.15.7:0/3824716218 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d28011b50 con 0x7f8d2c107e50 2024-06-10T00:54:11.021 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.199+0000 7f8d33e8b700 1 -- 172.21.15.7:0/3824716218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d2c107e50 msgr2=0x7f8d2c108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:11.021 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.199+0000 7f8d33e8b700 1 --2- 172.21.15.7:0/3824716218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d2c107e50 0x7f8d2c108270 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8d280098a0 tx=0x7f8d28009bb0 comp rx=0 tx=0).stop 2024-06-10T00:54:11.021 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.199+0000 7f8d33e8b700 1 -- 172.21.15.7:0/3824716218 shutdown_connections 2024-06-10T00:54:11.021 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.199+0000 7f8d33e8b700 1 --2- 172.21.15.7:0/3824716218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d2c107e50 0x7f8d2c108270 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:11.021 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.199+0000 7f8d33e8b700 1 -- 172.21.15.7:0/3824716218 >> 172.21.15.7:0/3824716218 conn(0x7f8d2c103460 msgr2=0x7f8d2c105880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:11.022 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.199+0000 7f8d33e8b700 1 -- 172.21.15.7:0/3824716218 shutdown_connections 2024-06-10T00:54:11.022 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.199+0000 7f8d33e8b700 1 -- 172.21.15.7:0/3824716218 wait complete. 2024-06-10T00:54:11.022 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.200+0000 7f8d33e8b700 1 Processor -- start 2024-06-10T00:54:11.022 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.200+0000 7f8d33e8b700 1 -- start start 2024-06-10T00:54:11.022 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.200+0000 7f8d33e8b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d2c107e50 0x7f8d2c07e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:11.022 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.200+0000 7f8d33e8b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c108930 con 0x7f8d2c107e50 2024-06-10T00:54:11.022 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.200+0000 7f8d31c27700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d2c107e50 0x7f8d2c07e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:11.022 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.200+0000 7f8d31c27700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d2c107e50 0x7f8d2c07e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33384/0 (socket says 172.21.15.7:33384) 2024-06-10T00:54:11.022 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.200+0000 7f8d31c27700 1 -- 172.21.15.7:0/4020903527 learned_addr learned my addr 172.21.15.7:0/4020903527 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:11.022 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.201+0000 7f8d31c27700 1 -- 172.21.15.7:0/4020903527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d2c07e950 con 0x7f8d2c107e50 2024-06-10T00:54:11.022 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.201+0000 7f8d31c27700 1 --2- 172.21.15.7:0/4020903527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d2c107e50 0x7f8d2c07e410 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8d28009fb0 tx=0x7f8d28011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:11.022 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.201+0000 7f8d22ffd700 1 -- 172.21.15.7:0/4020903527 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d280038b0 con 0x7f8d2c107e50 2024-06-10T00:54:11.022 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.201+0000 7f8d22ffd700 1 -- 172.21.15.7:0/4020903527 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8d28011e50 con 0x7f8d2c107e50 2024-06-10T00:54:11.023 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.201+0000 7f8d33e8b700 1 -- 172.21.15.7:0/4020903527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d2c07cb40 con 0x7f8d2c107e50 2024-06-10T00:54:11.023 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.201+0000 7f8d22ffd700 1 -- 172.21.15.7:0/4020903527 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d28019550 con 0x7f8d2c107e50 2024-06-10T00:54:11.023 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.201+0000 7f8d33e8b700 1 -- 172.21.15.7:0/4020903527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d2c07d080 con 0x7f8d2c107e50 2024-06-10T00:54:11.023 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.202+0000 7f8d22ffd700 1 -- 172.21.15.7:0/4020903527 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f8d28019800 con 0x7f8d2c107e50 2024-06-10T00:54:11.023 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.202+0000 7f8d22ffd700 1 --2- 172.21.15.7:0/4020903527 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f8d18039050 0x7f8d1803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:11.023 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.202+0000 7f8d33e8b700 1 -- 172.21.15.7:0/4020903527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d2c063af0 con 0x7f8d2c107e50 2024-06-10T00:54:11.023 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.202+0000 7f8d22ffd700 1 -- 172.21.15.7:0/4020903527 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8d2804d260 con 0x7f8d2c107e50 2024-06-10T00:54:11.023 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.202+0000 7f8d31426700 1 --2- 172.21.15.7:0/4020903527 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f8d18039050 0x7f8d1803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:11.023 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.203+0000 7f8d31426700 1 --2- 172.21.15.7:0/4020903527 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f8d18039050 0x7f8d1803b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c006fd0 tx=0x7f8d1c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:11.023 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.207+0000 7f8d22ffd700 1 -- 172.21.15.7:0/4020903527 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8d2804a080 con 0x7f8d2c107e50 2024-06-10T00:54:11.023 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.430+0000 7f8d33e8b700 1 -- 172.21.15.7:0/4020903527 --> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f8d2c07d9f0 con 0x7f8d18039050 2024-06-10T00:54:11.023 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.436+0000 7f8d22ffd700 1 -- 172.21.15.7:0/4020903527 <== mgr.14120 v2:172.21.15.7:6800/3298829389 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8d1c00a040 con 0x7f8d18039050 2024-06-10T00:54:11.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.438+0000 7f8d33e8b700 1 -- 172.21.15.7:0/4020903527 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f8d18039050 msgr2=0x7f8d1803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:11.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.438+0000 7f8d33e8b700 1 --2- 172.21.15.7:0/4020903527 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f8d18039050 0x7f8d1803b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c006fd0 tx=0x7f8d1c006e40 comp rx=0 tx=0).stop 2024-06-10T00:54:11.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.438+0000 7f8d33e8b700 1 -- 172.21.15.7:0/4020903527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d2c107e50 msgr2=0x7f8d2c07e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:11.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.438+0000 7f8d33e8b700 1 --2- 172.21.15.7:0/4020903527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d2c107e50 0x7f8d2c07e410 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8d28009fb0 tx=0x7f8d28011bf0 comp rx=0 tx=0).stop 2024-06-10T00:54:11.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.438+0000 7f8d33e8b700 1 -- 172.21.15.7:0/4020903527 shutdown_connections 2024-06-10T00:54:11.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.438+0000 7f8d33e8b700 1 --2- 172.21.15.7:0/4020903527 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f8d18039050 0x7f8d1803b510 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:11.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.438+0000 7f8d33e8b700 1 --2- 172.21.15.7:0/4020903527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d2c107e50 0x7f8d2c07e410 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:11.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.438+0000 7f8d33e8b700 1 -- 172.21.15.7:0/4020903527 >> 172.21.15.7:0/4020903527 conn(0x7f8d2c103460 msgr2=0x7f8d2c103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:11.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.438+0000 7f8d33e8b700 1 -- 172.21.15.7:0/4020903527 shutdown_connections 2024-06-10T00:54:11.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:10.438+0000 7f8d33e8b700 1 -- 172.21.15.7:0/4020903527 wait complete. 2024-06-10T00:54:11.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:11 smithi007 ceph-mon[32398]: mgrmap e8: x(active, since 2s) 2024-06-10T00:54:11.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:11 smithi007 ceph-mon[32398]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:54:11.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:11 smithi007 ceph-mon[32398]: from='mgr.14120 172.21.15.7:0/675400875' entity='mgr.x' 2024-06-10T00:54:11.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:11 smithi007 ceph-mon[32398]: from='mgr.14120 172.21.15.7:0/675400875' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:54:12.227 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:12 smithi007 ceph-mon[32398]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:54:12.235 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout value unchanged 2024-06-10T00:54:12.235 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.484+0000 7fa6b6a53700 1 Processor -- start 2024-06-10T00:54:12.235 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.484+0000 7fa6b6a53700 1 -- start start 2024-06-10T00:54:12.235 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.484+0000 7fa6b6a53700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b0108990 0x7fa6b010ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:12.235 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.484+0000 7fa6b6a53700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b007c6a0 con 0x7fa6b0108990 2024-06-10T00:54:12.235 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.485+0000 7fa6affff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b0108990 0x7fa6b010ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:12.236 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.485+0000 7fa6affff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b0108990 0x7fa6b010ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33386/0 (socket says 172.21.15.7:33386) 2024-06-10T00:54:12.236 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.485+0000 7fa6affff700 1 -- 172.21.15.7:0/1607094190 learned_addr learned my addr 172.21.15.7:0/1607094190 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:12.236 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.485+0000 7fa6affff700 1 -- 172.21.15.7:0/1607094190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b010b2c0 con 0x7fa6b0108990 2024-06-10T00:54:12.236 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.485+0000 7fa6affff700 1 --2- 172.21.15.7:0/1607094190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b0108990 0x7fa6b010ad80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa6a00098a0 tx=0x7fa6a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66131a7b48144980 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:12.236 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.485+0000 7fa6aeffd700 1 -- 172.21.15.7:0/1607094190 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6a0004160 con 0x7fa6b0108990 2024-06-10T00:54:12.236 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.485+0000 7fa6aeffd700 1 -- 172.21.15.7:0/1607094190 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa6a0004300 con 0x7fa6b0108990 2024-06-10T00:54:12.236 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.485+0000 7fa6aeffd700 1 -- 172.21.15.7:0/1607094190 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6a0011b50 con 0x7fa6b0108990 2024-06-10T00:54:12.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.486+0000 7fa6b6a53700 1 -- 172.21.15.7:0/1607094190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b0108990 msgr2=0x7fa6b010ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:12.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.486+0000 7fa6b6a53700 1 --2- 172.21.15.7:0/1607094190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b0108990 0x7fa6b010ad80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa6a00098a0 tx=0x7fa6a0009bb0 comp rx=0 tx=0).stop 2024-06-10T00:54:12.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.486+0000 7fa6b6a53700 1 -- 172.21.15.7:0/1607094190 shutdown_connections 2024-06-10T00:54:12.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.486+0000 7fa6b6a53700 1 --2- 172.21.15.7:0/1607094190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b0108990 0x7fa6b010ad80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:12.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.486+0000 7fa6b6a53700 1 -- 172.21.15.7:0/1607094190 >> 172.21.15.7:0/1607094190 conn(0x7fa6b0102340 msgr2=0x7fa6b01047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:12.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.486+0000 7fa6b6a53700 1 -- 172.21.15.7:0/1607094190 shutdown_connections 2024-06-10T00:54:12.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.486+0000 7fa6b6a53700 1 -- 172.21.15.7:0/1607094190 wait complete. 2024-06-10T00:54:12.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.486+0000 7fa6b6a53700 1 Processor -- start 2024-06-10T00:54:12.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.487+0000 7fa6b6a53700 1 -- start start 2024-06-10T00:54:12.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.487+0000 7fa6b6a53700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b0108990 0x7fa6b019bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:12.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.487+0000 7fa6b6a53700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b010c210 con 0x7fa6b0108990 2024-06-10T00:54:12.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.487+0000 7fa6affff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b0108990 0x7fa6b019bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:12.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.487+0000 7fa6affff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b0108990 0x7fa6b019bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33398/0 (socket says 172.21.15.7:33398) 2024-06-10T00:54:12.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.487+0000 7fa6affff700 1 -- 172.21.15.7:0/3756611843 learned_addr learned my addr 172.21.15.7:0/3756611843 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:12.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.487+0000 7fa6affff700 1 -- 172.21.15.7:0/3756611843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b019c2f0 con 0x7fa6b0108990 2024-06-10T00:54:12.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.488+0000 7fa6affff700 1 --2- 172.21.15.7:0/3756611843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b0108990 0x7fa6b019bdb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0011b60 tx=0x7fa6a0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:12.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.488+0000 7fa6ad7fa700 1 -- 172.21.15.7:0/3756611843 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6a0003970 con 0x7fa6b0108990 2024-06-10T00:54:12.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.488+0000 7fa6ad7fa700 1 -- 172.21.15.7:0/3756611843 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa6a0003b10 con 0x7fa6b0108990 2024-06-10T00:54:12.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.488+0000 7fa6b6a53700 1 -- 172.21.15.7:0/3756611843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b019c580 con 0x7fa6b0108990 2024-06-10T00:54:12.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.488+0000 7fa6ad7fa700 1 -- 172.21.15.7:0/3756611843 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6a0024610 con 0x7fa6b0108990 2024-06-10T00:54:12.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.488+0000 7fa6b6a53700 1 -- 172.21.15.7:0/3756611843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b019f270 con 0x7fa6b0108990 2024-06-10T00:54:12.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.489+0000 7fa6ad7fa700 1 -- 172.21.15.7:0/3756611843 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa6a001a430 con 0x7fa6b0108990 2024-06-10T00:54:12.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.489+0000 7fa6b6a53700 1 -- 172.21.15.7:0/3756611843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6b0063af0 con 0x7fa6b0108990 2024-06-10T00:54:12.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.489+0000 7fa6ad7fa700 1 --2- 172.21.15.7:0/3756611843 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fa6980390a0 0x7fa69803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:12.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.489+0000 7fa6ad7fa700 1 -- 172.21.15.7:0/3756611843 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa6a0011e00 con 0x7fa6b0108990 2024-06-10T00:54:12.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.489+0000 7fa6af7fe700 1 --2- 172.21.15.7:0/3756611843 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fa6980390a0 0x7fa69803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:12.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.490+0000 7fa6af7fe700 1 --2- 172.21.15.7:0/3756611843 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fa6980390a0 0x7fa69803b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4006fd0 tx=0x7fa6a4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:12.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.494+0000 7fa6ad7fa700 1 -- 172.21.15.7:0/3756611843 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa6a0016260 con 0x7fa6b0108990 2024-06-10T00:54:12.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.702+0000 7fa6b6a53700 1 -- 172.21.15.7:0/3756611843 --> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fa6b01035b0 con 0x7fa6980390a0 2024-06-10T00:54:12.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.703+0000 7fa6ad7fa700 1 -- 172.21.15.7:0/3756611843 <== mgr.14120 v2:172.21.15.7:6800/3298829389 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fa6b01035b0 con 0x7fa6980390a0 2024-06-10T00:54:12.240 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.705+0000 7fa6b6a53700 1 -- 172.21.15.7:0/3756611843 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fa6980390a0 msgr2=0x7fa69803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:12.240 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.705+0000 7fa6b6a53700 1 --2- 172.21.15.7:0/3756611843 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fa6980390a0 0x7fa69803b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4006fd0 tx=0x7fa6a4006e40 comp rx=0 tx=0).stop 2024-06-10T00:54:12.240 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.705+0000 7fa6b6a53700 1 -- 172.21.15.7:0/3756611843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b0108990 msgr2=0x7fa6b019bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:12.240 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.705+0000 7fa6b6a53700 1 --2- 172.21.15.7:0/3756611843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b0108990 0x7fa6b019bdb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0011b60 tx=0x7fa6a0011c40 comp rx=0 tx=0).stop 2024-06-10T00:54:12.240 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.705+0000 7fa6b6a53700 1 -- 172.21.15.7:0/3756611843 shutdown_connections 2024-06-10T00:54:12.240 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.705+0000 7fa6b6a53700 1 --2- 172.21.15.7:0/3756611843 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fa6980390a0 0x7fa69803b560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:12.240 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.705+0000 7fa6b6a53700 1 --2- 172.21.15.7:0/3756611843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b0108990 0x7fa6b019bdb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:12.240 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.705+0000 7fa6b6a53700 1 -- 172.21.15.7:0/3756611843 >> 172.21.15.7:0/3756611843 conn(0x7fa6b0102340 msgr2=0x7fa6b0102ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:12.240 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.705+0000 7fa6b6a53700 1 -- 172.21.15.7:0/3756611843 shutdown_connections 2024-06-10T00:54:12.240 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:11.705+0000 7fa6b6a53700 1 -- 172.21.15.7:0/3756611843 wait complete. 2024-06-10T00:54:12.240 INFO:teuthology.orchestra.run.smithi007.stdout:Generating ssh key... 2024-06-10T00:54:13.474 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:13 smithi007 ceph-mon[32398]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:54:13.474 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:13 smithi007 ceph-mon[32398]: Generating ssh key... 2024-06-10T00:54:13.474 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:13 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: Generating public/private rsa key pair. 2024-06-10T00:54:13.474 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:13 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: Your identification has been saved in /tmp/tmpn6skr10i/key. 2024-06-10T00:54:13.474 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:13 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: Your public key has been saved in /tmp/tmpn6skr10i/key.pub. 2024-06-10T00:54:13.475 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:13 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: The key fingerprint is: 2024-06-10T00:54:13.475 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:13 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: SHA256:syDJjBMUp0LtDAJXpwayxTBsSX5SzCNOgA78a6xW7Hg ceph-bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T00:54:13.475 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:13 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: The key's randomart image is: 2024-06-10T00:54:13.475 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:13 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: +---[RSA 3072]----+ 2024-06-10T00:54:13.475 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:13 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: |&=@+o . | 2024-06-10T00:54:13.475 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:13 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: |*/oO o | 2024-06-10T00:54:13.475 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:13 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: |XoO.+ | 2024-06-10T00:54:13.475 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:13 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: | +oX . | 2024-06-10T00:54:13.475 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:13 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: | = * . S | 2024-06-10T00:54:13.475 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:13 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: | B . . o | 2024-06-10T00:54:13.475 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:13 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: | * . | 2024-06-10T00:54:13.475 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:13 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: | + E | 2024-06-10T00:54:13.475 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:13 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: |. . | 2024-06-10T00:54:13.476 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:13 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: +----[SHA256]-----+ 2024-06-10T00:54:13.749 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.666+0000 7f77b1517700 1 Processor -- start 2024-06-10T00:54:13.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.666+0000 7f77b1517700 1 -- start start 2024-06-10T00:54:13.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.667+0000 7f77b1517700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77ac108440 0x7f77ac108860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:13.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.667+0000 7f77b1517700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac108e30 con 0x7f77ac108440 2024-06-10T00:54:13.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.667+0000 7f77aaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77ac108440 0x7f77ac108860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:13.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.667+0000 7f77aaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77ac108440 0x7f77ac108860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40382/0 (socket says 172.21.15.7:40382) 2024-06-10T00:54:13.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.667+0000 7f77aaffd700 1 -- 172.21.15.7:0/3395661404 learned_addr learned my addr 172.21.15.7:0/3395661404 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:13.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.667+0000 7f77aaffd700 1 -- 172.21.15.7:0/3395661404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77ac105f40 con 0x7f77ac108440 2024-06-10T00:54:13.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.667+0000 7f77aaffd700 1 --2- 172.21.15.7:0/3395661404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77ac108440 0x7f77ac108860 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f779c0098a0 tx=0x7f779c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b0d23494b06a49 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:13.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.667+0000 7f77a9ffb700 1 -- 172.21.15.7:0/3395661404 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f779c004160 con 0x7f77ac108440 2024-06-10T00:54:13.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.668+0000 7f77a9ffb700 1 -- 172.21.15.7:0/3395661404 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f779c004300 con 0x7f77ac108440 2024-06-10T00:54:13.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.668+0000 7f77a9ffb700 1 -- 172.21.15.7:0/3395661404 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f779c011b50 con 0x7f77ac108440 2024-06-10T00:54:13.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.668+0000 7f77b1517700 1 -- 172.21.15.7:0/3395661404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77ac108440 msgr2=0x7f77ac108860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:13.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.668+0000 7f77b1517700 1 --2- 172.21.15.7:0/3395661404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77ac108440 0x7f77ac108860 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f779c0098a0 tx=0x7f779c009bb0 comp rx=0 tx=0).stop 2024-06-10T00:54:13.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.668+0000 7f77b1517700 1 -- 172.21.15.7:0/3395661404 shutdown_connections 2024-06-10T00:54:13.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.668+0000 7f77b1517700 1 --2- 172.21.15.7:0/3395661404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77ac108440 0x7f77ac108860 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:13.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.668+0000 7f77b1517700 1 -- 172.21.15.7:0/3395661404 >> 172.21.15.7:0/3395661404 conn(0x7f77ac101ae0 msgr2=0x7f77ac103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:13.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.668+0000 7f77b1517700 1 -- 172.21.15.7:0/3395661404 shutdown_connections 2024-06-10T00:54:13.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.668+0000 7f77b1517700 1 -- 172.21.15.7:0/3395661404 wait complete. 2024-06-10T00:54:13.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.669+0000 7f77b1517700 1 Processor -- start 2024-06-10T00:54:13.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.669+0000 7f77b1517700 1 -- start start 2024-06-10T00:54:13.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.669+0000 7f77b1517700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77ac108440 0x7f77ac19df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:13.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.669+0000 7f77b1517700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac106f50 con 0x7f77ac108440 2024-06-10T00:54:13.752 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.669+0000 7f77aaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77ac108440 0x7f77ac19df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:13.752 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.669+0000 7f77aaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77ac108440 0x7f77ac19df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40396/0 (socket says 172.21.15.7:40396) 2024-06-10T00:54:13.752 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.669+0000 7f77aaffd700 1 -- 172.21.15.7:0/1933978804 learned_addr learned my addr 172.21.15.7:0/1933978804 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:13.752 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.670+0000 7f77aaffd700 1 -- 172.21.15.7:0/1933978804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77ac19e4b0 con 0x7f77ac108440 2024-06-10T00:54:13.752 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.670+0000 7f77aaffd700 1 --2- 172.21.15.7:0/1933978804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77ac108440 0x7f77ac19df70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f779c011670 tx=0x7f779c011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:13.752 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.670+0000 7f7793fff700 1 -- 172.21.15.7:0/1933978804 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f779c0039c0 con 0x7f77ac108440 2024-06-10T00:54:13.752 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.670+0000 7f7793fff700 1 -- 172.21.15.7:0/1933978804 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f779c011e50 con 0x7f77ac108440 2024-06-10T00:54:13.752 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.670+0000 7f77b1517700 1 -- 172.21.15.7:0/1933978804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77ac19e740 con 0x7f77ac108440 2024-06-10T00:54:13.752 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.670+0000 7f7793fff700 1 -- 172.21.15.7:0/1933978804 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f779c01a5c0 con 0x7f77ac108440 2024-06-10T00:54:13.752 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.670+0000 7f77b1517700 1 -- 172.21.15.7:0/1933978804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77ac1a1430 con 0x7f77ac108440 2024-06-10T00:54:13.752 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.671+0000 7f7793fff700 1 -- 172.21.15.7:0/1933978804 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f779c01a8c0 con 0x7f77ac108440 2024-06-10T00:54:13.752 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.671+0000 7f77b1517700 1 -- 172.21.15.7:0/1933978804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77ac063af0 con 0x7f77ac108440 2024-06-10T00:54:13.752 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.672+0000 7f7793fff700 1 --2- 172.21.15.7:0/1933978804 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f7794039000 0x7f779403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:13.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.672+0000 7f7793fff700 1 -- 172.21.15.7:0/1933978804 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f779c04d5b0 con 0x7f77ac108440 2024-06-10T00:54:13.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.672+0000 7f77aa7fc700 1 --2- 172.21.15.7:0/1933978804 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f7794039000 0x7f779403b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:13.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.672+0000 7f77aa7fc700 1 --2- 172.21.15.7:0/1933978804 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f7794039000 0x7f779403b4c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f77a0006fd0 tx=0x7f77a0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:13.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.677+0000 7f7793fff700 1 -- 172.21.15.7:0/1933978804 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f779c019c70 con 0x7f77ac108440 2024-06-10T00:54:13.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:12.882+0000 7f77b1517700 1 -- 172.21.15.7:0/1933978804 --> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f77ac0628d0 con 0x7f7794039000 2024-06-10T00:54:13.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:13.180+0000 7f7793fff700 1 -- 172.21.15.7:0/1933978804 <== mgr.14120 v2:172.21.15.7:6800/3298829389 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f77ac0628d0 con 0x7f7794039000 2024-06-10T00:54:13.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:13.182+0000 7f77b1517700 1 -- 172.21.15.7:0/1933978804 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f7794039000 msgr2=0x7f779403b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:13.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:13.182+0000 7f77b1517700 1 --2- 172.21.15.7:0/1933978804 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f7794039000 0x7f779403b4c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f77a0006fd0 tx=0x7f77a0006e40 comp rx=0 tx=0).stop 2024-06-10T00:54:13.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:13.182+0000 7f77b1517700 1 -- 172.21.15.7:0/1933978804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77ac108440 msgr2=0x7f77ac19df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:13.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:13.182+0000 7f77b1517700 1 --2- 172.21.15.7:0/1933978804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77ac108440 0x7f77ac19df70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f779c011670 tx=0x7f779c011b60 comp rx=0 tx=0).stop 2024-06-10T00:54:13.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:13.182+0000 7f77b1517700 1 -- 172.21.15.7:0/1933978804 shutdown_connections 2024-06-10T00:54:13.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:13.182+0000 7f77b1517700 1 --2- 172.21.15.7:0/1933978804 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f7794039000 0x7f779403b4c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:13.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:13.182+0000 7f77b1517700 1 --2- 172.21.15.7:0/1933978804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77ac108440 0x7f77ac19df70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:13.754 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:13.182+0000 7f77b1517700 1 -- 172.21.15.7:0/1933978804 >> 172.21.15.7:0/1933978804 conn(0x7f77ac101ae0 msgr2=0x7f77ac194630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:13.754 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:13.182+0000 7f77b1517700 1 -- 172.21.15.7:0/1933978804 shutdown_connections 2024-06-10T00:54:13.754 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:13.182+0000 7f77b1517700 1 -- 172.21.15.7:0/1933978804 wait complete. 2024-06-10T00:54:14.380 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:14 smithi007 ceph-mon[32398]: from='mgr.14120 172.21.15.7:0/675400875' entity='mgr.x' 2024-06-10T00:54:14.380 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:14 smithi007 ceph-mon[32398]: from='mgr.14120 172.21.15.7:0/675400875' entity='mgr.x' 2024-06-10T00:54:14.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrNSN1hwwYXdkBHnqcKM/Xx/5XPh3yb5JrIq4V63yp/JkF3Jv6j9EovvlxTjnMc+mzkCEmLL+FqMvTFqWMTdUaCyHeEsF/akdW+WdJIoWjVZ2ssyez8XKzk06evtLIJbuXDSlwgqcib0Y3SMTbWcpzOa6Lr04cRJW+dDP1Cnr3JjJRgrMJtR1EBbPaqXd9wQTVJzt6HH6Um4hoAsApo9FHkC9gEk3cNWlYjjDkz35qmwGbv8V8uoHeitWlyTbYe1Q/aOL25rZyAOz3k/y8g+CDvV9ob7WujoAdlPMdFxhjRx+QGAeRXHcgXlPLAt/I1GbVV+XZV3mAJaTVMA0PtAtQG60o+PRfdAvsxfgCFYyHZWWpTepCR2tYGPFwjLwq3JHeJu6liJUAP75cX1DiVqNG2RdY2fwouyZDQYRZevAm5x8XnVdcEc5swIZjRQMntQ6Josh9F1nwE966l4kK7+kbqRoTAyUAPMFLS9cO7KXGgbLrBH2rKVsz8W38T5Qusgc= ceph-bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T00:54:14.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.161+0000 7fabef6da700 1 Processor -- start 2024-06-10T00:54:14.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.162+0000 7fabef6da700 1 -- start start 2024-06-10T00:54:14.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.162+0000 7fabef6da700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabe807e760 0x7fabe807cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:14.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.162+0000 7fabef6da700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fabe807d190 con 0x7fabe807e760 2024-06-10T00:54:14.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.162+0000 7fabed476700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabe807e760 0x7fabe807cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:14.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.162+0000 7fabed476700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabe807e760 0x7fabe807cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40400/0 (socket says 172.21.15.7:40400) 2024-06-10T00:54:14.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.162+0000 7fabed476700 1 -- 172.21.15.7:0/1175021987 learned_addr learned my addr 172.21.15.7:0/1175021987 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:14.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.162+0000 7fabed476700 1 -- 172.21.15.7:0/1175021987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe807d310 con 0x7fabe807e760 2024-06-10T00:54:14.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.163+0000 7fabed476700 1 --2- 172.21.15.7:0/1175021987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabe807e760 0x7fabe807cbc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fabe40098a0 tx=0x7fabe4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83bd694881a2db22 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:14.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.163+0000 7fabdffff700 1 -- 172.21.15.7:0/1175021987 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabe4004160 con 0x7fabe807e760 2024-06-10T00:54:14.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.163+0000 7fabdffff700 1 -- 172.21.15.7:0/1175021987 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fabe4004300 con 0x7fabe807e760 2024-06-10T00:54:14.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.163+0000 7fabdffff700 1 -- 172.21.15.7:0/1175021987 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabe4011b50 con 0x7fabe807e760 2024-06-10T00:54:14.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.163+0000 7fabef6da700 1 -- 172.21.15.7:0/1175021987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabe807e760 msgr2=0x7fabe807cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:14.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.163+0000 7fabef6da700 1 --2- 172.21.15.7:0/1175021987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabe807e760 0x7fabe807cbc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fabe40098a0 tx=0x7fabe4009bb0 comp rx=0 tx=0).stop 2024-06-10T00:54:14.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.164+0000 7fabef6da700 1 -- 172.21.15.7:0/1175021987 shutdown_connections 2024-06-10T00:54:14.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.164+0000 7fabef6da700 1 --2- 172.21.15.7:0/1175021987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabe807e760 0x7fabe807cbc0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:14.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.164+0000 7fabef6da700 1 -- 172.21.15.7:0/1175021987 >> 172.21.15.7:0/1175021987 conn(0x7fabe8103460 msgr2=0x7fabe8105880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:14.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.164+0000 7fabef6da700 1 -- 172.21.15.7:0/1175021987 shutdown_connections 2024-06-10T00:54:14.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.164+0000 7fabef6da700 1 -- 172.21.15.7:0/1175021987 wait complete. 2024-06-10T00:54:14.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.164+0000 7fabef6da700 1 Processor -- start 2024-06-10T00:54:14.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.164+0000 7fabef6da700 1 -- start start 2024-06-10T00:54:14.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.164+0000 7fabef6da700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabe807e760 0x7fabe819e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:14.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.164+0000 7fabef6da700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fabe810e430 con 0x7fabe807e760 2024-06-10T00:54:14.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.165+0000 7fabed476700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabe807e760 0x7fabe819e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:14.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.165+0000 7fabed476700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabe807e760 0x7fabe819e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40410/0 (socket says 172.21.15.7:40410) 2024-06-10T00:54:14.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.165+0000 7fabed476700 1 -- 172.21.15.7:0/1639331643 learned_addr learned my addr 172.21.15.7:0/1639331643 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:14.941 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.165+0000 7fabed476700 1 -- 172.21.15.7:0/1639331643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe819e540 con 0x7fabe807e760 2024-06-10T00:54:14.941 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.165+0000 7fabed476700 1 --2- 172.21.15.7:0/1639331643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabe807e760 0x7fabe819e000 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fabe4011850 tx=0x7fabe4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:14.941 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.165+0000 7fabde7fc700 1 -- 172.21.15.7:0/1639331643 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabe4003af0 con 0x7fabe807e760 2024-06-10T00:54:14.941 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.165+0000 7fabde7fc700 1 -- 172.21.15.7:0/1639331643 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fabe4020070 con 0x7fabe807e760 2024-06-10T00:54:14.941 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.165+0000 7fabef6da700 1 -- 172.21.15.7:0/1639331643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe819e7d0 con 0x7fabe807e760 2024-06-10T00:54:14.941 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.165+0000 7fabde7fc700 1 -- 172.21.15.7:0/1639331643 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabe40195a0 con 0x7fabe807e760 2024-06-10T00:54:14.941 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.165+0000 7fabef6da700 1 -- 172.21.15.7:0/1639331643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe81a14c0 con 0x7fabe807e760 2024-06-10T00:54:14.941 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.166+0000 7fabde7fc700 1 -- 172.21.15.7:0/1639331643 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fabe4019850 con 0x7fabe807e760 2024-06-10T00:54:14.941 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.166+0000 7fabef6da700 1 -- 172.21.15.7:0/1639331643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabe8063af0 con 0x7fabe807e760 2024-06-10T00:54:14.941 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.166+0000 7fabde7fc700 1 --2- 172.21.15.7:0/1639331643 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fabd40390a0 0x7fabd403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:14.941 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.166+0000 7fabde7fc700 1 -- 172.21.15.7:0/1639331643 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fabe4011e00 con 0x7fabe807e760 2024-06-10T00:54:14.941 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.166+0000 7fabecc75700 1 --2- 172.21.15.7:0/1639331643 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fabd40390a0 0x7fabd403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:14.941 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.167+0000 7fabecc75700 1 --2- 172.21.15.7:0/1639331643 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fabd40390a0 0x7fabd403b560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fabd8006fd0 tx=0x7fabd8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:14.942 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.171+0000 7fabde7fc700 1 -- 172.21.15.7:0/1639331643 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fabe4003c90 con 0x7fabe807e760 2024-06-10T00:54:14.942 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.379+0000 7fabef6da700 1 -- 172.21.15.7:0/1639331643 --> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fabe8104670 con 0x7fabd40390a0 2024-06-10T00:54:14.942 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.380+0000 7fabde7fc700 1 -- 172.21.15.7:0/1639331643 <== mgr.14120 v2:172.21.15.7:6800/3298829389 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fabe8104670 con 0x7fabd40390a0 2024-06-10T00:54:14.942 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.383+0000 7fabef6da700 1 -- 172.21.15.7:0/1639331643 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fabd40390a0 msgr2=0x7fabd403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:14.942 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.383+0000 7fabef6da700 1 --2- 172.21.15.7:0/1639331643 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fabd40390a0 0x7fabd403b560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fabd8006fd0 tx=0x7fabd8006e40 comp rx=0 tx=0).stop 2024-06-10T00:54:14.942 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.383+0000 7fabef6da700 1 -- 172.21.15.7:0/1639331643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabe807e760 msgr2=0x7fabe819e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:14.942 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.383+0000 7fabef6da700 1 --2- 172.21.15.7:0/1639331643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabe807e760 0x7fabe819e000 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fabe4011850 tx=0x7fabe4011c10 comp rx=0 tx=0).stop 2024-06-10T00:54:14.942 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.383+0000 7fabef6da700 1 -- 172.21.15.7:0/1639331643 shutdown_connections 2024-06-10T00:54:14.942 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.383+0000 7fabef6da700 1 --2- 172.21.15.7:0/1639331643 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fabd40390a0 0x7fabd403b560 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:14.942 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.383+0000 7fabef6da700 1 --2- 172.21.15.7:0/1639331643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabe807e760 0x7fabe819e000 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:14.942 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.383+0000 7fabef6da700 1 -- 172.21.15.7:0/1639331643 >> 172.21.15.7:0/1639331643 conn(0x7fabe8103460 msgr2=0x7fabe8103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:14.942 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.383+0000 7fabef6da700 1 -- 172.21.15.7:0/1639331643 shutdown_connections 2024-06-10T00:54:14.942 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:14.383+0000 7fabef6da700 1 -- 172.21.15.7:0/1639331643 wait complete. 2024-06-10T00:54:14.943 INFO:teuthology.orchestra.run.smithi007.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-06-10T00:54:14.943 INFO:teuthology.orchestra.run.smithi007.stdout:Adding key to root@localhost authorized_keys... 2024-06-10T00:54:14.943 INFO:teuthology.orchestra.run.smithi007.stdout:Adding host smithi007... 2024-06-10T00:54:15.259 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:15 smithi007 ceph-mon[32398]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:54:16.509 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:16 smithi007 ceph-mon[32398]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi007", "addr": "172.21.15.7", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:54:17.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:17 smithi007 ceph-mon[32398]: Deploying cephadm binary to smithi007 2024-06-10T00:54:20.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Added host 'smithi007' with addr '172.21.15.7' 2024-06-10T00:54:20.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.372+0000 7fdf61310700 1 Processor -- start 2024-06-10T00:54:20.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.372+0000 7fdf61310700 1 -- start start 2024-06-10T00:54:20.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.373+0000 7fdf61310700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf5c07e6b0 0x7fdf5c07cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:20.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.373+0000 7fdf61310700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c07d0a0 con 0x7fdf5c07e6b0 2024-06-10T00:54:20.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.373+0000 7fdf5affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf5c07e6b0 0x7fdf5c07cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:20.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.373+0000 7fdf5affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf5c07e6b0 0x7fdf5c07cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40414/0 (socket says 172.21.15.7:40414) 2024-06-10T00:54:20.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.373+0000 7fdf5affd700 1 -- 172.21.15.7:0/1869267208 learned_addr learned my addr 172.21.15.7:0/1869267208 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:20.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.373+0000 7fdf5affd700 1 -- 172.21.15.7:0/1869267208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5c07d220 con 0x7fdf5c07e6b0 2024-06-10T00:54:20.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.373+0000 7fdf5affd700 1 --2- 172.21.15.7:0/1869267208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf5c07e6b0 0x7fdf5c07cb60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c0098a0 tx=0x7fdf4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6daa97307eaf3e41 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:20.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.374+0000 7fdf59ffb700 1 -- 172.21.15.7:0/1869267208 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf4c004160 con 0x7fdf5c07e6b0 2024-06-10T00:54:20.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.374+0000 7fdf59ffb700 1 -- 172.21.15.7:0/1869267208 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdf4c004300 con 0x7fdf5c07e6b0 2024-06-10T00:54:20.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.374+0000 7fdf59ffb700 1 -- 172.21.15.7:0/1869267208 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf4c011b50 con 0x7fdf5c07e6b0 2024-06-10T00:54:20.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.374+0000 7fdf61310700 1 -- 172.21.15.7:0/1869267208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf5c07e6b0 msgr2=0x7fdf5c07cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:20.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.374+0000 7fdf61310700 1 --2- 172.21.15.7:0/1869267208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf5c07e6b0 0x7fdf5c07cb60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c0098a0 tx=0x7fdf4c009bb0 comp rx=0 tx=0).stop 2024-06-10T00:54:20.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.374+0000 7fdf61310700 1 -- 172.21.15.7:0/1869267208 shutdown_connections 2024-06-10T00:54:20.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.374+0000 7fdf61310700 1 --2- 172.21.15.7:0/1869267208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf5c07e6b0 0x7fdf5c07cb60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:20.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.374+0000 7fdf61310700 1 -- 172.21.15.7:0/1869267208 >> 172.21.15.7:0/1869267208 conn(0x7fdf5c103470 msgr2=0x7fdf5c1058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:20.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.374+0000 7fdf61310700 1 -- 172.21.15.7:0/1869267208 shutdown_connections 2024-06-10T00:54:20.366 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.374+0000 7fdf61310700 1 -- 172.21.15.7:0/1869267208 wait complete. 2024-06-10T00:54:20.366 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.375+0000 7fdf61310700 1 Processor -- start 2024-06-10T00:54:20.366 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.375+0000 7fdf61310700 1 -- start start 2024-06-10T00:54:20.366 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.375+0000 7fdf61310700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf5c07e6b0 0x7fdf5c19dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:20.366 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.375+0000 7fdf61310700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c10e400 con 0x7fdf5c07e6b0 2024-06-10T00:54:20.366 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.375+0000 7fdf5affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf5c07e6b0 0x7fdf5c19dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:20.366 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.375+0000 7fdf5affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf5c07e6b0 0x7fdf5c19dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40426/0 (socket says 172.21.15.7:40426) 2024-06-10T00:54:20.366 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.375+0000 7fdf5affd700 1 -- 172.21.15.7:0/3560080457 learned_addr learned my addr 172.21.15.7:0/3560080457 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:20.366 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.376+0000 7fdf5affd700 1 -- 172.21.15.7:0/3560080457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5c19e4f0 con 0x7fdf5c07e6b0 2024-06-10T00:54:20.366 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.376+0000 7fdf5affd700 1 --2- 172.21.15.7:0/3560080457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf5c07e6b0 0x7fdf5c19dfb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c011b60 tx=0x7fdf4c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:20.366 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.376+0000 7fdf43fff700 1 -- 172.21.15.7:0/3560080457 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf4c003b20 con 0x7fdf5c07e6b0 2024-06-10T00:54:20.366 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.376+0000 7fdf61310700 1 -- 172.21.15.7:0/3560080457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf5c19e780 con 0x7fdf5c07e6b0 2024-06-10T00:54:20.367 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.376+0000 7fdf43fff700 1 -- 172.21.15.7:0/3560080457 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdf4c003cc0 con 0x7fdf5c07e6b0 2024-06-10T00:54:20.367 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.376+0000 7fdf43fff700 1 -- 172.21.15.7:0/3560080457 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf4c024840 con 0x7fdf5c07e6b0 2024-06-10T00:54:20.367 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.376+0000 7fdf61310700 1 -- 172.21.15.7:0/3560080457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf5c1a1470 con 0x7fdf5c07e6b0 2024-06-10T00:54:20.367 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.377+0000 7fdf61310700 1 -- 172.21.15.7:0/3560080457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf5c063af0 con 0x7fdf5c07e6b0 2024-06-10T00:54:20.367 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.377+0000 7fdf43fff700 1 -- 172.21.15.7:0/3560080457 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fdf4c022020 con 0x7fdf5c07e6b0 2024-06-10T00:54:20.367 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.377+0000 7fdf43fff700 1 --2- 172.21.15.7:0/3560080457 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fdf44039050 0x7fdf4403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:20.367 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.377+0000 7fdf43fff700 1 -- 172.21.15.7:0/3560080457 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdf4c011e00 con 0x7fdf5c07e6b0 2024-06-10T00:54:20.367 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.378+0000 7fdf5a7fc700 1 --2- 172.21.15.7:0/3560080457 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fdf44039050 0x7fdf4403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:20.367 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.378+0000 7fdf5a7fc700 1 --2- 172.21.15.7:0/3560080457 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fdf44039050 0x7fdf4403b510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdf50006fd0 tx=0x7fdf50006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:20.367 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.382+0000 7fdf43fff700 1 -- 172.21.15.7:0/3560080457 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdf4c024bc0 con 0x7fdf5c07e6b0 2024-06-10T00:54:20.367 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:15.594+0000 7fdf61310700 1 -- 172.21.15.7:0/3560080457 --> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi007", "addr": "172.21.15.7", "target": ["mon-mgr", ""]}) -- 0x7fdf5c1046a0 con 0x7fdf44039050 2024-06-10T00:54:20.367 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:19.847+0000 7fdf43fff700 1 -- 172.21.15.7:0/3560080457 <== mgr.14120 v2:172.21.15.7:6800/3298829389 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7fdf5c1046a0 con 0x7fdf44039050 2024-06-10T00:54:20.367 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:19.850+0000 7fdf61310700 1 -- 172.21.15.7:0/3560080457 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fdf44039050 msgr2=0x7fdf4403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:20.368 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:19.850+0000 7fdf61310700 1 --2- 172.21.15.7:0/3560080457 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fdf44039050 0x7fdf4403b510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdf50006fd0 tx=0x7fdf50006e40 comp rx=0 tx=0).stop 2024-06-10T00:54:20.368 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:19.850+0000 7fdf61310700 1 -- 172.21.15.7:0/3560080457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf5c07e6b0 msgr2=0x7fdf5c19dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:20.368 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:19.850+0000 7fdf61310700 1 --2- 172.21.15.7:0/3560080457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf5c07e6b0 0x7fdf5c19dfb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c011b60 tx=0x7fdf4c011c40 comp rx=0 tx=0).stop 2024-06-10T00:54:20.368 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:19.850+0000 7fdf61310700 1 -- 172.21.15.7:0/3560080457 shutdown_connections 2024-06-10T00:54:20.368 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:19.850+0000 7fdf61310700 1 --2- 172.21.15.7:0/3560080457 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fdf44039050 0x7fdf4403b510 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:20.368 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:19.850+0000 7fdf61310700 1 --2- 172.21.15.7:0/3560080457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf5c07e6b0 0x7fdf5c19dfb0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:20.368 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:19.850+0000 7fdf61310700 1 -- 172.21.15.7:0/3560080457 >> 172.21.15.7:0/3560080457 conn(0x7fdf5c103470 msgr2=0x7fdf5c103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:20.368 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:19.850+0000 7fdf61310700 1 -- 172.21.15.7:0/3560080457 shutdown_connections 2024-06-10T00:54:20.368 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:19.850+0000 7fdf61310700 1 -- 172.21.15.7:0/3560080457 wait complete. 2024-06-10T00:54:20.368 INFO:teuthology.orchestra.run.smithi007.stdout:Deploying unmanaged mon service... 2024-06-10T00:54:21.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:20 smithi007 ceph-mon[32398]: from='mgr.14120 172.21.15.7:0/675400875' entity='mgr.x' 2024-06-10T00:54:21.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:20 smithi007 ceph-mon[32398]: Added host smithi007 2024-06-10T00:54:21.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:20 smithi007 ceph-mon[32398]: from='mgr.14120 172.21.15.7:0/675400875' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:54:21.724 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-06-10T00:54:21.724 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.848+0000 7f6cfc171700 1 Processor -- start 2024-06-10T00:54:21.724 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.848+0000 7f6cfc171700 1 -- start start 2024-06-10T00:54:21.724 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.849+0000 7f6cfc171700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cf4106d90 0x7f6cf41071b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:21.724 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.849+0000 7f6cfc171700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf4107780 con 0x7f6cf4106d90 2024-06-10T00:54:21.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.849+0000 7f6cf9f0d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cf4106d90 0x7f6cf41071b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:21.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.849+0000 7f6cf9f0d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cf4106d90 0x7f6cf41071b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40428/0 (socket says 172.21.15.7:40428) 2024-06-10T00:54:21.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.849+0000 7f6cf9f0d700 1 -- 172.21.15.7:0/2704169167 learned_addr learned my addr 172.21.15.7:0/2704169167 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:21.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.849+0000 7f6cf9f0d700 1 -- 172.21.15.7:0/2704169167 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf4107f90 con 0x7f6cf4106d90 2024-06-10T00:54:21.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.849+0000 7f6cf9f0d700 1 --2- 172.21.15.7:0/2704169167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cf4106d90 0x7f6cf41071b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0009e80 tx=0x7f6cf000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8795969ebd8e7a6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:21.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.849+0000 7f6cf8f0b700 1 -- 172.21.15.7:0/2704169167 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cf0017070 con 0x7f6cf4106d90 2024-06-10T00:54:21.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.849+0000 7f6cf8f0b700 1 -- 172.21.15.7:0/2704169167 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6cf000b8c0 con 0x7f6cf4106d90 2024-06-10T00:54:21.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.849+0000 7f6cf8f0b700 1 -- 172.21.15.7:0/2704169167 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cf00045c0 con 0x7f6cf4106d90 2024-06-10T00:54:21.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.850+0000 7f6cfc171700 1 -- 172.21.15.7:0/2704169167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cf4106d90 msgr2=0x7f6cf41071b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:21.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.850+0000 7f6cfc171700 1 --2- 172.21.15.7:0/2704169167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cf4106d90 0x7f6cf41071b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0009e80 tx=0x7f6cf000b330 comp rx=0 tx=0).stop 2024-06-10T00:54:21.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.850+0000 7f6cfc171700 1 -- 172.21.15.7:0/2704169167 shutdown_connections 2024-06-10T00:54:21.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.850+0000 7f6cfc171700 1 --2- 172.21.15.7:0/2704169167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cf4106d90 0x7f6cf41071b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:21.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.850+0000 7f6cfc171700 1 -- 172.21.15.7:0/2704169167 >> 172.21.15.7:0/2704169167 conn(0x7f6cf4102310 msgr2=0x7f6cf4104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:21.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.850+0000 7f6cfc171700 1 -- 172.21.15.7:0/2704169167 shutdown_connections 2024-06-10T00:54:21.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.850+0000 7f6cfc171700 1 -- 172.21.15.7:0/2704169167 wait complete. 2024-06-10T00:54:21.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.851+0000 7f6cfc171700 1 Processor -- start 2024-06-10T00:54:21.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.851+0000 7f6cfc171700 1 -- start start 2024-06-10T00:54:21.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.851+0000 7f6cfc171700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cf4106d90 0x7f6cf419bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:21.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.851+0000 7f6cfc171700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf410b340 con 0x7f6cf4106d90 2024-06-10T00:54:21.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.851+0000 7f6cf9f0d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cf4106d90 0x7f6cf419bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:21.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.851+0000 7f6cf9f0d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cf4106d90 0x7f6cf419bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40430/0 (socket says 172.21.15.7:40430) 2024-06-10T00:54:21.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.851+0000 7f6cf9f0d700 1 -- 172.21.15.7:0/884059659 learned_addr learned my addr 172.21.15.7:0/884059659 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:21.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.851+0000 7f6cf9f0d700 1 -- 172.21.15.7:0/884059659 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf419c2f0 con 0x7f6cf4106d90 2024-06-10T00:54:21.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.851+0000 7f6cf9f0d700 1 --2- 172.21.15.7:0/884059659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cf4106d90 0x7f6cf419bdb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0009530 tx=0x7f6cf000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:21.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.852+0000 7f6ceaffd700 1 -- 172.21.15.7:0/884059659 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cf0020020 con 0x7f6cf4106d90 2024-06-10T00:54:21.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.852+0000 7f6ceaffd700 1 -- 172.21.15.7:0/884059659 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6cf0003b50 con 0x7f6cf4106d90 2024-06-10T00:54:21.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.852+0000 7f6ceaffd700 1 -- 172.21.15.7:0/884059659 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cf0017040 con 0x7f6cf4106d90 2024-06-10T00:54:21.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.852+0000 7f6cfc171700 1 -- 172.21.15.7:0/884059659 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cf419c580 con 0x7f6cf4106d90 2024-06-10T00:54:21.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.852+0000 7f6cfc171700 1 -- 172.21.15.7:0/884059659 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cf419f230 con 0x7f6cf4106d90 2024-06-10T00:54:21.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.852+0000 7f6ceaffd700 1 -- 172.21.15.7:0/884059659 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f6cf0023020 con 0x7f6cf4106d90 2024-06-10T00:54:21.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.853+0000 7f6cfc171700 1 -- 172.21.15.7:0/884059659 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cf4063af0 con 0x7f6cf4106d90 2024-06-10T00:54:21.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.853+0000 7f6ceaffd700 1 --2- 172.21.15.7:0/884059659 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f6ce00390a0 0x7f6ce003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:21.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.853+0000 7f6ceaffd700 1 -- 172.21.15.7:0/884059659 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6cf000f050 con 0x7f6cf4106d90 2024-06-10T00:54:21.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.853+0000 7f6cf970c700 1 --2- 172.21.15.7:0/884059659 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f6ce00390a0 0x7f6ce003b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:21.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.853+0000 7f6cf970c700 1 --2- 172.21.15.7:0/884059659 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f6ce00390a0 0x7f6ce003b560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4006fd0 tx=0x7f6ce4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:21.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:20.858+0000 7f6ceaffd700 1 -- 172.21.15.7:0/884059659 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6cf002a020 con 0x7f6cf4106d90 2024-06-10T00:54:21.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:21.082+0000 7f6cfc171700 1 -- 172.21.15.7:0/884059659 --> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f6cf402e040 con 0x7f6ce00390a0 2024-06-10T00:54:21.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:21.087+0000 7f6ceaffd700 1 -- 172.21.15.7:0/884059659 <== mgr.14120 v2:172.21.15.7:6800/3298829389 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6cf402e040 con 0x7f6ce00390a0 2024-06-10T00:54:21.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:21.089+0000 7f6cfc171700 1 -- 172.21.15.7:0/884059659 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f6ce00390a0 msgr2=0x7f6ce003b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:21.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:21.089+0000 7f6cfc171700 1 --2- 172.21.15.7:0/884059659 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f6ce00390a0 0x7f6ce003b560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4006fd0 tx=0x7f6ce4006e40 comp rx=0 tx=0).stop 2024-06-10T00:54:21.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:21.089+0000 7f6cfc171700 1 -- 172.21.15.7:0/884059659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cf4106d90 msgr2=0x7f6cf419bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:21.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:21.089+0000 7f6cfc171700 1 --2- 172.21.15.7:0/884059659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cf4106d90 0x7f6cf419bdb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0009530 tx=0x7f6cf000bd50 comp rx=0 tx=0).stop 2024-06-10T00:54:21.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:21.089+0000 7f6cfc171700 1 -- 172.21.15.7:0/884059659 shutdown_connections 2024-06-10T00:54:21.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:21.089+0000 7f6cfc171700 1 --2- 172.21.15.7:0/884059659 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f6ce00390a0 0x7f6ce003b560 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:21.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:21.089+0000 7f6cfc171700 1 --2- 172.21.15.7:0/884059659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cf4106d90 0x7f6cf419bdb0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:21.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:21.089+0000 7f6cfc171700 1 -- 172.21.15.7:0/884059659 >> 172.21.15.7:0/884059659 conn(0x7f6cf4102310 msgr2=0x7f6cf41031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:21.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:21.089+0000 7f6cfc171700 1 -- 172.21.15.7:0/884059659 shutdown_connections 2024-06-10T00:54:21.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:21.089+0000 7f6cfc171700 1 -- 172.21.15.7:0/884059659 wait complete. 2024-06-10T00:54:21.729 INFO:teuthology.orchestra.run.smithi007.stdout:Deploying unmanaged mgr service... 2024-06-10T00:54:22.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:22 smithi007 ceph-mon[32398]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:54:22.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:22 smithi007 ceph-mon[32398]: Saving service mon spec with placement count:5 2024-06-10T00:54:22.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:22 smithi007 ceph-mon[32398]: from='mgr.14120 172.21.15.7:0/675400875' entity='mgr.x' 2024-06-10T00:54:23.551 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-06-10T00:54:23.551 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.295+0000 7f712848c700 1 Processor -- start 2024-06-10T00:54:23.551 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.295+0000 7f712848c700 1 -- start start 2024-06-10T00:54:23.551 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.295+0000 7f712848c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7120106dc0 0x7f71201071e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:23.551 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.295+0000 7f712848c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f71201077b0 con 0x7f7120106dc0 2024-06-10T00:54:23.551 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.295+0000 7f7126228700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7120106dc0 0x7f71201071e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:23.551 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.295+0000 7f7126228700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7120106dc0 0x7f71201071e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32934/0 (socket says 172.21.15.7:32934) 2024-06-10T00:54:23.551 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.295+0000 7f7126228700 1 -- 172.21.15.7:0/1197031708 learned_addr learned my addr 172.21.15.7:0/1197031708 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:23.552 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.296+0000 7f7126228700 1 -- 172.21.15.7:0/1197031708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7120107f70 con 0x7f7120106dc0 2024-06-10T00:54:23.552 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.296+0000 7f7126228700 1 --2- 172.21.15.7:0/1197031708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7120106dc0 0x7f71201071e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f711c0098a0 tx=0x7f711c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22934abfddd4409e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:23.552 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.296+0000 7f7125226700 1 -- 172.21.15.7:0/1197031708 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f711c004160 con 0x7f7120106dc0 2024-06-10T00:54:23.552 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.296+0000 7f7125226700 1 -- 172.21.15.7:0/1197031708 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f711c004300 con 0x7f7120106dc0 2024-06-10T00:54:23.552 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.296+0000 7f7125226700 1 -- 172.21.15.7:0/1197031708 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f711c011b50 con 0x7f7120106dc0 2024-06-10T00:54:23.552 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.296+0000 7f712848c700 1 -- 172.21.15.7:0/1197031708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7120106dc0 msgr2=0x7f71201071e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:23.552 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.296+0000 7f712848c700 1 --2- 172.21.15.7:0/1197031708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7120106dc0 0x7f71201071e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f711c0098a0 tx=0x7f711c009bb0 comp rx=0 tx=0).stop 2024-06-10T00:54:23.552 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.297+0000 7f712848c700 1 -- 172.21.15.7:0/1197031708 shutdown_connections 2024-06-10T00:54:23.552 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.297+0000 7f712848c700 1 --2- 172.21.15.7:0/1197031708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7120106dc0 0x7f71201071e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:23.552 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.297+0000 7f712848c700 1 -- 172.21.15.7:0/1197031708 >> 172.21.15.7:0/1197031708 conn(0x7f7120102380 msgr2=0x7f71201047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:23.552 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.297+0000 7f712848c700 1 -- 172.21.15.7:0/1197031708 shutdown_connections 2024-06-10T00:54:23.552 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.297+0000 7f712848c700 1 -- 172.21.15.7:0/1197031708 wait complete. 2024-06-10T00:54:23.552 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.297+0000 7f712848c700 1 Processor -- start 2024-06-10T00:54:23.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.297+0000 7f712848c700 1 -- start start 2024-06-10T00:54:23.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.297+0000 7f712848c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7120106dc0 0x7f7120199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:23.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.297+0000 7f712848c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f712007b870 con 0x7f7120106dc0 2024-06-10T00:54:23.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.298+0000 7f7126228700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7120106dc0 0x7f7120199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:23.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.298+0000 7f7126228700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7120106dc0 0x7f7120199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32942/0 (socket says 172.21.15.7:32942) 2024-06-10T00:54:23.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.298+0000 7f7126228700 1 -- 172.21.15.7:0/3533481657 learned_addr learned my addr 172.21.15.7:0/3533481657 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:23.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.298+0000 7f7126228700 1 -- 172.21.15.7:0/3533481657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f712019a0b0 con 0x7f7120106dc0 2024-06-10T00:54:23.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.298+0000 7f7126228700 1 --2- 172.21.15.7:0/3533481657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7120106dc0 0x7f7120199b70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f711c011670 tx=0x7f711c011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:23.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.298+0000 7f71177fe700 1 -- 172.21.15.7:0/3533481657 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f711c0039c0 con 0x7f7120106dc0 2024-06-10T00:54:23.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.298+0000 7f71177fe700 1 -- 172.21.15.7:0/3533481657 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f711c011e50 con 0x7f7120106dc0 2024-06-10T00:54:23.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.298+0000 7f712848c700 1 -- 172.21.15.7:0/3533481657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f712019a340 con 0x7f7120106dc0 2024-06-10T00:54:23.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.298+0000 7f71177fe700 1 -- 172.21.15.7:0/3533481657 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f711c01a5c0 con 0x7f7120106dc0 2024-06-10T00:54:23.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.298+0000 7f712848c700 1 -- 172.21.15.7:0/3533481657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f712019d030 con 0x7f7120106dc0 2024-06-10T00:54:23.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.299+0000 7f71177fe700 1 -- 172.21.15.7:0/3533481657 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f711c024430 con 0x7f7120106dc0 2024-06-10T00:54:23.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.299+0000 7f71177fe700 1 --2- 172.21.15.7:0/3533481657 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f710c03b280 0x7f710c03d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:23.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.299+0000 7f712848c700 1 -- 172.21.15.7:0/3533481657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7120063af0 con 0x7f7120106dc0 2024-06-10T00:54:23.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.299+0000 7f71177fe700 1 -- 172.21.15.7:0/3533481657 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f711c04d4f0 con 0x7f7120106dc0 2024-06-10T00:54:23.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.299+0000 7f7125a27700 1 --2- 172.21.15.7:0/3533481657 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f710c03b280 0x7f710c03d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:23.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.300+0000 7f7125a27700 1 --2- 172.21.15.7:0/3533481657 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f710c03b280 0x7f710c03d740 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7110006fd0 tx=0x7f7110006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:23.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.304+0000 7f71177fe700 1 -- 172.21.15.7:0/3533481657 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f711c02d910 con 0x7f7120106dc0 2024-06-10T00:54:23.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.524+0000 7f712848c700 1 -- 172.21.15.7:0/3533481657 --> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f71200628d0 con 0x7f710c03b280 2024-06-10T00:54:23.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.529+0000 7f71177fe700 1 -- 172.21.15.7:0/3533481657 <== mgr.14120 v2:172.21.15.7:6800/3298829389 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f71200628d0 con 0x7f710c03b280 2024-06-10T00:54:23.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.531+0000 7f712848c700 1 -- 172.21.15.7:0/3533481657 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f710c03b280 msgr2=0x7f710c03d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:23.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.531+0000 7f712848c700 1 --2- 172.21.15.7:0/3533481657 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f710c03b280 0x7f710c03d740 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7110006fd0 tx=0x7f7110006e40 comp rx=0 tx=0).stop 2024-06-10T00:54:23.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.531+0000 7f712848c700 1 -- 172.21.15.7:0/3533481657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7120106dc0 msgr2=0x7f7120199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:23.555 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.531+0000 7f712848c700 1 --2- 172.21.15.7:0/3533481657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7120106dc0 0x7f7120199b70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f711c011670 tx=0x7f711c011b60 comp rx=0 tx=0).stop 2024-06-10T00:54:23.555 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.531+0000 7f712848c700 1 -- 172.21.15.7:0/3533481657 shutdown_connections 2024-06-10T00:54:23.555 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.531+0000 7f712848c700 1 --2- 172.21.15.7:0/3533481657 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f710c03b280 0x7f710c03d740 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:23.555 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.531+0000 7f712848c700 1 --2- 172.21.15.7:0/3533481657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7120106dc0 0x7f7120199b70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:23.555 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.531+0000 7f712848c700 1 -- 172.21.15.7:0/3533481657 >> 172.21.15.7:0/3533481657 conn(0x7f7120102380 msgr2=0x7f7120190190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:23.555 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.532+0000 7f712848c700 1 -- 172.21.15.7:0/3533481657 shutdown_connections 2024-06-10T00:54:23.555 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:22.532+0000 7f712848c700 1 -- 172.21.15.7:0/3533481657 wait complete. 2024-06-10T00:54:23.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:23 smithi007 ceph-mon[32398]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:54:23.795 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:23 smithi007 ceph-mon[32398]: Saving service mgr spec with placement count:2 2024-06-10T00:54:23.795 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:23 smithi007 ceph-mon[32398]: from='mgr.14120 172.21.15.7:0/675400875' entity='mgr.x' 2024-06-10T00:54:23.795 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:23 smithi007 ceph-mon[32398]: from='mgr.14120 172.21.15.7:0/675400875' entity='mgr.x' 2024-06-10T00:54:24.774 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.004+0000 7f7afe8f7700 1 Processor -- start 2024-06-10T00:54:24.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.004+0000 7f7afe8f7700 1 -- start start 2024-06-10T00:54:24.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.004+0000 7f7afe8f7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7af81089a0 0x7f7af810ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:24.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.004+0000 7f7afe8f7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7af807c6a0 con 0x7f7af81089a0 2024-06-10T00:54:24.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.005+0000 7f7af7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7af81089a0 0x7f7af810ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:24.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.005+0000 7f7af7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7af81089a0 0x7f7af810ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32950/0 (socket says 172.21.15.7:32950) 2024-06-10T00:54:24.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.005+0000 7f7af7fff700 1 -- 172.21.15.7:0/3374366012 learned_addr learned my addr 172.21.15.7:0/3374366012 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:24.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.005+0000 7f7af7fff700 1 -- 172.21.15.7:0/3374366012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af810b2d0 con 0x7f7af81089a0 2024-06-10T00:54:24.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.005+0000 7f7af7fff700 1 --2- 172.21.15.7:0/3374366012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7af81089a0 0x7f7af810ad90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80098a0 tx=0x7f7ae8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ec92dd078bed22 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:24.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.005+0000 7f7af6ffd700 1 -- 172.21.15.7:0/3374366012 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ae8004160 con 0x7f7af81089a0 2024-06-10T00:54:24.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.005+0000 7f7af6ffd700 1 -- 172.21.15.7:0/3374366012 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ae8004300 con 0x7f7af81089a0 2024-06-10T00:54:24.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.005+0000 7f7af6ffd700 1 -- 172.21.15.7:0/3374366012 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ae8011b20 con 0x7f7af81089a0 2024-06-10T00:54:24.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.006+0000 7f7afe8f7700 1 -- 172.21.15.7:0/3374366012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7af81089a0 msgr2=0x7f7af810ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:24.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.006+0000 7f7afe8f7700 1 --2- 172.21.15.7:0/3374366012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7af81089a0 0x7f7af810ad90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80098a0 tx=0x7f7ae8009bb0 comp rx=0 tx=0).stop 2024-06-10T00:54:24.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.006+0000 7f7afe8f7700 1 -- 172.21.15.7:0/3374366012 shutdown_connections 2024-06-10T00:54:24.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.006+0000 7f7afe8f7700 1 --2- 172.21.15.7:0/3374366012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7af81089a0 0x7f7af810ad90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:24.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.006+0000 7f7afe8f7700 1 -- 172.21.15.7:0/3374366012 >> 172.21.15.7:0/3374366012 conn(0x7f7af8102370 msgr2=0x7f7af81047b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:24.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.006+0000 7f7afe8f7700 1 -- 172.21.15.7:0/3374366012 shutdown_connections 2024-06-10T00:54:24.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.006+0000 7f7afe8f7700 1 -- 172.21.15.7:0/3374366012 wait complete. 2024-06-10T00:54:24.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.007+0000 7f7afe8f7700 1 Processor -- start 2024-06-10T00:54:24.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.007+0000 7f7afe8f7700 1 -- start start 2024-06-10T00:54:24.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.007+0000 7f7afe8f7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7af81089a0 0x7f7af819bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:24.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.007+0000 7f7afe8f7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7af810c2d0 con 0x7f7af81089a0 2024-06-10T00:54:24.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.007+0000 7f7af7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7af81089a0 0x7f7af819bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:24.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.007+0000 7f7af7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7af81089a0 0x7f7af819bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32958/0 (socket says 172.21.15.7:32958) 2024-06-10T00:54:24.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.007+0000 7f7af7fff700 1 -- 172.21.15.7:0/836596964 learned_addr learned my addr 172.21.15.7:0/836596964 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:24.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.008+0000 7f7af7fff700 1 -- 172.21.15.7:0/836596964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af819c250 con 0x7f7af81089a0 2024-06-10T00:54:24.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.008+0000 7f7af7fff700 1 --2- 172.21.15.7:0/836596964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7af81089a0 0x7f7af819bd10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80117e0 tx=0x7f7ae8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:24.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.008+0000 7f7af57fa700 1 -- 172.21.15.7:0/836596964 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ae8003bf0 con 0x7f7af81089a0 2024-06-10T00:54:24.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.008+0000 7f7af57fa700 1 -- 172.21.15.7:0/836596964 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ae801a910 con 0x7f7af81089a0 2024-06-10T00:54:24.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.008+0000 7f7afe8f7700 1 -- 172.21.15.7:0/836596964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7af819c4e0 con 0x7f7af81089a0 2024-06-10T00:54:24.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.008+0000 7f7af57fa700 1 -- 172.21.15.7:0/836596964 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ae8024970 con 0x7f7af81089a0 2024-06-10T00:54:24.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.008+0000 7f7afe8f7700 1 -- 172.21.15.7:0/836596964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7af819c940 con 0x7f7af81089a0 2024-06-10T00:54:24.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.009+0000 7f7af57fa700 1 -- 172.21.15.7:0/836596964 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f7ae8022020 con 0x7f7af81089a0 2024-06-10T00:54:24.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.009+0000 7f7af57fa700 1 --2- 172.21.15.7:0/836596964 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f7ae4039050 0x7f7ae403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:24.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.009+0000 7f7af57fa700 1 -- 172.21.15.7:0/836596964 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7ae804d390 con 0x7f7af81089a0 2024-06-10T00:54:24.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.009+0000 7f7af77fe700 1 --2- 172.21.15.7:0/836596964 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f7ae4039050 0x7f7ae403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:24.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.009+0000 7f7afe8f7700 1 -- 172.21.15.7:0/836596964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7af8063af0 con 0x7f7af81089a0 2024-06-10T00:54:24.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.010+0000 7f7af77fe700 1 --2- 172.21.15.7:0/836596964 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f7ae4039050 0x7f7ae403b510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0006fd0 tx=0x7f7ae0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:24.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.015+0000 7f7af57fa700 1 -- 172.21.15.7:0/836596964 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7ae8011e20 con 0x7f7af81089a0 2024-06-10T00:54:24.778 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.251+0000 7f7afe8f7700 1 -- 172.21.15.7:0/836596964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f7af819f6b0 con 0x7f7af81089a0 2024-06-10T00:54:24.778 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.254+0000 7f7af57fa700 1 -- 172.21.15.7:0/836596964 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f7ae80172b0 con 0x7f7af81089a0 2024-06-10T00:54:24.778 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.257+0000 7f7afe8f7700 1 -- 172.21.15.7:0/836596964 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f7ae4039050 msgr2=0x7f7ae403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:24.778 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.257+0000 7f7afe8f7700 1 --2- 172.21.15.7:0/836596964 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f7ae4039050 0x7f7ae403b510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0006fd0 tx=0x7f7ae0006e40 comp rx=0 tx=0).stop 2024-06-10T00:54:24.778 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.257+0000 7f7afe8f7700 1 -- 172.21.15.7:0/836596964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7af81089a0 msgr2=0x7f7af819bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:24.778 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.257+0000 7f7afe8f7700 1 --2- 172.21.15.7:0/836596964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7af81089a0 0x7f7af819bd10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80117e0 tx=0x7f7ae8011c10 comp rx=0 tx=0).stop 2024-06-10T00:54:24.778 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.257+0000 7f7afe8f7700 1 -- 172.21.15.7:0/836596964 shutdown_connections 2024-06-10T00:54:24.778 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.257+0000 7f7afe8f7700 1 --2- 172.21.15.7:0/836596964 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f7ae4039050 0x7f7ae403b510 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:24.778 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.257+0000 7f7afe8f7700 1 --2- 172.21.15.7:0/836596964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7af81089a0 0x7f7af819bd10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:24.778 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.257+0000 7f7afe8f7700 1 -- 172.21.15.7:0/836596964 >> 172.21.15.7:0/836596964 conn(0x7f7af8102370 msgr2=0x7f7af8102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:24.778 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.257+0000 7f7afe8f7700 1 -- 172.21.15.7:0/836596964 shutdown_connections 2024-06-10T00:54:24.778 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:24.257+0000 7f7afe8f7700 1 -- 172.21.15.7:0/836596964 wait complete. 2024-06-10T00:54:24.892 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:24 smithi007 ceph-mon[32398]: from='mgr.14120 172.21.15.7:0/675400875' entity='mgr.x' 2024-06-10T00:54:24.892 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:24 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/836596964' entity='client.admin' 2024-06-10T00:54:26.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:26 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/320240612' entity='client.admin' 2024-06-10T00:54:26.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:26 smithi007 ceph-mon[32398]: from='mgr.14120 172.21.15.7:0/675400875' entity='mgr.x' 2024-06-10T00:54:26.541 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.200+0000 7fad1d6cd700 1 Processor -- start 2024-06-10T00:54:26.541 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.200+0000 7fad1d6cd700 1 -- start start 2024-06-10T00:54:26.541 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.200+0000 7fad1d6cd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad1807deb0 0x7fad1807c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:26.541 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.200+0000 7fad1d6cd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fad1807c8e0 con 0x7fad1807deb0 2024-06-10T00:54:26.541 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.200+0000 7fad16ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad1807deb0 0x7fad1807c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:26.542 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.200+0000 7fad16ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad1807deb0 0x7fad1807c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32968/0 (socket says 172.21.15.7:32968) 2024-06-10T00:54:26.542 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.200+0000 7fad16ffd700 1 -- 172.21.15.7:0/1915914305 learned_addr learned my addr 172.21.15.7:0/1915914305 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:26.542 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.200+0000 7fad16ffd700 1 -- 172.21.15.7:0/1915914305 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad1807ca60 con 0x7fad1807deb0 2024-06-10T00:54:26.542 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.201+0000 7fad16ffd700 1 --2- 172.21.15.7:0/1915914305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad1807deb0 0x7fad1807c310 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fad080098a0 tx=0x7fad08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a926da45c8191e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:26.542 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.201+0000 7fad15ffb700 1 -- 172.21.15.7:0/1915914305 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad08004160 con 0x7fad1807deb0 2024-06-10T00:54:26.542 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.201+0000 7fad15ffb700 1 -- 172.21.15.7:0/1915914305 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad08004300 con 0x7fad1807deb0 2024-06-10T00:54:26.542 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.201+0000 7fad15ffb700 1 -- 172.21.15.7:0/1915914305 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad08011b20 con 0x7fad1807deb0 2024-06-10T00:54:26.542 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.201+0000 7fad1d6cd700 1 -- 172.21.15.7:0/1915914305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad1807deb0 msgr2=0x7fad1807c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:26.542 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.201+0000 7fad1d6cd700 1 --2- 172.21.15.7:0/1915914305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad1807deb0 0x7fad1807c310 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fad080098a0 tx=0x7fad08009bb0 comp rx=0 tx=0).stop 2024-06-10T00:54:26.542 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.201+0000 7fad1d6cd700 1 -- 172.21.15.7:0/1915914305 shutdown_connections 2024-06-10T00:54:26.542 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.201+0000 7fad1d6cd700 1 --2- 172.21.15.7:0/1915914305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad1807deb0 0x7fad1807c310 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:26.542 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.201+0000 7fad1d6cd700 1 -- 172.21.15.7:0/1915914305 >> 172.21.15.7:0/1915914305 conn(0x7fad18103420 msgr2=0x7fad18105880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:26.543 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.201+0000 7fad1d6cd700 1 -- 172.21.15.7:0/1915914305 shutdown_connections 2024-06-10T00:54:26.543 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.201+0000 7fad1d6cd700 1 -- 172.21.15.7:0/1915914305 wait complete. 2024-06-10T00:54:26.543 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.202+0000 7fad1d6cd700 1 Processor -- start 2024-06-10T00:54:26.543 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.202+0000 7fad1d6cd700 1 -- start start 2024-06-10T00:54:26.543 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.202+0000 7fad1d6cd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad1807deb0 0x7fad1819df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:26.543 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.202+0000 7fad1d6cd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fad1810e500 con 0x7fad1807deb0 2024-06-10T00:54:26.543 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.203+0000 7fad16ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad1807deb0 0x7fad1819df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:26.543 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.203+0000 7fad16ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad1807deb0 0x7fad1819df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32976/0 (socket says 172.21.15.7:32976) 2024-06-10T00:54:26.543 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.203+0000 7fad16ffd700 1 -- 172.21.15.7:0/320240612 learned_addr learned my addr 172.21.15.7:0/320240612 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:26.543 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.203+0000 7fad16ffd700 1 -- 172.21.15.7:0/320240612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad1819e4a0 con 0x7fad1807deb0 2024-06-10T00:54:26.543 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.203+0000 7fad16ffd700 1 --2- 172.21.15.7:0/320240612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad1807deb0 0x7fad1819df60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fad080117e0 tx=0x7fad08011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:26.543 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.203+0000 7facfffff700 1 -- 172.21.15.7:0/320240612 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad08003bf0 con 0x7fad1807deb0 2024-06-10T00:54:26.543 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.203+0000 7fad1d6cd700 1 -- 172.21.15.7:0/320240612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad1819e730 con 0x7fad1807deb0 2024-06-10T00:54:26.544 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.203+0000 7facfffff700 1 -- 172.21.15.7:0/320240612 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad0801a910 con 0x7fad1807deb0 2024-06-10T00:54:26.544 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.203+0000 7facfffff700 1 -- 172.21.15.7:0/320240612 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad08024970 con 0x7fad1807deb0 2024-06-10T00:54:26.544 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.203+0000 7fad1d6cd700 1 -- 172.21.15.7:0/320240612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad1819eb90 con 0x7fad1807deb0 2024-06-10T00:54:26.544 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.204+0000 7facfffff700 1 -- 172.21.15.7:0/320240612 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fad08022020 con 0x7fad1807deb0 2024-06-10T00:54:26.544 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.204+0000 7fad1d6cd700 1 -- 172.21.15.7:0/320240612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad18063af0 con 0x7fad1807deb0 2024-06-10T00:54:26.544 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.204+0000 7facfffff700 1 --2- 172.21.15.7:0/320240612 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fad00039050 0x7fad0003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:26.544 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.204+0000 7facfffff700 1 -- 172.21.15.7:0/320240612 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fad0804d820 con 0x7fad1807deb0 2024-06-10T00:54:26.544 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.204+0000 7fad167fc700 1 --2- 172.21.15.7:0/320240612 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fad00039050 0x7fad0003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:26.544 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.205+0000 7fad167fc700 1 --2- 172.21.15.7:0/320240612 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fad00039050 0x7fad0003b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fad0c006fd0 tx=0x7fad0c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:26.544 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.209+0000 7facfffff700 1 -- 172.21.15.7:0/320240612 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fad0802cdf0 con 0x7fad1807deb0 2024-06-10T00:54:26.544 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.421+0000 7fad1d6cd700 1 -- 172.21.15.7:0/320240612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fad181a1560 con 0x7fad1807deb0 2024-06-10T00:54:26.544 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.423+0000 7facfffff700 1 -- 172.21.15.7:0/320240612 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fad08020c10 con 0x7fad1807deb0 2024-06-10T00:54:26.545 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.426+0000 7fad1d6cd700 1 -- 172.21.15.7:0/320240612 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fad00039050 msgr2=0x7fad0003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:26.545 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.426+0000 7fad1d6cd700 1 --2- 172.21.15.7:0/320240612 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fad00039050 0x7fad0003b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fad0c006fd0 tx=0x7fad0c006e40 comp rx=0 tx=0).stop 2024-06-10T00:54:26.545 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.426+0000 7fad1d6cd700 1 -- 172.21.15.7:0/320240612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad1807deb0 msgr2=0x7fad1819df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:26.545 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.426+0000 7fad1d6cd700 1 --2- 172.21.15.7:0/320240612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad1807deb0 0x7fad1819df60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fad080117e0 tx=0x7fad08011ba0 comp rx=0 tx=0).stop 2024-06-10T00:54:26.545 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.426+0000 7fad1d6cd700 1 -- 172.21.15.7:0/320240612 shutdown_connections 2024-06-10T00:54:26.545 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.426+0000 7fad1d6cd700 1 --2- 172.21.15.7:0/320240612 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fad00039050 0x7fad0003b510 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:26.545 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.426+0000 7fad1d6cd700 1 --2- 172.21.15.7:0/320240612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad1807deb0 0x7fad1819df60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:26.545 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.426+0000 7fad1d6cd700 1 -- 172.21.15.7:0/320240612 >> 172.21.15.7:0/320240612 conn(0x7fad18103420 msgr2=0x7fad18103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:26.545 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.426+0000 7fad1d6cd700 1 -- 172.21.15.7:0/320240612 shutdown_connections 2024-06-10T00:54:26.545 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:25.426+0000 7fad1d6cd700 1 -- 172.21.15.7:0/320240612 wait complete. 2024-06-10T00:54:26.545 INFO:teuthology.orchestra.run.smithi007.stdout:Enabling the dashboard module... 2024-06-10T00:54:28.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:28 smithi007 ceph-mon[32398]: from='mgr.14120 172.21.15.7:0/675400875' entity='mgr.x' 2024-06-10T00:54:28.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:28 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1684900090' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-06-10T00:54:28.519 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:28 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: ignoring --setuser ceph since I am not root 2024-06-10T00:54:28.519 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:28 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: ignoring --setgroup ceph since I am not root 2024-06-10T00:54:28.519 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:28 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:28.434+0000 7f3ab3702200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-10T00:54:29.002 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:28 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:28.640+0000 7f3ab3702200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:54:29.003 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:28 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:28.748+0000 7f3ab3702200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-10T00:54:29.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.992+0000 7f18be0df700 1 Processor -- start 2024-06-10T00:54:29.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.992+0000 7f18be0df700 1 -- start start 2024-06-10T00:54:29.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.992+0000 7f18be0df700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18b8106dc0 0x7f18b81071e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:29.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.992+0000 7f18be0df700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f18b81077b0 con 0x7f18b8106dc0 2024-06-10T00:54:29.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.993+0000 7f18b77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18b8106dc0 0x7f18b81071e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:29.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.993+0000 7f18b77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18b8106dc0 0x7f18b81071e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32978/0 (socket says 172.21.15.7:32978) 2024-06-10T00:54:29.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.993+0000 7f18b77fe700 1 -- 172.21.15.7:0/3755979432 learned_addr learned my addr 172.21.15.7:0/3755979432 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:29.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.993+0000 7f18b77fe700 1 -- 172.21.15.7:0/3755979432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b8107fc0 con 0x7f18b8106dc0 2024-06-10T00:54:29.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.993+0000 7f18b77fe700 1 --2- 172.21.15.7:0/3755979432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18b8106dc0 0x7f18b81071e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f18a80098a0 tx=0x7f18a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9283061665ffa7d5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:29.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.994+0000 7f18b67fc700 1 -- 172.21.15.7:0/3755979432 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18a8004160 con 0x7f18b8106dc0 2024-06-10T00:54:29.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.994+0000 7f18b67fc700 1 -- 172.21.15.7:0/3755979432 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18a8004300 con 0x7f18b8106dc0 2024-06-10T00:54:29.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.994+0000 7f18b67fc700 1 -- 172.21.15.7:0/3755979432 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18a8011b20 con 0x7f18b8106dc0 2024-06-10T00:54:29.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.994+0000 7f18be0df700 1 -- 172.21.15.7:0/3755979432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18b8106dc0 msgr2=0x7f18b81071e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:29.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.994+0000 7f18be0df700 1 --2- 172.21.15.7:0/3755979432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18b8106dc0 0x7f18b81071e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f18a80098a0 tx=0x7f18a8009bb0 comp rx=0 tx=0).stop 2024-06-10T00:54:29.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.994+0000 7f18be0df700 1 -- 172.21.15.7:0/3755979432 shutdown_connections 2024-06-10T00:54:29.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.994+0000 7f18be0df700 1 --2- 172.21.15.7:0/3755979432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18b8106dc0 0x7f18b81071e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:29.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.994+0000 7f18be0df700 1 -- 172.21.15.7:0/3755979432 >> 172.21.15.7:0/3755979432 conn(0x7f18b8102340 msgr2=0x7f18b81047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:29.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.994+0000 7f18be0df700 1 -- 172.21.15.7:0/3755979432 shutdown_connections 2024-06-10T00:54:29.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.994+0000 7f18be0df700 1 -- 172.21.15.7:0/3755979432 wait complete. 2024-06-10T00:54:29.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.995+0000 7f18be0df700 1 Processor -- start 2024-06-10T00:54:29.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.995+0000 7f18be0df700 1 -- start start 2024-06-10T00:54:29.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.996+0000 7f18be0df700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18b8106dc0 0x7f18b819acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:29.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.996+0000 7f18be0df700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f18b810b370 con 0x7f18b8106dc0 2024-06-10T00:54:29.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.996+0000 7f18b77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18b8106dc0 0x7f18b819acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:29.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.996+0000 7f18b77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18b8106dc0 0x7f18b819acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32986/0 (socket says 172.21.15.7:32986) 2024-06-10T00:54:29.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.996+0000 7f18b77fe700 1 -- 172.21.15.7:0/1684900090 learned_addr learned my addr 172.21.15.7:0/1684900090 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:29.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.996+0000 7f18b77fe700 1 -- 172.21.15.7:0/1684900090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b819b1f0 con 0x7f18b8106dc0 2024-06-10T00:54:29.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.996+0000 7f18b77fe700 1 --2- 172.21.15.7:0/1684900090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18b8106dc0 0x7f18b819acb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f18a80117e0 tx=0x7f18a8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:29.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.996+0000 7f18b4ff9700 1 -- 172.21.15.7:0/1684900090 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18a8003bf0 con 0x7f18b8106dc0 2024-06-10T00:54:29.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.996+0000 7f18b4ff9700 1 -- 172.21.15.7:0/1684900090 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18a801a910 con 0x7f18b8106dc0 2024-06-10T00:54:29.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.996+0000 7f18be0df700 1 -- 172.21.15.7:0/1684900090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b819b480 con 0x7f18b8106dc0 2024-06-10T00:54:29.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.996+0000 7f18b4ff9700 1 -- 172.21.15.7:0/1684900090 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18a80249a0 con 0x7f18b8106dc0 2024-06-10T00:54:29.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.997+0000 7f18be0df700 1 -- 172.21.15.7:0/1684900090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b819b8e0 con 0x7f18b8106dc0 2024-06-10T00:54:29.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.997+0000 7f18b4ff9700 1 -- 172.21.15.7:0/1684900090 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f18a8022020 con 0x7f18b8106dc0 2024-06-10T00:54:29.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.998+0000 7f18b4ff9700 1 --2- 172.21.15.7:0/1684900090 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f18a0039000 0x7f18a003b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:29.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.998+0000 7f18b4ff9700 1 -- 172.21.15.7:0/1684900090 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f18a801aac0 con 0x7f18b8106dc0 2024-06-10T00:54:29.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.998+0000 7f18be0df700 1 -- 172.21.15.7:0/1684900090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18b8063af0 con 0x7f18b8106dc0 2024-06-10T00:54:29.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.998+0000 7f18b6ffd700 1 --2- 172.21.15.7:0/1684900090 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f18a0039000 0x7f18a003b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:29.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:26.998+0000 7f18b6ffd700 1 --2- 172.21.15.7:0/1684900090 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f18a0039000 0x7f18a003b4c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f18ac006fd0 tx=0x7f18ac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:29.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:27.003+0000 7f18b4ff9700 1 -- 172.21.15.7:0/1684900090 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f18a8011e20 con 0x7f18b8106dc0 2024-06-10T00:54:29.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:27.274+0000 7f18be0df700 1 -- 172.21.15.7:0/1684900090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f18b819e660 con 0x7f18b8106dc0 2024-06-10T00:54:29.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:28.245+0000 7f18b4ff9700 1 -- 172.21.15.7:0/1684900090 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f18a802c350 con 0x7f18b8106dc0 2024-06-10T00:54:29.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:28.247+0000 7f18b4ff9700 1 -- 172.21.15.7:0/1684900090 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f18a802da00 con 0x7f18b8106dc0 2024-06-10T00:54:29.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:28.248+0000 7f18be0df700 1 -- 172.21.15.7:0/1684900090 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f18a0039000 msgr2=0x7f18a003b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:29.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:28.248+0000 7f18be0df700 1 --2- 172.21.15.7:0/1684900090 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f18a0039000 0x7f18a003b4c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f18ac006fd0 tx=0x7f18ac006e40 comp rx=0 tx=0).stop 2024-06-10T00:54:29.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:28.248+0000 7f18be0df700 1 -- 172.21.15.7:0/1684900090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18b8106dc0 msgr2=0x7f18b819acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:29.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:28.248+0000 7f18be0df700 1 --2- 172.21.15.7:0/1684900090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18b8106dc0 0x7f18b819acb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f18a80117e0 tx=0x7f18a8011c10 comp rx=0 tx=0).stop 2024-06-10T00:54:29.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:28.248+0000 7f18be0df700 1 -- 172.21.15.7:0/1684900090 shutdown_connections 2024-06-10T00:54:29.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:28.248+0000 7f18be0df700 1 --2- 172.21.15.7:0/1684900090 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f18a0039000 0x7f18a003b4c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:29.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:28.248+0000 7f18be0df700 1 --2- 172.21.15.7:0/1684900090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18b8106dc0 0x7f18b819acb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:29.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:28.248+0000 7f18be0df700 1 -- 172.21.15.7:0/1684900090 >> 172.21.15.7:0/1684900090 conn(0x7f18b8102340 msgr2=0x7f18b8102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:29.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:28.249+0000 7f18be0df700 1 -- 172.21.15.7:0/1684900090 shutdown_connections 2024-06-10T00:54:29.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:28.249+0000 7f18be0df700 1 -- 172.21.15.7:0/1684900090 wait complete. 2024-06-10T00:54:29.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:29 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1684900090' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-06-10T00:54:29.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:29 smithi007 ceph-mon[32398]: mgrmap e9: x(active, since 20s) 2024-06-10T00:54:29.269 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:29 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:29.082+0000 7f3ab3702200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-10T00:54:29.269 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:29 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:29.173+0000 7f3ab3702200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-10T00:54:30.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-06-10T00:54:30.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-06-10T00:54:30.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-10T00:54:30.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-10T00:54:30.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-10T00:54:30.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-06-10T00:54:30.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.459+0000 7fbe5dcc1700 1 Processor -- start 2024-06-10T00:54:30.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.459+0000 7fbe5dcc1700 1 -- start start 2024-06-10T00:54:30.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.460+0000 7fbe5dcc1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe581061b0 0x7fbe581065d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:30.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.460+0000 7fbe5dcc1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbe58106ba0 con 0x7fbe581061b0 2024-06-10T00:54:30.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.460+0000 7fbe577fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe581061b0 0x7fbe581065d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:30.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.460+0000 7fbe577fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe581061b0 0x7fbe581065d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33010/0 (socket says 172.21.15.7:33010) 2024-06-10T00:54:30.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.460+0000 7fbe577fe700 1 -- 172.21.15.7:0/2702631289 learned_addr learned my addr 172.21.15.7:0/2702631289 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:30.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.461+0000 7fbe577fe700 1 -- 172.21.15.7:0/2702631289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe58106ed0 con 0x7fbe581061b0 2024-06-10T00:54:30.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.461+0000 7fbe577fe700 1 --2- 172.21.15.7:0/2702631289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe581061b0 0x7fbe581065d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbe480098a0 tx=0x7fbe48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6704ea0b4b6ec76 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:30.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.461+0000 7fbe567fc700 1 -- 172.21.15.7:0/2702631289 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe48004160 con 0x7fbe581061b0 2024-06-10T00:54:30.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.461+0000 7fbe567fc700 1 -- 172.21.15.7:0/2702631289 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe48004300 con 0x7fbe581061b0 2024-06-10T00:54:30.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.461+0000 7fbe567fc700 1 -- 172.21.15.7:0/2702631289 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe48004160 con 0x7fbe581061b0 2024-06-10T00:54:30.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.462+0000 7fbe5dcc1700 1 -- 172.21.15.7:0/2702631289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe581061b0 msgr2=0x7fbe581065d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:30.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.462+0000 7fbe5dcc1700 1 --2- 172.21.15.7:0/2702631289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe581061b0 0x7fbe581065d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbe480098a0 tx=0x7fbe48009bb0 comp rx=0 tx=0).stop 2024-06-10T00:54:30.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.462+0000 7fbe5dcc1700 1 -- 172.21.15.7:0/2702631289 shutdown_connections 2024-06-10T00:54:30.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.462+0000 7fbe5dcc1700 1 --2- 172.21.15.7:0/2702631289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe581061b0 0x7fbe581065d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:30.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.462+0000 7fbe5dcc1700 1 -- 172.21.15.7:0/2702631289 >> 172.21.15.7:0/2702631289 conn(0x7fbe58101ae0 msgr2=0x7fbe58103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:30.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.462+0000 7fbe5dcc1700 1 -- 172.21.15.7:0/2702631289 shutdown_connections 2024-06-10T00:54:30.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.463+0000 7fbe5dcc1700 1 -- 172.21.15.7:0/2702631289 wait complete. 2024-06-10T00:54:30.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.463+0000 7fbe5dcc1700 1 Processor -- start 2024-06-10T00:54:30.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.463+0000 7fbe5dcc1700 1 -- start start 2024-06-10T00:54:30.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.464+0000 7fbe5dcc1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe581061b0 0x7fbe581a2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:30.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.464+0000 7fbe5dcc1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5810a2a0 con 0x7fbe581061b0 2024-06-10T00:54:30.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.464+0000 7fbe577fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe581061b0 0x7fbe581a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:30.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.464+0000 7fbe577fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe581061b0 0x7fbe581a2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33022/0 (socket says 172.21.15.7:33022) 2024-06-10T00:54:30.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.464+0000 7fbe577fe700 1 -- 172.21.15.7:0/3505567330 learned_addr learned my addr 172.21.15.7:0/3505567330 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:30.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.464+0000 7fbe577fe700 1 -- 172.21.15.7:0/3505567330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe581a28d0 con 0x7fbe581061b0 2024-06-10T00:54:30.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.464+0000 7fbe577fe700 1 --2- 172.21.15.7:0/3505567330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe581061b0 0x7fbe581a2390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbe48006ba0 tx=0x7fbe48011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:30.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.464+0000 7fbe54ff9700 1 -- 172.21.15.7:0/3505567330 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe480039e0 con 0x7fbe581061b0 2024-06-10T00:54:30.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.465+0000 7fbe54ff9700 1 -- 172.21.15.7:0/3505567330 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe48003b80 con 0x7fbe581061b0 2024-06-10T00:54:30.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.465+0000 7fbe5dcc1700 1 -- 172.21.15.7:0/3505567330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe581a2b60 con 0x7fbe581061b0 2024-06-10T00:54:30.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.465+0000 7fbe54ff9700 1 -- 172.21.15.7:0/3505567330 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe480246c0 con 0x7fbe581061b0 2024-06-10T00:54:30.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.465+0000 7fbe5dcc1700 1 -- 172.21.15.7:0/3505567330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe581a2fc0 con 0x7fbe581061b0 2024-06-10T00:54:30.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.465+0000 7fbe54ff9700 1 -- 172.21.15.7:0/3505567330 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fbe48022020 con 0x7fbe581061b0 2024-06-10T00:54:30.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.466+0000 7fbe54ff9700 1 --2- 172.21.15.7:0/3505567330 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fbe40039090 0x7fbe4003b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:30.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.466+0000 7fbe5dcc1700 1 -- 172.21.15.7:0/3505567330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe58063af0 con 0x7fbe581061b0 2024-06-10T00:54:30.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.466+0000 7fbe54ff9700 1 -- 172.21.15.7:0/3505567330 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbe4804e120 con 0x7fbe581061b0 2024-06-10T00:54:30.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.466+0000 7fbe56ffd700 1 -- 172.21.15.7:0/3505567330 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fbe40039090 msgr2=0x7fbe4003b550 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3298829389 2024-06-10T00:54:30.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.466+0000 7fbe56ffd700 1 --2- 172.21.15.7:0/3505567330 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fbe40039090 0x7fbe4003b550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T00:54:30.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.471+0000 7fbe54ff9700 1 -- 172.21.15.7:0/3505567330 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbe4802a2a0 con 0x7fbe581061b0 2024-06-10T00:54:30.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.667+0000 7fbe56ffd700 1 -- 172.21.15.7:0/3505567330 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fbe40039090 msgr2=0x7fbe4003b550 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3298829389 2024-06-10T00:54:30.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.667+0000 7fbe56ffd700 1 --2- 172.21.15.7:0/3505567330 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fbe40039090 0x7fbe4003b550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-10T00:54:30.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.782+0000 7fbe5dcc1700 1 -- 172.21.15.7:0/3505567330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fbe580630a0 con 0x7fbe581061b0 2024-06-10T00:54:30.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.782+0000 7fbe54ff9700 1 -- 172.21.15.7:0/3505567330 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fbe4802d3f0 con 0x7fbe581061b0 2024-06-10T00:54:30.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.785+0000 7fbe5dcc1700 1 -- 172.21.15.7:0/3505567330 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fbe40039090 msgr2=0x7fbe4003b550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T00:54:30.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.785+0000 7fbe5dcc1700 1 --2- 172.21.15.7:0/3505567330 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fbe40039090 0x7fbe4003b550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:30.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.785+0000 7fbe5dcc1700 1 -- 172.21.15.7:0/3505567330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe581061b0 msgr2=0x7fbe581a2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:30.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.785+0000 7fbe5dcc1700 1 --2- 172.21.15.7:0/3505567330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe581061b0 0x7fbe581a2390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbe48006ba0 tx=0x7fbe48011df0 comp rx=0 tx=0).stop 2024-06-10T00:54:30.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.785+0000 7fbe5dcc1700 1 -- 172.21.15.7:0/3505567330 shutdown_connections 2024-06-10T00:54:30.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.785+0000 7fbe5dcc1700 1 --2- 172.21.15.7:0/3505567330 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7fbe40039090 0x7fbe4003b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:30.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.785+0000 7fbe5dcc1700 1 --2- 172.21.15.7:0/3505567330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe581061b0 0x7fbe581a2390 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:30.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.785+0000 7fbe5dcc1700 1 -- 172.21.15.7:0/3505567330 >> 172.21.15.7:0/3505567330 conn(0x7fbe58101ae0 msgr2=0x7fbe5810c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:30.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.785+0000 7fbe5dcc1700 1 -- 172.21.15.7:0/3505567330 shutdown_connections 2024-06-10T00:54:30.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:29.785+0000 7fbe5dcc1700 1 -- 172.21.15.7:0/3505567330 wait complete. 2024-06-10T00:54:30.301 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for the mgr to restart... 2024-06-10T00:54:30.302 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr epoch 9... 2024-06-10T00:54:30.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:30 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3505567330' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-10T00:54:30.519 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:30 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:30.450+0000 7f3ab3702200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-10T00:54:30.816 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:30 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:30.633+0000 7f3ab3702200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-10T00:54:30.816 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:30 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:30.815+0000 7f3ab3702200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-10T00:54:31.426 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:31 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:31.117+0000 7f3ab3702200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-10T00:54:31.426 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:31 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:31.214+0000 7f3ab3702200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-10T00:54:31.426 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:31 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:31.425+0000 7f3ab3702200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-10T00:54:31.768 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:31 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:31.627+0000 7f3ab3702200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:54:32.518 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:32 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:32.256+0000 7f3ab3702200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-10T00:54:33.012 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:32 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:32.699+0000 7f3ab3702200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-10T00:54:33.012 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:32 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:32.801+0000 7f3ab3702200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-10T00:54:33.012 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:33 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:33.011+0000 7f3ab3702200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-10T00:54:33.910 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:33 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:33.558+0000 7f3ab3702200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-10T00:54:33.911 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:33 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:33.654+0000 7f3ab3702200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-10T00:54:34.268 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:33 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:33.910+0000 7f3ab3702200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-10T00:54:34.269 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:34 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:34.050+0000 7f3ab3702200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-10T00:54:34.269 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:34 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:34.150+0000 7f3ab3702200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-10T00:54:34.582 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:34 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:34.271+0000 7f3ab3702200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-10T00:54:35.018 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:34 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:34.581+0000 7f3ab3702200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-10T00:54:35.492 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:35 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:35.210+0000 7f3ab3702200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-10T00:54:35.492 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 00:54:35 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T00:54:35.438+0000 7f3ab3702200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-10T00:54:35.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:35 smithi007 ceph-mon[32398]: Active manager daemon x restarted 2024-06-10T00:54:35.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:35 smithi007 ceph-mon[32398]: Activating manager daemon x 2024-06-10T00:54:35.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:35 smithi007 ceph-mon[32398]: osdmap e3: 0 total, 0 up, 0 in 2024-06-10T00:54:35.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:35 smithi007 ceph-mon[32398]: mgrmap e10: x(active, starting, since 0.00331648s) 2024-06-10T00:54:35.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:35 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:54:35.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:35 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-10T00:54:35.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:35 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-10T00:54:35.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:35 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-10T00:54:35.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:35 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-10T00:54:35.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:35 smithi007 ceph-mon[32398]: Manager daemon x is now available 2024-06-10T00:54:35.770 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:35 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:54:35.770 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:35 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-10T00:54:36.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-10T00:54:36.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:36 smithi007 ceph-mon[32398]: mgrmap e11: x(active, since 1.00596s) 2024-06-10T00:54:36.969 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-06-10T00:54:36.969 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-06-10T00:54:36.970 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-10T00:54:36.970 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-06-10T00:54:36.970 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.754+0000 7f09dac42700 1 Processor -- start 2024-06-10T00:54:36.970 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.754+0000 7f09dac42700 1 -- start start 2024-06-10T00:54:36.970 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.754+0000 7f09dac42700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d4106770 0x7f09d4106b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:36.970 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.754+0000 7f09dac42700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f09d407be40 con 0x7f09d4106770 2024-06-10T00:54:36.970 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.754+0000 7f09d89de700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d4106770 0x7f09d4106b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:36.970 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.754+0000 7f09d89de700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d4106770 0x7f09d4106b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33024/0 (socket says 172.21.15.7:33024) 2024-06-10T00:54:36.970 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.754+0000 7f09d89de700 1 -- 172.21.15.7:0/1471488526 learned_addr learned my addr 172.21.15.7:0/1471488526 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:36.970 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.755+0000 7f09d89de700 1 -- 172.21.15.7:0/1471488526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d41070d0 con 0x7f09d4106770 2024-06-10T00:54:36.970 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.755+0000 7f09d89de700 1 --2- 172.21.15.7:0/1471488526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d4106770 0x7f09d4106b90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f09c80098a0 tx=0x7f09c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=379a2698744fedea server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:36.970 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.755+0000 7f09d37fe700 1 -- 172.21.15.7:0/1471488526 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09c8004160 con 0x7f09d4106770 2024-06-10T00:54:36.971 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.755+0000 7f09d37fe700 1 -- 172.21.15.7:0/1471488526 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09c8004300 con 0x7f09d4106770 2024-06-10T00:54:36.971 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.755+0000 7f09d37fe700 1 -- 172.21.15.7:0/1471488526 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09c8011b20 con 0x7f09d4106770 2024-06-10T00:54:36.971 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.756+0000 7f09dac42700 1 -- 172.21.15.7:0/1471488526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d4106770 msgr2=0x7f09d4106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:36.971 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.756+0000 7f09dac42700 1 --2- 172.21.15.7:0/1471488526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d4106770 0x7f09d4106b90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f09c80098a0 tx=0x7f09c8009bb0 comp rx=0 tx=0).stop 2024-06-10T00:54:36.971 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.756+0000 7f09dac42700 1 -- 172.21.15.7:0/1471488526 shutdown_connections 2024-06-10T00:54:36.971 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.756+0000 7f09dac42700 1 --2- 172.21.15.7:0/1471488526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d4106770 0x7f09d4106b90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:36.971 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.756+0000 7f09dac42700 1 -- 172.21.15.7:0/1471488526 >> 172.21.15.7:0/1471488526 conn(0x7f09d41022f0 msgr2=0x7f09d4104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:36.971 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.756+0000 7f09dac42700 1 -- 172.21.15.7:0/1471488526 shutdown_connections 2024-06-10T00:54:36.971 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.756+0000 7f09dac42700 1 -- 172.21.15.7:0/1471488526 wait complete. 2024-06-10T00:54:36.971 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.757+0000 7f09dac42700 1 Processor -- start 2024-06-10T00:54:36.971 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.757+0000 7f09dac42700 1 -- start start 2024-06-10T00:54:36.971 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.757+0000 7f09dac42700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d4106770 0x7f09d419df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:36.971 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.757+0000 7f09dac42700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f09d41081a0 con 0x7f09d4106770 2024-06-10T00:54:36.972 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.757+0000 7f09d89de700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d4106770 0x7f09d419df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:36.972 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.757+0000 7f09d89de700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d4106770 0x7f09d419df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33030/0 (socket says 172.21.15.7:33030) 2024-06-10T00:54:36.972 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.757+0000 7f09d89de700 1 -- 172.21.15.7:0/2755895585 learned_addr learned my addr 172.21.15.7:0/2755895585 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:36.972 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.757+0000 7f09d89de700 1 -- 172.21.15.7:0/2755895585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d419e460 con 0x7f09d4106770 2024-06-10T00:54:36.972 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.757+0000 7f09d89de700 1 --2- 172.21.15.7:0/2755895585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d4106770 0x7f09d419df20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f09c80117e0 tx=0x7f09c8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:36.972 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.758+0000 7f09d1ffb700 1 -- 172.21.15.7:0/2755895585 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09c8003bf0 con 0x7f09d4106770 2024-06-10T00:54:36.972 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.758+0000 7f09dac42700 1 -- 172.21.15.7:0/2755895585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09d419e6f0 con 0x7f09d4106770 2024-06-10T00:54:36.972 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.758+0000 7f09d1ffb700 1 -- 172.21.15.7:0/2755895585 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09c801a910 con 0x7f09d4106770 2024-06-10T00:54:36.972 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.758+0000 7f09d1ffb700 1 -- 172.21.15.7:0/2755895585 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09c8024970 con 0x7f09d4106770 2024-06-10T00:54:36.972 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.758+0000 7f09dac42700 1 -- 172.21.15.7:0/2755895585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09d419eb50 con 0x7f09d4106770 2024-06-10T00:54:36.972 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.759+0000 7f09d1ffb700 1 -- 172.21.15.7:0/2755895585 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f09c8022020 con 0x7f09d4106770 2024-06-10T00:54:36.972 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.759+0000 7f09d1ffb700 1 --2- 172.21.15.7:0/2755895585 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f09c4039050 0x7f09c403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:36.972 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.759+0000 7f09d1ffb700 1 -- 172.21.15.7:0/2755895585 --> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f09c403bc20 con 0x7f09c4039050 2024-06-10T00:54:36.973 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.759+0000 7f09d1ffb700 1 -- 172.21.15.7:0/2755895585 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f09c804d510 con 0x7f09d4106770 2024-06-10T00:54:36.973 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.759+0000 7f09d3fff700 1 -- 172.21.15.7:0/2755895585 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f09c4039050 msgr2=0x7f09c403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3298829389 2024-06-10T00:54:36.973 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.759+0000 7f09d3fff700 1 --2- 172.21.15.7:0/2755895585 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f09c4039050 0x7f09c403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T00:54:36.973 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.960+0000 7f09d3fff700 1 -- 172.21.15.7:0/2755895585 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f09c4039050 msgr2=0x7f09c403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3298829389 2024-06-10T00:54:36.973 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:30.960+0000 7f09d3fff700 1 --2- 172.21.15.7:0/2755895585 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f09c4039050 0x7f09c403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-10T00:54:36.973 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:31.361+0000 7f09d3fff700 1 -- 172.21.15.7:0/2755895585 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f09c4039050 msgr2=0x7f09c403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3298829389 2024-06-10T00:54:36.973 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:31.361+0000 7f09d3fff700 1 --2- 172.21.15.7:0/2755895585 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f09c4039050 0x7f09c403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-10T00:54:36.973 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:32.162+0000 7f09d3fff700 1 -- 172.21.15.7:0/2755895585 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f09c4039050 msgr2=0x7f09c403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3298829389 2024-06-10T00:54:36.973 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:32.162+0000 7f09d3fff700 1 --2- 172.21.15.7:0/2755895585 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f09c4039050 0x7f09c403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-10T00:54:36.973 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:33.763+0000 7f09d3fff700 1 -- 172.21.15.7:0/2755895585 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f09c4039050 msgr2=0x7f09c403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3298829389 2024-06-10T00:54:36.973 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:33.763+0000 7f09d3fff700 1 --2- 172.21.15.7:0/2755895585 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f09c4039050 0x7f09c403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-10T00:54:36.974 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:35.441+0000 7f09d1ffb700 1 -- 172.21.15.7:0/2755895585 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f09c802d910 con 0x7f09d4106770 2024-06-10T00:54:36.974 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:35.441+0000 7f09d1ffb700 1 -- 172.21.15.7:0/2755895585 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f09c4039050 msgr2=0x7f09c403b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T00:54:36.974 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:35.441+0000 7f09d1ffb700 1 --2- 172.21.15.7:0/2755895585 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f09c4039050 0x7f09c403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:36.974 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:36.443+0000 7f09d1ffb700 1 -- 172.21.15.7:0/2755895585 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f09c804ec60 con 0x7f09d4106770 2024-06-10T00:54:36.974 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:36.443+0000 7f09d1ffb700 1 --2- 172.21.15.7:0/2755895585 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f09c403cb10 0x7f09c403ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:36.974 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:36.443+0000 7f09d1ffb700 1 -- 172.21.15.7:0/2755895585 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f09c8019e50 con 0x7f09c403cb10 2024-06-10T00:54:36.974 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:36.443+0000 7f09d3fff700 1 --2- 172.21.15.7:0/2755895585 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f09c403cb10 0x7f09c403ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:36.974 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:36.444+0000 7f09d3fff700 1 --2- 172.21.15.7:0/2755895585 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f09c403cb10 0x7f09c403ef00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f09c0003a10 tx=0x7f09c00092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:36.974 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:36.445+0000 7f09d1ffb700 1 -- 172.21.15.7:0/2755895585 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f09c8019e50 con 0x7f09c403cb10 2024-06-10T00:54:36.974 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:36.451+0000 7f09dac42700 1 -- 172.21.15.7:0/2755895585 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f09d4063af0 con 0x7f09c403cb10 2024-06-10T00:54:36.974 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:36.451+0000 7f09d1ffb700 1 -- 172.21.15.7:0/2755895585 <== mgr.14152 v2:172.21.15.7:6800/1804801756 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f09d4063af0 con 0x7f09c403cb10 2024-06-10T00:54:36.974 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:36.452+0000 7f09dac42700 1 -- 172.21.15.7:0/2755895585 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f09c403cb10 msgr2=0x7f09c403ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:36.974 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:36.452+0000 7f09dac42700 1 --2- 172.21.15.7:0/2755895585 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f09c403cb10 0x7f09c403ef00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f09c0003a10 tx=0x7f09c00092b0 comp rx=0 tx=0).stop 2024-06-10T00:54:36.975 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:36.452+0000 7f09dac42700 1 -- 172.21.15.7:0/2755895585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d4106770 msgr2=0x7f09d419df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:36.975 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:36.452+0000 7f09dac42700 1 --2- 172.21.15.7:0/2755895585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d4106770 0x7f09d419df20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f09c80117e0 tx=0x7f09c8011c10 comp rx=0 tx=0).stop 2024-06-10T00:54:36.975 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:36.452+0000 7f09dac42700 1 -- 172.21.15.7:0/2755895585 shutdown_connections 2024-06-10T00:54:36.975 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:36.452+0000 7f09dac42700 1 --2- 172.21.15.7:0/2755895585 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f09c403cb10 0x7f09c403ef00 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:36.975 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:36.452+0000 7f09dac42700 1 --2- 172.21.15.7:0/2755895585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d4106770 0x7f09d419df20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:36.975 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:36.452+0000 7f09dac42700 1 --2- 172.21.15.7:0/2755895585 >> [v2:172.21.15.7:6800/3298829389,v1:172.21.15.7:6801/3298829389] conn(0x7f09c4039050 0x7f09c403b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:36.975 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:36.452+0000 7f09dac42700 1 -- 172.21.15.7:0/2755895585 >> 172.21.15.7:0/2755895585 conn(0x7f09d41022f0 msgr2=0x7f09d4194b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:36.975 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:36.452+0000 7f09dac42700 1 -- 172.21.15.7:0/2755895585 shutdown_connections 2024-06-10T00:54:36.975 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:36.452+0000 7f09dac42700 1 -- 172.21.15.7:0/2755895585 wait complete. 2024-06-10T00:54:36.975 INFO:teuthology.orchestra.run.smithi007.stdout:mgr epoch 9 is available 2024-06-10T00:54:36.975 INFO:teuthology.orchestra.run.smithi007.stdout:Generating a dashboard self-signed certificate... 2024-06-10T00:54:37.650 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:37 smithi007 ceph-mon[32398]: [10/Jun/2024:00:54:36] ENGINE Bus STARTING 2024-06-10T00:54:37.650 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:37 smithi007 ceph-mon[32398]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-10T00:54:37.650 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:37 smithi007 ceph-mon[32398]: [10/Jun/2024:00:54:36] ENGINE Serving on https://172.21.15.7:7150 2024-06-10T00:54:37.650 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:37 smithi007 ceph-mon[32398]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-10T00:54:37.650 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:37 smithi007 ceph-mon[32398]: [10/Jun/2024:00:54:36] ENGINE Serving on http://172.21.15.7:8765 2024-06-10T00:54:37.650 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:37 smithi007 ceph-mon[32398]: [10/Jun/2024:00:54:36] ENGINE Bus STARTED 2024-06-10T00:54:38.317 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-06-10T00:54:38.317 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.422+0000 7f30f6da2700 1 Processor -- start 2024-06-10T00:54:38.317 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.422+0000 7f30f6da2700 1 -- start start 2024-06-10T00:54:38.317 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.422+0000 7f30f6da2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30f0107ed0 0x7f30f01082f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:38.317 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.423+0000 7f30f6da2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f30f0108830 con 0x7f30f0107ed0 2024-06-10T00:54:38.317 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.423+0000 7f30f4b3e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30f0107ed0 0x7f30f01082f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:38.317 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.423+0000 7f30f4b3e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30f0107ed0 0x7f30f01082f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43156/0 (socket says 172.21.15.7:43156) 2024-06-10T00:54:38.317 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.423+0000 7f30f4b3e700 1 -- 172.21.15.7:0/457208474 learned_addr learned my addr 172.21.15.7:0/457208474 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:38.317 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.423+0000 7f30f4b3e700 1 -- 172.21.15.7:0/457208474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f0108fe0 con 0x7f30f0107ed0 2024-06-10T00:54:38.318 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.423+0000 7f30f4b3e700 1 --2- 172.21.15.7:0/457208474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30f0107ed0 0x7f30f01082f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f30e0009e80 tx=0x7f30e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567ae649635f8d0d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:38.318 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.423+0000 7f30ef7fe700 1 -- 172.21.15.7:0/457208474 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30e0017070 con 0x7f30f0107ed0 2024-06-10T00:54:38.318 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.424+0000 7f30ef7fe700 1 -- 172.21.15.7:0/457208474 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30e000b8c0 con 0x7f30f0107ed0 2024-06-10T00:54:38.318 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.424+0000 7f30ef7fe700 1 -- 172.21.15.7:0/457208474 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30e00045c0 con 0x7f30f0107ed0 2024-06-10T00:54:38.318 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.424+0000 7f30f6da2700 1 -- 172.21.15.7:0/457208474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30f0107ed0 msgr2=0x7f30f01082f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:38.318 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.424+0000 7f30f6da2700 1 --2- 172.21.15.7:0/457208474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30f0107ed0 0x7f30f01082f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f30e0009e80 tx=0x7f30e000b330 comp rx=0 tx=0).stop 2024-06-10T00:54:38.318 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.424+0000 7f30f6da2700 1 -- 172.21.15.7:0/457208474 shutdown_connections 2024-06-10T00:54:38.318 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.424+0000 7f30f6da2700 1 --2- 172.21.15.7:0/457208474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30f0107ed0 0x7f30f01082f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:38.318 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.424+0000 7f30f6da2700 1 -- 172.21.15.7:0/457208474 >> 172.21.15.7:0/457208474 conn(0x7f30f0103470 msgr2=0x7f30f01058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:38.318 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.424+0000 7f30f6da2700 1 -- 172.21.15.7:0/457208474 shutdown_connections 2024-06-10T00:54:38.318 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.424+0000 7f30f6da2700 1 -- 172.21.15.7:0/457208474 wait complete. 2024-06-10T00:54:38.318 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.425+0000 7f30f6da2700 1 Processor -- start 2024-06-10T00:54:38.319 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.425+0000 7f30f6da2700 1 -- start start 2024-06-10T00:54:38.319 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.425+0000 7f30f6da2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30f0107ed0 0x7f30f007e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:38.319 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.425+0000 7f30f6da2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f30f010c420 con 0x7f30f0107ed0 2024-06-10T00:54:38.319 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.425+0000 7f30f4b3e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30f0107ed0 0x7f30f007e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:38.319 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.425+0000 7f30f4b3e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30f0107ed0 0x7f30f007e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43160/0 (socket says 172.21.15.7:43160) 2024-06-10T00:54:38.319 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.425+0000 7f30f4b3e700 1 -- 172.21.15.7:0/2041349460 learned_addr learned my addr 172.21.15.7:0/2041349460 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:38.319 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.426+0000 7f30f4b3e700 1 -- 172.21.15.7:0/2041349460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f007e900 con 0x7f30f0107ed0 2024-06-10T00:54:38.319 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.426+0000 7f30f4b3e700 1 --2- 172.21.15.7:0/2041349460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30f0107ed0 0x7f30f007e3c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f30e0009fd0 tx=0x7f30e0003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:38.319 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.426+0000 7f30edffb700 1 -- 172.21.15.7:0/2041349460 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30e0020020 con 0x7f30f0107ed0 2024-06-10T00:54:38.319 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.426+0000 7f30edffb700 1 -- 172.21.15.7:0/2041349460 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30e000bba0 con 0x7f30f0107ed0 2024-06-10T00:54:38.319 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.426+0000 7f30f6da2700 1 -- 172.21.15.7:0/2041349460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30f007cb40 con 0x7f30f0107ed0 2024-06-10T00:54:38.319 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.426+0000 7f30edffb700 1 -- 172.21.15.7:0/2041349460 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30e0017040 con 0x7f30f0107ed0 2024-06-10T00:54:38.319 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.426+0000 7f30f6da2700 1 -- 172.21.15.7:0/2041349460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30f007d080 con 0x7f30f0107ed0 2024-06-10T00:54:38.320 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.427+0000 7f30edffb700 1 -- 172.21.15.7:0/2041349460 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f30e0023020 con 0x7f30f0107ed0 2024-06-10T00:54:38.320 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.427+0000 7f30f6da2700 1 -- 172.21.15.7:0/2041349460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30f010c5a0 con 0x7f30f0107ed0 2024-06-10T00:54:38.320 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.427+0000 7f30edffb700 1 --2- 172.21.15.7:0/2041349460 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f30dc038ff0 0x7f30dc03b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:38.320 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.427+0000 7f30edffb700 1 -- 172.21.15.7:0/2041349460 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f30e000f050 con 0x7f30f0107ed0 2024-06-10T00:54:38.320 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.427+0000 7f30effff700 1 --2- 172.21.15.7:0/2041349460 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f30dc038ff0 0x7f30dc03b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:38.320 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.428+0000 7f30effff700 1 --2- 172.21.15.7:0/2041349460 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f30dc038ff0 0x7f30dc03b4b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f30d8006fd0 tx=0x7f30d8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:38.320 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.432+0000 7f30edffb700 1 -- 172.21.15.7:0/2041349460 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f30e002a020 con 0x7f30f0107ed0 2024-06-10T00:54:38.320 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.497+0000 7f30edffb700 1 -- 172.21.15.7:0/2041349460 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f30e0025400 con 0x7f30f0107ed0 2024-06-10T00:54:38.320 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.648+0000 7f30f6da2700 1 -- 172.21.15.7:0/2041349460 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f30f007d7f0 con 0x7f30dc038ff0 2024-06-10T00:54:38.320 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.769+0000 7f30edffb700 1 -- 172.21.15.7:0/2041349460 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f30f007d7f0 con 0x7f30dc038ff0 2024-06-10T00:54:38.320 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.771+0000 7f30f6da2700 1 -- 172.21.15.7:0/2041349460 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f30dc038ff0 msgr2=0x7f30dc03b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:38.320 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.771+0000 7f30f6da2700 1 --2- 172.21.15.7:0/2041349460 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f30dc038ff0 0x7f30dc03b4b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f30d8006fd0 tx=0x7f30d8006e40 comp rx=0 tx=0).stop 2024-06-10T00:54:38.320 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.771+0000 7f30f6da2700 1 -- 172.21.15.7:0/2041349460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30f0107ed0 msgr2=0x7f30f007e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:38.320 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.771+0000 7f30f6da2700 1 --2- 172.21.15.7:0/2041349460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30f0107ed0 0x7f30f007e3c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f30e0009fd0 tx=0x7f30e0003d00 comp rx=0 tx=0).stop 2024-06-10T00:54:38.321 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.771+0000 7f30f6da2700 1 -- 172.21.15.7:0/2041349460 shutdown_connections 2024-06-10T00:54:38.321 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.771+0000 7f30f6da2700 1 --2- 172.21.15.7:0/2041349460 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f30dc038ff0 0x7f30dc03b4b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:38.321 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.771+0000 7f30f6da2700 1 --2- 172.21.15.7:0/2041349460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30f0107ed0 0x7f30f007e3c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:38.321 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.771+0000 7f30f6da2700 1 -- 172.21.15.7:0/2041349460 >> 172.21.15.7:0/2041349460 conn(0x7f30f0103470 msgr2=0x7f30f0103ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:38.321 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.772+0000 7f30f6da2700 1 -- 172.21.15.7:0/2041349460 shutdown_connections 2024-06-10T00:54:38.321 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:37.772+0000 7f30f6da2700 1 -- 172.21.15.7:0/2041349460 wait complete. 2024-06-10T00:54:38.321 INFO:teuthology.orchestra.run.smithi007.stdout:Creating initial admin user... 2024-06-10T00:54:38.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:38 smithi007 ceph-mon[32398]: mgrmap e12: x(active, since 2s) 2024-06-10T00:54:38.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:38 smithi007 ceph-mon[32398]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:54:38.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:38 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:54:38.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:38 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:54:38.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:38 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:54:39.932 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$9FCZci7i6SrZ2MhtBcOUZO11/2nXkqPTI4SkE6jwGQnRFgrfz3kbO", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1717980879, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-06-10T00:54:39.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.798+0000 7f288869a700 1 Processor -- start 2024-06-10T00:54:39.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.798+0000 7f288869a700 1 -- start start 2024-06-10T00:54:39.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.798+0000 7f288869a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2880108390 0x7f28801087b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:39.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.798+0000 7f288869a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2880108d80 con 0x7f2880108390 2024-06-10T00:54:39.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.798+0000 7f2886436700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2880108390 0x7f28801087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:39.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.798+0000 7f2886436700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2880108390 0x7f28801087b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43162/0 (socket says 172.21.15.7:43162) 2024-06-10T00:54:39.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.798+0000 7f2886436700 1 -- 172.21.15.7:0/1898634643 learned_addr learned my addr 172.21.15.7:0/1898634643 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:39.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.799+0000 7f2886436700 1 -- 172.21.15.7:0/1898634643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2880106ff0 con 0x7f2880108390 2024-06-10T00:54:39.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.799+0000 7f2886436700 1 --2- 172.21.15.7:0/1898634643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2880108390 0x7f28801087b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f287c00bb40 tx=0x7f287c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd9ad4f7a9d8fca server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:39.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.799+0000 7f2885434700 1 -- 172.21.15.7:0/1898634643 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f287c00dbe0 con 0x7f2880108390 2024-06-10T00:54:39.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.799+0000 7f2885434700 1 -- 172.21.15.7:0/1898634643 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f287c004510 con 0x7f2880108390 2024-06-10T00:54:39.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.799+0000 7f2885434700 1 -- 172.21.15.7:0/1898634643 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f287c010470 con 0x7f2880108390 2024-06-10T00:54:39.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.800+0000 7f288869a700 1 -- 172.21.15.7:0/1898634643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2880108390 msgr2=0x7f28801087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:39.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.800+0000 7f288869a700 1 --2- 172.21.15.7:0/1898634643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2880108390 0x7f28801087b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f287c00bb40 tx=0x7f287c00be50 comp rx=0 tx=0).stop 2024-06-10T00:54:39.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.800+0000 7f288869a700 1 -- 172.21.15.7:0/1898634643 shutdown_connections 2024-06-10T00:54:39.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.800+0000 7f288869a700 1 --2- 172.21.15.7:0/1898634643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2880108390 0x7f28801087b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:39.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.800+0000 7f288869a700 1 -- 172.21.15.7:0/1898634643 >> 172.21.15.7:0/1898634643 conn(0x7f2880101ac0 msgr2=0x7f2880103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:39.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.800+0000 7f288869a700 1 -- 172.21.15.7:0/1898634643 shutdown_connections 2024-06-10T00:54:39.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.800+0000 7f288869a700 1 -- 172.21.15.7:0/1898634643 wait complete. 2024-06-10T00:54:39.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.801+0000 7f288869a700 1 Processor -- start 2024-06-10T00:54:39.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.801+0000 7f288869a700 1 -- start start 2024-06-10T00:54:39.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.801+0000 7f288869a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2880108390 0x7f288019e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:39.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.801+0000 7f288869a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2880105ee0 con 0x7f2880108390 2024-06-10T00:54:39.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.801+0000 7f2886436700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2880108390 0x7f288019e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:39.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.801+0000 7f2886436700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2880108390 0x7f288019e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43168/0 (socket says 172.21.15.7:43168) 2024-06-10T00:54:39.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.801+0000 7f2886436700 1 -- 172.21.15.7:0/1381811762 learned_addr learned my addr 172.21.15.7:0/1381811762 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:39.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.801+0000 7f2886436700 1 -- 172.21.15.7:0/1381811762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f288019e590 con 0x7f2880108390 2024-06-10T00:54:39.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.801+0000 7f2886436700 1 --2- 172.21.15.7:0/1381811762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2880108390 0x7f288019e050 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f287c00b340 tx=0x7f287c0040b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:39.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.802+0000 7f28737fe700 1 -- 172.21.15.7:0/1381811762 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f287c010770 con 0x7f2880108390 2024-06-10T00:54:39.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.802+0000 7f28737fe700 1 -- 172.21.15.7:0/1381811762 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f287c01d070 con 0x7f2880108390 2024-06-10T00:54:39.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.802+0000 7f288869a700 1 -- 172.21.15.7:0/1381811762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f288019e820 con 0x7f2880108390 2024-06-10T00:54:39.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.802+0000 7f28737fe700 1 -- 172.21.15.7:0/1381811762 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f287c021c30 con 0x7f2880108390 2024-06-10T00:54:39.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.802+0000 7f288869a700 1 -- 172.21.15.7:0/1381811762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28801a1510 con 0x7f2880108390 2024-06-10T00:54:39.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.803+0000 7f28737fe700 1 -- 172.21.15.7:0/1381811762 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f287c01f030 con 0x7f2880108390 2024-06-10T00:54:39.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.803+0000 7f288869a700 1 -- 172.21.15.7:0/1381811762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2880063af0 con 0x7f2880108390 2024-06-10T00:54:39.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.803+0000 7f28737fe700 1 --2- 172.21.15.7:0/1381811762 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f286c0390b0 0x7f286c03b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:39.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.803+0000 7f28737fe700 1 -- 172.21.15.7:0/1381811762 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f287c04d4d0 con 0x7f2880108390 2024-06-10T00:54:39.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.803+0000 7f2885c35700 1 --2- 172.21.15.7:0/1381811762 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f286c0390b0 0x7f286c03b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:39.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.803+0000 7f2885c35700 1 --2- 172.21.15.7:0/1381811762 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f286c0390b0 0x7f286c03b570 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2874006fd0 tx=0x7f2874006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:39.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:38.808+0000 7f28737fe700 1 -- 172.21.15.7:0/1381811762 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f287c015070 con 0x7f2880108390 2024-06-10T00:54:39.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:39.044+0000 7f288869a700 1 -- 172.21.15.7:0/1381811762 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f28801a17a0 con 0x7f286c0390b0 2024-06-10T00:54:39.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:39.290+0000 7f28737fe700 1 -- 172.21.15.7:0/1381811762 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f28801a17a0 con 0x7f286c0390b0 2024-06-10T00:54:39.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:39.293+0000 7f288869a700 1 -- 172.21.15.7:0/1381811762 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f286c0390b0 msgr2=0x7f286c03b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:39.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:39.293+0000 7f288869a700 1 --2- 172.21.15.7:0/1381811762 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f286c0390b0 0x7f286c03b570 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2874006fd0 tx=0x7f2874006e40 comp rx=0 tx=0).stop 2024-06-10T00:54:39.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:39.293+0000 7f288869a700 1 -- 172.21.15.7:0/1381811762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2880108390 msgr2=0x7f288019e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:39.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:39.293+0000 7f288869a700 1 --2- 172.21.15.7:0/1381811762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2880108390 0x7f288019e050 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f287c00b340 tx=0x7f287c0040b0 comp rx=0 tx=0).stop 2024-06-10T00:54:39.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:39.293+0000 7f288869a700 1 -- 172.21.15.7:0/1381811762 shutdown_connections 2024-06-10T00:54:39.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:39.293+0000 7f288869a700 1 --2- 172.21.15.7:0/1381811762 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f286c0390b0 0x7f286c03b570 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:39.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:39.293+0000 7f288869a700 1 --2- 172.21.15.7:0/1381811762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2880108390 0x7f288019e050 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:39.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:39.293+0000 7f288869a700 1 -- 172.21.15.7:0/1381811762 >> 172.21.15.7:0/1381811762 conn(0x7f2880101ac0 msgr2=0x7f2880194690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:39.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:39.293+0000 7f288869a700 1 -- 172.21.15.7:0/1381811762 shutdown_connections 2024-06-10T00:54:39.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:39.293+0000 7f288869a700 1 -- 172.21.15.7:0/1381811762 wait complete. 2024-06-10T00:54:39.937 INFO:teuthology.orchestra.run.smithi007.stdout:Fetching dashboard port number... 2024-06-10T00:54:40.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:39 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:54:40.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:39 smithi007 ceph-mon[32398]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:54:40.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:39 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:54:41.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:40 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1762040007' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-06-10T00:54:41.484 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 8443 2024-06-10T00:54:41.484 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.412+0000 7ff8c5fc0700 1 Processor -- start 2024-06-10T00:54:41.484 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.412+0000 7ff8c5fc0700 1 -- start start 2024-06-10T00:54:41.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.412+0000 7ff8c5fc0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c0108390 0x7ff8c01087b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:41.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.412+0000 7ff8c5fc0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c0108d80 con 0x7ff8c0108390 2024-06-10T00:54:41.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.412+0000 7ff8bf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c0108390 0x7ff8c01087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:41.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.412+0000 7ff8bf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c0108390 0x7ff8c01087b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43172/0 (socket says 172.21.15.7:43172) 2024-06-10T00:54:41.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.412+0000 7ff8bf7fe700 1 -- 172.21.15.7:0/3101251060 learned_addr learned my addr 172.21.15.7:0/3101251060 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:41.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.412+0000 7ff8bf7fe700 1 -- 172.21.15.7:0/3101251060 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c0106ff0 con 0x7ff8c0108390 2024-06-10T00:54:41.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.413+0000 7ff8bf7fe700 1 --2- 172.21.15.7:0/3101251060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c0108390 0x7ff8c01087b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff8b00098a0 tx=0x7ff8b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860950fa8323ecf1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:41.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.413+0000 7ff8be7fc700 1 -- 172.21.15.7:0/3101251060 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8b0004160 con 0x7ff8c0108390 2024-06-10T00:54:41.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.413+0000 7ff8be7fc700 1 -- 172.21.15.7:0/3101251060 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8b0004300 con 0x7ff8c0108390 2024-06-10T00:54:41.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.413+0000 7ff8be7fc700 1 -- 172.21.15.7:0/3101251060 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8b0011b20 con 0x7ff8c0108390 2024-06-10T00:54:41.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.413+0000 7ff8c5fc0700 1 -- 172.21.15.7:0/3101251060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c0108390 msgr2=0x7ff8c01087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:41.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.413+0000 7ff8c5fc0700 1 --2- 172.21.15.7:0/3101251060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c0108390 0x7ff8c01087b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff8b00098a0 tx=0x7ff8b0009bb0 comp rx=0 tx=0).stop 2024-06-10T00:54:41.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.414+0000 7ff8c5fc0700 1 -- 172.21.15.7:0/3101251060 shutdown_connections 2024-06-10T00:54:41.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.414+0000 7ff8c5fc0700 1 --2- 172.21.15.7:0/3101251060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c0108390 0x7ff8c01087b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:41.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.414+0000 7ff8c5fc0700 1 -- 172.21.15.7:0/3101251060 >> 172.21.15.7:0/3101251060 conn(0x7ff8c0101ac0 msgr2=0x7ff8c0103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:41.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.414+0000 7ff8c5fc0700 1 -- 172.21.15.7:0/3101251060 shutdown_connections 2024-06-10T00:54:41.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.414+0000 7ff8c5fc0700 1 -- 172.21.15.7:0/3101251060 wait complete. 2024-06-10T00:54:41.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.414+0000 7ff8c5fc0700 1 Processor -- start 2024-06-10T00:54:41.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.415+0000 7ff8c5fc0700 1 -- start start 2024-06-10T00:54:41.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.415+0000 7ff8c5fc0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c0108390 0x7ff8c0103eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:41.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.415+0000 7ff8c5fc0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c010b3b0 con 0x7ff8c0108390 2024-06-10T00:54:41.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.415+0000 7ff8bf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c0108390 0x7ff8c0103eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:41.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.415+0000 7ff8bf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c0108390 0x7ff8c0103eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43178/0 (socket says 172.21.15.7:43178) 2024-06-10T00:54:41.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.415+0000 7ff8bf7fe700 1 -- 172.21.15.7:0/1762040007 learned_addr learned my addr 172.21.15.7:0/1762040007 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:41.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.415+0000 7ff8bf7fe700 1 -- 172.21.15.7:0/1762040007 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c01043f0 con 0x7ff8c0108390 2024-06-10T00:54:41.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.415+0000 7ff8bf7fe700 1 --2- 172.21.15.7:0/1762040007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c0108390 0x7ff8c0103eb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff8b00117e0 tx=0x7ff8b0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:41.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.416+0000 7ff8bcff9700 1 -- 172.21.15.7:0/1762040007 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8b0003bf0 con 0x7ff8c0108390 2024-06-10T00:54:41.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.416+0000 7ff8bcff9700 1 -- 172.21.15.7:0/1762040007 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8b001a910 con 0x7ff8c0108390 2024-06-10T00:54:41.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.416+0000 7ff8c5fc0700 1 -- 172.21.15.7:0/1762040007 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8c01025e0 con 0x7ff8c0108390 2024-06-10T00:54:41.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.416+0000 7ff8bcff9700 1 -- 172.21.15.7:0/1762040007 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8b0024970 con 0x7ff8c0108390 2024-06-10T00:54:41.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.416+0000 7ff8c5fc0700 1 -- 172.21.15.7:0/1762040007 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8c0102aa0 con 0x7ff8c0108390 2024-06-10T00:54:41.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.417+0000 7ff8bcff9700 1 -- 172.21.15.7:0/1762040007 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7ff8b0022020 con 0x7ff8c0108390 2024-06-10T00:54:41.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.417+0000 7ff8c5fc0700 1 -- 172.21.15.7:0/1762040007 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8c0063af0 con 0x7ff8c0108390 2024-06-10T00:54:41.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.417+0000 7ff8bcff9700 1 --2- 172.21.15.7:0/1762040007 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff8a8039100 0x7ff8a803b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:41.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.417+0000 7ff8bcff9700 1 -- 172.21.15.7:0/1762040007 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff8b004e4f0 con 0x7ff8c0108390 2024-06-10T00:54:41.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.417+0000 7ff8beffd700 1 --2- 172.21.15.7:0/1762040007 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff8a8039100 0x7ff8a803b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:41.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.418+0000 7ff8beffd700 1 --2- 172.21.15.7:0/1762040007 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff8a8039100 0x7ff8a803b5c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4006fd0 tx=0x7ff8b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:41.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.424+0000 7ff8bcff9700 1 -- 172.21.15.7:0/1762040007 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff8b002cdf0 con 0x7ff8c0108390 2024-06-10T00:54:41.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.636+0000 7ff8c5fc0700 1 -- 172.21.15.7:0/1762040007 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7ff8c0195650 con 0x7ff8c0108390 2024-06-10T00:54:41.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.637+0000 7ff8bcff9700 1 -- 172.21.15.7:0/1762040007 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7ff8b0020c10 con 0x7ff8c0108390 2024-06-10T00:54:41.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.640+0000 7ff8c5fc0700 1 -- 172.21.15.7:0/1762040007 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff8a8039100 msgr2=0x7ff8a803b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:41.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.640+0000 7ff8c5fc0700 1 --2- 172.21.15.7:0/1762040007 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff8a8039100 0x7ff8a803b5c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4006fd0 tx=0x7ff8b4006e40 comp rx=0 tx=0).stop 2024-06-10T00:54:41.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.640+0000 7ff8c5fc0700 1 -- 172.21.15.7:0/1762040007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c0108390 msgr2=0x7ff8c0103eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:41.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.640+0000 7ff8c5fc0700 1 --2- 172.21.15.7:0/1762040007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c0108390 0x7ff8c0103eb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff8b00117e0 tx=0x7ff8b0011ba0 comp rx=0 tx=0).stop 2024-06-10T00:54:41.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.640+0000 7ff8c5fc0700 1 -- 172.21.15.7:0/1762040007 shutdown_connections 2024-06-10T00:54:41.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.640+0000 7ff8c5fc0700 1 --2- 172.21.15.7:0/1762040007 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff8a8039100 0x7ff8a803b5c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:41.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.640+0000 7ff8c5fc0700 1 --2- 172.21.15.7:0/1762040007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c0108390 0x7ff8c0103eb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:41.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.640+0000 7ff8c5fc0700 1 -- 172.21.15.7:0/1762040007 >> 172.21.15.7:0/1762040007 conn(0x7ff8c0101ac0 msgr2=0x7ff8c0192970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:41.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.640+0000 7ff8c5fc0700 1 -- 172.21.15.7:0/1762040007 shutdown_connections 2024-06-10T00:54:41.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:40.640+0000 7ff8c5fc0700 1 -- 172.21.15.7:0/1762040007 wait complete. 2024-06-10T00:54:41.495 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-10T00:54:41.495 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-06-10T00:54:41.503 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-10T00:54:41.503 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-06-10T00:54:41.503 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-06-10T00:54:41.503 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-06-10T00:54:41.505 INFO:teuthology.orchestra.run.smithi007.stdout:Ceph Dashboard is now available at: 2024-06-10T00:54:41.505 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:54:41.505 INFO:teuthology.orchestra.run.smithi007.stdout: URL: https://smithi007.front.sepia.ceph.com:8443/ 2024-06-10T00:54:41.505 INFO:teuthology.orchestra.run.smithi007.stdout: User: admin 2024-06-10T00:54:41.505 INFO:teuthology.orchestra.run.smithi007.stdout: Password: 58zq2w6p35 2024-06-10T00:54:41.505 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:54:41.506 INFO:teuthology.orchestra.run.smithi007.stdout:Saving cluster configuration to /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/config directory 2024-06-10T00:54:41.506 INFO:teuthology.orchestra.run.smithi007.stdout:Enabling autotune for osd_memory_target 2024-06-10T00:54:43.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.240+0000 7fe987565700 1 Processor -- start 2024-06-10T00:54:43.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.240+0000 7fe987565700 1 -- start start 2024-06-10T00:54:43.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.240+0000 7fe987565700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9801089b0 0x7fe98010ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:43.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.240+0000 7fe987565700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe98007c680 con 0x7fe9801089b0 2024-06-10T00:54:43.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.240+0000 7fe985301700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9801089b0 0x7fe98010ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:43.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.240+0000 7fe985301700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9801089b0 0x7fe98010ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45820/0 (socket says 172.21.15.7:45820) 2024-06-10T00:54:43.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.240+0000 7fe985301700 1 -- 172.21.15.7:0/1451030817 learned_addr learned my addr 172.21.15.7:0/1451030817 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:43.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.241+0000 7fe985301700 1 -- 172.21.15.7:0/1451030817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98010b2e0 con 0x7fe9801089b0 2024-06-10T00:54:43.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.241+0000 7fe985301700 1 --2- 172.21.15.7:0/1451030817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9801089b0 0x7fe98010ada0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe97c0098a0 tx=0x7fe97c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=decb7fea094b6e4f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:43.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.241+0000 7fe977fff700 1 -- 172.21.15.7:0/1451030817 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe97c004160 con 0x7fe9801089b0 2024-06-10T00:54:43.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.241+0000 7fe977fff700 1 -- 172.21.15.7:0/1451030817 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe97c004300 con 0x7fe9801089b0 2024-06-10T00:54:43.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.241+0000 7fe977fff700 1 -- 172.21.15.7:0/1451030817 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe97c011b20 con 0x7fe9801089b0 2024-06-10T00:54:43.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.241+0000 7fe987565700 1 -- 172.21.15.7:0/1451030817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9801089b0 msgr2=0x7fe98010ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:43.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.241+0000 7fe987565700 1 --2- 172.21.15.7:0/1451030817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9801089b0 0x7fe98010ada0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe97c0098a0 tx=0x7fe97c009bb0 comp rx=0 tx=0).stop 2024-06-10T00:54:43.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.242+0000 7fe987565700 1 -- 172.21.15.7:0/1451030817 shutdown_connections 2024-06-10T00:54:43.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.242+0000 7fe987565700 1 --2- 172.21.15.7:0/1451030817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9801089b0 0x7fe98010ada0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:43.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.242+0000 7fe987565700 1 -- 172.21.15.7:0/1451030817 >> 172.21.15.7:0/1451030817 conn(0x7fe980102380 msgr2=0x7fe9801047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:43.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.242+0000 7fe987565700 1 -- 172.21.15.7:0/1451030817 shutdown_connections 2024-06-10T00:54:43.003 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.242+0000 7fe987565700 1 -- 172.21.15.7:0/1451030817 wait complete. 2024-06-10T00:54:43.003 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.242+0000 7fe987565700 1 Processor -- start 2024-06-10T00:54:43.003 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.242+0000 7fe987565700 1 -- start start 2024-06-10T00:54:43.003 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.242+0000 7fe987565700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9801089b0 0x7fe98019bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:43.003 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.242+0000 7fe987565700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe98010c280 con 0x7fe9801089b0 2024-06-10T00:54:43.003 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.243+0000 7fe985301700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9801089b0 0x7fe98019bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:43.003 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.243+0000 7fe985301700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9801089b0 0x7fe98019bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45834/0 (socket says 172.21.15.7:45834) 2024-06-10T00:54:43.003 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.243+0000 7fe985301700 1 -- 172.21.15.7:0/3184711063 learned_addr learned my addr 172.21.15.7:0/3184711063 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:43.003 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.243+0000 7fe985301700 1 -- 172.21.15.7:0/3184711063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98019c220 con 0x7fe9801089b0 2024-06-10T00:54:43.003 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.243+0000 7fe985301700 1 --2- 172.21.15.7:0/3184711063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9801089b0 0x7fe98019bce0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe97c0117e0 tx=0x7fe97c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:43.003 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.243+0000 7fe9767fc700 1 -- 172.21.15.7:0/3184711063 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe97c003bf0 con 0x7fe9801089b0 2024-06-10T00:54:43.003 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.243+0000 7fe9767fc700 1 -- 172.21.15.7:0/3184711063 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe97c01a910 con 0x7fe9801089b0 2024-06-10T00:54:43.003 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.243+0000 7fe987565700 1 -- 172.21.15.7:0/3184711063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe98019c4b0 con 0x7fe9801089b0 2024-06-10T00:54:43.003 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.243+0000 7fe9767fc700 1 -- 172.21.15.7:0/3184711063 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe97c024970 con 0x7fe9801089b0 2024-06-10T00:54:43.003 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.243+0000 7fe987565700 1 -- 172.21.15.7:0/3184711063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe98019c910 con 0x7fe9801089b0 2024-06-10T00:54:43.004 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.244+0000 7fe9767fc700 1 -- 172.21.15.7:0/3184711063 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fe97c022020 con 0x7fe9801089b0 2024-06-10T00:54:43.004 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.244+0000 7fe987565700 1 -- 172.21.15.7:0/3184711063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe980063af0 con 0x7fe9801089b0 2024-06-10T00:54:43.004 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.244+0000 7fe9767fc700 1 --2- 172.21.15.7:0/3184711063 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe96c0390f0 0x7fe96c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:43.004 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.244+0000 7fe9767fc700 1 -- 172.21.15.7:0/3184711063 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe97c04d6b0 con 0x7fe9801089b0 2024-06-10T00:54:43.004 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.244+0000 7fe984b00700 1 --2- 172.21.15.7:0/3184711063 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe96c0390f0 0x7fe96c03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:43.004 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.245+0000 7fe984b00700 1 --2- 172.21.15.7:0/3184711063 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe96c0390f0 0x7fe96c03b5b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe970006fd0 tx=0x7fe970006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:43.004 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.250+0000 7fe9767fc700 1 -- 172.21.15.7:0/3184711063 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe97c011e20 con 0x7fe9801089b0 2024-06-10T00:54:43.004 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.454+0000 7fe987565700 1 -- 172.21.15.7:0/3184711063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fe98019f370 con 0x7fe9801089b0 2024-06-10T00:54:43.004 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.454+0000 7fe9767fc700 1 -- 172.21.15.7:0/3184711063 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fe97c00ba30 con 0x7fe9801089b0 2024-06-10T00:54:43.004 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.456+0000 7fe987565700 1 -- 172.21.15.7:0/3184711063 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe96c0390f0 msgr2=0x7fe96c03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:43.004 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.456+0000 7fe987565700 1 --2- 172.21.15.7:0/3184711063 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe96c0390f0 0x7fe96c03b5b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe970006fd0 tx=0x7fe970006e40 comp rx=0 tx=0).stop 2024-06-10T00:54:43.004 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.456+0000 7fe987565700 1 -- 172.21.15.7:0/3184711063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9801089b0 msgr2=0x7fe98019bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:43.004 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.456+0000 7fe987565700 1 --2- 172.21.15.7:0/3184711063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9801089b0 0x7fe98019bce0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe97c0117e0 tx=0x7fe97c011c10 comp rx=0 tx=0).stop 2024-06-10T00:54:43.004 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.456+0000 7fe987565700 1 -- 172.21.15.7:0/3184711063 shutdown_connections 2024-06-10T00:54:43.005 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.456+0000 7fe987565700 1 --2- 172.21.15.7:0/3184711063 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe96c0390f0 0x7fe96c03b5b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:43.005 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.456+0000 7fe987565700 1 --2- 172.21.15.7:0/3184711063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9801089b0 0x7fe98019bce0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:43.005 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.456+0000 7fe987565700 1 -- 172.21.15.7:0/3184711063 >> 172.21.15.7:0/3184711063 conn(0x7fe980102380 msgr2=0x7fe980102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:43.005 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.456+0000 7fe987565700 1 -- 172.21.15.7:0/3184711063 shutdown_connections 2024-06-10T00:54:43.005 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:42.456+0000 7fe987565700 1 -- 172.21.15.7:0/3184711063 wait complete. 2024-06-10T00:54:43.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:42 smithi007 ceph-mon[32398]: mgrmap e13: x(active, since 6s) 2024-06-10T00:54:44.307 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.426+0000 7fb44762b700 1 Processor -- start 2024-06-10T00:54:44.307 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.427+0000 7fb44762b700 1 -- start start 2024-06-10T00:54:44.307 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.427+0000 7fb44762b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb440107e50 0x7fb440108270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:44.307 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.427+0000 7fb44762b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb4401087b0 con 0x7fb440107e50 2024-06-10T00:54:44.307 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.427+0000 7fb4453c7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb440107e50 0x7fb440108270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:44.307 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.427+0000 7fb4453c7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb440107e50 0x7fb440108270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45838/0 (socket says 172.21.15.7:45838) 2024-06-10T00:54:44.307 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.427+0000 7fb4453c7700 1 -- 172.21.15.7:0/2476633732 learned_addr learned my addr 172.21.15.7:0/2476633732 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:44.307 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.427+0000 7fb4453c7700 1 -- 172.21.15.7:0/2476633732 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb440109040 con 0x7fb440107e50 2024-06-10T00:54:44.308 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.427+0000 7fb4453c7700 1 --2- 172.21.15.7:0/2476633732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb440107e50 0x7fb440108270 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb43c0098a0 tx=0x7fb43c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d81b97429680229 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:44.308 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.428+0000 7fb437fff700 1 -- 172.21.15.7:0/2476633732 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb43c004160 con 0x7fb440107e50 2024-06-10T00:54:44.308 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.428+0000 7fb437fff700 1 -- 172.21.15.7:0/2476633732 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb43c004300 con 0x7fb440107e50 2024-06-10T00:54:44.308 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.428+0000 7fb437fff700 1 -- 172.21.15.7:0/2476633732 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb43c011b20 con 0x7fb440107e50 2024-06-10T00:54:44.308 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.428+0000 7fb44762b700 1 -- 172.21.15.7:0/2476633732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb440107e50 msgr2=0x7fb440108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:44.308 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.428+0000 7fb44762b700 1 --2- 172.21.15.7:0/2476633732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb440107e50 0x7fb440108270 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb43c0098a0 tx=0x7fb43c009bb0 comp rx=0 tx=0).stop 2024-06-10T00:54:44.308 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.428+0000 7fb44762b700 1 -- 172.21.15.7:0/2476633732 shutdown_connections 2024-06-10T00:54:44.308 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.428+0000 7fb44762b700 1 --2- 172.21.15.7:0/2476633732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb440107e50 0x7fb440108270 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:44.308 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.428+0000 7fb44762b700 1 -- 172.21.15.7:0/2476633732 >> 172.21.15.7:0/2476633732 conn(0x7fb440103420 msgr2=0x7fb440105880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:44.308 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.428+0000 7fb44762b700 1 -- 172.21.15.7:0/2476633732 shutdown_connections 2024-06-10T00:54:44.308 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.428+0000 7fb44762b700 1 -- 172.21.15.7:0/2476633732 wait complete. 2024-06-10T00:54:44.308 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.429+0000 7fb44762b700 1 Processor -- start 2024-06-10T00:54:44.308 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.429+0000 7fb44762b700 1 -- start start 2024-06-10T00:54:44.308 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.429+0000 7fb44762b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb440107e50 0x7fb44007db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:44.309 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.429+0000 7fb44762b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb44010c300 con 0x7fb440107e50 2024-06-10T00:54:44.309 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.429+0000 7fb4453c7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb440107e50 0x7fb44007db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:44.309 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.429+0000 7fb4453c7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb440107e50 0x7fb44007db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45848/0 (socket says 172.21.15.7:45848) 2024-06-10T00:54:44.309 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.429+0000 7fb4453c7700 1 -- 172.21.15.7:0/3683555147 learned_addr learned my addr 172.21.15.7:0/3683555147 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:44.309 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.430+0000 7fb4453c7700 1 -- 172.21.15.7:0/3683555147 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb44007e0c0 con 0x7fb440107e50 2024-06-10T00:54:44.309 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.430+0000 7fb4453c7700 1 --2- 172.21.15.7:0/3683555147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb440107e50 0x7fb44007db80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb43c0117e0 tx=0x7fb43c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:44.309 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.430+0000 7fb4367fc700 1 -- 172.21.15.7:0/3683555147 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb43c003bf0 con 0x7fb440107e50 2024-06-10T00:54:44.309 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.430+0000 7fb4367fc700 1 -- 172.21.15.7:0/3683555147 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb43c01a910 con 0x7fb440107e50 2024-06-10T00:54:44.309 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.430+0000 7fb4367fc700 1 -- 172.21.15.7:0/3683555147 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb43c024970 con 0x7fb440107e50 2024-06-10T00:54:44.309 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.430+0000 7fb44762b700 1 -- 172.21.15.7:0/3683555147 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb44007c2b0 con 0x7fb440107e50 2024-06-10T00:54:44.309 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.430+0000 7fb44762b700 1 -- 172.21.15.7:0/3683555147 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb44007c770 con 0x7fb440107e50 2024-06-10T00:54:44.309 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.431+0000 7fb4367fc700 1 -- 172.21.15.7:0/3683555147 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fb43c022020 con 0x7fb440107e50 2024-06-10T00:54:44.309 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.431+0000 7fb44762b700 1 -- 172.21.15.7:0/3683555147 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb440063af0 con 0x7fb440107e50 2024-06-10T00:54:44.309 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.431+0000 7fb4367fc700 1 --2- 172.21.15.7:0/3683555147 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb42c0391a0 0x7fb42c03b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:44.309 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.431+0000 7fb4367fc700 1 -- 172.21.15.7:0/3683555147 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb43c04d580 con 0x7fb440107e50 2024-06-10T00:54:44.310 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.431+0000 7fb444bc6700 1 --2- 172.21.15.7:0/3683555147 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb42c0391a0 0x7fb42c03b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:44.310 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.432+0000 7fb444bc6700 1 --2- 172.21.15.7:0/3683555147 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb42c0391a0 0x7fb42c03b660 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb430006fd0 tx=0x7fb430006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:44.310 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.436+0000 7fb4367fc700 1 -- 172.21.15.7:0/3683555147 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb43c02d910 con 0x7fb440107e50 2024-06-10T00:54:44.310 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.755+0000 7fb44762b700 1 -- 172.21.15.7:0/3683555147 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fb44007c9f0 con 0x7fb440107e50 2024-06-10T00:54:44.310 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.758+0000 7fb4367fc700 1 -- 172.21.15.7:0/3683555147 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fb43c02d3f0 con 0x7fb440107e50 2024-06-10T00:54:44.310 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.760+0000 7fb44762b700 1 -- 172.21.15.7:0/3683555147 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb42c0391a0 msgr2=0x7fb42c03b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:44.310 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.760+0000 7fb44762b700 1 --2- 172.21.15.7:0/3683555147 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb42c0391a0 0x7fb42c03b660 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb430006fd0 tx=0x7fb430006e40 comp rx=0 tx=0).stop 2024-06-10T00:54:44.310 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.760+0000 7fb44762b700 1 -- 172.21.15.7:0/3683555147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb440107e50 msgr2=0x7fb44007db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:44.310 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.760+0000 7fb44762b700 1 --2- 172.21.15.7:0/3683555147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb440107e50 0x7fb44007db80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb43c0117e0 tx=0x7fb43c011ba0 comp rx=0 tx=0).stop 2024-06-10T00:54:44.310 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.760+0000 7fb44762b700 1 -- 172.21.15.7:0/3683555147 shutdown_connections 2024-06-10T00:54:44.310 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.760+0000 7fb44762b700 1 --2- 172.21.15.7:0/3683555147 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb42c0391a0 0x7fb42c03b660 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:44.310 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.760+0000 7fb44762b700 1 --2- 172.21.15.7:0/3683555147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb440107e50 0x7fb44007db80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:44.311 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.760+0000 7fb44762b700 1 -- 172.21.15.7:0/3683555147 >> 172.21.15.7:0/3683555147 conn(0x7fb440103420 msgr2=0x7fb440103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:44.311 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.760+0000 7fb44762b700 1 -- 172.21.15.7:0/3683555147 shutdown_connections 2024-06-10T00:54:44.311 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-10T00:54:43.760+0000 7fb44762b700 1 -- 172.21.15.7:0/3683555147 wait complete. 2024-06-10T00:54:44.311 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-06-10T00:54:44.311 INFO:teuthology.orchestra.run.smithi007.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-06-10T00:54:44.311 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:54:44.311 INFO:teuthology.orchestra.run.smithi007.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-06-10T00:54:44.311 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:54:44.311 INFO:teuthology.orchestra.run.smithi007.stdout:Or, if you are only running a single cluster on this host: 2024-06-10T00:54:44.311 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:54:44.311 INFO:teuthology.orchestra.run.smithi007.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-06-10T00:54:44.311 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:54:44.311 INFO:teuthology.orchestra.run.smithi007.stdout:Please consider enabling telemetry to help improve Ceph: 2024-06-10T00:54:44.311 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:54:44.311 INFO:teuthology.orchestra.run.smithi007.stdout: ceph telemetry on 2024-06-10T00:54:44.312 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:54:44.312 INFO:teuthology.orchestra.run.smithi007.stdout:For more information see: 2024-06-10T00:54:44.312 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:54:44.312 INFO:teuthology.orchestra.run.smithi007.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-06-10T00:54:44.312 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:54:44.312 INFO:teuthology.orchestra.run.smithi007.stdout:Bootstrap complete. 2024-06-10T00:54:44.347 INFO:tasks.cephadm:Fetching config... 2024-06-10T00:54:44.347 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-10T00:54:44.347 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-06-10T00:54:44.364 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-06-10T00:54:44.364 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-10T00:54:44.364 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-06-10T00:54:44.420 INFO:tasks.cephadm:Fetching mon keyring... 2024-06-10T00:54:44.420 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-10T00:54:44.420 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/keyring of=/dev/stdout 2024-06-10T00:54:44.487 INFO:tasks.cephadm:Fetching pub ssh key... 2024-06-10T00:54:44.487 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-10T00:54:44.487 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-06-10T00:54:44.543 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-06-10T00:54:44.543 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrNSN1hwwYXdkBHnqcKM/Xx/5XPh3yb5JrIq4V63yp/JkF3Jv6j9EovvlxTjnMc+mzkCEmLL+FqMvTFqWMTdUaCyHeEsF/akdW+WdJIoWjVZ2ssyez8XKzk06evtLIJbuXDSlwgqcib0Y3SMTbWcpzOa6Lr04cRJW+dDP1Cnr3JjJRgrMJtR1EBbPaqXd9wQTVJzt6HH6Um4hoAsApo9FHkC9gEk3cNWlYjjDkz35qmwGbv8V8uoHeitWlyTbYe1Q/aOL25rZyAOz3k/y8g+CDvV9ob7WujoAdlPMdFxhjRx+QGAeRXHcgXlPLAt/I1GbVV+XZV3mAJaTVMA0PtAtQG60o+PRfdAvsxfgCFYyHZWWpTepCR2tYGPFwjLwq3JHeJu6liJUAP75cX1DiVqNG2RdY2fwouyZDQYRZevAm5x8XnVdcEc5swIZjRQMntQ6Josh9F1nwE966l4kK7+kbqRoTAyUAPMFLS9cO7KXGgbLrBH2rKVsz8W38T5Qusgc= ceph-bded58f8-26c3-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-10T00:54:44.621 INFO:teuthology.orchestra.run.smithi007.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrNSN1hwwYXdkBHnqcKM/Xx/5XPh3yb5JrIq4V63yp/JkF3Jv6j9EovvlxTjnMc+mzkCEmLL+FqMvTFqWMTdUaCyHeEsF/akdW+WdJIoWjVZ2ssyez8XKzk06evtLIJbuXDSlwgqcib0Y3SMTbWcpzOa6Lr04cRJW+dDP1Cnr3JjJRgrMJtR1EBbPaqXd9wQTVJzt6HH6Um4hoAsApo9FHkC9gEk3cNWlYjjDkz35qmwGbv8V8uoHeitWlyTbYe1Q/aOL25rZyAOz3k/y8g+CDvV9ob7WujoAdlPMdFxhjRx+QGAeRXHcgXlPLAt/I1GbVV+XZV3mAJaTVMA0PtAtQG60o+PRfdAvsxfgCFYyHZWWpTepCR2tYGPFwjLwq3JHeJu6liJUAP75cX1DiVqNG2RdY2fwouyZDQYRZevAm5x8XnVdcEc5swIZjRQMntQ6Josh9F1nwE966l4kK7+kbqRoTAyUAPMFLS9cO7KXGgbLrBH2rKVsz8W38T5Qusgc= ceph-bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T00:54:44.632 DEBUG:teuthology.orchestra.run.smithi078:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrNSN1hwwYXdkBHnqcKM/Xx/5XPh3yb5JrIq4V63yp/JkF3Jv6j9EovvlxTjnMc+mzkCEmLL+FqMvTFqWMTdUaCyHeEsF/akdW+WdJIoWjVZ2ssyez8XKzk06evtLIJbuXDSlwgqcib0Y3SMTbWcpzOa6Lr04cRJW+dDP1Cnr3JjJRgrMJtR1EBbPaqXd9wQTVJzt6HH6Um4hoAsApo9FHkC9gEk3cNWlYjjDkz35qmwGbv8V8uoHeitWlyTbYe1Q/aOL25rZyAOz3k/y8g+CDvV9ob7WujoAdlPMdFxhjRx+QGAeRXHcgXlPLAt/I1GbVV+XZV3mAJaTVMA0PtAtQG60o+PRfdAvsxfgCFYyHZWWpTepCR2tYGPFwjLwq3JHeJu6liJUAP75cX1DiVqNG2RdY2fwouyZDQYRZevAm5x8XnVdcEc5swIZjRQMntQ6Josh9F1nwE966l4kK7+kbqRoTAyUAPMFLS9cO7KXGgbLrBH2rKVsz8W38T5Qusgc= ceph-bded58f8-26c3-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-10T00:54:44.671 INFO:teuthology.orchestra.run.smithi078.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrNSN1hwwYXdkBHnqcKM/Xx/5XPh3yb5JrIq4V63yp/JkF3Jv6j9EovvlxTjnMc+mzkCEmLL+FqMvTFqWMTdUaCyHeEsF/akdW+WdJIoWjVZ2ssyez8XKzk06evtLIJbuXDSlwgqcib0Y3SMTbWcpzOa6Lr04cRJW+dDP1Cnr3JjJRgrMJtR1EBbPaqXd9wQTVJzt6HH6Um4hoAsApo9FHkC9gEk3cNWlYjjDkz35qmwGbv8V8uoHeitWlyTbYe1Q/aOL25rZyAOz3k/y8g+CDvV9ob7WujoAdlPMdFxhjRx+QGAeRXHcgXlPLAt/I1GbVV+XZV3mAJaTVMA0PtAtQG60o+PRfdAvsxfgCFYyHZWWpTepCR2tYGPFwjLwq3JHeJu6liJUAP75cX1DiVqNG2RdY2fwouyZDQYRZevAm5x8XnVdcEc5swIZjRQMntQ6Josh9F1nwE966l4kK7+kbqRoTAyUAPMFLS9cO7KXGgbLrBH2rKVsz8W38T5Qusgc= ceph-bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T00:54:44.683 DEBUG:teuthology.orchestra.run.smithi175:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrNSN1hwwYXdkBHnqcKM/Xx/5XPh3yb5JrIq4V63yp/JkF3Jv6j9EovvlxTjnMc+mzkCEmLL+FqMvTFqWMTdUaCyHeEsF/akdW+WdJIoWjVZ2ssyez8XKzk06evtLIJbuXDSlwgqcib0Y3SMTbWcpzOa6Lr04cRJW+dDP1Cnr3JjJRgrMJtR1EBbPaqXd9wQTVJzt6HH6Um4hoAsApo9FHkC9gEk3cNWlYjjDkz35qmwGbv8V8uoHeitWlyTbYe1Q/aOL25rZyAOz3k/y8g+CDvV9ob7WujoAdlPMdFxhjRx+QGAeRXHcgXlPLAt/I1GbVV+XZV3mAJaTVMA0PtAtQG60o+PRfdAvsxfgCFYyHZWWpTepCR2tYGPFwjLwq3JHeJu6liJUAP75cX1DiVqNG2RdY2fwouyZDQYRZevAm5x8XnVdcEc5swIZjRQMntQ6Josh9F1nwE966l4kK7+kbqRoTAyUAPMFLS9cO7KXGgbLrBH2rKVsz8W38T5Qusgc= ceph-bded58f8-26c3-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-10T00:54:44.722 INFO:teuthology.orchestra.run.smithi175.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrNSN1hwwYXdkBHnqcKM/Xx/5XPh3yb5JrIq4V63yp/JkF3Jv6j9EovvlxTjnMc+mzkCEmLL+FqMvTFqWMTdUaCyHeEsF/akdW+WdJIoWjVZ2ssyez8XKzk06evtLIJbuXDSlwgqcib0Y3SMTbWcpzOa6Lr04cRJW+dDP1Cnr3JjJRgrMJtR1EBbPaqXd9wQTVJzt6HH6Um4hoAsApo9FHkC9gEk3cNWlYjjDkz35qmwGbv8V8uoHeitWlyTbYe1Q/aOL25rZyAOz3k/y8g+CDvV9ob7WujoAdlPMdFxhjRx+QGAeRXHcgXlPLAt/I1GbVV+XZV3mAJaTVMA0PtAtQG60o+PRfdAvsxfgCFYyHZWWpTepCR2tYGPFwjLwq3JHeJu6liJUAP75cX1DiVqNG2RdY2fwouyZDQYRZevAm5x8XnVdcEc5swIZjRQMntQ6Josh9F1nwE966l4kK7+kbqRoTAyUAPMFLS9cO7KXGgbLrBH2rKVsz8W38T5Qusgc= ceph-bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T00:54:44.734 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-06-10T00:54:44.983 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:45.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:44 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3683555147' entity='client.admin' 2024-06-10T00:54:47.454 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:47 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:54:47.454 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:47 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:54:47.454 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:47 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi007", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:54:47.454 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:47 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:54:47.454 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:47 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:54:47.454 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:47 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:54:47.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.478+0000 7fde862ba700 1 -- 172.21.15.7:0/2411063652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde80100210 msgr2=0x7fde80100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:47.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.478+0000 7fde862ba700 1 --2- 172.21.15.7:0/2411063652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde80100210 0x7fde80100630 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fde70009910 tx=0x7fde70009c20 comp rx=0 tx=0).stop 2024-06-10T00:54:47.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.478+0000 7fde862ba700 1 -- 172.21.15.7:0/2411063652 shutdown_connections 2024-06-10T00:54:47.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.478+0000 7fde862ba700 1 --2- 172.21.15.7:0/2411063652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde80100210 0x7fde80100630 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:47.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.478+0000 7fde862ba700 1 -- 172.21.15.7:0/2411063652 >> 172.21.15.7:0/2411063652 conn(0x7fde800fbb30 msgr2=0x7fde800fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:47.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.478+0000 7fde862ba700 1 -- 172.21.15.7:0/2411063652 shutdown_connections 2024-06-10T00:54:47.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.478+0000 7fde862ba700 1 -- 172.21.15.7:0/2411063652 wait complete. 2024-06-10T00:54:47.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.479+0000 7fde862ba700 1 Processor -- start 2024-06-10T00:54:47.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.479+0000 7fde862ba700 1 -- start start 2024-06-10T00:54:47.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.479+0000 7fde862ba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde80100210 0x7fde801975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:47.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.479+0000 7fde862ba700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fde80107f60 con 0x7fde80100210 2024-06-10T00:54:47.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.479+0000 7fde7ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde80100210 0x7fde801975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:47.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.479+0000 7fde7ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde80100210 0x7fde801975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45868/0 (socket says 172.21.15.7:45868) 2024-06-10T00:54:47.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.479+0000 7fde7ffff700 1 -- 172.21.15.7:0/1079398388 learned_addr learned my addr 172.21.15.7:0/1079398388 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:47.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.480+0000 7fde7ffff700 1 -- 172.21.15.7:0/1079398388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde80197ae0 con 0x7fde80100210 2024-06-10T00:54:47.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.480+0000 7fde7ffff700 1 --2- 172.21.15.7:0/1079398388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde80100210 0x7fde801975a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fde70006010 tx=0x7fde70005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:47.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.480+0000 7fde7d7fa700 1 -- 172.21.15.7:0/1079398388 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde70027070 con 0x7fde80100210 2024-06-10T00:54:47.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.480+0000 7fde7d7fa700 1 -- 172.21.15.7:0/1079398388 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde7000be40 con 0x7fde80100210 2024-06-10T00:54:47.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.480+0000 7fde7d7fa700 1 -- 172.21.15.7:0/1079398388 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde7000f460 con 0x7fde80100210 2024-06-10T00:54:47.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.480+0000 7fde862ba700 1 -- 172.21.15.7:0/1079398388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde80197dd0 con 0x7fde80100210 2024-06-10T00:54:47.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.480+0000 7fde862ba700 1 -- 172.21.15.7:0/1079398388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde8019aa60 con 0x7fde80100210 2024-06-10T00:54:47.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.481+0000 7fde7d7fa700 1 -- 172.21.15.7:0/1079398388 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fde7002a070 con 0x7fde80100210 2024-06-10T00:54:47.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.481+0000 7fde862ba700 1 -- 172.21.15.7:0/1079398388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde801086e0 con 0x7fde80100210 2024-06-10T00:54:47.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.481+0000 7fde7d7fa700 1 --2- 172.21.15.7:0/1079398388 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fde680390f0 0x7fde6803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:47.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.481+0000 7fde7d7fa700 1 -- 172.21.15.7:0/1079398388 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fde7001e030 con 0x7fde80100210 2024-06-10T00:54:47.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.481+0000 7fde7f7fe700 1 --2- 172.21.15.7:0/1079398388 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fde680390f0 0x7fde6803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:47.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.482+0000 7fde7f7fe700 1 --2- 172.21.15.7:0/1079398388 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fde680390f0 0x7fde6803b5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fde74006fd0 tx=0x7fde74006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:47.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.486+0000 7fde7d7fa700 1 -- 172.21.15.7:0/1079398388 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fde70013070 con 0x7fde80100210 2024-06-10T00:54:47.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.692+0000 7fde862ba700 1 -- 172.21.15.7:0/1079398388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fde8002de90 con 0x7fde80100210 2024-06-10T00:54:47.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.694+0000 7fde7d7fa700 1 -- 172.21.15.7:0/1079398388 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fde70031020 con 0x7fde80100210 2024-06-10T00:54:47.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.697+0000 7fde862ba700 1 -- 172.21.15.7:0/1079398388 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fde680390f0 msgr2=0x7fde6803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:47.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.697+0000 7fde862ba700 1 --2- 172.21.15.7:0/1079398388 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fde680390f0 0x7fde6803b5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fde74006fd0 tx=0x7fde74006e40 comp rx=0 tx=0).stop 2024-06-10T00:54:47.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.697+0000 7fde862ba700 1 -- 172.21.15.7:0/1079398388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde80100210 msgr2=0x7fde801975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:47.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.697+0000 7fde862ba700 1 --2- 172.21.15.7:0/1079398388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde80100210 0x7fde801975a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fde70006010 tx=0x7fde70005050 comp rx=0 tx=0).stop 2024-06-10T00:54:47.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.697+0000 7fde862ba700 1 -- 172.21.15.7:0/1079398388 shutdown_connections 2024-06-10T00:54:47.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.697+0000 7fde862ba700 1 --2- 172.21.15.7:0/1079398388 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fde680390f0 0x7fde6803b5b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:47.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.697+0000 7fde862ba700 1 --2- 172.21.15.7:0/1079398388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde80100210 0x7fde801975a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:47.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.697+0000 7fde862ba700 1 -- 172.21.15.7:0/1079398388 >> 172.21.15.7:0/1079398388 conn(0x7fde800fbb30 msgr2=0x7fde800fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:47.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.697+0000 7fde862ba700 1 -- 172.21.15.7:0/1079398388 shutdown_connections 2024-06-10T00:54:47.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:47.697+0000 7fde862ba700 1 -- 172.21.15.7:0/1079398388 wait complete. 2024-06-10T00:54:48.811 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-06-10T00:54:48.811 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-06-10T00:54:48.837 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:48 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1079398388' entity='client.admin' 2024-06-10T00:54:48.837 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:48 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:54:48.837 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:48 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:54:49.062 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:50.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.354+0000 7f24bd16a700 1 -- 172.21.15.7:0/3131264975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24b8103560 msgr2=0x7f24b8103980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:50.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.354+0000 7f24bd16a700 1 --2- 172.21.15.7:0/3131264975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24b8103560 0x7f24b8103980 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f24a8009910 tx=0x7f24a8009c20 comp rx=0 tx=0).stop 2024-06-10T00:54:50.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.354+0000 7f24bd16a700 1 -- 172.21.15.7:0/3131264975 shutdown_connections 2024-06-10T00:54:50.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.354+0000 7f24bd16a700 1 --2- 172.21.15.7:0/3131264975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24b8103560 0x7f24b8103980 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:50.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.354+0000 7f24bd16a700 1 -- 172.21.15.7:0/3131264975 >> 172.21.15.7:0/3131264975 conn(0x7f24b80fd4e0 msgr2=0x7f24b80ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:50.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.354+0000 7f24bd16a700 1 -- 172.21.15.7:0/3131264975 shutdown_connections 2024-06-10T00:54:50.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.354+0000 7f24bd16a700 1 -- 172.21.15.7:0/3131264975 wait complete. 2024-06-10T00:54:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.355+0000 7f24bd16a700 1 Processor -- start 2024-06-10T00:54:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.355+0000 7f24bd16a700 1 -- start start 2024-06-10T00:54:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.355+0000 7f24bd16a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24b8103560 0x7f24b8197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.355+0000 7f24bd16a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f24b8108030 con 0x7f24b8103560 2024-06-10T00:54:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.355+0000 7f24b6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24b8103560 0x7f24b8197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.355+0000 7f24b6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24b8103560 0x7f24b8197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45876/0 (socket says 172.21.15.7:45876) 2024-06-10T00:54:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.355+0000 7f24b6d9d700 1 -- 172.21.15.7:0/2150144645 learned_addr learned my addr 172.21.15.7:0/2150144645 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.356+0000 7f24b6d9d700 1 -- 172.21.15.7:0/2150144645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b8197c40 con 0x7f24b8103560 2024-06-10T00:54:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.356+0000 7f24b6d9d700 1 --2- 172.21.15.7:0/2150144645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24b8103560 0x7f24b8197700 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f24a8004fa0 tx=0x7f24a8005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.356+0000 7f249ffff700 1 -- 172.21.15.7:0/2150144645 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24a8027070 con 0x7f24b8103560 2024-06-10T00:54:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.356+0000 7f249ffff700 1 -- 172.21.15.7:0/2150144645 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24a800be40 con 0x7f24b8103560 2024-06-10T00:54:50.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.356+0000 7f24bd16a700 1 -- 172.21.15.7:0/2150144645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24b8197ed0 con 0x7f24b8103560 2024-06-10T00:54:50.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.356+0000 7f249ffff700 1 -- 172.21.15.7:0/2150144645 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24a800f460 con 0x7f24b8103560 2024-06-10T00:54:50.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.356+0000 7f24bd16a700 1 -- 172.21.15.7:0/2150144645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24b819abc0 con 0x7f24b8103560 2024-06-10T00:54:50.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.357+0000 7f249ffff700 1 -- 172.21.15.7:0/2150144645 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f24a802a070 con 0x7f24b8103560 2024-06-10T00:54:50.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.357+0000 7f24bd16a700 1 -- 172.21.15.7:0/2150144645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24b81087b0 con 0x7f24b8103560 2024-06-10T00:54:50.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.357+0000 7f249ffff700 1 --2- 172.21.15.7:0/2150144645 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f24a0039150 0x7f24a003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:50.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.357+0000 7f249ffff700 1 -- 172.21.15.7:0/2150144645 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f24a801e030 con 0x7f24b8103560 2024-06-10T00:54:50.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.357+0000 7f24b659c700 1 --2- 172.21.15.7:0/2150144645 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f24a0039150 0x7f24a003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:50.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.358+0000 7f24b659c700 1 --2- 172.21.15.7:0/2150144645 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f24a0039150 0x7f24a003b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f24ac006fd0 tx=0x7f24ac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:50.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.362+0000 7f249ffff700 1 -- 172.21.15.7:0/2150144645 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f24a8013070 con 0x7f24b8103560 2024-06-10T00:54:50.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.574+0000 7f24bd16a700 1 -- 172.21.15.7:0/2150144645 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f24b80628d0 con 0x7f24a0039150 2024-06-10T00:54:50.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.578+0000 7f249ffff700 1 -- 172.21.15.7:0/2150144645 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f24b80628d0 con 0x7f24a0039150 2024-06-10T00:54:50.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.580+0000 7f24bd16a700 1 -- 172.21.15.7:0/2150144645 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f24a0039150 msgr2=0x7f24a003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:50.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.580+0000 7f24bd16a700 1 --2- 172.21.15.7:0/2150144645 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f24a0039150 0x7f24a003b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f24ac006fd0 tx=0x7f24ac006e40 comp rx=0 tx=0).stop 2024-06-10T00:54:50.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.580+0000 7f24bd16a700 1 -- 172.21.15.7:0/2150144645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24b8103560 msgr2=0x7f24b8197700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:50.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.580+0000 7f24bd16a700 1 --2- 172.21.15.7:0/2150144645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24b8103560 0x7f24b8197700 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f24a8004fa0 tx=0x7f24a8005080 comp rx=0 tx=0).stop 2024-06-10T00:54:50.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.580+0000 7f24bd16a700 1 -- 172.21.15.7:0/2150144645 shutdown_connections 2024-06-10T00:54:50.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.580+0000 7f24bd16a700 1 --2- 172.21.15.7:0/2150144645 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f24a0039150 0x7f24a003b610 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:50.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.580+0000 7f24bd16a700 1 --2- 172.21.15.7:0/2150144645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24b8103560 0x7f24b8197700 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:50.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.580+0000 7f24bd16a700 1 -- 172.21.15.7:0/2150144645 >> 172.21.15.7:0/2150144645 conn(0x7f24b80fd4e0 msgr2=0x7f24b80fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:50.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.580+0000 7f24bd16a700 1 -- 172.21.15.7:0/2150144645 shutdown_connections 2024-06-10T00:54:50.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:50.580+0000 7f24bd16a700 1 -- 172.21.15.7:0/2150144645 wait complete. 2024-06-10T00:54:51.316 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi078 2024-06-10T00:54:51.316 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-06-10T00:54:51.316 DEBUG:teuthology.orchestra.run.smithi078:> dd of=/etc/ceph/ceph.conf 2024-06-10T00:54:51.334 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-06-10T00:54:51.334 DEBUG:teuthology.orchestra.run.smithi078:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:54:51.392 INFO:tasks.cephadm:Adding host smithi078 to orchestrator... 2024-06-10T00:54:51.392 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph orch host add smithi078 2024-06-10T00:54:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:51.880 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:51 smithi007 ceph-mon[32398]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:54:51.880 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:51 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:54:51.880 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:51 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:54:51.880 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:51 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:54:51.880 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:51 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:54:51.880 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:51 smithi007 ceph-mon[32398]: Updating smithi007:/etc/ceph/ceph.conf 2024-06-10T00:54:51.880 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:51 smithi007 ceph-mon[32398]: Updating smithi007:/var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:54:51.880 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:51 smithi007 ceph-mon[32398]: Updating smithi007:/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:54:52.753 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:52 smithi007 ceph-mon[32398]: Updating smithi007:/var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-10T00:54:52.753 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:52 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:54:52.753 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:52 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:54:52.753 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:52 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:54:52.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:52.981+0000 7f4533118700 1 -- 172.21.15.7:0/3158214982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f452c102100 msgr2=0x7f452c1024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:52.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:52.981+0000 7f4533118700 1 --2- 172.21.15.7:0/3158214982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f452c102100 0x7f452c1024e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4520009910 tx=0x7f4520009c20 comp rx=0 tx=0).stop 2024-06-10T00:54:52.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:52.981+0000 7f4533118700 1 -- 172.21.15.7:0/3158214982 shutdown_connections 2024-06-10T00:54:52.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:52.981+0000 7f4533118700 1 --2- 172.21.15.7:0/3158214982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f452c102100 0x7f452c1024e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:52.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:52.981+0000 7f4533118700 1 -- 172.21.15.7:0/3158214982 >> 172.21.15.7:0/3158214982 conn(0x7f452c0fd9d0 msgr2=0x7f452c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:52.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:52.981+0000 7f4533118700 1 -- 172.21.15.7:0/3158214982 shutdown_connections 2024-06-10T00:54:52.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:52.981+0000 7f4533118700 1 -- 172.21.15.7:0/3158214982 wait complete. 2024-06-10T00:54:52.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:52.982+0000 7f4533118700 1 Processor -- start 2024-06-10T00:54:52.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:52.982+0000 7f4533118700 1 -- start start 2024-06-10T00:54:52.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:52.982+0000 7f4533118700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f452c102100 0x7f452c19d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:52.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:52.982+0000 7f4533118700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f452c10c4d0 con 0x7f452c102100 2024-06-10T00:54:52.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:52.982+0000 7f4530eb4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f452c102100 0x7f452c19d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:52.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:52.982+0000 7f4530eb4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f452c102100 0x7f452c19d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45132/0 (socket says 172.21.15.7:45132) 2024-06-10T00:54:52.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:52.982+0000 7f4530eb4700 1 -- 172.21.15.7:0/1650287707 learned_addr learned my addr 172.21.15.7:0/1650287707 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:52.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:52.982+0000 7f4530eb4700 1 -- 172.21.15.7:0/1650287707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452c19dc20 con 0x7f452c102100 2024-06-10T00:54:52.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:52.982+0000 7f4530eb4700 1 --2- 172.21.15.7:0/1650287707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f452c102100 0x7f452c19d6e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4520005850 tx=0x7f4520005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:52.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:52.983+0000 7f4529ffb700 1 -- 172.21.15.7:0/1650287707 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4520026070 con 0x7f452c102100 2024-06-10T00:54:52.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:52.983+0000 7f4529ffb700 1 -- 172.21.15.7:0/1650287707 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f452000bbd0 con 0x7f452c102100 2024-06-10T00:54:52.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:52.983+0000 7f4533118700 1 -- 172.21.15.7:0/1650287707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f452c19df10 con 0x7f452c102100 2024-06-10T00:54:52.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:52.983+0000 7f4529ffb700 1 -- 172.21.15.7:0/1650287707 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4520003970 con 0x7f452c102100 2024-06-10T00:54:52.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:52.983+0000 7f4533118700 1 -- 172.21.15.7:0/1650287707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f452c1978a0 con 0x7f452c102100 2024-06-10T00:54:52.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:52.984+0000 7f4529ffb700 1 -- 172.21.15.7:0/1650287707 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f4520029070 con 0x7f452c102100 2024-06-10T00:54:52.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:52.984+0000 7f4533118700 1 -- 172.21.15.7:0/1650287707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f452c1035e0 con 0x7f452c102100 2024-06-10T00:54:52.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:52.984+0000 7f4529ffb700 1 --2- 172.21.15.7:0/1650287707 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f451c039150 0x7f451c03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:52.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:52.984+0000 7f4529ffb700 1 -- 172.21.15.7:0/1650287707 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f452001e070 con 0x7f452c102100 2024-06-10T00:54:52.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:52.984+0000 7f452bfff700 1 --2- 172.21.15.7:0/1650287707 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f451c039150 0x7f451c03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:52.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:52.984+0000 7f452bfff700 1 --2- 172.21.15.7:0/1650287707 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f451c039150 0x7f451c03b610 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4518006fd0 tx=0x7f4518006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:52.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:52.989+0000 7f4529ffb700 1 -- 172.21.15.7:0/1650287707 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4520013070 con 0x7f452c102100 2024-06-10T00:54:53.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:53.202+0000 7f4533118700 1 -- 172.21.15.7:0/1650287707 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi078", "target": ["mon-mgr", ""]}) -- 0x7f452c198dd0 con 0x7f451c039150 2024-06-10T00:54:54.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:53 smithi007 ceph-mon[32398]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi078", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:54:55.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:54 smithi007 ceph-mon[32398]: Deploying cephadm binary to smithi078 2024-06-10T00:54:57.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:56 smithi007 ceph-mon[32398]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:54:57.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:57.505+0000 7f4529ffb700 1 -- 172.21.15.7:0/1650287707 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f452c198dd0 con 0x7f451c039150 2024-06-10T00:54:57.506 INFO:teuthology.orchestra.run.smithi007.stdout:Added host 'smithi078' with addr '172.21.15.78' 2024-06-10T00:54:57.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:57.507+0000 7f4533118700 1 -- 172.21.15.7:0/1650287707 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f451c039150 msgr2=0x7f451c03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:57.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:57.507+0000 7f4533118700 1 --2- 172.21.15.7:0/1650287707 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f451c039150 0x7f451c03b610 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4518006fd0 tx=0x7f4518006e40 comp rx=0 tx=0).stop 2024-06-10T00:54:57.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:57.507+0000 7f4533118700 1 -- 172.21.15.7:0/1650287707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f452c102100 msgr2=0x7f452c19d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:57.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:57.507+0000 7f4533118700 1 --2- 172.21.15.7:0/1650287707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f452c102100 0x7f452c19d6e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4520005850 tx=0x7f4520005030 comp rx=0 tx=0).stop 2024-06-10T00:54:57.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:57.508+0000 7f4533118700 1 -- 172.21.15.7:0/1650287707 shutdown_connections 2024-06-10T00:54:57.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:57.508+0000 7f4533118700 1 --2- 172.21.15.7:0/1650287707 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f451c039150 0x7f451c03b610 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:57.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:57.508+0000 7f4533118700 1 --2- 172.21.15.7:0/1650287707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f452c102100 0x7f452c19d6e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:57.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:57.508+0000 7f4533118700 1 -- 172.21.15.7:0/1650287707 >> 172.21.15.7:0/1650287707 conn(0x7f452c0fd9d0 msgr2=0x7f452c104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:57.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:57.508+0000 7f4533118700 1 -- 172.21.15.7:0/1650287707 shutdown_connections 2024-06-10T00:54:57.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:57.508+0000 7f4533118700 1 -- 172.21.15.7:0/1650287707 wait complete. 2024-06-10T00:54:58.062 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-06-10T00:54:58.306 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:58.710 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:58 smithi007 ceph-mon[32398]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:54:58.710 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:58 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:54:58.710 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:58 smithi007 ceph-mon[32398]: Added host smithi078 2024-06-10T00:54:58.710 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:58 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:54:58.710 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:58 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:54:59.596 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:54:59 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:54:59.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:59.855+0000 7f5c82ec1700 1 -- 172.21.15.7:0/1475854322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c7c101040 msgr2=0x7f5c7c101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:59.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:59.855+0000 7f5c82ec1700 1 --2- 172.21.15.7:0/1475854322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c7c101040 0x7f5c7c101420 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c009910 tx=0x7f5c6c009c20 comp rx=0 tx=0).stop 2024-06-10T00:54:59.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:59.855+0000 7f5c82ec1700 1 -- 172.21.15.7:0/1475854322 shutdown_connections 2024-06-10T00:54:59.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:59.855+0000 7f5c82ec1700 1 --2- 172.21.15.7:0/1475854322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c7c101040 0x7f5c7c101420 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:59.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:59.855+0000 7f5c82ec1700 1 -- 172.21.15.7:0/1475854322 >> 172.21.15.7:0/1475854322 conn(0x7f5c7c0fc8f0 msgr2=0x7f5c7c0fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:59.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:59.855+0000 7f5c82ec1700 1 -- 172.21.15.7:0/1475854322 shutdown_connections 2024-06-10T00:54:59.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:59.855+0000 7f5c82ec1700 1 -- 172.21.15.7:0/1475854322 wait complete. 2024-06-10T00:54:59.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:59.856+0000 7f5c82ec1700 1 Processor -- start 2024-06-10T00:54:59.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:59.856+0000 7f5c82ec1700 1 -- start start 2024-06-10T00:54:59.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:59.856+0000 7f5c82ec1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c7c101040 0x7f5c7c197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:59.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:59.856+0000 7f5c82ec1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c10b410 con 0x7f5c7c101040 2024-06-10T00:54:59.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:59.856+0000 7f5c80c5d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c7c101040 0x7f5c7c197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:59.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:59.856+0000 7f5c80c5d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c7c101040 0x7f5c7c197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45146/0 (socket says 172.21.15.7:45146) 2024-06-10T00:54:59.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:59.856+0000 7f5c80c5d700 1 -- 172.21.15.7:0/2000787095 learned_addr learned my addr 172.21.15.7:0/2000787095 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:54:59.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:59.856+0000 7f5c80c5d700 1 -- 172.21.15.7:0/2000787095 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c7c197f80 con 0x7f5c7c101040 2024-06-10T00:54:59.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:59.857+0000 7f5c80c5d700 1 --2- 172.21.15.7:0/2000787095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c7c101040 0x7f5c7c197a40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c005850 tx=0x7f5c6c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:59.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:59.857+0000 7f5c79ffb700 1 -- 172.21.15.7:0/2000787095 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c6c026070 con 0x7f5c7c101040 2024-06-10T00:54:59.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:59.857+0000 7f5c82ec1700 1 -- 172.21.15.7:0/2000787095 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c7c198270 con 0x7f5c7c101040 2024-06-10T00:54:59.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:59.857+0000 7f5c79ffb700 1 -- 172.21.15.7:0/2000787095 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c6c00bbd0 con 0x7f5c7c101040 2024-06-10T00:54:59.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:59.857+0000 7f5c79ffb700 1 -- 172.21.15.7:0/2000787095 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c6c003970 con 0x7f5c7c101040 2024-06-10T00:54:59.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:59.857+0000 7f5c82ec1700 1 -- 172.21.15.7:0/2000787095 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c7c19c340 con 0x7f5c7c101040 2024-06-10T00:54:59.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:59.858+0000 7f5c79ffb700 1 -- 172.21.15.7:0/2000787095 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f5c6c029070 con 0x7f5c7c101040 2024-06-10T00:54:59.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:59.858+0000 7f5c82ec1700 1 -- 172.21.15.7:0/2000787095 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c7c102520 con 0x7f5c7c101040 2024-06-10T00:54:59.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:59.858+0000 7f5c79ffb700 1 --2- 172.21.15.7:0/2000787095 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5c640390f0 0x7f5c6403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:59.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:59.858+0000 7f5c79ffb700 1 -- 172.21.15.7:0/2000787095 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5c6c01e070 con 0x7f5c7c101040 2024-06-10T00:54:59.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:59.858+0000 7f5c7bfff700 1 --2- 172.21.15.7:0/2000787095 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5c640390f0 0x7f5c6403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:59.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:59.859+0000 7f5c7bfff700 1 --2- 172.21.15.7:0/2000787095 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5c640390f0 0x7f5c6403b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5c70006fd0 tx=0x7f5c70006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:59.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:54:59.863+0000 7f5c79ffb700 1 -- 172.21.15.7:0/2000787095 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5c6c013070 con 0x7f5c7c101040 2024-06-10T00:55:00.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:00.074+0000 7f5c82ec1700 1 -- 172.21.15.7:0/2000787095 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5c7c063af0 con 0x7f5c640390f0 2024-06-10T00:55:00.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:00.075+0000 7f5c79ffb700 1 -- 172.21.15.7:0/2000787095 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+158 (secure 0 0 0) 0x7f5c7c063af0 con 0x7f5c640390f0 2024-06-10T00:55:00.075 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:55:00.075 INFO:teuthology.orchestra.run.smithi007.stdout:[{"addr": "172.21.15.7", "hostname": "smithi007", "labels": [], "status": ""}, {"addr": "172.21.15.78", "hostname": "smithi078", "labels": [], "status": ""}] 2024-06-10T00:55:00.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:00.077+0000 7f5c82ec1700 1 -- 172.21.15.7:0/2000787095 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5c640390f0 msgr2=0x7f5c6403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:00.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:00.077+0000 7f5c82ec1700 1 --2- 172.21.15.7:0/2000787095 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5c640390f0 0x7f5c6403b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5c70006fd0 tx=0x7f5c70006e40 comp rx=0 tx=0).stop 2024-06-10T00:55:00.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:00.077+0000 7f5c82ec1700 1 -- 172.21.15.7:0/2000787095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c7c101040 msgr2=0x7f5c7c197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:00.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:00.077+0000 7f5c82ec1700 1 --2- 172.21.15.7:0/2000787095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c7c101040 0x7f5c7c197a40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c005850 tx=0x7f5c6c005030 comp rx=0 tx=0).stop 2024-06-10T00:55:00.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:00.077+0000 7f5c82ec1700 1 -- 172.21.15.7:0/2000787095 shutdown_connections 2024-06-10T00:55:00.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:00.077+0000 7f5c82ec1700 1 --2- 172.21.15.7:0/2000787095 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5c640390f0 0x7f5c6403b5b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:00.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:00.077+0000 7f5c82ec1700 1 --2- 172.21.15.7:0/2000787095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c7c101040 0x7f5c7c197a40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:00.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:00.077+0000 7f5c82ec1700 1 -- 172.21.15.7:0/2000787095 >> 172.21.15.7:0/2000787095 conn(0x7f5c7c0fc8f0 msgr2=0x7f5c7c06a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:00.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:00.077+0000 7f5c82ec1700 1 -- 172.21.15.7:0/2000787095 shutdown_connections 2024-06-10T00:55:00.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:00.077+0000 7f5c82ec1700 1 -- 172.21.15.7:0/2000787095 wait complete. 2024-06-10T00:55:00.619 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi175 2024-06-10T00:55:00.620 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-06-10T00:55:00.620 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/etc/ceph/ceph.conf 2024-06-10T00:55:00.638 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-06-10T00:55:00.639 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:55:00.695 INFO:tasks.cephadm:Adding host smithi175 to orchestrator... 2024-06-10T00:55:00.696 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph orch host add smithi175 2024-06-10T00:55:00.721 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:00 smithi007 ceph-mon[32398]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:00.721 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:00 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:00.721 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:00 smithi007 ceph-mon[32398]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:55:00.941 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:55:02.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:02.375+0000 7fde2b0cb700 1 -- 172.21.15.7:0/407266511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde24102130 msgr2=0x7fde24102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:02.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:02.375+0000 7fde2b0cb700 1 --2- 172.21.15.7:0/407266511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde24102130 0x7fde24102510 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fde14009910 tx=0x7fde14009c20 comp rx=0 tx=0).stop 2024-06-10T00:55:02.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:02.375+0000 7fde2b0cb700 1 -- 172.21.15.7:0/407266511 shutdown_connections 2024-06-10T00:55:02.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:02.375+0000 7fde2b0cb700 1 --2- 172.21.15.7:0/407266511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde24102130 0x7fde24102510 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:02.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:02.375+0000 7fde2b0cb700 1 -- 172.21.15.7:0/407266511 >> 172.21.15.7:0/407266511 conn(0x7fde240fd9c0 msgr2=0x7fde240ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:02.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:02.375+0000 7fde2b0cb700 1 -- 172.21.15.7:0/407266511 shutdown_connections 2024-06-10T00:55:02.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:02.375+0000 7fde2b0cb700 1 -- 172.21.15.7:0/407266511 wait complete. 2024-06-10T00:55:02.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:02.376+0000 7fde2b0cb700 1 Processor -- start 2024-06-10T00:55:02.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:02.376+0000 7fde2b0cb700 1 -- start start 2024-06-10T00:55:02.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:02.376+0000 7fde2b0cb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde24102130 0x7fde2419d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:02.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:02.376+0000 7fde2b0cb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fde2410c500 con 0x7fde24102130 2024-06-10T00:55:02.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:02.376+0000 7fde28e67700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde24102130 0x7fde2419d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:02.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:02.376+0000 7fde28e67700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde24102130 0x7fde2419d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37048/0 (socket says 172.21.15.7:37048) 2024-06-10T00:55:02.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:02.376+0000 7fde28e67700 1 -- 172.21.15.7:0/3618354340 learned_addr learned my addr 172.21.15.7:0/3618354340 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:55:02.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:02.376+0000 7fde28e67700 1 -- 172.21.15.7:0/3618354340 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde2419dbf0 con 0x7fde24102130 2024-06-10T00:55:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:02.376+0000 7fde28e67700 1 --2- 172.21.15.7:0/3618354340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde24102130 0x7fde2419d6b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fde14005850 tx=0x7fde14005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:02.377+0000 7fde21ffb700 1 -- 172.21.15.7:0/3618354340 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde14026070 con 0x7fde24102130 2024-06-10T00:55:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:02.377+0000 7fde21ffb700 1 -- 172.21.15.7:0/3618354340 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde1400bbd0 con 0x7fde24102130 2024-06-10T00:55:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:02.377+0000 7fde2b0cb700 1 -- 172.21.15.7:0/3618354340 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde2419dee0 con 0x7fde24102130 2024-06-10T00:55:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:02.377+0000 7fde21ffb700 1 -- 172.21.15.7:0/3618354340 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde14003970 con 0x7fde24102130 2024-06-10T00:55:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:02.377+0000 7fde2b0cb700 1 -- 172.21.15.7:0/3618354340 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde24197830 con 0x7fde24102130 2024-06-10T00:55:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:02.378+0000 7fde21ffb700 1 -- 172.21.15.7:0/3618354340 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fde14029070 con 0x7fde24102130 2024-06-10T00:55:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:02.378+0000 7fde2b0cb700 1 -- 172.21.15.7:0/3618354340 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde24103610 con 0x7fde24102130 2024-06-10T00:55:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:02.378+0000 7fde21ffb700 1 --2- 172.21.15.7:0/3618354340 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fde0c039150 0x7fde0c03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:02.378+0000 7fde21ffb700 1 -- 172.21.15.7:0/3618354340 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fde1401e070 con 0x7fde24102130 2024-06-10T00:55:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:02.378+0000 7fde23fff700 1 --2- 172.21.15.7:0/3618354340 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fde0c039150 0x7fde0c03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:02.378+0000 7fde23fff700 1 --2- 172.21.15.7:0/3618354340 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fde0c039150 0x7fde0c03b610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fde18006fd0 tx=0x7fde18006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:02.383+0000 7fde21ffb700 1 -- 172.21.15.7:0/3618354340 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fde14013070 con 0x7fde24102130 2024-06-10T00:55:02.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:02.597+0000 7fde2b0cb700 1 -- 172.21.15.7:0/3618354340 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi175", "target": ["mon-mgr", ""]}) -- 0x7fde24198d60 con 0x7fde0c039150 2024-06-10T00:55:03.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:02 smithi007 ceph-mon[32398]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:03.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:02 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:03.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:02 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:03.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:02 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:03.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:02 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:03.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:02 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi078", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:55:03.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:02 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:03.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:02 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:55:04.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:03 smithi007 ceph-mon[32398]: Updating smithi078:/etc/ceph/ceph.conf 2024-06-10T00:55:04.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:03 smithi007 ceph-mon[32398]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi175", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:55:04.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:03 smithi007 ceph-mon[32398]: Updating smithi078:/var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:55:04.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:03 smithi007 ceph-mon[32398]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:55:04.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:03 smithi007 ceph-mon[32398]: Deploying cephadm binary to smithi175 2024-06-10T00:55:04.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:03 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:04.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:03 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:04.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:03 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:05.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:04 smithi007 ceph-mon[32398]: Updating smithi078:/var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-10T00:55:05.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:04 smithi007 ceph-mon[32398]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:06.772 INFO:teuthology.orchestra.run.smithi007.stdout:Added host 'smithi175' with addr '172.21.15.175' 2024-06-10T00:55:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:06.771+0000 7fde21ffb700 1 -- 172.21.15.7:0/3618354340 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fde24198d60 con 0x7fde0c039150 2024-06-10T00:55:06.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:06.773+0000 7fde2b0cb700 1 -- 172.21.15.7:0/3618354340 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fde0c039150 msgr2=0x7fde0c03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:06.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:06.773+0000 7fde2b0cb700 1 --2- 172.21.15.7:0/3618354340 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fde0c039150 0x7fde0c03b610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fde18006fd0 tx=0x7fde18006e40 comp rx=0 tx=0).stop 2024-06-10T00:55:06.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:06.773+0000 7fde2b0cb700 1 -- 172.21.15.7:0/3618354340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde24102130 msgr2=0x7fde2419d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:06.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:06.773+0000 7fde2b0cb700 1 --2- 172.21.15.7:0/3618354340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde24102130 0x7fde2419d6b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fde14005850 tx=0x7fde14005030 comp rx=0 tx=0).stop 2024-06-10T00:55:06.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:06.773+0000 7fde2b0cb700 1 -- 172.21.15.7:0/3618354340 shutdown_connections 2024-06-10T00:55:06.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:06.773+0000 7fde2b0cb700 1 --2- 172.21.15.7:0/3618354340 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fde0c039150 0x7fde0c03b610 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:06.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:06.773+0000 7fde2b0cb700 1 --2- 172.21.15.7:0/3618354340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde24102130 0x7fde2419d6b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:06.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:06.773+0000 7fde2b0cb700 1 -- 172.21.15.7:0/3618354340 >> 172.21.15.7:0/3618354340 conn(0x7fde240fd9c0 msgr2=0x7fde24104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:06.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:06.774+0000 7fde2b0cb700 1 -- 172.21.15.7:0/3618354340 shutdown_connections 2024-06-10T00:55:06.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:06.774+0000 7fde2b0cb700 1 -- 172.21.15.7:0/3618354340 wait complete. 2024-06-10T00:55:07.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:06 smithi007 ceph-mon[32398]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:07.360 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-06-10T00:55:07.649 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:55:08.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:07 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:08.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:07 smithi007 ceph-mon[32398]: Added host smithi175 2024-06-10T00:55:08.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:07 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:55:08.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:07 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:09.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.042+0000 7fe17d730700 1 -- 172.21.15.7:0/2656615014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1781030a0 msgr2=0x7fe178103480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:09.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.042+0000 7fe17d730700 1 --2- 172.21.15.7:0/2656615014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1781030a0 0x7fe178103480 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe168009910 tx=0x7fe168009c20 comp rx=0 tx=0).stop 2024-06-10T00:55:09.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.042+0000 7fe17d730700 1 -- 172.21.15.7:0/2656615014 shutdown_connections 2024-06-10T00:55:09.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.042+0000 7fe17d730700 1 --2- 172.21.15.7:0/2656615014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1781030a0 0x7fe178103480 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:09.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.042+0000 7fe17d730700 1 -- 172.21.15.7:0/2656615014 >> 172.21.15.7:0/2656615014 conn(0x7fe1780fc890 msgr2=0x7fe1780fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:09.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.042+0000 7fe17d730700 1 -- 172.21.15.7:0/2656615014 shutdown_connections 2024-06-10T00:55:09.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.042+0000 7fe17d730700 1 -- 172.21.15.7:0/2656615014 wait complete. 2024-06-10T00:55:09.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.042+0000 7fe17d730700 1 Processor -- start 2024-06-10T00:55:09.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.043+0000 7fe17d730700 1 -- start start 2024-06-10T00:55:09.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.043+0000 7fe17d730700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1781030a0 0x7fe17819d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:09.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.043+0000 7fe17d730700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe17810c520 con 0x7fe1781030a0 2024-06-10T00:55:09.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.043+0000 7fe176ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1781030a0 0x7fe17819d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:09.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.043+0000 7fe176ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1781030a0 0x7fe17819d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37076/0 (socket says 172.21.15.7:37076) 2024-06-10T00:55:09.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.043+0000 7fe176ffd700 1 -- 172.21.15.7:0/719013490 learned_addr learned my addr 172.21.15.7:0/719013490 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:55:09.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.043+0000 7fe176ffd700 1 -- 172.21.15.7:0/719013490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe17819dc00 con 0x7fe1781030a0 2024-06-10T00:55:09.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.043+0000 7fe176ffd700 1 --2- 172.21.15.7:0/719013490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1781030a0 0x7fe17819d6c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe168006010 tx=0x7fe168005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:09.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.044+0000 7fe15ffff700 1 -- 172.21.15.7:0/719013490 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe168026070 con 0x7fe1781030a0 2024-06-10T00:55:09.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.044+0000 7fe15ffff700 1 -- 172.21.15.7:0/719013490 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe16800bc30 con 0x7fe1781030a0 2024-06-10T00:55:09.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.044+0000 7fe17d730700 1 -- 172.21.15.7:0/719013490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe17819de90 con 0x7fe1781030a0 2024-06-10T00:55:09.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.044+0000 7fe15ffff700 1 -- 172.21.15.7:0/719013490 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe168003ab0 con 0x7fe1781030a0 2024-06-10T00:55:09.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.044+0000 7fe17d730700 1 -- 172.21.15.7:0/719013490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe178197880 con 0x7fe1781030a0 2024-06-10T00:55:09.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.044+0000 7fe15ffff700 1 -- 172.21.15.7:0/719013490 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fe168029070 con 0x7fe1781030a0 2024-06-10T00:55:09.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.045+0000 7fe17d730700 1 -- 172.21.15.7:0/719013490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe17806a9c0 con 0x7fe1781030a0 2024-06-10T00:55:09.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.045+0000 7fe15ffff700 1 --2- 172.21.15.7:0/719013490 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe160039150 0x7fe16003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:09.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.045+0000 7fe15ffff700 1 -- 172.21.15.7:0/719013490 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe16801e070 con 0x7fe1781030a0 2024-06-10T00:55:09.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.045+0000 7fe1767fc700 1 --2- 172.21.15.7:0/719013490 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe160039150 0x7fe16003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:09.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.045+0000 7fe1767fc700 1 --2- 172.21.15.7:0/719013490 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe160039150 0x7fe16003b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe16c006fd0 tx=0x7fe16c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:09.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.050+0000 7fe15ffff700 1 -- 172.21.15.7:0/719013490 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe168013070 con 0x7fe1781030a0 2024-06-10T00:55:09.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.260+0000 7fe17d730700 1 -- 172.21.15.7:0/719013490 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe178198db0 con 0x7fe160039150 2024-06-10T00:55:09.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:08 smithi007 ceph-mon[32398]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:09.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:08 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:09.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.262+0000 7fe15ffff700 1 -- 172.21.15.7:0/719013490 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+238 (secure 0 0 0) 0x7fe178198db0 con 0x7fe160039150 2024-06-10T00:55:09.262 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T00:55:09.262 INFO:teuthology.orchestra.run.smithi007.stdout:[{"addr": "172.21.15.7", "hostname": "smithi007", "labels": [], "status": ""}, {"addr": "172.21.15.78", "hostname": "smithi078", "labels": [], "status": ""}, {"addr": "172.21.15.175", "hostname": "smithi175", "labels": [], "status": ""}] 2024-06-10T00:55:09.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.264+0000 7fe17d730700 1 -- 172.21.15.7:0/719013490 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe160039150 msgr2=0x7fe16003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:09.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.264+0000 7fe17d730700 1 --2- 172.21.15.7:0/719013490 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe160039150 0x7fe16003b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe16c006fd0 tx=0x7fe16c006e40 comp rx=0 tx=0).stop 2024-06-10T00:55:09.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.264+0000 7fe17d730700 1 -- 172.21.15.7:0/719013490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1781030a0 msgr2=0x7fe17819d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:09.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.264+0000 7fe17d730700 1 --2- 172.21.15.7:0/719013490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1781030a0 0x7fe17819d6c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe168006010 tx=0x7fe168005050 comp rx=0 tx=0).stop 2024-06-10T00:55:09.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.264+0000 7fe17d730700 1 -- 172.21.15.7:0/719013490 shutdown_connections 2024-06-10T00:55:09.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.264+0000 7fe17d730700 1 --2- 172.21.15.7:0/719013490 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe160039150 0x7fe16003b610 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:09.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.264+0000 7fe17d730700 1 --2- 172.21.15.7:0/719013490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1781030a0 0x7fe17819d6c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:09.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.264+0000 7fe17d730700 1 -- 172.21.15.7:0/719013490 >> 172.21.15.7:0/719013490 conn(0x7fe1780fc890 msgr2=0x7fe178103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:09.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.264+0000 7fe17d730700 1 -- 172.21.15.7:0/719013490 shutdown_connections 2024-06-10T00:55:09.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:09.264+0000 7fe17d730700 1 -- 172.21.15.7:0/719013490 wait complete. 2024-06-10T00:55:09.837 INFO:tasks.cephadm:Setting crush tunables to default 2024-06-10T00:55:09.837 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd crush tunables default 2024-06-10T00:55:10.079 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:55:10.202 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:09 smithi007 ceph-mon[32398]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:55:10.202 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:09 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:11.180 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:10 smithi007 ceph-mon[32398]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:11.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:11.650+0000 7f18213c8700 1 -- 172.21.15.7:0/2520994054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f181c106dd0 msgr2=0x7f181c0698b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:11.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:11.650+0000 7f18213c8700 1 --2- 172.21.15.7:0/2520994054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f181c106dd0 0x7f181c0698b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f180c009910 tx=0x7f180c009c20 comp rx=0 tx=0).stop 2024-06-10T00:55:11.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:11.650+0000 7f18213c8700 1 -- 172.21.15.7:0/2520994054 shutdown_connections 2024-06-10T00:55:11.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:11.650+0000 7f18213c8700 1 --2- 172.21.15.7:0/2520994054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f181c106dd0 0x7f181c0698b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:11.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:11.650+0000 7f18213c8700 1 -- 172.21.15.7:0/2520994054 >> 172.21.15.7:0/2520994054 conn(0x7f181c0768f0 msgr2=0x7f181c076d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:11.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:11.650+0000 7f18213c8700 1 -- 172.21.15.7:0/2520994054 shutdown_connections 2024-06-10T00:55:11.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:11.650+0000 7f18213c8700 1 -- 172.21.15.7:0/2520994054 wait complete. 2024-06-10T00:55:11.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:11.650+0000 7f18213c8700 1 Processor -- start 2024-06-10T00:55:11.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:11.650+0000 7f18213c8700 1 -- start start 2024-06-10T00:55:11.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:11.651+0000 7f18213c8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f181c106dd0 0x7f181c19e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:11.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:11.651+0000 7f18213c8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f181c112b50 con 0x7f181c106dd0 2024-06-10T00:55:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:11.651+0000 7f181affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f181c106dd0 0x7f181c19e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:11.651+0000 7f181affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f181c106dd0 0x7f181c19e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37084/0 (socket says 172.21.15.7:37084) 2024-06-10T00:55:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:11.651+0000 7f181affd700 1 -- 172.21.15.7:0/1961130065 learned_addr learned my addr 172.21.15.7:0/1961130065 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:55:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:11.651+0000 7f181affd700 1 -- 172.21.15.7:0/1961130065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181c19e550 con 0x7f181c106dd0 2024-06-10T00:55:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:11.651+0000 7f181affd700 1 --2- 172.21.15.7:0/1961130065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f181c106dd0 0x7f181c19e010 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f180c005850 tx=0x7f180c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:11.651+0000 7f1803fff700 1 -- 172.21.15.7:0/1961130065 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f180c026070 con 0x7f181c106dd0 2024-06-10T00:55:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:11.651+0000 7f1803fff700 1 -- 172.21.15.7:0/1961130065 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f180c00bc30 con 0x7f181c106dd0 2024-06-10T00:55:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:11.651+0000 7f18213c8700 1 -- 172.21.15.7:0/1961130065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f181c19e7e0 con 0x7f181c106dd0 2024-06-10T00:55:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:11.651+0000 7f1803fff700 1 -- 172.21.15.7:0/1961130065 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f180c003a20 con 0x7f181c106dd0 2024-06-10T00:55:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:11.651+0000 7f18213c8700 1 -- 172.21.15.7:0/1961130065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f181c19ec40 con 0x7f181c106dd0 2024-06-10T00:55:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:11.652+0000 7f1803fff700 1 -- 172.21.15.7:0/1961130065 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f180c029070 con 0x7f181c106dd0 2024-06-10T00:55:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:11.652+0000 7f18213c8700 1 -- 172.21.15.7:0/1961130065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f181c1013c0 con 0x7f181c106dd0 2024-06-10T00:55:11.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:11.653+0000 7f1803fff700 1 --2- 172.21.15.7:0/1961130065 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f18040390f0 0x7f180403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:11.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:11.653+0000 7f1803fff700 1 -- 172.21.15.7:0/1961130065 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f180c01e070 con 0x7f181c106dd0 2024-06-10T00:55:11.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:11.653+0000 7f181a7fc700 1 --2- 172.21.15.7:0/1961130065 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f18040390f0 0x7f180403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:11.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:11.653+0000 7f181a7fc700 1 --2- 172.21.15.7:0/1961130065 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f18040390f0 0x7f180403b5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1810006fd0 tx=0x7f1810006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:11.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:11.658+0000 7f1803fff700 1 -- 172.21.15.7:0/1961130065 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f180c013070 con 0x7f181c106dd0 2024-06-10T00:55:11.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:11.869+0000 7f18213c8700 1 -- 172.21.15.7:0/1961130065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f181c19f370 con 0x7f181c106dd0 2024-06-10T00:55:12.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:12.843+0000 7f1803fff700 1 -- 172.21.15.7:0/1961130065 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f180c030090 con 0x7f181c106dd0 2024-06-10T00:55:12.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:12.846+0000 7f18213c8700 1 -- 172.21.15.7:0/1961130065 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f18040390f0 msgr2=0x7f180403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:12.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:12.846+0000 7f18213c8700 1 --2- 172.21.15.7:0/1961130065 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f18040390f0 0x7f180403b5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1810006fd0 tx=0x7f1810006e40 comp rx=0 tx=0).stop 2024-06-10T00:55:12.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:12.846+0000 7f18213c8700 1 -- 172.21.15.7:0/1961130065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f181c106dd0 msgr2=0x7f181c19e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:12.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:12.846+0000 7f18213c8700 1 --2- 172.21.15.7:0/1961130065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f181c106dd0 0x7f181c19e010 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f180c005850 tx=0x7f180c005030 comp rx=0 tx=0).stop 2024-06-10T00:55:12.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:12.847+0000 7f18213c8700 1 -- 172.21.15.7:0/1961130065 shutdown_connections 2024-06-10T00:55:12.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:12.847+0000 7f18213c8700 1 --2- 172.21.15.7:0/1961130065 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f18040390f0 0x7f180403b5b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:12.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:12.847+0000 7f18213c8700 1 --2- 172.21.15.7:0/1961130065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f181c106dd0 0x7f181c19e010 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:12.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:12.847+0000 7f18213c8700 1 -- 172.21.15.7:0/1961130065 >> 172.21.15.7:0/1961130065 conn(0x7f181c0768f0 msgr2=0x7f181c0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:12.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:12.847+0000 7f18213c8700 1 -- 172.21.15.7:0/1961130065 shutdown_connections 2024-06-10T00:55:12.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:12.847+0000 7f18213c8700 1 -- 172.21.15.7:0/1961130065 wait complete. 2024-06-10T00:55:12.848 INFO:teuthology.orchestra.run.smithi007.stderr:adjusted tunables profile to default 2024-06-10T00:55:13.171 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:12 smithi007 ceph-mon[32398]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:13.171 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:12 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1961130065' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-10T00:55:13.172 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:12 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:13.172 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:12 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:13.172 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:12 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:13.172 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:12 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:13.172 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:12 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi175", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:55:13.172 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:12 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:13.172 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:12 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:55:13.468 INFO:tasks.cephadm:Adding mon.a on smithi007 2024-06-10T00:55:13.468 INFO:tasks.cephadm:Adding mon.b on smithi078 2024-06-10T00:55:13.468 INFO:tasks.cephadm:Adding mon.c on smithi175 2024-06-10T00:55:13.468 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph orch apply mon '3;smithi007:172.21.15.7=a;smithi078:172.21.15.78=b;smithi175:172.21.15.175=c' 2024-06-10T00:55:13.692 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:55:14.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:13 smithi007 ceph-mon[32398]: Updating smithi175:/etc/ceph/ceph.conf 2024-06-10T00:55:14.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:13 smithi007 ceph-mon[32398]: Updating smithi175:/var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:55:14.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:13 smithi007 ceph-mon[32398]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:55:14.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:13 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1961130065' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-10T00:55:14.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:13 smithi007 ceph-mon[32398]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T00:55:14.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:13 smithi007 ceph-mon[32398]: Updating smithi175:/var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-10T00:55:14.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:13 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:14.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:13 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:14.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:13 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:15.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.082+0000 7ff59276c700 1 -- 172.21.15.175:0/888782907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff58c108020 msgr2=0x7ff58c108400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:15.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.082+0000 7ff59276c700 1 --2- 172.21.15.175:0/888782907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff58c108020 0x7ff58c108400 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff57c009910 tx=0x7ff57c009c20 comp rx=0 tx=0).stop 2024-06-10T00:55:15.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.083+0000 7ff59276c700 1 -- 172.21.15.175:0/888782907 shutdown_connections 2024-06-10T00:55:15.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.083+0000 7ff59276c700 1 --2- 172.21.15.175:0/888782907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff58c108020 0x7ff58c108400 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:15.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.083+0000 7ff59276c700 1 -- 172.21.15.175:0/888782907 >> 172.21.15.175:0/888782907 conn(0x7ff58c077510 msgr2=0x7ff58c077920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:15.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.083+0000 7ff59276c700 1 -- 172.21.15.175:0/888782907 shutdown_connections 2024-06-10T00:55:15.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.083+0000 7ff59276c700 1 -- 172.21.15.175:0/888782907 wait complete. 2024-06-10T00:55:15.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.083+0000 7ff59276c700 1 Processor -- start 2024-06-10T00:55:15.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.083+0000 7ff59276c700 1 -- start start 2024-06-10T00:55:15.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.084+0000 7ff59276c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff58c108020 0x7ff58c19d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:15.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.084+0000 7ff59276c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c10c620 con 0x7ff58c108020 2024-06-10T00:55:15.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.084+0000 7ff58bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff58c108020 0x7ff58c19d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:15.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.084+0000 7ff58bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff58c108020 0x7ff58c19d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.175:36834/0 (socket says 172.21.15.175:36834) 2024-06-10T00:55:15.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.084+0000 7ff58bfff700 1 -- 172.21.15.175:0/2778763556 learned_addr learned my addr 172.21.15.175:0/2778763556 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-06-10T00:55:15.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.084+0000 7ff58bfff700 1 -- 172.21.15.175:0/2778763556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58c19dc60 con 0x7ff58c108020 2024-06-10T00:55:15.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.084+0000 7ff58bfff700 1 --2- 172.21.15.175:0/2778763556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff58c108020 0x7ff58c19d720 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff57c006010 tx=0x7ff57c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:15.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.085+0000 7ff5897fa700 1 -- 172.21.15.175:0/2778763556 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff57c026070 con 0x7ff58c108020 2024-06-10T00:55:15.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.085+0000 7ff59276c700 1 -- 172.21.15.175:0/2778763556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff58c19def0 con 0x7ff58c108020 2024-06-10T00:55:15.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.085+0000 7ff5897fa700 1 -- 172.21.15.175:0/2778763556 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff57c00bc30 con 0x7ff58c108020 2024-06-10T00:55:15.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.085+0000 7ff5897fa700 1 -- 172.21.15.175:0/2778763556 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff57c0039d0 con 0x7ff58c108020 2024-06-10T00:55:15.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.085+0000 7ff59276c700 1 -- 172.21.15.175:0/2778763556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff58c197930 con 0x7ff58c108020 2024-06-10T00:55:15.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.086+0000 7ff5897fa700 1 -- 172.21.15.175:0/2778763556 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7ff57c029070 con 0x7ff58c108020 2024-06-10T00:55:15.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.086+0000 7ff59276c700 1 -- 172.21.15.175:0/2778763556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff58c103740 con 0x7ff58c108020 2024-06-10T00:55:15.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.086+0000 7ff5897fa700 1 --2- 172.21.15.175:0/2778763556 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff5740390f0 0x7ff57403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:15.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.086+0000 7ff5897fa700 1 -- 172.21.15.175:0/2778763556 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff57c01e070 con 0x7ff58c108020 2024-06-10T00:55:15.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.086+0000 7ff58b7fe700 1 --2- 172.21.15.175:0/2778763556 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff5740390f0 0x7ff57403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:15.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.087+0000 7ff58b7fe700 1 --2- 172.21.15.175:0/2778763556 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff5740390f0 0x7ff57403b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff580006fd0 tx=0x7ff580006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:15.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.091+0000 7ff5897fa700 1 -- 172.21.15.175:0/2778763556 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff57c013070 con 0x7ff58c108020 2024-06-10T00:55:15.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.295+0000 7ff59276c700 1 -- 172.21.15.175:0/2778763556 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi007:172.21.15.7=a;smithi078:172.21.15.78=b;smithi175:172.21.15.175=c", "target": ["mon-mgr", ""]}) -- 0x7ff58c198e10 con 0x7ff5740390f0 2024-06-10T00:55:15.301 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.301+0000 7ff5897fa700 1 -- 172.21.15.175:0/2778763556 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff58c198e10 con 0x7ff5740390f0 2024-06-10T00:55:15.301 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled mon update... 2024-06-10T00:55:15.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.303+0000 7ff59276c700 1 -- 172.21.15.175:0/2778763556 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff5740390f0 msgr2=0x7ff57403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:15.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.303+0000 7ff59276c700 1 --2- 172.21.15.175:0/2778763556 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff5740390f0 0x7ff57403b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff580006fd0 tx=0x7ff580006e40 comp rx=0 tx=0).stop 2024-06-10T00:55:15.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.303+0000 7ff59276c700 1 -- 172.21.15.175:0/2778763556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff58c108020 msgr2=0x7ff58c19d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:15.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.303+0000 7ff59276c700 1 --2- 172.21.15.175:0/2778763556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff58c108020 0x7ff58c19d720 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff57c006010 tx=0x7ff57c005030 comp rx=0 tx=0).stop 2024-06-10T00:55:15.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.303+0000 7ff59276c700 1 -- 172.21.15.175:0/2778763556 shutdown_connections 2024-06-10T00:55:15.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.303+0000 7ff59276c700 1 --2- 172.21.15.175:0/2778763556 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff5740390f0 0x7ff57403b5b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:15.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.303+0000 7ff59276c700 1 --2- 172.21.15.175:0/2778763556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff58c108020 0x7ff58c19d720 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:15.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.303+0000 7ff59276c700 1 -- 172.21.15.175:0/2778763556 >> 172.21.15.175:0/2778763556 conn(0x7ff58c077510 msgr2=0x7ff58c103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:15.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.303+0000 7ff59276c700 1 -- 172.21.15.175:0/2778763556 shutdown_connections 2024-06-10T00:55:15.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:15.303+0000 7ff59276c700 1 -- 172.21.15.175:0/2778763556 wait complete. 2024-06-10T00:55:15.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:15 smithi007 ceph-mon[32398]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:15.852 DEBUG:teuthology.orchestra.run.smithi078:mon.b> sudo journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mon.b.service 2024-06-10T00:55:15.856 DEBUG:teuthology.orchestra.run.smithi175:mon.c> sudo journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mon.c.service 2024-06-10T00:55:15.858 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-10T00:55:15.859 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-10T00:55:16.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:16 smithi007 ceph-mon[32398]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi007:172.21.15.7=a;smithi078:172.21.15.78=b;smithi175:172.21.15.175=c", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:55:16.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:16 smithi007 ceph-mon[32398]: Saving service mon spec with placement smithi007:172.21.15.7=a;smithi078:172.21.15.78=b;smithi175:172.21.15.175=c;count:3 2024-06-10T00:55:16.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:16 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:16.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:16 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:55:16.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:16 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:16.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:16 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:55:16.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:16 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:16.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:16 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:55:16.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:16 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:17.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:17 smithi007 ceph-mon[32398]: Deploying daemon mon.c on smithi175 2024-06-10T00:55:17.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:17 smithi007 ceph-mon[32398]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:18.055 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:55:19.112 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 systemd[1]: Starting Ceph mon.c for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T00:55:19.627 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.625+0000 7f13b1fa1700 1 -- 172.21.15.175:0/3075600849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13ac1005b0 msgr2=0x7f13ac100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:19.627 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.625+0000 7f13b1fa1700 1 --2- 172.21.15.175:0/3075600849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13ac1005b0 0x7f13ac100990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f13a0009910 tx=0x7f13a0009c20 comp rx=0 tx=0).stop 2024-06-10T00:55:19.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.625+0000 7f13b1fa1700 1 -- 172.21.15.175:0/3075600849 shutdown_connections 2024-06-10T00:55:19.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.625+0000 7f13b1fa1700 1 --2- 172.21.15.175:0/3075600849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13ac1005b0 0x7f13ac100990 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:19.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.625+0000 7f13b1fa1700 1 -- 172.21.15.175:0/3075600849 >> 172.21.15.175:0/3075600849 conn(0x7f13ac0fbf50 msgr2=0x7f13ac0fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:19.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.625+0000 7f13b1fa1700 1 -- 172.21.15.175:0/3075600849 shutdown_connections 2024-06-10T00:55:19.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.625+0000 7f13b1fa1700 1 -- 172.21.15.175:0/3075600849 wait complete. 2024-06-10T00:55:19.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.626+0000 7f13b1fa1700 1 Processor -- start 2024-06-10T00:55:19.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.626+0000 7f13b1fa1700 1 -- start start 2024-06-10T00:55:19.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.626+0000 7f13b1fa1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13ac1005b0 0x7f13ac19d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:19.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.626+0000 7f13b1fa1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f13ac10c520 con 0x7f13ac1005b0 2024-06-10T00:55:19.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.626+0000 7f13b0f9f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13ac1005b0 0x7f13ac19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:19.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.626+0000 7f13b0f9f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13ac1005b0 0x7f13ac19d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.175:57530/0 (socket says 172.21.15.175:57530) 2024-06-10T00:55:19.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.626+0000 7f13b0f9f700 1 -- 172.21.15.175:0/1021866672 learned_addr learned my addr 172.21.15.175:0/1021866672 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-06-10T00:55:19.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.626+0000 7f13b0f9f700 1 -- 172.21.15.175:0/1021866672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13ac19dbd0 con 0x7f13ac1005b0 2024-06-10T00:55:19.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.627+0000 7f13b0f9f700 1 --2- 172.21.15.175:0/1021866672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13ac1005b0 0x7f13ac19d690 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f13a0005850 tx=0x7f13a0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:19.629 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.627+0000 7f13a9ffb700 1 -- 172.21.15.175:0/1021866672 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13a0026070 con 0x7f13ac1005b0 2024-06-10T00:55:19.629 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.627+0000 7f13b1fa1700 1 -- 172.21.15.175:0/1021866672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13ac19dec0 con 0x7f13ac1005b0 2024-06-10T00:55:19.629 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.627+0000 7f13a9ffb700 1 -- 172.21.15.175:0/1021866672 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13a000bbd0 con 0x7f13ac1005b0 2024-06-10T00:55:19.629 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.627+0000 7f13a9ffb700 1 -- 172.21.15.175:0/1021866672 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13a0003970 con 0x7f13ac1005b0 2024-06-10T00:55:19.629 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 podman[36580]: 2024-06-10 00:55:19.403849848 +0000 UTC m=+0.017533737 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:55:19.630 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.627+0000 7f13b1fa1700 1 -- 172.21.15.175:0/1021866672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13ac197810 con 0x7f13ac1005b0 2024-06-10T00:55:19.630 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.628+0000 7f13b1fa1700 1 -- 172.21.15.175:0/1021866672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13ac063af0 con 0x7f13ac1005b0 2024-06-10T00:55:19.630 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.628+0000 7f13a9ffb700 1 -- 172.21.15.175:0/1021866672 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f13a0029070 con 0x7f13ac1005b0 2024-06-10T00:55:19.630 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.628+0000 7f13a9ffb700 1 --2- 172.21.15.175:0/1021866672 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f139c0390f0 0x7f139c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:19.630 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.628+0000 7f13a9ffb700 1 -- 172.21.15.175:0/1021866672 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f13a001e070 con 0x7f13ac1005b0 2024-06-10T00:55:19.630 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.629+0000 7f13abfff700 1 --2- 172.21.15.175:0/1021866672 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f139c0390f0 0x7f139c03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:19.630 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.629+0000 7f13abfff700 1 --2- 172.21.15.175:0/1021866672 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f139c0390f0 0x7f139c03b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1398006fd0 tx=0x7f1398006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:19.634 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.634+0000 7f13a9ffb700 1 -- 172.21.15.175:0/1021866672 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f13a0013070 con 0x7f13ac1005b0 2024-06-10T00:55:19.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:19 smithi007 ceph-mon[32398]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:19.925 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 podman[36580]: 2024-06-10 00:55:19.732356974 +0000 UTC m=+0.346040851 container create 260eedf0597653cafbd704b522b7b72ca4322e912a5323d3c01b2afdc02c681e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-c, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-06-10T00:55:19.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.924+0000 7f13b1fa1700 1 -- 172.21.15.175:0/1021866672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f13ac103710 con 0x7f13ac1005b0 2024-06-10T00:55:19.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.924+0000 7f13a9ffb700 1 -- 172.21.15.175:0/1021866672 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7f13a0030020 con 0x7f13ac1005b0 2024-06-10T00:55:19.926 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-10T00:55:19.926 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"bded58f8-26c3-11ef-bc9b-c7b262605968","modified":"2024-06-10T00:53:26.101611Z","created":"2024-06-10T00:53:26.101611Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-10T00:55:19.927 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.926+0000 7f13b1fa1700 1 -- 172.21.15.175:0/1021866672 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f139c0390f0 msgr2=0x7f139c03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:19.927 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.926+0000 7f13b1fa1700 1 --2- 172.21.15.175:0/1021866672 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f139c0390f0 0x7f139c03b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1398006fd0 tx=0x7f1398006e40 comp rx=0 tx=0).stop 2024-06-10T00:55:19.927 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.926+0000 7f13b1fa1700 1 -- 172.21.15.175:0/1021866672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13ac1005b0 msgr2=0x7f13ac19d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:19.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.926+0000 7f13b1fa1700 1 --2- 172.21.15.175:0/1021866672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13ac1005b0 0x7f13ac19d690 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f13a0005850 tx=0x7f13a0005030 comp rx=0 tx=0).stop 2024-06-10T00:55:19.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.926+0000 7f13b1fa1700 1 -- 172.21.15.175:0/1021866672 shutdown_connections 2024-06-10T00:55:19.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.926+0000 7f13b1fa1700 1 --2- 172.21.15.175:0/1021866672 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f139c0390f0 0x7f139c03b5b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:19.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.926+0000 7f13b1fa1700 1 --2- 172.21.15.175:0/1021866672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13ac1005b0 0x7f13ac19d690 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:19.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.926+0000 7f13b1fa1700 1 -- 172.21.15.175:0/1021866672 >> 172.21.15.175:0/1021866672 conn(0x7f13ac0fbf50 msgr2=0x7f13ac103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:19.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.927+0000 7f13b1fa1700 1 -- 172.21.15.175:0/1021866672 shutdown_connections 2024-06-10T00:55:19.928 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:19.927+0000 7f13b1fa1700 1 -- 172.21.15.175:0/1021866672 wait complete. 2024-06-10T00:55:19.928 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2024-06-10T00:55:20.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 podman[36580]: 2024-06-10 00:55:19.959415341 +0000 UTC m=+0.573099219 container init 260eedf0597653cafbd704b522b7b72ca4322e912a5323d3c01b2afdc02c681e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T00:55:20.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 podman[36580]: 2024-06-10 00:55:19.962468659 +0000 UTC m=+0.576152538 container start 260eedf0597653cafbd704b522b7b72ca4322e912a5323d3c01b2afdc02c681e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-c, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-06-10T00:55:20.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: set uid:gid to 167:167 (ceph:ceph) 2024-06-10T00:55:20.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc), process ceph-mon, pid 2 2024-06-10T00:55:20.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: pidfile_write: ignore empty --pid-file 2024-06-10T00:55:20.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: load: jerasure load: lrc 2024-06-10T00:55:20.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: RocksDB version: 7.9.2 2024-06-10T00:55:20.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Git sha 0 2024-06-10T00:55:20.210 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Compile date 2024-05-30 15:42:47 2024-06-10T00:55:20.210 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: DB SUMMARY 2024-06-10T00:55:20.210 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: DB Session ID: GDMK4XK83WJZCVH03XSI 2024-06-10T00:55:20.210 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: CURRENT file: CURRENT 2024-06-10T00:55:20.210 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: IDENTITY file: IDENTITY 2024-06-10T00:55:20.210 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-06-10T00:55:20.210 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-06-10T00:55:20.210 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-06-10T00:55:20.210 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.error_if_exists: 0 2024-06-10T00:55:20.210 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.create_if_missing: 0 2024-06-10T00:55:20.210 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.paranoid_checks: 1 2024-06-10T00:55:20.210 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-10T00:55:20.210 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-10T00:55:20.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-10T00:55:20.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.env: 0x563bb6847c60 2024-06-10T00:55:20.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.fs: PosixFileSystem 2024-06-10T00:55:20.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.info_log: 0x563bb878db20 2024-06-10T00:55:20.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.max_file_opening_threads: 16 2024-06-10T00:55:20.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.statistics: (nil) 2024-06-10T00:55:20.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.use_fsync: 0 2024-06-10T00:55:20.212 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.max_log_file_size: 0 2024-06-10T00:55:20.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-10T00:55:20.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-10T00:55:20.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.keep_log_file_num: 1000 2024-06-10T00:55:20.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.recycle_log_file_num: 0 2024-06-10T00:55:20.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.allow_fallocate: 1 2024-06-10T00:55:20.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.allow_mmap_reads: 0 2024-06-10T00:55:20.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.allow_mmap_writes: 0 2024-06-10T00:55:20.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.use_direct_reads: 0 2024-06-10T00:55:20.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-10T00:55:20.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.create_missing_column_families: 0 2024-06-10T00:55:20.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.db_log_dir: 2024-06-10T00:55:20.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.wal_dir: 2024-06-10T00:55:20.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-10T00:55:20.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-10T00:55:20.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-10T00:55:20.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-10T00:55:20.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-10T00:55:20.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-10T00:55:20.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.advise_random_on_open: 1 2024-06-10T00:55:20.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.db_write_buffer_size: 0 2024-06-10T00:55:20.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.write_buffer_manager: 0x563bb875e1e0 2024-06-10T00:55:20.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-10T00:55:20.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-10T00:55:20.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-10T00:55:20.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.rate_limiter: (nil) 2024-06-10T00:55:20.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-10T00:55:20.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.wal_recovery_mode: 2 2024-06-10T00:55:20.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.enable_thread_tracking: 0 2024-06-10T00:55:20.215 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.enable_pipelined_write: 0 2024-06-10T00:55:20.215 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.unordered_write: 0 2024-06-10T00:55:20.215 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-10T00:55:20.215 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-10T00:55:20.215 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-10T00:55:20.215 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-10T00:55:20.215 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.row_cache: None 2024-06-10T00:55:20.215 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.wal_filter: None 2024-06-10T00:55:20.215 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-10T00:55:20.215 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.allow_ingest_behind: 0 2024-06-10T00:55:20.215 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.two_write_queues: 0 2024-06-10T00:55:20.215 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.manual_wal_flush: 0 2024-06-10T00:55:20.215 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.wal_compression: 0 2024-06-10T00:55:20.215 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.atomic_flush: 0 2024-06-10T00:55:20.215 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-10T00:55:20.216 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-10T00:55:20.216 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-10T00:55:20.216 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.log_readahead_size: 0 2024-06-10T00:55:20.216 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-10T00:55:20.216 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.best_efforts_recovery: 0 2024-06-10T00:55:20.216 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-10T00:55:20.216 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-10T00:55:20.216 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.allow_data_in_errors: 0 2024-06-10T00:55:20.216 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.db_host_id: __hostname__ 2024-06-10T00:55:20.216 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-10T00:55:20.216 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.max_background_jobs: 2 2024-06-10T00:55:20.216 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.max_background_compactions: -1 2024-06-10T00:55:20.216 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.max_subcompactions: 1 2024-06-10T00:55:20.216 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-10T00:55:20.217 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-10T00:55:20.217 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-10T00:55:20.217 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.max_total_wal_size: 0 2024-06-10T00:55:20.217 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-10T00:55:20.217 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-10T00:55:20.217 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-10T00:55:20.217 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-10T00:55:20.217 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.max_open_files: -1 2024-06-10T00:55:20.217 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.bytes_per_sync: 0 2024-06-10T00:55:20.217 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-10T00:55:20.217 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-10T00:55:20.217 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.compaction_readahead_size: 0 2024-06-10T00:55:20.217 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.max_background_flushes: -1 2024-06-10T00:55:20.217 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Compression algorithms supported: 2024-06-10T00:55:20.217 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-10T00:55:20.218 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: kZSTD supported: 0 2024-06-10T00:55:20.218 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: kXpressCompression supported: 0 2024-06-10T00:55:20.218 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: kLZ4HCCompression supported: 1 2024-06-10T00:55:20.218 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: kZlibCompression supported: 1 2024-06-10T00:55:20.218 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: kSnappyCompression supported: 1 2024-06-10T00:55:20.218 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: kLZ4Compression supported: 1 2024-06-10T00:55:20.218 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: kBZip2Compression supported: 0 2024-06-10T00:55:20.218 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-10T00:55:20.218 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-10T00:55:20.218 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-06-10T00:55:20.218 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-10T00:55:20.218 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-10T00:55:20.218 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.merge_operator: 2024-06-10T00:55:20.218 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.compaction_filter: None 2024-06-10T00:55:20.219 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.compaction_filter_factory: None 2024-06-10T00:55:20.219 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.sst_partitioner_factory: None 2024-06-10T00:55:20.219 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-10T00:55:20.219 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-10T00:55:20.219 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563bb878dc40) 2024-06-10T00:55:20.219 INFO:journalctl@ceph.mon.c.smithi175.stdout: cache_index_and_filter_blocks: 1 2024-06-10T00:55:20.219 INFO:journalctl@ceph.mon.c.smithi175.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-10T00:55:20.219 INFO:journalctl@ceph.mon.c.smithi175.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-10T00:55:20.219 INFO:journalctl@ceph.mon.c.smithi175.stdout: pin_top_level_index_and_filter: 1 2024-06-10T00:55:20.219 INFO:journalctl@ceph.mon.c.smithi175.stdout: index_type: 0 2024-06-10T00:55:20.219 INFO:journalctl@ceph.mon.c.smithi175.stdout: data_block_index_type: 0 2024-06-10T00:55:20.219 INFO:journalctl@ceph.mon.c.smithi175.stdout: index_shortening: 1 2024-06-10T00:55:20.219 INFO:journalctl@ceph.mon.c.smithi175.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-10T00:55:20.219 INFO:journalctl@ceph.mon.c.smithi175.stdout: checksum: 4 2024-06-10T00:55:20.219 INFO:journalctl@ceph.mon.c.smithi175.stdout: no_block_cache: 0 2024-06-10T00:55:20.219 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_cache: 0x563bb878f090 2024-06-10T00:55:20.220 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_cache_name: BinnedLRUCache 2024-06-10T00:55:20.220 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_cache_options: 2024-06-10T00:55:20.220 INFO:journalctl@ceph.mon.c.smithi175.stdout: capacity : 536870912 2024-06-10T00:55:20.220 INFO:journalctl@ceph.mon.c.smithi175.stdout: num_shard_bits : 4 2024-06-10T00:55:20.220 INFO:journalctl@ceph.mon.c.smithi175.stdout: strict_capacity_limit : 0 2024-06-10T00:55:20.220 INFO:journalctl@ceph.mon.c.smithi175.stdout: high_pri_pool_ratio: 0.000 2024-06-10T00:55:20.220 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_cache_compressed: (nil) 2024-06-10T00:55:20.220 INFO:journalctl@ceph.mon.c.smithi175.stdout: persistent_cache: (nil) 2024-06-10T00:55:20.220 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_size: 4096 2024-06-10T00:55:20.220 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_size_deviation: 10 2024-06-10T00:55:20.220 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_restart_interval: 16 2024-06-10T00:55:20.220 INFO:journalctl@ceph.mon.c.smithi175.stdout: index_block_restart_interval: 1 2024-06-10T00:55:20.220 INFO:journalctl@ceph.mon.c.smithi175.stdout: metadata_block_size: 4096 2024-06-10T00:55:20.220 INFO:journalctl@ceph.mon.c.smithi175.stdout: partition_filters: 0 2024-06-10T00:55:20.221 INFO:journalctl@ceph.mon.c.smithi175.stdout: use_delta_encoding: 1 2024-06-10T00:55:20.221 INFO:journalctl@ceph.mon.c.smithi175.stdout: filter_policy: bloomfilter 2024-06-10T00:55:20.221 INFO:journalctl@ceph.mon.c.smithi175.stdout: whole_key_filtering: 1 2024-06-10T00:55:20.221 INFO:journalctl@ceph.mon.c.smithi175.stdout: verify_compression: 0 2024-06-10T00:55:20.221 INFO:journalctl@ceph.mon.c.smithi175.stdout: read_amp_bytes_per_bit: 0 2024-06-10T00:55:20.221 INFO:journalctl@ceph.mon.c.smithi175.stdout: format_version: 5 2024-06-10T00:55:20.221 INFO:journalctl@ceph.mon.c.smithi175.stdout: enable_index_compression: 1 2024-06-10T00:55:20.221 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_align: 0 2024-06-10T00:55:20.221 INFO:journalctl@ceph.mon.c.smithi175.stdout: max_auto_readahead_size: 262144 2024-06-10T00:55:20.221 INFO:journalctl@ceph.mon.c.smithi175.stdout: prepopulate_block_cache: 0 2024-06-10T00:55:20.221 INFO:journalctl@ceph.mon.c.smithi175.stdout: initial_auto_readahead_size: 8192 2024-06-10T00:55:20.221 INFO:journalctl@ceph.mon.c.smithi175.stdout: num_file_reads_for_auto_readahead: 2 2024-06-10T00:55:20.221 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.write_buffer_size: 33554432 2024-06-10T00:55:20.221 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.max_write_buffer_number: 2 2024-06-10T00:55:20.221 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.compression: NoCompression 2024-06-10T00:55:20.222 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.bottommost_compression: Disabled 2024-06-10T00:55:20.222 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.prefix_extractor: nullptr 2024-06-10T00:55:20.222 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-10T00:55:20.222 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.num_levels: 7 2024-06-10T00:55:20.222 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-10T00:55:20.222 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-10T00:55:20.222 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-10T00:55:20.222 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-10T00:55:20.222 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-10T00:55:20.222 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-10T00:55:20.222 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-10T00:55:20.222 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-10T00:55:20.222 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-10T00:55:20.222 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-10T00:55:20.222 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-10T00:55:20.223 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-10T00:55:20.223 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-10T00:55:20.223 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.compression_opts.level: 32767 2024-06-10T00:55:20.223 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.compression_opts.strategy: 0 2024-06-10T00:55:20.223 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-10T00:55:20.223 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-10T00:55:20.223 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-10T00:55:20.223 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-10T00:55:20.223 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.compression_opts.enabled: false 2024-06-10T00:55:20.223 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-10T00:55:20.223 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-10T00:55:20.223 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-10T00:55:20.223 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-10T00:55:20.223 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.target_file_size_base: 67108864 2024-06-10T00:55:20.223 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-10T00:55:20.224 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-10T00:55:20.224 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-10T00:55:20.224 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-10T00:55:20.224 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-10T00:55:20.224 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-10T00:55:20.224 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-10T00:55:20.224 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-10T00:55:20.224 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-10T00:55:20.224 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-10T00:55:20.224 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-10T00:55:20.224 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-10T00:55:20.224 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-10T00:55:20.224 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-10T00:55:20.224 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.arena_block_size: 1048576 2024-06-10T00:55:20.224 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-10T00:55:20.225 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-10T00:55:20.225 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.disable_auto_compactions: 0 2024-06-10T00:55:20.225 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-10T00:55:20.225 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-10T00:55:20.225 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-10T00:55:20.225 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-10T00:55:20.225 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-10T00:55:20.225 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-10T00:55:20.225 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-10T00:55:20.225 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-10T00:55:20.225 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-10T00:55:20.225 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-10T00:55:20.225 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.table_properties_collectors: 2024-06-10T00:55:20.225 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.inplace_update_support: 0 2024-06-10T00:55:20.225 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-10T00:55:20.226 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-10T00:55:20.226 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-10T00:55:20.226 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-10T00:55:20.226 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.bloom_locality: 0 2024-06-10T00:55:20.226 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.max_successive_merges: 0 2024-06-10T00:55:20.226 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-10T00:55:20.226 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.paranoid_file_checks: 0 2024-06-10T00:55:20.226 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.force_consistency_checks: 1 2024-06-10T00:55:20.226 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.report_bg_io_stats: 0 2024-06-10T00:55:20.226 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.ttl: 2592000 2024-06-10T00:55:20.226 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-10T00:55:20.226 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-10T00:55:20.226 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-10T00:55:20.226 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.enable_blob_files: false 2024-06-10T00:55:20.226 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.min_blob_size: 0 2024-06-10T00:55:20.227 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.blob_file_size: 268435456 2024-06-10T00:55:20.227 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-10T00:55:20.227 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-10T00:55:20.227 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-10T00:55:20.227 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-10T00:55:20.227 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-10T00:55:20.227 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.blob_file_starting_level: 0 2024-06-10T00:55:20.227 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-10T00:55:20.227 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-06-10T00:55:20.227 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-06-10T00:55:20.227 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: be6f25c7-8589-4c0e-883e-27134697cc0d 2024-06-10T00:55:20.227 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717980919991854, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-06-10T00:55:20.227 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-06-10T00:55:20.227 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717980919992281, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1717980919, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "be6f25c7-8589-4c0e-883e-27134697cc0d", "db_session_id": "GDMK4XK83WJZCVH03XSI", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-06-10T00:55:20.228 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717980919992360, "job": 1, "event": "recovery_finished"} 2024-06-10T00:55:20.228 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-06-10T00:55:20.228 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-10T00:55:20.228 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563bb887e000 2024-06-10T00:55:20.228 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: DB pointer 0x563bb8868000 2024-06-10T00:55:20.228 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-10T00:55:20.228 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-10T00:55:20.228 INFO:journalctl@ceph.mon.c.smithi175.stdout: ** DB Stats ** 2024-06-10T00:55:20.228 INFO:journalctl@ceph.mon.c.smithi175.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-10T00:55:20.228 INFO:journalctl@ceph.mon.c.smithi175.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-10T00:55:20.228 INFO:journalctl@ceph.mon.c.smithi175.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-10T00:55:20.228 INFO:journalctl@ceph.mon.c.smithi175.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-10T00:55:20.228 INFO:journalctl@ceph.mon.c.smithi175.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-10T00:55:20.228 INFO:journalctl@ceph.mon.c.smithi175.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-10T00:55:20.228 INFO:journalctl@ceph.mon.c.smithi175.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-10T00:55:20.228 INFO:journalctl@ceph.mon.c.smithi175.stdout: 2024-06-10T00:55:20.229 INFO:journalctl@ceph.mon.c.smithi175.stdout: ** Compaction Stats [default] ** 2024-06-10T00:55:20.229 INFO:journalctl@ceph.mon.c.smithi175.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-10T00:55:20.229 INFO:journalctl@ceph.mon.c.smithi175.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-10T00:55:20.229 INFO:journalctl@ceph.mon.c.smithi175.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-10T00:55:20.229 INFO:journalctl@ceph.mon.c.smithi175.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-10T00:55:20.229 INFO:journalctl@ceph.mon.c.smithi175.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-10T00:55:20.229 INFO:journalctl@ceph.mon.c.smithi175.stdout: 2024-06-10T00:55:20.229 INFO:journalctl@ceph.mon.c.smithi175.stdout: ** Compaction Stats [default] ** 2024-06-10T00:55:20.229 INFO:journalctl@ceph.mon.c.smithi175.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-10T00:55:20.229 INFO:journalctl@ceph.mon.c.smithi175.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-10T00:55:20.229 INFO:journalctl@ceph.mon.c.smithi175.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-10T00:55:20.229 INFO:journalctl@ceph.mon.c.smithi175.stdout: 2024-06-10T00:55:20.229 INFO:journalctl@ceph.mon.c.smithi175.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-10T00:55:20.229 INFO:journalctl@ceph.mon.c.smithi175.stdout: 2024-06-10T00:55:20.230 INFO:journalctl@ceph.mon.c.smithi175.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-10T00:55:20.230 INFO:journalctl@ceph.mon.c.smithi175.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-10T00:55:20.230 INFO:journalctl@ceph.mon.c.smithi175.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-10T00:55:20.230 INFO:journalctl@ceph.mon.c.smithi175.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-10T00:55:20.230 INFO:journalctl@ceph.mon.c.smithi175.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-10T00:55:20.230 INFO:journalctl@ceph.mon.c.smithi175.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-10T00:55:20.230 INFO:journalctl@ceph.mon.c.smithi175.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-10T00:55:20.230 INFO:journalctl@ceph.mon.c.smithi175.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-10T00:55:20.230 INFO:journalctl@ceph.mon.c.smithi175.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-10T00:55:20.230 INFO:journalctl@ceph.mon.c.smithi175.stdout: Block cache BinnedLRUCache@0x563bb878f090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-06-10T00:55:20.230 INFO:journalctl@ceph.mon.c.smithi175.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-06-10T00:55:20.230 INFO:journalctl@ceph.mon.c.smithi175.stdout: 2024-06-10T00:55:20.230 INFO:journalctl@ceph.mon.c.smithi175.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-10T00:55:20.230 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-06-10T00:55:20.231 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: using public_addr v2:172.21.15.175:0/0 -> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2024-06-10T00:55:20.231 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: starting mon.c rank -1 at public addrs [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] at bind addrs [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T00:55:20.231 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:19 smithi175 ceph-mon[36636]: mon.c@-1(???) e0 preinit fsid bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T00:55:20.231 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: mon.c@-1(synchronizing).mds e1 new map 2024-06-10T00:55:20.231 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: mon.c@-1(synchronizing).mds e1 print_map 2024-06-10T00:55:20.231 INFO:journalctl@ceph.mon.c.smithi175.stdout: e1 2024-06-10T00:55:20.231 INFO:journalctl@ceph.mon.c.smithi175.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-10T00:55:20.231 INFO:journalctl@ceph.mon.c.smithi175.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-10T00:55:20.231 INFO:journalctl@ceph.mon.c.smithi175.stdout: legacy client fscid: -1 2024-06-10T00:55:20.231 INFO:journalctl@ceph.mon.c.smithi175.stdout: 2024-06-10T00:55:20.231 INFO:journalctl@ceph.mon.c.smithi175.stdout: No filesystems configured 2024-06-10T00:55:20.231 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-06-10T00:55:20.231 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-06-10T00:55:20.231 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-06-10T00:55:20.232 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-06-10T00:55:20.232 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-06-10T00:55:20.232 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-06-10T00:55:20.232 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-10T00:55:20.232 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-10T00:55:20.232 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-10T00:55:20.232 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-10T00:55:20.232 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:20.232 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:20.232 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: Added host smithi078 2024-06-10T00:55:20.232 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:55:20.232 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:20.232 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:20.233 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:20.233 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:20.233 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:55:20.233 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:20.233 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:20.233 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:20.233 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:20.233 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:20.233 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi078", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:55:20.233 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:20.233 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:55:20.233 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: Updating smithi078:/etc/ceph/ceph.conf 2024-06-10T00:55:20.233 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi175", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:55:20.234 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: Updating smithi078:/var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:55:20.234 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:55:20.234 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: Deploying cephadm binary to smithi175 2024-06-10T00:55:20.234 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:20.234 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:20.234 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:20.234 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: Updating smithi078:/var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-10T00:55:20.234 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:20.234 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:20.234 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:20.234 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: Added host smithi175 2024-06-10T00:55:20.234 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:55:20.234 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:20.234 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:20.235 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:20.235 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:55:20.235 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:20.235 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:20.235 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:20.235 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1961130065' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-10T00:55:20.235 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:20.235 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:20.235 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:20.235 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:20.235 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi175", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:55:20.235 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:20.235 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:55:20.235 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: Updating smithi175:/etc/ceph/ceph.conf 2024-06-10T00:55:20.235 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: Updating smithi175:/var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:55:20.236 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:55:20.236 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1961130065' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-10T00:55:20.236 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T00:55:20.236 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: Updating smithi175:/var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-10T00:55:20.236 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:20.236 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:20.236 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:20.236 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:20.236 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi007:172.21.15.7=a;smithi078:172.21.15.78=b;smithi175:172.21.15.175=c", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:55:20.236 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: Saving service mon spec with placement smithi007:172.21.15.7=a;smithi078:172.21.15.78=b;smithi175:172.21.15.175=c;count:3 2024-06-10T00:55:20.236 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:20.236 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:55:20.236 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:20.237 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:55:20.237 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:20.237 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:55:20.237 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:20.237 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: Deploying daemon mon.c on smithi175 2024-06-10T00:55:20.237 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:20.237 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:20.237 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 ceph-mon[36636]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-06-10T00:55:20.237 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 bash[36580]: 260eedf0597653cafbd704b522b7b72ca4322e912a5323d3c01b2afdc02c681e 2024-06-10T00:55:20.237 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:20 smithi175 systemd[1]: Started Ceph mon.c for bded58f8-26c3-11ef-bc9b-c7b262605968. 2024-06-10T00:55:21.549 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-10T00:55:21.549 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-10T00:55:21.793 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:55:25.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:25 smithi175 ceph-mon[36636]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:25.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:25 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:55:25.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:25 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:55:25.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:25 smithi175 ceph-mon[36636]: mon.a calling monitor election 2024-06-10T00:55:25.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:25 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:55:25.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:25 smithi175 ceph-mon[36636]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:25.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:25 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:55:25.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:25 smithi175 ceph-mon[36636]: mon.c calling monitor election 2024-06-10T00:55:25.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:25 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:55:25.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:25 smithi175 ceph-mon[36636]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:25.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:25 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:55:25.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:25 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:55:25.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:25 smithi175 ceph-mon[36636]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-10T00:55:25.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:25 smithi175 ceph-mon[36636]: monmap epoch 2 2024-06-10T00:55:25.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:25 smithi175 ceph-mon[36636]: fsid bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T00:55:25.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:25 smithi175 ceph-mon[36636]: last_changed 2024-06-10T00:55:20.017398+0000 2024-06-10T00:55:25.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:25 smithi175 ceph-mon[36636]: created 2024-06-10T00:53:26.101611+0000 2024-06-10T00:55:25.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:25 smithi175 ceph-mon[36636]: min_mon_release 19 (squid) 2024-06-10T00:55:25.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:25 smithi175 ceph-mon[36636]: election_strategy: 1 2024-06-10T00:55:25.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:25 smithi175 ceph-mon[36636]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-06-10T00:55:25.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:25 smithi175 ceph-mon[36636]: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.c 2024-06-10T00:55:25.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:25 smithi175 ceph-mon[36636]: fsmap 2024-06-10T00:55:25.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:25 smithi175 ceph-mon[36636]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T00:55:25.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:25 smithi175 ceph-mon[36636]: mgrmap e13: x(active, since 49s) 2024-06-10T00:55:25.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:25 smithi175 ceph-mon[36636]: overall HEALTH_OK 2024-06-10T00:55:25.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:25 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:25.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:25 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:25.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:25 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:25.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:25 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:55:25.461 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:25 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:25.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:25 smithi007 ceph-mon[32398]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:25.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:25 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:55:25.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:25 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:55:25.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:25 smithi007 ceph-mon[32398]: mon.a calling monitor election 2024-06-10T00:55:25.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:25 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:55:25.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:25 smithi007 ceph-mon[32398]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:25.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:25 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:55:25.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:25 smithi007 ceph-mon[32398]: mon.c calling monitor election 2024-06-10T00:55:25.520 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:25 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:55:25.520 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:25 smithi007 ceph-mon[32398]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:25.520 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:25 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:55:25.520 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:25 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:55:25.520 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:25 smithi007 ceph-mon[32398]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-10T00:55:25.520 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:25 smithi007 ceph-mon[32398]: monmap epoch 2 2024-06-10T00:55:25.520 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:25 smithi007 ceph-mon[32398]: fsid bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T00:55:25.520 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:25 smithi007 ceph-mon[32398]: last_changed 2024-06-10T00:55:20.017398+0000 2024-06-10T00:55:25.520 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:25 smithi007 ceph-mon[32398]: created 2024-06-10T00:53:26.101611+0000 2024-06-10T00:55:25.520 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:25 smithi007 ceph-mon[32398]: min_mon_release 19 (squid) 2024-06-10T00:55:25.520 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:25 smithi007 ceph-mon[32398]: election_strategy: 1 2024-06-10T00:55:25.520 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:25 smithi007 ceph-mon[32398]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-06-10T00:55:25.520 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:25 smithi007 ceph-mon[32398]: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.c 2024-06-10T00:55:25.520 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:25 smithi007 ceph-mon[32398]: fsmap 2024-06-10T00:55:25.521 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:25 smithi007 ceph-mon[32398]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T00:55:25.521 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:25 smithi007 ceph-mon[32398]: mgrmap e13: x(active, since 49s) 2024-06-10T00:55:25.521 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:25 smithi007 ceph-mon[32398]: overall HEALTH_OK 2024-06-10T00:55:25.521 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:25 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:25.521 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:25 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:25.521 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:25 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:25.521 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:25 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:55:25.521 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:25 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:26.306 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.305+0000 7f38a8843700 1 -- 172.21.15.175:0/2068371396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f388c0035c0 msgr2=0x7f388c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:26.306 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.305+0000 7f38a8843700 1 --2- 172.21.15.175:0/2068371396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f388c0035c0 0x7f388c005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f38a4100dd0 tx=0x7f389800feb0 comp rx=0 tx=0).stop 2024-06-10T00:55:26.306 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.305+0000 7f38a8843700 1 -- 172.21.15.175:0/2068371396 shutdown_connections 2024-06-10T00:55:26.306 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.305+0000 7f38a8843700 1 --2- 172.21.15.175:0/2068371396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f388c0035c0 0x7f388c005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:26.306 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.305+0000 7f38a8843700 1 -- 172.21.15.175:0/2068371396 >> 172.21.15.175:0/2068371396 conn(0x7f38a40fc9d0 msgr2=0x7f38a40fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:26.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.305+0000 7f38a8843700 1 -- 172.21.15.175:0/2068371396 shutdown_connections 2024-06-10T00:55:26.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.305+0000 7f38a8843700 1 -- 172.21.15.175:0/2068371396 wait complete. 2024-06-10T00:55:26.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.306+0000 7f38a8843700 1 Processor -- start 2024-06-10T00:55:26.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.307+0000 7f38a8843700 1 -- start start 2024-06-10T00:55:26.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.307+0000 7f38a8843700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f388c0035c0 0x7f38a419c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:26.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.307+0000 7f38a8843700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38a419cbe0 0x7f38a41968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:26.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.307+0000 7f38a8843700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f38a4108f30 con 0x7f388c0035c0 2024-06-10T00:55:26.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.307+0000 7f38a8843700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f38a4108db0 con 0x7f38a419cbe0 2024-06-10T00:55:26.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.307+0000 7f38a1d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38a419cbe0 0x7f38a41968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:26.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.307+0000 7f38a1d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38a419cbe0 0x7f38a41968b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:32778/0 (socket says 172.21.15.175:32778) 2024-06-10T00:55:26.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.307+0000 7f38a1d9b700 1 -- 172.21.15.175:0/3398092269 learned_addr learned my addr 172.21.15.175:0/3398092269 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-06-10T00:55:26.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.307+0000 7f38a259c700 1 --2- 172.21.15.175:0/3398092269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f388c0035c0 0x7f38a419c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:26.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.307+0000 7f38a1d9b700 1 -- 172.21.15.175:0/3398092269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38a419cbe0 msgr2=0x7f38a41968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-06-10T00:55:26.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.307+0000 7f38a1d9b700 1 -- 172.21.15.175:0/3398092269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38a419cbe0 msgr2=0x7f38a41968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-10T00:55:26.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.307+0000 7f38a1d9b700 1 --2- 172.21.15.175:0/3398092269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38a419cbe0 0x7f38a41968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-10T00:55:26.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.307+0000 7f38a1d9b700 1 --2- 172.21.15.175:0/3398092269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38a419cbe0 0x7f38a41968b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T00:55:26.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.308+0000 7f38a259c700 1 -- 172.21.15.175:0/3398092269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38a419cbe0 msgr2=0x7f38a41968b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T00:55:26.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.308+0000 7f38a259c700 1 --2- 172.21.15.175:0/3398092269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38a419cbe0 0x7f38a41968b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:26.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.308+0000 7f38a259c700 1 -- 172.21.15.175:0/3398092269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a419d220 con 0x7f388c0035c0 2024-06-10T00:55:26.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.308+0000 7f38a259c700 1 --2- 172.21.15.175:0/3398092269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f388c0035c0 0x7f38a419c6a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3898010010 tx=0x7f38980055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:26.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.308+0000 7f38937fe700 1 -- 172.21.15.175:0/3398092269 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f3898004b20 con 0x7f388c0035c0 2024-06-10T00:55:26.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.308+0000 7f38937fe700 1 -- 172.21.15.175:0/3398092269 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3898013e40 con 0x7f388c0035c0 2024-06-10T00:55:26.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.308+0000 7f38a8843700 1 -- 172.21.15.175:0/3398092269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a4196df0 con 0x7f388c0035c0 2024-06-10T00:55:26.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.308+0000 7f38937fe700 1 -- 172.21.15.175:0/3398092269 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f3898011b60 con 0x7f388c0035c0 2024-06-10T00:55:26.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.308+0000 7f38a8843700 1 -- 172.21.15.175:0/3398092269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a4197380 con 0x7f388c0035c0 2024-06-10T00:55:26.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.309+0000 7f38937fe700 1 -- 172.21.15.175:0/3398092269 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f3898023070 con 0x7f388c0035c0 2024-06-10T00:55:26.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.309+0000 7f38937fe700 1 --2- 172.21.15.175:0/3398092269 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3884038ee0 0x7f388403b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:26.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.310+0000 7f38937fe700 1 -- 172.21.15.175:0/3398092269 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3898020050 con 0x7f388c0035c0 2024-06-10T00:55:26.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.310+0000 7f38a8843700 1 -- 172.21.15.175:0/3398092269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38a41025e0 con 0x7f388c0035c0 2024-06-10T00:55:26.312 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.310+0000 7f38a1d9b700 1 --2- 172.21.15.175:0/3398092269 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3884038ee0 0x7f388403b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:26.312 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.310+0000 7f38a1d9b700 1 --2- 172.21.15.175:0/3398092269 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3884038ee0 0x7f388403b3a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3894009180 tx=0x7f3894008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:26.316 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.315+0000 7f38937fe700 1 -- 172.21.15.175:0/3398092269 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3898028020 con 0x7f388c0035c0 2024-06-10T00:55:26.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:26 smithi175 ceph-mon[36636]: Deploying daemon mon.b on smithi078 2024-06-10T00:55:26.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:26 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:55:26.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:26 smithi007 ceph-mon[32398]: Deploying daemon mon.b on smithi078 2024-06-10T00:55:26.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:26 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:55:26.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.596+0000 7f38a8843700 1 -- 172.21.15.175:0/3398092269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f38a4197d90 con 0x7f388c0035c0 2024-06-10T00:55:26.597 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.597+0000 7f38937fe700 1 -- 172.21.15.175:0/3398092269 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f389802b350 con 0x7f388c0035c0 2024-06-10T00:55:26.598 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-10T00:55:26.598 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":2,"fsid":"bded58f8-26c3-11ef-bc9b-c7b262605968","modified":"2024-06-10T00:55:20.017398Z","created":"2024-06-10T00:53:26.101611Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:3300","nonce":0},{"type":"v1","addr":"172.21.15.175:6789","nonce":0}]},"addr":"172.21.15.175:6789/0","public_addr":"172.21.15.175:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-10T00:55:26.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.599+0000 7f38a8843700 1 -- 172.21.15.175:0/3398092269 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3884038ee0 msgr2=0x7f388403b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:26.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.599+0000 7f38a8843700 1 --2- 172.21.15.175:0/3398092269 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3884038ee0 0x7f388403b3a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3894009180 tx=0x7f3894008040 comp rx=0 tx=0).stop 2024-06-10T00:55:26.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.599+0000 7f38a8843700 1 -- 172.21.15.175:0/3398092269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f388c0035c0 msgr2=0x7f38a419c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:26.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.599+0000 7f38a8843700 1 --2- 172.21.15.175:0/3398092269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f388c0035c0 0x7f38a419c6a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3898010010 tx=0x7f38980055a0 comp rx=0 tx=0).stop 2024-06-10T00:55:26.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.599+0000 7f38a8843700 1 -- 172.21.15.175:0/3398092269 shutdown_connections 2024-06-10T00:55:26.600 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.599+0000 7f38a8843700 1 --2- 172.21.15.175:0/3398092269 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3884038ee0 0x7f388403b3a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:26.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.599+0000 7f38a8843700 1 --2- 172.21.15.175:0/3398092269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f388c0035c0 0x7f38a419c6a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:26.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.599+0000 7f38a8843700 1 --2- 172.21.15.175:0/3398092269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38a419cbe0 0x7f38a41968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:26.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.599+0000 7f38a8843700 1 -- 172.21.15.175:0/3398092269 >> 172.21.15.175:0/3398092269 conn(0x7f38a40fc9d0 msgr2=0x7f38a40ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:26.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.599+0000 7f38a8843700 1 -- 172.21.15.175:0/3398092269 shutdown_connections 2024-06-10T00:55:26.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:26.599+0000 7f38a8843700 1 -- 172.21.15.175:0/3398092269 wait complete. 2024-06-10T00:55:26.601 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 2 2024-06-10T00:55:27.139 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:27 smithi175 ceph-mon[36636]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:27.140 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:27 smithi175 ceph-mon[36636]: from='client.? 172.21.15.175:0/3398092269' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-10T00:55:27.420 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:27 smithi007 ceph-mon[32398]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:27.420 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:27 smithi007 ceph-mon[32398]: from='client.? 172.21.15.175:0/3398092269' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-10T00:55:28.170 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-10T00:55:28.170 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-10T00:55:28.413 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:55:28.806 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:28 smithi078 ceph-mon[36683]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-06-10T00:55:29.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:28 smithi078 bash[36663]: e51fad75031b90f85bc64968f2bb50ac95abcf1da8f979447f83a6f499c4b6b4 2024-06-10T00:55:29.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:28 smithi078 systemd[1]: Started Ceph mon.b for bded58f8-26c3-11ef-bc9b-c7b262605968. 2024-06-10T00:55:34.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:33 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:55:34.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:33 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:55:34.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:33 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:55:34.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:33 smithi175 ceph-mon[36636]: mon.a calling monitor election 2024-06-10T00:55:34.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:33 smithi175 ceph-mon[36636]: mon.c calling monitor election 2024-06-10T00:55:34.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:33 smithi175 ceph-mon[36636]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:34.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:33 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:55:34.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:33 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:55:34.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:33 smithi175 ceph-mon[36636]: mon.b calling monitor election 2024-06-10T00:55:34.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:33 smithi175 ceph-mon[36636]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:34.210 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:33 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:55:34.210 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:33 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:55:34.210 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:33 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:55:34.210 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:33 smithi175 ceph-mon[36636]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-10T00:55:34.210 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:33 smithi175 ceph-mon[36636]: monmap epoch 3 2024-06-10T00:55:34.210 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:33 smithi175 ceph-mon[36636]: fsid bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T00:55:34.210 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:33 smithi175 ceph-mon[36636]: last_changed 2024-06-10T00:55:28.780516+0000 2024-06-10T00:55:34.210 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:33 smithi175 ceph-mon[36636]: created 2024-06-10T00:53:26.101611+0000 2024-06-10T00:55:34.210 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:33 smithi175 ceph-mon[36636]: min_mon_release 19 (squid) 2024-06-10T00:55:34.210 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:33 smithi175 ceph-mon[36636]: election_strategy: 1 2024-06-10T00:55:34.210 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:33 smithi175 ceph-mon[36636]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-06-10T00:55:34.210 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:33 smithi175 ceph-mon[36636]: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.c 2024-06-10T00:55:34.210 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:33 smithi175 ceph-mon[36636]: 2: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.b 2024-06-10T00:55:34.210 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:33 smithi175 ceph-mon[36636]: fsmap 2024-06-10T00:55:34.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:33 smithi175 ceph-mon[36636]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T00:55:34.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:33 smithi175 ceph-mon[36636]: mgrmap e13: x(active, since 58s) 2024-06-10T00:55:34.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:33 smithi175 ceph-mon[36636]: overall HEALTH_OK 2024-06-10T00:55:34.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:33 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:34.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:33 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:34.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.243+0000 7f9a919f1700 1 -- 172.21.15.175:0/3151483773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a740035c0 msgr2=0x7f9a74005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:34.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.243+0000 7f9a919f1700 1 --2- 172.21.15.175:0/3151483773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a740035c0 0x7f9a74005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c100db0 tx=0x7f9a84009e80 comp rx=0 tx=0).stop 2024-06-10T00:55:34.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.243+0000 7f9a919f1700 1 -- 172.21.15.175:0/3151483773 shutdown_connections 2024-06-10T00:55:34.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.243+0000 7f9a919f1700 1 --2- 172.21.15.175:0/3151483773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a740035c0 0x7f9a74005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:34.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.243+0000 7f9a919f1700 1 -- 172.21.15.175:0/3151483773 >> 172.21.15.175:0/3151483773 conn(0x7f9a8c0fc970 msgr2=0x7f9a8c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:34.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.243+0000 7f9a919f1700 1 -- 172.21.15.175:0/3151483773 shutdown_connections 2024-06-10T00:55:34.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.243+0000 7f9a919f1700 1 -- 172.21.15.175:0/3151483773 wait complete. 2024-06-10T00:55:34.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.244+0000 7f9a919f1700 1 Processor -- start 2024-06-10T00:55:34.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.244+0000 7f9a919f1700 1 -- start start 2024-06-10T00:55:34.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.244+0000 7f9a919f1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a740035c0 0x7f9a8c19d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:34.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.244+0000 7f9a919f1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c19dd30 0x7f9a8c197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:34.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.244+0000 7f9a919f1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a8c197f40 0x7f9a8c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:34.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.244+0000 7f9a919f1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c108ec0 con 0x7f9a8c19dd30 2024-06-10T00:55:34.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.244+0000 7f9a919f1700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c108d40 con 0x7f9a8c197f40 2024-06-10T00:55:34.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.244+0000 7f9a919f1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c109040 con 0x7f9a740035c0 2024-06-10T00:55:34.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.245+0000 7f9a8affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a740035c0 0x7f9a8c19d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:34.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.245+0000 7f9a8affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a740035c0 0x7f9a8c19d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:49014/0 (socket says 172.21.15.175:49014) 2024-06-10T00:55:34.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.245+0000 7f9a8affd700 1 -- 172.21.15.175:0/1634490941 learned_addr learned my addr 172.21.15.175:0/1634490941 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-06-10T00:55:34.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.245+0000 7f9a8a7fc700 1 --2- 172.21.15.175:0/1634490941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c19dd30 0x7f9a8c197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:34.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.245+0000 7f9a8affd700 1 -- 172.21.15.175:0/1634490941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a740035c0 msgr2=0x7f9a8c19d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-06-10T00:55:34.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.245+0000 7f9a8affd700 1 -- 172.21.15.175:0/1634490941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a740035c0 msgr2=0x7f9a8c19d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-10T00:55:34.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.245+0000 7f9a8affd700 1 --2- 172.21.15.175:0/1634490941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a740035c0 0x7f9a8c19d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-10T00:55:34.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.245+0000 7f9a8b7fe700 1 --2- 172.21.15.175:0/1634490941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a8c197f40 0x7f9a8c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:34.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.245+0000 7f9a8affd700 1 --2- 172.21.15.175:0/1634490941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a740035c0 0x7f9a8c19d7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T00:55:34.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.245+0000 7f9a8b7fe700 1 -- 172.21.15.175:0/1634490941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a8c197f40 msgr2=0x7f9a8c1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-06-10T00:55:34.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.245+0000 7f9a8b7fe700 1 -- 172.21.15.175:0/1634490941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a8c197f40 msgr2=0x7f9a8c1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-10T00:55:34.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.245+0000 7f9a8b7fe700 1 --2- 172.21.15.175:0/1634490941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a8c197f40 0x7f9a8c1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-10T00:55:34.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.245+0000 7f9a8b7fe700 1 --2- 172.21.15.175:0/1634490941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a8c197f40 0x7f9a8c1983f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T00:55:34.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.245+0000 7f9a8a7fc700 1 -- 172.21.15.175:0/1634490941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a740035c0 msgr2=0x7f9a8c19d7f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T00:55:34.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.245+0000 7f9a8a7fc700 1 --2- 172.21.15.175:0/1634490941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a740035c0 0x7f9a8c19d7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:34.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.245+0000 7f9a8a7fc700 1 -- 172.21.15.175:0/1634490941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a8c197f40 msgr2=0x7f9a8c1983f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T00:55:34.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.245+0000 7f9a8a7fc700 1 --2- 172.21.15.175:0/1634490941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a8c197f40 0x7f9a8c1983f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:34.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.245+0000 7f9a8a7fc700 1 -- 172.21.15.175:0/1634490941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a8c1a46c0 con 0x7f9a8c19dd30 2024-06-10T00:55:34.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.245+0000 7f9a8a7fc700 1 --2- 172.21.15.175:0/1634490941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c19dd30 0x7f9a8c197a00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9a8000b550 tx=0x7f9a8000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:34.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.245+0000 7f9a909ef700 1 -- 172.21.15.175:0/1634490941 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a80004230 con 0x7f9a8c19dd30 2024-06-10T00:55:34.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.245+0000 7f9a909ef700 1 -- 172.21.15.175:0/1634490941 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a800043d0 con 0x7f9a8c19dd30 2024-06-10T00:55:34.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.245+0000 7f9a919f1700 1 -- 172.21.15.175:0/1634490941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a8c1a48f0 con 0x7f9a8c19dd30 2024-06-10T00:55:34.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.245+0000 7f9a909ef700 1 -- 172.21.15.175:0/1634490941 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a80011c20 con 0x7f9a8c19dd30 2024-06-10T00:55:34.248 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.245+0000 7f9a919f1700 1 -- 172.21.15.175:0/1634490941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a8c1a4e30 con 0x7f9a8c19dd30 2024-06-10T00:55:34.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.246+0000 7f9a909ef700 1 -- 172.21.15.175:0/1634490941 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f9a8001a400 con 0x7f9a8c19dd30 2024-06-10T00:55:34.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.246+0000 7f9a919f1700 1 -- 172.21.15.175:0/1634490941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a8c063af0 con 0x7f9a8c19dd30 2024-06-10T00:55:34.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.247+0000 7f9a909ef700 1 --2- 172.21.15.175:0/1634490941 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9a78039580 0x7f9a7803ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:34.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.247+0000 7f9a909ef700 1 -- 172.21.15.175:0/1634490941 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9a8004bed0 con 0x7f9a8c19dd30 2024-06-10T00:55:34.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.247+0000 7f9a8affd700 1 --2- 172.21.15.175:0/1634490941 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9a78039580 0x7f9a7803ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:34.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.247+0000 7f9a8affd700 1 --2- 172.21.15.175:0/1634490941 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9a78039580 0x7f9a7803ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9a84009a30 tx=0x7f9a8400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:34.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.252+0000 7f9a909ef700 1 -- 172.21.15.175:0/1634490941 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9a8001a890 con 0x7f9a8c19dd30 2024-06-10T00:55:34.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:33 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:55:34.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:33 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:55:34.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:33 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:55:34.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:33 smithi007 ceph-mon[32398]: mon.a calling monitor election 2024-06-10T00:55:34.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:33 smithi007 ceph-mon[32398]: mon.c calling monitor election 2024-06-10T00:55:34.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:33 smithi007 ceph-mon[32398]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:34.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:33 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:55:34.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:33 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:55:34.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:33 smithi007 ceph-mon[32398]: mon.b calling monitor election 2024-06-10T00:55:34.270 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:33 smithi007 ceph-mon[32398]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:34.270 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:33 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:55:34.270 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:33 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:55:34.270 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:33 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:55:34.270 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:33 smithi007 ceph-mon[32398]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-10T00:55:34.270 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:33 smithi007 ceph-mon[32398]: monmap epoch 3 2024-06-10T00:55:34.270 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:33 smithi007 ceph-mon[32398]: fsid bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T00:55:34.270 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:33 smithi007 ceph-mon[32398]: last_changed 2024-06-10T00:55:28.780516+0000 2024-06-10T00:55:34.270 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:33 smithi007 ceph-mon[32398]: created 2024-06-10T00:53:26.101611+0000 2024-06-10T00:55:34.270 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:33 smithi007 ceph-mon[32398]: min_mon_release 19 (squid) 2024-06-10T00:55:34.270 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:33 smithi007 ceph-mon[32398]: election_strategy: 1 2024-06-10T00:55:34.270 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:33 smithi007 ceph-mon[32398]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-06-10T00:55:34.270 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:33 smithi007 ceph-mon[32398]: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.c 2024-06-10T00:55:34.271 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:33 smithi007 ceph-mon[32398]: 2: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.b 2024-06-10T00:55:34.271 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:33 smithi007 ceph-mon[32398]: fsmap 2024-06-10T00:55:34.271 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:33 smithi007 ceph-mon[32398]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T00:55:34.271 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:33 smithi007 ceph-mon[32398]: mgrmap e13: x(active, since 58s) 2024-06-10T00:55:34.271 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:33 smithi007 ceph-mon[32398]: overall HEALTH_OK 2024-06-10T00:55:34.271 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:33 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:34.271 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:33 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:34.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:33 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:55:34.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:33 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:55:34.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:33 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:55:34.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:33 smithi078 ceph-mon[36683]: mon.a calling monitor election 2024-06-10T00:55:34.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:33 smithi078 ceph-mon[36683]: mon.c calling monitor election 2024-06-10T00:55:34.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:33 smithi078 ceph-mon[36683]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:34.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:33 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:55:34.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:33 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:55:34.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:33 smithi078 ceph-mon[36683]: mon.b calling monitor election 2024-06-10T00:55:34.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:33 smithi078 ceph-mon[36683]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:34.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:33 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:55:34.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:33 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:55:34.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:33 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:55:34.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:33 smithi078 ceph-mon[36683]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-10T00:55:34.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:33 smithi078 ceph-mon[36683]: monmap epoch 3 2024-06-10T00:55:34.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:33 smithi078 ceph-mon[36683]: fsid bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T00:55:34.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:33 smithi078 ceph-mon[36683]: last_changed 2024-06-10T00:55:28.780516+0000 2024-06-10T00:55:34.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:33 smithi078 ceph-mon[36683]: created 2024-06-10T00:53:26.101611+0000 2024-06-10T00:55:34.337 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:33 smithi078 ceph-mon[36683]: min_mon_release 19 (squid) 2024-06-10T00:55:34.337 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:33 smithi078 ceph-mon[36683]: election_strategy: 1 2024-06-10T00:55:34.337 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:33 smithi078 ceph-mon[36683]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-06-10T00:55:34.337 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:33 smithi078 ceph-mon[36683]: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.c 2024-06-10T00:55:34.337 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:33 smithi078 ceph-mon[36683]: 2: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.b 2024-06-10T00:55:34.337 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:33 smithi078 ceph-mon[36683]: fsmap 2024-06-10T00:55:34.337 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:33 smithi078 ceph-mon[36683]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T00:55:34.337 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:33 smithi078 ceph-mon[36683]: mgrmap e13: x(active, since 58s) 2024-06-10T00:55:34.337 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:33 smithi078 ceph-mon[36683]: overall HEALTH_OK 2024-06-10T00:55:34.337 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:33 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:34.337 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:33 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:34.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.545+0000 7f9a919f1700 1 -- 172.21.15.175:0/1634490941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9a8c02dca0 con 0x7f9a8c19dd30 2024-06-10T00:55:34.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.546+0000 7f9a909ef700 1 -- 172.21.15.175:0/1634490941 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1277 (secure 0 0 0) 0x7f9a8004d030 con 0x7f9a8c19dd30 2024-06-10T00:55:34.547 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-10T00:55:34.547 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":3,"fsid":"bded58f8-26c3-11ef-bc9b-c7b262605968","modified":"2024-06-10T00:55:28.780516Z","created":"2024-06-10T00:53:26.101611Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:3300","nonce":0},{"type":"v1","addr":"172.21.15.175:6789","nonce":0}]},"addr":"172.21.15.175:6789/0","public_addr":"172.21.15.175:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:3300","nonce":0},{"type":"v1","addr":"172.21.15.78:6789","nonce":0}]},"addr":"172.21.15.78:6789/0","public_addr":"172.21.15.78:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-06-10T00:55:34.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.548+0000 7f9a919f1700 1 -- 172.21.15.175:0/1634490941 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9a78039580 msgr2=0x7f9a7803ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:34.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.548+0000 7f9a919f1700 1 --2- 172.21.15.175:0/1634490941 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9a78039580 0x7f9a7803ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9a84009a30 tx=0x7f9a8400b040 comp rx=0 tx=0).stop 2024-06-10T00:55:34.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.548+0000 7f9a919f1700 1 -- 172.21.15.175:0/1634490941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c19dd30 msgr2=0x7f9a8c197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:34.550 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.548+0000 7f9a919f1700 1 --2- 172.21.15.175:0/1634490941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c19dd30 0x7f9a8c197a00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9a8000b550 tx=0x7f9a8000b910 comp rx=0 tx=0).stop 2024-06-10T00:55:34.550 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.549+0000 7f9a919f1700 1 -- 172.21.15.175:0/1634490941 shutdown_connections 2024-06-10T00:55:34.550 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.549+0000 7f9a919f1700 1 --2- 172.21.15.175:0/1634490941 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9a78039580 0x7f9a7803ba40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:34.550 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.549+0000 7f9a919f1700 1 --2- 172.21.15.175:0/1634490941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a8c197f40 0x7f9a8c1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:34.550 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.549+0000 7f9a919f1700 1 --2- 172.21.15.175:0/1634490941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a740035c0 0x7f9a8c19d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:34.550 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.549+0000 7f9a919f1700 1 --2- 172.21.15.175:0/1634490941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c19dd30 0x7f9a8c197a00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:34.550 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.549+0000 7f9a919f1700 1 -- 172.21.15.175:0/1634490941 >> 172.21.15.175:0/1634490941 conn(0x7f9a8c0fc970 msgr2=0x7f9a8c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:34.550 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.549+0000 7f9a919f1700 1 -- 172.21.15.175:0/1634490941 shutdown_connections 2024-06-10T00:55:34.550 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:34.549+0000 7f9a919f1700 1 -- 172.21.15.175:0/1634490941 wait complete. 2024-06-10T00:55:34.550 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 3 2024-06-10T00:55:35.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:34 smithi078 ceph-mon[36683]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:35.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:34 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:35.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:34 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:35.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:34 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:55:35.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:34 smithi078 ceph-mon[36683]: from='client.? 172.21.15.175:0/1634490941' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-10T00:55:35.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:34 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:55:35.134 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:34 smithi175 ceph-mon[36636]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:35.134 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:34 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:35.134 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:34 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:35.134 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:34 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:55:35.134 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:34 smithi175 ceph-mon[36636]: from='client.? 172.21.15.175:0/1634490941' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-10T00:55:35.134 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:34 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:55:35.160 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-06-10T00:55:35.161 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph config generate-minimal-conf 2024-06-10T00:55:35.190 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:34 smithi007 ceph-mon[32398]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:35.190 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:34 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:35.190 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:34 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:35.190 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:34 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:55:35.190 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:34 smithi007 ceph-mon[32398]: from='client.? 172.21.15.175:0/1634490941' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-10T00:55:35.190 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:34 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:55:35.418 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:55:36.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:36.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:36.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:36.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:36.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:36.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:36.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:55:36.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:36.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:36.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:36.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:36.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:36.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:36.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:55:36.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:36.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:36.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:36.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:36.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:36.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:36.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:55:37.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.043+0000 7fcab0433700 1 -- 172.21.15.7:0/1842923700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcaa8101040 msgr2=0x7fcaa8101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.043+0000 7fcab0433700 1 --2- 172.21.15.7:0/1842923700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcaa8101040 0x7fcaa8101420 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009910 tx=0x7fcaa4009c20 comp rx=0 tx=0).stop 2024-06-10T00:55:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.043+0000 7fcab0433700 1 -- 172.21.15.7:0/1842923700 shutdown_connections 2024-06-10T00:55:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.043+0000 7fcab0433700 1 --2- 172.21.15.7:0/1842923700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcaa8101040 0x7fcaa8101420 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.043+0000 7fcab0433700 1 -- 172.21.15.7:0/1842923700 >> 172.21.15.7:0/1842923700 conn(0x7fcaa80fc8f0 msgr2=0x7fcaa80fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.043+0000 7fcab0433700 1 -- 172.21.15.7:0/1842923700 shutdown_connections 2024-06-10T00:55:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.043+0000 7fcab0433700 1 -- 172.21.15.7:0/1842923700 wait complete. 2024-06-10T00:55:37.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.044+0000 7fcab0433700 1 Processor -- start 2024-06-10T00:55:37.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.044+0000 7fcab0433700 1 -- start start 2024-06-10T00:55:37.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.044+0000 7fcab0433700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcaa8101040 0x7fcaa819d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:37.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.044+0000 7fcab0433700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcaa819de10 0x7fcaa8197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:37.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.044+0000 7fcab0433700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa8198020 0x7fcaa81984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:37.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.044+0000 7fcab0433700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa810b410 con 0x7fcaa8101040 2024-06-10T00:55:37.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.044+0000 7fcab0433700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa810b290 con 0x7fcaa8198020 2024-06-10T00:55:37.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.044+0000 7fcab0433700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa810b590 con 0x7fcaa819de10 2024-06-10T00:55:37.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.044+0000 7fcaae1cf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcaa8101040 0x7fcaa819d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:37.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.044+0000 7fcaae1cf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcaa8101040 0x7fcaa819d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58958/0 (socket says 172.21.15.7:58958) 2024-06-10T00:55:37.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.044+0000 7fcaae1cf700 1 -- 172.21.15.7:0/3309032326 learned_addr learned my addr 172.21.15.7:0/3309032326 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:55:37.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.045+0000 7fcaad9ce700 1 --2- 172.21.15.7:0/3309032326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcaa819de10 0x7fcaa8197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:37.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.045+0000 7fcaae9d0700 1 --2- 172.21.15.7:0/3309032326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa8198020 0x7fcaa81984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:37.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.045+0000 7fcaae1cf700 1 -- 172.21.15.7:0/3309032326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcaa819de10 msgr2=0x7fcaa8197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:37.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.045+0000 7fcaae1cf700 1 --2- 172.21.15.7:0/3309032326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcaa819de10 0x7fcaa8197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:37.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.045+0000 7fcaae1cf700 1 -- 172.21.15.7:0/3309032326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa8198020 msgr2=0x7fcaa81984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:37.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.045+0000 7fcaae1cf700 1 --2- 172.21.15.7:0/3309032326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa8198020 0x7fcaa81984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:37.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.045+0000 7fcaae1cf700 1 -- 172.21.15.7:0/3309032326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa81a47a0 con 0x7fcaa8101040 2024-06-10T00:55:37.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.045+0000 7fcaae9d0700 1 --2- 172.21.15.7:0/3309032326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa8198020 0x7fcaa81984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:55:37.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.045+0000 7fcaae1cf700 1 --2- 172.21.15.7:0/3309032326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcaa8101040 0x7fcaa819d8d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4005850 tx=0x7fcaa4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:37.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.045+0000 7fca9f7fe700 1 -- 172.21.15.7:0/3309032326 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4026070 con 0x7fcaa8101040 2024-06-10T00:55:37.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.045+0000 7fcab0433700 1 -- 172.21.15.7:0/3309032326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa81a49d0 con 0x7fcaa8101040 2024-06-10T00:55:37.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.045+0000 7fcab0433700 1 -- 172.21.15.7:0/3309032326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa81a4f60 con 0x7fcaa8101040 2024-06-10T00:55:37.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.045+0000 7fca9f7fe700 1 -- 172.21.15.7:0/3309032326 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaa400b6f0 con 0x7fcaa8101040 2024-06-10T00:55:37.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.045+0000 7fca9f7fe700 1 -- 172.21.15.7:0/3309032326 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4003a00 con 0x7fcaa8101040 2024-06-10T00:55:37.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.046+0000 7fca9f7fe700 1 -- 172.21.15.7:0/3309032326 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fcaa4029070 con 0x7fcaa8101040 2024-06-10T00:55:37.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.046+0000 7fcab0433700 1 -- 172.21.15.7:0/3309032326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca8c005330 con 0x7fcaa8101040 2024-06-10T00:55:37.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.046+0000 7fca9f7fe700 1 --2- 172.21.15.7:0/3309032326 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fca94039580 0x7fca9403ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:37.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.047+0000 7fca9f7fe700 1 -- 172.21.15.7:0/3309032326 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcaa404cd40 con 0x7fcaa8101040 2024-06-10T00:55:37.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.047+0000 7fcaad9ce700 1 --2- 172.21.15.7:0/3309032326 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fca94039580 0x7fca9403ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:37.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.047+0000 7fcaad9ce700 1 --2- 172.21.15.7:0/3309032326 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fca94039580 0x7fca9403ba40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fca98009850 tx=0x7fca98008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:37.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.053+0000 7fca9f7fe700 1 -- 172.21.15.7:0/3309032326 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcaa4013070 con 0x7fcaa8101040 2024-06-10T00:55:37.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.265+0000 7fcab0433700 1 -- 172.21.15.7:0/3309032326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fca8c005620 con 0x7fcaa8101040 2024-06-10T00:55:37.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.266+0000 7fca9f7fe700 1 -- 172.21.15.7:0/3309032326 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+271 (secure 0 0 0) 0x7fcaa4030070 con 0x7fcaa8101040 2024-06-10T00:55:37.267 INFO:teuthology.orchestra.run.smithi007.stdout:# minimal ceph.conf for bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T00:55:37.267 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2024-06-10T00:55:37.267 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T00:55:37.267 INFO:teuthology.orchestra.run.smithi007.stdout: mon_host = [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2024-06-10T00:55:37.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.268+0000 7fcab0433700 1 -- 172.21.15.7:0/3309032326 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fca94039580 msgr2=0x7fca9403ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:37.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.268+0000 7fcab0433700 1 --2- 172.21.15.7:0/3309032326 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fca94039580 0x7fca9403ba40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fca98009850 tx=0x7fca98008040 comp rx=0 tx=0).stop 2024-06-10T00:55:37.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.268+0000 7fcab0433700 1 -- 172.21.15.7:0/3309032326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcaa8101040 msgr2=0x7fcaa819d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:37.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.268+0000 7fcab0433700 1 --2- 172.21.15.7:0/3309032326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcaa8101040 0x7fcaa819d8d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4005850 tx=0x7fcaa4005030 comp rx=0 tx=0).stop 2024-06-10T00:55:37.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.268+0000 7fcab0433700 1 -- 172.21.15.7:0/3309032326 shutdown_connections 2024-06-10T00:55:37.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.268+0000 7fcab0433700 1 --2- 172.21.15.7:0/3309032326 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fca94039580 0x7fca9403ba40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:37.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.268+0000 7fcab0433700 1 --2- 172.21.15.7:0/3309032326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa8198020 0x7fcaa81984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:37.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.268+0000 7fcab0433700 1 --2- 172.21.15.7:0/3309032326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcaa8101040 0x7fcaa819d8d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:37.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.268+0000 7fcab0433700 1 --2- 172.21.15.7:0/3309032326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcaa819de10 0x7fcaa8197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:37.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.268+0000 7fcab0433700 1 -- 172.21.15.7:0/3309032326 >> 172.21.15.7:0/3309032326 conn(0x7fcaa80fc8f0 msgr2=0x7fcaa806a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:37.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.268+0000 7fcab0433700 1 -- 172.21.15.7:0/3309032326 shutdown_connections 2024-06-10T00:55:37.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:37.268+0000 7fcab0433700 1 -- 172.21.15.7:0/3309032326 wait complete. 2024-06-10T00:55:37.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:37 smithi175 ceph-mon[36636]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:37.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:37 smithi175 ceph-mon[36636]: Updating smithi007:/etc/ceph/ceph.conf 2024-06-10T00:55:37.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:37 smithi175 ceph-mon[36636]: Updating smithi078:/etc/ceph/ceph.conf 2024-06-10T00:55:37.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:37 smithi175 ceph-mon[36636]: Updating smithi175:/etc/ceph/ceph.conf 2024-06-10T00:55:37.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:37 smithi175 ceph-mon[36636]: Updating smithi078:/var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:55:37.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:37 smithi175 ceph-mon[36636]: Updating smithi175:/var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:55:37.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:37 smithi175 ceph-mon[36636]: Updating smithi007:/var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:55:37.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:55:37.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T00:55:37.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:37.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:37 smithi007 ceph-mon[32398]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:37.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:37 smithi007 ceph-mon[32398]: Updating smithi007:/etc/ceph/ceph.conf 2024-06-10T00:55:37.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:37 smithi007 ceph-mon[32398]: Updating smithi078:/etc/ceph/ceph.conf 2024-06-10T00:55:37.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:37 smithi007 ceph-mon[32398]: Updating smithi175:/etc/ceph/ceph.conf 2024-06-10T00:55:37.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:37 smithi007 ceph-mon[32398]: Updating smithi078:/var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:55:37.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:37 smithi007 ceph-mon[32398]: Updating smithi175:/var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:55:37.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:37 smithi007 ceph-mon[32398]: Updating smithi007:/var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:55:37.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.520 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.520 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.520 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.520 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:55:37.520 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T00:55:37.520 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:37.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:37 smithi078 ceph-mon[36683]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:37.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:37 smithi078 ceph-mon[36683]: Updating smithi007:/etc/ceph/ceph.conf 2024-06-10T00:55:37.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:37 smithi078 ceph-mon[36683]: Updating smithi078:/etc/ceph/ceph.conf 2024-06-10T00:55:37.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:37 smithi078 ceph-mon[36683]: Updating smithi175:/etc/ceph/ceph.conf 2024-06-10T00:55:37.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:37 smithi078 ceph-mon[36683]: Updating smithi078:/var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:55:37.586 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:37 smithi078 ceph-mon[36683]: Updating smithi175:/var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:55:37.586 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:37 smithi078 ceph-mon[36683]: Updating smithi007:/var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:55:37.586 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.586 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.586 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.586 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.586 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.586 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.586 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.586 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.586 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.586 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.586 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:37.587 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:55:37.587 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T00:55:37.587 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:38.305 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-06-10T00:55:38.305 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-10T00:55:38.305 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.conf 2024-06-10T00:55:38.377 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-10T00:55:38.377 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:55:38.441 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:38 smithi007 ceph-mon[32398]: Reconfiguring mon.a (unknown last config time)... 2024-06-10T00:55:38.441 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:38 smithi007 ceph-mon[32398]: Reconfiguring daemon mon.a on smithi007 2024-06-10T00:55:38.441 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:38 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3309032326' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:38.445 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-06-10T00:55:38.445 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.conf 2024-06-10T00:55:38.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:38 smithi175 ceph-mon[36636]: Reconfiguring mon.a (unknown last config time)... 2024-06-10T00:55:38.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:38 smithi175 ceph-mon[36636]: Reconfiguring daemon mon.a on smithi007 2024-06-10T00:55:38.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:38 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3309032326' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:38.471 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:38 smithi078 ceph-mon[36683]: Reconfiguring mon.a (unknown last config time)... 2024-06-10T00:55:38.472 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:38 smithi078 ceph-mon[36683]: Reconfiguring daemon mon.a on smithi007 2024-06-10T00:55:38.472 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:38 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3309032326' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:38.477 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-06-10T00:55:38.477 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:55:38.544 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-06-10T00:55:38.545 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.conf 2024-06-10T00:55:38.576 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-06-10T00:55:38.576 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:55:38.644 INFO:tasks.cephadm:Adding mgr.x on smithi007 2024-06-10T00:55:38.644 INFO:tasks.cephadm:Adding mgr.y on smithi078 2024-06-10T00:55:38.644 INFO:tasks.cephadm:Adding mgr.z on smithi175 2024-06-10T00:55:38.644 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph orch apply mgr '3;smithi007=x;smithi078=y;smithi175=z' 2024-06-10T00:55:38.918 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:55:39.356 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:39 smithi175 ceph-mon[36636]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:39.356 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:39 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:39.356 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:39 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:39.356 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:39 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:55:39.356 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:39 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T00:55:39.356 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:39 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:39.480 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:39 smithi078 ceph-mon[36683]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:39.480 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:39 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:39.480 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:39 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:39.480 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:39 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:55:39.480 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:39 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T00:55:39.480 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:39 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:39.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:39 smithi007 ceph-mon[32398]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:39.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:39 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:39.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:39 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:39.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:39 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:55:39.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:39 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T00:55:39.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:39 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:40.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.305+0000 7fd77d1e9700 1 -- 172.21.15.175:0/458491028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd778100ea0 msgr2=0x7fd7781012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:40.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.305+0000 7fd77d1e9700 1 --2- 172.21.15.175:0/458491028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd778100ea0 0x7fd7781012c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd77000b0a0 tx=0x7fd77000b3b0 comp rx=0 tx=0).stop 2024-06-10T00:55:40.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.305+0000 7fd77d1e9700 1 -- 172.21.15.175:0/458491028 shutdown_connections 2024-06-10T00:55:40.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.305+0000 7fd77d1e9700 1 --2- 172.21.15.175:0/458491028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd778100ea0 0x7fd7781012c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:40.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.305+0000 7fd77d1e9700 1 -- 172.21.15.175:0/458491028 >> 172.21.15.175:0/458491028 conn(0x7fd7780fc490 msgr2=0x7fd7780fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:40.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.305+0000 7fd77d1e9700 1 -- 172.21.15.175:0/458491028 shutdown_connections 2024-06-10T00:55:40.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.305+0000 7fd77d1e9700 1 -- 172.21.15.175:0/458491028 wait complete. 2024-06-10T00:55:40.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.306+0000 7fd77d1e9700 1 Processor -- start 2024-06-10T00:55:40.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.306+0000 7fd77d1e9700 1 -- start start 2024-06-10T00:55:40.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.306+0000 7fd77d1e9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd778100ea0 0x7fd778197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:40.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.306+0000 7fd77d1e9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd778197da0 0x7fd77819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:40.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.306+0000 7fd77d1e9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd778198220 0x7fd77819f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:40.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.306+0000 7fd77d1e9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd778106e90 con 0x7fd778197da0 2024-06-10T00:55:40.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.306+0000 7fd77d1e9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd778106d10 con 0x7fd778100ea0 2024-06-10T00:55:40.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.306+0000 7fd77d1e9700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd778107010 con 0x7fd778198220 2024-06-10T00:55:40.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.306+0000 7fd77759e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd778198220 0x7fd77819f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:40.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.307+0000 7fd77759e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd778198220 0x7fd77819f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39320/0 (socket says 172.21.15.175:39320) 2024-06-10T00:55:40.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.307+0000 7fd77759e700 1 -- 172.21.15.175:0/2962279278 learned_addr learned my addr 172.21.15.175:0/2962279278 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-06-10T00:55:40.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.307+0000 7fd77659c700 1 --2- 172.21.15.175:0/2962279278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd778197da0 0x7fd77819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:40.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.307+0000 7fd776d9d700 1 --2- 172.21.15.175:0/2962279278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd778100ea0 0x7fd778197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:40.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.307+0000 7fd77759e700 1 -- 172.21.15.175:0/2962279278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd778100ea0 msgr2=0x7fd778197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:40.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.307+0000 7fd77759e700 1 --2- 172.21.15.175:0/2962279278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd778100ea0 0x7fd778197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:40.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.307+0000 7fd77759e700 1 -- 172.21.15.175:0/2962279278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd778197da0 msgr2=0x7fd77819ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:40.310 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.307+0000 7fd77759e700 1 --2- 172.21.15.175:0/2962279278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd778197da0 0x7fd77819ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:40.310 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.307+0000 7fd77759e700 1 -- 172.21.15.175:0/2962279278 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd77819fc70 con 0x7fd778198220 2024-06-10T00:55:40.310 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.307+0000 7fd77759e700 1 --2- 172.21.15.175:0/2962279278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd778198220 0x7fd77819f3a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd76800b550 tx=0x7fd76800b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:40.310 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.307+0000 7fd76ffff700 1 -- 172.21.15.175:0/2962279278 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd768015020 con 0x7fd778198220 2024-06-10T00:55:40.310 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.307+0000 7fd76ffff700 1 -- 172.21.15.175:0/2962279278 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd768011d10 con 0x7fd778198220 2024-06-10T00:55:40.310 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.307+0000 7fd77d1e9700 1 -- 172.21.15.175:0/2962279278 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd77819ff60 con 0x7fd778198220 2024-06-10T00:55:40.310 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.307+0000 7fd76ffff700 1 -- 172.21.15.175:0/2962279278 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd768010dd0 con 0x7fd778198220 2024-06-10T00:55:40.310 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.307+0000 7fd77d1e9700 1 -- 172.21.15.175:0/2962279278 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7781a0490 con 0x7fd778198220 2024-06-10T00:55:40.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.308+0000 7fd76ffff700 1 -- 172.21.15.175:0/2962279278 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fd768018020 con 0x7fd778198220 2024-06-10T00:55:40.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.308+0000 7fd77d1e9700 1 -- 172.21.15.175:0/2962279278 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd778063af0 con 0x7fd778198220 2024-06-10T00:55:40.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.308+0000 7fd76ffff700 1 --2- 172.21.15.175:0/2962279278 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd764039580 0x7fd76403ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:40.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.309+0000 7fd76ffff700 1 -- 172.21.15.175:0/2962279278 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd76804ba30 con 0x7fd778198220 2024-06-10T00:55:40.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.309+0000 7fd776d9d700 1 --2- 172.21.15.175:0/2962279278 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd764039580 0x7fd76403ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:40.311 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.309+0000 7fd776d9d700 1 --2- 172.21.15.175:0/2962279278 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd764039580 0x7fd76403ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd7700076a0 tx=0x7fd770005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:40.314 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.314+0000 7fd76ffff700 1 -- 172.21.15.175:0/2962279278 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd76801f020 con 0x7fd778198220 2024-06-10T00:55:40.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:40 smithi175 ceph-mon[36636]: Reconfiguring mon.b (monmap changed)... 2024-06-10T00:55:40.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:40 smithi175 ceph-mon[36636]: Reconfiguring daemon mon.b on smithi078 2024-06-10T00:55:40.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:40 smithi007 ceph-mon[32398]: Reconfiguring mon.b (monmap changed)... 2024-06-10T00:55:40.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:40 smithi007 ceph-mon[32398]: Reconfiguring daemon mon.b on smithi078 2024-06-10T00:55:40.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.532+0000 7fd77d1e9700 1 -- 172.21.15.175:0/2962279278 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi078=y;smithi175=z", "target": ["mon-mgr", ""]}) -- 0x7fd778062960 con 0x7fd764039580 2024-06-10T00:55:40.539 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.538+0000 7fd76ffff700 1 -- 172.21.15.175:0/2962279278 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd778062960 con 0x7fd764039580 2024-06-10T00:55:40.539 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled mgr update... 2024-06-10T00:55:40.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.541+0000 7fd77d1e9700 1 -- 172.21.15.175:0/2962279278 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd764039580 msgr2=0x7fd76403ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:40.542 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.541+0000 7fd77d1e9700 1 --2- 172.21.15.175:0/2962279278 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd764039580 0x7fd76403ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd7700076a0 tx=0x7fd770005040 comp rx=0 tx=0).stop 2024-06-10T00:55:40.542 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.541+0000 7fd77d1e9700 1 -- 172.21.15.175:0/2962279278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd778198220 msgr2=0x7fd77819f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:40.542 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.541+0000 7fd77d1e9700 1 --2- 172.21.15.175:0/2962279278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd778198220 0x7fd77819f3a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd76800b550 tx=0x7fd76800b860 comp rx=0 tx=0).stop 2024-06-10T00:55:40.542 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.541+0000 7fd77d1e9700 1 -- 172.21.15.175:0/2962279278 shutdown_connections 2024-06-10T00:55:40.542 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.541+0000 7fd77d1e9700 1 --2- 172.21.15.175:0/2962279278 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd764039580 0x7fd76403ba40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:40.542 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.541+0000 7fd77d1e9700 1 --2- 172.21.15.175:0/2962279278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd778198220 0x7fd77819f3a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:40.542 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.541+0000 7fd77d1e9700 1 --2- 172.21.15.175:0/2962279278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd778100ea0 0x7fd778197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:40.542 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.541+0000 7fd77d1e9700 1 --2- 172.21.15.175:0/2962279278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd778197da0 0x7fd77819ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:40.542 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.541+0000 7fd77d1e9700 1 -- 172.21.15.175:0/2962279278 >> 172.21.15.175:0/2962279278 conn(0x7fd7780fc490 msgr2=0x7fd77818e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:40.542 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.541+0000 7fd77d1e9700 1 -- 172.21.15.175:0/2962279278 shutdown_connections 2024-06-10T00:55:40.542 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:55:40.541+0000 7fd77d1e9700 1 -- 172.21.15.175:0/2962279278 wait complete. 2024-06-10T00:55:40.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:40 smithi078 ceph-mon[36683]: Reconfiguring mon.b (monmap changed)... 2024-06-10T00:55:40.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:40 smithi078 ceph-mon[36683]: Reconfiguring daemon mon.b on smithi078 2024-06-10T00:55:41.142 DEBUG:teuthology.orchestra.run.smithi078:mgr.y> sudo journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mgr.y.service 2024-06-10T00:55:41.145 DEBUG:teuthology.orchestra.run.smithi175:mgr.z> sudo journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mgr.z.service 2024-06-10T00:55:41.147 INFO:tasks.cephadm:Deploying OSDs... 2024-06-10T00:55:41.147 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-10T00:55:41.147 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/scratch_devs of=/dev/stdout 2024-06-10T00:55:41.167 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-10T00:55:41.168 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_1 2024-06-10T00:55:41.228 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-10T00:55:41.228 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:55:41.228 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-06-10T00:55:41.228 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:55:41.228 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:55:41.228 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-06-10 00:54:47.671829435 +0000 2024-06-10T00:55:41.228 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-06-10 00:49:50.313308694 +0000 2024-06-10T00:55:41.228 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-06-10 00:49:50.313308694 +0000 2024-06-10T00:55:41.228 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-06-10 00:49:50.313308694 +0000 2024-06-10T00:55:41.228 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-10T00:55:41.295 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-06-10T00:55:41.295 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-06-10T00:55:41.296 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000124808 s, 4.1 MB/s 2024-06-10T00:55:41.297 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-10T00:55:41.357 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_2 2024-06-10T00:55:41.376 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:41 smithi175 ceph-mon[36636]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:41.377 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:41 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:41.377 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:41 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:41.377 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:41 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:55:41.377 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:41 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T00:55:41.377 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:41 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:41.377 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:41 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:41.415 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-10T00:55:41.415 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:55:41.415 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-06-10T00:55:41.415 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:55:41.415 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:55:41.415 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-06-10 00:54:47.706828964 +0000 2024-06-10T00:55:41.415 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-06-10 00:49:50.646304099 +0000 2024-06-10T00:55:41.415 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-06-10 00:49:50.646304099 +0000 2024-06-10T00:55:41.416 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-06-10 00:49:50.646304099 +0000 2024-06-10T00:55:41.416 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-10T00:55:41.479 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:41 smithi007 ceph-mon[32398]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:41.479 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:41 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:41.479 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:41 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:41.479 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:41 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:55:41.479 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:41 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T00:55:41.479 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:41 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:41.479 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:41 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:41.481 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-06-10T00:55:41.481 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-06-10T00:55:41.481 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000102885 s, 5.0 MB/s 2024-06-10T00:55:41.482 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-10T00:55:41.541 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_3 2024-06-10T00:55:41.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:41 smithi078 ceph-mon[36683]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:41.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:41 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:41.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:41 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:41.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:41 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:55:41.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:41 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T00:55:41.586 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:41 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:41.586 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:41 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:41.598 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-10T00:55:41.598 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:55:41.598 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-06-10T00:55:41.598 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:55:41.598 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:55:41.598 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-06-10 00:54:47.740828507 +0000 2024-06-10T00:55:41.598 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-06-10 00:49:50.983299448 +0000 2024-06-10T00:55:41.599 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-06-10 00:49:50.983299448 +0000 2024-06-10T00:55:41.599 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-06-10 00:49:50.983299448 +0000 2024-06-10T00:55:41.599 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-10T00:55:41.663 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-06-10T00:55:41.663 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-06-10T00:55:41.664 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000114057 s, 4.5 MB/s 2024-06-10T00:55:41.665 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-10T00:55:41.722 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_4 2024-06-10T00:55:41.780 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-10T00:55:41.780 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:55:41.780 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-06-10T00:55:41.780 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:55:41.780 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:55:41.780 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-06-10 00:54:47.774828050 +0000 2024-06-10T00:55:41.780 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-06-10 00:49:51.315294866 +0000 2024-06-10T00:55:41.780 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-06-10 00:49:51.315294866 +0000 2024-06-10T00:55:41.780 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-06-10 00:49:51.315294866 +0000 2024-06-10T00:55:41.781 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-10T00:55:41.845 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-06-10T00:55:41.845 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-06-10T00:55:41.845 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000113567 s, 4.5 MB/s 2024-06-10T00:55:41.846 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-10T00:55:41.903 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-06-10T00:55:41.903 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/scratch_devs of=/dev/stdout 2024-06-10T00:55:41.922 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-10T00:55:41.922 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_1 2024-06-10T00:55:41.980 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-10T00:55:41.980 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:55:41.980 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 658 Links: 1 2024-06-10T00:55:41.980 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:55:41.980 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:55:41.980 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-06-10 00:55:28.274819277 +0000 2024-06-10T00:55:41.980 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-06-10 00:49:58.403110480 +0000 2024-06-10T00:55:41.980 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-06-10 00:49:58.403110480 +0000 2024-06-10T00:55:41.980 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-06-10 00:49:58.403110480 +0000 2024-06-10T00:55:41.981 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-10T00:55:42.047 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-06-10T00:55:42.047 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-06-10T00:55:42.047 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000214901 s, 2.4 MB/s 2024-06-10T00:55:42.048 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-10T00:55:42.106 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_2 2024-06-10T00:55:42.163 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-10T00:55:42.163 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:55:42.163 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 676 Links: 1 2024-06-10T00:55:42.163 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:55:42.163 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:55:42.163 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-06-10 00:55:28.275819263 +0000 2024-06-10T00:55:42.163 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-06-10 00:49:58.732105458 +0000 2024-06-10T00:55:42.164 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-06-10 00:49:58.732105458 +0000 2024-06-10T00:55:42.164 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-06-10 00:49:58.732105458 +0000 2024-06-10T00:55:42.164 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-10T00:55:42.230 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-06-10T00:55:42.230 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-06-10T00:55:42.230 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000224945 s, 2.3 MB/s 2024-06-10T00:55:42.230 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-10T00:55:42.287 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_3 2024-06-10T00:55:42.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:42 smithi078 ceph-mon[36683]: Reconfiguring mon.c (monmap changed)... 2024-06-10T00:55:42.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:42 smithi078 ceph-mon[36683]: Reconfiguring daemon mon.c on smithi175 2024-06-10T00:55:42.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:42 smithi078 ceph-mon[36683]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi078=y;smithi175=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:55:42.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:42 smithi078 ceph-mon[36683]: Saving service mgr spec with placement smithi007=x;smithi078=y;smithi175=z;count:3 2024-06-10T00:55:42.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:42 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:42.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:42 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:42.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:42 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:55:42.343 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-10T00:55:42.343 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:55:42.343 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 697 Links: 1 2024-06-10T00:55:42.343 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:55:42.343 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:55:42.343 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-06-10 00:55:28.275819263 +0000 2024-06-10T00:55:42.344 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-06-10 00:49:59.038100787 +0000 2024-06-10T00:55:42.344 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-06-10 00:49:59.038100787 +0000 2024-06-10T00:55:42.344 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-06-10 00:49:59.038100787 +0000 2024-06-10T00:55:42.344 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-10T00:55:42.407 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-06-10T00:55:42.407 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-06-10T00:55:42.407 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000193103 s, 2.7 MB/s 2024-06-10T00:55:42.408 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-10T00:55:42.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:42 smithi175 ceph-mon[36636]: Reconfiguring mon.c (monmap changed)... 2024-06-10T00:55:42.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:42 smithi175 ceph-mon[36636]: Reconfiguring daemon mon.c on smithi175 2024-06-10T00:55:42.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:42 smithi175 ceph-mon[36636]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi078=y;smithi175=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:55:42.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:42 smithi175 ceph-mon[36636]: Saving service mgr spec with placement smithi007=x;smithi078=y;smithi175=z;count:3 2024-06-10T00:55:42.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:42 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:42.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:42 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:42.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:42 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:55:42.464 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_4 2024-06-10T00:55:42.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:42 smithi007 ceph-mon[32398]: Reconfiguring mon.c (monmap changed)... 2024-06-10T00:55:42.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:42 smithi007 ceph-mon[32398]: Reconfiguring daemon mon.c on smithi175 2024-06-10T00:55:42.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:42 smithi007 ceph-mon[32398]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi078=y;smithi175=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:55:42.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:42 smithi007 ceph-mon[32398]: Saving service mgr spec with placement smithi007=x;smithi078=y;smithi175=z;count:3 2024-06-10T00:55:42.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:42 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:42.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:42 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:42.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:42 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:55:42.520 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-10T00:55:42.520 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:55:42.520 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 717 Links: 1 2024-06-10T00:55:42.520 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:55:42.520 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:55:42.520 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-06-10 00:55:28.275819263 +0000 2024-06-10T00:55:42.520 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-06-10 00:49:59.390095414 +0000 2024-06-10T00:55:42.520 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-06-10 00:49:59.390095414 +0000 2024-06-10T00:55:42.520 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-06-10 00:49:59.390095414 +0000 2024-06-10T00:55:42.520 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-10T00:55:42.582 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-06-10T00:55:42.583 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-06-10T00:55:42.583 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000216824 s, 2.4 MB/s 2024-06-10T00:55:42.584 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-10T00:55:42.641 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-06-10T00:55:42.642 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/scratch_devs of=/dev/stdout 2024-06-10T00:55:42.661 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-10T00:55:42.661 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_1 2024-06-10T00:55:42.719 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-10T00:55:42.719 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:55:42.719 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-06-10T00:55:42.719 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:55:42.719 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:55:42.719 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-06-10 00:55:18.937551762 +0000 2024-06-10T00:55:42.719 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-06-10 00:49:57.977223421 +0000 2024-06-10T00:55:42.719 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-06-10 00:49:57.977223421 +0000 2024-06-10T00:55:42.719 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-06-10 00:49:57.977223421 +0000 2024-06-10T00:55:42.720 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-10T00:55:42.784 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-06-10T00:55:42.784 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-06-10T00:55:42.785 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000279717 s, 1.8 MB/s 2024-06-10T00:55:42.786 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-10T00:55:42.843 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_2 2024-06-10T00:55:42.898 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-10T00:55:42.898 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:55:42.898 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-06-10T00:55:42.898 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:55:42.898 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:55:42.898 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-06-10 00:55:18.937551762 +0000 2024-06-10T00:55:42.898 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-06-10 00:49:58.313218365 +0000 2024-06-10T00:55:42.898 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-06-10 00:49:58.313218365 +0000 2024-06-10T00:55:42.899 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-06-10 00:49:58.313218365 +0000 2024-06-10T00:55:42.899 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-10T00:55:42.960 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-06-10T00:55:42.960 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-06-10T00:55:42.960 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000244713 s, 2.1 MB/s 2024-06-10T00:55:42.962 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-10T00:55:43.017 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_3 2024-06-10T00:55:43.072 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-10T00:55:43.072 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:55:43.072 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-06-10T00:55:43.072 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:55:43.072 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:55:43.072 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-06-10 00:55:18.937551762 +0000 2024-06-10T00:55:43.072 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-06-10 00:49:58.662213114 +0000 2024-06-10T00:55:43.072 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-06-10 00:49:58.662213114 +0000 2024-06-10T00:55:43.073 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-06-10 00:49:58.662213114 +0000 2024-06-10T00:55:43.073 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-10T00:55:43.134 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-06-10T00:55:43.134 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-06-10T00:55:43.134 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000207634 s, 2.5 MB/s 2024-06-10T00:55:43.136 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-10T00:55:43.193 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_4 2024-06-10T00:55:43.250 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-10T00:55:43.250 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:55:43.250 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-06-10T00:55:43.250 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:55:43.250 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:55:43.250 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-06-10 00:55:18.937551762 +0000 2024-06-10T00:55:43.251 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-06-10 00:49:59.006207938 +0000 2024-06-10T00:55:43.251 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-06-10 00:49:59.006207938 +0000 2024-06-10T00:55:43.251 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-06-10 00:49:59.006207938 +0000 2024-06-10T00:55:43.251 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-10T00:55:43.313 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-06-10T00:55:43.313 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-06-10T00:55:43.313 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000173094 s, 3.0 MB/s 2024-06-10T00:55:43.314 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-10T00:55:43.369 INFO:tasks.cephadm:Deploying osd.0 on smithi007 with /dev/vg_nvme/lv_4... 2024-06-10T00:55:43.369 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-10T00:55:43.390 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:43 smithi175 ceph-mon[36636]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:43.390 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:43 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:43.390 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:43 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:55:43.390 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:43 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:43.390 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:43 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T00:55:43.390 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:43 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-10T00:55:43.391 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:43 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T00:55:43.391 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:43 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:43.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:43 smithi007 ceph-mon[32398]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:43.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:43 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:43.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:43 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:55:43.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:43 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:43.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:43 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T00:55:43.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:43 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-10T00:55:43.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:43 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T00:55:43.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:43 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:43.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:43 smithi078 ceph-mon[36683]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:43.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:43 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:43.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:43 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:55:43.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:43 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:43.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:43 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T00:55:43.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:43 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-10T00:55:43.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:43 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T00:55:43.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:43 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:43.623 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:55:44.401 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:44 smithi007 ceph-mon[32398]: Deploying daemon mgr.z on smithi175 2024-06-10T00:55:44.410 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:44 smithi175 ceph-mon[36636]: Deploying daemon mgr.z on smithi175 2024-06-10T00:55:44.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:44 smithi078 ceph-mon[36683]: Deploying daemon mgr.z on smithi175 2024-06-10T00:55:44.709 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:44 smithi175 systemd[1]: Starting Ceph mgr.z for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T00:55:45.092 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:44 smithi175 podman[38459]: 2024-06-10 00:55:44.688428961 +0000 UTC m=+0.017367055 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:55:45.092 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:44 smithi175 podman[38459]: 2024-06-10 00:55:44.798029641 +0000 UTC m=+0.126967719 container create 69ef91707a6fcf7645c2b37a125af7c2d90adbc7236287c0f6fba84543229fa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-06-10T00:55:45.092 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:44 smithi175 podman[38459]: 2024-06-10 00:55:44.992140792 +0000 UTC m=+0.321078871 container init 69ef91707a6fcf7645c2b37a125af7c2d90adbc7236287c0f6fba84543229fa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc) 2024-06-10T00:55:45.092 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:44 smithi175 podman[38459]: 2024-06-10 00:55:44.994606109 +0000 UTC m=+0.323544202 container start 69ef91707a6fcf7645c2b37a125af7c2d90adbc7236287c0f6fba84543229fa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_CLEAN=True, RELEASE=squid-3d89acc) 2024-06-10T00:55:45.092 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:45 smithi175 ceph-mgr[38478]: -- 172.21.15.175:0/2315452283 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563b89d605a0 con 0x563b89d34800 2024-06-10T00:55:45.092 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:45 smithi175 bash[38459]: 69ef91707a6fcf7645c2b37a125af7c2d90adbc7236287c0f6fba84543229fa8 2024-06-10T00:55:45.433 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:45 smithi175 systemd[1]: Started Ceph mgr.z for bded58f8-26c3-11ef-bc9b-c7b262605968. 2024-06-10T00:55:45.433 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:45 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-z[38474]: 2024-06-10T00:55:45.146+0000 7f69aac00200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-10T00:55:45.433 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:45 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-z[38474]: 2024-06-10T00:55:45.332+0000 7f69aac00200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:55:45.434 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:45 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-z[38474]: 2024-06-10T00:55:45.433+0000 7f69aac00200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-10T00:55:45.434 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:45 smithi175 ceph-mon[36636]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:45.434 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:45 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:45.434 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:45 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:45.434 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:45 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:45.434 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:45 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T00:55:45.434 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:45 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-10T00:55:45.434 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:45 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T00:55:45.434 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:45 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:45.478 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-10T00:55:45.478 INFO:teuthology.orchestra.run.smithi007.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-10T00:55:45.478 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10+0 records in 2024-06-10T00:55:45.478 INFO:teuthology.orchestra.run.smithi007.stderr:10+0 records out 2024-06-10T00:55:45.478 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00862466 s, 1.2 GB/s 2024-06-10T00:55:45.478 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping successful for: 2024-06-10T00:55:45.478 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:45 smithi007 ceph-mon[32398]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:45.478 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:45 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:45.478 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:45 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:45.479 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:45 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:45.479 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:45 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T00:55:45.479 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:45 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-10T00:55:45.479 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:45 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T00:55:45.479 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:45 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:45.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:45 smithi078 ceph-mon[36683]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:45.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:45 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:45.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:45 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:45.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:45 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:45.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:45 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T00:55:45.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:45 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-10T00:55:45.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:45 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T00:55:45.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:45 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:46.041 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi007:vg_nvme/lv_4 2024-06-10T00:55:46.172 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:45 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-z[38474]: 2024-06-10T00:55:45.742+0000 7f69aac00200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-10T00:55:46.173 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:45 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-z[38474]: 2024-06-10T00:55:45.828+0000 7f69aac00200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-10T00:55:46.291 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:55:46.392 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:46 smithi007 ceph-mon[32398]: Deploying daemon mgr.y on smithi078 2024-06-10T00:55:46.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:46 smithi175 ceph-mon[36636]: Deploying daemon mgr.y on smithi078 2024-06-10T00:55:46.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:46 smithi078 ceph-mon[36683]: Deploying daemon mgr.y on smithi078 2024-06-10T00:55:47.205 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:47 smithi078 systemd[1]: Starting Ceph mgr.y for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T00:55:47.418 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:47 smithi175 ceph-mon[36636]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:47.418 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:47 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-z[38474]: 2024-06-10T00:55:47.073+0000 7f69aac00200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-10T00:55:47.418 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:47 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-z[38474]: 2024-06-10T00:55:47.247+0000 7f69aac00200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-10T00:55:47.480 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:47 smithi007 ceph-mon[32398]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:47.533 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:47 smithi078 ceph-mon[36683]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:47.534 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:47 smithi078 podman[38041]: 2024-06-10 00:55:47.230752713 +0000 UTC m=+0.017445821 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:55:47.534 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:47 smithi078 podman[38041]: 2024-06-10 00:55:47.33203656 +0000 UTC m=+0.118729657 container create c999d5b9e20f1e337efc7fd2032be588c5196fcbb98d32b396d7acceaf375197 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-y, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-06-10T00:55:47.534 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:47 smithi078 podman[38041]: 2024-06-10 00:55:47.534218441 +0000 UTC m=+0.320911545 container init c999d5b9e20f1e337efc7fd2032be588c5196fcbb98d32b396d7acceaf375197 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, ceph=True) 2024-06-10T00:55:47.708 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:47 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-z[38474]: 2024-06-10T00:55:47.424+0000 7f69aac00200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-10T00:55:47.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.712+0000 7f4745473700 1 -- 172.21.15.7:0/3396145934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47401024f0 msgr2=0x7f4740102910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:47.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.712+0000 7f4745473700 1 --2- 172.21.15.7:0/3396145934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47401024f0 0x7f4740102910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f47300099c0 tx=0x7f4730009cd0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.712+0000 7f4745473700 1 -- 172.21.15.7:0/3396145934 shutdown_connections 2024-06-10T00:55:47.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.712+0000 7f4745473700 1 --2- 172.21.15.7:0/3396145934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4740104090 0x7f474010a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.712+0000 7f4745473700 1 --2- 172.21.15.7:0/3396145934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47401024f0 0x7f4740102910 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.712+0000 7f4745473700 1 --2- 172.21.15.7:0/3396145934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47401036f0 0x7f4740103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.712+0000 7f4745473700 1 -- 172.21.15.7:0/3396145934 >> 172.21.15.7:0/3396145934 conn(0x7f47400fda90 msgr2=0x7f47400ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:47.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.712+0000 7f4745473700 1 -- 172.21.15.7:0/3396145934 shutdown_connections 2024-06-10T00:55:47.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.712+0000 7f4745473700 1 -- 172.21.15.7:0/3396145934 wait complete. 2024-06-10T00:55:47.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.712+0000 7f4745473700 1 Processor -- start 2024-06-10T00:55:47.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.713+0000 7f4745473700 1 -- start start 2024-06-10T00:55:47.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.713+0000 7f4745473700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47401024f0 0x7f474019a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:47.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.713+0000 7f4745473700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47401036f0 0x7f474019aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:47.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.713+0000 7f4745473700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4740104090 0x7f47401a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:47.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.713+0000 7f4745473700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f474010e0f0 con 0x7f47401036f0 2024-06-10T00:55:47.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.713+0000 7f4745473700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f474010df70 con 0x7f47401024f0 2024-06-10T00:55:47.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.713+0000 7f4745473700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f474010e270 con 0x7f4740104090 2024-06-10T00:55:47.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.713+0000 7f473e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47401036f0 0x7f474019aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:47.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.713+0000 7f473effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47401024f0 0x7f474019a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:47.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.713+0000 7f473e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47401036f0 0x7f474019aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41692/0 (socket says 172.21.15.7:41692) 2024-06-10T00:55:47.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.713+0000 7f473e7fc700 1 -- 172.21.15.7:0/745911744 learned_addr learned my addr 172.21.15.7:0/745911744 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:55:47.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.713+0000 7f473f7fe700 1 --2- 172.21.15.7:0/745911744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4740104090 0x7f47401a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:47.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.713+0000 7f473effd700 1 -- 172.21.15.7:0/745911744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4740104090 msgr2=0x7f47401a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:47.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.713+0000 7f473effd700 1 --2- 172.21.15.7:0/745911744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4740104090 0x7f47401a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.713+0000 7f473effd700 1 -- 172.21.15.7:0/745911744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47401036f0 msgr2=0x7f474019aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:47.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.713+0000 7f473effd700 1 --2- 172.21.15.7:0/745911744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47401036f0 0x7f474019aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.713+0000 7f473effd700 1 -- 172.21.15.7:0/745911744 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47401a25d0 con 0x7f47401024f0 2024-06-10T00:55:47.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.713+0000 7f473e7fc700 1 --2- 172.21.15.7:0/745911744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47401036f0 0x7f474019aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:55:47.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.713+0000 7f473effd700 1 --2- 172.21.15.7:0/745911744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47401024f0 0x7f474019a970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4730003790 tx=0x7f4730009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:47.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.714+0000 7f4727fff700 1 -- 172.21.15.7:0/745911744 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4730009e60 con 0x7f47401024f0 2024-06-10T00:55:47.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.714+0000 7f4745473700 1 -- 172.21.15.7:0/745911744 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47401a2860 con 0x7f47401024f0 2024-06-10T00:55:47.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.714+0000 7f4727fff700 1 -- 172.21.15.7:0/745911744 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4730003960 con 0x7f47401024f0 2024-06-10T00:55:47.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.714+0000 7f4727fff700 1 -- 172.21.15.7:0/745911744 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4730005560 con 0x7f47401024f0 2024-06-10T00:55:47.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.714+0000 7f4745473700 1 -- 172.21.15.7:0/745911744 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47401a2d40 con 0x7f47401024f0 2024-06-10T00:55:47.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.714+0000 7f4727fff700 1 -- 172.21.15.7:0/745911744 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f473002b440 con 0x7f47401024f0 2024-06-10T00:55:47.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.715+0000 7f4745473700 1 -- 172.21.15.7:0/745911744 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4740069180 con 0x7f47401024f0 2024-06-10T00:55:47.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.715+0000 7f4727fff700 1 --2- 172.21.15.7:0/745911744 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4728039330 0x7f472803b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:47.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.715+0000 7f4727fff700 1 -- 172.21.15.7:0/745911744 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f473004d300 con 0x7f47401024f0 2024-06-10T00:55:47.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.715+0000 7f473e7fc700 1 --2- 172.21.15.7:0/745911744 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4728039330 0x7f472803b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:47.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.715+0000 7f473e7fc700 1 --2- 172.21.15.7:0/745911744 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4728039330 0x7f472803b7f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4734011590 tx=0x7f4734009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:47.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.720+0000 7f4727fff700 1 -- 172.21.15.7:0/745911744 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4730013070 con 0x7f47401024f0 2024-06-10T00:55:47.835 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:47 smithi078 podman[38041]: 2024-06-10 00:55:47.537591644 +0000 UTC m=+0.324284757 container start c999d5b9e20f1e337efc7fd2032be588c5196fcbb98d32b396d7acceaf375197 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-y, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T00:55:47.835 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:47 smithi078 ceph-mgr[38060]: -- 172.21.15.78:0/3034170605 <== mon.2 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556abb2705a0 con 0x556abb244c00 2024-06-10T00:55:47.835 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:47 smithi078 bash[38041]: c999d5b9e20f1e337efc7fd2032be588c5196fcbb98d32b396d7acceaf375197 2024-06-10T00:55:47.835 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:47 smithi078 systemd[1]: Started Ceph mgr.y for bded58f8-26c3-11ef-bc9b-c7b262605968. 2024-06-10T00:55:47.835 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:47 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-y[38056]: 2024-06-10T00:55:47.689+0000 7f9620bbb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-10T00:55:47.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:47.938+0000 7f4745473700 1 -- 172.21.15.7:0/745911744 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f4740075de0 con 0x7f4728039330 2024-06-10T00:55:48.018 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:47 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-z[38474]: 2024-06-10T00:55:47.713+0000 7f69aac00200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-10T00:55:48.018 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:47 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-z[38474]: 2024-06-10T00:55:47.807+0000 7f69aac00200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-10T00:55:48.018 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:48 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-z[38474]: 2024-06-10T00:55:48.017+0000 7f69aac00200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-10T00:55:48.307 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:47 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-y[38056]: 2024-06-10T00:55:47.877+0000 7f9620bbb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:55:48.307 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:47 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-y[38056]: 2024-06-10T00:55:47.983+0000 7f9620bbb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-10T00:55:48.307 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:48 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-y[38056]: 2024-06-10T00:55:48.306+0000 7f9620bbb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-10T00:55:48.458 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:48 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-z[38474]: 2024-06-10T00:55:48.217+0000 7f69aac00200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:55:48.585 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:48 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-y[38056]: 2024-06-10T00:55:48.398+0000 7f9620bbb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-10T00:55:48.829 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:48 smithi007 ceph-mon[32398]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:48.829 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:48 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:48.829 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:48 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:48.829 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:48 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:48.829 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:48 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:48.829 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:48 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:55:48.829 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:48 smithi007 ceph-mon[32398]: from='client.24113 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:55:48.829 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:48 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:55:48.829 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:48 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:55:48.830 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:48 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:48.958 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:48 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-z[38474]: 2024-06-10T00:55:48.856+0000 7f69aac00200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-10T00:55:48.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:48 smithi175 ceph-mon[36636]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:48.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:48 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:48.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:48 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:48.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:48 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:48.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:48 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:48.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:48 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:55:48.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:48 smithi175 ceph-mon[36636]: from='client.24113 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:55:48.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:48 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:55:48.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:48 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:55:48.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:48 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:48.991 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:48 smithi078 ceph-mon[36683]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:48.991 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:48 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:48.991 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:48 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:48.991 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:48 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:48.991 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:48 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:48.991 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:48 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:55:48.991 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:48 smithi078 ceph-mon[36683]: from='client.24113 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:55:48.991 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:48 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:55:48.991 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:48 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:55:48.991 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:48 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:49.596 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:49 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-z[38474]: 2024-06-10T00:55:49.293+0000 7f69aac00200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-10T00:55:49.596 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:49 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-z[38474]: 2024-06-10T00:55:49.392+0000 7f69aac00200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-10T00:55:49.596 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:49 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-z[38474]: 2024-06-10T00:55:49.595+0000 7f69aac00200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-10T00:55:49.980 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:49 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-y[38056]: 2024-06-10T00:55:49.639+0000 7f9620bbb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-10T00:55:49.980 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:49 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-y[38056]: 2024-06-10T00:55:49.810+0000 7f9620bbb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-10T00:55:49.980 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:49 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-y[38056]: 2024-06-10T00:55:49.979+0000 7f9620bbb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-10T00:55:50.231 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:50 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:50.231 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:50 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:50.231 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:50 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:50.231 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:50 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:50.231 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:50 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:50.232 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:50 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:55:50.232 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:50 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:50.232 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:50 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T00:55:50.232 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:50 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T00:55:50.232 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:50 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:50.458 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:50 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-z[38474]: 2024-06-10T00:55:50.123+0000 7f69aac00200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-10T00:55:50.459 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:50 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-z[38474]: 2024-06-10T00:55:50.216+0000 7f69aac00200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-10T00:55:50.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:50 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:50.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:50 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:50.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:50 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:50.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:50 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:50.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:50 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:50.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:50 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:55:50.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:50 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:50.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:50 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T00:55:50.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:50 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T00:55:50.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:50 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:50.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:50 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:50.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:50 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:50.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:50 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:50.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:50 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:50.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:50 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:50.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:50 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:55:50.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:50 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:50.520 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:50 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T00:55:50.520 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:50 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T00:55:50.520 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:50 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:50.568 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:50 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-y[38056]: 2024-06-10T00:55:50.275+0000 7f9620bbb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-10T00:55:50.568 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:50 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-y[38056]: 2024-06-10T00:55:50.367+0000 7f9620bbb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-10T00:55:50.569 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:50 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-y[38056]: 2024-06-10T00:55:50.568+0000 7f9620bbb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-10T00:55:50.808 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:50 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-z[38474]: 2024-06-10T00:55:50.461+0000 7f69aac00200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-10T00:55:50.808 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:50 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-z[38474]: 2024-06-10T00:55:50.597+0000 7f69aac00200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-10T00:55:50.808 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:50 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-z[38474]: 2024-06-10T00:55:50.691+0000 7f69aac00200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-10T00:55:50.808 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:50 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-z[38474]: 2024-06-10T00:55:50.807+0000 7f69aac00200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-10T00:55:50.835 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:50 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-y[38056]: 2024-06-10T00:55:50.758+0000 7f9620bbb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:55:51.458 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:51 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-z[38474]: 2024-06-10T00:55:51.108+0000 7f69aac00200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-10T00:55:51.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:51 smithi175 ceph-mon[36636]: Reconfiguring mgr.x (unknown last config time)... 2024-06-10T00:55:51.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:51 smithi175 ceph-mon[36636]: Reconfiguring daemon mgr.x on smithi007 2024-06-10T00:55:51.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:51 smithi175 ceph-mon[36636]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:51.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:51 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:51.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:51 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1334334341' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7c24c9f-b7ab-4c04-b2b4-120df9751667"}]: dispatch 2024-06-10T00:55:51.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:51 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1334334341' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7c24c9f-b7ab-4c04-b2b4-120df9751667"}]': finished 2024-06-10T00:55:51.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:51 smithi175 ceph-mon[36636]: osdmap e5: 1 total, 0 up, 1 in 2024-06-10T00:55:51.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:51 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:55:51.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:51 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/602449425' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:55:51.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:51 smithi007 ceph-mon[32398]: Reconfiguring mgr.x (unknown last config time)... 2024-06-10T00:55:51.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:51 smithi007 ceph-mon[32398]: Reconfiguring daemon mgr.x on smithi007 2024-06-10T00:55:51.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:51 smithi007 ceph-mon[32398]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:51.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:51 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:51.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:51 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1334334341' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7c24c9f-b7ab-4c04-b2b4-120df9751667"}]: dispatch 2024-06-10T00:55:51.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:51 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1334334341' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7c24c9f-b7ab-4c04-b2b4-120df9751667"}]': finished 2024-06-10T00:55:51.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:51 smithi007 ceph-mon[32398]: osdmap e5: 1 total, 0 up, 1 in 2024-06-10T00:55:51.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:51 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:55:51.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:51 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/602449425' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:55:51.585 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:51 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-y[38056]: 2024-06-10T00:55:51.374+0000 7f9620bbb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-10T00:55:51.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:51 smithi078 ceph-mon[36683]: Reconfiguring mgr.x (unknown last config time)... 2024-06-10T00:55:51.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:51 smithi078 ceph-mon[36683]: Reconfiguring daemon mgr.x on smithi007 2024-06-10T00:55:51.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:51 smithi078 ceph-mon[36683]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:51.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:51 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:51.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:51 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1334334341' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7c24c9f-b7ab-4c04-b2b4-120df9751667"}]: dispatch 2024-06-10T00:55:51.586 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:51 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1334334341' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7c24c9f-b7ab-4c04-b2b4-120df9751667"}]': finished 2024-06-10T00:55:51.586 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:51 smithi078 ceph-mon[36683]: osdmap e5: 1 total, 0 up, 1 in 2024-06-10T00:55:51.586 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:51 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:55:51.586 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:51 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/602449425' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:55:51.958 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:51 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-z[38474]: 2024-06-10T00:55:51.705+0000 7f69aac00200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-10T00:55:51.959 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 00:55:51 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-z[38474]: 2024-06-10T00:55:51.920+0000 7f69aac00200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-10T00:55:52.085 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:51 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-y[38056]: 2024-06-10T00:55:51.795+0000 7f9620bbb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-10T00:55:52.085 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:51 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-y[38056]: 2024-06-10T00:55:51.890+0000 7f9620bbb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-10T00:55:52.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:52.257+0000 7f4727fff700 1 -- 172.21.15.7:0/745911744 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 14) ==== 92059+0+0 (secure 0 0 0) 0x7f4730021cc0 con 0x7f47401024f0 2024-06-10T00:55:52.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:52 smithi007 ceph-mon[32398]: Standby manager daemon z started 2024-06-10T00:55:52.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:52 smithi007 ceph-mon[32398]: from='mgr.? 172.21.15.175:0/1647951116' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-10T00:55:52.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:52 smithi007 ceph-mon[32398]: from='mgr.? 172.21.15.175:0/1647951116' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-10T00:55:52.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:52 smithi007 ceph-mon[32398]: from='mgr.? 172.21.15.175:0/1647951116' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-10T00:55:52.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:52 smithi007 ceph-mon[32398]: from='mgr.? 172.21.15.175:0/1647951116' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-10T00:55:52.585 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:52 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-y[38056]: 2024-06-10T00:55:52.089+0000 7f9620bbb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-10T00:55:52.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:52 smithi078 ceph-mon[36683]: Standby manager daemon z started 2024-06-10T00:55:52.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:52 smithi078 ceph-mon[36683]: from='mgr.? 172.21.15.175:0/1647951116' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-10T00:55:52.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:52 smithi078 ceph-mon[36683]: from='mgr.? 172.21.15.175:0/1647951116' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-10T00:55:52.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:52 smithi078 ceph-mon[36683]: from='mgr.? 172.21.15.175:0/1647951116' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-10T00:55:52.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:52 smithi078 ceph-mon[36683]: from='mgr.? 172.21.15.175:0/1647951116' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-10T00:55:52.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:52 smithi175 ceph-mon[36636]: Standby manager daemon z started 2024-06-10T00:55:52.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:52 smithi175 ceph-mon[36636]: from='mgr.? 172.21.15.175:0/1647951116' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-10T00:55:52.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:52 smithi175 ceph-mon[36636]: from='mgr.? 172.21.15.175:0/1647951116' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-10T00:55:52.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:52 smithi175 ceph-mon[36636]: from='mgr.? 172.21.15.175:0/1647951116' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-10T00:55:52.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:52 smithi175 ceph-mon[36636]: from='mgr.? 172.21.15.175:0/1647951116' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-10T00:55:52.950 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:52 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-y[38056]: 2024-06-10T00:55:52.620+0000 7f9620bbb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-10T00:55:52.950 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:52 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-y[38056]: 2024-06-10T00:55:52.711+0000 7f9620bbb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-10T00:55:53.257 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:52 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-y[38056]: 2024-06-10T00:55:52.949+0000 7f9620bbb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-10T00:55:53.257 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:53 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-y[38056]: 2024-06-10T00:55:53.080+0000 7f9620bbb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-10T00:55:53.257 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:53 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-y[38056]: 2024-06-10T00:55:53.175+0000 7f9620bbb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-10T00:55:53.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:53 smithi007 ceph-mon[32398]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:53.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:53 smithi007 ceph-mon[32398]: mgrmap e14: x(active, since 76s), standbys: z 2024-06-10T00:55:53.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:53 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-10T00:55:53.585 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:53 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-y[38056]: 2024-06-10T00:55:53.289+0000 7f9620bbb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-10T00:55:53.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:53 smithi078 ceph-mon[36683]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:53.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:53 smithi078 ceph-mon[36683]: mgrmap e14: x(active, since 76s), standbys: z 2024-06-10T00:55:53.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:53 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-10T00:55:53.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:53 smithi175 ceph-mon[36636]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:53.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:53 smithi175 ceph-mon[36636]: mgrmap e14: x(active, since 76s), standbys: z 2024-06-10T00:55:53.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:53 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-10T00:55:54.085 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:53 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-y[38056]: 2024-06-10T00:55:53.586+0000 7f9620bbb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-10T00:55:54.585 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:54 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-y[38056]: 2024-06-10T00:55:54.181+0000 7f9620bbb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-10T00:55:54.585 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 00:55:54 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-y[38056]: 2024-06-10T00:55:54.396+0000 7f9620bbb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-10T00:55:55.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:55:55.264+0000 7f4727fff700 1 -- 172.21.15.7:0/745911744 <== mon.2 v2:172.21.15.78:3300/0 8 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7f47300163a0 con 0x7f47401024f0 2024-06-10T00:55:55.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:55 smithi007 ceph-mon[32398]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:55.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:55 smithi007 ceph-mon[32398]: Standby manager daemon y started 2024-06-10T00:55:55.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:55 smithi007 ceph-mon[32398]: from='mgr.? 172.21.15.78:0/2211200944' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-10T00:55:55.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:55 smithi007 ceph-mon[32398]: from='mgr.? 172.21.15.78:0/2211200944' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-10T00:55:55.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:55 smithi007 ceph-mon[32398]: from='mgr.? 172.21.15.78:0/2211200944' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-10T00:55:55.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:55 smithi007 ceph-mon[32398]: from='mgr.? 172.21.15.78:0/2211200944' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-10T00:55:55.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:55 smithi078 ceph-mon[36683]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:55.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:55 smithi078 ceph-mon[36683]: Standby manager daemon y started 2024-06-10T00:55:55.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:55 smithi078 ceph-mon[36683]: from='mgr.? 172.21.15.78:0/2211200944' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-10T00:55:55.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:55 smithi078 ceph-mon[36683]: from='mgr.? 172.21.15.78:0/2211200944' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-10T00:55:55.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:55 smithi078 ceph-mon[36683]: from='mgr.? 172.21.15.78:0/2211200944' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-10T00:55:55.586 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:55 smithi078 ceph-mon[36683]: from='mgr.? 172.21.15.78:0/2211200944' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-10T00:55:55.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:55 smithi175 ceph-mon[36636]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:55.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:55 smithi175 ceph-mon[36636]: Standby manager daemon y started 2024-06-10T00:55:55.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:55 smithi175 ceph-mon[36636]: from='mgr.? 172.21.15.78:0/2211200944' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-10T00:55:55.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:55 smithi175 ceph-mon[36636]: from='mgr.? 172.21.15.78:0/2211200944' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-10T00:55:55.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:55 smithi175 ceph-mon[36636]: from='mgr.? 172.21.15.78:0/2211200944' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-10T00:55:55.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:55 smithi175 ceph-mon[36636]: from='mgr.? 172.21.15.78:0/2211200944' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-10T00:55:56.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:56 smithi007 ceph-mon[32398]: mgrmap e15: x(active, since 79s), standbys: y, z 2024-06-10T00:55:56.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:56 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-10T00:55:56.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:56 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:56.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:56 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:56.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:56 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:55:56.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:56 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:56.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:56 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:55:56.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:56 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:56.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:56 smithi078 ceph-mon[36683]: mgrmap e15: x(active, since 79s), standbys: y, z 2024-06-10T00:55:56.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:56 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-10T00:55:56.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:56 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:56.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:56 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:56.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:56 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:55:56.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:56 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:56.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:56 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:55:56.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:56 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:56.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:56 smithi175 ceph-mon[36636]: mgrmap e15: x(active, since 79s), standbys: y, z 2024-06-10T00:55:56.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:56 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-10T00:55:56.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:56 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:56.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:56 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:56.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:56 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:55:56.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:56 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:56.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:56 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:55:56.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:56 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:55:57.463 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:57 smithi007 ceph-mon[32398]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:57.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:57 smithi078 ceph-mon[36683]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:57.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:57 smithi175 ceph-mon[36636]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:59.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:59 smithi007 ceph-mon[32398]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:59.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:59 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-10T00:55:59.516 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:55:59 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:59.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:59 smithi078 ceph-mon[36683]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:59.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:59 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-10T00:55:59.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:55:59 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:59.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:59 smithi175 ceph-mon[36636]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:55:59.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:59 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-10T00:55:59.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:55:59 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:56:00.494 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:00 smithi007 ceph-mon[32398]: Deploying daemon osd.0 on smithi007 2024-06-10T00:56:00.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:00 smithi078 ceph-mon[36683]: Deploying daemon osd.0 on smithi007 2024-06-10T00:56:00.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:00 smithi175 ceph-mon[36636]: Deploying daemon osd.0 on smithi007 2024-06-10T00:56:01.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:01 smithi007 ceph-mon[32398]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:56:01.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:01 smithi078 ceph-mon[36683]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:56:01.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:01 smithi175 ceph-mon[36636]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:56:03.452 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:03 smithi007 ceph-mon[32398]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:56:03.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:03 smithi078 ceph-mon[36683]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:56:03.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:03 smithi175 ceph-mon[36636]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:56:05.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:04 smithi007 ceph-mon[32398]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:56:05.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:04 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:05.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:04 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:05.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:04 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:56:05.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:04 smithi078 ceph-mon[36683]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:56:05.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:04 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:05.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:04 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:05.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:04 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:56:05.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:04 smithi175 ceph-mon[36636]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:56:05.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:04 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:05.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:04 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:05.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:04 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:56:07.221 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:06 smithi007 ceph-mon[32398]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:56:07.221 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:06 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:07.221 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:06 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:07.221 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:06 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:56:07.221 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:06 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:56:07.221 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:06 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:07.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:06 smithi078 ceph-mon[36683]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:56:07.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:06 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:07.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:06 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:07.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:07 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:56:07.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:07 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:56:07.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:07 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:07.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:07 smithi175 ceph-mon[36636]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:56:07.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:07 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:07.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:07 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:07.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:07 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:56:07.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:07 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:56:07.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:07 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:07.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:56:07.597+0000 7f4727fff700 1 -- 172.21.15.7:0/745911744 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4740075de0 con 0x7f4728039330 2024-06-10T00:56:07.597 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 0 on host 'smithi007' 2024-06-10T00:56:07.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:56:07.599+0000 7f4745473700 1 -- 172.21.15.7:0/745911744 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4728039330 msgr2=0x7f472803b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:07.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:56:07.599+0000 7f4745473700 1 --2- 172.21.15.7:0/745911744 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4728039330 0x7f472803b7f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4734011590 tx=0x7f4734009260 comp rx=0 tx=0).stop 2024-06-10T00:56:07.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:56:07.599+0000 7f4745473700 1 -- 172.21.15.7:0/745911744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47401024f0 msgr2=0x7f474019a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:07.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:56:07.599+0000 7f4745473700 1 --2- 172.21.15.7:0/745911744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47401024f0 0x7f474019a970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4730003790 tx=0x7f4730009580 comp rx=0 tx=0).stop 2024-06-10T00:56:07.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:56:07.599+0000 7f4745473700 1 -- 172.21.15.7:0/745911744 shutdown_connections 2024-06-10T00:56:07.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:56:07.599+0000 7f4745473700 1 --2- 172.21.15.7:0/745911744 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4728039330 0x7f472803b7f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:07.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:56:07.599+0000 7f4745473700 1 --2- 172.21.15.7:0/745911744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4740104090 0x7f47401a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:07.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:56:07.599+0000 7f4745473700 1 --2- 172.21.15.7:0/745911744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47401024f0 0x7f474019a970 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:07.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:56:07.599+0000 7f4745473700 1 --2- 172.21.15.7:0/745911744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47401036f0 0x7f474019aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:07.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:56:07.599+0000 7f4745473700 1 -- 172.21.15.7:0/745911744 >> 172.21.15.7:0/745911744 conn(0x7f47400fda90 msgr2=0x7f47400746c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:07.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:56:07.599+0000 7f4745473700 1 -- 172.21.15.7:0/745911744 shutdown_connections 2024-06-10T00:56:07.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:56:07.599+0000 7f4745473700 1 -- 172.21.15.7:0/745911744 wait complete. 2024-06-10T00:56:08.178 DEBUG:teuthology.orchestra.run.smithi007:osd.0> sudo journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.0.service 2024-06-10T00:56:08.181 INFO:tasks.cephadm:Deploying osd.1 on smithi078 with /dev/vg_nvme/lv_4... 2024-06-10T00:56:08.181 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-10T00:56:08.424 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T00:56:08.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:08 smithi007 ceph-mon[32398]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:56:08.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:08 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:08.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:08 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:08.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:08 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:56:08.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:08 smithi007 ceph-mon[32398]: from='osd.0 [v2:172.21.15.7:6802/404485726,v1:172.21.15.7:6803/404485726]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-10T00:56:08.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:08 smithi078 ceph-mon[36683]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:56:08.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:08 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:08.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:08 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:08.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:08 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:56:08.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:08 smithi078 ceph-mon[36683]: from='osd.0 [v2:172.21.15.7:6802/404485726,v1:172.21.15.7:6803/404485726]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-10T00:56:08.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:08 smithi175 ceph-mon[36636]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:56:08.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:08 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:08.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:08 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:08.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:08 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:56:08.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:08 smithi175 ceph-mon[36636]: from='osd.0 [v2:172.21.15.7:6802/404485726,v1:172.21.15.7:6803/404485726]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-10T00:56:09.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:09 smithi078 ceph-mon[36683]: from='osd.0 [v2:172.21.15.7:6802/404485726,v1:172.21.15.7:6803/404485726]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-10T00:56:09.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:09 smithi078 ceph-mon[36683]: osdmap e6: 1 total, 0 up, 1 in 2024-06-10T00:56:09.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:09 smithi078 ceph-mon[36683]: from='osd.0 [v2:172.21.15.7:6802/404485726,v1:172.21.15.7:6803/404485726]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-10T00:56:09.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:09 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:56:09.789 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:09 smithi007 ceph-mon[32398]: from='osd.0 [v2:172.21.15.7:6802/404485726,v1:172.21.15.7:6803/404485726]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-10T00:56:09.789 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:09 smithi007 ceph-mon[32398]: osdmap e6: 1 total, 0 up, 1 in 2024-06-10T00:56:09.789 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:09 smithi007 ceph-mon[32398]: from='osd.0 [v2:172.21.15.7:6802/404485726,v1:172.21.15.7:6803/404485726]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-10T00:56:09.789 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:09 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:56:09.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:09 smithi175 ceph-mon[36636]: from='osd.0 [v2:172.21.15.7:6802/404485726,v1:172.21.15.7:6803/404485726]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-10T00:56:09.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:09 smithi175 ceph-mon[36636]: osdmap e6: 1 total, 0 up, 1 in 2024-06-10T00:56:09.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:09 smithi175 ceph-mon[36636]: from='osd.0 [v2:172.21.15.7:6802/404485726,v1:172.21.15.7:6803/404485726]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-10T00:56:09.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:09 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:56:10.236 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-10T00:56:10.236 INFO:teuthology.orchestra.run.smithi078.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-10T00:56:10.236 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10+0 records in 2024-06-10T00:56:10.236 INFO:teuthology.orchestra.run.smithi078.stderr:10+0 records out 2024-06-10T00:56:10.236 INFO:teuthology.orchestra.run.smithi078.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0121275 s, 865 MB/s 2024-06-10T00:56:10.236 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping successful for: 2024-06-10T00:56:10.482 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 10 00:56:10 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-0[47139]: 2024-06-10T00:56:10.095+0000 7f1cbc110700 -1 osd.0 0 waiting for initial osdmap 2024-06-10T00:56:10.482 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 10 00:56:10 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-0[47139]: 2024-06-10T00:56:10.099+0000 7f1cb3efb700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:56:10.616 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:10 smithi078 ceph-mon[36683]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:56:10.616 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:10 smithi078 ceph-mon[36683]: from='osd.0 [v2:172.21.15.7:6802/404485726,v1:172.21.15.7:6803/404485726]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-06-10T00:56:10.616 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:10 smithi078 ceph-mon[36683]: osdmap e7: 1 total, 0 up, 1 in 2024-06-10T00:56:10.617 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:10 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:56:10.617 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:10 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:56:10.617 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:10 smithi078 ceph-mon[36683]: from='osd.0 [v2:172.21.15.7:6802/404485726,v1:172.21.15.7:6803/404485726]' entity='osd.0' 2024-06-10T00:56:10.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:10 smithi007 ceph-mon[32398]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:56:10.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:10 smithi007 ceph-mon[32398]: from='osd.0 [v2:172.21.15.7:6802/404485726,v1:172.21.15.7:6803/404485726]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-06-10T00:56:10.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:10 smithi007 ceph-mon[32398]: osdmap e7: 1 total, 0 up, 1 in 2024-06-10T00:56:10.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:10 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:56:10.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:10 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:56:10.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:10 smithi007 ceph-mon[32398]: from='osd.0 [v2:172.21.15.7:6802/404485726,v1:172.21.15.7:6803/404485726]' entity='osd.0' 2024-06-10T00:56:10.885 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_4 2024-06-10T00:56:10.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:10 smithi175 ceph-mon[36636]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:56:10.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:10 smithi175 ceph-mon[36636]: from='osd.0 [v2:172.21.15.7:6802/404485726,v1:172.21.15.7:6803/404485726]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-06-10T00:56:10.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:10 smithi175 ceph-mon[36636]: osdmap e7: 1 total, 0 up, 1 in 2024-06-10T00:56:10.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:10 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:56:10.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:10 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:56:10.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:10 smithi175 ceph-mon[36636]: from='osd.0 [v2:172.21.15.7:6802/404485726,v1:172.21.15.7:6803/404485726]' entity='osd.0' 2024-06-10T00:56:11.121 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T00:56:11.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:11 smithi078 ceph-mon[36683]: purged_snaps scrub starts 2024-06-10T00:56:11.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:11 smithi078 ceph-mon[36683]: purged_snaps scrub ok 2024-06-10T00:56:11.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:11 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:56:11.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:11 smithi078 ceph-mon[36683]: osd.0 [v2:172.21.15.7:6802/404485726,v1:172.21.15.7:6803/404485726] boot 2024-06-10T00:56:11.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:11 smithi078 ceph-mon[36683]: osdmap e8: 1 total, 1 up, 1 in 2024-06-10T00:56:11.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:11 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:56:11.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:11 smithi078 ceph-mon[36683]: Detected new or changed devices on smithi007 2024-06-10T00:56:11.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:11 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:11.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:11 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:11.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:11 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:56:11.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:11 smithi078 ceph-mon[36683]: Adjusting osd_memory_target on smithi007 to 17092M 2024-06-10T00:56:11.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:11 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:11.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:11 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:56:11.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:11 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:56:11.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:11 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:11.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:11 smithi175 ceph-mon[36636]: purged_snaps scrub starts 2024-06-10T00:56:11.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:11 smithi175 ceph-mon[36636]: purged_snaps scrub ok 2024-06-10T00:56:11.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:11 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:56:11.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:11 smithi175 ceph-mon[36636]: osd.0 [v2:172.21.15.7:6802/404485726,v1:172.21.15.7:6803/404485726] boot 2024-06-10T00:56:11.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:11 smithi175 ceph-mon[36636]: osdmap e8: 1 total, 1 up, 1 in 2024-06-10T00:56:11.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:11 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:56:11.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:11 smithi175 ceph-mon[36636]: Detected new or changed devices on smithi007 2024-06-10T00:56:11.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:11 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:11.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:11 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:11.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:11 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:56:11.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:11 smithi175 ceph-mon[36636]: Adjusting osd_memory_target on smithi007 to 17092M 2024-06-10T00:56:11.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:11 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:11.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:11 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:56:11.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:11 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:56:11.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:11 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:12.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:11 smithi007 ceph-mon[32398]: purged_snaps scrub starts 2024-06-10T00:56:12.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:11 smithi007 ceph-mon[32398]: purged_snaps scrub ok 2024-06-10T00:56:12.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:11 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:56:12.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:11 smithi007 ceph-mon[32398]: osd.0 [v2:172.21.15.7:6802/404485726,v1:172.21.15.7:6803/404485726] boot 2024-06-10T00:56:12.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:11 smithi007 ceph-mon[32398]: osdmap e8: 1 total, 1 up, 1 in 2024-06-10T00:56:12.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:11 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:56:12.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:11 smithi007 ceph-mon[32398]: Detected new or changed devices on smithi007 2024-06-10T00:56:12.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:11 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:12.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:11 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:12.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:11 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:56:12.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:11 smithi007 ceph-mon[32398]: Adjusting osd_memory_target on smithi007 to 17092M 2024-06-10T00:56:12.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:11 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:12.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:11 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:56:12.020 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:11 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:56:12.020 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:11 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:12.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.605+0000 7f1d937aa700 1 -- 172.21.15.78:0/3913557668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d8c108210 msgr2=0x7f1d8c10a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:12.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.605+0000 7f1d937aa700 1 --2- 172.21.15.78:0/3913557668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d8c108210 0x7f1d8c10a680 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400b0a0 tx=0x7f1d8400b3b0 comp rx=0 tx=0).stop 2024-06-10T00:56:12.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.605+0000 7f1d937aa700 1 -- 172.21.15.78:0/3913557668 shutdown_connections 2024-06-10T00:56:12.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.605+0000 7f1d937aa700 1 --2- 172.21.15.78:0/3913557668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d8c108210 0x7f1d8c10a680 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:12.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.605+0000 7f1d937aa700 1 --2- 172.21.15.78:0/3913557668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d8c102fb0 0x7f1d8c1053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:12.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.605+0000 7f1d937aa700 1 --2- 172.21.15.78:0/3913557668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d8c1058e0 0x7f1d8c107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:12.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.605+0000 7f1d937aa700 1 -- 172.21.15.78:0/3913557668 >> 172.21.15.78:0/3913557668 conn(0x7f1d8c0fc9e0 msgr2=0x7f1d8c0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:12.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.605+0000 7f1d937aa700 1 -- 172.21.15.78:0/3913557668 shutdown_connections 2024-06-10T00:56:12.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.605+0000 7f1d937aa700 1 -- 172.21.15.78:0/3913557668 wait complete. 2024-06-10T00:56:12.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.605+0000 7f1d937aa700 1 Processor -- start 2024-06-10T00:56:12.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.605+0000 7f1d937aa700 1 -- start start 2024-06-10T00:56:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.606+0000 7f1d937aa700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d8c102fb0 0x7f1d8c198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.606+0000 7f1d937aa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d8c1058e0 0x7f1d8c198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.606+0000 7f1d937aa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d8c108210 0x7f1d8c19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.606+0000 7f1d937aa700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8c10e130 con 0x7f1d8c1058e0 2024-06-10T00:56:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.606+0000 7f1d937aa700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8c10dfb0 con 0x7f1d8c108210 2024-06-10T00:56:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.606+0000 7f1d937aa700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8c10e2b0 con 0x7f1d8c102fb0 2024-06-10T00:56:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.606+0000 7f1d91d47700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d8c108210 0x7f1d8c19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.606+0000 7f1d91d47700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d8c108210 0x7f1d8c19fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32842/0 (socket says 172.21.15.78:32842) 2024-06-10T00:56:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.606+0000 7f1d91d47700 1 -- 172.21.15.78:0/3511714765 learned_addr learned my addr 172.21.15.78:0/3511714765 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-06-10T00:56:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.606+0000 7f1d91546700 1 --2- 172.21.15.78:0/3511714765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d8c102fb0 0x7f1d8c198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.606+0000 7f1d90d45700 1 --2- 172.21.15.78:0/3511714765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d8c1058e0 0x7f1d8c198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.606+0000 7f1d91d47700 1 -- 172.21.15.78:0/3511714765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d8c102fb0 msgr2=0x7f1d8c198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.606+0000 7f1d91d47700 1 --2- 172.21.15.78:0/3511714765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d8c102fb0 0x7f1d8c198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:12.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.606+0000 7f1d91d47700 1 -- 172.21.15.78:0/3511714765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d8c1058e0 msgr2=0x7f1d8c198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:12.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.606+0000 7f1d91d47700 1 --2- 172.21.15.78:0/3511714765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d8c1058e0 0x7f1d8c198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:12.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.606+0000 7f1d91d47700 1 -- 172.21.15.78:0/3511714765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d8c1a02a0 con 0x7f1d8c108210 2024-06-10T00:56:12.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.606+0000 7f1d91d47700 1 --2- 172.21.15.78:0/3511714765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d8c108210 0x7f1d8c19fc00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1d84007f00 tx=0x7f1d84004e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:12.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.606+0000 7f1d827fc700 1 -- 172.21.15.78:0/3511714765 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d84021070 con 0x7f1d8c108210 2024-06-10T00:56:12.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.606+0000 7f1d937aa700 1 -- 172.21.15.78:0/3511714765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d8c1a0530 con 0x7f1d8c108210 2024-06-10T00:56:12.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.606+0000 7f1d827fc700 1 -- 172.21.15.78:0/3511714765 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d84007cc0 con 0x7f1d8c108210 2024-06-10T00:56:12.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.606+0000 7f1d827fc700 1 -- 172.21.15.78:0/3511714765 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d84005500 con 0x7f1d8c108210 2024-06-10T00:56:12.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.606+0000 7f1d937aa700 1 -- 172.21.15.78:0/3511714765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d8c1a09e0 con 0x7f1d8c108210 2024-06-10T00:56:12.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.607+0000 7f1d937aa700 1 -- 172.21.15.78:0/3511714765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d8c069180 con 0x7f1d8c108210 2024-06-10T00:56:12.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.608+0000 7f1d827fc700 1 -- 172.21.15.78:0/3511714765 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7f1d84024070 con 0x7f1d8c108210 2024-06-10T00:56:12.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.609+0000 7f1d827fc700 1 --2- 172.21.15.78:0/3511714765 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1d780a2b90 0x7f1d780a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:12.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.609+0000 7f1d827fc700 1 -- 172.21.15.78:0/3511714765 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f1d8400f020 con 0x7f1d8c108210 2024-06-10T00:56:12.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.609+0000 7f1d91546700 1 --2- 172.21.15.78:0/3511714765 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1d780a2b90 0x7f1d780a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:12.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.609+0000 7f1d91546700 1 --2- 172.21.15.78:0/3511714765 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1d780a2b90 0x7f1d780a5050 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1d88005950 tx=0x7f1d8800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:12.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.613+0000 7f1d827fc700 1 -- 172.21.15.78:0/3511714765 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1d8402b020 con 0x7f1d8c108210 2024-06-10T00:56:12.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:12.818+0000 7f1d937aa700 1 -- 172.21.15.78:0/3511714765 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1d8c0628d0 con 0x7f1d780a2b90 2024-06-10T00:56:13.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:13 smithi078 ceph-mon[36683]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:56:13.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:13 smithi078 ceph-mon[36683]: osdmap e9: 1 total, 1 up, 1 in 2024-06-10T00:56:13.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:13 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:56:13.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:13 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:56:13.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:13 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:56:13.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:13 smithi175 ceph-mon[36636]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:56:13.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:13 smithi175 ceph-mon[36636]: osdmap e9: 1 total, 1 up, 1 in 2024-06-10T00:56:13.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:13 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:56:13.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:13 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:56:13.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:13 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:56:13.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:13 smithi007 ceph-mon[32398]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:56:13.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:13 smithi007 ceph-mon[32398]: osdmap e9: 1 total, 1 up, 1 in 2024-06-10T00:56:13.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:13 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:56:13.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:13 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:56:13.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:13 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:56:14.364 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:14 smithi078 ceph-mon[36683]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:56:14.610 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:14 smithi175 ceph-mon[36636]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:56:14.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:14 smithi007 ceph-mon[32398]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:56:15.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:15 smithi078 ceph-mon[36683]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:15.630 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:15 smithi175 ceph-mon[36636]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:15.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:15 smithi007 ceph-mon[32398]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:16.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:16 smithi078 ceph-mon[36683]: from='client.? 172.21.15.78:0/943421230' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9bf70a6b-f31c-4077-b5af-229f17bc2845"}]: dispatch 2024-06-10T00:56:16.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:16 smithi078 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9bf70a6b-f31c-4077-b5af-229f17bc2845"}]: dispatch 2024-06-10T00:56:16.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:16 smithi078 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9bf70a6b-f31c-4077-b5af-229f17bc2845"}]': finished 2024-06-10T00:56:16.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:16 smithi078 ceph-mon[36683]: osdmap e10: 2 total, 1 up, 2 in 2024-06-10T00:56:16.586 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:16 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:56:16.586 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:16 smithi078 ceph-mon[36683]: from='client.? 172.21.15.78:0/2155016276' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:56:16.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:16 smithi175 ceph-mon[36636]: from='client.? 172.21.15.78:0/943421230' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9bf70a6b-f31c-4077-b5af-229f17bc2845"}]: dispatch 2024-06-10T00:56:16.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:16 smithi175 ceph-mon[36636]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9bf70a6b-f31c-4077-b5af-229f17bc2845"}]: dispatch 2024-06-10T00:56:16.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:16 smithi175 ceph-mon[36636]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9bf70a6b-f31c-4077-b5af-229f17bc2845"}]': finished 2024-06-10T00:56:16.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:16 smithi175 ceph-mon[36636]: osdmap e10: 2 total, 1 up, 2 in 2024-06-10T00:56:16.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:16 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:56:16.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:16 smithi175 ceph-mon[36636]: from='client.? 172.21.15.78:0/2155016276' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:56:16.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:16 smithi007 ceph-mon[32398]: from='client.? 172.21.15.78:0/943421230' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9bf70a6b-f31c-4077-b5af-229f17bc2845"}]: dispatch 2024-06-10T00:56:16.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:16 smithi007 ceph-mon[32398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9bf70a6b-f31c-4077-b5af-229f17bc2845"}]: dispatch 2024-06-10T00:56:16.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:16 smithi007 ceph-mon[32398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9bf70a6b-f31c-4077-b5af-229f17bc2845"}]': finished 2024-06-10T00:56:16.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:16 smithi007 ceph-mon[32398]: osdmap e10: 2 total, 1 up, 2 in 2024-06-10T00:56:16.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:16 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:56:16.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:16 smithi007 ceph-mon[32398]: from='client.? 172.21.15.78:0/2155016276' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:56:17.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:17 smithi078 ceph-mon[36683]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:17.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:17 smithi175 ceph-mon[36636]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:17.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:17 smithi007 ceph-mon[32398]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:19.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:19 smithi078 ceph-mon[36683]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:19.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:19 smithi175 ceph-mon[36636]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:19.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:19 smithi007 ceph-mon[32398]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:21.512 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:21 smithi078 ceph-mon[36683]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:21.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:21 smithi007 ceph-mon[32398]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:21.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:21 smithi175 ceph-mon[36636]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:23.488 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:23 smithi078 ceph-mon[36683]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:23.488 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:23 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-10T00:56:23.488 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:23 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:56:23.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:23 smithi175 ceph-mon[36636]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:23.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:23 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-10T00:56:23.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:23 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:56:23.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:23 smithi007 ceph-mon[32398]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:23.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:23 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-10T00:56:23.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:23 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:56:24.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:24 smithi078 ceph-mon[36683]: Deploying daemon osd.1 on smithi078 2024-06-10T00:56:24.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:24 smithi175 ceph-mon[36636]: Deploying daemon osd.1 on smithi078 2024-06-10T00:56:24.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:24 smithi007 ceph-mon[32398]: Deploying daemon osd.1 on smithi078 2024-06-10T00:56:25.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:25 smithi078 ceph-mon[36683]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:25.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:25 smithi175 ceph-mon[36636]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:25.770 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:25 smithi007 ceph-mon[32398]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:27.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:27 smithi007 ceph-mon[32398]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:27.572 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:27 smithi078 ceph-mon[36683]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:27.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:27 smithi175 ceph-mon[36636]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:29.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:28 smithi078 ceph-mon[36683]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:29.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:28 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:29.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:28 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:56:29.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:28 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:29.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:28 smithi175 ceph-mon[36636]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:29.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:28 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:29.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:28 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:56:29.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:28 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:29.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:28 smithi007 ceph-mon[32398]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:29.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:28 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:29.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:28 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:56:29.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:28 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:31.164 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:30 smithi078 ceph-mon[36683]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:31.164 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:30 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:31.164 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:30 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:31.164 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:30 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:56:31.164 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:30 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:56:31.164 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:30 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:31.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:30 smithi175 ceph-mon[36636]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:31.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:30 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:31.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:30 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:31.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:30 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:56:31.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:30 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:56:31.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:30 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:31.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:30 smithi007 ceph-mon[32398]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:31.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:30 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:31.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:30 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:31.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:30 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:56:31.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:30 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:56:31.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:30 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:31.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:31.484+0000 7f1d827fc700 1 -- 172.21.15.78:0/3511714765 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1d8c0628d0 con 0x7f1d780a2b90 2024-06-10T00:56:31.485 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 1 on host 'smithi078' 2024-06-10T00:56:31.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:31.486+0000 7f1d937aa700 1 -- 172.21.15.78:0/3511714765 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1d780a2b90 msgr2=0x7f1d780a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:31.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:31.486+0000 7f1d937aa700 1 --2- 172.21.15.78:0/3511714765 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1d780a2b90 0x7f1d780a5050 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1d88005950 tx=0x7f1d8800a300 comp rx=0 tx=0).stop 2024-06-10T00:56:31.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:31.486+0000 7f1d937aa700 1 -- 172.21.15.78:0/3511714765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d8c108210 msgr2=0x7f1d8c19fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:31.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:31.486+0000 7f1d937aa700 1 --2- 172.21.15.78:0/3511714765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d8c108210 0x7f1d8c19fc00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1d84007f00 tx=0x7f1d84004e40 comp rx=0 tx=0).stop 2024-06-10T00:56:31.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:31.486+0000 7f1d937aa700 1 -- 172.21.15.78:0/3511714765 shutdown_connections 2024-06-10T00:56:31.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:31.486+0000 7f1d937aa700 1 --2- 172.21.15.78:0/3511714765 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1d780a2b90 0x7f1d780a5050 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:31.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:31.486+0000 7f1d937aa700 1 --2- 172.21.15.78:0/3511714765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d8c108210 0x7f1d8c19fc00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:31.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:31.486+0000 7f1d937aa700 1 --2- 172.21.15.78:0/3511714765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d8c102fb0 0x7f1d8c198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:31.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:31.486+0000 7f1d937aa700 1 --2- 172.21.15.78:0/3511714765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d8c1058e0 0x7f1d8c198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:31.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:31.486+0000 7f1d937aa700 1 -- 172.21.15.78:0/3511714765 >> 172.21.15.78:0/3511714765 conn(0x7f1d8c0fc9e0 msgr2=0x7f1d8c1038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:31.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:31.486+0000 7f1d937aa700 1 -- 172.21.15.78:0/3511714765 shutdown_connections 2024-06-10T00:56:31.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:56:31.486+0000 7f1d937aa700 1 -- 172.21.15.78:0/3511714765 wait complete. 2024-06-10T00:56:32.128 DEBUG:teuthology.orchestra.run.smithi078:osd.1> sudo journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.1.service 2024-06-10T00:56:32.130 INFO:tasks.cephadm:Deploying osd.2 on smithi175 with /dev/vg_nvme/lv_4... 2024-06-10T00:56:32.131 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-10T00:56:32.373 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:56:32.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:32 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:32.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:32 smithi078 ceph-mon[36683]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:32.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:32 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:32.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:32 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:56:32.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:32 smithi078 ceph-mon[36683]: from='osd.1 [v2:172.21.15.78:6800/1856804448,v1:172.21.15.78:6801/1856804448]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-10T00:56:32.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:32 smithi078 ceph-mon[36683]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-10T00:56:32.760 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:32 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:32.760 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:32 smithi175 ceph-mon[36636]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:32.760 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:32 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:32.760 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:32 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:56:32.760 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:32 smithi175 ceph-mon[36636]: from='osd.1 [v2:172.21.15.78:6800/1856804448,v1:172.21.15.78:6801/1856804448]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-10T00:56:32.760 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:32 smithi175 ceph-mon[36636]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-10T00:56:32.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:32 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:32.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:32 smithi007 ceph-mon[32398]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:32.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:32 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:32.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:32 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:56:32.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:32 smithi007 ceph-mon[32398]: from='osd.1 [v2:172.21.15.78:6800/1856804448,v1:172.21.15.78:6801/1856804448]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-10T00:56:32.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:32 smithi007 ceph-mon[32398]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-10T00:56:33.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:33 smithi078 ceph-mon[36683]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-10T00:56:33.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:33 smithi078 ceph-mon[36683]: osdmap e11: 2 total, 1 up, 2 in 2024-06-10T00:56:33.586 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:33 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:56:33.586 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:33 smithi078 ceph-mon[36683]: from='osd.1 [v2:172.21.15.78:6800/1856804448,v1:172.21.15.78:6801/1856804448]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-06-10T00:56:33.586 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:33 smithi078 ceph-mon[36683]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-06-10T00:56:33.586 INFO:journalctl@ceph.osd.1.smithi078.stdout:Jun 10 00:56:33 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-1[41229]: 2024-06-10T00:56:33.486+0000 7f83fe77d700 -1 osd.1 0 waiting for initial osdmap 2024-06-10T00:56:33.586 INFO:journalctl@ceph.osd.1.smithi078.stdout:Jun 10 00:56:33 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-1[41229]: 2024-06-10T00:56:33.489+0000 7f83fa570700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:56:33.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:33 smithi175 ceph-mon[36636]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-10T00:56:33.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:33 smithi175 ceph-mon[36636]: osdmap e11: 2 total, 1 up, 2 in 2024-06-10T00:56:33.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:33 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:56:33.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:33 smithi175 ceph-mon[36636]: from='osd.1 [v2:172.21.15.78:6800/1856804448,v1:172.21.15.78:6801/1856804448]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-06-10T00:56:33.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:33 smithi175 ceph-mon[36636]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-06-10T00:56:33.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:33 smithi007 ceph-mon[32398]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-10T00:56:33.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:33 smithi007 ceph-mon[32398]: osdmap e11: 2 total, 1 up, 2 in 2024-06-10T00:56:33.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:33 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:56:33.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:33 smithi007 ceph-mon[32398]: from='osd.1 [v2:172.21.15.78:6800/1856804448,v1:172.21.15.78:6801/1856804448]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-06-10T00:56:33.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:33 smithi007 ceph-mon[32398]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-06-10T00:56:34.169 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-10T00:56:34.170 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-10T00:56:34.170 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-06-10T00:56:34.170 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-06-10T00:56:34.170 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114553 s, 915 MB/s 2024-06-10T00:56:34.170 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-06-10T00:56:34.681 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:34 smithi078 ceph-mon[36683]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:34.681 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:34 smithi078 ceph-mon[36683]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-06-10T00:56:34.681 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:34 smithi078 ceph-mon[36683]: osdmap e12: 2 total, 1 up, 2 in 2024-06-10T00:56:34.681 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:34 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:56:34.681 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:34 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:56:34.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:34 smithi007 ceph-mon[32398]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:34.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:34 smithi007 ceph-mon[32398]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-06-10T00:56:34.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:34 smithi007 ceph-mon[32398]: osdmap e12: 2 total, 1 up, 2 in 2024-06-10T00:56:34.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:34 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:56:34.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:34 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:56:34.771 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:34 smithi175 ceph-mon[36636]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:56:34.771 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:34 smithi175 ceph-mon[36636]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-06-10T00:56:34.771 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:34 smithi175 ceph-mon[36636]: osdmap e12: 2 total, 1 up, 2 in 2024-06-10T00:56:34.771 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:34 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:56:34.771 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:34 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:56:34.798 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi175:vg_nvme/lv_4 2024-06-10T00:56:35.037 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:56:35.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:35 smithi175 ceph-mon[36636]: purged_snaps scrub starts 2024-06-10T00:56:35.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:35 smithi175 ceph-mon[36636]: purged_snaps scrub ok 2024-06-10T00:56:35.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:35 smithi175 ceph-mon[36636]: osd.1 [v2:172.21.15.78:6800/1856804448,v1:172.21.15.78:6801/1856804448] boot 2024-06-10T00:56:35.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:35 smithi175 ceph-mon[36636]: osdmap e13: 2 total, 2 up, 2 in 2024-06-10T00:56:35.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:35 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:56:35.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:35 smithi175 ceph-mon[36636]: Detected new or changed devices on smithi078 2024-06-10T00:56:35.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:35 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:35.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:35 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:35.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:35 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:56:35.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:35 smithi175 ceph-mon[36636]: Adjusting osd_memory_target on smithi078 to 17088M 2024-06-10T00:56:35.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:35 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:35.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:35 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:56:35.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:35 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:56:35.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:35 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:35.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:35 smithi007 ceph-mon[32398]: purged_snaps scrub starts 2024-06-10T00:56:35.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:35 smithi007 ceph-mon[32398]: purged_snaps scrub ok 2024-06-10T00:56:35.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:35 smithi007 ceph-mon[32398]: osd.1 [v2:172.21.15.78:6800/1856804448,v1:172.21.15.78:6801/1856804448] boot 2024-06-10T00:56:35.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:35 smithi007 ceph-mon[32398]: osdmap e13: 2 total, 2 up, 2 in 2024-06-10T00:56:35.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:35 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:56:35.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:35 smithi007 ceph-mon[32398]: Detected new or changed devices on smithi078 2024-06-10T00:56:35.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:35 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:35.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:35 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:35.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:35 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:56:35.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:35 smithi007 ceph-mon[32398]: Adjusting osd_memory_target on smithi078 to 17088M 2024-06-10T00:56:35.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:35 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:35.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:35 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:56:35.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:35 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:56:35.770 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:35 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:35.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:35 smithi078 ceph-mon[36683]: purged_snaps scrub starts 2024-06-10T00:56:35.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:35 smithi078 ceph-mon[36683]: purged_snaps scrub ok 2024-06-10T00:56:35.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:35 smithi078 ceph-mon[36683]: osd.1 [v2:172.21.15.78:6800/1856804448,v1:172.21.15.78:6801/1856804448] boot 2024-06-10T00:56:35.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:35 smithi078 ceph-mon[36683]: osdmap e13: 2 total, 2 up, 2 in 2024-06-10T00:56:35.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:35 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:56:35.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:35 smithi078 ceph-mon[36683]: Detected new or changed devices on smithi078 2024-06-10T00:56:35.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:35 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:35.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:35 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:35.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:35 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:56:35.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:35 smithi078 ceph-mon[36683]: Adjusting osd_memory_target on smithi078 to 17088M 2024-06-10T00:56:35.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:35 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:35.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:35 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:56:35.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:35 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:56:35.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:35 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:36.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.544+0000 7f35481ab700 1 -- 172.21.15.175:0/3382437884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3540100b80 msgr2=0x7f3540100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:36.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.544+0000 7f35481ab700 1 --2- 172.21.15.175:0/3382437884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3540100b80 0x7f3540100fa0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f353c009990 tx=0x7f353c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:56:36.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.545+0000 7f35481ab700 1 -- 172.21.15.175:0/3382437884 shutdown_connections 2024-06-10T00:56:36.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.545+0000 7f35481ab700 1 --2- 172.21.15.175:0/3382437884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3540102720 0x7f3540108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:36.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.545+0000 7f35481ab700 1 --2- 172.21.15.175:0/3382437884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3540100b80 0x7f3540100fa0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:36.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.545+0000 7f35481ab700 1 --2- 172.21.15.175:0/3382437884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3540101d80 0x7f35401021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:36.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.545+0000 7f35481ab700 1 -- 172.21.15.175:0/3382437884 >> 172.21.15.175:0/3382437884 conn(0x7f35400fc100 msgr2=0x7f35400fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:36.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.545+0000 7f35481ab700 1 -- 172.21.15.175:0/3382437884 shutdown_connections 2024-06-10T00:56:36.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.545+0000 7f35481ab700 1 -- 172.21.15.175:0/3382437884 wait complete. 2024-06-10T00:56:36.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.545+0000 7f35481ab700 1 Processor -- start 2024-06-10T00:56:36.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.545+0000 7f35481ab700 1 -- start start 2024-06-10T00:56:36.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.545+0000 7f35481ab700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3540100b80 0x7f3540198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:36.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.545+0000 7f35481ab700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3540101d80 0x7f3540198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:36.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.545+0000 7f35481ab700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3540102720 0x7f354019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:36.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.545+0000 7f35481ab700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f354010d040 con 0x7f3540101d80 2024-06-10T00:56:36.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.545+0000 7f35481ab700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f354010cec0 con 0x7f3540102720 2024-06-10T00:56:36.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.545+0000 7f35481ab700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f354010d1c0 con 0x7f3540100b80 2024-06-10T00:56:36.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.545+0000 7f3545f47700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3540100b80 0x7f3540198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:36.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.546+0000 7f3545f47700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3540100b80 0x7f3540198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:41664/0 (socket says 172.21.15.175:41664) 2024-06-10T00:56:36.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.546+0000 7f3545f47700 1 -- 172.21.15.175:0/4032309556 learned_addr learned my addr 172.21.15.175:0/4032309556 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-06-10T00:56:36.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.546+0000 7f3546748700 1 --2- 172.21.15.175:0/4032309556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3540102720 0x7f354019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:36.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.546+0000 7f3545f47700 1 -- 172.21.15.175:0/4032309556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3540102720 msgr2=0x7f354019fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:36.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.546+0000 7f3545f47700 1 --2- 172.21.15.175:0/4032309556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3540102720 0x7f354019fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:36.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.546+0000 7f3545f47700 1 -- 172.21.15.175:0/4032309556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3540101d80 msgr2=0x7f3540198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:36.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.546+0000 7f3545746700 1 --2- 172.21.15.175:0/4032309556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3540101d80 0x7f3540198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:36.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.546+0000 7f3545f47700 1 --2- 172.21.15.175:0/4032309556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3540101d80 0x7f3540198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:36.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.546+0000 7f3545f47700 1 -- 172.21.15.175:0/4032309556 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35401a03a0 con 0x7f3540100b80 2024-06-10T00:56:36.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.546+0000 7f3545746700 1 --2- 172.21.15.175:0/4032309556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3540101d80 0x7f3540198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:56:36.550 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.546+0000 7f3545f47700 1 --2- 172.21.15.175:0/4032309556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3540100b80 0x7f3540198740 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f353c0047b0 tx=0x7f353c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:36.550 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.546+0000 7f3536ffd700 1 -- 172.21.15.175:0/4032309556 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c027070 con 0x7f3540100b80 2024-06-10T00:56:36.550 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.546+0000 7f3536ffd700 1 -- 172.21.15.175:0/4032309556 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f353c022470 con 0x7f3540100b80 2024-06-10T00:56:36.550 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.546+0000 7f3536ffd700 1 -- 172.21.15.175:0/4032309556 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c00f6c0 con 0x7f3540100b80 2024-06-10T00:56:36.550 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.546+0000 7f35481ab700 1 -- 172.21.15.175:0/4032309556 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35401a0630 con 0x7f3540100b80 2024-06-10T00:56:36.550 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.546+0000 7f35481ab700 1 -- 172.21.15.175:0/4032309556 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35401a0ab0 con 0x7f3540100b80 2024-06-10T00:56:36.551 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.547+0000 7f35481ab700 1 -- 172.21.15.175:0/4032309556 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3540069180 con 0x7f3540100b80 2024-06-10T00:56:36.551 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.548+0000 7f3536ffd700 1 -- 172.21.15.175:0/4032309556 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7f353c02a070 con 0x7f3540100b80 2024-06-10T00:56:36.551 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.549+0000 7f3536ffd700 1 --2- 172.21.15.175:0/4032309556 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f352c0a2c10 0x7f352c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:36.551 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.549+0000 7f3536ffd700 1 -- 172.21.15.175:0/4032309556 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f353c0cef60 con 0x7f3540100b80 2024-06-10T00:56:36.551 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.549+0000 7f3545746700 1 --2- 172.21.15.175:0/4032309556 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f352c0a2c10 0x7f352c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:36.551 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.549+0000 7f3545746700 1 --2- 172.21.15.175:0/4032309556 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f352c0a2c10 0x7f352c0a50d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3530005fd0 tx=0x7f3530005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:36.553 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.552+0000 7f3536ffd700 1 -- 172.21.15.175:0/4032309556 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f353c031070 con 0x7f3540100b80 2024-06-10T00:56:36.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:36.756+0000 7f35481ab700 1 -- 172.21.15.175:0/4032309556 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f35400fe2b0 con 0x7f352c0a2c10 2024-06-10T00:56:36.757 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:36 smithi175 ceph-mon[36636]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:36.757 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:36 smithi175 ceph-mon[36636]: osdmap e14: 2 total, 2 up, 2 in 2024-06-10T00:56:36.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:36 smithi007 ceph-mon[32398]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:36.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:36 smithi007 ceph-mon[32398]: osdmap e14: 2 total, 2 up, 2 in 2024-06-10T00:56:36.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:36 smithi078 ceph-mon[36683]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:36.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:36 smithi078 ceph-mon[36683]: osdmap e14: 2 total, 2 up, 2 in 2024-06-10T00:56:37.658 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:37 smithi175 ceph-mon[36636]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:56:37.658 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:56:37.658 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:56:37.658 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:56:37.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:37 smithi007 ceph-mon[32398]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:56:37.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:56:37.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:56:37.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:56:37.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:37 smithi078 ceph-mon[36683]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:56:37.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:56:37.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:56:37.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:56:38.598 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:38 smithi175 ceph-mon[36636]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:38.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:38 smithi007 ceph-mon[32398]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:38.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:38 smithi078 ceph-mon[36683]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:39.759 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:39 smithi175 ceph-mon[36636]: from='client.? 172.21.15.175:0/1014512827' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62923c97-caf9-4f7e-9c14-9c9918843c78"}]: dispatch 2024-06-10T00:56:39.759 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:39 smithi175 ceph-mon[36636]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62923c97-caf9-4f7e-9c14-9c9918843c78"}]: dispatch 2024-06-10T00:56:39.759 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:39 smithi175 ceph-mon[36636]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "62923c97-caf9-4f7e-9c14-9c9918843c78"}]': finished 2024-06-10T00:56:39.759 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:39 smithi175 ceph-mon[36636]: osdmap e15: 3 total, 2 up, 3 in 2024-06-10T00:56:39.759 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:39 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:56:39.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:39 smithi007 ceph-mon[32398]: from='client.? 172.21.15.175:0/1014512827' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62923c97-caf9-4f7e-9c14-9c9918843c78"}]: dispatch 2024-06-10T00:56:39.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:39 smithi007 ceph-mon[32398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62923c97-caf9-4f7e-9c14-9c9918843c78"}]: dispatch 2024-06-10T00:56:39.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:39 smithi007 ceph-mon[32398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "62923c97-caf9-4f7e-9c14-9c9918843c78"}]': finished 2024-06-10T00:56:39.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:39 smithi007 ceph-mon[32398]: osdmap e15: 3 total, 2 up, 3 in 2024-06-10T00:56:39.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:39 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:56:39.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:39 smithi078 ceph-mon[36683]: from='client.? 172.21.15.175:0/1014512827' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62923c97-caf9-4f7e-9c14-9c9918843c78"}]: dispatch 2024-06-10T00:56:39.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:39 smithi078 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62923c97-caf9-4f7e-9c14-9c9918843c78"}]: dispatch 2024-06-10T00:56:39.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:39 smithi078 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "62923c97-caf9-4f7e-9c14-9c9918843c78"}]': finished 2024-06-10T00:56:39.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:39 smithi078 ceph-mon[36683]: osdmap e15: 3 total, 2 up, 3 in 2024-06-10T00:56:39.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:39 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:56:40.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:40 smithi007 ceph-mon[32398]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:40.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:40 smithi007 ceph-mon[32398]: from='client.? 172.21.15.175:0/3808185303' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:56:40.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:40 smithi078 ceph-mon[36683]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:40.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:40 smithi078 ceph-mon[36683]: from='client.? 172.21.15.175:0/3808185303' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:56:40.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:40 smithi175 ceph-mon[36636]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:40.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:40 smithi175 ceph-mon[36636]: from='client.? 172.21.15.175:0/3808185303' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:56:42.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:42 smithi007 ceph-mon[32398]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:42.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:42 smithi078 ceph-mon[36683]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:42.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:42 smithi175 ceph-mon[36636]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:44.606 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:44 smithi175 ceph-mon[36636]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:44.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:44 smithi007 ceph-mon[32398]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:44.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:44 smithi078 ceph-mon[36683]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:46.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:46 smithi175 ceph-mon[36636]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:46.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:46 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-10T00:56:46.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:46 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:56:46.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:46 smithi007 ceph-mon[32398]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:46.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:46 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-10T00:56:46.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:46 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:56:46.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:46 smithi078 ceph-mon[36683]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:46.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:46 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-10T00:56:46.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:46 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:56:47.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:47 smithi175 ceph-mon[36636]: Deploying daemon osd.2 on smithi175 2024-06-10T00:56:47.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:47 smithi007 ceph-mon[32398]: Deploying daemon osd.2 on smithi175 2024-06-10T00:56:47.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:47 smithi078 ceph-mon[36683]: Deploying daemon osd.2 on smithi175 2024-06-10T00:56:48.654 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:48 smithi175 ceph-mon[36636]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:48.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:48 smithi007 ceph-mon[32398]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:48.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:48 smithi078 ceph-mon[36683]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:50.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:50 smithi007 ceph-mon[32398]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:50.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:50 smithi078 ceph-mon[36683]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:50.886 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:50 smithi175 ceph-mon[36636]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:52.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:52 smithi175 ceph-mon[36636]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:52.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:52 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:52.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:52 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:52.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:52 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:56:53.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:52 smithi007 ceph-mon[32398]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:53.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:52 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:53.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:52 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:53.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:52 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:56:53.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:52 smithi078 ceph-mon[36683]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:53.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:52 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:53.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:52 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:53.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:52 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:56:54.823 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:54 smithi175 ceph-mon[36636]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:54.824 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:54.823+0000 7f3536ffd700 1 -- 172.21.15.175:0/4032309556 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f35400fe2b0 con 0x7f352c0a2c10 2024-06-10T00:56:54.824 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 2 on host 'smithi175' 2024-06-10T00:56:54.826 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:54.825+0000 7f35481ab700 1 -- 172.21.15.175:0/4032309556 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f352c0a2c10 msgr2=0x7f352c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:54.826 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:54.825+0000 7f35481ab700 1 --2- 172.21.15.175:0/4032309556 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f352c0a2c10 0x7f352c0a50d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3530005fd0 tx=0x7f3530005dc0 comp rx=0 tx=0).stop 2024-06-10T00:56:54.826 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:54.825+0000 7f35481ab700 1 -- 172.21.15.175:0/4032309556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3540100b80 msgr2=0x7f3540198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:54.826 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:54.825+0000 7f35481ab700 1 --2- 172.21.15.175:0/4032309556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3540100b80 0x7f3540198740 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f353c0047b0 tx=0x7f353c009f90 comp rx=0 tx=0).stop 2024-06-10T00:56:54.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:54.825+0000 7f35481ab700 1 -- 172.21.15.175:0/4032309556 shutdown_connections 2024-06-10T00:56:54.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:54.825+0000 7f35481ab700 1 --2- 172.21.15.175:0/4032309556 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f352c0a2c10 0x7f352c0a50d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:54.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:54.825+0000 7f35481ab700 1 --2- 172.21.15.175:0/4032309556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3540102720 0x7f354019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:54.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:54.825+0000 7f35481ab700 1 --2- 172.21.15.175:0/4032309556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3540100b80 0x7f3540198740 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:54.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:54.825+0000 7f35481ab700 1 --2- 172.21.15.175:0/4032309556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3540101d80 0x7f3540198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:54.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:54.825+0000 7f35481ab700 1 -- 172.21.15.175:0/4032309556 >> 172.21.15.175:0/4032309556 conn(0x7f35400fc100 msgr2=0x7f35400fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:54.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:54.825+0000 7f35481ab700 1 -- 172.21.15.175:0/4032309556 shutdown_connections 2024-06-10T00:56:54.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:56:54.826+0000 7f35481ab700 1 -- 172.21.15.175:0/4032309556 wait complete. 2024-06-10T00:56:55.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:54 smithi007 ceph-mon[32398]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:55.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:54 smithi078 ceph-mon[36683]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:55.438 DEBUG:teuthology.orchestra.run.smithi175:osd.2> sudo journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.2.service 2024-06-10T00:56:55.441 INFO:tasks.cephadm:Deploying osd.3 on smithi007 with /dev/vg_nvme/lv_3... 2024-06-10T00:56:55.441 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-10T00:56:55.700 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:56:55.708 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jun 10 00:56:55 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-2[41725]: 2024-06-10T00:56:55.585+0000 7f5e283527c0 -1 osd.2 0 log_to_monitors true 2024-06-10T00:56:56.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:55 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:56.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:55 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:56.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:55 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:56.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:55 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:56.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:55 smithi078 ceph-mon[36683]: from='osd.2 [v2:172.21.15.175:6800/918257455,v1:172.21.15.175:6801/918257455]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-10T00:56:56.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:55 smithi078 ceph-mon[36683]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-10T00:56:56.127 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:55 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:56.128 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:55 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:56.128 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:55 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:56.128 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:55 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:56.128 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:55 smithi007 ceph-mon[32398]: from='osd.2 [v2:172.21.15.175:6800/918257455,v1:172.21.15.175:6801/918257455]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-10T00:56:56.128 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:55 smithi007 ceph-mon[32398]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-10T00:56:56.179 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:55 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:56.180 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:55 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:56.180 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:55 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:56.180 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:55 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:56:56.180 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:55 smithi175 ceph-mon[36636]: from='osd.2 [v2:172.21.15.175:6800/918257455,v1:172.21.15.175:6801/918257455]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-10T00:56:56.180 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:55 smithi175 ceph-mon[36636]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-10T00:56:57.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:56 smithi078 ceph-mon[36683]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:57.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:56 smithi078 ceph-mon[36683]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-10T00:56:57.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:56 smithi078 ceph-mon[36683]: osdmap e16: 3 total, 2 up, 3 in 2024-06-10T00:56:57.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:56 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:56:57.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:56 smithi078 ceph-mon[36683]: from='osd.2 [v2:172.21.15.175:6800/918257455,v1:172.21.15.175:6801/918257455]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-10T00:56:57.086 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:56 smithi078 ceph-mon[36683]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-10T00:56:57.097 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:56 smithi007 ceph-mon[32398]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:57.097 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:56 smithi007 ceph-mon[32398]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-10T00:56:57.098 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:56 smithi007 ceph-mon[32398]: osdmap e16: 3 total, 2 up, 3 in 2024-06-10T00:56:57.098 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:56 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:56:57.098 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:56 smithi007 ceph-mon[32398]: from='osd.2 [v2:172.21.15.175:6800/918257455,v1:172.21.15.175:6801/918257455]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-10T00:56:57.098 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:56 smithi007 ceph-mon[32398]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-10T00:56:57.118 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:56 smithi175 ceph-mon[36636]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:57.118 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:56 smithi175 ceph-mon[36636]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-10T00:56:57.118 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:56 smithi175 ceph-mon[36636]: osdmap e16: 3 total, 2 up, 3 in 2024-06-10T00:56:57.118 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:56 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:56:57.118 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:56 smithi175 ceph-mon[36636]: from='osd.2 [v2:172.21.15.175:6800/918257455,v1:172.21.15.175:6801/918257455]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-10T00:56:57.118 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:56 smithi175 ceph-mon[36636]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-10T00:56:57.692 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-10T00:56:57.692 INFO:teuthology.orchestra.run.smithi007.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-10T00:56:57.692 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10+0 records in 2024-06-10T00:56:57.693 INFO:teuthology.orchestra.run.smithi007.stderr:10+0 records out 2024-06-10T00:56:57.693 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00832488 s, 1.3 GB/s 2024-06-10T00:56:57.693 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping successful for: 2024-06-10T00:56:57.708 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jun 10 00:56:57 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-2[41725]: 2024-06-10T00:56:57.599+0000 7f5e1d9b6700 -1 osd.2 0 waiting for initial osdmap 2024-06-10T00:56:57.708 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jun 10 00:56:57 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-2[41725]: 2024-06-10T00:56:57.604+0000 7f5e197a9700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:56:58.317 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi007:vg_nvme/lv_3 2024-06-10T00:56:58.567 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:56:58.759 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:58 smithi175 ceph-mon[36636]: purged_snaps scrub starts 2024-06-10T00:56:58.760 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:58 smithi175 ceph-mon[36636]: purged_snaps scrub ok 2024-06-10T00:56:58.760 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:58 smithi175 ceph-mon[36636]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:58.760 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:58 smithi175 ceph-mon[36636]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-06-10T00:56:58.760 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:58 smithi175 ceph-mon[36636]: osdmap e17: 3 total, 2 up, 3 in 2024-06-10T00:56:58.760 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:58 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:56:58.760 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:58 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:56:58.957 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:58 smithi007 ceph-mon[32398]: purged_snaps scrub starts 2024-06-10T00:56:58.957 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:58 smithi007 ceph-mon[32398]: purged_snaps scrub ok 2024-06-10T00:56:58.957 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:58 smithi007 ceph-mon[32398]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:58.957 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:58 smithi007 ceph-mon[32398]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-06-10T00:56:58.958 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:58 smithi007 ceph-mon[32398]: osdmap e17: 3 total, 2 up, 3 in 2024-06-10T00:56:58.958 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:58 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:56:58.958 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:58 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:56:59.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:58 smithi078 ceph-mon[36683]: purged_snaps scrub starts 2024-06-10T00:56:59.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:58 smithi078 ceph-mon[36683]: purged_snaps scrub ok 2024-06-10T00:56:59.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:58 smithi078 ceph-mon[36683]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:56:59.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:58 smithi078 ceph-mon[36683]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-06-10T00:56:59.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:58 smithi078 ceph-mon[36683]: osdmap e17: 3 total, 2 up, 3 in 2024-06-10T00:56:59.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:58 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:56:59.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:58 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:57:00.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:59 smithi007 ceph-mon[32398]: osd.2 [v2:172.21.15.175:6800/918257455,v1:172.21.15.175:6801/918257455] boot 2024-06-10T00:57:00.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:59 smithi007 ceph-mon[32398]: osdmap e18: 3 total, 3 up, 3 in 2024-06-10T00:57:00.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:59 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:57:00.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:59 smithi007 ceph-mon[32398]: Detected new or changed devices on smithi175 2024-06-10T00:57:00.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:59 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:00.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:59 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:00.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:59 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:57:00.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:59 smithi007 ceph-mon[32398]: Adjusting osd_memory_target on smithi175 to 17088M 2024-06-10T00:57:00.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:59 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:00.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:59 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:00.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:59 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:57:00.020 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:59 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:00.020 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:59 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:57:00.020 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:59 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:00.020 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:59 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:57:00.020 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:59 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:00.020 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:56:59 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-10T00:57:00.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.281+0000 7f1a77c45700 1 -- 172.21.15.7:0/1531724256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a70100d70 msgr2=0x7f1a70101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:00.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.281+0000 7f1a77c45700 1 --2- 172.21.15.7:0/1531724256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a70100d70 0x7f1a70101190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1a6000b0a0 tx=0x7f1a6000b3b0 comp rx=0 tx=0).stop 2024-06-10T00:57:00.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.281+0000 7f1a77c45700 1 -- 172.21.15.7:0/1531724256 shutdown_connections 2024-06-10T00:57:00.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.281+0000 7f1a77c45700 1 --2- 172.21.15.7:0/1531724256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a70102090 0x7f1a70102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:00.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.281+0000 7f1a77c45700 1 --2- 172.21.15.7:0/1531724256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a70100d70 0x7f1a70101190 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:00.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.281+0000 7f1a77c45700 1 --2- 172.21.15.7:0/1531724256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a701016d0 0x7f1a70101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:00.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.281+0000 7f1a77c45700 1 -- 172.21.15.7:0/1531724256 >> 172.21.15.7:0/1531724256 conn(0x7f1a700fc990 msgr2=0x7f1a700fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:00.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.282+0000 7f1a77c45700 1 -- 172.21.15.7:0/1531724256 shutdown_connections 2024-06-10T00:57:00.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.282+0000 7f1a77c45700 1 -- 172.21.15.7:0/1531724256 wait complete. 2024-06-10T00:57:00.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.282+0000 7f1a77c45700 1 Processor -- start 2024-06-10T00:57:00.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.282+0000 7f1a77c45700 1 -- start start 2024-06-10T00:57:00.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.282+0000 7f1a77c45700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a70100d70 0x7f1a7019a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:00.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.282+0000 7f1a77c45700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a701016d0 0x7f1a7019ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:00.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.282+0000 7f1a77c45700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a70102090 0x7f1a701a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:00.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.282+0000 7f1a77c45700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1a701102b0 con 0x7f1a701016d0 2024-06-10T00:57:00.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.282+0000 7f1a77c45700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1a70110130 con 0x7f1a70102090 2024-06-10T00:57:00.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.282+0000 7f1a77c45700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1a70110430 con 0x7f1a70100d70 2024-06-10T00:57:00.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.282+0000 7f1a751e0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a701016d0 0x7f1a7019ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:00.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.283+0000 7f1a751e0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a701016d0 0x7f1a7019ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55844/0 (socket says 172.21.15.7:55844) 2024-06-10T00:57:00.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.283+0000 7f1a751e0700 1 -- 172.21.15.7:0/3606892717 learned_addr learned my addr 172.21.15.7:0/3606892717 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:57:00.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.283+0000 7f1a759e1700 1 --2- 172.21.15.7:0/3606892717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a70100d70 0x7f1a7019a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:00.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.283+0000 7f1a761e2700 1 --2- 172.21.15.7:0/3606892717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a70102090 0x7f1a701a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:00.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.283+0000 7f1a751e0700 1 -- 172.21.15.7:0/3606892717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a70100d70 msgr2=0x7f1a7019a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:00.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.283+0000 7f1a751e0700 1 --2- 172.21.15.7:0/3606892717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a70100d70 0x7f1a7019a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:00.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.283+0000 7f1a751e0700 1 -- 172.21.15.7:0/3606892717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a70102090 msgr2=0x7f1a701a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:00.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.283+0000 7f1a751e0700 1 --2- 172.21.15.7:0/3606892717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a70102090 0x7f1a701a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:00.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.283+0000 7f1a751e0700 1 -- 172.21.15.7:0/3606892717 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a701a25b0 con 0x7f1a701016d0 2024-06-10T00:57:00.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.283+0000 7f1a761e2700 1 --2- 172.21.15.7:0/3606892717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a70102090 0x7f1a701a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:57:00.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.283+0000 7f1a751e0700 1 --2- 172.21.15.7:0/3606892717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a701016d0 0x7f1a7019ae90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1a6400d9b0 tx=0x7f1a6400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:00.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.283+0000 7f1a6effd700 1 -- 172.21.15.7:0/3606892717 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a64017070 con 0x7f1a701016d0 2024-06-10T00:57:00.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.283+0000 7f1a6effd700 1 -- 172.21.15.7:0/3606892717 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a64005020 con 0x7f1a701016d0 2024-06-10T00:57:00.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.283+0000 7f1a77c45700 1 -- 172.21.15.7:0/3606892717 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a701a28a0 con 0x7f1a701016d0 2024-06-10T00:57:00.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.283+0000 7f1a6effd700 1 -- 172.21.15.7:0/3606892717 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a64009bd0 con 0x7f1a701016d0 2024-06-10T00:57:00.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.283+0000 7f1a77c45700 1 -- 172.21.15.7:0/3606892717 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a701a2d90 con 0x7f1a701016d0 2024-06-10T00:57:00.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.284+0000 7f1a77c45700 1 -- 172.21.15.7:0/3606892717 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a70069180 con 0x7f1a701016d0 2024-06-10T00:57:00.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.286+0000 7f1a6effd700 1 -- 172.21.15.7:0/3606892717 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7f1a640051d0 con 0x7f1a701016d0 2024-06-10T00:57:00.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.287+0000 7f1a6effd700 1 --2- 172.21.15.7:0/3606892717 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1a580a2b90 0x7f1a580a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:00.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.287+0000 7f1a759e1700 1 --2- 172.21.15.7:0/3606892717 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1a580a2b90 0x7f1a580a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:00.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.287+0000 7f1a6effd700 1 -- 172.21.15.7:0/3606892717 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7f1a640cdef0 con 0x7f1a701016d0 2024-06-10T00:57:00.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.287+0000 7f1a759e1700 1 --2- 172.21.15.7:0/3606892717 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1a580a2b90 0x7f1a580a5050 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1a6000b070 tx=0x7f1a6000b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:00.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.291+0000 7f1a6effd700 1 -- 172.21.15.7:0/3606892717 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1a64020020 con 0x7f1a701016d0 2024-06-10T00:57:00.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:59 smithi078 ceph-mon[36683]: osd.2 [v2:172.21.15.175:6800/918257455,v1:172.21.15.175:6801/918257455] boot 2024-06-10T00:57:00.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:59 smithi078 ceph-mon[36683]: osdmap e18: 3 total, 3 up, 3 in 2024-06-10T00:57:00.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:59 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:57:00.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:59 smithi078 ceph-mon[36683]: Detected new or changed devices on smithi175 2024-06-10T00:57:00.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:59 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:00.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:59 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:00.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:59 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:57:00.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:59 smithi078 ceph-mon[36683]: Adjusting osd_memory_target on smithi175 to 17088M 2024-06-10T00:57:00.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:59 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:00.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:59 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:00.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:59 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:57:00.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:59 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:00.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:59 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:57:00.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:59 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:00.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:59 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:57:00.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:59 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:00.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:56:59 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-10T00:57:00.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:59 smithi175 ceph-mon[36636]: osd.2 [v2:172.21.15.175:6800/918257455,v1:172.21.15.175:6801/918257455] boot 2024-06-10T00:57:00.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:59 smithi175 ceph-mon[36636]: osdmap e18: 3 total, 3 up, 3 in 2024-06-10T00:57:00.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:59 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:57:00.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:59 smithi175 ceph-mon[36636]: Detected new or changed devices on smithi175 2024-06-10T00:57:00.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:59 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:00.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:59 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:00.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:59 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:57:00.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:59 smithi175 ceph-mon[36636]: Adjusting osd_memory_target on smithi175 to 17088M 2024-06-10T00:57:00.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:59 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:00.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:59 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:00.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:59 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:57:00.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:59 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:00.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:59 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:57:00.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:59 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:00.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:59 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:57:00.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:59 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:00.460 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:56:59 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-10T00:57:00.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:00.510+0000 7f1a77c45700 1 -- 172.21.15.7:0/3606892717 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f1a700628d0 con 0x7f1a580a2b90 2024-06-10T00:57:01.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:00 smithi007 ceph-mon[32398]: pgmap v80: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:57:01.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:00 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-10T00:57:01.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:00 smithi007 ceph-mon[32398]: osdmap e19: 3 total, 3 up, 3 in 2024-06-10T00:57:01.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:00 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-10T00:57:01.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:00 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:57:01.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:00 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:57:01.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:00 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:01.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:00 smithi078 ceph-mon[36683]: pgmap v80: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:57:01.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:00 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-10T00:57:01.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:00 smithi078 ceph-mon[36683]: osdmap e19: 3 total, 3 up, 3 in 2024-06-10T00:57:01.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:00 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-10T00:57:01.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:00 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:57:01.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:00 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:57:01.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:00 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:01.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:00 smithi175 ceph-mon[36636]: pgmap v80: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:57:01.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:00 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-10T00:57:01.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:00 smithi175 ceph-mon[36636]: osdmap e19: 3 total, 3 up, 3 in 2024-06-10T00:57:01.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:00 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-10T00:57:01.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:00 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:57:01.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:00 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:57:01.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:00 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:01.980 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 10 00:57:01 smithi007 sudo[51963]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-10T00:57:01.980 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 10 00:57:01 smithi007 sudo[51963]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:57:01.980 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 10 00:57:01 smithi007 sudo[51963]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:57:01.980 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 10 00:57:01 smithi007 sudo[51963]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:57:01.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:01.993+0000 7f1a6effd700 1 -- 172.21.15.7:0/3606892717 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f1a640981e0 con 0x7f1a701016d0 2024-06-10T00:57:02.249 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:01 smithi078 ceph-mon[36683]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:57:02.249 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:01 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-10T00:57:02.249 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:01 smithi078 ceph-mon[36683]: osdmap e20: 3 total, 3 up, 3 in 2024-06-10T00:57:02.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:01 smithi007 ceph-mon[32398]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:57:02.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:01 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-10T00:57:02.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:01 smithi007 ceph-mon[32398]: osdmap e20: 3 total, 3 up, 3 in 2024-06-10T00:57:02.269 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 10 00:57:01 smithi007 sudo[51969]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-10T00:57:02.269 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 10 00:57:01 smithi007 sudo[51969]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:57:02.269 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 10 00:57:01 smithi007 sudo[51969]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:57:02.269 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 10 00:57:01 smithi007 sudo[51969]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:57:02.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:01 smithi175 ceph-mon[36636]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:57:02.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:01 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-10T00:57:02.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:01 smithi175 ceph-mon[36636]: osdmap e20: 3 total, 3 up, 3 in 2024-06-10T00:57:02.503 INFO:journalctl@ceph.osd.1.smithi078.stdout:Jun 10 00:57:02 smithi078 sudo[45077]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-10T00:57:02.503 INFO:journalctl@ceph.osd.1.smithi078.stdout:Jun 10 00:57:02 smithi078 sudo[45077]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:57:02.503 INFO:journalctl@ceph.osd.1.smithi078.stdout:Jun 10 00:57:02 smithi078 sudo[45077]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:57:02.503 INFO:journalctl@ceph.osd.1.smithi078.stdout:Jun 10 00:57:02 smithi078 sudo[45077]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:57:02.835 INFO:journalctl@ceph.osd.1.smithi078.stdout:Jun 10 00:57:02 smithi078 sudo[45080]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-10T00:57:02.835 INFO:journalctl@ceph.osd.1.smithi078.stdout:Jun 10 00:57:02 smithi078 sudo[45080]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:57:02.835 INFO:journalctl@ceph.osd.1.smithi078.stdout:Jun 10 00:57:02 smithi078 sudo[45080]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:57:02.835 INFO:journalctl@ceph.osd.1.smithi078.stdout:Jun 10 00:57:02 smithi078 sudo[45080]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:57:03.024 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jun 10 00:57:02 smithi175 sudo[45323]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-10T00:57:03.025 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jun 10 00:57:02 smithi175 sudo[45323]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:57:03.025 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jun 10 00:57:02 smithi175 sudo[45323]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:57:03.025 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jun 10 00:57:02 smithi175 sudo[45323]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:57:03.025 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:02 smithi175 ceph-mon[36636]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:57:03.025 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:02 smithi175 ceph-mon[36636]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-06-10T00:57:03.025 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:02 smithi175 ceph-mon[36636]: osdmap e21: 3 total, 3 up, 3 in 2024-06-10T00:57:03.025 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:02 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:03.025 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:02 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:03.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:02 smithi007 ceph-mon[32398]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:57:03.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:02 smithi007 ceph-mon[32398]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-06-10T00:57:03.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:02 smithi007 ceph-mon[32398]: osdmap e21: 3 total, 3 up, 3 in 2024-06-10T00:57:03.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:02 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:03.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:02 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:03.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:02 smithi078 ceph-mon[36683]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:57:03.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:02 smithi078 ceph-mon[36683]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-06-10T00:57:03.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:02 smithi078 ceph-mon[36683]: osdmap e21: 3 total, 3 up, 3 in 2024-06-10T00:57:03.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:02 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:03.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:02 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:03.458 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jun 10 00:57:03 smithi175 sudo[45326]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-10T00:57:03.459 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jun 10 00:57:03 smithi175 sudo[45326]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:57:03.459 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jun 10 00:57:03 smithi175 sudo[45326]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:57:03.459 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jun 10 00:57:03 smithi175 sudo[45326]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:57:03.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:03 smithi007 sudo[52044]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-10T00:57:03.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:03 smithi007 sudo[52044]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:57:03.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:03 smithi007 sudo[52044]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:57:03.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:03 smithi007 sudo[52044]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:57:04.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:03 smithi007 sudo[52047]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-10T00:57:04.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:03 smithi007 sudo[52047]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:57:04.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:03 smithi007 sudo[52047]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:57:04.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:03 smithi007 sudo[52047]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:57:04.434 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:04 smithi175 sudo[45329]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-10T00:57:04.434 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:04 smithi175 sudo[45329]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:57:04.434 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:04 smithi175 sudo[45329]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:57:04.434 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:04 smithi175 sudo[45329]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:57:04.434 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:04 smithi175 sudo[45332]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-10T00:57:04.435 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:04 smithi175 sudo[45332]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:57:04.435 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:04 smithi175 sudo[45332]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:57:04.435 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:04 smithi175 sudo[45332]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:57:04.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:04 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:04.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:04 smithi175 ceph-mon[36636]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T00:57:04.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:04 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/4162997954' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65242098-e748-48ab-8f0e-bb25b5c62742"}]: dispatch 2024-06-10T00:57:04.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:04 smithi175 ceph-mon[36636]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T00:57:04.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:04 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:57:04.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:04 smithi175 ceph-mon[36636]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65242098-e748-48ab-8f0e-bb25b5c62742"}]: dispatch 2024-06-10T00:57:04.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:04 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:57:04.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:04 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:57:04.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:04 smithi175 ceph-mon[36636]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65242098-e748-48ab-8f0e-bb25b5c62742"}]': finished 2024-06-10T00:57:04.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:04 smithi175 ceph-mon[36636]: osdmap e22: 4 total, 3 up, 4 in 2024-06-10T00:57:04.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:04 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:57:04.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:04 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:57:04.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:04 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:57:04.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:04 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:57:04.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:04 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:04.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:04 smithi007 ceph-mon[32398]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T00:57:04.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:04 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/4162997954' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65242098-e748-48ab-8f0e-bb25b5c62742"}]: dispatch 2024-06-10T00:57:04.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:04 smithi007 ceph-mon[32398]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T00:57:04.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:04 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:57:04.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:04 smithi007 ceph-mon[32398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65242098-e748-48ab-8f0e-bb25b5c62742"}]: dispatch 2024-06-10T00:57:04.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:04 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:57:04.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:04 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:57:04.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:04 smithi007 ceph-mon[32398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65242098-e748-48ab-8f0e-bb25b5c62742"}]': finished 2024-06-10T00:57:04.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:04 smithi007 ceph-mon[32398]: osdmap e22: 4 total, 3 up, 4 in 2024-06-10T00:57:04.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:04 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:57:04.770 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:04 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:57:04.770 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:04 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:57:04.770 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:04 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:57:04.925 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:04 smithi078 sudo[45083]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-10T00:57:04.926 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:04 smithi078 sudo[45083]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:57:04.926 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:04 smithi078 sudo[45083]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:57:04.926 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:04 smithi078 sudo[45083]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:57:05.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:04 smithi078 sudo[45086]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-10T00:57:05.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:04 smithi078 sudo[45086]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:57:05.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:04 smithi078 sudo[45086]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:57:05.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:04 smithi078 sudo[45086]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:57:05.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:04 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:05.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:04 smithi078 ceph-mon[36683]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T00:57:05.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:04 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/4162997954' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65242098-e748-48ab-8f0e-bb25b5c62742"}]: dispatch 2024-06-10T00:57:05.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:04 smithi078 ceph-mon[36683]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T00:57:05.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:04 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:57:05.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:04 smithi078 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65242098-e748-48ab-8f0e-bb25b5c62742"}]: dispatch 2024-06-10T00:57:05.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:04 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:57:05.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:04 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:57:05.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:04 smithi078 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65242098-e748-48ab-8f0e-bb25b5c62742"}]': finished 2024-06-10T00:57:05.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:04 smithi078 ceph-mon[36683]: osdmap e22: 4 total, 3 up, 4 in 2024-06-10T00:57:05.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:04 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:57:05.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:04 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:57:05.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:04 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:57:05.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:04 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:57:05.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:05 smithi175 ceph-mon[36636]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:57:05.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:05 smithi175 ceph-mon[36636]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T00:57:05.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:05 smithi175 ceph-mon[36636]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T00:57:05.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:05 smithi175 ceph-mon[36636]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T00:57:05.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:05 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:57:05.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:05 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:57:05.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:05 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:57:05.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:05 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2870958598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:57:05.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:05 smithi175 ceph-mon[36636]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T00:57:05.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:05 smithi007 ceph-mon[32398]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:57:05.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:05 smithi007 ceph-mon[32398]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T00:57:05.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:05 smithi007 ceph-mon[32398]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T00:57:05.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:05 smithi007 ceph-mon[32398]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T00:57:05.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:05 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:57:05.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:05 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:57:05.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:05 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:57:05.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:05 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2870958598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:57:05.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:05 smithi007 ceph-mon[32398]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T00:57:05.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:05 smithi078 ceph-mon[36683]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:57:05.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:05 smithi078 ceph-mon[36683]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T00:57:05.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:05 smithi078 ceph-mon[36683]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T00:57:05.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:05 smithi078 ceph-mon[36683]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T00:57:05.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:05 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:57:05.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:05 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:57:05.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:05 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:57:05.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:05 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2870958598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:57:05.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:05 smithi078 ceph-mon[36683]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T00:57:06.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:06 smithi175 ceph-mon[36636]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:57:06.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:06 smithi007 ceph-mon[32398]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:57:06.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:06 smithi078 ceph-mon[36683]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:57:08.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:08 smithi078 ceph-mon[36683]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:57:08.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:08 smithi175 ceph-mon[36636]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:57:08.988 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:08 smithi007 ceph-mon[32398]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:57:10.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:10 smithi078 ceph-mon[36683]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:57:10.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:10 smithi175 ceph-mon[36636]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:57:11.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:10 smithi007 ceph-mon[32398]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:57:11.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:11 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-10T00:57:11.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:11 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:11.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:11 smithi078 ceph-mon[36683]: Deploying daemon osd.3 on smithi007 2024-06-10T00:57:11.950 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:11 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-10T00:57:11.950 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:11 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:11.950 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:11 smithi007 ceph-mon[32398]: Deploying daemon osd.3 on smithi007 2024-06-10T00:57:11.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:11 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-10T00:57:11.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:11 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:11.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:11 smithi175 ceph-mon[36636]: Deploying daemon osd.3 on smithi007 2024-06-10T00:57:12.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:12 smithi078 ceph-mon[36683]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-06-10T00:57:12.930 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:12 smithi007 ceph-mon[32398]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-06-10T00:57:12.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:12 smithi175 ceph-mon[36636]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-06-10T00:57:14.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:14 smithi007 ceph-mon[32398]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-06-10T00:57:14.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:14 smithi078 ceph-mon[36683]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-06-10T00:57:14.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:14 smithi175 ceph-mon[36636]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-06-10T00:57:16.747 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:16 smithi007 ceph-mon[32398]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-06-10T00:57:16.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:16 smithi078 ceph-mon[36683]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-06-10T00:57:16.948 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:16 smithi175 ceph-mon[36636]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-06-10T00:57:18.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:17 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:18.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:17 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:18.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:17 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:57:18.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:17 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:18.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:17 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:18.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:17 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:57:18.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:17 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:18.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:17 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:18.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:17 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:57:19.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:18 smithi175 ceph-mon[36636]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-10T00:57:19.222 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:18 smithi007 ceph-mon[32398]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-10T00:57:19.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:18 smithi078 ceph-mon[36683]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-10T00:57:20.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:20.922+0000 7f1a6effd700 1 -- 172.21.15.7:0/3606892717 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1a700628d0 con 0x7f1a580a2b90 2024-06-10T00:57:20.923 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 3 on host 'smithi007' 2024-06-10T00:57:20.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:20.924+0000 7f1a77c45700 1 -- 172.21.15.7:0/3606892717 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1a580a2b90 msgr2=0x7f1a580a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:20.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:20.924+0000 7f1a77c45700 1 --2- 172.21.15.7:0/3606892717 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1a580a2b90 0x7f1a580a5050 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1a6000b070 tx=0x7f1a6000b7b0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:20.924+0000 7f1a77c45700 1 -- 172.21.15.7:0/3606892717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a701016d0 msgr2=0x7f1a7019ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:20.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:20.924+0000 7f1a77c45700 1 --2- 172.21.15.7:0/3606892717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a701016d0 0x7f1a7019ae90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1a6400d9b0 tx=0x7f1a6400dcc0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:20.924+0000 7f1a77c45700 1 -- 172.21.15.7:0/3606892717 shutdown_connections 2024-06-10T00:57:20.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:20.924+0000 7f1a77c45700 1 --2- 172.21.15.7:0/3606892717 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1a580a2b90 0x7f1a580a5050 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:20.924+0000 7f1a77c45700 1 --2- 172.21.15.7:0/3606892717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a70102090 0x7f1a701a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:20.924+0000 7f1a77c45700 1 --2- 172.21.15.7:0/3606892717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a70100d70 0x7f1a7019a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:20.924+0000 7f1a77c45700 1 --2- 172.21.15.7:0/3606892717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a701016d0 0x7f1a7019ae90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:20.924+0000 7f1a77c45700 1 -- 172.21.15.7:0/3606892717 >> 172.21.15.7:0/3606892717 conn(0x7f1a700fc990 msgr2=0x7f1a700fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:20.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:20.924+0000 7f1a77c45700 1 -- 172.21.15.7:0/3606892717 shutdown_connections 2024-06-10T00:57:20.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:57:20.924+0000 7f1a77c45700 1 -- 172.21.15.7:0/3606892717 wait complete. 2024-06-10T00:57:21.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:20 smithi175 ceph-mon[36636]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-10T00:57:21.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:21.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:21.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:21.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:57:21.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:21.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:20 smithi175 ceph-mon[36636]: from='osd.3 [v2:172.21.15.7:6810/1797404966,v1:172.21.15.7:6811/1797404966]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-10T00:57:21.252 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:20 smithi007 ceph-mon[32398]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-10T00:57:21.252 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:20 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:21.252 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:20 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:21.253 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:20 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:21.253 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:20 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:57:21.253 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:20 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:21.253 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:20 smithi007 ceph-mon[32398]: from='osd.3 [v2:172.21.15.7:6810/1797404966,v1:172.21.15.7:6811/1797404966]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-10T00:57:21.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:20 smithi078 ceph-mon[36683]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-10T00:57:21.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:20 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:21.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:20 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:21.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:20 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:21.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:20 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:57:21.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:20 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:21.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:20 smithi078 ceph-mon[36683]: from='osd.3 [v2:172.21.15.7:6810/1797404966,v1:172.21.15.7:6811/1797404966]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-10T00:57:21.582 DEBUG:teuthology.orchestra.run.smithi007:osd.3> sudo journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.3.service 2024-06-10T00:57:21.584 INFO:tasks.cephadm:Deploying osd.4 on smithi078 with /dev/vg_nvme/lv_3... 2024-06-10T00:57:21.584 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-10T00:57:21.827 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T00:57:22.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:21 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:22.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:21 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:22.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:21 smithi175 ceph-mon[36636]: from='osd.3 [v2:172.21.15.7:6810/1797404966,v1:172.21.15.7:6811/1797404966]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-10T00:57:22.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:21 smithi175 ceph-mon[36636]: osdmap e23: 4 total, 3 up, 4 in 2024-06-10T00:57:22.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:21 smithi175 ceph-mon[36636]: from='osd.3 [v2:172.21.15.7:6810/1797404966,v1:172.21.15.7:6811/1797404966]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-10T00:57:22.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:21 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:57:22.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:21 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:57:22.214 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:21 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:22.214 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:21 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:22.215 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:21 smithi078 ceph-mon[36683]: from='osd.3 [v2:172.21.15.7:6810/1797404966,v1:172.21.15.7:6811/1797404966]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-10T00:57:22.215 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:21 smithi078 ceph-mon[36683]: osdmap e23: 4 total, 3 up, 4 in 2024-06-10T00:57:22.215 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:21 smithi078 ceph-mon[36683]: from='osd.3 [v2:172.21.15.7:6810/1797404966,v1:172.21.15.7:6811/1797404966]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-10T00:57:22.215 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:21 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:57:22.215 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:21 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:57:22.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:21 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:22.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:21 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:22.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:21 smithi007 ceph-mon[32398]: from='osd.3 [v2:172.21.15.7:6810/1797404966,v1:172.21.15.7:6811/1797404966]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-10T00:57:22.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:21 smithi007 ceph-mon[32398]: osdmap e23: 4 total, 3 up, 4 in 2024-06-10T00:57:22.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:21 smithi007 ceph-mon[32398]: from='osd.3 [v2:172.21.15.7:6810/1797404966,v1:172.21.15.7:6811/1797404966]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-10T00:57:22.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:21 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:57:22.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:21 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:57:22.269 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 10 00:57:21 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-3[54076]: 2024-06-10T00:57:21.927+0000 7f64cbdac700 -1 osd.3 0 waiting for initial osdmap 2024-06-10T00:57:22.270 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 10 00:57:21 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-3[54076]: 2024-06-10T00:57:21.931+0000 7f64c7b9f700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:57:23.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:22 smithi007 ceph-mon[32398]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:57:23.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:22 smithi007 ceph-mon[32398]: from='osd.3 [v2:172.21.15.7:6810/1797404966,v1:172.21.15.7:6811/1797404966]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-06-10T00:57:23.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:22 smithi007 ceph-mon[32398]: osdmap e24: 4 total, 3 up, 4 in 2024-06-10T00:57:23.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:22 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:57:23.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:22 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:57:23.196 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:22 smithi078 ceph-mon[36683]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:57:23.197 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:22 smithi078 ceph-mon[36683]: from='osd.3 [v2:172.21.15.7:6810/1797404966,v1:172.21.15.7:6811/1797404966]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-06-10T00:57:23.197 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:22 smithi078 ceph-mon[36683]: osdmap e24: 4 total, 3 up, 4 in 2024-06-10T00:57:23.197 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:22 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:57:23.197 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:22 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:57:23.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:22 smithi175 ceph-mon[36636]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:57:23.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:22 smithi175 ceph-mon[36636]: from='osd.3 [v2:172.21.15.7:6810/1797404966,v1:172.21.15.7:6811/1797404966]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-06-10T00:57:23.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:22 smithi175 ceph-mon[36636]: osdmap e24: 4 total, 3 up, 4 in 2024-06-10T00:57:23.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:22 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:57:23.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:22 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:57:23.741 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-10T00:57:23.741 INFO:teuthology.orchestra.run.smithi078.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-10T00:57:23.741 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10+0 records in 2024-06-10T00:57:23.741 INFO:teuthology.orchestra.run.smithi078.stderr:10+0 records out 2024-06-10T00:57:23.741 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0116217 s, 902 MB/s 2024-06-10T00:57:23.741 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping successful for: 2024-06-10T00:57:24.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:23 smithi175 ceph-mon[36636]: purged_snaps scrub starts 2024-06-10T00:57:24.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:23 smithi175 ceph-mon[36636]: purged_snaps scrub ok 2024-06-10T00:57:24.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:23 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:57:24.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:23 smithi175 ceph-mon[36636]: osd.3 [v2:172.21.15.7:6810/1797404966,v1:172.21.15.7:6811/1797404966] boot 2024-06-10T00:57:24.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:23 smithi175 ceph-mon[36636]: osdmap e25: 4 total, 4 up, 4 in 2024-06-10T00:57:24.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:23 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:57:24.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:23 smithi007 ceph-mon[32398]: purged_snaps scrub starts 2024-06-10T00:57:24.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:23 smithi007 ceph-mon[32398]: purged_snaps scrub ok 2024-06-10T00:57:24.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:23 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:57:24.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:23 smithi007 ceph-mon[32398]: osd.3 [v2:172.21.15.7:6810/1797404966,v1:172.21.15.7:6811/1797404966] boot 2024-06-10T00:57:24.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:23 smithi007 ceph-mon[32398]: osdmap e25: 4 total, 4 up, 4 in 2024-06-10T00:57:24.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:23 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:57:24.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:23 smithi078 ceph-mon[36683]: purged_snaps scrub starts 2024-06-10T00:57:24.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:23 smithi078 ceph-mon[36683]: purged_snaps scrub ok 2024-06-10T00:57:24.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:23 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:57:24.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:23 smithi078 ceph-mon[36683]: osd.3 [v2:172.21.15.7:6810/1797404966,v1:172.21.15.7:6811/1797404966] boot 2024-06-10T00:57:24.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:23 smithi078 ceph-mon[36683]: osdmap e25: 4 total, 4 up, 4 in 2024-06-10T00:57:24.336 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:23 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:57:24.444 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_3 2024-06-10T00:57:24.691 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T00:57:25.036 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:24 smithi078 ceph-mon[36683]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:57:25.036 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:24 smithi078 ceph-mon[36683]: osdmap e26: 4 total, 4 up, 4 in 2024-06-10T00:57:25.036 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:24 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:25.036 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:24 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:25.036 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:24 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:57:25.036 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:24 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:57:25.036 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:24 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:25.036 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:24 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:25.037 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:24 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:57:25.037 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:24 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:25.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:24 smithi175 ceph-mon[36636]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:57:25.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:24 smithi175 ceph-mon[36636]: osdmap e26: 4 total, 4 up, 4 in 2024-06-10T00:57:25.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:24 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:25.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:24 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:25.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:24 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:57:25.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:24 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:57:25.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:24 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:25.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:24 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:25.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:24 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:57:25.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:24 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:25.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:24 smithi007 ceph-mon[32398]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:57:25.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:24 smithi007 ceph-mon[32398]: osdmap e26: 4 total, 4 up, 4 in 2024-06-10T00:57:25.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:24 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:25.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:24 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:25.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:24 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:57:25.270 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:24 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:57:25.270 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:24 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:25.270 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:24 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:25.270 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:24 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:57:25.270 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:24 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:26.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.169+0000 7f27ebaf8700 1 -- 172.21.15.78:0/3276945818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27e4104090 msgr2=0x7f27e410a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:26.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.169+0000 7f27ebaf8700 1 --2- 172.21.15.78:0/3276945818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27e4104090 0x7f27e410a940 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00b0d0 tx=0x7f27dc00b3e0 comp rx=0 tx=0).stop 2024-06-10T00:57:26.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.169+0000 7f27ebaf8700 1 -- 172.21.15.78:0/3276945818 shutdown_connections 2024-06-10T00:57:26.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.169+0000 7f27ebaf8700 1 --2- 172.21.15.78:0/3276945818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27e4104090 0x7f27e410a940 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:26.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.169+0000 7f27ebaf8700 1 --2- 172.21.15.78:0/3276945818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27e41024f0 0x7f27e4102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:26.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.169+0000 7f27ebaf8700 1 --2- 172.21.15.78:0/3276945818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27e41036f0 0x7f27e4103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:26.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.170+0000 7f27ebaf8700 1 -- 172.21.15.78:0/3276945818 >> 172.21.15.78:0/3276945818 conn(0x7f27e40fda90 msgr2=0x7f27e40ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:26.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.170+0000 7f27ebaf8700 1 -- 172.21.15.78:0/3276945818 shutdown_connections 2024-06-10T00:57:26.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.170+0000 7f27ebaf8700 1 -- 172.21.15.78:0/3276945818 wait complete. 2024-06-10T00:57:26.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.170+0000 7f27ebaf8700 1 Processor -- start 2024-06-10T00:57:26.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.170+0000 7f27ebaf8700 1 -- start start 2024-06-10T00:57:26.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.170+0000 7f27ebaf8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27e41024f0 0x7f27e4198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:26.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.170+0000 7f27ebaf8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27e41036f0 0x7f27e4198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:26.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.170+0000 7f27ebaf8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27e4104090 0x7f27e419fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:26.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.170+0000 7f27ebaf8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f27e410e0e0 con 0x7f27e4104090 2024-06-10T00:57:26.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.170+0000 7f27ebaf8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f27e410df60 con 0x7f27e41036f0 2024-06-10T00:57:26.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.170+0000 7f27ebaf8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f27e410e260 con 0x7f27e41024f0 2024-06-10T00:57:26.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.170+0000 7f27e9093700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27e41036f0 0x7f27e4198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:26.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.170+0000 7f27e9093700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27e41036f0 0x7f27e4198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39658/0 (socket says 172.21.15.78:39658) 2024-06-10T00:57:26.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.170+0000 7f27e9093700 1 -- 172.21.15.78:0/625941849 learned_addr learned my addr 172.21.15.78:0/625941849 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-06-10T00:57:26.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.171+0000 7f27ea095700 1 --2- 172.21.15.78:0/625941849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27e4104090 0x7f27e419fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:26.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.171+0000 7f27e9894700 1 --2- 172.21.15.78:0/625941849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27e41024f0 0x7f27e4198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:26.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.171+0000 7f27e9093700 1 -- 172.21.15.78:0/625941849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27e41024f0 msgr2=0x7f27e4198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:26.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.171+0000 7f27e9093700 1 --2- 172.21.15.78:0/625941849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27e41024f0 0x7f27e4198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:26.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.171+0000 7f27e9093700 1 -- 172.21.15.78:0/625941849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27e4104090 msgr2=0x7f27e419fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:26.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.171+0000 7f27e9093700 1 --2- 172.21.15.78:0/625941849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27e4104090 0x7f27e419fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:26.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.171+0000 7f27e9093700 1 -- 172.21.15.78:0/625941849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e41a0260 con 0x7f27e41036f0 2024-06-10T00:57:26.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.171+0000 7f27e9093700 1 --2- 172.21.15.78:0/625941849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27e41036f0 0x7f27e4198b40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f27d400c9b0 tx=0x7f27d400ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:26.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.171+0000 7f27daffd700 1 -- 172.21.15.78:0/625941849 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d4007b00 con 0x7f27e41036f0 2024-06-10T00:57:26.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.171+0000 7f27ebaf8700 1 -- 172.21.15.78:0/625941849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e41a0550 con 0x7f27e41036f0 2024-06-10T00:57:26.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.171+0000 7f27daffd700 1 -- 172.21.15.78:0/625941849 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27d4007ca0 con 0x7f27e41036f0 2024-06-10T00:57:26.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.171+0000 7f27daffd700 1 -- 172.21.15.78:0/625941849 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d40186f0 con 0x7f27e41036f0 2024-06-10T00:57:26.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.171+0000 7f27ebaf8700 1 -- 172.21.15.78:0/625941849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e41a0ae0 con 0x7f27e41036f0 2024-06-10T00:57:26.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.172+0000 7f27ebaf8700 1 -- 172.21.15.78:0/625941849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27e4069180 con 0x7f27e41036f0 2024-06-10T00:57:26.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.173+0000 7f27daffd700 1 -- 172.21.15.78:0/625941849 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f27d4018890 con 0x7f27e41036f0 2024-06-10T00:57:26.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.174+0000 7f27daffd700 1 --2- 172.21.15.78:0/625941849 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f27d00a2c20 0x7f27d00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:26.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.174+0000 7f27daffd700 1 -- 172.21.15.78:0/625941849 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7f27d40ce070 con 0x7f27e41036f0 2024-06-10T00:57:26.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.174+0000 7f27e9894700 1 --2- 172.21.15.78:0/625941849 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f27d00a2c20 0x7f27d00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:26.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.175+0000 7f27e9894700 1 --2- 172.21.15.78:0/625941849 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f27d00a2c20 0x7f27d00a50e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f27e0005950 tx=0x7f27e0005d10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:26.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.178+0000 7f27daffd700 1 -- 172.21.15.78:0/625941849 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f27d4014030 con 0x7f27e41036f0 2024-06-10T00:57:26.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:25 smithi175 ceph-mon[36636]: Detected new or changed devices on smithi007 2024-06-10T00:57:26.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:25 smithi175 ceph-mon[36636]: Adjusting osd_memory_target on smithi007 to 8546M 2024-06-10T00:57:26.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:25 smithi175 ceph-mon[36636]: osdmap e27: 4 total, 4 up, 4 in 2024-06-10T00:57:26.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:25 smithi175 ceph-mon[36636]: osdmap e28: 4 total, 4 up, 4 in 2024-06-10T00:57:26.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:25 smithi007 ceph-mon[32398]: Detected new or changed devices on smithi007 2024-06-10T00:57:26.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:25 smithi007 ceph-mon[32398]: Adjusting osd_memory_target on smithi007 to 8546M 2024-06-10T00:57:26.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:25 smithi007 ceph-mon[32398]: osdmap e27: 4 total, 4 up, 4 in 2024-06-10T00:57:26.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:25 smithi007 ceph-mon[32398]: osdmap e28: 4 total, 4 up, 4 in 2024-06-10T00:57:26.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:25 smithi078 ceph-mon[36683]: Detected new or changed devices on smithi007 2024-06-10T00:57:26.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:25 smithi078 ceph-mon[36683]: Adjusting osd_memory_target on smithi007 to 8546M 2024-06-10T00:57:26.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:25 smithi078 ceph-mon[36683]: osdmap e27: 4 total, 4 up, 4 in 2024-06-10T00:57:26.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:25 smithi078 ceph-mon[36683]: osdmap e28: 4 total, 4 up, 4 in 2024-06-10T00:57:26.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:26.381+0000 7f27ebaf8700 1 -- 172.21.15.78:0/625941849 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f27e40628d0 con 0x7f27d00a2c20 2024-06-10T00:57:27.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:26 smithi175 ceph-mon[36636]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:57:27.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:26 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:57:27.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:26 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:57:27.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:26 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:27.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:26 smithi175 ceph-mon[36636]: osdmap e29: 4 total, 4 up, 4 in 2024-06-10T00:57:27.236 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:26 smithi078 ceph-mon[36683]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:57:27.236 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:26 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:57:27.236 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:26 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:57:27.236 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:26 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:27.236 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:26 smithi078 ceph-mon[36683]: osdmap e29: 4 total, 4 up, 4 in 2024-06-10T00:57:27.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:26 smithi007 ceph-mon[32398]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:57:27.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:26 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:57:27.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:26 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:57:27.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:26 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:27.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:26 smithi007 ceph-mon[32398]: osdmap e29: 4 total, 4 up, 4 in 2024-06-10T00:57:28.193 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:27 smithi078 ceph-mon[36683]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:57:28.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:27 smithi175 ceph-mon[36636]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:57:28.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:27 smithi007 ceph-mon[32398]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:57:29.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:28 smithi175 ceph-mon[36636]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:57:29.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:28 smithi007 ceph-mon[32398]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:57:29.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:28 smithi078 ceph-mon[36683]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:57:30.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:29 smithi175 ceph-mon[36636]: from='client.? 172.21.15.78:0/329011007' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7990305-de9e-400e-a432-d7854f73f339"}]: dispatch 2024-06-10T00:57:30.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:29 smithi175 ceph-mon[36636]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7990305-de9e-400e-a432-d7854f73f339"}]: dispatch 2024-06-10T00:57:30.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:29 smithi175 ceph-mon[36636]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7990305-de9e-400e-a432-d7854f73f339"}]': finished 2024-06-10T00:57:30.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:29 smithi175 ceph-mon[36636]: osdmap e30: 5 total, 4 up, 5 in 2024-06-10T00:57:30.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:29 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:57:30.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:29 smithi175 ceph-mon[36636]: from='client.? 172.21.15.78:0/2245338233' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:57:30.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:29 smithi007 ceph-mon[32398]: from='client.? 172.21.15.78:0/329011007' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7990305-de9e-400e-a432-d7854f73f339"}]: dispatch 2024-06-10T00:57:30.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:29 smithi007 ceph-mon[32398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7990305-de9e-400e-a432-d7854f73f339"}]: dispatch 2024-06-10T00:57:30.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:29 smithi007 ceph-mon[32398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7990305-de9e-400e-a432-d7854f73f339"}]': finished 2024-06-10T00:57:30.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:29 smithi007 ceph-mon[32398]: osdmap e30: 5 total, 4 up, 5 in 2024-06-10T00:57:30.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:29 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:57:30.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:29 smithi007 ceph-mon[32398]: from='client.? 172.21.15.78:0/2245338233' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:57:30.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:29 smithi078 ceph-mon[36683]: from='client.? 172.21.15.78:0/329011007' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7990305-de9e-400e-a432-d7854f73f339"}]: dispatch 2024-06-10T00:57:30.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:29 smithi078 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7990305-de9e-400e-a432-d7854f73f339"}]: dispatch 2024-06-10T00:57:30.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:29 smithi078 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7990305-de9e-400e-a432-d7854f73f339"}]': finished 2024-06-10T00:57:30.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:29 smithi078 ceph-mon[36683]: osdmap e30: 5 total, 4 up, 5 in 2024-06-10T00:57:30.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:29 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:57:30.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:29 smithi078 ceph-mon[36683]: from='client.? 172.21.15.78:0/2245338233' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:57:31.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:31 smithi078 ceph-mon[36683]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:57:31.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:31 smithi175 ceph-mon[36636]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:57:31.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:31 smithi007 ceph-mon[32398]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:57:33.332 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:33 smithi078 ceph-mon[36683]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-10T00:57:33.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:33 smithi175 ceph-mon[36636]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-10T00:57:33.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:33 smithi007 ceph-mon[32398]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-10T00:57:35.299 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:35 smithi078 ceph-mon[36683]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-06-10T00:57:35.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:35 smithi175 ceph-mon[36636]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-06-10T00:57:35.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:35 smithi007 ceph-mon[32398]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-06-10T00:57:37.306 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:37 smithi078 ceph-mon[36683]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-06-10T00:57:37.306 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-10T00:57:37.306 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:37.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:37 smithi175 ceph-mon[36636]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-06-10T00:57:37.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-10T00:57:37.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:37.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:37 smithi007 ceph-mon[32398]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-06-10T00:57:37.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-10T00:57:37.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:38.260 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:38 smithi078 ceph-mon[36683]: Deploying daemon osd.4 on smithi078 2024-06-10T00:57:38.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:38 smithi007 ceph-mon[32398]: Deploying daemon osd.4 on smithi078 2024-06-10T00:57:38.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:38 smithi175 ceph-mon[36636]: Deploying daemon osd.4 on smithi078 2024-06-10T00:57:39.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:39 smithi078 ceph-mon[36683]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-10T00:57:39.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:39 smithi175 ceph-mon[36636]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-10T00:57:39.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:39 smithi007 ceph-mon[32398]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-10T00:57:41.307 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:41 smithi078 ceph-mon[36683]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-06-10T00:57:41.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:41 smithi175 ceph-mon[36636]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-06-10T00:57:41.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:41 smithi007 ceph-mon[32398]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-06-10T00:57:43.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:42 smithi078 ceph-mon[36683]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-10T00:57:43.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:42 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:43.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:42 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:43.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:42 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:57:43.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:42 smithi175 ceph-mon[36636]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-10T00:57:43.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:42 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:43.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:42 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:43.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:42 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:57:43.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:42 smithi007 ceph-mon[32398]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-10T00:57:43.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:42 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:43.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:42 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:43.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:42 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:57:45.177 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:44 smithi078 ceph-mon[36683]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:57:45.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:44 smithi175 ceph-mon[36636]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:57:45.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:45.248+0000 7f27daffd700 1 -- 172.21.15.78:0/625941849 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f27e40628d0 con 0x7f27d00a2c20 2024-06-10T00:57:45.249 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 4 on host 'smithi078' 2024-06-10T00:57:45.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:45.250+0000 7f27ebaf8700 1 -- 172.21.15.78:0/625941849 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f27d00a2c20 msgr2=0x7f27d00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:45.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:45.250+0000 7f27ebaf8700 1 --2- 172.21.15.78:0/625941849 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f27d00a2c20 0x7f27d00a50e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f27e0005950 tx=0x7f27e0005d10 comp rx=0 tx=0).stop 2024-06-10T00:57:45.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:45.250+0000 7f27ebaf8700 1 -- 172.21.15.78:0/625941849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27e41036f0 msgr2=0x7f27e4198b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:45.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:45.250+0000 7f27ebaf8700 1 --2- 172.21.15.78:0/625941849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27e41036f0 0x7f27e4198b40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f27d400c9b0 tx=0x7f27d400ccc0 comp rx=0 tx=0).stop 2024-06-10T00:57:45.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:45.250+0000 7f27ebaf8700 1 -- 172.21.15.78:0/625941849 shutdown_connections 2024-06-10T00:57:45.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:45.250+0000 7f27ebaf8700 1 --2- 172.21.15.78:0/625941849 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f27d00a2c20 0x7f27d00a50e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:45.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:45.251+0000 7f27ebaf8700 1 --2- 172.21.15.78:0/625941849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27e4104090 0x7f27e419fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:45.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:45.251+0000 7f27ebaf8700 1 --2- 172.21.15.78:0/625941849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27e41024f0 0x7f27e4198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:45.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:45.251+0000 7f27ebaf8700 1 --2- 172.21.15.78:0/625941849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27e41036f0 0x7f27e4198b40 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:45.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:45.251+0000 7f27ebaf8700 1 -- 172.21.15.78:0/625941849 >> 172.21.15.78:0/625941849 conn(0x7f27e40fda90 msgr2=0x7f27e4106990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:45.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:45.251+0000 7f27ebaf8700 1 -- 172.21.15.78:0/625941849 shutdown_connections 2024-06-10T00:57:45.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:57:45.251+0000 7f27ebaf8700 1 -- 172.21.15.78:0/625941849 wait complete. 2024-06-10T00:57:45.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:44 smithi007 ceph-mon[32398]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:57:45.908 DEBUG:teuthology.orchestra.run.smithi078:osd.4> sudo journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.4.service 2024-06-10T00:57:45.911 INFO:tasks.cephadm:Deploying osd.5 on smithi175 with /dev/vg_nvme/lv_3... 2024-06-10T00:57:45.911 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-10T00:57:46.154 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:57:46.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:46 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:46.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:46 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:46.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:46 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:46.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:46 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:46.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:46 smithi175 ceph-mon[36636]: from='osd.4 [v2:172.21.15.78:6808/3767608357,v1:172.21.15.78:6809/3767608357]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-10T00:57:46.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:46 smithi175 ceph-mon[36636]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-10T00:57:46.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:46 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:46.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:46 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:46.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:46 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:46.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:46 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:46.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:46 smithi007 ceph-mon[32398]: from='osd.4 [v2:172.21.15.78:6808/3767608357,v1:172.21.15.78:6809/3767608357]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-10T00:57:46.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:46 smithi007 ceph-mon[32398]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-10T00:57:46.525 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:46 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:46.526 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:46 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:46.526 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:46 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:46.526 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:46 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:46.526 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:46 smithi078 ceph-mon[36683]: from='osd.4 [v2:172.21.15.78:6808/3767608357,v1:172.21.15.78:6809/3767608357]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-10T00:57:46.526 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:46 smithi078 ceph-mon[36683]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-10T00:57:47.423 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:47 smithi078 ceph-mon[36683]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:57:47.424 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:47 smithi078 ceph-mon[36683]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-10T00:57:47.424 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:47 smithi078 ceph-mon[36683]: osdmap e31: 5 total, 4 up, 5 in 2024-06-10T00:57:47.424 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:47 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:57:47.424 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:47 smithi078 ceph-mon[36683]: from='osd.4 [v2:172.21.15.78:6808/3767608357,v1:172.21.15.78:6809/3767608357]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-06-10T00:57:47.424 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:47 smithi078 ceph-mon[36683]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-06-10T00:57:47.427 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:47 smithi175 ceph-mon[36636]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:57:47.427 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:47 smithi175 ceph-mon[36636]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-10T00:57:47.427 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:47 smithi175 ceph-mon[36636]: osdmap e31: 5 total, 4 up, 5 in 2024-06-10T00:57:47.427 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:47 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:57:47.427 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:47 smithi175 ceph-mon[36636]: from='osd.4 [v2:172.21.15.78:6808/3767608357,v1:172.21.15.78:6809/3767608357]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-06-10T00:57:47.427 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:47 smithi175 ceph-mon[36636]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-06-10T00:57:47.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:47 smithi007 ceph-mon[32398]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:57:47.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:47 smithi007 ceph-mon[32398]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-10T00:57:47.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:47 smithi007 ceph-mon[32398]: osdmap e31: 5 total, 4 up, 5 in 2024-06-10T00:57:47.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:47 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:57:47.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:47 smithi007 ceph-mon[32398]: from='osd.4 [v2:172.21.15.78:6808/3767608357,v1:172.21.15.78:6809/3767608357]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-06-10T00:57:47.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:47 smithi007 ceph-mon[32398]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-06-10T00:57:47.690 INFO:journalctl@ceph.osd.4.smithi078.stdout:Jun 10 00:57:47 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-4[47863]: 2024-06-10T00:57:47.582+0000 7fdb23ca0700 -1 osd.4 0 waiting for initial osdmap 2024-06-10T00:57:47.691 INFO:journalctl@ceph.osd.4.smithi078.stdout:Jun 10 00:57:47 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-4[47863]: 2024-06-10T00:57:47.590+0000 7fdb1fa93700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:57:48.002 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-10T00:57:48.003 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-10T00:57:48.003 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-06-10T00:57:48.003 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-06-10T00:57:48.003 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.01312 s, 799 MB/s 2024-06-10T00:57:48.003 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-06-10T00:57:48.682 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi175:vg_nvme/lv_3 2024-06-10T00:57:48.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:48 smithi078 ceph-mon[36683]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:57:48.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:48 smithi078 ceph-mon[36683]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-06-10T00:57:48.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:48 smithi078 ceph-mon[36683]: osdmap e32: 5 total, 4 up, 5 in 2024-06-10T00:57:48.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:48 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:57:48.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:48 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:57:48.917 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:57:48.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:48 smithi175 ceph-mon[36636]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:57:48.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:48 smithi175 ceph-mon[36636]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-06-10T00:57:48.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:48 smithi175 ceph-mon[36636]: osdmap e32: 5 total, 4 up, 5 in 2024-06-10T00:57:48.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:48 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:57:48.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:48 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:57:49.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:48 smithi007 ceph-mon[32398]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:57:49.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:48 smithi007 ceph-mon[32398]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-06-10T00:57:49.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:48 smithi007 ceph-mon[32398]: osdmap e32: 5 total, 4 up, 5 in 2024-06-10T00:57:49.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:48 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:57:49.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:48 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:57:49.844 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:49 smithi175 ceph-mon[36636]: purged_snaps scrub starts 2024-06-10T00:57:49.844 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:49 smithi175 ceph-mon[36636]: purged_snaps scrub ok 2024-06-10T00:57:49.844 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:49 smithi175 ceph-mon[36636]: osd.4 [v2:172.21.15.78:6808/3767608357,v1:172.21.15.78:6809/3767608357] boot 2024-06-10T00:57:49.844 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:49 smithi175 ceph-mon[36636]: osdmap e33: 5 total, 5 up, 5 in 2024-06-10T00:57:49.844 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:49 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:57:49.844 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:49 smithi175 ceph-mon[36636]: Detected new or changed devices on smithi078 2024-06-10T00:57:49.844 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:49 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:49.844 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:49 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:49.845 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:49 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:57:49.845 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:49 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:57:49.845 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:49 smithi175 ceph-mon[36636]: Adjusting osd_memory_target on smithi078 to 8544M 2024-06-10T00:57:49.845 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:49 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:49.845 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:49 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:49.845 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:49 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:57:49.845 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:49 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:49.845 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:49 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:57:49.845 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:49 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:49.845 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:49 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:57:49.845 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:49 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:50.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:49 smithi007 ceph-mon[32398]: purged_snaps scrub starts 2024-06-10T00:57:50.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:49 smithi007 ceph-mon[32398]: purged_snaps scrub ok 2024-06-10T00:57:50.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:49 smithi007 ceph-mon[32398]: osd.4 [v2:172.21.15.78:6808/3767608357,v1:172.21.15.78:6809/3767608357] boot 2024-06-10T00:57:50.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:49 smithi007 ceph-mon[32398]: osdmap e33: 5 total, 5 up, 5 in 2024-06-10T00:57:50.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:49 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:57:50.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:49 smithi007 ceph-mon[32398]: Detected new or changed devices on smithi078 2024-06-10T00:57:50.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:49 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:50.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:49 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:50.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:49 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:57:50.020 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:49 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:57:50.020 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:49 smithi007 ceph-mon[32398]: Adjusting osd_memory_target on smithi078 to 8544M 2024-06-10T00:57:50.020 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:49 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:50.020 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:49 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:50.020 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:49 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:57:50.020 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:49 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:50.020 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:49 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:57:50.020 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:49 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:50.020 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:49 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:57:50.020 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:49 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:50.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:49 smithi078 ceph-mon[36683]: purged_snaps scrub starts 2024-06-10T00:57:50.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:49 smithi078 ceph-mon[36683]: purged_snaps scrub ok 2024-06-10T00:57:50.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:49 smithi078 ceph-mon[36683]: osd.4 [v2:172.21.15.78:6808/3767608357,v1:172.21.15.78:6809/3767608357] boot 2024-06-10T00:57:50.086 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:49 smithi078 ceph-mon[36683]: osdmap e33: 5 total, 5 up, 5 in 2024-06-10T00:57:50.086 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:49 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:57:50.086 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:49 smithi078 ceph-mon[36683]: Detected new or changed devices on smithi078 2024-06-10T00:57:50.086 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:49 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:50.086 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:49 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:50.086 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:49 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:57:50.086 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:49 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:57:50.086 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:49 smithi078 ceph-mon[36683]: Adjusting osd_memory_target on smithi078 to 8544M 2024-06-10T00:57:50.086 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:49 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:50.086 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:49 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:50.086 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:49 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:57:50.086 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:49 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:50.087 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:49 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:57:50.087 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:49 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:50.087 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:49 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:57:50.087 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:49 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:57:50.336 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.333+0000 7f46ef95c700 1 -- 172.21.15.175:0/2770352766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46e81020b0 msgr2=0x7f46e8109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:50.336 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.333+0000 7f46ef95c700 1 --2- 172.21.15.175:0/2770352766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46e81020b0 0x7f46e8109970 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f46e000b0d0 tx=0x7f46e000b3e0 comp rx=0 tx=0).stop 2024-06-10T00:57:50.336 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.333+0000 7f46ef95c700 1 -- 172.21.15.175:0/2770352766 shutdown_connections 2024-06-10T00:57:50.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.333+0000 7f46ef95c700 1 --2- 172.21.15.175:0/2770352766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46e81020b0 0x7f46e8109970 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:50.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.334+0000 7f46ef95c700 1 --2- 172.21.15.175:0/2770352766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46e81029d0 0x7f46e8102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:50.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.334+0000 7f46ef95c700 1 --2- 172.21.15.175:0/2770352766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46e8101710 0x7f46e8101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:50.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.334+0000 7f46ef95c700 1 -- 172.21.15.175:0/2770352766 >> 172.21.15.175:0/2770352766 conn(0x7f46e80fc160 msgr2=0x7f46e80fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:50.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.334+0000 7f46ef95c700 1 -- 172.21.15.175:0/2770352766 shutdown_connections 2024-06-10T00:57:50.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.334+0000 7f46ef95c700 1 -- 172.21.15.175:0/2770352766 wait complete. 2024-06-10T00:57:50.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.334+0000 7f46ef95c700 1 Processor -- start 2024-06-10T00:57:50.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.334+0000 7f46ef95c700 1 -- start start 2024-06-10T00:57:50.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.334+0000 7f46ef95c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46e8101710 0x7f46e81054b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:50.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.334+0000 7f46ef95c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46e81020b0 0x7f46e8103b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:50.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.334+0000 7f46ef95c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46e81029d0 0x7f46e8104040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:50.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.334+0000 7f46ef95c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f46e810d020 con 0x7f46e8101710 2024-06-10T00:57:50.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.334+0000 7f46ef95c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f46e810cea0 con 0x7f46e81029d0 2024-06-10T00:57:50.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.334+0000 7f46ef95c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f46e810d1a0 con 0x7f46e81020b0 2024-06-10T00:57:50.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.334+0000 7f46ecef7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46e81020b0 0x7f46e8103b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:50.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.334+0000 7f46ecef7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46e81020b0 0x7f46e8103b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:41286/0 (socket says 172.21.15.175:41286) 2024-06-10T00:57:50.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.334+0000 7f46ecef7700 1 -- 172.21.15.175:0/2224090734 learned_addr learned my addr 172.21.15.175:0/2224090734 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-06-10T00:57:50.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.335+0000 7f46edef9700 1 --2- 172.21.15.175:0/2224090734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46e81029d0 0x7f46e8104040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:50.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.335+0000 7f46ecef7700 1 -- 172.21.15.175:0/2224090734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46e81029d0 msgr2=0x7f46e8104040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:50.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.335+0000 7f46ecef7700 1 --2- 172.21.15.175:0/2224090734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46e81029d0 0x7f46e8104040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:50.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.335+0000 7f46ecef7700 1 -- 172.21.15.175:0/2224090734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46e8101710 msgr2=0x7f46e81054b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:50.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.335+0000 7f46ecef7700 1 --2- 172.21.15.175:0/2224090734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46e8101710 0x7f46e81054b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:50.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.335+0000 7f46ecef7700 1 -- 172.21.15.175:0/2224090734 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e81048f0 con 0x7f46e81020b0 2024-06-10T00:57:50.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.335+0000 7f46edef9700 1 --2- 172.21.15.175:0/2224090734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46e81029d0 0x7f46e8104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:57:50.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.335+0000 7f46ecef7700 1 --2- 172.21.15.175:0/2224090734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46e81020b0 0x7f46e8103b00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f46e400c970 tx=0x7f46e400cd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:50.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.335+0000 7f46de7fc700 1 -- 172.21.15.175:0/2224090734 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4007aa0 con 0x7f46e81020b0 2024-06-10T00:57:50.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.335+0000 7f46ef95c700 1 -- 172.21.15.175:0/2224090734 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46e81a2750 con 0x7f46e81020b0 2024-06-10T00:57:50.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.335+0000 7f46de7fc700 1 -- 172.21.15.175:0/2224090734 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46e4004d10 con 0x7f46e81020b0 2024-06-10T00:57:50.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.335+0000 7f46de7fc700 1 -- 172.21.15.175:0/2224090734 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4005690 con 0x7f46e81020b0 2024-06-10T00:57:50.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.335+0000 7f46ef95c700 1 -- 172.21.15.175:0/2224090734 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46e81a2c60 con 0x7f46e81020b0 2024-06-10T00:57:50.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.336+0000 7f46ef95c700 1 -- 172.21.15.175:0/2224090734 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46e8069180 con 0x7f46e81020b0 2024-06-10T00:57:50.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.337+0000 7f46de7fc700 1 -- 172.21.15.175:0/2224090734 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f46e4004750 con 0x7f46e81020b0 2024-06-10T00:57:50.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.338+0000 7f46de7fc700 1 --2- 172.21.15.175:0/2224090734 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f46d40a2c20 0x7f46d40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:50.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.338+0000 7f46de7fc700 1 -- 172.21.15.175:0/2224090734 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f46e40ce100 con 0x7f46e81020b0 2024-06-10T00:57:50.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.338+0000 7f46ed6f8700 1 --2- 172.21.15.175:0/2224090734 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f46d40a2c20 0x7f46d40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:50.340 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.338+0000 7f46ed6f8700 1 --2- 172.21.15.175:0/2224090734 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f46d40a2c20 0x7f46d40a50e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f46e806a960 tx=0x7f46d8005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:50.342 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.341+0000 7f46de7fc700 1 -- 172.21.15.175:0/2224090734 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f46e409c1f0 con 0x7f46e81020b0 2024-06-10T00:57:50.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:57:50.546+0000 7f46ef95c700 1 -- 172.21.15.175:0/2224090734 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f46e81a2f20 con 0x7f46d40a2c20 2024-06-10T00:57:50.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:50 smithi175 ceph-mon[36636]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:57:50.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:50 smithi175 ceph-mon[36636]: osdmap e34: 5 total, 5 up, 5 in 2024-06-10T00:57:50.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:50 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:57:50.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:50 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:57:50.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:50 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:51.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:50 smithi007 ceph-mon[32398]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:57:51.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:50 smithi007 ceph-mon[32398]: osdmap e34: 5 total, 5 up, 5 in 2024-06-10T00:57:51.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:50 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:57:51.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:50 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:57:51.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:50 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:51.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:50 smithi078 ceph-mon[36683]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:57:51.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:50 smithi078 ceph-mon[36683]: osdmap e34: 5 total, 5 up, 5 in 2024-06-10T00:57:51.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:50 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:57:51.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:50 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:57:51.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:50 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:51.820 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:51 smithi175 ceph-mon[36636]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:57:52.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:51 smithi007 ceph-mon[32398]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:57:52.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:51 smithi078 ceph-mon[36683]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:57:52.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:52 smithi175 ceph-mon[36636]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:57:53.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:52 smithi007 ceph-mon[32398]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:57:53.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:52 smithi078 ceph-mon[36683]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:57:53.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:53 smithi175 ceph-mon[36636]: from='client.? 172.21.15.175:0/3211467048' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bad9a7b3-da5a-48d0-905d-e6a3190317fe"}]: dispatch 2024-06-10T00:57:53.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:53 smithi175 ceph-mon[36636]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bad9a7b3-da5a-48d0-905d-e6a3190317fe"}]: dispatch 2024-06-10T00:57:53.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:53 smithi175 ceph-mon[36636]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bad9a7b3-da5a-48d0-905d-e6a3190317fe"}]': finished 2024-06-10T00:57:53.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:53 smithi175 ceph-mon[36636]: osdmap e35: 6 total, 5 up, 6 in 2024-06-10T00:57:53.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:53 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:57:54.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:53 smithi007 ceph-mon[32398]: from='client.? 172.21.15.175:0/3211467048' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bad9a7b3-da5a-48d0-905d-e6a3190317fe"}]: dispatch 2024-06-10T00:57:54.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:53 smithi007 ceph-mon[32398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bad9a7b3-da5a-48d0-905d-e6a3190317fe"}]: dispatch 2024-06-10T00:57:54.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:53 smithi007 ceph-mon[32398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bad9a7b3-da5a-48d0-905d-e6a3190317fe"}]': finished 2024-06-10T00:57:54.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:53 smithi007 ceph-mon[32398]: osdmap e35: 6 total, 5 up, 6 in 2024-06-10T00:57:54.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:53 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:57:54.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:53 smithi078 ceph-mon[36683]: from='client.? 172.21.15.175:0/3211467048' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bad9a7b3-da5a-48d0-905d-e6a3190317fe"}]: dispatch 2024-06-10T00:57:54.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:53 smithi078 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bad9a7b3-da5a-48d0-905d-e6a3190317fe"}]: dispatch 2024-06-10T00:57:54.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:53 smithi078 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bad9a7b3-da5a-48d0-905d-e6a3190317fe"}]': finished 2024-06-10T00:57:54.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:53 smithi078 ceph-mon[36683]: osdmap e35: 6 total, 5 up, 6 in 2024-06-10T00:57:54.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:53 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:57:54.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:54 smithi175 ceph-mon[36636]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:57:54.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:54 smithi175 ceph-mon[36636]: from='client.? 172.21.15.175:0/3995250144' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:57:55.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:54 smithi007 ceph-mon[32398]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:57:55.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:54 smithi007 ceph-mon[32398]: from='client.? 172.21.15.175:0/3995250144' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:57:55.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:54 smithi078 ceph-mon[36683]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:57:55.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:54 smithi078 ceph-mon[36683]: from='client.? 172.21.15.175:0/3995250144' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:57:56.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:56 smithi175 ceph-mon[36636]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:57:57.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:56 smithi007 ceph-mon[32398]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:57:57.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:56 smithi078 ceph-mon[36683]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:57:58.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:57:58 smithi175 ceph-mon[36636]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:57:59.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:57:58 smithi007 ceph-mon[32398]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:57:59.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:57:58 smithi078 ceph-mon[36683]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:58:00.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:00 smithi175 ceph-mon[36636]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:58:00.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:00 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-10T00:58:00.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:00 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:58:01.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:00 smithi007 ceph-mon[32398]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:58:01.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:00 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-10T00:58:01.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:00 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:58:01.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:00 smithi078 ceph-mon[36683]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:58:01.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:00 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-10T00:58:01.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:00 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:58:01.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:01 smithi175 ceph-mon[36636]: Deploying daemon osd.5 on smithi175 2024-06-10T00:58:02.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:01 smithi007 ceph-mon[32398]: Deploying daemon osd.5 on smithi175 2024-06-10T00:58:02.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:01 smithi078 ceph-mon[36683]: Deploying daemon osd.5 on smithi175 2024-06-10T00:58:02.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:02 smithi175 ceph-mon[36636]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:58:03.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:02 smithi007 ceph-mon[32398]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:58:03.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:02 smithi078 ceph-mon[36683]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:58:04.832 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:04 smithi175 ceph-mon[36636]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:58:05.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:04 smithi007 ceph-mon[32398]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:58:05.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:04 smithi078 ceph-mon[36683]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:58:07.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:06 smithi007 ceph-mon[32398]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:58:07.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:06 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:07.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:06 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:07.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:06 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:58:07.072 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:06 smithi175 ceph-mon[36636]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:58:07.072 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:06 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:07.072 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:06 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:07.073 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:06 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:58:07.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:06 smithi078 ceph-mon[36683]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:58:07.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:06 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:07.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:06 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:07.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:06 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:58:09.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:08 smithi007 ceph-mon[32398]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:58:09.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:08 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:09.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:08 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:09.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:08 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:58:09.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:08 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:58:09.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:08 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:09.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:08 smithi078 ceph-mon[36683]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:58:09.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:08 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:09.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:08 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:09.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:08 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:58:09.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:08 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:58:09.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:08 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:09.200 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:08 smithi175 ceph-mon[36636]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:58:09.200 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:08 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:09.200 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:08 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:09.200 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:08 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:58:09.200 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:08 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:58:09.200 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:08 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:09.567 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:58:09.567+0000 7f46de7fc700 1 -- 172.21.15.175:0/2224090734 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f46e81a2f20 con 0x7f46d40a2c20 2024-06-10T00:58:09.567 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 5 on host 'smithi175' 2024-06-10T00:58:09.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:58:09.569+0000 7f46ef95c700 1 -- 172.21.15.175:0/2224090734 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f46d40a2c20 msgr2=0x7f46d40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:09.570 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:58:09.569+0000 7f46ef95c700 1 --2- 172.21.15.175:0/2224090734 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f46d40a2c20 0x7f46d40a50e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f46e806a960 tx=0x7f46d8005d20 comp rx=0 tx=0).stop 2024-06-10T00:58:09.570 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:58:09.569+0000 7f46ef95c700 1 -- 172.21.15.175:0/2224090734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46e81020b0 msgr2=0x7f46e8103b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:09.570 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:58:09.569+0000 7f46ef95c700 1 --2- 172.21.15.175:0/2224090734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46e81020b0 0x7f46e8103b00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f46e400c970 tx=0x7f46e400cd30 comp rx=0 tx=0).stop 2024-06-10T00:58:09.570 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:58:09.569+0000 7f46ef95c700 1 -- 172.21.15.175:0/2224090734 shutdown_connections 2024-06-10T00:58:09.570 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:58:09.569+0000 7f46ef95c700 1 --2- 172.21.15.175:0/2224090734 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f46d40a2c20 0x7f46d40a50e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:09.570 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:58:09.569+0000 7f46ef95c700 1 --2- 172.21.15.175:0/2224090734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46e81029d0 0x7f46e8104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:09.570 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:58:09.569+0000 7f46ef95c700 1 --2- 172.21.15.175:0/2224090734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46e8101710 0x7f46e81054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:09.570 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:58:09.569+0000 7f46ef95c700 1 --2- 172.21.15.175:0/2224090734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46e81020b0 0x7f46e8103b00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:09.570 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:58:09.569+0000 7f46ef95c700 1 -- 172.21.15.175:0/2224090734 >> 172.21.15.175:0/2224090734 conn(0x7f46e80fc160 msgr2=0x7f46e81005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:09.571 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:58:09.569+0000 7f46ef95c700 1 -- 172.21.15.175:0/2224090734 shutdown_connections 2024-06-10T00:58:09.571 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:58:09.569+0000 7f46ef95c700 1 -- 172.21.15.175:0/2224090734 wait complete. 2024-06-10T00:58:10.213 DEBUG:teuthology.orchestra.run.smithi175:osd.5> sudo journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.5.service 2024-06-10T00:58:10.215 INFO:tasks.cephadm:Deploying osd.6 on smithi007 with /dev/vg_nvme/lv_2... 2024-06-10T00:58:10.215 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-10T00:58:10.470 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:58:10.833 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:10 smithi007 ceph-mon[32398]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:58:10.833 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:10 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:10.833 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:10 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:10.833 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:10 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:58:10.833 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:10 smithi007 ceph-mon[32398]: from='osd.5 [v2:172.21.15.175:6808/2157664731,v1:172.21.15.175:6809/2157664731]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-10T00:58:10.833 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:10 smithi007 ceph-mon[32398]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-10T00:58:10.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:10 smithi078 ceph-mon[36683]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:58:10.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:10 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:10.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:10 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:10.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:10 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:58:10.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:10 smithi078 ceph-mon[36683]: from='osd.5 [v2:172.21.15.175:6808/2157664731,v1:172.21.15.175:6809/2157664731]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-10T00:58:10.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:10 smithi078 ceph-mon[36683]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-10T00:58:10.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:10 smithi175 ceph-mon[36636]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:58:10.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:10 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:10.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:10 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:10.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:10 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:58:10.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:10 smithi175 ceph-mon[36636]: from='osd.5 [v2:172.21.15.175:6808/2157664731,v1:172.21.15.175:6809/2157664731]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-10T00:58:10.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:10 smithi175 ceph-mon[36636]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-10T00:58:11.545 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:11 smithi175 ceph-mon[36636]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-10T00:58:11.545 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:11 smithi175 ceph-mon[36636]: osdmap e36: 6 total, 5 up, 6 in 2024-06-10T00:58:11.545 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:11 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:58:11.545 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:11 smithi175 ceph-mon[36636]: from='osd.5 [v2:172.21.15.175:6808/2157664731,v1:172.21.15.175:6809/2157664731]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-10T00:58:11.545 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:11 smithi175 ceph-mon[36636]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-10T00:58:11.545 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 00:58:11 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T00:58:11.538+0000 7f310c55c700 -1 osd.5 0 waiting for initial osdmap 2024-06-10T00:58:11.546 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 00:58:11 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T00:58:11.544+0000 7f310834f700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:58:11.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:11 smithi078 ceph-mon[36683]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-10T00:58:11.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:11 smithi078 ceph-mon[36683]: osdmap e36: 6 total, 5 up, 6 in 2024-06-10T00:58:11.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:11 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:58:11.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:11 smithi078 ceph-mon[36683]: from='osd.5 [v2:172.21.15.175:6808/2157664731,v1:172.21.15.175:6809/2157664731]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-10T00:58:11.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:11 smithi078 ceph-mon[36683]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-10T00:58:11.838 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:11 smithi007 ceph-mon[32398]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-10T00:58:11.838 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:11 smithi007 ceph-mon[32398]: osdmap e36: 6 total, 5 up, 6 in 2024-06-10T00:58:11.838 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:11 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:58:11.838 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:11 smithi007 ceph-mon[32398]: from='osd.5 [v2:172.21.15.175:6808/2157664731,v1:172.21.15.175:6809/2157664731]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-10T00:58:11.838 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:11 smithi007 ceph-mon[32398]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-10T00:58:12.567 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-10T00:58:12.567 INFO:teuthology.orchestra.run.smithi007.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-10T00:58:12.567 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10+0 records in 2024-06-10T00:58:12.567 INFO:teuthology.orchestra.run.smithi007.stderr:10+0 records out 2024-06-10T00:58:12.567 INFO:teuthology.orchestra.run.smithi007.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00890444 s, 1.2 GB/s 2024-06-10T00:58:12.567 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping successful for: 2024-06-10T00:58:12.659 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:12 smithi175 ceph-mon[36636]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:58:12.660 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:12 smithi175 ceph-mon[36636]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-06-10T00:58:12.660 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:12 smithi175 ceph-mon[36636]: osdmap e37: 6 total, 5 up, 6 in 2024-06-10T00:58:12.660 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:12 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:58:12.660 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:12 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:58:12.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:12 smithi007 ceph-mon[32398]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:58:12.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:12 smithi007 ceph-mon[32398]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-06-10T00:58:12.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:12 smithi007 ceph-mon[32398]: osdmap e37: 6 total, 5 up, 6 in 2024-06-10T00:58:12.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:12 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:58:12.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:12 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:58:12.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:12 smithi078 ceph-mon[36683]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:58:12.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:12 smithi078 ceph-mon[36683]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-06-10T00:58:12.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:12 smithi078 ceph-mon[36683]: osdmap e37: 6 total, 5 up, 6 in 2024-06-10T00:58:12.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:12 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:58:12.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:12 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:58:13.489 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi007:vg_nvme/lv_2 2024-06-10T00:58:13.735 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:58:13.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:13 smithi007 ceph-mon[32398]: purged_snaps scrub starts 2024-06-10T00:58:13.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:13 smithi007 ceph-mon[32398]: purged_snaps scrub ok 2024-06-10T00:58:13.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:13 smithi007 ceph-mon[32398]: osd.5 [v2:172.21.15.175:6808/2157664731,v1:172.21.15.175:6809/2157664731] boot 2024-06-10T00:58:13.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:13 smithi007 ceph-mon[32398]: osdmap e38: 6 total, 6 up, 6 in 2024-06-10T00:58:13.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:13 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:58:13.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:13 smithi007 ceph-mon[32398]: Detected new or changed devices on smithi175 2024-06-10T00:58:13.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:13 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:13.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:13 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:13.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:13 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:58:13.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:13 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:58:13.770 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:13 smithi007 ceph-mon[32398]: Adjusting osd_memory_target on smithi175 to 8544M 2024-06-10T00:58:13.770 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:13 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:13.770 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:13 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:58:13.770 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:13 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:58:13.770 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:13 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:13.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:13 smithi078 ceph-mon[36683]: purged_snaps scrub starts 2024-06-10T00:58:13.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:13 smithi078 ceph-mon[36683]: purged_snaps scrub ok 2024-06-10T00:58:13.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:13 smithi078 ceph-mon[36683]: osd.5 [v2:172.21.15.175:6808/2157664731,v1:172.21.15.175:6809/2157664731] boot 2024-06-10T00:58:13.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:13 smithi078 ceph-mon[36683]: osdmap e38: 6 total, 6 up, 6 in 2024-06-10T00:58:13.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:13 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:58:13.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:13 smithi078 ceph-mon[36683]: Detected new or changed devices on smithi175 2024-06-10T00:58:13.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:13 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:13.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:13 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:13.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:13 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:58:13.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:13 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:58:13.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:13 smithi078 ceph-mon[36683]: Adjusting osd_memory_target on smithi175 to 8544M 2024-06-10T00:58:13.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:13 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:13.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:13 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:58:13.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:13 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:58:13.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:13 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:13.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:13 smithi175 ceph-mon[36636]: purged_snaps scrub starts 2024-06-10T00:58:13.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:13 smithi175 ceph-mon[36636]: purged_snaps scrub ok 2024-06-10T00:58:13.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:13 smithi175 ceph-mon[36636]: osd.5 [v2:172.21.15.175:6808/2157664731,v1:172.21.15.175:6809/2157664731] boot 2024-06-10T00:58:13.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:13 smithi175 ceph-mon[36636]: osdmap e38: 6 total, 6 up, 6 in 2024-06-10T00:58:13.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:13 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:58:13.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:13 smithi175 ceph-mon[36636]: Detected new or changed devices on smithi175 2024-06-10T00:58:13.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:13 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:13.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:13 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:13.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:13 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:58:13.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:13 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:58:13.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:13 smithi175 ceph-mon[36636]: Adjusting osd_memory_target on smithi175 to 8544M 2024-06-10T00:58:13.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:13 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:13.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:13 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:58:13.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:13 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:58:13.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:13 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:14.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:14 smithi007 ceph-mon[32398]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:58:14.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:14 smithi007 ceph-mon[32398]: osdmap e39: 6 total, 6 up, 6 in 2024-06-10T00:58:14.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:14 smithi078 ceph-mon[36683]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:58:14.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:14 smithi078 ceph-mon[36683]: osdmap e39: 6 total, 6 up, 6 in 2024-06-10T00:58:14.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:14 smithi175 ceph-mon[36636]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:58:14.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:14 smithi175 ceph-mon[36636]: osdmap e39: 6 total, 6 up, 6 in 2024-06-10T00:58:15.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.466+0000 7fd4c624a700 1 -- 172.21.15.7:0/2351768450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4c01036f0 msgr2=0x7fd4c0103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:15.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.466+0000 7fd4c624a700 1 --2- 172.21.15.7:0/2351768450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4c01036f0 0x7fd4c0103b50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd4a80099c0 tx=0x7fd4a8009cd0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.466+0000 7fd4c624a700 1 -- 172.21.15.7:0/2351768450 shutdown_connections 2024-06-10T00:58:15.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.466+0000 7fd4c624a700 1 --2- 172.21.15.7:0/2351768450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4c0104090 0x7fd4c010a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.466+0000 7fd4c624a700 1 --2- 172.21.15.7:0/2351768450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4c01024f0 0x7fd4c0102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.466+0000 7fd4c624a700 1 --2- 172.21.15.7:0/2351768450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4c01036f0 0x7fd4c0103b50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.466+0000 7fd4c624a700 1 -- 172.21.15.7:0/2351768450 >> 172.21.15.7:0/2351768450 conn(0x7fd4c00fda90 msgr2=0x7fd4c00ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.466+0000 7fd4c624a700 1 -- 172.21.15.7:0/2351768450 shutdown_connections 2024-06-10T00:58:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.466+0000 7fd4c624a700 1 -- 172.21.15.7:0/2351768450 wait complete. 2024-06-10T00:58:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.467+0000 7fd4c624a700 1 Processor -- start 2024-06-10T00:58:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.467+0000 7fd4c624a700 1 -- start start 2024-06-10T00:58:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.467+0000 7fd4c624a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4c01024f0 0x7fd4c01985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.467+0000 7fd4c624a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4c01036f0 0x7fd4c0198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.467+0000 7fd4c624a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4c0104090 0x7fd4c019fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.467+0000 7fd4c624a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c010e0c0 con 0x7fd4c01036f0 2024-06-10T00:58:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.467+0000 7fd4c624a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c010df40 con 0x7fd4c0104090 2024-06-10T00:58:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.467+0000 7fd4c624a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c010e240 con 0x7fd4c01024f0 2024-06-10T00:58:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.467+0000 7fd4beffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4c01036f0 0x7fd4c0198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.467+0000 7fd4beffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4c01036f0 0x7fd4c0198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53286/0 (socket says 172.21.15.7:53286) 2024-06-10T00:58:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.467+0000 7fd4beffd700 1 -- 172.21.15.7:0/2316502097 learned_addr learned my addr 172.21.15.7:0/2316502097 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:58:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.467+0000 7fd4bf7fe700 1 --2- 172.21.15.7:0/2316502097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4c01024f0 0x7fd4c01985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.467+0000 7fd4bffff700 1 --2- 172.21.15.7:0/2316502097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4c0104090 0x7fd4c019fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.468+0000 7fd4beffd700 1 -- 172.21.15.7:0/2316502097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4c01024f0 msgr2=0x7fd4c01985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.468+0000 7fd4beffd700 1 --2- 172.21.15.7:0/2316502097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4c01024f0 0x7fd4c01985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.468+0000 7fd4beffd700 1 -- 172.21.15.7:0/2316502097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4c0104090 msgr2=0x7fd4c019fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.468+0000 7fd4beffd700 1 --2- 172.21.15.7:0/2316502097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4c0104090 0x7fd4c019fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.468+0000 7fd4beffd700 1 -- 172.21.15.7:0/2316502097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c01a0220 con 0x7fd4c01036f0 2024-06-10T00:58:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.468+0000 7fd4bffff700 1 --2- 172.21.15.7:0/2316502097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4c0104090 0x7fd4c019fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:58:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.468+0000 7fd4beffd700 1 --2- 172.21.15.7:0/2316502097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4c01036f0 0x7fd4c0198b00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800b580 tx=0x7fd4a8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.468+0000 7fd4bcff9700 1 -- 172.21.15.7:0/2316502097 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8027070 con 0x7fd4c01036f0 2024-06-10T00:58:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.468+0000 7fd4bcff9700 1 -- 172.21.15.7:0/2316502097 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4a80041d0 con 0x7fd4c01036f0 2024-06-10T00:58:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.468+0000 7fd4c624a700 1 -- 172.21.15.7:0/2316502097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4c01a04b0 con 0x7fd4c01036f0 2024-06-10T00:58:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.468+0000 7fd4bcff9700 1 -- 172.21.15.7:0/2316502097 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a800f460 con 0x7fd4c01036f0 2024-06-10T00:58:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.468+0000 7fd4c624a700 1 -- 172.21.15.7:0/2316502097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4c01a0960 con 0x7fd4c01036f0 2024-06-10T00:58:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.469+0000 7fd4c624a700 1 -- 172.21.15.7:0/2316502097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4c0069180 con 0x7fd4c01036f0 2024-06-10T00:58:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.470+0000 7fd4bcff9700 1 -- 172.21.15.7:0/2316502097 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fd4a802a070 con 0x7fd4c01036f0 2024-06-10T00:58:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.471+0000 7fd4bcff9700 1 --2- 172.21.15.7:0/2316502097 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd4ac0a2d20 0x7fd4ac0a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.471+0000 7fd4bf7fe700 1 --2- 172.21.15.7:0/2316502097 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd4ac0a2d20 0x7fd4ac0a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.471+0000 7fd4bcff9700 1 -- 172.21.15.7:0/2316502097 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7fd4a80cf980 con 0x7fd4c01036f0 2024-06-10T00:58:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.471+0000 7fd4bf7fe700 1 --2- 172.21.15.7:0/2316502097 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd4ac0a2d20 0x7fd4ac0a51e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0103550 tx=0x7fd4b0009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:15.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.475+0000 7fd4bcff9700 1 -- 172.21.15.7:0/2316502097 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd4a8031020 con 0x7fd4c01036f0 2024-06-10T00:58:15.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:15.692+0000 7fd4c624a700 1 -- 172.21.15.7:0/2316502097 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd4c00628d0 con 0x7fd4ac0a2d20 2024-06-10T00:58:15.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:15 smithi007 ceph-mon[32398]: osdmap e40: 6 total, 6 up, 6 in 2024-06-10T00:58:15.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:15 smithi078 ceph-mon[36683]: osdmap e40: 6 total, 6 up, 6 in 2024-06-10T00:58:15.859 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:15 smithi175 ceph-mon[36636]: osdmap e40: 6 total, 6 up, 6 in 2024-06-10T00:58:16.611 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:16 smithi007 ceph-mon[32398]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:58:16.611 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:16 smithi007 ceph-mon[32398]: osdmap e41: 6 total, 6 up, 6 in 2024-06-10T00:58:16.611 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:16 smithi007 ceph-mon[32398]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:58:16.611 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:16 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:58:16.611 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:16 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:58:16.611 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:16 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:58:16.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:16 smithi078 ceph-mon[36683]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:58:16.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:16 smithi078 ceph-mon[36683]: osdmap e41: 6 total, 6 up, 6 in 2024-06-10T00:58:16.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:16 smithi078 ceph-mon[36683]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:58:16.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:16 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:58:16.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:16 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:58:16.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:16 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:58:16.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:16 smithi175 ceph-mon[36636]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:58:16.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:16 smithi175 ceph-mon[36636]: osdmap e41: 6 total, 6 up, 6 in 2024-06-10T00:58:16.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:16 smithi175 ceph-mon[36636]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:58:16.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:16 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:58:16.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:16 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:58:16.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:16 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:58:17.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:17 smithi078 ceph-mon[36683]: osdmap e42: 6 total, 6 up, 6 in 2024-06-10T00:58:17.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:17 smithi175 ceph-mon[36636]: osdmap e42: 6 total, 6 up, 6 in 2024-06-10T00:58:18.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:17 smithi007 ceph-mon[32398]: osdmap e42: 6 total, 6 up, 6 in 2024-06-10T00:58:18.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:18 smithi078 ceph-mon[36683]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:58:18.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:18 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2125774974' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c687653-157b-4b96-867b-a2eaa7d8ed8a"}]: dispatch 2024-06-10T00:58:18.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:18 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2125774974' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c687653-157b-4b96-867b-a2eaa7d8ed8a"}]': finished 2024-06-10T00:58:18.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:18 smithi078 ceph-mon[36683]: osdmap e43: 7 total, 6 up, 7 in 2024-06-10T00:58:18.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:18 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:58:18.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:18 smithi175 ceph-mon[36636]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:58:18.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:18 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2125774974' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c687653-157b-4b96-867b-a2eaa7d8ed8a"}]: dispatch 2024-06-10T00:58:18.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:18 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2125774974' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c687653-157b-4b96-867b-a2eaa7d8ed8a"}]': finished 2024-06-10T00:58:18.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:18 smithi175 ceph-mon[36636]: osdmap e43: 7 total, 6 up, 7 in 2024-06-10T00:58:18.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:18 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:58:19.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:18 smithi007 ceph-mon[32398]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:58:19.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:18 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2125774974' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c687653-157b-4b96-867b-a2eaa7d8ed8a"}]: dispatch 2024-06-10T00:58:19.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:18 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2125774974' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c687653-157b-4b96-867b-a2eaa7d8ed8a"}]': finished 2024-06-10T00:58:19.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:18 smithi007 ceph-mon[32398]: osdmap e43: 7 total, 6 up, 7 in 2024-06-10T00:58:19.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:18 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:58:19.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:19 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3352846343' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:58:19.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:19 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3352846343' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:58:20.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:19 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3352846343' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:58:20.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:20 smithi078 ceph-mon[36683]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:58:20.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:20 smithi175 ceph-mon[36636]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:58:21.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:20 smithi007 ceph-mon[32398]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:58:22.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:22 smithi007 ceph-mon[32398]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-10T00:58:22.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:22 smithi078 ceph-mon[36683]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-10T00:58:22.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:22 smithi175 ceph-mon[36636]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-10T00:58:24.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:24 smithi078 ceph-mon[36683]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-06-10T00:58:24.857 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:24 smithi007 ceph-mon[32398]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-06-10T00:58:24.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:24 smithi175 ceph-mon[36636]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-06-10T00:58:26.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:26 smithi078 ceph-mon[36683]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-06-10T00:58:26.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:26 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-10T00:58:26.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:26 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:58:26.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:26 smithi078 ceph-mon[36683]: Deploying daemon osd.6 on smithi007 2024-06-10T00:58:26.844 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:26 smithi007 ceph-mon[32398]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-06-10T00:58:26.844 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:26 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-10T00:58:26.844 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:26 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:58:26.844 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:26 smithi007 ceph-mon[32398]: Deploying daemon osd.6 on smithi007 2024-06-10T00:58:26.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:26 smithi175 ceph-mon[36636]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-06-10T00:58:26.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:26 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-10T00:58:26.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:26 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:58:26.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:26 smithi175 ceph-mon[36636]: Deploying daemon osd.6 on smithi007 2024-06-10T00:58:28.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:28 smithi007 ceph-mon[32398]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-10T00:58:28.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:28 smithi078 ceph-mon[36683]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-10T00:58:28.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:28 smithi175 ceph-mon[36636]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-10T00:58:30.904 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:30 smithi007 ceph-mon[32398]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-06-10T00:58:30.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:30 smithi175 ceph-mon[36636]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-06-10T00:58:31.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:30 smithi078 ceph-mon[36683]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-06-10T00:58:33.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:33 smithi007 ceph-mon[32398]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-10T00:58:33.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:33 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:33.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:33 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:33.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:33 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:58:33.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:33 smithi078 ceph-mon[36683]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-10T00:58:33.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:33 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:33.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:33 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:33.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:33 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:58:33.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:33 smithi175 ceph-mon[36636]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-10T00:58:33.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:33 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:33.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:33 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:33.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:33 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:58:35.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:35 smithi007 ceph-mon[32398]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:58:35.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:35.320+0000 7fd4bcff9700 1 -- 172.21.15.7:0/2316502097 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd4c00628d0 con 0x7fd4ac0a2d20 2024-06-10T00:58:35.320 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 6 on host 'smithi007' 2024-06-10T00:58:35.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:35.322+0000 7fd4c624a700 1 -- 172.21.15.7:0/2316502097 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd4ac0a2d20 msgr2=0x7fd4ac0a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:35.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:35.322+0000 7fd4c624a700 1 --2- 172.21.15.7:0/2316502097 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd4ac0a2d20 0x7fd4ac0a51e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0103550 tx=0x7fd4b0009040 comp rx=0 tx=0).stop 2024-06-10T00:58:35.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:35.322+0000 7fd4c624a700 1 -- 172.21.15.7:0/2316502097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4c01036f0 msgr2=0x7fd4c0198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:35.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:35.322+0000 7fd4c624a700 1 --2- 172.21.15.7:0/2316502097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4c01036f0 0x7fd4c0198b00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800b580 tx=0x7fd4a8005e80 comp rx=0 tx=0).stop 2024-06-10T00:58:35.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:35.322+0000 7fd4c624a700 1 -- 172.21.15.7:0/2316502097 shutdown_connections 2024-06-10T00:58:35.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:35.322+0000 7fd4c624a700 1 --2- 172.21.15.7:0/2316502097 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd4ac0a2d20 0x7fd4ac0a51e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:35.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:35.322+0000 7fd4c624a700 1 --2- 172.21.15.7:0/2316502097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4c0104090 0x7fd4c019fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:35.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:35.322+0000 7fd4c624a700 1 --2- 172.21.15.7:0/2316502097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4c01024f0 0x7fd4c01985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:35.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:35.322+0000 7fd4c624a700 1 --2- 172.21.15.7:0/2316502097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4c01036f0 0x7fd4c0198b00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:35.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:35.322+0000 7fd4c624a700 1 -- 172.21.15.7:0/2316502097 >> 172.21.15.7:0/2316502097 conn(0x7fd4c00fda90 msgr2=0x7fd4c0106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:35.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:35.323+0000 7fd4c624a700 1 -- 172.21.15.7:0/2316502097 shutdown_connections 2024-06-10T00:58:35.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:58:35.323+0000 7fd4c624a700 1 -- 172.21.15.7:0/2316502097 wait complete. 2024-06-10T00:58:35.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:35 smithi078 ceph-mon[36683]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:58:35.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:35 smithi175 ceph-mon[36636]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:58:35.938 DEBUG:teuthology.orchestra.run.smithi007:osd.6> sudo journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.6.service 2024-06-10T00:58:35.943 INFO:tasks.cephadm:Deploying osd.7 on smithi078 with /dev/vg_nvme/lv_2... 2024-06-10T00:58:35.943 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-10T00:58:36.197 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T00:58:36.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:36.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:36.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:36 smithi078 ceph-mon[36683]: from='osd.6 [v2:172.21.15.7:6818/838731756,v1:172.21.15.7:6819/838731756]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-10T00:58:36.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:36.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:36.639 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:36.639 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:36.640 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:36 smithi007 ceph-mon[32398]: from='osd.6 [v2:172.21.15.7:6818/838731756,v1:172.21.15.7:6819/838731756]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-10T00:58:36.640 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:36.640 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:36.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:36.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:36.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:36 smithi175 ceph-mon[36636]: from='osd.6 [v2:172.21.15.7:6818/838731756,v1:172.21.15.7:6819/838731756]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-10T00:58:36.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:36.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:37.560 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:37 smithi078 ceph-mon[36683]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:58:37.560 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:37 smithi078 ceph-mon[36683]: from='osd.6 [v2:172.21.15.7:6818/838731756,v1:172.21.15.7:6819/838731756]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-10T00:58:37.560 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:37 smithi078 ceph-mon[36683]: osdmap e44: 7 total, 6 up, 7 in 2024-06-10T00:58:37.560 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:37 smithi078 ceph-mon[36683]: from='osd.6 [v2:172.21.15.7:6818/838731756,v1:172.21.15.7:6819/838731756]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-10T00:58:37.561 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:58:37.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:37 smithi175 ceph-mon[36636]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:58:37.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:37 smithi175 ceph-mon[36636]: from='osd.6 [v2:172.21.15.7:6818/838731756,v1:172.21.15.7:6819/838731756]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-10T00:58:37.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:37 smithi175 ceph-mon[36636]: osdmap e44: 7 total, 6 up, 7 in 2024-06-10T00:58:37.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:37 smithi175 ceph-mon[36636]: from='osd.6 [v2:172.21.15.7:6818/838731756,v1:172.21.15.7:6819/838731756]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-10T00:58:37.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:58:37.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:37 smithi007 ceph-mon[32398]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:58:37.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:37 smithi007 ceph-mon[32398]: from='osd.6 [v2:172.21.15.7:6818/838731756,v1:172.21.15.7:6819/838731756]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-10T00:58:37.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:37 smithi007 ceph-mon[32398]: osdmap e44: 7 total, 6 up, 7 in 2024-06-10T00:58:37.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:37 smithi007 ceph-mon[32398]: from='osd.6 [v2:172.21.15.7:6818/838731756,v1:172.21.15.7:6819/838731756]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-10T00:58:37.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:58:37.769 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 00:58:37 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T00:58:37.308+0000 7ff8f884a700 -1 osd.6 0 waiting for initial osdmap 2024-06-10T00:58:37.769 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 00:58:37 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T00:58:37.318+0000 7ff8f3e3c700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:58:38.234 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-10T00:58:38.234 INFO:teuthology.orchestra.run.smithi078.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-10T00:58:38.234 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10+0 records in 2024-06-10T00:58:38.234 INFO:teuthology.orchestra.run.smithi078.stderr:10+0 records out 2024-06-10T00:58:38.234 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0134812 s, 778 MB/s 2024-06-10T00:58:38.234 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping successful for: 2024-06-10T00:58:38.404 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:38 smithi007 ceph-mon[32398]: from='osd.6 [v2:172.21.15.7:6818/838731756,v1:172.21.15.7:6819/838731756]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-06-10T00:58:38.404 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:38 smithi007 ceph-mon[32398]: osdmap e45: 7 total, 6 up, 7 in 2024-06-10T00:58:38.404 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:38 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:58:38.404 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:38 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:58:38.531 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:38 smithi078 ceph-mon[36683]: from='osd.6 [v2:172.21.15.7:6818/838731756,v1:172.21.15.7:6819/838731756]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-06-10T00:58:38.532 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:38 smithi078 ceph-mon[36683]: osdmap e45: 7 total, 6 up, 7 in 2024-06-10T00:58:38.532 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:38 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:58:38.532 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:38 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:58:38.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:38 smithi175 ceph-mon[36636]: from='osd.6 [v2:172.21.15.7:6818/838731756,v1:172.21.15.7:6819/838731756]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-06-10T00:58:38.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:38 smithi175 ceph-mon[36636]: osdmap e45: 7 total, 6 up, 7 in 2024-06-10T00:58:38.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:38 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:58:38.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:38 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:58:38.870 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_2 2024-06-10T00:58:39.108 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T00:58:39.494 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:39 smithi078 ceph-mon[36683]: purged_snaps scrub starts 2024-06-10T00:58:39.494 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:39 smithi078 ceph-mon[36683]: purged_snaps scrub ok 2024-06-10T00:58:39.494 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:39 smithi078 ceph-mon[36683]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:58:39.494 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:39 smithi078 ceph-mon[36683]: osd.6 [v2:172.21.15.7:6818/838731756,v1:172.21.15.7:6819/838731756] boot 2024-06-10T00:58:39.494 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:39 smithi078 ceph-mon[36683]: osdmap e46: 7 total, 7 up, 7 in 2024-06-10T00:58:39.494 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:39 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:58:39.667 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:39 smithi007 ceph-mon[32398]: purged_snaps scrub starts 2024-06-10T00:58:39.667 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:39 smithi007 ceph-mon[32398]: purged_snaps scrub ok 2024-06-10T00:58:39.667 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:39 smithi007 ceph-mon[32398]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:58:39.667 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:39 smithi007 ceph-mon[32398]: osd.6 [v2:172.21.15.7:6818/838731756,v1:172.21.15.7:6819/838731756] boot 2024-06-10T00:58:39.667 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:39 smithi007 ceph-mon[32398]: osdmap e46: 7 total, 7 up, 7 in 2024-06-10T00:58:39.667 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:39 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:58:39.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:39 smithi175 ceph-mon[36636]: purged_snaps scrub starts 2024-06-10T00:58:39.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:39 smithi175 ceph-mon[36636]: purged_snaps scrub ok 2024-06-10T00:58:39.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:39 smithi175 ceph-mon[36636]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:58:39.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:39 smithi175 ceph-mon[36636]: osd.6 [v2:172.21.15.7:6818/838731756,v1:172.21.15.7:6819/838731756] boot 2024-06-10T00:58:39.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:39 smithi175 ceph-mon[36636]: osdmap e46: 7 total, 7 up, 7 in 2024-06-10T00:58:39.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:39 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:58:40.534 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:40 smithi078 ceph-mon[36683]: osdmap e47: 7 total, 7 up, 7 in 2024-06-10T00:58:40.535 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:40 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:40.535 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:40 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:40.535 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:40 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:58:40.535 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:40 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:58:40.535 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:40 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:58:40.535 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:40 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:40.535 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:40 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:58:40.535 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:40 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:58:40.535 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:40 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:40.536 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:40 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:58:40.536 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:40 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:58:40.536 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:40 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:58:40.536 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:40 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:40.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.637+0000 7f703698a700 1 -- 172.21.15.78:0/3038363199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70301025f0 msgr2=0x7f7030102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:40.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.637+0000 7f703698a700 1 --2- 172.21.15.78:0/3038363199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70301025f0 0x7f7030102a50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f70240099c0 tx=0x7f7024009cd0 comp rx=0 tx=0).stop 2024-06-10T00:58:40.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.637+0000 7f703698a700 1 -- 172.21.15.78:0/3038363199 shutdown_connections 2024-06-10T00:58:40.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.637+0000 7f703698a700 1 --2- 172.21.15.78:0/3038363199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7030102f90 0x7f7030109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:40.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.637+0000 7f703698a700 1 --2- 172.21.15.78:0/3038363199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70301013f0 0x7f7030101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:40.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.637+0000 7f703698a700 1 --2- 172.21.15.78:0/3038363199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70301025f0 0x7f7030102a50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:40.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.637+0000 7f703698a700 1 -- 172.21.15.78:0/3038363199 >> 172.21.15.78:0/3038363199 conn(0x7f70300fc9b0 msgr2=0x7f70300fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:40.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.637+0000 7f703698a700 1 -- 172.21.15.78:0/3038363199 shutdown_connections 2024-06-10T00:58:40.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.637+0000 7f703698a700 1 -- 172.21.15.78:0/3038363199 wait complete. 2024-06-10T00:58:40.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.637+0000 7f703698a700 1 Processor -- start 2024-06-10T00:58:40.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.637+0000 7f703698a700 1 -- start start 2024-06-10T00:58:40.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.638+0000 7f703698a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70301013f0 0x7f7030197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:40.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.638+0000 7f703698a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70301025f0 0x7f7030197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:40.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.638+0000 7f703698a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7030102f90 0x7f703019ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:40.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.638+0000 7f703698a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f703010d030 con 0x7f70301013f0 2024-06-10T00:58:40.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.638+0000 7f703698a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f703010ceb0 con 0x7f70301025f0 2024-06-10T00:58:40.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.638+0000 7f703698a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f703010d1b0 con 0x7f7030102f90 2024-06-10T00:58:40.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.638+0000 7f702f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70301025f0 0x7f7030197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:40.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.638+0000 7f702f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70301025f0 0x7f7030197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35282/0 (socket says 172.21.15.78:35282) 2024-06-10T00:58:40.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.638+0000 7f702f7fe700 1 -- 172.21.15.78:0/1319131166 learned_addr learned my addr 172.21.15.78:0/1319131166 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-06-10T00:58:40.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.638+0000 7f702ffff700 1 --2- 172.21.15.78:0/1319131166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70301013f0 0x7f7030197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:40.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.638+0000 7f7034f27700 1 --2- 172.21.15.78:0/1319131166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7030102f90 0x7f703019ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:40.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.638+0000 7f702f7fe700 1 -- 172.21.15.78:0/1319131166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7030102f90 msgr2=0x7f703019ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:40.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.638+0000 7f702f7fe700 1 --2- 172.21.15.78:0/1319131166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7030102f90 0x7f703019ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:40.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.638+0000 7f702f7fe700 1 -- 172.21.15.78:0/1319131166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70301013f0 msgr2=0x7f7030197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:40.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.638+0000 7f702f7fe700 1 --2- 172.21.15.78:0/1319131166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70301013f0 0x7f7030197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:40.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.638+0000 7f702f7fe700 1 -- 172.21.15.78:0/1319131166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703019f2a0 con 0x7f70301025f0 2024-06-10T00:58:40.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.638+0000 7f702f7fe700 1 --2- 172.21.15.78:0/1319131166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70301025f0 0x7f7030197b80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7024003890 tx=0x7f70240047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:40.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.638+0000 7f702d7fa700 1 -- 172.21.15.78:0/1319131166 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024027070 con 0x7f70301025f0 2024-06-10T00:58:40.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.638+0000 7f703698a700 1 -- 172.21.15.78:0/1319131166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f703019f530 con 0x7f70301025f0 2024-06-10T00:58:40.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.638+0000 7f702d7fa700 1 -- 172.21.15.78:0/1319131166 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70240041d0 con 0x7f70301025f0 2024-06-10T00:58:40.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.638+0000 7f702d7fa700 1 -- 172.21.15.78:0/1319131166 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702400f460 con 0x7f70301025f0 2024-06-10T00:58:40.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.639+0000 7f703698a700 1 -- 172.21.15.78:0/1319131166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f703019fa60 con 0x7f70301025f0 2024-06-10T00:58:40.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.639+0000 7f703698a700 1 -- 172.21.15.78:0/1319131166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7030069180 con 0x7f70301025f0 2024-06-10T00:58:40.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.640+0000 7f702d7fa700 1 -- 172.21.15.78:0/1319131166 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f702402a070 con 0x7f70301025f0 2024-06-10T00:58:40.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.641+0000 7f702d7fa700 1 --2- 172.21.15.78:0/1319131166 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f70180a2c20 0x7f70180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:40.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.641+0000 7f702d7fa700 1 -- 172.21.15.78:0/1319131166 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f70240cf770 con 0x7f70301025f0 2024-06-10T00:58:40.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.641+0000 7f702ffff700 1 --2- 172.21.15.78:0/1319131166 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f70180a2c20 0x7f70180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:40.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.642+0000 7f702ffff700 1 --2- 172.21.15.78:0/1319131166 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f70180a2c20 0x7f70180a50e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7030102450 tx=0x7f702000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:40.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.645+0000 7f702d7fa700 1 -- 172.21.15.78:0/1319131166 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7024013070 con 0x7f70301025f0 2024-06-10T00:58:40.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:40 smithi175 ceph-mon[36636]: osdmap e47: 7 total, 7 up, 7 in 2024-06-10T00:58:40.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:40 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:40.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:40 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:40.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:40 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:58:40.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:40 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:58:40.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:40 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:58:40.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:40 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:40.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:40 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:58:40.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:40 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:58:40.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:40 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:40.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:40 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:58:40.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:40 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:58:40.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:40 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:58:40.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:40 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:40.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:40 smithi007 ceph-mon[32398]: osdmap e47: 7 total, 7 up, 7 in 2024-06-10T00:58:40.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:40 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:40.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:40 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:40.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:40 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:58:40.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:40 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:58:40.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:40 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:58:40.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:40 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:40.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:40 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:58:40.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:40 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:58:40.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:40 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:40.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:40 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:58:40.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:40 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:58:40.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:40 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:58:40.770 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:40 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:40.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:58:40.847+0000 7f703698a700 1 -- 172.21.15.78:0/1319131166 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f70300628d0 con 0x7f70180a2c20 2024-06-10T00:58:41.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:41 smithi078 ceph-mon[36683]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:58:41.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:41 smithi078 ceph-mon[36683]: Detected new or changed devices on smithi007 2024-06-10T00:58:41.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:41 smithi078 ceph-mon[36683]: Adjusting osd_memory_target on smithi007 to 5697M 2024-06-10T00:58:41.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:41 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:58:41.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:41 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:58:41.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:41 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:58:41.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:41 smithi175 ceph-mon[36636]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:58:41.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:41 smithi175 ceph-mon[36636]: Detected new or changed devices on smithi007 2024-06-10T00:58:41.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:41 smithi175 ceph-mon[36636]: Adjusting osd_memory_target on smithi007 to 5697M 2024-06-10T00:58:41.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:41 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:58:41.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:41 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:58:41.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:41 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:58:41.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:41 smithi007 ceph-mon[32398]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:58:41.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:41 smithi007 ceph-mon[32398]: Detected new or changed devices on smithi007 2024-06-10T00:58:41.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:41 smithi007 ceph-mon[32398]: Adjusting osd_memory_target on smithi007 to 5697M 2024-06-10T00:58:41.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:41 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:58:41.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:41 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:58:41.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:41 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:58:42.425 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:42 smithi078 ceph-mon[36683]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:58:42.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:42 smithi175 ceph-mon[36636]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:58:42.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:42 smithi007 ceph-mon[32398]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:58:43.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:43 smithi078 ceph-mon[36683]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:58:43.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:43 smithi175 ceph-mon[36636]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:58:43.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:43 smithi007 ceph-mon[32398]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:58:44.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:44 smithi175 ceph-mon[36636]: from='client.? 172.21.15.78:0/1325132792' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db40ddf1-9fd9-4859-b8c4-75f79582ab91"}]: dispatch 2024-06-10T00:58:44.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:44 smithi175 ceph-mon[36636]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db40ddf1-9fd9-4859-b8c4-75f79582ab91"}]: dispatch 2024-06-10T00:58:44.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:44 smithi175 ceph-mon[36636]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db40ddf1-9fd9-4859-b8c4-75f79582ab91"}]': finished 2024-06-10T00:58:44.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:44 smithi175 ceph-mon[36636]: osdmap e48: 8 total, 7 up, 8 in 2024-06-10T00:58:44.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:44 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:58:44.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:44 smithi175 ceph-mon[36636]: from='client.? 172.21.15.78:0/2394356508' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:58:44.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:44 smithi007 ceph-mon[32398]: from='client.? 172.21.15.78:0/1325132792' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db40ddf1-9fd9-4859-b8c4-75f79582ab91"}]: dispatch 2024-06-10T00:58:44.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:44 smithi007 ceph-mon[32398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db40ddf1-9fd9-4859-b8c4-75f79582ab91"}]: dispatch 2024-06-10T00:58:44.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:44 smithi007 ceph-mon[32398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db40ddf1-9fd9-4859-b8c4-75f79582ab91"}]': finished 2024-06-10T00:58:44.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:44 smithi007 ceph-mon[32398]: osdmap e48: 8 total, 7 up, 8 in 2024-06-10T00:58:44.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:44 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:58:44.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:44 smithi007 ceph-mon[32398]: from='client.? 172.21.15.78:0/2394356508' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:58:44.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:44 smithi078 ceph-mon[36683]: from='client.? 172.21.15.78:0/1325132792' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db40ddf1-9fd9-4859-b8c4-75f79582ab91"}]: dispatch 2024-06-10T00:58:44.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:44 smithi078 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db40ddf1-9fd9-4859-b8c4-75f79582ab91"}]: dispatch 2024-06-10T00:58:44.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:44 smithi078 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db40ddf1-9fd9-4859-b8c4-75f79582ab91"}]': finished 2024-06-10T00:58:44.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:44 smithi078 ceph-mon[36683]: osdmap e48: 8 total, 7 up, 8 in 2024-06-10T00:58:44.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:44 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:58:44.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:44 smithi078 ceph-mon[36683]: from='client.? 172.21.15.78:0/2394356508' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:58:45.609 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:45 smithi175 ceph-mon[36636]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:58:45.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:45 smithi007 ceph-mon[32398]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:58:45.837 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:45 smithi078 ceph-mon[36683]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:58:47.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:47 smithi078 ceph-mon[36683]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:58:47.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:47 smithi175 ceph-mon[36636]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:58:47.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:47 smithi007 ceph-mon[32398]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:58:49.557 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:49 smithi078 ceph-mon[36683]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:58:49.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:49 smithi175 ceph-mon[36636]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:58:49.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:49 smithi007 ceph-mon[32398]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:58:51.634 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:51 smithi078 ceph-mon[36683]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:58:51.635 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:51 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-10T00:58:51.635 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:51 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:58:51.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:51 smithi175 ceph-mon[36636]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:58:51.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:51 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-10T00:58:51.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:51 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:58:51.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:51 smithi007 ceph-mon[32398]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:58:51.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:51 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-10T00:58:51.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:51 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:58:52.654 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:52 smithi078 ceph-mon[36683]: Deploying daemon osd.7 on smithi078 2024-06-10T00:58:52.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:52 smithi175 ceph-mon[36636]: Deploying daemon osd.7 on smithi078 2024-06-10T00:58:52.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:52 smithi007 ceph-mon[32398]: Deploying daemon osd.7 on smithi078 2024-06-10T00:58:53.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:53 smithi078 ceph-mon[36683]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:58:53.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:53 smithi175 ceph-mon[36636]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:58:53.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:53 smithi007 ceph-mon[32398]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:58:54.471 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:54 smithi078 ceph-mon[36683]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:58:54.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:54 smithi175 ceph-mon[36636]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:58:54.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:54 smithi007 ceph-mon[32398]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:58:56.822 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:56 smithi078 ceph-mon[36683]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:58:56.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:56 smithi175 ceph-mon[36636]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:58:57.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:56 smithi007 ceph-mon[32398]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:58:58.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:57 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:58.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:57 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:58.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:57 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:58:58.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:57 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:58.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:57 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:58.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:57 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:58:58.312 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:57 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:58.312 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:57 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:58:58.312 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:57 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:58:59.158 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:58:58 smithi078 ceph-mon[36683]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:58:59.169 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:58:58 smithi007 ceph-mon[32398]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:58:59.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:58:58 smithi175 ceph-mon[36636]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:59:00.568 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 7 on host 'smithi078' 2024-06-10T00:59:00.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:00.567+0000 7f702d7fa700 1 -- 172.21.15.78:0/1319131166 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f70300628d0 con 0x7f70180a2c20 2024-06-10T00:59:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:00.570+0000 7f703698a700 1 -- 172.21.15.78:0/1319131166 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f70180a2c20 msgr2=0x7f70180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:00.570+0000 7f703698a700 1 --2- 172.21.15.78:0/1319131166 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f70180a2c20 0x7f70180a50e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7030102450 tx=0x7f702000a380 comp rx=0 tx=0).stop 2024-06-10T00:59:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:00.570+0000 7f703698a700 1 -- 172.21.15.78:0/1319131166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70301025f0 msgr2=0x7f7030197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:00.570+0000 7f703698a700 1 --2- 172.21.15.78:0/1319131166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70301025f0 0x7f7030197b80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7024003890 tx=0x7f70240047b0 comp rx=0 tx=0).stop 2024-06-10T00:59:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:00.570+0000 7f703698a700 1 -- 172.21.15.78:0/1319131166 shutdown_connections 2024-06-10T00:59:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:00.570+0000 7f703698a700 1 --2- 172.21.15.78:0/1319131166 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f70180a2c20 0x7f70180a50e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:00.570+0000 7f703698a700 1 --2- 172.21.15.78:0/1319131166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7030102f90 0x7f703019ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:00.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:00.570+0000 7f703698a700 1 --2- 172.21.15.78:0/1319131166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70301013f0 0x7f7030197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:00.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:00.570+0000 7f703698a700 1 --2- 172.21.15.78:0/1319131166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70301025f0 0x7f7030197b80 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:00.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:00.570+0000 7f703698a700 1 -- 172.21.15.78:0/1319131166 >> 172.21.15.78:0/1319131166 conn(0x7f70300fc9b0 msgr2=0x7f7030105970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:00.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:00.570+0000 7f703698a700 1 -- 172.21.15.78:0/1319131166 shutdown_connections 2024-06-10T00:59:00.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:00.570+0000 7f703698a700 1 -- 172.21.15.78:0/1319131166 wait complete. 2024-06-10T00:59:01.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:00 smithi175 ceph-mon[36636]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:59:01.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:00 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:01.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:00 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:01.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:00 smithi175 ceph-mon[36636]: from='osd.7 [v2:172.21.15.78:6816/3511921773,v1:172.21.15.78:6817/3511921773]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-10T00:59:01.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:00 smithi175 ceph-mon[36636]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-10T00:59:01.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:00 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:01.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:00 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:01.213 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:00 smithi078 ceph-mon[36683]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:59:01.213 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:00 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:01.213 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:00 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:01.213 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:00 smithi078 ceph-mon[36683]: from='osd.7 [v2:172.21.15.78:6816/3511921773,v1:172.21.15.78:6817/3511921773]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-10T00:59:01.213 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:00 smithi078 ceph-mon[36683]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-10T00:59:01.213 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:00 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:01.213 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:00 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:01.241 DEBUG:teuthology.orchestra.run.smithi078:osd.7> sudo journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.7.service 2024-06-10T00:59:01.243 INFO:tasks.cephadm:Deploying osd.8 on smithi175 with /dev/vg_nvme/lv_2... 2024-06-10T00:59:01.243 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-10T00:59:01.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:00 smithi007 ceph-mon[32398]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:59:01.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:00 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:01.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:00 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:01.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:00 smithi007 ceph-mon[32398]: from='osd.7 [v2:172.21.15.78:6816/3511921773,v1:172.21.15.78:6817/3511921773]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-10T00:59:01.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:00 smithi007 ceph-mon[32398]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-10T00:59:01.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:00 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:01.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:00 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:01.489 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:59:02.786 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:02 smithi078 ceph-mon[36683]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:59:02.786 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:02 smithi078 ceph-mon[36683]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-10T00:59:02.787 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:02 smithi078 ceph-mon[36683]: osdmap e49: 8 total, 7 up, 8 in 2024-06-10T00:59:02.787 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:02 smithi078 ceph-mon[36683]: from='osd.7 [v2:172.21.15.78:6816/3511921773,v1:172.21.15.78:6817/3511921773]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-06-10T00:59:02.787 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:02 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:59:02.787 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:02 smithi078 ceph-mon[36683]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-06-10T00:59:02.787 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 00:59:02 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T00:59:02.561+0000 7fc64ac89700 -1 osd.7 0 waiting for initial osdmap 2024-06-10T00:59:02.787 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 00:59:02 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T00:59:02.574+0000 7fc646a7c700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:59:02.862 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:02 smithi175 ceph-mon[36636]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:59:02.862 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:02 smithi175 ceph-mon[36636]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-10T00:59:02.862 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:02 smithi175 ceph-mon[36636]: osdmap e49: 8 total, 7 up, 8 in 2024-06-10T00:59:02.862 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:02 smithi175 ceph-mon[36636]: from='osd.7 [v2:172.21.15.78:6816/3511921773,v1:172.21.15.78:6817/3511921773]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-06-10T00:59:02.862 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:02 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:59:02.862 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:02 smithi175 ceph-mon[36636]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-06-10T00:59:03.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:02 smithi007 ceph-mon[32398]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:59:03.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:02 smithi007 ceph-mon[32398]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-10T00:59:03.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:02 smithi007 ceph-mon[32398]: osdmap e49: 8 total, 7 up, 8 in 2024-06-10T00:59:03.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:02 smithi007 ceph-mon[32398]: from='osd.7 [v2:172.21.15.78:6816/3511921773,v1:172.21.15.78:6817/3511921773]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-06-10T00:59:03.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:02 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:59:03.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:02 smithi007 ceph-mon[32398]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-06-10T00:59:03.586 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-10T00:59:03.586 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-10T00:59:03.586 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-06-10T00:59:03.586 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-06-10T00:59:03.586 INFO:teuthology.orchestra.run.smithi175.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0132839 s, 789 MB/s 2024-06-10T00:59:03.586 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-06-10T00:59:03.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:03 smithi175 ceph-mon[36636]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-06-10T00:59:03.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:03 smithi175 ceph-mon[36636]: osdmap e50: 8 total, 7 up, 8 in 2024-06-10T00:59:03.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:03 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:59:03.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:03 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:59:03.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:03 smithi078 ceph-mon[36683]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-06-10T00:59:03.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:03 smithi078 ceph-mon[36683]: osdmap e50: 8 total, 7 up, 8 in 2024-06-10T00:59:03.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:03 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:59:03.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:03 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:59:04.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:03 smithi007 ceph-mon[32398]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-06-10T00:59:04.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:03 smithi007 ceph-mon[32398]: osdmap e50: 8 total, 7 up, 8 in 2024-06-10T00:59:04.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:03 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:59:04.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:03 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:59:04.230 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi175:vg_nvme/lv_2 2024-06-10T00:59:04.476 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:59:04.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:04 smithi078 ceph-mon[36683]: purged_snaps scrub starts 2024-06-10T00:59:04.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:04 smithi078 ceph-mon[36683]: purged_snaps scrub ok 2024-06-10T00:59:04.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:04 smithi078 ceph-mon[36683]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:59:04.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:04 smithi078 ceph-mon[36683]: osd.7 [v2:172.21.15.78:6816/3511921773,v1:172.21.15.78:6817/3511921773] boot 2024-06-10T00:59:04.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:04 smithi078 ceph-mon[36683]: osdmap e51: 8 total, 8 up, 8 in 2024-06-10T00:59:04.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:04 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:59:04.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:04 smithi078 ceph-mon[36683]: Detected new or changed devices on smithi078 2024-06-10T00:59:04.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:04 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:04.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:04 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:04.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:04 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:59:04.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:04 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:59:04.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:04 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:59:04.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:04 smithi078 ceph-mon[36683]: Adjusting osd_memory_target on smithi078 to 5696M 2024-06-10T00:59:04.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:04 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:04.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:04 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:04.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:04 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:59:04.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:04 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:04.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:04 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:59:04.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:04 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:04.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:04 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:59:04.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:04 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:04.857 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:04 smithi175 ceph-mon[36636]: purged_snaps scrub starts 2024-06-10T00:59:04.857 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:04 smithi175 ceph-mon[36636]: purged_snaps scrub ok 2024-06-10T00:59:04.858 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:04 smithi175 ceph-mon[36636]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:59:04.858 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:04 smithi175 ceph-mon[36636]: osd.7 [v2:172.21.15.78:6816/3511921773,v1:172.21.15.78:6817/3511921773] boot 2024-06-10T00:59:04.858 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:04 smithi175 ceph-mon[36636]: osdmap e51: 8 total, 8 up, 8 in 2024-06-10T00:59:04.858 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:04 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:59:04.858 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:04 smithi175 ceph-mon[36636]: Detected new or changed devices on smithi078 2024-06-10T00:59:04.858 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:04 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:04.858 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:04 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:04.858 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:04 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:59:04.858 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:04 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:59:04.858 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:04 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:59:04.858 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:04 smithi175 ceph-mon[36636]: Adjusting osd_memory_target on smithi078 to 5696M 2024-06-10T00:59:04.859 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:04 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:04.859 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:04 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:04.859 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:04 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:59:04.859 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:04 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:04.859 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:04 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:59:04.859 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:04 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:04.859 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:04 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:59:04.859 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:04 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:05.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:04 smithi007 ceph-mon[32398]: purged_snaps scrub starts 2024-06-10T00:59:05.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:04 smithi007 ceph-mon[32398]: purged_snaps scrub ok 2024-06-10T00:59:05.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:04 smithi007 ceph-mon[32398]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:59:05.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:04 smithi007 ceph-mon[32398]: osd.7 [v2:172.21.15.78:6816/3511921773,v1:172.21.15.78:6817/3511921773] boot 2024-06-10T00:59:05.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:04 smithi007 ceph-mon[32398]: osdmap e51: 8 total, 8 up, 8 in 2024-06-10T00:59:05.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:04 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:59:05.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:04 smithi007 ceph-mon[32398]: Detected new or changed devices on smithi078 2024-06-10T00:59:05.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:04 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:05.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:04 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:05.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:04 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:59:05.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:04 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:59:05.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:04 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:59:05.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:04 smithi007 ceph-mon[32398]: Adjusting osd_memory_target on smithi078 to 5696M 2024-06-10T00:59:05.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:04 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:05.020 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:04 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:05.020 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:04 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:59:05.020 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:04 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:05.020 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:04 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:59:05.020 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:04 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:05.020 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:04 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:59:05.020 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:04 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:05.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:05 smithi078 ceph-mon[36683]: osdmap e52: 8 total, 8 up, 8 in 2024-06-10T00:59:05.864 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:05 smithi175 ceph-mon[36636]: osdmap e52: 8 total, 8 up, 8 in 2024-06-10T00:59:06.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:05 smithi007 ceph-mon[32398]: osdmap e52: 8 total, 8 up, 8 in 2024-06-10T00:59:06.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.085+0000 7fe463482700 1 -- 172.21.15.175:0/218040627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe45c1013f0 msgr2=0x7fe45c101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:06.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.085+0000 7fe463482700 1 --2- 172.21.15.175:0/218040627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe45c1013f0 0x7fe45c101810 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe458009990 tx=0x7fe458009ca0 comp rx=0 tx=0).stop 2024-06-10T00:59:06.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.085+0000 7fe463482700 1 -- 172.21.15.175:0/218040627 shutdown_connections 2024-06-10T00:59:06.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.085+0000 7fe463482700 1 --2- 172.21.15.175:0/218040627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe45c102f90 0x7fe45c109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:06.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.085+0000 7fe463482700 1 --2- 172.21.15.175:0/218040627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe45c1013f0 0x7fe45c101810 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:06.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.085+0000 7fe463482700 1 --2- 172.21.15.175:0/218040627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe45c1025f0 0x7fe45c102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:06.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.085+0000 7fe463482700 1 -- 172.21.15.175:0/218040627 >> 172.21.15.175:0/218040627 conn(0x7fe45c0fc990 msgr2=0x7fe45c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:06.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.085+0000 7fe463482700 1 -- 172.21.15.175:0/218040627 shutdown_connections 2024-06-10T00:59:06.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.085+0000 7fe463482700 1 -- 172.21.15.175:0/218040627 wait complete. 2024-06-10T00:59:06.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.086+0000 7fe463482700 1 Processor -- start 2024-06-10T00:59:06.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.086+0000 7fe463482700 1 -- start start 2024-06-10T00:59:06.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.086+0000 7fe463482700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe45c1013f0 0x7fe45c1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:06.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.086+0000 7fe463482700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe45c1025f0 0x7fe45c198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:06.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.086+0000 7fe463482700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe45c102f90 0x7fe45c19fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:06.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.086+0000 7fe463482700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c10d030 con 0x7fe45c1025f0 2024-06-10T00:59:06.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.086+0000 7fe463482700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c10ceb0 con 0x7fe45c102f90 2024-06-10T00:59:06.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.086+0000 7fe463482700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c10d1b0 con 0x7fe45c1013f0 2024-06-10T00:59:06.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.086+0000 7fe46121e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe45c1013f0 0x7fe45c1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:06.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.086+0000 7fe46121e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe45c1013f0 0x7fe45c1987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:44130/0 (socket says 172.21.15.175:44130) 2024-06-10T00:59:06.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.086+0000 7fe46121e700 1 -- 172.21.15.175:0/1513399198 learned_addr learned my addr 172.21.15.175:0/1513399198 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-06-10T00:59:06.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.087+0000 7fe46121e700 1 -- 172.21.15.175:0/1513399198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe45c102f90 msgr2=0x7fe45c19fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:06.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.087+0000 7fe46121e700 1 --2- 172.21.15.175:0/1513399198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe45c102f90 0x7fe45c19fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:06.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.087+0000 7fe460a1d700 1 --2- 172.21.15.175:0/1513399198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe45c1025f0 0x7fe45c198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:06.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.087+0000 7fe46121e700 1 -- 172.21.15.175:0/1513399198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe45c1025f0 msgr2=0x7fe45c198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:06.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.087+0000 7fe46121e700 1 --2- 172.21.15.175:0/1513399198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe45c1025f0 0x7fe45c198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:06.089 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.087+0000 7fe46121e700 1 -- 172.21.15.175:0/1513399198 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe45c1a0440 con 0x7fe45c1013f0 2024-06-10T00:59:06.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.087+0000 7fe460a1d700 1 --2- 172.21.15.175:0/1513399198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe45c1025f0 0x7fe45c198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:59:06.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.087+0000 7fe46121e700 1 --2- 172.21.15.175:0/1513399198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe45c1013f0 0x7fe45c1987e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe458000c00 tx=0x7fe458005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:06.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.087+0000 7fe4527fc700 1 -- 172.21.15.175:0/1513399198 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe458027070 con 0x7fe45c1013f0 2024-06-10T00:59:06.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.087+0000 7fe463482700 1 -- 172.21.15.175:0/1513399198 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe45c103810 con 0x7fe45c1013f0 2024-06-10T00:59:06.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.087+0000 7fe4527fc700 1 -- 172.21.15.175:0/1513399198 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe458004150 con 0x7fe45c1013f0 2024-06-10T00:59:06.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.087+0000 7fe4527fc700 1 -- 172.21.15.175:0/1513399198 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45800f460 con 0x7fe45c1013f0 2024-06-10T00:59:06.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.087+0000 7fe463482700 1 -- 172.21.15.175:0/1513399198 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe45c103cf0 con 0x7fe45c1013f0 2024-06-10T00:59:06.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.088+0000 7fe463482700 1 -- 172.21.15.175:0/1513399198 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe45c069180 con 0x7fe45c1013f0 2024-06-10T00:59:06.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.089+0000 7fe4527fc700 1 -- 172.21.15.175:0/1513399198 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fe45802a070 con 0x7fe45c1013f0 2024-06-10T00:59:06.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.090+0000 7fe4527fc700 1 --2- 172.21.15.175:0/1513399198 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe4480a2c20 0x7fe4480a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:06.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.090+0000 7fe4527fc700 1 -- 172.21.15.175:0/1513399198 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5397+0+0 (secure 0 0 0) 0x7fe4580cfd00 con 0x7fe45c1013f0 2024-06-10T00:59:06.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.090+0000 7fe460a1d700 1 --2- 172.21.15.175:0/1513399198 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe4480a2c20 0x7fe4480a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:06.091 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.091+0000 7fe460a1d700 1 --2- 172.21.15.175:0/1513399198 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe4480a2c20 0x7fe4480a50e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe44c005d90 tx=0x7fe44c005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:06.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.094+0000 7fe4527fc700 1 -- 172.21.15.175:0/1513399198 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe458031020 con 0x7fe45c1013f0 2024-06-10T00:59:06.299 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:06.299+0000 7fe463482700 1 -- 172.21.15.175:0/1513399198 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe45c0feb90 con 0x7fe4480a2c20 2024-06-10T00:59:06.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:06 smithi078 ceph-mon[36683]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:06.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:06 smithi078 ceph-mon[36683]: osdmap e53: 8 total, 8 up, 8 in 2024-06-10T00:59:06.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:06 smithi078 ceph-mon[36683]: osdmap e54: 8 total, 8 up, 8 in 2024-06-10T00:59:06.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:06 smithi078 ceph-mon[36683]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:59:06.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:06 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:59:06.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:06 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:59:06.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:06 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:06.906 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:06 smithi175 ceph-mon[36636]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:06.906 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:06 smithi175 ceph-mon[36636]: osdmap e53: 8 total, 8 up, 8 in 2024-06-10T00:59:06.906 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:06 smithi175 ceph-mon[36636]: osdmap e54: 8 total, 8 up, 8 in 2024-06-10T00:59:06.906 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:06 smithi175 ceph-mon[36636]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:59:06.907 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:06 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:59:06.907 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:06 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:59:06.907 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:06 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:07.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:06 smithi007 ceph-mon[32398]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:07.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:06 smithi007 ceph-mon[32398]: osdmap e53: 8 total, 8 up, 8 in 2024-06-10T00:59:07.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:06 smithi007 ceph-mon[32398]: osdmap e54: 8 total, 8 up, 8 in 2024-06-10T00:59:07.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:06 smithi007 ceph-mon[32398]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:59:07.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:06 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:59:07.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:06 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:59:07.019 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:06 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:08.022 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:07 smithi175 ceph-mon[36636]: osdmap e55: 8 total, 8 up, 8 in 2024-06-10T00:59:08.084 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:07 smithi078 ceph-mon[36683]: osdmap e55: 8 total, 8 up, 8 in 2024-06-10T00:59:08.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:07 smithi007 ceph-mon[32398]: osdmap e55: 8 total, 8 up, 8 in 2024-06-10T00:59:09.084 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:08 smithi078 ceph-mon[36683]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:09.198 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:08 smithi175 ceph-mon[36636]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:09.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:08 smithi007 ceph-mon[32398]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:10.084 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:09 smithi078 ceph-mon[36683]: from='client.? 172.21.15.175:0/4110665087' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f01cbd9-b193-4c70-a33c-4984167bf422"}]: dispatch 2024-06-10T00:59:10.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:09 smithi078 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f01cbd9-b193-4c70-a33c-4984167bf422"}]: dispatch 2024-06-10T00:59:10.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:09 smithi078 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f01cbd9-b193-4c70-a33c-4984167bf422"}]': finished 2024-06-10T00:59:10.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:09 smithi078 ceph-mon[36683]: osdmap e56: 9 total, 8 up, 9 in 2024-06-10T00:59:10.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:09 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:59:10.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:09 smithi078 ceph-mon[36683]: from='client.? 172.21.15.175:0/1086652694' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:59:10.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:09 smithi175 ceph-mon[36636]: from='client.? 172.21.15.175:0/4110665087' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f01cbd9-b193-4c70-a33c-4984167bf422"}]: dispatch 2024-06-10T00:59:10.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:09 smithi175 ceph-mon[36636]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f01cbd9-b193-4c70-a33c-4984167bf422"}]: dispatch 2024-06-10T00:59:10.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:09 smithi175 ceph-mon[36636]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f01cbd9-b193-4c70-a33c-4984167bf422"}]': finished 2024-06-10T00:59:10.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:09 smithi175 ceph-mon[36636]: osdmap e56: 9 total, 8 up, 9 in 2024-06-10T00:59:10.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:09 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:59:10.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:09 smithi175 ceph-mon[36636]: from='client.? 172.21.15.175:0/1086652694' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:59:10.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:09 smithi007 ceph-mon[32398]: from='client.? 172.21.15.175:0/4110665087' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f01cbd9-b193-4c70-a33c-4984167bf422"}]: dispatch 2024-06-10T00:59:10.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:09 smithi007 ceph-mon[32398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f01cbd9-b193-4c70-a33c-4984167bf422"}]: dispatch 2024-06-10T00:59:10.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:09 smithi007 ceph-mon[32398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f01cbd9-b193-4c70-a33c-4984167bf422"}]': finished 2024-06-10T00:59:10.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:09 smithi007 ceph-mon[32398]: osdmap e56: 9 total, 8 up, 9 in 2024-06-10T00:59:10.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:09 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:59:10.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:09 smithi007 ceph-mon[32398]: from='client.? 172.21.15.175:0/1086652694' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:59:11.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:10 smithi175 ceph-mon[36636]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:11.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:10 smithi007 ceph-mon[32398]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:11.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:10 smithi078 ceph-mon[36683]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:13.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:12 smithi175 ceph-mon[36636]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:13.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:12 smithi007 ceph-mon[32398]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:13.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:12 smithi078 ceph-mon[36683]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:15.189 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:14 smithi175 ceph-mon[36636]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:15.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:14 smithi007 ceph-mon[32398]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:15.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:14 smithi078 ceph-mon[36683]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:17.136 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:16 smithi078 ceph-mon[36683]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:17.137 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:16 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-10T00:59:17.137 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:16 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:17.137 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:16 smithi078 ceph-mon[36683]: Deploying daemon osd.8 on smithi175 2024-06-10T00:59:17.138 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:16 smithi175 ceph-mon[36636]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:17.138 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:16 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-10T00:59:17.138 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:16 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:17.138 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:16 smithi175 ceph-mon[36636]: Deploying daemon osd.8 on smithi175 2024-06-10T00:59:17.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:16 smithi007 ceph-mon[32398]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:17.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:16 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-10T00:59:17.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:16 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:17.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:16 smithi007 ceph-mon[32398]: Deploying daemon osd.8 on smithi175 2024-06-10T00:59:19.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:18 smithi175 ceph-mon[36636]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:19.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:18 smithi007 ceph-mon[32398]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:19.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:18 smithi078 ceph-mon[36683]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:21.024 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:20 smithi175 ceph-mon[36636]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:21.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:20 smithi007 ceph-mon[32398]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:21.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:20 smithi078 ceph-mon[36683]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:23.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:22 smithi175 ceph-mon[36636]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:23.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:22 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:23.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:22 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:23.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:22 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:59:23.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:22 smithi007 ceph-mon[32398]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:23.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:22 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:23.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:22 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:23.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:22 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:59:23.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:22 smithi078 ceph-mon[36683]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:23.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:22 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:23.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:22 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:23.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:22 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:59:25.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:24 smithi175 ceph-mon[36636]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:25.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:24 smithi007 ceph-mon[32398]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:25.318 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:25.318+0000 7fe4527fc700 1 -- 172.21.15.175:0/1513399198 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe45c0feb90 con 0x7fe4480a2c20 2024-06-10T00:59:25.318 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 8 on host 'smithi175' 2024-06-10T00:59:25.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:25.321+0000 7fe463482700 1 -- 172.21.15.175:0/1513399198 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe4480a2c20 msgr2=0x7fe4480a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:25.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:25.321+0000 7fe463482700 1 --2- 172.21.15.175:0/1513399198 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe4480a2c20 0x7fe4480a50e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe44c005d90 tx=0x7fe44c005d20 comp rx=0 tx=0).stop 2024-06-10T00:59:25.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:25.321+0000 7fe463482700 1 -- 172.21.15.175:0/1513399198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe45c1013f0 msgr2=0x7fe45c1987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:25.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:25.321+0000 7fe463482700 1 --2- 172.21.15.175:0/1513399198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe45c1013f0 0x7fe45c1987e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe458000c00 tx=0x7fe458005e80 comp rx=0 tx=0).stop 2024-06-10T00:59:25.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:25.321+0000 7fe463482700 1 -- 172.21.15.175:0/1513399198 shutdown_connections 2024-06-10T00:59:25.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:25.321+0000 7fe463482700 1 --2- 172.21.15.175:0/1513399198 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe4480a2c20 0x7fe4480a50e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:25.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:25.321+0000 7fe463482700 1 --2- 172.21.15.175:0/1513399198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe45c102f90 0x7fe45c19fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:25.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:25.321+0000 7fe463482700 1 --2- 172.21.15.175:0/1513399198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe45c1013f0 0x7fe45c1987e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:25.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:25.321+0000 7fe463482700 1 --2- 172.21.15.175:0/1513399198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe45c1025f0 0x7fe45c198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:25.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:25.321+0000 7fe463482700 1 -- 172.21.15.175:0/1513399198 >> 172.21.15.175:0/1513399198 conn(0x7fe45c0fc990 msgr2=0x7fe45c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:25.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:25.321+0000 7fe463482700 1 -- 172.21.15.175:0/1513399198 shutdown_connections 2024-06-10T00:59:25.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T00:59:25.321+0000 7fe463482700 1 -- 172.21.15.175:0/1513399198 wait complete. 2024-06-10T00:59:25.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:24 smithi078 ceph-mon[36683]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:26.011 DEBUG:teuthology.orchestra.run.smithi175:osd.8> sudo journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.8.service 2024-06-10T00:59:26.013 INFO:tasks.cephadm:Deploying osd.9 on smithi007 with /dev/vg_nvme/lv_1... 2024-06-10T00:59:26.013 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-10T00:59:26.272 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:59:26.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:26 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:26.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:26 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:26.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:26 smithi007 ceph-mon[32398]: from='osd.8 [v2:172.21.15.175:6816/1546325211,v1:172.21.15.175:6817/1546325211]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-10T00:59:26.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:26 smithi007 ceph-mon[32398]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-10T00:59:26.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:26 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:26.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:26 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:26.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:26 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:26.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:26 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:26.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:26 smithi078 ceph-mon[36683]: from='osd.8 [v2:172.21.15.175:6816/1546325211,v1:172.21.15.175:6817/1546325211]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-10T00:59:26.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:26 smithi078 ceph-mon[36683]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-10T00:59:26.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:26 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:26.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:26 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:26.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:26 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:26.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:26 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:26.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:26 smithi175 ceph-mon[36636]: from='osd.8 [v2:172.21.15.175:6816/1546325211,v1:172.21.15.175:6817/1546325211]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-10T00:59:26.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:26 smithi175 ceph-mon[36636]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-10T00:59:26.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:26 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:26.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:26 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:27.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:27 smithi175 ceph-mon[36636]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:27.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:27 smithi175 ceph-mon[36636]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-10T00:59:27.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:27 smithi175 ceph-mon[36636]: osdmap e57: 9 total, 8 up, 9 in 2024-06-10T00:59:27.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:27 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:59:27.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:27 smithi175 ceph-mon[36636]: from='osd.8 [v2:172.21.15.175:6816/1546325211,v1:172.21.15.175:6817/1546325211]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-10T00:59:27.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:27 smithi175 ceph-mon[36636]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-10T00:59:27.458 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 00:59:27 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T00:59:27.286+0000 7f61ced10700 -1 osd.8 0 waiting for initial osdmap 2024-06-10T00:59:27.459 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 00:59:27 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T00:59:27.297+0000 7f61cab03700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:59:27.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:27 smithi078 ceph-mon[36683]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:27.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:27 smithi078 ceph-mon[36683]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-10T00:59:27.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:27 smithi078 ceph-mon[36683]: osdmap e57: 9 total, 8 up, 9 in 2024-06-10T00:59:27.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:27 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:59:27.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:27 smithi078 ceph-mon[36683]: from='osd.8 [v2:172.21.15.175:6816/1546325211,v1:172.21.15.175:6817/1546325211]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-10T00:59:27.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:27 smithi078 ceph-mon[36683]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-10T00:59:27.647 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:27 smithi007 ceph-mon[32398]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:27.647 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:27 smithi007 ceph-mon[32398]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-10T00:59:27.647 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:27 smithi007 ceph-mon[32398]: osdmap e57: 9 total, 8 up, 9 in 2024-06-10T00:59:27.647 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:27 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:59:27.647 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:27 smithi007 ceph-mon[32398]: from='osd.8 [v2:172.21.15.175:6816/1546325211,v1:172.21.15.175:6817/1546325211]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-10T00:59:27.647 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:27 smithi007 ceph-mon[32398]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-10T00:59:28.402 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-10T00:59:28.402 INFO:teuthology.orchestra.run.smithi007.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-10T00:59:28.402 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10+0 records in 2024-06-10T00:59:28.402 INFO:teuthology.orchestra.run.smithi007.stderr:10+0 records out 2024-06-10T00:59:28.402 INFO:teuthology.orchestra.run.smithi007.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00877454 s, 1.2 GB/s 2024-06-10T00:59:28.403 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping successful for: 2024-06-10T00:59:28.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:28 smithi175 ceph-mon[36636]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-06-10T00:59:28.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:28 smithi175 ceph-mon[36636]: osdmap e58: 9 total, 8 up, 9 in 2024-06-10T00:59:28.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:28 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:59:28.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:28 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:59:28.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:28 smithi078 ceph-mon[36683]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-06-10T00:59:28.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:28 smithi078 ceph-mon[36683]: osdmap e58: 9 total, 8 up, 9 in 2024-06-10T00:59:28.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:28 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:59:28.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:28 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:59:28.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:28 smithi007 ceph-mon[32398]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-06-10T00:59:28.717 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:28 smithi007 ceph-mon[32398]: osdmap e58: 9 total, 8 up, 9 in 2024-06-10T00:59:28.717 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:28 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:59:28.717 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:28 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:59:29.131 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi007:vg_nvme/lv_1 2024-06-10T00:59:29.394 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:59:29.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:29 smithi175 ceph-mon[36636]: purged_snaps scrub starts 2024-06-10T00:59:29.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:29 smithi175 ceph-mon[36636]: purged_snaps scrub ok 2024-06-10T00:59:29.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:29 smithi175 ceph-mon[36636]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:29.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:29 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:59:29.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:29 smithi175 ceph-mon[36636]: osd.8 [v2:172.21.15.175:6816/1546325211,v1:172.21.15.175:6817/1546325211] boot 2024-06-10T00:59:29.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:29 smithi175 ceph-mon[36636]: osdmap e59: 9 total, 9 up, 9 in 2024-06-10T00:59:29.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:29 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:59:29.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:29 smithi078 ceph-mon[36683]: purged_snaps scrub starts 2024-06-10T00:59:29.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:29 smithi078 ceph-mon[36683]: purged_snaps scrub ok 2024-06-10T00:59:29.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:29 smithi078 ceph-mon[36683]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:29.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:29 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:59:29.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:29 smithi078 ceph-mon[36683]: osd.8 [v2:172.21.15.175:6816/1546325211,v1:172.21.15.175:6817/1546325211] boot 2024-06-10T00:59:29.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:29 smithi078 ceph-mon[36683]: osdmap e59: 9 total, 9 up, 9 in 2024-06-10T00:59:29.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:29 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:59:29.653 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:29 smithi007 ceph-mon[32398]: purged_snaps scrub starts 2024-06-10T00:59:29.654 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:29 smithi007 ceph-mon[32398]: purged_snaps scrub ok 2024-06-10T00:59:29.654 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:29 smithi007 ceph-mon[32398]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:59:29.654 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:29 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:59:29.654 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:29 smithi007 ceph-mon[32398]: osd.8 [v2:172.21.15.175:6816/1546325211,v1:172.21.15.175:6817/1546325211] boot 2024-06-10T00:59:29.654 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:29 smithi007 ceph-mon[32398]: osdmap e59: 9 total, 9 up, 9 in 2024-06-10T00:59:29.654 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:29 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:59:30.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:30 smithi007 ceph-mon[32398]: osdmap e60: 9 total, 9 up, 9 in 2024-06-10T00:59:30.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:30 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:30.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:30 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:30.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:30 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:59:30.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:30 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:59:30.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:30 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:59:30.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:30 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:30.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:30 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:30.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:30 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:59:30.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:30 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:30.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:30 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:59:30.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:30 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:30.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:30 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:59:30.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:30 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:30.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:30 smithi078 ceph-mon[36683]: osdmap e60: 9 total, 9 up, 9 in 2024-06-10T00:59:30.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:30 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:30.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:30 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:30.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:30 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:59:30.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:30 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:59:30.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:30 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:59:30.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:30 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:30.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:30 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:30.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:30 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:59:30.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:30 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:30.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:30 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:59:30.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:30 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:30.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:30 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:59:30.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:30 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:30.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:30 smithi175 ceph-mon[36636]: osdmap e60: 9 total, 9 up, 9 in 2024-06-10T00:59:30.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:30 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:30.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:30 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:30.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:30 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:59:30.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:30 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:59:30.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:30 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:59:30.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:30 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:30.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:30 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:30.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:30 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:59:30.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:30 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:30.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:30 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:59:30.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:30 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:30.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:30 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:59:30.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:30 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:31.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.075+0000 7fd89c15e700 1 -- 172.21.15.7:0/727477358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd894102080 msgr2=0x7fd894102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:31.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.075+0000 7fd89c15e700 1 --2- 172.21.15.7:0/727477358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd894102080 0x7fd894102500 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd88c00b0a0 tx=0x7fd88c00b3b0 comp rx=0 tx=0).stop 2024-06-10T00:59:31.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.075+0000 7fd89c15e700 1 -- 172.21.15.7:0/727477358 shutdown_connections 2024-06-10T00:59:31.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.075+0000 7fd89c15e700 1 --2- 172.21.15.7:0/727477358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd894102080 0x7fd894102500 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:31.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.075+0000 7fd89c15e700 1 --2- 172.21.15.7:0/727477358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd894100d60 0x7fd894101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:31.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.075+0000 7fd89c15e700 1 --2- 172.21.15.7:0/727477358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8941016c0 0x7fd894101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:31.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.075+0000 7fd89c15e700 1 -- 172.21.15.7:0/727477358 >> 172.21.15.7:0/727477358 conn(0x7fd8940fc960 msgr2=0x7fd8940fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:31.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.075+0000 7fd89c15e700 1 -- 172.21.15.7:0/727477358 shutdown_connections 2024-06-10T00:59:31.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.075+0000 7fd89c15e700 1 -- 172.21.15.7:0/727477358 wait complete. 2024-06-10T00:59:31.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.076+0000 7fd89c15e700 1 Processor -- start 2024-06-10T00:59:31.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.076+0000 7fd89c15e700 1 -- start start 2024-06-10T00:59:31.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.076+0000 7fd89c15e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd894100d60 0x7fd89419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:31.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.076+0000 7fd89c15e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8941016c0 0x7fd89419af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:31.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.076+0000 7fd89c15e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd894102080 0x7fd8941a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:31.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.076+0000 7fd89c15e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd894110380 con 0x7fd8941016c0 2024-06-10T00:59:31.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.076+0000 7fd89c15e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd894110200 con 0x7fd894100d60 2024-06-10T00:59:31.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.076+0000 7fd89c15e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd894110500 con 0x7fd894102080 2024-06-10T00:59:31.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.076+0000 7fd8996f9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8941016c0 0x7fd89419af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:31.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.076+0000 7fd8996f9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8941016c0 0x7fd89419af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59012/0 (socket says 172.21.15.7:59012) 2024-06-10T00:59:31.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.076+0000 7fd8996f9700 1 -- 172.21.15.7:0/412197212 learned_addr learned my addr 172.21.15.7:0/412197212 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T00:59:31.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.076+0000 7fd899efa700 1 --2- 172.21.15.7:0/412197212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd894100d60 0x7fd89419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:31.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.076+0000 7fd89a6fb700 1 --2- 172.21.15.7:0/412197212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd894102080 0x7fd8941a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:31.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.076+0000 7fd8996f9700 1 -- 172.21.15.7:0/412197212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd894102080 msgr2=0x7fd8941a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:31.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.076+0000 7fd8996f9700 1 --2- 172.21.15.7:0/412197212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd894102080 0x7fd8941a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:31.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.076+0000 7fd8996f9700 1 -- 172.21.15.7:0/412197212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd894100d60 msgr2=0x7fd89419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:31.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.076+0000 7fd8996f9700 1 --2- 172.21.15.7:0/412197212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd894100d60 0x7fd89419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:31.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.076+0000 7fd8996f9700 1 -- 172.21.15.7:0/412197212 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8941a25d0 con 0x7fd8941016c0 2024-06-10T00:59:31.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.076+0000 7fd89a6fb700 1 --2- 172.21.15.7:0/412197212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd894102080 0x7fd8941a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:59:31.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.077+0000 7fd8996f9700 1 --2- 172.21.15.7:0/412197212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8941016c0 0x7fd89419af00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd88400b560 tx=0x7fd88400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:31.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.077+0000 7fd88affd700 1 -- 172.21.15.7:0/412197212 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884015020 con 0x7fd8941016c0 2024-06-10T00:59:31.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.077+0000 7fd88affd700 1 -- 172.21.15.7:0/412197212 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd884011d10 con 0x7fd8941016c0 2024-06-10T00:59:31.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.077+0000 7fd89c15e700 1 -- 172.21.15.7:0/412197212 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8941a28c0 con 0x7fd8941016c0 2024-06-10T00:59:31.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.077+0000 7fd88affd700 1 -- 172.21.15.7:0/412197212 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884010e50 con 0x7fd8941016c0 2024-06-10T00:59:31.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.077+0000 7fd89c15e700 1 -- 172.21.15.7:0/412197212 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8941a2e20 con 0x7fd8941016c0 2024-06-10T00:59:31.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.078+0000 7fd89c15e700 1 -- 172.21.15.7:0/412197212 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd89402dca0 con 0x7fd8941016c0 2024-06-10T00:59:31.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.079+0000 7fd88affd700 1 -- 172.21.15.7:0/412197212 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fd884018020 con 0x7fd8941016c0 2024-06-10T00:59:31.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.080+0000 7fd88affd700 1 --2- 172.21.15.7:0/412197212 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd8800a2dc0 0x7fd8800a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:31.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.080+0000 7fd899efa700 1 --2- 172.21.15.7:0/412197212 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd8800a2dc0 0x7fd8800a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:31.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.080+0000 7fd88affd700 1 -- 172.21.15.7:0/412197212 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5829+0+0 (secure 0 0 0) 0x7fd8840cde40 con 0x7fd8941016c0 2024-06-10T00:59:31.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.080+0000 7fd899efa700 1 --2- 172.21.15.7:0/412197212 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd8800a2dc0 0x7fd8800a5280 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd890009a10 tx=0x7fd890009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:31.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.084+0000 7fd88affd700 1 -- 172.21.15.7:0/412197212 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd88401f020 con 0x7fd8941016c0 2024-06-10T00:59:31.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:31.302+0000 7fd89c15e700 1 -- 172.21.15.7:0/412197212 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd8940628d0 con 0x7fd8800a2dc0 2024-06-10T00:59:31.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:31 smithi007 ceph-mon[32398]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:59:31.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:31 smithi007 ceph-mon[32398]: Detected new or changed devices on smithi175 2024-06-10T00:59:31.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:31 smithi007 ceph-mon[32398]: Adjusting osd_memory_target on smithi175 to 5696M 2024-06-10T00:59:31.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:31 smithi007 ceph-mon[32398]: osdmap e61: 9 total, 9 up, 9 in 2024-06-10T00:59:31.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:31 smithi007 ceph-mon[32398]: osdmap e62: 9 total, 9 up, 9 in 2024-06-10T00:59:31.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:31 smithi078 ceph-mon[36683]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:59:31.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:31 smithi078 ceph-mon[36683]: Detected new or changed devices on smithi175 2024-06-10T00:59:31.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:31 smithi078 ceph-mon[36683]: Adjusting osd_memory_target on smithi175 to 5696M 2024-06-10T00:59:31.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:31 smithi078 ceph-mon[36683]: osdmap e61: 9 total, 9 up, 9 in 2024-06-10T00:59:31.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:31 smithi078 ceph-mon[36683]: osdmap e62: 9 total, 9 up, 9 in 2024-06-10T00:59:31.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:31 smithi175 ceph-mon[36636]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:59:31.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:31 smithi175 ceph-mon[36636]: Detected new or changed devices on smithi175 2024-06-10T00:59:31.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:31 smithi175 ceph-mon[36636]: Adjusting osd_memory_target on smithi175 to 5696M 2024-06-10T00:59:31.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:31 smithi175 ceph-mon[36636]: osdmap e61: 9 total, 9 up, 9 in 2024-06-10T00:59:31.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:31 smithi175 ceph-mon[36636]: osdmap e62: 9 total, 9 up, 9 in 2024-06-10T00:59:32.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:32 smithi078 ceph-mon[36683]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:59:32.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:32 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:59:32.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:32 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:59:32.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:32 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:32.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:32 smithi078 ceph-mon[36683]: osdmap e63: 9 total, 9 up, 9 in 2024-06-10T00:59:32.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:32 smithi175 ceph-mon[36636]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:59:32.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:32 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:59:32.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:32 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:59:32.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:32 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:32.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:32 smithi175 ceph-mon[36636]: osdmap e63: 9 total, 9 up, 9 in 2024-06-10T00:59:32.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:32 smithi007 ceph-mon[32398]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:59:32.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:32 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:59:32.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:32 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:59:32.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:32 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:32.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:32 smithi007 ceph-mon[32398]: osdmap e63: 9 total, 9 up, 9 in 2024-06-10T00:59:33.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:33 smithi007 ceph-mon[32398]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:59:33.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:33 smithi078 ceph-mon[36683]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:59:33.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:33 smithi175 ceph-mon[36636]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:59:34.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:34 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2192042539' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e5480998-f7fb-49cd-9aed-52405c88d830"}]: dispatch 2024-06-10T00:59:34.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:34 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2192042539' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e5480998-f7fb-49cd-9aed-52405c88d830"}]': finished 2024-06-10T00:59:34.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:34 smithi078 ceph-mon[36683]: osdmap e64: 10 total, 9 up, 10 in 2024-06-10T00:59:34.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:34 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:59:34.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:34 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2192042539' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e5480998-f7fb-49cd-9aed-52405c88d830"}]: dispatch 2024-06-10T00:59:34.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:34 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2192042539' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e5480998-f7fb-49cd-9aed-52405c88d830"}]': finished 2024-06-10T00:59:34.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:34 smithi175 ceph-mon[36636]: osdmap e64: 10 total, 9 up, 10 in 2024-06-10T00:59:34.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:34 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:59:34.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:34 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2192042539' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e5480998-f7fb-49cd-9aed-52405c88d830"}]: dispatch 2024-06-10T00:59:34.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:34 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2192042539' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e5480998-f7fb-49cd-9aed-52405c88d830"}]': finished 2024-06-10T00:59:34.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:34 smithi007 ceph-mon[32398]: osdmap e64: 10 total, 9 up, 10 in 2024-06-10T00:59:34.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:34 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:59:35.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:35 smithi078 ceph-mon[36683]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:59:35.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:35 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/415674101' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:59:35.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:35 smithi175 ceph-mon[36636]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:59:35.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:35 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/415674101' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:59:35.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:35 smithi007 ceph-mon[32398]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:59:35.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:35 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/415674101' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:59:37.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:37 smithi078 ceph-mon[36683]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-06-10T00:59:37.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:37 smithi175 ceph-mon[36636]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-06-10T00:59:37.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:37 smithi007 ceph-mon[32398]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-06-10T00:59:39.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:39 smithi078 ceph-mon[36683]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-06-10T00:59:39.647 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:39 smithi007 ceph-mon[32398]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-06-10T00:59:39.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:39 smithi175 ceph-mon[36636]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-06-10T00:59:41.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:41 smithi007 ceph-mon[32398]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-10T00:59:41.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:41 smithi078 ceph-mon[36683]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-10T00:59:41.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:41 smithi175 ceph-mon[36636]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-10T00:59:42.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:42 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-10T00:59:42.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:42 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:42.637 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:42 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-10T00:59:42.637 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:42 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:42.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:42 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-10T00:59:42.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:42 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:43.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:43 smithi078 ceph-mon[36683]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-06-10T00:59:43.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:43 smithi078 ceph-mon[36683]: Deploying daemon osd.9 on smithi007 2024-06-10T00:59:43.640 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:43 smithi007 ceph-mon[32398]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-06-10T00:59:43.640 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:43 smithi007 ceph-mon[32398]: Deploying daemon osd.9 on smithi007 2024-06-10T00:59:43.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:43 smithi175 ceph-mon[36636]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-06-10T00:59:43.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:43 smithi175 ceph-mon[36636]: Deploying daemon osd.9 on smithi007 2024-06-10T00:59:45.492 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:45 smithi007 ceph-mon[32398]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-10T00:59:45.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:45 smithi175 ceph-mon[36636]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-10T00:59:45.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:45 smithi078 ceph-mon[36683]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-10T00:59:47.636 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:47 smithi078 ceph-mon[36683]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:59:47.698 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:47 smithi007 ceph-mon[32398]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:59:47.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:47 smithi175 ceph-mon[36636]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:59:49.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:48 smithi175 ceph-mon[36636]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:59:49.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:48 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:49.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:48 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:49.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:48 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:59:49.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:48 smithi007 ceph-mon[32398]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:59:49.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:48 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:49.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:48 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:49.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:48 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:59:49.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:48 smithi078 ceph-mon[36683]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:59:49.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:48 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:49.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:48 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:49.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:48 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:59:51.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:50 smithi175 ceph-mon[36636]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:59:51.248 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:50 smithi007 ceph-mon[32398]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:59:51.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:50 smithi078 ceph-mon[36683]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:59:52.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:52.066+0000 7fd88affd700 1 -- 172.21.15.7:0/412197212 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd8940628d0 con 0x7fd8800a2dc0 2024-06-10T00:59:52.066 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 9 on host 'smithi007' 2024-06-10T00:59:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:52.069+0000 7fd89c15e700 1 -- 172.21.15.7:0/412197212 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd8800a2dc0 msgr2=0x7fd8800a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:52.069+0000 7fd89c15e700 1 --2- 172.21.15.7:0/412197212 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd8800a2dc0 0x7fd8800a5280 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd890009a10 tx=0x7fd890009190 comp rx=0 tx=0).stop 2024-06-10T00:59:52.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:52.069+0000 7fd89c15e700 1 -- 172.21.15.7:0/412197212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8941016c0 msgr2=0x7fd89419af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:52.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:52.069+0000 7fd89c15e700 1 --2- 172.21.15.7:0/412197212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8941016c0 0x7fd89419af00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd88400b560 tx=0x7fd88400b870 comp rx=0 tx=0).stop 2024-06-10T00:59:52.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:52.069+0000 7fd89c15e700 1 -- 172.21.15.7:0/412197212 shutdown_connections 2024-06-10T00:59:52.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:52.069+0000 7fd89c15e700 1 --2- 172.21.15.7:0/412197212 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd8800a2dc0 0x7fd8800a5280 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:52.069+0000 7fd89c15e700 1 --2- 172.21.15.7:0/412197212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd894102080 0x7fd8941a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:52.069+0000 7fd89c15e700 1 --2- 172.21.15.7:0/412197212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd894100d60 0x7fd89419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:52.069+0000 7fd89c15e700 1 --2- 172.21.15.7:0/412197212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8941016c0 0x7fd89419af00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:52.069+0000 7fd89c15e700 1 -- 172.21.15.7:0/412197212 >> 172.21.15.7:0/412197212 conn(0x7fd8940fc960 msgr2=0x7fd8940fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:52.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:52.069+0000 7fd89c15e700 1 -- 172.21.15.7:0/412197212 shutdown_connections 2024-06-10T00:59:52.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T00:59:52.069+0000 7fd89c15e700 1 -- 172.21.15.7:0/412197212 wait complete. 2024-06-10T00:59:52.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:52 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:52.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:52 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:52.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:52 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:52.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:52 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:59:52.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:52 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:52.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:52 smithi007 ceph-mon[32398]: from='osd.9 [v2:172.21.15.7:6826/1844550813,v1:172.21.15.7:6827/1844550813]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-10T00:59:52.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:52 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:52.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:52 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:59:52.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:52 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:52.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:52 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:52.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:52 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:52.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:52 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:52.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:52 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:59:52.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:52 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:52.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:52 smithi175 ceph-mon[36636]: from='osd.9 [v2:172.21.15.7:6826/1844550813,v1:172.21.15.7:6827/1844550813]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-10T00:59:52.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:52 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:52.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:52 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:59:52.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:52 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:52.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:52 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:52.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:52 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:52.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:52 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:52.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:52 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:59:52.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:52 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:52.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:52 smithi078 ceph-mon[36683]: from='osd.9 [v2:172.21.15.7:6826/1844550813,v1:172.21.15.7:6827/1844550813]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-10T00:59:52.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:52 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:52.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:52 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:59:52.585 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:52 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:52.790 DEBUG:teuthology.orchestra.run.smithi007:osd.9> sudo journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.9.service 2024-06-10T00:59:52.792 INFO:tasks.cephadm:Deploying osd.10 on smithi078 with /dev/vg_nvme/lv_1... 2024-06-10T00:59:52.793 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-10T00:59:53.041 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T00:59:53.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:53 smithi007 ceph-mon[32398]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:59:53.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:53 smithi007 ceph-mon[32398]: from='osd.9 [v2:172.21.15.7:6826/1844550813,v1:172.21.15.7:6827/1844550813]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-10T00:59:53.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:53 smithi007 ceph-mon[32398]: osdmap e65: 10 total, 9 up, 10 in 2024-06-10T00:59:53.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:53 smithi007 ceph-mon[32398]: from='osd.9 [v2:172.21.15.7:6826/1844550813,v1:172.21.15.7:6827/1844550813]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-10T00:59:53.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:53 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:59:53.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:53 smithi175 ceph-mon[36636]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:59:53.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:53 smithi175 ceph-mon[36636]: from='osd.9 [v2:172.21.15.7:6826/1844550813,v1:172.21.15.7:6827/1844550813]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-10T00:59:53.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:53 smithi175 ceph-mon[36636]: osdmap e65: 10 total, 9 up, 10 in 2024-06-10T00:59:53.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:53 smithi175 ceph-mon[36636]: from='osd.9 [v2:172.21.15.7:6826/1844550813,v1:172.21.15.7:6827/1844550813]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-10T00:59:53.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:53 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:59:53.474 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:53 smithi078 ceph-mon[36683]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:59:53.474 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:53 smithi078 ceph-mon[36683]: from='osd.9 [v2:172.21.15.7:6826/1844550813,v1:172.21.15.7:6827/1844550813]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-10T00:59:53.474 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:53 smithi078 ceph-mon[36683]: osdmap e65: 10 total, 9 up, 10 in 2024-06-10T00:59:53.474 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:53 smithi078 ceph-mon[36683]: from='osd.9 [v2:172.21.15.7:6826/1844550813,v1:172.21.15.7:6827/1844550813]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-10T00:59:53.474 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:53 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:59:54.018 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 00:59:53 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T00:59:53.593+0000 7f96f2c6b700 -1 osd.9 0 waiting for initial osdmap 2024-06-10T00:59:54.018 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 00:59:53 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T00:59:53.607+0000 7f96eaa56700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:59:54.381 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:54 smithi078 ceph-mon[36683]: from='osd.9 [v2:172.21.15.7:6826/1844550813,v1:172.21.15.7:6827/1844550813]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-06-10T00:59:54.381 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:54 smithi078 ceph-mon[36683]: osdmap e66: 10 total, 9 up, 10 in 2024-06-10T00:59:54.381 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:54 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:59:54.381 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:54 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:59:54.381 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:54 smithi078 ceph-mon[36683]: from='osd.9 [v2:172.21.15.7:6826/1844550813,v1:172.21.15.7:6827/1844550813]' entity='osd.9' 2024-06-10T00:59:54.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:54 smithi175 ceph-mon[36636]: from='osd.9 [v2:172.21.15.7:6826/1844550813,v1:172.21.15.7:6827/1844550813]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-06-10T00:59:54.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:54 smithi175 ceph-mon[36636]: osdmap e66: 10 total, 9 up, 10 in 2024-06-10T00:59:54.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:54 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:59:54.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:54 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:59:54.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:54 smithi175 ceph-mon[36636]: from='osd.9 [v2:172.21.15.7:6826/1844550813,v1:172.21.15.7:6827/1844550813]' entity='osd.9' 2024-06-10T00:59:54.458 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:54 smithi007 ceph-mon[32398]: from='osd.9 [v2:172.21.15.7:6826/1844550813,v1:172.21.15.7:6827/1844550813]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-06-10T00:59:54.458 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:54 smithi007 ceph-mon[32398]: osdmap e66: 10 total, 9 up, 10 in 2024-06-10T00:59:54.459 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:54 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:59:54.459 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:54 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:59:54.459 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:54 smithi007 ceph-mon[32398]: from='osd.9 [v2:172.21.15.7:6826/1844550813,v1:172.21.15.7:6827/1844550813]' entity='osd.9' 2024-06-10T00:59:54.973 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-10T00:59:54.973 INFO:teuthology.orchestra.run.smithi078.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-10T00:59:54.973 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10+0 records in 2024-06-10T00:59:54.973 INFO:teuthology.orchestra.run.smithi078.stderr:10+0 records out 2024-06-10T00:59:54.973 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0117296 s, 894 MB/s 2024-06-10T00:59:54.973 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping successful for: 2024-06-10T00:59:55.440 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:55 smithi078 ceph-mon[36683]: purged_snaps scrub starts 2024-06-10T00:59:55.440 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:55 smithi078 ceph-mon[36683]: purged_snaps scrub ok 2024-06-10T00:59:55.440 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:55 smithi078 ceph-mon[36683]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:59:55.440 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:55 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:59:55.440 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:55 smithi078 ceph-mon[36683]: osd.9 [v2:172.21.15.7:6826/1844550813,v1:172.21.15.7:6827/1844550813] boot 2024-06-10T00:59:55.440 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:55 smithi078 ceph-mon[36683]: osdmap e67: 10 total, 10 up, 10 in 2024-06-10T00:59:55.440 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:55 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:59:55.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:55 smithi175 ceph-mon[36636]: purged_snaps scrub starts 2024-06-10T00:59:55.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:55 smithi175 ceph-mon[36636]: purged_snaps scrub ok 2024-06-10T00:59:55.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:55 smithi175 ceph-mon[36636]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:59:55.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:55 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:59:55.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:55 smithi175 ceph-mon[36636]: osd.9 [v2:172.21.15.7:6826/1844550813,v1:172.21.15.7:6827/1844550813] boot 2024-06-10T00:59:55.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:55 smithi175 ceph-mon[36636]: osdmap e67: 10 total, 10 up, 10 in 2024-06-10T00:59:55.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:55 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:59:55.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:55 smithi007 ceph-mon[32398]: purged_snaps scrub starts 2024-06-10T00:59:55.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:55 smithi007 ceph-mon[32398]: purged_snaps scrub ok 2024-06-10T00:59:55.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:55 smithi007 ceph-mon[32398]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:59:55.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:55 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:59:55.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:55 smithi007 ceph-mon[32398]: osd.9 [v2:172.21.15.7:6826/1844550813,v1:172.21.15.7:6827/1844550813] boot 2024-06-10T00:59:55.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:55 smithi007 ceph-mon[32398]: osdmap e67: 10 total, 10 up, 10 in 2024-06-10T00:59:55.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:55 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:59:55.640 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_1 2024-06-10T00:59:55.882 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T00:59:56.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:56 smithi175 ceph-mon[36636]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:59:56.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:56 smithi175 ceph-mon[36636]: osdmap e68: 10 total, 10 up, 10 in 2024-06-10T00:59:56.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:56 smithi175 ceph-mon[36636]: Detected new or changed devices on smithi007 2024-06-10T00:59:56.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:56 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:56.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:56 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:56.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:56 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:59:56.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:56 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:59:56.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:56 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:59:56.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:56 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:59:56.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:56 smithi175 ceph-mon[36636]: Adjusting osd_memory_target on smithi007 to 4273M 2024-06-10T00:59:56.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:56 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:56.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:56 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:56.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:56 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:59:56.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:56 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:57.003 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:56 smithi078 ceph-mon[36683]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:59:57.003 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:56 smithi078 ceph-mon[36683]: osdmap e68: 10 total, 10 up, 10 in 2024-06-10T00:59:57.003 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:56 smithi078 ceph-mon[36683]: Detected new or changed devices on smithi007 2024-06-10T00:59:57.003 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:56 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:57.003 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:56 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:57.003 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:56 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:59:57.003 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:56 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:59:57.003 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:56 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:59:57.003 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:56 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:59:57.003 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:56 smithi078 ceph-mon[36683]: Adjusting osd_memory_target on smithi007 to 4273M 2024-06-10T00:59:57.004 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:56 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:57.004 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:56 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:57.004 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:56 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:59:57.004 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:56 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:57.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:56 smithi007 ceph-mon[32398]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:59:57.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:56 smithi007 ceph-mon[32398]: osdmap e68: 10 total, 10 up, 10 in 2024-06-10T00:59:57.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:56 smithi007 ceph-mon[32398]: Detected new or changed devices on smithi007 2024-06-10T00:59:57.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:56 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:57.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:56 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:57.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:56 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:59:57.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:56 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:59:57.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:56 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:59:57.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:56 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:59:57.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:56 smithi007 ceph-mon[32398]: Adjusting osd_memory_target on smithi007 to 4273M 2024-06-10T00:59:57.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:56 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:57.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:56 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:57.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:56 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:59:57.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:56 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T00:59:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.528+0000 7f7ad5a11700 1 -- 172.21.15.78:0/3887085318 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ad01013f0 msgr2=0x7f7ad0101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:57.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.528+0000 7f7ad5a11700 1 --2- 172.21.15.78:0/3887085318 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ad01013f0 0x7f7ad0101810 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7ac00099c0 tx=0x7f7ac0009cd0 comp rx=0 tx=0).stop 2024-06-10T00:59:57.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.528+0000 7f7ad5a11700 1 -- 172.21.15.78:0/3887085318 shutdown_connections 2024-06-10T00:59:57.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.528+0000 7f7ad5a11700 1 --2- 172.21.15.78:0/3887085318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad0102f90 0x7f7ad0109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:57.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.528+0000 7f7ad5a11700 1 --2- 172.21.15.78:0/3887085318 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ad01013f0 0x7f7ad0101810 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:57.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.528+0000 7f7ad5a11700 1 --2- 172.21.15.78:0/3887085318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ad01025f0 0x7f7ad0102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:57.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.528+0000 7f7ad5a11700 1 -- 172.21.15.78:0/3887085318 >> 172.21.15.78:0/3887085318 conn(0x7f7ad00fc990 msgr2=0x7f7ad00fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:57.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.528+0000 7f7ad5a11700 1 -- 172.21.15.78:0/3887085318 shutdown_connections 2024-06-10T00:59:57.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.528+0000 7f7ad5a11700 1 -- 172.21.15.78:0/3887085318 wait complete. 2024-06-10T00:59:57.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.529+0000 7f7ad5a11700 1 Processor -- start 2024-06-10T00:59:57.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.529+0000 7f7ad5a11700 1 -- start start 2024-06-10T00:59:57.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.529+0000 7f7ad5a11700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ad01013f0 0x7f7ad0105380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:57.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.529+0000 7f7ad5a11700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ad01025f0 0x7f7ad01039d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:57.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.529+0000 7f7ad5a11700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad0102f90 0x7f7ad0103f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:57.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.529+0000 7f7ad5a11700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad010cfb0 con 0x7f7ad01013f0 2024-06-10T00:59:57.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.529+0000 7f7ad5a11700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad010ce30 con 0x7f7ad01025f0 2024-06-10T00:59:57.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.529+0000 7f7ad5a11700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad010d130 con 0x7f7ad0102f90 2024-06-10T00:59:57.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.529+0000 7f7ac67fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ad01025f0 0x7f7ad01039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:57.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.529+0000 7f7ac67fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ad01025f0 0x7f7ad01039d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39670/0 (socket says 172.21.15.78:39670) 2024-06-10T00:59:57.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.529+0000 7f7ac67fc700 1 -- 172.21.15.78:0/3330575284 learned_addr learned my addr 172.21.15.78:0/3330575284 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-06-10T00:59:57.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.529+0000 7f7aceffd700 1 --2- 172.21.15.78:0/3330575284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ad01013f0 0x7f7ad0105380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:57.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.529+0000 7f7acf7fe700 1 --2- 172.21.15.78:0/3330575284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad0102f90 0x7f7ad0103f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:57.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.529+0000 7f7ac67fc700 1 -- 172.21.15.78:0/3330575284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad0102f90 msgr2=0x7f7ad0103f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:57.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.529+0000 7f7ac67fc700 1 --2- 172.21.15.78:0/3330575284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad0102f90 0x7f7ad0103f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:57.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.529+0000 7f7ac67fc700 1 -- 172.21.15.78:0/3330575284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ad01013f0 msgr2=0x7f7ad0105380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:57.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.529+0000 7f7ac67fc700 1 --2- 172.21.15.78:0/3330575284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ad01013f0 0x7f7ad0105380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:57.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.529+0000 7f7ac67fc700 1 -- 172.21.15.78:0/3330575284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad01047c0 con 0x7f7ad01025f0 2024-06-10T00:59:57.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.529+0000 7f7acf7fe700 1 --2- 172.21.15.78:0/3330575284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad0102f90 0x7f7ad0103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:59:57.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.530+0000 7f7ac67fc700 1 --2- 172.21.15.78:0/3330575284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ad01025f0 0x7f7ad01039d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800d980 tx=0x7f7ab800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:57.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.530+0000 7f7ad4a0f700 1 -- 172.21.15.78:0/3330575284 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8016070 con 0x7f7ad01025f0 2024-06-10T00:59:57.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.530+0000 7f7ad4a0f700 1 -- 172.21.15.78:0/3330575284 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ab8004d10 con 0x7f7ad01025f0 2024-06-10T00:59:57.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.530+0000 7f7ad5a11700 1 -- 172.21.15.78:0/3330575284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ad01a2910 con 0x7f7ad01025f0 2024-06-10T00:59:57.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.530+0000 7f7ad4a0f700 1 -- 172.21.15.78:0/3330575284 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab80055f0 con 0x7f7ad01025f0 2024-06-10T00:59:57.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.530+0000 7f7ad5a11700 1 -- 172.21.15.78:0/3330575284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ad01a2e20 con 0x7f7ad01025f0 2024-06-10T00:59:57.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.531+0000 7f7ad5a11700 1 -- 172.21.15.78:0/3330575284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ad0069180 con 0x7f7ad01025f0 2024-06-10T00:59:57.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.532+0000 7f7ad4a0f700 1 -- 172.21.15.78:0/3330575284 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f7ab8004750 con 0x7f7ad01025f0 2024-06-10T00:59:57.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.532+0000 7f7ad4a0f700 1 --2- 172.21.15.78:0/3330575284 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7abc0a2c20 0x7f7abc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:57.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.532+0000 7f7ad4a0f700 1 -- 172.21.15.78:0/3330575284 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7f7ab80ce690 con 0x7f7ad01025f0 2024-06-10T00:59:57.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.533+0000 7f7aceffd700 1 --2- 172.21.15.78:0/3330575284 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7abc0a2c20 0x7f7abc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:57.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.533+0000 7f7aceffd700 1 --2- 172.21.15.78:0/3330575284 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7abc0a2c20 0x7f7abc0a50e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0009990 tx=0x7f7ac0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:57.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.537+0000 7f7ad4a0f700 1 -- 172.21.15.78:0/3330575284 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7ab801f020 con 0x7f7ad01025f0 2024-06-10T00:59:57.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T00:59:57.740+0000 7f7ad5a11700 1 -- 172.21.15.78:0/3330575284 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f7ad00febd0 con 0x7f7abc0a2c20 2024-06-10T00:59:58.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:58 smithi175 ceph-mon[36636]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:59:58.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:58 smithi175 ceph-mon[36636]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:59:58.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:58 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:59:58.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:58 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:59:58.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 00:59:58 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:59.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:58 smithi007 ceph-mon[32398]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:59:59.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:58 smithi007 ceph-mon[32398]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:59:59.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:58 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:59:59.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:58 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:59:59.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 00:59:58 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:59.084 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:58 smithi078 ceph-mon[36683]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:59:59.084 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:58 smithi078 ceph-mon[36683]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:59:59.084 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:58 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:59:59.084 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:58 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:59:59.084 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 00:59:58 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:00.926 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:00 smithi078 ceph-mon[36683]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:00.926 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:00 smithi078 ceph-mon[36683]: overall HEALTH_OK 2024-06-10T01:00:00.926 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:00 smithi078 ceph-mon[36683]: from='client.? 172.21.15.78:0/3718744471' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caa82f33-a71c-4907-bf3b-30a7fdfeb71d"}]: dispatch 2024-06-10T01:00:00.927 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:00 smithi078 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caa82f33-a71c-4907-bf3b-30a7fdfeb71d"}]: dispatch 2024-06-10T01:00:00.927 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:00 smithi078 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "caa82f33-a71c-4907-bf3b-30a7fdfeb71d"}]': finished 2024-06-10T01:00:00.927 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:00 smithi078 ceph-mon[36683]: osdmap e69: 11 total, 10 up, 11 in 2024-06-10T01:00:00.927 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:00 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T01:00:00.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:00 smithi175 ceph-mon[36636]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:00.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:00 smithi175 ceph-mon[36636]: overall HEALTH_OK 2024-06-10T01:00:00.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:00 smithi175 ceph-mon[36636]: from='client.? 172.21.15.78:0/3718744471' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caa82f33-a71c-4907-bf3b-30a7fdfeb71d"}]: dispatch 2024-06-10T01:00:00.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:00 smithi175 ceph-mon[36636]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caa82f33-a71c-4907-bf3b-30a7fdfeb71d"}]: dispatch 2024-06-10T01:00:00.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:00 smithi175 ceph-mon[36636]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "caa82f33-a71c-4907-bf3b-30a7fdfeb71d"}]': finished 2024-06-10T01:00:00.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:00 smithi175 ceph-mon[36636]: osdmap e69: 11 total, 10 up, 11 in 2024-06-10T01:00:00.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:00 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T01:00:01.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:00 smithi007 ceph-mon[32398]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:01.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:00 smithi007 ceph-mon[32398]: overall HEALTH_OK 2024-06-10T01:00:01.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:00 smithi007 ceph-mon[32398]: from='client.? 172.21.15.78:0/3718744471' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caa82f33-a71c-4907-bf3b-30a7fdfeb71d"}]: dispatch 2024-06-10T01:00:01.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:00 smithi007 ceph-mon[32398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caa82f33-a71c-4907-bf3b-30a7fdfeb71d"}]: dispatch 2024-06-10T01:00:01.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:00 smithi007 ceph-mon[32398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "caa82f33-a71c-4907-bf3b-30a7fdfeb71d"}]': finished 2024-06-10T01:00:01.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:00 smithi007 ceph-mon[32398]: osdmap e69: 11 total, 10 up, 11 in 2024-06-10T01:00:01.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:00 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T01:00:01.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:01 smithi175 ceph-mon[36636]: from='client.? 172.21.15.78:0/2755901763' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T01:00:02.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:01 smithi007 ceph-mon[32398]: from='client.? 172.21.15.78:0/2755901763' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T01:00:02.084 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:01 smithi078 ceph-mon[36683]: from='client.? 172.21.15.78:0/2755901763' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T01:00:02.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:02 smithi175 ceph-mon[36636]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:03.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:02 smithi007 ceph-mon[32398]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:03.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:02 smithi078 ceph-mon[36683]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:04.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:04 smithi078 ceph-mon[36683]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:04.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:04 smithi175 ceph-mon[36636]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:05.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:04 smithi007 ceph-mon[32398]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:06.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:06 smithi078 ceph-mon[36683]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:06.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:06 smithi175 ceph-mon[36636]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:07.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:06 smithi007 ceph-mon[32398]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:08.881 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:08 smithi078 ceph-mon[36683]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:08.881 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:08 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-10T01:00:08.881 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:08 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:08.881 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:08 smithi078 ceph-mon[36683]: Deploying daemon osd.10 on smithi078 2024-06-10T01:00:08.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:08 smithi175 ceph-mon[36636]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:08.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:08 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-10T01:00:08.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:08 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:08.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:08 smithi175 ceph-mon[36636]: Deploying daemon osd.10 on smithi078 2024-06-10T01:00:09.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:08 smithi007 ceph-mon[32398]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:09.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:08 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-10T01:00:09.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:08 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:09.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:08 smithi007 ceph-mon[32398]: Deploying daemon osd.10 on smithi078 2024-06-10T01:00:10.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:10 smithi175 ceph-mon[36636]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:11.015 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:10 smithi078 ceph-mon[36683]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:11.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:10 smithi007 ceph-mon[32398]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:12.943 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:12 smithi078 ceph-mon[36683]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:12.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:12 smithi175 ceph-mon[36636]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:13.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:12 smithi007 ceph-mon[32398]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:14.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:14 smithi078 ceph-mon[36683]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:14.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:14 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:14.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:14 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:14.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:14 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:00:14.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:14 smithi175 ceph-mon[36636]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:14.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:14 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:14.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:14 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:14.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:14 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:00:15.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:14 smithi007 ceph-mon[32398]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:15.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:14 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:15.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:14 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:15.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:14 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:00:16.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:16 smithi078 ceph-mon[36683]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:16.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T01:00:16.914+0000 7f7ad4a0f700 1 -- 172.21.15.78:0/3330575284 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f7ad00febd0 con 0x7f7abc0a2c20 2024-06-10T01:00:16.914 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 10 on host 'smithi078' 2024-06-10T01:00:16.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T01:00:16.917+0000 7f7ad5a11700 1 -- 172.21.15.78:0/3330575284 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7abc0a2c20 msgr2=0x7f7abc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:16.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T01:00:16.917+0000 7f7ad5a11700 1 --2- 172.21.15.78:0/3330575284 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7abc0a2c20 0x7f7abc0a50e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0009990 tx=0x7f7ac0005fb0 comp rx=0 tx=0).stop 2024-06-10T01:00:16.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T01:00:16.917+0000 7f7ad5a11700 1 -- 172.21.15.78:0/3330575284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ad01025f0 msgr2=0x7f7ad01039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:16.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T01:00:16.917+0000 7f7ad5a11700 1 --2- 172.21.15.78:0/3330575284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ad01025f0 0x7f7ad01039d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800d980 tx=0x7f7ab800dc90 comp rx=0 tx=0).stop 2024-06-10T01:00:16.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T01:00:16.917+0000 7f7ad5a11700 1 -- 172.21.15.78:0/3330575284 shutdown_connections 2024-06-10T01:00:16.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T01:00:16.917+0000 7f7ad5a11700 1 --2- 172.21.15.78:0/3330575284 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7abc0a2c20 0x7f7abc0a50e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:16.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T01:00:16.917+0000 7f7ad5a11700 1 --2- 172.21.15.78:0/3330575284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad0102f90 0x7f7ad0103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:16.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T01:00:16.917+0000 7f7ad5a11700 1 --2- 172.21.15.78:0/3330575284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ad01013f0 0x7f7ad0105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:16.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T01:00:16.917+0000 7f7ad5a11700 1 --2- 172.21.15.78:0/3330575284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ad01025f0 0x7f7ad01039d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:16.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T01:00:16.917+0000 7f7ad5a11700 1 -- 172.21.15.78:0/3330575284 >> 172.21.15.78:0/3330575284 conn(0x7f7ad00fc990 msgr2=0x7f7ad00fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:16.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T01:00:16.917+0000 7f7ad5a11700 1 -- 172.21.15.78:0/3330575284 shutdown_connections 2024-06-10T01:00:16.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-06-10T01:00:16.917+0000 7f7ad5a11700 1 -- 172.21.15.78:0/3330575284 wait complete. 2024-06-10T01:00:16.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:16 smithi175 ceph-mon[36636]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:17.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:16 smithi007 ceph-mon[32398]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:17.606 DEBUG:teuthology.orchestra.run.smithi078:osd.10> sudo journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.10.service 2024-06-10T01:00:17.608 INFO:tasks.cephadm:Deploying osd.11 on smithi175 with /dev/vg_nvme/lv_1... 2024-06-10T01:00:17.608 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-10T01:00:17.855 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T01:00:18.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:17 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:18.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:17 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:18.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:17 smithi175 ceph-mon[36636]: from='osd.10 [v2:172.21.15.78:6824/2290824785,v1:172.21.15.78:6825/2290824785]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-10T01:00:18.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:17 smithi175 ceph-mon[36636]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-10T01:00:18.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:17 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:18.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:17 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:18.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:17 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:18.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:17 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:18.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:17 smithi007 ceph-mon[32398]: from='osd.10 [v2:172.21.15.78:6824/2290824785,v1:172.21.15.78:6825/2290824785]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-10T01:00:18.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:17 smithi007 ceph-mon[32398]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-10T01:00:18.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:17 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:18.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:17 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:18.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:17 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:18.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:17 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:18.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:17 smithi078 ceph-mon[36683]: from='osd.10 [v2:172.21.15.78:6824/2290824785,v1:172.21.15.78:6825/2290824785]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-10T01:00:18.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:17 smithi078 ceph-mon[36683]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-10T01:00:18.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:17 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:18.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:17 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:19.084 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:18 smithi078 ceph-mon[36683]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:19.084 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:18 smithi078 ceph-mon[36683]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-10T01:00:19.084 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:18 smithi078 ceph-mon[36683]: osdmap e70: 11 total, 10 up, 11 in 2024-06-10T01:00:19.084 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:18 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T01:00:19.084 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:18 smithi078 ceph-mon[36683]: from='osd.10 [v2:172.21.15.78:6824/2290824785,v1:172.21.15.78:6825/2290824785]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-06-10T01:00:19.084 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:18 smithi078 ceph-mon[36683]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-06-10T01:00:19.197 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:18 smithi175 ceph-mon[36636]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:19.197 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:18 smithi175 ceph-mon[36636]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-10T01:00:19.198 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:18 smithi175 ceph-mon[36636]: osdmap e70: 11 total, 10 up, 11 in 2024-06-10T01:00:19.198 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:18 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T01:00:19.198 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:18 smithi175 ceph-mon[36636]: from='osd.10 [v2:172.21.15.78:6824/2290824785,v1:172.21.15.78:6825/2290824785]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-06-10T01:00:19.198 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:18 smithi175 ceph-mon[36636]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-06-10T01:00:19.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:18 smithi007 ceph-mon[32398]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:19.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:18 smithi007 ceph-mon[32398]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-10T01:00:19.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:18 smithi007 ceph-mon[32398]: osdmap e70: 11 total, 10 up, 11 in 2024-06-10T01:00:19.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:18 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T01:00:19.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:18 smithi007 ceph-mon[32398]: from='osd.10 [v2:172.21.15.78:6824/2290824785,v1:172.21.15.78:6825/2290824785]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-06-10T01:00:19.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:18 smithi007 ceph-mon[32398]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-06-10T01:00:19.676 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:00:19 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:00:19.420+0000 7f1286376700 -1 osd.10 0 waiting for initial osdmap 2024-06-10T01:00:19.676 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:00:19 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:00:19.433+0000 7f127d960700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T01:00:19.885 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-10T01:00:19.885 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-10T01:00:19.885 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-06-10T01:00:19.885 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-06-10T01:00:19.885 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011477 s, 914 MB/s 2024-06-10T01:00:19.885 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-06-10T01:00:20.004 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:19 smithi078 ceph-mon[36683]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-06-10T01:00:20.005 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:19 smithi078 ceph-mon[36683]: osdmap e71: 11 total, 10 up, 11 in 2024-06-10T01:00:20.005 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:19 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T01:00:20.005 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:19 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T01:00:20.005 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:19 smithi078 ceph-mon[36683]: from='osd.10 ' entity='osd.10' 2024-06-10T01:00:20.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:19 smithi175 ceph-mon[36636]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-06-10T01:00:20.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:19 smithi175 ceph-mon[36636]: osdmap e71: 11 total, 10 up, 11 in 2024-06-10T01:00:20.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:19 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T01:00:20.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:19 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T01:00:20.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:19 smithi175 ceph-mon[36636]: from='osd.10 ' entity='osd.10' 2024-06-10T01:00:20.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:19 smithi007 ceph-mon[32398]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-06-10T01:00:20.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:19 smithi007 ceph-mon[32398]: osdmap e71: 11 total, 10 up, 11 in 2024-06-10T01:00:20.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:19 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T01:00:20.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:19 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T01:00:20.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:19 smithi007 ceph-mon[32398]: from='osd.10 ' entity='osd.10' 2024-06-10T01:00:20.655 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi175:vg_nvme/lv_1 2024-06-10T01:00:20.897 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T01:00:21.182 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:20 smithi175 ceph-mon[36636]: purged_snaps scrub starts 2024-06-10T01:00:21.183 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:20 smithi175 ceph-mon[36636]: purged_snaps scrub ok 2024-06-10T01:00:21.183 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:20 smithi175 ceph-mon[36636]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:21.183 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T01:00:21.183 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:20 smithi175 ceph-mon[36636]: osd.10 [v2:172.21.15.78:6824/2290824785,v1:172.21.15.78:6825/2290824785] boot 2024-06-10T01:00:21.183 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:20 smithi175 ceph-mon[36636]: osdmap e72: 11 total, 11 up, 11 in 2024-06-10T01:00:21.183 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T01:00:21.183 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:20 smithi175 ceph-mon[36636]: osdmap e73: 11 total, 11 up, 11 in 2024-06-10T01:00:21.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:20 smithi007 ceph-mon[32398]: purged_snaps scrub starts 2024-06-10T01:00:21.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:20 smithi007 ceph-mon[32398]: purged_snaps scrub ok 2024-06-10T01:00:21.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:20 smithi007 ceph-mon[32398]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:21.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:20 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T01:00:21.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:20 smithi007 ceph-mon[32398]: osd.10 [v2:172.21.15.78:6824/2290824785,v1:172.21.15.78:6825/2290824785] boot 2024-06-10T01:00:21.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:20 smithi007 ceph-mon[32398]: osdmap e72: 11 total, 11 up, 11 in 2024-06-10T01:00:21.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:20 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T01:00:21.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:20 smithi007 ceph-mon[32398]: osdmap e73: 11 total, 11 up, 11 in 2024-06-10T01:00:21.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:20 smithi078 ceph-mon[36683]: purged_snaps scrub starts 2024-06-10T01:00:21.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:20 smithi078 ceph-mon[36683]: purged_snaps scrub ok 2024-06-10T01:00:21.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:20 smithi078 ceph-mon[36683]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-06-10T01:00:21.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:20 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T01:00:21.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:20 smithi078 ceph-mon[36683]: osd.10 [v2:172.21.15.78:6824/2290824785,v1:172.21.15.78:6825/2290824785] boot 2024-06-10T01:00:21.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:20 smithi078 ceph-mon[36683]: osdmap e72: 11 total, 11 up, 11 in 2024-06-10T01:00:21.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:20 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T01:00:21.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:20 smithi078 ceph-mon[36683]: osdmap e73: 11 total, 11 up, 11 in 2024-06-10T01:00:22.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.579+0000 7f335ecdf700 1 -- 172.21.15.175:0/1030164484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33581025f0 msgr2=0x7f3358102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:22.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.579+0000 7f335ecdf700 1 --2- 172.21.15.175:0/1030164484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33581025f0 0x7f3358102a50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3344009990 tx=0x7f3344009ca0 comp rx=0 tx=0).stop 2024-06-10T01:00:22.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.579+0000 7f335ecdf700 1 -- 172.21.15.175:0/1030164484 shutdown_connections 2024-06-10T01:00:22.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.579+0000 7f335ecdf700 1 --2- 172.21.15.175:0/1030164484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3358102f90 0x7f3358109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:22.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.579+0000 7f335ecdf700 1 --2- 172.21.15.175:0/1030164484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33581013f0 0x7f3358101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:22.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.579+0000 7f335ecdf700 1 --2- 172.21.15.175:0/1030164484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33581025f0 0x7f3358102a50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:22.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.579+0000 7f335ecdf700 1 -- 172.21.15.175:0/1030164484 >> 172.21.15.175:0/1030164484 conn(0x7f33580fc9b0 msgr2=0x7f33580fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:22.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.579+0000 7f335ecdf700 1 -- 172.21.15.175:0/1030164484 shutdown_connections 2024-06-10T01:00:22.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.579+0000 7f335ecdf700 1 -- 172.21.15.175:0/1030164484 wait complete. 2024-06-10T01:00:22.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.579+0000 7f335ecdf700 1 Processor -- start 2024-06-10T01:00:22.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.579+0000 7f335ecdf700 1 -- start start 2024-06-10T01:00:22.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.579+0000 7f335ecdf700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33581013f0 0x7f335806b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:22.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.579+0000 7f335ecdf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33581025f0 0x7f335819d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:22.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.579+0000 7f335ecdf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3358102f90 0x7f335819fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:22.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.579+0000 7f335ecdf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f335810cfd0 con 0x7f3358102f90 2024-06-10T01:00:22.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.580+0000 7f335ecdf700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f335810ce50 con 0x7f33581025f0 2024-06-10T01:00:22.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.580+0000 7f335ecdf700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f335810d150 con 0x7f33581013f0 2024-06-10T01:00:22.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.580+0000 7f335ca7b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33581013f0 0x7f335806b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:22.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.580+0000 7f335ca7b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33581013f0 0x7f335806b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:33772/0 (socket says 172.21.15.175:33772) 2024-06-10T01:00:22.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.580+0000 7f335ca7b700 1 -- 172.21.15.175:0/576010188 learned_addr learned my addr 172.21.15.175:0/576010188 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-06-10T01:00:22.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.580+0000 7f3357fff700 1 --2- 172.21.15.175:0/576010188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33581025f0 0x7f335819d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:22.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.580+0000 7f335d27c700 1 --2- 172.21.15.175:0/576010188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3358102f90 0x7f335819fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:22.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.580+0000 7f335ca7b700 1 -- 172.21.15.175:0/576010188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33581025f0 msgr2=0x7f335819d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:22.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.580+0000 7f335ca7b700 1 --2- 172.21.15.175:0/576010188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33581025f0 0x7f335819d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:22.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.580+0000 7f335ca7b700 1 -- 172.21.15.175:0/576010188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3358102f90 msgr2=0x7f335819fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:22.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.580+0000 7f335ca7b700 1 --2- 172.21.15.175:0/576010188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3358102f90 0x7f335819fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:22.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.580+0000 7f335ca7b700 1 -- 172.21.15.175:0/576010188 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33581a0340 con 0x7f33581013f0 2024-06-10T01:00:22.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.580+0000 7f3357fff700 1 --2- 172.21.15.175:0/576010188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33581025f0 0x7f335819d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:00:22.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.580+0000 7f335d27c700 1 --2- 172.21.15.175:0/576010188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3358102f90 0x7f335819fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:00:22.583 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.580+0000 7f335ca7b700 1 --2- 172.21.15.175:0/576010188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33581013f0 0x7f335806b780 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f334c00d980 tx=0x7f334c00dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:22.583 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.580+0000 7f3355ffb700 1 -- 172.21.15.175:0/576010188 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334c017030 con 0x7f33581013f0 2024-06-10T01:00:22.583 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.580+0000 7f3355ffb700 1 -- 172.21.15.175:0/576010188 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f334c004500 con 0x7f33581013f0 2024-06-10T01:00:22.583 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.580+0000 7f335ecdf700 1 -- 172.21.15.175:0/576010188 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33581a0630 con 0x7f33581013f0 2024-06-10T01:00:22.583 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.580+0000 7f3355ffb700 1 -- 172.21.15.175:0/576010188 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334c003b20 con 0x7f33581013f0 2024-06-10T01:00:22.583 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.580+0000 7f335ecdf700 1 -- 172.21.15.175:0/576010188 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33581a0b70 con 0x7f33581013f0 2024-06-10T01:00:22.583 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.581+0000 7f335ecdf700 1 -- 172.21.15.175:0/576010188 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3358069180 con 0x7f33581013f0 2024-06-10T01:00:22.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.582+0000 7f3355ffb700 1 -- 172.21.15.175:0/576010188 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f334c01a070 con 0x7f33581013f0 2024-06-10T01:00:22.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.583+0000 7f3355ffb700 1 --2- 172.21.15.175:0/576010188 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f33480a2c20 0x7f33480a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:22.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.583+0000 7f3355ffb700 1 -- 172.21.15.175:0/576010188 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6723+0+0 (secure 0 0 0) 0x7f334c0ce400 con 0x7f33581013f0 2024-06-10T01:00:22.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.583+0000 7f3357fff700 1 --2- 172.21.15.175:0/576010188 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f33480a2c20 0x7f33480a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:22.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.584+0000 7f3357fff700 1 --2- 172.21.15.175:0/576010188 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f33480a2c20 0x7f33480a50e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f33440039c0 tx=0x7f3344005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:22.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.587+0000 7f3355ffb700 1 -- 172.21.15.175:0/576010188 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f334c009940 con 0x7f33581013f0 2024-06-10T01:00:22.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:22 smithi175 ceph-mon[36636]: Detected new or changed devices on smithi078 2024-06-10T01:00:22.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:22 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:22.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:22 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:22.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:22 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:22.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:22 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:22.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:22 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:22.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:22 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:22.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:22 smithi175 ceph-mon[36636]: Adjusting osd_memory_target on smithi078 to 4272M 2024-06-10T01:00:22.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:22 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:22.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:22 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:22.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:22 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:00:22.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:22 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:22.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:22 smithi175 ceph-mon[36636]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-06-10T01:00:22.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:22 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:00:22.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:22 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:22.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:22 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:00:22.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:22 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:22.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:22 smithi175 ceph-mon[36636]: osdmap e74: 11 total, 11 up, 11 in 2024-06-10T01:00:22.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:22 smithi007 ceph-mon[32398]: Detected new or changed devices on smithi078 2024-06-10T01:00:22.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:22 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:22.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:22 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:22.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:22 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:22.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:22 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:22.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:22 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:22.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:22 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:22.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:22 smithi007 ceph-mon[32398]: Adjusting osd_memory_target on smithi078 to 4272M 2024-06-10T01:00:22.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:22 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:22.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:22 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:22.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:22 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:00:22.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:22 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:22.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:22 smithi007 ceph-mon[32398]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-06-10T01:00:22.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:22 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:00:22.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:22 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:22.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:22 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:00:22.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:22 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:22.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:22 smithi007 ceph-mon[32398]: osdmap e74: 11 total, 11 up, 11 in 2024-06-10T01:00:22.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:22.797+0000 7f335ecdf700 1 -- 172.21.15.175:0/576010188 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f33580628d0 con 0x7f33480a2c20 2024-06-10T01:00:22.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:22 smithi078 ceph-mon[36683]: Detected new or changed devices on smithi078 2024-06-10T01:00:22.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:22 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:22.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:22 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:22.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:22 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:22.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:22 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:22.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:22 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:22.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:22 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:22.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:22 smithi078 ceph-mon[36683]: Adjusting osd_memory_target on smithi078 to 4272M 2024-06-10T01:00:22.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:22 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:22.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:22 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:22.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:22 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:00:22.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:22 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:22.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:22 smithi078 ceph-mon[36683]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-06-10T01:00:22.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:22 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:00:22.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:22 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:22.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:22 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:00:22.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:22 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:22.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:22 smithi078 ceph-mon[36683]: osdmap e74: 11 total, 11 up, 11 in 2024-06-10T01:00:23.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:23 smithi175 ceph-mon[36636]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:00:23.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:23 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T01:00:23.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:23 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T01:00:23.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:23 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:23.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:23 smithi175 ceph-mon[36636]: osdmap e75: 11 total, 11 up, 11 in 2024-06-10T01:00:23.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:23 smithi007 ceph-mon[32398]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:00:23.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:23 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T01:00:23.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:23 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T01:00:23.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:23 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:23.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:23 smithi007 ceph-mon[32398]: osdmap e75: 11 total, 11 up, 11 in 2024-06-10T01:00:23.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:23 smithi078 ceph-mon[36683]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:00:23.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:23 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T01:00:23.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:23 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T01:00:23.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:23 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:23.835 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:23 smithi078 ceph-mon[36683]: osdmap e75: 11 total, 11 up, 11 in 2024-06-10T01:00:25.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:24 smithi078 ceph-mon[36683]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-06-10T01:00:25.084 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:24 smithi078 ceph-mon[36683]: osdmap e76: 11 total, 11 up, 11 in 2024-06-10T01:00:25.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:24 smithi175 ceph-mon[36636]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-06-10T01:00:25.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:24 smithi175 ceph-mon[36636]: osdmap e76: 11 total, 11 up, 11 in 2024-06-10T01:00:25.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:24 smithi007 ceph-mon[32398]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-06-10T01:00:25.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:24 smithi007 ceph-mon[32398]: osdmap e76: 11 total, 11 up, 11 in 2024-06-10T01:00:26.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:25 smithi078 ceph-mon[36683]: from='client.? 172.21.15.175:0/1558974883' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "608cda77-77da-4da2-a7e2-76a773be27c8"}]: dispatch 2024-06-10T01:00:26.084 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:25 smithi078 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "608cda77-77da-4da2-a7e2-76a773be27c8"}]: dispatch 2024-06-10T01:00:26.084 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:25 smithi078 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "608cda77-77da-4da2-a7e2-76a773be27c8"}]': finished 2024-06-10T01:00:26.084 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:25 smithi078 ceph-mon[36683]: osdmap e77: 12 total, 11 up, 12 in 2024-06-10T01:00:26.084 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:25 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T01:00:26.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:25 smithi175 ceph-mon[36636]: from='client.? 172.21.15.175:0/1558974883' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "608cda77-77da-4da2-a7e2-76a773be27c8"}]: dispatch 2024-06-10T01:00:26.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:25 smithi175 ceph-mon[36636]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "608cda77-77da-4da2-a7e2-76a773be27c8"}]: dispatch 2024-06-10T01:00:26.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:25 smithi175 ceph-mon[36636]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "608cda77-77da-4da2-a7e2-76a773be27c8"}]': finished 2024-06-10T01:00:26.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:25 smithi175 ceph-mon[36636]: osdmap e77: 12 total, 11 up, 12 in 2024-06-10T01:00:26.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:25 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T01:00:26.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:25 smithi007 ceph-mon[32398]: from='client.? 172.21.15.175:0/1558974883' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "608cda77-77da-4da2-a7e2-76a773be27c8"}]: dispatch 2024-06-10T01:00:26.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:25 smithi007 ceph-mon[32398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "608cda77-77da-4da2-a7e2-76a773be27c8"}]: dispatch 2024-06-10T01:00:26.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:25 smithi007 ceph-mon[32398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "608cda77-77da-4da2-a7e2-76a773be27c8"}]': finished 2024-06-10T01:00:26.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:25 smithi007 ceph-mon[32398]: osdmap e77: 12 total, 11 up, 12 in 2024-06-10T01:00:26.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:25 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T01:00:27.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:26 smithi078 ceph-mon[36683]: pgmap v241: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-06-10T01:00:27.084 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:26 smithi078 ceph-mon[36683]: from='client.? 172.21.15.175:0/4168400808' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T01:00:27.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:26 smithi175 ceph-mon[36636]: pgmap v241: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-06-10T01:00:27.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:26 smithi175 ceph-mon[36636]: from='client.? 172.21.15.175:0/4168400808' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T01:00:27.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:26 smithi007 ceph-mon[32398]: pgmap v241: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-06-10T01:00:27.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:26 smithi007 ceph-mon[32398]: from='client.? 172.21.15.175:0/4168400808' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T01:00:29.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:28 smithi078 ceph-mon[36683]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-06-10T01:00:29.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:28 smithi175 ceph-mon[36636]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-06-10T01:00:29.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:28 smithi007 ceph-mon[32398]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-06-10T01:00:31.002 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:30 smithi175 ceph-mon[36636]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-06-10T01:00:31.083 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:30 smithi007 ceph-mon[32398]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-06-10T01:00:31.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:30 smithi078 ceph-mon[36683]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-06-10T01:00:32.999 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:32 smithi175 ceph-mon[36636]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-10T01:00:33.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:32 smithi078 ceph-mon[36683]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-10T01:00:33.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:32 smithi007 ceph-mon[32398]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-10T01:00:33.910 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:33 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-10T01:00:33.910 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:33 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:33.911 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:33 smithi175 ceph-mon[36636]: Deploying daemon osd.11 on smithi175 2024-06-10T01:00:34.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:33 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-10T01:00:34.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:33 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:34.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:33 smithi007 ceph-mon[32398]: Deploying daemon osd.11 on smithi175 2024-06-10T01:00:34.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:33 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-10T01:00:34.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:33 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:34.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:33 smithi078 ceph-mon[36683]: Deploying daemon osd.11 on smithi175 2024-06-10T01:00:35.088 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:34 smithi175 ceph-mon[36636]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-06-10T01:00:35.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:34 smithi007 ceph-mon[32398]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-06-10T01:00:35.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:34 smithi078 ceph-mon[36683]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-06-10T01:00:37.029 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:36 smithi175 ceph-mon[36636]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-06-10T01:00:37.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:36 smithi007 ceph-mon[32398]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-06-10T01:00:37.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:36 smithi078 ceph-mon[36683]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-06-10T01:00:39.102 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:38 smithi175 ceph-mon[36636]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-06-10T01:00:39.102 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:38 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:39.102 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:38 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:39.102 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:38 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:00:39.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:38 smithi007 ceph-mon[32398]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-06-10T01:00:39.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:38 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:39.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:38 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:39.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:38 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:00:39.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:38 smithi078 ceph-mon[36683]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-06-10T01:00:39.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:38 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:39.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:38 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:39.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:38 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:00:40.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:40 smithi175 ceph-mon[36636]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-06-10T01:00:41.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:40 smithi007 ceph-mon[32398]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-06-10T01:00:41.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:40 smithi078 ceph-mon[36683]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-06-10T01:00:42.871 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:42.871+0000 7f3355ffb700 1 -- 172.21.15.175:0/576010188 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f33580628d0 con 0x7f33480a2c20 2024-06-10T01:00:42.872 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 11 on host 'smithi175' 2024-06-10T01:00:42.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:42.873+0000 7f335ecdf700 1 -- 172.21.15.175:0/576010188 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f33480a2c20 msgr2=0x7f33480a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:42.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:42.873+0000 7f335ecdf700 1 --2- 172.21.15.175:0/576010188 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f33480a2c20 0x7f33480a50e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f33440039c0 tx=0x7f3344005fb0 comp rx=0 tx=0).stop 2024-06-10T01:00:42.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:42.873+0000 7f335ecdf700 1 -- 172.21.15.175:0/576010188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33581013f0 msgr2=0x7f335806b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:42.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:42.873+0000 7f335ecdf700 1 --2- 172.21.15.175:0/576010188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33581013f0 0x7f335806b780 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f334c00d980 tx=0x7f334c00dd40 comp rx=0 tx=0).stop 2024-06-10T01:00:42.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:42.873+0000 7f335ecdf700 1 -- 172.21.15.175:0/576010188 shutdown_connections 2024-06-10T01:00:42.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:42.873+0000 7f335ecdf700 1 --2- 172.21.15.175:0/576010188 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f33480a2c20 0x7f33480a50e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:42.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:42.873+0000 7f335ecdf700 1 --2- 172.21.15.175:0/576010188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3358102f90 0x7f335819fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:42.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:42.873+0000 7f335ecdf700 1 --2- 172.21.15.175:0/576010188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33581013f0 0x7f335806b780 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:42.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:42.873+0000 7f335ecdf700 1 --2- 172.21.15.175:0/576010188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33581025f0 0x7f335819d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:42.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:42.873+0000 7f335ecdf700 1 -- 172.21.15.175:0/576010188 >> 172.21.15.175:0/576010188 conn(0x7f33580fc9b0 msgr2=0x7f3358105970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:42.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:42.874+0000 7f335ecdf700 1 -- 172.21.15.175:0/576010188 shutdown_connections 2024-06-10T01:00:42.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:42.874+0000 7f335ecdf700 1 -- 172.21.15.175:0/576010188 wait complete. 2024-06-10T01:00:43.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:42 smithi175 ceph-mon[36636]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-06-10T01:00:43.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:42 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:43.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:42 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:43.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:42 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:43.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:42 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:00:43.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:42 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:43.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:42 smithi175 ceph-mon[36636]: from='osd.11 [v2:172.21.15.175:6824/1959979550,v1:172.21.15.175:6825/1959979550]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-10T01:00:43.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:42 smithi175 ceph-mon[36636]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-10T01:00:43.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:42 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:43.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:42 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:43.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:42 smithi007 ceph-mon[32398]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-06-10T01:00:43.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:42 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:43.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:42 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:43.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:42 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:43.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:42 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:00:43.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:42 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:43.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:42 smithi007 ceph-mon[32398]: from='osd.11 [v2:172.21.15.175:6824/1959979550,v1:172.21.15.175:6825/1959979550]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-10T01:00:43.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:42 smithi007 ceph-mon[32398]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-10T01:00:43.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:42 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:43.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:42 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:43.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:42 smithi078 ceph-mon[36683]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-06-10T01:00:43.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:42 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:43.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:42 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:43.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:42 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:43.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:42 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:00:43.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:42 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:43.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:42 smithi078 ceph-mon[36683]: from='osd.11 [v2:172.21.15.175:6824/1959979550,v1:172.21.15.175:6825/1959979550]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-10T01:00:43.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:42 smithi078 ceph-mon[36683]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-10T01:00:43.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:42 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:43.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:42 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:43.719 DEBUG:teuthology.orchestra.run.smithi175:osd.11> sudo journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.11.service 2024-06-10T01:00:43.801 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-06-10T01:00:43.801 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd stat -f json 2024-06-10T01:00:43.942 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:43 smithi175 ceph-mon[36636]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-10T01:00:43.942 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:43 smithi175 ceph-mon[36636]: osdmap e78: 12 total, 11 up, 12 in 2024-06-10T01:00:43.942 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:43 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T01:00:43.942 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:43 smithi175 ceph-mon[36636]: from='osd.11 [v2:172.21.15.175:6824/1959979550,v1:172.21.15.175:6825/1959979550]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-10T01:00:43.942 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:43 smithi175 ceph-mon[36636]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-10T01:00:43.943 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:43 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:00:44.061 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:00:44.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:43 smithi007 ceph-mon[32398]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-10T01:00:44.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:43 smithi007 ceph-mon[32398]: osdmap e78: 12 total, 11 up, 12 in 2024-06-10T01:00:44.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:43 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T01:00:44.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:43 smithi007 ceph-mon[32398]: from='osd.11 [v2:172.21.15.175:6824/1959979550,v1:172.21.15.175:6825/1959979550]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-10T01:00:44.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:43 smithi007 ceph-mon[32398]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-10T01:00:44.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:43 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:00:44.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:43 smithi078 ceph-mon[36683]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-10T01:00:44.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:43 smithi078 ceph-mon[36683]: osdmap e78: 12 total, 11 up, 12 in 2024-06-10T01:00:44.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:43 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T01:00:44.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:43 smithi078 ceph-mon[36683]: from='osd.11 [v2:172.21.15.175:6824/1959979550,v1:172.21.15.175:6825/1959979550]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-10T01:00:44.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:43 smithi078 ceph-mon[36683]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-10T01:00:44.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:43 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:00:44.457 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:00:44 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:00:44.378+0000 7facf9cfa700 -1 osd.11 0 waiting for initial osdmap 2024-06-10T01:00:44.457 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:00:44 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:00:44.391+0000 7facf1ae5700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T01:00:44.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:44 smithi175 ceph-mon[36636]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-06-10T01:00:44.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:44 smithi175 ceph-mon[36636]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-06-10T01:00:44.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:44 smithi175 ceph-mon[36636]: osdmap e79: 12 total, 11 up, 12 in 2024-06-10T01:00:44.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:44 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T01:00:44.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:44 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T01:00:44.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:44 smithi175 ceph-mon[36636]: from='osd.11 ' entity='osd.11' 2024-06-10T01:00:45.199 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:44 smithi007 ceph-mon[32398]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-06-10T01:00:45.199 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:44 smithi007 ceph-mon[32398]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-06-10T01:00:45.199 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:44 smithi007 ceph-mon[32398]: osdmap e79: 12 total, 11 up, 12 in 2024-06-10T01:00:45.199 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:44 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T01:00:45.200 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:44 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T01:00:45.200 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:44 smithi007 ceph-mon[32398]: from='osd.11 ' entity='osd.11' 2024-06-10T01:00:45.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:44 smithi078 ceph-mon[36683]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-06-10T01:00:45.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:44 smithi078 ceph-mon[36683]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-06-10T01:00:45.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:44 smithi078 ceph-mon[36683]: osdmap e79: 12 total, 11 up, 12 in 2024-06-10T01:00:45.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:44 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T01:00:45.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:44 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T01:00:45.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:44 smithi078 ceph-mon[36683]: from='osd.11 ' entity='osd.11' 2024-06-10T01:00:45.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.870+0000 7f7b99b0d700 1 -- 172.21.15.7:0/1629780664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b941016a0 msgr2=0x7f7b94101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:45.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.870+0000 7f7b99b0d700 1 --2- 172.21.15.7:0/1629780664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b941016a0 0x7f7b94101b20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7b88009990 tx=0x7f7b88009ca0 comp rx=0 tx=0).stop 2024-06-10T01:00:45.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.870+0000 7f7b99b0d700 1 -- 172.21.15.7:0/1629780664 shutdown_connections 2024-06-10T01:00:45.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.870+0000 7f7b99b0d700 1 --2- 172.21.15.7:0/1629780664 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b94102060 0x7f7b941024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:45.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.870+0000 7f7b99b0d700 1 --2- 172.21.15.7:0/1629780664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b94100d40 0x7f7b94101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:45.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.870+0000 7f7b99b0d700 1 --2- 172.21.15.7:0/1629780664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b941016a0 0x7f7b94101b20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:45.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.870+0000 7f7b99b0d700 1 -- 172.21.15.7:0/1629780664 >> 172.21.15.7:0/1629780664 conn(0x7f7b940fc960 msgr2=0x7f7b940feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:45.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.870+0000 7f7b99b0d700 1 -- 172.21.15.7:0/1629780664 shutdown_connections 2024-06-10T01:00:45.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.870+0000 7f7b99b0d700 1 -- 172.21.15.7:0/1629780664 wait complete. 2024-06-10T01:00:45.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.870+0000 7f7b99b0d700 1 Processor -- start 2024-06-10T01:00:45.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.871+0000 7f7b99b0d700 1 -- start start 2024-06-10T01:00:45.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.871+0000 7f7b99b0d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b94100d40 0x7f7b9419a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:45.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.871+0000 7f7b99b0d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b941016a0 0x7f7b9419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:45.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.871+0000 7f7b99b0d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b94102060 0x7f7b941a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:45.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.871+0000 7f7b99b0d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7b94110270 con 0x7f7b941016a0 2024-06-10T01:00:45.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.871+0000 7f7b99b0d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7b941100f0 con 0x7f7b94100d40 2024-06-10T01:00:45.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.871+0000 7f7b99b0d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7b941103f0 con 0x7f7b94102060 2024-06-10T01:00:45.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.871+0000 7f7b92ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b941016a0 0x7f7b9419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:45.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.871+0000 7f7b92ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b941016a0 0x7f7b9419adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51682/0 (socket says 172.21.15.7:51682) 2024-06-10T01:00:45.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.871+0000 7f7b92ffd700 1 -- 172.21.15.7:0/2470638452 learned_addr learned my addr 172.21.15.7:0/2470638452 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:00:45.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.871+0000 7f7b93fff700 1 --2- 172.21.15.7:0/2470638452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b94102060 0x7f7b941a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:45.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.871+0000 7f7b937fe700 1 --2- 172.21.15.7:0/2470638452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b94100d40 0x7f7b9419a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:45.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.871+0000 7f7b92ffd700 1 -- 172.21.15.7:0/2470638452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b94102060 msgr2=0x7f7b941a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:45.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.871+0000 7f7b92ffd700 1 --2- 172.21.15.7:0/2470638452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b94102060 0x7f7b941a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:45.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.871+0000 7f7b92ffd700 1 -- 172.21.15.7:0/2470638452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b94100d40 msgr2=0x7f7b9419a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:45.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.871+0000 7f7b92ffd700 1 --2- 172.21.15.7:0/2470638452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b94100d40 0x7f7b9419a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:45.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.871+0000 7f7b92ffd700 1 -- 172.21.15.7:0/2470638452 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b941a2510 con 0x7f7b941016a0 2024-06-10T01:00:45.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.871+0000 7f7b92ffd700 1 --2- 172.21.15.7:0/2470638452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b941016a0 0x7f7b9419adf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7b88004790 tx=0x7f7b88009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:45.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.872+0000 7f7b90ff9700 1 -- 172.21.15.7:0/2470638452 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88027070 con 0x7f7b941016a0 2024-06-10T01:00:45.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.872+0000 7f7b90ff9700 1 -- 172.21.15.7:0/2470638452 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b88004150 con 0x7f7b941016a0 2024-06-10T01:00:45.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.872+0000 7f7b99b0d700 1 -- 172.21.15.7:0/2470638452 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b941a27a0 con 0x7f7b941016a0 2024-06-10T01:00:45.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.872+0000 7f7b90ff9700 1 -- 172.21.15.7:0/2470638452 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88022e00 con 0x7f7b941016a0 2024-06-10T01:00:45.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.872+0000 7f7b99b0d700 1 -- 172.21.15.7:0/2470638452 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b941a2bf0 con 0x7f7b941016a0 2024-06-10T01:00:45.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.873+0000 7f7b99b0d700 1 -- 172.21.15.7:0/2470638452 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b94069180 con 0x7f7b941016a0 2024-06-10T01:00:45.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.874+0000 7f7b90ff9700 1 -- 172.21.15.7:0/2470638452 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f7b8802a070 con 0x7f7b941016a0 2024-06-10T01:00:45.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.875+0000 7f7b90ff9700 1 --2- 172.21.15.7:0/2470638452 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7b7c0a2c20 0x7f7b7c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:45.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.875+0000 7f7b937fe700 1 --2- 172.21.15.7:0/2470638452 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7b7c0a2c20 0x7f7b7c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:45.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.875+0000 7f7b90ff9700 1 -- 172.21.15.7:0/2470638452 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f7b880cfd90 con 0x7f7b941016a0 2024-06-10T01:00:45.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.875+0000 7f7b937fe700 1 --2- 172.21.15.7:0/2470638452 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7b7c0a2c20 0x7f7b7c0a50e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7b84005950 tx=0x7f7b8400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:45.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:45.878+0000 7f7b90ff9700 1 -- 172.21.15.7:0/2470638452 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7b88031020 con 0x7f7b941016a0 2024-06-10T01:00:46.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:45 smithi007 ceph-mon[32398]: purged_snaps scrub starts 2024-06-10T01:00:46.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:45 smithi007 ceph-mon[32398]: purged_snaps scrub ok 2024-06-10T01:00:46.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:45 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T01:00:46.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:45 smithi007 ceph-mon[32398]: osd.11 [v2:172.21.15.175:6824/1959979550,v1:172.21.15.175:6825/1959979550] boot 2024-06-10T01:00:46.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:45 smithi007 ceph-mon[32398]: osdmap e80: 12 total, 12 up, 12 in 2024-06-10T01:00:46.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:45 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T01:00:46.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:46.091+0000 7f7b99b0d700 1 -- 172.21.15.7:0/2470638452 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f7b9402dca0 con 0x7f7b941016a0 2024-06-10T01:00:46.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:46.091+0000 7f7b90ff9700 1 -- 172.21.15.7:0/2470638452 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7f7b88013070 con 0x7f7b941016a0 2024-06-10T01:00:46.090 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:00:46.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:46.094+0000 7f7b99b0d700 1 -- 172.21.15.7:0/2470638452 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7b7c0a2c20 msgr2=0x7f7b7c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:46.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:46.094+0000 7f7b99b0d700 1 --2- 172.21.15.7:0/2470638452 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7b7c0a2c20 0x7f7b7c0a50e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7b84005950 tx=0x7f7b8400b410 comp rx=0 tx=0).stop 2024-06-10T01:00:46.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:46.094+0000 7f7b99b0d700 1 -- 172.21.15.7:0/2470638452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b941016a0 msgr2=0x7f7b9419adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:46.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:46.094+0000 7f7b99b0d700 1 --2- 172.21.15.7:0/2470638452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b941016a0 0x7f7b9419adf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7b88004790 tx=0x7f7b88009580 comp rx=0 tx=0).stop 2024-06-10T01:00:46.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:46.094+0000 7f7b99b0d700 1 -- 172.21.15.7:0/2470638452 shutdown_connections 2024-06-10T01:00:46.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:46.094+0000 7f7b99b0d700 1 --2- 172.21.15.7:0/2470638452 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7b7c0a2c20 0x7f7b7c0a50e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:46.094+0000 7f7b99b0d700 1 --2- 172.21.15.7:0/2470638452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b94102060 0x7f7b941a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:46.094+0000 7f7b99b0d700 1 --2- 172.21.15.7:0/2470638452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b94100d40 0x7f7b9419a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:46.094+0000 7f7b99b0d700 1 --2- 172.21.15.7:0/2470638452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b941016a0 0x7f7b9419adf0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:46.094+0000 7f7b99b0d700 1 -- 172.21.15.7:0/2470638452 >> 172.21.15.7:0/2470638452 conn(0x7f7b940fc960 msgr2=0x7f7b940fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:46.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:46.094+0000 7f7b99b0d700 1 -- 172.21.15.7:0/2470638452 shutdown_connections 2024-06-10T01:00:46.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:46.094+0000 7f7b99b0d700 1 -- 172.21.15.7:0/2470638452 wait complete. 2024-06-10T01:00:46.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:45 smithi175 ceph-mon[36636]: purged_snaps scrub starts 2024-06-10T01:00:46.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:45 smithi175 ceph-mon[36636]: purged_snaps scrub ok 2024-06-10T01:00:46.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:45 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T01:00:46.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:45 smithi175 ceph-mon[36636]: osd.11 [v2:172.21.15.175:6824/1959979550,v1:172.21.15.175:6825/1959979550] boot 2024-06-10T01:00:46.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:45 smithi175 ceph-mon[36636]: osdmap e80: 12 total, 12 up, 12 in 2024-06-10T01:00:46.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:45 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T01:00:46.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:45 smithi078 ceph-mon[36683]: purged_snaps scrub starts 2024-06-10T01:00:46.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:45 smithi078 ceph-mon[36683]: purged_snaps scrub ok 2024-06-10T01:00:46.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:45 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T01:00:46.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:45 smithi078 ceph-mon[36683]: osd.11 [v2:172.21.15.175:6824/1959979550,v1:172.21.15.175:6825/1959979550] boot 2024-06-10T01:00:46.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:45 smithi078 ceph-mon[36683]: osdmap e80: 12 total, 12 up, 12 in 2024-06-10T01:00:46.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:45 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T01:00:46.835 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1717981245,"num_in_osds":12,"osd_in_since":1717981225,"num_remapped_pgs":0} 2024-06-10T01:00:46.836 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T01:00:46.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:46 smithi175 ceph-mon[36636]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-06-10T01:00:46.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:46 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2470638452' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-10T01:00:46.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:46 smithi175 ceph-mon[36636]: osdmap e81: 12 total, 12 up, 12 in 2024-06-10T01:00:46.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:46 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:46.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:46 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:46.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:46 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:46.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:46 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:46.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:46 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:46.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:46 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:46.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:46 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:46.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:46 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:46.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:46 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:00:46.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:46 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:47.088 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:00:47.219 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:46 smithi007 ceph-mon[32398]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-06-10T01:00:47.219 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:46 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2470638452' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-10T01:00:47.219 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:46 smithi007 ceph-mon[32398]: osdmap e81: 12 total, 12 up, 12 in 2024-06-10T01:00:47.219 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:46 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:47.219 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:46 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:47.220 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:46 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:47.220 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:46 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:47.220 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:46 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:47.220 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:46 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:47.220 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:46 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:47.220 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:46 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:47.221 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:46 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:00:47.221 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:46 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:47.235 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:46 smithi078 ceph-mon[36683]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-06-10T01:00:47.235 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:46 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2470638452' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-10T01:00:47.235 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:46 smithi078 ceph-mon[36683]: osdmap e81: 12 total, 12 up, 12 in 2024-06-10T01:00:47.235 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:46 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:47.235 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:46 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:47.235 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:46 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:47.236 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:46 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:47.236 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:46 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:47.236 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:46 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:47.236 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:46 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:47.236 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:46 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:47.236 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:46 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:00:47.236 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:46 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:48.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:47 smithi175 ceph-mon[36636]: Detected new or changed devices on smithi175 2024-06-10T01:00:48.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:47 smithi175 ceph-mon[36636]: Adjusting osd_memory_target on smithi175 to 4272M 2024-06-10T01:00:48.251 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:47 smithi078 ceph-mon[36683]: Detected new or changed devices on smithi175 2024-06-10T01:00:48.251 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:47 smithi078 ceph-mon[36683]: Adjusting osd_memory_target on smithi175 to 4272M 2024-06-10T01:00:48.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:47 smithi007 ceph-mon[32398]: Detected new or changed devices on smithi175 2024-06-10T01:00:48.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:47 smithi007 ceph-mon[32398]: Adjusting osd_memory_target on smithi175 to 4272M 2024-06-10T01:00:48.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.813+0000 7f2b816dc700 1 -- 172.21.15.7:0/1733678661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c1030f0 msgr2=0x7f2b7c107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:48.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.813+0000 7f2b816dc700 1 --2- 172.21.15.7:0/1733678661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c1030f0 0x7f2b7c107110 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2b700099c0 tx=0x7f2b70009cd0 comp rx=0 tx=0).stop 2024-06-10T01:00:48.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.813+0000 7f2b816dc700 1 -- 172.21.15.7:0/1733678661 shutdown_connections 2024-06-10T01:00:48.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.813+0000 7f2b816dc700 1 --2- 172.21.15.7:0/1733678661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b7c107840 0x7f2b7c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:48.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.813+0000 7f2b816dc700 1 --2- 172.21.15.7:0/1733678661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b7c102740 0x7f2b7c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:48.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.813+0000 7f2b816dc700 1 --2- 172.21.15.7:0/1733678661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c1030f0 0x7f2b7c107110 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:48.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.813+0000 7f2b816dc700 1 -- 172.21.15.7:0/1733678661 >> 172.21.15.7:0/1733678661 conn(0x7f2b7c0795f0 msgr2=0x7f2b7c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:48.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.813+0000 7f2b816dc700 1 -- 172.21.15.7:0/1733678661 shutdown_connections 2024-06-10T01:00:48.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.813+0000 7f2b816dc700 1 -- 172.21.15.7:0/1733678661 wait complete. 2024-06-10T01:00:48.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.814+0000 7f2b816dc700 1 Processor -- start 2024-06-10T01:00:48.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.814+0000 7f2b816dc700 1 -- start start 2024-06-10T01:00:48.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.814+0000 7f2b816dc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b7c102740 0x7f2b7c19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:48.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.814+0000 7f2b816dc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b7c1030f0 0x7f2b7c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:48.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.814+0000 7f2b816dc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c107840 0x7f2b7c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:48.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.814+0000 7f2b816dc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c112750 con 0x7f2b7c107840 2024-06-10T01:00:48.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.814+0000 7f2b816dc700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c1125d0 con 0x7f2b7c1030f0 2024-06-10T01:00:48.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.814+0000 7f2b816dc700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c1128d0 con 0x7f2b7c102740 2024-06-10T01:00:48.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.814+0000 7f2b7b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c107840 0x7f2b7c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:48.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.814+0000 7f2b7b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c107840 0x7f2b7c1989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51698/0 (socket says 172.21.15.7:51698) 2024-06-10T01:00:48.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.814+0000 7f2b7b7fe700 1 -- 172.21.15.7:0/1918731456 learned_addr learned my addr 172.21.15.7:0/1918731456 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:00:48.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.814+0000 7f2b7a7fc700 1 --2- 172.21.15.7:0/1918731456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b7c1030f0 0x7f2b7c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:48.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.815+0000 7f2b7affd700 1 --2- 172.21.15.7:0/1918731456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b7c102740 0x7f2b7c19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:48.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.815+0000 7f2b7b7fe700 1 -- 172.21.15.7:0/1918731456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b7c102740 msgr2=0x7f2b7c19e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:48.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.815+0000 7f2b7b7fe700 1 --2- 172.21.15.7:0/1918731456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b7c102740 0x7f2b7c19e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:48.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.815+0000 7f2b7b7fe700 1 -- 172.21.15.7:0/1918731456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b7c1030f0 msgr2=0x7f2b7c19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:48.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.815+0000 7f2b7b7fe700 1 --2- 172.21.15.7:0/1918731456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b7c1030f0 0x7f2b7c19ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:48.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.815+0000 7f2b7b7fe700 1 -- 172.21.15.7:0/1918731456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7c199130 con 0x7f2b7c107840 2024-06-10T01:00:48.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.815+0000 7f2b7b7fe700 1 --2- 172.21.15.7:0/1918731456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c107840 0x7f2b7c1989d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2b7400ef30 tx=0x7f2b7400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:48.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.815+0000 7f2b63fff700 1 -- 172.21.15.7:0/1918731456 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b740092e0 con 0x7f2b7c107840 2024-06-10T01:00:48.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.815+0000 7f2b816dc700 1 -- 172.21.15.7:0/1918731456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b7c1993c0 con 0x7f2b7c107840 2024-06-10T01:00:48.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.815+0000 7f2b63fff700 1 -- 172.21.15.7:0/1918731456 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b74004bb0 con 0x7f2b7c107840 2024-06-10T01:00:48.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.815+0000 7f2b63fff700 1 -- 172.21.15.7:0/1918731456 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b74004280 con 0x7f2b7c107840 2024-06-10T01:00:48.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.815+0000 7f2b816dc700 1 -- 172.21.15.7:0/1918731456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b7c1a56e0 con 0x7f2b7c107840 2024-06-10T01:00:48.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.816+0000 7f2b816dc700 1 -- 172.21.15.7:0/1918731456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b7c069180 con 0x7f2b7c107840 2024-06-10T01:00:48.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.817+0000 7f2b63fff700 1 -- 172.21.15.7:0/1918731456 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f2b7401b030 con 0x7f2b7c107840 2024-06-10T01:00:48.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.818+0000 7f2b63fff700 1 --2- 172.21.15.7:0/1918731456 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2b640a2ca0 0x7f2b640a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:48.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.818+0000 7f2b7affd700 1 --2- 172.21.15.7:0/1918731456 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2b640a2ca0 0x7f2b640a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:48.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.818+0000 7f2b63fff700 1 -- 172.21.15.7:0/1918731456 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f2b740ce760 con 0x7f2b7c107840 2024-06-10T01:00:48.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.818+0000 7f2b7affd700 1 --2- 172.21.15.7:0/1918731456 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2b640a2ca0 0x7f2b640a5160 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c005950 tx=0x7f2b6c005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:48.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:48.822+0000 7f2b63fff700 1 -- 172.21.15.7:0/1918731456 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2b74022020 con 0x7f2b7c107840 2024-06-10T01:00:49.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:49.028+0000 7f2b816dc700 1 -- 172.21.15.7:0/1918731456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b7c02dca0 con 0x7f2b7c107840 2024-06-10T01:00:49.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:49.029+0000 7f2b63fff700 1 -- 172.21.15.7:0/1918731456 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18350 (secure 0 0 0) 0x7f2b74010040 con 0x7f2b7c107840 2024-06-10T01:00:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:00:49.029 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":81,"fsid":"bded58f8-26c3-11ef-bc9b-c7b262605968","created":"2024-06-10T00:53:31.966085+0000","modified":"2024-06-10T01:00:46.384107+0000","last_up_change":"2024-06-10T01:00:45.382473+0000","last_in_change":"2024-06-10T01:00:25.608513+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:56:59.964055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f7c24c9f-b7ab-4c04-b2b4-120df9751667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6803","nonce":404485726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6805","nonce":404485726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6809","nonce":404485726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6807","nonce":404485726}]},"public_addr":"172.21.15.7:6803/404485726","cluster_addr":"172.21.15.7:6805/404485726","heartbeat_back_addr":"172.21.15.7:6809/404485726","heartbeat_front_addr":"172.21.15.7:6807/404485726","state":["exists","up"]},{"osd":1,"uuid":"9bf70a6b-f31c-4077-b5af-229f17bc2845","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6801","nonce":1856804448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6803","nonce":1856804448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6807","nonce":1856804448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6805","nonce":1856804448}]},"public_addr":"172.21.15.78:6801/1856804448","cluster_addr":"172.21.15.78:6803/1856804448","heartbeat_back_addr":"172.21.15.78:6807/1856804448","heartbeat_front_addr":"172.21.15.78:6805/1856804448","state":["exists","up"]},{"osd":2,"uuid":"62923c97-caf9-4f7e-9c14-9c9918843c78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6801","nonce":918257455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6803","nonce":918257455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6807","nonce":918257455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6805","nonce":918257455}]},"public_addr":"172.21.15.175:6801/918257455","cluster_addr":"172.21.15.175:6803/918257455","heartbeat_back_addr":"172.21.15.175:6807/918257455","heartbeat_front_addr":"172.21.15.175:6805/918257455","state":["exists","up"]},{"osd":3,"uuid":"65242098-e748-48ab-8f0e-bb25b5c62742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6811","nonce":1797404966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6813","nonce":1797404966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6817","nonce":1797404966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6815","nonce":1797404966}]},"public_addr":"172.21.15.7:6811/1797404966","cluster_addr":"172.21.15.7:6813/1797404966","heartbeat_back_addr":"172.21.15.7:6817/1797404966","heartbeat_front_addr":"172.21.15.7:6815/1797404966","state":["exists","up"]},{"osd":4,"uuid":"a7990305-de9e-400e-a432-d7854f73f339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6809","nonce":3767608357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6811","nonce":3767608357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6815","nonce":3767608357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6813","nonce":3767608357}]},"public_addr":"172.21.15.78:6809/3767608357","cluster_addr":"172.21.15.78:6811/3767608357","heartbeat_back_addr":"172.21.15.78:6815/3767608357","heartbeat_front_addr":"172.21.15.78:6813/3767608357","state":["exists","up"]},{"osd":5,"uuid":"bad9a7b3-da5a-48d0-905d-e6a3190317fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6809","nonce":2157664731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6811","nonce":2157664731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6815","nonce":2157664731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6813","nonce":2157664731}]},"public_addr":"172.21.15.175:6809/2157664731","cluster_addr":"172.21.15.175:6811/2157664731","heartbeat_back_addr":"172.21.15.175:6815/2157664731","heartbeat_front_addr":"172.21.15.175:6813/2157664731","state":["exists","up"]},{"osd":6,"uuid":"6c687653-157b-4b96-867b-a2eaa7d8ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6819","nonce":838731756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6821","nonce":838731756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6825","nonce":838731756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6823","nonce":838731756}]},"public_addr":"172.21.15.7:6819/838731756","cluster_addr":"172.21.15.7:6821/838731756","heartbeat_back_addr":"172.21.15.7:6825/838731756","heartbeat_front_addr":"172.21.15.7:6823/838731756","state":["exists","up"]},{"osd":7,"uuid":"db40ddf1-9fd9-4859-b8c4-75f79582ab91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6817","nonce":3511921773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6819","nonce":3511921773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6823","nonce":3511921773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6821","nonce":3511921773}]},"public_addr":"172.21.15.78:6817/3511921773","cluster_addr":"172.21.15.78:6819/3511921773","heartbeat_back_addr":"172.21.15.78:6823/3511921773","heartbeat_front_addr":"172.21.15.78:6821/3511921773","state":["exists","up"]},{"osd":8,"uuid":"0f01cbd9-b193-4c70-a33c-4984167bf422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6817","nonce":1546325211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6819","nonce":1546325211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6823","nonce":1546325211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6821","nonce":1546325211}]},"public_addr":"172.21.15.175:6817/1546325211","cluster_addr":"172.21.15.175:6819/1546325211","heartbeat_back_addr":"172.21.15.175:6823/1546325211","heartbeat_front_addr":"172.21.15.175:6821/1546325211","state":["exists","up"]},{"osd":9,"uuid":"e5480998-f7fb-49cd-9aed-52405c88d830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6827","nonce":1844550813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6829","nonce":1844550813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6833","nonce":1844550813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6831","nonce":1844550813}]},"public_addr":"172.21.15.7:6827/1844550813","cluster_addr":"172.21.15.7:6829/1844550813","heartbeat_back_addr":"172.21.15.7:6833/1844550813","heartbeat_front_addr":"172.21.15.7:6831/1844550813","state":["exists","up"]},{"osd":10,"uuid":"caa82f33-a71c-4907-bf3b-30a7fdfeb71d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6825","nonce":2290824785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6827","nonce":2290824785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6831","nonce":2290824785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6829","nonce":2290824785}]},"public_addr":"172.21.15.78:6825/2290824785","cluster_addr":"172.21.15.78:6827/2290824785","heartbeat_back_addr":"172.21.15.78:6831/2290824785","heartbeat_front_addr":"172.21.15.78:6829/2290824785","state":["exists","up"]},{"osd":11,"uuid":"608cda77-77da-4da2-a7e2-76a773be27c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6825","nonce":1959979550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6827","nonce":1959979550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6831","nonce":1959979550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6829","nonce":1959979550}]},"public_addr":"172.21.15.175:6825/1959979550","cluster_addr":"172.21.15.175:6827/1959979550","heartbeat_back_addr":"172.21.15.175:6831/1959979550","heartbeat_front_addr":"172.21.15.175:6829/1959979550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:08.850203+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:32.820161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:56.633656+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:21.759372+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:46.735874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:10.636541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:36.990099+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:01.767098+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:26.882776+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:52.757595+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:18.383938+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:43.544330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1833397470":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/322543993":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/212510583":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6801/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6800/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/2679436843":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6801/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/1210355277":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6800/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/3610017406":"2024-06-11T00:54:08.102405+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T01:00:49.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:49.031+0000 7f2b816dc700 1 -- 172.21.15.7:0/1918731456 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2b640a2ca0 msgr2=0x7f2b640a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:49.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:49.031+0000 7f2b816dc700 1 --2- 172.21.15.7:0/1918731456 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2b640a2ca0 0x7f2b640a5160 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c005950 tx=0x7f2b6c005ed0 comp rx=0 tx=0).stop 2024-06-10T01:00:49.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:49.031+0000 7f2b816dc700 1 -- 172.21.15.7:0/1918731456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c107840 msgr2=0x7f2b7c1989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:49.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:49.031+0000 7f2b816dc700 1 --2- 172.21.15.7:0/1918731456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c107840 0x7f2b7c1989d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2b7400ef30 tx=0x7f2b7400c3a0 comp rx=0 tx=0).stop 2024-06-10T01:00:49.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:49.031+0000 7f2b816dc700 1 -- 172.21.15.7:0/1918731456 shutdown_connections 2024-06-10T01:00:49.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:49.031+0000 7f2b816dc700 1 --2- 172.21.15.7:0/1918731456 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2b640a2ca0 0x7f2b640a5160 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:49.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:49.031+0000 7f2b816dc700 1 --2- 172.21.15.7:0/1918731456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c107840 0x7f2b7c1989d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:49.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:49.031+0000 7f2b816dc700 1 --2- 172.21.15.7:0/1918731456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b7c102740 0x7f2b7c19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:49.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:49.031+0000 7f2b816dc700 1 --2- 172.21.15.7:0/1918731456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b7c1030f0 0x7f2b7c19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:49.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:49.031+0000 7f2b816dc700 1 -- 172.21.15.7:0/1918731456 >> 172.21.15.7:0/1918731456 conn(0x7f2b7c0795f0 msgr2=0x7f2b7c100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:49.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:49.031+0000 7f2b816dc700 1 -- 172.21.15.7:0/1918731456 shutdown_connections 2024-06-10T01:00:49.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:49.031+0000 7f2b816dc700 1 -- 172.21.15.7:0/1918731456 wait complete. 2024-06-10T01:00:49.186 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:48 smithi175 ceph-mon[36636]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:00:49.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:48 smithi007 ceph-mon[32398]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:00:49.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:48 smithi078 ceph-mon[36683]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:00:49.694 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-10T00:56:59.964055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-06-10T01:00:49.695 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-06-10T01:00:49.948 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:00:50.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:49 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1918731456' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:00:50.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:49 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1918731456' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:00:50.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:49 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1918731456' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:00:51.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:50 smithi007 ceph-mon[32398]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:00:51.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:50 smithi175 ceph-mon[36636]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:00:51.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:50 smithi078 ceph-mon[36683]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:00:51.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.702+0000 7f1996439700 1 -- 172.21.15.7:0/3128905661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f199006a170 msgr2=0x7f199006a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:51.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.702+0000 7f1996439700 1 --2- 172.21.15.7:0/3128905661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f199006a170 0x7f199006a5f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1978009990 tx=0x7f1978009ca0 comp rx=0 tx=0).stop 2024-06-10T01:00:51.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.702+0000 7f1996439700 1 -- 172.21.15.7:0/3128905661 shutdown_connections 2024-06-10T01:00:51.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.702+0000 7f1996439700 1 --2- 172.21.15.7:0/3128905661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f199006ab30 0x7f199010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:51.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.702+0000 7f1996439700 1 --2- 172.21.15.7:0/3128905661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1990108500 0x7f19901088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:51.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.702+0000 7f1996439700 1 --2- 172.21.15.7:0/3128905661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f199006a170 0x7f199006a5f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:51.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.702+0000 7f1996439700 1 -- 172.21.15.7:0/3128905661 >> 172.21.15.7:0/3128905661 conn(0x7f1990078340 msgr2=0x7f1990078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:51.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.703+0000 7f1996439700 1 -- 172.21.15.7:0/3128905661 shutdown_connections 2024-06-10T01:00:51.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.703+0000 7f1996439700 1 -- 172.21.15.7:0/3128905661 wait complete. 2024-06-10T01:00:51.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.703+0000 7f1996439700 1 Processor -- start 2024-06-10T01:00:51.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.703+0000 7f1996439700 1 -- start start 2024-06-10T01:00:51.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.703+0000 7f1996439700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f199006a170 0x7f199019e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:51.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.704+0000 7f1996439700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f199006ab30 0x7f199019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:51.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.704+0000 7f1996439700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1990108500 0x7f1990198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:51.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.704+0000 7f1996439700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1990112780 con 0x7f1990108500 2024-06-10T01:00:51.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.704+0000 7f1996439700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1990112600 con 0x7f199006a170 2024-06-10T01:00:51.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.704+0000 7f1996439700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1990112900 con 0x7f199006ab30 2024-06-10T01:00:51.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.704+0000 7f19949d6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1990108500 0x7f1990198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:51.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.704+0000 7f19949d6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1990108500 0x7f1990198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51710/0 (socket says 172.21.15.7:51710) 2024-06-10T01:00:51.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.704+0000 7f19949d6700 1 -- 172.21.15.7:0/3883791084 learned_addr learned my addr 172.21.15.7:0/3883791084 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:00:51.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.704+0000 7f1987fff700 1 --2- 172.21.15.7:0/3883791084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f199006ab30 0x7f199019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:51.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.704+0000 7f198ffff700 1 --2- 172.21.15.7:0/3883791084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f199006a170 0x7f199019e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:51.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.704+0000 7f19949d6700 1 -- 172.21.15.7:0/3883791084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f199006ab30 msgr2=0x7f199019ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:51.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.704+0000 7f19949d6700 1 --2- 172.21.15.7:0/3883791084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f199006ab30 0x7f199019ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:51.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.704+0000 7f19949d6700 1 -- 172.21.15.7:0/3883791084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f199006a170 msgr2=0x7f199019e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:51.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.704+0000 7f19949d6700 1 --2- 172.21.15.7:0/3883791084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f199006a170 0x7f199019e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:51.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.704+0000 7f19949d6700 1 -- 172.21.15.7:0/3883791084 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1990199200 con 0x7f1990108500 2024-06-10T01:00:51.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.704+0000 7f198ffff700 1 --2- 172.21.15.7:0/3883791084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f199006a170 0x7f199019e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:00:51.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.704+0000 7f19949d6700 1 --2- 172.21.15.7:0/3883791084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1990108500 0x7f1990198aa0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1988007ee0 tx=0x7f198800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:51.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.704+0000 7f198dffb700 1 -- 172.21.15.7:0/3883791084 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198800dbd0 con 0x7f1990108500 2024-06-10T01:00:51.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.704+0000 7f1996439700 1 -- 172.21.15.7:0/3883791084 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19901994f0 con 0x7f1990108500 2024-06-10T01:00:51.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.705+0000 7f198dffb700 1 -- 172.21.15.7:0/3883791084 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f198800dd70 con 0x7f1990108500 2024-06-10T01:00:51.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.705+0000 7f198dffb700 1 -- 172.21.15.7:0/3883791084 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1988005a60 con 0x7f1990108500 2024-06-10T01:00:51.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.705+0000 7f1996439700 1 -- 172.21.15.7:0/3883791084 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19901a57b0 con 0x7f1990108500 2024-06-10T01:00:51.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.706+0000 7f1996439700 1 -- 172.21.15.7:0/3883791084 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1990069180 con 0x7f1990108500 2024-06-10T01:00:51.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.707+0000 7f198dffb700 1 -- 172.21.15.7:0/3883791084 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f198801a020 con 0x7f1990108500 2024-06-10T01:00:51.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.708+0000 7f198dffb700 1 --2- 172.21.15.7:0/3883791084 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f197c0a2ca0 0x7f197c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:51.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.708+0000 7f198ffff700 1 --2- 172.21.15.7:0/3883791084 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f197c0a2ca0 0x7f197c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:51.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.708+0000 7f198dffb700 1 -- 172.21.15.7:0/3883791084 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f19880ce2d0 con 0x7f1990108500 2024-06-10T01:00:51.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.708+0000 7f198ffff700 1 --2- 172.21.15.7:0/3883791084 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f197c0a2ca0 0x7f197c0a5160 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1980005e20 tx=0x7f1980005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:51.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.712+0000 7f198dffb700 1 -- 172.21.15.7:0/3883791084 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1988010070 con 0x7f1990108500 2024-06-10T01:00:51.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.917+0000 7f1996439700 1 -- 172.21.15.7:0/3883791084 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f199002dca0 con 0x7f1990108500 2024-06-10T01:00:51.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.917+0000 7f198dffb700 1 -- 172.21.15.7:0/3883791084 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f198809b630 con 0x7f1990108500 2024-06-10T01:00:51.917 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-06-10T01:00:51.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.920+0000 7f1996439700 1 -- 172.21.15.7:0/3883791084 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f197c0a2ca0 msgr2=0x7f197c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:51.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.920+0000 7f1996439700 1 --2- 172.21.15.7:0/3883791084 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f197c0a2ca0 0x7f197c0a5160 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1980005e20 tx=0x7f1980005db0 comp rx=0 tx=0).stop 2024-06-10T01:00:51.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.920+0000 7f1996439700 1 -- 172.21.15.7:0/3883791084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1990108500 msgr2=0x7f1990198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:51.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.920+0000 7f1996439700 1 --2- 172.21.15.7:0/3883791084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1990108500 0x7f1990198aa0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1988007ee0 tx=0x7f198800d3b0 comp rx=0 tx=0).stop 2024-06-10T01:00:51.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.920+0000 7f1996439700 1 -- 172.21.15.7:0/3883791084 shutdown_connections 2024-06-10T01:00:51.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.920+0000 7f1996439700 1 --2- 172.21.15.7:0/3883791084 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f197c0a2ca0 0x7f197c0a5160 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:51.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.920+0000 7f1996439700 1 --2- 172.21.15.7:0/3883791084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1990108500 0x7f1990198aa0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:51.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.920+0000 7f1996439700 1 --2- 172.21.15.7:0/3883791084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f199006a170 0x7f199019e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:51.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.920+0000 7f1996439700 1 --2- 172.21.15.7:0/3883791084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f199006ab30 0x7f199019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:51.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.920+0000 7f1996439700 1 -- 172.21.15.7:0/3883791084 >> 172.21.15.7:0/3883791084 conn(0x7f1990078340 msgr2=0x7f19901000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:51.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.920+0000 7f1996439700 1 -- 172.21.15.7:0/3883791084 shutdown_connections 2024-06-10T01:00:51.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:51.920+0000 7f1996439700 1 -- 172.21.15.7:0/3883791084 wait complete. 2024-06-10T01:00:52.517 INFO:tasks.cephadm:Adding mds.a on smithi007 2024-06-10T01:00:52.518 INFO:tasks.cephadm:Adding mds.d on smithi007 2024-06-10T01:00:52.518 INFO:tasks.cephadm:Adding mds.g on smithi007 2024-06-10T01:00:52.518 INFO:tasks.cephadm:Adding mds.j on smithi007 2024-06-10T01:00:52.518 INFO:tasks.cephadm:Adding mds.b on smithi078 2024-06-10T01:00:52.518 INFO:tasks.cephadm:Adding mds.e on smithi078 2024-06-10T01:00:52.518 INFO:tasks.cephadm:Adding mds.h on smithi078 2024-06-10T01:00:52.518 INFO:tasks.cephadm:Adding mds.k on smithi078 2024-06-10T01:00:52.518 INFO:tasks.cephadm:Adding mds.c on smithi175 2024-06-10T01:00:52.518 INFO:tasks.cephadm:Adding mds.f on smithi175 2024-06-10T01:00:52.519 INFO:tasks.cephadm:Adding mds.i on smithi175 2024-06-10T01:00:52.519 INFO:tasks.cephadm:Adding mds.l on smithi175 2024-06-10T01:00:52.519 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph orch apply mds all '12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi078=b;smithi078=e;smithi078=h;smithi078=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l' 2024-06-10T01:00:52.768 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T01:00:53.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:52 smithi175 ceph-mon[36636]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:00:53.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:52 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3883791084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T01:00:53.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:52 smithi007 ceph-mon[32398]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:00:53.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:52 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3883791084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T01:00:53.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:52 smithi078 ceph-mon[36683]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:00:53.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:52 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3883791084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T01:00:54.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.392+0000 7fd5ec1c4700 1 -- 172.21.15.175:0/3518405586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5e4102f80 msgr2=0x7fd5e4109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:54.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.392+0000 7fd5ec1c4700 1 --2- 172.21.15.175:0/3518405586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5e4102f80 0x7fd5e4109830 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc009990 tx=0x7fd5dc009ca0 comp rx=0 tx=0).stop 2024-06-10T01:00:54.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.392+0000 7fd5ec1c4700 1 -- 172.21.15.175:0/3518405586 shutdown_connections 2024-06-10T01:00:54.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.392+0000 7fd5ec1c4700 1 --2- 172.21.15.175:0/3518405586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5e4102f80 0x7fd5e4109830 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:54.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.392+0000 7fd5ec1c4700 1 --2- 172.21.15.175:0/3518405586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e41013e0 0x7fd5e4101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:54.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.392+0000 7fd5ec1c4700 1 --2- 172.21.15.175:0/3518405586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5e41025e0 0x7fd5e4102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:54.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.392+0000 7fd5ec1c4700 1 -- 172.21.15.175:0/3518405586 >> 172.21.15.175:0/3518405586 conn(0x7fd5e40fc980 msgr2=0x7fd5e40fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:54.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.392+0000 7fd5ec1c4700 1 -- 172.21.15.175:0/3518405586 shutdown_connections 2024-06-10T01:00:54.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.392+0000 7fd5ec1c4700 1 -- 172.21.15.175:0/3518405586 wait complete. 2024-06-10T01:00:54.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.393+0000 7fd5ec1c4700 1 Processor -- start 2024-06-10T01:00:54.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.393+0000 7fd5ec1c4700 1 -- start start 2024-06-10T01:00:54.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.393+0000 7fd5ec1c4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e41013e0 0x7fd5e406b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:54.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.393+0000 7fd5ec1c4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5e41025e0 0x7fd5e419d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:54.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.393+0000 7fd5ec1c4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5e4102f80 0x7fd5e419fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:54.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.393+0000 7fd5ec1c4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e410d0a0 con 0x7fd5e41013e0 2024-06-10T01:00:54.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.393+0000 7fd5ec1c4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e410cf20 con 0x7fd5e41025e0 2024-06-10T01:00:54.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.393+0000 7fd5ec1c4700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e410d220 con 0x7fd5e4102f80 2024-06-10T01:00:54.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.393+0000 7fd5ea761700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5e4102f80 0x7fd5e419fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:54.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.393+0000 7fd5ea761700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5e4102f80 0x7fd5e419fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:47120/0 (socket says 172.21.15.175:47120) 2024-06-10T01:00:54.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.393+0000 7fd5ea761700 1 -- 172.21.15.175:0/3006291144 learned_addr learned my addr 172.21.15.175:0/3006291144 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-06-10T01:00:54.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.393+0000 7fd5e975f700 1 --2- 172.21.15.175:0/3006291144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5e41025e0 0x7fd5e419d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:54.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.393+0000 7fd5ea761700 1 -- 172.21.15.175:0/3006291144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5e41025e0 msgr2=0x7fd5e419d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:54.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.393+0000 7fd5ea761700 1 --2- 172.21.15.175:0/3006291144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5e41025e0 0x7fd5e419d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:54.407 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.393+0000 7fd5e9f60700 1 --2- 172.21.15.175:0/3006291144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e41013e0 0x7fd5e406b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:54.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.393+0000 7fd5ea761700 1 -- 172.21.15.175:0/3006291144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e41013e0 msgr2=0x7fd5e406b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:54.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.393+0000 7fd5ea761700 1 --2- 172.21.15.175:0/3006291144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e41013e0 0x7fd5e406b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:54.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.393+0000 7fd5ea761700 1 -- 172.21.15.175:0/3006291144 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e41a0320 con 0x7fd5e4102f80 2024-06-10T01:00:54.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.393+0000 7fd5e9f60700 1 --2- 172.21.15.175:0/3006291144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e41013e0 0x7fd5e406b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:00:54.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.393+0000 7fd5ea761700 1 --2- 172.21.15.175:0/3006291144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5e4102f80 0x7fd5e419fa50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc003890 tx=0x7fd5dc009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:54.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.394+0000 7fd5daffd700 1 -- 172.21.15.175:0/3006291144 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc027070 con 0x7fd5e4102f80 2024-06-10T01:00:54.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.394+0000 7fd5daffd700 1 -- 172.21.15.175:0/3006291144 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5dc004150 con 0x7fd5e4102f80 2024-06-10T01:00:54.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.394+0000 7fd5ec1c4700 1 -- 172.21.15.175:0/3006291144 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e41a05b0 con 0x7fd5e4102f80 2024-06-10T01:00:54.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.394+0000 7fd5daffd700 1 -- 172.21.15.175:0/3006291144 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc00f460 con 0x7fd5e4102f80 2024-06-10T01:00:54.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.394+0000 7fd5ec1c4700 1 -- 172.21.15.175:0/3006291144 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e41a0a30 con 0x7fd5e4102f80 2024-06-10T01:00:54.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.395+0000 7fd5ec1c4700 1 -- 172.21.15.175:0/3006291144 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5e402dca0 con 0x7fd5e4102f80 2024-06-10T01:00:54.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.396+0000 7fd5daffd700 1 -- 172.21.15.175:0/3006291144 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fd5dc02a070 con 0x7fd5e4102f80 2024-06-10T01:00:54.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.396+0000 7fd5daffd700 1 --2- 172.21.15.175:0/3006291144 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd5d00a2c20 0x7fd5d00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:54.408 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.396+0000 7fd5daffd700 1 -- 172.21.15.175:0/3006291144 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fd5dc0cffb0 con 0x7fd5e4102f80 2024-06-10T01:00:54.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.397+0000 7fd5e9f60700 1 --2- 172.21.15.175:0/3006291144 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd5d00a2c20 0x7fd5d00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:54.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.397+0000 7fd5e9f60700 1 --2- 172.21.15.175:0/3006291144 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd5d00a2c20 0x7fd5d00a50e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4102440 tx=0x7fd5e00073c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:54.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.400+0000 7fd5daffd700 1 -- 172.21.15.175:0/3006291144 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd5dc013070 con 0x7fd5e4102f80 2024-06-10T01:00:54.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.604+0000 7fd5ec1c4700 1 -- 172.21.15.175:0/3006291144 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi078=b;smithi078=e;smithi078=h;smithi078=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l", "target": ["mon-mgr", ""]}) -- 0x7fd5e40629d0 con 0x7fd5d00a2c20 2024-06-10T01:00:54.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.613+0000 7fd5daffd700 1 -- 172.21.15.175:0/3006291144 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fd5e40629d0 con 0x7fd5d00a2c20 2024-06-10T01:00:54.611 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled mds.all update... 2024-06-10T01:00:54.613 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.615+0000 7fd5ec1c4700 1 -- 172.21.15.175:0/3006291144 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd5d00a2c20 msgr2=0x7fd5d00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:54.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.615+0000 7fd5ec1c4700 1 --2- 172.21.15.175:0/3006291144 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd5d00a2c20 0x7fd5d00a50e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4102440 tx=0x7fd5e00073c0 comp rx=0 tx=0).stop 2024-06-10T01:00:54.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.615+0000 7fd5ec1c4700 1 -- 172.21.15.175:0/3006291144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5e4102f80 msgr2=0x7fd5e419fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:54.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.615+0000 7fd5ec1c4700 1 --2- 172.21.15.175:0/3006291144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5e4102f80 0x7fd5e419fa50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc003890 tx=0x7fd5dc009ee0 comp rx=0 tx=0).stop 2024-06-10T01:00:54.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.615+0000 7fd5ec1c4700 1 -- 172.21.15.175:0/3006291144 shutdown_connections 2024-06-10T01:00:54.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.615+0000 7fd5ec1c4700 1 --2- 172.21.15.175:0/3006291144 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd5d00a2c20 0x7fd5d00a50e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:54.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.615+0000 7fd5ec1c4700 1 --2- 172.21.15.175:0/3006291144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5e4102f80 0x7fd5e419fa50 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:54.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.615+0000 7fd5ec1c4700 1 --2- 172.21.15.175:0/3006291144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e41013e0 0x7fd5e406b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:54.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.615+0000 7fd5ec1c4700 1 --2- 172.21.15.175:0/3006291144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5e41025e0 0x7fd5e419d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:54.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.615+0000 7fd5ec1c4700 1 -- 172.21.15.175:0/3006291144 >> 172.21.15.175:0/3006291144 conn(0x7fd5e40fc980 msgr2=0x7fd5e4105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:54.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.615+0000 7fd5ec1c4700 1 -- 172.21.15.175:0/3006291144 shutdown_connections 2024-06-10T01:00:54.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-10T01:00:54.615+0000 7fd5ec1c4700 1 -- 172.21.15.175:0/3006291144 wait complete. 2024-06-10T01:00:55.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:54 smithi175 ceph-mon[36636]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:00:55.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:54 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:55.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:54 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:00:55.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:54 smithi007 ceph-mon[32398]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:00:55.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:54 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:55.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:54 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:00:55.315 DEBUG:teuthology.orchestra.run.smithi007:mds.a> sudo journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.a.service 2024-06-10T01:00:55.317 DEBUG:teuthology.orchestra.run.smithi007:mds.d> sudo journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.d.service 2024-06-10T01:00:55.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:54 smithi078 ceph-mon[36683]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:00:55.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:54 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:55.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:54 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:00:55.360 DEBUG:teuthology.orchestra.run.smithi007:mds.g> sudo journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.g.service 2024-06-10T01:00:55.403 DEBUG:teuthology.orchestra.run.smithi007:mds.j> sudo journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.j.service 2024-06-10T01:00:55.446 DEBUG:teuthology.orchestra.run.smithi078:mds.b> sudo journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.b.service 2024-06-10T01:00:55.449 DEBUG:teuthology.orchestra.run.smithi078:mds.e> sudo journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.e.service 2024-06-10T01:00:55.491 DEBUG:teuthology.orchestra.run.smithi078:mds.h> sudo journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.h.service 2024-06-10T01:00:55.534 DEBUG:teuthology.orchestra.run.smithi078:mds.k> sudo journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.k.service 2024-06-10T01:00:55.577 DEBUG:teuthology.orchestra.run.smithi175:mds.c> sudo journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.c.service 2024-06-10T01:00:55.580 DEBUG:teuthology.orchestra.run.smithi175:mds.f> sudo journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.f.service 2024-06-10T01:00:55.623 DEBUG:teuthology.orchestra.run.smithi175:mds.i> sudo journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.i.service 2024-06-10T01:00:55.666 DEBUG:teuthology.orchestra.run.smithi175:mds.l> sudo journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.l.service 2024-06-10T01:00:55.710 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-06-10T01:00:55.711 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:00:55.962 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:00:56.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:55 smithi007 ceph-mon[32398]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi078=b;smithi078=e;smithi078=h;smithi078=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:00:56.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:55 smithi007 ceph-mon[32398]: Saving service mds.all spec with placement smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi078=b;smithi078=e;smithi078=h;smithi078=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l;count:12 2024-06-10T01:00:56.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:55 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:56.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:55 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:00:56.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:55 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:56.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:55 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:00:56.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:55 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:00:56.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:55 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:56.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:55 smithi007 ceph-mon[32398]: Deploying daemon mds.l on smithi175 2024-06-10T01:00:56.132 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:55 smithi175 ceph-mon[36636]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi078=b;smithi078=e;smithi078=h;smithi078=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:00:56.132 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:55 smithi175 ceph-mon[36636]: Saving service mds.all spec with placement smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi078=b;smithi078=e;smithi078=h;smithi078=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l;count:12 2024-06-10T01:00:56.132 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:55 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:56.132 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:55 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:00:56.132 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:55 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:56.132 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:55 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:00:56.133 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:55 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:00:56.133 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:55 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:56.133 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:55 smithi175 ceph-mon[36636]: Deploying daemon mds.l on smithi175 2024-06-10T01:00:56.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:55 smithi078 ceph-mon[36683]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi078=b;smithi078=e;smithi078=h;smithi078=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:00:56.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:55 smithi078 ceph-mon[36683]: Saving service mds.all spec with placement smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi078=b;smithi078=e;smithi078=h;smithi078=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l;count:12 2024-06-10T01:00:56.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:55 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:56.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:55 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:00:56.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:55 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:56.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:55 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:00:56.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:55 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:00:56.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:55 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:56.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:55 smithi078 ceph-mon[36683]: Deploying daemon mds.l on smithi175 2024-06-10T01:00:57.357 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:57 smithi007 ceph-mon[32398]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:00:57.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:57 smithi078 ceph-mon[36683]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:00:57.604 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:57 smithi175 ceph-mon[36636]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:00:57.604 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jun 10 01:00:57 smithi175 systemd[1]: Starting Ceph mds.l for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:00:57.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.726+0000 7fcf836a3700 1 -- 172.21.15.7:0/1806930791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf7c1066b0 msgr2=0x7fcf7c10e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:57.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.726+0000 7fcf836a3700 1 --2- 172.21.15.7:0/1806930791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf7c1066b0 0x7fcf7c10e280 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400b0d0 tx=0x7fcf7400b3e0 comp rx=0 tx=0).stop 2024-06-10T01:00:57.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.726+0000 7fcf836a3700 1 -- 172.21.15.7:0/1806930791 shutdown_connections 2024-06-10T01:00:57.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.726+0000 7fcf836a3700 1 --2- 172.21.15.7:0/1806930791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf7c1066b0 0x7fcf7c10e280 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:57.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.726+0000 7fcf836a3700 1 --2- 172.21.15.7:0/1806930791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf7c101620 0x7fcf7c101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:57.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.726+0000 7fcf836a3700 1 --2- 172.21.15.7:0/1806930791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf7c101fd0 0x7fcf7c105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:57.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.726+0000 7fcf836a3700 1 -- 172.21.15.7:0/1806930791 >> 172.21.15.7:0/1806930791 conn(0x7fcf7c078340 msgr2=0x7fcf7c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:57.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.726+0000 7fcf836a3700 1 -- 172.21.15.7:0/1806930791 shutdown_connections 2024-06-10T01:00:57.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.726+0000 7fcf836a3700 1 -- 172.21.15.7:0/1806930791 wait complete. 2024-06-10T01:00:57.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.726+0000 7fcf836a3700 1 Processor -- start 2024-06-10T01:00:57.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.726+0000 7fcf836a3700 1 -- start start 2024-06-10T01:00:57.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.727+0000 7fcf836a3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf7c101620 0x7fcf7c199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:57.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.727+0000 7fcf836a3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf7c101fd0 0x7fcf7c199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:57.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.727+0000 7fcf836a3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf7c1066b0 0x7fcf7c1a4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:57.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.727+0000 7fcf836a3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c111710 con 0x7fcf7c101620 2024-06-10T01:00:57.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.727+0000 7fcf836a3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c111590 con 0x7fcf7c1066b0 2024-06-10T01:00:57.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.727+0000 7fcf836a3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c111890 con 0x7fcf7c101fd0 2024-06-10T01:00:57.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.727+0000 7fcf8143f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf7c101620 0x7fcf7c199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:57.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.727+0000 7fcf8143f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf7c101620 0x7fcf7c199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59910/0 (socket says 172.21.15.7:59910) 2024-06-10T01:00:57.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.727+0000 7fcf8143f700 1 -- 172.21.15.7:0/4164414758 learned_addr learned my addr 172.21.15.7:0/4164414758 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:00:57.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.727+0000 7fcf80c3e700 1 --2- 172.21.15.7:0/4164414758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf7c101fd0 0x7fcf7c199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:57.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.727+0000 7fcf81c40700 1 --2- 172.21.15.7:0/4164414758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf7c1066b0 0x7fcf7c1a4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:57.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.727+0000 7fcf8143f700 1 -- 172.21.15.7:0/4164414758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf7c101fd0 msgr2=0x7fcf7c199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:57.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.727+0000 7fcf8143f700 1 --2- 172.21.15.7:0/4164414758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf7c101fd0 0x7fcf7c199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:57.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.727+0000 7fcf8143f700 1 -- 172.21.15.7:0/4164414758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf7c1066b0 msgr2=0x7fcf7c1a4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:57.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.727+0000 7fcf8143f700 1 --2- 172.21.15.7:0/4164414758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf7c1066b0 0x7fcf7c1a4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:57.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.727+0000 7fcf8143f700 1 -- 172.21.15.7:0/4164414758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7c1a5050 con 0x7fcf7c101620 2024-06-10T01:00:57.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.727+0000 7fcf81c40700 1 --2- 172.21.15.7:0/4164414758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf7c1066b0 0x7fcf7c1a4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:00:57.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.727+0000 7fcf8143f700 1 --2- 172.21.15.7:0/4164414758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf7c101620 0x7fcf7c199a90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcf7800b530 tx=0x7fcf7800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:57.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.728+0000 7fcf727fc700 1 -- 172.21.15.7:0/4164414758 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf78015020 con 0x7fcf7c101620 2024-06-10T01:00:57.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.728+0000 7fcf727fc700 1 -- 172.21.15.7:0/4164414758 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf78011d10 con 0x7fcf7c101620 2024-06-10T01:00:57.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.728+0000 7fcf836a3700 1 -- 172.21.15.7:0/4164414758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf7c1a52e0 con 0x7fcf7c101620 2024-06-10T01:00:57.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.728+0000 7fcf727fc700 1 -- 172.21.15.7:0/4164414758 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7800f2e0 con 0x7fcf7c101620 2024-06-10T01:00:57.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.728+0000 7fcf836a3700 1 -- 172.21.15.7:0/4164414758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf7c1a5840 con 0x7fcf7c101620 2024-06-10T01:00:57.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.729+0000 7fcf836a3700 1 -- 172.21.15.7:0/4164414758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf7c069180 con 0x7fcf7c101620 2024-06-10T01:00:57.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.730+0000 7fcf727fc700 1 -- 172.21.15.7:0/4164414758 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fcf78018020 con 0x7fcf7c101620 2024-06-10T01:00:57.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.731+0000 7fcf727fc700 1 --2- 172.21.15.7:0/4164414758 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcf680a2ca0 0x7fcf680a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:57.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.731+0000 7fcf80c3e700 1 --2- 172.21.15.7:0/4164414758 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcf680a2ca0 0x7fcf680a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:57.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.731+0000 7fcf727fc700 1 -- 172.21.15.7:0/4164414758 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fcf780ce410 con 0x7fcf7c101620 2024-06-10T01:00:57.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.731+0000 7fcf80c3e700 1 --2- 172.21.15.7:0/4164414758 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcf680a2ca0 0x7fcf680a5160 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c107af0 tx=0x7fcf6c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:57.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.736+0000 7fcf727fc700 1 -- 172.21.15.7:0/4164414758 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcf7801f020 con 0x7fcf7c101620 2024-06-10T01:00:57.918 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jun 10 01:00:57 smithi175 podman[65856]: 2024-06-10 01:00:57.508322158 +0000 UTC m=+0.017505135 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:00:57.918 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jun 10 01:00:57 smithi175 podman[65856]: 2024-06-10 01:00:57.608026633 +0000 UTC m=+0.117209610 container create 2b561a985a1f3854feaf3c8abacb66cf0ebdb0e0968e3b4c3bfb2fbcaf978cf4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-l, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T01:00:57.918 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jun 10 01:00:57 smithi175 podman[65856]: 2024-06-10 01:00:57.828296415 +0000 UTC m=+0.337479401 container init 2b561a985a1f3854feaf3c8abacb66cf0ebdb0e0968e3b4c3bfb2fbcaf978cf4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-l, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, ceph=True) 2024-06-10T01:00:57.918 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jun 10 01:00:57 smithi175 podman[65856]: 2024-06-10 01:00:57.831468298 +0000 UTC m=+0.340651283 container start 2b561a985a1f3854feaf3c8abacb66cf0ebdb0e0968e3b4c3bfb2fbcaf978cf4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-l, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T01:00:57.918 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jun 10 01:00:57 smithi175 ceph-mds[65875]: -- 172.21.15.175:0/1301271090 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5631dd861a40 con 0x5631dd834c00 2024-06-10T01:00:57.919 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jun 10 01:00:57 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-l[65870]: starting mds.l at 2024-06-10T01:00:57.919 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jun 10 01:00:57 smithi175 bash[65856]: 2b561a985a1f3854feaf3c8abacb66cf0ebdb0e0968e3b4c3bfb2fbcaf978cf4 2024-06-10T01:00:57.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.995+0000 7fcf836a3700 1 -- 172.21.15.7:0/4164414758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf7c02dca0 con 0x7fcf7c101620 2024-06-10T01:00:57.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.995+0000 7fcf727fc700 1 -- 172.21.15.7:0/4164414758 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fcf7809b770 con 0x7fcf7c101620 2024-06-10T01:00:57.995 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:00:57.995 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-06-10T01:00:57.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.998+0000 7fcf836a3700 1 -- 172.21.15.7:0/4164414758 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcf680a2ca0 msgr2=0x7fcf680a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:57.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.998+0000 7fcf836a3700 1 --2- 172.21.15.7:0/4164414758 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcf680a2ca0 0x7fcf680a5160 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c107af0 tx=0x7fcf6c00b410 comp rx=0 tx=0).stop 2024-06-10T01:00:57.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.998+0000 7fcf836a3700 1 -- 172.21.15.7:0/4164414758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf7c101620 msgr2=0x7fcf7c199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:57.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.998+0000 7fcf836a3700 1 --2- 172.21.15.7:0/4164414758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf7c101620 0x7fcf7c199a90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcf7800b530 tx=0x7fcf7800b840 comp rx=0 tx=0).stop 2024-06-10T01:00:57.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.998+0000 7fcf836a3700 1 -- 172.21.15.7:0/4164414758 shutdown_connections 2024-06-10T01:00:57.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.998+0000 7fcf836a3700 1 --2- 172.21.15.7:0/4164414758 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcf680a2ca0 0x7fcf680a5160 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:57.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.998+0000 7fcf836a3700 1 --2- 172.21.15.7:0/4164414758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf7c1066b0 0x7fcf7c1a4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:57.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.998+0000 7fcf836a3700 1 --2- 172.21.15.7:0/4164414758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf7c101620 0x7fcf7c199a90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:57.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.998+0000 7fcf836a3700 1 --2- 172.21.15.7:0/4164414758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf7c101fd0 0x7fcf7c199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:57.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.998+0000 7fcf836a3700 1 -- 172.21.15.7:0/4164414758 >> 172.21.15.7:0/4164414758 conn(0x7fcf7c078340 msgr2=0x7fcf7c0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:57.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.998+0000 7fcf836a3700 1 -- 172.21.15.7:0/4164414758 shutdown_connections 2024-06-10T01:00:57.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:00:57.998+0000 7fcf836a3700 1 -- 172.21.15.7:0/4164414758 wait complete. 2024-06-10T01:00:57.998 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 2 2024-06-10T01:00:58.206 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jun 10 01:00:57 smithi175 systemd[1]: Started Ceph mds.l for bded58f8-26c3-11ef-bc9b-c7b262605968. 2024-06-10T01:00:58.940 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:58 smithi078 ceph-mon[36683]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:00:58.941 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:58 smithi078 ceph-mon[36683]: mds.? [v2:172.21.15.175:6832/1281305765,v1:172.21.15.175:6833/1281305765] up:boot 2024-06-10T01:00:58.941 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:58 smithi078 ceph-mon[36683]: fsmap 1 up:standby 2024-06-10T01:00:58.941 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:58 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-10T01:00:58.941 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:58 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:58.941 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:58 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:58.941 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:58 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:58.941 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:58 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:00:58.942 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:58 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:00:58.942 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:58 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:58.942 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:58 smithi078 ceph-mon[36683]: Deploying daemon mds.k on smithi078 2024-06-10T01:00:58.942 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:00:58 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/4164414758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:00:59.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:58 smithi175 ceph-mon[36636]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:00:59.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:58 smithi175 ceph-mon[36636]: mds.? [v2:172.21.15.175:6832/1281305765,v1:172.21.15.175:6833/1281305765] up:boot 2024-06-10T01:00:59.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:58 smithi175 ceph-mon[36636]: fsmap 1 up:standby 2024-06-10T01:00:59.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:58 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-10T01:00:59.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:58 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:59.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:58 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:59.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:58 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:59.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:58 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:00:59.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:58 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:00:59.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:58 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:59.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:58 smithi175 ceph-mon[36636]: Deploying daemon mds.k on smithi078 2024-06-10T01:00:59.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:00:58 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/4164414758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:00:59.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:58 smithi007 ceph-mon[32398]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:00:59.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:58 smithi007 ceph-mon[32398]: mds.? [v2:172.21.15.175:6832/1281305765,v1:172.21.15.175:6833/1281305765] up:boot 2024-06-10T01:00:59.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:58 smithi007 ceph-mon[32398]: fsmap 1 up:standby 2024-06-10T01:00:59.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:58 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-10T01:00:59.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:58 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:59.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:58 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:59.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:58 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:00:59.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:58 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:00:59.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:58 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:00:59.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:58 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:59.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:58 smithi007 ceph-mon[32398]: Deploying daemon mds.k on smithi078 2024-06-10T01:00:59.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:00:58 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/4164414758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:01:00.334 INFO:journalctl@ceph.mds.k.smithi078.stdout:Jun 10 01:01:00 smithi078 systemd[1]: Starting Ceph mds.k for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:01:00.680 INFO:journalctl@ceph.mds.k.smithi078.stdout:Jun 10 01:01:00 smithi078 podman[65504]: 2024-06-10 01:01:00.299276408 +0000 UTC m=+0.017170633 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:01:00.680 INFO:journalctl@ceph.mds.k.smithi078.stdout:Jun 10 01:01:00 smithi078 podman[65504]: 2024-06-10 01:01:00.420725112 +0000 UTC m=+0.138619317 container create b77d167768fd4b0f5cc676d8dea7e9a0b3cf3aa131bc96bc0de236be363e2d52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-k, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-10T01:01:00.680 INFO:journalctl@ceph.mds.k.smithi078.stdout:Jun 10 01:01:00 smithi078 podman[65504]: 2024-06-10 01:01:00.681970154 +0000 UTC m=+0.399864361 container init b77d167768fd4b0f5cc676d8dea7e9a0b3cf3aa131bc96bc0de236be363e2d52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-k, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-06-10T01:01:00.844 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:01:01.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:00 smithi078 ceph-mon[36683]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:01.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:00 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:01.084 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:00 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:01.084 INFO:journalctl@ceph.mds.k.smithi078.stdout:Jun 10 01:01:00 smithi078 podman[65504]: 2024-06-10 01:01:00.684544352 +0000 UTC m=+0.402438577 container start b77d167768fd4b0f5cc676d8dea7e9a0b3cf3aa131bc96bc0de236be363e2d52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-k, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T01:01:01.084 INFO:journalctl@ceph.mds.k.smithi078.stdout:Jun 10 01:01:00 smithi078 ceph-mds[65523]: -- 172.21.15.78:0/3117273963 <== mon.2 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ec946f3a40 con 0x55ec946c7000 2024-06-10T01:01:01.084 INFO:journalctl@ceph.mds.k.smithi078.stdout:Jun 10 01:01:00 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-k[65519]: starting mds.k at 2024-06-10T01:01:01.084 INFO:journalctl@ceph.mds.k.smithi078.stdout:Jun 10 01:01:00 smithi078 bash[65504]: b77d167768fd4b0f5cc676d8dea7e9a0b3cf3aa131bc96bc0de236be363e2d52 2024-06-10T01:01:01.084 INFO:journalctl@ceph.mds.k.smithi078.stdout:Jun 10 01:01:00 smithi078 systemd[1]: Started Ceph mds.k for bded58f8-26c3-11ef-bc9b-c7b262605968. 2024-06-10T01:01:01.119 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:01:01.122 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:00 smithi007 ceph-mon[32398]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:01.122 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:00 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:01.122 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:00 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:01.206 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:00 smithi175 ceph-mon[36636]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:01.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:00 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:01.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:00 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:02.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:01 smithi007 ceph-mon[32398]: mds.? [v2:172.21.15.78:6832/1425832513,v1:172.21.15.78:6833/1425832513] up:boot 2024-06-10T01:01:02.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:01 smithi007 ceph-mon[32398]: fsmap 2 up:standby 2024-06-10T01:01:02.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:01 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-10T01:01:02.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:01 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:02.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:01 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:01:02.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:01 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:01:02.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:01 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:02.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:01 smithi007 ceph-mon[32398]: Deploying daemon mds.j on smithi007 2024-06-10T01:01:02.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:01 smithi175 ceph-mon[36636]: mds.? [v2:172.21.15.78:6832/1425832513,v1:172.21.15.78:6833/1425832513] up:boot 2024-06-10T01:01:02.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:01 smithi175 ceph-mon[36636]: fsmap 2 up:standby 2024-06-10T01:01:02.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:01 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-10T01:01:02.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:01 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:02.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:01 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:01:02.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:01 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:01:02.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:01 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:02.209 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:01 smithi175 ceph-mon[36636]: Deploying daemon mds.j on smithi007 2024-06-10T01:01:02.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:01 smithi078 ceph-mon[36683]: mds.? [v2:172.21.15.78:6832/1425832513,v1:172.21.15.78:6833/1425832513] up:boot 2024-06-10T01:01:02.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:01 smithi078 ceph-mon[36683]: fsmap 2 up:standby 2024-06-10T01:01:02.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:01 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-10T01:01:02.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:01 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:02.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:01 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:01:02.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:01 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:01:02.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:01 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:02.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:01 smithi078 ceph-mon[36683]: Deploying daemon mds.j on smithi007 2024-06-10T01:01:03.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:02 smithi078 ceph-mon[36683]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:03.206 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:02 smithi175 ceph-mon[36636]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:03.270 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:02 smithi007 ceph-mon[32398]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:04.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.207+0000 7f2368ef1700 1 -- 172.21.15.7:0/3032112969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2364101390 msgr2=0x7f2364101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:04.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.207+0000 7f2368ef1700 1 --2- 172.21.15.7:0/3032112969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2364101390 0x7f2364101810 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f234c009990 tx=0x7f234c009ca0 comp rx=0 tx=0).stop 2024-06-10T01:01:04.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.208+0000 7f2368ef1700 1 -- 172.21.15.7:0/3032112969 shutdown_connections 2024-06-10T01:01:04.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.208+0000 7f2368ef1700 1 --2- 172.21.15.7:0/3032112969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2364101d50 0x7f236410e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:04.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.208+0000 7f2368ef1700 1 --2- 172.21.15.7:0/3032112969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2364107400 0x7f23641077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:04.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.208+0000 7f2368ef1700 1 --2- 172.21.15.7:0/3032112969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2364101390 0x7f2364101810 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:04.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.208+0000 7f2368ef1700 1 -- 172.21.15.7:0/3032112969 >> 172.21.15.7:0/3032112969 conn(0x7f2364078340 msgr2=0x7f2364078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:04.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.208+0000 7f2368ef1700 1 -- 172.21.15.7:0/3032112969 shutdown_connections 2024-06-10T01:01:04.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.208+0000 7f2368ef1700 1 -- 172.21.15.7:0/3032112969 wait complete. 2024-06-10T01:01:04.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.208+0000 7f2368ef1700 1 Processor -- start 2024-06-10T01:01:04.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.208+0000 7f2368ef1700 1 -- start start 2024-06-10T01:01:04.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.209+0000 7f2368ef1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2364101390 0x7f236419ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:04.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.209+0000 7f2368ef1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2364101d50 0x7f236419b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:04.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.209+0000 7f2368ef1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2364107400 0x7f236419f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:04.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.209+0000 7f2368ef1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2364111670 con 0x7f2364101390 2024-06-10T01:01:04.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.209+0000 7f2368ef1700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f23641114f0 con 0x7f2364101d50 2024-06-10T01:01:04.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.209+0000 7f2368ef1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f23641117f0 con 0x7f2364107400 2024-06-10T01:01:04.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.209+0000 7f236259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2364101390 0x7f236419ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:04.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.209+0000 7f236259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2364101390 0x7f236419ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33658/0 (socket says 172.21.15.7:33658) 2024-06-10T01:01:04.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.209+0000 7f236259c700 1 -- 172.21.15.7:0/1205436804 learned_addr learned my addr 172.21.15.7:0/1205436804 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:01:04.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.209+0000 7f2361d9b700 1 --2- 172.21.15.7:0/1205436804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2364101d50 0x7f236419b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:04.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.209+0000 7f2362d9d700 1 --2- 172.21.15.7:0/1205436804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2364107400 0x7f236419f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:04.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.209+0000 7f236259c700 1 -- 172.21.15.7:0/1205436804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2364107400 msgr2=0x7f236419f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:04.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.209+0000 7f236259c700 1 --2- 172.21.15.7:0/1205436804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2364107400 0x7f236419f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:04.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.209+0000 7f236259c700 1 -- 172.21.15.7:0/1205436804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2364101d50 msgr2=0x7f236419b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:04.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.209+0000 7f236259c700 1 --2- 172.21.15.7:0/1205436804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2364101d50 0x7f236419b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:04.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.209+0000 7f236259c700 1 -- 172.21.15.7:0/1205436804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f236419fdb0 con 0x7f2364101390 2024-06-10T01:01:04.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.209+0000 7f236259c700 1 --2- 172.21.15.7:0/1205436804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2364101390 0x7f236419ada0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f235400c9b0 tx=0x7f235400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:04.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.209+0000 7f235b7fe700 1 -- 172.21.15.7:0/1205436804 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2354007b00 con 0x7f2364101390 2024-06-10T01:01:04.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.209+0000 7f235b7fe700 1 -- 172.21.15.7:0/1205436804 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2354007ca0 con 0x7f2364101390 2024-06-10T01:01:04.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.210+0000 7f2368ef1700 1 -- 172.21.15.7:0/1205436804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23641a00a0 con 0x7f2364101390 2024-06-10T01:01:04.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.210+0000 7f235b7fe700 1 -- 172.21.15.7:0/1205436804 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2354018780 con 0x7f2364101390 2024-06-10T01:01:04.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.210+0000 7f2368ef1700 1 -- 172.21.15.7:0/1205436804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23641a7920 con 0x7f2364101390 2024-06-10T01:01:04.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.210+0000 7f2368ef1700 1 -- 172.21.15.7:0/1205436804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2364069180 con 0x7f2364101390 2024-06-10T01:01:04.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.212+0000 7f235b7fe700 1 -- 172.21.15.7:0/1205436804 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f235400f450 con 0x7f2364101390 2024-06-10T01:01:04.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.212+0000 7f235b7fe700 1 --2- 172.21.15.7:0/1205436804 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f23500a2c20 0x7f23500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:04.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.213+0000 7f235b7fe700 1 -- 172.21.15.7:0/1205436804 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f23540cee80 con 0x7f2364101390 2024-06-10T01:01:04.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.213+0000 7f2361d9b700 1 --2- 172.21.15.7:0/1205436804 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f23500a2c20 0x7f23500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:04.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.213+0000 7f2361d9b700 1 --2- 172.21.15.7:0/1205436804 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f23500a2c20 0x7f23500a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f236406b1c0 tx=0x7f234c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:04.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.216+0000 7f235b7fe700 1 -- 172.21.15.7:0/1205436804 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2354014030 con 0x7f2364101390 2024-06-10T01:01:04.458 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jun 10 01:01:04 smithi007 podman[73265]: 2024-06-10 01:01:04.342281073 +0000 UTC m=+0.018306079 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:01:04.458 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jun 10 01:01:04 smithi007 podman[73265]: 2024-06-10 01:01:04.459937017 +0000 UTC m=+0.135962009 container create 7f40cbd8be744682b66fe5abfcb237bb6df2a59e7d9a0f5c5b3aec7e3f64c98b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-06-10T01:01:04.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.475+0000 7f2368ef1700 1 -- 172.21.15.7:0/1205436804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f236419c100 con 0x7f2364101390 2024-06-10T01:01:04.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.476+0000 7f235b7fe700 1 -- 172.21.15.7:0/1205436804 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f235409c310 con 0x7f2364101390 2024-06-10T01:01:04.475 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:01:04.475 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-06-10T01:01:04.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.478+0000 7f2368ef1700 1 -- 172.21.15.7:0/1205436804 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f23500a2c20 msgr2=0x7f23500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:04.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.478+0000 7f2368ef1700 1 --2- 172.21.15.7:0/1205436804 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f23500a2c20 0x7f23500a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f236406b1c0 tx=0x7f234c01a040 comp rx=0 tx=0).stop 2024-06-10T01:01:04.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.478+0000 7f2368ef1700 1 -- 172.21.15.7:0/1205436804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2364101390 msgr2=0x7f236419ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:04.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.478+0000 7f2368ef1700 1 --2- 172.21.15.7:0/1205436804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2364101390 0x7f236419ada0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f235400c9b0 tx=0x7f235400cd70 comp rx=0 tx=0).stop 2024-06-10T01:01:04.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.478+0000 7f2368ef1700 1 -- 172.21.15.7:0/1205436804 shutdown_connections 2024-06-10T01:01:04.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.478+0000 7f2368ef1700 1 --2- 172.21.15.7:0/1205436804 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f23500a2c20 0x7f23500a50e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:04.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.478+0000 7f2368ef1700 1 --2- 172.21.15.7:0/1205436804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2364107400 0x7f236419f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:04.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.478+0000 7f2368ef1700 1 --2- 172.21.15.7:0/1205436804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2364101390 0x7f236419ada0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:04.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.478+0000 7f2368ef1700 1 --2- 172.21.15.7:0/1205436804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2364101d50 0x7f236419b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:04.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.478+0000 7f2368ef1700 1 -- 172.21.15.7:0/1205436804 >> 172.21.15.7:0/1205436804 conn(0x7f2364078340 msgr2=0x7f236406a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:04.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.478+0000 7f2368ef1700 1 -- 172.21.15.7:0/1205436804 shutdown_connections 2024-06-10T01:01:04.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:04.479+0000 7f2368ef1700 1 -- 172.21.15.7:0/1205436804 wait complete. 2024-06-10T01:01:04.478 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 3 2024-06-10T01:01:05.157 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:04 smithi007 ceph-mon[32398]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:05.158 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:04 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1205436804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:01:05.158 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jun 10 01:01:04 smithi007 podman[73265]: 2024-06-10 01:01:04.898379172 +0000 UTC m=+0.574404166 container init 7f40cbd8be744682b66fe5abfcb237bb6df2a59e7d9a0f5c5b3aec7e3f64c98b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-j, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T01:01:05.158 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jun 10 01:01:04 smithi007 podman[73265]: 2024-06-10 01:01:04.901587167 +0000 UTC m=+0.577612167 container start 7f40cbd8be744682b66fe5abfcb237bb6df2a59e7d9a0f5c5b3aec7e3f64c98b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-j, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-06-10T01:01:05.158 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jun 10 01:01:04 smithi007 ceph-mds[73297]: -- 172.21.15.7:0/384921640 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d9e9121a40 con 0x55d9e90f5000 2024-06-10T01:01:05.158 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jun 10 01:01:04 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-j[73293]: starting mds.j at 2024-06-10T01:01:05.158 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jun 10 01:01:04 smithi007 bash[73265]: 7f40cbd8be744682b66fe5abfcb237bb6df2a59e7d9a0f5c5b3aec7e3f64c98b 2024-06-10T01:01:05.158 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jun 10 01:01:04 smithi007 systemd[1]: Started Ceph mds.j for bded58f8-26c3-11ef-bc9b-c7b262605968. 2024-06-10T01:01:05.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:04 smithi175 ceph-mon[36636]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:05.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:04 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1205436804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:01:05.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:04 smithi078 ceph-mon[36683]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:05.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:04 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1205436804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:01:06.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:06 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:06.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:06 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:06.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:06 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:06.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:06 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:01:06.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:06 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:01:06.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:06 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:06.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:06 smithi007 ceph-mon[32398]: Deploying daemon mds.d on smithi007 2024-06-10T01:01:06.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:06 smithi007 ceph-mon[32398]: mds.? [v2:172.21.15.7:6834/2172378766,v1:172.21.15.7:6835/2172378766] up:boot 2024-06-10T01:01:06.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:06 smithi007 ceph-mon[32398]: fsmap 3 up:standby 2024-06-10T01:01:06.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:06 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-10T01:01:06.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:06 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:06.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:06 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:06.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:06 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:06.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:06 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:01:06.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:06 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:01:06.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:06 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:06.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:06 smithi078 ceph-mon[36683]: Deploying daemon mds.d on smithi007 2024-06-10T01:01:06.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:06 smithi078 ceph-mon[36683]: mds.? [v2:172.21.15.7:6834/2172378766,v1:172.21.15.7:6835/2172378766] up:boot 2024-06-10T01:01:06.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:06 smithi078 ceph-mon[36683]: fsmap 3 up:standby 2024-06-10T01:01:06.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:06 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-10T01:01:06.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:06 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:06.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:06 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:06.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:06 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:06.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:06 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:01:06.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:06 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:01:06.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:06 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:06.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:06 smithi175 ceph-mon[36636]: Deploying daemon mds.d on smithi007 2024-06-10T01:01:06.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:06 smithi175 ceph-mon[36636]: mds.? [v2:172.21.15.7:6834/2172378766,v1:172.21.15.7:6835/2172378766] up:boot 2024-06-10T01:01:06.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:06 smithi175 ceph-mon[36636]: fsmap 3 up:standby 2024-06-10T01:01:06.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:06 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-10T01:01:07.331 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:07 smithi007 ceph-mon[32398]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:07.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:07 smithi078 ceph-mon[36683]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:07.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:07 smithi175 ceph-mon[36636]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:07.596 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:01:07.621 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jun 10 01:01:07 smithi007 podman[73584]: 2024-06-10 01:01:07.556519017 +0000 UTC m=+0.117851777 container create 894ef88f23c55c5926f06f506d8feff14b4dcc4243871f7bac1b35b4fe0b04c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-d, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-06-10T01:01:07.908 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jun 10 01:01:07 smithi007 podman[73584]: 2024-06-10 01:01:07.81782096 +0000 UTC m=+0.379153694 container init 894ef88f23c55c5926f06f506d8feff14b4dcc4243871f7bac1b35b4fe0b04c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-d, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-06-10T01:01:07.908 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jun 10 01:01:07 smithi007 podman[73584]: 2024-06-10 01:01:07.821635133 +0000 UTC m=+0.382967880 container start 894ef88f23c55c5926f06f506d8feff14b4dcc4243871f7bac1b35b4fe0b04c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-d, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T01:01:07.908 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jun 10 01:01:07 smithi007 ceph-mds[73632]: -- 172.21.15.7:0/3170285002 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5570ebb27a40 con 0x5570ebafa800 2024-06-10T01:01:07.908 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jun 10 01:01:07 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-d[73617]: starting mds.d at 2024-06-10T01:01:07.981 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:01:08.245 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jun 10 01:01:07 smithi007 bash[73584]: 894ef88f23c55c5926f06f506d8feff14b4dcc4243871f7bac1b35b4fe0b04c2 2024-06-10T01:01:08.245 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jun 10 01:01:07 smithi007 systemd[1]: Started Ceph mds.d for bded58f8-26c3-11ef-bc9b-c7b262605968. 2024-06-10T01:01:09.171 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:08 smithi007 ceph-mon[32398]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:09.171 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:08 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:09.171 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:08 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:09.171 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:08 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:09.171 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:08 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:01:09.171 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:08 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:01:09.171 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:08 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:09.171 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:08 smithi007 ceph-mon[32398]: Deploying daemon mds.e on smithi078 2024-06-10T01:01:09.171 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:08 smithi007 ceph-mon[32398]: mds.? [v2:172.21.15.7:6836/1734773915,v1:172.21.15.7:6837/1734773915] up:boot 2024-06-10T01:01:09.171 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:08 smithi007 ceph-mon[32398]: fsmap 4 up:standby 2024-06-10T01:01:09.172 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:08 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-10T01:01:09.320 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:08 smithi078 ceph-mon[36683]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:09.320 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:08 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:09.320 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:08 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:09.321 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:08 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:09.321 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:08 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:01:09.321 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:08 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:01:09.321 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:08 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:09.321 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:08 smithi078 ceph-mon[36683]: Deploying daemon mds.e on smithi078 2024-06-10T01:01:09.321 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:08 smithi078 ceph-mon[36683]: mds.? [v2:172.21.15.7:6836/1734773915,v1:172.21.15.7:6837/1734773915] up:boot 2024-06-10T01:01:09.321 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:08 smithi078 ceph-mon[36683]: fsmap 4 up:standby 2024-06-10T01:01:09.321 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:08 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-10T01:01:09.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:08 smithi175 ceph-mon[36636]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:09.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:08 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:09.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:08 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:09.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:08 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:09.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:08 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:01:09.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:08 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:01:09.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:08 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:09.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:08 smithi175 ceph-mon[36636]: Deploying daemon mds.e on smithi078 2024-06-10T01:01:09.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:08 smithi175 ceph-mon[36636]: mds.? [v2:172.21.15.7:6836/1734773915,v1:172.21.15.7:6837/1734773915] up:boot 2024-06-10T01:01:09.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:08 smithi175 ceph-mon[36636]: fsmap 4 up:standby 2024-06-10T01:01:09.459 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:08 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-10T01:01:09.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.712+0000 7f85f65d6700 1 -- 172.21.15.7:0/2644998319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85f0107400 msgr2=0x7f85f01077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:09.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.712+0000 7f85f65d6700 1 --2- 172.21.15.7:0/2644998319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85f0107400 0x7f85f01077e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009990 tx=0x7f85e0009ca0 comp rx=0 tx=0).stop 2024-06-10T01:01:09.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.712+0000 7f85f65d6700 1 -- 172.21.15.7:0/2644998319 shutdown_connections 2024-06-10T01:01:09.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.712+0000 7f85f65d6700 1 --2- 172.21.15.7:0/2644998319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f0101d50 0x7f85f010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:09.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.712+0000 7f85f65d6700 1 --2- 172.21.15.7:0/2644998319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85f0107400 0x7f85f01077e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:09.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.712+0000 7f85f65d6700 1 --2- 172.21.15.7:0/2644998319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85f0101390 0x7f85f0101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:09.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.712+0000 7f85f65d6700 1 -- 172.21.15.7:0/2644998319 >> 172.21.15.7:0/2644998319 conn(0x7f85f0078340 msgr2=0x7f85f0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:09.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.712+0000 7f85f65d6700 1 -- 172.21.15.7:0/2644998319 shutdown_connections 2024-06-10T01:01:09.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.712+0000 7f85f65d6700 1 -- 172.21.15.7:0/2644998319 wait complete. 2024-06-10T01:01:09.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.712+0000 7f85f65d6700 1 Processor -- start 2024-06-10T01:01:09.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.712+0000 7f85f65d6700 1 -- start start 2024-06-10T01:01:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.713+0000 7f85f65d6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85f0101390 0x7f85f019e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.713+0000 7f85f65d6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f0101d50 0x7f85f019ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.713+0000 7f85f65d6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85f0107400 0x7f85f0198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.713+0000 7f85f65d6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f85f0111740 con 0x7f85f0107400 2024-06-10T01:01:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.713+0000 7f85f65d6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f85f01115c0 con 0x7f85f0101390 2024-06-10T01:01:09.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.713+0000 7f85f65d6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f85f01118c0 con 0x7f85f0101d50 2024-06-10T01:01:09.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.713+0000 7f85f4b73700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85f0107400 0x7f85f0198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:09.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.713+0000 7f85f4b73700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85f0107400 0x7f85f0198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33690/0 (socket says 172.21.15.7:33690) 2024-06-10T01:01:09.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.713+0000 7f85f4b73700 1 -- 172.21.15.7:0/1024914610 learned_addr learned my addr 172.21.15.7:0/1024914610 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:01:09.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.713+0000 7f85effff700 1 --2- 172.21.15.7:0/1024914610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85f0101390 0x7f85f019e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:09.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.713+0000 7f85ef7fe700 1 --2- 172.21.15.7:0/1024914610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f0101d50 0x7f85f019ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:09.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.713+0000 7f85f4b73700 1 -- 172.21.15.7:0/1024914610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f0101d50 msgr2=0x7f85f019ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:09.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.713+0000 7f85f4b73700 1 --2- 172.21.15.7:0/1024914610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f0101d50 0x7f85f019ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:09.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.713+0000 7f85f4b73700 1 -- 172.21.15.7:0/1024914610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85f0101390 msgr2=0x7f85f019e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:09.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.713+0000 7f85f4b73700 1 --2- 172.21.15.7:0/1024914610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85f0101390 0x7f85f019e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:09.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.713+0000 7f85f4b73700 1 -- 172.21.15.7:0/1024914610 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f01991e0 con 0x7f85f0107400 2024-06-10T01:01:09.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.713+0000 7f85f4b73700 1 --2- 172.21.15.7:0/1024914610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85f0107400 0x7f85f0198a80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f85e800b970 tx=0x7f85e800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:09.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.714+0000 7f85ed7fa700 1 -- 172.21.15.7:0/1024914610 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e8019070 con 0x7f85f0107400 2024-06-10T01:01:09.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.714+0000 7f85f65d6700 1 -- 172.21.15.7:0/1024914610 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85f01994d0 con 0x7f85f0107400 2024-06-10T01:01:09.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.714+0000 7f85ed7fa700 1 -- 172.21.15.7:0/1024914610 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85e800ce10 con 0x7f85f0107400 2024-06-10T01:01:09.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.714+0000 7f85ed7fa700 1 -- 172.21.15.7:0/1024914610 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e8007370 con 0x7f85f0107400 2024-06-10T01:01:09.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.714+0000 7f85f65d6700 1 -- 172.21.15.7:0/1024914610 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85f01a5790 con 0x7f85f0107400 2024-06-10T01:01:09.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.715+0000 7f85f65d6700 1 -- 172.21.15.7:0/1024914610 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85f0069180 con 0x7f85f0107400 2024-06-10T01:01:09.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.716+0000 7f85ed7fa700 1 -- 172.21.15.7:0/1024914610 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f85e801c070 con 0x7f85f0107400 2024-06-10T01:01:09.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.717+0000 7f85ed7fa700 1 --2- 172.21.15.7:0/1024914610 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f85d80a2c20 0x7f85d80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:09.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.717+0000 7f85effff700 1 --2- 172.21.15.7:0/1024914610 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f85d80a2c20 0x7f85d80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:09.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.717+0000 7f85ed7fa700 1 -- 172.21.15.7:0/1024914610 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f85e80ce420 con 0x7f85f0107400 2024-06-10T01:01:09.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.717+0000 7f85effff700 1 --2- 172.21.15.7:0/1024914610 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f85d80a2c20 0x7f85d80a50e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009750 tx=0x7f85e0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:09.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.721+0000 7f85ed7fa700 1 -- 172.21.15.7:0/1024914610 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f85e809b820 con 0x7f85f0107400 2024-06-10T01:01:09.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.975+0000 7f85f65d6700 1 -- 172.21.15.7:0/1024914610 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85f01a59c0 con 0x7f85f0107400 2024-06-10T01:01:09.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.976+0000 7f85ed7fa700 1 -- 172.21.15.7:0/1024914610 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3865 (secure 0 0 0) 0x7f85e809b120 con 0x7f85f0107400 2024-06-10T01:01:09.975 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:01:09.975 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-06-10T01:01:09.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.978+0000 7f85f65d6700 1 -- 172.21.15.7:0/1024914610 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f85d80a2c20 msgr2=0x7f85d80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:09.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.978+0000 7f85f65d6700 1 --2- 172.21.15.7:0/1024914610 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f85d80a2c20 0x7f85d80a50e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009750 tx=0x7f85e0005fd0 comp rx=0 tx=0).stop 2024-06-10T01:01:09.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.978+0000 7f85f65d6700 1 -- 172.21.15.7:0/1024914610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85f0107400 msgr2=0x7f85f0198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:09.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.978+0000 7f85f65d6700 1 --2- 172.21.15.7:0/1024914610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85f0107400 0x7f85f0198a80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f85e800b970 tx=0x7f85e800bc80 comp rx=0 tx=0).stop 2024-06-10T01:01:09.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.978+0000 7f85f65d6700 1 -- 172.21.15.7:0/1024914610 shutdown_connections 2024-06-10T01:01:09.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.978+0000 7f85f65d6700 1 --2- 172.21.15.7:0/1024914610 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f85d80a2c20 0x7f85d80a50e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:09.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.978+0000 7f85f65d6700 1 --2- 172.21.15.7:0/1024914610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85f0107400 0x7f85f0198a80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:09.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.978+0000 7f85f65d6700 1 --2- 172.21.15.7:0/1024914610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85f0101390 0x7f85f019e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:09.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.978+0000 7f85f65d6700 1 --2- 172.21.15.7:0/1024914610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f0101d50 0x7f85f019ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:09.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.978+0000 7f85f65d6700 1 -- 172.21.15.7:0/1024914610 >> 172.21.15.7:0/1024914610 conn(0x7f85f0078340 msgr2=0x7f85f01005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:09.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.978+0000 7f85f65d6700 1 -- 172.21.15.7:0/1024914610 shutdown_connections 2024-06-10T01:01:09.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:09.978+0000 7f85f65d6700 1 -- 172.21.15.7:0/1024914610 wait complete. 2024-06-10T01:01:09.978 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 5 2024-06-10T01:01:10.333 INFO:journalctl@ceph.mds.e.smithi078.stdout:Jun 10 01:01:10 smithi078 systemd[1]: Starting Ceph mds.e for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:01:10.652 INFO:journalctl@ceph.mds.e.smithi078.stdout:Jun 10 01:01:10 smithi078 podman[65833]: 2024-06-10 01:01:10.276398752 +0000 UTC m=+0.017369030 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:01:10.652 INFO:journalctl@ceph.mds.e.smithi078.stdout:Jun 10 01:01:10 smithi078 podman[65833]: 2024-06-10 01:01:10.40110426 +0000 UTC m=+0.142074518 container create fa29baba2c0244d2122b9f36e06d10b3d147f8491497995ddf7ee0c66c79b0a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-e, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240528) 2024-06-10T01:01:10.652 INFO:journalctl@ceph.mds.e.smithi078.stdout:Jun 10 01:01:10 smithi078 podman[65833]: 2024-06-10 01:01:10.653823493 +0000 UTC m=+0.394793744 container init fa29baba2c0244d2122b9f36e06d10b3d147f8491497995ddf7ee0c66c79b0a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-e, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-06-10T01:01:10.971 INFO:journalctl@ceph.mds.e.smithi078.stdout:Jun 10 01:01:10 smithi078 podman[65833]: 2024-06-10 01:01:10.656426921 +0000 UTC m=+0.397397188 container start fa29baba2c0244d2122b9f36e06d10b3d147f8491497995ddf7ee0c66c79b0a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-e, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T01:01:10.971 INFO:journalctl@ceph.mds.e.smithi078.stdout:Jun 10 01:01:10 smithi078 ceph-mds[65852]: -- 172.21.15.78:0/386947795 <== mon.2 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556d8afada40 con 0x556d8af81000 2024-06-10T01:01:10.971 INFO:journalctl@ceph.mds.e.smithi078.stdout:Jun 10 01:01:10 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-e[65848]: starting mds.e at 2024-06-10T01:01:10.972 INFO:journalctl@ceph.mds.e.smithi078.stdout:Jun 10 01:01:10 smithi078 bash[65833]: fa29baba2c0244d2122b9f36e06d10b3d147f8491497995ddf7ee0c66c79b0a5 2024-06-10T01:01:10.972 INFO:journalctl@ceph.mds.e.smithi078.stdout:Jun 10 01:01:10 smithi078 systemd[1]: Started Ceph mds.e for bded58f8-26c3-11ef-bc9b-c7b262605968. 2024-06-10T01:01:11.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:10 smithi007 ceph-mon[32398]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:11.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:10 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1024914610' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:01:11.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:10 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:11.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:10 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:11.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:10 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:11.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:10 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:01:11.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:10 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:01:11.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:10 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:11.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:10 smithi078 ceph-mon[36683]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:11.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:10 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1024914610' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:01:11.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:10 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:11.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:10 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:11.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:10 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:11.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:10 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:01:11.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:10 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:01:11.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:10 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:11.449 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:10 smithi175 ceph-mon[36636]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:11.449 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:10 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1024914610' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:01:11.449 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:10 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:11.449 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:10 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:11.449 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:10 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:11.449 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:10 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:01:11.449 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:10 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:01:11.449 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:10 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:12.206 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:11 smithi175 ceph-mon[36636]: Deploying daemon mds.f on smithi175 2024-06-10T01:01:12.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:11 smithi175 ceph-mon[36636]: mds.? [v2:172.21.15.78:6834/938897057,v1:172.21.15.78:6835/938897057] up:boot 2024-06-10T01:01:12.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:11 smithi175 ceph-mon[36636]: fsmap 5 up:standby 2024-06-10T01:01:12.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:11 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-10T01:01:12.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:11 smithi007 ceph-mon[32398]: Deploying daemon mds.f on smithi175 2024-06-10T01:01:12.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:11 smithi007 ceph-mon[32398]: mds.? [v2:172.21.15.78:6834/938897057,v1:172.21.15.78:6835/938897057] up:boot 2024-06-10T01:01:12.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:11 smithi007 ceph-mon[32398]: fsmap 5 up:standby 2024-06-10T01:01:12.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:11 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-10T01:01:12.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:11 smithi078 ceph-mon[36683]: Deploying daemon mds.f on smithi175 2024-06-10T01:01:12.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:11 smithi078 ceph-mon[36683]: mds.? [v2:172.21.15.78:6834/938897057,v1:172.21.15.78:6835/938897057] up:boot 2024-06-10T01:01:12.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:11 smithi078 ceph-mon[36683]: fsmap 5 up:standby 2024-06-10T01:01:12.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:11 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-10T01:01:12.746 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:01:13.001 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:01:13.016 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:12 smithi007 ceph-mon[32398]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:13.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:12 smithi078 ceph-mon[36683]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:13.356 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:01:13 smithi175 systemd[1]: Starting Ceph mds.f for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:01:13.356 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:12 smithi175 ceph-mon[36636]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:13.607 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:01:13 smithi175 podman[66170]: 2024-06-10 01:01:13.260311864 +0000 UTC m=+0.016835577 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:01:13.607 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:01:13 smithi175 podman[66170]: 2024-06-10 01:01:13.390003236 +0000 UTC m=+0.146526978 container create 563aad482e1c65bef618894d0830b82b911274eb3e874d97aeb1d0bdccd7b593 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T01:01:13.607 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:01:13 smithi175 podman[66170]: 2024-06-10 01:01:13.609568103 +0000 UTC m=+0.366091803 container init 563aad482e1c65bef618894d0830b82b911274eb3e874d97aeb1d0bdccd7b593 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-06-10T01:01:13.957 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:01:13 smithi175 podman[66170]: 2024-06-10 01:01:13.61245708 +0000 UTC m=+0.368980781 container start 563aad482e1c65bef618894d0830b82b911274eb3e874d97aeb1d0bdccd7b593 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-f, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-06-10T01:01:13.957 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:01:13 smithi175 ceph-mds[66189]: -- 172.21.15.175:0/3430043125 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bfdf6afa40 con 0x55bfdf682800 2024-06-10T01:01:13.957 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:01:13 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-f[66185]: starting mds.f at 2024-06-10T01:01:13.957 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:01:13 smithi175 bash[66170]: 563aad482e1c65bef618894d0830b82b911274eb3e874d97aeb1d0bdccd7b593 2024-06-10T01:01:13.957 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:01:13 smithi175 systemd[1]: Started Ceph mds.f for bded58f8-26c3-11ef-bc9b-c7b262605968. 2024-06-10T01:01:14.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.590+0000 7f693c0ab700 1 -- 172.21.15.7:0/2699596531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6934101d30 msgr2=0x7f693410e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:14.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.590+0000 7f693c0ab700 1 --2- 172.21.15.7:0/2699596531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6934101d30 0x7f693410e210 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f692c00b0a0 tx=0x7f692c00b3b0 comp rx=0 tx=0).stop 2024-06-10T01:01:14.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.590+0000 7f693c0ab700 1 -- 172.21.15.7:0/2699596531 shutdown_connections 2024-06-10T01:01:14.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.590+0000 7f693c0ab700 1 --2- 172.21.15.7:0/2699596531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6934101d30 0x7f693410e210 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:14.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.590+0000 7f693c0ab700 1 --2- 172.21.15.7:0/2699596531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69341073d0 0x7f69341077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:14.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.590+0000 7f693c0ab700 1 --2- 172.21.15.7:0/2699596531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6934101390 0x7f69341017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:14.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.590+0000 7f693c0ab700 1 -- 172.21.15.7:0/2699596531 >> 172.21.15.7:0/2699596531 conn(0x7f69340782c0 msgr2=0x7f69340786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:14.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.590+0000 7f693c0ab700 1 -- 172.21.15.7:0/2699596531 shutdown_connections 2024-06-10T01:01:14.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.590+0000 7f693c0ab700 1 -- 172.21.15.7:0/2699596531 wait complete. 2024-06-10T01:01:14.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.591+0000 7f693c0ab700 1 Processor -- start 2024-06-10T01:01:14.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.591+0000 7f693c0ab700 1 -- start start 2024-06-10T01:01:14.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.592+0000 7f693c0ab700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6934101390 0x7f693419d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:14.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.592+0000 7f693c0ab700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6934101d30 0x7f693419dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:14.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.592+0000 7f693c0ab700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69341073d0 0x7f6934197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:14.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.592+0000 7f693c0ab700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6934111650 con 0x7f69341073d0 2024-06-10T01:01:14.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.592+0000 7f693c0ab700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f69341114d0 con 0x7f6934101390 2024-06-10T01:01:14.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.592+0000 7f693c0ab700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f69341117d0 con 0x7f6934101d30 2024-06-10T01:01:14.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.592+0000 7f693a648700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69341073d0 0x7f6934197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:14.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.592+0000 7f693a648700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69341073d0 0x7f6934197890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45612/0 (socket says 172.21.15.7:45612) 2024-06-10T01:01:14.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.592+0000 7f693a648700 1 -- 172.21.15.7:0/4043305777 learned_addr learned my addr 172.21.15.7:0/4043305777 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:01:14.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.592+0000 7f6939646700 1 --2- 172.21.15.7:0/4043305777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6934101d30 0x7f693419dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:14.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.592+0000 7f6939e47700 1 --2- 172.21.15.7:0/4043305777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6934101390 0x7f693419d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:14.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.592+0000 7f693a648700 1 -- 172.21.15.7:0/4043305777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6934101d30 msgr2=0x7f693419dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:14.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.592+0000 7f693a648700 1 --2- 172.21.15.7:0/4043305777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6934101d30 0x7f693419dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:14.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.592+0000 7f693a648700 1 -- 172.21.15.7:0/4043305777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6934101390 msgr2=0x7f693419d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:14.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.592+0000 7f693a648700 1 --2- 172.21.15.7:0/4043305777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6934101390 0x7f693419d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:14.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.592+0000 7f693a648700 1 -- 172.21.15.7:0/4043305777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6934198070 con 0x7f69341073d0 2024-06-10T01:01:14.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.592+0000 7f6939e47700 1 --2- 172.21.15.7:0/4043305777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6934101390 0x7f693419d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:01:14.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.592+0000 7f693a648700 1 --2- 172.21.15.7:0/4043305777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69341073d0 0x7f6934197890 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f692c00b780 tx=0x7f692c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:14.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.592+0000 7f692affd700 1 -- 172.21.15.7:0/4043305777 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c021070 con 0x7f69341073d0 2024-06-10T01:01:14.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.592+0000 7f693c0ab700 1 -- 172.21.15.7:0/4043305777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6934198300 con 0x7f69341073d0 2024-06-10T01:01:14.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.592+0000 7f692affd700 1 -- 172.21.15.7:0/4043305777 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f692c007b80 con 0x7f69341073d0 2024-06-10T01:01:14.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.592+0000 7f692affd700 1 -- 172.21.15.7:0/4043305777 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c005370 con 0x7f69341073d0 2024-06-10T01:01:14.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.593+0000 7f693c0ab700 1 -- 172.21.15.7:0/4043305777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f693406a170 con 0x7f69341073d0 2024-06-10T01:01:14.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.594+0000 7f693c0ab700 1 -- 172.21.15.7:0/4043305777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6934102a80 con 0x7f69341073d0 2024-06-10T01:01:14.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.595+0000 7f692affd700 1 -- 172.21.15.7:0/4043305777 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f692c024070 con 0x7f69341073d0 2024-06-10T01:01:14.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.595+0000 7f692affd700 1 --2- 172.21.15.7:0/4043305777 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f69200a4e30 0x7f69200a72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:14.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.595+0000 7f6939e47700 1 --2- 172.21.15.7:0/4043305777 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f69200a4e30 0x7f69200a72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:14.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.595+0000 7f692affd700 1 -- 172.21.15.7:0/4043305777 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f692c00f020 con 0x7f69341073d0 2024-06-10T01:01:14.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.596+0000 7f6939e47700 1 --2- 172.21.15.7:0/4043305777 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f69200a4e30 0x7f69200a72f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6930009c90 tx=0x7f6930009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:14.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.600+0000 7f692affd700 1 -- 172.21.15.7:0/4043305777 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f692c02b020 con 0x7f69341073d0 2024-06-10T01:01:14.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.852+0000 7f693c0ab700 1 -- 172.21.15.7:0/4043305777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f693406ab10 con 0x7f69341073d0 2024-06-10T01:01:14.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.853+0000 7f692affd700 1 -- 172.21.15.7:0/4043305777 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5515 (secure 0 0 0) 0x7f692c09d5b0 con 0x7f69341073d0 2024-06-10T01:01:14.852 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:01:14.852 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2784910673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2784910673},{"type":"v1","addr":"172.21.15.175:6835","nonce":2784910673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-06-10T01:01:14.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.855+0000 7f693c0ab700 1 -- 172.21.15.7:0/4043305777 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f69200a4e30 msgr2=0x7f69200a72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:14.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.855+0000 7f693c0ab700 1 --2- 172.21.15.7:0/4043305777 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f69200a4e30 0x7f69200a72f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6930009c90 tx=0x7f6930009260 comp rx=0 tx=0).stop 2024-06-10T01:01:14.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.855+0000 7f693c0ab700 1 -- 172.21.15.7:0/4043305777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69341073d0 msgr2=0x7f6934197890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:14.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.855+0000 7f693c0ab700 1 --2- 172.21.15.7:0/4043305777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69341073d0 0x7f6934197890 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f692c00b780 tx=0x7f692c004d00 comp rx=0 tx=0).stop 2024-06-10T01:01:14.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.855+0000 7f693c0ab700 1 -- 172.21.15.7:0/4043305777 shutdown_connections 2024-06-10T01:01:14.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.855+0000 7f693c0ab700 1 --2- 172.21.15.7:0/4043305777 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f69200a4e30 0x7f69200a72f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:14.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.855+0000 7f693c0ab700 1 --2- 172.21.15.7:0/4043305777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69341073d0 0x7f6934197890 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:14.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.855+0000 7f693c0ab700 1 --2- 172.21.15.7:0/4043305777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6934101390 0x7f693419d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:14.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.855+0000 7f693c0ab700 1 --2- 172.21.15.7:0/4043305777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6934101d30 0x7f693419dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:14.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.855+0000 7f693c0ab700 1 -- 172.21.15.7:0/4043305777 >> 172.21.15.7:0/4043305777 conn(0x7f69340782c0 msgr2=0x7f69340ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:14.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.855+0000 7f693c0ab700 1 -- 172.21.15.7:0/4043305777 shutdown_connections 2024-06-10T01:01:14.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:14.855+0000 7f693c0ab700 1 -- 172.21.15.7:0/4043305777 wait complete. 2024-06-10T01:01:14.855 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 7 2024-06-10T01:01:15.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:14 smithi007 ceph-mon[32398]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:15.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:14 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:15.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:14 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:15.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:14 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:15.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:14 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:01:15.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:14 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:01:15.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:14 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:15.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:14 smithi007 ceph-mon[32398]: Deploying daemon mds.c on smithi175 2024-06-10T01:01:15.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:14 smithi007 ceph-mon[32398]: mds.? [v2:172.21.15.175:6834/2784910673,v1:172.21.15.175:6835/2784910673] up:boot 2024-06-10T01:01:15.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:14 smithi007 ceph-mon[32398]: fsmap 6 up:standby 2024-06-10T01:01:15.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:14 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T01:01:15.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:14 smithi078 ceph-mon[36683]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:15.084 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:14 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:15.084 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:14 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:15.084 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:14 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:15.084 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:14 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:01:15.084 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:14 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:01:15.084 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:14 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:15.084 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:14 smithi078 ceph-mon[36683]: Deploying daemon mds.c on smithi175 2024-06-10T01:01:15.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:14 smithi078 ceph-mon[36683]: mds.? [v2:172.21.15.175:6834/2784910673,v1:172.21.15.175:6835/2784910673] up:boot 2024-06-10T01:01:15.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:14 smithi078 ceph-mon[36683]: fsmap 6 up:standby 2024-06-10T01:01:15.085 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:14 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T01:01:15.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:14 smithi175 ceph-mon[36636]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:15.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:14 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:15.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:14 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:15.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:14 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:15.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:14 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:01:15.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:14 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:01:15.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:14 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:15.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:14 smithi175 ceph-mon[36636]: Deploying daemon mds.c on smithi175 2024-06-10T01:01:15.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:14 smithi175 ceph-mon[36636]: mds.? [v2:172.21.15.175:6834/2784910673,v1:172.21.15.175:6835/2784910673] up:boot 2024-06-10T01:01:15.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:14 smithi175 ceph-mon[36636]: fsmap 6 up:standby 2024-06-10T01:01:15.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:14 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T01:01:15.956 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:15 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/4043305777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:01:15.957 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jun 10 01:01:15 smithi175 systemd[1]: Starting Ceph mds.c for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:01:16.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:15 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/4043305777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:01:16.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:15 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/4043305777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:01:16.356 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jun 10 01:01:16 smithi175 podman[66472]: 2024-06-10 01:01:16.001094018 +0000 UTC m=+0.017779518 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:01:16.356 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jun 10 01:01:16 smithi175 podman[66472]: 2024-06-10 01:01:16.122214087 +0000 UTC m=+0.138899568 container create 7d697cfd2a30bec8e039f1bf60185e3911d7136defa871d34412406b4cb42e5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240528) 2024-06-10T01:01:16.356 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jun 10 01:01:16 smithi175 podman[66472]: 2024-06-10 01:01:16.358633434 +0000 UTC m=+0.375318925 container init 7d697cfd2a30bec8e039f1bf60185e3911d7136defa871d34412406b4cb42e5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-c, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528) 2024-06-10T01:01:16.706 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jun 10 01:01:16 smithi175 podman[66472]: 2024-06-10 01:01:16.36198732 +0000 UTC m=+0.378672810 container start 7d697cfd2a30bec8e039f1bf60185e3911d7136defa871d34412406b4cb42e5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528) 2024-06-10T01:01:16.707 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jun 10 01:01:16 smithi175 ceph-mds[66491]: -- 172.21.15.175:0/3747532701 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56406ce51a40 con 0x56406ce24800 2024-06-10T01:01:16.707 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jun 10 01:01:16 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-c[66487]: starting mds.c at 2024-06-10T01:01:16.707 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jun 10 01:01:16 smithi175 bash[66472]: 7d697cfd2a30bec8e039f1bf60185e3911d7136defa871d34412406b4cb42e5c 2024-06-10T01:01:16.707 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jun 10 01:01:16 smithi175 systemd[1]: Started Ceph mds.c for bded58f8-26c3-11ef-bc9b-c7b262605968. 2024-06-10T01:01:17.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:16 smithi007 ceph-mon[32398]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:17.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:16 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:17.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:16 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:17.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:16 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:17.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:16 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:01:17.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:16 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:01:17.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:16 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:17.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:16 smithi078 ceph-mon[36683]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:17.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:16 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:17.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:16 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:17.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:16 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:17.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:16 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:01:17.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:16 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:01:17.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:16 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:17.206 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:16 smithi175 ceph-mon[36636]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:17.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:16 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:17.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:16 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:17.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:16 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:17.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:16 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:01:17.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:16 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:01:17.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:16 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:17.630 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:01:17.892 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:01:18.014 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:17 smithi007 ceph-mon[32398]: Deploying daemon mds.h on smithi078 2024-06-10T01:01:18.015 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:17 smithi007 ceph-mon[32398]: mds.? [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] up:boot 2024-06-10T01:01:18.015 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:17 smithi007 ceph-mon[32398]: fsmap 7 up:standby 2024-06-10T01:01:18.015 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:17 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-10T01:01:18.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:17 smithi078 ceph-mon[36683]: Deploying daemon mds.h on smithi078 2024-06-10T01:01:18.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:17 smithi078 ceph-mon[36683]: mds.? [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] up:boot 2024-06-10T01:01:18.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:17 smithi078 ceph-mon[36683]: fsmap 7 up:standby 2024-06-10T01:01:18.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:17 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-10T01:01:18.206 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:17 smithi175 ceph-mon[36636]: Deploying daemon mds.h on smithi078 2024-06-10T01:01:18.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:17 smithi175 ceph-mon[36636]: mds.? [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] up:boot 2024-06-10T01:01:18.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:17 smithi175 ceph-mon[36636]: fsmap 7 up:standby 2024-06-10T01:01:18.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:17 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-10T01:01:18.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:18 smithi175 ceph-mon[36636]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:19.000 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:18 smithi078 ceph-mon[36683]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:19.001 INFO:journalctl@ceph.mds.h.smithi078.stdout:Jun 10 01:01:18 smithi078 systemd[1]: Starting Ceph mds.h for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:01:19.016 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:18 smithi007 ceph-mon[32398]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:19.299 INFO:journalctl@ceph.mds.h.smithi078.stdout:Jun 10 01:01:19 smithi078 podman[66139]: 2024-06-10 01:01:18.905211142 +0000 UTC m=+0.017637064 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:01:19.299 INFO:journalctl@ceph.mds.h.smithi078.stdout:Jun 10 01:01:19 smithi078 podman[66139]: 2024-06-10 01:01:19.040733667 +0000 UTC m=+0.153159567 container create a9cd2441b6b2d8f7490969a6d94afa0324a36ec83db3b69be36fc71e29bd5c9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-h, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T01:01:19.299 INFO:journalctl@ceph.mds.h.smithi078.stdout:Jun 10 01:01:19 smithi078 podman[66139]: 2024-06-10 01:01:19.301027473 +0000 UTC m=+0.413453373 container init a9cd2441b6b2d8f7490969a6d94afa0324a36ec83db3b69be36fc71e29bd5c9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-h, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True) 2024-06-10T01:01:19.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.568+0000 7fc9959a7700 1 -- 172.21.15.7:0/1767484053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc990107840 msgr2=0x7fc99010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:19.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.568+0000 7fc9959a7700 1 --2- 172.21.15.7:0/1767484053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc990107840 0x7fc99010f3c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc98800b0d0 tx=0x7fc98800b3e0 comp rx=0 tx=0).stop 2024-06-10T01:01:19.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.568+0000 7fc9959a7700 1 -- 172.21.15.7:0/1767484053 shutdown_connections 2024-06-10T01:01:19.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.568+0000 7fc9959a7700 1 --2- 172.21.15.7:0/1767484053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc990107840 0x7fc99010f3c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:19.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.568+0000 7fc9959a7700 1 --2- 172.21.15.7:0/1767484053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc990102740 0x7fc990102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:19.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.568+0000 7fc9959a7700 1 --2- 172.21.15.7:0/1767484053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9901030f0 0x7fc990107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:19.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.568+0000 7fc9959a7700 1 -- 172.21.15.7:0/1767484053 >> 172.21.15.7:0/1767484053 conn(0x7fc9900795f0 msgr2=0x7fc990079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:19.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.569+0000 7fc9959a7700 1 -- 172.21.15.7:0/1767484053 shutdown_connections 2024-06-10T01:01:19.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.569+0000 7fc9959a7700 1 -- 172.21.15.7:0/1767484053 wait complete. 2024-06-10T01:01:19.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.569+0000 7fc9959a7700 1 Processor -- start 2024-06-10T01:01:19.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.569+0000 7fc9959a7700 1 -- start start 2024-06-10T01:01:19.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.570+0000 7fc9959a7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc990102740 0x7fc990074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:19.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.570+0000 7fc9959a7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9901030f0 0x7fc990074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:19.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.570+0000 7fc9959a7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc990107840 0x7fc990075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:19.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.570+0000 7fc9959a7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc990112780 con 0x7fc9901030f0 2024-06-10T01:01:19.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.570+0000 7fc9959a7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc990112600 con 0x7fc990102740 2024-06-10T01:01:19.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.570+0000 7fc9959a7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc990112900 con 0x7fc990107840 2024-06-10T01:01:19.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.570+0000 7fc98e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9901030f0 0x7fc990074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:19.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.570+0000 7fc98effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc990102740 0x7fc990074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:19.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.570+0000 7fc98effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc990102740 0x7fc990074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:40166/0 (socket says 172.21.15.7:40166) 2024-06-10T01:01:19.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.570+0000 7fc98effd700 1 -- 172.21.15.7:0/16252926 learned_addr learned my addr 172.21.15.7:0/16252926 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:01:19.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.570+0000 7fc98f7fe700 1 --2- 172.21.15.7:0/16252926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc990107840 0x7fc990075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:19.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.570+0000 7fc98effd700 1 -- 172.21.15.7:0/16252926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc990107840 msgr2=0x7fc990075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:19.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.570+0000 7fc98effd700 1 --2- 172.21.15.7:0/16252926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc990107840 0x7fc990075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:19.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.570+0000 7fc98effd700 1 -- 172.21.15.7:0/16252926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9901030f0 msgr2=0x7fc990074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:19.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.570+0000 7fc98effd700 1 --2- 172.21.15.7:0/16252926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9901030f0 0x7fc990074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:19.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.570+0000 7fc98effd700 1 -- 172.21.15.7:0/16252926 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9901a6fe0 con 0x7fc990102740 2024-06-10T01:01:19.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.570+0000 7fc98e7fc700 1 --2- 172.21.15.7:0/16252926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9901030f0 0x7fc990074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:01:19.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.570+0000 7fc98f7fe700 1 --2- 172.21.15.7:0/16252926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc990107840 0x7fc990075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:01:19.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.570+0000 7fc98effd700 1 --2- 172.21.15.7:0/16252926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc990102740 0x7fc990074790 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc98000d980 tx=0x7fc98000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:19.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.570+0000 7fc9949a5700 1 -- 172.21.15.7:0/16252926 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980017070 con 0x7fc990102740 2024-06-10T01:01:19.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.571+0000 7fc9949a5700 1 -- 172.21.15.7:0/16252926 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc980005020 con 0x7fc990102740 2024-06-10T01:01:19.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.571+0000 7fc9959a7700 1 -- 172.21.15.7:0/16252926 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9901a72d0 con 0x7fc990102740 2024-06-10T01:01:19.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.571+0000 7fc9949a5700 1 -- 172.21.15.7:0/16252926 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98000fbf0 con 0x7fc990102740 2024-06-10T01:01:19.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.571+0000 7fc9959a7700 1 -- 172.21.15.7:0/16252926 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9901a7810 con 0x7fc990102740 2024-06-10T01:01:19.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.572+0000 7fc9959a7700 1 -- 172.21.15.7:0/16252926 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc990069180 con 0x7fc990102740 2024-06-10T01:01:19.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.573+0000 7fc9949a5700 1 -- 172.21.15.7:0/16252926 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fc9800051d0 con 0x7fc990102740 2024-06-10T01:01:19.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.573+0000 7fc9949a5700 1 --2- 172.21.15.7:0/16252926 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc9780a2ca0 0x7fc9780a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:19.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.573+0000 7fc98e7fc700 1 --2- 172.21.15.7:0/16252926 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc9780a2ca0 0x7fc9780a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:19.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.573+0000 7fc9949a5700 1 -- 172.21.15.7:0/16252926 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc9800cea00 con 0x7fc990102740 2024-06-10T01:01:19.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.574+0000 7fc98e7fc700 1 --2- 172.21.15.7:0/16252926 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc9780a2ca0 0x7fc9780a5160 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc990108b90 tx=0x7fc98400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:19.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.578+0000 7fc9949a5700 1 -- 172.21.15.7:0/16252926 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc980020020 con 0x7fc990102740 2024-06-10T01:01:19.583 INFO:journalctl@ceph.mds.h.smithi078.stdout:Jun 10 01:01:19 smithi078 podman[66139]: 2024-06-10 01:01:19.303788124 +0000 UTC m=+0.416214025 container start a9cd2441b6b2d8f7490969a6d94afa0324a36ec83db3b69be36fc71e29bd5c9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-h, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-06-10T01:01:19.583 INFO:journalctl@ceph.mds.h.smithi078.stdout:Jun 10 01:01:19 smithi078 ceph-mds[66158]: -- 172.21.15.78:0/1024011553 <== mon.2 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dffd3eda40 con 0x55dffd3c1000 2024-06-10T01:01:19.583 INFO:journalctl@ceph.mds.h.smithi078.stdout:Jun 10 01:01:19 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-h[66154]: starting mds.h at 2024-06-10T01:01:19.583 INFO:journalctl@ceph.mds.h.smithi078.stdout:Jun 10 01:01:19 smithi078 bash[66139]: a9cd2441b6b2d8f7490969a6d94afa0324a36ec83db3b69be36fc71e29bd5c9a 2024-06-10T01:01:19.583 INFO:journalctl@ceph.mds.h.smithi078.stdout:Jun 10 01:01:19 smithi078 systemd[1]: Started Ceph mds.h for bded58f8-26c3-11ef-bc9b-c7b262605968. 2024-06-10T01:01:19.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.834+0000 7fc9959a7700 1 -- 172.21.15.7:0/16252926 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9901a7b80 con 0x7fc990102740 2024-06-10T01:01:19.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.835+0000 7fc9949a5700 1 -- 172.21.15.7:0/16252926 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7165 (secure 0 0 0) 0x7fc98009bd60 con 0x7fc990102740 2024-06-10T01:01:19.834 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:01:19.835 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2784910673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2784910673},{"type":"v1","addr":"172.21.15.175:6835","nonce":2784910673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-06-10T01:01:19.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.837+0000 7fc9959a7700 1 -- 172.21.15.7:0/16252926 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc9780a2ca0 msgr2=0x7fc9780a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:19.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.837+0000 7fc9959a7700 1 --2- 172.21.15.7:0/16252926 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc9780a2ca0 0x7fc9780a5160 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc990108b90 tx=0x7fc98400a380 comp rx=0 tx=0).stop 2024-06-10T01:01:19.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.837+0000 7fc9959a7700 1 -- 172.21.15.7:0/16252926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc990102740 msgr2=0x7fc990074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:19.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.837+0000 7fc9959a7700 1 --2- 172.21.15.7:0/16252926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc990102740 0x7fc990074790 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc98000d980 tx=0x7fc98000dc90 comp rx=0 tx=0).stop 2024-06-10T01:01:19.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.837+0000 7fc9959a7700 1 -- 172.21.15.7:0/16252926 shutdown_connections 2024-06-10T01:01:19.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.837+0000 7fc9959a7700 1 --2- 172.21.15.7:0/16252926 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc9780a2ca0 0x7fc9780a5160 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:19.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.837+0000 7fc9959a7700 1 --2- 172.21.15.7:0/16252926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc990107840 0x7fc990075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:19.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.837+0000 7fc9959a7700 1 --2- 172.21.15.7:0/16252926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc990102740 0x7fc990074790 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:19.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.837+0000 7fc9959a7700 1 --2- 172.21.15.7:0/16252926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9901030f0 0x7fc990074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:19.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.837+0000 7fc9959a7700 1 -- 172.21.15.7:0/16252926 >> 172.21.15.7:0/16252926 conn(0x7fc9900795f0 msgr2=0x7fc990100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:19.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.837+0000 7fc9959a7700 1 -- 172.21.15.7:0/16252926 shutdown_connections 2024-06-10T01:01:19.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:19.837+0000 7fc9959a7700 1 -- 172.21.15.7:0/16252926 wait complete. 2024-06-10T01:01:19.838 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 9 2024-06-10T01:01:20.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:20.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:20.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:20.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:01:20.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:01:20.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:20.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:20 smithi175 ceph-mon[36636]: Deploying daemon mds.i on smithi175 2024-06-10T01:01:20.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:20 smithi175 ceph-mon[36636]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:20.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:20 smithi175 ceph-mon[36636]: mds.? [v2:172.21.15.78:6836/3608622984,v1:172.21.15.78:6837/3608622984] up:boot 2024-06-10T01:01:20.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:20 smithi175 ceph-mon[36636]: fsmap 8 up:standby 2024-06-10T01:01:20.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:20 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-10T01:01:20.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:20 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/16252926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:01:20.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:20 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:20.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:20 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:20.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:20 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:20.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:20 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:01:20.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:20 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:01:20.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:20 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:20.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:20 smithi007 ceph-mon[32398]: Deploying daemon mds.i on smithi175 2024-06-10T01:01:20.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:20 smithi007 ceph-mon[32398]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:20.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:20 smithi007 ceph-mon[32398]: mds.? [v2:172.21.15.78:6836/3608622984,v1:172.21.15.78:6837/3608622984] up:boot 2024-06-10T01:01:20.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:20 smithi007 ceph-mon[32398]: fsmap 8 up:standby 2024-06-10T01:01:20.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:20 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-10T01:01:20.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:20 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/16252926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:01:20.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:20 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:20.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:20 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:20.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:20 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:20.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:20 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:01:20.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:20 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:01:20.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:20 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:20.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:20 smithi078 ceph-mon[36683]: Deploying daemon mds.i on smithi175 2024-06-10T01:01:20.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:20 smithi078 ceph-mon[36683]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:20.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:20 smithi078 ceph-mon[36683]: mds.? [v2:172.21.15.78:6836/3608622984,v1:172.21.15.78:6837/3608622984] up:boot 2024-06-10T01:01:20.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:20 smithi078 ceph-mon[36683]: fsmap 8 up:standby 2024-06-10T01:01:20.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:20 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-10T01:01:20.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:20 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/16252926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:01:21.706 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jun 10 01:01:21 smithi175 systemd[1]: Starting Ceph mds.i for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:01:22.111 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jun 10 01:01:21 smithi175 podman[66787]: 2024-06-10 01:01:21.74639938 +0000 UTC m=+0.017415876 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:01:22.112 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jun 10 01:01:21 smithi175 podman[66787]: 2024-06-10 01:01:21.861102964 +0000 UTC m=+0.132119446 container create 8e302b883cb81dae3b25d3e4d9088352fadd8ea2801bb07518cafb9914787fac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-i, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T01:01:22.456 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jun 10 01:01:22 smithi175 podman[66787]: 2024-06-10 01:01:22.114059201 +0000 UTC m=+0.385075684 container init 8e302b883cb81dae3b25d3e4d9088352fadd8ea2801bb07518cafb9914787fac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-i, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240528) 2024-06-10T01:01:22.457 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jun 10 01:01:22 smithi175 podman[66787]: 2024-06-10 01:01:22.116688945 +0000 UTC m=+0.387705431 container start 8e302b883cb81dae3b25d3e4d9088352fadd8ea2801bb07518cafb9914787fac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.build-date=20240528, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T01:01:22.457 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jun 10 01:01:22 smithi175 ceph-mds[66806]: -- 172.21.15.175:0/1478158698 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e19a56da40 con 0x55e19a540c00 2024-06-10T01:01:22.457 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jun 10 01:01:22 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-i[66802]: starting mds.i at 2024-06-10T01:01:22.457 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jun 10 01:01:22 smithi175 bash[66787]: 8e302b883cb81dae3b25d3e4d9088352fadd8ea2801bb07518cafb9914787fac 2024-06-10T01:01:22.457 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jun 10 01:01:22 smithi175 systemd[1]: Started Ceph mds.i for bded58f8-26c3-11ef-bc9b-c7b262605968. 2024-06-10T01:01:22.496 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:01:22.755 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:01:23.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:23 smithi007 ceph-mon[32398]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:23.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:23 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:23.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:23 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:23.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:23 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:23.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:23 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:01:23.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:23 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:01:23.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:23 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:23.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:23 smithi007 ceph-mon[32398]: Deploying daemon mds.g on smithi007 2024-06-10T01:01:23.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:23 smithi007 ceph-mon[32398]: mds.? [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] up:boot 2024-06-10T01:01:23.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:23 smithi007 ceph-mon[32398]: fsmap 9 up:standby 2024-06-10T01:01:23.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:23 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-10T01:01:23.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:23 smithi078 ceph-mon[36683]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:23.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:23 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:23.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:23 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:23.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:23 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:23.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:23 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:01:23.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:23 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:01:23.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:23 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:23.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:23 smithi078 ceph-mon[36683]: Deploying daemon mds.g on smithi007 2024-06-10T01:01:23.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:23 smithi078 ceph-mon[36683]: mds.? [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] up:boot 2024-06-10T01:01:23.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:23 smithi078 ceph-mon[36683]: fsmap 9 up:standby 2024-06-10T01:01:23.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:23 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-10T01:01:23.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:23 smithi175 ceph-mon[36636]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:23.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:23 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:23.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:23 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:23.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:23 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:23.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:23 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:01:23.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:23 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:01:23.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:23 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:23.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:23 smithi175 ceph-mon[36636]: Deploying daemon mds.g on smithi007 2024-06-10T01:01:23.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:23 smithi175 ceph-mon[36636]: mds.? [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] up:boot 2024-06-10T01:01:23.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:23 smithi175 ceph-mon[36636]: fsmap 9 up:standby 2024-06-10T01:01:23.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:23 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-10T01:01:25.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.085+0000 7f883ea3e700 1 -- 172.21.15.7:0/489705087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88381066b0 msgr2=0x7f883810e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:25.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.085+0000 7f883ea3e700 1 --2- 172.21.15.7:0/489705087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88381066b0 0x7f883810e280 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f883000b0d0 tx=0x7f883000b3e0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.085+0000 7f883ea3e700 1 -- 172.21.15.7:0/489705087 shutdown_connections 2024-06-10T01:01:25.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.085+0000 7f883ea3e700 1 --2- 172.21.15.7:0/489705087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88381066b0 0x7f883810e280 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.085+0000 7f883ea3e700 1 --2- 172.21.15.7:0/489705087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8838101620 0x7f8838101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.085+0000 7f883ea3e700 1 --2- 172.21.15.7:0/489705087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8838101fd0 0x7f8838105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.085+0000 7f883ea3e700 1 -- 172.21.15.7:0/489705087 >> 172.21.15.7:0/489705087 conn(0x7f8838078340 msgr2=0x7f8838078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:25.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.085+0000 7f883ea3e700 1 -- 172.21.15.7:0/489705087 shutdown_connections 2024-06-10T01:01:25.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.085+0000 7f883ea3e700 1 -- 172.21.15.7:0/489705087 wait complete. 2024-06-10T01:01:25.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.085+0000 7f883ea3e700 1 Processor -- start 2024-06-10T01:01:25.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.086+0000 7f883ea3e700 1 -- start start 2024-06-10T01:01:25.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.086+0000 7f883ea3e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8838101620 0x7f883819e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:25.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.086+0000 7f883ea3e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8838101fd0 0x7f883819ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:25.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.086+0000 7f883ea3e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f88381066b0 0x7f8838198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:25.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.086+0000 7f883ea3e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f88381116a0 con 0x7f8838101620 2024-06-10T01:01:25.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.086+0000 7f883ea3e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8838111520 con 0x7f88381066b0 2024-06-10T01:01:25.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.086+0000 7f883ea3e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8838111820 con 0x7f8838101fd0 2024-06-10T01:01:25.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.086+0000 7f8837fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8838101620 0x7f883819e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:25.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.086+0000 7f8837fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8838101620 0x7f883819e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34684/0 (socket says 172.21.15.7:34684) 2024-06-10T01:01:25.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.086+0000 7f8837fff700 1 -- 172.21.15.7:0/2765049384 learned_addr learned my addr 172.21.15.7:0/2765049384 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:01:25.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.086+0000 7f88377fe700 1 --2- 172.21.15.7:0/2765049384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8838101fd0 0x7f883819ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:25.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.086+0000 7f883cfdb700 1 --2- 172.21.15.7:0/2765049384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f88381066b0 0x7f8838198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:25.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.086+0000 7f8837fff700 1 -- 172.21.15.7:0/2765049384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8838101fd0 msgr2=0x7f883819ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:25.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.086+0000 7f8837fff700 1 --2- 172.21.15.7:0/2765049384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8838101fd0 0x7f883819ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.086+0000 7f8837fff700 1 -- 172.21.15.7:0/2765049384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f88381066b0 msgr2=0x7f8838198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:25.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.086+0000 7f8837fff700 1 --2- 172.21.15.7:0/2765049384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f88381066b0 0x7f8838198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.086+0000 7f8837fff700 1 -- 172.21.15.7:0/2765049384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88381991c0 con 0x7f8838101620 2024-06-10T01:01:25.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.086+0000 7f883cfdb700 1 --2- 172.21.15.7:0/2765049384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f88381066b0 0x7f8838198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:01:25.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.087+0000 7f8837fff700 1 --2- 172.21.15.7:0/2765049384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8838101620 0x7f883819e8a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f882800d950 tx=0x7f882800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:25.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.087+0000 7f88357fa700 1 -- 172.21.15.7:0/2765049384 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828016070 con 0x7f8838101620 2024-06-10T01:01:25.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.087+0000 7f883ea3e700 1 -- 172.21.15.7:0/2765049384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88381994b0 con 0x7f8838101620 2024-06-10T01:01:25.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.087+0000 7f88357fa700 1 -- 172.21.15.7:0/2765049384 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8828009b40 con 0x7f8838101620 2024-06-10T01:01:25.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.087+0000 7f88357fa700 1 -- 172.21.15.7:0/2765049384 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882800f620 con 0x7f8838101620 2024-06-10T01:01:25.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.087+0000 7f883ea3e700 1 -- 172.21.15.7:0/2765049384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88381a5770 con 0x7f8838101620 2024-06-10T01:01:25.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.088+0000 7f883ea3e700 1 -- 172.21.15.7:0/2765049384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8838069180 con 0x7f8838101620 2024-06-10T01:01:25.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.090+0000 7f88357fa700 1 -- 172.21.15.7:0/2765049384 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f8828019070 con 0x7f8838101620 2024-06-10T01:01:25.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.090+0000 7f88357fa700 1 --2- 172.21.15.7:0/2765049384 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f88200a2c20 0x7f88200a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:25.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.090+0000 7f88377fe700 1 --2- 172.21.15.7:0/2765049384 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f88200a2c20 0x7f88200a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:25.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.090+0000 7f88357fa700 1 -- 172.21.15.7:0/2765049384 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f88280ceca0 con 0x7f8838101620 2024-06-10T01:01:25.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.091+0000 7f88377fe700 1 --2- 172.21.15.7:0/2765049384 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f88200a2c20 0x7f88200a50e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f882c005950 tx=0x7f882c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:25.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.094+0000 7f88357fa700 1 -- 172.21.15.7:0/2765049384 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f882809c0a0 con 0x7f8838101620 2024-06-10T01:01:25.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.350+0000 7f883ea3e700 1 -- 172.21.15.7:0/2765049384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8838199f90 con 0x7f8838101620 2024-06-10T01:01:25.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.350+0000 7f88357fa700 1 -- 172.21.15.7:0/2765049384 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7995 (secure 0 0 0) 0x7f882809b9a0 con 0x7f8838101620 2024-06-10T01:01:25.349 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:01:25.349 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2784910673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2784910673},{"type":"v1","addr":"172.21.15.175:6835","nonce":2784910673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24451,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-06-10T01:01:25.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.352+0000 7f883ea3e700 1 -- 172.21.15.7:0/2765049384 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f88200a2c20 msgr2=0x7f88200a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:25.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.352+0000 7f883ea3e700 1 --2- 172.21.15.7:0/2765049384 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f88200a2c20 0x7f88200a50e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f882c005950 tx=0x7f882c0058e0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.352+0000 7f883ea3e700 1 -- 172.21.15.7:0/2765049384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8838101620 msgr2=0x7f883819e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:25.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.352+0000 7f883ea3e700 1 --2- 172.21.15.7:0/2765049384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8838101620 0x7f883819e8a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f882800d950 tx=0x7f882800dd10 comp rx=0 tx=0).stop 2024-06-10T01:01:25.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.352+0000 7f883ea3e700 1 -- 172.21.15.7:0/2765049384 shutdown_connections 2024-06-10T01:01:25.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.352+0000 7f883ea3e700 1 --2- 172.21.15.7:0/2765049384 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f88200a2c20 0x7f88200a50e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.352+0000 7f883ea3e700 1 --2- 172.21.15.7:0/2765049384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f88381066b0 0x7f8838198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.352+0000 7f883ea3e700 1 --2- 172.21.15.7:0/2765049384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8838101620 0x7f883819e8a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.352+0000 7f883ea3e700 1 --2- 172.21.15.7:0/2765049384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8838101fd0 0x7f883819ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.352+0000 7f883ea3e700 1 -- 172.21.15.7:0/2765049384 >> 172.21.15.7:0/2765049384 conn(0x7f8838078340 msgr2=0x7f883806aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:25.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.352+0000 7f883ea3e700 1 -- 172.21.15.7:0/2765049384 shutdown_connections 2024-06-10T01:01:25.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:25.352+0000 7f883ea3e700 1 -- 172.21.15.7:0/2765049384 wait complete. 2024-06-10T01:01:25.352 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 10 2024-06-10T01:01:25.516 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:25 smithi007 ceph-mon[32398]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:25.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:25 smithi078 ceph-mon[36683]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:25.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:25 smithi175 ceph-mon[36636]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:26.234 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 10 01:01:26 smithi007 systemd[1]: Starting Ceph mds.g for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:01:26.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:26 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2765049384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:01:26.517 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 10 01:01:26 smithi007 podman[74529]: 2024-06-10 01:01:26.374941872 +0000 UTC m=+0.018289924 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:01:26.517 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 10 01:01:26 smithi007 podman[74529]: 2024-06-10 01:01:26.478732582 +0000 UTC m=+0.122080622 container create 1a6ae155a59e721254e1b367535013e8d7b2f62f8a5014aa12e9ac7eb38707d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-g, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, ceph=True, maintainer=Guillaume Abrioux ) 2024-06-10T01:01:26.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:26 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2765049384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:01:26.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:26 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2765049384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:01:27.017 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 10 01:01:26 smithi007 podman[74529]: 2024-06-10 01:01:26.731948497 +0000 UTC m=+0.375296535 container init 1a6ae155a59e721254e1b367535013e8d7b2f62f8a5014aa12e9ac7eb38707d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-g, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-06-10T01:01:27.017 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 10 01:01:26 smithi007 podman[74529]: 2024-06-10 01:01:26.735425592 +0000 UTC m=+0.378773636 container start 1a6ae155a59e721254e1b367535013e8d7b2f62f8a5014aa12e9ac7eb38707d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-g, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T01:01:27.017 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 10 01:01:26 smithi007 ceph-mds[74548]: -- 172.21.15.7:0/1991247959 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fbdf05da40 con 0x55fbdf030800 2024-06-10T01:01:27.017 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 10 01:01:26 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-g[74544]: starting mds.g at 2024-06-10T01:01:27.017 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 10 01:01:26 smithi007 bash[74529]: 1a6ae155a59e721254e1b367535013e8d7b2f62f8a5014aa12e9ac7eb38707d0 2024-06-10T01:01:27.017 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 10 01:01:26 smithi007 systemd[1]: Started Ceph mds.g for bded58f8-26c3-11ef-bc9b-c7b262605968. 2024-06-10T01:01:27.568 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:27 smithi007 ceph-mon[32398]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:27.568 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:27 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:27.568 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:27 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:27.568 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:27 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:27.568 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:27 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:01:27.568 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:27 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:01:27.568 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:27 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:27.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:27 smithi078 ceph-mon[36683]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:27.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:27 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:27.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:27 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:27.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:27 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:27.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:27 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:01:27.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:27 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:01:27.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:27 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:27.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:27 smithi175 ceph-mon[36636]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:27.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:27 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:27.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:27 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:27.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:27 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:27.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:27 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:01:27.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:27 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:01:27.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:27 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:28.261 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:01:28.456 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:28 smithi175 ceph-mon[36636]: Deploying daemon mds.a on smithi007 2024-06-10T01:01:28.456 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:28 smithi175 ceph-mon[36636]: mds.? [v2:172.21.15.7:6838/159652731,v1:172.21.15.7:6839/159652731] up:boot 2024-06-10T01:01:28.456 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:28 smithi175 ceph-mon[36636]: fsmap 10 up:standby 2024-06-10T01:01:28.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:28 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-10T01:01:28.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:28 smithi078 ceph-mon[36683]: Deploying daemon mds.a on smithi007 2024-06-10T01:01:28.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:28 smithi078 ceph-mon[36683]: mds.? [v2:172.21.15.7:6838/159652731,v1:172.21.15.7:6839/159652731] up:boot 2024-06-10T01:01:28.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:28 smithi078 ceph-mon[36683]: fsmap 10 up:standby 2024-06-10T01:01:28.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:28 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-10T01:01:28.583 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:28 smithi007 ceph-mon[32398]: Deploying daemon mds.a on smithi007 2024-06-10T01:01:28.584 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:28 smithi007 ceph-mon[32398]: mds.? [v2:172.21.15.7:6838/159652731,v1:172.21.15.7:6839/159652731] up:boot 2024-06-10T01:01:28.584 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:28 smithi007 ceph-mon[32398]: fsmap 10 up:standby 2024-06-10T01:01:28.584 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:28 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-10T01:01:28.693 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:01:29.093 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 10 01:01:29 smithi007 systemd[1]: Starting Ceph mds.a for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:01:29.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:29 smithi007 ceph-mon[32398]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:29.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:29 smithi078 ceph-mon[36683]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:29.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:29 smithi175 ceph-mon[36636]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:30.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.505+0000 7fe85fe42700 1 -- 172.21.15.7:0/2356192127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe858107840 msgr2=0x7fe85810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:30.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.505+0000 7fe85fe42700 1 --2- 172.21.15.7:0/2356192127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe858107840 0x7fe85810f3c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe8500099c0 tx=0x7fe850009cd0 comp rx=0 tx=0).stop 2024-06-10T01:01:30.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.505+0000 7fe85fe42700 1 -- 172.21.15.7:0/2356192127 shutdown_connections 2024-06-10T01:01:30.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.505+0000 7fe85fe42700 1 --2- 172.21.15.7:0/2356192127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe858107840 0x7fe85810f3c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:30.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.505+0000 7fe85fe42700 1 --2- 172.21.15.7:0/2356192127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe858102740 0x7fe858102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:30.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.505+0000 7fe85fe42700 1 --2- 172.21.15.7:0/2356192127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8581030f0 0x7fe858107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:30.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.505+0000 7fe85fe42700 1 -- 172.21.15.7:0/2356192127 >> 172.21.15.7:0/2356192127 conn(0x7fe8580795f0 msgr2=0x7fe858079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:30.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.505+0000 7fe85fe42700 1 -- 172.21.15.7:0/2356192127 shutdown_connections 2024-06-10T01:01:30.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.506+0000 7fe85fe42700 1 -- 172.21.15.7:0/2356192127 wait complete. 2024-06-10T01:01:30.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.506+0000 7fe85fe42700 1 Processor -- start 2024-06-10T01:01:30.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.506+0000 7fe85fe42700 1 -- start start 2024-06-10T01:01:30.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.506+0000 7fe85fe42700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe858102740 0x7fe8581a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:30.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.506+0000 7fe85fe42700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8581030f0 0x7fe8581a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:30.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.506+0000 7fe85fe42700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe858107840 0x7fe85819ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:30.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.506+0000 7fe85fe42700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe858075350 con 0x7fe858102740 2024-06-10T01:01:30.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.506+0000 7fe85fe42700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe8580751d0 con 0x7fe858107840 2024-06-10T01:01:30.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.506+0000 7fe85fe42700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe8580754d0 con 0x7fe8581030f0 2024-06-10T01:01:30.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.507+0000 7fe85dbde700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe858102740 0x7fe8581a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:30.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.507+0000 7fe85dbde700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe858102740 0x7fe8581a08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34722/0 (socket says 172.21.15.7:34722) 2024-06-10T01:01:30.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.507+0000 7fe85dbde700 1 -- 172.21.15.7:0/3740065505 learned_addr learned my addr 172.21.15.7:0/3740065505 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:01:30.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.507+0000 7fe85d3dd700 1 --2- 172.21.15.7:0/3740065505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8581030f0 0x7fe8581a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:30.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.507+0000 7fe85e3df700 1 --2- 172.21.15.7:0/3740065505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe858107840 0x7fe85819ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:30.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.507+0000 7fe85dbde700 1 -- 172.21.15.7:0/3740065505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8581030f0 msgr2=0x7fe8581a0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:30.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.507+0000 7fe85dbde700 1 --2- 172.21.15.7:0/3740065505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8581030f0 0x7fe8581a0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:30.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.507+0000 7fe85dbde700 1 -- 172.21.15.7:0/3740065505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe858107840 msgr2=0x7fe85819ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:30.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.507+0000 7fe85dbde700 1 --2- 172.21.15.7:0/3740065505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe858107840 0x7fe85819ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:30.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.507+0000 7fe85dbde700 1 -- 172.21.15.7:0/3740065505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe85819b370 con 0x7fe858102740 2024-06-10T01:01:30.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.507+0000 7fe85e3df700 1 --2- 172.21.15.7:0/3740065505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe858107840 0x7fe85819ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:01:30.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.507+0000 7fe85dbde700 1 --2- 172.21.15.7:0/3740065505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe858102740 0x7fe8581a08f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe85400ef30 tx=0x7fe85400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:30.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.507+0000 7fe84effd700 1 -- 172.21.15.7:0/3740065505 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8540092e0 con 0x7fe858102740 2024-06-10T01:01:30.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.507+0000 7fe84effd700 1 -- 172.21.15.7:0/3740065505 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe854004bb0 con 0x7fe858102740 2024-06-10T01:01:30.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.507+0000 7fe85fe42700 1 -- 172.21.15.7:0/3740065505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe85819b600 con 0x7fe858102740 2024-06-10T01:01:30.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.507+0000 7fe84effd700 1 -- 172.21.15.7:0/3740065505 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe854004280 con 0x7fe858102740 2024-06-10T01:01:30.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.507+0000 7fe85fe42700 1 -- 172.21.15.7:0/3740065505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8581a77c0 con 0x7fe858102740 2024-06-10T01:01:30.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.508+0000 7fe85fe42700 1 -- 172.21.15.7:0/3740065505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe858069180 con 0x7fe858102740 2024-06-10T01:01:30.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.510+0000 7fe84effd700 1 -- 172.21.15.7:0/3740065505 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fe85401b030 con 0x7fe858102740 2024-06-10T01:01:30.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.511+0000 7fe84effd700 1 --2- 172.21.15.7:0/3740065505 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe8440a2c20 0x7fe8440a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:30.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.511+0000 7fe85d3dd700 1 --2- 172.21.15.7:0/3740065505 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe8440a2c20 0x7fe8440a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:30.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.511+0000 7fe84effd700 1 -- 172.21.15.7:0/3740065505 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe8540cf2b0 con 0x7fe858102740 2024-06-10T01:01:30.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.511+0000 7fe85d3dd700 1 --2- 172.21.15.7:0/3740065505 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe8440a2c20 0x7fe8440a50e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe858102070 tx=0x7fe84800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:30.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.514+0000 7fe84effd700 1 -- 172.21.15.7:0/3740065505 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe854022020 con 0x7fe858102740 2024-06-10T01:01:30.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.769+0000 7fe85fe42700 1 -- 172.21.15.7:0/3740065505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8581a79c0 con 0x7fe858102740 2024-06-10T01:01:30.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.770+0000 7fe84effd700 1 -- 172.21.15.7:0/3740065505 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8815 (secure 0 0 0) 0x7fe854010040 con 0x7fe858102740 2024-06-10T01:01:30.773 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:01:30.773 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2784910673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2784910673},{"type":"v1","addr":"172.21.15.175:6835","nonce":2784910673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24451,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-06-10T01:01:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.772+0000 7fe85fe42700 1 -- 172.21.15.7:0/3740065505 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe8440a2c20 msgr2=0x7fe8440a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.772+0000 7fe85fe42700 1 --2- 172.21.15.7:0/3740065505 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe8440a2c20 0x7fe8440a50e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe858102070 tx=0x7fe84800a380 comp rx=0 tx=0).stop 2024-06-10T01:01:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.772+0000 7fe85fe42700 1 -- 172.21.15.7:0/3740065505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe858102740 msgr2=0x7fe8581a08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.772+0000 7fe85fe42700 1 --2- 172.21.15.7:0/3740065505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe858102740 0x7fe8581a08f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe85400ef30 tx=0x7fe85400c3a0 comp rx=0 tx=0).stop 2024-06-10T01:01:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.772+0000 7fe85fe42700 1 -- 172.21.15.7:0/3740065505 shutdown_connections 2024-06-10T01:01:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.772+0000 7fe85fe42700 1 --2- 172.21.15.7:0/3740065505 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe8440a2c20 0x7fe8440a50e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.772+0000 7fe85fe42700 1 --2- 172.21.15.7:0/3740065505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe858107840 0x7fe85819ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.772+0000 7fe85fe42700 1 --2- 172.21.15.7:0/3740065505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe858102740 0x7fe8581a08f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.772+0000 7fe85fe42700 1 --2- 172.21.15.7:0/3740065505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8581030f0 0x7fe8581a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.772+0000 7fe85fe42700 1 -- 172.21.15.7:0/3740065505 >> 172.21.15.7:0/3740065505 conn(0x7fe8580795f0 msgr2=0x7fe8580ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.773+0000 7fe85fe42700 1 -- 172.21.15.7:0/3740065505 shutdown_connections 2024-06-10T01:01:30.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:30.773+0000 7fe85fe42700 1 -- 172.21.15.7:0/3740065505 wait complete. 2024-06-10T01:01:30.779 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 11 2024-06-10T01:01:30.892 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 10 01:01:30 smithi007 podman[74952]: 2024-06-10 01:01:30.641773384 +0000 UTC m=+0.342752357 container create b38668170db2a3e6c7e4d008712d7efe4c54d431fd3e4e1bf136aa07e55fa026 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528) 2024-06-10T01:01:30.892 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 10 01:01:30 smithi007 podman[74952]: 2024-06-10 01:01:30.546987066 +0000 UTC m=+0.247966052 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:01:30.892 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 10 01:01:30 smithi007 podman[74952]: 2024-06-10 01:01:30.894808446 +0000 UTC m=+0.595787435 container init b38668170db2a3e6c7e4d008712d7efe4c54d431fd3e4e1bf136aa07e55fa026 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, ceph=True) 2024-06-10T01:01:31.218 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 10 01:01:30 smithi007 podman[74952]: 2024-06-10 01:01:30.898255977 +0000 UTC m=+0.599234957 container start b38668170db2a3e6c7e4d008712d7efe4c54d431fd3e4e1bf136aa07e55fa026 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-a, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0) 2024-06-10T01:01:31.218 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 10 01:01:30 smithi007 ceph-mds[75007]: -- 172.21.15.7:0/1290196069 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564c6140fa40 con 0x564c613e3000 2024-06-10T01:01:31.218 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 10 01:01:30 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-a[74989]: starting mds.a at 2024-06-10T01:01:31.218 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 10 01:01:30 smithi007 bash[74952]: b38668170db2a3e6c7e4d008712d7efe4c54d431fd3e4e1bf136aa07e55fa026 2024-06-10T01:01:31.218 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 10 01:01:30 smithi007 systemd[1]: Started Ceph mds.a for bded58f8-26c3-11ef-bc9b-c7b262605968. 2024-06-10T01:01:31.516 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:31 smithi007 ceph-mon[32398]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:31.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:31 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3740065505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:01:31.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:31 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:31.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:31 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:31.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:31 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:31.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:31 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:01:31.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:31 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:01:31.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:31 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:31.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:31 smithi078 ceph-mon[36683]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:31.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:31 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3740065505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:01:31.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:31 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:31.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:31 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:31.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:31 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:31.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:31 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:01:31.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:31 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:01:31.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:31 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:31.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:31 smithi175 ceph-mon[36636]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:31.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:31 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3740065505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:01:31.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:31 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:31.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:31 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:31.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:31 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:31.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:31 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T01:01:31.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:31 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T01:01:31.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:31 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:32.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:32 smithi078 ceph-mon[36683]: Deploying daemon mds.b on smithi078 2024-06-10T01:01:32.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:32 smithi078 ceph-mon[36683]: mds.? [v2:172.21.15.7:6840/1788930795,v1:172.21.15.7:6841/1788930795] up:boot 2024-06-10T01:01:32.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:32 smithi078 ceph-mon[36683]: fsmap 11 up:standby 2024-06-10T01:01:32.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:32 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-10T01:01:32.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:32 smithi175 ceph-mon[36636]: Deploying daemon mds.b on smithi078 2024-06-10T01:01:32.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:32 smithi175 ceph-mon[36636]: mds.? [v2:172.21.15.7:6840/1788930795,v1:172.21.15.7:6841/1788930795] up:boot 2024-06-10T01:01:32.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:32 smithi175 ceph-mon[36636]: fsmap 11 up:standby 2024-06-10T01:01:32.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:32 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-10T01:01:32.766 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:32 smithi007 ceph-mon[32398]: Deploying daemon mds.b on smithi078 2024-06-10T01:01:32.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:32 smithi007 ceph-mon[32398]: mds.? [v2:172.21.15.7:6840/1788930795,v1:172.21.15.7:6841/1788930795] up:boot 2024-06-10T01:01:32.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:32 smithi007 ceph-mon[32398]: fsmap 11 up:standby 2024-06-10T01:01:32.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:32 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-10T01:01:33.484 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:33 smithi078 ceph-mon[36683]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:33.516 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:33 smithi007 ceph-mon[32398]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:33.643 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:01:33.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:33 smithi175 ceph-mon[36636]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:33.899 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:01:34.083 INFO:journalctl@ceph.mds.b.smithi078.stdout:Jun 10 01:01:33 smithi078 systemd[1]: Started Ceph mds.b for bded58f8-26c3-11ef-bc9b-c7b262605968. 2024-06-10T01:01:35.171 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:34 smithi078 ceph-mon[36683]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:35.171 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:34 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:35.171 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:34 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:35.171 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:34 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:35.171 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:34 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:35.172 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:34 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:35.172 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:34 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:01:35.172 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:34 smithi078 ceph-mon[36683]: mds.? [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] up:boot 2024-06-10T01:01:35.172 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:34 smithi078 ceph-mon[36683]: fsmap 12 up:standby 2024-06-10T01:01:35.172 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:34 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-10T01:01:35.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:34 smithi007 ceph-mon[32398]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:35.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:34 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:35.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:34 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:35.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:34 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:35.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:34 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:35.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:34 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:35.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:34 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:01:35.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:34 smithi007 ceph-mon[32398]: mds.? [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] up:boot 2024-06-10T01:01:35.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:34 smithi007 ceph-mon[32398]: fsmap 12 up:standby 2024-06-10T01:01:35.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:34 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-10T01:01:35.206 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:34 smithi175 ceph-mon[36636]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:35.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:34 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:35.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:34 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:35.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:34 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:35.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:34 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:35.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:34 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:35.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:34 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:01:35.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:34 smithi175 ceph-mon[36636]: mds.? [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] up:boot 2024-06-10T01:01:35.207 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:34 smithi175 ceph-mon[36636]: fsmap 12 up:standby 2024-06-10T01:01:35.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:34 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-10T01:01:35.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.573+0000 7fd7da7be700 1 -- 172.21.15.7:0/637818574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7d4102e80 msgr2=0x7fd7d410f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:35.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.573+0000 7fd7da7be700 1 --2- 172.21.15.7:0/637818574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7d4102e80 0x7fd7d410f330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc00b0a0 tx=0x7fd7cc00b3b0 comp rx=0 tx=0).stop 2024-06-10T01:01:35.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.573+0000 7fd7da7be700 1 -- 172.21.15.7:0/637818574 shutdown_connections 2024-06-10T01:01:35.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.573+0000 7fd7da7be700 1 --2- 172.21.15.7:0/637818574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7d4102e80 0x7fd7d410f330 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:35.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.573+0000 7fd7da7be700 1 --2- 172.21.15.7:0/637818574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7d41084f0 0x7fd7d41088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:35.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.573+0000 7fd7da7be700 1 --2- 172.21.15.7:0/637818574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7d41024e0 0x7fd7d4102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:35.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.573+0000 7fd7da7be700 1 -- 172.21.15.7:0/637818574 >> 172.21.15.7:0/637818574 conn(0x7fd7d4079530 msgr2=0x7fd7d4079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:35.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.573+0000 7fd7da7be700 1 -- 172.21.15.7:0/637818574 shutdown_connections 2024-06-10T01:01:35.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.573+0000 7fd7da7be700 1 -- 172.21.15.7:0/637818574 wait complete. 2024-06-10T01:01:35.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.574+0000 7fd7da7be700 1 Processor -- start 2024-06-10T01:01:35.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.574+0000 7fd7da7be700 1 -- start start 2024-06-10T01:01:35.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.574+0000 7fd7da7be700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7d41024e0 0x7fd7d419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:35.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.574+0000 7fd7da7be700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7d4102e80 0x7fd7d419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:35.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.574+0000 7fd7da7be700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7d41084f0 0x7fd7d41989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:35.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.574+0000 7fd7da7be700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d4112770 con 0x7fd7d41084f0 2024-06-10T01:01:35.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.574+0000 7fd7da7be700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d41125f0 con 0x7fd7d4102e80 2024-06-10T01:01:35.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.574+0000 7fd7da7be700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d41128f0 con 0x7fd7d41024e0 2024-06-10T01:01:35.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.574+0000 7fd7d8d5b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7d41084f0 0x7fd7d41989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:35.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.574+0000 7fd7d3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7d41024e0 0x7fd7d419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:35.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.575+0000 7fd7d3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7d41024e0 0x7fd7d419e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:41392/0 (socket says 172.21.15.7:41392) 2024-06-10T01:01:35.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.575+0000 7fd7d8d5b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7d41084f0 0x7fd7d41989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38590/0 (socket says 172.21.15.7:38590) 2024-06-10T01:01:35.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.575+0000 7fd7d3fff700 1 -- 172.21.15.7:0/1236123179 learned_addr learned my addr 172.21.15.7:0/1236123179 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:01:35.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.575+0000 7fd7d37fe700 1 --2- 172.21.15.7:0/1236123179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7d4102e80 0x7fd7d419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:35.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.575+0000 7fd7d3fff700 1 -- 172.21.15.7:0/1236123179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7d4102e80 msgr2=0x7fd7d419ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:35.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.575+0000 7fd7d3fff700 1 --2- 172.21.15.7:0/1236123179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7d4102e80 0x7fd7d419ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:35.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.575+0000 7fd7d3fff700 1 -- 172.21.15.7:0/1236123179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7d41084f0 msgr2=0x7fd7d41989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:35.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.575+0000 7fd7d3fff700 1 --2- 172.21.15.7:0/1236123179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7d41084f0 0x7fd7d41989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:35.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.575+0000 7fd7d3fff700 1 -- 172.21.15.7:0/1236123179 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d4199150 con 0x7fd7d41024e0 2024-06-10T01:01:35.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.575+0000 7fd7d8d5b700 1 --2- 172.21.15.7:0/1236123179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7d41084f0 0x7fd7d41989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:01:35.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.575+0000 7fd7d3fff700 1 --2- 172.21.15.7:0/1236123179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7d41024e0 0x7fd7d419e830 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400d980 tx=0x7fd7c400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:35.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.575+0000 7fd7d17fa700 1 -- 172.21.15.7:0/1236123179 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4016070 con 0x7fd7d41024e0 2024-06-10T01:01:35.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.575+0000 7fd7d17fa700 1 -- 172.21.15.7:0/1236123179 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7c4004d10 con 0x7fd7d41024e0 2024-06-10T01:01:35.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.575+0000 7fd7da7be700 1 -- 172.21.15.7:0/1236123179 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d4199440 con 0x7fd7d41024e0 2024-06-10T01:01:35.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.575+0000 7fd7d17fa700 1 -- 172.21.15.7:0/1236123179 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4005670 con 0x7fd7d41024e0 2024-06-10T01:01:35.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.575+0000 7fd7da7be700 1 -- 172.21.15.7:0/1236123179 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d40746c0 con 0x7fd7d41024e0 2024-06-10T01:01:35.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.576+0000 7fd7da7be700 1 -- 172.21.15.7:0/1236123179 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7d4069180 con 0x7fd7d41024e0 2024-06-10T01:01:35.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.578+0000 7fd7d17fa700 1 -- 172.21.15.7:0/1236123179 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fd7c4004750 con 0x7fd7d41024e0 2024-06-10T01:01:35.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.578+0000 7fd7d17fa700 1 --2- 172.21.15.7:0/1236123179 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd7bc0a2ca0 0x7fd7bc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:35.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.579+0000 7fd7d37fe700 1 --2- 172.21.15.7:0/1236123179 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd7bc0a2ca0 0x7fd7bc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:35.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.579+0000 7fd7d17fa700 1 -- 172.21.15.7:0/1236123179 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fd7c40ceb60 con 0x7fd7d41024e0 2024-06-10T01:01:35.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.579+0000 7fd7d37fe700 1 --2- 172.21.15.7:0/1236123179 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd7bc0a2ca0 0x7fd7bc0a5160 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4079ae0 tx=0x7fd7c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:35.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.582+0000 7fd7d17fa700 1 -- 172.21.15.7:0/1236123179 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd7c401f020 con 0x7fd7d41024e0 2024-06-10T01:01:35.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.879+0000 7fd7da7be700 1 -- 172.21.15.7:0/1236123179 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7d402dca0 con 0x7fd7d41024e0 2024-06-10T01:01:35.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.880+0000 7fd7d17fa700 1 -- 172.21.15.7:0/1236123179 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10464 (secure 0 0 0) 0x7fd7c409bec0 con 0x7fd7d41024e0 2024-06-10T01:01:35.879 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:01:35.879 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2784910673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2784910673},{"type":"v1","addr":"172.21.15.175:6835","nonce":2784910673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24434,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24451,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-06-10T01:01:35.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.882+0000 7fd7da7be700 1 -- 172.21.15.7:0/1236123179 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd7bc0a2ca0 msgr2=0x7fd7bc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:35.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.882+0000 7fd7da7be700 1 --2- 172.21.15.7:0/1236123179 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd7bc0a2ca0 0x7fd7bc0a5160 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4079ae0 tx=0x7fd7c8009260 comp rx=0 tx=0).stop 2024-06-10T01:01:35.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.882+0000 7fd7da7be700 1 -- 172.21.15.7:0/1236123179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7d41024e0 msgr2=0x7fd7d419e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:35.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.882+0000 7fd7da7be700 1 --2- 172.21.15.7:0/1236123179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7d41024e0 0x7fd7d419e830 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400d980 tx=0x7fd7c400dc90 comp rx=0 tx=0).stop 2024-06-10T01:01:35.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.883+0000 7fd7da7be700 1 -- 172.21.15.7:0/1236123179 shutdown_connections 2024-06-10T01:01:35.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.883+0000 7fd7da7be700 1 --2- 172.21.15.7:0/1236123179 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd7bc0a2ca0 0x7fd7bc0a5160 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:35.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.883+0000 7fd7da7be700 1 --2- 172.21.15.7:0/1236123179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7d41084f0 0x7fd7d41989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:35.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.883+0000 7fd7da7be700 1 --2- 172.21.15.7:0/1236123179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7d41024e0 0x7fd7d419e830 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:35.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.883+0000 7fd7da7be700 1 --2- 172.21.15.7:0/1236123179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7d4102e80 0x7fd7d419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:35.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.883+0000 7fd7da7be700 1 -- 172.21.15.7:0/1236123179 >> 172.21.15.7:0/1236123179 conn(0x7fd7d4079530 msgr2=0x7fd7d41004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:35.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.883+0000 7fd7da7be700 1 -- 172.21.15.7:0/1236123179 shutdown_connections 2024-06-10T01:01:35.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:35.883+0000 7fd7da7be700 1 -- 172.21.15.7:0/1236123179 wait complete. 2024-06-10T01:01:35.883 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 13 2024-06-10T01:01:36.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:36.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:36 smithi007 ceph-mon[32398]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:36.770 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:36.770 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:36.770 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:36.770 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:36.770 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:36 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1236123179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:01:36.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:36.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:36 smithi078 ceph-mon[36683]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:36.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:36.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:36.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:36.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:36.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:36 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1236123179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:01:36.841 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-06-10T01:01:36.956 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:36.956 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:36 smithi175 ceph-mon[36636]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:36.956 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:36.956 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:36.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:36.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:36.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:36 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1236123179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:01:37.109 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:01:38.706 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:38 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:38.706 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:38 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:38.706 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:38 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:38.706 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:38 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:01:38.706 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:38 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:38.706 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:38 smithi007 ceph-mon[32398]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:38.706 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:38 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:01:38.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:38 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:38.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:38 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:38.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:38 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:38.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:38 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:01:38.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:38 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:38.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:38 smithi175 ceph-mon[36636]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:38.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:38 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:01:38.720 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:38 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:38.720 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:38 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:38.720 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:38 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:38.720 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:38 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:01:38.721 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:38 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:38.721 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:38 smithi078 ceph-mon[36683]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:38.721 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:38 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:01:38.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.731+0000 7fdaf385d700 1 -- 172.21.15.7:0/1497698329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdaec1030f0 msgr2=0x7fdaec107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:38.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.731+0000 7fdaf385d700 1 --2- 172.21.15.7:0/1497698329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdaec1030f0 0x7fdaec107110 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0099c0 tx=0x7fdadc009cd0 comp rx=0 tx=0).stop 2024-06-10T01:01:38.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.731+0000 7fdaf385d700 1 -- 172.21.15.7:0/1497698329 shutdown_connections 2024-06-10T01:01:38.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.731+0000 7fdaf385d700 1 --2- 172.21.15.7:0/1497698329 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdaec107840 0x7fdaec10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:38.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.731+0000 7fdaf385d700 1 --2- 172.21.15.7:0/1497698329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdaec102740 0x7fdaec102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:38.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.731+0000 7fdaf385d700 1 --2- 172.21.15.7:0/1497698329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdaec1030f0 0x7fdaec107110 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:38.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.731+0000 7fdaf385d700 1 -- 172.21.15.7:0/1497698329 >> 172.21.15.7:0/1497698329 conn(0x7fdaec0795f0 msgr2=0x7fdaec079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:38.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.732+0000 7fdaf385d700 1 -- 172.21.15.7:0/1497698329 shutdown_connections 2024-06-10T01:01:38.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.732+0000 7fdaf385d700 1 -- 172.21.15.7:0/1497698329 wait complete. 2024-06-10T01:01:38.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.732+0000 7fdaf385d700 1 Processor -- start 2024-06-10T01:01:38.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.732+0000 7fdaf385d700 1 -- start start 2024-06-10T01:01:38.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.733+0000 7fdaf385d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdaec102740 0x7fdaec074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:38.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.733+0000 7fdaf385d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdaec1030f0 0x7fdaec074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:38.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.733+0000 7fdaf385d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdaec107840 0x7fdaec075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:38.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.733+0000 7fdaf385d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdaec112770 con 0x7fdaec107840 2024-06-10T01:01:38.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.733+0000 7fdaf385d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdaec1125f0 con 0x7fdaec1030f0 2024-06-10T01:01:38.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.733+0000 7fdaf385d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdaec1128f0 con 0x7fdaec102740 2024-06-10T01:01:38.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.733+0000 7fdaf1dfa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdaec107840 0x7fdaec075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:38.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.733+0000 7fdaf15f9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdaec102740 0x7fdaec074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:38.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.733+0000 7fdaf1dfa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdaec107840 0x7fdaec075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38600/0 (socket says 172.21.15.7:38600) 2024-06-10T01:01:38.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.733+0000 7fdaf1dfa700 1 -- 172.21.15.7:0/1459005882 learned_addr learned my addr 172.21.15.7:0/1459005882 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:01:38.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.733+0000 7fdaf0df8700 1 --2- 172.21.15.7:0/1459005882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdaec1030f0 0x7fdaec074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:38.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.733+0000 7fdaf15f9700 1 -- 172.21.15.7:0/1459005882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdaec1030f0 msgr2=0x7fdaec074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:38.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.733+0000 7fdaf15f9700 1 --2- 172.21.15.7:0/1459005882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdaec1030f0 0x7fdaec074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:38.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.733+0000 7fdaf15f9700 1 -- 172.21.15.7:0/1459005882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdaec107840 msgr2=0x7fdaec075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:38.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.733+0000 7fdaf15f9700 1 --2- 172.21.15.7:0/1459005882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdaec107840 0x7fdaec075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:38.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.733+0000 7fdaf15f9700 1 -- 172.21.15.7:0/1459005882 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaec1a7110 con 0x7fdaec102740 2024-06-10T01:01:38.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.733+0000 7fdaf1dfa700 1 --2- 172.21.15.7:0/1459005882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdaec107840 0x7fdaec075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:01:38.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.733+0000 7fdaf0df8700 1 --2- 172.21.15.7:0/1459005882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdaec1030f0 0x7fdaec074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:01:38.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.734+0000 7fdaf15f9700 1 --2- 172.21.15.7:0/1459005882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdaec102740 0x7fdaec074790 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdae800d750 tx=0x7fdae800da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:38.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.734+0000 7fdae27fc700 1 -- 172.21.15.7:0/1459005882 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8016070 con 0x7fdaec102740 2024-06-10T01:01:38.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.734+0000 7fdae27fc700 1 -- 172.21.15.7:0/1459005882 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdae8004d10 con 0x7fdaec102740 2024-06-10T01:01:38.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.734+0000 7fdaf385d700 1 -- 172.21.15.7:0/1459005882 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaec1a7400 con 0x7fdaec102740 2024-06-10T01:01:38.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.734+0000 7fdae27fc700 1 -- 172.21.15.7:0/1459005882 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae800f690 con 0x7fdaec102740 2024-06-10T01:01:38.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.734+0000 7fdaf385d700 1 -- 172.21.15.7:0/1459005882 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaec1a7960 con 0x7fdaec102740 2024-06-10T01:01:38.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.735+0000 7fdaf385d700 1 -- 172.21.15.7:0/1459005882 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaec069180 con 0x7fdaec102740 2024-06-10T01:01:38.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.736+0000 7fdae27fc700 1 -- 172.21.15.7:0/1459005882 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fdae8004750 con 0x7fdaec102740 2024-06-10T01:01:38.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.737+0000 7fdae27fc700 1 --2- 172.21.15.7:0/1459005882 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdad80a2ca0 0x7fdad80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:38.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.737+0000 7fdaf0df8700 1 --2- 172.21.15.7:0/1459005882 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdad80a2ca0 0x7fdad80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:38.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.737+0000 7fdae27fc700 1 -- 172.21.15.7:0/1459005882 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fdae80ce8a0 con 0x7fdaec102740 2024-06-10T01:01:38.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.738+0000 7fdaf0df8700 1 --2- 172.21.15.7:0/1459005882 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdad80a2ca0 0x7fdad80a5160 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdaec076220 tx=0x7fdadc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:38.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.742+0000 7fdae27fc700 1 -- 172.21.15.7:0/1459005882 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdae801f020 con 0x7fdaec102740 2024-06-10T01:01:38.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.958+0000 7fdaf385d700 1 -- 172.21.15.7:0/1459005882 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fdaec075cc0 con 0x7fdaec102740 2024-06-10T01:01:38.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.959+0000 7fdae27fc700 1 -- 172.21.15.7:0/1459005882 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7fdae809bc00 con 0x7fdaec102740 2024-06-10T01:01:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:01:38.957 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-06-10T01:01:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:01:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-06-10T01:01:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-06-10T01:01:38.958 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:01:38.958 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-06-10T01:01:38.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.961+0000 7fdaf385d700 1 -- 172.21.15.7:0/1459005882 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdad80a2ca0 msgr2=0x7fdad80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:38.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.961+0000 7fdaf385d700 1 --2- 172.21.15.7:0/1459005882 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdad80a2ca0 0x7fdad80a5160 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdaec076220 tx=0x7fdadc005fd0 comp rx=0 tx=0).stop 2024-06-10T01:01:38.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.961+0000 7fdaf385d700 1 -- 172.21.15.7:0/1459005882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdaec102740 msgr2=0x7fdaec074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:38.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.961+0000 7fdaf385d700 1 --2- 172.21.15.7:0/1459005882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdaec102740 0x7fdaec074790 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdae800d750 tx=0x7fdae800da60 comp rx=0 tx=0).stop 2024-06-10T01:01:38.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.961+0000 7fdaf385d700 1 -- 172.21.15.7:0/1459005882 shutdown_connections 2024-06-10T01:01:38.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.961+0000 7fdaf385d700 1 --2- 172.21.15.7:0/1459005882 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdad80a2ca0 0x7fdad80a5160 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:38.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.961+0000 7fdaf385d700 1 --2- 172.21.15.7:0/1459005882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdaec107840 0x7fdaec075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:38.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.961+0000 7fdaf385d700 1 --2- 172.21.15.7:0/1459005882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdaec102740 0x7fdaec074790 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:38.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.961+0000 7fdaf385d700 1 --2- 172.21.15.7:0/1459005882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdaec1030f0 0x7fdaec074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:38.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.961+0000 7fdaf385d700 1 -- 172.21.15.7:0/1459005882 >> 172.21.15.7:0/1459005882 conn(0x7fdaec0795f0 msgr2=0x7fdaec100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:38.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.961+0000 7fdaf385d700 1 -- 172.21.15.7:0/1459005882 shutdown_connections 2024-06-10T01:01:38.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:38.961+0000 7fdaf385d700 1 -- 172.21.15.7:0/1459005882 wait complete. 2024-06-10T01:01:39.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:39 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1459005882' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-10T01:01:39.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:39 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:39.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:39 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:39.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:39 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:39.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:39 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:39.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:39 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1459005882' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-10T01:01:39.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:39 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:39.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:39 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:39.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:39 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:39.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:39 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:39.827 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-06-10T01:01:39.828 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-06-10T01:01:39.832 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:39 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1459005882' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-10T01:01:39.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:39 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:39.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:39 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:39.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:39 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:39.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:39 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:40.272 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:01:40.711 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:40 smithi007 ceph-mon[32398]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:40.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:40 smithi078 ceph-mon[36683]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:40.956 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:40 smithi175 ceph-mon[36636]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.885+0000 7f46049b5700 1 -- 172.21.15.7:0/2213477054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46001015f0 msgr2=0x7f4600107150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.885+0000 7f46049b5700 1 --2- 172.21.15.7:0/2213477054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46001015f0 0x7f4600107150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f45f40099c0 tx=0x7f45f4009cd0 comp rx=0 tx=0).stop 2024-06-10T01:01:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.885+0000 7f46049b5700 1 -- 172.21.15.7:0/2213477054 shutdown_connections 2024-06-10T01:01:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.885+0000 7f46049b5700 1 --2- 172.21.15.7:0/2213477054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4600107880 0x7f460010f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.885+0000 7f46049b5700 1 --2- 172.21.15.7:0/2213477054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4600100c40 0x7f4600101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.885+0000 7f46049b5700 1 --2- 172.21.15.7:0/2213477054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46001015f0 0x7f4600107150 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.885+0000 7f46049b5700 1 -- 172.21.15.7:0/2213477054 >> 172.21.15.7:0/2213477054 conn(0x7f4600077990 msgr2=0x7f46000fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.885+0000 7f46049b5700 1 -- 172.21.15.7:0/2213477054 shutdown_connections 2024-06-10T01:01:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.885+0000 7f46049b5700 1 -- 172.21.15.7:0/2213477054 wait complete. 2024-06-10T01:01:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.886+0000 7f46049b5700 1 Processor -- start 2024-06-10T01:01:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.886+0000 7f46049b5700 1 -- start start 2024-06-10T01:01:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.886+0000 7f46049b5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4600100c40 0x7f460019e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.886+0000 7f46049b5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46001015f0 0x7f460019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.886+0000 7f46049b5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4600107880 0x7f4600198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.886+0000 7f46049b5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f46001127e0 con 0x7f46001015f0 2024-06-10T01:01:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.886+0000 7f46049b5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4600112660 con 0x7f4600100c40 2024-06-10T01:01:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.886+0000 7f46049b5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4600112960 con 0x7f4600107880 2024-06-10T01:01:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.886+0000 7f45fdd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46001015f0 0x7f460019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.886+0000 7f45fdd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46001015f0 0x7f460019ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45678/0 (socket says 172.21.15.7:45678) 2024-06-10T01:01:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.886+0000 7f45fdd9b700 1 -- 172.21.15.7:0/627423858 learned_addr learned my addr 172.21.15.7:0/627423858 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:01:41.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.887+0000 7f45fe59c700 1 --2- 172.21.15.7:0/627423858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4600100c40 0x7f460019e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:41.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.887+0000 7f45fed9d700 1 --2- 172.21.15.7:0/627423858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4600107880 0x7f4600198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:41.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.887+0000 7f45fdd9b700 1 -- 172.21.15.7:0/627423858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4600107880 msgr2=0x7f4600198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:41.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.887+0000 7f45fdd9b700 1 --2- 172.21.15.7:0/627423858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4600107880 0x7f4600198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:41.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.887+0000 7f45fdd9b700 1 -- 172.21.15.7:0/627423858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4600100c40 msgr2=0x7f460019e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:41.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.887+0000 7f45fdd9b700 1 --2- 172.21.15.7:0/627423858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4600100c40 0x7f460019e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:41.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.887+0000 7f45fdd9b700 1 -- 172.21.15.7:0/627423858 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46001991c0 con 0x7f46001015f0 2024-06-10T01:01:41.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.887+0000 7f45fdd9b700 1 --2- 172.21.15.7:0/627423858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46001015f0 0x7f460019ed90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f45f4005b40 tx=0x7f45f40038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:41.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.887+0000 7f45ef7fe700 1 -- 172.21.15.7:0/627423858 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f4026070 con 0x7f46001015f0 2024-06-10T01:01:41.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.887+0000 7f46049b5700 1 -- 172.21.15.7:0/627423858 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4600199450 con 0x7f46001015f0 2024-06-10T01:01:41.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.887+0000 7f45ef7fe700 1 -- 172.21.15.7:0/627423858 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45f4004050 con 0x7f46001015f0 2024-06-10T01:01:41.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.887+0000 7f46049b5700 1 -- 172.21.15.7:0/627423858 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4600101af0 con 0x7f46001015f0 2024-06-10T01:01:41.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.887+0000 7f45ef7fe700 1 -- 172.21.15.7:0/627423858 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f400fda0 con 0x7f46001015f0 2024-06-10T01:01:41.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.888+0000 7f46049b5700 1 -- 172.21.15.7:0/627423858 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4600199f40 con 0x7f46001015f0 2024-06-10T01:01:41.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.890+0000 7f45ef7fe700 1 -- 172.21.15.7:0/627423858 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f45f4029070 con 0x7f46001015f0 2024-06-10T01:01:41.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.890+0000 7f45ef7fe700 1 --2- 172.21.15.7:0/627423858 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f45e80a2c20 0x7f45e80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:41.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.890+0000 7f45fe59c700 1 --2- 172.21.15.7:0/627423858 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f45e80a2c20 0x7f45e80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:41.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.890+0000 7f45ef7fe700 1 -- 172.21.15.7:0/627423858 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f45f40cf940 con 0x7f46001015f0 2024-06-10T01:01:41.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.891+0000 7f45fe59c700 1 --2- 172.21.15.7:0/627423858 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f45e80a2c20 0x7f45e80a50e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4600107540 tx=0x7f45f00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:41.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:41.895+0000 7f45ef7fe700 1 -- 172.21.15.7:0/627423858 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f45f4030080 con 0x7f46001015f0 2024-06-10T01:01:42.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:42.119+0000 7f46049b5700 1 -- 172.21.15.7:0/627423858 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f460002dca0 con 0x7f46001015f0 2024-06-10T01:01:42.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:42.989+0000 7f45ef7fe700 1 -- 172.21.15.7:0/627423858 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7f45f4013070 con 0x7f46001015f0 2024-06-10T01:01:42.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:42.992+0000 7f46049b5700 1 -- 172.21.15.7:0/627423858 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f45e80a2c20 msgr2=0x7f45e80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:42.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:42.992+0000 7f46049b5700 1 --2- 172.21.15.7:0/627423858 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f45e80a2c20 0x7f45e80a50e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4600107540 tx=0x7f45f00073d0 comp rx=0 tx=0).stop 2024-06-10T01:01:42.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:42.992+0000 7f46049b5700 1 -- 172.21.15.7:0/627423858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46001015f0 msgr2=0x7f460019ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:42.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:42.992+0000 7f46049b5700 1 --2- 172.21.15.7:0/627423858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46001015f0 0x7f460019ed90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f45f4005b40 tx=0x7f45f40038b0 comp rx=0 tx=0).stop 2024-06-10T01:01:42.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:42.992+0000 7f46049b5700 1 -- 172.21.15.7:0/627423858 shutdown_connections 2024-06-10T01:01:42.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:42.992+0000 7f46049b5700 1 --2- 172.21.15.7:0/627423858 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f45e80a2c20 0x7f45e80a50e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:42.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:42.992+0000 7f46049b5700 1 --2- 172.21.15.7:0/627423858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4600107880 0x7f4600198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:42.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:42.992+0000 7f46049b5700 1 --2- 172.21.15.7:0/627423858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4600100c40 0x7f460019e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:42.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:42.992+0000 7f46049b5700 1 --2- 172.21.15.7:0/627423858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46001015f0 0x7f460019ed90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:42.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:42.992+0000 7f46049b5700 1 -- 172.21.15.7:0/627423858 >> 172.21.15.7:0/627423858 conn(0x7f4600077990 msgr2=0x7f46000fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:42.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:42.992+0000 7f46049b5700 1 -- 172.21.15.7:0/627423858 shutdown_connections 2024-06-10T01:01:42.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:42.992+0000 7f46049b5700 1 -- 172.21.15.7:0/627423858 wait complete. 2024-06-10T01:01:42.992 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-06-10T01:01:43.266 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:42 smithi007 ceph-mon[32398]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:43.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:42 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:43.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:42 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:43.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:42 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/627423858' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-10T01:01:43.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:42 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:43.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:42 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:01:43.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:42 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:43.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:42 smithi078 ceph-mon[36683]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:43.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:42 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:43.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:42 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:43.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:42 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/627423858' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-10T01:01:43.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:42 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:43.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:42 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:01:43.334 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:42 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:43.456 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:42 smithi175 ceph-mon[36636]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:43.456 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:42 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:43.456 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:42 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:43.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:42 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/627423858' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-10T01:01:43.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:42 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:43.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:42 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:01:43.457 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:42 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:01:43.721 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-06-10T01:01:43.973 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:01:44.266 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:43 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/627423858' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-10T01:01:44.266 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:43 smithi007 ceph-mon[32398]: osdmap e82: 12 total, 12 up, 12 in 2024-06-10T01:01:44.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:43 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/627423858' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-10T01:01:44.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:43 smithi078 ceph-mon[36683]: osdmap e82: 12 total, 12 up, 12 in 2024-06-10T01:01:44.456 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:43 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/627423858' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-10T01:01:44.456 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:43 smithi175 ceph-mon[36636]: osdmap e82: 12 total, 12 up, 12 in 2024-06-10T01:01:45.266 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:44 smithi007 ceph-mon[32398]: pgmap v286: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:45.266 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:44 smithi007 ceph-mon[32398]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-10T01:01:45.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:44 smithi007 ceph-mon[32398]: osdmap e83: 12 total, 12 up, 12 in 2024-06-10T01:01:45.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:44 smithi078 ceph-mon[36683]: pgmap v286: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:45.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:44 smithi078 ceph-mon[36683]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-10T01:01:45.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:44 smithi078 ceph-mon[36683]: osdmap e83: 12 total, 12 up, 12 in 2024-06-10T01:01:45.456 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:44 smithi175 ceph-mon[36636]: pgmap v286: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:45.456 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:44 smithi175 ceph-mon[36636]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-10T01:01:45.456 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:44 smithi175 ceph-mon[36636]: osdmap e83: 12 total, 12 up, 12 in 2024-06-10T01:01:45.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.527+0000 7f1a5b2a6700 1 -- 172.21.15.7:0/1588702198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a541084f0 msgr2=0x7f1a541088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:45.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.527+0000 7f1a5b2a6700 1 --2- 172.21.15.7:0/1588702198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a541084f0 0x7f1a541088d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1a50009990 tx=0x7f1a50009ca0 comp rx=0 tx=0).stop 2024-06-10T01:01:45.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.527+0000 7f1a5b2a6700 1 -- 172.21.15.7:0/1588702198 shutdown_connections 2024-06-10T01:01:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.527+0000 7f1a5b2a6700 1 --2- 172.21.15.7:0/1588702198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a54102e80 0x7f1a5410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.527+0000 7f1a5b2a6700 1 --2- 172.21.15.7:0/1588702198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a541084f0 0x7f1a541088d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.527+0000 7f1a5b2a6700 1 --2- 172.21.15.7:0/1588702198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a541024e0 0x7f1a54102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.527+0000 7f1a5b2a6700 1 -- 172.21.15.7:0/1588702198 >> 172.21.15.7:0/1588702198 conn(0x7f1a54079530 msgr2=0x7f1a54079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.527+0000 7f1a5b2a6700 1 -- 172.21.15.7:0/1588702198 shutdown_connections 2024-06-10T01:01:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.527+0000 7f1a5b2a6700 1 -- 172.21.15.7:0/1588702198 wait complete. 2024-06-10T01:01:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.527+0000 7f1a5b2a6700 1 Processor -- start 2024-06-10T01:01:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.527+0000 7f1a5b2a6700 1 -- start start 2024-06-10T01:01:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.528+0000 7f1a5b2a6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a541024e0 0x7f1a5419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.528+0000 7f1a5b2a6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a54102e80 0x7f1a5419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.528+0000 7f1a5b2a6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a541084f0 0x7f1a54198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.528+0000 7f1a5b2a6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1a541127a0 con 0x7f1a541024e0 2024-06-10T01:01:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.528+0000 7f1a5b2a6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1a54112620 con 0x7f1a541084f0 2024-06-10T01:01:45.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.528+0000 7f1a5b2a6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1a54112920 con 0x7f1a54102e80 2024-06-10T01:01:45.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.528+0000 7f1a59042700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a541024e0 0x7f1a5419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:45.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.528+0000 7f1a59042700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a541024e0 0x7f1a5419e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45696/0 (socket says 172.21.15.7:45696) 2024-06-10T01:01:45.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.528+0000 7f1a59042700 1 -- 172.21.15.7:0/3560776017 learned_addr learned my addr 172.21.15.7:0/3560776017 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:01:45.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.528+0000 7f1a58841700 1 --2- 172.21.15.7:0/3560776017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a54102e80 0x7f1a5419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:45.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.528+0000 7f1a59843700 1 --2- 172.21.15.7:0/3560776017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a541084f0 0x7f1a54198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:45.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.528+0000 7f1a59042700 1 -- 172.21.15.7:0/3560776017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a54102e80 msgr2=0x7f1a5419ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:45.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.528+0000 7f1a59042700 1 --2- 172.21.15.7:0/3560776017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a54102e80 0x7f1a5419ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:45.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.528+0000 7f1a59042700 1 -- 172.21.15.7:0/3560776017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a541084f0 msgr2=0x7f1a54198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:45.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.528+0000 7f1a59042700 1 --2- 172.21.15.7:0/3560776017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a541084f0 0x7f1a54198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:45.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.528+0000 7f1a59042700 1 -- 172.21.15.7:0/3560776017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a541991c0 con 0x7f1a541024e0 2024-06-10T01:01:45.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.528+0000 7f1a58841700 1 --2- 172.21.15.7:0/3560776017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a54102e80 0x7f1a5419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:01:45.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.528+0000 7f1a59042700 1 --2- 172.21.15.7:0/3560776017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a541024e0 0x7f1a5419e8a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1a50005910 tx=0x7f1a50009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:45.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.528+0000 7f1a4a7fc700 1 -- 172.21.15.7:0/3560776017 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a50027070 con 0x7f1a541024e0 2024-06-10T01:01:45.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.528+0000 7f1a5b2a6700 1 -- 172.21.15.7:0/3560776017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a54199450 con 0x7f1a541024e0 2024-06-10T01:01:45.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.528+0000 7f1a4a7fc700 1 -- 172.21.15.7:0/3560776017 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a50004050 con 0x7f1a541024e0 2024-06-10T01:01:45.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.529+0000 7f1a4a7fc700 1 -- 172.21.15.7:0/3560776017 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a50022e30 con 0x7f1a541024e0 2024-06-10T01:01:45.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.529+0000 7f1a5b2a6700 1 -- 172.21.15.7:0/3560776017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a541a5770 con 0x7f1a541024e0 2024-06-10T01:01:45.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.529+0000 7f1a5b2a6700 1 -- 172.21.15.7:0/3560776017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a54069180 con 0x7f1a541024e0 2024-06-10T01:01:45.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.531+0000 7f1a4a7fc700 1 -- 172.21.15.7:0/3560776017 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f1a5002a070 con 0x7f1a541024e0 2024-06-10T01:01:45.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.532+0000 7f1a4a7fc700 1 --2- 172.21.15.7:0/3560776017 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1a400a2c20 0x7f1a400a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:45.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.532+0000 7f1a58841700 1 --2- 172.21.15.7:0/3560776017 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1a400a2c20 0x7f1a400a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:45.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.532+0000 7f1a4a7fc700 1 -- 172.21.15.7:0/3560776017 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7497+0+0 (secure 0 0 0) 0x7f1a500cfcd0 con 0x7f1a541024e0 2024-06-10T01:01:45.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.532+0000 7f1a58841700 1 --2- 172.21.15.7:0/3560776017 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1a400a2c20 0x7f1a400a50e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1a44005950 tx=0x7f1a440058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:45.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.535+0000 7f1a4a7fc700 1 -- 172.21.15.7:0/3560776017 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1a50013070 con 0x7f1a541024e0 2024-06-10T01:01:45.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:45.741+0000 7f1a5b2a6700 1 -- 172.21.15.7:0/3560776017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f1a5402dca0 con 0x7f1a541024e0 2024-06-10T01:01:46.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:46.227+0000 7f1a4a7fc700 1 -- 172.21.15.7:0/3560776017 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v85) ==== 171+0+0 (secure 0 0 0) 0x7f1a5009cf50 con 0x7f1a541024e0 2024-06-10T01:01:46.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:46.230+0000 7f1a5b2a6700 1 -- 172.21.15.7:0/3560776017 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1a400a2c20 msgr2=0x7f1a400a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:46.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:46.230+0000 7f1a5b2a6700 1 --2- 172.21.15.7:0/3560776017 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1a400a2c20 0x7f1a400a50e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1a44005950 tx=0x7f1a440058e0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:46.230+0000 7f1a5b2a6700 1 -- 172.21.15.7:0/3560776017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a541024e0 msgr2=0x7f1a5419e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:46.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:46.230+0000 7f1a5b2a6700 1 --2- 172.21.15.7:0/3560776017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a541024e0 0x7f1a5419e8a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1a50005910 tx=0x7f1a50009f90 comp rx=0 tx=0).stop 2024-06-10T01:01:46.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:46.230+0000 7f1a5b2a6700 1 -- 172.21.15.7:0/3560776017 shutdown_connections 2024-06-10T01:01:46.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:46.230+0000 7f1a5b2a6700 1 --2- 172.21.15.7:0/3560776017 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1a400a2c20 0x7f1a400a50e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:46.230+0000 7f1a5b2a6700 1 --2- 172.21.15.7:0/3560776017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a541084f0 0x7f1a54198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:46.230+0000 7f1a5b2a6700 1 --2- 172.21.15.7:0/3560776017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a541024e0 0x7f1a5419e8a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:46.230+0000 7f1a5b2a6700 1 --2- 172.21.15.7:0/3560776017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a54102e80 0x7f1a5419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:46.230+0000 7f1a5b2a6700 1 -- 172.21.15.7:0/3560776017 >> 172.21.15.7:0/3560776017 conn(0x7f1a54079530 msgr2=0x7f1a54100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:46.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:46.230+0000 7f1a5b2a6700 1 -- 172.21.15.7:0/3560776017 shutdown_connections 2024-06-10T01:01:46.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:46.230+0000 7f1a5b2a6700 1 -- 172.21.15.7:0/3560776017 wait complete. 2024-06-10T01:01:46.237 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-06-10T01:01:46.456 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:46 smithi175 ceph-mon[36636]: osdmap e84: 12 total, 12 up, 12 in 2024-06-10T01:01:46.475 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:46 smithi078 ceph-mon[36683]: osdmap e84: 12 total, 12 up, 12 in 2024-06-10T01:01:46.516 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:46 smithi007 ceph-mon[32398]: osdmap e84: 12 total, 12 up, 12 in 2024-06-10T01:01:46.899 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-06-10T01:01:47.154 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:01:47.456 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:47 smithi175 ceph-mon[36636]: pgmap v289: 33 pgs: 3 creating+activating, 11 creating+peering, 8 unknown, 11 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:47.456 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:47 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3560776017' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-10T01:01:47.456 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:47 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3560776017' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-10T01:01:47.456 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:47 smithi175 ceph-mon[36636]: osdmap e85: 12 total, 12 up, 12 in 2024-06-10T01:01:47.486 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:47 smithi007 ceph-mon[32398]: pgmap v289: 33 pgs: 3 creating+activating, 11 creating+peering, 8 unknown, 11 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:47.487 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:47 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3560776017' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-10T01:01:47.487 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:47 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3560776017' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-10T01:01:47.487 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:47 smithi007 ceph-mon[32398]: osdmap e85: 12 total, 12 up, 12 in 2024-06-10T01:01:47.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:47 smithi078 ceph-mon[36683]: pgmap v289: 33 pgs: 3 creating+activating, 11 creating+peering, 8 unknown, 11 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:47.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:47 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3560776017' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-10T01:01:47.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:47 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3560776017' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-10T01:01:47.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:47 smithi078 ceph-mon[36683]: osdmap e85: 12 total, 12 up, 12 in 2024-06-10T01:01:48.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:48 smithi007 ceph-mon[32398]: osdmap e86: 12 total, 12 up, 12 in 2024-06-10T01:01:48.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:48 smithi078 ceph-mon[36683]: osdmap e86: 12 total, 12 up, 12 in 2024-06-10T01:01:48.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:48 smithi175 ceph-mon[36636]: osdmap e86: 12 total, 12 up, 12 in 2024-06-10T01:01:48.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.765+0000 7fa56f574700 1 -- 172.21.15.7:0/668764678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5681066b0 msgr2=0x7fa56810e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:48.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.765+0000 7fa56f574700 1 --2- 172.21.15.7:0/668764678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5681066b0 0x7fa56810e280 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa56000b0d0 tx=0x7fa56000b3e0 comp rx=0 tx=0).stop 2024-06-10T01:01:48.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.765+0000 7fa56f574700 1 -- 172.21.15.7:0/668764678 shutdown_connections 2024-06-10T01:01:48.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.765+0000 7fa56f574700 1 --2- 172.21.15.7:0/668764678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5681066b0 0x7fa56810e280 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:48.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.765+0000 7fa56f574700 1 --2- 172.21.15.7:0/668764678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa568101620 0x7fa568101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:48.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.765+0000 7fa56f574700 1 --2- 172.21.15.7:0/668764678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa568101fd0 0x7fa568105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:48.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.765+0000 7fa56f574700 1 -- 172.21.15.7:0/668764678 >> 172.21.15.7:0/668764678 conn(0x7fa568078340 msgr2=0x7fa568078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:48.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.766+0000 7fa56f574700 1 -- 172.21.15.7:0/668764678 shutdown_connections 2024-06-10T01:01:48.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.766+0000 7fa56f574700 1 -- 172.21.15.7:0/668764678 wait complete. 2024-06-10T01:01:48.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.766+0000 7fa56f574700 1 Processor -- start 2024-06-10T01:01:48.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.766+0000 7fa56f574700 1 -- start start 2024-06-10T01:01:48.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.766+0000 7fa56f574700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa568101620 0x7fa568103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:48.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.766+0000 7fa56f574700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa568101fd0 0x7fa5681035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:48.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.766+0000 7fa56f574700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5681066b0 0x7fa568103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:48.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.766+0000 7fa56f574700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa5681116a0 con 0x7fa568101620 2024-06-10T01:01:48.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.766+0000 7fa56f574700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa568111520 con 0x7fa5681066b0 2024-06-10T01:01:48.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.766+0000 7fa56f574700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa568111820 con 0x7fa568101fd0 2024-06-10T01:01:48.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.767+0000 7fa56d310700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa568101620 0x7fa568103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:48.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.767+0000 7fa56d310700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa568101620 0x7fa568103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45722/0 (socket says 172.21.15.7:45722) 2024-06-10T01:01:48.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.767+0000 7fa56d310700 1 -- 172.21.15.7:0/1462842466 learned_addr learned my addr 172.21.15.7:0/1462842466 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:01:48.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.767+0000 7fa56cb0f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa568101fd0 0x7fa5681035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:48.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.767+0000 7fa56db11700 1 --2- 172.21.15.7:0/1462842466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5681066b0 0x7fa568103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:48.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.767+0000 7fa56db11700 1 -- 172.21.15.7:0/1462842466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa568101fd0 msgr2=0x7fa5681035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:48.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.767+0000 7fa56db11700 1 --2- 172.21.15.7:0/1462842466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa568101fd0 0x7fa5681035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:48.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.767+0000 7fa56db11700 1 -- 172.21.15.7:0/1462842466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa568101620 msgr2=0x7fa568103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:48.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.767+0000 7fa56db11700 1 --2- 172.21.15.7:0/1462842466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa568101620 0x7fa568103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:48.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.767+0000 7fa56db11700 1 -- 172.21.15.7:0/1462842466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5681a70c0 con 0x7fa5681066b0 2024-06-10T01:01:48.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.767+0000 7fa56d310700 1 --2- 172.21.15.7:0/1462842466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa568101620 0x7fa568103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:01:48.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.767+0000 7fa56db11700 1 --2- 172.21.15.7:0/1462842466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5681066b0 0x7fa568103af0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa56000b770 tx=0x7fa560007a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:48.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.767+0000 7fa55e7fc700 1 -- 172.21.15.7:0/1462842466 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa560021070 con 0x7fa5681066b0 2024-06-10T01:01:48.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.767+0000 7fa56f574700 1 -- 172.21.15.7:0/1462842466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5681a7350 con 0x7fa5681066b0 2024-06-10T01:01:48.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.767+0000 7fa55e7fc700 1 -- 172.21.15.7:0/1462842466 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa56001db30 con 0x7fa5681066b0 2024-06-10T01:01:48.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.767+0000 7fa55e7fc700 1 -- 172.21.15.7:0/1462842466 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56001b9c0 con 0x7fa5681066b0 2024-06-10T01:01:48.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.768+0000 7fa56f574700 1 -- 172.21.15.7:0/1462842466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5681a77b0 con 0x7fa5681066b0 2024-06-10T01:01:48.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.768+0000 7fa56f574700 1 -- 172.21.15.7:0/1462842466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa568069180 con 0x7fa5681066b0 2024-06-10T01:01:48.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.771+0000 7fa55e7fc700 1 -- 172.21.15.7:0/1462842466 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fa560024070 con 0x7fa5681066b0 2024-06-10T01:01:48.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.772+0000 7fa55e7fc700 1 --2- 172.21.15.7:0/1462842466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa5540a2c20 0x7fa5540a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:48.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.772+0000 7fa56d310700 1 --2- 172.21.15.7:0/1462842466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa5540a2c20 0x7fa5540a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:48.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.772+0000 7fa55e7fc700 1 -- 172.21.15.7:0/1462842466 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7879+0+0 (secure 0 0 0) 0x7fa5600d03f0 con 0x7fa5681066b0 2024-06-10T01:01:48.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.772+0000 7fa56d310700 1 --2- 172.21.15.7:0/1462842466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa5540a2c20 0x7fa5540a50e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa56400f4d0 tx=0x7fa564005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:48.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:48.774+0000 7fa55e7fc700 1 -- 172.21.15.7:0/1462842466 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa560011390 con 0x7fa5681066b0 2024-06-10T01:01:49.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:49.027+0000 7fa56f574700 1 -- 172.21.15.7:0/1462842466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa56802de60 con 0x7fa5681066b0 2024-06-10T01:01:49.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:49.238+0000 7fa55e7fc700 1 -- 172.21.15.7:0/1462842466 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fa56009d4f0 con 0x7fa5681066b0 2024-06-10T01:01:49.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:49.241+0000 7fa56f574700 1 -- 172.21.15.7:0/1462842466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa5540a2c20 msgr2=0x7fa5540a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:49.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:49.241+0000 7fa56f574700 1 --2- 172.21.15.7:0/1462842466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa5540a2c20 0x7fa5540a50e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa56400f4d0 tx=0x7fa564005f90 comp rx=0 tx=0).stop 2024-06-10T01:01:49.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:49.241+0000 7fa56f574700 1 -- 172.21.15.7:0/1462842466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5681066b0 msgr2=0x7fa568103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:49.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:49.241+0000 7fa56f574700 1 --2- 172.21.15.7:0/1462842466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5681066b0 0x7fa568103af0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa56000b770 tx=0x7fa560007a20 comp rx=0 tx=0).stop 2024-06-10T01:01:49.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:49.241+0000 7fa56f574700 1 -- 172.21.15.7:0/1462842466 shutdown_connections 2024-06-10T01:01:49.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:49.241+0000 7fa56f574700 1 --2- 172.21.15.7:0/1462842466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa5540a2c20 0x7fa5540a50e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:49.241+0000 7fa56f574700 1 --2- 172.21.15.7:0/1462842466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5681066b0 0x7fa568103af0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:49.241+0000 7fa56f574700 1 --2- 172.21.15.7:0/1462842466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa568101620 0x7fa568103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:49.241+0000 7fa56f574700 1 --2- 172.21.15.7:0/1462842466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa568101fd0 0x7fa5681035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:49.241+0000 7fa56f574700 1 -- 172.21.15.7:0/1462842466 >> 172.21.15.7:0/1462842466 conn(0x7fa568078340 msgr2=0x7fa56806ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:49.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:49.242+0000 7fa56f574700 1 -- 172.21.15.7:0/1462842466 shutdown_connections 2024-06-10T01:01:49.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:49.242+0000 7fa56f574700 1 -- 172.21.15.7:0/1462842466 wait complete. 2024-06-10T01:01:49.241 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 2 and data pool 3 2024-06-10T01:01:49.516 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:49 smithi007 ceph-mon[32398]: pgmap v292: 97 pgs: 3 creating+activating, 19 creating+peering, 61 unknown, 14 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:49.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:49 smithi007 ceph-mon[32398]: osdmap e87: 12 total, 12 up, 12 in 2024-06-10T01:01:49.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:49 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1462842466' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-10T01:01:49.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:49 smithi007 ceph-mon[32398]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-10T01:01:49.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:49 smithi007 ceph-mon[32398]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-06-10T01:01:49.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:49 smithi078 ceph-mon[36683]: pgmap v292: 97 pgs: 3 creating+activating, 19 creating+peering, 61 unknown, 14 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:49.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:49 smithi078 ceph-mon[36683]: osdmap e87: 12 total, 12 up, 12 in 2024-06-10T01:01:49.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:49 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1462842466' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-10T01:01:49.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:49 smithi078 ceph-mon[36683]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-10T01:01:49.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:49 smithi078 ceph-mon[36683]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-06-10T01:01:49.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:49 smithi175 ceph-mon[36636]: pgmap v292: 97 pgs: 3 creating+activating, 19 creating+peering, 61 unknown, 14 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:01:49.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:49 smithi175 ceph-mon[36636]: osdmap e87: 12 total, 12 up, 12 in 2024-06-10T01:01:49.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:49 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1462842466' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-10T01:01:49.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:49 smithi175 ceph-mon[36636]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-10T01:01:49.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:49 smithi175 ceph-mon[36636]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-06-10T01:01:49.918 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-06-10T01:01:49.918 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-06-10T01:01:50.173 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:01:50.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:50 smithi007 ceph-mon[32398]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-10T01:01:50.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:50 smithi007 ceph-mon[32398]: osdmap e88: 12 total, 12 up, 12 in 2024-06-10T01:01:50.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:50 smithi007 ceph-mon[32398]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-06-10T01:01:50.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:50 smithi007 ceph-mon[32398]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-06-10T01:01:50.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:50 smithi078 ceph-mon[36683]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-10T01:01:50.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:50 smithi078 ceph-mon[36683]: osdmap e88: 12 total, 12 up, 12 in 2024-06-10T01:01:50.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:50 smithi078 ceph-mon[36683]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-06-10T01:01:50.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:50 smithi078 ceph-mon[36683]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-06-10T01:01:50.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:50 smithi175 ceph-mon[36636]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-10T01:01:50.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:50 smithi175 ceph-mon[36636]: osdmap e88: 12 total, 12 up, 12 in 2024-06-10T01:01:50.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:50 smithi175 ceph-mon[36636]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-06-10T01:01:50.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:50 smithi175 ceph-mon[36636]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-06-10T01:01:51.516 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:51 smithi007 ceph-mon[32398]: pgmap v295: 97 pgs: 8 creating+peering, 13 unknown, 76 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 5 op/s 2024-06-10T01:01:51.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:51 smithi007 ceph-mon[32398]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-10T01:01:51.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:51 smithi007 ceph-mon[32398]: Cluster is now healthy 2024-06-10T01:01:51.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:51 smithi007 ceph-mon[32398]: mds.? [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] up:active 2024-06-10T01:01:51.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:51 smithi007 ceph-mon[32398]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T01:01:51.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:51 smithi007 ceph-mon[32398]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T01:01:51.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:51 smithi078 ceph-mon[36683]: pgmap v295: 97 pgs: 8 creating+peering, 13 unknown, 76 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 5 op/s 2024-06-10T01:01:51.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:51 smithi078 ceph-mon[36683]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-10T01:01:51.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:51 smithi078 ceph-mon[36683]: Cluster is now healthy 2024-06-10T01:01:51.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:51 smithi078 ceph-mon[36683]: mds.? [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] up:active 2024-06-10T01:01:51.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:51 smithi078 ceph-mon[36683]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T01:01:51.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:51 smithi078 ceph-mon[36683]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T01:01:51.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:51 smithi175 ceph-mon[36636]: pgmap v295: 97 pgs: 8 creating+peering, 13 unknown, 76 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 5 op/s 2024-06-10T01:01:51.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:51 smithi175 ceph-mon[36636]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-10T01:01:51.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:51 smithi175 ceph-mon[36636]: Cluster is now healthy 2024-06-10T01:01:51.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:51 smithi175 ceph-mon[36636]: mds.? [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] up:active 2024-06-10T01:01:51.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:51 smithi175 ceph-mon[36636]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T01:01:51.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:51 smithi175 ceph-mon[36636]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T01:01:51.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.782+0000 7f0bef89c700 1 -- 172.21.15.7:0/3519215069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0be8107860 msgr2=0x7f0be810f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:51.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.782+0000 7f0bef89c700 1 --2- 172.21.15.7:0/3519215069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0be8107860 0x7f0be810f3e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0be000b0d0 tx=0x7f0be000b3e0 comp rx=0 tx=0).stop 2024-06-10T01:01:51.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.782+0000 7f0bef89c700 1 -- 172.21.15.7:0/3519215069 shutdown_connections 2024-06-10T01:01:51.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.782+0000 7f0bef89c700 1 --2- 172.21.15.7:0/3519215069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0be8107860 0x7f0be810f3e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:51.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.782+0000 7f0bef89c700 1 --2- 172.21.15.7:0/3519215069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0be8102770 0x7f0be8102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:51.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.782+0000 7f0bef89c700 1 --2- 172.21.15.7:0/3519215069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0be8103120 0x7f0be8107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:51.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.782+0000 7f0bef89c700 1 -- 172.21.15.7:0/3519215069 >> 172.21.15.7:0/3519215069 conn(0x7f0be8079600 msgr2=0x7f0be8079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:51.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.782+0000 7f0bef89c700 1 -- 172.21.15.7:0/3519215069 shutdown_connections 2024-06-10T01:01:51.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.782+0000 7f0bef89c700 1 -- 172.21.15.7:0/3519215069 wait complete. 2024-06-10T01:01:51.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.782+0000 7f0bef89c700 1 Processor -- start 2024-06-10T01:01:51.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.782+0000 7f0bef89c700 1 -- start start 2024-06-10T01:01:51.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.783+0000 7f0bef89c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0be8102770 0x7f0be819e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:51.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.783+0000 7f0bef89c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0be8103120 0x7f0be819ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:51.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.783+0000 7f0bef89c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0be8107860 0x7f0be8198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:51.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.783+0000 7f0bef89c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0be8112780 con 0x7f0be8102770 2024-06-10T01:01:51.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.783+0000 7f0bef89c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0be8112600 con 0x7f0be8107860 2024-06-10T01:01:51.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.783+0000 7f0bef89c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0be8112900 con 0x7f0be8103120 2024-06-10T01:01:51.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.783+0000 7f0bed638700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0be8102770 0x7f0be819e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:51.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.783+0000 7f0bed638700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0be8102770 0x7f0be819e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45750/0 (socket says 172.21.15.7:45750) 2024-06-10T01:01:51.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.783+0000 7f0bed638700 1 -- 172.21.15.7:0/1613364223 learned_addr learned my addr 172.21.15.7:0/1613364223 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:01:51.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.783+0000 7f0bece37700 1 --2- 172.21.15.7:0/1613364223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0be8103120 0x7f0be819ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:51.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.783+0000 7f0bede39700 1 --2- 172.21.15.7:0/1613364223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0be8107860 0x7f0be8198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:51.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.783+0000 7f0bed638700 1 -- 172.21.15.7:0/1613364223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0be8103120 msgr2=0x7f0be819ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:51.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.783+0000 7f0bed638700 1 --2- 172.21.15.7:0/1613364223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0be8103120 0x7f0be819ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:51.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.783+0000 7f0bed638700 1 -- 172.21.15.7:0/1613364223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0be8107860 msgr2=0x7f0be8198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:51.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.783+0000 7f0bed638700 1 --2- 172.21.15.7:0/1613364223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0be8107860 0x7f0be8198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:51.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.783+0000 7f0bed638700 1 -- 172.21.15.7:0/1613364223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be8199000 con 0x7f0be8102770 2024-06-10T01:01:51.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.783+0000 7f0bede39700 1 --2- 172.21.15.7:0/1613364223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0be8107860 0x7f0be8198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:01:51.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.783+0000 7f0bed638700 1 --2- 172.21.15.7:0/1613364223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0be8102770 0x7f0be819e6a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0be400c9b0 tx=0x7f0be400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:51.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.783+0000 7f0bde7fc700 1 -- 172.21.15.7:0/1613364223 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be4007b00 con 0x7f0be8102770 2024-06-10T01:01:51.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.784+0000 7f0bde7fc700 1 -- 172.21.15.7:0/1613364223 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0be4007ca0 con 0x7f0be8102770 2024-06-10T01:01:51.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.784+0000 7f0bef89c700 1 -- 172.21.15.7:0/1613364223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be81992f0 con 0x7f0be8102770 2024-06-10T01:01:51.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.784+0000 7f0bde7fc700 1 -- 172.21.15.7:0/1613364223 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be4018780 con 0x7f0be8102770 2024-06-10T01:01:51.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.784+0000 7f0bef89c700 1 -- 172.21.15.7:0/1613364223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be81a5500 con 0x7f0be8102770 2024-06-10T01:01:51.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.785+0000 7f0bef89c700 1 -- 172.21.15.7:0/1613364223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0be8103bf0 con 0x7f0be8102770 2024-06-10T01:01:51.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.786+0000 7f0bde7fc700 1 -- 172.21.15.7:0/1613364223 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f0be400f450 con 0x7f0be8102770 2024-06-10T01:01:51.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.786+0000 7f0bde7fc700 1 --2- 172.21.15.7:0/1613364223 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0bd40a2c20 0x7f0bd40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:51.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.787+0000 7f0bece37700 1 --2- 172.21.15.7:0/1613364223 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0bd40a2c20 0x7f0bd40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:51.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.787+0000 7f0bde7fc700 1 -- 172.21.15.7:0/1613364223 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f0be40cf220 con 0x7f0be8102770 2024-06-10T01:01:51.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.787+0000 7f0bece37700 1 --2- 172.21.15.7:0/1613364223 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0bd40a2c20 0x7f0bd40a50e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8005ea0 tx=0x7f0bd8005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:51.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:51.790+0000 7f0bde7fc700 1 -- 172.21.15.7:0/1613364223 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0be4014030 con 0x7f0be8102770 2024-06-10T01:01:52.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:52.006+0000 7f0bef89c700 1 -- 172.21.15.7:0/1613364223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f0be81a5790 con 0x7f0be8102770 2024-06-10T01:01:52.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:52.253+0000 7f0bde7fc700 1 -- 172.21.15.7:0/1613364223 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v89) ==== 157+0+0 (secure 0 0 0) 0x7f0be409c360 con 0x7f0be8102770 2024-06-10T01:01:52.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:52.256+0000 7f0bef89c700 1 -- 172.21.15.7:0/1613364223 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0bd40a2c20 msgr2=0x7f0bd40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:52.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:52.256+0000 7f0bef89c700 1 --2- 172.21.15.7:0/1613364223 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0bd40a2c20 0x7f0bd40a50e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8005ea0 tx=0x7f0bd8005e30 comp rx=0 tx=0).stop 2024-06-10T01:01:52.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:52.256+0000 7f0bef89c700 1 -- 172.21.15.7:0/1613364223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0be8102770 msgr2=0x7f0be819e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:52.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:52.256+0000 7f0bef89c700 1 --2- 172.21.15.7:0/1613364223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0be8102770 0x7f0be819e6a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0be400c9b0 tx=0x7f0be400cd70 comp rx=0 tx=0).stop 2024-06-10T01:01:52.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:52.256+0000 7f0bef89c700 1 -- 172.21.15.7:0/1613364223 shutdown_connections 2024-06-10T01:01:52.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:52.256+0000 7f0bef89c700 1 --2- 172.21.15.7:0/1613364223 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0bd40a2c20 0x7f0bd40a50e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:52.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:52.256+0000 7f0bef89c700 1 --2- 172.21.15.7:0/1613364223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0be8107860 0x7f0be8198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:52.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:52.256+0000 7f0bef89c700 1 --2- 172.21.15.7:0/1613364223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0be8102770 0x7f0be819e6a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:52.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:52.256+0000 7f0bef89c700 1 --2- 172.21.15.7:0/1613364223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0be8103120 0x7f0be819ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:52.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:52.256+0000 7f0bef89c700 1 -- 172.21.15.7:0/1613364223 >> 172.21.15.7:0/1613364223 conn(0x7f0be8079600 msgr2=0x7f0be8100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:52.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:52.256+0000 7f0bef89c700 1 -- 172.21.15.7:0/1613364223 shutdown_connections 2024-06-10T01:01:52.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:52.256+0000 7f0bef89c700 1 -- 172.21.15.7:0/1613364223 wait complete. 2024-06-10T01:01:52.255 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:52 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1613364223' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-10T01:01:52.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:52 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1613364223' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-10T01:01:52.587 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:52 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1613364223' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-10T01:01:52.956 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jun 10 01:01:52 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-i[66802]: 2024-06-10T01:01:52.587+0000 7fe50fb02700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-06-10T01:01:52.978 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-06-10T01:01:53.237 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:01:53.505 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:53 smithi007 ceph-mon[32398]: pgmap v296: 97 pgs: 8 creating+peering, 89 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 965 B/s wr, 6 op/s 2024-06-10T01:01:53.505 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:53 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1613364223' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-10T01:01:53.505 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:53 smithi007 ceph-mon[32398]: osdmap e89: 12 total, 12 up, 12 in 2024-06-10T01:01:53.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:53 smithi078 ceph-mon[36683]: pgmap v296: 97 pgs: 8 creating+peering, 89 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 965 B/s wr, 6 op/s 2024-06-10T01:01:53.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:53 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1613364223' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-10T01:01:53.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:53 smithi078 ceph-mon[36683]: osdmap e89: 12 total, 12 up, 12 in 2024-06-10T01:01:53.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:53 smithi175 ceph-mon[36636]: pgmap v296: 97 pgs: 8 creating+peering, 89 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 965 B/s wr, 6 op/s 2024-06-10T01:01:53.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:53 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1613364223' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-10T01:01:53.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:53 smithi175 ceph-mon[36636]: osdmap e89: 12 total, 12 up, 12 in 2024-06-10T01:01:54.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.935+0000 7fee29655700 1 -- 172.21.15.7:0/2682692467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee24103150 msgr2=0x7fee241070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:54.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.935+0000 7fee29655700 1 --2- 172.21.15.7:0/2682692467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee24103150 0x7fee241070b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fee180099c0 tx=0x7fee18009cd0 comp rx=0 tx=0).stop 2024-06-10T01:01:54.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.935+0000 7fee29655700 1 -- 172.21.15.7:0/2682692467 shutdown_connections 2024-06-10T01:01:54.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.935+0000 7fee29655700 1 --2- 172.21.15.7:0/2682692467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee24107790 0x7fee2410f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:54.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.935+0000 7fee29655700 1 --2- 172.21.15.7:0/2682692467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee241027a0 0x7fee24102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:54.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.935+0000 7fee29655700 1 --2- 172.21.15.7:0/2682692467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee24103150 0x7fee241070b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:54.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.935+0000 7fee29655700 1 -- 172.21.15.7:0/2682692467 >> 172.21.15.7:0/2682692467 conn(0x7fee24079560 msgr2=0x7fee24079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:54.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.935+0000 7fee29655700 1 -- 172.21.15.7:0/2682692467 shutdown_connections 2024-06-10T01:01:54.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.935+0000 7fee29655700 1 -- 172.21.15.7:0/2682692467 wait complete. 2024-06-10T01:01:54.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.936+0000 7fee29655700 1 Processor -- start 2024-06-10T01:01:54.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.936+0000 7fee29655700 1 -- start start 2024-06-10T01:01:54.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.936+0000 7fee29655700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee241027a0 0x7fee24074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:54.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.936+0000 7fee29655700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee24103150 0x7fee24074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:54.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.937+0000 7fee29655700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee24107790 0x7fee24075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:54.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.937+0000 7fee29655700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fee24112790 con 0x7fee241027a0 2024-06-10T01:01:54.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.937+0000 7fee29655700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fee24112610 con 0x7fee24103150 2024-06-10T01:01:54.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.937+0000 7fee22ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee241027a0 0x7fee24074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:54.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.937+0000 7fee29655700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fee24112910 con 0x7fee24107790 2024-06-10T01:01:54.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.937+0000 7fee22ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee241027a0 0x7fee24074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36638/0 (socket says 172.21.15.7:36638) 2024-06-10T01:01:54.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.937+0000 7fee22ffd700 1 -- 172.21.15.7:0/642632409 learned_addr learned my addr 172.21.15.7:0/642632409 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:01:54.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.937+0000 7fee227fc700 1 --2- 172.21.15.7:0/642632409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee24103150 0x7fee24074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:54.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.937+0000 7fee237fe700 1 --2- 172.21.15.7:0/642632409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee24107790 0x7fee24075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:54.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.937+0000 7fee22ffd700 1 -- 172.21.15.7:0/642632409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee24107790 msgr2=0x7fee24075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:54.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.937+0000 7fee22ffd700 1 --2- 172.21.15.7:0/642632409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee24107790 0x7fee24075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:54.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.937+0000 7fee22ffd700 1 -- 172.21.15.7:0/642632409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee24103150 msgr2=0x7fee24074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:54.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.937+0000 7fee22ffd700 1 --2- 172.21.15.7:0/642632409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee24103150 0x7fee24074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:54.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.937+0000 7fee22ffd700 1 -- 172.21.15.7:0/642632409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee241a70d0 con 0x7fee241027a0 2024-06-10T01:01:54.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.937+0000 7fee237fe700 1 --2- 172.21.15.7:0/642632409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee24107790 0x7fee24075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:01:54.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.937+0000 7fee22ffd700 1 --2- 172.21.15.7:0/642632409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee241027a0 0x7fee24074790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fee1400d950 tx=0x7fee1400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:54.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.937+0000 7fee0bfff700 1 -- 172.21.15.7:0/642632409 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee14016070 con 0x7fee241027a0 2024-06-10T01:01:54.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.938+0000 7fee29655700 1 -- 172.21.15.7:0/642632409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee241a73c0 con 0x7fee241027a0 2024-06-10T01:01:54.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.938+0000 7fee0bfff700 1 -- 172.21.15.7:0/642632409 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee14009e20 con 0x7fee241027a0 2024-06-10T01:01:54.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.938+0000 7fee0bfff700 1 -- 172.21.15.7:0/642632409 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1400f620 con 0x7fee241027a0 2024-06-10T01:01:54.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.938+0000 7fee29655700 1 -- 172.21.15.7:0/642632409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee241a78d0 con 0x7fee241027a0 2024-06-10T01:01:54.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.939+0000 7fee29655700 1 -- 172.21.15.7:0/642632409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee24050a00 con 0x7fee241027a0 2024-06-10T01:01:54.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.940+0000 7fee0bfff700 1 -- 172.21.15.7:0/642632409 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fee14019070 con 0x7fee241027a0 2024-06-10T01:01:54.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.941+0000 7fee0bfff700 1 --2- 172.21.15.7:0/642632409 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fee0c0a2cc0 0x7fee0c0a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:54.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.941+0000 7fee227fc700 1 --2- 172.21.15.7:0/642632409 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fee0c0a2cc0 0x7fee0c0a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:54.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.941+0000 7fee0bfff700 1 -- 172.21.15.7:0/642632409 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8277+0+0 (secure 0 0 0) 0x7fee140ceca0 con 0x7fee241027a0 2024-06-10T01:01:54.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.941+0000 7fee227fc700 1 --2- 172.21.15.7:0/642632409 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fee0c0a2cc0 0x7fee0c0a5180 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fee18009990 tx=0x7fee18005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:54.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:54.945+0000 7fee0bfff700 1 -- 172.21.15.7:0/642632409 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fee1409bb70 con 0x7fee241027a0 2024-06-10T01:01:55.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:55.151+0000 7fee29655700 1 -- 172.21.15.7:0/642632409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fee2402df10 con 0x7fee241027a0 2024-06-10T01:01:55.956 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:55 smithi175 ceph-mon[36636]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-06-10T01:01:55.956 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:55 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/642632409' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-10T01:01:56.016 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:55 smithi007 ceph-mon[32398]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-06-10T01:01:56.016 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:55 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/642632409' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-10T01:01:56.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:55 smithi078 ceph-mon[36683]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-06-10T01:01:56.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:55 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/642632409' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-10T01:01:56.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:56.263+0000 7fee0bfff700 1 -- 172.21.15.7:0/642632409 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v91) ==== 228+0+0 (secure 0 0 0) 0x7fee1409b470 con 0x7fee241027a0 2024-06-10T01:01:56.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:56.266+0000 7fee29655700 1 -- 172.21.15.7:0/642632409 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fee0c0a2cc0 msgr2=0x7fee0c0a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:56.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:56.266+0000 7fee29655700 1 --2- 172.21.15.7:0/642632409 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fee0c0a2cc0 0x7fee0c0a5180 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fee18009990 tx=0x7fee18005fb0 comp rx=0 tx=0).stop 2024-06-10T01:01:56.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:56.266+0000 7fee29655700 1 -- 172.21.15.7:0/642632409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee241027a0 msgr2=0x7fee24074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:56.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:56.266+0000 7fee29655700 1 --2- 172.21.15.7:0/642632409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee241027a0 0x7fee24074790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fee1400d950 tx=0x7fee1400dc60 comp rx=0 tx=0).stop 2024-06-10T01:01:56.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:56.266+0000 7fee29655700 1 -- 172.21.15.7:0/642632409 shutdown_connections 2024-06-10T01:01:56.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:56.266+0000 7fee29655700 1 --2- 172.21.15.7:0/642632409 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fee0c0a2cc0 0x7fee0c0a5180 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:56.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:56.266+0000 7fee29655700 1 --2- 172.21.15.7:0/642632409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee24107790 0x7fee24075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:56.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:56.266+0000 7fee29655700 1 --2- 172.21.15.7:0/642632409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee241027a0 0x7fee24074790 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:56.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:56.266+0000 7fee29655700 1 --2- 172.21.15.7:0/642632409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee24103150 0x7fee24074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:56.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:56.266+0000 7fee29655700 1 -- 172.21.15.7:0/642632409 >> 172.21.15.7:0/642632409 conn(0x7fee24079560 msgr2=0x7fee241016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:56.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:56.266+0000 7fee29655700 1 -- 172.21.15.7:0/642632409 shutdown_connections 2024-06-10T01:01:56.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:56.266+0000 7fee29655700 1 -- 172.21.15.7:0/642632409 wait complete. 2024-06-10T01:01:56.266 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data_ec' created 2024-06-10T01:01:56.956 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:56 smithi175 ceph-mon[36636]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-06-10T01:01:56.956 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:56 smithi175 ceph-mon[36636]: osdmap e90: 12 total, 12 up, 12 in 2024-06-10T01:01:56.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:56 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/642632409' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-10T01:01:56.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:56 smithi175 ceph-mon[36636]: osdmap e91: 12 total, 12 up, 12 in 2024-06-10T01:01:56.966 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:56 smithi007 ceph-mon[32398]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-06-10T01:01:56.966 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:56 smithi007 ceph-mon[32398]: osdmap e90: 12 total, 12 up, 12 in 2024-06-10T01:01:56.966 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:56 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/642632409' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-10T01:01:56.966 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:56 smithi007 ceph-mon[32398]: osdmap e91: 12 total, 12 up, 12 in 2024-06-10T01:01:57.000 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-06-10T01:01:57.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:56 smithi078 ceph-mon[36683]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-06-10T01:01:57.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:56 smithi078 ceph-mon[36683]: osdmap e90: 12 total, 12 up, 12 in 2024-06-10T01:01:57.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:56 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/642632409' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-10T01:01:57.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:56 smithi078 ceph-mon[36683]: osdmap e91: 12 total, 12 up, 12 in 2024-06-10T01:01:57.256 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:01:58.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:58 smithi007 ceph-mon[32398]: osdmap e92: 12 total, 12 up, 12 in 2024-06-10T01:01:58.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:58 smithi078 ceph-mon[36683]: osdmap e92: 12 total, 12 up, 12 in 2024-06-10T01:01:58.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:58 smithi175 ceph-mon[36636]: osdmap e92: 12 total, 12 up, 12 in 2024-06-10T01:01:58.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.973+0000 7fc3e7cea700 1 -- 172.21.15.7:0/1171621785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3e0107840 msgr2=0x7fc3e010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:58.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.973+0000 7fc3e7cea700 1 --2- 172.21.15.7:0/1171621785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3e0107840 0x7fc3e010f3c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc3d800b0d0 tx=0x7fc3d800b3e0 comp rx=0 tx=0).stop 2024-06-10T01:01:58.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.973+0000 7fc3e7cea700 1 -- 172.21.15.7:0/1171621785 shutdown_connections 2024-06-10T01:01:58.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.973+0000 7fc3e7cea700 1 --2- 172.21.15.7:0/1171621785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3e0107840 0x7fc3e010f3c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:58.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.973+0000 7fc3e7cea700 1 --2- 172.21.15.7:0/1171621785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3e0102740 0x7fc3e0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:58.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.973+0000 7fc3e7cea700 1 --2- 172.21.15.7:0/1171621785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3e01030f0 0x7fc3e0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:58.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.973+0000 7fc3e7cea700 1 -- 172.21.15.7:0/1171621785 >> 172.21.15.7:0/1171621785 conn(0x7fc3e00795f0 msgr2=0x7fc3e0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:58.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.973+0000 7fc3e7cea700 1 -- 172.21.15.7:0/1171621785 shutdown_connections 2024-06-10T01:01:58.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.973+0000 7fc3e7cea700 1 -- 172.21.15.7:0/1171621785 wait complete. 2024-06-10T01:01:58.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.974+0000 7fc3e7cea700 1 Processor -- start 2024-06-10T01:01:58.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.974+0000 7fc3e7cea700 1 -- start start 2024-06-10T01:01:58.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.975+0000 7fc3e7cea700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3e0102740 0x7fc3e019e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:58.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.975+0000 7fc3e7cea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3e01030f0 0x7fc3e019ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:58.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.975+0000 7fc3e7cea700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3e0107840 0x7fc3e0198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:58.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.975+0000 7fc3e7cea700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e0112760 con 0x7fc3e01030f0 2024-06-10T01:01:58.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.975+0000 7fc3e7cea700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e01125e0 con 0x7fc3e0102740 2024-06-10T01:01:58.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.975+0000 7fc3e7cea700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e01128e0 con 0x7fc3e0107840 2024-06-10T01:01:58.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.975+0000 7fc3e5285700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3e01030f0 0x7fc3e019ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:58.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.975+0000 7fc3e5a86700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3e0102740 0x7fc3e019e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:58.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.975+0000 7fc3e5285700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3e01030f0 0x7fc3e019ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36662/0 (socket says 172.21.15.7:36662) 2024-06-10T01:01:58.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.975+0000 7fc3e5285700 1 -- 172.21.15.7:0/3891268328 learned_addr learned my addr 172.21.15.7:0/3891268328 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:01:58.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.975+0000 7fc3e6287700 1 --2- 172.21.15.7:0/3891268328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3e0107840 0x7fc3e0198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:58.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.975+0000 7fc3e5285700 1 -- 172.21.15.7:0/3891268328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3e0107840 msgr2=0x7fc3e0198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:58.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.975+0000 7fc3e5285700 1 --2- 172.21.15.7:0/3891268328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3e0107840 0x7fc3e0198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:58.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.975+0000 7fc3e5285700 1 -- 172.21.15.7:0/3891268328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3e0102740 msgr2=0x7fc3e019e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:58.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.975+0000 7fc3e5285700 1 --2- 172.21.15.7:0/3891268328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3e0102740 0x7fc3e019e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:58.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.975+0000 7fc3e5285700 1 -- 172.21.15.7:0/3891268328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e01990e0 con 0x7fc3e01030f0 2024-06-10T01:01:58.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.975+0000 7fc3e5285700 1 --2- 172.21.15.7:0/3891268328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3e01030f0 0x7fc3e019ecb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000d950 tx=0x7fc3d000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:58.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.975+0000 7fc3d6ffd700 1 -- 172.21.15.7:0/3891268328 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0016070 con 0x7fc3e01030f0 2024-06-10T01:01:58.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.975+0000 7fc3d6ffd700 1 -- 172.21.15.7:0/3891268328 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3d000fd70 con 0x7fc3e01030f0 2024-06-10T01:01:58.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.975+0000 7fc3e7cea700 1 -- 172.21.15.7:0/3891268328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e01993d0 con 0x7fc3e01030f0 2024-06-10T01:01:58.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.975+0000 7fc3d6ffd700 1 -- 172.21.15.7:0/3891268328 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d000e610 con 0x7fc3e01030f0 2024-06-10T01:01:58.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.976+0000 7fc3e7cea700 1 -- 172.21.15.7:0/3891268328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e01a5640 con 0x7fc3e01030f0 2024-06-10T01:01:58.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.977+0000 7fc3e7cea700 1 -- 172.21.15.7:0/3891268328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e0069180 con 0x7fc3e01030f0 2024-06-10T01:01:58.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.979+0000 7fc3d6ffd700 1 -- 172.21.15.7:0/3891268328 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fc3d0019070 con 0x7fc3e01030f0 2024-06-10T01:01:58.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.979+0000 7fc3d6ffd700 1 --2- 172.21.15.7:0/3891268328 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc3cc0a2ca0 0x7fc3cc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:58.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.979+0000 7fc3e5a86700 1 --2- 172.21.15.7:0/3891268328 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc3cc0a2ca0 0x7fc3cc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:58.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.979+0000 7fc3d6ffd700 1 -- 172.21.15.7:0/3891268328 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8770+0+0 (secure 0 0 0) 0x7fc3d00cec40 con 0x7fc3e01030f0 2024-06-10T01:01:58.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.980+0000 7fc3e5a86700 1 --2- 172.21.15.7:0/3891268328 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc3cc0a2ca0 0x7fc3cc0a5160 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0107350 tx=0x7fc3dc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:58.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:58.983+0000 7fc3d6ffd700 1 -- 172.21.15.7:0/3891268328 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc3d009b980 con 0x7fc3e01030f0 2024-06-10T01:01:59.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:59.202+0000 7fc3e7cea700 1 -- 172.21.15.7:0/3891268328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fc3e002de60 con 0x7fc3e01030f0 2024-06-10T01:01:59.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:59.276+0000 7fc3d6ffd700 1 -- 172.21.15.7:0/3891268328 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v94) ==== 169+0+0 (secure 0 0 0) 0x7fc3d009b280 con 0x7fc3e01030f0 2024-06-10T01:01:59.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:59.279+0000 7fc3e7cea700 1 -- 172.21.15.7:0/3891268328 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc3cc0a2ca0 msgr2=0x7fc3cc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:59.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:59.279+0000 7fc3e7cea700 1 --2- 172.21.15.7:0/3891268328 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc3cc0a2ca0 0x7fc3cc0a5160 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0107350 tx=0x7fc3dc00b410 comp rx=0 tx=0).stop 2024-06-10T01:01:59.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:59.279+0000 7fc3e7cea700 1 -- 172.21.15.7:0/3891268328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3e01030f0 msgr2=0x7fc3e019ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:59.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:59.279+0000 7fc3e7cea700 1 --2- 172.21.15.7:0/3891268328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3e01030f0 0x7fc3e019ecb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000d950 tx=0x7fc3d000dc60 comp rx=0 tx=0).stop 2024-06-10T01:01:59.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:59.279+0000 7fc3e7cea700 1 -- 172.21.15.7:0/3891268328 shutdown_connections 2024-06-10T01:01:59.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:59.279+0000 7fc3e7cea700 1 --2- 172.21.15.7:0/3891268328 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc3cc0a2ca0 0x7fc3cc0a5160 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:59.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:59.279+0000 7fc3e7cea700 1 --2- 172.21.15.7:0/3891268328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3e0107840 0x7fc3e0198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:59.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:59.279+0000 7fc3e7cea700 1 --2- 172.21.15.7:0/3891268328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3e0102740 0x7fc3e019e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:59.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:59.279+0000 7fc3e7cea700 1 --2- 172.21.15.7:0/3891268328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3e01030f0 0x7fc3e019ecb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:59.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:59.279+0000 7fc3e7cea700 1 -- 172.21.15.7:0/3891268328 >> 172.21.15.7:0/3891268328 conn(0x7fc3e00795f0 msgr2=0x7fc3e0100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:59.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:59.279+0000 7fc3e7cea700 1 -- 172.21.15.7:0/3891268328 shutdown_connections 2024-06-10T01:01:59.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:01:59.279+0000 7fc3e7cea700 1 -- 172.21.15.7:0/3891268328 wait complete. 2024-06-10T01:01:59.278 INFO:teuthology.orchestra.run.smithi007.stderr:set pool 4 allow_ec_overwrites to true 2024-06-10T01:01:59.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:59 smithi078 ceph-mon[36683]: pgmap v303: 129 pgs: 2 creating+peering, 30 unknown, 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-06-10T01:01:59.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:59 smithi078 ceph-mon[36683]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-10T01:01:59.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:59 smithi078 ceph-mon[36683]: osdmap e93: 12 total, 12 up, 12 in 2024-06-10T01:01:59.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:01:59 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3891268328' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-10T01:01:59.640 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:59 smithi007 ceph-mon[32398]: pgmap v303: 129 pgs: 2 creating+peering, 30 unknown, 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-06-10T01:01:59.640 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:59 smithi007 ceph-mon[32398]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-10T01:01:59.640 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:59 smithi007 ceph-mon[32398]: osdmap e93: 12 total, 12 up, 12 in 2024-06-10T01:01:59.640 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:01:59 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3891268328' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-10T01:01:59.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:59 smithi175 ceph-mon[36636]: pgmap v303: 129 pgs: 2 creating+peering, 30 unknown, 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-06-10T01:01:59.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:59 smithi175 ceph-mon[36636]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-10T01:01:59.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:59 smithi175 ceph-mon[36636]: osdmap e93: 12 total, 12 up, 12 in 2024-06-10T01:01:59.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:01:59 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3891268328' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-10T01:02:00.041 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-06-10T01:02:00.296 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:02:00.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:00 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3891268328' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-10T01:02:00.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:00 smithi007 ceph-mon[32398]: osdmap e94: 12 total, 12 up, 12 in 2024-06-10T01:02:00.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:00 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3891268328' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-10T01:02:00.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:00 smithi078 ceph-mon[36683]: osdmap e94: 12 total, 12 up, 12 in 2024-06-10T01:02:00.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:00 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3891268328' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-10T01:02:00.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:00 smithi175 ceph-mon[36636]: osdmap e94: 12 total, 12 up, 12 in 2024-06-10T01:02:01.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:01 smithi078 ceph-mon[36683]: pgmap v306: 129 pgs: 2 creating+peering, 7 unknown, 120 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:01.690 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:01 smithi007 ceph-mon[32398]: pgmap v306: 129 pgs: 2 creating+peering, 7 unknown, 120 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:01.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:01 smithi175 ceph-mon[36636]: pgmap v306: 129 pgs: 2 creating+peering, 7 unknown, 120 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:02.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.040+0000 7f68099e0700 1 -- 172.21.15.7:0/1967538434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6804101d50 msgr2=0x7f680410e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:02.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.040+0000 7f68099e0700 1 --2- 172.21.15.7:0/1967538434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6804101d50 0x7f680410e240 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009990 tx=0x7f67fc009ca0 comp rx=0 tx=0).stop 2024-06-10T01:02:02.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.040+0000 7f68099e0700 1 -- 172.21.15.7:0/1967538434 shutdown_connections 2024-06-10T01:02:02.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.040+0000 7f68099e0700 1 --2- 172.21.15.7:0/1967538434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6804101d50 0x7f680410e240 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.040+0000 7f68099e0700 1 --2- 172.21.15.7:0/1967538434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6804107400 0x7f68041077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.040+0000 7f68099e0700 1 --2- 172.21.15.7:0/1967538434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6804101390 0x7f6804101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.040+0000 7f68099e0700 1 -- 172.21.15.7:0/1967538434 >> 172.21.15.7:0/1967538434 conn(0x7f6804078340 msgr2=0x7f6804078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.040+0000 7f68099e0700 1 -- 172.21.15.7:0/1967538434 shutdown_connections 2024-06-10T01:02:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.040+0000 7f68099e0700 1 -- 172.21.15.7:0/1967538434 wait complete. 2024-06-10T01:02:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.041+0000 7f68099e0700 1 Processor -- start 2024-06-10T01:02:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.041+0000 7f68099e0700 1 -- start start 2024-06-10T01:02:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.041+0000 7f68099e0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6804101390 0x7f680419ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.041+0000 7f68099e0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6804101d50 0x7f680419b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.041+0000 7f68099e0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6804107400 0x7f680419f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.041+0000 7f68099e0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f68041116c0 con 0x7f6804101d50 2024-06-10T01:02:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.041+0000 7f68099e0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6804111540 con 0x7f6804101390 2024-06-10T01:02:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.041+0000 7f68099e0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6804111840 con 0x7f6804107400 2024-06-10T01:02:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.041+0000 7f68027fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6804101d50 0x7f680419b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.041+0000 7f68027fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6804101d50 0x7f680419b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47876/0 (socket says 172.21.15.7:47876) 2024-06-10T01:02:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.041+0000 7f68027fc700 1 -- 172.21.15.7:0/398291072 learned_addr learned my addr 172.21.15.7:0/398291072 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:02:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.041+0000 7f6802ffd700 1 --2- 172.21.15.7:0/398291072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6804101390 0x7f680419ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.041+0000 7f68037fe700 1 --2- 172.21.15.7:0/398291072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6804107400 0x7f680419f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.041+0000 7f68027fc700 1 -- 172.21.15.7:0/398291072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6804107400 msgr2=0x7f680419f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.041+0000 7f68027fc700 1 --2- 172.21.15.7:0/398291072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6804107400 0x7f680419f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.041+0000 7f68027fc700 1 -- 172.21.15.7:0/398291072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6804101390 msgr2=0x7f680419ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.041+0000 7f68027fc700 1 --2- 172.21.15.7:0/398291072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6804101390 0x7f680419ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.041+0000 7f68027fc700 1 -- 172.21.15.7:0/398291072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680419fd80 con 0x7f6804101d50 2024-06-10T01:02:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.041+0000 7f6802ffd700 1 --2- 172.21.15.7:0/398291072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6804101390 0x7f680419ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:02:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.042+0000 7f68037fe700 1 --2- 172.21.15.7:0/398291072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6804107400 0x7f680419f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:02:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.042+0000 7f68027fc700 1 --2- 172.21.15.7:0/398291072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6804101d50 0x7f680419b2b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f67f800c9b0 tx=0x7f67f800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.042+0000 7f68089de700 1 -- 172.21.15.7:0/398291072 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f8007b00 con 0x7f6804101d50 2024-06-10T01:02:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.042+0000 7f68089de700 1 -- 172.21.15.7:0/398291072 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67f8007ca0 con 0x7f6804101d50 2024-06-10T01:02:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.042+0000 7f68099e0700 1 -- 172.21.15.7:0/398291072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68041a0070 con 0x7f6804101d50 2024-06-10T01:02:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.042+0000 7f68089de700 1 -- 172.21.15.7:0/398291072 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f8018780 con 0x7f6804101d50 2024-06-10T01:02:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.042+0000 7f68099e0700 1 -- 172.21.15.7:0/398291072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68041a78f0 con 0x7f6804101d50 2024-06-10T01:02:02.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.043+0000 7f68099e0700 1 -- 172.21.15.7:0/398291072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6804069180 con 0x7f6804101d50 2024-06-10T01:02:02.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.044+0000 7f68089de700 1 -- 172.21.15.7:0/398291072 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f67f800f450 con 0x7f6804101d50 2024-06-10T01:02:02.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.045+0000 7f68089de700 1 --2- 172.21.15.7:0/398291072 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f67ec0a2c20 0x7f67ec0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:02.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.045+0000 7f6802ffd700 1 --2- 172.21.15.7:0/398291072 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f67ec0a2c20 0x7f67ec0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:02.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.045+0000 7f68089de700 1 -- 172.21.15.7:0/398291072 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8770+0+0 (secure 0 0 0) 0x7f67f80cf590 con 0x7f6804101d50 2024-06-10T01:02:02.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.046+0000 7f6802ffd700 1 --2- 172.21.15.7:0/398291072 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f67ec0a2c20 0x7f67ec0a50e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f67f4009d50 tx=0x7f67f4009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:02.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.049+0000 7f68089de700 1 -- 172.21.15.7:0/398291072 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f67f8014030 con 0x7f6804101d50 2024-06-10T01:02:02.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:02.302+0000 7f68099e0700 1 -- 172.21.15.7:0/398291072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f68041a7b50 con 0x7f6804101d50 2024-06-10T01:02:03.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:03.290+0000 7f68089de700 1 -- 172.21.15.7:0/398291072 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f67f809c360 con 0x7f6804101d50 2024-06-10T01:02:03.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:03.293+0000 7f68099e0700 1 -- 172.21.15.7:0/398291072 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f67ec0a2c20 msgr2=0x7f67ec0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:03.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:03.293+0000 7f68099e0700 1 --2- 172.21.15.7:0/398291072 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f67ec0a2c20 0x7f67ec0a50e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f67f4009d50 tx=0x7f67f4009040 comp rx=0 tx=0).stop 2024-06-10T01:02:03.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:03.293+0000 7f68099e0700 1 -- 172.21.15.7:0/398291072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6804101d50 msgr2=0x7f680419b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:03.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:03.293+0000 7f68099e0700 1 --2- 172.21.15.7:0/398291072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6804101d50 0x7f680419b2b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f67f800c9b0 tx=0x7f67f800cd70 comp rx=0 tx=0).stop 2024-06-10T01:02:03.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:03.293+0000 7f68099e0700 1 -- 172.21.15.7:0/398291072 shutdown_connections 2024-06-10T01:02:03.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:03.294+0000 7f68099e0700 1 --2- 172.21.15.7:0/398291072 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f67ec0a2c20 0x7f67ec0a50e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:03.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:03.294+0000 7f68099e0700 1 --2- 172.21.15.7:0/398291072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6804107400 0x7f680419f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:03.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:03.294+0000 7f68099e0700 1 --2- 172.21.15.7:0/398291072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6804101390 0x7f680419ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:03.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:03.294+0000 7f68099e0700 1 --2- 172.21.15.7:0/398291072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6804101d50 0x7f680419b2b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:03.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:03.294+0000 7f68099e0700 1 -- 172.21.15.7:0/398291072 >> 172.21.15.7:0/398291072 conn(0x7f6804078340 msgr2=0x7f680406a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:03.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:03.294+0000 7f68099e0700 1 -- 172.21.15.7:0/398291072 shutdown_connections 2024-06-10T01:02:03.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:03.294+0000 7f68099e0700 1 -- 172.21.15.7:0/398291072 wait complete. 2024-06-10T01:02:03.294 INFO:teuthology.orchestra.run.smithi007.stderr:added data pool 4 to fsmap 2024-06-10T01:02:03.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:03 smithi078 ceph-mon[36683]: pgmap v307: 129 pgs: 2 creating+peering, 127 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:03.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:03 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/398291072' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-10T01:02:03.616 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:03 smithi007 ceph-mon[32398]: pgmap v307: 129 pgs: 2 creating+peering, 127 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:03.616 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:03 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/398291072' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-10T01:02:03.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:03 smithi175 ceph-mon[36636]: pgmap v307: 129 pgs: 2 creating+peering, 127 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:03.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:03 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/398291072' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-10T01:02:03.953 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:02:04.211 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:02:04.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:04 smithi007 ceph-mon[32398]: osdmap e95: 12 total, 12 up, 12 in 2024-06-10T01:02:04.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:04 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/398291072' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-10T01:02:04.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:04 smithi007 ceph-mon[32398]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T01:02:04.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:04 smithi078 ceph-mon[36683]: osdmap e95: 12 total, 12 up, 12 in 2024-06-10T01:02:04.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:04 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/398291072' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-10T01:02:04.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:04 smithi078 ceph-mon[36683]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T01:02:04.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:04 smithi175 ceph-mon[36636]: osdmap e95: 12 total, 12 up, 12 in 2024-06-10T01:02:04.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:04 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/398291072' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-10T01:02:04.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:04 smithi175 ceph-mon[36636]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T01:02:05.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:05 smithi078 ceph-mon[36683]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:05.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:05 smithi078 ceph-mon[36683]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-10T01:02:05.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:05 smithi078 ceph-mon[36683]: Cluster is now healthy 2024-06-10T01:02:05.640 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:05 smithi007 ceph-mon[32398]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:05.640 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:05 smithi007 ceph-mon[32398]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-10T01:02:05.640 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:05 smithi007 ceph-mon[32398]: Cluster is now healthy 2024-06-10T01:02:05.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:05 smithi175 ceph-mon[36636]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:05.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:05 smithi175 ceph-mon[36636]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-10T01:02:05.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:05 smithi175 ceph-mon[36636]: Cluster is now healthy 2024-06-10T01:02:05.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.871+0000 7fb8970a3700 1 -- 172.21.15.7:0/2608524392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb890108580 msgr2=0x7fb890108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:05.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.871+0000 7fb8970a3700 1 --2- 172.21.15.7:0/2608524392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb890108580 0x7fb890108960 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb8800099c0 tx=0x7fb880009cd0 comp rx=0 tx=0).stop 2024-06-10T01:02:05.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.871+0000 7fb8970a3700 1 -- 172.21.15.7:0/2608524392 shutdown_connections 2024-06-10T01:02:05.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.871+0000 7fb8970a3700 1 --2- 172.21.15.7:0/2608524392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb89006b620 0x7fb89010f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:05.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.871+0000 7fb8970a3700 1 --2- 172.21.15.7:0/2608524392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb890108580 0x7fb890108960 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:05.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.871+0000 7fb8970a3700 1 --2- 172.21.15.7:0/2608524392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb89006ac60 0x7fb89006b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:05.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.871+0000 7fb8970a3700 1 -- 172.21.15.7:0/2608524392 >> 172.21.15.7:0/2608524392 conn(0x7fb890078430 msgr2=0x7fb890078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:05.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.871+0000 7fb8970a3700 1 -- 172.21.15.7:0/2608524392 shutdown_connections 2024-06-10T01:02:05.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.872+0000 7fb8970a3700 1 -- 172.21.15.7:0/2608524392 wait complete. 2024-06-10T01:02:05.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.872+0000 7fb8970a3700 1 Processor -- start 2024-06-10T01:02:05.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.872+0000 7fb8970a3700 1 -- start start 2024-06-10T01:02:05.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.872+0000 7fb8970a3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb89006ac60 0x7fb890101400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:05.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.872+0000 7fb8970a3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb89006b620 0x7fb890101940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:05.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.872+0000 7fb8970a3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb890108580 0x7fb890101e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:05.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.872+0000 7fb8970a3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb8901127c0 con 0x7fb89006ac60 2024-06-10T01:02:05.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.872+0000 7fb8970a3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb890112640 con 0x7fb890108580 2024-06-10T01:02:05.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.872+0000 7fb8970a3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb890112940 con 0x7fb89006b620 2024-06-10T01:02:05.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.872+0000 7fb894e3f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb89006ac60 0x7fb890101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:05.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.872+0000 7fb894e3f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb89006ac60 0x7fb890101400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47896/0 (socket says 172.21.15.7:47896) 2024-06-10T01:02:05.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.872+0000 7fb894e3f700 1 -- 172.21.15.7:0/198188354 learned_addr learned my addr 172.21.15.7:0/198188354 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:02:05.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.873+0000 7fb88ffff700 1 --2- 172.21.15.7:0/198188354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb89006b620 0x7fb890101940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:05.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.873+0000 7fb895640700 1 --2- 172.21.15.7:0/198188354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb890108580 0x7fb890101e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:05.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.873+0000 7fb894e3f700 1 -- 172.21.15.7:0/198188354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb89006b620 msgr2=0x7fb890101940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:05.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.873+0000 7fb894e3f700 1 --2- 172.21.15.7:0/198188354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb89006b620 0x7fb890101940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:05.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.873+0000 7fb894e3f700 1 -- 172.21.15.7:0/198188354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb890108580 msgr2=0x7fb890101e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:05.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.873+0000 7fb894e3f700 1 --2- 172.21.15.7:0/198188354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb890108580 0x7fb890101e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:05.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.873+0000 7fb894e3f700 1 -- 172.21.15.7:0/198188354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8901a7120 con 0x7fb89006ac60 2024-06-10T01:02:05.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.873+0000 7fb894e3f700 1 --2- 172.21.15.7:0/198188354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb89006ac60 0x7fb890101400 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb880006010 tx=0x7fb880009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:05.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.873+0000 7fb88dffb700 1 -- 172.21.15.7:0/198188354 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb880027070 con 0x7fb89006ac60 2024-06-10T01:02:05.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.873+0000 7fb8970a3700 1 -- 172.21.15.7:0/198188354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8901a73b0 con 0x7fb89006ac60 2024-06-10T01:02:05.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.873+0000 7fb88dffb700 1 -- 172.21.15.7:0/198188354 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb880003780 con 0x7fb89006ac60 2024-06-10T01:02:05.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.873+0000 7fb88dffb700 1 -- 172.21.15.7:0/198188354 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb880022da0 con 0x7fb89006ac60 2024-06-10T01:02:05.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.873+0000 7fb8970a3700 1 -- 172.21.15.7:0/198188354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8901a7860 con 0x7fb89006ac60 2024-06-10T01:02:05.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.874+0000 7fb8970a3700 1 -- 172.21.15.7:0/198188354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb89010aa10 con 0x7fb89006ac60 2024-06-10T01:02:05.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.875+0000 7fb88dffb700 1 -- 172.21.15.7:0/198188354 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fb88002a070 con 0x7fb89006ac60 2024-06-10T01:02:05.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.876+0000 7fb88dffb700 1 --2- 172.21.15.7:0/198188354 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb8780a2ca0 0x7fb8780a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:05.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.876+0000 7fb88ffff700 1 --2- 172.21.15.7:0/198188354 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb8780a2ca0 0x7fb8780a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:05.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.876+0000 7fb88dffb700 1 -- 172.21.15.7:0/198188354 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7fb8800d0200 con 0x7fb89006ac60 2024-06-10T01:02:05.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.877+0000 7fb88ffff700 1 --2- 172.21.15.7:0/198188354 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb8780a2ca0 0x7fb8780a5160 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb890078a30 tx=0x7fb88400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:05.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:05.880+0000 7fb88dffb700 1 -- 172.21.15.7:0/198188354 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb880031080 con 0x7fb89006ac60 2024-06-10T01:02:06.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:06.143+0000 7fb8970a3700 1 -- 172.21.15.7:0/198188354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb89010a3f0 con 0x7fb89006ac60 2024-06-10T01:02:06.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:06.144+0000 7fb88dffb700 1 -- 172.21.15.7:0/198188354 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11782 (secure 0 0 0) 0x7fb880013070 con 0x7fb89006ac60 2024-06-10T01:02:06.143 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:02:06.143 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2784910673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2784910673},{"type":"v1","addr":"172.21.15.175:6835","nonce":2784910673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24434,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:02:02.305093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24451,"qdb_cluster":[24451]},"id":1}]} 2024-06-10T01:02:06.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:06.146+0000 7fb8970a3700 1 -- 172.21.15.7:0/198188354 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb8780a2ca0 msgr2=0x7fb8780a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:06.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:06.146+0000 7fb8970a3700 1 --2- 172.21.15.7:0/198188354 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb8780a2ca0 0x7fb8780a5160 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb890078a30 tx=0x7fb88400b410 comp rx=0 tx=0).stop 2024-06-10T01:02:06.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:06.146+0000 7fb8970a3700 1 -- 172.21.15.7:0/198188354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb89006ac60 msgr2=0x7fb890101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:06.146+0000 7fb8970a3700 1 --2- 172.21.15.7:0/198188354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb89006ac60 0x7fb890101400 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb880006010 tx=0x7fb880009f80 comp rx=0 tx=0).stop 2024-06-10T01:02:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:06.146+0000 7fb8970a3700 1 -- 172.21.15.7:0/198188354 shutdown_connections 2024-06-10T01:02:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:06.146+0000 7fb8970a3700 1 --2- 172.21.15.7:0/198188354 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb8780a2ca0 0x7fb8780a5160 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:06.146+0000 7fb8970a3700 1 --2- 172.21.15.7:0/198188354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb890108580 0x7fb890101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:06.146+0000 7fb8970a3700 1 --2- 172.21.15.7:0/198188354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb89006ac60 0x7fb890101400 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:06.146+0000 7fb8970a3700 1 --2- 172.21.15.7:0/198188354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb89006b620 0x7fb890101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:06.146+0000 7fb8970a3700 1 -- 172.21.15.7:0/198188354 >> 172.21.15.7:0/198188354 conn(0x7fb890078430 msgr2=0x7fb8900fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:06.146+0000 7fb8970a3700 1 -- 172.21.15.7:0/198188354 shutdown_connections 2024-06-10T01:02:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:06.146+0000 7fb8970a3700 1 -- 172.21.15.7:0/198188354 wait complete. 2024-06-10T01:02:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 17 2024-06-10T01:02:06.516 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:06 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/198188354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:02:06.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:06 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/198188354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:02:06.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:06 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/198188354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:02:06.776 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T01:02:07.031 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:02:07.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:07 smithi007 ceph-mon[32398]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:07.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:07 smithi078 ceph-mon[36683]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:07.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:07 smithi175 ceph-mon[36636]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:08.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.896+0000 7f401b9e7700 1 -- 172.21.15.7:0/643656500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4014102130 msgr2=0x7f401410e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:08.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.896+0000 7f401b9e7700 1 --2- 172.21.15.7:0/643656500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4014102130 0x7f401410e190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f400c00b0a0 tx=0x7f400c00b3b0 comp rx=0 tx=0).stop 2024-06-10T01:02:08.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.896+0000 7f401b9e7700 1 -- 172.21.15.7:0/643656500 shutdown_connections 2024-06-10T01:02:08.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.896+0000 7f401b9e7700 1 --2- 172.21.15.7:0/643656500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4014102130 0x7f401410e190 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:08.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.896+0000 7f401b9e7700 1 --2- 172.21.15.7:0/643656500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4014107550 0x7f4014107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:08.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.896+0000 7f401b9e7700 1 --2- 172.21.15.7:0/643656500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4014101770 0x7f4014101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:08.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.896+0000 7f401b9e7700 1 -- 172.21.15.7:0/643656500 >> 172.21.15.7:0/643656500 conn(0x7f4014078340 msgr2=0x7f4014078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:08.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.896+0000 7f401b9e7700 1 -- 172.21.15.7:0/643656500 shutdown_connections 2024-06-10T01:02:08.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.896+0000 7f401b9e7700 1 -- 172.21.15.7:0/643656500 wait complete. 2024-06-10T01:02:08.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.897+0000 7f401b9e7700 1 Processor -- start 2024-06-10T01:02:08.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.897+0000 7f401b9e7700 1 -- start start 2024-06-10T01:02:08.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.897+0000 7f401b9e7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4014101770 0x7f4014103080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:08.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.897+0000 7f401b9e7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4014102130 0x7f40141035c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:08.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.897+0000 7f401b9e7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4014107550 0x7f4014103b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:08.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.897+0000 7f401b9e7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f40141116d0 con 0x7f4014102130 2024-06-10T01:02:08.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.897+0000 7f401b9e7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4014111550 con 0x7f4014101770 2024-06-10T01:02:08.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.897+0000 7f401b9e7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4014111850 con 0x7f4014107550 2024-06-10T01:02:08.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.897+0000 7f4018f82700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4014102130 0x7f40141035c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:08.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.897+0000 7f4018f82700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4014102130 0x7f40141035c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47918/0 (socket says 172.21.15.7:47918) 2024-06-10T01:02:08.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.897+0000 7f4018f82700 1 -- 172.21.15.7:0/3289505728 learned_addr learned my addr 172.21.15.7:0/3289505728 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:02:08.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.897+0000 7f4019783700 1 --2- 172.21.15.7:0/3289505728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4014101770 0x7f4014103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:08.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.897+0000 7f4019f84700 1 --2- 172.21.15.7:0/3289505728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4014107550 0x7f4014103b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:08.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.897+0000 7f4018f82700 1 -- 172.21.15.7:0/3289505728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4014107550 msgr2=0x7f4014103b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:08.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.897+0000 7f4018f82700 1 --2- 172.21.15.7:0/3289505728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4014107550 0x7f4014103b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:08.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.897+0000 7f4018f82700 1 -- 172.21.15.7:0/3289505728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4014101770 msgr2=0x7f4014103080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:08.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.897+0000 7f4018f82700 1 --2- 172.21.15.7:0/3289505728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4014101770 0x7f4014103080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:08.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.898+0000 7f4018f82700 1 -- 172.21.15.7:0/3289505728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40141a7020 con 0x7f4014102130 2024-06-10T01:02:08.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.898+0000 7f4018f82700 1 --2- 172.21.15.7:0/3289505728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4014102130 0x7f40141035c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f400400d950 tx=0x7f400400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:08.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.898+0000 7f400a7fc700 1 -- 172.21.15.7:0/3289505728 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4004016070 con 0x7f4014102130 2024-06-10T01:02:08.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.898+0000 7f401b9e7700 1 -- 172.21.15.7:0/3289505728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40141a7310 con 0x7f4014102130 2024-06-10T01:02:08.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.898+0000 7f400a7fc700 1 -- 172.21.15.7:0/3289505728 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f400400fd70 con 0x7f4014102130 2024-06-10T01:02:08.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.898+0000 7f400a7fc700 1 -- 172.21.15.7:0/3289505728 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400400e610 con 0x7f4014102130 2024-06-10T01:02:08.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.898+0000 7f401b9e7700 1 -- 172.21.15.7:0/3289505728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40141a7850 con 0x7f4014102130 2024-06-10T01:02:08.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.899+0000 7f401b9e7700 1 -- 172.21.15.7:0/3289505728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4014069180 con 0x7f4014102130 2024-06-10T01:02:08.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.900+0000 7f400a7fc700 1 -- 172.21.15.7:0/3289505728 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f4004019070 con 0x7f4014102130 2024-06-10T01:02:08.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.901+0000 7f400a7fc700 1 --2- 172.21.15.7:0/3289505728 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f40000a2c20 0x7f40000a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:08.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.901+0000 7f4019783700 1 --2- 172.21.15.7:0/3289505728 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f40000a2c20 0x7f40000a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:08.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.901+0000 7f400a7fc700 1 -- 172.21.15.7:0/3289505728 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7f40040ced30 con 0x7f4014102130 2024-06-10T01:02:08.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.901+0000 7f4019783700 1 --2- 172.21.15.7:0/3289505728 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f40000a2c20 0x7f40000a50e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f401000abd0 tx=0x7f401000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:08.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:08.905+0000 7f400a7fc700 1 -- 172.21.15.7:0/3289505728 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f400409ba50 con 0x7f4014102130 2024-06-10T01:02:09.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:09.113+0000 7f401b9e7700 1 -- 172.21.15.7:0/3289505728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f40141a7ab0 con 0x7f4014102130 2024-06-10T01:02:09.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:09.115+0000 7f400a7fc700 1 -- 172.21.15.7:0/3289505728 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24022 (secure 0 0 0) 0x7f400409b350 con 0x7f4014102130 2024-06-10T01:02:09.114 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:02:09.115 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":95,"fsid":"bded58f8-26c3-11ef-bc9b-c7b262605968","created":"2024-06-10T00:53:31.966085+0000","modified":"2024-06-10T01:02:02.304963+0000","last_up_change":"2024-06-10T01:00:45.382473+0000","last_in_change":"2024-06-10T01:00:25.608513+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:56:59.964055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T01:01:42.563611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T01:01:46.175918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T01:01:55.695502+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f7c24c9f-b7ab-4c04-b2b4-120df9751667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6803","nonce":404485726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6805","nonce":404485726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6809","nonce":404485726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6807","nonce":404485726}]},"public_addr":"172.21.15.7:6803/404485726","cluster_addr":"172.21.15.7:6805/404485726","heartbeat_back_addr":"172.21.15.7:6809/404485726","heartbeat_front_addr":"172.21.15.7:6807/404485726","state":["exists","up"]},{"osd":1,"uuid":"9bf70a6b-f31c-4077-b5af-229f17bc2845","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6801","nonce":1856804448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6803","nonce":1856804448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6807","nonce":1856804448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6805","nonce":1856804448}]},"public_addr":"172.21.15.78:6801/1856804448","cluster_addr":"172.21.15.78:6803/1856804448","heartbeat_back_addr":"172.21.15.78:6807/1856804448","heartbeat_front_addr":"172.21.15.78:6805/1856804448","state":["exists","up"]},{"osd":2,"uuid":"62923c97-caf9-4f7e-9c14-9c9918843c78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6801","nonce":918257455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6803","nonce":918257455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6807","nonce":918257455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6805","nonce":918257455}]},"public_addr":"172.21.15.175:6801/918257455","cluster_addr":"172.21.15.175:6803/918257455","heartbeat_back_addr":"172.21.15.175:6807/918257455","heartbeat_front_addr":"172.21.15.175:6805/918257455","state":["exists","up"]},{"osd":3,"uuid":"65242098-e748-48ab-8f0e-bb25b5c62742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6811","nonce":1797404966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6813","nonce":1797404966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6817","nonce":1797404966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6815","nonce":1797404966}]},"public_addr":"172.21.15.7:6811/1797404966","cluster_addr":"172.21.15.7:6813/1797404966","heartbeat_back_addr":"172.21.15.7:6817/1797404966","heartbeat_front_addr":"172.21.15.7:6815/1797404966","state":["exists","up"]},{"osd":4,"uuid":"a7990305-de9e-400e-a432-d7854f73f339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6809","nonce":3767608357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6811","nonce":3767608357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6815","nonce":3767608357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6813","nonce":3767608357}]},"public_addr":"172.21.15.78:6809/3767608357","cluster_addr":"172.21.15.78:6811/3767608357","heartbeat_back_addr":"172.21.15.78:6815/3767608357","heartbeat_front_addr":"172.21.15.78:6813/3767608357","state":["exists","up"]},{"osd":5,"uuid":"bad9a7b3-da5a-48d0-905d-e6a3190317fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6809","nonce":2157664731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6811","nonce":2157664731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6815","nonce":2157664731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6813","nonce":2157664731}]},"public_addr":"172.21.15.175:6809/2157664731","cluster_addr":"172.21.15.175:6811/2157664731","heartbeat_back_addr":"172.21.15.175:6815/2157664731","heartbeat_front_addr":"172.21.15.175:6813/2157664731","state":["exists","up"]},{"osd":6,"uuid":"6c687653-157b-4b96-867b-a2eaa7d8ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6819","nonce":838731756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6821","nonce":838731756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6825","nonce":838731756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6823","nonce":838731756}]},"public_addr":"172.21.15.7:6819/838731756","cluster_addr":"172.21.15.7:6821/838731756","heartbeat_back_addr":"172.21.15.7:6825/838731756","heartbeat_front_addr":"172.21.15.7:6823/838731756","state":["exists","up"]},{"osd":7,"uuid":"db40ddf1-9fd9-4859-b8c4-75f79582ab91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6817","nonce":3511921773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6819","nonce":3511921773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6823","nonce":3511921773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6821","nonce":3511921773}]},"public_addr":"172.21.15.78:6817/3511921773","cluster_addr":"172.21.15.78:6819/3511921773","heartbeat_back_addr":"172.21.15.78:6823/3511921773","heartbeat_front_addr":"172.21.15.78:6821/3511921773","state":["exists","up"]},{"osd":8,"uuid":"0f01cbd9-b193-4c70-a33c-4984167bf422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6817","nonce":1546325211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6819","nonce":1546325211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6823","nonce":1546325211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6821","nonce":1546325211}]},"public_addr":"172.21.15.175:6817/1546325211","cluster_addr":"172.21.15.175:6819/1546325211","heartbeat_back_addr":"172.21.15.175:6823/1546325211","heartbeat_front_addr":"172.21.15.175:6821/1546325211","state":["exists","up"]},{"osd":9,"uuid":"e5480998-f7fb-49cd-9aed-52405c88d830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6827","nonce":1844550813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6829","nonce":1844550813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6833","nonce":1844550813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6831","nonce":1844550813}]},"public_addr":"172.21.15.7:6827/1844550813","cluster_addr":"172.21.15.7:6829/1844550813","heartbeat_back_addr":"172.21.15.7:6833/1844550813","heartbeat_front_addr":"172.21.15.7:6831/1844550813","state":["exists","up"]},{"osd":10,"uuid":"caa82f33-a71c-4907-bf3b-30a7fdfeb71d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6825","nonce":2290824785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6827","nonce":2290824785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6831","nonce":2290824785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6829","nonce":2290824785}]},"public_addr":"172.21.15.78:6825/2290824785","cluster_addr":"172.21.15.78:6827/2290824785","heartbeat_back_addr":"172.21.15.78:6831/2290824785","heartbeat_front_addr":"172.21.15.78:6829/2290824785","state":["exists","up"]},{"osd":11,"uuid":"608cda77-77da-4da2-a7e2-76a773be27c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6825","nonce":1959979550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6827","nonce":1959979550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6831","nonce":1959979550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6829","nonce":1959979550}]},"public_addr":"172.21.15.175:6825/1959979550","cluster_addr":"172.21.15.175:6827/1959979550","heartbeat_back_addr":"172.21.15.175:6831/1959979550","heartbeat_front_addr":"172.21.15.175:6829/1959979550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:08.850203+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:32.820161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:56.633656+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:21.759372+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:46.735874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:10.636541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:36.990099+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:01.767098+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:26.882776+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:52.757595+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:18.383938+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:43.544330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1833397470":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/322543993":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/212510583":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6801/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6800/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/2679436843":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6801/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/1210355277":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6800/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/3610017406":"2024-06-11T00:54:08.102405+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T01:02:09.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:09.117+0000 7f401b9e7700 1 -- 172.21.15.7:0/3289505728 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f40000a2c20 msgr2=0x7f40000a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:09.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:09.117+0000 7f401b9e7700 1 --2- 172.21.15.7:0/3289505728 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f40000a2c20 0x7f40000a50e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f401000abd0 tx=0x7f401000a380 comp rx=0 tx=0).stop 2024-06-10T01:02:09.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:09.117+0000 7f401b9e7700 1 -- 172.21.15.7:0/3289505728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4014102130 msgr2=0x7f40141035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:09.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:09.117+0000 7f401b9e7700 1 --2- 172.21.15.7:0/3289505728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4014102130 0x7f40141035c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f400400d950 tx=0x7f400400dc60 comp rx=0 tx=0).stop 2024-06-10T01:02:09.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:09.117+0000 7f401b9e7700 1 -- 172.21.15.7:0/3289505728 shutdown_connections 2024-06-10T01:02:09.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:09.117+0000 7f401b9e7700 1 --2- 172.21.15.7:0/3289505728 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f40000a2c20 0x7f40000a50e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:09.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:09.117+0000 7f401b9e7700 1 --2- 172.21.15.7:0/3289505728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4014107550 0x7f4014103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:09.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:09.117+0000 7f401b9e7700 1 --2- 172.21.15.7:0/3289505728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4014101770 0x7f4014103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:09.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:09.117+0000 7f401b9e7700 1 --2- 172.21.15.7:0/3289505728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4014102130 0x7f40141035c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:09.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:09.117+0000 7f401b9e7700 1 -- 172.21.15.7:0/3289505728 >> 172.21.15.7:0/3289505728 conn(0x7f4014078340 msgr2=0x7f4014100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:09.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:09.117+0000 7f401b9e7700 1 -- 172.21.15.7:0/3289505728 shutdown_connections 2024-06-10T01:02:09.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:09.117+0000 7f401b9e7700 1 -- 172.21.15.7:0/3289505728 wait complete. 2024-06-10T01:02:09.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:09 smithi078 ceph-mon[36683]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:09.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:09 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3289505728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:02:09.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:09 smithi175 ceph-mon[36636]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:09.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:09 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3289505728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:02:09.766 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:09 smithi007 ceph-mon[32398]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:09.766 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:09 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3289505728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:02:09.843 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T01:02:10.097 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:02:11.497 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:11 smithi007 ceph-mon[32398]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:11.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:11 smithi078 ceph-mon[36683]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:11.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:11 smithi175 ceph-mon[36636]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:11.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.849+0000 7fb5aa78f700 1 -- 172.21.15.7:0/3366799295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5a4107840 msgr2=0x7fb5a410f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:11.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.849+0000 7fb5aa78f700 1 --2- 172.21.15.7:0/3366799295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5a4107840 0x7fb5a410f3c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb59c00b0d0 tx=0x7fb59c00b3e0 comp rx=0 tx=0).stop 2024-06-10T01:02:11.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.849+0000 7fb5aa78f700 1 -- 172.21.15.7:0/3366799295 shutdown_connections 2024-06-10T01:02:11.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.849+0000 7fb5aa78f700 1 --2- 172.21.15.7:0/3366799295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5a4107840 0x7fb5a410f3c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:11.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.849+0000 7fb5aa78f700 1 --2- 172.21.15.7:0/3366799295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5a4102740 0x7fb5a4102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:11.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.849+0000 7fb5aa78f700 1 --2- 172.21.15.7:0/3366799295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5a41030f0 0x7fb5a4107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:11.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.849+0000 7fb5aa78f700 1 -- 172.21.15.7:0/3366799295 >> 172.21.15.7:0/3366799295 conn(0x7fb5a40795f0 msgr2=0x7fb5a4079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:11.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.849+0000 7fb5aa78f700 1 -- 172.21.15.7:0/3366799295 shutdown_connections 2024-06-10T01:02:11.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.850+0000 7fb5aa78f700 1 -- 172.21.15.7:0/3366799295 wait complete. 2024-06-10T01:02:11.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.850+0000 7fb5aa78f700 1 Processor -- start 2024-06-10T01:02:11.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.850+0000 7fb5aa78f700 1 -- start start 2024-06-10T01:02:11.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.851+0000 7fb5aa78f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5a4102740 0x7fb5a4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:11.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.851+0000 7fb5aa78f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5a41030f0 0x7fb5a4074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:11.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.851+0000 7fb5aa78f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5a4107840 0x7fb5a4075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:11.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.851+0000 7fb5aa78f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a4112780 con 0x7fb5a4102740 2024-06-10T01:02:11.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.851+0000 7fb5aa78f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a4112600 con 0x7fb5a4107840 2024-06-10T01:02:11.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.851+0000 7fb5aa78f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a4112900 con 0x7fb5a41030f0 2024-06-10T01:02:11.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.851+0000 7fb5a3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5a4102740 0x7fb5a4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:11.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.851+0000 7fb5a37fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5a41030f0 0x7fb5a4074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:11.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.851+0000 7fb5a3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5a4102740 0x7fb5a4074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60728/0 (socket says 172.21.15.7:60728) 2024-06-10T01:02:11.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.851+0000 7fb5a3fff700 1 -- 172.21.15.7:0/3716018353 learned_addr learned my addr 172.21.15.7:0/3716018353 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:02:11.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.851+0000 7fb5a37fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5a41030f0 0x7fb5a4074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:58140/0 (socket says 172.21.15.7:58140) 2024-06-10T01:02:11.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.851+0000 7fb5a8d2c700 1 --2- 172.21.15.7:0/3716018353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5a4107840 0x7fb5a4075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:11.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.851+0000 7fb5a3fff700 1 -- 172.21.15.7:0/3716018353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5a41030f0 msgr2=0x7fb5a4074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:11.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.851+0000 7fb5a3fff700 1 --2- 172.21.15.7:0/3716018353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5a41030f0 0x7fb5a4074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:11.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.851+0000 7fb5a3fff700 1 -- 172.21.15.7:0/3716018353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5a4107840 msgr2=0x7fb5a4075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:11.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.851+0000 7fb5a3fff700 1 --2- 172.21.15.7:0/3716018353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5a4107840 0x7fb5a4075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:11.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.851+0000 7fb5a3fff700 1 -- 172.21.15.7:0/3716018353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a41a70a0 con 0x7fb5a4102740 2024-06-10T01:02:11.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.851+0000 7fb5a3fff700 1 --2- 172.21.15.7:0/3716018353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5a4102740 0x7fb5a4074790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb59400d980 tx=0x7fb59400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:11.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.851+0000 7fb5a17fa700 1 -- 172.21.15.7:0/3716018353 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb594017070 con 0x7fb5a4102740 2024-06-10T01:02:11.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.852+0000 7fb5a17fa700 1 -- 172.21.15.7:0/3716018353 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb594005020 con 0x7fb5a4102740 2024-06-10T01:02:11.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.852+0000 7fb5aa78f700 1 -- 172.21.15.7:0/3716018353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5a41a7390 con 0x7fb5a4102740 2024-06-10T01:02:11.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.852+0000 7fb5a17fa700 1 -- 172.21.15.7:0/3716018353 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59400fbf0 con 0x7fb5a4102740 2024-06-10T01:02:11.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.852+0000 7fb5aa78f700 1 -- 172.21.15.7:0/3716018353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5a41a7880 con 0x7fb5a4102740 2024-06-10T01:02:11.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.853+0000 7fb5aa78f700 1 -- 172.21.15.7:0/3716018353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5a410aa10 con 0x7fb5a4102740 2024-06-10T01:02:11.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.854+0000 7fb5a17fa700 1 -- 172.21.15.7:0/3716018353 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fb5940051d0 con 0x7fb5a4102740 2024-06-10T01:02:11.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.854+0000 7fb5a17fa700 1 --2- 172.21.15.7:0/3716018353 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb58c0a2c20 0x7fb58c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:11.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.855+0000 7fb5a17fa700 1 -- 172.21.15.7:0/3716018353 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7fb5940cf490 con 0x7fb5a4102740 2024-06-10T01:02:11.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.855+0000 7fb5a37fe700 1 --2- 172.21.15.7:0/3716018353 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb58c0a2c20 0x7fb58c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:11.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.855+0000 7fb5a37fe700 1 --2- 172.21.15.7:0/3716018353 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb58c0a2c20 0x7fb58c0a50e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4102030 tx=0x7fb59800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:11.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:11.858+0000 7fb5a17fa700 1 -- 172.21.15.7:0/3716018353 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb594020020 con 0x7fb5a4102740 2024-06-10T01:02:12.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:12.064+0000 7fb5aa78f700 1 -- 172.21.15.7:0/3716018353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5a410a3f0 con 0x7fb5a4102740 2024-06-10T01:02:12.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:12.066+0000 7fb5a17fa700 1 -- 172.21.15.7:0/3716018353 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24022 (secure 0 0 0) 0x7fb59409c1b0 con 0x7fb5a4102740 2024-06-10T01:02:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:02:12.065 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":95,"fsid":"bded58f8-26c3-11ef-bc9b-c7b262605968","created":"2024-06-10T00:53:31.966085+0000","modified":"2024-06-10T01:02:02.304963+0000","last_up_change":"2024-06-10T01:00:45.382473+0000","last_in_change":"2024-06-10T01:00:25.608513+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:56:59.964055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T01:01:42.563611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T01:01:46.175918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T01:01:55.695502+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f7c24c9f-b7ab-4c04-b2b4-120df9751667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6803","nonce":404485726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6805","nonce":404485726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6809","nonce":404485726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6807","nonce":404485726}]},"public_addr":"172.21.15.7:6803/404485726","cluster_addr":"172.21.15.7:6805/404485726","heartbeat_back_addr":"172.21.15.7:6809/404485726","heartbeat_front_addr":"172.21.15.7:6807/404485726","state":["exists","up"]},{"osd":1,"uuid":"9bf70a6b-f31c-4077-b5af-229f17bc2845","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6801","nonce":1856804448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6803","nonce":1856804448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6807","nonce":1856804448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6805","nonce":1856804448}]},"public_addr":"172.21.15.78:6801/1856804448","cluster_addr":"172.21.15.78:6803/1856804448","heartbeat_back_addr":"172.21.15.78:6807/1856804448","heartbeat_front_addr":"172.21.15.78:6805/1856804448","state":["exists","up"]},{"osd":2,"uuid":"62923c97-caf9-4f7e-9c14-9c9918843c78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6801","nonce":918257455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6803","nonce":918257455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6807","nonce":918257455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6805","nonce":918257455}]},"public_addr":"172.21.15.175:6801/918257455","cluster_addr":"172.21.15.175:6803/918257455","heartbeat_back_addr":"172.21.15.175:6807/918257455","heartbeat_front_addr":"172.21.15.175:6805/918257455","state":["exists","up"]},{"osd":3,"uuid":"65242098-e748-48ab-8f0e-bb25b5c62742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6811","nonce":1797404966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6813","nonce":1797404966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6817","nonce":1797404966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6815","nonce":1797404966}]},"public_addr":"172.21.15.7:6811/1797404966","cluster_addr":"172.21.15.7:6813/1797404966","heartbeat_back_addr":"172.21.15.7:6817/1797404966","heartbeat_front_addr":"172.21.15.7:6815/1797404966","state":["exists","up"]},{"osd":4,"uuid":"a7990305-de9e-400e-a432-d7854f73f339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6809","nonce":3767608357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6811","nonce":3767608357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6815","nonce":3767608357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6813","nonce":3767608357}]},"public_addr":"172.21.15.78:6809/3767608357","cluster_addr":"172.21.15.78:6811/3767608357","heartbeat_back_addr":"172.21.15.78:6815/3767608357","heartbeat_front_addr":"172.21.15.78:6813/3767608357","state":["exists","up"]},{"osd":5,"uuid":"bad9a7b3-da5a-48d0-905d-e6a3190317fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6809","nonce":2157664731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6811","nonce":2157664731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6815","nonce":2157664731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6813","nonce":2157664731}]},"public_addr":"172.21.15.175:6809/2157664731","cluster_addr":"172.21.15.175:6811/2157664731","heartbeat_back_addr":"172.21.15.175:6815/2157664731","heartbeat_front_addr":"172.21.15.175:6813/2157664731","state":["exists","up"]},{"osd":6,"uuid":"6c687653-157b-4b96-867b-a2eaa7d8ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6819","nonce":838731756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6821","nonce":838731756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6825","nonce":838731756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6823","nonce":838731756}]},"public_addr":"172.21.15.7:6819/838731756","cluster_addr":"172.21.15.7:6821/838731756","heartbeat_back_addr":"172.21.15.7:6825/838731756","heartbeat_front_addr":"172.21.15.7:6823/838731756","state":["exists","up"]},{"osd":7,"uuid":"db40ddf1-9fd9-4859-b8c4-75f79582ab91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6817","nonce":3511921773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6819","nonce":3511921773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6823","nonce":3511921773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6821","nonce":3511921773}]},"public_addr":"172.21.15.78:6817/3511921773","cluster_addr":"172.21.15.78:6819/3511921773","heartbeat_back_addr":"172.21.15.78:6823/3511921773","heartbeat_front_addr":"172.21.15.78:6821/3511921773","state":["exists","up"]},{"osd":8,"uuid":"0f01cbd9-b193-4c70-a33c-4984167bf422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6817","nonce":1546325211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6819","nonce":1546325211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6823","nonce":1546325211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6821","nonce":1546325211}]},"public_addr":"172.21.15.175:6817/1546325211","cluster_addr":"172.21.15.175:6819/1546325211","heartbeat_back_addr":"172.21.15.175:6823/1546325211","heartbeat_front_addr":"172.21.15.175:6821/1546325211","state":["exists","up"]},{"osd":9,"uuid":"e5480998-f7fb-49cd-9aed-52405c88d830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6827","nonce":1844550813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6829","nonce":1844550813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6833","nonce":1844550813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6831","nonce":1844550813}]},"public_addr":"172.21.15.7:6827/1844550813","cluster_addr":"172.21.15.7:6829/1844550813","heartbeat_back_addr":"172.21.15.7:6833/1844550813","heartbeat_front_addr":"172.21.15.7:6831/1844550813","state":["exists","up"]},{"osd":10,"uuid":"caa82f33-a71c-4907-bf3b-30a7fdfeb71d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6825","nonce":2290824785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6827","nonce":2290824785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6831","nonce":2290824785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6829","nonce":2290824785}]},"public_addr":"172.21.15.78:6825/2290824785","cluster_addr":"172.21.15.78:6827/2290824785","heartbeat_back_addr":"172.21.15.78:6831/2290824785","heartbeat_front_addr":"172.21.15.78:6829/2290824785","state":["exists","up"]},{"osd":11,"uuid":"608cda77-77da-4da2-a7e2-76a773be27c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6825","nonce":1959979550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6827","nonce":1959979550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6831","nonce":1959979550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6829","nonce":1959979550}]},"public_addr":"172.21.15.175:6825/1959979550","cluster_addr":"172.21.15.175:6827/1959979550","heartbeat_back_addr":"172.21.15.175:6831/1959979550","heartbeat_front_addr":"172.21.15.175:6829/1959979550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:08.850203+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:32.820161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:56.633656+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:21.759372+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:46.735874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:10.636541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:36.990099+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:01.767098+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:26.882776+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:52.757595+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:18.383938+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:43.544330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1833397470":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/322543993":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/212510583":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6801/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6800/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/2679436843":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6801/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/1210355277":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6800/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/3610017406":"2024-06-11T00:54:08.102405+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T01:02:12.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:12.068+0000 7fb5aa78f700 1 -- 172.21.15.7:0/3716018353 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb58c0a2c20 msgr2=0x7fb58c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:12.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:12.068+0000 7fb5aa78f700 1 --2- 172.21.15.7:0/3716018353 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb58c0a2c20 0x7fb58c0a50e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4102030 tx=0x7fb59800b410 comp rx=0 tx=0).stop 2024-06-10T01:02:12.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:12.068+0000 7fb5aa78f700 1 -- 172.21.15.7:0/3716018353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5a4102740 msgr2=0x7fb5a4074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:12.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:12.068+0000 7fb5aa78f700 1 --2- 172.21.15.7:0/3716018353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5a4102740 0x7fb5a4074790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb59400d980 tx=0x7fb59400dc90 comp rx=0 tx=0).stop 2024-06-10T01:02:12.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:12.068+0000 7fb5aa78f700 1 -- 172.21.15.7:0/3716018353 shutdown_connections 2024-06-10T01:02:12.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:12.068+0000 7fb5aa78f700 1 --2- 172.21.15.7:0/3716018353 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb58c0a2c20 0x7fb58c0a50e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:12.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:12.068+0000 7fb5aa78f700 1 --2- 172.21.15.7:0/3716018353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5a4107840 0x7fb5a4075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:12.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:12.068+0000 7fb5aa78f700 1 --2- 172.21.15.7:0/3716018353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5a4102740 0x7fb5a4074790 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:12.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:12.068+0000 7fb5aa78f700 1 --2- 172.21.15.7:0/3716018353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5a41030f0 0x7fb5a4074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:12.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:12.068+0000 7fb5aa78f700 1 -- 172.21.15.7:0/3716018353 >> 172.21.15.7:0/3716018353 conn(0x7fb5a40795f0 msgr2=0x7fb5a41006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:12.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:12.068+0000 7fb5aa78f700 1 -- 172.21.15.7:0/3716018353 shutdown_connections 2024-06-10T01:02:12.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:12.068+0000 7fb5aa78f700 1 -- 172.21.15.7:0/3716018353 wait complete. 2024-06-10T01:02:12.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:12 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3716018353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:02:12.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:12 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3716018353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:02:12.766 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:12 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3716018353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:02:12.867 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:02:13.126 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:02:13.516 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:13 smithi007 ceph-mon[32398]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:13.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:13 smithi078 ceph-mon[36683]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:13.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:13 smithi175 ceph-mon[36636]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.005+0000 7f2f4ec08700 1 -- 172.21.15.7:0/2790366331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f48101390 msgr2=0x7f2f48101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.005+0000 7f2f4ec08700 1 --2- 172.21.15.7:0/2790366331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f48101390 0x7f2f48101810 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2f38009990 tx=0x7f2f38009ca0 comp rx=0 tx=0).stop 2024-06-10T01:02:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.005+0000 7f2f4ec08700 1 -- 172.21.15.7:0/2790366331 shutdown_connections 2024-06-10T01:02:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.005+0000 7f2f4ec08700 1 --2- 172.21.15.7:0/2790366331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f48101d50 0x7f2f4810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.005+0000 7f2f4ec08700 1 --2- 172.21.15.7:0/2790366331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f48107400 0x7f2f481077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.005+0000 7f2f4ec08700 1 --2- 172.21.15.7:0/2790366331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f48101390 0x7f2f48101810 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.005+0000 7f2f4ec08700 1 -- 172.21.15.7:0/2790366331 >> 172.21.15.7:0/2790366331 conn(0x7f2f48078340 msgr2=0x7f2f48078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.005+0000 7f2f4ec08700 1 -- 172.21.15.7:0/2790366331 shutdown_connections 2024-06-10T01:02:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.005+0000 7f2f4ec08700 1 -- 172.21.15.7:0/2790366331 wait complete. 2024-06-10T01:02:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.006+0000 7f2f4ec08700 1 Processor -- start 2024-06-10T01:02:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.006+0000 7f2f4ec08700 1 -- start start 2024-06-10T01:02:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.006+0000 7f2f4ec08700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f48101390 0x7f2f4819d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.006+0000 7f2f4ec08700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f48101d50 0x7f2f4819dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.006+0000 7f2f4ec08700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f48107400 0x7f2f48197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.006+0000 7f2f4ec08700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2f48111670 con 0x7f2f48107400 2024-06-10T01:02:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.006+0000 7f2f4ec08700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f481114f0 con 0x7f2f48101d50 2024-06-10T01:02:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.006+0000 7f2f4ec08700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2f481117f0 con 0x7f2f48101390 2024-06-10T01:02:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.006+0000 7f2f4d1a5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f48107400 0x7f2f48197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.006+0000 7f2f4d1a5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f48107400 0x7f2f48197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60754/0 (socket says 172.21.15.7:60754) 2024-06-10T01:02:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.006+0000 7f2f4d1a5700 1 -- 172.21.15.7:0/3339350924 learned_addr learned my addr 172.21.15.7:0/3339350924 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:02:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.007+0000 7f2f3ffff700 1 --2- 172.21.15.7:0/3339350924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f48101d50 0x7f2f4819dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.007+0000 7f2f4c9a4700 1 --2- 172.21.15.7:0/3339350924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f48101390 0x7f2f4819d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.007+0000 7f2f4d1a5700 1 -- 172.21.15.7:0/3339350924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f48101390 msgr2=0x7f2f4819d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.007+0000 7f2f4d1a5700 1 --2- 172.21.15.7:0/3339350924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f48101390 0x7f2f4819d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.007+0000 7f2f4d1a5700 1 -- 172.21.15.7:0/3339350924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f48101d50 msgr2=0x7f2f4819dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.007+0000 7f2f4d1a5700 1 --2- 172.21.15.7:0/3339350924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f48101d50 0x7f2f4819dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.007+0000 7f2f4d1a5700 1 -- 172.21.15.7:0/3339350924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f481980b0 con 0x7f2f48107400 2024-06-10T01:02:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.007+0000 7f2f4c9a4700 1 --2- 172.21.15.7:0/3339350924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f48101390 0x7f2f4819d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:02:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.007+0000 7f2f4d1a5700 1 --2- 172.21.15.7:0/3339350924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f48107400 0x7f2f48197950 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000efb0 tx=0x7f2f4000c600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.007+0000 7f2f3dffb700 1 -- 172.21.15.7:0/3339350924 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f40011670 con 0x7f2f48107400 2024-06-10T01:02:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.007+0000 7f2f3dffb700 1 -- 172.21.15.7:0/3339350924 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f40011cf0 con 0x7f2f48107400 2024-06-10T01:02:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.007+0000 7f2f4ec08700 1 -- 172.21.15.7:0/3339350924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f481983a0 con 0x7f2f48107400 2024-06-10T01:02:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.007+0000 7f2f3dffb700 1 -- 172.21.15.7:0/3339350924 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4001ae20 con 0x7f2f48107400 2024-06-10T01:02:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.007+0000 7f2f4ec08700 1 -- 172.21.15.7:0/3339350924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f4806a170 con 0x7f2f48107400 2024-06-10T01:02:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.008+0000 7f2f4ec08700 1 -- 172.21.15.7:0/3339350924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f48069180 con 0x7f2f48107400 2024-06-10T01:02:15.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.009+0000 7f2f3dffb700 1 -- 172.21.15.7:0/3339350924 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f2f40011810 con 0x7f2f48107400 2024-06-10T01:02:15.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.010+0000 7f2f3dffb700 1 --2- 172.21.15.7:0/3339350924 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2f300a2c20 0x7f2f300a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:15.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.010+0000 7f2f4c9a4700 1 --2- 172.21.15.7:0/3339350924 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2f300a2c20 0x7f2f300a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:15.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.010+0000 7f2f3dffb700 1 -- 172.21.15.7:0/3339350924 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7f2f400d1bf0 con 0x7f2f48107400 2024-06-10T01:02:15.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.010+0000 7f2f4c9a4700 1 --2- 172.21.15.7:0/3339350924 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2f300a2c20 0x7f2f300a50e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2f44005fd0 tx=0x7f2f44005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.014+0000 7f2f3dffb700 1 -- 172.21.15.7:0/3339350924 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2f40016030 con 0x7f2f48107400 2024-06-10T01:02:15.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.265+0000 7f2f4ec08700 1 -- 172.21.15.7:0/3339350924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f4806ac00 con 0x7f2f48107400 2024-06-10T01:02:15.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.266+0000 7f2f3dffb700 1 -- 172.21.15.7:0/3339350924 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11782 (secure 0 0 0) 0x7f2f4009e8b0 con 0x7f2f48107400 2024-06-10T01:02:15.264 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:02:15.264 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2784910673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2784910673},{"type":"v1","addr":"172.21.15.175:6835","nonce":2784910673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24434,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:02:02.305093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24451,"qdb_cluster":[24451]},"id":1}]} 2024-06-10T01:02:15.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.268+0000 7f2f4ec08700 1 -- 172.21.15.7:0/3339350924 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2f300a2c20 msgr2=0x7f2f300a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:15.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.268+0000 7f2f4ec08700 1 --2- 172.21.15.7:0/3339350924 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2f300a2c20 0x7f2f300a50e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2f44005fd0 tx=0x7f2f44005d60 comp rx=0 tx=0).stop 2024-06-10T01:02:15.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.268+0000 7f2f4ec08700 1 -- 172.21.15.7:0/3339350924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f48107400 msgr2=0x7f2f48197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:15.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.268+0000 7f2f4ec08700 1 --2- 172.21.15.7:0/3339350924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f48107400 0x7f2f48197950 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000efb0 tx=0x7f2f4000c600 comp rx=0 tx=0).stop 2024-06-10T01:02:15.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.268+0000 7f2f4ec08700 1 -- 172.21.15.7:0/3339350924 shutdown_connections 2024-06-10T01:02:15.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.268+0000 7f2f4ec08700 1 --2- 172.21.15.7:0/3339350924 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2f300a2c20 0x7f2f300a50e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:15.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.268+0000 7f2f4ec08700 1 --2- 172.21.15.7:0/3339350924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f48107400 0x7f2f48197950 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:15.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.268+0000 7f2f4ec08700 1 --2- 172.21.15.7:0/3339350924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f48101390 0x7f2f4819d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:15.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.268+0000 7f2f4ec08700 1 --2- 172.21.15.7:0/3339350924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f48101d50 0x7f2f4819dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:15.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.268+0000 7f2f4ec08700 1 -- 172.21.15.7:0/3339350924 >> 172.21.15.7:0/3339350924 conn(0x7f2f48078340 msgr2=0x7f2f480ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:15.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.268+0000 7f2f4ec08700 1 -- 172.21.15.7:0/3339350924 shutdown_connections 2024-06-10T01:02:15.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:15.268+0000 7f2f4ec08700 1 -- 172.21.15.7:0/3339350924 wait complete. 2024-06-10T01:02:15.267 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 17 2024-06-10T01:02:15.516 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:15 smithi007 ceph-mon[32398]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:15.516 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:15 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3339350924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:02:15.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:15 smithi078 ceph-mon[36683]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:15.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:15 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3339350924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:02:15.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:15 smithi175 ceph-mon[36636]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:15.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:15 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3339350924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:02:15.947 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T01:02:16.201 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:02:17.367 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:17 smithi007 ceph-mon[32398]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:17.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:17 smithi078 ceph-mon[36683]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:17.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:17 smithi175 ceph-mon[36636]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:17.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.889+0000 7fb2a2d53700 1 -- 172.21.15.7:0/2935185180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb29c107860 msgr2=0x7fb29c10f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:17.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.889+0000 7fb2a2d53700 1 --2- 172.21.15.7:0/2935185180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb29c107860 0x7fb29c10f3e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb29400b0d0 tx=0x7fb29400b3e0 comp rx=0 tx=0).stop 2024-06-10T01:02:17.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.890+0000 7fb2a2d53700 1 -- 172.21.15.7:0/2935185180 shutdown_connections 2024-06-10T01:02:17.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.890+0000 7fb2a2d53700 1 --2- 172.21.15.7:0/2935185180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb29c107860 0x7fb29c10f3e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:17.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.890+0000 7fb2a2d53700 1 --2- 172.21.15.7:0/2935185180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb29c102770 0x7fb29c102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:17.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.890+0000 7fb2a2d53700 1 --2- 172.21.15.7:0/2935185180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb29c103120 0x7fb29c107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:17.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.890+0000 7fb2a2d53700 1 -- 172.21.15.7:0/2935185180 >> 172.21.15.7:0/2935185180 conn(0x7fb29c079600 msgr2=0x7fb29c079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:17.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.890+0000 7fb2a2d53700 1 -- 172.21.15.7:0/2935185180 shutdown_connections 2024-06-10T01:02:17.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.890+0000 7fb2a2d53700 1 -- 172.21.15.7:0/2935185180 wait complete. 2024-06-10T01:02:17.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.890+0000 7fb2a2d53700 1 Processor -- start 2024-06-10T01:02:17.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.890+0000 7fb2a2d53700 1 -- start start 2024-06-10T01:02:17.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.890+0000 7fb2a2d53700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb29c102770 0x7fb29c19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:17.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.890+0000 7fb2a2d53700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb29c103120 0x7fb29c19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:17.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.890+0000 7fb2a2d53700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb29c107860 0x7fb29c198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:17.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.891+0000 7fb2a2d53700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c1127f0 con 0x7fb29c103120 2024-06-10T01:02:17.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.891+0000 7fb2a2d53700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c112670 con 0x7fb29c107860 2024-06-10T01:02:17.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.891+0000 7fb2a2d53700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c112970 con 0x7fb29c102770 2024-06-10T01:02:17.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.891+0000 7fb29bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb29c103120 0x7fb29c19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:17.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.891+0000 7fb29bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb29c103120 0x7fb29c19ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60782/0 (socket says 172.21.15.7:60782) 2024-06-10T01:02:17.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.891+0000 7fb29bfff700 1 -- 172.21.15.7:0/2319139276 learned_addr learned my addr 172.21.15.7:0/2319139276 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:02:17.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.891+0000 7fb2a0aef700 1 --2- 172.21.15.7:0/2319139276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb29c102770 0x7fb29c19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:17.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.891+0000 7fb2a12f0700 1 --2- 172.21.15.7:0/2319139276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb29c107860 0x7fb29c198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:17.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.891+0000 7fb29bfff700 1 -- 172.21.15.7:0/2319139276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb29c102770 msgr2=0x7fb29c19e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:17.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.891+0000 7fb29bfff700 1 --2- 172.21.15.7:0/2319139276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb29c102770 0x7fb29c19e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:17.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.891+0000 7fb29bfff700 1 -- 172.21.15.7:0/2319139276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb29c107860 msgr2=0x7fb29c198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:17.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.891+0000 7fb29bfff700 1 --2- 172.21.15.7:0/2319139276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb29c107860 0x7fb29c198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:17.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.891+0000 7fb29bfff700 1 -- 172.21.15.7:0/2319139276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29c199170 con 0x7fb29c103120 2024-06-10T01:02:17.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.891+0000 7fb2a0aef700 1 --2- 172.21.15.7:0/2319139276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb29c102770 0x7fb29c19e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:02:17.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.891+0000 7fb2a12f0700 1 --2- 172.21.15.7:0/2319139276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb29c107860 0x7fb29c198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:02:17.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.891+0000 7fb29bfff700 1 --2- 172.21.15.7:0/2319139276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb29c103120 0x7fb29c19ed40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb28800b530 tx=0x7fb28800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:17.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.891+0000 7fb299ffb700 1 -- 172.21.15.7:0/2319139276 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288015020 con 0x7fb29c103120 2024-06-10T01:02:17.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.891+0000 7fb299ffb700 1 -- 172.21.15.7:0/2319139276 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb288011d10 con 0x7fb29c103120 2024-06-10T01:02:17.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.891+0000 7fb2a2d53700 1 -- 172.21.15.7:0/2319139276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb29c199460 con 0x7fb29c103120 2024-06-10T01:02:17.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.891+0000 7fb299ffb700 1 -- 172.21.15.7:0/2319139276 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288010e50 con 0x7fb29c103120 2024-06-10T01:02:17.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.892+0000 7fb2a2d53700 1 -- 172.21.15.7:0/2319139276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb29c1a56d0 con 0x7fb29c103120 2024-06-10T01:02:17.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.893+0000 7fb2a2d53700 1 -- 172.21.15.7:0/2319139276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb29c069180 con 0x7fb29c103120 2024-06-10T01:02:17.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.894+0000 7fb299ffb700 1 -- 172.21.15.7:0/2319139276 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fb288018020 con 0x7fb29c103120 2024-06-10T01:02:17.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.894+0000 7fb299ffb700 1 --2- 172.21.15.7:0/2319139276 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb28c0a2dc0 0x7fb28c0a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:17.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.894+0000 7fb2a0aef700 1 --2- 172.21.15.7:0/2319139276 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb28c0a2dc0 0x7fb28c0a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:17.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.894+0000 7fb299ffb700 1 -- 172.21.15.7:0/2319139276 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7fb2880ceaf0 con 0x7fb29c103120 2024-06-10T01:02:17.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.895+0000 7fb2a0aef700 1 --2- 172.21.15.7:0/2319139276 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb28c0a2dc0 0x7fb28c0a5280 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb290005950 tx=0x7fb290005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:17.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:17.899+0000 7fb299ffb700 1 -- 172.21.15.7:0/2319139276 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb28801f020 con 0x7fb29c103120 2024-06-10T01:02:18.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:18.104+0000 7fb2a2d53700 1 -- 172.21.15.7:0/2319139276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb29c02dca0 con 0x7fb29c103120 2024-06-10T01:02:18.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:18.105+0000 7fb299ffb700 1 -- 172.21.15.7:0/2319139276 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24022 (secure 0 0 0) 0x7fb28809b810 con 0x7fb29c103120 2024-06-10T01:02:18.103 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:02:18.104 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":95,"fsid":"bded58f8-26c3-11ef-bc9b-c7b262605968","created":"2024-06-10T00:53:31.966085+0000","modified":"2024-06-10T01:02:02.304963+0000","last_up_change":"2024-06-10T01:00:45.382473+0000","last_in_change":"2024-06-10T01:00:25.608513+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:56:59.964055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T01:01:42.563611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T01:01:46.175918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T01:01:55.695502+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f7c24c9f-b7ab-4c04-b2b4-120df9751667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6803","nonce":404485726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6805","nonce":404485726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6809","nonce":404485726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6807","nonce":404485726}]},"public_addr":"172.21.15.7:6803/404485726","cluster_addr":"172.21.15.7:6805/404485726","heartbeat_back_addr":"172.21.15.7:6809/404485726","heartbeat_front_addr":"172.21.15.7:6807/404485726","state":["exists","up"]},{"osd":1,"uuid":"9bf70a6b-f31c-4077-b5af-229f17bc2845","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6801","nonce":1856804448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6803","nonce":1856804448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6807","nonce":1856804448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6805","nonce":1856804448}]},"public_addr":"172.21.15.78:6801/1856804448","cluster_addr":"172.21.15.78:6803/1856804448","heartbeat_back_addr":"172.21.15.78:6807/1856804448","heartbeat_front_addr":"172.21.15.78:6805/1856804448","state":["exists","up"]},{"osd":2,"uuid":"62923c97-caf9-4f7e-9c14-9c9918843c78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6801","nonce":918257455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6803","nonce":918257455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6807","nonce":918257455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6805","nonce":918257455}]},"public_addr":"172.21.15.175:6801/918257455","cluster_addr":"172.21.15.175:6803/918257455","heartbeat_back_addr":"172.21.15.175:6807/918257455","heartbeat_front_addr":"172.21.15.175:6805/918257455","state":["exists","up"]},{"osd":3,"uuid":"65242098-e748-48ab-8f0e-bb25b5c62742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6811","nonce":1797404966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6813","nonce":1797404966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6817","nonce":1797404966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6815","nonce":1797404966}]},"public_addr":"172.21.15.7:6811/1797404966","cluster_addr":"172.21.15.7:6813/1797404966","heartbeat_back_addr":"172.21.15.7:6817/1797404966","heartbeat_front_addr":"172.21.15.7:6815/1797404966","state":["exists","up"]},{"osd":4,"uuid":"a7990305-de9e-400e-a432-d7854f73f339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6809","nonce":3767608357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6811","nonce":3767608357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6815","nonce":3767608357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6813","nonce":3767608357}]},"public_addr":"172.21.15.78:6809/3767608357","cluster_addr":"172.21.15.78:6811/3767608357","heartbeat_back_addr":"172.21.15.78:6815/3767608357","heartbeat_front_addr":"172.21.15.78:6813/3767608357","state":["exists","up"]},{"osd":5,"uuid":"bad9a7b3-da5a-48d0-905d-e6a3190317fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6809","nonce":2157664731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6811","nonce":2157664731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6815","nonce":2157664731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6813","nonce":2157664731}]},"public_addr":"172.21.15.175:6809/2157664731","cluster_addr":"172.21.15.175:6811/2157664731","heartbeat_back_addr":"172.21.15.175:6815/2157664731","heartbeat_front_addr":"172.21.15.175:6813/2157664731","state":["exists","up"]},{"osd":6,"uuid":"6c687653-157b-4b96-867b-a2eaa7d8ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6819","nonce":838731756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6821","nonce":838731756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6825","nonce":838731756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6823","nonce":838731756}]},"public_addr":"172.21.15.7:6819/838731756","cluster_addr":"172.21.15.7:6821/838731756","heartbeat_back_addr":"172.21.15.7:6825/838731756","heartbeat_front_addr":"172.21.15.7:6823/838731756","state":["exists","up"]},{"osd":7,"uuid":"db40ddf1-9fd9-4859-b8c4-75f79582ab91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6817","nonce":3511921773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6819","nonce":3511921773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6823","nonce":3511921773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6821","nonce":3511921773}]},"public_addr":"172.21.15.78:6817/3511921773","cluster_addr":"172.21.15.78:6819/3511921773","heartbeat_back_addr":"172.21.15.78:6823/3511921773","heartbeat_front_addr":"172.21.15.78:6821/3511921773","state":["exists","up"]},{"osd":8,"uuid":"0f01cbd9-b193-4c70-a33c-4984167bf422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6817","nonce":1546325211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6819","nonce":1546325211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6823","nonce":1546325211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6821","nonce":1546325211}]},"public_addr":"172.21.15.175:6817/1546325211","cluster_addr":"172.21.15.175:6819/1546325211","heartbeat_back_addr":"172.21.15.175:6823/1546325211","heartbeat_front_addr":"172.21.15.175:6821/1546325211","state":["exists","up"]},{"osd":9,"uuid":"e5480998-f7fb-49cd-9aed-52405c88d830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6827","nonce":1844550813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6829","nonce":1844550813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6833","nonce":1844550813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6831","nonce":1844550813}]},"public_addr":"172.21.15.7:6827/1844550813","cluster_addr":"172.21.15.7:6829/1844550813","heartbeat_back_addr":"172.21.15.7:6833/1844550813","heartbeat_front_addr":"172.21.15.7:6831/1844550813","state":["exists","up"]},{"osd":10,"uuid":"caa82f33-a71c-4907-bf3b-30a7fdfeb71d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6825","nonce":2290824785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6827","nonce":2290824785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6831","nonce":2290824785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6829","nonce":2290824785}]},"public_addr":"172.21.15.78:6825/2290824785","cluster_addr":"172.21.15.78:6827/2290824785","heartbeat_back_addr":"172.21.15.78:6831/2290824785","heartbeat_front_addr":"172.21.15.78:6829/2290824785","state":["exists","up"]},{"osd":11,"uuid":"608cda77-77da-4da2-a7e2-76a773be27c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6825","nonce":1959979550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6827","nonce":1959979550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6831","nonce":1959979550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6829","nonce":1959979550}]},"public_addr":"172.21.15.175:6825/1959979550","cluster_addr":"172.21.15.175:6827/1959979550","heartbeat_back_addr":"172.21.15.175:6831/1959979550","heartbeat_front_addr":"172.21.15.175:6829/1959979550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:08.850203+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:32.820161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:56.633656+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:21.759372+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:46.735874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:10.636541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:36.990099+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:01.767098+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:26.882776+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:52.757595+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:18.383938+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:43.544330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1833397470":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/322543993":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/212510583":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6801/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6800/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/2679436843":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6801/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/1210355277":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6800/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/3610017406":"2024-06-11T00:54:08.102405+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T01:02:18.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:18.108+0000 7fb2a2d53700 1 -- 172.21.15.7:0/2319139276 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb28c0a2dc0 msgr2=0x7fb28c0a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:18.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:18.108+0000 7fb2a2d53700 1 --2- 172.21.15.7:0/2319139276 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb28c0a2dc0 0x7fb28c0a5280 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb290005950 tx=0x7fb290005f50 comp rx=0 tx=0).stop 2024-06-10T01:02:18.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:18.108+0000 7fb2a2d53700 1 -- 172.21.15.7:0/2319139276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb29c103120 msgr2=0x7fb29c19ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:18.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:18.108+0000 7fb2a2d53700 1 --2- 172.21.15.7:0/2319139276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb29c103120 0x7fb29c19ed40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb28800b530 tx=0x7fb28800b8f0 comp rx=0 tx=0).stop 2024-06-10T01:02:18.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:18.108+0000 7fb2a2d53700 1 -- 172.21.15.7:0/2319139276 shutdown_connections 2024-06-10T01:02:18.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:18.108+0000 7fb2a2d53700 1 --2- 172.21.15.7:0/2319139276 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb28c0a2dc0 0x7fb28c0a5280 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:18.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:18.108+0000 7fb2a2d53700 1 --2- 172.21.15.7:0/2319139276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb29c107860 0x7fb29c198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:18.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:18.108+0000 7fb2a2d53700 1 --2- 172.21.15.7:0/2319139276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb29c102770 0x7fb29c19e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:18.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:18.108+0000 7fb2a2d53700 1 --2- 172.21.15.7:0/2319139276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb29c103120 0x7fb29c19ed40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:18.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:18.108+0000 7fb2a2d53700 1 -- 172.21.15.7:0/2319139276 >> 172.21.15.7:0/2319139276 conn(0x7fb29c079600 msgr2=0x7fb29c1004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:18.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:18.108+0000 7fb2a2d53700 1 -- 172.21.15.7:0/2319139276 shutdown_connections 2024-06-10T01:02:18.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:18.108+0000 7fb2a2d53700 1 -- 172.21.15.7:0/2319139276 wait complete. 2024-06-10T01:02:18.537 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:18 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2319139276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:02:18.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:18 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2319139276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:02:18.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:18 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2319139276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:02:18.849 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-06-10T01:01:49.029348+0000', 'modified': '2024-06-10T01:02:02.305093+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24451}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24451': {'gid': 24451, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.175:6839/1983418024', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6838', 'nonce': 1983418024}, {'type': 'v1', 'addr': '172.21.15.175:6839', 'nonce': 1983418024}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24451, 'qdb_cluster': [24451]} 2024-06-10T01:02:18.849 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-06-10T01:02:18.850 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-10T01:02:19.101 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:02:19.475 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:19 smithi007 ceph-mon[32398]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:19.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:19 smithi078 ceph-mon[36683]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:19.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:19 smithi175 ceph-mon[36636]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:20.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.831+0000 7fa901039700 1 -- 172.21.15.7:0/2317100214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8fc107b60 msgr2=0x7fa8fc10e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:20.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.831+0000 7fa901039700 1 --2- 172.21.15.7:0/2317100214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8fc107b60 0x7fa8fc10e320 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa8f400b0d0 tx=0x7fa8f400b3e0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.832+0000 7fa901039700 1 -- 172.21.15.7:0/2317100214 shutdown_connections 2024-06-10T01:02:20.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.832+0000 7fa901039700 1 --2- 172.21.15.7:0/2317100214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8fc107b60 0x7fa8fc10e320 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.832+0000 7fa901039700 1 --2- 172.21.15.7:0/2317100214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8fc105330 0x7fa8fc105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.832+0000 7fa901039700 1 --2- 172.21.15.7:0/2317100214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8fc1071a0 0x7fa8fc107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.832+0000 7fa901039700 1 -- 172.21.15.7:0/2317100214 >> 172.21.15.7:0/2317100214 conn(0x7fa8fc0766b0 msgr2=0x7fa8fc076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:20.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.832+0000 7fa901039700 1 -- 172.21.15.7:0/2317100214 shutdown_connections 2024-06-10T01:02:20.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.832+0000 7fa901039700 1 -- 172.21.15.7:0/2317100214 wait complete. 2024-06-10T01:02:20.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.832+0000 7fa901039700 1 Processor -- start 2024-06-10T01:02:20.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.833+0000 7fa901039700 1 -- start start 2024-06-10T01:02:20.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.833+0000 7fa901039700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8fc105330 0x7fa8fc102030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:20.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.833+0000 7fa901039700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8fc1071a0 0x7fa8fc102570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:20.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.833+0000 7fa901039700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8fc107b60 0x7fa8fc102ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:20.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.833+0000 7fa901039700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa8fc1117b0 con 0x7fa8fc105330 2024-06-10T01:02:20.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.833+0000 7fa901039700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa8fc111630 con 0x7fa8fc107b60 2024-06-10T01:02:20.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.833+0000 7fa901039700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa8fc111930 con 0x7fa8fc1071a0 2024-06-10T01:02:20.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.833+0000 7fa8fad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8fc105330 0x7fa8fc102030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:20.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.833+0000 7fa8fad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8fc105330 0x7fa8fc102030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60802/0 (socket says 172.21.15.7:60802) 2024-06-10T01:02:20.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.833+0000 7fa8fad9d700 1 -- 172.21.15.7:0/1338575277 learned_addr learned my addr 172.21.15.7:0/1338575277 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:02:20.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.833+0000 7fa8fb59e700 1 --2- 172.21.15.7:0/1338575277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8fc107b60 0x7fa8fc102ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:20.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.833+0000 7fa8fa59c700 1 --2- 172.21.15.7:0/1338575277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8fc1071a0 0x7fa8fc102570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:20.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.834+0000 7fa8fad9d700 1 -- 172.21.15.7:0/1338575277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8fc1071a0 msgr2=0x7fa8fc102570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:20.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.834+0000 7fa8fad9d700 1 --2- 172.21.15.7:0/1338575277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8fc1071a0 0x7fa8fc102570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.834+0000 7fa8fad9d700 1 -- 172.21.15.7:0/1338575277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8fc107b60 msgr2=0x7fa8fc102ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:20.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.834+0000 7fa8fad9d700 1 --2- 172.21.15.7:0/1338575277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8fc107b60 0x7fa8fc102ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.834+0000 7fa8fad9d700 1 -- 172.21.15.7:0/1338575277 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8fc1a71e0 con 0x7fa8fc105330 2024-06-10T01:02:20.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.834+0000 7fa8fb59e700 1 --2- 172.21.15.7:0/1338575277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8fc107b60 0x7fa8fc102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:20.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.834+0000 7fa8fad9d700 1 --2- 172.21.15.7:0/1338575277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8fc105330 0x7fa8fc102030 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00d950 tx=0x7fa8ec00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:20.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.834+0000 7fa8e3fff700 1 -- 172.21.15.7:0/1338575277 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec016070 con 0x7fa8fc105330 2024-06-10T01:02:20.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.834+0000 7fa901039700 1 -- 172.21.15.7:0/1338575277 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8fc1a74d0 con 0x7fa8fc105330 2024-06-10T01:02:20.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.834+0000 7fa8e3fff700 1 -- 172.21.15.7:0/1338575277 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8ec00fd70 con 0x7fa8fc105330 2024-06-10T01:02:20.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.834+0000 7fa901039700 1 -- 172.21.15.7:0/1338575277 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8fc1a7a60 con 0x7fa8fc105330 2024-06-10T01:02:20.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.834+0000 7fa8e3fff700 1 -- 172.21.15.7:0/1338575277 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec00e610 con 0x7fa8fc105330 2024-06-10T01:02:20.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.836+0000 7fa8e3fff700 1 -- 172.21.15.7:0/1338575277 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fa8ec019070 con 0x7fa8fc105330 2024-06-10T01:02:20.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.837+0000 7fa8e3fff700 1 --2- 172.21.15.7:0/1338575277 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa8e40a2c20 0x7fa8e40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:20.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.837+0000 7fa8fa59c700 1 --2- 172.21.15.7:0/1338575277 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa8e40a2c20 0x7fa8e40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:20.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.837+0000 7fa8e3fff700 1 -- 172.21.15.7:0/1338575277 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7fa8ec0cf0e0 con 0x7fa8fc105330 2024-06-10T01:02:20.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.838+0000 7fa8fa59c700 1 --2- 172.21.15.7:0/1338575277 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa8e40a2c20 0x7fa8e40a50e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc076c90 tx=0x7fa8f0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:20.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.839+0000 7fa90b8bcb80 1 Processor -- start 2024-06-10T01:02:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.839+0000 7fa90b8bcb80 1 -- start start 2024-06-10T01:02:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.839+0000 7fa90b8bcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55efca3a1f10 0x55efca3a22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.839+0000 7fa90b8bcb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efca3a2830 0x55efca39c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.839+0000 7fa90b8bcb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efca39c750 0x55efca39cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.839+0000 7fa90b8bcb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55efca28ef10 con 0x55efca39c750 2024-06-10T01:02:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.839+0000 7fa90b8bcb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55efca2ed800 con 0x55efca3a1f10 2024-06-10T01:02:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.839+0000 7fa90b8bcb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55efca255410 con 0x55efca3a2830 2024-06-10T01:02:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.839+0000 7fa8fb59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efca39c750 0x55efca39cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.839+0000 7fa8fb59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efca39c750 0x55efca39cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60816/0 (socket says 172.21.15.7:60816) 2024-06-10T01:02:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.839+0000 7fa8fb59e700 1 -- 172.21.15.7:0/4111924555 learned_addr learned my addr 172.21.15.7:0/4111924555 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:02:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.839+0000 7fa8fad9d700 1 --2- 172.21.15.7:0/4111924555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55efca3a1f10 0x55efca3a22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.839+0000 7fa8fa59c700 1 --2- 172.21.15.7:0/4111924555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efca3a2830 0x55efca39c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:20.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.839+0000 7fa8fb59e700 1 -- 172.21.15.7:0/4111924555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efca3a2830 msgr2=0x55efca39c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:20.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.839+0000 7fa8fb59e700 1 --2- 172.21.15.7:0/4111924555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efca3a2830 0x55efca39c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.839+0000 7fa8fb59e700 1 -- 172.21.15.7:0/4111924555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55efca3a1f10 msgr2=0x55efca3a22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:20.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.839+0000 7fa8fb59e700 1 --2- 172.21.15.7:0/4111924555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55efca3a1f10 0x55efca3a22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.839+0000 7fa8fb59e700 1 -- 172.21.15.7:0/4111924555 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efca23c880 con 0x55efca39c750 2024-06-10T01:02:20.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.839+0000 7fa8fad9d700 1 --2- 172.21.15.7:0/4111924555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55efca3a1f10 0x55efca3a22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:20.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.840+0000 7fa8fb59e700 1 --2- 172.21.15.7:0/4111924555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efca39c750 0x55efca39cb70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4009c00 tx=0x7fa8f4006210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d48cef2f8f3f3ff server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:20.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.840+0000 7fa8e17fa700 1 -- 172.21.15.7:0/4111924555 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4004c30 con 0x55efca39c750 2024-06-10T01:02:20.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.840+0000 7fa8e17fa700 1 -- 172.21.15.7:0/4111924555 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8f4004dd0 con 0x55efca39c750 2024-06-10T01:02:20.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.840+0000 7fa8e17fa700 1 -- 172.21.15.7:0/4111924555 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f40054a0 con 0x55efca39c750 2024-06-10T01:02:20.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.840+0000 7fa90b8bcb80 1 -- 172.21.15.7:0/4111924555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efca39c750 msgr2=0x55efca39cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:20.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.840+0000 7fa90b8bcb80 1 --2- 172.21.15.7:0/4111924555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efca39c750 0x55efca39cb70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4009c00 tx=0x7fa8f4006210 comp rx=0 tx=0).stop 2024-06-10T01:02:20.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.841+0000 7fa90b8bcb80 1 -- 172.21.15.7:0/4111924555 shutdown_connections 2024-06-10T01:02:20.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.841+0000 7fa90b8bcb80 1 --2- 172.21.15.7:0/4111924555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efca39c750 0x55efca39cb70 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.841+0000 7fa90b8bcb80 1 --2- 172.21.15.7:0/4111924555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55efca3a1f10 0x55efca3a22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.841+0000 7fa90b8bcb80 1 --2- 172.21.15.7:0/4111924555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efca3a2830 0x55efca39c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.841+0000 7fa90b8bcb80 1 -- 172.21.15.7:0/4111924555 >> 172.21.15.7:0/4111924555 conn(0x55efca2c9030 msgr2=0x55efca3971d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:20.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.841+0000 7fa90b8bcb80 1 -- 172.21.15.7:0/4111924555 shutdown_connections 2024-06-10T01:02:20.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.841+0000 7fa90b8bcb80 1 -- 172.21.15.7:0/4111924555 wait complete. 2024-06-10T01:02:20.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.841+0000 7fa90b8bcb80 1 -- start start 2024-06-10T01:02:20.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.841+0000 7fa90b8bcb80 1 Processor -- start 2024-06-10T01:02:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.841+0000 7fa90b8bcb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efca39c750 0x55efca3a8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.841+0000 7fa90b8bcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55efca3a1f10 0x55efca3a5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.841+0000 7fa90b8bcb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efca3a2830 0x55efca3a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.841+0000 7fa90b8bcb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55efca28df70 con 0x55efca3a2830 2024-06-10T01:02:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.842+0000 7fa90b8bcb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55efca2e97c0 con 0x55efca3a1f10 2024-06-10T01:02:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.842+0000 7fa90b8bcb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55efca3a91b0 con 0x55efca39c750 2024-06-10T01:02:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.842+0000 7fa90b8bcb80 10 client.0 _ll_register_callbacks cb 0x55efca283560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:02:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.842+0000 7fa8fb59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efca3a2830 0x55efca3a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.842+0000 7fa8fb59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efca3a2830 0x55efca3a5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60824/0 (socket says 172.21.15.7:60824) 2024-06-10T01:02:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.842+0000 7fa8fb59e700 1 -- 172.21.15.7:0/2528878998 learned_addr learned my addr 172.21.15.7:0/2528878998 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:02:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.842+0000 7fa8fad9d700 1 --2- 172.21.15.7:0/2528878998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efca39c750 0x55efca3a8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.842+0000 7fa8fa59c700 1 --2- 172.21.15.7:0/2528878998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55efca3a1f10 0x55efca3a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.842+0000 7fa8fb59e700 1 -- 172.21.15.7:0/2528878998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efca39c750 msgr2=0x55efca3a8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.842+0000 7fa8fb59e700 1 --2- 172.21.15.7:0/2528878998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efca39c750 0x55efca3a8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.842+0000 7fa8fb59e700 1 -- 172.21.15.7:0/2528878998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55efca3a1f10 msgr2=0x55efca3a5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:20.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.842+0000 7fa8fb59e700 1 --2- 172.21.15.7:0/2528878998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55efca3a1f10 0x55efca3a5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.842+0000 7fa8fb59e700 1 -- 172.21.15.7:0/2528878998 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efca2b3700 con 0x55efca3a2830 2024-06-10T01:02:20.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.842+0000 7fa8fa59c700 1 --2- 172.21.15.7:0/2528878998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55efca3a1f10 0x55efca3a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:02:20.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.842+0000 7fa8fb59e700 1 --2- 172.21.15.7:0/2528878998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efca3a2830 0x55efca3a5d70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa8f400b0d0 tx=0x7fa8f4006120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304e619e129cc685 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:20.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.842+0000 7fa8deff5700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:02:20.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.842+0000 7fa8deff5700 1 -- 172.21.15.7:0/2528878998 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f40054f0 con 0x55efca3a2830 2024-06-10T01:02:20.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.842+0000 7fa8deff5700 1 -- 172.21.15.7:0/2528878998 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7fa8f4005690 con 0x55efca3a2830 2024-06-10T01:02:20.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.842+0000 7fa90b8bcb80 1 -- 172.21.15.7:0/2528878998 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8f4004dd0 con 0x55efca3a2830 2024-06-10T01:02:20.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.843+0000 7fa8deff5700 1 -- 172.21.15.7:0/2528878998 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fa8f4009480 con 0x55efca3a2830 2024-06-10T01:02:20.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.843+0000 7fa90b8bcb80 10 client.14694 fetch_fsmap learned FSMap version 17 2024-06-10T01:02:20.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.843+0000 7fa90b8bcb80 1 -- 172.21.15.7:0/2528878998 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55efca3b38c0 con 0x55efca3a2830 2024-06-10T01:02:20.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.843+0000 7fa8deff5700 1 -- 172.21.15.7:0/2528878998 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8f4004dd0 con 0x55efca3a2830 2024-06-10T01:02:20.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.843+0000 7fa8deff5700 1 -- 172.21.15.7:0/2528878998 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fa8f4011070 con 0x55efca3a2830 2024-06-10T01:02:20.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.843+0000 7fa90b8bcb80 10 client.14694 fetch_fsmap finished waiting for FSMap version 17 2024-06-10T01:02:20.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.843+0000 7fa90b8bcb80 10 client.14694 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-10T01:02:20.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.843+0000 7fa90b8bcb80 20 client.14694 populate_metadata read hostname 'smithi007' 2024-06-10T01:02:20.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.843+0000 7fa90b8bcb80 1 --2- 172.21.15.7:0/2528878998 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55efca3b4c10 0x55efca3b6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:20.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.843+0000 7fa90b8bcb80 4 client.14694 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:02:20.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.843+0000 7fa90b8bcb80 1 -- 172.21.15.7:0/2528878998 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efca2fad00 con 0x55efca3b4c10 2024-06-10T01:02:20.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.844+0000 7fa8fad9d700 1 --2- 172.21.15.7:0/2528878998 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55efca3b4c10 0x55efca3b6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:20.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.844+0000 7fa8fad9d700 1 --2- 172.21.15.7:0/2528878998 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55efca3b4c10 0x55efca3b6de0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c3b5086235b8fff server_cookie=8949fd1d0beb59c0 in_seq=0 out_seq=0 2024-06-10T01:02:20.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.844+0000 7fa8deff5700 10 client.14694 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:02:20.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.845+0000 7fa8deff5700 1 -- 172.21.15.7:0/2528878998 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55efca2fad00 con 0x55efca3b4c10 2024-06-10T01:02:20.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.845+0000 7fa8deff5700 10 client.14694 handle_command_reply: tid=0 2024-06-10T01:02:20.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.845+0000 7fa90b8bcb80 1 client.14694 shutdown 2024-06-10T01:02:20.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.845+0000 7fa90b8bcb80 1 -- 172.21.15.7:0/2528878998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efca3a2830 msgr2=0x55efca3a5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:20.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.845+0000 7fa90b8bcb80 1 --2- 172.21.15.7:0/2528878998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efca3a2830 0x55efca3a5d70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa8f400b0d0 tx=0x7fa8f4006120 comp rx=0 tx=0).stop 2024-06-10T01:02:20.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.845+0000 7fa90b8bcb80 1 -- 172.21.15.7:0/2528878998 shutdown_connections 2024-06-10T01:02:20.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.845+0000 7fa90b8bcb80 1 --2- 172.21.15.7:0/2528878998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efca3a2830 0x55efca3a5d70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.845+0000 7fa90b8bcb80 1 --2- 172.21.15.7:0/2528878998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efca39c750 0x55efca3a8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.845+0000 7fa90b8bcb80 1 --2- 172.21.15.7:0/2528878998 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55efca3b4c10 0x55efca3b6de0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.845+0000 7fa8deff5700 0 client.14694 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:02:20.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.845+0000 7fa90b8bcb80 1 --2- 172.21.15.7:0/2528878998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55efca3a1f10 0x55efca3a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.845+0000 7fa90b8bcb80 1 -- 172.21.15.7:0/2528878998 >> 172.21.15.7:0/2528878998 conn(0x55efca2c9030 msgr2=0x55efca3966b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:20.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.845+0000 7fa90b8bcb80 1 -- 172.21.15.7:0/2528878998 shutdown_connections 2024-06-10T01:02:20.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.845+0000 7fa90b8bcb80 1 -- 172.21.15.7:0/2528878998 wait complete. 2024-06-10T01:02:20.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.845+0000 7fa90b8bcb80 20 client.14694 trim_cache size 0 max 16384 2024-06-10T01:02:20.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.867+0000 7fa90b8bcb80 1 Processor -- start 2024-06-10T01:02:20.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.867+0000 7fa90b8bcb80 1 -- start start 2024-06-10T01:02:20.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.867+0000 7fa90b8bcb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efca39c750 0x55efca39c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:20.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.867+0000 7fa90b8bcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55efca3a1f10 0x55efca3a34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:20.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.867+0000 7fa90b8bcb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efca3a2830 0x55efca3a4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:20.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.867+0000 7fa90b8bcb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55efca3b5120 con 0x55efca3a2830 2024-06-10T01:02:20.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.867+0000 7fa90b8bcb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55efca3b57f0 con 0x55efca3a1f10 2024-06-10T01:02:20.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.867+0000 7fa8fb59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efca3a2830 0x55efca3a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:20.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.867+0000 7fa90b8bcb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55efca3b5ec0 con 0x55efca39c750 2024-06-10T01:02:20.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.867+0000 7fa8fb59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efca3a2830 0x55efca3a4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60840/0 (socket says 172.21.15.7:60840) 2024-06-10T01:02:20.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.867+0000 7fa8fb59e700 1 -- 172.21.15.7:0/2262634947 learned_addr learned my addr 172.21.15.7:0/2262634947 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:02:20.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.867+0000 7fa8fad9d700 1 --2- 172.21.15.7:0/2262634947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efca39c750 0x55efca39c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:20.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.867+0000 7fa8fb59e700 1 -- 172.21.15.7:0/2262634947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efca39c750 msgr2=0x55efca39c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:20.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.867+0000 7fa8fa59c700 1 --2- 172.21.15.7:0/2262634947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55efca3a1f10 0x55efca3a34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:20.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.867+0000 7fa8fb59e700 1 --2- 172.21.15.7:0/2262634947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efca39c750 0x55efca39c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.867+0000 7fa8fb59e700 1 -- 172.21.15.7:0/2262634947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55efca3a1f10 msgr2=0x55efca3a34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:20.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.867+0000 7fa8fb59e700 1 --2- 172.21.15.7:0/2262634947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55efca3a1f10 0x55efca3a34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.867+0000 7fa8fb59e700 1 -- 172.21.15.7:0/2262634947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efca3b7d40 con 0x55efca3a2830 2024-06-10T01:02:20.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.868+0000 7fa8fb59e700 1 --2- 172.21.15.7:0/2262634947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efca3a2830 0x55efca3a4830 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4002230 tx=0x7fa8f400b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ab65cceea5142d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:20.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.868+0000 7fa8e0ff9700 1 -- 172.21.15.7:0/2262634947 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4026410 con 0x55efca3a2830 2024-06-10T01:02:20.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.868+0000 7fa8e0ff9700 1 -- 172.21.15.7:0/2262634947 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8f40265b0 con 0x55efca3a2830 2024-06-10T01:02:20.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.868+0000 7fa8e0ff9700 1 -- 172.21.15.7:0/2262634947 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4031450 con 0x55efca3a2830 2024-06-10T01:02:20.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.868+0000 7fa90b8bcb80 1 -- 172.21.15.7:0/2262634947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efca3a2830 msgr2=0x55efca3a4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:20.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.868+0000 7fa90b8bcb80 1 --2- 172.21.15.7:0/2262634947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efca3a2830 0x55efca3a4830 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4002230 tx=0x7fa8f400b3e0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.868+0000 7fa90b8bcb80 1 -- 172.21.15.7:0/2262634947 shutdown_connections 2024-06-10T01:02:20.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.868+0000 7fa90b8bcb80 1 --2- 172.21.15.7:0/2262634947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efca3a2830 0x55efca3a4830 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.868+0000 7fa90b8bcb80 1 --2- 172.21.15.7:0/2262634947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efca39c750 0x55efca39c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.868+0000 7fa90b8bcb80 1 --2- 172.21.15.7:0/2262634947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55efca3a1f10 0x55efca3a34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.868+0000 7fa90b8bcb80 1 -- 172.21.15.7:0/2262634947 >> 172.21.15.7:0/2262634947 conn(0x55efca2c9030 msgr2=0x55efca3a4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:20.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.868+0000 7fa90b8bcb80 1 -- 172.21.15.7:0/2262634947 shutdown_connections 2024-06-10T01:02:20.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.868+0000 7fa90b8bcb80 1 -- 172.21.15.7:0/2262634947 wait complete. 2024-06-10T01:02:20.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.869+0000 7fa90b8bcb80 1 -- start start 2024-06-10T01:02:20.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.869+0000 7fa90b8bcb80 1 Processor -- start 2024-06-10T01:02:20.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.869+0000 7fa90b8bcb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efca39c750 0x55efca3c0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:20.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.869+0000 7fa90b8bcb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efca3a1f10 0x55efca3b9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:20.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.869+0000 7fa90b8bcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55efca3a2830 0x55efca3ba5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:20.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.869+0000 7fa90b8bcb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55efca3b6c60 con 0x55efca3a1f10 2024-06-10T01:02:20.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.869+0000 7fa90b8bcb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55efca3b6590 con 0x55efca3a2830 2024-06-10T01:02:20.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.869+0000 7fa90b8bcb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55efca3b7330 con 0x55efca39c750 2024-06-10T01:02:20.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.869+0000 7fa90b8bcb80 10 client.0 _ll_register_callbacks cb 0x55efca283560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:02:20.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.869+0000 7fa8fa59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efca3a1f10 0x55efca3b9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:20.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.869+0000 7fa8fa59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efca3a1f10 0x55efca3b9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60842/0 (socket says 172.21.15.7:60842) 2024-06-10T01:02:20.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.869+0000 7fa8fa59c700 1 -- 172.21.15.7:0/1822010336 learned_addr learned my addr 172.21.15.7:0/1822010336 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:02:20.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.869+0000 7fa8fad9d700 1 --2- 172.21.15.7:0/1822010336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efca39c750 0x55efca3c0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:20.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.869+0000 7fa8fb59e700 1 --2- 172.21.15.7:0/1822010336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55efca3a2830 0x55efca3ba5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:20.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.870+0000 7fa8fa59c700 1 -- 172.21.15.7:0/1822010336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efca39c750 msgr2=0x55efca3c0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:20.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.870+0000 7fa8fa59c700 1 --2- 172.21.15.7:0/1822010336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efca39c750 0x55efca3c0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.870+0000 7fa8fa59c700 1 -- 172.21.15.7:0/1822010336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55efca3a2830 msgr2=0x55efca3ba5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:20.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.870+0000 7fa8fa59c700 1 --2- 172.21.15.7:0/1822010336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55efca3a2830 0x55efca3ba5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.870+0000 7fa8fa59c700 1 -- 172.21.15.7:0/1822010336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efca3be670 con 0x55efca3a1f10 2024-06-10T01:02:20.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.870+0000 7fa8fa59c700 1 --2- 172.21.15.7:0/1822010336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efca3a1f10 0x55efca3b9eb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc103ac0 tx=0x7fa8f0004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5641ba189165806 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:20.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.870+0000 7fa8deff5700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:02:20.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.870+0000 7fa8deff5700 1 -- 172.21.15.7:0/1822010336 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f0025030 con 0x55efca3a1f10 2024-06-10T01:02:20.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.870+0000 7fa8deff5700 1 -- 172.21.15.7:0/1822010336 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7fa8f001b040 con 0x55efca3a1f10 2024-06-10T01:02:20.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.870+0000 7fa90b8bcb80 1 -- 172.21.15.7:0/1822010336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8f40265b0 con 0x55efca3a1f10 2024-06-10T01:02:20.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.870+0000 7fa8deff5700 1 -- 172.21.15.7:0/1822010336 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fa8f0015d00 con 0x55efca3a1f10 2024-06-10T01:02:20.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.870+0000 7fa90b8bcb80 10 client.14700 fetch_fsmap learned FSMap version 17 2024-06-10T01:02:20.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.870+0000 7fa90b8bcb80 1 -- 172.21.15.7:0/1822010336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55efca421490 con 0x55efca3a1f10 2024-06-10T01:02:20.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.870+0000 7fa8deff5700 1 -- 172.21.15.7:0/1822010336 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8f40265b0 con 0x55efca3a1f10 2024-06-10T01:02:20.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.870+0000 7fa8deff5700 1 -- 172.21.15.7:0/1822010336 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fa8f0013550 con 0x55efca3a1f10 2024-06-10T01:02:20.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.871+0000 7fa90b8bcb80 10 client.14700 fetch_fsmap finished waiting for FSMap version 17 2024-06-10T01:02:20.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.871+0000 7fa90b8bcb80 10 client.14700 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-10T01:02:20.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.871+0000 7fa90b8bcb80 20 client.14700 populate_metadata read hostname 'smithi007' 2024-06-10T01:02:20.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.871+0000 7fa90b8bcb80 1 --2- 172.21.15.7:0/1822010336 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55efca3b4c10 0x55efca424e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:20.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.871+0000 7fa90b8bcb80 4 client.14700 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T01:02:20.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.871+0000 7fa90b8bcb80 1 -- 172.21.15.7:0/1822010336 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "status"}) -- 0x55efca2f5280 con 0x55efca3b4c10 2024-06-10T01:02:20.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.871+0000 7fa8fad9d700 1 --2- 172.21.15.7:0/1822010336 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55efca3b4c10 0x55efca424e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:20.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.871+0000 7fa8fad9d700 1 --2- 172.21.15.7:0/1822010336 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55efca3b4c10 0x55efca424e50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2742b89afdc15960 server_cookie=79e1e79d54d4514b in_seq=0 out_seq=0 2024-06-10T01:02:20.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.871+0000 7fa8deff5700 10 client.14700 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:02:20.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.872+0000 7fa8deff5700 1 -- 172.21.15.7:0/1822010336 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55efca2f5280 con 0x55efca3b4c10 2024-06-10T01:02:20.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.872+0000 7fa8deff5700 10 client.14700 handle_command_reply: tid=0 2024-06-10T01:02:20.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.872+0000 7fa90b8bcb80 1 client.14700 shutdown 2024-06-10T01:02:20.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.872+0000 7fa90b8bcb80 1 -- 172.21.15.7:0/1822010336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efca3a1f10 msgr2=0x55efca3b9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:20.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.872+0000 7fa90b8bcb80 1 --2- 172.21.15.7:0/1822010336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efca3a1f10 0x55efca3b9eb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc103ac0 tx=0x7fa8f0004860 comp rx=0 tx=0).stop 2024-06-10T01:02:20.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.872+0000 7fa90b8bcb80 1 -- 172.21.15.7:0/1822010336 shutdown_connections 2024-06-10T01:02:20.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.872+0000 7fa90b8bcb80 1 --2- 172.21.15.7:0/1822010336 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55efca3b4c10 0x55efca424e50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.872+0000 7fa8deff5700 0 client.14700 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:02:20.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.872+0000 7fa90b8bcb80 1 --2- 172.21.15.7:0/1822010336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55efca3a2830 0x55efca3ba5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.872+0000 7fa90b8bcb80 1 --2- 172.21.15.7:0/1822010336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55efca39c750 0x55efca3c0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.872+0000 7fa90b8bcb80 1 --2- 172.21.15.7:0/1822010336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efca3a1f10 0x55efca3b9eb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.872+0000 7fa90b8bcb80 1 -- 172.21.15.7:0/1822010336 >> 172.21.15.7:0/1822010336 conn(0x55efca2c9030 msgr2=0x55efca3a78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:20.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.872+0000 7fa90b8bcb80 1 -- 172.21.15.7:0/1822010336 shutdown_connections 2024-06-10T01:02:20.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.872+0000 7fa90b8bcb80 1 -- 172.21.15.7:0/1822010336 wait complete. 2024-06-10T01:02:20.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.872+0000 7fa90b8bcb80 20 client.14700 trim_cache size 0 max 16384 2024-06-10T01:02:20.875 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:02:20.875 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "bded58f8-26c3-11ef-bc9b-c7b262605968", 2024-06-10T01:02:20.875 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-06-10T01:02:20.875 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24451, 2024-06-10T01:02:20.875 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-06-10T01:02:20.875 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-06-10T01:02:20.875 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-06-10T01:02:20.875 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 31.635030286999999, 2024-06-10T01:02:20.875 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 17, 2024-06-10T01:02:20.875 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 95, 2024-06-10T01:02:20.875 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 95, 2024-06-10T01:02:20.875 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 58.730281134000002 2024-06-10T01:02:20.876 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:02:20.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.872+0000 7fa901039700 1 -- 172.21.15.7:0/1338575277 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa8e40a2c20 msgr2=0x7fa8e40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:20.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.872+0000 7fa901039700 1 --2- 172.21.15.7:0/1338575277 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa8e40a2c20 0x7fa8e40a50e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc076c90 tx=0x7fa8f0009190 comp rx=0 tx=0).stop 2024-06-10T01:02:20.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.872+0000 7fa901039700 1 -- 172.21.15.7:0/1338575277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8fc105330 msgr2=0x7fa8fc102030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:20.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.872+0000 7fa901039700 1 --2- 172.21.15.7:0/1338575277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8fc105330 0x7fa8fc102030 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00d950 tx=0x7fa8ec00dc60 comp rx=0 tx=0).stop 2024-06-10T01:02:20.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.872+0000 7fa901039700 1 -- 172.21.15.7:0/1338575277 shutdown_connections 2024-06-10T01:02:20.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.873+0000 7fa901039700 1 --2- 172.21.15.7:0/1338575277 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa8e40a2c20 0x7fa8e40a50e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.873+0000 7fa901039700 1 --2- 172.21.15.7:0/1338575277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8fc107b60 0x7fa8fc102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.873+0000 7fa901039700 1 --2- 172.21.15.7:0/1338575277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8fc105330 0x7fa8fc102030 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.873+0000 7fa901039700 1 --2- 172.21.15.7:0/1338575277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8fc1071a0 0x7fa8fc102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.873+0000 7fa901039700 1 -- 172.21.15.7:0/1338575277 >> 172.21.15.7:0/1338575277 conn(0x7fa8fc0766b0 msgr2=0x7fa8fc100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:20.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.873+0000 7fa901039700 1 -- 172.21.15.7:0/1338575277 shutdown_connections 2024-06-10T01:02:20.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:20.873+0000 7fa901039700 1 -- 172.21.15.7:0/1338575277 wait complete. 2024-06-10T01:02:21.471 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:21 smithi007 ceph-mon[32398]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:21.502 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-10T01:02:21.502 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-06-10T01:02:21.502 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-06-10T01:02:21.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:21 smithi078 ceph-mon[36683]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:21.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:21 smithi175 ceph-mon[36636]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:21.754 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:02:23.476 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:23 smithi007 ceph-mon[32398]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:23.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.493+0000 7ff23d00f700 1 -- 172.21.15.7:0/2908664448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff238102770 msgr2=0x7ff238102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:23.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.493+0000 7ff23d00f700 1 --2- 172.21.15.7:0/2908664448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff238102770 0x7ff238102b50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff2280099c0 tx=0x7ff228009cd0 comp rx=0 tx=0).stop 2024-06-10T01:02:23.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.493+0000 7ff23d00f700 1 -- 172.21.15.7:0/2908664448 shutdown_connections 2024-06-10T01:02:23.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.493+0000 7ff23d00f700 1 --2- 172.21.15.7:0/2908664448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff238107860 0x7ff23810f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:23.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.493+0000 7ff23d00f700 1 --2- 172.21.15.7:0/2908664448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff238102770 0x7ff238102b50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:23.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.493+0000 7ff23d00f700 1 --2- 172.21.15.7:0/2908664448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff238103120 0x7ff238107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:23.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.493+0000 7ff23d00f700 1 -- 172.21.15.7:0/2908664448 >> 172.21.15.7:0/2908664448 conn(0x7ff238079600 msgr2=0x7ff238079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:23.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.493+0000 7ff23d00f700 1 -- 172.21.15.7:0/2908664448 shutdown_connections 2024-06-10T01:02:23.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.493+0000 7ff23d00f700 1 -- 172.21.15.7:0/2908664448 wait complete. 2024-06-10T01:02:23.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.494+0000 7ff23d00f700 1 Processor -- start 2024-06-10T01:02:23.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.494+0000 7ff23d00f700 1 -- start start 2024-06-10T01:02:23.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.495+0000 7ff23d00f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff238102770 0x7ff23819aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:23.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.495+0000 7ff23d00f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff238103120 0x7ff23819b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:23.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.495+0000 7ff23d00f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff238107860 0x7ff23819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:23.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.495+0000 7ff23d00f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff238112840 con 0x7ff238103120 2024-06-10T01:02:23.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.495+0000 7ff23d00f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff2381126c0 con 0x7ff238107860 2024-06-10T01:02:23.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.495+0000 7ff23d00f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff2381129c0 con 0x7ff238102770 2024-06-10T01:02:23.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.495+0000 7ff23659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff238103120 0x7ff23819b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:23.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.495+0000 7ff23659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff238103120 0x7ff23819b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55548/0 (socket says 172.21.15.7:55548) 2024-06-10T01:02:23.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.495+0000 7ff23659c700 1 -- 172.21.15.7:0/116390688 learned_addr learned my addr 172.21.15.7:0/116390688 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:02:23.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.495+0000 7ff23759e700 1 --2- 172.21.15.7:0/116390688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff238107860 0x7ff23819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:23.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.495+0000 7ff236d9d700 1 --2- 172.21.15.7:0/116390688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff238102770 0x7ff23819aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:23.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.495+0000 7ff23659c700 1 -- 172.21.15.7:0/116390688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff238102770 msgr2=0x7ff23819aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:23.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.495+0000 7ff23659c700 1 --2- 172.21.15.7:0/116390688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff238102770 0x7ff23819aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:23.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.495+0000 7ff23659c700 1 -- 172.21.15.7:0/116390688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff238107860 msgr2=0x7ff23819f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:23.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.495+0000 7ff23659c700 1 --2- 172.21.15.7:0/116390688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff238107860 0x7ff23819f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:23.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.495+0000 7ff23659c700 1 -- 172.21.15.7:0/116390688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff23819fec0 con 0x7ff238103120 2024-06-10T01:02:23.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.495+0000 7ff236d9d700 1 --2- 172.21.15.7:0/116390688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff238102770 0x7ff23819aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:02:23.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.495+0000 7ff23659c700 1 --2- 172.21.15.7:0/116390688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff238103120 0x7ff23819b3f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00b530 tx=0x7ff22c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:23.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.496+0000 7ff21ffff700 1 -- 172.21.15.7:0/116390688 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c015020 con 0x7ff238103120 2024-06-10T01:02:23.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.496+0000 7ff23d00f700 1 -- 172.21.15.7:0/116390688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2381a01b0 con 0x7ff238103120 2024-06-10T01:02:23.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.496+0000 7ff21ffff700 1 -- 172.21.15.7:0/116390688 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff22c011d10 con 0x7ff238103120 2024-06-10T01:02:23.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.496+0000 7ff21ffff700 1 -- 172.21.15.7:0/116390688 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c00f2e0 con 0x7ff238103120 2024-06-10T01:02:23.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.496+0000 7ff23d00f700 1 -- 172.21.15.7:0/116390688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2381a7a30 con 0x7ff238103120 2024-06-10T01:02:23.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.497+0000 7ff23d00f700 1 -- 172.21.15.7:0/116390688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff238069180 con 0x7ff238103120 2024-06-10T01:02:23.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.498+0000 7ff21ffff700 1 -- 172.21.15.7:0/116390688 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7ff22c018020 con 0x7ff238103120 2024-06-10T01:02:23.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.498+0000 7ff21ffff700 1 --2- 172.21.15.7:0/116390688 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff2200a2c20 0x7ff2200a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:23.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.499+0000 7ff21ffff700 1 -- 172.21.15.7:0/116390688 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7ff22c0cef60 con 0x7ff238103120 2024-06-10T01:02:23.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.499+0000 7ff236d9d700 1 --2- 172.21.15.7:0/116390688 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff2200a2c20 0x7ff2200a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:23.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.499+0000 7ff236d9d700 1 --2- 172.21.15.7:0/116390688 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff2200a2c20 0x7ff2200a50e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff228000c00 tx=0x7ff228005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:23.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.502+0000 7ff21ffff700 1 -- 172.21.15.7:0/116390688 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff22c01f020 con 0x7ff238103120 2024-06-10T01:02:23.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:23 smithi078 ceph-mon[36683]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:23.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:23 smithi175 ceph-mon[36636]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:23.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.758+0000 7ff23d00f700 1 -- 172.21.15.7:0/116390688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff23819c210 con 0x7ff238103120 2024-06-10T01:02:23.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.758+0000 7ff21ffff700 1 -- 172.21.15.7:0/116390688 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7ff23819c210 con 0x7ff238103120 2024-06-10T01:02:23.756 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-06-10T01:02:23.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.762+0000 7ff23d00f700 1 -- 172.21.15.7:0/116390688 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff2200a2c20 msgr2=0x7ff2200a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:23.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.762+0000 7ff23d00f700 1 --2- 172.21.15.7:0/116390688 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff2200a2c20 0x7ff2200a50e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff228000c00 tx=0x7ff228005fb0 comp rx=0 tx=0).stop 2024-06-10T01:02:23.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.762+0000 7ff23d00f700 1 -- 172.21.15.7:0/116390688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff238103120 msgr2=0x7ff23819b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:23.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.762+0000 7ff23d00f700 1 --2- 172.21.15.7:0/116390688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff238103120 0x7ff23819b3f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00b530 tx=0x7ff22c00b840 comp rx=0 tx=0).stop 2024-06-10T01:02:23.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.762+0000 7ff23d00f700 1 -- 172.21.15.7:0/116390688 shutdown_connections 2024-06-10T01:02:23.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.762+0000 7ff23d00f700 1 --2- 172.21.15.7:0/116390688 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff2200a2c20 0x7ff2200a50e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:23.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.762+0000 7ff23d00f700 1 --2- 172.21.15.7:0/116390688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff238107860 0x7ff23819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:23.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.762+0000 7ff23d00f700 1 --2- 172.21.15.7:0/116390688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff238102770 0x7ff23819aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:23.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.762+0000 7ff23d00f700 1 --2- 172.21.15.7:0/116390688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff238103120 0x7ff23819b3f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:23.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.762+0000 7ff23d00f700 1 -- 172.21.15.7:0/116390688 >> 172.21.15.7:0/116390688 conn(0x7ff238079600 msgr2=0x7ff2381089b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:23.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.762+0000 7ff23d00f700 1 -- 172.21.15.7:0/116390688 shutdown_connections 2024-06-10T01:02:23.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:23.762+0000 7ff23d00f700 1 -- 172.21.15.7:0/116390688 wait complete. 2024-06-10T01:02:24.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:24 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/116390688' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-10T01:02:24.501 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-06-10T01:02:24.501 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-10T01:02:24.501 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-06-10T01:02:24.501 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-06-10T01:02:24.501 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-06-10T01:02:24.501 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-06-10T01:02:24.501 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-10T01:02:24.501 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-10T01:02:24.501 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-06-10T01:02:24.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:02:24.502 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-06-10T01:02:24.521 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-10T01:02:24.521 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-10T01:02:24.521 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-06-10T01:02:24.521 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-06-10T01:02:24.521 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-06-10T01:02:24.521 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-06-10T01:02:24.521 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-10T01:02:24.521 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-06-10T01:02:24.522 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-06-10T01:02:24.522 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-10T01:02:24.522 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-06-10T01:02:24.522 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-06-10T01:02:24.522 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-10T01:02:24.522 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-06-10T01:02:24.522 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-06-10T01:02:24.522 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-06-10T01:02:24.522 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42212sec preferred_lft 42212sec 2024-06-10T01:02:24.522 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link noprefixroute 2024-06-10T01:02:24.522 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-06-10T01:02:24.522 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-10T01:02:24.522 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-06-10T01:02:24.523 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-06-10T01:02:24.523 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-10T01:02:24.523 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T01:02:24.523 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-06-10T01:02:24.523 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add name ceph-brx type bridge 2024-06-10T01:02:24.523 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr flush dev ceph-brx 2024-06-10T01:02:24.524 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set ceph-brx up 2024-06-10T01:02:24.524 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-10T01:02:24.524 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-06-10T01:02:24.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T01:02:24.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:24 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/116390688' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-10T01:02:24.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T01:02:24.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:02:24.735 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-10T01:02:24.762 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-06-10T01:02:24.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:02:24.764 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-06-10T01:02:24.829 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-06-10T01:02:24.829 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-10T01:02:24.829 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-06-10T01:02:24.829 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-06-10T01:02:24.829 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-10T01:02:24.830 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T01:02:24.830 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-06-10T01:02:24.830 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-06-10T01:02:24.830 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-06-10T01:02:24.830 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-06-10T01:02:24.830 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-06-10T01:02:24.832 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:24 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/116390688' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-10T01:02:24.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T01:02:25.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T01:02:25.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:02:25.161 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-06-10T01:02:25.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:02:25.178 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2024-06-10T01:02:25.234 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T01:02:25.234 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-06-10T01:02:25.234 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-10T01:02:25.234 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-06-10T01:02:25.234 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-06-10T01:02:25.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T01:02:25.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T01:02:25.355 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-06-10T01:02:25.355 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T01:02:25.355 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-06-10T01:02:25.355 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-06-10T01:02:25.355 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-10T01:02:25.355 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-06-10T01:02:25.356 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-06-10T01:02:25.356 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-06-10T01:02:25.356 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-06-10T01:02:25.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T01:02:25.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T01:02:25.489 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T01:02:25.489 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-06-10T01:02:25.489 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-06-10T01:02:25.489 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.0 master ceph-brx 2024-06-10T01:02:25.489 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-06-10T01:02:25.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T01:02:25.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T01:02:25.569 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-06-10T01:02:25.569 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-10T01:02:25.569 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-06-10T01:02:25.598 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-06-10T01:02:25.598 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-10T01:02:25.598 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-06-10T01:02:25.653 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-06-10T01:02:25.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:25 smithi175 ceph-mon[36636]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:25.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:25 smithi007 ceph-mon[32398]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:25.720 DEBUG:teuthology.orchestra.run.smithi007:> cat /proc/self/mounts | awk '{print $2}' 2024-06-10T01:02:25.777 INFO:teuthology.orchestra.run.smithi007.stdout:/proc 2024-06-10T01:02:25.777 INFO:teuthology.orchestra.run.smithi007.stdout:/sys 2024-06-10T01:02:25.777 INFO:teuthology.orchestra.run.smithi007.stdout:/dev 2024-06-10T01:02:25.777 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/security 2024-06-10T01:02:25.778 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/shm 2024-06-10T01:02:25.778 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/pts 2024-06-10T01:02:25.778 INFO:teuthology.orchestra.run.smithi007.stdout:/run 2024-06-10T01:02:25.778 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup 2024-06-10T01:02:25.778 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/pstore 2024-06-10T01:02:25.778 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/bpf 2024-06-10T01:02:25.778 INFO:teuthology.orchestra.run.smithi007.stdout:/ 2024-06-10T01:02:25.778 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/selinux 2024-06-10T01:02:25.778 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-06-10T01:02:25.778 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/mqueue 2024-06-10T01:02:25.778 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/debug 2024-06-10T01:02:25.778 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/tracing 2024-06-10T01:02:25.778 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/hugepages 2024-06-10T01:02:25.779 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/fuse/connections 2024-06-10T01:02:25.779 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/config 2024-06-10T01:02:25.779 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-sysctl.service 2024-06-10T01:02:25.779 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-10T01:02:25.779 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-10T01:02:25.779 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/nfs/rpc_pipefs 2024-06-10T01:02:25.779 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/1000 2024-06-10T01:02:25.779 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph 2024-06-10T01:02:25.779 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay 2024-06-10T01:02:25.779 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/0a0e586bacf1bfe1dd9d0a49b4601eeaafe55dd5a5f8521fb2db251bfaa232ac/merged 2024-06-10T01:02:25.779 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/8af986fd7ea3295f3ce02de6a6e03fbd6edf6e3e8bf09fb155fb623596c5d04d/merged 2024-06-10T01:02:25.779 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/0 2024-06-10T01:02:25.779 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-06-10T01:02:25.780 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/c1a689a27be89a1b21d4e0ff4b0bf849c63d494bf5ce39c17abc335bc422f196/merged 2024-06-10T01:02:25.780 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/92c27237951b20ceaa436dbd7c0e60edafbab22ffc73f5c06c979e5730340aa6/merged 2024-06-10T01:02:25.780 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/d3ce0993f7a4f08340e8fb003c3a5eb5d839ee56828ca8ac8a71450886b538bf/merged 2024-06-10T01:02:25.780 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/0daface4477d195ef09acefd0e4283ac407764e5f973504f71e394076b1715b3/merged 2024-06-10T01:02:25.780 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/598a8cd979b381dc743f2eb03e00dfa25bb947b77868f2150d5f59ec5edb752b/merged 2024-06-10T01:02:25.780 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/781141a98e7a0475b0a7754a82351a6796bef4910a7e7d257d29c2a3a59ccf8e/merged 2024-06-10T01:02:25.780 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/827aa451762745e69eccd76a320f354a1243b45dfe30391d653cb31408d54a3b/merged 2024-06-10T01:02:25.780 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/acaceea3ff205cb40f3ef66a99fe186a4132dc345ca3db7da23715f6bfa2a635/merged 2024-06-10T01:02:25.780 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns 2024-06-10T01:02:25.780 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-10T01:02:25.780 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-10T01:02:25.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:02:25.781 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-06-10T01:02:25.832 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:25 smithi078 ceph-mon[36683]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:25.838 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-06-10T01:02:25.838 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-06-10T01:02:25.880 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-06-10T01:02:25.905 DEBUG:teuthology.orchestra.run.smithi007:> cat /proc/self/mounts | awk '{print $2}' 2024-06-10T01:02:25.962 INFO:teuthology.orchestra.run.smithi007.stdout:/proc 2024-06-10T01:02:25.962 INFO:teuthology.orchestra.run.smithi007.stdout:/sys 2024-06-10T01:02:25.962 INFO:teuthology.orchestra.run.smithi007.stdout:/dev 2024-06-10T01:02:25.962 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/security 2024-06-10T01:02:25.963 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/shm 2024-06-10T01:02:25.963 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/pts 2024-06-10T01:02:25.963 INFO:teuthology.orchestra.run.smithi007.stdout:/run 2024-06-10T01:02:25.963 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup 2024-06-10T01:02:25.963 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/pstore 2024-06-10T01:02:25.963 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/bpf 2024-06-10T01:02:25.963 INFO:teuthology.orchestra.run.smithi007.stdout:/ 2024-06-10T01:02:25.963 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/selinux 2024-06-10T01:02:25.963 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-06-10T01:02:25.963 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/mqueue 2024-06-10T01:02:25.963 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/debug 2024-06-10T01:02:25.963 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/tracing 2024-06-10T01:02:25.964 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/hugepages 2024-06-10T01:02:25.964 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/fuse/connections 2024-06-10T01:02:25.964 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/config 2024-06-10T01:02:25.964 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-sysctl.service 2024-06-10T01:02:25.964 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-10T01:02:25.964 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-10T01:02:25.964 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/nfs/rpc_pipefs 2024-06-10T01:02:25.964 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/1000 2024-06-10T01:02:25.964 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph 2024-06-10T01:02:25.964 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay 2024-06-10T01:02:25.964 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/0a0e586bacf1bfe1dd9d0a49b4601eeaafe55dd5a5f8521fb2db251bfaa232ac/merged 2024-06-10T01:02:25.964 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/8af986fd7ea3295f3ce02de6a6e03fbd6edf6e3e8bf09fb155fb623596c5d04d/merged 2024-06-10T01:02:25.964 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/0 2024-06-10T01:02:25.965 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-06-10T01:02:25.965 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/c1a689a27be89a1b21d4e0ff4b0bf849c63d494bf5ce39c17abc335bc422f196/merged 2024-06-10T01:02:25.965 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/92c27237951b20ceaa436dbd7c0e60edafbab22ffc73f5c06c979e5730340aa6/merged 2024-06-10T01:02:25.965 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/d3ce0993f7a4f08340e8fb003c3a5eb5d839ee56828ca8ac8a71450886b538bf/merged 2024-06-10T01:02:25.965 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/0daface4477d195ef09acefd0e4283ac407764e5f973504f71e394076b1715b3/merged 2024-06-10T01:02:25.965 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/598a8cd979b381dc743f2eb03e00dfa25bb947b77868f2150d5f59ec5edb752b/merged 2024-06-10T01:02:25.965 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/781141a98e7a0475b0a7754a82351a6796bef4910a7e7d257d29c2a3a59ccf8e/merged 2024-06-10T01:02:25.965 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/827aa451762745e69eccd76a320f354a1243b45dfe30391d653cb31408d54a3b/merged 2024-06-10T01:02:25.965 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/acaceea3ff205cb40f3ef66a99fe186a4132dc345ca3db7da23715f6bfa2a635/merged 2024-06-10T01:02:25.965 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns 2024-06-10T01:02:25.965 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-10T01:02:25.965 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-10T01:02:25.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:02:25.966 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-06-10T01:02:26.020 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi007.stderr:2024-06-10T01:02:26.022+0000 7f3d9ec70480 -1 init, newargv = 0x55efd9996810 newargc=15 2024-06-10T01:02:26.020 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi007.stderr:ceph-fuse[78897]: starting ceph client 2024-06-10T01:02:26.026 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi007.stderr:ceph-fuse[78897]: starting fuse 2024-06-10T01:02:27.120 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-06-10T01:02:27.153 DEBUG:teuthology.orchestra.run.smithi007:> cat /proc/self/mounts | awk '{print $2}' 2024-06-10T01:02:27.213 INFO:teuthology.orchestra.run.smithi007.stdout:/proc 2024-06-10T01:02:27.213 INFO:teuthology.orchestra.run.smithi007.stdout:/sys 2024-06-10T01:02:27.213 INFO:teuthology.orchestra.run.smithi007.stdout:/dev 2024-06-10T01:02:27.213 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/security 2024-06-10T01:02:27.214 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/shm 2024-06-10T01:02:27.214 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/pts 2024-06-10T01:02:27.214 INFO:teuthology.orchestra.run.smithi007.stdout:/run 2024-06-10T01:02:27.214 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup 2024-06-10T01:02:27.214 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/pstore 2024-06-10T01:02:27.214 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/bpf 2024-06-10T01:02:27.214 INFO:teuthology.orchestra.run.smithi007.stdout:/ 2024-06-10T01:02:27.214 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/selinux 2024-06-10T01:02:27.214 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-06-10T01:02:27.214 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/mqueue 2024-06-10T01:02:27.214 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/debug 2024-06-10T01:02:27.214 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/tracing 2024-06-10T01:02:27.214 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/hugepages 2024-06-10T01:02:27.215 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/fuse/connections 2024-06-10T01:02:27.215 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/config 2024-06-10T01:02:27.215 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-sysctl.service 2024-06-10T01:02:27.215 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-10T01:02:27.215 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-10T01:02:27.215 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/nfs/rpc_pipefs 2024-06-10T01:02:27.215 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/1000 2024-06-10T01:02:27.215 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph 2024-06-10T01:02:27.215 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay 2024-06-10T01:02:27.215 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/0a0e586bacf1bfe1dd9d0a49b4601eeaafe55dd5a5f8521fb2db251bfaa232ac/merged 2024-06-10T01:02:27.215 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/8af986fd7ea3295f3ce02de6a6e03fbd6edf6e3e8bf09fb155fb623596c5d04d/merged 2024-06-10T01:02:27.215 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/0 2024-06-10T01:02:27.215 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-06-10T01:02:27.216 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/c1a689a27be89a1b21d4e0ff4b0bf849c63d494bf5ce39c17abc335bc422f196/merged 2024-06-10T01:02:27.216 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/92c27237951b20ceaa436dbd7c0e60edafbab22ffc73f5c06c979e5730340aa6/merged 2024-06-10T01:02:27.216 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/d3ce0993f7a4f08340e8fb003c3a5eb5d839ee56828ca8ac8a71450886b538bf/merged 2024-06-10T01:02:27.216 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/0daface4477d195ef09acefd0e4283ac407764e5f973504f71e394076b1715b3/merged 2024-06-10T01:02:27.216 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/598a8cd979b381dc743f2eb03e00dfa25bb947b77868f2150d5f59ec5edb752b/merged 2024-06-10T01:02:27.216 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/781141a98e7a0475b0a7754a82351a6796bef4910a7e7d257d29c2a3a59ccf8e/merged 2024-06-10T01:02:27.216 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/827aa451762745e69eccd76a320f354a1243b45dfe30391d653cb31408d54a3b/merged 2024-06-10T01:02:27.216 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/acaceea3ff205cb40f3ef66a99fe186a4132dc345ca3db7da23715f6bfa2a635/merged 2024-06-10T01:02:27.216 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns 2024-06-10T01:02:27.216 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-10T01:02:27.216 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-10T01:02:27.216 INFO:teuthology.orchestra.run.smithi007.stdout:/home/ubuntu/cephtest/mnt.admin 2024-06-10T01:02:27.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:02:27.217 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-06-10T01:02:27.271 INFO:teuthology.orchestra.run.smithi007.stdout:107 2024-06-10T01:02:27.272 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [107] 2024-06-10T01:02:27.272 DEBUG:teuthology.orchestra.run.smithi007:> sudo stdin-killer -- python3 -c ' 2024-06-10T01:02:27.272 DEBUG:teuthology.orchestra.run.smithi007:> import glob 2024-06-10T01:02:27.272 DEBUG:teuthology.orchestra.run.smithi007:> import re 2024-06-10T01:02:27.272 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-06-10T01:02:27.272 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2024-06-10T01:02:27.272 DEBUG:teuthology.orchestra.run.smithi007:> 2024-06-10T01:02:27.272 DEBUG:teuthology.orchestra.run.smithi007:> def _find_admin_socket(client_name): 2024-06-10T01:02:27.272 DEBUG:teuthology.orchestra.run.smithi007:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-06-10T01:02:27.273 DEBUG:teuthology.orchestra.run.smithi007:> files = glob.glob(asok_path) 2024-06-10T01:02:27.273 DEBUG:teuthology.orchestra.run.smithi007:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-06-10T01:02:27.273 DEBUG:teuthology.orchestra.run.smithi007:> 2024-06-10T01:02:27.273 DEBUG:teuthology.orchestra.run.smithi007:> # Given a non-glob path, it better be there 2024-06-10T01:02:27.273 DEBUG:teuthology.orchestra.run.smithi007:> if "*" not in asok_path: 2024-06-10T01:02:27.273 DEBUG:teuthology.orchestra.run.smithi007:> assert(len(files) == 1) 2024-06-10T01:02:27.273 DEBUG:teuthology.orchestra.run.smithi007:> return files[0] 2024-06-10T01:02:27.273 DEBUG:teuthology.orchestra.run.smithi007:> 2024-06-10T01:02:27.273 DEBUG:teuthology.orchestra.run.smithi007:> for f in files: 2024-06-10T01:02:27.273 DEBUG:teuthology.orchestra.run.smithi007:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-06-10T01:02:27.273 DEBUG:teuthology.orchestra.run.smithi007:> if os.path.exists("/proc/{0}".format(pid)): 2024-06-10T01:02:27.273 DEBUG:teuthology.orchestra.run.smithi007:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-06-10T01:02:27.273 DEBUG:teuthology.orchestra.run.smithi007:> contents = proc_f.read() 2024-06-10T01:02:27.273 DEBUG:teuthology.orchestra.run.smithi007:> if mountpoint in contents: 2024-06-10T01:02:27.274 DEBUG:teuthology.orchestra.run.smithi007:> return f 2024-06-10T01:02:27.274 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-06-10T01:02:27.274 DEBUG:teuthology.orchestra.run.smithi007:> 2024-06-10T01:02:27.274 DEBUG:teuthology.orchestra.run.smithi007:> print(_find_admin_socket("client.admin")) 2024-06-10T01:02:27.274 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-06-10T01:02:27.399 INFO:teuthology.orchestra.run.smithi007.stdout:/var/run/ceph/ceph-client.admin.78897.asok 2024-06-10T01:02:27.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T01:02:27.408 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:27 smithi007 ceph-mon[32398]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:27.410 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.78897.asok 2024-06-10T01:02:27.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:02:27.410 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.78897.asok status 2024-06-10T01:02:27.550 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:02:27.550 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": { 2024-06-10T01:02:27.550 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_sha1": "3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-10T01:02:27.550 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-10T01:02:27.550 INFO:teuthology.orchestra.run.smithi007.stdout: "entity_id": "admin", 2024-06-10T01:02:27.550 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": "smithi007", 2024-06-10T01:02:27.550 INFO:teuthology.orchestra.run.smithi007.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-06-10T01:02:27.550 INFO:teuthology.orchestra.run.smithi007.stdout: "pid": "78897", 2024-06-10T01:02:27.550 INFO:teuthology.orchestra.run.smithi007.stdout: "root": "/" 2024-06-10T01:02:27.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:02:27.550 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_count": 0, 2024-06-10T01:02:27.551 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_pinned_count": 0, 2024-06-10T01:02:27.551 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 14712, 2024-06-10T01:02:27.551 INFO:teuthology.orchestra.run.smithi007.stdout: "inst": { 2024-06-10T01:02:27.551 INFO:teuthology.orchestra.run.smithi007.stdout: "name": { 2024-06-10T01:02:27.551 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-06-10T01:02:27.551 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 14712 2024-06-10T01:02:27.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:02:27.551 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2024-06-10T01:02:27.551 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-06-10T01:02:27.551 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2024-06-10T01:02:27.551 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3723834346 2024-06-10T01:02:27.551 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:02:27.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:02:27.551 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2024-06-10T01:02:27.552 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-06-10T01:02:27.552 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2024-06-10T01:02:27.552 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3723834346 2024-06-10T01:02:27.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:02:27.552 INFO:teuthology.orchestra.run.smithi007.stdout: "inst_str": "client.14712 192.168.0.1:0/3723834346", 2024-06-10T01:02:27.552 INFO:teuthology.orchestra.run.smithi007.stdout: "addr_str": "192.168.0.1:0/3723834346", 2024-06-10T01:02:27.552 INFO:teuthology.orchestra.run.smithi007.stdout: "inode_count": 1, 2024-06-10T01:02:27.552 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_epoch": 17, 2024-06-10T01:02:27.552 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch": 95, 2024-06-10T01:02:27.552 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch_barrier": 0, 2024-06-10T01:02:27.552 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklisted": false, 2024-06-10T01:02:27.552 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs" 2024-06-10T01:02:27.552 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:02:27.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:02:27.561 DEBUG:teuthology.orchestra.run.smithi007:> stat --file-system '--printf=%T 2024-06-10T01:02:27.561 DEBUG:teuthology.orchestra.run.smithi007:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-06-10T01:02:27.577 INFO:teuthology.orchestra.run.smithi007.stdout:fuseblk 2024-06-10T01:02:27.578 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-06-10T01:02:27.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:02:27.578 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-06-10T01:02:27.644 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-06-10T01:02:27.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:27 smithi175 ceph-mon[36636]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:27.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-10T01:02:27.760 INFO:teuthology.orchestra.run.smithi007.stdout:# file: . 2024-06-10T01:02:27.760 INFO:teuthology.orchestra.run.smithi007.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-06-10T01:02:27.760 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:02:27.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T01:02:27.766 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-10T01:02:27.766 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-10T01:02:27.784 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-10T01:02:27.832 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:27 smithi078 ceph-mon[36683]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:28.074 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:02:29.497 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:29 smithi007 ceph-mon[32398]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:29.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:29 smithi175 ceph-mon[36636]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:29.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.741+0000 7f6074217700 1 -- 172.21.15.7:0/2356810003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f606c102740 msgr2=0x7f606c102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:29.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.741+0000 7f6074217700 1 --2- 172.21.15.7:0/2356810003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f606c102740 0x7f606c102b20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f60680099c0 tx=0x7f6068009cd0 comp rx=0 tx=0).stop 2024-06-10T01:02:29.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.741+0000 7f6074217700 1 -- 172.21.15.7:0/2356810003 shutdown_connections 2024-06-10T01:02:29.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.741+0000 7f6074217700 1 --2- 172.21.15.7:0/2356810003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f606c107840 0x7f606c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:29.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.741+0000 7f6074217700 1 --2- 172.21.15.7:0/2356810003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f606c102740 0x7f606c102b20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:29.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.741+0000 7f6074217700 1 --2- 172.21.15.7:0/2356810003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f606c1030f0 0x7f606c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:29.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.741+0000 7f6074217700 1 -- 172.21.15.7:0/2356810003 >> 172.21.15.7:0/2356810003 conn(0x7f606c0795f0 msgr2=0x7f606c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:29.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.741+0000 7f6074217700 1 -- 172.21.15.7:0/2356810003 shutdown_connections 2024-06-10T01:02:29.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.741+0000 7f6074217700 1 -- 172.21.15.7:0/2356810003 wait complete. 2024-06-10T01:02:29.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.742+0000 7f6074217700 1 Processor -- start 2024-06-10T01:02:29.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.742+0000 7f6074217700 1 -- start start 2024-06-10T01:02:29.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.742+0000 7f6074217700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f606c102740 0x7f606c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:29.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.742+0000 7f6074217700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f606c1030f0 0x7f606c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:29.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.742+0000 7f6074217700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f606c107840 0x7f606c075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:29.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.742+0000 7f6074217700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f606c112760 con 0x7f606c102740 2024-06-10T01:02:29.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.742+0000 7f6074217700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f606c1125e0 con 0x7f606c107840 2024-06-10T01:02:29.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.742+0000 7f6074217700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f606c1128e0 con 0x7f606c1030f0 2024-06-10T01:02:29.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.742+0000 7f6071fb3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f606c102740 0x7f606c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:29.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.742+0000 7f6071fb3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f606c102740 0x7f606c074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55572/0 (socket says 172.21.15.7:55572) 2024-06-10T01:02:29.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.742+0000 7f6071fb3700 1 -- 172.21.15.7:0/2310700749 learned_addr learned my addr 172.21.15.7:0/2310700749 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:02:29.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.742+0000 7f60717b2700 1 --2- 172.21.15.7:0/2310700749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f606c1030f0 0x7f606c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:29.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.742+0000 7f6071fb3700 1 -- 172.21.15.7:0/2310700749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f606c1030f0 msgr2=0x7f606c074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:29.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.742+0000 7f6071fb3700 1 --2- 172.21.15.7:0/2310700749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f606c1030f0 0x7f606c074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:29.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.742+0000 7f60727b4700 1 --2- 172.21.15.7:0/2310700749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f606c107840 0x7f606c075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:29.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.742+0000 7f6071fb3700 1 -- 172.21.15.7:0/2310700749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f606c107840 msgr2=0x7f606c075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:29.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.743+0000 7f6071fb3700 1 --2- 172.21.15.7:0/2310700749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f606c107840 0x7f606c075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:29.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.743+0000 7f6071fb3700 1 -- 172.21.15.7:0/2310700749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606c1a7000 con 0x7f606c102740 2024-06-10T01:02:29.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.743+0000 7f60727b4700 1 --2- 172.21.15.7:0/2310700749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f606c107840 0x7f606c075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:02:29.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.743+0000 7f60717b2700 1 --2- 172.21.15.7:0/2310700749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f606c1030f0 0x7f606c074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:02:29.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.743+0000 7f6071fb3700 1 --2- 172.21.15.7:0/2310700749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f606c102740 0x7f606c074790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6068006010 tx=0x7f6068005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:29.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.743+0000 7f6062ffd700 1 -- 172.21.15.7:0/2310700749 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6068027070 con 0x7f606c102740 2024-06-10T01:02:29.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.743+0000 7f6062ffd700 1 -- 172.21.15.7:0/2310700749 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60680040d0 con 0x7f606c102740 2024-06-10T01:02:29.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.743+0000 7f6074217700 1 -- 172.21.15.7:0/2310700749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f606c1a7290 con 0x7f606c102740 2024-06-10T01:02:29.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.743+0000 7f6062ffd700 1 -- 172.21.15.7:0/2310700749 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606800f460 con 0x7f606c102740 2024-06-10T01:02:29.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.743+0000 7f6074217700 1 -- 172.21.15.7:0/2310700749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f606c1a76e0 con 0x7f606c102740 2024-06-10T01:02:29.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.744+0000 7f6074217700 1 -- 172.21.15.7:0/2310700749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f606c069180 con 0x7f606c102740 2024-06-10T01:02:29.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.745+0000 7f6062ffd700 1 -- 172.21.15.7:0/2310700749 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f606802a070 con 0x7f606c102740 2024-06-10T01:02:29.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.746+0000 7f6062ffd700 1 --2- 172.21.15.7:0/2310700749 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f60580a2c20 0x7f60580a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:29.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.746+0000 7f60717b2700 1 --2- 172.21.15.7:0/2310700749 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f60580a2c20 0x7f60580a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:29.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.746+0000 7f6062ffd700 1 -- 172.21.15.7:0/2310700749 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7f60680d0980 con 0x7f606c102740 2024-06-10T01:02:29.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.746+0000 7f60717b2700 1 --2- 172.21.15.7:0/2310700749 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f60580a2c20 0x7f60580a50e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f605c005fd0 tx=0x7f605c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:29.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.750+0000 7f6062ffd700 1 -- 172.21.15.7:0/2310700749 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6068031020 con 0x7f606c102740 2024-06-10T01:02:29.832 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:29 smithi078 ceph-mon[36683]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:29.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.974+0000 7f6074217700 1 -- 172.21.15.7:0/2310700749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f606c1a7a90 con 0x7f606c102740 2024-06-10T01:02:29.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.975+0000 7f6062ffd700 1 -- 172.21.15.7:0/2310700749 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v95) ==== 81+0+580 (secure 0 0 0) 0x7f6068013070 con 0x7f606c102740 2024-06-10T01:02:29.973 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1833397470 2024-06-11T00:54:35.441123+0000 2024-06-10T01:02:29.973 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/322543993 2024-06-11T00:54:35.441123+0000 2024-06-10T01:02:29.973 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/212510583 2024-06-11T00:54:35.441123+0000 2024-06-10T01:02:29.973 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/3298829389 2024-06-11T00:54:35.441123+0000 2024-06-10T01:02:29.973 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/1100766262 2024-06-11T00:54:08.102405+0000 2024-06-10T01:02:29.973 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/2679436843 2024-06-11T00:54:08.102405+0000 2024-06-10T01:02:29.973 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/1100766262 2024-06-11T00:54:08.102405+0000 2024-06-10T01:02:29.973 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1210355277 2024-06-11T00:54:08.102405+0000 2024-06-10T01:02:29.973 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/3298829389 2024-06-11T00:54:35.441123+0000 2024-06-10T01:02:29.973 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3610017406 2024-06-11T00:54:08.102405+0000 2024-06-10T01:02:29.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.977+0000 7f6074217700 1 -- 172.21.15.7:0/2310700749 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f60580a2c20 msgr2=0x7f60580a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:29.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.977+0000 7f6074217700 1 --2- 172.21.15.7:0/2310700749 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f60580a2c20 0x7f60580a50e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f605c005fd0 tx=0x7f605c005e30 comp rx=0 tx=0).stop 2024-06-10T01:02:29.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.977+0000 7f6074217700 1 -- 172.21.15.7:0/2310700749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f606c102740 msgr2=0x7f606c074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:29.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.977+0000 7f6074217700 1 --2- 172.21.15.7:0/2310700749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f606c102740 0x7f606c074790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6068006010 tx=0x7f6068005640 comp rx=0 tx=0).stop 2024-06-10T01:02:29.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.977+0000 7f6074217700 1 -- 172.21.15.7:0/2310700749 shutdown_connections 2024-06-10T01:02:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.977+0000 7f6074217700 1 --2- 172.21.15.7:0/2310700749 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f60580a2c20 0x7f60580a50e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.977+0000 7f6074217700 1 --2- 172.21.15.7:0/2310700749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f606c107840 0x7f606c075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.977+0000 7f6074217700 1 --2- 172.21.15.7:0/2310700749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f606c102740 0x7f606c074790 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.977+0000 7f6074217700 1 --2- 172.21.15.7:0/2310700749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f606c1030f0 0x7f606c074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.977+0000 7f6074217700 1 -- 172.21.15.7:0/2310700749 >> 172.21.15.7:0/2310700749 conn(0x7f606c0795f0 msgr2=0x7f606c1003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.977+0000 7f6074217700 1 -- 172.21.15.7:0/2310700749 shutdown_connections 2024-06-10T01:02:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:29.977+0000 7f6074217700 1 -- 172.21.15.7:0/2310700749 wait complete. 2024-06-10T01:02:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:listed 10 entries 2024-06-10T01:02:30.637 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:30 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2310700749' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T01:02:30.670 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-10T01:02:30.670 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-10T01:02:30.692 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-10T01:02:30.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:30 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2310700749' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T01:02:30.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:30 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2310700749' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T01:02:30.988 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:02:31.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:31 smithi175 ceph-mon[36636]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:31.766 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:31 smithi007 ceph-mon[32398]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:31.832 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:31 smithi078 ceph-mon[36683]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:32.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.729+0000 7f87dec92700 1 -- 172.21.15.7:0/1519504913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87d8107840 msgr2=0x7f87d810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:32.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.729+0000 7f87dec92700 1 --2- 172.21.15.7:0/1519504913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87d8107840 0x7f87d810f3c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f87d000b0d0 tx=0x7f87d000b3e0 comp rx=0 tx=0).stop 2024-06-10T01:02:32.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.729+0000 7f87dec92700 1 -- 172.21.15.7:0/1519504913 shutdown_connections 2024-06-10T01:02:32.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.729+0000 7f87dec92700 1 --2- 172.21.15.7:0/1519504913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87d8107840 0x7f87d810f3c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:32.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.729+0000 7f87dec92700 1 --2- 172.21.15.7:0/1519504913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87d8102740 0x7f87d8102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:32.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.729+0000 7f87dec92700 1 --2- 172.21.15.7:0/1519504913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87d81030f0 0x7f87d8107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:32.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.729+0000 7f87dec92700 1 -- 172.21.15.7:0/1519504913 >> 172.21.15.7:0/1519504913 conn(0x7f87d80795f0 msgr2=0x7f87d8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:32.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.729+0000 7f87dec92700 1 -- 172.21.15.7:0/1519504913 shutdown_connections 2024-06-10T01:02:32.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.729+0000 7f87dec92700 1 -- 172.21.15.7:0/1519504913 wait complete. 2024-06-10T01:02:32.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.729+0000 7f87dec92700 1 Processor -- start 2024-06-10T01:02:32.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.729+0000 7f87dec92700 1 -- start start 2024-06-10T01:02:32.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.730+0000 7f87dec92700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87d8102740 0x7f87d819e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:32.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.730+0000 7f87dec92700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87d81030f0 0x7f87d819ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:32.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.730+0000 7f87dec92700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87d8107840 0x7f87d81988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:32.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.730+0000 7f87dec92700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f87d8112760 con 0x7f87d8107840 2024-06-10T01:02:32.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.730+0000 7f87dec92700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f87d81125e0 con 0x7f87d8102740 2024-06-10T01:02:32.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.730+0000 7f87dec92700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f87d81128e0 con 0x7f87d81030f0 2024-06-10T01:02:32.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.730+0000 7f87dd22f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87d8107840 0x7f87d81988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:32.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.730+0000 7f87dd22f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87d8107840 0x7f87d81988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59496/0 (socket says 172.21.15.7:59496) 2024-06-10T01:02:32.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.730+0000 7f87dca2e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87d8102740 0x7f87d819e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:32.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.730+0000 7f87dd22f700 1 -- 172.21.15.7:0/1735698743 learned_addr learned my addr 172.21.15.7:0/1735698743 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:02:32.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.730+0000 7f87d7fff700 1 --2- 172.21.15.7:0/1735698743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87d81030f0 0x7f87d819ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:32.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.730+0000 7f87dd22f700 1 -- 172.21.15.7:0/1735698743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87d81030f0 msgr2=0x7f87d819ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:32.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.730+0000 7f87dd22f700 1 --2- 172.21.15.7:0/1735698743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87d81030f0 0x7f87d819ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:32.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.730+0000 7f87dd22f700 1 -- 172.21.15.7:0/1735698743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87d8102740 msgr2=0x7f87d819e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:32.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.730+0000 7f87dd22f700 1 --2- 172.21.15.7:0/1735698743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87d8102740 0x7f87d819e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:32.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.730+0000 7f87dd22f700 1 -- 172.21.15.7:0/1735698743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d8199000 con 0x7f87d8107840 2024-06-10T01:02:32.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.730+0000 7f87dca2e700 1 --2- 172.21.15.7:0/1735698743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87d8102740 0x7f87d819e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:32.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.730+0000 7f87d7fff700 1 --2- 172.21.15.7:0/1735698743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87d81030f0 0x7f87d819ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:02:32.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.730+0000 7f87dd22f700 1 --2- 172.21.15.7:0/1735698743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87d8107840 0x7f87d81988a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f87d0006070 tx=0x7f87d0004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:32.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.731+0000 7f87d5ffb700 1 -- 172.21.15.7:0/1735698743 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d0021070 con 0x7f87d8107840 2024-06-10T01:02:32.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.731+0000 7f87dec92700 1 -- 172.21.15.7:0/1735698743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87d8199290 con 0x7f87d8107840 2024-06-10T01:02:32.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.731+0000 7f87d5ffb700 1 -- 172.21.15.7:0/1735698743 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87d0007b80 con 0x7f87d8107840 2024-06-10T01:02:32.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.731+0000 7f87d5ffb700 1 -- 172.21.15.7:0/1735698743 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d0005330 con 0x7f87d8107840 2024-06-10T01:02:32.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.731+0000 7f87dec92700 1 -- 172.21.15.7:0/1735698743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87d81a55b0 con 0x7f87d8107840 2024-06-10T01:02:32.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.732+0000 7f87dec92700 1 -- 172.21.15.7:0/1735698743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87d8069180 con 0x7f87d8107840 2024-06-10T01:02:32.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.733+0000 7f87d5ffb700 1 -- 172.21.15.7:0/1735698743 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f87d0024070 con 0x7f87d8107840 2024-06-10T01:02:32.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.734+0000 7f87d5ffb700 1 --2- 172.21.15.7:0/1735698743 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f87c80a2c20 0x7f87c80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:32.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.734+0000 7f87d5ffb700 1 -- 172.21.15.7:0/1735698743 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7f87d00d0340 con 0x7f87d8107840 2024-06-10T01:02:32.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.734+0000 7f87dca2e700 1 --2- 172.21.15.7:0/1735698743 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f87c80a2c20 0x7f87c80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:32.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.734+0000 7f87dca2e700 1 --2- 172.21.15.7:0/1735698743 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f87c80a2c20 0x7f87c80a50e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f87d81999e0 tx=0x7f87cc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.737+0000 7f87d5ffb700 1 -- 172.21.15.7:0/1735698743 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f87d009d0f0 con 0x7f87d8107840 2024-06-10T01:02:32.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.959+0000 7f87dec92700 1 -- 172.21.15.7:0/1735698743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f87d8199dd0 con 0x7f87d8107840 2024-06-10T01:02:32.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.959+0000 7f87d5ffb700 1 -- 172.21.15.7:0/1735698743 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v95) ==== 81+0+580 (secure 0 0 0) 0x7f87d8199dd0 con 0x7f87d8107840 2024-06-10T01:02:32.958 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1833397470 2024-06-11T00:54:35.441123+0000 2024-06-10T01:02:32.958 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/322543993 2024-06-11T00:54:35.441123+0000 2024-06-10T01:02:32.958 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/212510583 2024-06-11T00:54:35.441123+0000 2024-06-10T01:02:32.958 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/3298829389 2024-06-11T00:54:35.441123+0000 2024-06-10T01:02:32.958 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/1100766262 2024-06-11T00:54:08.102405+0000 2024-06-10T01:02:32.958 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/2679436843 2024-06-11T00:54:08.102405+0000 2024-06-10T01:02:32.959 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/1100766262 2024-06-11T00:54:08.102405+0000 2024-06-10T01:02:32.959 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1210355277 2024-06-11T00:54:08.102405+0000 2024-06-10T01:02:32.959 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/3298829389 2024-06-11T00:54:35.441123+0000 2024-06-10T01:02:32.959 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3610017406 2024-06-11T00:54:08.102405+0000 2024-06-10T01:02:32.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.962+0000 7f87dec92700 1 -- 172.21.15.7:0/1735698743 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f87c80a2c20 msgr2=0x7f87c80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:32.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.962+0000 7f87dec92700 1 --2- 172.21.15.7:0/1735698743 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f87c80a2c20 0x7f87c80a50e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f87d81999e0 tx=0x7f87cc00b410 comp rx=0 tx=0).stop 2024-06-10T01:02:32.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.962+0000 7f87dec92700 1 -- 172.21.15.7:0/1735698743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87d8107840 msgr2=0x7f87d81988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:32.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.962+0000 7f87dec92700 1 --2- 172.21.15.7:0/1735698743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87d8107840 0x7f87d81988a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f87d0006070 tx=0x7f87d0004d00 comp rx=0 tx=0).stop 2024-06-10T01:02:32.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.962+0000 7f87dec92700 1 -- 172.21.15.7:0/1735698743 shutdown_connections 2024-06-10T01:02:32.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.962+0000 7f87dec92700 1 --2- 172.21.15.7:0/1735698743 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f87c80a2c20 0x7f87c80a50e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:32.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.962+0000 7f87dec92700 1 --2- 172.21.15.7:0/1735698743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87d8107840 0x7f87d81988a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:32.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.962+0000 7f87dec92700 1 --2- 172.21.15.7:0/1735698743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87d8102740 0x7f87d819e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:32.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.962+0000 7f87dec92700 1 --2- 172.21.15.7:0/1735698743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87d81030f0 0x7f87d819ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:32.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.962+0000 7f87dec92700 1 -- 172.21.15.7:0/1735698743 >> 172.21.15.7:0/1735698743 conn(0x7f87d80795f0 msgr2=0x7f87d81003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:32.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.962+0000 7f87dec92700 1 -- 172.21.15.7:0/1735698743 shutdown_connections 2024-06-10T01:02:32.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:32.962+0000 7f87dec92700 1 -- 172.21.15.7:0/1735698743 wait complete. 2024-06-10T01:02:32.961 INFO:teuthology.orchestra.run.smithi007.stderr:listed 10 entries 2024-06-10T01:02:33.563 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:33 smithi007 ceph-mon[32398]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-10T01:02:33.564 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:33 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1735698743' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T01:02:33.593 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi007.front.sepia.ceph.com... 2024-06-10T01:02:33.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:02:33.594 DEBUG:teuthology.orchestra.run.smithi007:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-06-10T01:02:33.622 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi007.stderr:ceph-fuse[78897]: fuse finished with error 0 and tester_r 0 2024-06-10T01:02:33.624 INFO:teuthology.orchestra.run:waiting for 300 2024-06-10T01:02:33.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:33 smithi175 ceph-mon[36636]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-10T01:02:33.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:33 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1735698743' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T01:02:33.832 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:33 smithi078 ceph-mon[36683]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-10T01:02:33.832 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:33 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1735698743' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T01:02:35.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:35 smithi175 ceph-mon[36636]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-10T01:02:35.766 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:35 smithi007 ceph-mon[32398]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-10T01:02:35.832 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:35 smithi078 ceph-mon[36683]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-10T01:02:36.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-06-10T01:02:36.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-06-10T01:02:36.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-06-10T01:02:36.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-06-10T01:02:36.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-06-10T01:02:36.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-06-10T01:02:36.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-06-10T01:02:36.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-06-10T01:02:36.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 8]}]: dispatch 2024-06-10T01:02:36.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 10]}]: dispatch 2024-06-10T01:02:36.766 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-06-10T01:02:36.766 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-06-10T01:02:36.766 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-06-10T01:02:36.766 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-06-10T01:02:36.766 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-06-10T01:02:36.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-06-10T01:02:36.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-06-10T01:02:36.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-06-10T01:02:36.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 8]}]: dispatch 2024-06-10T01:02:36.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 10]}]: dispatch 2024-06-10T01:02:36.832 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-06-10T01:02:36.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-06-10T01:02:36.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-06-10T01:02:36.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-06-10T01:02:36.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-06-10T01:02:36.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-06-10T01:02:36.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-06-10T01:02:36.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-06-10T01:02:36.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 8]}]: dispatch 2024-06-10T01:02:36.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 10]}]: dispatch 2024-06-10T01:02:37.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:37 smithi175 ceph-mon[36636]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-10T01:02:37.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-06-10T01:02:37.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-06-10T01:02:37.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-06-10T01:02:37.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-06-10T01:02:37.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-06-10T01:02:37.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-06-10T01:02:37.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-06-10T01:02:37.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-06-10T01:02:37.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 8]}]': finished 2024-06-10T01:02:37.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 10]}]': finished 2024-06-10T01:02:37.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:37 smithi175 ceph-mon[36636]: osdmap e96: 12 total, 12 up, 12 in 2024-06-10T01:02:37.766 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:37 smithi007 ceph-mon[32398]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-10T01:02:37.766 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-06-10T01:02:37.766 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-06-10T01:02:37.766 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-06-10T01:02:37.766 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-06-10T01:02:37.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-06-10T01:02:37.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-06-10T01:02:37.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-06-10T01:02:37.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-06-10T01:02:37.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 8]}]': finished 2024-06-10T01:02:37.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 10]}]': finished 2024-06-10T01:02:37.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:37 smithi007 ceph-mon[32398]: osdmap e96: 12 total, 12 up, 12 in 2024-06-10T01:02:37.832 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:37 smithi078 ceph-mon[36683]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-10T01:02:37.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-06-10T01:02:37.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-06-10T01:02:37.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-06-10T01:02:37.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-06-10T01:02:37.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-06-10T01:02:37.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-06-10T01:02:37.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-06-10T01:02:37.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-06-10T01:02:37.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 8]}]': finished 2024-06-10T01:02:37.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 10]}]': finished 2024-06-10T01:02:37.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:37 smithi078 ceph-mon[36683]: osdmap e96: 12 total, 12 up, 12 in 2024-06-10T01:02:38.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:38 smithi175 ceph-mon[36636]: osdmap e97: 12 total, 12 up, 12 in 2024-06-10T01:02:38.766 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:38 smithi007 ceph-mon[32398]: osdmap e97: 12 total, 12 up, 12 in 2024-06-10T01:02:38.832 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:38 smithi078 ceph-mon[36683]: osdmap e97: 12 total, 12 up, 12 in 2024-06-10T01:02:39.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:39 smithi175 ceph-mon[36636]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s 2024-06-10T01:02:39.725 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-06-10T01:02:39.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:02:39.725 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-06-10T01:02:39.752 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T01:02:39.766 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:39 smithi007 ceph-mon[32398]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s 2024-06-10T01:02:39.832 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:39 smithi078 ceph-mon[36683]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s 2024-06-10T01:02:40.023 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:02:41.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:41 smithi007 ceph-mon[32398]: pgmap v329: 129 pgs: 1 activating, 128 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T01:02:41.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:41 smithi175 ceph-mon[36636]: pgmap v329: 129 pgs: 1 activating, 128 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T01:02:41.832 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:41 smithi078 ceph-mon[36683]: pgmap v329: 129 pgs: 1 activating, 128 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T01:02:41.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.993+0000 7f35ffad0700 1 -- 172.21.15.7:0/4218458479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35f8101390 msgr2=0x7f35f8101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:41.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.993+0000 7f35ffad0700 1 --2- 172.21.15.7:0/4218458479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35f8101390 0x7f35f8101810 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f35e8009990 tx=0x7f35e8009ca0 comp rx=0 tx=0).stop 2024-06-10T01:02:41.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.993+0000 7f35ffad0700 1 -- 172.21.15.7:0/4218458479 shutdown_connections 2024-06-10T01:02:41.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.993+0000 7f35ffad0700 1 --2- 172.21.15.7:0/4218458479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f8101d50 0x7f35f810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:41.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.993+0000 7f35ffad0700 1 --2- 172.21.15.7:0/4218458479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35f8107400 0x7f35f81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:41.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.993+0000 7f35ffad0700 1 --2- 172.21.15.7:0/4218458479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35f8101390 0x7f35f8101810 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:41.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.993+0000 7f35ffad0700 1 -- 172.21.15.7:0/4218458479 >> 172.21.15.7:0/4218458479 conn(0x7f35f8078340 msgr2=0x7f35f8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:41.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.994+0000 7f35ffad0700 1 -- 172.21.15.7:0/4218458479 shutdown_connections 2024-06-10T01:02:41.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.994+0000 7f35ffad0700 1 -- 172.21.15.7:0/4218458479 wait complete. 2024-06-10T01:02:41.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.994+0000 7f35ffad0700 1 Processor -- start 2024-06-10T01:02:41.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.994+0000 7f35ffad0700 1 -- start start 2024-06-10T01:02:41.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.995+0000 7f35ffad0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35f8101390 0x7f35f819d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:41.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.995+0000 7f35ffad0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f8101d50 0x7f35f819dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:41.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.995+0000 7f35ffad0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35f8107400 0x7f35f8197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:41.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.995+0000 7f35ffad0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f35f81116c0 con 0x7f35f8101390 2024-06-10T01:02:41.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.995+0000 7f35ffad0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f35f8111540 con 0x7f35f8101d50 2024-06-10T01:02:41.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.995+0000 7f35ffad0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f35f8111840 con 0x7f35f8107400 2024-06-10T01:02:41.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.995+0000 7f35fd86c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35f8101390 0x7f35f819d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:41.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.995+0000 7f35fd86c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35f8101390 0x7f35f819d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53696/0 (socket says 172.21.15.7:53696) 2024-06-10T01:02:41.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.995+0000 7f35fd86c700 1 -- 172.21.15.7:0/1985024026 learned_addr learned my addr 172.21.15.7:0/1985024026 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:02:41.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.995+0000 7f35fd06b700 1 --2- 172.21.15.7:0/1985024026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f8101d50 0x7f35f819dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:41.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.995+0000 7f35fe06d700 1 --2- 172.21.15.7:0/1985024026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35f8107400 0x7f35f8197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:41.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.995+0000 7f35fd86c700 1 -- 172.21.15.7:0/1985024026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35f8107400 msgr2=0x7f35f8197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:41.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.995+0000 7f35fd86c700 1 --2- 172.21.15.7:0/1985024026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35f8107400 0x7f35f8197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:41.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.995+0000 7f35fd86c700 1 -- 172.21.15.7:0/1985024026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f8101d50 msgr2=0x7f35f819dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:41.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.995+0000 7f35fd86c700 1 --2- 172.21.15.7:0/1985024026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f8101d50 0x7f35f819dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:41.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.995+0000 7f35fd86c700 1 -- 172.21.15.7:0/1985024026 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f81980a0 con 0x7f35f8101390 2024-06-10T01:02:41.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.995+0000 7f35fd06b700 1 --2- 172.21.15.7:0/1985024026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f8101d50 0x7f35f819dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:02:41.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.995+0000 7f35fd86c700 1 --2- 172.21.15.7:0/1985024026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35f8101390 0x7f35f819d780 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f35f400d950 tx=0x7f35f400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:41.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.996+0000 7f35eeffd700 1 -- 172.21.15.7:0/1985024026 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f4016070 con 0x7f35f8101390 2024-06-10T01:02:41.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.996+0000 7f35ffad0700 1 -- 172.21.15.7:0/1985024026 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35f8198390 con 0x7f35f8101390 2024-06-10T01:02:41.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.996+0000 7f35eeffd700 1 -- 172.21.15.7:0/1985024026 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35f4009b40 con 0x7f35f8101390 2024-06-10T01:02:41.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.996+0000 7f35eeffd700 1 -- 172.21.15.7:0/1985024026 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f400f620 con 0x7f35f8101390 2024-06-10T01:02:41.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.996+0000 7f35ffad0700 1 -- 172.21.15.7:0/1985024026 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35f806a170 con 0x7f35f8101390 2024-06-10T01:02:41.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.997+0000 7f35ffad0700 1 -- 172.21.15.7:0/1985024026 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35f8069180 con 0x7f35f8101390 2024-06-10T01:02:41.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.998+0000 7f35eeffd700 1 -- 172.21.15.7:0/1985024026 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f35f4019070 con 0x7f35f8101390 2024-06-10T01:02:41.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.998+0000 7f35eeffd700 1 --2- 172.21.15.7:0/1985024026 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35e40a2c20 0x7f35e40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:41.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.999+0000 7f35fd06b700 1 --2- 172.21.15.7:0/1985024026 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35e40a2c20 0x7f35e40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:41.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.999+0000 7f35eeffd700 1 -- 172.21.15.7:0/1985024026 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f35f40cf3c0 con 0x7f35f8101390 2024-06-10T01:02:41.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:41.999+0000 7f35fd06b700 1 --2- 172.21.15.7:0/1985024026 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35e40a2c20 0x7f35e40a50e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f35e800b5c0 tx=0x7f35e8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:42.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:42.003+0000 7f35eeffd700 1 -- 172.21.15.7:0/1985024026 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f35f409bf60 con 0x7f35f8101390 2024-06-10T01:02:42.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:42.209+0000 7f35ffad0700 1 -- 172.21.15.7:0/1985024026 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f35f806ab70 con 0x7f35f8101390 2024-06-10T01:02:42.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:42.211+0000 7f35eeffd700 1 -- 172.21.15.7:0/1985024026 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24493 (secure 0 0 0) 0x7f35f409b860 con 0x7f35f8101390 2024-06-10T01:02:42.209 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:02:42.209 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":97,"fsid":"bded58f8-26c3-11ef-bc9b-c7b262605968","created":"2024-06-10T00:53:31.966085+0000","modified":"2024-06-10T01:02:37.362590+0000","last_up_change":"2024-06-10T01:00:45.382473+0000","last_in_change":"2024-06-10T01:00:25.608513+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:56:59.964055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T01:01:42.563611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T01:01:46.175918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T01:01:55.695502+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f7c24c9f-b7ab-4c04-b2b4-120df9751667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6803","nonce":404485726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6805","nonce":404485726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6809","nonce":404485726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6807","nonce":404485726}]},"public_addr":"172.21.15.7:6803/404485726","cluster_addr":"172.21.15.7:6805/404485726","heartbeat_back_addr":"172.21.15.7:6809/404485726","heartbeat_front_addr":"172.21.15.7:6807/404485726","state":["exists","up"]},{"osd":1,"uuid":"9bf70a6b-f31c-4077-b5af-229f17bc2845","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6801","nonce":1856804448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6803","nonce":1856804448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6807","nonce":1856804448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6805","nonce":1856804448}]},"public_addr":"172.21.15.78:6801/1856804448","cluster_addr":"172.21.15.78:6803/1856804448","heartbeat_back_addr":"172.21.15.78:6807/1856804448","heartbeat_front_addr":"172.21.15.78:6805/1856804448","state":["exists","up"]},{"osd":2,"uuid":"62923c97-caf9-4f7e-9c14-9c9918843c78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6801","nonce":918257455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6803","nonce":918257455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6807","nonce":918257455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6805","nonce":918257455}]},"public_addr":"172.21.15.175:6801/918257455","cluster_addr":"172.21.15.175:6803/918257455","heartbeat_back_addr":"172.21.15.175:6807/918257455","heartbeat_front_addr":"172.21.15.175:6805/918257455","state":["exists","up"]},{"osd":3,"uuid":"65242098-e748-48ab-8f0e-bb25b5c62742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6811","nonce":1797404966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6813","nonce":1797404966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6817","nonce":1797404966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6815","nonce":1797404966}]},"public_addr":"172.21.15.7:6811/1797404966","cluster_addr":"172.21.15.7:6813/1797404966","heartbeat_back_addr":"172.21.15.7:6817/1797404966","heartbeat_front_addr":"172.21.15.7:6815/1797404966","state":["exists","up"]},{"osd":4,"uuid":"a7990305-de9e-400e-a432-d7854f73f339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6809","nonce":3767608357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6811","nonce":3767608357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6815","nonce":3767608357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6813","nonce":3767608357}]},"public_addr":"172.21.15.78:6809/3767608357","cluster_addr":"172.21.15.78:6811/3767608357","heartbeat_back_addr":"172.21.15.78:6815/3767608357","heartbeat_front_addr":"172.21.15.78:6813/3767608357","state":["exists","up"]},{"osd":5,"uuid":"bad9a7b3-da5a-48d0-905d-e6a3190317fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6809","nonce":2157664731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6811","nonce":2157664731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6815","nonce":2157664731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6813","nonce":2157664731}]},"public_addr":"172.21.15.175:6809/2157664731","cluster_addr":"172.21.15.175:6811/2157664731","heartbeat_back_addr":"172.21.15.175:6815/2157664731","heartbeat_front_addr":"172.21.15.175:6813/2157664731","state":["exists","up"]},{"osd":6,"uuid":"6c687653-157b-4b96-867b-a2eaa7d8ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6819","nonce":838731756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6821","nonce":838731756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6825","nonce":838731756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6823","nonce":838731756}]},"public_addr":"172.21.15.7:6819/838731756","cluster_addr":"172.21.15.7:6821/838731756","heartbeat_back_addr":"172.21.15.7:6825/838731756","heartbeat_front_addr":"172.21.15.7:6823/838731756","state":["exists","up"]},{"osd":7,"uuid":"db40ddf1-9fd9-4859-b8c4-75f79582ab91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6817","nonce":3511921773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6819","nonce":3511921773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6823","nonce":3511921773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6821","nonce":3511921773}]},"public_addr":"172.21.15.78:6817/3511921773","cluster_addr":"172.21.15.78:6819/3511921773","heartbeat_back_addr":"172.21.15.78:6823/3511921773","heartbeat_front_addr":"172.21.15.78:6821/3511921773","state":["exists","up"]},{"osd":8,"uuid":"0f01cbd9-b193-4c70-a33c-4984167bf422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6817","nonce":1546325211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6819","nonce":1546325211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6823","nonce":1546325211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6821","nonce":1546325211}]},"public_addr":"172.21.15.175:6817/1546325211","cluster_addr":"172.21.15.175:6819/1546325211","heartbeat_back_addr":"172.21.15.175:6823/1546325211","heartbeat_front_addr":"172.21.15.175:6821/1546325211","state":["exists","up"]},{"osd":9,"uuid":"e5480998-f7fb-49cd-9aed-52405c88d830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6827","nonce":1844550813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6829","nonce":1844550813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6833","nonce":1844550813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6831","nonce":1844550813}]},"public_addr":"172.21.15.7:6827/1844550813","cluster_addr":"172.21.15.7:6829/1844550813","heartbeat_back_addr":"172.21.15.7:6833/1844550813","heartbeat_front_addr":"172.21.15.7:6831/1844550813","state":["exists","up"]},{"osd":10,"uuid":"caa82f33-a71c-4907-bf3b-30a7fdfeb71d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6825","nonce":2290824785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6827","nonce":2290824785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6831","nonce":2290824785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6829","nonce":2290824785}]},"public_addr":"172.21.15.78:6825/2290824785","cluster_addr":"172.21.15.78:6827/2290824785","heartbeat_back_addr":"172.21.15.78:6831/2290824785","heartbeat_front_addr":"172.21.15.78:6829/2290824785","state":["exists","up"]},{"osd":11,"uuid":"608cda77-77da-4da2-a7e2-76a773be27c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6825","nonce":1959979550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6827","nonce":1959979550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6831","nonce":1959979550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6829","nonce":1959979550}]},"public_addr":"172.21.15.175:6825/1959979550","cluster_addr":"172.21.15.175:6827/1959979550","heartbeat_back_addr":"172.21.15.175:6831/1959979550","heartbeat_front_addr":"172.21.15.175:6829/1959979550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:08.850203+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:32.820161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:56.633656+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:21.759372+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:46.735874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:10.636541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:36.990099+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:01.767098+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:26.882776+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:52.757595+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:18.383938+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:43.544330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1833397470":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/322543993":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/212510583":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6801/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6800/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/2679436843":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6801/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/1210355277":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6800/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/3610017406":"2024-06-11T00:54:08.102405+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T01:02:42.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:42.213+0000 7f35ffad0700 1 -- 172.21.15.7:0/1985024026 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35e40a2c20 msgr2=0x7f35e40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:42.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:42.213+0000 7f35ffad0700 1 --2- 172.21.15.7:0/1985024026 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35e40a2c20 0x7f35e40a50e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f35e800b5c0 tx=0x7f35e8005fd0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:42.213+0000 7f35ffad0700 1 -- 172.21.15.7:0/1985024026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35f8101390 msgr2=0x7f35f819d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:42.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:42.213+0000 7f35ffad0700 1 --2- 172.21.15.7:0/1985024026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35f8101390 0x7f35f819d780 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f35f400d950 tx=0x7f35f400dd10 comp rx=0 tx=0).stop 2024-06-10T01:02:42.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:42.214+0000 7f35ffad0700 1 -- 172.21.15.7:0/1985024026 shutdown_connections 2024-06-10T01:02:42.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:42.214+0000 7f35ffad0700 1 --2- 172.21.15.7:0/1985024026 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35e40a2c20 0x7f35e40a50e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:42.214+0000 7f35ffad0700 1 --2- 172.21.15.7:0/1985024026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35f8107400 0x7f35f8197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:42.214+0000 7f35ffad0700 1 --2- 172.21.15.7:0/1985024026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35f8101390 0x7f35f819d780 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:42.214+0000 7f35ffad0700 1 --2- 172.21.15.7:0/1985024026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f8101d50 0x7f35f819dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:42.214+0000 7f35ffad0700 1 -- 172.21.15.7:0/1985024026 >> 172.21.15.7:0/1985024026 conn(0x7f35f8078340 msgr2=0x7f35f80ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:42.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:42.214+0000 7f35ffad0700 1 -- 172.21.15.7:0/1985024026 shutdown_connections 2024-06-10T01:02:42.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:42.214+0000 7f35ffad0700 1 -- 172.21.15.7:0/1985024026 wait complete. 2024-06-10T01:02:42.459 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:42 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1985024026' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:02:42.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:42 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1985024026' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:02:42.832 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:42 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1985024026' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:02:42.909 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T01:02:43.171 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:02:43.603 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:43 smithi007 ceph-mon[32398]: pgmap v330: 129 pgs: 1 activating, 128 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T01:02:43.604 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:43 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:02:43.635 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:43 smithi175 ceph-mon[36636]: pgmap v330: 129 pgs: 1 activating, 128 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T01:02:43.635 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:43 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:02:43.774 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:43 smithi078 ceph-mon[36683]: pgmap v330: 129 pgs: 1 activating, 128 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T01:02:43.774 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:43 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:02:44.656 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:44 smithi007 ceph-mon[32398]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T01:02:44.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:44 smithi175 ceph-mon[36636]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T01:02:44.832 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:44 smithi078 ceph-mon[36683]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T01:02:44.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.896+0000 7ff69204f700 1 -- 172.21.15.7:0/1190352478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff68c102740 msgr2=0x7ff68c102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:44.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.896+0000 7ff69204f700 1 --2- 172.21.15.7:0/1190352478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff68c102740 0x7ff68c102b20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff67c0099c0 tx=0x7ff67c009cd0 comp rx=0 tx=0).stop 2024-06-10T01:02:44.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.896+0000 7ff69204f700 1 -- 172.21.15.7:0/1190352478 shutdown_connections 2024-06-10T01:02:44.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.896+0000 7ff69204f700 1 --2- 172.21.15.7:0/1190352478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff68c107840 0x7ff68c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:44.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.896+0000 7ff69204f700 1 --2- 172.21.15.7:0/1190352478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff68c102740 0x7ff68c102b20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:44.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.896+0000 7ff69204f700 1 --2- 172.21.15.7:0/1190352478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff68c1030f0 0x7ff68c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:44.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.896+0000 7ff69204f700 1 -- 172.21.15.7:0/1190352478 >> 172.21.15.7:0/1190352478 conn(0x7ff68c0795f0 msgr2=0x7ff68c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:44.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.896+0000 7ff69204f700 1 -- 172.21.15.7:0/1190352478 shutdown_connections 2024-06-10T01:02:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.896+0000 7ff69204f700 1 -- 172.21.15.7:0/1190352478 wait complete. 2024-06-10T01:02:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.897+0000 7ff69204f700 1 Processor -- start 2024-06-10T01:02:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.897+0000 7ff69204f700 1 -- start start 2024-06-10T01:02:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.897+0000 7ff69204f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff68c102740 0x7ff68c19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.897+0000 7ff69204f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff68c1030f0 0x7ff68c19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.897+0000 7ff69204f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff68c107840 0x7ff68c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.897+0000 7ff69204f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c1127d0 con 0x7ff68c1030f0 2024-06-10T01:02:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.897+0000 7ff69204f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c112650 con 0x7ff68c102740 2024-06-10T01:02:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.897+0000 7ff69204f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c112950 con 0x7ff68c107840 2024-06-10T01:02:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.897+0000 7ff68affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff68c1030f0 0x7ff68c19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.897+0000 7ff68b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff68c102740 0x7ff68c19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.897+0000 7ff68b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff68c102740 0x7ff68c19e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:58916/0 (socket says 172.21.15.7:58916) 2024-06-10T01:02:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.897+0000 7ff68b7fe700 1 -- 172.21.15.7:0/1856281223 learned_addr learned my addr 172.21.15.7:0/1856281223 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:02:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.897+0000 7ff68bfff700 1 --2- 172.21.15.7:0/1856281223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff68c107840 0x7ff68c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:44.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.897+0000 7ff68affd700 1 -- 172.21.15.7:0/1856281223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff68c107840 msgr2=0x7ff68c198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:44.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.898+0000 7ff68affd700 1 --2- 172.21.15.7:0/1856281223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff68c107840 0x7ff68c198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:44.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.898+0000 7ff68affd700 1 -- 172.21.15.7:0/1856281223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff68c102740 msgr2=0x7ff68c19e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:44.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.898+0000 7ff68affd700 1 --2- 172.21.15.7:0/1856281223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff68c102740 0x7ff68c19e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:44.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.898+0000 7ff68affd700 1 -- 172.21.15.7:0/1856281223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff68c1991b0 con 0x7ff68c1030f0 2024-06-10T01:02:44.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.898+0000 7ff68bfff700 1 --2- 172.21.15.7:0/1856281223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff68c107840 0x7ff68c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:44.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.898+0000 7ff68b7fe700 1 --2- 172.21.15.7:0/1856281223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff68c102740 0x7ff68c19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:44.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.898+0000 7ff68affd700 1 --2- 172.21.15.7:0/1856281223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff68c1030f0 0x7ff68c19edd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff68000d750 tx=0x7ff68000da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:44.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.898+0000 7ff688ff9700 1 -- 172.21.15.7:0/1856281223 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff680014030 con 0x7ff68c1030f0 2024-06-10T01:02:44.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.898+0000 7ff69204f700 1 -- 172.21.15.7:0/1856281223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff68c1994a0 con 0x7ff68c1030f0 2024-06-10T01:02:44.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.898+0000 7ff688ff9700 1 -- 172.21.15.7:0/1856281223 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff680004500 con 0x7ff68c1030f0 2024-06-10T01:02:44.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.898+0000 7ff688ff9700 1 -- 172.21.15.7:0/1856281223 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff680005400 con 0x7ff68c1030f0 2024-06-10T01:02:44.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.898+0000 7ff69204f700 1 -- 172.21.15.7:0/1856281223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff68c1a5760 con 0x7ff68c1030f0 2024-06-10T01:02:44.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.899+0000 7ff69204f700 1 -- 172.21.15.7:0/1856281223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff68c10aa80 con 0x7ff68c1030f0 2024-06-10T01:02:44.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.900+0000 7ff688ff9700 1 -- 172.21.15.7:0/1856281223 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7ff680004750 con 0x7ff68c1030f0 2024-06-10T01:02:44.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.901+0000 7ff688ff9700 1 --2- 172.21.15.7:0/1856281223 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff6740a2c20 0x7ff6740a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:44.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.901+0000 7ff68b7fe700 1 --2- 172.21.15.7:0/1856281223 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff6740a2c20 0x7ff6740a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:44.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.901+0000 7ff688ff9700 1 -- 172.21.15.7:0/1856281223 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7ff6800ced20 con 0x7ff68c1030f0 2024-06-10T01:02:44.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.902+0000 7ff68b7fe700 1 --2- 172.21.15.7:0/1856281223 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff6740a2c20 0x7ff6740a50e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff67c000c00 tx=0x7ff67c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:44.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:44.906+0000 7ff688ff9700 1 -- 172.21.15.7:0/1856281223 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff68009b8c0 con 0x7ff68c1030f0 2024-06-10T01:02:45.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:45.118+0000 7ff69204f700 1 -- 172.21.15.7:0/1856281223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff68c10a460 con 0x7ff68c1030f0 2024-06-10T01:02:45.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:45.119+0000 7ff688ff9700 1 -- 172.21.15.7:0/1856281223 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24493 (secure 0 0 0) 0x7ff68009b1c0 con 0x7ff68c1030f0 2024-06-10T01:02:45.118 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:02:45.118 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":97,"fsid":"bded58f8-26c3-11ef-bc9b-c7b262605968","created":"2024-06-10T00:53:31.966085+0000","modified":"2024-06-10T01:02:37.362590+0000","last_up_change":"2024-06-10T01:00:45.382473+0000","last_in_change":"2024-06-10T01:00:25.608513+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:56:59.964055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T01:01:42.563611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T01:01:46.175918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T01:01:55.695502+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f7c24c9f-b7ab-4c04-b2b4-120df9751667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6803","nonce":404485726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6805","nonce":404485726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6809","nonce":404485726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6807","nonce":404485726}]},"public_addr":"172.21.15.7:6803/404485726","cluster_addr":"172.21.15.7:6805/404485726","heartbeat_back_addr":"172.21.15.7:6809/404485726","heartbeat_front_addr":"172.21.15.7:6807/404485726","state":["exists","up"]},{"osd":1,"uuid":"9bf70a6b-f31c-4077-b5af-229f17bc2845","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6801","nonce":1856804448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6803","nonce":1856804448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6807","nonce":1856804448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6805","nonce":1856804448}]},"public_addr":"172.21.15.78:6801/1856804448","cluster_addr":"172.21.15.78:6803/1856804448","heartbeat_back_addr":"172.21.15.78:6807/1856804448","heartbeat_front_addr":"172.21.15.78:6805/1856804448","state":["exists","up"]},{"osd":2,"uuid":"62923c97-caf9-4f7e-9c14-9c9918843c78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6801","nonce":918257455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6803","nonce":918257455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6807","nonce":918257455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6805","nonce":918257455}]},"public_addr":"172.21.15.175:6801/918257455","cluster_addr":"172.21.15.175:6803/918257455","heartbeat_back_addr":"172.21.15.175:6807/918257455","heartbeat_front_addr":"172.21.15.175:6805/918257455","state":["exists","up"]},{"osd":3,"uuid":"65242098-e748-48ab-8f0e-bb25b5c62742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6811","nonce":1797404966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6813","nonce":1797404966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6817","nonce":1797404966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6815","nonce":1797404966}]},"public_addr":"172.21.15.7:6811/1797404966","cluster_addr":"172.21.15.7:6813/1797404966","heartbeat_back_addr":"172.21.15.7:6817/1797404966","heartbeat_front_addr":"172.21.15.7:6815/1797404966","state":["exists","up"]},{"osd":4,"uuid":"a7990305-de9e-400e-a432-d7854f73f339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6809","nonce":3767608357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6811","nonce":3767608357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6815","nonce":3767608357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6813","nonce":3767608357}]},"public_addr":"172.21.15.78:6809/3767608357","cluster_addr":"172.21.15.78:6811/3767608357","heartbeat_back_addr":"172.21.15.78:6815/3767608357","heartbeat_front_addr":"172.21.15.78:6813/3767608357","state":["exists","up"]},{"osd":5,"uuid":"bad9a7b3-da5a-48d0-905d-e6a3190317fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6809","nonce":2157664731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6811","nonce":2157664731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6815","nonce":2157664731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6813","nonce":2157664731}]},"public_addr":"172.21.15.175:6809/2157664731","cluster_addr":"172.21.15.175:6811/2157664731","heartbeat_back_addr":"172.21.15.175:6815/2157664731","heartbeat_front_addr":"172.21.15.175:6813/2157664731","state":["exists","up"]},{"osd":6,"uuid":"6c687653-157b-4b96-867b-a2eaa7d8ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6819","nonce":838731756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6821","nonce":838731756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6825","nonce":838731756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6823","nonce":838731756}]},"public_addr":"172.21.15.7:6819/838731756","cluster_addr":"172.21.15.7:6821/838731756","heartbeat_back_addr":"172.21.15.7:6825/838731756","heartbeat_front_addr":"172.21.15.7:6823/838731756","state":["exists","up"]},{"osd":7,"uuid":"db40ddf1-9fd9-4859-b8c4-75f79582ab91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6817","nonce":3511921773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6819","nonce":3511921773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6823","nonce":3511921773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6821","nonce":3511921773}]},"public_addr":"172.21.15.78:6817/3511921773","cluster_addr":"172.21.15.78:6819/3511921773","heartbeat_back_addr":"172.21.15.78:6823/3511921773","heartbeat_front_addr":"172.21.15.78:6821/3511921773","state":["exists","up"]},{"osd":8,"uuid":"0f01cbd9-b193-4c70-a33c-4984167bf422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6817","nonce":1546325211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6819","nonce":1546325211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6823","nonce":1546325211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6821","nonce":1546325211}]},"public_addr":"172.21.15.175:6817/1546325211","cluster_addr":"172.21.15.175:6819/1546325211","heartbeat_back_addr":"172.21.15.175:6823/1546325211","heartbeat_front_addr":"172.21.15.175:6821/1546325211","state":["exists","up"]},{"osd":9,"uuid":"e5480998-f7fb-49cd-9aed-52405c88d830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6827","nonce":1844550813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6829","nonce":1844550813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6833","nonce":1844550813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6831","nonce":1844550813}]},"public_addr":"172.21.15.7:6827/1844550813","cluster_addr":"172.21.15.7:6829/1844550813","heartbeat_back_addr":"172.21.15.7:6833/1844550813","heartbeat_front_addr":"172.21.15.7:6831/1844550813","state":["exists","up"]},{"osd":10,"uuid":"caa82f33-a71c-4907-bf3b-30a7fdfeb71d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6825","nonce":2290824785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6827","nonce":2290824785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6831","nonce":2290824785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6829","nonce":2290824785}]},"public_addr":"172.21.15.78:6825/2290824785","cluster_addr":"172.21.15.78:6827/2290824785","heartbeat_back_addr":"172.21.15.78:6831/2290824785","heartbeat_front_addr":"172.21.15.78:6829/2290824785","state":["exists","up"]},{"osd":11,"uuid":"608cda77-77da-4da2-a7e2-76a773be27c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6825","nonce":1959979550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6827","nonce":1959979550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6831","nonce":1959979550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6829","nonce":1959979550}]},"public_addr":"172.21.15.175:6825/1959979550","cluster_addr":"172.21.15.175:6827/1959979550","heartbeat_back_addr":"172.21.15.175:6831/1959979550","heartbeat_front_addr":"172.21.15.175:6829/1959979550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:08.850203+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:32.820161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:56.633656+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:21.759372+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:46.735874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:10.636541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:36.990099+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:01.767098+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:26.882776+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:52.757595+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:18.383938+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:43.544330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1833397470":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/322543993":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/212510583":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6801/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6800/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/2679436843":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6801/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/1210355277":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6800/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/3610017406":"2024-06-11T00:54:08.102405+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T01:02:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:45.122+0000 7ff69204f700 1 -- 172.21.15.7:0/1856281223 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff6740a2c20 msgr2=0x7ff6740a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:45.122+0000 7ff69204f700 1 --2- 172.21.15.7:0/1856281223 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff6740a2c20 0x7ff6740a50e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff67c000c00 tx=0x7ff67c005fb0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:45.122+0000 7ff69204f700 1 -- 172.21.15.7:0/1856281223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff68c1030f0 msgr2=0x7ff68c19edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:45.122+0000 7ff69204f700 1 --2- 172.21.15.7:0/1856281223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff68c1030f0 0x7ff68c19edd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff68000d750 tx=0x7ff68000da60 comp rx=0 tx=0).stop 2024-06-10T01:02:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:45.122+0000 7ff69204f700 1 -- 172.21.15.7:0/1856281223 shutdown_connections 2024-06-10T01:02:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:45.122+0000 7ff69204f700 1 --2- 172.21.15.7:0/1856281223 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff6740a2c20 0x7ff6740a50e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:45.122+0000 7ff69204f700 1 --2- 172.21.15.7:0/1856281223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff68c107840 0x7ff68c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:45.122+0000 7ff69204f700 1 --2- 172.21.15.7:0/1856281223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff68c102740 0x7ff68c19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:45.122+0000 7ff69204f700 1 --2- 172.21.15.7:0/1856281223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff68c1030f0 0x7ff68c19edd0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:45.122+0000 7ff69204f700 1 -- 172.21.15.7:0/1856281223 >> 172.21.15.7:0/1856281223 conn(0x7ff68c0795f0 msgr2=0x7ff68c100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:45.122+0000 7ff69204f700 1 -- 172.21.15.7:0/1856281223 shutdown_connections 2024-06-10T01:02:45.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:45.122+0000 7ff69204f700 1 -- 172.21.15.7:0/1856281223 wait complete. 2024-06-10T01:02:45.766 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:45 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:02:45.766 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:45 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:02:45.766 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:45 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:02:45.766 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:45 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:02:45.766 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:45 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1856281223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:02:45.832 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:45 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:02:45.832 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:45 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:02:45.832 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:45 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:02:45.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:45 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:02:45.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:45 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1856281223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:02:45.956 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:45 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:02:45.956 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:45 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:02:45.956 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:45 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:02:45.956 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:45 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:02:45.956 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:45 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1856281223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:02:46.286 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-06-10T01:02:46.588 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:02:46.713 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:46 smithi007 ceph-mon[32398]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-10T01:02:46.832 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:46 smithi078 ceph-mon[36683]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-10T01:02:46.955 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:46 smithi175 ceph-mon[36636]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-10T01:02:48.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.416+0000 7f17dafed700 1 -- 172.21.15.7:0/2255200412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17d4108540 msgr2=0x7f17d4108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.416+0000 7f17dafed700 1 --2- 172.21.15.7:0/2255200412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17d4108540 0x7f17d4108920 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f17c80099c0 tx=0x7f17c8009cd0 comp rx=0 tx=0).stop 2024-06-10T01:02:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.416+0000 7f17dafed700 1 -- 172.21.15.7:0/2255200412 shutdown_connections 2024-06-10T01:02:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.416+0000 7f17dafed700 1 --2- 172.21.15.7:0/2255200412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17d4102e50 0x7f17d410f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.416+0000 7f17dafed700 1 --2- 172.21.15.7:0/2255200412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17d4108540 0x7f17d4108920 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.416+0000 7f17dafed700 1 --2- 172.21.15.7:0/2255200412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17d4102490 0x7f17d4102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.416+0000 7f17dafed700 1 -- 172.21.15.7:0/2255200412 >> 172.21.15.7:0/2255200412 conn(0x7f17d40fdf40 msgr2=0x7f17d4100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.416+0000 7f17dafed700 1 -- 172.21.15.7:0/2255200412 shutdown_connections 2024-06-10T01:02:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.416+0000 7f17dafed700 1 -- 172.21.15.7:0/2255200412 wait complete. 2024-06-10T01:02:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.416+0000 7f17dafed700 1 Processor -- start 2024-06-10T01:02:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.416+0000 7f17dafed700 1 -- start start 2024-06-10T01:02:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.417+0000 7f17dafed700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17d4102490 0x7f17d4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.417+0000 7f17dafed700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17d4102e50 0x7f17d4074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:48.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.417+0000 7f17dafed700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17d4108540 0x7f17d4075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:48.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.417+0000 7f17dafed700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f17d4112770 con 0x7f17d4102e50 2024-06-10T01:02:48.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.417+0000 7f17dafed700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f17d41125f0 con 0x7f17d4102490 2024-06-10T01:02:48.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.417+0000 7f17dafed700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f17d41128f0 con 0x7f17d4108540 2024-06-10T01:02:48.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.417+0000 7f17d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17d4102e50 0x7f17d4074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:48.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.417+0000 7f17d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17d4102e50 0x7f17d4074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53736/0 (socket says 172.21.15.7:53736) 2024-06-10T01:02:48.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.417+0000 7f17d3fff700 1 -- 172.21.15.7:0/3033151085 learned_addr learned my addr 172.21.15.7:0/3033151085 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:02:48.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.417+0000 7f17d8d89700 1 --2- 172.21.15.7:0/3033151085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17d4102490 0x7f17d4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:48.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.417+0000 7f17d958a700 1 --2- 172.21.15.7:0/3033151085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17d4108540 0x7f17d4075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:48.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.417+0000 7f17d3fff700 1 -- 172.21.15.7:0/3033151085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17d4108540 msgr2=0x7f17d4075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:48.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.417+0000 7f17d3fff700 1 --2- 172.21.15.7:0/3033151085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17d4108540 0x7f17d4075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:48.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.417+0000 7f17d3fff700 1 -- 172.21.15.7:0/3033151085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17d4102490 msgr2=0x7f17d4074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:48.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.417+0000 7f17d3fff700 1 --2- 172.21.15.7:0/3033151085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17d4102490 0x7f17d4074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:48.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.417+0000 7f17d3fff700 1 -- 172.21.15.7:0/3033151085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d41a7090 con 0x7f17d4102e50 2024-06-10T01:02:48.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.417+0000 7f17d3fff700 1 --2- 172.21.15.7:0/3033151085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17d4102e50 0x7f17d4074cd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f17c000c920 tx=0x7f17c000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:48.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.418+0000 7f17d1ffb700 1 -- 172.21.15.7:0/3033151085 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0004a30 con 0x7f17d4102e50 2024-06-10T01:02:48.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.418+0000 7f17dafed700 1 -- 172.21.15.7:0/3033151085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d41a7380 con 0x7f17d4102e50 2024-06-10T01:02:48.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.418+0000 7f17d1ffb700 1 -- 172.21.15.7:0/3033151085 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17c0005500 con 0x7f17d4102e50 2024-06-10T01:02:48.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.418+0000 7f17d1ffb700 1 -- 172.21.15.7:0/3033151085 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0007d00 con 0x7f17d4102e50 2024-06-10T01:02:48.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.418+0000 7f17dafed700 1 -- 172.21.15.7:0/3033151085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d41a78c0 con 0x7f17d4102e50 2024-06-10T01:02:48.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.419+0000 7f17dafed700 1 -- 172.21.15.7:0/3033151085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17d410a9d0 con 0x7f17d4102e50 2024-06-10T01:02:48.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.420+0000 7f17d1ffb700 1 -- 172.21.15.7:0/3033151085 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f17c0005000 con 0x7f17d4102e50 2024-06-10T01:02:48.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.421+0000 7f17d1ffb700 1 --2- 172.21.15.7:0/3033151085 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f17c40a2c20 0x7f17c40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:48.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.421+0000 7f17d8d89700 1 --2- 172.21.15.7:0/3033151085 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f17c40a2c20 0x7f17c40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:48.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.421+0000 7f17d1ffb700 1 -- 172.21.15.7:0/3033151085 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f17c00ceec0 con 0x7f17d4102e50 2024-06-10T01:02:48.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.422+0000 7f17d8d89700 1 --2- 172.21.15.7:0/3033151085 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f17c40a2c20 0x7f17c40a50e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f17c8000c00 tx=0x7f17c8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:48.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.426+0000 7f17d1ffb700 1 -- 172.21.15.7:0/3033151085 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f17c0014030 con 0x7f17d4102e50 2024-06-10T01:02:48.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:48.699+0000 7f17dafed700 1 -- 172.21.15.7:0/3033151085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f17d4075cc0 con 0x7f17d4102e50 2024-06-10T01:02:49.266 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:48 smithi007 ceph-mon[32398]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-10T01:02:49.266 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:48 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:02:49.266 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:48 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:02:49.266 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:48 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:02:49.266 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:48 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:02:49.266 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:48 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:02:49.266 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:48 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3033151085' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-10T01:02:49.332 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:48 smithi078 ceph-mon[36683]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-10T01:02:49.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:48 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:02:49.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:48 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:02:49.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:48 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:02:49.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:48 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:02:49.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:48 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:02:49.333 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:48 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3033151085' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-10T01:02:49.456 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:48 smithi175 ceph-mon[36636]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-10T01:02:49.456 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:48 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:02:49.456 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:48 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:02:49.456 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:48 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:02:49.456 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:48 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:02:49.456 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:48 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:02:49.456 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:48 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3033151085' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-10T01:02:49.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:49.671+0000 7f17d1ffb700 1 -- 172.21.15.7:0/3033151085 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f17c009bac0 con 0x7f17d4102e50 2024-06-10T01:02:49.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:49.676+0000 7f17dafed700 1 -- 172.21.15.7:0/3033151085 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f17c40a2c20 msgr2=0x7f17c40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:49.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:49.676+0000 7f17dafed700 1 --2- 172.21.15.7:0/3033151085 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f17c40a2c20 0x7f17c40a50e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f17c8000c00 tx=0x7f17c8005fb0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:49.676+0000 7f17dafed700 1 -- 172.21.15.7:0/3033151085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17d4102e50 msgr2=0x7f17d4074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:49.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:49.676+0000 7f17dafed700 1 --2- 172.21.15.7:0/3033151085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17d4102e50 0x7f17d4074cd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f17c000c920 tx=0x7f17c000cc30 comp rx=0 tx=0).stop 2024-06-10T01:02:49.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:49.676+0000 7f17dafed700 1 -- 172.21.15.7:0/3033151085 shutdown_connections 2024-06-10T01:02:49.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:49.676+0000 7f17dafed700 1 --2- 172.21.15.7:0/3033151085 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f17c40a2c20 0x7f17c40a50e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:49.676+0000 7f17dafed700 1 --2- 172.21.15.7:0/3033151085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17d4108540 0x7f17d4075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:49.676+0000 7f17dafed700 1 --2- 172.21.15.7:0/3033151085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17d4102490 0x7f17d4074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:49.676+0000 7f17dafed700 1 --2- 172.21.15.7:0/3033151085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17d4102e50 0x7f17d4074cd0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:49.676+0000 7f17dafed700 1 -- 172.21.15.7:0/3033151085 >> 172.21.15.7:0/3033151085 conn(0x7f17d40fdf40 msgr2=0x7f17d40ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:49.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:49.676+0000 7f17dafed700 1 -- 172.21.15.7:0/3033151085 shutdown_connections 2024-06-10T01:02:49.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:49.676+0000 7f17dafed700 1 -- 172.21.15.7:0/3033151085 wait complete. 2024-06-10T01:02:50.301 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-06-10T01:02:50.301 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-06-10T01:02:50.558 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:02:50.928 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:50 smithi007 ceph-mon[32398]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-10T01:02:50.929 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:50 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3033151085' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-10T01:02:50.929 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:50 smithi007 ceph-mon[32398]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T01:02:50.956 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:50 smithi175 ceph-mon[36636]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-10T01:02:50.956 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:50 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3033151085' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-10T01:02:50.956 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:50 smithi175 ceph-mon[36636]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T01:02:51.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:50 smithi078 ceph-mon[36683]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-10T01:02:51.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:50 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3033151085' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-10T01:02:51.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:50 smithi078 ceph-mon[36683]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T01:02:52.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.097+0000 7f8e3719d700 1 -- 172.21.15.7:0/2540454987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e3010ef40 msgr2=0x7f8e30111330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:52.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.097+0000 7f8e3719d700 1 --2- 172.21.15.7:0/2540454987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e3010ef40 0x7f8e30111330 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800b0a0 tx=0x7f8e2800b3b0 comp rx=0 tx=0).stop 2024-06-10T01:02:52.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.097+0000 7f8e3719d700 1 -- 172.21.15.7:0/2540454987 shutdown_connections 2024-06-10T01:02:52.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.097+0000 7f8e3719d700 1 --2- 172.21.15.7:0/2540454987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e3010ef40 0x7f8e30111330 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:52.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.097+0000 7f8e3719d700 1 --2- 172.21.15.7:0/2540454987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e3006a1a0 0x7f8e3006a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:52.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.097+0000 7f8e3719d700 1 --2- 172.21.15.7:0/2540454987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e3006aac0 0x7f8e3010ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:52.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.097+0000 7f8e3719d700 1 -- 172.21.15.7:0/2540454987 >> 172.21.15.7:0/2540454987 conn(0x7f8e30078340 msgr2=0x7f8e30078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:52.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.097+0000 7f8e3719d700 1 -- 172.21.15.7:0/2540454987 shutdown_connections 2024-06-10T01:02:52.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.097+0000 7f8e3719d700 1 -- 172.21.15.7:0/2540454987 wait complete. 2024-06-10T01:02:52.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.098+0000 7f8e3719d700 1 Processor -- start 2024-06-10T01:02:52.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.098+0000 7f8e3719d700 1 -- start start 2024-06-10T01:02:52.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.098+0000 7f8e3719d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e3006a1a0 0x7f8e3019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:52.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.098+0000 7f8e3719d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e3006aac0 0x7f8e3019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:52.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.098+0000 7f8e3719d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e3010ef40 0x7f8e3019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:52.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.098+0000 7f8e3719d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8e301149a0 con 0x7f8e3006a1a0 2024-06-10T01:02:52.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.098+0000 7f8e3719d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8e30114820 con 0x7f8e3010ef40 2024-06-10T01:02:52.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.098+0000 7f8e3719d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8e30114b20 con 0x7f8e3006aac0 2024-06-10T01:02:52.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.098+0000 7f8e34f39700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e3006a1a0 0x7f8e3019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:52.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.098+0000 7f8e34f39700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e3006a1a0 0x7f8e3019ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36112/0 (socket says 172.21.15.7:36112) 2024-06-10T01:02:52.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.098+0000 7f8e34f39700 1 -- 172.21.15.7:0/3521946456 learned_addr learned my addr 172.21.15.7:0/3521946456 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:02:52.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.099+0000 7f8e27fff700 1 --2- 172.21.15.7:0/3521946456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e3006aac0 0x7f8e3019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:52.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.099+0000 7f8e3573a700 1 --2- 172.21.15.7:0/3521946456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e3010ef40 0x7f8e3019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:52.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.099+0000 7f8e34f39700 1 -- 172.21.15.7:0/3521946456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e3006aac0 msgr2=0x7f8e3019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:52.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.099+0000 7f8e34f39700 1 --2- 172.21.15.7:0/3521946456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e3006aac0 0x7f8e3019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:52.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.099+0000 7f8e34f39700 1 -- 172.21.15.7:0/3521946456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e3010ef40 msgr2=0x7f8e3019f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:52.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.099+0000 7f8e34f39700 1 --2- 172.21.15.7:0/3521946456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e3010ef40 0x7f8e3019f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:52.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.099+0000 7f8e34f39700 1 -- 172.21.15.7:0/3521946456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e3019fe80 con 0x7f8e3006a1a0 2024-06-10T01:02:52.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.099+0000 7f8e34f39700 1 --2- 172.21.15.7:0/3521946456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e3006a1a0 0x7f8e3019ae70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c00d950 tx=0x7f8e2c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:52.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.099+0000 7f8e25ffb700 1 -- 172.21.15.7:0/3521946456 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c016070 con 0x7f8e3006a1a0 2024-06-10T01:02:52.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.099+0000 7f8e3719d700 1 -- 172.21.15.7:0/3521946456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e301a0170 con 0x7f8e3006a1a0 2024-06-10T01:02:52.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.099+0000 7f8e25ffb700 1 -- 172.21.15.7:0/3521946456 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e2c00fd70 con 0x7f8e3006a1a0 2024-06-10T01:02:52.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.099+0000 7f8e25ffb700 1 -- 172.21.15.7:0/3521946456 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c00e610 con 0x7f8e3006a1a0 2024-06-10T01:02:52.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.099+0000 7f8e3719d700 1 -- 172.21.15.7:0/3521946456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e301a79f0 con 0x7f8e3006a1a0 2024-06-10T01:02:52.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.100+0000 7f8e3719d700 1 -- 172.21.15.7:0/3521946456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e30069180 con 0x7f8e3006a1a0 2024-06-10T01:02:52.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.102+0000 7f8e25ffb700 1 -- 172.21.15.7:0/3521946456 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f8e2c019070 con 0x7f8e3006a1a0 2024-06-10T01:02:52.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.102+0000 7f8e25ffb700 1 --2- 172.21.15.7:0/3521946456 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8e180a2c20 0x7f8e180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:52.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.102+0000 7f8e27fff700 1 --2- 172.21.15.7:0/3521946456 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8e180a2c20 0x7f8e180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:52.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.102+0000 7f8e25ffb700 1 -- 172.21.15.7:0/3521946456 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f8e2c0cee50 con 0x7f8e3006a1a0 2024-06-10T01:02:52.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.103+0000 7f8e27fff700 1 --2- 172.21.15.7:0/3521946456 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8e180a2c20 0x7f8e180a50e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8e30100590 tx=0x7f8e2000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:52.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.107+0000 7f8e25ffb700 1 -- 172.21.15.7:0/3521946456 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8e2c09ba50 con 0x7f8e3006a1a0 2024-06-10T01:02:52.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.360+0000 7f8e3719d700 1 -- 172.21.15.7:0/3521946456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f8e3019c1d0 con 0x7f8e3006a1a0 2024-06-10T01:02:52.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.680+0000 7f8e25ffb700 1 -- 172.21.15.7:0/3521946456 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f8e2c09b350 con 0x7f8e3006a1a0 2024-06-10T01:02:52.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.683+0000 7f8e3719d700 1 -- 172.21.15.7:0/3521946456 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8e180a2c20 msgr2=0x7f8e180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:52.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.683+0000 7f8e3719d700 1 --2- 172.21.15.7:0/3521946456 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8e180a2c20 0x7f8e180a50e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8e30100590 tx=0x7f8e2000a300 comp rx=0 tx=0).stop 2024-06-10T01:02:52.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.683+0000 7f8e3719d700 1 -- 172.21.15.7:0/3521946456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e3006a1a0 msgr2=0x7f8e3019ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:52.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.683+0000 7f8e3719d700 1 --2- 172.21.15.7:0/3521946456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e3006a1a0 0x7f8e3019ae70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c00d950 tx=0x7f8e2c00dc60 comp rx=0 tx=0).stop 2024-06-10T01:02:52.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.683+0000 7f8e3719d700 1 -- 172.21.15.7:0/3521946456 shutdown_connections 2024-06-10T01:02:52.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.683+0000 7f8e3719d700 1 --2- 172.21.15.7:0/3521946456 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8e180a2c20 0x7f8e180a50e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:52.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.683+0000 7f8e3719d700 1 --2- 172.21.15.7:0/3521946456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e3010ef40 0x7f8e3019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:52.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.683+0000 7f8e3719d700 1 --2- 172.21.15.7:0/3521946456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e3006a1a0 0x7f8e3019ae70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:52.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.683+0000 7f8e3719d700 1 --2- 172.21.15.7:0/3521946456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e3006aac0 0x7f8e3019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:52.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.683+0000 7f8e3719d700 1 -- 172.21.15.7:0/3521946456 >> 172.21.15.7:0/3521946456 conn(0x7f8e30078340 msgr2=0x7f8e30110390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:52.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.684+0000 7f8e3719d700 1 -- 172.21.15.7:0/3521946456 shutdown_connections 2024-06-10T01:02:52.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:52.684+0000 7f8e3719d700 1 -- 172.21.15.7:0/3521946456 wait complete. 2024-06-10T01:02:52.955 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:52 smithi175 ceph-mon[36636]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:52.956 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:52 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3521946456' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-10T01:02:53.016 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:52 smithi007 ceph-mon[32398]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:53.016 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:52 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3521946456' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-10T01:02:53.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:52 smithi078 ceph-mon[36683]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:53.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:52 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3521946456' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-10T01:02:53.411 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-06-10T01:02:53.666 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:02:53.931 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:53 smithi007 ceph-mon[32398]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-10T01:02:53.931 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:53 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3521946456' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-06-10T01:02:53.931 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:53 smithi007 ceph-mon[32398]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-10T01:02:53.931 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:53 smithi007 ceph-mon[32398]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T01:02:53.931 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:53 smithi007 ceph-mon[32398]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-06-10T01:02:53.931 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:53 smithi007 ceph-mon[32398]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-06-10T01:02:53.956 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:53 smithi175 ceph-mon[36636]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-10T01:02:53.956 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:53 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3521946456' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-06-10T01:02:53.956 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:53 smithi175 ceph-mon[36636]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-10T01:02:53.956 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:53 smithi175 ceph-mon[36636]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T01:02:53.956 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:53 smithi175 ceph-mon[36636]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-06-10T01:02:53.956 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:53 smithi175 ceph-mon[36636]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-06-10T01:02:54.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:53 smithi078 ceph-mon[36683]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-10T01:02:54.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:53 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3521946456' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-06-10T01:02:54.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:53 smithi078 ceph-mon[36683]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-10T01:02:54.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:53 smithi078 ceph-mon[36683]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T01:02:54.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:53 smithi078 ceph-mon[36683]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-06-10T01:02:54.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:53 smithi078 ceph-mon[36683]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-06-10T01:02:54.859 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:54 smithi007 ceph-mon[32398]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:54.860 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:54 smithi007 ceph-mon[32398]: mds.? [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] up:active 2024-06-10T01:02:54.860 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:54 smithi007 ceph-mon[32398]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-10T01:02:54.860 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:54 smithi007 ceph-mon[32398]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-10T01:02:54.860 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:54 smithi007 ceph-mon[32398]: Cluster is now healthy 2024-06-10T01:02:54.860 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:54 smithi007 ceph-mon[32398]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-06-10T01:02:54.860 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:54 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:creating} 9 up:standby 2024-06-10T01:02:54.860 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:54 smithi007 ceph-mon[32398]: daemon mds.b is now active in filesystem cephfs as rank 2 2024-06-10T01:02:54.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:54 smithi175 ceph-mon[36636]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:54.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:54 smithi175 ceph-mon[36636]: mds.? [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] up:active 2024-06-10T01:02:54.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:54 smithi175 ceph-mon[36636]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-10T01:02:54.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:54 smithi175 ceph-mon[36636]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-10T01:02:54.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:54 smithi175 ceph-mon[36636]: Cluster is now healthy 2024-06-10T01:02:54.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:54 smithi175 ceph-mon[36636]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-06-10T01:02:54.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:54 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:creating} 9 up:standby 2024-06-10T01:02:54.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:54 smithi175 ceph-mon[36636]: daemon mds.b is now active in filesystem cephfs as rank 2 2024-06-10T01:02:55.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:54 smithi078 ceph-mon[36683]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:02:55.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:54 smithi078 ceph-mon[36683]: mds.? [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] up:active 2024-06-10T01:02:55.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:54 smithi078 ceph-mon[36683]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-10T01:02:55.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:54 smithi078 ceph-mon[36683]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-10T01:02:55.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:54 smithi078 ceph-mon[36683]: Cluster is now healthy 2024-06-10T01:02:55.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:54 smithi078 ceph-mon[36683]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-06-10T01:02:55.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:54 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:creating} 9 up:standby 2024-06-10T01:02:55.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:54 smithi078 ceph-mon[36683]: daemon mds.b is now active in filesystem cephfs as rank 2 2024-06-10T01:02:55.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.375+0000 7f35a8ee7700 1 -- 172.21.15.7:0/455016673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35a4107840 msgr2=0x7f35a410f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:55.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.375+0000 7f35a8ee7700 1 --2- 172.21.15.7:0/455016673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35a4107840 0x7f35a410f3c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f359c00b0d0 tx=0x7f359c00b3e0 comp rx=0 tx=0).stop 2024-06-10T01:02:55.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.376+0000 7f35a8ee7700 1 -- 172.21.15.7:0/455016673 shutdown_connections 2024-06-10T01:02:55.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.376+0000 7f35a8ee7700 1 --2- 172.21.15.7:0/455016673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35a4107840 0x7f35a410f3c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:55.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.376+0000 7f35a8ee7700 1 --2- 172.21.15.7:0/455016673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35a4102740 0x7f35a4102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:55.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.376+0000 7f35a8ee7700 1 --2- 172.21.15.7:0/455016673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35a41030f0 0x7f35a4107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:55.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.376+0000 7f35a8ee7700 1 -- 172.21.15.7:0/455016673 >> 172.21.15.7:0/455016673 conn(0x7f35a40795f0 msgr2=0x7f35a4079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:55.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.376+0000 7f35a8ee7700 1 -- 172.21.15.7:0/455016673 shutdown_connections 2024-06-10T01:02:55.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.376+0000 7f35a8ee7700 1 -- 172.21.15.7:0/455016673 wait complete. 2024-06-10T01:02:55.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.376+0000 7f35a8ee7700 1 Processor -- start 2024-06-10T01:02:55.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.376+0000 7f35a8ee7700 1 -- start start 2024-06-10T01:02:55.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.377+0000 7f35a8ee7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35a4102740 0x7f35a419e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:55.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.377+0000 7f35a8ee7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35a41030f0 0x7f35a419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:55.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.377+0000 7f35a8ee7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35a4107840 0x7f35a4198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:55.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.377+0000 7f35a8ee7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f35a4112850 con 0x7f35a41030f0 2024-06-10T01:02:55.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.377+0000 7f35a8ee7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f35a41126d0 con 0x7f35a4102740 2024-06-10T01:02:55.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.377+0000 7f35a8ee7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f35a41129d0 con 0x7f35a4107840 2024-06-10T01:02:55.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.377+0000 7f35a1d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35a41030f0 0x7f35a419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:55.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.377+0000 7f35a1d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35a41030f0 0x7f35a419ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36128/0 (socket says 172.21.15.7:36128) 2024-06-10T01:02:55.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.377+0000 7f35a1d9b700 1 -- 172.21.15.7:0/1014035869 learned_addr learned my addr 172.21.15.7:0/1014035869 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:02:55.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.377+0000 7f35a259c700 1 --2- 172.21.15.7:0/1014035869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35a4102740 0x7f35a419e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:55.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.377+0000 7f35a2d9d700 1 --2- 172.21.15.7:0/1014035869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35a4107840 0x7f35a4198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:55.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.377+0000 7f35a1d9b700 1 -- 172.21.15.7:0/1014035869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35a4107840 msgr2=0x7f35a4198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:55.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.377+0000 7f35a1d9b700 1 --2- 172.21.15.7:0/1014035869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35a4107840 0x7f35a4198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:55.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.377+0000 7f35a1d9b700 1 -- 172.21.15.7:0/1014035869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35a4102740 msgr2=0x7f35a419e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:55.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.377+0000 7f35a1d9b700 1 --2- 172.21.15.7:0/1014035869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35a4102740 0x7f35a419e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:55.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.377+0000 7f35a1d9b700 1 -- 172.21.15.7:0/1014035869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a41991f0 con 0x7f35a41030f0 2024-06-10T01:02:55.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.377+0000 7f35a259c700 1 --2- 172.21.15.7:0/1014035869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35a4102740 0x7f35a419e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:02:55.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.377+0000 7f35a1d9b700 1 --2- 172.21.15.7:0/1014035869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35a41030f0 0x7f35a419ee10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f359800b530 tx=0x7f359800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:55.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.377+0000 7f35937fe700 1 -- 172.21.15.7:0/1014035869 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3598015020 con 0x7f35a41030f0 2024-06-10T01:02:55.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.378+0000 7f35a8ee7700 1 -- 172.21.15.7:0/1014035869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35a41994e0 con 0x7f35a41030f0 2024-06-10T01:02:55.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.378+0000 7f35a8ee7700 1 -- 172.21.15.7:0/1014035869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35a41a5820 con 0x7f35a41030f0 2024-06-10T01:02:55.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.378+0000 7f35937fe700 1 -- 172.21.15.7:0/1014035869 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3598011d10 con 0x7f35a41030f0 2024-06-10T01:02:55.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.378+0000 7f35937fe700 1 -- 172.21.15.7:0/1014035869 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359801ae20 con 0x7f35a41030f0 2024-06-10T01:02:55.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.379+0000 7f35a8ee7700 1 -- 172.21.15.7:0/1014035869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3584005330 con 0x7f35a41030f0 2024-06-10T01:02:55.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.380+0000 7f35937fe700 1 -- 172.21.15.7:0/1014035869 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f3598018020 con 0x7f35a41030f0 2024-06-10T01:02:55.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.381+0000 7f35937fe700 1 --2- 172.21.15.7:0/1014035869 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f358c0a2c20 0x7f358c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:55.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.381+0000 7f35937fe700 1 -- 172.21.15.7:0/1014035869 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f35980ceb00 con 0x7f35a41030f0 2024-06-10T01:02:55.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.381+0000 7f35a259c700 1 --2- 172.21.15.7:0/1014035869 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f358c0a2c20 0x7f358c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:55.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.381+0000 7f35a259c700 1 --2- 172.21.15.7:0/1014035869 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f358c0a2c20 0x7f358c0a50e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f35a4199bd0 tx=0x7f3594008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:55.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.385+0000 7f35937fe700 1 -- 172.21.15.7:0/1014035869 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f359801f020 con 0x7f35a41030f0 2024-06-10T01:02:55.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.639+0000 7f35a8ee7700 1 -- 172.21.15.7:0/1014035869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f3584005620 con 0x7f35a41030f0 2024-06-10T01:02:55.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.704+0000 7f35937fe700 1 -- 172.21.15.7:0/1014035869 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7f359809b700 con 0x7f35a41030f0 2024-06-10T01:02:55.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.707+0000 7f35a8ee7700 1 -- 172.21.15.7:0/1014035869 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f358c0a2c20 msgr2=0x7f358c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:55.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.707+0000 7f35a8ee7700 1 --2- 172.21.15.7:0/1014035869 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f358c0a2c20 0x7f358c0a50e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f35a4199bd0 tx=0x7f3594008040 comp rx=0 tx=0).stop 2024-06-10T01:02:55.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.707+0000 7f35a8ee7700 1 -- 172.21.15.7:0/1014035869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35a41030f0 msgr2=0x7f35a419ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:55.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.707+0000 7f35a8ee7700 1 --2- 172.21.15.7:0/1014035869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35a41030f0 0x7f35a419ee10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f359800b530 tx=0x7f359800b8f0 comp rx=0 tx=0).stop 2024-06-10T01:02:55.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.707+0000 7f35a8ee7700 1 -- 172.21.15.7:0/1014035869 shutdown_connections 2024-06-10T01:02:55.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.707+0000 7f35a8ee7700 1 --2- 172.21.15.7:0/1014035869 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f358c0a2c20 0x7f358c0a50e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:55.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.707+0000 7f35a8ee7700 1 --2- 172.21.15.7:0/1014035869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35a4107840 0x7f35a4198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:55.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.707+0000 7f35a8ee7700 1 --2- 172.21.15.7:0/1014035869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35a4102740 0x7f35a419e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:55.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.707+0000 7f35a8ee7700 1 --2- 172.21.15.7:0/1014035869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35a41030f0 0x7f35a419ee10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:55.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.707+0000 7f35a8ee7700 1 -- 172.21.15.7:0/1014035869 >> 172.21.15.7:0/1014035869 conn(0x7f35a40795f0 msgr2=0x7f35a4101c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:55.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.707+0000 7f35a8ee7700 1 -- 172.21.15.7:0/1014035869 shutdown_connections 2024-06-10T01:02:55.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:55.707+0000 7f35a8ee7700 1 -- 172.21.15.7:0/1014035869 wait complete. 2024-06-10T01:02:55.956 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:55 smithi175 ceph-mon[36636]: mds.? [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] up:active 2024-06-10T01:02:55.956 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:55 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 9 up:standby 2024-06-10T01:02:55.956 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:55 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1014035869' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-10T01:02:56.016 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:55 smithi007 ceph-mon[32398]: mds.? [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] up:active 2024-06-10T01:02:56.016 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:55 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 9 up:standby 2024-06-10T01:02:56.016 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:55 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1014035869' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-10T01:02:56.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:55 smithi078 ceph-mon[36683]: mds.? [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] up:active 2024-06-10T01:02:56.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:55 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 9 up:standby 2024-06-10T01:02:56.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:55 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1014035869' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-10T01:02:56.366 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-06-10T01:02:56.625 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:02:56.909 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:56 smithi007 ceph-mon[32398]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-06-10T01:02:56.909 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:56 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1014035869' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-10T01:02:56.909 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:56 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 9 up:standby 2024-06-10T01:02:56.909 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:56 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:02:56.955 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:56 smithi175 ceph-mon[36636]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-06-10T01:02:56.956 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:56 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1014035869' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-10T01:02:56.956 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:56 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 9 up:standby 2024-06-10T01:02:56.956 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:56 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:02:57.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:56 smithi078 ceph-mon[36683]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-06-10T01:02:57.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:56 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1014035869' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-10T01:02:57.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:56 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 9 up:standby 2024-06-10T01:02:57.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:56 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:02:58.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.259+0000 7fa850d3b700 1 -- 172.21.15.7:0/1358049920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa84c101d60 msgr2=0x7fa84c10e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:58.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.259+0000 7fa850d3b700 1 --2- 172.21.15.7:0/1358049920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa84c101d60 0x7fa84c10e240 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa84400b0a0 tx=0x7fa84400b3b0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.259+0000 7fa850d3b700 1 -- 172.21.15.7:0/1358049920 shutdown_connections 2024-06-10T01:02:58.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.259+0000 7fa850d3b700 1 --2- 172.21.15.7:0/1358049920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa84c101d60 0x7fa84c10e240 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.259+0000 7fa850d3b700 1 --2- 172.21.15.7:0/1358049920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa84c107400 0x7fa84c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.259+0000 7fa850d3b700 1 --2- 172.21.15.7:0/1358049920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa84c1013c0 0x7fa84c101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.259+0000 7fa850d3b700 1 -- 172.21.15.7:0/1358049920 >> 172.21.15.7:0/1358049920 conn(0x7fa84c078330 msgr2=0x7fa84c078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:58.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.259+0000 7fa850d3b700 1 -- 172.21.15.7:0/1358049920 shutdown_connections 2024-06-10T01:02:58.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.259+0000 7fa850d3b700 1 -- 172.21.15.7:0/1358049920 wait complete. 2024-06-10T01:02:58.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.260+0000 7fa850d3b700 1 Processor -- start 2024-06-10T01:02:58.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.260+0000 7fa850d3b700 1 -- start start 2024-06-10T01:02:58.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.261+0000 7fa850d3b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa84c1013c0 0x7fa84c199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:58.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.261+0000 7fa850d3b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa84c101d60 0x7fa84c19a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:58.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.261+0000 7fa850d3b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa84c107400 0x7fa84c1a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:58.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.261+0000 7fa850d3b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c111740 con 0x7fa84c101d60 2024-06-10T01:02:58.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.261+0000 7fa850d3b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c1115c0 con 0x7fa84c107400 2024-06-10T01:02:58.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.261+0000 7fa850d3b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c1118c0 con 0x7fa84c1013c0 2024-06-10T01:02:58.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.261+0000 7fa841d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa84c101d60 0x7fa84c19a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:58.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.261+0000 7fa841d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa84c101d60 0x7fa84c19a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36148/0 (socket says 172.21.15.7:36148) 2024-06-10T01:02:58.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.261+0000 7fa841d9b700 1 -- 172.21.15.7:0/895171600 learned_addr learned my addr 172.21.15.7:0/895171600 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:02:58.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.261+0000 7fa84a59c700 1 --2- 172.21.15.7:0/895171600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa84c1013c0 0x7fa84c199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:58.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.261+0000 7fa84ad9d700 1 --2- 172.21.15.7:0/895171600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa84c107400 0x7fa84c1a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:58.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.261+0000 7fa841d9b700 1 -- 172.21.15.7:0/895171600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa84c1013c0 msgr2=0x7fa84c199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:58.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.261+0000 7fa841d9b700 1 --2- 172.21.15.7:0/895171600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa84c1013c0 0x7fa84c199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.261+0000 7fa841d9b700 1 -- 172.21.15.7:0/895171600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa84c107400 msgr2=0x7fa84c1a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:58.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.261+0000 7fa841d9b700 1 --2- 172.21.15.7:0/895171600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa84c107400 0x7fa84c1a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.261+0000 7fa841d9b700 1 -- 172.21.15.7:0/895171600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84c1a5160 con 0x7fa84c101d60 2024-06-10T01:02:58.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.262+0000 7fa841d9b700 1 --2- 172.21.15.7:0/895171600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa84c101d60 0x7fa84c19a0e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa834009aa0 tx=0x7fa834009db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:58.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.262+0000 7fa843fff700 1 -- 172.21.15.7:0/895171600 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa834011860 con 0x7fa84c101d60 2024-06-10T01:02:58.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.262+0000 7fa850d3b700 1 -- 172.21.15.7:0/895171600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa84c1979f0 con 0x7fa84c101d60 2024-06-10T01:02:58.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.262+0000 7fa843fff700 1 -- 172.21.15.7:0/895171600 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa834012430 con 0x7fa84c101d60 2024-06-10T01:02:58.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.262+0000 7fa843fff700 1 -- 172.21.15.7:0/895171600 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83401b660 con 0x7fa84c101d60 2024-06-10T01:02:58.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.262+0000 7fa850d3b700 1 -- 172.21.15.7:0/895171600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa84c197f80 con 0x7fa84c101d60 2024-06-10T01:02:58.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.263+0000 7fa850d3b700 1 -- 172.21.15.7:0/895171600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa84c102ab0 con 0x7fa84c101d60 2024-06-10T01:02:58.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.264+0000 7fa843fff700 1 -- 172.21.15.7:0/895171600 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fa8340125e0 con 0x7fa84c101d60 2024-06-10T01:02:58.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.265+0000 7fa843fff700 1 --2- 172.21.15.7:0/895171600 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa8380a2c20 0x7fa8380a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:58.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.265+0000 7fa84a59c700 1 --2- 172.21.15.7:0/895171600 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa8380a2c20 0x7fa8380a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:58.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.265+0000 7fa843fff700 1 -- 172.21.15.7:0/895171600 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7fa8340d2f90 con 0x7fa84c101d60 2024-06-10T01:02:58.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.265+0000 7fa84a59c700 1 --2- 172.21.15.7:0/895171600 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa8380a2c20 0x7fa8380a50e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa83c009a10 tx=0x7fa83c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:58.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.268+0000 7fa843fff700 1 -- 172.21.15.7:0/895171600 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa834017030 con 0x7fa84c101d60 2024-06-10T01:02:58.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.522+0000 7fa850d3b700 1 -- 172.21.15.7:0/895171600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fa84c06b1b0 con 0x7fa84c101d60 2024-06-10T01:02:58.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.708+0000 7fa843fff700 1 -- 172.21.15.7:0/895171600 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7fa83409fb90 con 0x7fa84c101d60 2024-06-10T01:02:58.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.711+0000 7fa850d3b700 1 -- 172.21.15.7:0/895171600 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa8380a2c20 msgr2=0x7fa8380a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:58.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.711+0000 7fa850d3b700 1 --2- 172.21.15.7:0/895171600 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa8380a2c20 0x7fa8380a50e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa83c009a10 tx=0x7fa83c009190 comp rx=0 tx=0).stop 2024-06-10T01:02:58.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.711+0000 7fa850d3b700 1 -- 172.21.15.7:0/895171600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa84c101d60 msgr2=0x7fa84c19a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:58.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.711+0000 7fa850d3b700 1 --2- 172.21.15.7:0/895171600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa84c101d60 0x7fa84c19a0e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa834009aa0 tx=0x7fa834009db0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.711+0000 7fa850d3b700 1 -- 172.21.15.7:0/895171600 shutdown_connections 2024-06-10T01:02:58.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.711+0000 7fa850d3b700 1 --2- 172.21.15.7:0/895171600 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa8380a2c20 0x7fa8380a50e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.711+0000 7fa850d3b700 1 --2- 172.21.15.7:0/895171600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa84c107400 0x7fa84c1a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.711+0000 7fa850d3b700 1 --2- 172.21.15.7:0/895171600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa84c1013c0 0x7fa84c199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.711+0000 7fa850d3b700 1 --2- 172.21.15.7:0/895171600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa84c101d60 0x7fa84c19a0e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.711+0000 7fa850d3b700 1 -- 172.21.15.7:0/895171600 >> 172.21.15.7:0/895171600 conn(0x7fa84c078330 msgr2=0x7fa84c100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:58.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.711+0000 7fa850d3b700 1 -- 172.21.15.7:0/895171600 shutdown_connections 2024-06-10T01:02:58.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:02:58.711+0000 7fa850d3b700 1 -- 172.21.15.7:0/895171600 wait complete. 2024-06-10T01:02:58.953 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:58 smithi007 ceph-mon[32398]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 1.5 KiB/s wr, 5 op/s 2024-06-10T01:02:58.953 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:58 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/895171600' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-10T01:02:59.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:58 smithi078 ceph-mon[36683]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 1.5 KiB/s wr, 5 op/s 2024-06-10T01:02:59.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:58 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/895171600' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-10T01:02:59.206 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:58 smithi175 ceph-mon[36636]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 1.5 KiB/s wr, 5 op/s 2024-06-10T01:02:59.206 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:58 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/895171600' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-10T01:02:59.263 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-06-10T01:02:59.263 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-06-10T01:02:59.516 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:02:59.957 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:59 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/895171600' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-10T01:02:59.958 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:02:59 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:03:00.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:59 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/895171600' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-10T01:03:00.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:02:59 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:03:00.166 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:59 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/895171600' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-10T01:03:00.166 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:02:59 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:03:00.921 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:00 smithi007 ceph-mon[32398]: pgmap v339: 129 pgs: 129 active+clean; 584 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-10T01:03:01.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:00 smithi078 ceph-mon[36683]: pgmap v339: 129 pgs: 129 active+clean; 584 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-10T01:03:01.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.165+0000 7fbb6bf38700 1 -- 172.21.15.7:0/1262943155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb641027a0 msgr2=0x7fbb64102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:01.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.165+0000 7fbb6bf38700 1 --2- 172.21.15.7:0/1262943155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb641027a0 0x7fbb64102b80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbb600099c0 tx=0x7fbb60009cd0 comp rx=0 tx=0).stop 2024-06-10T01:03:01.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.165+0000 7fbb6bf38700 1 -- 172.21.15.7:0/1262943155 shutdown_connections 2024-06-10T01:03:01.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.165+0000 7fbb6bf38700 1 --2- 172.21.15.7:0/1262943155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb64107870 0x7fbb6410f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:01.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.165+0000 7fbb6bf38700 1 --2- 172.21.15.7:0/1262943155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb641027a0 0x7fbb64102b80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:01.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.165+0000 7fbb6bf38700 1 --2- 172.21.15.7:0/1262943155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb64103150 0x7fbb64107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:01.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.165+0000 7fbb6bf38700 1 -- 172.21.15.7:0/1262943155 >> 172.21.15.7:0/1262943155 conn(0x7fbb64079570 msgr2=0x7fbb64079980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:01.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.165+0000 7fbb6bf38700 1 -- 172.21.15.7:0/1262943155 shutdown_connections 2024-06-10T01:03:01.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.165+0000 7fbb6bf38700 1 -- 172.21.15.7:0/1262943155 wait complete. 2024-06-10T01:03:01.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.166+0000 7fbb6bf38700 1 Processor -- start 2024-06-10T01:03:01.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.166+0000 7fbb6bf38700 1 -- start start 2024-06-10T01:03:01.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.166+0000 7fbb6bf38700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb641027a0 0x7fbb6419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:01.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.166+0000 7fbb6bf38700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb64103150 0x7fbb6419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:01.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.166+0000 7fbb6bf38700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb64107870 0x7fbb64198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:01.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.166+0000 7fbb6bf38700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbb64112800 con 0x7fbb64107870 2024-06-10T01:03:01.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.166+0000 7fbb6bf38700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbb64112680 con 0x7fbb64103150 2024-06-10T01:03:01.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.166+0000 7fbb6bf38700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbb64112980 con 0x7fbb641027a0 2024-06-10T01:03:01.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.166+0000 7fbb6a4d5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb64107870 0x7fbb64198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:01.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.166+0000 7fbb6a4d5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb64107870 0x7fbb64198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36166/0 (socket says 172.21.15.7:36166) 2024-06-10T01:03:01.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.166+0000 7fbb6a4d5700 1 -- 172.21.15.7:0/1556748700 learned_addr learned my addr 172.21.15.7:0/1556748700 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:03:01.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.167+0000 7fbb69cd4700 1 --2- 172.21.15.7:0/1556748700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb641027a0 0x7fbb6419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:01.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.167+0000 7fbb694d3700 1 --2- 172.21.15.7:0/1556748700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb64103150 0x7fbb6419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:01.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.167+0000 7fbb6a4d5700 1 -- 172.21.15.7:0/1556748700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb641027a0 msgr2=0x7fbb6419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:01.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.167+0000 7fbb6a4d5700 1 --2- 172.21.15.7:0/1556748700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb641027a0 0x7fbb6419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:01.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.167+0000 7fbb6a4d5700 1 -- 172.21.15.7:0/1556748700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb64103150 msgr2=0x7fbb6419eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:01.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.167+0000 7fbb6a4d5700 1 --2- 172.21.15.7:0/1556748700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb64103150 0x7fbb6419eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:01.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.167+0000 7fbb6a4d5700 1 -- 172.21.15.7:0/1556748700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb64199180 con 0x7fbb64107870 2024-06-10T01:03:01.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.167+0000 7fbb69cd4700 1 --2- 172.21.15.7:0/1556748700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb641027a0 0x7fbb6419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:03:01.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.167+0000 7fbb6a4d5700 1 --2- 172.21.15.7:0/1556748700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb64107870 0x7fbb64198a20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c007ee0 tx=0x7fbb5c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:01.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.167+0000 7fbb5affd700 1 -- 172.21.15.7:0/1556748700 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c00dbd0 con 0x7fbb64107870 2024-06-10T01:03:01.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.167+0000 7fbb5affd700 1 -- 172.21.15.7:0/1556748700 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb5c00dd70 con 0x7fbb64107870 2024-06-10T01:03:01.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.167+0000 7fbb6bf38700 1 -- 172.21.15.7:0/1556748700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb64199470 con 0x7fbb64107870 2024-06-10T01:03:01.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.167+0000 7fbb5affd700 1 -- 172.21.15.7:0/1556748700 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c0133f0 con 0x7fbb64107870 2024-06-10T01:03:01.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.167+0000 7fbb6bf38700 1 -- 172.21.15.7:0/1556748700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb641a5730 con 0x7fbb64107870 2024-06-10T01:03:01.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.168+0000 7fbb6bf38700 1 -- 172.21.15.7:0/1556748700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb64069180 con 0x7fbb64107870 2024-06-10T01:03:01.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.169+0000 7fbb5affd700 1 -- 172.21.15.7:0/1556748700 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fbb5c01a020 con 0x7fbb64107870 2024-06-10T01:03:01.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.170+0000 7fbb5affd700 1 --2- 172.21.15.7:0/1556748700 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fbb500a2c20 0x7fbb500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:01.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.170+0000 7fbb69cd4700 1 --2- 172.21.15.7:0/1556748700 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fbb500a2c20 0x7fbb500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:01.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.171+0000 7fbb5affd700 1 -- 172.21.15.7:0/1556748700 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7fbb5c0cf2a0 con 0x7fbb64107870 2024-06-10T01:03:01.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.171+0000 7fbb69cd4700 1 --2- 172.21.15.7:0/1556748700 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fbb500a2c20 0x7fbb500a50e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbb60000c00 tx=0x7fbb60005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:01.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.174+0000 7fbb5affd700 1 -- 172.21.15.7:0/1556748700 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbb5c010070 con 0x7fbb64107870 2024-06-10T01:03:01.206 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:00 smithi175 ceph-mon[36636]: pgmap v339: 129 pgs: 129 active+clean; 584 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-10T01:03:01.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.415+0000 7fbb6bf38700 1 -- 172.21.15.7:0/1556748700 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fbb64100c20 con 0x7fbb500a2c20 2024-06-10T01:03:01.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.437+0000 7fbb5affd700 1 -- 172.21.15.7:0/1556748700 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbb64100c20 con 0x7fbb500a2c20 2024-06-10T01:03:01.435 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 01:03:01 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T01:03:01.420+0000 7f3a15a81700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T01:03:01.435 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 01:03:01 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T01:03:01.420+0000 7f3a15a81700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T01:03:01.435 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 01:03:01 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T01:03:01.420+0000 7f3a15a81700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T01:03:01.435 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 01:03:01 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T01:03:01.420+0000 7f3a15a81700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T01:03:01.435 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 01:03:01 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x[32623]: 2024-06-10T01:03:01.420+0000 7f3a15a81700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T01:03:01.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.439+0000 7fbb6bf38700 1 -- 172.21.15.7:0/1556748700 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fbb500a2c20 msgr2=0x7fbb500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:01.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.439+0000 7fbb6bf38700 1 --2- 172.21.15.7:0/1556748700 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fbb500a2c20 0x7fbb500a50e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbb60000c00 tx=0x7fbb60005940 comp rx=0 tx=0).stop 2024-06-10T01:03:01.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.439+0000 7fbb6bf38700 1 -- 172.21.15.7:0/1556748700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb64107870 msgr2=0x7fbb64198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:01.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.439+0000 7fbb6bf38700 1 --2- 172.21.15.7:0/1556748700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb64107870 0x7fbb64198a20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c007ee0 tx=0x7fbb5c00d3b0 comp rx=0 tx=0).stop 2024-06-10T01:03:01.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.439+0000 7fbb6bf38700 1 -- 172.21.15.7:0/1556748700 shutdown_connections 2024-06-10T01:03:01.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.439+0000 7fbb6bf38700 1 --2- 172.21.15.7:0/1556748700 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fbb500a2c20 0x7fbb500a50e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:01.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.439+0000 7fbb6bf38700 1 --2- 172.21.15.7:0/1556748700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb64107870 0x7fbb64198a20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:01.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.439+0000 7fbb6bf38700 1 --2- 172.21.15.7:0/1556748700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb641027a0 0x7fbb6419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:01.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.439+0000 7fbb6bf38700 1 --2- 172.21.15.7:0/1556748700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb64103150 0x7fbb6419eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:01.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.439+0000 7fbb6bf38700 1 -- 172.21.15.7:0/1556748700 >> 172.21.15.7:0/1556748700 conn(0x7fbb64079570 msgr2=0x7fbb64100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:01.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.439+0000 7fbb6bf38700 1 -- 172.21.15.7:0/1556748700 shutdown_connections 2024-06-10T01:03:01.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:01.439+0000 7fbb6bf38700 1 -- 172.21.15.7:0/1556748700 wait complete. 2024-06-10T01:03:02.060 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-06-10T01:03:02.314 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:03:03.016 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:02 smithi007 ceph-mon[32398]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:03:03.016 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:02 smithi007 ceph-mon[32398]: pgmap v340: 129 pgs: 129 active+clean; 584 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-10T01:03:03.016 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:02 smithi007 ceph-mon[32398]: mgrmap e17: x(active, since 8m), standbys: y, z 2024-06-10T01:03:03.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:02 smithi078 ceph-mon[36683]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:03:03.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:02 smithi078 ceph-mon[36683]: pgmap v340: 129 pgs: 129 active+clean; 584 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-10T01:03:03.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:02 smithi078 ceph-mon[36683]: mgrmap e17: x(active, since 8m), standbys: y, z 2024-06-10T01:03:03.206 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:02 smithi175 ceph-mon[36636]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:03:03.206 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:02 smithi175 ceph-mon[36636]: pgmap v340: 129 pgs: 129 active+clean; 584 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-10T01:03:03.206 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:02 smithi175 ceph-mon[36636]: mgrmap e17: x(active, since 8m), standbys: y, z 2024-06-10T01:03:03.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.981+0000 7f15ce900700 1 -- 172.21.15.7:0/1514326203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15c8102770 msgr2=0x7f15c8102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:03.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.981+0000 7f15ce900700 1 --2- 172.21.15.7:0/1514326203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15c8102770 0x7f15c8102b50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f15b80099c0 tx=0x7f15b8009cd0 comp rx=0 tx=0).stop 2024-06-10T01:03:03.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.981+0000 7f15ce900700 1 -- 172.21.15.7:0/1514326203 shutdown_connections 2024-06-10T01:03:03.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.981+0000 7f15ce900700 1 --2- 172.21.15.7:0/1514326203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15c8107860 0x7f15c810f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:03.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.981+0000 7f15ce900700 1 --2- 172.21.15.7:0/1514326203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15c8102770 0x7f15c8102b50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:03.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.981+0000 7f15ce900700 1 --2- 172.21.15.7:0/1514326203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15c8103120 0x7f15c8107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:03.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.981+0000 7f15ce900700 1 -- 172.21.15.7:0/1514326203 >> 172.21.15.7:0/1514326203 conn(0x7f15c8079600 msgr2=0x7f15c8079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:03.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.981+0000 7f15ce900700 1 -- 172.21.15.7:0/1514326203 shutdown_connections 2024-06-10T01:03:03.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.981+0000 7f15ce900700 1 -- 172.21.15.7:0/1514326203 wait complete. 2024-06-10T01:03:03.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.982+0000 7f15ce900700 1 Processor -- start 2024-06-10T01:03:03.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.982+0000 7f15ce900700 1 -- start start 2024-06-10T01:03:03.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.982+0000 7f15ce900700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15c8102770 0x7f15c8073f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:03.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.982+0000 7f15ce900700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15c8103120 0x7f15c806bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:03.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.982+0000 7f15ce900700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15c8107860 0x7f15c806c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:03.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.982+0000 7f15ce900700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f15c81127c0 con 0x7f15c8102770 2024-06-10T01:03:03.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.982+0000 7f15ce900700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f15c8112640 con 0x7f15c8107860 2024-06-10T01:03:03.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.982+0000 7f15ce900700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f15c8112940 con 0x7f15c8103120 2024-06-10T01:03:03.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.982+0000 7f15c7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15c8102770 0x7f15c8073f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:03.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.982+0000 7f15c7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15c8102770 0x7f15c8073f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50210/0 (socket says 172.21.15.7:50210) 2024-06-10T01:03:03.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.983+0000 7f15c7fff700 1 -- 172.21.15.7:0/412566969 learned_addr learned my addr 172.21.15.7:0/412566969 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:03:03.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.983+0000 7f15c77fe700 1 --2- 172.21.15.7:0/412566969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15c8103120 0x7f15c806bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:03.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.983+0000 7f15cce9d700 1 --2- 172.21.15.7:0/412566969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15c8107860 0x7f15c806c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:03.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.983+0000 7f15c7fff700 1 -- 172.21.15.7:0/412566969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15c8103120 msgr2=0x7f15c806bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:03.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.983+0000 7f15c7fff700 1 --2- 172.21.15.7:0/412566969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15c8103120 0x7f15c806bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:03.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.983+0000 7f15c7fff700 1 -- 172.21.15.7:0/412566969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15c8107860 msgr2=0x7f15c806c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:03.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.983+0000 7f15c7fff700 1 --2- 172.21.15.7:0/412566969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15c8107860 0x7f15c806c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:03.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.983+0000 7f15c7fff700 1 -- 172.21.15.7:0/412566969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c806cd20 con 0x7f15c8102770 2024-06-10T01:03:03.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.983+0000 7f15c7fff700 1 --2- 172.21.15.7:0/412566969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15c8102770 0x7f15c8073f80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f15b8006010 tx=0x7f15b8009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:03.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.983+0000 7f15c57fa700 1 -- 172.21.15.7:0/412566969 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b8027070 con 0x7f15c8102770 2024-06-10T01:03:03.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.983+0000 7f15ce900700 1 -- 172.21.15.7:0/412566969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15c81ad980 con 0x7f15c8102770 2024-06-10T01:03:03.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.983+0000 7f15c57fa700 1 -- 172.21.15.7:0/412566969 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15b8004050 con 0x7f15c8102770 2024-06-10T01:03:03.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.983+0000 7f15c57fa700 1 -- 172.21.15.7:0/412566969 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b8022e10 con 0x7f15c8102770 2024-06-10T01:03:03.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.983+0000 7f15ce900700 1 -- 172.21.15.7:0/412566969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15c81addd0 con 0x7f15c8102770 2024-06-10T01:03:03.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.985+0000 7f15ce900700 1 -- 172.21.15.7:0/412566969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15c810aaa0 con 0x7f15c8102770 2024-06-10T01:03:03.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.986+0000 7f15c57fa700 1 -- 172.21.15.7:0/412566969 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f15b802a070 con 0x7f15c8102770 2024-06-10T01:03:03.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.986+0000 7f15c57fa700 1 --2- 172.21.15.7:0/412566969 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f15b00a2cb0 0x7f15b00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:03.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.986+0000 7f15c77fe700 1 --2- 172.21.15.7:0/412566969 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f15b00a2cb0 0x7f15b00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:03.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.986+0000 7f15c57fa700 1 -- 172.21.15.7:0/412566969 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f15b80d0210 con 0x7f15c8102770 2024-06-10T01:03:03.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.987+0000 7f15c77fe700 1 --2- 172.21.15.7:0/412566969 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f15b00a2cb0 0x7f15b00a5170 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f15c806d710 tx=0x7f15bc005eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:03.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:03.991+0000 7f15c57fa700 1 -- 172.21.15.7:0/412566969 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f15b8013070 con 0x7f15c8102770 2024-06-10T01:03:04.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:04.233+0000 7f15ce900700 1 -- 172.21.15.7:0/412566969 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f15c80628d0 con 0x7f15b00a2cb0 2024-06-10T01:03:04.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:04.262+0000 7f15c57fa700 1 -- 172.21.15.7:0/412566969 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f15c80628d0 con 0x7f15b00a2cb0 2024-06-10T01:03:04.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:04.264+0000 7f15ce900700 1 -- 172.21.15.7:0/412566969 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f15b00a2cb0 msgr2=0x7f15b00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:04.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:04.264+0000 7f15ce900700 1 --2- 172.21.15.7:0/412566969 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f15b00a2cb0 0x7f15b00a5170 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f15c806d710 tx=0x7f15bc005eb0 comp rx=0 tx=0).stop 2024-06-10T01:03:04.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:04.264+0000 7f15ce900700 1 -- 172.21.15.7:0/412566969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15c8102770 msgr2=0x7f15c8073f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:04.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:04.264+0000 7f15ce900700 1 --2- 172.21.15.7:0/412566969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15c8102770 0x7f15c8073f80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f15b8006010 tx=0x7f15b8009f80 comp rx=0 tx=0).stop 2024-06-10T01:03:04.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:04.264+0000 7f15ce900700 1 -- 172.21.15.7:0/412566969 shutdown_connections 2024-06-10T01:03:04.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:04.264+0000 7f15ce900700 1 --2- 172.21.15.7:0/412566969 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f15b00a2cb0 0x7f15b00a5170 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:04.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:04.264+0000 7f15ce900700 1 --2- 172.21.15.7:0/412566969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15c8107860 0x7f15c806c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:04.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:04.264+0000 7f15ce900700 1 --2- 172.21.15.7:0/412566969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15c8102770 0x7f15c8073f80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:04.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:04.264+0000 7f15ce900700 1 --2- 172.21.15.7:0/412566969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15c8103120 0x7f15c806bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:04.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:04.264+0000 7f15ce900700 1 -- 172.21.15.7:0/412566969 >> 172.21.15.7:0/412566969 conn(0x7f15c8079600 msgr2=0x7f15c8101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:04.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:04.264+0000 7f15ce900700 1 -- 172.21.15.7:0/412566969 shutdown_connections 2024-06-10T01:03:04.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:04.264+0000 7f15ce900700 1 -- 172.21.15.7:0/412566969 wait complete. 2024-06-10T01:03:04.831 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:04 smithi007 ceph-mon[32398]: pgmap v341: 129 pgs: 129 active+clean; 584 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-10T01:03:04.832 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:04 smithi007 ceph-mon[32398]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:03:04.861 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-06-10T01:03:05.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:04 smithi078 ceph-mon[36683]: pgmap v341: 129 pgs: 129 active+clean; 584 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-10T01:03:05.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:04 smithi078 ceph-mon[36683]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:03:05.112 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:03:05.205 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:04 smithi175 ceph-mon[36636]: pgmap v341: 129 pgs: 129 active+clean; 584 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-10T01:03:05.206 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:04 smithi175 ceph-mon[36636]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:03:06.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.636+0000 7efe07d0b700 1 -- 172.21.15.7:0/2325516059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe00107840 msgr2=0x7efe0010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:06.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.636+0000 7efe07d0b700 1 --2- 172.21.15.7:0/2325516059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe00107840 0x7efe0010f3c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7efdf800b0d0 tx=0x7efdf800b3e0 comp rx=0 tx=0).stop 2024-06-10T01:03:06.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.636+0000 7efe07d0b700 1 -- 172.21.15.7:0/2325516059 shutdown_connections 2024-06-10T01:03:06.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.636+0000 7efe07d0b700 1 --2- 172.21.15.7:0/2325516059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe00107840 0x7efe0010f3c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:06.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.636+0000 7efe07d0b700 1 --2- 172.21.15.7:0/2325516059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe00102740 0x7efe00102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:06.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.636+0000 7efe07d0b700 1 --2- 172.21.15.7:0/2325516059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe001030f0 0x7efe00107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:06.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.636+0000 7efe07d0b700 1 -- 172.21.15.7:0/2325516059 >> 172.21.15.7:0/2325516059 conn(0x7efe000795f0 msgr2=0x7efe00079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:06.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.637+0000 7efe07d0b700 1 -- 172.21.15.7:0/2325516059 shutdown_connections 2024-06-10T01:03:06.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.637+0000 7efe07d0b700 1 -- 172.21.15.7:0/2325516059 wait complete. 2024-06-10T01:03:06.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.637+0000 7efe07d0b700 1 Processor -- start 2024-06-10T01:03:06.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.637+0000 7efe07d0b700 1 -- start start 2024-06-10T01:03:06.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.637+0000 7efe07d0b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe00102740 0x7efe00074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:06.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.637+0000 7efe07d0b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe001030f0 0x7efe00074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:06.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.637+0000 7efe07d0b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe00107840 0x7efe00075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:06.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.637+0000 7efe07d0b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efe001127e0 con 0x7efe001030f0 2024-06-10T01:03:06.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.637+0000 7efe07d0b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efe00112660 con 0x7efe00107840 2024-06-10T01:03:06.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.637+0000 7efe07d0b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efe00112960 con 0x7efe00102740 2024-06-10T01:03:06.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.638+0000 7efe052a6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe001030f0 0x7efe00074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:06.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.638+0000 7efe052a6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe001030f0 0x7efe00074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50230/0 (socket says 172.21.15.7:50230) 2024-06-10T01:03:06.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.638+0000 7efe052a6700 1 -- 172.21.15.7:0/1249179355 learned_addr learned my addr 172.21.15.7:0/1249179355 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:03:06.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.638+0000 7efe05aa7700 1 --2- 172.21.15.7:0/1249179355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe00102740 0x7efe00074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:06.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.638+0000 7efe062a8700 1 --2- 172.21.15.7:0/1249179355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe00107840 0x7efe00075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:06.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.638+0000 7efe052a6700 1 -- 172.21.15.7:0/1249179355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe00102740 msgr2=0x7efe00074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:06.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.638+0000 7efe052a6700 1 --2- 172.21.15.7:0/1249179355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe00102740 0x7efe00074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:06.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.638+0000 7efe052a6700 1 -- 172.21.15.7:0/1249179355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe00107840 msgr2=0x7efe00075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:06.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.638+0000 7efe052a6700 1 --2- 172.21.15.7:0/1249179355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe00107840 0x7efe00075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:06.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.638+0000 7efe052a6700 1 -- 172.21.15.7:0/1249179355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe001a7040 con 0x7efe001030f0 2024-06-10T01:03:06.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.638+0000 7efe052a6700 1 --2- 172.21.15.7:0/1249179355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe001030f0 0x7efe00074cd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efdf000c9b0 tx=0x7efdf000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:06.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.638+0000 7efdf6ffd700 1 -- 172.21.15.7:0/1249179355 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf0007b00 con 0x7efe001030f0 2024-06-10T01:03:06.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.638+0000 7efdf6ffd700 1 -- 172.21.15.7:0/1249179355 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdf000ce50 con 0x7efe001030f0 2024-06-10T01:03:06.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.638+0000 7efe07d0b700 1 -- 172.21.15.7:0/1249179355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe001a7330 con 0x7efe001030f0 2024-06-10T01:03:06.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.638+0000 7efdf6ffd700 1 -- 172.21.15.7:0/1249179355 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf0018660 con 0x7efe001030f0 2024-06-10T01:03:06.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.639+0000 7efe07d0b700 1 -- 172.21.15.7:0/1249179355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe001a78c0 con 0x7efe001030f0 2024-06-10T01:03:06.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.639+0000 7efe07d0b700 1 -- 172.21.15.7:0/1249179355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe00069180 con 0x7efe001030f0 2024-06-10T01:03:06.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.641+0000 7efdf6ffd700 1 -- 172.21.15.7:0/1249179355 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7efdf000f450 con 0x7efe001030f0 2024-06-10T01:03:06.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.642+0000 7efdf6ffd700 1 --2- 172.21.15.7:0/1249179355 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efdec0a2d30 0x7efdec0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:06.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.642+0000 7efe05aa7700 1 --2- 172.21.15.7:0/1249179355 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efdec0a2d30 0x7efdec0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:06.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.642+0000 7efdf6ffd700 1 -- 172.21.15.7:0/1249179355 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7efdf00cf6c0 con 0x7efe001030f0 2024-06-10T01:03:06.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.642+0000 7efe05aa7700 1 --2- 172.21.15.7:0/1249179355 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efdec0a2d30 0x7efdec0a51f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009a10 tx=0x7efdfc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.645+0000 7efdf6ffd700 1 -- 172.21.15.7:0/1249179355 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efdf0014030 con 0x7efe001030f0 2024-06-10T01:03:06.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.888+0000 7efe07d0b700 1 -- 172.21.15.7:0/1249179355 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7efe00100b00 con 0x7efdec0a2d30 2024-06-10T01:03:06.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.894+0000 7efdf6ffd700 1 -- 172.21.15.7:0/1249179355 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7efe00100b00 con 0x7efdec0a2d30 2024-06-10T01:03:06.891 INFO:teuthology.orchestra.run.smithi007.stdout:/volumes/qa/sv_0/3421b3d7-2069-4bf7-9c0d-befe62146f90 2024-06-10T01:03:06.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.896+0000 7efe07d0b700 1 -- 172.21.15.7:0/1249179355 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efdec0a2d30 msgr2=0x7efdec0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:06.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.896+0000 7efe07d0b700 1 --2- 172.21.15.7:0/1249179355 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efdec0a2d30 0x7efdec0a51f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009a10 tx=0x7efdfc009190 comp rx=0 tx=0).stop 2024-06-10T01:03:06.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.896+0000 7efe07d0b700 1 -- 172.21.15.7:0/1249179355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe001030f0 msgr2=0x7efe00074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:06.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.896+0000 7efe07d0b700 1 --2- 172.21.15.7:0/1249179355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe001030f0 0x7efe00074cd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efdf000c9b0 tx=0x7efdf000ccc0 comp rx=0 tx=0).stop 2024-06-10T01:03:06.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.896+0000 7efe07d0b700 1 -- 172.21.15.7:0/1249179355 shutdown_connections 2024-06-10T01:03:06.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.896+0000 7efe07d0b700 1 --2- 172.21.15.7:0/1249179355 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efdec0a2d30 0x7efdec0a51f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:06.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.896+0000 7efe07d0b700 1 --2- 172.21.15.7:0/1249179355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe00107840 0x7efe00075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:06.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.896+0000 7efe07d0b700 1 --2- 172.21.15.7:0/1249179355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe00102740 0x7efe00074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:06.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.896+0000 7efe07d0b700 1 --2- 172.21.15.7:0/1249179355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe001030f0 0x7efe00074cd0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:06.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.896+0000 7efe07d0b700 1 -- 172.21.15.7:0/1249179355 >> 172.21.15.7:0/1249179355 conn(0x7efe000795f0 msgr2=0x7efe001003f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:06.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.896+0000 7efe07d0b700 1 -- 172.21.15.7:0/1249179355 shutdown_connections 2024-06-10T01:03:06.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:06.896+0000 7efe07d0b700 1 -- 172.21.15.7:0/1249179355 wait complete. 2024-06-10T01:03:07.016 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:06 smithi007 ceph-mon[32398]: pgmap v342: 129 pgs: 129 active+clean; 584 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-06-10T01:03:07.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:06 smithi078 ceph-mon[36683]: pgmap v342: 129 pgs: 129 active+clean; 584 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-06-10T01:03:07.205 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:06 smithi175 ceph-mon[36636]: pgmap v342: 129 pgs: 129 active+clean; 584 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-06-10T01:03:07.522 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-06-10T01:03:07.772 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:03:07.895 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:07 smithi007 ceph-mon[32398]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:03:08.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:07 smithi078 ceph-mon[36683]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:03:08.205 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:07 smithi175 ceph-mon[36636]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:03:08.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:08 smithi007 ceph-mon[32398]: pgmap v343: 129 pgs: 129 active+clean; 604 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-06-10T01:03:09.083 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:08 smithi078 ceph-mon[36683]: pgmap v343: 129 pgs: 129 active+clean; 604 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-06-10T01:03:09.205 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:08 smithi175 ceph-mon[36636]: pgmap v343: 129 pgs: 129 active+clean; 604 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-06-10T01:03:09.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.382+0000 7f02ca9e2700 1 -- 172.21.15.7:0/2692063603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02c4100e50 msgr2=0x7f02c41012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:09.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.382+0000 7f02ca9e2700 1 --2- 172.21.15.7:0/2692063603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02c4100e50 0x7f02c41012d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f02b8009990 tx=0x7f02b8009ca0 comp rx=0 tx=0).stop 2024-06-10T01:03:09.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.382+0000 7f02ca9e2700 1 -- 172.21.15.7:0/2692063603 shutdown_connections 2024-06-10T01:03:09.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.382+0000 7f02ca9e2700 1 --2- 172.21.15.7:0/2692063603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02c4101810 0x7f02c410f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:09.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.382+0000 7f02ca9e2700 1 --2- 172.21.15.7:0/2692063603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02c4107570 0x7f02c4107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:09.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.382+0000 7f02ca9e2700 1 --2- 172.21.15.7:0/2692063603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02c4100e50 0x7f02c41012d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:09.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.382+0000 7f02ca9e2700 1 -- 172.21.15.7:0/2692063603 >> 172.21.15.7:0/2692063603 conn(0x7f02c40766b0 msgr2=0x7f02c4076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:09.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.382+0000 7f02ca9e2700 1 -- 172.21.15.7:0/2692063603 shutdown_connections 2024-06-10T01:03:09.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.382+0000 7f02ca9e2700 1 -- 172.21.15.7:0/2692063603 wait complete. 2024-06-10T01:03:09.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.383+0000 7f02ca9e2700 1 Processor -- start 2024-06-10T01:03:09.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.383+0000 7f02ca9e2700 1 -- start start 2024-06-10T01:03:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.383+0000 7f02ca9e2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02c4100e50 0x7f02c419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.383+0000 7f02ca9e2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02c4101810 0x7f02c419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.383+0000 7f02ca9e2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02c4107570 0x7f02c4198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.383+0000 7f02ca9e2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f02c41127b0 con 0x7f02c4100e50 2024-06-10T01:03:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.383+0000 7f02ca9e2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f02c4112630 con 0x7f02c4101810 2024-06-10T01:03:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.383+0000 7f02ca9e2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f02c4112930 con 0x7f02c4107570 2024-06-10T01:03:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.384+0000 7f02c3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02c4100e50 0x7f02c419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.384+0000 7f02c8f7f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02c4107570 0x7f02c4198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.384+0000 7f02c3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02c4100e50 0x7f02c419e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50238/0 (socket says 172.21.15.7:50238) 2024-06-10T01:03:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.384+0000 7f02c3fff700 1 -- 172.21.15.7:0/1723738899 learned_addr learned my addr 172.21.15.7:0/1723738899 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:03:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.384+0000 7f02c37fe700 1 --2- 172.21.15.7:0/1723738899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02c4101810 0x7f02c419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.384+0000 7f02c3fff700 1 -- 172.21.15.7:0/1723738899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02c4107570 msgr2=0x7f02c4198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.384+0000 7f02c3fff700 1 --2- 172.21.15.7:0/1723738899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02c4107570 0x7f02c4198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:09.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.384+0000 7f02c3fff700 1 -- 172.21.15.7:0/1723738899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02c4101810 msgr2=0x7f02c419edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:09.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.384+0000 7f02c3fff700 1 --2- 172.21.15.7:0/1723738899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02c4101810 0x7f02c419edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:09.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.384+0000 7f02c3fff700 1 -- 172.21.15.7:0/1723738899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c41991d0 con 0x7f02c4100e50 2024-06-10T01:03:09.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.384+0000 7f02c3fff700 1 --2- 172.21.15.7:0/1723738899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02c4100e50 0x7f02c419e8b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f02b400d980 tx=0x7f02b400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:09.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.384+0000 7f02c17fa700 1 -- 172.21.15.7:0/1723738899 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b4016070 con 0x7f02c4100e50 2024-06-10T01:03:09.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.384+0000 7f02ca9e2700 1 -- 172.21.15.7:0/1723738899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c41994c0 con 0x7f02c4100e50 2024-06-10T01:03:09.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.384+0000 7f02ca9e2700 1 -- 172.21.15.7:0/1723738899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c41a5780 con 0x7f02c4100e50 2024-06-10T01:03:09.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.384+0000 7f02c17fa700 1 -- 172.21.15.7:0/1723738899 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02b400fd70 con 0x7f02c4100e50 2024-06-10T01:03:09.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.385+0000 7f02c17fa700 1 -- 172.21.15.7:0/1723738899 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b4019360 con 0x7f02c4100e50 2024-06-10T01:03:09.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.386+0000 7f02ca9e2700 1 -- 172.21.15.7:0/1723738899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02b0005330 con 0x7f02c4100e50 2024-06-10T01:03:09.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.387+0000 7f02c17fa700 1 -- 172.21.15.7:0/1723738899 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f02b4019070 con 0x7f02c4100e50 2024-06-10T01:03:09.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.387+0000 7f02c17fa700 1 --2- 172.21.15.7:0/1723738899 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f02ac0a2d30 0x7f02ac0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:09.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.387+0000 7f02c37fe700 1 --2- 172.21.15.7:0/1723738899 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f02ac0a2d30 0x7f02ac0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:09.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.387+0000 7f02c17fa700 1 -- 172.21.15.7:0/1723738899 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f02b40cedd0 con 0x7f02c4100e50 2024-06-10T01:03:09.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.388+0000 7f02c37fe700 1 --2- 172.21.15.7:0/1723738899 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f02ac0a2d30 0x7f02ac0a51f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f02b80098e0 tx=0x7f02b8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:09.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.392+0000 7f02c17fa700 1 -- 172.21.15.7:0/1723738899 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f02b409b870 con 0x7f02c4100e50 2024-06-10T01:03:09.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.632+0000 7f02ca9e2700 1 -- 172.21.15.7:0/1723738899 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f02b0000bc0 con 0x7f02ac0a2d30 2024-06-10T01:03:09.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.658+0000 7f02c17fa700 1 -- 172.21.15.7:0/1723738899 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f02b0000bc0 con 0x7f02ac0a2d30 2024-06-10T01:03:09.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.660+0000 7f02ca9e2700 1 -- 172.21.15.7:0/1723738899 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f02ac0a2d30 msgr2=0x7f02ac0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:09.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.660+0000 7f02ca9e2700 1 --2- 172.21.15.7:0/1723738899 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f02ac0a2d30 0x7f02ac0a51f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f02b80098e0 tx=0x7f02b8005960 comp rx=0 tx=0).stop 2024-06-10T01:03:09.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.660+0000 7f02ca9e2700 1 -- 172.21.15.7:0/1723738899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02c4100e50 msgr2=0x7f02c419e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:09.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.660+0000 7f02ca9e2700 1 --2- 172.21.15.7:0/1723738899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02c4100e50 0x7f02c419e8b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f02b400d980 tx=0x7f02b400dc90 comp rx=0 tx=0).stop 2024-06-10T01:03:09.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.660+0000 7f02ca9e2700 1 -- 172.21.15.7:0/1723738899 shutdown_connections 2024-06-10T01:03:09.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.660+0000 7f02ca9e2700 1 --2- 172.21.15.7:0/1723738899 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f02ac0a2d30 0x7f02ac0a51f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:09.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.660+0000 7f02ca9e2700 1 --2- 172.21.15.7:0/1723738899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02c4107570 0x7f02c4198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:09.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.660+0000 7f02ca9e2700 1 --2- 172.21.15.7:0/1723738899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02c4100e50 0x7f02c419e8b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:09.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.660+0000 7f02ca9e2700 1 --2- 172.21.15.7:0/1723738899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02c4101810 0x7f02c419edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:09.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.660+0000 7f02ca9e2700 1 -- 172.21.15.7:0/1723738899 >> 172.21.15.7:0/1723738899 conn(0x7f02c40766b0 msgr2=0x7f02c40ffab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:09.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.660+0000 7f02ca9e2700 1 -- 172.21.15.7:0/1723738899 shutdown_connections 2024-06-10T01:03:09.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:09.660+0000 7f02ca9e2700 1 -- 172.21.15.7:0/1723738899 wait complete. 2024-06-10T01:03:10.462 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-06-10T01:03:10.754 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:03:11.000 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:10 smithi007 ceph-mon[32398]: pgmap v344: 129 pgs: 129 active+clean; 604 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-06-10T01:03:11.000 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:10 smithi007 ceph-mon[32398]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:03:11.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:10 smithi078 ceph-mon[36683]: pgmap v344: 129 pgs: 129 active+clean; 604 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-06-10T01:03:11.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:10 smithi078 ceph-mon[36683]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:03:11.205 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:10 smithi175 ceph-mon[36636]: pgmap v344: 129 pgs: 129 active+clean; 604 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-06-10T01:03:11.206 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:10 smithi175 ceph-mon[36636]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:03:12.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.389+0000 7f435be77700 1 -- 172.21.15.7:0/4080840831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f435406b810 msgr2=0x7f43541070a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:12.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.389+0000 7f435be77700 1 --2- 172.21.15.7:0/4080840831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f435406b810 0x7f43541070a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f43480099c0 tx=0x7f4348009cd0 comp rx=0 tx=0).stop 2024-06-10T01:03:12.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.389+0000 7f435be77700 1 -- 172.21.15.7:0/4080840831 shutdown_connections 2024-06-10T01:03:12.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.389+0000 7f435be77700 1 --2- 172.21.15.7:0/4080840831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4354107780 0x7f435410f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:12.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.389+0000 7f435be77700 1 --2- 172.21.15.7:0/4080840831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f435406ae60 0x7f435406b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:12.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.389+0000 7f435be77700 1 --2- 172.21.15.7:0/4080840831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f435406b810 0x7f43541070a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:12.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.389+0000 7f435be77700 1 -- 172.21.15.7:0/4080840831 >> 172.21.15.7:0/4080840831 conn(0x7f43540773e0 msgr2=0x7f43540777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:12.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.389+0000 7f435be77700 1 -- 172.21.15.7:0/4080840831 shutdown_connections 2024-06-10T01:03:12.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.389+0000 7f435be77700 1 -- 172.21.15.7:0/4080840831 wait complete. 2024-06-10T01:03:12.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.390+0000 7f435be77700 1 Processor -- start 2024-06-10T01:03:12.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.390+0000 7f435be77700 1 -- start start 2024-06-10T01:03:12.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.390+0000 7f435be77700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f435406ae60 0x7f43541014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:12.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.390+0000 7f435be77700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f435406b810 0x7f4354101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:12.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.390+0000 7f435be77700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4354107780 0x7f4354101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:12.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.390+0000 7f435be77700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4354112780 con 0x7f435406ae60 2024-06-10T01:03:12.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.390+0000 7f435be77700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4354112600 con 0x7f435406b810 2024-06-10T01:03:12.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.390+0000 7f435be77700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4354112900 con 0x7f4354107780 2024-06-10T01:03:12.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.390+0000 7f4359c13700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f435406ae60 0x7f43541014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:12.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.390+0000 7f4359c13700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f435406ae60 0x7f43541014c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57392/0 (socket says 172.21.15.7:57392) 2024-06-10T01:03:12.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.390+0000 7f4359c13700 1 -- 172.21.15.7:0/3532964696 learned_addr learned my addr 172.21.15.7:0/3532964696 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:03:12.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.390+0000 7f435a414700 1 --2- 172.21.15.7:0/3532964696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4354107780 0x7f4354101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:12.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.390+0000 7f4359412700 1 --2- 172.21.15.7:0/3532964696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f435406b810 0x7f4354101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:12.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.390+0000 7f4359c13700 1 -- 172.21.15.7:0/3532964696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4354107780 msgr2=0x7f4354101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:12.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.390+0000 7f4359c13700 1 --2- 172.21.15.7:0/3532964696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4354107780 0x7f4354101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:12.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.390+0000 7f4359c13700 1 -- 172.21.15.7:0/3532964696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f435406b810 msgr2=0x7f4354101a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:12.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.390+0000 7f4359c13700 1 --2- 172.21.15.7:0/3532964696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f435406b810 0x7f4354101a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:12.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.390+0000 7f4359c13700 1 -- 172.21.15.7:0/3532964696 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43541a7180 con 0x7f435406ae60 2024-06-10T01:03:12.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.390+0000 7f435a414700 1 --2- 172.21.15.7:0/3532964696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4354107780 0x7f4354101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:03:12.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.390+0000 7f4359412700 1 --2- 172.21.15.7:0/3532964696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f435406b810 0x7f4354101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:03:12.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.391+0000 7f4359c13700 1 --2- 172.21.15.7:0/3532964696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f435406ae60 0x7f43541014c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f435000d980 tx=0x7f435000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:12.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.391+0000 7f4346ffd700 1 -- 172.21.15.7:0/3532964696 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4350016070 con 0x7f435406ae60 2024-06-10T01:03:12.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.391+0000 7f435be77700 1 -- 172.21.15.7:0/3532964696 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43541a7470 con 0x7f435406ae60 2024-06-10T01:03:12.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.391+0000 7f4346ffd700 1 -- 172.21.15.7:0/3532964696 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4350004500 con 0x7f435406ae60 2024-06-10T01:03:12.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.391+0000 7f4346ffd700 1 -- 172.21.15.7:0/3532964696 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4350004d00 con 0x7f435406ae60 2024-06-10T01:03:12.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.391+0000 7f435be77700 1 -- 172.21.15.7:0/3532964696 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43541a79b0 con 0x7f435406ae60 2024-06-10T01:03:12.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.392+0000 7f435be77700 1 -- 172.21.15.7:0/3532964696 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4354069180 con 0x7f435406ae60 2024-06-10T01:03:12.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.394+0000 7f4346ffd700 1 -- 172.21.15.7:0/3532964696 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f435000bbf0 con 0x7f435406ae60 2024-06-10T01:03:12.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.395+0000 7f4346ffd700 1 --2- 172.21.15.7:0/3532964696 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f43400a2cb0 0x7f43400a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:12.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.395+0000 7f4359412700 1 --2- 172.21.15.7:0/3532964696 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f43400a2cb0 0x7f43400a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:12.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.395+0000 7f4346ffd700 1 -- 172.21.15.7:0/3532964696 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f43500cf8a0 con 0x7f435406ae60 2024-06-10T01:03:12.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.395+0000 7f4359412700 1 --2- 172.21.15.7:0/3532964696 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f43400a2cb0 0x7f43400a5170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4348000c00 tx=0x7f4348009f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:12.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.398+0000 7f4346ffd700 1 -- 172.21.15.7:0/3532964696 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f435001f020 con 0x7f435406ae60 2024-06-10T01:03:12.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.641+0000 7f435be77700 1 -- 172.21.15.7:0/3532964696 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f43540628d0 con 0x7f43400a2cb0 2024-06-10T01:03:12.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.646+0000 7f4346ffd700 1 -- 172.21.15.7:0/3532964696 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f43540628d0 con 0x7f43400a2cb0 2024-06-10T01:03:12.643 INFO:teuthology.orchestra.run.smithi007.stdout:/volumes/qa/sv_1/56b71f3b-2893-4c82-ac3a-2d6fa4194c93 2024-06-10T01:03:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.648+0000 7f435be77700 1 -- 172.21.15.7:0/3532964696 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f43400a2cb0 msgr2=0x7f43400a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.648+0000 7f435be77700 1 --2- 172.21.15.7:0/3532964696 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f43400a2cb0 0x7f43400a5170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4348000c00 tx=0x7f4348009f30 comp rx=0 tx=0).stop 2024-06-10T01:03:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.648+0000 7f435be77700 1 -- 172.21.15.7:0/3532964696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f435406ae60 msgr2=0x7f43541014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.648+0000 7f435be77700 1 --2- 172.21.15.7:0/3532964696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f435406ae60 0x7f43541014c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f435000d980 tx=0x7f435000dd40 comp rx=0 tx=0).stop 2024-06-10T01:03:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.648+0000 7f435be77700 1 -- 172.21.15.7:0/3532964696 shutdown_connections 2024-06-10T01:03:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.648+0000 7f435be77700 1 --2- 172.21.15.7:0/3532964696 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f43400a2cb0 0x7f43400a5170 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:12.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.648+0000 7f435be77700 1 --2- 172.21.15.7:0/3532964696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4354107780 0x7f4354101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:12.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.648+0000 7f435be77700 1 --2- 172.21.15.7:0/3532964696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f435406ae60 0x7f43541014c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:12.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.649+0000 7f435be77700 1 --2- 172.21.15.7:0/3532964696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f435406b810 0x7f4354101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:12.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.649+0000 7f435be77700 1 -- 172.21.15.7:0/3532964696 >> 172.21.15.7:0/3532964696 conn(0x7f43540773e0 msgr2=0x7f4354106360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:12.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.649+0000 7f435be77700 1 -- 172.21.15.7:0/3532964696 shutdown_connections 2024-06-10T01:03:12.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:12.649+0000 7f435be77700 1 -- 172.21.15.7:0/3532964696 wait complete. 2024-06-10T01:03:13.004 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:12 smithi007 ceph-mon[32398]: pgmap v345: 129 pgs: 129 active+clean; 624 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-06-10T01:03:13.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:12 smithi078 ceph-mon[36683]: pgmap v345: 129 pgs: 129 active+clean; 624 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-06-10T01:03:13.205 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:12 smithi175 ceph-mon[36636]: pgmap v345: 129 pgs: 129 active+clean; 624 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-06-10T01:03:13.405 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:03:13.656 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:03:14.013 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:13 smithi007 ceph-mon[32398]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:03:14.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:13 smithi078 ceph-mon[36683]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:03:14.205 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:13 smithi175 ceph-mon[36636]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:03:15.016 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:14 smithi007 ceph-mon[32398]: pgmap v346: 129 pgs: 129 active+clean; 624 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-06-10T01:03:15.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:14 smithi078 ceph-mon[36683]: pgmap v346: 129 pgs: 129 active+clean; 624 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-06-10T01:03:15.205 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:14 smithi175 ceph-mon[36636]: pgmap v346: 129 pgs: 129 active+clean; 624 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-06-10T01:03:15.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.451+0000 7f37ccc71700 1 -- 172.21.15.7:0/733501813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37c8101d80 msgr2=0x7f37c810e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:15.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.451+0000 7f37ccc71700 1 --2- 172.21.15.7:0/733501813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37c8101d80 0x7f37c810e240 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f37c000b0a0 tx=0x7f37c000b3b0 comp rx=0 tx=0).stop 2024-06-10T01:03:15.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.451+0000 7f37ccc71700 1 -- 172.21.15.7:0/733501813 shutdown_connections 2024-06-10T01:03:15.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.451+0000 7f37ccc71700 1 --2- 172.21.15.7:0/733501813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37c8101d80 0x7f37c810e240 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:15.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.451+0000 7f37ccc71700 1 --2- 172.21.15.7:0/733501813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37c8107440 0x7f37c8107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:15.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.451+0000 7f37ccc71700 1 --2- 172.21.15.7:0/733501813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37c81013c0 0x7f37c8101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:15.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.451+0000 7f37ccc71700 1 -- 172.21.15.7:0/733501813 >> 172.21.15.7:0/733501813 conn(0x7f37c8077400 msgr2=0x7f37c8077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:15.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.451+0000 7f37ccc71700 1 -- 172.21.15.7:0/733501813 shutdown_connections 2024-06-10T01:03:15.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.451+0000 7f37ccc71700 1 -- 172.21.15.7:0/733501813 wait complete. 2024-06-10T01:03:15.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.451+0000 7f37ccc71700 1 Processor -- start 2024-06-10T01:03:15.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.452+0000 7f37ccc71700 1 -- start start 2024-06-10T01:03:15.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.452+0000 7f37ccc71700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37c81013c0 0x7f37c819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:15.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.452+0000 7f37ccc71700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37c8101d80 0x7f37c819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:15.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.452+0000 7f37ccc71700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37c8107440 0x7f37c81989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.452+0000 7f37ccc71700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f37c8111680 con 0x7f37c8101d80 2024-06-10T01:03:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.452+0000 7f37ccc71700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f37c8111500 con 0x7f37c81013c0 2024-06-10T01:03:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.452+0000 7f37ccc71700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f37c8111800 con 0x7f37c8107440 2024-06-10T01:03:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.452+0000 7f37c5d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37c8101d80 0x7f37c819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.452+0000 7f37c5d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37c8101d80 0x7f37c819ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57400/0 (socket says 172.21.15.7:57400) 2024-06-10T01:03:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.452+0000 7f37c5d9b700 1 -- 172.21.15.7:0/925237896 learned_addr learned my addr 172.21.15.7:0/925237896 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:03:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.452+0000 7f37c659c700 1 --2- 172.21.15.7:0/925237896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37c81013c0 0x7f37c819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.452+0000 7f37c6d9d700 1 --2- 172.21.15.7:0/925237896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37c8107440 0x7f37c81989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.452+0000 7f37c5d9b700 1 -- 172.21.15.7:0/925237896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37c8107440 msgr2=0x7f37c81989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.452+0000 7f37c5d9b700 1 --2- 172.21.15.7:0/925237896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37c8107440 0x7f37c81989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.452+0000 7f37c5d9b700 1 -- 172.21.15.7:0/925237896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37c81013c0 msgr2=0x7f37c819e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.452+0000 7f37c5d9b700 1 --2- 172.21.15.7:0/925237896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37c81013c0 0x7f37c819e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.452+0000 7f37c5d9b700 1 -- 172.21.15.7:0/925237896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c8199140 con 0x7f37c8101d80 2024-06-10T01:03:15.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.452+0000 7f37c659c700 1 --2- 172.21.15.7:0/925237896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37c81013c0 0x7f37c819e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:03:15.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.452+0000 7f37c5d9b700 1 --2- 172.21.15.7:0/925237896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37c8101d80 0x7f37c819ed60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00e9c0 tx=0x7f37bc00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:15.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.453+0000 7f37b77fe700 1 -- 172.21.15.7:0/925237896 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc00cbd0 con 0x7f37c8101d80 2024-06-10T01:03:15.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.453+0000 7f37b77fe700 1 -- 172.21.15.7:0/925237896 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37bc00cd70 con 0x7f37c8101d80 2024-06-10T01:03:15.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.453+0000 7f37ccc71700 1 -- 172.21.15.7:0/925237896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c8199430 con 0x7f37c8101d80 2024-06-10T01:03:15.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.453+0000 7f37b77fe700 1 -- 172.21.15.7:0/925237896 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc010430 con 0x7f37c8101d80 2024-06-10T01:03:15.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.453+0000 7f37ccc71700 1 -- 172.21.15.7:0/925237896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c81a56f0 con 0x7f37c8101d80 2024-06-10T01:03:15.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.454+0000 7f37ccc71700 1 -- 172.21.15.7:0/925237896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37c8102ab0 con 0x7f37c8101d80 2024-06-10T01:03:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.455+0000 7f37b77fe700 1 -- 172.21.15.7:0/925237896 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f37bc004750 con 0x7f37c8101d80 2024-06-10T01:03:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.456+0000 7f37b77fe700 1 --2- 172.21.15.7:0/925237896 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f37b00a2cb0 0x7f37b00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.456+0000 7f37b77fe700 1 -- 172.21.15.7:0/925237896 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f37bc0cfbf0 con 0x7f37c8101d80 2024-06-10T01:03:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.456+0000 7f37c659c700 1 --2- 172.21.15.7:0/925237896 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f37b00a2cb0 0x7f37b00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.456+0000 7f37c659c700 1 --2- 172.21.15.7:0/925237896 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f37b00a2cb0 0x7f37b00a5170 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f37b8005e10 tx=0x7f37b8005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:15.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.459+0000 7f37b77fe700 1 -- 172.21.15.7:0/925237896 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f37bc053850 con 0x7f37c8101d80 2024-06-10T01:03:15.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.710+0000 7f37ccc71700 1 -- 172.21.15.7:0/925237896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37c8069180 con 0x7f37c8101d80 2024-06-10T01:03:15.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.711+0000 7f37b77fe700 1 -- 172.21.15.7:0/925237896 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11855 (secure 0 0 0) 0x7f37c8069180 con 0x7f37c8101d80 2024-06-10T01:03:15.709 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:03:15.709 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:02:58.523980+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/2784910673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2784910673},{"type":"v1","addr":"172.21.15.175:6835","nonce":2784910673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:03:15.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.713+0000 7f37ccc71700 1 -- 172.21.15.7:0/925237896 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f37b00a2cb0 msgr2=0x7f37b00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:15.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.713+0000 7f37ccc71700 1 --2- 172.21.15.7:0/925237896 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f37b00a2cb0 0x7f37b00a5170 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f37b8005e10 tx=0x7f37b8005d60 comp rx=0 tx=0).stop 2024-06-10T01:03:15.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.713+0000 7f37ccc71700 1 -- 172.21.15.7:0/925237896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37c8101d80 msgr2=0x7f37c819ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:15.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.713+0000 7f37ccc71700 1 --2- 172.21.15.7:0/925237896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37c8101d80 0x7f37c819ed60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00e9c0 tx=0x7f37bc00ed80 comp rx=0 tx=0).stop 2024-06-10T01:03:15.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.713+0000 7f37ccc71700 1 -- 172.21.15.7:0/925237896 shutdown_connections 2024-06-10T01:03:15.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.713+0000 7f37ccc71700 1 --2- 172.21.15.7:0/925237896 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f37b00a2cb0 0x7f37b00a5170 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:15.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.713+0000 7f37ccc71700 1 --2- 172.21.15.7:0/925237896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37c8107440 0x7f37c81989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:15.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.713+0000 7f37ccc71700 1 --2- 172.21.15.7:0/925237896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37c81013c0 0x7f37c819e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:15.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.713+0000 7f37ccc71700 1 --2- 172.21.15.7:0/925237896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37c8101d80 0x7f37c819ed60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:15.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.713+0000 7f37ccc71700 1 -- 172.21.15.7:0/925237896 >> 172.21.15.7:0/925237896 conn(0x7f37c8077400 msgr2=0x7f37c8100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:15.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.714+0000 7f37ccc71700 1 -- 172.21.15.7:0/925237896 shutdown_connections 2024-06-10T01:03:15.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:15.714+0000 7f37ccc71700 1 -- 172.21.15.7:0/925237896 wait complete. 2024-06-10T01:03:15.712 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-06-10T01:03:16.016 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:15 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/925237896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:03:16.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:15 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/925237896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:03:16.205 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:15 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/925237896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:03:16.436 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T01:03:16.688 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:03:17.016 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:16 smithi007 ceph-mon[32398]: pgmap v347: 129 pgs: 129 active+clean; 624 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-06-10T01:03:17.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:16 smithi078 ceph-mon[36683]: pgmap v347: 129 pgs: 129 active+clean; 624 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-06-10T01:03:17.205 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:16 smithi175 ceph-mon[36636]: pgmap v347: 129 pgs: 129 active+clean; 624 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-06-10T01:03:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.342+0000 7f8fff431700 1 -- 172.21.15.7:0/2127569199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ff81038f0 msgr2=0x7f8ff8100f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.342+0000 7f8fff431700 1 --2- 172.21.15.7:0/2127569199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ff81038f0 0x7f8ff8100f20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009990 tx=0x7f8fe8009ca0 comp rx=0 tx=0).stop 2024-06-10T01:03:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.342+0000 7f8fff431700 1 -- 172.21.15.7:0/2127569199 shutdown_connections 2024-06-10T01:03:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.342+0000 7f8fff431700 1 --2- 172.21.15.7:0/2127569199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ff810f160 0x7f8ff8111550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.342+0000 7f8fff431700 1 --2- 172.21.15.7:0/2127569199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ff8103480 0x7f8ff81009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.342+0000 7f8fff431700 1 --2- 172.21.15.7:0/2127569199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ff81038f0 0x7f8ff8100f20 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.342+0000 7f8fff431700 1 -- 172.21.15.7:0/2127569199 >> 172.21.15.7:0/2127569199 conn(0x7f8ff8077990 msgr2=0x7f8ff80fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.342+0000 7f8fff431700 1 -- 172.21.15.7:0/2127569199 shutdown_connections 2024-06-10T01:03:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.342+0000 7f8fff431700 1 -- 172.21.15.7:0/2127569199 wait complete. 2024-06-10T01:03:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.343+0000 7f8fff431700 1 Processor -- start 2024-06-10T01:03:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.343+0000 7f8fff431700 1 -- start start 2024-06-10T01:03:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.343+0000 7f8fff431700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ff8103480 0x7f8ff819aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.343+0000 7f8fff431700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ff81038f0 0x7f8ff819b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.343+0000 7f8fff431700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ff810f160 0x7f8ff819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.343+0000 7f8fff431700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff8114a20 con 0x7f8ff81038f0 2024-06-10T01:03:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.343+0000 7f8fff431700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff81148a0 con 0x7f8ff810f160 2024-06-10T01:03:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.343+0000 7f8fff431700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff8114ba0 con 0x7f8ff8103480 2024-06-10T01:03:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.343+0000 7f8ffc9cc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ff81038f0 0x7f8ff819b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.343+0000 7f8ffc9cc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ff81038f0 0x7f8ff819b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57424/0 (socket says 172.21.15.7:57424) 2024-06-10T01:03:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.344+0000 7f8ffc9cc700 1 -- 172.21.15.7:0/1771848055 learned_addr learned my addr 172.21.15.7:0/1771848055 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:03:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.344+0000 7f8ffd9ce700 1 --2- 172.21.15.7:0/1771848055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ff810f160 0x7f8ff819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.344+0000 7f8ffd1cd700 1 --2- 172.21.15.7:0/1771848055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ff8103480 0x7f8ff819aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.344+0000 7f8ffc9cc700 1 -- 172.21.15.7:0/1771848055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ff8103480 msgr2=0x7f8ff819aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.344+0000 7f8ffc9cc700 1 --2- 172.21.15.7:0/1771848055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ff8103480 0x7f8ff819aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.344+0000 7f8ffc9cc700 1 -- 172.21.15.7:0/1771848055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ff810f160 msgr2=0x7f8ff819f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.344+0000 7f8ffc9cc700 1 --2- 172.21.15.7:0/1771848055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ff810f160 0x7f8ff819f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.344+0000 7f8ffc9cc700 1 -- 172.21.15.7:0/1771848055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff819fee0 con 0x7f8ff81038f0 2024-06-10T01:03:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.344+0000 7f8ffc9cc700 1 --2- 172.21.15.7:0/1771848055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ff81038f0 0x7f8ff819b410 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8006010 tx=0x7f8fe8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.344+0000 7f8fee7fc700 1 -- 172.21.15.7:0/1771848055 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8027070 con 0x7f8ff81038f0 2024-06-10T01:03:18.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.344+0000 7f8fee7fc700 1 -- 172.21.15.7:0/1771848055 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fe8004050 con 0x7f8ff81038f0 2024-06-10T01:03:18.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.344+0000 7f8fff431700 1 -- 172.21.15.7:0/1771848055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff81a0170 con 0x7f8ff81038f0 2024-06-10T01:03:18.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.344+0000 7f8fee7fc700 1 -- 172.21.15.7:0/1771848055 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8022c00 con 0x7f8ff81038f0 2024-06-10T01:03:18.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.344+0000 7f8fff431700 1 -- 172.21.15.7:0/1771848055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff81a7a50 con 0x7f8ff81038f0 2024-06-10T01:03:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.345+0000 7f8fff431700 1 -- 172.21.15.7:0/1771848055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ff8069180 con 0x7f8ff81038f0 2024-06-10T01:03:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.347+0000 7f8fee7fc700 1 -- 172.21.15.7:0/1771848055 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f8fe802a070 con 0x7f8ff81038f0 2024-06-10T01:03:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.347+0000 7f8fee7fc700 1 --2- 172.21.15.7:0/1771848055 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8fe40a2d30 0x7f8fe40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.347+0000 7f8fee7fc700 1 -- 172.21.15.7:0/1771848055 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f8fe80d0bd0 con 0x7f8ff81038f0 2024-06-10T01:03:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.347+0000 7f8ffd1cd700 1 --2- 172.21.15.7:0/1771848055 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8fe40a2d30 0x7f8fe40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.348+0000 7f8ffd1cd700 1 --2- 172.21.15.7:0/1771848055 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8fe40a2d30 0x7f8fe40a51f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4009d10 tx=0x7f8ff4009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.351+0000 7f8fee7fc700 1 -- 172.21.15.7:0/1771848055 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8fe8031070 con 0x7f8ff81038f0 2024-06-10T01:03:18.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.560+0000 7f8fff431700 1 -- 172.21.15.7:0/1771848055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ff81a7c20 con 0x7f8ff81038f0 2024-06-10T01:03:18.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.562+0000 7f8fee7fc700 1 -- 172.21.15.7:0/1771848055 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24493 (secure 0 0 0) 0x7f8fe8022e20 con 0x7f8ff81038f0 2024-06-10T01:03:18.561 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:03:18.561 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":97,"fsid":"bded58f8-26c3-11ef-bc9b-c7b262605968","created":"2024-06-10T00:53:31.966085+0000","modified":"2024-06-10T01:02:37.362590+0000","last_up_change":"2024-06-10T01:00:45.382473+0000","last_in_change":"2024-06-10T01:00:25.608513+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:56:59.964055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T01:01:42.563611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T01:01:46.175918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T01:01:55.695502+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f7c24c9f-b7ab-4c04-b2b4-120df9751667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6803","nonce":404485726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6805","nonce":404485726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6809","nonce":404485726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6807","nonce":404485726}]},"public_addr":"172.21.15.7:6803/404485726","cluster_addr":"172.21.15.7:6805/404485726","heartbeat_back_addr":"172.21.15.7:6809/404485726","heartbeat_front_addr":"172.21.15.7:6807/404485726","state":["exists","up"]},{"osd":1,"uuid":"9bf70a6b-f31c-4077-b5af-229f17bc2845","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6801","nonce":1856804448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6803","nonce":1856804448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6807","nonce":1856804448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6805","nonce":1856804448}]},"public_addr":"172.21.15.78:6801/1856804448","cluster_addr":"172.21.15.78:6803/1856804448","heartbeat_back_addr":"172.21.15.78:6807/1856804448","heartbeat_front_addr":"172.21.15.78:6805/1856804448","state":["exists","up"]},{"osd":2,"uuid":"62923c97-caf9-4f7e-9c14-9c9918843c78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6801","nonce":918257455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6803","nonce":918257455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6807","nonce":918257455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6805","nonce":918257455}]},"public_addr":"172.21.15.175:6801/918257455","cluster_addr":"172.21.15.175:6803/918257455","heartbeat_back_addr":"172.21.15.175:6807/918257455","heartbeat_front_addr":"172.21.15.175:6805/918257455","state":["exists","up"]},{"osd":3,"uuid":"65242098-e748-48ab-8f0e-bb25b5c62742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6811","nonce":1797404966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6813","nonce":1797404966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6817","nonce":1797404966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6815","nonce":1797404966}]},"public_addr":"172.21.15.7:6811/1797404966","cluster_addr":"172.21.15.7:6813/1797404966","heartbeat_back_addr":"172.21.15.7:6817/1797404966","heartbeat_front_addr":"172.21.15.7:6815/1797404966","state":["exists","up"]},{"osd":4,"uuid":"a7990305-de9e-400e-a432-d7854f73f339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6809","nonce":3767608357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6811","nonce":3767608357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6815","nonce":3767608357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6813","nonce":3767608357}]},"public_addr":"172.21.15.78:6809/3767608357","cluster_addr":"172.21.15.78:6811/3767608357","heartbeat_back_addr":"172.21.15.78:6815/3767608357","heartbeat_front_addr":"172.21.15.78:6813/3767608357","state":["exists","up"]},{"osd":5,"uuid":"bad9a7b3-da5a-48d0-905d-e6a3190317fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6809","nonce":2157664731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6811","nonce":2157664731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6815","nonce":2157664731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6813","nonce":2157664731}]},"public_addr":"172.21.15.175:6809/2157664731","cluster_addr":"172.21.15.175:6811/2157664731","heartbeat_back_addr":"172.21.15.175:6815/2157664731","heartbeat_front_addr":"172.21.15.175:6813/2157664731","state":["exists","up"]},{"osd":6,"uuid":"6c687653-157b-4b96-867b-a2eaa7d8ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6819","nonce":838731756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6821","nonce":838731756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6825","nonce":838731756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6823","nonce":838731756}]},"public_addr":"172.21.15.7:6819/838731756","cluster_addr":"172.21.15.7:6821/838731756","heartbeat_back_addr":"172.21.15.7:6825/838731756","heartbeat_front_addr":"172.21.15.7:6823/838731756","state":["exists","up"]},{"osd":7,"uuid":"db40ddf1-9fd9-4859-b8c4-75f79582ab91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6817","nonce":3511921773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6819","nonce":3511921773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6823","nonce":3511921773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6821","nonce":3511921773}]},"public_addr":"172.21.15.78:6817/3511921773","cluster_addr":"172.21.15.78:6819/3511921773","heartbeat_back_addr":"172.21.15.78:6823/3511921773","heartbeat_front_addr":"172.21.15.78:6821/3511921773","state":["exists","up"]},{"osd":8,"uuid":"0f01cbd9-b193-4c70-a33c-4984167bf422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6817","nonce":1546325211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6819","nonce":1546325211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6823","nonce":1546325211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6821","nonce":1546325211}]},"public_addr":"172.21.15.175:6817/1546325211","cluster_addr":"172.21.15.175:6819/1546325211","heartbeat_back_addr":"172.21.15.175:6823/1546325211","heartbeat_front_addr":"172.21.15.175:6821/1546325211","state":["exists","up"]},{"osd":9,"uuid":"e5480998-f7fb-49cd-9aed-52405c88d830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6827","nonce":1844550813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6829","nonce":1844550813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6833","nonce":1844550813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6831","nonce":1844550813}]},"public_addr":"172.21.15.7:6827/1844550813","cluster_addr":"172.21.15.7:6829/1844550813","heartbeat_back_addr":"172.21.15.7:6833/1844550813","heartbeat_front_addr":"172.21.15.7:6831/1844550813","state":["exists","up"]},{"osd":10,"uuid":"caa82f33-a71c-4907-bf3b-30a7fdfeb71d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6825","nonce":2290824785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6827","nonce":2290824785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6831","nonce":2290824785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6829","nonce":2290824785}]},"public_addr":"172.21.15.78:6825/2290824785","cluster_addr":"172.21.15.78:6827/2290824785","heartbeat_back_addr":"172.21.15.78:6831/2290824785","heartbeat_front_addr":"172.21.15.78:6829/2290824785","state":["exists","up"]},{"osd":11,"uuid":"608cda77-77da-4da2-a7e2-76a773be27c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6825","nonce":1959979550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6827","nonce":1959979550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6831","nonce":1959979550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6829","nonce":1959979550}]},"public_addr":"172.21.15.175:6825/1959979550","cluster_addr":"172.21.15.175:6827/1959979550","heartbeat_back_addr":"172.21.15.175:6831/1959979550","heartbeat_front_addr":"172.21.15.175:6829/1959979550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:08.850203+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:32.820161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:56.633656+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:21.759372+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:46.735874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:10.636541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:36.990099+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:01.767098+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:26.882776+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:52.757595+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:18.383938+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:43.544330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1833397470":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/322543993":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/212510583":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6801/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6800/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/2679436843":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6801/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/1210355277":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6800/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/3610017406":"2024-06-11T00:54:08.102405+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T01:03:18.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.564+0000 7f8fff431700 1 -- 172.21.15.7:0/1771848055 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8fe40a2d30 msgr2=0x7f8fe40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:18.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.564+0000 7f8fff431700 1 --2- 172.21.15.7:0/1771848055 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8fe40a2d30 0x7f8fe40a51f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4009d10 tx=0x7f8ff4009040 comp rx=0 tx=0).stop 2024-06-10T01:03:18.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.564+0000 7f8fff431700 1 -- 172.21.15.7:0/1771848055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ff81038f0 msgr2=0x7f8ff819b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:18.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.564+0000 7f8fff431700 1 --2- 172.21.15.7:0/1771848055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ff81038f0 0x7f8ff819b410 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8006010 tx=0x7f8fe8009f90 comp rx=0 tx=0).stop 2024-06-10T01:03:18.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.564+0000 7f8fff431700 1 -- 172.21.15.7:0/1771848055 shutdown_connections 2024-06-10T01:03:18.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.564+0000 7f8fff431700 1 --2- 172.21.15.7:0/1771848055 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8fe40a2d30 0x7f8fe40a51f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:18.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.564+0000 7f8fff431700 1 --2- 172.21.15.7:0/1771848055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ff810f160 0x7f8ff819f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:18.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.564+0000 7f8fff431700 1 --2- 172.21.15.7:0/1771848055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ff8103480 0x7f8ff819aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:18.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.564+0000 7f8fff431700 1 --2- 172.21.15.7:0/1771848055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ff81038f0 0x7f8ff819b410 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:18.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.564+0000 7f8fff431700 1 -- 172.21.15.7:0/1771848055 >> 172.21.15.7:0/1771848055 conn(0x7f8ff8077990 msgr2=0x7f8ff81106d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:18.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.564+0000 7f8fff431700 1 -- 172.21.15.7:0/1771848055 shutdown_connections 2024-06-10T01:03:18.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:18.564+0000 7f8fff431700 1 -- 172.21.15.7:0/1771848055 wait complete. 2024-06-10T01:03:18.871 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:18 smithi007 ceph-mon[32398]: pgmap v348: 129 pgs: 129 active+clean; 634 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-06-10T01:03:18.871 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:18 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1771848055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:03:19.008 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:18 smithi175 ceph-mon[36636]: pgmap v348: 129 pgs: 129 active+clean; 634 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-06-10T01:03:19.008 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:18 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1771848055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:03:19.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:18 smithi078 ceph-mon[36683]: pgmap v348: 129 pgs: 129 active+clean; 634 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-06-10T01:03:19.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:18 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1771848055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:03:19.205 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-10T01:03:19.205 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-10T01:03:19.458 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:03:20.803 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:20 smithi007 ceph-mon[32398]: pgmap v349: 129 pgs: 129 active+clean; 634 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-06-10T01:03:21.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.052+0000 7f9d6541d700 1 -- 172.21.15.7:0/2438528206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d60107840 msgr2=0x7f9d6010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:21.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.052+0000 7f9d6541d700 1 --2- 172.21.15.7:0/2438528206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d60107840 0x7f9d6010f3c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9d580099c0 tx=0x7f9d58009cd0 comp rx=0 tx=0).stop 2024-06-10T01:03:21.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.052+0000 7f9d6541d700 1 -- 172.21.15.7:0/2438528206 shutdown_connections 2024-06-10T01:03:21.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.052+0000 7f9d6541d700 1 --2- 172.21.15.7:0/2438528206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d60107840 0x7f9d6010f3c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:21.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.052+0000 7f9d6541d700 1 --2- 172.21.15.7:0/2438528206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d60102740 0x7f9d60102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:21.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.052+0000 7f9d6541d700 1 --2- 172.21.15.7:0/2438528206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d601030f0 0x7f9d60107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:21.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.052+0000 7f9d6541d700 1 -- 172.21.15.7:0/2438528206 >> 172.21.15.7:0/2438528206 conn(0x7f9d600795f0 msgr2=0x7f9d60079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:21.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.052+0000 7f9d6541d700 1 -- 172.21.15.7:0/2438528206 shutdown_connections 2024-06-10T01:03:21.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.052+0000 7f9d6541d700 1 -- 172.21.15.7:0/2438528206 wait complete. 2024-06-10T01:03:21.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.053+0000 7f9d6541d700 1 Processor -- start 2024-06-10T01:03:21.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.053+0000 7f9d6541d700 1 -- start start 2024-06-10T01:03:21.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.053+0000 7f9d6541d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d60102740 0x7f9d6019e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:21.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.053+0000 7f9d6541d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d601030f0 0x7f9d6019ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:21.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.053+0000 7f9d6541d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d60107840 0x7f9d60198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:21.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.053+0000 7f9d6541d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9d60112780 con 0x7f9d601030f0 2024-06-10T01:03:21.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.053+0000 7f9d6541d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9d60112600 con 0x7f9d60107840 2024-06-10T01:03:21.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.053+0000 7f9d6541d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9d60112900 con 0x7f9d60102740 2024-06-10T01:03:21.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.053+0000 7f9d5e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d601030f0 0x7f9d6019ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:21.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.054+0000 7f9d5e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d601030f0 0x7f9d6019ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57444/0 (socket says 172.21.15.7:57444) 2024-06-10T01:03:21.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.054+0000 7f9d5e7fc700 1 -- 172.21.15.7:0/2620473661 learned_addr learned my addr 172.21.15.7:0/2620473661 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:03:21.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.054+0000 7f9d5effd700 1 --2- 172.21.15.7:0/2620473661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d60102740 0x7f9d6019e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:21.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.054+0000 7f9d5f7fe700 1 --2- 172.21.15.7:0/2620473661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d60107840 0x7f9d60198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:21.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.054+0000 7f9d5e7fc700 1 -- 172.21.15.7:0/2620473661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d60102740 msgr2=0x7f9d6019e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:21.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.054+0000 7f9d5e7fc700 1 --2- 172.21.15.7:0/2620473661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d60102740 0x7f9d6019e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:21.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.054+0000 7f9d5e7fc700 1 -- 172.21.15.7:0/2620473661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d60107840 msgr2=0x7f9d60198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:21.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.054+0000 7f9d5e7fc700 1 --2- 172.21.15.7:0/2620473661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d60107840 0x7f9d60198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:21.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.054+0000 7f9d5e7fc700 1 -- 172.21.15.7:0/2620473661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d601990c0 con 0x7f9d601030f0 2024-06-10T01:03:21.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.054+0000 7f9d5f7fe700 1 --2- 172.21.15.7:0/2620473661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d60107840 0x7f9d60198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:03:21.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.054+0000 7f9d5e7fc700 1 --2- 172.21.15.7:0/2620473661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d601030f0 0x7f9d6019ece0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9d4800d950 tx=0x7f9d4800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:21.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.054+0000 7f9d57fff700 1 -- 172.21.15.7:0/2620473661 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d48016070 con 0x7f9d601030f0 2024-06-10T01:03:21.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.054+0000 7f9d6541d700 1 -- 172.21.15.7:0/2620473661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d601993b0 con 0x7f9d601030f0 2024-06-10T01:03:21.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.054+0000 7f9d57fff700 1 -- 172.21.15.7:0/2620473661 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d48009e20 con 0x7f9d601030f0 2024-06-10T01:03:21.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.054+0000 7f9d57fff700 1 -- 172.21.15.7:0/2620473661 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4800f620 con 0x7f9d601030f0 2024-06-10T01:03:21.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.054+0000 7f9d6541d700 1 -- 172.21.15.7:0/2620473661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d601a5670 con 0x7f9d601030f0 2024-06-10T01:03:21.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.055+0000 7f9d6541d700 1 -- 172.21.15.7:0/2620473661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d6010aa10 con 0x7f9d601030f0 2024-06-10T01:03:21.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.057+0000 7f9d57fff700 1 -- 172.21.15.7:0/2620473661 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f9d48019070 con 0x7f9d601030f0 2024-06-10T01:03:21.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.057+0000 7f9d57fff700 1 --2- 172.21.15.7:0/2620473661 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9d4c0a2cb0 0x7f9d4c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:21.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.057+0000 7f9d57fff700 1 -- 172.21.15.7:0/2620473661 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f9d480cf100 con 0x7f9d601030f0 2024-06-10T01:03:21.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.057+0000 7f9d5effd700 1 --2- 172.21.15.7:0/2620473661 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9d4c0a2cb0 0x7f9d4c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:21.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.058+0000 7f9d5effd700 1 --2- 172.21.15.7:0/2620473661 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9d4c0a2cb0 0x7f9d4c0a5170 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9d60199aa0 tx=0x7f9d50009180 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:21.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.062+0000 7f9d57fff700 1 -- 172.21.15.7:0/2620473661 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9d4809bba0 con 0x7f9d601030f0 2024-06-10T01:03:21.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:20 smithi078 ceph-mon[36683]: pgmap v349: 129 pgs: 129 active+clean; 634 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-06-10T01:03:21.205 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:20 smithi175 ceph-mon[36636]: pgmap v349: 129 pgs: 129 active+clean; 634 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-06-10T01:03:21.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.265+0000 7f9d6541d700 1 -- 172.21.15.7:0/2620473661 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9d600628d0 con 0x7f9d4c0a2cb0 2024-06-10T01:03:21.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.276+0000 7f9d57fff700 1 -- 172.21.15.7:0/2620473661 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340629 (secure 0 0 0) 0x7f9d600628d0 con 0x7f9d4c0a2cb0 2024-06-10T01:03:21.279 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:03:21.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.279+0000 7f9d6541d700 1 -- 172.21.15.7:0/2620473661 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9d4c0a2cb0 msgr2=0x7f9d4c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:21.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.279+0000 7f9d6541d700 1 --2- 172.21.15.7:0/2620473661 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9d4c0a2cb0 0x7f9d4c0a5170 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9d60199aa0 tx=0x7f9d50009180 comp rx=0 tx=0).stop 2024-06-10T01:03:21.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.279+0000 7f9d6541d700 1 -- 172.21.15.7:0/2620473661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d601030f0 msgr2=0x7f9d6019ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:21.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.279+0000 7f9d6541d700 1 --2- 172.21.15.7:0/2620473661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d601030f0 0x7f9d6019ece0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9d4800d950 tx=0x7f9d4800dc60 comp rx=0 tx=0).stop 2024-06-10T01:03:21.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.279+0000 7f9d6541d700 1 -- 172.21.15.7:0/2620473661 shutdown_connections 2024-06-10T01:03:21.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.279+0000 7f9d6541d700 1 --2- 172.21.15.7:0/2620473661 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9d4c0a2cb0 0x7f9d4c0a5170 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:21.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.279+0000 7f9d6541d700 1 --2- 172.21.15.7:0/2620473661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d60107840 0x7f9d60198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:21.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.279+0000 7f9d6541d700 1 --2- 172.21.15.7:0/2620473661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d60102740 0x7f9d6019e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:21.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.279+0000 7f9d6541d700 1 --2- 172.21.15.7:0/2620473661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d601030f0 0x7f9d6019ece0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:21.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.279+0000 7f9d6541d700 1 -- 172.21.15.7:0/2620473661 >> 172.21.15.7:0/2620473661 conn(0x7f9d600795f0 msgr2=0x7f9d601005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:21.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.279+0000 7f9d6541d700 1 -- 172.21.15.7:0/2620473661 shutdown_connections 2024-06-10T01:03:21.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:21.279+0000 7f9d6541d700 1 -- 172.21.15.7:0/2620473661 wait complete. 2024-06-10T01:03:21.283 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-06-10T01:03:21.875 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:21 smithi007 ceph-mon[32398]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T01:03:21.904 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-06-10T01:03:19.556527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649187,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":283,"num_read_kb":250,"num_write":345,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":277,"ondisk_log_size":277,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362164,"kb_used_data":13012,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579660,"statfs":{"total":1151940427776,"available":1151569571840,"internally_reserved":0,"allocated":13324288,"data_stored":8545115,"data_compressed":425322,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30635,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":66,"num_write":23,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003238"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.375628+0000","last_change":"2024-06-10T01:01:47.244728+0000","last_active":"2024-06-10T01:02:37.375628+0000","last_peered":"2024-06-10T01:02:37.375628+0000","last_clean":"2024-06-10T01:02:37.375628+0000","last_became_active":"2024-06-10T01:01:47.244198+0000","last_became_peered":"2024-06-10T01:01:47.244198+0000","last_unstale":"2024-06-10T01:02:37.375628+0000","last_undegraded":"2024-06-10T01:02:37.375628+0000","last_fullsized":"2024-06-10T01:02:37.375628+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:58:06.257952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.373467+0000","last_change":"2024-06-10T01:01:47.247620+0000","last_active":"2024-06-10T01:02:37.373467+0000","last_peered":"2024-06-10T01:02:37.373467+0000","last_clean":"2024-06-10T01:02:37.373467+0000","last_became_active":"2024-06-10T01:01:47.245475+0000","last_became_peered":"2024-06-10T01:01:47.245475+0000","last_unstale":"2024-06-10T01:02:37.373467+0000","last_undegraded":"2024-06-10T01:02:37.373467+0000","last_fullsized":"2024-06-10T01:02:37.373467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:29:09.367411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.372987+0000","last_change":"2024-06-10T01:01:47.252414+0000","last_active":"2024-06-10T01:02:37.372987+0000","last_peered":"2024-06-10T01:02:37.372987+0000","last_clean":"2024-06-10T01:02:37.372987+0000","last_became_active":"2024-06-10T01:01:47.252063+0000","last_became_peered":"2024-06-10T01:01:47.252063+0000","last_unstale":"2024-06-10T01:02:37.372987+0000","last_undegraded":"2024-06-10T01:02:37.372987+0000","last_fullsized":"2024-06-10T01:02:37.372987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:54:53.966209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.369842+0000","last_change":"2024-06-10T01:01:47.254875+0000","last_active":"2024-06-10T01:02:37.369842+0000","last_peered":"2024-06-10T01:02:37.369842+0000","last_clean":"2024-06-10T01:02:37.369842+0000","last_became_active":"2024-06-10T01:01:47.254542+0000","last_became_peered":"2024-06-10T01:01:47.254542+0000","last_unstale":"2024-06-10T01:02:37.369842+0000","last_undegraded":"2024-06-10T01:02:37.369842+0000","last_fullsized":"2024-06-10T01:02:37.369842+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:02:28.393133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.369100+0000","last_change":"2024-06-10T01:01:47.242596+0000","last_active":"2024-06-10T01:02:37.369100+0000","last_peered":"2024-06-10T01:02:37.369100+0000","last_clean":"2024-06-10T01:02:37.369100+0000","last_became_active":"2024-06-10T01:01:47.242308+0000","last_became_peered":"2024-06-10T01:01:47.242308+0000","last_unstale":"2024-06-10T01:02:37.369100+0000","last_undegraded":"2024-06-10T01:02:37.369100+0000","last_fullsized":"2024-06-10T01:02:37.369100+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:11:26.599954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.367730+0000","last_change":"2024-06-10T01:01:47.249085+0000","last_active":"2024-06-10T01:02:37.367730+0000","last_peered":"2024-06-10T01:02:37.367730+0000","last_clean":"2024-06-10T01:02:37.367730+0000","last_became_active":"2024-06-10T01:01:47.248749+0000","last_became_peered":"2024-06-10T01:01:47.248749+0000","last_unstale":"2024-06-10T01:02:37.367730+0000","last_undegraded":"2024-06-10T01:02:37.367730+0000","last_fullsized":"2024-06-10T01:02:37.367730+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:42:48.734566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.974291+0000","last_change":"2024-06-10T01:01:47.247334+0000","last_active":"2024-06-10T01:02:37.974291+0000","last_peered":"2024-06-10T01:02:37.974291+0000","last_clean":"2024-06-10T01:02:37.974291+0000","last_became_active":"2024-06-10T01:01:47.246964+0000","last_became_peered":"2024-06-10T01:01:47.246964+0000","last_unstale":"2024-06-10T01:02:37.974291+0000","last_undegraded":"2024-06-10T01:02:37.974291+0000","last_fullsized":"2024-06-10T01:02:37.974291+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:05:36.034950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.590922+0000","last_change":"2024-06-10T01:01:47.242523+0000","last_active":"2024-06-10T01:02:37.590922+0000","last_peered":"2024-06-10T01:02:37.590922+0000","last_clean":"2024-06-10T01:02:37.590922+0000","last_became_active":"2024-06-10T01:01:47.242198+0000","last_became_peered":"2024-06-10T01:01:47.242198+0000","last_unstale":"2024-06-10T01:02:37.590922+0000","last_undegraded":"2024-06-10T01:02:37.590922+0000","last_fullsized":"2024-06-10T01:02:37.590922+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:39:07.055970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.793070+0000","last_change":"2024-06-10T01:01:47.245277+0000","last_active":"2024-06-10T01:02:37.793070+0000","last_peered":"2024-06-10T01:02:37.793070+0000","last_clean":"2024-06-10T01:02:37.793070+0000","last_became_active":"2024-06-10T01:01:47.244932+0000","last_became_peered":"2024-06-10T01:01:47.244932+0000","last_unstale":"2024-06-10T01:02:37.793070+0000","last_undegraded":"2024-06-10T01:02:37.793070+0000","last_fullsized":"2024-06-10T01:02:37.793070+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:56:24.423713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.367630+0000","last_change":"2024-06-10T01:01:47.255531+0000","last_active":"2024-06-10T01:02:37.367630+0000","last_peered":"2024-06-10T01:02:37.367630+0000","last_clean":"2024-06-10T01:02:37.367630+0000","last_became_active":"2024-06-10T01:01:47.255150+0000","last_became_peered":"2024-06-10T01:01:47.255150+0000","last_unstale":"2024-06-10T01:02:37.367630+0000","last_undegraded":"2024-06-10T01:02:37.367630+0000","last_fullsized":"2024-06-10T01:02:37.367630+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:51:34.561396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.974825+0000","last_change":"2024-06-10T01:01:47.249832+0000","last_active":"2024-06-10T01:02:37.974825+0000","last_peered":"2024-06-10T01:02:37.974825+0000","last_clean":"2024-06-10T01:02:37.974825+0000","last_became_active":"2024-06-10T01:01:47.249303+0000","last_became_peered":"2024-06-10T01:01:47.249303+0000","last_unstale":"2024-06-10T01:02:37.974825+0000","last_undegraded":"2024-06-10T01:02:37.974825+0000","last_fullsized":"2024-06-10T01:02:37.974825+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:15:21.930157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.369968+0000","last_change":"2024-06-10T01:01:47.255927+0000","last_active":"2024-06-10T01:02:37.369968+0000","last_peered":"2024-06-10T01:02:37.369968+0000","last_clean":"2024-06-10T01:02:37.369968+0000","last_became_active":"2024-06-10T01:01:47.255591+0000","last_became_peered":"2024-06-10T01:01:47.255591+0000","last_unstale":"2024-06-10T01:02:37.369968+0000","last_undegraded":"2024-06-10T01:02:37.369968+0000","last_fullsized":"2024-06-10T01:02:37.369968+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:48:06.494430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.372069+0000","last_change":"2024-06-10T01:02:37.372069+0000","last_active":"2024-06-10T01:02:37.372069+0000","last_peered":"2024-06-10T01:02:37.372069+0000","last_clean":"2024-06-10T01:02:37.372069+0000","last_became_active":"2024-06-10T01:02:37.371599+0000","last_became_peered":"2024-06-10T01:02:37.371599+0000","last_unstale":"2024-06-10T01:02:37.372069+0000","last_undegraded":"2024-06-10T01:02:37.372069+0000","last_fullsized":"2024-06-10T01:02:37.372069+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:54:34.091576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.372531+0000","last_change":"2024-06-10T01:01:47.249553+0000","last_active":"2024-06-10T01:02:37.372531+0000","last_peered":"2024-06-10T01:02:37.372531+0000","last_clean":"2024-06-10T01:02:37.372531+0000","last_became_active":"2024-06-10T01:01:47.249202+0000","last_became_peered":"2024-06-10T01:01:47.249202+0000","last_unstale":"2024-06-10T01:02:37.372531+0000","last_undegraded":"2024-06-10T01:02:37.372531+0000","last_fullsized":"2024-06-10T01:02:37.372531+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:23:57.882264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.372324+0000","last_change":"2024-06-10T01:01:47.253987+0000","last_active":"2024-06-10T01:02:37.372324+0000","last_peered":"2024-06-10T01:02:37.372324+0000","last_clean":"2024-06-10T01:02:37.372324+0000","last_became_active":"2024-06-10T01:01:47.253578+0000","last_became_peered":"2024-06-10T01:01:47.253578+0000","last_unstale":"2024-06-10T01:02:37.372324+0000","last_undegraded":"2024-06-10T01:02:37.372324+0000","last_fullsized":"2024-06-10T01:02:37.372324+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:35:30.585482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.792280+0000","last_change":"2024-06-10T01:01:47.247089+0000","last_active":"2024-06-10T01:02:37.792280+0000","last_peered":"2024-06-10T01:02:37.792280+0000","last_clean":"2024-06-10T01:02:37.792280+0000","last_became_active":"2024-06-10T01:01:47.246066+0000","last_became_peered":"2024-06-10T01:01:47.246066+0000","last_unstale":"2024-06-10T01:02:37.792280+0000","last_undegraded":"2024-06-10T01:02:37.792280+0000","last_fullsized":"2024-06-10T01:02:37.792280+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:58:12.092981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.375425+0000","last_change":"2024-06-10T01:01:47.244491+0000","last_active":"2024-06-10T01:02:37.375425+0000","last_peered":"2024-06-10T01:02:37.375425+0000","last_clean":"2024-06-10T01:02:37.375425+0000","last_became_active":"2024-06-10T01:01:47.243747+0000","last_became_peered":"2024-06-10T01:01:47.243747+0000","last_unstale":"2024-06-10T01:02:37.375425+0000","last_undegraded":"2024-06-10T01:02:37.375425+0000","last_fullsized":"2024-06-10T01:02:37.375425+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:55:56.785016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.367139+0000","last_change":"2024-06-10T01:01:47.243141+0000","last_active":"2024-06-10T01:02:37.367139+0000","last_peered":"2024-06-10T01:02:37.367139+0000","last_clean":"2024-06-10T01:02:37.367139+0000","last_became_active":"2024-06-10T01:01:47.242464+0000","last_became_peered":"2024-06-10T01:01:47.242464+0000","last_unstale":"2024-06-10T01:02:37.367139+0000","last_undegraded":"2024-06-10T01:02:37.367139+0000","last_fullsized":"2024-06-10T01:02:37.367139+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:00:49.777130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.368037+0000","last_change":"2024-06-10T01:01:47.253463+0000","last_active":"2024-06-10T01:02:37.368037+0000","last_peered":"2024-06-10T01:02:37.368037+0000","last_clean":"2024-06-10T01:02:37.368037+0000","last_became_active":"2024-06-10T01:01:47.252929+0000","last_became_peered":"2024-06-10T01:01:47.252929+0000","last_unstale":"2024-06-10T01:02:37.368037+0000","last_undegraded":"2024-06-10T01:02:37.368037+0000","last_fullsized":"2024-06-10T01:02:37.368037+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:18:49.551276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.367213+0000","last_change":"2024-06-10T01:01:57.279561+0000","last_active":"2024-06-10T01:02:37.367213+0000","last_peered":"2024-06-10T01:02:37.367213+0000","last_clean":"2024-06-10T01:02:37.367213+0000","last_became_active":"2024-06-10T01:01:57.279132+0000","last_became_peered":"2024-06-10T01:01:57.279132+0000","last_unstale":"2024-06-10T01:02:37.367213+0000","last_undegraded":"2024-06-10T01:02:37.367213+0000","last_fullsized":"2024-06-10T01:02:37.367213+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:55:03.497701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'9","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:53.712475+0000","last_change":"2024-06-10T01:01:44.006667+0000","last_active":"2024-06-10T01:02:53.712475+0000","last_peered":"2024-06-10T01:02:53.712475+0000","last_clean":"2024-06-10T01:02:53.712475+0000","last_became_active":"2024-06-10T01:01:44.006343+0000","last_became_peered":"2024-06-10T01:01:44.006343+0000","last_unstale":"2024-06-10T01:02:53.712475+0000","last_undegraded":"2024-06-10T01:02:53.712475+0000","last_fullsized":"2024-06-10T01:02:53.712475+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:14:50.540084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.591617+0000","last_change":"2024-06-10T01:01:47.246139+0000","last_active":"2024-06-10T01:02:37.591617+0000","last_peered":"2024-06-10T01:02:37.591617+0000","last_clean":"2024-06-10T01:02:37.591617+0000","last_became_active":"2024-06-10T01:01:47.245743+0000","last_became_peered":"2024-06-10T01:01:47.245743+0000","last_unstale":"2024-06-10T01:02:37.591617+0000","last_undegraded":"2024-06-10T01:02:37.591617+0000","last_fullsized":"2024-06-10T01:02:37.591617+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:06:39.041569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.368964+0000","last_change":"2024-06-10T01:01:57.282843+0000","last_active":"2024-06-10T01:02:37.368964+0000","last_peered":"2024-06-10T01:02:37.368964+0000","last_clean":"2024-06-10T01:02:37.368964+0000","last_became_active":"2024-06-10T01:01:57.281797+0000","last_became_peered":"2024-06-10T01:01:57.281797+0000","last_unstale":"2024-06-10T01:02:37.368964+0000","last_undegraded":"2024-06-10T01:02:37.368964+0000","last_fullsized":"2024-06-10T01:02:37.368964+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:00.209628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.592003+0000","last_change":"2024-06-10T01:01:44.466714+0000","last_active":"2024-06-10T01:02:37.592003+0000","last_peered":"2024-06-10T01:02:37.592003+0000","last_clean":"2024-06-10T01:02:37.592003+0000","last_became_active":"2024-06-10T01:01:44.466296+0000","last_became_peered":"2024-06-10T01:01:44.466296+0000","last_unstale":"2024-06-10T01:02:37.592003+0000","last_undegraded":"2024-06-10T01:02:37.592003+0000","last_fullsized":"2024-06-10T01:02:37.592003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:16:36.082308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.373144+0000","last_change":"2024-06-10T01:01:47.244173+0000","last_active":"2024-06-10T01:02:37.373144+0000","last_peered":"2024-06-10T01:02:37.373144+0000","last_clean":"2024-06-10T01:02:37.373144+0000","last_became_active":"2024-06-10T01:01:47.243032+0000","last_became_peered":"2024-06-10T01:01:47.243032+0000","last_unstale":"2024-06-10T01:02:37.373144+0000","last_undegraded":"2024-06-10T01:02:37.373144+0000","last_fullsized":"2024-06-10T01:02:37.373144+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:14:33.965903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.592535+0000","last_change":"2024-06-10T01:02:37.592535+0000","last_active":"2024-06-10T01:02:37.592535+0000","last_peered":"2024-06-10T01:02:37.592535+0000","last_clean":"2024-06-10T01:02:37.592535+0000","last_became_active":"2024-06-10T01:02:37.592159+0000","last_became_peered":"2024-06-10T01:02:37.592159+0000","last_unstale":"2024-06-10T01:02:37.592535+0000","last_undegraded":"2024-06-10T01:02:37.592535+0000","last_fullsized":"2024-06-10T01:02:37.592535+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:50:08.626485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'4","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:53.709969+0000","last_change":"2024-06-10T01:01:44.009105+0000","last_active":"2024-06-10T01:02:53.709969+0000","last_peered":"2024-06-10T01:02:53.709969+0000","last_clean":"2024-06-10T01:02:53.709969+0000","last_became_active":"2024-06-10T01:01:44.008743+0000","last_became_peered":"2024-06-10T01:01:44.008743+0000","last_unstale":"2024-06-10T01:02:53.709969+0000","last_undegraded":"2024-06-10T01:02:53.709969+0000","last_fullsized":"2024-06-10T01:02:53.709969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:44:54.301378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.792728+0000","last_change":"2024-06-10T01:01:47.249007+0000","last_active":"2024-06-10T01:02:37.792728+0000","last_peered":"2024-06-10T01:02:37.792728+0000","last_clean":"2024-06-10T01:02:37.792728+0000","last_became_active":"2024-06-10T01:01:47.245632+0000","last_became_peered":"2024-06-10T01:01:47.245632+0000","last_unstale":"2024-06-10T01:02:37.792728+0000","last_undegraded":"2024-06-10T01:02:37.792728+0000","last_fullsized":"2024-06-10T01:02:37.792728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:29:18.916390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.368888+0000","last_change":"2024-06-10T01:01:57.283137+0000","last_active":"2024-06-10T01:02:37.368888+0000","last_peered":"2024-06-10T01:02:37.368888+0000","last_clean":"2024-06-10T01:02:37.368888+0000","last_became_active":"2024-06-10T01:01:57.282605+0000","last_became_peered":"2024-06-10T01:01:57.282605+0000","last_unstale":"2024-06-10T01:02:37.368888+0000","last_undegraded":"2024-06-10T01:02:37.368888+0000","last_fullsized":"2024-06-10T01:02:37.368888+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:26:16.967320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.369181+0000","last_change":"2024-06-10T01:01:44.010674+0000","last_active":"2024-06-10T01:02:37.369181+0000","last_peered":"2024-06-10T01:02:37.369181+0000","last_clean":"2024-06-10T01:02:37.369181+0000","last_became_active":"2024-06-10T01:01:44.010291+0000","last_became_peered":"2024-06-10T01:01:44.010291+0000","last_unstale":"2024-06-10T01:02:37.369181+0000","last_undegraded":"2024-06-10T01:02:37.369181+0000","last_fullsized":"2024-06-10T01:02:37.369181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:36:18.418362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.794580+0000","last_change":"2024-06-10T01:02:37.794580+0000","last_active":"2024-06-10T01:02:37.794580+0000","last_peered":"2024-06-10T01:02:37.794580+0000","last_clean":"2024-06-10T01:02:37.794580+0000","last_became_active":"2024-06-10T01:02:37.794127+0000","last_became_peered":"2024-06-10T01:02:37.794127+0000","last_unstale":"2024-06-10T01:02:37.794580+0000","last_undegraded":"2024-06-10T01:02:37.794580+0000","last_fullsized":"2024-06-10T01:02:37.794580+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:02.117871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.367911+0000","last_change":"2024-06-10T01:01:57.282494+0000","last_active":"2024-06-10T01:02:37.367911+0000","last_peered":"2024-06-10T01:02:37.367911+0000","last_clean":"2024-06-10T01:02:37.367911+0000","last_became_active":"2024-06-10T01:01:57.282077+0000","last_became_peered":"2024-06-10T01:01:57.282077+0000","last_unstale":"2024-06-10T01:02:37.367911+0000","last_undegraded":"2024-06-10T01:02:37.367911+0000","last_fullsized":"2024-06-10T01:02:37.367911+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:57:41.639788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"97'5","reported_seq":69,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:03:15.747316+0000","last_change":"2024-06-10T01:01:44.011134+0000","last_active":"2024-06-10T01:03:15.747316+0000","last_peered":"2024-06-10T01:03:15.747316+0000","last_clean":"2024-06-10T01:03:15.747316+0000","last_became_active":"2024-06-10T01:01:44.010788+0000","last_became_peered":"2024-06-10T01:01:44.010788+0000","last_unstale":"2024-06-10T01:03:15.747316+0000","last_undegraded":"2024-06-10T01:03:15.747316+0000","last_fullsized":"2024-06-10T01:03:15.747316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:22:55.228946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.369512+0000","last_change":"2024-06-10T01:01:47.248093+0000","last_active":"2024-06-10T01:02:37.369512+0000","last_peered":"2024-06-10T01:02:37.369512+0000","last_clean":"2024-06-10T01:02:37.369512+0000","last_became_active":"2024-06-10T01:01:47.247797+0000","last_became_peered":"2024-06-10T01:01:47.247797+0000","last_unstale":"2024-06-10T01:02:37.369512+0000","last_undegraded":"2024-06-10T01:02:37.369512+0000","last_fullsized":"2024-06-10T01:02:37.369512+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:45:50.159128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.377039+0000","last_change":"2024-06-10T01:02:37.377039+0000","last_active":"2024-06-10T01:02:37.377039+0000","last_peered":"2024-06-10T01:02:37.377039+0000","last_clean":"2024-06-10T01:02:37.377039+0000","last_became_active":"2024-06-10T01:02:37.376332+0000","last_became_peered":"2024-06-10T01:02:37.376332+0000","last_unstale":"2024-06-10T01:02:37.377039+0000","last_undegraded":"2024-06-10T01:02:37.377039+0000","last_fullsized":"2024-06-10T01:02:37.377039+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:15.000505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,10,5],"acting":[2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'3","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:53.710218+0000","last_change":"2024-06-10T01:01:44.008342+0000","last_active":"2024-06-10T01:02:53.710218+0000","last_peered":"2024-06-10T01:02:53.710218+0000","last_clean":"2024-06-10T01:02:53.710218+0000","last_became_active":"2024-06-10T01:01:44.007959+0000","last_became_peered":"2024-06-10T01:01:44.007959+0000","last_unstale":"2024-06-10T01:02:53.710218+0000","last_undegraded":"2024-06-10T01:02:53.710218+0000","last_fullsized":"2024-06-10T01:02:53.710218+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:36:03.143609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.372957+0000","last_change":"2024-06-10T01:01:47.247854+0000","last_active":"2024-06-10T01:02:37.372957+0000","last_peered":"2024-06-10T01:02:37.372957+0000","last_clean":"2024-06-10T01:02:37.372957+0000","last_became_active":"2024-06-10T01:01:47.247563+0000","last_became_peered":"2024-06-10T01:01:47.247563+0000","last_unstale":"2024-06-10T01:02:37.372957+0000","last_undegraded":"2024-06-10T01:02:37.372957+0000","last_fullsized":"2024-06-10T01:02:37.372957+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:32:13.024815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.375326+0000","last_change":"2024-06-10T01:01:57.282427+0000","last_active":"2024-06-10T01:02:37.375326+0000","last_peered":"2024-06-10T01:02:37.375326+0000","last_clean":"2024-06-10T01:02:37.375326+0000","last_became_active":"2024-06-10T01:01:57.281987+0000","last_became_peered":"2024-06-10T01:01:57.281987+0000","last_unstale":"2024-06-10T01:02:37.375326+0000","last_undegraded":"2024-06-10T01:02:37.375326+0000","last_fullsized":"2024-06-10T01:02:37.375326+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:22:31.736804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:53.709620+0000","last_change":"2024-06-10T01:01:44.006567+0000","last_active":"2024-06-10T01:02:53.709620+0000","last_peered":"2024-06-10T01:02:53.709620+0000","last_clean":"2024-06-10T01:02:53.709620+0000","last_became_active":"2024-06-10T01:01:44.006216+0000","last_became_peered":"2024-06-10T01:01:44.006216+0000","last_unstale":"2024-06-10T01:02:53.709620+0000","last_undegraded":"2024-06-10T01:02:53.709620+0000","last_fullsized":"2024-06-10T01:02:53.709620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:33:55.696873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.590667+0000","last_change":"2024-06-10T01:01:47.245326+0000","last_active":"2024-06-10T01:02:37.590667+0000","last_peered":"2024-06-10T01:02:37.590667+0000","last_clean":"2024-06-10T01:02:37.590667+0000","last_became_active":"2024-06-10T01:01:47.245013+0000","last_became_peered":"2024-06-10T01:01:47.245013+0000","last_unstale":"2024-06-10T01:02:37.590667+0000","last_undegraded":"2024-06-10T01:02:37.590667+0000","last_fullsized":"2024-06-10T01:02:37.590667+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:17:56.416627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.373215+0000","last_change":"2024-06-10T01:01:57.288997+0000","last_active":"2024-06-10T01:02:37.373215+0000","last_peered":"2024-06-10T01:02:37.373215+0000","last_clean":"2024-06-10T01:02:37.373215+0000","last_became_active":"2024-06-10T01:01:57.288596+0000","last_became_peered":"2024-06-10T01:01:57.288596+0000","last_unstale":"2024-06-10T01:02:37.373215+0000","last_undegraded":"2024-06-10T01:02:37.373215+0000","last_fullsized":"2024-06-10T01:02:37.373215+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:08:14.739737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.372286+0000","last_change":"2024-06-10T01:01:44.010079+0000","last_active":"2024-06-10T01:02:37.372286+0000","last_peered":"2024-06-10T01:02:37.372286+0000","last_clean":"2024-06-10T01:02:37.372286+0000","last_became_active":"2024-06-10T01:01:44.009752+0000","last_became_peered":"2024-06-10T01:01:44.009752+0000","last_unstale":"2024-06-10T01:02:37.372286+0000","last_undegraded":"2024-06-10T01:02:37.372286+0000","last_fullsized":"2024-06-10T01:02:37.372286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:29:35.660935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.366907+0000","last_change":"2024-06-10T01:01:47.245015+0000","last_active":"2024-06-10T01:02:37.366907+0000","last_peered":"2024-06-10T01:02:37.366907+0000","last_clean":"2024-06-10T01:02:37.366907+0000","last_became_active":"2024-06-10T01:01:47.244595+0000","last_became_peered":"2024-06-10T01:01:47.244595+0000","last_unstale":"2024-06-10T01:02:37.366907+0000","last_undegraded":"2024-06-10T01:02:37.366907+0000","last_fullsized":"2024-06-10T01:02:37.366907+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:01:48.446526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.591036+0000","last_change":"2024-06-10T01:01:57.281656+0000","last_active":"2024-06-10T01:02:37.591036+0000","last_peered":"2024-06-10T01:02:37.591036+0000","last_clean":"2024-06-10T01:02:37.591036+0000","last_became_active":"2024-06-10T01:01:57.281295+0000","last_became_peered":"2024-06-10T01:01:57.281295+0000","last_unstale":"2024-06-10T01:02:37.591036+0000","last_undegraded":"2024-06-10T01:02:37.591036+0000","last_fullsized":"2024-06-10T01:02:37.591036+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:04:41.619864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"97'3","reported_seq":64,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:03:14.745207+0000","last_change":"2024-06-10T01:01:44.011941+0000","last_active":"2024-06-10T01:03:14.745207+0000","last_peered":"2024-06-10T01:03:14.745207+0000","last_clean":"2024-06-10T01:03:14.745207+0000","last_became_active":"2024-06-10T01:01:44.010156+0000","last_became_peered":"2024-06-10T01:01:44.010156+0000","last_unstale":"2024-06-10T01:03:14.745207+0000","last_undegraded":"2024-06-10T01:03:14.745207+0000","last_fullsized":"2024-06-10T01:03:14.745207+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:16:15.709544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.793074+0000","last_change":"2024-06-10T01:01:47.241720+0000","last_active":"2024-06-10T01:02:37.793074+0000","last_peered":"2024-06-10T01:02:37.793074+0000","last_clean":"2024-06-10T01:02:37.793074+0000","last_became_active":"2024-06-10T01:01:47.241386+0000","last_became_peered":"2024-06-10T01:01:47.241386+0000","last_unstale":"2024-06-10T01:02:37.793074+0000","last_undegraded":"2024-06-10T01:02:37.793074+0000","last_fullsized":"2024-06-10T01:02:37.793074+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:42:01.304787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.366174+0000","last_change":"2024-06-10T01:01:57.283226+0000","last_active":"2024-06-10T01:02:37.366174+0000","last_peered":"2024-06-10T01:02:37.366174+0000","last_clean":"2024-06-10T01:02:37.366174+0000","last_became_active":"2024-06-10T01:01:57.282720+0000","last_became_peered":"2024-06-10T01:01:57.282720+0000","last_unstale":"2024-06-10T01:02:37.366174+0000","last_undegraded":"2024-06-10T01:02:37.366174+0000","last_fullsized":"2024-06-10T01:02:37.366174+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:35:31.342042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:52.695045+0000","last_change":"2024-06-10T01:01:44.009278+0000","last_active":"2024-06-10T01:02:52.695045+0000","last_peered":"2024-06-10T01:02:52.695045+0000","last_clean":"2024-06-10T01:02:52.695045+0000","last_became_active":"2024-06-10T01:01:44.008841+0000","last_became_peered":"2024-06-10T01:01:44.008841+0000","last_unstale":"2024-06-10T01:02:52.695045+0000","last_undegraded":"2024-06-10T01:02:52.695045+0000","last_fullsized":"2024-06-10T01:02:52.695045+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:30:58.392080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.373314+0000","last_change":"2024-06-10T01:01:47.256451+0000","last_active":"2024-06-10T01:02:37.373314+0000","last_peered":"2024-06-10T01:02:37.373314+0000","last_clean":"2024-06-10T01:02:37.373314+0000","last_became_active":"2024-06-10T01:01:47.256036+0000","last_became_peered":"2024-06-10T01:01:47.256036+0000","last_unstale":"2024-06-10T01:02:37.373314+0000","last_undegraded":"2024-06-10T01:02:37.373314+0000","last_fullsized":"2024-06-10T01:02:37.373314+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:57:26.640109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.794160+0000","last_change":"2024-06-10T01:02:37.794160+0000","last_active":"2024-06-10T01:02:37.794160+0000","last_peered":"2024-06-10T01:02:37.794160+0000","last_clean":"2024-06-10T01:02:37.794160+0000","last_became_active":"2024-06-10T01:02:37.793633+0000","last_became_peered":"2024-06-10T01:02:37.793633+0000","last_unstale":"2024-06-10T01:02:37.794160+0000","last_undegraded":"2024-06-10T01:02:37.794160+0000","last_fullsized":"2024-06-10T01:02:37.794160+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:48:48.083620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:52.694810+0000","last_change":"2024-06-10T01:01:44.469179+0000","last_active":"2024-06-10T01:02:52.694810+0000","last_peered":"2024-06-10T01:02:52.694810+0000","last_clean":"2024-06-10T01:02:52.694810+0000","last_became_active":"2024-06-10T01:01:44.468669+0000","last_became_peered":"2024-06-10T01:01:44.468669+0000","last_unstale":"2024-06-10T01:02:52.694810+0000","last_undegraded":"2024-06-10T01:02:52.694810+0000","last_fullsized":"2024-06-10T01:02:52.694810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:07:51.629030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.792660+0000","last_change":"2024-06-10T01:01:47.247588+0000","last_active":"2024-06-10T01:02:37.792660+0000","last_peered":"2024-06-10T01:02:37.792660+0000","last_clean":"2024-06-10T01:02:37.792660+0000","last_became_active":"2024-06-10T01:01:47.245750+0000","last_became_peered":"2024-06-10T01:01:47.245750+0000","last_unstale":"2024-06-10T01:02:37.792660+0000","last_undegraded":"2024-06-10T01:02:37.792660+0000","last_fullsized":"2024-06-10T01:02:37.792660+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:12:10.513777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.366245+0000","last_change":"2024-06-10T01:01:57.283042+0000","last_active":"2024-06-10T01:02:37.366245+0000","last_peered":"2024-06-10T01:02:37.366245+0000","last_clean":"2024-06-10T01:02:37.366245+0000","last_became_active":"2024-06-10T01:01:57.282600+0000","last_became_peered":"2024-06-10T01:01:57.282600+0000","last_unstale":"2024-06-10T01:02:37.366245+0000","last_undegraded":"2024-06-10T01:02:37.366245+0000","last_fullsized":"2024-06-10T01:02:37.366245+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:06:36.969968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"97'30","reported_seq":119,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:03:15.747487+0000","last_change":"2024-06-10T01:01:44.011783+0000","last_active":"2024-06-10T01:03:15.747487+0000","last_peered":"2024-06-10T01:03:15.747487+0000","last_clean":"2024-06-10T01:03:15.747487+0000","last_became_active":"2024-06-10T01:01:44.011164+0000","last_became_peered":"2024-06-10T01:01:44.011164+0000","last_unstale":"2024-06-10T01:03:15.747487+0000","last_undegraded":"2024-06-10T01:03:15.747487+0000","last_fullsized":"2024-06-10T01:03:15.747487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:16:13.642714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54975,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":78,"num_write":31,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.369707+0000","last_change":"2024-06-10T01:01:47.250751+0000","last_active":"2024-06-10T01:02:37.369707+0000","last_peered":"2024-06-10T01:02:37.369707+0000","last_clean":"2024-06-10T01:02:37.369707+0000","last_became_active":"2024-06-10T01:01:47.250451+0000","last_became_peered":"2024-06-10T01:01:47.250451+0000","last_unstale":"2024-06-10T01:02:37.369707+0000","last_undegraded":"2024-06-10T01:02:37.369707+0000","last_fullsized":"2024-06-10T01:02:37.369707+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:22:10.121363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.374459+0000","last_change":"2024-06-10T01:01:57.289687+0000","last_active":"2024-06-10T01:02:37.374459+0000","last_peered":"2024-06-10T01:02:37.374459+0000","last_clean":"2024-06-10T01:02:37.374459+0000","last_became_active":"2024-06-10T01:01:57.289274+0000","last_became_peered":"2024-06-10T01:01:57.289274+0000","last_unstale":"2024-06-10T01:02:37.374459+0000","last_undegraded":"2024-06-10T01:02:37.374459+0000","last_fullsized":"2024-06-10T01:02:37.374459+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:34:46.873974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'6","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:53.710812+0000","last_change":"2024-06-10T01:01:44.010159+0000","last_active":"2024-06-10T01:02:53.710812+0000","last_peered":"2024-06-10T01:02:53.710812+0000","last_clean":"2024-06-10T01:02:53.710812+0000","last_became_active":"2024-06-10T01:01:44.009744+0000","last_became_peered":"2024-06-10T01:01:44.009744+0000","last_unstale":"2024-06-10T01:02:53.710812+0000","last_undegraded":"2024-06-10T01:02:53.710812+0000","last_fullsized":"2024-06-10T01:02:53.710812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:30.507613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.369580+0000","last_change":"2024-06-10T01:01:47.246662+0000","last_active":"2024-06-10T01:02:37.369580+0000","last_peered":"2024-06-10T01:02:37.369580+0000","last_clean":"2024-06-10T01:02:37.369580+0000","last_became_active":"2024-06-10T01:01:47.246268+0000","last_became_peered":"2024-06-10T01:01:47.246268+0000","last_unstale":"2024-06-10T01:02:37.369580+0000","last_undegraded":"2024-06-10T01:02:37.369580+0000","last_fullsized":"2024-06-10T01:02:37.369580+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:08:38.089054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.374428+0000","last_change":"2024-06-10T01:01:57.289937+0000","last_active":"2024-06-10T01:02:37.374428+0000","last_peered":"2024-06-10T01:02:37.374428+0000","last_clean":"2024-06-10T01:02:37.374428+0000","last_became_active":"2024-06-10T01:01:57.289407+0000","last_became_peered":"2024-06-10T01:01:57.289407+0000","last_unstale":"2024-06-10T01:02:37.374428+0000","last_undegraded":"2024-06-10T01:02:37.374428+0000","last_fullsized":"2024-06-10T01:02:37.374428+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:01:48.546762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:52.695496+0000","last_change":"2024-06-10T01:01:44.012654+0000","last_active":"2024-06-10T01:02:52.695496+0000","last_peered":"2024-06-10T01:02:52.695496+0000","last_clean":"2024-06-10T01:02:52.695496+0000","last_became_active":"2024-06-10T01:01:44.012246+0000","last_became_peered":"2024-06-10T01:01:44.012246+0000","last_unstale":"2024-06-10T01:02:52.695496+0000","last_undegraded":"2024-06-10T01:02:52.695496+0000","last_fullsized":"2024-06-10T01:02:52.695496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:31:36.173554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.974442+0000","last_change":"2024-06-10T01:01:47.250514+0000","last_active":"2024-06-10T01:02:37.974442+0000","last_peered":"2024-06-10T01:02:37.974442+0000","last_clean":"2024-06-10T01:02:37.974442+0000","last_became_active":"2024-06-10T01:01:47.249050+0000","last_became_peered":"2024-06-10T01:01:47.249050+0000","last_unstale":"2024-06-10T01:02:37.974442+0000","last_undegraded":"2024-06-10T01:02:37.974442+0000","last_fullsized":"2024-06-10T01:02:37.974442+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:51:30.789392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.367105+0000","last_change":"2024-06-10T01:01:57.282389+0000","last_active":"2024-06-10T01:02:37.367105+0000","last_peered":"2024-06-10T01:02:37.367105+0000","last_clean":"2024-06-10T01:02:37.367105+0000","last_became_active":"2024-06-10T01:01:57.281857+0000","last_became_peered":"2024-06-10T01:01:57.281857+0000","last_unstale":"2024-06-10T01:02:37.367105+0000","last_undegraded":"2024-06-10T01:02:37.367105+0000","last_fullsized":"2024-06-10T01:02:37.367105+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:59:14.863020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"97'3","reported_seq":68,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:03:16.749033+0000","last_change":"2024-06-10T01:01:44.011331+0000","last_active":"2024-06-10T01:03:16.749033+0000","last_peered":"2024-06-10T01:03:16.749033+0000","last_clean":"2024-06-10T01:03:16.749033+0000","last_became_active":"2024-06-10T01:01:44.010968+0000","last_became_peered":"2024-06-10T01:01:44.010968+0000","last_unstale":"2024-06-10T01:03:16.749033+0000","last_undegraded":"2024-06-10T01:03:16.749033+0000","last_fullsized":"2024-06-10T01:03:16.749033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:43:37.931225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.974460+0000","last_change":"2024-06-10T01:01:47.248349+0000","last_active":"2024-06-10T01:02:37.974460+0000","last_peered":"2024-06-10T01:02:37.974460+0000","last_clean":"2024-06-10T01:02:37.974460+0000","last_became_active":"2024-06-10T01:01:47.248008+0000","last_became_peered":"2024-06-10T01:01:47.248008+0000","last_unstale":"2024-06-10T01:02:37.974460+0000","last_undegraded":"2024-06-10T01:02:37.974460+0000","last_fullsized":"2024-06-10T01:02:37.974460+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:45:20.747704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.369929+0000","last_change":"2024-06-10T01:01:57.289773+0000","last_active":"2024-06-10T01:02:37.369929+0000","last_peered":"2024-06-10T01:02:37.369929+0000","last_clean":"2024-06-10T01:02:37.369929+0000","last_became_active":"2024-06-10T01:01:57.289371+0000","last_became_peered":"2024-06-10T01:01:57.289371+0000","last_unstale":"2024-06-10T01:02:37.369929+0000","last_undegraded":"2024-06-10T01:02:37.369929+0000","last_fullsized":"2024-06-10T01:02:37.369929+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:24:52.209402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'15","reported_seq":72,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:53.711765+0000","last_change":"2024-06-10T01:02:37.382175+0000","last_active":"2024-06-10T01:02:53.711765+0000","last_peered":"2024-06-10T01:02:53.711765+0000","last_clean":"2024-06-10T01:02:53.711765+0000","last_became_active":"2024-06-10T01:02:37.374506+0000","last_became_peered":"2024-06-10T01:02:37.374506+0000","last_unstale":"2024-06-10T01:02:53.711765+0000","last_undegraded":"2024-06-10T01:02:53.711765+0000","last_fullsized":"2024-06-10T01:02:53.711765+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:22:34.017650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.372221+0000","last_change":"2024-06-10T01:01:47.245439+0000","last_active":"2024-06-10T01:02:37.372221+0000","last_peered":"2024-06-10T01:02:37.372221+0000","last_clean":"2024-06-10T01:02:37.372221+0000","last_became_active":"2024-06-10T01:01:47.242917+0000","last_became_peered":"2024-06-10T01:01:47.242917+0000","last_unstale":"2024-06-10T01:02:37.372221+0000","last_undegraded":"2024-06-10T01:02:37.372221+0000","last_fullsized":"2024-06-10T01:02:37.372221+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:13:12.614230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.372096+0000","last_change":"2024-06-10T01:01:57.284731+0000","last_active":"2024-06-10T01:02:37.372096+0000","last_peered":"2024-06-10T01:02:37.372096+0000","last_clean":"2024-06-10T01:02:37.372096+0000","last_became_active":"2024-06-10T01:01:57.284301+0000","last_became_peered":"2024-06-10T01:01:57.284301+0000","last_unstale":"2024-06-10T01:02:37.372096+0000","last_undegraded":"2024-06-10T01:02:37.372096+0000","last_fullsized":"2024-06-10T01:02:37.372096+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:01:36.518214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:54.707212+0000","last_change":"2024-06-10T01:01:44.010154+0000","last_active":"2024-06-10T01:02:54.707212+0000","last_peered":"2024-06-10T01:02:54.707212+0000","last_clean":"2024-06-10T01:02:54.707212+0000","last_became_active":"2024-06-10T01:01:44.009656+0000","last_became_peered":"2024-06-10T01:01:44.009656+0000","last_unstale":"2024-06-10T01:02:54.707212+0000","last_undegraded":"2024-06-10T01:02:54.707212+0000","last_fullsized":"2024-06-10T01:02:54.707212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:33:08.789946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.378484+0000","last_change":"2024-06-10T01:02:37.378484+0000","last_active":"2024-06-10T01:02:37.378484+0000","last_peered":"2024-06-10T01:02:37.378484+0000","last_clean":"2024-06-10T01:02:37.378484+0000","last_became_active":"2024-06-10T01:02:37.378092+0000","last_became_peered":"2024-06-10T01:02:37.378092+0000","last_unstale":"2024-06-10T01:02:37.378484+0000","last_undegraded":"2024-06-10T01:02:37.378484+0000","last_fullsized":"2024-06-10T01:02:37.378484+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:19:46.599235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.369406+0000","last_change":"2024-06-10T01:01:57.332887+0000","last_active":"2024-06-10T01:02:37.369406+0000","last_peered":"2024-06-10T01:02:37.369406+0000","last_clean":"2024-06-10T01:02:37.369406+0000","last_became_active":"2024-06-10T01:01:57.332285+0000","last_became_peered":"2024-06-10T01:01:57.332285+0000","last_unstale":"2024-06-10T01:02:37.369406+0000","last_undegraded":"2024-06-10T01:02:37.369406+0000","last_fullsized":"2024-06-10T01:02:37.369406+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:09:12.233405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.366543+0000","last_change":"2024-06-10T01:01:44.466831+0000","last_active":"2024-06-10T01:02:37.366543+0000","last_peered":"2024-06-10T01:02:37.366543+0000","last_clean":"2024-06-10T01:02:37.366543+0000","last_became_active":"2024-06-10T01:01:44.466484+0000","last_became_peered":"2024-06-10T01:01:44.466484+0000","last_unstale":"2024-06-10T01:02:37.366543+0000","last_undegraded":"2024-06-10T01:02:37.366543+0000","last_fullsized":"2024-06-10T01:02:37.366543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:19:28.385504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.367319+0000","last_change":"2024-06-10T01:01:47.252399+0000","last_active":"2024-06-10T01:02:37.367319+0000","last_peered":"2024-06-10T01:02:37.367319+0000","last_clean":"2024-06-10T01:02:37.367319+0000","last_became_active":"2024-06-10T01:01:47.252069+0000","last_became_peered":"2024-06-10T01:01:47.252069+0000","last_unstale":"2024-06-10T01:02:37.367319+0000","last_undegraded":"2024-06-10T01:02:37.367319+0000","last_fullsized":"2024-06-10T01:02:37.367319+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:02:02.291174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.369699+0000","last_change":"2024-06-10T01:01:57.285670+0000","last_active":"2024-06-10T01:02:37.369699+0000","last_peered":"2024-06-10T01:02:37.369699+0000","last_clean":"2024-06-10T01:02:37.369699+0000","last_became_active":"2024-06-10T01:01:57.285281+0000","last_became_peered":"2024-06-10T01:01:57.285281+0000","last_unstale":"2024-06-10T01:02:37.369699+0000","last_undegraded":"2024-06-10T01:02:37.369699+0000","last_fullsized":"2024-06-10T01:02:37.369699+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:58.956934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'1","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.974404+0000","last_change":"2024-06-10T01:01:44.008013+0000","last_active":"2024-06-10T01:02:37.974404+0000","last_peered":"2024-06-10T01:02:37.974404+0000","last_clean":"2024-06-10T01:02:37.974404+0000","last_became_active":"2024-06-10T01:01:44.007591+0000","last_became_peered":"2024-06-10T01:01:44.007591+0000","last_unstale":"2024-06-10T01:02:37.974404+0000","last_undegraded":"2024-06-10T01:02:37.974404+0000","last_fullsized":"2024-06-10T01:02:37.974404+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:18:20.862340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.367130+0000","last_change":"2024-06-10T01:01:47.254756+0000","last_active":"2024-06-10T01:02:37.367130+0000","last_peered":"2024-06-10T01:02:37.367130+0000","last_clean":"2024-06-10T01:02:37.367130+0000","last_became_active":"2024-06-10T01:01:47.254418+0000","last_became_peered":"2024-06-10T01:01:47.254418+0000","last_unstale":"2024-06-10T01:02:37.367130+0000","last_undegraded":"2024-06-10T01:02:37.367130+0000","last_fullsized":"2024-06-10T01:02:37.367130+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:46:31.452539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.370058+0000","last_change":"2024-06-10T01:01:57.332792+0000","last_active":"2024-06-10T01:02:37.370058+0000","last_peered":"2024-06-10T01:02:37.370058+0000","last_clean":"2024-06-10T01:02:37.370058+0000","last_became_active":"2024-06-10T01:01:57.332331+0000","last_became_peered":"2024-06-10T01:01:57.332331+0000","last_unstale":"2024-06-10T01:02:37.370058+0000","last_undegraded":"2024-06-10T01:02:37.370058+0000","last_fullsized":"2024-06-10T01:02:37.370058+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:46:41.419807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.372568+0000","last_change":"2024-06-10T01:01:47.247857+0000","last_active":"2024-06-10T01:02:37.372568+0000","last_peered":"2024-06-10T01:02:37.372568+0000","last_clean":"2024-06-10T01:02:37.372568+0000","last_became_active":"2024-06-10T01:01:47.247552+0000","last_became_peered":"2024-06-10T01:01:47.247552+0000","last_unstale":"2024-06-10T01:02:37.372568+0000","last_undegraded":"2024-06-10T01:02:37.372568+0000","last_fullsized":"2024-06-10T01:02:37.372568+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:29:40.521887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"97'4","reported_seq":68,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:03:17.749968+0000","last_change":"2024-06-10T01:01:44.466631+0000","last_active":"2024-06-10T01:03:17.749968+0000","last_peered":"2024-06-10T01:03:17.749968+0000","last_clean":"2024-06-10T01:03:17.749968+0000","last_became_active":"2024-06-10T01:01:44.466280+0000","last_became_peered":"2024-06-10T01:01:44.466280+0000","last_unstale":"2024-06-10T01:03:17.749968+0000","last_undegraded":"2024-06-10T01:03:17.749968+0000","last_fullsized":"2024-06-10T01:03:17.749968+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:09:33.658112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":25,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.974760+0000","last_change":"2024-06-10T01:01:57.281973+0000","last_active":"2024-06-10T01:02:37.974760+0000","last_peered":"2024-06-10T01:02:37.974760+0000","last_clean":"2024-06-10T01:02:37.974760+0000","last_became_active":"2024-06-10T01:01:57.281386+0000","last_became_peered":"2024-06-10T01:01:57.281386+0000","last_unstale":"2024-06-10T01:02:37.974760+0000","last_undegraded":"2024-06-10T01:02:37.974760+0000","last_fullsized":"2024-06-10T01:02:37.974760+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:10:28.236250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.591038+0000","last_change":"2024-06-10T01:01:47.237436+0000","last_active":"2024-06-10T01:02:37.591038+0000","last_peered":"2024-06-10T01:02:37.591038+0000","last_clean":"2024-06-10T01:02:37.591038+0000","last_became_active":"2024-06-10T01:01:47.237099+0000","last_became_peered":"2024-06-10T01:01:47.237099+0000","last_unstale":"2024-06-10T01:02:37.591038+0000","last_undegraded":"2024-06-10T01:02:37.591038+0000","last_fullsized":"2024-06-10T01:02:37.591038+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:34:53.838877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'3","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:55.715038+0000","last_change":"2024-06-10T01:01:44.010655+0000","last_active":"2024-06-10T01:02:55.715038+0000","last_peered":"2024-06-10T01:02:55.715038+0000","last_clean":"2024-06-10T01:02:55.715038+0000","last_became_active":"2024-06-10T01:01:44.009918+0000","last_became_peered":"2024-06-10T01:01:44.009918+0000","last_unstale":"2024-06-10T01:02:55.715038+0000","last_undegraded":"2024-06-10T01:02:55.715038+0000","last_fullsized":"2024-06-10T01:02:55.715038+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:47:32.052650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":362,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.974272+0000","last_change":"2024-06-10T01:00:23.822447+0000","last_active":"2024-06-10T01:02:37.974272+0000","last_peered":"2024-06-10T01:02:37.974272+0000","last_clean":"2024-06-10T01:02:37.974272+0000","last_became_active":"2024-06-10T01:00:23.821966+0000","last_became_peered":"2024-06-10T01:00:23.821966+0000","last_unstale":"2024-06-10T01:02:37.974272+0000","last_undegraded":"2024-06-10T01:02:37.974272+0000","last_fullsized":"2024-06-10T01:02:37.974272+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:56:59.964661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:56:59.964661+0000","last_clean_scrub_stamp":"2024-06-10T00:56:59.964661+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:03:43.732875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.373138+0000","last_change":"2024-06-10T01:01:57.332980+0000","last_active":"2024-06-10T01:02:37.373138+0000","last_peered":"2024-06-10T01:02:37.373138+0000","last_clean":"2024-06-10T01:02:37.373138+0000","last_became_active":"2024-06-10T01:01:57.332437+0000","last_became_peered":"2024-06-10T01:01:57.332437+0000","last_unstale":"2024-06-10T01:02:37.373138+0000","last_undegraded":"2024-06-10T01:02:37.373138+0000","last_fullsized":"2024-06-10T01:02:37.373138+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:27:35.029079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.368754+0000","last_change":"2024-06-10T01:01:47.255926+0000","last_active":"2024-06-10T01:02:37.368754+0000","last_peered":"2024-06-10T01:02:37.368754+0000","last_clean":"2024-06-10T01:02:37.368754+0000","last_became_active":"2024-06-10T01:01:47.255571+0000","last_became_peered":"2024-06-10T01:01:47.255571+0000","last_unstale":"2024-06-10T01:02:37.368754+0000","last_undegraded":"2024-06-10T01:02:37.368754+0000","last_fullsized":"2024-06-10T01:02:37.368754+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:05:22.084037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'3","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:53.711111+0000","last_change":"2024-06-10T01:01:44.007509+0000","last_active":"2024-06-10T01:02:53.711111+0000","last_peered":"2024-06-10T01:02:53.711111+0000","last_clean":"2024-06-10T01:02:53.711111+0000","last_became_active":"2024-06-10T01:01:44.007178+0000","last_became_peered":"2024-06-10T01:01:44.007178+0000","last_unstale":"2024-06-10T01:02:53.711111+0000","last_undegraded":"2024-06-10T01:02:53.711111+0000","last_fullsized":"2024-06-10T01:02:53.711111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:02:52.791886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.374290+0000","last_change":"2024-06-10T01:01:57.285626+0000","last_active":"2024-06-10T01:02:37.374290+0000","last_peered":"2024-06-10T01:02:37.374290+0000","last_clean":"2024-06-10T01:02:37.374290+0000","last_became_active":"2024-06-10T01:01:57.285098+0000","last_became_peered":"2024-06-10T01:01:57.285098+0000","last_unstale":"2024-06-10T01:02:37.374290+0000","last_undegraded":"2024-06-10T01:02:37.374290+0000","last_fullsized":"2024-06-10T01:02:37.374290+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:13:49.881770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:52.694460+0000","last_change":"2024-06-10T01:01:44.469028+0000","last_active":"2024-06-10T01:02:52.694460+0000","last_peered":"2024-06-10T01:02:52.694460+0000","last_clean":"2024-06-10T01:02:52.694460+0000","last_became_active":"2024-06-10T01:01:44.468570+0000","last_became_peered":"2024-06-10T01:01:44.468570+0000","last_unstale":"2024-06-10T01:02:52.694460+0000","last_undegraded":"2024-06-10T01:02:52.694460+0000","last_fullsized":"2024-06-10T01:02:52.694460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:12:11.287959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.369854+0000","last_change":"2024-06-10T01:01:47.253414+0000","last_active":"2024-06-10T01:02:37.369854+0000","last_peered":"2024-06-10T01:02:37.369854+0000","last_clean":"2024-06-10T01:02:37.369854+0000","last_became_active":"2024-06-10T01:01:47.253060+0000","last_became_peered":"2024-06-10T01:01:47.253060+0000","last_unstale":"2024-06-10T01:02:37.369854+0000","last_undegraded":"2024-06-10T01:02:37.369854+0000","last_fullsized":"2024-06-10T01:02:37.369854+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:24:55.534164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.590637+0000","last_change":"2024-06-10T01:01:57.332756+0000","last_active":"2024-06-10T01:02:37.590637+0000","last_peered":"2024-06-10T01:02:37.590637+0000","last_clean":"2024-06-10T01:02:37.590637+0000","last_became_active":"2024-06-10T01:01:57.332361+0000","last_became_peered":"2024-06-10T01:01:57.332361+0000","last_unstale":"2024-06-10T01:02:37.590637+0000","last_undegraded":"2024-06-10T01:02:37.590637+0000","last_fullsized":"2024-06-10T01:02:37.590637+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:02:28.275407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"97'10","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:03:12.254982+0000","last_change":"2024-06-10T01:01:44.005164+0000","last_active":"2024-06-10T01:03:12.254982+0000","last_peered":"2024-06-10T01:03:12.254982+0000","last_clean":"2024-06-10T01:03:12.254982+0000","last_became_active":"2024-06-10T01:01:44.004816+0000","last_became_peered":"2024-06-10T01:01:44.004816+0000","last_unstale":"2024-06-10T01:03:12.254982+0000","last_undegraded":"2024-06-10T01:03:12.254982+0000","last_fullsized":"2024-06-10T01:03:12.254982+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:11:56.385267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.592118+0000","last_change":"2024-06-10T01:01:47.247951+0000","last_active":"2024-06-10T01:02:37.592118+0000","last_peered":"2024-06-10T01:02:37.592118+0000","last_clean":"2024-06-10T01:02:37.592118+0000","last_became_active":"2024-06-10T01:01:47.246996+0000","last_became_peered":"2024-06-10T01:01:47.246996+0000","last_unstale":"2024-06-10T01:02:37.592118+0000","last_undegraded":"2024-06-10T01:02:37.592118+0000","last_fullsized":"2024-06-10T01:02:37.592118+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:39:18.167021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.366850+0000","last_change":"2024-06-10T01:01:57.278130+0000","last_active":"2024-06-10T01:02:37.366850+0000","last_peered":"2024-06-10T01:02:37.366850+0000","last_clean":"2024-06-10T01:02:37.366850+0000","last_became_active":"2024-06-10T01:01:57.277697+0000","last_became_peered":"2024-06-10T01:01:57.277697+0000","last_unstale":"2024-06-10T01:02:37.366850+0000","last_undegraded":"2024-06-10T01:02:37.366850+0000","last_fullsized":"2024-06-10T01:02:37.366850+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:17:01.662152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:52.697670+0000","last_change":"2024-06-10T01:01:44.553176+0000","last_active":"2024-06-10T01:02:52.697670+0000","last_peered":"2024-06-10T01:02:52.697670+0000","last_clean":"2024-06-10T01:02:52.697670+0000","last_became_active":"2024-06-10T01:01:44.552797+0000","last_became_peered":"2024-06-10T01:01:44.552797+0000","last_unstale":"2024-06-10T01:02:52.697670+0000","last_undegraded":"2024-06-10T01:02:52.697670+0000","last_fullsized":"2024-06-10T01:02:52.697670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:21:13.960304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.373615+0000","last_change":"2024-06-10T01:01:47.255796+0000","last_active":"2024-06-10T01:02:37.373615+0000","last_peered":"2024-06-10T01:02:37.373615+0000","last_clean":"2024-06-10T01:02:37.373615+0000","last_became_active":"2024-06-10T01:01:47.255495+0000","last_became_peered":"2024-06-10T01:01:47.255495+0000","last_unstale":"2024-06-10T01:02:37.373615+0000","last_undegraded":"2024-06-10T01:02:37.373615+0000","last_fullsized":"2024-06-10T01:02:37.373615+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:06:42.209005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.368892+0000","last_change":"2024-06-10T01:01:57.332869+0000","last_active":"2024-06-10T01:02:37.368892+0000","last_peered":"2024-06-10T01:02:37.368892+0000","last_clean":"2024-06-10T01:02:37.368892+0000","last_became_active":"2024-06-10T01:01:57.332391+0000","last_became_peered":"2024-06-10T01:01:57.332391+0000","last_unstale":"2024-06-10T01:02:37.368892+0000","last_undegraded":"2024-06-10T01:02:37.368892+0000","last_fullsized":"2024-06-10T01:02:37.368892+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:58:31.443626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:52.698476+0000","last_change":"2024-06-10T01:01:44.466861+0000","last_active":"2024-06-10T01:02:52.698476+0000","last_peered":"2024-06-10T01:02:52.698476+0000","last_clean":"2024-06-10T01:02:52.698476+0000","last_became_active":"2024-06-10T01:01:44.466553+0000","last_became_peered":"2024-06-10T01:01:44.466553+0000","last_unstale":"2024-06-10T01:02:52.698476+0000","last_undegraded":"2024-06-10T01:02:52.698476+0000","last_fullsized":"2024-06-10T01:02:52.698476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:55:52.545594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.368847+0000","last_change":"2024-06-10T01:01:47.244972+0000","last_active":"2024-06-10T01:02:37.368847+0000","last_peered":"2024-06-10T01:02:37.368847+0000","last_clean":"2024-06-10T01:02:37.368847+0000","last_became_active":"2024-06-10T01:01:47.244649+0000","last_became_peered":"2024-06-10T01:01:47.244649+0000","last_unstale":"2024-06-10T01:02:37.368847+0000","last_undegraded":"2024-06-10T01:02:37.368847+0000","last_fullsized":"2024-06-10T01:02:37.368847+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:10:46.909200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.593332+0000","last_change":"2024-06-10T01:02:37.592380+0000","last_active":"2024-06-10T01:02:37.593332+0000","last_peered":"2024-06-10T01:02:37.593332+0000","last_clean":"2024-06-10T01:02:37.593332+0000","last_became_active":"2024-06-10T01:02:37.591856+0000","last_became_peered":"2024-06-10T01:02:37.591856+0000","last_unstale":"2024-06-10T01:02:37.593332+0000","last_undegraded":"2024-06-10T01:02:37.593332+0000","last_fullsized":"2024-06-10T01:02:37.593332+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:00:25.717192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:52.699377+0000","last_change":"2024-06-10T01:01:44.008116+0000","last_active":"2024-06-10T01:02:52.699377+0000","last_peered":"2024-06-10T01:02:52.699377+0000","last_clean":"2024-06-10T01:02:52.699377+0000","last_became_active":"2024-06-10T01:01:44.007778+0000","last_became_peered":"2024-06-10T01:01:44.007778+0000","last_unstale":"2024-06-10T01:02:52.699377+0000","last_undegraded":"2024-06-10T01:02:52.699377+0000","last_fullsized":"2024-06-10T01:02:52.699377+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:18:57.267296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.374063+0000","last_change":"2024-06-10T01:01:47.252833+0000","last_active":"2024-06-10T01:02:37.374063+0000","last_peered":"2024-06-10T01:02:37.374063+0000","last_clean":"2024-06-10T01:02:37.374063+0000","last_became_active":"2024-06-10T01:01:47.252326+0000","last_became_peered":"2024-06-10T01:01:47.252326+0000","last_unstale":"2024-06-10T01:02:37.374063+0000","last_undegraded":"2024-06-10T01:02:37.374063+0000","last_fullsized":"2024-06-10T01:02:37.374063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:00:42.673838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":25,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.369673+0000","last_change":"2024-06-10T01:01:57.275975+0000","last_active":"2024-06-10T01:02:37.369673+0000","last_peered":"2024-06-10T01:02:37.369673+0000","last_clean":"2024-06-10T01:02:37.369673+0000","last_became_active":"2024-06-10T01:01:57.275408+0000","last_became_peered":"2024-06-10T01:01:57.275408+0000","last_unstale":"2024-06-10T01:02:37.369673+0000","last_undegraded":"2024-06-10T01:02:37.369673+0000","last_fullsized":"2024-06-10T01:02:37.369673+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:12:42.376636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'1","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.367168+0000","last_change":"2024-06-10T01:01:44.006246+0000","last_active":"2024-06-10T01:02:37.367168+0000","last_peered":"2024-06-10T01:02:37.367168+0000","last_clean":"2024-06-10T01:02:37.367168+0000","last_became_active":"2024-06-10T01:01:44.005852+0000","last_became_peered":"2024-06-10T01:01:44.005852+0000","last_unstale":"2024-06-10T01:02:37.367168+0000","last_undegraded":"2024-06-10T01:02:37.367168+0000","last_fullsized":"2024-06-10T01:02:37.367168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:56:47.260279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.591178+0000","last_change":"2024-06-10T01:01:47.243154+0000","last_active":"2024-06-10T01:02:37.591178+0000","last_peered":"2024-06-10T01:02:37.591178+0000","last_clean":"2024-06-10T01:02:37.591178+0000","last_became_active":"2024-06-10T01:01:47.242851+0000","last_became_peered":"2024-06-10T01:01:47.242851+0000","last_unstale":"2024-06-10T01:02:37.591178+0000","last_undegraded":"2024-06-10T01:02:37.591178+0000","last_fullsized":"2024-06-10T01:02:37.591178+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:41:05.857777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.369132+0000","last_change":"2024-06-10T01:01:57.333613+0000","last_active":"2024-06-10T01:02:37.369132+0000","last_peered":"2024-06-10T01:02:37.369132+0000","last_clean":"2024-06-10T01:02:37.369132+0000","last_became_active":"2024-06-10T01:01:57.333019+0000","last_became_peered":"2024-06-10T01:01:57.333019+0000","last_unstale":"2024-06-10T01:02:37.369132+0000","last_undegraded":"2024-06-10T01:02:37.369132+0000","last_fullsized":"2024-06-10T01:02:37.369132+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:11.912885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:52.696696+0000","last_change":"2024-06-10T01:01:44.008221+0000","last_active":"2024-06-10T01:02:52.696696+0000","last_peered":"2024-06-10T01:02:52.696696+0000","last_clean":"2024-06-10T01:02:52.696696+0000","last_became_active":"2024-06-10T01:01:44.007880+0000","last_became_peered":"2024-06-10T01:01:44.007880+0000","last_unstale":"2024-06-10T01:02:52.696696+0000","last_undegraded":"2024-06-10T01:02:52.696696+0000","last_fullsized":"2024-06-10T01:02:52.696696+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:40:33.162925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.974467+0000","last_change":"2024-06-10T01:01:47.254902+0000","last_active":"2024-06-10T01:02:37.974467+0000","last_peered":"2024-06-10T01:02:37.974467+0000","last_clean":"2024-06-10T01:02:37.974467+0000","last_became_active":"2024-06-10T01:01:47.254555+0000","last_became_peered":"2024-06-10T01:01:47.254555+0000","last_unstale":"2024-06-10T01:02:37.974467+0000","last_undegraded":"2024-06-10T01:02:37.974467+0000","last_fullsized":"2024-06-10T01:02:37.974467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:03:17.571001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.792433+0000","last_change":"2024-06-10T01:01:57.284491+0000","last_active":"2024-06-10T01:02:37.792433+0000","last_peered":"2024-06-10T01:02:37.792433+0000","last_clean":"2024-06-10T01:02:37.792433+0000","last_became_active":"2024-06-10T01:01:57.283795+0000","last_became_peered":"2024-06-10T01:01:57.283795+0000","last_unstale":"2024-06-10T01:02:37.792433+0000","last_undegraded":"2024-06-10T01:02:37.792433+0000","last_fullsized":"2024-06-10T01:02:37.792433+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:49:00.381664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'3","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:53.710646+0000","last_change":"2024-06-10T01:01:44.553159+0000","last_active":"2024-06-10T01:02:53.710646+0000","last_peered":"2024-06-10T01:02:53.710646+0000","last_clean":"2024-06-10T01:02:53.710646+0000","last_became_active":"2024-06-10T01:01:44.552759+0000","last_became_peered":"2024-06-10T01:01:44.552759+0000","last_unstale":"2024-06-10T01:02:53.710646+0000","last_undegraded":"2024-06-10T01:02:53.710646+0000","last_fullsized":"2024-06-10T01:02:53.710646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:19:00.110790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.974880+0000","last_change":"2024-06-10T01:01:47.247803+0000","last_active":"2024-06-10T01:02:37.974880+0000","last_peered":"2024-06-10T01:02:37.974880+0000","last_clean":"2024-06-10T01:02:37.974880+0000","last_became_active":"2024-06-10T01:01:47.246715+0000","last_became_peered":"2024-06-10T01:01:47.246715+0000","last_unstale":"2024-06-10T01:02:37.974880+0000","last_undegraded":"2024-06-10T01:02:37.974880+0000","last_fullsized":"2024-06-10T01:02:37.974880+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:22.429002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.372674+0000","last_change":"2024-06-10T01:01:57.287668+0000","last_active":"2024-06-10T01:02:37.372674+0000","last_peered":"2024-06-10T01:02:37.372674+0000","last_clean":"2024-06-10T01:02:37.372674+0000","last_became_active":"2024-06-10T01:01:57.287060+0000","last_became_peered":"2024-06-10T01:01:57.287060+0000","last_unstale":"2024-06-10T01:02:37.372674+0000","last_undegraded":"2024-06-10T01:02:37.372674+0000","last_fullsized":"2024-06-10T01:02:37.372674+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:04:35.883613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.374270+0000","last_change":"2024-06-10T01:01:44.013051+0000","last_active":"2024-06-10T01:02:37.374270+0000","last_peered":"2024-06-10T01:02:37.374270+0000","last_clean":"2024-06-10T01:02:37.374270+0000","last_became_active":"2024-06-10T01:01:44.012667+0000","last_became_peered":"2024-06-10T01:01:44.012667+0000","last_unstale":"2024-06-10T01:02:37.374270+0000","last_undegraded":"2024-06-10T01:02:37.374270+0000","last_fullsized":"2024-06-10T01:02:37.374270+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:10:49.650082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.974462+0000","last_change":"2024-06-10T01:01:47.251209+0000","last_active":"2024-06-10T01:02:37.974462+0000","last_peered":"2024-06-10T01:02:37.974462+0000","last_clean":"2024-06-10T01:02:37.974462+0000","last_became_active":"2024-06-10T01:01:47.250858+0000","last_became_peered":"2024-06-10T01:01:47.250858+0000","last_unstale":"2024-06-10T01:02:37.974462+0000","last_undegraded":"2024-06-10T01:02:37.974462+0000","last_fullsized":"2024-06-10T01:02:37.974462+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:47.272727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.371843+0000","last_change":"2024-06-10T01:01:57.284187+0000","last_active":"2024-06-10T01:02:37.371843+0000","last_peered":"2024-06-10T01:02:37.371843+0000","last_clean":"2024-06-10T01:02:37.371843+0000","last_became_active":"2024-06-10T01:01:57.283739+0000","last_became_peered":"2024-06-10T01:01:57.283739+0000","last_unstale":"2024-06-10T01:02:37.371843+0000","last_undegraded":"2024-06-10T01:02:37.371843+0000","last_fullsized":"2024-06-10T01:02:37.371843+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:13:34.630708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.370086+0000","last_change":"2024-06-10T01:01:44.005702+0000","last_active":"2024-06-10T01:02:37.370086+0000","last_peered":"2024-06-10T01:02:37.370086+0000","last_clean":"2024-06-10T01:02:37.370086+0000","last_became_active":"2024-06-10T01:01:44.005354+0000","last_became_peered":"2024-06-10T01:01:44.005354+0000","last_unstale":"2024-06-10T01:02:37.370086+0000","last_undegraded":"2024-06-10T01:02:37.370086+0000","last_fullsized":"2024-06-10T01:02:37.370086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:51:14.819688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.974705+0000","last_change":"2024-06-10T01:01:47.250325+0000","last_active":"2024-06-10T01:02:37.974705+0000","last_peered":"2024-06-10T01:02:37.974705+0000","last_clean":"2024-06-10T01:02:37.974705+0000","last_became_active":"2024-06-10T01:01:47.249591+0000","last_became_peered":"2024-06-10T01:01:47.249591+0000","last_unstale":"2024-06-10T01:02:37.974705+0000","last_undegraded":"2024-06-10T01:02:37.974705+0000","last_fullsized":"2024-06-10T01:02:37.974705+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:17:16.268717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.374167+0000","last_change":"2024-06-10T01:01:47.253449+0000","last_active":"2024-06-10T01:02:37.374167+0000","last_peered":"2024-06-10T01:02:37.374167+0000","last_clean":"2024-06-10T01:02:37.374167+0000","last_became_active":"2024-06-10T01:01:47.253144+0000","last_became_peered":"2024-06-10T01:01:47.253144+0000","last_unstale":"2024-06-10T01:02:37.374167+0000","last_undegraded":"2024-06-10T01:02:37.374167+0000","last_fullsized":"2024-06-10T01:02:37.374167+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:32:22.353409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.367553+0000","last_change":"2024-06-10T01:01:47.258241+0000","last_active":"2024-06-10T01:02:37.367553+0000","last_peered":"2024-06-10T01:02:37.367553+0000","last_clean":"2024-06-10T01:02:37.367553+0000","last_became_active":"2024-06-10T01:01:47.257823+0000","last_became_peered":"2024-06-10T01:01:47.257823+0000","last_unstale":"2024-06-10T01:02:37.367553+0000","last_undegraded":"2024-06-10T01:02:37.367553+0000","last_fullsized":"2024-06-10T01:02:37.367553+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:11:23.652479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.366971+0000","last_change":"2024-06-10T01:01:47.256218+0000","last_active":"2024-06-10T01:02:37.366971+0000","last_peered":"2024-06-10T01:02:37.366971+0000","last_clean":"2024-06-10T01:02:37.366971+0000","last_became_active":"2024-06-10T01:01:47.255950+0000","last_became_peered":"2024-06-10T01:01:47.255950+0000","last_unstale":"2024-06-10T01:02:37.366971+0000","last_undegraded":"2024-06-10T01:02:37.366971+0000","last_fullsized":"2024-06-10T01:02:37.366971+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:59:37.283646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.373530+0000","last_change":"2024-06-10T01:01:47.253109+0000","last_active":"2024-06-10T01:02:37.373530+0000","last_peered":"2024-06-10T01:02:37.373530+0000","last_clean":"2024-06-10T01:02:37.373530+0000","last_became_active":"2024-06-10T01:01:47.252767+0000","last_became_peered":"2024-06-10T01:01:47.252767+0000","last_unstale":"2024-06-10T01:02:37.373530+0000","last_undegraded":"2024-06-10T01:02:37.373530+0000","last_fullsized":"2024-06-10T01:02:37.373530+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:57:13.177230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.375094+0000","last_change":"2024-06-10T01:02:37.375094+0000","last_active":"2024-06-10T01:02:37.375094+0000","last_peered":"2024-06-10T01:02:37.375094+0000","last_clean":"2024-06-10T01:02:37.375094+0000","last_became_active":"2024-06-10T01:02:37.374527+0000","last_became_peered":"2024-06-10T01:02:37.374527+0000","last_unstale":"2024-06-10T01:02:37.375094+0000","last_undegraded":"2024-06-10T01:02:37.375094+0000","last_fullsized":"2024-06-10T01:02:37.375094+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:34:41.205816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.374992+0000","last_change":"2024-06-10T01:01:47.250243+0000","last_active":"2024-06-10T01:02:37.374992+0000","last_peered":"2024-06-10T01:02:37.374992+0000","last_clean":"2024-06-10T01:02:37.374992+0000","last_became_active":"2024-06-10T01:01:47.249907+0000","last_became_peered":"2024-06-10T01:01:47.249907+0000","last_unstale":"2024-06-10T01:02:37.374992+0000","last_undegraded":"2024-06-10T01:02:37.374992+0000","last_fullsized":"2024-06-10T01:02:37.374992+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:31:32.933660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.368310+0000","last_change":"2024-06-10T01:01:47.254173+0000","last_active":"2024-06-10T01:02:37.368310+0000","last_peered":"2024-06-10T01:02:37.368310+0000","last_clean":"2024-06-10T01:02:37.368310+0000","last_became_active":"2024-06-10T01:01:47.253796+0000","last_became_peered":"2024-06-10T01:01:47.253796+0000","last_unstale":"2024-06-10T01:02:37.368310+0000","last_undegraded":"2024-06-10T01:02:37.368310+0000","last_fullsized":"2024-06-10T01:02:37.368310+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:41:12.199114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.592242+0000","last_change":"2024-06-10T01:01:47.253920+0000","last_active":"2024-06-10T01:02:37.592242+0000","last_peered":"2024-06-10T01:02:37.592242+0000","last_clean":"2024-06-10T01:02:37.592242+0000","last_became_active":"2024-06-10T01:01:47.253288+0000","last_became_peered":"2024-06-10T01:01:47.253288+0000","last_unstale":"2024-06-10T01:02:37.592242+0000","last_undegraded":"2024-06-10T01:02:37.592242+0000","last_fullsized":"2024-06-10T01:02:37.592242+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:41:20.406263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.372341+0000","last_change":"2024-06-10T01:01:47.240772+0000","last_active":"2024-06-10T01:02:37.372341+0000","last_peered":"2024-06-10T01:02:37.372341+0000","last_clean":"2024-06-10T01:02:37.372341+0000","last_became_active":"2024-06-10T01:01:47.238258+0000","last_became_peered":"2024-06-10T01:01:47.238258+0000","last_unstale":"2024-06-10T01:02:37.372341+0000","last_undegraded":"2024-06-10T01:02:37.372341+0000","last_fullsized":"2024-06-10T01:02:37.372341+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:55:25.211434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.377151+0000","last_change":"2024-06-10T01:02:37.376724+0000","last_active":"2024-06-10T01:02:37.377151+0000","last_peered":"2024-06-10T01:02:37.377151+0000","last_clean":"2024-06-10T01:02:37.377151+0000","last_became_active":"2024-06-10T01:02:37.376329+0000","last_became_peered":"2024-06-10T01:02:37.376329+0000","last_unstale":"2024-06-10T01:02:37.377151+0000","last_undegraded":"2024-06-10T01:02:37.377151+0000","last_fullsized":"2024-06-10T01:02:37.377151+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:38:21.837958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.792781+0000","last_change":"2024-06-10T01:01:47.244463+0000","last_active":"2024-06-10T01:02:37.792781+0000","last_peered":"2024-06-10T01:02:37.792781+0000","last_clean":"2024-06-10T01:02:37.792781+0000","last_became_active":"2024-06-10T01:01:47.244114+0000","last_became_peered":"2024-06-10T01:01:47.244114+0000","last_unstale":"2024-06-10T01:02:37.792781+0000","last_undegraded":"2024-06-10T01:02:37.792781+0000","last_fullsized":"2024-06-10T01:02:37.792781+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:47:37.098894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.370498+0000","last_change":"2024-06-10T01:01:47.248317+0000","last_active":"2024-06-10T01:02:37.370498+0000","last_peered":"2024-06-10T01:02:37.370498+0000","last_clean":"2024-06-10T01:02:37.370498+0000","last_became_active":"2024-06-10T01:01:47.247958+0000","last_became_peered":"2024-06-10T01:01:47.247958+0000","last_unstale":"2024-06-10T01:02:37.370498+0000","last_undegraded":"2024-06-10T01:02:37.370498+0000","last_fullsized":"2024-06-10T01:02:37.370498+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:39:59.267957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.374610+0000","last_change":"2024-06-10T01:01:47.252542+0000","last_active":"2024-06-10T01:02:37.374610+0000","last_peered":"2024-06-10T01:02:37.374610+0000","last_clean":"2024-06-10T01:02:37.374610+0000","last_became_active":"2024-06-10T01:01:47.252216+0000","last_became_peered":"2024-06-10T01:01:47.252216+0000","last_unstale":"2024-06-10T01:02:37.374610+0000","last_undegraded":"2024-06-10T01:02:37.374610+0000","last_fullsized":"2024-06-10T01:02:37.374610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:12:22.196263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285691,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58585,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":157,"num_read_kb":141,"num_write":162,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":185391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383713,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":550999,"data_compressed":26362,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645350,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32356,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712796,"statfs":{"total":95995035648,"available":95961903104,"internally_reserved":0,"allocated":1208320,"data_stored":1193242,"data_compressed":62688,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808875,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1089536,"data_stored":555331,"data_compressed":26362,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070513,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":551155,"data_compressed":26362,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332149,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":1146673,"data_compressed":62688,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495674,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":546411,"data_compressed":26362,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1130496,"data_stored":606114,"data_compressed":26362,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1089536,"data_stored":547799,"data_compressed":26362,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1146880,"data_stored":607352,"data_compressed":26362,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":546605,"data_compressed":26362,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920836,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":551173,"data_compressed":26362,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757311,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1142784,"data_stored":1142261,"data_compressed":62688,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34687,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34687,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34687,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":61031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":56553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T01:03:21.910 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-10T01:03:22.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:21 smithi078 ceph-mon[36683]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T01:03:22.161 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:03:22.205 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:21 smithi175 ceph-mon[36636]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T01:03:22.976 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:22 smithi007 ceph-mon[32398]: pgmap v350: 129 pgs: 129 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-06-10T01:03:23.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:22 smithi078 ceph-mon[36683]: pgmap v350: 129 pgs: 129 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-06-10T01:03:23.205 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:22 smithi175 ceph-mon[36636]: pgmap v350: 129 pgs: 129 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-06-10T01:03:23.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.924+0000 7fed71426700 1 -- 172.21.15.7:0/4052384028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed6c06ac60 msgr2=0x7fed6c06b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:23.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.924+0000 7fed71426700 1 --2- 172.21.15.7:0/4052384028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed6c06ac60 0x7fed6c06b0e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fed600099c0 tx=0x7fed60009cd0 comp rx=0 tx=0).stop 2024-06-10T01:03:23.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.924+0000 7fed71426700 1 -- 172.21.15.7:0/4052384028 shutdown_connections 2024-06-10T01:03:23.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.924+0000 7fed71426700 1 --2- 172.21.15.7:0/4052384028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed6c06b620 0x7fed6c10f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:23.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.924+0000 7fed71426700 1 --2- 172.21.15.7:0/4052384028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed6c108580 0x7fed6c108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:23.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.924+0000 7fed71426700 1 --2- 172.21.15.7:0/4052384028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed6c06ac60 0x7fed6c06b0e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:23.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.924+0000 7fed71426700 1 -- 172.21.15.7:0/4052384028 >> 172.21.15.7:0/4052384028 conn(0x7fed6c078430 msgr2=0x7fed6c078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:23.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.924+0000 7fed71426700 1 -- 172.21.15.7:0/4052384028 shutdown_connections 2024-06-10T01:03:23.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.924+0000 7fed71426700 1 -- 172.21.15.7:0/4052384028 wait complete. 2024-06-10T01:03:23.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.925+0000 7fed71426700 1 Processor -- start 2024-06-10T01:03:23.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.925+0000 7fed71426700 1 -- start start 2024-06-10T01:03:23.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.925+0000 7fed71426700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed6c06ac60 0x7fed6c19e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:23.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.925+0000 7fed71426700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed6c06b620 0x7fed6c19ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:23.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.925+0000 7fed71426700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed6c108580 0x7fed6c1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:23.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.925+0000 7fed71426700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fed6c1127e0 con 0x7fed6c06b620 2024-06-10T01:03:23.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.925+0000 7fed71426700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fed6c112660 con 0x7fed6c06ac60 2024-06-10T01:03:23.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.925+0000 7fed71426700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fed6c112960 con 0x7fed6c108580 2024-06-10T01:03:23.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.925+0000 7fed6a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed6c06b620 0x7fed6c19ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:23.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.925+0000 7fed6a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed6c06b620 0x7fed6c19ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40852/0 (socket says 172.21.15.7:40852) 2024-06-10T01:03:23.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.925+0000 7fed6a7fc700 1 -- 172.21.15.7:0/1864296248 learned_addr learned my addr 172.21.15.7:0/1864296248 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:03:23.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.925+0000 7fed6affd700 1 --2- 172.21.15.7:0/1864296248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed6c06ac60 0x7fed6c19e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:23.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.925+0000 7fed6b7fe700 1 --2- 172.21.15.7:0/1864296248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed6c108580 0x7fed6c1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:23.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.925+0000 7fed6a7fc700 1 -- 172.21.15.7:0/1864296248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed6c108580 msgr2=0x7fed6c1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:23.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.925+0000 7fed6a7fc700 1 --2- 172.21.15.7:0/1864296248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed6c108580 0x7fed6c1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:23.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.925+0000 7fed6a7fc700 1 -- 172.21.15.7:0/1864296248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed6c06ac60 msgr2=0x7fed6c19e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:23.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.925+0000 7fed6a7fc700 1 --2- 172.21.15.7:0/1864296248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed6c06ac60 0x7fed6c19e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:23.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.926+0000 7fed6a7fc700 1 -- 172.21.15.7:0/1864296248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed6c199120 con 0x7fed6c06b620 2024-06-10T01:03:23.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.926+0000 7fed6b7fe700 1 --2- 172.21.15.7:0/1864296248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed6c108580 0x7fed6c1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:03:23.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.926+0000 7fed6affd700 1 --2- 172.21.15.7:0/1864296248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed6c06ac60 0x7fed6c19e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:03:23.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.926+0000 7fed6a7fc700 1 --2- 172.21.15.7:0/1864296248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed6c06b620 0x7fed6c19ecf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fed60009fd0 tx=0x7fed60003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:23.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.926+0000 7fed53fff700 1 -- 172.21.15.7:0/1864296248 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60027070 con 0x7fed6c06b620 2024-06-10T01:03:23.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.926+0000 7fed71426700 1 -- 172.21.15.7:0/1864296248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed6c1993b0 con 0x7fed6c06b620 2024-06-10T01:03:23.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.926+0000 7fed53fff700 1 -- 172.21.15.7:0/1864296248 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed60004150 con 0x7fed6c06b620 2024-06-10T01:03:23.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.926+0000 7fed53fff700 1 -- 172.21.15.7:0/1864296248 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6000f460 con 0x7fed6c06b620 2024-06-10T01:03:23.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.926+0000 7fed71426700 1 -- 172.21.15.7:0/1864296248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed6c1a5680 con 0x7fed6c06b620 2024-06-10T01:03:23.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.927+0000 7fed71426700 1 -- 172.21.15.7:0/1864296248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed6c10aa10 con 0x7fed6c06b620 2024-06-10T01:03:23.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.928+0000 7fed53fff700 1 -- 172.21.15.7:0/1864296248 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fed6002a070 con 0x7fed6c06b620 2024-06-10T01:03:23.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.929+0000 7fed53fff700 1 --2- 172.21.15.7:0/1864296248 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fed540a2cb0 0x7fed540a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:23.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.929+0000 7fed53fff700 1 -- 172.21.15.7:0/1864296248 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7fed600d0500 con 0x7fed6c06b620 2024-06-10T01:03:23.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.929+0000 7fed6affd700 1 --2- 172.21.15.7:0/1864296248 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fed540a2cb0 0x7fed540a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:23.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.929+0000 7fed6affd700 1 --2- 172.21.15.7:0/1864296248 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fed540a2cb0 0x7fed540a5170 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fed5c007fd0 tx=0x7fed5c007810 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:23.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:23.933+0000 7fed53fff700 1 -- 172.21.15.7:0/1864296248 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fed6005e850 con 0x7fed6c06b620 2024-06-10T01:03:24.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:24.136+0000 7fed71426700 1 -- 172.21.15.7:0/1864296248 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fed6c0628d0 con 0x7fed540a2cb0 2024-06-10T01:03:24.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:24.146+0000 7fed53fff700 1 -- 172.21.15.7:0/1864296248 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340630 (secure 0 0 0) 0x7fed5c06d2b0 con 0x7fed540a2cb0 2024-06-10T01:03:24.149 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:03:24.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:24.150+0000 7fed71426700 1 -- 172.21.15.7:0/1864296248 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fed540a2cb0 msgr2=0x7fed540a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:24.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:24.150+0000 7fed71426700 1 --2- 172.21.15.7:0/1864296248 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fed540a2cb0 0x7fed540a5170 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fed5c007fd0 tx=0x7fed5c007810 comp rx=0 tx=0).stop 2024-06-10T01:03:24.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:24.150+0000 7fed71426700 1 -- 172.21.15.7:0/1864296248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed6c06b620 msgr2=0x7fed6c19ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:24.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:24.150+0000 7fed71426700 1 --2- 172.21.15.7:0/1864296248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed6c06b620 0x7fed6c19ecf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fed60009fd0 tx=0x7fed60003730 comp rx=0 tx=0).stop 2024-06-10T01:03:24.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:24.150+0000 7fed71426700 1 -- 172.21.15.7:0/1864296248 shutdown_connections 2024-06-10T01:03:24.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:24.150+0000 7fed71426700 1 --2- 172.21.15.7:0/1864296248 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fed540a2cb0 0x7fed540a5170 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:24.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:24.150+0000 7fed71426700 1 --2- 172.21.15.7:0/1864296248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed6c108580 0x7fed6c1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:24.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:24.150+0000 7fed71426700 1 --2- 172.21.15.7:0/1864296248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed6c06ac60 0x7fed6c19e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:24.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:24.150+0000 7fed71426700 1 --2- 172.21.15.7:0/1864296248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed6c06b620 0x7fed6c19ecf0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:24.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:24.150+0000 7fed71426700 1 -- 172.21.15.7:0/1864296248 >> 172.21.15.7:0/1864296248 conn(0x7fed6c078430 msgr2=0x7fed6c0ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:24.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:24.150+0000 7fed71426700 1 -- 172.21.15.7:0/1864296248 shutdown_connections 2024-06-10T01:03:24.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:24.150+0000 7fed71426700 1 -- 172.21.15.7:0/1864296248 wait complete. 2024-06-10T01:03:24.153 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-06-10T01:03:24.887 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:24 smithi007 ceph-mon[32398]: pgmap v351: 129 pgs: 129 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-06-10T01:03:24.887 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:24 smithi007 ceph-mon[32398]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T01:03:24.923 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":351,"stamp":"2024-06-10T01:03:23.557820+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654005,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":304,"num_read_kb":265,"num_write":352,"num_write_kb":3054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":282,"ondisk_log_size":282,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362168,"kb_used_data":13016,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579656,"statfs":{"total":1151940427776,"available":1151569567744,"internally_reserved":0,"allocated":13328384,"data_stored":8549933,"data_compressed":425322,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15433,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":67,"num_read_kb":46,"num_write":19,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003523"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.375628+0000","last_change":"2024-06-10T01:01:47.244728+0000","last_active":"2024-06-10T01:02:37.375628+0000","last_peered":"2024-06-10T01:02:37.375628+0000","last_clean":"2024-06-10T01:02:37.375628+0000","last_became_active":"2024-06-10T01:01:47.244198+0000","last_became_peered":"2024-06-10T01:01:47.244198+0000","last_unstale":"2024-06-10T01:02:37.375628+0000","last_undegraded":"2024-06-10T01:02:37.375628+0000","last_fullsized":"2024-06-10T01:02:37.375628+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:58:06.257952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.373467+0000","last_change":"2024-06-10T01:01:47.247620+0000","last_active":"2024-06-10T01:02:37.373467+0000","last_peered":"2024-06-10T01:02:37.373467+0000","last_clean":"2024-06-10T01:02:37.373467+0000","last_became_active":"2024-06-10T01:01:47.245475+0000","last_became_peered":"2024-06-10T01:01:47.245475+0000","last_unstale":"2024-06-10T01:02:37.373467+0000","last_undegraded":"2024-06-10T01:02:37.373467+0000","last_fullsized":"2024-06-10T01:02:37.373467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:29:09.367411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.372987+0000","last_change":"2024-06-10T01:01:47.252414+0000","last_active":"2024-06-10T01:02:37.372987+0000","last_peered":"2024-06-10T01:02:37.372987+0000","last_clean":"2024-06-10T01:02:37.372987+0000","last_became_active":"2024-06-10T01:01:47.252063+0000","last_became_peered":"2024-06-10T01:01:47.252063+0000","last_unstale":"2024-06-10T01:02:37.372987+0000","last_undegraded":"2024-06-10T01:02:37.372987+0000","last_fullsized":"2024-06-10T01:02:37.372987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:54:53.966209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.369842+0000","last_change":"2024-06-10T01:01:47.254875+0000","last_active":"2024-06-10T01:02:37.369842+0000","last_peered":"2024-06-10T01:02:37.369842+0000","last_clean":"2024-06-10T01:02:37.369842+0000","last_became_active":"2024-06-10T01:01:47.254542+0000","last_became_peered":"2024-06-10T01:01:47.254542+0000","last_unstale":"2024-06-10T01:02:37.369842+0000","last_undegraded":"2024-06-10T01:02:37.369842+0000","last_fullsized":"2024-06-10T01:02:37.369842+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:02:28.393133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.369100+0000","last_change":"2024-06-10T01:01:47.242596+0000","last_active":"2024-06-10T01:02:37.369100+0000","last_peered":"2024-06-10T01:02:37.369100+0000","last_clean":"2024-06-10T01:02:37.369100+0000","last_became_active":"2024-06-10T01:01:47.242308+0000","last_became_peered":"2024-06-10T01:01:47.242308+0000","last_unstale":"2024-06-10T01:02:37.369100+0000","last_undegraded":"2024-06-10T01:02:37.369100+0000","last_fullsized":"2024-06-10T01:02:37.369100+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:11:26.599954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.367730+0000","last_change":"2024-06-10T01:01:47.249085+0000","last_active":"2024-06-10T01:02:37.367730+0000","last_peered":"2024-06-10T01:02:37.367730+0000","last_clean":"2024-06-10T01:02:37.367730+0000","last_became_active":"2024-06-10T01:01:47.248749+0000","last_became_peered":"2024-06-10T01:01:47.248749+0000","last_unstale":"2024-06-10T01:02:37.367730+0000","last_undegraded":"2024-06-10T01:02:37.367730+0000","last_fullsized":"2024-06-10T01:02:37.367730+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:42:48.734566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.974291+0000","last_change":"2024-06-10T01:01:47.247334+0000","last_active":"2024-06-10T01:02:37.974291+0000","last_peered":"2024-06-10T01:02:37.974291+0000","last_clean":"2024-06-10T01:02:37.974291+0000","last_became_active":"2024-06-10T01:01:47.246964+0000","last_became_peered":"2024-06-10T01:01:47.246964+0000","last_unstale":"2024-06-10T01:02:37.974291+0000","last_undegraded":"2024-06-10T01:02:37.974291+0000","last_fullsized":"2024-06-10T01:02:37.974291+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:05:36.034950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.590922+0000","last_change":"2024-06-10T01:01:47.242523+0000","last_active":"2024-06-10T01:02:37.590922+0000","last_peered":"2024-06-10T01:02:37.590922+0000","last_clean":"2024-06-10T01:02:37.590922+0000","last_became_active":"2024-06-10T01:01:47.242198+0000","last_became_peered":"2024-06-10T01:01:47.242198+0000","last_unstale":"2024-06-10T01:02:37.590922+0000","last_undegraded":"2024-06-10T01:02:37.590922+0000","last_fullsized":"2024-06-10T01:02:37.590922+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:39:07.055970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.793070+0000","last_change":"2024-06-10T01:01:47.245277+0000","last_active":"2024-06-10T01:02:37.793070+0000","last_peered":"2024-06-10T01:02:37.793070+0000","last_clean":"2024-06-10T01:02:37.793070+0000","last_became_active":"2024-06-10T01:01:47.244932+0000","last_became_peered":"2024-06-10T01:01:47.244932+0000","last_unstale":"2024-06-10T01:02:37.793070+0000","last_undegraded":"2024-06-10T01:02:37.793070+0000","last_fullsized":"2024-06-10T01:02:37.793070+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:56:24.423713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.367630+0000","last_change":"2024-06-10T01:01:47.255531+0000","last_active":"2024-06-10T01:02:37.367630+0000","last_peered":"2024-06-10T01:02:37.367630+0000","last_clean":"2024-06-10T01:02:37.367630+0000","last_became_active":"2024-06-10T01:01:47.255150+0000","last_became_peered":"2024-06-10T01:01:47.255150+0000","last_unstale":"2024-06-10T01:02:37.367630+0000","last_undegraded":"2024-06-10T01:02:37.367630+0000","last_fullsized":"2024-06-10T01:02:37.367630+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:51:34.561396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.974825+0000","last_change":"2024-06-10T01:01:47.249832+0000","last_active":"2024-06-10T01:02:37.974825+0000","last_peered":"2024-06-10T01:02:37.974825+0000","last_clean":"2024-06-10T01:02:37.974825+0000","last_became_active":"2024-06-10T01:01:47.249303+0000","last_became_peered":"2024-06-10T01:01:47.249303+0000","last_unstale":"2024-06-10T01:02:37.974825+0000","last_undegraded":"2024-06-10T01:02:37.974825+0000","last_fullsized":"2024-06-10T01:02:37.974825+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:15:21.930157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.369968+0000","last_change":"2024-06-10T01:01:47.255927+0000","last_active":"2024-06-10T01:02:37.369968+0000","last_peered":"2024-06-10T01:02:37.369968+0000","last_clean":"2024-06-10T01:02:37.369968+0000","last_became_active":"2024-06-10T01:01:47.255591+0000","last_became_peered":"2024-06-10T01:01:47.255591+0000","last_unstale":"2024-06-10T01:02:37.369968+0000","last_undegraded":"2024-06-10T01:02:37.369968+0000","last_fullsized":"2024-06-10T01:02:37.369968+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:48:06.494430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.372069+0000","last_change":"2024-06-10T01:02:37.372069+0000","last_active":"2024-06-10T01:02:37.372069+0000","last_peered":"2024-06-10T01:02:37.372069+0000","last_clean":"2024-06-10T01:02:37.372069+0000","last_became_active":"2024-06-10T01:02:37.371599+0000","last_became_peered":"2024-06-10T01:02:37.371599+0000","last_unstale":"2024-06-10T01:02:37.372069+0000","last_undegraded":"2024-06-10T01:02:37.372069+0000","last_fullsized":"2024-06-10T01:02:37.372069+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:54:34.091576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.372531+0000","last_change":"2024-06-10T01:01:47.249553+0000","last_active":"2024-06-10T01:02:37.372531+0000","last_peered":"2024-06-10T01:02:37.372531+0000","last_clean":"2024-06-10T01:02:37.372531+0000","last_became_active":"2024-06-10T01:01:47.249202+0000","last_became_peered":"2024-06-10T01:01:47.249202+0000","last_unstale":"2024-06-10T01:02:37.372531+0000","last_undegraded":"2024-06-10T01:02:37.372531+0000","last_fullsized":"2024-06-10T01:02:37.372531+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:23:57.882264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.372324+0000","last_change":"2024-06-10T01:01:47.253987+0000","last_active":"2024-06-10T01:02:37.372324+0000","last_peered":"2024-06-10T01:02:37.372324+0000","last_clean":"2024-06-10T01:02:37.372324+0000","last_became_active":"2024-06-10T01:01:47.253578+0000","last_became_peered":"2024-06-10T01:01:47.253578+0000","last_unstale":"2024-06-10T01:02:37.372324+0000","last_undegraded":"2024-06-10T01:02:37.372324+0000","last_fullsized":"2024-06-10T01:02:37.372324+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:35:30.585482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.792280+0000","last_change":"2024-06-10T01:01:47.247089+0000","last_active":"2024-06-10T01:02:37.792280+0000","last_peered":"2024-06-10T01:02:37.792280+0000","last_clean":"2024-06-10T01:02:37.792280+0000","last_became_active":"2024-06-10T01:01:47.246066+0000","last_became_peered":"2024-06-10T01:01:47.246066+0000","last_unstale":"2024-06-10T01:02:37.792280+0000","last_undegraded":"2024-06-10T01:02:37.792280+0000","last_fullsized":"2024-06-10T01:02:37.792280+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:58:12.092981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.375425+0000","last_change":"2024-06-10T01:01:47.244491+0000","last_active":"2024-06-10T01:02:37.375425+0000","last_peered":"2024-06-10T01:02:37.375425+0000","last_clean":"2024-06-10T01:02:37.375425+0000","last_became_active":"2024-06-10T01:01:47.243747+0000","last_became_peered":"2024-06-10T01:01:47.243747+0000","last_unstale":"2024-06-10T01:02:37.375425+0000","last_undegraded":"2024-06-10T01:02:37.375425+0000","last_fullsized":"2024-06-10T01:02:37.375425+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:55:56.785016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.367139+0000","last_change":"2024-06-10T01:01:47.243141+0000","last_active":"2024-06-10T01:02:37.367139+0000","last_peered":"2024-06-10T01:02:37.367139+0000","last_clean":"2024-06-10T01:02:37.367139+0000","last_became_active":"2024-06-10T01:01:47.242464+0000","last_became_peered":"2024-06-10T01:01:47.242464+0000","last_unstale":"2024-06-10T01:02:37.367139+0000","last_undegraded":"2024-06-10T01:02:37.367139+0000","last_fullsized":"2024-06-10T01:02:37.367139+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:00:49.777130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.368037+0000","last_change":"2024-06-10T01:01:47.253463+0000","last_active":"2024-06-10T01:02:37.368037+0000","last_peered":"2024-06-10T01:02:37.368037+0000","last_clean":"2024-06-10T01:02:37.368037+0000","last_became_active":"2024-06-10T01:01:47.252929+0000","last_became_peered":"2024-06-10T01:01:47.252929+0000","last_unstale":"2024-06-10T01:02:37.368037+0000","last_undegraded":"2024-06-10T01:02:37.368037+0000","last_fullsized":"2024-06-10T01:02:37.368037+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:18:49.551276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.367213+0000","last_change":"2024-06-10T01:01:57.279561+0000","last_active":"2024-06-10T01:02:37.367213+0000","last_peered":"2024-06-10T01:02:37.367213+0000","last_clean":"2024-06-10T01:02:37.367213+0000","last_became_active":"2024-06-10T01:01:57.279132+0000","last_became_peered":"2024-06-10T01:01:57.279132+0000","last_unstale":"2024-06-10T01:02:37.367213+0000","last_undegraded":"2024-06-10T01:02:37.367213+0000","last_fullsized":"2024-06-10T01:02:37.367213+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:55:03.497701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'9","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:53.712475+0000","last_change":"2024-06-10T01:01:44.006667+0000","last_active":"2024-06-10T01:02:53.712475+0000","last_peered":"2024-06-10T01:02:53.712475+0000","last_clean":"2024-06-10T01:02:53.712475+0000","last_became_active":"2024-06-10T01:01:44.006343+0000","last_became_peered":"2024-06-10T01:01:44.006343+0000","last_unstale":"2024-06-10T01:02:53.712475+0000","last_undegraded":"2024-06-10T01:02:53.712475+0000","last_fullsized":"2024-06-10T01:02:53.712475+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:14:50.540084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.591617+0000","last_change":"2024-06-10T01:01:47.246139+0000","last_active":"2024-06-10T01:02:37.591617+0000","last_peered":"2024-06-10T01:02:37.591617+0000","last_clean":"2024-06-10T01:02:37.591617+0000","last_became_active":"2024-06-10T01:01:47.245743+0000","last_became_peered":"2024-06-10T01:01:47.245743+0000","last_unstale":"2024-06-10T01:02:37.591617+0000","last_undegraded":"2024-06-10T01:02:37.591617+0000","last_fullsized":"2024-06-10T01:02:37.591617+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:06:39.041569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.368964+0000","last_change":"2024-06-10T01:01:57.282843+0000","last_active":"2024-06-10T01:02:37.368964+0000","last_peered":"2024-06-10T01:02:37.368964+0000","last_clean":"2024-06-10T01:02:37.368964+0000","last_became_active":"2024-06-10T01:01:57.281797+0000","last_became_peered":"2024-06-10T01:01:57.281797+0000","last_unstale":"2024-06-10T01:02:37.368964+0000","last_undegraded":"2024-06-10T01:02:37.368964+0000","last_fullsized":"2024-06-10T01:02:37.368964+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:00.209628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.592003+0000","last_change":"2024-06-10T01:01:44.466714+0000","last_active":"2024-06-10T01:02:37.592003+0000","last_peered":"2024-06-10T01:02:37.592003+0000","last_clean":"2024-06-10T01:02:37.592003+0000","last_became_active":"2024-06-10T01:01:44.466296+0000","last_became_peered":"2024-06-10T01:01:44.466296+0000","last_unstale":"2024-06-10T01:02:37.592003+0000","last_undegraded":"2024-06-10T01:02:37.592003+0000","last_fullsized":"2024-06-10T01:02:37.592003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:16:36.082308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.373144+0000","last_change":"2024-06-10T01:01:47.244173+0000","last_active":"2024-06-10T01:02:37.373144+0000","last_peered":"2024-06-10T01:02:37.373144+0000","last_clean":"2024-06-10T01:02:37.373144+0000","last_became_active":"2024-06-10T01:01:47.243032+0000","last_became_peered":"2024-06-10T01:01:47.243032+0000","last_unstale":"2024-06-10T01:02:37.373144+0000","last_undegraded":"2024-06-10T01:02:37.373144+0000","last_fullsized":"2024-06-10T01:02:37.373144+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:14:33.965903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.592535+0000","last_change":"2024-06-10T01:02:37.592535+0000","last_active":"2024-06-10T01:02:37.592535+0000","last_peered":"2024-06-10T01:02:37.592535+0000","last_clean":"2024-06-10T01:02:37.592535+0000","last_became_active":"2024-06-10T01:02:37.592159+0000","last_became_peered":"2024-06-10T01:02:37.592159+0000","last_unstale":"2024-06-10T01:02:37.592535+0000","last_undegraded":"2024-06-10T01:02:37.592535+0000","last_fullsized":"2024-06-10T01:02:37.592535+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:50:08.626485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'4","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:53.709969+0000","last_change":"2024-06-10T01:01:44.009105+0000","last_active":"2024-06-10T01:02:53.709969+0000","last_peered":"2024-06-10T01:02:53.709969+0000","last_clean":"2024-06-10T01:02:53.709969+0000","last_became_active":"2024-06-10T01:01:44.008743+0000","last_became_peered":"2024-06-10T01:01:44.008743+0000","last_unstale":"2024-06-10T01:02:53.709969+0000","last_undegraded":"2024-06-10T01:02:53.709969+0000","last_fullsized":"2024-06-10T01:02:53.709969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:44:54.301378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.792728+0000","last_change":"2024-06-10T01:01:47.249007+0000","last_active":"2024-06-10T01:02:37.792728+0000","last_peered":"2024-06-10T01:02:37.792728+0000","last_clean":"2024-06-10T01:02:37.792728+0000","last_became_active":"2024-06-10T01:01:47.245632+0000","last_became_peered":"2024-06-10T01:01:47.245632+0000","last_unstale":"2024-06-10T01:02:37.792728+0000","last_undegraded":"2024-06-10T01:02:37.792728+0000","last_fullsized":"2024-06-10T01:02:37.792728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:29:18.916390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.368888+0000","last_change":"2024-06-10T01:01:57.283137+0000","last_active":"2024-06-10T01:02:37.368888+0000","last_peered":"2024-06-10T01:02:37.368888+0000","last_clean":"2024-06-10T01:02:37.368888+0000","last_became_active":"2024-06-10T01:01:57.282605+0000","last_became_peered":"2024-06-10T01:01:57.282605+0000","last_unstale":"2024-06-10T01:02:37.368888+0000","last_undegraded":"2024-06-10T01:02:37.368888+0000","last_fullsized":"2024-06-10T01:02:37.368888+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:26:16.967320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.369181+0000","last_change":"2024-06-10T01:01:44.010674+0000","last_active":"2024-06-10T01:02:37.369181+0000","last_peered":"2024-06-10T01:02:37.369181+0000","last_clean":"2024-06-10T01:02:37.369181+0000","last_became_active":"2024-06-10T01:01:44.010291+0000","last_became_peered":"2024-06-10T01:01:44.010291+0000","last_unstale":"2024-06-10T01:02:37.369181+0000","last_undegraded":"2024-06-10T01:02:37.369181+0000","last_fullsized":"2024-06-10T01:02:37.369181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:36:18.418362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.794580+0000","last_change":"2024-06-10T01:02:37.794580+0000","last_active":"2024-06-10T01:02:37.794580+0000","last_peered":"2024-06-10T01:02:37.794580+0000","last_clean":"2024-06-10T01:02:37.794580+0000","last_became_active":"2024-06-10T01:02:37.794127+0000","last_became_peered":"2024-06-10T01:02:37.794127+0000","last_unstale":"2024-06-10T01:02:37.794580+0000","last_undegraded":"2024-06-10T01:02:37.794580+0000","last_fullsized":"2024-06-10T01:02:37.794580+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:02.117871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.367911+0000","last_change":"2024-06-10T01:01:57.282494+0000","last_active":"2024-06-10T01:02:37.367911+0000","last_peered":"2024-06-10T01:02:37.367911+0000","last_clean":"2024-06-10T01:02:37.367911+0000","last_became_active":"2024-06-10T01:01:57.282077+0000","last_became_peered":"2024-06-10T01:01:57.282077+0000","last_unstale":"2024-06-10T01:02:37.367911+0000","last_undegraded":"2024-06-10T01:02:37.367911+0000","last_fullsized":"2024-06-10T01:02:37.367911+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:57:41.639788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"97'5","reported_seq":74,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:03:20.754051+0000","last_change":"2024-06-10T01:01:44.011134+0000","last_active":"2024-06-10T01:03:20.754051+0000","last_peered":"2024-06-10T01:03:20.754051+0000","last_clean":"2024-06-10T01:03:20.754051+0000","last_became_active":"2024-06-10T01:01:44.010788+0000","last_became_peered":"2024-06-10T01:01:44.010788+0000","last_unstale":"2024-06-10T01:03:20.754051+0000","last_undegraded":"2024-06-10T01:03:20.754051+0000","last_fullsized":"2024-06-10T01:03:20.754051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:22:55.228946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.369512+0000","last_change":"2024-06-10T01:01:47.248093+0000","last_active":"2024-06-10T01:02:37.369512+0000","last_peered":"2024-06-10T01:02:37.369512+0000","last_clean":"2024-06-10T01:02:37.369512+0000","last_became_active":"2024-06-10T01:01:47.247797+0000","last_became_peered":"2024-06-10T01:01:47.247797+0000","last_unstale":"2024-06-10T01:02:37.369512+0000","last_undegraded":"2024-06-10T01:02:37.369512+0000","last_fullsized":"2024-06-10T01:02:37.369512+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:45:50.159128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.377039+0000","last_change":"2024-06-10T01:02:37.377039+0000","last_active":"2024-06-10T01:02:37.377039+0000","last_peered":"2024-06-10T01:02:37.377039+0000","last_clean":"2024-06-10T01:02:37.377039+0000","last_became_active":"2024-06-10T01:02:37.376332+0000","last_became_peered":"2024-06-10T01:02:37.376332+0000","last_unstale":"2024-06-10T01:02:37.377039+0000","last_undegraded":"2024-06-10T01:02:37.377039+0000","last_fullsized":"2024-06-10T01:02:37.377039+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:15.000505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,10,5],"acting":[2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'3","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:53.710218+0000","last_change":"2024-06-10T01:01:44.008342+0000","last_active":"2024-06-10T01:02:53.710218+0000","last_peered":"2024-06-10T01:02:53.710218+0000","last_clean":"2024-06-10T01:02:53.710218+0000","last_became_active":"2024-06-10T01:01:44.007959+0000","last_became_peered":"2024-06-10T01:01:44.007959+0000","last_unstale":"2024-06-10T01:02:53.710218+0000","last_undegraded":"2024-06-10T01:02:53.710218+0000","last_fullsized":"2024-06-10T01:02:53.710218+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:36:03.143609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.372957+0000","last_change":"2024-06-10T01:01:47.247854+0000","last_active":"2024-06-10T01:02:37.372957+0000","last_peered":"2024-06-10T01:02:37.372957+0000","last_clean":"2024-06-10T01:02:37.372957+0000","last_became_active":"2024-06-10T01:01:47.247563+0000","last_became_peered":"2024-06-10T01:01:47.247563+0000","last_unstale":"2024-06-10T01:02:37.372957+0000","last_undegraded":"2024-06-10T01:02:37.372957+0000","last_fullsized":"2024-06-10T01:02:37.372957+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:32:13.024815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.375326+0000","last_change":"2024-06-10T01:01:57.282427+0000","last_active":"2024-06-10T01:02:37.375326+0000","last_peered":"2024-06-10T01:02:37.375326+0000","last_clean":"2024-06-10T01:02:37.375326+0000","last_became_active":"2024-06-10T01:01:57.281987+0000","last_became_peered":"2024-06-10T01:01:57.281987+0000","last_unstale":"2024-06-10T01:02:37.375326+0000","last_undegraded":"2024-06-10T01:02:37.375326+0000","last_fullsized":"2024-06-10T01:02:37.375326+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:22:31.736804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:53.709620+0000","last_change":"2024-06-10T01:01:44.006567+0000","last_active":"2024-06-10T01:02:53.709620+0000","last_peered":"2024-06-10T01:02:53.709620+0000","last_clean":"2024-06-10T01:02:53.709620+0000","last_became_active":"2024-06-10T01:01:44.006216+0000","last_became_peered":"2024-06-10T01:01:44.006216+0000","last_unstale":"2024-06-10T01:02:53.709620+0000","last_undegraded":"2024-06-10T01:02:53.709620+0000","last_fullsized":"2024-06-10T01:02:53.709620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:33:55.696873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.590667+0000","last_change":"2024-06-10T01:01:47.245326+0000","last_active":"2024-06-10T01:02:37.590667+0000","last_peered":"2024-06-10T01:02:37.590667+0000","last_clean":"2024-06-10T01:02:37.590667+0000","last_became_active":"2024-06-10T01:01:47.245013+0000","last_became_peered":"2024-06-10T01:01:47.245013+0000","last_unstale":"2024-06-10T01:02:37.590667+0000","last_undegraded":"2024-06-10T01:02:37.590667+0000","last_fullsized":"2024-06-10T01:02:37.590667+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:17:56.416627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.373215+0000","last_change":"2024-06-10T01:01:57.288997+0000","last_active":"2024-06-10T01:02:37.373215+0000","last_peered":"2024-06-10T01:02:37.373215+0000","last_clean":"2024-06-10T01:02:37.373215+0000","last_became_active":"2024-06-10T01:01:57.288596+0000","last_became_peered":"2024-06-10T01:01:57.288596+0000","last_unstale":"2024-06-10T01:02:37.373215+0000","last_undegraded":"2024-06-10T01:02:37.373215+0000","last_fullsized":"2024-06-10T01:02:37.373215+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:08:14.739737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.372286+0000","last_change":"2024-06-10T01:01:44.010079+0000","last_active":"2024-06-10T01:02:37.372286+0000","last_peered":"2024-06-10T01:02:37.372286+0000","last_clean":"2024-06-10T01:02:37.372286+0000","last_became_active":"2024-06-10T01:01:44.009752+0000","last_became_peered":"2024-06-10T01:01:44.009752+0000","last_unstale":"2024-06-10T01:02:37.372286+0000","last_undegraded":"2024-06-10T01:02:37.372286+0000","last_fullsized":"2024-06-10T01:02:37.372286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:29:35.660935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.366907+0000","last_change":"2024-06-10T01:01:47.245015+0000","last_active":"2024-06-10T01:02:37.366907+0000","last_peered":"2024-06-10T01:02:37.366907+0000","last_clean":"2024-06-10T01:02:37.366907+0000","last_became_active":"2024-06-10T01:01:47.244595+0000","last_became_peered":"2024-06-10T01:01:47.244595+0000","last_unstale":"2024-06-10T01:02:37.366907+0000","last_undegraded":"2024-06-10T01:02:37.366907+0000","last_fullsized":"2024-06-10T01:02:37.366907+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:01:48.446526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.591036+0000","last_change":"2024-06-10T01:01:57.281656+0000","last_active":"2024-06-10T01:02:37.591036+0000","last_peered":"2024-06-10T01:02:37.591036+0000","last_clean":"2024-06-10T01:02:37.591036+0000","last_became_active":"2024-06-10T01:01:57.281295+0000","last_became_peered":"2024-06-10T01:01:57.281295+0000","last_unstale":"2024-06-10T01:02:37.591036+0000","last_undegraded":"2024-06-10T01:02:37.591036+0000","last_fullsized":"2024-06-10T01:02:37.591036+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:04:41.619864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"97'3","reported_seq":69,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:03:19.752026+0000","last_change":"2024-06-10T01:01:44.011941+0000","last_active":"2024-06-10T01:03:19.752026+0000","last_peered":"2024-06-10T01:03:19.752026+0000","last_clean":"2024-06-10T01:03:19.752026+0000","last_became_active":"2024-06-10T01:01:44.010156+0000","last_became_peered":"2024-06-10T01:01:44.010156+0000","last_unstale":"2024-06-10T01:03:19.752026+0000","last_undegraded":"2024-06-10T01:03:19.752026+0000","last_fullsized":"2024-06-10T01:03:19.752026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:16:15.709544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.793074+0000","last_change":"2024-06-10T01:01:47.241720+0000","last_active":"2024-06-10T01:02:37.793074+0000","last_peered":"2024-06-10T01:02:37.793074+0000","last_clean":"2024-06-10T01:02:37.793074+0000","last_became_active":"2024-06-10T01:01:47.241386+0000","last_became_peered":"2024-06-10T01:01:47.241386+0000","last_unstale":"2024-06-10T01:02:37.793074+0000","last_undegraded":"2024-06-10T01:02:37.793074+0000","last_fullsized":"2024-06-10T01:02:37.793074+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:42:01.304787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.366174+0000","last_change":"2024-06-10T01:01:57.283226+0000","last_active":"2024-06-10T01:02:37.366174+0000","last_peered":"2024-06-10T01:02:37.366174+0000","last_clean":"2024-06-10T01:02:37.366174+0000","last_became_active":"2024-06-10T01:01:57.282720+0000","last_became_peered":"2024-06-10T01:01:57.282720+0000","last_unstale":"2024-06-10T01:02:37.366174+0000","last_undegraded":"2024-06-10T01:02:37.366174+0000","last_fullsized":"2024-06-10T01:02:37.366174+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:35:31.342042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:52.695045+0000","last_change":"2024-06-10T01:01:44.009278+0000","last_active":"2024-06-10T01:02:52.695045+0000","last_peered":"2024-06-10T01:02:52.695045+0000","last_clean":"2024-06-10T01:02:52.695045+0000","last_became_active":"2024-06-10T01:01:44.008841+0000","last_became_peered":"2024-06-10T01:01:44.008841+0000","last_unstale":"2024-06-10T01:02:52.695045+0000","last_undegraded":"2024-06-10T01:02:52.695045+0000","last_fullsized":"2024-06-10T01:02:52.695045+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:30:58.392080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.373314+0000","last_change":"2024-06-10T01:01:47.256451+0000","last_active":"2024-06-10T01:02:37.373314+0000","last_peered":"2024-06-10T01:02:37.373314+0000","last_clean":"2024-06-10T01:02:37.373314+0000","last_became_active":"2024-06-10T01:01:47.256036+0000","last_became_peered":"2024-06-10T01:01:47.256036+0000","last_unstale":"2024-06-10T01:02:37.373314+0000","last_undegraded":"2024-06-10T01:02:37.373314+0000","last_fullsized":"2024-06-10T01:02:37.373314+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:57:26.640109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.794160+0000","last_change":"2024-06-10T01:02:37.794160+0000","last_active":"2024-06-10T01:02:37.794160+0000","last_peered":"2024-06-10T01:02:37.794160+0000","last_clean":"2024-06-10T01:02:37.794160+0000","last_became_active":"2024-06-10T01:02:37.793633+0000","last_became_peered":"2024-06-10T01:02:37.793633+0000","last_unstale":"2024-06-10T01:02:37.794160+0000","last_undegraded":"2024-06-10T01:02:37.794160+0000","last_fullsized":"2024-06-10T01:02:37.794160+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:48:48.083620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:52.694810+0000","last_change":"2024-06-10T01:01:44.469179+0000","last_active":"2024-06-10T01:02:52.694810+0000","last_peered":"2024-06-10T01:02:52.694810+0000","last_clean":"2024-06-10T01:02:52.694810+0000","last_became_active":"2024-06-10T01:01:44.468669+0000","last_became_peered":"2024-06-10T01:01:44.468669+0000","last_unstale":"2024-06-10T01:02:52.694810+0000","last_undegraded":"2024-06-10T01:02:52.694810+0000","last_fullsized":"2024-06-10T01:02:52.694810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:07:51.629030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.792660+0000","last_change":"2024-06-10T01:01:47.247588+0000","last_active":"2024-06-10T01:02:37.792660+0000","last_peered":"2024-06-10T01:02:37.792660+0000","last_clean":"2024-06-10T01:02:37.792660+0000","last_became_active":"2024-06-10T01:01:47.245750+0000","last_became_peered":"2024-06-10T01:01:47.245750+0000","last_unstale":"2024-06-10T01:02:37.792660+0000","last_undegraded":"2024-06-10T01:02:37.792660+0000","last_fullsized":"2024-06-10T01:02:37.792660+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:12:10.513777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.366245+0000","last_change":"2024-06-10T01:01:57.283042+0000","last_active":"2024-06-10T01:02:37.366245+0000","last_peered":"2024-06-10T01:02:37.366245+0000","last_clean":"2024-06-10T01:02:37.366245+0000","last_became_active":"2024-06-10T01:01:57.282600+0000","last_became_peered":"2024-06-10T01:01:57.282600+0000","last_unstale":"2024-06-10T01:02:37.366245+0000","last_undegraded":"2024-06-10T01:02:37.366245+0000","last_fullsized":"2024-06-10T01:02:37.366245+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:06:36.969968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"97'33","reported_seq":133,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:03:20.753505+0000","last_change":"2024-06-10T01:01:44.011783+0000","last_active":"2024-06-10T01:03:20.753505+0000","last_peered":"2024-06-10T01:03:20.753505+0000","last_clean":"2024-06-10T01:03:20.753505+0000","last_became_active":"2024-06-10T01:01:44.011164+0000","last_became_peered":"2024-06-10T01:01:44.011164+0000","last_unstale":"2024-06-10T01:03:20.753505+0000","last_undegraded":"2024-06-10T01:03:20.753505+0000","last_fullsized":"2024-06-10T01:03:20.753505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:16:13.642714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59793,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":60,"num_read_kb":88,"num_write":34,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.369707+0000","last_change":"2024-06-10T01:01:47.250751+0000","last_active":"2024-06-10T01:02:37.369707+0000","last_peered":"2024-06-10T01:02:37.369707+0000","last_clean":"2024-06-10T01:02:37.369707+0000","last_became_active":"2024-06-10T01:01:47.250451+0000","last_became_peered":"2024-06-10T01:01:47.250451+0000","last_unstale":"2024-06-10T01:02:37.369707+0000","last_undegraded":"2024-06-10T01:02:37.369707+0000","last_fullsized":"2024-06-10T01:02:37.369707+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:22:10.121363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.374459+0000","last_change":"2024-06-10T01:01:57.289687+0000","last_active":"2024-06-10T01:02:37.374459+0000","last_peered":"2024-06-10T01:02:37.374459+0000","last_clean":"2024-06-10T01:02:37.374459+0000","last_became_active":"2024-06-10T01:01:57.289274+0000","last_became_peered":"2024-06-10T01:01:57.289274+0000","last_unstale":"2024-06-10T01:02:37.374459+0000","last_undegraded":"2024-06-10T01:02:37.374459+0000","last_fullsized":"2024-06-10T01:02:37.374459+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:34:46.873974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'6","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:53.710812+0000","last_change":"2024-06-10T01:01:44.010159+0000","last_active":"2024-06-10T01:02:53.710812+0000","last_peered":"2024-06-10T01:02:53.710812+0000","last_clean":"2024-06-10T01:02:53.710812+0000","last_became_active":"2024-06-10T01:01:44.009744+0000","last_became_peered":"2024-06-10T01:01:44.009744+0000","last_unstale":"2024-06-10T01:02:53.710812+0000","last_undegraded":"2024-06-10T01:02:53.710812+0000","last_fullsized":"2024-06-10T01:02:53.710812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:30.507613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.369580+0000","last_change":"2024-06-10T01:01:47.246662+0000","last_active":"2024-06-10T01:02:37.369580+0000","last_peered":"2024-06-10T01:02:37.369580+0000","last_clean":"2024-06-10T01:02:37.369580+0000","last_became_active":"2024-06-10T01:01:47.246268+0000","last_became_peered":"2024-06-10T01:01:47.246268+0000","last_unstale":"2024-06-10T01:02:37.369580+0000","last_undegraded":"2024-06-10T01:02:37.369580+0000","last_fullsized":"2024-06-10T01:02:37.369580+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:08:38.089054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.374428+0000","last_change":"2024-06-10T01:01:57.289937+0000","last_active":"2024-06-10T01:02:37.374428+0000","last_peered":"2024-06-10T01:02:37.374428+0000","last_clean":"2024-06-10T01:02:37.374428+0000","last_became_active":"2024-06-10T01:01:57.289407+0000","last_became_peered":"2024-06-10T01:01:57.289407+0000","last_unstale":"2024-06-10T01:02:37.374428+0000","last_undegraded":"2024-06-10T01:02:37.374428+0000","last_fullsized":"2024-06-10T01:02:37.374428+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:01:48.546762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:52.695496+0000","last_change":"2024-06-10T01:01:44.012654+0000","last_active":"2024-06-10T01:02:52.695496+0000","last_peered":"2024-06-10T01:02:52.695496+0000","last_clean":"2024-06-10T01:02:52.695496+0000","last_became_active":"2024-06-10T01:01:44.012246+0000","last_became_peered":"2024-06-10T01:01:44.012246+0000","last_unstale":"2024-06-10T01:02:52.695496+0000","last_undegraded":"2024-06-10T01:02:52.695496+0000","last_fullsized":"2024-06-10T01:02:52.695496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:31:36.173554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.974442+0000","last_change":"2024-06-10T01:01:47.250514+0000","last_active":"2024-06-10T01:02:37.974442+0000","last_peered":"2024-06-10T01:02:37.974442+0000","last_clean":"2024-06-10T01:02:37.974442+0000","last_became_active":"2024-06-10T01:01:47.249050+0000","last_became_peered":"2024-06-10T01:01:47.249050+0000","last_unstale":"2024-06-10T01:02:37.974442+0000","last_undegraded":"2024-06-10T01:02:37.974442+0000","last_fullsized":"2024-06-10T01:02:37.974442+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:51:30.789392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.367105+0000","last_change":"2024-06-10T01:01:57.282389+0000","last_active":"2024-06-10T01:02:37.367105+0000","last_peered":"2024-06-10T01:02:37.367105+0000","last_clean":"2024-06-10T01:02:37.367105+0000","last_became_active":"2024-06-10T01:01:57.281857+0000","last_became_peered":"2024-06-10T01:01:57.281857+0000","last_unstale":"2024-06-10T01:02:37.367105+0000","last_undegraded":"2024-06-10T01:02:37.367105+0000","last_fullsized":"2024-06-10T01:02:37.367105+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:59:14.863020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"97'3","reported_seq":68,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:03:16.749033+0000","last_change":"2024-06-10T01:01:44.011331+0000","last_active":"2024-06-10T01:03:16.749033+0000","last_peered":"2024-06-10T01:03:16.749033+0000","last_clean":"2024-06-10T01:03:16.749033+0000","last_became_active":"2024-06-10T01:01:44.010968+0000","last_became_peered":"2024-06-10T01:01:44.010968+0000","last_unstale":"2024-06-10T01:03:16.749033+0000","last_undegraded":"2024-06-10T01:03:16.749033+0000","last_fullsized":"2024-06-10T01:03:16.749033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:43:37.931225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.974460+0000","last_change":"2024-06-10T01:01:47.248349+0000","last_active":"2024-06-10T01:02:37.974460+0000","last_peered":"2024-06-10T01:02:37.974460+0000","last_clean":"2024-06-10T01:02:37.974460+0000","last_became_active":"2024-06-10T01:01:47.248008+0000","last_became_peered":"2024-06-10T01:01:47.248008+0000","last_unstale":"2024-06-10T01:02:37.974460+0000","last_undegraded":"2024-06-10T01:02:37.974460+0000","last_fullsized":"2024-06-10T01:02:37.974460+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:45:20.747704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.369929+0000","last_change":"2024-06-10T01:01:57.289773+0000","last_active":"2024-06-10T01:02:37.369929+0000","last_peered":"2024-06-10T01:02:37.369929+0000","last_clean":"2024-06-10T01:02:37.369929+0000","last_became_active":"2024-06-10T01:01:57.289371+0000","last_became_peered":"2024-06-10T01:01:57.289371+0000","last_unstale":"2024-06-10T01:02:37.369929+0000","last_undegraded":"2024-06-10T01:02:37.369929+0000","last_fullsized":"2024-06-10T01:02:37.369929+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:24:52.209402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'15","reported_seq":72,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:53.711765+0000","last_change":"2024-06-10T01:02:37.382175+0000","last_active":"2024-06-10T01:02:53.711765+0000","last_peered":"2024-06-10T01:02:53.711765+0000","last_clean":"2024-06-10T01:02:53.711765+0000","last_became_active":"2024-06-10T01:02:37.374506+0000","last_became_peered":"2024-06-10T01:02:37.374506+0000","last_unstale":"2024-06-10T01:02:53.711765+0000","last_undegraded":"2024-06-10T01:02:53.711765+0000","last_fullsized":"2024-06-10T01:02:53.711765+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:22:34.017650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.372221+0000","last_change":"2024-06-10T01:01:47.245439+0000","last_active":"2024-06-10T01:02:37.372221+0000","last_peered":"2024-06-10T01:02:37.372221+0000","last_clean":"2024-06-10T01:02:37.372221+0000","last_became_active":"2024-06-10T01:01:47.242917+0000","last_became_peered":"2024-06-10T01:01:47.242917+0000","last_unstale":"2024-06-10T01:02:37.372221+0000","last_undegraded":"2024-06-10T01:02:37.372221+0000","last_fullsized":"2024-06-10T01:02:37.372221+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:13:12.614230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.372096+0000","last_change":"2024-06-10T01:01:57.284731+0000","last_active":"2024-06-10T01:02:37.372096+0000","last_peered":"2024-06-10T01:02:37.372096+0000","last_clean":"2024-06-10T01:02:37.372096+0000","last_became_active":"2024-06-10T01:01:57.284301+0000","last_became_peered":"2024-06-10T01:01:57.284301+0000","last_unstale":"2024-06-10T01:02:37.372096+0000","last_undegraded":"2024-06-10T01:02:37.372096+0000","last_fullsized":"2024-06-10T01:02:37.372096+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:01:36.518214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:54.707212+0000","last_change":"2024-06-10T01:01:44.010154+0000","last_active":"2024-06-10T01:02:54.707212+0000","last_peered":"2024-06-10T01:02:54.707212+0000","last_clean":"2024-06-10T01:02:54.707212+0000","last_became_active":"2024-06-10T01:01:44.009656+0000","last_became_peered":"2024-06-10T01:01:44.009656+0000","last_unstale":"2024-06-10T01:02:54.707212+0000","last_undegraded":"2024-06-10T01:02:54.707212+0000","last_fullsized":"2024-06-10T01:02:54.707212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:33:08.789946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.378484+0000","last_change":"2024-06-10T01:02:37.378484+0000","last_active":"2024-06-10T01:02:37.378484+0000","last_peered":"2024-06-10T01:02:37.378484+0000","last_clean":"2024-06-10T01:02:37.378484+0000","last_became_active":"2024-06-10T01:02:37.378092+0000","last_became_peered":"2024-06-10T01:02:37.378092+0000","last_unstale":"2024-06-10T01:02:37.378484+0000","last_undegraded":"2024-06-10T01:02:37.378484+0000","last_fullsized":"2024-06-10T01:02:37.378484+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:19:46.599235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.369406+0000","last_change":"2024-06-10T01:01:57.332887+0000","last_active":"2024-06-10T01:02:37.369406+0000","last_peered":"2024-06-10T01:02:37.369406+0000","last_clean":"2024-06-10T01:02:37.369406+0000","last_became_active":"2024-06-10T01:01:57.332285+0000","last_became_peered":"2024-06-10T01:01:57.332285+0000","last_unstale":"2024-06-10T01:02:37.369406+0000","last_undegraded":"2024-06-10T01:02:37.369406+0000","last_fullsized":"2024-06-10T01:02:37.369406+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:09:12.233405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.366543+0000","last_change":"2024-06-10T01:01:44.466831+0000","last_active":"2024-06-10T01:02:37.366543+0000","last_peered":"2024-06-10T01:02:37.366543+0000","last_clean":"2024-06-10T01:02:37.366543+0000","last_became_active":"2024-06-10T01:01:44.466484+0000","last_became_peered":"2024-06-10T01:01:44.466484+0000","last_unstale":"2024-06-10T01:02:37.366543+0000","last_undegraded":"2024-06-10T01:02:37.366543+0000","last_fullsized":"2024-06-10T01:02:37.366543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:19:28.385504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.367319+0000","last_change":"2024-06-10T01:01:47.252399+0000","last_active":"2024-06-10T01:02:37.367319+0000","last_peered":"2024-06-10T01:02:37.367319+0000","last_clean":"2024-06-10T01:02:37.367319+0000","last_became_active":"2024-06-10T01:01:47.252069+0000","last_became_peered":"2024-06-10T01:01:47.252069+0000","last_unstale":"2024-06-10T01:02:37.367319+0000","last_undegraded":"2024-06-10T01:02:37.367319+0000","last_fullsized":"2024-06-10T01:02:37.367319+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:02:02.291174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.369699+0000","last_change":"2024-06-10T01:01:57.285670+0000","last_active":"2024-06-10T01:02:37.369699+0000","last_peered":"2024-06-10T01:02:37.369699+0000","last_clean":"2024-06-10T01:02:37.369699+0000","last_became_active":"2024-06-10T01:01:57.285281+0000","last_became_peered":"2024-06-10T01:01:57.285281+0000","last_unstale":"2024-06-10T01:02:37.369699+0000","last_undegraded":"2024-06-10T01:02:37.369699+0000","last_fullsized":"2024-06-10T01:02:37.369699+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:58.956934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'1","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.974404+0000","last_change":"2024-06-10T01:01:44.008013+0000","last_active":"2024-06-10T01:02:37.974404+0000","last_peered":"2024-06-10T01:02:37.974404+0000","last_clean":"2024-06-10T01:02:37.974404+0000","last_became_active":"2024-06-10T01:01:44.007591+0000","last_became_peered":"2024-06-10T01:01:44.007591+0000","last_unstale":"2024-06-10T01:02:37.974404+0000","last_undegraded":"2024-06-10T01:02:37.974404+0000","last_fullsized":"2024-06-10T01:02:37.974404+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:18:20.862340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.367130+0000","last_change":"2024-06-10T01:01:47.254756+0000","last_active":"2024-06-10T01:02:37.367130+0000","last_peered":"2024-06-10T01:02:37.367130+0000","last_clean":"2024-06-10T01:02:37.367130+0000","last_became_active":"2024-06-10T01:01:47.254418+0000","last_became_peered":"2024-06-10T01:01:47.254418+0000","last_unstale":"2024-06-10T01:02:37.367130+0000","last_undegraded":"2024-06-10T01:02:37.367130+0000","last_fullsized":"2024-06-10T01:02:37.367130+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:46:31.452539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.370058+0000","last_change":"2024-06-10T01:01:57.332792+0000","last_active":"2024-06-10T01:02:37.370058+0000","last_peered":"2024-06-10T01:02:37.370058+0000","last_clean":"2024-06-10T01:02:37.370058+0000","last_became_active":"2024-06-10T01:01:57.332331+0000","last_became_peered":"2024-06-10T01:01:57.332331+0000","last_unstale":"2024-06-10T01:02:37.370058+0000","last_undegraded":"2024-06-10T01:02:37.370058+0000","last_fullsized":"2024-06-10T01:02:37.370058+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:46:41.419807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.372568+0000","last_change":"2024-06-10T01:01:47.247857+0000","last_active":"2024-06-10T01:02:37.372568+0000","last_peered":"2024-06-10T01:02:37.372568+0000","last_clean":"2024-06-10T01:02:37.372568+0000","last_became_active":"2024-06-10T01:01:47.247552+0000","last_became_peered":"2024-06-10T01:01:47.247552+0000","last_unstale":"2024-06-10T01:02:37.372568+0000","last_undegraded":"2024-06-10T01:02:37.372568+0000","last_fullsized":"2024-06-10T01:02:37.372568+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:29:40.521887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"97'4","reported_seq":68,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:03:17.749968+0000","last_change":"2024-06-10T01:01:44.466631+0000","last_active":"2024-06-10T01:03:17.749968+0000","last_peered":"2024-06-10T01:03:17.749968+0000","last_clean":"2024-06-10T01:03:17.749968+0000","last_became_active":"2024-06-10T01:01:44.466280+0000","last_became_peered":"2024-06-10T01:01:44.466280+0000","last_unstale":"2024-06-10T01:03:17.749968+0000","last_undegraded":"2024-06-10T01:03:17.749968+0000","last_fullsized":"2024-06-10T01:03:17.749968+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:09:33.658112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":25,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.974760+0000","last_change":"2024-06-10T01:01:57.281973+0000","last_active":"2024-06-10T01:02:37.974760+0000","last_peered":"2024-06-10T01:02:37.974760+0000","last_clean":"2024-06-10T01:02:37.974760+0000","last_became_active":"2024-06-10T01:01:57.281386+0000","last_became_peered":"2024-06-10T01:01:57.281386+0000","last_unstale":"2024-06-10T01:02:37.974760+0000","last_undegraded":"2024-06-10T01:02:37.974760+0000","last_fullsized":"2024-06-10T01:02:37.974760+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:10:28.236250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.591038+0000","last_change":"2024-06-10T01:01:47.237436+0000","last_active":"2024-06-10T01:02:37.591038+0000","last_peered":"2024-06-10T01:02:37.591038+0000","last_clean":"2024-06-10T01:02:37.591038+0000","last_became_active":"2024-06-10T01:01:47.237099+0000","last_became_peered":"2024-06-10T01:01:47.237099+0000","last_unstale":"2024-06-10T01:02:37.591038+0000","last_undegraded":"2024-06-10T01:02:37.591038+0000","last_fullsized":"2024-06-10T01:02:37.591038+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:34:53.838877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'3","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:55.715038+0000","last_change":"2024-06-10T01:01:44.010655+0000","last_active":"2024-06-10T01:02:55.715038+0000","last_peered":"2024-06-10T01:02:55.715038+0000","last_clean":"2024-06-10T01:02:55.715038+0000","last_became_active":"2024-06-10T01:01:44.009918+0000","last_became_peered":"2024-06-10T01:01:44.009918+0000","last_unstale":"2024-06-10T01:02:55.715038+0000","last_undegraded":"2024-06-10T01:02:55.715038+0000","last_fullsized":"2024-06-10T01:02:55.715038+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:47:32.052650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":362,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.974272+0000","last_change":"2024-06-10T01:00:23.822447+0000","last_active":"2024-06-10T01:02:37.974272+0000","last_peered":"2024-06-10T01:02:37.974272+0000","last_clean":"2024-06-10T01:02:37.974272+0000","last_became_active":"2024-06-10T01:00:23.821966+0000","last_became_peered":"2024-06-10T01:00:23.821966+0000","last_unstale":"2024-06-10T01:02:37.974272+0000","last_undegraded":"2024-06-10T01:02:37.974272+0000","last_fullsized":"2024-06-10T01:02:37.974272+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:56:59.964661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:56:59.964661+0000","last_clean_scrub_stamp":"2024-06-10T00:56:59.964661+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:03:43.732875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.373138+0000","last_change":"2024-06-10T01:01:57.332980+0000","last_active":"2024-06-10T01:02:37.373138+0000","last_peered":"2024-06-10T01:02:37.373138+0000","last_clean":"2024-06-10T01:02:37.373138+0000","last_became_active":"2024-06-10T01:01:57.332437+0000","last_became_peered":"2024-06-10T01:01:57.332437+0000","last_unstale":"2024-06-10T01:02:37.373138+0000","last_undegraded":"2024-06-10T01:02:37.373138+0000","last_fullsized":"2024-06-10T01:02:37.373138+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:27:35.029079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.368754+0000","last_change":"2024-06-10T01:01:47.255926+0000","last_active":"2024-06-10T01:02:37.368754+0000","last_peered":"2024-06-10T01:02:37.368754+0000","last_clean":"2024-06-10T01:02:37.368754+0000","last_became_active":"2024-06-10T01:01:47.255571+0000","last_became_peered":"2024-06-10T01:01:47.255571+0000","last_unstale":"2024-06-10T01:02:37.368754+0000","last_undegraded":"2024-06-10T01:02:37.368754+0000","last_fullsized":"2024-06-10T01:02:37.368754+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:05:22.084037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'3","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:53.711111+0000","last_change":"2024-06-10T01:01:44.007509+0000","last_active":"2024-06-10T01:02:53.711111+0000","last_peered":"2024-06-10T01:02:53.711111+0000","last_clean":"2024-06-10T01:02:53.711111+0000","last_became_active":"2024-06-10T01:01:44.007178+0000","last_became_peered":"2024-06-10T01:01:44.007178+0000","last_unstale":"2024-06-10T01:02:53.711111+0000","last_undegraded":"2024-06-10T01:02:53.711111+0000","last_fullsized":"2024-06-10T01:02:53.711111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:02:52.791886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.374290+0000","last_change":"2024-06-10T01:01:57.285626+0000","last_active":"2024-06-10T01:02:37.374290+0000","last_peered":"2024-06-10T01:02:37.374290+0000","last_clean":"2024-06-10T01:02:37.374290+0000","last_became_active":"2024-06-10T01:01:57.285098+0000","last_became_peered":"2024-06-10T01:01:57.285098+0000","last_unstale":"2024-06-10T01:02:37.374290+0000","last_undegraded":"2024-06-10T01:02:37.374290+0000","last_fullsized":"2024-06-10T01:02:37.374290+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:13:49.881770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:52.694460+0000","last_change":"2024-06-10T01:01:44.469028+0000","last_active":"2024-06-10T01:02:52.694460+0000","last_peered":"2024-06-10T01:02:52.694460+0000","last_clean":"2024-06-10T01:02:52.694460+0000","last_became_active":"2024-06-10T01:01:44.468570+0000","last_became_peered":"2024-06-10T01:01:44.468570+0000","last_unstale":"2024-06-10T01:02:52.694460+0000","last_undegraded":"2024-06-10T01:02:52.694460+0000","last_fullsized":"2024-06-10T01:02:52.694460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:12:11.287959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.369854+0000","last_change":"2024-06-10T01:01:47.253414+0000","last_active":"2024-06-10T01:02:37.369854+0000","last_peered":"2024-06-10T01:02:37.369854+0000","last_clean":"2024-06-10T01:02:37.369854+0000","last_became_active":"2024-06-10T01:01:47.253060+0000","last_became_peered":"2024-06-10T01:01:47.253060+0000","last_unstale":"2024-06-10T01:02:37.369854+0000","last_undegraded":"2024-06-10T01:02:37.369854+0000","last_fullsized":"2024-06-10T01:02:37.369854+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:24:55.534164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.590637+0000","last_change":"2024-06-10T01:01:57.332756+0000","last_active":"2024-06-10T01:02:37.590637+0000","last_peered":"2024-06-10T01:02:37.590637+0000","last_clean":"2024-06-10T01:02:37.590637+0000","last_became_active":"2024-06-10T01:01:57.332361+0000","last_became_peered":"2024-06-10T01:01:57.332361+0000","last_unstale":"2024-06-10T01:02:37.590637+0000","last_undegraded":"2024-06-10T01:02:37.590637+0000","last_fullsized":"2024-06-10T01:02:37.590637+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:02:28.275407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"97'12","reported_seq":52,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:03:16.255913+0000","last_change":"2024-06-10T01:01:44.005164+0000","last_active":"2024-06-10T01:03:16.255913+0000","last_peered":"2024-06-10T01:03:16.255913+0000","last_clean":"2024-06-10T01:03:16.255913+0000","last_became_active":"2024-06-10T01:01:44.004816+0000","last_became_peered":"2024-06-10T01:01:44.004816+0000","last_unstale":"2024-06-10T01:03:16.255913+0000","last_undegraded":"2024-06-10T01:03:16.255913+0000","last_fullsized":"2024-06-10T01:03:16.255913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:11:56.385267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.592118+0000","last_change":"2024-06-10T01:01:47.247951+0000","last_active":"2024-06-10T01:02:37.592118+0000","last_peered":"2024-06-10T01:02:37.592118+0000","last_clean":"2024-06-10T01:02:37.592118+0000","last_became_active":"2024-06-10T01:01:47.246996+0000","last_became_peered":"2024-06-10T01:01:47.246996+0000","last_unstale":"2024-06-10T01:02:37.592118+0000","last_undegraded":"2024-06-10T01:02:37.592118+0000","last_fullsized":"2024-06-10T01:02:37.592118+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:39:18.167021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.366850+0000","last_change":"2024-06-10T01:01:57.278130+0000","last_active":"2024-06-10T01:02:37.366850+0000","last_peered":"2024-06-10T01:02:37.366850+0000","last_clean":"2024-06-10T01:02:37.366850+0000","last_became_active":"2024-06-10T01:01:57.277697+0000","last_became_peered":"2024-06-10T01:01:57.277697+0000","last_unstale":"2024-06-10T01:02:37.366850+0000","last_undegraded":"2024-06-10T01:02:37.366850+0000","last_fullsized":"2024-06-10T01:02:37.366850+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:17:01.662152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:52.697670+0000","last_change":"2024-06-10T01:01:44.553176+0000","last_active":"2024-06-10T01:02:52.697670+0000","last_peered":"2024-06-10T01:02:52.697670+0000","last_clean":"2024-06-10T01:02:52.697670+0000","last_became_active":"2024-06-10T01:01:44.552797+0000","last_became_peered":"2024-06-10T01:01:44.552797+0000","last_unstale":"2024-06-10T01:02:52.697670+0000","last_undegraded":"2024-06-10T01:02:52.697670+0000","last_fullsized":"2024-06-10T01:02:52.697670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:21:13.960304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.373615+0000","last_change":"2024-06-10T01:01:47.255796+0000","last_active":"2024-06-10T01:02:37.373615+0000","last_peered":"2024-06-10T01:02:37.373615+0000","last_clean":"2024-06-10T01:02:37.373615+0000","last_became_active":"2024-06-10T01:01:47.255495+0000","last_became_peered":"2024-06-10T01:01:47.255495+0000","last_unstale":"2024-06-10T01:02:37.373615+0000","last_undegraded":"2024-06-10T01:02:37.373615+0000","last_fullsized":"2024-06-10T01:02:37.373615+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:06:42.209005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.368892+0000","last_change":"2024-06-10T01:01:57.332869+0000","last_active":"2024-06-10T01:02:37.368892+0000","last_peered":"2024-06-10T01:02:37.368892+0000","last_clean":"2024-06-10T01:02:37.368892+0000","last_became_active":"2024-06-10T01:01:57.332391+0000","last_became_peered":"2024-06-10T01:01:57.332391+0000","last_unstale":"2024-06-10T01:02:37.368892+0000","last_undegraded":"2024-06-10T01:02:37.368892+0000","last_fullsized":"2024-06-10T01:02:37.368892+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:58:31.443626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:52.698476+0000","last_change":"2024-06-10T01:01:44.466861+0000","last_active":"2024-06-10T01:02:52.698476+0000","last_peered":"2024-06-10T01:02:52.698476+0000","last_clean":"2024-06-10T01:02:52.698476+0000","last_became_active":"2024-06-10T01:01:44.466553+0000","last_became_peered":"2024-06-10T01:01:44.466553+0000","last_unstale":"2024-06-10T01:02:52.698476+0000","last_undegraded":"2024-06-10T01:02:52.698476+0000","last_fullsized":"2024-06-10T01:02:52.698476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:55:52.545594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.368847+0000","last_change":"2024-06-10T01:01:47.244972+0000","last_active":"2024-06-10T01:02:37.368847+0000","last_peered":"2024-06-10T01:02:37.368847+0000","last_clean":"2024-06-10T01:02:37.368847+0000","last_became_active":"2024-06-10T01:01:47.244649+0000","last_became_peered":"2024-06-10T01:01:47.244649+0000","last_unstale":"2024-06-10T01:02:37.368847+0000","last_undegraded":"2024-06-10T01:02:37.368847+0000","last_fullsized":"2024-06-10T01:02:37.368847+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:10:46.909200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.593332+0000","last_change":"2024-06-10T01:02:37.592380+0000","last_active":"2024-06-10T01:02:37.593332+0000","last_peered":"2024-06-10T01:02:37.593332+0000","last_clean":"2024-06-10T01:02:37.593332+0000","last_became_active":"2024-06-10T01:02:37.591856+0000","last_became_peered":"2024-06-10T01:02:37.591856+0000","last_unstale":"2024-06-10T01:02:37.593332+0000","last_undegraded":"2024-06-10T01:02:37.593332+0000","last_fullsized":"2024-06-10T01:02:37.593332+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:00:25.717192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:52.699377+0000","last_change":"2024-06-10T01:01:44.008116+0000","last_active":"2024-06-10T01:02:52.699377+0000","last_peered":"2024-06-10T01:02:52.699377+0000","last_clean":"2024-06-10T01:02:52.699377+0000","last_became_active":"2024-06-10T01:01:44.007778+0000","last_became_peered":"2024-06-10T01:01:44.007778+0000","last_unstale":"2024-06-10T01:02:52.699377+0000","last_undegraded":"2024-06-10T01:02:52.699377+0000","last_fullsized":"2024-06-10T01:02:52.699377+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:18:57.267296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.374063+0000","last_change":"2024-06-10T01:01:47.252833+0000","last_active":"2024-06-10T01:02:37.374063+0000","last_peered":"2024-06-10T01:02:37.374063+0000","last_clean":"2024-06-10T01:02:37.374063+0000","last_became_active":"2024-06-10T01:01:47.252326+0000","last_became_peered":"2024-06-10T01:01:47.252326+0000","last_unstale":"2024-06-10T01:02:37.374063+0000","last_undegraded":"2024-06-10T01:02:37.374063+0000","last_fullsized":"2024-06-10T01:02:37.374063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:00:42.673838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":25,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.369673+0000","last_change":"2024-06-10T01:01:57.275975+0000","last_active":"2024-06-10T01:02:37.369673+0000","last_peered":"2024-06-10T01:02:37.369673+0000","last_clean":"2024-06-10T01:02:37.369673+0000","last_became_active":"2024-06-10T01:01:57.275408+0000","last_became_peered":"2024-06-10T01:01:57.275408+0000","last_unstale":"2024-06-10T01:02:37.369673+0000","last_undegraded":"2024-06-10T01:02:37.369673+0000","last_fullsized":"2024-06-10T01:02:37.369673+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:12:42.376636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'1","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.367168+0000","last_change":"2024-06-10T01:01:44.006246+0000","last_active":"2024-06-10T01:02:37.367168+0000","last_peered":"2024-06-10T01:02:37.367168+0000","last_clean":"2024-06-10T01:02:37.367168+0000","last_became_active":"2024-06-10T01:01:44.005852+0000","last_became_peered":"2024-06-10T01:01:44.005852+0000","last_unstale":"2024-06-10T01:02:37.367168+0000","last_undegraded":"2024-06-10T01:02:37.367168+0000","last_fullsized":"2024-06-10T01:02:37.367168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:56:47.260279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.591178+0000","last_change":"2024-06-10T01:01:47.243154+0000","last_active":"2024-06-10T01:02:37.591178+0000","last_peered":"2024-06-10T01:02:37.591178+0000","last_clean":"2024-06-10T01:02:37.591178+0000","last_became_active":"2024-06-10T01:01:47.242851+0000","last_became_peered":"2024-06-10T01:01:47.242851+0000","last_unstale":"2024-06-10T01:02:37.591178+0000","last_undegraded":"2024-06-10T01:02:37.591178+0000","last_fullsized":"2024-06-10T01:02:37.591178+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:41:05.857777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.369132+0000","last_change":"2024-06-10T01:01:57.333613+0000","last_active":"2024-06-10T01:02:37.369132+0000","last_peered":"2024-06-10T01:02:37.369132+0000","last_clean":"2024-06-10T01:02:37.369132+0000","last_became_active":"2024-06-10T01:01:57.333019+0000","last_became_peered":"2024-06-10T01:01:57.333019+0000","last_unstale":"2024-06-10T01:02:37.369132+0000","last_undegraded":"2024-06-10T01:02:37.369132+0000","last_fullsized":"2024-06-10T01:02:37.369132+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:11.912885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:52.696696+0000","last_change":"2024-06-10T01:01:44.008221+0000","last_active":"2024-06-10T01:02:52.696696+0000","last_peered":"2024-06-10T01:02:52.696696+0000","last_clean":"2024-06-10T01:02:52.696696+0000","last_became_active":"2024-06-10T01:01:44.007880+0000","last_became_peered":"2024-06-10T01:01:44.007880+0000","last_unstale":"2024-06-10T01:02:52.696696+0000","last_undegraded":"2024-06-10T01:02:52.696696+0000","last_fullsized":"2024-06-10T01:02:52.696696+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:40:33.162925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.974467+0000","last_change":"2024-06-10T01:01:47.254902+0000","last_active":"2024-06-10T01:02:37.974467+0000","last_peered":"2024-06-10T01:02:37.974467+0000","last_clean":"2024-06-10T01:02:37.974467+0000","last_became_active":"2024-06-10T01:01:47.254555+0000","last_became_peered":"2024-06-10T01:01:47.254555+0000","last_unstale":"2024-06-10T01:02:37.974467+0000","last_undegraded":"2024-06-10T01:02:37.974467+0000","last_fullsized":"2024-06-10T01:02:37.974467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:03:17.571001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.792433+0000","last_change":"2024-06-10T01:01:57.284491+0000","last_active":"2024-06-10T01:02:37.792433+0000","last_peered":"2024-06-10T01:02:37.792433+0000","last_clean":"2024-06-10T01:02:37.792433+0000","last_became_active":"2024-06-10T01:01:57.283795+0000","last_became_peered":"2024-06-10T01:01:57.283795+0000","last_unstale":"2024-06-10T01:02:37.792433+0000","last_undegraded":"2024-06-10T01:02:37.792433+0000","last_fullsized":"2024-06-10T01:02:37.792433+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:49:00.381664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'3","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:53.710646+0000","last_change":"2024-06-10T01:01:44.553159+0000","last_active":"2024-06-10T01:02:53.710646+0000","last_peered":"2024-06-10T01:02:53.710646+0000","last_clean":"2024-06-10T01:02:53.710646+0000","last_became_active":"2024-06-10T01:01:44.552759+0000","last_became_peered":"2024-06-10T01:01:44.552759+0000","last_unstale":"2024-06-10T01:02:53.710646+0000","last_undegraded":"2024-06-10T01:02:53.710646+0000","last_fullsized":"2024-06-10T01:02:53.710646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:19:00.110790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.974880+0000","last_change":"2024-06-10T01:01:47.247803+0000","last_active":"2024-06-10T01:02:37.974880+0000","last_peered":"2024-06-10T01:02:37.974880+0000","last_clean":"2024-06-10T01:02:37.974880+0000","last_became_active":"2024-06-10T01:01:47.246715+0000","last_became_peered":"2024-06-10T01:01:47.246715+0000","last_unstale":"2024-06-10T01:02:37.974880+0000","last_undegraded":"2024-06-10T01:02:37.974880+0000","last_fullsized":"2024-06-10T01:02:37.974880+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:22.429002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.372674+0000","last_change":"2024-06-10T01:01:57.287668+0000","last_active":"2024-06-10T01:02:37.372674+0000","last_peered":"2024-06-10T01:02:37.372674+0000","last_clean":"2024-06-10T01:02:37.372674+0000","last_became_active":"2024-06-10T01:01:57.287060+0000","last_became_peered":"2024-06-10T01:01:57.287060+0000","last_unstale":"2024-06-10T01:02:37.372674+0000","last_undegraded":"2024-06-10T01:02:37.372674+0000","last_fullsized":"2024-06-10T01:02:37.372674+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:04:35.883613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.374270+0000","last_change":"2024-06-10T01:01:44.013051+0000","last_active":"2024-06-10T01:02:37.374270+0000","last_peered":"2024-06-10T01:02:37.374270+0000","last_clean":"2024-06-10T01:02:37.374270+0000","last_became_active":"2024-06-10T01:01:44.012667+0000","last_became_peered":"2024-06-10T01:01:44.012667+0000","last_unstale":"2024-06-10T01:02:37.374270+0000","last_undegraded":"2024-06-10T01:02:37.374270+0000","last_fullsized":"2024-06-10T01:02:37.374270+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:10:49.650082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.974462+0000","last_change":"2024-06-10T01:01:47.251209+0000","last_active":"2024-06-10T01:02:37.974462+0000","last_peered":"2024-06-10T01:02:37.974462+0000","last_clean":"2024-06-10T01:02:37.974462+0000","last_became_active":"2024-06-10T01:01:47.250858+0000","last_became_peered":"2024-06-10T01:01:47.250858+0000","last_unstale":"2024-06-10T01:02:37.974462+0000","last_undegraded":"2024-06-10T01:02:37.974462+0000","last_fullsized":"2024-06-10T01:02:37.974462+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:47.272727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.371843+0000","last_change":"2024-06-10T01:01:57.284187+0000","last_active":"2024-06-10T01:02:37.371843+0000","last_peered":"2024-06-10T01:02:37.371843+0000","last_clean":"2024-06-10T01:02:37.371843+0000","last_became_active":"2024-06-10T01:01:57.283739+0000","last_became_peered":"2024-06-10T01:01:57.283739+0000","last_unstale":"2024-06-10T01:02:37.371843+0000","last_undegraded":"2024-06-10T01:02:37.371843+0000","last_fullsized":"2024-06-10T01:02:37.371843+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:13:34.630708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.370086+0000","last_change":"2024-06-10T01:01:44.005702+0000","last_active":"2024-06-10T01:02:37.370086+0000","last_peered":"2024-06-10T01:02:37.370086+0000","last_clean":"2024-06-10T01:02:37.370086+0000","last_became_active":"2024-06-10T01:01:44.005354+0000","last_became_peered":"2024-06-10T01:01:44.005354+0000","last_unstale":"2024-06-10T01:02:37.370086+0000","last_undegraded":"2024-06-10T01:02:37.370086+0000","last_fullsized":"2024-06-10T01:02:37.370086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:51:14.819688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.974705+0000","last_change":"2024-06-10T01:01:47.250325+0000","last_active":"2024-06-10T01:02:37.974705+0000","last_peered":"2024-06-10T01:02:37.974705+0000","last_clean":"2024-06-10T01:02:37.974705+0000","last_became_active":"2024-06-10T01:01:47.249591+0000","last_became_peered":"2024-06-10T01:01:47.249591+0000","last_unstale":"2024-06-10T01:02:37.974705+0000","last_undegraded":"2024-06-10T01:02:37.974705+0000","last_fullsized":"2024-06-10T01:02:37.974705+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:17:16.268717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.374167+0000","last_change":"2024-06-10T01:01:47.253449+0000","last_active":"2024-06-10T01:02:37.374167+0000","last_peered":"2024-06-10T01:02:37.374167+0000","last_clean":"2024-06-10T01:02:37.374167+0000","last_became_active":"2024-06-10T01:01:47.253144+0000","last_became_peered":"2024-06-10T01:01:47.253144+0000","last_unstale":"2024-06-10T01:02:37.374167+0000","last_undegraded":"2024-06-10T01:02:37.374167+0000","last_fullsized":"2024-06-10T01:02:37.374167+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:32:22.353409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.367553+0000","last_change":"2024-06-10T01:01:47.258241+0000","last_active":"2024-06-10T01:02:37.367553+0000","last_peered":"2024-06-10T01:02:37.367553+0000","last_clean":"2024-06-10T01:02:37.367553+0000","last_became_active":"2024-06-10T01:01:47.257823+0000","last_became_peered":"2024-06-10T01:01:47.257823+0000","last_unstale":"2024-06-10T01:02:37.367553+0000","last_undegraded":"2024-06-10T01:02:37.367553+0000","last_fullsized":"2024-06-10T01:02:37.367553+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:11:23.652479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.366971+0000","last_change":"2024-06-10T01:01:47.256218+0000","last_active":"2024-06-10T01:02:37.366971+0000","last_peered":"2024-06-10T01:02:37.366971+0000","last_clean":"2024-06-10T01:02:37.366971+0000","last_became_active":"2024-06-10T01:01:47.255950+0000","last_became_peered":"2024-06-10T01:01:47.255950+0000","last_unstale":"2024-06-10T01:02:37.366971+0000","last_undegraded":"2024-06-10T01:02:37.366971+0000","last_fullsized":"2024-06-10T01:02:37.366971+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:59:37.283646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.373530+0000","last_change":"2024-06-10T01:01:47.253109+0000","last_active":"2024-06-10T01:02:37.373530+0000","last_peered":"2024-06-10T01:02:37.373530+0000","last_clean":"2024-06-10T01:02:37.373530+0000","last_became_active":"2024-06-10T01:01:47.252767+0000","last_became_peered":"2024-06-10T01:01:47.252767+0000","last_unstale":"2024-06-10T01:02:37.373530+0000","last_undegraded":"2024-06-10T01:02:37.373530+0000","last_fullsized":"2024-06-10T01:02:37.373530+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:57:13.177230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.375094+0000","last_change":"2024-06-10T01:02:37.375094+0000","last_active":"2024-06-10T01:02:37.375094+0000","last_peered":"2024-06-10T01:02:37.375094+0000","last_clean":"2024-06-10T01:02:37.375094+0000","last_became_active":"2024-06-10T01:02:37.374527+0000","last_became_peered":"2024-06-10T01:02:37.374527+0000","last_unstale":"2024-06-10T01:02:37.375094+0000","last_undegraded":"2024-06-10T01:02:37.375094+0000","last_fullsized":"2024-06-10T01:02:37.375094+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:34:41.205816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.374992+0000","last_change":"2024-06-10T01:01:47.250243+0000","last_active":"2024-06-10T01:02:37.374992+0000","last_peered":"2024-06-10T01:02:37.374992+0000","last_clean":"2024-06-10T01:02:37.374992+0000","last_became_active":"2024-06-10T01:01:47.249907+0000","last_became_peered":"2024-06-10T01:01:47.249907+0000","last_unstale":"2024-06-10T01:02:37.374992+0000","last_undegraded":"2024-06-10T01:02:37.374992+0000","last_fullsized":"2024-06-10T01:02:37.374992+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:31:32.933660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.368310+0000","last_change":"2024-06-10T01:01:47.254173+0000","last_active":"2024-06-10T01:02:37.368310+0000","last_peered":"2024-06-10T01:02:37.368310+0000","last_clean":"2024-06-10T01:02:37.368310+0000","last_became_active":"2024-06-10T01:01:47.253796+0000","last_became_peered":"2024-06-10T01:01:47.253796+0000","last_unstale":"2024-06-10T01:02:37.368310+0000","last_undegraded":"2024-06-10T01:02:37.368310+0000","last_fullsized":"2024-06-10T01:02:37.368310+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:41:12.199114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.592242+0000","last_change":"2024-06-10T01:01:47.253920+0000","last_active":"2024-06-10T01:02:37.592242+0000","last_peered":"2024-06-10T01:02:37.592242+0000","last_clean":"2024-06-10T01:02:37.592242+0000","last_became_active":"2024-06-10T01:01:47.253288+0000","last_became_peered":"2024-06-10T01:01:47.253288+0000","last_unstale":"2024-06-10T01:02:37.592242+0000","last_undegraded":"2024-06-10T01:02:37.592242+0000","last_fullsized":"2024-06-10T01:02:37.592242+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:41:20.406263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.372341+0000","last_change":"2024-06-10T01:01:47.240772+0000","last_active":"2024-06-10T01:02:37.372341+0000","last_peered":"2024-06-10T01:02:37.372341+0000","last_clean":"2024-06-10T01:02:37.372341+0000","last_became_active":"2024-06-10T01:01:47.238258+0000","last_became_peered":"2024-06-10T01:01:47.238258+0000","last_unstale":"2024-06-10T01:02:37.372341+0000","last_undegraded":"2024-06-10T01:02:37.372341+0000","last_fullsized":"2024-06-10T01:02:37.372341+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:55:25.211434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.377151+0000","last_change":"2024-06-10T01:02:37.376724+0000","last_active":"2024-06-10T01:02:37.377151+0000","last_peered":"2024-06-10T01:02:37.377151+0000","last_clean":"2024-06-10T01:02:37.377151+0000","last_became_active":"2024-06-10T01:02:37.376329+0000","last_became_peered":"2024-06-10T01:02:37.376329+0000","last_unstale":"2024-06-10T01:02:37.377151+0000","last_undegraded":"2024-06-10T01:02:37.377151+0000","last_fullsized":"2024-06-10T01:02:37.377151+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:38:21.837958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.792781+0000","last_change":"2024-06-10T01:01:47.244463+0000","last_active":"2024-06-10T01:02:37.792781+0000","last_peered":"2024-06-10T01:02:37.792781+0000","last_clean":"2024-06-10T01:02:37.792781+0000","last_became_active":"2024-06-10T01:01:47.244114+0000","last_became_peered":"2024-06-10T01:01:47.244114+0000","last_unstale":"2024-06-10T01:02:37.792781+0000","last_undegraded":"2024-06-10T01:02:37.792781+0000","last_fullsized":"2024-06-10T01:02:37.792781+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:47:37.098894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.370498+0000","last_change":"2024-06-10T01:01:47.248317+0000","last_active":"2024-06-10T01:02:37.370498+0000","last_peered":"2024-06-10T01:02:37.370498+0000","last_clean":"2024-06-10T01:02:37.370498+0000","last_became_active":"2024-06-10T01:01:47.247958+0000","last_became_peered":"2024-06-10T01:01:47.247958+0000","last_unstale":"2024-06-10T01:02:37.370498+0000","last_undegraded":"2024-06-10T01:02:37.370498+0000","last_fullsized":"2024-06-10T01:02:37.370498+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:39:59.267957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T01:02:37.374610+0000","last_change":"2024-06-10T01:01:47.252542+0000","last_active":"2024-06-10T01:02:37.374610+0000","last_peered":"2024-06-10T01:02:37.374610+0000","last_clean":"2024-06-10T01:02:37.374610+0000","last_became_active":"2024-06-10T01:01:47.252216+0000","last_became_peered":"2024-06-10T01:01:47.252216+0000","last_unstale":"2024-06-10T01:02:37.374610+0000","last_undegraded":"2024-06-10T01:02:37.374610+0000","last_fullsized":"2024-06-10T01:02:37.374610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:12:22.196263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285691,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63403,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":178,"num_read_kb":156,"num_write":169,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":190209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383714,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":550999,"data_compressed":26362,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645351,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32360,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712792,"statfs":{"total":95995035648,"available":95961899008,"internally_reserved":0,"allocated":1212416,"data_stored":1198060,"data_compressed":62688,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808876,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1089536,"data_stored":555331,"data_compressed":26362,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070513,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":551155,"data_compressed":26362,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332149,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":1146673,"data_compressed":62688,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495674,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":546411,"data_compressed":26362,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1130496,"data_stored":606114,"data_compressed":26362,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1089536,"data_stored":547799,"data_compressed":26362,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1146880,"data_stored":607352,"data_compressed":26362,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":546605,"data_compressed":26362,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920836,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":551173,"data_compressed":26362,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757312,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1142784,"data_stored":1142261,"data_compressed":62688,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34687,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34687,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34687,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":61031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T01:03:24.933 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-10T01:03:24.934 INFO:tasks.cephadm:Setting up client nodes... 2024-06-10T01:03:24.934 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-06-10T01:03:25.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:24 smithi078 ceph-mon[36683]: pgmap v351: 129 pgs: 129 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-06-10T01:03:25.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:24 smithi078 ceph-mon[36683]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T01:03:25.193 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:03:25.205 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:24 smithi175 ceph-mon[36636]: pgmap v351: 129 pgs: 129 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-06-10T01:03:25.205 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:24 smithi175 ceph-mon[36636]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T01:03:26.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.866+0000 7f09bb59e700 1 -- 172.21.15.7:0/3410547984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09bc103ef0 msgr2=0x7f09bc10a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:26.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.866+0000 7f09bb59e700 1 --2- 172.21.15.7:0/3410547984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09bc103ef0 0x7f09bc10a7a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f09b400b0d0 tx=0x7f09b400b3e0 comp rx=0 tx=0).stop 2024-06-10T01:03:26.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.866+0000 7f09bb59e700 1 -- 172.21.15.7:0/3410547984 shutdown_connections 2024-06-10T01:03:26.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.866+0000 7f09bb59e700 1 --2- 172.21.15.7:0/3410547984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09bc103ef0 0x7f09bc10a7a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:26.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.866+0000 7f09bb59e700 1 --2- 172.21.15.7:0/3410547984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09bc102350 0x7f09bc102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:26.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.866+0000 7f09bb59e700 1 --2- 172.21.15.7:0/3410547984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09bc103550 0x7f09bc1039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:26.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.866+0000 7f09bb59e700 1 -- 172.21.15.7:0/3410547984 >> 172.21.15.7:0/3410547984 conn(0x7f09bc0fd8d0 msgr2=0x7f09bc0ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:26.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.866+0000 7f09bb59e700 1 -- 172.21.15.7:0/3410547984 shutdown_connections 2024-06-10T01:03:26.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.866+0000 7f09bb59e700 1 -- 172.21.15.7:0/3410547984 wait complete. 2024-06-10T01:03:26.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.867+0000 7f09bb59e700 1 Processor -- start 2024-06-10T01:03:26.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.867+0000 7f09bb59e700 1 -- start start 2024-06-10T01:03:26.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.867+0000 7f09bb59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09bc102350 0x7f09bc198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:26.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.867+0000 7f09bb59e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09bc103550 0x7f09bc198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:26.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.867+0000 7f09bb59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09bc103ef0 0x7f09bc19f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:26.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.867+0000 7f09bb59e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f09bc10df40 con 0x7f09bc102350 2024-06-10T01:03:26.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.867+0000 7f09bb59e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f09bc10ddc0 con 0x7f09bc103ef0 2024-06-10T01:03:26.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.867+0000 7f09bb59e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f09bc10e0c0 con 0x7f09bc103550 2024-06-10T01:03:26.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.867+0000 7f09ba59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09bc102350 0x7f09bc198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:26.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.867+0000 7f09ba59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09bc102350 0x7f09bc198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40872/0 (socket says 172.21.15.7:40872) 2024-06-10T01:03:26.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.868+0000 7f09ba59c700 1 -- 172.21.15.7:0/96905346 learned_addr learned my addr 172.21.15.7:0/96905346 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:03:26.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.868+0000 7f09b9d9b700 1 --2- 172.21.15.7:0/96905346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09bc103550 0x7f09bc198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:26.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.868+0000 7f09bad9d700 1 --2- 172.21.15.7:0/96905346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09bc103ef0 0x7f09bc19f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:26.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.868+0000 7f09ba59c700 1 -- 172.21.15.7:0/96905346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09bc103550 msgr2=0x7f09bc198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:26.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.868+0000 7f09ba59c700 1 --2- 172.21.15.7:0/96905346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09bc103550 0x7f09bc198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:26.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.868+0000 7f09ba59c700 1 -- 172.21.15.7:0/96905346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09bc103ef0 msgr2=0x7f09bc19f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:26.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.868+0000 7f09ba59c700 1 --2- 172.21.15.7:0/96905346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09bc103ef0 0x7f09bc19f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:26.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.868+0000 7f09ba59c700 1 -- 172.21.15.7:0/96905346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09bc1a0060 con 0x7f09bc102350 2024-06-10T01:03:26.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.868+0000 7f09bad9d700 1 --2- 172.21.15.7:0/96905346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09bc103ef0 0x7f09bc19f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:03:26.868 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:26 smithi007 ceph-mon[32398]: pgmap v352: 129 pgs: 129 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-06-10T01:03:26.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.868+0000 7f09ba59c700 1 --2- 172.21.15.7:0/96905346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09bc102350 0x7f09bc198400 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f09ac00d9b0 tx=0x7f09ac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:26.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.868+0000 7f09ab7fe700 1 -- 172.21.15.7:0/96905346 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ac017070 con 0x7f09bc102350 2024-06-10T01:03:26.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.868+0000 7f09bb59e700 1 -- 172.21.15.7:0/96905346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09bc1a0350 con 0x7f09bc102350 2024-06-10T01:03:26.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.868+0000 7f09bb59e700 1 -- 172.21.15.7:0/96905346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09bc1a0890 con 0x7f09bc102350 2024-06-10T01:03:26.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.869+0000 7f09ab7fe700 1 -- 172.21.15.7:0/96905346 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09ac009e10 con 0x7f09bc102350 2024-06-10T01:03:26.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.869+0000 7f09ab7fe700 1 -- 172.21.15.7:0/96905346 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ac00f620 con 0x7f09bc102350 2024-06-10T01:03:26.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.870+0000 7f09bb59e700 1 -- 172.21.15.7:0/96905346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f099c005360 con 0x7f09bc102350 2024-06-10T01:03:26.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.871+0000 7f09ab7fe700 1 -- 172.21.15.7:0/96905346 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f09ac010710 con 0x7f09bc102350 2024-06-10T01:03:26.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.872+0000 7f09ab7fe700 1 --2- 172.21.15.7:0/96905346 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f09a40a2de0 0x7f09a40a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:26.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.872+0000 7f09b9d9b700 1 --2- 172.21.15.7:0/96905346 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f09a40a2de0 0x7f09a40a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:26.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.872+0000 7f09ab7fe700 1 -- 172.21.15.7:0/96905346 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f09ac0cef50 con 0x7f09bc102350 2024-06-10T01:03:26.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.873+0000 7f09b9d9b700 1 --2- 172.21.15.7:0/96905346 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f09a40a2de0 0x7f09a40a52a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f09b0007be0 tx=0x7f09b00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:26.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:26.875+0000 7f09ab7fe700 1 -- 172.21.15.7:0/96905346 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f09ac09ba70 con 0x7f09bc102350 2024-06-10T01:03:27.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:26 smithi078 ceph-mon[36683]: pgmap v352: 129 pgs: 129 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-06-10T01:03:27.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:27.167+0000 7f09bb59e700 1 -- 172.21.15.7:0/96905346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f099c005650 con 0x7f09bc102350 2024-06-10T01:03:27.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:27.170+0000 7f09ab7fe700 1 -- 172.21.15.7:0/96905346 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f09ac09b370 con 0x7f09bc102350 2024-06-10T01:03:27.167 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-06-10T01:03:27.167 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDfUGZmDlIVChAAa/yDzmMyo5sjlB54JjsTgQ== 2024-06-10T01:03:27.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:27.172+0000 7f09bb59e700 1 -- 172.21.15.7:0/96905346 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f09a40a2de0 msgr2=0x7f09a40a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:27.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:27.172+0000 7f09bb59e700 1 --2- 172.21.15.7:0/96905346 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f09a40a2de0 0x7f09a40a52a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f09b0007be0 tx=0x7f09b00073d0 comp rx=0 tx=0).stop 2024-06-10T01:03:27.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:27.172+0000 7f09bb59e700 1 -- 172.21.15.7:0/96905346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09bc102350 msgr2=0x7f09bc198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:27.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:27.172+0000 7f09bb59e700 1 --2- 172.21.15.7:0/96905346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09bc102350 0x7f09bc198400 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f09ac00d9b0 tx=0x7f09ac00dcc0 comp rx=0 tx=0).stop 2024-06-10T01:03:27.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:27.172+0000 7f09bb59e700 1 -- 172.21.15.7:0/96905346 shutdown_connections 2024-06-10T01:03:27.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:27.172+0000 7f09bb59e700 1 --2- 172.21.15.7:0/96905346 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f09a40a2de0 0x7f09a40a52a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:27.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:27.172+0000 7f09bb59e700 1 --2- 172.21.15.7:0/96905346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09bc103ef0 0x7f09bc19f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:27.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:27.172+0000 7f09bb59e700 1 --2- 172.21.15.7:0/96905346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09bc102350 0x7f09bc198400 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:27.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:27.172+0000 7f09bb59e700 1 --2- 172.21.15.7:0/96905346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09bc103550 0x7f09bc198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:27.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:27.172+0000 7f09bb59e700 1 -- 172.21.15.7:0/96905346 >> 172.21.15.7:0/96905346 conn(0x7f09bc0fd8d0 msgr2=0x7f09bc106780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:27.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:27.172+0000 7f09bb59e700 1 -- 172.21.15.7:0/96905346 shutdown_connections 2024-06-10T01:03:27.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:27.172+0000 7f09bb59e700 1 -- 172.21.15.7:0/96905346 wait complete. 2024-06-10T01:03:27.205 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:26 smithi175 ceph-mon[36636]: pgmap v352: 129 pgs: 129 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-06-10T01:03:27.892 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-10T01:03:27.892 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-06-10T01:03:27.892 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-06-10T01:03:27.932 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-06-10T01:03:27.932 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-06-10T01:03:27.932 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph mgr dump --format=json 2024-06-10T01:03:28.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:27 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/96905346' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-10T01:03:28.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:27 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/96905346' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-10T01:03:28.205 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:27 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/96905346' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-10T01:03:28.205 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:27 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/96905346' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-10T01:03:28.226 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:03:28.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:27 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/96905346' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-10T01:03:28.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:27 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/96905346' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-10T01:03:29.449 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:29 smithi007 ceph-mon[32398]: pgmap v353: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-06-10T01:03:29.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:29 smithi175 ceph-mon[36636]: pgmap v353: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-06-10T01:03:29.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:29 smithi078 ceph-mon[36683]: pgmap v353: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-06-10T01:03:29.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.995+0000 7fcd86bd6700 1 -- 172.21.15.7:0/3867578647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd80101620 msgr2=0x7fcd80101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:29.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.995+0000 7fcd86bd6700 1 --2- 172.21.15.7:0/3867578647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd80101620 0x7fcd80101a00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcd740099c0 tx=0x7fcd74009cd0 comp rx=0 tx=0).stop 2024-06-10T01:03:29.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.995+0000 7fcd86bd6700 1 -- 172.21.15.7:0/3867578647 shutdown_connections 2024-06-10T01:03:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.995+0000 7fcd86bd6700 1 --2- 172.21.15.7:0/3867578647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd801066b0 0x7fcd8010e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.995+0000 7fcd86bd6700 1 --2- 172.21.15.7:0/3867578647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd80101620 0x7fcd80101a00 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.995+0000 7fcd86bd6700 1 --2- 172.21.15.7:0/3867578647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd80101fd0 0x7fcd80105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.995+0000 7fcd86bd6700 1 -- 172.21.15.7:0/3867578647 >> 172.21.15.7:0/3867578647 conn(0x7fcd80078360 msgr2=0x7fcd80078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.995+0000 7fcd86bd6700 1 -- 172.21.15.7:0/3867578647 shutdown_connections 2024-06-10T01:03:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.995+0000 7fcd86bd6700 1 -- 172.21.15.7:0/3867578647 wait complete. 2024-06-10T01:03:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.995+0000 7fcd86bd6700 1 Processor -- start 2024-06-10T01:03:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.996+0000 7fcd86bd6700 1 -- start start 2024-06-10T01:03:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.996+0000 7fcd86bd6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd80101620 0x7fcd80199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.996+0000 7fcd86bd6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd80101fd0 0x7fcd8019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.996+0000 7fcd86bd6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd801066b0 0x7fcd801a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.996+0000 7fcd86bd6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcd80111690 con 0x7fcd80101620 2024-06-10T01:03:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.996+0000 7fcd86bd6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcd80111510 con 0x7fcd801066b0 2024-06-10T01:03:29.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.996+0000 7fcd86bd6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcd80111810 con 0x7fcd80101fd0 2024-06-10T01:03:29.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.996+0000 7fcd84972700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd80101620 0x7fcd80199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:29.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.996+0000 7fcd84972700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd80101620 0x7fcd80199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40886/0 (socket says 172.21.15.7:40886) 2024-06-10T01:03:29.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.996+0000 7fcd84972700 1 -- 172.21.15.7:0/745146431 learned_addr learned my addr 172.21.15.7:0/745146431 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:03:29.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.996+0000 7fcd85173700 1 --2- 172.21.15.7:0/745146431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd801066b0 0x7fcd801a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:29.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.996+0000 7fcd7ffff700 1 --2- 172.21.15.7:0/745146431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd80101fd0 0x7fcd8019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:29.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.996+0000 7fcd84972700 1 -- 172.21.15.7:0/745146431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd80101fd0 msgr2=0x7fcd8019a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:29.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.996+0000 7fcd84972700 1 --2- 172.21.15.7:0/745146431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd80101fd0 0x7fcd8019a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:29.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.996+0000 7fcd84972700 1 -- 172.21.15.7:0/745146431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd801066b0 msgr2=0x7fcd801a4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:29.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.996+0000 7fcd84972700 1 --2- 172.21.15.7:0/745146431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd801066b0 0x7fcd801a4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:29.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.996+0000 7fcd84972700 1 -- 172.21.15.7:0/745146431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd801a5090 con 0x7fcd80101620 2024-06-10T01:03:29.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.996+0000 7fcd7ffff700 1 --2- 172.21.15.7:0/745146431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd80101fd0 0x7fcd8019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:03:29.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.996+0000 7fcd84972700 1 --2- 172.21.15.7:0/745146431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd80101620 0x7fcd80199ad0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fcd74006010 tx=0x7fcd74009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:29.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.997+0000 7fcd7dffb700 1 -- 172.21.15.7:0/745146431 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd74027070 con 0x7fcd80101620 2024-06-10T01:03:29.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.997+0000 7fcd7dffb700 1 -- 172.21.15.7:0/745146431 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd74003780 con 0x7fcd80101620 2024-06-10T01:03:29.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.997+0000 7fcd7dffb700 1 -- 172.21.15.7:0/745146431 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd74022da0 con 0x7fcd80101620 2024-06-10T01:03:29.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.997+0000 7fcd86bd6700 1 -- 172.21.15.7:0/745146431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd801a5260 con 0x7fcd80101620 2024-06-10T01:03:29.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.997+0000 7fcd86bd6700 1 -- 172.21.15.7:0/745146431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd801a5600 con 0x7fcd80101620 2024-06-10T01:03:29.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.998+0000 7fcd86bd6700 1 -- 172.21.15.7:0/745146431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd80069180 con 0x7fcd80101620 2024-06-10T01:03:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:29.999+0000 7fcd7dffb700 1 -- 172.21.15.7:0/745146431 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fcd7402a070 con 0x7fcd80101620 2024-06-10T01:03:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:30.000+0000 7fcd7dffb700 1 --2- 172.21.15.7:0/745146431 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcd700a2d30 0x7fcd700a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:30.000+0000 7fcd7ffff700 1 --2- 172.21.15.7:0/745146431 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcd700a2d30 0x7fcd700a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:30.000+0000 7fcd7dffb700 1 -- 172.21.15.7:0/745146431 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7fcd740d0320 con 0x7fcd80101620 2024-06-10T01:03:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:30.001+0000 7fcd7ffff700 1 --2- 172.21.15.7:0/745146431 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcd700a2d30 0x7fcd700a51f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c005950 tx=0x7fcd6c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:30.004+0000 7fcd7dffb700 1 -- 172.21.15.7:0/745146431 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcd7405e850 con 0x7fcd80101620 2024-06-10T01:03:30.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:30.273+0000 7fcd86bd6700 1 -- 172.21.15.7:0/745146431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fcd8002df00 con 0x7fcd80101620 2024-06-10T01:03:30.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:30.278+0000 7fcd7dffb700 1 -- 172.21.15.7:0/745146431 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263189 (secure 0 0 0) 0x7fcd74013070 con 0x7fcd80101620 2024-06-10T01:03:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:03:30.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:30.281+0000 7fcd86bd6700 1 -- 172.21.15.7:0/745146431 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcd700a2d30 msgr2=0x7fcd700a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:30.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:30.281+0000 7fcd86bd6700 1 --2- 172.21.15.7:0/745146431 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcd700a2d30 0x7fcd700a51f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c005950 tx=0x7fcd6c0058e0 comp rx=0 tx=0).stop 2024-06-10T01:03:30.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:30.281+0000 7fcd86bd6700 1 -- 172.21.15.7:0/745146431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd80101620 msgr2=0x7fcd80199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:30.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:30.281+0000 7fcd86bd6700 1 --2- 172.21.15.7:0/745146431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd80101620 0x7fcd80199ad0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fcd74006010 tx=0x7fcd74009f80 comp rx=0 tx=0).stop 2024-06-10T01:03:30.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:30.281+0000 7fcd86bd6700 1 -- 172.21.15.7:0/745146431 shutdown_connections 2024-06-10T01:03:30.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:30.281+0000 7fcd86bd6700 1 --2- 172.21.15.7:0/745146431 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcd700a2d30 0x7fcd700a51f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:30.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:30.281+0000 7fcd86bd6700 1 --2- 172.21.15.7:0/745146431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd801066b0 0x7fcd801a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:30.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:30.281+0000 7fcd86bd6700 1 --2- 172.21.15.7:0/745146431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd80101620 0x7fcd80199ad0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:30.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:30.281+0000 7fcd86bd6700 1 --2- 172.21.15.7:0/745146431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd80101fd0 0x7fcd8019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:30.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:30.281+0000 7fcd86bd6700 1 -- 172.21.15.7:0/745146431 >> 172.21.15.7:0/745146431 conn(0x7fcd80078360 msgr2=0x7fcd80100b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:30.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:30.281+0000 7fcd86bd6700 1 -- 172.21.15.7:0/745146431 shutdown_connections 2024-06-10T01:03:30.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:30.281+0000 7fcd86bd6700 1 -- 172.21.15.7:0/745146431 wait complete. 2024-06-10T01:03:30.981 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":1804801756},{"type":"v1","addr":"172.21.15.7:6801","nonce":1804801756}]},"active_addr":"172.21.15.7:6801/1804801756","active_change":"2024-06-10T00:54:35.441249+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.7:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":307439401}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":2795460416}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":951050500}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":4252116269}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":497115802}]}]} 2024-06-10T01:03:30.986 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-06-10T01:03:30.986 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-06-10T01:03:30.987 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T01:03:31.242 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:03:31.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:31 smithi007 ceph-mon[32398]: pgmap v354: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-10T01:03:31.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:31 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/745146431' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-10T01:03:31.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:31 smithi175 ceph-mon[36636]: pgmap v354: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-10T01:03:31.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:31 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/745146431' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-10T01:03:31.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:31 smithi078 ceph-mon[36683]: pgmap v354: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-10T01:03:31.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:31 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/745146431' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-10T01:03:32.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.815+0000 7fd20c3e0700 1 -- 172.21.15.7:0/2720959942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd204107840 msgr2=0x7fd20410f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:32.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.815+0000 7fd20c3e0700 1 --2- 172.21.15.7:0/2720959942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd204107840 0x7fd20410f3c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc00b0d0 tx=0x7fd1fc00b3e0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.815+0000 7fd20c3e0700 1 -- 172.21.15.7:0/2720959942 shutdown_connections 2024-06-10T01:03:32.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.815+0000 7fd20c3e0700 1 --2- 172.21.15.7:0/2720959942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd204107840 0x7fd20410f3c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.815+0000 7fd20c3e0700 1 --2- 172.21.15.7:0/2720959942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd204102740 0x7fd204102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.815+0000 7fd20c3e0700 1 --2- 172.21.15.7:0/2720959942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2041030f0 0x7fd204107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.815+0000 7fd20c3e0700 1 -- 172.21.15.7:0/2720959942 >> 172.21.15.7:0/2720959942 conn(0x7fd2040795f0 msgr2=0x7fd204079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:32.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.815+0000 7fd20c3e0700 1 -- 172.21.15.7:0/2720959942 shutdown_connections 2024-06-10T01:03:32.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.815+0000 7fd20c3e0700 1 -- 172.21.15.7:0/2720959942 wait complete. 2024-06-10T01:03:32.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.816+0000 7fd20c3e0700 1 Processor -- start 2024-06-10T01:03:32.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.816+0000 7fd20c3e0700 1 -- start start 2024-06-10T01:03:32.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.816+0000 7fd20c3e0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd204102740 0x7fd20419e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:32.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.816+0000 7fd20c3e0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2041030f0 0x7fd20419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:32.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.816+0000 7fd20c3e0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd204107840 0x7fd204198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:32.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.816+0000 7fd20c3e0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd204112760 con 0x7fd204107840 2024-06-10T01:03:32.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.816+0000 7fd20c3e0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd2041125e0 con 0x7fd2041030f0 2024-06-10T01:03:32.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.816+0000 7fd20c3e0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd2041128e0 con 0x7fd204102740 2024-06-10T01:03:32.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.816+0000 7fd20a97d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd204107840 0x7fd204198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:32.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.816+0000 7fd20a97d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd204107840 0x7fd204198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49728/0 (socket says 172.21.15.7:49728) 2024-06-10T01:03:32.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.816+0000 7fd20a97d700 1 -- 172.21.15.7:0/1261485713 learned_addr learned my addr 172.21.15.7:0/1261485713 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:03:32.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.816+0000 7fd20997b700 1 --2- 172.21.15.7:0/1261485713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2041030f0 0x7fd20419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:32.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.816+0000 7fd20a17c700 1 --2- 172.21.15.7:0/1261485713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd204102740 0x7fd20419e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:32.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.816+0000 7fd20a97d700 1 -- 172.21.15.7:0/1261485713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd204102740 msgr2=0x7fd20419e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:32.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.816+0000 7fd20a97d700 1 --2- 172.21.15.7:0/1261485713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd204102740 0x7fd20419e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.816+0000 7fd20a97d700 1 -- 172.21.15.7:0/1261485713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2041030f0 msgr2=0x7fd20419ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:32.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.816+0000 7fd20a97d700 1 --2- 172.21.15.7:0/1261485713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2041030f0 0x7fd20419ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.817+0000 7fd20a97d700 1 -- 172.21.15.7:0/1261485713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2041990e0 con 0x7fd204107840 2024-06-10T01:03:32.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.817+0000 7fd20a97d700 1 --2- 172.21.15.7:0/1261485713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd204107840 0x7fd204198980 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc006070 tx=0x7fd1fc004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:32.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.817+0000 7fd1fb7fe700 1 -- 172.21.15.7:0/1261485713 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc021070 con 0x7fd204107840 2024-06-10T01:03:32.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.817+0000 7fd20c3e0700 1 -- 172.21.15.7:0/1261485713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd204199370 con 0x7fd204107840 2024-06-10T01:03:32.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.817+0000 7fd1fb7fe700 1 -- 172.21.15.7:0/1261485713 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1fc007b80 con 0x7fd204107840 2024-06-10T01:03:32.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.817+0000 7fd1fb7fe700 1 -- 172.21.15.7:0/1261485713 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc0052a0 con 0x7fd204107840 2024-06-10T01:03:32.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.817+0000 7fd20c3e0700 1 -- 172.21.15.7:0/1261485713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2041a5640 con 0x7fd204107840 2024-06-10T01:03:32.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.818+0000 7fd20c3e0700 1 -- 172.21.15.7:0/1261485713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd204069180 con 0x7fd204107840 2024-06-10T01:03:32.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.819+0000 7fd1fb7fe700 1 -- 172.21.15.7:0/1261485713 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fd1fc024070 con 0x7fd204107840 2024-06-10T01:03:32.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.820+0000 7fd1fb7fe700 1 --2- 172.21.15.7:0/1261485713 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd1f00a2cb0 0x7fd1f00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:32.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.820+0000 7fd20a17c700 1 --2- 172.21.15.7:0/1261485713 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd1f00a2cb0 0x7fd1f00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:32.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.820+0000 7fd1fb7fe700 1 -- 172.21.15.7:0/1261485713 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7fd1fc0d0a40 con 0x7fd204107840 2024-06-10T01:03:32.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.820+0000 7fd20a17c700 1 --2- 172.21.15.7:0/1261485713 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd1f00a2cb0 0x7fd1f00a5170 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd2041072f0 tx=0x7fd2000073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:32.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:32.823+0000 7fd1fb7fe700 1 -- 172.21.15.7:0/1261485713 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd1fc09d570 con 0x7fd204107840 2024-06-10T01:03:33.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:33.030+0000 7fd20c3e0700 1 -- 172.21.15.7:0/1261485713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd204199e60 con 0x7fd204107840 2024-06-10T01:03:33.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:33.031+0000 7fd1fb7fe700 1 -- 172.21.15.7:0/1261485713 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24493 (secure 0 0 0) 0x7fd1fc09ce70 con 0x7fd204107840 2024-06-10T01:03:33.029 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:03:33.029 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":97,"fsid":"bded58f8-26c3-11ef-bc9b-c7b262605968","created":"2024-06-10T00:53:31.966085+0000","modified":"2024-06-10T01:02:37.362590+0000","last_up_change":"2024-06-10T01:00:45.382473+0000","last_in_change":"2024-06-10T01:00:25.608513+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:56:59.964055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T01:01:42.563611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T01:01:46.175918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T01:01:55.695502+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f7c24c9f-b7ab-4c04-b2b4-120df9751667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6803","nonce":404485726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6805","nonce":404485726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6809","nonce":404485726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6807","nonce":404485726}]},"public_addr":"172.21.15.7:6803/404485726","cluster_addr":"172.21.15.7:6805/404485726","heartbeat_back_addr":"172.21.15.7:6809/404485726","heartbeat_front_addr":"172.21.15.7:6807/404485726","state":["exists","up"]},{"osd":1,"uuid":"9bf70a6b-f31c-4077-b5af-229f17bc2845","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6801","nonce":1856804448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6803","nonce":1856804448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6807","nonce":1856804448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6805","nonce":1856804448}]},"public_addr":"172.21.15.78:6801/1856804448","cluster_addr":"172.21.15.78:6803/1856804448","heartbeat_back_addr":"172.21.15.78:6807/1856804448","heartbeat_front_addr":"172.21.15.78:6805/1856804448","state":["exists","up"]},{"osd":2,"uuid":"62923c97-caf9-4f7e-9c14-9c9918843c78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6801","nonce":918257455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6803","nonce":918257455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6807","nonce":918257455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6805","nonce":918257455}]},"public_addr":"172.21.15.175:6801/918257455","cluster_addr":"172.21.15.175:6803/918257455","heartbeat_back_addr":"172.21.15.175:6807/918257455","heartbeat_front_addr":"172.21.15.175:6805/918257455","state":["exists","up"]},{"osd":3,"uuid":"65242098-e748-48ab-8f0e-bb25b5c62742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6811","nonce":1797404966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6813","nonce":1797404966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6817","nonce":1797404966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6815","nonce":1797404966}]},"public_addr":"172.21.15.7:6811/1797404966","cluster_addr":"172.21.15.7:6813/1797404966","heartbeat_back_addr":"172.21.15.7:6817/1797404966","heartbeat_front_addr":"172.21.15.7:6815/1797404966","state":["exists","up"]},{"osd":4,"uuid":"a7990305-de9e-400e-a432-d7854f73f339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6809","nonce":3767608357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6811","nonce":3767608357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6815","nonce":3767608357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6813","nonce":3767608357}]},"public_addr":"172.21.15.78:6809/3767608357","cluster_addr":"172.21.15.78:6811/3767608357","heartbeat_back_addr":"172.21.15.78:6815/3767608357","heartbeat_front_addr":"172.21.15.78:6813/3767608357","state":["exists","up"]},{"osd":5,"uuid":"bad9a7b3-da5a-48d0-905d-e6a3190317fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6809","nonce":2157664731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6811","nonce":2157664731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6815","nonce":2157664731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6813","nonce":2157664731}]},"public_addr":"172.21.15.175:6809/2157664731","cluster_addr":"172.21.15.175:6811/2157664731","heartbeat_back_addr":"172.21.15.175:6815/2157664731","heartbeat_front_addr":"172.21.15.175:6813/2157664731","state":["exists","up"]},{"osd":6,"uuid":"6c687653-157b-4b96-867b-a2eaa7d8ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6819","nonce":838731756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6821","nonce":838731756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6825","nonce":838731756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6823","nonce":838731756}]},"public_addr":"172.21.15.7:6819/838731756","cluster_addr":"172.21.15.7:6821/838731756","heartbeat_back_addr":"172.21.15.7:6825/838731756","heartbeat_front_addr":"172.21.15.7:6823/838731756","state":["exists","up"]},{"osd":7,"uuid":"db40ddf1-9fd9-4859-b8c4-75f79582ab91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6817","nonce":3511921773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6819","nonce":3511921773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6823","nonce":3511921773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6821","nonce":3511921773}]},"public_addr":"172.21.15.78:6817/3511921773","cluster_addr":"172.21.15.78:6819/3511921773","heartbeat_back_addr":"172.21.15.78:6823/3511921773","heartbeat_front_addr":"172.21.15.78:6821/3511921773","state":["exists","up"]},{"osd":8,"uuid":"0f01cbd9-b193-4c70-a33c-4984167bf422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6817","nonce":1546325211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6819","nonce":1546325211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6823","nonce":1546325211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6821","nonce":1546325211}]},"public_addr":"172.21.15.175:6817/1546325211","cluster_addr":"172.21.15.175:6819/1546325211","heartbeat_back_addr":"172.21.15.175:6823/1546325211","heartbeat_front_addr":"172.21.15.175:6821/1546325211","state":["exists","up"]},{"osd":9,"uuid":"e5480998-f7fb-49cd-9aed-52405c88d830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6827","nonce":1844550813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6829","nonce":1844550813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6833","nonce":1844550813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6831","nonce":1844550813}]},"public_addr":"172.21.15.7:6827/1844550813","cluster_addr":"172.21.15.7:6829/1844550813","heartbeat_back_addr":"172.21.15.7:6833/1844550813","heartbeat_front_addr":"172.21.15.7:6831/1844550813","state":["exists","up"]},{"osd":10,"uuid":"caa82f33-a71c-4907-bf3b-30a7fdfeb71d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6825","nonce":2290824785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6827","nonce":2290824785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6831","nonce":2290824785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6829","nonce":2290824785}]},"public_addr":"172.21.15.78:6825/2290824785","cluster_addr":"172.21.15.78:6827/2290824785","heartbeat_back_addr":"172.21.15.78:6831/2290824785","heartbeat_front_addr":"172.21.15.78:6829/2290824785","state":["exists","up"]},{"osd":11,"uuid":"608cda77-77da-4da2-a7e2-76a773be27c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6825","nonce":1959979550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6827","nonce":1959979550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6831","nonce":1959979550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6829","nonce":1959979550}]},"public_addr":"172.21.15.175:6825/1959979550","cluster_addr":"172.21.15.175:6827/1959979550","heartbeat_back_addr":"172.21.15.175:6831/1959979550","heartbeat_front_addr":"172.21.15.175:6829/1959979550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:08.850203+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:32.820161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:56.633656+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:21.759372+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:46.735874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:10.636541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:36.990099+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:01.767098+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:26.882776+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:52.757595+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:18.383938+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:43.544330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1833397470":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/322543993":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/212510583":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6801/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6800/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/2679436843":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6801/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/1210355277":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6800/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/3610017406":"2024-06-11T00:54:08.102405+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T01:03:33.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:33.033+0000 7fd20c3e0700 1 -- 172.21.15.7:0/1261485713 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd1f00a2cb0 msgr2=0x7fd1f00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:33.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:33.033+0000 7fd20c3e0700 1 --2- 172.21.15.7:0/1261485713 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd1f00a2cb0 0x7fd1f00a5170 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd2041072f0 tx=0x7fd2000073d0 comp rx=0 tx=0).stop 2024-06-10T01:03:33.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:33.033+0000 7fd20c3e0700 1 -- 172.21.15.7:0/1261485713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd204107840 msgr2=0x7fd204198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:33.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:33.033+0000 7fd20c3e0700 1 --2- 172.21.15.7:0/1261485713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd204107840 0x7fd204198980 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc006070 tx=0x7fd1fc004d00 comp rx=0 tx=0).stop 2024-06-10T01:03:33.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:33.033+0000 7fd20c3e0700 1 -- 172.21.15.7:0/1261485713 shutdown_connections 2024-06-10T01:03:33.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:33.033+0000 7fd20c3e0700 1 --2- 172.21.15.7:0/1261485713 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd1f00a2cb0 0x7fd1f00a5170 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:33.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:33.033+0000 7fd20c3e0700 1 --2- 172.21.15.7:0/1261485713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd204107840 0x7fd204198980 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:33.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:33.033+0000 7fd20c3e0700 1 --2- 172.21.15.7:0/1261485713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd204102740 0x7fd20419e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:33.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:33.033+0000 7fd20c3e0700 1 --2- 172.21.15.7:0/1261485713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2041030f0 0x7fd20419ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:33.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:33.033+0000 7fd20c3e0700 1 -- 172.21.15.7:0/1261485713 >> 172.21.15.7:0/1261485713 conn(0x7fd2040795f0 msgr2=0x7fd2041005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:33.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:33.034+0000 7fd20c3e0700 1 -- 172.21.15.7:0/1261485713 shutdown_connections 2024-06-10T01:03:33.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:33.034+0000 7fd20c3e0700 1 -- 172.21.15.7:0/1261485713 wait complete. 2024-06-10T01:03:33.504 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:33 smithi007 ceph-mon[32398]: pgmap v355: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-10T01:03:33.504 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:33 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1261485713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:03:33.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:33 smithi078 ceph-mon[36683]: pgmap v355: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-10T01:03:33.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:33 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1261485713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:03:33.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:33 smithi175 ceph-mon[36636]: pgmap v355: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-10T01:03:33.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:33 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1261485713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:03:33.753 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-06-10T01:03:33.753 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T01:03:34.008 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:03:35.455 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:35 smithi007 ceph-mon[32398]: pgmap v356: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-10T01:03:35.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:35 smithi078 ceph-mon[36683]: pgmap v356: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-10T01:03:35.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.685+0000 7ff64bcdc700 1 -- 172.21.15.7:0/1491973909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff644101df0 msgr2=0x7ff6441021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:35.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.685+0000 7ff64bcdc700 1 --2- 172.21.15.7:0/1491973909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff644101df0 0x7ff6441021d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff6400099c0 tx=0x7ff640009cd0 comp rx=0 tx=0).stop 2024-06-10T01:03:35.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.685+0000 7ff64bcdc700 1 -- 172.21.15.7:0/1491973909 shutdown_connections 2024-06-10T01:03:35.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.685+0000 7ff64bcdc700 1 --2- 172.21.15.7:0/1491973909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff644106750 0x7ff64410e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:35.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.685+0000 7ff64bcdc700 1 --2- 172.21.15.7:0/1491973909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff644101df0 0x7ff6441021d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:35.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.685+0000 7ff64bcdc700 1 --2- 172.21.15.7:0/1491973909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6441027a0 0x7ff644106020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:35.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.685+0000 7ff64bcdc700 1 -- 172.21.15.7:0/1491973909 >> 172.21.15.7:0/1491973909 conn(0x7ff644077a00 msgr2=0x7ff6440fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:35.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.686+0000 7ff64bcdc700 1 -- 172.21.15.7:0/1491973909 shutdown_connections 2024-06-10T01:03:35.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.686+0000 7ff64bcdc700 1 -- 172.21.15.7:0/1491973909 wait complete. 2024-06-10T01:03:35.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.686+0000 7ff64bcdc700 1 Processor -- start 2024-06-10T01:03:35.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.686+0000 7ff64bcdc700 1 -- start start 2024-06-10T01:03:35.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.686+0000 7ff64bcdc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff644101df0 0x7ff6440fce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:35.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.686+0000 7ff64bcdc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6441027a0 0x7ff6440fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:35.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.686+0000 7ff64bcdc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff644106750 0x7ff6440fd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:35.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.686+0000 7ff64bcdc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff644111670 con 0x7ff644101df0 2024-06-10T01:03:35.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.686+0000 7ff64bcdc700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff6441114f0 con 0x7ff644106750 2024-06-10T01:03:35.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.686+0000 7ff64bcdc700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff6441117f0 con 0x7ff6441027a0 2024-06-10T01:03:35.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.687+0000 7ff649a78700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff644101df0 0x7ff6440fce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:35.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.687+0000 7ff649a78700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff644101df0 0x7ff6440fce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49754/0 (socket says 172.21.15.7:49754) 2024-06-10T01:03:35.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.687+0000 7ff649a78700 1 -- 172.21.15.7:0/1993993433 learned_addr learned my addr 172.21.15.7:0/1993993433 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:03:35.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.687+0000 7ff649277700 1 --2- 172.21.15.7:0/1993993433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6441027a0 0x7ff6440fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:35.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.687+0000 7ff64a279700 1 --2- 172.21.15.7:0/1993993433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff644106750 0x7ff6440fd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:35.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.687+0000 7ff649a78700 1 -- 172.21.15.7:0/1993993433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6441027a0 msgr2=0x7ff6440fd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:35.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.687+0000 7ff649a78700 1 --2- 172.21.15.7:0/1993993433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6441027a0 0x7ff6440fd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:35.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.687+0000 7ff649a78700 1 -- 172.21.15.7:0/1993993433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff644106750 msgr2=0x7ff6440fd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:35.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.687+0000 7ff649a78700 1 --2- 172.21.15.7:0/1993993433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff644106750 0x7ff6440fd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:35.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.687+0000 7ff649a78700 1 -- 172.21.15.7:0/1993993433 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6441a7140 con 0x7ff644101df0 2024-06-10T01:03:35.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.687+0000 7ff649a78700 1 --2- 172.21.15.7:0/1993993433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff644101df0 0x7ff6440fce70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff640006010 tx=0x7ff640009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:35.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.687+0000 7ff63affd700 1 -- 172.21.15.7:0/1993993433 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff640026070 con 0x7ff644101df0 2024-06-10T01:03:35.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.687+0000 7ff64bcdc700 1 -- 172.21.15.7:0/1993993433 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6441a73d0 con 0x7ff644101df0 2024-06-10T01:03:35.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.687+0000 7ff63affd700 1 -- 172.21.15.7:0/1993993433 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff640004cd0 con 0x7ff644101df0 2024-06-10T01:03:35.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.687+0000 7ff63affd700 1 -- 172.21.15.7:0/1993993433 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff640003ba0 con 0x7ff644101df0 2024-06-10T01:03:35.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.687+0000 7ff64bcdc700 1 -- 172.21.15.7:0/1993993433 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6441a7820 con 0x7ff644101df0 2024-06-10T01:03:35.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.688+0000 7ff64bcdc700 1 -- 172.21.15.7:0/1993993433 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff644069180 con 0x7ff644101df0 2024-06-10T01:03:35.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.690+0000 7ff63affd700 1 -- 172.21.15.7:0/1993993433 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7ff640029070 con 0x7ff644101df0 2024-06-10T01:03:35.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.690+0000 7ff63affd700 1 --2- 172.21.15.7:0/1993993433 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff6300a2cb0 0x7ff6300a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:35.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.690+0000 7ff649277700 1 --2- 172.21.15.7:0/1993993433 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff6300a2cb0 0x7ff6300a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:35.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.690+0000 7ff63affd700 1 -- 172.21.15.7:0/1993993433 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7ff6400d0210 con 0x7ff644101df0 2024-06-10T01:03:35.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.691+0000 7ff649277700 1 --2- 172.21.15.7:0/1993993433 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff6300a2cb0 0x7ff6300a5170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff644103fc0 tx=0x7ff63400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:35.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.694+0000 7ff63affd700 1 -- 172.21.15.7:0/1993993433 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff640030020 con 0x7ff644101df0 2024-06-10T01:03:35.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:35 smithi175 ceph-mon[36636]: pgmap v356: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-10T01:03:35.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.902+0000 7ff64bcdc700 1 -- 172.21.15.7:0/1993993433 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff6441a7b40 con 0x7ff644101df0 2024-06-10T01:03:35.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.904+0000 7ff63affd700 1 -- 172.21.15.7:0/1993993433 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24493 (secure 0 0 0) 0x7ff640098020 con 0x7ff644101df0 2024-06-10T01:03:35.901 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:03:35.901 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":97,"fsid":"bded58f8-26c3-11ef-bc9b-c7b262605968","created":"2024-06-10T00:53:31.966085+0000","modified":"2024-06-10T01:02:37.362590+0000","last_up_change":"2024-06-10T01:00:45.382473+0000","last_in_change":"2024-06-10T01:00:25.608513+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:56:59.964055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T01:01:42.563611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T01:01:46.175918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T01:01:55.695502+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f7c24c9f-b7ab-4c04-b2b4-120df9751667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6803","nonce":404485726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6805","nonce":404485726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6809","nonce":404485726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6807","nonce":404485726}]},"public_addr":"172.21.15.7:6803/404485726","cluster_addr":"172.21.15.7:6805/404485726","heartbeat_back_addr":"172.21.15.7:6809/404485726","heartbeat_front_addr":"172.21.15.7:6807/404485726","state":["exists","up"]},{"osd":1,"uuid":"9bf70a6b-f31c-4077-b5af-229f17bc2845","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6801","nonce":1856804448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6803","nonce":1856804448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6807","nonce":1856804448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6805","nonce":1856804448}]},"public_addr":"172.21.15.78:6801/1856804448","cluster_addr":"172.21.15.78:6803/1856804448","heartbeat_back_addr":"172.21.15.78:6807/1856804448","heartbeat_front_addr":"172.21.15.78:6805/1856804448","state":["exists","up"]},{"osd":2,"uuid":"62923c97-caf9-4f7e-9c14-9c9918843c78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6801","nonce":918257455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6803","nonce":918257455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6807","nonce":918257455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6805","nonce":918257455}]},"public_addr":"172.21.15.175:6801/918257455","cluster_addr":"172.21.15.175:6803/918257455","heartbeat_back_addr":"172.21.15.175:6807/918257455","heartbeat_front_addr":"172.21.15.175:6805/918257455","state":["exists","up"]},{"osd":3,"uuid":"65242098-e748-48ab-8f0e-bb25b5c62742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6811","nonce":1797404966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6813","nonce":1797404966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6817","nonce":1797404966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6815","nonce":1797404966}]},"public_addr":"172.21.15.7:6811/1797404966","cluster_addr":"172.21.15.7:6813/1797404966","heartbeat_back_addr":"172.21.15.7:6817/1797404966","heartbeat_front_addr":"172.21.15.7:6815/1797404966","state":["exists","up"]},{"osd":4,"uuid":"a7990305-de9e-400e-a432-d7854f73f339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6809","nonce":3767608357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6811","nonce":3767608357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6815","nonce":3767608357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6813","nonce":3767608357}]},"public_addr":"172.21.15.78:6809/3767608357","cluster_addr":"172.21.15.78:6811/3767608357","heartbeat_back_addr":"172.21.15.78:6815/3767608357","heartbeat_front_addr":"172.21.15.78:6813/3767608357","state":["exists","up"]},{"osd":5,"uuid":"bad9a7b3-da5a-48d0-905d-e6a3190317fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6809","nonce":2157664731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6811","nonce":2157664731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6815","nonce":2157664731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6813","nonce":2157664731}]},"public_addr":"172.21.15.175:6809/2157664731","cluster_addr":"172.21.15.175:6811/2157664731","heartbeat_back_addr":"172.21.15.175:6815/2157664731","heartbeat_front_addr":"172.21.15.175:6813/2157664731","state":["exists","up"]},{"osd":6,"uuid":"6c687653-157b-4b96-867b-a2eaa7d8ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6819","nonce":838731756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6821","nonce":838731756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6825","nonce":838731756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6823","nonce":838731756}]},"public_addr":"172.21.15.7:6819/838731756","cluster_addr":"172.21.15.7:6821/838731756","heartbeat_back_addr":"172.21.15.7:6825/838731756","heartbeat_front_addr":"172.21.15.7:6823/838731756","state":["exists","up"]},{"osd":7,"uuid":"db40ddf1-9fd9-4859-b8c4-75f79582ab91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6817","nonce":3511921773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6819","nonce":3511921773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6823","nonce":3511921773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6821","nonce":3511921773}]},"public_addr":"172.21.15.78:6817/3511921773","cluster_addr":"172.21.15.78:6819/3511921773","heartbeat_back_addr":"172.21.15.78:6823/3511921773","heartbeat_front_addr":"172.21.15.78:6821/3511921773","state":["exists","up"]},{"osd":8,"uuid":"0f01cbd9-b193-4c70-a33c-4984167bf422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6817","nonce":1546325211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6819","nonce":1546325211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6823","nonce":1546325211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6821","nonce":1546325211}]},"public_addr":"172.21.15.175:6817/1546325211","cluster_addr":"172.21.15.175:6819/1546325211","heartbeat_back_addr":"172.21.15.175:6823/1546325211","heartbeat_front_addr":"172.21.15.175:6821/1546325211","state":["exists","up"]},{"osd":9,"uuid":"e5480998-f7fb-49cd-9aed-52405c88d830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6827","nonce":1844550813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6829","nonce":1844550813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6833","nonce":1844550813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6831","nonce":1844550813}]},"public_addr":"172.21.15.7:6827/1844550813","cluster_addr":"172.21.15.7:6829/1844550813","heartbeat_back_addr":"172.21.15.7:6833/1844550813","heartbeat_front_addr":"172.21.15.7:6831/1844550813","state":["exists","up"]},{"osd":10,"uuid":"caa82f33-a71c-4907-bf3b-30a7fdfeb71d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6825","nonce":2290824785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6827","nonce":2290824785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6831","nonce":2290824785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6829","nonce":2290824785}]},"public_addr":"172.21.15.78:6825/2290824785","cluster_addr":"172.21.15.78:6827/2290824785","heartbeat_back_addr":"172.21.15.78:6831/2290824785","heartbeat_front_addr":"172.21.15.78:6829/2290824785","state":["exists","up"]},{"osd":11,"uuid":"608cda77-77da-4da2-a7e2-76a773be27c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6825","nonce":1959979550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6827","nonce":1959979550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6831","nonce":1959979550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6829","nonce":1959979550}]},"public_addr":"172.21.15.175:6825/1959979550","cluster_addr":"172.21.15.175:6827/1959979550","heartbeat_back_addr":"172.21.15.175:6831/1959979550","heartbeat_front_addr":"172.21.15.175:6829/1959979550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:08.850203+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:32.820161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:56.633656+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:21.759372+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:46.735874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:10.636541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:36.990099+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:01.767098+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:26.882776+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:52.757595+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:18.383938+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:43.544330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1833397470":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/322543993":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/212510583":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6801/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6800/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/2679436843":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6801/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/1210355277":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6800/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/3610017406":"2024-06-11T00:54:08.102405+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T01:03:35.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.906+0000 7ff64bcdc700 1 -- 172.21.15.7:0/1993993433 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff6300a2cb0 msgr2=0x7ff6300a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:35.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.906+0000 7ff64bcdc700 1 --2- 172.21.15.7:0/1993993433 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff6300a2cb0 0x7ff6300a5170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff644103fc0 tx=0x7ff63400a300 comp rx=0 tx=0).stop 2024-06-10T01:03:35.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.906+0000 7ff64bcdc700 1 -- 172.21.15.7:0/1993993433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff644101df0 msgr2=0x7ff6440fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:35.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.906+0000 7ff64bcdc700 1 --2- 172.21.15.7:0/1993993433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff644101df0 0x7ff6440fce70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff640006010 tx=0x7ff640009f80 comp rx=0 tx=0).stop 2024-06-10T01:03:35.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.906+0000 7ff64bcdc700 1 -- 172.21.15.7:0/1993993433 shutdown_connections 2024-06-10T01:03:35.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.906+0000 7ff64bcdc700 1 --2- 172.21.15.7:0/1993993433 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff6300a2cb0 0x7ff6300a5170 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:35.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.906+0000 7ff64bcdc700 1 --2- 172.21.15.7:0/1993993433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff644106750 0x7ff6440fd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:35.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.906+0000 7ff64bcdc700 1 --2- 172.21.15.7:0/1993993433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff644101df0 0x7ff6440fce70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:35.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.906+0000 7ff64bcdc700 1 --2- 172.21.15.7:0/1993993433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6441027a0 0x7ff6440fd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:35.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.906+0000 7ff64bcdc700 1 -- 172.21.15.7:0/1993993433 >> 172.21.15.7:0/1993993433 conn(0x7ff644077a00 msgr2=0x7ff644100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:35.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.906+0000 7ff64bcdc700 1 -- 172.21.15.7:0/1993993433 shutdown_connections 2024-06-10T01:03:35.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:35.906+0000 7ff64bcdc700 1 -- 172.21.15.7:0/1993993433 wait complete. 2024-06-10T01:03:36.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:36 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1993993433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:03:36.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:36 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1993993433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:03:36.680 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-06-10T01:03:36.680 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-06-10T01:03:36.681 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-06-10T01:03:36.681 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-06-10T01:03:36.682 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-06-10T01:03:36.682 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-06-10T01:03:36.683 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-06-10T01:03:36.683 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-06-10T01:03:36.684 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-06-10T01:03:36.684 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-06-10T01:03:36.685 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-06-10T01:03:36.685 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-06-10T01:03:36.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:36 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1993993433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:03:37.434 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:03:37.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:37 smithi007 ceph-mon[32398]: pgmap v357: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-10T01:03:37.516 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-06-10T01:03:37.516 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]: dispatch 2024-06-10T01:03:37.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:37 smithi078 ceph-mon[36683]: pgmap v357: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-10T01:03:37.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-06-10T01:03:37.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]: dispatch 2024-06-10T01:03:37.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:37 smithi175 ceph-mon[36636]: pgmap v357: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-10T01:03:37.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-06-10T01:03:37.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]: dispatch 2024-06-10T01:03:37.754 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:03:37.755 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:03:37.756 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:03:37.757 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:03:37.757 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:03:37.757 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:03:37.861 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:03:38.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:38 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-06-10T01:03:38.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:38 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]': finished 2024-06-10T01:03:38.516 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:38 smithi007 ceph-mon[32398]: osdmap e98: 12 total, 12 up, 12 in 2024-06-10T01:03:38.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:38 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-06-10T01:03:38.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:38 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]': finished 2024-06-10T01:03:38.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:38 smithi078 ceph-mon[36683]: osdmap e98: 12 total, 12 up, 12 in 2024-06-10T01:03:38.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:38 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-06-10T01:03:38.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:38 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]': finished 2024-06-10T01:03:38.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:38 smithi175 ceph-mon[36636]: osdmap e98: 12 total, 12 up, 12 in 2024-06-10T01:03:39.482 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:39 smithi007 ceph-mon[32398]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-06-10T01:03:39.482 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:39 smithi007 ceph-mon[32398]: osdmap e99: 12 total, 12 up, 12 in 2024-06-10T01:03:39.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:39 smithi078 ceph-mon[36683]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-06-10T01:03:39.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:39 smithi078 ceph-mon[36683]: osdmap e99: 12 total, 12 up, 12 in 2024-06-10T01:03:39.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:39 smithi175 ceph-mon[36636]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-06-10T01:03:39.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:39 smithi175 ceph-mon[36636]: osdmap e99: 12 total, 12 up, 12 in 2024-06-10T01:03:40.694 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:03:41.450 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:41 smithi007 ceph-mon[32398]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:03:41.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:41 smithi078 ceph-mon[36683]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:03:41.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:41 smithi175 ceph-mon[36636]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:03:42.034 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:03:42.034 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:03:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:03:43.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:43 smithi007 ceph-mon[32398]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 13 B/s, 1 objects/s recovering 2024-06-10T01:03:43.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:43 smithi078 ceph-mon[36683]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 13 B/s, 1 objects/s recovering 2024-06-10T01:03:43.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:43 smithi175 ceph-mon[36636]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 13 B/s, 1 objects/s recovering 2024-06-10T01:03:45.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:45 smithi007 ceph-mon[32398]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 13 B/s, 1 objects/s recovering 2024-06-10T01:03:45.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:45 smithi078 ceph-mon[36683]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 13 B/s, 1 objects/s recovering 2024-06-10T01:03:45.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:45 smithi175 ceph-mon[36636]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 13 B/s, 1 objects/s recovering 2024-06-10T01:03:47.483 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:47 smithi007 ceph-mon[32398]: pgmap v364: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s; 13 B/s, 1 objects/s recovering 2024-06-10T01:03:47.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:47 smithi078 ceph-mon[36683]: pgmap v364: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s; 13 B/s, 1 objects/s recovering 2024-06-10T01:03:47.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.294+0000 7f73dc48b700 1 -- 172.21.15.7:0/1983204245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73d4101f80 msgr2=0x7f73d4105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:47.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.294+0000 7f73dc48b700 1 --2- 172.21.15.7:0/1983204245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73d4101f80 0x7f73d4105fb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f73c4009990 tx=0x7f73c4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:03:47.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.294+0000 7f73dc48b700 1 -- 172.21.15.7:0/1983204245 shutdown_connections 2024-06-10T01:03:47.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.294+0000 7f73dc48b700 1 --2- 172.21.15.7:0/1983204245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73d4106690 0x7f73d410e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:47.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.294+0000 7f73dc48b700 1 --2- 172.21.15.7:0/1983204245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73d41015d0 0x7f73d41019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:47.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.294+0000 7f73dc48b700 1 --2- 172.21.15.7:0/1983204245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73d4101f80 0x7f73d4105fb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:47.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.294+0000 7f73dc48b700 1 -- 172.21.15.7:0/1983204245 >> 172.21.15.7:0/1983204245 conn(0x7f73d4078330 msgr2=0x7f73d4078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:47.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.294+0000 7f73dc48b700 1 -- 172.21.15.7:0/1983204245 shutdown_connections 2024-06-10T01:03:47.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.294+0000 7f73dc48b700 1 -- 172.21.15.7:0/1983204245 wait complete. 2024-06-10T01:03:47.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.295+0000 7f73dc48b700 1 Processor -- start 2024-06-10T01:03:47.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.295+0000 7f73dc48b700 1 -- start start 2024-06-10T01:03:47.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.295+0000 7f73dc48b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73d41015d0 0x7f73d419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:47.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.296+0000 7f73dc48b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73d4101f80 0x7f73d419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:47.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.296+0000 7f73dc48b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73d4106690 0x7f73d4198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:47.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.296+0000 7f73dc48b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f73d4111650 con 0x7f73d4106690 2024-06-10T01:03:47.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.296+0000 7f73dc48b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f73d41114d0 con 0x7f73d41015d0 2024-06-10T01:03:47.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.296+0000 7f73dc48b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f73d41117d0 con 0x7f73d4101f80 2024-06-10T01:03:47.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.296+0000 7f73daa28700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73d4106690 0x7f73d4198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:47.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.296+0000 7f73daa28700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73d4106690 0x7f73d4198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43056/0 (socket says 172.21.15.7:43056) 2024-06-10T01:03:47.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.296+0000 7f73daa28700 1 -- 172.21.15.7:0/4269521090 learned_addr learned my addr 172.21.15.7:0/4269521090 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:03:47.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.296+0000 7f73d9a26700 1 --2- 172.21.15.7:0/4269521090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73d4101f80 0x7f73d419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:47.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.296+0000 7f73da227700 1 --2- 172.21.15.7:0/4269521090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73d41015d0 0x7f73d419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:47.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.296+0000 7f73d9a26700 1 -- 172.21.15.7:0/4269521090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73d41015d0 msgr2=0x7f73d419e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:47.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.296+0000 7f73d9a26700 1 --2- 172.21.15.7:0/4269521090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73d41015d0 0x7f73d419e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:47.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.296+0000 7f73d9a26700 1 -- 172.21.15.7:0/4269521090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73d4106690 msgr2=0x7f73d4198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:47.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.296+0000 7f73d9a26700 1 --2- 172.21.15.7:0/4269521090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73d4106690 0x7f73d4198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:47.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.296+0000 7f73d9a26700 1 -- 172.21.15.7:0/4269521090 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d4199490 con 0x7f73d4101f80 2024-06-10T01:03:47.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.296+0000 7f73daa28700 1 --2- 172.21.15.7:0/4269521090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73d4106690 0x7f73d4198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:03:47.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.297+0000 7f73d9a26700 1 --2- 172.21.15.7:0/4269521090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73d4101f80 0x7f73d419ede0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f73c40098e0 tx=0x7f73c400b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:47.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.297+0000 7f73cb7fe700 1 -- 172.21.15.7:0/4269521090 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c4027070 con 0x7f73d4101f80 2024-06-10T01:03:47.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.297+0000 7f73cb7fe700 1 -- 172.21.15.7:0/4269521090 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73c4003d70 con 0x7f73d4101f80 2024-06-10T01:03:47.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.297+0000 7f73dc48b700 1 -- 172.21.15.7:0/4269521090 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73d4199720 con 0x7f73d4101f80 2024-06-10T01:03:47.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.297+0000 7f73cb7fe700 1 -- 172.21.15.7:0/4269521090 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c400f460 con 0x7f73d4101f80 2024-06-10T01:03:47.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.297+0000 7f73dc48b700 1 -- 172.21.15.7:0/4269521090 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73d41a5770 con 0x7f73d4101f80 2024-06-10T01:03:47.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.298+0000 7f73dc48b700 1 -- 172.21.15.7:0/4269521090 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f73d4100e60 con 0x7f73d4101f80 2024-06-10T01:03:47.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.299+0000 7f73cb7fe700 1 -- 172.21.15.7:0/4269521090 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f73c402a070 con 0x7f73d4101f80 2024-06-10T01:03:47.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.300+0000 7f73cb7fe700 1 --2- 172.21.15.7:0/4269521090 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f73c00a2cb0 0x7f73c00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:47.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.300+0000 7f73da227700 1 --2- 172.21.15.7:0/4269521090 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f73c00a2cb0 0x7f73c00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:47.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.300+0000 7f73cb7fe700 1 -- 172.21.15.7:0/4269521090 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f73c40d09d0 con 0x7f73d4101f80 2024-06-10T01:03:47.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.300+0000 7f73cb7fe700 1 --2- 172.21.15.7:0/4269521090 >> [v2:172.21.15.78:6800/1856804448,v1:172.21.15.78:6801/1856804448] conn(0x7f73c00ade90 0x7f73c00b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:47.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.300+0000 7f73cb7fe700 1 -- 172.21.15.7:0/4269521090 --> [v2:172.21.15.78:6800/1856804448,v1:172.21.15.78:6801/1856804448] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f73c402be50 con 0x7f73c00ade90 2024-06-10T01:03:47.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.300+0000 7f73cb7fe700 1 -- 172.21.15.7:0/4269521090 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f73c40d0d80 con 0x7f73d4101f80 2024-06-10T01:03:47.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.300+0000 7f73da227700 1 --2- 172.21.15.7:0/4269521090 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f73c00a2cb0 0x7f73c00a5170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f73d410e600 tx=0x7f73d000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:47.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.301+0000 7f73daa28700 1 --2- 172.21.15.7:0/4269521090 >> [v2:172.21.15.78:6800/1856804448,v1:172.21.15.78:6801/1856804448] conn(0x7f73c00ade90 0x7f73c00b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:47.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.301+0000 7f73daa28700 1 --2- 172.21.15.7:0/4269521090 >> [v2:172.21.15.78:6800/1856804448,v1:172.21.15.78:6801/1856804448] conn(0x7f73c00ade90 0x7f73c00b0330 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:47.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.302+0000 7f73cb7fe700 1 -- 172.21.15.7:0/4269521090 <== osd.1 v2:172.21.15.78:6800/1856804448 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f73c402be50 con 0x7f73c00ade90 2024-06-10T01:03:47.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.318+0000 7f73dc48b700 1 -- 172.21.15.7:0/4269521090 --> [v2:172.21.15.78:6800/1856804448,v1:172.21.15.78:6801/1856804448] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f73d4102a90 con 0x7f73c00ade90 2024-06-10T01:03:47.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.319+0000 7f73cb7fe700 1 -- 172.21.15.7:0/4269521090 <== osd.1 v2:172.21.15.78:6800/1856804448 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f73d4102a90 con 0x7f73c00ade90 2024-06-10T01:03:47.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.320+0000 7f73dc48b700 1 -- 172.21.15.7:0/4269521090 >> [v2:172.21.15.78:6800/1856804448,v1:172.21.15.78:6801/1856804448] conn(0x7f73c00ade90 msgr2=0x7f73c00b0330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:47.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.320+0000 7f73dc48b700 1 --2- 172.21.15.7:0/4269521090 >> [v2:172.21.15.78:6800/1856804448,v1:172.21.15.78:6801/1856804448] conn(0x7f73c00ade90 0x7f73c00b0330 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:47.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.320+0000 7f73dc48b700 1 -- 172.21.15.7:0/4269521090 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f73c00a2cb0 msgr2=0x7f73c00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:47.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.320+0000 7f73dc48b700 1 --2- 172.21.15.7:0/4269521090 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f73c00a2cb0 0x7f73c00a5170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f73d410e600 tx=0x7f73d000b3c0 comp rx=0 tx=0).stop 2024-06-10T01:03:47.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.320+0000 7f73dc48b700 1 -- 172.21.15.7:0/4269521090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73d4101f80 msgr2=0x7f73d419ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:47.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.320+0000 7f73dc48b700 1 --2- 172.21.15.7:0/4269521090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73d4101f80 0x7f73d419ede0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f73c40098e0 tx=0x7f73c400b7e0 comp rx=0 tx=0).stop 2024-06-10T01:03:47.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.320+0000 7f73daa28700 1 -- 172.21.15.7:0/4269521090 reap_dead start 2024-06-10T01:03:47.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.320+0000 7f73dc48b700 1 -- 172.21.15.7:0/4269521090 shutdown_connections 2024-06-10T01:03:47.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.320+0000 7f73dc48b700 1 -- 172.21.15.7:0/4269521090 >> 172.21.15.7:0/4269521090 conn(0x7f73d4078330 msgr2=0x7f73d4104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:47.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.320+0000 7f73dc48b700 1 -- 172.21.15.7:0/4269521090 shutdown_connections 2024-06-10T01:03:47.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.320+0000 7f73dc48b700 1 -- 172.21.15.7:0/4269521090 wait complete. 2024-06-10T01:03:47.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:47 smithi175 ceph-mon[36636]: pgmap v364: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s; 13 B/s, 1 objects/s recovering 2024-06-10T01:03:48.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.492+0000 7fcfc3206700 1 -- 172.21.15.7:0/3956238981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfbc074fe0 msgr2=0x7fcfbc10e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:48.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.492+0000 7fcfc3206700 1 --2- 172.21.15.7:0/3956238981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfbc074fe0 0x7fcfbc10e980 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009990 tx=0x7fcfa8009ca0 comp rx=0 tx=0).stop 2024-06-10T01:03:48.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.492+0000 7fcfc3206700 1 -- 172.21.15.7:0/3956238981 shutdown_connections 2024-06-10T01:03:48.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.492+0000 7fcfc3206700 1 --2- 172.21.15.7:0/3956238981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfbc10eec0 0x7fcfbc1112b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:48.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.492+0000 7fcfc3206700 1 --2- 172.21.15.7:0/3956238981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfbc0746c0 0x7fcfbc074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:48.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.492+0000 7fcfc3206700 1 --2- 172.21.15.7:0/3956238981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfbc074fe0 0x7fcfbc10e980 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:48.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.492+0000 7fcfc3206700 1 -- 172.21.15.7:0/3956238981 >> 172.21.15.7:0/3956238981 conn(0x7fcfbc079560 msgr2=0x7fcfbc079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:48.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.492+0000 7fcfc3206700 1 -- 172.21.15.7:0/3956238981 shutdown_connections 2024-06-10T01:03:48.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.492+0000 7fcfc3206700 1 -- 172.21.15.7:0/3956238981 wait complete. 2024-06-10T01:03:48.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.492+0000 7fcfc3206700 1 Processor -- start 2024-06-10T01:03:48.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.492+0000 7fcfc3206700 1 -- start start 2024-06-10T01:03:48.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.493+0000 7fcfc3206700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfbc0746c0 0x7fcfbc19add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:48.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.493+0000 7fcfc3206700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfbc074fe0 0x7fcfbc19b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:48.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.493+0000 7fcfc3206700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfbc10eec0 0x7fcfbc19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:48.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.493+0000 7fcfc3206700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc114a20 con 0x7fcfbc0746c0 2024-06-10T01:03:48.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.493+0000 7fcfc3206700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc1148a0 con 0x7fcfbc10eec0 2024-06-10T01:03:48.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.493+0000 7fcfc3206700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc114ba0 con 0x7fcfbc074fe0 2024-06-10T01:03:48.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.493+0000 7fcfc0fa2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfbc0746c0 0x7fcfbc19add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:48.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.493+0000 7fcfc0fa2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfbc0746c0 0x7fcfbc19add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43074/0 (socket says 172.21.15.7:43074) 2024-06-10T01:03:48.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.493+0000 7fcfc0fa2700 1 -- 172.21.15.7:0/1400672527 learned_addr learned my addr 172.21.15.7:0/1400672527 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:03:48.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.493+0000 7fcfc17a3700 1 --2- 172.21.15.7:0/1400672527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfbc10eec0 0x7fcfbc19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:48.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.493+0000 7fcfbbfff700 1 --2- 172.21.15.7:0/1400672527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfbc074fe0 0x7fcfbc19b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:48.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.493+0000 7fcfc0fa2700 1 -- 172.21.15.7:0/1400672527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfbc074fe0 msgr2=0x7fcfbc19b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:48.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.493+0000 7fcfc0fa2700 1 --2- 172.21.15.7:0/1400672527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfbc074fe0 0x7fcfbc19b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:48.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.493+0000 7fcfc0fa2700 1 -- 172.21.15.7:0/1400672527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfbc10eec0 msgr2=0x7fcfbc19f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:48.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.493+0000 7fcfc0fa2700 1 --2- 172.21.15.7:0/1400672527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfbc10eec0 0x7fcfbc19f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:48.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.493+0000 7fcfc0fa2700 1 -- 172.21.15.7:0/1400672527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfbc19fde0 con 0x7fcfbc0746c0 2024-06-10T01:03:48.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.493+0000 7fcfbbfff700 1 --2- 172.21.15.7:0/1400672527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfbc074fe0 0x7fcfbc19b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:03:48.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.493+0000 7fcfc0fa2700 1 --2- 172.21.15.7:0/1400672527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfbc0746c0 0x7fcfbc19add0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000d950 tx=0x7fcfb000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:48.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.493+0000 7fcfb9ffb700 1 -- 172.21.15.7:0/1400672527 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0016070 con 0x7fcfbc0746c0 2024-06-10T01:03:48.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.493+0000 7fcfc3206700 1 -- 172.21.15.7:0/1400672527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfbc1a00d0 con 0x7fcfbc0746c0 2024-06-10T01:03:48.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.493+0000 7fcfb9ffb700 1 -- 172.21.15.7:0/1400672527 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfb0009e20 con 0x7fcfbc0746c0 2024-06-10T01:03:48.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.493+0000 7fcfb9ffb700 1 -- 172.21.15.7:0/1400672527 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb000f620 con 0x7fcfbc0746c0 2024-06-10T01:03:48.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.494+0000 7fcfc3206700 1 -- 172.21.15.7:0/1400672527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfbc1a7950 con 0x7fcfbc0746c0 2024-06-10T01:03:48.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.494+0000 7fcfc3206700 1 -- 172.21.15.7:0/1400672527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcfbc0ffff0 con 0x7fcfbc0746c0 2024-06-10T01:03:48.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.496+0000 7fcfb9ffb700 1 -- 172.21.15.7:0/1400672527 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fcfb0019070 con 0x7fcfbc0746c0 2024-06-10T01:03:48.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.496+0000 7fcfb9ffb700 1 --2- 172.21.15.7:0/1400672527 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcfac0a2cb0 0x7fcfac0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:48.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.496+0000 7fcfbbfff700 1 --2- 172.21.15.7:0/1400672527 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcfac0a2cb0 0x7fcfac0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:48.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.496+0000 7fcfb9ffb700 1 -- 172.21.15.7:0/1400672527 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fcfb00cf620 con 0x7fcfbc0746c0 2024-06-10T01:03:48.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.497+0000 7fcfb9ffb700 1 --2- 172.21.15.7:0/1400672527 >> [v2:172.21.15.175:6808/2157664731,v1:172.21.15.175:6809/2157664731] conn(0x7fcfac0ade90 0x7fcfac0b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:48.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.497+0000 7fcfb9ffb700 1 -- 172.21.15.7:0/1400672527 --> [v2:172.21.15.175:6808/2157664731,v1:172.21.15.175:6809/2157664731] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcfb0010e50 con 0x7fcfac0ade90 2024-06-10T01:03:48.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.497+0000 7fcfb9ffb700 1 -- 172.21.15.7:0/1400672527 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7fcfb00cfa70 con 0x7fcfbc0746c0 2024-06-10T01:03:48.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.497+0000 7fcfbbfff700 1 --2- 172.21.15.7:0/1400672527 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcfac0a2cb0 0x7fcfac0a5170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800b5c0 tx=0x7fcfa8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:48.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.497+0000 7fcfc17a3700 1 --2- 172.21.15.7:0/1400672527 >> [v2:172.21.15.175:6808/2157664731,v1:172.21.15.175:6809/2157664731] conn(0x7fcfac0ade90 0x7fcfac0b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:48.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.497+0000 7fcfc17a3700 1 --2- 172.21.15.7:0/1400672527 >> [v2:172.21.15.175:6808/2157664731,v1:172.21.15.175:6809/2157664731] conn(0x7fcfac0ade90 0x7fcfac0b0330 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:48.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.498+0000 7fcfb9ffb700 1 -- 172.21.15.7:0/1400672527 <== osd.5 v2:172.21.15.175:6808/2157664731 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcfb0010e50 con 0x7fcfac0ade90 2024-06-10T01:03:48.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.516+0000 7fcfc3206700 1 -- 172.21.15.7:0/1400672527 --> [v2:172.21.15.175:6808/2157664731,v1:172.21.15.175:6809/2157664731] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcfbc075db0 con 0x7fcfac0ade90 2024-06-10T01:03:48.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.516+0000 7fcfb9ffb700 1 -- 172.21.15.7:0/1400672527 <== osd.5 v2:172.21.15.175:6808/2157664731 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcfbc075db0 con 0x7fcfac0ade90 2024-06-10T01:03:48.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.517+0000 7fcfc3206700 1 -- 172.21.15.7:0/1400672527 >> [v2:172.21.15.175:6808/2157664731,v1:172.21.15.175:6809/2157664731] conn(0x7fcfac0ade90 msgr2=0x7fcfac0b0330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:48.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.517+0000 7fcfc3206700 1 --2- 172.21.15.7:0/1400672527 >> [v2:172.21.15.175:6808/2157664731,v1:172.21.15.175:6809/2157664731] conn(0x7fcfac0ade90 0x7fcfac0b0330 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:48.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.517+0000 7fcfc3206700 1 -- 172.21.15.7:0/1400672527 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcfac0a2cb0 msgr2=0x7fcfac0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:48.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.517+0000 7fcfc3206700 1 --2- 172.21.15.7:0/1400672527 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcfac0a2cb0 0x7fcfac0a5170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800b5c0 tx=0x7fcfa8005fd0 comp rx=0 tx=0).stop 2024-06-10T01:03:48.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.517+0000 7fcfc3206700 1 -- 172.21.15.7:0/1400672527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfbc0746c0 msgr2=0x7fcfbc19add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:48.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.517+0000 7fcfc3206700 1 --2- 172.21.15.7:0/1400672527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfbc0746c0 0x7fcfbc19add0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000d950 tx=0x7fcfb000dc60 comp rx=0 tx=0).stop 2024-06-10T01:03:48.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.517+0000 7fcfc17a3700 1 -- 172.21.15.7:0/1400672527 reap_dead start 2024-06-10T01:03:48.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.517+0000 7fcfc3206700 1 -- 172.21.15.7:0/1400672527 shutdown_connections 2024-06-10T01:03:48.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.517+0000 7fcfc3206700 1 -- 172.21.15.7:0/1400672527 >> 172.21.15.7:0/1400672527 conn(0x7fcfbc079560 msgr2=0x7fcfbc0ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:48.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.517+0000 7fcfc3206700 1 -- 172.21.15.7:0/1400672527 shutdown_connections 2024-06-10T01:03:48.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:46.517+0000 7fcfc3206700 1 -- 172.21.15.7:0/1400672527 wait complete. 2024-06-10T01:03:48.632 INFO:teuthology.orchestra.run.smithi007.stdout:163208757317 2024-06-10T01:03:48.632 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-06-10T01:03:49.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.155+0000 7fc424b2d700 1 -- 172.21.15.7:0/1527635683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc420101520 msgr2=0x7fc420111210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:49.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.155+0000 7fc424b2d700 1 --2- 172.21.15.7:0/1527635683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc420101520 0x7fc420111210 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc418009f90 tx=0x7fc41800b330 comp rx=0 tx=0).stop 2024-06-10T01:03:49.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.156+0000 7fc424b2d700 1 -- 172.21.15.7:0/1527635683 shutdown_connections 2024-06-10T01:03:49.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.156+0000 7fc424b2d700 1 --2- 172.21.15.7:0/1527635683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc420101520 0x7fc420111210 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:49.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.156+0000 7fc424b2d700 1 --2- 172.21.15.7:0/1527635683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4201033a0 0x7fc420103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:49.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.156+0000 7fc424b2d700 1 --2- 172.21.15.7:0/1527635683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc420100a80 0x7fc420100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:49.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.156+0000 7fc424b2d700 1 -- 172.21.15.7:0/1527635683 >> 172.21.15.7:0/1527635683 conn(0x7fc420077a00 msgr2=0x7fc4200fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:49.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.156+0000 7fc424b2d700 1 -- 172.21.15.7:0/1527635683 shutdown_connections 2024-06-10T01:03:49.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.156+0000 7fc424b2d700 1 -- 172.21.15.7:0/1527635683 wait complete. 2024-06-10T01:03:49.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.156+0000 7fc424b2d700 1 Processor -- start 2024-06-10T01:03:49.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.157+0000 7fc424b2d700 1 -- start start 2024-06-10T01:03:49.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.157+0000 7fc424b2d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc420100a80 0x7fc42019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:49.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.157+0000 7fc424b2d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc420101520 0x7fc42019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:49.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.157+0000 7fc424b2d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4201033a0 0x7fc42019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:49.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.157+0000 7fc424b2d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc4201149a0 con 0x7fc420100a80 2024-06-10T01:03:49.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.157+0000 7fc424b2d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc420114820 con 0x7fc4201033a0 2024-06-10T01:03:49.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.157+0000 7fc424b2d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc420114b20 con 0x7fc420101520 2024-06-10T01:03:49.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.157+0000 7fc41e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc420100a80 0x7fc42019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:49.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.157+0000 7fc41e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc420100a80 0x7fc42019ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43096/0 (socket says 172.21.15.7:43096) 2024-06-10T01:03:49.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.157+0000 7fc41ed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4201033a0 0x7fc42019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:49.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.157+0000 7fc41e59c700 1 -- 172.21.15.7:0/4163184826 learned_addr learned my addr 172.21.15.7:0/4163184826 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:03:49.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.157+0000 7fc41dd9b700 1 --2- 172.21.15.7:0/4163184826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc420101520 0x7fc42019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:49.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.158+0000 7fc41e59c700 1 -- 172.21.15.7:0/4163184826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc420101520 msgr2=0x7fc42019b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:49.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.158+0000 7fc41e59c700 1 --2- 172.21.15.7:0/4163184826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc420101520 0x7fc42019b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:49.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.158+0000 7fc41e59c700 1 -- 172.21.15.7:0/4163184826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4201033a0 msgr2=0x7fc42019f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:49.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.158+0000 7fc41e59c700 1 --2- 172.21.15.7:0/4163184826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4201033a0 0x7fc42019f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:49.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.158+0000 7fc41e59c700 1 -- 172.21.15.7:0/4163184826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42019fdf0 con 0x7fc420100a80 2024-06-10T01:03:49.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.159+0000 7fc41e59c700 1 --2- 172.21.15.7:0/4163184826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc420100a80 0x7fc42019ae30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc410009f70 tx=0x7fc41000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:49.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.160+0000 7fc40f7fe700 1 -- 172.21.15.7:0/4163184826 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41000ee60 con 0x7fc420100a80 2024-06-10T01:03:49.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.160+0000 7fc424b2d700 1 -- 172.21.15.7:0/4163184826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4201a00e0 con 0x7fc420100a80 2024-06-10T01:03:49.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.160+0000 7fc424b2d700 1 -- 172.21.15.7:0/4163184826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc420103da0 con 0x7fc420100a80 2024-06-10T01:03:49.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.160+0000 7fc40f7fe700 1 -- 172.21.15.7:0/4163184826 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc410004d10 con 0x7fc420100a80 2024-06-10T01:03:49.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.161+0000 7fc40f7fe700 1 -- 172.21.15.7:0/4163184826 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc410005350 con 0x7fc420100a80 2024-06-10T01:03:49.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.161+0000 7fc424b2d700 1 -- 172.21.15.7:0/4163184826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc400001030 con 0x7fc420100a80 2024-06-10T01:03:49.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.164+0000 7fc40f7fe700 1 -- 172.21.15.7:0/4163184826 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fc41000c980 con 0x7fc420100a80 2024-06-10T01:03:49.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.165+0000 7fc40f7fe700 1 --2- 172.21.15.7:0/4163184826 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc4080a2d30 0x7fc4080a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:49.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.165+0000 7fc40f7fe700 1 -- 172.21.15.7:0/4163184826 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fc4100d8360 con 0x7fc420100a80 2024-06-10T01:03:49.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.165+0000 7fc41dd9b700 1 --2- 172.21.15.7:0/4163184826 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc4080a2d30 0x7fc4080a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:49.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.165+0000 7fc40f7fe700 1 --2- 172.21.15.7:0/4163184826 >> [v2:172.21.15.78:6816/3511921773,v1:172.21.15.78:6817/3511921773] conn(0x7fc4080adf10 0x7fc4080b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:49.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.165+0000 7fc40f7fe700 1 -- 172.21.15.7:0/4163184826 --> [v2:172.21.15.78:6816/3511921773,v1:172.21.15.78:6817/3511921773] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc41000ae50 con 0x7fc4080adf10 2024-06-10T01:03:49.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.165+0000 7fc40f7fe700 1 -- 172.21.15.7:0/4163184826 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7fc4100d87b0 con 0x7fc420100a80 2024-06-10T01:03:49.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.165+0000 7fc41ed9d700 1 --2- 172.21.15.7:0/4163184826 >> [v2:172.21.15.78:6816/3511921773,v1:172.21.15.78:6817/3511921773] conn(0x7fc4080adf10 0x7fc4080b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:49.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.165+0000 7fc41dd9b700 1 --2- 172.21.15.7:0/4163184826 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc4080a2d30 0x7fc4080a51f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc414009680 tx=0x7fc414009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:49.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.166+0000 7fc41ed9d700 1 --2- 172.21.15.7:0/4163184826 >> [v2:172.21.15.78:6816/3511921773,v1:172.21.15.78:6817/3511921773] conn(0x7fc4080adf10 0x7fc4080b03b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:49.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.167+0000 7fc40f7fe700 1 -- 172.21.15.7:0/4163184826 <== osd.7 v2:172.21.15.78:6816/3511921773 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc41000ae50 con 0x7fc4080adf10 2024-06-10T01:03:49.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.183+0000 7fc424b2d700 1 -- 172.21.15.7:0/4163184826 --> [v2:172.21.15.78:6816/3511921773,v1:172.21.15.78:6817/3511921773] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc400002e60 con 0x7fc4080adf10 2024-06-10T01:03:49.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.184+0000 7fc40f7fe700 1 -- 172.21.15.7:0/4163184826 <== osd.7 v2:172.21.15.78:6816/3511921773 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc400002e60 con 0x7fc4080adf10 2024-06-10T01:03:49.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.184+0000 7fc424b2d700 1 -- 172.21.15.7:0/4163184826 >> [v2:172.21.15.78:6816/3511921773,v1:172.21.15.78:6817/3511921773] conn(0x7fc4080adf10 msgr2=0x7fc4080b03b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:49.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.184+0000 7fc424b2d700 1 --2- 172.21.15.7:0/4163184826 >> [v2:172.21.15.78:6816/3511921773,v1:172.21.15.78:6817/3511921773] conn(0x7fc4080adf10 0x7fc4080b03b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:49.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.184+0000 7fc424b2d700 1 -- 172.21.15.7:0/4163184826 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc4080a2d30 msgr2=0x7fc4080a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:49.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.184+0000 7fc424b2d700 1 --2- 172.21.15.7:0/4163184826 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc4080a2d30 0x7fc4080a51f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc414009680 tx=0x7fc414009260 comp rx=0 tx=0).stop 2024-06-10T01:03:49.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.184+0000 7fc424b2d700 1 -- 172.21.15.7:0/4163184826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc420100a80 msgr2=0x7fc42019ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:49.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.184+0000 7fc424b2d700 1 --2- 172.21.15.7:0/4163184826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc420100a80 0x7fc42019ae30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc410009f70 tx=0x7fc41000ec20 comp rx=0 tx=0).stop 2024-06-10T01:03:49.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.184+0000 7fc41ed9d700 1 -- 172.21.15.7:0/4163184826 reap_dead start 2024-06-10T01:03:49.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.184+0000 7fc424b2d700 1 -- 172.21.15.7:0/4163184826 shutdown_connections 2024-06-10T01:03:49.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.184+0000 7fc424b2d700 1 -- 172.21.15.7:0/4163184826 >> 172.21.15.7:0/4163184826 conn(0x7fc420077a00 msgr2=0x7fc4200fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:49.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.184+0000 7fc424b2d700 1 -- 172.21.15.7:0/4163184826 shutdown_connections 2024-06-10T01:03:49.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:49.185+0000 7fc424b2d700 1 -- 172.21.15.7:0/4163184826 wait complete. 2024-06-10T01:03:49.399 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:49 smithi007 ceph-mon[32398]: pgmap v365: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 11 B/s, 0 objects/s recovering 2024-06-10T01:03:49.401 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:49 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:03:49.507 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:49 smithi175 ceph-mon[36636]: pgmap v365: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 11 B/s, 0 objects/s recovering 2024-06-10T01:03:49.508 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:49 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:03:49.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:49 smithi078 ceph-mon[36683]: pgmap v365: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 11 B/s, 0 objects/s recovering 2024-06-10T01:03:49.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:49 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:03:51.069 INFO:teuthology.orchestra.run.smithi007.stdout:219043332156 2024-06-10T01:03:51.069 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-06-10T01:03:51.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:51 smithi007 ceph-mon[32398]: pgmap v366: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3 KiB/s rd, 6 op/s; 9 B/s, 0 objects/s recovering 2024-06-10T01:03:51.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:51 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:03:51.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:51 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:03:51.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:51 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:03:51.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:51 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:03:51.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.485+0000 7f65e234d700 1 -- 172.21.15.7:0/2446970287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65dc102740 msgr2=0x7f65dc102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:51.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.485+0000 7f65e234d700 1 --2- 172.21.15.7:0/2446970287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65dc102740 0x7f65dc102b20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f65cc0099c0 tx=0x7f65cc009cd0 comp rx=0 tx=0).stop 2024-06-10T01:03:51.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.485+0000 7f65e234d700 1 -- 172.21.15.7:0/2446970287 shutdown_connections 2024-06-10T01:03:51.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.485+0000 7f65e234d700 1 --2- 172.21.15.7:0/2446970287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65dc107840 0x7f65dc10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:51.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.485+0000 7f65e234d700 1 --2- 172.21.15.7:0/2446970287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65dc102740 0x7f65dc102b20 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:51.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.485+0000 7f65e234d700 1 --2- 172.21.15.7:0/2446970287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65dc1030f0 0x7f65dc107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:51.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.485+0000 7f65e234d700 1 -- 172.21.15.7:0/2446970287 >> 172.21.15.7:0/2446970287 conn(0x7f65dc0795f0 msgr2=0x7f65dc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:51.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.485+0000 7f65e234d700 1 -- 172.21.15.7:0/2446970287 shutdown_connections 2024-06-10T01:03:51.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.485+0000 7f65e234d700 1 -- 172.21.15.7:0/2446970287 wait complete. 2024-06-10T01:03:51.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.486+0000 7f65e234d700 1 Processor -- start 2024-06-10T01:03:51.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.486+0000 7f65e234d700 1 -- start start 2024-06-10T01:03:51.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.486+0000 7f65e234d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65dc102740 0x7f65dc19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:51.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.486+0000 7f65e234d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65dc1030f0 0x7f65dc19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:51.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.486+0000 7f65e234d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65dc107840 0x7f65dc1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:51.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.486+0000 7f65e234d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc112780 con 0x7f65dc107840 2024-06-10T01:03:51.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.486+0000 7f65e234d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc112600 con 0x7f65dc1030f0 2024-06-10T01:03:51.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.486+0000 7f65e234d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc112900 con 0x7f65dc102740 2024-06-10T01:03:51.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.486+0000 7f65dbfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65dc102740 0x7f65dc19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:51.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.486+0000 7f65e08ea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65dc107840 0x7f65dc1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:51.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.486+0000 7f65e08ea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65dc107840 0x7f65dc1989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43126/0 (socket says 172.21.15.7:43126) 2024-06-10T01:03:51.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.486+0000 7f65e08ea700 1 -- 172.21.15.7:0/1811385190 learned_addr learned my addr 172.21.15.7:0/1811385190 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:03:51.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.486+0000 7f65dbfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65dc102740 0x7f65dc19e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36390/0 (socket says 172.21.15.7:36390) 2024-06-10T01:03:51.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.487+0000 7f65d3fff700 1 --2- 172.21.15.7:0/1811385190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65dc1030f0 0x7f65dc19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:51.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.487+0000 7f65e08ea700 1 -- 172.21.15.7:0/1811385190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65dc102740 msgr2=0x7f65dc19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:51.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.487+0000 7f65e08ea700 1 --2- 172.21.15.7:0/1811385190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65dc102740 0x7f65dc19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:51.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.487+0000 7f65e08ea700 1 -- 172.21.15.7:0/1811385190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65dc1030f0 msgr2=0x7f65dc19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:51.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.487+0000 7f65e08ea700 1 --2- 172.21.15.7:0/1811385190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65dc1030f0 0x7f65dc19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:51.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.487+0000 7f65e08ea700 1 -- 172.21.15.7:0/1811385190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65dc199140 con 0x7f65dc107840 2024-06-10T01:03:51.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.487+0000 7f65dbfff700 1 --2- 172.21.15.7:0/1811385190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65dc102740 0x7f65dc19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:03:51.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.487+0000 7f65e08ea700 1 --2- 172.21.15.7:0/1811385190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65dc107840 0x7f65dc1989e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f65d400ef30 tx=0x7f65d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:51.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.487+0000 7f65d9ffb700 1 -- 172.21.15.7:0/1811385190 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d40092e0 con 0x7f65dc107840 2024-06-10T01:03:51.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.487+0000 7f65d9ffb700 1 -- 172.21.15.7:0/1811385190 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65d4004bb0 con 0x7f65dc107840 2024-06-10T01:03:51.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.487+0000 7f65e234d700 1 -- 172.21.15.7:0/1811385190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65dc1993d0 con 0x7f65dc107840 2024-06-10T01:03:51.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.487+0000 7f65d9ffb700 1 -- 172.21.15.7:0/1811385190 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d4004280 con 0x7f65dc107840 2024-06-10T01:03:51.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.487+0000 7f65e234d700 1 -- 172.21.15.7:0/1811385190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65dc1a56f0 con 0x7f65dc107840 2024-06-10T01:03:51.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.488+0000 7f65e234d700 1 -- 172.21.15.7:0/1811385190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f65dc062be0 con 0x7f65dc107840 2024-06-10T01:03:51.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.490+0000 7f65d9ffb700 1 -- 172.21.15.7:0/1811385190 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f65d401b030 con 0x7f65dc107840 2024-06-10T01:03:51.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.490+0000 7f65d9ffb700 1 --2- 172.21.15.7:0/1811385190 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f65c80a7140 0x7f65c80a9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:51.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.490+0000 7f65dbfff700 1 --2- 172.21.15.7:0/1811385190 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f65c80a7140 0x7f65c80a9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:51.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.490+0000 7f65d9ffb700 1 -- 172.21.15.7:0/1811385190 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f65d40cf810 con 0x7f65dc107840 2024-06-10T01:03:51.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.490+0000 7f65d9ffb700 1 --2- 172.21.15.7:0/1811385190 >> [v2:172.21.15.7:6826/1844550813,v1:172.21.15.7:6827/1844550813] conn(0x7f65c80b2320 0x7f65c80b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:51.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.491+0000 7f65d9ffb700 1 -- 172.21.15.7:0/1811385190 --> [v2:172.21.15.7:6826/1844550813,v1:172.21.15.7:6827/1844550813] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f65d401fe50 con 0x7f65c80b2320 2024-06-10T01:03:51.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.491+0000 7f65d9ffb700 1 -- 172.21.15.7:0/1811385190 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f65d40cfc60 con 0x7f65dc107840 2024-06-10T01:03:51.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.491+0000 7f65d3fff700 1 --2- 172.21.15.7:0/1811385190 >> [v2:172.21.15.7:6826/1844550813,v1:172.21.15.7:6827/1844550813] conn(0x7f65c80b2320 0x7f65c80b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:51.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.491+0000 7f65dbfff700 1 --2- 172.21.15.7:0/1811385190 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f65c80a7140 0x7f65c80a9600 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f65dc199b20 tx=0x7f65cc005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:51.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.491+0000 7f65d3fff700 1 --2- 172.21.15.7:0/1811385190 >> [v2:172.21.15.7:6826/1844550813,v1:172.21.15.7:6827/1844550813] conn(0x7f65c80b2320 0x7f65c80b47c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:51.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.492+0000 7f65d9ffb700 1 -- 172.21.15.7:0/1811385190 <== osd.9 v2:172.21.15.7:6826/1844550813 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f65d401fe50 con 0x7f65c80b2320 2024-06-10T01:03:51.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.508+0000 7f65e234d700 1 -- 172.21.15.7:0/1811385190 --> [v2:172.21.15.7:6826/1844550813,v1:172.21.15.7:6827/1844550813] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f65dc063010 con 0x7f65c80b2320 2024-06-10T01:03:51.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.508+0000 7f65d9ffb700 1 -- 172.21.15.7:0/1811385190 <== osd.9 v2:172.21.15.7:6826/1844550813 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f65dc063010 con 0x7f65c80b2320 2024-06-10T01:03:51.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.509+0000 7f65e234d700 1 -- 172.21.15.7:0/1811385190 >> [v2:172.21.15.7:6826/1844550813,v1:172.21.15.7:6827/1844550813] conn(0x7f65c80b2320 msgr2=0x7f65c80b47c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:51.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.509+0000 7f65e234d700 1 --2- 172.21.15.7:0/1811385190 >> [v2:172.21.15.7:6826/1844550813,v1:172.21.15.7:6827/1844550813] conn(0x7f65c80b2320 0x7f65c80b47c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:51.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.509+0000 7f65e234d700 1 -- 172.21.15.7:0/1811385190 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f65c80a7140 msgr2=0x7f65c80a9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:51.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.509+0000 7f65e234d700 1 --2- 172.21.15.7:0/1811385190 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f65c80a7140 0x7f65c80a9600 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f65dc199b20 tx=0x7f65cc005a90 comp rx=0 tx=0).stop 2024-06-10T01:03:51.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.509+0000 7f65e234d700 1 -- 172.21.15.7:0/1811385190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65dc107840 msgr2=0x7f65dc1989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:51.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.509+0000 7f65e234d700 1 --2- 172.21.15.7:0/1811385190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65dc107840 0x7f65dc1989e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f65d400ef30 tx=0x7f65d400c3a0 comp rx=0 tx=0).stop 2024-06-10T01:03:51.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.509+0000 7f65e08ea700 1 -- 172.21.15.7:0/1811385190 reap_dead start 2024-06-10T01:03:51.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.509+0000 7f65e234d700 1 -- 172.21.15.7:0/1811385190 shutdown_connections 2024-06-10T01:03:51.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.509+0000 7f65e234d700 1 -- 172.21.15.7:0/1811385190 >> 172.21.15.7:0/1811385190 conn(0x7f65dc0795f0 msgr2=0x7f65dc1006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:51.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.509+0000 7f65e234d700 1 -- 172.21.15.7:0/1811385190 shutdown_connections 2024-06-10T01:03:51.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.509+0000 7f65e234d700 1 -- 172.21.15.7:0/1811385190 wait complete. 2024-06-10T01:03:51.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:51 smithi078 ceph-mon[36683]: pgmap v366: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3 KiB/s rd, 6 op/s; 9 B/s, 0 objects/s recovering 2024-06-10T01:03:51.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:51 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:03:51.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:51 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:03:51.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:51 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:03:51.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:51 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:03:51.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:51 smithi175 ceph-mon[36636]: pgmap v366: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3 KiB/s rd, 6 op/s; 9 B/s, 0 objects/s recovering 2024-06-10T01:03:51.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:51 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:03:51.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:51 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:03:51.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:51 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:03:51.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:51 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:03:52.255 INFO:teuthology.orchestra.run.smithi007.stdout:287762808883 2024-06-10T01:03:52.255 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-06-10T01:03:52.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.871+0000 7f0e9a801700 1 -- 172.21.15.7:0/1142107485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e9406ab10 msgr2=0x7f0e9410f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:52.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.871+0000 7f0e9a801700 1 --2- 172.21.15.7:0/1142107485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e9406ab10 0x7f0e9410f320 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c00b0d0 tx=0x7f0e8c00b3e0 comp rx=0 tx=0).stop 2024-06-10T01:03:52.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.871+0000 7f0e9a801700 1 -- 172.21.15.7:0/1142107485 shutdown_connections 2024-06-10T01:03:52.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.871+0000 7f0e9a801700 1 --2- 172.21.15.7:0/1142107485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e9406ab10 0x7f0e9410f320 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:52.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.871+0000 7f0e9a801700 1 --2- 172.21.15.7:0/1142107485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e941084e0 0x7f0e941088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:52.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.871+0000 7f0e9a801700 1 --2- 172.21.15.7:0/1142107485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e9406a170 0x7f0e9406a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:52.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.871+0000 7f0e9a801700 1 -- 172.21.15.7:0/1142107485 >> 172.21.15.7:0/1142107485 conn(0x7f0e940782c0 msgr2=0x7f0e940786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:52.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.871+0000 7f0e9a801700 1 -- 172.21.15.7:0/1142107485 shutdown_connections 2024-06-10T01:03:52.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.871+0000 7f0e9a801700 1 -- 172.21.15.7:0/1142107485 wait complete. 2024-06-10T01:03:52.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.872+0000 7f0e9a801700 1 Processor -- start 2024-06-10T01:03:52.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.872+0000 7f0e9a801700 1 -- start start 2024-06-10T01:03:52.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.872+0000 7f0e9a801700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e9406a170 0x7f0e94101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:52.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.872+0000 7f0e9a801700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e9406ab10 0x7f0e941019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:52.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.872+0000 7f0e9a801700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e941084e0 0x7f0e94101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:52.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.872+0000 7f0e9a801700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0e94112730 con 0x7f0e9406ab10 2024-06-10T01:03:52.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.872+0000 7f0e9a801700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e941125b0 con 0x7f0e941084e0 2024-06-10T01:03:52.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.872+0000 7f0e9a801700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0e941128b0 con 0x7f0e9406a170 2024-06-10T01:03:52.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.873+0000 7f0e937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e9406ab10 0x7f0e941019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:52.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.873+0000 7f0e93fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e9406a170 0x7f0e94101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:52.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.873+0000 7f0e937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e9406ab10 0x7f0e941019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54612/0 (socket says 172.21.15.7:54612) 2024-06-10T01:03:52.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.873+0000 7f0e93fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e9406a170 0x7f0e94101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:35094/0 (socket says 172.21.15.7:35094) 2024-06-10T01:03:52.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.873+0000 7f0e93fff700 1 -- 172.21.15.7:0/1398338595 learned_addr learned my addr 172.21.15.7:0/1398338595 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:03:52.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.873+0000 7f0e98d9e700 1 --2- 172.21.15.7:0/1398338595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e941084e0 0x7f0e94101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:52.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.873+0000 7f0e93fff700 1 -- 172.21.15.7:0/1398338595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e941084e0 msgr2=0x7f0e94101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:52.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.873+0000 7f0e93fff700 1 --2- 172.21.15.7:0/1398338595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e941084e0 0x7f0e94101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:52.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.873+0000 7f0e93fff700 1 -- 172.21.15.7:0/1398338595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e9406ab10 msgr2=0x7f0e941019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:52.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.873+0000 7f0e93fff700 1 --2- 172.21.15.7:0/1398338595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e9406ab10 0x7f0e941019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:52.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.873+0000 7f0e93fff700 1 -- 172.21.15.7:0/1398338595 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e941a7090 con 0x7f0e9406a170 2024-06-10T01:03:52.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.873+0000 7f0e98d9e700 1 --2- 172.21.15.7:0/1398338595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e941084e0 0x7f0e94101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:03:52.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.873+0000 7f0e937fe700 1 --2- 172.21.15.7:0/1398338595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e9406ab10 0x7f0e941019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:03:52.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.873+0000 7f0e93fff700 1 --2- 172.21.15.7:0/1398338595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e9406a170 0x7f0e94101460 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400d980 tx=0x7f0e8400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:52.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.874+0000 7f0e917fa700 1 -- 172.21.15.7:0/1398338595 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e84016070 con 0x7f0e9406a170 2024-06-10T01:03:52.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.874+0000 7f0e917fa700 1 -- 172.21.15.7:0/1398338595 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e84004d10 con 0x7f0e9406a170 2024-06-10T01:03:52.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.874+0000 7f0e9a801700 1 -- 172.21.15.7:0/1398338595 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e941a7380 con 0x7f0e9406a170 2024-06-10T01:03:52.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.874+0000 7f0e917fa700 1 -- 172.21.15.7:0/1398338595 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e84005670 con 0x7f0e9406a170 2024-06-10T01:03:52.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.874+0000 7f0e9a801700 1 -- 172.21.15.7:0/1398338595 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e941a78c0 con 0x7f0e9406a170 2024-06-10T01:03:52.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.874+0000 7f0e9a801700 1 -- 172.21.15.7:0/1398338595 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0e94062b60 con 0x7f0e9406a170 2024-06-10T01:03:52.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.876+0000 7f0e917fa700 1 -- 172.21.15.7:0/1398338595 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0e84004750 con 0x7f0e9406a170 2024-06-10T01:03:52.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.877+0000 7f0e917fa700 1 --2- 172.21.15.7:0/1398338595 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0e7c0a2cb0 0x7f0e7c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:52.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.878+0000 7f0e937fe700 1 --2- 172.21.15.7:0/1398338595 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0e7c0a2cb0 0x7f0e7c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:52.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.878+0000 7f0e917fa700 1 -- 172.21.15.7:0/1398338595 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f0e840cf980 con 0x7f0e9406a170 2024-06-10T01:03:52.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.878+0000 7f0e917fa700 1 --2- 172.21.15.7:0/1398338595 >> [v2:172.21.15.78:6824/2290824785,v1:172.21.15.78:6825/2290824785] conn(0x7f0e7c0ade90 0x7f0e7c0b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:52.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.878+0000 7f0e917fa700 1 -- 172.21.15.7:0/1398338595 --> [v2:172.21.15.78:6824/2290824785,v1:172.21.15.78:6825/2290824785] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0e84013ec0 con 0x7f0e7c0ade90 2024-06-10T01:03:52.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.878+0000 7f0e917fa700 1 -- 172.21.15.7:0/1398338595 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f0e840cfd30 con 0x7f0e9406a170 2024-06-10T01:03:52.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.878+0000 7f0e937fe700 1 --2- 172.21.15.7:0/1398338595 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0e7c0a2cb0 0x7f0e7c0a5170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0e94078850 tx=0x7f0e8800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:52.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.878+0000 7f0e98d9e700 1 --2- 172.21.15.7:0/1398338595 >> [v2:172.21.15.78:6824/2290824785,v1:172.21.15.78:6825/2290824785] conn(0x7f0e7c0ade90 0x7f0e7c0b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:52.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.879+0000 7f0e98d9e700 1 --2- 172.21.15.7:0/1398338595 >> [v2:172.21.15.78:6824/2290824785,v1:172.21.15.78:6825/2290824785] conn(0x7f0e7c0ade90 0x7f0e7c0b0330 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:52.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.879+0000 7f0e917fa700 1 -- 172.21.15.7:0/1398338595 <== osd.10 v2:172.21.15.78:6824/2290824785 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0e84013ec0 con 0x7f0e7c0ade90 2024-06-10T01:03:52.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.898+0000 7f0e9a801700 1 -- 172.21.15.7:0/1398338595 --> [v2:172.21.15.78:6824/2290824785,v1:172.21.15.78:6825/2290824785] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0e94062fe0 con 0x7f0e7c0ade90 2024-06-10T01:03:52.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.898+0000 7f0e917fa700 1 -- 172.21.15.7:0/1398338595 <== osd.10 v2:172.21.15.78:6824/2290824785 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0e94062fe0 con 0x7f0e7c0ade90 2024-06-10T01:03:52.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.899+0000 7f0e9a801700 1 -- 172.21.15.7:0/1398338595 >> [v2:172.21.15.78:6824/2290824785,v1:172.21.15.78:6825/2290824785] conn(0x7f0e7c0ade90 msgr2=0x7f0e7c0b0330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:52.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.899+0000 7f0e9a801700 1 --2- 172.21.15.7:0/1398338595 >> [v2:172.21.15.78:6824/2290824785,v1:172.21.15.78:6825/2290824785] conn(0x7f0e7c0ade90 0x7f0e7c0b0330 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:52.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.899+0000 7f0e9a801700 1 -- 172.21.15.7:0/1398338595 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0e7c0a2cb0 msgr2=0x7f0e7c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:52.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.899+0000 7f0e9a801700 1 --2- 172.21.15.7:0/1398338595 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0e7c0a2cb0 0x7f0e7c0a5170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0e94078850 tx=0x7f0e8800b410 comp rx=0 tx=0).stop 2024-06-10T01:03:52.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.899+0000 7f0e9a801700 1 -- 172.21.15.7:0/1398338595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e9406a170 msgr2=0x7f0e94101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:52.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.899+0000 7f0e9a801700 1 --2- 172.21.15.7:0/1398338595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e9406a170 0x7f0e94101460 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400d980 tx=0x7f0e8400dc90 comp rx=0 tx=0).stop 2024-06-10T01:03:52.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.899+0000 7f0e98d9e700 1 -- 172.21.15.7:0/1398338595 reap_dead start 2024-06-10T01:03:52.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.899+0000 7f0e9a801700 1 -- 172.21.15.7:0/1398338595 shutdown_connections 2024-06-10T01:03:52.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.899+0000 7f0e9a801700 1 -- 172.21.15.7:0/1398338595 >> 172.21.15.7:0/1398338595 conn(0x7f0e940782c0 msgr2=0x7f0e940ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:52.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.899+0000 7f0e9a801700 1 -- 172.21.15.7:0/1398338595 shutdown_connections 2024-06-10T01:03:52.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:52.899+0000 7f0e9a801700 1 -- 172.21.15.7:0/1398338595 wait complete. 2024-06-10T01:03:53.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:53 smithi007 ceph-mon[32398]: pgmap v367: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 9 B/s, 0 objects/s recovering 2024-06-10T01:03:53.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:53 smithi078 ceph-mon[36683]: pgmap v367: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 9 B/s, 0 objects/s recovering 2024-06-10T01:03:53.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:53 smithi175 ceph-mon[36636]: pgmap v367: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 9 B/s, 0 objects/s recovering 2024-06-10T01:03:53.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.677+0000 7f17f81ed700 1 -- 172.21.15.7:0/2344000525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17f0101d50 msgr2=0x7f17f010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:53.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.677+0000 7f17f81ed700 1 --2- 172.21.15.7:0/2344000525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17f0101d50 0x7f17f010e240 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f17e800b0a0 tx=0x7f17e800b3b0 comp rx=0 tx=0).stop 2024-06-10T01:03:53.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.677+0000 7f17f81ed700 1 -- 172.21.15.7:0/2344000525 shutdown_connections 2024-06-10T01:03:53.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.677+0000 7f17f81ed700 1 --2- 172.21.15.7:0/2344000525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17f0101d50 0x7f17f010e240 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:53.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.677+0000 7f17f81ed700 1 --2- 172.21.15.7:0/2344000525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17f0107400 0x7f17f01077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:53.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.677+0000 7f17f81ed700 1 --2- 172.21.15.7:0/2344000525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f0101390 0x7f17f0101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:53.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.677+0000 7f17f81ed700 1 -- 172.21.15.7:0/2344000525 >> 172.21.15.7:0/2344000525 conn(0x7f17f0078340 msgr2=0x7f17f0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:53.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.677+0000 7f17f81ed700 1 -- 172.21.15.7:0/2344000525 shutdown_connections 2024-06-10T01:03:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.677+0000 7f17f81ed700 1 -- 172.21.15.7:0/2344000525 wait complete. 2024-06-10T01:03:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.678+0000 7f17f81ed700 1 Processor -- start 2024-06-10T01:03:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.678+0000 7f17f81ed700 1 -- start start 2024-06-10T01:03:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.678+0000 7f17f81ed700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17f0101390 0x7f17f019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.678+0000 7f17f81ed700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17f0101d50 0x7f17f019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.678+0000 7f17f81ed700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f0107400 0x7f17f0198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.678+0000 7f17f81ed700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f17f0111690 con 0x7f17f0101390 2024-06-10T01:03:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.678+0000 7f17f81ed700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f17f0111510 con 0x7f17f0107400 2024-06-10T01:03:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.678+0000 7f17f81ed700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f17f0111810 con 0x7f17f0101d50 2024-06-10T01:03:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.678+0000 7f17f5f89700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17f0101390 0x7f17f019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.678+0000 7f17f5f89700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17f0101390 0x7f17f019e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43150/0 (socket says 172.21.15.7:43150) 2024-06-10T01:03:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.678+0000 7f17f5f89700 1 -- 172.21.15.7:0/1826838937 learned_addr learned my addr 172.21.15.7:0/1826838937 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:03:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.679+0000 7f17f5788700 1 --2- 172.21.15.7:0/1826838937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17f0101d50 0x7f17f019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:53.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.679+0000 7f17f678a700 1 --2- 172.21.15.7:0/1826838937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f0107400 0x7f17f0198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:53.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.679+0000 7f17f5f89700 1 -- 172.21.15.7:0/1826838937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17f0101d50 msgr2=0x7f17f019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:53.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.679+0000 7f17f5f89700 1 --2- 172.21.15.7:0/1826838937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17f0101d50 0x7f17f019ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:53.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.679+0000 7f17f5f89700 1 -- 172.21.15.7:0/1826838937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f0107400 msgr2=0x7f17f0198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:53.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.679+0000 7f17f5f89700 1 --2- 172.21.15.7:0/1826838937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f0107400 0x7f17f0198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:53.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.679+0000 7f17f5f89700 1 -- 172.21.15.7:0/1826838937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f0199160 con 0x7f17f0101390 2024-06-10T01:03:53.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.679+0000 7f17f678a700 1 --2- 172.21.15.7:0/1826838937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f0107400 0x7f17f0198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:03:53.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.679+0000 7f17f5f89700 1 --2- 172.21.15.7:0/1826838937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17f0101390 0x7f17f019e840 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f17ec00e9c0 tx=0x7f17ec00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:53.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.679+0000 7f17e6ffd700 1 -- 172.21.15.7:0/1826838937 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec00cbd0 con 0x7f17f0101390 2024-06-10T01:03:53.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.679+0000 7f17f81ed700 1 -- 172.21.15.7:0/1826838937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17f0199450 con 0x7f17f0101390 2024-06-10T01:03:53.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.679+0000 7f17e6ffd700 1 -- 172.21.15.7:0/1826838937 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17ec00cd70 con 0x7f17f0101390 2024-06-10T01:03:53.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.679+0000 7f17e6ffd700 1 -- 172.21.15.7:0/1826838937 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec010430 con 0x7f17f0101390 2024-06-10T01:03:53.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.679+0000 7f17f81ed700 1 -- 172.21.15.7:0/1826838937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17f01a5710 con 0x7f17f0101390 2024-06-10T01:03:53.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.680+0000 7f17f81ed700 1 -- 172.21.15.7:0/1826838937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f17f0105500 con 0x7f17f0101390 2024-06-10T01:03:53.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.682+0000 7f17e6ffd700 1 -- 172.21.15.7:0/1826838937 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f17ec004750 con 0x7f17f0101390 2024-06-10T01:03:53.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.682+0000 7f17e6ffd700 1 --2- 172.21.15.7:0/1826838937 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f17dc0a2cb0 0x7f17dc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:53.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.682+0000 7f17f5788700 1 --2- 172.21.15.7:0/1826838937 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f17dc0a2cb0 0x7f17dc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:53.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.682+0000 7f17e6ffd700 1 -- 172.21.15.7:0/1826838937 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f17ec0cfd80 con 0x7f17f0101390 2024-06-10T01:03:53.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.682+0000 7f17e6ffd700 1 --2- 172.21.15.7:0/1826838937 >> [v2:172.21.15.7:6810/1797404966,v1:172.21.15.7:6811/1797404966] conn(0x7f17dc0ade90 0x7f17dc0b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:53.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.682+0000 7f17e6ffd700 1 -- 172.21.15.7:0/1826838937 --> [v2:172.21.15.7:6810/1797404966,v1:172.21.15.7:6811/1797404966] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f17dc0b0a00 con 0x7f17dc0ade90 2024-06-10T01:03:53.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.683+0000 7f17e6ffd700 1 -- 172.21.15.7:0/1826838937 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f17ec0d01d0 con 0x7f17f0101390 2024-06-10T01:03:53.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.683+0000 7f17f678a700 1 --2- 172.21.15.7:0/1826838937 >> [v2:172.21.15.7:6810/1797404966,v1:172.21.15.7:6811/1797404966] conn(0x7f17dc0ade90 0x7f17dc0b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:53.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.683+0000 7f17f5788700 1 --2- 172.21.15.7:0/1826838937 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f17dc0a2cb0 0x7f17dc0a5170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f17f00788d0 tx=0x7f17e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:53.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.683+0000 7f17f678a700 1 --2- 172.21.15.7:0/1826838937 >> [v2:172.21.15.7:6810/1797404966,v1:172.21.15.7:6811/1797404966] conn(0x7f17dc0ade90 0x7f17dc0b0330 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:53.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.684+0000 7f17e6ffd700 1 -- 172.21.15.7:0/1826838937 <== osd.3 v2:172.21.15.7:6810/1797404966 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f17dc0b0a00 con 0x7f17dc0ade90 2024-06-10T01:03:53.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.701+0000 7f17f81ed700 1 -- 172.21.15.7:0/1826838937 --> [v2:172.21.15.7:6810/1797404966,v1:172.21.15.7:6811/1797404966] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f17f0102ab0 con 0x7f17dc0ade90 2024-06-10T01:03:53.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.701+0000 7f17e6ffd700 1 -- 172.21.15.7:0/1826838937 <== osd.3 v2:172.21.15.7:6810/1797404966 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f17f0102ab0 con 0x7f17dc0ade90 2024-06-10T01:03:53.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.701+0000 7f17f81ed700 1 -- 172.21.15.7:0/1826838937 >> [v2:172.21.15.7:6810/1797404966,v1:172.21.15.7:6811/1797404966] conn(0x7f17dc0ade90 msgr2=0x7f17dc0b0330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:53.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.701+0000 7f17f81ed700 1 --2- 172.21.15.7:0/1826838937 >> [v2:172.21.15.7:6810/1797404966,v1:172.21.15.7:6811/1797404966] conn(0x7f17dc0ade90 0x7f17dc0b0330 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:53.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.701+0000 7f17f81ed700 1 -- 172.21.15.7:0/1826838937 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f17dc0a2cb0 msgr2=0x7f17dc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:53.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.701+0000 7f17f81ed700 1 --2- 172.21.15.7:0/1826838937 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f17dc0a2cb0 0x7f17dc0a5170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f17f00788d0 tx=0x7f17e0009260 comp rx=0 tx=0).stop 2024-06-10T01:03:53.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.701+0000 7f17f81ed700 1 -- 172.21.15.7:0/1826838937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17f0101390 msgr2=0x7f17f019e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:53.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.701+0000 7f17f81ed700 1 --2- 172.21.15.7:0/1826838937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17f0101390 0x7f17f019e840 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f17ec00e9c0 tx=0x7f17ec00ed80 comp rx=0 tx=0).stop 2024-06-10T01:03:53.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.701+0000 7f17f678a700 1 -- 172.21.15.7:0/1826838937 reap_dead start 2024-06-10T01:03:53.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.702+0000 7f17f81ed700 1 -- 172.21.15.7:0/1826838937 shutdown_connections 2024-06-10T01:03:53.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.702+0000 7f17f81ed700 1 -- 172.21.15.7:0/1826838937 >> 172.21.15.7:0/1826838937 conn(0x7f17f0078340 msgr2=0x7f17f01005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:53.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.702+0000 7f17f81ed700 1 -- 172.21.15.7:0/1826838937 shutdown_connections 2024-06-10T01:03:53.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:51.702+0000 7f17f81ed700 1 -- 172.21.15.7:0/1826838937 wait complete. 2024-06-10T01:03:55.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:55 smithi007 ceph-mon[32398]: pgmap v368: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-06-10T01:03:55.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:55 smithi078 ceph-mon[36683]: pgmap v368: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-06-10T01:03:55.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:55 smithi175 ceph-mon[36636]: pgmap v368: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-06-10T01:03:55.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.911+0000 7f535271f700 1 -- 172.21.15.7:0/2655584813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f534c1017f0 msgr2=0x7f534c10f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:55.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.911+0000 7f535271f700 1 --2- 172.21.15.7:0/2655584813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f534c1017f0 0x7f534c10f260 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f534400b0a0 tx=0x7f534400b3b0 comp rx=0 tx=0).stop 2024-06-10T01:03:55.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.911+0000 7f535271f700 1 -- 172.21.15.7:0/2655584813 shutdown_connections 2024-06-10T01:03:55.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.911+0000 7f535271f700 1 --2- 172.21.15.7:0/2655584813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f534c1017f0 0x7f534c10f260 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:55.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.911+0000 7f535271f700 1 --2- 172.21.15.7:0/2655584813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f534c107550 0x7f534c107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:55.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.911+0000 7f535271f700 1 --2- 172.21.15.7:0/2655584813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f534c100e30 0x7f534c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:55.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.911+0000 7f535271f700 1 -- 172.21.15.7:0/2655584813 >> 172.21.15.7:0/2655584813 conn(0x7f534c0766b0 msgr2=0x7f534c076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:55.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.911+0000 7f535271f700 1 -- 172.21.15.7:0/2655584813 shutdown_connections 2024-06-10T01:03:55.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.911+0000 7f535271f700 1 -- 172.21.15.7:0/2655584813 wait complete. 2024-06-10T01:03:55.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.911+0000 7f535271f700 1 Processor -- start 2024-06-10T01:03:55.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.911+0000 7f535271f700 1 -- start start 2024-06-10T01:03:55.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.912+0000 7f535271f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f534c100e30 0x7f534c19e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:55.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.912+0000 7f535271f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f534c1017f0 0x7f534c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:55.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.912+0000 7f535271f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f534c107550 0x7f534c198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:55.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.912+0000 7f535271f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f534c112880 con 0x7f534c107550 2024-06-10T01:03:55.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.912+0000 7f535271f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f534c112700 con 0x7f534c1017f0 2024-06-10T01:03:55.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.912+0000 7f535271f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f534c112a00 con 0x7f534c100e30 2024-06-10T01:03:55.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.912+0000 7f5350cbc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f534c107550 0x7f534c198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:55.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.912+0000 7f5350cbc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f534c107550 0x7f534c198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54630/0 (socket says 172.21.15.7:54630) 2024-06-10T01:03:55.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.912+0000 7f5350cbc700 1 -- 172.21.15.7:0/3316173130 learned_addr learned my addr 172.21.15.7:0/3316173130 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:03:55.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.912+0000 7f534bfff700 1 --2- 172.21.15.7:0/3316173130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f534c100e30 0x7f534c19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:55.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.912+0000 7f5343fff700 1 --2- 172.21.15.7:0/3316173130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f534c1017f0 0x7f534c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:55.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.912+0000 7f5350cbc700 1 -- 172.21.15.7:0/3316173130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f534c100e30 msgr2=0x7f534c19e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:55.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.912+0000 7f5350cbc700 1 --2- 172.21.15.7:0/3316173130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f534c100e30 0x7f534c19e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:55.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.912+0000 7f5350cbc700 1 -- 172.21.15.7:0/3316173130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f534c1017f0 msgr2=0x7f534c19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:55.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.912+0000 7f5350cbc700 1 --2- 172.21.15.7:0/3316173130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f534c1017f0 0x7f534c19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:55.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.912+0000 7f5350cbc700 1 -- 172.21.15.7:0/3316173130 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f534c1991a0 con 0x7f534c107550 2024-06-10T01:03:55.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.912+0000 7f5343fff700 1 --2- 172.21.15.7:0/3316173130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f534c1017f0 0x7f534c19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:03:55.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.912+0000 7f5350cbc700 1 --2- 172.21.15.7:0/3316173130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f534c107550 0x7f534c198a40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f534400d010 tx=0x7f534400bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:55.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.912+0000 7f5349ffb700 1 -- 172.21.15.7:0/3316173130 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5344021070 con 0x7f534c107550 2024-06-10T01:03:55.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.912+0000 7f5349ffb700 1 -- 172.21.15.7:0/3316173130 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5344007c80 con 0x7f534c107550 2024-06-10T01:03:55.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.912+0000 7f535271f700 1 -- 172.21.15.7:0/3316173130 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f534c199430 con 0x7f534c107550 2024-06-10T01:03:55.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.913+0000 7f5349ffb700 1 -- 172.21.15.7:0/3316173130 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534401d450 con 0x7f534c107550 2024-06-10T01:03:55.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.913+0000 7f535271f700 1 -- 172.21.15.7:0/3316173130 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f534c1a5700 con 0x7f534c107550 2024-06-10T01:03:55.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.913+0000 7f535271f700 1 -- 172.21.15.7:0/3316173130 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f534c100260 con 0x7f534c107550 2024-06-10T01:03:55.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.915+0000 7f5349ffb700 1 -- 172.21.15.7:0/3316173130 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f53440040b0 con 0x7f534c107550 2024-06-10T01:03:55.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.915+0000 7f5349ffb700 1 --2- 172.21.15.7:0/3316173130 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f53380a2d30 0x7f53380a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:55.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.915+0000 7f5349ffb700 1 -- 172.21.15.7:0/3316173130 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f534400f020 con 0x7f534c107550 2024-06-10T01:03:55.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.915+0000 7f534bfff700 1 --2- 172.21.15.7:0/3316173130 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f53380a2d30 0x7f53380a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:55.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.916+0000 7f5349ffb700 1 --2- 172.21.15.7:0/3316173130 >> [v2:172.21.15.175:6816/1546325211,v1:172.21.15.175:6817/1546325211] conn(0x7f53380adf10 0x7f53380b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:55.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.916+0000 7f5349ffb700 1 -- 172.21.15.7:0/3316173130 --> [v2:172.21.15.175:6816/1546325211,v1:172.21.15.175:6817/1546325211] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f534401fe80 con 0x7f53380adf10 2024-06-10T01:03:55.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.916+0000 7f5349ffb700 1 -- 172.21.15.7:0/3316173130 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f53440d1170 con 0x7f534c107550 2024-06-10T01:03:55.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.916+0000 7f534bfff700 1 --2- 172.21.15.7:0/3316173130 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f53380a2d30 0x7f53380a51f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f534c0ff500 tx=0x7f533c00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:55.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.916+0000 7f5343fff700 1 --2- 172.21.15.7:0/3316173130 >> [v2:172.21.15.175:6816/1546325211,v1:172.21.15.175:6817/1546325211] conn(0x7f53380adf10 0x7f53380b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:55.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.916+0000 7f5343fff700 1 --2- 172.21.15.7:0/3316173130 >> [v2:172.21.15.175:6816/1546325211,v1:172.21.15.175:6817/1546325211] conn(0x7f53380adf10 0x7f53380b03b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:55.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.917+0000 7f5349ffb700 1 -- 172.21.15.7:0/3316173130 <== osd.8 v2:172.21.15.175:6816/1546325211 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f534401fe80 con 0x7f53380adf10 2024-06-10T01:03:55.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.934+0000 7f535271f700 1 -- 172.21.15.7:0/3316173130 --> [v2:172.21.15.175:6816/1546325211,v1:172.21.15.175:6817/1546325211] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f534c06a6e0 con 0x7f53380adf10 2024-06-10T01:03:55.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.934+0000 7f5349ffb700 1 -- 172.21.15.7:0/3316173130 <== osd.8 v2:172.21.15.175:6816/1546325211 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f534c06a6e0 con 0x7f53380adf10 2024-06-10T01:03:55.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.934+0000 7f535271f700 1 -- 172.21.15.7:0/3316173130 >> [v2:172.21.15.175:6816/1546325211,v1:172.21.15.175:6817/1546325211] conn(0x7f53380adf10 msgr2=0x7f53380b03b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:55.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.934+0000 7f535271f700 1 --2- 172.21.15.7:0/3316173130 >> [v2:172.21.15.175:6816/1546325211,v1:172.21.15.175:6817/1546325211] conn(0x7f53380adf10 0x7f53380b03b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:55.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.934+0000 7f535271f700 1 -- 172.21.15.7:0/3316173130 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f53380a2d30 msgr2=0x7f53380a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:55.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.934+0000 7f535271f700 1 --2- 172.21.15.7:0/3316173130 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f53380a2d30 0x7f53380a51f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f534c0ff500 tx=0x7f533c00a3b0 comp rx=0 tx=0).stop 2024-06-10T01:03:55.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.934+0000 7f535271f700 1 -- 172.21.15.7:0/3316173130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f534c107550 msgr2=0x7f534c198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:55.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.934+0000 7f535271f700 1 --2- 172.21.15.7:0/3316173130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f534c107550 0x7f534c198a40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f534400d010 tx=0x7f534400bde0 comp rx=0 tx=0).stop 2024-06-10T01:03:55.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.934+0000 7f5350cbc700 1 -- 172.21.15.7:0/3316173130 reap_dead start 2024-06-10T01:03:55.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.935+0000 7f535271f700 1 -- 172.21.15.7:0/3316173130 shutdown_connections 2024-06-10T01:03:55.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.935+0000 7f535271f700 1 -- 172.21.15.7:0/3316173130 >> 172.21.15.7:0/3316173130 conn(0x7f534c0766b0 msgr2=0x7f534c0ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:55.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.935+0000 7f535271f700 1 -- 172.21.15.7:0/3316173130 shutdown_connections 2024-06-10T01:03:55.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:53.935+0000 7f535271f700 1 -- 172.21.15.7:0/3316173130 wait complete. 2024-06-10T01:03:56.921 INFO:teuthology.orchestra.run.smithi007.stdout:107374182481 2024-06-10T01:03:56.921 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-06-10T01:03:57.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.893+0000 7f1876845700 1 -- 172.21.15.7:0/1834929999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f187006ab30 msgr2=0x7f187010f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:57.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.893+0000 7f1876845700 1 --2- 172.21.15.7:0/1834929999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f187006ab30 0x7f187010f320 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f186800b0a0 tx=0x7f186800b3b0 comp rx=0 tx=0).stop 2024-06-10T01:03:57.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.893+0000 7f1876845700 1 -- 172.21.15.7:0/1834929999 shutdown_connections 2024-06-10T01:03:57.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.893+0000 7f1876845700 1 --2- 172.21.15.7:0/1834929999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f187006ab30 0x7f187010f320 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:57.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.893+0000 7f1876845700 1 --2- 172.21.15.7:0/1834929999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1870108500 0x7f18701088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:57.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.893+0000 7f1876845700 1 --2- 172.21.15.7:0/1834929999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f187006a170 0x7f187006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:57.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.893+0000 7f1876845700 1 -- 172.21.15.7:0/1834929999 >> 172.21.15.7:0/1834929999 conn(0x7f1870078340 msgr2=0x7f1870078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:57.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.893+0000 7f1876845700 1 -- 172.21.15.7:0/1834929999 shutdown_connections 2024-06-10T01:03:57.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.893+0000 7f1876845700 1 -- 172.21.15.7:0/1834929999 wait complete. 2024-06-10T01:03:57.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.894+0000 7f1876845700 1 Processor -- start 2024-06-10T01:03:57.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.894+0000 7f1876845700 1 -- start start 2024-06-10T01:03:57.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.894+0000 7f1876845700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f187006a170 0x7f1870101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:57.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.894+0000 7f1876845700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f187006ab30 0x7f18701019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:57.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.894+0000 7f1876845700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1870108500 0x7f1870101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:57.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.894+0000 7f1876845700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1870112760 con 0x7f1870108500 2024-06-10T01:03:57.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.894+0000 7f1876845700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f18701125e0 con 0x7f187006ab30 2024-06-10T01:03:57.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.894+0000 7f1876845700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f18701128e0 con 0x7f187006a170 2024-06-10T01:03:57.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.895+0000 7f1874de2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1870108500 0x7f1870101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:57.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.895+0000 7f1874de2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1870108500 0x7f1870101ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54672/0 (socket says 172.21.15.7:54672) 2024-06-10T01:03:57.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.895+0000 7f1874de2700 1 -- 172.21.15.7:0/1465816020 learned_addr learned my addr 172.21.15.7:0/1465816020 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:03:57.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.895+0000 7f186ffff700 1 --2- 172.21.15.7:0/1465816020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f187006a170 0x7f1870101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:57.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.895+0000 7f186f7fe700 1 --2- 172.21.15.7:0/1465816020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f187006ab30 0x7f18701019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:57.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.895+0000 7f1874de2700 1 -- 172.21.15.7:0/1465816020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f187006a170 msgr2=0x7f1870101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:57.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.895+0000 7f1874de2700 1 --2- 172.21.15.7:0/1465816020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f187006a170 0x7f1870101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:57.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.895+0000 7f1874de2700 1 -- 172.21.15.7:0/1465816020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f187006ab30 msgr2=0x7f18701019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:57.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.895+0000 7f1874de2700 1 --2- 172.21.15.7:0/1465816020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f187006ab30 0x7f18701019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:57.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.895+0000 7f1874de2700 1 -- 172.21.15.7:0/1465816020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18701ab580 con 0x7f1870108500 2024-06-10T01:03:57.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.895+0000 7f1874de2700 1 --2- 172.21.15.7:0/1465816020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1870108500 0x7f1870101ee0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f186800b780 tx=0x7f1868004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:57.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.895+0000 7f186d7fa700 1 -- 172.21.15.7:0/1465816020 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1868021070 con 0x7f1870108500 2024-06-10T01:03:57.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.895+0000 7f186d7fa700 1 -- 172.21.15.7:0/1465816020 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1868007b80 con 0x7f1870108500 2024-06-10T01:03:57.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.895+0000 7f1876845700 1 -- 172.21.15.7:0/1465816020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18701ab7b0 con 0x7f1870108500 2024-06-10T01:03:57.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.895+0000 7f186d7fa700 1 -- 172.21.15.7:0/1465816020 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18680052a0 con 0x7f1870108500 2024-06-10T01:03:57.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.896+0000 7f1876845700 1 -- 172.21.15.7:0/1465816020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18701abcb0 con 0x7f1870108500 2024-06-10T01:03:57.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.896+0000 7f1876845700 1 -- 172.21.15.7:0/1465816020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1870062ba0 con 0x7f1870108500 2024-06-10T01:03:57.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.899+0000 7f186d7fa700 1 -- 172.21.15.7:0/1465816020 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f1868024070 con 0x7f1870108500 2024-06-10T01:03:57.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.899+0000 7f186d7fa700 1 --2- 172.21.15.7:0/1465816020 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f18580a2d30 0x7f18580a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:57.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.899+0000 7f186d7fa700 1 -- 172.21.15.7:0/1465816020 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f186800f020 con 0x7f1870108500 2024-06-10T01:03:57.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.899+0000 7f186ffff700 1 --2- 172.21.15.7:0/1465816020 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f18580a2d30 0x7f18580a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:57.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.900+0000 7f186d7fa700 1 --2- 172.21.15.7:0/1465816020 >> [v2:172.21.15.7:6802/404485726,v1:172.21.15.7:6803/404485726] conn(0x7f18580adf10 0x7f18580b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:57.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.900+0000 7f186d7fa700 1 -- 172.21.15.7:0/1465816020 --> [v2:172.21.15.7:6802/404485726,v1:172.21.15.7:6803/404485726] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1868008ea0 con 0x7f18580adf10 2024-06-10T01:03:57.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.900+0000 7f186d7fa700 1 -- 172.21.15.7:0/1465816020 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f18680d1270 con 0x7f1870108500 2024-06-10T01:03:57.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.900+0000 7f186ffff700 1 --2- 172.21.15.7:0/1465816020 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f18580a2d30 0x7f18580a51f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1860009be0 tx=0x7f1860009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:57.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.900+0000 7f186f7fe700 1 --2- 172.21.15.7:0/1465816020 >> [v2:172.21.15.7:6802/404485726,v1:172.21.15.7:6803/404485726] conn(0x7f18580adf10 0x7f18580b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:57.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.900+0000 7f186f7fe700 1 --2- 172.21.15.7:0/1465816020 >> [v2:172.21.15.7:6802/404485726,v1:172.21.15.7:6803/404485726] conn(0x7f18580adf10 0x7f18580b03b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:57.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.901+0000 7f186d7fa700 1 -- 172.21.15.7:0/1465816020 <== osd.0 v2:172.21.15.7:6802/404485726 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1868008ea0 con 0x7f18580adf10 2024-06-10T01:03:57.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.919+0000 7f1876845700 1 -- 172.21.15.7:0/1465816020 --> [v2:172.21.15.7:6802/404485726,v1:172.21.15.7:6803/404485726] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1870062fd0 con 0x7f18580adf10 2024-06-10T01:03:57.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.919+0000 7f186d7fa700 1 -- 172.21.15.7:0/1465816020 <== osd.0 v2:172.21.15.7:6802/404485726 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1870062fd0 con 0x7f18580adf10 2024-06-10T01:03:57.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.920+0000 7f1876845700 1 -- 172.21.15.7:0/1465816020 >> [v2:172.21.15.7:6802/404485726,v1:172.21.15.7:6803/404485726] conn(0x7f18580adf10 msgr2=0x7f18580b03b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:57.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.920+0000 7f1876845700 1 --2- 172.21.15.7:0/1465816020 >> [v2:172.21.15.7:6802/404485726,v1:172.21.15.7:6803/404485726] conn(0x7f18580adf10 0x7f18580b03b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:57.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.920+0000 7f1876845700 1 -- 172.21.15.7:0/1465816020 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f18580a2d30 msgr2=0x7f18580a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:57.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.920+0000 7f1876845700 1 --2- 172.21.15.7:0/1465816020 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f18580a2d30 0x7f18580a51f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1860009be0 tx=0x7f1860009260 comp rx=0 tx=0).stop 2024-06-10T01:03:57.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.920+0000 7f1876845700 1 -- 172.21.15.7:0/1465816020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1870108500 msgr2=0x7f1870101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:57.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.920+0000 7f1876845700 1 --2- 172.21.15.7:0/1465816020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1870108500 0x7f1870101ee0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f186800b780 tx=0x7f1868004d00 comp rx=0 tx=0).stop 2024-06-10T01:03:57.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.920+0000 7f1874de2700 1 -- 172.21.15.7:0/1465816020 reap_dead start 2024-06-10T01:03:57.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.920+0000 7f1876845700 1 -- 172.21.15.7:0/1465816020 shutdown_connections 2024-06-10T01:03:57.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.920+0000 7f1876845700 1 -- 172.21.15.7:0/1465816020 >> 172.21.15.7:0/1465816020 conn(0x7f1870078340 msgr2=0x7f18700fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:57.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.920+0000 7f1876845700 1 -- 172.21.15.7:0/1465816020 shutdown_connections 2024-06-10T01:03:57.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.920+0000 7f1876845700 1 -- 172.21.15.7:0/1465816020 wait complete. 2024-06-10T01:03:57.394 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:57 smithi007 ceph-mon[32398]: pgmap v369: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-06-10T01:03:57.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:57 smithi078 ceph-mon[36683]: pgmap v369: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-06-10T01:03:57.706 INFO:teuthology.orchestra.run.smithi007.stdout:34359738465 2024-06-10T01:03:57.706 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-06-10T01:03:57.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:57 smithi175 ceph-mon[36636]: pgmap v369: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-06-10T01:03:57.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.911+0000 7f1fee15b700 1 -- 172.21.15.7:0/184835950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fe806aa30 msgr2=0x7f1fe806ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:57.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.911+0000 7f1fee15b700 1 --2- 172.21.15.7:0/184835950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fe806aa30 0x7f1fe806ae10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8009990 tx=0x7f1fd8009ca0 comp rx=0 tx=0).stop 2024-06-10T01:03:57.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.912+0000 7f1fee15b700 1 -- 172.21.15.7:0/184835950 shutdown_connections 2024-06-10T01:03:57.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.912+0000 7f1fee15b700 1 --2- 172.21.15.7:0/184835950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fe810ef30 0x7f1fe8111320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:57.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.912+0000 7f1fee15b700 1 --2- 172.21.15.7:0/184835950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fe806aa30 0x7f1fe806ae10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:57.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.912+0000 7f1fee15b700 1 --2- 172.21.15.7:0/184835950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fe806b350 0x7f1fe810e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:57.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.912+0000 7f1fee15b700 1 -- 172.21.15.7:0/184835950 >> 172.21.15.7:0/184835950 conn(0x7f1fe8078300 msgr2=0x7f1fe8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:57.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.912+0000 7f1fee15b700 1 -- 172.21.15.7:0/184835950 shutdown_connections 2024-06-10T01:03:57.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.912+0000 7f1fee15b700 1 -- 172.21.15.7:0/184835950 wait complete. 2024-06-10T01:03:57.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.913+0000 7f1fee15b700 1 Processor -- start 2024-06-10T01:03:57.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.913+0000 7f1fee15b700 1 -- start start 2024-06-10T01:03:57.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.913+0000 7f1fee15b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fe806aa30 0x7f1fe819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:57.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.913+0000 7f1fee15b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fe806b350 0x7f1fe819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:57.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.913+0000 7f1fee15b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fe810ef30 0x7f1fe819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:57.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.913+0000 7f1fee15b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe81149a0 con 0x7f1fe806aa30 2024-06-10T01:03:57.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.913+0000 7f1fee15b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe8114820 con 0x7f1fe810ef30 2024-06-10T01:03:57.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.913+0000 7f1fee15b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe8114b20 con 0x7f1fe806b350 2024-06-10T01:03:57.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.913+0000 7f1fe6ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fe806b350 0x7f1fe819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:57.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.913+0000 7f1fe6ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fe806b350 0x7f1fe819b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:35168/0 (socket says 172.21.15.7:35168) 2024-06-10T01:03:57.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.913+0000 7f1fe6ffd700 1 -- 172.21.15.7:0/1712338456 learned_addr learned my addr 172.21.15.7:0/1712338456 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:03:57.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.914+0000 7f1fe77fe700 1 --2- 172.21.15.7:0/1712338456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fe806aa30 0x7f1fe819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:57.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.914+0000 7f1fe7fff700 1 --2- 172.21.15.7:0/1712338456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fe810ef30 0x7f1fe819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:57.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.914+0000 7f1fe6ffd700 1 -- 172.21.15.7:0/1712338456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fe810ef30 msgr2=0x7f1fe819f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:57.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.914+0000 7f1fe6ffd700 1 --2- 172.21.15.7:0/1712338456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fe810ef30 0x7f1fe819f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:57.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.914+0000 7f1fe6ffd700 1 -- 172.21.15.7:0/1712338456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fe806aa30 msgr2=0x7f1fe819ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:57.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.914+0000 7f1fe6ffd700 1 --2- 172.21.15.7:0/1712338456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fe806aa30 0x7f1fe819ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.914+0000 7f1fe6ffd700 1 -- 172.21.15.7:0/1712338456 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe819fe20 con 0x7f1fe806b350 2024-06-10T01:03:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.914+0000 7f1fe7fff700 1 --2- 172.21.15.7:0/1712338456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fe810ef30 0x7f1fe819f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:03:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.914+0000 7f1fe6ffd700 1 --2- 172.21.15.7:0/1712338456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fe806b350 0x7f1fe819b350 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc00e9a0 tx=0x7f1fdc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.914+0000 7f1fe4ff9700 1 -- 172.21.15.7:0/1712338456 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc00cbd0 con 0x7f1fe806b350 2024-06-10T01:03:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.914+0000 7f1fee15b700 1 -- 172.21.15.7:0/1712338456 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe81a0110 con 0x7f1fe806b350 2024-06-10T01:03:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.914+0000 7f1fe4ff9700 1 -- 172.21.15.7:0/1712338456 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fdc00cd70 con 0x7f1fe806b350 2024-06-10T01:03:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.915+0000 7f1fee15b700 1 -- 172.21.15.7:0/1712338456 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe81a7990 con 0x7f1fe806b350 2024-06-10T01:03:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.915+0000 7f1fe4ff9700 1 -- 172.21.15.7:0/1712338456 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc018750 con 0x7f1fe806b350 2024-06-10T01:03:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.916+0000 7f1fee15b700 1 -- 172.21.15.7:0/1712338456 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1fd4001030 con 0x7f1fe806b350 2024-06-10T01:03:57.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.917+0000 7f1fe4ff9700 1 -- 172.21.15.7:0/1712338456 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f1fdc010430 con 0x7f1fe806b350 2024-06-10T01:03:57.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.918+0000 7f1fe4ff9700 1 --2- 172.21.15.7:0/1712338456 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1fd00a2d30 0x7f1fd00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:57.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.918+0000 7f1fe77fe700 1 --2- 172.21.15.7:0/1712338456 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1fd00a2d30 0x7f1fd00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:57.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.918+0000 7f1fe4ff9700 1 -- 172.21.15.7:0/1712338456 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f1fdc0d0200 con 0x7f1fe806b350 2024-06-10T01:03:57.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.918+0000 7f1fe4ff9700 1 --2- 172.21.15.7:0/1712338456 >> [v2:172.21.15.175:6800/918257455,v1:172.21.15.175:6801/918257455] conn(0x7f1fd00adf10 0x7f1fd00b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:57.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.918+0000 7f1fe4ff9700 1 -- 172.21.15.7:0/1712338456 --> [v2:172.21.15.175:6800/918257455,v1:172.21.15.175:6801/918257455] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1fdc01bec0 con 0x7f1fd00adf10 2024-06-10T01:03:57.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.918+0000 7f1fe4ff9700 1 -- 172.21.15.7:0/1712338456 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f1fdc0d0640 con 0x7f1fe806b350 2024-06-10T01:03:57.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.918+0000 7f1fe77fe700 1 --2- 172.21.15.7:0/1712338456 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1fd00a2d30 0x7f1fd00a51f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800b2e0 tx=0x7f1fd8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:57.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.918+0000 7f1fe7fff700 1 --2- 172.21.15.7:0/1712338456 >> [v2:172.21.15.175:6800/918257455,v1:172.21.15.175:6801/918257455] conn(0x7f1fd00adf10 0x7f1fd00b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:57.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.919+0000 7f1fe7fff700 1 --2- 172.21.15.7:0/1712338456 >> [v2:172.21.15.175:6800/918257455,v1:172.21.15.175:6801/918257455] conn(0x7f1fd00adf10 0x7f1fd00b03b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:57.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.920+0000 7f1fe4ff9700 1 -- 172.21.15.7:0/1712338456 <== osd.2 v2:172.21.15.175:6800/918257455 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1fdc01bec0 con 0x7f1fd00adf10 2024-06-10T01:03:57.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.937+0000 7f1fee15b700 1 -- 172.21.15.7:0/1712338456 --> [v2:172.21.15.175:6800/918257455,v1:172.21.15.175:6801/918257455] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1fd4002de0 con 0x7f1fd00adf10 2024-06-10T01:03:57.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.938+0000 7f1fe4ff9700 1 -- 172.21.15.7:0/1712338456 <== osd.2 v2:172.21.15.175:6800/918257455 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1fd4002de0 con 0x7f1fd00adf10 2024-06-10T01:03:57.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.938+0000 7f1fee15b700 1 -- 172.21.15.7:0/1712338456 >> [v2:172.21.15.175:6800/918257455,v1:172.21.15.175:6801/918257455] conn(0x7f1fd00adf10 msgr2=0x7f1fd00b03b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:57.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.938+0000 7f1fee15b700 1 --2- 172.21.15.7:0/1712338456 >> [v2:172.21.15.175:6800/918257455,v1:172.21.15.175:6801/918257455] conn(0x7f1fd00adf10 0x7f1fd00b03b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:57.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.938+0000 7f1fee15b700 1 -- 172.21.15.7:0/1712338456 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1fd00a2d30 msgr2=0x7f1fd00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:57.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.938+0000 7f1fee15b700 1 --2- 172.21.15.7:0/1712338456 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1fd00a2d30 0x7f1fd00a51f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800b2e0 tx=0x7f1fd8005960 comp rx=0 tx=0).stop 2024-06-10T01:03:57.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.938+0000 7f1fee15b700 1 -- 172.21.15.7:0/1712338456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fe806b350 msgr2=0x7f1fe819b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:57.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.938+0000 7f1fee15b700 1 --2- 172.21.15.7:0/1712338456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fe806b350 0x7f1fe819b350 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc00e9a0 tx=0x7f1fdc00ecb0 comp rx=0 tx=0).stop 2024-06-10T01:03:57.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.938+0000 7f1fe7fff700 1 -- 172.21.15.7:0/1712338456 reap_dead start 2024-06-10T01:03:57.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.938+0000 7f1fee15b700 1 -- 172.21.15.7:0/1712338456 shutdown_connections 2024-06-10T01:03:57.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.938+0000 7f1fee15b700 1 -- 172.21.15.7:0/1712338456 >> 172.21.15.7:0/1712338456 conn(0x7f1fe8078300 msgr2=0x7f1fe8110340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:57.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.939+0000 7f1fee15b700 1 -- 172.21.15.7:0/1712338456 shutdown_connections 2024-06-10T01:03:57.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:57.939+0000 7f1fee15b700 1 -- 172.21.15.7:0/1712338456 wait complete. 2024-06-10T01:03:58.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.782+0000 7f630ec99700 1 -- 172.21.15.7:0/3900637349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6308101d50 msgr2=0x7f630810e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:58.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.782+0000 7f630ec99700 1 --2- 172.21.15.7:0/3900637349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6308101d50 0x7f630810e240 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f630000b0a0 tx=0x7f630000b3b0 comp rx=0 tx=0).stop 2024-06-10T01:03:58.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.783+0000 7f630ec99700 1 -- 172.21.15.7:0/3900637349 shutdown_connections 2024-06-10T01:03:58.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.783+0000 7f630ec99700 1 --2- 172.21.15.7:0/3900637349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6308101d50 0x7f630810e240 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:58.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.783+0000 7f630ec99700 1 --2- 172.21.15.7:0/3900637349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6308107400 0x7f63081077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:58.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.783+0000 7f630ec99700 1 --2- 172.21.15.7:0/3900637349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6308101390 0x7f6308101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:58.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.783+0000 7f630ec99700 1 -- 172.21.15.7:0/3900637349 >> 172.21.15.7:0/3900637349 conn(0x7f6308078340 msgr2=0x7f6308078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:58.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.783+0000 7f630ec99700 1 -- 172.21.15.7:0/3900637349 shutdown_connections 2024-06-10T01:03:58.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.783+0000 7f630ec99700 1 -- 172.21.15.7:0/3900637349 wait complete. 2024-06-10T01:03:58.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.784+0000 7f630ec99700 1 Processor -- start 2024-06-10T01:03:58.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.784+0000 7f630ec99700 1 -- start start 2024-06-10T01:03:58.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.784+0000 7f630ec99700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6308101390 0x7f6308198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:58.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.784+0000 7f630ec99700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6308101d50 0x7f63081991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:58.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.784+0000 7f630ec99700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6308107400 0x7f630819d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:58.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.784+0000 7f630ec99700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6308112790 con 0x7f6308101390 2024-06-10T01:03:58.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.784+0000 7f630ec99700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6308112610 con 0x7f6308101d50 2024-06-10T01:03:58.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.784+0000 7f630ec99700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6308112910 con 0x7f6308107400 2024-06-10T01:03:58.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.784+0000 7f630ca35700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6308101390 0x7f6308198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:58.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.784+0000 7f630ca35700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6308101390 0x7f6308198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54646/0 (socket says 172.21.15.7:54646) 2024-06-10T01:03:58.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.784+0000 7f630ca35700 1 -- 172.21.15.7:0/2890329660 learned_addr learned my addr 172.21.15.7:0/2890329660 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:03:58.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.784+0000 7f630d236700 1 --2- 172.21.15.7:0/2890329660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6308107400 0x7f630819d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:58.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.784+0000 7f6307fff700 1 --2- 172.21.15.7:0/2890329660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6308101d50 0x7f63081991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:58.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.785+0000 7f630ca35700 1 -- 172.21.15.7:0/2890329660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6308107400 msgr2=0x7f630819d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:58.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.785+0000 7f630ca35700 1 --2- 172.21.15.7:0/2890329660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6308107400 0x7f630819d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:58.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.785+0000 7f630ca35700 1 -- 172.21.15.7:0/2890329660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6308101d50 msgr2=0x7f63081991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:58.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.785+0000 7f630ca35700 1 --2- 172.21.15.7:0/2890329660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6308101d50 0x7f63081991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:58.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.785+0000 7f630ca35700 1 -- 172.21.15.7:0/2890329660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630819dc80 con 0x7f6308101390 2024-06-10T01:03:58.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.785+0000 7f6307fff700 1 --2- 172.21.15.7:0/2890329660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6308101d50 0x7f63081991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:03:58.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.785+0000 7f630ca35700 1 --2- 172.21.15.7:0/2890329660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6308101390 0x7f6308198c80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00d980 tx=0x7f62fc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:58.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.785+0000 7f6305ffb700 1 -- 172.21.15.7:0/2890329660 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc016070 con 0x7f6308101390 2024-06-10T01:03:58.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.785+0000 7f630ec99700 1 -- 172.21.15.7:0/2890329660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f630819df70 con 0x7f6308101390 2024-06-10T01:03:58.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.785+0000 7f6305ffb700 1 -- 172.21.15.7:0/2890329660 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62fc004d10 con 0x7f6308101390 2024-06-10T01:03:58.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.785+0000 7f6305ffb700 1 -- 172.21.15.7:0/2890329660 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc0055f0 con 0x7f6308101390 2024-06-10T01:03:58.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.785+0000 7f630ec99700 1 -- 172.21.15.7:0/2890329660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63081a5830 con 0x7f6308101390 2024-06-10T01:03:58.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.786+0000 7f62f37fe700 1 -- 172.21.15.7:0/2890329660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6308062ba0 con 0x7f6308101390 2024-06-10T01:03:58.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.787+0000 7f6305ffb700 1 -- 172.21.15.7:0/2890329660 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f62fc004830 con 0x7f6308101390 2024-06-10T01:03:58.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.788+0000 7f6305ffb700 1 --2- 172.21.15.7:0/2890329660 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f62f80a2cb0 0x7f62f80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:58.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.788+0000 7f6307fff700 1 --2- 172.21.15.7:0/2890329660 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f62f80a2cb0 0x7f62f80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:58.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.788+0000 7f6305ffb700 1 -- 172.21.15.7:0/2890329660 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f62fc0cf3b0 con 0x7f6308101390 2024-06-10T01:03:58.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.789+0000 7f6305ffb700 1 --2- 172.21.15.7:0/2890329660 >> [v2:172.21.15.78:6808/3767608357,v1:172.21.15.78:6809/3767608357] conn(0x7f62f80ade90 0x7f62f80b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:58.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.789+0000 7f6305ffb700 1 -- 172.21.15.7:0/2890329660 --> [v2:172.21.15.78:6808/3767608357,v1:172.21.15.78:6809/3767608357] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f62f80b0a00 con 0x7f62f80ade90 2024-06-10T01:03:58.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.789+0000 7f6305ffb700 1 -- 172.21.15.7:0/2890329660 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f62fc0d6050 con 0x7f6308101390 2024-06-10T01:03:58.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.789+0000 7f6307fff700 1 --2- 172.21.15.7:0/2890329660 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f62f80a2cb0 0x7f62f80a5170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f63080787d0 tx=0x7f62f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:58.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.789+0000 7f630d236700 1 --2- 172.21.15.7:0/2890329660 >> [v2:172.21.15.78:6808/3767608357,v1:172.21.15.78:6809/3767608357] conn(0x7f62f80ade90 0x7f62f80b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:58.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.790+0000 7f630d236700 1 --2- 172.21.15.7:0/2890329660 >> [v2:172.21.15.78:6808/3767608357,v1:172.21.15.78:6809/3767608357] conn(0x7f62f80ade90 0x7f62f80b0330 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:58.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.791+0000 7f6305ffb700 1 -- 172.21.15.7:0/2890329660 <== osd.4 v2:172.21.15.78:6808/3767608357 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f62f80b0a00 con 0x7f62f80ade90 2024-06-10T01:03:58.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.826+0000 7f62f37fe700 1 -- 172.21.15.7:0/2890329660 --> [v2:172.21.15.78:6808/3767608357,v1:172.21.15.78:6809/3767608357] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6308062fd0 con 0x7f62f80ade90 2024-06-10T01:03:58.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.827+0000 7f6305ffb700 1 -- 172.21.15.7:0/2890329660 <== osd.4 v2:172.21.15.78:6808/3767608357 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6308062fd0 con 0x7f62f80ade90 2024-06-10T01:03:58.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.827+0000 7f62f37fe700 1 -- 172.21.15.7:0/2890329660 >> [v2:172.21.15.78:6808/3767608357,v1:172.21.15.78:6809/3767608357] conn(0x7f62f80ade90 msgr2=0x7f62f80b0330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:58.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.827+0000 7f62f37fe700 1 --2- 172.21.15.7:0/2890329660 >> [v2:172.21.15.78:6808/3767608357,v1:172.21.15.78:6809/3767608357] conn(0x7f62f80ade90 0x7f62f80b0330 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:58.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.827+0000 7f62f37fe700 1 -- 172.21.15.7:0/2890329660 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f62f80a2cb0 msgr2=0x7f62f80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:58.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.827+0000 7f62f37fe700 1 --2- 172.21.15.7:0/2890329660 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f62f80a2cb0 0x7f62f80a5170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f63080787d0 tx=0x7f62f4009260 comp rx=0 tx=0).stop 2024-06-10T01:03:58.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.827+0000 7f62f37fe700 1 -- 172.21.15.7:0/2890329660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6308101390 msgr2=0x7f6308198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:58.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.827+0000 7f62f37fe700 1 --2- 172.21.15.7:0/2890329660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6308101390 0x7f6308198c80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00d980 tx=0x7f62fc00dc90 comp rx=0 tx=0).stop 2024-06-10T01:03:58.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.828+0000 7f630d236700 1 -- 172.21.15.7:0/2890329660 reap_dead start 2024-06-10T01:03:58.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.828+0000 7f62f37fe700 1 -- 172.21.15.7:0/2890329660 shutdown_connections 2024-06-10T01:03:58.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.828+0000 7f62f37fe700 1 -- 172.21.15.7:0/2890329660 >> 172.21.15.7:0/2890329660 conn(0x7f6308078340 msgr2=0x7f63080ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:58.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.828+0000 7f62f37fe700 1 -- 172.21.15.7:0/2890329660 shutdown_connections 2024-06-10T01:03:58.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:03:56.828+0000 7f62f37fe700 1 -- 172.21.15.7:0/2890329660 wait complete. 2024-06-10T01:03:58.660 INFO:teuthology.orchestra.run.smithi007.stdout:141733920844 2024-06-10T01:03:58.660 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-06-10T01:03:59.032 INFO:teuthology.orchestra.run.smithi007.stdout:309237645358 2024-06-10T01:03:59.033 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-06-10T01:03:59.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:03:59 smithi007 ceph-mon[32398]: pgmap v370: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:03:59.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:03:59 smithi078 ceph-mon[36683]: pgmap v370: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:03:59.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:03:59 smithi175 ceph-mon[36636]: pgmap v370: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:03:59.924 INFO:teuthology.orchestra.run.smithi007.stdout:55834574937 2024-06-10T01:03:59.924 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-06-10T01:04:01.048 INFO:teuthology.orchestra.run.smithi007.stdout:77309411414 2024-06-10T01:04:01.048 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-06-10T01:04:01.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:01 smithi007 ceph-mon[32398]: pgmap v371: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:04:01.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.304+0000 7fdb38f7e700 1 -- 172.21.15.7:0/3346367164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb341015d0 msgr2=0x7fdb341019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:01.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.304+0000 7fdb38f7e700 1 --2- 172.21.15.7:0/3346367164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb341015d0 0x7fdb341019b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdb240099c0 tx=0x7fdb24009cd0 comp rx=0 tx=0).stop 2024-06-10T01:04:01.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.304+0000 7fdb38f7e700 1 -- 172.21.15.7:0/3346367164 shutdown_connections 2024-06-10T01:04:01.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.304+0000 7fdb38f7e700 1 --2- 172.21.15.7:0/3346367164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb341066b0 0x7fdb3410e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:01.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.304+0000 7fdb38f7e700 1 --2- 172.21.15.7:0/3346367164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb341015d0 0x7fdb341019b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:01.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.304+0000 7fdb38f7e700 1 --2- 172.21.15.7:0/3346367164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb34101f80 0x7fdb34105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:01.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.305+0000 7fdb38f7e700 1 -- 172.21.15.7:0/3346367164 >> 172.21.15.7:0/3346367164 conn(0x7fdb34078400 msgr2=0x7fdb34078810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:01.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.305+0000 7fdb38f7e700 1 -- 172.21.15.7:0/3346367164 shutdown_connections 2024-06-10T01:04:01.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.305+0000 7fdb38f7e700 1 -- 172.21.15.7:0/3346367164 wait complete. 2024-06-10T01:04:01.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.305+0000 7fdb38f7e700 1 Processor -- start 2024-06-10T01:04:01.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.306+0000 7fdb38f7e700 1 -- start start 2024-06-10T01:04:01.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.306+0000 7fdb38f7e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb341015d0 0x7fdb34103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:01.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.306+0000 7fdb38f7e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb34101f80 0x7fdb341035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:01.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.306+0000 7fdb38f7e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb341066b0 0x7fdb34103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:01.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.306+0000 7fdb38f7e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdb34111690 con 0x7fdb341066b0 2024-06-10T01:04:01.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.306+0000 7fdb38f7e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb34111510 con 0x7fdb34101f80 2024-06-10T01:04:01.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.306+0000 7fdb38f7e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdb34111810 con 0x7fdb341015d0 2024-06-10T01:04:01.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.306+0000 7fdb32d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb341066b0 0x7fdb34103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:01.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.306+0000 7fdb3259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb341015d0 0x7fdb34103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:01.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.306+0000 7fdb32d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb341066b0 0x7fdb34103af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54704/0 (socket says 172.21.15.7:54704) 2024-06-10T01:04:01.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.306+0000 7fdb32d9d700 1 -- 172.21.15.7:0/3871955466 learned_addr learned my addr 172.21.15.7:0/3871955466 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:01.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.306+0000 7fdb31d9b700 1 --2- 172.21.15.7:0/3871955466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb34101f80 0x7fdb341035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:01.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.307+0000 7fdb32d9d700 1 -- 172.21.15.7:0/3871955466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb341015d0 msgr2=0x7fdb34103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:01.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.307+0000 7fdb32d9d700 1 --2- 172.21.15.7:0/3871955466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb341015d0 0x7fdb34103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:01.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.307+0000 7fdb32d9d700 1 -- 172.21.15.7:0/3871955466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb34101f80 msgr2=0x7fdb341035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:01.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.307+0000 7fdb32d9d700 1 --2- 172.21.15.7:0/3871955466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb34101f80 0x7fdb341035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:01.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.307+0000 7fdb32d9d700 1 -- 172.21.15.7:0/3871955466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb341a7140 con 0x7fdb341066b0 2024-06-10T01:04:01.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.307+0000 7fdb3259c700 1 --2- 172.21.15.7:0/3871955466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb341015d0 0x7fdb34103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:01.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.307+0000 7fdb32d9d700 1 --2- 172.21.15.7:0/3871955466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb341066b0 0x7fdb34103af0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c007ee0 tx=0x7fdb2c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:01.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.307+0000 7fdb237fe700 1 -- 172.21.15.7:0/3871955466 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c004bb0 con 0x7fdb341066b0 2024-06-10T01:04:01.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.307+0000 7fdb38f7e700 1 -- 172.21.15.7:0/3871955466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb341a7430 con 0x7fdb341066b0 2024-06-10T01:04:01.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.307+0000 7fdb237fe700 1 -- 172.21.15.7:0/3871955466 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb2c004d50 con 0x7fdb341066b0 2024-06-10T01:04:01.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.307+0000 7fdb237fe700 1 -- 172.21.15.7:0/3871955466 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c005960 con 0x7fdb341066b0 2024-06-10T01:04:01.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.307+0000 7fdb38f7e700 1 -- 172.21.15.7:0/3871955466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb341a7920 con 0x7fdb341066b0 2024-06-10T01:04:01.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.308+0000 7fdb38f7e700 1 -- 172.21.15.7:0/3871955466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdb34062ba0 con 0x7fdb341066b0 2024-06-10T01:04:01.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.310+0000 7fdb237fe700 1 -- 172.21.15.7:0/3871955466 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fdb2c01a020 con 0x7fdb341066b0 2024-06-10T01:04:01.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.311+0000 7fdb237fe700 1 --2- 172.21.15.7:0/3871955466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdb1c0a2cb0 0x7fdb1c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:01.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.311+0000 7fdb237fe700 1 -- 172.21.15.7:0/3871955466 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fdb2c0cec80 con 0x7fdb341066b0 2024-06-10T01:04:01.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.311+0000 7fdb3259c700 1 --2- 172.21.15.7:0/3871955466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdb1c0a2cb0 0x7fdb1c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:01.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.311+0000 7fdb237fe700 1 --2- 172.21.15.7:0/3871955466 >> [v2:172.21.15.175:6824/1959979550,v1:172.21.15.175:6825/1959979550] conn(0x7fdb1c0ade80 0x7fdb1c0b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:01.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.311+0000 7fdb237fe700 1 -- 172.21.15.7:0/3871955466 --> [v2:172.21.15.175:6824/1959979550,v1:172.21.15.175:6825/1959979550] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdb2c016ec0 con 0x7fdb1c0ade80 2024-06-10T01:04:01.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.311+0000 7fdb237fe700 1 -- 172.21.15.7:0/3871955466 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7fdb2c0d5020 con 0x7fdb341066b0 2024-06-10T01:04:01.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.312+0000 7fdb31d9b700 1 --2- 172.21.15.7:0/3871955466 >> [v2:172.21.15.175:6824/1959979550,v1:172.21.15.175:6825/1959979550] conn(0x7fdb1c0ade80 0x7fdb1c0b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:01.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.312+0000 7fdb3259c700 1 --2- 172.21.15.7:0/3871955466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdb1c0a2cb0 0x7fdb1c0a5170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdb24000c00 tx=0x7fdb24005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:01.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.312+0000 7fdb31d9b700 1 --2- 172.21.15.7:0/3871955466 >> [v2:172.21.15.175:6824/1959979550,v1:172.21.15.175:6825/1959979550] conn(0x7fdb1c0ade80 0x7fdb1c0b0320 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:01.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.313+0000 7fdb237fe700 1 -- 172.21.15.7:0/3871955466 <== osd.11 v2:172.21.15.175:6824/1959979550 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdb2c016ec0 con 0x7fdb1c0ade80 2024-06-10T01:04:01.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.332+0000 7fdb38f7e700 1 -- 172.21.15.7:0/3871955466 --> [v2:172.21.15.175:6824/1959979550,v1:172.21.15.175:6825/1959979550] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdb34062fd0 con 0x7fdb1c0ade80 2024-06-10T01:04:01.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.333+0000 7fdb237fe700 1 -- 172.21.15.7:0/3871955466 <== osd.11 v2:172.21.15.175:6824/1959979550 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdb34062fd0 con 0x7fdb1c0ade80 2024-06-10T01:04:01.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.333+0000 7fdb38f7e700 1 -- 172.21.15.7:0/3871955466 >> [v2:172.21.15.175:6824/1959979550,v1:172.21.15.175:6825/1959979550] conn(0x7fdb1c0ade80 msgr2=0x7fdb1c0b0320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:01.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.333+0000 7fdb38f7e700 1 --2- 172.21.15.7:0/3871955466 >> [v2:172.21.15.175:6824/1959979550,v1:172.21.15.175:6825/1959979550] conn(0x7fdb1c0ade80 0x7fdb1c0b0320 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:01.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.333+0000 7fdb38f7e700 1 -- 172.21.15.7:0/3871955466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdb1c0a2cb0 msgr2=0x7fdb1c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:01.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.333+0000 7fdb38f7e700 1 --2- 172.21.15.7:0/3871955466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdb1c0a2cb0 0x7fdb1c0a5170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdb24000c00 tx=0x7fdb24005fb0 comp rx=0 tx=0).stop 2024-06-10T01:04:01.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.333+0000 7fdb38f7e700 1 -- 172.21.15.7:0/3871955466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb341066b0 msgr2=0x7fdb34103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:01.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.333+0000 7fdb38f7e700 1 --2- 172.21.15.7:0/3871955466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb341066b0 0x7fdb34103af0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c007ee0 tx=0x7fdb2c00d3b0 comp rx=0 tx=0).stop 2024-06-10T01:04:01.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.334+0000 7fdb32d9d700 1 -- 172.21.15.7:0/3871955466 reap_dead start 2024-06-10T01:04:01.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.334+0000 7fdb38f7e700 1 -- 172.21.15.7:0/3871955466 shutdown_connections 2024-06-10T01:04:01.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.334+0000 7fdb38f7e700 1 -- 172.21.15.7:0/3871955466 >> 172.21.15.7:0/3871955466 conn(0x7fdb34078400 msgr2=0x7fdb341054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:01.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.334+0000 7fdb38f7e700 1 -- 172.21.15.7:0/3871955466 shutdown_connections 2024-06-10T01:04:01.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:01.334+0000 7fdb38f7e700 1 -- 172.21.15.7:0/3871955466 wait complete. 2024-06-10T01:04:01.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:01 smithi078 ceph-mon[36683]: pgmap v371: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:04:01.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:01 smithi175 ceph-mon[36636]: pgmap v371: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:04:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.089+0000 7f3368b42700 1 -- 172.21.15.7:0/3045789778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f336406ab30 msgr2=0x7f336410f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.089+0000 7f3368b42700 1 --2- 172.21.15.7:0/3045789778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f336406ab30 0x7f336410f320 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f335c00b0a0 tx=0x7f335c00b3b0 comp rx=0 tx=0).stop 2024-06-10T01:04:02.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.089+0000 7f3368b42700 1 -- 172.21.15.7:0/3045789778 shutdown_connections 2024-06-10T01:04:02.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.089+0000 7f3368b42700 1 --2- 172.21.15.7:0/3045789778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f336406ab30 0x7f336410f320 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:02.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.089+0000 7f3368b42700 1 --2- 172.21.15.7:0/3045789778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3364108500 0x7f33641088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:02.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.089+0000 7f3368b42700 1 --2- 172.21.15.7:0/3045789778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f336406a170 0x7f336406a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:02.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.089+0000 7f3368b42700 1 -- 172.21.15.7:0/3045789778 >> 172.21.15.7:0/3045789778 conn(0x7f3364078340 msgr2=0x7f3364078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:02.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.089+0000 7f3368b42700 1 -- 172.21.15.7:0/3045789778 shutdown_connections 2024-06-10T01:04:02.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.089+0000 7f3368b42700 1 -- 172.21.15.7:0/3045789778 wait complete. 2024-06-10T01:04:02.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.090+0000 7f3368b42700 1 Processor -- start 2024-06-10T01:04:02.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.090+0000 7f3368b42700 1 -- start start 2024-06-10T01:04:02.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.090+0000 7f3368b42700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f336406a170 0x7f3364101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:02.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.090+0000 7f3368b42700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f336406ab30 0x7f33641019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:02.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.090+0000 7f3368b42700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3364108500 0x7f3364101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:02.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.090+0000 7f3368b42700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3364112760 con 0x7f3364108500 2024-06-10T01:04:02.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.090+0000 7f3368b42700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f33641125e0 con 0x7f336406ab30 2024-06-10T01:04:02.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.090+0000 7f3368b42700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f33641128e0 con 0x7f336406a170 2024-06-10T01:04:02.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.090+0000 7f3362d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3364108500 0x7f3364101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:02.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.090+0000 7f3362d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3364108500 0x7f3364101ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48058/0 (socket says 172.21.15.7:48058) 2024-06-10T01:04:02.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.090+0000 7f3362d9d700 1 -- 172.21.15.7:0/1076203459 learned_addr learned my addr 172.21.15.7:0/1076203459 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:02.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.090+0000 7f336259c700 1 --2- 172.21.15.7:0/1076203459 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f336406a170 0x7f3364101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:02.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.091+0000 7f3361d9b700 1 --2- 172.21.15.7:0/1076203459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f336406ab30 0x7f33641019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:02.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.091+0000 7f3362d9d700 1 -- 172.21.15.7:0/1076203459 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f336406a170 msgr2=0x7f3364101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:02.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.091+0000 7f3362d9d700 1 --2- 172.21.15.7:0/1076203459 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f336406a170 0x7f3364101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:02.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.091+0000 7f3362d9d700 1 -- 172.21.15.7:0/1076203459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f336406ab30 msgr2=0x7f33641019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:02.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.091+0000 7f3362d9d700 1 --2- 172.21.15.7:0/1076203459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f336406ab30 0x7f33641019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:02.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.091+0000 7f3362d9d700 1 -- 172.21.15.7:0/1076203459 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33641a70a0 con 0x7f3364108500 2024-06-10T01:04:02.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.091+0000 7f3361d9b700 1 --2- 172.21.15.7:0/1076203459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f336406ab30 0x7f33641019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:04:02.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.091+0000 7f336259c700 1 --2- 172.21.15.7:0/1076203459 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f336406a170 0x7f3364101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:04:02.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.091+0000 7f3362d9d700 1 --2- 172.21.15.7:0/1076203459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3364108500 0x7f3364101ee0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f335c0062a0 tx=0x7f335c004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:02.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.091+0000 7f33537fe700 1 -- 172.21.15.7:0/1076203459 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c021070 con 0x7f3364108500 2024-06-10T01:04:02.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.091+0000 7f33537fe700 1 -- 172.21.15.7:0/1076203459 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f335c007c80 con 0x7f3364108500 2024-06-10T01:04:02.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.091+0000 7f3368b42700 1 -- 172.21.15.7:0/1076203459 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33641a7330 con 0x7f3364108500 2024-06-10T01:04:02.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.091+0000 7f33537fe700 1 -- 172.21.15.7:0/1076203459 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c01d450 con 0x7f3364108500 2024-06-10T01:04:02.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.091+0000 7f3368b42700 1 -- 172.21.15.7:0/1076203459 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33641a7830 con 0x7f3364108500 2024-06-10T01:04:02.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.092+0000 7f3368b42700 1 -- 172.21.15.7:0/1076203459 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3364062ba0 con 0x7f3364108500 2024-06-10T01:04:02.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.093+0000 7f33537fe700 1 -- 172.21.15.7:0/1076203459 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f335c004000 con 0x7f3364108500 2024-06-10T01:04:02.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.094+0000 7f33537fe700 1 --2- 172.21.15.7:0/1076203459 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f334c0a2d30 0x7f334c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:02.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.094+0000 7f33537fe700 1 -- 172.21.15.7:0/1076203459 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f335c00f020 con 0x7f3364108500 2024-06-10T01:04:02.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.094+0000 7f336259c700 1 --2- 172.21.15.7:0/1076203459 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f334c0a2d30 0x7f334c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.094+0000 7f33537fe700 1 --2- 172.21.15.7:0/1076203459 >> [v2:172.21.15.7:6818/838731756,v1:172.21.15.7:6819/838731756] conn(0x7f334c0adf10 0x7f334c0b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.094+0000 7f33537fe700 1 -- 172.21.15.7:0/1076203459 --> [v2:172.21.15.7:6818/838731756,v1:172.21.15.7:6819/838731756] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f335c008ea0 con 0x7f334c0adf10 2024-06-10T01:04:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.094+0000 7f33537fe700 1 -- 172.21.15.7:0/1076203459 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f335c0d1140 con 0x7f3364108500 2024-06-10T01:04:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.094+0000 7f3361d9b700 1 --2- 172.21.15.7:0/1076203459 >> [v2:172.21.15.7:6818/838731756,v1:172.21.15.7:6819/838731756] conn(0x7f334c0adf10 0x7f334c0b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.095+0000 7f336259c700 1 --2- 172.21.15.7:0/1076203459 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f334c0a2d30 0x7f334c0a51f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3354005fd0 tx=0x7f335400a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.095+0000 7f3361d9b700 1 --2- 172.21.15.7:0/1076203459 >> [v2:172.21.15.7:6818/838731756,v1:172.21.15.7:6819/838731756] conn(0x7f334c0adf10 0x7f334c0b03b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.096+0000 7f33537fe700 1 -- 172.21.15.7:0/1076203459 <== osd.6 v2:172.21.15.7:6818/838731756 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f335c008ea0 con 0x7f334c0adf10 2024-06-10T01:04:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.112+0000 7f3368b42700 1 -- 172.21.15.7:0/1076203459 --> [v2:172.21.15.7:6818/838731756,v1:172.21.15.7:6819/838731756] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3364063020 con 0x7f334c0adf10 2024-06-10T01:04:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.112+0000 7f33537fe700 1 -- 172.21.15.7:0/1076203459 <== osd.6 v2:172.21.15.7:6818/838731756 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3364063020 con 0x7f334c0adf10 2024-06-10T01:04:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.113+0000 7f3368b42700 1 -- 172.21.15.7:0/1076203459 >> [v2:172.21.15.7:6818/838731756,v1:172.21.15.7:6819/838731756] conn(0x7f334c0adf10 msgr2=0x7f334c0b03b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.113+0000 7f3368b42700 1 --2- 172.21.15.7:0/1076203459 >> [v2:172.21.15.7:6818/838731756,v1:172.21.15.7:6819/838731756] conn(0x7f334c0adf10 0x7f334c0b03b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.113+0000 7f3368b42700 1 -- 172.21.15.7:0/1076203459 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f334c0a2d30 msgr2=0x7f334c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.113+0000 7f3368b42700 1 --2- 172.21.15.7:0/1076203459 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f334c0a2d30 0x7f334c0a51f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3354005fd0 tx=0x7f335400a5e0 comp rx=0 tx=0).stop 2024-06-10T01:04:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.113+0000 7f3368b42700 1 -- 172.21.15.7:0/1076203459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3364108500 msgr2=0x7f3364101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.113+0000 7f3368b42700 1 --2- 172.21.15.7:0/1076203459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3364108500 0x7f3364101ee0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f335c0062a0 tx=0x7f335c004e20 comp rx=0 tx=0).stop 2024-06-10T01:04:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.113+0000 7f3362d9d700 1 -- 172.21.15.7:0/1076203459 reap_dead start 2024-06-10T01:04:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.113+0000 7f3368b42700 1 -- 172.21.15.7:0/1076203459 shutdown_connections 2024-06-10T01:04:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.113+0000 7f3368b42700 1 -- 172.21.15.7:0/1076203459 >> 172.21.15.7:0/1076203459 conn(0x7f3364078340 msgr2=0x7f33640ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.113+0000 7f3368b42700 1 -- 172.21.15.7:0/1076203459 shutdown_connections 2024-06-10T01:04:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:02.113+0000 7f3368b42700 1 -- 172.21.15.7:0/1076203459 wait complete. 2024-06-10T01:04:02.416 INFO:teuthology.orchestra.run.smithi007.stdout:343597383723 2024-06-10T01:04:02.416 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-06-10T01:04:02.981 INFO:teuthology.orchestra.run.smithi007.stdout:197568495683 2024-06-10T01:04:02.982 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-06-10T01:04:03.505 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:03 smithi007 ceph-mon[32398]: pgmap v372: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:04:03.531 INFO:teuthology.orchestra.run.smithi007.stdout:253403070520 2024-06-10T01:04:03.531 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-06-10T01:04:03.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:03 smithi078 ceph-mon[36683]: pgmap v372: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:04:03.651 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:03.665 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:03.665 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:03.672 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:03.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:03 smithi175 ceph-mon[36636]: pgmap v372: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:04:05.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:05 smithi007 ceph-mon[32398]: pgmap v373: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-10T01:04:05.529 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:05.531 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:05.532 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:05.536 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:05.536 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:05.539 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:05.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:05 smithi078 ceph-mon[36683]: pgmap v373: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-10T01:04:05.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:05 smithi175 ceph-mon[36636]: pgmap v373: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-10T01:04:07.429 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:07 smithi007 ceph-mon[32398]: pgmap v374: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-06-10T01:04:07.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:07 smithi078 ceph-mon[36683]: pgmap v374: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-06-10T01:04:07.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:07 smithi175 ceph-mon[36636]: pgmap v374: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-06-10T01:04:09.463 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:09 smithi007 ceph-mon[32398]: pgmap v375: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-06-10T01:04:09.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:09 smithi078 ceph-mon[36683]: pgmap v375: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-06-10T01:04:09.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:09 smithi175 ceph-mon[36636]: pgmap v375: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-06-10T01:04:09.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.903+0000 7f4ba1fa5700 1 -- 172.21.15.7:0/1300604811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b9c1030f0 msgr2=0x7f4b9c107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:09.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.903+0000 7f4ba1fa5700 1 --2- 172.21.15.7:0/1300604811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b9c1030f0 0x7f4b9c107110 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4b900099c0 tx=0x7f4b90009cd0 comp rx=0 tx=0).stop 2024-06-10T01:04:09.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.903+0000 7f4ba1fa5700 1 -- 172.21.15.7:0/1300604811 shutdown_connections 2024-06-10T01:04:09.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.903+0000 7f4ba1fa5700 1 --2- 172.21.15.7:0/1300604811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b9c107840 0x7f4b9c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:09.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.903+0000 7f4ba1fa5700 1 --2- 172.21.15.7:0/1300604811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b9c102740 0x7f4b9c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:09.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.903+0000 7f4ba1fa5700 1 --2- 172.21.15.7:0/1300604811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b9c1030f0 0x7f4b9c107110 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:09.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.903+0000 7f4ba1fa5700 1 -- 172.21.15.7:0/1300604811 >> 172.21.15.7:0/1300604811 conn(0x7f4b9c0795f0 msgr2=0x7f4b9c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:09.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.903+0000 7f4ba1fa5700 1 -- 172.21.15.7:0/1300604811 shutdown_connections 2024-06-10T01:04:09.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.903+0000 7f4ba1fa5700 1 -- 172.21.15.7:0/1300604811 wait complete. 2024-06-10T01:04:09.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.904+0000 7f4ba1fa5700 1 Processor -- start 2024-06-10T01:04:09.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.904+0000 7f4ba1fa5700 1 -- start start 2024-06-10T01:04:09.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.905+0000 7f4ba1fa5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b9c102740 0x7f4b9c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:09.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.905+0000 7f4ba1fa5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b9c1030f0 0x7f4b9c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:09.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.905+0000 7f4ba1fa5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b9c107840 0x7f4b9c075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:09.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.905+0000 7f4ba1fa5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c112770 con 0x7f4b9c102740 2024-06-10T01:04:09.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.905+0000 7f4ba1fa5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c1125f0 con 0x7f4b9c107840 2024-06-10T01:04:09.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.905+0000 7f4ba1fa5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c1128f0 con 0x7f4b9c1030f0 2024-06-10T01:04:09.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.905+0000 7f4b9b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b9c102740 0x7f4b9c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:09.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.905+0000 7f4b9b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b9c102740 0x7f4b9c074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48078/0 (socket says 172.21.15.7:48078) 2024-06-10T01:04:09.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.905+0000 7f4b9b7fe700 1 -- 172.21.15.7:0/1568593706 learned_addr learned my addr 172.21.15.7:0/1568593706 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:09.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.905+0000 7f4b9affd700 1 --2- 172.21.15.7:0/1568593706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b9c1030f0 0x7f4b9c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:09.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.905+0000 7f4b9bfff700 1 --2- 172.21.15.7:0/1568593706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b9c107840 0x7f4b9c075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:09.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.905+0000 7f4b9b7fe700 1 -- 172.21.15.7:0/1568593706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b9c1030f0 msgr2=0x7f4b9c074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:09.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.905+0000 7f4b9b7fe700 1 --2- 172.21.15.7:0/1568593706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b9c1030f0 0x7f4b9c074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:09.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.905+0000 7f4b9b7fe700 1 -- 172.21.15.7:0/1568593706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b9c107840 msgr2=0x7f4b9c075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:09.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.905+0000 7f4b9b7fe700 1 --2- 172.21.15.7:0/1568593706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b9c107840 0x7f4b9c075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:09.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.905+0000 7f4b9b7fe700 1 -- 172.21.15.7:0/1568593706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b9c1a7080 con 0x7f4b9c102740 2024-06-10T01:04:09.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.905+0000 7f4b9bfff700 1 --2- 172.21.15.7:0/1568593706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b9c107840 0x7f4b9c075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:04:09.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.906+0000 7f4b9b7fe700 1 --2- 172.21.15.7:0/1568593706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b9c102740 0x7f4b9c074790 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c00d980 tx=0x7f4b8c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:09.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.906+0000 7f4b98ff9700 1 -- 172.21.15.7:0/1568593706 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c017070 con 0x7f4b9c102740 2024-06-10T01:04:09.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.906+0000 7f4b98ff9700 1 -- 172.21.15.7:0/1568593706 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b8c005020 con 0x7f4b9c102740 2024-06-10T01:04:09.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.906+0000 7f4ba1fa5700 1 -- 172.21.15.7:0/1568593706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b9c1a7370 con 0x7f4b9c102740 2024-06-10T01:04:09.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.906+0000 7f4b98ff9700 1 -- 172.21.15.7:0/1568593706 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c009bc0 con 0x7f4b9c102740 2024-06-10T01:04:09.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.906+0000 7f4ba1fa5700 1 -- 172.21.15.7:0/1568593706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b9c1a7830 con 0x7f4b9c102740 2024-06-10T01:04:09.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.908+0000 7f4ba1fa5700 1 -- 172.21.15.7:0/1568593706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b9c10aa10 con 0x7f4b9c102740 2024-06-10T01:04:09.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.909+0000 7f4b98ff9700 1 -- 172.21.15.7:0/1568593706 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f4b8c0051d0 con 0x7f4b9c102740 2024-06-10T01:04:09.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.910+0000 7f4b98ff9700 1 --2- 172.21.15.7:0/1568593706 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4b840a2cb0 0x7f4b840a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:09.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.910+0000 7f4b9affd700 1 --2- 172.21.15.7:0/1568593706 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4b840a2cb0 0x7f4b840a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:09.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.910+0000 7f4b98ff9700 1 -- 172.21.15.7:0/1568593706 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f4b8c0cf480 con 0x7f4b9c102740 2024-06-10T01:04:09.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.910+0000 7f4b9affd700 1 --2- 172.21.15.7:0/1568593706 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4b840a2cb0 0x7f4b840a5170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c0761d0 tx=0x7f4b90005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:09.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:09.914+0000 7f4b98ff9700 1 -- 172.21.15.7:0/1568593706 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4b8c020020 con 0x7f4b9c102740 2024-06-10T01:04:10.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:10.123+0000 7f4ba1fa5700 1 -- 172.21.15.7:0/1568593706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f4b9c10a3f0 con 0x7f4b9c102740 2024-06-10T01:04:10.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:10.123+0000 7f4b98ff9700 1 -- 172.21.15.7:0/1568593706 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4b8c09bf00 con 0x7f4b9c102740 2024-06-10T01:04:10.121 INFO:teuthology.orchestra.run.smithi007.stdout:55834574941 2024-06-10T01:04:10.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:10.126+0000 7f4ba1fa5700 1 -- 172.21.15.7:0/1568593706 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4b840a2cb0 msgr2=0x7f4b840a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:10.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:10.126+0000 7f4ba1fa5700 1 --2- 172.21.15.7:0/1568593706 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4b840a2cb0 0x7f4b840a5170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c0761d0 tx=0x7f4b90005fb0 comp rx=0 tx=0).stop 2024-06-10T01:04:10.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:10.126+0000 7f4ba1fa5700 1 -- 172.21.15.7:0/1568593706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b9c102740 msgr2=0x7f4b9c074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:10.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:10.126+0000 7f4ba1fa5700 1 --2- 172.21.15.7:0/1568593706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b9c102740 0x7f4b9c074790 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c00d980 tx=0x7f4b8c00dc90 comp rx=0 tx=0).stop 2024-06-10T01:04:10.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:10.126+0000 7f4ba1fa5700 1 -- 172.21.15.7:0/1568593706 shutdown_connections 2024-06-10T01:04:10.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:10.126+0000 7f4ba1fa5700 1 --2- 172.21.15.7:0/1568593706 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4b840a2cb0 0x7f4b840a5170 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:10.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:10.126+0000 7f4ba1fa5700 1 --2- 172.21.15.7:0/1568593706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b9c107840 0x7f4b9c075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:10.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:10.126+0000 7f4ba1fa5700 1 --2- 172.21.15.7:0/1568593706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b9c102740 0x7f4b9c074790 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:10.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:10.126+0000 7f4ba1fa5700 1 --2- 172.21.15.7:0/1568593706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b9c1030f0 0x7f4b9c074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:10.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:10.126+0000 7f4ba1fa5700 1 -- 172.21.15.7:0/1568593706 >> 172.21.15.7:0/1568593706 conn(0x7f4b9c0795f0 msgr2=0x7f4b9c101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:10.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:10.126+0000 7f4ba1fa5700 1 -- 172.21.15.7:0/1568593706 shutdown_connections 2024-06-10T01:04:10.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:10.126+0000 7f4ba1fa5700 1 -- 172.21.15.7:0/1568593706 wait complete. 2024-06-10T01:04:10.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:10 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1568593706' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-10T01:04:10.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:10 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1568593706' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-10T01:04:10.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:10 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1568593706' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-10T01:04:11.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:11 smithi007 ceph-mon[32398]: pgmap v376: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:04:11.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:11 smithi078 ceph-mon[36683]: pgmap v376: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:04:11.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:11 smithi175 ceph-mon[36636]: pgmap v376: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:04:13.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:13 smithi007 ceph-mon[32398]: pgmap v377: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:04:13.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:13 smithi078 ceph-mon[36683]: pgmap v377: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:04:13.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:13 smithi175 ceph-mon[36636]: pgmap v377: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:04:14.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.150+0000 7fe5e805f700 1 -- 172.21.15.7:0/3463588832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5e0102540 msgr2=0x7fe5e01029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:14.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.150+0000 7fe5e805f700 1 --2- 172.21.15.7:0/3463588832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5e0102540 0x7fe5e01029a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4009990 tx=0x7fe5d4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.150+0000 7fe5e805f700 1 -- 172.21.15.7:0/3463588832 shutdown_connections 2024-06-10T01:04:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.150+0000 7fe5e805f700 1 --2- 172.21.15.7:0/3463588832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e0102ee0 0x7fe5e010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.150+0000 7fe5e805f700 1 --2- 172.21.15.7:0/3463588832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5e0107450 0x7fe5e0107830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.150+0000 7fe5e805f700 1 --2- 172.21.15.7:0/3463588832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5e0102540 0x7fe5e01029a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.150+0000 7fe5e805f700 1 -- 172.21.15.7:0/3463588832 >> 172.21.15.7:0/3463588832 conn(0x7fe5e00795f0 msgr2=0x7fe5e0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.150+0000 7fe5e805f700 1 -- 172.21.15.7:0/3463588832 shutdown_connections 2024-06-10T01:04:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.150+0000 7fe5e805f700 1 -- 172.21.15.7:0/3463588832 wait complete. 2024-06-10T01:04:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.151+0000 7fe5e805f700 1 Processor -- start 2024-06-10T01:04:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.151+0000 7fe5e805f700 1 -- start start 2024-06-10T01:04:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.151+0000 7fe5e805f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5e0102540 0x7fe5e019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.151+0000 7fe5e805f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e0102ee0 0x7fe5e019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.151+0000 7fe5e805f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5e0107450 0x7fe5e01989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.151+0000 7fe5e805f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e01127d0 con 0x7fe5e0107450 2024-06-10T01:04:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.151+0000 7fe5e805f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e0112650 con 0x7fe5e0102ee0 2024-06-10T01:04:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.151+0000 7fe5e805f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e0112950 con 0x7fe5e0102540 2024-06-10T01:04:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.151+0000 7fe5e65fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5e0107450 0x7fe5e01989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.151+0000 7fe5e65fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5e0107450 0x7fe5e01989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48810/0 (socket says 172.21.15.7:48810) 2024-06-10T01:04:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.151+0000 7fe5e65fc700 1 -- 172.21.15.7:0/140105184 learned_addr learned my addr 172.21.15.7:0/140105184 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.152+0000 7fe5e55fa700 1 --2- 172.21.15.7:0/140105184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e0102ee0 0x7fe5e019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.152+0000 7fe5e5dfb700 1 --2- 172.21.15.7:0/140105184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5e0102540 0x7fe5e019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.152+0000 7fe5e65fc700 1 -- 172.21.15.7:0/140105184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5e0102540 msgr2=0x7fe5e019e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.152+0000 7fe5e65fc700 1 --2- 172.21.15.7:0/140105184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5e0102540 0x7fe5e019e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.152+0000 7fe5e65fc700 1 -- 172.21.15.7:0/140105184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e0102ee0 msgr2=0x7fe5e019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.152+0000 7fe5e65fc700 1 --2- 172.21.15.7:0/140105184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e0102ee0 0x7fe5e019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.152+0000 7fe5e65fc700 1 -- 172.21.15.7:0/140105184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e0199130 con 0x7fe5e0107450 2024-06-10T01:04:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.152+0000 7fe5e65fc700 1 --2- 172.21.15.7:0/140105184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5e0107450 0x7fe5e01989d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8007f10 tx=0x7fe5d800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:14.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.152+0000 7fe5d2ffd700 1 -- 172.21.15.7:0/140105184 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8009e20 con 0x7fe5e0107450 2024-06-10T01:04:14.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.152+0000 7fe5e805f700 1 -- 172.21.15.7:0/140105184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e0199420 con 0x7fe5e0107450 2024-06-10T01:04:14.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.152+0000 7fe5d2ffd700 1 -- 172.21.15.7:0/140105184 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d800dbd0 con 0x7fe5e0107450 2024-06-10T01:04:14.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.152+0000 7fe5d2ffd700 1 -- 172.21.15.7:0/140105184 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8005a60 con 0x7fe5e0107450 2024-06-10T01:04:14.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.152+0000 7fe5e805f700 1 -- 172.21.15.7:0/140105184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e01a56e0 con 0x7fe5e0107450 2024-06-10T01:04:14.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.153+0000 7fe5e805f700 1 -- 172.21.15.7:0/140105184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5e0069180 con 0x7fe5e0107450 2024-06-10T01:04:14.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.154+0000 7fe5d2ffd700 1 -- 172.21.15.7:0/140105184 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fe5d801a020 con 0x7fe5e0107450 2024-06-10T01:04:14.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.155+0000 7fe5d2ffd700 1 --2- 172.21.15.7:0/140105184 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe5cc0a2cb0 0x7fe5cc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:14.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.155+0000 7fe5e5dfb700 1 --2- 172.21.15.7:0/140105184 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe5cc0a2cb0 0x7fe5cc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:14.257 INFO:teuthology.orchestra.run.smithi007.stdout:141733920847 2024-06-10T01:04:14.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.155+0000 7fe5d2ffd700 1 -- 172.21.15.7:0/140105184 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fe5d80ced10 con 0x7fe5e0107450 2024-06-10T01:04:14.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.155+0000 7fe5e5dfb700 1 --2- 172.21.15.7:0/140105184 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe5cc0a2cb0 0x7fe5cc0a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0199b10 tx=0x7fe5dc00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:14.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.160+0000 7fe5d2ffd700 1 -- 172.21.15.7:0/140105184 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe5d80d4020 con 0x7fe5e0107450 2024-06-10T01:04:14.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.388+0000 7fe5e805f700 1 -- 172.21.15.7:0/140105184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fe5e002dca0 con 0x7fe5e0107450 2024-06-10T01:04:14.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.388+0000 7fe5d2ffd700 1 -- 172.21.15.7:0/140105184 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe5d8010070 con 0x7fe5e0107450 2024-06-10T01:04:14.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.390+0000 7fe5e805f700 1 -- 172.21.15.7:0/140105184 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe5cc0a2cb0 msgr2=0x7fe5cc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:14.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.390+0000 7fe5e805f700 1 --2- 172.21.15.7:0/140105184 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe5cc0a2cb0 0x7fe5cc0a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0199b10 tx=0x7fe5dc00a560 comp rx=0 tx=0).stop 2024-06-10T01:04:14.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.390+0000 7fe5e805f700 1 -- 172.21.15.7:0/140105184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5e0107450 msgr2=0x7fe5e01989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.390+0000 7fe5e805f700 1 --2- 172.21.15.7:0/140105184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5e0107450 0x7fe5e01989d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8007f10 tx=0x7fe5d800d3b0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.390+0000 7fe5e805f700 1 -- 172.21.15.7:0/140105184 shutdown_connections 2024-06-10T01:04:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.390+0000 7fe5e805f700 1 --2- 172.21.15.7:0/140105184 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe5cc0a2cb0 0x7fe5cc0a5170 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.390+0000 7fe5e805f700 1 --2- 172.21.15.7:0/140105184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5e0107450 0x7fe5e01989d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.390+0000 7fe5e805f700 1 --2- 172.21.15.7:0/140105184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5e0102540 0x7fe5e019e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.390+0000 7fe5e805f700 1 --2- 172.21.15.7:0/140105184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e0102ee0 0x7fe5e019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.391+0000 7fe5e805f700 1 -- 172.21.15.7:0/140105184 >> 172.21.15.7:0/140105184 conn(0x7fe5e00795f0 msgr2=0x7fe5e01004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.391+0000 7fe5e805f700 1 -- 172.21.15.7:0/140105184 shutdown_connections 2024-06-10T01:04:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:13.391+0000 7fe5e805f700 1 -- 172.21.15.7:0/140105184 wait complete. 2024-06-10T01:04:14.498 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:14 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/140105184' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-10T01:04:14.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:14 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/140105184' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-10T01:04:14.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:14 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/140105184' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-10T01:04:15.417 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:15 smithi007 ceph-mon[32398]: pgmap v378: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:04:15.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:15 smithi078 ceph-mon[36683]: pgmap v378: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:04:15.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:15 smithi175 ceph-mon[36636]: pgmap v378: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:04:15.739 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574937 got 55834574941 for osd.1 2024-06-10T01:04:15.739 DEBUG:teuthology.parallel:result is None 2024-06-10T01:04:17.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:17 smithi007 ceph-mon[32398]: pgmap v379: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-10T01:04:17.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.445+0000 7f983dff3700 1 -- 172.21.15.7:0/3725843134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f983806ab30 msgr2=0x7f983810f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:17.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.445+0000 7f983dff3700 1 --2- 172.21.15.7:0/3725843134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f983806ab30 0x7f983810f320 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f983000b0a0 tx=0x7f983000b3b0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.445+0000 7f983dff3700 1 -- 172.21.15.7:0/3725843134 shutdown_connections 2024-06-10T01:04:17.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.445+0000 7f983dff3700 1 --2- 172.21.15.7:0/3725843134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f983806ab30 0x7f983810f320 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.445+0000 7f983dff3700 1 --2- 172.21.15.7:0/3725843134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9838108500 0x7f98381088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.445+0000 7f983dff3700 1 --2- 172.21.15.7:0/3725843134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f983806a170 0x7f983806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.445+0000 7f983dff3700 1 -- 172.21.15.7:0/3725843134 >> 172.21.15.7:0/3725843134 conn(0x7f9838078340 msgr2=0x7f9838078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:17.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.445+0000 7f983dff3700 1 -- 172.21.15.7:0/3725843134 shutdown_connections 2024-06-10T01:04:17.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.445+0000 7f983dff3700 1 -- 172.21.15.7:0/3725843134 wait complete. 2024-06-10T01:04:17.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.446+0000 7f983dff3700 1 Processor -- start 2024-06-10T01:04:17.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.446+0000 7f983dff3700 1 -- start start 2024-06-10T01:04:17.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.446+0000 7f983dff3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f983806a170 0x7f9838101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:17.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.446+0000 7f983dff3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f983806ab30 0x7f98381019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:17.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.446+0000 7f983dff3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9838108500 0x7f9838101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:17.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.446+0000 7f983dff3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9838112780 con 0x7f9838108500 2024-06-10T01:04:17.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.446+0000 7f983dff3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9838112600 con 0x7f983806ab30 2024-06-10T01:04:17.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.446+0000 7f983dff3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9838112900 con 0x7f983806a170 2024-06-10T01:04:17.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.446+0000 7f9837fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9838108500 0x7f9838101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:17.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.446+0000 7f9837fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9838108500 0x7f9838101ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48824/0 (socket says 172.21.15.7:48824) 2024-06-10T01:04:17.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.446+0000 7f9837fff700 1 -- 172.21.15.7:0/2446138014 learned_addr learned my addr 172.21.15.7:0/2446138014 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:17.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.446+0000 7f982effd700 1 --2- 172.21.15.7:0/2446138014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f983806ab30 0x7f98381019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:17.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.446+0000 7f98377fe700 1 --2- 172.21.15.7:0/2446138014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f983806a170 0x7f9838101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:17.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.447+0000 7f9837fff700 1 -- 172.21.15.7:0/2446138014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f983806a170 msgr2=0x7f9838101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:17.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.447+0000 7f9837fff700 1 --2- 172.21.15.7:0/2446138014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f983806a170 0x7f9838101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.447+0000 7f9837fff700 1 -- 172.21.15.7:0/2446138014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f983806ab30 msgr2=0x7f98381019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:17.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.447+0000 7f9837fff700 1 --2- 172.21.15.7:0/2446138014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f983806ab30 0x7f98381019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.447+0000 7f9837fff700 1 -- 172.21.15.7:0/2446138014 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98381a7100 con 0x7f9838108500 2024-06-10T01:04:17.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.447+0000 7f98377fe700 1 --2- 172.21.15.7:0/2446138014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f983806a170 0x7f9838101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:04:17.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.447+0000 7f9837fff700 1 --2- 172.21.15.7:0/2446138014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9838108500 0x7f9838101ee0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f98300062a0 tx=0x7f9830004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:17.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.447+0000 7f98357fa700 1 -- 172.21.15.7:0/2446138014 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830021070 con 0x7f9838108500 2024-06-10T01:04:17.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.447+0000 7f983dff3700 1 -- 172.21.15.7:0/2446138014 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98381a7390 con 0x7f9838108500 2024-06-10T01:04:17.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.447+0000 7f98357fa700 1 -- 172.21.15.7:0/2446138014 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9830007c00 con 0x7f9838108500 2024-06-10T01:04:17.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.447+0000 7f98357fa700 1 -- 172.21.15.7:0/2446138014 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983001d450 con 0x7f9838108500 2024-06-10T01:04:17.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.447+0000 7f983dff3700 1 -- 172.21.15.7:0/2446138014 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98381a7840 con 0x7f9838108500 2024-06-10T01:04:17.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.448+0000 7f983dff3700 1 -- 172.21.15.7:0/2446138014 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f983810a9e0 con 0x7f9838108500 2024-06-10T01:04:17.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.449+0000 7f98357fa700 1 -- 172.21.15.7:0/2446138014 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f9830004000 con 0x7f9838108500 2024-06-10T01:04:17.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.450+0000 7f98357fa700 1 --2- 172.21.15.7:0/2446138014 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f98240a2cb0 0x7f98240a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:17.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.450+0000 7f98377fe700 1 --2- 172.21.15.7:0/2446138014 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f98240a2cb0 0x7f98240a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:17.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.450+0000 7f98357fa700 1 -- 172.21.15.7:0/2446138014 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f983000f020 con 0x7f9838108500 2024-06-10T01:04:17.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.450+0000 7f98377fe700 1 --2- 172.21.15.7:0/2446138014 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f98240a2cb0 0x7f98240a5170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9828005e10 tx=0x7f9828005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:17.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.453+0000 7f98357fa700 1 -- 172.21.15.7:0/2446138014 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f983002a020 con 0x7f9838108500 2024-06-10T01:04:17.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:17 smithi078 ceph-mon[36683]: pgmap v379: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-10T01:04:17.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.664+0000 7f983dff3700 1 -- 172.21.15.7:0/2446138014 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f9838102990 con 0x7f9838108500 2024-06-10T01:04:17.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.664+0000 7f98357fa700 1 -- 172.21.15.7:0/2446138014 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f983009d590 con 0x7f9838108500 2024-06-10T01:04:17.661 INFO:teuthology.orchestra.run.smithi007.stdout:107374182486 2024-06-10T01:04:17.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.666+0000 7f983dff3700 1 -- 172.21.15.7:0/2446138014 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f98240a2cb0 msgr2=0x7f98240a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:17.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.666+0000 7f983dff3700 1 --2- 172.21.15.7:0/2446138014 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f98240a2cb0 0x7f98240a5170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9828005e10 tx=0x7f9828005d80 comp rx=0 tx=0).stop 2024-06-10T01:04:17.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.666+0000 7f983dff3700 1 -- 172.21.15.7:0/2446138014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9838108500 msgr2=0x7f9838101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:17.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.666+0000 7f983dff3700 1 --2- 172.21.15.7:0/2446138014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9838108500 0x7f9838101ee0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f98300062a0 tx=0x7f9830004d70 comp rx=0 tx=0).stop 2024-06-10T01:04:17.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.666+0000 7f983dff3700 1 -- 172.21.15.7:0/2446138014 shutdown_connections 2024-06-10T01:04:17.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.666+0000 7f983dff3700 1 --2- 172.21.15.7:0/2446138014 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f98240a2cb0 0x7f98240a5170 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.666+0000 7f983dff3700 1 --2- 172.21.15.7:0/2446138014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9838108500 0x7f9838101ee0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.666+0000 7f983dff3700 1 --2- 172.21.15.7:0/2446138014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f983806a170 0x7f9838101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.666+0000 7f983dff3700 1 --2- 172.21.15.7:0/2446138014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f983806ab30 0x7f98381019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.666+0000 7f983dff3700 1 -- 172.21.15.7:0/2446138014 >> 172.21.15.7:0/2446138014 conn(0x7f9838078340 msgr2=0x7f98380fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:17.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.666+0000 7f983dff3700 1 -- 172.21.15.7:0/2446138014 shutdown_connections 2024-06-10T01:04:17.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:17.666+0000 7f983dff3700 1 -- 172.21.15.7:0/2446138014 wait complete. 2024-06-10T01:04:17.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:17 smithi175 ceph-mon[36636]: pgmap v379: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-10T01:04:18.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:18 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2446138014' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-10T01:04:18.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:18 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2446138014' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-10T01:04:18.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:18 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2446138014' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-10T01:04:18.825 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182481 got 107374182486 for osd.3 2024-06-10T01:04:18.825 DEBUG:teuthology.parallel:result is None 2024-06-10T01:04:19.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.018+0000 7f48b25fd700 1 -- 172.21.15.7:0/3595229750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48ac108580 msgr2=0x7f48ac108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:19.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.018+0000 7f48b25fd700 1 --2- 172.21.15.7:0/3595229750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48ac108580 0x7f48ac108960 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f489c0099c0 tx=0x7f489c009cd0 comp rx=0 tx=0).stop 2024-06-10T01:04:19.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.018+0000 7f48b25fd700 1 -- 172.21.15.7:0/3595229750 shutdown_connections 2024-06-10T01:04:19.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.018+0000 7f48b25fd700 1 --2- 172.21.15.7:0/3595229750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48ac06b620 0x7f48ac10f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:19.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.018+0000 7f48b25fd700 1 --2- 172.21.15.7:0/3595229750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48ac108580 0x7f48ac108960 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:19.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.018+0000 7f48b25fd700 1 --2- 172.21.15.7:0/3595229750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48ac06ac60 0x7f48ac06b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:19.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.018+0000 7f48b25fd700 1 -- 172.21.15.7:0/3595229750 >> 172.21.15.7:0/3595229750 conn(0x7f48ac078430 msgr2=0x7f48ac078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:19.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.018+0000 7f48b25fd700 1 -- 172.21.15.7:0/3595229750 shutdown_connections 2024-06-10T01:04:19.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.018+0000 7f48b25fd700 1 -- 172.21.15.7:0/3595229750 wait complete. 2024-06-10T01:04:19.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.019+0000 7f48b25fd700 1 Processor -- start 2024-06-10T01:04:19.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.019+0000 7f48b25fd700 1 -- start start 2024-06-10T01:04:19.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.019+0000 7f48b25fd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48ac06ac60 0x7f48ac19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:19.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.019+0000 7f48b25fd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48ac06b620 0x7f48ac19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:19.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.019+0000 7f48b25fd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48ac108580 0x7f48ac198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:19.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.019+0000 7f48b25fd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac112810 con 0x7f48ac06b620 2024-06-10T01:04:19.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.019+0000 7f48b25fd700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac112690 con 0x7f48ac108580 2024-06-10T01:04:19.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.019+0000 7f48b25fd700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac112990 con 0x7f48ac06ac60 2024-06-10T01:04:19.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.019+0000 7f48a3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48ac06b620 0x7f48ac19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:19.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.019+0000 7f48a3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48ac06b620 0x7f48ac19ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48862/0 (socket says 172.21.15.7:48862) 2024-06-10T01:04:19.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.019+0000 7f48a3fff700 1 -- 172.21.15.7:0/220564005 learned_addr learned my addr 172.21.15.7:0/220564005 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:19.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.019+0000 7f48abfff700 1 --2- 172.21.15.7:0/220564005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48ac06ac60 0x7f48ac19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:19.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.020+0000 7f48a3fff700 1 -- 172.21.15.7:0/220564005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48ac06ac60 msgr2=0x7f48ac19e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:19.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.020+0000 7f48a3fff700 1 --2- 172.21.15.7:0/220564005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48ac06ac60 0x7f48ac19e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:19.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.020+0000 7f48a3fff700 1 -- 172.21.15.7:0/220564005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48ac108580 msgr2=0x7f48ac198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:19.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.020+0000 7f48a3fff700 1 --2- 172.21.15.7:0/220564005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48ac108580 0x7f48ac198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:19.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.020+0000 7f48a3fff700 1 -- 172.21.15.7:0/220564005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48ac1991f0 con 0x7f48ac06b620 2024-06-10T01:04:19.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.020+0000 7f48a3fff700 1 --2- 172.21.15.7:0/220564005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48ac06b620 0x7f48ac19ee10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f489400d980 tx=0x7f489400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:19.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.020+0000 7f48a9ffb700 1 -- 172.21.15.7:0/220564005 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4894016070 con 0x7f48ac06b620 2024-06-10T01:04:19.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.020+0000 7f48b25fd700 1 -- 172.21.15.7:0/220564005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48ac1994e0 con 0x7f48ac06b620 2024-06-10T01:04:19.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.020+0000 7f48a9ffb700 1 -- 172.21.15.7:0/220564005 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4894004d10 con 0x7f48ac06b620 2024-06-10T01:04:19.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.020+0000 7f48a9ffb700 1 -- 172.21.15.7:0/220564005 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48940055f0 con 0x7f48ac06b620 2024-06-10T01:04:19.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.020+0000 7f48b25fd700 1 -- 172.21.15.7:0/220564005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48ac1a57a0 con 0x7f48ac06b620 2024-06-10T01:04:19.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.021+0000 7f48b25fd700 1 -- 172.21.15.7:0/220564005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f488c005330 con 0x7f48ac06b620 2024-06-10T01:04:19.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.022+0000 7f48a9ffb700 1 -- 172.21.15.7:0/220564005 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f4894004220 con 0x7f48ac06b620 2024-06-10T01:04:19.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.023+0000 7f48a9ffb700 1 --2- 172.21.15.7:0/220564005 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f48980a2cb0 0x7f48980a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:19.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.024+0000 7f48abfff700 1 --2- 172.21.15.7:0/220564005 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f48980a2cb0 0x7f48980a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:19.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.024+0000 7f48a9ffb700 1 -- 172.21.15.7:0/220564005 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f48940cf230 con 0x7f48ac06b620 2024-06-10T01:04:19.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.024+0000 7f48abfff700 1 --2- 172.21.15.7:0/220564005 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f48980a2cb0 0x7f48980a5170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f489c000c00 tx=0x7f489c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:19.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.029+0000 7f48a9ffb700 1 -- 172.21.15.7:0/220564005 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f489401f020 con 0x7f48ac06b620 2024-06-10T01:04:19.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.239+0000 7f48b25fd700 1 -- 172.21.15.7:0/220564005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f488c0058f0 con 0x7f48ac06b620 2024-06-10T01:04:19.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.239+0000 7f48a9ffb700 1 -- 172.21.15.7:0/220564005 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f489400f3c0 con 0x7f48ac06b620 2024-06-10T01:04:19.236 INFO:teuthology.orchestra.run.smithi007.stdout:309237645363 2024-06-10T01:04:19.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.241+0000 7f48b25fd700 1 -- 172.21.15.7:0/220564005 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f48980a2cb0 msgr2=0x7f48980a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:19.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.241+0000 7f48b25fd700 1 --2- 172.21.15.7:0/220564005 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f48980a2cb0 0x7f48980a5170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f489c000c00 tx=0x7f489c005fb0 comp rx=0 tx=0).stop 2024-06-10T01:04:19.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.241+0000 7f48b25fd700 1 -- 172.21.15.7:0/220564005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48ac06b620 msgr2=0x7f48ac19ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:19.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.241+0000 7f48b25fd700 1 --2- 172.21.15.7:0/220564005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48ac06b620 0x7f48ac19ee10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f489400d980 tx=0x7f489400dc90 comp rx=0 tx=0).stop 2024-06-10T01:04:19.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.241+0000 7f48b25fd700 1 -- 172.21.15.7:0/220564005 shutdown_connections 2024-06-10T01:04:19.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.241+0000 7f48b25fd700 1 --2- 172.21.15.7:0/220564005 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f48980a2cb0 0x7f48980a5170 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:19.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.241+0000 7f48b25fd700 1 --2- 172.21.15.7:0/220564005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48ac108580 0x7f48ac198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:19.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.241+0000 7f48b25fd700 1 --2- 172.21.15.7:0/220564005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48ac06ac60 0x7f48ac19e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:19.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.241+0000 7f48b25fd700 1 --2- 172.21.15.7:0/220564005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48ac06b620 0x7f48ac19ee10 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:19.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.242+0000 7f48b25fd700 1 -- 172.21.15.7:0/220564005 >> 172.21.15.7:0/220564005 conn(0x7f48ac078430 msgr2=0x7f48ac0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:19.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.242+0000 7f48b25fd700 1 -- 172.21.15.7:0/220564005 shutdown_connections 2024-06-10T01:04:19.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:19.242+0000 7f48b25fd700 1 -- 172.21.15.7:0/220564005 wait complete. 2024-06-10T01:04:19.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:19 smithi007 ceph-mon[32398]: pgmap v380: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:04:19.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:19 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2158427821' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-10T01:04:19.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:19 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/220564005' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-10T01:04:19.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:19 smithi078 ceph-mon[36683]: pgmap v380: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:04:19.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:19 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2158427821' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-10T01:04:19.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:19 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/220564005' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-10T01:04:19.606 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:19 smithi175 ceph-mon[36636]: pgmap v380: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:04:19.606 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:19 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2158427821' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-10T01:04:19.607 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:19 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/220564005' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-10T01:04:20.034 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645358 got 309237645363 for osd.10 2024-06-10T01:04:20.034 DEBUG:teuthology.parallel:result is None 2024-06-10T01:04:20.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.460+0000 7f64c5667700 1 -- 172.21.15.7:0/3427519344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64c01084f0 msgr2=0x7f64c01088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:20.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.460+0000 7f64c5667700 1 --2- 172.21.15.7:0/3427519344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64c01084f0 0x7f64c01088d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f64b0009990 tx=0x7f64b0009ca0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.460+0000 7f64c5667700 1 -- 172.21.15.7:0/3427519344 shutdown_connections 2024-06-10T01:04:20.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.460+0000 7f64c5667700 1 --2- 172.21.15.7:0/3427519344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64c006b3c0 0x7f64c010f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.460+0000 7f64c5667700 1 --2- 172.21.15.7:0/3427519344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64c01084f0 0x7f64c01088d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.460+0000 7f64c5667700 1 --2- 172.21.15.7:0/3427519344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64c006aa00 0x7f64c006ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.460+0000 7f64c5667700 1 -- 172.21.15.7:0/3427519344 >> 172.21.15.7:0/3427519344 conn(0x7f64c0078300 msgr2=0x7f64c0078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:20.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.460+0000 7f64c5667700 1 -- 172.21.15.7:0/3427519344 shutdown_connections 2024-06-10T01:04:20.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.460+0000 7f64c5667700 1 -- 172.21.15.7:0/3427519344 wait complete. 2024-06-10T01:04:20.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.461+0000 7f64c5667700 1 Processor -- start 2024-06-10T01:04:20.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.461+0000 7f64c5667700 1 -- start start 2024-06-10T01:04:20.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.461+0000 7f64c5667700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64c006aa00 0x7f64c019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:20.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.461+0000 7f64c5667700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64c006b3c0 0x7f64c019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:20.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.461+0000 7f64c5667700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64c01084f0 0x7f64c0198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:20.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.461+0000 7f64c5667700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f64c0112770 con 0x7f64c006b3c0 2024-06-10T01:04:20.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.461+0000 7f64c5667700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f64c01125f0 con 0x7f64c006aa00 2024-06-10T01:04:20.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.461+0000 7f64c5667700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f64c01128f0 con 0x7f64c01084f0 2024-06-10T01:04:20.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.461+0000 7f64be7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64c006b3c0 0x7f64c019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:20.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.461+0000 7f64be7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64c006b3c0 0x7f64c019eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48888/0 (socket says 172.21.15.7:48888) 2024-06-10T01:04:20.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.461+0000 7f64be7fc700 1 -- 172.21.15.7:0/1592613144 learned_addr learned my addr 172.21.15.7:0/1592613144 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:20.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.461+0000 7f64bf7fe700 1 --2- 172.21.15.7:0/1592613144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64c01084f0 0x7f64c0198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:20.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.462+0000 7f64beffd700 1 --2- 172.21.15.7:0/1592613144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64c006aa00 0x7f64c019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:20.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.462+0000 7f64bf7fe700 1 -- 172.21.15.7:0/1592613144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64c006aa00 msgr2=0x7f64c019e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:20.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.462+0000 7f64bf7fe700 1 --2- 172.21.15.7:0/1592613144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64c006aa00 0x7f64c019e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.462+0000 7f64bf7fe700 1 -- 172.21.15.7:0/1592613144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64c006b3c0 msgr2=0x7f64c019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:20.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.462+0000 7f64bf7fe700 1 --2- 172.21.15.7:0/1592613144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64c006b3c0 0x7f64c019eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.462+0000 7f64bf7fe700 1 -- 172.21.15.7:0/1592613144 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c01991d0 con 0x7f64c01084f0 2024-06-10T01:04:20.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.462+0000 7f64be7fc700 1 --2- 172.21.15.7:0/1592613144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64c006b3c0 0x7f64c019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:20.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.462+0000 7f64bf7fe700 1 --2- 172.21.15.7:0/1592613144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64c01084f0 0x7f64c0198a70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f64b800efc0 tx=0x7f64b800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:20.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.462+0000 7f64a7fff700 1 -- 172.21.15.7:0/1592613144 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b80092e0 con 0x7f64c01084f0 2024-06-10T01:04:20.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.462+0000 7f64a7fff700 1 -- 172.21.15.7:0/1592613144 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64b8004bb0 con 0x7f64c01084f0 2024-06-10T01:04:20.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.462+0000 7f64c5667700 1 -- 172.21.15.7:0/1592613144 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64c0199460 con 0x7f64c01084f0 2024-06-10T01:04:20.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.462+0000 7f64a7fff700 1 -- 172.21.15.7:0/1592613144 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b8004280 con 0x7f64c01084f0 2024-06-10T01:04:20.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.462+0000 7f64c5667700 1 -- 172.21.15.7:0/1592613144 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64c01a5730 con 0x7f64c01084f0 2024-06-10T01:04:20.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.463+0000 7f64c5667700 1 -- 172.21.15.7:0/1592613144 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64c010a9d0 con 0x7f64c01084f0 2024-06-10T01:04:20.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.464+0000 7f64a7fff700 1 -- 172.21.15.7:0/1592613144 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f64b801b030 con 0x7f64c01084f0 2024-06-10T01:04:20.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.465+0000 7f64a7fff700 1 --2- 172.21.15.7:0/1592613144 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f64a80a2cb0 0x7f64a80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:20.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.465+0000 7f64beffd700 1 --2- 172.21.15.7:0/1592613144 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f64a80a2cb0 0x7f64a80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:20.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.465+0000 7f64a7fff700 1 -- 172.21.15.7:0/1592613144 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f64b80cf360 con 0x7f64c01084f0 2024-06-10T01:04:20.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.466+0000 7f64beffd700 1 --2- 172.21.15.7:0/1592613144 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f64a80a2cb0 0x7f64a80a5170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f64b000b2e0 tx=0x7f64b001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:20.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.468+0000 7f64a7fff700 1 -- 172.21.15.7:0/1592613144 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f64b8022020 con 0x7f64c01084f0 2024-06-10T01:04:20.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.677+0000 7f64c5667700 1 -- 172.21.15.7:0/1592613144 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f64c0199f50 con 0x7f64c01084f0 2024-06-10T01:04:20.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.677+0000 7f64a7fff700 1 -- 172.21.15.7:0/1592613144 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f64b8010040 con 0x7f64c01084f0 2024-06-10T01:04:20.675 INFO:teuthology.orchestra.run.smithi007.stdout:197568495687 2024-06-10T01:04:20.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.680+0000 7f64c5667700 1 -- 172.21.15.7:0/1592613144 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f64a80a2cb0 msgr2=0x7f64a80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:20.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.680+0000 7f64c5667700 1 --2- 172.21.15.7:0/1592613144 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f64a80a2cb0 0x7f64a80a5170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f64b000b2e0 tx=0x7f64b001a040 comp rx=0 tx=0).stop 2024-06-10T01:04:20.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.680+0000 7f64c5667700 1 -- 172.21.15.7:0/1592613144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64c01084f0 msgr2=0x7f64c0198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:20.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.680+0000 7f64c5667700 1 --2- 172.21.15.7:0/1592613144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64c01084f0 0x7f64c0198a70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f64b800efc0 tx=0x7f64b800c3a0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.680+0000 7f64c5667700 1 -- 172.21.15.7:0/1592613144 shutdown_connections 2024-06-10T01:04:20.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.680+0000 7f64c5667700 1 --2- 172.21.15.7:0/1592613144 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f64a80a2cb0 0x7f64a80a5170 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.680+0000 7f64c5667700 1 --2- 172.21.15.7:0/1592613144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64c01084f0 0x7f64c0198a70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.680+0000 7f64c5667700 1 --2- 172.21.15.7:0/1592613144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64c006aa00 0x7f64c019e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.680+0000 7f64c5667700 1 --2- 172.21.15.7:0/1592613144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64c006b3c0 0x7f64c019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.680+0000 7f64c5667700 1 -- 172.21.15.7:0/1592613144 >> 172.21.15.7:0/1592613144 conn(0x7f64c0078300 msgr2=0x7f64c0100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:20.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.680+0000 7f64c5667700 1 -- 172.21.15.7:0/1592613144 shutdown_connections 2024-06-10T01:04:20.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.680+0000 7f64c5667700 1 -- 172.21.15.7:0/1592613144 wait complete. 2024-06-10T01:04:20.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.683+0000 7f2165983700 1 -- 172.21.15.7:0/3594584311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2160101390 msgr2=0x7f2160101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:20.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.683+0000 7f2165983700 1 --2- 172.21.15.7:0/3594584311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2160101390 0x7f2160101810 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2154009990 tx=0x7f2154009ca0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.683+0000 7f2165983700 1 -- 172.21.15.7:0/3594584311 shutdown_connections 2024-06-10T01:04:20.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.683+0000 7f2165983700 1 --2- 172.21.15.7:0/3594584311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2160101d50 0x7f216010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.683+0000 7f2165983700 1 --2- 172.21.15.7:0/3594584311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2160107400 0x7f21601077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.683+0000 7f2165983700 1 --2- 172.21.15.7:0/3594584311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2160101390 0x7f2160101810 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.683+0000 7f2165983700 1 -- 172.21.15.7:0/3594584311 >> 172.21.15.7:0/3594584311 conn(0x7f2160078340 msgr2=0x7f2160078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:20.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.683+0000 7f2165983700 1 -- 172.21.15.7:0/3594584311 shutdown_connections 2024-06-10T01:04:20.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.683+0000 7f2165983700 1 -- 172.21.15.7:0/3594584311 wait complete. 2024-06-10T01:04:20.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.684+0000 7f2165983700 1 Processor -- start 2024-06-10T01:04:20.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.684+0000 7f2165983700 1 -- start start 2024-06-10T01:04:20.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.684+0000 7f2165983700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2160101390 0x7f216019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:20.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.684+0000 7f2165983700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2160101d50 0x7f216019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:20.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.684+0000 7f2165983700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2160107400 0x7f216019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:20.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.684+0000 7f2165983700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2160111670 con 0x7f2160101390 2024-06-10T01:04:20.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.684+0000 7f2165983700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f21601114f0 con 0x7f2160107400 2024-06-10T01:04:20.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.684+0000 7f2165983700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f21601117f0 con 0x7f2160101d50 2024-06-10T01:04:20.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.685+0000 7f215e7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2160101d50 0x7f216019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:20.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.685+0000 7f215effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2160101390 0x7f216019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:20.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.685+0000 7f215e7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2160101d50 0x7f216019b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:41906/0 (socket says 172.21.15.7:41906) 2024-06-10T01:04:20.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.685+0000 7f215e7fc700 1 -- 172.21.15.7:0/913805719 learned_addr learned my addr 172.21.15.7:0/913805719 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:20.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.685+0000 7f215f7fe700 1 --2- 172.21.15.7:0/913805719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2160107400 0x7f216019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:20.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.685+0000 7f215e7fc700 1 -- 172.21.15.7:0/913805719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2160107400 msgr2=0x7f216019f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:20.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.685+0000 7f215e7fc700 1 --2- 172.21.15.7:0/913805719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2160107400 0x7f216019f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.685+0000 7f215e7fc700 1 -- 172.21.15.7:0/913805719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2160101390 msgr2=0x7f216019ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:20.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.685+0000 7f215e7fc700 1 --2- 172.21.15.7:0/913805719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2160101390 0x7f216019ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.685+0000 7f215e7fc700 1 -- 172.21.15.7:0/913805719 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f216019fe30 con 0x7f2160101d50 2024-06-10T01:04:20.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.685+0000 7f215effd700 1 --2- 172.21.15.7:0/913805719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2160101390 0x7f216019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:20.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.685+0000 7f215e7fc700 1 --2- 172.21.15.7:0/913805719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2160101d50 0x7f216019b360 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2154000c00 tx=0x7f215400bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:20.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.685+0000 7f2164981700 1 -- 172.21.15.7:0/913805719 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2154027070 con 0x7f2160101d50 2024-06-10T01:04:20.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.685+0000 7f2164981700 1 -- 172.21.15.7:0/913805719 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2154003d70 con 0x7f2160101d50 2024-06-10T01:04:20.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.685+0000 7f2164981700 1 -- 172.21.15.7:0/913805719 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215400f460 con 0x7f2160101d50 2024-06-10T01:04:20.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.685+0000 7f2165983700 1 -- 172.21.15.7:0/913805719 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21601a00c0 con 0x7f2160101d50 2024-06-10T01:04:20.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.686+0000 7f2165983700 1 -- 172.21.15.7:0/913805719 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21601a79a0 con 0x7f2160101d50 2024-06-10T01:04:20.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.686+0000 7f2165983700 1 -- 172.21.15.7:0/913805719 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2160069180 con 0x7f2160101d50 2024-06-10T01:04:20.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.688+0000 7f2164981700 1 -- 172.21.15.7:0/913805719 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f215402a070 con 0x7f2160101d50 2024-06-10T01:04:20.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.688+0000 7f2164981700 1 --2- 172.21.15.7:0/913805719 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21480a2d30 0x7f21480a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:20.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.689+0000 7f215effd700 1 --2- 172.21.15.7:0/913805719 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21480a2d30 0x7f21480a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:20.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.689+0000 7f2164981700 1 -- 172.21.15.7:0/913805719 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f21540d0600 con 0x7f2160101d50 2024-06-10T01:04:20.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.689+0000 7f215effd700 1 --2- 172.21.15.7:0/913805719 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21480a2d30 0x7f21480a51f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2150005950 tx=0x7f215000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:20.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.692+0000 7f2164981700 1 -- 172.21.15.7:0/913805719 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2154013070 con 0x7f2160101d50 2024-06-10T01:04:20.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.648+0000 7f1dd40ad700 1 -- 172.21.15.7:0/4197672411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dcc102740 msgr2=0x7f1dcc102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:20.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.648+0000 7f1dd40ad700 1 --2- 172.21.15.7:0/4197672411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dcc102740 0x7f1dcc102b20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1dc80099c0 tx=0x7f1dc8009cd0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.648+0000 7f1dd40ad700 1 -- 172.21.15.7:0/4197672411 shutdown_connections 2024-06-10T01:04:20.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.648+0000 7f1dd40ad700 1 --2- 172.21.15.7:0/4197672411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1dcc107840 0x7f1dcc10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.648+0000 7f1dd40ad700 1 --2- 172.21.15.7:0/4197672411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dcc102740 0x7f1dcc102b20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.648+0000 7f1dd40ad700 1 --2- 172.21.15.7:0/4197672411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1dcc1030f0 0x7f1dcc107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.648+0000 7f1dd40ad700 1 -- 172.21.15.7:0/4197672411 >> 172.21.15.7:0/4197672411 conn(0x7f1dcc0795f0 msgr2=0x7f1dcc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.648+0000 7f1dd40ad700 1 -- 172.21.15.7:0/4197672411 shutdown_connections 2024-06-10T01:04:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.648+0000 7f1dd40ad700 1 -- 172.21.15.7:0/4197672411 wait complete. 2024-06-10T01:04:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.649+0000 7f1dd40ad700 1 Processor -- start 2024-06-10T01:04:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.649+0000 7f1dd40ad700 1 -- start start 2024-06-10T01:04:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.649+0000 7f1dd40ad700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dcc102740 0x7f1dcc19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.649+0000 7f1dd40ad700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1dcc1030f0 0x7f1dcc19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.649+0000 7f1dd40ad700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1dcc107840 0x7f1dcc1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.649+0000 7f1dd40ad700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1dcc112760 con 0x7f1dcc102740 2024-06-10T01:04:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.649+0000 7f1dd40ad700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1dcc1125e0 con 0x7f1dcc107840 2024-06-10T01:04:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.649+0000 7f1dd40ad700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1dcc1128e0 con 0x7f1dcc1030f0 2024-06-10T01:04:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.649+0000 7f1dd1e49700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dcc102740 0x7f1dcc19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.649+0000 7f1dd1e49700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dcc102740 0x7f1dcc19e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48840/0 (socket says 172.21.15.7:48840) 2024-06-10T01:04:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.649+0000 7f1dd1e49700 1 -- 172.21.15.7:0/2158427821 learned_addr learned my addr 172.21.15.7:0/2158427821 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:20.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.649+0000 7f1dd264a700 1 --2- 172.21.15.7:0/2158427821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1dcc107840 0x7f1dcc1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:20.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.649+0000 7f1dd1648700 1 --2- 172.21.15.7:0/2158427821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1dcc1030f0 0x7f1dcc19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:20.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.649+0000 7f1dd1e49700 1 -- 172.21.15.7:0/2158427821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1dcc1030f0 msgr2=0x7f1dcc19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:20.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.649+0000 7f1dd1e49700 1 --2- 172.21.15.7:0/2158427821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1dcc1030f0 0x7f1dcc19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.649+0000 7f1dd1e49700 1 -- 172.21.15.7:0/2158427821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1dcc107840 msgr2=0x7f1dcc1989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:20.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.649+0000 7f1dd1e49700 1 --2- 172.21.15.7:0/2158427821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1dcc107840 0x7f1dcc1989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.649+0000 7f1dd1e49700 1 -- 172.21.15.7:0/2158427821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dcc199140 con 0x7f1dcc102740 2024-06-10T01:04:20.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.650+0000 7f1dd1e49700 1 --2- 172.21.15.7:0/2158427821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dcc102740 0x7f1dcc19e820 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8006010 tx=0x7f1dc8009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:20.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.650+0000 7f1dc2ffd700 1 -- 172.21.15.7:0/2158427821 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8027070 con 0x7f1dcc102740 2024-06-10T01:04:20.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.650+0000 7f1dd40ad700 1 -- 172.21.15.7:0/2158427821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dcc1993d0 con 0x7f1dcc102740 2024-06-10T01:04:20.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.650+0000 7f1dc2ffd700 1 -- 172.21.15.7:0/2158427821 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1dc8003780 con 0x7f1dcc102740 2024-06-10T01:04:20.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.650+0000 7f1dc2ffd700 1 -- 172.21.15.7:0/2158427821 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8022da0 con 0x7f1dcc102740 2024-06-10T01:04:20.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.650+0000 7f1dd40ad700 1 -- 172.21.15.7:0/2158427821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dcc1a56f0 con 0x7f1dcc102740 2024-06-10T01:04:20.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.651+0000 7f1dd40ad700 1 -- 172.21.15.7:0/2158427821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dcc103bd0 con 0x7f1dcc102740 2024-06-10T01:04:20.769 INFO:teuthology.orchestra.run.smithi007.stdout:163208757323 2024-06-10T01:04:20.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.652+0000 7f1dc2ffd700 1 -- 172.21.15.7:0/2158427821 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f1dc802a070 con 0x7f1dcc102740 2024-06-10T01:04:20.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.653+0000 7f1dc2ffd700 1 --2- 172.21.15.7:0/2158427821 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1db80a2cb0 0x7f1db80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:20.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.653+0000 7f1dd1648700 1 --2- 172.21.15.7:0/2158427821 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1db80a2cb0 0x7f1db80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:20.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.653+0000 7f1dc2ffd700 1 -- 172.21.15.7:0/2158427821 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f1dc80d0520 con 0x7f1dcc102740 2024-06-10T01:04:20.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.653+0000 7f1dd1648700 1 --2- 172.21.15.7:0/2158427821 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1db80a2cb0 0x7f1db80a5170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc005950 tx=0x7f1dbc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:20.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.656+0000 7f1dc2ffd700 1 -- 172.21.15.7:0/2158427821 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1dc8031080 con 0x7f1dcc102740 2024-06-10T01:04:20.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.867+0000 7f1dd40ad700 1 -- 172.21.15.7:0/2158427821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f1dcc02dca0 con 0x7f1dcc102740 2024-06-10T01:04:20.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.868+0000 7f1dc2ffd700 1 -- 172.21.15.7:0/2158427821 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1dc8013070 con 0x7f1dcc102740 2024-06-10T01:04:20.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.870+0000 7f1dd40ad700 1 -- 172.21.15.7:0/2158427821 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1db80a2cb0 msgr2=0x7f1db80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:20.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.870+0000 7f1dd40ad700 1 --2- 172.21.15.7:0/2158427821 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1db80a2cb0 0x7f1db80a5170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc005950 tx=0x7f1dbc0058e0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.870+0000 7f1dd40ad700 1 -- 172.21.15.7:0/2158427821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dcc102740 msgr2=0x7f1dcc19e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:20.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.870+0000 7f1dd40ad700 1 --2- 172.21.15.7:0/2158427821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dcc102740 0x7f1dcc19e820 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8006010 tx=0x7f1dc8009f80 comp rx=0 tx=0).stop 2024-06-10T01:04:20.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.870+0000 7f1dd40ad700 1 -- 172.21.15.7:0/2158427821 shutdown_connections 2024-06-10T01:04:20.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.870+0000 7f1dd40ad700 1 --2- 172.21.15.7:0/2158427821 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1db80a2cb0 0x7f1db80a5170 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.870+0000 7f1dd40ad700 1 --2- 172.21.15.7:0/2158427821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1dcc107840 0x7f1dcc1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.870+0000 7f1dd40ad700 1 --2- 172.21.15.7:0/2158427821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dcc102740 0x7f1dcc19e820 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.870+0000 7f1dd40ad700 1 --2- 172.21.15.7:0/2158427821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1dcc1030f0 0x7f1dcc19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.870+0000 7f1dd40ad700 1 -- 172.21.15.7:0/2158427821 >> 172.21.15.7:0/2158427821 conn(0x7f1dcc0795f0 msgr2=0x7f1dcc100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:20.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.870+0000 7f1dd40ad700 1 -- 172.21.15.7:0/2158427821 shutdown_connections 2024-06-10T01:04:20.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:18.870+0000 7f1dd40ad700 1 -- 172.21.15.7:0/2158427821 wait complete. 2024-06-10T01:04:20.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.906+0000 7f2165983700 1 -- 172.21.15.7:0/913805719 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f21601a7b70 con 0x7f2160101d50 2024-06-10T01:04:20.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.907+0000 7f2164981700 1 -- 172.21.15.7:0/913805719 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f215409d0b0 con 0x7f2160101d50 2024-06-10T01:04:20.904 INFO:teuthology.orchestra.run.smithi007.stdout:219043332162 2024-06-10T01:04:20.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.909+0000 7f2165983700 1 -- 172.21.15.7:0/913805719 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21480a2d30 msgr2=0x7f21480a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:20.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.909+0000 7f2165983700 1 --2- 172.21.15.7:0/913805719 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21480a2d30 0x7f21480a51f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2150005950 tx=0x7f215000b410 comp rx=0 tx=0).stop 2024-06-10T01:04:20.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.909+0000 7f2165983700 1 -- 172.21.15.7:0/913805719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2160101d50 msgr2=0x7f216019b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:20.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.909+0000 7f2165983700 1 --2- 172.21.15.7:0/913805719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2160101d50 0x7f216019b360 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2154000c00 tx=0x7f215400bab0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.909+0000 7f2165983700 1 -- 172.21.15.7:0/913805719 shutdown_connections 2024-06-10T01:04:20.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.909+0000 7f2165983700 1 --2- 172.21.15.7:0/913805719 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21480a2d30 0x7f21480a51f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.909+0000 7f2165983700 1 --2- 172.21.15.7:0/913805719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2160107400 0x7f216019f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.909+0000 7f2165983700 1 --2- 172.21.15.7:0/913805719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2160101390 0x7f216019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.909+0000 7f2165983700 1 --2- 172.21.15.7:0/913805719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2160101d50 0x7f216019b360 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:20.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.909+0000 7f2165983700 1 -- 172.21.15.7:0/913805719 >> 172.21.15.7:0/913805719 conn(0x7f2160078340 msgr2=0x7f216006a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:20.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.909+0000 7f2165983700 1 -- 172.21.15.7:0/913805719 shutdown_connections 2024-06-10T01:04:20.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:20.909+0000 7f2165983700 1 -- 172.21.15.7:0/913805719 wait complete. 2024-06-10T01:04:21.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:21 smithi078 ceph-mon[36683]: pgmap v381: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T01:04:21.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:21 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1592613144' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-10T01:04:21.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:21 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/913805719' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-10T01:04:21.649 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:21 smithi007 ceph-mon[32398]: pgmap v381: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T01:04:21.649 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:21 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1592613144' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-10T01:04:21.649 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:21 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/913805719' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-10T01:04:21.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:21 smithi175 ceph-mon[36636]: pgmap v381: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T01:04:21.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:21 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1592613144' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-10T01:04:21.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:21 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/913805719' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-10T01:04:21.764 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:21.793 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920844 got 141733920847 for osd.4 2024-06-10T01:04:21.793 DEBUG:teuthology.parallel:result is None 2024-06-10T01:04:22.181 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332156 got 219043332162 for osd.7 2024-06-10T01:04:22.181 DEBUG:teuthology.parallel:result is None 2024-06-10T01:04:22.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.811+0000 7f0d79d51700 1 -- 172.21.15.7:0/2541430524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d741017d0 msgr2=0x7f0d7410f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:22.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.811+0000 7f0d79d51700 1 --2- 172.21.15.7:0/2541430524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d741017d0 0x7f0d7410f260 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c00b0a0 tx=0x7f0d6c00b3b0 comp rx=0 tx=0).stop 2024-06-10T01:04:22.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.811+0000 7f0d79d51700 1 -- 172.21.15.7:0/2541430524 shutdown_connections 2024-06-10T01:04:22.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.811+0000 7f0d79d51700 1 --2- 172.21.15.7:0/2541430524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d741017d0 0x7f0d7410f260 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:22.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.811+0000 7f0d79d51700 1 --2- 172.21.15.7:0/2541430524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d74107570 0x7f0d74107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:22.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.811+0000 7f0d79d51700 1 --2- 172.21.15.7:0/2541430524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d74100e30 0x7f0d74101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:22.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.811+0000 7f0d79d51700 1 -- 172.21.15.7:0/2541430524 >> 172.21.15.7:0/2541430524 conn(0x7f0d74076680 msgr2=0x7f0d74076a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:22.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.811+0000 7f0d79d51700 1 -- 172.21.15.7:0/2541430524 shutdown_connections 2024-06-10T01:04:22.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.811+0000 7f0d79d51700 1 -- 172.21.15.7:0/2541430524 wait complete. 2024-06-10T01:04:22.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.811+0000 7f0d79d51700 1 Processor -- start 2024-06-10T01:04:22.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.812+0000 7f0d79d51700 1 -- start start 2024-06-10T01:04:22.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.812+0000 7f0d79d51700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d74100e30 0x7f0d7419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:22.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.812+0000 7f0d79d51700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d741017d0 0x7f0d7419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:22.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.812+0000 7f0d79d51700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d74107570 0x7f0d74198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:22.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.812+0000 7f0d79d51700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0d74112780 con 0x7f0d741017d0 2024-06-10T01:04:22.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.812+0000 7f0d79d51700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0d74112600 con 0x7f0d74100e30 2024-06-10T01:04:22.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.812+0000 7f0d79d51700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0d74112900 con 0x7f0d74107570 2024-06-10T01:04:22.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.812+0000 7f0d72ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d741017d0 0x7f0d7419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:22.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.812+0000 7f0d72ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d741017d0 0x7f0d7419ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52812/0 (socket says 172.21.15.7:52812) 2024-06-10T01:04:22.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.812+0000 7f0d72ffd700 1 -- 172.21.15.7:0/1361665422 learned_addr learned my addr 172.21.15.7:0/1361665422 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:22.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.812+0000 7f0d73fff700 1 --2- 172.21.15.7:0/1361665422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d74107570 0x7f0d74198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:22.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.812+0000 7f0d737fe700 1 --2- 172.21.15.7:0/1361665422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d74100e30 0x7f0d7419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:22.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.812+0000 7f0d72ffd700 1 -- 172.21.15.7:0/1361665422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d74107570 msgr2=0x7f0d74198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:22.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.812+0000 7f0d72ffd700 1 --2- 172.21.15.7:0/1361665422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d74107570 0x7f0d74198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:22.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.812+0000 7f0d72ffd700 1 -- 172.21.15.7:0/1361665422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d74100e30 msgr2=0x7f0d7419e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:22.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.812+0000 7f0d72ffd700 1 --2- 172.21.15.7:0/1361665422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d74100e30 0x7f0d7419e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:22.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.812+0000 7f0d72ffd700 1 -- 172.21.15.7:0/1361665422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d741991c0 con 0x7f0d741017d0 2024-06-10T01:04:22.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.812+0000 7f0d737fe700 1 --2- 172.21.15.7:0/1361665422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d74100e30 0x7f0d7419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:04:22.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.812+0000 7f0d72ffd700 1 --2- 172.21.15.7:0/1361665422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d741017d0 0x7f0d7419ed90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0d6800d950 tx=0x7f0d6800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:22.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.813+0000 7f0d70ff9700 1 -- 172.21.15.7:0/1361665422 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68016070 con 0x7f0d741017d0 2024-06-10T01:04:22.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.813+0000 7f0d70ff9700 1 -- 172.21.15.7:0/1361665422 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d68009e20 con 0x7f0d741017d0 2024-06-10T01:04:22.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.813+0000 7f0d79d51700 1 -- 172.21.15.7:0/1361665422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d741994b0 con 0x7f0d741017d0 2024-06-10T01:04:22.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.813+0000 7f0d70ff9700 1 -- 172.21.15.7:0/1361665422 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6800f620 con 0x7f0d741017d0 2024-06-10T01:04:22.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.813+0000 7f0d79d51700 1 -- 172.21.15.7:0/1361665422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d741a5720 con 0x7f0d741017d0 2024-06-10T01:04:22.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.814+0000 7f0d79d51700 1 -- 172.21.15.7:0/1361665422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d74069180 con 0x7f0d741017d0 2024-06-10T01:04:22.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.815+0000 7f0d70ff9700 1 -- 172.21.15.7:0/1361665422 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0d68019070 con 0x7f0d741017d0 2024-06-10T01:04:22.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.816+0000 7f0d70ff9700 1 --2- 172.21.15.7:0/1361665422 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0d5c0a2d30 0x7f0d5c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:22.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.816+0000 7f0d737fe700 1 --2- 172.21.15.7:0/1361665422 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0d5c0a2d30 0x7f0d5c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:22.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.816+0000 7f0d70ff9700 1 -- 172.21.15.7:0/1361665422 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f0d680cf110 con 0x7f0d741017d0 2024-06-10T01:04:22.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.816+0000 7f0d737fe700 1 --2- 172.21.15.7:0/1361665422 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0d5c0a2d30 0x7f0d5c0a51f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0d64005e10 tx=0x7f0d64005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:22.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:22.819+0000 7f0d70ff9700 1 -- 172.21.15.7:0/1361665422 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0d6809bb30 con 0x7f0d741017d0 2024-06-10T01:04:23.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.034+0000 7f0d79d51700 1 -- 172.21.15.7:0/1361665422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f0d7402dca0 con 0x7f0d741017d0 2024-06-10T01:04:23.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.034+0000 7f0d70ff9700 1 -- 172.21.15.7:0/1361665422 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0d6809b430 con 0x7f0d741017d0 2024-06-10T01:04:23.032 INFO:teuthology.orchestra.run.smithi007.stdout:77309411419 2024-06-10T01:04:23.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.037+0000 7f0d79d51700 1 -- 172.21.15.7:0/1361665422 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0d5c0a2d30 msgr2=0x7f0d5c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:23.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.037+0000 7f0d79d51700 1 --2- 172.21.15.7:0/1361665422 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0d5c0a2d30 0x7f0d5c0a51f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0d64005e10 tx=0x7f0d64005d60 comp rx=0 tx=0).stop 2024-06-10T01:04:23.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.037+0000 7f0d79d51700 1 -- 172.21.15.7:0/1361665422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d741017d0 msgr2=0x7f0d7419ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:23.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.037+0000 7f0d79d51700 1 --2- 172.21.15.7:0/1361665422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d741017d0 0x7f0d7419ed90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0d6800d950 tx=0x7f0d6800dc60 comp rx=0 tx=0).stop 2024-06-10T01:04:23.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.037+0000 7f0d79d51700 1 -- 172.21.15.7:0/1361665422 shutdown_connections 2024-06-10T01:04:23.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.037+0000 7f0d79d51700 1 --2- 172.21.15.7:0/1361665422 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0d5c0a2d30 0x7f0d5c0a51f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:23.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.037+0000 7f0d79d51700 1 --2- 172.21.15.7:0/1361665422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d74107570 0x7f0d74198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:23.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.037+0000 7f0d79d51700 1 --2- 172.21.15.7:0/1361665422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d74100e30 0x7f0d7419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:23.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.037+0000 7f0d79d51700 1 --2- 172.21.15.7:0/1361665422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d741017d0 0x7f0d7419ed90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:23.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.037+0000 7f0d79d51700 1 -- 172.21.15.7:0/1361665422 >> 172.21.15.7:0/1361665422 conn(0x7f0d74076680 msgr2=0x7f0d740ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:23.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.037+0000 7f0d79d51700 1 -- 172.21.15.7:0/1361665422 shutdown_connections 2024-06-10T01:04:23.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.037+0000 7f0d79d51700 1 -- 172.21.15.7:0/1361665422 wait complete. 2024-06-10T01:04:23.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:23 smithi078 ceph-mon[36683]: pgmap v382: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:04:23.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:23 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1361665422' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-10T01:04:23.613 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:23 smithi007 ceph-mon[32398]: pgmap v382: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:04:23.613 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:23 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1361665422' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-10T01:04:23.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:23 smithi175 ceph-mon[36636]: pgmap v382: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:04:23.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:23 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1361665422' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-10T01:04:23.941 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411414 got 77309411419 for osd.2 2024-06-10T01:04:23.941 DEBUG:teuthology.parallel:result is None 2024-06-10T01:04:24.345 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757317 got 163208757323 for osd.5 2024-06-10T01:04:24.345 DEBUG:teuthology.parallel:result is None 2024-06-10T01:04:24.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:24 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2968065810' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-10T01:04:24.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.620+0000 7f0a5ad60700 1 -- 172.21.15.7:0/1724251988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a5406ab10 msgr2=0x7f0a5410f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:24.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.620+0000 7f0a5ad60700 1 --2- 172.21.15.7:0/1724251988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a5406ab10 0x7f0a5410f320 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c00b0d0 tx=0x7f0a4c00b3e0 comp rx=0 tx=0).stop 2024-06-10T01:04:24.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.620+0000 7f0a5ad60700 1 -- 172.21.15.7:0/1724251988 shutdown_connections 2024-06-10T01:04:24.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.620+0000 7f0a5ad60700 1 --2- 172.21.15.7:0/1724251988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a5406ab10 0x7f0a5410f320 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:24.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.620+0000 7f0a5ad60700 1 --2- 172.21.15.7:0/1724251988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a541084e0 0x7f0a541088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:24.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.620+0000 7f0a5ad60700 1 --2- 172.21.15.7:0/1724251988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a5406a170 0x7f0a5406a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:24.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.620+0000 7f0a5ad60700 1 -- 172.21.15.7:0/1724251988 >> 172.21.15.7:0/1724251988 conn(0x7f0a540782c0 msgr2=0x7f0a540786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:24.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.620+0000 7f0a5ad60700 1 -- 172.21.15.7:0/1724251988 shutdown_connections 2024-06-10T01:04:24.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.620+0000 7f0a5ad60700 1 -- 172.21.15.7:0/1724251988 wait complete. 2024-06-10T01:04:24.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.621+0000 7f0a5ad60700 1 Processor -- start 2024-06-10T01:04:24.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.621+0000 7f0a5ad60700 1 -- start start 2024-06-10T01:04:24.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.621+0000 7f0a5ad60700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a5406a170 0x7f0a5419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:24.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.621+0000 7f0a5ad60700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a5406ab10 0x7f0a5419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:24.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.621+0000 7f0a5ad60700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a541084e0 0x7f0a541989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:24.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.621+0000 7f0a5ad60700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0a54112760 con 0x7f0a5406a170 2024-06-10T01:04:24.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.621+0000 7f0a5ad60700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0a541125e0 con 0x7f0a5406ab10 2024-06-10T01:04:24.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.621+0000 7f0a5ad60700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0a541128e0 con 0x7f0a541084e0 2024-06-10T01:04:24.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.621+0000 7f0a58afc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a5406a170 0x7f0a5419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:24.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.621+0000 7f0a58afc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a5406a170 0x7f0a5419e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52820/0 (socket says 172.21.15.7:52820) 2024-06-10T01:04:24.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.621+0000 7f0a58afc700 1 -- 172.21.15.7:0/2968065810 learned_addr learned my addr 172.21.15.7:0/2968065810 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:24.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.621+0000 7f0a592fd700 1 --2- 172.21.15.7:0/2968065810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a541084e0 0x7f0a541989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:24.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.621+0000 7f0a4bfff700 1 --2- 172.21.15.7:0/2968065810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a5406ab10 0x7f0a5419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:24.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.621+0000 7f0a58afc700 1 -- 172.21.15.7:0/2968065810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a541084e0 msgr2=0x7f0a541989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:24.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.621+0000 7f0a58afc700 1 --2- 172.21.15.7:0/2968065810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a541084e0 0x7f0a541989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:24.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.621+0000 7f0a58afc700 1 -- 172.21.15.7:0/2968065810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a5406ab10 msgr2=0x7f0a5419ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:24.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.621+0000 7f0a58afc700 1 --2- 172.21.15.7:0/2968065810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a5406ab10 0x7f0a5419ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:24.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.621+0000 7f0a58afc700 1 -- 172.21.15.7:0/2968065810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a54199120 con 0x7f0a5406a170 2024-06-10T01:04:24.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.622+0000 7f0a4bfff700 1 --2- 172.21.15.7:0/2968065810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a5406ab10 0x7f0a5419ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:04:24.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.622+0000 7f0a58afc700 1 --2- 172.21.15.7:0/2968065810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a5406a170 0x7f0a5419e7b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0a4400e9a0 tx=0x7f0a4400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:24.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.622+0000 7f0a49ffb700 1 -- 172.21.15.7:0/2968065810 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4400cbd0 con 0x7f0a5406a170 2024-06-10T01:04:24.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.622+0000 7f0a49ffb700 1 -- 172.21.15.7:0/2968065810 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a4400cd70 con 0x7f0a5406a170 2024-06-10T01:04:24.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.622+0000 7f0a5ad60700 1 -- 172.21.15.7:0/2968065810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a54199410 con 0x7f0a5406a170 2024-06-10T01:04:24.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.622+0000 7f0a49ffb700 1 -- 172.21.15.7:0/2968065810 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a440187c0 con 0x7f0a5406a170 2024-06-10T01:04:24.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.622+0000 7f0a5ad60700 1 -- 172.21.15.7:0/2968065810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a541a5680 con 0x7f0a5406a170 2024-06-10T01:04:24.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.623+0000 7f0a5ad60700 1 -- 172.21.15.7:0/2968065810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a54069180 con 0x7f0a5406a170 2024-06-10T01:04:24.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.625+0000 7f0a49ffb700 1 -- 172.21.15.7:0/2968065810 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0a44010430 con 0x7f0a5406a170 2024-06-10T01:04:24.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.625+0000 7f0a49ffb700 1 --2- 172.21.15.7:0/2968065810 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0a3c0a2d30 0x7f0a3c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:24.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.625+0000 7f0a4bfff700 1 --2- 172.21.15.7:0/2968065810 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0a3c0a2d30 0x7f0a3c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:24.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.625+0000 7f0a49ffb700 1 -- 172.21.15.7:0/2968065810 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f0a440cfc20 con 0x7f0a5406a170 2024-06-10T01:04:24.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.626+0000 7f0a4bfff700 1 --2- 172.21.15.7:0/2968065810 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0a3c0a2d30 0x7f0a3c0a51f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0a50005fd0 tx=0x7f0a50005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:24.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.628+0000 7f0a49ffb700 1 -- 172.21.15.7:0/2968065810 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0a4409c640 con 0x7f0a5406a170 2024-06-10T01:04:24.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.836+0000 7f0a5ad60700 1 -- 172.21.15.7:0/2968065810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f0a5402dca0 con 0x7f0a5406a170 2024-06-10T01:04:24.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.836+0000 7f0a49ffb700 1 -- 172.21.15.7:0/2968065810 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f0a4409bf40 con 0x7f0a5406a170 2024-06-10T01:04:24.639 INFO:teuthology.orchestra.run.smithi007.stdout:343597383727 2024-06-10T01:04:24.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.838+0000 7f0a5ad60700 1 -- 172.21.15.7:0/2968065810 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0a3c0a2d30 msgr2=0x7f0a3c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:24.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.838+0000 7f0a5ad60700 1 --2- 172.21.15.7:0/2968065810 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0a3c0a2d30 0x7f0a3c0a51f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0a50005fd0 tx=0x7f0a50005ee0 comp rx=0 tx=0).stop 2024-06-10T01:04:24.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.838+0000 7f0a5ad60700 1 -- 172.21.15.7:0/2968065810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a5406a170 msgr2=0x7f0a5419e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:24.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.838+0000 7f0a5ad60700 1 --2- 172.21.15.7:0/2968065810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a5406a170 0x7f0a5419e7b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0a4400e9a0 tx=0x7f0a4400ed60 comp rx=0 tx=0).stop 2024-06-10T01:04:24.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.838+0000 7f0a5ad60700 1 -- 172.21.15.7:0/2968065810 shutdown_connections 2024-06-10T01:04:24.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.838+0000 7f0a5ad60700 1 --2- 172.21.15.7:0/2968065810 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0a3c0a2d30 0x7f0a3c0a51f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:24.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.838+0000 7f0a5ad60700 1 --2- 172.21.15.7:0/2968065810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a541084e0 0x7f0a541989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:24.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.839+0000 7f0a5ad60700 1 --2- 172.21.15.7:0/2968065810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a5406a170 0x7f0a5419e7b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:24.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.839+0000 7f0a5ad60700 1 --2- 172.21.15.7:0/2968065810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a5406ab10 0x7f0a5419ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:24.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.839+0000 7f0a5ad60700 1 -- 172.21.15.7:0/2968065810 >> 172.21.15.7:0/2968065810 conn(0x7f0a540782c0 msgr2=0x7f0a540ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:24.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.839+0000 7f0a5ad60700 1 -- 172.21.15.7:0/2968065810 shutdown_connections 2024-06-10T01:04:24.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:23.839+0000 7f0a5ad60700 1 -- 172.21.15.7:0/2968065810 wait complete. 2024-06-10T01:04:24.640 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:24 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2968065810' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-10T01:04:24.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:24 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2968065810' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-10T01:04:25.248 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383723 got 343597383727 for osd.11 2024-06-10T01:04:25.248 DEBUG:teuthology.parallel:result is None 2024-06-10T01:04:25.460 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:25 smithi007 ceph-mon[32398]: pgmap v383: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:04:25.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:25 smithi078 ceph-mon[36683]: pgmap v383: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:04:25.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.657+0000 7f3fe37ff700 1 -- 172.21.15.7:0/591723263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fdc06abe0 msgr2=0x7f3fdc06b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:25.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.657+0000 7f3fe37ff700 1 --2- 172.21.15.7:0/591723263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fdc06abe0 0x7f3fdc06b060 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc009990 tx=0x7f3fcc009ca0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.657+0000 7f3fe37ff700 1 -- 172.21.15.7:0/591723263 shutdown_connections 2024-06-10T01:04:25.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.657+0000 7f3fe37ff700 1 --2- 172.21.15.7:0/591723263 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fdc06b5a0 0x7f3fdc10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.657+0000 7f3fe37ff700 1 --2- 172.21.15.7:0/591723263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fdc108500 0x7f3fdc1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.657+0000 7f3fe37ff700 1 --2- 172.21.15.7:0/591723263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fdc06abe0 0x7f3fdc06b060 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.657+0000 7f3fe37ff700 1 -- 172.21.15.7:0/591723263 >> 172.21.15.7:0/591723263 conn(0x7f3fdc078360 msgr2=0x7f3fdc078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:25.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.657+0000 7f3fe37ff700 1 -- 172.21.15.7:0/591723263 shutdown_connections 2024-06-10T01:04:25.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.657+0000 7f3fe37ff700 1 -- 172.21.15.7:0/591723263 wait complete. 2024-06-10T01:04:25.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.657+0000 7f3fe37ff700 1 Processor -- start 2024-06-10T01:04:25.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.658+0000 7f3fe37ff700 1 -- start start 2024-06-10T01:04:25.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.658+0000 7f3fe37ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fdc06abe0 0x7f3fdc19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:25.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.658+0000 7f3fe37ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fdc06b5a0 0x7f3fdc19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:25.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.658+0000 7f3fe37ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fdc108500 0x7f3fdc198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:25.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.658+0000 7f3fe37ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc1127e0 con 0x7f3fdc06abe0 2024-06-10T01:04:25.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.658+0000 7f3fe37ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc112660 con 0x7f3fdc108500 2024-06-10T01:04:25.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.658+0000 7f3fe37ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc112960 con 0x7f3fdc06b5a0 2024-06-10T01:04:25.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.658+0000 7f3fe159b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fdc06abe0 0x7f3fdc19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:25.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.658+0000 7f3fe159b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fdc06abe0 0x7f3fdc19e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52834/0 (socket says 172.21.15.7:52834) 2024-06-10T01:04:25.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.658+0000 7f3fe159b700 1 -- 172.21.15.7:0/2696765840 learned_addr learned my addr 172.21.15.7:0/2696765840 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:25.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.658+0000 7f3fe0d9a700 1 --2- 172.21.15.7:0/2696765840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fdc06b5a0 0x7f3fdc19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:25.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.658+0000 7f3fe1d9c700 1 --2- 172.21.15.7:0/2696765840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fdc108500 0x7f3fdc198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:25.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.658+0000 7f3fe159b700 1 -- 172.21.15.7:0/2696765840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fdc06b5a0 msgr2=0x7f3fdc19ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:25.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.658+0000 7f3fe159b700 1 --2- 172.21.15.7:0/2696765840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fdc06b5a0 0x7f3fdc19ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.658+0000 7f3fe159b700 1 -- 172.21.15.7:0/2696765840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fdc108500 msgr2=0x7f3fdc198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:25.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.658+0000 7f3fe159b700 1 --2- 172.21.15.7:0/2696765840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fdc108500 0x7f3fdc198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.658+0000 7f3fe159b700 1 -- 172.21.15.7:0/2696765840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fdc199240 con 0x7f3fdc06abe0 2024-06-10T01:04:25.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.659+0000 7f3fe159b700 1 --2- 172.21.15.7:0/2696765840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fdc06abe0 0x7f3fdc19e920 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3fd800b530 tx=0x7f3fd800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:25.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.659+0000 7f3fd27fc700 1 -- 172.21.15.7:0/2696765840 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8015020 con 0x7f3fdc06abe0 2024-06-10T01:04:25.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.659+0000 7f3fe37ff700 1 -- 172.21.15.7:0/2696765840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fdc199530 con 0x7f3fdc06abe0 2024-06-10T01:04:25.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.659+0000 7f3fd27fc700 1 -- 172.21.15.7:0/2696765840 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fd8011d10 con 0x7f3fdc06abe0 2024-06-10T01:04:25.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.659+0000 7f3fd27fc700 1 -- 172.21.15.7:0/2696765840 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd800f2e0 con 0x7f3fdc06abe0 2024-06-10T01:04:25.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.659+0000 7f3fe37ff700 1 -- 172.21.15.7:0/2696765840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fdc1a57f0 con 0x7f3fdc06abe0 2024-06-10T01:04:25.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.660+0000 7f3fe37ff700 1 -- 172.21.15.7:0/2696765840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fdc10a9e0 con 0x7f3fdc06abe0 2024-06-10T01:04:25.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.661+0000 7f3fd27fc700 1 -- 172.21.15.7:0/2696765840 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f3fd8018020 con 0x7f3fdc06abe0 2024-06-10T01:04:25.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.662+0000 7f3fd27fc700 1 --2- 172.21.15.7:0/2696765840 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3fc80a2cb0 0x7f3fc80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:25.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.662+0000 7f3fe0d9a700 1 --2- 172.21.15.7:0/2696765840 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3fc80a2cb0 0x7f3fc80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:25.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.662+0000 7f3fd27fc700 1 -- 172.21.15.7:0/2696765840 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f3fd80cecf0 con 0x7f3fdc06abe0 2024-06-10T01:04:25.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.662+0000 7f3fe0d9a700 1 --2- 172.21.15.7:0/2696765840 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3fc80a2cb0 0x7f3fc80a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00b5c0 tx=0x7f3fcc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:25.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.667+0000 7f3fd27fc700 1 -- 172.21.15.7:0/2696765840 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3fd801f020 con 0x7f3fdc06abe0 2024-06-10T01:04:25.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:25 smithi175 ceph-mon[36636]: pgmap v383: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:04:25.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.878+0000 7f3fe37ff700 1 -- 172.21.15.7:0/2696765840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f3fdc199fd0 con 0x7f3fdc06abe0 2024-06-10T01:04:25.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.878+0000 7f3fd27fc700 1 -- 172.21.15.7:0/2696765840 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3fd809b820 con 0x7f3fdc06abe0 2024-06-10T01:04:25.875 INFO:teuthology.orchestra.run.smithi007.stdout:287762808889 2024-06-10T01:04:25.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.881+0000 7f3fe37ff700 1 -- 172.21.15.7:0/2696765840 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3fc80a2cb0 msgr2=0x7f3fc80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:25.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.881+0000 7f3fe37ff700 1 --2- 172.21.15.7:0/2696765840 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3fc80a2cb0 0x7f3fc80a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00b5c0 tx=0x7f3fcc005960 comp rx=0 tx=0).stop 2024-06-10T01:04:25.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.881+0000 7f3fe37ff700 1 -- 172.21.15.7:0/2696765840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fdc06abe0 msgr2=0x7f3fdc19e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:25.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.881+0000 7f3fe37ff700 1 --2- 172.21.15.7:0/2696765840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fdc06abe0 0x7f3fdc19e920 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3fd800b530 tx=0x7f3fd800b840 comp rx=0 tx=0).stop 2024-06-10T01:04:25.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.881+0000 7f3fe37ff700 1 -- 172.21.15.7:0/2696765840 shutdown_connections 2024-06-10T01:04:25.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.881+0000 7f3fe37ff700 1 --2- 172.21.15.7:0/2696765840 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3fc80a2cb0 0x7f3fc80a5170 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.881+0000 7f3fe37ff700 1 --2- 172.21.15.7:0/2696765840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fdc108500 0x7f3fdc198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.881+0000 7f3fe37ff700 1 --2- 172.21.15.7:0/2696765840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fdc06abe0 0x7f3fdc19e920 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.881+0000 7f3fe37ff700 1 --2- 172.21.15.7:0/2696765840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fdc06b5a0 0x7f3fdc19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.881+0000 7f3fe37ff700 1 -- 172.21.15.7:0/2696765840 >> 172.21.15.7:0/2696765840 conn(0x7f3fdc078360 msgr2=0x7f3fdc100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:25.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.881+0000 7f3fe37ff700 1 -- 172.21.15.7:0/2696765840 shutdown_connections 2024-06-10T01:04:25.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:25.881+0000 7f3fe37ff700 1 -- 172.21.15.7:0/2696765840 wait complete. 2024-06-10T01:04:26.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:26 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2696765840' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-10T01:04:26.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:26 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2696765840' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-10T01:04:26.744 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:26 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2696765840' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-10T01:04:26.774 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808883 got 287762808889 for osd.9 2024-06-10T01:04:26.774 DEBUG:teuthology.parallel:result is None 2024-06-10T01:04:27.197 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495683 got 197568495687 for osd.6 2024-06-10T01:04:27.198 DEBUG:teuthology.parallel:result is None 2024-06-10T01:04:27.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:27 smithi007 ceph-mon[32398]: pgmap v384: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:04:27.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:27 smithi078 ceph-mon[36683]: pgmap v384: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:04:27.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:27 smithi175 ceph-mon[36636]: pgmap v384: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:04:27.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.973+0000 7fe4f8101700 1 -- 172.21.15.7:0/2801542603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4f01066d0 msgr2=0x7fe4f010e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:27.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.973+0000 7fe4f8101700 1 --2- 172.21.15.7:0/2801542603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4f01066d0 0x7fe4f010e2a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800b0d0 tx=0x7fe4e800b3e0 comp rx=0 tx=0).stop 2024-06-10T01:04:27.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.973+0000 7fe4f8101700 1 -- 172.21.15.7:0/2801542603 shutdown_connections 2024-06-10T01:04:27.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.973+0000 7fe4f8101700 1 --2- 172.21.15.7:0/2801542603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4f01066d0 0x7fe4f010e2a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:27.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.973+0000 7fe4f8101700 1 --2- 172.21.15.7:0/2801542603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4f0101680 0x7fe4f0101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:27.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.973+0000 7fe4f8101700 1 --2- 172.21.15.7:0/2801542603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f0102030 0x7fe4f0105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:27.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.973+0000 7fe4f8101700 1 -- 172.21.15.7:0/2801542603 >> 172.21.15.7:0/2801542603 conn(0x7fe4f00773e0 msgr2=0x7fe4f00777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:27.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.973+0000 7fe4f8101700 1 -- 172.21.15.7:0/2801542603 shutdown_connections 2024-06-10T01:04:27.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.973+0000 7fe4f8101700 1 -- 172.21.15.7:0/2801542603 wait complete. 2024-06-10T01:04:27.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.974+0000 7fe4f8101700 1 Processor -- start 2024-06-10T01:04:27.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.974+0000 7fe4f8101700 1 -- start start 2024-06-10T01:04:27.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.974+0000 7fe4f8101700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f0101680 0x7fe4f0197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:27.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.974+0000 7fe4f8101700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4f0102030 0x7fe4f0197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:27.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.974+0000 7fe4f8101700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4f01066d0 0x7fe4f019c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:27.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.974+0000 7fe4f8101700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f01116c0 con 0x7fe4f0102030 2024-06-10T01:04:27.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.974+0000 7fe4f8101700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f0111540 con 0x7fe4f0101680 2024-06-10T01:04:27.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.974+0000 7fe4f8101700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f0111840 con 0x7fe4f01066d0 2024-06-10T01:04:27.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.974+0000 7fe4f569c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4f0102030 0x7fe4f0197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:27.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.974+0000 7fe4f569c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4f0102030 0x7fe4f0197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52852/0 (socket says 172.21.15.7:52852) 2024-06-10T01:04:27.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.974+0000 7fe4f569c700 1 -- 172.21.15.7:0/1394350682 learned_addr learned my addr 172.21.15.7:0/1394350682 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:27.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.975+0000 7fe4f5e9d700 1 --2- 172.21.15.7:0/1394350682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f0101680 0x7fe4f0197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:27.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.975+0000 7fe4f669e700 1 --2- 172.21.15.7:0/1394350682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4f01066d0 0x7fe4f019c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:27.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.975+0000 7fe4f569c700 1 -- 172.21.15.7:0/1394350682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4f01066d0 msgr2=0x7fe4f019c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:27.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.975+0000 7fe4f569c700 1 --2- 172.21.15.7:0/1394350682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4f01066d0 0x7fe4f019c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:27.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.975+0000 7fe4f569c700 1 -- 172.21.15.7:0/1394350682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f0101680 msgr2=0x7fe4f0197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:27.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.975+0000 7fe4f569c700 1 --2- 172.21.15.7:0/1394350682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f0101680 0x7fe4f0197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:27.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.975+0000 7fe4f569c700 1 -- 172.21.15.7:0/1394350682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f019cce0 con 0x7fe4f0102030 2024-06-10T01:04:27.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.975+0000 7fe4f669e700 1 --2- 172.21.15.7:0/1394350682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4f01066d0 0x7fe4f019c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:04:27.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.975+0000 7fe4f569c700 1 --2- 172.21.15.7:0/1394350682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4f0102030 0x7fe4f0197c90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe4e000c9e0 tx=0x7fe4e000cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:27.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.975+0000 7fe4e6ffd700 1 -- 172.21.15.7:0/1394350682 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0007aa0 con 0x7fe4f0102030 2024-06-10T01:04:27.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.975+0000 7fe4f8101700 1 -- 172.21.15.7:0/1394350682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4f019cfd0 con 0x7fe4f0102030 2024-06-10T01:04:27.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.975+0000 7fe4e6ffd700 1 -- 172.21.15.7:0/1394350682 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e0007c40 con 0x7fe4f0102030 2024-06-10T01:04:27.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.975+0000 7fe4e6ffd700 1 -- 172.21.15.7:0/1394350682 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0018780 con 0x7fe4f0102030 2024-06-10T01:04:27.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.975+0000 7fe4f8101700 1 -- 172.21.15.7:0/1394350682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4f019d560 con 0x7fe4f0102030 2024-06-10T01:04:27.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.976+0000 7fe4f8101700 1 -- 172.21.15.7:0/1394350682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4f0069180 con 0x7fe4f0102030 2024-06-10T01:04:27.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.978+0000 7fe4e6ffd700 1 -- 172.21.15.7:0/1394350682 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fe4e000f450 con 0x7fe4f0102030 2024-06-10T01:04:27.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.978+0000 7fe4e6ffd700 1 --2- 172.21.15.7:0/1394350682 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe4dc0a2cb0 0x7fe4dc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:27.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.978+0000 7fe4f5e9d700 1 --2- 172.21.15.7:0/1394350682 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe4dc0a2cb0 0x7fe4dc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:27.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.978+0000 7fe4e6ffd700 1 -- 172.21.15.7:0/1394350682 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fe4e00cf660 con 0x7fe4f0102030 2024-06-10T01:04:27.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.979+0000 7fe4f5e9d700 1 --2- 172.21.15.7:0/1394350682 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe4dc0a2cb0 0x7fe4dc0a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec005ea0 tx=0x7fe4ec005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:27.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:27.982+0000 7fe4e6ffd700 1 -- 172.21.15.7:0/1394350682 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe4e0014030 con 0x7fe4f0102030 2024-06-10T01:04:28.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.200+0000 7fe4f8101700 1 -- 172.21.15.7:0/1394350682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fe4f006b250 con 0x7fe4f0102030 2024-06-10T01:04:28.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.201+0000 7fe4e6ffd700 1 -- 172.21.15.7:0/1394350682 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe4e009c0e0 con 0x7fe4f0102030 2024-06-10T01:04:28.198 INFO:teuthology.orchestra.run.smithi007.stdout:34359738470 2024-06-10T01:04:28.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.204+0000 7fe4f8101700 1 -- 172.21.15.7:0/1394350682 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe4dc0a2cb0 msgr2=0x7fe4dc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:28.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.204+0000 7fe4f8101700 1 --2- 172.21.15.7:0/1394350682 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe4dc0a2cb0 0x7fe4dc0a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec005ea0 tx=0x7fe4ec005e30 comp rx=0 tx=0).stop 2024-06-10T01:04:28.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.204+0000 7fe4f8101700 1 -- 172.21.15.7:0/1394350682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4f0102030 msgr2=0x7fe4f0197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:28.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.204+0000 7fe4f8101700 1 --2- 172.21.15.7:0/1394350682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4f0102030 0x7fe4f0197c90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe4e000c9e0 tx=0x7fe4e000cda0 comp rx=0 tx=0).stop 2024-06-10T01:04:28.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.204+0000 7fe4f8101700 1 -- 172.21.15.7:0/1394350682 shutdown_connections 2024-06-10T01:04:28.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.204+0000 7fe4f8101700 1 --2- 172.21.15.7:0/1394350682 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe4dc0a2cb0 0x7fe4dc0a5170 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:28.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.204+0000 7fe4f8101700 1 --2- 172.21.15.7:0/1394350682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4f01066d0 0x7fe4f019c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:28.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.204+0000 7fe4f8101700 1 --2- 172.21.15.7:0/1394350682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f0101680 0x7fe4f0197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:28.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.204+0000 7fe4f8101700 1 --2- 172.21.15.7:0/1394350682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4f0102030 0x7fe4f0197c90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:28.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.204+0000 7fe4f8101700 1 -- 172.21.15.7:0/1394350682 >> 172.21.15.7:0/1394350682 conn(0x7fe4f00773e0 msgr2=0x7fe4f00ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:28.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.204+0000 7fe4f8101700 1 -- 172.21.15.7:0/1394350682 shutdown_connections 2024-06-10T01:04:28.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.204+0000 7fe4f8101700 1 -- 172.21.15.7:0/1394350682 wait complete. 2024-06-10T01:04:28.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.721+0000 7fe5c7ee8700 1 -- 172.21.15.7:0/2753128723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5c0102ef0 msgr2=0x7fe5c010f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:28.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.721+0000 7fe5c7ee8700 1 --2- 172.21.15.7:0/2753128723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5c0102ef0 0x7fe5c010f3a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800b0d0 tx=0x7fe5b800b3e0 comp rx=0 tx=0).stop 2024-06-10T01:04:28.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.721+0000 7fe5c7ee8700 1 -- 172.21.15.7:0/2753128723 shutdown_connections 2024-06-10T01:04:28.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.721+0000 7fe5c7ee8700 1 --2- 172.21.15.7:0/2753128723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5c0102ef0 0x7fe5c010f3a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:28.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.721+0000 7fe5c7ee8700 1 --2- 172.21.15.7:0/2753128723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5c01085d0 0x7fe5c01089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:28.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.721+0000 7fe5c7ee8700 1 --2- 172.21.15.7:0/2753128723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5c0102530 0x7fe5c01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:28.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.721+0000 7fe5c7ee8700 1 -- 172.21.15.7:0/2753128723 >> 172.21.15.7:0/2753128723 conn(0x7fe5c0079640 msgr2=0x7fe5c0079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:28.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.721+0000 7fe5c7ee8700 1 -- 172.21.15.7:0/2753128723 shutdown_connections 2024-06-10T01:04:28.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.721+0000 7fe5c7ee8700 1 -- 172.21.15.7:0/2753128723 wait complete. 2024-06-10T01:04:28.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.722+0000 7fe5c7ee8700 1 Processor -- start 2024-06-10T01:04:28.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.722+0000 7fe5c7ee8700 1 -- start start 2024-06-10T01:04:28.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.722+0000 7fe5c7ee8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5c0102530 0x7fe5c0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:28.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.722+0000 7fe5c7ee8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5c0102ef0 0x7fe5c0074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:28.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.722+0000 7fe5c7ee8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5c01085d0 0x7fe5c0075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:28.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.722+0000 7fe5c7ee8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c0112890 con 0x7fe5c01085d0 2024-06-10T01:04:28.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.722+0000 7fe5c7ee8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c0112710 con 0x7fe5c0102530 2024-06-10T01:04:28.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.722+0000 7fe5c7ee8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c0112a10 con 0x7fe5c0102ef0 2024-06-10T01:04:28.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.723+0000 7fe5c6485700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5c01085d0 0x7fe5c0075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:28.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.723+0000 7fe5c6485700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5c01085d0 0x7fe5c0075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52872/0 (socket says 172.21.15.7:52872) 2024-06-10T01:04:28.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.723+0000 7fe5c6485700 1 -- 172.21.15.7:0/1068540911 learned_addr learned my addr 172.21.15.7:0/1068540911 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:28.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.723+0000 7fe5c5c84700 1 --2- 172.21.15.7:0/1068540911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5c0102530 0x7fe5c0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:28.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.723+0000 7fe5c5483700 1 --2- 172.21.15.7:0/1068540911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5c0102ef0 0x7fe5c0074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:28.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.723+0000 7fe5c6485700 1 -- 172.21.15.7:0/1068540911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5c0102ef0 msgr2=0x7fe5c0074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:28.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.723+0000 7fe5c6485700 1 --2- 172.21.15.7:0/1068540911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5c0102ef0 0x7fe5c0074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:28.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.723+0000 7fe5c6485700 1 -- 172.21.15.7:0/1068540911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5c0102530 msgr2=0x7fe5c0074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:28.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.723+0000 7fe5c6485700 1 --2- 172.21.15.7:0/1068540911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5c0102530 0x7fe5c0074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:28.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.723+0000 7fe5c6485700 1 -- 172.21.15.7:0/1068540911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c01a7170 con 0x7fe5c01085d0 2024-06-10T01:04:28.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.723+0000 7fe5c5483700 1 --2- 172.21.15.7:0/1068540911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5c0102ef0 0x7fe5c0074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:04:28.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.723+0000 7fe5c6485700 1 --2- 172.21.15.7:0/1068540911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5c01085d0 0x7fe5c0075210 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8006070 tx=0x7fe5b8004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:28.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.723+0000 7fe5b6ffd700 1 -- 172.21.15.7:0/1068540911 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8021070 con 0x7fe5c01085d0 2024-06-10T01:04:28.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.724+0000 7fe5c7ee8700 1 -- 172.21.15.7:0/1068540911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c01a7400 con 0x7fe5c01085d0 2024-06-10T01:04:28.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.724+0000 7fe5b6ffd700 1 -- 172.21.15.7:0/1068540911 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b8007b80 con 0x7fe5c01085d0 2024-06-10T01:04:28.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.724+0000 7fe5b6ffd700 1 -- 172.21.15.7:0/1068540911 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8005330 con 0x7fe5c01085d0 2024-06-10T01:04:28.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.724+0000 7fe5c7ee8700 1 -- 172.21.15.7:0/1068540911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c01a7900 con 0x7fe5c01085d0 2024-06-10T01:04:28.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.725+0000 7fe5c7ee8700 1 -- 172.21.15.7:0/1068540911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5c0103c40 con 0x7fe5c01085d0 2024-06-10T01:04:28.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.726+0000 7fe5b6ffd700 1 -- 172.21.15.7:0/1068540911 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fe5b8024070 con 0x7fe5c01085d0 2024-06-10T01:04:28.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.727+0000 7fe5b6ffd700 1 --2- 172.21.15.7:0/1068540911 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe5ac0a2d30 0x7fe5ac0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:28.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.727+0000 7fe5c5c84700 1 --2- 172.21.15.7:0/1068540911 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe5ac0a2d30 0x7fe5ac0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:28.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.727+0000 7fe5b6ffd700 1 -- 172.21.15.7:0/1068540911 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fe5b800f020 con 0x7fe5c01085d0 2024-06-10T01:04:28.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.728+0000 7fe5c5c84700 1 --2- 172.21.15.7:0/1068540911 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe5ac0a2d30 0x7fe5ac0a51f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc005950 tx=0x7fe5bc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:28.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.731+0000 7fe5b6ffd700 1 -- 172.21.15.7:0/1068540911 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe5b802b020 con 0x7fe5c01085d0 2024-06-10T01:04:28.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.951+0000 7fe5c7ee8700 1 -- 172.21.15.7:0/1068540911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fe5c002dca0 con 0x7fe5c01085d0 2024-06-10T01:04:28.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.952+0000 7fe5b6ffd700 1 -- 172.21.15.7:0/1068540911 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe5b809d9a0 con 0x7fe5c01085d0 2024-06-10T01:04:28.949 INFO:teuthology.orchestra.run.smithi007.stdout:253403070527 2024-06-10T01:04:28.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.954+0000 7fe5c7ee8700 1 -- 172.21.15.7:0/1068540911 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe5ac0a2d30 msgr2=0x7fe5ac0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:28.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.954+0000 7fe5c7ee8700 1 --2- 172.21.15.7:0/1068540911 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe5ac0a2d30 0x7fe5ac0a51f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc005950 tx=0x7fe5bc00b410 comp rx=0 tx=0).stop 2024-06-10T01:04:28.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.954+0000 7fe5c7ee8700 1 -- 172.21.15.7:0/1068540911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5c01085d0 msgr2=0x7fe5c0075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:28.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.954+0000 7fe5c7ee8700 1 --2- 172.21.15.7:0/1068540911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5c01085d0 0x7fe5c0075210 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8006070 tx=0x7fe5b8004d00 comp rx=0 tx=0).stop 2024-06-10T01:04:28.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.954+0000 7fe5c7ee8700 1 -- 172.21.15.7:0/1068540911 shutdown_connections 2024-06-10T01:04:28.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.954+0000 7fe5c7ee8700 1 --2- 172.21.15.7:0/1068540911 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe5ac0a2d30 0x7fe5ac0a51f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:28.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.954+0000 7fe5c7ee8700 1 --2- 172.21.15.7:0/1068540911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5c01085d0 0x7fe5c0075210 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:28.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.954+0000 7fe5c7ee8700 1 --2- 172.21.15.7:0/1068540911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5c0102530 0x7fe5c0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:28.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.954+0000 7fe5c7ee8700 1 --2- 172.21.15.7:0/1068540911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5c0102ef0 0x7fe5c0074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:28.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.954+0000 7fe5c7ee8700 1 -- 172.21.15.7:0/1068540911 >> 172.21.15.7:0/1068540911 conn(0x7fe5c0079640 msgr2=0x7fe5c0101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:28.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.955+0000 7fe5c7ee8700 1 -- 172.21.15.7:0/1068540911 shutdown_connections 2024-06-10T01:04:28.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:28.955+0000 7fe5c7ee8700 1 -- 172.21.15.7:0/1068540911 wait complete. 2024-06-10T01:04:29.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:29 smithi007 ceph-mon[32398]: pgmap v385: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:04:29.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:29 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:04:29.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:29 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:04:29.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:29 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1394350682' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-10T01:04:29.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:29 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:04:29.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:29 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:04:29.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:29 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:04:29.266 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:29 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1068540911' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-10T01:04:29.332 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:29 smithi078 ceph-mon[36683]: pgmap v385: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:04:29.332 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:29 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:04:29.332 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:29 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:04:29.332 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:29 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1394350682' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-10T01:04:29.332 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:29 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:04:29.332 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:29 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:04:29.332 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:29 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:04:29.332 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:29 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1068540911' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-10T01:04:29.411 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738465 got 34359738470 for osd.0 2024-06-10T01:04:29.411 DEBUG:teuthology.parallel:result is None 2024-06-10T01:04:29.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:29 smithi175 ceph-mon[36636]: pgmap v385: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:04:29.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:29 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:04:29.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:29 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:04:29.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:29 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1394350682' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-10T01:04:29.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:29 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:04:29.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:29 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:04:29.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:29 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:04:29.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:29 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1068540911' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-10T01:04:29.944 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070520 got 253403070527 for osd.8 2024-06-10T01:04:29.944 DEBUG:teuthology.parallel:result is None 2024-06-10T01:04:29.944 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-10T01:04:29.944 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-10T01:04:30.198 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:31.205 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:31 smithi175 ceph-mon[36636]: pgmap v386: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T01:04:31.331 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:31 smithi078 ceph-mon[36683]: pgmap v386: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T01:04:31.350 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:31 smithi007 ceph-mon[32398]: pgmap v386: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T01:04:31.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.904+0000 7fa35cb04700 1 -- 172.21.15.7:0/1404777850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa358107860 msgr2=0x7fa35810f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:31.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.904+0000 7fa35cb04700 1 --2- 172.21.15.7:0/1404777850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa358107860 0x7fa35810f3e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa35000b0d0 tx=0x7fa35000b3e0 comp rx=0 tx=0).stop 2024-06-10T01:04:31.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.904+0000 7fa35cb04700 1 -- 172.21.15.7:0/1404777850 shutdown_connections 2024-06-10T01:04:31.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.904+0000 7fa35cb04700 1 --2- 172.21.15.7:0/1404777850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa358107860 0x7fa35810f3e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:31.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.904+0000 7fa35cb04700 1 --2- 172.21.15.7:0/1404777850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa358102770 0x7fa358102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:31.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.904+0000 7fa35cb04700 1 --2- 172.21.15.7:0/1404777850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa358103120 0x7fa358107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:31.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.904+0000 7fa35cb04700 1 -- 172.21.15.7:0/1404777850 >> 172.21.15.7:0/1404777850 conn(0x7fa358079600 msgr2=0x7fa358079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:31.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.904+0000 7fa35cb04700 1 -- 172.21.15.7:0/1404777850 shutdown_connections 2024-06-10T01:04:31.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.904+0000 7fa35cb04700 1 -- 172.21.15.7:0/1404777850 wait complete. 2024-06-10T01:04:31.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.904+0000 7fa35cb04700 1 Processor -- start 2024-06-10T01:04:31.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.905+0000 7fa35cb04700 1 -- start start 2024-06-10T01:04:31.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.905+0000 7fa35cb04700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa358102770 0x7fa35819e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:31.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.905+0000 7fa35cb04700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa358103120 0x7fa35819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:31.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.905+0000 7fa35cb04700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa358107860 0x7fa358198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:31.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.905+0000 7fa35cb04700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa358112870 con 0x7fa358102770 2024-06-10T01:04:31.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.905+0000 7fa35cb04700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa3581126f0 con 0x7fa358107860 2024-06-10T01:04:31.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.905+0000 7fa35cb04700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa3581129f0 con 0x7fa358103120 2024-06-10T01:04:31.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.905+0000 7fa35659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa358102770 0x7fa35819e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:31.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.905+0000 7fa35659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa358102770 0x7fa35819e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55552/0 (socket says 172.21.15.7:55552) 2024-06-10T01:04:31.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.905+0000 7fa35659c700 1 -- 172.21.15.7:0/346277783 learned_addr learned my addr 172.21.15.7:0/346277783 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.905+0000 7fa355d9b700 1 --2- 172.21.15.7:0/346277783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa358103120 0x7fa35819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.905+0000 7fa356d9d700 1 --2- 172.21.15.7:0/346277783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa358107860 0x7fa358198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.905+0000 7fa35659c700 1 -- 172.21.15.7:0/346277783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa358103120 msgr2=0x7fa35819ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.905+0000 7fa35659c700 1 --2- 172.21.15.7:0/346277783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa358103120 0x7fa35819ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.905+0000 7fa35659c700 1 -- 172.21.15.7:0/346277783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa358107860 msgr2=0x7fa358198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.905+0000 7fa35659c700 1 --2- 172.21.15.7:0/346277783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa358107860 0x7fa358198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.905+0000 7fa35659c700 1 -- 172.21.15.7:0/346277783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa358199230 con 0x7fa358102770 2024-06-10T01:04:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.905+0000 7fa356d9d700 1 --2- 172.21.15.7:0/346277783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa358107860 0x7fa358198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:04:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.906+0000 7fa35659c700 1 --2- 172.21.15.7:0/346277783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa358102770 0x7fa35819e910 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa34800b530 tx=0x7fa34800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.906+0000 7fa3477fe700 1 -- 172.21.15.7:0/346277783 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa348015020 con 0x7fa358102770 2024-06-10T01:04:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.906+0000 7fa3477fe700 1 -- 172.21.15.7:0/346277783 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa348011d10 con 0x7fa358102770 2024-06-10T01:04:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.906+0000 7fa35cb04700 1 -- 172.21.15.7:0/346277783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa358199520 con 0x7fa358102770 2024-06-10T01:04:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.906+0000 7fa3477fe700 1 -- 172.21.15.7:0/346277783 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa348010e50 con 0x7fa358102770 2024-06-10T01:04:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.906+0000 7fa35cb04700 1 -- 172.21.15.7:0/346277783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3581a57e0 con 0x7fa358102770 2024-06-10T01:04:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.907+0000 7fa35cb04700 1 -- 172.21.15.7:0/346277783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa35810aaa0 con 0x7fa358102770 2024-06-10T01:04:31.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.908+0000 7fa3477fe700 1 -- 172.21.15.7:0/346277783 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fa348018020 con 0x7fa358102770 2024-06-10T01:04:31.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.909+0000 7fa3477fe700 1 --2- 172.21.15.7:0/346277783 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa3400a2ed0 0x7fa3400a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:31.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.909+0000 7fa3477fe700 1 -- 172.21.15.7:0/346277783 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fa3480cf1e0 con 0x7fa358102770 2024-06-10T01:04:31.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.909+0000 7fa355d9b700 1 --2- 172.21.15.7:0/346277783 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa3400a2ed0 0x7fa3400a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:31.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.909+0000 7fa355d9b700 1 --2- 172.21.15.7:0/346277783 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa3400a2ed0 0x7fa3400a5390 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa34c006fd0 tx=0x7fa34c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:31.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:31.913+0000 7fa3477fe700 1 -- 172.21.15.7:0/346277783 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa34801f020 con 0x7fa358102770 2024-06-10T01:04:32.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:32.119+0000 7fa35cb04700 1 -- 172.21.15.7:0/346277783 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa3580628d0 con 0x7fa3400a2ed0 2024-06-10T01:04:32.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:32.128+0000 7fa3477fe700 1 -- 172.21.15.7:0/346277783 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340648 (secure 0 0 0) 0x7fa3580628d0 con 0x7fa3400a2ed0 2024-06-10T01:04:32.131 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:04:32.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:32.131+0000 7fa35cb04700 1 -- 172.21.15.7:0/346277783 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa3400a2ed0 msgr2=0x7fa3400a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:32.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:32.131+0000 7fa35cb04700 1 --2- 172.21.15.7:0/346277783 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa3400a2ed0 0x7fa3400a5390 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa34c006fd0 tx=0x7fa34c008040 comp rx=0 tx=0).stop 2024-06-10T01:04:32.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:32.132+0000 7fa35cb04700 1 -- 172.21.15.7:0/346277783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa358102770 msgr2=0x7fa35819e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:32.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:32.132+0000 7fa35cb04700 1 --2- 172.21.15.7:0/346277783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa358102770 0x7fa35819e910 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa34800b530 tx=0x7fa34800b8f0 comp rx=0 tx=0).stop 2024-06-10T01:04:32.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:32.132+0000 7fa35cb04700 1 -- 172.21.15.7:0/346277783 shutdown_connections 2024-06-10T01:04:32.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:32.132+0000 7fa35cb04700 1 --2- 172.21.15.7:0/346277783 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa3400a2ed0 0x7fa3400a5390 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:32.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:32.132+0000 7fa35cb04700 1 --2- 172.21.15.7:0/346277783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa358107860 0x7fa358198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:32.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:32.132+0000 7fa35cb04700 1 --2- 172.21.15.7:0/346277783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa358102770 0x7fa35819e910 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:32.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:32.132+0000 7fa35cb04700 1 --2- 172.21.15.7:0/346277783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa358103120 0x7fa35819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:32.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:32.132+0000 7fa35cb04700 1 -- 172.21.15.7:0/346277783 >> 172.21.15.7:0/346277783 conn(0x7fa358079600 msgr2=0x7fa358100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:32.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:32.132+0000 7fa35cb04700 1 -- 172.21.15.7:0/346277783 shutdown_connections 2024-06-10T01:04:32.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:32.132+0000 7fa35cb04700 1 -- 172.21.15.7:0/346277783 wait complete. 2024-06-10T01:04:32.134 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-06-10T01:04:32.961 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-06-10T01:04:31.577571+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658270,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":719,"num_read_kb":479,"num_write":353,"num_write_kb":3059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":283,"ondisk_log_size":283,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362540,"kb_used_data":13316,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579284,"statfs":{"total":1151940427776,"available":1151569186816,"internally_reserved":0,"allocated":13635584,"data_stored":8787512,"data_compressed":496626,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003173"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.237314+0000","last_change":"2024-06-10T01:01:47.244728+0000","last_active":"2024-06-10T01:03:38.237314+0000","last_peered":"2024-06-10T01:03:38.237314+0000","last_clean":"2024-06-10T01:03:38.237314+0000","last_became_active":"2024-06-10T01:01:47.244198+0000","last_became_peered":"2024-06-10T01:01:47.244198+0000","last_unstale":"2024-06-10T01:03:38.237314+0000","last_undegraded":"2024-06-10T01:03:38.237314+0000","last_fullsized":"2024-06-10T01:03:38.237314+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:58:06.257952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234562+0000","last_change":"2024-06-10T01:01:47.247620+0000","last_active":"2024-06-10T01:03:38.234562+0000","last_peered":"2024-06-10T01:03:38.234562+0000","last_clean":"2024-06-10T01:03:38.234562+0000","last_became_active":"2024-06-10T01:01:47.245475+0000","last_became_peered":"2024-06-10T01:01:47.245475+0000","last_unstale":"2024-06-10T01:03:38.234562+0000","last_undegraded":"2024-06-10T01:03:38.234562+0000","last_fullsized":"2024-06-10T01:03:38.234562+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:29:09.367411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234327+0000","last_change":"2024-06-10T01:01:47.252414+0000","last_active":"2024-06-10T01:03:38.234327+0000","last_peered":"2024-06-10T01:03:38.234327+0000","last_clean":"2024-06-10T01:03:38.234327+0000","last_became_active":"2024-06-10T01:01:47.252063+0000","last_became_peered":"2024-06-10T01:01:47.252063+0000","last_unstale":"2024-06-10T01:03:38.234327+0000","last_undegraded":"2024-06-10T01:03:38.234327+0000","last_fullsized":"2024-06-10T01:03:38.234327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:54:53.966209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239632+0000","last_change":"2024-06-10T01:01:47.254875+0000","last_active":"2024-06-10T01:03:38.239632+0000","last_peered":"2024-06-10T01:03:38.239632+0000","last_clean":"2024-06-10T01:03:38.239632+0000","last_became_active":"2024-06-10T01:01:47.254542+0000","last_became_peered":"2024-06-10T01:01:47.254542+0000","last_unstale":"2024-06-10T01:03:38.239632+0000","last_undegraded":"2024-06-10T01:03:38.239632+0000","last_fullsized":"2024-06-10T01:03:38.239632+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:02:28.393133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235286+0000","last_change":"2024-06-10T01:01:47.242596+0000","last_active":"2024-06-10T01:03:38.235286+0000","last_peered":"2024-06-10T01:03:38.235286+0000","last_clean":"2024-06-10T01:03:38.235286+0000","last_became_active":"2024-06-10T01:01:47.242308+0000","last_became_peered":"2024-06-10T01:01:47.242308+0000","last_unstale":"2024-06-10T01:03:38.235286+0000","last_undegraded":"2024-06-10T01:03:38.235286+0000","last_fullsized":"2024-06-10T01:03:38.235286+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:11:26.599954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.570472+0000","last_change":"2024-06-10T01:01:47.249085+0000","last_active":"2024-06-10T01:03:38.570472+0000","last_peered":"2024-06-10T01:03:38.570472+0000","last_clean":"2024-06-10T01:03:38.570472+0000","last_became_active":"2024-06-10T01:01:47.248749+0000","last_became_peered":"2024-06-10T01:01:47.248749+0000","last_unstale":"2024-06-10T01:03:38.570472+0000","last_undegraded":"2024-06-10T01:03:38.570472+0000","last_fullsized":"2024-06-10T01:03:38.570472+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:42:48.734566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235057+0000","last_change":"2024-06-10T01:01:47.247334+0000","last_active":"2024-06-10T01:03:38.235057+0000","last_peered":"2024-06-10T01:03:38.235057+0000","last_clean":"2024-06-10T01:03:38.235057+0000","last_became_active":"2024-06-10T01:01:47.246964+0000","last_became_peered":"2024-06-10T01:01:47.246964+0000","last_unstale":"2024-06-10T01:03:38.235057+0000","last_undegraded":"2024-06-10T01:03:38.235057+0000","last_fullsized":"2024-06-10T01:03:38.235057+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:05:36.034950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.238961+0000","last_change":"2024-06-10T01:01:47.242523+0000","last_active":"2024-06-10T01:03:38.238961+0000","last_peered":"2024-06-10T01:03:38.238961+0000","last_clean":"2024-06-10T01:03:38.238961+0000","last_became_active":"2024-06-10T01:01:47.242198+0000","last_became_peered":"2024-06-10T01:01:47.242198+0000","last_unstale":"2024-06-10T01:03:38.238961+0000","last_undegraded":"2024-06-10T01:03:38.238961+0000","last_fullsized":"2024-06-10T01:03:38.238961+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:39:07.055970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771277+0000","last_change":"2024-06-10T01:01:47.245277+0000","last_active":"2024-06-10T01:03:38.771277+0000","last_peered":"2024-06-10T01:03:38.771277+0000","last_clean":"2024-06-10T01:03:38.771277+0000","last_became_active":"2024-06-10T01:01:47.244932+0000","last_became_peered":"2024-06-10T01:01:47.244932+0000","last_unstale":"2024-06-10T01:03:38.771277+0000","last_undegraded":"2024-06-10T01:03:38.771277+0000","last_fullsized":"2024-06-10T01:03:38.771277+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:56:24.423713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.570817+0000","last_change":"2024-06-10T01:01:47.255531+0000","last_active":"2024-06-10T01:03:38.570817+0000","last_peered":"2024-06-10T01:03:38.570817+0000","last_clean":"2024-06-10T01:03:38.570817+0000","last_became_active":"2024-06-10T01:01:47.255150+0000","last_became_peered":"2024-06-10T01:01:47.255150+0000","last_unstale":"2024-06-10T01:03:38.570817+0000","last_undegraded":"2024-06-10T01:03:38.570817+0000","last_fullsized":"2024-06-10T01:03:38.570817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:51:34.561396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235538+0000","last_change":"2024-06-10T01:01:47.249832+0000","last_active":"2024-06-10T01:03:38.235538+0000","last_peered":"2024-06-10T01:03:38.235538+0000","last_clean":"2024-06-10T01:03:38.235538+0000","last_became_active":"2024-06-10T01:01:47.249303+0000","last_became_peered":"2024-06-10T01:01:47.249303+0000","last_unstale":"2024-06-10T01:03:38.235538+0000","last_undegraded":"2024-06-10T01:03:38.235538+0000","last_fullsized":"2024-06-10T01:03:38.235538+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:15:21.930157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239476+0000","last_change":"2024-06-10T01:01:47.255927+0000","last_active":"2024-06-10T01:03:38.239476+0000","last_peered":"2024-06-10T01:03:38.239476+0000","last_clean":"2024-06-10T01:03:38.239476+0000","last_became_active":"2024-06-10T01:01:47.255591+0000","last_became_peered":"2024-06-10T01:01:47.255591+0000","last_unstale":"2024-06-10T01:03:38.239476+0000","last_undegraded":"2024-06-10T01:03:38.239476+0000","last_fullsized":"2024-06-10T01:03:38.239476+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:48:06.494430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771241+0000","last_change":"2024-06-10T01:02:37.372069+0000","last_active":"2024-06-10T01:03:38.771241+0000","last_peered":"2024-06-10T01:03:38.771241+0000","last_clean":"2024-06-10T01:03:38.771241+0000","last_became_active":"2024-06-10T01:02:37.371599+0000","last_became_peered":"2024-06-10T01:02:37.371599+0000","last_unstale":"2024-06-10T01:03:38.771241+0000","last_undegraded":"2024-06-10T01:03:38.771241+0000","last_fullsized":"2024-06-10T01:03:38.771241+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:54:34.091576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234211+0000","last_change":"2024-06-10T01:01:47.249553+0000","last_active":"2024-06-10T01:03:38.234211+0000","last_peered":"2024-06-10T01:03:38.234211+0000","last_clean":"2024-06-10T01:03:38.234211+0000","last_became_active":"2024-06-10T01:01:47.249202+0000","last_became_peered":"2024-06-10T01:01:47.249202+0000","last_unstale":"2024-06-10T01:03:38.234211+0000","last_undegraded":"2024-06-10T01:03:38.234211+0000","last_fullsized":"2024-06-10T01:03:38.234211+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:23:57.882264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234200+0000","last_change":"2024-06-10T01:01:47.253987+0000","last_active":"2024-06-10T01:03:38.234200+0000","last_peered":"2024-06-10T01:03:38.234200+0000","last_clean":"2024-06-10T01:03:38.234200+0000","last_became_active":"2024-06-10T01:01:47.253578+0000","last_became_peered":"2024-06-10T01:01:47.253578+0000","last_unstale":"2024-06-10T01:03:38.234200+0000","last_undegraded":"2024-06-10T01:03:38.234200+0000","last_fullsized":"2024-06-10T01:03:38.234200+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:35:30.585482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770674+0000","last_change":"2024-06-10T01:01:47.247089+0000","last_active":"2024-06-10T01:03:38.770674+0000","last_peered":"2024-06-10T01:03:38.770674+0000","last_clean":"2024-06-10T01:03:38.770674+0000","last_became_active":"2024-06-10T01:01:47.246066+0000","last_became_peered":"2024-06-10T01:01:47.246066+0000","last_unstale":"2024-06-10T01:03:38.770674+0000","last_undegraded":"2024-06-10T01:03:38.770674+0000","last_fullsized":"2024-06-10T01:03:38.770674+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:58:12.092981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.237077+0000","last_change":"2024-06-10T01:01:47.244491+0000","last_active":"2024-06-10T01:03:38.237077+0000","last_peered":"2024-06-10T01:03:38.237077+0000","last_clean":"2024-06-10T01:03:38.237077+0000","last_became_active":"2024-06-10T01:01:47.243747+0000","last_became_peered":"2024-06-10T01:01:47.243747+0000","last_unstale":"2024-06-10T01:03:38.237077+0000","last_undegraded":"2024-06-10T01:03:38.237077+0000","last_fullsized":"2024-06-10T01:03:38.237077+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:55:56.785016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.772072+0000","last_change":"2024-06-10T01:01:47.243141+0000","last_active":"2024-06-10T01:03:38.772072+0000","last_peered":"2024-06-10T01:03:38.772072+0000","last_clean":"2024-06-10T01:03:38.772072+0000","last_became_active":"2024-06-10T01:01:47.242464+0000","last_became_peered":"2024-06-10T01:01:47.242464+0000","last_unstale":"2024-06-10T01:03:38.772072+0000","last_undegraded":"2024-06-10T01:03:38.772072+0000","last_fullsized":"2024-06-10T01:03:38.772072+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:00:49.777130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.240034+0000","last_change":"2024-06-10T01:01:47.253463+0000","last_active":"2024-06-10T01:03:38.240034+0000","last_peered":"2024-06-10T01:03:38.240034+0000","last_clean":"2024-06-10T01:03:38.240034+0000","last_became_active":"2024-06-10T01:01:47.252929+0000","last_became_peered":"2024-06-10T01:01:47.252929+0000","last_unstale":"2024-06-10T01:03:38.240034+0000","last_undegraded":"2024-06-10T01:03:38.240034+0000","last_fullsized":"2024-06-10T01:03:38.240034+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:18:49.551276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239453+0000","last_change":"2024-06-10T01:01:57.279561+0000","last_active":"2024-06-10T01:03:38.239453+0000","last_peered":"2024-06-10T01:03:38.239453+0000","last_clean":"2024-06-10T01:03:38.239453+0000","last_became_active":"2024-06-10T01:01:57.279132+0000","last_became_peered":"2024-06-10T01:01:57.279132+0000","last_unstale":"2024-06-10T01:03:38.239453+0000","last_undegraded":"2024-06-10T01:03:38.239453+0000","last_fullsized":"2024-06-10T01:03:38.239453+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:55:03.497701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'9","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.772124+0000","last_change":"2024-06-10T01:01:44.006667+0000","last_active":"2024-06-10T01:03:38.772124+0000","last_peered":"2024-06-10T01:03:38.772124+0000","last_clean":"2024-06-10T01:03:38.772124+0000","last_became_active":"2024-06-10T01:01:44.006343+0000","last_became_peered":"2024-06-10T01:01:44.006343+0000","last_unstale":"2024-06-10T01:03:38.772124+0000","last_undegraded":"2024-06-10T01:03:38.772124+0000","last_fullsized":"2024-06-10T01:03:38.772124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:14:50.540084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239291+0000","last_change":"2024-06-10T01:01:47.246139+0000","last_active":"2024-06-10T01:03:38.239291+0000","last_peered":"2024-06-10T01:03:38.239291+0000","last_clean":"2024-06-10T01:03:38.239291+0000","last_became_active":"2024-06-10T01:01:47.245743+0000","last_became_peered":"2024-06-10T01:01:47.245743+0000","last_unstale":"2024-06-10T01:03:38.239291+0000","last_undegraded":"2024-06-10T01:03:38.239291+0000","last_fullsized":"2024-06-10T01:03:38.239291+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:06:39.041569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239398+0000","last_change":"2024-06-10T01:01:57.282843+0000","last_active":"2024-06-10T01:03:38.239398+0000","last_peered":"2024-06-10T01:03:38.239398+0000","last_clean":"2024-06-10T01:03:38.239398+0000","last_became_active":"2024-06-10T01:01:57.281797+0000","last_became_peered":"2024-06-10T01:01:57.281797+0000","last_unstale":"2024-06-10T01:03:38.239398+0000","last_undegraded":"2024-06-10T01:03:38.239398+0000","last_fullsized":"2024-06-10T01:03:38.239398+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:00.209628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239550+0000","last_change":"2024-06-10T01:01:44.466714+0000","last_active":"2024-06-10T01:03:38.239550+0000","last_peered":"2024-06-10T01:03:38.239550+0000","last_clean":"2024-06-10T01:03:38.239550+0000","last_became_active":"2024-06-10T01:01:44.466296+0000","last_became_peered":"2024-06-10T01:01:44.466296+0000","last_unstale":"2024-06-10T01:03:38.239550+0000","last_undegraded":"2024-06-10T01:03:38.239550+0000","last_fullsized":"2024-06-10T01:03:38.239550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:16:36.082308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234498+0000","last_change":"2024-06-10T01:01:47.244173+0000","last_active":"2024-06-10T01:03:38.234498+0000","last_peered":"2024-06-10T01:03:38.234498+0000","last_clean":"2024-06-10T01:03:38.234498+0000","last_became_active":"2024-06-10T01:01:47.243032+0000","last_became_peered":"2024-06-10T01:01:47.243032+0000","last_unstale":"2024-06-10T01:03:38.234498+0000","last_undegraded":"2024-06-10T01:03:38.234498+0000","last_fullsized":"2024-06-10T01:03:38.234498+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:14:33.965903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239057+0000","last_change":"2024-06-10T01:02:37.592535+0000","last_active":"2024-06-10T01:03:38.239057+0000","last_peered":"2024-06-10T01:03:38.239057+0000","last_clean":"2024-06-10T01:03:38.239057+0000","last_became_active":"2024-06-10T01:02:37.592159+0000","last_became_peered":"2024-06-10T01:02:37.592159+0000","last_unstale":"2024-06-10T01:03:38.239057+0000","last_undegraded":"2024-06-10T01:03:38.239057+0000","last_fullsized":"2024-06-10T01:03:38.239057+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:50:08.626485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235433+0000","last_change":"2024-06-10T01:01:44.009105+0000","last_active":"2024-06-10T01:03:38.235433+0000","last_peered":"2024-06-10T01:03:38.235433+0000","last_clean":"2024-06-10T01:03:38.235433+0000","last_became_active":"2024-06-10T01:01:44.008743+0000","last_became_peered":"2024-06-10T01:01:44.008743+0000","last_unstale":"2024-06-10T01:03:38.235433+0000","last_undegraded":"2024-06-10T01:03:38.235433+0000","last_fullsized":"2024-06-10T01:03:38.235433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:44:54.301378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770936+0000","last_change":"2024-06-10T01:01:47.249007+0000","last_active":"2024-06-10T01:03:38.770936+0000","last_peered":"2024-06-10T01:03:38.770936+0000","last_clean":"2024-06-10T01:03:38.770936+0000","last_became_active":"2024-06-10T01:01:47.245632+0000","last_became_peered":"2024-06-10T01:01:47.245632+0000","last_unstale":"2024-06-10T01:03:38.770936+0000","last_undegraded":"2024-06-10T01:03:38.770936+0000","last_fullsized":"2024-06-10T01:03:38.770936+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:29:18.916390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.238983+0000","last_change":"2024-06-10T01:01:57.283137+0000","last_active":"2024-06-10T01:03:38.238983+0000","last_peered":"2024-06-10T01:03:38.238983+0000","last_clean":"2024-06-10T01:03:38.238983+0000","last_became_active":"2024-06-10T01:01:57.282605+0000","last_became_peered":"2024-06-10T01:01:57.282605+0000","last_unstale":"2024-06-10T01:03:38.238983+0000","last_undegraded":"2024-06-10T01:03:38.238983+0000","last_fullsized":"2024-06-10T01:03:38.238983+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:26:16.967320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235734+0000","last_change":"2024-06-10T01:01:44.010674+0000","last_active":"2024-06-10T01:03:38.235734+0000","last_peered":"2024-06-10T01:03:38.235734+0000","last_clean":"2024-06-10T01:03:38.235734+0000","last_became_active":"2024-06-10T01:01:44.010291+0000","last_became_peered":"2024-06-10T01:01:44.010291+0000","last_unstale":"2024-06-10T01:03:38.235734+0000","last_undegraded":"2024-06-10T01:03:38.235734+0000","last_fullsized":"2024-06-10T01:03:38.235734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:36:18.418362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270169+0000","last_change":"2024-06-10T01:02:37.794580+0000","last_active":"2024-06-10T01:03:38.270169+0000","last_peered":"2024-06-10T01:03:38.270169+0000","last_clean":"2024-06-10T01:03:38.270169+0000","last_became_active":"2024-06-10T01:02:37.794127+0000","last_became_peered":"2024-06-10T01:02:37.794127+0000","last_unstale":"2024-06-10T01:03:38.270169+0000","last_undegraded":"2024-06-10T01:03:38.270169+0000","last_fullsized":"2024-06-10T01:03:38.270169+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:02.117871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239746+0000","last_change":"2024-06-10T01:01:57.282494+0000","last_active":"2024-06-10T01:03:38.239746+0000","last_peered":"2024-06-10T01:03:38.239746+0000","last_clean":"2024-06-10T01:03:38.239746+0000","last_became_active":"2024-06-10T01:01:57.282077+0000","last_became_peered":"2024-06-10T01:01:57.282077+0000","last_unstale":"2024-06-10T01:03:38.239746+0000","last_undegraded":"2024-06-10T01:03:38.239746+0000","last_fullsized":"2024-06-10T01:03:38.239746+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:57:41.639788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"97'5","reported_seq":144,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:25.845372+0000","last_change":"2024-06-10T01:01:44.011134+0000","last_active":"2024-06-10T01:04:25.845372+0000","last_peered":"2024-06-10T01:04:25.845372+0000","last_clean":"2024-06-10T01:04:25.845372+0000","last_became_active":"2024-06-10T01:01:44.010788+0000","last_became_peered":"2024-06-10T01:01:44.010788+0000","last_unstale":"2024-06-10T01:04:25.845372+0000","last_undegraded":"2024-06-10T01:04:25.845372+0000","last_fullsized":"2024-06-10T01:04:25.845372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:22:55.228946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235465+0000","last_change":"2024-06-10T01:01:47.248093+0000","last_active":"2024-06-10T01:03:38.235465+0000","last_peered":"2024-06-10T01:03:38.235465+0000","last_clean":"2024-06-10T01:03:38.235465+0000","last_became_active":"2024-06-10T01:01:47.247797+0000","last_became_peered":"2024-06-10T01:01:47.247797+0000","last_unstale":"2024-06-10T01:03:38.235465+0000","last_undegraded":"2024-06-10T01:03:38.235465+0000","last_fullsized":"2024-06-10T01:03:38.235465+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:45:50.159128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239671+0000","last_change":"2024-06-10T01:02:37.377039+0000","last_active":"2024-06-10T01:03:38.239671+0000","last_peered":"2024-06-10T01:03:38.239671+0000","last_clean":"2024-06-10T01:03:38.239671+0000","last_became_active":"2024-06-10T01:02:37.376332+0000","last_became_peered":"2024-06-10T01:02:37.376332+0000","last_unstale":"2024-06-10T01:03:38.239671+0000","last_undegraded":"2024-06-10T01:03:38.239671+0000","last_fullsized":"2024-06-10T01:03:38.239671+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:15.000505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,10,5],"acting":[2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.570465+0000","last_change":"2024-06-10T01:01:44.008342+0000","last_active":"2024-06-10T01:03:38.570465+0000","last_peered":"2024-06-10T01:03:38.570465+0000","last_clean":"2024-06-10T01:03:38.570465+0000","last_became_active":"2024-06-10T01:01:44.007959+0000","last_became_peered":"2024-06-10T01:01:44.007959+0000","last_unstale":"2024-06-10T01:03:38.570465+0000","last_undegraded":"2024-06-10T01:03:38.570465+0000","last_fullsized":"2024-06-10T01:03:38.570465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:36:03.143609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.269968+0000","last_change":"2024-06-10T01:01:47.247854+0000","last_active":"2024-06-10T01:03:38.269968+0000","last_peered":"2024-06-10T01:03:38.269968+0000","last_clean":"2024-06-10T01:03:38.269968+0000","last_became_active":"2024-06-10T01:01:47.247563+0000","last_became_peered":"2024-06-10T01:01:47.247563+0000","last_unstale":"2024-06-10T01:03:38.269968+0000","last_undegraded":"2024-06-10T01:03:38.269968+0000","last_fullsized":"2024-06-10T01:03:38.269968+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:32:13.024815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.237041+0000","last_change":"2024-06-10T01:01:57.282427+0000","last_active":"2024-06-10T01:03:38.237041+0000","last_peered":"2024-06-10T01:03:38.237041+0000","last_clean":"2024-06-10T01:03:38.237041+0000","last_became_active":"2024-06-10T01:01:57.281987+0000","last_became_peered":"2024-06-10T01:01:57.281987+0000","last_unstale":"2024-06-10T01:03:38.237041+0000","last_undegraded":"2024-06-10T01:03:38.237041+0000","last_fullsized":"2024-06-10T01:03:38.237041+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:22:31.736804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770902+0000","last_change":"2024-06-10T01:01:44.006567+0000","last_active":"2024-06-10T01:03:38.770902+0000","last_peered":"2024-06-10T01:03:38.770902+0000","last_clean":"2024-06-10T01:03:38.770902+0000","last_became_active":"2024-06-10T01:01:44.006216+0000","last_became_peered":"2024-06-10T01:01:44.006216+0000","last_unstale":"2024-06-10T01:03:38.770902+0000","last_undegraded":"2024-06-10T01:03:38.770902+0000","last_fullsized":"2024-06-10T01:03:38.770902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:33:55.696873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.238845+0000","last_change":"2024-06-10T01:01:47.245326+0000","last_active":"2024-06-10T01:03:38.238845+0000","last_peered":"2024-06-10T01:03:38.238845+0000","last_clean":"2024-06-10T01:03:38.238845+0000","last_became_active":"2024-06-10T01:01:47.245013+0000","last_became_peered":"2024-06-10T01:01:47.245013+0000","last_unstale":"2024-06-10T01:03:38.238845+0000","last_undegraded":"2024-06-10T01:03:38.238845+0000","last_fullsized":"2024-06-10T01:03:38.238845+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:17:56.416627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234530+0000","last_change":"2024-06-10T01:01:57.288997+0000","last_active":"2024-06-10T01:03:38.234530+0000","last_peered":"2024-06-10T01:03:38.234530+0000","last_clean":"2024-06-10T01:03:38.234530+0000","last_became_active":"2024-06-10T01:01:57.288596+0000","last_became_peered":"2024-06-10T01:01:57.288596+0000","last_unstale":"2024-06-10T01:03:38.234530+0000","last_undegraded":"2024-06-10T01:03:38.234530+0000","last_fullsized":"2024-06-10T01:03:38.234530+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:08:14.739737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.269738+0000","last_change":"2024-06-10T01:01:44.010079+0000","last_active":"2024-06-10T01:03:38.269738+0000","last_peered":"2024-06-10T01:03:38.269738+0000","last_clean":"2024-06-10T01:03:38.269738+0000","last_became_active":"2024-06-10T01:01:44.009752+0000","last_became_peered":"2024-06-10T01:01:44.009752+0000","last_unstale":"2024-06-10T01:03:38.269738+0000","last_undegraded":"2024-06-10T01:03:38.269738+0000","last_fullsized":"2024-06-10T01:03:38.269738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:29:35.660935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770966+0000","last_change":"2024-06-10T01:01:47.245015+0000","last_active":"2024-06-10T01:03:38.770966+0000","last_peered":"2024-06-10T01:03:38.770966+0000","last_clean":"2024-06-10T01:03:38.770966+0000","last_became_active":"2024-06-10T01:01:47.244595+0000","last_became_peered":"2024-06-10T01:01:47.244595+0000","last_unstale":"2024-06-10T01:03:38.770966+0000","last_undegraded":"2024-06-10T01:03:38.770966+0000","last_fullsized":"2024-06-10T01:03:38.770966+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:01:48.446526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.238926+0000","last_change":"2024-06-10T01:01:57.281656+0000","last_active":"2024-06-10T01:03:38.238926+0000","last_peered":"2024-06-10T01:03:38.238926+0000","last_clean":"2024-06-10T01:03:38.238926+0000","last_became_active":"2024-06-10T01:01:57.281295+0000","last_became_peered":"2024-06-10T01:01:57.281295+0000","last_unstale":"2024-06-10T01:03:38.238926+0000","last_undegraded":"2024-06-10T01:03:38.238926+0000","last_fullsized":"2024-06-10T01:03:38.238926+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:04:41.619864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"97'3","reported_seq":144,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:29.850140+0000","last_change":"2024-06-10T01:01:44.011941+0000","last_active":"2024-06-10T01:04:29.850140+0000","last_peered":"2024-06-10T01:04:29.850140+0000","last_clean":"2024-06-10T01:04:29.850140+0000","last_became_active":"2024-06-10T01:01:44.010156+0000","last_became_peered":"2024-06-10T01:01:44.010156+0000","last_unstale":"2024-06-10T01:04:29.850140+0000","last_undegraded":"2024-06-10T01:04:29.850140+0000","last_fullsized":"2024-06-10T01:04:29.850140+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:16:15.709544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771303+0000","last_change":"2024-06-10T01:01:47.241720+0000","last_active":"2024-06-10T01:03:38.771303+0000","last_peered":"2024-06-10T01:03:38.771303+0000","last_clean":"2024-06-10T01:03:38.771303+0000","last_became_active":"2024-06-10T01:01:47.241386+0000","last_became_peered":"2024-06-10T01:01:47.241386+0000","last_unstale":"2024-06-10T01:03:38.771303+0000","last_undegraded":"2024-06-10T01:03:38.771303+0000","last_fullsized":"2024-06-10T01:03:38.771303+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:42:01.304787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770709+0000","last_change":"2024-06-10T01:01:57.283226+0000","last_active":"2024-06-10T01:03:38.770709+0000","last_peered":"2024-06-10T01:03:38.770709+0000","last_clean":"2024-06-10T01:03:38.770709+0000","last_became_active":"2024-06-10T01:01:57.282720+0000","last_became_peered":"2024-06-10T01:01:57.282720+0000","last_unstale":"2024-06-10T01:03:38.770709+0000","last_undegraded":"2024-06-10T01:03:38.770709+0000","last_fullsized":"2024-06-10T01:03:38.770709+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:35:31.342042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235489+0000","last_change":"2024-06-10T01:01:44.009278+0000","last_active":"2024-06-10T01:03:38.235489+0000","last_peered":"2024-06-10T01:03:38.235489+0000","last_clean":"2024-06-10T01:03:38.235489+0000","last_became_active":"2024-06-10T01:01:44.008841+0000","last_became_peered":"2024-06-10T01:01:44.008841+0000","last_unstale":"2024-06-10T01:03:38.235489+0000","last_undegraded":"2024-06-10T01:03:38.235489+0000","last_fullsized":"2024-06-10T01:03:38.235489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:30:58.392080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234525+0000","last_change":"2024-06-10T01:01:47.256451+0000","last_active":"2024-06-10T01:03:38.234525+0000","last_peered":"2024-06-10T01:03:38.234525+0000","last_clean":"2024-06-10T01:03:38.234525+0000","last_became_active":"2024-06-10T01:01:47.256036+0000","last_became_peered":"2024-06-10T01:01:47.256036+0000","last_unstale":"2024-06-10T01:03:38.234525+0000","last_undegraded":"2024-06-10T01:03:38.234525+0000","last_fullsized":"2024-06-10T01:03:38.234525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:57:26.640109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239192+0000","last_change":"2024-06-10T01:02:37.794160+0000","last_active":"2024-06-10T01:03:38.239192+0000","last_peered":"2024-06-10T01:03:38.239192+0000","last_clean":"2024-06-10T01:03:38.239192+0000","last_became_active":"2024-06-10T01:02:37.793633+0000","last_became_peered":"2024-06-10T01:02:37.793633+0000","last_unstale":"2024-06-10T01:03:38.239192+0000","last_undegraded":"2024-06-10T01:03:38.239192+0000","last_fullsized":"2024-06-10T01:03:38.239192+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:48:48.083620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270125+0000","last_change":"2024-06-10T01:01:44.469179+0000","last_active":"2024-06-10T01:03:38.270125+0000","last_peered":"2024-06-10T01:03:38.270125+0000","last_clean":"2024-06-10T01:03:38.270125+0000","last_became_active":"2024-06-10T01:01:44.468669+0000","last_became_peered":"2024-06-10T01:01:44.468669+0000","last_unstale":"2024-06-10T01:03:38.270125+0000","last_undegraded":"2024-06-10T01:03:38.270125+0000","last_fullsized":"2024-06-10T01:03:38.270125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:07:51.629030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770954+0000","last_change":"2024-06-10T01:01:47.247588+0000","last_active":"2024-06-10T01:03:38.770954+0000","last_peered":"2024-06-10T01:03:38.770954+0000","last_clean":"2024-06-10T01:03:38.770954+0000","last_became_active":"2024-06-10T01:01:47.245750+0000","last_became_peered":"2024-06-10T01:01:47.245750+0000","last_unstale":"2024-06-10T01:03:38.770954+0000","last_undegraded":"2024-06-10T01:03:38.770954+0000","last_fullsized":"2024-06-10T01:03:38.770954+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:12:10.513777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771292+0000","last_change":"2024-06-10T01:01:57.283042+0000","last_active":"2024-06-10T01:03:38.771292+0000","last_peered":"2024-06-10T01:03:38.771292+0000","last_clean":"2024-06-10T01:03:38.771292+0000","last_became_active":"2024-06-10T01:01:57.282600+0000","last_became_peered":"2024-06-10T01:01:57.282600+0000","last_unstale":"2024-06-10T01:03:38.771292+0000","last_undegraded":"2024-06-10T01:03:38.771292+0000","last_fullsized":"2024-06-10T01:03:38.771292+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:06:36.969968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"97'34","reported_seq":280,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:30.851642+0000","last_change":"2024-06-10T01:01:44.011783+0000","last_active":"2024-06-10T01:04:30.851642+0000","last_peered":"2024-06-10T01:04:30.851642+0000","last_clean":"2024-06-10T01:04:30.851642+0000","last_became_active":"2024-06-10T01:01:44.011164+0000","last_became_peered":"2024-06-10T01:01:44.011164+0000","last_unstale":"2024-06-10T01:04:30.851642+0000","last_undegraded":"2024-06-10T01:04:30.851642+0000","last_fullsized":"2024-06-10T01:04:30.851642+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:16:13.642714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64058,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":201,"num_read_kb":163,"num_write":35,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235185+0000","last_change":"2024-06-10T01:01:47.250751+0000","last_active":"2024-06-10T01:03:38.235185+0000","last_peered":"2024-06-10T01:03:38.235185+0000","last_clean":"2024-06-10T01:03:38.235185+0000","last_became_active":"2024-06-10T01:01:47.250451+0000","last_became_peered":"2024-06-10T01:01:47.250451+0000","last_unstale":"2024-06-10T01:03:38.235185+0000","last_undegraded":"2024-06-10T01:03:38.235185+0000","last_fullsized":"2024-06-10T01:03:38.235185+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:22:10.121363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270382+0000","last_change":"2024-06-10T01:01:57.289687+0000","last_active":"2024-06-10T01:03:38.270382+0000","last_peered":"2024-06-10T01:03:38.270382+0000","last_clean":"2024-06-10T01:03:38.270382+0000","last_became_active":"2024-06-10T01:01:57.289274+0000","last_became_peered":"2024-06-10T01:01:57.289274+0000","last_unstale":"2024-06-10T01:03:38.270382+0000","last_undegraded":"2024-06-10T01:03:38.270382+0000","last_fullsized":"2024-06-10T01:03:38.270382+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:34:46.873974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'6","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239529+0000","last_change":"2024-06-10T01:01:44.010159+0000","last_active":"2024-06-10T01:03:38.239529+0000","last_peered":"2024-06-10T01:03:38.239529+0000","last_clean":"2024-06-10T01:03:38.239529+0000","last_became_active":"2024-06-10T01:01:44.009744+0000","last_became_peered":"2024-06-10T01:01:44.009744+0000","last_unstale":"2024-06-10T01:03:38.239529+0000","last_undegraded":"2024-06-10T01:03:38.239529+0000","last_fullsized":"2024-06-10T01:03:38.239529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:30.507613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771040+0000","last_change":"2024-06-10T01:01:47.246662+0000","last_active":"2024-06-10T01:03:38.771040+0000","last_peered":"2024-06-10T01:03:38.771040+0000","last_clean":"2024-06-10T01:03:38.771040+0000","last_became_active":"2024-06-10T01:01:47.246268+0000","last_became_peered":"2024-06-10T01:01:47.246268+0000","last_unstale":"2024-06-10T01:03:38.771040+0000","last_undegraded":"2024-06-10T01:03:38.771040+0000","last_fullsized":"2024-06-10T01:03:38.771040+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:08:38.089054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.236925+0000","last_change":"2024-06-10T01:01:57.289937+0000","last_active":"2024-06-10T01:03:38.236925+0000","last_peered":"2024-06-10T01:03:38.236925+0000","last_clean":"2024-06-10T01:03:38.236925+0000","last_became_active":"2024-06-10T01:01:57.289407+0000","last_became_peered":"2024-06-10T01:01:57.289407+0000","last_unstale":"2024-06-10T01:03:38.236925+0000","last_undegraded":"2024-06-10T01:03:38.236925+0000","last_fullsized":"2024-06-10T01:03:38.236925+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:01:48.546762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270079+0000","last_change":"2024-06-10T01:01:44.012654+0000","last_active":"2024-06-10T01:03:38.270079+0000","last_peered":"2024-06-10T01:03:38.270079+0000","last_clean":"2024-06-10T01:03:38.270079+0000","last_became_active":"2024-06-10T01:01:44.012246+0000","last_became_peered":"2024-06-10T01:01:44.012246+0000","last_unstale":"2024-06-10T01:03:38.270079+0000","last_undegraded":"2024-06-10T01:03:38.270079+0000","last_fullsized":"2024-06-10T01:03:38.270079+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:31:36.173554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235282+0000","last_change":"2024-06-10T01:01:47.250514+0000","last_active":"2024-06-10T01:03:38.235282+0000","last_peered":"2024-06-10T01:03:38.235282+0000","last_clean":"2024-06-10T01:03:38.235282+0000","last_became_active":"2024-06-10T01:01:47.249050+0000","last_became_peered":"2024-06-10T01:01:47.249050+0000","last_unstale":"2024-06-10T01:03:38.235282+0000","last_undegraded":"2024-06-10T01:03:38.235282+0000","last_fullsized":"2024-06-10T01:03:38.235282+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:51:30.789392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771928+0000","last_change":"2024-06-10T01:01:57.282389+0000","last_active":"2024-06-10T01:03:38.771928+0000","last_peered":"2024-06-10T01:03:38.771928+0000","last_clean":"2024-06-10T01:03:38.771928+0000","last_became_active":"2024-06-10T01:01:57.281857+0000","last_became_peered":"2024-06-10T01:01:57.281857+0000","last_unstale":"2024-06-10T01:03:38.771928+0000","last_undegraded":"2024-06-10T01:03:38.771928+0000","last_fullsized":"2024-06-10T01:03:38.771928+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:59:14.863020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"97'3","reported_seq":143,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:26.847151+0000","last_change":"2024-06-10T01:01:44.011331+0000","last_active":"2024-06-10T01:04:26.847151+0000","last_peered":"2024-06-10T01:04:26.847151+0000","last_clean":"2024-06-10T01:04:26.847151+0000","last_became_active":"2024-06-10T01:01:44.010968+0000","last_became_peered":"2024-06-10T01:01:44.010968+0000","last_unstale":"2024-06-10T01:04:26.847151+0000","last_undegraded":"2024-06-10T01:04:26.847151+0000","last_fullsized":"2024-06-10T01:04:26.847151+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:43:37.931225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235103+0000","last_change":"2024-06-10T01:01:47.248349+0000","last_active":"2024-06-10T01:03:38.235103+0000","last_peered":"2024-06-10T01:03:38.235103+0000","last_clean":"2024-06-10T01:03:38.235103+0000","last_became_active":"2024-06-10T01:01:47.248008+0000","last_became_peered":"2024-06-10T01:01:47.248008+0000","last_unstale":"2024-06-10T01:03:38.235103+0000","last_undegraded":"2024-06-10T01:03:38.235103+0000","last_fullsized":"2024-06-10T01:03:38.235103+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:45:20.747704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771221+0000","last_change":"2024-06-10T01:01:57.289773+0000","last_active":"2024-06-10T01:03:38.771221+0000","last_peered":"2024-06-10T01:03:38.771221+0000","last_clean":"2024-06-10T01:03:38.771221+0000","last_became_active":"2024-06-10T01:01:57.289371+0000","last_became_peered":"2024-06-10T01:01:57.289371+0000","last_unstale":"2024-06-10T01:03:38.771221+0000","last_undegraded":"2024-06-10T01:03:38.771221+0000","last_fullsized":"2024-06-10T01:03:38.771221+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:24:52.209402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'15","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.257548+0000","last_change":"2024-06-10T01:03:38.257122+0000","last_active":"2024-06-10T01:03:38.257548+0000","last_peered":"2024-06-10T01:03:38.257548+0000","last_clean":"2024-06-10T01:03:38.257548+0000","last_became_active":"2024-06-10T01:03:38.240737+0000","last_became_peered":"2024-06-10T01:03:38.240737+0000","last_unstale":"2024-06-10T01:03:38.257548+0000","last_undegraded":"2024-06-10T01:03:38.257548+0000","last_fullsized":"2024-06-10T01:03:38.257548+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:37:10.925004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.269868+0000","last_change":"2024-06-10T01:01:47.245439+0000","last_active":"2024-06-10T01:03:38.269868+0000","last_peered":"2024-06-10T01:03:38.269868+0000","last_clean":"2024-06-10T01:03:38.269868+0000","last_became_active":"2024-06-10T01:01:47.242917+0000","last_became_peered":"2024-06-10T01:01:47.242917+0000","last_unstale":"2024-06-10T01:03:38.269868+0000","last_undegraded":"2024-06-10T01:03:38.269868+0000","last_fullsized":"2024-06-10T01:03:38.269868+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:13:12.614230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.269795+0000","last_change":"2024-06-10T01:01:57.284731+0000","last_active":"2024-06-10T01:03:38.269795+0000","last_peered":"2024-06-10T01:03:38.269795+0000","last_clean":"2024-06-10T01:03:38.269795+0000","last_became_active":"2024-06-10T01:01:57.284301+0000","last_became_peered":"2024-06-10T01:01:57.284301+0000","last_unstale":"2024-06-10T01:03:38.269795+0000","last_undegraded":"2024-06-10T01:03:38.269795+0000","last_fullsized":"2024-06-10T01:03:38.269795+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:01:36.518214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235624+0000","last_change":"2024-06-10T01:01:44.010154+0000","last_active":"2024-06-10T01:03:38.235624+0000","last_peered":"2024-06-10T01:03:38.235624+0000","last_clean":"2024-06-10T01:03:38.235624+0000","last_became_active":"2024-06-10T01:01:44.009656+0000","last_became_peered":"2024-06-10T01:01:44.009656+0000","last_unstale":"2024-06-10T01:03:38.235624+0000","last_undegraded":"2024-06-10T01:03:38.235624+0000","last_fullsized":"2024-06-10T01:03:38.235624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:33:08.789946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.236902+0000","last_change":"2024-06-10T01:02:37.378484+0000","last_active":"2024-06-10T01:03:38.236902+0000","last_peered":"2024-06-10T01:03:38.236902+0000","last_clean":"2024-06-10T01:03:38.236902+0000","last_became_active":"2024-06-10T01:02:37.378092+0000","last_became_peered":"2024-06-10T01:02:37.378092+0000","last_unstale":"2024-06-10T01:03:38.236902+0000","last_undegraded":"2024-06-10T01:03:38.236902+0000","last_fullsized":"2024-06-10T01:03:38.236902+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:19:46.599235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235375+0000","last_change":"2024-06-10T01:01:57.332887+0000","last_active":"2024-06-10T01:03:38.235375+0000","last_peered":"2024-06-10T01:03:38.235375+0000","last_clean":"2024-06-10T01:03:38.235375+0000","last_became_active":"2024-06-10T01:01:57.332285+0000","last_became_peered":"2024-06-10T01:01:57.332285+0000","last_unstale":"2024-06-10T01:03:38.235375+0000","last_undegraded":"2024-06-10T01:03:38.235375+0000","last_fullsized":"2024-06-10T01:03:38.235375+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:09:12.233405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771642+0000","last_change":"2024-06-10T01:01:44.466831+0000","last_active":"2024-06-10T01:03:38.771642+0000","last_peered":"2024-06-10T01:03:38.771642+0000","last_clean":"2024-06-10T01:03:38.771642+0000","last_became_active":"2024-06-10T01:01:44.466484+0000","last_became_peered":"2024-06-10T01:01:44.466484+0000","last_unstale":"2024-06-10T01:03:38.771642+0000","last_undegraded":"2024-06-10T01:03:38.771642+0000","last_fullsized":"2024-06-10T01:03:38.771642+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:19:28.385504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.570599+0000","last_change":"2024-06-10T01:01:47.252399+0000","last_active":"2024-06-10T01:03:38.570599+0000","last_peered":"2024-06-10T01:03:38.570599+0000","last_clean":"2024-06-10T01:03:38.570599+0000","last_became_active":"2024-06-10T01:01:47.252069+0000","last_became_peered":"2024-06-10T01:01:47.252069+0000","last_unstale":"2024-06-10T01:03:38.570599+0000","last_undegraded":"2024-06-10T01:03:38.570599+0000","last_fullsized":"2024-06-10T01:03:38.570599+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:02:02.291174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771406+0000","last_change":"2024-06-10T01:01:57.285670+0000","last_active":"2024-06-10T01:03:38.771406+0000","last_peered":"2024-06-10T01:03:38.771406+0000","last_clean":"2024-06-10T01:03:38.771406+0000","last_became_active":"2024-06-10T01:01:57.285281+0000","last_became_peered":"2024-06-10T01:01:57.285281+0000","last_unstale":"2024-06-10T01:03:38.771406+0000","last_undegraded":"2024-06-10T01:03:38.771406+0000","last_fullsized":"2024-06-10T01:03:38.771406+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:58.956934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235189+0000","last_change":"2024-06-10T01:01:44.008013+0000","last_active":"2024-06-10T01:03:38.235189+0000","last_peered":"2024-06-10T01:03:38.235189+0000","last_clean":"2024-06-10T01:03:38.235189+0000","last_became_active":"2024-06-10T01:01:44.007591+0000","last_became_peered":"2024-06-10T01:01:44.007591+0000","last_unstale":"2024-06-10T01:03:38.235189+0000","last_undegraded":"2024-06-10T01:03:38.235189+0000","last_fullsized":"2024-06-10T01:03:38.235189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:18:20.862340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.570439+0000","last_change":"2024-06-10T01:01:47.254756+0000","last_active":"2024-06-10T01:03:38.570439+0000","last_peered":"2024-06-10T01:03:38.570439+0000","last_clean":"2024-06-10T01:03:38.570439+0000","last_became_active":"2024-06-10T01:01:47.254418+0000","last_became_peered":"2024-06-10T01:01:47.254418+0000","last_unstale":"2024-06-10T01:03:38.570439+0000","last_undegraded":"2024-06-10T01:03:38.570439+0000","last_fullsized":"2024-06-10T01:03:38.570439+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:46:31.452539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770995+0000","last_change":"2024-06-10T01:01:57.332792+0000","last_active":"2024-06-10T01:03:38.770995+0000","last_peered":"2024-06-10T01:03:38.770995+0000","last_clean":"2024-06-10T01:03:38.770995+0000","last_became_active":"2024-06-10T01:01:57.332331+0000","last_became_peered":"2024-06-10T01:01:57.332331+0000","last_unstale":"2024-06-10T01:03:38.770995+0000","last_undegraded":"2024-06-10T01:03:38.770995+0000","last_fullsized":"2024-06-10T01:03:38.770995+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:46:41.419807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270036+0000","last_change":"2024-06-10T01:01:47.247857+0000","last_active":"2024-06-10T01:03:38.270036+0000","last_peered":"2024-06-10T01:03:38.270036+0000","last_clean":"2024-06-10T01:03:38.270036+0000","last_became_active":"2024-06-10T01:01:47.247552+0000","last_became_peered":"2024-06-10T01:01:47.247552+0000","last_unstale":"2024-06-10T01:03:38.270036+0000","last_undegraded":"2024-06-10T01:03:38.270036+0000","last_fullsized":"2024-06-10T01:03:38.270036+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:29:40.521887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"97'4","reported_seq":143,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:27.848040+0000","last_change":"2024-06-10T01:01:44.466631+0000","last_active":"2024-06-10T01:04:27.848040+0000","last_peered":"2024-06-10T01:04:27.848040+0000","last_clean":"2024-06-10T01:04:27.848040+0000","last_became_active":"2024-06-10T01:01:44.466280+0000","last_became_peered":"2024-06-10T01:01:44.466280+0000","last_unstale":"2024-06-10T01:04:27.848040+0000","last_undegraded":"2024-06-10T01:04:27.848040+0000","last_fullsized":"2024-06-10T01:04:27.848040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:09:33.658112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235365+0000","last_change":"2024-06-10T01:01:57.281973+0000","last_active":"2024-06-10T01:03:38.235365+0000","last_peered":"2024-06-10T01:03:38.235365+0000","last_clean":"2024-06-10T01:03:38.235365+0000","last_became_active":"2024-06-10T01:01:57.281386+0000","last_became_peered":"2024-06-10T01:01:57.281386+0000","last_unstale":"2024-06-10T01:03:38.235365+0000","last_undegraded":"2024-06-10T01:03:38.235365+0000","last_fullsized":"2024-06-10T01:03:38.235365+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:10:28.236250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239023+0000","last_change":"2024-06-10T01:01:47.237436+0000","last_active":"2024-06-10T01:03:38.239023+0000","last_peered":"2024-06-10T01:03:38.239023+0000","last_clean":"2024-06-10T01:03:38.239023+0000","last_became_active":"2024-06-10T01:01:47.237099+0000","last_became_peered":"2024-06-10T01:01:47.237099+0000","last_unstale":"2024-06-10T01:03:38.239023+0000","last_undegraded":"2024-06-10T01:03:38.239023+0000","last_fullsized":"2024-06-10T01:03:38.239023+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:34:53.838877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'3","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239178+0000","last_change":"2024-06-10T01:01:44.010655+0000","last_active":"2024-06-10T01:03:38.239178+0000","last_peered":"2024-06-10T01:03:38.239178+0000","last_clean":"2024-06-10T01:03:38.239178+0000","last_became_active":"2024-06-10T01:01:44.009918+0000","last_became_peered":"2024-06-10T01:01:44.009918+0000","last_unstale":"2024-06-10T01:03:38.239178+0000","last_undegraded":"2024-06-10T01:03:38.239178+0000","last_fullsized":"2024-06-10T01:03:38.239178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:47:32.052650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":366,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234835+0000","last_change":"2024-06-10T01:00:23.822447+0000","last_active":"2024-06-10T01:03:38.234835+0000","last_peered":"2024-06-10T01:03:38.234835+0000","last_clean":"2024-06-10T01:03:38.234835+0000","last_became_active":"2024-06-10T01:00:23.821966+0000","last_became_peered":"2024-06-10T01:00:23.821966+0000","last_unstale":"2024-06-10T01:03:38.234835+0000","last_undegraded":"2024-06-10T01:03:38.234835+0000","last_fullsized":"2024-06-10T01:03:38.234835+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:56:59.964661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:56:59.964661+0000","last_clean_scrub_stamp":"2024-06-10T00:56:59.964661+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:03:43.732875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270272+0000","last_change":"2024-06-10T01:01:57.332980+0000","last_active":"2024-06-10T01:03:38.270272+0000","last_peered":"2024-06-10T01:03:38.270272+0000","last_clean":"2024-06-10T01:03:38.270272+0000","last_became_active":"2024-06-10T01:01:57.332437+0000","last_became_peered":"2024-06-10T01:01:57.332437+0000","last_unstale":"2024-06-10T01:03:38.270272+0000","last_undegraded":"2024-06-10T01:03:38.270272+0000","last_fullsized":"2024-06-10T01:03:38.270272+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:27:35.029079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770659+0000","last_change":"2024-06-10T01:01:47.255926+0000","last_active":"2024-06-10T01:03:38.770659+0000","last_peered":"2024-06-10T01:03:38.770659+0000","last_clean":"2024-06-10T01:03:38.770659+0000","last_became_active":"2024-06-10T01:01:47.255571+0000","last_became_peered":"2024-06-10T01:01:47.255571+0000","last_unstale":"2024-06-10T01:03:38.770659+0000","last_undegraded":"2024-06-10T01:03:38.770659+0000","last_fullsized":"2024-06-10T01:03:38.770659+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:05:22.084037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234997+0000","last_change":"2024-06-10T01:01:44.007509+0000","last_active":"2024-06-10T01:03:38.234997+0000","last_peered":"2024-06-10T01:03:38.234997+0000","last_clean":"2024-06-10T01:03:38.234997+0000","last_became_active":"2024-06-10T01:01:44.007178+0000","last_became_peered":"2024-06-10T01:01:44.007178+0000","last_unstale":"2024-06-10T01:03:38.234997+0000","last_undegraded":"2024-06-10T01:03:38.234997+0000","last_fullsized":"2024-06-10T01:03:38.234997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:02:52.791886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.236790+0000","last_change":"2024-06-10T01:01:57.285626+0000","last_active":"2024-06-10T01:03:38.236790+0000","last_peered":"2024-06-10T01:03:38.236790+0000","last_clean":"2024-06-10T01:03:38.236790+0000","last_became_active":"2024-06-10T01:01:57.285098+0000","last_became_peered":"2024-06-10T01:01:57.285098+0000","last_unstale":"2024-06-10T01:03:38.236790+0000","last_undegraded":"2024-06-10T01:03:38.236790+0000","last_fullsized":"2024-06-10T01:03:38.236790+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:13:49.881770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235332+0000","last_change":"2024-06-10T01:01:44.469028+0000","last_active":"2024-06-10T01:03:38.235332+0000","last_peered":"2024-06-10T01:03:38.235332+0000","last_clean":"2024-06-10T01:03:38.235332+0000","last_became_active":"2024-06-10T01:01:44.468570+0000","last_became_peered":"2024-06-10T01:01:44.468570+0000","last_unstale":"2024-06-10T01:03:38.235332+0000","last_undegraded":"2024-06-10T01:03:38.235332+0000","last_fullsized":"2024-06-10T01:03:38.235332+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:12:11.287959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771533+0000","last_change":"2024-06-10T01:01:47.253414+0000","last_active":"2024-06-10T01:03:38.771533+0000","last_peered":"2024-06-10T01:03:38.771533+0000","last_clean":"2024-06-10T01:03:38.771533+0000","last_became_active":"2024-06-10T01:01:47.253060+0000","last_became_peered":"2024-06-10T01:01:47.253060+0000","last_unstale":"2024-06-10T01:03:38.771533+0000","last_undegraded":"2024-06-10T01:03:38.771533+0000","last_fullsized":"2024-06-10T01:03:38.771533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:24:55.534164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.238849+0000","last_change":"2024-06-10T01:01:57.332756+0000","last_active":"2024-06-10T01:03:38.238849+0000","last_peered":"2024-06-10T01:03:38.238849+0000","last_clean":"2024-06-10T01:03:38.238849+0000","last_became_active":"2024-06-10T01:01:57.332361+0000","last_became_peered":"2024-06-10T01:01:57.332361+0000","last_unstale":"2024-06-10T01:03:38.238849+0000","last_undegraded":"2024-06-10T01:03:38.238849+0000","last_fullsized":"2024-06-10T01:03:38.238849+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:02:28.275407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"97'12","reported_seq":57,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.772073+0000","last_change":"2024-06-10T01:01:44.005164+0000","last_active":"2024-06-10T01:03:38.772073+0000","last_peered":"2024-06-10T01:03:38.772073+0000","last_clean":"2024-06-10T01:03:38.772073+0000","last_became_active":"2024-06-10T01:01:44.004816+0000","last_became_peered":"2024-06-10T01:01:44.004816+0000","last_unstale":"2024-06-10T01:03:38.772073+0000","last_undegraded":"2024-06-10T01:03:38.772073+0000","last_fullsized":"2024-06-10T01:03:38.772073+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:11:56.385267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239548+0000","last_change":"2024-06-10T01:01:47.247951+0000","last_active":"2024-06-10T01:03:38.239548+0000","last_peered":"2024-06-10T01:03:38.239548+0000","last_clean":"2024-06-10T01:03:38.239548+0000","last_became_active":"2024-06-10T01:01:47.246996+0000","last_became_peered":"2024-06-10T01:01:47.246996+0000","last_unstale":"2024-06-10T01:03:38.239548+0000","last_undegraded":"2024-06-10T01:03:38.239548+0000","last_fullsized":"2024-06-10T01:03:38.239548+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:39:18.167021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239228+0000","last_change":"2024-06-10T01:01:57.278130+0000","last_active":"2024-06-10T01:03:38.239228+0000","last_peered":"2024-06-10T01:03:38.239228+0000","last_clean":"2024-06-10T01:03:38.239228+0000","last_became_active":"2024-06-10T01:01:57.277697+0000","last_became_peered":"2024-06-10T01:01:57.277697+0000","last_unstale":"2024-06-10T01:03:38.239228+0000","last_undegraded":"2024-06-10T01:03:38.239228+0000","last_fullsized":"2024-06-10T01:03:38.239228+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:17:01.662152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239618+0000","last_change":"2024-06-10T01:01:44.553176+0000","last_active":"2024-06-10T01:03:38.239618+0000","last_peered":"2024-06-10T01:03:38.239618+0000","last_clean":"2024-06-10T01:03:38.239618+0000","last_became_active":"2024-06-10T01:01:44.552797+0000","last_became_peered":"2024-06-10T01:01:44.552797+0000","last_unstale":"2024-06-10T01:03:38.239618+0000","last_undegraded":"2024-06-10T01:03:38.239618+0000","last_fullsized":"2024-06-10T01:03:38.239618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:21:13.960304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234635+0000","last_change":"2024-06-10T01:01:47.255796+0000","last_active":"2024-06-10T01:03:38.234635+0000","last_peered":"2024-06-10T01:03:38.234635+0000","last_clean":"2024-06-10T01:03:38.234635+0000","last_became_active":"2024-06-10T01:01:47.255495+0000","last_became_peered":"2024-06-10T01:01:47.255495+0000","last_unstale":"2024-06-10T01:03:38.234635+0000","last_undegraded":"2024-06-10T01:03:38.234635+0000","last_fullsized":"2024-06-10T01:03:38.234635+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:06:42.209005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239491+0000","last_change":"2024-06-10T01:01:57.332869+0000","last_active":"2024-06-10T01:03:38.239491+0000","last_peered":"2024-06-10T01:03:38.239491+0000","last_clean":"2024-06-10T01:03:38.239491+0000","last_became_active":"2024-06-10T01:01:57.332391+0000","last_became_peered":"2024-06-10T01:01:57.332391+0000","last_unstale":"2024-06-10T01:03:38.239491+0000","last_undegraded":"2024-06-10T01:03:38.239491+0000","last_fullsized":"2024-06-10T01:03:38.239491+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:58:31.443626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770788+0000","last_change":"2024-06-10T01:01:44.466861+0000","last_active":"2024-06-10T01:03:38.770788+0000","last_peered":"2024-06-10T01:03:38.770788+0000","last_clean":"2024-06-10T01:03:38.770788+0000","last_became_active":"2024-06-10T01:01:44.466553+0000","last_became_peered":"2024-06-10T01:01:44.466553+0000","last_unstale":"2024-06-10T01:03:38.770788+0000","last_undegraded":"2024-06-10T01:03:38.770788+0000","last_fullsized":"2024-06-10T01:03:38.770788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:55:52.545594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234920+0000","last_change":"2024-06-10T01:01:47.244972+0000","last_active":"2024-06-10T01:03:38.234920+0000","last_peered":"2024-06-10T01:03:38.234920+0000","last_clean":"2024-06-10T01:03:38.234920+0000","last_became_active":"2024-06-10T01:01:47.244649+0000","last_became_peered":"2024-06-10T01:01:47.244649+0000","last_unstale":"2024-06-10T01:03:38.234920+0000","last_undegraded":"2024-06-10T01:03:38.234920+0000","last_fullsized":"2024-06-10T01:03:38.234920+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:10:46.909200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235402+0000","last_change":"2024-06-10T01:02:37.592380+0000","last_active":"2024-06-10T01:03:38.235402+0000","last_peered":"2024-06-10T01:03:38.235402+0000","last_clean":"2024-06-10T01:03:38.235402+0000","last_became_active":"2024-06-10T01:02:37.591856+0000","last_became_peered":"2024-06-10T01:02:37.591856+0000","last_unstale":"2024-06-10T01:03:38.235402+0000","last_undegraded":"2024-06-10T01:03:38.235402+0000","last_fullsized":"2024-06-10T01:03:38.235402+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:00:25.717192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239006+0000","last_change":"2024-06-10T01:01:44.008116+0000","last_active":"2024-06-10T01:03:38.239006+0000","last_peered":"2024-06-10T01:03:38.239006+0000","last_clean":"2024-06-10T01:03:38.239006+0000","last_became_active":"2024-06-10T01:01:44.007778+0000","last_became_peered":"2024-06-10T01:01:44.007778+0000","last_unstale":"2024-06-10T01:03:38.239006+0000","last_undegraded":"2024-06-10T01:03:38.239006+0000","last_fullsized":"2024-06-10T01:03:38.239006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:18:57.267296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.237657+0000","last_change":"2024-06-10T01:01:47.252833+0000","last_active":"2024-06-10T01:03:38.237657+0000","last_peered":"2024-06-10T01:03:38.237657+0000","last_clean":"2024-06-10T01:03:38.237657+0000","last_became_active":"2024-06-10T01:01:47.252326+0000","last_became_peered":"2024-06-10T01:01:47.252326+0000","last_unstale":"2024-06-10T01:03:38.237657+0000","last_undegraded":"2024-06-10T01:03:38.237657+0000","last_fullsized":"2024-06-10T01:03:38.237657+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:00:42.673838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239284+0000","last_change":"2024-06-10T01:01:57.275975+0000","last_active":"2024-06-10T01:03:38.239284+0000","last_peered":"2024-06-10T01:03:38.239284+0000","last_clean":"2024-06-10T01:03:38.239284+0000","last_became_active":"2024-06-10T01:01:57.275408+0000","last_became_peered":"2024-06-10T01:01:57.275408+0000","last_unstale":"2024-06-10T01:03:38.239284+0000","last_undegraded":"2024-06-10T01:03:38.239284+0000","last_fullsized":"2024-06-10T01:03:38.239284+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:12:42.376636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'1","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.243874+0000","last_change":"2024-06-10T01:03:38.243422+0000","last_active":"2024-06-10T01:03:38.243874+0000","last_peered":"2024-06-10T01:03:38.243874+0000","last_clean":"2024-06-10T01:03:38.243874+0000","last_became_active":"2024-06-10T01:03:38.239916+0000","last_became_peered":"2024-06-10T01:03:38.239916+0000","last_unstale":"2024-06-10T01:03:38.243874+0000","last_undegraded":"2024-06-10T01:03:38.243874+0000","last_fullsized":"2024-06-10T01:03:38.243874+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:30:01.399956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239179+0000","last_change":"2024-06-10T01:01:47.243154+0000","last_active":"2024-06-10T01:03:38.239179+0000","last_peered":"2024-06-10T01:03:38.239179+0000","last_clean":"2024-06-10T01:03:38.239179+0000","last_became_active":"2024-06-10T01:01:47.242851+0000","last_became_peered":"2024-06-10T01:01:47.242851+0000","last_unstale":"2024-06-10T01:03:38.239179+0000","last_undegraded":"2024-06-10T01:03:38.239179+0000","last_fullsized":"2024-06-10T01:03:38.239179+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:41:05.857777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239584+0000","last_change":"2024-06-10T01:01:57.333613+0000","last_active":"2024-06-10T01:03:38.239584+0000","last_peered":"2024-06-10T01:03:38.239584+0000","last_clean":"2024-06-10T01:03:38.239584+0000","last_became_active":"2024-06-10T01:01:57.333019+0000","last_became_peered":"2024-06-10T01:01:57.333019+0000","last_unstale":"2024-06-10T01:03:38.239584+0000","last_undegraded":"2024-06-10T01:03:38.239584+0000","last_fullsized":"2024-06-10T01:03:38.239584+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:11.912885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235303+0000","last_change":"2024-06-10T01:01:44.008221+0000","last_active":"2024-06-10T01:03:38.235303+0000","last_peered":"2024-06-10T01:03:38.235303+0000","last_clean":"2024-06-10T01:03:38.235303+0000","last_became_active":"2024-06-10T01:01:44.007880+0000","last_became_peered":"2024-06-10T01:01:44.007880+0000","last_unstale":"2024-06-10T01:03:38.235303+0000","last_undegraded":"2024-06-10T01:03:38.235303+0000","last_fullsized":"2024-06-10T01:03:38.235303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:40:33.162925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235307+0000","last_change":"2024-06-10T01:01:47.254902+0000","last_active":"2024-06-10T01:03:38.235307+0000","last_peered":"2024-06-10T01:03:38.235307+0000","last_clean":"2024-06-10T01:03:38.235307+0000","last_became_active":"2024-06-10T01:01:47.254555+0000","last_became_peered":"2024-06-10T01:01:47.254555+0000","last_unstale":"2024-06-10T01:03:38.235307+0000","last_undegraded":"2024-06-10T01:03:38.235307+0000","last_fullsized":"2024-06-10T01:03:38.235307+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:03:17.571001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770786+0000","last_change":"2024-06-10T01:01:57.284491+0000","last_active":"2024-06-10T01:03:38.770786+0000","last_peered":"2024-06-10T01:03:38.770786+0000","last_clean":"2024-06-10T01:03:38.770786+0000","last_became_active":"2024-06-10T01:01:57.283795+0000","last_became_peered":"2024-06-10T01:01:57.283795+0000","last_unstale":"2024-06-10T01:03:38.770786+0000","last_undegraded":"2024-06-10T01:03:38.770786+0000","last_fullsized":"2024-06-10T01:03:38.770786+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:49:00.381664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239889+0000","last_change":"2024-06-10T01:01:44.553159+0000","last_active":"2024-06-10T01:03:38.239889+0000","last_peered":"2024-06-10T01:03:38.239889+0000","last_clean":"2024-06-10T01:03:38.239889+0000","last_became_active":"2024-06-10T01:01:44.552759+0000","last_became_peered":"2024-06-10T01:01:44.552759+0000","last_unstale":"2024-06-10T01:03:38.239889+0000","last_undegraded":"2024-06-10T01:03:38.239889+0000","last_fullsized":"2024-06-10T01:03:38.239889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:19:00.110790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235665+0000","last_change":"2024-06-10T01:01:47.247803+0000","last_active":"2024-06-10T01:03:38.235665+0000","last_peered":"2024-06-10T01:03:38.235665+0000","last_clean":"2024-06-10T01:03:38.235665+0000","last_became_active":"2024-06-10T01:01:47.246715+0000","last_became_peered":"2024-06-10T01:01:47.246715+0000","last_unstale":"2024-06-10T01:03:38.235665+0000","last_undegraded":"2024-06-10T01:03:38.235665+0000","last_fullsized":"2024-06-10T01:03:38.235665+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:22.429002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234192+0000","last_change":"2024-06-10T01:01:57.287668+0000","last_active":"2024-06-10T01:03:38.234192+0000","last_peered":"2024-06-10T01:03:38.234192+0000","last_clean":"2024-06-10T01:03:38.234192+0000","last_became_active":"2024-06-10T01:01:57.287060+0000","last_became_peered":"2024-06-10T01:01:57.287060+0000","last_unstale":"2024-06-10T01:03:38.234192+0000","last_undegraded":"2024-06-10T01:03:38.234192+0000","last_fullsized":"2024-06-10T01:03:38.234192+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:04:35.883613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270251+0000","last_change":"2024-06-10T01:01:44.013051+0000","last_active":"2024-06-10T01:03:38.270251+0000","last_peered":"2024-06-10T01:03:38.270251+0000","last_clean":"2024-06-10T01:03:38.270251+0000","last_became_active":"2024-06-10T01:01:44.012667+0000","last_became_peered":"2024-06-10T01:01:44.012667+0000","last_unstale":"2024-06-10T01:03:38.270251+0000","last_undegraded":"2024-06-10T01:03:38.270251+0000","last_fullsized":"2024-06-10T01:03:38.270251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:10:49.650082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235239+0000","last_change":"2024-06-10T01:01:47.251209+0000","last_active":"2024-06-10T01:03:38.235239+0000","last_peered":"2024-06-10T01:03:38.235239+0000","last_clean":"2024-06-10T01:03:38.235239+0000","last_became_active":"2024-06-10T01:01:47.250858+0000","last_became_peered":"2024-06-10T01:01:47.250858+0000","last_unstale":"2024-06-10T01:03:38.235239+0000","last_undegraded":"2024-06-10T01:03:38.235239+0000","last_fullsized":"2024-06-10T01:03:38.235239+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:47.272727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.269600+0000","last_change":"2024-06-10T01:01:57.284187+0000","last_active":"2024-06-10T01:03:38.269600+0000","last_peered":"2024-06-10T01:03:38.269600+0000","last_clean":"2024-06-10T01:03:38.269600+0000","last_became_active":"2024-06-10T01:01:57.283739+0000","last_became_peered":"2024-06-10T01:01:57.283739+0000","last_unstale":"2024-06-10T01:03:38.269600+0000","last_undegraded":"2024-06-10T01:03:38.269600+0000","last_fullsized":"2024-06-10T01:03:38.269600+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:13:34.630708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771085+0000","last_change":"2024-06-10T01:01:44.005702+0000","last_active":"2024-06-10T01:03:38.771085+0000","last_peered":"2024-06-10T01:03:38.771085+0000","last_clean":"2024-06-10T01:03:38.771085+0000","last_became_active":"2024-06-10T01:01:44.005354+0000","last_became_peered":"2024-06-10T01:01:44.005354+0000","last_unstale":"2024-06-10T01:03:38.771085+0000","last_undegraded":"2024-06-10T01:03:38.771085+0000","last_fullsized":"2024-06-10T01:03:38.771085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:51:14.819688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235610+0000","last_change":"2024-06-10T01:01:47.250325+0000","last_active":"2024-06-10T01:03:38.235610+0000","last_peered":"2024-06-10T01:03:38.235610+0000","last_clean":"2024-06-10T01:03:38.235610+0000","last_became_active":"2024-06-10T01:01:47.249591+0000","last_became_peered":"2024-06-10T01:01:47.249591+0000","last_unstale":"2024-06-10T01:03:38.235610+0000","last_undegraded":"2024-06-10T01:03:38.235610+0000","last_fullsized":"2024-06-10T01:03:38.235610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:17:16.268717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.237787+0000","last_change":"2024-06-10T01:01:47.253449+0000","last_active":"2024-06-10T01:03:38.237787+0000","last_peered":"2024-06-10T01:03:38.237787+0000","last_clean":"2024-06-10T01:03:38.237787+0000","last_became_active":"2024-06-10T01:01:47.253144+0000","last_became_peered":"2024-06-10T01:01:47.253144+0000","last_unstale":"2024-06-10T01:03:38.237787+0000","last_undegraded":"2024-06-10T01:03:38.237787+0000","last_fullsized":"2024-06-10T01:03:38.237787+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:32:22.353409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239727+0000","last_change":"2024-06-10T01:01:47.258241+0000","last_active":"2024-06-10T01:03:38.239727+0000","last_peered":"2024-06-10T01:03:38.239727+0000","last_clean":"2024-06-10T01:03:38.239727+0000","last_became_active":"2024-06-10T01:01:47.257823+0000","last_became_peered":"2024-06-10T01:01:47.257823+0000","last_unstale":"2024-06-10T01:03:38.239727+0000","last_undegraded":"2024-06-10T01:03:38.239727+0000","last_fullsized":"2024-06-10T01:03:38.239727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:11:23.652479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239265+0000","last_change":"2024-06-10T01:01:47.256218+0000","last_active":"2024-06-10T01:03:38.239265+0000","last_peered":"2024-06-10T01:03:38.239265+0000","last_clean":"2024-06-10T01:03:38.239265+0000","last_became_active":"2024-06-10T01:01:47.255950+0000","last_became_peered":"2024-06-10T01:01:47.255950+0000","last_unstale":"2024-06-10T01:03:38.239265+0000","last_undegraded":"2024-06-10T01:03:38.239265+0000","last_fullsized":"2024-06-10T01:03:38.239265+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:59:37.283646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234361+0000","last_change":"2024-06-10T01:01:47.253109+0000","last_active":"2024-06-10T01:03:38.234361+0000","last_peered":"2024-06-10T01:03:38.234361+0000","last_clean":"2024-06-10T01:03:38.234361+0000","last_became_active":"2024-06-10T01:01:47.252767+0000","last_became_peered":"2024-06-10T01:01:47.252767+0000","last_unstale":"2024-06-10T01:03:38.234361+0000","last_undegraded":"2024-06-10T01:03:38.234361+0000","last_fullsized":"2024-06-10T01:03:38.234361+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:57:13.177230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771539+0000","last_change":"2024-06-10T01:02:37.375094+0000","last_active":"2024-06-10T01:03:38.771539+0000","last_peered":"2024-06-10T01:03:38.771539+0000","last_clean":"2024-06-10T01:03:38.771539+0000","last_became_active":"2024-06-10T01:02:37.374527+0000","last_became_peered":"2024-06-10T01:02:37.374527+0000","last_unstale":"2024-06-10T01:03:38.771539+0000","last_undegraded":"2024-06-10T01:03:38.771539+0000","last_fullsized":"2024-06-10T01:03:38.771539+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:34:41.205816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.237368+0000","last_change":"2024-06-10T01:01:47.250243+0000","last_active":"2024-06-10T01:03:38.237368+0000","last_peered":"2024-06-10T01:03:38.237368+0000","last_clean":"2024-06-10T01:03:38.237368+0000","last_became_active":"2024-06-10T01:01:47.249907+0000","last_became_peered":"2024-06-10T01:01:47.249907+0000","last_unstale":"2024-06-10T01:03:38.237368+0000","last_undegraded":"2024-06-10T01:03:38.237368+0000","last_fullsized":"2024-06-10T01:03:38.237368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:31:32.933660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239994+0000","last_change":"2024-06-10T01:01:47.254173+0000","last_active":"2024-06-10T01:03:38.239994+0000","last_peered":"2024-06-10T01:03:38.239994+0000","last_clean":"2024-06-10T01:03:38.239994+0000","last_became_active":"2024-06-10T01:01:47.253796+0000","last_became_peered":"2024-06-10T01:01:47.253796+0000","last_unstale":"2024-06-10T01:03:38.239994+0000","last_undegraded":"2024-06-10T01:03:38.239994+0000","last_fullsized":"2024-06-10T01:03:38.239994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:41:12.199114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239585+0000","last_change":"2024-06-10T01:01:47.253920+0000","last_active":"2024-06-10T01:03:38.239585+0000","last_peered":"2024-06-10T01:03:38.239585+0000","last_clean":"2024-06-10T01:03:38.239585+0000","last_became_active":"2024-06-10T01:01:47.253288+0000","last_became_peered":"2024-06-10T01:01:47.253288+0000","last_unstale":"2024-06-10T01:03:38.239585+0000","last_undegraded":"2024-06-10T01:03:38.239585+0000","last_fullsized":"2024-06-10T01:03:38.239585+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:41:20.406263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.269809+0000","last_change":"2024-06-10T01:01:47.240772+0000","last_active":"2024-06-10T01:03:38.269809+0000","last_peered":"2024-06-10T01:03:38.269809+0000","last_clean":"2024-06-10T01:03:38.269809+0000","last_became_active":"2024-06-10T01:01:47.238258+0000","last_became_peered":"2024-06-10T01:01:47.238258+0000","last_unstale":"2024-06-10T01:03:38.269809+0000","last_undegraded":"2024-06-10T01:03:38.269809+0000","last_fullsized":"2024-06-10T01:03:38.269809+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:55:25.211434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770662+0000","last_change":"2024-06-10T01:02:37.376724+0000","last_active":"2024-06-10T01:03:38.770662+0000","last_peered":"2024-06-10T01:03:38.770662+0000","last_clean":"2024-06-10T01:03:38.770662+0000","last_became_active":"2024-06-10T01:02:37.376329+0000","last_became_peered":"2024-06-10T01:02:37.376329+0000","last_unstale":"2024-06-10T01:03:38.770662+0000","last_undegraded":"2024-06-10T01:03:38.770662+0000","last_fullsized":"2024-06-10T01:03:38.770662+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:38:21.837958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771019+0000","last_change":"2024-06-10T01:01:47.244463+0000","last_active":"2024-06-10T01:03:38.771019+0000","last_peered":"2024-06-10T01:03:38.771019+0000","last_clean":"2024-06-10T01:03:38.771019+0000","last_became_active":"2024-06-10T01:01:47.244114+0000","last_became_peered":"2024-06-10T01:01:47.244114+0000","last_unstale":"2024-06-10T01:03:38.771019+0000","last_undegraded":"2024-06-10T01:03:38.771019+0000","last_fullsized":"2024-06-10T01:03:38.771019+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:47:37.098894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771060+0000","last_change":"2024-06-10T01:01:47.248317+0000","last_active":"2024-06-10T01:03:38.771060+0000","last_peered":"2024-06-10T01:03:38.771060+0000","last_clean":"2024-06-10T01:03:38.771060+0000","last_became_active":"2024-06-10T01:01:47.247958+0000","last_became_peered":"2024-06-10T01:01:47.247958+0000","last_unstale":"2024-06-10T01:03:38.771060+0000","last_undegraded":"2024-06-10T01:03:38.771060+0000","last_fullsized":"2024-06-10T01:03:38.771060+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:39:59.267957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.237181+0000","last_change":"2024-06-10T01:01:47.252542+0000","last_active":"2024-06-10T01:03:38.237181+0000","last_peered":"2024-06-10T01:03:38.237181+0000","last_clean":"2024-06-10T01:03:38.237181+0000","last_became_active":"2024-06-10T01:01:47.252216+0000","last_became_peered":"2024-06-10T01:01:47.252216+0000","last_unstale":"2024-06-10T01:03:38.237181+0000","last_undegraded":"2024-06-10T01:03:38.237181+0000","last_fullsized":"2024-06-10T01:03:38.237181+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:12:22.196263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285691,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67668,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":593,"num_read_kb":370,"num_write":170,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":203004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383729,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1101824,"data_stored":569843,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645366,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32392,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712760,"statfs":{"total":95995035648,"available":95961866240,"internally_reserved":0,"allocated":1241088,"data_stored":1221057,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808891,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1114112,"data_stored":574063,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070528,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":569775,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332164,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1159168,"data_stored":1165405,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495689,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":565143,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574946,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1159168,"data_stored":629111,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738471,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1114112,"data_stored":566531,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411421,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28172,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716980,"statfs":{"total":95995035648,"available":95966187520,"internally_reserved":0,"allocated":1175552,"data_stored":630349,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182489,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":565337,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920851,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1110016,"data_stored":569905,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757326,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1167360,"data_stored":1160993,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34687,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34687,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34687,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":65296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T01:04:32.966 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-10T01:04:33.225 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:33.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:33 smithi007 ceph-mon[32398]: pgmap v387: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:04:33.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:33 smithi007 ceph-mon[32398]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T01:04:33.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:33 smithi175 ceph-mon[36636]: pgmap v387: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:04:33.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:33 smithi175 ceph-mon[36636]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T01:04:33.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:33 smithi078 ceph-mon[36683]: pgmap v387: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:04:33.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:33 smithi078 ceph-mon[36683]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T01:04:34.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.898+0000 7f84196c2700 1 -- 172.21.15.7:0/2961075215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84141015d0 msgr2=0x7f84141019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:34.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.898+0000 7f84196c2700 1 --2- 172.21.15.7:0/2961075215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84141015d0 0x7f84141019b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f84040099c0 tx=0x7f8404009cd0 comp rx=0 tx=0).stop 2024-06-10T01:04:34.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.898+0000 7f84196c2700 1 -- 172.21.15.7:0/2961075215 shutdown_connections 2024-06-10T01:04:34.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.898+0000 7f84196c2700 1 --2- 172.21.15.7:0/2961075215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84141066b0 0x7f841410e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:34.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.898+0000 7f84196c2700 1 --2- 172.21.15.7:0/2961075215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84141015d0 0x7f84141019b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:34.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.898+0000 7f84196c2700 1 --2- 172.21.15.7:0/2961075215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8414101f80 0x7f8414105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:34.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.898+0000 7f84196c2700 1 -- 172.21.15.7:0/2961075215 >> 172.21.15.7:0/2961075215 conn(0x7f8414078400 msgr2=0x7f8414078810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:34.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.898+0000 7f84196c2700 1 -- 172.21.15.7:0/2961075215 shutdown_connections 2024-06-10T01:04:34.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.898+0000 7f84196c2700 1 -- 172.21.15.7:0/2961075215 wait complete. 2024-06-10T01:04:34.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.899+0000 7f84196c2700 1 Processor -- start 2024-06-10T01:04:34.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.899+0000 7f84196c2700 1 -- start start 2024-06-10T01:04:34.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.899+0000 7f84196c2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84141015d0 0x7f841419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:34.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.899+0000 7f84196c2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8414101f80 0x7f841419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:34.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.899+0000 7f84196c2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84141066b0 0x7f8414198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:34.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.899+0000 7f84196c2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8414111690 con 0x7f8414101f80 2024-06-10T01:04:34.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.899+0000 7f84196c2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8414111510 con 0x7f84141066b0 2024-06-10T01:04:34.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.899+0000 7f84196c2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8414111810 con 0x7f84141015d0 2024-06-10T01:04:34.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.899+0000 7f84127fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8414101f80 0x7f841419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:34.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.900+0000 7f84127fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8414101f80 0x7f841419ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55578/0 (socket says 172.21.15.7:55578) 2024-06-10T01:04:34.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.900+0000 7f84127fc700 1 -- 172.21.15.7:0/3872035689 learned_addr learned my addr 172.21.15.7:0/3872035689 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:34.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.900+0000 7f8412ffd700 1 --2- 172.21.15.7:0/3872035689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84141015d0 0x7f841419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:34.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.900+0000 7f84137fe700 1 --2- 172.21.15.7:0/3872035689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84141066b0 0x7f8414198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:34.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.900+0000 7f84127fc700 1 -- 172.21.15.7:0/3872035689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84141015d0 msgr2=0x7f841419e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:34.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.900+0000 7f84127fc700 1 --2- 172.21.15.7:0/3872035689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84141015d0 0x7f841419e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:34.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.900+0000 7f84127fc700 1 -- 172.21.15.7:0/3872035689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84141066b0 msgr2=0x7f8414198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:34.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.900+0000 7f84127fc700 1 --2- 172.21.15.7:0/3872035689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84141066b0 0x7f8414198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:34.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.900+0000 7f84127fc700 1 -- 172.21.15.7:0/3872035689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8414199200 con 0x7f8414101f80 2024-06-10T01:04:34.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.900+0000 7f8412ffd700 1 --2- 172.21.15.7:0/3872035689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84141015d0 0x7f841419e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:04:34.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.900+0000 7f84137fe700 1 --2- 172.21.15.7:0/3872035689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84141066b0 0x7f8414198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:04:34.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.900+0000 7f84127fc700 1 --2- 172.21.15.7:0/3872035689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8414101f80 0x7f841419ee20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f840800d980 tx=0x7f840800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:34.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.900+0000 7f83fbfff700 1 -- 172.21.15.7:0/3872035689 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8408017070 con 0x7f8414101f80 2024-06-10T01:04:34.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.900+0000 7f83fbfff700 1 -- 172.21.15.7:0/3872035689 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8408009b40 con 0x7f8414101f80 2024-06-10T01:04:34.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.900+0000 7f83fbfff700 1 -- 172.21.15.7:0/3872035689 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840800f620 con 0x7f8414101f80 2024-06-10T01:04:34.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.900+0000 7f84196c2700 1 -- 172.21.15.7:0/3872035689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84141994f0 con 0x7f8414101f80 2024-06-10T01:04:34.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.900+0000 7f84196c2700 1 -- 172.21.15.7:0/3872035689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84141a57b0 con 0x7f8414101f80 2024-06-10T01:04:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.902+0000 7f84196c2700 1 -- 172.21.15.7:0/3872035689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8414069180 con 0x7f8414101f80 2024-06-10T01:04:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.903+0000 7f83fbfff700 1 -- 172.21.15.7:0/3872035689 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f8408009cf0 con 0x7f8414101f80 2024-06-10T01:04:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.904+0000 7f83fbfff700 1 --2- 172.21.15.7:0/3872035689 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f83fc0a2cb0 0x7f83fc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.904+0000 7f8412ffd700 1 --2- 172.21.15.7:0/3872035689 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f83fc0a2cb0 0x7f83fc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.904+0000 7f83fbfff700 1 -- 172.21.15.7:0/3872035689 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f84080cf4a0 con 0x7f8414101f80 2024-06-10T01:04:34.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.904+0000 7f8412ffd700 1 --2- 172.21.15.7:0/3872035689 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f83fc0a2cb0 0x7f83fc0a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8404000c00 tx=0x7f8404005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:34.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:34.908+0000 7f83fbfff700 1 -- 172.21.15.7:0/3872035689 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f840809bf20 con 0x7f8414101f80 2024-06-10T01:04:35.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:35.112+0000 7f84196c2700 1 -- 172.21.15.7:0/3872035689 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f84140628d0 con 0x7f83fc0a2cb0 2024-06-10T01:04:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:35.122+0000 7f83fbfff700 1 -- 172.21.15.7:0/3872035689 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340648 (secure 0 0 0) 0x7f84140628d0 con 0x7f83fc0a2cb0 2024-06-10T01:04:35.124 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:04:35.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:35.124+0000 7f84196c2700 1 -- 172.21.15.7:0/3872035689 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f83fc0a2cb0 msgr2=0x7f83fc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:35.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:35.124+0000 7f84196c2700 1 --2- 172.21.15.7:0/3872035689 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f83fc0a2cb0 0x7f83fc0a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8404000c00 tx=0x7f8404005ab0 comp rx=0 tx=0).stop 2024-06-10T01:04:35.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:35.125+0000 7f84196c2700 1 -- 172.21.15.7:0/3872035689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8414101f80 msgr2=0x7f841419ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:35.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:35.125+0000 7f84196c2700 1 --2- 172.21.15.7:0/3872035689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8414101f80 0x7f841419ee20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f840800d980 tx=0x7f840800dd40 comp rx=0 tx=0).stop 2024-06-10T01:04:35.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:35.125+0000 7f84196c2700 1 -- 172.21.15.7:0/3872035689 shutdown_connections 2024-06-10T01:04:35.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:35.125+0000 7f84196c2700 1 --2- 172.21.15.7:0/3872035689 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f83fc0a2cb0 0x7f83fc0a5170 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:35.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:35.125+0000 7f84196c2700 1 --2- 172.21.15.7:0/3872035689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84141066b0 0x7f8414198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:35.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:35.125+0000 7f84196c2700 1 --2- 172.21.15.7:0/3872035689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84141015d0 0x7f841419e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:35.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:35.125+0000 7f84196c2700 1 --2- 172.21.15.7:0/3872035689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8414101f80 0x7f841419ee20 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:35.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:35.125+0000 7f84196c2700 1 -- 172.21.15.7:0/3872035689 >> 172.21.15.7:0/3872035689 conn(0x7f8414078400 msgr2=0x7f841406af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:35.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:35.125+0000 7f84196c2700 1 -- 172.21.15.7:0/3872035689 shutdown_connections 2024-06-10T01:04:35.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:35.125+0000 7f84196c2700 1 -- 172.21.15.7:0/3872035689 wait complete. 2024-06-10T01:04:35.128 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-06-10T01:04:35.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:35 smithi007 ceph-mon[32398]: pgmap v388: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:04:35.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:35 smithi175 ceph-mon[36636]: pgmap v388: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:04:35.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:35 smithi078 ceph-mon[36683]: pgmap v388: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:04:35.807 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2024-06-10T01:04:33.578001+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658270,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":724,"num_read_kb":479,"num_write":353,"num_write_kb":3059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":283,"ondisk_log_size":283,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362540,"kb_used_data":13316,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579284,"statfs":{"total":1151940427776,"available":1151569186816,"internally_reserved":0,"allocated":13635584,"data_stored":8787512,"data_compressed":496626,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002922"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.237314+0000","last_change":"2024-06-10T01:01:47.244728+0000","last_active":"2024-06-10T01:03:38.237314+0000","last_peered":"2024-06-10T01:03:38.237314+0000","last_clean":"2024-06-10T01:03:38.237314+0000","last_became_active":"2024-06-10T01:01:47.244198+0000","last_became_peered":"2024-06-10T01:01:47.244198+0000","last_unstale":"2024-06-10T01:03:38.237314+0000","last_undegraded":"2024-06-10T01:03:38.237314+0000","last_fullsized":"2024-06-10T01:03:38.237314+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:58:06.257952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234562+0000","last_change":"2024-06-10T01:01:47.247620+0000","last_active":"2024-06-10T01:03:38.234562+0000","last_peered":"2024-06-10T01:03:38.234562+0000","last_clean":"2024-06-10T01:03:38.234562+0000","last_became_active":"2024-06-10T01:01:47.245475+0000","last_became_peered":"2024-06-10T01:01:47.245475+0000","last_unstale":"2024-06-10T01:03:38.234562+0000","last_undegraded":"2024-06-10T01:03:38.234562+0000","last_fullsized":"2024-06-10T01:03:38.234562+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:29:09.367411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234327+0000","last_change":"2024-06-10T01:01:47.252414+0000","last_active":"2024-06-10T01:03:38.234327+0000","last_peered":"2024-06-10T01:03:38.234327+0000","last_clean":"2024-06-10T01:03:38.234327+0000","last_became_active":"2024-06-10T01:01:47.252063+0000","last_became_peered":"2024-06-10T01:01:47.252063+0000","last_unstale":"2024-06-10T01:03:38.234327+0000","last_undegraded":"2024-06-10T01:03:38.234327+0000","last_fullsized":"2024-06-10T01:03:38.234327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:54:53.966209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239632+0000","last_change":"2024-06-10T01:01:47.254875+0000","last_active":"2024-06-10T01:03:38.239632+0000","last_peered":"2024-06-10T01:03:38.239632+0000","last_clean":"2024-06-10T01:03:38.239632+0000","last_became_active":"2024-06-10T01:01:47.254542+0000","last_became_peered":"2024-06-10T01:01:47.254542+0000","last_unstale":"2024-06-10T01:03:38.239632+0000","last_undegraded":"2024-06-10T01:03:38.239632+0000","last_fullsized":"2024-06-10T01:03:38.239632+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:02:28.393133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235286+0000","last_change":"2024-06-10T01:01:47.242596+0000","last_active":"2024-06-10T01:03:38.235286+0000","last_peered":"2024-06-10T01:03:38.235286+0000","last_clean":"2024-06-10T01:03:38.235286+0000","last_became_active":"2024-06-10T01:01:47.242308+0000","last_became_peered":"2024-06-10T01:01:47.242308+0000","last_unstale":"2024-06-10T01:03:38.235286+0000","last_undegraded":"2024-06-10T01:03:38.235286+0000","last_fullsized":"2024-06-10T01:03:38.235286+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:11:26.599954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.570472+0000","last_change":"2024-06-10T01:01:47.249085+0000","last_active":"2024-06-10T01:03:38.570472+0000","last_peered":"2024-06-10T01:03:38.570472+0000","last_clean":"2024-06-10T01:03:38.570472+0000","last_became_active":"2024-06-10T01:01:47.248749+0000","last_became_peered":"2024-06-10T01:01:47.248749+0000","last_unstale":"2024-06-10T01:03:38.570472+0000","last_undegraded":"2024-06-10T01:03:38.570472+0000","last_fullsized":"2024-06-10T01:03:38.570472+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:42:48.734566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235057+0000","last_change":"2024-06-10T01:01:47.247334+0000","last_active":"2024-06-10T01:03:38.235057+0000","last_peered":"2024-06-10T01:03:38.235057+0000","last_clean":"2024-06-10T01:03:38.235057+0000","last_became_active":"2024-06-10T01:01:47.246964+0000","last_became_peered":"2024-06-10T01:01:47.246964+0000","last_unstale":"2024-06-10T01:03:38.235057+0000","last_undegraded":"2024-06-10T01:03:38.235057+0000","last_fullsized":"2024-06-10T01:03:38.235057+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:05:36.034950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.238961+0000","last_change":"2024-06-10T01:01:47.242523+0000","last_active":"2024-06-10T01:03:38.238961+0000","last_peered":"2024-06-10T01:03:38.238961+0000","last_clean":"2024-06-10T01:03:38.238961+0000","last_became_active":"2024-06-10T01:01:47.242198+0000","last_became_peered":"2024-06-10T01:01:47.242198+0000","last_unstale":"2024-06-10T01:03:38.238961+0000","last_undegraded":"2024-06-10T01:03:38.238961+0000","last_fullsized":"2024-06-10T01:03:38.238961+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:39:07.055970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771277+0000","last_change":"2024-06-10T01:01:47.245277+0000","last_active":"2024-06-10T01:03:38.771277+0000","last_peered":"2024-06-10T01:03:38.771277+0000","last_clean":"2024-06-10T01:03:38.771277+0000","last_became_active":"2024-06-10T01:01:47.244932+0000","last_became_peered":"2024-06-10T01:01:47.244932+0000","last_unstale":"2024-06-10T01:03:38.771277+0000","last_undegraded":"2024-06-10T01:03:38.771277+0000","last_fullsized":"2024-06-10T01:03:38.771277+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:56:24.423713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.570817+0000","last_change":"2024-06-10T01:01:47.255531+0000","last_active":"2024-06-10T01:03:38.570817+0000","last_peered":"2024-06-10T01:03:38.570817+0000","last_clean":"2024-06-10T01:03:38.570817+0000","last_became_active":"2024-06-10T01:01:47.255150+0000","last_became_peered":"2024-06-10T01:01:47.255150+0000","last_unstale":"2024-06-10T01:03:38.570817+0000","last_undegraded":"2024-06-10T01:03:38.570817+0000","last_fullsized":"2024-06-10T01:03:38.570817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:51:34.561396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235538+0000","last_change":"2024-06-10T01:01:47.249832+0000","last_active":"2024-06-10T01:03:38.235538+0000","last_peered":"2024-06-10T01:03:38.235538+0000","last_clean":"2024-06-10T01:03:38.235538+0000","last_became_active":"2024-06-10T01:01:47.249303+0000","last_became_peered":"2024-06-10T01:01:47.249303+0000","last_unstale":"2024-06-10T01:03:38.235538+0000","last_undegraded":"2024-06-10T01:03:38.235538+0000","last_fullsized":"2024-06-10T01:03:38.235538+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:15:21.930157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239476+0000","last_change":"2024-06-10T01:01:47.255927+0000","last_active":"2024-06-10T01:03:38.239476+0000","last_peered":"2024-06-10T01:03:38.239476+0000","last_clean":"2024-06-10T01:03:38.239476+0000","last_became_active":"2024-06-10T01:01:47.255591+0000","last_became_peered":"2024-06-10T01:01:47.255591+0000","last_unstale":"2024-06-10T01:03:38.239476+0000","last_undegraded":"2024-06-10T01:03:38.239476+0000","last_fullsized":"2024-06-10T01:03:38.239476+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:48:06.494430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771241+0000","last_change":"2024-06-10T01:02:37.372069+0000","last_active":"2024-06-10T01:03:38.771241+0000","last_peered":"2024-06-10T01:03:38.771241+0000","last_clean":"2024-06-10T01:03:38.771241+0000","last_became_active":"2024-06-10T01:02:37.371599+0000","last_became_peered":"2024-06-10T01:02:37.371599+0000","last_unstale":"2024-06-10T01:03:38.771241+0000","last_undegraded":"2024-06-10T01:03:38.771241+0000","last_fullsized":"2024-06-10T01:03:38.771241+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:54:34.091576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234211+0000","last_change":"2024-06-10T01:01:47.249553+0000","last_active":"2024-06-10T01:03:38.234211+0000","last_peered":"2024-06-10T01:03:38.234211+0000","last_clean":"2024-06-10T01:03:38.234211+0000","last_became_active":"2024-06-10T01:01:47.249202+0000","last_became_peered":"2024-06-10T01:01:47.249202+0000","last_unstale":"2024-06-10T01:03:38.234211+0000","last_undegraded":"2024-06-10T01:03:38.234211+0000","last_fullsized":"2024-06-10T01:03:38.234211+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:23:57.882264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234200+0000","last_change":"2024-06-10T01:01:47.253987+0000","last_active":"2024-06-10T01:03:38.234200+0000","last_peered":"2024-06-10T01:03:38.234200+0000","last_clean":"2024-06-10T01:03:38.234200+0000","last_became_active":"2024-06-10T01:01:47.253578+0000","last_became_peered":"2024-06-10T01:01:47.253578+0000","last_unstale":"2024-06-10T01:03:38.234200+0000","last_undegraded":"2024-06-10T01:03:38.234200+0000","last_fullsized":"2024-06-10T01:03:38.234200+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:35:30.585482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770674+0000","last_change":"2024-06-10T01:01:47.247089+0000","last_active":"2024-06-10T01:03:38.770674+0000","last_peered":"2024-06-10T01:03:38.770674+0000","last_clean":"2024-06-10T01:03:38.770674+0000","last_became_active":"2024-06-10T01:01:47.246066+0000","last_became_peered":"2024-06-10T01:01:47.246066+0000","last_unstale":"2024-06-10T01:03:38.770674+0000","last_undegraded":"2024-06-10T01:03:38.770674+0000","last_fullsized":"2024-06-10T01:03:38.770674+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:58:12.092981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.237077+0000","last_change":"2024-06-10T01:01:47.244491+0000","last_active":"2024-06-10T01:03:38.237077+0000","last_peered":"2024-06-10T01:03:38.237077+0000","last_clean":"2024-06-10T01:03:38.237077+0000","last_became_active":"2024-06-10T01:01:47.243747+0000","last_became_peered":"2024-06-10T01:01:47.243747+0000","last_unstale":"2024-06-10T01:03:38.237077+0000","last_undegraded":"2024-06-10T01:03:38.237077+0000","last_fullsized":"2024-06-10T01:03:38.237077+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:55:56.785016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.772072+0000","last_change":"2024-06-10T01:01:47.243141+0000","last_active":"2024-06-10T01:03:38.772072+0000","last_peered":"2024-06-10T01:03:38.772072+0000","last_clean":"2024-06-10T01:03:38.772072+0000","last_became_active":"2024-06-10T01:01:47.242464+0000","last_became_peered":"2024-06-10T01:01:47.242464+0000","last_unstale":"2024-06-10T01:03:38.772072+0000","last_undegraded":"2024-06-10T01:03:38.772072+0000","last_fullsized":"2024-06-10T01:03:38.772072+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:00:49.777130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.240034+0000","last_change":"2024-06-10T01:01:47.253463+0000","last_active":"2024-06-10T01:03:38.240034+0000","last_peered":"2024-06-10T01:03:38.240034+0000","last_clean":"2024-06-10T01:03:38.240034+0000","last_became_active":"2024-06-10T01:01:47.252929+0000","last_became_peered":"2024-06-10T01:01:47.252929+0000","last_unstale":"2024-06-10T01:03:38.240034+0000","last_undegraded":"2024-06-10T01:03:38.240034+0000","last_fullsized":"2024-06-10T01:03:38.240034+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:18:49.551276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239453+0000","last_change":"2024-06-10T01:01:57.279561+0000","last_active":"2024-06-10T01:03:38.239453+0000","last_peered":"2024-06-10T01:03:38.239453+0000","last_clean":"2024-06-10T01:03:38.239453+0000","last_became_active":"2024-06-10T01:01:57.279132+0000","last_became_peered":"2024-06-10T01:01:57.279132+0000","last_unstale":"2024-06-10T01:03:38.239453+0000","last_undegraded":"2024-06-10T01:03:38.239453+0000","last_fullsized":"2024-06-10T01:03:38.239453+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:55:03.497701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'9","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.772124+0000","last_change":"2024-06-10T01:01:44.006667+0000","last_active":"2024-06-10T01:03:38.772124+0000","last_peered":"2024-06-10T01:03:38.772124+0000","last_clean":"2024-06-10T01:03:38.772124+0000","last_became_active":"2024-06-10T01:01:44.006343+0000","last_became_peered":"2024-06-10T01:01:44.006343+0000","last_unstale":"2024-06-10T01:03:38.772124+0000","last_undegraded":"2024-06-10T01:03:38.772124+0000","last_fullsized":"2024-06-10T01:03:38.772124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:14:50.540084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239291+0000","last_change":"2024-06-10T01:01:47.246139+0000","last_active":"2024-06-10T01:03:38.239291+0000","last_peered":"2024-06-10T01:03:38.239291+0000","last_clean":"2024-06-10T01:03:38.239291+0000","last_became_active":"2024-06-10T01:01:47.245743+0000","last_became_peered":"2024-06-10T01:01:47.245743+0000","last_unstale":"2024-06-10T01:03:38.239291+0000","last_undegraded":"2024-06-10T01:03:38.239291+0000","last_fullsized":"2024-06-10T01:03:38.239291+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:06:39.041569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239398+0000","last_change":"2024-06-10T01:01:57.282843+0000","last_active":"2024-06-10T01:03:38.239398+0000","last_peered":"2024-06-10T01:03:38.239398+0000","last_clean":"2024-06-10T01:03:38.239398+0000","last_became_active":"2024-06-10T01:01:57.281797+0000","last_became_peered":"2024-06-10T01:01:57.281797+0000","last_unstale":"2024-06-10T01:03:38.239398+0000","last_undegraded":"2024-06-10T01:03:38.239398+0000","last_fullsized":"2024-06-10T01:03:38.239398+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:00.209628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239550+0000","last_change":"2024-06-10T01:01:44.466714+0000","last_active":"2024-06-10T01:03:38.239550+0000","last_peered":"2024-06-10T01:03:38.239550+0000","last_clean":"2024-06-10T01:03:38.239550+0000","last_became_active":"2024-06-10T01:01:44.466296+0000","last_became_peered":"2024-06-10T01:01:44.466296+0000","last_unstale":"2024-06-10T01:03:38.239550+0000","last_undegraded":"2024-06-10T01:03:38.239550+0000","last_fullsized":"2024-06-10T01:03:38.239550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:16:36.082308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234498+0000","last_change":"2024-06-10T01:01:47.244173+0000","last_active":"2024-06-10T01:03:38.234498+0000","last_peered":"2024-06-10T01:03:38.234498+0000","last_clean":"2024-06-10T01:03:38.234498+0000","last_became_active":"2024-06-10T01:01:47.243032+0000","last_became_peered":"2024-06-10T01:01:47.243032+0000","last_unstale":"2024-06-10T01:03:38.234498+0000","last_undegraded":"2024-06-10T01:03:38.234498+0000","last_fullsized":"2024-06-10T01:03:38.234498+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:14:33.965903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239057+0000","last_change":"2024-06-10T01:02:37.592535+0000","last_active":"2024-06-10T01:03:38.239057+0000","last_peered":"2024-06-10T01:03:38.239057+0000","last_clean":"2024-06-10T01:03:38.239057+0000","last_became_active":"2024-06-10T01:02:37.592159+0000","last_became_peered":"2024-06-10T01:02:37.592159+0000","last_unstale":"2024-06-10T01:03:38.239057+0000","last_undegraded":"2024-06-10T01:03:38.239057+0000","last_fullsized":"2024-06-10T01:03:38.239057+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:50:08.626485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235433+0000","last_change":"2024-06-10T01:01:44.009105+0000","last_active":"2024-06-10T01:03:38.235433+0000","last_peered":"2024-06-10T01:03:38.235433+0000","last_clean":"2024-06-10T01:03:38.235433+0000","last_became_active":"2024-06-10T01:01:44.008743+0000","last_became_peered":"2024-06-10T01:01:44.008743+0000","last_unstale":"2024-06-10T01:03:38.235433+0000","last_undegraded":"2024-06-10T01:03:38.235433+0000","last_fullsized":"2024-06-10T01:03:38.235433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:44:54.301378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770936+0000","last_change":"2024-06-10T01:01:47.249007+0000","last_active":"2024-06-10T01:03:38.770936+0000","last_peered":"2024-06-10T01:03:38.770936+0000","last_clean":"2024-06-10T01:03:38.770936+0000","last_became_active":"2024-06-10T01:01:47.245632+0000","last_became_peered":"2024-06-10T01:01:47.245632+0000","last_unstale":"2024-06-10T01:03:38.770936+0000","last_undegraded":"2024-06-10T01:03:38.770936+0000","last_fullsized":"2024-06-10T01:03:38.770936+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:29:18.916390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.238983+0000","last_change":"2024-06-10T01:01:57.283137+0000","last_active":"2024-06-10T01:03:38.238983+0000","last_peered":"2024-06-10T01:03:38.238983+0000","last_clean":"2024-06-10T01:03:38.238983+0000","last_became_active":"2024-06-10T01:01:57.282605+0000","last_became_peered":"2024-06-10T01:01:57.282605+0000","last_unstale":"2024-06-10T01:03:38.238983+0000","last_undegraded":"2024-06-10T01:03:38.238983+0000","last_fullsized":"2024-06-10T01:03:38.238983+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:26:16.967320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235734+0000","last_change":"2024-06-10T01:01:44.010674+0000","last_active":"2024-06-10T01:03:38.235734+0000","last_peered":"2024-06-10T01:03:38.235734+0000","last_clean":"2024-06-10T01:03:38.235734+0000","last_became_active":"2024-06-10T01:01:44.010291+0000","last_became_peered":"2024-06-10T01:01:44.010291+0000","last_unstale":"2024-06-10T01:03:38.235734+0000","last_undegraded":"2024-06-10T01:03:38.235734+0000","last_fullsized":"2024-06-10T01:03:38.235734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:36:18.418362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270169+0000","last_change":"2024-06-10T01:02:37.794580+0000","last_active":"2024-06-10T01:03:38.270169+0000","last_peered":"2024-06-10T01:03:38.270169+0000","last_clean":"2024-06-10T01:03:38.270169+0000","last_became_active":"2024-06-10T01:02:37.794127+0000","last_became_peered":"2024-06-10T01:02:37.794127+0000","last_unstale":"2024-06-10T01:03:38.270169+0000","last_undegraded":"2024-06-10T01:03:38.270169+0000","last_fullsized":"2024-06-10T01:03:38.270169+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:02.117871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239746+0000","last_change":"2024-06-10T01:01:57.282494+0000","last_active":"2024-06-10T01:03:38.239746+0000","last_peered":"2024-06-10T01:03:38.239746+0000","last_clean":"2024-06-10T01:03:38.239746+0000","last_became_active":"2024-06-10T01:01:57.282077+0000","last_became_peered":"2024-06-10T01:01:57.282077+0000","last_unstale":"2024-06-10T01:03:38.239746+0000","last_undegraded":"2024-06-10T01:03:38.239746+0000","last_fullsized":"2024-06-10T01:03:38.239746+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:57:41.639788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"97'5","reported_seq":149,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:30.852251+0000","last_change":"2024-06-10T01:01:44.011134+0000","last_active":"2024-06-10T01:04:30.852251+0000","last_peered":"2024-06-10T01:04:30.852251+0000","last_clean":"2024-06-10T01:04:30.852251+0000","last_became_active":"2024-06-10T01:01:44.010788+0000","last_became_peered":"2024-06-10T01:01:44.010788+0000","last_unstale":"2024-06-10T01:04:30.852251+0000","last_undegraded":"2024-06-10T01:04:30.852251+0000","last_fullsized":"2024-06-10T01:04:30.852251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:22:55.228946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235465+0000","last_change":"2024-06-10T01:01:47.248093+0000","last_active":"2024-06-10T01:03:38.235465+0000","last_peered":"2024-06-10T01:03:38.235465+0000","last_clean":"2024-06-10T01:03:38.235465+0000","last_became_active":"2024-06-10T01:01:47.247797+0000","last_became_peered":"2024-06-10T01:01:47.247797+0000","last_unstale":"2024-06-10T01:03:38.235465+0000","last_undegraded":"2024-06-10T01:03:38.235465+0000","last_fullsized":"2024-06-10T01:03:38.235465+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:45:50.159128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239671+0000","last_change":"2024-06-10T01:02:37.377039+0000","last_active":"2024-06-10T01:03:38.239671+0000","last_peered":"2024-06-10T01:03:38.239671+0000","last_clean":"2024-06-10T01:03:38.239671+0000","last_became_active":"2024-06-10T01:02:37.376332+0000","last_became_peered":"2024-06-10T01:02:37.376332+0000","last_unstale":"2024-06-10T01:03:38.239671+0000","last_undegraded":"2024-06-10T01:03:38.239671+0000","last_fullsized":"2024-06-10T01:03:38.239671+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:15.000505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,10,5],"acting":[2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.570465+0000","last_change":"2024-06-10T01:01:44.008342+0000","last_active":"2024-06-10T01:03:38.570465+0000","last_peered":"2024-06-10T01:03:38.570465+0000","last_clean":"2024-06-10T01:03:38.570465+0000","last_became_active":"2024-06-10T01:01:44.007959+0000","last_became_peered":"2024-06-10T01:01:44.007959+0000","last_unstale":"2024-06-10T01:03:38.570465+0000","last_undegraded":"2024-06-10T01:03:38.570465+0000","last_fullsized":"2024-06-10T01:03:38.570465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:36:03.143609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.269968+0000","last_change":"2024-06-10T01:01:47.247854+0000","last_active":"2024-06-10T01:03:38.269968+0000","last_peered":"2024-06-10T01:03:38.269968+0000","last_clean":"2024-06-10T01:03:38.269968+0000","last_became_active":"2024-06-10T01:01:47.247563+0000","last_became_peered":"2024-06-10T01:01:47.247563+0000","last_unstale":"2024-06-10T01:03:38.269968+0000","last_undegraded":"2024-06-10T01:03:38.269968+0000","last_fullsized":"2024-06-10T01:03:38.269968+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:32:13.024815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.237041+0000","last_change":"2024-06-10T01:01:57.282427+0000","last_active":"2024-06-10T01:03:38.237041+0000","last_peered":"2024-06-10T01:03:38.237041+0000","last_clean":"2024-06-10T01:03:38.237041+0000","last_became_active":"2024-06-10T01:01:57.281987+0000","last_became_peered":"2024-06-10T01:01:57.281987+0000","last_unstale":"2024-06-10T01:03:38.237041+0000","last_undegraded":"2024-06-10T01:03:38.237041+0000","last_fullsized":"2024-06-10T01:03:38.237041+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:22:31.736804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770902+0000","last_change":"2024-06-10T01:01:44.006567+0000","last_active":"2024-06-10T01:03:38.770902+0000","last_peered":"2024-06-10T01:03:38.770902+0000","last_clean":"2024-06-10T01:03:38.770902+0000","last_became_active":"2024-06-10T01:01:44.006216+0000","last_became_peered":"2024-06-10T01:01:44.006216+0000","last_unstale":"2024-06-10T01:03:38.770902+0000","last_undegraded":"2024-06-10T01:03:38.770902+0000","last_fullsized":"2024-06-10T01:03:38.770902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:33:55.696873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.238845+0000","last_change":"2024-06-10T01:01:47.245326+0000","last_active":"2024-06-10T01:03:38.238845+0000","last_peered":"2024-06-10T01:03:38.238845+0000","last_clean":"2024-06-10T01:03:38.238845+0000","last_became_active":"2024-06-10T01:01:47.245013+0000","last_became_peered":"2024-06-10T01:01:47.245013+0000","last_unstale":"2024-06-10T01:03:38.238845+0000","last_undegraded":"2024-06-10T01:03:38.238845+0000","last_fullsized":"2024-06-10T01:03:38.238845+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:17:56.416627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234530+0000","last_change":"2024-06-10T01:01:57.288997+0000","last_active":"2024-06-10T01:03:38.234530+0000","last_peered":"2024-06-10T01:03:38.234530+0000","last_clean":"2024-06-10T01:03:38.234530+0000","last_became_active":"2024-06-10T01:01:57.288596+0000","last_became_peered":"2024-06-10T01:01:57.288596+0000","last_unstale":"2024-06-10T01:03:38.234530+0000","last_undegraded":"2024-06-10T01:03:38.234530+0000","last_fullsized":"2024-06-10T01:03:38.234530+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:08:14.739737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.269738+0000","last_change":"2024-06-10T01:01:44.010079+0000","last_active":"2024-06-10T01:03:38.269738+0000","last_peered":"2024-06-10T01:03:38.269738+0000","last_clean":"2024-06-10T01:03:38.269738+0000","last_became_active":"2024-06-10T01:01:44.009752+0000","last_became_peered":"2024-06-10T01:01:44.009752+0000","last_unstale":"2024-06-10T01:03:38.269738+0000","last_undegraded":"2024-06-10T01:03:38.269738+0000","last_fullsized":"2024-06-10T01:03:38.269738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:29:35.660935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770966+0000","last_change":"2024-06-10T01:01:47.245015+0000","last_active":"2024-06-10T01:03:38.770966+0000","last_peered":"2024-06-10T01:03:38.770966+0000","last_clean":"2024-06-10T01:03:38.770966+0000","last_became_active":"2024-06-10T01:01:47.244595+0000","last_became_peered":"2024-06-10T01:01:47.244595+0000","last_unstale":"2024-06-10T01:03:38.770966+0000","last_undegraded":"2024-06-10T01:03:38.770966+0000","last_fullsized":"2024-06-10T01:03:38.770966+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:01:48.446526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.238926+0000","last_change":"2024-06-10T01:01:57.281656+0000","last_active":"2024-06-10T01:03:38.238926+0000","last_peered":"2024-06-10T01:03:38.238926+0000","last_clean":"2024-06-10T01:03:38.238926+0000","last_became_active":"2024-06-10T01:01:57.281295+0000","last_became_peered":"2024-06-10T01:01:57.281295+0000","last_unstale":"2024-06-10T01:03:38.238926+0000","last_undegraded":"2024-06-10T01:03:38.238926+0000","last_fullsized":"2024-06-10T01:03:38.238926+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:04:41.619864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"97'3","reported_seq":144,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:29.850140+0000","last_change":"2024-06-10T01:01:44.011941+0000","last_active":"2024-06-10T01:04:29.850140+0000","last_peered":"2024-06-10T01:04:29.850140+0000","last_clean":"2024-06-10T01:04:29.850140+0000","last_became_active":"2024-06-10T01:01:44.010156+0000","last_became_peered":"2024-06-10T01:01:44.010156+0000","last_unstale":"2024-06-10T01:04:29.850140+0000","last_undegraded":"2024-06-10T01:04:29.850140+0000","last_fullsized":"2024-06-10T01:04:29.850140+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:16:15.709544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771303+0000","last_change":"2024-06-10T01:01:47.241720+0000","last_active":"2024-06-10T01:03:38.771303+0000","last_peered":"2024-06-10T01:03:38.771303+0000","last_clean":"2024-06-10T01:03:38.771303+0000","last_became_active":"2024-06-10T01:01:47.241386+0000","last_became_peered":"2024-06-10T01:01:47.241386+0000","last_unstale":"2024-06-10T01:03:38.771303+0000","last_undegraded":"2024-06-10T01:03:38.771303+0000","last_fullsized":"2024-06-10T01:03:38.771303+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:42:01.304787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770709+0000","last_change":"2024-06-10T01:01:57.283226+0000","last_active":"2024-06-10T01:03:38.770709+0000","last_peered":"2024-06-10T01:03:38.770709+0000","last_clean":"2024-06-10T01:03:38.770709+0000","last_became_active":"2024-06-10T01:01:57.282720+0000","last_became_peered":"2024-06-10T01:01:57.282720+0000","last_unstale":"2024-06-10T01:03:38.770709+0000","last_undegraded":"2024-06-10T01:03:38.770709+0000","last_fullsized":"2024-06-10T01:03:38.770709+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:35:31.342042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235489+0000","last_change":"2024-06-10T01:01:44.009278+0000","last_active":"2024-06-10T01:03:38.235489+0000","last_peered":"2024-06-10T01:03:38.235489+0000","last_clean":"2024-06-10T01:03:38.235489+0000","last_became_active":"2024-06-10T01:01:44.008841+0000","last_became_peered":"2024-06-10T01:01:44.008841+0000","last_unstale":"2024-06-10T01:03:38.235489+0000","last_undegraded":"2024-06-10T01:03:38.235489+0000","last_fullsized":"2024-06-10T01:03:38.235489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:30:58.392080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234525+0000","last_change":"2024-06-10T01:01:47.256451+0000","last_active":"2024-06-10T01:03:38.234525+0000","last_peered":"2024-06-10T01:03:38.234525+0000","last_clean":"2024-06-10T01:03:38.234525+0000","last_became_active":"2024-06-10T01:01:47.256036+0000","last_became_peered":"2024-06-10T01:01:47.256036+0000","last_unstale":"2024-06-10T01:03:38.234525+0000","last_undegraded":"2024-06-10T01:03:38.234525+0000","last_fullsized":"2024-06-10T01:03:38.234525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:57:26.640109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239192+0000","last_change":"2024-06-10T01:02:37.794160+0000","last_active":"2024-06-10T01:03:38.239192+0000","last_peered":"2024-06-10T01:03:38.239192+0000","last_clean":"2024-06-10T01:03:38.239192+0000","last_became_active":"2024-06-10T01:02:37.793633+0000","last_became_peered":"2024-06-10T01:02:37.793633+0000","last_unstale":"2024-06-10T01:03:38.239192+0000","last_undegraded":"2024-06-10T01:03:38.239192+0000","last_fullsized":"2024-06-10T01:03:38.239192+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:48:48.083620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270125+0000","last_change":"2024-06-10T01:01:44.469179+0000","last_active":"2024-06-10T01:03:38.270125+0000","last_peered":"2024-06-10T01:03:38.270125+0000","last_clean":"2024-06-10T01:03:38.270125+0000","last_became_active":"2024-06-10T01:01:44.468669+0000","last_became_peered":"2024-06-10T01:01:44.468669+0000","last_unstale":"2024-06-10T01:03:38.270125+0000","last_undegraded":"2024-06-10T01:03:38.270125+0000","last_fullsized":"2024-06-10T01:03:38.270125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:07:51.629030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770954+0000","last_change":"2024-06-10T01:01:47.247588+0000","last_active":"2024-06-10T01:03:38.770954+0000","last_peered":"2024-06-10T01:03:38.770954+0000","last_clean":"2024-06-10T01:03:38.770954+0000","last_became_active":"2024-06-10T01:01:47.245750+0000","last_became_peered":"2024-06-10T01:01:47.245750+0000","last_unstale":"2024-06-10T01:03:38.770954+0000","last_undegraded":"2024-06-10T01:03:38.770954+0000","last_fullsized":"2024-06-10T01:03:38.770954+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:12:10.513777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771292+0000","last_change":"2024-06-10T01:01:57.283042+0000","last_active":"2024-06-10T01:03:38.771292+0000","last_peered":"2024-06-10T01:03:38.771292+0000","last_clean":"2024-06-10T01:03:38.771292+0000","last_became_active":"2024-06-10T01:01:57.282600+0000","last_became_peered":"2024-06-10T01:01:57.282600+0000","last_unstale":"2024-06-10T01:03:38.771292+0000","last_undegraded":"2024-06-10T01:03:38.771292+0000","last_fullsized":"2024-06-10T01:03:38.771292+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:06:36.969968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"97'34","reported_seq":280,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:30.851642+0000","last_change":"2024-06-10T01:01:44.011783+0000","last_active":"2024-06-10T01:04:30.851642+0000","last_peered":"2024-06-10T01:04:30.851642+0000","last_clean":"2024-06-10T01:04:30.851642+0000","last_became_active":"2024-06-10T01:01:44.011164+0000","last_became_peered":"2024-06-10T01:01:44.011164+0000","last_unstale":"2024-06-10T01:04:30.851642+0000","last_undegraded":"2024-06-10T01:04:30.851642+0000","last_fullsized":"2024-06-10T01:04:30.851642+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:16:13.642714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64058,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":201,"num_read_kb":163,"num_write":35,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235185+0000","last_change":"2024-06-10T01:01:47.250751+0000","last_active":"2024-06-10T01:03:38.235185+0000","last_peered":"2024-06-10T01:03:38.235185+0000","last_clean":"2024-06-10T01:03:38.235185+0000","last_became_active":"2024-06-10T01:01:47.250451+0000","last_became_peered":"2024-06-10T01:01:47.250451+0000","last_unstale":"2024-06-10T01:03:38.235185+0000","last_undegraded":"2024-06-10T01:03:38.235185+0000","last_fullsized":"2024-06-10T01:03:38.235185+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:22:10.121363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270382+0000","last_change":"2024-06-10T01:01:57.289687+0000","last_active":"2024-06-10T01:03:38.270382+0000","last_peered":"2024-06-10T01:03:38.270382+0000","last_clean":"2024-06-10T01:03:38.270382+0000","last_became_active":"2024-06-10T01:01:57.289274+0000","last_became_peered":"2024-06-10T01:01:57.289274+0000","last_unstale":"2024-06-10T01:03:38.270382+0000","last_undegraded":"2024-06-10T01:03:38.270382+0000","last_fullsized":"2024-06-10T01:03:38.270382+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:34:46.873974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'6","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239529+0000","last_change":"2024-06-10T01:01:44.010159+0000","last_active":"2024-06-10T01:03:38.239529+0000","last_peered":"2024-06-10T01:03:38.239529+0000","last_clean":"2024-06-10T01:03:38.239529+0000","last_became_active":"2024-06-10T01:01:44.009744+0000","last_became_peered":"2024-06-10T01:01:44.009744+0000","last_unstale":"2024-06-10T01:03:38.239529+0000","last_undegraded":"2024-06-10T01:03:38.239529+0000","last_fullsized":"2024-06-10T01:03:38.239529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:30.507613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771040+0000","last_change":"2024-06-10T01:01:47.246662+0000","last_active":"2024-06-10T01:03:38.771040+0000","last_peered":"2024-06-10T01:03:38.771040+0000","last_clean":"2024-06-10T01:03:38.771040+0000","last_became_active":"2024-06-10T01:01:47.246268+0000","last_became_peered":"2024-06-10T01:01:47.246268+0000","last_unstale":"2024-06-10T01:03:38.771040+0000","last_undegraded":"2024-06-10T01:03:38.771040+0000","last_fullsized":"2024-06-10T01:03:38.771040+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:08:38.089054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.236925+0000","last_change":"2024-06-10T01:01:57.289937+0000","last_active":"2024-06-10T01:03:38.236925+0000","last_peered":"2024-06-10T01:03:38.236925+0000","last_clean":"2024-06-10T01:03:38.236925+0000","last_became_active":"2024-06-10T01:01:57.289407+0000","last_became_peered":"2024-06-10T01:01:57.289407+0000","last_unstale":"2024-06-10T01:03:38.236925+0000","last_undegraded":"2024-06-10T01:03:38.236925+0000","last_fullsized":"2024-06-10T01:03:38.236925+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:01:48.546762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270079+0000","last_change":"2024-06-10T01:01:44.012654+0000","last_active":"2024-06-10T01:03:38.270079+0000","last_peered":"2024-06-10T01:03:38.270079+0000","last_clean":"2024-06-10T01:03:38.270079+0000","last_became_active":"2024-06-10T01:01:44.012246+0000","last_became_peered":"2024-06-10T01:01:44.012246+0000","last_unstale":"2024-06-10T01:03:38.270079+0000","last_undegraded":"2024-06-10T01:03:38.270079+0000","last_fullsized":"2024-06-10T01:03:38.270079+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:31:36.173554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235282+0000","last_change":"2024-06-10T01:01:47.250514+0000","last_active":"2024-06-10T01:03:38.235282+0000","last_peered":"2024-06-10T01:03:38.235282+0000","last_clean":"2024-06-10T01:03:38.235282+0000","last_became_active":"2024-06-10T01:01:47.249050+0000","last_became_peered":"2024-06-10T01:01:47.249050+0000","last_unstale":"2024-06-10T01:03:38.235282+0000","last_undegraded":"2024-06-10T01:03:38.235282+0000","last_fullsized":"2024-06-10T01:03:38.235282+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:51:30.789392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771928+0000","last_change":"2024-06-10T01:01:57.282389+0000","last_active":"2024-06-10T01:03:38.771928+0000","last_peered":"2024-06-10T01:03:38.771928+0000","last_clean":"2024-06-10T01:03:38.771928+0000","last_became_active":"2024-06-10T01:01:57.281857+0000","last_became_peered":"2024-06-10T01:01:57.281857+0000","last_unstale":"2024-06-10T01:03:38.771928+0000","last_undegraded":"2024-06-10T01:03:38.771928+0000","last_fullsized":"2024-06-10T01:03:38.771928+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:59:14.863020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"97'3","reported_seq":143,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:26.847151+0000","last_change":"2024-06-10T01:01:44.011331+0000","last_active":"2024-06-10T01:04:26.847151+0000","last_peered":"2024-06-10T01:04:26.847151+0000","last_clean":"2024-06-10T01:04:26.847151+0000","last_became_active":"2024-06-10T01:01:44.010968+0000","last_became_peered":"2024-06-10T01:01:44.010968+0000","last_unstale":"2024-06-10T01:04:26.847151+0000","last_undegraded":"2024-06-10T01:04:26.847151+0000","last_fullsized":"2024-06-10T01:04:26.847151+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:43:37.931225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235103+0000","last_change":"2024-06-10T01:01:47.248349+0000","last_active":"2024-06-10T01:03:38.235103+0000","last_peered":"2024-06-10T01:03:38.235103+0000","last_clean":"2024-06-10T01:03:38.235103+0000","last_became_active":"2024-06-10T01:01:47.248008+0000","last_became_peered":"2024-06-10T01:01:47.248008+0000","last_unstale":"2024-06-10T01:03:38.235103+0000","last_undegraded":"2024-06-10T01:03:38.235103+0000","last_fullsized":"2024-06-10T01:03:38.235103+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:45:20.747704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771221+0000","last_change":"2024-06-10T01:01:57.289773+0000","last_active":"2024-06-10T01:03:38.771221+0000","last_peered":"2024-06-10T01:03:38.771221+0000","last_clean":"2024-06-10T01:03:38.771221+0000","last_became_active":"2024-06-10T01:01:57.289371+0000","last_became_peered":"2024-06-10T01:01:57.289371+0000","last_unstale":"2024-06-10T01:03:38.771221+0000","last_undegraded":"2024-06-10T01:03:38.771221+0000","last_fullsized":"2024-06-10T01:03:38.771221+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:24:52.209402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'15","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.257548+0000","last_change":"2024-06-10T01:03:38.257122+0000","last_active":"2024-06-10T01:03:38.257548+0000","last_peered":"2024-06-10T01:03:38.257548+0000","last_clean":"2024-06-10T01:03:38.257548+0000","last_became_active":"2024-06-10T01:03:38.240737+0000","last_became_peered":"2024-06-10T01:03:38.240737+0000","last_unstale":"2024-06-10T01:03:38.257548+0000","last_undegraded":"2024-06-10T01:03:38.257548+0000","last_fullsized":"2024-06-10T01:03:38.257548+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:37:10.925004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.269868+0000","last_change":"2024-06-10T01:01:47.245439+0000","last_active":"2024-06-10T01:03:38.269868+0000","last_peered":"2024-06-10T01:03:38.269868+0000","last_clean":"2024-06-10T01:03:38.269868+0000","last_became_active":"2024-06-10T01:01:47.242917+0000","last_became_peered":"2024-06-10T01:01:47.242917+0000","last_unstale":"2024-06-10T01:03:38.269868+0000","last_undegraded":"2024-06-10T01:03:38.269868+0000","last_fullsized":"2024-06-10T01:03:38.269868+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:13:12.614230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.269795+0000","last_change":"2024-06-10T01:01:57.284731+0000","last_active":"2024-06-10T01:03:38.269795+0000","last_peered":"2024-06-10T01:03:38.269795+0000","last_clean":"2024-06-10T01:03:38.269795+0000","last_became_active":"2024-06-10T01:01:57.284301+0000","last_became_peered":"2024-06-10T01:01:57.284301+0000","last_unstale":"2024-06-10T01:03:38.269795+0000","last_undegraded":"2024-06-10T01:03:38.269795+0000","last_fullsized":"2024-06-10T01:03:38.269795+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:01:36.518214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235624+0000","last_change":"2024-06-10T01:01:44.010154+0000","last_active":"2024-06-10T01:03:38.235624+0000","last_peered":"2024-06-10T01:03:38.235624+0000","last_clean":"2024-06-10T01:03:38.235624+0000","last_became_active":"2024-06-10T01:01:44.009656+0000","last_became_peered":"2024-06-10T01:01:44.009656+0000","last_unstale":"2024-06-10T01:03:38.235624+0000","last_undegraded":"2024-06-10T01:03:38.235624+0000","last_fullsized":"2024-06-10T01:03:38.235624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:33:08.789946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.236902+0000","last_change":"2024-06-10T01:02:37.378484+0000","last_active":"2024-06-10T01:03:38.236902+0000","last_peered":"2024-06-10T01:03:38.236902+0000","last_clean":"2024-06-10T01:03:38.236902+0000","last_became_active":"2024-06-10T01:02:37.378092+0000","last_became_peered":"2024-06-10T01:02:37.378092+0000","last_unstale":"2024-06-10T01:03:38.236902+0000","last_undegraded":"2024-06-10T01:03:38.236902+0000","last_fullsized":"2024-06-10T01:03:38.236902+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:19:46.599235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235375+0000","last_change":"2024-06-10T01:01:57.332887+0000","last_active":"2024-06-10T01:03:38.235375+0000","last_peered":"2024-06-10T01:03:38.235375+0000","last_clean":"2024-06-10T01:03:38.235375+0000","last_became_active":"2024-06-10T01:01:57.332285+0000","last_became_peered":"2024-06-10T01:01:57.332285+0000","last_unstale":"2024-06-10T01:03:38.235375+0000","last_undegraded":"2024-06-10T01:03:38.235375+0000","last_fullsized":"2024-06-10T01:03:38.235375+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:09:12.233405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771642+0000","last_change":"2024-06-10T01:01:44.466831+0000","last_active":"2024-06-10T01:03:38.771642+0000","last_peered":"2024-06-10T01:03:38.771642+0000","last_clean":"2024-06-10T01:03:38.771642+0000","last_became_active":"2024-06-10T01:01:44.466484+0000","last_became_peered":"2024-06-10T01:01:44.466484+0000","last_unstale":"2024-06-10T01:03:38.771642+0000","last_undegraded":"2024-06-10T01:03:38.771642+0000","last_fullsized":"2024-06-10T01:03:38.771642+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:19:28.385504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.570599+0000","last_change":"2024-06-10T01:01:47.252399+0000","last_active":"2024-06-10T01:03:38.570599+0000","last_peered":"2024-06-10T01:03:38.570599+0000","last_clean":"2024-06-10T01:03:38.570599+0000","last_became_active":"2024-06-10T01:01:47.252069+0000","last_became_peered":"2024-06-10T01:01:47.252069+0000","last_unstale":"2024-06-10T01:03:38.570599+0000","last_undegraded":"2024-06-10T01:03:38.570599+0000","last_fullsized":"2024-06-10T01:03:38.570599+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:02:02.291174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771406+0000","last_change":"2024-06-10T01:01:57.285670+0000","last_active":"2024-06-10T01:03:38.771406+0000","last_peered":"2024-06-10T01:03:38.771406+0000","last_clean":"2024-06-10T01:03:38.771406+0000","last_became_active":"2024-06-10T01:01:57.285281+0000","last_became_peered":"2024-06-10T01:01:57.285281+0000","last_unstale":"2024-06-10T01:03:38.771406+0000","last_undegraded":"2024-06-10T01:03:38.771406+0000","last_fullsized":"2024-06-10T01:03:38.771406+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:58.956934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235189+0000","last_change":"2024-06-10T01:01:44.008013+0000","last_active":"2024-06-10T01:03:38.235189+0000","last_peered":"2024-06-10T01:03:38.235189+0000","last_clean":"2024-06-10T01:03:38.235189+0000","last_became_active":"2024-06-10T01:01:44.007591+0000","last_became_peered":"2024-06-10T01:01:44.007591+0000","last_unstale":"2024-06-10T01:03:38.235189+0000","last_undegraded":"2024-06-10T01:03:38.235189+0000","last_fullsized":"2024-06-10T01:03:38.235189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:18:20.862340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.570439+0000","last_change":"2024-06-10T01:01:47.254756+0000","last_active":"2024-06-10T01:03:38.570439+0000","last_peered":"2024-06-10T01:03:38.570439+0000","last_clean":"2024-06-10T01:03:38.570439+0000","last_became_active":"2024-06-10T01:01:47.254418+0000","last_became_peered":"2024-06-10T01:01:47.254418+0000","last_unstale":"2024-06-10T01:03:38.570439+0000","last_undegraded":"2024-06-10T01:03:38.570439+0000","last_fullsized":"2024-06-10T01:03:38.570439+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:46:31.452539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770995+0000","last_change":"2024-06-10T01:01:57.332792+0000","last_active":"2024-06-10T01:03:38.770995+0000","last_peered":"2024-06-10T01:03:38.770995+0000","last_clean":"2024-06-10T01:03:38.770995+0000","last_became_active":"2024-06-10T01:01:57.332331+0000","last_became_peered":"2024-06-10T01:01:57.332331+0000","last_unstale":"2024-06-10T01:03:38.770995+0000","last_undegraded":"2024-06-10T01:03:38.770995+0000","last_fullsized":"2024-06-10T01:03:38.770995+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:46:41.419807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270036+0000","last_change":"2024-06-10T01:01:47.247857+0000","last_active":"2024-06-10T01:03:38.270036+0000","last_peered":"2024-06-10T01:03:38.270036+0000","last_clean":"2024-06-10T01:03:38.270036+0000","last_became_active":"2024-06-10T01:01:47.247552+0000","last_became_peered":"2024-06-10T01:01:47.247552+0000","last_unstale":"2024-06-10T01:03:38.270036+0000","last_undegraded":"2024-06-10T01:03:38.270036+0000","last_fullsized":"2024-06-10T01:03:38.270036+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:29:40.521887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"97'4","reported_seq":143,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:27.848040+0000","last_change":"2024-06-10T01:01:44.466631+0000","last_active":"2024-06-10T01:04:27.848040+0000","last_peered":"2024-06-10T01:04:27.848040+0000","last_clean":"2024-06-10T01:04:27.848040+0000","last_became_active":"2024-06-10T01:01:44.466280+0000","last_became_peered":"2024-06-10T01:01:44.466280+0000","last_unstale":"2024-06-10T01:04:27.848040+0000","last_undegraded":"2024-06-10T01:04:27.848040+0000","last_fullsized":"2024-06-10T01:04:27.848040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:09:33.658112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235365+0000","last_change":"2024-06-10T01:01:57.281973+0000","last_active":"2024-06-10T01:03:38.235365+0000","last_peered":"2024-06-10T01:03:38.235365+0000","last_clean":"2024-06-10T01:03:38.235365+0000","last_became_active":"2024-06-10T01:01:57.281386+0000","last_became_peered":"2024-06-10T01:01:57.281386+0000","last_unstale":"2024-06-10T01:03:38.235365+0000","last_undegraded":"2024-06-10T01:03:38.235365+0000","last_fullsized":"2024-06-10T01:03:38.235365+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:10:28.236250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239023+0000","last_change":"2024-06-10T01:01:47.237436+0000","last_active":"2024-06-10T01:03:38.239023+0000","last_peered":"2024-06-10T01:03:38.239023+0000","last_clean":"2024-06-10T01:03:38.239023+0000","last_became_active":"2024-06-10T01:01:47.237099+0000","last_became_peered":"2024-06-10T01:01:47.237099+0000","last_unstale":"2024-06-10T01:03:38.239023+0000","last_undegraded":"2024-06-10T01:03:38.239023+0000","last_fullsized":"2024-06-10T01:03:38.239023+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:34:53.838877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'3","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239178+0000","last_change":"2024-06-10T01:01:44.010655+0000","last_active":"2024-06-10T01:03:38.239178+0000","last_peered":"2024-06-10T01:03:38.239178+0000","last_clean":"2024-06-10T01:03:38.239178+0000","last_became_active":"2024-06-10T01:01:44.009918+0000","last_became_peered":"2024-06-10T01:01:44.009918+0000","last_unstale":"2024-06-10T01:03:38.239178+0000","last_undegraded":"2024-06-10T01:03:38.239178+0000","last_fullsized":"2024-06-10T01:03:38.239178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:47:32.052650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":366,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234835+0000","last_change":"2024-06-10T01:00:23.822447+0000","last_active":"2024-06-10T01:03:38.234835+0000","last_peered":"2024-06-10T01:03:38.234835+0000","last_clean":"2024-06-10T01:03:38.234835+0000","last_became_active":"2024-06-10T01:00:23.821966+0000","last_became_peered":"2024-06-10T01:00:23.821966+0000","last_unstale":"2024-06-10T01:03:38.234835+0000","last_undegraded":"2024-06-10T01:03:38.234835+0000","last_fullsized":"2024-06-10T01:03:38.234835+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:56:59.964661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:56:59.964661+0000","last_clean_scrub_stamp":"2024-06-10T00:56:59.964661+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:03:43.732875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270272+0000","last_change":"2024-06-10T01:01:57.332980+0000","last_active":"2024-06-10T01:03:38.270272+0000","last_peered":"2024-06-10T01:03:38.270272+0000","last_clean":"2024-06-10T01:03:38.270272+0000","last_became_active":"2024-06-10T01:01:57.332437+0000","last_became_peered":"2024-06-10T01:01:57.332437+0000","last_unstale":"2024-06-10T01:03:38.270272+0000","last_undegraded":"2024-06-10T01:03:38.270272+0000","last_fullsized":"2024-06-10T01:03:38.270272+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:27:35.029079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770659+0000","last_change":"2024-06-10T01:01:47.255926+0000","last_active":"2024-06-10T01:03:38.770659+0000","last_peered":"2024-06-10T01:03:38.770659+0000","last_clean":"2024-06-10T01:03:38.770659+0000","last_became_active":"2024-06-10T01:01:47.255571+0000","last_became_peered":"2024-06-10T01:01:47.255571+0000","last_unstale":"2024-06-10T01:03:38.770659+0000","last_undegraded":"2024-06-10T01:03:38.770659+0000","last_fullsized":"2024-06-10T01:03:38.770659+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:05:22.084037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234997+0000","last_change":"2024-06-10T01:01:44.007509+0000","last_active":"2024-06-10T01:03:38.234997+0000","last_peered":"2024-06-10T01:03:38.234997+0000","last_clean":"2024-06-10T01:03:38.234997+0000","last_became_active":"2024-06-10T01:01:44.007178+0000","last_became_peered":"2024-06-10T01:01:44.007178+0000","last_unstale":"2024-06-10T01:03:38.234997+0000","last_undegraded":"2024-06-10T01:03:38.234997+0000","last_fullsized":"2024-06-10T01:03:38.234997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:02:52.791886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.236790+0000","last_change":"2024-06-10T01:01:57.285626+0000","last_active":"2024-06-10T01:03:38.236790+0000","last_peered":"2024-06-10T01:03:38.236790+0000","last_clean":"2024-06-10T01:03:38.236790+0000","last_became_active":"2024-06-10T01:01:57.285098+0000","last_became_peered":"2024-06-10T01:01:57.285098+0000","last_unstale":"2024-06-10T01:03:38.236790+0000","last_undegraded":"2024-06-10T01:03:38.236790+0000","last_fullsized":"2024-06-10T01:03:38.236790+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:13:49.881770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235332+0000","last_change":"2024-06-10T01:01:44.469028+0000","last_active":"2024-06-10T01:03:38.235332+0000","last_peered":"2024-06-10T01:03:38.235332+0000","last_clean":"2024-06-10T01:03:38.235332+0000","last_became_active":"2024-06-10T01:01:44.468570+0000","last_became_peered":"2024-06-10T01:01:44.468570+0000","last_unstale":"2024-06-10T01:03:38.235332+0000","last_undegraded":"2024-06-10T01:03:38.235332+0000","last_fullsized":"2024-06-10T01:03:38.235332+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:12:11.287959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771533+0000","last_change":"2024-06-10T01:01:47.253414+0000","last_active":"2024-06-10T01:03:38.771533+0000","last_peered":"2024-06-10T01:03:38.771533+0000","last_clean":"2024-06-10T01:03:38.771533+0000","last_became_active":"2024-06-10T01:01:47.253060+0000","last_became_peered":"2024-06-10T01:01:47.253060+0000","last_unstale":"2024-06-10T01:03:38.771533+0000","last_undegraded":"2024-06-10T01:03:38.771533+0000","last_fullsized":"2024-06-10T01:03:38.771533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:24:55.534164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.238849+0000","last_change":"2024-06-10T01:01:57.332756+0000","last_active":"2024-06-10T01:03:38.238849+0000","last_peered":"2024-06-10T01:03:38.238849+0000","last_clean":"2024-06-10T01:03:38.238849+0000","last_became_active":"2024-06-10T01:01:57.332361+0000","last_became_peered":"2024-06-10T01:01:57.332361+0000","last_unstale":"2024-06-10T01:03:38.238849+0000","last_undegraded":"2024-06-10T01:03:38.238849+0000","last_fullsized":"2024-06-10T01:03:38.238849+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:02:28.275407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"97'12","reported_seq":57,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.772073+0000","last_change":"2024-06-10T01:01:44.005164+0000","last_active":"2024-06-10T01:03:38.772073+0000","last_peered":"2024-06-10T01:03:38.772073+0000","last_clean":"2024-06-10T01:03:38.772073+0000","last_became_active":"2024-06-10T01:01:44.004816+0000","last_became_peered":"2024-06-10T01:01:44.004816+0000","last_unstale":"2024-06-10T01:03:38.772073+0000","last_undegraded":"2024-06-10T01:03:38.772073+0000","last_fullsized":"2024-06-10T01:03:38.772073+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:11:56.385267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239548+0000","last_change":"2024-06-10T01:01:47.247951+0000","last_active":"2024-06-10T01:03:38.239548+0000","last_peered":"2024-06-10T01:03:38.239548+0000","last_clean":"2024-06-10T01:03:38.239548+0000","last_became_active":"2024-06-10T01:01:47.246996+0000","last_became_peered":"2024-06-10T01:01:47.246996+0000","last_unstale":"2024-06-10T01:03:38.239548+0000","last_undegraded":"2024-06-10T01:03:38.239548+0000","last_fullsized":"2024-06-10T01:03:38.239548+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:39:18.167021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239228+0000","last_change":"2024-06-10T01:01:57.278130+0000","last_active":"2024-06-10T01:03:38.239228+0000","last_peered":"2024-06-10T01:03:38.239228+0000","last_clean":"2024-06-10T01:03:38.239228+0000","last_became_active":"2024-06-10T01:01:57.277697+0000","last_became_peered":"2024-06-10T01:01:57.277697+0000","last_unstale":"2024-06-10T01:03:38.239228+0000","last_undegraded":"2024-06-10T01:03:38.239228+0000","last_fullsized":"2024-06-10T01:03:38.239228+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:17:01.662152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239618+0000","last_change":"2024-06-10T01:01:44.553176+0000","last_active":"2024-06-10T01:03:38.239618+0000","last_peered":"2024-06-10T01:03:38.239618+0000","last_clean":"2024-06-10T01:03:38.239618+0000","last_became_active":"2024-06-10T01:01:44.552797+0000","last_became_peered":"2024-06-10T01:01:44.552797+0000","last_unstale":"2024-06-10T01:03:38.239618+0000","last_undegraded":"2024-06-10T01:03:38.239618+0000","last_fullsized":"2024-06-10T01:03:38.239618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:21:13.960304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234635+0000","last_change":"2024-06-10T01:01:47.255796+0000","last_active":"2024-06-10T01:03:38.234635+0000","last_peered":"2024-06-10T01:03:38.234635+0000","last_clean":"2024-06-10T01:03:38.234635+0000","last_became_active":"2024-06-10T01:01:47.255495+0000","last_became_peered":"2024-06-10T01:01:47.255495+0000","last_unstale":"2024-06-10T01:03:38.234635+0000","last_undegraded":"2024-06-10T01:03:38.234635+0000","last_fullsized":"2024-06-10T01:03:38.234635+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:06:42.209005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239491+0000","last_change":"2024-06-10T01:01:57.332869+0000","last_active":"2024-06-10T01:03:38.239491+0000","last_peered":"2024-06-10T01:03:38.239491+0000","last_clean":"2024-06-10T01:03:38.239491+0000","last_became_active":"2024-06-10T01:01:57.332391+0000","last_became_peered":"2024-06-10T01:01:57.332391+0000","last_unstale":"2024-06-10T01:03:38.239491+0000","last_undegraded":"2024-06-10T01:03:38.239491+0000","last_fullsized":"2024-06-10T01:03:38.239491+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:58:31.443626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770788+0000","last_change":"2024-06-10T01:01:44.466861+0000","last_active":"2024-06-10T01:03:38.770788+0000","last_peered":"2024-06-10T01:03:38.770788+0000","last_clean":"2024-06-10T01:03:38.770788+0000","last_became_active":"2024-06-10T01:01:44.466553+0000","last_became_peered":"2024-06-10T01:01:44.466553+0000","last_unstale":"2024-06-10T01:03:38.770788+0000","last_undegraded":"2024-06-10T01:03:38.770788+0000","last_fullsized":"2024-06-10T01:03:38.770788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:55:52.545594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234920+0000","last_change":"2024-06-10T01:01:47.244972+0000","last_active":"2024-06-10T01:03:38.234920+0000","last_peered":"2024-06-10T01:03:38.234920+0000","last_clean":"2024-06-10T01:03:38.234920+0000","last_became_active":"2024-06-10T01:01:47.244649+0000","last_became_peered":"2024-06-10T01:01:47.244649+0000","last_unstale":"2024-06-10T01:03:38.234920+0000","last_undegraded":"2024-06-10T01:03:38.234920+0000","last_fullsized":"2024-06-10T01:03:38.234920+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:10:46.909200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235402+0000","last_change":"2024-06-10T01:02:37.592380+0000","last_active":"2024-06-10T01:03:38.235402+0000","last_peered":"2024-06-10T01:03:38.235402+0000","last_clean":"2024-06-10T01:03:38.235402+0000","last_became_active":"2024-06-10T01:02:37.591856+0000","last_became_peered":"2024-06-10T01:02:37.591856+0000","last_unstale":"2024-06-10T01:03:38.235402+0000","last_undegraded":"2024-06-10T01:03:38.235402+0000","last_fullsized":"2024-06-10T01:03:38.235402+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:00:25.717192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239006+0000","last_change":"2024-06-10T01:01:44.008116+0000","last_active":"2024-06-10T01:03:38.239006+0000","last_peered":"2024-06-10T01:03:38.239006+0000","last_clean":"2024-06-10T01:03:38.239006+0000","last_became_active":"2024-06-10T01:01:44.007778+0000","last_became_peered":"2024-06-10T01:01:44.007778+0000","last_unstale":"2024-06-10T01:03:38.239006+0000","last_undegraded":"2024-06-10T01:03:38.239006+0000","last_fullsized":"2024-06-10T01:03:38.239006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:18:57.267296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.237657+0000","last_change":"2024-06-10T01:01:47.252833+0000","last_active":"2024-06-10T01:03:38.237657+0000","last_peered":"2024-06-10T01:03:38.237657+0000","last_clean":"2024-06-10T01:03:38.237657+0000","last_became_active":"2024-06-10T01:01:47.252326+0000","last_became_peered":"2024-06-10T01:01:47.252326+0000","last_unstale":"2024-06-10T01:03:38.237657+0000","last_undegraded":"2024-06-10T01:03:38.237657+0000","last_fullsized":"2024-06-10T01:03:38.237657+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:00:42.673838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239284+0000","last_change":"2024-06-10T01:01:57.275975+0000","last_active":"2024-06-10T01:03:38.239284+0000","last_peered":"2024-06-10T01:03:38.239284+0000","last_clean":"2024-06-10T01:03:38.239284+0000","last_became_active":"2024-06-10T01:01:57.275408+0000","last_became_peered":"2024-06-10T01:01:57.275408+0000","last_unstale":"2024-06-10T01:03:38.239284+0000","last_undegraded":"2024-06-10T01:03:38.239284+0000","last_fullsized":"2024-06-10T01:03:38.239284+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:12:42.376636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'1","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.243874+0000","last_change":"2024-06-10T01:03:38.243422+0000","last_active":"2024-06-10T01:03:38.243874+0000","last_peered":"2024-06-10T01:03:38.243874+0000","last_clean":"2024-06-10T01:03:38.243874+0000","last_became_active":"2024-06-10T01:03:38.239916+0000","last_became_peered":"2024-06-10T01:03:38.239916+0000","last_unstale":"2024-06-10T01:03:38.243874+0000","last_undegraded":"2024-06-10T01:03:38.243874+0000","last_fullsized":"2024-06-10T01:03:38.243874+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:30:01.399956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239179+0000","last_change":"2024-06-10T01:01:47.243154+0000","last_active":"2024-06-10T01:03:38.239179+0000","last_peered":"2024-06-10T01:03:38.239179+0000","last_clean":"2024-06-10T01:03:38.239179+0000","last_became_active":"2024-06-10T01:01:47.242851+0000","last_became_peered":"2024-06-10T01:01:47.242851+0000","last_unstale":"2024-06-10T01:03:38.239179+0000","last_undegraded":"2024-06-10T01:03:38.239179+0000","last_fullsized":"2024-06-10T01:03:38.239179+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:41:05.857777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239584+0000","last_change":"2024-06-10T01:01:57.333613+0000","last_active":"2024-06-10T01:03:38.239584+0000","last_peered":"2024-06-10T01:03:38.239584+0000","last_clean":"2024-06-10T01:03:38.239584+0000","last_became_active":"2024-06-10T01:01:57.333019+0000","last_became_peered":"2024-06-10T01:01:57.333019+0000","last_unstale":"2024-06-10T01:03:38.239584+0000","last_undegraded":"2024-06-10T01:03:38.239584+0000","last_fullsized":"2024-06-10T01:03:38.239584+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:11.912885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235303+0000","last_change":"2024-06-10T01:01:44.008221+0000","last_active":"2024-06-10T01:03:38.235303+0000","last_peered":"2024-06-10T01:03:38.235303+0000","last_clean":"2024-06-10T01:03:38.235303+0000","last_became_active":"2024-06-10T01:01:44.007880+0000","last_became_peered":"2024-06-10T01:01:44.007880+0000","last_unstale":"2024-06-10T01:03:38.235303+0000","last_undegraded":"2024-06-10T01:03:38.235303+0000","last_fullsized":"2024-06-10T01:03:38.235303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:40:33.162925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235307+0000","last_change":"2024-06-10T01:01:47.254902+0000","last_active":"2024-06-10T01:03:38.235307+0000","last_peered":"2024-06-10T01:03:38.235307+0000","last_clean":"2024-06-10T01:03:38.235307+0000","last_became_active":"2024-06-10T01:01:47.254555+0000","last_became_peered":"2024-06-10T01:01:47.254555+0000","last_unstale":"2024-06-10T01:03:38.235307+0000","last_undegraded":"2024-06-10T01:03:38.235307+0000","last_fullsized":"2024-06-10T01:03:38.235307+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:03:17.571001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770786+0000","last_change":"2024-06-10T01:01:57.284491+0000","last_active":"2024-06-10T01:03:38.770786+0000","last_peered":"2024-06-10T01:03:38.770786+0000","last_clean":"2024-06-10T01:03:38.770786+0000","last_became_active":"2024-06-10T01:01:57.283795+0000","last_became_peered":"2024-06-10T01:01:57.283795+0000","last_unstale":"2024-06-10T01:03:38.770786+0000","last_undegraded":"2024-06-10T01:03:38.770786+0000","last_fullsized":"2024-06-10T01:03:38.770786+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:49:00.381664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239889+0000","last_change":"2024-06-10T01:01:44.553159+0000","last_active":"2024-06-10T01:03:38.239889+0000","last_peered":"2024-06-10T01:03:38.239889+0000","last_clean":"2024-06-10T01:03:38.239889+0000","last_became_active":"2024-06-10T01:01:44.552759+0000","last_became_peered":"2024-06-10T01:01:44.552759+0000","last_unstale":"2024-06-10T01:03:38.239889+0000","last_undegraded":"2024-06-10T01:03:38.239889+0000","last_fullsized":"2024-06-10T01:03:38.239889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:19:00.110790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235665+0000","last_change":"2024-06-10T01:01:47.247803+0000","last_active":"2024-06-10T01:03:38.235665+0000","last_peered":"2024-06-10T01:03:38.235665+0000","last_clean":"2024-06-10T01:03:38.235665+0000","last_became_active":"2024-06-10T01:01:47.246715+0000","last_became_peered":"2024-06-10T01:01:47.246715+0000","last_unstale":"2024-06-10T01:03:38.235665+0000","last_undegraded":"2024-06-10T01:03:38.235665+0000","last_fullsized":"2024-06-10T01:03:38.235665+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:22.429002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234192+0000","last_change":"2024-06-10T01:01:57.287668+0000","last_active":"2024-06-10T01:03:38.234192+0000","last_peered":"2024-06-10T01:03:38.234192+0000","last_clean":"2024-06-10T01:03:38.234192+0000","last_became_active":"2024-06-10T01:01:57.287060+0000","last_became_peered":"2024-06-10T01:01:57.287060+0000","last_unstale":"2024-06-10T01:03:38.234192+0000","last_undegraded":"2024-06-10T01:03:38.234192+0000","last_fullsized":"2024-06-10T01:03:38.234192+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:04:35.883613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270251+0000","last_change":"2024-06-10T01:01:44.013051+0000","last_active":"2024-06-10T01:03:38.270251+0000","last_peered":"2024-06-10T01:03:38.270251+0000","last_clean":"2024-06-10T01:03:38.270251+0000","last_became_active":"2024-06-10T01:01:44.012667+0000","last_became_peered":"2024-06-10T01:01:44.012667+0000","last_unstale":"2024-06-10T01:03:38.270251+0000","last_undegraded":"2024-06-10T01:03:38.270251+0000","last_fullsized":"2024-06-10T01:03:38.270251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:10:49.650082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235239+0000","last_change":"2024-06-10T01:01:47.251209+0000","last_active":"2024-06-10T01:03:38.235239+0000","last_peered":"2024-06-10T01:03:38.235239+0000","last_clean":"2024-06-10T01:03:38.235239+0000","last_became_active":"2024-06-10T01:01:47.250858+0000","last_became_peered":"2024-06-10T01:01:47.250858+0000","last_unstale":"2024-06-10T01:03:38.235239+0000","last_undegraded":"2024-06-10T01:03:38.235239+0000","last_fullsized":"2024-06-10T01:03:38.235239+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:47.272727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.269600+0000","last_change":"2024-06-10T01:01:57.284187+0000","last_active":"2024-06-10T01:03:38.269600+0000","last_peered":"2024-06-10T01:03:38.269600+0000","last_clean":"2024-06-10T01:03:38.269600+0000","last_became_active":"2024-06-10T01:01:57.283739+0000","last_became_peered":"2024-06-10T01:01:57.283739+0000","last_unstale":"2024-06-10T01:03:38.269600+0000","last_undegraded":"2024-06-10T01:03:38.269600+0000","last_fullsized":"2024-06-10T01:03:38.269600+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:13:34.630708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771085+0000","last_change":"2024-06-10T01:01:44.005702+0000","last_active":"2024-06-10T01:03:38.771085+0000","last_peered":"2024-06-10T01:03:38.771085+0000","last_clean":"2024-06-10T01:03:38.771085+0000","last_became_active":"2024-06-10T01:01:44.005354+0000","last_became_peered":"2024-06-10T01:01:44.005354+0000","last_unstale":"2024-06-10T01:03:38.771085+0000","last_undegraded":"2024-06-10T01:03:38.771085+0000","last_fullsized":"2024-06-10T01:03:38.771085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:51:14.819688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235610+0000","last_change":"2024-06-10T01:01:47.250325+0000","last_active":"2024-06-10T01:03:38.235610+0000","last_peered":"2024-06-10T01:03:38.235610+0000","last_clean":"2024-06-10T01:03:38.235610+0000","last_became_active":"2024-06-10T01:01:47.249591+0000","last_became_peered":"2024-06-10T01:01:47.249591+0000","last_unstale":"2024-06-10T01:03:38.235610+0000","last_undegraded":"2024-06-10T01:03:38.235610+0000","last_fullsized":"2024-06-10T01:03:38.235610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:17:16.268717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.237787+0000","last_change":"2024-06-10T01:01:47.253449+0000","last_active":"2024-06-10T01:03:38.237787+0000","last_peered":"2024-06-10T01:03:38.237787+0000","last_clean":"2024-06-10T01:03:38.237787+0000","last_became_active":"2024-06-10T01:01:47.253144+0000","last_became_peered":"2024-06-10T01:01:47.253144+0000","last_unstale":"2024-06-10T01:03:38.237787+0000","last_undegraded":"2024-06-10T01:03:38.237787+0000","last_fullsized":"2024-06-10T01:03:38.237787+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:32:22.353409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239727+0000","last_change":"2024-06-10T01:01:47.258241+0000","last_active":"2024-06-10T01:03:38.239727+0000","last_peered":"2024-06-10T01:03:38.239727+0000","last_clean":"2024-06-10T01:03:38.239727+0000","last_became_active":"2024-06-10T01:01:47.257823+0000","last_became_peered":"2024-06-10T01:01:47.257823+0000","last_unstale":"2024-06-10T01:03:38.239727+0000","last_undegraded":"2024-06-10T01:03:38.239727+0000","last_fullsized":"2024-06-10T01:03:38.239727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:11:23.652479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239265+0000","last_change":"2024-06-10T01:01:47.256218+0000","last_active":"2024-06-10T01:03:38.239265+0000","last_peered":"2024-06-10T01:03:38.239265+0000","last_clean":"2024-06-10T01:03:38.239265+0000","last_became_active":"2024-06-10T01:01:47.255950+0000","last_became_peered":"2024-06-10T01:01:47.255950+0000","last_unstale":"2024-06-10T01:03:38.239265+0000","last_undegraded":"2024-06-10T01:03:38.239265+0000","last_fullsized":"2024-06-10T01:03:38.239265+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:59:37.283646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234361+0000","last_change":"2024-06-10T01:01:47.253109+0000","last_active":"2024-06-10T01:03:38.234361+0000","last_peered":"2024-06-10T01:03:38.234361+0000","last_clean":"2024-06-10T01:03:38.234361+0000","last_became_active":"2024-06-10T01:01:47.252767+0000","last_became_peered":"2024-06-10T01:01:47.252767+0000","last_unstale":"2024-06-10T01:03:38.234361+0000","last_undegraded":"2024-06-10T01:03:38.234361+0000","last_fullsized":"2024-06-10T01:03:38.234361+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:57:13.177230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771539+0000","last_change":"2024-06-10T01:02:37.375094+0000","last_active":"2024-06-10T01:03:38.771539+0000","last_peered":"2024-06-10T01:03:38.771539+0000","last_clean":"2024-06-10T01:03:38.771539+0000","last_became_active":"2024-06-10T01:02:37.374527+0000","last_became_peered":"2024-06-10T01:02:37.374527+0000","last_unstale":"2024-06-10T01:03:38.771539+0000","last_undegraded":"2024-06-10T01:03:38.771539+0000","last_fullsized":"2024-06-10T01:03:38.771539+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:34:41.205816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.237368+0000","last_change":"2024-06-10T01:01:47.250243+0000","last_active":"2024-06-10T01:03:38.237368+0000","last_peered":"2024-06-10T01:03:38.237368+0000","last_clean":"2024-06-10T01:03:38.237368+0000","last_became_active":"2024-06-10T01:01:47.249907+0000","last_became_peered":"2024-06-10T01:01:47.249907+0000","last_unstale":"2024-06-10T01:03:38.237368+0000","last_undegraded":"2024-06-10T01:03:38.237368+0000","last_fullsized":"2024-06-10T01:03:38.237368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:31:32.933660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239994+0000","last_change":"2024-06-10T01:01:47.254173+0000","last_active":"2024-06-10T01:03:38.239994+0000","last_peered":"2024-06-10T01:03:38.239994+0000","last_clean":"2024-06-10T01:03:38.239994+0000","last_became_active":"2024-06-10T01:01:47.253796+0000","last_became_peered":"2024-06-10T01:01:47.253796+0000","last_unstale":"2024-06-10T01:03:38.239994+0000","last_undegraded":"2024-06-10T01:03:38.239994+0000","last_fullsized":"2024-06-10T01:03:38.239994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:41:12.199114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239585+0000","last_change":"2024-06-10T01:01:47.253920+0000","last_active":"2024-06-10T01:03:38.239585+0000","last_peered":"2024-06-10T01:03:38.239585+0000","last_clean":"2024-06-10T01:03:38.239585+0000","last_became_active":"2024-06-10T01:01:47.253288+0000","last_became_peered":"2024-06-10T01:01:47.253288+0000","last_unstale":"2024-06-10T01:03:38.239585+0000","last_undegraded":"2024-06-10T01:03:38.239585+0000","last_fullsized":"2024-06-10T01:03:38.239585+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:41:20.406263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.269809+0000","last_change":"2024-06-10T01:01:47.240772+0000","last_active":"2024-06-10T01:03:38.269809+0000","last_peered":"2024-06-10T01:03:38.269809+0000","last_clean":"2024-06-10T01:03:38.269809+0000","last_became_active":"2024-06-10T01:01:47.238258+0000","last_became_peered":"2024-06-10T01:01:47.238258+0000","last_unstale":"2024-06-10T01:03:38.269809+0000","last_undegraded":"2024-06-10T01:03:38.269809+0000","last_fullsized":"2024-06-10T01:03:38.269809+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:55:25.211434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770662+0000","last_change":"2024-06-10T01:02:37.376724+0000","last_active":"2024-06-10T01:03:38.770662+0000","last_peered":"2024-06-10T01:03:38.770662+0000","last_clean":"2024-06-10T01:03:38.770662+0000","last_became_active":"2024-06-10T01:02:37.376329+0000","last_became_peered":"2024-06-10T01:02:37.376329+0000","last_unstale":"2024-06-10T01:03:38.770662+0000","last_undegraded":"2024-06-10T01:03:38.770662+0000","last_fullsized":"2024-06-10T01:03:38.770662+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:38:21.837958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771019+0000","last_change":"2024-06-10T01:01:47.244463+0000","last_active":"2024-06-10T01:03:38.771019+0000","last_peered":"2024-06-10T01:03:38.771019+0000","last_clean":"2024-06-10T01:03:38.771019+0000","last_became_active":"2024-06-10T01:01:47.244114+0000","last_became_peered":"2024-06-10T01:01:47.244114+0000","last_unstale":"2024-06-10T01:03:38.771019+0000","last_undegraded":"2024-06-10T01:03:38.771019+0000","last_fullsized":"2024-06-10T01:03:38.771019+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:47:37.098894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771060+0000","last_change":"2024-06-10T01:01:47.248317+0000","last_active":"2024-06-10T01:03:38.771060+0000","last_peered":"2024-06-10T01:03:38.771060+0000","last_clean":"2024-06-10T01:03:38.771060+0000","last_became_active":"2024-06-10T01:01:47.247958+0000","last_became_peered":"2024-06-10T01:01:47.247958+0000","last_unstale":"2024-06-10T01:03:38.771060+0000","last_undegraded":"2024-06-10T01:03:38.771060+0000","last_fullsized":"2024-06-10T01:03:38.771060+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:39:59.267957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.237181+0000","last_change":"2024-06-10T01:01:47.252542+0000","last_active":"2024-06-10T01:03:38.237181+0000","last_peered":"2024-06-10T01:03:38.237181+0000","last_clean":"2024-06-10T01:03:38.237181+0000","last_became_active":"2024-06-10T01:01:47.252216+0000","last_became_peered":"2024-06-10T01:01:47.252216+0000","last_unstale":"2024-06-10T01:03:38.237181+0000","last_undegraded":"2024-06-10T01:03:38.237181+0000","last_fullsized":"2024-06-10T01:03:38.237181+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:12:22.196263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285691,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67668,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":598,"num_read_kb":370,"num_write":170,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":203004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383729,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1101824,"data_stored":569843,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645366,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32392,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712760,"statfs":{"total":95995035648,"available":95961866240,"internally_reserved":0,"allocated":1241088,"data_stored":1221057,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808891,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1114112,"data_stored":574063,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070528,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":569775,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332164,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1159168,"data_stored":1165405,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495689,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":565143,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574946,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1159168,"data_stored":629111,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738472,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1114112,"data_stored":566531,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411421,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28172,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716980,"statfs":{"total":95995035648,"available":95966187520,"internally_reserved":0,"allocated":1175552,"data_stored":630349,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182489,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":565337,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920851,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1110016,"data_stored":569905,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757327,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1167360,"data_stored":1160993,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34687,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34687,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34687,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":65296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T01:04:35.812 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-10T01:04:35.812 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-06-10T01:04:35.812 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-06-10T01:04:35.813 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph health --format=json 2024-06-10T01:04:36.063 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:36.092 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:36 smithi007 ceph-mon[32398]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T01:04:36.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:36 smithi175 ceph-mon[36636]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T01:04:36.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:36 smithi078 ceph-mon[36683]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T01:04:37.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:37 smithi175 ceph-mon[36636]: pgmap v389: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:04:37.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:37 smithi175 ceph-mon[36636]: mgrmap e18: x(active, since 10m), standbys: y, z 2024-06-10T01:04:37.474 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:37 smithi007 ceph-mon[32398]: pgmap v389: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:04:37.474 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:37 smithi007 ceph-mon[32398]: mgrmap e18: x(active, since 10m), standbys: y, z 2024-06-10T01:04:37.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:37 smithi078 ceph-mon[36683]: pgmap v389: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:04:37.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:37 smithi078 ceph-mon[36683]: mgrmap e18: x(active, since 10m), standbys: y, z 2024-06-10T01:04:37.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.858+0000 7fc2bc6c5700 1 -- 172.21.15.7:0/2018918045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2b41024e0 msgr2=0x7fc2b4102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:37.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.858+0000 7fc2bc6c5700 1 --2- 172.21.15.7:0/2018918045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2b41024e0 0x7fc2b4102940 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009990 tx=0x7fc2a4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:04:37.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.859+0000 7fc2bc6c5700 1 -- 172.21.15.7:0/2018918045 shutdown_connections 2024-06-10T01:04:37.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.859+0000 7fc2bc6c5700 1 --2- 172.21.15.7:0/2018918045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b4102e80 0x7fc2b410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:37.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.859+0000 7fc2bc6c5700 1 --2- 172.21.15.7:0/2018918045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2b41084f0 0x7fc2b41088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:37.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.859+0000 7fc2bc6c5700 1 --2- 172.21.15.7:0/2018918045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2b41024e0 0x7fc2b4102940 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:37.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.859+0000 7fc2bc6c5700 1 -- 172.21.15.7:0/2018918045 >> 172.21.15.7:0/2018918045 conn(0x7fc2b4079530 msgr2=0x7fc2b4079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:37.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.859+0000 7fc2bc6c5700 1 -- 172.21.15.7:0/2018918045 shutdown_connections 2024-06-10T01:04:37.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.859+0000 7fc2bc6c5700 1 -- 172.21.15.7:0/2018918045 wait complete. 2024-06-10T01:04:37.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.859+0000 7fc2bc6c5700 1 Processor -- start 2024-06-10T01:04:37.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.859+0000 7fc2bc6c5700 1 -- start start 2024-06-10T01:04:37.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.860+0000 7fc2bc6c5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2b41024e0 0x7fc2b419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:37.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.860+0000 7fc2bc6c5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b4102e80 0x7fc2b419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:37.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.860+0000 7fc2bc6c5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2b41084f0 0x7fc2b41989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:37.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.860+0000 7fc2bc6c5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b4112750 con 0x7fc2b41024e0 2024-06-10T01:04:37.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.860+0000 7fc2bc6c5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b41125d0 con 0x7fc2b4102e80 2024-06-10T01:04:37.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.860+0000 7fc2bc6c5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b41128d0 con 0x7fc2b41084f0 2024-06-10T01:04:37.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.860+0000 7fc2ba461700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2b41024e0 0x7fc2b419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:37.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.860+0000 7fc2ba461700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2b41024e0 0x7fc2b419e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55586/0 (socket says 172.21.15.7:55586) 2024-06-10T01:04:37.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.860+0000 7fc2ba461700 1 -- 172.21.15.7:0/3362815616 learned_addr learned my addr 172.21.15.7:0/3362815616 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:37.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.860+0000 7fc2b9c60700 1 --2- 172.21.15.7:0/3362815616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b4102e80 0x7fc2b419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:37.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.860+0000 7fc2bac62700 1 --2- 172.21.15.7:0/3362815616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2b41084f0 0x7fc2b41989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:37.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.860+0000 7fc2ba461700 1 -- 172.21.15.7:0/3362815616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2b41084f0 msgr2=0x7fc2b41989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:37.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.860+0000 7fc2ba461700 1 --2- 172.21.15.7:0/3362815616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2b41084f0 0x7fc2b41989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:37.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.860+0000 7fc2ba461700 1 -- 172.21.15.7:0/3362815616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b4102e80 msgr2=0x7fc2b419ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:37.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.860+0000 7fc2ba461700 1 --2- 172.21.15.7:0/3362815616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b4102e80 0x7fc2b419ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:37.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.860+0000 7fc2ba461700 1 -- 172.21.15.7:0/3362815616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b4199150 con 0x7fc2b41024e0 2024-06-10T01:04:37.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.860+0000 7fc2ba461700 1 --2- 172.21.15.7:0/3362815616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2b41024e0 0x7fc2b419e830 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc2b000d950 tx=0x7fc2b000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:37.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.860+0000 7fc2ab7fe700 1 -- 172.21.15.7:0/3362815616 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0016070 con 0x7fc2b41024e0 2024-06-10T01:04:37.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.860+0000 7fc2ab7fe700 1 -- 172.21.15.7:0/3362815616 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2b000fd70 con 0x7fc2b41024e0 2024-06-10T01:04:37.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.860+0000 7fc2bc6c5700 1 -- 172.21.15.7:0/3362815616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2b4199440 con 0x7fc2b41024e0 2024-06-10T01:04:37.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.861+0000 7fc2bc6c5700 1 -- 172.21.15.7:0/3362815616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2b40746c0 con 0x7fc2b41024e0 2024-06-10T01:04:37.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.861+0000 7fc2ab7fe700 1 -- 172.21.15.7:0/3362815616 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b000e610 con 0x7fc2b41024e0 2024-06-10T01:04:37.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.862+0000 7fc2bc6c5700 1 -- 172.21.15.7:0/3362815616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2b4069180 con 0x7fc2b41024e0 2024-06-10T01:04:37.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.863+0000 7fc2ab7fe700 1 -- 172.21.15.7:0/3362815616 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fc2b0019070 con 0x7fc2b41024e0 2024-06-10T01:04:37.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.863+0000 7fc2ab7fe700 1 --2- 172.21.15.7:0/3362815616 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc2a00a2ca0 0x7fc2a00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:37.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.864+0000 7fc2b9c60700 1 --2- 172.21.15.7:0/3362815616 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc2a00a2ca0 0x7fc2a00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:37.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.864+0000 7fc2ab7fe700 1 -- 172.21.15.7:0/3362815616 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fc2b00cf110 con 0x7fc2b41024e0 2024-06-10T01:04:37.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.864+0000 7fc2b9c60700 1 --2- 172.21.15.7:0/3362815616 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc2a00a2ca0 0x7fc2a00a5160 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400b2e0 tx=0x7fc2a4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:37.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:37.868+0000 7fc2ab7fe700 1 -- 172.21.15.7:0/3362815616 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc2b009bcf0 con 0x7fc2b41024e0 2024-06-10T01:04:38.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:38.123+0000 7fc2bc6c5700 1 -- 172.21.15.7:0/3362815616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc2b402dca0 con 0x7fc2b41024e0 2024-06-10T01:04:38.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:38.123+0000 7fc2ab7fe700 1 -- 172.21.15.7:0/3362815616 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fc2b009b5f0 con 0x7fc2b41024e0 2024-06-10T01:04:38.122 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:04:38.122 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-06-10T01:04:38.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:38.126+0000 7fc2bc6c5700 1 -- 172.21.15.7:0/3362815616 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc2a00a2ca0 msgr2=0x7fc2a00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:38.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:38.126+0000 7fc2bc6c5700 1 --2- 172.21.15.7:0/3362815616 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc2a00a2ca0 0x7fc2a00a5160 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400b2e0 tx=0x7fc2a4005960 comp rx=0 tx=0).stop 2024-06-10T01:04:38.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:38.126+0000 7fc2bc6c5700 1 -- 172.21.15.7:0/3362815616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2b41024e0 msgr2=0x7fc2b419e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:38.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:38.126+0000 7fc2bc6c5700 1 --2- 172.21.15.7:0/3362815616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2b41024e0 0x7fc2b419e830 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc2b000d950 tx=0x7fc2b000dc60 comp rx=0 tx=0).stop 2024-06-10T01:04:38.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:38.126+0000 7fc2bc6c5700 1 -- 172.21.15.7:0/3362815616 shutdown_connections 2024-06-10T01:04:38.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:38.126+0000 7fc2bc6c5700 1 --2- 172.21.15.7:0/3362815616 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc2a00a2ca0 0x7fc2a00a5160 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:38.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:38.126+0000 7fc2bc6c5700 1 --2- 172.21.15.7:0/3362815616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2b41084f0 0x7fc2b41989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:38.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:38.126+0000 7fc2bc6c5700 1 --2- 172.21.15.7:0/3362815616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2b41024e0 0x7fc2b419e830 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:38.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:38.126+0000 7fc2bc6c5700 1 --2- 172.21.15.7:0/3362815616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b4102e80 0x7fc2b419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:38.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:38.126+0000 7fc2bc6c5700 1 -- 172.21.15.7:0/3362815616 >> 172.21.15.7:0/3362815616 conn(0x7fc2b4079530 msgr2=0x7fc2b4100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:38.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:38.126+0000 7fc2bc6c5700 1 -- 172.21.15.7:0/3362815616 shutdown_connections 2024-06-10T01:04:38.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:38.126+0000 7fc2bc6c5700 1 -- 172.21.15.7:0/3362815616 wait complete. 2024-06-10T01:04:38.780 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-06-10T01:04:38.780 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:04:39.034 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:39.369 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:39 smithi007 ceph-mon[32398]: pgmap v390: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:04:39.369 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:39 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3362815616' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-10T01:04:39.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:39 smithi175 ceph-mon[36636]: pgmap v390: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:04:39.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:39 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3362815616' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-10T01:04:39.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:39 smithi078 ceph-mon[36683]: pgmap v390: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:04:39.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:39 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3362815616' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-10T01:04:40.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.715+0000 7fa85ffb8700 1 -- 172.21.15.7:0/4177629821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8581084f0 msgr2=0x7fa8581088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:40.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.715+0000 7fa85ffb8700 1 --2- 172.21.15.7:0/4177629821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8581084f0 0x7fa8581088d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa84800b0a0 tx=0x7fa84800b3b0 comp rx=0 tx=0).stop 2024-06-10T01:04:40.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.715+0000 7fa85ffb8700 1 -- 172.21.15.7:0/4177629821 shutdown_connections 2024-06-10T01:04:40.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.715+0000 7fa85ffb8700 1 --2- 172.21.15.7:0/4177629821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa85806b3c0 0x7fa85810f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:40.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.715+0000 7fa85ffb8700 1 --2- 172.21.15.7:0/4177629821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8581084f0 0x7fa8581088d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:40.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.715+0000 7fa85ffb8700 1 --2- 172.21.15.7:0/4177629821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa85806aa00 0x7fa85806ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:40.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.715+0000 7fa85ffb8700 1 -- 172.21.15.7:0/4177629821 >> 172.21.15.7:0/4177629821 conn(0x7fa858078300 msgr2=0x7fa858078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:40.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.715+0000 7fa85ffb8700 1 -- 172.21.15.7:0/4177629821 shutdown_connections 2024-06-10T01:04:40.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.715+0000 7fa85ffb8700 1 -- 172.21.15.7:0/4177629821 wait complete. 2024-06-10T01:04:40.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.716+0000 7fa85ffb8700 1 Processor -- start 2024-06-10T01:04:40.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.716+0000 7fa85ffb8700 1 -- start start 2024-06-10T01:04:40.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.716+0000 7fa85ffb8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa85806aa00 0x7fa858101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:40.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.716+0000 7fa85ffb8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa85806b3c0 0x7fa858101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:40.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.716+0000 7fa85ffb8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8581084f0 0x7fa858101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:40.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.716+0000 7fa85ffb8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa8581127d0 con 0x7fa8581084f0 2024-06-10T01:04:40.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.716+0000 7fa85ffb8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa858112650 con 0x7fa85806b3c0 2024-06-10T01:04:40.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.716+0000 7fa85ffb8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa858112950 con 0x7fa85806aa00 2024-06-10T01:04:40.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.717+0000 7fa85e555700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8581084f0 0x7fa858101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:40.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.717+0000 7fa85e555700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8581084f0 0x7fa858101ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55612/0 (socket says 172.21.15.7:55612) 2024-06-10T01:04:40.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.717+0000 7fa85e555700 1 -- 172.21.15.7:0/3905411264 learned_addr learned my addr 172.21.15.7:0/3905411264 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:40.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.717+0000 7fa85dd54700 1 --2- 172.21.15.7:0/3905411264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa85806aa00 0x7fa858101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:40.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.717+0000 7fa85d553700 1 --2- 172.21.15.7:0/3905411264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa85806b3c0 0x7fa858101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:40.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.717+0000 7fa85e555700 1 -- 172.21.15.7:0/3905411264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa85806aa00 msgr2=0x7fa858101450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:40.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.717+0000 7fa85e555700 1 --2- 172.21.15.7:0/3905411264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa85806aa00 0x7fa858101450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:40.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.717+0000 7fa85e555700 1 -- 172.21.15.7:0/3905411264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa85806b3c0 msgr2=0x7fa858101990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:40.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.717+0000 7fa85e555700 1 --2- 172.21.15.7:0/3905411264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa85806b3c0 0x7fa858101990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:40.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.717+0000 7fa85e555700 1 -- 172.21.15.7:0/3905411264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8581a71f0 con 0x7fa8581084f0 2024-06-10T01:04:40.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.717+0000 7fa85e555700 1 --2- 172.21.15.7:0/3905411264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8581084f0 0x7fa858101ed0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa85000c970 tx=0x7fa85000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:40.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.717+0000 7fa856ffd700 1 -- 172.21.15.7:0/3905411264 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85000ce60 con 0x7fa8581084f0 2024-06-10T01:04:40.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.717+0000 7fa856ffd700 1 -- 172.21.15.7:0/3905411264 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa850004830 con 0x7fa8581084f0 2024-06-10T01:04:40.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.717+0000 7fa85ffb8700 1 -- 172.21.15.7:0/3905411264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8581a74e0 con 0x7fa8581084f0 2024-06-10T01:04:40.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.717+0000 7fa856ffd700 1 -- 172.21.15.7:0/3905411264 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8500055a0 con 0x7fa8581084f0 2024-06-10T01:04:40.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.718+0000 7fa85ffb8700 1 -- 172.21.15.7:0/3905411264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8581a79d0 con 0x7fa8581084f0 2024-06-10T01:04:40.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.719+0000 7fa85ffb8700 1 -- 172.21.15.7:0/3905411264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa85810a9d0 con 0x7fa8581084f0 2024-06-10T01:04:40.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.719+0000 7fa856ffd700 1 -- 172.21.15.7:0/3905411264 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa8500049e0 con 0x7fa8581084f0 2024-06-10T01:04:40.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.720+0000 7fa856ffd700 1 --2- 172.21.15.7:0/3905411264 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa8400a2c20 0x7fa8400a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:40.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.720+0000 7fa85dd54700 1 --2- 172.21.15.7:0/3905411264 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa8400a2c20 0x7fa8400a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:40.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.720+0000 7fa856ffd700 1 -- 172.21.15.7:0/3905411264 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fa8500cec40 con 0x7fa8581084f0 2024-06-10T01:04:40.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.721+0000 7fa85dd54700 1 --2- 172.21.15.7:0/3905411264 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa8400a2c20 0x7fa8400a50e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa84800b070 tx=0x7fa84800b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:40.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.725+0000 7fa856ffd700 1 -- 172.21.15.7:0/3905411264 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa850014030 con 0x7fa8581084f0 2024-06-10T01:04:40.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.976+0000 7fa85ffb8700 1 -- 172.21.15.7:0/3905411264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa85810a3b0 con 0x7fa8581084f0 2024-06-10T01:04:40.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.977+0000 7fa856ffd700 1 -- 172.21.15.7:0/3905411264 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11855 (secure 0 0 0) 0x7fa85009b7c0 con 0x7fa8581084f0 2024-06-10T01:04:40.974 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:04:40.975 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:02:58.523980+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/2784910673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2784910673},{"type":"v1","addr":"172.21.15.175:6835","nonce":2784910673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:04:40.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.979+0000 7fa85ffb8700 1 -- 172.21.15.7:0/3905411264 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa8400a2c20 msgr2=0x7fa8400a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:40.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.979+0000 7fa85ffb8700 1 --2- 172.21.15.7:0/3905411264 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa8400a2c20 0x7fa8400a50e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa84800b070 tx=0x7fa84800b7b0 comp rx=0 tx=0).stop 2024-06-10T01:04:40.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.979+0000 7fa85ffb8700 1 -- 172.21.15.7:0/3905411264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8581084f0 msgr2=0x7fa858101ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:40.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.979+0000 7fa85ffb8700 1 --2- 172.21.15.7:0/3905411264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8581084f0 0x7fa858101ed0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa85000c970 tx=0x7fa85000cc80 comp rx=0 tx=0).stop 2024-06-10T01:04:40.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.979+0000 7fa85ffb8700 1 -- 172.21.15.7:0/3905411264 shutdown_connections 2024-06-10T01:04:40.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.979+0000 7fa85ffb8700 1 --2- 172.21.15.7:0/3905411264 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa8400a2c20 0x7fa8400a50e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:40.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.979+0000 7fa85ffb8700 1 --2- 172.21.15.7:0/3905411264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8581084f0 0x7fa858101ed0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:40.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.979+0000 7fa85ffb8700 1 --2- 172.21.15.7:0/3905411264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa85806aa00 0x7fa858101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:40.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.979+0000 7fa85ffb8700 1 --2- 172.21.15.7:0/3905411264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa85806b3c0 0x7fa858101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:40.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.979+0000 7fa85ffb8700 1 -- 172.21.15.7:0/3905411264 >> 172.21.15.7:0/3905411264 conn(0x7fa858078300 msgr2=0x7fa858100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:40.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.980+0000 7fa85ffb8700 1 -- 172.21.15.7:0/3905411264 shutdown_connections 2024-06-10T01:04:40.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:40.980+0000 7fa85ffb8700 1 -- 172.21.15.7:0/3905411264 wait complete. 2024-06-10T01:04:40.977 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-06-10T01:04:41.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:41 smithi007 ceph-mon[32398]: pgmap v391: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:04:41.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:41 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3905411264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:04:41.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:41 smithi175 ceph-mon[36636]: pgmap v391: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:04:41.458 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:41 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3905411264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:04:41.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:41 smithi078 ceph-mon[36683]: pgmap v391: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:04:41.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:41 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3905411264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:04:41.738 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:04:41.997 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:43.379 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:43 smithi007 ceph-mon[32398]: pgmap v392: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:04:43.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:43 smithi175 ceph-mon[36636]: pgmap v392: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:04:43.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:43 smithi078 ceph-mon[36683]: pgmap v392: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:04:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.750+0000 7fda5f5f9700 1 -- 172.21.15.7:0/1212128238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda58101d50 msgr2=0x7fda5810e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.750+0000 7fda5f5f9700 1 --2- 172.21.15.7:0/1212128238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda58101d50 0x7fda5810e240 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fda5000b0a0 tx=0x7fda5000b3b0 comp rx=0 tx=0).stop 2024-06-10T01:04:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.750+0000 7fda5f5f9700 1 -- 172.21.15.7:0/1212128238 shutdown_connections 2024-06-10T01:04:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.750+0000 7fda5f5f9700 1 --2- 172.21.15.7:0/1212128238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda58101d50 0x7fda5810e240 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.750+0000 7fda5f5f9700 1 --2- 172.21.15.7:0/1212128238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda58107400 0x7fda581077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.750+0000 7fda5f5f9700 1 --2- 172.21.15.7:0/1212128238 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda58101390 0x7fda58101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.750+0000 7fda5f5f9700 1 -- 172.21.15.7:0/1212128238 >> 172.21.15.7:0/1212128238 conn(0x7fda58078340 msgr2=0x7fda58078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.750+0000 7fda5f5f9700 1 -- 172.21.15.7:0/1212128238 shutdown_connections 2024-06-10T01:04:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.750+0000 7fda5f5f9700 1 -- 172.21.15.7:0/1212128238 wait complete. 2024-06-10T01:04:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.751+0000 7fda5f5f9700 1 Processor -- start 2024-06-10T01:04:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.751+0000 7fda5f5f9700 1 -- start start 2024-06-10T01:04:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.751+0000 7fda5f5f9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda58101390 0x7fda58103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.751+0000 7fda5f5f9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda58101d50 0x7fda581035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.751+0000 7fda5f5f9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda58107400 0x7fda58103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.751+0000 7fda5f5f9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fda58111770 con 0x7fda58107400 2024-06-10T01:04:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.751+0000 7fda5f5f9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fda581115f0 con 0x7fda58101390 2024-06-10T01:04:43.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.751+0000 7fda5f5f9700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fda581118f0 con 0x7fda58101d50 2024-06-10T01:04:43.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.751+0000 7fda5db96700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda58107400 0x7fda58103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:43.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.751+0000 7fda5db96700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda58107400 0x7fda58103af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52662/0 (socket says 172.21.15.7:52662) 2024-06-10T01:04:43.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.751+0000 7fda5db96700 1 -- 172.21.15.7:0/653859967 learned_addr learned my addr 172.21.15.7:0/653859967 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:43.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.751+0000 7fda5d395700 1 --2- 172.21.15.7:0/653859967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda58101390 0x7fda58103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:43.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.752+0000 7fda5cb94700 1 --2- 172.21.15.7:0/653859967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda58101d50 0x7fda581035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:43.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.752+0000 7fda5db96700 1 -- 172.21.15.7:0/653859967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda58101d50 msgr2=0x7fda581035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:43.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.752+0000 7fda5db96700 1 --2- 172.21.15.7:0/653859967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda58101d50 0x7fda581035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:43.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.752+0000 7fda5db96700 1 -- 172.21.15.7:0/653859967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda58101390 msgr2=0x7fda58103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:43.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.752+0000 7fda5db96700 1 --2- 172.21.15.7:0/653859967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda58101390 0x7fda58103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:43.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.752+0000 7fda5db96700 1 -- 172.21.15.7:0/653859967 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda581a71e0 con 0x7fda58107400 2024-06-10T01:04:43.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.752+0000 7fda5db96700 1 --2- 172.21.15.7:0/653859967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda58107400 0x7fda58103af0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fda500062a0 tx=0x7fda5000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:43.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.752+0000 7fda4e7fc700 1 -- 172.21.15.7:0/653859967 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda50021070 con 0x7fda58107400 2024-06-10T01:04:43.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.752+0000 7fda5f5f9700 1 -- 172.21.15.7:0/653859967 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda581a7470 con 0x7fda58107400 2024-06-10T01:04:43.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.752+0000 7fda4e7fc700 1 -- 172.21.15.7:0/653859967 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda50007d00 con 0x7fda58107400 2024-06-10T01:04:43.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.752+0000 7fda4e7fc700 1 -- 172.21.15.7:0/653859967 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda50005460 con 0x7fda58107400 2024-06-10T01:04:43.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.752+0000 7fda5f5f9700 1 -- 172.21.15.7:0/653859967 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda581a7920 con 0x7fda58107400 2024-06-10T01:04:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.753+0000 7fda5f5f9700 1 -- 172.21.15.7:0/653859967 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda58069180 con 0x7fda58107400 2024-06-10T01:04:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.754+0000 7fda4e7fc700 1 -- 172.21.15.7:0/653859967 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fda50024070 con 0x7fda58107400 2024-06-10T01:04:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.755+0000 7fda4e7fc700 1 --2- 172.21.15.7:0/653859967 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fda440a2c20 0x7fda440a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.755+0000 7fda5d395700 1 --2- 172.21.15.7:0/653859967 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fda440a2c20 0x7fda440a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.755+0000 7fda4e7fc700 1 -- 172.21.15.7:0/653859967 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fda5000f020 con 0x7fda58107400 2024-06-10T01:04:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.756+0000 7fda5d395700 1 --2- 172.21.15.7:0/653859967 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fda440a2c20 0x7fda440a50e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fda54009a10 tx=0x7fda54009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:43.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:43.760+0000 7fda4e7fc700 1 -- 172.21.15.7:0/653859967 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fda5002b020 con 0x7fda58107400 2024-06-10T01:04:44.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:44.012+0000 7fda5f5f9700 1 -- 172.21.15.7:0/653859967 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda581a7bc0 con 0x7fda58107400 2024-06-10T01:04:44.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:44.012+0000 7fda4e7fc700 1 -- 172.21.15.7:0/653859967 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11855 (secure 0 0 0) 0x7fda5009d740 con 0x7fda58107400 2024-06-10T01:04:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:04:44.010 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:02:58.523980+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/2784910673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2784910673},{"type":"v1","addr":"172.21.15.175:6835","nonce":2784910673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:04:44.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:44.015+0000 7fda5f5f9700 1 -- 172.21.15.7:0/653859967 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fda440a2c20 msgr2=0x7fda440a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:44.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:44.015+0000 7fda5f5f9700 1 --2- 172.21.15.7:0/653859967 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fda440a2c20 0x7fda440a50e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fda54009a10 tx=0x7fda54009190 comp rx=0 tx=0).stop 2024-06-10T01:04:44.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:44.015+0000 7fda5f5f9700 1 -- 172.21.15.7:0/653859967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda58107400 msgr2=0x7fda58103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:44.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:44.015+0000 7fda5f5f9700 1 --2- 172.21.15.7:0/653859967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda58107400 0x7fda58103af0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fda500062a0 tx=0x7fda5000be60 comp rx=0 tx=0).stop 2024-06-10T01:04:44.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:44.015+0000 7fda5f5f9700 1 -- 172.21.15.7:0/653859967 shutdown_connections 2024-06-10T01:04:44.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:44.015+0000 7fda5f5f9700 1 --2- 172.21.15.7:0/653859967 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fda440a2c20 0x7fda440a50e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:44.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:44.015+0000 7fda5f5f9700 1 --2- 172.21.15.7:0/653859967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda58107400 0x7fda58103af0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:44.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:44.015+0000 7fda5f5f9700 1 --2- 172.21.15.7:0/653859967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda58101390 0x7fda58103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:44.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:44.015+0000 7fda5f5f9700 1 --2- 172.21.15.7:0/653859967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda58101d50 0x7fda581035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:44.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:44.015+0000 7fda5f5f9700 1 -- 172.21.15.7:0/653859967 >> 172.21.15.7:0/653859967 conn(0x7fda58078340 msgr2=0x7fda581005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:44.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:44.015+0000 7fda5f5f9700 1 -- 172.21.15.7:0/653859967 shutdown_connections 2024-06-10T01:04:44.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:44.015+0000 7fda5f5f9700 1 -- 172.21.15.7:0/653859967 wait complete. 2024-06-10T01:04:44.013 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-06-10T01:04:44.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:44 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/653859967' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:04:44.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:44 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/653859967' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:04:44.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:44 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/653859967' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:04:44.841 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T01:04:45.098 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:45.433 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:45 smithi007 ceph-mon[32398]: pgmap v393: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-10T01:04:45.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:45 smithi175 ceph-mon[36636]: pgmap v393: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-10T01:04:45.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:45 smithi078 ceph-mon[36683]: pgmap v393: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-10T01:04:46.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.718+0000 7f58a7e29700 1 -- 172.21.15.7:0/1985198605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58a0107840 msgr2=0x7f58a010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:46.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.719+0000 7f58a7e29700 1 --2- 172.21.15.7:0/1985198605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58a0107840 0x7f58a010f3c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f589800b0d0 tx=0x7f589800b3e0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.719+0000 7f58a7e29700 1 -- 172.21.15.7:0/1985198605 shutdown_connections 2024-06-10T01:04:46.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.719+0000 7f58a7e29700 1 --2- 172.21.15.7:0/1985198605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58a0107840 0x7f58a010f3c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.719+0000 7f58a7e29700 1 --2- 172.21.15.7:0/1985198605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58a0102740 0x7f58a0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.719+0000 7f58a7e29700 1 --2- 172.21.15.7:0/1985198605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58a01030f0 0x7f58a0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.719+0000 7f58a7e29700 1 -- 172.21.15.7:0/1985198605 >> 172.21.15.7:0/1985198605 conn(0x7f58a00795f0 msgr2=0x7f58a0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:46.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.719+0000 7f58a7e29700 1 -- 172.21.15.7:0/1985198605 shutdown_connections 2024-06-10T01:04:46.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.719+0000 7f58a7e29700 1 -- 172.21.15.7:0/1985198605 wait complete. 2024-06-10T01:04:46.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.719+0000 7f58a7e29700 1 Processor -- start 2024-06-10T01:04:46.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.719+0000 7f58a7e29700 1 -- start start 2024-06-10T01:04:46.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.720+0000 7f58a7e29700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58a0102740 0x7f58a019e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:46.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.720+0000 7f58a7e29700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58a01030f0 0x7f58a019ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:46.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.720+0000 7f58a7e29700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58a0107840 0x7f58a0198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:46.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.720+0000 7f58a7e29700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f58a0112760 con 0x7f58a0107840 2024-06-10T01:04:46.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.720+0000 7f58a7e29700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f58a01125e0 con 0x7f58a01030f0 2024-06-10T01:04:46.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.720+0000 7f58a7e29700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f58a01128e0 con 0x7f58a0102740 2024-06-10T01:04:46.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.720+0000 7f58a63c6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58a0107840 0x7f58a0198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:46.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.720+0000 7f58a63c6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58a0107840 0x7f58a0198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52682/0 (socket says 172.21.15.7:52682) 2024-06-10T01:04:46.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.720+0000 7f58a63c6700 1 -- 172.21.15.7:0/428402683 learned_addr learned my addr 172.21.15.7:0/428402683 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:46.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.720+0000 7f58a5bc5700 1 --2- 172.21.15.7:0/428402683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58a0102740 0x7f58a019e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:46.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.720+0000 7f58a53c4700 1 --2- 172.21.15.7:0/428402683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58a01030f0 0x7f58a019ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:46.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.720+0000 7f58a63c6700 1 -- 172.21.15.7:0/428402683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58a0102740 msgr2=0x7f58a019e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:46.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.720+0000 7f58a63c6700 1 --2- 172.21.15.7:0/428402683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58a0102740 0x7f58a019e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.720+0000 7f58a63c6700 1 -- 172.21.15.7:0/428402683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58a01030f0 msgr2=0x7f58a019ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:46.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.720+0000 7f58a63c6700 1 --2- 172.21.15.7:0/428402683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58a01030f0 0x7f58a019ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.720+0000 7f58a63c6700 1 -- 172.21.15.7:0/428402683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a01990e0 con 0x7f58a0107840 2024-06-10T01:04:46.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.720+0000 7f58a63c6700 1 --2- 172.21.15.7:0/428402683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58a0107840 0x7f58a0198980 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5898006070 tx=0x7f5898004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:46.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.720+0000 7f5896ffd700 1 -- 172.21.15.7:0/428402683 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5898021070 con 0x7f58a0107840 2024-06-10T01:04:46.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.720+0000 7f5896ffd700 1 -- 172.21.15.7:0/428402683 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5898007b80 con 0x7f58a0107840 2024-06-10T01:04:46.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.720+0000 7f58a7e29700 1 -- 172.21.15.7:0/428402683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a0199370 con 0x7f58a0107840 2024-06-10T01:04:46.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.720+0000 7f5896ffd700 1 -- 172.21.15.7:0/428402683 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5898005330 con 0x7f58a0107840 2024-06-10T01:04:46.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.720+0000 7f58a7e29700 1 -- 172.21.15.7:0/428402683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a01a5640 con 0x7f58a0107840 2024-06-10T01:04:46.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.721+0000 7f58a7e29700 1 -- 172.21.15.7:0/428402683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58a0069180 con 0x7f58a0107840 2024-06-10T01:04:46.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.723+0000 7f5896ffd700 1 -- 172.21.15.7:0/428402683 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f5898024070 con 0x7f58a0107840 2024-06-10T01:04:46.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.723+0000 7f5896ffd700 1 --2- 172.21.15.7:0/428402683 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f588c0a2c20 0x7f588c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:46.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.723+0000 7f58a5bc5700 1 --2- 172.21.15.7:0/428402683 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f588c0a2c20 0x7f588c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:46.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.723+0000 7f5896ffd700 1 -- 172.21.15.7:0/428402683 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f589800f020 con 0x7f58a0107840 2024-06-10T01:04:46.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.724+0000 7f58a5bc5700 1 --2- 172.21.15.7:0/428402683 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f588c0a2c20 0x7f588c0a50e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f589c005950 tx=0x7f589c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:46.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.727+0000 7f5896ffd700 1 -- 172.21.15.7:0/428402683 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f589802b020 con 0x7f58a0107840 2024-06-10T01:04:46.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.934+0000 7f58a7e29700 1 -- 172.21.15.7:0/428402683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58a002dca0 con 0x7f58a0107840 2024-06-10T01:04:46.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.936+0000 7f5896ffd700 1 -- 172.21.15.7:0/428402683 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24615 (secure 0 0 0) 0x7f589809db10 con 0x7f58a0107840 2024-06-10T01:04:46.933 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:04:46.934 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":99,"fsid":"bded58f8-26c3-11ef-bc9b-c7b262605968","created":"2024-06-10T00:53:31.966085+0000","modified":"2024-06-10T01:03:38.227703+0000","last_up_change":"2024-06-10T01:00:45.382473+0000","last_in_change":"2024-06-10T01:00:25.608513+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:56:59.964055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T01:01:42.563611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T01:01:46.175918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T01:01:55.695502+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f7c24c9f-b7ab-4c04-b2b4-120df9751667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6803","nonce":404485726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6805","nonce":404485726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6809","nonce":404485726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6807","nonce":404485726}]},"public_addr":"172.21.15.7:6803/404485726","cluster_addr":"172.21.15.7:6805/404485726","heartbeat_back_addr":"172.21.15.7:6809/404485726","heartbeat_front_addr":"172.21.15.7:6807/404485726","state":["exists","up"]},{"osd":1,"uuid":"9bf70a6b-f31c-4077-b5af-229f17bc2845","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6801","nonce":1856804448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6803","nonce":1856804448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6807","nonce":1856804448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6805","nonce":1856804448}]},"public_addr":"172.21.15.78:6801/1856804448","cluster_addr":"172.21.15.78:6803/1856804448","heartbeat_back_addr":"172.21.15.78:6807/1856804448","heartbeat_front_addr":"172.21.15.78:6805/1856804448","state":["exists","up"]},{"osd":2,"uuid":"62923c97-caf9-4f7e-9c14-9c9918843c78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6801","nonce":918257455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6803","nonce":918257455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6807","nonce":918257455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6805","nonce":918257455}]},"public_addr":"172.21.15.175:6801/918257455","cluster_addr":"172.21.15.175:6803/918257455","heartbeat_back_addr":"172.21.15.175:6807/918257455","heartbeat_front_addr":"172.21.15.175:6805/918257455","state":["exists","up"]},{"osd":3,"uuid":"65242098-e748-48ab-8f0e-bb25b5c62742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6811","nonce":1797404966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6813","nonce":1797404966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6817","nonce":1797404966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6815","nonce":1797404966}]},"public_addr":"172.21.15.7:6811/1797404966","cluster_addr":"172.21.15.7:6813/1797404966","heartbeat_back_addr":"172.21.15.7:6817/1797404966","heartbeat_front_addr":"172.21.15.7:6815/1797404966","state":["exists","up"]},{"osd":4,"uuid":"a7990305-de9e-400e-a432-d7854f73f339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6809","nonce":3767608357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6811","nonce":3767608357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6815","nonce":3767608357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6813","nonce":3767608357}]},"public_addr":"172.21.15.78:6809/3767608357","cluster_addr":"172.21.15.78:6811/3767608357","heartbeat_back_addr":"172.21.15.78:6815/3767608357","heartbeat_front_addr":"172.21.15.78:6813/3767608357","state":["exists","up"]},{"osd":5,"uuid":"bad9a7b3-da5a-48d0-905d-e6a3190317fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6809","nonce":2157664731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6811","nonce":2157664731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6815","nonce":2157664731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6813","nonce":2157664731}]},"public_addr":"172.21.15.175:6809/2157664731","cluster_addr":"172.21.15.175:6811/2157664731","heartbeat_back_addr":"172.21.15.175:6815/2157664731","heartbeat_front_addr":"172.21.15.175:6813/2157664731","state":["exists","up"]},{"osd":6,"uuid":"6c687653-157b-4b96-867b-a2eaa7d8ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6819","nonce":838731756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6821","nonce":838731756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6825","nonce":838731756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6823","nonce":838731756}]},"public_addr":"172.21.15.7:6819/838731756","cluster_addr":"172.21.15.7:6821/838731756","heartbeat_back_addr":"172.21.15.7:6825/838731756","heartbeat_front_addr":"172.21.15.7:6823/838731756","state":["exists","up"]},{"osd":7,"uuid":"db40ddf1-9fd9-4859-b8c4-75f79582ab91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6817","nonce":3511921773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6819","nonce":3511921773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6823","nonce":3511921773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6821","nonce":3511921773}]},"public_addr":"172.21.15.78:6817/3511921773","cluster_addr":"172.21.15.78:6819/3511921773","heartbeat_back_addr":"172.21.15.78:6823/3511921773","heartbeat_front_addr":"172.21.15.78:6821/3511921773","state":["exists","up"]},{"osd":8,"uuid":"0f01cbd9-b193-4c70-a33c-4984167bf422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6817","nonce":1546325211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6819","nonce":1546325211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6823","nonce":1546325211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6821","nonce":1546325211}]},"public_addr":"172.21.15.175:6817/1546325211","cluster_addr":"172.21.15.175:6819/1546325211","heartbeat_back_addr":"172.21.15.175:6823/1546325211","heartbeat_front_addr":"172.21.15.175:6821/1546325211","state":["exists","up"]},{"osd":9,"uuid":"e5480998-f7fb-49cd-9aed-52405c88d830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6827","nonce":1844550813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6829","nonce":1844550813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6833","nonce":1844550813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6831","nonce":1844550813}]},"public_addr":"172.21.15.7:6827/1844550813","cluster_addr":"172.21.15.7:6829/1844550813","heartbeat_back_addr":"172.21.15.7:6833/1844550813","heartbeat_front_addr":"172.21.15.7:6831/1844550813","state":["exists","up"]},{"osd":10,"uuid":"caa82f33-a71c-4907-bf3b-30a7fdfeb71d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6825","nonce":2290824785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6827","nonce":2290824785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6831","nonce":2290824785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6829","nonce":2290824785}]},"public_addr":"172.21.15.78:6825/2290824785","cluster_addr":"172.21.15.78:6827/2290824785","heartbeat_back_addr":"172.21.15.78:6831/2290824785","heartbeat_front_addr":"172.21.15.78:6829/2290824785","state":["exists","up"]},{"osd":11,"uuid":"608cda77-77da-4da2-a7e2-76a773be27c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6825","nonce":1959979550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6827","nonce":1959979550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6831","nonce":1959979550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6829","nonce":1959979550}]},"public_addr":"172.21.15.175:6825/1959979550","cluster_addr":"172.21.15.175:6827/1959979550","heartbeat_back_addr":"172.21.15.175:6831/1959979550","heartbeat_front_addr":"172.21.15.175:6829/1959979550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:08.850203+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:32.820161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:56.633656+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:21.759372+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:46.735874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:10.636541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:36.990099+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:01.767098+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:26.882776+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:52.757595+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:18.383938+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:43.544330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1833397470":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/322543993":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/212510583":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6801/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6800/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/2679436843":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6801/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/1210355277":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6800/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/3610017406":"2024-06-11T00:54:08.102405+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T01:04:46.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.938+0000 7f58a7e29700 1 -- 172.21.15.7:0/428402683 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f588c0a2c20 msgr2=0x7f588c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:46.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.938+0000 7f58a7e29700 1 --2- 172.21.15.7:0/428402683 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f588c0a2c20 0x7f588c0a50e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f589c005950 tx=0x7f589c00a300 comp rx=0 tx=0).stop 2024-06-10T01:04:46.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.938+0000 7f58a7e29700 1 -- 172.21.15.7:0/428402683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58a0107840 msgr2=0x7f58a0198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:46.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.938+0000 7f58a7e29700 1 --2- 172.21.15.7:0/428402683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58a0107840 0x7f58a0198980 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5898006070 tx=0x7f5898004d00 comp rx=0 tx=0).stop 2024-06-10T01:04:46.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.938+0000 7f58a7e29700 1 -- 172.21.15.7:0/428402683 shutdown_connections 2024-06-10T01:04:46.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.938+0000 7f58a7e29700 1 --2- 172.21.15.7:0/428402683 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f588c0a2c20 0x7f588c0a50e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.938+0000 7f58a7e29700 1 --2- 172.21.15.7:0/428402683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58a0107840 0x7f58a0198980 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.938+0000 7f58a7e29700 1 --2- 172.21.15.7:0/428402683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58a0102740 0x7f58a019e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.938+0000 7f58a7e29700 1 --2- 172.21.15.7:0/428402683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58a01030f0 0x7f58a019ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.938+0000 7f58a7e29700 1 -- 172.21.15.7:0/428402683 >> 172.21.15.7:0/428402683 conn(0x7f58a00795f0 msgr2=0x7f58a01005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:46.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.938+0000 7f58a7e29700 1 -- 172.21.15.7:0/428402683 shutdown_connections 2024-06-10T01:04:46.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:46.938+0000 7f58a7e29700 1 -- 172.21.15.7:0/428402683 wait complete. 2024-06-10T01:04:47.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:47 smithi007 ceph-mon[32398]: pgmap v394: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:04:47.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:47 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/428402683' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:04:47.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:47 smithi175 ceph-mon[36636]: pgmap v394: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:04:47.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:47 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/428402683' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:04:47.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:47 smithi078 ceph-mon[36683]: pgmap v394: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:04:47.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:47 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/428402683' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:04:47.721 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:04:47.979 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:49.448 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:49 smithi007 ceph-mon[32398]: pgmap v395: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:04:49.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:49 smithi175 ceph-mon[36636]: pgmap v395: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:04:49.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:49 smithi078 ceph-mon[36683]: pgmap v395: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:04:49.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.684+0000 7fa38fce1700 1 -- 172.21.15.7:0/1267560138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa388101620 msgr2=0x7fa388101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:49.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.684+0000 7fa38fce1700 1 --2- 172.21.15.7:0/1267560138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa388101620 0x7fa388101a00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa3840099c0 tx=0x7fa384009cd0 comp rx=0 tx=0).stop 2024-06-10T01:04:49.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.684+0000 7fa38fce1700 1 -- 172.21.15.7:0/1267560138 shutdown_connections 2024-06-10T01:04:49.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.684+0000 7fa38fce1700 1 --2- 172.21.15.7:0/1267560138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3881066b0 0x7fa38810e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:49.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.684+0000 7fa38fce1700 1 --2- 172.21.15.7:0/1267560138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa388101620 0x7fa388101a00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:49.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.684+0000 7fa38fce1700 1 --2- 172.21.15.7:0/1267560138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa388101fd0 0x7fa388105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:49.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.684+0000 7fa38fce1700 1 -- 172.21.15.7:0/1267560138 >> 172.21.15.7:0/1267560138 conn(0x7fa388078340 msgr2=0x7fa388078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:49.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.684+0000 7fa38fce1700 1 -- 172.21.15.7:0/1267560138 shutdown_connections 2024-06-10T01:04:49.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.684+0000 7fa38fce1700 1 -- 172.21.15.7:0/1267560138 wait complete. 2024-06-10T01:04:49.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.685+0000 7fa38fce1700 1 Processor -- start 2024-06-10T01:04:49.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.685+0000 7fa38fce1700 1 -- start start 2024-06-10T01:04:49.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.685+0000 7fa38fce1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa388101620 0x7fa38819d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:49.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.685+0000 7fa38fce1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa388101fd0 0x7fa38819dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:49.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.685+0000 7fa38fce1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3881066b0 0x7fa3881979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:49.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.686+0000 7fa38fce1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa3881116c0 con 0x7fa388101620 2024-06-10T01:04:49.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.686+0000 7fa38fce1700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa388111540 con 0x7fa388101fd0 2024-06-10T01:04:49.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.686+0000 7fa38fce1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa388111840 con 0x7fa3881066b0 2024-06-10T01:04:49.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.686+0000 7fa38da7d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa388101620 0x7fa38819d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:49.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.686+0000 7fa38da7d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa388101620 0x7fa38819d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52712/0 (socket says 172.21.15.7:52712) 2024-06-10T01:04:49.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.686+0000 7fa38da7d700 1 -- 172.21.15.7:0/1545040312 learned_addr learned my addr 172.21.15.7:0/1545040312 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:49.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.686+0000 7fa38e27e700 1 --2- 172.21.15.7:0/1545040312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3881066b0 0x7fa3881979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:49.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.686+0000 7fa38da7d700 1 -- 172.21.15.7:0/1545040312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3881066b0 msgr2=0x7fa3881979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:49.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.686+0000 7fa38da7d700 1 --2- 172.21.15.7:0/1545040312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3881066b0 0x7fa3881979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:49.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.686+0000 7fa38da7d700 1 -- 172.21.15.7:0/1545040312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa388101fd0 msgr2=0x7fa38819dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:49.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.686+0000 7fa38d27c700 1 --2- 172.21.15.7:0/1545040312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa388101fd0 0x7fa38819dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:49.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.686+0000 7fa38da7d700 1 --2- 172.21.15.7:0/1545040312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa388101fd0 0x7fa38819dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:49.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.686+0000 7fa38da7d700 1 -- 172.21.15.7:0/1545040312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa388198100 con 0x7fa388101620 2024-06-10T01:04:49.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.686+0000 7fa38e27e700 1 --2- 172.21.15.7:0/1545040312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3881066b0 0x7fa3881979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:04:49.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.686+0000 7fa38d27c700 1 --2- 172.21.15.7:0/1545040312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa388101fd0 0x7fa38819dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:04:49.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.686+0000 7fa38da7d700 1 --2- 172.21.15.7:0/1545040312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa388101620 0x7fa38819d7e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa384006010 tx=0x7fa384009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:49.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.686+0000 7fa37effd700 1 -- 172.21.15.7:0/1545040312 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa384027070 con 0x7fa388101620 2024-06-10T01:04:49.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.686+0000 7fa37effd700 1 -- 172.21.15.7:0/1545040312 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa384004050 con 0x7fa388101620 2024-06-10T01:04:49.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.686+0000 7fa38fce1700 1 -- 172.21.15.7:0/1545040312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa388198390 con 0x7fa388101620 2024-06-10T01:04:49.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.686+0000 7fa37effd700 1 -- 172.21.15.7:0/1545040312 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa384022e10 con 0x7fa388101620 2024-06-10T01:04:49.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.686+0000 7fa38fce1700 1 -- 172.21.15.7:0/1545040312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3881a46b0 con 0x7fa388101620 2024-06-10T01:04:49.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.687+0000 7fa38fce1700 1 -- 172.21.15.7:0/1545040312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa388069180 con 0x7fa388101620 2024-06-10T01:04:49.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.689+0000 7fa37effd700 1 -- 172.21.15.7:0/1545040312 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa38402a070 con 0x7fa388101620 2024-06-10T01:04:49.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.690+0000 7fa37effd700 1 --2- 172.21.15.7:0/1545040312 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa3740a2ca0 0x7fa3740a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:49.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.690+0000 7fa38d27c700 1 --2- 172.21.15.7:0/1545040312 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa3740a2ca0 0x7fa3740a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:49.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.690+0000 7fa37effd700 1 -- 172.21.15.7:0/1545040312 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fa3840d0250 con 0x7fa388101620 2024-06-10T01:04:49.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.690+0000 7fa38d27c700 1 --2- 172.21.15.7:0/1545040312 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa3740a2ca0 0x7fa3740a5160 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa378005fd0 tx=0x7fa378005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:49.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.693+0000 7fa37effd700 1 -- 172.21.15.7:0/1545040312 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa384013070 con 0x7fa388101620 2024-06-10T01:04:49.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.948+0000 7fa38fce1700 1 -- 172.21.15.7:0/1545040312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa388198ed0 con 0x7fa388101620 2024-06-10T01:04:49.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.948+0000 7fa37effd700 1 -- 172.21.15.7:0/1545040312 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11855 (secure 0 0 0) 0x7fa38409cec0 con 0x7fa388101620 2024-06-10T01:04:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:04:49.946 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:02:58.523980+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/2784910673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2784910673},{"type":"v1","addr":"172.21.15.175:6835","nonce":2784910673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:04:49.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.951+0000 7fa38fce1700 1 -- 172.21.15.7:0/1545040312 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa3740a2ca0 msgr2=0x7fa3740a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:49.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.951+0000 7fa38fce1700 1 --2- 172.21.15.7:0/1545040312 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa3740a2ca0 0x7fa3740a5160 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa378005fd0 tx=0x7fa378005dc0 comp rx=0 tx=0).stop 2024-06-10T01:04:49.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.951+0000 7fa38fce1700 1 -- 172.21.15.7:0/1545040312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa388101620 msgr2=0x7fa38819d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:49.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.951+0000 7fa38fce1700 1 --2- 172.21.15.7:0/1545040312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa388101620 0x7fa38819d7e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa384006010 tx=0x7fa384009f80 comp rx=0 tx=0).stop 2024-06-10T01:04:49.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.951+0000 7fa38fce1700 1 -- 172.21.15.7:0/1545040312 shutdown_connections 2024-06-10T01:04:49.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.951+0000 7fa38fce1700 1 --2- 172.21.15.7:0/1545040312 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa3740a2ca0 0x7fa3740a5160 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:49.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.951+0000 7fa38fce1700 1 --2- 172.21.15.7:0/1545040312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3881066b0 0x7fa3881979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:49.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.951+0000 7fa38fce1700 1 --2- 172.21.15.7:0/1545040312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa388101620 0x7fa38819d7e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:49.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.951+0000 7fa38fce1700 1 --2- 172.21.15.7:0/1545040312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa388101fd0 0x7fa38819dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:49.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.951+0000 7fa38fce1700 1 -- 172.21.15.7:0/1545040312 >> 172.21.15.7:0/1545040312 conn(0x7fa388078340 msgr2=0x7fa388100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:49.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.951+0000 7fa38fce1700 1 -- 172.21.15.7:0/1545040312 shutdown_connections 2024-06-10T01:04:49.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:49.951+0000 7fa38fce1700 1 -- 172.21.15.7:0/1545040312 wait complete. 2024-06-10T01:04:49.949 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-06-10T01:04:50.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:50 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1545040312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:04:50.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:50 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1545040312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:04:50.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:50 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1545040312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:04:50.676 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-10T01:01:49.029348+0000', 'modified': '2024-06-10T01:02:58.523980+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24451, 'mds_1': 24436, 'mds_2': 24434}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24403': {'gid': 24403, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.175:6833/1281305765', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6832', 'nonce': 1281305765}, {'type': 'v1', 'addr': '172.21.15.175:6833', 'nonce': 1281305765}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24413': {'gid': 24413, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.78:6837/3608622984', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6836', 'nonce': 3608622984}, {'type': 'v1', 'addr': '172.21.15.78:6837', 'nonce': 3608622984}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24424': {'gid': 24424, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.175:6835/2784910673', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6834', 'nonce': 2784910673}, {'type': 'v1', 'addr': '172.21.15.175:6835', 'nonce': 2784910673}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'b', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.78:6839/1946947484', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6838', 'nonce': 1946947484}, {'type': 'v1', 'addr': '172.21.15.78:6839', 'nonce': 1946947484}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'c', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.175:6837/779740919', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6836', 'nonce': 779740919}, {'type': 'v1', 'addr': '172.21.15.175:6837', 'nonce': 779740919}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.175:6839/1983418024', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6838', 'nonce': 1983418024}, {'type': 'v1', 'addr': '172.21.15.175:6839', 'nonce': 1983418024}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24451, 'qdb_cluster': [24436, 24451, 24434]} 2024-06-10T01:04:50.676 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-06-10T01:04:50.676 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-10T01:04:50.928 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:51.302 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:51 smithi007 ceph-mon[32398]: pgmap v396: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-06-10T01:04:51.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:51 smithi175 ceph-mon[36636]: pgmap v396: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-06-10T01:04:51.483 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:51 smithi078 ceph-mon[36683]: pgmap v396: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-06-10T01:04:52.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.578+0000 7faceafbf700 1 -- 172.21.15.7:0/2511947706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face406b810 msgr2=0x7face41070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:52.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.578+0000 7faceafbf700 1 --2- 172.21.15.7:0/2511947706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face406b810 0x7face41070d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7facd00099c0 tx=0x7facd0009cd0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.578+0000 7faceafbf700 1 -- 172.21.15.7:0/2511947706 shutdown_connections 2024-06-10T01:04:52.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.578+0000 7faceafbf700 1 --2- 172.21.15.7:0/2511947706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7face41077b0 0x7face410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.578+0000 7faceafbf700 1 --2- 172.21.15.7:0/2511947706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7face406ae60 0x7face406b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.578+0000 7faceafbf700 1 --2- 172.21.15.7:0/2511947706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face406b810 0x7face41070d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.578+0000 7faceafbf700 1 -- 172.21.15.7:0/2511947706 >> 172.21.15.7:0/2511947706 conn(0x7face40773e0 msgr2=0x7face40777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:52.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.578+0000 7faceafbf700 1 -- 172.21.15.7:0/2511947706 shutdown_connections 2024-06-10T01:04:52.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.578+0000 7faceafbf700 1 -- 172.21.15.7:0/2511947706 wait complete. 2024-06-10T01:04:52.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.579+0000 7faceafbf700 1 Processor -- start 2024-06-10T01:04:52.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.579+0000 7faceafbf700 1 -- start start 2024-06-10T01:04:52.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.579+0000 7faceafbf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7face406ae60 0x7face419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:52.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.579+0000 7faceafbf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face406b810 0x7face419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:52.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.579+0000 7faceafbf700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7face41077b0 0x7face419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:52.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.579+0000 7faceafbf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7face4112800 con 0x7face406b810 2024-06-10T01:04:52.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.579+0000 7faceafbf700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7face4112680 con 0x7face406ae60 2024-06-10T01:04:52.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.579+0000 7faceafbf700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7face4112980 con 0x7face41077b0 2024-06-10T01:04:52.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.579+0000 7face3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face406b810 0x7face419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:52.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.579+0000 7face3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face406b810 0x7face419b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44916/0 (socket says 172.21.15.7:44916) 2024-06-10T01:04:52.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.579+0000 7face3fff700 1 -- 172.21.15.7:0/1842427782 learned_addr learned my addr 172.21.15.7:0/1842427782 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:52.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.579+0000 7face8d5b700 1 --2- 172.21.15.7:0/1842427782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7face406ae60 0x7face419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:52.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.579+0000 7face955c700 1 --2- 172.21.15.7:0/1842427782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7face41077b0 0x7face419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:52.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.579+0000 7face3fff700 1 -- 172.21.15.7:0/1842427782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7face41077b0 msgr2=0x7face419f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:52.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.579+0000 7face3fff700 1 --2- 172.21.15.7:0/1842427782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7face41077b0 0x7face419f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.579+0000 7face3fff700 1 -- 172.21.15.7:0/1842427782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7face406ae60 msgr2=0x7face419ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:52.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.579+0000 7face3fff700 1 --2- 172.21.15.7:0/1842427782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7face406ae60 0x7face419ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.580+0000 7face3fff700 1 -- 172.21.15.7:0/1842427782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face419fe40 con 0x7face406b810 2024-06-10T01:04:52.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.580+0000 7face3fff700 1 --2- 172.21.15.7:0/1842427782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face406b810 0x7face419b370 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7facd000c010 tx=0x7facd00038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:52.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.580+0000 7face1ffb700 1 -- 172.21.15.7:0/1842427782 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd0026070 con 0x7face406b810 2024-06-10T01:04:52.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.580+0000 7faceafbf700 1 -- 172.21.15.7:0/1842427782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7face41a00d0 con 0x7face406b810 2024-06-10T01:04:52.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.580+0000 7face1ffb700 1 -- 172.21.15.7:0/1842427782 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facd0005690 con 0x7face406b810 2024-06-10T01:04:52.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.580+0000 7face1ffb700 1 -- 172.21.15.7:0/1842427782 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd000fcf0 con 0x7face406b810 2024-06-10T01:04:52.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.580+0000 7faceafbf700 1 -- 172.21.15.7:0/1842427782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7face41a79b0 con 0x7face406b810 2024-06-10T01:04:52.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.583+0000 7face1ffb700 1 -- 172.21.15.7:0/1842427782 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7facd0029070 con 0x7face406b810 2024-06-10T01:04:52.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.583+0000 7face1ffb700 1 --2- 172.21.15.7:0/1842427782 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7facd40a2c20 0x7facd40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:52.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.583+0000 7face8d5b700 1 --2- 172.21.15.7:0/1842427782 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7facd40a2c20 0x7facd40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:52.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.583+0000 7face1ffb700 1 -- 172.21.15.7:0/1842427782 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7facd00cfd60 con 0x7face406b810 2024-06-10T01:04:52.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.584+0000 7face8d5b700 1 --2- 172.21.15.7:0/1842427782 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7facd40a2c20 0x7facd40a50e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7facd8005950 tx=0x7facd800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:52.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.584+0000 7facf5842b80 1 Processor -- start 2024-06-10T01:04:52.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.584+0000 7facf5842b80 1 -- start start 2024-06-10T01:04:52.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.584+0000 7facf5842b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca70c08f10 0x55ca70c092f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:52.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.584+0000 7facf5842b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca70c09830 0x55ca70c03210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:52.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.584+0000 7facf5842b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca70c03750 0x55ca70c03b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:52.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.584+0000 7facf5842b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ca70af5f10 con 0x55ca70c09830 2024-06-10T01:04:52.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.584+0000 7facf5842b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ca70b54800 con 0x55ca70c08f10 2024-06-10T01:04:52.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.584+0000 7facf5842b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ca70abc410 con 0x55ca70c03750 2024-06-10T01:04:52.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.584+0000 7face3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca70c09830 0x55ca70c03210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:52.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.584+0000 7face3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca70c09830 0x55ca70c03210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44932/0 (socket says 172.21.15.7:44932) 2024-06-10T01:04:52.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.585+0000 7face3fff700 1 -- 172.21.15.7:0/2440848914 learned_addr learned my addr 172.21.15.7:0/2440848914 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:52.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.585+0000 7face8d5b700 1 --2- 172.21.15.7:0/2440848914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca70c08f10 0x55ca70c092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:52.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.585+0000 7face955c700 1 --2- 172.21.15.7:0/2440848914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca70c03750 0x55ca70c03b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:52.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.585+0000 7face3fff700 1 -- 172.21.15.7:0/2440848914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca70c03750 msgr2=0x55ca70c03b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:52.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.585+0000 7face3fff700 1 --2- 172.21.15.7:0/2440848914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca70c03750 0x55ca70c03b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.585+0000 7face3fff700 1 -- 172.21.15.7:0/2440848914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca70c08f10 msgr2=0x55ca70c092f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:52.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.585+0000 7face3fff700 1 --2- 172.21.15.7:0/2440848914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca70c08f10 0x55ca70c092f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.585+0000 7face3fff700 1 -- 172.21.15.7:0/2440848914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca70aa3880 con 0x55ca70c09830 2024-06-10T01:04:52.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.585+0000 7face3fff700 1 --2- 172.21.15.7:0/2440848914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca70c09830 0x55ca70c03210 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7facd00a4980 tx=0x7facd00042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1e66ccb96666d8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:52.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.585+0000 7facceffd700 1 -- 172.21.15.7:0/2440848914 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd000f460 con 0x55ca70c09830 2024-06-10T01:04:52.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.585+0000 7facceffd700 1 -- 172.21.15.7:0/2440848914 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facd009c9d0 con 0x55ca70c09830 2024-06-10T01:04:52.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.585+0000 7facceffd700 1 -- 172.21.15.7:0/2440848914 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd00a4c40 con 0x55ca70c09830 2024-06-10T01:04:52.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.586+0000 7facf5842b80 1 -- 172.21.15.7:0/2440848914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca70c09830 msgr2=0x55ca70c03210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:52.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.586+0000 7facf5842b80 1 --2- 172.21.15.7:0/2440848914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca70c09830 0x55ca70c03210 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7facd00a4980 tx=0x7facd00042e0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.586+0000 7facf5842b80 1 -- 172.21.15.7:0/2440848914 shutdown_connections 2024-06-10T01:04:52.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.586+0000 7facf5842b80 1 --2- 172.21.15.7:0/2440848914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca70c03750 0x55ca70c03b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.586+0000 7facf5842b80 1 --2- 172.21.15.7:0/2440848914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca70c08f10 0x55ca70c092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.586+0000 7facf5842b80 1 --2- 172.21.15.7:0/2440848914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca70c09830 0x55ca70c03210 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.586+0000 7facf5842b80 1 -- 172.21.15.7:0/2440848914 >> 172.21.15.7:0/2440848914 conn(0x55ca70b30030 msgr2=0x55ca70bfe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:52.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.586+0000 7facf5842b80 1 -- 172.21.15.7:0/2440848914 shutdown_connections 2024-06-10T01:04:52.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.586+0000 7facf5842b80 1 -- 172.21.15.7:0/2440848914 wait complete. 2024-06-10T01:04:52.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.586+0000 7facf5842b80 1 -- start start 2024-06-10T01:04:52.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.586+0000 7facf5842b80 1 Processor -- start 2024-06-10T01:04:52.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.587+0000 7facf5842b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca70c03750 0x55ca70c0f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:52.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.587+0000 7facf5842b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca70c08f10 0x55ca70c0c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:52.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.587+0000 7facf5842b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca70c09830 0x55ca70c0cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:52.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.587+0000 7facf5842b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ca70af4f70 con 0x55ca70c09830 2024-06-10T01:04:52.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.587+0000 7facf5842b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ca70b507c0 con 0x55ca70c03750 2024-06-10T01:04:52.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.587+0000 7facf5842b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ca70c0ffc0 con 0x55ca70c08f10 2024-06-10T01:04:52.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.587+0000 7facf5842b80 10 client.0 _ll_register_callbacks cb 0x55ca70aea560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:04:52.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.587+0000 7face955c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca70c09830 0x55ca70c0cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:52.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.587+0000 7face955c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca70c09830 0x55ca70c0cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44936/0 (socket says 172.21.15.7:44936) 2024-06-10T01:04:52.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.587+0000 7face955c700 1 -- 172.21.15.7:0/3920340896 learned_addr learned my addr 172.21.15.7:0/3920340896 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:52.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.587+0000 7face3fff700 1 --2- 172.21.15.7:0/3920340896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca70c08f10 0x55ca70c0c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:52.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.587+0000 7face8d5b700 1 --2- 172.21.15.7:0/3920340896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca70c03750 0x55ca70c0f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:52.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.587+0000 7face955c700 1 -- 172.21.15.7:0/3920340896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca70c08f10 msgr2=0x55ca70c0c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:52.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.587+0000 7face955c700 1 --2- 172.21.15.7:0/3920340896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca70c08f10 0x55ca70c0c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.587+0000 7face955c700 1 -- 172.21.15.7:0/3920340896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca70c03750 msgr2=0x55ca70c0f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:52.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.587+0000 7face955c700 1 --2- 172.21.15.7:0/3920340896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca70c03750 0x55ca70c0f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.587+0000 7face955c700 1 -- 172.21.15.7:0/3920340896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca70b1a700 con 0x55ca70c09830 2024-06-10T01:04:52.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.587+0000 7face955c700 1 --2- 172.21.15.7:0/3920340896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca70c09830 0x55ca70c0cdc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7facdc010b30 tx=0x7facdc010e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c847e140ad0e7c12 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:52.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.587+0000 7facc3fff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:04:52.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.588+0000 7facc3fff700 1 -- 172.21.15.7:0/3920340896 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc003c00 con 0x55ca70c09830 2024-06-10T01:04:52.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.588+0000 7facc3fff700 1 -- 172.21.15.7:0/3920340896 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7facdc003da0 con 0x55ca70c09830 2024-06-10T01:04:52.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.588+0000 7facf5842b80 1 -- 172.21.15.7:0/3920340896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facd009c9d0 con 0x55ca70c09830 2024-06-10T01:04:52.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.588+0000 7facc3fff700 1 -- 172.21.15.7:0/3920340896 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7facdc024050 con 0x55ca70c09830 2024-06-10T01:04:52.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.588+0000 7facf5842b80 10 client.15042 fetch_fsmap learned FSMap version 26 2024-06-10T01:04:52.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.588+0000 7facf5842b80 1 -- 172.21.15.7:0/3920340896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ca70c1a670 con 0x55ca70c09830 2024-06-10T01:04:52.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.588+0000 7facc3fff700 1 -- 172.21.15.7:0/3920340896 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facd009c9d0 con 0x55ca70c09830 2024-06-10T01:04:52.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.588+0000 7facc3fff700 1 -- 172.21.15.7:0/3920340896 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7facdc0044f0 con 0x55ca70c09830 2024-06-10T01:04:52.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.588+0000 7facf5842b80 10 client.15042 fetch_fsmap finished waiting for FSMap version 26 2024-06-10T01:04:52.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.588+0000 7facf5842b80 10 client.15042 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-10T01:04:52.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.588+0000 7facf5842b80 20 client.15042 populate_metadata read hostname 'smithi007' 2024-06-10T01:04:52.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.588+0000 7facf5842b80 1 --2- 172.21.15.7:0/3920340896 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55ca70c1ccd0 0x55ca70c1e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:52.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.588+0000 7facf5842b80 4 client.15042 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:04:52.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.588+0000 7facf5842b80 1 -- 172.21.15.7:0/3920340896 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca70b61d00 con 0x55ca70c1ccd0 2024-06-10T01:04:52.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.589+0000 7face8d5b700 1 --2- 172.21.15.7:0/3920340896 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55ca70c1ccd0 0x55ca70c1e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:52.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.589+0000 7face8d5b700 1 --2- 172.21.15.7:0/3920340896 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55ca70c1ccd0 0x55ca70c1e140 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d99c10aec3a9cf4a server_cookie=abb7ab4a1fc11eb8 in_seq=0 out_seq=0 2024-06-10T01:04:52.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.589+0000 7facc3fff700 10 client.15042 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:04:52.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.590+0000 7facc3fff700 1 -- 172.21.15.7:0/3920340896 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ca70b61d00 con 0x55ca70c1ccd0 2024-06-10T01:04:52.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.590+0000 7facc3fff700 10 client.15042 handle_command_reply: tid=0 2024-06-10T01:04:52.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.590+0000 7facf5842b80 1 client.15042 shutdown 2024-06-10T01:04:52.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.590+0000 7facf5842b80 1 -- 172.21.15.7:0/3920340896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca70c09830 msgr2=0x55ca70c0cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:52.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.590+0000 7facf5842b80 1 --2- 172.21.15.7:0/3920340896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca70c09830 0x55ca70c0cdc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7facdc010b30 tx=0x7facdc010e40 comp rx=0 tx=0).stop 2024-06-10T01:04:52.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.590+0000 7facf5842b80 1 -- 172.21.15.7:0/3920340896 shutdown_connections 2024-06-10T01:04:52.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.590+0000 7facf5842b80 1 --2- 172.21.15.7:0/3920340896 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55ca70c1ccd0 0x55ca70c1e140 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.590+0000 7facc3fff700 0 client.15042 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:04:52.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.590+0000 7facf5842b80 1 --2- 172.21.15.7:0/3920340896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca70c09830 0x55ca70c0cdc0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.590+0000 7facf5842b80 1 --2- 172.21.15.7:0/3920340896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca70c03750 0x55ca70c0f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.590+0000 7facf5842b80 1 --2- 172.21.15.7:0/3920340896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca70c08f10 0x55ca70c0c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.590+0000 7facf5842b80 1 -- 172.21.15.7:0/3920340896 >> 172.21.15.7:0/3920340896 conn(0x55ca70b30030 msgr2=0x55ca70bfd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:52.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.590+0000 7facf5842b80 1 -- 172.21.15.7:0/3920340896 shutdown_connections 2024-06-10T01:04:52.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.590+0000 7facf5842b80 1 -- 172.21.15.7:0/3920340896 wait complete. 2024-06-10T01:04:52.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.590+0000 7facf5842b80 20 client.15042 trim_cache size 0 max 16384 2024-06-10T01:04:52.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.612+0000 7facf5842b80 1 Processor -- start 2024-06-10T01:04:52.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.612+0000 7facf5842b80 1 -- start start 2024-06-10T01:04:52.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.612+0000 7facf5842b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca70c03750 0x55ca70c03210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:52.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.612+0000 7facf5842b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca70c08f10 0x55ca70bff950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:52.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.612+0000 7facf5842b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca70c09830 0x55ca70c0ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:52.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.612+0000 7facf5842b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ca70c1c480 con 0x55ca70c03750 2024-06-10T01:04:52.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.612+0000 7facf5842b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ca70c1cb50 con 0x55ca70c08f10 2024-06-10T01:04:52.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.612+0000 7facf5842b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ca70c1d220 con 0x55ca70c09830 2024-06-10T01:04:52.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.612+0000 7face8d5b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca70c03750 0x55ca70c03210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:52.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.612+0000 7face8d5b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca70c03750 0x55ca70c03210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44938/0 (socket says 172.21.15.7:44938) 2024-06-10T01:04:52.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.613+0000 7face8d5b700 1 -- 172.21.15.7:0/4046631532 learned_addr learned my addr 172.21.15.7:0/4046631532 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:52.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.613+0000 7face955c700 1 --2- 172.21.15.7:0/4046631532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca70c09830 0x55ca70c0ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:52.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.613+0000 7face3fff700 1 --2- 172.21.15.7:0/4046631532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca70c08f10 0x55ca70bff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.613+0000 7face8d5b700 1 -- 172.21.15.7:0/4046631532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca70c09830 msgr2=0x55ca70c0ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.613+0000 7face8d5b700 1 --2- 172.21.15.7:0/4046631532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca70c09830 0x55ca70c0ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.613+0000 7face8d5b700 1 -- 172.21.15.7:0/4046631532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca70c08f10 msgr2=0x55ca70bff950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.613+0000 7face8d5b700 1 --2- 172.21.15.7:0/4046631532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca70c08f10 0x55ca70bff950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.613+0000 7face8d5b700 1 -- 172.21.15.7:0/4046631532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca70c0ae30 con 0x55ca70c03750 2024-06-10T01:04:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.613+0000 7face3fff700 1 --2- 172.21.15.7:0/4046631532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca70c08f10 0x55ca70bff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:04:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.613+0000 7face8d5b700 1 --2- 172.21.15.7:0/4046631532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca70c03750 0x55ca70c03210 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7facd8011540 tx=0x7facd8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c041f88d554e90d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:52.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.613+0000 7facce7fc700 1 -- 172.21.15.7:0/4046631532 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd8011d50 con 0x55ca70c03750 2024-06-10T01:04:52.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.613+0000 7facce7fc700 1 -- 172.21.15.7:0/4046631532 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facd801e900 con 0x55ca70c03750 2024-06-10T01:04:52.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.613+0000 7facce7fc700 1 -- 172.21.15.7:0/4046631532 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd8027720 con 0x55ca70c03750 2024-06-10T01:04:52.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.614+0000 7facf5842b80 1 -- 172.21.15.7:0/4046631532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca70c03750 msgr2=0x55ca70c03210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:52.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.614+0000 7facf5842b80 1 --2- 172.21.15.7:0/4046631532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca70c03750 0x55ca70c03210 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7facd8011540 tx=0x7facd8013000 comp rx=0 tx=0).stop 2024-06-10T01:04:52.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.614+0000 7facf5842b80 1 -- 172.21.15.7:0/4046631532 shutdown_connections 2024-06-10T01:04:52.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.614+0000 7facf5842b80 1 --2- 172.21.15.7:0/4046631532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca70c09830 0x55ca70c0ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.614+0000 7facf5842b80 1 --2- 172.21.15.7:0/4046631532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca70c03750 0x55ca70c03210 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.614+0000 7facf5842b80 1 --2- 172.21.15.7:0/4046631532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca70c08f10 0x55ca70bff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.614+0000 7facf5842b80 1 -- 172.21.15.7:0/4046631532 >> 172.21.15.7:0/4046631532 conn(0x55ca70b30030 msgr2=0x55ca70c09cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:52.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.614+0000 7facf5842b80 1 -- 172.21.15.7:0/4046631532 shutdown_connections 2024-06-10T01:04:52.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.614+0000 7facf5842b80 1 -- 172.21.15.7:0/4046631532 wait complete. 2024-06-10T01:04:52.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.614+0000 7facf5842b80 1 -- start start 2024-06-10T01:04:52.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.615+0000 7facf5842b80 1 Processor -- start 2024-06-10T01:04:52.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.615+0000 7facf5842b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca70c03750 0x55ca70c27b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:52.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.615+0000 7facf5842b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca70c08f10 0x55ca70bfd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:52.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.615+0000 7facf5842b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca70c09830 0x55ca70bfd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.615+0000 7facf5842b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ca70c1dfc0 con 0x55ca70c09830 2024-06-10T01:04:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.615+0000 7facf5842b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ca70c1d8f0 con 0x55ca70c08f10 2024-06-10T01:04:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.615+0000 7facf5842b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ca70c1e690 con 0x55ca70c03750 2024-06-10T01:04:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.615+0000 7facf5842b80 10 client.0 _ll_register_callbacks cb 0x55ca70aea560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:04:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.615+0000 7face955c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca70c09830 0x55ca70bfd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.615+0000 7face955c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca70c09830 0x55ca70bfd720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44946/0 (socket says 172.21.15.7:44946) 2024-06-10T01:04:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.615+0000 7face955c700 1 -- 172.21.15.7:0/445304526 learned_addr learned my addr 172.21.15.7:0/445304526 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.615+0000 7face8d5b700 1 --2- 172.21.15.7:0/445304526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca70c03750 0x55ca70c27b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.615+0000 7face3fff700 1 --2- 172.21.15.7:0/445304526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca70c08f10 0x55ca70bfd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.616+0000 7face955c700 1 -- 172.21.15.7:0/445304526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca70c03750 msgr2=0x55ca70c27b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.616+0000 7face955c700 1 --2- 172.21.15.7:0/445304526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca70c03750 0x55ca70c27b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.616+0000 7face955c700 1 -- 172.21.15.7:0/445304526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca70c08f10 msgr2=0x55ca70bfd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.616+0000 7face955c700 1 --2- 172.21.15.7:0/445304526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca70c08f10 0x55ca70bfd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.616+0000 7face955c700 1 -- 172.21.15.7:0/445304526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca70bfdce0 con 0x55ca70c09830 2024-06-10T01:04:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.616+0000 7face955c700 1 --2- 172.21.15.7:0/445304526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca70c09830 0x55ca70bfd720 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7facdc004990 tx=0x7facdc007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b78a041eae1dff9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.616+0000 7facc3fff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:04:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.616+0000 7facc3fff700 1 -- 172.21.15.7:0/445304526 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc00e860 con 0x55ca70c09830 2024-06-10T01:04:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.616+0000 7facf5842b80 1 -- 172.21.15.7:0/445304526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facd801e900 con 0x55ca70c09830 2024-06-10T01:04:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.616+0000 7facc3fff700 1 -- 172.21.15.7:0/445304526 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7facdc00ea00 con 0x55ca70c09830 2024-06-10T01:04:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.616+0000 7facc3fff700 1 -- 172.21.15.7:0/445304526 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7facdc00ec20 con 0x55ca70c09830 2024-06-10T01:04:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.616+0000 7facf5842b80 10 client.15048 fetch_fsmap learned FSMap version 26 2024-06-10T01:04:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.616+0000 7facf5842b80 1 -- 172.21.15.7:0/445304526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ca70c26370 con 0x55ca70c09830 2024-06-10T01:04:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.616+0000 7facc3fff700 1 -- 172.21.15.7:0/445304526 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facd801e900 con 0x55ca70c09830 2024-06-10T01:04:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.617+0000 7facc3fff700 1 -- 172.21.15.7:0/445304526 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7facdc005040 con 0x55ca70c09830 2024-06-10T01:04:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.617+0000 7facf5842b80 10 client.15048 fetch_fsmap finished waiting for FSMap version 26 2024-06-10T01:04:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.617+0000 7facf5842b80 10 client.15048 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-10T01:04:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.617+0000 7facf5842b80 20 client.15048 populate_metadata read hostname 'smithi007' 2024-06-10T01:04:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.617+0000 7facf5842b80 1 --2- 172.21.15.7:0/445304526 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55ca70c1ccd0 0x55ca70c8bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.617+0000 7facf5842b80 4 client.15048 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T01:04:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.617+0000 7facf5842b80 1 -- 172.21.15.7:0/445304526 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "status"}) -- 0x55ca70b5c280 con 0x55ca70c1ccd0 2024-06-10T01:04:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.617+0000 7face8d5b700 1 --2- 172.21.15.7:0/445304526 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55ca70c1ccd0 0x55ca70c8bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.618+0000 7face8d5b700 1 --2- 172.21.15.7:0/445304526 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55ca70c1ccd0 0x55ca70c8bbc0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=44f36a20fdacdddd server_cookie=ec4ac60b10f8a4b1 in_seq=0 out_seq=0 2024-06-10T01:04:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.618+0000 7facc3fff700 10 client.15048 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:04:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.618+0000 7facc3fff700 1 -- 172.21.15.7:0/445304526 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ca70b5c280 con 0x55ca70c1ccd0 2024-06-10T01:04:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.618+0000 7facc3fff700 10 client.15048 handle_command_reply: tid=0 2024-06-10T01:04:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.618+0000 7facf5842b80 1 client.15048 shutdown 2024-06-10T01:04:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.618+0000 7facf5842b80 1 -- 172.21.15.7:0/445304526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca70c09830 msgr2=0x55ca70bfd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.618+0000 7facf5842b80 1 --2- 172.21.15.7:0/445304526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca70c09830 0x55ca70bfd720 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7facdc004990 tx=0x7facdc007f30 comp rx=0 tx=0).stop 2024-06-10T01:04:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.618+0000 7facf5842b80 1 -- 172.21.15.7:0/445304526 shutdown_connections 2024-06-10T01:04:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.618+0000 7facf5842b80 1 --2- 172.21.15.7:0/445304526 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55ca70c1ccd0 0x55ca70c8bbc0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.618+0000 7facf5842b80 1 --2- 172.21.15.7:0/445304526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca70c09830 0x55ca70bfd720 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.618+0000 7facf5842b80 1 --2- 172.21.15.7:0/445304526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca70c03750 0x55ca70c27b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.618+0000 7facf5842b80 1 --2- 172.21.15.7:0/445304526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca70c08f10 0x55ca70bfd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.618+0000 7facf5842b80 1 -- 172.21.15.7:0/445304526 >> 172.21.15.7:0/445304526 conn(0x55ca70b30030 msgr2=0x55ca70c0e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:52.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.618+0000 7facc3fff700 0 client.15048 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:04:52.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.618+0000 7facf5842b80 1 -- 172.21.15.7:0/445304526 shutdown_connections 2024-06-10T01:04:52.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.618+0000 7facf5842b80 1 -- 172.21.15.7:0/445304526 wait complete. 2024-06-10T01:04:52.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.619+0000 7facf5842b80 20 client.15048 trim_cache size 0 max 16384 2024-06-10T01:04:52.620 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:04:52.620 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "bded58f8-26c3-11ef-bc9b-c7b262605968", 2024-06-10T01:04:52.620 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 2, 2024-06-10T01:04:52.621 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24434, 2024-06-10T01:04:52.621 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-06-10T01:04:52.621 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-06-10T01:04:52.621 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-06-10T01:04:52.621 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 118.916414182, 2024-06-10T01:04:52.621 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 26, 2024-06-10T01:04:52.621 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 99, 2024-06-10T01:04:52.621 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 97, 2024-06-10T01:04:52.621 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 198.85196909199999 2024-06-10T01:04:52.621 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:04:52.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.619+0000 7faceafbf700 1 -- 172.21.15.7:0/1842427782 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7facd40a2c20 msgr2=0x7facd40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:52.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.619+0000 7faceafbf700 1 --2- 172.21.15.7:0/1842427782 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7facd40a2c20 0x7facd40a50e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7facd8005950 tx=0x7facd800b410 comp rx=0 tx=0).stop 2024-06-10T01:04:52.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.619+0000 7faceafbf700 1 -- 172.21.15.7:0/1842427782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face406b810 msgr2=0x7face419b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:52.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.619+0000 7faceafbf700 1 --2- 172.21.15.7:0/1842427782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face406b810 0x7face419b370 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7facd000c010 tx=0x7facd00038b0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.619+0000 7faceafbf700 1 -- 172.21.15.7:0/1842427782 shutdown_connections 2024-06-10T01:04:52.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.619+0000 7faceafbf700 1 --2- 172.21.15.7:0/1842427782 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7facd40a2c20 0x7facd40a50e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.619+0000 7faceafbf700 1 --2- 172.21.15.7:0/1842427782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7face41077b0 0x7face419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.619+0000 7faceafbf700 1 --2- 172.21.15.7:0/1842427782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7face406ae60 0x7face419ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.619+0000 7faceafbf700 1 --2- 172.21.15.7:0/1842427782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face406b810 0x7face419b370 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.619+0000 7faceafbf700 1 -- 172.21.15.7:0/1842427782 >> 172.21.15.7:0/1842427782 conn(0x7face40773e0 msgr2=0x7face4100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:52.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.619+0000 7faceafbf700 1 -- 172.21.15.7:0/1842427782 shutdown_connections 2024-06-10T01:04:52.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:52.620+0000 7faceafbf700 1 -- 172.21.15.7:0/1842427782 wait complete. 2024-06-10T01:04:53.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:53 smithi007 ceph-mon[32398]: pgmap v397: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-06-10T01:04:53.411 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.c status 2024-06-10T01:04:53.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:53 smithi175 ceph-mon[36636]: pgmap v397: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-06-10T01:04:53.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:53 smithi078 ceph-mon[36683]: pgmap v397: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-06-10T01:04:53.664 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:55.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.237+0000 7f0bca0d7700 1 -- 172.21.15.7:0/2130264701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc4108500 msgr2=0x7f0bc41088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:55.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.237+0000 7f0bca0d7700 1 --2- 172.21.15.7:0/2130264701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc4108500 0x7f0bc41088e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4009990 tx=0x7f0bb4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.237+0000 7f0bca0d7700 1 -- 172.21.15.7:0/2130264701 shutdown_connections 2024-06-10T01:04:55.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.237+0000 7f0bca0d7700 1 --2- 172.21.15.7:0/2130264701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0bc406ab30 0x7f0bc410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.237+0000 7f0bca0d7700 1 --2- 172.21.15.7:0/2130264701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc4108500 0x7f0bc41088e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.237+0000 7f0bca0d7700 1 --2- 172.21.15.7:0/2130264701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bc406a170 0x7f0bc406a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.237+0000 7f0bca0d7700 1 -- 172.21.15.7:0/2130264701 >> 172.21.15.7:0/2130264701 conn(0x7f0bc4078340 msgr2=0x7f0bc4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:55.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.237+0000 7f0bca0d7700 1 -- 172.21.15.7:0/2130264701 shutdown_connections 2024-06-10T01:04:55.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.237+0000 7f0bca0d7700 1 -- 172.21.15.7:0/2130264701 wait complete. 2024-06-10T01:04:55.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.238+0000 7f0bca0d7700 1 Processor -- start 2024-06-10T01:04:55.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.238+0000 7f0bca0d7700 1 -- start start 2024-06-10T01:04:55.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.238+0000 7f0bca0d7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc406a170 0x7f0bc419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:55.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.238+0000 7f0bca0d7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bc406ab30 0x7f0bc419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:55.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.238+0000 7f0bca0d7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0bc4108500 0x7f0bc4198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:55.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.238+0000 7f0bca0d7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc4112780 con 0x7f0bc406a170 2024-06-10T01:04:55.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.238+0000 7f0bca0d7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc4112600 con 0x7f0bc4108500 2024-06-10T01:04:55.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.238+0000 7f0bca0d7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc4112900 con 0x7f0bc406ab30 2024-06-10T01:04:55.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.238+0000 7f0bc37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc406a170 0x7f0bc419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:55.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.238+0000 7f0bc37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc406a170 0x7f0bc419e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44960/0 (socket says 172.21.15.7:44960) 2024-06-10T01:04:55.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.238+0000 7f0bc37fe700 1 -- 172.21.15.7:0/1058699038 learned_addr learned my addr 172.21.15.7:0/1058699038 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:55.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.239+0000 7f0bc2ffd700 1 --2- 172.21.15.7:0/1058699038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bc406ab30 0x7f0bc419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:55.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.239+0000 7f0bc37fe700 1 -- 172.21.15.7:0/1058699038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bc406ab30 msgr2=0x7f0bc419ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:55.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.239+0000 7f0bc3fff700 1 --2- 172.21.15.7:0/1058699038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0bc4108500 0x7f0bc4198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:55.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.239+0000 7f0bc37fe700 1 --2- 172.21.15.7:0/1058699038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bc406ab30 0x7f0bc419ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.239+0000 7f0bc37fe700 1 -- 172.21.15.7:0/1058699038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0bc4108500 msgr2=0x7f0bc4198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:55.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.239+0000 7f0bc37fe700 1 --2- 172.21.15.7:0/1058699038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0bc4108500 0x7f0bc4198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.239+0000 7f0bc37fe700 1 -- 172.21.15.7:0/1058699038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc41991c0 con 0x7f0bc406a170 2024-06-10T01:04:55.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.239+0000 7f0bc2ffd700 1 --2- 172.21.15.7:0/1058699038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bc406ab30 0x7f0bc419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:04:55.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.239+0000 7f0bc37fe700 1 --2- 172.21.15.7:0/1058699038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc406a170 0x7f0bc419e8a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4000c00 tx=0x7f0bb4003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:55.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.239+0000 7f0bc0ff9700 1 -- 172.21.15.7:0/1058699038 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4027070 con 0x7f0bc406a170 2024-06-10T01:04:55.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.239+0000 7f0bca0d7700 1 -- 172.21.15.7:0/1058699038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc4199450 con 0x7f0bc406a170 2024-06-10T01:04:55.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.239+0000 7f0bc0ff9700 1 -- 172.21.15.7:0/1058699038 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bb4004190 con 0x7f0bc406a170 2024-06-10T01:04:55.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.239+0000 7f0bc0ff9700 1 -- 172.21.15.7:0/1058699038 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb400f460 con 0x7f0bc406a170 2024-06-10T01:04:55.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.239+0000 7f0bca0d7700 1 -- 172.21.15.7:0/1058699038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc41a5770 con 0x7f0bc406a170 2024-06-10T01:04:55.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.241+0000 7f0bc0ff9700 1 -- 172.21.15.7:0/1058699038 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0bb402a070 con 0x7f0bc406a170 2024-06-10T01:04:55.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.242+0000 7f0bc0ff9700 1 --2- 172.21.15.7:0/1058699038 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0bac0a2c20 0x7f0bac0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:55.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.243+0000 7f0bc2ffd700 1 --2- 172.21.15.7:0/1058699038 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0bac0a2c20 0x7f0bac0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:55.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.243+0000 7f0bc0ff9700 1 -- 172.21.15.7:0/1058699038 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f0bb40d0290 con 0x7f0bc406a170 2024-06-10T01:04:55.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.243+0000 7f0bc2ffd700 1 --2- 172.21.15.7:0/1058699038 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0bac0a2c20 0x7f0bac0a50e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8005fd0 tx=0x7f0bb8005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:55.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.243+0000 7f0bd495ab80 1 Processor -- start 2024-06-10T01:04:55.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.243+0000 7f0bd495ab80 1 -- start start 2024-06-10T01:04:55.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.244+0000 7f0bd495ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a1b4cd320 0x556a1b4cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:55.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.244+0000 7f0bd495ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a1b4cdc40 0x556a1b4c7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:55.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.244+0000 7f0bd495ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a1b4c7b60 0x556a1b4c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:55.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.244+0000 7f0bd495ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556a1b3b5f10 con 0x556a1b4cd320 2024-06-10T01:04:55.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.244+0000 7f0bd495ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556a1b414800 con 0x556a1b4cdc40 2024-06-10T01:04:55.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.244+0000 7f0bd495ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556a1b37c410 con 0x556a1b4c7b60 2024-06-10T01:04:55.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.244+0000 7f0bc37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a1b4cd320 0x556a1b4cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:55.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.244+0000 7f0bc37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a1b4cd320 0x556a1b4cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44966/0 (socket says 172.21.15.7:44966) 2024-06-10T01:04:55.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.244+0000 7f0bc37fe700 1 -- 172.21.15.7:0/808761145 learned_addr learned my addr 172.21.15.7:0/808761145 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:55.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.244+0000 7f0bc2ffd700 1 --2- 172.21.15.7:0/808761145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a1b4cdc40 0x556a1b4c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:55.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.244+0000 7f0bc3fff700 1 --2- 172.21.15.7:0/808761145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a1b4c7b60 0x556a1b4c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:55.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.244+0000 7f0bc37fe700 1 -- 172.21.15.7:0/808761145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a1b4c7b60 msgr2=0x556a1b4c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:55.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.244+0000 7f0bc37fe700 1 --2- 172.21.15.7:0/808761145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a1b4c7b60 0x556a1b4c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.244+0000 7f0bc37fe700 1 -- 172.21.15.7:0/808761145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a1b4cdc40 msgr2=0x556a1b4c7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:55.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.244+0000 7f0bc37fe700 1 --2- 172.21.15.7:0/808761145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a1b4cdc40 0x556a1b4c7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.244+0000 7f0bc37fe700 1 -- 172.21.15.7:0/808761145 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a1b363880 con 0x556a1b4cd320 2024-06-10T01:04:55.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.244+0000 7f0bc37fe700 1 --2- 172.21.15.7:0/808761145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a1b4cd320 0x556a1b4cd700 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0bb40a4e50 tx=0x7f0bb4022aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28500c471d6c8e2c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:55.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.244+0000 7f0ba9ffb700 1 -- 172.21.15.7:0/808761145 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb409c7a0 con 0x556a1b4cd320 2024-06-10T01:04:55.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.244+0000 7f0ba9ffb700 1 -- 172.21.15.7:0/808761145 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bb40a0b50 con 0x556a1b4cd320 2024-06-10T01:04:55.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.244+0000 7f0ba9ffb700 1 -- 172.21.15.7:0/808761145 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4022500 con 0x556a1b4cd320 2024-06-10T01:04:55.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.245+0000 7f0bd495ab80 1 -- 172.21.15.7:0/808761145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a1b4cd320 msgr2=0x556a1b4cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:55.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.245+0000 7f0bd495ab80 1 --2- 172.21.15.7:0/808761145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a1b4cd320 0x556a1b4cd700 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0bb40a4e50 tx=0x7f0bb4022aa0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.245+0000 7f0bd495ab80 1 -- 172.21.15.7:0/808761145 shutdown_connections 2024-06-10T01:04:55.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.245+0000 7f0bd495ab80 1 --2- 172.21.15.7:0/808761145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a1b4c7b60 0x556a1b4c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.245+0000 7f0bd495ab80 1 --2- 172.21.15.7:0/808761145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a1b4cd320 0x556a1b4cd700 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.245+0000 7f0bd495ab80 1 --2- 172.21.15.7:0/808761145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a1b4cdc40 0x556a1b4c7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.245+0000 7f0bd495ab80 1 -- 172.21.15.7:0/808761145 >> 172.21.15.7:0/808761145 conn(0x556a1b3f0030 msgr2=0x556a1b4be1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:55.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.245+0000 7f0bd495ab80 1 -- 172.21.15.7:0/808761145 shutdown_connections 2024-06-10T01:04:55.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.245+0000 7f0bd495ab80 1 -- 172.21.15.7:0/808761145 wait complete. 2024-06-10T01:04:55.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.246+0000 7f0bd495ab80 1 -- start start 2024-06-10T01:04:55.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.246+0000 7f0bd495ab80 1 Processor -- start 2024-06-10T01:04:55.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.246+0000 7f0bd495ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a1b4c7b60 0x556a1b4d3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:55.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.246+0000 7f0bd495ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a1b4cd320 0x556a1b4d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:55.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.246+0000 7f0bd495ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a1b4cdc40 0x556a1b4d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:55.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.246+0000 7f0bd495ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556a1b3b4f70 con 0x556a1b4cd320 2024-06-10T01:04:55.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.246+0000 7f0bd495ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556a1b4107c0 con 0x556a1b4c7b60 2024-06-10T01:04:55.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.246+0000 7f0bd495ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556a1b4d43e0 con 0x556a1b4cdc40 2024-06-10T01:04:55.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.246+0000 7f0bd495ab80 10 client.0 _ll_register_callbacks cb 0x556a1b3aa560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:04:55.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.246+0000 7f0bc2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a1b4cd320 0x556a1b4d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:55.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.246+0000 7f0bc2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a1b4cd320 0x556a1b4d0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44974/0 (socket says 172.21.15.7:44974) 2024-06-10T01:04:55.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.246+0000 7f0bc2ffd700 1 -- 172.21.15.7:0/3760994185 learned_addr learned my addr 172.21.15.7:0/3760994185 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:55.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.246+0000 7f0bc37fe700 1 --2- 172.21.15.7:0/3760994185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a1b4c7b60 0x556a1b4d3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:55.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.246+0000 7f0bc3fff700 1 --2- 172.21.15.7:0/3760994185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a1b4cdc40 0x556a1b4d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:55.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.247+0000 7f0bc2ffd700 1 -- 172.21.15.7:0/3760994185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a1b4cdc40 msgr2=0x556a1b4d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:55.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.247+0000 7f0bc2ffd700 1 --2- 172.21.15.7:0/3760994185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a1b4cdc40 0x556a1b4d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.247+0000 7f0bc2ffd700 1 -- 172.21.15.7:0/3760994185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a1b4c7b60 msgr2=0x556a1b4d3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:55.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.247+0000 7f0bc2ffd700 1 --2- 172.21.15.7:0/3760994185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a1b4c7b60 0x556a1b4d3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.247+0000 7f0bc2ffd700 1 -- 172.21.15.7:0/3760994185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a1b3da700 con 0x556a1b4cd320 2024-06-10T01:04:55.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.247+0000 7f0bc3fff700 1 --2- 172.21.15.7:0/3760994185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a1b4cdc40 0x556a1b4d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:04:55.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.247+0000 7f0bc37fe700 1 --2- 172.21.15.7:0/3760994185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a1b4c7b60 0x556a1b4d3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:04:55.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.247+0000 7f0bc2ffd700 1 --2- 172.21.15.7:0/3760994185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a1b4cd320 0x556a1b4d0d10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0bb800f750 tx=0x7f0bb8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17331161d6d2528 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:55.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.247+0000 7f0ba2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:04:55.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.247+0000 7f0ba2ffd700 1 -- 172.21.15.7:0/3760994185 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8026670 con 0x556a1b4cd320 2024-06-10T01:04:55.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.247+0000 7f0bd495ab80 1 -- 172.21.15.7:0/3760994185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bb40a0b50 con 0x556a1b4cd320 2024-06-10T01:04:55.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.247+0000 7f0ba2ffd700 1 -- 172.21.15.7:0/3760994185 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f0bb8026810 con 0x556a1b4cd320 2024-06-10T01:04:55.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.247+0000 7f0ba2ffd700 1 -- 172.21.15.7:0/3760994185 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0bb802f050 con 0x556a1b4cd320 2024-06-10T01:04:55.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.247+0000 7f0bd495ab80 10 client.15060 fetch_fsmap learned FSMap version 26 2024-06-10T01:04:55.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.247+0000 7f0bd495ab80 1 -- 172.21.15.7:0/3760994185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556a1b4debb0 con 0x556a1b4cd320 2024-06-10T01:04:55.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.247+0000 7f0ba2ffd700 1 -- 172.21.15.7:0/3760994185 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bb40a0b50 con 0x556a1b4cd320 2024-06-10T01:04:55.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.248+0000 7f0ba2ffd700 1 -- 172.21.15.7:0/3760994185 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f0bb801dc80 con 0x556a1b4cd320 2024-06-10T01:04:55.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.248+0000 7f0bd495ab80 10 client.15060 fetch_fsmap finished waiting for FSMap version 26 2024-06-10T01:04:55.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.248+0000 7f0bd495ab80 10 client.15060 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-10T01:04:55.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.248+0000 7f0bd495ab80 20 client.15060 populate_metadata read hostname 'smithi007' 2024-06-10T01:04:55.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.248+0000 7f0bd495ab80 1 --2- 172.21.15.7:0/3760994185 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x556a1b4e1220 0x556a1b4e2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:55.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.248+0000 7f0bd495ab80 4 client.15060 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:04:55.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.248+0000 7f0bd495ab80 1 -- 172.21.15.7:0/3760994185 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a1b421d00 con 0x556a1b4e1220 2024-06-10T01:04:55.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.248+0000 7f0bc37fe700 1 --2- 172.21.15.7:0/3760994185 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x556a1b4e1220 0x556a1b4e2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:55.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.249+0000 7f0bc37fe700 1 --2- 172.21.15.7:0/3760994185 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x556a1b4e1220 0x556a1b4e2690 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9a08ab1af612b059 server_cookie=cb930fa899892183 in_seq=0 out_seq=0 2024-06-10T01:04:55.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.249+0000 7f0ba2ffd700 10 client.15060 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:04:55.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.249+0000 7f0ba2ffd700 1 -- 172.21.15.7:0/3760994185 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556a1b421d00 con 0x556a1b4e1220 2024-06-10T01:04:55.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.249+0000 7f0ba2ffd700 10 client.15060 handle_command_reply: tid=0 2024-06-10T01:04:55.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.249+0000 7f0bd495ab80 1 client.15060 shutdown 2024-06-10T01:04:55.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.250+0000 7f0bd495ab80 1 -- 172.21.15.7:0/3760994185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a1b4cd320 msgr2=0x556a1b4d0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:55.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.250+0000 7f0bd495ab80 1 --2- 172.21.15.7:0/3760994185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a1b4cd320 0x556a1b4d0d10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0bb800f750 tx=0x7f0bb8004750 comp rx=0 tx=0).stop 2024-06-10T01:04:55.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.250+0000 7f0bd495ab80 1 -- 172.21.15.7:0/3760994185 shutdown_connections 2024-06-10T01:04:55.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.250+0000 7f0bd495ab80 1 --2- 172.21.15.7:0/3760994185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a1b4cdc40 0x556a1b4d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.250+0000 7f0bd495ab80 1 --2- 172.21.15.7:0/3760994185 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x556a1b4e1220 0x556a1b4e2690 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.250+0000 7f0bd495ab80 1 --2- 172.21.15.7:0/3760994185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a1b4c7b60 0x556a1b4d3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.250+0000 7f0bd495ab80 1 --2- 172.21.15.7:0/3760994185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a1b4cd320 0x556a1b4d0d10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.250+0000 7f0ba2ffd700 0 client.15060 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:04:55.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.250+0000 7f0bd495ab80 1 -- 172.21.15.7:0/3760994185 >> 172.21.15.7:0/3760994185 conn(0x556a1b3f0030 msgr2=0x556a1b4bd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:55.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.250+0000 7f0bd495ab80 1 -- 172.21.15.7:0/3760994185 shutdown_connections 2024-06-10T01:04:55.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.250+0000 7f0bd495ab80 1 -- 172.21.15.7:0/3760994185 wait complete. 2024-06-10T01:04:55.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.250+0000 7f0bd495ab80 20 client.15060 trim_cache size 0 max 16384 2024-06-10T01:04:55.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:55 smithi007 ceph-mon[32398]: pgmap v398: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:04:55.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.272+0000 7f0bd495ab80 1 Processor -- start 2024-06-10T01:04:55.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.272+0000 7f0bd495ab80 1 -- start start 2024-06-10T01:04:55.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.272+0000 7f0bd495ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a1b4c7b60 0x556a1b4cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:55.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.272+0000 7f0bd495ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a1b4cd320 0x556a1b4ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:55.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.272+0000 7f0bd495ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a1b4cdc40 0x556a1b4cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:55.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.272+0000 7f0bd495ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556a1b4e09d0 con 0x556a1b4cd320 2024-06-10T01:04:55.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.272+0000 7f0bd495ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556a1b4e10a0 con 0x556a1b4cdc40 2024-06-10T01:04:55.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.272+0000 7f0bd495ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556a1b4e1770 con 0x556a1b4c7b60 2024-06-10T01:04:55.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.272+0000 7f0bc2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a1b4cd320 0x556a1b4ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:55.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.272+0000 7f0bc2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a1b4cd320 0x556a1b4ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44990/0 (socket says 172.21.15.7:44990) 2024-06-10T01:04:55.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.272+0000 7f0bc2ffd700 1 -- 172.21.15.7:0/3524738258 learned_addr learned my addr 172.21.15.7:0/3524738258 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:55.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.273+0000 7f0bc3fff700 1 --2- 172.21.15.7:0/3524738258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a1b4cdc40 0x556a1b4cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:55.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.273+0000 7f0bc37fe700 1 --2- 172.21.15.7:0/3524738258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a1b4c7b60 0x556a1b4cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:55.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.273+0000 7f0bc2ffd700 1 -- 172.21.15.7:0/3524738258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a1b4c7b60 msgr2=0x556a1b4cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:55.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.273+0000 7f0bc2ffd700 1 --2- 172.21.15.7:0/3524738258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a1b4c7b60 0x556a1b4cb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.273+0000 7f0bc2ffd700 1 -- 172.21.15.7:0/3524738258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a1b4cdc40 msgr2=0x556a1b4cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:55.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.273+0000 7f0bc2ffd700 1 --2- 172.21.15.7:0/3524738258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a1b4cdc40 0x556a1b4cfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.273+0000 7f0bc2ffd700 1 -- 172.21.15.7:0/3524738258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a1b4d4380 con 0x556a1b4cd320 2024-06-10T01:04:55.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.273+0000 7f0bc37fe700 1 --2- 172.21.15.7:0/3524738258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a1b4c7b60 0x556a1b4cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:04:55.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.273+0000 7f0bc2ffd700 1 --2- 172.21.15.7:0/3524738258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a1b4cd320 0x556a1b4ce8b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8012040 tx=0x7f0bb8005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdfa25756b7dcb1e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:55.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.273+0000 7f0ba97fa700 1 -- 172.21.15.7:0/3524738258 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8004910 con 0x556a1b4cd320 2024-06-10T01:04:55.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.273+0000 7f0ba97fa700 1 -- 172.21.15.7:0/3524738258 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bb8038070 con 0x556a1b4cd320 2024-06-10T01:04:55.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.273+0000 7f0ba97fa700 1 -- 172.21.15.7:0/3524738258 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb801c950 con 0x556a1b4cd320 2024-06-10T01:04:55.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.274+0000 7f0bd495ab80 1 -- 172.21.15.7:0/3524738258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a1b4cd320 msgr2=0x556a1b4ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:55.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.274+0000 7f0bd495ab80 1 --2- 172.21.15.7:0/3524738258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a1b4cd320 0x556a1b4ce8b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8012040 tx=0x7f0bb8005e30 comp rx=0 tx=0).stop 2024-06-10T01:04:55.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.274+0000 7f0bd495ab80 1 -- 172.21.15.7:0/3524738258 shutdown_connections 2024-06-10T01:04:55.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.274+0000 7f0bd495ab80 1 --2- 172.21.15.7:0/3524738258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a1b4cdc40 0x556a1b4cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.274+0000 7f0bd495ab80 1 --2- 172.21.15.7:0/3524738258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a1b4c7b60 0x556a1b4cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.274+0000 7f0bd495ab80 1 --2- 172.21.15.7:0/3524738258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a1b4cd320 0x556a1b4ce8b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.274+0000 7f0bd495ab80 1 -- 172.21.15.7:0/3524738258 >> 172.21.15.7:0/3524738258 conn(0x556a1b3f0030 msgr2=0x556a1b4cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:55.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.274+0000 7f0bd495ab80 1 -- 172.21.15.7:0/3524738258 shutdown_connections 2024-06-10T01:04:55.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.274+0000 7f0bd495ab80 1 -- 172.21.15.7:0/3524738258 wait complete. 2024-06-10T01:04:55.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.274+0000 7f0bd495ab80 1 -- start start 2024-06-10T01:04:55.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.275+0000 7f0bd495ab80 1 Processor -- start 2024-06-10T01:04:55.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.275+0000 7f0bd495ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a1b4c7b60 0x556a1b4ec070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:55.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.275+0000 7f0bd495ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a1b4cd320 0x556a1b4e53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:55.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.275+0000 7f0bd495ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a1b4cdc40 0x556a1b4e5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:55.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.275+0000 7f0bd495ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556a1b4e2510 con 0x556a1b4cd320 2024-06-10T01:04:55.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.275+0000 7f0bd495ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556a1b4e1e40 con 0x556a1b4cdc40 2024-06-10T01:04:55.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.275+0000 7f0bd495ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556a1b4e2be0 con 0x556a1b4c7b60 2024-06-10T01:04:55.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.275+0000 7f0bd495ab80 10 client.0 _ll_register_callbacks cb 0x556a1b3aa560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:04:55.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.275+0000 7f0bc2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a1b4cd320 0x556a1b4e53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:55.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.275+0000 7f0bc2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a1b4cd320 0x556a1b4e53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44996/0 (socket says 172.21.15.7:44996) 2024-06-10T01:04:55.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.275+0000 7f0bc2ffd700 1 -- 172.21.15.7:0/2146549490 learned_addr learned my addr 172.21.15.7:0/2146549490 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:55.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.275+0000 7f0bc3fff700 1 --2- 172.21.15.7:0/2146549490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a1b4cdc40 0x556a1b4e5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:55.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.275+0000 7f0bc37fe700 1 --2- 172.21.15.7:0/2146549490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a1b4c7b60 0x556a1b4ec070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:55.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.275+0000 7f0bc2ffd700 1 -- 172.21.15.7:0/2146549490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a1b4c7b60 msgr2=0x556a1b4ec070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:55.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.275+0000 7f0bc2ffd700 1 --2- 172.21.15.7:0/2146549490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a1b4c7b60 0x556a1b4ec070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.275+0000 7f0bc2ffd700 1 -- 172.21.15.7:0/2146549490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a1b4cdc40 msgr2=0x556a1b4e5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:55.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.275+0000 7f0bc2ffd700 1 --2- 172.21.15.7:0/2146549490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a1b4cdc40 0x556a1b4e5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.275+0000 7f0bc2ffd700 1 -- 172.21.15.7:0/2146549490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a1b4e5f50 con 0x556a1b4cd320 2024-06-10T01:04:55.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.275+0000 7f0bc37fe700 1 --2- 172.21.15.7:0/2146549490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a1b4c7b60 0x556a1b4ec070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:04:55.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.276+0000 7f0bc2ffd700 1 --2- 172.21.15.7:0/2146549490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a1b4cd320 0x556a1b4e53d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0bb801c500 tx=0x7f0bb801c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbd658187b0dd50a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:55.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.276+0000 7f0ba2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:04:55.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.276+0000 7f0ba2ffd700 1 -- 172.21.15.7:0/2146549490 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8042020 con 0x556a1b4cd320 2024-06-10T01:04:55.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.276+0000 7f0bd495ab80 1 -- 172.21.15.7:0/2146549490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bb8038070 con 0x556a1b4cd320 2024-06-10T01:04:55.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.276+0000 7f0ba2ffd700 1 -- 172.21.15.7:0/2146549490 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f0bb801cc10 con 0x556a1b4cd320 2024-06-10T01:04:55.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.276+0000 7f0ba2ffd700 1 -- 172.21.15.7:0/2146549490 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0bb804d050 con 0x556a1b4cd320 2024-06-10T01:04:55.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.276+0000 7f0bd495ab80 10 client.15066 fetch_fsmap learned FSMap version 26 2024-06-10T01:04:55.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.276+0000 7f0bd495ab80 1 -- 172.21.15.7:0/2146549490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556a1b4ea970 con 0x556a1b4cd320 2024-06-10T01:04:55.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.276+0000 7f0ba2ffd700 1 -- 172.21.15.7:0/2146549490 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bb8038070 con 0x556a1b4cd320 2024-06-10T01:04:55.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.277+0000 7f0ba2ffd700 1 -- 172.21.15.7:0/2146549490 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f0bb8026500 con 0x556a1b4cd320 2024-06-10T01:04:55.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.277+0000 7f0bd495ab80 10 client.15066 fetch_fsmap finished waiting for FSMap version 26 2024-06-10T01:04:55.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.277+0000 7f0bd495ab80 10 client.15066 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-10T01:04:55.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.277+0000 7f0bd495ab80 20 client.15066 populate_metadata read hostname 'smithi007' 2024-06-10T01:04:55.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.277+0000 7f0bd495ab80 1 --2- 172.21.15.7:0/2146549490 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x556a1b551060 0x556a1b54fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:55.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.277+0000 7f0bd495ab80 4 client.15066 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T01:04:55.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.277+0000 7f0bd495ab80 1 -- 172.21.15.7:0/2146549490 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "status"}) -- 0x556a1b41c280 con 0x556a1b551060 2024-06-10T01:04:55.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.277+0000 7f0bc37fe700 1 --2- 172.21.15.7:0/2146549490 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x556a1b551060 0x556a1b54fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:55.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.277+0000 7f0bc37fe700 1 --2- 172.21.15.7:0/2146549490 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x556a1b551060 0x556a1b54fd70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1b829396c5454c85 server_cookie=3d09c6fa25a675d2 in_seq=0 out_seq=0 2024-06-10T01:04:55.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.277+0000 7f0ba2ffd700 10 client.15066 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:04:55.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.278+0000 7f0ba2ffd700 1 -- 172.21.15.7:0/2146549490 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x556a1b41c280 con 0x556a1b551060 2024-06-10T01:04:55.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.278+0000 7f0ba2ffd700 10 client.15066 handle_command_reply: tid=0 2024-06-10T01:04:55.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.278+0000 7f0bd495ab80 1 client.15066 shutdown 2024-06-10T01:04:55.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.278+0000 7f0bd495ab80 1 -- 172.21.15.7:0/2146549490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a1b4cd320 msgr2=0x556a1b4e53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:55.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.278+0000 7f0bd495ab80 1 --2- 172.21.15.7:0/2146549490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a1b4cd320 0x556a1b4e53d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0bb801c500 tx=0x7f0bb801c6a0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.278+0000 7f0bd495ab80 1 -- 172.21.15.7:0/2146549490 shutdown_connections 2024-06-10T01:04:55.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.278+0000 7f0bd495ab80 1 --2- 172.21.15.7:0/2146549490 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x556a1b551060 0x556a1b54fd70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.278+0000 7f0bd495ab80 1 --2- 172.21.15.7:0/2146549490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a1b4cdc40 0x556a1b4e5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.278+0000 7f0bd495ab80 1 --2- 172.21.15.7:0/2146549490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a1b4c7b60 0x556a1b4ec070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.278+0000 7f0bd495ab80 1 --2- 172.21.15.7:0/2146549490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a1b4cd320 0x556a1b4e53d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.278+0000 7f0bd495ab80 1 -- 172.21.15.7:0/2146549490 >> 172.21.15.7:0/2146549490 conn(0x556a1b3f0030 msgr2=0x556a1b4c2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:55.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.278+0000 7f0ba2ffd700 0 client.15066 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:04:55.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.278+0000 7f0bd495ab80 1 -- 172.21.15.7:0/2146549490 shutdown_connections 2024-06-10T01:04:55.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.278+0000 7f0bd495ab80 1 -- 172.21.15.7:0/2146549490 wait complete. 2024-06-10T01:04:55.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.278+0000 7f0bd495ab80 20 client.15066 trim_cache size 0 max 16384 2024-06-10T01:04:55.280 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:04:55.280 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "bded58f8-26c3-11ef-bc9b-c7b262605968", 2024-06-10T01:04:55.281 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 1, 2024-06-10T01:04:55.281 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24436, 2024-06-10T01:04:55.281 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-06-10T01:04:55.281 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-06-10T01:04:55.281 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-06-10T01:04:55.281 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 122.59160617800001, 2024-06-10T01:04:55.281 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 26, 2024-06-10T01:04:55.281 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 99, 2024-06-10T01:04:55.281 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 97, 2024-06-10T01:04:55.281 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 218.89140356199999 2024-06-10T01:04:55.281 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:04:55.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.279+0000 7f0bca0d7700 1 -- 172.21.15.7:0/1058699038 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0bac0a2c20 msgr2=0x7f0bac0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:55.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.279+0000 7f0bca0d7700 1 --2- 172.21.15.7:0/1058699038 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0bac0a2c20 0x7f0bac0a50e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8005fd0 tx=0x7f0bb8005dc0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.279+0000 7f0bca0d7700 1 -- 172.21.15.7:0/1058699038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc406a170 msgr2=0x7f0bc419e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:55.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.279+0000 7f0bca0d7700 1 --2- 172.21.15.7:0/1058699038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc406a170 0x7f0bc419e8a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4000c00 tx=0x7f0bb4003730 comp rx=0 tx=0).stop 2024-06-10T01:04:55.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.279+0000 7f0bca0d7700 1 -- 172.21.15.7:0/1058699038 shutdown_connections 2024-06-10T01:04:55.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.279+0000 7f0bca0d7700 1 --2- 172.21.15.7:0/1058699038 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0bac0a2c20 0x7f0bac0a50e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.279+0000 7f0bca0d7700 1 --2- 172.21.15.7:0/1058699038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0bc4108500 0x7f0bc4198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.279+0000 7f0bca0d7700 1 --2- 172.21.15.7:0/1058699038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc406a170 0x7f0bc419e8a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.279+0000 7f0bca0d7700 1 --2- 172.21.15.7:0/1058699038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bc406ab30 0x7f0bc419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:55.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.279+0000 7f0bca0d7700 1 -- 172.21.15.7:0/1058699038 >> 172.21.15.7:0/1058699038 conn(0x7f0bc4078340 msgr2=0x7f0bc4100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:55.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.279+0000 7f0bca0d7700 1 -- 172.21.15.7:0/1058699038 shutdown_connections 2024-06-10T01:04:55.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:55.279+0000 7f0bca0d7700 1 -- 172.21.15.7:0/1058699038 wait complete. 2024-06-10T01:04:55.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:55 smithi175 ceph-mon[36636]: pgmap v398: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:04:55.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:55 smithi078 ceph-mon[36683]: pgmap v398: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:04:56.079 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-10T01:04:56.333 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:57.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:57 smithi175 ceph-mon[36636]: pgmap v399: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:04:57.490 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:57 smithi007 ceph-mon[32398]: pgmap v399: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:04:57.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:57 smithi078 ceph-mon[36683]: pgmap v399: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:04:58.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.005+0000 7f81753c9700 1 -- 172.21.15.7:0/2786577472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8170108560 msgr2=0x7f8170108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:58.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.005+0000 7f81753c9700 1 --2- 172.21.15.7:0/2786577472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8170108560 0x7f8170108940 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f81600099c0 tx=0x7f8160009cd0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.005+0000 7f81753c9700 1 -- 172.21.15.7:0/2786577472 shutdown_connections 2024-06-10T01:04:58.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.005+0000 7f81753c9700 1 --2- 172.21.15.7:0/2786577472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81701013a0 0x7f817010f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.005+0000 7f81753c9700 1 --2- 172.21.15.7:0/2786577472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8170108560 0x7f8170108940 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.005+0000 7f81753c9700 1 --2- 172.21.15.7:0/2786577472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81701009e0 0x7f8170100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.005+0000 7f81753c9700 1 -- 172.21.15.7:0/2786577472 >> 172.21.15.7:0/2786577472 conn(0x7f8170077990 msgr2=0x7f81700fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:58.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.005+0000 7f81753c9700 1 -- 172.21.15.7:0/2786577472 shutdown_connections 2024-06-10T01:04:58.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.005+0000 7f81753c9700 1 -- 172.21.15.7:0/2786577472 wait complete. 2024-06-10T01:04:58.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.005+0000 7f81753c9700 1 Processor -- start 2024-06-10T01:04:58.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.006+0000 7f81753c9700 1 -- start start 2024-06-10T01:04:58.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.006+0000 7f81753c9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81701009e0 0x7f817019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:58.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.006+0000 7f81753c9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81701013a0 0x7f817019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:58.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.006+0000 7f81753c9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8170108560 0x7f8170198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:58.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.006+0000 7f81753c9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8170112790 con 0x7f81701013a0 2024-06-10T01:04:58.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.006+0000 7f81753c9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8170112610 con 0x7f81701009e0 2024-06-10T01:04:58.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.006+0000 7f81753c9700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8170112910 con 0x7f8170108560 2024-06-10T01:04:58.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.006+0000 7f816e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81701013a0 0x7f817019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:58.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.006+0000 7f816e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81701013a0 0x7f817019ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45018/0 (socket says 172.21.15.7:45018) 2024-06-10T01:04:58.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.006+0000 7f816e7fc700 1 -- 172.21.15.7:0/2186888372 learned_addr learned my addr 172.21.15.7:0/2186888372 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:58.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.006+0000 7f816effd700 1 --2- 172.21.15.7:0/2186888372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81701009e0 0x7f817019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:58.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.006+0000 7f816f7fe700 1 --2- 172.21.15.7:0/2186888372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8170108560 0x7f8170198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:58.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.006+0000 7f816e7fc700 1 -- 172.21.15.7:0/2186888372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8170108560 msgr2=0x7f8170198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:58.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.006+0000 7f816e7fc700 1 --2- 172.21.15.7:0/2186888372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8170108560 0x7f8170198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.006+0000 7f816e7fc700 1 -- 172.21.15.7:0/2186888372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81701009e0 msgr2=0x7f817019e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:58.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.006+0000 7f816e7fc700 1 --2- 172.21.15.7:0/2186888372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81701009e0 0x7f817019e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.006+0000 7f816e7fc700 1 -- 172.21.15.7:0/2186888372 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8170199190 con 0x7f81701013a0 2024-06-10T01:04:58.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.007+0000 7f816e7fc700 1 --2- 172.21.15.7:0/2186888372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81701013a0 0x7f817019ed60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f816400d950 tx=0x7f816400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:58.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.007+0000 7f8157fff700 1 -- 172.21.15.7:0/2186888372 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164016070 con 0x7f81701013a0 2024-06-10T01:04:58.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.007+0000 7f8157fff700 1 -- 172.21.15.7:0/2186888372 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f816400fd70 con 0x7f81701013a0 2024-06-10T01:04:58.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.007+0000 7f81753c9700 1 -- 172.21.15.7:0/2186888372 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8170199480 con 0x7f81701013a0 2024-06-10T01:04:58.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.007+0000 7f8157fff700 1 -- 172.21.15.7:0/2186888372 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816400e610 con 0x7f81701013a0 2024-06-10T01:04:58.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.007+0000 7f81753c9700 1 -- 172.21.15.7:0/2186888372 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81701a56f0 con 0x7f81701013a0 2024-06-10T01:04:58.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.009+0000 7f8157fff700 1 -- 172.21.15.7:0/2186888372 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f8164019070 con 0x7f81701013a0 2024-06-10T01:04:58.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.010+0000 7f8157fff700 1 --2- 172.21.15.7:0/2186888372 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f81580a2c20 0x7f81580a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:58.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.010+0000 7f8157fff700 1 -- 172.21.15.7:0/2186888372 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f81640cee50 con 0x7f81701013a0 2024-06-10T01:04:58.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.010+0000 7f816effd700 1 --2- 172.21.15.7:0/2186888372 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f81580a2c20 0x7f81580a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:58.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.011+0000 7f816effd700 1 --2- 172.21.15.7:0/2186888372 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f81580a2c20 0x7f81580a50e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8160000c00 tx=0x7f8160005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:58.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.011+0000 7f817fc4cb80 1 Processor -- start 2024-06-10T01:04:58.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.011+0000 7f817fc4cb80 1 -- start start 2024-06-10T01:04:58.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.011+0000 7f817fc4cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdbb0575f0 0x55bdbb0579d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:58.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.011+0000 7f817fc4cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdbb057f10 0x55bdbb051990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:58.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.011+0000 7f817fc4cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdbb051ed0 0x55bdbb0522f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:58.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.011+0000 7f817fc4cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bdbafa37c0 con 0x55bdbb0575f0 2024-06-10T01:04:58.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.011+0000 7f817fc4cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bdbaf4b6d0 con 0x55bdbb057f10 2024-06-10T01:04:58.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.011+0000 7f817fc4cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bdbaf9f780 con 0x55bdbb051ed0 2024-06-10T01:04:58.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.011+0000 7f816effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdbb0575f0 0x55bdbb0579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:58.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.011+0000 7f816effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdbb0575f0 0x55bdbb0579d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45020/0 (socket says 172.21.15.7:45020) 2024-06-10T01:04:58.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.011+0000 7f816effd700 1 -- 172.21.15.7:0/4137857999 learned_addr learned my addr 172.21.15.7:0/4137857999 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:58.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.011+0000 7f816f7fe700 1 --2- 172.21.15.7:0/4137857999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdbb051ed0 0x55bdbb0522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:58.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.011+0000 7f816e7fc700 1 --2- 172.21.15.7:0/4137857999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdbb057f10 0x55bdbb051990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:58.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.011+0000 7f816effd700 1 -- 172.21.15.7:0/4137857999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdbb051ed0 msgr2=0x55bdbb0522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:58.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.011+0000 7f816effd700 1 --2- 172.21.15.7:0/4137857999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdbb051ed0 0x55bdbb0522f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.011+0000 7f816effd700 1 -- 172.21.15.7:0/4137857999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdbb057f10 msgr2=0x55bdbb051990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:58.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.011+0000 7f816effd700 1 --2- 172.21.15.7:0/4137857999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdbb057f10 0x55bdbb051990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.011+0000 7f816effd700 1 -- 172.21.15.7:0/4137857999 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdbaf41f00 con 0x55bdbb0575f0 2024-06-10T01:04:58.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.012+0000 7f816effd700 1 --2- 172.21.15.7:0/4137857999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdbb0575f0 0x55bdbb0579d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8160015000 tx=0x7f8160020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e5bd542e4e64d9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:58.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.012+0000 7f81557fa700 1 -- 172.21.15.7:0/4137857999 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816000be70 con 0x55bdbb0575f0 2024-06-10T01:04:58.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.012+0000 7f81557fa700 1 -- 172.21.15.7:0/4137857999 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f816000fd50 con 0x55bdbb0575f0 2024-06-10T01:04:58.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.012+0000 7f81557fa700 1 -- 172.21.15.7:0/4137857999 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816002f400 con 0x55bdbb0575f0 2024-06-10T01:04:58.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.012+0000 7f817fc4cb80 1 -- 172.21.15.7:0/4137857999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdbb0575f0 msgr2=0x55bdbb0579d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:58.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.012+0000 7f817fc4cb80 1 --2- 172.21.15.7:0/4137857999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdbb0575f0 0x55bdbb0579d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8160015000 tx=0x7f8160020000 comp rx=0 tx=0).stop 2024-06-10T01:04:58.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.013+0000 7f817fc4cb80 1 -- 172.21.15.7:0/4137857999 shutdown_connections 2024-06-10T01:04:58.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.013+0000 7f817fc4cb80 1 --2- 172.21.15.7:0/4137857999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdbb051ed0 0x55bdbb0522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.013+0000 7f817fc4cb80 1 --2- 172.21.15.7:0/4137857999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdbb0575f0 0x55bdbb0579d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.013+0000 7f817fc4cb80 1 --2- 172.21.15.7:0/4137857999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdbb057f10 0x55bdbb051990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.013+0000 7f817fc4cb80 1 -- 172.21.15.7:0/4137857999 >> 172.21.15.7:0/4137857999 conn(0x55bdbaf7cc70 msgr2=0x55bdbb04c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:58.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.013+0000 7f817fc4cb80 1 -- 172.21.15.7:0/4137857999 shutdown_connections 2024-06-10T01:04:58.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.013+0000 7f817fc4cb80 1 -- 172.21.15.7:0/4137857999 wait complete. 2024-06-10T01:04:58.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.013+0000 7f817fc4cb80 1 -- start start 2024-06-10T01:04:58.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.013+0000 7f817fc4cb80 1 Processor -- start 2024-06-10T01:04:58.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.013+0000 7f817fc4cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdbb051ed0 0x55bdbb05dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:58.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.013+0000 7f817fc4cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdbb0575f0 0x55bdbb05af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:58.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.013+0000 7f817fc4cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdbb057f10 0x55bdbb05b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:58.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.013+0000 7f817fc4cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bdbb05e8c0 con 0x55bdbb057f10 2024-06-10T01:04:58.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.013+0000 7f817fc4cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bdbaf43ba0 con 0x55bdbb051ed0 2024-06-10T01:04:58.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.013+0000 7f817fc4cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bdbb05ea40 con 0x55bdbb0575f0 2024-06-10T01:04:58.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.013+0000 7f817fc4cb80 10 client.0 _ll_register_callbacks cb 0x55bdbaf3d8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:04:58.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.013+0000 7f816f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdbb057f10 0x55bdbb05b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:58.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.013+0000 7f816f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdbb057f10 0x55bdbb05b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45024/0 (socket says 172.21.15.7:45024) 2024-06-10T01:04:58.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.013+0000 7f816effd700 1 --2- 172.21.15.7:0/4242648705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdbb051ed0 0x55bdbb05dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:58.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.013+0000 7f816f7fe700 1 -- 172.21.15.7:0/4242648705 learned_addr learned my addr 172.21.15.7:0/4242648705 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:58.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.014+0000 7f816e7fc700 1 --2- 172.21.15.7:0/4242648705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdbb0575f0 0x55bdbb05af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:58.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.014+0000 7f816f7fe700 1 -- 172.21.15.7:0/4242648705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdbb0575f0 msgr2=0x55bdbb05af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:58.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.014+0000 7f816f7fe700 1 --2- 172.21.15.7:0/4242648705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdbb0575f0 0x55bdbb05af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.014+0000 7f816f7fe700 1 -- 172.21.15.7:0/4242648705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdbb051ed0 msgr2=0x55bdbb05dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:58.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.014+0000 7f816f7fe700 1 --2- 172.21.15.7:0/4242648705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdbb051ed0 0x55bdbb05dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.014+0000 7f816f7fe700 1 -- 172.21.15.7:0/4242648705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdbaef2880 con 0x55bdbb057f10 2024-06-10T01:04:58.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.014+0000 7f816effd700 1 --2- 172.21.15.7:0/4242648705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdbb051ed0 0x55bdbb05dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:58.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.014+0000 7f816f7fe700 1 --2- 172.21.15.7:0/4242648705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdbb057f10 0x55bdbb05b4c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8168011cc0 tx=0x7f8168007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e493a24d9ed772 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:58.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.014+0000 7f8152ff5700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:04:58.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.014+0000 7f8152ff5700 1 -- 172.21.15.7:0/4242648705 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816801b020 con 0x55bdbb057f10 2024-06-10T01:04:58.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.014+0000 7f8152ff5700 1 -- 172.21.15.7:0/4242648705 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f816800f940 con 0x55bdbb057f10 2024-06-10T01:04:58.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.014+0000 7f817fc4cb80 1 -- 172.21.15.7:0/4242648705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f816000fd50 con 0x55bdbb057f10 2024-06-10T01:04:58.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.014+0000 7f8152ff5700 1 -- 172.21.15.7:0/4242648705 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8168026050 con 0x55bdbb057f10 2024-06-10T01:04:58.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.014+0000 7f817fc4cb80 10 client.15078 fetch_fsmap learned FSMap version 26 2024-06-10T01:04:58.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.014+0000 7f817fc4cb80 1 -- 172.21.15.7:0/4242648705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bdbb069280 con 0x55bdbb057f10 2024-06-10T01:04:58.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.015+0000 7f8152ff5700 1 -- 172.21.15.7:0/4242648705 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f816000fd50 con 0x55bdbb057f10 2024-06-10T01:04:58.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.015+0000 7f8152ff5700 1 -- 172.21.15.7:0/4242648705 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f8168015430 con 0x55bdbb057f10 2024-06-10T01:04:58.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.015+0000 7f817fc4cb80 10 client.15078 fetch_fsmap finished waiting for FSMap version 26 2024-06-10T01:04:58.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.015+0000 7f817fc4cb80 10 client.15078 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-10T01:04:58.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.015+0000 7f817fc4cb80 20 client.15078 populate_metadata read hostname 'smithi007' 2024-06-10T01:04:58.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.015+0000 7f817fc4cb80 1 --2- 172.21.15.7:0/4242648705 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55bdbb06b900 0x55bdbb06cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:58.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.015+0000 7f817fc4cb80 4 client.15078 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:04:58.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.015+0000 7f817fc4cb80 1 -- 172.21.15.7:0/4242648705 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdbafb0cc0 con 0x55bdbb06b900 2024-06-10T01:04:58.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.015+0000 7f816effd700 1 --2- 172.21.15.7:0/4242648705 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55bdbb06b900 0x55bdbb06cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:58.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.016+0000 7f816effd700 1 --2- 172.21.15.7:0/4242648705 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55bdbb06b900 0x55bdbb06cd70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fe0e8686df262b6 server_cookie=4a22aeea2d89a048 in_seq=0 out_seq=0 2024-06-10T01:04:58.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.016+0000 7f8152ff5700 10 client.15078 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:04:58.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.016+0000 7f8152ff5700 1 -- 172.21.15.7:0/4242648705 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bdbafb0cc0 con 0x55bdbb06b900 2024-06-10T01:04:58.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.016+0000 7f8152ff5700 10 client.15078 handle_command_reply: tid=0 2024-06-10T01:04:58.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.016+0000 7f817fc4cb80 1 client.15078 shutdown 2024-06-10T01:04:58.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.017+0000 7f817fc4cb80 1 -- 172.21.15.7:0/4242648705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdbb057f10 msgr2=0x55bdbb05b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:58.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.017+0000 7f817fc4cb80 1 --2- 172.21.15.7:0/4242648705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdbb057f10 0x55bdbb05b4c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8168011cc0 tx=0x7f8168007f10 comp rx=0 tx=0).stop 2024-06-10T01:04:58.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.017+0000 7f817fc4cb80 1 -- 172.21.15.7:0/4242648705 shutdown_connections 2024-06-10T01:04:58.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.017+0000 7f817fc4cb80 1 --2- 172.21.15.7:0/4242648705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdbb057f10 0x55bdbb05b4c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.017+0000 7f817fc4cb80 1 --2- 172.21.15.7:0/4242648705 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55bdbb06b900 0x55bdbb06cd70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.017+0000 7f817fc4cb80 1 --2- 172.21.15.7:0/4242648705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdbb051ed0 0x55bdbb05dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.017+0000 7f817fc4cb80 1 --2- 172.21.15.7:0/4242648705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdbb0575f0 0x55bdbb05af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.017+0000 7f817fc4cb80 1 -- 172.21.15.7:0/4242648705 >> 172.21.15.7:0/4242648705 conn(0x55bdbaf7cc70 msgr2=0x55bdbb04c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:58.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.017+0000 7f8152ff5700 0 client.15078 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:04:58.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.017+0000 7f817fc4cb80 1 -- 172.21.15.7:0/4242648705 shutdown_connections 2024-06-10T01:04:58.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.017+0000 7f817fc4cb80 1 -- 172.21.15.7:0/4242648705 wait complete. 2024-06-10T01:04:58.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.017+0000 7f817fc4cb80 20 client.15078 trim_cache size 0 max 16384 2024-06-10T01:04:58.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.039+0000 7f817fc4cb80 1 Processor -- start 2024-06-10T01:04:58.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.039+0000 7f817fc4cb80 1 -- start start 2024-06-10T01:04:58.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.040+0000 7f817fc4cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdbb051ed0 0x55bdbb051990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:58.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.040+0000 7f817fc4cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdbb0575f0 0x55bdbb05b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:58.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.040+0000 7f817fc4cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdbb057f10 0x55bdbb0598b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:58.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.040+0000 7f817fc4cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bdbb06b0b0 con 0x55bdbb0575f0 2024-06-10T01:04:58.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.040+0000 7f817fc4cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bdbb06b780 con 0x55bdbb057f10 2024-06-10T01:04:58.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.040+0000 7f817fc4cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bdbb06be50 con 0x55bdbb051ed0 2024-06-10T01:04:58.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.040+0000 7f816e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdbb0575f0 0x55bdbb05b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:58.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.040+0000 7f816effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdbb051ed0 0x55bdbb051990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:58.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.040+0000 7f816e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdbb0575f0 0x55bdbb05b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45038/0 (socket says 172.21.15.7:45038) 2024-06-10T01:04:58.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.040+0000 7f816e7fc700 1 -- 172.21.15.7:0/886112365 learned_addr learned my addr 172.21.15.7:0/886112365 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:58.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.040+0000 7f816f7fe700 1 --2- 172.21.15.7:0/886112365 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdbb057f10 0x55bdbb0598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:58.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.040+0000 7f816e7fc700 1 -- 172.21.15.7:0/886112365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdbb051ed0 msgr2=0x55bdbb051990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:58.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.040+0000 7f816e7fc700 1 --2- 172.21.15.7:0/886112365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdbb051ed0 0x55bdbb051990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.040+0000 7f816e7fc700 1 -- 172.21.15.7:0/886112365 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdbb057f10 msgr2=0x55bdbb0598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:58.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.040+0000 7f816e7fc700 1 --2- 172.21.15.7:0/886112365 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdbb057f10 0x55bdbb0598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.040+0000 7f816e7fc700 1 -- 172.21.15.7:0/886112365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdbb05d8c0 con 0x55bdbb0575f0 2024-06-10T01:04:58.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.040+0000 7f816f7fe700 1 --2- 172.21.15.7:0/886112365 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdbb057f10 0x55bdbb0598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:58.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.040+0000 7f816effd700 1 --2- 172.21.15.7:0/886112365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdbb051ed0 0x55bdbb051990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:58.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.041+0000 7f816e7fc700 1 --2- 172.21.15.7:0/886112365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdbb0575f0 0x55bdbb05b530 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f816409b2c0 tx=0x7f8164010b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec60db23e4268902 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:58.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.041+0000 7f8154ff9700 1 -- 172.21.15.7:0/886112365 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164020040 con 0x55bdbb0575f0 2024-06-10T01:04:58.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.041+0000 7f8154ff9700 1 -- 172.21.15.7:0/886112365 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f816409f6e0 con 0x55bdbb0575f0 2024-06-10T01:04:58.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.041+0000 7f8154ff9700 1 -- 172.21.15.7:0/886112365 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81640a3b10 con 0x55bdbb0575f0 2024-06-10T01:04:58.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.042+0000 7f817fc4cb80 1 -- 172.21.15.7:0/886112365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdbb0575f0 msgr2=0x55bdbb05b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:58.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.042+0000 7f817fc4cb80 1 --2- 172.21.15.7:0/886112365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdbb0575f0 0x55bdbb05b530 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f816409b2c0 tx=0x7f8164010b10 comp rx=0 tx=0).stop 2024-06-10T01:04:58.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.042+0000 7f817fc4cb80 1 -- 172.21.15.7:0/886112365 shutdown_connections 2024-06-10T01:04:58.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.042+0000 7f817fc4cb80 1 --2- 172.21.15.7:0/886112365 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdbb057f10 0x55bdbb0598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.042+0000 7f817fc4cb80 1 --2- 172.21.15.7:0/886112365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdbb051ed0 0x55bdbb051990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.042+0000 7f817fc4cb80 1 --2- 172.21.15.7:0/886112365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdbb0575f0 0x55bdbb05b530 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.042+0000 7f817fc4cb80 1 -- 172.21.15.7:0/886112365 >> 172.21.15.7:0/886112365 conn(0x55bdbaf7cc70 msgr2=0x55bdbb04c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:58.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.042+0000 7f817fc4cb80 1 -- 172.21.15.7:0/886112365 shutdown_connections 2024-06-10T01:04:58.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.042+0000 7f817fc4cb80 1 -- 172.21.15.7:0/886112365 wait complete. 2024-06-10T01:04:58.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.042+0000 7f817fc4cb80 1 -- start start 2024-06-10T01:04:58.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.042+0000 7f817fc4cb80 1 Processor -- start 2024-06-10T01:04:58.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.043+0000 7f817fc4cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdbb051ed0 0x55bdbb05a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:58.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.043+0000 7f817fc4cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdbb0575f0 0x55bdbb05ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:58.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.043+0000 7f817fc4cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdbb057f10 0x55bdbb0751e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:58.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.043+0000 7f817fc4cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bdbb06cbf0 con 0x55bdbb0575f0 2024-06-10T01:04:58.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.043+0000 7f817fc4cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bdbb06c520 con 0x55bdbb057f10 2024-06-10T01:04:58.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.043+0000 7f817fc4cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bdbb06d2c0 con 0x55bdbb051ed0 2024-06-10T01:04:58.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.043+0000 7f817fc4cb80 10 client.0 _ll_register_callbacks cb 0x7f81680115f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:04:58.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.043+0000 7f816e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdbb0575f0 0x55bdbb05ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:58.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.043+0000 7f816e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdbb0575f0 0x55bdbb05ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45054/0 (socket says 172.21.15.7:45054) 2024-06-10T01:04:58.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.043+0000 7f816e7fc700 1 -- 172.21.15.7:0/1696537689 learned_addr learned my addr 172.21.15.7:0/1696537689 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:04:58.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.043+0000 7f816effd700 1 --2- 172.21.15.7:0/1696537689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdbb051ed0 0x55bdbb05a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:58.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.043+0000 7f816f7fe700 1 --2- 172.21.15.7:0/1696537689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdbb057f10 0x55bdbb0751e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:58.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.043+0000 7f816e7fc700 1 -- 172.21.15.7:0/1696537689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdbb051ed0 msgr2=0x55bdbb05a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:58.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.043+0000 7f816e7fc700 1 --2- 172.21.15.7:0/1696537689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdbb051ed0 0x55bdbb05a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.043+0000 7f816e7fc700 1 -- 172.21.15.7:0/1696537689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdbb057f10 msgr2=0x55bdbb0751e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:58.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.043+0000 7f816e7fc700 1 --2- 172.21.15.7:0/1696537689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdbb057f10 0x55bdbb0751e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.043+0000 7f816e7fc700 1 -- 172.21.15.7:0/1696537689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdbb075820 con 0x55bdbb0575f0 2024-06-10T01:04:58.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.043+0000 7f816effd700 1 --2- 172.21.15.7:0/1696537689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdbb051ed0 0x55bdbb05a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:04:58.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.043+0000 7f816e7fc700 1 --2- 172.21.15.7:0/1696537689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdbb0575f0 0x55bdbb05ac40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f816409b030 tx=0x7f816409b230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acac09bef2ba4c85 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:58.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.043+0000 7f8152ff5700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:04:58.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.044+0000 7f8152ff5700 1 -- 172.21.15.7:0/1696537689 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81640b8020 con 0x55bdbb0575f0 2024-06-10T01:04:58.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.044+0000 7f8152ff5700 1 -- 172.21.15.7:0/1696537689 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f81640b4230 con 0x55bdbb0575f0 2024-06-10T01:04:58.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.044+0000 7f817fc4cb80 1 -- 172.21.15.7:0/1696537689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f816409f6e0 con 0x55bdbb0575f0 2024-06-10T01:04:58.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.044+0000 7f8152ff5700 1 -- 172.21.15.7:0/1696537689 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8164020070 con 0x55bdbb0575f0 2024-06-10T01:04:58.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.044+0000 7f817fc4cb80 10 client.15084 fetch_fsmap learned FSMap version 26 2024-06-10T01:04:58.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.044+0000 7f817fc4cb80 1 -- 172.21.15.7:0/1696537689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bdbb076b00 con 0x55bdbb0575f0 2024-06-10T01:04:58.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.044+0000 7f8152ff5700 1 -- 172.21.15.7:0/1696537689 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f816409f6e0 con 0x55bdbb0575f0 2024-06-10T01:04:58.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.044+0000 7f8152ff5700 1 -- 172.21.15.7:0/1696537689 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f816409f170 con 0x55bdbb0575f0 2024-06-10T01:04:58.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.044+0000 7f817fc4cb80 10 client.15084 fetch_fsmap finished waiting for FSMap version 26 2024-06-10T01:04:58.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.044+0000 7f817fc4cb80 10 client.15084 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-10T01:04:58.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.044+0000 7f817fc4cb80 20 client.15084 populate_metadata read hostname 'smithi007' 2024-06-10T01:04:58.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.044+0000 7f817fc4cb80 1 --2- 172.21.15.7:0/1696537689 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55bdbb0e3bb0 0x55bdbb0e28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:58.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.044+0000 7f817fc4cb80 4 client.15084 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T01:04:58.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.044+0000 7f817fc4cb80 1 -- 172.21.15.7:0/1696537689 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "status"}) -- 0x55bdbafab240 con 0x55bdbb0e3bb0 2024-06-10T01:04:58.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.045+0000 7f816effd700 1 --2- 172.21.15.7:0/1696537689 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55bdbb0e3bb0 0x55bdbb0e28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:58.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.045+0000 7f8152ff5700 10 client.15084 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:04:58.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.045+0000 7f816effd700 1 --2- 172.21.15.7:0/1696537689 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55bdbb0e3bb0 0x55bdbb0e28c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4660827dbaea064 server_cookie=e7a53ae882ebc074 in_seq=0 out_seq=0 2024-06-10T01:04:58.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.045+0000 7f8152ff5700 1 -- 172.21.15.7:0/1696537689 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55bdbafab240 con 0x55bdbb0e3bb0 2024-06-10T01:04:58.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.045+0000 7f8152ff5700 10 client.15084 handle_command_reply: tid=0 2024-06-10T01:04:58.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.045+0000 7f817fc4cb80 1 client.15084 shutdown 2024-06-10T01:04:58.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.045+0000 7f817fc4cb80 1 -- 172.21.15.7:0/1696537689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdbb0575f0 msgr2=0x55bdbb05ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:58.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.045+0000 7f817fc4cb80 1 --2- 172.21.15.7:0/1696537689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdbb0575f0 0x55bdbb05ac40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f816409b030 tx=0x7f816409b230 comp rx=0 tx=0).stop 2024-06-10T01:04:58.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.046+0000 7f817fc4cb80 1 -- 172.21.15.7:0/1696537689 shutdown_connections 2024-06-10T01:04:58.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.046+0000 7f817fc4cb80 1 --2- 172.21.15.7:0/1696537689 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55bdbb0e3bb0 0x55bdbb0e28c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.046+0000 7f817fc4cb80 1 --2- 172.21.15.7:0/1696537689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdbb057f10 0x55bdbb0751e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.046+0000 7f817fc4cb80 1 --2- 172.21.15.7:0/1696537689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdbb051ed0 0x55bdbb05a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.046+0000 7f8152ff5700 0 client.15084 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:04:58.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.046+0000 7f817fc4cb80 1 --2- 172.21.15.7:0/1696537689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdbb0575f0 0x55bdbb05ac40 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.046+0000 7f817fc4cb80 1 -- 172.21.15.7:0/1696537689 >> 172.21.15.7:0/1696537689 conn(0x55bdbaf7cc70 msgr2=0x55bdbb045db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:58.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.046+0000 7f817fc4cb80 1 -- 172.21.15.7:0/1696537689 shutdown_connections 2024-06-10T01:04:58.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.046+0000 7f817fc4cb80 1 -- 172.21.15.7:0/1696537689 wait complete. 2024-06-10T01:04:58.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.046+0000 7f817fc4cb80 20 client.15084 trim_cache size 0 max 16384 2024-06-10T01:04:58.048 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:04:58.048 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "bded58f8-26c3-11ef-bc9b-c7b262605968", 2024-06-10T01:04:58.048 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-06-10T01:04:58.048 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24451, 2024-06-10T01:04:58.048 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-06-10T01:04:58.048 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-06-10T01:04:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-06-10T01:04:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 188.808875075, 2024-06-10T01:04:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 26, 2024-06-10T01:04:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 99, 2024-06-10T01:04:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 97, 2024-06-10T01:04:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 215.90412583700001 2024-06-10T01:04:58.049 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:04:58.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.046+0000 7f81753c9700 1 -- 172.21.15.7:0/2186888372 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f81580a2c20 msgr2=0x7f81580a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:58.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.046+0000 7f81753c9700 1 --2- 172.21.15.7:0/2186888372 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f81580a2c20 0x7f81580a50e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8160000c00 tx=0x7f8160005fb0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.046+0000 7f81753c9700 1 -- 172.21.15.7:0/2186888372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81701013a0 msgr2=0x7f817019ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:58.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.046+0000 7f81753c9700 1 --2- 172.21.15.7:0/2186888372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81701013a0 0x7f817019ed60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f816400d950 tx=0x7f816400dc60 comp rx=0 tx=0).stop 2024-06-10T01:04:58.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.046+0000 7f81753c9700 1 -- 172.21.15.7:0/2186888372 shutdown_connections 2024-06-10T01:04:58.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.046+0000 7f81753c9700 1 --2- 172.21.15.7:0/2186888372 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f81580a2c20 0x7f81580a50e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.047+0000 7f81753c9700 1 --2- 172.21.15.7:0/2186888372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8170108560 0x7f8170198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.047+0000 7f81753c9700 1 --2- 172.21.15.7:0/2186888372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81701009e0 0x7f817019e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.047+0000 7f81753c9700 1 --2- 172.21.15.7:0/2186888372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81701013a0 0x7f817019ed60 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:58.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.047+0000 7f81753c9700 1 -- 172.21.15.7:0/2186888372 >> 172.21.15.7:0/2186888372 conn(0x7f8170077990 msgr2=0x7f81700fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:58.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.047+0000 7f81753c9700 1 -- 172.21.15.7:0/2186888372 shutdown_connections 2024-06-10T01:04:58.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:04:58.047+0000 7f81753c9700 1 -- 172.21.15.7:0/2186888372 wait complete. 2024-06-10T01:04:58.628 INFO:tasks.cephadm:Setup complete, yielding 2024-06-10T01:04:58.628 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-10T01:04:58.641 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-06-10T01:04:58.641 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch status' 2024-06-10T01:04:58.895 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:59.278 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:04:59 smithi007 ceph-mon[32398]: pgmap v400: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:04:59.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:04:59 smithi175 ceph-mon[36636]: pgmap v400: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:04:59.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:04:59 smithi078 ceph-mon[36683]: pgmap v400: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:05:00.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.358+0000 7ff4c025d700 1 -- 172.21.15.7:0/3959023531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4b810ead0 msgr2=0x7ff4b8110ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:00.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.358+0000 7ff4c025d700 1 --2- 172.21.15.7:0/3959023531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4b810ead0 0x7ff4b8110ec0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff4b000b0a0 tx=0x7ff4b000b3b0 comp rx=0 tx=0).stop 2024-06-10T01:05:00.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.358+0000 7ff4c025d700 1 -- 172.21.15.7:0/3959023531 shutdown_connections 2024-06-10T01:05:00.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.358+0000 7ff4c025d700 1 --2- 172.21.15.7:0/3959023531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4b810ead0 0x7ff4b8110ec0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:00.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.358+0000 7ff4c025d700 1 --2- 172.21.15.7:0/3959023531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4b80764c0 0x7ff4b8074920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:00.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.358+0000 7ff4c025d700 1 --2- 172.21.15.7:0/3959023531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4b8074ef0 0x7ff4b8075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:00.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.358+0000 7ff4c025d700 1 -- 172.21.15.7:0/3959023531 >> 172.21.15.7:0/3959023531 conn(0x7ff4b80fda90 msgr2=0x7ff4b80ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:00.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.358+0000 7ff4c025d700 1 -- 172.21.15.7:0/3959023531 shutdown_connections 2024-06-10T01:05:00.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.358+0000 7ff4c025d700 1 -- 172.21.15.7:0/3959023531 wait complete. 2024-06-10T01:05:00.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.359+0000 7ff4c025d700 1 Processor -- start 2024-06-10T01:05:00.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.359+0000 7ff4c025d700 1 -- start start 2024-06-10T01:05:00.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.359+0000 7ff4c025d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4b8074ef0 0x7ff4b819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:00.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.359+0000 7ff4c025d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4b80764c0 0x7ff4b819f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:00.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.359+0000 7ff4c025d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4b810ead0 0x7ff4b81a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:00.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.359+0000 7ff4c025d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b8114740 con 0x7ff4b80764c0 2024-06-10T01:05:00.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.359+0000 7ff4c025d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b81145c0 con 0x7ff4b810ead0 2024-06-10T01:05:00.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.359+0000 7ff4c025d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b81148c0 con 0x7ff4b8074ef0 2024-06-10T01:05:00.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.359+0000 7ff4bd7f8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4b80764c0 0x7ff4b819f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:00.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.359+0000 7ff4bd7f8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4b80764c0 0x7ff4b819f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45072/0 (socket says 172.21.15.7:45072) 2024-06-10T01:05:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.359+0000 7ff4bd7f8700 1 -- 172.21.15.7:0/76191267 learned_addr learned my addr 172.21.15.7:0/76191267 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.359+0000 7ff4bdff9700 1 --2- 172.21.15.7:0/76191267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4b8074ef0 0x7ff4b819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.359+0000 7ff4be7fa700 1 --2- 172.21.15.7:0/76191267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4b810ead0 0x7ff4b81a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.359+0000 7ff4bd7f8700 1 -- 172.21.15.7:0/76191267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4b8074ef0 msgr2=0x7ff4b819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.359+0000 7ff4bd7f8700 1 --2- 172.21.15.7:0/76191267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4b8074ef0 0x7ff4b819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.359+0000 7ff4bd7f8700 1 -- 172.21.15.7:0/76191267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4b810ead0 msgr2=0x7ff4b81a62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.359+0000 7ff4bd7f8700 1 --2- 172.21.15.7:0/76191267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4b810ead0 0x7ff4b81a62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.360+0000 7ff4bd7f8700 1 -- 172.21.15.7:0/76191267 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b81a6970 con 0x7ff4b80764c0 2024-06-10T01:05:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.360+0000 7ff4bd7f8700 1 --2- 172.21.15.7:0/76191267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4b80764c0 0x7ff4b819f2b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff4a800d980 tx=0x7ff4a800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.360+0000 7ff4aeffd700 1 -- 172.21.15.7:0/76191267 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8017070 con 0x7ff4b80764c0 2024-06-10T01:05:00.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.360+0000 7ff4aeffd700 1 -- 172.21.15.7:0/76191267 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a800fd70 con 0x7ff4b80764c0 2024-06-10T01:05:00.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.360+0000 7ff4c025d700 1 -- 172.21.15.7:0/76191267 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b81a6c60 con 0x7ff4b80764c0 2024-06-10T01:05:00.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.360+0000 7ff4aeffd700 1 -- 172.21.15.7:0/76191267 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a800e3b0 con 0x7ff4b80764c0 2024-06-10T01:05:00.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.360+0000 7ff4c025d700 1 -- 172.21.15.7:0/76191267 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4b81a71a0 con 0x7ff4b80764c0 2024-06-10T01:05:00.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.361+0000 7ff4c025d700 1 -- 172.21.15.7:0/76191267 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4b8069180 con 0x7ff4b80764c0 2024-06-10T01:05:00.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.362+0000 7ff4aeffd700 1 -- 172.21.15.7:0/76191267 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7ff4a800f890 con 0x7ff4b80764c0 2024-06-10T01:05:00.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.363+0000 7ff4aeffd700 1 --2- 172.21.15.7:0/76191267 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff4a40a2ca0 0x7ff4a40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:00.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.363+0000 7ff4bdff9700 1 --2- 172.21.15.7:0/76191267 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff4a40a2ca0 0x7ff4a40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:00.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.363+0000 7ff4aeffd700 1 -- 172.21.15.7:0/76191267 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7ff4a80cee60 con 0x7ff4b80764c0 2024-06-10T01:05:00.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.363+0000 7ff4bdff9700 1 --2- 172.21.15.7:0/76191267 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff4a40a2ca0 0x7ff4a40a5160 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4005950 tx=0x7ff4b400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:00.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.367+0000 7ff4aeffd700 1 -- 172.21.15.7:0/76191267 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff4a8010460 con 0x7ff4b80764c0 2024-06-10T01:05:00.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.592+0000 7ff4c025d700 1 -- 172.21.15.7:0/76191267 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7ff4b80628d0 con 0x7ff4a40a2ca0 2024-06-10T01:05:00.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.593+0000 7ff4aeffd700 1 -- 172.21.15.7:0/76191267 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7ff4b80628d0 con 0x7ff4a40a2ca0 2024-06-10T01:05:00.590 INFO:teuthology.orchestra.run.smithi007.stdout:Backend: cephadm 2024-06-10T01:05:00.590 INFO:teuthology.orchestra.run.smithi007.stdout:Available: Yes 2024-06-10T01:05:00.590 INFO:teuthology.orchestra.run.smithi007.stdout:Paused: No 2024-06-10T01:05:00.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.595+0000 7ff4c025d700 1 -- 172.21.15.7:0/76191267 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff4a40a2ca0 msgr2=0x7ff4a40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:00.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.595+0000 7ff4c025d700 1 --2- 172.21.15.7:0/76191267 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff4a40a2ca0 0x7ff4a40a5160 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4005950 tx=0x7ff4b400a300 comp rx=0 tx=0).stop 2024-06-10T01:05:00.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.595+0000 7ff4c025d700 1 -- 172.21.15.7:0/76191267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4b80764c0 msgr2=0x7ff4b819f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:00.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.595+0000 7ff4c025d700 1 --2- 172.21.15.7:0/76191267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4b80764c0 0x7ff4b819f2b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff4a800d980 tx=0x7ff4a800dc90 comp rx=0 tx=0).stop 2024-06-10T01:05:00.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.596+0000 7ff4c025d700 1 -- 172.21.15.7:0/76191267 shutdown_connections 2024-06-10T01:05:00.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.596+0000 7ff4c025d700 1 --2- 172.21.15.7:0/76191267 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff4a40a2ca0 0x7ff4a40a5160 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:00.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.596+0000 7ff4c025d700 1 --2- 172.21.15.7:0/76191267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4b810ead0 0x7ff4b81a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:00.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.596+0000 7ff4c025d700 1 --2- 172.21.15.7:0/76191267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4b8074ef0 0x7ff4b819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:00.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.596+0000 7ff4c025d700 1 --2- 172.21.15.7:0/76191267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4b80764c0 0x7ff4b819f2b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:00.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.596+0000 7ff4c025d700 1 -- 172.21.15.7:0/76191267 >> 172.21.15.7:0/76191267 conn(0x7ff4b80fda90 msgr2=0x7ff4b8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:00.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.596+0000 7ff4c025d700 1 -- 172.21.15.7:0/76191267 shutdown_connections 2024-06-10T01:05:00.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:00.596+0000 7ff4c025d700 1 -- 172.21.15.7:0/76191267 wait complete. 2024-06-10T01:05:01.191 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:01 smithi007 ceph-mon[32398]: pgmap v401: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T01:05:01.225 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ps' 2024-06-10T01:05:01.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:01 smithi175 ceph-mon[36636]: pgmap v401: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T01:05:01.482 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:05:01.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:01 smithi078 ceph-mon[36683]: pgmap v401: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T01:05:02.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:02 smithi007 ceph-mon[32398]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:05:02.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:02 smithi175 ceph-mon[36636]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:05:02.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:02 smithi078 ceph-mon[36683]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:05:03.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.119+0000 7f0473d7e700 1 -- 172.21.15.7:0/821618102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f046c105970 msgr2=0x7f046c107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:03.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.119+0000 7f0473d7e700 1 --2- 172.21.15.7:0/821618102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f046c105970 0x7f046c107d60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0460009990 tx=0x7f0460009ca0 comp rx=0 tx=0).stop 2024-06-10T01:05:03.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.120+0000 7f0473d7e700 1 -- 172.21.15.7:0/821618102 shutdown_connections 2024-06-10T01:05:03.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.120+0000 7f0473d7e700 1 --2- 172.21.15.7:0/821618102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f046c1082a0 0x7f046c10a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:03.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.120+0000 7f0473d7e700 1 --2- 172.21.15.7:0/821618102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f046c103040 0x7f046c105430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:03.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.120+0000 7f0473d7e700 1 --2- 172.21.15.7:0/821618102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f046c105970 0x7f046c107d60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:03.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.120+0000 7f0473d7e700 1 -- 172.21.15.7:0/821618102 >> 172.21.15.7:0/821618102 conn(0x7f046c0fc990 msgr2=0x7f046c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:03.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.120+0000 7f0473d7e700 1 -- 172.21.15.7:0/821618102 shutdown_connections 2024-06-10T01:05:03.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.120+0000 7f0473d7e700 1 -- 172.21.15.7:0/821618102 wait complete. 2024-06-10T01:05:03.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.120+0000 7f0473d7e700 1 Processor -- start 2024-06-10T01:05:03.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.121+0000 7f0473d7e700 1 -- start start 2024-06-10T01:05:03.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.121+0000 7f0473d7e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f046c103040 0x7f046c198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:03.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.121+0000 7f0473d7e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f046c105970 0x7f046c198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:03.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.121+0000 7f0473d7e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f046c1082a0 0x7f046c19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:03.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.121+0000 7f0473d7e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f046c10e0b0 con 0x7f046c105970 2024-06-10T01:05:03.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.121+0000 7f0473d7e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f046c10df30 con 0x7f046c103040 2024-06-10T01:05:03.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.121+0000 7f0473d7e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f046c10e230 con 0x7f046c1082a0 2024-06-10T01:05:03.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.121+0000 7f0471319700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f046c105970 0x7f046c198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:03.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.121+0000 7f0471319700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f046c105970 0x7f046c198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44756/0 (socket says 172.21.15.7:44756) 2024-06-10T01:05:03.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.121+0000 7f0471319700 1 -- 172.21.15.7:0/2420060433 learned_addr learned my addr 172.21.15.7:0/2420060433 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:03.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.121+0000 7f0471b1a700 1 --2- 172.21.15.7:0/2420060433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f046c103040 0x7f046c198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:03.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.121+0000 7f047231b700 1 --2- 172.21.15.7:0/2420060433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f046c1082a0 0x7f046c19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:03.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.121+0000 7f0471319700 1 -- 172.21.15.7:0/2420060433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f046c1082a0 msgr2=0x7f046c19fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:03.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.121+0000 7f0471319700 1 --2- 172.21.15.7:0/2420060433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f046c1082a0 0x7f046c19fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:03.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.121+0000 7f0471319700 1 -- 172.21.15.7:0/2420060433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f046c103040 msgr2=0x7f046c198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:03.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.121+0000 7f0471319700 1 --2- 172.21.15.7:0/2420060433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f046c103040 0x7f046c198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:03.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.121+0000 7f0471319700 1 -- 172.21.15.7:0/2420060433 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f046c1a0390 con 0x7f046c105970 2024-06-10T01:05:03.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.121+0000 7f047231b700 1 --2- 172.21.15.7:0/2420060433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f046c1082a0 0x7f046c19fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:05:03.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.122+0000 7f0471319700 1 --2- 172.21.15.7:0/2420060433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f046c105970 0x7f046c198c70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0460000c00 tx=0x7f046000bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:03.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.122+0000 7f045effd700 1 -- 172.21.15.7:0/2420060433 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460027070 con 0x7f046c105970 2024-06-10T01:05:03.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.122+0000 7f045effd700 1 -- 172.21.15.7:0/2420060433 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0460003d70 con 0x7f046c105970 2024-06-10T01:05:03.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.122+0000 7f0473d7e700 1 -- 172.21.15.7:0/2420060433 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f046c1a0620 con 0x7f046c105970 2024-06-10T01:05:03.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.122+0000 7f045effd700 1 -- 172.21.15.7:0/2420060433 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046000f460 con 0x7f046c105970 2024-06-10T01:05:03.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.122+0000 7f0473d7e700 1 -- 172.21.15.7:0/2420060433 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f046c1a0ad0 con 0x7f046c105970 2024-06-10T01:05:03.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.123+0000 7f0473d7e700 1 -- 172.21.15.7:0/2420060433 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f046c069180 con 0x7f046c105970 2024-06-10T01:05:03.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.124+0000 7f045effd700 1 -- 172.21.15.7:0/2420060433 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f046002a070 con 0x7f046c105970 2024-06-10T01:05:03.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.125+0000 7f045effd700 1 --2- 172.21.15.7:0/2420060433 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f04580a2ca0 0x7f04580a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:03.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.125+0000 7f0471b1a700 1 --2- 172.21.15.7:0/2420060433 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f04580a2ca0 0x7f04580a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:03.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.125+0000 7f045effd700 1 -- 172.21.15.7:0/2420060433 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f04600d0230 con 0x7f046c105970 2024-06-10T01:05:03.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.125+0000 7f0471b1a700 1 --2- 172.21.15.7:0/2420060433 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f04580a2ca0 0x7f04580a5160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0468009c60 tx=0x7f0468009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:03.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.129+0000 7f045effd700 1 -- 172.21.15.7:0/2420060433 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0460013070 con 0x7f046c105970 2024-06-10T01:05:03.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.347+0000 7f0473d7e700 1 -- 172.21.15.7:0/2420060433 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f046c0628d0 con 0x7f04580a2ca0 2024-06-10T01:05:03.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.361+0000 7f045effd700 1 -- 172.21.15.7:0/2420060433 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f046c0628d0 con 0x7f04580a2ca0 2024-06-10T01:05:03.358 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-10T01:05:03.358 INFO:teuthology.orchestra.run.smithi007.stdout:mds.a smithi007 running (3m) 35s ago 3m 16.8M - 19.0.0-2946-g3d89accf b045ac545f82 b38668170db2 2024-06-10T01:05:03.358 INFO:teuthology.orchestra.run.smithi007.stdout:mds.b smithi078 running (3m) 72s ago 3m 20.2M - 19.0.0-2946-g3d89accf b045ac545f82 9dbe77293d68 2024-06-10T01:05:03.358 INFO:teuthology.orchestra.run.smithi007.stdout:mds.c smithi175 running (3m) 72s ago 3m 17.9M - 19.0.0-2946-g3d89accf b045ac545f82 7d697cfd2a30 2024-06-10T01:05:03.358 INFO:teuthology.orchestra.run.smithi007.stdout:mds.d smithi007 running (3m) 35s ago 3m 13.5M - 19.0.0-2946-g3d89accf b045ac545f82 894ef88f23c5 2024-06-10T01:05:03.358 INFO:teuthology.orchestra.run.smithi007.stdout:mds.e smithi078 running (3m) 72s ago 3m 14.5M - 19.0.0-2946-g3d89accf b045ac545f82 fa29baba2c02 2024-06-10T01:05:03.358 INFO:teuthology.orchestra.run.smithi007.stdout:mds.f smithi175 running (3m) 72s ago 3m 19.8M - 19.0.0-2946-g3d89accf b045ac545f82 563aad482e1c 2024-06-10T01:05:03.359 INFO:teuthology.orchestra.run.smithi007.stdout:mds.g smithi007 running (3m) 35s ago 3m 14.1M - 19.0.0-2946-g3d89accf b045ac545f82 1a6ae155a59e 2024-06-10T01:05:03.359 INFO:teuthology.orchestra.run.smithi007.stdout:mds.h smithi078 running (3m) 72s ago 3m 19.6M - 19.0.0-2946-g3d89accf b045ac545f82 a9cd2441b6b2 2024-06-10T01:05:03.359 INFO:teuthology.orchestra.run.smithi007.stdout:mds.i smithi175 running (3m) 72s ago 3m 26.1M - 19.0.0-2946-g3d89accf b045ac545f82 8e302b883cb8 2024-06-10T01:05:03.359 INFO:teuthology.orchestra.run.smithi007.stdout:mds.j smithi007 running (3m) 35s ago 3m 13.6M - 19.0.0-2946-g3d89accf b045ac545f82 7f40cbd8be74 2024-06-10T01:05:03.359 INFO:teuthology.orchestra.run.smithi007.stdout:mds.k smithi078 running (4m) 72s ago 4m 13.7M - 19.0.0-2946-g3d89accf b045ac545f82 b77d167768fd 2024-06-10T01:05:03.359 INFO:teuthology.orchestra.run.smithi007.stdout:mds.l smithi175 running (4m) 72s ago 4m 17.1M - 19.0.0-2946-g3d89accf b045ac545f82 2b561a985a1f 2024-06-10T01:05:03.359 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.x smithi007 *:9283,8765 running (11m) 35s ago 11m 519M - 19.0.0-2946-g3d89accf b045ac545f82 4bce828fe08d 2024-06-10T01:05:03.359 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.y smithi078 *:8443,8765 running (9m) 72s ago 9m 454M - 19.0.0-2946-g3d89accf b045ac545f82 c999d5b9e20f 2024-06-10T01:05:03.359 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.z smithi175 *:8443,8765 running (9m) 72s ago 9m 454M - 19.0.0-2946-g3d89accf b045ac545f82 69ef91707a6f 2024-06-10T01:05:03.359 INFO:teuthology.orchestra.run.smithi007.stdout:mon.a smithi007 running (11m) 35s ago 11m 53.3M 2048M 19.0.0-2946-g3d89accf b045ac545f82 afb11117c1a1 2024-06-10T01:05:03.359 INFO:teuthology.orchestra.run.smithi007.stdout:mon.b smithi078 running (9m) 72s ago 9m 42.4M 2048M 19.0.0-2946-g3d89accf b045ac545f82 e51fad75031b 2024-06-10T01:05:03.359 INFO:teuthology.orchestra.run.smithi007.stdout:mon.c smithi175 running (9m) 72s ago 9m 40.3M 2048M 19.0.0-2946-g3d89accf b045ac545f82 260eedf05976 2024-06-10T01:05:03.359 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (8m) 35s ago 8m 55.4M 4273M 19.0.0-2946-g3d89accf b045ac545f82 f38bf96bcb02 2024-06-10T01:05:03.360 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi078 running (8m) 72s ago 8m 49.0M 4272M 19.0.0-2946-g3d89accf b045ac545f82 4f6724a1934f 2024-06-10T01:05:03.360 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi175 running (8m) 72s ago 8m 47.6M 4272M 19.0.0-2946-g3d89accf b045ac545f82 0a21f6282f93 2024-06-10T01:05:03.360 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi007 running (7m) 35s ago 7m 44.6M 4273M 19.0.0-2946-g3d89accf b045ac545f82 4a8ca8b53658 2024-06-10T01:05:03.360 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi078 running (7m) 72s ago 7m 43.9M 4272M 19.0.0-2946-g3d89accf b045ac545f82 08f853dcb81e 2024-06-10T01:05:03.360 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi175 running (6m) 72s ago 6m 48.3M 4272M 19.0.0-2946-g3d89accf b045ac545f82 635dcc0e148a 2024-06-10T01:05:03.360 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6 smithi007 running (6m) 35s ago 6m 41.9M 4273M 19.0.0-2946-g3d89accf b045ac545f82 6a93b8ca5f4d 2024-06-10T01:05:03.360 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7 smithi078 running (6m) 72s ago 6m 44.9M 4272M 19.0.0-2946-g3d89accf b045ac545f82 78bc30409072 2024-06-10T01:05:03.360 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8 smithi175 running (5m) 72s ago 5m 44.5M 4272M 19.0.0-2946-g3d89accf b045ac545f82 da89affc86f5 2024-06-10T01:05:03.360 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9 smithi007 running (5m) 35s ago 5m 53.6M 4273M 19.0.0-2946-g3d89accf b045ac545f82 9a637f3e3124 2024-06-10T01:05:03.360 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10 smithi078 running (4m) 72s ago 4m 55.4M 4272M 19.0.0-2946-g3d89accf b045ac545f82 e8a573a639cd 2024-06-10T01:05:03.360 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11 smithi175 running (4m) 72s ago 4m 56.5M 4272M 19.0.0-2946-g3d89accf b045ac545f82 9861c67ad596 2024-06-10T01:05:03.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.363+0000 7f0473d7e700 1 -- 172.21.15.7:0/2420060433 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f04580a2ca0 msgr2=0x7f04580a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:03.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.363+0000 7f0473d7e700 1 --2- 172.21.15.7:0/2420060433 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f04580a2ca0 0x7f04580a5160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0468009c60 tx=0x7f0468009260 comp rx=0 tx=0).stop 2024-06-10T01:05:03.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.363+0000 7f0473d7e700 1 -- 172.21.15.7:0/2420060433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f046c105970 msgr2=0x7f046c198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:03.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.363+0000 7f0473d7e700 1 --2- 172.21.15.7:0/2420060433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f046c105970 0x7f046c198c70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0460000c00 tx=0x7f046000bbb0 comp rx=0 tx=0).stop 2024-06-10T01:05:03.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.363+0000 7f0473d7e700 1 -- 172.21.15.7:0/2420060433 shutdown_connections 2024-06-10T01:05:03.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.363+0000 7f0473d7e700 1 --2- 172.21.15.7:0/2420060433 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f04580a2ca0 0x7f04580a5160 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:03.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.363+0000 7f0473d7e700 1 --2- 172.21.15.7:0/2420060433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f046c1082a0 0x7f046c19fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:03.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.363+0000 7f0473d7e700 1 --2- 172.21.15.7:0/2420060433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f046c103040 0x7f046c198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:03.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.363+0000 7f0473d7e700 1 --2- 172.21.15.7:0/2420060433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f046c105970 0x7f046c198c70 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:03.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.363+0000 7f0473d7e700 1 -- 172.21.15.7:0/2420060433 >> 172.21.15.7:0/2420060433 conn(0x7f046c0fc990 msgr2=0x7f046c103ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:03.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.363+0000 7f0473d7e700 1 -- 172.21.15.7:0/2420060433 shutdown_connections 2024-06-10T01:05:03.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:03.363+0000 7f0473d7e700 1 -- 172.21.15.7:0/2420060433 wait complete. 2024-06-10T01:05:03.371 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:03 smithi007 ceph-mon[32398]: pgmap v402: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:05:03.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:03 smithi175 ceph-mon[36636]: pgmap v402: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:05:03.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:03 smithi078 ceph-mon[36683]: pgmap v402: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:05:03.990 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ls' 2024-06-10T01:05:04.245 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:05:04.367 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:04 smithi007 ceph-mon[32398]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:05:04.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:04 smithi175 ceph-mon[36636]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:05:04.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:04 smithi078 ceph-mon[36683]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:05:05.334 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:05 smithi007 ceph-mon[32398]: pgmap v403: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:05:05.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:05 smithi175 ceph-mon[36636]: pgmap v403: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:05:05.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:05 smithi078 ceph-mon[36683]: pgmap v403: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:05:05.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.894+0000 7f6bb214c700 1 -- 172.21.15.7:0/3497613889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bac104090 msgr2=0x7f6bac10a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:05.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.894+0000 7f6bb214c700 1 --2- 172.21.15.7:0/3497613889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bac104090 0x7f6bac10a940 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400b0d0 tx=0x7f6ba400b3e0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.894+0000 7f6bb214c700 1 -- 172.21.15.7:0/3497613889 shutdown_connections 2024-06-10T01:05:05.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.894+0000 7f6bb214c700 1 --2- 172.21.15.7:0/3497613889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bac104090 0x7f6bac10a940 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.894+0000 7f6bb214c700 1 --2- 172.21.15.7:0/3497613889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bac1024f0 0x7f6bac102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.894+0000 7f6bb214c700 1 --2- 172.21.15.7:0/3497613889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bac1036f0 0x7f6bac103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.894+0000 7f6bb214c700 1 -- 172.21.15.7:0/3497613889 >> 172.21.15.7:0/3497613889 conn(0x7f6bac0fda90 msgr2=0x7f6bac0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:05.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.894+0000 7f6bb214c700 1 -- 172.21.15.7:0/3497613889 shutdown_connections 2024-06-10T01:05:05.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.894+0000 7f6bb214c700 1 -- 172.21.15.7:0/3497613889 wait complete. 2024-06-10T01:05:05.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.895+0000 7f6bb214c700 1 Processor -- start 2024-06-10T01:05:05.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.895+0000 7f6bb214c700 1 -- start start 2024-06-10T01:05:05.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.895+0000 7f6bb214c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bac1024f0 0x7f6bac19eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:05.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.895+0000 7f6bb214c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bac1036f0 0x7f6bac19f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:05.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.895+0000 7f6bb214c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bac104090 0x7f6bac1a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:05.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.895+0000 7f6bb214c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6bac10e0e0 con 0x7f6bac104090 2024-06-10T01:05:05.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.895+0000 7f6bb214c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6bac10df60 con 0x7f6bac1024f0 2024-06-10T01:05:05.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.895+0000 7f6bb214c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6bac10e260 con 0x7f6bac1036f0 2024-06-10T01:05:05.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.895+0000 7f6babfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bac104090 0x7f6bac1a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:05.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.895+0000 7f6babfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bac104090 0x7f6bac1a6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44772/0 (socket says 172.21.15.7:44772) 2024-06-10T01:05:05.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.895+0000 7f6babfff700 1 -- 172.21.15.7:0/4247800505 learned_addr learned my addr 172.21.15.7:0/4247800505 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:05.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.895+0000 7f6bab7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bac1024f0 0x7f6bac19eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:05.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.896+0000 7f6ba2ffd700 1 --2- 172.21.15.7:0/4247800505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bac1036f0 0x7f6bac19f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:05.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.896+0000 7f6babfff700 1 -- 172.21.15.7:0/4247800505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bac1036f0 msgr2=0x7f6bac19f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:05.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.896+0000 7f6babfff700 1 --2- 172.21.15.7:0/4247800505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bac1036f0 0x7f6bac19f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.896+0000 7f6babfff700 1 -- 172.21.15.7:0/4247800505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bac1024f0 msgr2=0x7f6bac19eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:05.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.896+0000 7f6babfff700 1 --2- 172.21.15.7:0/4247800505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bac1024f0 0x7f6bac19eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.896+0000 7f6babfff700 1 -- 172.21.15.7:0/4247800505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bac1a6920 con 0x7f6bac104090 2024-06-10T01:05:05.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.896+0000 7f6ba2ffd700 1 --2- 172.21.15.7:0/4247800505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bac1036f0 0x7f6bac19f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:05:05.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.896+0000 7f6babfff700 1 --2- 172.21.15.7:0/4247800505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bac104090 0x7f6bac1a6250 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4006070 tx=0x7f6ba4007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:05.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.896+0000 7f6ba97fa700 1 -- 172.21.15.7:0/4247800505 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4021020 con 0x7f6bac104090 2024-06-10T01:05:05.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.896+0000 7f6ba97fa700 1 -- 172.21.15.7:0/4247800505 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ba401d950 con 0x7f6bac104090 2024-06-10T01:05:05.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.896+0000 7f6bb214c700 1 -- 172.21.15.7:0/4247800505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bac1a6bb0 con 0x7f6bac104090 2024-06-10T01:05:05.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.896+0000 7f6ba97fa700 1 -- 172.21.15.7:0/4247800505 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4005390 con 0x7f6bac104090 2024-06-10T01:05:05.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.896+0000 7f6bb214c700 1 -- 172.21.15.7:0/4247800505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bac1a70f0 con 0x7f6bac104090 2024-06-10T01:05:05.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.897+0000 7f6bb214c700 1 -- 172.21.15.7:0/4247800505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bac069180 con 0x7f6bac104090 2024-06-10T01:05:05.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.899+0000 7f6ba97fa700 1 -- 172.21.15.7:0/4247800505 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f6ba4024020 con 0x7f6bac104090 2024-06-10T01:05:05.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.900+0000 7f6ba97fa700 1 --2- 172.21.15.7:0/4247800505 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6b980a2c20 0x7f6b980a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:05.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.900+0000 7f6bab7fe700 1 --2- 172.21.15.7:0/4247800505 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6b980a2c20 0x7f6b980a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:05.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.900+0000 7f6ba97fa700 1 -- 172.21.15.7:0/4247800505 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f6ba400f020 con 0x7f6bac104090 2024-06-10T01:05:05.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.900+0000 7f6bab7fe700 1 --2- 172.21.15.7:0/4247800505 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6b980a2c20 0x7f6b980a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6bac19fb80 tx=0x7f6b9c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:05.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:05.904+0000 7f6ba97fa700 1 -- 172.21.15.7:0/4247800505 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6ba409d9a0 con 0x7f6bac104090 2024-06-10T01:05:06.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:06.117+0000 7f6bb214c700 1 -- 172.21.15.7:0/4247800505 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f6bac0628d0 con 0x7f6b980a2c20 2024-06-10T01:05:06.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:06.123+0000 7f6ba97fa700 1 -- 172.21.15.7:0/4247800505 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f6bac0628d0 con 0x7f6b980a2c20 2024-06-10T01:05:06.120 INFO:teuthology.orchestra.run.smithi007.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-06-10T01:05:06.120 INFO:teuthology.orchestra.run.smithi007.stdout:mds.all 12/12 75s ago 4m smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi078=b;smithi078=e;smithi078=h;smithi078=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l;count:12 2024-06-10T01:05:06.120 INFO:teuthology.orchestra.run.smithi007.stdout:mgr 3/3 75s ago 9m smithi007=x;smithi078=y;smithi175=z;count:3 2024-06-10T01:05:06.120 INFO:teuthology.orchestra.run.smithi007.stdout:mon 3/3 75s ago 9m smithi007:172.21.15.7=a;smithi078:172.21.15.78=b;smithi175:172.21.15.175=c;count:3 2024-06-10T01:05:06.120 INFO:teuthology.orchestra.run.smithi007.stdout:osd 12 75s ago - 2024-06-10T01:05:06.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:06.125+0000 7f6bb214c700 1 -- 172.21.15.7:0/4247800505 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6b980a2c20 msgr2=0x7f6b980a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:06.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:06.125+0000 7f6bb214c700 1 --2- 172.21.15.7:0/4247800505 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6b980a2c20 0x7f6b980a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6bac19fb80 tx=0x7f6b9c00b410 comp rx=0 tx=0).stop 2024-06-10T01:05:06.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:06.125+0000 7f6bb214c700 1 -- 172.21.15.7:0/4247800505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bac104090 msgr2=0x7f6bac1a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:06.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:06.125+0000 7f6bb214c700 1 --2- 172.21.15.7:0/4247800505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bac104090 0x7f6bac1a6250 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4006070 tx=0x7f6ba4007d10 comp rx=0 tx=0).stop 2024-06-10T01:05:06.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:06.125+0000 7f6bb214c700 1 -- 172.21.15.7:0/4247800505 shutdown_connections 2024-06-10T01:05:06.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:06.125+0000 7f6bb214c700 1 --2- 172.21.15.7:0/4247800505 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6b980a2c20 0x7f6b980a50e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:06.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:06.125+0000 7f6bb214c700 1 --2- 172.21.15.7:0/4247800505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bac104090 0x7f6bac1a6250 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:06.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:06.125+0000 7f6bb214c700 1 --2- 172.21.15.7:0/4247800505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bac1024f0 0x7f6bac19eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:06.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:06.125+0000 7f6bb214c700 1 --2- 172.21.15.7:0/4247800505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bac1036f0 0x7f6bac19f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:06.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:06.125+0000 7f6bb214c700 1 -- 172.21.15.7:0/4247800505 >> 172.21.15.7:0/4247800505 conn(0x7f6bac0fda90 msgr2=0x7f6bac1049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:06.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:06.125+0000 7f6bb214c700 1 -- 172.21.15.7:0/4247800505 shutdown_connections 2024-06-10T01:05:06.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:06.125+0000 7f6bb214c700 1 -- 172.21.15.7:0/4247800505 wait complete. 2024-06-10T01:05:06.752 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch host ls' 2024-06-10T01:05:07.009 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:05:07.404 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:07 smithi007 ceph-mon[32398]: pgmap v404: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:05:07.404 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:07 smithi007 ceph-mon[32398]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:05:07.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:07 smithi175 ceph-mon[36636]: pgmap v404: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:05:07.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:07 smithi175 ceph-mon[36636]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:05:07.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:07 smithi078 ceph-mon[36683]: pgmap v404: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:05:07.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:07 smithi078 ceph-mon[36683]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:05:08.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.691+0000 7f81b54a9700 1 -- 172.21.15.7:0/2594782975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b01024f0 msgr2=0x7f81b0102910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.691+0000 7f81b54a9700 1 --2- 172.21.15.7:0/2594782975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b01024f0 0x7f81b0102910 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f81a00099c0 tx=0x7f81a0009cd0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.691+0000 7f81b54a9700 1 -- 172.21.15.7:0/2594782975 shutdown_connections 2024-06-10T01:05:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.691+0000 7f81b54a9700 1 --2- 172.21.15.7:0/2594782975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81b0104090 0x7f81b010a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.691+0000 7f81b54a9700 1 --2- 172.21.15.7:0/2594782975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b01024f0 0x7f81b0102910 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.691+0000 7f81b54a9700 1 --2- 172.21.15.7:0/2594782975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81b01036f0 0x7f81b0103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.691+0000 7f81b54a9700 1 -- 172.21.15.7:0/2594782975 >> 172.21.15.7:0/2594782975 conn(0x7f81b00fda90 msgr2=0x7f81b00ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.691+0000 7f81b54a9700 1 -- 172.21.15.7:0/2594782975 shutdown_connections 2024-06-10T01:05:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.691+0000 7f81b54a9700 1 -- 172.21.15.7:0/2594782975 wait complete. 2024-06-10T01:05:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.692+0000 7f81b54a9700 1 Processor -- start 2024-06-10T01:05:08.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.692+0000 7f81b54a9700 1 -- start start 2024-06-10T01:05:08.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.693+0000 7f81b54a9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81b01024f0 0x7f81b0198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:08.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.693+0000 7f81b54a9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81b01036f0 0x7f81b0198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:08.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.693+0000 7f81b54a9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b0104090 0x7f81b019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:08.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.693+0000 7f81b54a9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f81b010e0d0 con 0x7f81b0104090 2024-06-10T01:05:08.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.693+0000 7f81b54a9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f81b010df50 con 0x7f81b01024f0 2024-06-10T01:05:08.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.693+0000 7f81b54a9700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f81b010e250 con 0x7f81b01036f0 2024-06-10T01:05:08.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.693+0000 7f81af7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b0104090 0x7f81b019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:08.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.693+0000 7f81af7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b0104090 0x7f81b019fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44796/0 (socket says 172.21.15.7:44796) 2024-06-10T01:05:08.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.693+0000 7f81af7fe700 1 -- 172.21.15.7:0/633526210 learned_addr learned my addr 172.21.15.7:0/633526210 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:08.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.693+0000 7f81aeffd700 1 --2- 172.21.15.7:0/633526210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81b01024f0 0x7f81b0198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:08.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.693+0000 7f81ae7fc700 1 --2- 172.21.15.7:0/633526210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81b01036f0 0x7f81b0198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:08.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.693+0000 7f81af7fe700 1 -- 172.21.15.7:0/633526210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81b01036f0 msgr2=0x7f81b0198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:08.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.693+0000 7f81af7fe700 1 --2- 172.21.15.7:0/633526210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81b01036f0 0x7f81b0198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.693+0000 7f81af7fe700 1 -- 172.21.15.7:0/633526210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81b01024f0 msgr2=0x7f81b0198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:08.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.693+0000 7f81af7fe700 1 --2- 172.21.15.7:0/633526210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81b01024f0 0x7f81b0198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.693+0000 7f81af7fe700 1 -- 172.21.15.7:0/633526210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b01a03d0 con 0x7f81b0104090 2024-06-10T01:05:08.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.693+0000 7f81af7fe700 1 --2- 172.21.15.7:0/633526210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b0104090 0x7f81b019fd30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f81a8007f10 tx=0x7f81a800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:08.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.693+0000 7f81a7fff700 1 -- 172.21.15.7:0/633526210 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a800dd20 con 0x7f81b0104090 2024-06-10T01:05:08.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.694+0000 7f81a7fff700 1 -- 172.21.15.7:0/633526210 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81a8004020 con 0x7f81b0104090 2024-06-10T01:05:08.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.694+0000 7f81b54a9700 1 -- 172.21.15.7:0/633526210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b01a06c0 con 0x7f81b0104090 2024-06-10T01:05:08.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.694+0000 7f81a7fff700 1 -- 172.21.15.7:0/633526210 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8015660 con 0x7f81b0104090 2024-06-10T01:05:08.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.694+0000 7f81b54a9700 1 -- 172.21.15.7:0/633526210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b01a0bb0 con 0x7f81b0104090 2024-06-10T01:05:08.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.695+0000 7f81b54a9700 1 -- 172.21.15.7:0/633526210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81b0069180 con 0x7f81b0104090 2024-06-10T01:05:08.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.696+0000 7f81a7fff700 1 -- 172.21.15.7:0/633526210 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f81a80041d0 con 0x7f81b0104090 2024-06-10T01:05:08.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.696+0000 7f81a7fff700 1 --2- 172.21.15.7:0/633526210 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f819c0a2c20 0x7f819c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:08.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.697+0000 7f81aeffd700 1 --2- 172.21.15.7:0/633526210 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f819c0a2c20 0x7f819c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:08.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.697+0000 7f81a7fff700 1 -- 172.21.15.7:0/633526210 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f81a80cf5c0 con 0x7f81b0104090 2024-06-10T01:05:08.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.697+0000 7f81aeffd700 1 --2- 172.21.15.7:0/633526210 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f819c0a2c20 0x7f819c0a50e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f81a000b580 tx=0x7f81a0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:08.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.700+0000 7f81a7fff700 1 -- 172.21.15.7:0/633526210 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f81a8010070 con 0x7f81b0104090 2024-06-10T01:05:08.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.913+0000 7f81b54a9700 1 -- 172.21.15.7:0/633526210 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f81b0062990 con 0x7f819c0a2c20 2024-06-10T01:05:08.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.914+0000 7f81a7fff700 1 -- 172.21.15.7:0/633526210 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f81b0062990 con 0x7f819c0a2c20 2024-06-10T01:05:08.911 INFO:teuthology.orchestra.run.smithi007.stdout:HOST ADDR LABELS STATUS 2024-06-10T01:05:08.911 INFO:teuthology.orchestra.run.smithi007.stdout:smithi007 172.21.15.7 2024-06-10T01:05:08.911 INFO:teuthology.orchestra.run.smithi007.stdout:smithi078 172.21.15.78 2024-06-10T01:05:08.911 INFO:teuthology.orchestra.run.smithi007.stdout:smithi175 172.21.15.175 2024-06-10T01:05:08.911 INFO:teuthology.orchestra.run.smithi007.stdout:3 hosts in cluster 2024-06-10T01:05:08.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.916+0000 7f81b54a9700 1 -- 172.21.15.7:0/633526210 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f819c0a2c20 msgr2=0x7f819c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:08.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.916+0000 7f81b54a9700 1 --2- 172.21.15.7:0/633526210 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f819c0a2c20 0x7f819c0a50e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f81a000b580 tx=0x7f81a0005fb0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.916+0000 7f81b54a9700 1 -- 172.21.15.7:0/633526210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b0104090 msgr2=0x7f81b019fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:08.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.916+0000 7f81b54a9700 1 --2- 172.21.15.7:0/633526210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b0104090 0x7f81b019fd30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f81a8007f10 tx=0x7f81a800d3b0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.916+0000 7f81b54a9700 1 -- 172.21.15.7:0/633526210 shutdown_connections 2024-06-10T01:05:08.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.916+0000 7f81b54a9700 1 --2- 172.21.15.7:0/633526210 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f819c0a2c20 0x7f819c0a50e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.916+0000 7f81b54a9700 1 --2- 172.21.15.7:0/633526210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b0104090 0x7f81b019fd30 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.916+0000 7f81b54a9700 1 --2- 172.21.15.7:0/633526210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81b01024f0 0x7f81b0198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.916+0000 7f81b54a9700 1 --2- 172.21.15.7:0/633526210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81b01036f0 0x7f81b0198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.916+0000 7f81b54a9700 1 -- 172.21.15.7:0/633526210 >> 172.21.15.7:0/633526210 conn(0x7f81b00fda90 msgr2=0x7f81b0106b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:08.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.917+0000 7f81b54a9700 1 -- 172.21.15.7:0/633526210 shutdown_connections 2024-06-10T01:05:08.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:08.917+0000 7f81b54a9700 1 -- 172.21.15.7:0/633526210 wait complete. 2024-06-10T01:05:09.234 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:09 smithi007 ceph-mon[32398]: pgmap v405: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:05:09.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:09 smithi175 ceph-mon[36636]: pgmap v405: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:05:09.540 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch device ls' 2024-06-10T01:05:09.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:09 smithi078 ceph-mon[36683]: pgmap v405: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:05:09.793 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:05:10.256 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:10 smithi007 ceph-mon[32398]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:05:10.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:10 smithi175 ceph-mon[36636]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:05:10.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:10 smithi078 ceph-mon[36683]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:05:11.240 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:11 smithi007 ceph-mon[32398]: pgmap v406: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:05:11.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:11 smithi175 ceph-mon[36636]: pgmap v406: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:05:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.479+0000 7f4e83591700 1 -- 172.21.15.7:0/3366334905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e7c102f70 msgr2=0x7f4e7c105360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.479+0000 7f4e83591700 1 --2- 172.21.15.7:0/3366334905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e7c102f70 0x7f4e7c105360 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4e78009990 tx=0x7f4e78009ca0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.480+0000 7f4e83591700 1 -- 172.21.15.7:0/3366334905 shutdown_connections 2024-06-10T01:05:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.480+0000 7f4e83591700 1 --2- 172.21.15.7:0/3366334905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e7c1081d0 0x7f4e7c10a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.480+0000 7f4e83591700 1 --2- 172.21.15.7:0/3366334905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e7c102f70 0x7f4e7c105360 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.480+0000 7f4e83591700 1 --2- 172.21.15.7:0/3366334905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e7c1058a0 0x7f4e7c107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.480+0000 7f4e83591700 1 -- 172.21.15.7:0/3366334905 >> 172.21.15.7:0/3366334905 conn(0x7f4e7c0fc960 msgr2=0x7f4e7c0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.480+0000 7f4e83591700 1 -- 172.21.15.7:0/3366334905 shutdown_connections 2024-06-10T01:05:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.480+0000 7f4e83591700 1 -- 172.21.15.7:0/3366334905 wait complete. 2024-06-10T01:05:11.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.480+0000 7f4e83591700 1 Processor -- start 2024-06-10T01:05:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.480+0000 7f4e83591700 1 -- start start 2024-06-10T01:05:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.480+0000 7f4e83591700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e7c102f70 0x7f4e7c198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.481+0000 7f4e83591700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e7c1058a0 0x7f4e7c198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.481+0000 7f4e83591700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e7c1081d0 0x7f4e7c19fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.481+0000 7f4e83591700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c10e140 con 0x7f4e7c1058a0 2024-06-10T01:05:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.481+0000 7f4e83591700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c10dfc0 con 0x7f4e7c102f70 2024-06-10T01:05:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.481+0000 7f4e83591700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c10e2c0 con 0x7f4e7c1081d0 2024-06-10T01:05:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.481+0000 7f4e80b2c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e7c1058a0 0x7f4e7c198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.481+0000 7f4e80b2c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e7c1058a0 0x7f4e7c198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44804/0 (socket says 172.21.15.7:44804) 2024-06-10T01:05:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.481+0000 7f4e80b2c700 1 -- 172.21.15.7:0/317398668 learned_addr learned my addr 172.21.15.7:0/317398668 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.481+0000 7f4e81b2e700 1 --2- 172.21.15.7:0/317398668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e7c1081d0 0x7f4e7c19fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.481+0000 7f4e8132d700 1 --2- 172.21.15.7:0/317398668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e7c102f70 0x7f4e7c198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.481+0000 7f4e80b2c700 1 -- 172.21.15.7:0/317398668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e7c1081d0 msgr2=0x7f4e7c19fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.481+0000 7f4e80b2c700 1 --2- 172.21.15.7:0/317398668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e7c1081d0 0x7f4e7c19fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.481+0000 7f4e80b2c700 1 -- 172.21.15.7:0/317398668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e7c102f70 msgr2=0x7f4e7c198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.481+0000 7f4e80b2c700 1 --2- 172.21.15.7:0/317398668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e7c102f70 0x7f4e7c198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.481+0000 7f4e80b2c700 1 -- 172.21.15.7:0/317398668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7c1a03d0 con 0x7f4e7c1058a0 2024-06-10T01:05:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.481+0000 7f4e80b2c700 1 --2- 172.21.15.7:0/317398668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e7c1058a0 0x7f4e7c198cb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00d9b0 tx=0x7f4e6c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.481+0000 7f4e727fc700 1 -- 172.21.15.7:0/317398668 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c017070 con 0x7f4e7c1058a0 2024-06-10T01:05:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.481+0000 7f4e727fc700 1 -- 172.21.15.7:0/317398668 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e6c005020 con 0x7f4e7c1058a0 2024-06-10T01:05:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.481+0000 7f4e83591700 1 -- 172.21.15.7:0/317398668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e7c1a06c0 con 0x7f4e7c1058a0 2024-06-10T01:05:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.481+0000 7f4e727fc700 1 -- 172.21.15.7:0/317398668 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c009a90 con 0x7f4e7c1058a0 2024-06-10T01:05:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.481+0000 7f4e83591700 1 -- 172.21.15.7:0/317398668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e7c1a0c00 con 0x7f4e7c1058a0 2024-06-10T01:05:11.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.482+0000 7f4e83591700 1 -- 172.21.15.7:0/317398668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e7c069180 con 0x7f4e7c1058a0 2024-06-10T01:05:11.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.484+0000 7f4e727fc700 1 -- 172.21.15.7:0/317398668 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f4e6c0051d0 con 0x7f4e7c1058a0 2024-06-10T01:05:11.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.484+0000 7f4e727fc700 1 --2- 172.21.15.7:0/317398668 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4e680a2ca0 0x7f4e680a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:11.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.484+0000 7f4e727fc700 1 -- 172.21.15.7:0/317398668 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f4e6c0cf5b0 con 0x7f4e7c1058a0 2024-06-10T01:05:11.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.484+0000 7f4e8132d700 1 --2- 172.21.15.7:0/317398668 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4e680a2ca0 0x7f4e680a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.485+0000 7f4e8132d700 1 --2- 172.21.15.7:0/317398668 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4e680a2ca0 0x7f4e680a5160 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800b5c0 tx=0x7f4e78005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:11.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.488+0000 7f4e727fc700 1 -- 172.21.15.7:0/317398668 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4e6c020020 con 0x7f4e7c1058a0 2024-06-10T01:05:11.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:11 smithi078 ceph-mon[36683]: pgmap v406: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:05:11.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.707+0000 7f4e83591700 1 -- 172.21.15.7:0/317398668 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f4e7c0628d0 con 0x7f4e680a2ca0 2024-06-10T01:05:11.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.708+0000 7f4e727fc700 1 -- 172.21.15.7:0/317398668 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f4e7c0628d0 con 0x7f4e680a2ca0 2024-06-10T01:05:11.705 INFO:teuthology.orchestra.run.smithi007.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-06-10T01:05:11.705 INFO:teuthology.orchestra.run.smithi007.stdout:smithi007 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300GE480DGN 447G No 5m ago Has a FileSystem, LVM detected 2024-06-10T01:05:11.705 INFO:teuthology.orchestra.run.smithi007.stdout:smithi078 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400QR400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-06-10T01:05:11.705 INFO:teuthology.orchestra.run.smithi007.stdout:smithi175 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300M0400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-06-10T01:05:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.710+0000 7f4e83591700 1 -- 172.21.15.7:0/317398668 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4e680a2ca0 msgr2=0x7f4e680a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.710+0000 7f4e83591700 1 --2- 172.21.15.7:0/317398668 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4e680a2ca0 0x7f4e680a5160 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800b5c0 tx=0x7f4e78005fb0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.710+0000 7f4e83591700 1 -- 172.21.15.7:0/317398668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e7c1058a0 msgr2=0x7f4e7c198cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.710+0000 7f4e83591700 1 --2- 172.21.15.7:0/317398668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e7c1058a0 0x7f4e7c198cb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00d9b0 tx=0x7f4e6c00dcc0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.711+0000 7f4e83591700 1 -- 172.21.15.7:0/317398668 shutdown_connections 2024-06-10T01:05:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.711+0000 7f4e83591700 1 --2- 172.21.15.7:0/317398668 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4e680a2ca0 0x7f4e680a5160 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.711+0000 7f4e83591700 1 --2- 172.21.15.7:0/317398668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e7c1081d0 0x7f4e7c19fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.711+0000 7f4e83591700 1 --2- 172.21.15.7:0/317398668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e7c102f70 0x7f4e7c198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.711+0000 7f4e83591700 1 --2- 172.21.15.7:0/317398668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e7c1058a0 0x7f4e7c198cb0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.711+0000 7f4e83591700 1 -- 172.21.15.7:0/317398668 >> 172.21.15.7:0/317398668 conn(0x7f4e7c0fc960 msgr2=0x7f4e7c1038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.711+0000 7f4e83591700 1 -- 172.21.15.7:0/317398668 shutdown_connections 2024-06-10T01:05:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:11.711+0000 7f4e83591700 1 -- 172.21.15.7:0/317398668 wait complete. 2024-06-10T01:05:12.330 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-10T01:05:12.339 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-06-10T01:05:12.339 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs dump' 2024-06-10T01:05:12.588 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:05:13.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:13 smithi007 ceph-mon[32398]: pgmap v407: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:05:13.386 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:13 smithi007 ceph-mon[32398]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:05:13.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:13 smithi175 ceph-mon[36636]: pgmap v407: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:05:13.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:13 smithi175 ceph-mon[36636]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:05:13.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:13 smithi078 ceph-mon[36683]: pgmap v407: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:05:13.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:13 smithi078 ceph-mon[36683]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T01:05:14.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.222+0000 7fad6c306700 1 -- 172.21.15.7:0/2059946688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad641036f0 msgr2=0x7fad64103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:14.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.222+0000 7fad6c306700 1 --2- 172.21.15.7:0/2059946688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad641036f0 0x7fad64103b50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fad540099c0 tx=0x7fad54009cd0 comp rx=0 tx=0).stop 2024-06-10T01:05:14.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.222+0000 7fad6c306700 1 -- 172.21.15.7:0/2059946688 shutdown_connections 2024-06-10T01:05:14.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.222+0000 7fad6c306700 1 --2- 172.21.15.7:0/2059946688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad64104090 0x7fad6410a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:14.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.222+0000 7fad6c306700 1 --2- 172.21.15.7:0/2059946688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad641024f0 0x7fad64102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:14.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.222+0000 7fad6c306700 1 --2- 172.21.15.7:0/2059946688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad641036f0 0x7fad64103b50 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:14.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.222+0000 7fad6c306700 1 -- 172.21.15.7:0/2059946688 >> 172.21.15.7:0/2059946688 conn(0x7fad640fda90 msgr2=0x7fad640ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:14.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.222+0000 7fad6c306700 1 -- 172.21.15.7:0/2059946688 shutdown_connections 2024-06-10T01:05:14.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.222+0000 7fad6c306700 1 -- 172.21.15.7:0/2059946688 wait complete. 2024-06-10T01:05:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.223+0000 7fad6c306700 1 Processor -- start 2024-06-10T01:05:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.223+0000 7fad6c306700 1 -- start start 2024-06-10T01:05:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.223+0000 7fad6c306700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad641024f0 0x7fad64198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.223+0000 7fad6c306700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad641036f0 0x7fad64198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.223+0000 7fad6c306700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad64104090 0x7fad6419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.223+0000 7fad6c306700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fad6410e0d0 con 0x7fad64104090 2024-06-10T01:05:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.223+0000 7fad6c306700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fad6410df50 con 0x7fad641036f0 2024-06-10T01:05:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.223+0000 7fad6c306700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fad6410e250 con 0x7fad641024f0 2024-06-10T01:05:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.223+0000 7fad6a8a3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad64104090 0x7fad6419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.223+0000 7fad6a8a3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad64104090 0x7fad6419fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37210/0 (socket says 172.21.15.7:37210) 2024-06-10T01:05:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.223+0000 7fad6a8a3700 1 -- 172.21.15.7:0/3401197416 learned_addr learned my addr 172.21.15.7:0/3401197416 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.223+0000 7fad698a1700 1 --2- 172.21.15.7:0/3401197416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad641036f0 0x7fad64198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:14.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.223+0000 7fad6a0a2700 1 --2- 172.21.15.7:0/3401197416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad641024f0 0x7fad64198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:14.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.223+0000 7fad6a8a3700 1 -- 172.21.15.7:0/3401197416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad641024f0 msgr2=0x7fad64198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:14.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.223+0000 7fad6a8a3700 1 --2- 172.21.15.7:0/3401197416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad641024f0 0x7fad64198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:14.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.223+0000 7fad6a8a3700 1 -- 172.21.15.7:0/3401197416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad641036f0 msgr2=0x7fad64198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:14.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.223+0000 7fad6a8a3700 1 --2- 172.21.15.7:0/3401197416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad641036f0 0x7fad64198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:14.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.223+0000 7fad6a8a3700 1 -- 172.21.15.7:0/3401197416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad641a0370 con 0x7fad64104090 2024-06-10T01:05:14.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.224+0000 7fad6a8a3700 1 --2- 172.21.15.7:0/3401197416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad64104090 0x7fad6419fcd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00ef60 tx=0x7fad5c00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:14.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.224+0000 7fad5b7fe700 1 -- 172.21.15.7:0/3401197416 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c0092e0 con 0x7fad64104090 2024-06-10T01:05:14.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.224+0000 7fad5b7fe700 1 -- 172.21.15.7:0/3401197416 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad5c004bb0 con 0x7fad64104090 2024-06-10T01:05:14.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.224+0000 7fad6c306700 1 -- 172.21.15.7:0/3401197416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad640746c0 con 0x7fad64104090 2024-06-10T01:05:14.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.224+0000 7fad5b7fe700 1 -- 172.21.15.7:0/3401197416 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c004280 con 0x7fad64104090 2024-06-10T01:05:14.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.224+0000 7fad6c306700 1 -- 172.21.15.7:0/3401197416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad64074c00 con 0x7fad64104090 2024-06-10T01:05:14.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.225+0000 7fad6c306700 1 -- 172.21.15.7:0/3401197416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad64069180 con 0x7fad64104090 2024-06-10T01:05:14.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.226+0000 7fad5b7fe700 1 -- 172.21.15.7:0/3401197416 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fad5c01b030 con 0x7fad64104090 2024-06-10T01:05:14.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.227+0000 7fad5b7fe700 1 --2- 172.21.15.7:0/3401197416 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fad500a2ca0 0x7fad500a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:14.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.227+0000 7fad6a0a2700 1 --2- 172.21.15.7:0/3401197416 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fad500a2ca0 0x7fad500a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:14.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.227+0000 7fad5b7fe700 1 -- 172.21.15.7:0/3401197416 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fad5c0ceda0 con 0x7fad64104090 2024-06-10T01:05:14.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.227+0000 7fad6a0a2700 1 --2- 172.21.15.7:0/3401197416 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fad500a2ca0 0x7fad500a5160 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fad64103550 tx=0x7fad6000f820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:14.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.231+0000 7fad5b7fe700 1 -- 172.21.15.7:0/3401197416 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fad5c022020 con 0x7fad64104090 2024-06-10T01:05:14.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.488+0000 7fad6c306700 1 -- 172.21.15.7:0/3401197416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fad64075fe0 con 0x7fad64104090 2024-06-10T01:05:14.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.489+0000 7fad5b7fe700 1 -- 172.21.15.7:0/3401197416 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3040 (secure 0 0 0) 0x7fad5c010040 con 0x7fad64104090 2024-06-10T01:05:14.486 INFO:teuthology.orchestra.run.smithi007.stdout:e26 2024-06-10T01:05:14.486 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-10T01:05:14.486 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-10T01:05:14.486 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-06-10T01:05:14.486 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:05:14.486 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-06-10T01:05:14.486 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-06-10T01:05:14.486 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 26 2024-06-10T01:05:14.486 INFO:teuthology.orchestra.run.smithi007.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-06-10T01:05:14.486 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-06-10T01:01:49.029348+0000 2024-06-10T01:05:14.487 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-06-10T01:02:58.523980+0000 2024-06-10T01:05:14.487 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-06-10T01:05:14.487 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-06-10T01:05:14.487 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 300 2024-06-10T01:05:14.487 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-06-10T01:05:14.487 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-06-10T01:05:14.487 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-06-10T01:05:14.487 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-06-10T01:05:14.487 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-06-10T01:05:14.487 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 0 2024-06-10T01:05:14.487 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-10T01:05:14.487 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 3 2024-06-10T01:05:14.487 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1,2 2024-06-10T01:05:14.487 INFO:teuthology.orchestra.run.smithi007.stdout:up {0=24451,1=24436,2=24434} 2024-06-10T01:05:14.488 INFO:teuthology.orchestra.run.smithi007.stdout:failed 2024-06-10T01:05:14.488 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-06-10T01:05:14.488 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-06-10T01:05:14.488 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3,4] 2024-06-10T01:05:14.488 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-06-10T01:05:14.488 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data disabled 2024-06-10T01:05:14.488 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-06-10T01:05:14.488 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-06-10T01:05:14.488 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 0 2024-06-10T01:05:14.488 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 24451 members: 24436,24451,24434 2024-06-10T01:05:14.488 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.i{0:24451} state up:active seq 8 addr [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T01:05:14.488 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.f{0:24424} state up:standby-replay seq 1 addr [v2:172.21.15.175:6834/2784910673,v1:172.21.15.175:6835/2784910673] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T01:05:14.489 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c{1:24436} state up:active seq 26 addr [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T01:05:14.489 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.h{1:24413} state up:standby-replay seq 1 addr [v2:172.21.15.78:6836/3608622984,v1:172.21.15.78:6837/3608622984] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T01:05:14.489 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b{2:24434} state up:active seq 21 addr [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T01:05:14.489 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.l{2:24403} state up:standby-replay seq 1 addr [v2:172.21.15.175:6832/1281305765,v1:172.21.15.175:6833/1281305765] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T01:05:14.489 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:05:14.489 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:05:14.489 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-06-10T01:05:14.489 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:05:14.489 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.j{-1:14532} state up:standby seq 1 addr [v2:172.21.15.7:6834/2172378766,v1:172.21.15.7:6835/2172378766] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T01:05:14.489 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d{-1:14538} state up:standby seq 1 addr [v2:172.21.15.7:6836/1734773915,v1:172.21.15.7:6837/1734773915] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T01:05:14.489 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.g{-1:14583} state up:standby seq 1 addr [v2:172.21.15.7:6838/159652731,v1:172.21.15.7:6839/159652731] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T01:05:14.489 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a{-1:14595} state up:standby seq 1 addr [v2:172.21.15.7:6840/1788930795,v1:172.21.15.7:6841/1788930795] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T01:05:14.489 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.k{-1:24383} state up:standby seq 1 addr [v2:172.21.15.78:6832/1425832513,v1:172.21.15.78:6833/1425832513] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T01:05:14.489 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.e{-1:24401} state up:standby seq 1 addr [v2:172.21.15.78:6834/938897057,v1:172.21.15.78:6835/938897057] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T01:05:14.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.491+0000 7fad6c306700 1 -- 172.21.15.7:0/3401197416 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fad500a2ca0 msgr2=0x7fad500a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:14.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.491+0000 7fad6c306700 1 --2- 172.21.15.7:0/3401197416 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fad500a2ca0 0x7fad500a5160 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fad64103550 tx=0x7fad6000f820 comp rx=0 tx=0).stop 2024-06-10T01:05:14.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.491+0000 7fad6c306700 1 -- 172.21.15.7:0/3401197416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad64104090 msgr2=0x7fad6419fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:14.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.491+0000 7fad6c306700 1 --2- 172.21.15.7:0/3401197416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad64104090 0x7fad6419fcd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00ef60 tx=0x7fad5c00ef90 comp rx=0 tx=0).stop 2024-06-10T01:05:14.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.491+0000 7fad6c306700 1 -- 172.21.15.7:0/3401197416 shutdown_connections 2024-06-10T01:05:14.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.491+0000 7fad6c306700 1 --2- 172.21.15.7:0/3401197416 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fad500a2ca0 0x7fad500a5160 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:14.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.491+0000 7fad6c306700 1 --2- 172.21.15.7:0/3401197416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad64104090 0x7fad6419fcd0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:14.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.491+0000 7fad6c306700 1 --2- 172.21.15.7:0/3401197416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad641024f0 0x7fad64198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:14.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.491+0000 7fad6c306700 1 --2- 172.21.15.7:0/3401197416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad641036f0 0x7fad64198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:14.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.491+0000 7fad6c306700 1 -- 172.21.15.7:0/3401197416 >> 172.21.15.7:0/3401197416 conn(0x7fad640fda90 msgr2=0x7fad64106a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:14.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.491+0000 7fad6c306700 1 -- 172.21.15.7:0/3401197416 shutdown_connections 2024-06-10T01:05:14.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:14.491+0000 7fad6c306700 1 -- 172.21.15.7:0/3401197416 wait complete. 2024-06-10T01:05:14.491 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-06-10T01:05:15.151 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- bash -c 'ceph osd dump' 2024-06-10T01:05:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:05:15.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:15 smithi175 ceph-mon[36636]: pgmap v408: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:05:15.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:15 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3401197416' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-10T01:05:15.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:15 smithi007 ceph-mon[32398]: pgmap v408: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:05:15.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:15 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3401197416' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-10T01:05:15.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:15 smithi078 ceph-mon[36683]: pgmap v408: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:05:15.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:15 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3401197416' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-10T01:05:17.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.098+0000 7f73fab8d700 1 -- 172.21.15.7:0/1187460228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73f4102f60 msgr2=0x7f73f4109810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:17.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.098+0000 7f73fab8d700 1 --2- 172.21.15.7:0/1187460228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73f4102f60 0x7f73f4109810 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f73ec00b0d0 tx=0x7f73ec00b3e0 comp rx=0 tx=0).stop 2024-06-10T01:05:17.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.098+0000 7f73fab8d700 1 -- 172.21.15.7:0/1187460228 shutdown_connections 2024-06-10T01:05:17.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.098+0000 7f73fab8d700 1 --2- 172.21.15.7:0/1187460228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73f4102f60 0x7f73f4109810 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:17.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.098+0000 7f73fab8d700 1 --2- 172.21.15.7:0/1187460228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73f41013c0 0x7f73f41017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:17.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.098+0000 7f73fab8d700 1 --2- 172.21.15.7:0/1187460228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73f41025c0 0x7f73f4102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:17.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.098+0000 7f73fab8d700 1 -- 172.21.15.7:0/1187460228 >> 172.21.15.7:0/1187460228 conn(0x7f73f40fc960 msgr2=0x7f73f40feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:17.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.098+0000 7f73fab8d700 1 -- 172.21.15.7:0/1187460228 shutdown_connections 2024-06-10T01:05:17.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.098+0000 7f73fab8d700 1 -- 172.21.15.7:0/1187460228 wait complete. 2024-06-10T01:05:17.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.099+0000 7f73fab8d700 1 Processor -- start 2024-06-10T01:05:17.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.099+0000 7f73fab8d700 1 -- start start 2024-06-10T01:05:17.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.099+0000 7f73fab8d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73f41013c0 0x7f73f406ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:17.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.099+0000 7f73fab8d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73f41025c0 0x7f73f419d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:17.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.099+0000 7f73fab8d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73f4102f60 0x7f73f419f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:17.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.099+0000 7f73fab8d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f73f410cf90 con 0x7f73f4102f60 2024-06-10T01:05:17.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.099+0000 7f73fab8d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f73f410ce10 con 0x7f73f41025c0 2024-06-10T01:05:17.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.099+0000 7f73fab8d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f73f410d110 con 0x7f73f41013c0 2024-06-10T01:05:17.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.099+0000 7f73f912a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73f4102f60 0x7f73f419f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:17.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.099+0000 7f73f912a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73f4102f60 0x7f73f419f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37220/0 (socket says 172.21.15.7:37220) 2024-06-10T01:05:17.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.099+0000 7f73f912a700 1 -- 172.21.15.7:0/3994467931 learned_addr learned my addr 172.21.15.7:0/3994467931 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:17.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.099+0000 7f73ebfff700 1 --2- 172.21.15.7:0/3994467931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73f41025c0 0x7f73f419d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:17.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.099+0000 7f73f8929700 1 --2- 172.21.15.7:0/3994467931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73f41013c0 0x7f73f406ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:17.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.100+0000 7f73f912a700 1 -- 172.21.15.7:0/3994467931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73f41013c0 msgr2=0x7f73f406ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:17.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.100+0000 7f73f912a700 1 --2- 172.21.15.7:0/3994467931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73f41013c0 0x7f73f406ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:17.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.100+0000 7f73f912a700 1 -- 172.21.15.7:0/3994467931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73f41025c0 msgr2=0x7f73f419d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:17.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.100+0000 7f73f912a700 1 --2- 172.21.15.7:0/3994467931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73f41025c0 0x7f73f419d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:17.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.100+0000 7f73f912a700 1 -- 172.21.15.7:0/3994467931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f41a0280 con 0x7f73f4102f60 2024-06-10T01:05:17.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.100+0000 7f73ebfff700 1 --2- 172.21.15.7:0/3994467931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73f41025c0 0x7f73f419d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:05:17.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.100+0000 7f73f912a700 1 --2- 172.21.15.7:0/3994467931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73f4102f60 0x7f73f419f9b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f73ec006070 tx=0x7f73ec007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:17.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.100+0000 7f73e9ffb700 1 -- 172.21.15.7:0/3994467931 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ec021020 con 0x7f73f4102f60 2024-06-10T01:05:17.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.100+0000 7f73e9ffb700 1 -- 172.21.15.7:0/3994467931 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73ec01d950 con 0x7f73f4102f60 2024-06-10T01:05:17.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.100+0000 7f73e9ffb700 1 -- 172.21.15.7:0/3994467931 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ec005510 con 0x7f73f4102f60 2024-06-10T01:05:17.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.100+0000 7f73fab8d700 1 -- 172.21.15.7:0/3994467931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f41a0510 con 0x7f73f4102f60 2024-06-10T01:05:17.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.100+0000 7f73fab8d700 1 -- 172.21.15.7:0/3994467931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f41a0a00 con 0x7f73f4102f60 2024-06-10T01:05:17.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.101+0000 7f73fab8d700 1 -- 172.21.15.7:0/3994467931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73f4069180 con 0x7f73f4102f60 2024-06-10T01:05:17.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.102+0000 7f73e9ffb700 1 -- 172.21.15.7:0/3994467931 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f73ec024020 con 0x7f73f4102f60 2024-06-10T01:05:17.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.103+0000 7f73e9ffb700 1 --2- 172.21.15.7:0/3994467931 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f73dc0a2ca0 0x7f73dc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:17.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.103+0000 7f73f8929700 1 --2- 172.21.15.7:0/3994467931 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f73dc0a2ca0 0x7f73dc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:17.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.103+0000 7f73e9ffb700 1 -- 172.21.15.7:0/3994467931 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f73ec00f020 con 0x7f73f4102f60 2024-06-10T01:05:17.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.104+0000 7f73f8929700 1 --2- 172.21.15.7:0/3994467931 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f73dc0a2ca0 0x7f73dc0a5160 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f73f0005fd0 tx=0x7f73f0005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:17.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.107+0000 7f73e9ffb700 1 -- 172.21.15.7:0/3994467931 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f73ec09d9f0 con 0x7f73f4102f60 2024-06-10T01:05:17.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.317+0000 7f73fab8d700 1 -- 172.21.15.7:0/3994467931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f73f40630a0 con 0x7f73f4102f60 2024-06-10T01:05:17.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.317+0000 7f73e9ffb700 1 -- 172.21.15.7:0/3994467931 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v99) ==== 56+0+5591 (secure 0 0 0) 0x7f73ec09d2f0 con 0x7f73f4102f60 2024-06-10T01:05:17.314 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 99 2024-06-10T01:05:17.314 INFO:teuthology.orchestra.run.smithi007.stdout:fsid bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T01:05:17.314 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-06-10T00:53:31.966085+0000 2024-06-10T01:05:17.315 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-06-10T01:03:38.227703+0000 2024-06-10T01:05:17.315 INFO:teuthology.orchestra.run.smithi007.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-06-10T01:05:17.315 INFO:teuthology.orchestra.run.smithi007.stdout:crush_version 27 2024-06-10T01:05:17.315 INFO:teuthology.orchestra.run.smithi007.stdout:full_ratio 0.95 2024-06-10T01:05:17.315 INFO:teuthology.orchestra.run.smithi007.stdout:backfillfull_ratio 0.9 2024-06-10T01:05:17.315 INFO:teuthology.orchestra.run.smithi007.stdout:nearfull_ratio 0.85 2024-06-10T01:05:17.315 INFO:teuthology.orchestra.run.smithi007.stdout:require_min_compat_client luminous 2024-06-10T01:05:17.315 INFO:teuthology.orchestra.run.smithi007.stdout:min_compat_client luminous 2024-06-10T01:05:17.315 INFO:teuthology.orchestra.run.smithi007.stdout:require_osd_release squid 2024-06-10T01:05:17.315 INFO:teuthology.orchestra.run.smithi007.stdout:stretch_mode_enabled false 2024-06-10T01:05:17.315 INFO:teuthology.orchestra.run.smithi007.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-06-10T01:05:17.315 INFO:teuthology.orchestra.run.smithi007.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-06-10T01:05:17.315 INFO:teuthology.orchestra.run.smithi007.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-06-10T01:05:17.316 INFO:teuthology.orchestra.run.smithi007.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 95 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-06-10T01:05:17.316 INFO:teuthology.orchestra.run.smithi007.stdout:max_osd 12 2024-06-10T01:05:17.316 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 up in weight 1 up_from 8 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6802/404485726,v1:172.21.15.7:6803/404485726] [v2:172.21.15.7:6804/404485726,v1:172.21.15.7:6805/404485726] exists,up f7c24c9f-b7ab-4c04-b2b4-120df9751667 2024-06-10T01:05:17.316 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 up in weight 1 up_from 13 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6800/1856804448,v1:172.21.15.78:6801/1856804448] [v2:172.21.15.78:6802/1856804448,v1:172.21.15.78:6803/1856804448] exists,up 9bf70a6b-f31c-4077-b5af-229f17bc2845 2024-06-10T01:05:17.316 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 up in weight 1 up_from 18 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6800/918257455,v1:172.21.15.175:6801/918257455] [v2:172.21.15.175:6802/918257455,v1:172.21.15.175:6803/918257455] exists,up 62923c97-caf9-4f7e-9c14-9c9918843c78 2024-06-10T01:05:17.316 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 up in weight 1 up_from 25 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6810/1797404966,v1:172.21.15.7:6811/1797404966] [v2:172.21.15.7:6812/1797404966,v1:172.21.15.7:6813/1797404966] exists,up 65242098-e748-48ab-8f0e-bb25b5c62742 2024-06-10T01:05:17.316 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 up in weight 1 up_from 33 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6808/3767608357,v1:172.21.15.78:6809/3767608357] [v2:172.21.15.78:6810/3767608357,v1:172.21.15.78:6811/3767608357] exists,up a7990305-de9e-400e-a432-d7854f73f339 2024-06-10T01:05:17.316 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 up in weight 1 up_from 38 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6808/2157664731,v1:172.21.15.175:6809/2157664731] [v2:172.21.15.175:6810/2157664731,v1:172.21.15.175:6811/2157664731] exists,up bad9a7b3-da5a-48d0-905d-e6a3190317fe 2024-06-10T01:05:17.316 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6 up in weight 1 up_from 46 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6818/838731756,v1:172.21.15.7:6819/838731756] [v2:172.21.15.7:6820/838731756,v1:172.21.15.7:6821/838731756] exists,up 6c687653-157b-4b96-867b-a2eaa7d8ed8a 2024-06-10T01:05:17.316 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7 up in weight 1 up_from 51 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6816/3511921773,v1:172.21.15.78:6817/3511921773] [v2:172.21.15.78:6818/3511921773,v1:172.21.15.78:6819/3511921773] exists,up db40ddf1-9fd9-4859-b8c4-75f79582ab91 2024-06-10T01:05:17.316 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8 up in weight 1 up_from 59 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6816/1546325211,v1:172.21.15.175:6817/1546325211] [v2:172.21.15.175:6818/1546325211,v1:172.21.15.175:6819/1546325211] exists,up 0f01cbd9-b193-4c70-a33c-4984167bf422 2024-06-10T01:05:17.316 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9 up in weight 1 up_from 67 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6826/1844550813,v1:172.21.15.7:6827/1844550813] [v2:172.21.15.7:6828/1844550813,v1:172.21.15.7:6829/1844550813] exists,up e5480998-f7fb-49cd-9aed-52405c88d830 2024-06-10T01:05:17.317 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10 up in weight 1 up_from 72 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6824/2290824785,v1:172.21.15.78:6825/2290824785] [v2:172.21.15.78:6826/2290824785,v1:172.21.15.78:6827/2290824785] exists,up caa82f33-a71c-4907-bf3b-30a7fdfeb71d 2024-06-10T01:05:17.317 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11 up in weight 1 up_from 80 up_thru 98 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6824/1959979550,v1:172.21.15.175:6825/1959979550] [v2:172.21.15.175:6826/1959979550,v1:172.21.15.175:6827/1959979550] exists,up 608cda77-77da-4da2-a7e2-76a773be27c8 2024-06-10T01:05:17.317 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 2.a [2,11] 2024-06-10T01:05:17.317 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 2.10 [2,4,8,11] 2024-06-10T01:05:17.317 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.e [4,8] 2024-06-10T01:05:17.317 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.1d [4,6] 2024-06-10T01:05:17.317 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.24 [4,9] 2024-06-10T01:05:17.317 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.29 [3,9] 2024-06-10T01:05:17.317 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.33 [5,9] 2024-06-10T01:05:17.317 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 4.f [9,8] 2024-06-10T01:05:17.317 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 4.13 [9,8] 2024-06-10T01:05:17.317 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 4.1b [5,8] 2024-06-10T01:05:17.317 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 4.1c [9,10] 2024-06-10T01:05:17.317 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/1833397470 expires 2024-06-11T00:54:35.441123+0000 2024-06-10T01:05:17.318 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/322543993 expires 2024-06-11T00:54:35.441123+0000 2024-06-10T01:05:17.318 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/212510583 expires 2024-06-11T00:54:35.441123+0000 2024-06-10T01:05:17.318 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6801/3298829389 expires 2024-06-11T00:54:35.441123+0000 2024-06-10T01:05:17.318 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6800/1100766262 expires 2024-06-11T00:54:08.102405+0000 2024-06-10T01:05:17.318 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/2679436843 expires 2024-06-11T00:54:08.102405+0000 2024-06-10T01:05:17.318 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6801/1100766262 expires 2024-06-11T00:54:08.102405+0000 2024-06-10T01:05:17.318 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/1210355277 expires 2024-06-11T00:54:08.102405+0000 2024-06-10T01:05:17.318 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6800/3298829389 expires 2024-06-11T00:54:35.441123+0000 2024-06-10T01:05:17.318 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/3610017406 expires 2024-06-11T00:54:08.102405+0000 2024-06-10T01:05:17.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.320+0000 7f73fab8d700 1 -- 172.21.15.7:0/3994467931 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f73dc0a2ca0 msgr2=0x7f73dc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:17.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.320+0000 7f73fab8d700 1 --2- 172.21.15.7:0/3994467931 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f73dc0a2ca0 0x7f73dc0a5160 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f73f0005fd0 tx=0x7f73f0005e50 comp rx=0 tx=0).stop 2024-06-10T01:05:17.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.320+0000 7f73fab8d700 1 -- 172.21.15.7:0/3994467931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73f4102f60 msgr2=0x7f73f419f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:17.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.320+0000 7f73fab8d700 1 --2- 172.21.15.7:0/3994467931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73f4102f60 0x7f73f419f9b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f73ec006070 tx=0x7f73ec007e40 comp rx=0 tx=0).stop 2024-06-10T01:05:17.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.320+0000 7f73fab8d700 1 -- 172.21.15.7:0/3994467931 shutdown_connections 2024-06-10T01:05:17.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.320+0000 7f73fab8d700 1 --2- 172.21.15.7:0/3994467931 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f73dc0a2ca0 0x7f73dc0a5160 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:17.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.320+0000 7f73fab8d700 1 --2- 172.21.15.7:0/3994467931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73f4102f60 0x7f73f419f9b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:17.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.320+0000 7f73fab8d700 1 --2- 172.21.15.7:0/3994467931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73f41013c0 0x7f73f406ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:17.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.320+0000 7f73fab8d700 1 --2- 172.21.15.7:0/3994467931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73f41025c0 0x7f73f419d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:17.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.320+0000 7f73fab8d700 1 -- 172.21.15.7:0/3994467931 >> 172.21.15.7:0/3994467931 conn(0x7f73f40fc960 msgr2=0x7f73f4105860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:17.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.320+0000 7f73fab8d700 1 -- 172.21.15.7:0/3994467931 shutdown_connections 2024-06-10T01:05:17.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:17.320+0000 7f73fab8d700 1 -- 172.21.15.7:0/3994467931 wait complete. 2024-06-10T01:05:17.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:17 smithi175 ceph-mon[36636]: pgmap v409: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:05:17.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:17 smithi007 ceph-mon[32398]: pgmap v409: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:05:17.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:17 smithi078 ceph-mon[36683]: pgmap v409: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:05:18.012 INFO:teuthology.run_tasks:Running task fs.ready... 2024-06-10T01:05:18.024 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:05:18.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:18 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3994467931' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-10T01:05:18.277 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:05:18.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:18 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3994467931' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-10T01:05:18.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:18 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3994467931' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-10T01:05:19.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:19 smithi007 ceph-mon[32398]: pgmap v410: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:05:19.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:19 smithi175 ceph-mon[36636]: pgmap v410: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:05:19.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:19 smithi078 ceph-mon[36683]: pgmap v410: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:05:19.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.813+0000 7f2b841b2700 1 -- 172.21.15.7:0/2723553748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c1066b0 msgr2=0x7f2b7c10e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:19.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.813+0000 7f2b841b2700 1 --2- 172.21.15.7:0/2723553748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c1066b0 0x7f2b7c10e280 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2b7400b0d0 tx=0x7f2b7400b3e0 comp rx=0 tx=0).stop 2024-06-10T01:05:19.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.813+0000 7f2b841b2700 1 -- 172.21.15.7:0/2723553748 shutdown_connections 2024-06-10T01:05:19.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.813+0000 7f2b841b2700 1 --2- 172.21.15.7:0/2723553748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c1066b0 0x7f2b7c10e280 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:19.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.813+0000 7f2b841b2700 1 --2- 172.21.15.7:0/2723553748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b7c101620 0x7f2b7c101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:19.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.813+0000 7f2b841b2700 1 --2- 172.21.15.7:0/2723553748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b7c101fd0 0x7f2b7c105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:19.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.813+0000 7f2b841b2700 1 -- 172.21.15.7:0/2723553748 >> 172.21.15.7:0/2723553748 conn(0x7f2b7c078340 msgr2=0x7f2b7c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:19.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.813+0000 7f2b841b2700 1 -- 172.21.15.7:0/2723553748 shutdown_connections 2024-06-10T01:05:19.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.813+0000 7f2b841b2700 1 -- 172.21.15.7:0/2723553748 wait complete. 2024-06-10T01:05:19.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.814+0000 7f2b841b2700 1 Processor -- start 2024-06-10T01:05:19.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.814+0000 7f2b841b2700 1 -- start start 2024-06-10T01:05:19.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.814+0000 7f2b841b2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c101620 0x7f2b7c199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:19.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.814+0000 7f2b841b2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b7c101fd0 0x7f2b7c19a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:19.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.814+0000 7f2b841b2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b7c1066b0 0x7f2b7c1a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:19.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.814+0000 7f2b841b2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c1116c0 con 0x7f2b7c101620 2024-06-10T01:05:19.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.814+0000 7f2b841b2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c111540 con 0x7f2b7c101fd0 2024-06-10T01:05:19.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.814+0000 7f2b841b2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c111840 con 0x7f2b7c1066b0 2024-06-10T01:05:19.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.814+0000 7f2b81f4e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c101620 0x7f2b7c199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:19.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.814+0000 7f2b81f4e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c101620 0x7f2b7c199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37232/0 (socket says 172.21.15.7:37232) 2024-06-10T01:05:19.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.814+0000 7f2b81f4e700 1 -- 172.21.15.7:0/1587845983 learned_addr learned my addr 172.21.15.7:0/1587845983 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:19.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.814+0000 7f2b8274f700 1 --2- 172.21.15.7:0/1587845983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b7c1066b0 0x7f2b7c1a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:19.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.814+0000 7f2b8174d700 1 --2- 172.21.15.7:0/1587845983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b7c101fd0 0x7f2b7c19a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:19.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.814+0000 7f2b81f4e700 1 -- 172.21.15.7:0/1587845983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b7c1066b0 msgr2=0x7f2b7c1a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:19.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.814+0000 7f2b81f4e700 1 --2- 172.21.15.7:0/1587845983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b7c1066b0 0x7f2b7c1a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:19.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.814+0000 7f2b81f4e700 1 -- 172.21.15.7:0/1587845983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b7c101fd0 msgr2=0x7f2b7c19a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:19.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.814+0000 7f2b81f4e700 1 --2- 172.21.15.7:0/1587845983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b7c101fd0 0x7f2b7c19a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:19.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.814+0000 7f2b81f4e700 1 -- 172.21.15.7:0/1587845983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7c1a5080 con 0x7f2b7c101620 2024-06-10T01:05:19.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.814+0000 7f2b8274f700 1 --2- 172.21.15.7:0/1587845983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b7c1066b0 0x7f2b7c1a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:05:19.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.814+0000 7f2b8174d700 1 --2- 172.21.15.7:0/1587845983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b7c101fd0 0x7f2b7c19a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:05:19.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.815+0000 7f2b81f4e700 1 --2- 172.21.15.7:0/1587845983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c101620 0x7f2b7c199ac0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800c9b0 tx=0x7f2b7800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:19.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.815+0000 7f2b72ffd700 1 -- 172.21.15.7:0/1587845983 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b78007b00 con 0x7f2b7c101620 2024-06-10T01:05:19.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.815+0000 7f2b72ffd700 1 -- 172.21.15.7:0/1587845983 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b78007ca0 con 0x7f2b7c101620 2024-06-10T01:05:19.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.815+0000 7f2b841b2700 1 -- 172.21.15.7:0/1587845983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b7c1a5250 con 0x7f2b7c101620 2024-06-10T01:05:19.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.815+0000 7f2b72ffd700 1 -- 172.21.15.7:0/1587845983 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b78018780 con 0x7f2b7c101620 2024-06-10T01:05:19.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.815+0000 7f2b841b2700 1 -- 172.21.15.7:0/1587845983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b7c1a5760 con 0x7f2b7c101620 2024-06-10T01:05:19.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.816+0000 7f2b841b2700 1 -- 172.21.15.7:0/1587845983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b7c069180 con 0x7f2b7c101620 2024-06-10T01:05:19.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.817+0000 7f2b72ffd700 1 -- 172.21.15.7:0/1587845983 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f2b7800f450 con 0x7f2b7c101620 2024-06-10T01:05:19.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.818+0000 7f2b72ffd700 1 --2- 172.21.15.7:0/1587845983 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2b680a2c20 0x7f2b680a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:19.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.818+0000 7f2b8174d700 1 --2- 172.21.15.7:0/1587845983 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2b680a2c20 0x7f2b680a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:19.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.818+0000 7f2b72ffd700 1 -- 172.21.15.7:0/1587845983 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f2b780cf6f0 con 0x7f2b7c101620 2024-06-10T01:05:19.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.818+0000 7f2b8174d700 1 --2- 172.21.15.7:0/1587845983 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2b680a2c20 0x7f2b680a50e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c107ac0 tx=0x7f2b6c009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:19.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:19.821+0000 7f2b72ffd700 1 -- 172.21.15.7:0/1587845983 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2b78014030 con 0x7f2b7c101620 2024-06-10T01:05:20.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:20.076+0000 7f2b841b2700 1 -- 172.21.15.7:0/1587845983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b7c06b160 con 0x7f2b7c101620 2024-06-10T01:05:20.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:20.076+0000 7f2b72ffd700 1 -- 172.21.15.7:0/1587845983 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11855 (secure 0 0 0) 0x7f2b7809c360 con 0x7f2b7c101620 2024-06-10T01:05:20.073 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:05:20.073 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:02:58.523980+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/2784910673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2784910673},{"type":"v1","addr":"172.21.15.175:6835","nonce":2784910673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:05:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:20.078+0000 7f2b841b2700 1 -- 172.21.15.7:0/1587845983 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2b680a2c20 msgr2=0x7f2b680a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:20.078+0000 7f2b841b2700 1 --2- 172.21.15.7:0/1587845983 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2b680a2c20 0x7f2b680a50e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c107ac0 tx=0x7f2b6c009550 comp rx=0 tx=0).stop 2024-06-10T01:05:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:20.078+0000 7f2b841b2700 1 -- 172.21.15.7:0/1587845983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c101620 msgr2=0x7f2b7c199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:20.078+0000 7f2b841b2700 1 --2- 172.21.15.7:0/1587845983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c101620 0x7f2b7c199ac0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800c9b0 tx=0x7f2b7800cd70 comp rx=0 tx=0).stop 2024-06-10T01:05:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:20.078+0000 7f2b841b2700 1 -- 172.21.15.7:0/1587845983 shutdown_connections 2024-06-10T01:05:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:20.078+0000 7f2b841b2700 1 --2- 172.21.15.7:0/1587845983 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2b680a2c20 0x7f2b680a50e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:20.078+0000 7f2b841b2700 1 --2- 172.21.15.7:0/1587845983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b7c1066b0 0x7f2b7c1a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:20.078+0000 7f2b841b2700 1 --2- 172.21.15.7:0/1587845983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c101620 0x7f2b7c199ac0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:20.078+0000 7f2b841b2700 1 --2- 172.21.15.7:0/1587845983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b7c101fd0 0x7f2b7c19a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:20.078+0000 7f2b841b2700 1 -- 172.21.15.7:0/1587845983 >> 172.21.15.7:0/1587845983 conn(0x7f2b7c078340 msgr2=0x7f2b7c0ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:20.079+0000 7f2b841b2700 1 -- 172.21.15.7:0/1587845983 shutdown_connections 2024-06-10T01:05:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:20.079+0000 7f2b841b2700 1 -- 172.21.15.7:0/1587845983 wait complete. 2024-06-10T01:05:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-06-10T01:05:20.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:20 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1587845983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:05:20.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:20 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1587845983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:05:20.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:20 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1587845983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:05:20.679 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:05:20.930 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:05:21.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:21 smithi007 ceph-mon[32398]: pgmap v411: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:05:21.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:21 smithi175 ceph-mon[36636]: pgmap v411: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:05:21.482 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:21 smithi078 ceph-mon[36683]: pgmap v411: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:05:22.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.673+0000 7f6d31eef700 1 -- 172.21.15.7:0/3232787108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d2c107400 msgr2=0x7f6d2c1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:22.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.673+0000 7f6d31eef700 1 --2- 172.21.15.7:0/3232787108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d2c107400 0x7f6d2c1077e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c009990 tx=0x7f6d1c009ca0 comp rx=0 tx=0).stop 2024-06-10T01:05:22.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.673+0000 7f6d31eef700 1 -- 172.21.15.7:0/3232787108 shutdown_connections 2024-06-10T01:05:22.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.673+0000 7f6d31eef700 1 --2- 172.21.15.7:0/3232787108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d2c101d50 0x7f6d2c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:22.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.673+0000 7f6d31eef700 1 --2- 172.21.15.7:0/3232787108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d2c107400 0x7f6d2c1077e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:22.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.673+0000 7f6d31eef700 1 --2- 172.21.15.7:0/3232787108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d2c101390 0x7f6d2c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:22.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.673+0000 7f6d31eef700 1 -- 172.21.15.7:0/3232787108 >> 172.21.15.7:0/3232787108 conn(0x7f6d2c078340 msgr2=0x7f6d2c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:22.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.673+0000 7f6d31eef700 1 -- 172.21.15.7:0/3232787108 shutdown_connections 2024-06-10T01:05:22.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.673+0000 7f6d31eef700 1 -- 172.21.15.7:0/3232787108 wait complete. 2024-06-10T01:05:22.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.674+0000 7f6d31eef700 1 Processor -- start 2024-06-10T01:05:22.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.674+0000 7f6d31eef700 1 -- start start 2024-06-10T01:05:22.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.674+0000 7f6d31eef700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d2c101390 0x7f6d2c199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:22.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.674+0000 7f6d31eef700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d2c101d50 0x7f6d2c19a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:22.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.674+0000 7f6d31eef700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d2c107400 0x7f6d2c1a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:22.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.674+0000 7f6d31eef700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2c1116a0 con 0x7f6d2c107400 2024-06-10T01:05:22.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.674+0000 7f6d2bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d2c107400 0x7f6d2c1a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:22.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.674+0000 7f6d31eef700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2c111520 con 0x7f6d2c101d50 2024-06-10T01:05:22.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.674+0000 7f6d31eef700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2c111820 con 0x7f6d2c101390 2024-06-10T01:05:22.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.674+0000 7f6d2bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d2c107400 0x7f6d2c1a4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39074/0 (socket says 172.21.15.7:39074) 2024-06-10T01:05:22.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.675+0000 7f6d2bfff700 1 -- 172.21.15.7:0/3853998163 learned_addr learned my addr 172.21.15.7:0/3853998163 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:22.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.675+0000 7f6d2b7fe700 1 --2- 172.21.15.7:0/3853998163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d2c101390 0x7f6d2c199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:22.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.675+0000 7f6d22ffd700 1 --2- 172.21.15.7:0/3853998163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d2c101d50 0x7f6d2c19a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:22.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.675+0000 7f6d2bfff700 1 -- 172.21.15.7:0/3853998163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d2c101390 msgr2=0x7f6d2c199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:22.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.675+0000 7f6d2bfff700 1 --2- 172.21.15.7:0/3853998163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d2c101390 0x7f6d2c199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:22.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.675+0000 7f6d2bfff700 1 -- 172.21.15.7:0/3853998163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d2c101d50 msgr2=0x7f6d2c19a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:22.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.675+0000 7f6d2bfff700 1 --2- 172.21.15.7:0/3853998163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d2c101d50 0x7f6d2c19a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:22.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.675+0000 7f6d2bfff700 1 -- 172.21.15.7:0/3853998163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d2c1a50a0 con 0x7f6d2c107400 2024-06-10T01:05:22.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.675+0000 7f6d2bfff700 1 --2- 172.21.15.7:0/3853998163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d2c107400 0x7f6d2c1a4b60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6d24007ee0 tx=0x7f6d2400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:22.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.675+0000 7f6d297fa700 1 -- 172.21.15.7:0/3853998163 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2400dd20 con 0x7f6d2c107400 2024-06-10T01:05:22.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.675+0000 7f6d297fa700 1 -- 172.21.15.7:0/3853998163 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d24004020 con 0x7f6d2c107400 2024-06-10T01:05:22.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.675+0000 7f6d31eef700 1 -- 172.21.15.7:0/3853998163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d2c1a5270 con 0x7f6d2c107400 2024-06-10T01:05:22.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.675+0000 7f6d297fa700 1 -- 172.21.15.7:0/3853998163 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d24015660 con 0x7f6d2c107400 2024-06-10T01:05:22.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.675+0000 7f6d31eef700 1 -- 172.21.15.7:0/3853998163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d2c1a5760 con 0x7f6d2c107400 2024-06-10T01:05:22.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.676+0000 7f6d31eef700 1 -- 172.21.15.7:0/3853998163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d2c069180 con 0x7f6d2c107400 2024-06-10T01:05:22.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.678+0000 7f6d297fa700 1 -- 172.21.15.7:0/3853998163 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f6d240041d0 con 0x7f6d2c107400 2024-06-10T01:05:22.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.678+0000 7f6d297fa700 1 --2- 172.21.15.7:0/3853998163 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6d180a2c20 0x7f6d180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:22.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.678+0000 7f6d297fa700 1 -- 172.21.15.7:0/3853998163 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f6d240cf5f0 con 0x7f6d2c107400 2024-06-10T01:05:22.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.678+0000 7f6d2b7fe700 1 --2- 172.21.15.7:0/3853998163 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6d180a2c20 0x7f6d180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:22.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.679+0000 7f6d2b7fe700 1 --2- 172.21.15.7:0/3853998163 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6d180a2c20 0x7f6d180a50e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00b5c0 tx=0x7f6d1c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:22.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.682+0000 7f6d297fa700 1 -- 172.21.15.7:0/3853998163 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6d24010070 con 0x7f6d2c107400 2024-06-10T01:05:22.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.934+0000 7f6d31eef700 1 -- 172.21.15.7:0/3853998163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d2c02dca0 con 0x7f6d2c107400 2024-06-10T01:05:22.932 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:05:22.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.935+0000 7f6d297fa700 1 -- 172.21.15.7:0/3853998163 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11855 (secure 0 0 0) 0x7f6d2409c1d0 con 0x7f6d2c107400 2024-06-10T01:05:22.933 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:02:58.523980+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/2784910673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2784910673},{"type":"v1","addr":"172.21.15.175:6835","nonce":2784910673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:05:22.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.937+0000 7f6d31eef700 1 -- 172.21.15.7:0/3853998163 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6d180a2c20 msgr2=0x7f6d180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:22.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.937+0000 7f6d31eef700 1 --2- 172.21.15.7:0/3853998163 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6d180a2c20 0x7f6d180a50e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00b5c0 tx=0x7f6d1c005960 comp rx=0 tx=0).stop 2024-06-10T01:05:22.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.937+0000 7f6d31eef700 1 -- 172.21.15.7:0/3853998163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d2c107400 msgr2=0x7f6d2c1a4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:22.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.937+0000 7f6d31eef700 1 --2- 172.21.15.7:0/3853998163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d2c107400 0x7f6d2c1a4b60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6d24007ee0 tx=0x7f6d2400d3b0 comp rx=0 tx=0).stop 2024-06-10T01:05:22.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.937+0000 7f6d31eef700 1 -- 172.21.15.7:0/3853998163 shutdown_connections 2024-06-10T01:05:22.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.937+0000 7f6d31eef700 1 --2- 172.21.15.7:0/3853998163 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6d180a2c20 0x7f6d180a50e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:22.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.937+0000 7f6d31eef700 1 --2- 172.21.15.7:0/3853998163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d2c107400 0x7f6d2c1a4b60 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:22.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.937+0000 7f6d31eef700 1 --2- 172.21.15.7:0/3853998163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d2c101390 0x7f6d2c199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:22.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.937+0000 7f6d31eef700 1 --2- 172.21.15.7:0/3853998163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d2c101d50 0x7f6d2c19a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:22.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.937+0000 7f6d31eef700 1 -- 172.21.15.7:0/3853998163 >> 172.21.15.7:0/3853998163 conn(0x7f6d2c078340 msgr2=0x7f6d2c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:22.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.937+0000 7f6d31eef700 1 -- 172.21.15.7:0/3853998163 shutdown_connections 2024-06-10T01:05:22.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:22.938+0000 7f6d31eef700 1 -- 172.21.15.7:0/3853998163 wait complete. 2024-06-10T01:05:22.935 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-06-10T01:05:23.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:23 smithi175 ceph-mon[36636]: pgmap v412: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:05:23.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:23 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3853998163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:05:23.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:23 smithi007 ceph-mon[32398]: pgmap v412: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:05:23.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:23 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3853998163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:05:23.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:23 smithi078 ceph-mon[36683]: pgmap v412: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:05:23.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:23 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3853998163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:05:23.641 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T01:05:23.892 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:05:25.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:25 smithi007 ceph-mon[32398]: pgmap v413: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:05:25.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:25 smithi175 ceph-mon[36636]: pgmap v413: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:05:25.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.542+0000 7fb173e87700 1 -- 172.21.15.7:0/3406929685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb16c107860 msgr2=0x7fb16c10f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:25.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.542+0000 7fb173e87700 1 --2- 172.21.15.7:0/3406929685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb16c107860 0x7fb16c10f3e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb1640099c0 tx=0x7fb164009cd0 comp rx=0 tx=0).stop 2024-06-10T01:05:25.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.542+0000 7fb173e87700 1 -- 172.21.15.7:0/3406929685 shutdown_connections 2024-06-10T01:05:25.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.542+0000 7fb173e87700 1 --2- 172.21.15.7:0/3406929685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb16c107860 0x7fb16c10f3e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:25.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.542+0000 7fb173e87700 1 --2- 172.21.15.7:0/3406929685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb16c102770 0x7fb16c102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:25.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.542+0000 7fb173e87700 1 --2- 172.21.15.7:0/3406929685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb16c103120 0x7fb16c107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:25.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.542+0000 7fb173e87700 1 -- 172.21.15.7:0/3406929685 >> 172.21.15.7:0/3406929685 conn(0x7fb16c079600 msgr2=0x7fb16c079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:25.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.542+0000 7fb173e87700 1 -- 172.21.15.7:0/3406929685 shutdown_connections 2024-06-10T01:05:25.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.542+0000 7fb173e87700 1 -- 172.21.15.7:0/3406929685 wait complete. 2024-06-10T01:05:25.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.543+0000 7fb173e87700 1 Processor -- start 2024-06-10T01:05:25.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.543+0000 7fb173e87700 1 -- start start 2024-06-10T01:05:25.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.543+0000 7fb173e87700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb16c102770 0x7fb16c19e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:25.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.543+0000 7fb173e87700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb16c103120 0x7fb16c19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:25.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.543+0000 7fb173e87700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb16c107860 0x7fb16c1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:25.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.543+0000 7fb173e87700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c112780 con 0x7fb16c103120 2024-06-10T01:05:25.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.543+0000 7fb173e87700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c112600 con 0x7fb16c102770 2024-06-10T01:05:25.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.543+0000 7fb173e87700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c112900 con 0x7fb16c107860 2024-06-10T01:05:25.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.543+0000 7fb171422700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb16c103120 0x7fb16c19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:25.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.543+0000 7fb171422700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb16c103120 0x7fb16c19ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39100/0 (socket says 172.21.15.7:39100) 2024-06-10T01:05:25.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.543+0000 7fb171422700 1 -- 172.21.15.7:0/39920694 learned_addr learned my addr 172.21.15.7:0/39920694 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:25.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.544+0000 7fb171c23700 1 --2- 172.21.15.7:0/39920694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb16c102770 0x7fb16c19e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:25.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.544+0000 7fb172424700 1 --2- 172.21.15.7:0/39920694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb16c107860 0x7fb16c1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:25.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.544+0000 7fb171422700 1 -- 172.21.15.7:0/39920694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb16c107860 msgr2=0x7fb16c1988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:25.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.544+0000 7fb171422700 1 --2- 172.21.15.7:0/39920694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb16c107860 0x7fb16c1988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:25.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.544+0000 7fb171422700 1 -- 172.21.15.7:0/39920694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb16c102770 msgr2=0x7fb16c19e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:25.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.544+0000 7fb171422700 1 --2- 172.21.15.7:0/39920694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb16c102770 0x7fb16c19e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:25.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.544+0000 7fb171422700 1 -- 172.21.15.7:0/39920694 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16c199020 con 0x7fb16c103120 2024-06-10T01:05:25.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.544+0000 7fb172424700 1 --2- 172.21.15.7:0/39920694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb16c107860 0x7fb16c1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:05:25.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.544+0000 7fb171422700 1 --2- 172.21.15.7:0/39920694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb16c103120 0x7fb16c19ebf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb16000d950 tx=0x7fb16000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:25.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.544+0000 7fb16affd700 1 -- 172.21.15.7:0/39920694 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160016070 con 0x7fb16c103120 2024-06-10T01:05:25.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.544+0000 7fb16affd700 1 -- 172.21.15.7:0/39920694 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb160009e20 con 0x7fb16c103120 2024-06-10T01:05:25.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.544+0000 7fb173e87700 1 -- 172.21.15.7:0/39920694 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb16c199310 con 0x7fb16c103120 2024-06-10T01:05:25.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.544+0000 7fb16affd700 1 -- 172.21.15.7:0/39920694 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16000f620 con 0x7fb16c103120 2024-06-10T01:05:25.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.544+0000 7fb173e87700 1 -- 172.21.15.7:0/39920694 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb16c1a5580 con 0x7fb16c103120 2024-06-10T01:05:25.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.545+0000 7fb173e87700 1 -- 172.21.15.7:0/39920694 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb16c069180 con 0x7fb16c103120 2024-06-10T01:05:25.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.546+0000 7fb16affd700 1 -- 172.21.15.7:0/39920694 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb160019070 con 0x7fb16c103120 2024-06-10T01:05:25.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.547+0000 7fb16affd700 1 --2- 172.21.15.7:0/39920694 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb1540a2ca0 0x7fb1540a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:25.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.547+0000 7fb171c23700 1 --2- 172.21.15.7:0/39920694 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb1540a2ca0 0x7fb1540a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:25.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.547+0000 7fb16affd700 1 -- 172.21.15.7:0/39920694 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fb1600cef20 con 0x7fb16c103120 2024-06-10T01:05:25.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.548+0000 7fb171c23700 1 --2- 172.21.15.7:0/39920694 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb1540a2ca0 0x7fb1540a5160 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb16c07a090 tx=0x7fb15c0079e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:25.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.551+0000 7fb16affd700 1 -- 172.21.15.7:0/39920694 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb16009bb00 con 0x7fb16c103120 2024-06-10T01:05:25.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:25 smithi078 ceph-mon[36683]: pgmap v413: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:05:25.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.758+0000 7fb173e87700 1 -- 172.21.15.7:0/39920694 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb16c1a57e0 con 0x7fb16c103120 2024-06-10T01:05:25.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.760+0000 7fb16affd700 1 -- 172.21.15.7:0/39920694 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24615 (secure 0 0 0) 0x7fb16009b400 con 0x7fb16c103120 2024-06-10T01:05:25.757 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:05:25.757 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":99,"fsid":"bded58f8-26c3-11ef-bc9b-c7b262605968","created":"2024-06-10T00:53:31.966085+0000","modified":"2024-06-10T01:03:38.227703+0000","last_up_change":"2024-06-10T01:00:45.382473+0000","last_in_change":"2024-06-10T01:00:25.608513+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:56:59.964055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T01:01:42.563611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T01:01:46.175918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T01:01:55.695502+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f7c24c9f-b7ab-4c04-b2b4-120df9751667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6803","nonce":404485726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6805","nonce":404485726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6809","nonce":404485726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6807","nonce":404485726}]},"public_addr":"172.21.15.7:6803/404485726","cluster_addr":"172.21.15.7:6805/404485726","heartbeat_back_addr":"172.21.15.7:6809/404485726","heartbeat_front_addr":"172.21.15.7:6807/404485726","state":["exists","up"]},{"osd":1,"uuid":"9bf70a6b-f31c-4077-b5af-229f17bc2845","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6801","nonce":1856804448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6803","nonce":1856804448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6807","nonce":1856804448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6805","nonce":1856804448}]},"public_addr":"172.21.15.78:6801/1856804448","cluster_addr":"172.21.15.78:6803/1856804448","heartbeat_back_addr":"172.21.15.78:6807/1856804448","heartbeat_front_addr":"172.21.15.78:6805/1856804448","state":["exists","up"]},{"osd":2,"uuid":"62923c97-caf9-4f7e-9c14-9c9918843c78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6801","nonce":918257455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6803","nonce":918257455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6807","nonce":918257455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6805","nonce":918257455}]},"public_addr":"172.21.15.175:6801/918257455","cluster_addr":"172.21.15.175:6803/918257455","heartbeat_back_addr":"172.21.15.175:6807/918257455","heartbeat_front_addr":"172.21.15.175:6805/918257455","state":["exists","up"]},{"osd":3,"uuid":"65242098-e748-48ab-8f0e-bb25b5c62742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6811","nonce":1797404966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6813","nonce":1797404966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6817","nonce":1797404966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6815","nonce":1797404966}]},"public_addr":"172.21.15.7:6811/1797404966","cluster_addr":"172.21.15.7:6813/1797404966","heartbeat_back_addr":"172.21.15.7:6817/1797404966","heartbeat_front_addr":"172.21.15.7:6815/1797404966","state":["exists","up"]},{"osd":4,"uuid":"a7990305-de9e-400e-a432-d7854f73f339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6809","nonce":3767608357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6811","nonce":3767608357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6815","nonce":3767608357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6813","nonce":3767608357}]},"public_addr":"172.21.15.78:6809/3767608357","cluster_addr":"172.21.15.78:6811/3767608357","heartbeat_back_addr":"172.21.15.78:6815/3767608357","heartbeat_front_addr":"172.21.15.78:6813/3767608357","state":["exists","up"]},{"osd":5,"uuid":"bad9a7b3-da5a-48d0-905d-e6a3190317fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6809","nonce":2157664731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6811","nonce":2157664731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6815","nonce":2157664731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6813","nonce":2157664731}]},"public_addr":"172.21.15.175:6809/2157664731","cluster_addr":"172.21.15.175:6811/2157664731","heartbeat_back_addr":"172.21.15.175:6815/2157664731","heartbeat_front_addr":"172.21.15.175:6813/2157664731","state":["exists","up"]},{"osd":6,"uuid":"6c687653-157b-4b96-867b-a2eaa7d8ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6819","nonce":838731756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6821","nonce":838731756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6825","nonce":838731756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6823","nonce":838731756}]},"public_addr":"172.21.15.7:6819/838731756","cluster_addr":"172.21.15.7:6821/838731756","heartbeat_back_addr":"172.21.15.7:6825/838731756","heartbeat_front_addr":"172.21.15.7:6823/838731756","state":["exists","up"]},{"osd":7,"uuid":"db40ddf1-9fd9-4859-b8c4-75f79582ab91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6817","nonce":3511921773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6819","nonce":3511921773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6823","nonce":3511921773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6821","nonce":3511921773}]},"public_addr":"172.21.15.78:6817/3511921773","cluster_addr":"172.21.15.78:6819/3511921773","heartbeat_back_addr":"172.21.15.78:6823/3511921773","heartbeat_front_addr":"172.21.15.78:6821/3511921773","state":["exists","up"]},{"osd":8,"uuid":"0f01cbd9-b193-4c70-a33c-4984167bf422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6817","nonce":1546325211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6819","nonce":1546325211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6823","nonce":1546325211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6821","nonce":1546325211}]},"public_addr":"172.21.15.175:6817/1546325211","cluster_addr":"172.21.15.175:6819/1546325211","heartbeat_back_addr":"172.21.15.175:6823/1546325211","heartbeat_front_addr":"172.21.15.175:6821/1546325211","state":["exists","up"]},{"osd":9,"uuid":"e5480998-f7fb-49cd-9aed-52405c88d830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6827","nonce":1844550813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6829","nonce":1844550813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6833","nonce":1844550813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6831","nonce":1844550813}]},"public_addr":"172.21.15.7:6827/1844550813","cluster_addr":"172.21.15.7:6829/1844550813","heartbeat_back_addr":"172.21.15.7:6833/1844550813","heartbeat_front_addr":"172.21.15.7:6831/1844550813","state":["exists","up"]},{"osd":10,"uuid":"caa82f33-a71c-4907-bf3b-30a7fdfeb71d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6825","nonce":2290824785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6827","nonce":2290824785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6831","nonce":2290824785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6829","nonce":2290824785}]},"public_addr":"172.21.15.78:6825/2290824785","cluster_addr":"172.21.15.78:6827/2290824785","heartbeat_back_addr":"172.21.15.78:6831/2290824785","heartbeat_front_addr":"172.21.15.78:6829/2290824785","state":["exists","up"]},{"osd":11,"uuid":"608cda77-77da-4da2-a7e2-76a773be27c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6825","nonce":1959979550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6827","nonce":1959979550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6831","nonce":1959979550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6829","nonce":1959979550}]},"public_addr":"172.21.15.175:6825/1959979550","cluster_addr":"172.21.15.175:6827/1959979550","heartbeat_back_addr":"172.21.15.175:6831/1959979550","heartbeat_front_addr":"172.21.15.175:6829/1959979550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:08.850203+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:32.820161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:56.633656+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:21.759372+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:46.735874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:10.636541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:36.990099+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:01.767098+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:26.882776+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:52.757595+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:18.383938+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:43.544330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1833397470":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/322543993":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/212510583":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6801/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6800/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/2679436843":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6801/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/1210355277":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6800/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/3610017406":"2024-06-11T00:54:08.102405+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T01:05:25.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.763+0000 7fb173e87700 1 -- 172.21.15.7:0/39920694 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb1540a2ca0 msgr2=0x7fb1540a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:25.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.763+0000 7fb173e87700 1 --2- 172.21.15.7:0/39920694 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb1540a2ca0 0x7fb1540a5160 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb16c07a090 tx=0x7fb15c0079e0 comp rx=0 tx=0).stop 2024-06-10T01:05:25.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.763+0000 7fb173e87700 1 -- 172.21.15.7:0/39920694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb16c103120 msgr2=0x7fb16c19ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:25.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.763+0000 7fb173e87700 1 --2- 172.21.15.7:0/39920694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb16c103120 0x7fb16c19ebf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb16000d950 tx=0x7fb16000dc60 comp rx=0 tx=0).stop 2024-06-10T01:05:25.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.763+0000 7fb173e87700 1 -- 172.21.15.7:0/39920694 shutdown_connections 2024-06-10T01:05:25.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.763+0000 7fb173e87700 1 --2- 172.21.15.7:0/39920694 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb1540a2ca0 0x7fb1540a5160 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:25.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.763+0000 7fb173e87700 1 --2- 172.21.15.7:0/39920694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb16c107860 0x7fb16c1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:25.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.763+0000 7fb173e87700 1 --2- 172.21.15.7:0/39920694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb16c102770 0x7fb16c19e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:25.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.763+0000 7fb173e87700 1 --2- 172.21.15.7:0/39920694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb16c103120 0x7fb16c19ebf0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:25.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.763+0000 7fb173e87700 1 -- 172.21.15.7:0/39920694 >> 172.21.15.7:0/39920694 conn(0x7fb16c079600 msgr2=0x7fb16c100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:25.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.763+0000 7fb173e87700 1 -- 172.21.15.7:0/39920694 shutdown_connections 2024-06-10T01:05:25.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:25.763+0000 7fb173e87700 1 -- 172.21.15.7:0/39920694 wait complete. 2024-06-10T01:05:26.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:26 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/39920694' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:05:26.445 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-10T01:01:49.029348+0000', 'modified': '2024-06-10T01:02:58.523980+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24451, 'mds_1': 24436, 'mds_2': 24434}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24403': {'gid': 24403, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.175:6833/1281305765', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6832', 'nonce': 1281305765}, {'type': 'v1', 'addr': '172.21.15.175:6833', 'nonce': 1281305765}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24413': {'gid': 24413, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.78:6837/3608622984', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6836', 'nonce': 3608622984}, {'type': 'v1', 'addr': '172.21.15.78:6837', 'nonce': 3608622984}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24424': {'gid': 24424, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.175:6835/2784910673', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6834', 'nonce': 2784910673}, {'type': 'v1', 'addr': '172.21.15.175:6835', 'nonce': 2784910673}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'b', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.78:6839/1946947484', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6838', 'nonce': 1946947484}, {'type': 'v1', 'addr': '172.21.15.78:6839', 'nonce': 1946947484}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'c', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.175:6837/779740919', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6836', 'nonce': 779740919}, {'type': 'v1', 'addr': '172.21.15.175:6837', 'nonce': 779740919}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.175:6839/1983418024', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6838', 'nonce': 1983418024}, {'type': 'v1', 'addr': '172.21.15.175:6839', 'nonce': 1983418024}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24451, 'qdb_cluster': [24436, 24451, 24434]} 2024-06-10T01:05:26.445 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-06-10T01:05:26.445 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-10T01:05:26.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:26 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/39920694' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:05:26.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:26 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/39920694' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:05:26.700 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:05:27.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:27 smithi175 ceph-mon[36636]: pgmap v414: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:05:27.496 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:27 smithi007 ceph-mon[32398]: pgmap v414: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:05:27.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:27 smithi078 ceph-mon[36683]: pgmap v414: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:05:28.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.338+0000 7f0645db0700 1 -- 172.21.15.7:0/4252802394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0640074fe0 msgr2=0x7f064010e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:28.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.338+0000 7f0645db0700 1 --2- 172.21.15.7:0/4252802394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0640074fe0 0x7f064010e950 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0634009990 tx=0x7f0634009ca0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.338+0000 7f0645db0700 1 -- 172.21.15.7:0/4252802394 shutdown_connections 2024-06-10T01:05:28.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.338+0000 7f0645db0700 1 --2- 172.21.15.7:0/4252802394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f064010ee90 0x7f0640111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.338+0000 7f0645db0700 1 --2- 172.21.15.7:0/4252802394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06400746c0 0x7f0640074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.338+0000 7f0645db0700 1 --2- 172.21.15.7:0/4252802394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0640074fe0 0x7f064010e950 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.338+0000 7f0645db0700 1 -- 172.21.15.7:0/4252802394 >> 172.21.15.7:0/4252802394 conn(0x7f06400795f0 msgr2=0x7f0640079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:28.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.338+0000 7f0645db0700 1 -- 172.21.15.7:0/4252802394 shutdown_connections 2024-06-10T01:05:28.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.338+0000 7f0645db0700 1 -- 172.21.15.7:0/4252802394 wait complete. 2024-06-10T01:05:28.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.338+0000 7f0645db0700 1 Processor -- start 2024-06-10T01:05:28.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.339+0000 7f0645db0700 1 -- start start 2024-06-10T01:05:28.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.339+0000 7f0645db0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06400746c0 0x7f064019ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:28.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.339+0000 7f0645db0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0640074fe0 0x7f064019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:28.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.339+0000 7f0645db0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f064010ee90 0x7f064019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:28.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.339+0000 7f0645db0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f06401149c0 con 0x7f06400746c0 2024-06-10T01:05:28.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.339+0000 7f0645db0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0640114840 con 0x7f064010ee90 2024-06-10T01:05:28.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.339+0000 7f0645db0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0640114b40 con 0x7f0640074fe0 2024-06-10T01:05:28.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.339+0000 7f063f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06400746c0 0x7f064019ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:28.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.339+0000 7f063f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06400746c0 0x7f064019ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39110/0 (socket says 172.21.15.7:39110) 2024-06-10T01:05:28.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.339+0000 7f063f7fe700 1 -- 172.21.15.7:0/274356836 learned_addr learned my addr 172.21.15.7:0/274356836 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:28.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.339+0000 7f063effd700 1 --2- 172.21.15.7:0/274356836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0640074fe0 0x7f064019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:28.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.339+0000 7f063ffff700 1 --2- 172.21.15.7:0/274356836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f064010ee90 0x7f064019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:28.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.339+0000 7f063f7fe700 1 -- 172.21.15.7:0/274356836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0640074fe0 msgr2=0x7f064019b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:28.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.339+0000 7f063f7fe700 1 --2- 172.21.15.7:0/274356836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0640074fe0 0x7f064019b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.339+0000 7f063f7fe700 1 -- 172.21.15.7:0/274356836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f064010ee90 msgr2=0x7f064019f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:28.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.339+0000 7f063f7fe700 1 --2- 172.21.15.7:0/274356836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f064010ee90 0x7f064019f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.339+0000 7f063f7fe700 1 -- 172.21.15.7:0/274356836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f064019fd80 con 0x7f06400746c0 2024-06-10T01:05:28.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.340+0000 7f063f7fe700 1 --2- 172.21.15.7:0/274356836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06400746c0 0x7f064019ad70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f063000c9b0 tx=0x7f063000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:28.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.340+0000 7f063cff9700 1 -- 172.21.15.7:0/274356836 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0630007b00 con 0x7f06400746c0 2024-06-10T01:05:28.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.340+0000 7f0645db0700 1 -- 172.21.15.7:0/274356836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06401a0070 con 0x7f06400746c0 2024-06-10T01:05:28.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.340+0000 7f063cff9700 1 -- 172.21.15.7:0/274356836 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0630007ca0 con 0x7f06400746c0 2024-06-10T01:05:28.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.340+0000 7f063cff9700 1 -- 172.21.15.7:0/274356836 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0630018780 con 0x7f06400746c0 2024-06-10T01:05:28.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.340+0000 7f0645db0700 1 -- 172.21.15.7:0/274356836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06401a78f0 con 0x7f06400746c0 2024-06-10T01:05:28.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.342+0000 7f063cff9700 1 -- 172.21.15.7:0/274356836 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f063000f450 con 0x7f06400746c0 2024-06-10T01:05:28.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.343+0000 7f063cff9700 1 --2- 172.21.15.7:0/274356836 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f06280a2c20 0x7f06280a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:28.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.343+0000 7f063effd700 1 --2- 172.21.15.7:0/274356836 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f06280a2c20 0x7f06280a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:28.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.343+0000 7f063cff9700 1 -- 172.21.15.7:0/274356836 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f06300cf430 con 0x7f06400746c0 2024-06-10T01:05:28.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.343+0000 7f063effd700 1 --2- 172.21.15.7:0/274356836 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f06280a2c20 0x7f06280a50e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0634006010 tx=0x7f063400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:28.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.343+0000 7f0650633b80 1 Processor -- start 2024-06-10T01:05:28.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.343+0000 7f0650633b80 1 -- start start 2024-06-10T01:05:28.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.344+0000 7f0650633b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621b3f71f10 0x5621b3f722f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:28.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.344+0000 7f0650633b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621b3f72830 0x5621b3f6c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:28.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.344+0000 7f0650633b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b3f6c750 0x5621b3f6cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:28.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.344+0000 7f0650633b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5621b3e5ef10 con 0x5621b3f71f10 2024-06-10T01:05:28.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.344+0000 7f0650633b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5621b3ebd800 con 0x5621b3f6c750 2024-06-10T01:05:28.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.344+0000 7f0650633b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5621b3e25410 con 0x5621b3f72830 2024-06-10T01:05:28.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.344+0000 7f063f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621b3f71f10 0x5621b3f722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:28.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.344+0000 7f063f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621b3f71f10 0x5621b3f722f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39120/0 (socket says 172.21.15.7:39120) 2024-06-10T01:05:28.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.344+0000 7f063f7fe700 1 -- 172.21.15.7:0/1369614531 learned_addr learned my addr 172.21.15.7:0/1369614531 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:28.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.344+0000 7f063ffff700 1 --2- 172.21.15.7:0/1369614531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b3f6c750 0x5621b3f6cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:28.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.344+0000 7f063effd700 1 --2- 172.21.15.7:0/1369614531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621b3f72830 0x5621b3f6c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:28.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.344+0000 7f063f7fe700 1 -- 172.21.15.7:0/1369614531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621b3f72830 msgr2=0x5621b3f6c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:28.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.344+0000 7f063f7fe700 1 --2- 172.21.15.7:0/1369614531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621b3f72830 0x5621b3f6c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.344+0000 7f063f7fe700 1 -- 172.21.15.7:0/1369614531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b3f6c750 msgr2=0x5621b3f6cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:28.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.344+0000 7f063f7fe700 1 --2- 172.21.15.7:0/1369614531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b3f6c750 0x5621b3f6cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.344+0000 7f063f7fe700 1 -- 172.21.15.7:0/1369614531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621b3e0c880 con 0x5621b3f71f10 2024-06-10T01:05:28.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.344+0000 7f063ffff700 1 --2- 172.21.15.7:0/1369614531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b3f6c750 0x5621b3f6cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:05:28.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.344+0000 7f063f7fe700 1 --2- 172.21.15.7:0/1369614531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621b3f71f10 0x5621b3f722f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f06300a4050 tx=0x7f063009c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0738ea9236a7a09 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:28.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.344+0000 7f0625ffb700 1 -- 172.21.15.7:0/1369614531 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06300a4580 con 0x5621b3f71f10 2024-06-10T01:05:28.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.345+0000 7f0625ffb700 1 -- 172.21.15.7:0/1369614531 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f063000dda0 con 0x5621b3f71f10 2024-06-10T01:05:28.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.345+0000 7f0625ffb700 1 -- 172.21.15.7:0/1369614531 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063009bb70 con 0x5621b3f71f10 2024-06-10T01:05:28.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.345+0000 7f0650633b80 1 -- 172.21.15.7:0/1369614531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621b3f71f10 msgr2=0x5621b3f722f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:28.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.345+0000 7f0650633b80 1 --2- 172.21.15.7:0/1369614531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621b3f71f10 0x5621b3f722f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f06300a4050 tx=0x7f063009c010 comp rx=0 tx=0).stop 2024-06-10T01:05:28.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.345+0000 7f0650633b80 1 -- 172.21.15.7:0/1369614531 shutdown_connections 2024-06-10T01:05:28.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.345+0000 7f0650633b80 1 --2- 172.21.15.7:0/1369614531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b3f6c750 0x5621b3f6cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.345+0000 7f0650633b80 1 --2- 172.21.15.7:0/1369614531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621b3f71f10 0x5621b3f722f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.345+0000 7f0650633b80 1 --2- 172.21.15.7:0/1369614531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621b3f72830 0x5621b3f6c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.345+0000 7f0650633b80 1 -- 172.21.15.7:0/1369614531 >> 172.21.15.7:0/1369614531 conn(0x5621b3e99030 msgr2=0x5621b3f671d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:28.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.345+0000 7f0650633b80 1 -- 172.21.15.7:0/1369614531 shutdown_connections 2024-06-10T01:05:28.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.345+0000 7f0650633b80 1 -- 172.21.15.7:0/1369614531 wait complete. 2024-06-10T01:05:28.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.346+0000 7f0650633b80 1 -- start start 2024-06-10T01:05:28.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.346+0000 7f0650633b80 1 Processor -- start 2024-06-10T01:05:28.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.346+0000 7f0650633b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621b3f6c750 0x5621b3f785c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:28.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.346+0000 7f0650633b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621b3f71f10 0x5621b3f758c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:28.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.346+0000 7f0650633b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b3f72830 0x5621b3f75e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:28.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.346+0000 7f0650633b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5621b3e5df70 con 0x5621b3f71f10 2024-06-10T01:05:28.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.346+0000 7f0650633b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5621b3eb97c0 con 0x5621b3f72830 2024-06-10T01:05:28.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.346+0000 7f0650633b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5621b3f79210 con 0x5621b3f6c750 2024-06-10T01:05:28.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.346+0000 7f0650633b80 10 client.0 _ll_register_callbacks cb 0x5621b3e53560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:05:28.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.346+0000 7f063effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621b3f71f10 0x5621b3f758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:28.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.346+0000 7f063effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621b3f71f10 0x5621b3f758c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39136/0 (socket says 172.21.15.7:39136) 2024-06-10T01:05:28.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.346+0000 7f063effd700 1 -- 172.21.15.7:0/1433651025 learned_addr learned my addr 172.21.15.7:0/1433651025 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:28.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.347+0000 7f063f7fe700 1 --2- 172.21.15.7:0/1433651025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621b3f6c750 0x5621b3f785c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:28.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.347+0000 7f063ffff700 1 --2- 172.21.15.7:0/1433651025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b3f72830 0x5621b3f75e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:28.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.347+0000 7f063effd700 1 -- 172.21.15.7:0/1433651025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621b3f6c750 msgr2=0x5621b3f785c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:28.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.347+0000 7f063effd700 1 --2- 172.21.15.7:0/1433651025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621b3f6c750 0x5621b3f785c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.347+0000 7f063effd700 1 -- 172.21.15.7:0/1433651025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b3f72830 msgr2=0x5621b3f75e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:28.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.347+0000 7f063effd700 1 --2- 172.21.15.7:0/1433651025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b3f72830 0x5621b3f75e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.347+0000 7f063effd700 1 -- 172.21.15.7:0/1433651025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621b3e83700 con 0x5621b3f71f10 2024-06-10T01:05:28.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.347+0000 7f063effd700 1 --2- 172.21.15.7:0/1433651025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621b3f71f10 0x5621b3f758c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0634015000 tx=0x7f0634009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a0b9470d463cd3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:28.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.347+0000 7f061effd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:05:28.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.347+0000 7f061effd700 1 -- 172.21.15.7:0/1433651025 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0634035060 con 0x5621b3f71f10 2024-06-10T01:05:28.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.347+0000 7f061effd700 1 -- 172.21.15.7:0/1433651025 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f063400fc80 con 0x5621b3f71f10 2024-06-10T01:05:28.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.347+0000 7f0650633b80 1 -- 172.21.15.7:0/1433651025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f063000dda0 con 0x5621b3f71f10 2024-06-10T01:05:28.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.347+0000 7f061effd700 1 -- 172.21.15.7:0/1433651025 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f063403c050 con 0x5621b3f71f10 2024-06-10T01:05:28.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.347+0000 7f0650633b80 10 client.15156 fetch_fsmap learned FSMap version 26 2024-06-10T01:05:28.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.347+0000 7f0650633b80 1 -- 172.21.15.7:0/1433651025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5621b3f839f0 con 0x5621b3f71f10 2024-06-10T01:05:28.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.348+0000 7f061effd700 1 -- 172.21.15.7:0/1433651025 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f063000dda0 con 0x5621b3f71f10 2024-06-10T01:05:28.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.348+0000 7f061effd700 1 -- 172.21.15.7:0/1433651025 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f063402f070 con 0x5621b3f71f10 2024-06-10T01:05:28.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.348+0000 7f0650633b80 10 client.15156 fetch_fsmap finished waiting for FSMap version 26 2024-06-10T01:05:28.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.348+0000 7f0650633b80 10 client.15156 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-10T01:05:28.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.348+0000 7f0650633b80 20 client.15156 populate_metadata read hostname 'smithi007' 2024-06-10T01:05:28.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.348+0000 7f0650633b80 1 --2- 172.21.15.7:0/1433651025 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5621b3f85fd0 0x5621b3f87440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:28.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.348+0000 7f0650633b80 4 client.15156 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:05:28.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.348+0000 7f0650633b80 1 -- 172.21.15.7:0/1433651025 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621b3ecad00 con 0x5621b3f85fd0 2024-06-10T01:05:28.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.348+0000 7f063f7fe700 1 --2- 172.21.15.7:0/1433651025 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5621b3f85fd0 0x5621b3f87440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:28.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.349+0000 7f063f7fe700 1 --2- 172.21.15.7:0/1433651025 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5621b3f85fd0 0x5621b3f87440 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=99360501ee2fe147 server_cookie=80cba0826e013b3b in_seq=0 out_seq=0 2024-06-10T01:05:28.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.349+0000 7f061effd700 10 client.15156 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:05:28.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.349+0000 7f061effd700 1 -- 172.21.15.7:0/1433651025 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5621b3ecad00 con 0x5621b3f85fd0 2024-06-10T01:05:28.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.349+0000 7f061effd700 10 client.15156 handle_command_reply: tid=0 2024-06-10T01:05:28.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.350+0000 7f0650633b80 1 client.15156 shutdown 2024-06-10T01:05:28.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.350+0000 7f0650633b80 1 -- 172.21.15.7:0/1433651025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621b3f71f10 msgr2=0x5621b3f758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:28.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.350+0000 7f0650633b80 1 --2- 172.21.15.7:0/1433651025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621b3f71f10 0x5621b3f758c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0634015000 tx=0x7f0634009ca0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.350+0000 7f0650633b80 1 -- 172.21.15.7:0/1433651025 shutdown_connections 2024-06-10T01:05:28.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.350+0000 7f0650633b80 1 --2- 172.21.15.7:0/1433651025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b3f72830 0x5621b3f75e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.350+0000 7f0650633b80 1 --2- 172.21.15.7:0/1433651025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621b3f6c750 0x5621b3f785c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.350+0000 7f0650633b80 1 --2- 172.21.15.7:0/1433651025 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5621b3f85fd0 0x5621b3f87440 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.350+0000 7f0650633b80 1 --2- 172.21.15.7:0/1433651025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621b3f71f10 0x5621b3f758c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.350+0000 7f0650633b80 1 -- 172.21.15.7:0/1433651025 >> 172.21.15.7:0/1433651025 conn(0x5621b3e99030 msgr2=0x5621b3f666b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:28.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.350+0000 7f061effd700 0 client.15156 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:05:28.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.350+0000 7f0650633b80 1 -- 172.21.15.7:0/1433651025 shutdown_connections 2024-06-10T01:05:28.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.350+0000 7f0650633b80 1 -- 172.21.15.7:0/1433651025 wait complete. 2024-06-10T01:05:28.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.350+0000 7f0650633b80 20 client.15156 trim_cache size 0 max 16384 2024-06-10T01:05:28.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.372+0000 7f0650633b80 1 Processor -- start 2024-06-10T01:05:28.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.372+0000 7f0650633b80 1 -- start start 2024-06-10T01:05:28.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.372+0000 7f0650633b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621b3f6c750 0x5621b3f75f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:28.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.372+0000 7f0650633b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b3f71f10 0x5621b3f78100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:28.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.372+0000 7f0650633b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621b3f72830 0x5621b3f66e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:28.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.372+0000 7f0650633b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5621b3f85780 con 0x5621b3f6c750 2024-06-10T01:05:28.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.372+0000 7f0650633b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5621b3f85e50 con 0x5621b3f71f10 2024-06-10T01:05:28.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.372+0000 7f0650633b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5621b3f86520 con 0x5621b3f72830 2024-06-10T01:05:28.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.372+0000 7f063f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621b3f6c750 0x5621b3f75f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:28.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.372+0000 7f063f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621b3f6c750 0x5621b3f75f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39152/0 (socket says 172.21.15.7:39152) 2024-06-10T01:05:28.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.372+0000 7f063f7fe700 1 -- 172.21.15.7:0/2193453927 learned_addr learned my addr 172.21.15.7:0/2193453927 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:28.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.372+0000 7f063ffff700 1 --2- 172.21.15.7:0/2193453927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621b3f72830 0x5621b3f66e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:28.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.372+0000 7f063effd700 1 --2- 172.21.15.7:0/2193453927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b3f71f10 0x5621b3f78100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:28.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.373+0000 7f063f7fe700 1 -- 172.21.15.7:0/2193453927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621b3f72830 msgr2=0x5621b3f66e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:28.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.373+0000 7f063f7fe700 1 --2- 172.21.15.7:0/2193453927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621b3f72830 0x5621b3f66e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.373+0000 7f063f7fe700 1 -- 172.21.15.7:0/2193453927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b3f71f10 msgr2=0x5621b3f78100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:28.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.373+0000 7f063f7fe700 1 --2- 172.21.15.7:0/2193453927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b3f71f10 0x5621b3f78100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.373+0000 7f063f7fe700 1 -- 172.21.15.7:0/2193453927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621b3f883a0 con 0x5621b3f6c750 2024-06-10T01:05:28.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.373+0000 7f063effd700 1 --2- 172.21.15.7:0/2193453927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b3f71f10 0x5621b3f78100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:05:28.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.373+0000 7f063f7fe700 1 --2- 172.21.15.7:0/2193453927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621b3f6c750 0x5621b3f75f90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f063000afd0 tx=0x7f0630000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc27a0c6cabd6ec7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:28.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.373+0000 7f06257fa700 1 -- 172.21.15.7:0/2193453927 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06300b8020 con 0x5621b3f6c750 2024-06-10T01:05:28.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.373+0000 7f06257fa700 1 -- 172.21.15.7:0/2193453927 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06300999b0 con 0x5621b3f6c750 2024-06-10T01:05:28.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.373+0000 7f06257fa700 1 -- 172.21.15.7:0/2193453927 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063000f740 con 0x5621b3f6c750 2024-06-10T01:05:28.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.374+0000 7f0650633b80 1 -- 172.21.15.7:0/2193453927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621b3f6c750 msgr2=0x5621b3f75f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:28.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.374+0000 7f0650633b80 1 --2- 172.21.15.7:0/2193453927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621b3f6c750 0x5621b3f75f90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f063000afd0 tx=0x7f0630000bc0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.374+0000 7f0650633b80 1 -- 172.21.15.7:0/2193453927 shutdown_connections 2024-06-10T01:05:28.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.374+0000 7f0650633b80 1 --2- 172.21.15.7:0/2193453927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621b3f72830 0x5621b3f66e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.374+0000 7f0650633b80 1 --2- 172.21.15.7:0/2193453927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621b3f6c750 0x5621b3f75f90 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.374+0000 7f0650633b80 1 --2- 172.21.15.7:0/2193453927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b3f71f10 0x5621b3f78100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.374+0000 7f0650633b80 1 -- 172.21.15.7:0/2193453927 >> 172.21.15.7:0/2193453927 conn(0x5621b3e99030 msgr2=0x5621b3f75880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:28.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.374+0000 7f0650633b80 1 -- 172.21.15.7:0/2193453927 shutdown_connections 2024-06-10T01:05:28.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.374+0000 7f0650633b80 1 -- 172.21.15.7:0/2193453927 wait complete. 2024-06-10T01:05:28.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.374+0000 7f0650633b80 1 -- start start 2024-06-10T01:05:28.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.374+0000 7f0650633b80 1 Processor -- start 2024-06-10T01:05:28.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.375+0000 7f0650633b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621b3f6c750 0x5621b3f666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:28.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.375+0000 7f0650633b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b3f71f10 0x5621b3f66bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:28.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.375+0000 7f0650633b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621b3f72830 0x5621b3f67130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:28.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.375+0000 7f0650633b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5621b3f872c0 con 0x5621b3f6c750 2024-06-10T01:05:28.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.375+0000 7f0650633b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5621b3f86bf0 con 0x5621b3f71f10 2024-06-10T01:05:28.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.375+0000 7f0650633b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5621b3f87990 con 0x5621b3f72830 2024-06-10T01:05:28.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.375+0000 7f0650633b80 10 client.0 _ll_register_callbacks cb 0x5621b3e53560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:05:28.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.375+0000 7f063f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621b3f6c750 0x5621b3f666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:28.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.375+0000 7f063f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621b3f6c750 0x5621b3f666b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39156/0 (socket says 172.21.15.7:39156) 2024-06-10T01:05:28.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.375+0000 7f063f7fe700 1 -- 172.21.15.7:0/3548333162 learned_addr learned my addr 172.21.15.7:0/3548333162 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:28.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.375+0000 7f063effd700 1 --2- 172.21.15.7:0/3548333162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b3f71f10 0x5621b3f66bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:28.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.375+0000 7f063ffff700 1 --2- 172.21.15.7:0/3548333162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621b3f72830 0x5621b3f67130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:28.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.375+0000 7f063f7fe700 1 -- 172.21.15.7:0/3548333162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621b3f72830 msgr2=0x5621b3f67130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:28.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.375+0000 7f063f7fe700 1 --2- 172.21.15.7:0/3548333162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621b3f72830 0x5621b3f67130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.375+0000 7f063f7fe700 1 -- 172.21.15.7:0/3548333162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b3f71f10 msgr2=0x5621b3f66bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:28.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.375+0000 7f063f7fe700 1 --2- 172.21.15.7:0/3548333162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b3f71f10 0x5621b3f66bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.375+0000 7f063f7fe700 1 -- 172.21.15.7:0/3548333162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621b3f8ef90 con 0x5621b3f6c750 2024-06-10T01:05:28.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.375+0000 7f063f7fe700 1 --2- 172.21.15.7:0/3548333162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621b3f6c750 0x5621b3f666b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f06300ccc00 tx=0x7f063009ba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315b83f598fb7b64 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:28.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.375+0000 7f061effd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:05:28.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.376+0000 7f061effd700 1 -- 172.21.15.7:0/3548333162 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06300b8070 con 0x5621b3f6c750 2024-06-10T01:05:28.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.376+0000 7f061effd700 1 -- 172.21.15.7:0/3548333162 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f063000fa30 con 0x5621b3f6c750 2024-06-10T01:05:28.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.376+0000 7f0650633b80 1 -- 172.21.15.7:0/3548333162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06300999b0 con 0x5621b3f6c750 2024-06-10T01:05:28.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.376+0000 7f061effd700 1 -- 172.21.15.7:0/3548333162 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f063000f740 con 0x5621b3f6c750 2024-06-10T01:05:28.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.376+0000 7f0650633b80 10 client.15162 fetch_fsmap learned FSMap version 26 2024-06-10T01:05:28.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.376+0000 7f0650633b80 1 -- 172.21.15.7:0/3548333162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5621b3ff1b80 con 0x5621b3f6c750 2024-06-10T01:05:28.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.376+0000 7f061effd700 1 -- 172.21.15.7:0/3548333162 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06300999b0 con 0x5621b3f6c750 2024-06-10T01:05:28.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.376+0000 7f061effd700 1 -- 172.21.15.7:0/3548333162 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f06300a40f0 con 0x5621b3f6c750 2024-06-10T01:05:28.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.376+0000 7f0650633b80 10 client.15162 fetch_fsmap finished waiting for FSMap version 26 2024-06-10T01:05:28.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.376+0000 7f0650633b80 10 client.15162 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-10T01:05:28.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.376+0000 7f0650633b80 20 client.15162 populate_metadata read hostname 'smithi007' 2024-06-10T01:05:28.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.376+0000 7f0650633b80 1 --2- 172.21.15.7:0/3548333162 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5621b3ff5e90 0x5621b3ff4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:28.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.376+0000 7f0650633b80 4 client.15162 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T01:05:28.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.376+0000 7f0650633b80 1 -- 172.21.15.7:0/3548333162 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "status"}) -- 0x5621b3ec5280 con 0x5621b3ff5e90 2024-06-10T01:05:28.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.377+0000 7f063effd700 1 --2- 172.21.15.7:0/3548333162 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5621b3ff5e90 0x5621b3ff4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:28.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.377+0000 7f061effd700 10 client.15162 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:05:28.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.377+0000 7f063effd700 1 --2- 172.21.15.7:0/3548333162 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5621b3ff5e90 0x5621b3ff4ba0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=80b664e40a5c257a server_cookie=ab52a626d0ec778d in_seq=0 out_seq=0 2024-06-10T01:05:28.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.377+0000 7f061effd700 1 -- 172.21.15.7:0/3548333162 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5621b3ec5280 con 0x5621b3ff5e90 2024-06-10T01:05:28.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.377+0000 7f061effd700 10 client.15162 handle_command_reply: tid=0 2024-06-10T01:05:28.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.377+0000 7f0650633b80 1 client.15162 shutdown 2024-06-10T01:05:28.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.378+0000 7f0650633b80 1 -- 172.21.15.7:0/3548333162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621b3f6c750 msgr2=0x5621b3f666b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:28.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.378+0000 7f0650633b80 1 --2- 172.21.15.7:0/3548333162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621b3f6c750 0x5621b3f666b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f06300ccc00 tx=0x7f063009ba00 comp rx=0 tx=0).stop 2024-06-10T01:05:28.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.378+0000 7f0650633b80 1 -- 172.21.15.7:0/3548333162 shutdown_connections 2024-06-10T01:05:28.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.378+0000 7f0650633b80 1 --2- 172.21.15.7:0/3548333162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621b3f72830 0x5621b3f67130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.378+0000 7f0650633b80 1 --2- 172.21.15.7:0/3548333162 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5621b3ff5e90 0x5621b3ff4ba0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.378+0000 7f0650633b80 1 --2- 172.21.15.7:0/3548333162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621b3f6c750 0x5621b3f666b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.378+0000 7f0650633b80 1 --2- 172.21.15.7:0/3548333162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b3f71f10 0x5621b3f66bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.378+0000 7f0650633b80 1 -- 172.21.15.7:0/3548333162 >> 172.21.15.7:0/3548333162 conn(0x5621b3e99030 msgr2=0x5621b3f72c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:28.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.378+0000 7f061effd700 0 client.15162 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:05:28.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.378+0000 7f0650633b80 1 -- 172.21.15.7:0/3548333162 shutdown_connections 2024-06-10T01:05:28.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.378+0000 7f0650633b80 1 -- 172.21.15.7:0/3548333162 wait complete. 2024-06-10T01:05:28.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.378+0000 7f0650633b80 20 client.15162 trim_cache size 0 max 16384 2024-06-10T01:05:28.393 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:05:28.393 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "bded58f8-26c3-11ef-bc9b-c7b262605968", 2024-06-10T01:05:28.393 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 2, 2024-06-10T01:05:28.393 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24434, 2024-06-10T01:05:28.393 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-06-10T01:05:28.394 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-06-10T01:05:28.394 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-06-10T01:05:28.394 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 154.675943612, 2024-06-10T01:05:28.394 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 26, 2024-06-10T01:05:28.394 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 99, 2024-06-10T01:05:28.394 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 97, 2024-06-10T01:05:28.395 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 234.61149828999999 2024-06-10T01:05:28.395 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:05:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.378+0000 7f0645db0700 1 -- 172.21.15.7:0/274356836 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f06280a2c20 msgr2=0x7f06280a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.378+0000 7f0645db0700 1 --2- 172.21.15.7:0/274356836 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f06280a2c20 0x7f06280a50e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0634006010 tx=0x7f063400b540 comp rx=0 tx=0).stop 2024-06-10T01:05:28.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.378+0000 7f0645db0700 1 -- 172.21.15.7:0/274356836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06400746c0 msgr2=0x7f064019ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:28.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.378+0000 7f0645db0700 1 --2- 172.21.15.7:0/274356836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06400746c0 0x7f064019ad70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f063000c9b0 tx=0x7f063000cd70 comp rx=0 tx=0).stop 2024-06-10T01:05:28.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.379+0000 7f0645db0700 1 -- 172.21.15.7:0/274356836 shutdown_connections 2024-06-10T01:05:28.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.379+0000 7f0645db0700 1 --2- 172.21.15.7:0/274356836 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f06280a2c20 0x7f06280a50e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.379+0000 7f0645db0700 1 --2- 172.21.15.7:0/274356836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f064010ee90 0x7f064019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.379+0000 7f0645db0700 1 --2- 172.21.15.7:0/274356836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06400746c0 0x7f064019ad70 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.379+0000 7f0645db0700 1 --2- 172.21.15.7:0/274356836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0640074fe0 0x7f064019b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:28.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.379+0000 7f0645db0700 1 -- 172.21.15.7:0/274356836 >> 172.21.15.7:0/274356836 conn(0x7f06400795f0 msgr2=0x7f06400ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:28.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.379+0000 7f0645db0700 1 -- 172.21.15.7:0/274356836 shutdown_connections 2024-06-10T01:05:28.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:28.379+0000 7f0645db0700 1 -- 172.21.15.7:0/274356836 wait complete. 2024-06-10T01:05:29.026 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.c status 2024-06-10T01:05:29.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:29 smithi007 ceph-mon[32398]: pgmap v415: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:05:29.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:29 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:05:29.299 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:05:29.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:29 smithi175 ceph-mon[36636]: pgmap v415: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:05:29.455 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:29 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:05:29.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:29 smithi078 ceph-mon[36683]: pgmap v415: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:05:29.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:29 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:05:30.749 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:30 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:05:30.749 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:30 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:05:30.749 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:30 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:05:30.749 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:30 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:05:30.749 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:30 smithi007 ceph-mon[32398]: pgmap v416: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:05:30.749 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:30 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:05:30.749 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:30 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:05:30.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:30 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:05:30.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:30 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:05:30.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:30 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:05:30.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:30 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:05:30.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:30 smithi078 ceph-mon[36683]: pgmap v416: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:05:30.832 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:30 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:05:30.832 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:30 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:05:30.954 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:30 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:05:30.955 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:30 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:05:30.955 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:30 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:05:30.955 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:30 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:05:30.955 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:30 smithi175 ceph-mon[36636]: pgmap v416: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:05:30.955 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:30 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:05:30.955 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:30 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:05:31.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.005+0000 7f16f528b700 1 -- 172.21.15.7:0/1563496724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16f010a6a0 msgr2=0x7f16f010aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:31.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.005+0000 7f16f528b700 1 --2- 172.21.15.7:0/1563496724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16f010a6a0 0x7f16f010aa80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f16e0009990 tx=0x7f16e0009ca0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.005+0000 7f16f528b700 1 -- 172.21.15.7:0/1563496724 shutdown_connections 2024-06-10T01:05:31.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.005+0000 7f16f528b700 1 --2- 172.21.15.7:0/1563496724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16f0105060 0x7f16f0111570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.005+0000 7f16f528b700 1 --2- 172.21.15.7:0/1563496724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16f010a6a0 0x7f16f010aa80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.005+0000 7f16f528b700 1 --2- 172.21.15.7:0/1563496724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16f01046a0 0x7f16f0104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.005+0000 7f16f528b700 1 -- 172.21.15.7:0/1563496724 >> 172.21.15.7:0/1563496724 conn(0x7f16f0100190 msgr2=0x7f16f01025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:31.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.005+0000 7f16f528b700 1 -- 172.21.15.7:0/1563496724 shutdown_connections 2024-06-10T01:05:31.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.005+0000 7f16f528b700 1 -- 172.21.15.7:0/1563496724 wait complete. 2024-06-10T01:05:31.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.006+0000 7f16f528b700 1 Processor -- start 2024-06-10T01:05:31.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.006+0000 7f16f528b700 1 -- start start 2024-06-10T01:05:31.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.006+0000 7f16f528b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16f01046a0 0x7f16f01113c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:31.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.006+0000 7f16f528b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16f0105060 0x7f16f010e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:31.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.006+0000 7f16f528b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16f010a6a0 0x7f16f010ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:31.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.006+0000 7f16f528b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f16f01149f0 con 0x7f16f01046a0 2024-06-10T01:05:31.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.006+0000 7f16f528b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16f0114870 con 0x7f16f010a6a0 2024-06-10T01:05:31.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.006+0000 7f16f528b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f16f0114b70 con 0x7f16f0105060 2024-06-10T01:05:31.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.006+0000 7f16eeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16f01046a0 0x7f16f01113c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:31.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.006+0000 7f16eeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16f01046a0 0x7f16f01113c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39172/0 (socket says 172.21.15.7:39172) 2024-06-10T01:05:31.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.006+0000 7f16eeffd700 1 -- 172.21.15.7:0/141797820 learned_addr learned my addr 172.21.15.7:0/141797820 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:31.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.007+0000 7f16ee7fc700 1 --2- 172.21.15.7:0/141797820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16f0105060 0x7f16f010e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:31.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.007+0000 7f16ef7fe700 1 --2- 172.21.15.7:0/141797820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16f010a6a0 0x7f16f010ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:31.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.007+0000 7f16eeffd700 1 -- 172.21.15.7:0/141797820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16f0105060 msgr2=0x7f16f010e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:31.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.007+0000 7f16eeffd700 1 --2- 172.21.15.7:0/141797820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16f0105060 0x7f16f010e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.007+0000 7f16eeffd700 1 -- 172.21.15.7:0/141797820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16f010a6a0 msgr2=0x7f16f010ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:31.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.007+0000 7f16eeffd700 1 --2- 172.21.15.7:0/141797820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16f010a6a0 0x7f16f010ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.007+0000 7f16eeffd700 1 -- 172.21.15.7:0/141797820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f01ab6f0 con 0x7f16f01046a0 2024-06-10T01:05:31.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.007+0000 7f16eeffd700 1 --2- 172.21.15.7:0/141797820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16f01046a0 0x7f16f01113c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f16e0000c00 tx=0x7f16e00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:31.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.007+0000 7f16d7fff700 1 -- 172.21.15.7:0/141797820 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0027070 con 0x7f16f01046a0 2024-06-10T01:05:31.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.007+0000 7f16f528b700 1 -- 172.21.15.7:0/141797820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16f01ab8c0 con 0x7f16f01046a0 2024-06-10T01:05:31.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.007+0000 7f16d7fff700 1 -- 172.21.15.7:0/141797820 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16e0003e40 con 0x7f16f01046a0 2024-06-10T01:05:31.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.007+0000 7f16d7fff700 1 -- 172.21.15.7:0/141797820 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0022d10 con 0x7f16f01046a0 2024-06-10T01:05:31.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.007+0000 7f16f528b700 1 -- 172.21.15.7:0/141797820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16f01abcb0 con 0x7f16f01046a0 2024-06-10T01:05:31.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.010+0000 7f16d7fff700 1 -- 172.21.15.7:0/141797820 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f16e002a070 con 0x7f16f01046a0 2024-06-10T01:05:31.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.011+0000 7f16d7fff700 1 --2- 172.21.15.7:0/141797820 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f16d80a2ca0 0x7f16d80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:31.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.011+0000 7f16ee7fc700 1 --2- 172.21.15.7:0/141797820 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f16d80a2ca0 0x7f16d80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:31.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.011+0000 7f16d7fff700 1 -- 172.21.15.7:0/141797820 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f16e00d0180 con 0x7f16f01046a0 2024-06-10T01:05:31.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.011+0000 7f16ee7fc700 1 --2- 172.21.15.7:0/141797820 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f16d80a2ca0 0x7f16d80a5160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f16f01099a0 tx=0x7f16e400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:31.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.012+0000 7f16ffb0eb80 1 Processor -- start 2024-06-10T01:05:31.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.012+0000 7f16ffb0eb80 1 -- start start 2024-06-10T01:05:31.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.012+0000 7f16ffb0eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b52015df10 0x55b52015e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:31.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.012+0000 7f16ffb0eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b52015e830 0x55b520158210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:31.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.012+0000 7f16ffb0eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b520158750 0x55b520158b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:31.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.012+0000 7f16ffb0eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b52004af10 con 0x55b520158750 2024-06-10T01:05:31.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.012+0000 7f16ffb0eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b5200a9800 con 0x55b52015e830 2024-06-10T01:05:31.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.012+0000 7f16ffb0eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b520011410 con 0x55b52015df10 2024-06-10T01:05:31.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.012+0000 7f16ef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b520158750 0x55b520158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:31.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.012+0000 7f16ef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b520158750 0x55b520158b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39182/0 (socket says 172.21.15.7:39182) 2024-06-10T01:05:31.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.012+0000 7f16ef7fe700 1 -- 172.21.15.7:0/769815672 learned_addr learned my addr 172.21.15.7:0/769815672 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:31.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.012+0000 7f16eeffd700 1 --2- 172.21.15.7:0/769815672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b52015df10 0x55b52015e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:31.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.012+0000 7f16ee7fc700 1 --2- 172.21.15.7:0/769815672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b52015e830 0x55b520158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:31.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.013+0000 7f16ef7fe700 1 -- 172.21.15.7:0/769815672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b52015df10 msgr2=0x55b52015e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:31.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.013+0000 7f16ef7fe700 1 --2- 172.21.15.7:0/769815672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b52015df10 0x55b52015e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.013+0000 7f16ef7fe700 1 -- 172.21.15.7:0/769815672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b52015e830 msgr2=0x55b520158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:31.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.013+0000 7f16ef7fe700 1 --2- 172.21.15.7:0/769815672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b52015e830 0x55b520158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.013+0000 7f16ef7fe700 1 -- 172.21.15.7:0/769815672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b51fff8880 con 0x55b520158750 2024-06-10T01:05:31.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.013+0000 7f16ef7fe700 1 --2- 172.21.15.7:0/769815672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b520158750 0x55b520158b70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f16e800f3e0 tx=0x7f16e800f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=302cfafff17242b7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:31.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.013+0000 7f16d57fa700 1 -- 172.21.15.7:0/769815672 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e8018030 con 0x55b520158750 2024-06-10T01:05:31.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.013+0000 7f16d57fa700 1 -- 172.21.15.7:0/769815672 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16e8004bb0 con 0x55b520158750 2024-06-10T01:05:31.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.013+0000 7f16d57fa700 1 -- 172.21.15.7:0/769815672 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e8005020 con 0x55b520158750 2024-06-10T01:05:31.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.014+0000 7f16ffb0eb80 1 -- 172.21.15.7:0/769815672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b520158750 msgr2=0x55b520158b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:31.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.014+0000 7f16ffb0eb80 1 --2- 172.21.15.7:0/769815672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b520158750 0x55b520158b70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f16e800f3e0 tx=0x7f16e800f6f0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.014+0000 7f16ffb0eb80 1 -- 172.21.15.7:0/769815672 shutdown_connections 2024-06-10T01:05:31.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.014+0000 7f16ffb0eb80 1 --2- 172.21.15.7:0/769815672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b520158750 0x55b520158b70 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.014+0000 7f16ffb0eb80 1 --2- 172.21.15.7:0/769815672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b52015df10 0x55b52015e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.014+0000 7f16ffb0eb80 1 --2- 172.21.15.7:0/769815672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b52015e830 0x55b520158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.014+0000 7f16ffb0eb80 1 -- 172.21.15.7:0/769815672 >> 172.21.15.7:0/769815672 conn(0x55b520085030 msgr2=0x55b5201531d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:31.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.014+0000 7f16ffb0eb80 1 -- 172.21.15.7:0/769815672 shutdown_connections 2024-06-10T01:05:31.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.014+0000 7f16ffb0eb80 1 -- 172.21.15.7:0/769815672 wait complete. 2024-06-10T01:05:31.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.014+0000 7f16ffb0eb80 1 -- start start 2024-06-10T01:05:31.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.014+0000 7f16ffb0eb80 1 Processor -- start 2024-06-10T01:05:31.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.015+0000 7f16ffb0eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b520158750 0x55b5201644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:31.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.015+0000 7f16ffb0eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b52015df10 0x55b5201617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:31.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.015+0000 7f16ffb0eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b52015e830 0x55b520161d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:31.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.015+0000 7f16ffb0eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b520049f70 con 0x55b52015df10 2024-06-10T01:05:31.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.015+0000 7f16ffb0eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b5200a57c0 con 0x55b520158750 2024-06-10T01:05:31.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.015+0000 7f16ffb0eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b520165210 con 0x55b52015e830 2024-06-10T01:05:31.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.015+0000 7f16ffb0eb80 10 client.0 _ll_register_callbacks cb 0x55b52003f560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:05:31.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.015+0000 7f16ee7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b52015df10 0x55b5201617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:31.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.015+0000 7f16eeffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b520158750 0x55b5201644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:31.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.015+0000 7f16ee7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b52015df10 0x55b5201617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39186/0 (socket says 172.21.15.7:39186) 2024-06-10T01:05:31.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.015+0000 7f16ee7fc700 1 -- 172.21.15.7:0/1484832826 learned_addr learned my addr 172.21.15.7:0/1484832826 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:31.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.015+0000 7f16ef7fe700 1 --2- 172.21.15.7:0/1484832826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b52015e830 0x55b520161d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:31.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.015+0000 7f16ee7fc700 1 -- 172.21.15.7:0/1484832826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b52015e830 msgr2=0x55b520161d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:31.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.015+0000 7f16ee7fc700 1 --2- 172.21.15.7:0/1484832826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b52015e830 0x55b520161d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.015+0000 7f16ee7fc700 1 -- 172.21.15.7:0/1484832826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b520158750 msgr2=0x55b5201644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:31.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.015+0000 7f16ee7fc700 1 --2- 172.21.15.7:0/1484832826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b520158750 0x55b5201644e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.015+0000 7f16ee7fc700 1 -- 172.21.15.7:0/1484832826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b52006f700 con 0x55b52015df10 2024-06-10T01:05:31.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.015+0000 7f16eeffd700 1 --2- 172.21.15.7:0/1484832826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b520158750 0x55b5201644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:31.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.015+0000 7f16ee7fc700 1 --2- 172.21.15.7:0/1484832826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b52015df10 0x55b5201617e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f16e4003e40 tx=0x7f16e40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e183872e7e359866 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:31.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.015+0000 7f16d2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:05:31.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.016+0000 7f16d2ff5700 1 -- 172.21.15.7:0/1484832826 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e4011ce0 con 0x55b52015df10 2024-06-10T01:05:31.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.016+0000 7f16d2ff5700 1 -- 172.21.15.7:0/1484832826 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f16e4028470 con 0x55b52015df10 2024-06-10T01:05:31.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.016+0000 7f16ffb0eb80 1 -- 172.21.15.7:0/1484832826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16e8004bb0 con 0x55b52015df10 2024-06-10T01:05:31.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.016+0000 7f16d2ff5700 1 -- 172.21.15.7:0/1484832826 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f16e4003e70 con 0x55b52015df10 2024-06-10T01:05:31.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.016+0000 7f16ffb0eb80 10 client.15174 fetch_fsmap learned FSMap version 26 2024-06-10T01:05:31.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.016+0000 7f16ffb0eb80 1 -- 172.21.15.7:0/1484832826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b52016f9c0 con 0x55b52015df10 2024-06-10T01:05:31.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.016+0000 7f16d2ff5700 1 -- 172.21.15.7:0/1484832826 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16e8004bb0 con 0x55b52015df10 2024-06-10T01:05:31.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.016+0000 7f16d2ff5700 1 -- 172.21.15.7:0/1484832826 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f16e401f790 con 0x55b52015df10 2024-06-10T01:05:31.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.016+0000 7f16ffb0eb80 10 client.15174 fetch_fsmap finished waiting for FSMap version 26 2024-06-10T01:05:31.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.016+0000 7f16ffb0eb80 10 client.15174 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-10T01:05:31.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.016+0000 7f16ffb0eb80 20 client.15174 populate_metadata read hostname 'smithi007' 2024-06-10T01:05:31.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.016+0000 7f16ffb0eb80 1 --2- 172.21.15.7:0/1484832826 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55b520172010 0x55b520173480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:31.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.016+0000 7f16ffb0eb80 4 client.15174 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:05:31.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.016+0000 7f16ffb0eb80 1 -- 172.21.15.7:0/1484832826 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5200b6d00 con 0x55b520172010 2024-06-10T01:05:31.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.017+0000 7f16eeffd700 1 --2- 172.21.15.7:0/1484832826 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55b520172010 0x55b520173480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:31.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.017+0000 7f16eeffd700 1 --2- 172.21.15.7:0/1484832826 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55b520172010 0x55b520173480 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=59e52de242bd596 server_cookie=ae210d353f522ce in_seq=0 out_seq=0 2024-06-10T01:05:31.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.017+0000 7f16d2ff5700 10 client.15174 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:05:31.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.018+0000 7f16d2ff5700 1 -- 172.21.15.7:0/1484832826 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b5200b6d00 con 0x55b520172010 2024-06-10T01:05:31.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.018+0000 7f16d2ff5700 10 client.15174 handle_command_reply: tid=0 2024-06-10T01:05:31.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.018+0000 7f16ffb0eb80 1 client.15174 shutdown 2024-06-10T01:05:31.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.018+0000 7f16ffb0eb80 1 -- 172.21.15.7:0/1484832826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b52015df10 msgr2=0x55b5201617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:31.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.018+0000 7f16ffb0eb80 1 --2- 172.21.15.7:0/1484832826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b52015df10 0x55b5201617e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f16e4003e40 tx=0x7f16e40047c0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.018+0000 7f16ffb0eb80 1 -- 172.21.15.7:0/1484832826 shutdown_connections 2024-06-10T01:05:31.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.018+0000 7f16ffb0eb80 1 --2- 172.21.15.7:0/1484832826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b52015e830 0x55b520161d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.018+0000 7f16ffb0eb80 1 --2- 172.21.15.7:0/1484832826 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55b520172010 0x55b520173480 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.018+0000 7f16ffb0eb80 1 --2- 172.21.15.7:0/1484832826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b520158750 0x55b5201644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.018+0000 7f16ffb0eb80 1 --2- 172.21.15.7:0/1484832826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b52015df10 0x55b5201617e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.018+0000 7f16ffb0eb80 1 -- 172.21.15.7:0/1484832826 >> 172.21.15.7:0/1484832826 conn(0x55b520085030 msgr2=0x55b5201526b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:31.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.018+0000 7f16d2ff5700 0 client.15174 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:05:31.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.018+0000 7f16ffb0eb80 1 -- 172.21.15.7:0/1484832826 shutdown_connections 2024-06-10T01:05:31.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.018+0000 7f16ffb0eb80 1 -- 172.21.15.7:0/1484832826 wait complete. 2024-06-10T01:05:31.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.018+0000 7f16ffb0eb80 20 client.15174 trim_cache size 0 max 16384 2024-06-10T01:05:31.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.040+0000 7f16ffb0eb80 1 Processor -- start 2024-06-10T01:05:31.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.040+0000 7f16ffb0eb80 1 -- start start 2024-06-10T01:05:31.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.040+0000 7f16ffb0eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b520158750 0x55b520158210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:31.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.040+0000 7f16ffb0eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b52015df10 0x55b5201607e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:31.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.040+0000 7f16ffb0eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b52015e830 0x55b520152e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:31.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.040+0000 7f16ffb0eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b5201717c0 con 0x55b520158750 2024-06-10T01:05:31.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.040+0000 7f16ffb0eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b520171e90 con 0x55b52015e830 2024-06-10T01:05:31.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.040+0000 7f16ffb0eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b520172560 con 0x55b52015df10 2024-06-10T01:05:31.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.040+0000 7f16eeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b520158750 0x55b520158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:31.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.040+0000 7f16eeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b520158750 0x55b520158210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39200/0 (socket says 172.21.15.7:39200) 2024-06-10T01:05:31.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.040+0000 7f16eeffd700 1 -- 172.21.15.7:0/2447887388 learned_addr learned my addr 172.21.15.7:0/2447887388 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:31.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.040+0000 7f16ee7fc700 1 --2- 172.21.15.7:0/2447887388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b52015df10 0x55b5201607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:31.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.040+0000 7f16eeffd700 1 -- 172.21.15.7:0/2447887388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b52015df10 msgr2=0x55b5201607e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:31.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.040+0000 7f16eeffd700 1 --2- 172.21.15.7:0/2447887388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b52015df10 0x55b5201607e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.040+0000 7f16eeffd700 1 -- 172.21.15.7:0/2447887388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b52015e830 msgr2=0x55b520152e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:31.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.040+0000 7f16eeffd700 1 --2- 172.21.15.7:0/2447887388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b52015e830 0x55b520152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.040+0000 7f16eeffd700 1 -- 172.21.15.7:0/2447887388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b520153ff0 con 0x55b520158750 2024-06-10T01:05:31.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.041+0000 7f16eeffd700 1 --2- 172.21.15.7:0/2447887388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b520158750 0x55b520158210 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f16e009c390 tx=0x7f16e00cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb425f94f826a3d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:31.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.041+0000 7f16d4ff9700 1 -- 172.21.15.7:0/2447887388 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e00ba070 con 0x55b520158750 2024-06-10T01:05:31.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.041+0000 7f16d4ff9700 1 -- 172.21.15.7:0/2447887388 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16e00a0a40 con 0x55b520158750 2024-06-10T01:05:31.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.041+0000 7f16d4ff9700 1 -- 172.21.15.7:0/2447887388 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e00b4030 con 0x55b520158750 2024-06-10T01:05:31.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.041+0000 7f16ffb0eb80 1 -- 172.21.15.7:0/2447887388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b520158750 msgr2=0x55b520158210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:31.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.041+0000 7f16ffb0eb80 1 --2- 172.21.15.7:0/2447887388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b520158750 0x55b520158210 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f16e009c390 tx=0x7f16e00cf040 comp rx=0 tx=0).stop 2024-06-10T01:05:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.041+0000 7f16ffb0eb80 1 -- 172.21.15.7:0/2447887388 shutdown_connections 2024-06-10T01:05:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.041+0000 7f16ffb0eb80 1 --2- 172.21.15.7:0/2447887388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b52015e830 0x55b520152e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.041+0000 7f16ffb0eb80 1 --2- 172.21.15.7:0/2447887388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b520158750 0x55b520158210 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.041+0000 7f16ffb0eb80 1 --2- 172.21.15.7:0/2447887388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b52015df10 0x55b5201607e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.041+0000 7f16ffb0eb80 1 -- 172.21.15.7:0/2447887388 >> 172.21.15.7:0/2447887388 conn(0x55b520085030 msgr2=0x55b5201600d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.042+0000 7f16ffb0eb80 1 -- 172.21.15.7:0/2447887388 shutdown_connections 2024-06-10T01:05:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.042+0000 7f16ffb0eb80 1 -- 172.21.15.7:0/2447887388 wait complete. 2024-06-10T01:05:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.042+0000 7f16ffb0eb80 1 -- start start 2024-06-10T01:05:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.042+0000 7f16ffb0eb80 1 Processor -- start 2024-06-10T01:05:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.042+0000 7f16ffb0eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b520158750 0x55b52017d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.042+0000 7f16ffb0eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b52015df10 0x55b5201764e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.042+0000 7f16ffb0eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b52015e830 0x55b520176b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.042+0000 7f16ffb0eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b520173300 con 0x55b52015df10 2024-06-10T01:05:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.042+0000 7f16ffb0eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b520172c30 con 0x55b520158750 2024-06-10T01:05:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.042+0000 7f16ffb0eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b5201739d0 con 0x55b52015e830 2024-06-10T01:05:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.042+0000 7f16ffb0eb80 10 client.0 _ll_register_callbacks cb 0x55b52003f560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:05:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.042+0000 7f16ee7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b52015df10 0x55b5201764e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.042+0000 7f16ee7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b52015df10 0x55b5201764e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39206/0 (socket says 172.21.15.7:39206) 2024-06-10T01:05:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.042+0000 7f16ee7fc700 1 -- 172.21.15.7:0/731027447 learned_addr learned my addr 172.21.15.7:0/731027447 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.042+0000 7f16eeffd700 1 --2- 172.21.15.7:0/731027447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b520158750 0x55b52017d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.042+0000 7f16ef7fe700 1 --2- 172.21.15.7:0/731027447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b52015e830 0x55b520176b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.043+0000 7f16ee7fc700 1 -- 172.21.15.7:0/731027447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b52015e830 msgr2=0x55b520176b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.043+0000 7f16ee7fc700 1 --2- 172.21.15.7:0/731027447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b52015e830 0x55b520176b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.043+0000 7f16ee7fc700 1 -- 172.21.15.7:0/731027447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b520158750 msgr2=0x55b52017d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.043+0000 7f16ee7fc700 1 --2- 172.21.15.7:0/731027447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b520158750 0x55b52017d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.043+0000 7f16ee7fc700 1 -- 172.21.15.7:0/731027447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5201772f0 con 0x55b52015df10 2024-06-10T01:05:31.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.043+0000 7f16ee7fc700 1 --2- 172.21.15.7:0/731027447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b52015df10 0x55b5201764e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x55b520154bd0 tx=0x7f16e40038f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2dcdf559a240376 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:31.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.043+0000 7f16d2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:05:31.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.043+0000 7f16d2ff5700 1 -- 172.21.15.7:0/731027447 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e4039070 con 0x55b52015df10 2024-06-10T01:05:31.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.043+0000 7f16d2ff5700 1 -- 172.21.15.7:0/731027447 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f16e401f840 con 0x55b52015df10 2024-06-10T01:05:31.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.043+0000 7f16ffb0eb80 1 -- 172.21.15.7:0/731027447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16e00a0a40 con 0x55b52015df10 2024-06-10T01:05:31.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.043+0000 7f16d2ff5700 1 -- 172.21.15.7:0/731027447 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f16e4044050 con 0x55b52015df10 2024-06-10T01:05:31.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.043+0000 7f16ffb0eb80 10 client.15180 fetch_fsmap learned FSMap version 26 2024-06-10T01:05:31.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.043+0000 7f16ffb0eb80 1 -- 172.21.15.7:0/731027447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b5201dd900 con 0x55b52015df10 2024-06-10T01:05:31.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.043+0000 7f16d2ff5700 1 -- 172.21.15.7:0/731027447 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16e00a0a40 con 0x55b52015df10 2024-06-10T01:05:31.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.044+0000 7f16d2ff5700 1 -- 172.21.15.7:0/731027447 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f16e4010600 con 0x55b52015df10 2024-06-10T01:05:31.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.044+0000 7f16ffb0eb80 10 client.15180 fetch_fsmap finished waiting for FSMap version 26 2024-06-10T01:05:31.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.044+0000 7f16ffb0eb80 10 client.15180 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-10T01:05:31.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.044+0000 7f16ffb0eb80 20 client.15180 populate_metadata read hostname 'smithi007' 2024-06-10T01:05:31.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.044+0000 7f16ffb0eb80 1 --2- 172.21.15.7:0/731027447 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55b5201e2480 0x55b5201e1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:31.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.044+0000 7f16ffb0eb80 4 client.15180 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T01:05:31.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.044+0000 7f16ffb0eb80 1 -- 172.21.15.7:0/731027447 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "status"}) -- 0x55b5200b1280 con 0x55b5201e2480 2024-06-10T01:05:31.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.044+0000 7f16eeffd700 1 --2- 172.21.15.7:0/731027447 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55b5201e2480 0x55b5201e1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:31.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.044+0000 7f16eeffd700 1 --2- 172.21.15.7:0/731027447 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55b5201e2480 0x55b5201e1190 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7caec09df7164fcf server_cookie=3c5807cafd6fe2e0 in_seq=0 out_seq=0 2024-06-10T01:05:31.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.044+0000 7f16d2ff5700 10 client.15180 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:05:31.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.044+0000 7f16d2ff5700 1 -- 172.21.15.7:0/731027447 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b5200b1280 con 0x55b5201e2480 2024-06-10T01:05:31.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.044+0000 7f16d2ff5700 10 client.15180 handle_command_reply: tid=0 2024-06-10T01:05:31.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.044+0000 7f16ffb0eb80 1 client.15180 shutdown 2024-06-10T01:05:31.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.045+0000 7f16ffb0eb80 1 -- 172.21.15.7:0/731027447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b52015df10 msgr2=0x55b5201764e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:31.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.045+0000 7f16ffb0eb80 1 --2- 172.21.15.7:0/731027447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b52015df10 0x55b5201764e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x55b520154bd0 tx=0x7f16e40038f0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.045+0000 7f16ffb0eb80 1 -- 172.21.15.7:0/731027447 shutdown_connections 2024-06-10T01:05:31.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.045+0000 7f16ffb0eb80 1 --2- 172.21.15.7:0/731027447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b52015e830 0x55b520176b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.045+0000 7f16ffb0eb80 1 --2- 172.21.15.7:0/731027447 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55b5201e2480 0x55b5201e1190 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.045+0000 7f16ffb0eb80 1 --2- 172.21.15.7:0/731027447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b520158750 0x55b52017d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.045+0000 7f16ffb0eb80 1 --2- 172.21.15.7:0/731027447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b52015df10 0x55b5201764e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.045+0000 7f16ffb0eb80 1 -- 172.21.15.7:0/731027447 >> 172.21.15.7:0/731027447 conn(0x55b520085030 msgr2=0x55b5201638a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:31.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.045+0000 7f16d2ff5700 0 client.15180 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:05:31.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.045+0000 7f16ffb0eb80 1 -- 172.21.15.7:0/731027447 shutdown_connections 2024-06-10T01:05:31.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.045+0000 7f16ffb0eb80 1 -- 172.21.15.7:0/731027447 wait complete. 2024-06-10T01:05:31.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.045+0000 7f16ffb0eb80 20 client.15180 trim_cache size 0 max 16384 2024-06-10T01:05:31.047 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:05:31.047 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "bded58f8-26c3-11ef-bc9b-c7b262605968", 2024-06-10T01:05:31.047 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 1, 2024-06-10T01:05:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24436, 2024-06-10T01:05:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-06-10T01:05:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-06-10T01:05:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-06-10T01:05:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 158.35837611900001, 2024-06-10T01:05:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 26, 2024-06-10T01:05:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 99, 2024-06-10T01:05:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 97, 2024-06-10T01:05:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 254.65817150399999 2024-06-10T01:05:31.048 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:05:31.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.045+0000 7f16f528b700 1 -- 172.21.15.7:0/141797820 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f16d80a2ca0 msgr2=0x7f16d80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:31.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.045+0000 7f16f528b700 1 --2- 172.21.15.7:0/141797820 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f16d80a2ca0 0x7f16d80a5160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f16f01099a0 tx=0x7f16e400b3c0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.045+0000 7f16f528b700 1 -- 172.21.15.7:0/141797820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16f01046a0 msgr2=0x7f16f01113c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:31.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.045+0000 7f16f528b700 1 --2- 172.21.15.7:0/141797820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16f01046a0 0x7f16f01113c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f16e0000c00 tx=0x7f16e00047b0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.045+0000 7f16f528b700 1 -- 172.21.15.7:0/141797820 shutdown_connections 2024-06-10T01:05:31.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.045+0000 7f16f528b700 1 --2- 172.21.15.7:0/141797820 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f16d80a2ca0 0x7f16d80a5160 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.045+0000 7f16f528b700 1 --2- 172.21.15.7:0/141797820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16f010a6a0 0x7f16f010ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.045+0000 7f16f528b700 1 --2- 172.21.15.7:0/141797820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16f01046a0 0x7f16f01113c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.045+0000 7f16f528b700 1 --2- 172.21.15.7:0/141797820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16f0105060 0x7f16f010e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:31.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.045+0000 7f16f528b700 1 -- 172.21.15.7:0/141797820 >> 172.21.15.7:0/141797820 conn(0x7f16f0100190 msgr2=0x7f16f0100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:31.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.046+0000 7f16f528b700 1 -- 172.21.15.7:0/141797820 shutdown_connections 2024-06-10T01:05:31.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:31.046+0000 7f16f528b700 1 -- 172.21.15.7:0/141797820 wait complete. 2024-06-10T01:05:31.701 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:31 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:05:31.701 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:31 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:05:31.701 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:31 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:05:31.728 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-10T01:05:31.954 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:31 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:05:31.955 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:31 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:05:31.955 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:31 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:05:31.979 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:05:32.081 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:31 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:05:32.081 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:31 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:05:32.081 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:31 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:05:32.765 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:32 smithi007 ceph-mon[32398]: pgmap v417: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:05:32.955 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:32 smithi175 ceph-mon[36636]: pgmap v417: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:05:33.081 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:32 smithi078 ceph-mon[36683]: pgmap v417: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:05:33.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.650+0000 7ff5f6e44700 1 -- 172.21.15.7:0/317041240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5f0108500 msgr2=0x7ff5f01088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:33.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.650+0000 7ff5f6e44700 1 --2- 172.21.15.7:0/317041240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5f0108500 0x7ff5f01088e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4009990 tx=0x7ff5e4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.650+0000 7ff5f6e44700 1 -- 172.21.15.7:0/317041240 shutdown_connections 2024-06-10T01:05:33.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.650+0000 7ff5f6e44700 1 --2- 172.21.15.7:0/317041240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5f006ab30 0x7ff5f010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.650+0000 7ff5f6e44700 1 --2- 172.21.15.7:0/317041240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5f0108500 0x7ff5f01088e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.650+0000 7ff5f6e44700 1 --2- 172.21.15.7:0/317041240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5f006a170 0x7ff5f006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.650+0000 7ff5f6e44700 1 -- 172.21.15.7:0/317041240 >> 172.21.15.7:0/317041240 conn(0x7ff5f0078340 msgr2=0x7ff5f0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:33.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.650+0000 7ff5f6e44700 1 -- 172.21.15.7:0/317041240 shutdown_connections 2024-06-10T01:05:33.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.650+0000 7ff5f6e44700 1 -- 172.21.15.7:0/317041240 wait complete. 2024-06-10T01:05:33.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.651+0000 7ff5f6e44700 1 Processor -- start 2024-06-10T01:05:33.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.651+0000 7ff5f6e44700 1 -- start start 2024-06-10T01:05:33.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.651+0000 7ff5f6e44700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5f006a170 0x7ff5f019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:33.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.651+0000 7ff5f6e44700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5f006ab30 0x7ff5f019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:33.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.651+0000 7ff5f6e44700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5f0108500 0x7ff5f0198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:33.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.651+0000 7ff5f6e44700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f0112780 con 0x7ff5f006a170 2024-06-10T01:05:33.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.651+0000 7ff5f6e44700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f0112600 con 0x7ff5f0108500 2024-06-10T01:05:33.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.651+0000 7ff5f6e44700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f0112900 con 0x7ff5f006ab30 2024-06-10T01:05:33.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.651+0000 7ff5f4be0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5f006a170 0x7ff5f019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:33.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.651+0000 7ff5f4be0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5f006a170 0x7ff5f019e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44750/0 (socket says 172.21.15.7:44750) 2024-06-10T01:05:33.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.651+0000 7ff5f4be0700 1 -- 172.21.15.7:0/3839920266 learned_addr learned my addr 172.21.15.7:0/3839920266 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:33.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.652+0000 7ff5effff700 1 --2- 172.21.15.7:0/3839920266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5f006ab30 0x7ff5f019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:33.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.652+0000 7ff5f53e1700 1 --2- 172.21.15.7:0/3839920266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5f0108500 0x7ff5f0198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.652+0000 7ff5f4be0700 1 -- 172.21.15.7:0/3839920266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5f006ab30 msgr2=0x7ff5f019ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.652+0000 7ff5f4be0700 1 --2- 172.21.15.7:0/3839920266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5f006ab30 0x7ff5f019ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.652+0000 7ff5f4be0700 1 -- 172.21.15.7:0/3839920266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5f0108500 msgr2=0x7ff5f0198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.652+0000 7ff5f4be0700 1 --2- 172.21.15.7:0/3839920266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5f0108500 0x7ff5f0198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.652+0000 7ff5f4be0700 1 -- 172.21.15.7:0/3839920266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f01991c0 con 0x7ff5f006a170 2024-06-10T01:05:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.652+0000 7ff5f53e1700 1 --2- 172.21.15.7:0/3839920266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5f0108500 0x7ff5f0198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:05:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.652+0000 7ff5f4be0700 1 --2- 172.21.15.7:0/3839920266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5f006a170 0x7ff5f019e8a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4000c00 tx=0x7ff5e40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.652+0000 7ff5edffb700 1 -- 172.21.15.7:0/3839920266 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4027070 con 0x7ff5f006a170 2024-06-10T01:05:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.652+0000 7ff5edffb700 1 -- 172.21.15.7:0/3839920266 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5e4003780 con 0x7ff5f006a170 2024-06-10T01:05:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.652+0000 7ff5f6e44700 1 -- 172.21.15.7:0/3839920266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5f0199450 con 0x7ff5f006a170 2024-06-10T01:05:33.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.652+0000 7ff5edffb700 1 -- 172.21.15.7:0/3839920266 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4022dd0 con 0x7ff5f006a170 2024-06-10T01:05:33.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.653+0000 7ff5f6e44700 1 -- 172.21.15.7:0/3839920266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5f01a5770 con 0x7ff5f006a170 2024-06-10T01:05:33.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.655+0000 7ff5edffb700 1 -- 172.21.15.7:0/3839920266 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7ff5e402a070 con 0x7ff5f006a170 2024-06-10T01:05:33.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.655+0000 7ff5edffb700 1 --2- 172.21.15.7:0/3839920266 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff5e00a2c20 0x7ff5e00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:33.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.656+0000 7ff5effff700 1 --2- 172.21.15.7:0/3839920266 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff5e00a2c20 0x7ff5e00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:33.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.656+0000 7ff5edffb700 1 -- 172.21.15.7:0/3839920266 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7ff5e40d00a0 con 0x7ff5f006a170 2024-06-10T01:05:33.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.656+0000 7ff5effff700 1 --2- 172.21.15.7:0/3839920266 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff5e00a2c20 0x7ff5e00a50e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0078a20 tx=0x7ff5dc00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:33.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.656+0000 7ff6016c7b80 1 Processor -- start 2024-06-10T01:05:33.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.656+0000 7ff6016c7b80 1 -- start start 2024-06-10T01:05:33.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.657+0000 7ff6016c7b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a95a61f10 0x555a95a622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:33.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.657+0000 7ff6016c7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a95a62830 0x555a95a5c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:33.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.657+0000 7ff6016c7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a95a5c750 0x555a95a5cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:33.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.657+0000 7ff6016c7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555a9594ef10 con 0x555a95a62830 2024-06-10T01:05:33.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.657+0000 7ff6016c7b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555a959ad800 con 0x555a95a61f10 2024-06-10T01:05:33.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.657+0000 7ff6016c7b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555a95915410 con 0x555a95a5c750 2024-06-10T01:05:33.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.657+0000 7ff5f4be0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a95a61f10 0x555a95a622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:33.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.657+0000 7ff5f53e1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a95a5c750 0x555a95a5cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:33.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.657+0000 7ff5f4be0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a95a61f10 0x555a95a622f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:58634/0 (socket says 172.21.15.7:58634) 2024-06-10T01:05:33.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.657+0000 7ff5f4be0700 1 -- 172.21.15.7:0/2371839787 learned_addr learned my addr 172.21.15.7:0/2371839787 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:33.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.657+0000 7ff5effff700 1 --2- 172.21.15.7:0/2371839787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a95a62830 0x555a95a5c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:33.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.657+0000 7ff5f53e1700 1 -- 172.21.15.7:0/2371839787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a95a61f10 msgr2=0x555a95a622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:33.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.657+0000 7ff5f53e1700 1 --2- 172.21.15.7:0/2371839787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a95a61f10 0x555a95a622f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.657+0000 7ff5f53e1700 1 -- 172.21.15.7:0/2371839787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a95a62830 msgr2=0x555a95a5c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:33.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.657+0000 7ff5f53e1700 1 --2- 172.21.15.7:0/2371839787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a95a62830 0x555a95a5c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.657+0000 7ff5f53e1700 1 -- 172.21.15.7:0/2371839787 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a958fc880 con 0x555a95a5c750 2024-06-10T01:05:33.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.657+0000 7ff5f4be0700 1 --2- 172.21.15.7:0/2371839787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a95a61f10 0x555a95a622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:33.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.657+0000 7ff5effff700 1 --2- 172.21.15.7:0/2371839787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a95a62830 0x555a95a5c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:05:33.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.658+0000 7ff5f53e1700 1 --2- 172.21.15.7:0/2371839787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a95a5c750 0x555a95a5cb70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8009d10 tx=0x7ff5e80077e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8818dc3a0fe3ade9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:33.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.658+0000 7ff5daffd700 1 -- 172.21.15.7:0/2371839787 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8009e60 con 0x555a95a5c750 2024-06-10T01:05:33.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.658+0000 7ff5daffd700 1 -- 172.21.15.7:0/2371839787 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5e8019070 con 0x555a95a5c750 2024-06-10T01:05:33.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.658+0000 7ff5daffd700 1 -- 172.21.15.7:0/2371839787 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8013940 con 0x555a95a5c750 2024-06-10T01:05:33.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.659+0000 7ff6016c7b80 1 -- 172.21.15.7:0/2371839787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a95a5c750 msgr2=0x555a95a5cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:33.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.659+0000 7ff6016c7b80 1 --2- 172.21.15.7:0/2371839787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a95a5c750 0x555a95a5cb70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8009d10 tx=0x7ff5e80077e0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.659+0000 7ff6016c7b80 1 -- 172.21.15.7:0/2371839787 shutdown_connections 2024-06-10T01:05:33.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.659+0000 7ff6016c7b80 1 --2- 172.21.15.7:0/2371839787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a95a5c750 0x555a95a5cb70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.659+0000 7ff6016c7b80 1 --2- 172.21.15.7:0/2371839787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a95a61f10 0x555a95a622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.659+0000 7ff6016c7b80 1 --2- 172.21.15.7:0/2371839787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a95a62830 0x555a95a5c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.659+0000 7ff6016c7b80 1 -- 172.21.15.7:0/2371839787 >> 172.21.15.7:0/2371839787 conn(0x555a95989030 msgr2=0x555a95a571d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:33.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.659+0000 7ff6016c7b80 1 -- 172.21.15.7:0/2371839787 shutdown_connections 2024-06-10T01:05:33.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.659+0000 7ff6016c7b80 1 -- 172.21.15.7:0/2371839787 wait complete. 2024-06-10T01:05:33.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.659+0000 7ff6016c7b80 1 -- start start 2024-06-10T01:05:33.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.659+0000 7ff6016c7b80 1 Processor -- start 2024-06-10T01:05:33.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.660+0000 7ff6016c7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a95a5c750 0x555a95a68520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:33.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.660+0000 7ff6016c7b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a95a61f10 0x555a95a65870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:33.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.660+0000 7ff6016c7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a95a62830 0x555a95a65db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:33.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.660+0000 7ff6016c7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555a9594df70 con 0x555a95a5c750 2024-06-10T01:05:33.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.660+0000 7ff6016c7b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555a959a97c0 con 0x555a95a61f10 2024-06-10T01:05:33.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.660+0000 7ff6016c7b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555a95a69210 con 0x555a95a62830 2024-06-10T01:05:33.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.660+0000 7ff6016c7b80 10 client.0 _ll_register_callbacks cb 0x555a95943560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:05:33.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.660+0000 7ff5f4be0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a95a5c750 0x555a95a68520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:33.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.660+0000 7ff5f4be0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a95a5c750 0x555a95a68520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44770/0 (socket says 172.21.15.7:44770) 2024-06-10T01:05:33.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.660+0000 7ff5f4be0700 1 -- 172.21.15.7:0/4165953656 learned_addr learned my addr 172.21.15.7:0/4165953656 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:33.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.660+0000 7ff5f53e1700 1 --2- 172.21.15.7:0/4165953656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a95a62830 0x555a95a65db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:33.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.660+0000 7ff5effff700 1 --2- 172.21.15.7:0/4165953656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a95a61f10 0x555a95a65870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:33.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.660+0000 7ff5f4be0700 1 -- 172.21.15.7:0/4165953656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a95a62830 msgr2=0x555a95a65db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:33.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.660+0000 7ff5f4be0700 1 --2- 172.21.15.7:0/4165953656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a95a62830 0x555a95a65db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.660+0000 7ff5f4be0700 1 -- 172.21.15.7:0/4165953656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a95a61f10 msgr2=0x555a95a65870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:33.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.660+0000 7ff5f4be0700 1 --2- 172.21.15.7:0/4165953656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a95a61f10 0x555a95a65870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.660+0000 7ff5f4be0700 1 -- 172.21.15.7:0/4165953656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a95973700 con 0x555a95a5c750 2024-06-10T01:05:33.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.660+0000 7ff5effff700 1 --2- 172.21.15.7:0/4165953656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a95a61f10 0x555a95a65870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:05:33.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.660+0000 7ff5f4be0700 1 --2- 172.21.15.7:0/4165953656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a95a5c750 0x555a95a68520 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4006010 tx=0x7ff5e4022750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60148f324b626b6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:33.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.660+0000 7ff5cffff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:05:33.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.661+0000 7ff5cffff700 1 -- 172.21.15.7:0/4165953656 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e409cb00 con 0x555a95a5c750 2024-06-10T01:05:33.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.661+0000 7ff6016c7b80 1 -- 172.21.15.7:0/4165953656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5e8019070 con 0x555a95a5c750 2024-06-10T01:05:33.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.661+0000 7ff5cffff700 1 -- 172.21.15.7:0/4165953656 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7ff5e40a4df0 con 0x555a95a5c750 2024-06-10T01:05:33.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.661+0000 7ff5cffff700 1 -- 172.21.15.7:0/4165953656 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff5e40a4ff0 con 0x555a95a5c750 2024-06-10T01:05:33.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.661+0000 7ff6016c7b80 10 client.15192 fetch_fsmap learned FSMap version 26 2024-06-10T01:05:33.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.661+0000 7ff6016c7b80 1 -- 172.21.15.7:0/4165953656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555a95a73920 con 0x555a95a5c750 2024-06-10T01:05:33.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.661+0000 7ff5cffff700 1 -- 172.21.15.7:0/4165953656 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5e8019070 con 0x555a95a5c750 2024-06-10T01:05:33.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.661+0000 7ff5cffff700 1 -- 172.21.15.7:0/4165953656 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7ff5e40cdce0 con 0x555a95a5c750 2024-06-10T01:05:33.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.661+0000 7ff6016c7b80 10 client.15192 fetch_fsmap finished waiting for FSMap version 26 2024-06-10T01:05:33.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.661+0000 7ff6016c7b80 10 client.15192 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-10T01:05:33.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.661+0000 7ff6016c7b80 20 client.15192 populate_metadata read hostname 'smithi007' 2024-06-10T01:05:33.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.661+0000 7ff6016c7b80 1 --2- 172.21.15.7:0/4165953656 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555a95a75fc0 0x555a95a77430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:33.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.661+0000 7ff6016c7b80 4 client.15192 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:05:33.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.661+0000 7ff6016c7b80 1 -- 172.21.15.7:0/4165953656 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a959bad00 con 0x555a95a75fc0 2024-06-10T01:05:33.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.662+0000 7ff5effff700 1 --2- 172.21.15.7:0/4165953656 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555a95a75fc0 0x555a95a77430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:33.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.662+0000 7ff5effff700 1 --2- 172.21.15.7:0/4165953656 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555a95a75fc0 0x555a95a77430 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f367e6c287220788 server_cookie=2dd12ef94981f2eb in_seq=0 out_seq=0 2024-06-10T01:05:33.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.662+0000 7ff5cffff700 10 client.15192 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:05:33.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.663+0000 7ff5cffff700 1 -- 172.21.15.7:0/4165953656 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555a959bad00 con 0x555a95a75fc0 2024-06-10T01:05:33.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.663+0000 7ff5cffff700 10 client.15192 handle_command_reply: tid=0 2024-06-10T01:05:33.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.663+0000 7ff6016c7b80 1 client.15192 shutdown 2024-06-10T01:05:33.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.663+0000 7ff6016c7b80 1 -- 172.21.15.7:0/4165953656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a95a5c750 msgr2=0x555a95a68520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:33.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.663+0000 7ff6016c7b80 1 --2- 172.21.15.7:0/4165953656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a95a5c750 0x555a95a68520 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4006010 tx=0x7ff5e4022750 comp rx=0 tx=0).stop 2024-06-10T01:05:33.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.663+0000 7ff6016c7b80 1 -- 172.21.15.7:0/4165953656 shutdown_connections 2024-06-10T01:05:33.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.663+0000 7ff6016c7b80 1 --2- 172.21.15.7:0/4165953656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a95a62830 0x555a95a65db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.663+0000 7ff6016c7b80 1 --2- 172.21.15.7:0/4165953656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a95a5c750 0x555a95a68520 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.663+0000 7ff6016c7b80 1 --2- 172.21.15.7:0/4165953656 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555a95a75fc0 0x555a95a77430 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.663+0000 7ff6016c7b80 1 --2- 172.21.15.7:0/4165953656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a95a61f10 0x555a95a65870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.663+0000 7ff5cffff700 0 client.15192 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:05:33.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.663+0000 7ff6016c7b80 1 -- 172.21.15.7:0/4165953656 >> 172.21.15.7:0/4165953656 conn(0x555a95989030 msgr2=0x555a95a566b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:33.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.663+0000 7ff6016c7b80 1 -- 172.21.15.7:0/4165953656 shutdown_connections 2024-06-10T01:05:33.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.663+0000 7ff6016c7b80 1 -- 172.21.15.7:0/4165953656 wait complete. 2024-06-10T01:05:33.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.663+0000 7ff6016c7b80 20 client.15192 trim_cache size 0 max 16384 2024-06-10T01:05:33.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.686+0000 7ff6016c7b80 1 Processor -- start 2024-06-10T01:05:33.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.686+0000 7ff6016c7b80 1 -- start start 2024-06-10T01:05:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.686+0000 7ff6016c7b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a95a5c750 0x555a95a5c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.686+0000 7ff6016c7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a95a61f10 0x555a95a63320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.686+0000 7ff6016c7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a95a62830 0x555a95a56e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.686+0000 7ff6016c7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555a95a75770 con 0x555a95a62830 2024-06-10T01:05:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.686+0000 7ff6016c7b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555a95a75e40 con 0x555a95a5c750 2024-06-10T01:05:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.686+0000 7ff6016c7b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555a95a76510 con 0x555a95a61f10 2024-06-10T01:05:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.686+0000 7ff5f53e1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a95a62830 0x555a95a56e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.686+0000 7ff5f53e1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a95a62830 0x555a95a56e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44776/0 (socket says 172.21.15.7:44776) 2024-06-10T01:05:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.686+0000 7ff5f53e1700 1 -- 172.21.15.7:0/3058430646 learned_addr learned my addr 172.21.15.7:0/3058430646 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:33.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.686+0000 7ff5effff700 1 --2- 172.21.15.7:0/3058430646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a95a61f10 0x555a95a63320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:33.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.686+0000 7ff5f4be0700 1 --2- 172.21.15.7:0/3058430646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a95a5c750 0x555a95a5c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:33.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.686+0000 7ff5f53e1700 1 -- 172.21.15.7:0/3058430646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a95a61f10 msgr2=0x555a95a63320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:33.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.686+0000 7ff5f53e1700 1 --2- 172.21.15.7:0/3058430646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a95a61f10 0x555a95a63320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.686+0000 7ff5f53e1700 1 -- 172.21.15.7:0/3058430646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a95a5c750 msgr2=0x555a95a5c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:33.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.686+0000 7ff5f53e1700 1 --2- 172.21.15.7:0/3058430646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a95a5c750 0x555a95a5c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.686+0000 7ff5f53e1700 1 -- 172.21.15.7:0/3058430646 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a95a63de0 con 0x555a95a62830 2024-06-10T01:05:33.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.687+0000 7ff5f53e1700 1 --2- 172.21.15.7:0/3058430646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a95a62830 0x555a95a56e30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8006130 tx=0x7ff5e8009270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03628fcc6a46423 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:33.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.687+0000 7ff5da7fc700 1 -- 172.21.15.7:0/3058430646 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8009e60 con 0x555a95a62830 2024-06-10T01:05:33.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.687+0000 7ff5da7fc700 1 -- 172.21.15.7:0/3058430646 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5e8019040 con 0x555a95a62830 2024-06-10T01:05:33.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.687+0000 7ff5da7fc700 1 -- 172.21.15.7:0/3058430646 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e80046c0 con 0x555a95a62830 2024-06-10T01:05:33.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.687+0000 7ff6016c7b80 1 -- 172.21.15.7:0/3058430646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a95a62830 msgr2=0x555a95a56e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:33.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.687+0000 7ff6016c7b80 1 --2- 172.21.15.7:0/3058430646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a95a62830 0x555a95a56e30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8006130 tx=0x7ff5e8009270 comp rx=0 tx=0).stop 2024-06-10T01:05:33.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.687+0000 7ff6016c7b80 1 -- 172.21.15.7:0/3058430646 shutdown_connections 2024-06-10T01:05:33.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.687+0000 7ff6016c7b80 1 --2- 172.21.15.7:0/3058430646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a95a62830 0x555a95a56e30 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.687+0000 7ff6016c7b80 1 --2- 172.21.15.7:0/3058430646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a95a5c750 0x555a95a5c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.687+0000 7ff6016c7b80 1 --2- 172.21.15.7:0/3058430646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a95a61f10 0x555a95a63320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.687+0000 7ff6016c7b80 1 -- 172.21.15.7:0/3058430646 >> 172.21.15.7:0/3058430646 conn(0x555a95989030 msgr2=0x555a95a62c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:33.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.687+0000 7ff6016c7b80 1 -- 172.21.15.7:0/3058430646 shutdown_connections 2024-06-10T01:05:33.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.687+0000 7ff6016c7b80 1 -- 172.21.15.7:0/3058430646 wait complete. 2024-06-10T01:05:33.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.688+0000 7ff6016c7b80 1 -- start start 2024-06-10T01:05:33.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.688+0000 7ff6016c7b80 1 Processor -- start 2024-06-10T01:05:33.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.688+0000 7ff6016c7b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a95a5c750 0x555a95a7eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:33.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.688+0000 7ff6016c7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a95a61f10 0x555a95a7f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:33.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.688+0000 7ff6016c7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a95a62830 0x555a95a7fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:33.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.688+0000 7ff6016c7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555a95a772b0 con 0x555a95a61f10 2024-06-10T01:05:33.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.688+0000 7ff6016c7b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555a95a76be0 con 0x555a95a5c750 2024-06-10T01:05:33.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.688+0000 7ff6016c7b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555a95a77980 con 0x555a95a62830 2024-06-10T01:05:33.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.688+0000 7ff6016c7b80 10 client.0 _ll_register_callbacks cb 0x555a95943560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:05:33.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.688+0000 7ff5effff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a95a61f10 0x555a95a7f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:33.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.688+0000 7ff5effff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a95a61f10 0x555a95a7f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44790/0 (socket says 172.21.15.7:44790) 2024-06-10T01:05:33.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.688+0000 7ff5f4be0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a95a5c750 0x555a95a7eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:33.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.688+0000 7ff5effff700 1 -- 172.21.15.7:0/2191725267 learned_addr learned my addr 172.21.15.7:0/2191725267 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:33.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.688+0000 7ff5f53e1700 1 --2- 172.21.15.7:0/2191725267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a95a62830 0x555a95a7fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:33.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.689+0000 7ff5effff700 1 -- 172.21.15.7:0/2191725267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a95a62830 msgr2=0x555a95a7fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:33.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.689+0000 7ff5effff700 1 --2- 172.21.15.7:0/2191725267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a95a62830 0x555a95a7fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.689+0000 7ff5effff700 1 -- 172.21.15.7:0/2191725267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a95a5c750 msgr2=0x555a95a7eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:33.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.689+0000 7ff5effff700 1 --2- 172.21.15.7:0/2191725267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a95a5c750 0x555a95a7eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.689+0000 7ff5effff700 1 -- 172.21.15.7:0/2191725267 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a95a56b80 con 0x555a95a61f10 2024-06-10T01:05:33.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.689+0000 7ff5f4be0700 1 --2- 172.21.15.7:0/2191725267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a95a5c750 0x555a95a7eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:33.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.689+0000 7ff5effff700 1 --2- 172.21.15.7:0/2191725267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a95a61f10 0x555a95a7f530 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x555a95a58a80 tx=0x7ff5dc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97e07febc88ea3c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:33.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.689+0000 7ff5cffff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:05:33.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.689+0000 7ff5cffff700 1 -- 172.21.15.7:0/2191725267 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc011c00 con 0x555a95a61f10 2024-06-10T01:05:33.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.689+0000 7ff6016c7b80 1 -- 172.21.15.7:0/2191725267 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5e8019040 con 0x555a95a61f10 2024-06-10T01:05:33.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.689+0000 7ff5cffff700 1 -- 172.21.15.7:0/2191725267 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7ff5dc011da0 con 0x555a95a61f10 2024-06-10T01:05:33.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.689+0000 7ff5cffff700 1 -- 172.21.15.7:0/2191725267 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff5dc032050 con 0x555a95a61f10 2024-06-10T01:05:33.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.689+0000 7ff6016c7b80 10 client.15198 fetch_fsmap learned FSMap version 26 2024-06-10T01:05:33.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.689+0000 7ff6016c7b80 1 -- 172.21.15.7:0/2191725267 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555a95ae1dd0 con 0x555a95a61f10 2024-06-10T01:05:33.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.689+0000 7ff5cffff700 1 -- 172.21.15.7:0/2191725267 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5e8019040 con 0x555a95a61f10 2024-06-10T01:05:33.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.690+0000 7ff5cffff700 1 -- 172.21.15.7:0/2191725267 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7ff5dc0272d0 con 0x555a95a61f10 2024-06-10T01:05:33.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.690+0000 7ff6016c7b80 10 client.15198 fetch_fsmap finished waiting for FSMap version 26 2024-06-10T01:05:33.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.690+0000 7ff6016c7b80 10 client.15198 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-10T01:05:33.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.690+0000 7ff6016c7b80 20 client.15198 populate_metadata read hostname 'smithi007' 2024-06-10T01:05:33.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.690+0000 7ff6016c7b80 1 --2- 172.21.15.7:0/2191725267 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555a95ae6a20 0x555a95ae5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:33.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.690+0000 7ff6016c7b80 4 client.15198 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T01:05:33.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.690+0000 7ff6016c7b80 1 -- 172.21.15.7:0/2191725267 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "status"}) -- 0x555a959b5280 con 0x555a95ae6a20 2024-06-10T01:05:33.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.690+0000 7ff5f4be0700 1 --2- 172.21.15.7:0/2191725267 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555a95ae6a20 0x555a95ae5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:33.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.690+0000 7ff5f4be0700 1 --2- 172.21.15.7:0/2191725267 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555a95ae6a20 0x555a95ae5730 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a74682b5ef47a362 server_cookie=ad2a5968ba011085 in_seq=0 out_seq=0 2024-06-10T01:05:33.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.690+0000 7ff5cffff700 10 client.15198 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:05:33.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.691+0000 7ff5cffff700 1 -- 172.21.15.7:0/2191725267 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x555a959b5280 con 0x555a95ae6a20 2024-06-10T01:05:33.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.691+0000 7ff5cffff700 10 client.15198 handle_command_reply: tid=0 2024-06-10T01:05:33.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.691+0000 7ff6016c7b80 1 client.15198 shutdown 2024-06-10T01:05:33.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.691+0000 7ff6016c7b80 1 -- 172.21.15.7:0/2191725267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a95a61f10 msgr2=0x555a95a7f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:33.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.691+0000 7ff6016c7b80 1 --2- 172.21.15.7:0/2191725267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a95a61f10 0x555a95a7f530 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x555a95a58a80 tx=0x7ff5dc0047c0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.691+0000 7ff6016c7b80 1 -- 172.21.15.7:0/2191725267 shutdown_connections 2024-06-10T01:05:33.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.691+0000 7ff6016c7b80 1 --2- 172.21.15.7:0/2191725267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a95a62830 0x555a95a7fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.691+0000 7ff6016c7b80 1 --2- 172.21.15.7:0/2191725267 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555a95ae6a20 0x555a95ae5730 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.691+0000 7ff5cffff700 0 client.15198 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:05:33.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.691+0000 7ff6016c7b80 1 --2- 172.21.15.7:0/2191725267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a95a5c750 0x555a95a7eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.691+0000 7ff6016c7b80 1 --2- 172.21.15.7:0/2191725267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a95a61f10 0x555a95a7f530 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.691+0000 7ff6016c7b80 1 -- 172.21.15.7:0/2191725267 >> 172.21.15.7:0/2191725267 conn(0x555a95989030 msgr2=0x555a95a67050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:33.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.691+0000 7ff6016c7b80 1 -- 172.21.15.7:0/2191725267 shutdown_connections 2024-06-10T01:05:33.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.691+0000 7ff6016c7b80 1 -- 172.21.15.7:0/2191725267 wait complete. 2024-06-10T01:05:33.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.691+0000 7ff6016c7b80 20 client.15198 trim_cache size 0 max 16384 2024-06-10T01:05:33.694 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:05:33.694 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "bded58f8-26c3-11ef-bc9b-c7b262605968", 2024-06-10T01:05:33.694 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-06-10T01:05:33.694 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24451, 2024-06-10T01:05:33.694 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-06-10T01:05:33.694 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-06-10T01:05:33.694 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-06-10T01:05:33.694 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 224.454199426, 2024-06-10T01:05:33.694 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 26, 2024-06-10T01:05:33.694 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 99, 2024-06-10T01:05:33.694 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 97, 2024-06-10T01:05:33.694 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 251.549449987 2024-06-10T01:05:33.695 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:05:33.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.692+0000 7ff5f6e44700 1 -- 172.21.15.7:0/3839920266 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff5e00a2c20 msgr2=0x7ff5e00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:33.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.692+0000 7ff5f6e44700 1 --2- 172.21.15.7:0/3839920266 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff5e00a2c20 0x7ff5e00a50e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0078a20 tx=0x7ff5dc00b3c0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.692+0000 7ff5f6e44700 1 -- 172.21.15.7:0/3839920266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5f006a170 msgr2=0x7ff5f019e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:33.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.692+0000 7ff5f6e44700 1 --2- 172.21.15.7:0/3839920266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5f006a170 0x7ff5f019e8a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4000c00 tx=0x7ff5e40047b0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.692+0000 7ff5f6e44700 1 -- 172.21.15.7:0/3839920266 shutdown_connections 2024-06-10T01:05:33.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.692+0000 7ff5f6e44700 1 --2- 172.21.15.7:0/3839920266 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff5e00a2c20 0x7ff5e00a50e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.692+0000 7ff5f6e44700 1 --2- 172.21.15.7:0/3839920266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5f0108500 0x7ff5f0198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.692+0000 7ff5f6e44700 1 --2- 172.21.15.7:0/3839920266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5f006a170 0x7ff5f019e8a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.692+0000 7ff5f6e44700 1 --2- 172.21.15.7:0/3839920266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5f006ab30 0x7ff5f019ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:33.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.692+0000 7ff5f6e44700 1 -- 172.21.15.7:0/3839920266 >> 172.21.15.7:0/3839920266 conn(0x7ff5f0078340 msgr2=0x7ff5f0100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:33.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.692+0000 7ff5f6e44700 1 -- 172.21.15.7:0/3839920266 shutdown_connections 2024-06-10T01:05:33.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:33.692+0000 7ff5f6e44700 1 -- 172.21.15.7:0/3839920266 wait complete. 2024-06-10T01:05:34.341 INFO:teuthology.run_tasks:Running task sequential... 2024-06-10T01:05:34.350 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-06-10T01:05:34.350 INFO:teuthology.task.sequential:In sequential, running task print... 2024-06-10T01:05:34.350 INFO:teuthology.task.print:Enabling mgr modules 2024-06-10T01:05:34.350 INFO:teuthology.run_tasks:Running task kclient... 2024-06-10T01:05:34.362 INFO:tasks.kclient:Mounting kernel clients... 2024-06-10T01:05:34.362 INFO:tasks.kclient:config is {'client.0': {}} 2024-06-10T01:05:34.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:05:34.362 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-06-10T01:05:34.380 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-06-10T01:05:34.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:05:34.381 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-10T01:05:34.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:05:34.452 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-06-10T01:05:34.553 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-06-10T01:05:34.553 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-10T01:05:34.553 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-10T01:05:34.554 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-06-10T01:05:34.765 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:34 smithi007 ceph-mon[32398]: pgmap v418: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:05:34.804 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:05:34.954 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:34 smithi175 ceph-mon[36636]: pgmap v418: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:05:35.081 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:34 smithi078 ceph-mon[36683]: pgmap v418: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:05:36.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.395+0000 7fbd7ac40700 1 -- 172.21.15.7:0/1806909849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd7406b850 msgr2=0x7fbd74107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:36.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.395+0000 7fbd7ac40700 1 --2- 172.21.15.7:0/1806909849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd7406b850 0x7fbd74107130 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbd600099c0 tx=0x7fbd60009cd0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.395+0000 7fbd7ac40700 1 -- 172.21.15.7:0/1806909849 shutdown_connections 2024-06-10T01:05:36.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.395+0000 7fbd7ac40700 1 --2- 172.21.15.7:0/1806909849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd74107860 0x7fbd7410f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.395+0000 7fbd7ac40700 1 --2- 172.21.15.7:0/1806909849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd7406aea0 0x7fbd7406b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.395+0000 7fbd7ac40700 1 --2- 172.21.15.7:0/1806909849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd7406b850 0x7fbd74107130 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.395+0000 7fbd7ac40700 1 -- 172.21.15.7:0/1806909849 >> 172.21.15.7:0/1806909849 conn(0x7fbd740784c0 msgr2=0x7fbd740788d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:36.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.395+0000 7fbd7ac40700 1 -- 172.21.15.7:0/1806909849 shutdown_connections 2024-06-10T01:05:36.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.395+0000 7fbd7ac40700 1 -- 172.21.15.7:0/1806909849 wait complete. 2024-06-10T01:05:36.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.396+0000 7fbd7ac40700 1 Processor -- start 2024-06-10T01:05:36.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.396+0000 7fbd7ac40700 1 -- start start 2024-06-10T01:05:36.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.396+0000 7fbd7ac40700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd7406aea0 0x7fbd7419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:36.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.396+0000 7fbd7ac40700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd7406b850 0x7fbd7419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:36.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.396+0000 7fbd7ac40700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd74107860 0x7fbd741989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:36.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.396+0000 7fbd7ac40700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbd74112790 con 0x7fbd74107860 2024-06-10T01:05:36.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.396+0000 7fbd7ac40700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbd74112610 con 0x7fbd7406aea0 2024-06-10T01:05:36.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.396+0000 7fbd7ac40700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbd74112910 con 0x7fbd7406b850 2024-06-10T01:05:36.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.397+0000 7fbd791dd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd74107860 0x7fbd741989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:36.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.397+0000 7fbd791dd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd74107860 0x7fbd741989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44798/0 (socket says 172.21.15.7:44798) 2024-06-10T01:05:36.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.397+0000 7fbd791dd700 1 -- 172.21.15.7:0/112848825 learned_addr learned my addr 172.21.15.7:0/112848825 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:36.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.397+0000 7fbd789dc700 1 --2- 172.21.15.7:0/112848825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd7406aea0 0x7fbd7419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:36.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.397+0000 7fbd73fff700 1 --2- 172.21.15.7:0/112848825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd7406b850 0x7fbd7419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:36.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.397+0000 7fbd791dd700 1 -- 172.21.15.7:0/112848825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd7406b850 msgr2=0x7fbd7419ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:36.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.397+0000 7fbd791dd700 1 --2- 172.21.15.7:0/112848825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd7406b850 0x7fbd7419ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.397+0000 7fbd791dd700 1 -- 172.21.15.7:0/112848825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd7406aea0 msgr2=0x7fbd7419e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:36.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.397+0000 7fbd791dd700 1 --2- 172.21.15.7:0/112848825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd7406aea0 0x7fbd7419e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.397+0000 7fbd791dd700 1 -- 172.21.15.7:0/112848825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd74199130 con 0x7fbd74107860 2024-06-10T01:05:36.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.397+0000 7fbd73fff700 1 --2- 172.21.15.7:0/112848825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd7406b850 0x7fbd7419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:05:36.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.397+0000 7fbd791dd700 1 --2- 172.21.15.7:0/112848825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd74107860 0x7fbd741989d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c007ee0 tx=0x7fbd6c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:36.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.397+0000 7fbd71ffb700 1 -- 172.21.15.7:0/112848825 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c00dbd0 con 0x7fbd74107860 2024-06-10T01:05:36.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.397+0000 7fbd71ffb700 1 -- 172.21.15.7:0/112848825 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd6c00dd70 con 0x7fbd74107860 2024-06-10T01:05:36.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.397+0000 7fbd7ac40700 1 -- 172.21.15.7:0/112848825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd74199420 con 0x7fbd74107860 2024-06-10T01:05:36.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.397+0000 7fbd71ffb700 1 -- 172.21.15.7:0/112848825 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c005a60 con 0x7fbd74107860 2024-06-10T01:05:36.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.398+0000 7fbd7ac40700 1 -- 172.21.15.7:0/112848825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd741a56e0 con 0x7fbd74107860 2024-06-10T01:05:36.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.399+0000 7fbd7ac40700 1 -- 172.21.15.7:0/112848825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd74069180 con 0x7fbd74107860 2024-06-10T01:05:36.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.400+0000 7fbd71ffb700 1 -- 172.21.15.7:0/112848825 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fbd6c01a020 con 0x7fbd74107860 2024-06-10T01:05:36.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.400+0000 7fbd71ffb700 1 --2- 172.21.15.7:0/112848825 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fbd640a2c20 0x7fbd640a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:36.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.400+0000 7fbd789dc700 1 --2- 172.21.15.7:0/112848825 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fbd640a2c20 0x7fbd640a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:36.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.400+0000 7fbd71ffb700 1 -- 172.21.15.7:0/112848825 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fbd6c0cea90 con 0x7fbd74107860 2024-06-10T01:05:36.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.401+0000 7fbd789dc700 1 --2- 172.21.15.7:0/112848825 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fbd640a2c20 0x7fbd640a50e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbd74199b10 tx=0x7fbd680073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:36.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.405+0000 7fbd71ffb700 1 -- 172.21.15.7:0/112848825 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbd6c010070 con 0x7fbd74107860 2024-06-10T01:05:36.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.664+0000 7fbd7ac40700 1 -- 172.21.15.7:0/112848825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbd7402dca0 con 0x7fbd74107860 2024-06-10T01:05:36.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.664+0000 7fbd71ffb700 1 -- 172.21.15.7:0/112848825 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+88 (secure 0 0 0) 0x7fbd6c09b670 con 0x7fbd74107860 2024-06-10T01:05:36.661 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-06-10T01:05:36.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.667+0000 7fbd7ac40700 1 -- 172.21.15.7:0/112848825 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fbd640a2c20 msgr2=0x7fbd640a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:36.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.667+0000 7fbd7ac40700 1 --2- 172.21.15.7:0/112848825 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fbd640a2c20 0x7fbd640a50e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbd74199b10 tx=0x7fbd680073d0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.667+0000 7fbd7ac40700 1 -- 172.21.15.7:0/112848825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd74107860 msgr2=0x7fbd741989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:36.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.667+0000 7fbd7ac40700 1 --2- 172.21.15.7:0/112848825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd74107860 0x7fbd741989d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c007ee0 tx=0x7fbd6c00d3b0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.667+0000 7fbd7ac40700 1 -- 172.21.15.7:0/112848825 shutdown_connections 2024-06-10T01:05:36.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.667+0000 7fbd7ac40700 1 --2- 172.21.15.7:0/112848825 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fbd640a2c20 0x7fbd640a50e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.667+0000 7fbd7ac40700 1 --2- 172.21.15.7:0/112848825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd74107860 0x7fbd741989d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.667+0000 7fbd7ac40700 1 --2- 172.21.15.7:0/112848825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd7406aea0 0x7fbd7419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.667+0000 7fbd7ac40700 1 --2- 172.21.15.7:0/112848825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd7406b850 0x7fbd7419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.667+0000 7fbd7ac40700 1 -- 172.21.15.7:0/112848825 >> 172.21.15.7:0/112848825 conn(0x7fbd740784c0 msgr2=0x7fbd741089b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:36.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.667+0000 7fbd7ac40700 1 -- 172.21.15.7:0/112848825 shutdown_connections 2024-06-10T01:05:36.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:36.667+0000 7fbd7ac40700 1 -- 172.21.15.7:0/112848825 wait complete. 2024-06-10T01:05:37.324 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:37 smithi007 ceph-mon[32398]: pgmap v419: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:05:37.324 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:37 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/112848825' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-10T01:05:37.354 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-06-10T01:05:37.355 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-10T01:05:37.355 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-06-10T01:05:37.355 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-06-10T01:05:37.355 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-10T01:05:37.355 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-06-10T01:05:37.355 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-10T01:05:37.355 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-10T01:05:37.355 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-06-10T01:05:37.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:05:37.355 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-06-10T01:05:37.374 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-10T01:05:37.374 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-10T01:05:37.374 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-06-10T01:05:37.374 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-06-10T01:05:37.375 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-06-10T01:05:37.375 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-06-10T01:05:37.375 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-10T01:05:37.375 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-06-10T01:05:37.375 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-06-10T01:05:37.375 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-10T01:05:37.375 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-06-10T01:05:37.375 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-06-10T01:05:37.375 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-10T01:05:37.375 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-06-10T01:05:37.375 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-06-10T01:05:37.375 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-06-10T01:05:37.376 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42019sec preferred_lft 42019sec 2024-06-10T01:05:37.376 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link noprefixroute 2024-06-10T01:05:37.376 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-06-10T01:05:37.376 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-10T01:05:37.376 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-06-10T01:05:37.376 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-06-10T01:05:37.376 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-10T01:05:37.376 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T01:05:37.376 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-06-10T01:05:37.376 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add name ceph-brx type bridge 2024-06-10T01:05:37.377 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr flush dev ceph-brx 2024-06-10T01:05:37.377 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set ceph-brx up 2024-06-10T01:05:37.377 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-10T01:05:37.377 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-06-10T01:05:37.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T01:05:37.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T01:05:37.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:05:37.523 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-10T01:05:37.548 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-06-10T01:05:37.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:05:37.550 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-06-10T01:05:37.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:37 smithi078 ceph-mon[36683]: pgmap v419: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:05:37.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:37 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/112848825' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-10T01:05:37.607 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-06-10T01:05:37.608 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-10T01:05:37.608 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-06-10T01:05:37.608 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-06-10T01:05:37.608 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-10T01:05:37.608 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T01:05:37.608 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-06-10T01:05:37.608 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-06-10T01:05:37.608 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-06-10T01:05:37.609 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-06-10T01:05:37.609 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-06-10T01:05:37.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T01:05:37.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:37 smithi175 ceph-mon[36636]: pgmap v419: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T01:05:37.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:37 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/112848825' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-10T01:05:37.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T01:05:37.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:05:37.740 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-06-10T01:05:37.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:05:37.756 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2024-06-10T01:05:37.812 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T01:05:37.812 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-06-10T01:05:37.812 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-10T01:05:37.813 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-06-10T01:05:37.813 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-06-10T01:05:37.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T01:05:37.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T01:05:37.932 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-06-10T01:05:37.933 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T01:05:37.933 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-06-10T01:05:37.933 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-06-10T01:05:37.933 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-10T01:05:37.933 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-06-10T01:05:37.933 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-06-10T01:05:37.933 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-06-10T01:05:37.933 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-06-10T01:05:37.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T01:05:38.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T01:05:38.052 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T01:05:38.052 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-06-10T01:05:38.052 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-06-10T01:05:38.052 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.0 master ceph-brx 2024-06-10T01:05:38.052 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-06-10T01:05:38.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T01:05:38.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T01:05:38.132 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-10T01:05:38.132 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-06-10T01:05:38.148 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-06-10T01:05:38.148 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-10T01:05:38.148 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-06-10T01:05:38.204 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-06-10T01:05:38.204 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-06-10T01:05:38.204 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/56b71f3b-2893-4c82-ac3a-2d6fa4194c93 2024-06-10T01:05:38.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:05:38.204 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/56b71f3b-2893-4c82-ac3a-2d6fa4194c93 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-06-10T01:05:38.289 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-06-10T01:05:38.289 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync". 2024-06-10T01:05:38.438 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-06-10T01:05:38.438 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync". 2024-06-10T01:05:38.439 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.78:3300,172.21.15.175:3300" 2024-06-10T01:05:38.439 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 0@bded58f8-26c3-11ef-bc9b-c7b262605968.cephfs=/volumes/qa/sv_1/56b71f3b-2893-4c82-ac3a-2d6fa4194c93 2024-06-10T01:05:38.439 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync,name=0,key=0,mon_addr=172.21.15.7:3300/172.21.15.78:3300/172.21.15.175:3300" will pass to kernel 2024-06-10T01:05:38.439 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-06-10T01:05:38.439 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-06-10T01:05:38.439 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-06-10T01:05:38.439 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-06-10T01:05:38.440 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-06-10T01:05:38.441 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-06-10T01:05:38.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:05:38.441 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-06-10T01:05:38.472 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-10T01:05:38.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-10T01:05:38.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T01:05:38.571 INFO:teuthology.orchestra.run.smithi007.stdout:client24973 2024-06-10T01:05:38.571 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-10T01:05:38.571 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-10T01:05:38.589 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-10T01:05:38.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-10T01:05:38.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T01:05:38.688 INFO:teuthology.orchestra.run.smithi007.stdout:client24973 2024-06-10T01:05:38.688 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-10T01:05:38.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-10T01:05:38.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T01:05:38.746 INFO:teuthology.orchestra.run.smithi007.stdout:client24973 2024-06-10T01:05:38.746 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968.client24973/status') 2024-06-10T01:05:38.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T01:05:38.806 INFO:teuthology.orchestra.run.smithi007.stdout:instance: client.24973 (3)172.21.15.7:0/2899150270 2024-06-10T01:05:38.806 INFO:teuthology.orchestra.run.smithi007.stdout:blocklisted: false 2024-06-10T01:05:38.806 INFO:teuthology.orchestra.run.smithi007.stderr:0+1 records in 2024-06-10T01:05:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:0+1 records out 2024-06-10T01:05:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:70 bytes copied, 2.47e-05 s, 2.8 MB/s 2024-06-10T01:05:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T01:05:38.814 INFO:teuthology.run_tasks:Running task exec... 2024-06-10T01:05:38.824 INFO:teuthology.task.exec:Executing custom commands... 2024-06-10T01:05:38.824 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-06-10T01:05:38.824 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-06-10T01:05:38.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.933+0000 7f98b55f9640 1 -- 172.21.15.7:0/2383062355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98b00ffa80 msgr2=0x7f98b00fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:38.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.933+0000 7f98b55f9640 1 --2- 172.21.15.7:0/2383062355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98b00ffa80 0x7f98b00fff00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9898009900 tx=0x7f989802a410 comp rx=0 tx=0).stop 2024-06-10T01:05:38.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.933+0000 7f98b55f9640 1 -- 172.21.15.7:0/2383062355 shutdown_connections 2024-06-10T01:05:38.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.933+0000 7f98b55f9640 1 --2- 172.21.15.7:0/2383062355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98b0100440 0x7f98b0109ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:38.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.933+0000 7f98b55f9640 1 --2- 172.21.15.7:0/2383062355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98b00ffa80 0x7f98b00fff00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:38.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.933+0000 7f98b55f9640 1 --2- 172.21.15.7:0/2383062355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98b0103010 0x7f98b0103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:38.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.933+0000 7f98b55f9640 1 -- 172.21.15.7:0/2383062355 >> 172.21.15.7:0/2383062355 conn(0x7f98b00fb600 msgr2=0x7f98b00fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:38.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.933+0000 7f98b55f9640 1 -- 172.21.15.7:0/2383062355 shutdown_connections 2024-06-10T01:05:38.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.933+0000 7f98b55f9640 1 -- 172.21.15.7:0/2383062355 wait complete. 2024-06-10T01:05:38.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.934+0000 7f98b55f9640 1 Processor -- start 2024-06-10T01:05:38.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.934+0000 7f98b55f9640 1 -- start start 2024-06-10T01:05:38.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.935+0000 7f98b55f9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98b00ffa80 0x7f98b0197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:38.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.935+0000 7f98b55f9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98b0100440 0x7f98b0198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:38.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.935+0000 7f98b55f9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98b0103010 0x7f98b019f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:38.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.935+0000 7f98b55f9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f98b010d6c0 con 0x7f98b0103010 2024-06-10T01:05:38.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.935+0000 7f98b55f9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f98b010d540 con 0x7f98b00ffa80 2024-06-10T01:05:38.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.935+0000 7f98b55f9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f98b010d840 con 0x7f98b0100440 2024-06-10T01:05:38.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.935+0000 7f98af7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98b0103010 0x7f98b019f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:38.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.935+0000 7f98af7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98b0103010 0x7f98b019f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44810/0 (socket says 172.21.15.7:44810) 2024-06-10T01:05:38.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.935+0000 7f98af7fe640 1 -- 172.21.15.7:0/2633995688 learned_addr learned my addr 172.21.15.7:0/2633995688 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:38.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.935+0000 7f98aeffd640 1 --2- 172.21.15.7:0/2633995688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98b00ffa80 0x7f98b0197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:38.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.935+0000 7f98ae7fc640 1 --2- 172.21.15.7:0/2633995688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98b0100440 0x7f98b0198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:38.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.935+0000 7f98af7fe640 1 -- 172.21.15.7:0/2633995688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98b0100440 msgr2=0x7f98b0198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:38.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.935+0000 7f98af7fe640 1 --2- 172.21.15.7:0/2633995688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98b0100440 0x7f98b0198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:38.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.935+0000 7f98af7fe640 1 -- 172.21.15.7:0/2633995688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98b00ffa80 msgr2=0x7f98b0197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:38.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.935+0000 7f98af7fe640 1 --2- 172.21.15.7:0/2633995688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98b00ffa80 0x7f98b0197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:38.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.935+0000 7f98af7fe640 1 -- 172.21.15.7:0/2633995688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b019f930 con 0x7f98b0103010 2024-06-10T01:05:38.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.935+0000 7f98af7fe640 1 --2- 172.21.15.7:0/2633995688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98b0103010 0x7f98b019f290 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f98a800eaa0 tx=0x7f98a800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:38.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.935+0000 7f98a7fff640 1 -- 172.21.15.7:0/2633995688 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a800cdb0 con 0x7f98b0103010 2024-06-10T01:05:38.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.935+0000 7f98a7fff640 1 -- 172.21.15.7:0/2633995688 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98a8005020 con 0x7f98b0103010 2024-06-10T01:05:38.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.935+0000 7f98b55f9640 1 -- 172.21.15.7:0/2633995688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98b019fc20 con 0x7f98b0103010 2024-06-10T01:05:38.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.935+0000 7f98a7fff640 1 -- 172.21.15.7:0/2633995688 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a80116b0 con 0x7f98b0103010 2024-06-10T01:05:38.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.936+0000 7f98b55f9640 1 -- 172.21.15.7:0/2633995688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98b01a0170 con 0x7f98b0103010 2024-06-10T01:05:38.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.936+0000 7f98b55f9640 1 -- 172.21.15.7:0/2633995688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98b0108ed0 con 0x7f98b0103010 2024-06-10T01:05:38.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.938+0000 7f98a7fff640 1 -- 172.21.15.7:0/2633995688 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f98a8005560 con 0x7f98b0103010 2024-06-10T01:05:38.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.938+0000 7f98a7fff640 1 --2- 172.21.15.7:0/2633995688 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f989c0a2b80 0x7f989c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:38.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.939+0000 7f98aeffd640 1 --2- 172.21.15.7:0/2633995688 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f989c0a2b80 0x7f989c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:38.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.939+0000 7f98a7fff640 1 -- 172.21.15.7:0/2633995688 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f98a80cf6a0 con 0x7f98b0103010 2024-06-10T01:05:38.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.939+0000 7f98aeffd640 1 --2- 172.21.15.7:0/2633995688 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f989c0a2b80 0x7f989c0a5040 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f98b01991f0 tx=0x7f98a000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:38.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:38.943+0000 7f98a7fff640 1 -- 172.21.15.7:0/2633995688 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f98a8016030 con 0x7f98b0103010 2024-06-10T01:05:39.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.174+0000 7f98b55f9640 1 -- 172.21.15.7:0/2633995688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f98b0062640 con 0x7f98b0103010 2024-06-10T01:05:39.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.238+0000 7f98a7fff640 1 -- 172.21.15.7:0/2633995688 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v27) ==== 117+0+0 (secure 0 0 0) 0x7f98a809c280 con 0x7f98b0103010 2024-06-10T01:05:39.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.241+0000 7f98b55f9640 1 -- 172.21.15.7:0/2633995688 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f989c0a2b80 msgr2=0x7f989c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:39.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.241+0000 7f98b55f9640 1 --2- 172.21.15.7:0/2633995688 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f989c0a2b80 0x7f989c0a5040 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f98b01991f0 tx=0x7f98a000a2d0 comp rx=0 tx=0).stop 2024-06-10T01:05:39.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.241+0000 7f98b55f9640 1 -- 172.21.15.7:0/2633995688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98b0103010 msgr2=0x7f98b019f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:39.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.241+0000 7f98b55f9640 1 --2- 172.21.15.7:0/2633995688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98b0103010 0x7f98b019f290 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f98a800eaa0 tx=0x7f98a800ef60 comp rx=0 tx=0).stop 2024-06-10T01:05:39.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.241+0000 7f98b55f9640 1 -- 172.21.15.7:0/2633995688 shutdown_connections 2024-06-10T01:05:39.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.241+0000 7f98b55f9640 1 --2- 172.21.15.7:0/2633995688 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f989c0a2b80 0x7f989c0a5040 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:39.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.241+0000 7f98b55f9640 1 --2- 172.21.15.7:0/2633995688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98b0103010 0x7f98b019f290 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:39.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.241+0000 7f98b55f9640 1 --2- 172.21.15.7:0/2633995688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98b0100440 0x7f98b0198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:39.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.241+0000 7f98b55f9640 1 --2- 172.21.15.7:0/2633995688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98b00ffa80 0x7f98b0197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:39.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.241+0000 7f98b55f9640 1 -- 172.21.15.7:0/2633995688 >> 172.21.15.7:0/2633995688 conn(0x7f98b00fb600 msgr2=0x7f98b01060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:39.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.241+0000 7f98b55f9640 1 -- 172.21.15.7:0/2633995688 shutdown_connections 2024-06-10T01:05:39.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.241+0000 7f98b55f9640 1 -- 172.21.15.7:0/2633995688 wait complete. 2024-06-10T01:05:39.250 INFO:teuthology.run_tasks:Running task check-counter... 2024-06-10T01:05:39.261 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-06-10T01:05:39.271 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-10T01:05:39.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.380+0000 7f9c07437640 1 -- 172.21.15.7:0/4143573282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c00102d00 msgr2=0x7f9c00103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:39.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.380+0000 7f9c07437640 1 --2- 172.21.15.7:0/4143573282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c00102d00 0x7f9c00103180 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009930 tx=0x7f9bf402a420 comp rx=0 tx=0).stop 2024-06-10T01:05:39.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.380+0000 7f9c07437640 1 -- 172.21.15.7:0/4143573282 shutdown_connections 2024-06-10T01:05:39.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.380+0000 7f9c07437640 1 --2- 172.21.15.7:0/4143573282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c001036c0 0x7f9c00109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:39.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.380+0000 7f9c07437640 1 --2- 172.21.15.7:0/4143573282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c00102d00 0x7f9c00103180 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:39.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.380+0000 7f9c07437640 1 --2- 172.21.15.7:0/4143573282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c000ffd30 0x7f9c00100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:39.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.380+0000 7f9c07437640 1 -- 172.21.15.7:0/4143573282 >> 172.21.15.7:0/4143573282 conn(0x7f9c000fb660 msgr2=0x7f9c000fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:39.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.380+0000 7f9c07437640 1 -- 172.21.15.7:0/4143573282 shutdown_connections 2024-06-10T01:05:39.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.380+0000 7f9c07437640 1 -- 172.21.15.7:0/4143573282 wait complete. 2024-06-10T01:05:39.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.380+0000 7f9c07437640 1 Processor -- start 2024-06-10T01:05:39.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.380+0000 7f9c07437640 1 -- start start 2024-06-10T01:05:39.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.380+0000 7f9c07437640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c000ffd30 0x7f9c00102670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:39.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.381+0000 7f9c07437640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c00102d00 0x7f9c00100cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:39.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.381+0000 7f9c07437640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c001036c0 0x7f9c001011f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:39.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.381+0000 7f9c07437640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0010d710 con 0x7f9c000ffd30 2024-06-10T01:05:39.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.381+0000 7f9c07437640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0010d590 con 0x7f9c001036c0 2024-06-10T01:05:39.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.381+0000 7f9c07437640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0010d890 con 0x7f9c00102d00 2024-06-10T01:05:39.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.381+0000 7f9c051d3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c000ffd30 0x7f9c00102670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:39.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.381+0000 7f9c051d3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c000ffd30 0x7f9c00102670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44830/0 (socket says 172.21.15.7:44830) 2024-06-10T01:05:39.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.381+0000 7f9c051d3640 1 -- 172.21.15.7:0/624483772 learned_addr learned my addr 172.21.15.7:0/624483772 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:39.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.381+0000 7f9c059d4640 1 --2- 172.21.15.7:0/624483772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c001036c0 0x7f9c001011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:39.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.381+0000 7f9c049d2640 1 --2- 172.21.15.7:0/624483772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c00102d00 0x7f9c00100cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:39.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.381+0000 7f9c051d3640 1 -- 172.21.15.7:0/624483772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c00102d00 msgr2=0x7f9c00100cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:39.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.381+0000 7f9c051d3640 1 --2- 172.21.15.7:0/624483772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c00102d00 0x7f9c00100cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:39.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.381+0000 7f9c051d3640 1 -- 172.21.15.7:0/624483772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c001036c0 msgr2=0x7f9c001011f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:39.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.381+0000 7f9c051d3640 1 --2- 172.21.15.7:0/624483772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c001036c0 0x7f9c001011f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:39.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.381+0000 7f9c051d3640 1 -- 172.21.15.7:0/624483772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c00101a80 con 0x7f9c000ffd30 2024-06-10T01:05:39.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.381+0000 7f9c049d2640 1 --2- 172.21.15.7:0/624483772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c00102d00 0x7f9c00100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:05:39.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.381+0000 7f9c059d4640 1 --2- 172.21.15.7:0/624483772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c001036c0 0x7f9c001011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:05:39.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.381+0000 7f9c051d3640 1 --2- 172.21.15.7:0/624483772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c000ffd30 0x7f9c00102670 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9bf000b9c0 tx=0x7f9bf000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:39.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.381+0000 7f9bfe7fc640 1 -- 172.21.15.7:0/624483772 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0019070 con 0x7f9c000ffd30 2024-06-10T01:05:39.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.381+0000 7f9bfe7fc640 1 -- 172.21.15.7:0/624483772 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bf000cba0 con 0x7f9c000ffd30 2024-06-10T01:05:39.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.381+0000 7f9bfe7fc640 1 -- 172.21.15.7:0/624483772 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0012640 con 0x7f9c000ffd30 2024-06-10T01:05:39.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.381+0000 7f9c07437640 1 -- 172.21.15.7:0/624483772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c001a1f00 con 0x7f9c000ffd30 2024-06-10T01:05:39.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.381+0000 7f9c07437640 1 -- 172.21.15.7:0/624483772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c001a2390 con 0x7f9c000ffd30 2024-06-10T01:05:39.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.382+0000 7f9c07437640 1 -- 172.21.15.7:0/624483772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c00108eb0 con 0x7f9c000ffd30 2024-06-10T01:05:39.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.384+0000 7f9bfe7fc640 1 -- 172.21.15.7:0/624483772 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9bf001c070 con 0x7f9c000ffd30 2024-06-10T01:05:39.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.385+0000 7f9bfe7fc640 1 --2- 172.21.15.7:0/624483772 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9be80a2b90 0x7f9be80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:39.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.385+0000 7f9c049d2640 1 --2- 172.21.15.7:0/624483772 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9be80a2b90 0x7f9be80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:39.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.385+0000 7f9bfe7fc640 1 -- 172.21.15.7:0/624483772 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f9bf00cea90 con 0x7f9c000ffd30 2024-06-10T01:05:39.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.385+0000 7f9c049d2640 1 --2- 172.21.15.7:0/624483772 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9be80a2b90 0x7f9be80a5050 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4006010 tx=0x7f9bf4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:39.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.389+0000 7f9bfe7fc640 1 -- 172.21.15.7:0/624483772 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9bf009b610 con 0x7f9c000ffd30 2024-06-10T01:05:39.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:39 smithi007 ceph-mon[32398]: pgmap v420: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:05:39.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:39 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2633995688' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-10T01:05:39.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.576+0000 7f9c07437640 1 -- 172.21.15.7:0/624483772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c000626e0 con 0x7f9c000ffd30 2024-06-10T01:05:39.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.578+0000 7f9bfe7fc640 1 -- 172.21.15.7:0/624483772 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24615 (secure 0 0 0) 0x7f9bf009af10 con 0x7f9c000ffd30 2024-06-10T01:05:39.575 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:05:39.575 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":99,"fsid":"bded58f8-26c3-11ef-bc9b-c7b262605968","created":"2024-06-10T00:53:31.966085+0000","modified":"2024-06-10T01:03:38.227703+0000","last_up_change":"2024-06-10T01:00:45.382473+0000","last_in_change":"2024-06-10T01:00:25.608513+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:56:59.964055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T01:01:42.563611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T01:01:46.175918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T01:01:55.695502+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f7c24c9f-b7ab-4c04-b2b4-120df9751667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6803","nonce":404485726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6805","nonce":404485726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6809","nonce":404485726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6807","nonce":404485726}]},"public_addr":"172.21.15.7:6803/404485726","cluster_addr":"172.21.15.7:6805/404485726","heartbeat_back_addr":"172.21.15.7:6809/404485726","heartbeat_front_addr":"172.21.15.7:6807/404485726","state":["exists","up"]},{"osd":1,"uuid":"9bf70a6b-f31c-4077-b5af-229f17bc2845","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6801","nonce":1856804448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6803","nonce":1856804448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6807","nonce":1856804448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6805","nonce":1856804448}]},"public_addr":"172.21.15.78:6801/1856804448","cluster_addr":"172.21.15.78:6803/1856804448","heartbeat_back_addr":"172.21.15.78:6807/1856804448","heartbeat_front_addr":"172.21.15.78:6805/1856804448","state":["exists","up"]},{"osd":2,"uuid":"62923c97-caf9-4f7e-9c14-9c9918843c78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6801","nonce":918257455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6803","nonce":918257455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6807","nonce":918257455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6805","nonce":918257455}]},"public_addr":"172.21.15.175:6801/918257455","cluster_addr":"172.21.15.175:6803/918257455","heartbeat_back_addr":"172.21.15.175:6807/918257455","heartbeat_front_addr":"172.21.15.175:6805/918257455","state":["exists","up"]},{"osd":3,"uuid":"65242098-e748-48ab-8f0e-bb25b5c62742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6811","nonce":1797404966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6813","nonce":1797404966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6817","nonce":1797404966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6815","nonce":1797404966}]},"public_addr":"172.21.15.7:6811/1797404966","cluster_addr":"172.21.15.7:6813/1797404966","heartbeat_back_addr":"172.21.15.7:6817/1797404966","heartbeat_front_addr":"172.21.15.7:6815/1797404966","state":["exists","up"]},{"osd":4,"uuid":"a7990305-de9e-400e-a432-d7854f73f339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6809","nonce":3767608357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6811","nonce":3767608357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6815","nonce":3767608357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6813","nonce":3767608357}]},"public_addr":"172.21.15.78:6809/3767608357","cluster_addr":"172.21.15.78:6811/3767608357","heartbeat_back_addr":"172.21.15.78:6815/3767608357","heartbeat_front_addr":"172.21.15.78:6813/3767608357","state":["exists","up"]},{"osd":5,"uuid":"bad9a7b3-da5a-48d0-905d-e6a3190317fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6809","nonce":2157664731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6811","nonce":2157664731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6815","nonce":2157664731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6813","nonce":2157664731}]},"public_addr":"172.21.15.175:6809/2157664731","cluster_addr":"172.21.15.175:6811/2157664731","heartbeat_back_addr":"172.21.15.175:6815/2157664731","heartbeat_front_addr":"172.21.15.175:6813/2157664731","state":["exists","up"]},{"osd":6,"uuid":"6c687653-157b-4b96-867b-a2eaa7d8ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6819","nonce":838731756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6821","nonce":838731756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6825","nonce":838731756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6823","nonce":838731756}]},"public_addr":"172.21.15.7:6819/838731756","cluster_addr":"172.21.15.7:6821/838731756","heartbeat_back_addr":"172.21.15.7:6825/838731756","heartbeat_front_addr":"172.21.15.7:6823/838731756","state":["exists","up"]},{"osd":7,"uuid":"db40ddf1-9fd9-4859-b8c4-75f79582ab91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6817","nonce":3511921773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6819","nonce":3511921773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6823","nonce":3511921773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6821","nonce":3511921773}]},"public_addr":"172.21.15.78:6817/3511921773","cluster_addr":"172.21.15.78:6819/3511921773","heartbeat_back_addr":"172.21.15.78:6823/3511921773","heartbeat_front_addr":"172.21.15.78:6821/3511921773","state":["exists","up"]},{"osd":8,"uuid":"0f01cbd9-b193-4c70-a33c-4984167bf422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6817","nonce":1546325211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6819","nonce":1546325211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6823","nonce":1546325211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6821","nonce":1546325211}]},"public_addr":"172.21.15.175:6817/1546325211","cluster_addr":"172.21.15.175:6819/1546325211","heartbeat_back_addr":"172.21.15.175:6823/1546325211","heartbeat_front_addr":"172.21.15.175:6821/1546325211","state":["exists","up"]},{"osd":9,"uuid":"e5480998-f7fb-49cd-9aed-52405c88d830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6827","nonce":1844550813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6829","nonce":1844550813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6833","nonce":1844550813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6831","nonce":1844550813}]},"public_addr":"172.21.15.7:6827/1844550813","cluster_addr":"172.21.15.7:6829/1844550813","heartbeat_back_addr":"172.21.15.7:6833/1844550813","heartbeat_front_addr":"172.21.15.7:6831/1844550813","state":["exists","up"]},{"osd":10,"uuid":"caa82f33-a71c-4907-bf3b-30a7fdfeb71d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6825","nonce":2290824785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6827","nonce":2290824785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6831","nonce":2290824785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6829","nonce":2290824785}]},"public_addr":"172.21.15.78:6825/2290824785","cluster_addr":"172.21.15.78:6827/2290824785","heartbeat_back_addr":"172.21.15.78:6831/2290824785","heartbeat_front_addr":"172.21.15.78:6829/2290824785","state":["exists","up"]},{"osd":11,"uuid":"608cda77-77da-4da2-a7e2-76a773be27c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6825","nonce":1959979550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6827","nonce":1959979550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6831","nonce":1959979550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6829","nonce":1959979550}]},"public_addr":"172.21.15.175:6825/1959979550","cluster_addr":"172.21.15.175:6827/1959979550","heartbeat_back_addr":"172.21.15.175:6831/1959979550","heartbeat_front_addr":"172.21.15.175:6829/1959979550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:08.850203+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:32.820161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:56.633656+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:21.759372+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:46.735874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:10.636541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:36.990099+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:01.767098+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:26.882776+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:52.757595+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:18.383938+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:43.544330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1833397470":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/322543993":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/212510583":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6801/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6800/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/2679436843":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6801/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/1210355277":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6800/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/3610017406":"2024-06-11T00:54:08.102405+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T01:05:39.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.580+0000 7f9c07437640 1 -- 172.21.15.7:0/624483772 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9be80a2b90 msgr2=0x7f9be80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:39.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.580+0000 7f9c07437640 1 --2- 172.21.15.7:0/624483772 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9be80a2b90 0x7f9be80a5050 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4006010 tx=0x7f9bf4000bc0 comp rx=0 tx=0).stop 2024-06-10T01:05:39.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.580+0000 7f9c07437640 1 -- 172.21.15.7:0/624483772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c000ffd30 msgr2=0x7f9c00102670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:39.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.580+0000 7f9c07437640 1 --2- 172.21.15.7:0/624483772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c000ffd30 0x7f9c00102670 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9bf000b9c0 tx=0x7f9bf000be80 comp rx=0 tx=0).stop 2024-06-10T01:05:39.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.580+0000 7f9c07437640 1 -- 172.21.15.7:0/624483772 shutdown_connections 2024-06-10T01:05:39.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.580+0000 7f9c07437640 1 --2- 172.21.15.7:0/624483772 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9be80a2b90 0x7f9be80a5050 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:39.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.580+0000 7f9c07437640 1 --2- 172.21.15.7:0/624483772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c001036c0 0x7f9c001011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:39.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.580+0000 7f9c07437640 1 --2- 172.21.15.7:0/624483772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c00102d00 0x7f9c00100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:39.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.580+0000 7f9c07437640 1 --2- 172.21.15.7:0/624483772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c000ffd30 0x7f9c00102670 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:39.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.580+0000 7f9c07437640 1 -- 172.21.15.7:0/624483772 >> 172.21.15.7:0/624483772 conn(0x7f9c000fb660 msgr2=0x7f9c00106080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:39.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.580+0000 7f9c07437640 1 -- 172.21.15.7:0/624483772 shutdown_connections 2024-06-10T01:05:39.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.580+0000 7f9c07437640 1 -- 172.21.15.7:0/624483772 wait complete. 2024-06-10T01:05:39.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:39 smithi078 ceph-mon[36683]: pgmap v420: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:05:39.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:39 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2633995688' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-10T01:05:39.588 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-10T00:56:59.964055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-06-10T01:01:42.563611+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-06-10T01:01:46.175918+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-06-10T01:01:55.695502+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-06-10T01:05:39.589 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-10T01:05:39.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.700+0000 7fa7f585d640 1 -- 172.21.15.7:0/4028556029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7f0102be0 msgr2=0x7fa7f0103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:39.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.700+0000 7fa7f585d640 1 --2- 172.21.15.7:0/4028556029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7f0102be0 0x7fa7f0103060 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8009900 tx=0x7fa7d802a410 comp rx=0 tx=0).stop 2024-06-10T01:05:39.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.700+0000 7fa7f585d640 1 -- 172.21.15.7:0/4028556029 shutdown_connections 2024-06-10T01:05:39.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.700+0000 7fa7f585d640 1 --2- 172.21.15.7:0/4028556029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7f01035a0 0x7fa7f0109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:39.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.700+0000 7fa7f585d640 1 --2- 172.21.15.7:0/4028556029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7f0102be0 0x7fa7f0103060 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:39.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.700+0000 7fa7f585d640 1 --2- 172.21.15.7:0/4028556029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7f01019f0 0x7fa7f0101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:39.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.700+0000 7fa7f585d640 1 -- 172.21.15.7:0/4028556029 >> 172.21.15.7:0/4028556029 conn(0x7fa7f00fcf60 msgr2=0x7fa7f00ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:39.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.700+0000 7fa7f585d640 1 -- 172.21.15.7:0/4028556029 shutdown_connections 2024-06-10T01:05:39.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.700+0000 7fa7f585d640 1 -- 172.21.15.7:0/4028556029 wait complete. 2024-06-10T01:05:39.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.701+0000 7fa7f585d640 1 Processor -- start 2024-06-10T01:05:39.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.701+0000 7fa7f585d640 1 -- start start 2024-06-10T01:05:39.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.701+0000 7fa7f585d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7f01019f0 0x7fa7f0075750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:39.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.701+0000 7fa7f585d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7f0102be0 0x7fa7f0073d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:39.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.701+0000 7fa7f585d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7f01035a0 0x7fa7f00742d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:39.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.701+0000 7fa7f585d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f010d6c0 con 0x7fa7f0102be0 2024-06-10T01:05:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.701+0000 7fa7f585d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f010d540 con 0x7fa7f01035a0 2024-06-10T01:05:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.701+0000 7fa7f585d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f010d840 con 0x7fa7f01019f0 2024-06-10T01:05:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.701+0000 7fa7e67fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7f0102be0 0x7fa7f0073d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.701+0000 7fa7e67fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7f0102be0 0x7fa7f0073d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44848/0 (socket says 172.21.15.7:44848) 2024-06-10T01:05:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.701+0000 7fa7e67fc640 1 -- 172.21.15.7:0/1290848157 learned_addr learned my addr 172.21.15.7:0/1290848157 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.701+0000 7fa7ef7fe640 1 --2- 172.21.15.7:0/1290848157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7f01035a0 0x7fa7f00742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.701+0000 7fa7eeffd640 1 --2- 172.21.15.7:0/1290848157 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7f01019f0 0x7fa7f0075750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.701+0000 7fa7e67fc640 1 -- 172.21.15.7:0/1290848157 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7f01019f0 msgr2=0x7fa7f0075750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.701+0000 7fa7e67fc640 1 --2- 172.21.15.7:0/1290848157 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7f01019f0 0x7fa7f0075750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.701+0000 7fa7e67fc640 1 -- 172.21.15.7:0/1290848157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7f01035a0 msgr2=0x7fa7f00742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.701+0000 7fa7e67fc640 1 --2- 172.21.15.7:0/1290848157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7f01035a0 0x7fa7f00742d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.702+0000 7fa7e67fc640 1 -- 172.21.15.7:0/1290848157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f0074b60 con 0x7fa7f0102be0 2024-06-10T01:05:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.702+0000 7fa7e67fc640 1 --2- 172.21.15.7:0/1290848157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7f0102be0 0x7fa7f0073d90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8031440 tx=0x7fa7d8032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:39.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.702+0000 7fa7f485b640 1 -- 172.21.15.7:0/1290848157 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d803f030 con 0x7fa7f0102be0 2024-06-10T01:05:39.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.702+0000 7fa7f585d640 1 -- 172.21.15.7:0/1290848157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7f01a1e20 con 0x7fa7f0102be0 2024-06-10T01:05:39.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.702+0000 7fa7f485b640 1 -- 172.21.15.7:0/1290848157 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7d8003ce0 con 0x7fa7f0102be0 2024-06-10T01:05:39.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.702+0000 7fa7f485b640 1 -- 172.21.15.7:0/1290848157 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8035070 con 0x7fa7f0102be0 2024-06-10T01:05:39.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.702+0000 7fa7f585d640 1 -- 172.21.15.7:0/1290848157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7f01a2280 con 0x7fa7f0102be0 2024-06-10T01:05:39.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.703+0000 7fa7f585d640 1 -- 172.21.15.7:0/1290848157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7f0108d10 con 0x7fa7f0102be0 2024-06-10T01:05:39.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.704+0000 7fa7f485b640 1 -- 172.21.15.7:0/1290848157 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa7d8042030 con 0x7fa7f0102be0 2024-06-10T01:05:39.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.705+0000 7fa7f485b640 1 --2- 172.21.15.7:0/1290848157 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa7dc0a2b90 0x7fa7dc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:39.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.705+0000 7fa7eeffd640 1 --2- 172.21.15.7:0/1290848157 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa7dc0a2b90 0x7fa7dc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:39.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.705+0000 7fa7f485b640 1 -- 172.21.15.7:0/1290848157 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fa7d80ef9d0 con 0x7fa7f0102be0 2024-06-10T01:05:39.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.705+0000 7fa7eeffd640 1 --2- 172.21.15.7:0/1290848157 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa7dc0a2b90 0x7fa7dc0a5050 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0005fd0 tx=0x7fa7e0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:39.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:39 smithi175 ceph-mon[36636]: pgmap v420: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T01:05:39.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:39 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2633995688' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-10T01:05:39.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.709+0000 7fa7f485b640 1 -- 172.21.15.7:0/1290848157 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa7d80bc5b0 con 0x7fa7f0102be0 2024-06-10T01:05:39.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.894+0000 7fa7f585d640 1 -- 172.21.15.7:0/1290848157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa7f0062640 con 0x7fa7f0102be0 2024-06-10T01:05:39.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.894+0000 7fa7f485b640 1 -- 172.21.15.7:0/1290848157 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7fa7d80bbeb0 con 0x7fa7f0102be0 2024-06-10T01:05:39.891 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-06-10T01:05:39.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.896+0000 7fa7f585d640 1 -- 172.21.15.7:0/1290848157 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa7dc0a2b90 msgr2=0x7fa7dc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:39.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.896+0000 7fa7f585d640 1 --2- 172.21.15.7:0/1290848157 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa7dc0a2b90 0x7fa7dc0a5050 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0005fd0 tx=0x7fa7e0005850 comp rx=0 tx=0).stop 2024-06-10T01:05:39.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.896+0000 7fa7f585d640 1 -- 172.21.15.7:0/1290848157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7f0102be0 msgr2=0x7fa7f0073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:39.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.896+0000 7fa7f585d640 1 --2- 172.21.15.7:0/1290848157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7f0102be0 0x7fa7f0073d90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8031440 tx=0x7fa7d8032e20 comp rx=0 tx=0).stop 2024-06-10T01:05:39.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.896+0000 7fa7f585d640 1 -- 172.21.15.7:0/1290848157 shutdown_connections 2024-06-10T01:05:39.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.896+0000 7fa7f585d640 1 --2- 172.21.15.7:0/1290848157 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa7dc0a2b90 0x7fa7dc0a5050 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:39.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.896+0000 7fa7f585d640 1 --2- 172.21.15.7:0/1290848157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7f01035a0 0x7fa7f00742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:39.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.896+0000 7fa7f585d640 1 --2- 172.21.15.7:0/1290848157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7f0102be0 0x7fa7f0073d90 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:39.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.896+0000 7fa7f585d640 1 --2- 172.21.15.7:0/1290848157 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7f01019f0 0x7fa7f0075750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:39.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.896+0000 7fa7f585d640 1 -- 172.21.15.7:0/1290848157 >> 172.21.15.7:0/1290848157 conn(0x7fa7f00fcf60 msgr2=0x7fa7f0105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:39.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.896+0000 7fa7f585d640 1 -- 172.21.15.7:0/1290848157 shutdown_connections 2024-06-10T01:05:39.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:39.896+0000 7fa7f585d640 1 -- 172.21.15.7:0/1290848157 wait complete. 2024-06-10T01:05:39.904 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-06-10T01:05:40.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.015+0000 7f92ff914640 1 -- 172.21.15.7:0/3995773705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92f8102be0 msgr2=0x7f92f8103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:40.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.015+0000 7f92ff914640 1 --2- 172.21.15.7:0/3995773705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92f8102be0 0x7f92f8103060 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009900 tx=0x7f92ec02a410 comp rx=0 tx=0).stop 2024-06-10T01:05:40.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.015+0000 7f92ff914640 1 -- 172.21.15.7:0/3995773705 shutdown_connections 2024-06-10T01:05:40.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.015+0000 7f92ff914640 1 --2- 172.21.15.7:0/3995773705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92f81035a0 0x7f92f8109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:40.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.015+0000 7f92ff914640 1 --2- 172.21.15.7:0/3995773705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92f8102be0 0x7f92f8103060 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:40.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.015+0000 7f92ff914640 1 --2- 172.21.15.7:0/3995773705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92f81019f0 0x7f92f8101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:40.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.015+0000 7f92ff914640 1 -- 172.21.15.7:0/3995773705 >> 172.21.15.7:0/3995773705 conn(0x7f92f80fcf60 msgr2=0x7f92f80ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:40.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.015+0000 7f92ff914640 1 -- 172.21.15.7:0/3995773705 shutdown_connections 2024-06-10T01:05:40.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.015+0000 7f92ff914640 1 -- 172.21.15.7:0/3995773705 wait complete. 2024-06-10T01:05:40.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.015+0000 7f92ff914640 1 Processor -- start 2024-06-10T01:05:40.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.015+0000 7f92ff914640 1 -- start start 2024-06-10T01:05:40.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.016+0000 7f92ff914640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92f81019f0 0x7f92f8197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:40.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.016+0000 7f92ff914640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92f8102be0 0x7f92f8198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:40.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.016+0000 7f92ff914640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92f81035a0 0x7f92f819f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:40.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.016+0000 7f92ff914640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f92f810d670 con 0x7f92f81019f0 2024-06-10T01:05:40.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.016+0000 7f92ff914640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f92f810d4f0 con 0x7f92f81035a0 2024-06-10T01:05:40.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.016+0000 7f92ff914640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f92f810d7f0 con 0x7f92f8102be0 2024-06-10T01:05:40.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.016+0000 7f92fd6b0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92f81019f0 0x7f92f8197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:40.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.016+0000 7f92fd6b0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92f81019f0 0x7f92f8197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44872/0 (socket says 172.21.15.7:44872) 2024-06-10T01:05:40.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.016+0000 7f92fd6b0640 1 -- 172.21.15.7:0/443438500 learned_addr learned my addr 172.21.15.7:0/443438500 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:40.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.016+0000 7f92fdeb1640 1 --2- 172.21.15.7:0/443438500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92f81035a0 0x7f92f819f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:40.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.016+0000 7f92fceaf640 1 --2- 172.21.15.7:0/443438500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92f8102be0 0x7f92f8198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:40.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.016+0000 7f92fd6b0640 1 -- 172.21.15.7:0/443438500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92f8102be0 msgr2=0x7f92f8198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:40.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.016+0000 7f92fd6b0640 1 --2- 172.21.15.7:0/443438500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92f8102be0 0x7f92f8198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:40.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.016+0000 7f92fd6b0640 1 -- 172.21.15.7:0/443438500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92f81035a0 msgr2=0x7f92f819f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:40.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.016+0000 7f92fd6b0640 1 --2- 172.21.15.7:0/443438500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92f81035a0 0x7f92f819f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:40.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.016+0000 7f92fd6b0640 1 -- 172.21.15.7:0/443438500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f819f960 con 0x7f92f81019f0 2024-06-10T01:05:40.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.016+0000 7f92fd6b0640 1 --2- 172.21.15.7:0/443438500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92f81019f0 0x7f92f8197d40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f92f000d950 tx=0x7f92f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:40.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.016+0000 7f92f67fc640 1 -- 172.21.15.7:0/443438500 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f0017070 con 0x7f92f81019f0 2024-06-10T01:05:40.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.016+0000 7f92f67fc640 1 -- 172.21.15.7:0/443438500 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92f0009c30 con 0x7f92f81019f0 2024-06-10T01:05:40.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.016+0000 7f92ff914640 1 -- 172.21.15.7:0/443438500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f819fc50 con 0x7f92f81019f0 2024-06-10T01:05:40.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.016+0000 7f92f67fc640 1 -- 172.21.15.7:0/443438500 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f000f600 con 0x7f92f81019f0 2024-06-10T01:05:40.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.016+0000 7f92ff914640 1 -- 172.21.15.7:0/443438500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f81a0170 con 0x7f92f81019f0 2024-06-10T01:05:40.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.017+0000 7f92ff914640 1 -- 172.21.15.7:0/443438500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92f8108d60 con 0x7f92f81019f0 2024-06-10T01:05:40.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.018+0000 7f92f67fc640 1 -- 172.21.15.7:0/443438500 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f92f001a070 con 0x7f92f81019f0 2024-06-10T01:05:40.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.019+0000 7f92f67fc640 1 --2- 172.21.15.7:0/443438500 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f92e00a2c10 0x7f92e00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:40.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.019+0000 7f92fceaf640 1 --2- 172.21.15.7:0/443438500 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f92e00a2c10 0x7f92e00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:40.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.019+0000 7f92f67fc640 1 -- 172.21.15.7:0/443438500 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f92f00cee00 con 0x7f92f81019f0 2024-06-10T01:05:40.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.020+0000 7f92fceaf640 1 --2- 172.21.15.7:0/443438500 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f92e00a2c10 0x7f92e00a50d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f92ec000c00 tx=0x7f92ec0315d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:40.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.023+0000 7f92f67fc640 1 -- 172.21.15.7:0/443438500 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f92f0009de0 con 0x7f92f81019f0 2024-06-10T01:05:40.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.208+0000 7f92ff914640 1 -- 172.21.15.7:0/443438500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f92f80625a0 con 0x7f92f81019f0 2024-06-10T01:05:40.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.208+0000 7f92f67fc640 1 -- 172.21.15.7:0/443438500 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7f92f009b980 con 0x7f92f81019f0 2024-06-10T01:05:40.205 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-06-10T01:05:40.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.210+0000 7f92ff914640 1 -- 172.21.15.7:0/443438500 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f92e00a2c10 msgr2=0x7f92e00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:40.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.210+0000 7f92ff914640 1 --2- 172.21.15.7:0/443438500 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f92e00a2c10 0x7f92e00a50d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f92ec000c00 tx=0x7f92ec0315d0 comp rx=0 tx=0).stop 2024-06-10T01:05:40.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.210+0000 7f92ff914640 1 -- 172.21.15.7:0/443438500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92f81019f0 msgr2=0x7f92f8197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:40.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.210+0000 7f92ff914640 1 --2- 172.21.15.7:0/443438500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92f81019f0 0x7f92f8197d40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f92f000d950 tx=0x7f92f000de10 comp rx=0 tx=0).stop 2024-06-10T01:05:40.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.210+0000 7f92ff914640 1 -- 172.21.15.7:0/443438500 shutdown_connections 2024-06-10T01:05:40.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.210+0000 7f92ff914640 1 --2- 172.21.15.7:0/443438500 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f92e00a2c10 0x7f92e00a50d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:40.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.210+0000 7f92ff914640 1 --2- 172.21.15.7:0/443438500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92f81035a0 0x7f92f819f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:40.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.210+0000 7f92ff914640 1 --2- 172.21.15.7:0/443438500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92f8102be0 0x7f92f8198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:40.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.210+0000 7f92ff914640 1 --2- 172.21.15.7:0/443438500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92f81019f0 0x7f92f8197d40 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:40.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.210+0000 7f92ff914640 1 -- 172.21.15.7:0/443438500 >> 172.21.15.7:0/443438500 conn(0x7f92f80fcf60 msgr2=0x7f92f8105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:40.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.210+0000 7f92ff914640 1 -- 172.21.15.7:0/443438500 shutdown_connections 2024-06-10T01:05:40.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.210+0000 7f92ff914640 1 -- 172.21.15.7:0/443438500 wait complete. 2024-06-10T01:05:40.218 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-06-10T01:05:40.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.329+0000 7f118cfdc640 1 -- 172.21.15.7:0/211549494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1188104da0 msgr2=0x7f1188107190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:40.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.329+0000 7f118cfdc640 1 --2- 172.21.15.7:0/211549494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1188104da0 0x7f1188107190 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f117c009900 tx=0x7f117c02a410 comp rx=0 tx=0).stop 2024-06-10T01:05:40.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.329+0000 7f118cfdc640 1 -- 172.21.15.7:0/211549494 shutdown_connections 2024-06-10T01:05:40.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.329+0000 7f118cfdc640 1 --2- 172.21.15.7:0/211549494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11881076d0 0x7f1188109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:40.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.329+0000 7f118cfdc640 1 --2- 172.21.15.7:0/211549494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1188104da0 0x7f1188107190 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:40.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.329+0000 7f118cfdc640 1 --2- 172.21.15.7:0/211549494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1188102470 0x7f1188104860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:40.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.329+0000 7f118cfdc640 1 -- 172.21.15.7:0/211549494 >> 172.21.15.7:0/211549494 conn(0x7f11880fbe80 msgr2=0x7f11880fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:40.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.329+0000 7f118cfdc640 1 -- 172.21.15.7:0/211549494 shutdown_connections 2024-06-10T01:05:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.329+0000 7f118cfdc640 1 -- 172.21.15.7:0/211549494 wait complete. 2024-06-10T01:05:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.330+0000 7f118cfdc640 1 Processor -- start 2024-06-10T01:05:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.330+0000 7f118cfdc640 1 -- start start 2024-06-10T01:05:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.330+0000 7f118cfdc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1188102470 0x7f1188197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.330+0000 7f118cfdc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1188104da0 0x7f1188198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.330+0000 7f118cfdc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11881076d0 0x7f118819f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.330+0000 7f118cfdc640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f118810d660 con 0x7f1188104da0 2024-06-10T01:05:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.330+0000 7f118cfdc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f118810d4e0 con 0x7f11881076d0 2024-06-10T01:05:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.330+0000 7f118cfdc640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f118810d7e0 con 0x7f1188102470 2024-06-10T01:05:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.330+0000 7f1185d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1188104da0 0x7f1188198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.330+0000 7f1185d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1188104da0 0x7f1188198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44898/0 (socket says 172.21.15.7:44898) 2024-06-10T01:05:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.330+0000 7f1185d9b640 1 -- 172.21.15.7:0/3856965755 learned_addr learned my addr 172.21.15.7:0/3856965755 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.331+0000 7f118659c640 1 --2- 172.21.15.7:0/3856965755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1188102470 0x7f1188197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:40.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.331+0000 7f1186d9d640 1 --2- 172.21.15.7:0/3856965755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11881076d0 0x7f118819f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:40.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.331+0000 7f1185d9b640 1 -- 172.21.15.7:0/3856965755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1188102470 msgr2=0x7f1188197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:40.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.331+0000 7f1185d9b640 1 --2- 172.21.15.7:0/3856965755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1188102470 0x7f1188197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:40.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.331+0000 7f1185d9b640 1 -- 172.21.15.7:0/3856965755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11881076d0 msgr2=0x7f118819f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:40.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.331+0000 7f1185d9b640 1 --2- 172.21.15.7:0/3856965755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11881076d0 0x7f118819f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:40.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.331+0000 7f1185d9b640 1 -- 172.21.15.7:0/3856965755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f118819f960 con 0x7f1188104da0 2024-06-10T01:05:40.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.331+0000 7f1186d9d640 1 --2- 172.21.15.7:0/3856965755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11881076d0 0x7f118819f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:05:40.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.331+0000 7f1185d9b640 1 --2- 172.21.15.7:0/3856965755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1188104da0 0x7f1188198290 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f117c031610 tx=0x7f117c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:40.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.331+0000 7f11777fe640 1 -- 172.21.15.7:0/3856965755 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c03f030 con 0x7f1188104da0 2024-06-10T01:05:40.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.331+0000 7f118cfdc640 1 -- 172.21.15.7:0/3856965755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f118819fbf0 con 0x7f1188104da0 2024-06-10T01:05:40.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.331+0000 7f11777fe640 1 -- 172.21.15.7:0/3856965755 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f117c003ce0 con 0x7f1188104da0 2024-06-10T01:05:40.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.331+0000 7f11777fe640 1 -- 172.21.15.7:0/3856965755 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c035070 con 0x7f1188104da0 2024-06-10T01:05:40.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.331+0000 7f118cfdc640 1 -- 172.21.15.7:0/3856965755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11881a0130 con 0x7f1188104da0 2024-06-10T01:05:40.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.332+0000 7f118cfdc640 1 -- 172.21.15.7:0/3856965755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11880fd970 con 0x7f1188104da0 2024-06-10T01:05:40.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.333+0000 7f11777fe640 1 -- 172.21.15.7:0/3856965755 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f117c042030 con 0x7f1188104da0 2024-06-10T01:05:40.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.334+0000 7f11777fe640 1 --2- 172.21.15.7:0/3856965755 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f11700a2b90 0x7f11700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:40.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.334+0000 7f11777fe640 1 -- 172.21.15.7:0/3856965755 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f117c0efad0 con 0x7f1188104da0 2024-06-10T01:05:40.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.334+0000 7f118659c640 1 --2- 172.21.15.7:0/3856965755 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f11700a2b90 0x7f11700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:40.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.334+0000 7f118659c640 1 --2- 172.21.15.7:0/3856965755 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f11700a2b90 0x7f11700a5050 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f11780058c0 tx=0x7f1178005c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:40.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.338+0000 7f11777fe640 1 -- 172.21.15.7:0/3856965755 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f117c0bc6b0 con 0x7f1188104da0 2024-06-10T01:05:40.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:40 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2633995688' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-06-10T01:05:40.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:40 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:05:40.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:40 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/624483772' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:05:40.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:40 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1290848157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T01:05:40.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:40 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/443438500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-10T01:05:40.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.523+0000 7f118cfdc640 1 -- 172.21.15.7:0/3856965755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f11880625a0 con 0x7f1188104da0 2024-06-10T01:05:40.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.523+0000 7f11777fe640 1 -- 172.21.15.7:0/3856965755 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7f117c0bbfb0 con 0x7f1188104da0 2024-06-10T01:05:40.520 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-06-10T01:05:40.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.525+0000 7f118cfdc640 1 -- 172.21.15.7:0/3856965755 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f11700a2b90 msgr2=0x7f11700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:40.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.525+0000 7f118cfdc640 1 --2- 172.21.15.7:0/3856965755 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f11700a2b90 0x7f11700a5050 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f11780058c0 tx=0x7f1178005c80 comp rx=0 tx=0).stop 2024-06-10T01:05:40.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.525+0000 7f118cfdc640 1 -- 172.21.15.7:0/3856965755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1188104da0 msgr2=0x7f1188198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:40.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.525+0000 7f118cfdc640 1 --2- 172.21.15.7:0/3856965755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1188104da0 0x7f1188198290 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f117c031610 tx=0x7f117c0043b0 comp rx=0 tx=0).stop 2024-06-10T01:05:40.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.525+0000 7f118cfdc640 1 -- 172.21.15.7:0/3856965755 shutdown_connections 2024-06-10T01:05:40.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.525+0000 7f118cfdc640 1 --2- 172.21.15.7:0/3856965755 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f11700a2b90 0x7f11700a5050 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:40.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.525+0000 7f118cfdc640 1 --2- 172.21.15.7:0/3856965755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11881076d0 0x7f118819f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:40.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.525+0000 7f118cfdc640 1 --2- 172.21.15.7:0/3856965755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1188104da0 0x7f1188198290 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:40.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.525+0000 7f118cfdc640 1 --2- 172.21.15.7:0/3856965755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1188102470 0x7f1188197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:40.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.525+0000 7f118cfdc640 1 -- 172.21.15.7:0/3856965755 >> 172.21.15.7:0/3856965755 conn(0x7f11880fbe80 msgr2=0x7f1188102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:40.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.525+0000 7f118cfdc640 1 -- 172.21.15.7:0/3856965755 shutdown_connections 2024-06-10T01:05:40.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.525+0000 7f118cfdc640 1 -- 172.21.15.7:0/3856965755 wait complete. 2024-06-10T01:05:40.533 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-06-10T01:05:40.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:40 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2633995688' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-06-10T01:05:40.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:40 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:05:40.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:40 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/624483772' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:05:40.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:40 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1290848157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T01:05:40.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:40 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/443438500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-10T01:05:40.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.644+0000 7f0c61488640 1 -- 172.21.15.7:0/2637450626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c5c102bf0 msgr2=0x7f0c5c103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:40.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.644+0000 7f0c61488640 1 --2- 172.21.15.7:0/2637450626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c5c102bf0 0x7f0c5c103070 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0c50009900 tx=0x7f0c5002a410 comp rx=0 tx=0).stop 2024-06-10T01:05:40.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.644+0000 7f0c61488640 1 -- 172.21.15.7:0/2637450626 shutdown_connections 2024-06-10T01:05:40.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.644+0000 7f0c61488640 1 --2- 172.21.15.7:0/2637450626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c5c1035b0 0x7f0c5c109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:40.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.644+0000 7f0c61488640 1 --2- 172.21.15.7:0/2637450626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c5c102bf0 0x7f0c5c103070 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:40.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.644+0000 7f0c61488640 1 --2- 172.21.15.7:0/2637450626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c5c101a00 0x7f0c5c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:40.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.644+0000 7f0c61488640 1 -- 172.21.15.7:0/2637450626 >> 172.21.15.7:0/2637450626 conn(0x7f0c5c0fcf70 msgr2=0x7f0c5c0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:40.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.644+0000 7f0c61488640 1 -- 172.21.15.7:0/2637450626 shutdown_connections 2024-06-10T01:05:40.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.644+0000 7f0c61488640 1 -- 172.21.15.7:0/2637450626 wait complete. 2024-06-10T01:05:40.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.645+0000 7f0c61488640 1 Processor -- start 2024-06-10T01:05:40.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.645+0000 7f0c61488640 1 -- start start 2024-06-10T01:05:40.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.645+0000 7f0c61488640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c5c101a00 0x7f0c5c075750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:40.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.645+0000 7f0c61488640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c5c102bf0 0x7f0c5c073d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:40.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.645+0000 7f0c61488640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c5c1035b0 0x7f0c5c0742d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:40.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.645+0000 7f0c61488640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5c10d680 con 0x7f0c5c101a00 2024-06-10T01:05:40.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.645+0000 7f0c61488640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5c10d500 con 0x7f0c5c102bf0 2024-06-10T01:05:40.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.645+0000 7f0c61488640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5c10d800 con 0x7f0c5c1035b0 2024-06-10T01:05:40.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.645+0000 7f0c5affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c5c101a00 0x7f0c5c075750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:40.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.645+0000 7f0c5affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c5c101a00 0x7f0c5c075750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44918/0 (socket says 172.21.15.7:44918) 2024-06-10T01:05:40.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.645+0000 7f0c5affd640 1 -- 172.21.15.7:0/3694882071 learned_addr learned my addr 172.21.15.7:0/3694882071 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:40.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.645+0000 7f0c5b7fe640 1 --2- 172.21.15.7:0/3694882071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c5c1035b0 0x7f0c5c0742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:40.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.645+0000 7f0c5affd640 1 -- 172.21.15.7:0/3694882071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c5c1035b0 msgr2=0x7f0c5c0742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:40.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.645+0000 7f0c5affd640 1 --2- 172.21.15.7:0/3694882071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c5c1035b0 0x7f0c5c0742d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:40.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.645+0000 7f0c5affd640 1 -- 172.21.15.7:0/3694882071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c5c102bf0 msgr2=0x7f0c5c073d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:40.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.645+0000 7f0c5a7fc640 1 --2- 172.21.15.7:0/3694882071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c5c102bf0 0x7f0c5c073d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:40.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.645+0000 7f0c5affd640 1 --2- 172.21.15.7:0/3694882071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c5c102bf0 0x7f0c5c073d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:40.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.646+0000 7f0c5affd640 1 -- 172.21.15.7:0/3694882071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c5c074b60 con 0x7f0c5c101a00 2024-06-10T01:05:40.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.646+0000 7f0c5a7fc640 1 --2- 172.21.15.7:0/3694882071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c5c102bf0 0x7f0c5c073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:05:40.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.646+0000 7f0c5affd640 1 --2- 172.21.15.7:0/3694882071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c5c101a00 0x7f0c5c075750 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c00e9c0 tx=0x7f0c4c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:40.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.646+0000 7f0c43fff640 1 -- 172.21.15.7:0/3694882071 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c00cd80 con 0x7f0c5c101a00 2024-06-10T01:05:40.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.646+0000 7f0c43fff640 1 -- 172.21.15.7:0/3694882071 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c4c0035f0 con 0x7f0c5c101a00 2024-06-10T01:05:40.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.646+0000 7f0c61488640 1 -- 172.21.15.7:0/3694882071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c5c1a1e50 con 0x7f0c5c101a00 2024-06-10T01:05:40.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.646+0000 7f0c43fff640 1 -- 172.21.15.7:0/3694882071 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c010690 con 0x7f0c5c101a00 2024-06-10T01:05:40.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.646+0000 7f0c61488640 1 -- 172.21.15.7:0/3694882071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c5c1a2260 con 0x7f0c5c101a00 2024-06-10T01:05:40.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.647+0000 7f0c61488640 1 -- 172.21.15.7:0/3694882071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c5c108da0 con 0x7f0c5c101a00 2024-06-10T01:05:40.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.648+0000 7f0c43fff640 1 -- 172.21.15.7:0/3694882071 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0c4c0037a0 con 0x7f0c5c101a00 2024-06-10T01:05:40.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.649+0000 7f0c43fff640 1 --2- 172.21.15.7:0/3694882071 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0c440a2b90 0x7f0c440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:40.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.649+0000 7f0c5a7fc640 1 --2- 172.21.15.7:0/3694882071 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0c440a2b90 0x7f0c440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:40.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.649+0000 7f0c43fff640 1 -- 172.21.15.7:0/3694882071 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f0c4c0cf940 con 0x7f0c5c101a00 2024-06-10T01:05:40.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.650+0000 7f0c5a7fc640 1 --2- 172.21.15.7:0/3694882071 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0c440a2b90 0x7f0c440a5050 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0c50000c00 tx=0x7f0c50031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:40.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.653+0000 7f0c43fff640 1 -- 172.21.15.7:0/3694882071 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0c4c09c520 con 0x7f0c5c101a00 2024-06-10T01:05:40.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:40 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2633995688' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-06-10T01:05:40.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:40 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:05:40.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:40 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/624483772' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:05:40.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:40 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1290848157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T01:05:40.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:40 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/443438500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-10T01:05:40.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.839+0000 7f0c61488640 1 -- 172.21.15.7:0/3694882071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0c5c02d2e0 con 0x7f0c5c101a00 2024-06-10T01:05:40.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.840+0000 7f0c43fff640 1 -- 172.21.15.7:0/3694882071 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v99) ==== 103+0+11 (secure 0 0 0) 0x7f0c4c09be20 con 0x7f0c5c101a00 2024-06-10T01:05:40.837 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-06-10T01:05:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.842+0000 7f0c61488640 1 -- 172.21.15.7:0/3694882071 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0c440a2b90 msgr2=0x7f0c440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.842+0000 7f0c61488640 1 --2- 172.21.15.7:0/3694882071 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0c440a2b90 0x7f0c440a5050 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0c50000c00 tx=0x7f0c50031400 comp rx=0 tx=0).stop 2024-06-10T01:05:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.842+0000 7f0c61488640 1 -- 172.21.15.7:0/3694882071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c5c101a00 msgr2=0x7f0c5c075750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.842+0000 7f0c61488640 1 --2- 172.21.15.7:0/3694882071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c5c101a00 0x7f0c5c075750 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c00e9c0 tx=0x7f0c4c00ee80 comp rx=0 tx=0).stop 2024-06-10T01:05:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.842+0000 7f0c61488640 1 -- 172.21.15.7:0/3694882071 shutdown_connections 2024-06-10T01:05:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.842+0000 7f0c61488640 1 --2- 172.21.15.7:0/3694882071 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0c440a2b90 0x7f0c440a5050 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.842+0000 7f0c61488640 1 --2- 172.21.15.7:0/3694882071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c5c1035b0 0x7f0c5c0742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.842+0000 7f0c61488640 1 --2- 172.21.15.7:0/3694882071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c5c102bf0 0x7f0c5c073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.842+0000 7f0c61488640 1 --2- 172.21.15.7:0/3694882071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c5c101a00 0x7f0c5c075750 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:40.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.842+0000 7f0c61488640 1 -- 172.21.15.7:0/3694882071 >> 172.21.15.7:0/3694882071 conn(0x7f0c5c0fcf70 msgr2=0x7f0c5c105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:40.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.842+0000 7f0c61488640 1 -- 172.21.15.7:0/3694882071 shutdown_connections 2024-06-10T01:05:40.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:40.842+0000 7f0c61488640 1 -- 172.21.15.7:0/3694882071 wait complete. 2024-06-10T01:05:40.850 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-06-10T01:05:40.850 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:05:41.106 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:05:41.455 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:41 smithi007 ceph-mon[32398]: pgmap v421: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:05:41.455 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:41 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3856965755' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-10T01:05:41.455 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:41 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3694882071' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-10T01:05:41.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:41 smithi078 ceph-mon[36683]: pgmap v421: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:05:41.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:41 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3856965755' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-10T01:05:41.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:41 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3694882071' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-10T01:05:41.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:41 smithi175 ceph-mon[36636]: pgmap v421: 129 pgs: 129 active+clean; 643 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:05:41.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:41 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3856965755' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-10T01:05:41.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:41 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3694882071' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-10T01:05:42.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.756+0000 7f0b1103c700 1 -- 172.21.15.7:0/2076539781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b0c102740 msgr2=0x7f0b0c102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:42.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.756+0000 7f0b1103c700 1 --2- 172.21.15.7:0/2076539781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b0c102740 0x7f0b0c102b20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0afc00b0d0 tx=0x7f0afc00b3e0 comp rx=0 tx=0).stop 2024-06-10T01:05:42.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.756+0000 7f0b1103c700 1 -- 172.21.15.7:0/2076539781 shutdown_connections 2024-06-10T01:05:42.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.756+0000 7f0b1103c700 1 --2- 172.21.15.7:0/2076539781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b0c107840 0x7f0b0c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:42.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.756+0000 7f0b1103c700 1 --2- 172.21.15.7:0/2076539781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b0c102740 0x7f0b0c102b20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.756+0000 7f0b1103c700 1 --2- 172.21.15.7:0/2076539781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b0c1030f0 0x7f0b0c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.756+0000 7f0b1103c700 1 -- 172.21.15.7:0/2076539781 >> 172.21.15.7:0/2076539781 conn(0x7f0b0c0795f0 msgr2=0x7f0b0c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.756+0000 7f0b1103c700 1 -- 172.21.15.7:0/2076539781 shutdown_connections 2024-06-10T01:05:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.756+0000 7f0b1103c700 1 -- 172.21.15.7:0/2076539781 wait complete. 2024-06-10T01:05:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.757+0000 7f0b1103c700 1 Processor -- start 2024-06-10T01:05:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.757+0000 7f0b1103c700 1 -- start start 2024-06-10T01:05:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.757+0000 7f0b1103c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b0c102740 0x7f0b0c19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.757+0000 7f0b1103c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b0c1030f0 0x7f0b0c19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.757+0000 7f0b1103c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b0c107840 0x7f0b0c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.757+0000 7f0b1103c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c1127d0 con 0x7f0b0c1030f0 2024-06-10T01:05:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.757+0000 7f0b1103c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c112650 con 0x7f0b0c107840 2024-06-10T01:05:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.757+0000 7f0b1103c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c112950 con 0x7f0b0c102740 2024-06-10T01:05:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.758+0000 7f0b0259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b0c1030f0 0x7f0b0c19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.758+0000 7f0b0ad9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b0c102740 0x7f0b0c19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.758+0000 7f0b0259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b0c1030f0 0x7f0b0c19edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58536/0 (socket says 172.21.15.7:58536) 2024-06-10T01:05:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.758+0000 7f0b0259c700 1 -- 172.21.15.7:0/1936040630 learned_addr learned my addr 172.21.15.7:0/1936040630 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.758+0000 7f0b0b59e700 1 --2- 172.21.15.7:0/1936040630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b0c107840 0x7f0b0c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.758+0000 7f0b0259c700 1 -- 172.21.15.7:0/1936040630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b0c102740 msgr2=0x7f0b0c19e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.758+0000 7f0b0259c700 1 --2- 172.21.15.7:0/1936040630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b0c102740 0x7f0b0c19e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.758+0000 7f0b0259c700 1 -- 172.21.15.7:0/1936040630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b0c107840 msgr2=0x7f0b0c198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.758+0000 7f0b0259c700 1 --2- 172.21.15.7:0/1936040630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b0c107840 0x7f0b0c198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.758+0000 7f0b0259c700 1 -- 172.21.15.7:0/1936040630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0c1991b0 con 0x7f0b0c1030f0 2024-06-10T01:05:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.758+0000 7f0b0259c700 1 --2- 172.21.15.7:0/1936040630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b0c1030f0 0x7f0b0c19edd0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0af400b530 tx=0x7f0af400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.758+0000 7f0b08d99700 1 -- 172.21.15.7:0/1936040630 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af4015020 con 0x7f0b0c1030f0 2024-06-10T01:05:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.758+0000 7f0b1103c700 1 -- 172.21.15.7:0/1936040630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b0c1994a0 con 0x7f0b0c1030f0 2024-06-10T01:05:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.758+0000 7f0b08d99700 1 -- 172.21.15.7:0/1936040630 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0af4011d10 con 0x7f0b0c1030f0 2024-06-10T01:05:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.758+0000 7f0b08d99700 1 -- 172.21.15.7:0/1936040630 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af400f2e0 con 0x7f0b0c1030f0 2024-06-10T01:05:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.758+0000 7f0b1103c700 1 -- 172.21.15.7:0/1936040630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b0c1a5760 con 0x7f0b0c1030f0 2024-06-10T01:05:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.759+0000 7f0b1103c700 1 -- 172.21.15.7:0/1936040630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b0c10aa80 con 0x7f0b0c1030f0 2024-06-10T01:05:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.761+0000 7f0b08d99700 1 -- 172.21.15.7:0/1936040630 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0af4018020 con 0x7f0b0c1030f0 2024-06-10T01:05:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.761+0000 7f0b08d99700 1 --2- 172.21.15.7:0/1936040630 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0af80a2c20 0x7f0af80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:42.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.761+0000 7f0b0ad9d700 1 --2- 172.21.15.7:0/1936040630 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0af80a2c20 0x7f0af80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:42.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.761+0000 7f0b08d99700 1 -- 172.21.15.7:0/1936040630 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f0af40cead0 con 0x7f0b0c1030f0 2024-06-10T01:05:42.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.762+0000 7f0b0ad9d700 1 --2- 172.21.15.7:0/1936040630 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0af80a2c20 0x7f0af80a50e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0afc009fd0 tx=0x7f0afc00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:42.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:42.766+0000 7f0b08d99700 1 -- 172.21.15.7:0/1936040630 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0af401f020 con 0x7f0b0c1030f0 2024-06-10T01:05:43.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.021+0000 7f0b1103c700 1 -- 172.21.15.7:0/1936040630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b0c069180 con 0x7f0b0c1030f0 2024-06-10T01:05:43.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.022+0000 7f0b08d99700 1 -- 172.21.15.7:0/1936040630 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11855 (secure 0 0 0) 0x7f0af409b6b0 con 0x7f0b0c1030f0 2024-06-10T01:05:43.018 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:05:43.019 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:05:39.175358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/2784910673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2784910673},{"type":"v1","addr":"172.21.15.175:6835","nonce":2784910673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24434,24436,24451]},"id":1}]} 2024-06-10T01:05:43.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.023+0000 7f0b1103c700 1 -- 172.21.15.7:0/1936040630 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0af80a2c20 msgr2=0x7f0af80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:43.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.023+0000 7f0b1103c700 1 --2- 172.21.15.7:0/1936040630 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0af80a2c20 0x7f0af80a50e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0afc009fd0 tx=0x7f0afc00b040 comp rx=0 tx=0).stop 2024-06-10T01:05:43.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.023+0000 7f0b1103c700 1 -- 172.21.15.7:0/1936040630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b0c1030f0 msgr2=0x7f0b0c19edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:43.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.023+0000 7f0b1103c700 1 --2- 172.21.15.7:0/1936040630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b0c1030f0 0x7f0b0c19edd0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0af400b530 tx=0x7f0af400b840 comp rx=0 tx=0).stop 2024-06-10T01:05:43.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.023+0000 7f0b1103c700 1 -- 172.21.15.7:0/1936040630 shutdown_connections 2024-06-10T01:05:43.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.023+0000 7f0b1103c700 1 --2- 172.21.15.7:0/1936040630 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0af80a2c20 0x7f0af80a50e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:43.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.023+0000 7f0b1103c700 1 --2- 172.21.15.7:0/1936040630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b0c107840 0x7f0b0c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:43.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.023+0000 7f0b1103c700 1 --2- 172.21.15.7:0/1936040630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b0c102740 0x7f0b0c19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:43.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.023+0000 7f0b1103c700 1 --2- 172.21.15.7:0/1936040630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b0c1030f0 0x7f0b0c19edd0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:43.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.023+0000 7f0b1103c700 1 -- 172.21.15.7:0/1936040630 >> 172.21.15.7:0/1936040630 conn(0x7f0b0c0795f0 msgr2=0x7f0b0c100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:43.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.023+0000 7f0b1103c700 1 -- 172.21.15.7:0/1936040630 shutdown_connections 2024-06-10T01:05:43.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.023+0000 7f0b1103c700 1 -- 172.21.15.7:0/1936040630 wait complete. 2024-06-10T01:05:43.022 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-06-10T01:05:43.335 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:43 smithi007 ceph-mon[32398]: pgmap v422: 129 pgs: 129 active+clean; 644 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 511 B/s wr, 6 op/s 2024-06-10T01:05:43.335 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:43 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1936040630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:05:43.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:43 smithi078 ceph-mon[36683]: pgmap v422: 129 pgs: 129 active+clean; 644 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 511 B/s wr, 6 op/s 2024-06-10T01:05:43.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:43 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1936040630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:05:43.676 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-06-10T01:05:43.677 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-06-10T01:05:43.677 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-10T01:05:43.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:43 smithi175 ceph-mon[36636]: pgmap v422: 129 pgs: 129 active+clean; 644 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 511 B/s wr, 6 op/s 2024-06-10T01:05:43.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:43 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1936040630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:05:43.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.790+0000 7f51e3464640 1 -- 172.21.15.7:0/1472742326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51dc102590 msgr2=0x7f51dc108e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:44.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.790+0000 7f51e3464640 1 --2- 172.21.15.7:0/1472742326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51dc102590 0x7f51dc108e20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f51d4009ca0 tx=0x7f51d402a8c0 comp rx=0 tx=0).stop 2024-06-10T01:05:44.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.791+0000 7f51e3464640 1 -- 172.21.15.7:0/1472742326 shutdown_connections 2024-06-10T01:05:44.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.791+0000 7f51e3464640 1 --2- 172.21.15.7:0/1472742326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51dc102590 0x7f51dc108e20 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:44.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.791+0000 7f51e3464640 1 --2- 172.21.15.7:0/1472742326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51dc101bd0 0x7f51dc102050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:44.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.791+0000 7f51e3464640 1 --2- 172.21.15.7:0/1472742326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51dc1009e0 0x7f51dc100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:44.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.791+0000 7f51e3464640 1 -- 172.21.15.7:0/1472742326 >> 172.21.15.7:0/1472742326 conn(0x7f51dc0fbf70 msgr2=0x7f51dc0fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:44.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.791+0000 7f51e3464640 1 -- 172.21.15.7:0/1472742326 shutdown_connections 2024-06-10T01:05:44.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.791+0000 7f51e3464640 1 -- 172.21.15.7:0/1472742326 wait complete. 2024-06-10T01:05:44.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.791+0000 7f51e3464640 1 Processor -- start 2024-06-10T01:05:44.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.791+0000 7f51e3464640 1 -- start start 2024-06-10T01:05:44.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.792+0000 7f51e3464640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51dc1009e0 0x7f51dc06ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:44.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.792+0000 7f51e3464640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51dc101bd0 0x7f51dc19c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:44.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.792+0000 7f51e3464640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51dc102590 0x7f51dc19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:44.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.792+0000 7f51e3464640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc10c580 con 0x7f51dc1009e0 2024-06-10T01:05:44.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.792+0000 7f51e3464640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc10c400 con 0x7f51dc102590 2024-06-10T01:05:44.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.792+0000 7f51e3464640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc10c700 con 0x7f51dc101bd0 2024-06-10T01:05:44.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.792+0000 7f51e1200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51dc1009e0 0x7f51dc06ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:44.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.792+0000 7f51e1200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51dc1009e0 0x7f51dc06ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58550/0 (socket says 172.21.15.7:58550) 2024-06-10T01:05:44.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.792+0000 7f51e1200640 1 -- 172.21.15.7:0/4133097436 learned_addr learned my addr 172.21.15.7:0/4133097436 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.792+0000 7f51e1a01640 1 --2- 172.21.15.7:0/4133097436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51dc102590 0x7f51dc19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.792+0000 7f51e09ff640 1 --2- 172.21.15.7:0/4133097436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51dc101bd0 0x7f51dc19c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.792+0000 7f51e1200640 1 -- 172.21.15.7:0/4133097436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51dc101bd0 msgr2=0x7f51dc19c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.792+0000 7f51e1200640 1 --2- 172.21.15.7:0/4133097436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51dc101bd0 0x7f51dc19c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.792+0000 7f51e1200640 1 -- 172.21.15.7:0/4133097436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51dc102590 msgr2=0x7f51dc19f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.792+0000 7f51e1200640 1 --2- 172.21.15.7:0/4133097436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51dc102590 0x7f51dc19f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.792+0000 7f51e1200640 1 -- 172.21.15.7:0/4133097436 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51dc19f910 con 0x7f51dc1009e0 2024-06-10T01:05:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.792+0000 7f51e09ff640 1 --2- 172.21.15.7:0/4133097436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51dc101bd0 0x7f51dc19c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:05:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.792+0000 7f51e1200640 1 --2- 172.21.15.7:0/4133097436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51dc1009e0 0x7f51dc06ac20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f51d800e9c0 tx=0x7f51d800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.792+0000 7f51d27fc640 1 -- 172.21.15.7:0/4133097436 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d800cde0 con 0x7f51dc1009e0 2024-06-10T01:05:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.792+0000 7f51e3464640 1 -- 172.21.15.7:0/4133097436 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51dc19fba0 con 0x7f51dc1009e0 2024-06-10T01:05:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.792+0000 7f51d27fc640 1 -- 172.21.15.7:0/4133097436 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51d8010910 con 0x7f51dc1009e0 2024-06-10T01:05:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.793+0000 7f51d27fc640 1 -- 172.21.15.7:0/4133097436 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d8018a10 con 0x7f51dc1009e0 2024-06-10T01:05:44.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.793+0000 7f51e3464640 1 -- 172.21.15.7:0/4133097436 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51dc1a00f0 con 0x7f51dc1009e0 2024-06-10T01:05:44.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.793+0000 7f51e3464640 1 -- 172.21.15.7:0/4133097436 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51dc107ca0 con 0x7f51dc1009e0 2024-06-10T01:05:44.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.795+0000 7f51d27fc640 1 -- 172.21.15.7:0/4133097436 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f51d801f020 con 0x7f51dc1009e0 2024-06-10T01:05:44.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.795+0000 7f51d27fc640 1 --2- 172.21.15.7:0/4133097436 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f51c80a2b90 0x7f51c80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:44.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.795+0000 7f51e09ff640 1 --2- 172.21.15.7:0/4133097436 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f51c80a2b90 0x7f51c80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:44.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.795+0000 7f51d27fc640 1 -- 172.21.15.7:0/4133097436 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f51d80cf680 con 0x7f51dc1009e0 2024-06-10T01:05:44.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.796+0000 7f51e09ff640 1 --2- 172.21.15.7:0/4133097436 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f51c80a2b90 0x7f51c80a5050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f51dc06a990 tx=0x7f51cc005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:44.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.800+0000 7f51d27fc640 1 -- 172.21.15.7:0/4133097436 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f51d809c260 con 0x7f51dc1009e0 2024-06-10T01:05:44.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.981+0000 7f51e3464640 1 -- 172.21.15.7:0/4133097436 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f51dc061de0 con 0x7f51c80a2b90 2024-06-10T01:05:44.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.991+0000 7f51d27fc640 1 -- 172.21.15.7:0/4133097436 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340766 (secure 0 0 0) 0x7f51dc061de0 con 0x7f51c80a2b90 2024-06-10T01:05:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:05:44.045 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-06-10T01:05:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.995+0000 7f51e3464640 1 -- 172.21.15.7:0/4133097436 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f51c80a2b90 msgr2=0x7f51c80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.995+0000 7f51e3464640 1 --2- 172.21.15.7:0/4133097436 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f51c80a2b90 0x7f51c80a5050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f51dc06a990 tx=0x7f51cc005e50 comp rx=0 tx=0).stop 2024-06-10T01:05:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.995+0000 7f51e3464640 1 -- 172.21.15.7:0/4133097436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51dc1009e0 msgr2=0x7f51dc06ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.995+0000 7f51e3464640 1 --2- 172.21.15.7:0/4133097436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51dc1009e0 0x7f51dc06ac20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f51d800e9c0 tx=0x7f51d800ee80 comp rx=0 tx=0).stop 2024-06-10T01:05:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.995+0000 7f51e3464640 1 -- 172.21.15.7:0/4133097436 shutdown_connections 2024-06-10T01:05:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.995+0000 7f51e3464640 1 --2- 172.21.15.7:0/4133097436 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f51c80a2b90 0x7f51c80a5050 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.995+0000 7f51e3464640 1 --2- 172.21.15.7:0/4133097436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51dc102590 0x7f51dc19f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:44.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.995+0000 7f51e3464640 1 --2- 172.21.15.7:0/4133097436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51dc101bd0 0x7f51dc19c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:44.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.995+0000 7f51e3464640 1 --2- 172.21.15.7:0/4133097436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51dc1009e0 0x7f51dc06ac20 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:44.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.995+0000 7f51e3464640 1 -- 172.21.15.7:0/4133097436 >> 172.21.15.7:0/4133097436 conn(0x7f51dc0fbf70 msgr2=0x7f51dc104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:44.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.995+0000 7f51e3464640 1 -- 172.21.15.7:0/4133097436 shutdown_connections 2024-06-10T01:05:44.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:43.995+0000 7f51e3464640 1 -- 172.21.15.7:0/4133097436 wait complete. 2024-06-10T01:05:44.048 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":423,"stamp":"2024-06-10T01:05:43.597017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659135,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1144,"num_read_kb":690,"num_write":366,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":793,"ondisk_log_size":793,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362468,"kb_used_data":13316,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579356,"statfs":{"total":1151940427776,"available":1151569260544,"internally_reserved":0,"allocated":13635584,"data_stored":8789183,"data_compressed":496626,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":64,"num_read_kb":30,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003015"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.569445+0000","last_change":"2024-06-10T01:01:47.244728+0000","last_active":"2024-06-10T01:04:35.569445+0000","last_peered":"2024-06-10T01:04:35.569445+0000","last_clean":"2024-06-10T01:04:35.569445+0000","last_became_active":"2024-06-10T01:01:47.244198+0000","last_became_peered":"2024-06-10T01:01:47.244198+0000","last_unstale":"2024-06-10T01:04:35.569445+0000","last_undegraded":"2024-06-10T01:04:35.569445+0000","last_fullsized":"2024-06-10T01:04:35.569445+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:58:06.257952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.557217+0000","last_change":"2024-06-10T01:01:47.247620+0000","last_active":"2024-06-10T01:04:35.557217+0000","last_peered":"2024-06-10T01:04:35.557217+0000","last_clean":"2024-06-10T01:04:35.557217+0000","last_became_active":"2024-06-10T01:01:47.245475+0000","last_became_peered":"2024-06-10T01:01:47.245475+0000","last_unstale":"2024-06-10T01:04:35.557217+0000","last_undegraded":"2024-06-10T01:04:35.557217+0000","last_fullsized":"2024-06-10T01:04:35.557217+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:29:09.367411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.569660+0000","last_change":"2024-06-10T01:01:47.252414+0000","last_active":"2024-06-10T01:04:35.569660+0000","last_peered":"2024-06-10T01:04:35.569660+0000","last_clean":"2024-06-10T01:04:35.569660+0000","last_became_active":"2024-06-10T01:01:47.252063+0000","last_became_peered":"2024-06-10T01:01:47.252063+0000","last_unstale":"2024-06-10T01:04:35.569660+0000","last_undegraded":"2024-06-10T01:04:35.569660+0000","last_fullsized":"2024-06-10T01:04:35.569660+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:54:53.966209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.553564+0000","last_change":"2024-06-10T01:01:47.254875+0000","last_active":"2024-06-10T01:04:35.553564+0000","last_peered":"2024-06-10T01:04:35.553564+0000","last_clean":"2024-06-10T01:04:35.553564+0000","last_became_active":"2024-06-10T01:01:47.254542+0000","last_became_peered":"2024-06-10T01:01:47.254542+0000","last_unstale":"2024-06-10T01:04:35.553564+0000","last_undegraded":"2024-06-10T01:04:35.553564+0000","last_fullsized":"2024-06-10T01:04:35.553564+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:02:28.393133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558663+0000","last_change":"2024-06-10T01:01:47.242596+0000","last_active":"2024-06-10T01:04:35.558663+0000","last_peered":"2024-06-10T01:04:35.558663+0000","last_clean":"2024-06-10T01:04:35.558663+0000","last_became_active":"2024-06-10T01:01:47.242308+0000","last_became_peered":"2024-06-10T01:01:47.242308+0000","last_unstale":"2024-06-10T01:04:35.558663+0000","last_undegraded":"2024-06-10T01:04:35.558663+0000","last_fullsized":"2024-06-10T01:04:35.558663+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:11:26.599954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.550600+0000","last_change":"2024-06-10T01:01:47.249085+0000","last_active":"2024-06-10T01:04:35.550600+0000","last_peered":"2024-06-10T01:04:35.550600+0000","last_clean":"2024-06-10T01:04:35.550600+0000","last_became_active":"2024-06-10T01:01:47.248749+0000","last_became_peered":"2024-06-10T01:01:47.248749+0000","last_unstale":"2024-06-10T01:04:35.550600+0000","last_undegraded":"2024-06-10T01:04:35.550600+0000","last_fullsized":"2024-06-10T01:04:35.550600+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:42:48.734566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562583+0000","last_change":"2024-06-10T01:01:47.247334+0000","last_active":"2024-06-10T01:04:35.562583+0000","last_peered":"2024-06-10T01:04:35.562583+0000","last_clean":"2024-06-10T01:04:35.562583+0000","last_became_active":"2024-06-10T01:01:47.246964+0000","last_became_peered":"2024-06-10T01:01:47.246964+0000","last_unstale":"2024-06-10T01:04:35.562583+0000","last_undegraded":"2024-06-10T01:04:35.562583+0000","last_fullsized":"2024-06-10T01:04:35.562583+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:05:36.034950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.563056+0000","last_change":"2024-06-10T01:01:47.242523+0000","last_active":"2024-06-10T01:04:35.563056+0000","last_peered":"2024-06-10T01:04:35.563056+0000","last_clean":"2024-06-10T01:04:35.563056+0000","last_became_active":"2024-06-10T01:01:47.242198+0000","last_became_peered":"2024-06-10T01:01:47.242198+0000","last_unstale":"2024-06-10T01:04:35.563056+0000","last_undegraded":"2024-06-10T01:04:35.563056+0000","last_fullsized":"2024-06-10T01:04:35.563056+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:39:07.055970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.567116+0000","last_change":"2024-06-10T01:01:47.245277+0000","last_active":"2024-06-10T01:04:35.567116+0000","last_peered":"2024-06-10T01:04:35.567116+0000","last_clean":"2024-06-10T01:04:35.567116+0000","last_became_active":"2024-06-10T01:01:47.244932+0000","last_became_peered":"2024-06-10T01:01:47.244932+0000","last_unstale":"2024-06-10T01:04:35.567116+0000","last_undegraded":"2024-06-10T01:04:35.567116+0000","last_fullsized":"2024-06-10T01:04:35.567116+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:56:24.423713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559784+0000","last_change":"2024-06-10T01:01:47.255531+0000","last_active":"2024-06-10T01:04:35.559784+0000","last_peered":"2024-06-10T01:04:35.559784+0000","last_clean":"2024-06-10T01:04:35.559784+0000","last_became_active":"2024-06-10T01:01:47.255150+0000","last_became_peered":"2024-06-10T01:01:47.255150+0000","last_unstale":"2024-06-10T01:04:35.559784+0000","last_undegraded":"2024-06-10T01:04:35.559784+0000","last_fullsized":"2024-06-10T01:04:35.559784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:51:34.561396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559164+0000","last_change":"2024-06-10T01:01:47.249832+0000","last_active":"2024-06-10T01:04:35.559164+0000","last_peered":"2024-06-10T01:04:35.559164+0000","last_clean":"2024-06-10T01:04:35.559164+0000","last_became_active":"2024-06-10T01:01:47.249303+0000","last_became_peered":"2024-06-10T01:01:47.249303+0000","last_unstale":"2024-06-10T01:04:35.559164+0000","last_undegraded":"2024-06-10T01:04:35.559164+0000","last_fullsized":"2024-06-10T01:04:35.559164+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:15:21.930157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.551184+0000","last_change":"2024-06-10T01:01:47.255927+0000","last_active":"2024-06-10T01:04:35.551184+0000","last_peered":"2024-06-10T01:04:35.551184+0000","last_clean":"2024-06-10T01:04:35.551184+0000","last_became_active":"2024-06-10T01:01:47.255591+0000","last_became_peered":"2024-06-10T01:01:47.255591+0000","last_unstale":"2024-06-10T01:04:35.551184+0000","last_undegraded":"2024-06-10T01:04:35.551184+0000","last_fullsized":"2024-06-10T01:04:35.551184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:48:06.494430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.557880+0000","last_change":"2024-06-10T01:02:37.372069+0000","last_active":"2024-06-10T01:04:35.557880+0000","last_peered":"2024-06-10T01:04:35.557880+0000","last_clean":"2024-06-10T01:04:35.557880+0000","last_became_active":"2024-06-10T01:02:37.371599+0000","last_became_peered":"2024-06-10T01:02:37.371599+0000","last_unstale":"2024-06-10T01:04:35.557880+0000","last_undegraded":"2024-06-10T01:04:35.557880+0000","last_fullsized":"2024-06-10T01:04:35.557880+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:54:34.091576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.551120+0000","last_change":"2024-06-10T01:01:47.249553+0000","last_active":"2024-06-10T01:04:35.551120+0000","last_peered":"2024-06-10T01:04:35.551120+0000","last_clean":"2024-06-10T01:04:35.551120+0000","last_became_active":"2024-06-10T01:01:47.249202+0000","last_became_peered":"2024-06-10T01:01:47.249202+0000","last_unstale":"2024-06-10T01:04:35.551120+0000","last_undegraded":"2024-06-10T01:04:35.551120+0000","last_fullsized":"2024-06-10T01:04:35.551120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:23:57.882264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.564289+0000","last_change":"2024-06-10T01:01:47.253987+0000","last_active":"2024-06-10T01:04:35.564289+0000","last_peered":"2024-06-10T01:04:35.564289+0000","last_clean":"2024-06-10T01:04:35.564289+0000","last_became_active":"2024-06-10T01:01:47.253578+0000","last_became_peered":"2024-06-10T01:01:47.253578+0000","last_unstale":"2024-06-10T01:04:35.564289+0000","last_undegraded":"2024-06-10T01:04:35.564289+0000","last_fullsized":"2024-06-10T01:04:35.564289+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:35:30.585482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.567601+0000","last_change":"2024-06-10T01:01:47.247089+0000","last_active":"2024-06-10T01:04:35.567601+0000","last_peered":"2024-06-10T01:04:35.567601+0000","last_clean":"2024-06-10T01:04:35.567601+0000","last_became_active":"2024-06-10T01:01:47.246066+0000","last_became_peered":"2024-06-10T01:01:47.246066+0000","last_unstale":"2024-06-10T01:04:35.567601+0000","last_undegraded":"2024-06-10T01:04:35.567601+0000","last_fullsized":"2024-06-10T01:04:35.567601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:58:12.092981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.569804+0000","last_change":"2024-06-10T01:01:47.244491+0000","last_active":"2024-06-10T01:04:35.569804+0000","last_peered":"2024-06-10T01:04:35.569804+0000","last_clean":"2024-06-10T01:04:35.569804+0000","last_became_active":"2024-06-10T01:01:47.243747+0000","last_became_peered":"2024-06-10T01:01:47.243747+0000","last_unstale":"2024-06-10T01:04:35.569804+0000","last_undegraded":"2024-06-10T01:04:35.569804+0000","last_fullsized":"2024-06-10T01:04:35.569804+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:55:56.785016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.567988+0000","last_change":"2024-06-10T01:01:47.243141+0000","last_active":"2024-06-10T01:04:35.567988+0000","last_peered":"2024-06-10T01:04:35.567988+0000","last_clean":"2024-06-10T01:04:35.567988+0000","last_became_active":"2024-06-10T01:01:47.242464+0000","last_became_peered":"2024-06-10T01:01:47.242464+0000","last_unstale":"2024-06-10T01:04:35.567988+0000","last_undegraded":"2024-06-10T01:04:35.567988+0000","last_fullsized":"2024-06-10T01:04:35.567988+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:00:49.777130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559919+0000","last_change":"2024-06-10T01:01:47.253463+0000","last_active":"2024-06-10T01:04:35.559919+0000","last_peered":"2024-06-10T01:04:35.559919+0000","last_clean":"2024-06-10T01:04:35.559919+0000","last_became_active":"2024-06-10T01:01:47.252929+0000","last_became_peered":"2024-06-10T01:01:47.252929+0000","last_unstale":"2024-06-10T01:04:35.559919+0000","last_undegraded":"2024-06-10T01:04:35.559919+0000","last_fullsized":"2024-06-10T01:04:35.559919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:18:49.551276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239453+0000","last_change":"2024-06-10T01:01:57.279561+0000","last_active":"2024-06-10T01:03:38.239453+0000","last_peered":"2024-06-10T01:03:38.239453+0000","last_clean":"2024-06-10T01:03:38.239453+0000","last_became_active":"2024-06-10T01:01:57.279132+0000","last_became_peered":"2024-06-10T01:01:57.279132+0000","last_unstale":"2024-06-10T01:03:38.239453+0000","last_undegraded":"2024-06-10T01:03:38.239453+0000","last_fullsized":"2024-06-10T01:03:38.239453+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:55:03.497701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'9","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.772124+0000","last_change":"2024-06-10T01:01:44.006667+0000","last_active":"2024-06-10T01:03:38.772124+0000","last_peered":"2024-06-10T01:03:38.772124+0000","last_clean":"2024-06-10T01:03:38.772124+0000","last_became_active":"2024-06-10T01:01:44.006343+0000","last_became_peered":"2024-06-10T01:01:44.006343+0000","last_unstale":"2024-06-10T01:03:38.772124+0000","last_undegraded":"2024-06-10T01:03:38.772124+0000","last_fullsized":"2024-06-10T01:03:38.772124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:14:50.540084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562836+0000","last_change":"2024-06-10T01:01:47.246139+0000","last_active":"2024-06-10T01:04:35.562836+0000","last_peered":"2024-06-10T01:04:35.562836+0000","last_clean":"2024-06-10T01:04:35.562836+0000","last_became_active":"2024-06-10T01:01:47.245743+0000","last_became_peered":"2024-06-10T01:01:47.245743+0000","last_unstale":"2024-06-10T01:04:35.562836+0000","last_undegraded":"2024-06-10T01:04:35.562836+0000","last_fullsized":"2024-06-10T01:04:35.562836+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:06:39.041569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239398+0000","last_change":"2024-06-10T01:01:57.282843+0000","last_active":"2024-06-10T01:03:38.239398+0000","last_peered":"2024-06-10T01:03:38.239398+0000","last_clean":"2024-06-10T01:03:38.239398+0000","last_became_active":"2024-06-10T01:01:57.281797+0000","last_became_peered":"2024-06-10T01:01:57.281797+0000","last_unstale":"2024-06-10T01:03:38.239398+0000","last_undegraded":"2024-06-10T01:03:38.239398+0000","last_fullsized":"2024-06-10T01:03:38.239398+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:00.209628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239550+0000","last_change":"2024-06-10T01:01:44.466714+0000","last_active":"2024-06-10T01:03:38.239550+0000","last_peered":"2024-06-10T01:03:38.239550+0000","last_clean":"2024-06-10T01:03:38.239550+0000","last_became_active":"2024-06-10T01:01:44.466296+0000","last_became_peered":"2024-06-10T01:01:44.466296+0000","last_unstale":"2024-06-10T01:03:38.239550+0000","last_undegraded":"2024-06-10T01:03:38.239550+0000","last_fullsized":"2024-06-10T01:03:38.239550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:16:36.082308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.569808+0000","last_change":"2024-06-10T01:01:47.244173+0000","last_active":"2024-06-10T01:04:35.569808+0000","last_peered":"2024-06-10T01:04:35.569808+0000","last_clean":"2024-06-10T01:04:35.569808+0000","last_became_active":"2024-06-10T01:01:47.243032+0000","last_became_peered":"2024-06-10T01:01:47.243032+0000","last_unstale":"2024-06-10T01:04:35.569808+0000","last_undegraded":"2024-06-10T01:04:35.569808+0000","last_fullsized":"2024-06-10T01:04:35.569808+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:14:33.965903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239057+0000","last_change":"2024-06-10T01:02:37.592535+0000","last_active":"2024-06-10T01:03:38.239057+0000","last_peered":"2024-06-10T01:03:38.239057+0000","last_clean":"2024-06-10T01:03:38.239057+0000","last_became_active":"2024-06-10T01:02:37.592159+0000","last_became_peered":"2024-06-10T01:02:37.592159+0000","last_unstale":"2024-06-10T01:03:38.239057+0000","last_undegraded":"2024-06-10T01:03:38.239057+0000","last_fullsized":"2024-06-10T01:03:38.239057+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:50:08.626485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235433+0000","last_change":"2024-06-10T01:01:44.009105+0000","last_active":"2024-06-10T01:03:38.235433+0000","last_peered":"2024-06-10T01:03:38.235433+0000","last_clean":"2024-06-10T01:03:38.235433+0000","last_became_active":"2024-06-10T01:01:44.008743+0000","last_became_peered":"2024-06-10T01:01:44.008743+0000","last_unstale":"2024-06-10T01:03:38.235433+0000","last_undegraded":"2024-06-10T01:03:38.235433+0000","last_fullsized":"2024-06-10T01:03:38.235433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:44:54.301378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.570239+0000","last_change":"2024-06-10T01:01:47.249007+0000","last_active":"2024-06-10T01:04:35.570239+0000","last_peered":"2024-06-10T01:04:35.570239+0000","last_clean":"2024-06-10T01:04:35.570239+0000","last_became_active":"2024-06-10T01:01:47.245632+0000","last_became_peered":"2024-06-10T01:01:47.245632+0000","last_unstale":"2024-06-10T01:04:35.570239+0000","last_undegraded":"2024-06-10T01:04:35.570239+0000","last_fullsized":"2024-06-10T01:04:35.570239+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:29:18.916390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.238983+0000","last_change":"2024-06-10T01:01:57.283137+0000","last_active":"2024-06-10T01:03:38.238983+0000","last_peered":"2024-06-10T01:03:38.238983+0000","last_clean":"2024-06-10T01:03:38.238983+0000","last_became_active":"2024-06-10T01:01:57.282605+0000","last_became_peered":"2024-06-10T01:01:57.282605+0000","last_unstale":"2024-06-10T01:03:38.238983+0000","last_undegraded":"2024-06-10T01:03:38.238983+0000","last_fullsized":"2024-06-10T01:03:38.238983+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:26:16.967320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235734+0000","last_change":"2024-06-10T01:01:44.010674+0000","last_active":"2024-06-10T01:03:38.235734+0000","last_peered":"2024-06-10T01:03:38.235734+0000","last_clean":"2024-06-10T01:03:38.235734+0000","last_became_active":"2024-06-10T01:01:44.010291+0000","last_became_peered":"2024-06-10T01:01:44.010291+0000","last_unstale":"2024-06-10T01:03:38.235734+0000","last_undegraded":"2024-06-10T01:03:38.235734+0000","last_fullsized":"2024-06-10T01:03:38.235734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:36:18.418362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":60,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.566078+0000","last_change":"2024-06-10T01:02:37.794580+0000","last_active":"2024-06-10T01:04:35.566078+0000","last_peered":"2024-06-10T01:04:35.566078+0000","last_clean":"2024-06-10T01:04:35.566078+0000","last_became_active":"2024-06-10T01:02:37.794127+0000","last_became_peered":"2024-06-10T01:02:37.794127+0000","last_unstale":"2024-06-10T01:04:35.566078+0000","last_undegraded":"2024-06-10T01:04:35.566078+0000","last_fullsized":"2024-06-10T01:04:35.566078+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:02.117871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239746+0000","last_change":"2024-06-10T01:01:57.282494+0000","last_active":"2024-06-10T01:03:38.239746+0000","last_peered":"2024-06-10T01:03:38.239746+0000","last_clean":"2024-06-10T01:03:38.239746+0000","last_became_active":"2024-06-10T01:01:57.282077+0000","last_became_peered":"2024-06-10T01:01:57.282077+0000","last_unstale":"2024-06-10T01:03:38.239746+0000","last_undegraded":"2024-06-10T01:03:38.239746+0000","last_fullsized":"2024-06-10T01:03:38.239746+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:57:41.639788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"97'5","reported_seq":219,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:40.951609+0000","last_change":"2024-06-10T01:01:44.011134+0000","last_active":"2024-06-10T01:05:40.951609+0000","last_peered":"2024-06-10T01:05:40.951609+0000","last_clean":"2024-06-10T01:05:40.951609+0000","last_became_active":"2024-06-10T01:01:44.010788+0000","last_became_peered":"2024-06-10T01:01:44.010788+0000","last_unstale":"2024-06-10T01:05:40.951609+0000","last_undegraded":"2024-06-10T01:05:40.951609+0000","last_fullsized":"2024-06-10T01:05:40.951609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:22:55.228946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":169,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.564421+0000","last_change":"2024-06-10T01:01:47.248093+0000","last_active":"2024-06-10T01:04:35.564421+0000","last_peered":"2024-06-10T01:04:35.564421+0000","last_clean":"2024-06-10T01:04:35.564421+0000","last_became_active":"2024-06-10T01:01:47.247797+0000","last_became_peered":"2024-06-10T01:01:47.247797+0000","last_unstale":"2024-06-10T01:04:35.564421+0000","last_undegraded":"2024-06-10T01:04:35.564421+0000","last_fullsized":"2024-06-10T01:04:35.564421+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:45:50.159128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239671+0000","last_change":"2024-06-10T01:02:37.377039+0000","last_active":"2024-06-10T01:03:38.239671+0000","last_peered":"2024-06-10T01:03:38.239671+0000","last_clean":"2024-06-10T01:03:38.239671+0000","last_became_active":"2024-06-10T01:02:37.376332+0000","last_became_peered":"2024-06-10T01:02:37.376332+0000","last_unstale":"2024-06-10T01:03:38.239671+0000","last_undegraded":"2024-06-10T01:03:38.239671+0000","last_fullsized":"2024-06-10T01:03:38.239671+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:15.000505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,10,5],"acting":[2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.570465+0000","last_change":"2024-06-10T01:01:44.008342+0000","last_active":"2024-06-10T01:03:38.570465+0000","last_peered":"2024-06-10T01:03:38.570465+0000","last_clean":"2024-06-10T01:03:38.570465+0000","last_became_active":"2024-06-10T01:01:44.007959+0000","last_became_peered":"2024-06-10T01:01:44.007959+0000","last_unstale":"2024-06-10T01:03:38.570465+0000","last_undegraded":"2024-06-10T01:03:38.570465+0000","last_fullsized":"2024-06-10T01:03:38.570465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:36:03.143609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.565044+0000","last_change":"2024-06-10T01:01:47.247854+0000","last_active":"2024-06-10T01:04:35.565044+0000","last_peered":"2024-06-10T01:04:35.565044+0000","last_clean":"2024-06-10T01:04:35.565044+0000","last_became_active":"2024-06-10T01:01:47.247563+0000","last_became_peered":"2024-06-10T01:01:47.247563+0000","last_unstale":"2024-06-10T01:04:35.565044+0000","last_undegraded":"2024-06-10T01:04:35.565044+0000","last_fullsized":"2024-06-10T01:04:35.565044+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:32:13.024815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.237041+0000","last_change":"2024-06-10T01:01:57.282427+0000","last_active":"2024-06-10T01:03:38.237041+0000","last_peered":"2024-06-10T01:03:38.237041+0000","last_clean":"2024-06-10T01:03:38.237041+0000","last_became_active":"2024-06-10T01:01:57.281987+0000","last_became_peered":"2024-06-10T01:01:57.281987+0000","last_unstale":"2024-06-10T01:03:38.237041+0000","last_undegraded":"2024-06-10T01:03:38.237041+0000","last_fullsized":"2024-06-10T01:03:38.237041+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:22:31.736804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770902+0000","last_change":"2024-06-10T01:01:44.006567+0000","last_active":"2024-06-10T01:03:38.770902+0000","last_peered":"2024-06-10T01:03:38.770902+0000","last_clean":"2024-06-10T01:03:38.770902+0000","last_became_active":"2024-06-10T01:01:44.006216+0000","last_became_peered":"2024-06-10T01:01:44.006216+0000","last_unstale":"2024-06-10T01:03:38.770902+0000","last_undegraded":"2024-06-10T01:03:38.770902+0000","last_fullsized":"2024-06-10T01:03:38.770902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:33:55.696873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558008+0000","last_change":"2024-06-10T01:01:47.245326+0000","last_active":"2024-06-10T01:04:35.558008+0000","last_peered":"2024-06-10T01:04:35.558008+0000","last_clean":"2024-06-10T01:04:35.558008+0000","last_became_active":"2024-06-10T01:01:47.245013+0000","last_became_peered":"2024-06-10T01:01:47.245013+0000","last_unstale":"2024-06-10T01:04:35.558008+0000","last_undegraded":"2024-06-10T01:04:35.558008+0000","last_fullsized":"2024-06-10T01:04:35.558008+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:17:56.416627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234530+0000","last_change":"2024-06-10T01:01:57.288997+0000","last_active":"2024-06-10T01:03:38.234530+0000","last_peered":"2024-06-10T01:03:38.234530+0000","last_clean":"2024-06-10T01:03:38.234530+0000","last_became_active":"2024-06-10T01:01:57.288596+0000","last_became_peered":"2024-06-10T01:01:57.288596+0000","last_unstale":"2024-06-10T01:03:38.234530+0000","last_undegraded":"2024-06-10T01:03:38.234530+0000","last_fullsized":"2024-06-10T01:03:38.234530+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:08:14.739737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.269738+0000","last_change":"2024-06-10T01:01:44.010079+0000","last_active":"2024-06-10T01:03:38.269738+0000","last_peered":"2024-06-10T01:03:38.269738+0000","last_clean":"2024-06-10T01:03:38.269738+0000","last_became_active":"2024-06-10T01:01:44.009752+0000","last_became_peered":"2024-06-10T01:01:44.009752+0000","last_unstale":"2024-06-10T01:03:38.269738+0000","last_undegraded":"2024-06-10T01:03:38.269738+0000","last_fullsized":"2024-06-10T01:03:38.269738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:29:35.660935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.554398+0000","last_change":"2024-06-10T01:01:47.245015+0000","last_active":"2024-06-10T01:04:35.554398+0000","last_peered":"2024-06-10T01:04:35.554398+0000","last_clean":"2024-06-10T01:04:35.554398+0000","last_became_active":"2024-06-10T01:01:47.244595+0000","last_became_peered":"2024-06-10T01:01:47.244595+0000","last_unstale":"2024-06-10T01:04:35.554398+0000","last_undegraded":"2024-06-10T01:04:35.554398+0000","last_fullsized":"2024-06-10T01:04:35.554398+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:01:48.446526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.238926+0000","last_change":"2024-06-10T01:01:57.281656+0000","last_active":"2024-06-10T01:03:38.238926+0000","last_peered":"2024-06-10T01:03:38.238926+0000","last_clean":"2024-06-10T01:03:38.238926+0000","last_became_active":"2024-06-10T01:01:57.281295+0000","last_became_peered":"2024-06-10T01:01:57.281295+0000","last_unstale":"2024-06-10T01:03:38.238926+0000","last_undegraded":"2024-06-10T01:03:38.238926+0000","last_fullsized":"2024-06-10T01:03:38.238926+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:04:41.619864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"97'3","reported_seq":214,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:39.949599+0000","last_change":"2024-06-10T01:01:44.011941+0000","last_active":"2024-06-10T01:05:39.949599+0000","last_peered":"2024-06-10T01:05:39.949599+0000","last_clean":"2024-06-10T01:05:39.949599+0000","last_became_active":"2024-06-10T01:01:44.010156+0000","last_became_peered":"2024-06-10T01:01:44.010156+0000","last_unstale":"2024-06-10T01:05:39.949599+0000","last_undegraded":"2024-06-10T01:05:39.949599+0000","last_fullsized":"2024-06-10T01:05:39.949599+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:16:15.709544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":166,"num_read_kb":166,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.570257+0000","last_change":"2024-06-10T01:01:47.241720+0000","last_active":"2024-06-10T01:04:35.570257+0000","last_peered":"2024-06-10T01:04:35.570257+0000","last_clean":"2024-06-10T01:04:35.570257+0000","last_became_active":"2024-06-10T01:01:47.241386+0000","last_became_peered":"2024-06-10T01:01:47.241386+0000","last_unstale":"2024-06-10T01:04:35.570257+0000","last_undegraded":"2024-06-10T01:04:35.570257+0000","last_fullsized":"2024-06-10T01:04:35.570257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:42:01.304787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770709+0000","last_change":"2024-06-10T01:01:57.283226+0000","last_active":"2024-06-10T01:03:38.770709+0000","last_peered":"2024-06-10T01:03:38.770709+0000","last_clean":"2024-06-10T01:03:38.770709+0000","last_became_active":"2024-06-10T01:01:57.282720+0000","last_became_peered":"2024-06-10T01:01:57.282720+0000","last_unstale":"2024-06-10T01:03:38.770709+0000","last_undegraded":"2024-06-10T01:03:38.770709+0000","last_fullsized":"2024-06-10T01:03:38.770709+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:35:31.342042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235489+0000","last_change":"2024-06-10T01:01:44.009278+0000","last_active":"2024-06-10T01:03:38.235489+0000","last_peered":"2024-06-10T01:03:38.235489+0000","last_clean":"2024-06-10T01:03:38.235489+0000","last_became_active":"2024-06-10T01:01:44.008841+0000","last_became_peered":"2024-06-10T01:01:44.008841+0000","last_unstale":"2024-06-10T01:03:38.235489+0000","last_undegraded":"2024-06-10T01:03:38.235489+0000","last_fullsized":"2024-06-10T01:03:38.235489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:30:58.392080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.557141+0000","last_change":"2024-06-10T01:01:47.256451+0000","last_active":"2024-06-10T01:04:35.557141+0000","last_peered":"2024-06-10T01:04:35.557141+0000","last_clean":"2024-06-10T01:04:35.557141+0000","last_became_active":"2024-06-10T01:01:47.256036+0000","last_became_peered":"2024-06-10T01:01:47.256036+0000","last_unstale":"2024-06-10T01:04:35.557141+0000","last_undegraded":"2024-06-10T01:04:35.557141+0000","last_fullsized":"2024-06-10T01:04:35.557141+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:57:26.640109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239192+0000","last_change":"2024-06-10T01:02:37.794160+0000","last_active":"2024-06-10T01:03:38.239192+0000","last_peered":"2024-06-10T01:03:38.239192+0000","last_clean":"2024-06-10T01:03:38.239192+0000","last_became_active":"2024-06-10T01:02:37.793633+0000","last_became_peered":"2024-06-10T01:02:37.793633+0000","last_unstale":"2024-06-10T01:03:38.239192+0000","last_undegraded":"2024-06-10T01:03:38.239192+0000","last_fullsized":"2024-06-10T01:03:38.239192+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:48:48.083620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270125+0000","last_change":"2024-06-10T01:01:44.469179+0000","last_active":"2024-06-10T01:03:38.270125+0000","last_peered":"2024-06-10T01:03:38.270125+0000","last_clean":"2024-06-10T01:03:38.270125+0000","last_became_active":"2024-06-10T01:01:44.468669+0000","last_became_peered":"2024-06-10T01:01:44.468669+0000","last_unstale":"2024-06-10T01:03:38.270125+0000","last_undegraded":"2024-06-10T01:03:38.270125+0000","last_fullsized":"2024-06-10T01:03:38.270125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:07:51.629030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.570053+0000","last_change":"2024-06-10T01:01:47.247588+0000","last_active":"2024-06-10T01:04:35.570053+0000","last_peered":"2024-06-10T01:04:35.570053+0000","last_clean":"2024-06-10T01:04:35.570053+0000","last_became_active":"2024-06-10T01:01:47.245750+0000","last_became_peered":"2024-06-10T01:01:47.245750+0000","last_unstale":"2024-06-10T01:04:35.570053+0000","last_undegraded":"2024-06-10T01:04:35.570053+0000","last_fullsized":"2024-06-10T01:04:35.570053+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:12:10.513777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771292+0000","last_change":"2024-06-10T01:01:57.283042+0000","last_active":"2024-06-10T01:03:38.771292+0000","last_peered":"2024-06-10T01:03:38.771292+0000","last_clean":"2024-06-10T01:03:38.771292+0000","last_became_active":"2024-06-10T01:01:57.282600+0000","last_became_peered":"2024-06-10T01:01:57.282600+0000","last_unstale":"2024-06-10T01:03:38.771292+0000","last_undegraded":"2024-06-10T01:03:38.771292+0000","last_fullsized":"2024-06-10T01:03:38.771292+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:06:36.969968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"99'39","reported_seq":425,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:39.950344+0000","last_change":"2024-06-10T01:01:44.011783+0000","last_active":"2024-06-10T01:05:39.950344+0000","last_peered":"2024-06-10T01:05:39.950344+0000","last_clean":"2024-06-10T01:05:39.950344+0000","last_became_active":"2024-06-10T01:01:44.011164+0000","last_became_peered":"2024-06-10T01:01:44.011164+0000","last_unstale":"2024-06-10T01:05:39.950344+0000","last_undegraded":"2024-06-10T01:05:39.950344+0000","last_fullsized":"2024-06-10T01:05:39.950344+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:16:13.642714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64615,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":341,"num_read_kb":234,"num_write":40,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562407+0000","last_change":"2024-06-10T01:01:47.250751+0000","last_active":"2024-06-10T01:04:35.562407+0000","last_peered":"2024-06-10T01:04:35.562407+0000","last_clean":"2024-06-10T01:04:35.562407+0000","last_became_active":"2024-06-10T01:01:47.250451+0000","last_became_peered":"2024-06-10T01:01:47.250451+0000","last_unstale":"2024-06-10T01:04:35.562407+0000","last_undegraded":"2024-06-10T01:04:35.562407+0000","last_fullsized":"2024-06-10T01:04:35.562407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:22:10.121363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270382+0000","last_change":"2024-06-10T01:01:57.289687+0000","last_active":"2024-06-10T01:03:38.270382+0000","last_peered":"2024-06-10T01:03:38.270382+0000","last_clean":"2024-06-10T01:03:38.270382+0000","last_became_active":"2024-06-10T01:01:57.289274+0000","last_became_peered":"2024-06-10T01:01:57.289274+0000","last_unstale":"2024-06-10T01:03:38.270382+0000","last_undegraded":"2024-06-10T01:03:38.270382+0000","last_fullsized":"2024-06-10T01:03:38.270382+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:34:46.873974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'6","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239529+0000","last_change":"2024-06-10T01:01:44.010159+0000","last_active":"2024-06-10T01:03:38.239529+0000","last_peered":"2024-06-10T01:03:38.239529+0000","last_clean":"2024-06-10T01:03:38.239529+0000","last_became_active":"2024-06-10T01:01:44.009744+0000","last_became_peered":"2024-06-10T01:01:44.009744+0000","last_unstale":"2024-06-10T01:03:38.239529+0000","last_undegraded":"2024-06-10T01:03:38.239529+0000","last_fullsized":"2024-06-10T01:03:38.239529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:30.507613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558994+0000","last_change":"2024-06-10T01:01:47.246662+0000","last_active":"2024-06-10T01:04:35.558994+0000","last_peered":"2024-06-10T01:04:35.558994+0000","last_clean":"2024-06-10T01:04:35.558994+0000","last_became_active":"2024-06-10T01:01:47.246268+0000","last_became_peered":"2024-06-10T01:01:47.246268+0000","last_unstale":"2024-06-10T01:04:35.558994+0000","last_undegraded":"2024-06-10T01:04:35.558994+0000","last_fullsized":"2024-06-10T01:04:35.558994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:08:38.089054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.236925+0000","last_change":"2024-06-10T01:01:57.289937+0000","last_active":"2024-06-10T01:03:38.236925+0000","last_peered":"2024-06-10T01:03:38.236925+0000","last_clean":"2024-06-10T01:03:38.236925+0000","last_became_active":"2024-06-10T01:01:57.289407+0000","last_became_peered":"2024-06-10T01:01:57.289407+0000","last_unstale":"2024-06-10T01:03:38.236925+0000","last_undegraded":"2024-06-10T01:03:38.236925+0000","last_fullsized":"2024-06-10T01:03:38.236925+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:01:48.546762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270079+0000","last_change":"2024-06-10T01:01:44.012654+0000","last_active":"2024-06-10T01:03:38.270079+0000","last_peered":"2024-06-10T01:03:38.270079+0000","last_clean":"2024-06-10T01:03:38.270079+0000","last_became_active":"2024-06-10T01:01:44.012246+0000","last_became_peered":"2024-06-10T01:01:44.012246+0000","last_unstale":"2024-06-10T01:03:38.270079+0000","last_undegraded":"2024-06-10T01:03:38.270079+0000","last_fullsized":"2024-06-10T01:03:38.270079+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:31:36.173554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558783+0000","last_change":"2024-06-10T01:01:47.250514+0000","last_active":"2024-06-10T01:04:35.558783+0000","last_peered":"2024-06-10T01:04:35.558783+0000","last_clean":"2024-06-10T01:04:35.558783+0000","last_became_active":"2024-06-10T01:01:47.249050+0000","last_became_peered":"2024-06-10T01:01:47.249050+0000","last_unstale":"2024-06-10T01:04:35.558783+0000","last_undegraded":"2024-06-10T01:04:35.558783+0000","last_fullsized":"2024-06-10T01:04:35.558783+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:51:30.789392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771928+0000","last_change":"2024-06-10T01:01:57.282389+0000","last_active":"2024-06-10T01:03:38.771928+0000","last_peered":"2024-06-10T01:03:38.771928+0000","last_clean":"2024-06-10T01:03:38.771928+0000","last_became_active":"2024-06-10T01:01:57.281857+0000","last_became_peered":"2024-06-10T01:01:57.281857+0000","last_unstale":"2024-06-10T01:03:38.771928+0000","last_undegraded":"2024-06-10T01:03:38.771928+0000","last_fullsized":"2024-06-10T01:03:38.771928+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:59:14.863020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"99'4","reported_seq":214,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:37.947518+0000","last_change":"2024-06-10T01:01:44.011331+0000","last_active":"2024-06-10T01:05:37.947518+0000","last_peered":"2024-06-10T01:05:37.947518+0000","last_clean":"2024-06-10T01:05:37.947518+0000","last_became_active":"2024-06-10T01:01:44.010968+0000","last_became_peered":"2024-06-10T01:01:44.010968+0000","last_unstale":"2024-06-10T01:05:37.947518+0000","last_undegraded":"2024-06-10T01:05:37.947518+0000","last_fullsized":"2024-06-10T01:05:37.947518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:43:37.931225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":896,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":166,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562879+0000","last_change":"2024-06-10T01:01:47.248349+0000","last_active":"2024-06-10T01:04:35.562879+0000","last_peered":"2024-06-10T01:04:35.562879+0000","last_clean":"2024-06-10T01:04:35.562879+0000","last_became_active":"2024-06-10T01:01:47.248008+0000","last_became_peered":"2024-06-10T01:01:47.248008+0000","last_unstale":"2024-06-10T01:04:35.562879+0000","last_undegraded":"2024-06-10T01:04:35.562879+0000","last_fullsized":"2024-06-10T01:04:35.562879+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:45:20.747704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771221+0000","last_change":"2024-06-10T01:01:57.289773+0000","last_active":"2024-06-10T01:03:38.771221+0000","last_peered":"2024-06-10T01:03:38.771221+0000","last_clean":"2024-06-10T01:03:38.771221+0000","last_became_active":"2024-06-10T01:01:57.289371+0000","last_became_peered":"2024-06-10T01:01:57.289371+0000","last_unstale":"2024-06-10T01:03:38.771221+0000","last_undegraded":"2024-06-10T01:03:38.771221+0000","last_fullsized":"2024-06-10T01:03:38.771221+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:24:52.209402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'15","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.257548+0000","last_change":"2024-06-10T01:03:38.257122+0000","last_active":"2024-06-10T01:03:38.257548+0000","last_peered":"2024-06-10T01:03:38.257548+0000","last_clean":"2024-06-10T01:03:38.257548+0000","last_became_active":"2024-06-10T01:03:38.240737+0000","last_became_peered":"2024-06-10T01:03:38.240737+0000","last_unstale":"2024-06-10T01:03:38.257548+0000","last_undegraded":"2024-06-10T01:03:38.257548+0000","last_fullsized":"2024-06-10T01:03:38.257548+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:37:10.925004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562425+0000","last_change":"2024-06-10T01:01:47.245439+0000","last_active":"2024-06-10T01:04:35.562425+0000","last_peered":"2024-06-10T01:04:35.562425+0000","last_clean":"2024-06-10T01:04:35.562425+0000","last_became_active":"2024-06-10T01:01:47.242917+0000","last_became_peered":"2024-06-10T01:01:47.242917+0000","last_unstale":"2024-06-10T01:04:35.562425+0000","last_undegraded":"2024-06-10T01:04:35.562425+0000","last_fullsized":"2024-06-10T01:04:35.562425+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:13:12.614230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.269795+0000","last_change":"2024-06-10T01:01:57.284731+0000","last_active":"2024-06-10T01:03:38.269795+0000","last_peered":"2024-06-10T01:03:38.269795+0000","last_clean":"2024-06-10T01:03:38.269795+0000","last_became_active":"2024-06-10T01:01:57.284301+0000","last_became_peered":"2024-06-10T01:01:57.284301+0000","last_unstale":"2024-06-10T01:03:38.269795+0000","last_undegraded":"2024-06-10T01:03:38.269795+0000","last_fullsized":"2024-06-10T01:03:38.269795+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:01:36.518214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"99'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:59.314831+0000","last_change":"2024-06-10T01:01:44.010154+0000","last_active":"2024-06-10T01:03:59.314831+0000","last_peered":"2024-06-10T01:03:59.314831+0000","last_clean":"2024-06-10T01:03:59.314831+0000","last_became_active":"2024-06-10T01:01:44.009656+0000","last_became_peered":"2024-06-10T01:01:44.009656+0000","last_unstale":"2024-06-10T01:03:59.314831+0000","last_undegraded":"2024-06-10T01:03:59.314831+0000","last_fullsized":"2024-06-10T01:03:59.314831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:33:08.789946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.550140+0000","last_change":"2024-06-10T01:02:37.378484+0000","last_active":"2024-06-10T01:04:35.550140+0000","last_peered":"2024-06-10T01:04:35.550140+0000","last_clean":"2024-06-10T01:04:35.550140+0000","last_became_active":"2024-06-10T01:02:37.378092+0000","last_became_peered":"2024-06-10T01:02:37.378092+0000","last_unstale":"2024-06-10T01:04:35.550140+0000","last_undegraded":"2024-06-10T01:04:35.550140+0000","last_fullsized":"2024-06-10T01:04:35.550140+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:19:46.599235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235375+0000","last_change":"2024-06-10T01:01:57.332887+0000","last_active":"2024-06-10T01:03:38.235375+0000","last_peered":"2024-06-10T01:03:38.235375+0000","last_clean":"2024-06-10T01:03:38.235375+0000","last_became_active":"2024-06-10T01:01:57.332285+0000","last_became_peered":"2024-06-10T01:01:57.332285+0000","last_unstale":"2024-06-10T01:03:38.235375+0000","last_undegraded":"2024-06-10T01:03:38.235375+0000","last_fullsized":"2024-06-10T01:03:38.235375+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:09:12.233405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771642+0000","last_change":"2024-06-10T01:01:44.466831+0000","last_active":"2024-06-10T01:03:38.771642+0000","last_peered":"2024-06-10T01:03:38.771642+0000","last_clean":"2024-06-10T01:03:38.771642+0000","last_became_active":"2024-06-10T01:01:44.466484+0000","last_became_peered":"2024-06-10T01:01:44.466484+0000","last_unstale":"2024-06-10T01:03:38.771642+0000","last_undegraded":"2024-06-10T01:03:38.771642+0000","last_fullsized":"2024-06-10T01:03:38.771642+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:19:28.385504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.556135+0000","last_change":"2024-06-10T01:01:47.252399+0000","last_active":"2024-06-10T01:04:35.556135+0000","last_peered":"2024-06-10T01:04:35.556135+0000","last_clean":"2024-06-10T01:04:35.556135+0000","last_became_active":"2024-06-10T01:01:47.252069+0000","last_became_peered":"2024-06-10T01:01:47.252069+0000","last_unstale":"2024-06-10T01:04:35.556135+0000","last_undegraded":"2024-06-10T01:04:35.556135+0000","last_fullsized":"2024-06-10T01:04:35.556135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:02:02.291174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771406+0000","last_change":"2024-06-10T01:01:57.285670+0000","last_active":"2024-06-10T01:03:38.771406+0000","last_peered":"2024-06-10T01:03:38.771406+0000","last_clean":"2024-06-10T01:03:38.771406+0000","last_became_active":"2024-06-10T01:01:57.285281+0000","last_became_peered":"2024-06-10T01:01:57.285281+0000","last_unstale":"2024-06-10T01:03:38.771406+0000","last_undegraded":"2024-06-10T01:03:38.771406+0000","last_fullsized":"2024-06-10T01:03:38.771406+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:58.956934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235189+0000","last_change":"2024-06-10T01:01:44.008013+0000","last_active":"2024-06-10T01:03:38.235189+0000","last_peered":"2024-06-10T01:03:38.235189+0000","last_clean":"2024-06-10T01:03:38.235189+0000","last_became_active":"2024-06-10T01:01:44.007591+0000","last_became_peered":"2024-06-10T01:01:44.007591+0000","last_unstale":"2024-06-10T01:03:38.235189+0000","last_undegraded":"2024-06-10T01:03:38.235189+0000","last_fullsized":"2024-06-10T01:03:38.235189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:18:20.862340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.550926+0000","last_change":"2024-06-10T01:01:47.254756+0000","last_active":"2024-06-10T01:04:35.550926+0000","last_peered":"2024-06-10T01:04:35.550926+0000","last_clean":"2024-06-10T01:04:35.550926+0000","last_became_active":"2024-06-10T01:01:47.254418+0000","last_became_peered":"2024-06-10T01:01:47.254418+0000","last_unstale":"2024-06-10T01:04:35.550926+0000","last_undegraded":"2024-06-10T01:04:35.550926+0000","last_fullsized":"2024-06-10T01:04:35.550926+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:46:31.452539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770995+0000","last_change":"2024-06-10T01:01:57.332792+0000","last_active":"2024-06-10T01:03:38.770995+0000","last_peered":"2024-06-10T01:03:38.770995+0000","last_clean":"2024-06-10T01:03:38.770995+0000","last_became_active":"2024-06-10T01:01:57.332331+0000","last_became_peered":"2024-06-10T01:01:57.332331+0000","last_unstale":"2024-06-10T01:03:38.770995+0000","last_undegraded":"2024-06-10T01:03:38.770995+0000","last_fullsized":"2024-06-10T01:03:38.770995+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:46:41.419807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.565658+0000","last_change":"2024-06-10T01:01:47.247857+0000","last_active":"2024-06-10T01:04:35.565658+0000","last_peered":"2024-06-10T01:04:35.565658+0000","last_clean":"2024-06-10T01:04:35.565658+0000","last_became_active":"2024-06-10T01:01:47.247552+0000","last_became_peered":"2024-06-10T01:01:47.247552+0000","last_unstale":"2024-06-10T01:04:35.565658+0000","last_undegraded":"2024-06-10T01:04:35.565658+0000","last_fullsized":"2024-06-10T01:04:35.565658+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:29:40.521887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"99'5","reported_seq":214,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:38.440204+0000","last_change":"2024-06-10T01:01:44.466631+0000","last_active":"2024-06-10T01:05:38.440204+0000","last_peered":"2024-06-10T01:05:38.440204+0000","last_clean":"2024-06-10T01:05:38.440204+0000","last_became_active":"2024-06-10T01:01:44.466280+0000","last_became_peered":"2024-06-10T01:01:44.466280+0000","last_unstale":"2024-06-10T01:05:38.440204+0000","last_undegraded":"2024-06-10T01:05:38.440204+0000","last_fullsized":"2024-06-10T01:05:38.440204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:09:33.658112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":164,"num_read_kb":164,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235365+0000","last_change":"2024-06-10T01:01:57.281973+0000","last_active":"2024-06-10T01:03:38.235365+0000","last_peered":"2024-06-10T01:03:38.235365+0000","last_clean":"2024-06-10T01:03:38.235365+0000","last_became_active":"2024-06-10T01:01:57.281386+0000","last_became_peered":"2024-06-10T01:01:57.281386+0000","last_unstale":"2024-06-10T01:03:38.235365+0000","last_undegraded":"2024-06-10T01:03:38.235365+0000","last_fullsized":"2024-06-10T01:03:38.235365+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:10:28.236250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562415+0000","last_change":"2024-06-10T01:01:47.237436+0000","last_active":"2024-06-10T01:04:35.562415+0000","last_peered":"2024-06-10T01:04:35.562415+0000","last_clean":"2024-06-10T01:04:35.562415+0000","last_became_active":"2024-06-10T01:01:47.237099+0000","last_became_peered":"2024-06-10T01:01:47.237099+0000","last_unstale":"2024-06-10T01:04:35.562415+0000","last_undegraded":"2024-06-10T01:04:35.562415+0000","last_fullsized":"2024-06-10T01:04:35.562415+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:34:53.838877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'3","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239178+0000","last_change":"2024-06-10T01:01:44.010655+0000","last_active":"2024-06-10T01:03:38.239178+0000","last_peered":"2024-06-10T01:03:38.239178+0000","last_clean":"2024-06-10T01:03:38.239178+0000","last_became_active":"2024-06-10T01:01:44.009918+0000","last_became_peered":"2024-06-10T01:01:44.009918+0000","last_unstale":"2024-06-10T01:03:38.239178+0000","last_undegraded":"2024-06-10T01:03:38.239178+0000","last_fullsized":"2024-06-10T01:03:38.239178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:47:32.052650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":367,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234906+0000","last_change":"2024-06-10T01:00:23.822447+0000","last_active":"2024-06-10T01:03:38.234906+0000","last_peered":"2024-06-10T01:03:38.234906+0000","last_clean":"2024-06-10T01:03:38.234906+0000","last_became_active":"2024-06-10T01:00:23.821966+0000","last_became_peered":"2024-06-10T01:00:23.821966+0000","last_unstale":"2024-06-10T01:03:38.234906+0000","last_undegraded":"2024-06-10T01:03:38.234906+0000","last_fullsized":"2024-06-10T01:03:38.234906+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:56:59.964661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:56:59.964661+0000","last_clean_scrub_stamp":"2024-06-10T00:56:59.964661+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:03:43.732875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270272+0000","last_change":"2024-06-10T01:01:57.332980+0000","last_active":"2024-06-10T01:03:38.270272+0000","last_peered":"2024-06-10T01:03:38.270272+0000","last_clean":"2024-06-10T01:03:38.270272+0000","last_became_active":"2024-06-10T01:01:57.332437+0000","last_became_peered":"2024-06-10T01:01:57.332437+0000","last_unstale":"2024-06-10T01:03:38.270272+0000","last_undegraded":"2024-06-10T01:03:38.270272+0000","last_fullsized":"2024-06-10T01:03:38.270272+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:27:35.029079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559754+0000","last_change":"2024-06-10T01:01:47.255926+0000","last_active":"2024-06-10T01:04:35.559754+0000","last_peered":"2024-06-10T01:04:35.559754+0000","last_clean":"2024-06-10T01:04:35.559754+0000","last_became_active":"2024-06-10T01:01:47.255571+0000","last_became_peered":"2024-06-10T01:01:47.255571+0000","last_unstale":"2024-06-10T01:04:35.559754+0000","last_undegraded":"2024-06-10T01:04:35.559754+0000","last_fullsized":"2024-06-10T01:04:35.559754+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:05:22.084037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234997+0000","last_change":"2024-06-10T01:01:44.007509+0000","last_active":"2024-06-10T01:03:38.234997+0000","last_peered":"2024-06-10T01:03:38.234997+0000","last_clean":"2024-06-10T01:03:38.234997+0000","last_became_active":"2024-06-10T01:01:44.007178+0000","last_became_peered":"2024-06-10T01:01:44.007178+0000","last_unstale":"2024-06-10T01:03:38.234997+0000","last_undegraded":"2024-06-10T01:03:38.234997+0000","last_fullsized":"2024-06-10T01:03:38.234997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:02:52.791886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.236790+0000","last_change":"2024-06-10T01:01:57.285626+0000","last_active":"2024-06-10T01:03:38.236790+0000","last_peered":"2024-06-10T01:03:38.236790+0000","last_clean":"2024-06-10T01:03:38.236790+0000","last_became_active":"2024-06-10T01:01:57.285098+0000","last_became_peered":"2024-06-10T01:01:57.285098+0000","last_unstale":"2024-06-10T01:03:38.236790+0000","last_undegraded":"2024-06-10T01:03:38.236790+0000","last_fullsized":"2024-06-10T01:03:38.236790+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:13:49.881770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235332+0000","last_change":"2024-06-10T01:01:44.469028+0000","last_active":"2024-06-10T01:03:38.235332+0000","last_peered":"2024-06-10T01:03:38.235332+0000","last_clean":"2024-06-10T01:03:38.235332+0000","last_became_active":"2024-06-10T01:01:44.468570+0000","last_became_peered":"2024-06-10T01:01:44.468570+0000","last_unstale":"2024-06-10T01:03:38.235332+0000","last_undegraded":"2024-06-10T01:03:38.235332+0000","last_fullsized":"2024-06-10T01:03:38.235332+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:12:11.287959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.560160+0000","last_change":"2024-06-10T01:01:47.253414+0000","last_active":"2024-06-10T01:04:35.560160+0000","last_peered":"2024-06-10T01:04:35.560160+0000","last_clean":"2024-06-10T01:04:35.560160+0000","last_became_active":"2024-06-10T01:01:47.253060+0000","last_became_peered":"2024-06-10T01:01:47.253060+0000","last_unstale":"2024-06-10T01:04:35.560160+0000","last_undegraded":"2024-06-10T01:04:35.560160+0000","last_fullsized":"2024-06-10T01:04:35.560160+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:24:55.534164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.238849+0000","last_change":"2024-06-10T01:01:57.332756+0000","last_active":"2024-06-10T01:03:38.238849+0000","last_peered":"2024-06-10T01:03:38.238849+0000","last_clean":"2024-06-10T01:03:38.238849+0000","last_became_active":"2024-06-10T01:01:57.332361+0000","last_became_peered":"2024-06-10T01:01:57.332361+0000","last_unstale":"2024-06-10T01:03:38.238849+0000","last_undegraded":"2024-06-10T01:03:38.238849+0000","last_fullsized":"2024-06-10T01:03:38.238849+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:02:28.275407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"99'14","reported_seq":59,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:36.265950+0000","last_change":"2024-06-10T01:01:44.005164+0000","last_active":"2024-06-10T01:04:36.265950+0000","last_peered":"2024-06-10T01:04:36.265950+0000","last_clean":"2024-06-10T01:04:36.265950+0000","last_became_active":"2024-06-10T01:01:44.004816+0000","last_became_peered":"2024-06-10T01:01:44.004816+0000","last_unstale":"2024-06-10T01:04:36.265950+0000","last_undegraded":"2024-06-10T01:04:36.265950+0000","last_fullsized":"2024-06-10T01:04:36.265950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:11:56.385267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.557511+0000","last_change":"2024-06-10T01:01:47.247951+0000","last_active":"2024-06-10T01:04:35.557511+0000","last_peered":"2024-06-10T01:04:35.557511+0000","last_clean":"2024-06-10T01:04:35.557511+0000","last_became_active":"2024-06-10T01:01:47.246996+0000","last_became_peered":"2024-06-10T01:01:47.246996+0000","last_unstale":"2024-06-10T01:04:35.557511+0000","last_undegraded":"2024-06-10T01:04:35.557511+0000","last_fullsized":"2024-06-10T01:04:35.557511+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:39:18.167021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239228+0000","last_change":"2024-06-10T01:01:57.278130+0000","last_active":"2024-06-10T01:03:38.239228+0000","last_peered":"2024-06-10T01:03:38.239228+0000","last_clean":"2024-06-10T01:03:38.239228+0000","last_became_active":"2024-06-10T01:01:57.277697+0000","last_became_peered":"2024-06-10T01:01:57.277697+0000","last_unstale":"2024-06-10T01:03:38.239228+0000","last_undegraded":"2024-06-10T01:03:38.239228+0000","last_fullsized":"2024-06-10T01:03:38.239228+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:17:01.662152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239618+0000","last_change":"2024-06-10T01:01:44.553176+0000","last_active":"2024-06-10T01:03:38.239618+0000","last_peered":"2024-06-10T01:03:38.239618+0000","last_clean":"2024-06-10T01:03:38.239618+0000","last_became_active":"2024-06-10T01:01:44.552797+0000","last_became_peered":"2024-06-10T01:01:44.552797+0000","last_unstale":"2024-06-10T01:03:38.239618+0000","last_undegraded":"2024-06-10T01:03:38.239618+0000","last_fullsized":"2024-06-10T01:03:38.239618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:21:13.960304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.556104+0000","last_change":"2024-06-10T01:01:47.255796+0000","last_active":"2024-06-10T01:04:35.556104+0000","last_peered":"2024-06-10T01:04:35.556104+0000","last_clean":"2024-06-10T01:04:35.556104+0000","last_became_active":"2024-06-10T01:01:47.255495+0000","last_became_peered":"2024-06-10T01:01:47.255495+0000","last_unstale":"2024-06-10T01:04:35.556104+0000","last_undegraded":"2024-06-10T01:04:35.556104+0000","last_fullsized":"2024-06-10T01:04:35.556104+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:06:42.209005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239491+0000","last_change":"2024-06-10T01:01:57.332869+0000","last_active":"2024-06-10T01:03:38.239491+0000","last_peered":"2024-06-10T01:03:38.239491+0000","last_clean":"2024-06-10T01:03:38.239491+0000","last_became_active":"2024-06-10T01:01:57.332391+0000","last_became_peered":"2024-06-10T01:01:57.332391+0000","last_unstale":"2024-06-10T01:03:38.239491+0000","last_undegraded":"2024-06-10T01:03:38.239491+0000","last_fullsized":"2024-06-10T01:03:38.239491+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:58:31.443626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770788+0000","last_change":"2024-06-10T01:01:44.466861+0000","last_active":"2024-06-10T01:03:38.770788+0000","last_peered":"2024-06-10T01:03:38.770788+0000","last_clean":"2024-06-10T01:03:38.770788+0000","last_became_active":"2024-06-10T01:01:44.466553+0000","last_became_peered":"2024-06-10T01:01:44.466553+0000","last_unstale":"2024-06-10T01:03:38.770788+0000","last_undegraded":"2024-06-10T01:03:38.770788+0000","last_fullsized":"2024-06-10T01:03:38.770788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:55:52.545594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":41,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.544035+0000","last_change":"2024-06-10T01:01:47.244972+0000","last_active":"2024-06-10T01:04:35.544035+0000","last_peered":"2024-06-10T01:04:35.544035+0000","last_clean":"2024-06-10T01:04:35.544035+0000","last_became_active":"2024-06-10T01:01:47.244649+0000","last_became_peered":"2024-06-10T01:01:47.244649+0000","last_unstale":"2024-06-10T01:04:35.544035+0000","last_undegraded":"2024-06-10T01:04:35.544035+0000","last_fullsized":"2024-06-10T01:04:35.544035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:10:46.909200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235461+0000","last_change":"2024-06-10T01:02:37.592380+0000","last_active":"2024-06-10T01:03:38.235461+0000","last_peered":"2024-06-10T01:03:38.235461+0000","last_clean":"2024-06-10T01:03:38.235461+0000","last_became_active":"2024-06-10T01:02:37.591856+0000","last_became_peered":"2024-06-10T01:02:37.591856+0000","last_unstale":"2024-06-10T01:03:38.235461+0000","last_undegraded":"2024-06-10T01:03:38.235461+0000","last_fullsized":"2024-06-10T01:03:38.235461+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:00:25.717192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239006+0000","last_change":"2024-06-10T01:01:44.008116+0000","last_active":"2024-06-10T01:03:38.239006+0000","last_peered":"2024-06-10T01:03:38.239006+0000","last_clean":"2024-06-10T01:03:38.239006+0000","last_became_active":"2024-06-10T01:01:44.007778+0000","last_became_peered":"2024-06-10T01:01:44.007778+0000","last_unstale":"2024-06-10T01:03:38.239006+0000","last_undegraded":"2024-06-10T01:03:38.239006+0000","last_fullsized":"2024-06-10T01:03:38.239006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:18:57.267296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.556264+0000","last_change":"2024-06-10T01:01:47.252833+0000","last_active":"2024-06-10T01:04:35.556264+0000","last_peered":"2024-06-10T01:04:35.556264+0000","last_clean":"2024-06-10T01:04:35.556264+0000","last_became_active":"2024-06-10T01:01:47.252326+0000","last_became_peered":"2024-06-10T01:01:47.252326+0000","last_unstale":"2024-06-10T01:04:35.556264+0000","last_undegraded":"2024-06-10T01:04:35.556264+0000","last_fullsized":"2024-06-10T01:04:35.556264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:00:42.673838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239284+0000","last_change":"2024-06-10T01:01:57.275975+0000","last_active":"2024-06-10T01:03:38.239284+0000","last_peered":"2024-06-10T01:03:38.239284+0000","last_clean":"2024-06-10T01:03:38.239284+0000","last_became_active":"2024-06-10T01:01:57.275408+0000","last_became_peered":"2024-06-10T01:01:57.275408+0000","last_unstale":"2024-06-10T01:03:38.239284+0000","last_undegraded":"2024-06-10T01:03:38.239284+0000","last_fullsized":"2024-06-10T01:03:38.239284+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:12:42.376636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'1","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.243874+0000","last_change":"2024-06-10T01:03:38.243422+0000","last_active":"2024-06-10T01:03:38.243874+0000","last_peered":"2024-06-10T01:03:38.243874+0000","last_clean":"2024-06-10T01:03:38.243874+0000","last_became_active":"2024-06-10T01:03:38.239916+0000","last_became_peered":"2024-06-10T01:03:38.239916+0000","last_unstale":"2024-06-10T01:03:38.243874+0000","last_undegraded":"2024-06-10T01:03:38.243874+0000","last_fullsized":"2024-06-10T01:03:38.243874+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:30:01.399956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.555295+0000","last_change":"2024-06-10T01:01:47.243154+0000","last_active":"2024-06-10T01:04:35.555295+0000","last_peered":"2024-06-10T01:04:35.555295+0000","last_clean":"2024-06-10T01:04:35.555295+0000","last_became_active":"2024-06-10T01:01:47.242851+0000","last_became_peered":"2024-06-10T01:01:47.242851+0000","last_unstale":"2024-06-10T01:04:35.555295+0000","last_undegraded":"2024-06-10T01:04:35.555295+0000","last_fullsized":"2024-06-10T01:04:35.555295+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:41:05.857777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239584+0000","last_change":"2024-06-10T01:01:57.333613+0000","last_active":"2024-06-10T01:03:38.239584+0000","last_peered":"2024-06-10T01:03:38.239584+0000","last_clean":"2024-06-10T01:03:38.239584+0000","last_became_active":"2024-06-10T01:01:57.333019+0000","last_became_peered":"2024-06-10T01:01:57.333019+0000","last_unstale":"2024-06-10T01:03:38.239584+0000","last_undegraded":"2024-06-10T01:03:38.239584+0000","last_fullsized":"2024-06-10T01:03:38.239584+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:11.912885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235303+0000","last_change":"2024-06-10T01:01:44.008221+0000","last_active":"2024-06-10T01:03:38.235303+0000","last_peered":"2024-06-10T01:03:38.235303+0000","last_clean":"2024-06-10T01:03:38.235303+0000","last_became_active":"2024-06-10T01:01:44.007880+0000","last_became_peered":"2024-06-10T01:01:44.007880+0000","last_unstale":"2024-06-10T01:03:38.235303+0000","last_undegraded":"2024-06-10T01:03:38.235303+0000","last_fullsized":"2024-06-10T01:03:38.235303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:40:33.162925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558463+0000","last_change":"2024-06-10T01:01:47.254902+0000","last_active":"2024-06-10T01:04:35.558463+0000","last_peered":"2024-06-10T01:04:35.558463+0000","last_clean":"2024-06-10T01:04:35.558463+0000","last_became_active":"2024-06-10T01:01:47.254555+0000","last_became_peered":"2024-06-10T01:01:47.254555+0000","last_unstale":"2024-06-10T01:04:35.558463+0000","last_undegraded":"2024-06-10T01:04:35.558463+0000","last_fullsized":"2024-06-10T01:04:35.558463+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:03:17.571001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770786+0000","last_change":"2024-06-10T01:01:57.284491+0000","last_active":"2024-06-10T01:03:38.770786+0000","last_peered":"2024-06-10T01:03:38.770786+0000","last_clean":"2024-06-10T01:03:38.770786+0000","last_became_active":"2024-06-10T01:01:57.283795+0000","last_became_peered":"2024-06-10T01:01:57.283795+0000","last_unstale":"2024-06-10T01:03:38.770786+0000","last_undegraded":"2024-06-10T01:03:38.770786+0000","last_fullsized":"2024-06-10T01:03:38.770786+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:49:00.381664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239889+0000","last_change":"2024-06-10T01:01:44.553159+0000","last_active":"2024-06-10T01:03:38.239889+0000","last_peered":"2024-06-10T01:03:38.239889+0000","last_clean":"2024-06-10T01:03:38.239889+0000","last_became_active":"2024-06-10T01:01:44.552759+0000","last_became_peered":"2024-06-10T01:01:44.552759+0000","last_unstale":"2024-06-10T01:03:38.239889+0000","last_undegraded":"2024-06-10T01:03:38.239889+0000","last_fullsized":"2024-06-10T01:03:38.239889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:19:00.110790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559093+0000","last_change":"2024-06-10T01:01:47.247803+0000","last_active":"2024-06-10T01:04:35.559093+0000","last_peered":"2024-06-10T01:04:35.559093+0000","last_clean":"2024-06-10T01:04:35.559093+0000","last_became_active":"2024-06-10T01:01:47.246715+0000","last_became_peered":"2024-06-10T01:01:47.246715+0000","last_unstale":"2024-06-10T01:04:35.559093+0000","last_undegraded":"2024-06-10T01:04:35.559093+0000","last_fullsized":"2024-06-10T01:04:35.559093+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:22.429002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234192+0000","last_change":"2024-06-10T01:01:57.287668+0000","last_active":"2024-06-10T01:03:38.234192+0000","last_peered":"2024-06-10T01:03:38.234192+0000","last_clean":"2024-06-10T01:03:38.234192+0000","last_became_active":"2024-06-10T01:01:57.287060+0000","last_became_peered":"2024-06-10T01:01:57.287060+0000","last_unstale":"2024-06-10T01:03:38.234192+0000","last_undegraded":"2024-06-10T01:03:38.234192+0000","last_fullsized":"2024-06-10T01:03:38.234192+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:04:35.883613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270251+0000","last_change":"2024-06-10T01:01:44.013051+0000","last_active":"2024-06-10T01:03:38.270251+0000","last_peered":"2024-06-10T01:03:38.270251+0000","last_clean":"2024-06-10T01:03:38.270251+0000","last_became_active":"2024-06-10T01:01:44.012667+0000","last_became_peered":"2024-06-10T01:01:44.012667+0000","last_unstale":"2024-06-10T01:03:38.270251+0000","last_undegraded":"2024-06-10T01:03:38.270251+0000","last_fullsized":"2024-06-10T01:03:38.270251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:10:49.650082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.563083+0000","last_change":"2024-06-10T01:01:47.251209+0000","last_active":"2024-06-10T01:04:35.563083+0000","last_peered":"2024-06-10T01:04:35.563083+0000","last_clean":"2024-06-10T01:04:35.563083+0000","last_became_active":"2024-06-10T01:01:47.250858+0000","last_became_peered":"2024-06-10T01:01:47.250858+0000","last_unstale":"2024-06-10T01:04:35.563083+0000","last_undegraded":"2024-06-10T01:04:35.563083+0000","last_fullsized":"2024-06-10T01:04:35.563083+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:47.272727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.269600+0000","last_change":"2024-06-10T01:01:57.284187+0000","last_active":"2024-06-10T01:03:38.269600+0000","last_peered":"2024-06-10T01:03:38.269600+0000","last_clean":"2024-06-10T01:03:38.269600+0000","last_became_active":"2024-06-10T01:01:57.283739+0000","last_became_peered":"2024-06-10T01:01:57.283739+0000","last_unstale":"2024-06-10T01:03:38.269600+0000","last_undegraded":"2024-06-10T01:03:38.269600+0000","last_fullsized":"2024-06-10T01:03:38.269600+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:13:34.630708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771085+0000","last_change":"2024-06-10T01:01:44.005702+0000","last_active":"2024-06-10T01:03:38.771085+0000","last_peered":"2024-06-10T01:03:38.771085+0000","last_clean":"2024-06-10T01:03:38.771085+0000","last_became_active":"2024-06-10T01:01:44.005354+0000","last_became_peered":"2024-06-10T01:01:44.005354+0000","last_unstale":"2024-06-10T01:03:38.771085+0000","last_undegraded":"2024-06-10T01:03:38.771085+0000","last_fullsized":"2024-06-10T01:03:38.771085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:51:14.819688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558942+0000","last_change":"2024-06-10T01:01:47.250325+0000","last_active":"2024-06-10T01:04:35.558942+0000","last_peered":"2024-06-10T01:04:35.558942+0000","last_clean":"2024-06-10T01:04:35.558942+0000","last_became_active":"2024-06-10T01:01:47.249591+0000","last_became_peered":"2024-06-10T01:01:47.249591+0000","last_unstale":"2024-06-10T01:04:35.558942+0000","last_undegraded":"2024-06-10T01:04:35.558942+0000","last_fullsized":"2024-06-10T01:04:35.558942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:17:16.268717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.563635+0000","last_change":"2024-06-10T01:01:47.253449+0000","last_active":"2024-06-10T01:04:35.563635+0000","last_peered":"2024-06-10T01:04:35.563635+0000","last_clean":"2024-06-10T01:04:35.563635+0000","last_became_active":"2024-06-10T01:01:47.253144+0000","last_became_peered":"2024-06-10T01:01:47.253144+0000","last_unstale":"2024-06-10T01:04:35.563635+0000","last_undegraded":"2024-06-10T01:04:35.563635+0000","last_fullsized":"2024-06-10T01:04:35.563635+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:32:22.353409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.564016+0000","last_change":"2024-06-10T01:01:47.258241+0000","last_active":"2024-06-10T01:04:35.564016+0000","last_peered":"2024-06-10T01:04:35.564016+0000","last_clean":"2024-06-10T01:04:35.564016+0000","last_became_active":"2024-06-10T01:01:47.257823+0000","last_became_peered":"2024-06-10T01:01:47.257823+0000","last_unstale":"2024-06-10T01:04:35.564016+0000","last_undegraded":"2024-06-10T01:04:35.564016+0000","last_fullsized":"2024-06-10T01:04:35.564016+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:11:23.652479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559410+0000","last_change":"2024-06-10T01:01:47.256218+0000","last_active":"2024-06-10T01:04:35.559410+0000","last_peered":"2024-06-10T01:04:35.559410+0000","last_clean":"2024-06-10T01:04:35.559410+0000","last_became_active":"2024-06-10T01:01:47.255950+0000","last_became_peered":"2024-06-10T01:01:47.255950+0000","last_unstale":"2024-06-10T01:04:35.559410+0000","last_undegraded":"2024-06-10T01:04:35.559410+0000","last_fullsized":"2024-06-10T01:04:35.559410+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:59:37.283646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.556085+0000","last_change":"2024-06-10T01:01:47.253109+0000","last_active":"2024-06-10T01:04:35.556085+0000","last_peered":"2024-06-10T01:04:35.556085+0000","last_clean":"2024-06-10T01:04:35.556085+0000","last_became_active":"2024-06-10T01:01:47.252767+0000","last_became_peered":"2024-06-10T01:01:47.252767+0000","last_unstale":"2024-06-10T01:04:35.556085+0000","last_undegraded":"2024-06-10T01:04:35.556085+0000","last_fullsized":"2024-06-10T01:04:35.556085+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:57:13.177230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.550864+0000","last_change":"2024-06-10T01:02:37.375094+0000","last_active":"2024-06-10T01:04:35.550864+0000","last_peered":"2024-06-10T01:04:35.550864+0000","last_clean":"2024-06-10T01:04:35.550864+0000","last_became_active":"2024-06-10T01:02:37.374527+0000","last_became_peered":"2024-06-10T01:02:37.374527+0000","last_unstale":"2024-06-10T01:04:35.550864+0000","last_undegraded":"2024-06-10T01:04:35.550864+0000","last_fullsized":"2024-06-10T01:04:35.550864+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:34:41.205816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.551206+0000","last_change":"2024-06-10T01:01:47.250243+0000","last_active":"2024-06-10T01:04:35.551206+0000","last_peered":"2024-06-10T01:04:35.551206+0000","last_clean":"2024-06-10T01:04:35.551206+0000","last_became_active":"2024-06-10T01:01:47.249907+0000","last_became_peered":"2024-06-10T01:01:47.249907+0000","last_unstale":"2024-06-10T01:04:35.551206+0000","last_undegraded":"2024-06-10T01:04:35.551206+0000","last_fullsized":"2024-06-10T01:04:35.551206+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:31:32.933660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.564063+0000","last_change":"2024-06-10T01:01:47.254173+0000","last_active":"2024-06-10T01:04:35.564063+0000","last_peered":"2024-06-10T01:04:35.564063+0000","last_clean":"2024-06-10T01:04:35.564063+0000","last_became_active":"2024-06-10T01:01:47.253796+0000","last_became_peered":"2024-06-10T01:01:47.253796+0000","last_unstale":"2024-06-10T01:04:35.564063+0000","last_undegraded":"2024-06-10T01:04:35.564063+0000","last_fullsized":"2024-06-10T01:04:35.564063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:41:12.199114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":41,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.548118+0000","last_change":"2024-06-10T01:01:47.253920+0000","last_active":"2024-06-10T01:04:35.548118+0000","last_peered":"2024-06-10T01:04:35.548118+0000","last_clean":"2024-06-10T01:04:35.548118+0000","last_became_active":"2024-06-10T01:01:47.253288+0000","last_became_peered":"2024-06-10T01:01:47.253288+0000","last_unstale":"2024-06-10T01:04:35.548118+0000","last_undegraded":"2024-06-10T01:04:35.548118+0000","last_fullsized":"2024-06-10T01:04:35.548118+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:41:20.406263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.564816+0000","last_change":"2024-06-10T01:01:47.240772+0000","last_active":"2024-06-10T01:04:35.564816+0000","last_peered":"2024-06-10T01:04:35.564816+0000","last_clean":"2024-06-10T01:04:35.564816+0000","last_became_active":"2024-06-10T01:01:47.238258+0000","last_became_peered":"2024-06-10T01:01:47.238258+0000","last_unstale":"2024-06-10T01:04:35.564816+0000","last_undegraded":"2024-06-10T01:04:35.564816+0000","last_fullsized":"2024-06-10T01:04:35.564816+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:55:25.211434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":24,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.544703+0000","last_change":"2024-06-10T01:02:37.376724+0000","last_active":"2024-06-10T01:04:35.544703+0000","last_peered":"2024-06-10T01:04:35.544703+0000","last_clean":"2024-06-10T01:04:35.544703+0000","last_became_active":"2024-06-10T01:02:37.376329+0000","last_became_peered":"2024-06-10T01:02:37.376329+0000","last_unstale":"2024-06-10T01:04:35.544703+0000","last_undegraded":"2024-06-10T01:04:35.544703+0000","last_fullsized":"2024-06-10T01:04:35.544703+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:38:21.837958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.565197+0000","last_change":"2024-06-10T01:01:47.244463+0000","last_active":"2024-06-10T01:04:35.565197+0000","last_peered":"2024-06-10T01:04:35.565197+0000","last_clean":"2024-06-10T01:04:35.565197+0000","last_became_active":"2024-06-10T01:01:47.244114+0000","last_became_peered":"2024-06-10T01:01:47.244114+0000","last_unstale":"2024-06-10T01:04:35.565197+0000","last_undegraded":"2024-06-10T01:04:35.565197+0000","last_fullsized":"2024-06-10T01:04:35.565197+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:47:37.098894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559211+0000","last_change":"2024-06-10T01:01:47.248317+0000","last_active":"2024-06-10T01:04:35.559211+0000","last_peered":"2024-06-10T01:04:35.559211+0000","last_clean":"2024-06-10T01:04:35.559211+0000","last_became_active":"2024-06-10T01:01:47.247958+0000","last_became_peered":"2024-06-10T01:01:47.247958+0000","last_unstale":"2024-06-10T01:04:35.559211+0000","last_undegraded":"2024-06-10T01:04:35.559211+0000","last_fullsized":"2024-06-10T01:04:35.559211+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:39:59.267957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.552389+0000","last_change":"2024-06-10T01:01:47.252542+0000","last_active":"2024-06-10T01:04:35.552389+0000","last_peered":"2024-06-10T01:04:35.552389+0000","last_clean":"2024-06-10T01:04:35.552389+0000","last_became_active":"2024-06-10T01:01:47.252216+0000","last_became_peered":"2024-06-10T01:01:47.252216+0000","last_unstale":"2024-06-10T01:04:35.552389+0000","last_undegraded":"2024-06-10T01:04:35.552389+0000","last_fullsized":"2024-06-10T01:04:35.552389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:12:22.196263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285691,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68533,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1018,"num_read_kb":581,"num_write":183,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":205599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383743,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":569843,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645380,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32388,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712764,"statfs":{"total":95995035648,"available":95961870336,"internally_reserved":0,"allocated":1241088,"data_stored":1221922,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808905,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":574063,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070542,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":569775,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332178,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":1165405,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495703,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":565143,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574960,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":629668,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738486,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":566531,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411435,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":630598,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182503,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":565337,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920865,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":569905,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757341,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":1160993,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34687,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34687,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34687,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":65853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T01:05:44.054 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-10T01:05:44.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.162+0000 7f6a99a12640 1 -- 172.21.15.7:0/3365328374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a94101a00 msgr2=0x7f6a94101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:44.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.162+0000 7f6a99a12640 1 --2- 172.21.15.7:0/3365328374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a94101a00 0x7f6a94101e20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6a84009900 tx=0x7f6a8402a410 comp rx=0 tx=0).stop 2024-06-10T01:05:44.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.162+0000 7f6a99a12640 1 -- 172.21.15.7:0/3365328374 shutdown_connections 2024-06-10T01:05:44.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.162+0000 7f6a99a12640 1 --2- 172.21.15.7:0/3365328374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a941035b0 0x7f6a94109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:44.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.162+0000 7f6a99a12640 1 --2- 172.21.15.7:0/3365328374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a94102bf0 0x7f6a94103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:44.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.162+0000 7f6a99a12640 1 --2- 172.21.15.7:0/3365328374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a94101a00 0x7f6a94101e20 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:44.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.162+0000 7f6a99a12640 1 -- 172.21.15.7:0/3365328374 >> 172.21.15.7:0/3365328374 conn(0x7f6a940fcf90 msgr2=0x7f6a940ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:44.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.162+0000 7f6a99a12640 1 -- 172.21.15.7:0/3365328374 shutdown_connections 2024-06-10T01:05:44.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.162+0000 7f6a99a12640 1 -- 172.21.15.7:0/3365328374 wait complete. 2024-06-10T01:05:44.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.163+0000 7f6a99a12640 1 Processor -- start 2024-06-10T01:05:44.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.163+0000 7f6a99a12640 1 -- start start 2024-06-10T01:05:44.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.163+0000 7f6a99a12640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a94101a00 0x7f6a94199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:44.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.163+0000 7f6a99a12640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a94102bf0 0x7f6a9419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:44.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.163+0000 7f6a99a12640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a941035b0 0x7f6a941a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:44.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.163+0000 7f6a99a12640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9410d650 con 0x7f6a941035b0 2024-06-10T01:05:44.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.163+0000 7f6a99a12640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9410d4d0 con 0x7f6a94101a00 2024-06-10T01:05:44.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.163+0000 7f6a99a12640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9410d7d0 con 0x7f6a94102bf0 2024-06-10T01:05:44.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.163+0000 7f6a937fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a941035b0 0x7f6a941a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:44.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.163+0000 7f6a937fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a941035b0 0x7f6a941a1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58570/0 (socket says 172.21.15.7:58570) 2024-06-10T01:05:44.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.163+0000 7f6a937fe640 1 -- 172.21.15.7:0/4282678223 learned_addr learned my addr 172.21.15.7:0/4282678223 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:44.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.163+0000 7f6a92ffd640 1 --2- 172.21.15.7:0/4282678223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a94101a00 0x7f6a94199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:44.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.163+0000 7f6a8bfff640 1 --2- 172.21.15.7:0/4282678223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a94102bf0 0x7f6a9419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:44.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.163+0000 7f6a937fe640 1 -- 172.21.15.7:0/4282678223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a94102bf0 msgr2=0x7f6a9419a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:44.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.163+0000 7f6a937fe640 1 --2- 172.21.15.7:0/4282678223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a94102bf0 0x7f6a9419a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:44.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.163+0000 7f6a937fe640 1 -- 172.21.15.7:0/4282678223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a94101a00 msgr2=0x7f6a94199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:44.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.163+0000 7f6a937fe640 1 --2- 172.21.15.7:0/4282678223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a94101a00 0x7f6a94199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:44.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.163+0000 7f6a937fe640 1 -- 172.21.15.7:0/4282678223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a941a1ad0 con 0x7f6a941035b0 2024-06-10T01:05:44.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.163+0000 7f6a8bfff640 1 --2- 172.21.15.7:0/4282678223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a94102bf0 0x7f6a9419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:05:44.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.164+0000 7f6a937fe640 1 --2- 172.21.15.7:0/4282678223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a941035b0 0x7f6a941a1460 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00eaa0 tx=0x7f6a8c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:44.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.164+0000 7f6a98a10640 1 -- 172.21.15.7:0/4282678223 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c00cdb0 con 0x7f6a941035b0 2024-06-10T01:05:44.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.164+0000 7f6a98a10640 1 -- 172.21.15.7:0/4282678223 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a8c005020 con 0x7f6a941035b0 2024-06-10T01:05:44.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.164+0000 7f6a99a12640 1 -- 172.21.15.7:0/4282678223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a941a1dc0 con 0x7f6a941035b0 2024-06-10T01:05:44.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.164+0000 7f6a98a10640 1 -- 172.21.15.7:0/4282678223 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c0116b0 con 0x7f6a941035b0 2024-06-10T01:05:44.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.164+0000 7f6a99a12640 1 -- 172.21.15.7:0/4282678223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a941a2230 con 0x7f6a941035b0 2024-06-10T01:05:44.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.165+0000 7f6a99a12640 1 -- 172.21.15.7:0/4282678223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a941a24f0 con 0x7f6a941035b0 2024-06-10T01:05:44.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.166+0000 7f6a98a10640 1 -- 172.21.15.7:0/4282678223 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f6a8c005560 con 0x7f6a941035b0 2024-06-10T01:05:44.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.167+0000 7f6a98a10640 1 --2- 172.21.15.7:0/4282678223 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6a800a2b90 0x7f6a800a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:44.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.167+0000 7f6a98a10640 1 -- 172.21.15.7:0/4282678223 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f6a8c0cf9b0 con 0x7f6a941035b0 2024-06-10T01:05:44.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.167+0000 7f6a92ffd640 1 --2- 172.21.15.7:0/4282678223 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6a800a2b90 0x7f6a800a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:44.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.167+0000 7f6a92ffd640 1 --2- 172.21.15.7:0/4282678223 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6a800a2b90 0x7f6a800a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6a84005ab0 tx=0x7f6a84000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:44.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.170+0000 7f6a98a10640 1 -- 172.21.15.7:0/4282678223 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6a8c016030 con 0x7f6a941035b0 2024-06-10T01:05:44.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.351+0000 7f6a99a12640 1 -- 172.21.15.7:0/4282678223 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6a941a26c0 con 0x7f6a800a2b90 2024-06-10T01:05:44.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.361+0000 7f6a98a10640 1 -- 172.21.15.7:0/4282678223 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340766 (secure 0 0 0) 0x7f6a941a26c0 con 0x7f6a800a2b90 2024-06-10T01:05:44.362 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:05:44.364 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-06-10T01:05:44.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.364+0000 7f6a99a12640 1 -- 172.21.15.7:0/4282678223 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6a800a2b90 msgr2=0x7f6a800a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:44.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.364+0000 7f6a99a12640 1 --2- 172.21.15.7:0/4282678223 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6a800a2b90 0x7f6a800a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6a84005ab0 tx=0x7f6a84000bc0 comp rx=0 tx=0).stop 2024-06-10T01:05:44.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.364+0000 7f6a99a12640 1 -- 172.21.15.7:0/4282678223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a941035b0 msgr2=0x7f6a941a1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:44.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.364+0000 7f6a99a12640 1 --2- 172.21.15.7:0/4282678223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a941035b0 0x7f6a941a1460 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00eaa0 tx=0x7f6a8c00ef60 comp rx=0 tx=0).stop 2024-06-10T01:05:44.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.364+0000 7f6a99a12640 1 -- 172.21.15.7:0/4282678223 shutdown_connections 2024-06-10T01:05:44.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.364+0000 7f6a99a12640 1 --2- 172.21.15.7:0/4282678223 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6a800a2b90 0x7f6a800a5050 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:44.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.364+0000 7f6a99a12640 1 --2- 172.21.15.7:0/4282678223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a941035b0 0x7f6a941a1460 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:44.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.364+0000 7f6a99a12640 1 --2- 172.21.15.7:0/4282678223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a94102bf0 0x7f6a9419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:44.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.364+0000 7f6a99a12640 1 --2- 172.21.15.7:0/4282678223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a94101a00 0x7f6a94199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:44.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.364+0000 7f6a99a12640 1 -- 172.21.15.7:0/4282678223 >> 172.21.15.7:0/4282678223 conn(0x7f6a940fcf90 msgr2=0x7f6a94073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:44.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.364+0000 7f6a99a12640 1 -- 172.21.15.7:0/4282678223 shutdown_connections 2024-06-10T01:05:44.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.364+0000 7f6a99a12640 1 -- 172.21.15.7:0/4282678223 wait complete. 2024-06-10T01:05:44.373 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":423,"stamp":"2024-06-10T01:05:43.597017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659135,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1144,"num_read_kb":690,"num_write":366,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":793,"ondisk_log_size":793,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362468,"kb_used_data":13316,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579356,"statfs":{"total":1151940427776,"available":1151569260544,"internally_reserved":0,"allocated":13635584,"data_stored":8789183,"data_compressed":496626,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":64,"num_read_kb":30,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003015"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.569445+0000","last_change":"2024-06-10T01:01:47.244728+0000","last_active":"2024-06-10T01:04:35.569445+0000","last_peered":"2024-06-10T01:04:35.569445+0000","last_clean":"2024-06-10T01:04:35.569445+0000","last_became_active":"2024-06-10T01:01:47.244198+0000","last_became_peered":"2024-06-10T01:01:47.244198+0000","last_unstale":"2024-06-10T01:04:35.569445+0000","last_undegraded":"2024-06-10T01:04:35.569445+0000","last_fullsized":"2024-06-10T01:04:35.569445+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:58:06.257952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.557217+0000","last_change":"2024-06-10T01:01:47.247620+0000","last_active":"2024-06-10T01:04:35.557217+0000","last_peered":"2024-06-10T01:04:35.557217+0000","last_clean":"2024-06-10T01:04:35.557217+0000","last_became_active":"2024-06-10T01:01:47.245475+0000","last_became_peered":"2024-06-10T01:01:47.245475+0000","last_unstale":"2024-06-10T01:04:35.557217+0000","last_undegraded":"2024-06-10T01:04:35.557217+0000","last_fullsized":"2024-06-10T01:04:35.557217+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:29:09.367411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.569660+0000","last_change":"2024-06-10T01:01:47.252414+0000","last_active":"2024-06-10T01:04:35.569660+0000","last_peered":"2024-06-10T01:04:35.569660+0000","last_clean":"2024-06-10T01:04:35.569660+0000","last_became_active":"2024-06-10T01:01:47.252063+0000","last_became_peered":"2024-06-10T01:01:47.252063+0000","last_unstale":"2024-06-10T01:04:35.569660+0000","last_undegraded":"2024-06-10T01:04:35.569660+0000","last_fullsized":"2024-06-10T01:04:35.569660+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:54:53.966209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.553564+0000","last_change":"2024-06-10T01:01:47.254875+0000","last_active":"2024-06-10T01:04:35.553564+0000","last_peered":"2024-06-10T01:04:35.553564+0000","last_clean":"2024-06-10T01:04:35.553564+0000","last_became_active":"2024-06-10T01:01:47.254542+0000","last_became_peered":"2024-06-10T01:01:47.254542+0000","last_unstale":"2024-06-10T01:04:35.553564+0000","last_undegraded":"2024-06-10T01:04:35.553564+0000","last_fullsized":"2024-06-10T01:04:35.553564+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:02:28.393133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558663+0000","last_change":"2024-06-10T01:01:47.242596+0000","last_active":"2024-06-10T01:04:35.558663+0000","last_peered":"2024-06-10T01:04:35.558663+0000","last_clean":"2024-06-10T01:04:35.558663+0000","last_became_active":"2024-06-10T01:01:47.242308+0000","last_became_peered":"2024-06-10T01:01:47.242308+0000","last_unstale":"2024-06-10T01:04:35.558663+0000","last_undegraded":"2024-06-10T01:04:35.558663+0000","last_fullsized":"2024-06-10T01:04:35.558663+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:11:26.599954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.550600+0000","last_change":"2024-06-10T01:01:47.249085+0000","last_active":"2024-06-10T01:04:35.550600+0000","last_peered":"2024-06-10T01:04:35.550600+0000","last_clean":"2024-06-10T01:04:35.550600+0000","last_became_active":"2024-06-10T01:01:47.248749+0000","last_became_peered":"2024-06-10T01:01:47.248749+0000","last_unstale":"2024-06-10T01:04:35.550600+0000","last_undegraded":"2024-06-10T01:04:35.550600+0000","last_fullsized":"2024-06-10T01:04:35.550600+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:42:48.734566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562583+0000","last_change":"2024-06-10T01:01:47.247334+0000","last_active":"2024-06-10T01:04:35.562583+0000","last_peered":"2024-06-10T01:04:35.562583+0000","last_clean":"2024-06-10T01:04:35.562583+0000","last_became_active":"2024-06-10T01:01:47.246964+0000","last_became_peered":"2024-06-10T01:01:47.246964+0000","last_unstale":"2024-06-10T01:04:35.562583+0000","last_undegraded":"2024-06-10T01:04:35.562583+0000","last_fullsized":"2024-06-10T01:04:35.562583+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:05:36.034950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.563056+0000","last_change":"2024-06-10T01:01:47.242523+0000","last_active":"2024-06-10T01:04:35.563056+0000","last_peered":"2024-06-10T01:04:35.563056+0000","last_clean":"2024-06-10T01:04:35.563056+0000","last_became_active":"2024-06-10T01:01:47.242198+0000","last_became_peered":"2024-06-10T01:01:47.242198+0000","last_unstale":"2024-06-10T01:04:35.563056+0000","last_undegraded":"2024-06-10T01:04:35.563056+0000","last_fullsized":"2024-06-10T01:04:35.563056+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:39:07.055970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.567116+0000","last_change":"2024-06-10T01:01:47.245277+0000","last_active":"2024-06-10T01:04:35.567116+0000","last_peered":"2024-06-10T01:04:35.567116+0000","last_clean":"2024-06-10T01:04:35.567116+0000","last_became_active":"2024-06-10T01:01:47.244932+0000","last_became_peered":"2024-06-10T01:01:47.244932+0000","last_unstale":"2024-06-10T01:04:35.567116+0000","last_undegraded":"2024-06-10T01:04:35.567116+0000","last_fullsized":"2024-06-10T01:04:35.567116+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:56:24.423713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559784+0000","last_change":"2024-06-10T01:01:47.255531+0000","last_active":"2024-06-10T01:04:35.559784+0000","last_peered":"2024-06-10T01:04:35.559784+0000","last_clean":"2024-06-10T01:04:35.559784+0000","last_became_active":"2024-06-10T01:01:47.255150+0000","last_became_peered":"2024-06-10T01:01:47.255150+0000","last_unstale":"2024-06-10T01:04:35.559784+0000","last_undegraded":"2024-06-10T01:04:35.559784+0000","last_fullsized":"2024-06-10T01:04:35.559784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:51:34.561396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559164+0000","last_change":"2024-06-10T01:01:47.249832+0000","last_active":"2024-06-10T01:04:35.559164+0000","last_peered":"2024-06-10T01:04:35.559164+0000","last_clean":"2024-06-10T01:04:35.559164+0000","last_became_active":"2024-06-10T01:01:47.249303+0000","last_became_peered":"2024-06-10T01:01:47.249303+0000","last_unstale":"2024-06-10T01:04:35.559164+0000","last_undegraded":"2024-06-10T01:04:35.559164+0000","last_fullsized":"2024-06-10T01:04:35.559164+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:15:21.930157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.551184+0000","last_change":"2024-06-10T01:01:47.255927+0000","last_active":"2024-06-10T01:04:35.551184+0000","last_peered":"2024-06-10T01:04:35.551184+0000","last_clean":"2024-06-10T01:04:35.551184+0000","last_became_active":"2024-06-10T01:01:47.255591+0000","last_became_peered":"2024-06-10T01:01:47.255591+0000","last_unstale":"2024-06-10T01:04:35.551184+0000","last_undegraded":"2024-06-10T01:04:35.551184+0000","last_fullsized":"2024-06-10T01:04:35.551184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:48:06.494430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.557880+0000","last_change":"2024-06-10T01:02:37.372069+0000","last_active":"2024-06-10T01:04:35.557880+0000","last_peered":"2024-06-10T01:04:35.557880+0000","last_clean":"2024-06-10T01:04:35.557880+0000","last_became_active":"2024-06-10T01:02:37.371599+0000","last_became_peered":"2024-06-10T01:02:37.371599+0000","last_unstale":"2024-06-10T01:04:35.557880+0000","last_undegraded":"2024-06-10T01:04:35.557880+0000","last_fullsized":"2024-06-10T01:04:35.557880+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:54:34.091576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.551120+0000","last_change":"2024-06-10T01:01:47.249553+0000","last_active":"2024-06-10T01:04:35.551120+0000","last_peered":"2024-06-10T01:04:35.551120+0000","last_clean":"2024-06-10T01:04:35.551120+0000","last_became_active":"2024-06-10T01:01:47.249202+0000","last_became_peered":"2024-06-10T01:01:47.249202+0000","last_unstale":"2024-06-10T01:04:35.551120+0000","last_undegraded":"2024-06-10T01:04:35.551120+0000","last_fullsized":"2024-06-10T01:04:35.551120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:23:57.882264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.564289+0000","last_change":"2024-06-10T01:01:47.253987+0000","last_active":"2024-06-10T01:04:35.564289+0000","last_peered":"2024-06-10T01:04:35.564289+0000","last_clean":"2024-06-10T01:04:35.564289+0000","last_became_active":"2024-06-10T01:01:47.253578+0000","last_became_peered":"2024-06-10T01:01:47.253578+0000","last_unstale":"2024-06-10T01:04:35.564289+0000","last_undegraded":"2024-06-10T01:04:35.564289+0000","last_fullsized":"2024-06-10T01:04:35.564289+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:35:30.585482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.567601+0000","last_change":"2024-06-10T01:01:47.247089+0000","last_active":"2024-06-10T01:04:35.567601+0000","last_peered":"2024-06-10T01:04:35.567601+0000","last_clean":"2024-06-10T01:04:35.567601+0000","last_became_active":"2024-06-10T01:01:47.246066+0000","last_became_peered":"2024-06-10T01:01:47.246066+0000","last_unstale":"2024-06-10T01:04:35.567601+0000","last_undegraded":"2024-06-10T01:04:35.567601+0000","last_fullsized":"2024-06-10T01:04:35.567601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:58:12.092981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.569804+0000","last_change":"2024-06-10T01:01:47.244491+0000","last_active":"2024-06-10T01:04:35.569804+0000","last_peered":"2024-06-10T01:04:35.569804+0000","last_clean":"2024-06-10T01:04:35.569804+0000","last_became_active":"2024-06-10T01:01:47.243747+0000","last_became_peered":"2024-06-10T01:01:47.243747+0000","last_unstale":"2024-06-10T01:04:35.569804+0000","last_undegraded":"2024-06-10T01:04:35.569804+0000","last_fullsized":"2024-06-10T01:04:35.569804+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:55:56.785016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.567988+0000","last_change":"2024-06-10T01:01:47.243141+0000","last_active":"2024-06-10T01:04:35.567988+0000","last_peered":"2024-06-10T01:04:35.567988+0000","last_clean":"2024-06-10T01:04:35.567988+0000","last_became_active":"2024-06-10T01:01:47.242464+0000","last_became_peered":"2024-06-10T01:01:47.242464+0000","last_unstale":"2024-06-10T01:04:35.567988+0000","last_undegraded":"2024-06-10T01:04:35.567988+0000","last_fullsized":"2024-06-10T01:04:35.567988+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:00:49.777130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559919+0000","last_change":"2024-06-10T01:01:47.253463+0000","last_active":"2024-06-10T01:04:35.559919+0000","last_peered":"2024-06-10T01:04:35.559919+0000","last_clean":"2024-06-10T01:04:35.559919+0000","last_became_active":"2024-06-10T01:01:47.252929+0000","last_became_peered":"2024-06-10T01:01:47.252929+0000","last_unstale":"2024-06-10T01:04:35.559919+0000","last_undegraded":"2024-06-10T01:04:35.559919+0000","last_fullsized":"2024-06-10T01:04:35.559919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:18:49.551276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239453+0000","last_change":"2024-06-10T01:01:57.279561+0000","last_active":"2024-06-10T01:03:38.239453+0000","last_peered":"2024-06-10T01:03:38.239453+0000","last_clean":"2024-06-10T01:03:38.239453+0000","last_became_active":"2024-06-10T01:01:57.279132+0000","last_became_peered":"2024-06-10T01:01:57.279132+0000","last_unstale":"2024-06-10T01:03:38.239453+0000","last_undegraded":"2024-06-10T01:03:38.239453+0000","last_fullsized":"2024-06-10T01:03:38.239453+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:55:03.497701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'9","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.772124+0000","last_change":"2024-06-10T01:01:44.006667+0000","last_active":"2024-06-10T01:03:38.772124+0000","last_peered":"2024-06-10T01:03:38.772124+0000","last_clean":"2024-06-10T01:03:38.772124+0000","last_became_active":"2024-06-10T01:01:44.006343+0000","last_became_peered":"2024-06-10T01:01:44.006343+0000","last_unstale":"2024-06-10T01:03:38.772124+0000","last_undegraded":"2024-06-10T01:03:38.772124+0000","last_fullsized":"2024-06-10T01:03:38.772124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:14:50.540084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562836+0000","last_change":"2024-06-10T01:01:47.246139+0000","last_active":"2024-06-10T01:04:35.562836+0000","last_peered":"2024-06-10T01:04:35.562836+0000","last_clean":"2024-06-10T01:04:35.562836+0000","last_became_active":"2024-06-10T01:01:47.245743+0000","last_became_peered":"2024-06-10T01:01:47.245743+0000","last_unstale":"2024-06-10T01:04:35.562836+0000","last_undegraded":"2024-06-10T01:04:35.562836+0000","last_fullsized":"2024-06-10T01:04:35.562836+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:06:39.041569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239398+0000","last_change":"2024-06-10T01:01:57.282843+0000","last_active":"2024-06-10T01:03:38.239398+0000","last_peered":"2024-06-10T01:03:38.239398+0000","last_clean":"2024-06-10T01:03:38.239398+0000","last_became_active":"2024-06-10T01:01:57.281797+0000","last_became_peered":"2024-06-10T01:01:57.281797+0000","last_unstale":"2024-06-10T01:03:38.239398+0000","last_undegraded":"2024-06-10T01:03:38.239398+0000","last_fullsized":"2024-06-10T01:03:38.239398+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:00.209628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239550+0000","last_change":"2024-06-10T01:01:44.466714+0000","last_active":"2024-06-10T01:03:38.239550+0000","last_peered":"2024-06-10T01:03:38.239550+0000","last_clean":"2024-06-10T01:03:38.239550+0000","last_became_active":"2024-06-10T01:01:44.466296+0000","last_became_peered":"2024-06-10T01:01:44.466296+0000","last_unstale":"2024-06-10T01:03:38.239550+0000","last_undegraded":"2024-06-10T01:03:38.239550+0000","last_fullsized":"2024-06-10T01:03:38.239550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:16:36.082308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.569808+0000","last_change":"2024-06-10T01:01:47.244173+0000","last_active":"2024-06-10T01:04:35.569808+0000","last_peered":"2024-06-10T01:04:35.569808+0000","last_clean":"2024-06-10T01:04:35.569808+0000","last_became_active":"2024-06-10T01:01:47.243032+0000","last_became_peered":"2024-06-10T01:01:47.243032+0000","last_unstale":"2024-06-10T01:04:35.569808+0000","last_undegraded":"2024-06-10T01:04:35.569808+0000","last_fullsized":"2024-06-10T01:04:35.569808+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:14:33.965903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239057+0000","last_change":"2024-06-10T01:02:37.592535+0000","last_active":"2024-06-10T01:03:38.239057+0000","last_peered":"2024-06-10T01:03:38.239057+0000","last_clean":"2024-06-10T01:03:38.239057+0000","last_became_active":"2024-06-10T01:02:37.592159+0000","last_became_peered":"2024-06-10T01:02:37.592159+0000","last_unstale":"2024-06-10T01:03:38.239057+0000","last_undegraded":"2024-06-10T01:03:38.239057+0000","last_fullsized":"2024-06-10T01:03:38.239057+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:50:08.626485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235433+0000","last_change":"2024-06-10T01:01:44.009105+0000","last_active":"2024-06-10T01:03:38.235433+0000","last_peered":"2024-06-10T01:03:38.235433+0000","last_clean":"2024-06-10T01:03:38.235433+0000","last_became_active":"2024-06-10T01:01:44.008743+0000","last_became_peered":"2024-06-10T01:01:44.008743+0000","last_unstale":"2024-06-10T01:03:38.235433+0000","last_undegraded":"2024-06-10T01:03:38.235433+0000","last_fullsized":"2024-06-10T01:03:38.235433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:44:54.301378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.570239+0000","last_change":"2024-06-10T01:01:47.249007+0000","last_active":"2024-06-10T01:04:35.570239+0000","last_peered":"2024-06-10T01:04:35.570239+0000","last_clean":"2024-06-10T01:04:35.570239+0000","last_became_active":"2024-06-10T01:01:47.245632+0000","last_became_peered":"2024-06-10T01:01:47.245632+0000","last_unstale":"2024-06-10T01:04:35.570239+0000","last_undegraded":"2024-06-10T01:04:35.570239+0000","last_fullsized":"2024-06-10T01:04:35.570239+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:29:18.916390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.238983+0000","last_change":"2024-06-10T01:01:57.283137+0000","last_active":"2024-06-10T01:03:38.238983+0000","last_peered":"2024-06-10T01:03:38.238983+0000","last_clean":"2024-06-10T01:03:38.238983+0000","last_became_active":"2024-06-10T01:01:57.282605+0000","last_became_peered":"2024-06-10T01:01:57.282605+0000","last_unstale":"2024-06-10T01:03:38.238983+0000","last_undegraded":"2024-06-10T01:03:38.238983+0000","last_fullsized":"2024-06-10T01:03:38.238983+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:26:16.967320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235734+0000","last_change":"2024-06-10T01:01:44.010674+0000","last_active":"2024-06-10T01:03:38.235734+0000","last_peered":"2024-06-10T01:03:38.235734+0000","last_clean":"2024-06-10T01:03:38.235734+0000","last_became_active":"2024-06-10T01:01:44.010291+0000","last_became_peered":"2024-06-10T01:01:44.010291+0000","last_unstale":"2024-06-10T01:03:38.235734+0000","last_undegraded":"2024-06-10T01:03:38.235734+0000","last_fullsized":"2024-06-10T01:03:38.235734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:36:18.418362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":60,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.566078+0000","last_change":"2024-06-10T01:02:37.794580+0000","last_active":"2024-06-10T01:04:35.566078+0000","last_peered":"2024-06-10T01:04:35.566078+0000","last_clean":"2024-06-10T01:04:35.566078+0000","last_became_active":"2024-06-10T01:02:37.794127+0000","last_became_peered":"2024-06-10T01:02:37.794127+0000","last_unstale":"2024-06-10T01:04:35.566078+0000","last_undegraded":"2024-06-10T01:04:35.566078+0000","last_fullsized":"2024-06-10T01:04:35.566078+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:02.117871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239746+0000","last_change":"2024-06-10T01:01:57.282494+0000","last_active":"2024-06-10T01:03:38.239746+0000","last_peered":"2024-06-10T01:03:38.239746+0000","last_clean":"2024-06-10T01:03:38.239746+0000","last_became_active":"2024-06-10T01:01:57.282077+0000","last_became_peered":"2024-06-10T01:01:57.282077+0000","last_unstale":"2024-06-10T01:03:38.239746+0000","last_undegraded":"2024-06-10T01:03:38.239746+0000","last_fullsized":"2024-06-10T01:03:38.239746+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:57:41.639788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"97'5","reported_seq":219,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:40.951609+0000","last_change":"2024-06-10T01:01:44.011134+0000","last_active":"2024-06-10T01:05:40.951609+0000","last_peered":"2024-06-10T01:05:40.951609+0000","last_clean":"2024-06-10T01:05:40.951609+0000","last_became_active":"2024-06-10T01:01:44.010788+0000","last_became_peered":"2024-06-10T01:01:44.010788+0000","last_unstale":"2024-06-10T01:05:40.951609+0000","last_undegraded":"2024-06-10T01:05:40.951609+0000","last_fullsized":"2024-06-10T01:05:40.951609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:22:55.228946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":169,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.564421+0000","last_change":"2024-06-10T01:01:47.248093+0000","last_active":"2024-06-10T01:04:35.564421+0000","last_peered":"2024-06-10T01:04:35.564421+0000","last_clean":"2024-06-10T01:04:35.564421+0000","last_became_active":"2024-06-10T01:01:47.247797+0000","last_became_peered":"2024-06-10T01:01:47.247797+0000","last_unstale":"2024-06-10T01:04:35.564421+0000","last_undegraded":"2024-06-10T01:04:35.564421+0000","last_fullsized":"2024-06-10T01:04:35.564421+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:45:50.159128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239671+0000","last_change":"2024-06-10T01:02:37.377039+0000","last_active":"2024-06-10T01:03:38.239671+0000","last_peered":"2024-06-10T01:03:38.239671+0000","last_clean":"2024-06-10T01:03:38.239671+0000","last_became_active":"2024-06-10T01:02:37.376332+0000","last_became_peered":"2024-06-10T01:02:37.376332+0000","last_unstale":"2024-06-10T01:03:38.239671+0000","last_undegraded":"2024-06-10T01:03:38.239671+0000","last_fullsized":"2024-06-10T01:03:38.239671+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:15.000505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,10,5],"acting":[2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.570465+0000","last_change":"2024-06-10T01:01:44.008342+0000","last_active":"2024-06-10T01:03:38.570465+0000","last_peered":"2024-06-10T01:03:38.570465+0000","last_clean":"2024-06-10T01:03:38.570465+0000","last_became_active":"2024-06-10T01:01:44.007959+0000","last_became_peered":"2024-06-10T01:01:44.007959+0000","last_unstale":"2024-06-10T01:03:38.570465+0000","last_undegraded":"2024-06-10T01:03:38.570465+0000","last_fullsized":"2024-06-10T01:03:38.570465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:36:03.143609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.565044+0000","last_change":"2024-06-10T01:01:47.247854+0000","last_active":"2024-06-10T01:04:35.565044+0000","last_peered":"2024-06-10T01:04:35.565044+0000","last_clean":"2024-06-10T01:04:35.565044+0000","last_became_active":"2024-06-10T01:01:47.247563+0000","last_became_peered":"2024-06-10T01:01:47.247563+0000","last_unstale":"2024-06-10T01:04:35.565044+0000","last_undegraded":"2024-06-10T01:04:35.565044+0000","last_fullsized":"2024-06-10T01:04:35.565044+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:32:13.024815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.237041+0000","last_change":"2024-06-10T01:01:57.282427+0000","last_active":"2024-06-10T01:03:38.237041+0000","last_peered":"2024-06-10T01:03:38.237041+0000","last_clean":"2024-06-10T01:03:38.237041+0000","last_became_active":"2024-06-10T01:01:57.281987+0000","last_became_peered":"2024-06-10T01:01:57.281987+0000","last_unstale":"2024-06-10T01:03:38.237041+0000","last_undegraded":"2024-06-10T01:03:38.237041+0000","last_fullsized":"2024-06-10T01:03:38.237041+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:22:31.736804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770902+0000","last_change":"2024-06-10T01:01:44.006567+0000","last_active":"2024-06-10T01:03:38.770902+0000","last_peered":"2024-06-10T01:03:38.770902+0000","last_clean":"2024-06-10T01:03:38.770902+0000","last_became_active":"2024-06-10T01:01:44.006216+0000","last_became_peered":"2024-06-10T01:01:44.006216+0000","last_unstale":"2024-06-10T01:03:38.770902+0000","last_undegraded":"2024-06-10T01:03:38.770902+0000","last_fullsized":"2024-06-10T01:03:38.770902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:33:55.696873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558008+0000","last_change":"2024-06-10T01:01:47.245326+0000","last_active":"2024-06-10T01:04:35.558008+0000","last_peered":"2024-06-10T01:04:35.558008+0000","last_clean":"2024-06-10T01:04:35.558008+0000","last_became_active":"2024-06-10T01:01:47.245013+0000","last_became_peered":"2024-06-10T01:01:47.245013+0000","last_unstale":"2024-06-10T01:04:35.558008+0000","last_undegraded":"2024-06-10T01:04:35.558008+0000","last_fullsized":"2024-06-10T01:04:35.558008+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:17:56.416627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234530+0000","last_change":"2024-06-10T01:01:57.288997+0000","last_active":"2024-06-10T01:03:38.234530+0000","last_peered":"2024-06-10T01:03:38.234530+0000","last_clean":"2024-06-10T01:03:38.234530+0000","last_became_active":"2024-06-10T01:01:57.288596+0000","last_became_peered":"2024-06-10T01:01:57.288596+0000","last_unstale":"2024-06-10T01:03:38.234530+0000","last_undegraded":"2024-06-10T01:03:38.234530+0000","last_fullsized":"2024-06-10T01:03:38.234530+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:08:14.739737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.269738+0000","last_change":"2024-06-10T01:01:44.010079+0000","last_active":"2024-06-10T01:03:38.269738+0000","last_peered":"2024-06-10T01:03:38.269738+0000","last_clean":"2024-06-10T01:03:38.269738+0000","last_became_active":"2024-06-10T01:01:44.009752+0000","last_became_peered":"2024-06-10T01:01:44.009752+0000","last_unstale":"2024-06-10T01:03:38.269738+0000","last_undegraded":"2024-06-10T01:03:38.269738+0000","last_fullsized":"2024-06-10T01:03:38.269738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:29:35.660935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.554398+0000","last_change":"2024-06-10T01:01:47.245015+0000","last_active":"2024-06-10T01:04:35.554398+0000","last_peered":"2024-06-10T01:04:35.554398+0000","last_clean":"2024-06-10T01:04:35.554398+0000","last_became_active":"2024-06-10T01:01:47.244595+0000","last_became_peered":"2024-06-10T01:01:47.244595+0000","last_unstale":"2024-06-10T01:04:35.554398+0000","last_undegraded":"2024-06-10T01:04:35.554398+0000","last_fullsized":"2024-06-10T01:04:35.554398+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:01:48.446526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.238926+0000","last_change":"2024-06-10T01:01:57.281656+0000","last_active":"2024-06-10T01:03:38.238926+0000","last_peered":"2024-06-10T01:03:38.238926+0000","last_clean":"2024-06-10T01:03:38.238926+0000","last_became_active":"2024-06-10T01:01:57.281295+0000","last_became_peered":"2024-06-10T01:01:57.281295+0000","last_unstale":"2024-06-10T01:03:38.238926+0000","last_undegraded":"2024-06-10T01:03:38.238926+0000","last_fullsized":"2024-06-10T01:03:38.238926+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:04:41.619864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"97'3","reported_seq":214,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:39.949599+0000","last_change":"2024-06-10T01:01:44.011941+0000","last_active":"2024-06-10T01:05:39.949599+0000","last_peered":"2024-06-10T01:05:39.949599+0000","last_clean":"2024-06-10T01:05:39.949599+0000","last_became_active":"2024-06-10T01:01:44.010156+0000","last_became_peered":"2024-06-10T01:01:44.010156+0000","last_unstale":"2024-06-10T01:05:39.949599+0000","last_undegraded":"2024-06-10T01:05:39.949599+0000","last_fullsized":"2024-06-10T01:05:39.949599+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:16:15.709544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":166,"num_read_kb":166,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.570257+0000","last_change":"2024-06-10T01:01:47.241720+0000","last_active":"2024-06-10T01:04:35.570257+0000","last_peered":"2024-06-10T01:04:35.570257+0000","last_clean":"2024-06-10T01:04:35.570257+0000","last_became_active":"2024-06-10T01:01:47.241386+0000","last_became_peered":"2024-06-10T01:01:47.241386+0000","last_unstale":"2024-06-10T01:04:35.570257+0000","last_undegraded":"2024-06-10T01:04:35.570257+0000","last_fullsized":"2024-06-10T01:04:35.570257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:42:01.304787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770709+0000","last_change":"2024-06-10T01:01:57.283226+0000","last_active":"2024-06-10T01:03:38.770709+0000","last_peered":"2024-06-10T01:03:38.770709+0000","last_clean":"2024-06-10T01:03:38.770709+0000","last_became_active":"2024-06-10T01:01:57.282720+0000","last_became_peered":"2024-06-10T01:01:57.282720+0000","last_unstale":"2024-06-10T01:03:38.770709+0000","last_undegraded":"2024-06-10T01:03:38.770709+0000","last_fullsized":"2024-06-10T01:03:38.770709+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:35:31.342042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235489+0000","last_change":"2024-06-10T01:01:44.009278+0000","last_active":"2024-06-10T01:03:38.235489+0000","last_peered":"2024-06-10T01:03:38.235489+0000","last_clean":"2024-06-10T01:03:38.235489+0000","last_became_active":"2024-06-10T01:01:44.008841+0000","last_became_peered":"2024-06-10T01:01:44.008841+0000","last_unstale":"2024-06-10T01:03:38.235489+0000","last_undegraded":"2024-06-10T01:03:38.235489+0000","last_fullsized":"2024-06-10T01:03:38.235489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:30:58.392080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.557141+0000","last_change":"2024-06-10T01:01:47.256451+0000","last_active":"2024-06-10T01:04:35.557141+0000","last_peered":"2024-06-10T01:04:35.557141+0000","last_clean":"2024-06-10T01:04:35.557141+0000","last_became_active":"2024-06-10T01:01:47.256036+0000","last_became_peered":"2024-06-10T01:01:47.256036+0000","last_unstale":"2024-06-10T01:04:35.557141+0000","last_undegraded":"2024-06-10T01:04:35.557141+0000","last_fullsized":"2024-06-10T01:04:35.557141+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:57:26.640109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239192+0000","last_change":"2024-06-10T01:02:37.794160+0000","last_active":"2024-06-10T01:03:38.239192+0000","last_peered":"2024-06-10T01:03:38.239192+0000","last_clean":"2024-06-10T01:03:38.239192+0000","last_became_active":"2024-06-10T01:02:37.793633+0000","last_became_peered":"2024-06-10T01:02:37.793633+0000","last_unstale":"2024-06-10T01:03:38.239192+0000","last_undegraded":"2024-06-10T01:03:38.239192+0000","last_fullsized":"2024-06-10T01:03:38.239192+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:48:48.083620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270125+0000","last_change":"2024-06-10T01:01:44.469179+0000","last_active":"2024-06-10T01:03:38.270125+0000","last_peered":"2024-06-10T01:03:38.270125+0000","last_clean":"2024-06-10T01:03:38.270125+0000","last_became_active":"2024-06-10T01:01:44.468669+0000","last_became_peered":"2024-06-10T01:01:44.468669+0000","last_unstale":"2024-06-10T01:03:38.270125+0000","last_undegraded":"2024-06-10T01:03:38.270125+0000","last_fullsized":"2024-06-10T01:03:38.270125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:07:51.629030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.570053+0000","last_change":"2024-06-10T01:01:47.247588+0000","last_active":"2024-06-10T01:04:35.570053+0000","last_peered":"2024-06-10T01:04:35.570053+0000","last_clean":"2024-06-10T01:04:35.570053+0000","last_became_active":"2024-06-10T01:01:47.245750+0000","last_became_peered":"2024-06-10T01:01:47.245750+0000","last_unstale":"2024-06-10T01:04:35.570053+0000","last_undegraded":"2024-06-10T01:04:35.570053+0000","last_fullsized":"2024-06-10T01:04:35.570053+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:12:10.513777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771292+0000","last_change":"2024-06-10T01:01:57.283042+0000","last_active":"2024-06-10T01:03:38.771292+0000","last_peered":"2024-06-10T01:03:38.771292+0000","last_clean":"2024-06-10T01:03:38.771292+0000","last_became_active":"2024-06-10T01:01:57.282600+0000","last_became_peered":"2024-06-10T01:01:57.282600+0000","last_unstale":"2024-06-10T01:03:38.771292+0000","last_undegraded":"2024-06-10T01:03:38.771292+0000","last_fullsized":"2024-06-10T01:03:38.771292+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:06:36.969968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"99'39","reported_seq":425,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:39.950344+0000","last_change":"2024-06-10T01:01:44.011783+0000","last_active":"2024-06-10T01:05:39.950344+0000","last_peered":"2024-06-10T01:05:39.950344+0000","last_clean":"2024-06-10T01:05:39.950344+0000","last_became_active":"2024-06-10T01:01:44.011164+0000","last_became_peered":"2024-06-10T01:01:44.011164+0000","last_unstale":"2024-06-10T01:05:39.950344+0000","last_undegraded":"2024-06-10T01:05:39.950344+0000","last_fullsized":"2024-06-10T01:05:39.950344+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:16:13.642714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64615,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":341,"num_read_kb":234,"num_write":40,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562407+0000","last_change":"2024-06-10T01:01:47.250751+0000","last_active":"2024-06-10T01:04:35.562407+0000","last_peered":"2024-06-10T01:04:35.562407+0000","last_clean":"2024-06-10T01:04:35.562407+0000","last_became_active":"2024-06-10T01:01:47.250451+0000","last_became_peered":"2024-06-10T01:01:47.250451+0000","last_unstale":"2024-06-10T01:04:35.562407+0000","last_undegraded":"2024-06-10T01:04:35.562407+0000","last_fullsized":"2024-06-10T01:04:35.562407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:22:10.121363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270382+0000","last_change":"2024-06-10T01:01:57.289687+0000","last_active":"2024-06-10T01:03:38.270382+0000","last_peered":"2024-06-10T01:03:38.270382+0000","last_clean":"2024-06-10T01:03:38.270382+0000","last_became_active":"2024-06-10T01:01:57.289274+0000","last_became_peered":"2024-06-10T01:01:57.289274+0000","last_unstale":"2024-06-10T01:03:38.270382+0000","last_undegraded":"2024-06-10T01:03:38.270382+0000","last_fullsized":"2024-06-10T01:03:38.270382+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:34:46.873974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'6","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239529+0000","last_change":"2024-06-10T01:01:44.010159+0000","last_active":"2024-06-10T01:03:38.239529+0000","last_peered":"2024-06-10T01:03:38.239529+0000","last_clean":"2024-06-10T01:03:38.239529+0000","last_became_active":"2024-06-10T01:01:44.009744+0000","last_became_peered":"2024-06-10T01:01:44.009744+0000","last_unstale":"2024-06-10T01:03:38.239529+0000","last_undegraded":"2024-06-10T01:03:38.239529+0000","last_fullsized":"2024-06-10T01:03:38.239529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:30.507613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558994+0000","last_change":"2024-06-10T01:01:47.246662+0000","last_active":"2024-06-10T01:04:35.558994+0000","last_peered":"2024-06-10T01:04:35.558994+0000","last_clean":"2024-06-10T01:04:35.558994+0000","last_became_active":"2024-06-10T01:01:47.246268+0000","last_became_peered":"2024-06-10T01:01:47.246268+0000","last_unstale":"2024-06-10T01:04:35.558994+0000","last_undegraded":"2024-06-10T01:04:35.558994+0000","last_fullsized":"2024-06-10T01:04:35.558994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:08:38.089054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.236925+0000","last_change":"2024-06-10T01:01:57.289937+0000","last_active":"2024-06-10T01:03:38.236925+0000","last_peered":"2024-06-10T01:03:38.236925+0000","last_clean":"2024-06-10T01:03:38.236925+0000","last_became_active":"2024-06-10T01:01:57.289407+0000","last_became_peered":"2024-06-10T01:01:57.289407+0000","last_unstale":"2024-06-10T01:03:38.236925+0000","last_undegraded":"2024-06-10T01:03:38.236925+0000","last_fullsized":"2024-06-10T01:03:38.236925+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:01:48.546762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270079+0000","last_change":"2024-06-10T01:01:44.012654+0000","last_active":"2024-06-10T01:03:38.270079+0000","last_peered":"2024-06-10T01:03:38.270079+0000","last_clean":"2024-06-10T01:03:38.270079+0000","last_became_active":"2024-06-10T01:01:44.012246+0000","last_became_peered":"2024-06-10T01:01:44.012246+0000","last_unstale":"2024-06-10T01:03:38.270079+0000","last_undegraded":"2024-06-10T01:03:38.270079+0000","last_fullsized":"2024-06-10T01:03:38.270079+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:31:36.173554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558783+0000","last_change":"2024-06-10T01:01:47.250514+0000","last_active":"2024-06-10T01:04:35.558783+0000","last_peered":"2024-06-10T01:04:35.558783+0000","last_clean":"2024-06-10T01:04:35.558783+0000","last_became_active":"2024-06-10T01:01:47.249050+0000","last_became_peered":"2024-06-10T01:01:47.249050+0000","last_unstale":"2024-06-10T01:04:35.558783+0000","last_undegraded":"2024-06-10T01:04:35.558783+0000","last_fullsized":"2024-06-10T01:04:35.558783+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:51:30.789392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771928+0000","last_change":"2024-06-10T01:01:57.282389+0000","last_active":"2024-06-10T01:03:38.771928+0000","last_peered":"2024-06-10T01:03:38.771928+0000","last_clean":"2024-06-10T01:03:38.771928+0000","last_became_active":"2024-06-10T01:01:57.281857+0000","last_became_peered":"2024-06-10T01:01:57.281857+0000","last_unstale":"2024-06-10T01:03:38.771928+0000","last_undegraded":"2024-06-10T01:03:38.771928+0000","last_fullsized":"2024-06-10T01:03:38.771928+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:59:14.863020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"99'4","reported_seq":214,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:37.947518+0000","last_change":"2024-06-10T01:01:44.011331+0000","last_active":"2024-06-10T01:05:37.947518+0000","last_peered":"2024-06-10T01:05:37.947518+0000","last_clean":"2024-06-10T01:05:37.947518+0000","last_became_active":"2024-06-10T01:01:44.010968+0000","last_became_peered":"2024-06-10T01:01:44.010968+0000","last_unstale":"2024-06-10T01:05:37.947518+0000","last_undegraded":"2024-06-10T01:05:37.947518+0000","last_fullsized":"2024-06-10T01:05:37.947518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:43:37.931225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":896,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":166,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562879+0000","last_change":"2024-06-10T01:01:47.248349+0000","last_active":"2024-06-10T01:04:35.562879+0000","last_peered":"2024-06-10T01:04:35.562879+0000","last_clean":"2024-06-10T01:04:35.562879+0000","last_became_active":"2024-06-10T01:01:47.248008+0000","last_became_peered":"2024-06-10T01:01:47.248008+0000","last_unstale":"2024-06-10T01:04:35.562879+0000","last_undegraded":"2024-06-10T01:04:35.562879+0000","last_fullsized":"2024-06-10T01:04:35.562879+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:45:20.747704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771221+0000","last_change":"2024-06-10T01:01:57.289773+0000","last_active":"2024-06-10T01:03:38.771221+0000","last_peered":"2024-06-10T01:03:38.771221+0000","last_clean":"2024-06-10T01:03:38.771221+0000","last_became_active":"2024-06-10T01:01:57.289371+0000","last_became_peered":"2024-06-10T01:01:57.289371+0000","last_unstale":"2024-06-10T01:03:38.771221+0000","last_undegraded":"2024-06-10T01:03:38.771221+0000","last_fullsized":"2024-06-10T01:03:38.771221+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:24:52.209402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'15","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.257548+0000","last_change":"2024-06-10T01:03:38.257122+0000","last_active":"2024-06-10T01:03:38.257548+0000","last_peered":"2024-06-10T01:03:38.257548+0000","last_clean":"2024-06-10T01:03:38.257548+0000","last_became_active":"2024-06-10T01:03:38.240737+0000","last_became_peered":"2024-06-10T01:03:38.240737+0000","last_unstale":"2024-06-10T01:03:38.257548+0000","last_undegraded":"2024-06-10T01:03:38.257548+0000","last_fullsized":"2024-06-10T01:03:38.257548+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:37:10.925004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562425+0000","last_change":"2024-06-10T01:01:47.245439+0000","last_active":"2024-06-10T01:04:35.562425+0000","last_peered":"2024-06-10T01:04:35.562425+0000","last_clean":"2024-06-10T01:04:35.562425+0000","last_became_active":"2024-06-10T01:01:47.242917+0000","last_became_peered":"2024-06-10T01:01:47.242917+0000","last_unstale":"2024-06-10T01:04:35.562425+0000","last_undegraded":"2024-06-10T01:04:35.562425+0000","last_fullsized":"2024-06-10T01:04:35.562425+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:13:12.614230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.269795+0000","last_change":"2024-06-10T01:01:57.284731+0000","last_active":"2024-06-10T01:03:38.269795+0000","last_peered":"2024-06-10T01:03:38.269795+0000","last_clean":"2024-06-10T01:03:38.269795+0000","last_became_active":"2024-06-10T01:01:57.284301+0000","last_became_peered":"2024-06-10T01:01:57.284301+0000","last_unstale":"2024-06-10T01:03:38.269795+0000","last_undegraded":"2024-06-10T01:03:38.269795+0000","last_fullsized":"2024-06-10T01:03:38.269795+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:01:36.518214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"99'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:59.314831+0000","last_change":"2024-06-10T01:01:44.010154+0000","last_active":"2024-06-10T01:03:59.314831+0000","last_peered":"2024-06-10T01:03:59.314831+0000","last_clean":"2024-06-10T01:03:59.314831+0000","last_became_active":"2024-06-10T01:01:44.009656+0000","last_became_peered":"2024-06-10T01:01:44.009656+0000","last_unstale":"2024-06-10T01:03:59.314831+0000","last_undegraded":"2024-06-10T01:03:59.314831+0000","last_fullsized":"2024-06-10T01:03:59.314831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:33:08.789946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.550140+0000","last_change":"2024-06-10T01:02:37.378484+0000","last_active":"2024-06-10T01:04:35.550140+0000","last_peered":"2024-06-10T01:04:35.550140+0000","last_clean":"2024-06-10T01:04:35.550140+0000","last_became_active":"2024-06-10T01:02:37.378092+0000","last_became_peered":"2024-06-10T01:02:37.378092+0000","last_unstale":"2024-06-10T01:04:35.550140+0000","last_undegraded":"2024-06-10T01:04:35.550140+0000","last_fullsized":"2024-06-10T01:04:35.550140+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:19:46.599235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235375+0000","last_change":"2024-06-10T01:01:57.332887+0000","last_active":"2024-06-10T01:03:38.235375+0000","last_peered":"2024-06-10T01:03:38.235375+0000","last_clean":"2024-06-10T01:03:38.235375+0000","last_became_active":"2024-06-10T01:01:57.332285+0000","last_became_peered":"2024-06-10T01:01:57.332285+0000","last_unstale":"2024-06-10T01:03:38.235375+0000","last_undegraded":"2024-06-10T01:03:38.235375+0000","last_fullsized":"2024-06-10T01:03:38.235375+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:09:12.233405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771642+0000","last_change":"2024-06-10T01:01:44.466831+0000","last_active":"2024-06-10T01:03:38.771642+0000","last_peered":"2024-06-10T01:03:38.771642+0000","last_clean":"2024-06-10T01:03:38.771642+0000","last_became_active":"2024-06-10T01:01:44.466484+0000","last_became_peered":"2024-06-10T01:01:44.466484+0000","last_unstale":"2024-06-10T01:03:38.771642+0000","last_undegraded":"2024-06-10T01:03:38.771642+0000","last_fullsized":"2024-06-10T01:03:38.771642+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:19:28.385504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.556135+0000","last_change":"2024-06-10T01:01:47.252399+0000","last_active":"2024-06-10T01:04:35.556135+0000","last_peered":"2024-06-10T01:04:35.556135+0000","last_clean":"2024-06-10T01:04:35.556135+0000","last_became_active":"2024-06-10T01:01:47.252069+0000","last_became_peered":"2024-06-10T01:01:47.252069+0000","last_unstale":"2024-06-10T01:04:35.556135+0000","last_undegraded":"2024-06-10T01:04:35.556135+0000","last_fullsized":"2024-06-10T01:04:35.556135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:02:02.291174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771406+0000","last_change":"2024-06-10T01:01:57.285670+0000","last_active":"2024-06-10T01:03:38.771406+0000","last_peered":"2024-06-10T01:03:38.771406+0000","last_clean":"2024-06-10T01:03:38.771406+0000","last_became_active":"2024-06-10T01:01:57.285281+0000","last_became_peered":"2024-06-10T01:01:57.285281+0000","last_unstale":"2024-06-10T01:03:38.771406+0000","last_undegraded":"2024-06-10T01:03:38.771406+0000","last_fullsized":"2024-06-10T01:03:38.771406+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:58.956934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235189+0000","last_change":"2024-06-10T01:01:44.008013+0000","last_active":"2024-06-10T01:03:38.235189+0000","last_peered":"2024-06-10T01:03:38.235189+0000","last_clean":"2024-06-10T01:03:38.235189+0000","last_became_active":"2024-06-10T01:01:44.007591+0000","last_became_peered":"2024-06-10T01:01:44.007591+0000","last_unstale":"2024-06-10T01:03:38.235189+0000","last_undegraded":"2024-06-10T01:03:38.235189+0000","last_fullsized":"2024-06-10T01:03:38.235189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:18:20.862340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.550926+0000","last_change":"2024-06-10T01:01:47.254756+0000","last_active":"2024-06-10T01:04:35.550926+0000","last_peered":"2024-06-10T01:04:35.550926+0000","last_clean":"2024-06-10T01:04:35.550926+0000","last_became_active":"2024-06-10T01:01:47.254418+0000","last_became_peered":"2024-06-10T01:01:47.254418+0000","last_unstale":"2024-06-10T01:04:35.550926+0000","last_undegraded":"2024-06-10T01:04:35.550926+0000","last_fullsized":"2024-06-10T01:04:35.550926+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:46:31.452539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770995+0000","last_change":"2024-06-10T01:01:57.332792+0000","last_active":"2024-06-10T01:03:38.770995+0000","last_peered":"2024-06-10T01:03:38.770995+0000","last_clean":"2024-06-10T01:03:38.770995+0000","last_became_active":"2024-06-10T01:01:57.332331+0000","last_became_peered":"2024-06-10T01:01:57.332331+0000","last_unstale":"2024-06-10T01:03:38.770995+0000","last_undegraded":"2024-06-10T01:03:38.770995+0000","last_fullsized":"2024-06-10T01:03:38.770995+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:46:41.419807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.565658+0000","last_change":"2024-06-10T01:01:47.247857+0000","last_active":"2024-06-10T01:04:35.565658+0000","last_peered":"2024-06-10T01:04:35.565658+0000","last_clean":"2024-06-10T01:04:35.565658+0000","last_became_active":"2024-06-10T01:01:47.247552+0000","last_became_peered":"2024-06-10T01:01:47.247552+0000","last_unstale":"2024-06-10T01:04:35.565658+0000","last_undegraded":"2024-06-10T01:04:35.565658+0000","last_fullsized":"2024-06-10T01:04:35.565658+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:29:40.521887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"99'5","reported_seq":214,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:38.440204+0000","last_change":"2024-06-10T01:01:44.466631+0000","last_active":"2024-06-10T01:05:38.440204+0000","last_peered":"2024-06-10T01:05:38.440204+0000","last_clean":"2024-06-10T01:05:38.440204+0000","last_became_active":"2024-06-10T01:01:44.466280+0000","last_became_peered":"2024-06-10T01:01:44.466280+0000","last_unstale":"2024-06-10T01:05:38.440204+0000","last_undegraded":"2024-06-10T01:05:38.440204+0000","last_fullsized":"2024-06-10T01:05:38.440204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:09:33.658112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":164,"num_read_kb":164,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235365+0000","last_change":"2024-06-10T01:01:57.281973+0000","last_active":"2024-06-10T01:03:38.235365+0000","last_peered":"2024-06-10T01:03:38.235365+0000","last_clean":"2024-06-10T01:03:38.235365+0000","last_became_active":"2024-06-10T01:01:57.281386+0000","last_became_peered":"2024-06-10T01:01:57.281386+0000","last_unstale":"2024-06-10T01:03:38.235365+0000","last_undegraded":"2024-06-10T01:03:38.235365+0000","last_fullsized":"2024-06-10T01:03:38.235365+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:10:28.236250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562415+0000","last_change":"2024-06-10T01:01:47.237436+0000","last_active":"2024-06-10T01:04:35.562415+0000","last_peered":"2024-06-10T01:04:35.562415+0000","last_clean":"2024-06-10T01:04:35.562415+0000","last_became_active":"2024-06-10T01:01:47.237099+0000","last_became_peered":"2024-06-10T01:01:47.237099+0000","last_unstale":"2024-06-10T01:04:35.562415+0000","last_undegraded":"2024-06-10T01:04:35.562415+0000","last_fullsized":"2024-06-10T01:04:35.562415+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:34:53.838877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'3","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239178+0000","last_change":"2024-06-10T01:01:44.010655+0000","last_active":"2024-06-10T01:03:38.239178+0000","last_peered":"2024-06-10T01:03:38.239178+0000","last_clean":"2024-06-10T01:03:38.239178+0000","last_became_active":"2024-06-10T01:01:44.009918+0000","last_became_peered":"2024-06-10T01:01:44.009918+0000","last_unstale":"2024-06-10T01:03:38.239178+0000","last_undegraded":"2024-06-10T01:03:38.239178+0000","last_fullsized":"2024-06-10T01:03:38.239178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:47:32.052650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":367,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234906+0000","last_change":"2024-06-10T01:00:23.822447+0000","last_active":"2024-06-10T01:03:38.234906+0000","last_peered":"2024-06-10T01:03:38.234906+0000","last_clean":"2024-06-10T01:03:38.234906+0000","last_became_active":"2024-06-10T01:00:23.821966+0000","last_became_peered":"2024-06-10T01:00:23.821966+0000","last_unstale":"2024-06-10T01:03:38.234906+0000","last_undegraded":"2024-06-10T01:03:38.234906+0000","last_fullsized":"2024-06-10T01:03:38.234906+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:56:59.964661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:56:59.964661+0000","last_clean_scrub_stamp":"2024-06-10T00:56:59.964661+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:03:43.732875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270272+0000","last_change":"2024-06-10T01:01:57.332980+0000","last_active":"2024-06-10T01:03:38.270272+0000","last_peered":"2024-06-10T01:03:38.270272+0000","last_clean":"2024-06-10T01:03:38.270272+0000","last_became_active":"2024-06-10T01:01:57.332437+0000","last_became_peered":"2024-06-10T01:01:57.332437+0000","last_unstale":"2024-06-10T01:03:38.270272+0000","last_undegraded":"2024-06-10T01:03:38.270272+0000","last_fullsized":"2024-06-10T01:03:38.270272+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:27:35.029079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559754+0000","last_change":"2024-06-10T01:01:47.255926+0000","last_active":"2024-06-10T01:04:35.559754+0000","last_peered":"2024-06-10T01:04:35.559754+0000","last_clean":"2024-06-10T01:04:35.559754+0000","last_became_active":"2024-06-10T01:01:47.255571+0000","last_became_peered":"2024-06-10T01:01:47.255571+0000","last_unstale":"2024-06-10T01:04:35.559754+0000","last_undegraded":"2024-06-10T01:04:35.559754+0000","last_fullsized":"2024-06-10T01:04:35.559754+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:05:22.084037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234997+0000","last_change":"2024-06-10T01:01:44.007509+0000","last_active":"2024-06-10T01:03:38.234997+0000","last_peered":"2024-06-10T01:03:38.234997+0000","last_clean":"2024-06-10T01:03:38.234997+0000","last_became_active":"2024-06-10T01:01:44.007178+0000","last_became_peered":"2024-06-10T01:01:44.007178+0000","last_unstale":"2024-06-10T01:03:38.234997+0000","last_undegraded":"2024-06-10T01:03:38.234997+0000","last_fullsized":"2024-06-10T01:03:38.234997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:02:52.791886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.236790+0000","last_change":"2024-06-10T01:01:57.285626+0000","last_active":"2024-06-10T01:03:38.236790+0000","last_peered":"2024-06-10T01:03:38.236790+0000","last_clean":"2024-06-10T01:03:38.236790+0000","last_became_active":"2024-06-10T01:01:57.285098+0000","last_became_peered":"2024-06-10T01:01:57.285098+0000","last_unstale":"2024-06-10T01:03:38.236790+0000","last_undegraded":"2024-06-10T01:03:38.236790+0000","last_fullsized":"2024-06-10T01:03:38.236790+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:13:49.881770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235332+0000","last_change":"2024-06-10T01:01:44.469028+0000","last_active":"2024-06-10T01:03:38.235332+0000","last_peered":"2024-06-10T01:03:38.235332+0000","last_clean":"2024-06-10T01:03:38.235332+0000","last_became_active":"2024-06-10T01:01:44.468570+0000","last_became_peered":"2024-06-10T01:01:44.468570+0000","last_unstale":"2024-06-10T01:03:38.235332+0000","last_undegraded":"2024-06-10T01:03:38.235332+0000","last_fullsized":"2024-06-10T01:03:38.235332+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:12:11.287959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.560160+0000","last_change":"2024-06-10T01:01:47.253414+0000","last_active":"2024-06-10T01:04:35.560160+0000","last_peered":"2024-06-10T01:04:35.560160+0000","last_clean":"2024-06-10T01:04:35.560160+0000","last_became_active":"2024-06-10T01:01:47.253060+0000","last_became_peered":"2024-06-10T01:01:47.253060+0000","last_unstale":"2024-06-10T01:04:35.560160+0000","last_undegraded":"2024-06-10T01:04:35.560160+0000","last_fullsized":"2024-06-10T01:04:35.560160+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:24:55.534164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.238849+0000","last_change":"2024-06-10T01:01:57.332756+0000","last_active":"2024-06-10T01:03:38.238849+0000","last_peered":"2024-06-10T01:03:38.238849+0000","last_clean":"2024-06-10T01:03:38.238849+0000","last_became_active":"2024-06-10T01:01:57.332361+0000","last_became_peered":"2024-06-10T01:01:57.332361+0000","last_unstale":"2024-06-10T01:03:38.238849+0000","last_undegraded":"2024-06-10T01:03:38.238849+0000","last_fullsized":"2024-06-10T01:03:38.238849+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:02:28.275407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"99'14","reported_seq":59,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:36.265950+0000","last_change":"2024-06-10T01:01:44.005164+0000","last_active":"2024-06-10T01:04:36.265950+0000","last_peered":"2024-06-10T01:04:36.265950+0000","last_clean":"2024-06-10T01:04:36.265950+0000","last_became_active":"2024-06-10T01:01:44.004816+0000","last_became_peered":"2024-06-10T01:01:44.004816+0000","last_unstale":"2024-06-10T01:04:36.265950+0000","last_undegraded":"2024-06-10T01:04:36.265950+0000","last_fullsized":"2024-06-10T01:04:36.265950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:11:56.385267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.557511+0000","last_change":"2024-06-10T01:01:47.247951+0000","last_active":"2024-06-10T01:04:35.557511+0000","last_peered":"2024-06-10T01:04:35.557511+0000","last_clean":"2024-06-10T01:04:35.557511+0000","last_became_active":"2024-06-10T01:01:47.246996+0000","last_became_peered":"2024-06-10T01:01:47.246996+0000","last_unstale":"2024-06-10T01:04:35.557511+0000","last_undegraded":"2024-06-10T01:04:35.557511+0000","last_fullsized":"2024-06-10T01:04:35.557511+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:39:18.167021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239228+0000","last_change":"2024-06-10T01:01:57.278130+0000","last_active":"2024-06-10T01:03:38.239228+0000","last_peered":"2024-06-10T01:03:38.239228+0000","last_clean":"2024-06-10T01:03:38.239228+0000","last_became_active":"2024-06-10T01:01:57.277697+0000","last_became_peered":"2024-06-10T01:01:57.277697+0000","last_unstale":"2024-06-10T01:03:38.239228+0000","last_undegraded":"2024-06-10T01:03:38.239228+0000","last_fullsized":"2024-06-10T01:03:38.239228+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:17:01.662152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239618+0000","last_change":"2024-06-10T01:01:44.553176+0000","last_active":"2024-06-10T01:03:38.239618+0000","last_peered":"2024-06-10T01:03:38.239618+0000","last_clean":"2024-06-10T01:03:38.239618+0000","last_became_active":"2024-06-10T01:01:44.552797+0000","last_became_peered":"2024-06-10T01:01:44.552797+0000","last_unstale":"2024-06-10T01:03:38.239618+0000","last_undegraded":"2024-06-10T01:03:38.239618+0000","last_fullsized":"2024-06-10T01:03:38.239618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:21:13.960304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.556104+0000","last_change":"2024-06-10T01:01:47.255796+0000","last_active":"2024-06-10T01:04:35.556104+0000","last_peered":"2024-06-10T01:04:35.556104+0000","last_clean":"2024-06-10T01:04:35.556104+0000","last_became_active":"2024-06-10T01:01:47.255495+0000","last_became_peered":"2024-06-10T01:01:47.255495+0000","last_unstale":"2024-06-10T01:04:35.556104+0000","last_undegraded":"2024-06-10T01:04:35.556104+0000","last_fullsized":"2024-06-10T01:04:35.556104+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:06:42.209005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239491+0000","last_change":"2024-06-10T01:01:57.332869+0000","last_active":"2024-06-10T01:03:38.239491+0000","last_peered":"2024-06-10T01:03:38.239491+0000","last_clean":"2024-06-10T01:03:38.239491+0000","last_became_active":"2024-06-10T01:01:57.332391+0000","last_became_peered":"2024-06-10T01:01:57.332391+0000","last_unstale":"2024-06-10T01:03:38.239491+0000","last_undegraded":"2024-06-10T01:03:38.239491+0000","last_fullsized":"2024-06-10T01:03:38.239491+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:58:31.443626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770788+0000","last_change":"2024-06-10T01:01:44.466861+0000","last_active":"2024-06-10T01:03:38.770788+0000","last_peered":"2024-06-10T01:03:38.770788+0000","last_clean":"2024-06-10T01:03:38.770788+0000","last_became_active":"2024-06-10T01:01:44.466553+0000","last_became_peered":"2024-06-10T01:01:44.466553+0000","last_unstale":"2024-06-10T01:03:38.770788+0000","last_undegraded":"2024-06-10T01:03:38.770788+0000","last_fullsized":"2024-06-10T01:03:38.770788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:55:52.545594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":41,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.544035+0000","last_change":"2024-06-10T01:01:47.244972+0000","last_active":"2024-06-10T01:04:35.544035+0000","last_peered":"2024-06-10T01:04:35.544035+0000","last_clean":"2024-06-10T01:04:35.544035+0000","last_became_active":"2024-06-10T01:01:47.244649+0000","last_became_peered":"2024-06-10T01:01:47.244649+0000","last_unstale":"2024-06-10T01:04:35.544035+0000","last_undegraded":"2024-06-10T01:04:35.544035+0000","last_fullsized":"2024-06-10T01:04:35.544035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:10:46.909200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235461+0000","last_change":"2024-06-10T01:02:37.592380+0000","last_active":"2024-06-10T01:03:38.235461+0000","last_peered":"2024-06-10T01:03:38.235461+0000","last_clean":"2024-06-10T01:03:38.235461+0000","last_became_active":"2024-06-10T01:02:37.591856+0000","last_became_peered":"2024-06-10T01:02:37.591856+0000","last_unstale":"2024-06-10T01:03:38.235461+0000","last_undegraded":"2024-06-10T01:03:38.235461+0000","last_fullsized":"2024-06-10T01:03:38.235461+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:00:25.717192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239006+0000","last_change":"2024-06-10T01:01:44.008116+0000","last_active":"2024-06-10T01:03:38.239006+0000","last_peered":"2024-06-10T01:03:38.239006+0000","last_clean":"2024-06-10T01:03:38.239006+0000","last_became_active":"2024-06-10T01:01:44.007778+0000","last_became_peered":"2024-06-10T01:01:44.007778+0000","last_unstale":"2024-06-10T01:03:38.239006+0000","last_undegraded":"2024-06-10T01:03:38.239006+0000","last_fullsized":"2024-06-10T01:03:38.239006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:18:57.267296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.556264+0000","last_change":"2024-06-10T01:01:47.252833+0000","last_active":"2024-06-10T01:04:35.556264+0000","last_peered":"2024-06-10T01:04:35.556264+0000","last_clean":"2024-06-10T01:04:35.556264+0000","last_became_active":"2024-06-10T01:01:47.252326+0000","last_became_peered":"2024-06-10T01:01:47.252326+0000","last_unstale":"2024-06-10T01:04:35.556264+0000","last_undegraded":"2024-06-10T01:04:35.556264+0000","last_fullsized":"2024-06-10T01:04:35.556264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:00:42.673838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239284+0000","last_change":"2024-06-10T01:01:57.275975+0000","last_active":"2024-06-10T01:03:38.239284+0000","last_peered":"2024-06-10T01:03:38.239284+0000","last_clean":"2024-06-10T01:03:38.239284+0000","last_became_active":"2024-06-10T01:01:57.275408+0000","last_became_peered":"2024-06-10T01:01:57.275408+0000","last_unstale":"2024-06-10T01:03:38.239284+0000","last_undegraded":"2024-06-10T01:03:38.239284+0000","last_fullsized":"2024-06-10T01:03:38.239284+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:12:42.376636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'1","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.243874+0000","last_change":"2024-06-10T01:03:38.243422+0000","last_active":"2024-06-10T01:03:38.243874+0000","last_peered":"2024-06-10T01:03:38.243874+0000","last_clean":"2024-06-10T01:03:38.243874+0000","last_became_active":"2024-06-10T01:03:38.239916+0000","last_became_peered":"2024-06-10T01:03:38.239916+0000","last_unstale":"2024-06-10T01:03:38.243874+0000","last_undegraded":"2024-06-10T01:03:38.243874+0000","last_fullsized":"2024-06-10T01:03:38.243874+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:30:01.399956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.555295+0000","last_change":"2024-06-10T01:01:47.243154+0000","last_active":"2024-06-10T01:04:35.555295+0000","last_peered":"2024-06-10T01:04:35.555295+0000","last_clean":"2024-06-10T01:04:35.555295+0000","last_became_active":"2024-06-10T01:01:47.242851+0000","last_became_peered":"2024-06-10T01:01:47.242851+0000","last_unstale":"2024-06-10T01:04:35.555295+0000","last_undegraded":"2024-06-10T01:04:35.555295+0000","last_fullsized":"2024-06-10T01:04:35.555295+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:41:05.857777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239584+0000","last_change":"2024-06-10T01:01:57.333613+0000","last_active":"2024-06-10T01:03:38.239584+0000","last_peered":"2024-06-10T01:03:38.239584+0000","last_clean":"2024-06-10T01:03:38.239584+0000","last_became_active":"2024-06-10T01:01:57.333019+0000","last_became_peered":"2024-06-10T01:01:57.333019+0000","last_unstale":"2024-06-10T01:03:38.239584+0000","last_undegraded":"2024-06-10T01:03:38.239584+0000","last_fullsized":"2024-06-10T01:03:38.239584+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:11.912885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235303+0000","last_change":"2024-06-10T01:01:44.008221+0000","last_active":"2024-06-10T01:03:38.235303+0000","last_peered":"2024-06-10T01:03:38.235303+0000","last_clean":"2024-06-10T01:03:38.235303+0000","last_became_active":"2024-06-10T01:01:44.007880+0000","last_became_peered":"2024-06-10T01:01:44.007880+0000","last_unstale":"2024-06-10T01:03:38.235303+0000","last_undegraded":"2024-06-10T01:03:38.235303+0000","last_fullsized":"2024-06-10T01:03:38.235303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:40:33.162925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558463+0000","last_change":"2024-06-10T01:01:47.254902+0000","last_active":"2024-06-10T01:04:35.558463+0000","last_peered":"2024-06-10T01:04:35.558463+0000","last_clean":"2024-06-10T01:04:35.558463+0000","last_became_active":"2024-06-10T01:01:47.254555+0000","last_became_peered":"2024-06-10T01:01:47.254555+0000","last_unstale":"2024-06-10T01:04:35.558463+0000","last_undegraded":"2024-06-10T01:04:35.558463+0000","last_fullsized":"2024-06-10T01:04:35.558463+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:03:17.571001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770786+0000","last_change":"2024-06-10T01:01:57.284491+0000","last_active":"2024-06-10T01:03:38.770786+0000","last_peered":"2024-06-10T01:03:38.770786+0000","last_clean":"2024-06-10T01:03:38.770786+0000","last_became_active":"2024-06-10T01:01:57.283795+0000","last_became_peered":"2024-06-10T01:01:57.283795+0000","last_unstale":"2024-06-10T01:03:38.770786+0000","last_undegraded":"2024-06-10T01:03:38.770786+0000","last_fullsized":"2024-06-10T01:03:38.770786+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:49:00.381664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239889+0000","last_change":"2024-06-10T01:01:44.553159+0000","last_active":"2024-06-10T01:03:38.239889+0000","last_peered":"2024-06-10T01:03:38.239889+0000","last_clean":"2024-06-10T01:03:38.239889+0000","last_became_active":"2024-06-10T01:01:44.552759+0000","last_became_peered":"2024-06-10T01:01:44.552759+0000","last_unstale":"2024-06-10T01:03:38.239889+0000","last_undegraded":"2024-06-10T01:03:38.239889+0000","last_fullsized":"2024-06-10T01:03:38.239889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:19:00.110790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559093+0000","last_change":"2024-06-10T01:01:47.247803+0000","last_active":"2024-06-10T01:04:35.559093+0000","last_peered":"2024-06-10T01:04:35.559093+0000","last_clean":"2024-06-10T01:04:35.559093+0000","last_became_active":"2024-06-10T01:01:47.246715+0000","last_became_peered":"2024-06-10T01:01:47.246715+0000","last_unstale":"2024-06-10T01:04:35.559093+0000","last_undegraded":"2024-06-10T01:04:35.559093+0000","last_fullsized":"2024-06-10T01:04:35.559093+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:22.429002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234192+0000","last_change":"2024-06-10T01:01:57.287668+0000","last_active":"2024-06-10T01:03:38.234192+0000","last_peered":"2024-06-10T01:03:38.234192+0000","last_clean":"2024-06-10T01:03:38.234192+0000","last_became_active":"2024-06-10T01:01:57.287060+0000","last_became_peered":"2024-06-10T01:01:57.287060+0000","last_unstale":"2024-06-10T01:03:38.234192+0000","last_undegraded":"2024-06-10T01:03:38.234192+0000","last_fullsized":"2024-06-10T01:03:38.234192+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:04:35.883613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270251+0000","last_change":"2024-06-10T01:01:44.013051+0000","last_active":"2024-06-10T01:03:38.270251+0000","last_peered":"2024-06-10T01:03:38.270251+0000","last_clean":"2024-06-10T01:03:38.270251+0000","last_became_active":"2024-06-10T01:01:44.012667+0000","last_became_peered":"2024-06-10T01:01:44.012667+0000","last_unstale":"2024-06-10T01:03:38.270251+0000","last_undegraded":"2024-06-10T01:03:38.270251+0000","last_fullsized":"2024-06-10T01:03:38.270251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:10:49.650082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.563083+0000","last_change":"2024-06-10T01:01:47.251209+0000","last_active":"2024-06-10T01:04:35.563083+0000","last_peered":"2024-06-10T01:04:35.563083+0000","last_clean":"2024-06-10T01:04:35.563083+0000","last_became_active":"2024-06-10T01:01:47.250858+0000","last_became_peered":"2024-06-10T01:01:47.250858+0000","last_unstale":"2024-06-10T01:04:35.563083+0000","last_undegraded":"2024-06-10T01:04:35.563083+0000","last_fullsized":"2024-06-10T01:04:35.563083+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:47.272727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.269600+0000","last_change":"2024-06-10T01:01:57.284187+0000","last_active":"2024-06-10T01:03:38.269600+0000","last_peered":"2024-06-10T01:03:38.269600+0000","last_clean":"2024-06-10T01:03:38.269600+0000","last_became_active":"2024-06-10T01:01:57.283739+0000","last_became_peered":"2024-06-10T01:01:57.283739+0000","last_unstale":"2024-06-10T01:03:38.269600+0000","last_undegraded":"2024-06-10T01:03:38.269600+0000","last_fullsized":"2024-06-10T01:03:38.269600+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:13:34.630708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771085+0000","last_change":"2024-06-10T01:01:44.005702+0000","last_active":"2024-06-10T01:03:38.771085+0000","last_peered":"2024-06-10T01:03:38.771085+0000","last_clean":"2024-06-10T01:03:38.771085+0000","last_became_active":"2024-06-10T01:01:44.005354+0000","last_became_peered":"2024-06-10T01:01:44.005354+0000","last_unstale":"2024-06-10T01:03:38.771085+0000","last_undegraded":"2024-06-10T01:03:38.771085+0000","last_fullsized":"2024-06-10T01:03:38.771085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:51:14.819688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558942+0000","last_change":"2024-06-10T01:01:47.250325+0000","last_active":"2024-06-10T01:04:35.558942+0000","last_peered":"2024-06-10T01:04:35.558942+0000","last_clean":"2024-06-10T01:04:35.558942+0000","last_became_active":"2024-06-10T01:01:47.249591+0000","last_became_peered":"2024-06-10T01:01:47.249591+0000","last_unstale":"2024-06-10T01:04:35.558942+0000","last_undegraded":"2024-06-10T01:04:35.558942+0000","last_fullsized":"2024-06-10T01:04:35.558942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:17:16.268717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.563635+0000","last_change":"2024-06-10T01:01:47.253449+0000","last_active":"2024-06-10T01:04:35.563635+0000","last_peered":"2024-06-10T01:04:35.563635+0000","last_clean":"2024-06-10T01:04:35.563635+0000","last_became_active":"2024-06-10T01:01:47.253144+0000","last_became_peered":"2024-06-10T01:01:47.253144+0000","last_unstale":"2024-06-10T01:04:35.563635+0000","last_undegraded":"2024-06-10T01:04:35.563635+0000","last_fullsized":"2024-06-10T01:04:35.563635+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:32:22.353409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.564016+0000","last_change":"2024-06-10T01:01:47.258241+0000","last_active":"2024-06-10T01:04:35.564016+0000","last_peered":"2024-06-10T01:04:35.564016+0000","last_clean":"2024-06-10T01:04:35.564016+0000","last_became_active":"2024-06-10T01:01:47.257823+0000","last_became_peered":"2024-06-10T01:01:47.257823+0000","last_unstale":"2024-06-10T01:04:35.564016+0000","last_undegraded":"2024-06-10T01:04:35.564016+0000","last_fullsized":"2024-06-10T01:04:35.564016+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:11:23.652479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559410+0000","last_change":"2024-06-10T01:01:47.256218+0000","last_active":"2024-06-10T01:04:35.559410+0000","last_peered":"2024-06-10T01:04:35.559410+0000","last_clean":"2024-06-10T01:04:35.559410+0000","last_became_active":"2024-06-10T01:01:47.255950+0000","last_became_peered":"2024-06-10T01:01:47.255950+0000","last_unstale":"2024-06-10T01:04:35.559410+0000","last_undegraded":"2024-06-10T01:04:35.559410+0000","last_fullsized":"2024-06-10T01:04:35.559410+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:59:37.283646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.556085+0000","last_change":"2024-06-10T01:01:47.253109+0000","last_active":"2024-06-10T01:04:35.556085+0000","last_peered":"2024-06-10T01:04:35.556085+0000","last_clean":"2024-06-10T01:04:35.556085+0000","last_became_active":"2024-06-10T01:01:47.252767+0000","last_became_peered":"2024-06-10T01:01:47.252767+0000","last_unstale":"2024-06-10T01:04:35.556085+0000","last_undegraded":"2024-06-10T01:04:35.556085+0000","last_fullsized":"2024-06-10T01:04:35.556085+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:57:13.177230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.550864+0000","last_change":"2024-06-10T01:02:37.375094+0000","last_active":"2024-06-10T01:04:35.550864+0000","last_peered":"2024-06-10T01:04:35.550864+0000","last_clean":"2024-06-10T01:04:35.550864+0000","last_became_active":"2024-06-10T01:02:37.374527+0000","last_became_peered":"2024-06-10T01:02:37.374527+0000","last_unstale":"2024-06-10T01:04:35.550864+0000","last_undegraded":"2024-06-10T01:04:35.550864+0000","last_fullsized":"2024-06-10T01:04:35.550864+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:34:41.205816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.551206+0000","last_change":"2024-06-10T01:01:47.250243+0000","last_active":"2024-06-10T01:04:35.551206+0000","last_peered":"2024-06-10T01:04:35.551206+0000","last_clean":"2024-06-10T01:04:35.551206+0000","last_became_active":"2024-06-10T01:01:47.249907+0000","last_became_peered":"2024-06-10T01:01:47.249907+0000","last_unstale":"2024-06-10T01:04:35.551206+0000","last_undegraded":"2024-06-10T01:04:35.551206+0000","last_fullsized":"2024-06-10T01:04:35.551206+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:31:32.933660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.564063+0000","last_change":"2024-06-10T01:01:47.254173+0000","last_active":"2024-06-10T01:04:35.564063+0000","last_peered":"2024-06-10T01:04:35.564063+0000","last_clean":"2024-06-10T01:04:35.564063+0000","last_became_active":"2024-06-10T01:01:47.253796+0000","last_became_peered":"2024-06-10T01:01:47.253796+0000","last_unstale":"2024-06-10T01:04:35.564063+0000","last_undegraded":"2024-06-10T01:04:35.564063+0000","last_fullsized":"2024-06-10T01:04:35.564063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:41:12.199114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":41,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.548118+0000","last_change":"2024-06-10T01:01:47.253920+0000","last_active":"2024-06-10T01:04:35.548118+0000","last_peered":"2024-06-10T01:04:35.548118+0000","last_clean":"2024-06-10T01:04:35.548118+0000","last_became_active":"2024-06-10T01:01:47.253288+0000","last_became_peered":"2024-06-10T01:01:47.253288+0000","last_unstale":"2024-06-10T01:04:35.548118+0000","last_undegraded":"2024-06-10T01:04:35.548118+0000","last_fullsized":"2024-06-10T01:04:35.548118+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:41:20.406263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.564816+0000","last_change":"2024-06-10T01:01:47.240772+0000","last_active":"2024-06-10T01:04:35.564816+0000","last_peered":"2024-06-10T01:04:35.564816+0000","last_clean":"2024-06-10T01:04:35.564816+0000","last_became_active":"2024-06-10T01:01:47.238258+0000","last_became_peered":"2024-06-10T01:01:47.238258+0000","last_unstale":"2024-06-10T01:04:35.564816+0000","last_undegraded":"2024-06-10T01:04:35.564816+0000","last_fullsized":"2024-06-10T01:04:35.564816+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:55:25.211434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":24,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.544703+0000","last_change":"2024-06-10T01:02:37.376724+0000","last_active":"2024-06-10T01:04:35.544703+0000","last_peered":"2024-06-10T01:04:35.544703+0000","last_clean":"2024-06-10T01:04:35.544703+0000","last_became_active":"2024-06-10T01:02:37.376329+0000","last_became_peered":"2024-06-10T01:02:37.376329+0000","last_unstale":"2024-06-10T01:04:35.544703+0000","last_undegraded":"2024-06-10T01:04:35.544703+0000","last_fullsized":"2024-06-10T01:04:35.544703+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:38:21.837958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.565197+0000","last_change":"2024-06-10T01:01:47.244463+0000","last_active":"2024-06-10T01:04:35.565197+0000","last_peered":"2024-06-10T01:04:35.565197+0000","last_clean":"2024-06-10T01:04:35.565197+0000","last_became_active":"2024-06-10T01:01:47.244114+0000","last_became_peered":"2024-06-10T01:01:47.244114+0000","last_unstale":"2024-06-10T01:04:35.565197+0000","last_undegraded":"2024-06-10T01:04:35.565197+0000","last_fullsized":"2024-06-10T01:04:35.565197+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:47:37.098894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559211+0000","last_change":"2024-06-10T01:01:47.248317+0000","last_active":"2024-06-10T01:04:35.559211+0000","last_peered":"2024-06-10T01:04:35.559211+0000","last_clean":"2024-06-10T01:04:35.559211+0000","last_became_active":"2024-06-10T01:01:47.247958+0000","last_became_peered":"2024-06-10T01:01:47.247958+0000","last_unstale":"2024-06-10T01:04:35.559211+0000","last_undegraded":"2024-06-10T01:04:35.559211+0000","last_fullsized":"2024-06-10T01:04:35.559211+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:39:59.267957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.552389+0000","last_change":"2024-06-10T01:01:47.252542+0000","last_active":"2024-06-10T01:04:35.552389+0000","last_peered":"2024-06-10T01:04:35.552389+0000","last_clean":"2024-06-10T01:04:35.552389+0000","last_became_active":"2024-06-10T01:01:47.252216+0000","last_became_peered":"2024-06-10T01:01:47.252216+0000","last_unstale":"2024-06-10T01:04:35.552389+0000","last_undegraded":"2024-06-10T01:04:35.552389+0000","last_fullsized":"2024-06-10T01:04:35.552389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:12:22.196263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285691,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68533,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1018,"num_read_kb":581,"num_write":183,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":205599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383743,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":569843,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645380,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32388,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712764,"statfs":{"total":95995035648,"available":95961870336,"internally_reserved":0,"allocated":1241088,"data_stored":1221922,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808905,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":574063,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070542,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":569775,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332178,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":1165405,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495703,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":565143,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574960,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":629668,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738486,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":566531,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411435,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":630598,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182503,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":565337,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920865,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":569905,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757341,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":1160993,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34687,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34687,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34687,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":65853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T01:05:44.378 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-06-10T01:05:44.378 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-10T01:05:44.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.521+0000 7fc432770640 1 -- 172.21.15.7:0/574675110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc42c101af0 msgr2=0x7fc42c101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:44.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.521+0000 7fc432770640 1 --2- 172.21.15.7:0/574675110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc42c101af0 0x7fc42c101f70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc414009900 tx=0x7fc41402a410 comp rx=0 tx=0).stop 2024-06-10T01:05:44.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.521+0000 7fc432770640 1 -- 172.21.15.7:0/574675110 shutdown_connections 2024-06-10T01:05:44.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.521+0000 7fc432770640 1 --2- 172.21.15.7:0/574675110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc42c1024b0 0x7fc42c108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:44.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.521+0000 7fc432770640 1 --2- 172.21.15.7:0/574675110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc42c101af0 0x7fc42c101f70 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:44.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.521+0000 7fc432770640 1 --2- 172.21.15.7:0/574675110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc42c100900 0x7fc42c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:44.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.521+0000 7fc432770640 1 -- 172.21.15.7:0/574675110 >> 172.21.15.7:0/574675110 conn(0x7fc42c0fbeb0 msgr2=0x7fc42c0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:44.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.521+0000 7fc432770640 1 -- 172.21.15.7:0/574675110 shutdown_connections 2024-06-10T01:05:44.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.521+0000 7fc432770640 1 -- 172.21.15.7:0/574675110 wait complete. 2024-06-10T01:05:44.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.522+0000 7fc432770640 1 Processor -- start 2024-06-10T01:05:44.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.522+0000 7fc432770640 1 -- start start 2024-06-10T01:05:44.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.522+0000 7fc432770640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc42c100900 0x7fc42c06ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:44.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.522+0000 7fc432770640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc42c101af0 0x7fc42c19ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:44.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.522+0000 7fc432770640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc42c1024b0 0x7fc42c19f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:44.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.522+0000 7fc432770640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c10c560 con 0x7fc42c101af0 2024-06-10T01:05:44.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.522+0000 7fc432770640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c10c3e0 con 0x7fc42c1024b0 2024-06-10T01:05:44.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.522+0000 7fc432770640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c10c6e0 con 0x7fc42c100900 2024-06-10T01:05:44.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.522+0000 7fc423fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc42c101af0 0x7fc42c19ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:44.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.522+0000 7fc423fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc42c101af0 0x7fc42c19ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58594/0 (socket says 172.21.15.7:58594) 2024-06-10T01:05:44.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.522+0000 7fc423fff640 1 -- 172.21.15.7:0/2443418752 learned_addr learned my addr 172.21.15.7:0/2443418752 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:44.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.522+0000 7fc42bfff640 1 --2- 172.21.15.7:0/2443418752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc42c100900 0x7fc42c06ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:44.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.522+0000 7fc430d0d640 1 --2- 172.21.15.7:0/2443418752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc42c1024b0 0x7fc42c19f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.522+0000 7fc423fff640 1 -- 172.21.15.7:0/2443418752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc42c100900 msgr2=0x7fc42c06ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.522+0000 7fc423fff640 1 --2- 172.21.15.7:0/2443418752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc42c100900 0x7fc42c06ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.522+0000 7fc423fff640 1 -- 172.21.15.7:0/2443418752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc42c1024b0 msgr2=0x7fc42c19f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.522+0000 7fc423fff640 1 --2- 172.21.15.7:0/2443418752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc42c1024b0 0x7fc42c19f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.522+0000 7fc423fff640 1 -- 172.21.15.7:0/2443418752 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42c19f990 con 0x7fc42c101af0 2024-06-10T01:05:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.522+0000 7fc430d0d640 1 --2- 172.21.15.7:0/2443418752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc42c1024b0 0x7fc42c19f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:05:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.522+0000 7fc42bfff640 1 --2- 172.21.15.7:0/2443418752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc42c100900 0x7fc42c06ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:05:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.523+0000 7fc423fff640 1 --2- 172.21.15.7:0/2443418752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc42c101af0 0x7fc42c19ca90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc414009230 tx=0x7fc414031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.523+0000 7fc429ffb640 1 -- 172.21.15.7:0/2443418752 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41403f030 con 0x7fc42c101af0 2024-06-10T01:05:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.523+0000 7fc432770640 1 -- 172.21.15.7:0/2443418752 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc42c19fc20 con 0x7fc42c101af0 2024-06-10T01:05:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.523+0000 7fc429ffb640 1 -- 172.21.15.7:0/2443418752 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc414003ad0 con 0x7fc42c101af0 2024-06-10T01:05:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.523+0000 7fc429ffb640 1 -- 172.21.15.7:0/2443418752 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414035070 con 0x7fc42c101af0 2024-06-10T01:05:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.523+0000 7fc432770640 1 -- 172.21.15.7:0/2443418752 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc42c1a0160 con 0x7fc42c101af0 2024-06-10T01:05:44.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.524+0000 7fc432770640 1 -- 172.21.15.7:0/2443418752 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc42c107ca0 con 0x7fc42c101af0 2024-06-10T01:05:44.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.525+0000 7fc429ffb640 1 -- 172.21.15.7:0/2443418752 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fc414042030 con 0x7fc42c101af0 2024-06-10T01:05:44.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.526+0000 7fc429ffb640 1 --2- 172.21.15.7:0/2443418752 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc4180a2b90 0x7fc4180a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:44.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.526+0000 7fc42bfff640 1 --2- 172.21.15.7:0/2443418752 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc4180a2b90 0x7fc4180a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:44.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.526+0000 7fc429ffb640 1 -- 172.21.15.7:0/2443418752 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fc4140efa50 con 0x7fc42c101af0 2024-06-10T01:05:44.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.526+0000 7fc42bfff640 1 --2- 172.21.15.7:0/2443418752 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc4180a2b90 0x7fc4180a5050 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc42c101950 tx=0x7fc41c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:44.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.530+0000 7fc429ffb640 1 -- 172.21.15.7:0/2443418752 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc4140bc5d0 con 0x7fc42c101af0 2024-06-10T01:05:44.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.713+0000 7fc432770640 1 -- 172.21.15.7:0/2443418752 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc42c061de0 con 0x7fc4180a2b90 2024-06-10T01:05:44.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.722+0000 7fc429ffb640 1 -- 172.21.15.7:0/2443418752 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340766 (secure 0 0 0) 0x7fc42c061de0 con 0x7fc4180a2b90 2024-06-10T01:05:44.723 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:05:44.725 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-06-10T01:05:44.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.725+0000 7fc432770640 1 -- 172.21.15.7:0/2443418752 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc4180a2b90 msgr2=0x7fc4180a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:44.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.725+0000 7fc432770640 1 --2- 172.21.15.7:0/2443418752 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc4180a2b90 0x7fc4180a5050 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc42c101950 tx=0x7fc41c005f90 comp rx=0 tx=0).stop 2024-06-10T01:05:44.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.725+0000 7fc432770640 1 -- 172.21.15.7:0/2443418752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc42c101af0 msgr2=0x7fc42c19ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:44.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.725+0000 7fc432770640 1 --2- 172.21.15.7:0/2443418752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc42c101af0 0x7fc42c19ca90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc414009230 tx=0x7fc414031b60 comp rx=0 tx=0).stop 2024-06-10T01:05:44.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.726+0000 7fc432770640 1 -- 172.21.15.7:0/2443418752 shutdown_connections 2024-06-10T01:05:44.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.726+0000 7fc432770640 1 --2- 172.21.15.7:0/2443418752 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc4180a2b90 0x7fc4180a5050 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:44.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.726+0000 7fc432770640 1 --2- 172.21.15.7:0/2443418752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc42c1024b0 0x7fc42c19f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:44.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.726+0000 7fc432770640 1 --2- 172.21.15.7:0/2443418752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc42c101af0 0x7fc42c19ca90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:44.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.726+0000 7fc432770640 1 --2- 172.21.15.7:0/2443418752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc42c100900 0x7fc42c06ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:44.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.726+0000 7fc432770640 1 -- 172.21.15.7:0/2443418752 >> 172.21.15.7:0/2443418752 conn(0x7fc42c0fbeb0 msgr2=0x7fc42c104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:44.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.726+0000 7fc432770640 1 -- 172.21.15.7:0/2443418752 shutdown_connections 2024-06-10T01:05:44.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:44.726+0000 7fc432770640 1 -- 172.21.15.7:0/2443418752 wait complete. 2024-06-10T01:05:44.734 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":423,"stamp":"2024-06-10T01:05:43.597017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659135,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1144,"num_read_kb":690,"num_write":366,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":793,"ondisk_log_size":793,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362468,"kb_used_data":13316,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579356,"statfs":{"total":1151940427776,"available":1151569260544,"internally_reserved":0,"allocated":13635584,"data_stored":8789183,"data_compressed":496626,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":64,"num_read_kb":30,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003015"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.569445+0000","last_change":"2024-06-10T01:01:47.244728+0000","last_active":"2024-06-10T01:04:35.569445+0000","last_peered":"2024-06-10T01:04:35.569445+0000","last_clean":"2024-06-10T01:04:35.569445+0000","last_became_active":"2024-06-10T01:01:47.244198+0000","last_became_peered":"2024-06-10T01:01:47.244198+0000","last_unstale":"2024-06-10T01:04:35.569445+0000","last_undegraded":"2024-06-10T01:04:35.569445+0000","last_fullsized":"2024-06-10T01:04:35.569445+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:58:06.257952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.557217+0000","last_change":"2024-06-10T01:01:47.247620+0000","last_active":"2024-06-10T01:04:35.557217+0000","last_peered":"2024-06-10T01:04:35.557217+0000","last_clean":"2024-06-10T01:04:35.557217+0000","last_became_active":"2024-06-10T01:01:47.245475+0000","last_became_peered":"2024-06-10T01:01:47.245475+0000","last_unstale":"2024-06-10T01:04:35.557217+0000","last_undegraded":"2024-06-10T01:04:35.557217+0000","last_fullsized":"2024-06-10T01:04:35.557217+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:29:09.367411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.569660+0000","last_change":"2024-06-10T01:01:47.252414+0000","last_active":"2024-06-10T01:04:35.569660+0000","last_peered":"2024-06-10T01:04:35.569660+0000","last_clean":"2024-06-10T01:04:35.569660+0000","last_became_active":"2024-06-10T01:01:47.252063+0000","last_became_peered":"2024-06-10T01:01:47.252063+0000","last_unstale":"2024-06-10T01:04:35.569660+0000","last_undegraded":"2024-06-10T01:04:35.569660+0000","last_fullsized":"2024-06-10T01:04:35.569660+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:54:53.966209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.553564+0000","last_change":"2024-06-10T01:01:47.254875+0000","last_active":"2024-06-10T01:04:35.553564+0000","last_peered":"2024-06-10T01:04:35.553564+0000","last_clean":"2024-06-10T01:04:35.553564+0000","last_became_active":"2024-06-10T01:01:47.254542+0000","last_became_peered":"2024-06-10T01:01:47.254542+0000","last_unstale":"2024-06-10T01:04:35.553564+0000","last_undegraded":"2024-06-10T01:04:35.553564+0000","last_fullsized":"2024-06-10T01:04:35.553564+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:02:28.393133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558663+0000","last_change":"2024-06-10T01:01:47.242596+0000","last_active":"2024-06-10T01:04:35.558663+0000","last_peered":"2024-06-10T01:04:35.558663+0000","last_clean":"2024-06-10T01:04:35.558663+0000","last_became_active":"2024-06-10T01:01:47.242308+0000","last_became_peered":"2024-06-10T01:01:47.242308+0000","last_unstale":"2024-06-10T01:04:35.558663+0000","last_undegraded":"2024-06-10T01:04:35.558663+0000","last_fullsized":"2024-06-10T01:04:35.558663+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:11:26.599954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.550600+0000","last_change":"2024-06-10T01:01:47.249085+0000","last_active":"2024-06-10T01:04:35.550600+0000","last_peered":"2024-06-10T01:04:35.550600+0000","last_clean":"2024-06-10T01:04:35.550600+0000","last_became_active":"2024-06-10T01:01:47.248749+0000","last_became_peered":"2024-06-10T01:01:47.248749+0000","last_unstale":"2024-06-10T01:04:35.550600+0000","last_undegraded":"2024-06-10T01:04:35.550600+0000","last_fullsized":"2024-06-10T01:04:35.550600+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:42:48.734566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562583+0000","last_change":"2024-06-10T01:01:47.247334+0000","last_active":"2024-06-10T01:04:35.562583+0000","last_peered":"2024-06-10T01:04:35.562583+0000","last_clean":"2024-06-10T01:04:35.562583+0000","last_became_active":"2024-06-10T01:01:47.246964+0000","last_became_peered":"2024-06-10T01:01:47.246964+0000","last_unstale":"2024-06-10T01:04:35.562583+0000","last_undegraded":"2024-06-10T01:04:35.562583+0000","last_fullsized":"2024-06-10T01:04:35.562583+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:05:36.034950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.563056+0000","last_change":"2024-06-10T01:01:47.242523+0000","last_active":"2024-06-10T01:04:35.563056+0000","last_peered":"2024-06-10T01:04:35.563056+0000","last_clean":"2024-06-10T01:04:35.563056+0000","last_became_active":"2024-06-10T01:01:47.242198+0000","last_became_peered":"2024-06-10T01:01:47.242198+0000","last_unstale":"2024-06-10T01:04:35.563056+0000","last_undegraded":"2024-06-10T01:04:35.563056+0000","last_fullsized":"2024-06-10T01:04:35.563056+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:39:07.055970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.567116+0000","last_change":"2024-06-10T01:01:47.245277+0000","last_active":"2024-06-10T01:04:35.567116+0000","last_peered":"2024-06-10T01:04:35.567116+0000","last_clean":"2024-06-10T01:04:35.567116+0000","last_became_active":"2024-06-10T01:01:47.244932+0000","last_became_peered":"2024-06-10T01:01:47.244932+0000","last_unstale":"2024-06-10T01:04:35.567116+0000","last_undegraded":"2024-06-10T01:04:35.567116+0000","last_fullsized":"2024-06-10T01:04:35.567116+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:56:24.423713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559784+0000","last_change":"2024-06-10T01:01:47.255531+0000","last_active":"2024-06-10T01:04:35.559784+0000","last_peered":"2024-06-10T01:04:35.559784+0000","last_clean":"2024-06-10T01:04:35.559784+0000","last_became_active":"2024-06-10T01:01:47.255150+0000","last_became_peered":"2024-06-10T01:01:47.255150+0000","last_unstale":"2024-06-10T01:04:35.559784+0000","last_undegraded":"2024-06-10T01:04:35.559784+0000","last_fullsized":"2024-06-10T01:04:35.559784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:51:34.561396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559164+0000","last_change":"2024-06-10T01:01:47.249832+0000","last_active":"2024-06-10T01:04:35.559164+0000","last_peered":"2024-06-10T01:04:35.559164+0000","last_clean":"2024-06-10T01:04:35.559164+0000","last_became_active":"2024-06-10T01:01:47.249303+0000","last_became_peered":"2024-06-10T01:01:47.249303+0000","last_unstale":"2024-06-10T01:04:35.559164+0000","last_undegraded":"2024-06-10T01:04:35.559164+0000","last_fullsized":"2024-06-10T01:04:35.559164+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:15:21.930157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.551184+0000","last_change":"2024-06-10T01:01:47.255927+0000","last_active":"2024-06-10T01:04:35.551184+0000","last_peered":"2024-06-10T01:04:35.551184+0000","last_clean":"2024-06-10T01:04:35.551184+0000","last_became_active":"2024-06-10T01:01:47.255591+0000","last_became_peered":"2024-06-10T01:01:47.255591+0000","last_unstale":"2024-06-10T01:04:35.551184+0000","last_undegraded":"2024-06-10T01:04:35.551184+0000","last_fullsized":"2024-06-10T01:04:35.551184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:48:06.494430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.557880+0000","last_change":"2024-06-10T01:02:37.372069+0000","last_active":"2024-06-10T01:04:35.557880+0000","last_peered":"2024-06-10T01:04:35.557880+0000","last_clean":"2024-06-10T01:04:35.557880+0000","last_became_active":"2024-06-10T01:02:37.371599+0000","last_became_peered":"2024-06-10T01:02:37.371599+0000","last_unstale":"2024-06-10T01:04:35.557880+0000","last_undegraded":"2024-06-10T01:04:35.557880+0000","last_fullsized":"2024-06-10T01:04:35.557880+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:54:34.091576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.551120+0000","last_change":"2024-06-10T01:01:47.249553+0000","last_active":"2024-06-10T01:04:35.551120+0000","last_peered":"2024-06-10T01:04:35.551120+0000","last_clean":"2024-06-10T01:04:35.551120+0000","last_became_active":"2024-06-10T01:01:47.249202+0000","last_became_peered":"2024-06-10T01:01:47.249202+0000","last_unstale":"2024-06-10T01:04:35.551120+0000","last_undegraded":"2024-06-10T01:04:35.551120+0000","last_fullsized":"2024-06-10T01:04:35.551120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:23:57.882264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.564289+0000","last_change":"2024-06-10T01:01:47.253987+0000","last_active":"2024-06-10T01:04:35.564289+0000","last_peered":"2024-06-10T01:04:35.564289+0000","last_clean":"2024-06-10T01:04:35.564289+0000","last_became_active":"2024-06-10T01:01:47.253578+0000","last_became_peered":"2024-06-10T01:01:47.253578+0000","last_unstale":"2024-06-10T01:04:35.564289+0000","last_undegraded":"2024-06-10T01:04:35.564289+0000","last_fullsized":"2024-06-10T01:04:35.564289+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:35:30.585482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.567601+0000","last_change":"2024-06-10T01:01:47.247089+0000","last_active":"2024-06-10T01:04:35.567601+0000","last_peered":"2024-06-10T01:04:35.567601+0000","last_clean":"2024-06-10T01:04:35.567601+0000","last_became_active":"2024-06-10T01:01:47.246066+0000","last_became_peered":"2024-06-10T01:01:47.246066+0000","last_unstale":"2024-06-10T01:04:35.567601+0000","last_undegraded":"2024-06-10T01:04:35.567601+0000","last_fullsized":"2024-06-10T01:04:35.567601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:58:12.092981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.569804+0000","last_change":"2024-06-10T01:01:47.244491+0000","last_active":"2024-06-10T01:04:35.569804+0000","last_peered":"2024-06-10T01:04:35.569804+0000","last_clean":"2024-06-10T01:04:35.569804+0000","last_became_active":"2024-06-10T01:01:47.243747+0000","last_became_peered":"2024-06-10T01:01:47.243747+0000","last_unstale":"2024-06-10T01:04:35.569804+0000","last_undegraded":"2024-06-10T01:04:35.569804+0000","last_fullsized":"2024-06-10T01:04:35.569804+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:55:56.785016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.567988+0000","last_change":"2024-06-10T01:01:47.243141+0000","last_active":"2024-06-10T01:04:35.567988+0000","last_peered":"2024-06-10T01:04:35.567988+0000","last_clean":"2024-06-10T01:04:35.567988+0000","last_became_active":"2024-06-10T01:01:47.242464+0000","last_became_peered":"2024-06-10T01:01:47.242464+0000","last_unstale":"2024-06-10T01:04:35.567988+0000","last_undegraded":"2024-06-10T01:04:35.567988+0000","last_fullsized":"2024-06-10T01:04:35.567988+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:00:49.777130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559919+0000","last_change":"2024-06-10T01:01:47.253463+0000","last_active":"2024-06-10T01:04:35.559919+0000","last_peered":"2024-06-10T01:04:35.559919+0000","last_clean":"2024-06-10T01:04:35.559919+0000","last_became_active":"2024-06-10T01:01:47.252929+0000","last_became_peered":"2024-06-10T01:01:47.252929+0000","last_unstale":"2024-06-10T01:04:35.559919+0000","last_undegraded":"2024-06-10T01:04:35.559919+0000","last_fullsized":"2024-06-10T01:04:35.559919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:18:49.551276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239453+0000","last_change":"2024-06-10T01:01:57.279561+0000","last_active":"2024-06-10T01:03:38.239453+0000","last_peered":"2024-06-10T01:03:38.239453+0000","last_clean":"2024-06-10T01:03:38.239453+0000","last_became_active":"2024-06-10T01:01:57.279132+0000","last_became_peered":"2024-06-10T01:01:57.279132+0000","last_unstale":"2024-06-10T01:03:38.239453+0000","last_undegraded":"2024-06-10T01:03:38.239453+0000","last_fullsized":"2024-06-10T01:03:38.239453+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:55:03.497701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'9","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.772124+0000","last_change":"2024-06-10T01:01:44.006667+0000","last_active":"2024-06-10T01:03:38.772124+0000","last_peered":"2024-06-10T01:03:38.772124+0000","last_clean":"2024-06-10T01:03:38.772124+0000","last_became_active":"2024-06-10T01:01:44.006343+0000","last_became_peered":"2024-06-10T01:01:44.006343+0000","last_unstale":"2024-06-10T01:03:38.772124+0000","last_undegraded":"2024-06-10T01:03:38.772124+0000","last_fullsized":"2024-06-10T01:03:38.772124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:14:50.540084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562836+0000","last_change":"2024-06-10T01:01:47.246139+0000","last_active":"2024-06-10T01:04:35.562836+0000","last_peered":"2024-06-10T01:04:35.562836+0000","last_clean":"2024-06-10T01:04:35.562836+0000","last_became_active":"2024-06-10T01:01:47.245743+0000","last_became_peered":"2024-06-10T01:01:47.245743+0000","last_unstale":"2024-06-10T01:04:35.562836+0000","last_undegraded":"2024-06-10T01:04:35.562836+0000","last_fullsized":"2024-06-10T01:04:35.562836+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:06:39.041569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239398+0000","last_change":"2024-06-10T01:01:57.282843+0000","last_active":"2024-06-10T01:03:38.239398+0000","last_peered":"2024-06-10T01:03:38.239398+0000","last_clean":"2024-06-10T01:03:38.239398+0000","last_became_active":"2024-06-10T01:01:57.281797+0000","last_became_peered":"2024-06-10T01:01:57.281797+0000","last_unstale":"2024-06-10T01:03:38.239398+0000","last_undegraded":"2024-06-10T01:03:38.239398+0000","last_fullsized":"2024-06-10T01:03:38.239398+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:00.209628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239550+0000","last_change":"2024-06-10T01:01:44.466714+0000","last_active":"2024-06-10T01:03:38.239550+0000","last_peered":"2024-06-10T01:03:38.239550+0000","last_clean":"2024-06-10T01:03:38.239550+0000","last_became_active":"2024-06-10T01:01:44.466296+0000","last_became_peered":"2024-06-10T01:01:44.466296+0000","last_unstale":"2024-06-10T01:03:38.239550+0000","last_undegraded":"2024-06-10T01:03:38.239550+0000","last_fullsized":"2024-06-10T01:03:38.239550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:16:36.082308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.569808+0000","last_change":"2024-06-10T01:01:47.244173+0000","last_active":"2024-06-10T01:04:35.569808+0000","last_peered":"2024-06-10T01:04:35.569808+0000","last_clean":"2024-06-10T01:04:35.569808+0000","last_became_active":"2024-06-10T01:01:47.243032+0000","last_became_peered":"2024-06-10T01:01:47.243032+0000","last_unstale":"2024-06-10T01:04:35.569808+0000","last_undegraded":"2024-06-10T01:04:35.569808+0000","last_fullsized":"2024-06-10T01:04:35.569808+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:14:33.965903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239057+0000","last_change":"2024-06-10T01:02:37.592535+0000","last_active":"2024-06-10T01:03:38.239057+0000","last_peered":"2024-06-10T01:03:38.239057+0000","last_clean":"2024-06-10T01:03:38.239057+0000","last_became_active":"2024-06-10T01:02:37.592159+0000","last_became_peered":"2024-06-10T01:02:37.592159+0000","last_unstale":"2024-06-10T01:03:38.239057+0000","last_undegraded":"2024-06-10T01:03:38.239057+0000","last_fullsized":"2024-06-10T01:03:38.239057+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:50:08.626485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235433+0000","last_change":"2024-06-10T01:01:44.009105+0000","last_active":"2024-06-10T01:03:38.235433+0000","last_peered":"2024-06-10T01:03:38.235433+0000","last_clean":"2024-06-10T01:03:38.235433+0000","last_became_active":"2024-06-10T01:01:44.008743+0000","last_became_peered":"2024-06-10T01:01:44.008743+0000","last_unstale":"2024-06-10T01:03:38.235433+0000","last_undegraded":"2024-06-10T01:03:38.235433+0000","last_fullsized":"2024-06-10T01:03:38.235433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:44:54.301378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.570239+0000","last_change":"2024-06-10T01:01:47.249007+0000","last_active":"2024-06-10T01:04:35.570239+0000","last_peered":"2024-06-10T01:04:35.570239+0000","last_clean":"2024-06-10T01:04:35.570239+0000","last_became_active":"2024-06-10T01:01:47.245632+0000","last_became_peered":"2024-06-10T01:01:47.245632+0000","last_unstale":"2024-06-10T01:04:35.570239+0000","last_undegraded":"2024-06-10T01:04:35.570239+0000","last_fullsized":"2024-06-10T01:04:35.570239+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:29:18.916390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.238983+0000","last_change":"2024-06-10T01:01:57.283137+0000","last_active":"2024-06-10T01:03:38.238983+0000","last_peered":"2024-06-10T01:03:38.238983+0000","last_clean":"2024-06-10T01:03:38.238983+0000","last_became_active":"2024-06-10T01:01:57.282605+0000","last_became_peered":"2024-06-10T01:01:57.282605+0000","last_unstale":"2024-06-10T01:03:38.238983+0000","last_undegraded":"2024-06-10T01:03:38.238983+0000","last_fullsized":"2024-06-10T01:03:38.238983+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:26:16.967320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235734+0000","last_change":"2024-06-10T01:01:44.010674+0000","last_active":"2024-06-10T01:03:38.235734+0000","last_peered":"2024-06-10T01:03:38.235734+0000","last_clean":"2024-06-10T01:03:38.235734+0000","last_became_active":"2024-06-10T01:01:44.010291+0000","last_became_peered":"2024-06-10T01:01:44.010291+0000","last_unstale":"2024-06-10T01:03:38.235734+0000","last_undegraded":"2024-06-10T01:03:38.235734+0000","last_fullsized":"2024-06-10T01:03:38.235734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:36:18.418362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":60,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.566078+0000","last_change":"2024-06-10T01:02:37.794580+0000","last_active":"2024-06-10T01:04:35.566078+0000","last_peered":"2024-06-10T01:04:35.566078+0000","last_clean":"2024-06-10T01:04:35.566078+0000","last_became_active":"2024-06-10T01:02:37.794127+0000","last_became_peered":"2024-06-10T01:02:37.794127+0000","last_unstale":"2024-06-10T01:04:35.566078+0000","last_undegraded":"2024-06-10T01:04:35.566078+0000","last_fullsized":"2024-06-10T01:04:35.566078+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:02.117871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239746+0000","last_change":"2024-06-10T01:01:57.282494+0000","last_active":"2024-06-10T01:03:38.239746+0000","last_peered":"2024-06-10T01:03:38.239746+0000","last_clean":"2024-06-10T01:03:38.239746+0000","last_became_active":"2024-06-10T01:01:57.282077+0000","last_became_peered":"2024-06-10T01:01:57.282077+0000","last_unstale":"2024-06-10T01:03:38.239746+0000","last_undegraded":"2024-06-10T01:03:38.239746+0000","last_fullsized":"2024-06-10T01:03:38.239746+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:57:41.639788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"97'5","reported_seq":219,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:40.951609+0000","last_change":"2024-06-10T01:01:44.011134+0000","last_active":"2024-06-10T01:05:40.951609+0000","last_peered":"2024-06-10T01:05:40.951609+0000","last_clean":"2024-06-10T01:05:40.951609+0000","last_became_active":"2024-06-10T01:01:44.010788+0000","last_became_peered":"2024-06-10T01:01:44.010788+0000","last_unstale":"2024-06-10T01:05:40.951609+0000","last_undegraded":"2024-06-10T01:05:40.951609+0000","last_fullsized":"2024-06-10T01:05:40.951609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:22:55.228946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":169,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.564421+0000","last_change":"2024-06-10T01:01:47.248093+0000","last_active":"2024-06-10T01:04:35.564421+0000","last_peered":"2024-06-10T01:04:35.564421+0000","last_clean":"2024-06-10T01:04:35.564421+0000","last_became_active":"2024-06-10T01:01:47.247797+0000","last_became_peered":"2024-06-10T01:01:47.247797+0000","last_unstale":"2024-06-10T01:04:35.564421+0000","last_undegraded":"2024-06-10T01:04:35.564421+0000","last_fullsized":"2024-06-10T01:04:35.564421+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:45:50.159128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239671+0000","last_change":"2024-06-10T01:02:37.377039+0000","last_active":"2024-06-10T01:03:38.239671+0000","last_peered":"2024-06-10T01:03:38.239671+0000","last_clean":"2024-06-10T01:03:38.239671+0000","last_became_active":"2024-06-10T01:02:37.376332+0000","last_became_peered":"2024-06-10T01:02:37.376332+0000","last_unstale":"2024-06-10T01:03:38.239671+0000","last_undegraded":"2024-06-10T01:03:38.239671+0000","last_fullsized":"2024-06-10T01:03:38.239671+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:15.000505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,10,5],"acting":[2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.570465+0000","last_change":"2024-06-10T01:01:44.008342+0000","last_active":"2024-06-10T01:03:38.570465+0000","last_peered":"2024-06-10T01:03:38.570465+0000","last_clean":"2024-06-10T01:03:38.570465+0000","last_became_active":"2024-06-10T01:01:44.007959+0000","last_became_peered":"2024-06-10T01:01:44.007959+0000","last_unstale":"2024-06-10T01:03:38.570465+0000","last_undegraded":"2024-06-10T01:03:38.570465+0000","last_fullsized":"2024-06-10T01:03:38.570465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:36:03.143609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.565044+0000","last_change":"2024-06-10T01:01:47.247854+0000","last_active":"2024-06-10T01:04:35.565044+0000","last_peered":"2024-06-10T01:04:35.565044+0000","last_clean":"2024-06-10T01:04:35.565044+0000","last_became_active":"2024-06-10T01:01:47.247563+0000","last_became_peered":"2024-06-10T01:01:47.247563+0000","last_unstale":"2024-06-10T01:04:35.565044+0000","last_undegraded":"2024-06-10T01:04:35.565044+0000","last_fullsized":"2024-06-10T01:04:35.565044+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:32:13.024815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.237041+0000","last_change":"2024-06-10T01:01:57.282427+0000","last_active":"2024-06-10T01:03:38.237041+0000","last_peered":"2024-06-10T01:03:38.237041+0000","last_clean":"2024-06-10T01:03:38.237041+0000","last_became_active":"2024-06-10T01:01:57.281987+0000","last_became_peered":"2024-06-10T01:01:57.281987+0000","last_unstale":"2024-06-10T01:03:38.237041+0000","last_undegraded":"2024-06-10T01:03:38.237041+0000","last_fullsized":"2024-06-10T01:03:38.237041+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:22:31.736804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770902+0000","last_change":"2024-06-10T01:01:44.006567+0000","last_active":"2024-06-10T01:03:38.770902+0000","last_peered":"2024-06-10T01:03:38.770902+0000","last_clean":"2024-06-10T01:03:38.770902+0000","last_became_active":"2024-06-10T01:01:44.006216+0000","last_became_peered":"2024-06-10T01:01:44.006216+0000","last_unstale":"2024-06-10T01:03:38.770902+0000","last_undegraded":"2024-06-10T01:03:38.770902+0000","last_fullsized":"2024-06-10T01:03:38.770902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:33:55.696873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558008+0000","last_change":"2024-06-10T01:01:47.245326+0000","last_active":"2024-06-10T01:04:35.558008+0000","last_peered":"2024-06-10T01:04:35.558008+0000","last_clean":"2024-06-10T01:04:35.558008+0000","last_became_active":"2024-06-10T01:01:47.245013+0000","last_became_peered":"2024-06-10T01:01:47.245013+0000","last_unstale":"2024-06-10T01:04:35.558008+0000","last_undegraded":"2024-06-10T01:04:35.558008+0000","last_fullsized":"2024-06-10T01:04:35.558008+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:17:56.416627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234530+0000","last_change":"2024-06-10T01:01:57.288997+0000","last_active":"2024-06-10T01:03:38.234530+0000","last_peered":"2024-06-10T01:03:38.234530+0000","last_clean":"2024-06-10T01:03:38.234530+0000","last_became_active":"2024-06-10T01:01:57.288596+0000","last_became_peered":"2024-06-10T01:01:57.288596+0000","last_unstale":"2024-06-10T01:03:38.234530+0000","last_undegraded":"2024-06-10T01:03:38.234530+0000","last_fullsized":"2024-06-10T01:03:38.234530+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:08:14.739737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.269738+0000","last_change":"2024-06-10T01:01:44.010079+0000","last_active":"2024-06-10T01:03:38.269738+0000","last_peered":"2024-06-10T01:03:38.269738+0000","last_clean":"2024-06-10T01:03:38.269738+0000","last_became_active":"2024-06-10T01:01:44.009752+0000","last_became_peered":"2024-06-10T01:01:44.009752+0000","last_unstale":"2024-06-10T01:03:38.269738+0000","last_undegraded":"2024-06-10T01:03:38.269738+0000","last_fullsized":"2024-06-10T01:03:38.269738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:29:35.660935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.554398+0000","last_change":"2024-06-10T01:01:47.245015+0000","last_active":"2024-06-10T01:04:35.554398+0000","last_peered":"2024-06-10T01:04:35.554398+0000","last_clean":"2024-06-10T01:04:35.554398+0000","last_became_active":"2024-06-10T01:01:47.244595+0000","last_became_peered":"2024-06-10T01:01:47.244595+0000","last_unstale":"2024-06-10T01:04:35.554398+0000","last_undegraded":"2024-06-10T01:04:35.554398+0000","last_fullsized":"2024-06-10T01:04:35.554398+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:01:48.446526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.238926+0000","last_change":"2024-06-10T01:01:57.281656+0000","last_active":"2024-06-10T01:03:38.238926+0000","last_peered":"2024-06-10T01:03:38.238926+0000","last_clean":"2024-06-10T01:03:38.238926+0000","last_became_active":"2024-06-10T01:01:57.281295+0000","last_became_peered":"2024-06-10T01:01:57.281295+0000","last_unstale":"2024-06-10T01:03:38.238926+0000","last_undegraded":"2024-06-10T01:03:38.238926+0000","last_fullsized":"2024-06-10T01:03:38.238926+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:04:41.619864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"97'3","reported_seq":214,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:39.949599+0000","last_change":"2024-06-10T01:01:44.011941+0000","last_active":"2024-06-10T01:05:39.949599+0000","last_peered":"2024-06-10T01:05:39.949599+0000","last_clean":"2024-06-10T01:05:39.949599+0000","last_became_active":"2024-06-10T01:01:44.010156+0000","last_became_peered":"2024-06-10T01:01:44.010156+0000","last_unstale":"2024-06-10T01:05:39.949599+0000","last_undegraded":"2024-06-10T01:05:39.949599+0000","last_fullsized":"2024-06-10T01:05:39.949599+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:16:15.709544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":166,"num_read_kb":166,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.570257+0000","last_change":"2024-06-10T01:01:47.241720+0000","last_active":"2024-06-10T01:04:35.570257+0000","last_peered":"2024-06-10T01:04:35.570257+0000","last_clean":"2024-06-10T01:04:35.570257+0000","last_became_active":"2024-06-10T01:01:47.241386+0000","last_became_peered":"2024-06-10T01:01:47.241386+0000","last_unstale":"2024-06-10T01:04:35.570257+0000","last_undegraded":"2024-06-10T01:04:35.570257+0000","last_fullsized":"2024-06-10T01:04:35.570257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:42:01.304787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770709+0000","last_change":"2024-06-10T01:01:57.283226+0000","last_active":"2024-06-10T01:03:38.770709+0000","last_peered":"2024-06-10T01:03:38.770709+0000","last_clean":"2024-06-10T01:03:38.770709+0000","last_became_active":"2024-06-10T01:01:57.282720+0000","last_became_peered":"2024-06-10T01:01:57.282720+0000","last_unstale":"2024-06-10T01:03:38.770709+0000","last_undegraded":"2024-06-10T01:03:38.770709+0000","last_fullsized":"2024-06-10T01:03:38.770709+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:35:31.342042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235489+0000","last_change":"2024-06-10T01:01:44.009278+0000","last_active":"2024-06-10T01:03:38.235489+0000","last_peered":"2024-06-10T01:03:38.235489+0000","last_clean":"2024-06-10T01:03:38.235489+0000","last_became_active":"2024-06-10T01:01:44.008841+0000","last_became_peered":"2024-06-10T01:01:44.008841+0000","last_unstale":"2024-06-10T01:03:38.235489+0000","last_undegraded":"2024-06-10T01:03:38.235489+0000","last_fullsized":"2024-06-10T01:03:38.235489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:30:58.392080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.557141+0000","last_change":"2024-06-10T01:01:47.256451+0000","last_active":"2024-06-10T01:04:35.557141+0000","last_peered":"2024-06-10T01:04:35.557141+0000","last_clean":"2024-06-10T01:04:35.557141+0000","last_became_active":"2024-06-10T01:01:47.256036+0000","last_became_peered":"2024-06-10T01:01:47.256036+0000","last_unstale":"2024-06-10T01:04:35.557141+0000","last_undegraded":"2024-06-10T01:04:35.557141+0000","last_fullsized":"2024-06-10T01:04:35.557141+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:57:26.640109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239192+0000","last_change":"2024-06-10T01:02:37.794160+0000","last_active":"2024-06-10T01:03:38.239192+0000","last_peered":"2024-06-10T01:03:38.239192+0000","last_clean":"2024-06-10T01:03:38.239192+0000","last_became_active":"2024-06-10T01:02:37.793633+0000","last_became_peered":"2024-06-10T01:02:37.793633+0000","last_unstale":"2024-06-10T01:03:38.239192+0000","last_undegraded":"2024-06-10T01:03:38.239192+0000","last_fullsized":"2024-06-10T01:03:38.239192+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:48:48.083620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270125+0000","last_change":"2024-06-10T01:01:44.469179+0000","last_active":"2024-06-10T01:03:38.270125+0000","last_peered":"2024-06-10T01:03:38.270125+0000","last_clean":"2024-06-10T01:03:38.270125+0000","last_became_active":"2024-06-10T01:01:44.468669+0000","last_became_peered":"2024-06-10T01:01:44.468669+0000","last_unstale":"2024-06-10T01:03:38.270125+0000","last_undegraded":"2024-06-10T01:03:38.270125+0000","last_fullsized":"2024-06-10T01:03:38.270125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:07:51.629030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.570053+0000","last_change":"2024-06-10T01:01:47.247588+0000","last_active":"2024-06-10T01:04:35.570053+0000","last_peered":"2024-06-10T01:04:35.570053+0000","last_clean":"2024-06-10T01:04:35.570053+0000","last_became_active":"2024-06-10T01:01:47.245750+0000","last_became_peered":"2024-06-10T01:01:47.245750+0000","last_unstale":"2024-06-10T01:04:35.570053+0000","last_undegraded":"2024-06-10T01:04:35.570053+0000","last_fullsized":"2024-06-10T01:04:35.570053+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:12:10.513777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771292+0000","last_change":"2024-06-10T01:01:57.283042+0000","last_active":"2024-06-10T01:03:38.771292+0000","last_peered":"2024-06-10T01:03:38.771292+0000","last_clean":"2024-06-10T01:03:38.771292+0000","last_became_active":"2024-06-10T01:01:57.282600+0000","last_became_peered":"2024-06-10T01:01:57.282600+0000","last_unstale":"2024-06-10T01:03:38.771292+0000","last_undegraded":"2024-06-10T01:03:38.771292+0000","last_fullsized":"2024-06-10T01:03:38.771292+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:06:36.969968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"99'39","reported_seq":425,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:39.950344+0000","last_change":"2024-06-10T01:01:44.011783+0000","last_active":"2024-06-10T01:05:39.950344+0000","last_peered":"2024-06-10T01:05:39.950344+0000","last_clean":"2024-06-10T01:05:39.950344+0000","last_became_active":"2024-06-10T01:01:44.011164+0000","last_became_peered":"2024-06-10T01:01:44.011164+0000","last_unstale":"2024-06-10T01:05:39.950344+0000","last_undegraded":"2024-06-10T01:05:39.950344+0000","last_fullsized":"2024-06-10T01:05:39.950344+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:16:13.642714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64615,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":341,"num_read_kb":234,"num_write":40,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562407+0000","last_change":"2024-06-10T01:01:47.250751+0000","last_active":"2024-06-10T01:04:35.562407+0000","last_peered":"2024-06-10T01:04:35.562407+0000","last_clean":"2024-06-10T01:04:35.562407+0000","last_became_active":"2024-06-10T01:01:47.250451+0000","last_became_peered":"2024-06-10T01:01:47.250451+0000","last_unstale":"2024-06-10T01:04:35.562407+0000","last_undegraded":"2024-06-10T01:04:35.562407+0000","last_fullsized":"2024-06-10T01:04:35.562407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:22:10.121363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270382+0000","last_change":"2024-06-10T01:01:57.289687+0000","last_active":"2024-06-10T01:03:38.270382+0000","last_peered":"2024-06-10T01:03:38.270382+0000","last_clean":"2024-06-10T01:03:38.270382+0000","last_became_active":"2024-06-10T01:01:57.289274+0000","last_became_peered":"2024-06-10T01:01:57.289274+0000","last_unstale":"2024-06-10T01:03:38.270382+0000","last_undegraded":"2024-06-10T01:03:38.270382+0000","last_fullsized":"2024-06-10T01:03:38.270382+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:34:46.873974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'6","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239529+0000","last_change":"2024-06-10T01:01:44.010159+0000","last_active":"2024-06-10T01:03:38.239529+0000","last_peered":"2024-06-10T01:03:38.239529+0000","last_clean":"2024-06-10T01:03:38.239529+0000","last_became_active":"2024-06-10T01:01:44.009744+0000","last_became_peered":"2024-06-10T01:01:44.009744+0000","last_unstale":"2024-06-10T01:03:38.239529+0000","last_undegraded":"2024-06-10T01:03:38.239529+0000","last_fullsized":"2024-06-10T01:03:38.239529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:30.507613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558994+0000","last_change":"2024-06-10T01:01:47.246662+0000","last_active":"2024-06-10T01:04:35.558994+0000","last_peered":"2024-06-10T01:04:35.558994+0000","last_clean":"2024-06-10T01:04:35.558994+0000","last_became_active":"2024-06-10T01:01:47.246268+0000","last_became_peered":"2024-06-10T01:01:47.246268+0000","last_unstale":"2024-06-10T01:04:35.558994+0000","last_undegraded":"2024-06-10T01:04:35.558994+0000","last_fullsized":"2024-06-10T01:04:35.558994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:08:38.089054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.236925+0000","last_change":"2024-06-10T01:01:57.289937+0000","last_active":"2024-06-10T01:03:38.236925+0000","last_peered":"2024-06-10T01:03:38.236925+0000","last_clean":"2024-06-10T01:03:38.236925+0000","last_became_active":"2024-06-10T01:01:57.289407+0000","last_became_peered":"2024-06-10T01:01:57.289407+0000","last_unstale":"2024-06-10T01:03:38.236925+0000","last_undegraded":"2024-06-10T01:03:38.236925+0000","last_fullsized":"2024-06-10T01:03:38.236925+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:01:48.546762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270079+0000","last_change":"2024-06-10T01:01:44.012654+0000","last_active":"2024-06-10T01:03:38.270079+0000","last_peered":"2024-06-10T01:03:38.270079+0000","last_clean":"2024-06-10T01:03:38.270079+0000","last_became_active":"2024-06-10T01:01:44.012246+0000","last_became_peered":"2024-06-10T01:01:44.012246+0000","last_unstale":"2024-06-10T01:03:38.270079+0000","last_undegraded":"2024-06-10T01:03:38.270079+0000","last_fullsized":"2024-06-10T01:03:38.270079+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:31:36.173554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558783+0000","last_change":"2024-06-10T01:01:47.250514+0000","last_active":"2024-06-10T01:04:35.558783+0000","last_peered":"2024-06-10T01:04:35.558783+0000","last_clean":"2024-06-10T01:04:35.558783+0000","last_became_active":"2024-06-10T01:01:47.249050+0000","last_became_peered":"2024-06-10T01:01:47.249050+0000","last_unstale":"2024-06-10T01:04:35.558783+0000","last_undegraded":"2024-06-10T01:04:35.558783+0000","last_fullsized":"2024-06-10T01:04:35.558783+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:51:30.789392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771928+0000","last_change":"2024-06-10T01:01:57.282389+0000","last_active":"2024-06-10T01:03:38.771928+0000","last_peered":"2024-06-10T01:03:38.771928+0000","last_clean":"2024-06-10T01:03:38.771928+0000","last_became_active":"2024-06-10T01:01:57.281857+0000","last_became_peered":"2024-06-10T01:01:57.281857+0000","last_unstale":"2024-06-10T01:03:38.771928+0000","last_undegraded":"2024-06-10T01:03:38.771928+0000","last_fullsized":"2024-06-10T01:03:38.771928+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:59:14.863020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"99'4","reported_seq":214,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:37.947518+0000","last_change":"2024-06-10T01:01:44.011331+0000","last_active":"2024-06-10T01:05:37.947518+0000","last_peered":"2024-06-10T01:05:37.947518+0000","last_clean":"2024-06-10T01:05:37.947518+0000","last_became_active":"2024-06-10T01:01:44.010968+0000","last_became_peered":"2024-06-10T01:01:44.010968+0000","last_unstale":"2024-06-10T01:05:37.947518+0000","last_undegraded":"2024-06-10T01:05:37.947518+0000","last_fullsized":"2024-06-10T01:05:37.947518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:43:37.931225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":896,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":166,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562879+0000","last_change":"2024-06-10T01:01:47.248349+0000","last_active":"2024-06-10T01:04:35.562879+0000","last_peered":"2024-06-10T01:04:35.562879+0000","last_clean":"2024-06-10T01:04:35.562879+0000","last_became_active":"2024-06-10T01:01:47.248008+0000","last_became_peered":"2024-06-10T01:01:47.248008+0000","last_unstale":"2024-06-10T01:04:35.562879+0000","last_undegraded":"2024-06-10T01:04:35.562879+0000","last_fullsized":"2024-06-10T01:04:35.562879+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:45:20.747704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771221+0000","last_change":"2024-06-10T01:01:57.289773+0000","last_active":"2024-06-10T01:03:38.771221+0000","last_peered":"2024-06-10T01:03:38.771221+0000","last_clean":"2024-06-10T01:03:38.771221+0000","last_became_active":"2024-06-10T01:01:57.289371+0000","last_became_peered":"2024-06-10T01:01:57.289371+0000","last_unstale":"2024-06-10T01:03:38.771221+0000","last_undegraded":"2024-06-10T01:03:38.771221+0000","last_fullsized":"2024-06-10T01:03:38.771221+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:24:52.209402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'15","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.257548+0000","last_change":"2024-06-10T01:03:38.257122+0000","last_active":"2024-06-10T01:03:38.257548+0000","last_peered":"2024-06-10T01:03:38.257548+0000","last_clean":"2024-06-10T01:03:38.257548+0000","last_became_active":"2024-06-10T01:03:38.240737+0000","last_became_peered":"2024-06-10T01:03:38.240737+0000","last_unstale":"2024-06-10T01:03:38.257548+0000","last_undegraded":"2024-06-10T01:03:38.257548+0000","last_fullsized":"2024-06-10T01:03:38.257548+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:37:10.925004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562425+0000","last_change":"2024-06-10T01:01:47.245439+0000","last_active":"2024-06-10T01:04:35.562425+0000","last_peered":"2024-06-10T01:04:35.562425+0000","last_clean":"2024-06-10T01:04:35.562425+0000","last_became_active":"2024-06-10T01:01:47.242917+0000","last_became_peered":"2024-06-10T01:01:47.242917+0000","last_unstale":"2024-06-10T01:04:35.562425+0000","last_undegraded":"2024-06-10T01:04:35.562425+0000","last_fullsized":"2024-06-10T01:04:35.562425+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:13:12.614230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.269795+0000","last_change":"2024-06-10T01:01:57.284731+0000","last_active":"2024-06-10T01:03:38.269795+0000","last_peered":"2024-06-10T01:03:38.269795+0000","last_clean":"2024-06-10T01:03:38.269795+0000","last_became_active":"2024-06-10T01:01:57.284301+0000","last_became_peered":"2024-06-10T01:01:57.284301+0000","last_unstale":"2024-06-10T01:03:38.269795+0000","last_undegraded":"2024-06-10T01:03:38.269795+0000","last_fullsized":"2024-06-10T01:03:38.269795+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:01:36.518214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"99'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:59.314831+0000","last_change":"2024-06-10T01:01:44.010154+0000","last_active":"2024-06-10T01:03:59.314831+0000","last_peered":"2024-06-10T01:03:59.314831+0000","last_clean":"2024-06-10T01:03:59.314831+0000","last_became_active":"2024-06-10T01:01:44.009656+0000","last_became_peered":"2024-06-10T01:01:44.009656+0000","last_unstale":"2024-06-10T01:03:59.314831+0000","last_undegraded":"2024-06-10T01:03:59.314831+0000","last_fullsized":"2024-06-10T01:03:59.314831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:33:08.789946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.550140+0000","last_change":"2024-06-10T01:02:37.378484+0000","last_active":"2024-06-10T01:04:35.550140+0000","last_peered":"2024-06-10T01:04:35.550140+0000","last_clean":"2024-06-10T01:04:35.550140+0000","last_became_active":"2024-06-10T01:02:37.378092+0000","last_became_peered":"2024-06-10T01:02:37.378092+0000","last_unstale":"2024-06-10T01:04:35.550140+0000","last_undegraded":"2024-06-10T01:04:35.550140+0000","last_fullsized":"2024-06-10T01:04:35.550140+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:19:46.599235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235375+0000","last_change":"2024-06-10T01:01:57.332887+0000","last_active":"2024-06-10T01:03:38.235375+0000","last_peered":"2024-06-10T01:03:38.235375+0000","last_clean":"2024-06-10T01:03:38.235375+0000","last_became_active":"2024-06-10T01:01:57.332285+0000","last_became_peered":"2024-06-10T01:01:57.332285+0000","last_unstale":"2024-06-10T01:03:38.235375+0000","last_undegraded":"2024-06-10T01:03:38.235375+0000","last_fullsized":"2024-06-10T01:03:38.235375+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:09:12.233405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771642+0000","last_change":"2024-06-10T01:01:44.466831+0000","last_active":"2024-06-10T01:03:38.771642+0000","last_peered":"2024-06-10T01:03:38.771642+0000","last_clean":"2024-06-10T01:03:38.771642+0000","last_became_active":"2024-06-10T01:01:44.466484+0000","last_became_peered":"2024-06-10T01:01:44.466484+0000","last_unstale":"2024-06-10T01:03:38.771642+0000","last_undegraded":"2024-06-10T01:03:38.771642+0000","last_fullsized":"2024-06-10T01:03:38.771642+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:19:28.385504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.556135+0000","last_change":"2024-06-10T01:01:47.252399+0000","last_active":"2024-06-10T01:04:35.556135+0000","last_peered":"2024-06-10T01:04:35.556135+0000","last_clean":"2024-06-10T01:04:35.556135+0000","last_became_active":"2024-06-10T01:01:47.252069+0000","last_became_peered":"2024-06-10T01:01:47.252069+0000","last_unstale":"2024-06-10T01:04:35.556135+0000","last_undegraded":"2024-06-10T01:04:35.556135+0000","last_fullsized":"2024-06-10T01:04:35.556135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:02:02.291174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771406+0000","last_change":"2024-06-10T01:01:57.285670+0000","last_active":"2024-06-10T01:03:38.771406+0000","last_peered":"2024-06-10T01:03:38.771406+0000","last_clean":"2024-06-10T01:03:38.771406+0000","last_became_active":"2024-06-10T01:01:57.285281+0000","last_became_peered":"2024-06-10T01:01:57.285281+0000","last_unstale":"2024-06-10T01:03:38.771406+0000","last_undegraded":"2024-06-10T01:03:38.771406+0000","last_fullsized":"2024-06-10T01:03:38.771406+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:58.956934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235189+0000","last_change":"2024-06-10T01:01:44.008013+0000","last_active":"2024-06-10T01:03:38.235189+0000","last_peered":"2024-06-10T01:03:38.235189+0000","last_clean":"2024-06-10T01:03:38.235189+0000","last_became_active":"2024-06-10T01:01:44.007591+0000","last_became_peered":"2024-06-10T01:01:44.007591+0000","last_unstale":"2024-06-10T01:03:38.235189+0000","last_undegraded":"2024-06-10T01:03:38.235189+0000","last_fullsized":"2024-06-10T01:03:38.235189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:18:20.862340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.550926+0000","last_change":"2024-06-10T01:01:47.254756+0000","last_active":"2024-06-10T01:04:35.550926+0000","last_peered":"2024-06-10T01:04:35.550926+0000","last_clean":"2024-06-10T01:04:35.550926+0000","last_became_active":"2024-06-10T01:01:47.254418+0000","last_became_peered":"2024-06-10T01:01:47.254418+0000","last_unstale":"2024-06-10T01:04:35.550926+0000","last_undegraded":"2024-06-10T01:04:35.550926+0000","last_fullsized":"2024-06-10T01:04:35.550926+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:46:31.452539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770995+0000","last_change":"2024-06-10T01:01:57.332792+0000","last_active":"2024-06-10T01:03:38.770995+0000","last_peered":"2024-06-10T01:03:38.770995+0000","last_clean":"2024-06-10T01:03:38.770995+0000","last_became_active":"2024-06-10T01:01:57.332331+0000","last_became_peered":"2024-06-10T01:01:57.332331+0000","last_unstale":"2024-06-10T01:03:38.770995+0000","last_undegraded":"2024-06-10T01:03:38.770995+0000","last_fullsized":"2024-06-10T01:03:38.770995+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:46:41.419807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.565658+0000","last_change":"2024-06-10T01:01:47.247857+0000","last_active":"2024-06-10T01:04:35.565658+0000","last_peered":"2024-06-10T01:04:35.565658+0000","last_clean":"2024-06-10T01:04:35.565658+0000","last_became_active":"2024-06-10T01:01:47.247552+0000","last_became_peered":"2024-06-10T01:01:47.247552+0000","last_unstale":"2024-06-10T01:04:35.565658+0000","last_undegraded":"2024-06-10T01:04:35.565658+0000","last_fullsized":"2024-06-10T01:04:35.565658+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:29:40.521887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"99'5","reported_seq":214,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:38.440204+0000","last_change":"2024-06-10T01:01:44.466631+0000","last_active":"2024-06-10T01:05:38.440204+0000","last_peered":"2024-06-10T01:05:38.440204+0000","last_clean":"2024-06-10T01:05:38.440204+0000","last_became_active":"2024-06-10T01:01:44.466280+0000","last_became_peered":"2024-06-10T01:01:44.466280+0000","last_unstale":"2024-06-10T01:05:38.440204+0000","last_undegraded":"2024-06-10T01:05:38.440204+0000","last_fullsized":"2024-06-10T01:05:38.440204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:09:33.658112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":164,"num_read_kb":164,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235365+0000","last_change":"2024-06-10T01:01:57.281973+0000","last_active":"2024-06-10T01:03:38.235365+0000","last_peered":"2024-06-10T01:03:38.235365+0000","last_clean":"2024-06-10T01:03:38.235365+0000","last_became_active":"2024-06-10T01:01:57.281386+0000","last_became_peered":"2024-06-10T01:01:57.281386+0000","last_unstale":"2024-06-10T01:03:38.235365+0000","last_undegraded":"2024-06-10T01:03:38.235365+0000","last_fullsized":"2024-06-10T01:03:38.235365+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:10:28.236250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562415+0000","last_change":"2024-06-10T01:01:47.237436+0000","last_active":"2024-06-10T01:04:35.562415+0000","last_peered":"2024-06-10T01:04:35.562415+0000","last_clean":"2024-06-10T01:04:35.562415+0000","last_became_active":"2024-06-10T01:01:47.237099+0000","last_became_peered":"2024-06-10T01:01:47.237099+0000","last_unstale":"2024-06-10T01:04:35.562415+0000","last_undegraded":"2024-06-10T01:04:35.562415+0000","last_fullsized":"2024-06-10T01:04:35.562415+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:34:53.838877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'3","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239178+0000","last_change":"2024-06-10T01:01:44.010655+0000","last_active":"2024-06-10T01:03:38.239178+0000","last_peered":"2024-06-10T01:03:38.239178+0000","last_clean":"2024-06-10T01:03:38.239178+0000","last_became_active":"2024-06-10T01:01:44.009918+0000","last_became_peered":"2024-06-10T01:01:44.009918+0000","last_unstale":"2024-06-10T01:03:38.239178+0000","last_undegraded":"2024-06-10T01:03:38.239178+0000","last_fullsized":"2024-06-10T01:03:38.239178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:47:32.052650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":367,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234906+0000","last_change":"2024-06-10T01:00:23.822447+0000","last_active":"2024-06-10T01:03:38.234906+0000","last_peered":"2024-06-10T01:03:38.234906+0000","last_clean":"2024-06-10T01:03:38.234906+0000","last_became_active":"2024-06-10T01:00:23.821966+0000","last_became_peered":"2024-06-10T01:00:23.821966+0000","last_unstale":"2024-06-10T01:03:38.234906+0000","last_undegraded":"2024-06-10T01:03:38.234906+0000","last_fullsized":"2024-06-10T01:03:38.234906+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:56:59.964661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:56:59.964661+0000","last_clean_scrub_stamp":"2024-06-10T00:56:59.964661+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:03:43.732875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270272+0000","last_change":"2024-06-10T01:01:57.332980+0000","last_active":"2024-06-10T01:03:38.270272+0000","last_peered":"2024-06-10T01:03:38.270272+0000","last_clean":"2024-06-10T01:03:38.270272+0000","last_became_active":"2024-06-10T01:01:57.332437+0000","last_became_peered":"2024-06-10T01:01:57.332437+0000","last_unstale":"2024-06-10T01:03:38.270272+0000","last_undegraded":"2024-06-10T01:03:38.270272+0000","last_fullsized":"2024-06-10T01:03:38.270272+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:27:35.029079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559754+0000","last_change":"2024-06-10T01:01:47.255926+0000","last_active":"2024-06-10T01:04:35.559754+0000","last_peered":"2024-06-10T01:04:35.559754+0000","last_clean":"2024-06-10T01:04:35.559754+0000","last_became_active":"2024-06-10T01:01:47.255571+0000","last_became_peered":"2024-06-10T01:01:47.255571+0000","last_unstale":"2024-06-10T01:04:35.559754+0000","last_undegraded":"2024-06-10T01:04:35.559754+0000","last_fullsized":"2024-06-10T01:04:35.559754+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:05:22.084037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234997+0000","last_change":"2024-06-10T01:01:44.007509+0000","last_active":"2024-06-10T01:03:38.234997+0000","last_peered":"2024-06-10T01:03:38.234997+0000","last_clean":"2024-06-10T01:03:38.234997+0000","last_became_active":"2024-06-10T01:01:44.007178+0000","last_became_peered":"2024-06-10T01:01:44.007178+0000","last_unstale":"2024-06-10T01:03:38.234997+0000","last_undegraded":"2024-06-10T01:03:38.234997+0000","last_fullsized":"2024-06-10T01:03:38.234997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:02:52.791886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.236790+0000","last_change":"2024-06-10T01:01:57.285626+0000","last_active":"2024-06-10T01:03:38.236790+0000","last_peered":"2024-06-10T01:03:38.236790+0000","last_clean":"2024-06-10T01:03:38.236790+0000","last_became_active":"2024-06-10T01:01:57.285098+0000","last_became_peered":"2024-06-10T01:01:57.285098+0000","last_unstale":"2024-06-10T01:03:38.236790+0000","last_undegraded":"2024-06-10T01:03:38.236790+0000","last_fullsized":"2024-06-10T01:03:38.236790+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:13:49.881770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235332+0000","last_change":"2024-06-10T01:01:44.469028+0000","last_active":"2024-06-10T01:03:38.235332+0000","last_peered":"2024-06-10T01:03:38.235332+0000","last_clean":"2024-06-10T01:03:38.235332+0000","last_became_active":"2024-06-10T01:01:44.468570+0000","last_became_peered":"2024-06-10T01:01:44.468570+0000","last_unstale":"2024-06-10T01:03:38.235332+0000","last_undegraded":"2024-06-10T01:03:38.235332+0000","last_fullsized":"2024-06-10T01:03:38.235332+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:12:11.287959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.560160+0000","last_change":"2024-06-10T01:01:47.253414+0000","last_active":"2024-06-10T01:04:35.560160+0000","last_peered":"2024-06-10T01:04:35.560160+0000","last_clean":"2024-06-10T01:04:35.560160+0000","last_became_active":"2024-06-10T01:01:47.253060+0000","last_became_peered":"2024-06-10T01:01:47.253060+0000","last_unstale":"2024-06-10T01:04:35.560160+0000","last_undegraded":"2024-06-10T01:04:35.560160+0000","last_fullsized":"2024-06-10T01:04:35.560160+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:24:55.534164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.238849+0000","last_change":"2024-06-10T01:01:57.332756+0000","last_active":"2024-06-10T01:03:38.238849+0000","last_peered":"2024-06-10T01:03:38.238849+0000","last_clean":"2024-06-10T01:03:38.238849+0000","last_became_active":"2024-06-10T01:01:57.332361+0000","last_became_peered":"2024-06-10T01:01:57.332361+0000","last_unstale":"2024-06-10T01:03:38.238849+0000","last_undegraded":"2024-06-10T01:03:38.238849+0000","last_fullsized":"2024-06-10T01:03:38.238849+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:02:28.275407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"99'14","reported_seq":59,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:36.265950+0000","last_change":"2024-06-10T01:01:44.005164+0000","last_active":"2024-06-10T01:04:36.265950+0000","last_peered":"2024-06-10T01:04:36.265950+0000","last_clean":"2024-06-10T01:04:36.265950+0000","last_became_active":"2024-06-10T01:01:44.004816+0000","last_became_peered":"2024-06-10T01:01:44.004816+0000","last_unstale":"2024-06-10T01:04:36.265950+0000","last_undegraded":"2024-06-10T01:04:36.265950+0000","last_fullsized":"2024-06-10T01:04:36.265950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:11:56.385267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.557511+0000","last_change":"2024-06-10T01:01:47.247951+0000","last_active":"2024-06-10T01:04:35.557511+0000","last_peered":"2024-06-10T01:04:35.557511+0000","last_clean":"2024-06-10T01:04:35.557511+0000","last_became_active":"2024-06-10T01:01:47.246996+0000","last_became_peered":"2024-06-10T01:01:47.246996+0000","last_unstale":"2024-06-10T01:04:35.557511+0000","last_undegraded":"2024-06-10T01:04:35.557511+0000","last_fullsized":"2024-06-10T01:04:35.557511+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:39:18.167021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239228+0000","last_change":"2024-06-10T01:01:57.278130+0000","last_active":"2024-06-10T01:03:38.239228+0000","last_peered":"2024-06-10T01:03:38.239228+0000","last_clean":"2024-06-10T01:03:38.239228+0000","last_became_active":"2024-06-10T01:01:57.277697+0000","last_became_peered":"2024-06-10T01:01:57.277697+0000","last_unstale":"2024-06-10T01:03:38.239228+0000","last_undegraded":"2024-06-10T01:03:38.239228+0000","last_fullsized":"2024-06-10T01:03:38.239228+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:17:01.662152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239618+0000","last_change":"2024-06-10T01:01:44.553176+0000","last_active":"2024-06-10T01:03:38.239618+0000","last_peered":"2024-06-10T01:03:38.239618+0000","last_clean":"2024-06-10T01:03:38.239618+0000","last_became_active":"2024-06-10T01:01:44.552797+0000","last_became_peered":"2024-06-10T01:01:44.552797+0000","last_unstale":"2024-06-10T01:03:38.239618+0000","last_undegraded":"2024-06-10T01:03:38.239618+0000","last_fullsized":"2024-06-10T01:03:38.239618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:21:13.960304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.556104+0000","last_change":"2024-06-10T01:01:47.255796+0000","last_active":"2024-06-10T01:04:35.556104+0000","last_peered":"2024-06-10T01:04:35.556104+0000","last_clean":"2024-06-10T01:04:35.556104+0000","last_became_active":"2024-06-10T01:01:47.255495+0000","last_became_peered":"2024-06-10T01:01:47.255495+0000","last_unstale":"2024-06-10T01:04:35.556104+0000","last_undegraded":"2024-06-10T01:04:35.556104+0000","last_fullsized":"2024-06-10T01:04:35.556104+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:06:42.209005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239491+0000","last_change":"2024-06-10T01:01:57.332869+0000","last_active":"2024-06-10T01:03:38.239491+0000","last_peered":"2024-06-10T01:03:38.239491+0000","last_clean":"2024-06-10T01:03:38.239491+0000","last_became_active":"2024-06-10T01:01:57.332391+0000","last_became_peered":"2024-06-10T01:01:57.332391+0000","last_unstale":"2024-06-10T01:03:38.239491+0000","last_undegraded":"2024-06-10T01:03:38.239491+0000","last_fullsized":"2024-06-10T01:03:38.239491+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:58:31.443626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770788+0000","last_change":"2024-06-10T01:01:44.466861+0000","last_active":"2024-06-10T01:03:38.770788+0000","last_peered":"2024-06-10T01:03:38.770788+0000","last_clean":"2024-06-10T01:03:38.770788+0000","last_became_active":"2024-06-10T01:01:44.466553+0000","last_became_peered":"2024-06-10T01:01:44.466553+0000","last_unstale":"2024-06-10T01:03:38.770788+0000","last_undegraded":"2024-06-10T01:03:38.770788+0000","last_fullsized":"2024-06-10T01:03:38.770788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:55:52.545594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":41,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.544035+0000","last_change":"2024-06-10T01:01:47.244972+0000","last_active":"2024-06-10T01:04:35.544035+0000","last_peered":"2024-06-10T01:04:35.544035+0000","last_clean":"2024-06-10T01:04:35.544035+0000","last_became_active":"2024-06-10T01:01:47.244649+0000","last_became_peered":"2024-06-10T01:01:47.244649+0000","last_unstale":"2024-06-10T01:04:35.544035+0000","last_undegraded":"2024-06-10T01:04:35.544035+0000","last_fullsized":"2024-06-10T01:04:35.544035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:10:46.909200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235461+0000","last_change":"2024-06-10T01:02:37.592380+0000","last_active":"2024-06-10T01:03:38.235461+0000","last_peered":"2024-06-10T01:03:38.235461+0000","last_clean":"2024-06-10T01:03:38.235461+0000","last_became_active":"2024-06-10T01:02:37.591856+0000","last_became_peered":"2024-06-10T01:02:37.591856+0000","last_unstale":"2024-06-10T01:03:38.235461+0000","last_undegraded":"2024-06-10T01:03:38.235461+0000","last_fullsized":"2024-06-10T01:03:38.235461+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:00:25.717192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239006+0000","last_change":"2024-06-10T01:01:44.008116+0000","last_active":"2024-06-10T01:03:38.239006+0000","last_peered":"2024-06-10T01:03:38.239006+0000","last_clean":"2024-06-10T01:03:38.239006+0000","last_became_active":"2024-06-10T01:01:44.007778+0000","last_became_peered":"2024-06-10T01:01:44.007778+0000","last_unstale":"2024-06-10T01:03:38.239006+0000","last_undegraded":"2024-06-10T01:03:38.239006+0000","last_fullsized":"2024-06-10T01:03:38.239006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:18:57.267296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.556264+0000","last_change":"2024-06-10T01:01:47.252833+0000","last_active":"2024-06-10T01:04:35.556264+0000","last_peered":"2024-06-10T01:04:35.556264+0000","last_clean":"2024-06-10T01:04:35.556264+0000","last_became_active":"2024-06-10T01:01:47.252326+0000","last_became_peered":"2024-06-10T01:01:47.252326+0000","last_unstale":"2024-06-10T01:04:35.556264+0000","last_undegraded":"2024-06-10T01:04:35.556264+0000","last_fullsized":"2024-06-10T01:04:35.556264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:00:42.673838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239284+0000","last_change":"2024-06-10T01:01:57.275975+0000","last_active":"2024-06-10T01:03:38.239284+0000","last_peered":"2024-06-10T01:03:38.239284+0000","last_clean":"2024-06-10T01:03:38.239284+0000","last_became_active":"2024-06-10T01:01:57.275408+0000","last_became_peered":"2024-06-10T01:01:57.275408+0000","last_unstale":"2024-06-10T01:03:38.239284+0000","last_undegraded":"2024-06-10T01:03:38.239284+0000","last_fullsized":"2024-06-10T01:03:38.239284+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:12:42.376636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'1","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.243874+0000","last_change":"2024-06-10T01:03:38.243422+0000","last_active":"2024-06-10T01:03:38.243874+0000","last_peered":"2024-06-10T01:03:38.243874+0000","last_clean":"2024-06-10T01:03:38.243874+0000","last_became_active":"2024-06-10T01:03:38.239916+0000","last_became_peered":"2024-06-10T01:03:38.239916+0000","last_unstale":"2024-06-10T01:03:38.243874+0000","last_undegraded":"2024-06-10T01:03:38.243874+0000","last_fullsized":"2024-06-10T01:03:38.243874+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:30:01.399956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.555295+0000","last_change":"2024-06-10T01:01:47.243154+0000","last_active":"2024-06-10T01:04:35.555295+0000","last_peered":"2024-06-10T01:04:35.555295+0000","last_clean":"2024-06-10T01:04:35.555295+0000","last_became_active":"2024-06-10T01:01:47.242851+0000","last_became_peered":"2024-06-10T01:01:47.242851+0000","last_unstale":"2024-06-10T01:04:35.555295+0000","last_undegraded":"2024-06-10T01:04:35.555295+0000","last_fullsized":"2024-06-10T01:04:35.555295+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:41:05.857777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239584+0000","last_change":"2024-06-10T01:01:57.333613+0000","last_active":"2024-06-10T01:03:38.239584+0000","last_peered":"2024-06-10T01:03:38.239584+0000","last_clean":"2024-06-10T01:03:38.239584+0000","last_became_active":"2024-06-10T01:01:57.333019+0000","last_became_peered":"2024-06-10T01:01:57.333019+0000","last_unstale":"2024-06-10T01:03:38.239584+0000","last_undegraded":"2024-06-10T01:03:38.239584+0000","last_fullsized":"2024-06-10T01:03:38.239584+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:11.912885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235303+0000","last_change":"2024-06-10T01:01:44.008221+0000","last_active":"2024-06-10T01:03:38.235303+0000","last_peered":"2024-06-10T01:03:38.235303+0000","last_clean":"2024-06-10T01:03:38.235303+0000","last_became_active":"2024-06-10T01:01:44.007880+0000","last_became_peered":"2024-06-10T01:01:44.007880+0000","last_unstale":"2024-06-10T01:03:38.235303+0000","last_undegraded":"2024-06-10T01:03:38.235303+0000","last_fullsized":"2024-06-10T01:03:38.235303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:40:33.162925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558463+0000","last_change":"2024-06-10T01:01:47.254902+0000","last_active":"2024-06-10T01:04:35.558463+0000","last_peered":"2024-06-10T01:04:35.558463+0000","last_clean":"2024-06-10T01:04:35.558463+0000","last_became_active":"2024-06-10T01:01:47.254555+0000","last_became_peered":"2024-06-10T01:01:47.254555+0000","last_unstale":"2024-06-10T01:04:35.558463+0000","last_undegraded":"2024-06-10T01:04:35.558463+0000","last_fullsized":"2024-06-10T01:04:35.558463+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:03:17.571001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770786+0000","last_change":"2024-06-10T01:01:57.284491+0000","last_active":"2024-06-10T01:03:38.770786+0000","last_peered":"2024-06-10T01:03:38.770786+0000","last_clean":"2024-06-10T01:03:38.770786+0000","last_became_active":"2024-06-10T01:01:57.283795+0000","last_became_peered":"2024-06-10T01:01:57.283795+0000","last_unstale":"2024-06-10T01:03:38.770786+0000","last_undegraded":"2024-06-10T01:03:38.770786+0000","last_fullsized":"2024-06-10T01:03:38.770786+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:49:00.381664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239889+0000","last_change":"2024-06-10T01:01:44.553159+0000","last_active":"2024-06-10T01:03:38.239889+0000","last_peered":"2024-06-10T01:03:38.239889+0000","last_clean":"2024-06-10T01:03:38.239889+0000","last_became_active":"2024-06-10T01:01:44.552759+0000","last_became_peered":"2024-06-10T01:01:44.552759+0000","last_unstale":"2024-06-10T01:03:38.239889+0000","last_undegraded":"2024-06-10T01:03:38.239889+0000","last_fullsized":"2024-06-10T01:03:38.239889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:19:00.110790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559093+0000","last_change":"2024-06-10T01:01:47.247803+0000","last_active":"2024-06-10T01:04:35.559093+0000","last_peered":"2024-06-10T01:04:35.559093+0000","last_clean":"2024-06-10T01:04:35.559093+0000","last_became_active":"2024-06-10T01:01:47.246715+0000","last_became_peered":"2024-06-10T01:01:47.246715+0000","last_unstale":"2024-06-10T01:04:35.559093+0000","last_undegraded":"2024-06-10T01:04:35.559093+0000","last_fullsized":"2024-06-10T01:04:35.559093+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:22.429002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234192+0000","last_change":"2024-06-10T01:01:57.287668+0000","last_active":"2024-06-10T01:03:38.234192+0000","last_peered":"2024-06-10T01:03:38.234192+0000","last_clean":"2024-06-10T01:03:38.234192+0000","last_became_active":"2024-06-10T01:01:57.287060+0000","last_became_peered":"2024-06-10T01:01:57.287060+0000","last_unstale":"2024-06-10T01:03:38.234192+0000","last_undegraded":"2024-06-10T01:03:38.234192+0000","last_fullsized":"2024-06-10T01:03:38.234192+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:04:35.883613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270251+0000","last_change":"2024-06-10T01:01:44.013051+0000","last_active":"2024-06-10T01:03:38.270251+0000","last_peered":"2024-06-10T01:03:38.270251+0000","last_clean":"2024-06-10T01:03:38.270251+0000","last_became_active":"2024-06-10T01:01:44.012667+0000","last_became_peered":"2024-06-10T01:01:44.012667+0000","last_unstale":"2024-06-10T01:03:38.270251+0000","last_undegraded":"2024-06-10T01:03:38.270251+0000","last_fullsized":"2024-06-10T01:03:38.270251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:10:49.650082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.563083+0000","last_change":"2024-06-10T01:01:47.251209+0000","last_active":"2024-06-10T01:04:35.563083+0000","last_peered":"2024-06-10T01:04:35.563083+0000","last_clean":"2024-06-10T01:04:35.563083+0000","last_became_active":"2024-06-10T01:01:47.250858+0000","last_became_peered":"2024-06-10T01:01:47.250858+0000","last_unstale":"2024-06-10T01:04:35.563083+0000","last_undegraded":"2024-06-10T01:04:35.563083+0000","last_fullsized":"2024-06-10T01:04:35.563083+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:47.272727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.269600+0000","last_change":"2024-06-10T01:01:57.284187+0000","last_active":"2024-06-10T01:03:38.269600+0000","last_peered":"2024-06-10T01:03:38.269600+0000","last_clean":"2024-06-10T01:03:38.269600+0000","last_became_active":"2024-06-10T01:01:57.283739+0000","last_became_peered":"2024-06-10T01:01:57.283739+0000","last_unstale":"2024-06-10T01:03:38.269600+0000","last_undegraded":"2024-06-10T01:03:38.269600+0000","last_fullsized":"2024-06-10T01:03:38.269600+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:13:34.630708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771085+0000","last_change":"2024-06-10T01:01:44.005702+0000","last_active":"2024-06-10T01:03:38.771085+0000","last_peered":"2024-06-10T01:03:38.771085+0000","last_clean":"2024-06-10T01:03:38.771085+0000","last_became_active":"2024-06-10T01:01:44.005354+0000","last_became_peered":"2024-06-10T01:01:44.005354+0000","last_unstale":"2024-06-10T01:03:38.771085+0000","last_undegraded":"2024-06-10T01:03:38.771085+0000","last_fullsized":"2024-06-10T01:03:38.771085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:51:14.819688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558942+0000","last_change":"2024-06-10T01:01:47.250325+0000","last_active":"2024-06-10T01:04:35.558942+0000","last_peered":"2024-06-10T01:04:35.558942+0000","last_clean":"2024-06-10T01:04:35.558942+0000","last_became_active":"2024-06-10T01:01:47.249591+0000","last_became_peered":"2024-06-10T01:01:47.249591+0000","last_unstale":"2024-06-10T01:04:35.558942+0000","last_undegraded":"2024-06-10T01:04:35.558942+0000","last_fullsized":"2024-06-10T01:04:35.558942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:17:16.268717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.563635+0000","last_change":"2024-06-10T01:01:47.253449+0000","last_active":"2024-06-10T01:04:35.563635+0000","last_peered":"2024-06-10T01:04:35.563635+0000","last_clean":"2024-06-10T01:04:35.563635+0000","last_became_active":"2024-06-10T01:01:47.253144+0000","last_became_peered":"2024-06-10T01:01:47.253144+0000","last_unstale":"2024-06-10T01:04:35.563635+0000","last_undegraded":"2024-06-10T01:04:35.563635+0000","last_fullsized":"2024-06-10T01:04:35.563635+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:32:22.353409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.564016+0000","last_change":"2024-06-10T01:01:47.258241+0000","last_active":"2024-06-10T01:04:35.564016+0000","last_peered":"2024-06-10T01:04:35.564016+0000","last_clean":"2024-06-10T01:04:35.564016+0000","last_became_active":"2024-06-10T01:01:47.257823+0000","last_became_peered":"2024-06-10T01:01:47.257823+0000","last_unstale":"2024-06-10T01:04:35.564016+0000","last_undegraded":"2024-06-10T01:04:35.564016+0000","last_fullsized":"2024-06-10T01:04:35.564016+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:11:23.652479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559410+0000","last_change":"2024-06-10T01:01:47.256218+0000","last_active":"2024-06-10T01:04:35.559410+0000","last_peered":"2024-06-10T01:04:35.559410+0000","last_clean":"2024-06-10T01:04:35.559410+0000","last_became_active":"2024-06-10T01:01:47.255950+0000","last_became_peered":"2024-06-10T01:01:47.255950+0000","last_unstale":"2024-06-10T01:04:35.559410+0000","last_undegraded":"2024-06-10T01:04:35.559410+0000","last_fullsized":"2024-06-10T01:04:35.559410+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:59:37.283646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.556085+0000","last_change":"2024-06-10T01:01:47.253109+0000","last_active":"2024-06-10T01:04:35.556085+0000","last_peered":"2024-06-10T01:04:35.556085+0000","last_clean":"2024-06-10T01:04:35.556085+0000","last_became_active":"2024-06-10T01:01:47.252767+0000","last_became_peered":"2024-06-10T01:01:47.252767+0000","last_unstale":"2024-06-10T01:04:35.556085+0000","last_undegraded":"2024-06-10T01:04:35.556085+0000","last_fullsized":"2024-06-10T01:04:35.556085+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:57:13.177230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.550864+0000","last_change":"2024-06-10T01:02:37.375094+0000","last_active":"2024-06-10T01:04:35.550864+0000","last_peered":"2024-06-10T01:04:35.550864+0000","last_clean":"2024-06-10T01:04:35.550864+0000","last_became_active":"2024-06-10T01:02:37.374527+0000","last_became_peered":"2024-06-10T01:02:37.374527+0000","last_unstale":"2024-06-10T01:04:35.550864+0000","last_undegraded":"2024-06-10T01:04:35.550864+0000","last_fullsized":"2024-06-10T01:04:35.550864+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:34:41.205816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.551206+0000","last_change":"2024-06-10T01:01:47.250243+0000","last_active":"2024-06-10T01:04:35.551206+0000","last_peered":"2024-06-10T01:04:35.551206+0000","last_clean":"2024-06-10T01:04:35.551206+0000","last_became_active":"2024-06-10T01:01:47.249907+0000","last_became_peered":"2024-06-10T01:01:47.249907+0000","last_unstale":"2024-06-10T01:04:35.551206+0000","last_undegraded":"2024-06-10T01:04:35.551206+0000","last_fullsized":"2024-06-10T01:04:35.551206+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:31:32.933660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.564063+0000","last_change":"2024-06-10T01:01:47.254173+0000","last_active":"2024-06-10T01:04:35.564063+0000","last_peered":"2024-06-10T01:04:35.564063+0000","last_clean":"2024-06-10T01:04:35.564063+0000","last_became_active":"2024-06-10T01:01:47.253796+0000","last_became_peered":"2024-06-10T01:01:47.253796+0000","last_unstale":"2024-06-10T01:04:35.564063+0000","last_undegraded":"2024-06-10T01:04:35.564063+0000","last_fullsized":"2024-06-10T01:04:35.564063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:41:12.199114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":41,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.548118+0000","last_change":"2024-06-10T01:01:47.253920+0000","last_active":"2024-06-10T01:04:35.548118+0000","last_peered":"2024-06-10T01:04:35.548118+0000","last_clean":"2024-06-10T01:04:35.548118+0000","last_became_active":"2024-06-10T01:01:47.253288+0000","last_became_peered":"2024-06-10T01:01:47.253288+0000","last_unstale":"2024-06-10T01:04:35.548118+0000","last_undegraded":"2024-06-10T01:04:35.548118+0000","last_fullsized":"2024-06-10T01:04:35.548118+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:41:20.406263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.564816+0000","last_change":"2024-06-10T01:01:47.240772+0000","last_active":"2024-06-10T01:04:35.564816+0000","last_peered":"2024-06-10T01:04:35.564816+0000","last_clean":"2024-06-10T01:04:35.564816+0000","last_became_active":"2024-06-10T01:01:47.238258+0000","last_became_peered":"2024-06-10T01:01:47.238258+0000","last_unstale":"2024-06-10T01:04:35.564816+0000","last_undegraded":"2024-06-10T01:04:35.564816+0000","last_fullsized":"2024-06-10T01:04:35.564816+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:55:25.211434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":24,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.544703+0000","last_change":"2024-06-10T01:02:37.376724+0000","last_active":"2024-06-10T01:04:35.544703+0000","last_peered":"2024-06-10T01:04:35.544703+0000","last_clean":"2024-06-10T01:04:35.544703+0000","last_became_active":"2024-06-10T01:02:37.376329+0000","last_became_peered":"2024-06-10T01:02:37.376329+0000","last_unstale":"2024-06-10T01:04:35.544703+0000","last_undegraded":"2024-06-10T01:04:35.544703+0000","last_fullsized":"2024-06-10T01:04:35.544703+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:38:21.837958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.565197+0000","last_change":"2024-06-10T01:01:47.244463+0000","last_active":"2024-06-10T01:04:35.565197+0000","last_peered":"2024-06-10T01:04:35.565197+0000","last_clean":"2024-06-10T01:04:35.565197+0000","last_became_active":"2024-06-10T01:01:47.244114+0000","last_became_peered":"2024-06-10T01:01:47.244114+0000","last_unstale":"2024-06-10T01:04:35.565197+0000","last_undegraded":"2024-06-10T01:04:35.565197+0000","last_fullsized":"2024-06-10T01:04:35.565197+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:47:37.098894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559211+0000","last_change":"2024-06-10T01:01:47.248317+0000","last_active":"2024-06-10T01:04:35.559211+0000","last_peered":"2024-06-10T01:04:35.559211+0000","last_clean":"2024-06-10T01:04:35.559211+0000","last_became_active":"2024-06-10T01:01:47.247958+0000","last_became_peered":"2024-06-10T01:01:47.247958+0000","last_unstale":"2024-06-10T01:04:35.559211+0000","last_undegraded":"2024-06-10T01:04:35.559211+0000","last_fullsized":"2024-06-10T01:04:35.559211+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:39:59.267957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.552389+0000","last_change":"2024-06-10T01:01:47.252542+0000","last_active":"2024-06-10T01:04:35.552389+0000","last_peered":"2024-06-10T01:04:35.552389+0000","last_clean":"2024-06-10T01:04:35.552389+0000","last_became_active":"2024-06-10T01:01:47.252216+0000","last_became_peered":"2024-06-10T01:01:47.252216+0000","last_unstale":"2024-06-10T01:04:35.552389+0000","last_undegraded":"2024-06-10T01:04:35.552389+0000","last_fullsized":"2024-06-10T01:04:35.552389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:12:22.196263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285691,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68533,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1018,"num_read_kb":581,"num_write":183,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":205599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383743,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":569843,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645380,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32388,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712764,"statfs":{"total":95995035648,"available":95961870336,"internally_reserved":0,"allocated":1241088,"data_stored":1221922,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808905,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":574063,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070542,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":569775,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332178,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":1165405,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495703,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":565143,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574960,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":629668,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738486,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":566531,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411435,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":630598,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182503,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":565337,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920865,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":569905,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757341,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":1160993,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34687,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34687,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34687,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":65853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T01:05:44.740 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:05:45.022 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:05:45.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:45 smithi007 ceph-mon[32398]: pgmap v423: 129 pgs: 129 active+clean; 644 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 511 B/s wr, 5 op/s 2024-06-10T01:05:45.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:45 smithi007 ceph-mon[32398]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T01:05:45.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:45 smithi007 ceph-mon[32398]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T01:05:45.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:45 smithi078 ceph-mon[36683]: pgmap v423: 129 pgs: 129 active+clean; 644 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 511 B/s wr, 5 op/s 2024-06-10T01:05:45.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:45 smithi078 ceph-mon[36683]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T01:05:45.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:45 smithi078 ceph-mon[36683]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T01:05:45.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:45 smithi175 ceph-mon[36636]: pgmap v423: 129 pgs: 129 active+clean; 644 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 511 B/s wr, 5 op/s 2024-06-10T01:05:45.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:45 smithi175 ceph-mon[36636]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T01:05:45.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:45 smithi175 ceph-mon[36636]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T01:05:46.427 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:46 smithi007 ceph-mon[32398]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T01:05:46.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:46 smithi078 ceph-mon[36683]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T01:05:46.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:46 smithi175 ceph-mon[36636]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T01:05:46.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.802+0000 7f477cc3f700 1 -- 172.21.15.7:0/2876875538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4778108500 msgr2=0x7f47781088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:46.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.802+0000 7f477cc3f700 1 --2- 172.21.15.7:0/2876875538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4778108500 0x7f47781088e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f476800b0a0 tx=0x7f476800b3b0 comp rx=0 tx=0).stop 2024-06-10T01:05:46.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.802+0000 7f477cc3f700 1 -- 172.21.15.7:0/2876875538 shutdown_connections 2024-06-10T01:05:46.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.802+0000 7f477cc3f700 1 --2- 172.21.15.7:0/2876875538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f477806ab30 0x7f477810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:46.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.802+0000 7f477cc3f700 1 --2- 172.21.15.7:0/2876875538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4778108500 0x7f47781088e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:46.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.802+0000 7f477cc3f700 1 --2- 172.21.15.7:0/2876875538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f477806a170 0x7f477806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:46.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.802+0000 7f477cc3f700 1 -- 172.21.15.7:0/2876875538 >> 172.21.15.7:0/2876875538 conn(0x7f4778078340 msgr2=0x7f4778078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:46.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.802+0000 7f477cc3f700 1 -- 172.21.15.7:0/2876875538 shutdown_connections 2024-06-10T01:05:46.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.802+0000 7f477cc3f700 1 -- 172.21.15.7:0/2876875538 wait complete. 2024-06-10T01:05:46.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.803+0000 7f477cc3f700 1 Processor -- start 2024-06-10T01:05:46.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.803+0000 7f477cc3f700 1 -- start start 2024-06-10T01:05:46.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.803+0000 7f477cc3f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f477806a170 0x7f4778101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:46.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.803+0000 7f477cc3f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f477806ab30 0x7f47781019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:46.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.803+0000 7f477cc3f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4778108500 0x7f4778101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:46.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.803+0000 7f477cc3f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4778112760 con 0x7f477806ab30 2024-06-10T01:05:46.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.803+0000 7f477cc3f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f47781125e0 con 0x7f4778108500 2024-06-10T01:05:46.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.803+0000 7f477cc3f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f47781128e0 con 0x7f477806a170 2024-06-10T01:05:46.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.804+0000 7f4775d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f477806ab30 0x7f47781019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:46.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.804+0000 7f4775d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f477806ab30 0x7f47781019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58614/0 (socket says 172.21.15.7:58614) 2024-06-10T01:05:46.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.804+0000 7f477659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f477806a170 0x7f4778101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:46.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.804+0000 7f4775d9b700 1 -- 172.21.15.7:0/2683165577 learned_addr learned my addr 172.21.15.7:0/2683165577 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:46.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.804+0000 7f4776d9d700 1 --2- 172.21.15.7:0/2683165577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4778108500 0x7f4778101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:46.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.804+0000 7f4775d9b700 1 -- 172.21.15.7:0/2683165577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f477806a170 msgr2=0x7f4778101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:46.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.804+0000 7f4775d9b700 1 --2- 172.21.15.7:0/2683165577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f477806a170 0x7f4778101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:46.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.804+0000 7f4775d9b700 1 -- 172.21.15.7:0/2683165577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4778108500 msgr2=0x7f4778101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:46.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.804+0000 7f4775d9b700 1 --2- 172.21.15.7:0/2683165577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4778108500 0x7f4778101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:46.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.804+0000 7f4775d9b700 1 -- 172.21.15.7:0/2683165577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47781a7110 con 0x7f477806ab30 2024-06-10T01:05:46.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.804+0000 7f4775d9b700 1 --2- 172.21.15.7:0/2683165577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f477806ab30 0x7f47781019a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f476c00d980 tx=0x7f476c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:46.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.804+0000 7f47677fe700 1 -- 172.21.15.7:0/2683165577 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c016070 con 0x7f477806ab30 2024-06-10T01:05:46.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.804+0000 7f47677fe700 1 -- 172.21.15.7:0/2683165577 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f476c004d10 con 0x7f477806ab30 2024-06-10T01:05:46.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.804+0000 7f477cc3f700 1 -- 172.21.15.7:0/2683165577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47781a7400 con 0x7f477806ab30 2024-06-10T01:05:46.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.804+0000 7f47677fe700 1 -- 172.21.15.7:0/2683165577 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c0055f0 con 0x7f477806ab30 2024-06-10T01:05:46.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.804+0000 7f477cc3f700 1 -- 172.21.15.7:0/2683165577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47781a7940 con 0x7f477806ab30 2024-06-10T01:05:46.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.805+0000 7f477cc3f700 1 -- 172.21.15.7:0/2683165577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f477810a9e0 con 0x7f477806ab30 2024-06-10T01:05:46.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.807+0000 7f47677fe700 1 -- 172.21.15.7:0/2683165577 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f476c004750 con 0x7f477806ab30 2024-06-10T01:05:46.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.807+0000 7f47677fe700 1 --2- 172.21.15.7:0/2683165577 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f47600a2c20 0x7f47600a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:46.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.807+0000 7f477659c700 1 --2- 172.21.15.7:0/2683165577 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f47600a2c20 0x7f47600a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:46.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.807+0000 7f47677fe700 1 -- 172.21.15.7:0/2683165577 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f476c0cf1f0 con 0x7f477806ab30 2024-06-10T01:05:46.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.808+0000 7f477659c700 1 --2- 172.21.15.7:0/2683165577 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f47600a2c20 0x7f47600a50e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4768009e60 tx=0x7f476800b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:46.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:46.812+0000 7f47677fe700 1 -- 172.21.15.7:0/2683165577 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f476c01f020 con 0x7f477806ab30 2024-06-10T01:05:47.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:47.065+0000 7f477cc3f700 1 -- 172.21.15.7:0/2683165577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4778102940 con 0x7f477806ab30 2024-06-10T01:05:47.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:47.066+0000 7f47677fe700 1 -- 172.21.15.7:0/2683165577 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11855 (secure 0 0 0) 0x7f476c09bdd0 con 0x7f477806ab30 2024-06-10T01:05:47.064 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:05:47.064 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:05:39.175358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/2784910673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2784910673},{"type":"v1","addr":"172.21.15.175:6835","nonce":2784910673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24434,24436,24451]},"id":1}]} 2024-06-10T01:05:47.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:47.068+0000 7f477cc3f700 1 -- 172.21.15.7:0/2683165577 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f47600a2c20 msgr2=0x7f47600a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:47.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:47.068+0000 7f477cc3f700 1 --2- 172.21.15.7:0/2683165577 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f47600a2c20 0x7f47600a50e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4768009e60 tx=0x7f476800b7b0 comp rx=0 tx=0).stop 2024-06-10T01:05:47.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:47.068+0000 7f477cc3f700 1 -- 172.21.15.7:0/2683165577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f477806ab30 msgr2=0x7f47781019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:47.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:47.068+0000 7f477cc3f700 1 --2- 172.21.15.7:0/2683165577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f477806ab30 0x7f47781019a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f476c00d980 tx=0x7f476c00dc90 comp rx=0 tx=0).stop 2024-06-10T01:05:47.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:47.068+0000 7f477cc3f700 1 -- 172.21.15.7:0/2683165577 shutdown_connections 2024-06-10T01:05:47.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:47.068+0000 7f477cc3f700 1 --2- 172.21.15.7:0/2683165577 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f47600a2c20 0x7f47600a50e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:47.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:47.068+0000 7f477cc3f700 1 --2- 172.21.15.7:0/2683165577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4778108500 0x7f4778101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:47.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:47.068+0000 7f477cc3f700 1 --2- 172.21.15.7:0/2683165577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f477806a170 0x7f4778101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:47.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:47.068+0000 7f477cc3f700 1 --2- 172.21.15.7:0/2683165577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f477806ab30 0x7f47781019a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:47.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:47.068+0000 7f477cc3f700 1 -- 172.21.15.7:0/2683165577 >> 172.21.15.7:0/2683165577 conn(0x7f4778078340 msgr2=0x7f47780ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:47.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:47.068+0000 7f477cc3f700 1 -- 172.21.15.7:0/2683165577 shutdown_connections 2024-06-10T01:05:47.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:47.068+0000 7f477cc3f700 1 -- 172.21.15.7:0/2683165577 wait complete. 2024-06-10T01:05:47.067 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-06-10T01:05:47.355 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:47 smithi007 ceph-mon[32398]: pgmap v424: 129 pgs: 129 active+clean; 644 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 6 op/s 2024-06-10T01:05:47.355 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:47 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2683165577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:05:47.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:47 smithi078 ceph-mon[36683]: pgmap v424: 129 pgs: 129 active+clean; 644 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 6 op/s 2024-06-10T01:05:47.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:47 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2683165577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:05:47.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:47 smithi175 ceph-mon[36636]: pgmap v424: 129 pgs: 129 active+clean; 644 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 6 op/s 2024-06-10T01:05:47.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:47 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2683165577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:05:47.768 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T01:05:48.018 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:05:49.418 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:49 smithi007 ceph-mon[32398]: pgmap v425: 129 pgs: 129 active+clean; 648 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 7 op/s 2024-06-10T01:05:49.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:49 smithi078 ceph-mon[36683]: pgmap v425: 129 pgs: 129 active+clean; 648 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 7 op/s 2024-06-10T01:05:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.669+0000 7f0538ff9700 1 -- 172.21.15.7:0/748027115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05341017d0 msgr2=0x7f053410f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.669+0000 7f0538ff9700 1 --2- 172.21.15.7:0/748027115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05341017d0 0x7f053410f260 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f052c00b0a0 tx=0x7f052c00b3b0 comp rx=0 tx=0).stop 2024-06-10T01:05:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.669+0000 7f0538ff9700 1 -- 172.21.15.7:0/748027115 shutdown_connections 2024-06-10T01:05:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.669+0000 7f0538ff9700 1 --2- 172.21.15.7:0/748027115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05341017d0 0x7f053410f260 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.669+0000 7f0538ff9700 1 --2- 172.21.15.7:0/748027115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0534107570 0x7f0534107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.669+0000 7f0538ff9700 1 --2- 172.21.15.7:0/748027115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0534100e30 0x7f0534101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.669+0000 7f0538ff9700 1 -- 172.21.15.7:0/748027115 >> 172.21.15.7:0/748027115 conn(0x7f0534076680 msgr2=0x7f0534076a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.669+0000 7f0538ff9700 1 -- 172.21.15.7:0/748027115 shutdown_connections 2024-06-10T01:05:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.669+0000 7f0538ff9700 1 -- 172.21.15.7:0/748027115 wait complete. 2024-06-10T01:05:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.670+0000 7f0538ff9700 1 Processor -- start 2024-06-10T01:05:49.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.671+0000 7f0538ff9700 1 -- start start 2024-06-10T01:05:49.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.671+0000 7f0538ff9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0534100e30 0x7f053419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:49.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.671+0000 7f0538ff9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05341017d0 0x7f053419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:49.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.671+0000 7f0538ff9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0534107570 0x7f05341989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.671+0000 7f0538ff9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0534112780 con 0x7f05341017d0 2024-06-10T01:05:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.671+0000 7f0538ff9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0534112600 con 0x7f0534107570 2024-06-10T01:05:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.671+0000 7f0538ff9700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0534112900 con 0x7f0534100e30 2024-06-10T01:05:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.671+0000 7f0529d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05341017d0 0x7f053419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.671+0000 7f053259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0534100e30 0x7f053419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.671+0000 7f053259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0534100e30 0x7f053419e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47164/0 (socket says 172.21.15.7:47164) 2024-06-10T01:05:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.671+0000 7f053259c700 1 -- 172.21.15.7:0/2082293124 learned_addr learned my addr 172.21.15.7:0/2082293124 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.671+0000 7f0529d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05341017d0 0x7f053419ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58624/0 (socket says 172.21.15.7:58624) 2024-06-10T01:05:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.671+0000 7f0532d9d700 1 --2- 172.21.15.7:0/2082293124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0534107570 0x7f05341989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.671+0000 7f053259c700 1 -- 172.21.15.7:0/2082293124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0534107570 msgr2=0x7f05341989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.671+0000 7f053259c700 1 --2- 172.21.15.7:0/2082293124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0534107570 0x7f05341989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.671+0000 7f053259c700 1 -- 172.21.15.7:0/2082293124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05341017d0 msgr2=0x7f053419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.671+0000 7f053259c700 1 --2- 172.21.15.7:0/2082293124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05341017d0 0x7f053419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.671+0000 7f053259c700 1 -- 172.21.15.7:0/2082293124 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0534199120 con 0x7f0534100e30 2024-06-10T01:05:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.671+0000 7f0529d9b700 1 --2- 172.21.15.7:0/2082293124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05341017d0 0x7f053419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:49.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.672+0000 7f053259c700 1 --2- 172.21.15.7:0/2082293124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0534100e30 0x7f053419e800 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f052400d980 tx=0x7f052400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:49.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.672+0000 7f052bfff700 1 -- 172.21.15.7:0/2082293124 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0524017030 con 0x7f0534100e30 2024-06-10T01:05:49.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.672+0000 7f0538ff9700 1 -- 172.21.15.7:0/2082293124 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0534199410 con 0x7f0534100e30 2024-06-10T01:05:49.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.672+0000 7f052bfff700 1 -- 172.21.15.7:0/2082293124 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0524004500 con 0x7f0534100e30 2024-06-10T01:05:49.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.672+0000 7f052bfff700 1 -- 172.21.15.7:0/2082293124 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05240039e0 con 0x7f0534100e30 2024-06-10T01:05:49.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.672+0000 7f0538ff9700 1 -- 172.21.15.7:0/2082293124 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05341a56d0 con 0x7f0534100e30 2024-06-10T01:05:49.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.673+0000 7f0538ff9700 1 -- 172.21.15.7:0/2082293124 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0534069180 con 0x7f0534100e30 2024-06-10T01:05:49.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.675+0000 7f052bfff700 1 -- 172.21.15.7:0/2082293124 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f052401a070 con 0x7f0534100e30 2024-06-10T01:05:49.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.675+0000 7f052bfff700 1 --2- 172.21.15.7:0/2082293124 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f05200a2c20 0x7f05200a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:49.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.676+0000 7f0529d9b700 1 --2- 172.21.15.7:0/2082293124 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f05200a2c20 0x7f05200a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:49.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.676+0000 7f052bfff700 1 -- 172.21.15.7:0/2082293124 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f05240cefe0 con 0x7f0534100e30 2024-06-10T01:05:49.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.676+0000 7f0529d9b700 1 --2- 172.21.15.7:0/2082293124 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f05200a2c20 0x7f05200a50e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0534076bc0 tx=0x7f051c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:49.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.680+0000 7f052bfff700 1 -- 172.21.15.7:0/2082293124 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f052401a360 con 0x7f0534100e30 2024-06-10T01:05:49.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:49 smithi175 ceph-mon[36636]: pgmap v425: 129 pgs: 129 active+clean; 648 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 7 op/s 2024-06-10T01:05:49.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.884+0000 7f0538ff9700 1 -- 172.21.15.7:0/2082293124 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f053402dca0 con 0x7f0534100e30 2024-06-10T01:05:49.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.886+0000 7f052bfff700 1 -- 172.21.15.7:0/2082293124 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24615 (secure 0 0 0) 0x7f052409bbc0 con 0x7f0534100e30 2024-06-10T01:05:49.884 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:05:49.884 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":99,"fsid":"bded58f8-26c3-11ef-bc9b-c7b262605968","created":"2024-06-10T00:53:31.966085+0000","modified":"2024-06-10T01:03:38.227703+0000","last_up_change":"2024-06-10T01:00:45.382473+0000","last_in_change":"2024-06-10T01:00:25.608513+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:56:59.964055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T01:01:42.563611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T01:01:46.175918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T01:01:55.695502+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f7c24c9f-b7ab-4c04-b2b4-120df9751667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6803","nonce":404485726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6805","nonce":404485726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6809","nonce":404485726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6807","nonce":404485726}]},"public_addr":"172.21.15.7:6803/404485726","cluster_addr":"172.21.15.7:6805/404485726","heartbeat_back_addr":"172.21.15.7:6809/404485726","heartbeat_front_addr":"172.21.15.7:6807/404485726","state":["exists","up"]},{"osd":1,"uuid":"9bf70a6b-f31c-4077-b5af-229f17bc2845","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6801","nonce":1856804448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6803","nonce":1856804448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6807","nonce":1856804448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6805","nonce":1856804448}]},"public_addr":"172.21.15.78:6801/1856804448","cluster_addr":"172.21.15.78:6803/1856804448","heartbeat_back_addr":"172.21.15.78:6807/1856804448","heartbeat_front_addr":"172.21.15.78:6805/1856804448","state":["exists","up"]},{"osd":2,"uuid":"62923c97-caf9-4f7e-9c14-9c9918843c78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6801","nonce":918257455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6803","nonce":918257455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6807","nonce":918257455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6805","nonce":918257455}]},"public_addr":"172.21.15.175:6801/918257455","cluster_addr":"172.21.15.175:6803/918257455","heartbeat_back_addr":"172.21.15.175:6807/918257455","heartbeat_front_addr":"172.21.15.175:6805/918257455","state":["exists","up"]},{"osd":3,"uuid":"65242098-e748-48ab-8f0e-bb25b5c62742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6811","nonce":1797404966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6813","nonce":1797404966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6817","nonce":1797404966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6815","nonce":1797404966}]},"public_addr":"172.21.15.7:6811/1797404966","cluster_addr":"172.21.15.7:6813/1797404966","heartbeat_back_addr":"172.21.15.7:6817/1797404966","heartbeat_front_addr":"172.21.15.7:6815/1797404966","state":["exists","up"]},{"osd":4,"uuid":"a7990305-de9e-400e-a432-d7854f73f339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6809","nonce":3767608357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6811","nonce":3767608357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6815","nonce":3767608357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6813","nonce":3767608357}]},"public_addr":"172.21.15.78:6809/3767608357","cluster_addr":"172.21.15.78:6811/3767608357","heartbeat_back_addr":"172.21.15.78:6815/3767608357","heartbeat_front_addr":"172.21.15.78:6813/3767608357","state":["exists","up"]},{"osd":5,"uuid":"bad9a7b3-da5a-48d0-905d-e6a3190317fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6809","nonce":2157664731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6811","nonce":2157664731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6815","nonce":2157664731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6813","nonce":2157664731}]},"public_addr":"172.21.15.175:6809/2157664731","cluster_addr":"172.21.15.175:6811/2157664731","heartbeat_back_addr":"172.21.15.175:6815/2157664731","heartbeat_front_addr":"172.21.15.175:6813/2157664731","state":["exists","up"]},{"osd":6,"uuid":"6c687653-157b-4b96-867b-a2eaa7d8ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6819","nonce":838731756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6821","nonce":838731756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6825","nonce":838731756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6823","nonce":838731756}]},"public_addr":"172.21.15.7:6819/838731756","cluster_addr":"172.21.15.7:6821/838731756","heartbeat_back_addr":"172.21.15.7:6825/838731756","heartbeat_front_addr":"172.21.15.7:6823/838731756","state":["exists","up"]},{"osd":7,"uuid":"db40ddf1-9fd9-4859-b8c4-75f79582ab91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6817","nonce":3511921773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6819","nonce":3511921773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6823","nonce":3511921773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6821","nonce":3511921773}]},"public_addr":"172.21.15.78:6817/3511921773","cluster_addr":"172.21.15.78:6819/3511921773","heartbeat_back_addr":"172.21.15.78:6823/3511921773","heartbeat_front_addr":"172.21.15.78:6821/3511921773","state":["exists","up"]},{"osd":8,"uuid":"0f01cbd9-b193-4c70-a33c-4984167bf422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6817","nonce":1546325211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6819","nonce":1546325211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6823","nonce":1546325211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6821","nonce":1546325211}]},"public_addr":"172.21.15.175:6817/1546325211","cluster_addr":"172.21.15.175:6819/1546325211","heartbeat_back_addr":"172.21.15.175:6823/1546325211","heartbeat_front_addr":"172.21.15.175:6821/1546325211","state":["exists","up"]},{"osd":9,"uuid":"e5480998-f7fb-49cd-9aed-52405c88d830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6827","nonce":1844550813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6829","nonce":1844550813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6833","nonce":1844550813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6831","nonce":1844550813}]},"public_addr":"172.21.15.7:6827/1844550813","cluster_addr":"172.21.15.7:6829/1844550813","heartbeat_back_addr":"172.21.15.7:6833/1844550813","heartbeat_front_addr":"172.21.15.7:6831/1844550813","state":["exists","up"]},{"osd":10,"uuid":"caa82f33-a71c-4907-bf3b-30a7fdfeb71d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6825","nonce":2290824785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6827","nonce":2290824785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6831","nonce":2290824785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6829","nonce":2290824785}]},"public_addr":"172.21.15.78:6825/2290824785","cluster_addr":"172.21.15.78:6827/2290824785","heartbeat_back_addr":"172.21.15.78:6831/2290824785","heartbeat_front_addr":"172.21.15.78:6829/2290824785","state":["exists","up"]},{"osd":11,"uuid":"608cda77-77da-4da2-a7e2-76a773be27c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6825","nonce":1959979550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6827","nonce":1959979550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6831","nonce":1959979550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6829","nonce":1959979550}]},"public_addr":"172.21.15.175:6825/1959979550","cluster_addr":"172.21.15.175:6827/1959979550","heartbeat_back_addr":"172.21.15.175:6831/1959979550","heartbeat_front_addr":"172.21.15.175:6829/1959979550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:08.850203+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:32.820161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:56.633656+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:21.759372+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:46.735874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:10.636541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:36.990099+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:01.767098+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:26.882776+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:52.757595+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:18.383938+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:43.544330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1833397470":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/322543993":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/212510583":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6801/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6800/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/2679436843":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6801/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/1210355277":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6800/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/3610017406":"2024-06-11T00:54:08.102405+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T01:05:49.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.888+0000 7f0538ff9700 1 -- 172.21.15.7:0/2082293124 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f05200a2c20 msgr2=0x7f05200a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:49.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.888+0000 7f0538ff9700 1 --2- 172.21.15.7:0/2082293124 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f05200a2c20 0x7f05200a50e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0534076bc0 tx=0x7f051c009260 comp rx=0 tx=0).stop 2024-06-10T01:05:49.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.888+0000 7f0538ff9700 1 -- 172.21.15.7:0/2082293124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0534100e30 msgr2=0x7f053419e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:49.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.888+0000 7f0538ff9700 1 --2- 172.21.15.7:0/2082293124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0534100e30 0x7f053419e800 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f052400d980 tx=0x7f052400dc90 comp rx=0 tx=0).stop 2024-06-10T01:05:49.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.888+0000 7f0538ff9700 1 -- 172.21.15.7:0/2082293124 shutdown_connections 2024-06-10T01:05:49.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.888+0000 7f0538ff9700 1 --2- 172.21.15.7:0/2082293124 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f05200a2c20 0x7f05200a50e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:49.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.888+0000 7f0538ff9700 1 --2- 172.21.15.7:0/2082293124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0534107570 0x7f05341989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:49.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.888+0000 7f0538ff9700 1 --2- 172.21.15.7:0/2082293124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0534100e30 0x7f053419e800 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:49.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.889+0000 7f0538ff9700 1 --2- 172.21.15.7:0/2082293124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05341017d0 0x7f053419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:49.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.889+0000 7f0538ff9700 1 -- 172.21.15.7:0/2082293124 >> 172.21.15.7:0/2082293124 conn(0x7f0534076680 msgr2=0x7f05340ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:49.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.889+0000 7f0538ff9700 1 -- 172.21.15.7:0/2082293124 shutdown_connections 2024-06-10T01:05:49.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:49.889+0000 7f0538ff9700 1 -- 172.21.15.7:0/2082293124 wait complete. 2024-06-10T01:05:50.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:50 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2082293124' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:05:50.608 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:50 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2082293124' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:05:50.638 DEBUG:tasks.fwd_scrub:Yielding 2024-06-10T01:05:50.638 INFO:teuthology.run_tasks:Running task background_exec... 2024-06-10T01:05:50.649 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-06-10T01:05:50.649 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-06-10T01:05:50.649 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-06-10T01:05:50.649 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:05:50.649 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:05:50.652 INFO:teuthology.run_tasks:Running task quiescer... 2024-06-10T01:05:50.664 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-10T01:05:50.664 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-10T01:05:50.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:50 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2082293124' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:05:50.908 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:05:50.920 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:05:51.387 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:51 smithi007 ceph-mon[32398]: pgmap v426: 129 pgs: 129 active+clean; 648 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 6 op/s 2024-06-10T01:05:51.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:51 smithi078 ceph-mon[36683]: pgmap v426: 129 pgs: 129 active+clean; 648 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 6 op/s 2024-06-10T01:05:51.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:51 smithi175 ceph-mon[36636]: pgmap v426: 129 pgs: 129 active+clean; 648 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 6 op/s 2024-06-10T01:05:53.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:53 smithi078 ceph-mon[36683]: pgmap v427: 129 pgs: 129 active+clean; 648 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-06-10T01:05:53.605 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:53 smithi007 ceph-mon[32398]: pgmap v427: 129 pgs: 129 active+clean; 648 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-06-10T01:05:53.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:53 smithi175 ceph-mon[36636]: pgmap v427: 129 pgs: 129 active+clean; 648 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-06-10T01:05:54.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.078+0000 7fb9c7301700 1 -- 172.21.15.7:0/2259226047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9c006ac00 msgr2=0x7fb9c006afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:54.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.078+0000 7fb9c7301700 1 --2- 172.21.15.7:0/2259226047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9c006ac00 0x7fb9c006afe0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc009990 tx=0x7fb9bc009ca0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.078+0000 7fb9c7301700 1 -- 172.21.15.7:0/2259226047 shutdown_connections 2024-06-10T01:05:54.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.078+0000 7fb9c7301700 1 --2- 172.21.15.7:0/2259226047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9c010efd0 0x7fb9c01113c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.078+0000 7fb9c7301700 1 --2- 172.21.15.7:0/2259226047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9c006ac00 0x7fb9c006afe0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.078+0000 7fb9c7301700 1 --2- 172.21.15.7:0/2259226047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9c006b520 0x7fb9c010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.078+0000 7fb9c7301700 1 -- 172.21.15.7:0/2259226047 >> 172.21.15.7:0/2259226047 conn(0x7fb9c00773e0 msgr2=0x7fb9c00777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:54.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.078+0000 7fb9c7301700 1 -- 172.21.15.7:0/2259226047 shutdown_connections 2024-06-10T01:05:54.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.078+0000 7fb9c7301700 1 -- 172.21.15.7:0/2259226047 wait complete. 2024-06-10T01:05:54.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.079+0000 7fb9c7301700 1 Processor -- start 2024-06-10T01:05:54.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.079+0000 7fb9c7301700 1 -- start start 2024-06-10T01:05:54.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.079+0000 7fb9c7301700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9c006ac00 0x7fb9c019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:54.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.079+0000 7fb9c7301700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9c006b520 0x7fb9c019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:54.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.079+0000 7fb9c7301700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9c010efd0 0x7fb9c019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:54.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.079+0000 7fb9c7301700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c0114a20 con 0x7fb9c010efd0 2024-06-10T01:05:54.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.079+0000 7fb9c7301700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c01148a0 con 0x7fb9c006b520 2024-06-10T01:05:54.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.079+0000 7fb9c7301700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c0114ba0 con 0x7fb9c006ac00 2024-06-10T01:05:54.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.079+0000 7fb9c589e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9c010efd0 0x7fb9c019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:54.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.079+0000 7fb9c589e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9c010efd0 0x7fb9c019f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39972/0 (socket says 172.21.15.7:39972) 2024-06-10T01:05:54.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.079+0000 7fb9c589e700 1 -- 172.21.15.7:0/1858881229 learned_addr learned my addr 172.21.15.7:0/1858881229 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:54.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.079+0000 7fb9c489c700 1 --2- 172.21.15.7:0/1858881229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9c006b520 0x7fb9c019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:54.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.079+0000 7fb9c509d700 1 --2- 172.21.15.7:0/1858881229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9c006ac00 0x7fb9c019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:54.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.079+0000 7fb9c589e700 1 -- 172.21.15.7:0/1858881229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9c006ac00 msgr2=0x7fb9c019ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:54.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.079+0000 7fb9c589e700 1 --2- 172.21.15.7:0/1858881229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9c006ac00 0x7fb9c019ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.079+0000 7fb9c589e700 1 -- 172.21.15.7:0/1858881229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9c006b520 msgr2=0x7fb9c019b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:54.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.079+0000 7fb9c589e700 1 --2- 172.21.15.7:0/1858881229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9c006b520 0x7fb9c019b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.079+0000 7fb9c589e700 1 -- 172.21.15.7:0/1858881229 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c019fdd0 con 0x7fb9c010efd0 2024-06-10T01:05:54.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.080+0000 7fb9c589e700 1 --2- 172.21.15.7:0/1858881229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9c010efd0 0x7fb9c019f690 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb9b80094f0 tx=0x7fb9b800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:54.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.080+0000 7fb9b67fc700 1 -- 172.21.15.7:0/1858881229 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b800ddd0 con 0x7fb9c010efd0 2024-06-10T01:05:54.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.080+0000 7fb9b67fc700 1 -- 172.21.15.7:0/1858881229 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9b8009e20 con 0x7fb9c010efd0 2024-06-10T01:05:54.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.080+0000 7fb9c7301700 1 -- 172.21.15.7:0/1858881229 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9c01a00c0 con 0x7fb9c010efd0 2024-06-10T01:05:54.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.080+0000 7fb9b67fc700 1 -- 172.21.15.7:0/1858881229 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8015660 con 0x7fb9c010efd0 2024-06-10T01:05:54.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.080+0000 7fb9c7301700 1 -- 172.21.15.7:0/1858881229 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9c01a7940 con 0x7fb9c010efd0 2024-06-10T01:05:54.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.081+0000 7fb9c7301700 1 -- 172.21.15.7:0/1858881229 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9c0069180 con 0x7fb9c010efd0 2024-06-10T01:05:54.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.083+0000 7fb9b67fc700 1 -- 172.21.15.7:0/1858881229 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb9b8004ad0 con 0x7fb9c010efd0 2024-06-10T01:05:54.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.084+0000 7fb9b67fc700 1 --2- 172.21.15.7:0/1858881229 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb9ac0a2c20 0x7fb9ac0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:54.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.084+0000 7fb9c509d700 1 --2- 172.21.15.7:0/1858881229 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb9ac0a2c20 0x7fb9ac0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:54.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.084+0000 7fb9b67fc700 1 -- 172.21.15.7:0/1858881229 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fb9b80cf610 con 0x7fb9c010efd0 2024-06-10T01:05:54.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.084+0000 7fb9c509d700 1 --2- 172.21.15.7:0/1858881229 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb9ac0a2c20 0x7fb9ac0a50e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc006010 tx=0x7fb9bc00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:54.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.087+0000 7fb9b67fc700 1 -- 172.21.15.7:0/1858881229 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb9b8010070 con 0x7fb9c010efd0 2024-06-10T01:05:54.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.259+0000 7ff6dc48d700 1 -- 172.21.15.7:0/3207561813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d4100e30 msgr2=0x7ff6d4101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:54.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.259+0000 7ff6dc48d700 1 --2- 172.21.15.7:0/3207561813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d4100e30 0x7ff6d4101290 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4009990 tx=0x7ff6c4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.259+0000 7ff6dc48d700 1 -- 172.21.15.7:0/3207561813 shutdown_connections 2024-06-10T01:05:54.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.259+0000 7ff6dc48d700 1 --2- 172.21.15.7:0/3207561813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6d41017d0 0x7ff6d410f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.259+0000 7ff6dc48d700 1 --2- 172.21.15.7:0/3207561813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6d4107570 0x7ff6d4107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.259+0000 7ff6dc48d700 1 --2- 172.21.15.7:0/3207561813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d4100e30 0x7ff6d4101290 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.259+0000 7ff6dc48d700 1 -- 172.21.15.7:0/3207561813 >> 172.21.15.7:0/3207561813 conn(0x7ff6d4076680 msgr2=0x7ff6d4076a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:54.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.259+0000 7ff6dc48d700 1 -- 172.21.15.7:0/3207561813 shutdown_connections 2024-06-10T01:05:54.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.259+0000 7ff6dc48d700 1 -- 172.21.15.7:0/3207561813 wait complete. 2024-06-10T01:05:54.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.260+0000 7ff6dc48d700 1 Processor -- start 2024-06-10T01:05:54.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.260+0000 7ff6dc48d700 1 -- start start 2024-06-10T01:05:54.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.260+0000 7ff6dc48d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6d4100e30 0x7ff6d419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:54.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.260+0000 7ff6dc48d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6d41017d0 0x7ff6d419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:54.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.260+0000 7ff6dc48d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d4107570 0x7ff6d4198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:54.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.260+0000 7ff6dc48d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d4112760 con 0x7ff6d4107570 2024-06-10T01:05:54.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.260+0000 7ff6dc48d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d41125e0 con 0x7ff6d41017d0 2024-06-10T01:05:54.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.260+0000 7ff6dc48d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d41128e0 con 0x7ff6d4100e30 2024-06-10T01:05:54.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.260+0000 7ff6daa2a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d4107570 0x7ff6d4198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:54.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.260+0000 7ff6daa2a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d4107570 0x7ff6d4198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39996/0 (socket says 172.21.15.7:39996) 2024-06-10T01:05:54.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.260+0000 7ff6daa2a700 1 -- 172.21.15.7:0/2469452940 learned_addr learned my addr 172.21.15.7:0/2469452940 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:54.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.260+0000 7ff6da229700 1 --2- 172.21.15.7:0/2469452940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6d4100e30 0x7ff6d419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:54.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.261+0000 7ff6d9a28700 1 --2- 172.21.15.7:0/2469452940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6d41017d0 0x7ff6d419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:54.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.261+0000 7ff6daa2a700 1 -- 172.21.15.7:0/2469452940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6d4100e30 msgr2=0x7ff6d419e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:54.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.261+0000 7ff6daa2a700 1 --2- 172.21.15.7:0/2469452940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6d4100e30 0x7ff6d419e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.261+0000 7ff6daa2a700 1 -- 172.21.15.7:0/2469452940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6d41017d0 msgr2=0x7ff6d419ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:54.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.261+0000 7ff6daa2a700 1 --2- 172.21.15.7:0/2469452940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6d41017d0 0x7ff6d419ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.261+0000 7ff6daa2a700 1 -- 172.21.15.7:0/2469452940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d4199200 con 0x7ff6d4107570 2024-06-10T01:05:54.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.261+0000 7ff6da229700 1 --2- 172.21.15.7:0/2469452940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6d4100e30 0x7ff6d419e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:54.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.261+0000 7ff6daa2a700 1 --2- 172.21.15.7:0/2469452940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d4107570 0x7ff6d4198aa0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc007ec0 tx=0x7ff6cc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:54.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.261+0000 7ff6cb7fe700 1 -- 172.21.15.7:0/2469452940 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc00dd20 con 0x7ff6d4107570 2024-06-10T01:05:54.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.261+0000 7ff6cb7fe700 1 -- 172.21.15.7:0/2469452940 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6cc018070 con 0x7ff6d4107570 2024-06-10T01:05:54.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.261+0000 7ff6dc48d700 1 -- 172.21.15.7:0/2469452940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6d41994f0 con 0x7ff6d4107570 2024-06-10T01:05:54.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.262+0000 7ff6cb7fe700 1 -- 172.21.15.7:0/2469452940 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc012690 con 0x7ff6d4107570 2024-06-10T01:05:54.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.262+0000 7ff6dc48d700 1 -- 172.21.15.7:0/2469452940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6d41a57b0 con 0x7ff6d4107570 2024-06-10T01:05:54.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.264+0000 7ff6cb7fe700 1 -- 172.21.15.7:0/2469452940 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7ff6cc012830 con 0x7ff6d4107570 2024-06-10T01:05:54.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.264+0000 7ff6cb7fe700 1 --2- 172.21.15.7:0/2469452940 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff6c00a2b80 0x7ff6c00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:54.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.264+0000 7ff6cb7fe700 1 -- 172.21.15.7:0/2469452940 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7ff6cc0ce8d0 con 0x7ff6d4107570 2024-06-10T01:05:54.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.264+0000 7ff6da229700 1 --2- 172.21.15.7:0/2469452940 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff6c00a2b80 0x7ff6c00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.265+0000 7ff6da229700 1 --2- 172.21.15.7:0/2469452940 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff6c00a2b80 0x7ff6c00a5040 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4199be0 tx=0x7ff6d000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.265+0000 7ff6e6d10b80 1 Processor -- start 2024-06-10T01:05:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.265+0000 7ff6e6d10b80 1 -- start start 2024-06-10T01:05:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.266+0000 7ff6e6d10b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8e16138d0 0x55d8e1613cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.266+0000 7ff6e6d10b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8e16141f0 0x55d8e1618090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.266+0000 7ff6e6d10b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8e16185d0 0x55d8e1618a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.266+0000 7ff6e6d10b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d8e1501fe0 con 0x55d8e16138d0 2024-06-10T01:05:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.266+0000 7ff6e6d10b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d8e15608d0 con 0x55d8e16185d0 2024-06-10T01:05:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.266+0000 7ff6e6d10b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d8e14c84e0 con 0x55d8e16141f0 2024-06-10T01:05:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.266+0000 7ff6da229700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8e16138d0 0x55d8e1613cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.266+0000 7ff6da229700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8e16138d0 0x55d8e1613cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39998/0 (socket says 172.21.15.7:39998) 2024-06-10T01:05:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.266+0000 7ff6da229700 1 -- 172.21.15.7:0/4262652980 learned_addr learned my addr 172.21.15.7:0/4262652980 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.266+0000 7ff6d9a28700 1 --2- 172.21.15.7:0/4262652980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8e16141f0 0x55d8e1618090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.266+0000 7ff6daa2a700 1 --2- 172.21.15.7:0/4262652980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8e16185d0 0x55d8e1618a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.266+0000 7ff6da229700 1 -- 172.21.15.7:0/4262652980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8e16141f0 msgr2=0x55d8e1618090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.266+0000 7ff6da229700 1 --2- 172.21.15.7:0/4262652980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8e16141f0 0x55d8e1618090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.266+0000 7ff6da229700 1 -- 172.21.15.7:0/4262652980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8e16185d0 msgr2=0x55d8e1618a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.266+0000 7ff6da229700 1 --2- 172.21.15.7:0/4262652980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8e16185d0 0x55d8e1618a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.266+0000 7ff6da229700 1 -- 172.21.15.7:0/4262652980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8e14af950 con 0x55d8e16138d0 2024-06-10T01:05:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.266+0000 7ff6daa2a700 1 --2- 172.21.15.7:0/4262652980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8e16185d0 0x55d8e1618a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:05:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.267+0000 7ff6da229700 1 --2- 172.21.15.7:0/4262652980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8e16138d0 0x55d8e1613cb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0011760 tx=0x7ff6d0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50bb6d443f18fcb6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.267+0000 7ff6c8d70700 1 -- 172.21.15.7:0/4262652980 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d001e420 con 0x55d8e16138d0 2024-06-10T01:05:54.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.267+0000 7ff6c8d70700 1 -- 172.21.15.7:0/4262652980 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6d001eaa0 con 0x55d8e16138d0 2024-06-10T01:05:54.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.267+0000 7ff6c8d70700 1 -- 172.21.15.7:0/4262652980 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0026c10 con 0x55d8e16138d0 2024-06-10T01:05:54.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.267+0000 7ff6e6d10b80 1 -- 172.21.15.7:0/4262652980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8e16138d0 msgr2=0x55d8e1613cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.267+0000 7ff6e6d10b80 1 --2- 172.21.15.7:0/4262652980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8e16138d0 0x55d8e1613cb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0011760 tx=0x7ff6d0013000 comp rx=0 tx=0).stop 2024-06-10T01:05:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.268+0000 7ff6e6d10b80 1 -- 172.21.15.7:0/4262652980 shutdown_connections 2024-06-10T01:05:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.268+0000 7ff6e6d10b80 1 --2- 172.21.15.7:0/4262652980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8e16185d0 0x55d8e1618a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.268+0000 7ff6e6d10b80 1 --2- 172.21.15.7:0/4262652980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8e16138d0 0x55d8e1613cb0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.268+0000 7ff6e6d10b80 1 --2- 172.21.15.7:0/4262652980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8e16141f0 0x55d8e1618090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.268+0000 7ff6e6d10b80 1 -- 172.21.15.7:0/4262652980 >> 172.21.15.7:0/4262652980 conn(0x55d8e153c100 msgr2=0x55d8e160a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.268+0000 7ff6e6d10b80 1 -- 172.21.15.7:0/4262652980 shutdown_connections 2024-06-10T01:05:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.268+0000 7ff6e6d10b80 1 -- 172.21.15.7:0/4262652980 wait complete. 2024-06-10T01:05:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.268+0000 7ff6e6d10b80 1 -- start start 2024-06-10T01:05:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.268+0000 7ff6e6d10b80 1 Processor -- start 2024-06-10T01:05:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.268+0000 7ff6e6d10b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8e16138d0 0x55d8e161fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.268+0000 7ff6e6d10b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8e16141f0 0x55d8e161ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.268+0000 7ff6e6d10b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8e16185d0 0x55d8e161d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.269+0000 7ff6e6d10b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d8e1501040 con 0x55d8e16138d0 2024-06-10T01:05:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.269+0000 7ff6e6d10b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d8e155c890 con 0x55d8e16141f0 2024-06-10T01:05:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.269+0000 7ff6e6d10b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d8e1620700 con 0x55d8e16185d0 2024-06-10T01:05:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.269+0000 7ff6e6d10b80 10 client.0 _ll_register_callbacks cb 0x55d8e14f6630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:05:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.269+0000 7ff6da229700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8e16138d0 0x55d8e161fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.269+0000 7ff6da229700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8e16138d0 0x55d8e161fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40012/0 (socket says 172.21.15.7:40012) 2024-06-10T01:05:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.269+0000 7ff6da229700 1 -- 172.21.15.7:0/148780399 learned_addr learned my addr 172.21.15.7:0/148780399 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.269+0000 7ff6d9a28700 1 --2- 172.21.15.7:0/148780399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8e16141f0 0x55d8e161ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.269+0000 7ff6daa2a700 1 --2- 172.21.15.7:0/148780399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8e16185d0 0x55d8e161d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.269+0000 7ff6da229700 1 -- 172.21.15.7:0/148780399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8e16185d0 msgr2=0x55d8e161d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.269+0000 7ff6da229700 1 --2- 172.21.15.7:0/148780399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8e16185d0 0x55d8e161d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.269+0000 7ff6da229700 1 -- 172.21.15.7:0/148780399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8e16141f0 msgr2=0x55d8e161ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.269+0000 7ff6da229700 1 --2- 172.21.15.7:0/148780399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8e16141f0 0x55d8e161ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.269+0000 7ff6da229700 1 -- 172.21.15.7:0/148780399 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8e15267d0 con 0x55d8e16138d0 2024-06-10T01:05:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.269+0000 7ff6da229700 1 --2- 172.21.15.7:0/148780399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8e16138d0 0x55d8e161fbb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000b650 tx=0x7ff6d00266a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cfbac1b09d4705 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.269+0000 7ff6bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:05:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.269+0000 7ff6bdffb700 1 -- 172.21.15.7:0/148780399 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d001ec50 con 0x55d8e16138d0 2024-06-10T01:05:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.270+0000 7ff6bdffb700 1 -- 172.21.15.7:0/148780399 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7ff6d001e420 con 0x55d8e16138d0 2024-06-10T01:05:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.270+0000 7ff6e6d10b80 1 -- 172.21.15.7:0/148780399 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6d001eaa0 con 0x55d8e16138d0 2024-06-10T01:05:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.270+0000 7ff6bdffb700 1 -- 172.21.15.7:0/148780399 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff6d001e620 con 0x55d8e16138d0 2024-06-10T01:05:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.270+0000 7ff6e6d10b80 10 client.15294 fetch_fsmap learned FSMap version 27 2024-06-10T01:05:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.270+0000 7ff6e6d10b80 1 -- 172.21.15.7:0/148780399 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d8e162ae80 con 0x55d8e16138d0 2024-06-10T01:05:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.270+0000 7ff6bdffb700 1 -- 172.21.15.7:0/148780399 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6d001eaa0 con 0x55d8e16138d0 2024-06-10T01:05:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.270+0000 7ff6bdffb700 1 -- 172.21.15.7:0/148780399 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff6d0026ac0 con 0x55d8e16138d0 2024-06-10T01:05:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.270+0000 7ff6e6d10b80 10 client.15294 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:05:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.270+0000 7ff6e6d10b80 10 client.15294 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:05:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.270+0000 7ff6e6d10b80 20 client.15294 populate_metadata read hostname 'smithi007' 2024-06-10T01:05:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.270+0000 7ff6e6d10b80 1 --2- 172.21.15.7:0/148780399 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d8e162d400 0x55d8e162f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.270+0000 7ff6e6d10b80 4 client.15294 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:05:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.270+0000 7ff6e6d10b80 1 -- 172.21.15.7:0/148780399 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8e156ddd0 con 0x55d8e162d400 2024-06-10T01:05:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.271+0000 7ff6d9a28700 1 --2- 172.21.15.7:0/148780399 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d8e162d400 0x55d8e162f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.271+0000 7ff6bdffb700 10 client.15294 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:05:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.271+0000 7ff6d9a28700 1 --2- 172.21.15.7:0/148780399 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d8e162d400 0x55d8e162f7f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58aa9bc2ef0730ea server_cookie=f3c6597df725249c in_seq=0 out_seq=0 2024-06-10T01:05:54.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.272+0000 7ff6bdffb700 1 -- 172.21.15.7:0/148780399 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d8e156ddd0 con 0x55d8e162d400 2024-06-10T01:05:54.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.272+0000 7ff6bdffb700 10 client.15294 handle_command_reply: tid=0 2024-06-10T01:05:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.272+0000 7ff6e6d10b80 1 client.15294 shutdown 2024-06-10T01:05:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.272+0000 7ff6e6d10b80 1 -- 172.21.15.7:0/148780399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8e16138d0 msgr2=0x55d8e161fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.272+0000 7ff6e6d10b80 1 --2- 172.21.15.7:0/148780399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8e16138d0 0x55d8e161fbb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000b650 tx=0x7ff6d00266a0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.272+0000 7ff6e6d10b80 1 -- 172.21.15.7:0/148780399 shutdown_connections 2024-06-10T01:05:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.272+0000 7ff6e6d10b80 1 --2- 172.21.15.7:0/148780399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8e16185d0 0x55d8e161d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.272+0000 7ff6e6d10b80 1 --2- 172.21.15.7:0/148780399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8e16138d0 0x55d8e161fbb0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.272+0000 7ff6e6d10b80 1 --2- 172.21.15.7:0/148780399 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d8e162d400 0x55d8e162f7f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.272+0000 7ff6bdffb700 0 client.15294 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:05:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.272+0000 7ff6e6d10b80 1 --2- 172.21.15.7:0/148780399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8e16141f0 0x55d8e161ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.272+0000 7ff6e6d10b80 1 -- 172.21.15.7:0/148780399 >> 172.21.15.7:0/148780399 conn(0x55d8e153c100 msgr2=0x55d8e1619aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.272+0000 7ff6e6d10b80 1 -- 172.21.15.7:0/148780399 shutdown_connections 2024-06-10T01:05:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.272+0000 7ff6e6d10b80 1 -- 172.21.15.7:0/148780399 wait complete. 2024-06-10T01:05:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.272+0000 7ff6e6d10b80 20 client.15294 trim_cache size 0 max 16384 2024-06-10T01:05:54.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.293+0000 7ff6e6d10b80 1 Processor -- start 2024-06-10T01:05:54.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.293+0000 7ff6e6d10b80 1 -- start start 2024-06-10T01:05:54.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.294+0000 7ff6e6d10b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8e16138d0 0x55d8e1629820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:54.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.294+0000 7ff6e6d10b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8e16141f0 0x55d8e16199c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:54.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.294+0000 7ff6e6d10b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8e16185d0 0x55d8e161f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:54.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.294+0000 7ff6e6d10b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d41ae7c0 con 0x55d8e16141f0 2024-06-10T01:05:54.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.294+0000 7ff6e6d10b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d41ac050 con 0x55d8e16138d0 2024-06-10T01:05:54.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.294+0000 7ff6e6d10b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d41a9ea0 con 0x55d8e16185d0 2024-06-10T01:05:54.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.294+0000 7ff6d9a28700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8e16141f0 0x55d8e16199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:54.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.294+0000 7ff6d9a28700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8e16141f0 0x55d8e16199c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40020/0 (socket says 172.21.15.7:40020) 2024-06-10T01:05:54.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.294+0000 7ff6d9a28700 1 -- 172.21.15.7:0/3365311492 learned_addr learned my addr 172.21.15.7:0/3365311492 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:54.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.294+0000 7ff6da229700 1 --2- 172.21.15.7:0/3365311492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8e16138d0 0x55d8e1629820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:54.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.294+0000 7ff6daa2a700 1 --2- 172.21.15.7:0/3365311492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8e16185d0 0x55d8e161f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:54.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.294+0000 7ff6d9a28700 1 -- 172.21.15.7:0/3365311492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8e16185d0 msgr2=0x55d8e161f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:54.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.294+0000 7ff6d9a28700 1 --2- 172.21.15.7:0/3365311492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8e16185d0 0x55d8e161f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.294+0000 7ff6d9a28700 1 -- 172.21.15.7:0/3365311492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8e16138d0 msgr2=0x55d8e1629820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:54.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.294+0000 7ff6d9a28700 1 --2- 172.21.15.7:0/3365311492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8e16138d0 0x55d8e1629820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.294+0000 7ff6d9a28700 1 -- 172.21.15.7:0/3365311492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8e161d9f0 con 0x55d8e16141f0 2024-06-10T01:05:54.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.294+0000 7ff6da229700 1 --2- 172.21.15.7:0/3365311492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8e16138d0 0x55d8e1629820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:54.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.294+0000 7ff6d9a28700 1 --2- 172.21.15.7:0/3365311492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8e16141f0 0x55d8e16199c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40098e0 tx=0x7ff6c401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f78584eee1042c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:54.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.295+0000 7ff6bffff700 1 -- 172.21.15.7:0/3365311492 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c40050c0 con 0x55d8e16141f0 2024-06-10T01:05:54.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.295+0000 7ff6bffff700 1 -- 172.21.15.7:0/3365311492 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6c4029070 con 0x55d8e16141f0 2024-06-10T01:05:54.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.295+0000 7ff6bffff700 1 -- 172.21.15.7:0/3365311492 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c400f6c0 con 0x55d8e16141f0 2024-06-10T01:05:54.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.295+0000 7ff6e6d10b80 1 -- 172.21.15.7:0/3365311492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8e16141f0 msgr2=0x55d8e16199c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:54.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.295+0000 7ff6e6d10b80 1 --2- 172.21.15.7:0/3365311492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8e16141f0 0x55d8e16199c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40098e0 tx=0x7ff6c401a040 comp rx=0 tx=0).stop 2024-06-10T01:05:54.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.295+0000 7ff6e6d10b80 1 -- 172.21.15.7:0/3365311492 shutdown_connections 2024-06-10T01:05:54.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.295+0000 7ff6e6d10b80 1 --2- 172.21.15.7:0/3365311492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8e16185d0 0x55d8e161f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.295+0000 7ff6e6d10b80 1 --2- 172.21.15.7:0/3365311492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8e16138d0 0x55d8e1629820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.295+0000 7ff6e6d10b80 1 --2- 172.21.15.7:0/3365311492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8e16141f0 0x55d8e16199c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.295+0000 7ff6e6d10b80 1 -- 172.21.15.7:0/3365311492 >> 172.21.15.7:0/3365311492 conn(0x55d8e153c100 msgr2=0x55d8e1609870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:54.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.296+0000 7ff6e6d10b80 1 -- 172.21.15.7:0/3365311492 shutdown_connections 2024-06-10T01:05:54.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.296+0000 7ff6e6d10b80 1 -- 172.21.15.7:0/3365311492 wait complete. 2024-06-10T01:05:54.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.296+0000 7ff6e6d10b80 1 -- start start 2024-06-10T01:05:54.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.296+0000 7ff6e6d10b80 1 Processor -- start 2024-06-10T01:05:54.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.296+0000 7ff6e6d10b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8e16138d0 0x55d8e1625990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:54.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.296+0000 7ff6e6d10b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8e16141f0 0x55d8e1625ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:54.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.296+0000 7ff6e6d10b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8e16185d0 0x55d8e1622960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:54.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.296+0000 7ff6e6d10b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d41aa510 con 0x55d8e16185d0 2024-06-10T01:05:54.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.296+0000 7ff6e6d10b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d41ab980 con 0x55d8e16141f0 2024-06-10T01:05:54.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.296+0000 7ff6e6d10b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d41aabe0 con 0x55d8e16138d0 2024-06-10T01:05:54.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.296+0000 7ff6e6d10b80 10 client.0 _ll_register_callbacks cb 0x55d8e14f6630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:05:54.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.297+0000 7ff6daa2a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8e16185d0 0x55d8e1622960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:54.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.297+0000 7ff6daa2a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8e16185d0 0x55d8e1622960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40036/0 (socket says 172.21.15.7:40036) 2024-06-10T01:05:54.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.297+0000 7ff6daa2a700 1 -- 172.21.15.7:0/4053696320 learned_addr learned my addr 172.21.15.7:0/4053696320 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:54.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.297+0000 7ff6da229700 1 --2- 172.21.15.7:0/4053696320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8e16138d0 0x55d8e1625990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:54.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.297+0000 7ff6d9a28700 1 --2- 172.21.15.7:0/4053696320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8e16141f0 0x55d8e1625ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:54.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.297+0000 7ff6daa2a700 1 -- 172.21.15.7:0/4053696320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8e16138d0 msgr2=0x55d8e1625990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:54.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.297+0000 7ff6daa2a700 1 --2- 172.21.15.7:0/4053696320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8e16138d0 0x55d8e1625990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.297+0000 7ff6daa2a700 1 -- 172.21.15.7:0/4053696320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8e16141f0 msgr2=0x55d8e1625ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:54.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.297+0000 7ff6daa2a700 1 --2- 172.21.15.7:0/4053696320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8e16141f0 0x55d8e1625ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.297+0000 7ff6daa2a700 1 -- 172.21.15.7:0/4053696320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8e1622ea0 con 0x55d8e16185d0 2024-06-10T01:05:54.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.297+0000 7ff6da229700 1 --2- 172.21.15.7:0/4053696320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8e16138d0 0x55d8e1625990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:54.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.297+0000 7ff6daa2a700 1 --2- 172.21.15.7:0/4053696320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8e16185d0 0x55d8e1622960 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc097210 tx=0x7ff6cc09b580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225973cc341d8380 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:54.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.297+0000 7ff6bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:05:54.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.297+0000 7ff6bdffb700 1 -- 172.21.15.7:0/4053696320 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc09f160 con 0x55d8e16185d0 2024-06-10T01:05:54.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.297+0000 7ff6e6d10b80 1 -- 172.21.15.7:0/4053696320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6c4029070 con 0x55d8e16185d0 2024-06-10T01:05:54.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.297+0000 7ff6bdffb700 1 -- 172.21.15.7:0/4053696320 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7ff6cc0a3590 con 0x55d8e16185d0 2024-06-10T01:05:54.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.297+0000 7ff6bdffb700 1 -- 172.21.15.7:0/4053696320 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff6cc0a3a20 con 0x55d8e16185d0 2024-06-10T01:05:54.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.297+0000 7ff6e6d10b80 10 client.15300 fetch_fsmap learned FSMap version 27 2024-06-10T01:05:54.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.298+0000 7ff6e6d10b80 1 -- 172.21.15.7:0/4053696320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d8e1698750 con 0x55d8e16185d0 2024-06-10T01:05:54.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.298+0000 7ff6bdffb700 1 -- 172.21.15.7:0/4053696320 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6c4029070 con 0x55d8e16185d0 2024-06-10T01:05:54.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.298+0000 7ff6bdffb700 1 -- 172.21.15.7:0/4053696320 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff6cc004b60 con 0x55d8e16185d0 2024-06-10T01:05:54.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.298+0000 7ff6e6d10b80 10 client.15300 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:05:54.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.298+0000 7ff6e6d10b80 10 client.15300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:05:54.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.298+0000 7ff6e6d10b80 20 client.15300 populate_metadata read hostname 'smithi007' 2024-06-10T01:05:54.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.298+0000 7ff6e6d10b80 1 --2- 172.21.15.7:0/4053696320 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d8e162d400 0x55d8e169ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:54.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.298+0000 7ff6e6d10b80 4 client.15300 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:05:54.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.298+0000 7ff6e6d10b80 1 -- 172.21.15.7:0/4053696320 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d8e1568350 con 0x55d8e162d400 2024-06-10T01:05:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.298+0000 7ff6da229700 1 --2- 172.21.15.7:0/4053696320 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d8e162d400 0x55d8e169ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.299+0000 7ff6da229700 1 --2- 172.21.15.7:0/4053696320 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d8e162d400 0x55d8e169ae20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d354c5f577da3c7 server_cookie=ac794c59ed24de1f in_seq=0 out_seq=0 2024-06-10T01:05:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.299+0000 7ff6bdffb700 10 client.15300 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:05:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.300+0000 7ff6bdffb700 1 -- 172.21.15.7:0/4053696320 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d8e1568350 con 0x55d8e162d400 2024-06-10T01:05:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.300+0000 7ff6bdffb700 10 client.15300 handle_command_reply: tid=0 2024-06-10T01:05:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.300+0000 7ff6e6d10b80 1 client.15300 shutdown 2024-06-10T01:05:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.300+0000 7ff6e6d10b80 1 -- 172.21.15.7:0/4053696320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8e16185d0 msgr2=0x55d8e1622960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.300+0000 7ff6e6d10b80 1 --2- 172.21.15.7:0/4053696320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8e16185d0 0x55d8e1622960 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc097210 tx=0x7ff6cc09b580 comp rx=0 tx=0).stop 2024-06-10T01:05:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.300+0000 7ff6e6d10b80 1 -- 172.21.15.7:0/4053696320 shutdown_connections 2024-06-10T01:05:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.300+0000 7ff6e6d10b80 1 --2- 172.21.15.7:0/4053696320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8e16185d0 0x55d8e1622960 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.300+0000 7ff6e6d10b80 1 --2- 172.21.15.7:0/4053696320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8e16138d0 0x55d8e1625990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.300+0000 7ff6e6d10b80 1 --2- 172.21.15.7:0/4053696320 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d8e162d400 0x55d8e169ae20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.300+0000 7ff6e6d10b80 1 --2- 172.21.15.7:0/4053696320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8e16141f0 0x55d8e1625ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.300+0000 7ff6e6d10b80 1 -- 172.21.15.7:0/4053696320 >> 172.21.15.7:0/4053696320 conn(0x55d8e153c100 msgr2=0x55d8e1617d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.300+0000 7ff6bdffb700 0 client.15300 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:05:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.301+0000 7ff6e6d10b80 1 -- 172.21.15.7:0/4053696320 shutdown_connections 2024-06-10T01:05:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.301+0000 7ff6e6d10b80 1 -- 172.21.15.7:0/4053696320 wait complete. 2024-06-10T01:05:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.301+0000 7ff6e6d10b80 20 client.15300 trim_cache size 0 max 16384 2024-06-10T01:05:54.301 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:05:54.301 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-10T01:05:54.301 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "333bd08f-d8ff-40fd-8495-172914b960e5", 2024-06-10T01:05:54.301 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-10T01:05:54.302 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:05:54.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.301+0000 7fb9c7301700 1 -- 172.21.15.7:0/1858881229 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb9c00628d0 con 0x7fb9ac0a2c20 2024-06-10T01:05:54.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.301+0000 7ff6dc48d700 1 -- 172.21.15.7:0/2469452940 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff6c00a2b80 msgr2=0x7ff6c00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:54.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.301+0000 7ff6dc48d700 1 --2- 172.21.15.7:0/2469452940 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff6c00a2b80 0x7ff6c00a5040 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4199be0 tx=0x7ff6d000b410 comp rx=0 tx=0).stop 2024-06-10T01:05:54.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.301+0000 7ff6dc48d700 1 -- 172.21.15.7:0/2469452940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d4107570 msgr2=0x7ff6d4198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:54.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.301+0000 7ff6dc48d700 1 --2- 172.21.15.7:0/2469452940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d4107570 0x7ff6d4198aa0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc007ec0 tx=0x7ff6cc00d3b0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.301+0000 7ff6dc48d700 1 -- 172.21.15.7:0/2469452940 shutdown_connections 2024-06-10T01:05:54.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.301+0000 7ff6dc48d700 1 --2- 172.21.15.7:0/2469452940 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff6c00a2b80 0x7ff6c00a5040 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.301+0000 7ff6dc48d700 1 --2- 172.21.15.7:0/2469452940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d4107570 0x7ff6d4198aa0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.301+0000 7ff6dc48d700 1 --2- 172.21.15.7:0/2469452940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6d4100e30 0x7ff6d419e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.301+0000 7ff6dc48d700 1 --2- 172.21.15.7:0/2469452940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6d41017d0 0x7ff6d419ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.301+0000 7ff6dc48d700 1 -- 172.21.15.7:0/2469452940 >> 172.21.15.7:0/2469452940 conn(0x7ff6d4076680 msgr2=0x7ff6d40ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:54.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.301+0000 7ff6dc48d700 1 -- 172.21.15.7:0/2469452940 shutdown_connections 2024-06-10T01:05:54.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.301+0000 7ff6dc48d700 1 -- 172.21.15.7:0/2469452940 wait complete. 2024-06-10T01:05:54.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.311+0000 7fb9b67fc700 1 -- 172.21.15.7:0/1858881229 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340767 (secure 0 0 0) 0x7fb9c00628d0 con 0x7fb9ac0a2c20 2024-06-10T01:05:54.312 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:05:54.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.314+0000 7fb9c7301700 1 -- 172.21.15.7:0/1858881229 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb9ac0a2c20 msgr2=0x7fb9ac0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:54.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.314+0000 7fb9c7301700 1 --2- 172.21.15.7:0/1858881229 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb9ac0a2c20 0x7fb9ac0a50e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc006010 tx=0x7fb9bc00b540 comp rx=0 tx=0).stop 2024-06-10T01:05:54.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.314+0000 7fb9c7301700 1 -- 172.21.15.7:0/1858881229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9c010efd0 msgr2=0x7fb9c019f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:54.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.314+0000 7fb9c7301700 1 --2- 172.21.15.7:0/1858881229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9c010efd0 0x7fb9c019f690 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb9b80094f0 tx=0x7fb9b800d3b0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.314+0000 7fb9c7301700 1 -- 172.21.15.7:0/1858881229 shutdown_connections 2024-06-10T01:05:54.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.314+0000 7fb9c7301700 1 --2- 172.21.15.7:0/1858881229 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb9ac0a2c20 0x7fb9ac0a50e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.314+0000 7fb9c7301700 1 --2- 172.21.15.7:0/1858881229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9c010efd0 0x7fb9c019f690 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.314+0000 7fb9c7301700 1 --2- 172.21.15.7:0/1858881229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9c006ac00 0x7fb9c019ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.314+0000 7fb9c7301700 1 --2- 172.21.15.7:0/1858881229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9c006b520 0x7fb9c019b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:54.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.314+0000 7fb9c7301700 1 -- 172.21.15.7:0/1858881229 >> 172.21.15.7:0/1858881229 conn(0x7fb9c00773e0 msgr2=0x7fb9c0110380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:54.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.314+0000 7fb9c7301700 1 -- 172.21.15.7:0/1858881229 shutdown_connections 2024-06-10T01:05:54.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:54.314+0000 7fb9c7301700 1 -- 172.21.15.7:0/1858881229 wait complete. 2024-06-10T01:05:54.316 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-06-10T01:05:55.040 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:05:55.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:55 smithi078 ceph-mon[36683]: pgmap v428: 129 pgs: 129 active+clean; 648 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-10T01:05:55.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:55 smithi078 ceph-mon[36683]: scrub queued for path: / 2024-06-10T01:05:55.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:55 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:05:55.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:55 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:05:55.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:55 smithi078 ceph-mon[36683]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T01:05:55.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:55 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:05:55.691 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:55 smithi007 ceph-mon[32398]: pgmap v428: 129 pgs: 129 active+clean; 648 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-10T01:05:55.691 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:55 smithi007 ceph-mon[32398]: scrub queued for path: / 2024-06-10T01:05:55.691 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:55 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:05:55.691 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:55 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:05:55.691 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:55 smithi007 ceph-mon[32398]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T01:05:55.691 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:55 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:05:55.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:55 smithi175 ceph-mon[36636]: pgmap v428: 129 pgs: 129 active+clean; 648 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-10T01:05:55.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:55 smithi175 ceph-mon[36636]: scrub queued for path: / 2024-06-10T01:05:55.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:55 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:05:55.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:55 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:05:55.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:55 smithi175 ceph-mon[36636]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T01:05:55.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:55 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:05:55.720 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":428,"stamp":"2024-06-10T01:05:53.600044+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663449,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1205,"num_read_kb":725,"num_write":367,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":794,"ondisk_log_size":794,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362480,"kb_used_data":13328,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579344,"statfs":{"total":1151940427776,"available":1151569248256,"internally_reserved":0,"allocated":13647872,"data_stored":8803049,"data_compressed":496626,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4314,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":35,"num_write":1,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003454"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.569445+0000","last_change":"2024-06-10T01:01:47.244728+0000","last_active":"2024-06-10T01:04:35.569445+0000","last_peered":"2024-06-10T01:04:35.569445+0000","last_clean":"2024-06-10T01:04:35.569445+0000","last_became_active":"2024-06-10T01:01:47.244198+0000","last_became_peered":"2024-06-10T01:01:47.244198+0000","last_unstale":"2024-06-10T01:04:35.569445+0000","last_undegraded":"2024-06-10T01:04:35.569445+0000","last_fullsized":"2024-06-10T01:04:35.569445+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:58:06.257952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.557217+0000","last_change":"2024-06-10T01:01:47.247620+0000","last_active":"2024-06-10T01:04:35.557217+0000","last_peered":"2024-06-10T01:04:35.557217+0000","last_clean":"2024-06-10T01:04:35.557217+0000","last_became_active":"2024-06-10T01:01:47.245475+0000","last_became_peered":"2024-06-10T01:01:47.245475+0000","last_unstale":"2024-06-10T01:04:35.557217+0000","last_undegraded":"2024-06-10T01:04:35.557217+0000","last_fullsized":"2024-06-10T01:04:35.557217+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:29:09.367411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.569660+0000","last_change":"2024-06-10T01:01:47.252414+0000","last_active":"2024-06-10T01:04:35.569660+0000","last_peered":"2024-06-10T01:04:35.569660+0000","last_clean":"2024-06-10T01:04:35.569660+0000","last_became_active":"2024-06-10T01:01:47.252063+0000","last_became_peered":"2024-06-10T01:01:47.252063+0000","last_unstale":"2024-06-10T01:04:35.569660+0000","last_undegraded":"2024-06-10T01:04:35.569660+0000","last_fullsized":"2024-06-10T01:04:35.569660+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:54:53.966209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.553817+0000","last_change":"2024-06-10T01:01:47.254875+0000","last_active":"2024-06-10T01:04:35.553817+0000","last_peered":"2024-06-10T01:04:35.553817+0000","last_clean":"2024-06-10T01:04:35.553817+0000","last_became_active":"2024-06-10T01:01:47.254542+0000","last_became_peered":"2024-06-10T01:01:47.254542+0000","last_unstale":"2024-06-10T01:04:35.553817+0000","last_undegraded":"2024-06-10T01:04:35.553817+0000","last_fullsized":"2024-06-10T01:04:35.553817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:02:28.393133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558663+0000","last_change":"2024-06-10T01:01:47.242596+0000","last_active":"2024-06-10T01:04:35.558663+0000","last_peered":"2024-06-10T01:04:35.558663+0000","last_clean":"2024-06-10T01:04:35.558663+0000","last_became_active":"2024-06-10T01:01:47.242308+0000","last_became_peered":"2024-06-10T01:01:47.242308+0000","last_unstale":"2024-06-10T01:04:35.558663+0000","last_undegraded":"2024-06-10T01:04:35.558663+0000","last_fullsized":"2024-06-10T01:04:35.558663+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:11:26.599954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.550600+0000","last_change":"2024-06-10T01:01:47.249085+0000","last_active":"2024-06-10T01:04:35.550600+0000","last_peered":"2024-06-10T01:04:35.550600+0000","last_clean":"2024-06-10T01:04:35.550600+0000","last_became_active":"2024-06-10T01:01:47.248749+0000","last_became_peered":"2024-06-10T01:01:47.248749+0000","last_unstale":"2024-06-10T01:04:35.550600+0000","last_undegraded":"2024-06-10T01:04:35.550600+0000","last_fullsized":"2024-06-10T01:04:35.550600+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:42:48.734566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562583+0000","last_change":"2024-06-10T01:01:47.247334+0000","last_active":"2024-06-10T01:04:35.562583+0000","last_peered":"2024-06-10T01:04:35.562583+0000","last_clean":"2024-06-10T01:04:35.562583+0000","last_became_active":"2024-06-10T01:01:47.246964+0000","last_became_peered":"2024-06-10T01:01:47.246964+0000","last_unstale":"2024-06-10T01:04:35.562583+0000","last_undegraded":"2024-06-10T01:04:35.562583+0000","last_fullsized":"2024-06-10T01:04:35.562583+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:05:36.034950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.563056+0000","last_change":"2024-06-10T01:01:47.242523+0000","last_active":"2024-06-10T01:04:35.563056+0000","last_peered":"2024-06-10T01:04:35.563056+0000","last_clean":"2024-06-10T01:04:35.563056+0000","last_became_active":"2024-06-10T01:01:47.242198+0000","last_became_peered":"2024-06-10T01:01:47.242198+0000","last_unstale":"2024-06-10T01:04:35.563056+0000","last_undegraded":"2024-06-10T01:04:35.563056+0000","last_fullsized":"2024-06-10T01:04:35.563056+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:39:07.055970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.567116+0000","last_change":"2024-06-10T01:01:47.245277+0000","last_active":"2024-06-10T01:04:35.567116+0000","last_peered":"2024-06-10T01:04:35.567116+0000","last_clean":"2024-06-10T01:04:35.567116+0000","last_became_active":"2024-06-10T01:01:47.244932+0000","last_became_peered":"2024-06-10T01:01:47.244932+0000","last_unstale":"2024-06-10T01:04:35.567116+0000","last_undegraded":"2024-06-10T01:04:35.567116+0000","last_fullsized":"2024-06-10T01:04:35.567116+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:56:24.423713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559784+0000","last_change":"2024-06-10T01:01:47.255531+0000","last_active":"2024-06-10T01:04:35.559784+0000","last_peered":"2024-06-10T01:04:35.559784+0000","last_clean":"2024-06-10T01:04:35.559784+0000","last_became_active":"2024-06-10T01:01:47.255150+0000","last_became_peered":"2024-06-10T01:01:47.255150+0000","last_unstale":"2024-06-10T01:04:35.559784+0000","last_undegraded":"2024-06-10T01:04:35.559784+0000","last_fullsized":"2024-06-10T01:04:35.559784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:51:34.561396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559164+0000","last_change":"2024-06-10T01:01:47.249832+0000","last_active":"2024-06-10T01:04:35.559164+0000","last_peered":"2024-06-10T01:04:35.559164+0000","last_clean":"2024-06-10T01:04:35.559164+0000","last_became_active":"2024-06-10T01:01:47.249303+0000","last_became_peered":"2024-06-10T01:01:47.249303+0000","last_unstale":"2024-06-10T01:04:35.559164+0000","last_undegraded":"2024-06-10T01:04:35.559164+0000","last_fullsized":"2024-06-10T01:04:35.559164+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:15:21.930157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.551184+0000","last_change":"2024-06-10T01:01:47.255927+0000","last_active":"2024-06-10T01:04:35.551184+0000","last_peered":"2024-06-10T01:04:35.551184+0000","last_clean":"2024-06-10T01:04:35.551184+0000","last_became_active":"2024-06-10T01:01:47.255591+0000","last_became_peered":"2024-06-10T01:01:47.255591+0000","last_unstale":"2024-06-10T01:04:35.551184+0000","last_undegraded":"2024-06-10T01:04:35.551184+0000","last_fullsized":"2024-06-10T01:04:35.551184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:48:06.494430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.557880+0000","last_change":"2024-06-10T01:02:37.372069+0000","last_active":"2024-06-10T01:04:35.557880+0000","last_peered":"2024-06-10T01:04:35.557880+0000","last_clean":"2024-06-10T01:04:35.557880+0000","last_became_active":"2024-06-10T01:02:37.371599+0000","last_became_peered":"2024-06-10T01:02:37.371599+0000","last_unstale":"2024-06-10T01:04:35.557880+0000","last_undegraded":"2024-06-10T01:04:35.557880+0000","last_fullsized":"2024-06-10T01:04:35.557880+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:54:34.091576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.551120+0000","last_change":"2024-06-10T01:01:47.249553+0000","last_active":"2024-06-10T01:04:35.551120+0000","last_peered":"2024-06-10T01:04:35.551120+0000","last_clean":"2024-06-10T01:04:35.551120+0000","last_became_active":"2024-06-10T01:01:47.249202+0000","last_became_peered":"2024-06-10T01:01:47.249202+0000","last_unstale":"2024-06-10T01:04:35.551120+0000","last_undegraded":"2024-06-10T01:04:35.551120+0000","last_fullsized":"2024-06-10T01:04:35.551120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:23:57.882264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.564289+0000","last_change":"2024-06-10T01:01:47.253987+0000","last_active":"2024-06-10T01:04:35.564289+0000","last_peered":"2024-06-10T01:04:35.564289+0000","last_clean":"2024-06-10T01:04:35.564289+0000","last_became_active":"2024-06-10T01:01:47.253578+0000","last_became_peered":"2024-06-10T01:01:47.253578+0000","last_unstale":"2024-06-10T01:04:35.564289+0000","last_undegraded":"2024-06-10T01:04:35.564289+0000","last_fullsized":"2024-06-10T01:04:35.564289+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:35:30.585482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.567601+0000","last_change":"2024-06-10T01:01:47.247089+0000","last_active":"2024-06-10T01:04:35.567601+0000","last_peered":"2024-06-10T01:04:35.567601+0000","last_clean":"2024-06-10T01:04:35.567601+0000","last_became_active":"2024-06-10T01:01:47.246066+0000","last_became_peered":"2024-06-10T01:01:47.246066+0000","last_unstale":"2024-06-10T01:04:35.567601+0000","last_undegraded":"2024-06-10T01:04:35.567601+0000","last_fullsized":"2024-06-10T01:04:35.567601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:58:12.092981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.569804+0000","last_change":"2024-06-10T01:01:47.244491+0000","last_active":"2024-06-10T01:04:35.569804+0000","last_peered":"2024-06-10T01:04:35.569804+0000","last_clean":"2024-06-10T01:04:35.569804+0000","last_became_active":"2024-06-10T01:01:47.243747+0000","last_became_peered":"2024-06-10T01:01:47.243747+0000","last_unstale":"2024-06-10T01:04:35.569804+0000","last_undegraded":"2024-06-10T01:04:35.569804+0000","last_fullsized":"2024-06-10T01:04:35.569804+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:55:56.785016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.567988+0000","last_change":"2024-06-10T01:01:47.243141+0000","last_active":"2024-06-10T01:04:35.567988+0000","last_peered":"2024-06-10T01:04:35.567988+0000","last_clean":"2024-06-10T01:04:35.567988+0000","last_became_active":"2024-06-10T01:01:47.242464+0000","last_became_peered":"2024-06-10T01:01:47.242464+0000","last_unstale":"2024-06-10T01:04:35.567988+0000","last_undegraded":"2024-06-10T01:04:35.567988+0000","last_fullsized":"2024-06-10T01:04:35.567988+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:00:49.777130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559919+0000","last_change":"2024-06-10T01:01:47.253463+0000","last_active":"2024-06-10T01:04:35.559919+0000","last_peered":"2024-06-10T01:04:35.559919+0000","last_clean":"2024-06-10T01:04:35.559919+0000","last_became_active":"2024-06-10T01:01:47.252929+0000","last_became_peered":"2024-06-10T01:01:47.252929+0000","last_unstale":"2024-06-10T01:04:35.559919+0000","last_undegraded":"2024-06-10T01:04:35.559919+0000","last_fullsized":"2024-06-10T01:04:35.559919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:18:49.551276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239453+0000","last_change":"2024-06-10T01:01:57.279561+0000","last_active":"2024-06-10T01:03:38.239453+0000","last_peered":"2024-06-10T01:03:38.239453+0000","last_clean":"2024-06-10T01:03:38.239453+0000","last_became_active":"2024-06-10T01:01:57.279132+0000","last_became_peered":"2024-06-10T01:01:57.279132+0000","last_unstale":"2024-06-10T01:03:38.239453+0000","last_undegraded":"2024-06-10T01:03:38.239453+0000","last_fullsized":"2024-06-10T01:03:38.239453+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:55:03.497701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'9","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.772124+0000","last_change":"2024-06-10T01:01:44.006667+0000","last_active":"2024-06-10T01:03:38.772124+0000","last_peered":"2024-06-10T01:03:38.772124+0000","last_clean":"2024-06-10T01:03:38.772124+0000","last_became_active":"2024-06-10T01:01:44.006343+0000","last_became_peered":"2024-06-10T01:01:44.006343+0000","last_unstale":"2024-06-10T01:03:38.772124+0000","last_undegraded":"2024-06-10T01:03:38.772124+0000","last_fullsized":"2024-06-10T01:03:38.772124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:14:50.540084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562931+0000","last_change":"2024-06-10T01:01:47.246139+0000","last_active":"2024-06-10T01:04:35.562931+0000","last_peered":"2024-06-10T01:04:35.562931+0000","last_clean":"2024-06-10T01:04:35.562931+0000","last_became_active":"2024-06-10T01:01:47.245743+0000","last_became_peered":"2024-06-10T01:01:47.245743+0000","last_unstale":"2024-06-10T01:04:35.562931+0000","last_undegraded":"2024-06-10T01:04:35.562931+0000","last_fullsized":"2024-06-10T01:04:35.562931+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:06:39.041569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239398+0000","last_change":"2024-06-10T01:01:57.282843+0000","last_active":"2024-06-10T01:03:38.239398+0000","last_peered":"2024-06-10T01:03:38.239398+0000","last_clean":"2024-06-10T01:03:38.239398+0000","last_became_active":"2024-06-10T01:01:57.281797+0000","last_became_peered":"2024-06-10T01:01:57.281797+0000","last_unstale":"2024-06-10T01:03:38.239398+0000","last_undegraded":"2024-06-10T01:03:38.239398+0000","last_fullsized":"2024-06-10T01:03:38.239398+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:00.209628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:43.959967+0000","last_change":"2024-06-10T01:01:44.466714+0000","last_active":"2024-06-10T01:03:43.959967+0000","last_peered":"2024-06-10T01:03:43.959967+0000","last_clean":"2024-06-10T01:03:43.959967+0000","last_became_active":"2024-06-10T01:01:44.466296+0000","last_became_peered":"2024-06-10T01:01:44.466296+0000","last_unstale":"2024-06-10T01:03:43.959967+0000","last_undegraded":"2024-06-10T01:03:43.959967+0000","last_fullsized":"2024-06-10T01:03:43.959967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:16:36.082308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.569808+0000","last_change":"2024-06-10T01:01:47.244173+0000","last_active":"2024-06-10T01:04:35.569808+0000","last_peered":"2024-06-10T01:04:35.569808+0000","last_clean":"2024-06-10T01:04:35.569808+0000","last_became_active":"2024-06-10T01:01:47.243032+0000","last_became_peered":"2024-06-10T01:01:47.243032+0000","last_unstale":"2024-06-10T01:04:35.569808+0000","last_undegraded":"2024-06-10T01:04:35.569808+0000","last_fullsized":"2024-06-10T01:04:35.569808+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:14:33.965903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239057+0000","last_change":"2024-06-10T01:02:37.592535+0000","last_active":"2024-06-10T01:03:38.239057+0000","last_peered":"2024-06-10T01:03:38.239057+0000","last_clean":"2024-06-10T01:03:38.239057+0000","last_became_active":"2024-06-10T01:02:37.592159+0000","last_became_peered":"2024-06-10T01:02:37.592159+0000","last_unstale":"2024-06-10T01:03:38.239057+0000","last_undegraded":"2024-06-10T01:03:38.239057+0000","last_fullsized":"2024-06-10T01:03:38.239057+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:50:08.626485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235433+0000","last_change":"2024-06-10T01:01:44.009105+0000","last_active":"2024-06-10T01:03:38.235433+0000","last_peered":"2024-06-10T01:03:38.235433+0000","last_clean":"2024-06-10T01:03:38.235433+0000","last_became_active":"2024-06-10T01:01:44.008743+0000","last_became_peered":"2024-06-10T01:01:44.008743+0000","last_unstale":"2024-06-10T01:03:38.235433+0000","last_undegraded":"2024-06-10T01:03:38.235433+0000","last_fullsized":"2024-06-10T01:03:38.235433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:44:54.301378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.570239+0000","last_change":"2024-06-10T01:01:47.249007+0000","last_active":"2024-06-10T01:04:35.570239+0000","last_peered":"2024-06-10T01:04:35.570239+0000","last_clean":"2024-06-10T01:04:35.570239+0000","last_became_active":"2024-06-10T01:01:47.245632+0000","last_became_peered":"2024-06-10T01:01:47.245632+0000","last_unstale":"2024-06-10T01:04:35.570239+0000","last_undegraded":"2024-06-10T01:04:35.570239+0000","last_fullsized":"2024-06-10T01:04:35.570239+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:29:18.916390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.238983+0000","last_change":"2024-06-10T01:01:57.283137+0000","last_active":"2024-06-10T01:03:38.238983+0000","last_peered":"2024-06-10T01:03:38.238983+0000","last_clean":"2024-06-10T01:03:38.238983+0000","last_became_active":"2024-06-10T01:01:57.282605+0000","last_became_peered":"2024-06-10T01:01:57.282605+0000","last_unstale":"2024-06-10T01:03:38.238983+0000","last_undegraded":"2024-06-10T01:03:38.238983+0000","last_fullsized":"2024-06-10T01:03:38.238983+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:26:16.967320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235734+0000","last_change":"2024-06-10T01:01:44.010674+0000","last_active":"2024-06-10T01:03:38.235734+0000","last_peered":"2024-06-10T01:03:38.235734+0000","last_clean":"2024-06-10T01:03:38.235734+0000","last_became_active":"2024-06-10T01:01:44.010291+0000","last_became_peered":"2024-06-10T01:01:44.010291+0000","last_unstale":"2024-06-10T01:03:38.235734+0000","last_undegraded":"2024-06-10T01:03:38.235734+0000","last_fullsized":"2024-06-10T01:03:38.235734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:36:18.418362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":60,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.566078+0000","last_change":"2024-06-10T01:02:37.794580+0000","last_active":"2024-06-10T01:04:35.566078+0000","last_peered":"2024-06-10T01:04:35.566078+0000","last_clean":"2024-06-10T01:04:35.566078+0000","last_became_active":"2024-06-10T01:02:37.794127+0000","last_became_peered":"2024-06-10T01:02:37.794127+0000","last_unstale":"2024-06-10T01:04:35.566078+0000","last_undegraded":"2024-06-10T01:04:35.566078+0000","last_fullsized":"2024-06-10T01:04:35.566078+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:02.117871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239746+0000","last_change":"2024-06-10T01:01:57.282494+0000","last_active":"2024-06-10T01:03:38.239746+0000","last_peered":"2024-06-10T01:03:38.239746+0000","last_clean":"2024-06-10T01:03:38.239746+0000","last_became_active":"2024-06-10T01:01:57.282077+0000","last_became_peered":"2024-06-10T01:01:57.282077+0000","last_unstale":"2024-06-10T01:03:38.239746+0000","last_undegraded":"2024-06-10T01:03:38.239746+0000","last_fullsized":"2024-06-10T01:03:38.239746+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:57:41.639788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"97'5","reported_seq":229,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:50.966743+0000","last_change":"2024-06-10T01:01:44.011134+0000","last_active":"2024-06-10T01:05:50.966743+0000","last_peered":"2024-06-10T01:05:50.966743+0000","last_clean":"2024-06-10T01:05:50.966743+0000","last_became_active":"2024-06-10T01:01:44.010788+0000","last_became_peered":"2024-06-10T01:01:44.010788+0000","last_unstale":"2024-06-10T01:05:50.966743+0000","last_undegraded":"2024-06-10T01:05:50.966743+0000","last_fullsized":"2024-06-10T01:05:50.966743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:22:55.228946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.564421+0000","last_change":"2024-06-10T01:01:47.248093+0000","last_active":"2024-06-10T01:04:35.564421+0000","last_peered":"2024-06-10T01:04:35.564421+0000","last_clean":"2024-06-10T01:04:35.564421+0000","last_became_active":"2024-06-10T01:01:47.247797+0000","last_became_peered":"2024-06-10T01:01:47.247797+0000","last_unstale":"2024-06-10T01:04:35.564421+0000","last_undegraded":"2024-06-10T01:04:35.564421+0000","last_fullsized":"2024-06-10T01:04:35.564421+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:45:50.159128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239671+0000","last_change":"2024-06-10T01:02:37.377039+0000","last_active":"2024-06-10T01:03:38.239671+0000","last_peered":"2024-06-10T01:03:38.239671+0000","last_clean":"2024-06-10T01:03:38.239671+0000","last_became_active":"2024-06-10T01:02:37.376332+0000","last_became_peered":"2024-06-10T01:02:37.376332+0000","last_unstale":"2024-06-10T01:03:38.239671+0000","last_undegraded":"2024-06-10T01:03:38.239671+0000","last_fullsized":"2024-06-10T01:03:38.239671+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:15.000505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,10,5],"acting":[2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.570465+0000","last_change":"2024-06-10T01:01:44.008342+0000","last_active":"2024-06-10T01:03:38.570465+0000","last_peered":"2024-06-10T01:03:38.570465+0000","last_clean":"2024-06-10T01:03:38.570465+0000","last_became_active":"2024-06-10T01:01:44.007959+0000","last_became_peered":"2024-06-10T01:01:44.007959+0000","last_unstale":"2024-06-10T01:03:38.570465+0000","last_undegraded":"2024-06-10T01:03:38.570465+0000","last_fullsized":"2024-06-10T01:03:38.570465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:36:03.143609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.565044+0000","last_change":"2024-06-10T01:01:47.247854+0000","last_active":"2024-06-10T01:04:35.565044+0000","last_peered":"2024-06-10T01:04:35.565044+0000","last_clean":"2024-06-10T01:04:35.565044+0000","last_became_active":"2024-06-10T01:01:47.247563+0000","last_became_peered":"2024-06-10T01:01:47.247563+0000","last_unstale":"2024-06-10T01:04:35.565044+0000","last_undegraded":"2024-06-10T01:04:35.565044+0000","last_fullsized":"2024-06-10T01:04:35.565044+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:32:13.024815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.237041+0000","last_change":"2024-06-10T01:01:57.282427+0000","last_active":"2024-06-10T01:03:38.237041+0000","last_peered":"2024-06-10T01:03:38.237041+0000","last_clean":"2024-06-10T01:03:38.237041+0000","last_became_active":"2024-06-10T01:01:57.281987+0000","last_became_peered":"2024-06-10T01:01:57.281987+0000","last_unstale":"2024-06-10T01:03:38.237041+0000","last_undegraded":"2024-06-10T01:03:38.237041+0000","last_fullsized":"2024-06-10T01:03:38.237041+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:22:31.736804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770902+0000","last_change":"2024-06-10T01:01:44.006567+0000","last_active":"2024-06-10T01:03:38.770902+0000","last_peered":"2024-06-10T01:03:38.770902+0000","last_clean":"2024-06-10T01:03:38.770902+0000","last_became_active":"2024-06-10T01:01:44.006216+0000","last_became_peered":"2024-06-10T01:01:44.006216+0000","last_unstale":"2024-06-10T01:03:38.770902+0000","last_undegraded":"2024-06-10T01:03:38.770902+0000","last_fullsized":"2024-06-10T01:03:38.770902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:33:55.696873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558008+0000","last_change":"2024-06-10T01:01:47.245326+0000","last_active":"2024-06-10T01:04:35.558008+0000","last_peered":"2024-06-10T01:04:35.558008+0000","last_clean":"2024-06-10T01:04:35.558008+0000","last_became_active":"2024-06-10T01:01:47.245013+0000","last_became_peered":"2024-06-10T01:01:47.245013+0000","last_unstale":"2024-06-10T01:04:35.558008+0000","last_undegraded":"2024-06-10T01:04:35.558008+0000","last_fullsized":"2024-06-10T01:04:35.558008+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:17:56.416627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234530+0000","last_change":"2024-06-10T01:01:57.288997+0000","last_active":"2024-06-10T01:03:38.234530+0000","last_peered":"2024-06-10T01:03:38.234530+0000","last_clean":"2024-06-10T01:03:38.234530+0000","last_became_active":"2024-06-10T01:01:57.288596+0000","last_became_peered":"2024-06-10T01:01:57.288596+0000","last_unstale":"2024-06-10T01:03:38.234530+0000","last_undegraded":"2024-06-10T01:03:38.234530+0000","last_fullsized":"2024-06-10T01:03:38.234530+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:08:14.739737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.269738+0000","last_change":"2024-06-10T01:01:44.010079+0000","last_active":"2024-06-10T01:03:38.269738+0000","last_peered":"2024-06-10T01:03:38.269738+0000","last_clean":"2024-06-10T01:03:38.269738+0000","last_became_active":"2024-06-10T01:01:44.009752+0000","last_became_peered":"2024-06-10T01:01:44.009752+0000","last_unstale":"2024-06-10T01:03:38.269738+0000","last_undegraded":"2024-06-10T01:03:38.269738+0000","last_fullsized":"2024-06-10T01:03:38.269738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:29:35.660935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.554398+0000","last_change":"2024-06-10T01:01:47.245015+0000","last_active":"2024-06-10T01:04:35.554398+0000","last_peered":"2024-06-10T01:04:35.554398+0000","last_clean":"2024-06-10T01:04:35.554398+0000","last_became_active":"2024-06-10T01:01:47.244595+0000","last_became_peered":"2024-06-10T01:01:47.244595+0000","last_unstale":"2024-06-10T01:04:35.554398+0000","last_undegraded":"2024-06-10T01:04:35.554398+0000","last_fullsized":"2024-06-10T01:04:35.554398+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:01:48.446526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.238926+0000","last_change":"2024-06-10T01:01:57.281656+0000","last_active":"2024-06-10T01:03:38.238926+0000","last_peered":"2024-06-10T01:03:38.238926+0000","last_clean":"2024-06-10T01:03:38.238926+0000","last_became_active":"2024-06-10T01:01:57.281295+0000","last_became_peered":"2024-06-10T01:01:57.281295+0000","last_unstale":"2024-06-10T01:03:38.238926+0000","last_undegraded":"2024-06-10T01:03:38.238926+0000","last_fullsized":"2024-06-10T01:03:38.238926+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:04:41.619864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"97'3","reported_seq":223,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:48.962945+0000","last_change":"2024-06-10T01:01:44.011941+0000","last_active":"2024-06-10T01:05:48.962945+0000","last_peered":"2024-06-10T01:05:48.962945+0000","last_clean":"2024-06-10T01:05:48.962945+0000","last_became_active":"2024-06-10T01:01:44.010156+0000","last_became_peered":"2024-06-10T01:01:44.010156+0000","last_unstale":"2024-06-10T01:05:48.962945+0000","last_undegraded":"2024-06-10T01:05:48.962945+0000","last_fullsized":"2024-06-10T01:05:48.962945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:16:15.709544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":175,"num_read_kb":175,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.570257+0000","last_change":"2024-06-10T01:01:47.241720+0000","last_active":"2024-06-10T01:04:35.570257+0000","last_peered":"2024-06-10T01:04:35.570257+0000","last_clean":"2024-06-10T01:04:35.570257+0000","last_became_active":"2024-06-10T01:01:47.241386+0000","last_became_peered":"2024-06-10T01:01:47.241386+0000","last_unstale":"2024-06-10T01:04:35.570257+0000","last_undegraded":"2024-06-10T01:04:35.570257+0000","last_fullsized":"2024-06-10T01:04:35.570257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:42:01.304787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770709+0000","last_change":"2024-06-10T01:01:57.283226+0000","last_active":"2024-06-10T01:03:38.770709+0000","last_peered":"2024-06-10T01:03:38.770709+0000","last_clean":"2024-06-10T01:03:38.770709+0000","last_became_active":"2024-06-10T01:01:57.282720+0000","last_became_peered":"2024-06-10T01:01:57.282720+0000","last_unstale":"2024-06-10T01:03:38.770709+0000","last_undegraded":"2024-06-10T01:03:38.770709+0000","last_fullsized":"2024-06-10T01:03:38.770709+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:35:31.342042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235489+0000","last_change":"2024-06-10T01:01:44.009278+0000","last_active":"2024-06-10T01:03:38.235489+0000","last_peered":"2024-06-10T01:03:38.235489+0000","last_clean":"2024-06-10T01:03:38.235489+0000","last_became_active":"2024-06-10T01:01:44.008841+0000","last_became_peered":"2024-06-10T01:01:44.008841+0000","last_unstale":"2024-06-10T01:03:38.235489+0000","last_undegraded":"2024-06-10T01:03:38.235489+0000","last_fullsized":"2024-06-10T01:03:38.235489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:30:58.392080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.557141+0000","last_change":"2024-06-10T01:01:47.256451+0000","last_active":"2024-06-10T01:04:35.557141+0000","last_peered":"2024-06-10T01:04:35.557141+0000","last_clean":"2024-06-10T01:04:35.557141+0000","last_became_active":"2024-06-10T01:01:47.256036+0000","last_became_peered":"2024-06-10T01:01:47.256036+0000","last_unstale":"2024-06-10T01:04:35.557141+0000","last_undegraded":"2024-06-10T01:04:35.557141+0000","last_fullsized":"2024-06-10T01:04:35.557141+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:57:26.640109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239192+0000","last_change":"2024-06-10T01:02:37.794160+0000","last_active":"2024-06-10T01:03:38.239192+0000","last_peered":"2024-06-10T01:03:38.239192+0000","last_clean":"2024-06-10T01:03:38.239192+0000","last_became_active":"2024-06-10T01:02:37.793633+0000","last_became_peered":"2024-06-10T01:02:37.793633+0000","last_unstale":"2024-06-10T01:03:38.239192+0000","last_undegraded":"2024-06-10T01:03:38.239192+0000","last_fullsized":"2024-06-10T01:03:38.239192+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:48:48.083620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270125+0000","last_change":"2024-06-10T01:01:44.469179+0000","last_active":"2024-06-10T01:03:38.270125+0000","last_peered":"2024-06-10T01:03:38.270125+0000","last_clean":"2024-06-10T01:03:38.270125+0000","last_became_active":"2024-06-10T01:01:44.468669+0000","last_became_peered":"2024-06-10T01:01:44.468669+0000","last_unstale":"2024-06-10T01:03:38.270125+0000","last_undegraded":"2024-06-10T01:03:38.270125+0000","last_fullsized":"2024-06-10T01:03:38.270125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:07:51.629030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.570053+0000","last_change":"2024-06-10T01:01:47.247588+0000","last_active":"2024-06-10T01:04:35.570053+0000","last_peered":"2024-06-10T01:04:35.570053+0000","last_clean":"2024-06-10T01:04:35.570053+0000","last_became_active":"2024-06-10T01:01:47.245750+0000","last_became_peered":"2024-06-10T01:01:47.245750+0000","last_unstale":"2024-06-10T01:04:35.570053+0000","last_undegraded":"2024-06-10T01:04:35.570053+0000","last_fullsized":"2024-06-10T01:04:35.570053+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:12:10.513777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771292+0000","last_change":"2024-06-10T01:01:57.283042+0000","last_active":"2024-06-10T01:03:38.771292+0000","last_peered":"2024-06-10T01:03:38.771292+0000","last_clean":"2024-06-10T01:03:38.771292+0000","last_became_active":"2024-06-10T01:01:57.282600+0000","last_became_peered":"2024-06-10T01:01:57.282600+0000","last_unstale":"2024-06-10T01:03:38.771292+0000","last_undegraded":"2024-06-10T01:03:38.771292+0000","last_fullsized":"2024-06-10T01:03:38.771292+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:06:36.969968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"99'40","reported_seq":447,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:49.964777+0000","last_change":"2024-06-10T01:01:44.011783+0000","last_active":"2024-06-10T01:05:49.964777+0000","last_peered":"2024-06-10T01:05:49.964777+0000","last_clean":"2024-06-10T01:05:49.964777+0000","last_became_active":"2024-06-10T01:01:44.011164+0000","last_became_peered":"2024-06-10T01:01:44.011164+0000","last_unstale":"2024-06-10T01:05:49.964777+0000","last_undegraded":"2024-06-10T01:05:49.964777+0000","last_fullsized":"2024-06-10T01:05:49.964777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:16:13.642714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68929,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":362,"num_read_kb":249,"num_write":41,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562407+0000","last_change":"2024-06-10T01:01:47.250751+0000","last_active":"2024-06-10T01:04:35.562407+0000","last_peered":"2024-06-10T01:04:35.562407+0000","last_clean":"2024-06-10T01:04:35.562407+0000","last_became_active":"2024-06-10T01:01:47.250451+0000","last_became_peered":"2024-06-10T01:01:47.250451+0000","last_unstale":"2024-06-10T01:04:35.562407+0000","last_undegraded":"2024-06-10T01:04:35.562407+0000","last_fullsized":"2024-06-10T01:04:35.562407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:22:10.121363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270382+0000","last_change":"2024-06-10T01:01:57.289687+0000","last_active":"2024-06-10T01:03:38.270382+0000","last_peered":"2024-06-10T01:03:38.270382+0000","last_clean":"2024-06-10T01:03:38.270382+0000","last_became_active":"2024-06-10T01:01:57.289274+0000","last_became_peered":"2024-06-10T01:01:57.289274+0000","last_unstale":"2024-06-10T01:03:38.270382+0000","last_undegraded":"2024-06-10T01:03:38.270382+0000","last_fullsized":"2024-06-10T01:03:38.270382+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:34:46.873974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'6","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239529+0000","last_change":"2024-06-10T01:01:44.010159+0000","last_active":"2024-06-10T01:03:38.239529+0000","last_peered":"2024-06-10T01:03:38.239529+0000","last_clean":"2024-06-10T01:03:38.239529+0000","last_became_active":"2024-06-10T01:01:44.009744+0000","last_became_peered":"2024-06-10T01:01:44.009744+0000","last_unstale":"2024-06-10T01:03:38.239529+0000","last_undegraded":"2024-06-10T01:03:38.239529+0000","last_fullsized":"2024-06-10T01:03:38.239529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:30.507613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558994+0000","last_change":"2024-06-10T01:01:47.246662+0000","last_active":"2024-06-10T01:04:35.558994+0000","last_peered":"2024-06-10T01:04:35.558994+0000","last_clean":"2024-06-10T01:04:35.558994+0000","last_became_active":"2024-06-10T01:01:47.246268+0000","last_became_peered":"2024-06-10T01:01:47.246268+0000","last_unstale":"2024-06-10T01:04:35.558994+0000","last_undegraded":"2024-06-10T01:04:35.558994+0000","last_fullsized":"2024-06-10T01:04:35.558994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:08:38.089054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.236925+0000","last_change":"2024-06-10T01:01:57.289937+0000","last_active":"2024-06-10T01:03:38.236925+0000","last_peered":"2024-06-10T01:03:38.236925+0000","last_clean":"2024-06-10T01:03:38.236925+0000","last_became_active":"2024-06-10T01:01:57.289407+0000","last_became_peered":"2024-06-10T01:01:57.289407+0000","last_unstale":"2024-06-10T01:03:38.236925+0000","last_undegraded":"2024-06-10T01:03:38.236925+0000","last_fullsized":"2024-06-10T01:03:38.236925+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:01:48.546762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270079+0000","last_change":"2024-06-10T01:01:44.012654+0000","last_active":"2024-06-10T01:03:38.270079+0000","last_peered":"2024-06-10T01:03:38.270079+0000","last_clean":"2024-06-10T01:03:38.270079+0000","last_became_active":"2024-06-10T01:01:44.012246+0000","last_became_peered":"2024-06-10T01:01:44.012246+0000","last_unstale":"2024-06-10T01:03:38.270079+0000","last_undegraded":"2024-06-10T01:03:38.270079+0000","last_fullsized":"2024-06-10T01:03:38.270079+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:31:36.173554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558783+0000","last_change":"2024-06-10T01:01:47.250514+0000","last_active":"2024-06-10T01:04:35.558783+0000","last_peered":"2024-06-10T01:04:35.558783+0000","last_clean":"2024-06-10T01:04:35.558783+0000","last_became_active":"2024-06-10T01:01:47.249050+0000","last_became_peered":"2024-06-10T01:01:47.249050+0000","last_unstale":"2024-06-10T01:04:35.558783+0000","last_undegraded":"2024-06-10T01:04:35.558783+0000","last_fullsized":"2024-06-10T01:04:35.558783+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:51:30.789392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771928+0000","last_change":"2024-06-10T01:01:57.282389+0000","last_active":"2024-06-10T01:03:38.771928+0000","last_peered":"2024-06-10T01:03:38.771928+0000","last_clean":"2024-06-10T01:03:38.771928+0000","last_became_active":"2024-06-10T01:01:57.281857+0000","last_became_peered":"2024-06-10T01:01:57.281857+0000","last_unstale":"2024-06-10T01:03:38.771928+0000","last_undegraded":"2024-06-10T01:03:38.771928+0000","last_fullsized":"2024-06-10T01:03:38.771928+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:59:14.863020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"99'4","reported_seq":225,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:46.961953+0000","last_change":"2024-06-10T01:01:44.011331+0000","last_active":"2024-06-10T01:05:46.961953+0000","last_peered":"2024-06-10T01:05:46.961953+0000","last_clean":"2024-06-10T01:05:46.961953+0000","last_became_active":"2024-06-10T01:01:44.010968+0000","last_became_peered":"2024-06-10T01:01:44.010968+0000","last_unstale":"2024-06-10T01:05:46.961953+0000","last_undegraded":"2024-06-10T01:05:46.961953+0000","last_fullsized":"2024-06-10T01:05:46.961953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:43:37.931225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":896,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":177,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562879+0000","last_change":"2024-06-10T01:01:47.248349+0000","last_active":"2024-06-10T01:04:35.562879+0000","last_peered":"2024-06-10T01:04:35.562879+0000","last_clean":"2024-06-10T01:04:35.562879+0000","last_became_active":"2024-06-10T01:01:47.248008+0000","last_became_peered":"2024-06-10T01:01:47.248008+0000","last_unstale":"2024-06-10T01:04:35.562879+0000","last_undegraded":"2024-06-10T01:04:35.562879+0000","last_fullsized":"2024-06-10T01:04:35.562879+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:45:20.747704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771221+0000","last_change":"2024-06-10T01:01:57.289773+0000","last_active":"2024-06-10T01:03:38.771221+0000","last_peered":"2024-06-10T01:03:38.771221+0000","last_clean":"2024-06-10T01:03:38.771221+0000","last_became_active":"2024-06-10T01:01:57.289371+0000","last_became_peered":"2024-06-10T01:01:57.289371+0000","last_unstale":"2024-06-10T01:03:38.771221+0000","last_undegraded":"2024-06-10T01:03:38.771221+0000","last_fullsized":"2024-06-10T01:03:38.771221+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:24:52.209402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'15","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.257548+0000","last_change":"2024-06-10T01:03:38.257122+0000","last_active":"2024-06-10T01:03:38.257548+0000","last_peered":"2024-06-10T01:03:38.257548+0000","last_clean":"2024-06-10T01:03:38.257548+0000","last_became_active":"2024-06-10T01:03:38.240737+0000","last_became_peered":"2024-06-10T01:03:38.240737+0000","last_unstale":"2024-06-10T01:03:38.257548+0000","last_undegraded":"2024-06-10T01:03:38.257548+0000","last_fullsized":"2024-06-10T01:03:38.257548+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:37:10.925004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562425+0000","last_change":"2024-06-10T01:01:47.245439+0000","last_active":"2024-06-10T01:04:35.562425+0000","last_peered":"2024-06-10T01:04:35.562425+0000","last_clean":"2024-06-10T01:04:35.562425+0000","last_became_active":"2024-06-10T01:01:47.242917+0000","last_became_peered":"2024-06-10T01:01:47.242917+0000","last_unstale":"2024-06-10T01:04:35.562425+0000","last_undegraded":"2024-06-10T01:04:35.562425+0000","last_fullsized":"2024-06-10T01:04:35.562425+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:13:12.614230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.269795+0000","last_change":"2024-06-10T01:01:57.284731+0000","last_active":"2024-06-10T01:03:38.269795+0000","last_peered":"2024-06-10T01:03:38.269795+0000","last_clean":"2024-06-10T01:03:38.269795+0000","last_became_active":"2024-06-10T01:01:57.284301+0000","last_became_peered":"2024-06-10T01:01:57.284301+0000","last_unstale":"2024-06-10T01:03:38.269795+0000","last_undegraded":"2024-06-10T01:03:38.269795+0000","last_fullsized":"2024-06-10T01:03:38.269795+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:01:36.518214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"99'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:59.314831+0000","last_change":"2024-06-10T01:01:44.010154+0000","last_active":"2024-06-10T01:03:59.314831+0000","last_peered":"2024-06-10T01:03:59.314831+0000","last_clean":"2024-06-10T01:03:59.314831+0000","last_became_active":"2024-06-10T01:01:44.009656+0000","last_became_peered":"2024-06-10T01:01:44.009656+0000","last_unstale":"2024-06-10T01:03:59.314831+0000","last_undegraded":"2024-06-10T01:03:59.314831+0000","last_fullsized":"2024-06-10T01:03:59.314831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:33:08.789946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.550140+0000","last_change":"2024-06-10T01:02:37.378484+0000","last_active":"2024-06-10T01:04:35.550140+0000","last_peered":"2024-06-10T01:04:35.550140+0000","last_clean":"2024-06-10T01:04:35.550140+0000","last_became_active":"2024-06-10T01:02:37.378092+0000","last_became_peered":"2024-06-10T01:02:37.378092+0000","last_unstale":"2024-06-10T01:04:35.550140+0000","last_undegraded":"2024-06-10T01:04:35.550140+0000","last_fullsized":"2024-06-10T01:04:35.550140+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:19:46.599235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235375+0000","last_change":"2024-06-10T01:01:57.332887+0000","last_active":"2024-06-10T01:03:38.235375+0000","last_peered":"2024-06-10T01:03:38.235375+0000","last_clean":"2024-06-10T01:03:38.235375+0000","last_became_active":"2024-06-10T01:01:57.332285+0000","last_became_peered":"2024-06-10T01:01:57.332285+0000","last_unstale":"2024-06-10T01:03:38.235375+0000","last_undegraded":"2024-06-10T01:03:38.235375+0000","last_fullsized":"2024-06-10T01:03:38.235375+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:09:12.233405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771642+0000","last_change":"2024-06-10T01:01:44.466831+0000","last_active":"2024-06-10T01:03:38.771642+0000","last_peered":"2024-06-10T01:03:38.771642+0000","last_clean":"2024-06-10T01:03:38.771642+0000","last_became_active":"2024-06-10T01:01:44.466484+0000","last_became_peered":"2024-06-10T01:01:44.466484+0000","last_unstale":"2024-06-10T01:03:38.771642+0000","last_undegraded":"2024-06-10T01:03:38.771642+0000","last_fullsized":"2024-06-10T01:03:38.771642+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:19:28.385504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.556135+0000","last_change":"2024-06-10T01:01:47.252399+0000","last_active":"2024-06-10T01:04:35.556135+0000","last_peered":"2024-06-10T01:04:35.556135+0000","last_clean":"2024-06-10T01:04:35.556135+0000","last_became_active":"2024-06-10T01:01:47.252069+0000","last_became_peered":"2024-06-10T01:01:47.252069+0000","last_unstale":"2024-06-10T01:04:35.556135+0000","last_undegraded":"2024-06-10T01:04:35.556135+0000","last_fullsized":"2024-06-10T01:04:35.556135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:02:02.291174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771406+0000","last_change":"2024-06-10T01:01:57.285670+0000","last_active":"2024-06-10T01:03:38.771406+0000","last_peered":"2024-06-10T01:03:38.771406+0000","last_clean":"2024-06-10T01:03:38.771406+0000","last_became_active":"2024-06-10T01:01:57.285281+0000","last_became_peered":"2024-06-10T01:01:57.285281+0000","last_unstale":"2024-06-10T01:03:38.771406+0000","last_undegraded":"2024-06-10T01:03:38.771406+0000","last_fullsized":"2024-06-10T01:03:38.771406+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:58.956934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235189+0000","last_change":"2024-06-10T01:01:44.008013+0000","last_active":"2024-06-10T01:03:38.235189+0000","last_peered":"2024-06-10T01:03:38.235189+0000","last_clean":"2024-06-10T01:03:38.235189+0000","last_became_active":"2024-06-10T01:01:44.007591+0000","last_became_peered":"2024-06-10T01:01:44.007591+0000","last_unstale":"2024-06-10T01:03:38.235189+0000","last_undegraded":"2024-06-10T01:03:38.235189+0000","last_fullsized":"2024-06-10T01:03:38.235189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:18:20.862340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.550926+0000","last_change":"2024-06-10T01:01:47.254756+0000","last_active":"2024-06-10T01:04:35.550926+0000","last_peered":"2024-06-10T01:04:35.550926+0000","last_clean":"2024-06-10T01:04:35.550926+0000","last_became_active":"2024-06-10T01:01:47.254418+0000","last_became_peered":"2024-06-10T01:01:47.254418+0000","last_unstale":"2024-06-10T01:04:35.550926+0000","last_undegraded":"2024-06-10T01:04:35.550926+0000","last_fullsized":"2024-06-10T01:04:35.550926+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:46:31.452539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770995+0000","last_change":"2024-06-10T01:01:57.332792+0000","last_active":"2024-06-10T01:03:38.770995+0000","last_peered":"2024-06-10T01:03:38.770995+0000","last_clean":"2024-06-10T01:03:38.770995+0000","last_became_active":"2024-06-10T01:01:57.332331+0000","last_became_peered":"2024-06-10T01:01:57.332331+0000","last_unstale":"2024-06-10T01:03:38.770995+0000","last_undegraded":"2024-06-10T01:03:38.770995+0000","last_fullsized":"2024-06-10T01:03:38.770995+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:46:41.419807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.565745+0000","last_change":"2024-06-10T01:01:47.247857+0000","last_active":"2024-06-10T01:04:35.565745+0000","last_peered":"2024-06-10T01:04:35.565745+0000","last_clean":"2024-06-10T01:04:35.565745+0000","last_became_active":"2024-06-10T01:01:47.247552+0000","last_became_peered":"2024-06-10T01:01:47.247552+0000","last_unstale":"2024-06-10T01:04:35.565745+0000","last_undegraded":"2024-06-10T01:04:35.565745+0000","last_fullsized":"2024-06-10T01:04:35.565745+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:29:40.521887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"99'5","reported_seq":224,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:47.962652+0000","last_change":"2024-06-10T01:01:44.466631+0000","last_active":"2024-06-10T01:05:47.962652+0000","last_peered":"2024-06-10T01:05:47.962652+0000","last_clean":"2024-06-10T01:05:47.962652+0000","last_became_active":"2024-06-10T01:01:44.466280+0000","last_became_peered":"2024-06-10T01:01:44.466280+0000","last_unstale":"2024-06-10T01:05:47.962652+0000","last_undegraded":"2024-06-10T01:05:47.962652+0000","last_fullsized":"2024-06-10T01:05:47.962652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:09:33.658112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":174,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235365+0000","last_change":"2024-06-10T01:01:57.281973+0000","last_active":"2024-06-10T01:03:38.235365+0000","last_peered":"2024-06-10T01:03:38.235365+0000","last_clean":"2024-06-10T01:03:38.235365+0000","last_became_active":"2024-06-10T01:01:57.281386+0000","last_became_peered":"2024-06-10T01:01:57.281386+0000","last_unstale":"2024-06-10T01:03:38.235365+0000","last_undegraded":"2024-06-10T01:03:38.235365+0000","last_fullsized":"2024-06-10T01:03:38.235365+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:10:28.236250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562415+0000","last_change":"2024-06-10T01:01:47.237436+0000","last_active":"2024-06-10T01:04:35.562415+0000","last_peered":"2024-06-10T01:04:35.562415+0000","last_clean":"2024-06-10T01:04:35.562415+0000","last_became_active":"2024-06-10T01:01:47.237099+0000","last_became_peered":"2024-06-10T01:01:47.237099+0000","last_unstale":"2024-06-10T01:04:35.562415+0000","last_undegraded":"2024-06-10T01:04:35.562415+0000","last_fullsized":"2024-06-10T01:04:35.562415+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:34:53.838877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'3","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239178+0000","last_change":"2024-06-10T01:01:44.010655+0000","last_active":"2024-06-10T01:03:38.239178+0000","last_peered":"2024-06-10T01:03:38.239178+0000","last_clean":"2024-06-10T01:03:38.239178+0000","last_became_active":"2024-06-10T01:01:44.009918+0000","last_became_peered":"2024-06-10T01:01:44.009918+0000","last_unstale":"2024-06-10T01:03:38.239178+0000","last_undegraded":"2024-06-10T01:03:38.239178+0000","last_fullsized":"2024-06-10T01:03:38.239178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:47:32.052650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":367,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234906+0000","last_change":"2024-06-10T01:00:23.822447+0000","last_active":"2024-06-10T01:03:38.234906+0000","last_peered":"2024-06-10T01:03:38.234906+0000","last_clean":"2024-06-10T01:03:38.234906+0000","last_became_active":"2024-06-10T01:00:23.821966+0000","last_became_peered":"2024-06-10T01:00:23.821966+0000","last_unstale":"2024-06-10T01:03:38.234906+0000","last_undegraded":"2024-06-10T01:03:38.234906+0000","last_fullsized":"2024-06-10T01:03:38.234906+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:56:59.964661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:56:59.964661+0000","last_clean_scrub_stamp":"2024-06-10T00:56:59.964661+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:03:43.732875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270272+0000","last_change":"2024-06-10T01:01:57.332980+0000","last_active":"2024-06-10T01:03:38.270272+0000","last_peered":"2024-06-10T01:03:38.270272+0000","last_clean":"2024-06-10T01:03:38.270272+0000","last_became_active":"2024-06-10T01:01:57.332437+0000","last_became_peered":"2024-06-10T01:01:57.332437+0000","last_unstale":"2024-06-10T01:03:38.270272+0000","last_undegraded":"2024-06-10T01:03:38.270272+0000","last_fullsized":"2024-06-10T01:03:38.270272+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:27:35.029079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559754+0000","last_change":"2024-06-10T01:01:47.255926+0000","last_active":"2024-06-10T01:04:35.559754+0000","last_peered":"2024-06-10T01:04:35.559754+0000","last_clean":"2024-06-10T01:04:35.559754+0000","last_became_active":"2024-06-10T01:01:47.255571+0000","last_became_peered":"2024-06-10T01:01:47.255571+0000","last_unstale":"2024-06-10T01:04:35.559754+0000","last_undegraded":"2024-06-10T01:04:35.559754+0000","last_fullsized":"2024-06-10T01:04:35.559754+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:05:22.084037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:43.603848+0000","last_change":"2024-06-10T01:01:44.007509+0000","last_active":"2024-06-10T01:03:43.603848+0000","last_peered":"2024-06-10T01:03:43.603848+0000","last_clean":"2024-06-10T01:03:43.603848+0000","last_became_active":"2024-06-10T01:01:44.007178+0000","last_became_peered":"2024-06-10T01:01:44.007178+0000","last_unstale":"2024-06-10T01:03:43.603848+0000","last_undegraded":"2024-06-10T01:03:43.603848+0000","last_fullsized":"2024-06-10T01:03:43.603848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:02:52.791886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.236790+0000","last_change":"2024-06-10T01:01:57.285626+0000","last_active":"2024-06-10T01:03:38.236790+0000","last_peered":"2024-06-10T01:03:38.236790+0000","last_clean":"2024-06-10T01:03:38.236790+0000","last_became_active":"2024-06-10T01:01:57.285098+0000","last_became_peered":"2024-06-10T01:01:57.285098+0000","last_unstale":"2024-06-10T01:03:38.236790+0000","last_undegraded":"2024-06-10T01:03:38.236790+0000","last_fullsized":"2024-06-10T01:03:38.236790+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:13:49.881770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235332+0000","last_change":"2024-06-10T01:01:44.469028+0000","last_active":"2024-06-10T01:03:38.235332+0000","last_peered":"2024-06-10T01:03:38.235332+0000","last_clean":"2024-06-10T01:03:38.235332+0000","last_became_active":"2024-06-10T01:01:44.468570+0000","last_became_peered":"2024-06-10T01:01:44.468570+0000","last_unstale":"2024-06-10T01:03:38.235332+0000","last_undegraded":"2024-06-10T01:03:38.235332+0000","last_fullsized":"2024-06-10T01:03:38.235332+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:12:11.287959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.560160+0000","last_change":"2024-06-10T01:01:47.253414+0000","last_active":"2024-06-10T01:04:35.560160+0000","last_peered":"2024-06-10T01:04:35.560160+0000","last_clean":"2024-06-10T01:04:35.560160+0000","last_became_active":"2024-06-10T01:01:47.253060+0000","last_became_peered":"2024-06-10T01:01:47.253060+0000","last_unstale":"2024-06-10T01:04:35.560160+0000","last_undegraded":"2024-06-10T01:04:35.560160+0000","last_fullsized":"2024-06-10T01:04:35.560160+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:24:55.534164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.238849+0000","last_change":"2024-06-10T01:01:57.332756+0000","last_active":"2024-06-10T01:03:38.238849+0000","last_peered":"2024-06-10T01:03:38.238849+0000","last_clean":"2024-06-10T01:03:38.238849+0000","last_became_active":"2024-06-10T01:01:57.332361+0000","last_became_peered":"2024-06-10T01:01:57.332361+0000","last_unstale":"2024-06-10T01:03:38.238849+0000","last_undegraded":"2024-06-10T01:03:38.238849+0000","last_fullsized":"2024-06-10T01:03:38.238849+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:02:28.275407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"99'14","reported_seq":59,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:36.265950+0000","last_change":"2024-06-10T01:01:44.005164+0000","last_active":"2024-06-10T01:04:36.265950+0000","last_peered":"2024-06-10T01:04:36.265950+0000","last_clean":"2024-06-10T01:04:36.265950+0000","last_became_active":"2024-06-10T01:01:44.004816+0000","last_became_peered":"2024-06-10T01:01:44.004816+0000","last_unstale":"2024-06-10T01:04:36.265950+0000","last_undegraded":"2024-06-10T01:04:36.265950+0000","last_fullsized":"2024-06-10T01:04:36.265950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:11:56.385267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.557511+0000","last_change":"2024-06-10T01:01:47.247951+0000","last_active":"2024-06-10T01:04:35.557511+0000","last_peered":"2024-06-10T01:04:35.557511+0000","last_clean":"2024-06-10T01:04:35.557511+0000","last_became_active":"2024-06-10T01:01:47.246996+0000","last_became_peered":"2024-06-10T01:01:47.246996+0000","last_unstale":"2024-06-10T01:04:35.557511+0000","last_undegraded":"2024-06-10T01:04:35.557511+0000","last_fullsized":"2024-06-10T01:04:35.557511+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:39:18.167021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239228+0000","last_change":"2024-06-10T01:01:57.278130+0000","last_active":"2024-06-10T01:03:38.239228+0000","last_peered":"2024-06-10T01:03:38.239228+0000","last_clean":"2024-06-10T01:03:38.239228+0000","last_became_active":"2024-06-10T01:01:57.277697+0000","last_became_peered":"2024-06-10T01:01:57.277697+0000","last_unstale":"2024-06-10T01:03:38.239228+0000","last_undegraded":"2024-06-10T01:03:38.239228+0000","last_fullsized":"2024-06-10T01:03:38.239228+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:17:01.662152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239618+0000","last_change":"2024-06-10T01:01:44.553176+0000","last_active":"2024-06-10T01:03:38.239618+0000","last_peered":"2024-06-10T01:03:38.239618+0000","last_clean":"2024-06-10T01:03:38.239618+0000","last_became_active":"2024-06-10T01:01:44.552797+0000","last_became_peered":"2024-06-10T01:01:44.552797+0000","last_unstale":"2024-06-10T01:03:38.239618+0000","last_undegraded":"2024-06-10T01:03:38.239618+0000","last_fullsized":"2024-06-10T01:03:38.239618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:21:13.960304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.556104+0000","last_change":"2024-06-10T01:01:47.255796+0000","last_active":"2024-06-10T01:04:35.556104+0000","last_peered":"2024-06-10T01:04:35.556104+0000","last_clean":"2024-06-10T01:04:35.556104+0000","last_became_active":"2024-06-10T01:01:47.255495+0000","last_became_peered":"2024-06-10T01:01:47.255495+0000","last_unstale":"2024-06-10T01:04:35.556104+0000","last_undegraded":"2024-06-10T01:04:35.556104+0000","last_fullsized":"2024-06-10T01:04:35.556104+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:06:42.209005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239491+0000","last_change":"2024-06-10T01:01:57.332869+0000","last_active":"2024-06-10T01:03:38.239491+0000","last_peered":"2024-06-10T01:03:38.239491+0000","last_clean":"2024-06-10T01:03:38.239491+0000","last_became_active":"2024-06-10T01:01:57.332391+0000","last_became_peered":"2024-06-10T01:01:57.332391+0000","last_unstale":"2024-06-10T01:03:38.239491+0000","last_undegraded":"2024-06-10T01:03:38.239491+0000","last_fullsized":"2024-06-10T01:03:38.239491+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:58:31.443626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770788+0000","last_change":"2024-06-10T01:01:44.466861+0000","last_active":"2024-06-10T01:03:38.770788+0000","last_peered":"2024-06-10T01:03:38.770788+0000","last_clean":"2024-06-10T01:03:38.770788+0000","last_became_active":"2024-06-10T01:01:44.466553+0000","last_became_peered":"2024-06-10T01:01:44.466553+0000","last_unstale":"2024-06-10T01:03:38.770788+0000","last_undegraded":"2024-06-10T01:03:38.770788+0000","last_fullsized":"2024-06-10T01:03:38.770788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:55:52.545594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":41,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.544035+0000","last_change":"2024-06-10T01:01:47.244972+0000","last_active":"2024-06-10T01:04:35.544035+0000","last_peered":"2024-06-10T01:04:35.544035+0000","last_clean":"2024-06-10T01:04:35.544035+0000","last_became_active":"2024-06-10T01:01:47.244649+0000","last_became_peered":"2024-06-10T01:01:47.244649+0000","last_unstale":"2024-06-10T01:04:35.544035+0000","last_undegraded":"2024-06-10T01:04:35.544035+0000","last_fullsized":"2024-06-10T01:04:35.544035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:10:46.909200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235461+0000","last_change":"2024-06-10T01:02:37.592380+0000","last_active":"2024-06-10T01:03:38.235461+0000","last_peered":"2024-06-10T01:03:38.235461+0000","last_clean":"2024-06-10T01:03:38.235461+0000","last_became_active":"2024-06-10T01:02:37.591856+0000","last_became_peered":"2024-06-10T01:02:37.591856+0000","last_unstale":"2024-06-10T01:03:38.235461+0000","last_undegraded":"2024-06-10T01:03:38.235461+0000","last_fullsized":"2024-06-10T01:03:38.235461+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:00:25.717192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239006+0000","last_change":"2024-06-10T01:01:44.008116+0000","last_active":"2024-06-10T01:03:38.239006+0000","last_peered":"2024-06-10T01:03:38.239006+0000","last_clean":"2024-06-10T01:03:38.239006+0000","last_became_active":"2024-06-10T01:01:44.007778+0000","last_became_peered":"2024-06-10T01:01:44.007778+0000","last_unstale":"2024-06-10T01:03:38.239006+0000","last_undegraded":"2024-06-10T01:03:38.239006+0000","last_fullsized":"2024-06-10T01:03:38.239006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:18:57.267296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.556264+0000","last_change":"2024-06-10T01:01:47.252833+0000","last_active":"2024-06-10T01:04:35.556264+0000","last_peered":"2024-06-10T01:04:35.556264+0000","last_clean":"2024-06-10T01:04:35.556264+0000","last_became_active":"2024-06-10T01:01:47.252326+0000","last_became_peered":"2024-06-10T01:01:47.252326+0000","last_unstale":"2024-06-10T01:04:35.556264+0000","last_undegraded":"2024-06-10T01:04:35.556264+0000","last_fullsized":"2024-06-10T01:04:35.556264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:00:42.673838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239284+0000","last_change":"2024-06-10T01:01:57.275975+0000","last_active":"2024-06-10T01:03:38.239284+0000","last_peered":"2024-06-10T01:03:38.239284+0000","last_clean":"2024-06-10T01:03:38.239284+0000","last_became_active":"2024-06-10T01:01:57.275408+0000","last_became_peered":"2024-06-10T01:01:57.275408+0000","last_unstale":"2024-06-10T01:03:38.239284+0000","last_undegraded":"2024-06-10T01:03:38.239284+0000","last_fullsized":"2024-06-10T01:03:38.239284+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:12:42.376636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'1","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:40.912786+0000","last_change":"2024-06-10T01:03:38.243422+0000","last_active":"2024-06-10T01:03:40.912786+0000","last_peered":"2024-06-10T01:03:40.912786+0000","last_clean":"2024-06-10T01:03:40.912786+0000","last_became_active":"2024-06-10T01:03:38.239916+0000","last_became_peered":"2024-06-10T01:03:38.239916+0000","last_unstale":"2024-06-10T01:03:40.912786+0000","last_undegraded":"2024-06-10T01:03:40.912786+0000","last_fullsized":"2024-06-10T01:03:40.912786+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:30:01.399956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.555295+0000","last_change":"2024-06-10T01:01:47.243154+0000","last_active":"2024-06-10T01:04:35.555295+0000","last_peered":"2024-06-10T01:04:35.555295+0000","last_clean":"2024-06-10T01:04:35.555295+0000","last_became_active":"2024-06-10T01:01:47.242851+0000","last_became_peered":"2024-06-10T01:01:47.242851+0000","last_unstale":"2024-06-10T01:04:35.555295+0000","last_undegraded":"2024-06-10T01:04:35.555295+0000","last_fullsized":"2024-06-10T01:04:35.555295+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:41:05.857777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239584+0000","last_change":"2024-06-10T01:01:57.333613+0000","last_active":"2024-06-10T01:03:38.239584+0000","last_peered":"2024-06-10T01:03:38.239584+0000","last_clean":"2024-06-10T01:03:38.239584+0000","last_became_active":"2024-06-10T01:01:57.333019+0000","last_became_peered":"2024-06-10T01:01:57.333019+0000","last_unstale":"2024-06-10T01:03:38.239584+0000","last_undegraded":"2024-06-10T01:03:38.239584+0000","last_fullsized":"2024-06-10T01:03:38.239584+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:11.912885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235303+0000","last_change":"2024-06-10T01:01:44.008221+0000","last_active":"2024-06-10T01:03:38.235303+0000","last_peered":"2024-06-10T01:03:38.235303+0000","last_clean":"2024-06-10T01:03:38.235303+0000","last_became_active":"2024-06-10T01:01:44.007880+0000","last_became_peered":"2024-06-10T01:01:44.007880+0000","last_unstale":"2024-06-10T01:03:38.235303+0000","last_undegraded":"2024-06-10T01:03:38.235303+0000","last_fullsized":"2024-06-10T01:03:38.235303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:40:33.162925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558463+0000","last_change":"2024-06-10T01:01:47.254902+0000","last_active":"2024-06-10T01:04:35.558463+0000","last_peered":"2024-06-10T01:04:35.558463+0000","last_clean":"2024-06-10T01:04:35.558463+0000","last_became_active":"2024-06-10T01:01:47.254555+0000","last_became_peered":"2024-06-10T01:01:47.254555+0000","last_unstale":"2024-06-10T01:04:35.558463+0000","last_undegraded":"2024-06-10T01:04:35.558463+0000","last_fullsized":"2024-06-10T01:04:35.558463+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:03:17.571001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770786+0000","last_change":"2024-06-10T01:01:57.284491+0000","last_active":"2024-06-10T01:03:38.770786+0000","last_peered":"2024-06-10T01:03:38.770786+0000","last_clean":"2024-06-10T01:03:38.770786+0000","last_became_active":"2024-06-10T01:01:57.283795+0000","last_became_peered":"2024-06-10T01:01:57.283795+0000","last_unstale":"2024-06-10T01:03:38.770786+0000","last_undegraded":"2024-06-10T01:03:38.770786+0000","last_fullsized":"2024-06-10T01:03:38.770786+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:49:00.381664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239889+0000","last_change":"2024-06-10T01:01:44.553159+0000","last_active":"2024-06-10T01:03:38.239889+0000","last_peered":"2024-06-10T01:03:38.239889+0000","last_clean":"2024-06-10T01:03:38.239889+0000","last_became_active":"2024-06-10T01:01:44.552759+0000","last_became_peered":"2024-06-10T01:01:44.552759+0000","last_unstale":"2024-06-10T01:03:38.239889+0000","last_undegraded":"2024-06-10T01:03:38.239889+0000","last_fullsized":"2024-06-10T01:03:38.239889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:19:00.110790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559093+0000","last_change":"2024-06-10T01:01:47.247803+0000","last_active":"2024-06-10T01:04:35.559093+0000","last_peered":"2024-06-10T01:04:35.559093+0000","last_clean":"2024-06-10T01:04:35.559093+0000","last_became_active":"2024-06-10T01:01:47.246715+0000","last_became_peered":"2024-06-10T01:01:47.246715+0000","last_unstale":"2024-06-10T01:04:35.559093+0000","last_undegraded":"2024-06-10T01:04:35.559093+0000","last_fullsized":"2024-06-10T01:04:35.559093+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:22.429002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234192+0000","last_change":"2024-06-10T01:01:57.287668+0000","last_active":"2024-06-10T01:03:38.234192+0000","last_peered":"2024-06-10T01:03:38.234192+0000","last_clean":"2024-06-10T01:03:38.234192+0000","last_became_active":"2024-06-10T01:01:57.287060+0000","last_became_peered":"2024-06-10T01:01:57.287060+0000","last_unstale":"2024-06-10T01:03:38.234192+0000","last_undegraded":"2024-06-10T01:03:38.234192+0000","last_fullsized":"2024-06-10T01:03:38.234192+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:04:35.883613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270251+0000","last_change":"2024-06-10T01:01:44.013051+0000","last_active":"2024-06-10T01:03:38.270251+0000","last_peered":"2024-06-10T01:03:38.270251+0000","last_clean":"2024-06-10T01:03:38.270251+0000","last_became_active":"2024-06-10T01:01:44.012667+0000","last_became_peered":"2024-06-10T01:01:44.012667+0000","last_unstale":"2024-06-10T01:03:38.270251+0000","last_undegraded":"2024-06-10T01:03:38.270251+0000","last_fullsized":"2024-06-10T01:03:38.270251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:10:49.650082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.563083+0000","last_change":"2024-06-10T01:01:47.251209+0000","last_active":"2024-06-10T01:04:35.563083+0000","last_peered":"2024-06-10T01:04:35.563083+0000","last_clean":"2024-06-10T01:04:35.563083+0000","last_became_active":"2024-06-10T01:01:47.250858+0000","last_became_peered":"2024-06-10T01:01:47.250858+0000","last_unstale":"2024-06-10T01:04:35.563083+0000","last_undegraded":"2024-06-10T01:04:35.563083+0000","last_fullsized":"2024-06-10T01:04:35.563083+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:47.272727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.269600+0000","last_change":"2024-06-10T01:01:57.284187+0000","last_active":"2024-06-10T01:03:38.269600+0000","last_peered":"2024-06-10T01:03:38.269600+0000","last_clean":"2024-06-10T01:03:38.269600+0000","last_became_active":"2024-06-10T01:01:57.283739+0000","last_became_peered":"2024-06-10T01:01:57.283739+0000","last_unstale":"2024-06-10T01:03:38.269600+0000","last_undegraded":"2024-06-10T01:03:38.269600+0000","last_fullsized":"2024-06-10T01:03:38.269600+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:13:34.630708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771085+0000","last_change":"2024-06-10T01:01:44.005702+0000","last_active":"2024-06-10T01:03:38.771085+0000","last_peered":"2024-06-10T01:03:38.771085+0000","last_clean":"2024-06-10T01:03:38.771085+0000","last_became_active":"2024-06-10T01:01:44.005354+0000","last_became_peered":"2024-06-10T01:01:44.005354+0000","last_unstale":"2024-06-10T01:03:38.771085+0000","last_undegraded":"2024-06-10T01:03:38.771085+0000","last_fullsized":"2024-06-10T01:03:38.771085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:51:14.819688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558942+0000","last_change":"2024-06-10T01:01:47.250325+0000","last_active":"2024-06-10T01:04:35.558942+0000","last_peered":"2024-06-10T01:04:35.558942+0000","last_clean":"2024-06-10T01:04:35.558942+0000","last_became_active":"2024-06-10T01:01:47.249591+0000","last_became_peered":"2024-06-10T01:01:47.249591+0000","last_unstale":"2024-06-10T01:04:35.558942+0000","last_undegraded":"2024-06-10T01:04:35.558942+0000","last_fullsized":"2024-06-10T01:04:35.558942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:17:16.268717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.563635+0000","last_change":"2024-06-10T01:01:47.253449+0000","last_active":"2024-06-10T01:04:35.563635+0000","last_peered":"2024-06-10T01:04:35.563635+0000","last_clean":"2024-06-10T01:04:35.563635+0000","last_became_active":"2024-06-10T01:01:47.253144+0000","last_became_peered":"2024-06-10T01:01:47.253144+0000","last_unstale":"2024-06-10T01:04:35.563635+0000","last_undegraded":"2024-06-10T01:04:35.563635+0000","last_fullsized":"2024-06-10T01:04:35.563635+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:32:22.353409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.564016+0000","last_change":"2024-06-10T01:01:47.258241+0000","last_active":"2024-06-10T01:04:35.564016+0000","last_peered":"2024-06-10T01:04:35.564016+0000","last_clean":"2024-06-10T01:04:35.564016+0000","last_became_active":"2024-06-10T01:01:47.257823+0000","last_became_peered":"2024-06-10T01:01:47.257823+0000","last_unstale":"2024-06-10T01:04:35.564016+0000","last_undegraded":"2024-06-10T01:04:35.564016+0000","last_fullsized":"2024-06-10T01:04:35.564016+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:11:23.652479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559410+0000","last_change":"2024-06-10T01:01:47.256218+0000","last_active":"2024-06-10T01:04:35.559410+0000","last_peered":"2024-06-10T01:04:35.559410+0000","last_clean":"2024-06-10T01:04:35.559410+0000","last_became_active":"2024-06-10T01:01:47.255950+0000","last_became_peered":"2024-06-10T01:01:47.255950+0000","last_unstale":"2024-06-10T01:04:35.559410+0000","last_undegraded":"2024-06-10T01:04:35.559410+0000","last_fullsized":"2024-06-10T01:04:35.559410+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:59:37.283646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.556085+0000","last_change":"2024-06-10T01:01:47.253109+0000","last_active":"2024-06-10T01:04:35.556085+0000","last_peered":"2024-06-10T01:04:35.556085+0000","last_clean":"2024-06-10T01:04:35.556085+0000","last_became_active":"2024-06-10T01:01:47.252767+0000","last_became_peered":"2024-06-10T01:01:47.252767+0000","last_unstale":"2024-06-10T01:04:35.556085+0000","last_undegraded":"2024-06-10T01:04:35.556085+0000","last_fullsized":"2024-06-10T01:04:35.556085+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:57:13.177230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.550864+0000","last_change":"2024-06-10T01:02:37.375094+0000","last_active":"2024-06-10T01:04:35.550864+0000","last_peered":"2024-06-10T01:04:35.550864+0000","last_clean":"2024-06-10T01:04:35.550864+0000","last_became_active":"2024-06-10T01:02:37.374527+0000","last_became_peered":"2024-06-10T01:02:37.374527+0000","last_unstale":"2024-06-10T01:04:35.550864+0000","last_undegraded":"2024-06-10T01:04:35.550864+0000","last_fullsized":"2024-06-10T01:04:35.550864+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:34:41.205816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.551206+0000","last_change":"2024-06-10T01:01:47.250243+0000","last_active":"2024-06-10T01:04:35.551206+0000","last_peered":"2024-06-10T01:04:35.551206+0000","last_clean":"2024-06-10T01:04:35.551206+0000","last_became_active":"2024-06-10T01:01:47.249907+0000","last_became_peered":"2024-06-10T01:01:47.249907+0000","last_unstale":"2024-06-10T01:04:35.551206+0000","last_undegraded":"2024-06-10T01:04:35.551206+0000","last_fullsized":"2024-06-10T01:04:35.551206+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:31:32.933660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.564063+0000","last_change":"2024-06-10T01:01:47.254173+0000","last_active":"2024-06-10T01:04:35.564063+0000","last_peered":"2024-06-10T01:04:35.564063+0000","last_clean":"2024-06-10T01:04:35.564063+0000","last_became_active":"2024-06-10T01:01:47.253796+0000","last_became_peered":"2024-06-10T01:01:47.253796+0000","last_unstale":"2024-06-10T01:04:35.564063+0000","last_undegraded":"2024-06-10T01:04:35.564063+0000","last_fullsized":"2024-06-10T01:04:35.564063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:41:12.199114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":41,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.548118+0000","last_change":"2024-06-10T01:01:47.253920+0000","last_active":"2024-06-10T01:04:35.548118+0000","last_peered":"2024-06-10T01:04:35.548118+0000","last_clean":"2024-06-10T01:04:35.548118+0000","last_became_active":"2024-06-10T01:01:47.253288+0000","last_became_peered":"2024-06-10T01:01:47.253288+0000","last_unstale":"2024-06-10T01:04:35.548118+0000","last_undegraded":"2024-06-10T01:04:35.548118+0000","last_fullsized":"2024-06-10T01:04:35.548118+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:41:20.406263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.564816+0000","last_change":"2024-06-10T01:01:47.240772+0000","last_active":"2024-06-10T01:04:35.564816+0000","last_peered":"2024-06-10T01:04:35.564816+0000","last_clean":"2024-06-10T01:04:35.564816+0000","last_became_active":"2024-06-10T01:01:47.238258+0000","last_became_peered":"2024-06-10T01:01:47.238258+0000","last_unstale":"2024-06-10T01:04:35.564816+0000","last_undegraded":"2024-06-10T01:04:35.564816+0000","last_fullsized":"2024-06-10T01:04:35.564816+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:55:25.211434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":24,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.544703+0000","last_change":"2024-06-10T01:02:37.376724+0000","last_active":"2024-06-10T01:04:35.544703+0000","last_peered":"2024-06-10T01:04:35.544703+0000","last_clean":"2024-06-10T01:04:35.544703+0000","last_became_active":"2024-06-10T01:02:37.376329+0000","last_became_peered":"2024-06-10T01:02:37.376329+0000","last_unstale":"2024-06-10T01:04:35.544703+0000","last_undegraded":"2024-06-10T01:04:35.544703+0000","last_fullsized":"2024-06-10T01:04:35.544703+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:38:21.837958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.565197+0000","last_change":"2024-06-10T01:01:47.244463+0000","last_active":"2024-06-10T01:04:35.565197+0000","last_peered":"2024-06-10T01:04:35.565197+0000","last_clean":"2024-06-10T01:04:35.565197+0000","last_became_active":"2024-06-10T01:01:47.244114+0000","last_became_peered":"2024-06-10T01:01:47.244114+0000","last_unstale":"2024-06-10T01:04:35.565197+0000","last_undegraded":"2024-06-10T01:04:35.565197+0000","last_fullsized":"2024-06-10T01:04:35.565197+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:47:37.098894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559211+0000","last_change":"2024-06-10T01:01:47.248317+0000","last_active":"2024-06-10T01:04:35.559211+0000","last_peered":"2024-06-10T01:04:35.559211+0000","last_clean":"2024-06-10T01:04:35.559211+0000","last_became_active":"2024-06-10T01:01:47.247958+0000","last_became_peered":"2024-06-10T01:01:47.247958+0000","last_unstale":"2024-06-10T01:04:35.559211+0000","last_undegraded":"2024-06-10T01:04:35.559211+0000","last_fullsized":"2024-06-10T01:04:35.559211+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:39:59.267957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.552389+0000","last_change":"2024-06-10T01:01:47.252542+0000","last_active":"2024-06-10T01:04:35.552389+0000","last_peered":"2024-06-10T01:04:35.552389+0000","last_clean":"2024-06-10T01:04:35.552389+0000","last_became_active":"2024-06-10T01:01:47.252216+0000","last_became_peered":"2024-06-10T01:01:47.252216+0000","last_unstale":"2024-06-10T01:04:35.552389+0000","last_undegraded":"2024-06-10T01:04:35.552389+0000","last_fullsized":"2024-06-10T01:04:35.552389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:12:22.196263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285691,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72847,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1079,"num_read_kb":616,"num_write":184,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":218541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383745,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":569843,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645382,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32392,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712760,"statfs":{"total":95995035648,"available":95961866240,"internally_reserved":0,"allocated":1245184,"data_stored":1226236,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808907,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":574063,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070544,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":569775,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332180,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":1165713,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495705,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":565143,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574962,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1163264,"data_stored":633982,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738488,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":566531,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411437,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1179648,"data_stored":635220,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182505,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":565337,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920867,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":570213,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757343,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":1160993,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34687,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34687,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34687,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":70167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T01:05:55.726 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-10T01:05:55.729 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:05:56.459 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:05:57.524 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:57 smithi007 ceph-mon[32398]: pgmap v429: 129 pgs: 129 active+clean; 648 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-10T01:05:57.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:57 smithi078 ceph-mon[36683]: pgmap v429: 129 pgs: 129 active+clean; 648 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-10T01:05:57.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:57 smithi175 ceph-mon[36636]: pgmap v429: 129 pgs: 129 active+clean; 648 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-10T01:05:58.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.206+0000 7f476abd2700 1 -- 172.21.15.7:0/3142475078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764102900 msgr2=0x7f476410e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:58.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.206+0000 7f476abd2700 1 --2- 172.21.15.7:0/3142475078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764102900 0x7f476410e310 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f475c00b0d0 tx=0x7f475c00b3e0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.206+0000 7f476abd2700 1 -- 172.21.15.7:0/3142475078 shutdown_connections 2024-06-10T01:05:58.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.206+0000 7f476abd2700 1 --2- 172.21.15.7:0/3142475078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764102900 0x7f476410e310 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.206+0000 7f476abd2700 1 --2- 172.21.15.7:0/3142475078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4764107660 0x7f4764107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.206+0000 7f476abd2700 1 --2- 172.21.15.7:0/3142475078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4764101f40 0x7f47641023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.206+0000 7f476abd2700 1 -- 172.21.15.7:0/3142475078 >> 172.21.15.7:0/3142475078 conn(0x7f47640766b0 msgr2=0x7f4764076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:58.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.206+0000 7f476abd2700 1 -- 172.21.15.7:0/3142475078 shutdown_connections 2024-06-10T01:05:58.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.206+0000 7f476abd2700 1 -- 172.21.15.7:0/3142475078 wait complete. 2024-06-10T01:05:58.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.207+0000 7f476abd2700 1 Processor -- start 2024-06-10T01:05:58.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.207+0000 7f476abd2700 1 -- start start 2024-06-10T01:05:58.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.207+0000 7f476abd2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4764101f40 0x7f4764199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:58.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.207+0000 7f476abd2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4764102900 0x7f4764199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:58.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.207+0000 7f476abd2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764107660 0x7f47641a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:58.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.207+0000 7f476abd2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f47641117b0 con 0x7f4764107660 2024-06-10T01:05:58.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.207+0000 7f476abd2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4764111630 con 0x7f4764102900 2024-06-10T01:05:58.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.207+0000 7f476abd2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4764111930 con 0x7f4764101f40 2024-06-10T01:05:58.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.207+0000 7f476916f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764107660 0x7f47641a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:58.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.208+0000 7f476916f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764107660 0x7f47641a4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40056/0 (socket says 172.21.15.7:40056) 2024-06-10T01:05:58.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.208+0000 7f476916f700 1 -- 172.21.15.7:0/3200595947 learned_addr learned my addr 172.21.15.7:0/3200595947 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:58.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.208+0000 7f476896e700 1 --2- 172.21.15.7:0/3200595947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4764101f40 0x7f4764199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:58.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.208+0000 7f475bfff700 1 --2- 172.21.15.7:0/3200595947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4764102900 0x7f4764199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:58.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.208+0000 7f476916f700 1 -- 172.21.15.7:0/3200595947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4764101f40 msgr2=0x7f4764199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:58.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.208+0000 7f476916f700 1 --2- 172.21.15.7:0/3200595947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4764101f40 0x7f4764199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.208+0000 7f476916f700 1 -- 172.21.15.7:0/3200595947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4764102900 msgr2=0x7f4764199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:58.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.208+0000 7f476916f700 1 --2- 172.21.15.7:0/3200595947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4764102900 0x7f4764199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.208+0000 7f476916f700 1 -- 172.21.15.7:0/3200595947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47641a5070 con 0x7f4764107660 2024-06-10T01:05:58.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.208+0000 7f476916f700 1 --2- 172.21.15.7:0/3200595947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764107660 0x7f47641a4b30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f475c009850 tx=0x7f475c007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:58.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.208+0000 7f4759ffb700 1 -- 172.21.15.7:0/3200595947 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c021070 con 0x7f4764107660 2024-06-10T01:05:58.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.208+0000 7f476abd2700 1 -- 172.21.15.7:0/3200595947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47641a5240 con 0x7f4764107660 2024-06-10T01:05:58.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.208+0000 7f4759ffb700 1 -- 172.21.15.7:0/3200595947 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f475c01db30 con 0x7f4764107660 2024-06-10T01:05:58.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.208+0000 7f4759ffb700 1 -- 172.21.15.7:0/3200595947 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c01ba10 con 0x7f4764107660 2024-06-10T01:05:58.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.209+0000 7f476abd2700 1 -- 172.21.15.7:0/3200595947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47641a56a0 con 0x7f4764107660 2024-06-10T01:05:58.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.211+0000 7f4759ffb700 1 -- 172.21.15.7:0/3200595947 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f475c024070 con 0x7f4764107660 2024-06-10T01:05:58.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.212+0000 7f4759ffb700 1 --2- 172.21.15.7:0/3200595947 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f474c0a2c20 0x7f474c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:58.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.212+0000 7f4759ffb700 1 -- 172.21.15.7:0/3200595947 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f475c0d0310 con 0x7f4764107660 2024-06-10T01:05:58.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.212+0000 7f476896e700 1 --2- 172.21.15.7:0/3200595947 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f474c0a2c20 0x7f474c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:58.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.213+0000 7f476896e700 1 --2- 172.21.15.7:0/3200595947 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f474c0a2c20 0x7f474c0a50e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4760005950 tx=0x7f476000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:58.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.213+0000 7f4775455b80 1 Processor -- start 2024-06-10T01:05:58.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.213+0000 7f4775455b80 1 -- start start 2024-06-10T01:05:58.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.213+0000 7f4775455b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f93a3a50 0x5648f93a3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:58.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.213+0000 7f4775455b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648f93a4370 0x5648f939ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:58.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.213+0000 7f4775455b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5648f939e330 0x5648f939e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:58.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.213+0000 7f4775455b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5648f92eb810 con 0x5648f939e330 2024-06-10T01:05:58.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.213+0000 7f4775455b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5648f9293720 con 0x5648f93a4370 2024-06-10T01:05:58.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.213+0000 7f4775455b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5648f92e77d0 con 0x5648f93a3a50 2024-06-10T01:05:58.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.213+0000 7f476916f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5648f939e330 0x5648f939e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:58.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.213+0000 7f476916f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5648f939e330 0x5648f939e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40058/0 (socket says 172.21.15.7:40058) 2024-06-10T01:05:58.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.214+0000 7f476916f700 1 -- 172.21.15.7:0/1899197154 learned_addr learned my addr 172.21.15.7:0/1899197154 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.214+0000 7f476896e700 1 --2- 172.21.15.7:0/1899197154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f93a3a50 0x5648f93a3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.214+0000 7f475bfff700 1 --2- 172.21.15.7:0/1899197154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648f93a4370 0x5648f939ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.214+0000 7f476916f700 1 -- 172.21.15.7:0/1899197154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f93a3a50 msgr2=0x5648f93a3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.214+0000 7f476916f700 1 --2- 172.21.15.7:0/1899197154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f93a3a50 0x5648f93a3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.214+0000 7f476916f700 1 -- 172.21.15.7:0/1899197154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648f93a4370 msgr2=0x5648f939ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.214+0000 7f476916f700 1 --2- 172.21.15.7:0/1899197154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648f93a4370 0x5648f939ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.214+0000 7f476916f700 1 -- 172.21.15.7:0/1899197154 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648f9289f50 con 0x5648f939e330 2024-06-10T01:05:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.214+0000 7f476916f700 1 --2- 172.21.15.7:0/1899197154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5648f939e330 0x5648f939e750 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f475c09c580 tx=0x7f475c0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75cc713f2b0dfbef server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.214+0000 7f474affd700 1 -- 172.21.15.7:0/1899197154 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c02b070 con 0x5648f939e330 2024-06-10T01:05:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.214+0000 7f474affd700 1 -- 172.21.15.7:0/1899197154 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f475c0a0c30 con 0x5648f939e330 2024-06-10T01:05:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.214+0000 7f474affd700 1 -- 172.21.15.7:0/1899197154 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c098ef0 con 0x5648f939e330 2024-06-10T01:05:58.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.215+0000 7f4775455b80 1 -- 172.21.15.7:0/1899197154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5648f939e330 msgr2=0x5648f939e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:58.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.215+0000 7f4775455b80 1 --2- 172.21.15.7:0/1899197154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5648f939e330 0x5648f939e750 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f475c09c580 tx=0x7f475c0099e0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.215+0000 7f4775455b80 1 -- 172.21.15.7:0/1899197154 shutdown_connections 2024-06-10T01:05:58.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.215+0000 7f4775455b80 1 --2- 172.21.15.7:0/1899197154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5648f939e330 0x5648f939e750 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.215+0000 7f4775455b80 1 --2- 172.21.15.7:0/1899197154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f93a3a50 0x5648f93a3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.215+0000 7f4775455b80 1 --2- 172.21.15.7:0/1899197154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648f93a4370 0x5648f939ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.215+0000 7f4775455b80 1 -- 172.21.15.7:0/1899197154 >> 172.21.15.7:0/1899197154 conn(0x5648f92c4cc0 msgr2=0x5648f9394850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:58.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.215+0000 7f4775455b80 1 -- 172.21.15.7:0/1899197154 shutdown_connections 2024-06-10T01:05:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.215+0000 7f4775455b80 1 -- 172.21.15.7:0/1899197154 wait complete. 2024-06-10T01:05:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.215+0000 7f4775455b80 1 -- start start 2024-06-10T01:05:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.216+0000 7f4775455b80 1 Processor -- start 2024-06-10T01:05:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.216+0000 7f4775455b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5648f939e330 0x5648f93aa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.216+0000 7f4775455b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648f93a3a50 0x5648f93a7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.216+0000 7f4775455b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f93a4370 0x5648f93a79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.216+0000 7f4775455b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5648f93aab00 con 0x5648f939e330 2024-06-10T01:05:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.216+0000 7f4775455b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5648f928bbf0 con 0x5648f93a3a50 2024-06-10T01:05:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.216+0000 7f4775455b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5648f93aac80 con 0x5648f93a4370 2024-06-10T01:05:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.216+0000 7f4775455b80 10 client.0 _ll_register_callbacks cb 0x5648f9285920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:05:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.216+0000 7f476896e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5648f939e330 0x5648f93aa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.216+0000 7f476896e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5648f939e330 0x5648f93aa0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40066/0 (socket says 172.21.15.7:40066) 2024-06-10T01:05:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.216+0000 7f476896e700 1 -- 172.21.15.7:0/1893190725 learned_addr learned my addr 172.21.15.7:0/1893190725 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.216+0000 7f475bfff700 1 --2- 172.21.15.7:0/1893190725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648f93a3a50 0x5648f93a7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.216+0000 7f476916f700 1 --2- 172.21.15.7:0/1893190725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f93a4370 0x5648f93a79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.217+0000 7f476896e700 1 -- 172.21.15.7:0/1893190725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f93a4370 msgr2=0x5648f93a79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.217+0000 7f476896e700 1 --2- 172.21.15.7:0/1893190725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f93a4370 0x5648f93a79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.217+0000 7f476896e700 1 -- 172.21.15.7:0/1893190725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648f93a3a50 msgr2=0x5648f93a7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.217+0000 7f476896e700 1 --2- 172.21.15.7:0/1893190725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648f93a3a50 0x5648f93a7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.217+0000 7f476896e700 1 -- 172.21.15.7:0/1893190725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648f923a8d0 con 0x5648f939e330 2024-06-10T01:05:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.217+0000 7f476896e700 1 --2- 172.21.15.7:0/1893190725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5648f939e330 0x5648f93aa0f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f47600116f0 tx=0x7f47600047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a3d2fe3453f677 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.217+0000 7f4743fff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:05:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.217+0000 7f4743fff700 1 -- 172.21.15.7:0/1893190725 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4760004a30 con 0x5648f939e330 2024-06-10T01:05:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.217+0000 7f4743fff700 1 -- 172.21.15.7:0/1893190725 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f4760004bd0 con 0x5648f939e330 2024-06-10T01:05:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.217+0000 7f4775455b80 1 -- 172.21.15.7:0/1893190725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f475c0a0c30 con 0x5648f939e330 2024-06-10T01:05:58.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.217+0000 7f4743fff700 1 -- 172.21.15.7:0/1893190725 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4760004dd0 con 0x5648f939e330 2024-06-10T01:05:58.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.217+0000 7f4775455b80 10 client.15312 fetch_fsmap learned FSMap version 27 2024-06-10T01:05:58.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.217+0000 7f4775455b80 1 -- 172.21.15.7:0/1893190725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5648f93b5400 con 0x5648f939e330 2024-06-10T01:05:58.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.217+0000 7f4743fff700 1 -- 172.21.15.7:0/1893190725 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f475c0a0c30 con 0x5648f939e330 2024-06-10T01:05:58.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.218+0000 7f4743fff700 1 -- 172.21.15.7:0/1893190725 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f476001f9e0 con 0x5648f939e330 2024-06-10T01:05:58.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.218+0000 7f4775455b80 10 client.15312 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:05:58.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.218+0000 7f4775455b80 10 client.15312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:05:58.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.218+0000 7f4775455b80 20 client.15312 populate_metadata read hostname 'smithi007' 2024-06-10T01:05:58.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.218+0000 7f4775455b80 1 --2- 172.21.15.7:0/1893190725 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5648f93b7a40 0x5648f93b9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:58.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.218+0000 7f4775455b80 4 client.15312 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:05:58.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.218+0000 7f4775455b80 1 -- 172.21.15.7:0/1893190725 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648f92f8d10 con 0x5648f93b7a40 2024-06-10T01:05:58.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.218+0000 7f475bfff700 1 --2- 172.21.15.7:0/1893190725 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5648f93b7a40 0x5648f93b9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:58.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.218+0000 7f475bfff700 1 --2- 172.21.15.7:0/1893190725 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5648f93b7a40 0x5648f93b9e50 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af5923254cc31a65 server_cookie=cba55b417b8246d0 in_seq=0 out_seq=0 2024-06-10T01:05:58.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.219+0000 7f4743fff700 10 client.15312 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:05:58.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.219+0000 7f4743fff700 1 -- 172.21.15.7:0/1893190725 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5648f92f8d10 con 0x5648f93b7a40 2024-06-10T01:05:58.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.219+0000 7f4743fff700 10 client.15312 handle_command_reply: tid=0 2024-06-10T01:05:58.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.219+0000 7f4775455b80 1 client.15312 shutdown 2024-06-10T01:05:58.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.219+0000 7f4775455b80 1 -- 172.21.15.7:0/1893190725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5648f939e330 msgr2=0x5648f93aa0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:58.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.219+0000 7f4775455b80 1 --2- 172.21.15.7:0/1893190725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5648f939e330 0x5648f93aa0f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f47600116f0 tx=0x7f47600047c0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.219+0000 7f4775455b80 1 -- 172.21.15.7:0/1893190725 shutdown_connections 2024-06-10T01:05:58.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.219+0000 7f4775455b80 1 --2- 172.21.15.7:0/1893190725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f93a4370 0x5648f93a79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.219+0000 7f4775455b80 1 --2- 172.21.15.7:0/1893190725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5648f939e330 0x5648f93aa0f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.219+0000 7f4775455b80 1 --2- 172.21.15.7:0/1893190725 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5648f93b7a40 0x5648f93b9e50 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.219+0000 7f4775455b80 1 --2- 172.21.15.7:0/1893190725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648f93a3a50 0x5648f93a7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.219+0000 7f4775455b80 1 -- 172.21.15.7:0/1893190725 >> 172.21.15.7:0/1893190725 conn(0x5648f92c4cc0 msgr2=0x5648f939cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:58.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.219+0000 7f4743fff700 0 client.15312 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:05:58.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.220+0000 7f4775455b80 1 -- 172.21.15.7:0/1893190725 shutdown_connections 2024-06-10T01:05:58.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.220+0000 7f4775455b80 1 -- 172.21.15.7:0/1893190725 wait complete. 2024-06-10T01:05:58.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.220+0000 7f4775455b80 20 client.15312 trim_cache size 0 max 16384 2024-06-10T01:05:58.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.241+0000 7f4775455b80 1 Processor -- start 2024-06-10T01:05:58.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.241+0000 7f4775455b80 1 -- start start 2024-06-10T01:05:58.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.241+0000 7f4775455b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5648f939e330 0x5648f939ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:58.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.241+0000 7f4775455b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648f93a3a50 0x5648f938e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:58.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.241+0000 7f4775455b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f93a4370 0x5648f93a6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:58.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.241+0000 7f4775455b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f47641ae890 con 0x5648f939e330 2024-06-10T01:05:58.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.241+0000 7f4775455b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f47641ac120 con 0x5648f93a3a50 2024-06-10T01:05:58.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.241+0000 7f4775455b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f47641a9f70 con 0x5648f93a4370 2024-06-10T01:05:58.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.241+0000 7f476896e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5648f939e330 0x5648f939ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:58.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.241+0000 7f476896e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5648f939e330 0x5648f939ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40074/0 (socket says 172.21.15.7:40074) 2024-06-10T01:05:58.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.241+0000 7f476896e700 1 -- 172.21.15.7:0/3116076713 learned_addr learned my addr 172.21.15.7:0/3116076713 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:58.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.241+0000 7f475bfff700 1 --2- 172.21.15.7:0/3116076713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648f93a3a50 0x5648f938e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:58.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.241+0000 7f476896e700 1 -- 172.21.15.7:0/3116076713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f93a4370 msgr2=0x5648f93a6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:58.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.241+0000 7f476896e700 1 --2- 172.21.15.7:0/3116076713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f93a4370 0x5648f93a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.241+0000 7f476896e700 1 -- 172.21.15.7:0/3116076713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648f93a3a50 msgr2=0x5648f938e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:58.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.241+0000 7f476896e700 1 --2- 172.21.15.7:0/3116076713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648f93a3a50 0x5648f938e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.241+0000 7f476896e700 1 -- 172.21.15.7:0/3116076713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648f93a5a00 con 0x5648f939e330 2024-06-10T01:05:58.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.241+0000 7f475bfff700 1 --2- 172.21.15.7:0/3116076713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648f93a3a50 0x5648f938e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:05:58.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.242+0000 7f476896e700 1 --2- 172.21.15.7:0/3116076713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5648f939e330 0x5648f939ddf0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4760000c00 tx=0x7f4760010ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef74047195d9cff server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:58.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.242+0000 7f474a7fc700 1 -- 172.21.15.7:0/3116076713 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4760039070 con 0x5648f939e330 2024-06-10T01:05:58.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.242+0000 7f474a7fc700 1 -- 172.21.15.7:0/3116076713 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4760023070 con 0x5648f939e330 2024-06-10T01:05:58.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.242+0000 7f474a7fc700 1 -- 172.21.15.7:0/3116076713 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47600278c0 con 0x5648f939e330 2024-06-10T01:05:58.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.242+0000 7f4775455b80 1 -- 172.21.15.7:0/3116076713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5648f939e330 msgr2=0x5648f939ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:58.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.242+0000 7f4775455b80 1 --2- 172.21.15.7:0/3116076713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5648f939e330 0x5648f939ddf0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4760000c00 tx=0x7f4760010ed0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.242+0000 7f4775455b80 1 -- 172.21.15.7:0/3116076713 shutdown_connections 2024-06-10T01:05:58.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.242+0000 7f4775455b80 1 --2- 172.21.15.7:0/3116076713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f93a4370 0x5648f93a6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.242+0000 7f4775455b80 1 --2- 172.21.15.7:0/3116076713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5648f939e330 0x5648f939ddf0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.242+0000 7f4775455b80 1 --2- 172.21.15.7:0/3116076713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648f93a3a50 0x5648f938e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.242+0000 7f4775455b80 1 -- 172.21.15.7:0/3116076713 >> 172.21.15.7:0/3116076713 conn(0x5648f92c4cc0 msgr2=0x5648f93a5d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:58.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.242+0000 7f4775455b80 1 -- 172.21.15.7:0/3116076713 shutdown_connections 2024-06-10T01:05:58.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.242+0000 7f4775455b80 1 -- 172.21.15.7:0/3116076713 wait complete. 2024-06-10T01:05:58.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.243+0000 7f4775455b80 1 -- start start 2024-06-10T01:05:58.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.243+0000 7f4775455b80 1 Processor -- start 2024-06-10T01:05:58.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.243+0000 7f4775455b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648f939e330 0x5648f93abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:58.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.243+0000 7f4775455b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5648f93a3a50 0x5648f93ac490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:58.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.243+0000 7f4775455b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f93a4370 0x5648f93aca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:58.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.243+0000 7f4775455b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f47641aa5e0 con 0x5648f93a3a50 2024-06-10T01:05:58.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.243+0000 7f4775455b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f47641aba50 con 0x5648f939e330 2024-06-10T01:05:58.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.243+0000 7f4775455b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f47641aacb0 con 0x5648f93a4370 2024-06-10T01:05:58.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.243+0000 7f4775455b80 10 client.0 _ll_register_callbacks cb 0x5648f9285920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:05:58.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.244+0000 7f475bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5648f93a3a50 0x5648f93ac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:58.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.244+0000 7f475bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5648f93a3a50 0x5648f93ac490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40080/0 (socket says 172.21.15.7:40080) 2024-06-10T01:05:58.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.244+0000 7f475bfff700 1 -- 172.21.15.7:0/2527960094 learned_addr learned my addr 172.21.15.7:0/2527960094 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:58.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.244+0000 7f476896e700 1 --2- 172.21.15.7:0/2527960094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648f939e330 0x5648f93abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:58.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.244+0000 7f476916f700 1 --2- 172.21.15.7:0/2527960094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f93a4370 0x5648f93aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:58.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.244+0000 7f475bfff700 1 -- 172.21.15.7:0/2527960094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f93a4370 msgr2=0x5648f93aca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:58.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.244+0000 7f475bfff700 1 --2- 172.21.15.7:0/2527960094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f93a4370 0x5648f93aca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.244+0000 7f475bfff700 1 -- 172.21.15.7:0/2527960094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648f939e330 msgr2=0x5648f93abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:58.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.244+0000 7f475bfff700 1 --2- 172.21.15.7:0/2527960094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648f939e330 0x5648f93abf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.244+0000 7f475bfff700 1 -- 172.21.15.7:0/2527960094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648f9421a40 con 0x5648f93a3a50 2024-06-10T01:05:58.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.244+0000 7f475bfff700 1 --2- 172.21.15.7:0/2527960094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5648f93a3a50 0x5648f93ac490 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f47540097c0 tx=0x7f47540107c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f2b2fb87c129b0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:58.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.244+0000 7f4743fff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:05:58.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.244+0000 7f4743fff700 1 -- 172.21.15.7:0/2527960094 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4754004230 con 0x5648f93a3a50 2024-06-10T01:05:58.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.244+0000 7f4743fff700 1 -- 172.21.15.7:0/2527960094 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f47540043d0 con 0x5648f93a3a50 2024-06-10T01:05:58.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.244+0000 7f4775455b80 1 -- 172.21.15.7:0/2527960094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4760023070 con 0x5648f93a3a50 2024-06-10T01:05:58.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.244+0000 7f4743fff700 1 -- 172.21.15.7:0/2527960094 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4754003680 con 0x5648f93a3a50 2024-06-10T01:05:58.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.244+0000 7f4775455b80 10 client.15318 fetch_fsmap learned FSMap version 27 2024-06-10T01:05:58.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.245+0000 7f4775455b80 1 -- 172.21.15.7:0/2527960094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5648f9422d50 con 0x5648f93a3a50 2024-06-10T01:05:58.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.245+0000 7f4743fff700 1 -- 172.21.15.7:0/2527960094 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4760023070 con 0x5648f93a3a50 2024-06-10T01:05:58.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.245+0000 7f4743fff700 1 -- 172.21.15.7:0/2527960094 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4754025070 con 0x5648f93a3a50 2024-06-10T01:05:58.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.245+0000 7f4775455b80 10 client.15318 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:05:58.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.245+0000 7f4775455b80 10 client.15318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:05:58.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.245+0000 7f4775455b80 20 client.15318 populate_metadata read hostname 'smithi007' 2024-06-10T01:05:58.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.245+0000 7f4775455b80 1 --2- 172.21.15.7:0/2527960094 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5648f94234e0 0x5648f9425900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:58.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.245+0000 7f4775455b80 4 client.15318 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:05:58.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.245+0000 7f4775455b80 1 -- 172.21.15.7:0/2527960094 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5648f92f3290 con 0x5648f94234e0 2024-06-10T01:05:58.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.245+0000 7f476896e700 1 --2- 172.21.15.7:0/2527960094 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5648f94234e0 0x5648f9425900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:58.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.246+0000 7f4743fff700 10 client.15318 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:05:58.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.246+0000 7f476896e700 1 --2- 172.21.15.7:0/2527960094 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5648f94234e0 0x5648f9425900 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f4fb4ad5681719c server_cookie=3d35310be97a138d in_seq=0 out_seq=0 2024-06-10T01:05:58.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.246+0000 7f4743fff700 1 -- 172.21.15.7:0/2527960094 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5648f92f3290 con 0x5648f94234e0 2024-06-10T01:05:58.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.246+0000 7f4743fff700 10 client.15318 handle_command_reply: tid=0 2024-06-10T01:05:58.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.246+0000 7f4775455b80 1 client.15318 shutdown 2024-06-10T01:05:58.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.246+0000 7f4775455b80 1 -- 172.21.15.7:0/2527960094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5648f93a3a50 msgr2=0x5648f93ac490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:58.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.246+0000 7f4775455b80 1 --2- 172.21.15.7:0/2527960094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5648f93a3a50 0x5648f93ac490 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f47540097c0 tx=0x7f47540107c0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.246+0000 7f4775455b80 1 -- 172.21.15.7:0/2527960094 shutdown_connections 2024-06-10T01:05:58.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.246+0000 7f4775455b80 1 --2- 172.21.15.7:0/2527960094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f93a4370 0x5648f93aca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.246+0000 7f4775455b80 1 --2- 172.21.15.7:0/2527960094 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5648f94234e0 0x5648f9425900 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.246+0000 7f4775455b80 1 --2- 172.21.15.7:0/2527960094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648f939e330 0x5648f93abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.246+0000 7f4775455b80 1 --2- 172.21.15.7:0/2527960094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5648f93a3a50 0x5648f93ac490 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.246+0000 7f4775455b80 1 -- 172.21.15.7:0/2527960094 >> 172.21.15.7:0/2527960094 conn(0x5648f92c4cc0 msgr2=0x5648f93a2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:58.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.246+0000 7f4743fff700 0 client.15318 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:05:58.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.246+0000 7f4775455b80 1 -- 172.21.15.7:0/2527960094 shutdown_connections 2024-06-10T01:05:58.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.246+0000 7f4775455b80 1 -- 172.21.15.7:0/2527960094 wait complete. 2024-06-10T01:05:58.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.246+0000 7f4775455b80 20 client.15318 trim_cache size 0 max 16384 2024-06-10T01:05:58.249 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:05:58.249 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T01:05:58.249 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-10T01:05:58.249 INFO:teuthology.orchestra.run.smithi007.stdout: "333bd08f-d8ff-40fd-8495-172914b960e5": { 2024-06-10T01:05:58.249 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-10T01:05:58.249 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "333bd08f-d8ff-40fd-8495-172914b960e5", 2024-06-10T01:05:58.249 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-10T01:05:58.249 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:05:58.249 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:05:58.249 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:05:58.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.247+0000 7f476abd2700 1 -- 172.21.15.7:0/3200595947 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f474c0a2c20 msgr2=0x7f474c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:58.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.247+0000 7f476abd2700 1 --2- 172.21.15.7:0/3200595947 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f474c0a2c20 0x7f474c0a50e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4760005950 tx=0x7f476000b410 comp rx=0 tx=0).stop 2024-06-10T01:05:58.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.247+0000 7f476abd2700 1 -- 172.21.15.7:0/3200595947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764107660 msgr2=0x7f47641a4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:58.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.247+0000 7f476abd2700 1 --2- 172.21.15.7:0/3200595947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764107660 0x7f47641a4b30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f475c009850 tx=0x7f475c007a20 comp rx=0 tx=0).stop 2024-06-10T01:05:58.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.247+0000 7f476abd2700 1 -- 172.21.15.7:0/3200595947 shutdown_connections 2024-06-10T01:05:58.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.247+0000 7f476abd2700 1 --2- 172.21.15.7:0/3200595947 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f474c0a2c20 0x7f474c0a50e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.247+0000 7f476abd2700 1 --2- 172.21.15.7:0/3200595947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764107660 0x7f47641a4b30 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.247+0000 7f476abd2700 1 --2- 172.21.15.7:0/3200595947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4764101f40 0x7f4764199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.247+0000 7f476abd2700 1 --2- 172.21.15.7:0/3200595947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4764102900 0x7f4764199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:58.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.247+0000 7f476abd2700 1 -- 172.21.15.7:0/3200595947 >> 172.21.15.7:0/3200595947 conn(0x7f47640766b0 msgr2=0x7f47640fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:58.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.247+0000 7f476abd2700 1 -- 172.21.15.7:0/3200595947 shutdown_connections 2024-06-10T01:05:58.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:58.247+0000 7f476abd2700 1 -- 172.21.15.7:0/3200595947 wait complete. 2024-06-10T01:05:59.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:05:59 smithi007 ceph-mon[32398]: pgmap v430: 129 pgs: 129 active+clean; 649 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 7 op/s 2024-06-10T01:05:59.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:05:59 smithi078 ceph-mon[36683]: pgmap v430: 129 pgs: 129 active+clean; 649 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 7 op/s 2024-06-10T01:05:59.670 INFO:tasks.cephfs.filesystem:scrub status for tag:333bd08f-d8ff-40fd-8495-172914b960e5 - {'path': '/', 'tag': '333bd08f-d8ff-40fd-8495-172914b960e5', 'options': 'recursive,force'} 2024-06-10T01:05:59.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:05:59 smithi175 ceph-mon[36636]: pgmap v430: 129 pgs: 129 active+clean; 649 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 7 op/s 2024-06-10T01:05:59.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.922+0000 7f8239eef700 1 -- 172.21.15.7:0/2642421836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8234108560 msgr2=0x7f8234108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:59.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.922+0000 7f8239eef700 1 --2- 172.21.15.7:0/2642421836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8234108560 0x7f8234108940 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f82240099c0 tx=0x7f8224009cd0 comp rx=0 tx=0).stop 2024-06-10T01:05:59.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.922+0000 7f8239eef700 1 -- 172.21.15.7:0/2642421836 shutdown_connections 2024-06-10T01:05:59.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.922+0000 7f8239eef700 1 --2- 172.21.15.7:0/2642421836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8234102ed0 0x7f823410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:59.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.922+0000 7f8239eef700 1 --2- 172.21.15.7:0/2642421836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8234108560 0x7f8234108940 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:59.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.922+0000 7f8239eef700 1 --2- 172.21.15.7:0/2642421836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8234102510 0x7f8234102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:59.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.922+0000 7f8239eef700 1 -- 172.21.15.7:0/2642421836 >> 172.21.15.7:0/2642421836 conn(0x7f82340795e0 msgr2=0x7f82340799f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:59.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.922+0000 7f8239eef700 1 -- 172.21.15.7:0/2642421836 shutdown_connections 2024-06-10T01:05:59.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.922+0000 7f8239eef700 1 -- 172.21.15.7:0/2642421836 wait complete. 2024-06-10T01:05:59.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.923+0000 7f8239eef700 1 Processor -- start 2024-06-10T01:05:59.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.923+0000 7f8239eef700 1 -- start start 2024-06-10T01:05:59.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.923+0000 7f8239eef700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8234102510 0x7f823419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:59.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.923+0000 7f8239eef700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8234102ed0 0x7f823419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:59.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.923+0000 7f8239eef700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8234108560 0x7f8234198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:59.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.923+0000 7f8239eef700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f82341127c0 con 0x7f8234108560 2024-06-10T01:05:59.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.923+0000 7f8239eef700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8234112640 con 0x7f8234102ed0 2024-06-10T01:05:59.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.923+0000 7f8239eef700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8234112940 con 0x7f8234102510 2024-06-10T01:05:59.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.923+0000 7f8233fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8234108560 0x7f8234198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:59.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.923+0000 7f8233fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8234108560 0x7f8234198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40100/0 (socket says 172.21.15.7:40100) 2024-06-10T01:05:59.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.923+0000 7f8233fff700 1 -- 172.21.15.7:0/498944889 learned_addr learned my addr 172.21.15.7:0/498944889 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:05:59.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.923+0000 7f82337fe700 1 --2- 172.21.15.7:0/498944889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8234102510 0x7f823419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:59.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.924+0000 7f822affd700 1 --2- 172.21.15.7:0/498944889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8234102ed0 0x7f823419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:59.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.924+0000 7f8233fff700 1 -- 172.21.15.7:0/498944889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8234102510 msgr2=0x7f823419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:59.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.924+0000 7f8233fff700 1 --2- 172.21.15.7:0/498944889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8234102510 0x7f823419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:59.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.924+0000 7f8233fff700 1 -- 172.21.15.7:0/498944889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8234102ed0 msgr2=0x7f823419eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:59.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.924+0000 7f8233fff700 1 --2- 172.21.15.7:0/498944889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8234102ed0 0x7f823419eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:59.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.924+0000 7f8233fff700 1 -- 172.21.15.7:0/498944889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8234199180 con 0x7f8234108560 2024-06-10T01:05:59.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.924+0000 7f822affd700 1 --2- 172.21.15.7:0/498944889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8234102ed0 0x7f823419eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:05:59.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.924+0000 7f8233fff700 1 --2- 172.21.15.7:0/498944889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8234108560 0x7f8234198a20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f822c007f10 tx=0x7f822c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:59.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.924+0000 7f82317fa700 1 -- 172.21.15.7:0/498944889 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c009e20 con 0x7f8234108560 2024-06-10T01:05:59.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.924+0000 7f8239eef700 1 -- 172.21.15.7:0/498944889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8234199470 con 0x7f8234108560 2024-06-10T01:05:59.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.924+0000 7f82317fa700 1 -- 172.21.15.7:0/498944889 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f822c00dbd0 con 0x7f8234108560 2024-06-10T01:05:59.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.924+0000 7f82317fa700 1 -- 172.21.15.7:0/498944889 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c005a60 con 0x7f8234108560 2024-06-10T01:05:59.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.924+0000 7f8239eef700 1 -- 172.21.15.7:0/498944889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82341a5730 con 0x7f8234108560 2024-06-10T01:05:59.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.925+0000 7f8239eef700 1 -- 172.21.15.7:0/498944889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f823410a9f0 con 0x7f8234108560 2024-06-10T01:05:59.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.927+0000 7f82317fa700 1 -- 172.21.15.7:0/498944889 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f822c01a020 con 0x7f8234108560 2024-06-10T01:05:59.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.927+0000 7f82317fa700 1 --2- 172.21.15.7:0/498944889 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f82200a2c20 0x7f82200a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:59.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.927+0000 7f82317fa700 1 -- 172.21.15.7:0/498944889 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f822c0cf0e0 con 0x7f8234108560 2024-06-10T01:05:59.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.927+0000 7f82337fe700 1 --2- 172.21.15.7:0/498944889 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f82200a2c20 0x7f82200a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:59.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.928+0000 7f82337fe700 1 --2- 172.21.15.7:0/498944889 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f82200a2c20 0x7f82200a50e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8224000c00 tx=0x7f8224005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:59.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:05:59.935+0000 7f82317fa700 1 -- 172.21.15.7:0/498944889 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f822c010070 con 0x7f8234108560 2024-06-10T01:06:00.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:00.138+0000 7f8239eef700 1 -- 172.21.15.7:0/498944889 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8234199f50 con 0x7f82200a2c20 2024-06-10T01:06:00.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:00.157+0000 7f82317fa700 1 -- 172.21.15.7:0/498944889 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340767 (secure 0 0 0) 0x7f8234199f50 con 0x7f82200a2c20 2024-06-10T01:06:00.159 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:06:00.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:00.160+0000 7f8239eef700 1 -- 172.21.15.7:0/498944889 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f82200a2c20 msgr2=0x7f82200a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:00.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:00.160+0000 7f8239eef700 1 --2- 172.21.15.7:0/498944889 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f82200a2c20 0x7f82200a50e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8224000c00 tx=0x7f8224005fb0 comp rx=0 tx=0).stop 2024-06-10T01:06:00.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:00.160+0000 7f8239eef700 1 -- 172.21.15.7:0/498944889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8234108560 msgr2=0x7f8234198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:00.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:00.160+0000 7f8239eef700 1 --2- 172.21.15.7:0/498944889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8234108560 0x7f8234198a20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f822c007f10 tx=0x7f822c00d3b0 comp rx=0 tx=0).stop 2024-06-10T01:06:00.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:00.161+0000 7f8239eef700 1 -- 172.21.15.7:0/498944889 shutdown_connections 2024-06-10T01:06:00.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:00.161+0000 7f8239eef700 1 --2- 172.21.15.7:0/498944889 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f82200a2c20 0x7f82200a50e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:00.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:00.161+0000 7f8239eef700 1 --2- 172.21.15.7:0/498944889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8234108560 0x7f8234198a20 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:00.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:00.161+0000 7f8239eef700 1 --2- 172.21.15.7:0/498944889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8234102510 0x7f823419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:00.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:00.161+0000 7f8239eef700 1 --2- 172.21.15.7:0/498944889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8234102ed0 0x7f823419eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:00.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:00.161+0000 7f8239eef700 1 -- 172.21.15.7:0/498944889 >> 172.21.15.7:0/498944889 conn(0x7f82340795e0 msgr2=0x7f8234100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:00.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:00.161+0000 7f8239eef700 1 -- 172.21.15.7:0/498944889 shutdown_connections 2024-06-10T01:06:00.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:00.161+0000 7f8239eef700 1 -- 172.21.15.7:0/498944889 wait complete. 2024-06-10T01:06:00.163 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-06-10T01:06:00.852 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":431,"stamp":"2024-06-10T01:05:59.601529+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664587,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1248,"num_read_kb":749,"num_write":369,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":796,"ondisk_log_size":796,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362480,"kb_used_data":13328,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579344,"statfs":{"total":1151940427776,"available":1151569248256,"internally_reserved":0,"allocated":13647872,"data_stored":8806463,"data_compressed":496626,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1138,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":39,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003273"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.569445+0000","last_change":"2024-06-10T01:01:47.244728+0000","last_active":"2024-06-10T01:04:35.569445+0000","last_peered":"2024-06-10T01:04:35.569445+0000","last_clean":"2024-06-10T01:04:35.569445+0000","last_became_active":"2024-06-10T01:01:47.244198+0000","last_became_peered":"2024-06-10T01:01:47.244198+0000","last_unstale":"2024-06-10T01:04:35.569445+0000","last_undegraded":"2024-06-10T01:04:35.569445+0000","last_fullsized":"2024-06-10T01:04:35.569445+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:58:06.257952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.557753+0000","last_change":"2024-06-10T01:01:47.247620+0000","last_active":"2024-06-10T01:04:35.557753+0000","last_peered":"2024-06-10T01:04:35.557753+0000","last_clean":"2024-06-10T01:04:35.557753+0000","last_became_active":"2024-06-10T01:01:47.245475+0000","last_became_peered":"2024-06-10T01:01:47.245475+0000","last_unstale":"2024-06-10T01:04:35.557753+0000","last_undegraded":"2024-06-10T01:04:35.557753+0000","last_fullsized":"2024-06-10T01:04:35.557753+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:29:09.367411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.569730+0000","last_change":"2024-06-10T01:01:47.252414+0000","last_active":"2024-06-10T01:04:35.569730+0000","last_peered":"2024-06-10T01:04:35.569730+0000","last_clean":"2024-06-10T01:04:35.569730+0000","last_became_active":"2024-06-10T01:01:47.252063+0000","last_became_peered":"2024-06-10T01:01:47.252063+0000","last_unstale":"2024-06-10T01:04:35.569730+0000","last_undegraded":"2024-06-10T01:04:35.569730+0000","last_fullsized":"2024-06-10T01:04:35.569730+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:54:53.966209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.553817+0000","last_change":"2024-06-10T01:01:47.254875+0000","last_active":"2024-06-10T01:04:35.553817+0000","last_peered":"2024-06-10T01:04:35.553817+0000","last_clean":"2024-06-10T01:04:35.553817+0000","last_became_active":"2024-06-10T01:01:47.254542+0000","last_became_peered":"2024-06-10T01:01:47.254542+0000","last_unstale":"2024-06-10T01:04:35.553817+0000","last_undegraded":"2024-06-10T01:04:35.553817+0000","last_fullsized":"2024-06-10T01:04:35.553817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:02:28.393133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558663+0000","last_change":"2024-06-10T01:01:47.242596+0000","last_active":"2024-06-10T01:04:35.558663+0000","last_peered":"2024-06-10T01:04:35.558663+0000","last_clean":"2024-06-10T01:04:35.558663+0000","last_became_active":"2024-06-10T01:01:47.242308+0000","last_became_peered":"2024-06-10T01:01:47.242308+0000","last_unstale":"2024-06-10T01:04:35.558663+0000","last_undegraded":"2024-06-10T01:04:35.558663+0000","last_fullsized":"2024-06-10T01:04:35.558663+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:11:26.599954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.550600+0000","last_change":"2024-06-10T01:01:47.249085+0000","last_active":"2024-06-10T01:04:35.550600+0000","last_peered":"2024-06-10T01:04:35.550600+0000","last_clean":"2024-06-10T01:04:35.550600+0000","last_became_active":"2024-06-10T01:01:47.248749+0000","last_became_peered":"2024-06-10T01:01:47.248749+0000","last_unstale":"2024-06-10T01:04:35.550600+0000","last_undegraded":"2024-06-10T01:04:35.550600+0000","last_fullsized":"2024-06-10T01:04:35.550600+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:42:48.734566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562583+0000","last_change":"2024-06-10T01:01:47.247334+0000","last_active":"2024-06-10T01:04:35.562583+0000","last_peered":"2024-06-10T01:04:35.562583+0000","last_clean":"2024-06-10T01:04:35.562583+0000","last_became_active":"2024-06-10T01:01:47.246964+0000","last_became_peered":"2024-06-10T01:01:47.246964+0000","last_unstale":"2024-06-10T01:04:35.562583+0000","last_undegraded":"2024-06-10T01:04:35.562583+0000","last_fullsized":"2024-06-10T01:04:35.562583+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:05:36.034950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.563056+0000","last_change":"2024-06-10T01:01:47.242523+0000","last_active":"2024-06-10T01:04:35.563056+0000","last_peered":"2024-06-10T01:04:35.563056+0000","last_clean":"2024-06-10T01:04:35.563056+0000","last_became_active":"2024-06-10T01:01:47.242198+0000","last_became_peered":"2024-06-10T01:01:47.242198+0000","last_unstale":"2024-06-10T01:04:35.563056+0000","last_undegraded":"2024-06-10T01:04:35.563056+0000","last_fullsized":"2024-06-10T01:04:35.563056+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:39:07.055970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.567116+0000","last_change":"2024-06-10T01:01:47.245277+0000","last_active":"2024-06-10T01:04:35.567116+0000","last_peered":"2024-06-10T01:04:35.567116+0000","last_clean":"2024-06-10T01:04:35.567116+0000","last_became_active":"2024-06-10T01:01:47.244932+0000","last_became_peered":"2024-06-10T01:01:47.244932+0000","last_unstale":"2024-06-10T01:04:35.567116+0000","last_undegraded":"2024-06-10T01:04:35.567116+0000","last_fullsized":"2024-06-10T01:04:35.567116+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:56:24.423713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559784+0000","last_change":"2024-06-10T01:01:47.255531+0000","last_active":"2024-06-10T01:04:35.559784+0000","last_peered":"2024-06-10T01:04:35.559784+0000","last_clean":"2024-06-10T01:04:35.559784+0000","last_became_active":"2024-06-10T01:01:47.255150+0000","last_became_peered":"2024-06-10T01:01:47.255150+0000","last_unstale":"2024-06-10T01:04:35.559784+0000","last_undegraded":"2024-06-10T01:04:35.559784+0000","last_fullsized":"2024-06-10T01:04:35.559784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:51:34.561396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559164+0000","last_change":"2024-06-10T01:01:47.249832+0000","last_active":"2024-06-10T01:04:35.559164+0000","last_peered":"2024-06-10T01:04:35.559164+0000","last_clean":"2024-06-10T01:04:35.559164+0000","last_became_active":"2024-06-10T01:01:47.249303+0000","last_became_peered":"2024-06-10T01:01:47.249303+0000","last_unstale":"2024-06-10T01:04:35.559164+0000","last_undegraded":"2024-06-10T01:04:35.559164+0000","last_fullsized":"2024-06-10T01:04:35.559164+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:15:21.930157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.551184+0000","last_change":"2024-06-10T01:01:47.255927+0000","last_active":"2024-06-10T01:04:35.551184+0000","last_peered":"2024-06-10T01:04:35.551184+0000","last_clean":"2024-06-10T01:04:35.551184+0000","last_became_active":"2024-06-10T01:01:47.255591+0000","last_became_peered":"2024-06-10T01:01:47.255591+0000","last_unstale":"2024-06-10T01:04:35.551184+0000","last_undegraded":"2024-06-10T01:04:35.551184+0000","last_fullsized":"2024-06-10T01:04:35.551184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:48:06.494430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.557880+0000","last_change":"2024-06-10T01:02:37.372069+0000","last_active":"2024-06-10T01:04:35.557880+0000","last_peered":"2024-06-10T01:04:35.557880+0000","last_clean":"2024-06-10T01:04:35.557880+0000","last_became_active":"2024-06-10T01:02:37.371599+0000","last_became_peered":"2024-06-10T01:02:37.371599+0000","last_unstale":"2024-06-10T01:04:35.557880+0000","last_undegraded":"2024-06-10T01:04:35.557880+0000","last_fullsized":"2024-06-10T01:04:35.557880+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:54:34.091576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.551120+0000","last_change":"2024-06-10T01:01:47.249553+0000","last_active":"2024-06-10T01:04:35.551120+0000","last_peered":"2024-06-10T01:04:35.551120+0000","last_clean":"2024-06-10T01:04:35.551120+0000","last_became_active":"2024-06-10T01:01:47.249202+0000","last_became_peered":"2024-06-10T01:01:47.249202+0000","last_unstale":"2024-06-10T01:04:35.551120+0000","last_undegraded":"2024-06-10T01:04:35.551120+0000","last_fullsized":"2024-06-10T01:04:35.551120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:23:57.882264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.564289+0000","last_change":"2024-06-10T01:01:47.253987+0000","last_active":"2024-06-10T01:04:35.564289+0000","last_peered":"2024-06-10T01:04:35.564289+0000","last_clean":"2024-06-10T01:04:35.564289+0000","last_became_active":"2024-06-10T01:01:47.253578+0000","last_became_peered":"2024-06-10T01:01:47.253578+0000","last_unstale":"2024-06-10T01:04:35.564289+0000","last_undegraded":"2024-06-10T01:04:35.564289+0000","last_fullsized":"2024-06-10T01:04:35.564289+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:35:30.585482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.567601+0000","last_change":"2024-06-10T01:01:47.247089+0000","last_active":"2024-06-10T01:04:35.567601+0000","last_peered":"2024-06-10T01:04:35.567601+0000","last_clean":"2024-06-10T01:04:35.567601+0000","last_became_active":"2024-06-10T01:01:47.246066+0000","last_became_peered":"2024-06-10T01:01:47.246066+0000","last_unstale":"2024-06-10T01:04:35.567601+0000","last_undegraded":"2024-06-10T01:04:35.567601+0000","last_fullsized":"2024-06-10T01:04:35.567601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:58:12.092981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.569804+0000","last_change":"2024-06-10T01:01:47.244491+0000","last_active":"2024-06-10T01:04:35.569804+0000","last_peered":"2024-06-10T01:04:35.569804+0000","last_clean":"2024-06-10T01:04:35.569804+0000","last_became_active":"2024-06-10T01:01:47.243747+0000","last_became_peered":"2024-06-10T01:01:47.243747+0000","last_unstale":"2024-06-10T01:04:35.569804+0000","last_undegraded":"2024-06-10T01:04:35.569804+0000","last_fullsized":"2024-06-10T01:04:35.569804+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:55:56.785016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.567988+0000","last_change":"2024-06-10T01:01:47.243141+0000","last_active":"2024-06-10T01:04:35.567988+0000","last_peered":"2024-06-10T01:04:35.567988+0000","last_clean":"2024-06-10T01:04:35.567988+0000","last_became_active":"2024-06-10T01:01:47.242464+0000","last_became_peered":"2024-06-10T01:01:47.242464+0000","last_unstale":"2024-06-10T01:04:35.567988+0000","last_undegraded":"2024-06-10T01:04:35.567988+0000","last_fullsized":"2024-06-10T01:04:35.567988+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:00:49.777130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559919+0000","last_change":"2024-06-10T01:01:47.253463+0000","last_active":"2024-06-10T01:04:35.559919+0000","last_peered":"2024-06-10T01:04:35.559919+0000","last_clean":"2024-06-10T01:04:35.559919+0000","last_became_active":"2024-06-10T01:01:47.252929+0000","last_became_peered":"2024-06-10T01:01:47.252929+0000","last_unstale":"2024-06-10T01:04:35.559919+0000","last_undegraded":"2024-06-10T01:04:35.559919+0000","last_fullsized":"2024-06-10T01:04:35.559919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:18:49.551276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239453+0000","last_change":"2024-06-10T01:01:57.279561+0000","last_active":"2024-06-10T01:03:38.239453+0000","last_peered":"2024-06-10T01:03:38.239453+0000","last_clean":"2024-06-10T01:03:38.239453+0000","last_became_active":"2024-06-10T01:01:57.279132+0000","last_became_peered":"2024-06-10T01:01:57.279132+0000","last_unstale":"2024-06-10T01:03:38.239453+0000","last_undegraded":"2024-06-10T01:03:38.239453+0000","last_fullsized":"2024-06-10T01:03:38.239453+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:55:03.497701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'9","reported_seq":57,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:54.300918+0000","last_change":"2024-06-10T01:01:44.006667+0000","last_active":"2024-06-10T01:05:54.300918+0000","last_peered":"2024-06-10T01:05:54.300918+0000","last_clean":"2024-06-10T01:05:54.300918+0000","last_became_active":"2024-06-10T01:01:44.006343+0000","last_became_peered":"2024-06-10T01:01:44.006343+0000","last_unstale":"2024-06-10T01:05:54.300918+0000","last_undegraded":"2024-06-10T01:05:54.300918+0000","last_fullsized":"2024-06-10T01:05:54.300918+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:14:50.540084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562931+0000","last_change":"2024-06-10T01:01:47.246139+0000","last_active":"2024-06-10T01:04:35.562931+0000","last_peered":"2024-06-10T01:04:35.562931+0000","last_clean":"2024-06-10T01:04:35.562931+0000","last_became_active":"2024-06-10T01:01:47.245743+0000","last_became_peered":"2024-06-10T01:01:47.245743+0000","last_unstale":"2024-06-10T01:04:35.562931+0000","last_undegraded":"2024-06-10T01:04:35.562931+0000","last_fullsized":"2024-06-10T01:04:35.562931+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:06:39.041569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239398+0000","last_change":"2024-06-10T01:01:57.282843+0000","last_active":"2024-06-10T01:03:38.239398+0000","last_peered":"2024-06-10T01:03:38.239398+0000","last_clean":"2024-06-10T01:03:38.239398+0000","last_became_active":"2024-06-10T01:01:57.281797+0000","last_became_peered":"2024-06-10T01:01:57.281797+0000","last_unstale":"2024-06-10T01:03:38.239398+0000","last_undegraded":"2024-06-10T01:03:38.239398+0000","last_fullsized":"2024-06-10T01:03:38.239398+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:00.209628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:43.959967+0000","last_change":"2024-06-10T01:01:44.466714+0000","last_active":"2024-06-10T01:03:43.959967+0000","last_peered":"2024-06-10T01:03:43.959967+0000","last_clean":"2024-06-10T01:03:43.959967+0000","last_became_active":"2024-06-10T01:01:44.466296+0000","last_became_peered":"2024-06-10T01:01:44.466296+0000","last_unstale":"2024-06-10T01:03:43.959967+0000","last_undegraded":"2024-06-10T01:03:43.959967+0000","last_fullsized":"2024-06-10T01:03:43.959967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:16:36.082308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.569882+0000","last_change":"2024-06-10T01:01:47.244173+0000","last_active":"2024-06-10T01:04:35.569882+0000","last_peered":"2024-06-10T01:04:35.569882+0000","last_clean":"2024-06-10T01:04:35.569882+0000","last_became_active":"2024-06-10T01:01:47.243032+0000","last_became_peered":"2024-06-10T01:01:47.243032+0000","last_unstale":"2024-06-10T01:04:35.569882+0000","last_undegraded":"2024-06-10T01:04:35.569882+0000","last_fullsized":"2024-06-10T01:04:35.569882+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:14:33.965903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239057+0000","last_change":"2024-06-10T01:02:37.592535+0000","last_active":"2024-06-10T01:03:38.239057+0000","last_peered":"2024-06-10T01:03:38.239057+0000","last_clean":"2024-06-10T01:03:38.239057+0000","last_became_active":"2024-06-10T01:02:37.592159+0000","last_became_peered":"2024-06-10T01:02:37.592159+0000","last_unstale":"2024-06-10T01:03:38.239057+0000","last_undegraded":"2024-06-10T01:03:38.239057+0000","last_fullsized":"2024-06-10T01:03:38.239057+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:50:08.626485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235433+0000","last_change":"2024-06-10T01:01:44.009105+0000","last_active":"2024-06-10T01:03:38.235433+0000","last_peered":"2024-06-10T01:03:38.235433+0000","last_clean":"2024-06-10T01:03:38.235433+0000","last_became_active":"2024-06-10T01:01:44.008743+0000","last_became_peered":"2024-06-10T01:01:44.008743+0000","last_unstale":"2024-06-10T01:03:38.235433+0000","last_undegraded":"2024-06-10T01:03:38.235433+0000","last_fullsized":"2024-06-10T01:03:38.235433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:44:54.301378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.570239+0000","last_change":"2024-06-10T01:01:47.249007+0000","last_active":"2024-06-10T01:04:35.570239+0000","last_peered":"2024-06-10T01:04:35.570239+0000","last_clean":"2024-06-10T01:04:35.570239+0000","last_became_active":"2024-06-10T01:01:47.245632+0000","last_became_peered":"2024-06-10T01:01:47.245632+0000","last_unstale":"2024-06-10T01:04:35.570239+0000","last_undegraded":"2024-06-10T01:04:35.570239+0000","last_fullsized":"2024-06-10T01:04:35.570239+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:29:18.916390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.238983+0000","last_change":"2024-06-10T01:01:57.283137+0000","last_active":"2024-06-10T01:03:38.238983+0000","last_peered":"2024-06-10T01:03:38.238983+0000","last_clean":"2024-06-10T01:03:38.238983+0000","last_became_active":"2024-06-10T01:01:57.282605+0000","last_became_peered":"2024-06-10T01:01:57.282605+0000","last_unstale":"2024-06-10T01:03:38.238983+0000","last_undegraded":"2024-06-10T01:03:38.238983+0000","last_fullsized":"2024-06-10T01:03:38.238983+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:26:16.967320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235734+0000","last_change":"2024-06-10T01:01:44.010674+0000","last_active":"2024-06-10T01:03:38.235734+0000","last_peered":"2024-06-10T01:03:38.235734+0000","last_clean":"2024-06-10T01:03:38.235734+0000","last_became_active":"2024-06-10T01:01:44.010291+0000","last_became_peered":"2024-06-10T01:01:44.010291+0000","last_unstale":"2024-06-10T01:03:38.235734+0000","last_undegraded":"2024-06-10T01:03:38.235734+0000","last_fullsized":"2024-06-10T01:03:38.235734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:36:18.418362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":60,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.566078+0000","last_change":"2024-06-10T01:02:37.794580+0000","last_active":"2024-06-10T01:04:35.566078+0000","last_peered":"2024-06-10T01:04:35.566078+0000","last_clean":"2024-06-10T01:04:35.566078+0000","last_became_active":"2024-06-10T01:02:37.794127+0000","last_became_peered":"2024-06-10T01:02:37.794127+0000","last_unstale":"2024-06-10T01:04:35.566078+0000","last_undegraded":"2024-06-10T01:04:35.566078+0000","last_fullsized":"2024-06-10T01:04:35.566078+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:02.117871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239746+0000","last_change":"2024-06-10T01:01:57.282494+0000","last_active":"2024-06-10T01:03:38.239746+0000","last_peered":"2024-06-10T01:03:38.239746+0000","last_clean":"2024-06-10T01:03:38.239746+0000","last_became_active":"2024-06-10T01:01:57.282077+0000","last_became_peered":"2024-06-10T01:01:57.282077+0000","last_unstale":"2024-06-10T01:03:38.239746+0000","last_undegraded":"2024-06-10T01:03:38.239746+0000","last_fullsized":"2024-06-10T01:03:38.239746+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:57:41.639788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"97'5","reported_seq":234,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:55.973995+0000","last_change":"2024-06-10T01:01:44.011134+0000","last_active":"2024-06-10T01:05:55.973995+0000","last_peered":"2024-06-10T01:05:55.973995+0000","last_clean":"2024-06-10T01:05:55.973995+0000","last_became_active":"2024-06-10T01:01:44.010788+0000","last_became_peered":"2024-06-10T01:01:44.010788+0000","last_unstale":"2024-06-10T01:05:55.973995+0000","last_undegraded":"2024-06-10T01:05:55.973995+0000","last_fullsized":"2024-06-10T01:05:55.973995+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:22:55.228946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":184,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.564421+0000","last_change":"2024-06-10T01:01:47.248093+0000","last_active":"2024-06-10T01:04:35.564421+0000","last_peered":"2024-06-10T01:04:35.564421+0000","last_clean":"2024-06-10T01:04:35.564421+0000","last_became_active":"2024-06-10T01:01:47.247797+0000","last_became_peered":"2024-06-10T01:01:47.247797+0000","last_unstale":"2024-06-10T01:04:35.564421+0000","last_undegraded":"2024-06-10T01:04:35.564421+0000","last_fullsized":"2024-06-10T01:04:35.564421+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:45:50.159128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239671+0000","last_change":"2024-06-10T01:02:37.377039+0000","last_active":"2024-06-10T01:03:38.239671+0000","last_peered":"2024-06-10T01:03:38.239671+0000","last_clean":"2024-06-10T01:03:38.239671+0000","last_became_active":"2024-06-10T01:02:37.376332+0000","last_became_peered":"2024-06-10T01:02:37.376332+0000","last_unstale":"2024-06-10T01:03:38.239671+0000","last_undegraded":"2024-06-10T01:03:38.239671+0000","last_fullsized":"2024-06-10T01:03:38.239671+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:15.000505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,10,5],"acting":[2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.570465+0000","last_change":"2024-06-10T01:01:44.008342+0000","last_active":"2024-06-10T01:03:38.570465+0000","last_peered":"2024-06-10T01:03:38.570465+0000","last_clean":"2024-06-10T01:03:38.570465+0000","last_became_active":"2024-06-10T01:01:44.007959+0000","last_became_peered":"2024-06-10T01:01:44.007959+0000","last_unstale":"2024-06-10T01:03:38.570465+0000","last_undegraded":"2024-06-10T01:03:38.570465+0000","last_fullsized":"2024-06-10T01:03:38.570465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:36:03.143609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.565044+0000","last_change":"2024-06-10T01:01:47.247854+0000","last_active":"2024-06-10T01:04:35.565044+0000","last_peered":"2024-06-10T01:04:35.565044+0000","last_clean":"2024-06-10T01:04:35.565044+0000","last_became_active":"2024-06-10T01:01:47.247563+0000","last_became_peered":"2024-06-10T01:01:47.247563+0000","last_unstale":"2024-06-10T01:04:35.565044+0000","last_undegraded":"2024-06-10T01:04:35.565044+0000","last_fullsized":"2024-06-10T01:04:35.565044+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:32:13.024815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.237041+0000","last_change":"2024-06-10T01:01:57.282427+0000","last_active":"2024-06-10T01:03:38.237041+0000","last_peered":"2024-06-10T01:03:38.237041+0000","last_clean":"2024-06-10T01:03:38.237041+0000","last_became_active":"2024-06-10T01:01:57.281987+0000","last_became_peered":"2024-06-10T01:01:57.281987+0000","last_unstale":"2024-06-10T01:03:38.237041+0000","last_undegraded":"2024-06-10T01:03:38.237041+0000","last_fullsized":"2024-06-10T01:03:38.237041+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:22:31.736804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770902+0000","last_change":"2024-06-10T01:01:44.006567+0000","last_active":"2024-06-10T01:03:38.770902+0000","last_peered":"2024-06-10T01:03:38.770902+0000","last_clean":"2024-06-10T01:03:38.770902+0000","last_became_active":"2024-06-10T01:01:44.006216+0000","last_became_peered":"2024-06-10T01:01:44.006216+0000","last_unstale":"2024-06-10T01:03:38.770902+0000","last_undegraded":"2024-06-10T01:03:38.770902+0000","last_fullsized":"2024-06-10T01:03:38.770902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:33:55.696873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558008+0000","last_change":"2024-06-10T01:01:47.245326+0000","last_active":"2024-06-10T01:04:35.558008+0000","last_peered":"2024-06-10T01:04:35.558008+0000","last_clean":"2024-06-10T01:04:35.558008+0000","last_became_active":"2024-06-10T01:01:47.245013+0000","last_became_peered":"2024-06-10T01:01:47.245013+0000","last_unstale":"2024-06-10T01:04:35.558008+0000","last_undegraded":"2024-06-10T01:04:35.558008+0000","last_fullsized":"2024-06-10T01:04:35.558008+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:17:56.416627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:56.631218+0000","last_change":"2024-06-10T01:01:57.288997+0000","last_active":"2024-06-10T01:03:56.631218+0000","last_peered":"2024-06-10T01:03:56.631218+0000","last_clean":"2024-06-10T01:03:56.631218+0000","last_became_active":"2024-06-10T01:01:57.288596+0000","last_became_peered":"2024-06-10T01:01:57.288596+0000","last_unstale":"2024-06-10T01:03:56.631218+0000","last_undegraded":"2024-06-10T01:03:56.631218+0000","last_fullsized":"2024-06-10T01:03:56.631218+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:08:14.739737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.269738+0000","last_change":"2024-06-10T01:01:44.010079+0000","last_active":"2024-06-10T01:03:38.269738+0000","last_peered":"2024-06-10T01:03:38.269738+0000","last_clean":"2024-06-10T01:03:38.269738+0000","last_became_active":"2024-06-10T01:01:44.009752+0000","last_became_peered":"2024-06-10T01:01:44.009752+0000","last_unstale":"2024-06-10T01:03:38.269738+0000","last_undegraded":"2024-06-10T01:03:38.269738+0000","last_fullsized":"2024-06-10T01:03:38.269738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:29:35.660935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.554398+0000","last_change":"2024-06-10T01:01:47.245015+0000","last_active":"2024-06-10T01:04:35.554398+0000","last_peered":"2024-06-10T01:04:35.554398+0000","last_clean":"2024-06-10T01:04:35.554398+0000","last_became_active":"2024-06-10T01:01:47.244595+0000","last_became_peered":"2024-06-10T01:01:47.244595+0000","last_unstale":"2024-06-10T01:04:35.554398+0000","last_undegraded":"2024-06-10T01:04:35.554398+0000","last_fullsized":"2024-06-10T01:04:35.554398+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:01:48.446526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.238926+0000","last_change":"2024-06-10T01:01:57.281656+0000","last_active":"2024-06-10T01:03:38.238926+0000","last_peered":"2024-06-10T01:03:38.238926+0000","last_clean":"2024-06-10T01:03:38.238926+0000","last_became_active":"2024-06-10T01:01:57.281295+0000","last_became_peered":"2024-06-10T01:01:57.281295+0000","last_unstale":"2024-06-10T01:03:38.238926+0000","last_undegraded":"2024-06-10T01:03:38.238926+0000","last_fullsized":"2024-06-10T01:03:38.238926+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:04:41.619864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"97'3","reported_seq":228,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:53.970500+0000","last_change":"2024-06-10T01:01:44.011941+0000","last_active":"2024-06-10T01:05:53.970500+0000","last_peered":"2024-06-10T01:05:53.970500+0000","last_clean":"2024-06-10T01:05:53.970500+0000","last_became_active":"2024-06-10T01:01:44.010156+0000","last_became_peered":"2024-06-10T01:01:44.010156+0000","last_unstale":"2024-06-10T01:05:53.970500+0000","last_undegraded":"2024-06-10T01:05:53.970500+0000","last_fullsized":"2024-06-10T01:05:53.970500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:16:15.709544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":180,"num_read_kb":180,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.570257+0000","last_change":"2024-06-10T01:01:47.241720+0000","last_active":"2024-06-10T01:04:35.570257+0000","last_peered":"2024-06-10T01:04:35.570257+0000","last_clean":"2024-06-10T01:04:35.570257+0000","last_became_active":"2024-06-10T01:01:47.241386+0000","last_became_peered":"2024-06-10T01:01:47.241386+0000","last_unstale":"2024-06-10T01:04:35.570257+0000","last_undegraded":"2024-06-10T01:04:35.570257+0000","last_fullsized":"2024-06-10T01:04:35.570257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:42:01.304787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770709+0000","last_change":"2024-06-10T01:01:57.283226+0000","last_active":"2024-06-10T01:03:38.770709+0000","last_peered":"2024-06-10T01:03:38.770709+0000","last_clean":"2024-06-10T01:03:38.770709+0000","last_became_active":"2024-06-10T01:01:57.282720+0000","last_became_peered":"2024-06-10T01:01:57.282720+0000","last_unstale":"2024-06-10T01:03:38.770709+0000","last_undegraded":"2024-06-10T01:03:38.770709+0000","last_fullsized":"2024-06-10T01:03:38.770709+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:35:31.342042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235489+0000","last_change":"2024-06-10T01:01:44.009278+0000","last_active":"2024-06-10T01:03:38.235489+0000","last_peered":"2024-06-10T01:03:38.235489+0000","last_clean":"2024-06-10T01:03:38.235489+0000","last_became_active":"2024-06-10T01:01:44.008841+0000","last_became_peered":"2024-06-10T01:01:44.008841+0000","last_unstale":"2024-06-10T01:03:38.235489+0000","last_undegraded":"2024-06-10T01:03:38.235489+0000","last_fullsized":"2024-06-10T01:03:38.235489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:30:58.392080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.557141+0000","last_change":"2024-06-10T01:01:47.256451+0000","last_active":"2024-06-10T01:04:35.557141+0000","last_peered":"2024-06-10T01:04:35.557141+0000","last_clean":"2024-06-10T01:04:35.557141+0000","last_became_active":"2024-06-10T01:01:47.256036+0000","last_became_peered":"2024-06-10T01:01:47.256036+0000","last_unstale":"2024-06-10T01:04:35.557141+0000","last_undegraded":"2024-06-10T01:04:35.557141+0000","last_fullsized":"2024-06-10T01:04:35.557141+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:57:26.640109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239192+0000","last_change":"2024-06-10T01:02:37.794160+0000","last_active":"2024-06-10T01:03:38.239192+0000","last_peered":"2024-06-10T01:03:38.239192+0000","last_clean":"2024-06-10T01:03:38.239192+0000","last_became_active":"2024-06-10T01:02:37.793633+0000","last_became_peered":"2024-06-10T01:02:37.793633+0000","last_unstale":"2024-06-10T01:03:38.239192+0000","last_undegraded":"2024-06-10T01:03:38.239192+0000","last_fullsized":"2024-06-10T01:03:38.239192+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:48:48.083620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270125+0000","last_change":"2024-06-10T01:01:44.469179+0000","last_active":"2024-06-10T01:03:38.270125+0000","last_peered":"2024-06-10T01:03:38.270125+0000","last_clean":"2024-06-10T01:03:38.270125+0000","last_became_active":"2024-06-10T01:01:44.468669+0000","last_became_peered":"2024-06-10T01:01:44.468669+0000","last_unstale":"2024-06-10T01:03:38.270125+0000","last_undegraded":"2024-06-10T01:03:38.270125+0000","last_fullsized":"2024-06-10T01:03:38.270125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:07:51.629030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.570053+0000","last_change":"2024-06-10T01:01:47.247588+0000","last_active":"2024-06-10T01:04:35.570053+0000","last_peered":"2024-06-10T01:04:35.570053+0000","last_clean":"2024-06-10T01:04:35.570053+0000","last_became_active":"2024-06-10T01:01:47.245750+0000","last_became_peered":"2024-06-10T01:01:47.245750+0000","last_unstale":"2024-06-10T01:04:35.570053+0000","last_undegraded":"2024-06-10T01:04:35.570053+0000","last_fullsized":"2024-06-10T01:04:35.570053+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:12:10.513777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771292+0000","last_change":"2024-06-10T01:01:57.283042+0000","last_active":"2024-06-10T01:03:38.771292+0000","last_peered":"2024-06-10T01:03:38.771292+0000","last_clean":"2024-06-10T01:03:38.771292+0000","last_became_active":"2024-06-10T01:01:57.282600+0000","last_became_peered":"2024-06-10T01:01:57.282600+0000","last_unstale":"2024-06-10T01:03:38.771292+0000","last_undegraded":"2024-06-10T01:03:38.771292+0000","last_fullsized":"2024-06-10T01:03:38.771292+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:06:36.969968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"99'42","reported_seq":461,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:54.972721+0000","last_change":"2024-06-10T01:01:44.011783+0000","last_active":"2024-06-10T01:05:54.972721+0000","last_peered":"2024-06-10T01:05:54.972721+0000","last_clean":"2024-06-10T01:05:54.972721+0000","last_became_active":"2024-06-10T01:01:44.011164+0000","last_became_peered":"2024-06-10T01:01:44.011164+0000","last_unstale":"2024-06-10T01:05:54.972721+0000","last_undegraded":"2024-06-10T01:05:54.972721+0000","last_fullsized":"2024-06-10T01:05:54.972721+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:16:13.642714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70067,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":374,"num_read_kb":257,"num_write":43,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562407+0000","last_change":"2024-06-10T01:01:47.250751+0000","last_active":"2024-06-10T01:04:35.562407+0000","last_peered":"2024-06-10T01:04:35.562407+0000","last_clean":"2024-06-10T01:04:35.562407+0000","last_became_active":"2024-06-10T01:01:47.250451+0000","last_became_peered":"2024-06-10T01:01:47.250451+0000","last_unstale":"2024-06-10T01:04:35.562407+0000","last_undegraded":"2024-06-10T01:04:35.562407+0000","last_fullsized":"2024-06-10T01:04:35.562407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:22:10.121363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270382+0000","last_change":"2024-06-10T01:01:57.289687+0000","last_active":"2024-06-10T01:03:38.270382+0000","last_peered":"2024-06-10T01:03:38.270382+0000","last_clean":"2024-06-10T01:03:38.270382+0000","last_became_active":"2024-06-10T01:01:57.289274+0000","last_became_peered":"2024-06-10T01:01:57.289274+0000","last_unstale":"2024-06-10T01:03:38.270382+0000","last_undegraded":"2024-06-10T01:03:38.270382+0000","last_fullsized":"2024-06-10T01:03:38.270382+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:34:46.873974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'6","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239529+0000","last_change":"2024-06-10T01:01:44.010159+0000","last_active":"2024-06-10T01:03:38.239529+0000","last_peered":"2024-06-10T01:03:38.239529+0000","last_clean":"2024-06-10T01:03:38.239529+0000","last_became_active":"2024-06-10T01:01:44.009744+0000","last_became_peered":"2024-06-10T01:01:44.009744+0000","last_unstale":"2024-06-10T01:03:38.239529+0000","last_undegraded":"2024-06-10T01:03:38.239529+0000","last_fullsized":"2024-06-10T01:03:38.239529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:30.507613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558994+0000","last_change":"2024-06-10T01:01:47.246662+0000","last_active":"2024-06-10T01:04:35.558994+0000","last_peered":"2024-06-10T01:04:35.558994+0000","last_clean":"2024-06-10T01:04:35.558994+0000","last_became_active":"2024-06-10T01:01:47.246268+0000","last_became_peered":"2024-06-10T01:01:47.246268+0000","last_unstale":"2024-06-10T01:04:35.558994+0000","last_undegraded":"2024-06-10T01:04:35.558994+0000","last_fullsized":"2024-06-10T01:04:35.558994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:08:38.089054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.236925+0000","last_change":"2024-06-10T01:01:57.289937+0000","last_active":"2024-06-10T01:03:38.236925+0000","last_peered":"2024-06-10T01:03:38.236925+0000","last_clean":"2024-06-10T01:03:38.236925+0000","last_became_active":"2024-06-10T01:01:57.289407+0000","last_became_peered":"2024-06-10T01:01:57.289407+0000","last_unstale":"2024-06-10T01:03:38.236925+0000","last_undegraded":"2024-06-10T01:03:38.236925+0000","last_fullsized":"2024-06-10T01:03:38.236925+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:01:48.546762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270079+0000","last_change":"2024-06-10T01:01:44.012654+0000","last_active":"2024-06-10T01:03:38.270079+0000","last_peered":"2024-06-10T01:03:38.270079+0000","last_clean":"2024-06-10T01:03:38.270079+0000","last_became_active":"2024-06-10T01:01:44.012246+0000","last_became_peered":"2024-06-10T01:01:44.012246+0000","last_unstale":"2024-06-10T01:03:38.270079+0000","last_undegraded":"2024-06-10T01:03:38.270079+0000","last_fullsized":"2024-06-10T01:03:38.270079+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:31:36.173554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558783+0000","last_change":"2024-06-10T01:01:47.250514+0000","last_active":"2024-06-10T01:04:35.558783+0000","last_peered":"2024-06-10T01:04:35.558783+0000","last_clean":"2024-06-10T01:04:35.558783+0000","last_became_active":"2024-06-10T01:01:47.249050+0000","last_became_peered":"2024-06-10T01:01:47.249050+0000","last_unstale":"2024-06-10T01:04:35.558783+0000","last_undegraded":"2024-06-10T01:04:35.558783+0000","last_fullsized":"2024-06-10T01:04:35.558783+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:51:30.789392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771928+0000","last_change":"2024-06-10T01:01:57.282389+0000","last_active":"2024-06-10T01:03:38.771928+0000","last_peered":"2024-06-10T01:03:38.771928+0000","last_clean":"2024-06-10T01:03:38.771928+0000","last_became_active":"2024-06-10T01:01:57.281857+0000","last_became_peered":"2024-06-10T01:01:57.281857+0000","last_unstale":"2024-06-10T01:03:38.771928+0000","last_undegraded":"2024-06-10T01:03:38.771928+0000","last_fullsized":"2024-06-10T01:03:38.771928+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:59:14.863020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"99'4","reported_seq":235,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:56.977143+0000","last_change":"2024-06-10T01:01:44.011331+0000","last_active":"2024-06-10T01:05:56.977143+0000","last_peered":"2024-06-10T01:05:56.977143+0000","last_clean":"2024-06-10T01:05:56.977143+0000","last_became_active":"2024-06-10T01:01:44.010968+0000","last_became_peered":"2024-06-10T01:01:44.010968+0000","last_unstale":"2024-06-10T01:05:56.977143+0000","last_undegraded":"2024-06-10T01:05:56.977143+0000","last_fullsized":"2024-06-10T01:05:56.977143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:43:37.931225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":896,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":187,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562879+0000","last_change":"2024-06-10T01:01:47.248349+0000","last_active":"2024-06-10T01:04:35.562879+0000","last_peered":"2024-06-10T01:04:35.562879+0000","last_clean":"2024-06-10T01:04:35.562879+0000","last_became_active":"2024-06-10T01:01:47.248008+0000","last_became_peered":"2024-06-10T01:01:47.248008+0000","last_unstale":"2024-06-10T01:04:35.562879+0000","last_undegraded":"2024-06-10T01:04:35.562879+0000","last_fullsized":"2024-06-10T01:04:35.562879+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:45:20.747704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771221+0000","last_change":"2024-06-10T01:01:57.289773+0000","last_active":"2024-06-10T01:03:38.771221+0000","last_peered":"2024-06-10T01:03:38.771221+0000","last_clean":"2024-06-10T01:03:38.771221+0000","last_became_active":"2024-06-10T01:01:57.289371+0000","last_became_peered":"2024-06-10T01:01:57.289371+0000","last_unstale":"2024-06-10T01:03:38.771221+0000","last_undegraded":"2024-06-10T01:03:38.771221+0000","last_fullsized":"2024-06-10T01:03:38.771221+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:24:52.209402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'15","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.257548+0000","last_change":"2024-06-10T01:03:38.257122+0000","last_active":"2024-06-10T01:03:38.257548+0000","last_peered":"2024-06-10T01:03:38.257548+0000","last_clean":"2024-06-10T01:03:38.257548+0000","last_became_active":"2024-06-10T01:03:38.240737+0000","last_became_peered":"2024-06-10T01:03:38.240737+0000","last_unstale":"2024-06-10T01:03:38.257548+0000","last_undegraded":"2024-06-10T01:03:38.257548+0000","last_fullsized":"2024-06-10T01:03:38.257548+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:37:10.925004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562425+0000","last_change":"2024-06-10T01:01:47.245439+0000","last_active":"2024-06-10T01:04:35.562425+0000","last_peered":"2024-06-10T01:04:35.562425+0000","last_clean":"2024-06-10T01:04:35.562425+0000","last_became_active":"2024-06-10T01:01:47.242917+0000","last_became_peered":"2024-06-10T01:01:47.242917+0000","last_unstale":"2024-06-10T01:04:35.562425+0000","last_undegraded":"2024-06-10T01:04:35.562425+0000","last_fullsized":"2024-06-10T01:04:35.562425+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:13:12.614230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.269795+0000","last_change":"2024-06-10T01:01:57.284731+0000","last_active":"2024-06-10T01:03:38.269795+0000","last_peered":"2024-06-10T01:03:38.269795+0000","last_clean":"2024-06-10T01:03:38.269795+0000","last_became_active":"2024-06-10T01:01:57.284301+0000","last_became_peered":"2024-06-10T01:01:57.284301+0000","last_unstale":"2024-06-10T01:03:38.269795+0000","last_undegraded":"2024-06-10T01:03:38.269795+0000","last_fullsized":"2024-06-10T01:03:38.269795+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:01:36.518214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"99'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:59.314831+0000","last_change":"2024-06-10T01:01:44.010154+0000","last_active":"2024-06-10T01:03:59.314831+0000","last_peered":"2024-06-10T01:03:59.314831+0000","last_clean":"2024-06-10T01:03:59.314831+0000","last_became_active":"2024-06-10T01:01:44.009656+0000","last_became_peered":"2024-06-10T01:01:44.009656+0000","last_unstale":"2024-06-10T01:03:59.314831+0000","last_undegraded":"2024-06-10T01:03:59.314831+0000","last_fullsized":"2024-06-10T01:03:59.314831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:33:08.789946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.550140+0000","last_change":"2024-06-10T01:02:37.378484+0000","last_active":"2024-06-10T01:04:35.550140+0000","last_peered":"2024-06-10T01:04:35.550140+0000","last_clean":"2024-06-10T01:04:35.550140+0000","last_became_active":"2024-06-10T01:02:37.378092+0000","last_became_peered":"2024-06-10T01:02:37.378092+0000","last_unstale":"2024-06-10T01:04:35.550140+0000","last_undegraded":"2024-06-10T01:04:35.550140+0000","last_fullsized":"2024-06-10T01:04:35.550140+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:19:46.599235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235375+0000","last_change":"2024-06-10T01:01:57.332887+0000","last_active":"2024-06-10T01:03:38.235375+0000","last_peered":"2024-06-10T01:03:38.235375+0000","last_clean":"2024-06-10T01:03:38.235375+0000","last_became_active":"2024-06-10T01:01:57.332285+0000","last_became_peered":"2024-06-10T01:01:57.332285+0000","last_unstale":"2024-06-10T01:03:38.235375+0000","last_undegraded":"2024-06-10T01:03:38.235375+0000","last_fullsized":"2024-06-10T01:03:38.235375+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:09:12.233405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:50.167401+0000","last_change":"2024-06-10T01:01:44.466831+0000","last_active":"2024-06-10T01:03:50.167401+0000","last_peered":"2024-06-10T01:03:50.167401+0000","last_clean":"2024-06-10T01:03:50.167401+0000","last_became_active":"2024-06-10T01:01:44.466484+0000","last_became_peered":"2024-06-10T01:01:44.466484+0000","last_unstale":"2024-06-10T01:03:50.167401+0000","last_undegraded":"2024-06-10T01:03:50.167401+0000","last_fullsized":"2024-06-10T01:03:50.167401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:19:28.385504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.556135+0000","last_change":"2024-06-10T01:01:47.252399+0000","last_active":"2024-06-10T01:04:35.556135+0000","last_peered":"2024-06-10T01:04:35.556135+0000","last_clean":"2024-06-10T01:04:35.556135+0000","last_became_active":"2024-06-10T01:01:47.252069+0000","last_became_peered":"2024-06-10T01:01:47.252069+0000","last_unstale":"2024-06-10T01:04:35.556135+0000","last_undegraded":"2024-06-10T01:04:35.556135+0000","last_fullsized":"2024-06-10T01:04:35.556135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:02:02.291174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771406+0000","last_change":"2024-06-10T01:01:57.285670+0000","last_active":"2024-06-10T01:03:38.771406+0000","last_peered":"2024-06-10T01:03:38.771406+0000","last_clean":"2024-06-10T01:03:38.771406+0000","last_became_active":"2024-06-10T01:01:57.285281+0000","last_became_peered":"2024-06-10T01:01:57.285281+0000","last_unstale":"2024-06-10T01:03:38.771406+0000","last_undegraded":"2024-06-10T01:03:38.771406+0000","last_fullsized":"2024-06-10T01:03:38.771406+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:58.956934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:48.604327+0000","last_change":"2024-06-10T01:01:44.008013+0000","last_active":"2024-06-10T01:03:48.604327+0000","last_peered":"2024-06-10T01:03:48.604327+0000","last_clean":"2024-06-10T01:03:48.604327+0000","last_became_active":"2024-06-10T01:01:44.007591+0000","last_became_peered":"2024-06-10T01:01:44.007591+0000","last_unstale":"2024-06-10T01:03:48.604327+0000","last_undegraded":"2024-06-10T01:03:48.604327+0000","last_fullsized":"2024-06-10T01:03:48.604327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:18:20.862340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.550926+0000","last_change":"2024-06-10T01:01:47.254756+0000","last_active":"2024-06-10T01:04:35.550926+0000","last_peered":"2024-06-10T01:04:35.550926+0000","last_clean":"2024-06-10T01:04:35.550926+0000","last_became_active":"2024-06-10T01:01:47.254418+0000","last_became_peered":"2024-06-10T01:01:47.254418+0000","last_unstale":"2024-06-10T01:04:35.550926+0000","last_undegraded":"2024-06-10T01:04:35.550926+0000","last_fullsized":"2024-06-10T01:04:35.550926+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:46:31.452539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770995+0000","last_change":"2024-06-10T01:01:57.332792+0000","last_active":"2024-06-10T01:03:38.770995+0000","last_peered":"2024-06-10T01:03:38.770995+0000","last_clean":"2024-06-10T01:03:38.770995+0000","last_became_active":"2024-06-10T01:01:57.332331+0000","last_became_peered":"2024-06-10T01:01:57.332331+0000","last_unstale":"2024-06-10T01:03:38.770995+0000","last_undegraded":"2024-06-10T01:03:38.770995+0000","last_fullsized":"2024-06-10T01:03:38.770995+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:46:41.419807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.565745+0000","last_change":"2024-06-10T01:01:47.247857+0000","last_active":"2024-06-10T01:04:35.565745+0000","last_peered":"2024-06-10T01:04:35.565745+0000","last_clean":"2024-06-10T01:04:35.565745+0000","last_became_active":"2024-06-10T01:01:47.247552+0000","last_became_peered":"2024-06-10T01:01:47.247552+0000","last_unstale":"2024-06-10T01:04:35.565745+0000","last_undegraded":"2024-06-10T01:04:35.565745+0000","last_fullsized":"2024-06-10T01:04:35.565745+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:29:40.521887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"99'5","reported_seq":235,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:57.978068+0000","last_change":"2024-06-10T01:01:44.466631+0000","last_active":"2024-06-10T01:05:57.978068+0000","last_peered":"2024-06-10T01:05:57.978068+0000","last_clean":"2024-06-10T01:05:57.978068+0000","last_became_active":"2024-06-10T01:01:44.466280+0000","last_became_peered":"2024-06-10T01:01:44.466280+0000","last_unstale":"2024-06-10T01:05:57.978068+0000","last_undegraded":"2024-06-10T01:05:57.978068+0000","last_fullsized":"2024-06-10T01:05:57.978068+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:09:33.658112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":184,"num_read_kb":184,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235365+0000","last_change":"2024-06-10T01:01:57.281973+0000","last_active":"2024-06-10T01:03:38.235365+0000","last_peered":"2024-06-10T01:03:38.235365+0000","last_clean":"2024-06-10T01:03:38.235365+0000","last_became_active":"2024-06-10T01:01:57.281386+0000","last_became_peered":"2024-06-10T01:01:57.281386+0000","last_unstale":"2024-06-10T01:03:38.235365+0000","last_undegraded":"2024-06-10T01:03:38.235365+0000","last_fullsized":"2024-06-10T01:03:38.235365+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:10:28.236250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562526+0000","last_change":"2024-06-10T01:01:47.237436+0000","last_active":"2024-06-10T01:04:35.562526+0000","last_peered":"2024-06-10T01:04:35.562526+0000","last_clean":"2024-06-10T01:04:35.562526+0000","last_became_active":"2024-06-10T01:01:47.237099+0000","last_became_peered":"2024-06-10T01:01:47.237099+0000","last_unstale":"2024-06-10T01:04:35.562526+0000","last_undegraded":"2024-06-10T01:04:35.562526+0000","last_fullsized":"2024-06-10T01:04:35.562526+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:34:53.838877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'3","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:48.634491+0000","last_change":"2024-06-10T01:01:44.010655+0000","last_active":"2024-06-10T01:03:48.634491+0000","last_peered":"2024-06-10T01:03:48.634491+0000","last_clean":"2024-06-10T01:03:48.634491+0000","last_became_active":"2024-06-10T01:01:44.009918+0000","last_became_peered":"2024-06-10T01:01:44.009918+0000","last_unstale":"2024-06-10T01:03:48.634491+0000","last_undegraded":"2024-06-10T01:03:48.634491+0000","last_fullsized":"2024-06-10T01:03:48.634491+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:47:32.052650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":367,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234906+0000","last_change":"2024-06-10T01:00:23.822447+0000","last_active":"2024-06-10T01:03:38.234906+0000","last_peered":"2024-06-10T01:03:38.234906+0000","last_clean":"2024-06-10T01:03:38.234906+0000","last_became_active":"2024-06-10T01:00:23.821966+0000","last_became_peered":"2024-06-10T01:00:23.821966+0000","last_unstale":"2024-06-10T01:03:38.234906+0000","last_undegraded":"2024-06-10T01:03:38.234906+0000","last_fullsized":"2024-06-10T01:03:38.234906+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:56:59.964661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:56:59.964661+0000","last_clean_scrub_stamp":"2024-06-10T00:56:59.964661+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:03:43.732875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270272+0000","last_change":"2024-06-10T01:01:57.332980+0000","last_active":"2024-06-10T01:03:38.270272+0000","last_peered":"2024-06-10T01:03:38.270272+0000","last_clean":"2024-06-10T01:03:38.270272+0000","last_became_active":"2024-06-10T01:01:57.332437+0000","last_became_peered":"2024-06-10T01:01:57.332437+0000","last_unstale":"2024-06-10T01:03:38.270272+0000","last_undegraded":"2024-06-10T01:03:38.270272+0000","last_fullsized":"2024-06-10T01:03:38.270272+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:27:35.029079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559893+0000","last_change":"2024-06-10T01:01:47.255926+0000","last_active":"2024-06-10T01:04:35.559893+0000","last_peered":"2024-06-10T01:04:35.559893+0000","last_clean":"2024-06-10T01:04:35.559893+0000","last_became_active":"2024-06-10T01:01:47.255571+0000","last_became_peered":"2024-06-10T01:01:47.255571+0000","last_unstale":"2024-06-10T01:04:35.559893+0000","last_undegraded":"2024-06-10T01:04:35.559893+0000","last_fullsized":"2024-06-10T01:04:35.559893+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:05:22.084037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:43.603848+0000","last_change":"2024-06-10T01:01:44.007509+0000","last_active":"2024-06-10T01:03:43.603848+0000","last_peered":"2024-06-10T01:03:43.603848+0000","last_clean":"2024-06-10T01:03:43.603848+0000","last_became_active":"2024-06-10T01:01:44.007178+0000","last_became_peered":"2024-06-10T01:01:44.007178+0000","last_unstale":"2024-06-10T01:03:43.603848+0000","last_undegraded":"2024-06-10T01:03:43.603848+0000","last_fullsized":"2024-06-10T01:03:43.603848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:02:52.791886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.236790+0000","last_change":"2024-06-10T01:01:57.285626+0000","last_active":"2024-06-10T01:03:38.236790+0000","last_peered":"2024-06-10T01:03:38.236790+0000","last_clean":"2024-06-10T01:03:38.236790+0000","last_became_active":"2024-06-10T01:01:57.285098+0000","last_became_peered":"2024-06-10T01:01:57.285098+0000","last_unstale":"2024-06-10T01:03:38.236790+0000","last_undegraded":"2024-06-10T01:03:38.236790+0000","last_fullsized":"2024-06-10T01:03:38.236790+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:13:49.881770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:49.313610+0000","last_change":"2024-06-10T01:01:44.469028+0000","last_active":"2024-06-10T01:03:49.313610+0000","last_peered":"2024-06-10T01:03:49.313610+0000","last_clean":"2024-06-10T01:03:49.313610+0000","last_became_active":"2024-06-10T01:01:44.468570+0000","last_became_peered":"2024-06-10T01:01:44.468570+0000","last_unstale":"2024-06-10T01:03:49.313610+0000","last_undegraded":"2024-06-10T01:03:49.313610+0000","last_fullsized":"2024-06-10T01:03:49.313610+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:12:11.287959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.560378+0000","last_change":"2024-06-10T01:01:47.253414+0000","last_active":"2024-06-10T01:04:35.560378+0000","last_peered":"2024-06-10T01:04:35.560378+0000","last_clean":"2024-06-10T01:04:35.560378+0000","last_became_active":"2024-06-10T01:01:47.253060+0000","last_became_peered":"2024-06-10T01:01:47.253060+0000","last_unstale":"2024-06-10T01:04:35.560378+0000","last_undegraded":"2024-06-10T01:04:35.560378+0000","last_fullsized":"2024-06-10T01:04:35.560378+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:24:55.534164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:58.961104+0000","last_change":"2024-06-10T01:01:57.332756+0000","last_active":"2024-06-10T01:03:58.961104+0000","last_peered":"2024-06-10T01:03:58.961104+0000","last_clean":"2024-06-10T01:03:58.961104+0000","last_became_active":"2024-06-10T01:01:57.332361+0000","last_became_peered":"2024-06-10T01:01:57.332361+0000","last_unstale":"2024-06-10T01:03:58.961104+0000","last_undegraded":"2024-06-10T01:03:58.961104+0000","last_fullsized":"2024-06-10T01:03:58.961104+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:02:28.275407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"99'14","reported_seq":59,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:36.265950+0000","last_change":"2024-06-10T01:01:44.005164+0000","last_active":"2024-06-10T01:04:36.265950+0000","last_peered":"2024-06-10T01:04:36.265950+0000","last_clean":"2024-06-10T01:04:36.265950+0000","last_became_active":"2024-06-10T01:01:44.004816+0000","last_became_peered":"2024-06-10T01:01:44.004816+0000","last_unstale":"2024-06-10T01:04:36.265950+0000","last_undegraded":"2024-06-10T01:04:36.265950+0000","last_fullsized":"2024-06-10T01:04:36.265950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:11:56.385267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.557511+0000","last_change":"2024-06-10T01:01:47.247951+0000","last_active":"2024-06-10T01:04:35.557511+0000","last_peered":"2024-06-10T01:04:35.557511+0000","last_clean":"2024-06-10T01:04:35.557511+0000","last_became_active":"2024-06-10T01:01:47.246996+0000","last_became_peered":"2024-06-10T01:01:47.246996+0000","last_unstale":"2024-06-10T01:04:35.557511+0000","last_undegraded":"2024-06-10T01:04:35.557511+0000","last_fullsized":"2024-06-10T01:04:35.557511+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:39:18.167021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239228+0000","last_change":"2024-06-10T01:01:57.278130+0000","last_active":"2024-06-10T01:03:38.239228+0000","last_peered":"2024-06-10T01:03:38.239228+0000","last_clean":"2024-06-10T01:03:38.239228+0000","last_became_active":"2024-06-10T01:01:57.277697+0000","last_became_peered":"2024-06-10T01:01:57.277697+0000","last_unstale":"2024-06-10T01:03:38.239228+0000","last_undegraded":"2024-06-10T01:03:38.239228+0000","last_fullsized":"2024-06-10T01:03:38.239228+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:17:01.662152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239618+0000","last_change":"2024-06-10T01:01:44.553176+0000","last_active":"2024-06-10T01:03:38.239618+0000","last_peered":"2024-06-10T01:03:38.239618+0000","last_clean":"2024-06-10T01:03:38.239618+0000","last_became_active":"2024-06-10T01:01:44.552797+0000","last_became_peered":"2024-06-10T01:01:44.552797+0000","last_unstale":"2024-06-10T01:03:38.239618+0000","last_undegraded":"2024-06-10T01:03:38.239618+0000","last_fullsized":"2024-06-10T01:03:38.239618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:21:13.960304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.556104+0000","last_change":"2024-06-10T01:01:47.255796+0000","last_active":"2024-06-10T01:04:35.556104+0000","last_peered":"2024-06-10T01:04:35.556104+0000","last_clean":"2024-06-10T01:04:35.556104+0000","last_became_active":"2024-06-10T01:01:47.255495+0000","last_became_peered":"2024-06-10T01:01:47.255495+0000","last_unstale":"2024-06-10T01:04:35.556104+0000","last_undegraded":"2024-06-10T01:04:35.556104+0000","last_fullsized":"2024-06-10T01:04:35.556104+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:06:42.209005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239491+0000","last_change":"2024-06-10T01:01:57.332869+0000","last_active":"2024-06-10T01:03:38.239491+0000","last_peered":"2024-06-10T01:03:38.239491+0000","last_clean":"2024-06-10T01:03:38.239491+0000","last_became_active":"2024-06-10T01:01:57.332391+0000","last_became_peered":"2024-06-10T01:01:57.332391+0000","last_unstale":"2024-06-10T01:03:38.239491+0000","last_undegraded":"2024-06-10T01:03:38.239491+0000","last_fullsized":"2024-06-10T01:03:38.239491+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:58:31.443626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770788+0000","last_change":"2024-06-10T01:01:44.466861+0000","last_active":"2024-06-10T01:03:38.770788+0000","last_peered":"2024-06-10T01:03:38.770788+0000","last_clean":"2024-06-10T01:03:38.770788+0000","last_became_active":"2024-06-10T01:01:44.466553+0000","last_became_peered":"2024-06-10T01:01:44.466553+0000","last_unstale":"2024-06-10T01:03:38.770788+0000","last_undegraded":"2024-06-10T01:03:38.770788+0000","last_fullsized":"2024-06-10T01:03:38.770788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:55:52.545594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":41,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.544035+0000","last_change":"2024-06-10T01:01:47.244972+0000","last_active":"2024-06-10T01:04:35.544035+0000","last_peered":"2024-06-10T01:04:35.544035+0000","last_clean":"2024-06-10T01:04:35.544035+0000","last_became_active":"2024-06-10T01:01:47.244649+0000","last_became_peered":"2024-06-10T01:01:47.244649+0000","last_unstale":"2024-06-10T01:04:35.544035+0000","last_undegraded":"2024-06-10T01:04:35.544035+0000","last_fullsized":"2024-06-10T01:04:35.544035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:10:46.909200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235461+0000","last_change":"2024-06-10T01:02:37.592380+0000","last_active":"2024-06-10T01:03:38.235461+0000","last_peered":"2024-06-10T01:03:38.235461+0000","last_clean":"2024-06-10T01:03:38.235461+0000","last_became_active":"2024-06-10T01:02:37.591856+0000","last_became_peered":"2024-06-10T01:02:37.591856+0000","last_unstale":"2024-06-10T01:03:38.235461+0000","last_undegraded":"2024-06-10T01:03:38.235461+0000","last_fullsized":"2024-06-10T01:03:38.235461+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:00:25.717192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239006+0000","last_change":"2024-06-10T01:01:44.008116+0000","last_active":"2024-06-10T01:03:38.239006+0000","last_peered":"2024-06-10T01:03:38.239006+0000","last_clean":"2024-06-10T01:03:38.239006+0000","last_became_active":"2024-06-10T01:01:44.007778+0000","last_became_peered":"2024-06-10T01:01:44.007778+0000","last_unstale":"2024-06-10T01:03:38.239006+0000","last_undegraded":"2024-06-10T01:03:38.239006+0000","last_fullsized":"2024-06-10T01:03:38.239006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:18:57.267296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.556264+0000","last_change":"2024-06-10T01:01:47.252833+0000","last_active":"2024-06-10T01:04:35.556264+0000","last_peered":"2024-06-10T01:04:35.556264+0000","last_clean":"2024-06-10T01:04:35.556264+0000","last_became_active":"2024-06-10T01:01:47.252326+0000","last_became_peered":"2024-06-10T01:01:47.252326+0000","last_unstale":"2024-06-10T01:04:35.556264+0000","last_undegraded":"2024-06-10T01:04:35.556264+0000","last_fullsized":"2024-06-10T01:04:35.556264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:00:42.673838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239284+0000","last_change":"2024-06-10T01:01:57.275975+0000","last_active":"2024-06-10T01:03:38.239284+0000","last_peered":"2024-06-10T01:03:38.239284+0000","last_clean":"2024-06-10T01:03:38.239284+0000","last_became_active":"2024-06-10T01:01:57.275408+0000","last_became_peered":"2024-06-10T01:01:57.275408+0000","last_unstale":"2024-06-10T01:03:38.239284+0000","last_undegraded":"2024-06-10T01:03:38.239284+0000","last_fullsized":"2024-06-10T01:03:38.239284+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:12:42.376636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'1","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:40.912786+0000","last_change":"2024-06-10T01:03:38.243422+0000","last_active":"2024-06-10T01:03:40.912786+0000","last_peered":"2024-06-10T01:03:40.912786+0000","last_clean":"2024-06-10T01:03:40.912786+0000","last_became_active":"2024-06-10T01:03:38.239916+0000","last_became_peered":"2024-06-10T01:03:38.239916+0000","last_unstale":"2024-06-10T01:03:40.912786+0000","last_undegraded":"2024-06-10T01:03:40.912786+0000","last_fullsized":"2024-06-10T01:03:40.912786+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:30:01.399956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.555295+0000","last_change":"2024-06-10T01:01:47.243154+0000","last_active":"2024-06-10T01:04:35.555295+0000","last_peered":"2024-06-10T01:04:35.555295+0000","last_clean":"2024-06-10T01:04:35.555295+0000","last_became_active":"2024-06-10T01:01:47.242851+0000","last_became_peered":"2024-06-10T01:01:47.242851+0000","last_unstale":"2024-06-10T01:04:35.555295+0000","last_undegraded":"2024-06-10T01:04:35.555295+0000","last_fullsized":"2024-06-10T01:04:35.555295+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:41:05.857777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239584+0000","last_change":"2024-06-10T01:01:57.333613+0000","last_active":"2024-06-10T01:03:38.239584+0000","last_peered":"2024-06-10T01:03:38.239584+0000","last_clean":"2024-06-10T01:03:38.239584+0000","last_became_active":"2024-06-10T01:01:57.333019+0000","last_became_peered":"2024-06-10T01:01:57.333019+0000","last_unstale":"2024-06-10T01:03:38.239584+0000","last_undegraded":"2024-06-10T01:03:38.239584+0000","last_fullsized":"2024-06-10T01:03:38.239584+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:11.912885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235303+0000","last_change":"2024-06-10T01:01:44.008221+0000","last_active":"2024-06-10T01:03:38.235303+0000","last_peered":"2024-06-10T01:03:38.235303+0000","last_clean":"2024-06-10T01:03:38.235303+0000","last_became_active":"2024-06-10T01:01:44.007880+0000","last_became_peered":"2024-06-10T01:01:44.007880+0000","last_unstale":"2024-06-10T01:03:38.235303+0000","last_undegraded":"2024-06-10T01:03:38.235303+0000","last_fullsized":"2024-06-10T01:03:38.235303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:40:33.162925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558463+0000","last_change":"2024-06-10T01:01:47.254902+0000","last_active":"2024-06-10T01:04:35.558463+0000","last_peered":"2024-06-10T01:04:35.558463+0000","last_clean":"2024-06-10T01:04:35.558463+0000","last_became_active":"2024-06-10T01:01:47.254555+0000","last_became_peered":"2024-06-10T01:01:47.254555+0000","last_unstale":"2024-06-10T01:04:35.558463+0000","last_undegraded":"2024-06-10T01:04:35.558463+0000","last_fullsized":"2024-06-10T01:04:35.558463+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:03:17.571001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770786+0000","last_change":"2024-06-10T01:01:57.284491+0000","last_active":"2024-06-10T01:03:38.770786+0000","last_peered":"2024-06-10T01:03:38.770786+0000","last_clean":"2024-06-10T01:03:38.770786+0000","last_became_active":"2024-06-10T01:01:57.283795+0000","last_became_peered":"2024-06-10T01:01:57.283795+0000","last_unstale":"2024-06-10T01:03:38.770786+0000","last_undegraded":"2024-06-10T01:03:38.770786+0000","last_fullsized":"2024-06-10T01:03:38.770786+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:49:00.381664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239889+0000","last_change":"2024-06-10T01:01:44.553159+0000","last_active":"2024-06-10T01:03:38.239889+0000","last_peered":"2024-06-10T01:03:38.239889+0000","last_clean":"2024-06-10T01:03:38.239889+0000","last_became_active":"2024-06-10T01:01:44.552759+0000","last_became_peered":"2024-06-10T01:01:44.552759+0000","last_unstale":"2024-06-10T01:03:38.239889+0000","last_undegraded":"2024-06-10T01:03:38.239889+0000","last_fullsized":"2024-06-10T01:03:38.239889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:19:00.110790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559093+0000","last_change":"2024-06-10T01:01:47.247803+0000","last_active":"2024-06-10T01:04:35.559093+0000","last_peered":"2024-06-10T01:04:35.559093+0000","last_clean":"2024-06-10T01:04:35.559093+0000","last_became_active":"2024-06-10T01:01:47.246715+0000","last_became_peered":"2024-06-10T01:01:47.246715+0000","last_unstale":"2024-06-10T01:04:35.559093+0000","last_undegraded":"2024-06-10T01:04:35.559093+0000","last_fullsized":"2024-06-10T01:04:35.559093+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:22.429002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234192+0000","last_change":"2024-06-10T01:01:57.287668+0000","last_active":"2024-06-10T01:03:38.234192+0000","last_peered":"2024-06-10T01:03:38.234192+0000","last_clean":"2024-06-10T01:03:38.234192+0000","last_became_active":"2024-06-10T01:01:57.287060+0000","last_became_peered":"2024-06-10T01:01:57.287060+0000","last_unstale":"2024-06-10T01:03:38.234192+0000","last_undegraded":"2024-06-10T01:03:38.234192+0000","last_fullsized":"2024-06-10T01:03:38.234192+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:04:35.883613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270251+0000","last_change":"2024-06-10T01:01:44.013051+0000","last_active":"2024-06-10T01:03:38.270251+0000","last_peered":"2024-06-10T01:03:38.270251+0000","last_clean":"2024-06-10T01:03:38.270251+0000","last_became_active":"2024-06-10T01:01:44.012667+0000","last_became_peered":"2024-06-10T01:01:44.012667+0000","last_unstale":"2024-06-10T01:03:38.270251+0000","last_undegraded":"2024-06-10T01:03:38.270251+0000","last_fullsized":"2024-06-10T01:03:38.270251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:10:49.650082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.563083+0000","last_change":"2024-06-10T01:01:47.251209+0000","last_active":"2024-06-10T01:04:35.563083+0000","last_peered":"2024-06-10T01:04:35.563083+0000","last_clean":"2024-06-10T01:04:35.563083+0000","last_became_active":"2024-06-10T01:01:47.250858+0000","last_became_peered":"2024-06-10T01:01:47.250858+0000","last_unstale":"2024-06-10T01:04:35.563083+0000","last_undegraded":"2024-06-10T01:04:35.563083+0000","last_fullsized":"2024-06-10T01:04:35.563083+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:47.272727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.269600+0000","last_change":"2024-06-10T01:01:57.284187+0000","last_active":"2024-06-10T01:03:38.269600+0000","last_peered":"2024-06-10T01:03:38.269600+0000","last_clean":"2024-06-10T01:03:38.269600+0000","last_became_active":"2024-06-10T01:01:57.283739+0000","last_became_peered":"2024-06-10T01:01:57.283739+0000","last_unstale":"2024-06-10T01:03:38.269600+0000","last_undegraded":"2024-06-10T01:03:38.269600+0000","last_fullsized":"2024-06-10T01:03:38.269600+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:13:34.630708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771085+0000","last_change":"2024-06-10T01:01:44.005702+0000","last_active":"2024-06-10T01:03:38.771085+0000","last_peered":"2024-06-10T01:03:38.771085+0000","last_clean":"2024-06-10T01:03:38.771085+0000","last_became_active":"2024-06-10T01:01:44.005354+0000","last_became_peered":"2024-06-10T01:01:44.005354+0000","last_unstale":"2024-06-10T01:03:38.771085+0000","last_undegraded":"2024-06-10T01:03:38.771085+0000","last_fullsized":"2024-06-10T01:03:38.771085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:51:14.819688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558942+0000","last_change":"2024-06-10T01:01:47.250325+0000","last_active":"2024-06-10T01:04:35.558942+0000","last_peered":"2024-06-10T01:04:35.558942+0000","last_clean":"2024-06-10T01:04:35.558942+0000","last_became_active":"2024-06-10T01:01:47.249591+0000","last_became_peered":"2024-06-10T01:01:47.249591+0000","last_unstale":"2024-06-10T01:04:35.558942+0000","last_undegraded":"2024-06-10T01:04:35.558942+0000","last_fullsized":"2024-06-10T01:04:35.558942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:17:16.268717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.563635+0000","last_change":"2024-06-10T01:01:47.253449+0000","last_active":"2024-06-10T01:04:35.563635+0000","last_peered":"2024-06-10T01:04:35.563635+0000","last_clean":"2024-06-10T01:04:35.563635+0000","last_became_active":"2024-06-10T01:01:47.253144+0000","last_became_peered":"2024-06-10T01:01:47.253144+0000","last_unstale":"2024-06-10T01:04:35.563635+0000","last_undegraded":"2024-06-10T01:04:35.563635+0000","last_fullsized":"2024-06-10T01:04:35.563635+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:32:22.353409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.564183+0000","last_change":"2024-06-10T01:01:47.258241+0000","last_active":"2024-06-10T01:04:35.564183+0000","last_peered":"2024-06-10T01:04:35.564183+0000","last_clean":"2024-06-10T01:04:35.564183+0000","last_became_active":"2024-06-10T01:01:47.257823+0000","last_became_peered":"2024-06-10T01:01:47.257823+0000","last_unstale":"2024-06-10T01:04:35.564183+0000","last_undegraded":"2024-06-10T01:04:35.564183+0000","last_fullsized":"2024-06-10T01:04:35.564183+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:11:23.652479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559614+0000","last_change":"2024-06-10T01:01:47.256218+0000","last_active":"2024-06-10T01:04:35.559614+0000","last_peered":"2024-06-10T01:04:35.559614+0000","last_clean":"2024-06-10T01:04:35.559614+0000","last_became_active":"2024-06-10T01:01:47.255950+0000","last_became_peered":"2024-06-10T01:01:47.255950+0000","last_unstale":"2024-06-10T01:04:35.559614+0000","last_undegraded":"2024-06-10T01:04:35.559614+0000","last_fullsized":"2024-06-10T01:04:35.559614+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:59:37.283646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.556367+0000","last_change":"2024-06-10T01:01:47.253109+0000","last_active":"2024-06-10T01:04:35.556367+0000","last_peered":"2024-06-10T01:04:35.556367+0000","last_clean":"2024-06-10T01:04:35.556367+0000","last_became_active":"2024-06-10T01:01:47.252767+0000","last_became_peered":"2024-06-10T01:01:47.252767+0000","last_unstale":"2024-06-10T01:04:35.556367+0000","last_undegraded":"2024-06-10T01:04:35.556367+0000","last_fullsized":"2024-06-10T01:04:35.556367+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:57:13.177230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.550864+0000","last_change":"2024-06-10T01:02:37.375094+0000","last_active":"2024-06-10T01:04:35.550864+0000","last_peered":"2024-06-10T01:04:35.550864+0000","last_clean":"2024-06-10T01:04:35.550864+0000","last_became_active":"2024-06-10T01:02:37.374527+0000","last_became_peered":"2024-06-10T01:02:37.374527+0000","last_unstale":"2024-06-10T01:04:35.550864+0000","last_undegraded":"2024-06-10T01:04:35.550864+0000","last_fullsized":"2024-06-10T01:04:35.550864+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:34:41.205816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.551206+0000","last_change":"2024-06-10T01:01:47.250243+0000","last_active":"2024-06-10T01:04:35.551206+0000","last_peered":"2024-06-10T01:04:35.551206+0000","last_clean":"2024-06-10T01:04:35.551206+0000","last_became_active":"2024-06-10T01:01:47.249907+0000","last_became_peered":"2024-06-10T01:01:47.249907+0000","last_unstale":"2024-06-10T01:04:35.551206+0000","last_undegraded":"2024-06-10T01:04:35.551206+0000","last_fullsized":"2024-06-10T01:04:35.551206+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:31:32.933660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.564063+0000","last_change":"2024-06-10T01:01:47.254173+0000","last_active":"2024-06-10T01:04:35.564063+0000","last_peered":"2024-06-10T01:04:35.564063+0000","last_clean":"2024-06-10T01:04:35.564063+0000","last_became_active":"2024-06-10T01:01:47.253796+0000","last_became_peered":"2024-06-10T01:01:47.253796+0000","last_unstale":"2024-06-10T01:04:35.564063+0000","last_undegraded":"2024-06-10T01:04:35.564063+0000","last_fullsized":"2024-06-10T01:04:35.564063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:41:12.199114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":41,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.548118+0000","last_change":"2024-06-10T01:01:47.253920+0000","last_active":"2024-06-10T01:04:35.548118+0000","last_peered":"2024-06-10T01:04:35.548118+0000","last_clean":"2024-06-10T01:04:35.548118+0000","last_became_active":"2024-06-10T01:01:47.253288+0000","last_became_peered":"2024-06-10T01:01:47.253288+0000","last_unstale":"2024-06-10T01:04:35.548118+0000","last_undegraded":"2024-06-10T01:04:35.548118+0000","last_fullsized":"2024-06-10T01:04:35.548118+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:41:20.406263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.564816+0000","last_change":"2024-06-10T01:01:47.240772+0000","last_active":"2024-06-10T01:04:35.564816+0000","last_peered":"2024-06-10T01:04:35.564816+0000","last_clean":"2024-06-10T01:04:35.564816+0000","last_became_active":"2024-06-10T01:01:47.238258+0000","last_became_peered":"2024-06-10T01:01:47.238258+0000","last_unstale":"2024-06-10T01:04:35.564816+0000","last_undegraded":"2024-06-10T01:04:35.564816+0000","last_fullsized":"2024-06-10T01:04:35.564816+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:55:25.211434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":24,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.544703+0000","last_change":"2024-06-10T01:02:37.376724+0000","last_active":"2024-06-10T01:04:35.544703+0000","last_peered":"2024-06-10T01:04:35.544703+0000","last_clean":"2024-06-10T01:04:35.544703+0000","last_became_active":"2024-06-10T01:02:37.376329+0000","last_became_peered":"2024-06-10T01:02:37.376329+0000","last_unstale":"2024-06-10T01:04:35.544703+0000","last_undegraded":"2024-06-10T01:04:35.544703+0000","last_fullsized":"2024-06-10T01:04:35.544703+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:38:21.837958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.565197+0000","last_change":"2024-06-10T01:01:47.244463+0000","last_active":"2024-06-10T01:04:35.565197+0000","last_peered":"2024-06-10T01:04:35.565197+0000","last_clean":"2024-06-10T01:04:35.565197+0000","last_became_active":"2024-06-10T01:01:47.244114+0000","last_became_peered":"2024-06-10T01:01:47.244114+0000","last_unstale":"2024-06-10T01:04:35.565197+0000","last_undegraded":"2024-06-10T01:04:35.565197+0000","last_fullsized":"2024-06-10T01:04:35.565197+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:47:37.098894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559211+0000","last_change":"2024-06-10T01:01:47.248317+0000","last_active":"2024-06-10T01:04:35.559211+0000","last_peered":"2024-06-10T01:04:35.559211+0000","last_clean":"2024-06-10T01:04:35.559211+0000","last_became_active":"2024-06-10T01:01:47.247958+0000","last_became_peered":"2024-06-10T01:01:47.247958+0000","last_unstale":"2024-06-10T01:04:35.559211+0000","last_undegraded":"2024-06-10T01:04:35.559211+0000","last_fullsized":"2024-06-10T01:04:35.559211+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:39:59.267957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.552389+0000","last_change":"2024-06-10T01:01:47.252542+0000","last_active":"2024-06-10T01:04:35.552389+0000","last_peered":"2024-06-10T01:04:35.552389+0000","last_clean":"2024-06-10T01:04:35.552389+0000","last_became_active":"2024-06-10T01:01:47.252216+0000","last_became_peered":"2024-06-10T01:01:47.252216+0000","last_unstale":"2024-06-10T01:04:35.552389+0000","last_undegraded":"2024-06-10T01:04:35.552389+0000","last_fullsized":"2024-06-10T01:04:35.552389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:12:22.196263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285691,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73985,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1122,"num_read_kb":640,"num_write":186,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":221955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383746,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":569843,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645383,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32392,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712760,"statfs":{"total":95995035648,"available":95961866240,"internally_reserved":0,"allocated":1245184,"data_stored":1227374,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808908,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":574063,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070546,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":569775,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332182,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":1165713,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495707,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":565143,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1163264,"data_stored":635120,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738489,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":566531,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1179648,"data_stored":636358,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182507,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":565337,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920869,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":570213,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757344,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":1160993,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34687,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34687,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34687,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":70067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T01:06:00.858 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-10T01:06:00.858 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-10T01:06:01.109 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:06:01.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:01 smithi007 ceph-mon[32398]: pgmap v431: 129 pgs: 129 active+clean; 649 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:06:01.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:01 smithi007 ceph-mon[32398]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T01:06:01.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:01 smithi078 ceph-mon[36683]: pgmap v431: 129 pgs: 129 active+clean; 649 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:06:01.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:01 smithi078 ceph-mon[36683]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T01:06:01.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:01 smithi175 ceph-mon[36636]: pgmap v431: 129 pgs: 129 active+clean; 649 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:06:01.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:01 smithi175 ceph-mon[36636]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T01:06:02.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.883+0000 7f8eea1f1700 1 -- 172.21.15.7:0/3341400079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ee4108500 msgr2=0x7f8ee41088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:02.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.883+0000 7f8eea1f1700 1 --2- 172.21.15.7:0/3341400079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ee4108500 0x7f8ee41088e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4009990 tx=0x7f8ed4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:06:02.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.883+0000 7f8eea1f1700 1 -- 172.21.15.7:0/3341400079 shutdown_connections 2024-06-10T01:06:02.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.883+0000 7f8eea1f1700 1 --2- 172.21.15.7:0/3341400079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ee406ab30 0x7f8ee410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:02.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.883+0000 7f8eea1f1700 1 --2- 172.21.15.7:0/3341400079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ee4108500 0x7f8ee41088e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:02.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.883+0000 7f8eea1f1700 1 --2- 172.21.15.7:0/3341400079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ee406a170 0x7f8ee406a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:02.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.883+0000 7f8eea1f1700 1 -- 172.21.15.7:0/3341400079 >> 172.21.15.7:0/3341400079 conn(0x7f8ee4078340 msgr2=0x7f8ee4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:02.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.883+0000 7f8eea1f1700 1 -- 172.21.15.7:0/3341400079 shutdown_connections 2024-06-10T01:06:02.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.883+0000 7f8eea1f1700 1 -- 172.21.15.7:0/3341400079 wait complete. 2024-06-10T01:06:02.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.884+0000 7f8eea1f1700 1 Processor -- start 2024-06-10T01:06:02.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.884+0000 7f8eea1f1700 1 -- start start 2024-06-10T01:06:02.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.884+0000 7f8eea1f1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ee406a170 0x7f8ee4101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:02.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.884+0000 7f8eea1f1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ee406ab30 0x7f8ee41019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:02.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.884+0000 7f8eea1f1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ee4108500 0x7f8ee4101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:02.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.884+0000 7f8eea1f1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee4112780 con 0x7f8ee406ab30 2024-06-10T01:06:02.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.884+0000 7f8eea1f1700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee4112600 con 0x7f8ee4108500 2024-06-10T01:06:02.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.884+0000 7f8eea1f1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee4112900 con 0x7f8ee406a170 2024-06-10T01:06:02.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.884+0000 7f8ee2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ee406ab30 0x7f8ee41019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:02.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.884+0000 7f8ee2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ee406ab30 0x7f8ee41019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52912/0 (socket says 172.21.15.7:52912) 2024-06-10T01:06:02.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.884+0000 7f8ee2ffd700 1 -- 172.21.15.7:0/1403076036 learned_addr learned my addr 172.21.15.7:0/1403076036 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:02.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.884+0000 7f8ee37fe700 1 --2- 172.21.15.7:0/1403076036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ee406a170 0x7f8ee4101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:02.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.885+0000 7f8ee3fff700 1 --2- 172.21.15.7:0/1403076036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ee4108500 0x7f8ee4101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:02.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.885+0000 7f8ee2ffd700 1 -- 172.21.15.7:0/1403076036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ee406a170 msgr2=0x7f8ee4101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:02.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.885+0000 7f8ee2ffd700 1 --2- 172.21.15.7:0/1403076036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ee406a170 0x7f8ee4101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:02.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.885+0000 7f8ee2ffd700 1 -- 172.21.15.7:0/1403076036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ee4108500 msgr2=0x7f8ee4101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:02.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.885+0000 7f8ee2ffd700 1 --2- 172.21.15.7:0/1403076036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ee4108500 0x7f8ee4101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:02.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.885+0000 7f8ee2ffd700 1 -- 172.21.15.7:0/1403076036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee41a7200 con 0x7f8ee406ab30 2024-06-10T01:06:02.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.885+0000 7f8ee2ffd700 1 --2- 172.21.15.7:0/1403076036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ee406ab30 0x7f8ee41019a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8ed800d980 tx=0x7f8ed800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:02.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.885+0000 7f8ee0ff9700 1 -- 172.21.15.7:0/1403076036 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8017070 con 0x7f8ee406ab30 2024-06-10T01:06:02.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.885+0000 7f8eea1f1700 1 -- 172.21.15.7:0/1403076036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ee41a74f0 con 0x7f8ee406ab30 2024-06-10T01:06:02.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.885+0000 7f8ee0ff9700 1 -- 172.21.15.7:0/1403076036 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ed8005020 con 0x7f8ee406ab30 2024-06-10T01:06:02.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.885+0000 7f8ee0ff9700 1 -- 172.21.15.7:0/1403076036 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed800fbf0 con 0x7f8ee406ab30 2024-06-10T01:06:02.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.885+0000 7f8eea1f1700 1 -- 172.21.15.7:0/1403076036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ee41a79e0 con 0x7f8ee406ab30 2024-06-10T01:06:02.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.886+0000 7f8eea1f1700 1 -- 172.21.15.7:0/1403076036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ee410a9e0 con 0x7f8ee406ab30 2024-06-10T01:06:02.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.887+0000 7f8ee0ff9700 1 -- 172.21.15.7:0/1403076036 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f8ed80051d0 con 0x7f8ee406ab30 2024-06-10T01:06:02.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.888+0000 7f8ee0ff9700 1 --2- 172.21.15.7:0/1403076036 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8ecc0a2c20 0x7f8ecc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:02.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.888+0000 7f8ee37fe700 1 --2- 172.21.15.7:0/1403076036 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8ecc0a2c20 0x7f8ecc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:02.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.888+0000 7f8ee0ff9700 1 -- 172.21.15.7:0/1403076036 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f8ed80cf1f0 con 0x7f8ee406ab30 2024-06-10T01:06:02.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.889+0000 7f8ee37fe700 1 --2- 172.21.15.7:0/1403076036 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8ecc0a2c20 0x7f8ecc0a50e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8ed40098e0 tx=0x7f8ed4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:02.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:02.893+0000 7f8ee0ff9700 1 -- 172.21.15.7:0/1403076036 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8ed8020020 con 0x7f8ee406ab30 2024-06-10T01:06:03.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:03.093+0000 7f8eea1f1700 1 -- 172.21.15.7:0/1403076036 --> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8ee4102990 con 0x7f8ecc0a2c20 2024-06-10T01:06:03.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:03.103+0000 7f8ee0ff9700 1 -- 172.21.15.7:0/1403076036 <== mgr.14152 v2:172.21.15.7:6800/1804801756 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340767 (secure 0 0 0) 0x7f8ee4102990 con 0x7f8ecc0a2c20 2024-06-10T01:06:03.104 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:06:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:03.106+0000 7f8eea1f1700 1 -- 172.21.15.7:0/1403076036 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8ecc0a2c20 msgr2=0x7f8ecc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:03.106+0000 7f8eea1f1700 1 --2- 172.21.15.7:0/1403076036 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8ecc0a2c20 0x7f8ecc0a50e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8ed40098e0 tx=0x7f8ed4005960 comp rx=0 tx=0).stop 2024-06-10T01:06:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:03.106+0000 7f8eea1f1700 1 -- 172.21.15.7:0/1403076036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ee406ab30 msgr2=0x7f8ee41019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:03.106+0000 7f8eea1f1700 1 --2- 172.21.15.7:0/1403076036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ee406ab30 0x7f8ee41019a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8ed800d980 tx=0x7f8ed800dc90 comp rx=0 tx=0).stop 2024-06-10T01:06:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:03.107+0000 7f8eea1f1700 1 -- 172.21.15.7:0/1403076036 shutdown_connections 2024-06-10T01:06:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:03.107+0000 7f8eea1f1700 1 --2- 172.21.15.7:0/1403076036 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8ecc0a2c20 0x7f8ecc0a50e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:03.107+0000 7f8eea1f1700 1 --2- 172.21.15.7:0/1403076036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ee4108500 0x7f8ee4101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:03.107+0000 7f8eea1f1700 1 --2- 172.21.15.7:0/1403076036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ee406a170 0x7f8ee4101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:03.107+0000 7f8eea1f1700 1 --2- 172.21.15.7:0/1403076036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ee406ab30 0x7f8ee41019a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:03.107+0000 7f8eea1f1700 1 -- 172.21.15.7:0/1403076036 >> 172.21.15.7:0/1403076036 conn(0x7f8ee4078340 msgr2=0x7f8ee40fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:03.107+0000 7f8eea1f1700 1 -- 172.21.15.7:0/1403076036 shutdown_connections 2024-06-10T01:06:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:03.107+0000 7f8eea1f1700 1 -- 172.21.15.7:0/1403076036 wait complete. 2024-06-10T01:06:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-06-10T01:06:03.444 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:03 smithi007 ceph-mon[32398]: pgmap v432: 129 pgs: 129 active+clean; 649 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:06:03.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:03 smithi078 ceph-mon[36683]: pgmap v432: 129 pgs: 129 active+clean; 649 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:06:03.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:03 smithi175 ceph-mon[36636]: pgmap v432: 129 pgs: 129 active+clean; 649 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:06:03.767 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-06-10T01:06:01.602139+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664587,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1263,"num_read_kb":759,"num_write":369,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":796,"ondisk_log_size":796,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362480,"kb_used_data":13328,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579344,"statfs":{"total":1151940427776,"available":1151569248256,"internally_reserved":0,"allocated":13647872,"data_stored":8806463,"data_compressed":496626,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1138,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":44,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003263"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.569548+0000","last_change":"2024-06-10T01:01:47.244728+0000","last_active":"2024-06-10T01:04:35.569548+0000","last_peered":"2024-06-10T01:04:35.569548+0000","last_clean":"2024-06-10T01:04:35.569548+0000","last_became_active":"2024-06-10T01:01:47.244198+0000","last_became_peered":"2024-06-10T01:01:47.244198+0000","last_unstale":"2024-06-10T01:04:35.569548+0000","last_undegraded":"2024-06-10T01:04:35.569548+0000","last_fullsized":"2024-06-10T01:04:35.569548+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:58:06.257952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.557753+0000","last_change":"2024-06-10T01:01:47.247620+0000","last_active":"2024-06-10T01:04:35.557753+0000","last_peered":"2024-06-10T01:04:35.557753+0000","last_clean":"2024-06-10T01:04:35.557753+0000","last_became_active":"2024-06-10T01:01:47.245475+0000","last_became_peered":"2024-06-10T01:01:47.245475+0000","last_unstale":"2024-06-10T01:04:35.557753+0000","last_undegraded":"2024-06-10T01:04:35.557753+0000","last_fullsized":"2024-06-10T01:04:35.557753+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:29:09.367411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.569730+0000","last_change":"2024-06-10T01:01:47.252414+0000","last_active":"2024-06-10T01:04:35.569730+0000","last_peered":"2024-06-10T01:04:35.569730+0000","last_clean":"2024-06-10T01:04:35.569730+0000","last_became_active":"2024-06-10T01:01:47.252063+0000","last_became_peered":"2024-06-10T01:01:47.252063+0000","last_unstale":"2024-06-10T01:04:35.569730+0000","last_undegraded":"2024-06-10T01:04:35.569730+0000","last_fullsized":"2024-06-10T01:04:35.569730+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:54:53.966209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.553817+0000","last_change":"2024-06-10T01:01:47.254875+0000","last_active":"2024-06-10T01:04:35.553817+0000","last_peered":"2024-06-10T01:04:35.553817+0000","last_clean":"2024-06-10T01:04:35.553817+0000","last_became_active":"2024-06-10T01:01:47.254542+0000","last_became_peered":"2024-06-10T01:01:47.254542+0000","last_unstale":"2024-06-10T01:04:35.553817+0000","last_undegraded":"2024-06-10T01:04:35.553817+0000","last_fullsized":"2024-06-10T01:04:35.553817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:02:28.393133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558663+0000","last_change":"2024-06-10T01:01:47.242596+0000","last_active":"2024-06-10T01:04:35.558663+0000","last_peered":"2024-06-10T01:04:35.558663+0000","last_clean":"2024-06-10T01:04:35.558663+0000","last_became_active":"2024-06-10T01:01:47.242308+0000","last_became_peered":"2024-06-10T01:01:47.242308+0000","last_unstale":"2024-06-10T01:04:35.558663+0000","last_undegraded":"2024-06-10T01:04:35.558663+0000","last_fullsized":"2024-06-10T01:04:35.558663+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:11:26.599954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.550600+0000","last_change":"2024-06-10T01:01:47.249085+0000","last_active":"2024-06-10T01:04:35.550600+0000","last_peered":"2024-06-10T01:04:35.550600+0000","last_clean":"2024-06-10T01:04:35.550600+0000","last_became_active":"2024-06-10T01:01:47.248749+0000","last_became_peered":"2024-06-10T01:01:47.248749+0000","last_unstale":"2024-06-10T01:04:35.550600+0000","last_undegraded":"2024-06-10T01:04:35.550600+0000","last_fullsized":"2024-06-10T01:04:35.550600+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:42:48.734566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562583+0000","last_change":"2024-06-10T01:01:47.247334+0000","last_active":"2024-06-10T01:04:35.562583+0000","last_peered":"2024-06-10T01:04:35.562583+0000","last_clean":"2024-06-10T01:04:35.562583+0000","last_became_active":"2024-06-10T01:01:47.246964+0000","last_became_peered":"2024-06-10T01:01:47.246964+0000","last_unstale":"2024-06-10T01:04:35.562583+0000","last_undegraded":"2024-06-10T01:04:35.562583+0000","last_fullsized":"2024-06-10T01:04:35.562583+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:05:36.034950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.563056+0000","last_change":"2024-06-10T01:01:47.242523+0000","last_active":"2024-06-10T01:04:35.563056+0000","last_peered":"2024-06-10T01:04:35.563056+0000","last_clean":"2024-06-10T01:04:35.563056+0000","last_became_active":"2024-06-10T01:01:47.242198+0000","last_became_peered":"2024-06-10T01:01:47.242198+0000","last_unstale":"2024-06-10T01:04:35.563056+0000","last_undegraded":"2024-06-10T01:04:35.563056+0000","last_fullsized":"2024-06-10T01:04:35.563056+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:39:07.055970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.567116+0000","last_change":"2024-06-10T01:01:47.245277+0000","last_active":"2024-06-10T01:04:35.567116+0000","last_peered":"2024-06-10T01:04:35.567116+0000","last_clean":"2024-06-10T01:04:35.567116+0000","last_became_active":"2024-06-10T01:01:47.244932+0000","last_became_peered":"2024-06-10T01:01:47.244932+0000","last_unstale":"2024-06-10T01:04:35.567116+0000","last_undegraded":"2024-06-10T01:04:35.567116+0000","last_fullsized":"2024-06-10T01:04:35.567116+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:56:24.423713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559784+0000","last_change":"2024-06-10T01:01:47.255531+0000","last_active":"2024-06-10T01:04:35.559784+0000","last_peered":"2024-06-10T01:04:35.559784+0000","last_clean":"2024-06-10T01:04:35.559784+0000","last_became_active":"2024-06-10T01:01:47.255150+0000","last_became_peered":"2024-06-10T01:01:47.255150+0000","last_unstale":"2024-06-10T01:04:35.559784+0000","last_undegraded":"2024-06-10T01:04:35.559784+0000","last_fullsized":"2024-06-10T01:04:35.559784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:51:34.561396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559164+0000","last_change":"2024-06-10T01:01:47.249832+0000","last_active":"2024-06-10T01:04:35.559164+0000","last_peered":"2024-06-10T01:04:35.559164+0000","last_clean":"2024-06-10T01:04:35.559164+0000","last_became_active":"2024-06-10T01:01:47.249303+0000","last_became_peered":"2024-06-10T01:01:47.249303+0000","last_unstale":"2024-06-10T01:04:35.559164+0000","last_undegraded":"2024-06-10T01:04:35.559164+0000","last_fullsized":"2024-06-10T01:04:35.559164+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:15:21.930157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.551184+0000","last_change":"2024-06-10T01:01:47.255927+0000","last_active":"2024-06-10T01:04:35.551184+0000","last_peered":"2024-06-10T01:04:35.551184+0000","last_clean":"2024-06-10T01:04:35.551184+0000","last_became_active":"2024-06-10T01:01:47.255591+0000","last_became_peered":"2024-06-10T01:01:47.255591+0000","last_unstale":"2024-06-10T01:04:35.551184+0000","last_undegraded":"2024-06-10T01:04:35.551184+0000","last_fullsized":"2024-06-10T01:04:35.551184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:48:06.494430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.557880+0000","last_change":"2024-06-10T01:02:37.372069+0000","last_active":"2024-06-10T01:04:35.557880+0000","last_peered":"2024-06-10T01:04:35.557880+0000","last_clean":"2024-06-10T01:04:35.557880+0000","last_became_active":"2024-06-10T01:02:37.371599+0000","last_became_peered":"2024-06-10T01:02:37.371599+0000","last_unstale":"2024-06-10T01:04:35.557880+0000","last_undegraded":"2024-06-10T01:04:35.557880+0000","last_fullsized":"2024-06-10T01:04:35.557880+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:54:34.091576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.551120+0000","last_change":"2024-06-10T01:01:47.249553+0000","last_active":"2024-06-10T01:04:35.551120+0000","last_peered":"2024-06-10T01:04:35.551120+0000","last_clean":"2024-06-10T01:04:35.551120+0000","last_became_active":"2024-06-10T01:01:47.249202+0000","last_became_peered":"2024-06-10T01:01:47.249202+0000","last_unstale":"2024-06-10T01:04:35.551120+0000","last_undegraded":"2024-06-10T01:04:35.551120+0000","last_fullsized":"2024-06-10T01:04:35.551120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:23:57.882264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.564289+0000","last_change":"2024-06-10T01:01:47.253987+0000","last_active":"2024-06-10T01:04:35.564289+0000","last_peered":"2024-06-10T01:04:35.564289+0000","last_clean":"2024-06-10T01:04:35.564289+0000","last_became_active":"2024-06-10T01:01:47.253578+0000","last_became_peered":"2024-06-10T01:01:47.253578+0000","last_unstale":"2024-06-10T01:04:35.564289+0000","last_undegraded":"2024-06-10T01:04:35.564289+0000","last_fullsized":"2024-06-10T01:04:35.564289+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:35:30.585482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.567601+0000","last_change":"2024-06-10T01:01:47.247089+0000","last_active":"2024-06-10T01:04:35.567601+0000","last_peered":"2024-06-10T01:04:35.567601+0000","last_clean":"2024-06-10T01:04:35.567601+0000","last_became_active":"2024-06-10T01:01:47.246066+0000","last_became_peered":"2024-06-10T01:01:47.246066+0000","last_unstale":"2024-06-10T01:04:35.567601+0000","last_undegraded":"2024-06-10T01:04:35.567601+0000","last_fullsized":"2024-06-10T01:04:35.567601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:58:12.092981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.569804+0000","last_change":"2024-06-10T01:01:47.244491+0000","last_active":"2024-06-10T01:04:35.569804+0000","last_peered":"2024-06-10T01:04:35.569804+0000","last_clean":"2024-06-10T01:04:35.569804+0000","last_became_active":"2024-06-10T01:01:47.243747+0000","last_became_peered":"2024-06-10T01:01:47.243747+0000","last_unstale":"2024-06-10T01:04:35.569804+0000","last_undegraded":"2024-06-10T01:04:35.569804+0000","last_fullsized":"2024-06-10T01:04:35.569804+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:55:56.785016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.567988+0000","last_change":"2024-06-10T01:01:47.243141+0000","last_active":"2024-06-10T01:04:35.567988+0000","last_peered":"2024-06-10T01:04:35.567988+0000","last_clean":"2024-06-10T01:04:35.567988+0000","last_became_active":"2024-06-10T01:01:47.242464+0000","last_became_peered":"2024-06-10T01:01:47.242464+0000","last_unstale":"2024-06-10T01:04:35.567988+0000","last_undegraded":"2024-06-10T01:04:35.567988+0000","last_fullsized":"2024-06-10T01:04:35.567988+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:00:49.777130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559919+0000","last_change":"2024-06-10T01:01:47.253463+0000","last_active":"2024-06-10T01:04:35.559919+0000","last_peered":"2024-06-10T01:04:35.559919+0000","last_clean":"2024-06-10T01:04:35.559919+0000","last_became_active":"2024-06-10T01:01:47.252929+0000","last_became_peered":"2024-06-10T01:01:47.252929+0000","last_unstale":"2024-06-10T01:04:35.559919+0000","last_undegraded":"2024-06-10T01:04:35.559919+0000","last_fullsized":"2024-06-10T01:04:35.559919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:18:49.551276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239453+0000","last_change":"2024-06-10T01:01:57.279561+0000","last_active":"2024-06-10T01:03:38.239453+0000","last_peered":"2024-06-10T01:03:38.239453+0000","last_clean":"2024-06-10T01:03:38.239453+0000","last_became_active":"2024-06-10T01:01:57.279132+0000","last_became_peered":"2024-06-10T01:01:57.279132+0000","last_unstale":"2024-06-10T01:03:38.239453+0000","last_undegraded":"2024-06-10T01:03:38.239453+0000","last_fullsized":"2024-06-10T01:03:38.239453+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:55:03.497701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'9","reported_seq":57,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:54.300918+0000","last_change":"2024-06-10T01:01:44.006667+0000","last_active":"2024-06-10T01:05:54.300918+0000","last_peered":"2024-06-10T01:05:54.300918+0000","last_clean":"2024-06-10T01:05:54.300918+0000","last_became_active":"2024-06-10T01:01:44.006343+0000","last_became_peered":"2024-06-10T01:01:44.006343+0000","last_unstale":"2024-06-10T01:05:54.300918+0000","last_undegraded":"2024-06-10T01:05:54.300918+0000","last_fullsized":"2024-06-10T01:05:54.300918+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:14:50.540084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562931+0000","last_change":"2024-06-10T01:01:47.246139+0000","last_active":"2024-06-10T01:04:35.562931+0000","last_peered":"2024-06-10T01:04:35.562931+0000","last_clean":"2024-06-10T01:04:35.562931+0000","last_became_active":"2024-06-10T01:01:47.245743+0000","last_became_peered":"2024-06-10T01:01:47.245743+0000","last_unstale":"2024-06-10T01:04:35.562931+0000","last_undegraded":"2024-06-10T01:04:35.562931+0000","last_fullsized":"2024-06-10T01:04:35.562931+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:06:39.041569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239398+0000","last_change":"2024-06-10T01:01:57.282843+0000","last_active":"2024-06-10T01:03:38.239398+0000","last_peered":"2024-06-10T01:03:38.239398+0000","last_clean":"2024-06-10T01:03:38.239398+0000","last_became_active":"2024-06-10T01:01:57.281797+0000","last_became_peered":"2024-06-10T01:01:57.281797+0000","last_unstale":"2024-06-10T01:03:38.239398+0000","last_undegraded":"2024-06-10T01:03:38.239398+0000","last_fullsized":"2024-06-10T01:03:38.239398+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:00.209628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:43.959967+0000","last_change":"2024-06-10T01:01:44.466714+0000","last_active":"2024-06-10T01:03:43.959967+0000","last_peered":"2024-06-10T01:03:43.959967+0000","last_clean":"2024-06-10T01:03:43.959967+0000","last_became_active":"2024-06-10T01:01:44.466296+0000","last_became_peered":"2024-06-10T01:01:44.466296+0000","last_unstale":"2024-06-10T01:03:43.959967+0000","last_undegraded":"2024-06-10T01:03:43.959967+0000","last_fullsized":"2024-06-10T01:03:43.959967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:16:36.082308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.569882+0000","last_change":"2024-06-10T01:01:47.244173+0000","last_active":"2024-06-10T01:04:35.569882+0000","last_peered":"2024-06-10T01:04:35.569882+0000","last_clean":"2024-06-10T01:04:35.569882+0000","last_became_active":"2024-06-10T01:01:47.243032+0000","last_became_peered":"2024-06-10T01:01:47.243032+0000","last_unstale":"2024-06-10T01:04:35.569882+0000","last_undegraded":"2024-06-10T01:04:35.569882+0000","last_fullsized":"2024-06-10T01:04:35.569882+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:14:33.965903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239057+0000","last_change":"2024-06-10T01:02:37.592535+0000","last_active":"2024-06-10T01:03:38.239057+0000","last_peered":"2024-06-10T01:03:38.239057+0000","last_clean":"2024-06-10T01:03:38.239057+0000","last_became_active":"2024-06-10T01:02:37.592159+0000","last_became_peered":"2024-06-10T01:02:37.592159+0000","last_unstale":"2024-06-10T01:03:38.239057+0000","last_undegraded":"2024-06-10T01:03:38.239057+0000","last_fullsized":"2024-06-10T01:03:38.239057+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:50:08.626485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235433+0000","last_change":"2024-06-10T01:01:44.009105+0000","last_active":"2024-06-10T01:03:38.235433+0000","last_peered":"2024-06-10T01:03:38.235433+0000","last_clean":"2024-06-10T01:03:38.235433+0000","last_became_active":"2024-06-10T01:01:44.008743+0000","last_became_peered":"2024-06-10T01:01:44.008743+0000","last_unstale":"2024-06-10T01:03:38.235433+0000","last_undegraded":"2024-06-10T01:03:38.235433+0000","last_fullsized":"2024-06-10T01:03:38.235433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:44:54.301378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.570239+0000","last_change":"2024-06-10T01:01:47.249007+0000","last_active":"2024-06-10T01:04:35.570239+0000","last_peered":"2024-06-10T01:04:35.570239+0000","last_clean":"2024-06-10T01:04:35.570239+0000","last_became_active":"2024-06-10T01:01:47.245632+0000","last_became_peered":"2024-06-10T01:01:47.245632+0000","last_unstale":"2024-06-10T01:04:35.570239+0000","last_undegraded":"2024-06-10T01:04:35.570239+0000","last_fullsized":"2024-06-10T01:04:35.570239+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:29:18.916390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.238983+0000","last_change":"2024-06-10T01:01:57.283137+0000","last_active":"2024-06-10T01:03:38.238983+0000","last_peered":"2024-06-10T01:03:38.238983+0000","last_clean":"2024-06-10T01:03:38.238983+0000","last_became_active":"2024-06-10T01:01:57.282605+0000","last_became_peered":"2024-06-10T01:01:57.282605+0000","last_unstale":"2024-06-10T01:03:38.238983+0000","last_undegraded":"2024-06-10T01:03:38.238983+0000","last_fullsized":"2024-06-10T01:03:38.238983+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:26:16.967320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235734+0000","last_change":"2024-06-10T01:01:44.010674+0000","last_active":"2024-06-10T01:03:38.235734+0000","last_peered":"2024-06-10T01:03:38.235734+0000","last_clean":"2024-06-10T01:03:38.235734+0000","last_became_active":"2024-06-10T01:01:44.010291+0000","last_became_peered":"2024-06-10T01:01:44.010291+0000","last_unstale":"2024-06-10T01:03:38.235734+0000","last_undegraded":"2024-06-10T01:03:38.235734+0000","last_fullsized":"2024-06-10T01:03:38.235734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:36:18.418362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":60,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.566078+0000","last_change":"2024-06-10T01:02:37.794580+0000","last_active":"2024-06-10T01:04:35.566078+0000","last_peered":"2024-06-10T01:04:35.566078+0000","last_clean":"2024-06-10T01:04:35.566078+0000","last_became_active":"2024-06-10T01:02:37.794127+0000","last_became_peered":"2024-06-10T01:02:37.794127+0000","last_unstale":"2024-06-10T01:04:35.566078+0000","last_undegraded":"2024-06-10T01:04:35.566078+0000","last_fullsized":"2024-06-10T01:04:35.566078+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:02.117871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239746+0000","last_change":"2024-06-10T01:01:57.282494+0000","last_active":"2024-06-10T01:03:38.239746+0000","last_peered":"2024-06-10T01:03:38.239746+0000","last_clean":"2024-06-10T01:03:38.239746+0000","last_became_active":"2024-06-10T01:01:57.282077+0000","last_became_peered":"2024-06-10T01:01:57.282077+0000","last_unstale":"2024-06-10T01:03:38.239746+0000","last_undegraded":"2024-06-10T01:03:38.239746+0000","last_fullsized":"2024-06-10T01:03:38.239746+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:57:41.639788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"97'5","reported_seq":234,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:55.973995+0000","last_change":"2024-06-10T01:01:44.011134+0000","last_active":"2024-06-10T01:05:55.973995+0000","last_peered":"2024-06-10T01:05:55.973995+0000","last_clean":"2024-06-10T01:05:55.973995+0000","last_became_active":"2024-06-10T01:01:44.010788+0000","last_became_peered":"2024-06-10T01:01:44.010788+0000","last_unstale":"2024-06-10T01:05:55.973995+0000","last_undegraded":"2024-06-10T01:05:55.973995+0000","last_fullsized":"2024-06-10T01:05:55.973995+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:22:55.228946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":184,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.564421+0000","last_change":"2024-06-10T01:01:47.248093+0000","last_active":"2024-06-10T01:04:35.564421+0000","last_peered":"2024-06-10T01:04:35.564421+0000","last_clean":"2024-06-10T01:04:35.564421+0000","last_became_active":"2024-06-10T01:01:47.247797+0000","last_became_peered":"2024-06-10T01:01:47.247797+0000","last_unstale":"2024-06-10T01:04:35.564421+0000","last_undegraded":"2024-06-10T01:04:35.564421+0000","last_fullsized":"2024-06-10T01:04:35.564421+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:45:50.159128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239671+0000","last_change":"2024-06-10T01:02:37.377039+0000","last_active":"2024-06-10T01:03:38.239671+0000","last_peered":"2024-06-10T01:03:38.239671+0000","last_clean":"2024-06-10T01:03:38.239671+0000","last_became_active":"2024-06-10T01:02:37.376332+0000","last_became_peered":"2024-06-10T01:02:37.376332+0000","last_unstale":"2024-06-10T01:03:38.239671+0000","last_undegraded":"2024-06-10T01:03:38.239671+0000","last_fullsized":"2024-06-10T01:03:38.239671+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:49:15.000505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,10,5],"acting":[2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.570465+0000","last_change":"2024-06-10T01:01:44.008342+0000","last_active":"2024-06-10T01:03:38.570465+0000","last_peered":"2024-06-10T01:03:38.570465+0000","last_clean":"2024-06-10T01:03:38.570465+0000","last_became_active":"2024-06-10T01:01:44.007959+0000","last_became_peered":"2024-06-10T01:01:44.007959+0000","last_unstale":"2024-06-10T01:03:38.570465+0000","last_undegraded":"2024-06-10T01:03:38.570465+0000","last_fullsized":"2024-06-10T01:03:38.570465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:36:03.143609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.565044+0000","last_change":"2024-06-10T01:01:47.247854+0000","last_active":"2024-06-10T01:04:35.565044+0000","last_peered":"2024-06-10T01:04:35.565044+0000","last_clean":"2024-06-10T01:04:35.565044+0000","last_became_active":"2024-06-10T01:01:47.247563+0000","last_became_peered":"2024-06-10T01:01:47.247563+0000","last_unstale":"2024-06-10T01:04:35.565044+0000","last_undegraded":"2024-06-10T01:04:35.565044+0000","last_fullsized":"2024-06-10T01:04:35.565044+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:32:13.024815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.237041+0000","last_change":"2024-06-10T01:01:57.282427+0000","last_active":"2024-06-10T01:03:38.237041+0000","last_peered":"2024-06-10T01:03:38.237041+0000","last_clean":"2024-06-10T01:03:38.237041+0000","last_became_active":"2024-06-10T01:01:57.281987+0000","last_became_peered":"2024-06-10T01:01:57.281987+0000","last_unstale":"2024-06-10T01:03:38.237041+0000","last_undegraded":"2024-06-10T01:03:38.237041+0000","last_fullsized":"2024-06-10T01:03:38.237041+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:22:31.736804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770902+0000","last_change":"2024-06-10T01:01:44.006567+0000","last_active":"2024-06-10T01:03:38.770902+0000","last_peered":"2024-06-10T01:03:38.770902+0000","last_clean":"2024-06-10T01:03:38.770902+0000","last_became_active":"2024-06-10T01:01:44.006216+0000","last_became_peered":"2024-06-10T01:01:44.006216+0000","last_unstale":"2024-06-10T01:03:38.770902+0000","last_undegraded":"2024-06-10T01:03:38.770902+0000","last_fullsized":"2024-06-10T01:03:38.770902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:33:55.696873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558008+0000","last_change":"2024-06-10T01:01:47.245326+0000","last_active":"2024-06-10T01:04:35.558008+0000","last_peered":"2024-06-10T01:04:35.558008+0000","last_clean":"2024-06-10T01:04:35.558008+0000","last_became_active":"2024-06-10T01:01:47.245013+0000","last_became_peered":"2024-06-10T01:01:47.245013+0000","last_unstale":"2024-06-10T01:04:35.558008+0000","last_undegraded":"2024-06-10T01:04:35.558008+0000","last_fullsized":"2024-06-10T01:04:35.558008+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:17:56.416627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:56.631218+0000","last_change":"2024-06-10T01:01:57.288997+0000","last_active":"2024-06-10T01:03:56.631218+0000","last_peered":"2024-06-10T01:03:56.631218+0000","last_clean":"2024-06-10T01:03:56.631218+0000","last_became_active":"2024-06-10T01:01:57.288596+0000","last_became_peered":"2024-06-10T01:01:57.288596+0000","last_unstale":"2024-06-10T01:03:56.631218+0000","last_undegraded":"2024-06-10T01:03:56.631218+0000","last_fullsized":"2024-06-10T01:03:56.631218+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:08:14.739737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.269738+0000","last_change":"2024-06-10T01:01:44.010079+0000","last_active":"2024-06-10T01:03:38.269738+0000","last_peered":"2024-06-10T01:03:38.269738+0000","last_clean":"2024-06-10T01:03:38.269738+0000","last_became_active":"2024-06-10T01:01:44.009752+0000","last_became_peered":"2024-06-10T01:01:44.009752+0000","last_unstale":"2024-06-10T01:03:38.269738+0000","last_undegraded":"2024-06-10T01:03:38.269738+0000","last_fullsized":"2024-06-10T01:03:38.269738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:29:35.660935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.554398+0000","last_change":"2024-06-10T01:01:47.245015+0000","last_active":"2024-06-10T01:04:35.554398+0000","last_peered":"2024-06-10T01:04:35.554398+0000","last_clean":"2024-06-10T01:04:35.554398+0000","last_became_active":"2024-06-10T01:01:47.244595+0000","last_became_peered":"2024-06-10T01:01:47.244595+0000","last_unstale":"2024-06-10T01:04:35.554398+0000","last_undegraded":"2024-06-10T01:04:35.554398+0000","last_fullsized":"2024-06-10T01:04:35.554398+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:01:48.446526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.238926+0000","last_change":"2024-06-10T01:01:57.281656+0000","last_active":"2024-06-10T01:03:38.238926+0000","last_peered":"2024-06-10T01:03:38.238926+0000","last_clean":"2024-06-10T01:03:38.238926+0000","last_became_active":"2024-06-10T01:01:57.281295+0000","last_became_peered":"2024-06-10T01:01:57.281295+0000","last_unstale":"2024-06-10T01:03:38.238926+0000","last_undegraded":"2024-06-10T01:03:38.238926+0000","last_fullsized":"2024-06-10T01:03:38.238926+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:04:41.619864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"97'3","reported_seq":233,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:58.978576+0000","last_change":"2024-06-10T01:01:44.011941+0000","last_active":"2024-06-10T01:05:58.978576+0000","last_peered":"2024-06-10T01:05:58.978576+0000","last_clean":"2024-06-10T01:05:58.978576+0000","last_became_active":"2024-06-10T01:01:44.010156+0000","last_became_peered":"2024-06-10T01:01:44.010156+0000","last_unstale":"2024-06-10T01:05:58.978576+0000","last_undegraded":"2024-06-10T01:05:58.978576+0000","last_fullsized":"2024-06-10T01:05:58.978576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:16:15.709544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":185,"num_read_kb":185,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.570257+0000","last_change":"2024-06-10T01:01:47.241720+0000","last_active":"2024-06-10T01:04:35.570257+0000","last_peered":"2024-06-10T01:04:35.570257+0000","last_clean":"2024-06-10T01:04:35.570257+0000","last_became_active":"2024-06-10T01:01:47.241386+0000","last_became_peered":"2024-06-10T01:01:47.241386+0000","last_unstale":"2024-06-10T01:04:35.570257+0000","last_undegraded":"2024-06-10T01:04:35.570257+0000","last_fullsized":"2024-06-10T01:04:35.570257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:42:01.304787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770709+0000","last_change":"2024-06-10T01:01:57.283226+0000","last_active":"2024-06-10T01:03:38.770709+0000","last_peered":"2024-06-10T01:03:38.770709+0000","last_clean":"2024-06-10T01:03:38.770709+0000","last_became_active":"2024-06-10T01:01:57.282720+0000","last_became_peered":"2024-06-10T01:01:57.282720+0000","last_unstale":"2024-06-10T01:03:38.770709+0000","last_undegraded":"2024-06-10T01:03:38.770709+0000","last_fullsized":"2024-06-10T01:03:38.770709+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:35:31.342042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235489+0000","last_change":"2024-06-10T01:01:44.009278+0000","last_active":"2024-06-10T01:03:38.235489+0000","last_peered":"2024-06-10T01:03:38.235489+0000","last_clean":"2024-06-10T01:03:38.235489+0000","last_became_active":"2024-06-10T01:01:44.008841+0000","last_became_peered":"2024-06-10T01:01:44.008841+0000","last_unstale":"2024-06-10T01:03:38.235489+0000","last_undegraded":"2024-06-10T01:03:38.235489+0000","last_fullsized":"2024-06-10T01:03:38.235489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:30:58.392080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.557141+0000","last_change":"2024-06-10T01:01:47.256451+0000","last_active":"2024-06-10T01:04:35.557141+0000","last_peered":"2024-06-10T01:04:35.557141+0000","last_clean":"2024-06-10T01:04:35.557141+0000","last_became_active":"2024-06-10T01:01:47.256036+0000","last_became_peered":"2024-06-10T01:01:47.256036+0000","last_unstale":"2024-06-10T01:04:35.557141+0000","last_undegraded":"2024-06-10T01:04:35.557141+0000","last_fullsized":"2024-06-10T01:04:35.557141+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:57:26.640109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239192+0000","last_change":"2024-06-10T01:02:37.794160+0000","last_active":"2024-06-10T01:03:38.239192+0000","last_peered":"2024-06-10T01:03:38.239192+0000","last_clean":"2024-06-10T01:03:38.239192+0000","last_became_active":"2024-06-10T01:02:37.793633+0000","last_became_peered":"2024-06-10T01:02:37.793633+0000","last_unstale":"2024-06-10T01:03:38.239192+0000","last_undegraded":"2024-06-10T01:03:38.239192+0000","last_fullsized":"2024-06-10T01:03:38.239192+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:48:48.083620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270125+0000","last_change":"2024-06-10T01:01:44.469179+0000","last_active":"2024-06-10T01:03:38.270125+0000","last_peered":"2024-06-10T01:03:38.270125+0000","last_clean":"2024-06-10T01:03:38.270125+0000","last_became_active":"2024-06-10T01:01:44.468669+0000","last_became_peered":"2024-06-10T01:01:44.468669+0000","last_unstale":"2024-06-10T01:03:38.270125+0000","last_undegraded":"2024-06-10T01:03:38.270125+0000","last_fullsized":"2024-06-10T01:03:38.270125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:07:51.629030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.570053+0000","last_change":"2024-06-10T01:01:47.247588+0000","last_active":"2024-06-10T01:04:35.570053+0000","last_peered":"2024-06-10T01:04:35.570053+0000","last_clean":"2024-06-10T01:04:35.570053+0000","last_became_active":"2024-06-10T01:01:47.245750+0000","last_became_peered":"2024-06-10T01:01:47.245750+0000","last_unstale":"2024-06-10T01:04:35.570053+0000","last_undegraded":"2024-06-10T01:04:35.570053+0000","last_fullsized":"2024-06-10T01:04:35.570053+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:12:10.513777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771292+0000","last_change":"2024-06-10T01:01:57.283042+0000","last_active":"2024-06-10T01:03:38.771292+0000","last_peered":"2024-06-10T01:03:38.771292+0000","last_clean":"2024-06-10T01:03:38.771292+0000","last_became_active":"2024-06-10T01:01:57.282600+0000","last_became_peered":"2024-06-10T01:01:57.282600+0000","last_unstale":"2024-06-10T01:03:38.771292+0000","last_undegraded":"2024-06-10T01:03:38.771292+0000","last_fullsized":"2024-06-10T01:03:38.771292+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:06:36.969968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"99'42","reported_seq":471,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:59.979980+0000","last_change":"2024-06-10T01:01:44.011783+0000","last_active":"2024-06-10T01:05:59.979980+0000","last_peered":"2024-06-10T01:05:59.979980+0000","last_clean":"2024-06-10T01:05:59.979980+0000","last_became_active":"2024-06-10T01:01:44.011164+0000","last_became_peered":"2024-06-10T01:01:44.011164+0000","last_unstale":"2024-06-10T01:05:59.979980+0000","last_undegraded":"2024-06-10T01:05:59.979980+0000","last_fullsized":"2024-06-10T01:05:59.979980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:16:13.642714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70067,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":384,"num_read_kb":262,"num_write":43,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562407+0000","last_change":"2024-06-10T01:01:47.250751+0000","last_active":"2024-06-10T01:04:35.562407+0000","last_peered":"2024-06-10T01:04:35.562407+0000","last_clean":"2024-06-10T01:04:35.562407+0000","last_became_active":"2024-06-10T01:01:47.250451+0000","last_became_peered":"2024-06-10T01:01:47.250451+0000","last_unstale":"2024-06-10T01:04:35.562407+0000","last_undegraded":"2024-06-10T01:04:35.562407+0000","last_fullsized":"2024-06-10T01:04:35.562407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:22:10.121363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270382+0000","last_change":"2024-06-10T01:01:57.289687+0000","last_active":"2024-06-10T01:03:38.270382+0000","last_peered":"2024-06-10T01:03:38.270382+0000","last_clean":"2024-06-10T01:03:38.270382+0000","last_became_active":"2024-06-10T01:01:57.289274+0000","last_became_peered":"2024-06-10T01:01:57.289274+0000","last_unstale":"2024-06-10T01:03:38.270382+0000","last_undegraded":"2024-06-10T01:03:38.270382+0000","last_fullsized":"2024-06-10T01:03:38.270382+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:34:46.873974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'6","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239529+0000","last_change":"2024-06-10T01:01:44.010159+0000","last_active":"2024-06-10T01:03:38.239529+0000","last_peered":"2024-06-10T01:03:38.239529+0000","last_clean":"2024-06-10T01:03:38.239529+0000","last_became_active":"2024-06-10T01:01:44.009744+0000","last_became_peered":"2024-06-10T01:01:44.009744+0000","last_unstale":"2024-06-10T01:03:38.239529+0000","last_undegraded":"2024-06-10T01:03:38.239529+0000","last_fullsized":"2024-06-10T01:03:38.239529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:30.507613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558994+0000","last_change":"2024-06-10T01:01:47.246662+0000","last_active":"2024-06-10T01:04:35.558994+0000","last_peered":"2024-06-10T01:04:35.558994+0000","last_clean":"2024-06-10T01:04:35.558994+0000","last_became_active":"2024-06-10T01:01:47.246268+0000","last_became_peered":"2024-06-10T01:01:47.246268+0000","last_unstale":"2024-06-10T01:04:35.558994+0000","last_undegraded":"2024-06-10T01:04:35.558994+0000","last_fullsized":"2024-06-10T01:04:35.558994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:08:38.089054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.236925+0000","last_change":"2024-06-10T01:01:57.289937+0000","last_active":"2024-06-10T01:03:38.236925+0000","last_peered":"2024-06-10T01:03:38.236925+0000","last_clean":"2024-06-10T01:03:38.236925+0000","last_became_active":"2024-06-10T01:01:57.289407+0000","last_became_peered":"2024-06-10T01:01:57.289407+0000","last_unstale":"2024-06-10T01:03:38.236925+0000","last_undegraded":"2024-06-10T01:03:38.236925+0000","last_fullsized":"2024-06-10T01:03:38.236925+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:01:48.546762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270079+0000","last_change":"2024-06-10T01:01:44.012654+0000","last_active":"2024-06-10T01:03:38.270079+0000","last_peered":"2024-06-10T01:03:38.270079+0000","last_clean":"2024-06-10T01:03:38.270079+0000","last_became_active":"2024-06-10T01:01:44.012246+0000","last_became_peered":"2024-06-10T01:01:44.012246+0000","last_unstale":"2024-06-10T01:03:38.270079+0000","last_undegraded":"2024-06-10T01:03:38.270079+0000","last_fullsized":"2024-06-10T01:03:38.270079+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:31:36.173554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558783+0000","last_change":"2024-06-10T01:01:47.250514+0000","last_active":"2024-06-10T01:04:35.558783+0000","last_peered":"2024-06-10T01:04:35.558783+0000","last_clean":"2024-06-10T01:04:35.558783+0000","last_became_active":"2024-06-10T01:01:47.249050+0000","last_became_peered":"2024-06-10T01:01:47.249050+0000","last_unstale":"2024-06-10T01:04:35.558783+0000","last_undegraded":"2024-06-10T01:04:35.558783+0000","last_fullsized":"2024-06-10T01:04:35.558783+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:51:30.789392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771928+0000","last_change":"2024-06-10T01:01:57.282389+0000","last_active":"2024-06-10T01:03:38.771928+0000","last_peered":"2024-06-10T01:03:38.771928+0000","last_clean":"2024-06-10T01:03:38.771928+0000","last_became_active":"2024-06-10T01:01:57.281857+0000","last_became_peered":"2024-06-10T01:01:57.281857+0000","last_unstale":"2024-06-10T01:03:38.771928+0000","last_undegraded":"2024-06-10T01:03:38.771928+0000","last_fullsized":"2024-06-10T01:03:38.771928+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:59:14.863020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"99'4","reported_seq":235,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:56.977143+0000","last_change":"2024-06-10T01:01:44.011331+0000","last_active":"2024-06-10T01:05:56.977143+0000","last_peered":"2024-06-10T01:05:56.977143+0000","last_clean":"2024-06-10T01:05:56.977143+0000","last_became_active":"2024-06-10T01:01:44.010968+0000","last_became_peered":"2024-06-10T01:01:44.010968+0000","last_unstale":"2024-06-10T01:05:56.977143+0000","last_undegraded":"2024-06-10T01:05:56.977143+0000","last_fullsized":"2024-06-10T01:05:56.977143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:43:37.931225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":896,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":187,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562879+0000","last_change":"2024-06-10T01:01:47.248349+0000","last_active":"2024-06-10T01:04:35.562879+0000","last_peered":"2024-06-10T01:04:35.562879+0000","last_clean":"2024-06-10T01:04:35.562879+0000","last_became_active":"2024-06-10T01:01:47.248008+0000","last_became_peered":"2024-06-10T01:01:47.248008+0000","last_unstale":"2024-06-10T01:04:35.562879+0000","last_undegraded":"2024-06-10T01:04:35.562879+0000","last_fullsized":"2024-06-10T01:04:35.562879+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:45:20.747704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771221+0000","last_change":"2024-06-10T01:01:57.289773+0000","last_active":"2024-06-10T01:03:38.771221+0000","last_peered":"2024-06-10T01:03:38.771221+0000","last_clean":"2024-06-10T01:03:38.771221+0000","last_became_active":"2024-06-10T01:01:57.289371+0000","last_became_peered":"2024-06-10T01:01:57.289371+0000","last_unstale":"2024-06-10T01:03:38.771221+0000","last_undegraded":"2024-06-10T01:03:38.771221+0000","last_fullsized":"2024-06-10T01:03:38.771221+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:24:52.209402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'15","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:40.912752+0000","last_change":"2024-06-10T01:03:38.257122+0000","last_active":"2024-06-10T01:03:40.912752+0000","last_peered":"2024-06-10T01:03:40.912752+0000","last_clean":"2024-06-10T01:03:40.912752+0000","last_became_active":"2024-06-10T01:03:38.240737+0000","last_became_peered":"2024-06-10T01:03:38.240737+0000","last_unstale":"2024-06-10T01:03:40.912752+0000","last_undegraded":"2024-06-10T01:03:40.912752+0000","last_fullsized":"2024-06-10T01:03:40.912752+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:37:10.925004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562425+0000","last_change":"2024-06-10T01:01:47.245439+0000","last_active":"2024-06-10T01:04:35.562425+0000","last_peered":"2024-06-10T01:04:35.562425+0000","last_clean":"2024-06-10T01:04:35.562425+0000","last_became_active":"2024-06-10T01:01:47.242917+0000","last_became_peered":"2024-06-10T01:01:47.242917+0000","last_unstale":"2024-06-10T01:04:35.562425+0000","last_undegraded":"2024-06-10T01:04:35.562425+0000","last_fullsized":"2024-06-10T01:04:35.562425+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:13:12.614230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.269795+0000","last_change":"2024-06-10T01:01:57.284731+0000","last_active":"2024-06-10T01:03:38.269795+0000","last_peered":"2024-06-10T01:03:38.269795+0000","last_clean":"2024-06-10T01:03:38.269795+0000","last_became_active":"2024-06-10T01:01:57.284301+0000","last_became_peered":"2024-06-10T01:01:57.284301+0000","last_unstale":"2024-06-10T01:03:38.269795+0000","last_undegraded":"2024-06-10T01:03:38.269795+0000","last_fullsized":"2024-06-10T01:03:38.269795+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:01:36.518214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"99'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:59.314831+0000","last_change":"2024-06-10T01:01:44.010154+0000","last_active":"2024-06-10T01:03:59.314831+0000","last_peered":"2024-06-10T01:03:59.314831+0000","last_clean":"2024-06-10T01:03:59.314831+0000","last_became_active":"2024-06-10T01:01:44.009656+0000","last_became_peered":"2024-06-10T01:01:44.009656+0000","last_unstale":"2024-06-10T01:03:59.314831+0000","last_undegraded":"2024-06-10T01:03:59.314831+0000","last_fullsized":"2024-06-10T01:03:59.314831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:33:08.789946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.550140+0000","last_change":"2024-06-10T01:02:37.378484+0000","last_active":"2024-06-10T01:04:35.550140+0000","last_peered":"2024-06-10T01:04:35.550140+0000","last_clean":"2024-06-10T01:04:35.550140+0000","last_became_active":"2024-06-10T01:02:37.378092+0000","last_became_peered":"2024-06-10T01:02:37.378092+0000","last_unstale":"2024-06-10T01:04:35.550140+0000","last_undegraded":"2024-06-10T01:04:35.550140+0000","last_fullsized":"2024-06-10T01:04:35.550140+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:19:46.599235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235375+0000","last_change":"2024-06-10T01:01:57.332887+0000","last_active":"2024-06-10T01:03:38.235375+0000","last_peered":"2024-06-10T01:03:38.235375+0000","last_clean":"2024-06-10T01:03:38.235375+0000","last_became_active":"2024-06-10T01:01:57.332285+0000","last_became_peered":"2024-06-10T01:01:57.332285+0000","last_unstale":"2024-06-10T01:03:38.235375+0000","last_undegraded":"2024-06-10T01:03:38.235375+0000","last_fullsized":"2024-06-10T01:03:38.235375+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:09:12.233405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:50.167401+0000","last_change":"2024-06-10T01:01:44.466831+0000","last_active":"2024-06-10T01:03:50.167401+0000","last_peered":"2024-06-10T01:03:50.167401+0000","last_clean":"2024-06-10T01:03:50.167401+0000","last_became_active":"2024-06-10T01:01:44.466484+0000","last_became_peered":"2024-06-10T01:01:44.466484+0000","last_unstale":"2024-06-10T01:03:50.167401+0000","last_undegraded":"2024-06-10T01:03:50.167401+0000","last_fullsized":"2024-06-10T01:03:50.167401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:19:28.385504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.556249+0000","last_change":"2024-06-10T01:01:47.252399+0000","last_active":"2024-06-10T01:04:35.556249+0000","last_peered":"2024-06-10T01:04:35.556249+0000","last_clean":"2024-06-10T01:04:35.556249+0000","last_became_active":"2024-06-10T01:01:47.252069+0000","last_became_peered":"2024-06-10T01:01:47.252069+0000","last_unstale":"2024-06-10T01:04:35.556249+0000","last_undegraded":"2024-06-10T01:04:35.556249+0000","last_fullsized":"2024-06-10T01:04:35.556249+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:02:02.291174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771406+0000","last_change":"2024-06-10T01:01:57.285670+0000","last_active":"2024-06-10T01:03:38.771406+0000","last_peered":"2024-06-10T01:03:38.771406+0000","last_clean":"2024-06-10T01:03:38.771406+0000","last_became_active":"2024-06-10T01:01:57.285281+0000","last_became_peered":"2024-06-10T01:01:57.285281+0000","last_unstale":"2024-06-10T01:03:38.771406+0000","last_undegraded":"2024-06-10T01:03:38.771406+0000","last_fullsized":"2024-06-10T01:03:38.771406+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:58.956934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:48.604327+0000","last_change":"2024-06-10T01:01:44.008013+0000","last_active":"2024-06-10T01:03:48.604327+0000","last_peered":"2024-06-10T01:03:48.604327+0000","last_clean":"2024-06-10T01:03:48.604327+0000","last_became_active":"2024-06-10T01:01:44.007591+0000","last_became_peered":"2024-06-10T01:01:44.007591+0000","last_unstale":"2024-06-10T01:03:48.604327+0000","last_undegraded":"2024-06-10T01:03:48.604327+0000","last_fullsized":"2024-06-10T01:03:48.604327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:18:20.862340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.551776+0000","last_change":"2024-06-10T01:01:47.254756+0000","last_active":"2024-06-10T01:04:35.551776+0000","last_peered":"2024-06-10T01:04:35.551776+0000","last_clean":"2024-06-10T01:04:35.551776+0000","last_became_active":"2024-06-10T01:01:47.254418+0000","last_became_peered":"2024-06-10T01:01:47.254418+0000","last_unstale":"2024-06-10T01:04:35.551776+0000","last_undegraded":"2024-06-10T01:04:35.551776+0000","last_fullsized":"2024-06-10T01:04:35.551776+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:46:31.452539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770995+0000","last_change":"2024-06-10T01:01:57.332792+0000","last_active":"2024-06-10T01:03:38.770995+0000","last_peered":"2024-06-10T01:03:38.770995+0000","last_clean":"2024-06-10T01:03:38.770995+0000","last_became_active":"2024-06-10T01:01:57.332331+0000","last_became_peered":"2024-06-10T01:01:57.332331+0000","last_unstale":"2024-06-10T01:03:38.770995+0000","last_undegraded":"2024-06-10T01:03:38.770995+0000","last_fullsized":"2024-06-10T01:03:38.770995+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:46:41.419807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.565745+0000","last_change":"2024-06-10T01:01:47.247857+0000","last_active":"2024-06-10T01:04:35.565745+0000","last_peered":"2024-06-10T01:04:35.565745+0000","last_clean":"2024-06-10T01:04:35.565745+0000","last_became_active":"2024-06-10T01:01:47.247552+0000","last_became_peered":"2024-06-10T01:01:47.247552+0000","last_unstale":"2024-06-10T01:04:35.565745+0000","last_undegraded":"2024-06-10T01:04:35.565745+0000","last_fullsized":"2024-06-10T01:04:35.565745+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:29:40.521887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"99'5","reported_seq":235,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:57.978068+0000","last_change":"2024-06-10T01:01:44.466631+0000","last_active":"2024-06-10T01:05:57.978068+0000","last_peered":"2024-06-10T01:05:57.978068+0000","last_clean":"2024-06-10T01:05:57.978068+0000","last_became_active":"2024-06-10T01:01:44.466280+0000","last_became_peered":"2024-06-10T01:01:44.466280+0000","last_unstale":"2024-06-10T01:05:57.978068+0000","last_undegraded":"2024-06-10T01:05:57.978068+0000","last_fullsized":"2024-06-10T01:05:57.978068+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:09:33.658112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":184,"num_read_kb":184,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235365+0000","last_change":"2024-06-10T01:01:57.281973+0000","last_active":"2024-06-10T01:03:38.235365+0000","last_peered":"2024-06-10T01:03:38.235365+0000","last_clean":"2024-06-10T01:03:38.235365+0000","last_became_active":"2024-06-10T01:01:57.281386+0000","last_became_peered":"2024-06-10T01:01:57.281386+0000","last_unstale":"2024-06-10T01:03:38.235365+0000","last_undegraded":"2024-06-10T01:03:38.235365+0000","last_fullsized":"2024-06-10T01:03:38.235365+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:10:28.236250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.562526+0000","last_change":"2024-06-10T01:01:47.237436+0000","last_active":"2024-06-10T01:04:35.562526+0000","last_peered":"2024-06-10T01:04:35.562526+0000","last_clean":"2024-06-10T01:04:35.562526+0000","last_became_active":"2024-06-10T01:01:47.237099+0000","last_became_peered":"2024-06-10T01:01:47.237099+0000","last_unstale":"2024-06-10T01:04:35.562526+0000","last_undegraded":"2024-06-10T01:04:35.562526+0000","last_fullsized":"2024-06-10T01:04:35.562526+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:34:53.838877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'3","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:48.634491+0000","last_change":"2024-06-10T01:01:44.010655+0000","last_active":"2024-06-10T01:03:48.634491+0000","last_peered":"2024-06-10T01:03:48.634491+0000","last_clean":"2024-06-10T01:03:48.634491+0000","last_became_active":"2024-06-10T01:01:44.009918+0000","last_became_peered":"2024-06-10T01:01:44.009918+0000","last_unstale":"2024-06-10T01:03:48.634491+0000","last_undegraded":"2024-06-10T01:03:48.634491+0000","last_fullsized":"2024-06-10T01:03:48.634491+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:47:32.052650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":367,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234906+0000","last_change":"2024-06-10T01:00:23.822447+0000","last_active":"2024-06-10T01:03:38.234906+0000","last_peered":"2024-06-10T01:03:38.234906+0000","last_clean":"2024-06-10T01:03:38.234906+0000","last_became_active":"2024-06-10T01:00:23.821966+0000","last_became_peered":"2024-06-10T01:00:23.821966+0000","last_unstale":"2024-06-10T01:03:38.234906+0000","last_undegraded":"2024-06-10T01:03:38.234906+0000","last_fullsized":"2024-06-10T01:03:38.234906+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:56:59.964661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:56:59.964661+0000","last_clean_scrub_stamp":"2024-06-10T00:56:59.964661+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:03:43.732875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270272+0000","last_change":"2024-06-10T01:01:57.332980+0000","last_active":"2024-06-10T01:03:38.270272+0000","last_peered":"2024-06-10T01:03:38.270272+0000","last_clean":"2024-06-10T01:03:38.270272+0000","last_became_active":"2024-06-10T01:01:57.332437+0000","last_became_peered":"2024-06-10T01:01:57.332437+0000","last_unstale":"2024-06-10T01:03:38.270272+0000","last_undegraded":"2024-06-10T01:03:38.270272+0000","last_fullsized":"2024-06-10T01:03:38.270272+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:27:35.029079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559893+0000","last_change":"2024-06-10T01:01:47.255926+0000","last_active":"2024-06-10T01:04:35.559893+0000","last_peered":"2024-06-10T01:04:35.559893+0000","last_clean":"2024-06-10T01:04:35.559893+0000","last_became_active":"2024-06-10T01:01:47.255571+0000","last_became_peered":"2024-06-10T01:01:47.255571+0000","last_unstale":"2024-06-10T01:04:35.559893+0000","last_undegraded":"2024-06-10T01:04:35.559893+0000","last_fullsized":"2024-06-10T01:04:35.559893+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:05:22.084037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:43.603848+0000","last_change":"2024-06-10T01:01:44.007509+0000","last_active":"2024-06-10T01:03:43.603848+0000","last_peered":"2024-06-10T01:03:43.603848+0000","last_clean":"2024-06-10T01:03:43.603848+0000","last_became_active":"2024-06-10T01:01:44.007178+0000","last_became_peered":"2024-06-10T01:01:44.007178+0000","last_unstale":"2024-06-10T01:03:43.603848+0000","last_undegraded":"2024-06-10T01:03:43.603848+0000","last_fullsized":"2024-06-10T01:03:43.603848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:02:52.791886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.236790+0000","last_change":"2024-06-10T01:01:57.285626+0000","last_active":"2024-06-10T01:03:38.236790+0000","last_peered":"2024-06-10T01:03:38.236790+0000","last_clean":"2024-06-10T01:03:38.236790+0000","last_became_active":"2024-06-10T01:01:57.285098+0000","last_became_peered":"2024-06-10T01:01:57.285098+0000","last_unstale":"2024-06-10T01:03:38.236790+0000","last_undegraded":"2024-06-10T01:03:38.236790+0000","last_fullsized":"2024-06-10T01:03:38.236790+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:13:49.881770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:49.313610+0000","last_change":"2024-06-10T01:01:44.469028+0000","last_active":"2024-06-10T01:03:49.313610+0000","last_peered":"2024-06-10T01:03:49.313610+0000","last_clean":"2024-06-10T01:03:49.313610+0000","last_became_active":"2024-06-10T01:01:44.468570+0000","last_became_peered":"2024-06-10T01:01:44.468570+0000","last_unstale":"2024-06-10T01:03:49.313610+0000","last_undegraded":"2024-06-10T01:03:49.313610+0000","last_fullsized":"2024-06-10T01:03:49.313610+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:12:11.287959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.560378+0000","last_change":"2024-06-10T01:01:47.253414+0000","last_active":"2024-06-10T01:04:35.560378+0000","last_peered":"2024-06-10T01:04:35.560378+0000","last_clean":"2024-06-10T01:04:35.560378+0000","last_became_active":"2024-06-10T01:01:47.253060+0000","last_became_peered":"2024-06-10T01:01:47.253060+0000","last_unstale":"2024-06-10T01:04:35.560378+0000","last_undegraded":"2024-06-10T01:04:35.560378+0000","last_fullsized":"2024-06-10T01:04:35.560378+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:24:55.534164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:58.961104+0000","last_change":"2024-06-10T01:01:57.332756+0000","last_active":"2024-06-10T01:03:58.961104+0000","last_peered":"2024-06-10T01:03:58.961104+0000","last_clean":"2024-06-10T01:03:58.961104+0000","last_became_active":"2024-06-10T01:01:57.332361+0000","last_became_peered":"2024-06-10T01:01:57.332361+0000","last_unstale":"2024-06-10T01:03:58.961104+0000","last_undegraded":"2024-06-10T01:03:58.961104+0000","last_fullsized":"2024-06-10T01:03:58.961104+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:02:28.275407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"99'14","reported_seq":60,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:05:39.274900+0000","last_change":"2024-06-10T01:01:44.005164+0000","last_active":"2024-06-10T01:05:39.274900+0000","last_peered":"2024-06-10T01:05:39.274900+0000","last_clean":"2024-06-10T01:05:39.274900+0000","last_became_active":"2024-06-10T01:01:44.004816+0000","last_became_peered":"2024-06-10T01:01:44.004816+0000","last_unstale":"2024-06-10T01:05:39.274900+0000","last_undegraded":"2024-06-10T01:05:39.274900+0000","last_fullsized":"2024-06-10T01:05:39.274900+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:11:56.385267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.557511+0000","last_change":"2024-06-10T01:01:47.247951+0000","last_active":"2024-06-10T01:04:35.557511+0000","last_peered":"2024-06-10T01:04:35.557511+0000","last_clean":"2024-06-10T01:04:35.557511+0000","last_became_active":"2024-06-10T01:01:47.246996+0000","last_became_peered":"2024-06-10T01:01:47.246996+0000","last_unstale":"2024-06-10T01:04:35.557511+0000","last_undegraded":"2024-06-10T01:04:35.557511+0000","last_fullsized":"2024-06-10T01:04:35.557511+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:39:18.167021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239228+0000","last_change":"2024-06-10T01:01:57.278130+0000","last_active":"2024-06-10T01:03:38.239228+0000","last_peered":"2024-06-10T01:03:38.239228+0000","last_clean":"2024-06-10T01:03:38.239228+0000","last_became_active":"2024-06-10T01:01:57.277697+0000","last_became_peered":"2024-06-10T01:01:57.277697+0000","last_unstale":"2024-06-10T01:03:38.239228+0000","last_undegraded":"2024-06-10T01:03:38.239228+0000","last_fullsized":"2024-06-10T01:03:38.239228+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:17:01.662152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239618+0000","last_change":"2024-06-10T01:01:44.553176+0000","last_active":"2024-06-10T01:03:38.239618+0000","last_peered":"2024-06-10T01:03:38.239618+0000","last_clean":"2024-06-10T01:03:38.239618+0000","last_became_active":"2024-06-10T01:01:44.552797+0000","last_became_peered":"2024-06-10T01:01:44.552797+0000","last_unstale":"2024-06-10T01:03:38.239618+0000","last_undegraded":"2024-06-10T01:03:38.239618+0000","last_fullsized":"2024-06-10T01:03:38.239618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:21:13.960304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.556104+0000","last_change":"2024-06-10T01:01:47.255796+0000","last_active":"2024-06-10T01:04:35.556104+0000","last_peered":"2024-06-10T01:04:35.556104+0000","last_clean":"2024-06-10T01:04:35.556104+0000","last_became_active":"2024-06-10T01:01:47.255495+0000","last_became_peered":"2024-06-10T01:01:47.255495+0000","last_unstale":"2024-06-10T01:04:35.556104+0000","last_undegraded":"2024-06-10T01:04:35.556104+0000","last_fullsized":"2024-06-10T01:04:35.556104+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:06:42.209005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239491+0000","last_change":"2024-06-10T01:01:57.332869+0000","last_active":"2024-06-10T01:03:38.239491+0000","last_peered":"2024-06-10T01:03:38.239491+0000","last_clean":"2024-06-10T01:03:38.239491+0000","last_became_active":"2024-06-10T01:01:57.332391+0000","last_became_peered":"2024-06-10T01:01:57.332391+0000","last_unstale":"2024-06-10T01:03:38.239491+0000","last_undegraded":"2024-06-10T01:03:38.239491+0000","last_fullsized":"2024-06-10T01:03:38.239491+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:58:31.443626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770788+0000","last_change":"2024-06-10T01:01:44.466861+0000","last_active":"2024-06-10T01:03:38.770788+0000","last_peered":"2024-06-10T01:03:38.770788+0000","last_clean":"2024-06-10T01:03:38.770788+0000","last_became_active":"2024-06-10T01:01:44.466553+0000","last_became_peered":"2024-06-10T01:01:44.466553+0000","last_unstale":"2024-06-10T01:03:38.770788+0000","last_undegraded":"2024-06-10T01:03:38.770788+0000","last_fullsized":"2024-06-10T01:03:38.770788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:55:52.545594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":41,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.544035+0000","last_change":"2024-06-10T01:01:47.244972+0000","last_active":"2024-06-10T01:04:35.544035+0000","last_peered":"2024-06-10T01:04:35.544035+0000","last_clean":"2024-06-10T01:04:35.544035+0000","last_became_active":"2024-06-10T01:01:47.244649+0000","last_became_peered":"2024-06-10T01:01:47.244649+0000","last_unstale":"2024-06-10T01:04:35.544035+0000","last_undegraded":"2024-06-10T01:04:35.544035+0000","last_fullsized":"2024-06-10T01:04:35.544035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:10:46.909200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235461+0000","last_change":"2024-06-10T01:02:37.592380+0000","last_active":"2024-06-10T01:03:38.235461+0000","last_peered":"2024-06-10T01:03:38.235461+0000","last_clean":"2024-06-10T01:03:38.235461+0000","last_became_active":"2024-06-10T01:02:37.591856+0000","last_became_peered":"2024-06-10T01:02:37.591856+0000","last_unstale":"2024-06-10T01:03:38.235461+0000","last_undegraded":"2024-06-10T01:03:38.235461+0000","last_fullsized":"2024-06-10T01:03:38.235461+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:00:25.717192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239006+0000","last_change":"2024-06-10T01:01:44.008116+0000","last_active":"2024-06-10T01:03:38.239006+0000","last_peered":"2024-06-10T01:03:38.239006+0000","last_clean":"2024-06-10T01:03:38.239006+0000","last_became_active":"2024-06-10T01:01:44.007778+0000","last_became_peered":"2024-06-10T01:01:44.007778+0000","last_unstale":"2024-06-10T01:03:38.239006+0000","last_undegraded":"2024-06-10T01:03:38.239006+0000","last_fullsized":"2024-06-10T01:03:38.239006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:18:57.267296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.556264+0000","last_change":"2024-06-10T01:01:47.252833+0000","last_active":"2024-06-10T01:04:35.556264+0000","last_peered":"2024-06-10T01:04:35.556264+0000","last_clean":"2024-06-10T01:04:35.556264+0000","last_became_active":"2024-06-10T01:01:47.252326+0000","last_became_peered":"2024-06-10T01:01:47.252326+0000","last_unstale":"2024-06-10T01:04:35.556264+0000","last_undegraded":"2024-06-10T01:04:35.556264+0000","last_fullsized":"2024-06-10T01:04:35.556264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:00:42.673838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239284+0000","last_change":"2024-06-10T01:01:57.275975+0000","last_active":"2024-06-10T01:03:38.239284+0000","last_peered":"2024-06-10T01:03:38.239284+0000","last_clean":"2024-06-10T01:03:38.239284+0000","last_became_active":"2024-06-10T01:01:57.275408+0000","last_became_peered":"2024-06-10T01:01:57.275408+0000","last_unstale":"2024-06-10T01:03:38.239284+0000","last_undegraded":"2024-06-10T01:03:38.239284+0000","last_fullsized":"2024-06-10T01:03:38.239284+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:12:42.376636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'1","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:40.912786+0000","last_change":"2024-06-10T01:03:38.243422+0000","last_active":"2024-06-10T01:03:40.912786+0000","last_peered":"2024-06-10T01:03:40.912786+0000","last_clean":"2024-06-10T01:03:40.912786+0000","last_became_active":"2024-06-10T01:03:38.239916+0000","last_became_peered":"2024-06-10T01:03:38.239916+0000","last_unstale":"2024-06-10T01:03:40.912786+0000","last_undegraded":"2024-06-10T01:03:40.912786+0000","last_fullsized":"2024-06-10T01:03:40.912786+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:30:01.399956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.555295+0000","last_change":"2024-06-10T01:01:47.243154+0000","last_active":"2024-06-10T01:04:35.555295+0000","last_peered":"2024-06-10T01:04:35.555295+0000","last_clean":"2024-06-10T01:04:35.555295+0000","last_became_active":"2024-06-10T01:01:47.242851+0000","last_became_peered":"2024-06-10T01:01:47.242851+0000","last_unstale":"2024-06-10T01:04:35.555295+0000","last_undegraded":"2024-06-10T01:04:35.555295+0000","last_fullsized":"2024-06-10T01:04:35.555295+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:41:05.857777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239584+0000","last_change":"2024-06-10T01:01:57.333613+0000","last_active":"2024-06-10T01:03:38.239584+0000","last_peered":"2024-06-10T01:03:38.239584+0000","last_clean":"2024-06-10T01:03:38.239584+0000","last_became_active":"2024-06-10T01:01:57.333019+0000","last_became_peered":"2024-06-10T01:01:57.333019+0000","last_unstale":"2024-06-10T01:03:38.239584+0000","last_undegraded":"2024-06-10T01:03:38.239584+0000","last_fullsized":"2024-06-10T01:03:38.239584+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:11.912885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.235303+0000","last_change":"2024-06-10T01:01:44.008221+0000","last_active":"2024-06-10T01:03:38.235303+0000","last_peered":"2024-06-10T01:03:38.235303+0000","last_clean":"2024-06-10T01:03:38.235303+0000","last_became_active":"2024-06-10T01:01:44.007880+0000","last_became_peered":"2024-06-10T01:01:44.007880+0000","last_unstale":"2024-06-10T01:03:38.235303+0000","last_undegraded":"2024-06-10T01:03:38.235303+0000","last_fullsized":"2024-06-10T01:03:38.235303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:40:33.162925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558463+0000","last_change":"2024-06-10T01:01:47.254902+0000","last_active":"2024-06-10T01:04:35.558463+0000","last_peered":"2024-06-10T01:04:35.558463+0000","last_clean":"2024-06-10T01:04:35.558463+0000","last_became_active":"2024-06-10T01:01:47.254555+0000","last_became_peered":"2024-06-10T01:01:47.254555+0000","last_unstale":"2024-06-10T01:04:35.558463+0000","last_undegraded":"2024-06-10T01:04:35.558463+0000","last_fullsized":"2024-06-10T01:04:35.558463+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:03:17.571001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.770786+0000","last_change":"2024-06-10T01:01:57.284491+0000","last_active":"2024-06-10T01:03:38.770786+0000","last_peered":"2024-06-10T01:03:38.770786+0000","last_clean":"2024-06-10T01:03:38.770786+0000","last_became_active":"2024-06-10T01:01:57.283795+0000","last_became_peered":"2024-06-10T01:01:57.283795+0000","last_unstale":"2024-06-10T01:03:38.770786+0000","last_undegraded":"2024-06-10T01:03:38.770786+0000","last_fullsized":"2024-06-10T01:03:38.770786+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:49:00.381664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.239889+0000","last_change":"2024-06-10T01:01:44.553159+0000","last_active":"2024-06-10T01:03:38.239889+0000","last_peered":"2024-06-10T01:03:38.239889+0000","last_clean":"2024-06-10T01:03:38.239889+0000","last_became_active":"2024-06-10T01:01:44.552759+0000","last_became_peered":"2024-06-10T01:01:44.552759+0000","last_unstale":"2024-06-10T01:03:38.239889+0000","last_undegraded":"2024-06-10T01:03:38.239889+0000","last_fullsized":"2024-06-10T01:03:38.239889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:19:00.110790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559093+0000","last_change":"2024-06-10T01:01:47.247803+0000","last_active":"2024-06-10T01:04:35.559093+0000","last_peered":"2024-06-10T01:04:35.559093+0000","last_clean":"2024-06-10T01:04:35.559093+0000","last_became_active":"2024-06-10T01:01:47.246715+0000","last_became_peered":"2024-06-10T01:01:47.246715+0000","last_unstale":"2024-06-10T01:04:35.559093+0000","last_undegraded":"2024-06-10T01:04:35.559093+0000","last_fullsized":"2024-06-10T01:04:35.559093+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:22.429002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.234192+0000","last_change":"2024-06-10T01:01:57.287668+0000","last_active":"2024-06-10T01:03:38.234192+0000","last_peered":"2024-06-10T01:03:38.234192+0000","last_clean":"2024-06-10T01:03:38.234192+0000","last_became_active":"2024-06-10T01:01:57.287060+0000","last_became_peered":"2024-06-10T01:01:57.287060+0000","last_unstale":"2024-06-10T01:03:38.234192+0000","last_undegraded":"2024-06-10T01:03:38.234192+0000","last_fullsized":"2024-06-10T01:03:38.234192+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:04:35.883613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.270251+0000","last_change":"2024-06-10T01:01:44.013051+0000","last_active":"2024-06-10T01:03:38.270251+0000","last_peered":"2024-06-10T01:03:38.270251+0000","last_clean":"2024-06-10T01:03:38.270251+0000","last_became_active":"2024-06-10T01:01:44.012667+0000","last_became_peered":"2024-06-10T01:01:44.012667+0000","last_unstale":"2024-06-10T01:03:38.270251+0000","last_undegraded":"2024-06-10T01:03:38.270251+0000","last_fullsized":"2024-06-10T01:03:38.270251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:10:49.650082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.563083+0000","last_change":"2024-06-10T01:01:47.251209+0000","last_active":"2024-06-10T01:04:35.563083+0000","last_peered":"2024-06-10T01:04:35.563083+0000","last_clean":"2024-06-10T01:04:35.563083+0000","last_became_active":"2024-06-10T01:01:47.250858+0000","last_became_peered":"2024-06-10T01:01:47.250858+0000","last_unstale":"2024-06-10T01:04:35.563083+0000","last_undegraded":"2024-06-10T01:04:35.563083+0000","last_fullsized":"2024-06-10T01:04:35.563083+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:08:47.272727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.269600+0000","last_change":"2024-06-10T01:01:57.284187+0000","last_active":"2024-06-10T01:03:38.269600+0000","last_peered":"2024-06-10T01:03:38.269600+0000","last_clean":"2024-06-10T01:03:38.269600+0000","last_became_active":"2024-06-10T01:01:57.283739+0000","last_became_peered":"2024-06-10T01:01:57.283739+0000","last_unstale":"2024-06-10T01:03:38.269600+0000","last_undegraded":"2024-06-10T01:03:38.269600+0000","last_fullsized":"2024-06-10T01:03:38.269600+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:56.258976+0000","last_clean_scrub_stamp":"2024-06-10T01:01:56.258976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:13:34.630708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:03:38.771085+0000","last_change":"2024-06-10T01:01:44.005702+0000","last_active":"2024-06-10T01:03:38.771085+0000","last_peered":"2024-06-10T01:03:38.771085+0000","last_clean":"2024-06-10T01:03:38.771085+0000","last_became_active":"2024-06-10T01:01:44.005354+0000","last_became_peered":"2024-06-10T01:01:44.005354+0000","last_unstale":"2024-06-10T01:03:38.771085+0000","last_undegraded":"2024-06-10T01:03:38.771085+0000","last_fullsized":"2024-06-10T01:03:38.771085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:42.985660+0000","last_clean_scrub_stamp":"2024-06-10T01:01:42.985660+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:51:14.819688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.558942+0000","last_change":"2024-06-10T01:01:47.250325+0000","last_active":"2024-06-10T01:04:35.558942+0000","last_peered":"2024-06-10T01:04:35.558942+0000","last_clean":"2024-06-10T01:04:35.558942+0000","last_became_active":"2024-06-10T01:01:47.249591+0000","last_became_peered":"2024-06-10T01:01:47.249591+0000","last_unstale":"2024-06-10T01:04:35.558942+0000","last_undegraded":"2024-06-10T01:04:35.558942+0000","last_fullsized":"2024-06-10T01:04:35.558942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:17:16.268717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.563751+0000","last_change":"2024-06-10T01:01:47.253449+0000","last_active":"2024-06-10T01:04:35.563751+0000","last_peered":"2024-06-10T01:04:35.563751+0000","last_clean":"2024-06-10T01:04:35.563751+0000","last_became_active":"2024-06-10T01:01:47.253144+0000","last_became_peered":"2024-06-10T01:01:47.253144+0000","last_unstale":"2024-06-10T01:04:35.563751+0000","last_undegraded":"2024-06-10T01:04:35.563751+0000","last_fullsized":"2024-06-10T01:04:35.563751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:32:22.353409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.564183+0000","last_change":"2024-06-10T01:01:47.258241+0000","last_active":"2024-06-10T01:04:35.564183+0000","last_peered":"2024-06-10T01:04:35.564183+0000","last_clean":"2024-06-10T01:04:35.564183+0000","last_became_active":"2024-06-10T01:01:47.257823+0000","last_became_peered":"2024-06-10T01:01:47.257823+0000","last_unstale":"2024-06-10T01:04:35.564183+0000","last_undegraded":"2024-06-10T01:04:35.564183+0000","last_fullsized":"2024-06-10T01:04:35.564183+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:11:23.652479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559614+0000","last_change":"2024-06-10T01:01:47.256218+0000","last_active":"2024-06-10T01:04:35.559614+0000","last_peered":"2024-06-10T01:04:35.559614+0000","last_clean":"2024-06-10T01:04:35.559614+0000","last_became_active":"2024-06-10T01:01:47.255950+0000","last_became_peered":"2024-06-10T01:01:47.255950+0000","last_unstale":"2024-06-10T01:04:35.559614+0000","last_undegraded":"2024-06-10T01:04:35.559614+0000","last_fullsized":"2024-06-10T01:04:35.559614+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:59:37.283646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.556367+0000","last_change":"2024-06-10T01:01:47.253109+0000","last_active":"2024-06-10T01:04:35.556367+0000","last_peered":"2024-06-10T01:04:35.556367+0000","last_clean":"2024-06-10T01:04:35.556367+0000","last_became_active":"2024-06-10T01:01:47.252767+0000","last_became_peered":"2024-06-10T01:01:47.252767+0000","last_unstale":"2024-06-10T01:04:35.556367+0000","last_undegraded":"2024-06-10T01:04:35.556367+0000","last_fullsized":"2024-06-10T01:04:35.556367+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:57:13.177230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.550864+0000","last_change":"2024-06-10T01:02:37.375094+0000","last_active":"2024-06-10T01:04:35.550864+0000","last_peered":"2024-06-10T01:04:35.550864+0000","last_clean":"2024-06-10T01:04:35.550864+0000","last_became_active":"2024-06-10T01:02:37.374527+0000","last_became_peered":"2024-06-10T01:02:37.374527+0000","last_unstale":"2024-06-10T01:04:35.550864+0000","last_undegraded":"2024-06-10T01:04:35.550864+0000","last_fullsized":"2024-06-10T01:04:35.550864+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:34:41.205816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.551206+0000","last_change":"2024-06-10T01:01:47.250243+0000","last_active":"2024-06-10T01:04:35.551206+0000","last_peered":"2024-06-10T01:04:35.551206+0000","last_clean":"2024-06-10T01:04:35.551206+0000","last_became_active":"2024-06-10T01:01:47.249907+0000","last_became_peered":"2024-06-10T01:01:47.249907+0000","last_unstale":"2024-06-10T01:04:35.551206+0000","last_undegraded":"2024-06-10T01:04:35.551206+0000","last_fullsized":"2024-06-10T01:04:35.551206+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:31:32.933660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.564063+0000","last_change":"2024-06-10T01:01:47.254173+0000","last_active":"2024-06-10T01:04:35.564063+0000","last_peered":"2024-06-10T01:04:35.564063+0000","last_clean":"2024-06-10T01:04:35.564063+0000","last_became_active":"2024-06-10T01:01:47.253796+0000","last_became_peered":"2024-06-10T01:01:47.253796+0000","last_unstale":"2024-06-10T01:04:35.564063+0000","last_undegraded":"2024-06-10T01:04:35.564063+0000","last_fullsized":"2024-06-10T01:04:35.564063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:41:12.199114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":41,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.548118+0000","last_change":"2024-06-10T01:01:47.253920+0000","last_active":"2024-06-10T01:04:35.548118+0000","last_peered":"2024-06-10T01:04:35.548118+0000","last_clean":"2024-06-10T01:04:35.548118+0000","last_became_active":"2024-06-10T01:01:47.253288+0000","last_became_peered":"2024-06-10T01:01:47.253288+0000","last_unstale":"2024-06-10T01:04:35.548118+0000","last_undegraded":"2024-06-10T01:04:35.548118+0000","last_fullsized":"2024-06-10T01:04:35.548118+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:41:20.406263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.564816+0000","last_change":"2024-06-10T01:01:47.240772+0000","last_active":"2024-06-10T01:04:35.564816+0000","last_peered":"2024-06-10T01:04:35.564816+0000","last_clean":"2024-06-10T01:04:35.564816+0000","last_became_active":"2024-06-10T01:01:47.238258+0000","last_became_peered":"2024-06-10T01:01:47.238258+0000","last_unstale":"2024-06-10T01:04:35.564816+0000","last_undegraded":"2024-06-10T01:04:35.564816+0000","last_fullsized":"2024-06-10T01:04:35.564816+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:55:25.211434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":24,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.544703+0000","last_change":"2024-06-10T01:02:37.376724+0000","last_active":"2024-06-10T01:04:35.544703+0000","last_peered":"2024-06-10T01:04:35.544703+0000","last_clean":"2024-06-10T01:04:35.544703+0000","last_became_active":"2024-06-10T01:02:37.376329+0000","last_became_peered":"2024-06-10T01:02:37.376329+0000","last_unstale":"2024-06-10T01:04:35.544703+0000","last_undegraded":"2024-06-10T01:04:35.544703+0000","last_fullsized":"2024-06-10T01:04:35.544703+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:38:21.837958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.565197+0000","last_change":"2024-06-10T01:01:47.244463+0000","last_active":"2024-06-10T01:04:35.565197+0000","last_peered":"2024-06-10T01:04:35.565197+0000","last_clean":"2024-06-10T01:04:35.565197+0000","last_became_active":"2024-06-10T01:01:47.244114+0000","last_became_peered":"2024-06-10T01:01:47.244114+0000","last_unstale":"2024-06-10T01:04:35.565197+0000","last_undegraded":"2024-06-10T01:04:35.565197+0000","last_fullsized":"2024-06-10T01:04:35.565197+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:47:37.098894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.559211+0000","last_change":"2024-06-10T01:01:47.248317+0000","last_active":"2024-06-10T01:04:35.559211+0000","last_peered":"2024-06-10T01:04:35.559211+0000","last_clean":"2024-06-10T01:04:35.559211+0000","last_became_active":"2024-06-10T01:01:47.247958+0000","last_became_peered":"2024-06-10T01:01:47.247958+0000","last_unstale":"2024-06-10T01:04:35.559211+0000","last_undegraded":"2024-06-10T01:04:35.559211+0000","last_fullsized":"2024-06-10T01:04:35.559211+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:39:59.267957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T01:04:35.552389+0000","last_change":"2024-06-10T01:01:47.252542+0000","last_active":"2024-06-10T01:04:35.552389+0000","last_peered":"2024-06-10T01:04:35.552389+0000","last_clean":"2024-06-10T01:04:35.552389+0000","last_became_active":"2024-06-10T01:01:47.252216+0000","last_became_peered":"2024-06-10T01:01:47.252216+0000","last_unstale":"2024-06-10T01:04:35.552389+0000","last_undegraded":"2024-06-10T01:04:35.552389+0000","last_fullsized":"2024-06-10T01:04:35.552389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T01:01:46.226105+0000","last_clean_scrub_stamp":"2024-06-10T01:01:46.226105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:12:22.196263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285691,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73985,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1137,"num_read_kb":650,"num_write":186,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":221955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383747,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":569843,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645384,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32392,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712760,"statfs":{"total":95995035648,"available":95961866240,"internally_reserved":0,"allocated":1245184,"data_stored":1227374,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808909,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":574063,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070546,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":569775,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332182,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":1165713,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495707,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":565143,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1163264,"data_stored":635120,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738489,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":566531,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1179648,"data_stored":636358,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182507,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":565337,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920869,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":570213,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757344,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":1160993,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34687,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34687,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34687,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36326,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":70067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T01:06:03.773 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:06:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.798+0000 7fa46f190640 1 -- 172.21.15.7:0/4075355659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa468107780 msgr2=0x7fa468109b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.798+0000 7fa46f190640 1 --2- 172.21.15.7:0/4075355659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa468107780 0x7fa468109b70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa460009ca0 tx=0x7fa46002a8c0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.798+0000 7fa46f190640 1 -- 172.21.15.7:0/4075355659 shutdown_connections 2024-06-10T01:06:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.798+0000 7fa46f190640 1 --2- 172.21.15.7:0/4075355659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa468107780 0x7fa468109b70 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.798+0000 7fa46f190640 1 --2- 172.21.15.7:0/4075355659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa468104e50 0x7fa468107240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.798+0000 7fa46f190640 1 --2- 172.21.15.7:0/4075355659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa468102520 0x7fa468104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.798+0000 7fa46f190640 1 -- 172.21.15.7:0/4075355659 >> 172.21.15.7:0/4075355659 conn(0x7fa4680fbed0 msgr2=0x7fa4680fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.798+0000 7fa46f190640 1 -- 172.21.15.7:0/4075355659 shutdown_connections 2024-06-10T01:06:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.798+0000 7fa46f190640 1 -- 172.21.15.7:0/4075355659 wait complete. 2024-06-10T01:06:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.799+0000 7fa46f190640 1 Processor -- start 2024-06-10T01:06:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.799+0000 7fa46f190640 1 -- start start 2024-06-10T01:06:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.799+0000 7fa46f190640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa468102520 0x7fa468197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.799+0000 7fa46f190640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa468104e50 0x7fa468198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.799+0000 7fa46f190640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa468107780 0x7fa46819f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.799+0000 7fa46f190640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa46810d6e0 con 0x7fa468104e50 2024-06-10T01:06:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.799+0000 7fa46f190640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa46810d560 con 0x7fa468102520 2024-06-10T01:06:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.799+0000 7fa46f190640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa46810d860 con 0x7fa468107780 2024-06-10T01:06:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.799+0000 7fa45ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa468104e50 0x7fa468198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.799+0000 7fa45ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa468104e50 0x7fa468198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52930/0 (socket says 172.21.15.7:52930) 2024-06-10T01:06:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.799+0000 7fa45ffff640 1 -- 172.21.15.7:0/1097860218 learned_addr learned my addr 172.21.15.7:0/1097860218 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.799+0000 7fa46cf2c640 1 --2- 172.21.15.7:0/1097860218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa468102520 0x7fa468197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.799+0000 7fa46d72d640 1 --2- 172.21.15.7:0/1097860218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa468107780 0x7fa46819f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.800+0000 7fa45ffff640 1 -- 172.21.15.7:0/1097860218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa468107780 msgr2=0x7fa46819f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.800+0000 7fa45ffff640 1 --2- 172.21.15.7:0/1097860218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa468107780 0x7fa46819f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.800+0000 7fa45ffff640 1 -- 172.21.15.7:0/1097860218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa468102520 msgr2=0x7fa468197d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.800+0000 7fa45ffff640 1 --2- 172.21.15.7:0/1097860218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa468102520 0x7fa468197d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.800+0000 7fa45ffff640 1 -- 172.21.15.7:0/1097860218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa46819f920 con 0x7fa468104e50 2024-06-10T01:06:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.800+0000 7fa45ffff640 1 --2- 172.21.15.7:0/1097860218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa468104e50 0x7fa468198250 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa45800e9c0 tx=0x7fa45800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.800+0000 7fa45dffb640 1 -- 172.21.15.7:0/1097860218 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45800cd80 con 0x7fa468104e50 2024-06-10T01:06:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.800+0000 7fa45dffb640 1 -- 172.21.15.7:0/1097860218 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa458005500 con 0x7fa468104e50 2024-06-10T01:06:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.800+0000 7fa46f190640 1 -- 172.21.15.7:0/1097860218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa46819fc10 con 0x7fa468104e50 2024-06-10T01:06:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.800+0000 7fa45dffb640 1 -- 172.21.15.7:0/1097860218 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa458010690 con 0x7fa468104e50 2024-06-10T01:06:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.800+0000 7fa46f190640 1 -- 172.21.15.7:0/1097860218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4681a0160 con 0x7fa468104e50 2024-06-10T01:06:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.802+0000 7fa45dffb640 1 -- 172.21.15.7:0/1097860218 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa4580046c0 con 0x7fa468104e50 2024-06-10T01:06:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.803+0000 7fa471f7e740 1 Processor -- start 2024-06-10T01:06:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.803+0000 7fa471f7e740 1 -- start start 2024-06-10T01:06:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.803+0000 7fa471f7e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c94998010 0x557c94998430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.803+0000 7fa471f7e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c94998a00 0x557c9499d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.803+0000 7fa471f7e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c9499db70 0x557c949a0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.803+0000 7fa471f7e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557c948fb4f0 con 0x557c94998010 2024-06-10T01:06:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.803+0000 7fa471f7e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557c94899870 con 0x557c94998a00 2024-06-10T01:06:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.803+0000 7fa471f7e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557c948f74b0 con 0x557c9499db70 2024-06-10T01:06:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.803+0000 7fa45dffb640 1 --2- 172.21.15.7:0/1097860218 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa4500a2b80 0x7fa4500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.803+0000 7fa46d72d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c94998010 0x557c94998430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.803+0000 7fa46d72d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c94998010 0x557c94998430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52942/0 (socket says 172.21.15.7:52942) 2024-06-10T01:06:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.803+0000 7fa46d72d640 1 -- 172.21.15.7:0/3165118587 learned_addr learned my addr 172.21.15.7:0/3165118587 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.804+0000 7fa46cf2c640 1 --2- 172.21.15.7:0/3165118587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c94998a00 0x557c9499d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.804+0000 7fa46d72d640 1 --2- 172.21.15.7:0/1097860218 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa4500a2b80 0x7fa4500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.804+0000 7fa45dffb640 1 -- 172.21.15.7:0/1097860218 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fa4580cf900 con 0x7fa468104e50 2024-06-10T01:06:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.804+0000 7fa45ffff640 1 --2- 172.21.15.7:0/3165118587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c9499db70 0x557c949a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.804+0000 7fa46d72d640 1 -- 172.21.15.7:0/3165118587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c9499db70 msgr2=0x557c949a0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.804+0000 7fa46d72d640 1 --2- 172.21.15.7:0/3165118587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c9499db70 0x557c949a0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.804+0000 7fa46d72d640 1 -- 172.21.15.7:0/3165118587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c94998a00 msgr2=0x557c9499d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.804+0000 7fa46d72d640 1 --2- 172.21.15.7:0/3165118587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c94998a00 0x557c9499d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.804+0000 7fa46d72d640 1 -- 172.21.15.7:0/3165118587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c948b59b0 con 0x557c94998010 2024-06-10T01:06:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.804+0000 7fa46cf2c640 1 --2- 172.21.15.7:0/3165118587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c94998a00 0x557c9499d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:03.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.804+0000 7fa46d72d640 1 --2- 172.21.15.7:0/3165118587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c94998010 0x557c94998430 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa460005cc0 tx=0x7fa460031550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=493c9e32b07ae3d1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:03.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.804+0000 7fa46d72d640 1 --2- 172.21.15.7:0/1097860218 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa4500a2b80 0x7fa4500a5040 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa460004af0 tx=0x7fa46004c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:03.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.804+0000 7fa44effd640 1 -- 172.21.15.7:0/3165118587 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460056070 con 0x557c94998010 2024-06-10T01:06:03.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.804+0000 7fa44effd640 1 -- 172.21.15.7:0/3165118587 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa46004d070 con 0x557c94998010 2024-06-10T01:06:03.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.804+0000 7fa44effd640 1 -- 172.21.15.7:0/3165118587 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460004030 con 0x557c94998010 2024-06-10T01:06:03.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.805+0000 7fa471f7e740 1 -- 172.21.15.7:0/3165118587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c94998010 msgr2=0x557c94998430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:03.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.805+0000 7fa471f7e740 1 --2- 172.21.15.7:0/3165118587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c94998010 0x557c94998430 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa460005cc0 tx=0x7fa460031550 comp rx=0 tx=0).stop 2024-06-10T01:06:03.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.805+0000 7fa471f7e740 1 -- 172.21.15.7:0/3165118587 shutdown_connections 2024-06-10T01:06:03.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.805+0000 7fa471f7e740 1 --2- 172.21.15.7:0/3165118587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c9499db70 0x557c949a0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.805+0000 7fa471f7e740 1 --2- 172.21.15.7:0/3165118587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c94998a00 0x557c9499d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.805+0000 7fa471f7e740 1 --2- 172.21.15.7:0/3165118587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c94998010 0x557c94998430 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.805+0000 7fa471f7e740 1 -- 172.21.15.7:0/3165118587 >> 172.21.15.7:0/3165118587 conn(0x557c94993680 msgr2=0x557c94993a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:03.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.805+0000 7fa471f7e740 1 -- 172.21.15.7:0/3165118587 shutdown_connections 2024-06-10T01:06:03.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.805+0000 7fa471f7e740 1 -- 172.21.15.7:0/3165118587 wait complete. 2024-06-10T01:06:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.806+0000 7fa471f7e740 1 -- start start 2024-06-10T01:06:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.806+0000 7fa471f7e740 1 Processor -- start 2024-06-10T01:06:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.806+0000 7fa471f7e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c94998010 0x557c949a2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.806+0000 7fa471f7e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c94998a00 0x557c949a27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.806+0000 7fa471f7e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c9499db70 0x557c949a97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.806+0000 7fa471f7e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557c949a0ac0 con 0x557c94998010 2024-06-10T01:06:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.806+0000 7fa471f7e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557c94884ae0 con 0x557c94998a00 2024-06-10T01:06:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.806+0000 7fa471f7e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557c949a0c40 con 0x557c9499db70 2024-06-10T01:06:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.806+0000 7fa471f7e740 10 client.0 _ll_register_callbacks cb 0x557c948bd9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.806+0000 7fa46cf2c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c94998010 0x557c949a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.806+0000 7fa46cf2c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c94998010 0x557c949a2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52956/0 (socket says 172.21.15.7:52956) 2024-06-10T01:06:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.806+0000 7fa46cf2c640 1 -- 172.21.15.7:0/915657086 learned_addr learned my addr 172.21.15.7:0/915657086 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.806+0000 7fa45ffff640 1 --2- 172.21.15.7:0/915657086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c94998a00 0x557c949a27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.807+0000 7fa46cf2c640 1 -- 172.21.15.7:0/915657086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c9499db70 msgr2=0x557c949a97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.807+0000 7fa46cf2c640 1 --2- 172.21.15.7:0/915657086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c9499db70 0x557c949a97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.807+0000 7fa46cf2c640 1 -- 172.21.15.7:0/915657086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c94998a00 msgr2=0x557c949a27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.807+0000 7fa46cf2c640 1 --2- 172.21.15.7:0/915657086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c94998a00 0x557c949a27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.807+0000 7fa46cf2c640 1 -- 172.21.15.7:0/915657086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c9460f210 con 0x557c94998010 2024-06-10T01:06:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.807+0000 7fa45ffff640 1 --2- 172.21.15.7:0/915657086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c94998a00 0x557c949a27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:06:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.807+0000 7fa46cf2c640 1 --2- 172.21.15.7:0/915657086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c94998010 0x557c949a2260 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa46400b8f0 tx=0x7fa46400bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f619b4a44ae233a0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.807+0000 7fa447fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:06:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.807+0000 7fa447fff640 1 -- 172.21.15.7:0/915657086 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa464003d00 con 0x557c94998010 2024-06-10T01:06:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.807+0000 7fa447fff640 1 -- 172.21.15.7:0/915657086 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fa464011440 con 0x557c94998010 2024-06-10T01:06:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.807+0000 7fa471f7e740 1 -- 172.21.15.7:0/915657086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa46004d070 con 0x557c94998010 2024-06-10T01:06:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.807+0000 7fa447fff640 1 -- 172.21.15.7:0/915657086 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa464003ea0 con 0x557c94998010 2024-06-10T01:06:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.807+0000 7fa471f7e740 10 client.15342 fetch_fsmap learned FSMap version 27 2024-06-10T01:06:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.807+0000 7fa471f7e740 1 -- 172.21.15.7:0/915657086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557c946db620 con 0x557c94998010 2024-06-10T01:06:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.807+0000 7fa447fff640 1 -- 172.21.15.7:0/915657086 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa46004d070 con 0x557c94998010 2024-06-10T01:06:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.807+0000 7fa447fff640 1 -- 172.21.15.7:0/915657086 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa464011640 con 0x557c94998010 2024-06-10T01:06:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.808+0000 7fa471f7e740 10 client.15342 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:06:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.808+0000 7fa471f7e740 10 client.15342 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:06:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.808+0000 7fa471f7e740 20 client.15342 populate_metadata read hostname 'smithi007' 2024-06-10T01:06:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.808+0000 7fa471f7e740 1 --2- 172.21.15.7:0/915657086 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557c949ae1b0 0x557c949b0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.808+0000 7fa471f7e740 4 client.15342 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:06:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.808+0000 7fa471f7e740 1 -- 172.21.15.7:0/915657086 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c9490ef10 con 0x557c949ae1b0 2024-06-10T01:06:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.808+0000 7fa45ffff640 1 --2- 172.21.15.7:0/915657086 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557c949ae1b0 0x557c949b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.808+0000 7fa45ffff640 1 --2- 172.21.15.7:0/915657086 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557c949ae1b0 0x557c949b0640 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a2d80a319fc9f55 server_cookie=3d960980943c84c3 in_seq=0 out_seq=0 2024-06-10T01:06:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.808+0000 7fa447fff640 10 client.15342 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.809+0000 7fa447fff640 1 -- 172.21.15.7:0/915657086 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557c9490ef10 con 0x557c949ae1b0 2024-06-10T01:06:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.809+0000 7fa447fff640 10 client.15342 handle_command_reply: tid=0 2024-06-10T01:06:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.809+0000 7fa471f7e740 1 client.15342 shutdown 2024-06-10T01:06:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.809+0000 7fa471f7e740 1 -- 172.21.15.7:0/915657086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c94998010 msgr2=0x557c949a2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.809+0000 7fa471f7e740 1 --2- 172.21.15.7:0/915657086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c94998010 0x557c949a2260 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa46400b8f0 tx=0x7fa46400bdb0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.809+0000 7fa471f7e740 1 -- 172.21.15.7:0/915657086 shutdown_connections 2024-06-10T01:06:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.809+0000 7fa471f7e740 1 --2- 172.21.15.7:0/915657086 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557c949ae1b0 0x557c949b0640 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.809+0000 7fa471f7e740 1 --2- 172.21.15.7:0/915657086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c9499db70 0x557c949a97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.809+0000 7fa471f7e740 1 --2- 172.21.15.7:0/915657086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c94998a00 0x557c949a27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.809+0000 7fa471f7e740 1 --2- 172.21.15.7:0/915657086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c94998010 0x557c949a2260 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.809+0000 7fa447fff640 0 client.15342 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.809+0000 7fa471f7e740 1 -- 172.21.15.7:0/915657086 >> 172.21.15.7:0/915657086 conn(0x557c94993680 msgr2=0x557c949927e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.809+0000 7fa471f7e740 1 -- 172.21.15.7:0/915657086 shutdown_connections 2024-06-10T01:06:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.809+0000 7fa471f7e740 1 -- 172.21.15.7:0/915657086 wait complete. 2024-06-10T01:06:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.810+0000 7fa471f7e740 20 client.15342 trim_cache size 0 max 16384 2024-06-10T01:06:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.825+0000 7fa471f7e740 1 Processor -- start 2024-06-10T01:06:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.825+0000 7fa471f7e740 1 -- start start 2024-06-10T01:06:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.825+0000 7fa471f7e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c94998010 0x557c94996cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.825+0000 7fa471f7e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c94998a00 0x557c949a97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.825+0000 7fa471f7e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c9499db70 0x557c949a1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.825+0000 7fa471f7e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa4681aad70 con 0x557c94998010 2024-06-10T01:06:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.825+0000 7fa471f7e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa4681a8640 con 0x557c94998a00 2024-06-10T01:06:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.825+0000 7fa471f7e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa4681a6490 con 0x557c9499db70 2024-06-10T01:06:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.825+0000 7fa46cf2c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c94998010 0x557c94996cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.825+0000 7fa46cf2c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c94998010 0x557c94996cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52968/0 (socket says 172.21.15.7:52968) 2024-06-10T01:06:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.825+0000 7fa46cf2c640 1 -- 172.21.15.7:0/4098946586 learned_addr learned my addr 172.21.15.7:0/4098946586 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.825+0000 7fa46d72d640 1 --2- 172.21.15.7:0/4098946586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c9499db70 0x557c949a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.825+0000 7fa46cf2c640 1 -- 172.21.15.7:0/4098946586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c9499db70 msgr2=0x557c949a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.825+0000 7fa46cf2c640 1 --2- 172.21.15.7:0/4098946586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c9499db70 0x557c949a1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.825+0000 7fa46cf2c640 1 -- 172.21.15.7:0/4098946586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c94998a00 msgr2=0x557c949a97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.825+0000 7fa45ffff640 1 --2- 172.21.15.7:0/4098946586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c94998a00 0x557c949a97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.825+0000 7fa46cf2c640 1 --2- 172.21.15.7:0/4098946586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c94998a00 0x557c949a97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.825+0000 7fa46cf2c640 1 -- 172.21.15.7:0/4098946586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c94994350 con 0x557c94998010 2024-06-10T01:06:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.825+0000 7fa45ffff640 1 --2- 172.21.15.7:0/4098946586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c94998a00 0x557c949a97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:06:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.826+0000 7fa46cf2c640 1 --2- 172.21.15.7:0/4098946586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c94998010 0x557c94996cc0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa464011b00 tx=0x7fa464012d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ccb62b26666667 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.826+0000 7fa44e7fc640 1 -- 172.21.15.7:0/4098946586 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46402d030 con 0x557c94998010 2024-06-10T01:06:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.826+0000 7fa44e7fc640 1 -- 172.21.15.7:0/4098946586 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa464004f70 con 0x557c94998010 2024-06-10T01:06:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.826+0000 7fa44e7fc640 1 -- 172.21.15.7:0/4098946586 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa464000e60 con 0x557c94998010 2024-06-10T01:06:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.826+0000 7fa471f7e740 1 -- 172.21.15.7:0/4098946586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c94998010 msgr2=0x557c94996cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.826+0000 7fa471f7e740 1 --2- 172.21.15.7:0/4098946586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c94998010 0x557c94996cc0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa464011b00 tx=0x7fa464012d80 comp rx=0 tx=0).stop 2024-06-10T01:06:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.826+0000 7fa471f7e740 1 -- 172.21.15.7:0/4098946586 shutdown_connections 2024-06-10T01:06:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.826+0000 7fa471f7e740 1 --2- 172.21.15.7:0/4098946586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c9499db70 0x557c949a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.826+0000 7fa471f7e740 1 --2- 172.21.15.7:0/4098946586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c94998a00 0x557c949a97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.826+0000 7fa471f7e740 1 --2- 172.21.15.7:0/4098946586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c94998010 0x557c94996cc0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.826+0000 7fa471f7e740 1 -- 172.21.15.7:0/4098946586 >> 172.21.15.7:0/4098946586 conn(0x557c94993680 msgr2=0x557c949965b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.827+0000 7fa471f7e740 1 -- 172.21.15.7:0/4098946586 shutdown_connections 2024-06-10T01:06:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.827+0000 7fa471f7e740 1 -- 172.21.15.7:0/4098946586 wait complete. 2024-06-10T01:06:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.827+0000 7fa471f7e740 1 -- start start 2024-06-10T01:06:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.827+0000 7fa471f7e740 1 Processor -- start 2024-06-10T01:06:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.827+0000 7fa471f7e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c94998010 0x557c949a66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.827+0000 7fa471f7e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c94998a00 0x557c949a6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.827+0000 7fa471f7e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c9499db70 0x557c94a16f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.827+0000 7fa471f7e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa4681a6b00 con 0x557c94998010 2024-06-10T01:06:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.827+0000 7fa471f7e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa4681a7f70 con 0x557c9499db70 2024-06-10T01:06:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.827+0000 7fa471f7e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa4681a71d0 con 0x557c94998a00 2024-06-10T01:06:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.827+0000 7fa471f7e740 10 client.0 _ll_register_callbacks cb 0x7fa464004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.827+0000 7fa46cf2c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c94998010 0x557c949a66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.827+0000 7fa46cf2c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c94998010 0x557c949a66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52972/0 (socket says 172.21.15.7:52972) 2024-06-10T01:06:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.827+0000 7fa46cf2c640 1 -- 172.21.15.7:0/946356616 learned_addr learned my addr 172.21.15.7:0/946356616 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.828+0000 7fa46d72d640 1 --2- 172.21.15.7:0/946356616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c9499db70 0x557c94a16f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.828+0000 7fa45ffff640 1 --2- 172.21.15.7:0/946356616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c94998a00 0x557c949a6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.828+0000 7fa46cf2c640 1 -- 172.21.15.7:0/946356616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c94998a00 msgr2=0x557c949a6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.828+0000 7fa46cf2c640 1 --2- 172.21.15.7:0/946356616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c94998a00 0x557c949a6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.828+0000 7fa46cf2c640 1 -- 172.21.15.7:0/946356616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c9499db70 msgr2=0x557c94a16f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.828+0000 7fa46cf2c640 1 --2- 172.21.15.7:0/946356616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c9499db70 0x557c94a16f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.828+0000 7fa46cf2c640 1 -- 172.21.15.7:0/946356616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c949934b0 con 0x557c94998010 2024-06-10T01:06:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.828+0000 7fa45ffff640 1 --2- 172.21.15.7:0/946356616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c94998a00 0x557c949a6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:06:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.828+0000 7fa46cf2c640 1 --2- 172.21.15.7:0/946356616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c94998010 0x557c949a66e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa464005930 tx=0x7fa464005350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b36fa5257fcab296 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.828+0000 7fa447fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:06:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.828+0000 7fa447fff640 1 -- 172.21.15.7:0/946356616 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46400b390 con 0x557c94998010 2024-06-10T01:06:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.828+0000 7fa447fff640 1 -- 172.21.15.7:0/946356616 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fa464004610 con 0x557c94998010 2024-06-10T01:06:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.828+0000 7fa471f7e740 1 -- 172.21.15.7:0/946356616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa464004f70 con 0x557c94998010 2024-06-10T01:06:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.828+0000 7fa447fff640 1 -- 172.21.15.7:0/946356616 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa464043050 con 0x557c94998010 2024-06-10T01:06:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.828+0000 7fa471f7e740 10 client.15348 fetch_fsmap learned FSMap version 27 2024-06-10T01:06:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.828+0000 7fa471f7e740 1 -- 172.21.15.7:0/946356616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557c94a18410 con 0x557c94998010 2024-06-10T01:06:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.829+0000 7fa447fff640 1 -- 172.21.15.7:0/946356616 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa464004f70 con 0x557c94998010 2024-06-10T01:06:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.829+0000 7fa447fff640 1 -- 172.21.15.7:0/946356616 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa464029b40 con 0x557c94998010 2024-06-10T01:06:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.829+0000 7fa471f7e740 10 client.15348 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:06:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.829+0000 7fa471f7e740 10 client.15348 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:06:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.829+0000 7fa471f7e740 20 client.15348 populate_metadata read hostname 'smithi007' 2024-06-10T01:06:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.829+0000 7fa471f7e740 1 --2- 172.21.15.7:0/946356616 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557c949ae1b0 0x557c94a1acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.829+0000 7fa471f7e740 4 client.15348 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:06:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.829+0000 7fa471f7e740 1 -- 172.21.15.7:0/946356616 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557c94908c20 con 0x557c949ae1b0 2024-06-10T01:06:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.829+0000 7fa45ffff640 1 --2- 172.21.15.7:0/946356616 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557c949ae1b0 0x557c94a1acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.830+0000 7fa447fff640 10 client.15348 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.830+0000 7fa45ffff640 1 --2- 172.21.15.7:0/946356616 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557c949ae1b0 0x557c94a1acf0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75fd073a15d88609 server_cookie=7d084dfbe4504757 in_seq=0 out_seq=0 2024-06-10T01:06:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.842+0000 7fa447fff640 1 -- 172.21.15.7:0/946356616 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557c94908c20 con 0x557c949ae1b0 2024-06-10T01:06:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.842+0000 7fa447fff640 10 client.15348 handle_command_reply: tid=0 2024-06-10T01:06:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.842+0000 7fa471f7e740 1 client.15348 shutdown 2024-06-10T01:06:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.842+0000 7fa471f7e740 1 -- 172.21.15.7:0/946356616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c94998010 msgr2=0x557c949a66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.842+0000 7fa471f7e740 1 --2- 172.21.15.7:0/946356616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c94998010 0x557c949a66e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa464005930 tx=0x7fa464005350 comp rx=0 tx=0).stop 2024-06-10T01:06:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.842+0000 7fa471f7e740 1 -- 172.21.15.7:0/946356616 shutdown_connections 2024-06-10T01:06:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.842+0000 7fa471f7e740 1 --2- 172.21.15.7:0/946356616 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557c949ae1b0 0x557c94a1acf0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.842+0000 7fa471f7e740 1 --2- 172.21.15.7:0/946356616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c9499db70 0x557c94a16f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.842+0000 7fa447fff640 0 client.15348 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.842+0000 7fa471f7e740 1 --2- 172.21.15.7:0/946356616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c94998a00 0x557c949a6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.842+0000 7fa471f7e740 1 --2- 172.21.15.7:0/946356616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c94998010 0x557c949a66e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.842+0000 7fa471f7e740 1 -- 172.21.15.7:0/946356616 >> 172.21.15.7:0/946356616 conn(0x557c94993680 msgr2=0x557c94991f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.842+0000 7fa471f7e740 1 -- 172.21.15.7:0/946356616 shutdown_connections 2024-06-10T01:06:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.842+0000 7fa471f7e740 1 -- 172.21.15.7:0/946356616 wait complete. 2024-06-10T01:06:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.842+0000 7fa471f7e740 20 client.15348 trim_cache size 0 max 16384 2024-06-10T01:06:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:06:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:06:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:06:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.843+0000 7fa46f190640 1 -- 172.21.15.7:0/1097860218 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa4500a2b80 msgr2=0x7fa4500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.843+0000 7fa46f190640 1 --2- 172.21.15.7:0/1097860218 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa4500a2b80 0x7fa4500a5040 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa460004af0 tx=0x7fa46004c000 comp rx=0 tx=0).stop 2024-06-10T01:06:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.843+0000 7fa46f190640 1 -- 172.21.15.7:0/1097860218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa468104e50 msgr2=0x7fa468198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.843+0000 7fa46f190640 1 --2- 172.21.15.7:0/1097860218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa468104e50 0x7fa468198250 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa45800e9c0 tx=0x7fa45800ee80 comp rx=0 tx=0).stop 2024-06-10T01:06:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.843+0000 7fa46f190640 1 -- 172.21.15.7:0/1097860218 shutdown_connections 2024-06-10T01:06:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.843+0000 7fa46f190640 1 --2- 172.21.15.7:0/1097860218 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa4500a2b80 0x7fa4500a5040 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.843+0000 7fa46f190640 1 --2- 172.21.15.7:0/1097860218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa468107780 0x7fa46819f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.843+0000 7fa46f190640 1 --2- 172.21.15.7:0/1097860218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa468104e50 0x7fa468198250 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.843+0000 7fa46f190640 1 --2- 172.21.15.7:0/1097860218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa468102520 0x7fa468197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.843+0000 7fa46f190640 1 -- 172.21.15.7:0/1097860218 >> 172.21.15.7:0/1097860218 conn(0x7fa4680fbed0 msgr2=0x7fa468102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.843+0000 7fa46f190640 1 -- 172.21.15.7:0/1097860218 shutdown_connections 2024-06-10T01:06:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:03.843+0000 7fa46f190640 1 -- 172.21.15.7:0/1097860218 wait complete. 2024-06-10T01:06:04.034 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:06:04.483 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:04 smithi007 ceph-mon[32398]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T01:06:04.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:04 smithi078 ceph-mon[36683]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T01:06:04.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:04 smithi175 ceph-mon[36636]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T01:06:05.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:05 smithi007 ceph-mon[32398]: pgmap v433: 129 pgs: 129 active+clean; 649 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-10T01:06:05.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:05 smithi078 ceph-mon[36683]: pgmap v433: 129 pgs: 129 active+clean; 649 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-10T01:06:05.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:05 smithi175 ceph-mon[36636]: pgmap v433: 129 pgs: 129 active+clean; 649 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-10T01:06:05.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.853+0000 7f67c1181700 1 -- 172.21.15.7:0/3410250599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67bc108500 msgr2=0x7f67bc1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:05.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.853+0000 7f67c1181700 1 --2- 172.21.15.7:0/3410250599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67bc108500 0x7f67bc1088e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f67ac009990 tx=0x7f67ac009ca0 comp rx=0 tx=0).stop 2024-06-10T01:06:05.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.853+0000 7f67c1181700 1 -- 172.21.15.7:0/3410250599 shutdown_connections 2024-06-10T01:06:05.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.853+0000 7f67c1181700 1 --2- 172.21.15.7:0/3410250599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67bc06ab30 0x7f67bc10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:05.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.853+0000 7f67c1181700 1 --2- 172.21.15.7:0/3410250599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67bc108500 0x7f67bc1088e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:05.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.853+0000 7f67c1181700 1 --2- 172.21.15.7:0/3410250599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67bc06a170 0x7f67bc06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:05.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.853+0000 7f67c1181700 1 -- 172.21.15.7:0/3410250599 >> 172.21.15.7:0/3410250599 conn(0x7f67bc078340 msgr2=0x7f67bc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:05.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.853+0000 7f67c1181700 1 -- 172.21.15.7:0/3410250599 shutdown_connections 2024-06-10T01:06:05.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.853+0000 7f67c1181700 1 -- 172.21.15.7:0/3410250599 wait complete. 2024-06-10T01:06:05.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.854+0000 7f67c1181700 1 Processor -- start 2024-06-10T01:06:05.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.854+0000 7f67c1181700 1 -- start start 2024-06-10T01:06:05.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.854+0000 7f67c1181700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67bc06a170 0x7f67bc19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:05.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.854+0000 7f67c1181700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67bc06ab30 0x7f67bc19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:05.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.854+0000 7f67c1181700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67bc108500 0x7f67bc198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:05.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.854+0000 7f67c1181700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc112760 con 0x7f67bc108500 2024-06-10T01:06:05.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.854+0000 7f67c1181700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc1125e0 con 0x7f67bc06ab30 2024-06-10T01:06:05.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.854+0000 7f67c1181700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc1128e0 con 0x7f67bc06a170 2024-06-10T01:06:05.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.855+0000 7f67bb59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67bc108500 0x7f67bc198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:05.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.855+0000 7f67bb59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67bc108500 0x7f67bc198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52996/0 (socket says 172.21.15.7:52996) 2024-06-10T01:06:05.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.855+0000 7f67bb59e700 1 -- 172.21.15.7:0/938445977 learned_addr learned my addr 172.21.15.7:0/938445977 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:05.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.855+0000 7f67ba59c700 1 --2- 172.21.15.7:0/938445977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67bc06ab30 0x7f67bc19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:05.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.855+0000 7f67bad9d700 1 --2- 172.21.15.7:0/938445977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67bc06a170 0x7f67bc19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:05.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.855+0000 7f67bb59e700 1 -- 172.21.15.7:0/938445977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67bc06a170 msgr2=0x7f67bc19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:05.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.855+0000 7f67bb59e700 1 --2- 172.21.15.7:0/938445977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67bc06a170 0x7f67bc19e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:05.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.855+0000 7f67bb59e700 1 -- 172.21.15.7:0/938445977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67bc06ab30 msgr2=0x7f67bc19ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:05.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.855+0000 7f67bb59e700 1 --2- 172.21.15.7:0/938445977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67bc06ab30 0x7f67bc19ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:05.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.855+0000 7f67bb59e700 1 -- 172.21.15.7:0/938445977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67bc1991c0 con 0x7f67bc108500 2024-06-10T01:06:05.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.855+0000 7f67ba59c700 1 --2- 172.21.15.7:0/938445977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67bc06ab30 0x7f67bc19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:05.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.855+0000 7f67bb59e700 1 --2- 172.21.15.7:0/938445977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67bc108500 0x7f67bc198a60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f67b4007ee0 tx=0x7f67b400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:05.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.855+0000 7f67a3fff700 1 -- 172.21.15.7:0/938445977 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4004bb0 con 0x7f67bc108500 2024-06-10T01:06:05.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.855+0000 7f67a3fff700 1 -- 172.21.15.7:0/938445977 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67b4004d50 con 0x7f67bc108500 2024-06-10T01:06:05.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.855+0000 7f67c1181700 1 -- 172.21.15.7:0/938445977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67bc1994b0 con 0x7f67bc108500 2024-06-10T01:06:05.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.855+0000 7f67a3fff700 1 -- 172.21.15.7:0/938445977 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4005960 con 0x7f67bc108500 2024-06-10T01:06:05.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.855+0000 7f67c1181700 1 -- 172.21.15.7:0/938445977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67bc1a5720 con 0x7f67bc108500 2024-06-10T01:06:05.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.857+0000 7f67c1181700 1 -- 172.21.15.7:0/938445977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67bc10a9e0 con 0x7f67bc108500 2024-06-10T01:06:05.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.858+0000 7f67a3fff700 1 -- 172.21.15.7:0/938445977 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f67b401a020 con 0x7f67bc108500 2024-06-10T01:06:05.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.858+0000 7f67a3fff700 1 --2- 172.21.15.7:0/938445977 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f67a40a2ca0 0x7f67a40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:05.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.859+0000 7f67bad9d700 1 --2- 172.21.15.7:0/938445977 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f67a40a2ca0 0x7f67a40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:05.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.859+0000 7f67a3fff700 1 -- 172.21.15.7:0/938445977 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f67b40ce9b0 con 0x7f67bc108500 2024-06-10T01:06:05.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.859+0000 7f67bad9d700 1 --2- 172.21.15.7:0/938445977 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f67a40a2ca0 0x7f67a40a5160 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f67ac006010 tx=0x7f67ac00b560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:05.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:05.863+0000 7f67a3fff700 1 -- 172.21.15.7:0/938445977 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f67b4010070 con 0x7f67bc108500 2024-06-10T01:06:06.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:06.118+0000 7f67c1181700 1 -- 172.21.15.7:0/938445977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67bc069180 con 0x7f67bc108500 2024-06-10T01:06:06.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:06.119+0000 7f67a3fff700 1 -- 172.21.15.7:0/938445977 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11855 (secure 0 0 0) 0x7f67b409b590 con 0x7f67bc108500 2024-06-10T01:06:06.118 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:06:06.118 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:05:39.175358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/2784910673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2784910673},{"type":"v1","addr":"172.21.15.175:6835","nonce":2784910673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24434,24436,24451]},"id":1}]} 2024-06-10T01:06:06.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:06.121+0000 7f67c1181700 1 -- 172.21.15.7:0/938445977 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f67a40a2ca0 msgr2=0x7f67a40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:06.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:06.121+0000 7f67c1181700 1 --2- 172.21.15.7:0/938445977 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f67a40a2ca0 0x7f67a40a5160 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f67ac006010 tx=0x7f67ac00b560 comp rx=0 tx=0).stop 2024-06-10T01:06:06.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:06.121+0000 7f67c1181700 1 -- 172.21.15.7:0/938445977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67bc108500 msgr2=0x7f67bc198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:06.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:06.121+0000 7f67c1181700 1 --2- 172.21.15.7:0/938445977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67bc108500 0x7f67bc198a60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f67b4007ee0 tx=0x7f67b400d3b0 comp rx=0 tx=0).stop 2024-06-10T01:06:06.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:06.122+0000 7f67c1181700 1 -- 172.21.15.7:0/938445977 shutdown_connections 2024-06-10T01:06:06.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:06.122+0000 7f67c1181700 1 --2- 172.21.15.7:0/938445977 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f67a40a2ca0 0x7f67a40a5160 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:06.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:06.122+0000 7f67c1181700 1 --2- 172.21.15.7:0/938445977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67bc108500 0x7f67bc198a60 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:06.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:06.122+0000 7f67c1181700 1 --2- 172.21.15.7:0/938445977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67bc06a170 0x7f67bc19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:06.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:06.122+0000 7f67c1181700 1 --2- 172.21.15.7:0/938445977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67bc06ab30 0x7f67bc19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:06.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:06.122+0000 7f67c1181700 1 -- 172.21.15.7:0/938445977 >> 172.21.15.7:0/938445977 conn(0x7f67bc078340 msgr2=0x7f67bc0ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:06.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:06.122+0000 7f67c1181700 1 -- 172.21.15.7:0/938445977 shutdown_connections 2024-06-10T01:06:06.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:06.122+0000 7f67c1181700 1 -- 172.21.15.7:0/938445977 wait complete. 2024-06-10T01:06:06.120 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-06-10T01:06:06.464 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:06 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/938445977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:06:06.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:06 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/938445977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:06:06.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:06 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/938445977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:06:06.882 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:06:07.177 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:06:07.552 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:07 smithi007 ceph-mon[32398]: pgmap v434: 129 pgs: 129 active+clean; 649 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-10T01:06:07.553 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:07 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:06:07.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:07 smithi078 ceph-mon[36683]: pgmap v434: 129 pgs: 129 active+clean; 649 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-10T01:06:07.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:07 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:06:07.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:07 smithi175 ceph-mon[36636]: pgmap v434: 129 pgs: 129 active+clean; 649 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-10T01:06:07.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:07 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:06:08.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.747+0000 7f3f350ee700 1 -- 172.21.15.7:0/3852491469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f30101d80 msgr2=0x7f3f3010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:08.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.747+0000 7f3f350ee700 1 --2- 172.21.15.7:0/3852491469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f30101d80 0x7f3f3010e240 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800b0a0 tx=0x7f3f2800b3b0 comp rx=0 tx=0).stop 2024-06-10T01:06:08.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.747+0000 7f3f350ee700 1 -- 172.21.15.7:0/3852491469 shutdown_connections 2024-06-10T01:06:08.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.747+0000 7f3f350ee700 1 --2- 172.21.15.7:0/3852491469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f30101d80 0x7f3f3010e240 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:08.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.747+0000 7f3f350ee700 1 --2- 172.21.15.7:0/3852491469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f30107440 0x7f3f30107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:08.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.747+0000 7f3f350ee700 1 --2- 172.21.15.7:0/3852491469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f301013c0 0x7f3f30101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:08.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.747+0000 7f3f350ee700 1 -- 172.21.15.7:0/3852491469 >> 172.21.15.7:0/3852491469 conn(0x7f3f30077400 msgr2=0x7f3f30077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:08.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.747+0000 7f3f350ee700 1 -- 172.21.15.7:0/3852491469 shutdown_connections 2024-06-10T01:06:08.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.747+0000 7f3f350ee700 1 -- 172.21.15.7:0/3852491469 wait complete. 2024-06-10T01:06:08.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.748+0000 7f3f350ee700 1 Processor -- start 2024-06-10T01:06:08.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.748+0000 7f3f350ee700 1 -- start start 2024-06-10T01:06:08.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.749+0000 7f3f350ee700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f301013c0 0x7f3f3010ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:08.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.749+0000 7f3f350ee700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f30101d80 0x7f3f3010f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:08.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.749+0000 7f3f350ee700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f30107440 0x7f3f3010f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:08.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.749+0000 7f3f350ee700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3f30112800 con 0x7f3f301013c0 2024-06-10T01:06:08.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.749+0000 7f3f350ee700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3f30112680 con 0x7f3f30107440 2024-06-10T01:06:08.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.749+0000 7f3f350ee700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3f30112980 con 0x7f3f30101d80 2024-06-10T01:06:08.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.749+0000 7f3f2ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f301013c0 0x7f3f3010ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:08.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.749+0000 7f3f2ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f301013c0 0x7f3f3010ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53018/0 (socket says 172.21.15.7:53018) 2024-06-10T01:06:08.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.749+0000 7f3f2ed9d700 1 -- 172.21.15.7:0/3430984604 learned_addr learned my addr 172.21.15.7:0/3430984604 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:08.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.749+0000 7f3f2659c700 1 --2- 172.21.15.7:0/3430984604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f30101d80 0x7f3f3010f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:08.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.749+0000 7f3f2f59e700 1 --2- 172.21.15.7:0/3430984604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f30107440 0x7f3f3010f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:08.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.749+0000 7f3f2ed9d700 1 -- 172.21.15.7:0/3430984604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f30101d80 msgr2=0x7f3f3010f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:08.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.749+0000 7f3f2ed9d700 1 --2- 172.21.15.7:0/3430984604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f30101d80 0x7f3f3010f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:08.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.749+0000 7f3f2ed9d700 1 -- 172.21.15.7:0/3430984604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f30107440 msgr2=0x7f3f3010f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:08.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.749+0000 7f3f2ed9d700 1 --2- 172.21.15.7:0/3430984604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f30107440 0x7f3f3010f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:08.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.749+0000 7f3f2ed9d700 1 -- 172.21.15.7:0/3430984604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f301a7120 con 0x7f3f301013c0 2024-06-10T01:06:08.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.749+0000 7f3f2ed9d700 1 --2- 172.21.15.7:0/3430984604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f301013c0 0x7f3f3010ecd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3f2000b530 tx=0x7f3f2000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:08.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.749+0000 7f3f2cd99700 1 -- 172.21.15.7:0/3430984604 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f20015020 con 0x7f3f301013c0 2024-06-10T01:06:08.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.750+0000 7f3f350ee700 1 -- 172.21.15.7:0/3430984604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f301a7410 con 0x7f3f301013c0 2024-06-10T01:06:08.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.750+0000 7f3f2cd99700 1 -- 172.21.15.7:0/3430984604 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f20011d10 con 0x7f3f301013c0 2024-06-10T01:06:08.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.750+0000 7f3f2cd99700 1 -- 172.21.15.7:0/3430984604 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2000f2e0 con 0x7f3f301013c0 2024-06-10T01:06:08.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.750+0000 7f3f350ee700 1 -- 172.21.15.7:0/3430984604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f301a7900 con 0x7f3f301013c0 2024-06-10T01:06:08.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.751+0000 7f3f350ee700 1 -- 172.21.15.7:0/3430984604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f30102ab0 con 0x7f3f301013c0 2024-06-10T01:06:08.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.752+0000 7f3f2cd99700 1 -- 172.21.15.7:0/3430984604 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f3f20018020 con 0x7f3f301013c0 2024-06-10T01:06:08.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.753+0000 7f3f2cd99700 1 --2- 172.21.15.7:0/3430984604 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3f1c0a2c20 0x7f3f1c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:08.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.754+0000 7f3f2659c700 1 --2- 172.21.15.7:0/3430984604 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3f1c0a2c20 0x7f3f1c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:08.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.754+0000 7f3f2cd99700 1 -- 172.21.15.7:0/3430984604 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f3f200cf260 con 0x7f3f301013c0 2024-06-10T01:06:08.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.754+0000 7f3f2659c700 1 --2- 172.21.15.7:0/3430984604 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3f1c0a2c20 0x7f3f1c0a50e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3f30106180 tx=0x7f3f1800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:08.757+0000 7f3f2cd99700 1 -- 172.21.15.7:0/3430984604 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3f2001f020 con 0x7f3f301013c0 2024-06-10T01:06:09.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:09.007+0000 7f3f350ee700 1 -- 172.21.15.7:0/3430984604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f30069180 con 0x7f3f301013c0 2024-06-10T01:06:09.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:09.008+0000 7f3f2cd99700 1 -- 172.21.15.7:0/3430984604 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11855 (secure 0 0 0) 0x7f3f2009bde0 con 0x7f3f301013c0 2024-06-10T01:06:09.004 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:06:09.004 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:05:39.175358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/2784910673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2784910673},{"type":"v1","addr":"172.21.15.175:6835","nonce":2784910673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24434,24436,24451]},"id":1}]} 2024-06-10T01:06:09.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:09.010+0000 7f3f350ee700 1 -- 172.21.15.7:0/3430984604 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3f1c0a2c20 msgr2=0x7f3f1c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:09.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:09.010+0000 7f3f350ee700 1 --2- 172.21.15.7:0/3430984604 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3f1c0a2c20 0x7f3f1c0a50e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3f30106180 tx=0x7f3f1800a380 comp rx=0 tx=0).stop 2024-06-10T01:06:09.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:09.010+0000 7f3f350ee700 1 -- 172.21.15.7:0/3430984604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f301013c0 msgr2=0x7f3f3010ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:09.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:09.010+0000 7f3f350ee700 1 --2- 172.21.15.7:0/3430984604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f301013c0 0x7f3f3010ecd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3f2000b530 tx=0x7f3f2000b840 comp rx=0 tx=0).stop 2024-06-10T01:06:09.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:09.010+0000 7f3f350ee700 1 -- 172.21.15.7:0/3430984604 shutdown_connections 2024-06-10T01:06:09.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:09.010+0000 7f3f350ee700 1 --2- 172.21.15.7:0/3430984604 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3f1c0a2c20 0x7f3f1c0a50e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:09.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:09.010+0000 7f3f350ee700 1 --2- 172.21.15.7:0/3430984604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f30107440 0x7f3f3010f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:09.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:09.010+0000 7f3f350ee700 1 --2- 172.21.15.7:0/3430984604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f301013c0 0x7f3f3010ecd0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:09.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:09.010+0000 7f3f350ee700 1 --2- 172.21.15.7:0/3430984604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f30101d80 0x7f3f3010f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:09.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:09.010+0000 7f3f350ee700 1 -- 172.21.15.7:0/3430984604 >> 172.21.15.7:0/3430984604 conn(0x7f3f30077400 msgr2=0x7f3f300ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:09.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:09.010+0000 7f3f350ee700 1 -- 172.21.15.7:0/3430984604 shutdown_connections 2024-06-10T01:06:09.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:09.010+0000 7f3f350ee700 1 -- 172.21.15.7:0/3430984604 wait complete. 2024-06-10T01:06:09.007 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-06-10T01:06:09.555 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:09 smithi007 ceph-mon[32398]: pgmap v435: 129 pgs: 129 active+clean; 650 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-06-10T01:06:09.555 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:09 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3430984604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:06:09.582 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:09 smithi078 ceph-mon[36683]: pgmap v435: 129 pgs: 129 active+clean; 650 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-06-10T01:06:09.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:09 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3430984604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:06:09.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:09 smithi175 ceph-mon[36636]: pgmap v435: 129 pgs: 129 active+clean; 650 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-06-10T01:06:09.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:09 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3430984604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:06:09.804 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T01:06:10.061 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:06:11.473 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:11 smithi007 ceph-mon[32398]: pgmap v436: 129 pgs: 129 active+clean; 650 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-06-10T01:06:11.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:11 smithi078 ceph-mon[36683]: pgmap v436: 129 pgs: 129 active+clean; 650 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-06-10T01:06:11.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.707+0000 7fe66d686700 1 -- 172.21.15.7:0/836152015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe668101d40 msgr2=0x7fe66810e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:11.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.707+0000 7fe66d686700 1 --2- 172.21.15.7:0/836152015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe668101d40 0x7fe66810e230 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe66000b0a0 tx=0x7fe66000b3b0 comp rx=0 tx=0).stop 2024-06-10T01:06:11.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.707+0000 7fe66d686700 1 -- 172.21.15.7:0/836152015 shutdown_connections 2024-06-10T01:06:11.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.707+0000 7fe66d686700 1 --2- 172.21.15.7:0/836152015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe668101d40 0x7fe66810e230 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:11.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.707+0000 7fe66d686700 1 --2- 172.21.15.7:0/836152015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6681073f0 0x7fe6681077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:11.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.707+0000 7fe66d686700 1 --2- 172.21.15.7:0/836152015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe668101380 0x7fe668101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:11.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.707+0000 7fe66d686700 1 -- 172.21.15.7:0/836152015 >> 172.21.15.7:0/836152015 conn(0x7fe668078300 msgr2=0x7fe668078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:11.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.707+0000 7fe66d686700 1 -- 172.21.15.7:0/836152015 shutdown_connections 2024-06-10T01:06:11.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.707+0000 7fe66d686700 1 -- 172.21.15.7:0/836152015 wait complete. 2024-06-10T01:06:11.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:11 smithi175 ceph-mon[36636]: pgmap v436: 129 pgs: 129 active+clean; 650 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-06-10T01:06:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.708+0000 7fe66d686700 1 Processor -- start 2024-06-10T01:06:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.708+0000 7fe66d686700 1 -- start start 2024-06-10T01:06:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.708+0000 7fe66d686700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe668101380 0x7fe668199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.708+0000 7fe66d686700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe668101d40 0x7fe668199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.709+0000 7fe66d686700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6681073f0 0x7fe6681a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.709+0000 7fe66d686700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe668111660 con 0x7fe668101380 2024-06-10T01:06:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.709+0000 7fe66d686700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe6681114e0 con 0x7fe668101d40 2024-06-10T01:06:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.709+0000 7fe66d686700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe6681117e0 con 0x7fe6681073f0 2024-06-10T01:06:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.709+0000 7fe666ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe668101380 0x7fe668199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.709+0000 7fe666ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe668101380 0x7fe668199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53036/0 (socket says 172.21.15.7:53036) 2024-06-10T01:06:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.709+0000 7fe666ffd700 1 -- 172.21.15.7:0/305045832 learned_addr learned my addr 172.21.15.7:0/305045832 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.709+0000 7fe6677fe700 1 --2- 172.21.15.7:0/305045832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6681073f0 0x7fe6681a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.709+0000 7fe6667fc700 1 --2- 172.21.15.7:0/305045832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe668101d40 0x7fe668199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.709+0000 7fe666ffd700 1 -- 172.21.15.7:0/305045832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6681073f0 msgr2=0x7fe6681a4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.709+0000 7fe666ffd700 1 --2- 172.21.15.7:0/305045832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6681073f0 0x7fe6681a4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.709+0000 7fe666ffd700 1 -- 172.21.15.7:0/305045832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe668101d40 msgr2=0x7fe668199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.709+0000 7fe666ffd700 1 --2- 172.21.15.7:0/305045832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe668101d40 0x7fe668199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.709+0000 7fe666ffd700 1 -- 172.21.15.7:0/305045832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6681a4fc0 con 0x7fe668101380 2024-06-10T01:06:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.709+0000 7fe666ffd700 1 --2- 172.21.15.7:0/305045832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe668101380 0x7fe668199a00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe65800d950 tx=0x7fe65800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.709+0000 7fe64ffff700 1 -- 172.21.15.7:0/305045832 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe658016070 con 0x7fe668101380 2024-06-10T01:06:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.709+0000 7fe64ffff700 1 -- 172.21.15.7:0/305045832 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe65800fd70 con 0x7fe668101380 2024-06-10T01:06:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.709+0000 7fe66d686700 1 -- 172.21.15.7:0/305045832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6681a51f0 con 0x7fe668101380 2024-06-10T01:06:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.709+0000 7fe64ffff700 1 -- 172.21.15.7:0/305045832 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65800e610 con 0x7fe668101380 2024-06-10T01:06:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.709+0000 7fe66d686700 1 -- 172.21.15.7:0/305045832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6681a5750 con 0x7fe668101380 2024-06-10T01:06:11.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.711+0000 7fe66d686700 1 -- 172.21.15.7:0/305045832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe668069180 con 0x7fe668101380 2024-06-10T01:06:11.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.712+0000 7fe64ffff700 1 -- 172.21.15.7:0/305045832 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fe658019070 con 0x7fe668101380 2024-06-10T01:06:11.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.712+0000 7fe64ffff700 1 --2- 172.21.15.7:0/305045832 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe6500a2ca0 0x7fe6500a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:11.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.712+0000 7fe6667fc700 1 --2- 172.21.15.7:0/305045832 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe6500a2ca0 0x7fe6500a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:11.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.712+0000 7fe64ffff700 1 -- 172.21.15.7:0/305045832 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fe6580ced50 con 0x7fe668101380 2024-06-10T01:06:11.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.713+0000 7fe6667fc700 1 --2- 172.21.15.7:0/305045832 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe6500a2ca0 0x7fe6500a5160 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe668078820 tx=0x7fe65c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:11.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.717+0000 7fe64ffff700 1 -- 172.21.15.7:0/305045832 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe6580d2030 con 0x7fe668101380 2024-06-10T01:06:11.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.925+0000 7fe66d686700 1 -- 172.21.15.7:0/305045832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6681a5a70 con 0x7fe668101380 2024-06-10T01:06:11.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.927+0000 7fe64ffff700 1 -- 172.21.15.7:0/305045832 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24615 (secure 0 0 0) 0x7fe65809b930 con 0x7fe668101380 2024-06-10T01:06:11.923 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:06:11.924 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":99,"fsid":"bded58f8-26c3-11ef-bc9b-c7b262605968","created":"2024-06-10T00:53:31.966085+0000","modified":"2024-06-10T01:03:38.227703+0000","last_up_change":"2024-06-10T01:00:45.382473+0000","last_in_change":"2024-06-10T01:00:25.608513+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:56:59.964055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T01:01:42.563611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T01:01:46.175918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T01:01:55.695502+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f7c24c9f-b7ab-4c04-b2b4-120df9751667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6803","nonce":404485726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6805","nonce":404485726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6809","nonce":404485726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6807","nonce":404485726}]},"public_addr":"172.21.15.7:6803/404485726","cluster_addr":"172.21.15.7:6805/404485726","heartbeat_back_addr":"172.21.15.7:6809/404485726","heartbeat_front_addr":"172.21.15.7:6807/404485726","state":["exists","up"]},{"osd":1,"uuid":"9bf70a6b-f31c-4077-b5af-229f17bc2845","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6801","nonce":1856804448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6803","nonce":1856804448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6807","nonce":1856804448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6805","nonce":1856804448}]},"public_addr":"172.21.15.78:6801/1856804448","cluster_addr":"172.21.15.78:6803/1856804448","heartbeat_back_addr":"172.21.15.78:6807/1856804448","heartbeat_front_addr":"172.21.15.78:6805/1856804448","state":["exists","up"]},{"osd":2,"uuid":"62923c97-caf9-4f7e-9c14-9c9918843c78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6801","nonce":918257455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6803","nonce":918257455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6807","nonce":918257455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6805","nonce":918257455}]},"public_addr":"172.21.15.175:6801/918257455","cluster_addr":"172.21.15.175:6803/918257455","heartbeat_back_addr":"172.21.15.175:6807/918257455","heartbeat_front_addr":"172.21.15.175:6805/918257455","state":["exists","up"]},{"osd":3,"uuid":"65242098-e748-48ab-8f0e-bb25b5c62742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6811","nonce":1797404966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6813","nonce":1797404966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6817","nonce":1797404966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6815","nonce":1797404966}]},"public_addr":"172.21.15.7:6811/1797404966","cluster_addr":"172.21.15.7:6813/1797404966","heartbeat_back_addr":"172.21.15.7:6817/1797404966","heartbeat_front_addr":"172.21.15.7:6815/1797404966","state":["exists","up"]},{"osd":4,"uuid":"a7990305-de9e-400e-a432-d7854f73f339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6809","nonce":3767608357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6811","nonce":3767608357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6815","nonce":3767608357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6813","nonce":3767608357}]},"public_addr":"172.21.15.78:6809/3767608357","cluster_addr":"172.21.15.78:6811/3767608357","heartbeat_back_addr":"172.21.15.78:6815/3767608357","heartbeat_front_addr":"172.21.15.78:6813/3767608357","state":["exists","up"]},{"osd":5,"uuid":"bad9a7b3-da5a-48d0-905d-e6a3190317fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6809","nonce":2157664731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6811","nonce":2157664731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6815","nonce":2157664731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6813","nonce":2157664731}]},"public_addr":"172.21.15.175:6809/2157664731","cluster_addr":"172.21.15.175:6811/2157664731","heartbeat_back_addr":"172.21.15.175:6815/2157664731","heartbeat_front_addr":"172.21.15.175:6813/2157664731","state":["exists","up"]},{"osd":6,"uuid":"6c687653-157b-4b96-867b-a2eaa7d8ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6819","nonce":838731756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6821","nonce":838731756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6825","nonce":838731756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6823","nonce":838731756}]},"public_addr":"172.21.15.7:6819/838731756","cluster_addr":"172.21.15.7:6821/838731756","heartbeat_back_addr":"172.21.15.7:6825/838731756","heartbeat_front_addr":"172.21.15.7:6823/838731756","state":["exists","up"]},{"osd":7,"uuid":"db40ddf1-9fd9-4859-b8c4-75f79582ab91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6817","nonce":3511921773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6819","nonce":3511921773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6823","nonce":3511921773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6821","nonce":3511921773}]},"public_addr":"172.21.15.78:6817/3511921773","cluster_addr":"172.21.15.78:6819/3511921773","heartbeat_back_addr":"172.21.15.78:6823/3511921773","heartbeat_front_addr":"172.21.15.78:6821/3511921773","state":["exists","up"]},{"osd":8,"uuid":"0f01cbd9-b193-4c70-a33c-4984167bf422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6817","nonce":1546325211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6819","nonce":1546325211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6823","nonce":1546325211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6821","nonce":1546325211}]},"public_addr":"172.21.15.175:6817/1546325211","cluster_addr":"172.21.15.175:6819/1546325211","heartbeat_back_addr":"172.21.15.175:6823/1546325211","heartbeat_front_addr":"172.21.15.175:6821/1546325211","state":["exists","up"]},{"osd":9,"uuid":"e5480998-f7fb-49cd-9aed-52405c88d830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6827","nonce":1844550813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6829","nonce":1844550813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6833","nonce":1844550813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6831","nonce":1844550813}]},"public_addr":"172.21.15.7:6827/1844550813","cluster_addr":"172.21.15.7:6829/1844550813","heartbeat_back_addr":"172.21.15.7:6833/1844550813","heartbeat_front_addr":"172.21.15.7:6831/1844550813","state":["exists","up"]},{"osd":10,"uuid":"caa82f33-a71c-4907-bf3b-30a7fdfeb71d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6825","nonce":2290824785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6827","nonce":2290824785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6831","nonce":2290824785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6829","nonce":2290824785}]},"public_addr":"172.21.15.78:6825/2290824785","cluster_addr":"172.21.15.78:6827/2290824785","heartbeat_back_addr":"172.21.15.78:6831/2290824785","heartbeat_front_addr":"172.21.15.78:6829/2290824785","state":["exists","up"]},{"osd":11,"uuid":"608cda77-77da-4da2-a7e2-76a773be27c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6825","nonce":1959979550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6827","nonce":1959979550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6831","nonce":1959979550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6829","nonce":1959979550}]},"public_addr":"172.21.15.175:6825/1959979550","cluster_addr":"172.21.15.175:6827/1959979550","heartbeat_back_addr":"172.21.15.175:6831/1959979550","heartbeat_front_addr":"172.21.15.175:6829/1959979550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:08.850203+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:32.820161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:56.633656+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:21.759372+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:46.735874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:10.636541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:36.990099+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:01.767098+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:26.882776+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:52.757595+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:18.383938+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:43.544330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1833397470":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/322543993":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/212510583":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6801/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6800/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/2679436843":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6801/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/1210355277":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6800/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/3610017406":"2024-06-11T00:54:08.102405+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T01:06:11.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.929+0000 7fe66d686700 1 -- 172.21.15.7:0/305045832 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe6500a2ca0 msgr2=0x7fe6500a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:11.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.929+0000 7fe66d686700 1 --2- 172.21.15.7:0/305045832 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe6500a2ca0 0x7fe6500a5160 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe668078820 tx=0x7fe65c00b410 comp rx=0 tx=0).stop 2024-06-10T01:06:11.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.929+0000 7fe66d686700 1 -- 172.21.15.7:0/305045832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe668101380 msgr2=0x7fe668199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:11.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.929+0000 7fe66d686700 1 --2- 172.21.15.7:0/305045832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe668101380 0x7fe668199a00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe65800d950 tx=0x7fe65800dc60 comp rx=0 tx=0).stop 2024-06-10T01:06:11.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.929+0000 7fe66d686700 1 -- 172.21.15.7:0/305045832 shutdown_connections 2024-06-10T01:06:11.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.929+0000 7fe66d686700 1 --2- 172.21.15.7:0/305045832 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe6500a2ca0 0x7fe6500a5160 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:11.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.929+0000 7fe66d686700 1 --2- 172.21.15.7:0/305045832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6681073f0 0x7fe6681a4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:11.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.929+0000 7fe66d686700 1 --2- 172.21.15.7:0/305045832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe668101380 0x7fe668199a00 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:11.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.929+0000 7fe66d686700 1 --2- 172.21.15.7:0/305045832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe668101d40 0x7fe668199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:11.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.929+0000 7fe66d686700 1 -- 172.21.15.7:0/305045832 >> 172.21.15.7:0/305045832 conn(0x7fe668078300 msgr2=0x7fe6680ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:11.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.929+0000 7fe66d686700 1 -- 172.21.15.7:0/305045832 shutdown_connections 2024-06-10T01:06:11.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:11.929+0000 7fe66d686700 1 -- 172.21.15.7:0/305045832 wait complete. 2024-06-10T01:06:12.495 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:12 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/305045832' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:06:12.537 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-06-10T01:06:12.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:12 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/305045832' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:06:12.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:12 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/305045832' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:06:12.789 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:06:13.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:13 smithi007 ceph-mon[32398]: pgmap v437: 129 pgs: 129 active+clean; 650 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-06-10T01:06:13.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:13 smithi078 ceph-mon[36683]: pgmap v437: 129 pgs: 129 active+clean; 650 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-06-10T01:06:13.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:13 smithi175 ceph-mon[36636]: pgmap v437: 129 pgs: 129 active+clean; 650 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-06-10T01:06:15.240 INFO:teuthology.orchestra.run.smithi007.stdout:{"run_dir":"/var/run/ceph"} 2024-06-10T01:06:15.240 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-06-10T01:06:15.241 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 178 2024-06-10T01:06:15.241 DEBUG:tasks.quiescer:Yielding 2024-06-10T01:06:15.241 INFO:teuthology.run_tasks:Running task workunit... 2024-06-10T01:06:15.255 INFO:tasks.workunit:Pulling workunits from ref 7b7a5d375cf4e4afab96b9d4d229b699ae8304bb 2024-06-10T01:06:15.255 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-06-10T01:06:15.255 INFO:tasks.workunit:timeout=3h 2024-06-10T01:06:15.255 INFO:tasks.workunit:cleanup=True 2024-06-10T01:06:15.256 DEBUG:teuthology.orchestra.run.smithi007:> stat -- /home/ubuntu/cephtest/mnt.0 2024-06-10T01:06:15.256 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:06:15.275 INFO:teuthology.orchestra.run.smithi007.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-06-10T01:06:15.275 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-06-10T01:06:15.275 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6bh/107d Inode: 1099511628283 Links: 2 2024-06-10T01:06:15.276 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T01:06:15.276 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-06-10T01:06:15.276 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-06-10 01:03:09.637370185 +0000 2024-06-10T01:06:15.276 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-06-10 01:03:09.637370185 +0000 2024-06-10T01:06:15.276 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-06-10 01:05:38.469062018 +0000 2024-06-10T01:06:15.276 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-06-10 01:03:09.637370185 +0000 2024-06-10T01:06:15.276 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-06-10T01:06:15.276 DEBUG:teuthology.orchestra.run.smithi007:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-06-10T01:06:15.354 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 7b7a5d375cf4e4afab96b9d4d229b699ae8304bb 2024-06-10T01:06:15.410 INFO:tasks.workunit.client.0.smithi007.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-06-10T01:06:15.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:15 smithi007 ceph-mon[32398]: pgmap v438: 129 pgs: 129 active+clean; 650 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-06-10T01:06:15.523 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:06:15.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:15 smithi078 ceph-mon[36683]: pgmap v438: 129 pgs: 129 active+clean; 650 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-06-10T01:06:15.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:15 smithi175 ceph-mon[36636]: pgmap v438: 129 pgs: 129 active+clean; 650 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-06-10T01:06:16.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.944+0000 7f0cf0177640 1 -- 172.21.15.7:0/4019694769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ce81024b0 msgr2=0x7f0ce8108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.944+0000 7f0cf0177640 1 --2- 172.21.15.7:0/4019694769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ce81024b0 0x7f0ce8108d40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0009ca0 tx=0x7f0ce002a8c0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.944+0000 7f0cf0177640 1 -- 172.21.15.7:0/4019694769 shutdown_connections 2024-06-10T01:06:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.944+0000 7f0cf0177640 1 --2- 172.21.15.7:0/4019694769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ce81024b0 0x7f0ce8108d40 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.944+0000 7f0cf0177640 1 --2- 172.21.15.7:0/4019694769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ce8101af0 0x7f0ce8101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.944+0000 7f0cf0177640 1 --2- 172.21.15.7:0/4019694769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ce8100900 0x7f0ce8100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.944+0000 7f0cf0177640 1 -- 172.21.15.7:0/4019694769 >> 172.21.15.7:0/4019694769 conn(0x7f0ce80fbe70 msgr2=0x7f0ce80fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.944+0000 7f0cf0177640 1 -- 172.21.15.7:0/4019694769 shutdown_connections 2024-06-10T01:06:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.944+0000 7f0cf0177640 1 -- 172.21.15.7:0/4019694769 wait complete. 2024-06-10T01:06:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.944+0000 7f0cf0177640 1 Processor -- start 2024-06-10T01:06:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.944+0000 7f0cf0177640 1 -- start start 2024-06-10T01:06:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.945+0000 7f0cf0177640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ce8100900 0x7f0ce8196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.945+0000 7f0cf0177640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ce8101af0 0x7f0ce8197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.945+0000 7f0cf0177640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ce81024b0 0x7f0ce819e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.945+0000 7f0cf0177640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce810c580 con 0x7f0ce81024b0 2024-06-10T01:06:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.945+0000 7f0cf0177640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce810c400 con 0x7f0ce8101af0 2024-06-10T01:06:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.945+0000 7f0cf0177640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce810c700 con 0x7f0ce8100900 2024-06-10T01:06:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.945+0000 7f0cee714640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ce81024b0 0x7f0ce819e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.945+0000 7f0cee714640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ce81024b0 0x7f0ce819e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41566/0 (socket says 172.21.15.7:41566) 2024-06-10T01:06:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.945+0000 7f0cee714640 1 -- 172.21.15.7:0/3220970870 learned_addr learned my addr 172.21.15.7:0/3220970870 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.945+0000 7f0ced712640 1 --2- 172.21.15.7:0/3220970870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ce8101af0 0x7f0ce8197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.945+0000 7f0cedf13640 1 --2- 172.21.15.7:0/3220970870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ce8100900 0x7f0ce8196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.945+0000 7f0cee714640 1 -- 172.21.15.7:0/3220970870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ce8100900 msgr2=0x7f0ce8196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.945+0000 7f0cee714640 1 --2- 172.21.15.7:0/3220970870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ce8100900 0x7f0ce8196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.945+0000 7f0cee714640 1 -- 172.21.15.7:0/3220970870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ce8101af0 msgr2=0x7f0ce8197130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.945+0000 7f0cee714640 1 --2- 172.21.15.7:0/3220970870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ce8101af0 0x7f0ce8197130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.945+0000 7f0cee714640 1 -- 172.21.15.7:0/3220970870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce819e810 con 0x7f0ce81024b0 2024-06-10T01:06:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.945+0000 7f0cee714640 1 --2- 172.21.15.7:0/3220970870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ce81024b0 0x7f0ce819e1a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0005e30 tx=0x7f0ce0031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.946+0000 7f0cdeffd640 1 -- 172.21.15.7:0/3220970870 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0003c30 con 0x7f0ce81024b0 2024-06-10T01:06:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.946+0000 7f0cf0177640 1 -- 172.21.15.7:0/3220970870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ce819eaa0 con 0x7f0ce81024b0 2024-06-10T01:06:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.946+0000 7f0cdeffd640 1 -- 172.21.15.7:0/3220970870 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ce0003dd0 con 0x7f0ce81024b0 2024-06-10T01:06:16.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.946+0000 7f0cdeffd640 1 -- 172.21.15.7:0/3220970870 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce002fde0 con 0x7f0ce81024b0 2024-06-10T01:06:16.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.946+0000 7f0cf0177640 1 -- 172.21.15.7:0/3220970870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ce819f040 con 0x7f0ce81024b0 2024-06-10T01:06:16.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.948+0000 7f0cdeffd640 1 -- 172.21.15.7:0/3220970870 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0ce0004050 con 0x7f0ce81024b0 2024-06-10T01:06:16.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.948+0000 7f0cf2f64740 1 Processor -- start 2024-06-10T01:06:16.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.949+0000 7f0cf2f64740 1 -- start start 2024-06-10T01:06:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.949+0000 7f0cf2f64740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4113a5220 0x55f4113a5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.949+0000 7f0cf2f64740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4113a5c10 0x55f4113aa710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.949+0000 7f0cf2f64740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4113aad80 0x55f4113ad210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.949+0000 7f0cf2f64740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f4113064f0 con 0x55f4113aad80 2024-06-10T01:06:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.949+0000 7f0cf2f64740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f4112a4870 con 0x55f4113a5c10 2024-06-10T01:06:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.949+0000 7f0cf2f64740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f4113024b0 con 0x55f4113a5220 2024-06-10T01:06:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.949+0000 7f0ced712640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4113aad80 0x55f4113ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.949+0000 7f0ced712640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4113aad80 0x55f4113ad210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41570/0 (socket says 172.21.15.7:41570) 2024-06-10T01:06:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.949+0000 7f0cdeffd640 1 --2- 172.21.15.7:0/3220970870 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0cd40a2b80 0x7f0cd40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.949+0000 7f0ced712640 1 -- 172.21.15.7:0/243218639 learned_addr learned my addr 172.21.15.7:0/243218639 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.949+0000 7f0cedf13640 1 --2- 172.21.15.7:0/243218639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4113a5c10 0x55f4113aa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.949+0000 7f0cee714640 1 --2- 172.21.15.7:0/3220970870 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0cd40a2b80 0x7f0cd40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.949+0000 7f0cdeffd640 1 -- 172.21.15.7:0/3220970870 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f0ce00f0550 con 0x7f0ce81024b0 2024-06-10T01:06:16.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.950+0000 7f0ced712640 1 -- 172.21.15.7:0/243218639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4113a5220 msgr2=0x55f4113a5640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:06:16.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.950+0000 7f0ced712640 1 --2- 172.21.15.7:0/243218639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4113a5220 0x55f4113a5640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.950+0000 7f0ced712640 1 -- 172.21.15.7:0/243218639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4113a5c10 msgr2=0x55f4113aa710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.950+0000 7f0ced712640 1 --2- 172.21.15.7:0/243218639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4113a5c10 0x55f4113aa710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.950+0000 7f0ced712640 1 -- 172.21.15.7:0/243218639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4112c09b0 con 0x55f4113aad80 2024-06-10T01:06:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.950+0000 7f0cee714640 1 --2- 172.21.15.7:0/3220970870 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0cd40a2b80 0x7f0cd40a5040 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00edd20 tx=0x7f0ce0040fc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.950+0000 7f0ced712640 1 --2- 172.21.15.7:0/243218639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4113aad80 0x55f4113ad210 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0cd800bc50 tx=0x7f0cd80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d169b0159dcef33 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.950+0000 7f0cd3fff640 1 -- 172.21.15.7:0/243218639 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd801c480 con 0x55f4113aad80 2024-06-10T01:06:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.950+0000 7f0cd3fff640 1 -- 172.21.15.7:0/243218639 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cd801cb00 con 0x55f4113aad80 2024-06-10T01:06:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.950+0000 7f0cd3fff640 1 -- 172.21.15.7:0/243218639 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8011c30 con 0x55f4113aad80 2024-06-10T01:06:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.951+0000 7f0cf2f64740 1 -- 172.21.15.7:0/243218639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4113aad80 msgr2=0x55f4113ad210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.951+0000 7f0cf2f64740 1 --2- 172.21.15.7:0/243218639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4113aad80 0x55f4113ad210 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0cd800bc50 tx=0x7f0cd80107a0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.951+0000 7f0cf2f64740 1 -- 172.21.15.7:0/243218639 shutdown_connections 2024-06-10T01:06:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.951+0000 7f0cf2f64740 1 --2- 172.21.15.7:0/243218639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4113aad80 0x55f4113ad210 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.951+0000 7f0cf2f64740 1 --2- 172.21.15.7:0/243218639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4113a5c10 0x55f4113aa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.951+0000 7f0cf2f64740 1 --2- 172.21.15.7:0/243218639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4113a5220 0x55f4113a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.951+0000 7f0cf2f64740 1 -- 172.21.15.7:0/243218639 >> 172.21.15.7:0/243218639 conn(0x55f4113a0890 msgr2=0x55f4113a0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.951+0000 7f0cf2f64740 1 -- 172.21.15.7:0/243218639 shutdown_connections 2024-06-10T01:06:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.951+0000 7f0cf2f64740 1 -- 172.21.15.7:0/243218639 wait complete. 2024-06-10T01:06:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.951+0000 7f0cf2f64740 1 -- start start 2024-06-10T01:06:16.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.951+0000 7f0cf2f64740 1 Processor -- start 2024-06-10T01:06:16.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.952+0000 7f0cf2f64740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4113a5220 0x55f4113af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:16.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.952+0000 7f0cf2f64740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4113a5c10 0x55f4113afa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:16.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.952+0000 7f0cf2f64740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4113aad80 0x55f4113b6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:16.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.952+0000 7f0cf2f64740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f4113add30 con 0x55f4113aad80 2024-06-10T01:06:16.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.952+0000 7f0cf2f64740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f41128fae0 con 0x55f4113a5c10 2024-06-10T01:06:16.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.952+0000 7f0cf2f64740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f4113adeb0 con 0x55f4113a5220 2024-06-10T01:06:16.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.952+0000 7f0cf2f64740 10 client.0 _ll_register_callbacks cb 0x55f4112c89b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:16.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.952+0000 7f0ced712640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4113aad80 0x55f4113b6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:16.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.952+0000 7f0ced712640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4113aad80 0x55f4113b6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41580/0 (socket says 172.21.15.7:41580) 2024-06-10T01:06:16.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.952+0000 7f0ced712640 1 -- 172.21.15.7:0/1967921037 learned_addr learned my addr 172.21.15.7:0/1967921037 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:16.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.952+0000 7f0cedf13640 1 --2- 172.21.15.7:0/1967921037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4113a5c10 0x55f4113afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:16.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.952+0000 7f0ced712640 1 --2- 172.21.15.7:0/1967921037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4113a5220 0x55f4113af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:16.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.952+0000 7f0ced712640 1 -- 172.21.15.7:0/1967921037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4113a5220 msgr2=0x55f4113af4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:16.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.952+0000 7f0ced712640 1 --2- 172.21.15.7:0/1967921037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4113a5220 0x55f4113af4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.952+0000 7f0ced712640 1 -- 172.21.15.7:0/1967921037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4113a5c10 msgr2=0x55f4113afa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:16.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.952+0000 7f0ced712640 1 --2- 172.21.15.7:0/1967921037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4113a5c10 0x55f4113afa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.952+0000 7f0ced712640 1 -- 172.21.15.7:0/1967921037 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f41101a210 con 0x55f4113aad80 2024-06-10T01:06:16.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.952+0000 7f0ced712640 1 --2- 172.21.15.7:0/1967921037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4113aad80 0x55f4113b6a70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8005940 tx=0x7f0cd800b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46adde04e6e00b8b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:16.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.952+0000 7f0cd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:06:16.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.953+0000 7f0cd17fa640 1 -- 172.21.15.7:0/1967921037 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8011dc0 con 0x55f4113aad80 2024-06-10T01:06:16.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.953+0000 7f0cd17fa640 1 -- 172.21.15.7:0/1967921037 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f0cd8004030 con 0x55f4113aad80 2024-06-10T01:06:16.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.953+0000 7f0cf2f64740 1 -- 172.21.15.7:0/1967921037 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cd801cb00 con 0x55f4113aad80 2024-06-10T01:06:16.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.953+0000 7f0cd17fa640 1 -- 172.21.15.7:0/1967921037 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0cd8034050 con 0x55f4113aad80 2024-06-10T01:06:16.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.953+0000 7f0cf2f64740 10 client.15378 fetch_fsmap learned FSMap version 27 2024-06-10T01:06:16.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.953+0000 7f0cf2f64740 1 -- 172.21.15.7:0/1967921037 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f4110e6620 con 0x55f4113aad80 2024-06-10T01:06:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.953+0000 7f0cd17fa640 1 -- 172.21.15.7:0/1967921037 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cd801cb00 con 0x55f4113aad80 2024-06-10T01:06:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.954+0000 7f0cd17fa640 1 -- 172.21.15.7:0/1967921037 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0cd8003710 con 0x55f4113aad80 2024-06-10T01:06:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.954+0000 7f0cf2f64740 10 client.15378 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:06:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.954+0000 7f0cf2f64740 10 client.15378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:06:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.954+0000 7f0cf2f64740 20 client.15378 populate_metadata read hostname 'smithi007' 2024-06-10T01:06:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.954+0000 7f0cf2f64740 1 --2- 172.21.15.7:0/1967921037 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f41139fd30 0x55f4113bf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.954+0000 7f0cf2f64740 4 client.15378 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:06:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.954+0000 7f0cf2f64740 1 -- 172.21.15.7:0/1967921037 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f411319f10 con 0x55f41139fd30 2024-06-10T01:06:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.954+0000 7f0cedf13640 1 --2- 172.21.15.7:0/1967921037 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f41139fd30 0x55f4113bf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.954+0000 7f0cedf13640 1 --2- 172.21.15.7:0/1967921037 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f41139fd30 0x55f4113bf700 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dac3978f5355e6 server_cookie=1fd83fefdd3aadb1 in_seq=0 out_seq=0 2024-06-10T01:06:16.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.954+0000 7f0cd17fa640 10 client.15378 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:16.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.955+0000 7f0cd17fa640 1 -- 172.21.15.7:0/1967921037 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f411319f10 con 0x55f41139fd30 2024-06-10T01:06:16.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.955+0000 7f0cd17fa640 10 client.15378 handle_command_reply: tid=0 2024-06-10T01:06:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.955+0000 7f0cf2f64740 1 client.15378 shutdown 2024-06-10T01:06:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.955+0000 7f0cf2f64740 1 -- 172.21.15.7:0/1967921037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4113aad80 msgr2=0x55f4113b6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.955+0000 7f0cf2f64740 1 --2- 172.21.15.7:0/1967921037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4113aad80 0x55f4113b6a70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8005940 tx=0x7f0cd800b590 comp rx=0 tx=0).stop 2024-06-10T01:06:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.955+0000 7f0cf2f64740 1 -- 172.21.15.7:0/1967921037 shutdown_connections 2024-06-10T01:06:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.955+0000 7f0cf2f64740 1 --2- 172.21.15.7:0/1967921037 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f41139fd30 0x55f4113bf700 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.955+0000 7f0cf2f64740 1 --2- 172.21.15.7:0/1967921037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4113aad80 0x55f4113b6a70 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.955+0000 7f0cf2f64740 1 --2- 172.21.15.7:0/1967921037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4113a5c10 0x55f4113afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.955+0000 7f0cf2f64740 1 --2- 172.21.15.7:0/1967921037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4113a5220 0x55f4113af4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.955+0000 7f0cf2f64740 1 -- 172.21.15.7:0/1967921037 >> 172.21.15.7:0/1967921037 conn(0x55f4113a0890 msgr2=0x55f41139d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.956+0000 7f0cd17fa640 0 client.15378 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.956+0000 7f0cf2f64740 1 -- 172.21.15.7:0/1967921037 shutdown_connections 2024-06-10T01:06:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.956+0000 7f0cf2f64740 1 -- 172.21.15.7:0/1967921037 wait complete. 2024-06-10T01:06:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.965+0000 7f0cf2f64740 20 client.15378 trim_cache size 0 max 16384 2024-06-10T01:06:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.980+0000 7f0cf2f64740 1 Processor -- start 2024-06-10T01:06:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.980+0000 7f0cf2f64740 1 -- start start 2024-06-10T01:06:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.980+0000 7f0cf2f64740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4113a5220 0x55f4113ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.980+0000 7f0cf2f64740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4113a5c10 0x55f4113b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.980+0000 7f0cf2f64740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4113aad80 0x55f4113a6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.980+0000 7f0cf2f64740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce81aaae0 con 0x55f4113a5220 2024-06-10T01:06:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.980+0000 7f0cf2f64740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce81a83b0 con 0x55f4113aad80 2024-06-10T01:06:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.980+0000 7f0cf2f64740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce81a62b0 con 0x55f4113a5c10 2024-06-10T01:06:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.980+0000 7f0ced712640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4113a5220 0x55f4113ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.980+0000 7f0ced712640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4113a5220 0x55f4113ae2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41596/0 (socket says 172.21.15.7:41596) 2024-06-10T01:06:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.980+0000 7f0ced712640 1 -- 172.21.15.7:0/722140103 learned_addr learned my addr 172.21.15.7:0/722140103 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.980+0000 7f0ced712640 1 -- 172.21.15.7:0/722140103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4113a5c10 msgr2=0x55f4113b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.980+0000 7f0cedf13640 1 --2- 172.21.15.7:0/722140103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4113a5c10 0x55f4113b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.980+0000 7f0ced712640 1 --2- 172.21.15.7:0/722140103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4113a5c10 0x55f4113b6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.980+0000 7f0ced712640 1 -- 172.21.15.7:0/722140103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4113aad80 msgr2=0x55f4113a6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.980+0000 7f0ced712640 1 --2- 172.21.15.7:0/722140103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4113aad80 0x55f4113a6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.980+0000 7f0ced712640 1 -- 172.21.15.7:0/722140103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4113a06c0 con 0x55f4113a5220 2024-06-10T01:06:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.980+0000 7f0cedf13640 1 --2- 172.21.15.7:0/722140103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4113a5c10 0x55f4113b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:06:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.981+0000 7f0ced712640 1 --2- 172.21.15.7:0/722140103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4113a5220 0x55f4113ae2b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80107e0 tx=0x7f0cd800bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce8b3cf7f475ec0a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.981+0000 7f0cd37fe640 1 -- 172.21.15.7:0/722140103 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd803d070 con 0x55f4113a5220 2024-06-10T01:06:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.981+0000 7f0cd37fe640 1 -- 172.21.15.7:0/722140103 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cd8003c80 con 0x55f4113a5220 2024-06-10T01:06:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.981+0000 7f0cd37fe640 1 -- 172.21.15.7:0/722140103 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd801cb00 con 0x55f4113a5220 2024-06-10T01:06:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.981+0000 7f0cf2f64740 1 -- 172.21.15.7:0/722140103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4113a5220 msgr2=0x55f4113ae2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.981+0000 7f0cf2f64740 1 --2- 172.21.15.7:0/722140103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4113a5220 0x55f4113ae2b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80107e0 tx=0x7f0cd800bb50 comp rx=0 tx=0).stop 2024-06-10T01:06:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.982+0000 7f0cf2f64740 1 -- 172.21.15.7:0/722140103 shutdown_connections 2024-06-10T01:06:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.982+0000 7f0cf2f64740 1 --2- 172.21.15.7:0/722140103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4113aad80 0x55f4113a6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.982+0000 7f0cf2f64740 1 --2- 172.21.15.7:0/722140103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4113a5c10 0x55f4113b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.982+0000 7f0cf2f64740 1 --2- 172.21.15.7:0/722140103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4113a5220 0x55f4113ae2b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.982+0000 7f0cf2f64740 1 -- 172.21.15.7:0/722140103 >> 172.21.15.7:0/722140103 conn(0x55f4113a0890 msgr2=0x55f4113aa4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.982+0000 7f0cf2f64740 1 -- 172.21.15.7:0/722140103 shutdown_connections 2024-06-10T01:06:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.982+0000 7f0cf2f64740 1 -- 172.21.15.7:0/722140103 wait complete. 2024-06-10T01:06:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.982+0000 7f0cf2f64740 1 -- start start 2024-06-10T01:06:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.982+0000 7f0cf2f64740 1 Processor -- start 2024-06-10T01:06:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.982+0000 7f0cf2f64740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4113a5220 0x55f4113b6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.982+0000 7f0cf2f64740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4113a5c10 0x55f4113b3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.982+0000 7f0cf2f64740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4113aad80 0x55f4113b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.982+0000 7f0cf2f64740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce81a6870 con 0x55f4113a5c10 2024-06-10T01:06:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.982+0000 7f0cf2f64740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce81a7ce0 con 0x55f4113aad80 2024-06-10T01:06:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.982+0000 7f0cf2f64740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce81a6f40 con 0x55f4113a5220 2024-06-10T01:06:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.982+0000 7f0cf2f64740 10 client.0 _ll_register_callbacks cb 0x7f0ce400b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.982+0000 7f0cedf13640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4113a5c10 0x55f4113b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.982+0000 7f0cedf13640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4113a5c10 0x55f4113b3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41610/0 (socket says 172.21.15.7:41610) 2024-06-10T01:06:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.982+0000 7f0cedf13640 1 -- 172.21.15.7:0/900317221 learned_addr learned my addr 172.21.15.7:0/900317221 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.983+0000 7f0cedf13640 1 -- 172.21.15.7:0/900317221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4113a5220 msgr2=0x55f4113b6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.983+0000 7f0cedf13640 1 --2- 172.21.15.7:0/900317221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4113a5220 0x55f4113b6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.983+0000 7f0cedf13640 1 -- 172.21.15.7:0/900317221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4113aad80 msgr2=0x55f4113b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.983+0000 7f0ced712640 1 --2- 172.21.15.7:0/900317221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4113aad80 0x55f4113b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.983+0000 7f0cedf13640 1 --2- 172.21.15.7:0/900317221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4113aad80 0x55f4113b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.983+0000 7f0cedf13640 1 -- 172.21.15.7:0/900317221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f411425d50 con 0x55f4113a5c10 2024-06-10T01:06:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.983+0000 7f0ced712640 1 --2- 172.21.15.7:0/900317221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4113aad80 0x55f4113b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:06:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.983+0000 7f0cedf13640 1 --2- 172.21.15.7:0/900317221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4113a5c10 0x55f4113b3210 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x55f4113a6680 tx=0x7f0ce40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e04e5ae6f24f11 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.983+0000 7f0cd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:06:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.983+0000 7f0cd17fa640 1 -- 172.21.15.7:0/900317221 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce402b060 con 0x55f4113a5c10 2024-06-10T01:06:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.983+0000 7f0cd17fa640 1 -- 172.21.15.7:0/900317221 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f0ce401e990 con 0x55f4113a5c10 2024-06-10T01:06:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.983+0000 7f0cf2f64740 1 -- 172.21.15.7:0/900317221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cd8003c80 con 0x55f4113a5c10 2024-06-10T01:06:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.983+0000 7f0cd17fa640 1 -- 172.21.15.7:0/900317221 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0ce4032050 con 0x55f4113a5c10 2024-06-10T01:06:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.983+0000 7f0cf2f64740 10 client.15384 fetch_fsmap learned FSMap version 27 2024-06-10T01:06:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.983+0000 7f0cf2f64740 1 -- 172.21.15.7:0/900317221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f411426f00 con 0x55f4113a5c10 2024-06-10T01:06:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.984+0000 7f0cd17fa640 1 -- 172.21.15.7:0/900317221 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cd8003c80 con 0x55f4113a5c10 2024-06-10T01:06:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.984+0000 7f0cd17fa640 1 -- 172.21.15.7:0/900317221 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0ce4010430 con 0x55f4113a5c10 2024-06-10T01:06:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.984+0000 7f0cf2f64740 10 client.15384 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:06:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.984+0000 7f0cf2f64740 10 client.15384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:06:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.984+0000 7f0cf2f64740 20 client.15384 populate_metadata read hostname 'smithi007' 2024-06-10T01:06:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.984+0000 7f0cf2f64740 1 --2- 172.21.15.7:0/900317221 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f411427890 0x55f411429cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.984+0000 7f0cf2f64740 4 client.15384 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:06:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.984+0000 7f0cf2f64740 1 -- 172.21.15.7:0/900317221 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f411313c20 con 0x55f411427890 2024-06-10T01:06:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.984+0000 7f0ced712640 1 --2- 172.21.15.7:0/900317221 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f411427890 0x55f411429cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.984+0000 7f0ced712640 1 --2- 172.21.15.7:0/900317221 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f411427890 0x55f411429cb0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5ec7e291aee3aa6 server_cookie=b185240371ecf6cb in_seq=0 out_seq=0 2024-06-10T01:06:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.984+0000 7f0cd17fa640 10 client.15384 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.992+0000 7f0cd17fa640 1 -- 172.21.15.7:0/900317221 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f411313c20 con 0x55f411427890 2024-06-10T01:06:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.992+0000 7f0cd17fa640 10 client.15384 handle_command_reply: tid=0 2024-06-10T01:06:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.992+0000 7f0cf2f64740 1 client.15384 shutdown 2024-06-10T01:06:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.992+0000 7f0cf2f64740 1 -- 172.21.15.7:0/900317221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4113a5c10 msgr2=0x55f4113b3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.992+0000 7f0cf2f64740 1 --2- 172.21.15.7:0/900317221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4113a5c10 0x55f4113b3210 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x55f4113a6680 tx=0x7f0ce40118d0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.992+0000 7f0cf2f64740 1 -- 172.21.15.7:0/900317221 shutdown_connections 2024-06-10T01:06:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.992+0000 7f0cf2f64740 1 --2- 172.21.15.7:0/900317221 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f411427890 0x55f411429cb0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.992+0000 7f0cf2f64740 1 --2- 172.21.15.7:0/900317221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4113aad80 0x55f4113b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.992+0000 7f0cf2f64740 1 --2- 172.21.15.7:0/900317221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4113a5c10 0x55f4113b3210 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.992+0000 7f0cf2f64740 1 --2- 172.21.15.7:0/900317221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4113a5220 0x55f4113b6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.992+0000 7f0cd17fa640 0 client.15384 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.992+0000 7f0cf2f64740 1 -- 172.21.15.7:0/900317221 >> 172.21.15.7:0/900317221 conn(0x55f4113a0890 msgr2=0x55f4113ae2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.992+0000 7f0cf2f64740 1 -- 172.21.15.7:0/900317221 shutdown_connections 2024-06-10T01:06:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.992+0000 7f0cf2f64740 1 -- 172.21.15.7:0/900317221 wait complete. 2024-06-10T01:06:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.992+0000 7f0cf2f64740 20 client.15384 trim_cache size 0 max 16384 2024-06-10T01:06:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:06:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:06:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:06:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.993+0000 7f0cf0177640 1 -- 172.21.15.7:0/3220970870 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0cd40a2b80 msgr2=0x7f0cd40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.993+0000 7f0cf0177640 1 --2- 172.21.15.7:0/3220970870 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0cd40a2b80 0x7f0cd40a5040 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00edd20 tx=0x7f0ce0040fc0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.993+0000 7f0cf0177640 1 -- 172.21.15.7:0/3220970870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ce81024b0 msgr2=0x7f0ce819e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.993+0000 7f0cf0177640 1 --2- 172.21.15.7:0/3220970870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ce81024b0 0x7f0ce819e1a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0005e30 tx=0x7f0ce0031ee0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.993+0000 7f0cf0177640 1 -- 172.21.15.7:0/3220970870 shutdown_connections 2024-06-10T01:06:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.993+0000 7f0cf0177640 1 --2- 172.21.15.7:0/3220970870 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0cd40a2b80 0x7f0cd40a5040 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.993+0000 7f0cf0177640 1 --2- 172.21.15.7:0/3220970870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ce81024b0 0x7f0ce819e1a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.993+0000 7f0cf0177640 1 --2- 172.21.15.7:0/3220970870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ce8101af0 0x7f0ce8197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.993+0000 7f0cf0177640 1 --2- 172.21.15.7:0/3220970870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ce8100900 0x7f0ce8196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:16.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.993+0000 7f0cf0177640 1 -- 172.21.15.7:0/3220970870 >> 172.21.15.7:0/3220970870 conn(0x7f0ce80fbe70 msgr2=0x7f0ce8104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:16.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.993+0000 7f0cf0177640 1 -- 172.21.15.7:0/3220970870 shutdown_connections 2024-06-10T01:06:16.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:16.993+0000 7f0cf0177640 1 -- 172.21.15.7:0/3220970870 wait complete. 2024-06-10T01:06:17.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.162+0000 7fe21306d700 1 -- 172.21.15.7:0/3839361032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe20c102740 msgr2=0x7fe20c102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:17.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.162+0000 7fe21306d700 1 --2- 172.21.15.7:0/3839361032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe20c102740 0x7fe20c102b20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc0099c0 tx=0x7fe1fc009cd0 comp rx=0 tx=0).stop 2024-06-10T01:06:17.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.162+0000 7fe21306d700 1 -- 172.21.15.7:0/3839361032 shutdown_connections 2024-06-10T01:06:17.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.162+0000 7fe21306d700 1 --2- 172.21.15.7:0/3839361032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe20c107840 0x7fe20c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:17.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.162+0000 7fe21306d700 1 --2- 172.21.15.7:0/3839361032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe20c102740 0x7fe20c102b20 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:17.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.162+0000 7fe21306d700 1 --2- 172.21.15.7:0/3839361032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe20c1030f0 0x7fe20c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:17.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.162+0000 7fe21306d700 1 -- 172.21.15.7:0/3839361032 >> 172.21.15.7:0/3839361032 conn(0x7fe20c0795f0 msgr2=0x7fe20c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:17.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.162+0000 7fe21306d700 1 -- 172.21.15.7:0/3839361032 shutdown_connections 2024-06-10T01:06:17.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.162+0000 7fe21306d700 1 -- 172.21.15.7:0/3839361032 wait complete. 2024-06-10T01:06:17.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.163+0000 7fe21306d700 1 Processor -- start 2024-06-10T01:06:17.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.163+0000 7fe21306d700 1 -- start start 2024-06-10T01:06:17.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.163+0000 7fe21306d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe20c102740 0x7fe20c19e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:17.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.163+0000 7fe21306d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe20c1030f0 0x7fe20c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:17.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.163+0000 7fe21306d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe20c107840 0x7fe20c198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:17.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.163+0000 7fe21306d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe20c112780 con 0x7fe20c102740 2024-06-10T01:06:17.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.163+0000 7fe21306d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe20c112600 con 0x7fe20c1030f0 2024-06-10T01:06:17.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.163+0000 7fe21306d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe20c112900 con 0x7fe20c107840 2024-06-10T01:06:17.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.163+0000 7fe210e09700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe20c102740 0x7fe20c19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:17.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.164+0000 7fe210e09700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe20c102740 0x7fe20c19e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41634/0 (socket says 172.21.15.7:41634) 2024-06-10T01:06:17.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.164+0000 7fe210e09700 1 -- 172.21.15.7:0/2830765759 learned_addr learned my addr 172.21.15.7:0/2830765759 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:17.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.164+0000 7fe203fff700 1 --2- 172.21.15.7:0/2830765759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe20c1030f0 0x7fe20c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:17.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.164+0000 7fe21160a700 1 --2- 172.21.15.7:0/2830765759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe20c107840 0x7fe20c198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:17.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.164+0000 7fe210e09700 1 -- 172.21.15.7:0/2830765759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe20c107840 msgr2=0x7fe20c198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:17.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.164+0000 7fe210e09700 1 --2- 172.21.15.7:0/2830765759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe20c107840 0x7fe20c198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:17.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.164+0000 7fe210e09700 1 -- 172.21.15.7:0/2830765759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe20c1030f0 msgr2=0x7fe20c19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:17.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.164+0000 7fe210e09700 1 --2- 172.21.15.7:0/2830765759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe20c1030f0 0x7fe20c19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:17.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.164+0000 7fe210e09700 1 -- 172.21.15.7:0/2830765759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe20c1991a0 con 0x7fe20c102740 2024-06-10T01:06:17.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.164+0000 7fe210e09700 1 --2- 172.21.15.7:0/2830765759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe20c102740 0x7fe20c19e830 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc0059d0 tx=0x7fe1fc00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:17.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.164+0000 7fe201ffb700 1 -- 172.21.15.7:0/2830765759 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc027070 con 0x7fe20c102740 2024-06-10T01:06:17.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.164+0000 7fe21306d700 1 -- 172.21.15.7:0/2830765759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe20c199430 con 0x7fe20c102740 2024-06-10T01:06:17.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.164+0000 7fe201ffb700 1 -- 172.21.15.7:0/2830765759 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1fc003680 con 0x7fe20c102740 2024-06-10T01:06:17.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.164+0000 7fe201ffb700 1 -- 172.21.15.7:0/2830765759 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc0173f0 con 0x7fe20c102740 2024-06-10T01:06:17.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.165+0000 7fe21306d700 1 -- 172.21.15.7:0/2830765759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe20c1a5700 con 0x7fe20c102740 2024-06-10T01:06:17.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.165+0000 7fe21306d700 1 -- 172.21.15.7:0/2830765759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe20c103bd0 con 0x7fe20c102740 2024-06-10T01:06:17.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.167+0000 7fe201ffb700 1 -- 172.21.15.7:0/2830765759 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fe1fc02a070 con 0x7fe20c102740 2024-06-10T01:06:17.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.167+0000 7fe201ffb700 1 --2- 172.21.15.7:0/2830765759 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe1f40a2c20 0x7fe1f40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:17.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.168+0000 7fe201ffb700 1 -- 172.21.15.7:0/2830765759 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fe1fc0cff30 con 0x7fe20c102740 2024-06-10T01:06:17.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.168+0000 7fe203fff700 1 --2- 172.21.15.7:0/2830765759 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe1f40a2c20 0x7fe1f40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:17.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.168+0000 7fe203fff700 1 --2- 172.21.15.7:0/2830765759 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe1f40a2c20 0x7fe1f40a50e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe20c101fd0 tx=0x7fe20800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:17.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.171+0000 7fe201ffb700 1 -- 172.21.15.7:0/2830765759 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe1fc013070 con 0x7fe20c102740 2024-06-10T01:06:17.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.426+0000 7fe21306d700 1 -- 172.21.15.7:0/2830765759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe20c02dca0 con 0x7fe20c102740 2024-06-10T01:06:17.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.427+0000 7fe201ffb700 1 -- 172.21.15.7:0/2830765759 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11855 (secure 0 0 0) 0x7fe1fc09cba0 con 0x7fe20c102740 2024-06-10T01:06:17.424 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:06:17.424 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:05:39.175358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/2784910673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2784910673},{"type":"v1","addr":"172.21.15.175:6835","nonce":2784910673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24434,24436,24451]},"id":1}]} 2024-06-10T01:06:17.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.429+0000 7fe21306d700 1 -- 172.21.15.7:0/2830765759 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe1f40a2c20 msgr2=0x7fe1f40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:17.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.429+0000 7fe21306d700 1 --2- 172.21.15.7:0/2830765759 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe1f40a2c20 0x7fe1f40a50e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe20c101fd0 tx=0x7fe20800b3c0 comp rx=0 tx=0).stop 2024-06-10T01:06:17.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.429+0000 7fe21306d700 1 -- 172.21.15.7:0/2830765759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe20c102740 msgr2=0x7fe20c19e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:17.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.429+0000 7fe21306d700 1 --2- 172.21.15.7:0/2830765759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe20c102740 0x7fe20c19e830 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc0059d0 tx=0x7fe1fc00fa00 comp rx=0 tx=0).stop 2024-06-10T01:06:17.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.429+0000 7fe21306d700 1 -- 172.21.15.7:0/2830765759 shutdown_connections 2024-06-10T01:06:17.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.429+0000 7fe21306d700 1 --2- 172.21.15.7:0/2830765759 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe1f40a2c20 0x7fe1f40a50e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:17.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.429+0000 7fe21306d700 1 --2- 172.21.15.7:0/2830765759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe20c107840 0x7fe20c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:17.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.429+0000 7fe21306d700 1 --2- 172.21.15.7:0/2830765759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe20c102740 0x7fe20c19e830 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:17.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.429+0000 7fe21306d700 1 --2- 172.21.15.7:0/2830765759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe20c1030f0 0x7fe20c19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:17.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.429+0000 7fe21306d700 1 -- 172.21.15.7:0/2830765759 >> 172.21.15.7:0/2830765759 conn(0x7fe20c0795f0 msgr2=0x7fe20c108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:17.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.430+0000 7fe21306d700 1 -- 172.21.15.7:0/2830765759 shutdown_connections 2024-06-10T01:06:17.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:17.430+0000 7fe21306d700 1 -- 172.21.15.7:0/2830765759 wait complete. 2024-06-10T01:06:17.427 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-06-10T01:06:17.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:17 smithi078 ceph-mon[36683]: pgmap v439: 129 pgs: 129 active+clean; 650 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-06-10T01:06:17.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:17 smithi175 ceph-mon[36636]: pgmap v439: 129 pgs: 129 active+clean; 650 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-06-10T01:06:17.750 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:17 smithi007 ceph-mon[32398]: pgmap v439: 129 pgs: 129 active+clean; 650 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-06-10T01:06:18.082 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-10T01:01:49.029348+0000', 'modified': '2024-06-10T01:05:39.175358+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24451, 'mds_1': 24436, 'mds_2': 24434}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24403': {'gid': 24403, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.175:6833/1281305765', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6832', 'nonce': 1281305765}, {'type': 'v1', 'addr': '172.21.15.175:6833', 'nonce': 1281305765}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24413': {'gid': 24413, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.78:6837/3608622984', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6836', 'nonce': 3608622984}, {'type': 'v1', 'addr': '172.21.15.78:6837', 'nonce': 3608622984}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24424': {'gid': 24424, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.175:6835/2784910673', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6834', 'nonce': 2784910673}, {'type': 'v1', 'addr': '172.21.15.175:6835', 'nonce': 2784910673}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'b', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.78:6839/1946947484', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6838', 'nonce': 1946947484}, {'type': 'v1', 'addr': '172.21.15.78:6839', 'nonce': 1946947484}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'c', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.175:6837/779740919', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6836', 'nonce': 779740919}, {'type': 'v1', 'addr': '172.21.15.175:6837', 'nonce': 779740919}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.175:6839/1983418024', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6838', 'nonce': 1983418024}, {'type': 'v1', 'addr': '172.21.15.175:6839', 'nonce': 1983418024}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24451, 'qdb_cluster': [24434, 24436, 24451]} 2024-06-10T01:06:18.082 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-06-10T01:06:18.082 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-10T01:06:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:06:18.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:18 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2830765759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:06:18.603 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:18 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2830765759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:06:18.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:18 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2830765759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:06:19.487 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:19 smithi007 ceph-mon[32398]: pgmap v440: 129 pgs: 129 active+clean; 659 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-10T01:06:19.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:19 smithi078 ceph-mon[36683]: pgmap v440: 129 pgs: 129 active+clean; 659 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-10T01:06:19.605 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:19 smithi175 ceph-mon[36636]: pgmap v440: 129 pgs: 129 active+clean; 659 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-10T01:06:20.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.036+0000 7f671b506700 1 -- 172.21.15.7:0/3998467310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67140764c0 msgr2=0x7f67140746c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:20.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.036+0000 7f671b506700 1 --2- 172.21.15.7:0/3998467310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67140764c0 0x7f67140746c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6710009990 tx=0x7f6710009ca0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.036+0000 7f671b506700 1 -- 172.21.15.7:0/3998467310 shutdown_connections 2024-06-10T01:06:20.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.036+0000 7f671b506700 1 --2- 172.21.15.7:0/3998467310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6714113120 0x7f6714115510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.036+0000 7f671b506700 1 --2- 172.21.15.7:0/3998467310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67140764c0 0x7f67140746c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.036+0000 7f671b506700 1 --2- 172.21.15.7:0/3998467310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6714074c00 0x7f6714075080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.036+0000 7f671b506700 1 -- 172.21.15.7:0/3998467310 >> 172.21.15.7:0/3998467310 conn(0x7f6714079530 msgr2=0x7f6714079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:20.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.036+0000 7f671b506700 1 -- 172.21.15.7:0/3998467310 shutdown_connections 2024-06-10T01:06:20.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.036+0000 7f671b506700 1 -- 172.21.15.7:0/3998467310 wait complete. 2024-06-10T01:06:20.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.037+0000 7f671b506700 1 Processor -- start 2024-06-10T01:06:20.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.037+0000 7f671b506700 1 -- start start 2024-06-10T01:06:20.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.037+0000 7f671b506700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6714074c00 0x7f671419f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:20.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.037+0000 7f671b506700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67140764c0 0x7f671419f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:20.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.037+0000 7f671b506700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6714113120 0x7f67141a3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:20.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.037+0000 7f671b506700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6714118e70 con 0x7f6714113120 2024-06-10T01:06:20.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.037+0000 7f671b506700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6714118cf0 con 0x7f6714074c00 2024-06-10T01:06:20.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.037+0000 7f671b506700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6714118ff0 con 0x7f67140764c0 2024-06-10T01:06:20.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.037+0000 7f6719aa3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6714113120 0x7f67141a3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:20.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.037+0000 7f6719aa3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6714113120 0x7f67141a3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41658/0 (socket says 172.21.15.7:41658) 2024-06-10T01:06:20.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.037+0000 7f6719aa3700 1 -- 172.21.15.7:0/2177581125 learned_addr learned my addr 172.21.15.7:0/2177581125 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:20.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.037+0000 7f67192a2700 1 --2- 172.21.15.7:0/2177581125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6714074c00 0x7f671419f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:20.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.037+0000 7f6718aa1700 1 --2- 172.21.15.7:0/2177581125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67140764c0 0x7f671419f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:20.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.037+0000 7f6719aa3700 1 -- 172.21.15.7:0/2177581125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67140764c0 msgr2=0x7f671419f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:20.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.037+0000 7f6719aa3700 1 --2- 172.21.15.7:0/2177581125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67140764c0 0x7f671419f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.037+0000 7f6719aa3700 1 -- 172.21.15.7:0/2177581125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6714074c00 msgr2=0x7f671419f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:20.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.037+0000 7f6719aa3700 1 --2- 172.21.15.7:0/2177581125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6714074c00 0x7f671419f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.037+0000 7f6719aa3700 1 -- 172.21.15.7:0/2177581125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67141a4370 con 0x7f6714113120 2024-06-10T01:06:20.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.037+0000 7f6718aa1700 1 --2- 172.21.15.7:0/2177581125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67140764c0 0x7f671419f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:06:20.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.038+0000 7f6719aa3700 1 --2- 172.21.15.7:0/2177581125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6714113120 0x7f67141a3c30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f670c00b9e0 tx=0x7f670c00bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:20.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.038+0000 7f67067fc700 1 -- 172.21.15.7:0/2177581125 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c019070 con 0x7f6714113120 2024-06-10T01:06:20.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.038+0000 7f67067fc700 1 -- 172.21.15.7:0/2177581125 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f670c00ce10 con 0x7f6714113120 2024-06-10T01:06:20.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.038+0000 7f67067fc700 1 -- 172.21.15.7:0/2177581125 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c012360 con 0x7f6714113120 2024-06-10T01:06:20.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.038+0000 7f671b506700 1 -- 172.21.15.7:0/2177581125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67141a4660 con 0x7f6714113120 2024-06-10T01:06:20.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.038+0000 7f671b506700 1 -- 172.21.15.7:0/2177581125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67141abee0 con 0x7f6714113120 2024-06-10T01:06:20.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.040+0000 7f67067fc700 1 -- 172.21.15.7:0/2177581125 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f670c01c070 con 0x7f6714113120 2024-06-10T01:06:20.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.041+0000 7f67067fc700 1 --2- 172.21.15.7:0/2177581125 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f67000a2c20 0x7f67000a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:20.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.041+0000 7f67192a2700 1 --2- 172.21.15.7:0/2177581125 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f67000a2c20 0x7f67000a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:20.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.041+0000 7f67067fc700 1 -- 172.21.15.7:0/2177581125 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f670c0ce730 con 0x7f6714113120 2024-06-10T01:06:20.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.041+0000 7f67192a2700 1 --2- 172.21.15.7:0/2177581125 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f67000a2c20 0x7f67000a50e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f671000b5c0 tx=0x7f6710005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:20.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.042+0000 7f6725d89b80 1 Processor -- start 2024-06-10T01:06:20.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.042+0000 7f6725d89b80 1 -- start start 2024-06-10T01:06:20.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.042+0000 7f6725d89b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575794cbf10 0x5575794cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:20.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.042+0000 7f6725d89b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575794cc830 0x5575794c6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:20.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.042+0000 7f6725d89b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575794c6750 0x5575794c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:20.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.042+0000 7f6725d89b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5575793b8f10 con 0x5575794cc830 2024-06-10T01:06:20.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.042+0000 7f6725d89b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557579417800 con 0x5575794c6750 2024-06-10T01:06:20.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.042+0000 7f6725d89b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55757937f410 con 0x5575794cbf10 2024-06-10T01:06:20.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.042+0000 7f6718aa1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575794cc830 0x5575794c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:20.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.042+0000 7f6718aa1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575794cc830 0x5575794c6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41660/0 (socket says 172.21.15.7:41660) 2024-06-10T01:06:20.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.042+0000 7f6718aa1700 1 -- 172.21.15.7:0/4113128593 learned_addr learned my addr 172.21.15.7:0/4113128593 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:20.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.042+0000 7f6719aa3700 1 --2- 172.21.15.7:0/4113128593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575794c6750 0x5575794c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:20.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.042+0000 7f67192a2700 1 --2- 172.21.15.7:0/4113128593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575794cbf10 0x5575794cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:20.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.042+0000 7f6718aa1700 1 -- 172.21.15.7:0/4113128593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575794cbf10 msgr2=0x5575794cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:20.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.042+0000 7f6718aa1700 1 --2- 172.21.15.7:0/4113128593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575794cbf10 0x5575794cc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.042+0000 7f6718aa1700 1 -- 172.21.15.7:0/4113128593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575794c6750 msgr2=0x5575794c6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:20.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.042+0000 7f6718aa1700 1 --2- 172.21.15.7:0/4113128593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575794c6750 0x5575794c6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.042+0000 7f6718aa1700 1 -- 172.21.15.7:0/4113128593 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557579366880 con 0x5575794cc830 2024-06-10T01:06:20.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.043+0000 7f6718aa1700 1 --2- 172.21.15.7:0/4113128593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575794cc830 0x5575794c6210 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6708007870 tx=0x7f6708007b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d277890a81b164e9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:20.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.043+0000 7f66ff7fe700 1 -- 172.21.15.7:0/4113128593 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67080125f0 con 0x5575794cc830 2024-06-10T01:06:20.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.043+0000 7f66ff7fe700 1 -- 172.21.15.7:0/4113128593 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6708018070 con 0x5575794cc830 2024-06-10T01:06:20.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.043+0000 7f66ff7fe700 1 -- 172.21.15.7:0/4113128593 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6708013be0 con 0x5575794cc830 2024-06-10T01:06:20.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.044+0000 7f6725d89b80 1 -- 172.21.15.7:0/4113128593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575794cc830 msgr2=0x5575794c6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:20.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.044+0000 7f6725d89b80 1 --2- 172.21.15.7:0/4113128593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575794cc830 0x5575794c6210 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6708007870 tx=0x7f6708007b80 comp rx=0 tx=0).stop 2024-06-10T01:06:20.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.044+0000 7f6725d89b80 1 -- 172.21.15.7:0/4113128593 shutdown_connections 2024-06-10T01:06:20.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.044+0000 7f6725d89b80 1 --2- 172.21.15.7:0/4113128593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575794c6750 0x5575794c6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.044+0000 7f6725d89b80 1 --2- 172.21.15.7:0/4113128593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575794cbf10 0x5575794cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.044+0000 7f6725d89b80 1 --2- 172.21.15.7:0/4113128593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575794cc830 0x5575794c6210 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.044+0000 7f6725d89b80 1 -- 172.21.15.7:0/4113128593 >> 172.21.15.7:0/4113128593 conn(0x5575793f3030 msgr2=0x5575794c11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:20.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.044+0000 7f6725d89b80 1 -- 172.21.15.7:0/4113128593 shutdown_connections 2024-06-10T01:06:20.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.044+0000 7f6725d89b80 1 -- 172.21.15.7:0/4113128593 wait complete. 2024-06-10T01:06:20.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.044+0000 7f6725d89b80 1 -- start start 2024-06-10T01:06:20.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.044+0000 7f6725d89b80 1 Processor -- start 2024-06-10T01:06:20.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.045+0000 7f6725d89b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575794c6750 0x5575794d25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:20.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.045+0000 7f6725d89b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575794cbf10 0x5575794cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:20.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.045+0000 7f6725d89b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575794cc830 0x5575794cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:20.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.045+0000 7f6725d89b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5575793b7f70 con 0x5575794cbf10 2024-06-10T01:06:20.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.045+0000 7f6725d89b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5575794137c0 con 0x5575794cc830 2024-06-10T01:06:20.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.045+0000 7f6725d89b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5575794d3210 con 0x5575794c6750 2024-06-10T01:06:20.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.045+0000 7f6725d89b80 10 client.0 _ll_register_callbacks cb 0x5575793ad560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:20.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.045+0000 7f6718aa1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575794cbf10 0x5575794cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:20.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.045+0000 7f6718aa1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575794cbf10 0x5575794cf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41672/0 (socket says 172.21.15.7:41672) 2024-06-10T01:06:20.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.045+0000 7f6718aa1700 1 -- 172.21.15.7:0/9502786 learned_addr learned my addr 172.21.15.7:0/9502786 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:20.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.045+0000 7f6719aa3700 1 --2- 172.21.15.7:0/9502786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575794cc830 0x5575794cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:20.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.045+0000 7f67192a2700 1 --2- 172.21.15.7:0/9502786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575794c6750 0x5575794d25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:20.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.045+0000 7f6718aa1700 1 -- 172.21.15.7:0/9502786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575794c6750 msgr2=0x5575794d25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:20.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.045+0000 7f6718aa1700 1 --2- 172.21.15.7:0/9502786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575794c6750 0x5575794d25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.045+0000 7f6718aa1700 1 -- 172.21.15.7:0/9502786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575794cc830 msgr2=0x5575794cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:20.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.045+0000 7f6718aa1700 1 --2- 172.21.15.7:0/9502786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575794cc830 0x5575794cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.045+0000 7f6718aa1700 1 -- 172.21.15.7:0/9502786 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575793dd700 con 0x5575794cbf10 2024-06-10T01:06:20.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.045+0000 7f67192a2700 1 --2- 172.21.15.7:0/9502786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575794c6750 0x5575794d25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:06:20.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.045+0000 7f6718aa1700 1 --2- 172.21.15.7:0/9502786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575794cbf10 0x5575794cf8c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f67080048a0 tx=0x7f6708013b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8cb26a33fa2e67 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:20.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.046+0000 7f66fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:06:20.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.046+0000 7f66fcff9700 1 -- 172.21.15.7:0/9502786 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67080074c0 con 0x5575794cbf10 2024-06-10T01:06:20.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.046+0000 7f66fcff9700 1 -- 172.21.15.7:0/9502786 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f6708012800 con 0x5575794cbf10 2024-06-10T01:06:20.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.046+0000 7f6725d89b80 1 -- 172.21.15.7:0/9502786 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6708018070 con 0x5575794cbf10 2024-06-10T01:06:20.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.046+0000 7f66fcff9700 1 -- 172.21.15.7:0/9502786 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6708030050 con 0x5575794cbf10 2024-06-10T01:06:20.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.046+0000 7f6725d89b80 10 client.15402 fetch_fsmap learned FSMap version 27 2024-06-10T01:06:20.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.046+0000 7f6725d89b80 1 -- 172.21.15.7:0/9502786 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5575794dd9f0 con 0x5575794cbf10 2024-06-10T01:06:20.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.046+0000 7f66fcff9700 1 -- 172.21.15.7:0/9502786 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6708018070 con 0x5575794cbf10 2024-06-10T01:06:20.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.046+0000 7f66fcff9700 1 -- 172.21.15.7:0/9502786 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f67080279f0 con 0x5575794cbf10 2024-06-10T01:06:20.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.046+0000 7f6725d89b80 10 client.15402 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:06:20.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.046+0000 7f6725d89b80 10 client.15402 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-10T01:06:20.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.046+0000 7f6725d89b80 20 client.15402 populate_metadata read hostname 'smithi007' 2024-06-10T01:06:20.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.046+0000 7f6725d89b80 1 --2- 172.21.15.7:0/9502786 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5575794dfc30 0x5575794e10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:20.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.047+0000 7f6725d89b80 4 client.15402 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:06:20.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.047+0000 7f6725d89b80 1 -- 172.21.15.7:0/9502786 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557579424d00 con 0x5575794dfc30 2024-06-10T01:06:20.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.047+0000 7f67192a2700 1 --2- 172.21.15.7:0/9502786 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5575794dfc30 0x5575794e10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:20.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.047+0000 7f66fcff9700 10 client.15402 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:06:20.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.047+0000 7f67192a2700 1 --2- 172.21.15.7:0/9502786 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5575794dfc30 0x5575794e10a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e07b4412777a9b99 server_cookie=c3832cbba3d239bc in_seq=0 out_seq=0 2024-06-10T01:06:20.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.048+0000 7f66fcff9700 1 -- 172.21.15.7:0/9502786 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557579424d00 con 0x5575794dfc30 2024-06-10T01:06:20.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.048+0000 7f66fcff9700 10 client.15402 handle_command_reply: tid=0 2024-06-10T01:06:20.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.048+0000 7f6725d89b80 1 client.15402 shutdown 2024-06-10T01:06:20.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.048+0000 7f6725d89b80 1 -- 172.21.15.7:0/9502786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575794cbf10 msgr2=0x5575794cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:20.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.048+0000 7f6725d89b80 1 --2- 172.21.15.7:0/9502786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575794cbf10 0x5575794cf8c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f67080048a0 tx=0x7f6708013b80 comp rx=0 tx=0).stop 2024-06-10T01:06:20.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.048+0000 7f6725d89b80 1 -- 172.21.15.7:0/9502786 shutdown_connections 2024-06-10T01:06:20.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.048+0000 7f6725d89b80 1 --2- 172.21.15.7:0/9502786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575794cc830 0x5575794cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.048+0000 7f6725d89b80 1 --2- 172.21.15.7:0/9502786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575794c6750 0x5575794d25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.048+0000 7f6725d89b80 1 --2- 172.21.15.7:0/9502786 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5575794dfc30 0x5575794e10a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.048+0000 7f6725d89b80 1 --2- 172.21.15.7:0/9502786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575794cbf10 0x5575794cf8c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.048+0000 7f6725d89b80 1 -- 172.21.15.7:0/9502786 >> 172.21.15.7:0/9502786 conn(0x5575793f3030 msgr2=0x5575794c06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:20.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.048+0000 7f66fcff9700 0 client.15402 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:06:20.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.048+0000 7f6725d89b80 1 -- 172.21.15.7:0/9502786 shutdown_connections 2024-06-10T01:06:20.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.048+0000 7f6725d89b80 1 -- 172.21.15.7:0/9502786 wait complete. 2024-06-10T01:06:20.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.048+0000 7f6725d89b80 20 client.15402 trim_cache size 0 max 16384 2024-06-10T01:06:20.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.070+0000 7f6725d89b80 1 Processor -- start 2024-06-10T01:06:20.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.070+0000 7f6725d89b80 1 -- start start 2024-06-10T01:06:20.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.070+0000 7f6725d89b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575794c6750 0x5575794c0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:20.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.070+0000 7f6725d89b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575794cbf10 0x5575794c1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:20.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.070+0000 7f6725d89b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575794cc830 0x5575794ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:20.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.070+0000 7f6725d89b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5575794df3e0 con 0x5575794cbf10 2024-06-10T01:06:20.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.070+0000 7f6725d89b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5575794dfab0 con 0x5575794cc830 2024-06-10T01:06:20.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.070+0000 7f6725d89b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5575794e0180 con 0x5575794c6750 2024-06-10T01:06:20.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.070+0000 7f6718aa1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575794cbf10 0x5575794c1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:20.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.070+0000 7f6718aa1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575794cbf10 0x5575794c1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41684/0 (socket says 172.21.15.7:41684) 2024-06-10T01:06:20.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.070+0000 7f6718aa1700 1 -- 172.21.15.7:0/2359667794 learned_addr learned my addr 172.21.15.7:0/2359667794 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:20.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.070+0000 7f67192a2700 1 --2- 172.21.15.7:0/2359667794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575794c6750 0x5575794c0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:20.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.070+0000 7f6718aa1700 1 -- 172.21.15.7:0/2359667794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575794c6750 msgr2=0x5575794c0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:20.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.070+0000 7f6718aa1700 1 --2- 172.21.15.7:0/2359667794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575794c6750 0x5575794c0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.070+0000 7f6718aa1700 1 -- 172.21.15.7:0/2359667794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575794cc830 msgr2=0x5575794ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:20.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.070+0000 7f6719aa3700 1 --2- 172.21.15.7:0/2359667794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575794cc830 0x5575794ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:20.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.070+0000 7f6718aa1700 1 --2- 172.21.15.7:0/2359667794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575794cc830 0x5575794ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.070+0000 7f6718aa1700 1 -- 172.21.15.7:0/2359667794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575794df8d0 con 0x5575794cbf10 2024-06-10T01:06:20.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.071+0000 7f67192a2700 1 --2- 172.21.15.7:0/2359667794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575794c6750 0x5575794c0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:06:20.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.071+0000 7f6719aa3700 1 --2- 172.21.15.7:0/2359667794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575794cc830 0x5575794ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:06:20.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.071+0000 7f6718aa1700 1 --2- 172.21.15.7:0/2359667794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575794cbf10 0x5575794c1370 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6708027e40 tx=0x7f6708006ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a797fcda38555e1d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:20.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.071+0000 7f66feffd700 1 -- 172.21.15.7:0/2359667794 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6708013e60 con 0x5575794cbf10 2024-06-10T01:06:20.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.071+0000 7f66feffd700 1 -- 172.21.15.7:0/2359667794 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6708018070 con 0x5575794cbf10 2024-06-10T01:06:20.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.071+0000 7f66feffd700 1 -- 172.21.15.7:0/2359667794 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670801d400 con 0x5575794cbf10 2024-06-10T01:06:20.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.071+0000 7f6725d89b80 1 -- 172.21.15.7:0/2359667794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575794cbf10 msgr2=0x5575794c1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:20.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.071+0000 7f6725d89b80 1 --2- 172.21.15.7:0/2359667794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575794cbf10 0x5575794c1370 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6708027e40 tx=0x7f6708006ac0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.072+0000 7f6725d89b80 1 -- 172.21.15.7:0/2359667794 shutdown_connections 2024-06-10T01:06:20.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.072+0000 7f6725d89b80 1 --2- 172.21.15.7:0/2359667794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575794cc830 0x5575794ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.072+0000 7f6725d89b80 1 --2- 172.21.15.7:0/2359667794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575794c6750 0x5575794c0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.072+0000 7f6725d89b80 1 --2- 172.21.15.7:0/2359667794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575794cbf10 0x5575794c1370 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.072+0000 7f6725d89b80 1 -- 172.21.15.7:0/2359667794 >> 172.21.15.7:0/2359667794 conn(0x5575793f3030 msgr2=0x5575794ce1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:20.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.072+0000 7f6725d89b80 1 -- 172.21.15.7:0/2359667794 shutdown_connections 2024-06-10T01:06:20.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.072+0000 7f6725d89b80 1 -- 172.21.15.7:0/2359667794 wait complete. 2024-06-10T01:06:20.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.072+0000 7f6725d89b80 1 -- start start 2024-06-10T01:06:20.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.072+0000 7f6725d89b80 1 Processor -- start 2024-06-10T01:06:20.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.072+0000 7f6725d89b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575794c6750 0x5575794c0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:20.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.072+0000 7f6725d89b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575794cbf10 0x5575794c0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:20.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.072+0000 7f6725d89b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575794cc830 0x5575794e9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.072+0000 7f6725d89b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5575794e0f20 con 0x5575794c6750 2024-06-10T01:06:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.072+0000 7f6725d89b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5575794e0850 con 0x5575794cbf10 2024-06-10T01:06:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.072+0000 7f6725d89b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5575794e15f0 con 0x5575794cc830 2024-06-10T01:06:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.073+0000 7f6725d89b80 10 client.0 _ll_register_callbacks cb 0x5575793ad560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.073+0000 7f67192a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575794c6750 0x5575794c0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.073+0000 7f67192a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575794c6750 0x5575794c0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41690/0 (socket says 172.21.15.7:41690) 2024-06-10T01:06:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.073+0000 7f67192a2700 1 -- 172.21.15.7:0/2630627454 learned_addr learned my addr 172.21.15.7:0/2630627454 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.073+0000 7f6719aa3700 1 --2- 172.21.15.7:0/2630627454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575794cc830 0x5575794e9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.073+0000 7f6718aa1700 1 --2- 172.21.15.7:0/2630627454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575794cbf10 0x5575794c0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.073+0000 7f67192a2700 1 -- 172.21.15.7:0/2630627454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575794cc830 msgr2=0x5575794e9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.073+0000 7f67192a2700 1 --2- 172.21.15.7:0/2630627454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575794cc830 0x5575794e9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.073+0000 7f67192a2700 1 -- 172.21.15.7:0/2630627454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575794cbf10 msgr2=0x5575794c0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.073+0000 7f67192a2700 1 --2- 172.21.15.7:0/2630627454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575794cbf10 0x5575794c0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.073+0000 7f67192a2700 1 -- 172.21.15.7:0/2630627454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575794e9a50 con 0x5575794c6750 2024-06-10T01:06:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.073+0000 7f6718aa1700 1 --2- 172.21.15.7:0/2630627454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575794cbf10 0x5575794c0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:06:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.073+0000 7f6719aa3700 1 --2- 172.21.15.7:0/2630627454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575794cc830 0x5575794e9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:06:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.073+0000 7f67192a2700 1 --2- 172.21.15.7:0/2630627454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575794c6750 0x5575794c0000 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6710009990 tx=0x7f6710009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3fd31b417a23fd8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.073+0000 7f66fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:06:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.073+0000 7f66fcff9700 1 -- 172.21.15.7:0/2630627454 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671000fe60 con 0x5575794c6750 2024-06-10T01:06:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.073+0000 7f66fcff9700 1 -- 172.21.15.7:0/2630627454 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f6710032450 con 0x5575794c6750 2024-06-10T01:06:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.073+0000 7f6725d89b80 1 -- 172.21.15.7:0/2630627454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6708018070 con 0x5575794c6750 2024-06-10T01:06:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.074+0000 7f66fcff9700 1 -- 172.21.15.7:0/2630627454 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6710043050 con 0x5575794c6750 2024-06-10T01:06:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.074+0000 7f6725d89b80 10 client.15408 fetch_fsmap learned FSMap version 27 2024-06-10T01:06:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.074+0000 7f6725d89b80 1 -- 172.21.15.7:0/2630627454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5575794eac90 con 0x5575794c6750 2024-06-10T01:06:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.074+0000 7f66fcff9700 1 -- 172.21.15.7:0/2630627454 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6708018070 con 0x5575794c6750 2024-06-10T01:06:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.074+0000 7f66fcff9700 1 -- 172.21.15.7:0/2630627454 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6710024880 con 0x5575794c6750 2024-06-10T01:06:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.074+0000 7f6725d89b80 10 client.15408 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:06:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.074+0000 7f6725d89b80 10 client.15408 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-10T01:06:20.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.074+0000 7f6725d89b80 20 client.15408 populate_metadata read hostname 'smithi007' 2024-06-10T01:06:20.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.074+0000 7f6725d89b80 1 --2- 172.21.15.7:0/2630627454 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55757954fa70 0x55757954e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:20.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.074+0000 7f6725d89b80 4 client.15408 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T01:06:20.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.074+0000 7f6725d89b80 1 -- 172.21.15.7:0/2630627454 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "status"}) -- 0x55757941f280 con 0x55757954fa70 2024-06-10T01:06:20.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.074+0000 7f6718aa1700 1 --2- 172.21.15.7:0/2630627454 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55757954fa70 0x55757954e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:20.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.075+0000 7f66fcff9700 10 client.15408 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:06:20.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.075+0000 7f6718aa1700 1 --2- 172.21.15.7:0/2630627454 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55757954fa70 0x55757954e780 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f0efb6dc4ffb1f6 server_cookie=5d98b448b266deb5 in_seq=0 out_seq=0 2024-06-10T01:06:20.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.075+0000 7f66fcff9700 1 -- 172.21.15.7:0/2630627454 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55757941f280 con 0x55757954fa70 2024-06-10T01:06:20.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.075+0000 7f66fcff9700 10 client.15408 handle_command_reply: tid=0 2024-06-10T01:06:20.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.075+0000 7f6725d89b80 1 client.15408 shutdown 2024-06-10T01:06:20.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.075+0000 7f6725d89b80 1 -- 172.21.15.7:0/2630627454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575794c6750 msgr2=0x5575794c0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:20.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.075+0000 7f6725d89b80 1 --2- 172.21.15.7:0/2630627454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575794c6750 0x5575794c0000 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6710009990 tx=0x7f6710009ca0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.075+0000 7f6725d89b80 1 -- 172.21.15.7:0/2630627454 shutdown_connections 2024-06-10T01:06:20.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.075+0000 7f6725d89b80 1 --2- 172.21.15.7:0/2630627454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5575794cc830 0x5575794e9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.075+0000 7f6725d89b80 1 --2- 172.21.15.7:0/2630627454 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55757954fa70 0x55757954e780 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.075+0000 7f6725d89b80 1 --2- 172.21.15.7:0/2630627454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575794c6750 0x5575794c0000 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.075+0000 7f6725d89b80 1 --2- 172.21.15.7:0/2630627454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575794cbf10 0x5575794c0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.075+0000 7f66fcff9700 0 client.15408 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:06:20.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.075+0000 7f6725d89b80 1 -- 172.21.15.7:0/2630627454 >> 172.21.15.7:0/2630627454 conn(0x5575793f3030 msgr2=0x5575794cdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:20.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.075+0000 7f6725d89b80 1 -- 172.21.15.7:0/2630627454 shutdown_connections 2024-06-10T01:06:20.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.075+0000 7f6725d89b80 1 -- 172.21.15.7:0/2630627454 wait complete. 2024-06-10T01:06:20.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.075+0000 7f6725d89b80 20 client.15408 trim_cache size 0 max 16384 2024-06-10T01:06:20.078 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:06:20.079 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "bded58f8-26c3-11ef-bc9b-c7b262605968", 2024-06-10T01:06:20.079 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 2, 2024-06-10T01:06:20.079 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24434, 2024-06-10T01:06:20.079 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-06-10T01:06:20.079 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-06-10T01:06:20.079 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-06-10T01:06:20.079 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 206.373506574, 2024-06-10T01:06:20.079 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 27, 2024-06-10T01:06:20.079 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 99, 2024-06-10T01:06:20.079 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 99, 2024-06-10T01:06:20.079 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 286.30906136300001 2024-06-10T01:06:20.079 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:06:20.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.076+0000 7f671b506700 1 -- 172.21.15.7:0/2177581125 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f67000a2c20 msgr2=0x7f67000a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:20.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.076+0000 7f671b506700 1 --2- 172.21.15.7:0/2177581125 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f67000a2c20 0x7f67000a50e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f671000b5c0 tx=0x7f6710005960 comp rx=0 tx=0).stop 2024-06-10T01:06:20.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.076+0000 7f671b506700 1 -- 172.21.15.7:0/2177581125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6714113120 msgr2=0x7f67141a3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:20.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.076+0000 7f671b506700 1 --2- 172.21.15.7:0/2177581125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6714113120 0x7f67141a3c30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f670c00b9e0 tx=0x7f670c00bcf0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.076+0000 7f671b506700 1 -- 172.21.15.7:0/2177581125 shutdown_connections 2024-06-10T01:06:20.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.076+0000 7f671b506700 1 --2- 172.21.15.7:0/2177581125 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f67000a2c20 0x7f67000a50e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.076+0000 7f671b506700 1 --2- 172.21.15.7:0/2177581125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6714113120 0x7f67141a3c30 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.076+0000 7f671b506700 1 --2- 172.21.15.7:0/2177581125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6714074c00 0x7f671419f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.076+0000 7f671b506700 1 --2- 172.21.15.7:0/2177581125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67140764c0 0x7f671419f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:20.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.076+0000 7f671b506700 1 -- 172.21.15.7:0/2177581125 >> 172.21.15.7:0/2177581125 conn(0x7f6714079530 msgr2=0x7f6714111ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:20.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.076+0000 7f671b506700 1 -- 172.21.15.7:0/2177581125 shutdown_connections 2024-06-10T01:06:20.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:20.076+0000 7f671b506700 1 -- 172.21.15.7:0/2177581125 wait complete. 2024-06-10T01:06:20.729 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.c status 2024-06-10T01:06:20.984 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:06:21.366 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:21 smithi007 ceph-mon[32398]: pgmap v441: 129 pgs: 129 active+clean; 659 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-06-10T01:06:21.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:21 smithi078 ceph-mon[36683]: pgmap v441: 129 pgs: 129 active+clean; 659 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-06-10T01:06:21.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:21 smithi175 ceph-mon[36636]: pgmap v441: 129 pgs: 129 active+clean; 659 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-06-10T01:06:22.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.723+0000 7fb37b81d700 1 -- 172.21.15.7:0/754367127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb374108520 msgr2=0x7fb374108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:22.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.723+0000 7fb37b81d700 1 --2- 172.21.15.7:0/754367127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb374108520 0x7fb374108900 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb370009990 tx=0x7fb370009ca0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.724+0000 7fb37b81d700 1 -- 172.21.15.7:0/754367127 shutdown_connections 2024-06-10T01:06:22.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.724+0000 7fb37b81d700 1 --2- 172.21.15.7:0/754367127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb37406b5c0 0x7fb37410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.724+0000 7fb37b81d700 1 --2- 172.21.15.7:0/754367127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb374108520 0x7fb374108900 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.724+0000 7fb37b81d700 1 --2- 172.21.15.7:0/754367127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb37406ac00 0x7fb37406b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.724+0000 7fb37b81d700 1 -- 172.21.15.7:0/754367127 >> 172.21.15.7:0/754367127 conn(0x7fb3740773b0 msgr2=0x7fb3740777c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:22.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.724+0000 7fb37b81d700 1 -- 172.21.15.7:0/754367127 shutdown_connections 2024-06-10T01:06:22.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.724+0000 7fb37b81d700 1 -- 172.21.15.7:0/754367127 wait complete. 2024-06-10T01:06:22.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.724+0000 7fb37b81d700 1 Processor -- start 2024-06-10T01:06:22.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.724+0000 7fb37b81d700 1 -- start start 2024-06-10T01:06:22.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.725+0000 7fb37b81d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb37406ac00 0x7fb374101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:22.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.725+0000 7fb37b81d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb37406b5c0 0x7fb3741019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:22.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.725+0000 7fb37b81d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb374108520 0x7fb374101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:22.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.725+0000 7fb37b81d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb3741127a0 con 0x7fb37406b5c0 2024-06-10T01:06:22.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.725+0000 7fb37b81d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb374112620 con 0x7fb374108520 2024-06-10T01:06:22.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.725+0000 7fb37b81d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb374112920 con 0x7fb37406ac00 2024-06-10T01:06:22.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.725+0000 7fb378db8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb37406b5c0 0x7fb3741019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:22.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.725+0000 7fb378db8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb37406b5c0 0x7fb3741019d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57084/0 (socket says 172.21.15.7:57084) 2024-06-10T01:06:22.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.725+0000 7fb378db8700 1 -- 172.21.15.7:0/1061121779 learned_addr learned my addr 172.21.15.7:0/1061121779 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:22.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.725+0000 7fb3795b9700 1 --2- 172.21.15.7:0/1061121779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb37406ac00 0x7fb374101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:22.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.725+0000 7fb379dba700 1 --2- 172.21.15.7:0/1061121779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb374108520 0x7fb374101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:22.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.725+0000 7fb378db8700 1 -- 172.21.15.7:0/1061121779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb37406ac00 msgr2=0x7fb374101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:22.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.725+0000 7fb378db8700 1 --2- 172.21.15.7:0/1061121779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb37406ac00 0x7fb374101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.725+0000 7fb378db8700 1 -- 172.21.15.7:0/1061121779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb374108520 msgr2=0x7fb374101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:22.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.725+0000 7fb378db8700 1 --2- 172.21.15.7:0/1061121779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb374108520 0x7fb374101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.725+0000 7fb378db8700 1 -- 172.21.15.7:0/1061121779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3741a7110 con 0x7fb37406b5c0 2024-06-10T01:06:22.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.725+0000 7fb378db8700 1 --2- 172.21.15.7:0/1061121779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb37406b5c0 0x7fb3741019d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb36400d950 tx=0x7fb36400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:22.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.725+0000 7fb36a7fc700 1 -- 172.21.15.7:0/1061121779 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb364016070 con 0x7fb37406b5c0 2024-06-10T01:06:22.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.725+0000 7fb36a7fc700 1 -- 172.21.15.7:0/1061121779 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb36400fd70 con 0x7fb37406b5c0 2024-06-10T01:06:22.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.726+0000 7fb37b81d700 1 -- 172.21.15.7:0/1061121779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3741a7400 con 0x7fb37406b5c0 2024-06-10T01:06:22.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.726+0000 7fb36a7fc700 1 -- 172.21.15.7:0/1061121779 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36400e610 con 0x7fb37406b5c0 2024-06-10T01:06:22.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.726+0000 7fb37b81d700 1 -- 172.21.15.7:0/1061121779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3741a7990 con 0x7fb37406b5c0 2024-06-10T01:06:22.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.728+0000 7fb36a7fc700 1 -- 172.21.15.7:0/1061121779 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb364019070 con 0x7fb37406b5c0 2024-06-10T01:06:22.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.729+0000 7fb36a7fc700 1 --2- 172.21.15.7:0/1061121779 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb3600a2d20 0x7fb3600a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:22.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.729+0000 7fb3795b9700 1 --2- 172.21.15.7:0/1061121779 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb3600a2d20 0x7fb3600a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:22.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.729+0000 7fb36a7fc700 1 -- 172.21.15.7:0/1061121779 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fb3640ce8c0 con 0x7fb37406b5c0 2024-06-10T01:06:22.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.729+0000 7fb3795b9700 1 --2- 172.21.15.7:0/1061121779 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb3600a2d20 0x7fb3600a51e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb370006010 tx=0x7fb37000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:22.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.729+0000 7fb3860a0b80 1 Processor -- start 2024-06-10T01:06:22.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.729+0000 7fb3860a0b80 1 -- start start 2024-06-10T01:06:22.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.730+0000 7fb3860a0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fa43595f0 0x558fa43599d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:22.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.730+0000 7fb3860a0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fa4359f10 0x558fa4353990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:22.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.730+0000 7fb3860a0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558fa4353ed0 0x558fa43542f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:22.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.730+0000 7fb3860a0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558fa42a57c0 con 0x558fa43595f0 2024-06-10T01:06:22.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.730+0000 7fb3860a0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558fa424d6d0 con 0x558fa4359f10 2024-06-10T01:06:22.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.730+0000 7fb3860a0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558fa42a1780 con 0x558fa4353ed0 2024-06-10T01:06:22.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.730+0000 7fb3795b9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fa43595f0 0x558fa43599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:22.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.730+0000 7fb3795b9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fa43595f0 0x558fa43599d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57086/0 (socket says 172.21.15.7:57086) 2024-06-10T01:06:22.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.730+0000 7fb3795b9700 1 -- 172.21.15.7:0/1973188091 learned_addr learned my addr 172.21.15.7:0/1973188091 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:22.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.730+0000 7fb378db8700 1 --2- 172.21.15.7:0/1973188091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fa4359f10 0x558fa4353990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:22.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.730+0000 7fb379dba700 1 --2- 172.21.15.7:0/1973188091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558fa4353ed0 0x558fa43542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:22.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.730+0000 7fb3795b9700 1 -- 172.21.15.7:0/1973188091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558fa4353ed0 msgr2=0x558fa43542f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:22.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.730+0000 7fb3795b9700 1 --2- 172.21.15.7:0/1973188091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558fa4353ed0 0x558fa43542f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.730+0000 7fb3795b9700 1 -- 172.21.15.7:0/1973188091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fa4359f10 msgr2=0x558fa4353990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:22.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.730+0000 7fb3795b9700 1 --2- 172.21.15.7:0/1973188091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fa4359f10 0x558fa4353990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.730+0000 7fb3795b9700 1 -- 172.21.15.7:0/1973188091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fa4243f00 con 0x558fa43595f0 2024-06-10T01:06:22.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.730+0000 7fb378db8700 1 --2- 172.21.15.7:0/1973188091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fa4359f10 0x558fa4353990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:22.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.730+0000 7fb3795b9700 1 --2- 172.21.15.7:0/1973188091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fa43595f0 0x558fa43599d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb37000ba60 tx=0x7fb370020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd9131f6188d6afd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:22.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.731+0000 7fb35f7fe700 1 -- 172.21.15.7:0/1973188091 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37000bd70 con 0x558fa43595f0 2024-06-10T01:06:22.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.731+0000 7fb35f7fe700 1 -- 172.21.15.7:0/1973188091 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb37000fd50 con 0x558fa43595f0 2024-06-10T01:06:22.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.731+0000 7fb35f7fe700 1 -- 172.21.15.7:0/1973188091 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370024d10 con 0x558fa43595f0 2024-06-10T01:06:22.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.731+0000 7fb3860a0b80 1 -- 172.21.15.7:0/1973188091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fa43595f0 msgr2=0x558fa43599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:22.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.731+0000 7fb3860a0b80 1 --2- 172.21.15.7:0/1973188091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fa43595f0 0x558fa43599d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb37000ba60 tx=0x7fb370020000 comp rx=0 tx=0).stop 2024-06-10T01:06:22.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.731+0000 7fb3860a0b80 1 -- 172.21.15.7:0/1973188091 shutdown_connections 2024-06-10T01:06:22.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.731+0000 7fb3860a0b80 1 --2- 172.21.15.7:0/1973188091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558fa4353ed0 0x558fa43542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.731+0000 7fb3860a0b80 1 --2- 172.21.15.7:0/1973188091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fa43595f0 0x558fa43599d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.731+0000 7fb3860a0b80 1 --2- 172.21.15.7:0/1973188091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fa4359f10 0x558fa4353990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.731+0000 7fb3860a0b80 1 -- 172.21.15.7:0/1973188091 >> 172.21.15.7:0/1973188091 conn(0x558fa427ec70 msgr2=0x558fa434e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:22.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.731+0000 7fb3860a0b80 1 -- 172.21.15.7:0/1973188091 shutdown_connections 2024-06-10T01:06:22.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.732+0000 7fb3860a0b80 1 -- 172.21.15.7:0/1973188091 wait complete. 2024-06-10T01:06:22.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.732+0000 7fb3860a0b80 1 -- start start 2024-06-10T01:06:22.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.732+0000 7fb3860a0b80 1 Processor -- start 2024-06-10T01:06:22.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.732+0000 7fb3860a0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fa4353ed0 0x558fa435fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:22.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.732+0000 7fb3860a0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fa43595f0 0x558fa435cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:22.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.732+0000 7fb3860a0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558fa4359f10 0x558fa435d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:22.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.732+0000 7fb3860a0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558fa43608c0 con 0x558fa4353ed0 2024-06-10T01:06:22.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.732+0000 7fb3860a0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558fa4245ba0 con 0x558fa43595f0 2024-06-10T01:06:22.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.732+0000 7fb3860a0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558fa4360a40 con 0x558fa4359f10 2024-06-10T01:06:22.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.732+0000 7fb3860a0b80 10 client.0 _ll_register_callbacks cb 0x558fa423f8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:22.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.733+0000 7fb3795b9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fa4353ed0 0x558fa435fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:22.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.733+0000 7fb3795b9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fa4353ed0 0x558fa435fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57096/0 (socket says 172.21.15.7:57096) 2024-06-10T01:06:22.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.733+0000 7fb3795b9700 1 -- 172.21.15.7:0/3640853059 learned_addr learned my addr 172.21.15.7:0/3640853059 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:22.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.733+0000 7fb379dba700 1 --2- 172.21.15.7:0/3640853059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558fa4359f10 0x558fa435d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:22.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.733+0000 7fb378db8700 1 --2- 172.21.15.7:0/3640853059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fa43595f0 0x558fa435cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:22.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.733+0000 7fb3795b9700 1 -- 172.21.15.7:0/3640853059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558fa4359f10 msgr2=0x558fa435d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:22.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.733+0000 7fb3795b9700 1 --2- 172.21.15.7:0/3640853059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558fa4359f10 0x558fa435d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.733+0000 7fb3795b9700 1 -- 172.21.15.7:0/3640853059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fa43595f0 msgr2=0x558fa435cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:22.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.733+0000 7fb3795b9700 1 --2- 172.21.15.7:0/3640853059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fa43595f0 0x558fa435cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.733+0000 7fb3795b9700 1 -- 172.21.15.7:0/3640853059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fa41f4880 con 0x558fa4353ed0 2024-06-10T01:06:22.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.733+0000 7fb3795b9700 1 --2- 172.21.15.7:0/3640853059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fa4353ed0 0x558fa435fc50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb370024fd0 tx=0x7fb370024670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=716cfc020e779b2b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:22.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.733+0000 7fb35cff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:06:22.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.733+0000 7fb35cff9700 1 -- 172.21.15.7:0/3640853059 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370024410 con 0x558fa4353ed0 2024-06-10T01:06:22.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.733+0000 7fb35cff9700 1 -- 172.21.15.7:0/3640853059 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fb37002c050 con 0x558fa4353ed0 2024-06-10T01:06:22.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.733+0000 7fb3860a0b80 1 -- 172.21.15.7:0/3640853059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb37000fd50 con 0x558fa4353ed0 2024-06-10T01:06:22.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.733+0000 7fb35cff9700 1 -- 172.21.15.7:0/3640853059 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb370024c00 con 0x558fa4353ed0 2024-06-10T01:06:22.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.734+0000 7fb3860a0b80 10 client.15420 fetch_fsmap learned FSMap version 27 2024-06-10T01:06:22.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.734+0000 7fb3860a0b80 1 -- 172.21.15.7:0/3640853059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558fa436b1a0 con 0x558fa4353ed0 2024-06-10T01:06:22.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.734+0000 7fb35cff9700 1 -- 172.21.15.7:0/3640853059 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb37000fd50 con 0x558fa4353ed0 2024-06-10T01:06:22.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.734+0000 7fb35cff9700 1 -- 172.21.15.7:0/3640853059 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb3700173f0 con 0x558fa4353ed0 2024-06-10T01:06:22.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.734+0000 7fb3860a0b80 10 client.15420 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:06:22.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.734+0000 7fb3860a0b80 10 client.15420 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-10T01:06:22.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.734+0000 7fb3860a0b80 20 client.15420 populate_metadata read hostname 'smithi007' 2024-06-10T01:06:22.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.734+0000 7fb3860a0b80 1 --2- 172.21.15.7:0/3640853059 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x558fa436c4f0 0x558fa436e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:22.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.734+0000 7fb3860a0b80 4 client.15420 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:06:22.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.734+0000 7fb3860a0b80 1 -- 172.21.15.7:0/3640853059 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fa42b2cc0 con 0x558fa436c4f0 2024-06-10T01:06:22.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.734+0000 7fb378db8700 1 --2- 172.21.15.7:0/3640853059 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x558fa436c4f0 0x558fa436e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:22.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.735+0000 7fb378db8700 1 --2- 172.21.15.7:0/3640853059 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x558fa436c4f0 0x558fa436e5b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b80bca2f3be18a35 server_cookie=8c1a490372f840c4 in_seq=0 out_seq=0 2024-06-10T01:06:22.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.735+0000 7fb35cff9700 10 client.15420 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:06:22.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.735+0000 7fb35cff9700 1 -- 172.21.15.7:0/3640853059 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558fa42b2cc0 con 0x558fa436c4f0 2024-06-10T01:06:22.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.735+0000 7fb35cff9700 10 client.15420 handle_command_reply: tid=0 2024-06-10T01:06:22.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.736+0000 7fb3860a0b80 1 client.15420 shutdown 2024-06-10T01:06:22.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.736+0000 7fb3860a0b80 1 -- 172.21.15.7:0/3640853059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fa4353ed0 msgr2=0x558fa435fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:22.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.736+0000 7fb3860a0b80 1 --2- 172.21.15.7:0/3640853059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fa4353ed0 0x558fa435fc50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb370024fd0 tx=0x7fb370024670 comp rx=0 tx=0).stop 2024-06-10T01:06:22.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.736+0000 7fb3860a0b80 1 -- 172.21.15.7:0/3640853059 shutdown_connections 2024-06-10T01:06:22.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.736+0000 7fb3860a0b80 1 --2- 172.21.15.7:0/3640853059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558fa4359f10 0x558fa435d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.736+0000 7fb3860a0b80 1 --2- 172.21.15.7:0/3640853059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fa4353ed0 0x558fa435fc50 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.736+0000 7fb3860a0b80 1 --2- 172.21.15.7:0/3640853059 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x558fa436c4f0 0x558fa436e5b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.736+0000 7fb3860a0b80 1 --2- 172.21.15.7:0/3640853059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fa43595f0 0x558fa435cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.736+0000 7fb3860a0b80 1 -- 172.21.15.7:0/3640853059 >> 172.21.15.7:0/3640853059 conn(0x558fa427ec70 msgr2=0x558fa434e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:22.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.736+0000 7fb35cff9700 0 client.15420 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:06:22.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.736+0000 7fb3860a0b80 1 -- 172.21.15.7:0/3640853059 shutdown_connections 2024-06-10T01:06:22.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.736+0000 7fb3860a0b80 1 -- 172.21.15.7:0/3640853059 wait complete. 2024-06-10T01:06:22.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.736+0000 7fb3860a0b80 20 client.15420 trim_cache size 0 max 16384 2024-06-10T01:06:22.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.758+0000 7fb3860a0b80 1 Processor -- start 2024-06-10T01:06:22.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.758+0000 7fb3860a0b80 1 -- start start 2024-06-10T01:06:22.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.758+0000 7fb3860a0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fa4353ed0 0x558fa4353990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:22.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.758+0000 7fb3860a0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558fa43595f0 0x558fa435a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:22.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.758+0000 7fb3860a0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fa4359f10 0x558fa435a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:22.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.758+0000 7fb3860a0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558fa436c8f0 con 0x558fa4359f10 2024-06-10T01:06:22.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.758+0000 7fb3860a0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558fa436cfc0 con 0x558fa4353ed0 2024-06-10T01:06:22.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.758+0000 7fb3860a0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558fa436d690 con 0x558fa43595f0 2024-06-10T01:06:22.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.758+0000 7fb379dba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fa4359f10 0x558fa435a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:22.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.758+0000 7fb379dba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fa4359f10 0x558fa435a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57104/0 (socket says 172.21.15.7:57104) 2024-06-10T01:06:22.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.758+0000 7fb379dba700 1 -- 172.21.15.7:0/3625051254 learned_addr learned my addr 172.21.15.7:0/3625051254 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:22.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.758+0000 7fb3795b9700 1 --2- 172.21.15.7:0/3625051254 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fa4353ed0 0x558fa4353990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:22.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.758+0000 7fb378db8700 1 --2- 172.21.15.7:0/3625051254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558fa43595f0 0x558fa435a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:22.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.758+0000 7fb379dba700 1 -- 172.21.15.7:0/3625051254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558fa43595f0 msgr2=0x558fa435a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:22.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.758+0000 7fb379dba700 1 --2- 172.21.15.7:0/3625051254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558fa43595f0 0x558fa435a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.758+0000 7fb379dba700 1 -- 172.21.15.7:0/3625051254 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fa4353ed0 msgr2=0x558fa4353990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:22.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.758+0000 7fb379dba700 1 --2- 172.21.15.7:0/3625051254 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fa4353ed0 0x558fa4353990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.758+0000 7fb379dba700 1 -- 172.21.15.7:0/3625051254 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fa436cde0 con 0x558fa4359f10 2024-06-10T01:06:22.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.759+0000 7fb379dba700 1 --2- 172.21.15.7:0/3625051254 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fa4359f10 0x558fa435a830 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb36c014690 tx=0x7fb36c007950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0ffb5c8a41cc55 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:22.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.759+0000 7fb35effd700 1 -- 172.21.15.7:0/3625051254 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c01a070 con 0x558fa4359f10 2024-06-10T01:06:22.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.759+0000 7fb35effd700 1 -- 172.21.15.7:0/3625051254 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb36c016b10 con 0x558fa4359f10 2024-06-10T01:06:22.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.759+0000 7fb35effd700 1 -- 172.21.15.7:0/3625051254 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c01fc50 con 0x558fa4359f10 2024-06-10T01:06:22.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.759+0000 7fb3860a0b80 1 -- 172.21.15.7:0/3625051254 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fa4359f10 msgr2=0x558fa435a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:22.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.759+0000 7fb3860a0b80 1 --2- 172.21.15.7:0/3625051254 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fa4359f10 0x558fa435a830 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb36c014690 tx=0x7fb36c007950 comp rx=0 tx=0).stop 2024-06-10T01:06:22.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.759+0000 7fb3860a0b80 1 -- 172.21.15.7:0/3625051254 shutdown_connections 2024-06-10T01:06:22.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.759+0000 7fb3860a0b80 1 --2- 172.21.15.7:0/3625051254 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fa4359f10 0x558fa435a830 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.759+0000 7fb3860a0b80 1 --2- 172.21.15.7:0/3625051254 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fa4353ed0 0x558fa4353990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.759+0000 7fb3860a0b80 1 --2- 172.21.15.7:0/3625051254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558fa43595f0 0x558fa435a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.759+0000 7fb3860a0b80 1 -- 172.21.15.7:0/3625051254 >> 172.21.15.7:0/3625051254 conn(0x558fa427ec70 msgr2=0x558fa43542b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:22.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.760+0000 7fb3860a0b80 1 -- 172.21.15.7:0/3625051254 shutdown_connections 2024-06-10T01:06:22.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.760+0000 7fb3860a0b80 1 -- 172.21.15.7:0/3625051254 wait complete. 2024-06-10T01:06:22.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.760+0000 7fb3860a0b80 1 -- start start 2024-06-10T01:06:22.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.760+0000 7fb3860a0b80 1 Processor -- start 2024-06-10T01:06:22.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.760+0000 7fb3860a0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fa4353ed0 0x558fa4369380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:22.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.760+0000 7fb3860a0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558fa43595f0 0x558fa43698c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:22.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.760+0000 7fb3860a0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fa4359f10 0x558fa4369e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:22.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.761+0000 7fb3860a0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558fa436e430 con 0x558fa4359f10 2024-06-10T01:06:22.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.761+0000 7fb3860a0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558fa436dd60 con 0x558fa4353ed0 2024-06-10T01:06:22.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.761+0000 7fb3860a0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558fa436eb00 con 0x558fa43595f0 2024-06-10T01:06:22.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.761+0000 7fb379dba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fa4359f10 0x558fa4369e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:22.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.761+0000 7fb3860a0b80 10 client.0 _ll_register_callbacks cb 0x558fa423f8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:22.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.761+0000 7fb379dba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fa4359f10 0x558fa4369e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57118/0 (socket says 172.21.15.7:57118) 2024-06-10T01:06:22.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.761+0000 7fb379dba700 1 -- 172.21.15.7:0/733857629 learned_addr learned my addr 172.21.15.7:0/733857629 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:22.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.761+0000 7fb3795b9700 1 --2- 172.21.15.7:0/733857629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fa4353ed0 0x558fa4369380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:22.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.761+0000 7fb378db8700 1 --2- 172.21.15.7:0/733857629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558fa43595f0 0x558fa43698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:22.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.761+0000 7fb379dba700 1 -- 172.21.15.7:0/733857629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558fa43595f0 msgr2=0x558fa43698c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:22.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.761+0000 7fb379dba700 1 --2- 172.21.15.7:0/733857629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558fa43595f0 0x558fa43698c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.761+0000 7fb379dba700 1 -- 172.21.15.7:0/733857629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fa4353ed0 msgr2=0x558fa4369380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:22.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.761+0000 7fb379dba700 1 --2- 172.21.15.7:0/733857629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fa4353ed0 0x558fa4369380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.761+0000 7fb379dba700 1 -- 172.21.15.7:0/733857629 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fa4376100 con 0x558fa4359f10 2024-06-10T01:06:22.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.761+0000 7fb378db8700 1 --2- 172.21.15.7:0/733857629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558fa43595f0 0x558fa43698c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:06:22.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.761+0000 7fb379dba700 1 --2- 172.21.15.7:0/733857629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fa4359f10 0x558fa4369e00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb36c009010 tx=0x7fb36c0168e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63fd218f6f598f24 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:22.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.761+0000 7fb35cff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:06:22.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.761+0000 7fb35cff9700 1 -- 172.21.15.7:0/733857629 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c01a040 con 0x558fa4359f10 2024-06-10T01:06:22.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.761+0000 7fb3860a0b80 1 -- 172.21.15.7:0/733857629 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb36c016b10 con 0x558fa4359f10 2024-06-10T01:06:22.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.761+0000 7fb35cff9700 1 -- 172.21.15.7:0/733857629 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fb36c003c10 con 0x558fa4359f10 2024-06-10T01:06:22.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.762+0000 7fb35cff9700 1 -- 172.21.15.7:0/733857629 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb36c003e30 con 0x558fa4359f10 2024-06-10T01:06:22.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.762+0000 7fb3860a0b80 10 client.15426 fetch_fsmap learned FSMap version 27 2024-06-10T01:06:22.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.762+0000 7fb3860a0b80 1 -- 172.21.15.7:0/733857629 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558fa43d8e10 con 0x558fa4359f10 2024-06-10T01:06:22.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.762+0000 7fb35cff9700 1 -- 172.21.15.7:0/733857629 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb36c016b10 con 0x558fa4359f10 2024-06-10T01:06:22.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.762+0000 7fb35cff9700 1 -- 172.21.15.7:0/733857629 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb36c004530 con 0x558fa4359f10 2024-06-10T01:06:22.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.762+0000 7fb3860a0b80 10 client.15426 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:06:22.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.762+0000 7fb3860a0b80 10 client.15426 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-10T01:06:22.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.762+0000 7fb3860a0b80 20 client.15426 populate_metadata read hostname 'smithi007' 2024-06-10T01:06:22.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.762+0000 7fb3860a0b80 1 --2- 172.21.15.7:0/733857629 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x558fa43dd610 0x558fa43dc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:22.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.762+0000 7fb3860a0b80 4 client.15426 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T01:06:22.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.762+0000 7fb3860a0b80 1 -- 172.21.15.7:0/733857629 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "status"}) -- 0x558fa42ad240 con 0x558fa43dd610 2024-06-10T01:06:22.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.762+0000 7fb3795b9700 1 --2- 172.21.15.7:0/733857629 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x558fa43dd610 0x558fa43dc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:22.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.763+0000 7fb3795b9700 1 --2- 172.21.15.7:0/733857629 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x558fa43dd610 0x558fa43dc320 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=588e422ee4fe9005 server_cookie=e9365c733b095398 in_seq=0 out_seq=0 2024-06-10T01:06:22.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.763+0000 7fb35cff9700 10 client.15426 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:06:22.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.763+0000 7fb35cff9700 1 -- 172.21.15.7:0/733857629 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x558fa42ad240 con 0x558fa43dd610 2024-06-10T01:06:22.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.763+0000 7fb35cff9700 10 client.15426 handle_command_reply: tid=0 2024-06-10T01:06:22.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.763+0000 7fb3860a0b80 1 client.15426 shutdown 2024-06-10T01:06:22.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.763+0000 7fb3860a0b80 1 -- 172.21.15.7:0/733857629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fa4359f10 msgr2=0x558fa4369e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:22.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.763+0000 7fb3860a0b80 1 --2- 172.21.15.7:0/733857629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fa4359f10 0x558fa4369e00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb36c009010 tx=0x7fb36c0168e0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.763+0000 7fb3860a0b80 1 -- 172.21.15.7:0/733857629 shutdown_connections 2024-06-10T01:06:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.763+0000 7fb3860a0b80 1 --2- 172.21.15.7:0/733857629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fa4359f10 0x558fa4369e00 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.763+0000 7fb3860a0b80 1 --2- 172.21.15.7:0/733857629 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x558fa43dd610 0x558fa43dc320 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.763+0000 7fb3860a0b80 1 --2- 172.21.15.7:0/733857629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fa4353ed0 0x558fa4369380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.763+0000 7fb35cff9700 0 client.15426 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:06:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.763+0000 7fb3860a0b80 1 --2- 172.21.15.7:0/733857629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558fa43595f0 0x558fa43698c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.763+0000 7fb3860a0b80 1 -- 172.21.15.7:0/733857629 >> 172.21.15.7:0/733857629 conn(0x558fa427ec70 msgr2=0x558fa435a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.763+0000 7fb3860a0b80 1 -- 172.21.15.7:0/733857629 shutdown_connections 2024-06-10T01:06:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.763+0000 7fb3860a0b80 1 -- 172.21.15.7:0/733857629 wait complete. 2024-06-10T01:06:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.764+0000 7fb3860a0b80 20 client.15426 trim_cache size 0 max 16384 2024-06-10T01:06:22.767 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:06:22.767 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "bded58f8-26c3-11ef-bc9b-c7b262605968", 2024-06-10T01:06:22.767 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 1, 2024-06-10T01:06:22.767 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24436, 2024-06-10T01:06:22.767 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-06-10T01:06:22.767 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-06-10T01:06:22.767 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-06-10T01:06:22.767 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 210.07701239900001, 2024-06-10T01:06:22.767 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 27, 2024-06-10T01:06:22.767 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 99, 2024-06-10T01:06:22.767 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 99, 2024-06-10T01:06:22.767 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 306.37680843300001 2024-06-10T01:06:22.768 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:06:22.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.764+0000 7fb37b81d700 1 -- 172.21.15.7:0/1061121779 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb3600a2d20 msgr2=0x7fb3600a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:22.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.764+0000 7fb37b81d700 1 --2- 172.21.15.7:0/1061121779 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb3600a2d20 0x7fb3600a51e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb370006010 tx=0x7fb37000b540 comp rx=0 tx=0).stop 2024-06-10T01:06:22.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.764+0000 7fb37b81d700 1 -- 172.21.15.7:0/1061121779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb37406b5c0 msgr2=0x7fb3741019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:22.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.764+0000 7fb37b81d700 1 --2- 172.21.15.7:0/1061121779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb37406b5c0 0x7fb3741019d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb36400d950 tx=0x7fb36400dc60 comp rx=0 tx=0).stop 2024-06-10T01:06:22.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.764+0000 7fb37b81d700 1 -- 172.21.15.7:0/1061121779 shutdown_connections 2024-06-10T01:06:22.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.764+0000 7fb37b81d700 1 --2- 172.21.15.7:0/1061121779 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb3600a2d20 0x7fb3600a51e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.764+0000 7fb37b81d700 1 --2- 172.21.15.7:0/1061121779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb374108520 0x7fb374101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.764+0000 7fb37b81d700 1 --2- 172.21.15.7:0/1061121779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb37406ac00 0x7fb374101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.764+0000 7fb37b81d700 1 --2- 172.21.15.7:0/1061121779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb37406b5c0 0x7fb3741019d0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:22.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.764+0000 7fb37b81d700 1 -- 172.21.15.7:0/1061121779 >> 172.21.15.7:0/1061121779 conn(0x7fb3740773b0 msgr2=0x7fb3740fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:22.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.764+0000 7fb37b81d700 1 -- 172.21.15.7:0/1061121779 shutdown_connections 2024-06-10T01:06:22.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:22.764+0000 7fb37b81d700 1 -- 172.21.15.7:0/1061121779 wait complete. 2024-06-10T01:06:23.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:23 smithi007 ceph-mon[32398]: pgmap v442: 129 pgs: 129 active+clean; 660 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-06-10T01:06:23.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:23 smithi078 ceph-mon[36683]: pgmap v442: 129 pgs: 129 active+clean; 660 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-06-10T01:06:23.593 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-10T01:06:23.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:23 smithi175 ceph-mon[36636]: pgmap v442: 129 pgs: 129 active+clean; 660 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-06-10T01:06:23.849 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:06:25.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:25 smithi078 ceph-mon[36683]: pgmap v443: 129 pgs: 129 active+clean; 660 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-06-10T01:06:25.589 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:25 smithi007 ceph-mon[32398]: pgmap v443: 129 pgs: 129 active+clean; 660 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-06-10T01:06:25.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.616+0000 7f4d98c9b700 1 -- 172.21.15.7:0/810119114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d94102740 msgr2=0x7f4d94102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:25.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.616+0000 7f4d98c9b700 1 --2- 172.21.15.7:0/810119114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d94102740 0x7f4d94102b20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4d840099c0 tx=0x7f4d84009cd0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.616+0000 7f4d98c9b700 1 -- 172.21.15.7:0/810119114 shutdown_connections 2024-06-10T01:06:25.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.616+0000 7f4d98c9b700 1 --2- 172.21.15.7:0/810119114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d94107830 0x7f4d9410f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.616+0000 7f4d98c9b700 1 --2- 172.21.15.7:0/810119114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d94102740 0x7f4d94102b20 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.616+0000 7f4d98c9b700 1 --2- 172.21.15.7:0/810119114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d941030f0 0x7f4d94107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.616+0000 7f4d98c9b700 1 -- 172.21.15.7:0/810119114 >> 172.21.15.7:0/810119114 conn(0x7f4d940795b0 msgr2=0x7f4d940799c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:25.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.616+0000 7f4d98c9b700 1 -- 172.21.15.7:0/810119114 shutdown_connections 2024-06-10T01:06:25.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.616+0000 7f4d98c9b700 1 -- 172.21.15.7:0/810119114 wait complete. 2024-06-10T01:06:25.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.617+0000 7f4d98c9b700 1 Processor -- start 2024-06-10T01:06:25.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.617+0000 7f4d98c9b700 1 -- start start 2024-06-10T01:06:25.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.617+0000 7f4d98c9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d94102740 0x7f4d9419e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:25.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.617+0000 7f4d98c9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d941030f0 0x7f4d9419ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:25.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.617+0000 7f4d98c9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94107830 0x7f4d94198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:25.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.617+0000 7f4d98c9b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4d94112770 con 0x7f4d94102740 2024-06-10T01:06:25.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.617+0000 7f4d98c9b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4d941125f0 con 0x7f4d94107830 2024-06-10T01:06:25.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.617+0000 7f4d98c9b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4d941128f0 con 0x7f4d941030f0 2024-06-10T01:06:25.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.617+0000 7f4d9259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d94102740 0x7f4d9419e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:25.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.617+0000 7f4d9259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d94102740 0x7f4d9419e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57134/0 (socket says 172.21.15.7:57134) 2024-06-10T01:06:25.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.617+0000 7f4d9259c700 1 -- 172.21.15.7:0/1324337967 learned_addr learned my addr 172.21.15.7:0/1324337967 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:25.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.618+0000 7f4d91d9b700 1 --2- 172.21.15.7:0/1324337967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d941030f0 0x7f4d9419ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:25.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.618+0000 7f4d92d9d700 1 --2- 172.21.15.7:0/1324337967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94107830 0x7f4d94198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:25.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.618+0000 7f4d9259c700 1 -- 172.21.15.7:0/1324337967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d941030f0 msgr2=0x7f4d9419ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:25.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.618+0000 7f4d9259c700 1 --2- 172.21.15.7:0/1324337967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d941030f0 0x7f4d9419ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.618+0000 7f4d9259c700 1 -- 172.21.15.7:0/1324337967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94107830 msgr2=0x7f4d94198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:25.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.618+0000 7f4d9259c700 1 --2- 172.21.15.7:0/1324337967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94107830 0x7f4d94198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.618+0000 7f4d9259c700 1 -- 172.21.15.7:0/1324337967 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d94199090 con 0x7f4d94102740 2024-06-10T01:06:25.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.618+0000 7f4d92d9d700 1 --2- 172.21.15.7:0/1324337967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94107830 0x7f4d94198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:06:25.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.618+0000 7f4d9259c700 1 --2- 172.21.15.7:0/1324337967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d94102740 0x7f4d9419e720 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4d84006010 tx=0x7f4d84009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:25.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.618+0000 7f4d837fe700 1 -- 172.21.15.7:0/1324337967 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d84027070 con 0x7f4d94102740 2024-06-10T01:06:25.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.618+0000 7f4d837fe700 1 -- 172.21.15.7:0/1324337967 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d84003780 con 0x7f4d94102740 2024-06-10T01:06:25.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.618+0000 7f4d98c9b700 1 -- 172.21.15.7:0/1324337967 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d94199320 con 0x7f4d94102740 2024-06-10T01:06:25.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.618+0000 7f4d837fe700 1 -- 172.21.15.7:0/1324337967 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d84022da0 con 0x7f4d94102740 2024-06-10T01:06:25.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.618+0000 7f4d98c9b700 1 -- 172.21.15.7:0/1324337967 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d941a55f0 con 0x7f4d94102740 2024-06-10T01:06:25.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.621+0000 7f4d837fe700 1 -- 172.21.15.7:0/1324337967 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f4d8402a070 con 0x7f4d94102740 2024-06-10T01:06:25.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.622+0000 7f4d837fe700 1 --2- 172.21.15.7:0/1324337967 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4d7c0a2c20 0x7f4d7c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:25.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.622+0000 7f4d91d9b700 1 --2- 172.21.15.7:0/1324337967 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4d7c0a2c20 0x7f4d7c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:25.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.622+0000 7f4d837fe700 1 -- 172.21.15.7:0/1324337967 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f4d840d01d0 con 0x7f4d94102740 2024-06-10T01:06:25.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.623+0000 7f4d91d9b700 1 --2- 172.21.15.7:0/1324337967 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4d7c0a2c20 0x7f4d7c0a50e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4d94108c80 tx=0x7f4d8800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:25.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.623+0000 7f4da351eb80 1 Processor -- start 2024-06-10T01:06:25.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.623+0000 7f4da351eb80 1 -- start start 2024-06-10T01:06:25.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.623+0000 7f4da351eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570b6a8ba20 0x5570b6a8be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:25.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.623+0000 7f4da351eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570b6a8c340 0x5570b6a85da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:25.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.623+0000 7f4da351eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b6a862e0 0x5570b6a86700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:25.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.623+0000 7f4da351eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5570b69d37c0 con 0x5570b6a862e0 2024-06-10T01:06:25.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.623+0000 7f4da351eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5570b697b6d0 con 0x5570b6a8ba20 2024-06-10T01:06:25.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.623+0000 7f4da351eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5570b69cf780 con 0x5570b6a8c340 2024-06-10T01:06:25.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.623+0000 7f4d92d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b6a862e0 0x5570b6a86700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:25.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.623+0000 7f4d92d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b6a862e0 0x5570b6a86700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57148/0 (socket says 172.21.15.7:57148) 2024-06-10T01:06:25.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.623+0000 7f4d92d9d700 1 -- 172.21.15.7:0/4255193888 learned_addr learned my addr 172.21.15.7:0/4255193888 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:25.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.624+0000 7f4d91d9b700 1 --2- 172.21.15.7:0/4255193888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570b6a8c340 0x5570b6a85da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:25.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.624+0000 7f4d9259c700 1 --2- 172.21.15.7:0/4255193888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570b6a8ba20 0x5570b6a8be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:25.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.624+0000 7f4d92d9d700 1 -- 172.21.15.7:0/4255193888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570b6a8c340 msgr2=0x5570b6a85da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:25.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.624+0000 7f4d92d9d700 1 --2- 172.21.15.7:0/4255193888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570b6a8c340 0x5570b6a85da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.624+0000 7f4d92d9d700 1 -- 172.21.15.7:0/4255193888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570b6a8ba20 msgr2=0x5570b6a8be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:25.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.624+0000 7f4d92d9d700 1 --2- 172.21.15.7:0/4255193888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570b6a8ba20 0x5570b6a8be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.624+0000 7f4d92d9d700 1 -- 172.21.15.7:0/4255193888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570b6971f00 con 0x5570b6a862e0 2024-06-10T01:06:25.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.624+0000 7f4d91d9b700 1 --2- 172.21.15.7:0/4255193888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570b6a8c340 0x5570b6a85da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:06:25.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.624+0000 7f4d92d9d700 1 --2- 172.21.15.7:0/4255193888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b6a862e0 0x5570b6a86700 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c009e60 tx=0x7f4d8c0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb605bd47b02c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:25.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.624+0000 7f4d80ff9700 1 -- 172.21.15.7:0/4255193888 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c014610 con 0x5570b6a862e0 2024-06-10T01:06:25.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.624+0000 7f4d80ff9700 1 -- 172.21.15.7:0/4255193888 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d8c019070 con 0x5570b6a862e0 2024-06-10T01:06:25.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.624+0000 7f4d80ff9700 1 -- 172.21.15.7:0/4255193888 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c013b80 con 0x5570b6a862e0 2024-06-10T01:06:25.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.625+0000 7f4da351eb80 1 -- 172.21.15.7:0/4255193888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b6a862e0 msgr2=0x5570b6a86700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:25.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.625+0000 7f4da351eb80 1 --2- 172.21.15.7:0/4255193888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b6a862e0 0x5570b6a86700 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c009e60 tx=0x7f4d8c0077e0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.625+0000 7f4da351eb80 1 -- 172.21.15.7:0/4255193888 shutdown_connections 2024-06-10T01:06:25.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.625+0000 7f4da351eb80 1 --2- 172.21.15.7:0/4255193888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b6a862e0 0x5570b6a86700 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.625+0000 7f4da351eb80 1 --2- 172.21.15.7:0/4255193888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570b6a8ba20 0x5570b6a8be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.625+0000 7f4da351eb80 1 --2- 172.21.15.7:0/4255193888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570b6a8c340 0x5570b6a85da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.625+0000 7f4da351eb80 1 -- 172.21.15.7:0/4255193888 >> 172.21.15.7:0/4255193888 conn(0x5570b69acc70 msgr2=0x5570b6a7c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:25.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.625+0000 7f4da351eb80 1 -- 172.21.15.7:0/4255193888 shutdown_connections 2024-06-10T01:06:25.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.625+0000 7f4da351eb80 1 -- 172.21.15.7:0/4255193888 wait complete. 2024-06-10T01:06:25.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.625+0000 7f4da351eb80 1 -- start start 2024-06-10T01:06:25.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.626+0000 7f4da351eb80 1 Processor -- start 2024-06-10T01:06:25.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.626+0000 7f4da351eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b6a862e0 0x5570b6a920a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:25.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.626+0000 7f4da351eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570b6a8ba20 0x5570b6a8f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:25.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.626+0000 7f4da351eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570b6a8c340 0x5570b6a8f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:25.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.626+0000 7f4da351eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5570b6a92d80 con 0x5570b6a862e0 2024-06-10T01:06:25.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.626+0000 7f4da351eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5570b6973ba0 con 0x5570b6a8c340 2024-06-10T01:06:25.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.626+0000 7f4da351eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5570b6a92f00 con 0x5570b6a8ba20 2024-06-10T01:06:25.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.626+0000 7f4da351eb80 10 client.0 _ll_register_callbacks cb 0x5570b696d8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:25.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.626+0000 7f4d9259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b6a862e0 0x5570b6a920a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:25.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.626+0000 7f4d9259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b6a862e0 0x5570b6a920a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57156/0 (socket says 172.21.15.7:57156) 2024-06-10T01:06:25.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.626+0000 7f4d9259c700 1 -- 172.21.15.7:0/501148582 learned_addr learned my addr 172.21.15.7:0/501148582 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:25.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.626+0000 7f4d92d9d700 1 --2- 172.21.15.7:0/501148582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570b6a8c340 0x5570b6a8f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:25.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.626+0000 7f4d91d9b700 1 --2- 172.21.15.7:0/501148582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570b6a8ba20 0x5570b6a8f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:25.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.626+0000 7f4d9259c700 1 -- 172.21.15.7:0/501148582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570b6a8ba20 msgr2=0x5570b6a8f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:25.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.626+0000 7f4d9259c700 1 --2- 172.21.15.7:0/501148582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570b6a8ba20 0x5570b6a8f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.626+0000 7f4d9259c700 1 -- 172.21.15.7:0/501148582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570b6a8c340 msgr2=0x5570b6a8f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:25.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.626+0000 7f4d9259c700 1 --2- 172.21.15.7:0/501148582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570b6a8c340 0x5570b6a8f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.626+0000 7f4d9259c700 1 -- 172.21.15.7:0/501148582 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570b6922880 con 0x5570b6a862e0 2024-06-10T01:06:25.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.627+0000 7f4d9259c700 1 --2- 172.21.15.7:0/501148582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b6a862e0 0x5570b6a920a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4d8400c010 tx=0x7f4d8409c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd875267884f2f2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:25.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.627+0000 7f4d79ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:06:25.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.627+0000 7f4d79ffb700 1 -- 172.21.15.7:0/501148582 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8409ce40 con 0x5570b6a862e0 2024-06-10T01:06:25.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.627+0000 7f4d79ffb700 1 -- 172.21.15.7:0/501148582 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f4d840a4e90 con 0x5570b6a862e0 2024-06-10T01:06:25.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.627+0000 7f4da351eb80 1 -- 172.21.15.7:0/501148582 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d8c019070 con 0x5570b6a862e0 2024-06-10T01:06:25.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.627+0000 7f4d79ffb700 1 -- 172.21.15.7:0/501148582 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4d8409c740 con 0x5570b6a862e0 2024-06-10T01:06:25.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.627+0000 7f4da351eb80 10 client.15438 fetch_fsmap learned FSMap version 27 2024-06-10T01:06:25.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.627+0000 7f4da351eb80 1 -- 172.21.15.7:0/501148582 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5570b6a9d680 con 0x5570b6a862e0 2024-06-10T01:06:25.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.627+0000 7f4d79ffb700 1 -- 172.21.15.7:0/501148582 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d8c019070 con 0x5570b6a862e0 2024-06-10T01:06:25.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.627+0000 7f4d79ffb700 1 -- 172.21.15.7:0/501148582 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4d840bc040 con 0x5570b6a862e0 2024-06-10T01:06:25.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.627+0000 7f4da351eb80 10 client.15438 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:06:25.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.627+0000 7f4da351eb80 10 client.15438 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-10T01:06:25.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.627+0000 7f4da351eb80 20 client.15438 populate_metadata read hostname 'smithi007' 2024-06-10T01:06:25.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.627+0000 7f4da351eb80 1 --2- 172.21.15.7:0/501148582 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5570b6a9fcc0 0x5570b6aa1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:25.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.627+0000 7f4da351eb80 4 client.15438 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:06:25.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.627+0000 7f4da351eb80 1 -- 172.21.15.7:0/501148582 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570b69e0cc0 con 0x5570b6a9fcc0 2024-06-10T01:06:25.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.628+0000 7f4d91d9b700 1 --2- 172.21.15.7:0/501148582 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5570b6a9fcc0 0x5570b6aa1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:25.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.628+0000 7f4d91d9b700 1 --2- 172.21.15.7:0/501148582 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5570b6a9fcc0 0x5570b6aa1130 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=543fd6b40553890c server_cookie=9460d6e771f8f929 in_seq=0 out_seq=0 2024-06-10T01:06:25.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.628+0000 7f4d79ffb700 10 client.15438 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:25.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.629+0000 7f4d79ffb700 1 -- 172.21.15.7:0/501148582 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5570b69e0cc0 con 0x5570b6a9fcc0 2024-06-10T01:06:25.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.629+0000 7f4d79ffb700 10 client.15438 handle_command_reply: tid=0 2024-06-10T01:06:25.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.629+0000 7f4da351eb80 1 client.15438 shutdown 2024-06-10T01:06:25.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.629+0000 7f4da351eb80 1 -- 172.21.15.7:0/501148582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b6a862e0 msgr2=0x5570b6a920a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:25.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.629+0000 7f4da351eb80 1 --2- 172.21.15.7:0/501148582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b6a862e0 0x5570b6a920a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4d8400c010 tx=0x7f4d8409c350 comp rx=0 tx=0).stop 2024-06-10T01:06:25.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.629+0000 7f4da351eb80 1 -- 172.21.15.7:0/501148582 shutdown_connections 2024-06-10T01:06:25.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.629+0000 7f4da351eb80 1 --2- 172.21.15.7:0/501148582 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5570b6a9fcc0 0x5570b6aa1130 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.629+0000 7f4da351eb80 1 --2- 172.21.15.7:0/501148582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570b6a8c340 0x5570b6a8f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.629+0000 7f4da351eb80 1 --2- 172.21.15.7:0/501148582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b6a862e0 0x5570b6a920a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.629+0000 7f4d79ffb700 0 client.15438 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:25.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.629+0000 7f4da351eb80 1 --2- 172.21.15.7:0/501148582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570b6a8ba20 0x5570b6a8f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.629+0000 7f4da351eb80 1 -- 172.21.15.7:0/501148582 >> 172.21.15.7:0/501148582 conn(0x5570b69acc70 msgr2=0x5570b6a80980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:25.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.629+0000 7f4da351eb80 1 -- 172.21.15.7:0/501148582 shutdown_connections 2024-06-10T01:06:25.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.629+0000 7f4da351eb80 1 -- 172.21.15.7:0/501148582 wait complete. 2024-06-10T01:06:25.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.629+0000 7f4da351eb80 20 client.15438 trim_cache size 0 max 16384 2024-06-10T01:06:25.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.651+0000 7f4da351eb80 1 Processor -- start 2024-06-10T01:06:25.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.651+0000 7f4da351eb80 1 -- start start 2024-06-10T01:06:25.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.651+0000 7f4da351eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b6a862e0 0x5570b6a8f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:25.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.651+0000 7f4da351eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570b6a8ba20 0x5570b6a8dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:25.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.651+0000 7f4da351eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570b6a8c340 0x5570b6a8e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:25.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.651+0000 7f4da351eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5570b6a9f470 con 0x5570b6a862e0 2024-06-10T01:06:25.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.651+0000 7f4da351eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5570b6a9fb40 con 0x5570b6a8ba20 2024-06-10T01:06:25.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.651+0000 7f4da351eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5570b6aa0210 con 0x5570b6a8c340 2024-06-10T01:06:25.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.651+0000 7f4d9259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b6a862e0 0x5570b6a8f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:25.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.651+0000 7f4d9259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b6a862e0 0x5570b6a8f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57166/0 (socket says 172.21.15.7:57166) 2024-06-10T01:06:25.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.651+0000 7f4d9259c700 1 -- 172.21.15.7:0/4180774525 learned_addr learned my addr 172.21.15.7:0/4180774525 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:25.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.651+0000 7f4d91d9b700 1 --2- 172.21.15.7:0/4180774525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570b6a8ba20 0x5570b6a8dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:25.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.651+0000 7f4d92d9d700 1 --2- 172.21.15.7:0/4180774525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570b6a8c340 0x5570b6a8e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:25.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.651+0000 7f4d9259c700 1 -- 172.21.15.7:0/4180774525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570b6a8c340 msgr2=0x5570b6a8e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:25.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.651+0000 7f4d9259c700 1 --2- 172.21.15.7:0/4180774525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570b6a8c340 0x5570b6a8e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.651+0000 7f4d9259c700 1 -- 172.21.15.7:0/4180774525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570b6a8ba20 msgr2=0x5570b6a8dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:25.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.651+0000 7f4d9259c700 1 --2- 172.21.15.7:0/4180774525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570b6a8ba20 0x5570b6a8dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.651+0000 7f4d9259c700 1 -- 172.21.15.7:0/4180774525 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570b6a91ce0 con 0x5570b6a862e0 2024-06-10T01:06:25.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.651+0000 7f4d91d9b700 1 --2- 172.21.15.7:0/4180774525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570b6a8ba20 0x5570b6a8dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:06:25.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.652+0000 7f4d9259c700 1 --2- 172.21.15.7:0/4180774525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b6a862e0 0x5570b6a8f980 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4d840cdcc0 tx=0x7f4d84005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec1caaacebada1a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:25.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.652+0000 7f4d7bfff700 1 -- 172.21.15.7:0/4180774525 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d840a0af0 con 0x5570b6a862e0 2024-06-10T01:06:25.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.652+0000 7f4d7bfff700 1 -- 172.21.15.7:0/4180774525 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d840a5320 con 0x5570b6a862e0 2024-06-10T01:06:25.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.652+0000 7f4d7bfff700 1 -- 172.21.15.7:0/4180774525 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8409b600 con 0x5570b6a862e0 2024-06-10T01:06:25.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.652+0000 7f4da351eb80 1 -- 172.21.15.7:0/4180774525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b6a862e0 msgr2=0x5570b6a8f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:25.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.652+0000 7f4da351eb80 1 --2- 172.21.15.7:0/4180774525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b6a862e0 0x5570b6a8f980 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4d840cdcc0 tx=0x7f4d84005a90 comp rx=0 tx=0).stop 2024-06-10T01:06:25.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.652+0000 7f4da351eb80 1 -- 172.21.15.7:0/4180774525 shutdown_connections 2024-06-10T01:06:25.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.652+0000 7f4da351eb80 1 --2- 172.21.15.7:0/4180774525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570b6a8c340 0x5570b6a8e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.652+0000 7f4da351eb80 1 --2- 172.21.15.7:0/4180774525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b6a862e0 0x5570b6a8f980 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.652+0000 7f4da351eb80 1 --2- 172.21.15.7:0/4180774525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570b6a8ba20 0x5570b6a8dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.652+0000 7f4da351eb80 1 -- 172.21.15.7:0/4180774525 >> 172.21.15.7:0/4180774525 conn(0x5570b69acc70 msgr2=0x5570b6a7c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:25.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.652+0000 7f4da351eb80 1 -- 172.21.15.7:0/4180774525 shutdown_connections 2024-06-10T01:06:25.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.652+0000 7f4da351eb80 1 -- 172.21.15.7:0/4180774525 wait complete. 2024-06-10T01:06:25.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.653+0000 7f4da351eb80 1 -- start start 2024-06-10T01:06:25.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.653+0000 7f4da351eb80 1 Processor -- start 2024-06-10T01:06:25.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.653+0000 7f4da351eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570b6a862e0 0x5570b6aaa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:25.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.653+0000 7f4da351eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b6a8ba20 0x5570b6aaaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:25.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.653+0000 7f4da351eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570b6a8c340 0x5570b6aa5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:25.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.653+0000 7f4da351eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5570b6aa0fb0 con 0x5570b6a8ba20 2024-06-10T01:06:25.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.653+0000 7f4da351eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5570b6aa08e0 con 0x5570b6a8c340 2024-06-10T01:06:25.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.653+0000 7f4da351eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5570b6aa1680 con 0x5570b6a862e0 2024-06-10T01:06:25.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.653+0000 7f4da351eb80 10 client.0 _ll_register_callbacks cb 0x5570b696d8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:25.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.653+0000 7f4d91d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b6a8ba20 0x5570b6aaaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:25.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.653+0000 7f4d91d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b6a8ba20 0x5570b6aaaaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57182/0 (socket says 172.21.15.7:57182) 2024-06-10T01:06:25.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.653+0000 7f4d91d9b700 1 -- 172.21.15.7:0/2695546546 learned_addr learned my addr 172.21.15.7:0/2695546546 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:25.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.653+0000 7f4d9259c700 1 --2- 172.21.15.7:0/2695546546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570b6a862e0 0x5570b6aaa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:25.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.653+0000 7f4d92d9d700 1 --2- 172.21.15.7:0/2695546546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570b6a8c340 0x5570b6aa5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:25.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.654+0000 7f4d91d9b700 1 -- 172.21.15.7:0/2695546546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570b6a862e0 msgr2=0x5570b6aaa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:25.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.654+0000 7f4d91d9b700 1 --2- 172.21.15.7:0/2695546546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570b6a862e0 0x5570b6aaa5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.654+0000 7f4d91d9b700 1 -- 172.21.15.7:0/2695546546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570b6a8c340 msgr2=0x5570b6aa5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:25.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.654+0000 7f4d91d9b700 1 --2- 172.21.15.7:0/2695546546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570b6a8c340 0x5570b6aa5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.654+0000 7f4d91d9b700 1 -- 172.21.15.7:0/2695546546 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570b6aa5b50 con 0x5570b6a8ba20 2024-06-10T01:06:25.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.654+0000 7f4d92d9d700 1 --2- 172.21.15.7:0/2695546546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570b6a8c340 0x5570b6aa5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:06:25.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.654+0000 7f4d91d9b700 1 --2- 172.21.15.7:0/2695546546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b6a8ba20 0x5570b6aaaaf0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x5570b6a90970 tx=0x7f4d880047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316d34252b40b7b1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:25.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.654+0000 7f4d79ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:06:25.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.654+0000 7f4d79ffb700 1 -- 172.21.15.7:0/2695546546 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d88016070 con 0x5570b6a8ba20 2024-06-10T01:06:25.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.654+0000 7f4d79ffb700 1 -- 172.21.15.7:0/2695546546 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f4d880262d0 con 0x5570b6a8ba20 2024-06-10T01:06:25.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.654+0000 7f4da351eb80 1 -- 172.21.15.7:0/2695546546 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d840a5320 con 0x5570b6a8ba20 2024-06-10T01:06:25.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.654+0000 7f4d79ffb700 1 -- 172.21.15.7:0/2695546546 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4d88034050 con 0x5570b6a8ba20 2024-06-10T01:06:25.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.654+0000 7f4da351eb80 10 client.15444 fetch_fsmap learned FSMap version 27 2024-06-10T01:06:25.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.654+0000 7f4da351eb80 1 -- 172.21.15.7:0/2695546546 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5570b6b0be60 con 0x5570b6a8ba20 2024-06-10T01:06:25.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.654+0000 7f4d79ffb700 1 -- 172.21.15.7:0/2695546546 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d840a5320 con 0x5570b6a8ba20 2024-06-10T01:06:25.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.655+0000 7f4d79ffb700 1 -- 172.21.15.7:0/2695546546 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4d88019040 con 0x5570b6a8ba20 2024-06-10T01:06:25.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.655+0000 7f4da351eb80 10 client.15444 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:06:25.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.655+0000 7f4da351eb80 10 client.15444 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-10T01:06:25.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.655+0000 7f4da351eb80 20 client.15444 populate_metadata read hostname 'smithi007' 2024-06-10T01:06:25.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.655+0000 7f4da351eb80 1 --2- 172.21.15.7:0/2695546546 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5570b6b10ae0 0x5570b6b0f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:25.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.655+0000 7f4da351eb80 4 client.15444 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T01:06:25.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.655+0000 7f4da351eb80 1 -- 172.21.15.7:0/2695546546 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "status"}) -- 0x5570b69db240 con 0x5570b6b10ae0 2024-06-10T01:06:25.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.655+0000 7f4d9259c700 1 --2- 172.21.15.7:0/2695546546 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5570b6b10ae0 0x5570b6b0f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:25.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.655+0000 7f4d9259c700 1 --2- 172.21.15.7:0/2695546546 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5570b6b10ae0 0x5570b6b0f7f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bc28f0e3572343b server_cookie=8969d348dfb9923 in_seq=0 out_seq=0 2024-06-10T01:06:25.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.655+0000 7f4d79ffb700 10 client.15444 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:25.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.655+0000 7f4d79ffb700 1 -- 172.21.15.7:0/2695546546 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5570b69db240 con 0x5570b6b10ae0 2024-06-10T01:06:25.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.655+0000 7f4d79ffb700 10 client.15444 handle_command_reply: tid=0 2024-06-10T01:06:25.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.655+0000 7f4da351eb80 1 client.15444 shutdown 2024-06-10T01:06:25.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.656+0000 7f4da351eb80 1 -- 172.21.15.7:0/2695546546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b6a8ba20 msgr2=0x5570b6aaaaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:25.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.656+0000 7f4da351eb80 1 --2- 172.21.15.7:0/2695546546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b6a8ba20 0x5570b6aaaaf0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x5570b6a90970 tx=0x7f4d880047c0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.656+0000 7f4da351eb80 1 -- 172.21.15.7:0/2695546546 shutdown_connections 2024-06-10T01:06:25.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.656+0000 7f4da351eb80 1 --2- 172.21.15.7:0/2695546546 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5570b6b10ae0 0x5570b6b0f7f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.656+0000 7f4da351eb80 1 --2- 172.21.15.7:0/2695546546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570b6a8c340 0x5570b6aa5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.656+0000 7f4da351eb80 1 --2- 172.21.15.7:0/2695546546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570b6a862e0 0x5570b6aaa5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.656+0000 7f4d79ffb700 0 client.15444 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:25.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.656+0000 7f4da351eb80 1 --2- 172.21.15.7:0/2695546546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b6a8ba20 0x5570b6aaaaf0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.656+0000 7f4da351eb80 1 -- 172.21.15.7:0/2695546546 >> 172.21.15.7:0/2695546546 conn(0x5570b69acc70 msgr2=0x5570b6a8dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:25.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.656+0000 7f4da351eb80 1 -- 172.21.15.7:0/2695546546 shutdown_connections 2024-06-10T01:06:25.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.656+0000 7f4da351eb80 1 -- 172.21.15.7:0/2695546546 wait complete. 2024-06-10T01:06:25.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.656+0000 7f4da351eb80 20 client.15444 trim_cache size 0 max 16384 2024-06-10T01:06:25.659 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:06:25.659 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "bded58f8-26c3-11ef-bc9b-c7b262605968", 2024-06-10T01:06:25.659 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-06-10T01:06:25.660 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24451, 2024-06-10T01:06:25.660 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-06-10T01:06:25.660 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-06-10T01:06:25.660 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-06-10T01:06:25.660 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 276.41896894899998, 2024-06-10T01:06:25.660 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 27, 2024-06-10T01:06:25.660 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 99, 2024-06-10T01:06:25.660 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 99, 2024-06-10T01:06:25.660 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 303.51421950700001 2024-06-10T01:06:25.660 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:06:25.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.656+0000 7f4d98c9b700 1 -- 172.21.15.7:0/1324337967 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4d7c0a2c20 msgr2=0x7f4d7c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:25.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.656+0000 7f4d98c9b700 1 --2- 172.21.15.7:0/1324337967 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4d7c0a2c20 0x7f4d7c0a50e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4d94108c80 tx=0x7f4d8800a380 comp rx=0 tx=0).stop 2024-06-10T01:06:25.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.656+0000 7f4d98c9b700 1 -- 172.21.15.7:0/1324337967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d94102740 msgr2=0x7f4d9419e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:25.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.656+0000 7f4d98c9b700 1 --2- 172.21.15.7:0/1324337967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d94102740 0x7f4d9419e720 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4d84006010 tx=0x7f4d84009f80 comp rx=0 tx=0).stop 2024-06-10T01:06:25.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.656+0000 7f4d98c9b700 1 -- 172.21.15.7:0/1324337967 shutdown_connections 2024-06-10T01:06:25.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.656+0000 7f4d98c9b700 1 --2- 172.21.15.7:0/1324337967 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4d7c0a2c20 0x7f4d7c0a50e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.656+0000 7f4d98c9b700 1 --2- 172.21.15.7:0/1324337967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94107830 0x7f4d94198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.656+0000 7f4d98c9b700 1 --2- 172.21.15.7:0/1324337967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d94102740 0x7f4d9419e720 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.656+0000 7f4d98c9b700 1 --2- 172.21.15.7:0/1324337967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d941030f0 0x7f4d9419ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:25.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.656+0000 7f4d98c9b700 1 -- 172.21.15.7:0/1324337967 >> 172.21.15.7:0/1324337967 conn(0x7f4d940795b0 msgr2=0x7f4d94100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:25.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.656+0000 7f4d98c9b700 1 -- 172.21.15.7:0/1324337967 shutdown_connections 2024-06-10T01:06:25.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:25.656+0000 7f4d98c9b700 1 -- 172.21.15.7:0/1324337967 wait complete. 2024-06-10T01:06:25.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:25 smithi175 ceph-mon[36636]: pgmap v443: 129 pgs: 129 active+clean; 660 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-06-10T01:06:26.520 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-06-10T01:06:27.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:27 smithi175 ceph-mon[36636]: pgmap v444: 129 pgs: 129 active+clean; 660 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-06-10T01:06:27.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:27 smithi007 ceph-mon[32398]: pgmap v444: 129 pgs: 129 active+clean; 660 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-06-10T01:06:27.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:27 smithi078 ceph-mon[36683]: pgmap v444: 129 pgs: 129 active+clean; 660 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-06-10T01:06:29.670 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:06:29.694 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:29 smithi007 ceph-mon[32398]: pgmap v445: 129 pgs: 129 active+clean; 660 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-06-10T01:06:29.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:29 smithi175 ceph-mon[36636]: pgmap v445: 129 pgs: 129 active+clean; 660 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-06-10T01:06:29.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:29 smithi078 ceph-mon[36683]: pgmap v445: 129 pgs: 129 active+clean; 660 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-06-10T01:06:29.931 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:06:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.090+0000 7f15121b0640 1 -- 172.21.15.7:0/4165083727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f150c1024e0 msgr2=0x7f150c1048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.090+0000 7f15121b0640 1 --2- 172.21.15.7:0/4165083727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f150c1024e0 0x7f150c1048d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f14fc009900 tx=0x7f14fc02a410 comp rx=0 tx=0).stop 2024-06-10T01:06:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.090+0000 7f15121b0640 1 -- 172.21.15.7:0/4165083727 shutdown_connections 2024-06-10T01:06:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.090+0000 7f15121b0640 1 --2- 172.21.15.7:0/4165083727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f150c107740 0x7f150c109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.090+0000 7f15121b0640 1 --2- 172.21.15.7:0/4165083727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f150c104e10 0x7f150c107200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.090+0000 7f15121b0640 1 --2- 172.21.15.7:0/4165083727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f150c1024e0 0x7f150c1048d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.090+0000 7f15121b0640 1 -- 172.21.15.7:0/4165083727 >> 172.21.15.7:0/4165083727 conn(0x7f150c0fbe70 msgr2=0x7f150c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.090+0000 7f15121b0640 1 -- 172.21.15.7:0/4165083727 shutdown_connections 2024-06-10T01:06:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.091+0000 7f15121b0640 1 -- 172.21.15.7:0/4165083727 wait complete. 2024-06-10T01:06:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.091+0000 7f15121b0640 1 Processor -- start 2024-06-10T01:06:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.091+0000 7f15121b0640 1 -- start start 2024-06-10T01:06:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.091+0000 7f15121b0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f150c1024e0 0x7f150c197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.091+0000 7f15121b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f150c104e10 0x7f150c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.092+0000 7f15121b0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f150c107740 0x7f150c19f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.092+0000 7f15121b0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f150c10d670 con 0x7f150c1024e0 2024-06-10T01:06:30.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.092+0000 7f15121b0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f150c10d4f0 con 0x7f150c104e10 2024-06-10T01:06:30.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.092+0000 7f15121b0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f150c10d7f0 con 0x7f150c107740 2024-06-10T01:06:30.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.092+0000 7f150b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f150c1024e0 0x7f150c197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:30.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.092+0000 7f150b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f150c1024e0 0x7f150c197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57202/0 (socket says 172.21.15.7:57202) 2024-06-10T01:06:30.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.092+0000 7f150b7fe640 1 -- 172.21.15.7:0/520354291 learned_addr learned my addr 172.21.15.7:0/520354291 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:30.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.092+0000 7f150affd640 1 --2- 172.21.15.7:0/520354291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f150c104e10 0x7f150c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:30.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.092+0000 7f150bfff640 1 --2- 172.21.15.7:0/520354291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f150c107740 0x7f150c19f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:30.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.092+0000 7f150b7fe640 1 -- 172.21.15.7:0/520354291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f150c107740 msgr2=0x7f150c19f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:30.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.092+0000 7f150b7fe640 1 --2- 172.21.15.7:0/520354291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f150c107740 0x7f150c19f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.092+0000 7f150b7fe640 1 -- 172.21.15.7:0/520354291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f150c104e10 msgr2=0x7f150c198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:30.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.092+0000 7f150b7fe640 1 --2- 172.21.15.7:0/520354291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f150c104e10 0x7f150c198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.092+0000 7f150b7fe640 1 -- 172.21.15.7:0/520354291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150c19f960 con 0x7f150c1024e0 2024-06-10T01:06:30.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.092+0000 7f150b7fe640 1 --2- 172.21.15.7:0/520354291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f150c1024e0 0x7f150c197d40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f14fc009230 tx=0x7f14fc004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:30.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.092+0000 7f1508ff9640 1 -- 172.21.15.7:0/520354291 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc03f030 con 0x7f150c1024e0 2024-06-10T01:06:30.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.092+0000 7f1508ff9640 1 -- 172.21.15.7:0/520354291 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14fc003ce0 con 0x7f150c1024e0 2024-06-10T01:06:30.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.092+0000 7f15121b0640 1 -- 172.21.15.7:0/520354291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f150c19fbf0 con 0x7f150c1024e0 2024-06-10T01:06:30.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.093+0000 7f1508ff9640 1 -- 172.21.15.7:0/520354291 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc035070 con 0x7f150c1024e0 2024-06-10T01:06:30.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.093+0000 7f15121b0640 1 -- 172.21.15.7:0/520354291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f150c1a00e0 con 0x7f150c1024e0 2024-06-10T01:06:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.095+0000 7f1508ff9640 1 -- 172.21.15.7:0/520354291 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f14fc042030 con 0x7f150c1024e0 2024-06-10T01:06:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.095+0000 7f1514f9e740 1 Processor -- start 2024-06-10T01:06:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.095+0000 7f1514f9e740 1 -- start start 2024-06-10T01:06:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.095+0000 7f1514f9e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3b2028010 0x55a3b2028430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.095+0000 7f1514f9e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3b2028a00 0x55a3b202d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.095+0000 7f1514f9e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3b202db70 0x55a3b2030000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.095+0000 7f1514f9e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a3b1f8b4f0 con 0x55a3b2028010 2024-06-10T01:06:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.095+0000 7f1514f9e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a3b1f29870 con 0x55a3b2028a00 2024-06-10T01:06:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.095+0000 7f1514f9e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a3b1f874b0 con 0x55a3b202db70 2024-06-10T01:06:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.095+0000 7f1508ff9640 1 --2- 172.21.15.7:0/520354291 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f14f40a2b80 0x7f14f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.096+0000 7f150bfff640 1 --2- 172.21.15.7:0/520354291 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f14f40a2b80 0x7f14f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.096+0000 7f1508ff9640 1 -- 172.21.15.7:0/520354291 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f14fc0efd60 con 0x7f150c1024e0 2024-06-10T01:06:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.096+0000 7f150affd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3b202db70 0x55a3b2030000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.096+0000 7f150affd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3b202db70 0x55a3b2030000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:51846/0 (socket says 172.21.15.7:51846) 2024-06-10T01:06:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.096+0000 7f150b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3b2028a00 0x55a3b202d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.096+0000 7f150affd640 1 -- 172.21.15.7:0/2221588125 learned_addr learned my addr 172.21.15.7:0/2221588125 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.096+0000 7f150bfff640 1 --2- 172.21.15.7:0/2221588125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3b2028010 0x55a3b2028430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.096+0000 7f150affd640 1 -- 172.21.15.7:0/2221588125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3b2028a00 msgr2=0x55a3b202d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.096+0000 7f150affd640 1 --2- 172.21.15.7:0/2221588125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3b2028a00 0x55a3b202d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.096+0000 7f150affd640 1 -- 172.21.15.7:0/2221588125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3b2028010 msgr2=0x55a3b2028430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.096+0000 7f150affd640 1 --2- 172.21.15.7:0/2221588125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3b2028010 0x55a3b2028430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.096+0000 7f150affd640 1 -- 172.21.15.7:0/2221588125 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3b1f459b0 con 0x55a3b202db70 2024-06-10T01:06:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.096+0000 7f150b7fe640 1 --2- 172.21.15.7:0/2221588125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3b2028a00 0x55a3b202d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.096+0000 7f150bfff640 1 --2- 172.21.15.7:0/520354291 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f14f40a2b80 0x7f14f40a5040 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f15040098b0 tx=0x7f1504009e00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.096+0000 7f150affd640 1 --2- 172.21.15.7:0/2221588125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3b202db70 0x55a3b2030000 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f150000bc70 tx=0x7f15000107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1c8863960a8aa9d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.096+0000 7f14f1ffb640 1 -- 172.21.15.7:0/2221588125 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500005020 con 0x55a3b202db70 2024-06-10T01:06:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.097+0000 7f14f1ffb640 1 -- 172.21.15.7:0/2221588125 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15000047a0 con 0x55a3b202db70 2024-06-10T01:06:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.097+0000 7f14f1ffb640 1 -- 172.21.15.7:0/2221588125 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15000116a0 con 0x55a3b202db70 2024-06-10T01:06:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.097+0000 7f1514f9e740 1 -- 172.21.15.7:0/2221588125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3b202db70 msgr2=0x55a3b2030000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.097+0000 7f1514f9e740 1 --2- 172.21.15.7:0/2221588125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3b202db70 0x55a3b2030000 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f150000bc70 tx=0x7f15000107a0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.097+0000 7f1514f9e740 1 -- 172.21.15.7:0/2221588125 shutdown_connections 2024-06-10T01:06:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.097+0000 7f1514f9e740 1 --2- 172.21.15.7:0/2221588125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3b202db70 0x55a3b2030000 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.097+0000 7f1514f9e740 1 --2- 172.21.15.7:0/2221588125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3b2028a00 0x55a3b202d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.097+0000 7f1514f9e740 1 --2- 172.21.15.7:0/2221588125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3b2028010 0x55a3b2028430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.097+0000 7f1514f9e740 1 -- 172.21.15.7:0/2221588125 >> 172.21.15.7:0/2221588125 conn(0x55a3b2023680 msgr2=0x55a3b2023a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.097+0000 7f1514f9e740 1 -- 172.21.15.7:0/2221588125 shutdown_connections 2024-06-10T01:06:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.097+0000 7f1514f9e740 1 -- 172.21.15.7:0/2221588125 wait complete. 2024-06-10T01:06:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.098+0000 7f1514f9e740 1 -- start start 2024-06-10T01:06:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.098+0000 7f1514f9e740 1 Processor -- start 2024-06-10T01:06:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.098+0000 7f1514f9e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3b2028010 0x55a3b20322f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.098+0000 7f1514f9e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3b2028a00 0x55a3b2032830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.098+0000 7f1514f9e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3b202db70 0x55a3b2039870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.098+0000 7f1514f9e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a3b2030b20 con 0x55a3b2028010 2024-06-10T01:06:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.098+0000 7f1514f9e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a3b1f14ae0 con 0x55a3b2028a00 2024-06-10T01:06:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.098+0000 7f1514f9e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a3b2030ca0 con 0x55a3b202db70 2024-06-10T01:06:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.098+0000 7f1514f9e740 10 client.0 _ll_register_callbacks cb 0x55a3b1f4d9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.098+0000 7f150b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3b2028010 0x55a3b20322f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.098+0000 7f150b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3b2028010 0x55a3b20322f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57220/0 (socket says 172.21.15.7:57220) 2024-06-10T01:06:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.098+0000 7f150b7fe640 1 -- 172.21.15.7:0/3361218969 learned_addr learned my addr 172.21.15.7:0/3361218969 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.098+0000 7f150bfff640 1 --2- 172.21.15.7:0/3361218969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3b202db70 0x55a3b2039870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.098+0000 7f150affd640 1 --2- 172.21.15.7:0/3361218969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3b2028a00 0x55a3b2032830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.099+0000 7f150b7fe640 1 -- 172.21.15.7:0/3361218969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3b202db70 msgr2=0x55a3b2039870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.099+0000 7f150b7fe640 1 --2- 172.21.15.7:0/3361218969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3b202db70 0x55a3b2039870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.099+0000 7f150b7fe640 1 -- 172.21.15.7:0/3361218969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3b2028a00 msgr2=0x55a3b2032830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.099+0000 7f150b7fe640 1 --2- 172.21.15.7:0/3361218969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3b2028a00 0x55a3b2032830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.099+0000 7f150b7fe640 1 -- 172.21.15.7:0/3361218969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3b1c9f210 con 0x55a3b2028010 2024-06-10T01:06:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.099+0000 7f150affd640 1 --2- 172.21.15.7:0/3361218969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3b2028a00 0x55a3b2032830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:06:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.099+0000 7f150bfff640 1 --2- 172.21.15.7:0/3361218969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3b202db70 0x55a3b2039870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.099+0000 7f150b7fe640 1 --2- 172.21.15.7:0/3361218969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3b2028010 0x55a3b20322f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f14fc0314e0 tx=0x7f14fc0bbe30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4ff9b419e5fddc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.099+0000 7f14eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:06:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.099+0000 7f14eaffd640 1 -- 172.21.15.7:0/3361218969 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc049040 con 0x55a3b2028010 2024-06-10T01:06:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.099+0000 7f14eaffd640 1 -- 172.21.15.7:0/3361218969 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f14fc0b8960 con 0x55a3b2028010 2024-06-10T01:06:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.099+0000 7f1514f9e740 1 -- 172.21.15.7:0/3361218969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15000047a0 con 0x55a3b2028010 2024-06-10T01:06:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.099+0000 7f14eaffd640 1 -- 172.21.15.7:0/3361218969 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f14fc0c05f0 con 0x55a3b2028010 2024-06-10T01:06:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.099+0000 7f1514f9e740 10 client.15456 fetch_fsmap learned FSMap version 27 2024-06-10T01:06:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.099+0000 7f1514f9e740 1 -- 172.21.15.7:0/3361218969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a3b1d6b620 con 0x55a3b2028010 2024-06-10T01:06:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.099+0000 7f14eaffd640 1 -- 172.21.15.7:0/3361218969 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15000047a0 con 0x55a3b2028010 2024-06-10T01:06:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.099+0000 7f14eaffd640 1 -- 172.21.15.7:0/3361218969 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f14fc0bc5e0 con 0x55a3b2028010 2024-06-10T01:06:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.100+0000 7f1514f9e740 10 client.15456 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:06:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.100+0000 7f1514f9e740 10 client.15456 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:06:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.100+0000 7f1514f9e740 20 client.15456 populate_metadata read hostname 'smithi007' 2024-06-10T01:06:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.100+0000 7f1514f9e740 1 --2- 172.21.15.7:0/3361218969 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a3b203e170 0x55a3b20405e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.100+0000 7f1514f9e740 4 client.15456 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:06:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.100+0000 7f1514f9e740 1 -- 172.21.15.7:0/3361218969 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3b1f9ef10 con 0x55a3b203e170 2024-06-10T01:06:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.100+0000 7f150affd640 1 --2- 172.21.15.7:0/3361218969 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a3b203e170 0x55a3b20405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.100+0000 7f150affd640 1 --2- 172.21.15.7:0/3361218969 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a3b203e170 0x55a3b20405e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d04518838f9e161 server_cookie=c0ab26bc464ebc76 in_seq=0 out_seq=0 2024-06-10T01:06:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.100+0000 7f14eaffd640 10 client.15456 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.101+0000 7f14eaffd640 1 -- 172.21.15.7:0/3361218969 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a3b1f9ef10 con 0x55a3b203e170 2024-06-10T01:06:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.101+0000 7f14eaffd640 10 client.15456 handle_command_reply: tid=0 2024-06-10T01:06:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.101+0000 7f1514f9e740 1 client.15456 shutdown 2024-06-10T01:06:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.101+0000 7f1514f9e740 1 -- 172.21.15.7:0/3361218969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3b2028010 msgr2=0x55a3b20322f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.101+0000 7f1514f9e740 1 --2- 172.21.15.7:0/3361218969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3b2028010 0x55a3b20322f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f14fc0314e0 tx=0x7f14fc0bbe30 comp rx=0 tx=0).stop 2024-06-10T01:06:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.101+0000 7f1514f9e740 1 -- 172.21.15.7:0/3361218969 shutdown_connections 2024-06-10T01:06:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.101+0000 7f1514f9e740 1 --2- 172.21.15.7:0/3361218969 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a3b203e170 0x55a3b20405e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.101+0000 7f1514f9e740 1 --2- 172.21.15.7:0/3361218969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3b202db70 0x55a3b2039870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.101+0000 7f1514f9e740 1 --2- 172.21.15.7:0/3361218969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3b2028a00 0x55a3b2032830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.101+0000 7f1514f9e740 1 --2- 172.21.15.7:0/3361218969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3b2028010 0x55a3b20322f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.101+0000 7f1514f9e740 1 -- 172.21.15.7:0/3361218969 >> 172.21.15.7:0/3361218969 conn(0x55a3b2023680 msgr2=0x55a3b20227e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.101+0000 7f14eaffd640 0 client.15456 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.101+0000 7f1514f9e740 1 -- 172.21.15.7:0/3361218969 shutdown_connections 2024-06-10T01:06:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.101+0000 7f1514f9e740 1 -- 172.21.15.7:0/3361218969 wait complete. 2024-06-10T01:06:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.102+0000 7f1514f9e740 20 client.15456 trim_cache size 0 max 16384 2024-06-10T01:06:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.117+0000 7f1514f9e740 1 Processor -- start 2024-06-10T01:06:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.117+0000 7f1514f9e740 1 -- start start 2024-06-10T01:06:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.117+0000 7f1514f9e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3b2028010 0x55a3b2029c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.117+0000 7f1514f9e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3b2028a00 0x55a3b2039830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.117+0000 7f1514f9e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3b202db70 0x55a3b202a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.117+0000 7f1514f9e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1acdd0 con 0x55a3b202db70 2024-06-10T01:06:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.117+0000 7f1514f9e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1aa6a0 con 0x55a3b2028a00 2024-06-10T01:06:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.117+0000 7f1514f9e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1a84f0 con 0x55a3b2028010 2024-06-10T01:06:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.117+0000 7f150bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3b202db70 0x55a3b202a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.117+0000 7f150bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3b202db70 0x55a3b202a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57232/0 (socket says 172.21.15.7:57232) 2024-06-10T01:06:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.117+0000 7f150bfff640 1 -- 172.21.15.7:0/4234920512 learned_addr learned my addr 172.21.15.7:0/4234920512 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.117+0000 7f150b7fe640 1 --2- 172.21.15.7:0/4234920512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3b2028010 0x55a3b2029c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.117+0000 7f150affd640 1 --2- 172.21.15.7:0/4234920512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3b2028a00 0x55a3b2039830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.118+0000 7f150bfff640 1 -- 172.21.15.7:0/4234920512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3b2028010 msgr2=0x55a3b2029c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.118+0000 7f150bfff640 1 --2- 172.21.15.7:0/4234920512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3b2028010 0x55a3b2029c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.118+0000 7f150bfff640 1 -- 172.21.15.7:0/4234920512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3b2028a00 msgr2=0x55a3b2039830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.118+0000 7f150bfff640 1 --2- 172.21.15.7:0/4234920512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3b2028a00 0x55a3b2039830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.118+0000 7f150bfff640 1 -- 172.21.15.7:0/4234920512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3b202d8c0 con 0x55a3b202db70 2024-06-10T01:06:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.118+0000 7f150bfff640 1 --2- 172.21.15.7:0/4234920512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3b202db70 0x55a3b202a1d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1504009470 tx=0x7f1504004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71c78851fea422c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.118+0000 7f14f17fa640 1 -- 172.21.15.7:0/4234920512 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150402f070 con 0x55a3b202db70 2024-06-10T01:06:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.118+0000 7f14f17fa640 1 -- 172.21.15.7:0/4234920512 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f150402a410 con 0x55a3b202db70 2024-06-10T01:06:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.118+0000 7f14f17fa640 1 -- 172.21.15.7:0/4234920512 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1504020400 con 0x55a3b202db70 2024-06-10T01:06:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.119+0000 7f1514f9e740 1 -- 172.21.15.7:0/4234920512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3b202db70 msgr2=0x55a3b202a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.119+0000 7f1514f9e740 1 --2- 172.21.15.7:0/4234920512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3b202db70 0x55a3b202a1d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1504009470 tx=0x7f1504004a40 comp rx=0 tx=0).stop 2024-06-10T01:06:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.119+0000 7f1514f9e740 1 -- 172.21.15.7:0/4234920512 shutdown_connections 2024-06-10T01:06:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.119+0000 7f1514f9e740 1 --2- 172.21.15.7:0/4234920512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3b202db70 0x55a3b202a1d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.119+0000 7f1514f9e740 1 --2- 172.21.15.7:0/4234920512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3b2028a00 0x55a3b2039830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.119+0000 7f1514f9e740 1 --2- 172.21.15.7:0/4234920512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3b2028010 0x55a3b2029c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.119+0000 7f1514f9e740 1 -- 172.21.15.7:0/4234920512 >> 172.21.15.7:0/4234920512 conn(0x55a3b2023680 msgr2=0x55a3b2023a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.119+0000 7f1514f9e740 1 -- 172.21.15.7:0/4234920512 shutdown_connections 2024-06-10T01:06:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.119+0000 7f1514f9e740 1 -- 172.21.15.7:0/4234920512 wait complete. 2024-06-10T01:06:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.119+0000 7f1514f9e740 1 -- start start 2024-06-10T01:06:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.119+0000 7f1514f9e740 1 Processor -- start 2024-06-10T01:06:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.120+0000 7f1514f9e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3b2028010 0x55a3b2039890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.120+0000 7f1514f9e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3b2028a00 0x55a3b2036330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.120+0000 7f1514f9e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3b202db70 0x55a3b2036960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.120+0000 7f1514f9e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1a8b60 con 0x55a3b2028a00 2024-06-10T01:06:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.120+0000 7f1514f9e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1a9fd0 con 0x55a3b202db70 2024-06-10T01:06:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.120+0000 7f1514f9e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1a9230 con 0x55a3b2028010 2024-06-10T01:06:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.120+0000 7f1514f9e740 10 client.0 _ll_register_callbacks cb 0x7f14fc03dfc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.120+0000 7f150affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3b2028a00 0x55a3b2036330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.120+0000 7f150affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3b2028a00 0x55a3b2036330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57248/0 (socket says 172.21.15.7:57248) 2024-06-10T01:06:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.120+0000 7f150affd640 1 -- 172.21.15.7:0/4070066710 learned_addr learned my addr 172.21.15.7:0/4070066710 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.120+0000 7f150bfff640 1 --2- 172.21.15.7:0/4070066710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3b202db70 0x55a3b2036960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.120+0000 7f150b7fe640 1 --2- 172.21.15.7:0/4070066710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3b2028010 0x55a3b2039890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.120+0000 7f150affd640 1 -- 172.21.15.7:0/4070066710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3b2028010 msgr2=0x55a3b2039890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.120+0000 7f150affd640 1 --2- 172.21.15.7:0/4070066710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3b2028010 0x55a3b2039890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.120+0000 7f150affd640 1 -- 172.21.15.7:0/4070066710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3b202db70 msgr2=0x55a3b2036960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.120+0000 7f150affd640 1 --2- 172.21.15.7:0/4070066710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3b202db70 0x55a3b2036960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.120+0000 7f150affd640 1 -- 172.21.15.7:0/4070066710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3b20a6f00 con 0x55a3b2028a00 2024-06-10T01:06:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.120+0000 7f150bfff640 1 --2- 172.21.15.7:0/4070066710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3b202db70 0x55a3b2036960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:06:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.120+0000 7f150affd640 1 --2- 172.21.15.7:0/4070066710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3b2028a00 0x55a3b2036330 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x55a3b2024550 tx=0x7f150000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb265a9739781a9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.120+0000 7f14eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:06:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.120+0000 7f14eaffd640 1 -- 172.21.15.7:0/4070066710 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150002f060 con 0x55a3b2028a00 2024-06-10T01:06:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.120+0000 7f14eaffd640 1 -- 172.21.15.7:0/4070066710 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f1500004450 con 0x55a3b2028a00 2024-06-10T01:06:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.120+0000 7f1514f9e740 1 -- 172.21.15.7:0/4070066710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f150402a410 con 0x55a3b2028a00 2024-06-10T01:06:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.121+0000 7f14eaffd640 1 -- 172.21.15.7:0/4070066710 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1500036050 con 0x55a3b2028a00 2024-06-10T01:06:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.121+0000 7f1514f9e740 10 client.15462 fetch_fsmap learned FSMap version 27 2024-06-10T01:06:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.121+0000 7f1514f9e740 1 -- 172.21.15.7:0/4070066710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a3b20a8240 con 0x55a3b2028a00 2024-06-10T01:06:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.121+0000 7f14eaffd640 1 -- 172.21.15.7:0/4070066710 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f150402a410 con 0x55a3b2028a00 2024-06-10T01:06:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.121+0000 7f14eaffd640 1 -- 172.21.15.7:0/4070066710 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f15000035f0 con 0x55a3b2028a00 2024-06-10T01:06:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.121+0000 7f1514f9e740 10 client.15462 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:06:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.121+0000 7f1514f9e740 10 client.15462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:06:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.121+0000 7f1514f9e740 20 client.15462 populate_metadata read hostname 'smithi007' 2024-06-10T01:06:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.121+0000 7f1514f9e740 1 --2- 172.21.15.7:0/4070066710 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a3b20a88b0 0x55a3b20aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.121+0000 7f1514f9e740 4 client.15462 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:06:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.121+0000 7f1514f9e740 1 -- 172.21.15.7:0/4070066710 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a3b1f98c20 con 0x55a3b20a88b0 2024-06-10T01:06:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.121+0000 7f150b7fe640 1 --2- 172.21.15.7:0/4070066710 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a3b20a88b0 0x55a3b20aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.122+0000 7f14eaffd640 10 client.15462 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.122+0000 7f150b7fe640 1 --2- 172.21.15.7:0/4070066710 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a3b20a88b0 0x55a3b20aaca0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce28831b52d05c68 server_cookie=85346752520727e1 in_seq=0 out_seq=0 2024-06-10T01:06:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.125+0000 7f14eaffd640 1 -- 172.21.15.7:0/4070066710 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a3b1f98c20 con 0x55a3b20a88b0 2024-06-10T01:06:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.125+0000 7f14eaffd640 10 client.15462 handle_command_reply: tid=0 2024-06-10T01:06:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.125+0000 7f1514f9e740 1 client.15462 shutdown 2024-06-10T01:06:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.126+0000 7f1514f9e740 1 -- 172.21.15.7:0/4070066710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3b2028a00 msgr2=0x55a3b2036330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.126+0000 7f1514f9e740 1 --2- 172.21.15.7:0/4070066710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3b2028a00 0x55a3b2036330 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x55a3b2024550 tx=0x7f150000be50 comp rx=0 tx=0).stop 2024-06-10T01:06:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.126+0000 7f1514f9e740 1 -- 172.21.15.7:0/4070066710 shutdown_connections 2024-06-10T01:06:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.126+0000 7f1514f9e740 1 --2- 172.21.15.7:0/4070066710 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a3b20a88b0 0x55a3b20aaca0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.126+0000 7f1514f9e740 1 --2- 172.21.15.7:0/4070066710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3b202db70 0x55a3b2036960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.126+0000 7f1514f9e740 1 --2- 172.21.15.7:0/4070066710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3b2028a00 0x55a3b2036330 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.126+0000 7f14eaffd640 0 client.15462 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:30.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.126+0000 7f1514f9e740 1 --2- 172.21.15.7:0/4070066710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3b2028010 0x55a3b2039890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.126+0000 7f1514f9e740 1 -- 172.21.15.7:0/4070066710 >> 172.21.15.7:0/4070066710 conn(0x55a3b2023680 msgr2=0x55a3b2029c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:30.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.126+0000 7f1514f9e740 1 -- 172.21.15.7:0/4070066710 shutdown_connections 2024-06-10T01:06:30.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.126+0000 7f1514f9e740 1 -- 172.21.15.7:0/4070066710 wait complete. 2024-06-10T01:06:30.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.126+0000 7f1514f9e740 20 client.15462 trim_cache size 0 max 16384 2024-06-10T01:06:30.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:06:30.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:06:30.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:06:30.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.126+0000 7f15121b0640 1 -- 172.21.15.7:0/520354291 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f14f40a2b80 msgr2=0x7f14f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:30.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.126+0000 7f15121b0640 1 --2- 172.21.15.7:0/520354291 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f14f40a2b80 0x7f14f40a5040 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f15040098b0 tx=0x7f1504009e00 comp rx=0 tx=0).stop 2024-06-10T01:06:30.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.126+0000 7f15121b0640 1 -- 172.21.15.7:0/520354291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f150c1024e0 msgr2=0x7f150c197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:30.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.126+0000 7f15121b0640 1 --2- 172.21.15.7:0/520354291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f150c1024e0 0x7f150c197d40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f14fc009230 tx=0x7f14fc004360 comp rx=0 tx=0).stop 2024-06-10T01:06:30.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.126+0000 7f15121b0640 1 -- 172.21.15.7:0/520354291 shutdown_connections 2024-06-10T01:06:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.126+0000 7f15121b0640 1 --2- 172.21.15.7:0/520354291 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f14f40a2b80 0x7f14f40a5040 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.126+0000 7f15121b0640 1 --2- 172.21.15.7:0/520354291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f150c107740 0x7f150c19f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.126+0000 7f15121b0640 1 --2- 172.21.15.7:0/520354291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f150c104e10 0x7f150c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.126+0000 7f15121b0640 1 --2- 172.21.15.7:0/520354291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f150c1024e0 0x7f150c197d40 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.126+0000 7f15121b0640 1 -- 172.21.15.7:0/520354291 >> 172.21.15.7:0/520354291 conn(0x7f150c0fbe70 msgr2=0x7f150c103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.127+0000 7f15121b0640 1 -- 172.21.15.7:0/520354291 shutdown_connections 2024-06-10T01:06:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:30.127+0000 7f15121b0640 1 -- 172.21.15.7:0/520354291 wait complete. 2024-06-10T01:06:31.604 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:31 smithi007 ceph-mon[32398]: pgmap v446: 129 pgs: 129 active+clean; 660 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-06-10T01:06:31.604 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:31 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:06:31.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:31 smithi175 ceph-mon[36636]: pgmap v446: 129 pgs: 129 active+clean; 660 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-06-10T01:06:31.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:31 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:06:31.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.733+0000 7f8dc1e4f700 1 -- 172.21.15.7:0/3826792093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dbc108500 msgr2=0x7f8dbc1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:31.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.733+0000 7f8dc1e4f700 1 --2- 172.21.15.7:0/3826792093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dbc108500 0x7f8dbc1088e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8dac009990 tx=0x7f8dac009ca0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.734+0000 7f8dc1e4f700 1 -- 172.21.15.7:0/3826792093 shutdown_connections 2024-06-10T01:06:31.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.734+0000 7f8dc1e4f700 1 --2- 172.21.15.7:0/3826792093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8dbc06ab30 0x7f8dbc10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.734+0000 7f8dc1e4f700 1 --2- 172.21.15.7:0/3826792093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dbc108500 0x7f8dbc1088e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.734+0000 7f8dc1e4f700 1 --2- 172.21.15.7:0/3826792093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8dbc06a170 0x7f8dbc06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.734+0000 7f8dc1e4f700 1 -- 172.21.15.7:0/3826792093 >> 172.21.15.7:0/3826792093 conn(0x7f8dbc078340 msgr2=0x7f8dbc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:31.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.734+0000 7f8dc1e4f700 1 -- 172.21.15.7:0/3826792093 shutdown_connections 2024-06-10T01:06:31.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.734+0000 7f8dc1e4f700 1 -- 172.21.15.7:0/3826792093 wait complete. 2024-06-10T01:06:31.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.735+0000 7f8dc1e4f700 1 Processor -- start 2024-06-10T01:06:31.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.735+0000 7f8dc1e4f700 1 -- start start 2024-06-10T01:06:31.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.735+0000 7f8dc1e4f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8dbc06a170 0x7f8dbc19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:31.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.736+0000 7f8dc1e4f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dbc06ab30 0x7f8dbc19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:31.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.736+0000 7f8dc1e4f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8dbc108500 0x7f8dbc198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:31.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.736+0000 7f8dc1e4f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc112760 con 0x7f8dbc06ab30 2024-06-10T01:06:31.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.736+0000 7f8dc1e4f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc1125e0 con 0x7f8dbc108500 2024-06-10T01:06:31.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.736+0000 7f8dc1e4f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc1128e0 con 0x7f8dbc06a170 2024-06-10T01:06:31.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.736+0000 7f8dbaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dbc06ab30 0x7f8dbc19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:31.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.736+0000 7f8dbb7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8dbc06a170 0x7f8dbc19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:31.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.736+0000 7f8dbaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dbc06ab30 0x7f8dbc19eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57276/0 (socket says 172.21.15.7:57276) 2024-06-10T01:06:31.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.736+0000 7f8dbaffd700 1 -- 172.21.15.7:0/593145843 learned_addr learned my addr 172.21.15.7:0/593145843 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:31.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.736+0000 7f8dbbfff700 1 --2- 172.21.15.7:0/593145843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8dbc108500 0x7f8dbc198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:31.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.736+0000 7f8dbb7fe700 1 -- 172.21.15.7:0/593145843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8dbc108500 msgr2=0x7f8dbc198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:31.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.736+0000 7f8dbb7fe700 1 --2- 172.21.15.7:0/593145843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8dbc108500 0x7f8dbc198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.736+0000 7f8dbb7fe700 1 -- 172.21.15.7:0/593145843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dbc06ab30 msgr2=0x7f8dbc19eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:31.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.736+0000 7f8dbb7fe700 1 --2- 172.21.15.7:0/593145843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dbc06ab30 0x7f8dbc19eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.736+0000 7f8dbb7fe700 1 -- 172.21.15.7:0/593145843 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dbc199180 con 0x7f8dbc06a170 2024-06-10T01:06:31.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.737+0000 7f8dbb7fe700 1 --2- 172.21.15.7:0/593145843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8dbc06a170 0x7f8dbc19e860 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8dac006010 tx=0x7f8dac003890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:31.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.737+0000 7f8db8ff9700 1 -- 172.21.15.7:0/593145843 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dac027070 con 0x7f8dbc06a170 2024-06-10T01:06:31.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.737+0000 7f8db8ff9700 1 -- 172.21.15.7:0/593145843 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8dac0056d0 con 0x7f8dbc06a170 2024-06-10T01:06:31.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.737+0000 7f8dc1e4f700 1 -- 172.21.15.7:0/593145843 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dbc199410 con 0x7f8dbc06a170 2024-06-10T01:06:31.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.737+0000 7f8db8ff9700 1 -- 172.21.15.7:0/593145843 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dac022bd0 con 0x7f8dbc06a170 2024-06-10T01:06:31.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.737+0000 7f8dc1e4f700 1 -- 172.21.15.7:0/593145843 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dbc101390 con 0x7f8dbc06a170 2024-06-10T01:06:31.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.739+0000 7f8db8ff9700 1 -- 172.21.15.7:0/593145843 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f8dac02a070 con 0x7f8dbc06a170 2024-06-10T01:06:31.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.740+0000 7f8db8ff9700 1 --2- 172.21.15.7:0/593145843 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8da40a7030 0x7f8da40a94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:31.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.740+0000 7f8dbaffd700 1 --2- 172.21.15.7:0/593145843 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8da40a7030 0x7f8da40a94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:31.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.740+0000 7f8db8ff9700 1 -- 172.21.15.7:0/593145843 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f8dac09a730 con 0x7f8dbc06a170 2024-06-10T01:06:31.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.741+0000 7f8dbaffd700 1 --2- 172.21.15.7:0/593145843 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8da40a7030 0x7f8da40a94f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc0ffbc0 tx=0x7f8db000a400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:31.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.741+0000 7f8dcc6d2b80 1 Processor -- start 2024-06-10T01:06:31.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.741+0000 7f8dcc6d2b80 1 -- start start 2024-06-10T01:06:31.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.742+0000 7f8dcc6d2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164ab86640 0x56164ab86a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:31.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.742+0000 7f8dcc6d2b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164ab86f60 0x56164ab809e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:31.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.742+0000 7f8dcc6d2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56164ab80f20 0x56164ab81340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:31.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.742+0000 7f8dcc6d2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56164aad2810 con 0x56164ab80f20 2024-06-10T01:06:31.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.742+0000 7f8dcc6d2b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56164aa7a720 con 0x56164ab86f60 2024-06-10T01:06:31.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.742+0000 7f8dcc6d2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56164aace7d0 con 0x56164ab86640 2024-06-10T01:06:31.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.742+0000 7f8dbb7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164ab86640 0x56164ab86a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:31.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.742+0000 7f8dbbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56164ab80f20 0x56164ab81340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:31.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.742+0000 7f8dbb7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164ab86640 0x56164ab86a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:51910/0 (socket says 172.21.15.7:51910) 2024-06-10T01:06:31.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.742+0000 7f8dbb7fe700 1 -- 172.21.15.7:0/2893601812 learned_addr learned my addr 172.21.15.7:0/2893601812 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:31.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.742+0000 7f8dbaffd700 1 --2- 172.21.15.7:0/2893601812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164ab86f60 0x56164ab809e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:31.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.742+0000 7f8dbb7fe700 1 -- 172.21.15.7:0/2893601812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164ab86f60 msgr2=0x56164ab809e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:31.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.742+0000 7f8dbb7fe700 1 --2- 172.21.15.7:0/2893601812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164ab86f60 0x56164ab809e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.742+0000 7f8dbb7fe700 1 -- 172.21.15.7:0/2893601812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56164ab80f20 msgr2=0x56164ab81340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:31.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.742+0000 7f8dbb7fe700 1 --2- 172.21.15.7:0/2893601812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56164ab80f20 0x56164ab81340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.742+0000 7f8dbb7fe700 1 -- 172.21.15.7:0/2893601812 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56164aa70f50 con 0x56164ab86640 2024-06-10T01:06:31.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.742+0000 7f8dbaffd700 1 --2- 172.21.15.7:0/2893601812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164ab86f60 0x56164ab809e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:06:31.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.743+0000 7f8dbb7fe700 1 --2- 172.21.15.7:0/2893601812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164ab86640 0x56164ab86a20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8dac021000 tx=0x7f8dac009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=728a20aa4aa66e85 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:31.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.743+0000 7f8da1ffb700 1 -- 172.21.15.7:0/2893601812 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dac06de60 con 0x56164ab86640 2024-06-10T01:06:31.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.743+0000 7f8da1ffb700 1 -- 172.21.15.7:0/2893601812 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8dac09cd90 con 0x56164ab86640 2024-06-10T01:06:31.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.743+0000 7f8da1ffb700 1 -- 172.21.15.7:0/2893601812 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dac0a0a40 con 0x56164ab86640 2024-06-10T01:06:31.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.744+0000 7f8dcc6d2b80 1 -- 172.21.15.7:0/2893601812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164ab86640 msgr2=0x56164ab86a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:31.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.744+0000 7f8dcc6d2b80 1 --2- 172.21.15.7:0/2893601812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164ab86640 0x56164ab86a20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8dac021000 tx=0x7f8dac009310 comp rx=0 tx=0).stop 2024-06-10T01:06:31.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.744+0000 7f8dcc6d2b80 1 -- 172.21.15.7:0/2893601812 shutdown_connections 2024-06-10T01:06:31.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.744+0000 7f8dcc6d2b80 1 --2- 172.21.15.7:0/2893601812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56164ab80f20 0x56164ab81340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.744+0000 7f8dcc6d2b80 1 --2- 172.21.15.7:0/2893601812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164ab86640 0x56164ab86a20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.744+0000 7f8dcc6d2b80 1 --2- 172.21.15.7:0/2893601812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164ab86f60 0x56164ab809e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.744+0000 7f8dcc6d2b80 1 -- 172.21.15.7:0/2893601812 >> 172.21.15.7:0/2893601812 conn(0x56164aaabcc0 msgr2=0x56164ab7b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:31.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.744+0000 7f8dcc6d2b80 1 -- 172.21.15.7:0/2893601812 shutdown_connections 2024-06-10T01:06:31.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.744+0000 7f8dcc6d2b80 1 -- 172.21.15.7:0/2893601812 wait complete. 2024-06-10T01:06:31.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.744+0000 7f8dcc6d2b80 1 -- start start 2024-06-10T01:06:31.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.745+0000 7f8dcc6d2b80 1 Processor -- start 2024-06-10T01:06:31.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.745+0000 7f8dcc6d2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164ab80f20 0x56164ab8cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:31.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.745+0000 7f8dcc6d2b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164ab86640 0x56164ab8a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:31.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.745+0000 7f8dcc6d2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56164ab86f60 0x56164ab8a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:31.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.745+0000 7f8dcc6d2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56164ab8d6f0 con 0x56164ab86f60 2024-06-10T01:06:31.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.745+0000 7f8dcc6d2b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56164aa72bf0 con 0x56164ab86640 2024-06-10T01:06:31.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.745+0000 7f8dcc6d2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56164ab8d870 con 0x56164ab80f20 2024-06-10T01:06:31.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.745+0000 7f8dcc6d2b80 10 client.0 _ll_register_callbacks cb 0x56164aa6c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:31.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.745+0000 7f8dbbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56164ab86f60 0x56164ab8a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:31.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.745+0000 7f8dbbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56164ab86f60 0x56164ab8a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57292/0 (socket says 172.21.15.7:57292) 2024-06-10T01:06:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.745+0000 7f8dbbfff700 1 -- 172.21.15.7:0/2515035586 learned_addr learned my addr 172.21.15.7:0/2515035586 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.745+0000 7f8dbaffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164ab86640 0x56164ab8a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.746+0000 7f8dbb7fe700 1 --2- 172.21.15.7:0/2515035586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164ab80f20 0x56164ab8cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.746+0000 7f8dbaffd700 1 -- 172.21.15.7:0/2515035586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164ab80f20 msgr2=0x56164ab8cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.746+0000 7f8dbaffd700 1 --2- 172.21.15.7:0/2515035586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164ab80f20 0x56164ab8cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.746+0000 7f8dbaffd700 1 -- 172.21.15.7:0/2515035586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56164ab86f60 msgr2=0x56164ab8a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.746+0000 7f8dbaffd700 1 --2- 172.21.15.7:0/2515035586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56164ab86f60 0x56164ab8a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.746+0000 7f8dbaffd700 1 -- 172.21.15.7:0/2515035586 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56164aa218d0 con 0x56164ab86640 2024-06-10T01:06:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.746+0000 7f8dbbfff700 1 --2- 172.21.15.7:0/2515035586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56164ab86f60 0x56164ab8a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.746+0000 7f8dbaffd700 1 --2- 172.21.15.7:0/2515035586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164ab86640 0x56164ab8a050 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8db0005950 tx=0x7f8db00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a514de0165b7594 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.746+0000 7f8d9affd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:06:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.747+0000 7f8d9affd700 1 -- 172.21.15.7:0/2515035586 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db0016070 con 0x56164ab86640 2024-06-10T01:06:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.747+0000 7f8dcc6d2b80 1 -- 172.21.15.7:0/2515035586 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8dac09cd90 con 0x56164ab86640 2024-06-10T01:06:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.747+0000 7f8d9affd700 1 -- 172.21.15.7:0/2515035586 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f8db0010840 con 0x56164ab86640 2024-06-10T01:06:31.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.747+0000 7f8d9affd700 1 -- 172.21.15.7:0/2515035586 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8db0031050 con 0x56164ab86640 2024-06-10T01:06:31.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.747+0000 7f8dcc6d2b80 10 client.25124 fetch_fsmap learned FSMap version 27 2024-06-10T01:06:31.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.747+0000 7f8dcc6d2b80 1 -- 172.21.15.7:0/2515035586 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56164ab98010 con 0x56164ab86640 2024-06-10T01:06:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.747+0000 7f8d9affd700 1 -- 172.21.15.7:0/2515035586 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8dac09cd90 con 0x56164ab86640 2024-06-10T01:06:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.747+0000 7f8d9affd700 1 -- 172.21.15.7:0/2515035586 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8db0028bc0 con 0x56164ab86640 2024-06-10T01:06:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.748+0000 7f8dcc6d2b80 10 client.25124 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:06:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.748+0000 7f8dcc6d2b80 10 client.25124 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:06:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.748+0000 7f8dcc6d2b80 20 client.25124 populate_metadata read hostname 'smithi007' 2024-06-10T01:06:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.748+0000 7f8dcc6d2b80 1 --2- 172.21.15.7:0/2515035586 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56164ab9a5f0 0x56164ab9ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.748+0000 7f8dcc6d2b80 4 client.25124 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:06:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.748+0000 7f8dcc6d2b80 1 -- 172.21.15.7:0/2515035586 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56164aadfd10 con 0x56164ab9a5f0 2024-06-10T01:06:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.748+0000 7f8dbb7fe700 1 --2- 172.21.15.7:0/2515035586 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56164ab9a5f0 0x56164ab9ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.748+0000 7f8d9affd700 10 client.25124 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.749+0000 7f8dbb7fe700 1 --2- 172.21.15.7:0/2515035586 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56164ab9a5f0 0x56164ab9ca10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7740e34a2d12959b server_cookie=719dfbc32f814c27 in_seq=0 out_seq=0 2024-06-10T01:06:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.750+0000 7f8d9affd700 1 -- 172.21.15.7:0/2515035586 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56164aadfd10 con 0x56164ab9a5f0 2024-06-10T01:06:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.750+0000 7f8d9affd700 10 client.25124 handle_command_reply: tid=0 2024-06-10T01:06:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.750+0000 7f8dcc6d2b80 1 client.25124 shutdown 2024-06-10T01:06:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.750+0000 7f8dcc6d2b80 1 -- 172.21.15.7:0/2515035586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164ab86640 msgr2=0x56164ab8a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.750+0000 7f8dcc6d2b80 1 --2- 172.21.15.7:0/2515035586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164ab86640 0x56164ab8a050 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8db0005950 tx=0x7f8db00047c0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.750+0000 7f8dcc6d2b80 1 -- 172.21.15.7:0/2515035586 shutdown_connections 2024-06-10T01:06:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.750+0000 7f8dcc6d2b80 1 --2- 172.21.15.7:0/2515035586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56164ab86f60 0x56164ab8a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.750+0000 7f8dcc6d2b80 1 --2- 172.21.15.7:0/2515035586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164ab80f20 0x56164ab8cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.750+0000 7f8dcc6d2b80 1 --2- 172.21.15.7:0/2515035586 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56164ab9a5f0 0x56164ab9ca10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.750+0000 7f8dcc6d2b80 1 --2- 172.21.15.7:0/2515035586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164ab86640 0x56164ab8a050 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.750+0000 7f8dcc6d2b80 1 -- 172.21.15.7:0/2515035586 >> 172.21.15.7:0/2515035586 conn(0x56164aaabcc0 msgr2=0x56164ab7b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.750+0000 7f8d9affd700 0 client.25124 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.750+0000 7f8dcc6d2b80 1 -- 172.21.15.7:0/2515035586 shutdown_connections 2024-06-10T01:06:31.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.750+0000 7f8dcc6d2b80 1 -- 172.21.15.7:0/2515035586 wait complete. 2024-06-10T01:06:31.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.750+0000 7f8dcc6d2b80 20 client.25124 trim_cache size 0 max 16384 2024-06-10T01:06:31.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.773+0000 7f8dcc6d2b80 1 Processor -- start 2024-06-10T01:06:31.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.773+0000 7f8dcc6d2b80 1 -- start start 2024-06-10T01:06:31.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.774+0000 7f8dcc6d2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164ab80f20 0x56164ab809e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:31.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.774+0000 7f8dcc6d2b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164ab86640 0x56164ab75690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:31.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.774+0000 7f8dcc6d2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56164ab86f60 0x56164aba2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:31.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.774+0000 7f8dcc6d2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc1ae8c0 con 0x56164ab86f60 2024-06-10T01:06:31.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.774+0000 7f8dcc6d2b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc1ac150 con 0x56164ab86640 2024-06-10T01:06:31.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.774+0000 7f8dcc6d2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc1a9fa0 con 0x56164ab80f20 2024-06-10T01:06:31.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.774+0000 7f8dbbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56164ab86f60 0x56164aba2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:31.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.774+0000 7f8dbbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56164ab86f60 0x56164aba2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57294/0 (socket says 172.21.15.7:57294) 2024-06-10T01:06:31.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.774+0000 7f8dbbfff700 1 -- 172.21.15.7:0/1675667654 learned_addr learned my addr 172.21.15.7:0/1675667654 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:31.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.774+0000 7f8dbb7fe700 1 --2- 172.21.15.7:0/1675667654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164ab80f20 0x56164ab809e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.774+0000 7f8dbaffd700 1 --2- 172.21.15.7:0/1675667654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164ab86640 0x56164ab75690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.774+0000 7f8dbb7fe700 1 -- 172.21.15.7:0/1675667654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164ab86640 msgr2=0x56164ab75690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.774+0000 7f8dbb7fe700 1 --2- 172.21.15.7:0/1675667654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164ab86640 0x56164ab75690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.774+0000 7f8dbb7fe700 1 -- 172.21.15.7:0/1675667654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56164ab86f60 msgr2=0x56164aba2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.774+0000 7f8dbb7fe700 1 --2- 172.21.15.7:0/1675667654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56164ab86f60 0x56164aba2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.774+0000 7f8dbb7fe700 1 -- 172.21.15.7:0/1675667654 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56164ab885e0 con 0x56164ab80f20 2024-06-10T01:06:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.774+0000 7f8dbbfff700 1 --2- 172.21.15.7:0/1675667654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56164ab86f60 0x56164aba2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.775+0000 7f8dbb7fe700 1 --2- 172.21.15.7:0/1675667654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164ab80f20 0x56164ab809e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8dac0098e0 tx=0x7f8dac01a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c62bbaba29348387 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.775+0000 7f8da17fa700 1 -- 172.21.15.7:0/1675667654 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dac066070 con 0x56164ab80f20 2024-06-10T01:06:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.775+0000 7f8da17fa700 1 -- 172.21.15.7:0/1675667654 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8dac005450 con 0x56164ab80f20 2024-06-10T01:06:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.775+0000 7f8da17fa700 1 -- 172.21.15.7:0/1675667654 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dac06b070 con 0x56164ab80f20 2024-06-10T01:06:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.776+0000 7f8dcc6d2b80 1 -- 172.21.15.7:0/1675667654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164ab80f20 msgr2=0x56164ab809e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.776+0000 7f8dcc6d2b80 1 --2- 172.21.15.7:0/1675667654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164ab80f20 0x56164ab809e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8dac0098e0 tx=0x7f8dac01a040 comp rx=0 tx=0).stop 2024-06-10T01:06:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.776+0000 7f8dcc6d2b80 1 -- 172.21.15.7:0/1675667654 shutdown_connections 2024-06-10T01:06:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.776+0000 7f8dcc6d2b80 1 --2- 172.21.15.7:0/1675667654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56164ab86f60 0x56164aba2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.776+0000 7f8dcc6d2b80 1 --2- 172.21.15.7:0/1675667654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164ab80f20 0x56164ab809e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.776+0000 7f8dcc6d2b80 1 --2- 172.21.15.7:0/1675667654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164ab86640 0x56164ab75690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.776+0000 7f8dcc6d2b80 1 -- 172.21.15.7:0/1675667654 >> 172.21.15.7:0/1675667654 conn(0x56164aaabcc0 msgr2=0x56164ab879b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:31.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.776+0000 7f8dcc6d2b80 1 -- 172.21.15.7:0/1675667654 shutdown_connections 2024-06-10T01:06:31.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.776+0000 7f8dcc6d2b80 1 -- 172.21.15.7:0/1675667654 wait complete. 2024-06-10T01:06:31.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.776+0000 7f8dcc6d2b80 1 -- start start 2024-06-10T01:06:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.777+0000 7f8dcc6d2b80 1 Processor -- start 2024-06-10T01:06:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.777+0000 7f8dcc6d2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164ab80f20 0x56164ab92fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.777+0000 7f8dcc6d2b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164ab86640 0x56164ab90350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.777+0000 7f8dcc6d2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56164ab86f60 0x56164ab90890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.777+0000 7f8dcc6d2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc1aa610 con 0x56164ab86f60 2024-06-10T01:06:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.777+0000 7f8dcc6d2b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc1aba80 con 0x56164ab86640 2024-06-10T01:06:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.777+0000 7f8dcc6d2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc1aace0 con 0x56164ab80f20 2024-06-10T01:06:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.777+0000 7f8dcc6d2b80 10 client.0 _ll_register_callbacks cb 0x56164aa6c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.777+0000 7f8dbbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56164ab86f60 0x56164ab90890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.777+0000 7f8dbaffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164ab86640 0x56164ab90350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.777+0000 7f8dbbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56164ab86f60 0x56164ab90890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57304/0 (socket says 172.21.15.7:57304) 2024-06-10T01:06:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.777+0000 7f8dbbfff700 1 -- 172.21.15.7:0/1387075162 learned_addr learned my addr 172.21.15.7:0/1387075162 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.777+0000 7f8dbaffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164ab86640 0x56164ab90350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:44528/0 (socket says 172.21.15.7:44528) 2024-06-10T01:06:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.777+0000 7f8dbb7fe700 1 --2- 172.21.15.7:0/1387075162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164ab80f20 0x56164ab92fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.778+0000 7f8dbbfff700 1 -- 172.21.15.7:0/1387075162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164ab80f20 msgr2=0x56164ab92fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.778+0000 7f8dbbfff700 1 --2- 172.21.15.7:0/1387075162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164ab80f20 0x56164ab92fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.778+0000 7f8dbbfff700 1 -- 172.21.15.7:0/1387075162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164ab86640 msgr2=0x56164ab90350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.778+0000 7f8dbbfff700 1 --2- 172.21.15.7:0/1387075162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164ab86640 0x56164ab90350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.778+0000 7f8dbbfff700 1 -- 172.21.15.7:0/1387075162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56164ab90e50 con 0x56164ab86f60 2024-06-10T01:06:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.778+0000 7f8dbbfff700 1 --2- 172.21.15.7:0/1387075162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56164ab86f60 0x56164ab90890 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8db40176f0 tx=0x7f8db4017720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d797e3510b4f063 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.778+0000 7f8d9affd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:06:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.778+0000 7f8d9affd700 1 -- 172.21.15.7:0/1387075162 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db4017d90 con 0x56164ab86f60 2024-06-10T01:06:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.778+0000 7f8d9affd700 1 -- 172.21.15.7:0/1387075162 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f8db40203c0 con 0x56164ab86f60 2024-06-10T01:06:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.778+0000 7f8dcc6d2b80 1 -- 172.21.15.7:0/1387075162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56164aada290 con 0x56164ab86f60 2024-06-10T01:06:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.778+0000 7f8d9affd700 1 -- 172.21.15.7:0/1387075162 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8db40205c0 con 0x56164ab86f60 2024-06-10T01:06:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.778+0000 7f8dcc6d2b80 10 client.15480 fetch_fsmap learned FSMap version 27 2024-06-10T01:06:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.778+0000 7f8dcc6d2b80 1 -- 172.21.15.7:0/1387075162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56164ac050b0 con 0x56164ab86f60 2024-06-10T01:06:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.779+0000 7f8d9affd700 1 -- 172.21.15.7:0/1387075162 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56164aada290 con 0x56164ab86f60 2024-06-10T01:06:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.779+0000 7f8d9affd700 1 -- 172.21.15.7:0/1387075162 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8db4021a70 con 0x56164ab86f60 2024-06-10T01:06:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.779+0000 7f8dcc6d2b80 10 client.15480 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:06:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.779+0000 7f8dcc6d2b80 10 client.15480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:06:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.779+0000 7f8dcc6d2b80 20 client.15480 populate_metadata read hostname 'smithi007' 2024-06-10T01:06:31.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.779+0000 7f8dcc6d2b80 1 --2- 172.21.15.7:0/1387075162 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56164ac05810 0x56164ac07c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:31.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.779+0000 7f8dcc6d2b80 4 client.15480 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:06:31.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.779+0000 7f8dcc6d2b80 1 -- 172.21.15.7:0/1387075162 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56164a954250 con 0x56164ac05810 2024-06-10T01:06:31.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.779+0000 7f8dbb7fe700 1 --2- 172.21.15.7:0/1387075162 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56164ac05810 0x56164ac07c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:31.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.780+0000 7f8d9affd700 10 client.15480 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:31.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.780+0000 7f8dbb7fe700 1 --2- 172.21.15.7:0/1387075162 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56164ac05810 0x56164ac07c30 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f0eb8d542d77232 server_cookie=941b228432842b44 in_seq=0 out_seq=0 2024-06-10T01:06:31.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.780+0000 7f8d9affd700 1 -- 172.21.15.7:0/1387075162 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56164a954250 con 0x56164ac05810 2024-06-10T01:06:31.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.780+0000 7f8d9affd700 10 client.15480 handle_command_reply: tid=0 2024-06-10T01:06:31.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.780+0000 7f8dcc6d2b80 1 client.15480 shutdown 2024-06-10T01:06:31.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.780+0000 7f8dcc6d2b80 1 -- 172.21.15.7:0/1387075162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56164ab86f60 msgr2=0x56164ab90890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:31.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.780+0000 7f8dcc6d2b80 1 --2- 172.21.15.7:0/1387075162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56164ab86f60 0x56164ab90890 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8db40176f0 tx=0x7f8db4017720 comp rx=0 tx=0).stop 2024-06-10T01:06:31.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.781+0000 7f8dcc6d2b80 1 -- 172.21.15.7:0/1387075162 shutdown_connections 2024-06-10T01:06:31.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.781+0000 7f8dcc6d2b80 1 --2- 172.21.15.7:0/1387075162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56164ab86f60 0x56164ab90890 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.781+0000 7f8dcc6d2b80 1 --2- 172.21.15.7:0/1387075162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164ab80f20 0x56164ab92fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.781+0000 7f8dcc6d2b80 1 --2- 172.21.15.7:0/1387075162 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56164ac05810 0x56164ac07c30 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.781+0000 7f8d9affd700 0 client.15480 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:31.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.781+0000 7f8dcc6d2b80 1 --2- 172.21.15.7:0/1387075162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164ab86640 0x56164ab90350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.781+0000 7f8dcc6d2b80 1 -- 172.21.15.7:0/1387075162 >> 172.21.15.7:0/1387075162 conn(0x56164aaabcc0 msgr2=0x56164ab85120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:31.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.781+0000 7f8dcc6d2b80 1 -- 172.21.15.7:0/1387075162 shutdown_connections 2024-06-10T01:06:31.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.781+0000 7f8dcc6d2b80 1 -- 172.21.15.7:0/1387075162 wait complete. 2024-06-10T01:06:31.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.781+0000 7f8dcc6d2b80 20 client.15480 trim_cache size 0 max 16384 2024-06-10T01:06:31.781 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:06:31.781 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-10T01:06:31.781 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-10T01:06:31.782 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:06:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.782+0000 7f8dc1e4f700 1 -- 172.21.15.7:0/593145843 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8da40a7030 msgr2=0x7f8da40a94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.782+0000 7f8dc1e4f700 1 --2- 172.21.15.7:0/593145843 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8da40a7030 0x7f8da40a94f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc0ffbc0 tx=0x7f8db000a400 comp rx=0 tx=0).stop 2024-06-10T01:06:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.782+0000 7f8dc1e4f700 1 -- 172.21.15.7:0/593145843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8dbc06a170 msgr2=0x7f8dbc19e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.782+0000 7f8dc1e4f700 1 --2- 172.21.15.7:0/593145843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8dbc06a170 0x7f8dbc19e860 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8dac006010 tx=0x7f8dac003890 comp rx=0 tx=0).stop 2024-06-10T01:06:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.782+0000 7f8dc1e4f700 1 -- 172.21.15.7:0/593145843 shutdown_connections 2024-06-10T01:06:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.782+0000 7f8dc1e4f700 1 --2- 172.21.15.7:0/593145843 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8da40a7030 0x7f8da40a94f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.782+0000 7f8dc1e4f700 1 --2- 172.21.15.7:0/593145843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8dbc108500 0x7f8dbc198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.782+0000 7f8dc1e4f700 1 --2- 172.21.15.7:0/593145843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8dbc06a170 0x7f8dbc19e860 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.782+0000 7f8dc1e4f700 1 --2- 172.21.15.7:0/593145843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dbc06ab30 0x7f8dbc19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.782+0000 7f8dc1e4f700 1 -- 172.21.15.7:0/593145843 >> 172.21.15.7:0/593145843 conn(0x7f8dbc078340 msgr2=0x7f8dbc0fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.782+0000 7f8dc1e4f700 1 -- 172.21.15.7:0/593145843 shutdown_connections 2024-06-10T01:06:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:31.782+0000 7f8dc1e4f700 1 -- 172.21.15.7:0/593145843 wait complete. 2024-06-10T01:06:31.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:31 smithi078 ceph-mon[36683]: pgmap v446: 129 pgs: 129 active+clean; 660 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-06-10T01:06:31.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:31 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:06:32.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:32 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:06:32.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:32 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:06:32.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:32 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:06:32.614 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:06:32.614 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:06:32.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:32 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:06:32.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:32 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:06:32.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:32 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:06:32.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:32 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:06:32.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:32 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:06:32.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:32 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:06:32.934 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:06:33.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:33 smithi007 ceph-mon[32398]: pgmap v447: 129 pgs: 129 active+clean; 661 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 8 op/s 2024-06-10T01:06:33.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:33 smithi175 ceph-mon[36636]: pgmap v447: 129 pgs: 129 active+clean; 661 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 8 op/s 2024-06-10T01:06:33.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:33 smithi078 ceph-mon[36683]: pgmap v447: 129 pgs: 129 active+clean; 661 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 8 op/s 2024-06-10T01:06:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.683+0000 7fa66d482700 1 -- 172.21.15.7:0/3292070571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa668107400 msgr2=0x7fa6681077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.683+0000 7fa66d482700 1 --2- 172.21.15.7:0/3292070571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa668107400 0x7fa6681077e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa658009990 tx=0x7fa658009ca0 comp rx=0 tx=0).stop 2024-06-10T01:06:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.683+0000 7fa66d482700 1 -- 172.21.15.7:0/3292070571 shutdown_connections 2024-06-10T01:06:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.683+0000 7fa66d482700 1 --2- 172.21.15.7:0/3292070571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa668101d50 0x7fa66810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.683+0000 7fa66d482700 1 --2- 172.21.15.7:0/3292070571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa668107400 0x7fa6681077e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.683+0000 7fa66d482700 1 --2- 172.21.15.7:0/3292070571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa668101390 0x7fa668101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.683+0000 7fa66d482700 1 -- 172.21.15.7:0/3292070571 >> 172.21.15.7:0/3292070571 conn(0x7fa668078340 msgr2=0x7fa668078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.683+0000 7fa66d482700 1 -- 172.21.15.7:0/3292070571 shutdown_connections 2024-06-10T01:06:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.683+0000 7fa66d482700 1 -- 172.21.15.7:0/3292070571 wait complete. 2024-06-10T01:06:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.684+0000 7fa66d482700 1 Processor -- start 2024-06-10T01:06:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.684+0000 7fa66d482700 1 -- start start 2024-06-10T01:06:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.684+0000 7fa66d482700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa668101390 0x7fa6681999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.684+0000 7fa66d482700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa668101d50 0x7fa668199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.684+0000 7fa66d482700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa668107400 0x7fa6681a4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.684+0000 7fa66d482700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa668111670 con 0x7fa668101390 2024-06-10T01:06:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.684+0000 7fa66d482700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa6681114f0 con 0x7fa668107400 2024-06-10T01:06:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.684+0000 7fa66d482700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa6681117f0 con 0x7fa668101d50 2024-06-10T01:06:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.684+0000 7fa666ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa668101390 0x7fa6681999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.684+0000 7fa666ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa668101390 0x7fa6681999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38100/0 (socket says 172.21.15.7:38100) 2024-06-10T01:06:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.684+0000 7fa666ffd700 1 -- 172.21.15.7:0/3191297989 learned_addr learned my addr 172.21.15.7:0/3191297989 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.684+0000 7fa6667fc700 1 --2- 172.21.15.7:0/3191297989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa668101d50 0x7fa668199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.684+0000 7fa6677fe700 1 --2- 172.21.15.7:0/3191297989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa668107400 0x7fa6681a4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.685+0000 7fa666ffd700 1 -- 172.21.15.7:0/3191297989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa668101d50 msgr2=0x7fa668199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.685+0000 7fa666ffd700 1 --2- 172.21.15.7:0/3191297989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa668101d50 0x7fa668199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.685+0000 7fa666ffd700 1 -- 172.21.15.7:0/3191297989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa668107400 msgr2=0x7fa6681a4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.685+0000 7fa666ffd700 1 --2- 172.21.15.7:0/3191297989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa668107400 0x7fa6681a4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.685+0000 7fa666ffd700 1 -- 172.21.15.7:0/3191297989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6681a4f90 con 0x7fa668101390 2024-06-10T01:06:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.685+0000 7fa666ffd700 1 --2- 172.21.15.7:0/3191297989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa668101390 0x7fa6681999d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa6580038b0 tx=0x7fa658009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.685+0000 7fa64ffff700 1 -- 172.21.15.7:0/3191297989 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658027070 con 0x7fa668101390 2024-06-10T01:06:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.685+0000 7fa64ffff700 1 -- 172.21.15.7:0/3191297989 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa658004050 con 0x7fa668101390 2024-06-10T01:06:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.685+0000 7fa66d482700 1 -- 172.21.15.7:0/3191297989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6681a5160 con 0x7fa668101390 2024-06-10T01:06:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.685+0000 7fa64ffff700 1 -- 172.21.15.7:0/3191297989 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658022c00 con 0x7fa668101390 2024-06-10T01:06:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.685+0000 7fa66d482700 1 -- 172.21.15.7:0/3191297989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6681a5610 con 0x7fa668101390 2024-06-10T01:06:34.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.686+0000 7fa66d482700 1 -- 172.21.15.7:0/3191297989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa668069180 con 0x7fa668101390 2024-06-10T01:06:34.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.688+0000 7fa64ffff700 1 -- 172.21.15.7:0/3191297989 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa65802a070 con 0x7fa668101390 2024-06-10T01:06:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.688+0000 7fa64ffff700 1 --2- 172.21.15.7:0/3191297989 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa6500a2c20 0x7fa6500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.688+0000 7fa6667fc700 1 --2- 172.21.15.7:0/3191297989 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa6500a2c20 0x7fa6500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.688+0000 7fa64ffff700 1 -- 172.21.15.7:0/3191297989 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fa6580d0a40 con 0x7fa668101390 2024-06-10T01:06:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.689+0000 7fa6667fc700 1 --2- 172.21.15.7:0/3191297989 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa6500a2c20 0x7fa6500a50e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa6680788b0 tx=0x7fa65c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:34.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:34.692+0000 7fa64ffff700 1 -- 172.21.15.7:0/3191297989 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa658031020 con 0x7fa668101390 2024-06-10T01:06:35.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:35.023+0000 7fa66d482700 1 -- 172.21.15.7:0/3191297989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa66802dca0 con 0x7fa668101390 2024-06-10T01:06:35.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:35.024+0000 7fa64ffff700 1 -- 172.21.15.7:0/3191297989 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11855 (secure 0 0 0) 0x7fa658013070 con 0x7fa668101390 2024-06-10T01:06:35.020 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:06:35.021 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:05:39.175358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/2784910673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2784910673},{"type":"v1","addr":"172.21.15.175:6835","nonce":2784910673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24434,24436,24451]},"id":1}]} 2024-06-10T01:06:35.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:35.026+0000 7fa66d482700 1 -- 172.21.15.7:0/3191297989 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa6500a2c20 msgr2=0x7fa6500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:35.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:35.026+0000 7fa66d482700 1 --2- 172.21.15.7:0/3191297989 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa6500a2c20 0x7fa6500a50e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa6680788b0 tx=0x7fa65c00a380 comp rx=0 tx=0).stop 2024-06-10T01:06:35.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:35.026+0000 7fa66d482700 1 -- 172.21.15.7:0/3191297989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa668101390 msgr2=0x7fa6681999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:35.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:35.026+0000 7fa66d482700 1 --2- 172.21.15.7:0/3191297989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa668101390 0x7fa6681999d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa6580038b0 tx=0x7fa658009f50 comp rx=0 tx=0).stop 2024-06-10T01:06:35.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:35.026+0000 7fa66d482700 1 -- 172.21.15.7:0/3191297989 shutdown_connections 2024-06-10T01:06:35.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:35.026+0000 7fa66d482700 1 --2- 172.21.15.7:0/3191297989 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa6500a2c20 0x7fa6500a50e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:35.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:35.026+0000 7fa66d482700 1 --2- 172.21.15.7:0/3191297989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa668107400 0x7fa6681a4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:35.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:35.026+0000 7fa66d482700 1 --2- 172.21.15.7:0/3191297989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa668101390 0x7fa6681999d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:35.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:35.026+0000 7fa66d482700 1 --2- 172.21.15.7:0/3191297989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa668101d50 0x7fa668199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:35.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:35.026+0000 7fa66d482700 1 -- 172.21.15.7:0/3191297989 >> 172.21.15.7:0/3191297989 conn(0x7fa668078340 msgr2=0x7fa6680fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:35.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:35.027+0000 7fa66d482700 1 -- 172.21.15.7:0/3191297989 shutdown_connections 2024-06-10T01:06:35.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:35.027+0000 7fa66d482700 1 -- 172.21.15.7:0/3191297989 wait complete. 2024-06-10T01:06:35.024 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-06-10T01:06:35.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:35 smithi175 ceph-mon[36636]: pgmap v448: 129 pgs: 129 active+clean; 661 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-10T01:06:35.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:35 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3191297989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:06:35.758 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:35 smithi007 ceph-mon[32398]: pgmap v448: 129 pgs: 129 active+clean; 661 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-10T01:06:35.758 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:35 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3191297989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:06:35.785 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:06:35.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:35 smithi078 ceph-mon[36683]: pgmap v448: 129 pgs: 129 active+clean; 661 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-10T01:06:35.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:35 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3191297989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:06:36.050 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:06:37.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:37 smithi007 ceph-mon[32398]: pgmap v449: 129 pgs: 129 active+clean; 661 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:06:37.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:37 smithi175 ceph-mon[36636]: pgmap v449: 129 pgs: 129 active+clean; 661 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:06:37.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.777+0000 7ff8425d3700 1 -- 172.21.15.7:0/1109803181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff83c101d50 msgr2=0x7ff83c10e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:37.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.777+0000 7ff8425d3700 1 --2- 172.21.15.7:0/1109803181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff83c101d50 0x7ff83c10e240 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff83400b0a0 tx=0x7ff83400b3b0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.777+0000 7ff8425d3700 1 -- 172.21.15.7:0/1109803181 shutdown_connections 2024-06-10T01:06:37.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.777+0000 7ff8425d3700 1 --2- 172.21.15.7:0/1109803181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff83c101d50 0x7ff83c10e240 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.777+0000 7ff8425d3700 1 --2- 172.21.15.7:0/1109803181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff83c107400 0x7ff83c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.777+0000 7ff8425d3700 1 --2- 172.21.15.7:0/1109803181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff83c101390 0x7ff83c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.777+0000 7ff8425d3700 1 -- 172.21.15.7:0/1109803181 >> 172.21.15.7:0/1109803181 conn(0x7ff83c078340 msgr2=0x7ff83c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:37.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.777+0000 7ff8425d3700 1 -- 172.21.15.7:0/1109803181 shutdown_connections 2024-06-10T01:06:37.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.777+0000 7ff8425d3700 1 -- 172.21.15.7:0/1109803181 wait complete. 2024-06-10T01:06:37.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.778+0000 7ff8425d3700 1 Processor -- start 2024-06-10T01:06:37.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.778+0000 7ff8425d3700 1 -- start start 2024-06-10T01:06:37.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.779+0000 7ff8425d3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff83c101390 0x7ff83c19d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:37.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.779+0000 7ff8425d3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff83c101d50 0x7ff83c19dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:37.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.779+0000 7ff8425d3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff83c107400 0x7ff83c197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:37.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.779+0000 7ff8425d3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c111670 con 0x7ff83c107400 2024-06-10T01:06:37.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.779+0000 7ff8425d3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c1114f0 con 0x7ff83c101390 2024-06-10T01:06:37.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.779+0000 7ff8425d3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c1117f0 con 0x7ff83c101d50 2024-06-10T01:06:37.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.779+0000 7ff840b70700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff83c107400 0x7ff83c197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:37.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.779+0000 7ff840b70700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff83c107400 0x7ff83c197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38114/0 (socket says 172.21.15.7:38114) 2024-06-10T01:06:37.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.779+0000 7ff840b70700 1 -- 172.21.15.7:0/1757710958 learned_addr learned my addr 172.21.15.7:0/1757710958 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:37.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.779+0000 7ff83bfff700 1 --2- 172.21.15.7:0/1757710958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff83c101390 0x7ff83c19d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:37.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.779+0000 7ff83b7fe700 1 --2- 172.21.15.7:0/1757710958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff83c101d50 0x7ff83c19dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:37.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.780+0000 7ff840b70700 1 -- 172.21.15.7:0/1757710958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff83c101d50 msgr2=0x7ff83c19dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:37.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.780+0000 7ff840b70700 1 --2- 172.21.15.7:0/1757710958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff83c101d50 0x7ff83c19dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.780+0000 7ff840b70700 1 -- 172.21.15.7:0/1757710958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff83c101390 msgr2=0x7ff83c19d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:37.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.780+0000 7ff840b70700 1 --2- 172.21.15.7:0/1757710958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff83c101390 0x7ff83c19d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.780+0000 7ff840b70700 1 -- 172.21.15.7:0/1757710958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff83c197fd0 con 0x7ff83c107400 2024-06-10T01:06:37.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.780+0000 7ff83b7fe700 1 --2- 172.21.15.7:0/1757710958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff83c101d50 0x7ff83c19dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:06:37.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.780+0000 7ff840b70700 1 --2- 172.21.15.7:0/1757710958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff83c107400 0x7ff83c197870 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff8340062a0 tx=0x7ff834004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:37.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.780+0000 7ff8397fa700 1 -- 172.21.15.7:0/1757710958 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff834021070 con 0x7ff83c107400 2024-06-10T01:06:37.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.780+0000 7ff8397fa700 1 -- 172.21.15.7:0/1757710958 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff834007c00 con 0x7ff83c107400 2024-06-10T01:06:37.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.780+0000 7ff8397fa700 1 -- 172.21.15.7:0/1757710958 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83401d450 con 0x7ff83c107400 2024-06-10T01:06:37.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.780+0000 7ff8425d3700 1 -- 172.21.15.7:0/1757710958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff83c198260 con 0x7ff83c107400 2024-06-10T01:06:37.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.780+0000 7ff8425d3700 1 -- 172.21.15.7:0/1757710958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff83c06a170 con 0x7ff83c107400 2024-06-10T01:06:37.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.783+0000 7ff8397fa700 1 -- 172.21.15.7:0/1757710958 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7ff834004020 con 0x7ff83c107400 2024-06-10T01:06:37.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.784+0000 7ff8397fa700 1 --2- 172.21.15.7:0/1757710958 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff8240a2ca0 0x7ff8240a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:37.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.784+0000 7ff83bfff700 1 --2- 172.21.15.7:0/1757710958 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff8240a2ca0 0x7ff8240a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:37.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.784+0000 7ff8397fa700 1 -- 172.21.15.7:0/1757710958 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7ff83400f020 con 0x7ff83c107400 2024-06-10T01:06:37.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.784+0000 7ff84ce56b80 1 Processor -- start 2024-06-10T01:06:37.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.785+0000 7ff84ce56b80 1 -- start start 2024-06-10T01:06:37.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.785+0000 7ff84ce56b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964d5cf640 0x55964d5cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:37.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.785+0000 7ff84ce56b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55964d5cff60 0x55964d5c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:37.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.785+0000 7ff84ce56b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964d5c9f20 0x55964d5ca340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:37.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.785+0000 7ff84ce56b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55964d51b810 con 0x55964d5cf640 2024-06-10T01:06:37.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.785+0000 7ff84ce56b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55964d4c3720 con 0x55964d5c9f20 2024-06-10T01:06:37.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.785+0000 7ff84ce56b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55964d5177d0 con 0x55964d5cff60 2024-06-10T01:06:37.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.785+0000 7ff83b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55964d5cff60 0x55964d5c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:37.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.785+0000 7ff83bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964d5cf640 0x55964d5cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:37.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.785+0000 7ff83b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55964d5cff60 0x55964d5c99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:54490/0 (socket says 172.21.15.7:54490) 2024-06-10T01:06:37.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.785+0000 7ff83b7fe700 1 -- 172.21.15.7:0/2607803319 learned_addr learned my addr 172.21.15.7:0/2607803319 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:37.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.785+0000 7ff83bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964d5cf640 0x55964d5cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38116/0 (socket says 172.21.15.7:38116) 2024-06-10T01:06:37.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.785+0000 7ff83bfff700 1 --2- 172.21.15.7:0/1757710958 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff8240a2ca0 0x7ff8240a5160 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff83c1989b0 tx=0x7ff82c00f3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:37.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.785+0000 7ff840b70700 1 --2- 172.21.15.7:0/2607803319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964d5c9f20 0x55964d5ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:37.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.785+0000 7ff83bfff700 1 -- 172.21.15.7:0/2607803319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55964d5cff60 msgr2=0x55964d5c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:37.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.785+0000 7ff83bfff700 1 --2- 172.21.15.7:0/2607803319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55964d5cff60 0x55964d5c99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.785+0000 7ff83bfff700 1 -- 172.21.15.7:0/2607803319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964d5c9f20 msgr2=0x55964d5ca340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:37.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.785+0000 7ff83bfff700 1 --2- 172.21.15.7:0/2607803319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964d5c9f20 0x55964d5ca340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.785+0000 7ff83bfff700 1 -- 172.21.15.7:0/2607803319 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55964d4b9f50 con 0x55964d5cf640 2024-06-10T01:06:37.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.786+0000 7ff840b70700 1 --2- 172.21.15.7:0/2607803319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964d5c9f20 0x55964d5ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:06:37.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.786+0000 7ff83b7fe700 1 --2- 172.21.15.7:0/2607803319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55964d5cff60 0x55964d5c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:37.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.786+0000 7ff83bfff700 1 --2- 172.21.15.7:0/2607803319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964d5cf640 0x55964d5cfa20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff82c00f7c0 tx=0x7ff82c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801ddc2fd0504330 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:37.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.786+0000 7ff8227fc700 1 -- 172.21.15.7:0/2607803319 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c018070 con 0x55964d5cf640 2024-06-10T01:06:37.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.786+0000 7ff8227fc700 1 -- 172.21.15.7:0/2607803319 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff82c00fe60 con 0x55964d5cf640 2024-06-10T01:06:37.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.786+0000 7ff8227fc700 1 -- 172.21.15.7:0/2607803319 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c003c10 con 0x55964d5cf640 2024-06-10T01:06:37.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.787+0000 7ff84ce56b80 1 -- 172.21.15.7:0/2607803319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964d5cf640 msgr2=0x55964d5cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:37.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.787+0000 7ff84ce56b80 1 --2- 172.21.15.7:0/2607803319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964d5cf640 0x55964d5cfa20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff82c00f7c0 tx=0x7ff82c01a000 comp rx=0 tx=0).stop 2024-06-10T01:06:37.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.787+0000 7ff84ce56b80 1 -- 172.21.15.7:0/2607803319 shutdown_connections 2024-06-10T01:06:37.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.787+0000 7ff84ce56b80 1 --2- 172.21.15.7:0/2607803319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964d5c9f20 0x55964d5ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.787+0000 7ff84ce56b80 1 --2- 172.21.15.7:0/2607803319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964d5cf640 0x55964d5cfa20 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.787+0000 7ff84ce56b80 1 --2- 172.21.15.7:0/2607803319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55964d5cff60 0x55964d5c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.787+0000 7ff84ce56b80 1 -- 172.21.15.7:0/2607803319 >> 172.21.15.7:0/2607803319 conn(0x55964d4f4cc0 msgr2=0x55964d5c4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:37.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.787+0000 7ff84ce56b80 1 -- 172.21.15.7:0/2607803319 shutdown_connections 2024-06-10T01:06:37.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.787+0000 7ff84ce56b80 1 -- 172.21.15.7:0/2607803319 wait complete. 2024-06-10T01:06:37.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.787+0000 7ff84ce56b80 1 -- start start 2024-06-10T01:06:37.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.787+0000 7ff84ce56b80 1 Processor -- start 2024-06-10T01:06:37.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.788+0000 7ff84ce56b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964d5c9f20 0x55964d5d5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:37.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.788+0000 7ff84ce56b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964d5cf640 0x55964d5d3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:37.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.788+0000 7ff84ce56b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55964d5cff60 0x55964d5d3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:37.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.788+0000 7ff84ce56b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55964d5d69d0 con 0x55964d5c9f20 2024-06-10T01:06:37.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.788+0000 7ff84ce56b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55964d4bbbf0 con 0x55964d5cf640 2024-06-10T01:06:37.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.788+0000 7ff84ce56b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55964d5d6b50 con 0x55964d5cff60 2024-06-10T01:06:37.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.788+0000 7ff84ce56b80 10 client.0 _ll_register_callbacks cb 0x55964d4b5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:37.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.788+0000 7ff83bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964d5c9f20 0x55964d5d5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:37.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.788+0000 7ff83bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964d5c9f20 0x55964d5d5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38120/0 (socket says 172.21.15.7:38120) 2024-06-10T01:06:37.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.788+0000 7ff83bfff700 1 -- 172.21.15.7:0/3121668277 learned_addr learned my addr 172.21.15.7:0/3121668277 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:37.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.788+0000 7ff840b70700 1 --2- 172.21.15.7:0/3121668277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55964d5cff60 0x55964d5d3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:37.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.788+0000 7ff83b7fe700 1 --2- 172.21.15.7:0/3121668277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964d5cf640 0x55964d5d3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:37.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.788+0000 7ff83bfff700 1 -- 172.21.15.7:0/3121668277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55964d5cff60 msgr2=0x55964d5d3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:37.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.788+0000 7ff83bfff700 1 --2- 172.21.15.7:0/3121668277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55964d5cff60 0x55964d5d3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.788+0000 7ff83bfff700 1 -- 172.21.15.7:0/3121668277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964d5cf640 msgr2=0x55964d5d3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:37.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.788+0000 7ff83bfff700 1 --2- 172.21.15.7:0/3121668277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964d5cf640 0x55964d5d3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.788+0000 7ff83bfff700 1 -- 172.21.15.7:0/3121668277 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55964d46a8d0 con 0x55964d5c9f20 2024-06-10T01:06:37.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.788+0000 7ff840b70700 1 --2- 172.21.15.7:0/3121668277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55964d5cff60 0x55964d5d3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:06:37.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.788+0000 7ff83b7fe700 1 --2- 172.21.15.7:0/3121668277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964d5cf640 0x55964d5d3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:06:37.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.789+0000 7ff83bfff700 1 --2- 172.21.15.7:0/3121668277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964d5c9f20 0x55964d5d5d30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff82c00a990 tx=0x7ff82c003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf694f20d8298be server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:37.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.789+0000 7ff81b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:06:37.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.789+0000 7ff81b7fe700 1 -- 172.21.15.7:0/3121668277 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c018030 con 0x55964d5c9f20 2024-06-10T01:06:37.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.789+0000 7ff81b7fe700 1 -- 172.21.15.7:0/3121668277 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7ff82c0054d0 con 0x55964d5c9f20 2024-06-10T01:06:37.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.789+0000 7ff84ce56b80 1 -- 172.21.15.7:0/3121668277 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff82c00fe60 con 0x55964d5c9f20 2024-06-10T01:06:37.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.789+0000 7ff81b7fe700 1 -- 172.21.15.7:0/3121668277 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff82c02d450 con 0x55964d5c9f20 2024-06-10T01:06:37.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.789+0000 7ff84ce56b80 10 client.15498 fetch_fsmap learned FSMap version 27 2024-06-10T01:06:37.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.789+0000 7ff84ce56b80 1 -- 172.21.15.7:0/3121668277 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55964d5e12d0 con 0x55964d5c9f20 2024-06-10T01:06:37.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.789+0000 7ff81b7fe700 1 -- 172.21.15.7:0/3121668277 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff82c00fe60 con 0x55964d5c9f20 2024-06-10T01:06:37.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.789+0000 7ff81b7fe700 1 -- 172.21.15.7:0/3121668277 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff82c0104c0 con 0x55964d5c9f20 2024-06-10T01:06:37.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.789+0000 7ff84ce56b80 10 client.15498 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:06:37.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.789+0000 7ff84ce56b80 10 client.15498 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:06:37.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.789+0000 7ff84ce56b80 20 client.15498 populate_metadata read hostname 'smithi007' 2024-06-10T01:06:37.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.789+0000 7ff84ce56b80 1 --2- 172.21.15.7:0/3121668277 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55964d5e38b0 0x55964d5e5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:37.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.789+0000 7ff84ce56b80 4 client.15498 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:06:37.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.789+0000 7ff84ce56b80 1 -- 172.21.15.7:0/3121668277 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55964d528d10 con 0x55964d5e38b0 2024-06-10T01:06:37.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.790+0000 7ff83b7fe700 1 --2- 172.21.15.7:0/3121668277 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55964d5e38b0 0x55964d5e5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:37.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.790+0000 7ff81b7fe700 10 client.15498 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:06:37.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.790+0000 7ff83b7fe700 1 --2- 172.21.15.7:0/3121668277 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55964d5e38b0 0x55964d5e5ca0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b8ee6f69f7f23d86 server_cookie=f7e90d79eb4723b2 in_seq=0 out_seq=0 2024-06-10T01:06:37.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.791+0000 7ff81b7fe700 1 -- 172.21.15.7:0/3121668277 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55964d528d10 con 0x55964d5e38b0 2024-06-10T01:06:37.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.791+0000 7ff81b7fe700 10 client.15498 handle_command_reply: tid=0 2024-06-10T01:06:37.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.791+0000 7ff84ce56b80 1 client.15498 shutdown 2024-06-10T01:06:37.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.791+0000 7ff84ce56b80 1 -- 172.21.15.7:0/3121668277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964d5c9f20 msgr2=0x55964d5d5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:37.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.791+0000 7ff84ce56b80 1 --2- 172.21.15.7:0/3121668277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964d5c9f20 0x55964d5d5d30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff82c00a990 tx=0x7ff82c003c30 comp rx=0 tx=0).stop 2024-06-10T01:06:37.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.791+0000 7ff84ce56b80 1 -- 172.21.15.7:0/3121668277 shutdown_connections 2024-06-10T01:06:37.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.791+0000 7ff84ce56b80 1 --2- 172.21.15.7:0/3121668277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55964d5cff60 0x55964d5d3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.791+0000 7ff84ce56b80 1 --2- 172.21.15.7:0/3121668277 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55964d5e38b0 0x55964d5e5ca0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.791+0000 7ff84ce56b80 1 --2- 172.21.15.7:0/3121668277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964d5c9f20 0x55964d5d5d30 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.791+0000 7ff84ce56b80 1 --2- 172.21.15.7:0/3121668277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964d5cf640 0x55964d5d3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.791+0000 7ff84ce56b80 1 -- 172.21.15.7:0/3121668277 >> 172.21.15.7:0/3121668277 conn(0x55964d4f4cc0 msgr2=0x55964d5c89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:37.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.791+0000 7ff81b7fe700 0 client.15498 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:06:37.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.791+0000 7ff84ce56b80 1 -- 172.21.15.7:0/3121668277 shutdown_connections 2024-06-10T01:06:37.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.792+0000 7ff84ce56b80 1 -- 172.21.15.7:0/3121668277 wait complete. 2024-06-10T01:06:37.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.792+0000 7ff84ce56b80 20 client.15498 trim_cache size 0 max 16384 2024-06-10T01:06:37.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.811+0000 7ff84ce56b80 1 Processor -- start 2024-06-10T01:06:37.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.811+0000 7ff84ce56b80 1 -- start start 2024-06-10T01:06:37.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.811+0000 7ff84ce56b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55964d5c9f20 0x55964d5c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:37.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.811+0000 7ff84ce56b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964d5cf640 0x55964d5d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:37.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.811+0000 7ff84ce56b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964d5cff60 0x55964d5be510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:37.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.811+0000 7ff84ce56b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c1b2c50 con 0x55964d5cf640 2024-06-10T01:06:37.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.811+0000 7ff84ce56b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c1b04e0 con 0x55964d5cff60 2024-06-10T01:06:37.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.811+0000 7ff84ce56b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c1ae330 con 0x55964d5c9f20 2024-06-10T01:06:37.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.811+0000 7ff83b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964d5cf640 0x55964d5d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:37.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.811+0000 7ff83b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964d5cf640 0x55964d5d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38130/0 (socket says 172.21.15.7:38130) 2024-06-10T01:06:37.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.811+0000 7ff83b7fe700 1 -- 172.21.15.7:0/2017264487 learned_addr learned my addr 172.21.15.7:0/2017264487 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:37.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.811+0000 7ff840b70700 1 --2- 172.21.15.7:0/2017264487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964d5cff60 0x55964d5be510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:37.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.811+0000 7ff83bfff700 1 --2- 172.21.15.7:0/2017264487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55964d5c9f20 0x55964d5c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:37.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.811+0000 7ff83b7fe700 1 -- 172.21.15.7:0/2017264487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55964d5c9f20 msgr2=0x55964d5c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:37.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.811+0000 7ff83b7fe700 1 --2- 172.21.15.7:0/2017264487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55964d5c9f20 0x55964d5c99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.812+0000 7ff83b7fe700 1 -- 172.21.15.7:0/2017264487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964d5cff60 msgr2=0x55964d5be510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:37.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.812+0000 7ff83b7fe700 1 --2- 172.21.15.7:0/2017264487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964d5cff60 0x55964d5be510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.812+0000 7ff83b7fe700 1 -- 172.21.15.7:0/2017264487 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55964d5d5970 con 0x55964d5cf640 2024-06-10T01:06:37.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.812+0000 7ff840b70700 1 --2- 172.21.15.7:0/2017264487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964d5cff60 0x55964d5be510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:06:37.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.812+0000 7ff83b7fe700 1 --2- 172.21.15.7:0/2017264487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964d5cf640 0x55964d5d0bd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x55964d5d4600 tx=0x7ff83000b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6e89025211ab21 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:37.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.812+0000 7ff821ffb700 1 -- 172.21.15.7:0/2017264487 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830023070 con 0x55964d5cf640 2024-06-10T01:06:37.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.812+0000 7ff821ffb700 1 -- 172.21.15.7:0/2017264487 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff830012e20 con 0x55964d5cf640 2024-06-10T01:06:37.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.812+0000 7ff821ffb700 1 -- 172.21.15.7:0/2017264487 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83001f730 con 0x55964d5cf640 2024-06-10T01:06:37.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.813+0000 7ff84ce56b80 1 -- 172.21.15.7:0/2017264487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964d5cf640 msgr2=0x55964d5d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:37.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.813+0000 7ff84ce56b80 1 --2- 172.21.15.7:0/2017264487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964d5cf640 0x55964d5d0bd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x55964d5d4600 tx=0x7ff83000b680 comp rx=0 tx=0).stop 2024-06-10T01:06:37.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.813+0000 7ff84ce56b80 1 -- 172.21.15.7:0/2017264487 shutdown_connections 2024-06-10T01:06:37.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.813+0000 7ff84ce56b80 1 --2- 172.21.15.7:0/2017264487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964d5cff60 0x55964d5be510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.813+0000 7ff84ce56b80 1 --2- 172.21.15.7:0/2017264487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55964d5c9f20 0x55964d5c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.813+0000 7ff84ce56b80 1 --2- 172.21.15.7:0/2017264487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964d5cf640 0x55964d5d0bd0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.813+0000 7ff84ce56b80 1 -- 172.21.15.7:0/2017264487 >> 172.21.15.7:0/2017264487 conn(0x55964d4f4cc0 msgr2=0x55964d5bde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:37.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.813+0000 7ff84ce56b80 1 -- 172.21.15.7:0/2017264487 shutdown_connections 2024-06-10T01:06:37.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.813+0000 7ff84ce56b80 1 -- 172.21.15.7:0/2017264487 wait complete. 2024-06-10T01:06:37.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.813+0000 7ff84ce56b80 1 -- start start 2024-06-10T01:06:37.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.814+0000 7ff84ce56b80 1 Processor -- start 2024-06-10T01:06:37.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.814+0000 7ff84ce56b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964d5c9f20 0x55964d6465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:37.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.814+0000 7ff84ce56b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55964d5cf640 0x55964d646b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:37.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.814+0000 7ff84ce56b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964d5cff60 0x55964d5e6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:37.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.814+0000 7ff84ce56b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c1ae9a0 con 0x55964d5c9f20 2024-06-10T01:06:37.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.814+0000 7ff83bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964d5c9f20 0x55964d6465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:37.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.814+0000 7ff84ce56b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c1afe10 con 0x55964d5cff60 2024-06-10T01:06:37.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.814+0000 7ff84ce56b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c1af070 con 0x55964d5cf640 2024-06-10T01:06:37.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.814+0000 7ff83bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964d5c9f20 0x55964d6465c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38144/0 (socket says 172.21.15.7:38144) 2024-06-10T01:06:37.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.814+0000 7ff83bfff700 1 -- 172.21.15.7:0/827677674 learned_addr learned my addr 172.21.15.7:0/827677674 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:37.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.814+0000 7ff84ce56b80 10 client.0 _ll_register_callbacks cb 0x55964d4b5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:37.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.814+0000 7ff83b7fe700 1 --2- 172.21.15.7:0/827677674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55964d5cf640 0x55964d646b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:37.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.814+0000 7ff840b70700 1 --2- 172.21.15.7:0/827677674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964d5cff60 0x55964d5e6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:37.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.814+0000 7ff83bfff700 1 -- 172.21.15.7:0/827677674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55964d5cf640 msgr2=0x55964d646b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:37.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.814+0000 7ff83bfff700 1 --2- 172.21.15.7:0/827677674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55964d5cf640 0x55964d646b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.814+0000 7ff83bfff700 1 -- 172.21.15.7:0/827677674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964d5cff60 msgr2=0x55964d5e6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:37.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.814+0000 7ff83bfff700 1 --2- 172.21.15.7:0/827677674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964d5cff60 0x55964d5e6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.814+0000 7ff83bfff700 1 -- 172.21.15.7:0/827677674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55964d5e6a80 con 0x55964d5c9f20 2024-06-10T01:06:37.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.814+0000 7ff840b70700 1 --2- 172.21.15.7:0/827677674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964d5cff60 0x55964d5e6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:06:37.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.815+0000 7ff83bfff700 1 --2- 172.21.15.7:0/827677674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964d5c9f20 0x55964d6465c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff82c00fd70 tx=0x7ff82c012b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9271140031ee29c1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:37.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.815+0000 7ff81b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:06:37.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.815+0000 7ff81b7fe700 1 -- 172.21.15.7:0/827677674 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c018070 con 0x55964d5c9f20 2024-06-10T01:06:37.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.815+0000 7ff81b7fe700 1 -- 172.21.15.7:0/827677674 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7ff82c0052d0 con 0x55964d5c9f20 2024-06-10T01:06:37.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.815+0000 7ff84ce56b80 1 -- 172.21.15.7:0/827677674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff830012e20 con 0x55964d5c9f20 2024-06-10T01:06:37.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.815+0000 7ff81b7fe700 1 -- 172.21.15.7:0/827677674 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff82c00f7c0 con 0x55964d5c9f20 2024-06-10T01:06:37.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.815+0000 7ff84ce56b80 10 client.15504 fetch_fsmap learned FSMap version 27 2024-06-10T01:06:37.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.815+0000 7ff84ce56b80 1 -- 172.21.15.7:0/827677674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55964d64de00 con 0x55964d5c9f20 2024-06-10T01:06:37.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.815+0000 7ff81b7fe700 1 -- 172.21.15.7:0/827677674 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff830012e20 con 0x55964d5c9f20 2024-06-10T01:06:37.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.816+0000 7ff81b7fe700 1 -- 172.21.15.7:0/827677674 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff82c039030 con 0x55964d5c9f20 2024-06-10T01:06:37.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.816+0000 7ff84ce56b80 10 client.15504 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:06:37.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.816+0000 7ff84ce56b80 10 client.15504 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:06:37.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.816+0000 7ff84ce56b80 20 client.15504 populate_metadata read hostname 'smithi007' 2024-06-10T01:06:37.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.816+0000 7ff84ce56b80 1 --2- 172.21.15.7:0/827677674 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55964d64e540 0x55964d650960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:37.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.816+0000 7ff84ce56b80 4 client.15504 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:06:37.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.816+0000 7ff84ce56b80 1 -- 172.21.15.7:0/827677674 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55964d523290 con 0x55964d64e540 2024-06-10T01:06:37.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.816+0000 7ff83b7fe700 1 --2- 172.21.15.7:0/827677674 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55964d64e540 0x55964d650960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:37.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.816+0000 7ff83b7fe700 1 --2- 172.21.15.7:0/827677674 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55964d64e540 0x55964d650960 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=81b656a5cd0d89b server_cookie=6b152328451b57a5 in_seq=0 out_seq=0 2024-06-10T01:06:37.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.816+0000 7ff81b7fe700 10 client.15504 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:06:37.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.817+0000 7ff81b7fe700 1 -- 172.21.15.7:0/827677674 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55964d523290 con 0x55964d64e540 2024-06-10T01:06:37.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.817+0000 7ff81b7fe700 10 client.15504 handle_command_reply: tid=0 2024-06-10T01:06:37.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.817+0000 7ff84ce56b80 1 client.15504 shutdown 2024-06-10T01:06:37.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.817+0000 7ff84ce56b80 1 -- 172.21.15.7:0/827677674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964d5c9f20 msgr2=0x55964d6465c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:37.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.817+0000 7ff84ce56b80 1 --2- 172.21.15.7:0/827677674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964d5c9f20 0x55964d6465c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff82c00fd70 tx=0x7ff82c012b30 comp rx=0 tx=0).stop 2024-06-10T01:06:37.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.817+0000 7ff84ce56b80 1 -- 172.21.15.7:0/827677674 shutdown_connections 2024-06-10T01:06:37.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.817+0000 7ff84ce56b80 1 --2- 172.21.15.7:0/827677674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964d5cff60 0x55964d5e6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.817+0000 7ff84ce56b80 1 --2- 172.21.15.7:0/827677674 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55964d64e540 0x55964d650960 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.817+0000 7ff81b7fe700 0 client.15504 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:06:37.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.817+0000 7ff84ce56b80 1 --2- 172.21.15.7:0/827677674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964d5c9f20 0x55964d6465c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.817+0000 7ff84ce56b80 1 --2- 172.21.15.7:0/827677674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55964d5cf640 0x55964d646b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.817+0000 7ff84ce56b80 1 -- 172.21.15.7:0/827677674 >> 172.21.15.7:0/827677674 conn(0x55964d4f4cc0 msgr2=0x55964d5be510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:37.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.817+0000 7ff84ce56b80 1 -- 172.21.15.7:0/827677674 shutdown_connections 2024-06-10T01:06:37.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.817+0000 7ff84ce56b80 1 -- 172.21.15.7:0/827677674 wait complete. 2024-06-10T01:06:37.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.817+0000 7ff84ce56b80 20 client.15504 trim_cache size 0 max 16384 2024-06-10T01:06:37.818 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:06:37.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.818+0000 7ff8425d3700 1 -- 172.21.15.7:0/1757710958 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff8240a2ca0 msgr2=0x7ff8240a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:37.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.818+0000 7ff8425d3700 1 --2- 172.21.15.7:0/1757710958 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff8240a2ca0 0x7ff8240a5160 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff83c1989b0 tx=0x7ff82c00f3c0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.818+0000 7ff8425d3700 1 -- 172.21.15.7:0/1757710958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff83c107400 msgr2=0x7ff83c197870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:37.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.818+0000 7ff8425d3700 1 --2- 172.21.15.7:0/1757710958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff83c107400 0x7ff83c197870 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff8340062a0 tx=0x7ff834004d70 comp rx=0 tx=0).stop 2024-06-10T01:06:37.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.818+0000 7ff8425d3700 1 -- 172.21.15.7:0/1757710958 shutdown_connections 2024-06-10T01:06:37.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.818+0000 7ff8425d3700 1 --2- 172.21.15.7:0/1757710958 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff8240a2ca0 0x7ff8240a5160 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.818+0000 7ff8425d3700 1 --2- 172.21.15.7:0/1757710958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff83c107400 0x7ff83c197870 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.818+0000 7ff8425d3700 1 --2- 172.21.15.7:0/1757710958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff83c101390 0x7ff83c19d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.818+0000 7ff8425d3700 1 --2- 172.21.15.7:0/1757710958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff83c101d50 0x7ff83c19dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:37.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.818+0000 7ff8425d3700 1 -- 172.21.15.7:0/1757710958 >> 172.21.15.7:0/1757710958 conn(0x7ff83c078340 msgr2=0x7ff83c0ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:37.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.818+0000 7ff8425d3700 1 -- 172.21.15.7:0/1757710958 shutdown_connections 2024-06-10T01:06:37.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:37.818+0000 7ff8425d3700 1 -- 172.21.15.7:0/1757710958 wait complete. 2024-06-10T01:06:37.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:37 smithi078 ceph-mon[36683]: pgmap v449: 129 pgs: 129 active+clean; 661 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:06:38.619 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:06:38.879 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:06:39.699 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:39 smithi007 ceph-mon[32398]: pgmap v450: 129 pgs: 129 active+clean; 661 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-10T01:06:39.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:39 smithi175 ceph-mon[36636]: pgmap v450: 129 pgs: 129 active+clean; 661 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-10T01:06:39.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:39 smithi078 ceph-mon[36683]: pgmap v450: 129 pgs: 129 active+clean; 661 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-10T01:06:40.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.665+0000 7f8902346700 1 -- 172.21.15.7:0/1583182732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88fc108510 msgr2=0x7f88fc1088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:40.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.665+0000 7f8902346700 1 --2- 172.21.15.7:0/1583182732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88fc108510 0x7f88fc1088f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f88ec009990 tx=0x7f88ec009ca0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.666+0000 7f8902346700 1 -- 172.21.15.7:0/1583182732 shutdown_connections 2024-06-10T01:06:40.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.666+0000 7f8902346700 1 --2- 172.21.15.7:0/1583182732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88fc101440 0x7f88fc10f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.666+0000 7f8902346700 1 --2- 172.21.15.7:0/1583182732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88fc108510 0x7f88fc1088f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.666+0000 7f8902346700 1 --2- 172.21.15.7:0/1583182732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f88fc100a80 0x7f88fc100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.666+0000 7f8902346700 1 -- 172.21.15.7:0/1583182732 >> 172.21.15.7:0/1583182732 conn(0x7f88fc077a00 msgr2=0x7f88fc0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:40.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.666+0000 7f8902346700 1 -- 172.21.15.7:0/1583182732 shutdown_connections 2024-06-10T01:06:40.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.666+0000 7f8902346700 1 -- 172.21.15.7:0/1583182732 wait complete. 2024-06-10T01:06:40.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.666+0000 7f8902346700 1 Processor -- start 2024-06-10T01:06:40.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.666+0000 7f8902346700 1 -- start start 2024-06-10T01:06:40.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.667+0000 7f8902346700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88fc100a80 0x7f88fc19e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:40.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.667+0000 7f8902346700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88fc101440 0x7f88fc19ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.667+0000 7f8902346700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f88fc108510 0x7f88fc198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.667+0000 7f8902346700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc112790 con 0x7f88fc101440 2024-06-10T01:06:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.667+0000 7f8902346700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc112610 con 0x7f88fc108510 2024-06-10T01:06:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.667+0000 7f8902346700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc112910 con 0x7f88fc100a80 2024-06-10T01:06:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.667+0000 7f88f3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88fc101440 0x7f88fc19ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.667+0000 7f88f3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88fc101440 0x7f88fc19ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38164/0 (socket says 172.21.15.7:38164) 2024-06-10T01:06:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.667+0000 7f88f3fff700 1 -- 172.21.15.7:0/2088198301 learned_addr learned my addr 172.21.15.7:0/2088198301 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.667+0000 7f88fbfff700 1 --2- 172.21.15.7:0/2088198301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88fc100a80 0x7f88fc19e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.667+0000 7f89008e3700 1 --2- 172.21.15.7:0/2088198301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f88fc108510 0x7f88fc198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.667+0000 7f88f3fff700 1 -- 172.21.15.7:0/2088198301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88fc100a80 msgr2=0x7f88fc19e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.667+0000 7f88f3fff700 1 --2- 172.21.15.7:0/2088198301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88fc100a80 0x7f88fc19e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.667+0000 7f88f3fff700 1 -- 172.21.15.7:0/2088198301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f88fc108510 msgr2=0x7f88fc198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.667+0000 7f88f3fff700 1 --2- 172.21.15.7:0/2088198301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f88fc108510 0x7f88fc198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.667+0000 7f88f3fff700 1 -- 172.21.15.7:0/2088198301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88fc1990d0 con 0x7f88fc101440 2024-06-10T01:06:40.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.667+0000 7f89008e3700 1 --2- 172.21.15.7:0/2088198301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f88fc108510 0x7f88fc198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:06:40.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.668+0000 7f88f3fff700 1 --2- 172.21.15.7:0/2088198301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88fc101440 0x7f88fc19ecf0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f88e400c9b0 tx=0x7f88e400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:40.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.668+0000 7f88f9ffb700 1 -- 172.21.15.7:0/2088198301 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e4007b00 con 0x7f88fc101440 2024-06-10T01:06:40.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.668+0000 7f8902346700 1 -- 172.21.15.7:0/2088198301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88fc1993c0 con 0x7f88fc101440 2024-06-10T01:06:40.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.668+0000 7f88f9ffb700 1 -- 172.21.15.7:0/2088198301 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88e4004d10 con 0x7f88fc101440 2024-06-10T01:06:40.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.668+0000 7f88f9ffb700 1 -- 172.21.15.7:0/2088198301 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e4005650 con 0x7f88fc101440 2024-06-10T01:06:40.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.669+0000 7f8902346700 1 -- 172.21.15.7:0/2088198301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88fc101b90 con 0x7f88fc101440 2024-06-10T01:06:40.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.671+0000 7f88f9ffb700 1 -- 172.21.15.7:0/2088198301 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f88e4007ca0 con 0x7f88fc101440 2024-06-10T01:06:40.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.672+0000 7f88f9ffb700 1 --2- 172.21.15.7:0/2088198301 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f88e80a2ca0 0x7f88e80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:40.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.672+0000 7f88f9ffb700 1 -- 172.21.15.7:0/2088198301 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f88e40cf030 con 0x7f88fc101440 2024-06-10T01:06:40.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.672+0000 7f88fbfff700 1 --2- 172.21.15.7:0/2088198301 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f88e80a2ca0 0x7f88e80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:40.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.673+0000 7f88fbfff700 1 --2- 172.21.15.7:0/2088198301 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f88e80a2ca0 0x7f88e80a5160 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f88fc199ab0 tx=0x7f88ec005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:40.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.673+0000 7f890cbc9b80 1 Processor -- start 2024-06-10T01:06:40.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.673+0000 7f890cbc9b80 1 -- start start 2024-06-10T01:06:40.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.673+0000 7f890cbc9b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561fc1baf60 0x5561fc1bb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:40.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.673+0000 7f890cbc9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561fc1bb880 0x5561fc1b5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:40.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.673+0000 7f890cbc9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5561fc1b57a0 0x5561fc1b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:40.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.673+0000 7f890cbc9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5561fc0a7f60 con 0x5561fc1b57a0 2024-06-10T01:06:40.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.673+0000 7f890cbc9b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5561fc106850 con 0x5561fc1baf60 2024-06-10T01:06:40.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.673+0000 7f890cbc9b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5561fc06e460 con 0x5561fc1bb880 2024-06-10T01:06:40.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.673+0000 7f89008e3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5561fc1b57a0 0x5561fc1b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:40.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.674+0000 7f89008e3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5561fc1b57a0 0x5561fc1b5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38168/0 (socket says 172.21.15.7:38168) 2024-06-10T01:06:40.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.674+0000 7f89008e3700 1 -- 172.21.15.7:0/2148017364 learned_addr learned my addr 172.21.15.7:0/2148017364 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:40.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.674+0000 7f88fbfff700 1 --2- 172.21.15.7:0/2148017364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561fc1baf60 0x5561fc1bb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:40.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.674+0000 7f88f3fff700 1 --2- 172.21.15.7:0/2148017364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561fc1bb880 0x5561fc1b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:40.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.674+0000 7f89008e3700 1 -- 172.21.15.7:0/2148017364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561fc1bb880 msgr2=0x5561fc1b5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:40.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.674+0000 7f89008e3700 1 --2- 172.21.15.7:0/2148017364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561fc1bb880 0x5561fc1b5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.674+0000 7f89008e3700 1 -- 172.21.15.7:0/2148017364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561fc1baf60 msgr2=0x5561fc1bb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:40.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.674+0000 7f89008e3700 1 --2- 172.21.15.7:0/2148017364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561fc1baf60 0x5561fc1bb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.674+0000 7f89008e3700 1 -- 172.21.15.7:0/2148017364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561fc0558d0 con 0x5561fc1b57a0 2024-06-10T01:06:40.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.674+0000 7f88f3fff700 1 --2- 172.21.15.7:0/2148017364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561fc1bb880 0x5561fc1b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:06:40.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.674+0000 7f89008e3700 1 --2- 172.21.15.7:0/2148017364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5561fc1b57a0 0x5561fc1b5bc0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f88f4009cd0 tx=0x7f88f4007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb264c4d5220d620 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:40.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.674+0000 7f88f27fc700 1 -- 172.21.15.7:0/2148017364 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f4019070 con 0x5561fc1b57a0 2024-06-10T01:06:40.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.674+0000 7f88f27fc700 1 -- 172.21.15.7:0/2148017364 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88f40148e0 con 0x5561fc1b57a0 2024-06-10T01:06:40.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.674+0000 7f88f27fc700 1 -- 172.21.15.7:0/2148017364 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f40139a0 con 0x5561fc1b57a0 2024-06-10T01:06:40.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.675+0000 7f890cbc9b80 1 -- 172.21.15.7:0/2148017364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5561fc1b57a0 msgr2=0x5561fc1b5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:40.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.675+0000 7f890cbc9b80 1 --2- 172.21.15.7:0/2148017364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5561fc1b57a0 0x5561fc1b5bc0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f88f4009cd0 tx=0x7f88f4007500 comp rx=0 tx=0).stop 2024-06-10T01:06:40.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.675+0000 7f890cbc9b80 1 -- 172.21.15.7:0/2148017364 shutdown_connections 2024-06-10T01:06:40.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.675+0000 7f890cbc9b80 1 --2- 172.21.15.7:0/2148017364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5561fc1b57a0 0x5561fc1b5bc0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.675+0000 7f890cbc9b80 1 --2- 172.21.15.7:0/2148017364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561fc1baf60 0x5561fc1bb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.675+0000 7f890cbc9b80 1 --2- 172.21.15.7:0/2148017364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561fc1bb880 0x5561fc1b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.675+0000 7f890cbc9b80 1 -- 172.21.15.7:0/2148017364 >> 172.21.15.7:0/2148017364 conn(0x5561fc0e2080 msgr2=0x5561fc1b0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:40.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.675+0000 7f890cbc9b80 1 -- 172.21.15.7:0/2148017364 shutdown_connections 2024-06-10T01:06:40.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.675+0000 7f890cbc9b80 1 -- 172.21.15.7:0/2148017364 wait complete. 2024-06-10T01:06:40.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.676+0000 7f890cbc9b80 1 -- start start 2024-06-10T01:06:40.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.676+0000 7f890cbc9b80 1 Processor -- start 2024-06-10T01:06:40.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.676+0000 7f890cbc9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561fc1b57a0 0x5561fc1c15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.676+0000 7f890cbc9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5561fc1baf60 0x5561fc1be8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.676+0000 7f890cbc9b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561fc1bb880 0x5561fc1bee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.676+0000 7f890cbc9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5561fc0a6fc0 con 0x5561fc1baf60 2024-06-10T01:06:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.676+0000 7f890cbc9b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5561fc102810 con 0x5561fc1bb880 2024-06-10T01:06:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.676+0000 7f890cbc9b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5561fc1c2260 con 0x5561fc1b57a0 2024-06-10T01:06:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.676+0000 7f890cbc9b80 10 client.0 _ll_register_callbacks cb 0x5561fc09c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.677+0000 7f88f3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5561fc1baf60 0x5561fc1be8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.677+0000 7f88f3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5561fc1baf60 0x5561fc1be8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38184/0 (socket says 172.21.15.7:38184) 2024-06-10T01:06:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.677+0000 7f88f3fff700 1 -- 172.21.15.7:0/1015563468 learned_addr learned my addr 172.21.15.7:0/1015563468 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.677+0000 7f88fbfff700 1 --2- 172.21.15.7:0/1015563468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561fc1b57a0 0x5561fc1c15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.677+0000 7f88f3fff700 1 -- 172.21.15.7:0/1015563468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561fc1b57a0 msgr2=0x5561fc1c15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.677+0000 7f88f3fff700 1 --2- 172.21.15.7:0/1015563468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561fc1b57a0 0x5561fc1c15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.677+0000 7f88f3fff700 1 -- 172.21.15.7:0/1015563468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561fc1bb880 msgr2=0x5561fc1bee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.677+0000 7f88f3fff700 1 --2- 172.21.15.7:0/1015563468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561fc1bb880 0x5561fc1bee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.677+0000 7f88f3fff700 1 -- 172.21.15.7:0/1015563468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561fc0cc750 con 0x5561fc1baf60 2024-06-10T01:06:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.677+0000 7f88f3fff700 1 --2- 172.21.15.7:0/1015563468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5561fc1baf60 0x5561fc1be8c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f88e4005060 tx=0x7f88e409afa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1953945c17145239 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.677+0000 7f88db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:06:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.677+0000 7f88db7fe700 1 -- 172.21.15.7:0/1015563468 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e409b510 con 0x5561fc1baf60 2024-06-10T01:06:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.677+0000 7f88db7fe700 1 -- 172.21.15.7:0/1015563468 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f88e4099810 con 0x5561fc1baf60 2024-06-10T01:06:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.677+0000 7f890cbc9b80 1 -- 172.21.15.7:0/1015563468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88f40148e0 con 0x5561fc1baf60 2024-06-10T01:06:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.677+0000 7f88db7fe700 1 -- 172.21.15.7:0/1015563468 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f88e40ab050 con 0x5561fc1baf60 2024-06-10T01:06:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.677+0000 7f890cbc9b80 10 client.15516 fetch_fsmap learned FSMap version 27 2024-06-10T01:06:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.678+0000 7f890cbc9b80 1 -- 172.21.15.7:0/1015563468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5561fc1ccaa0 con 0x5561fc1baf60 2024-06-10T01:06:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.678+0000 7f88db7fe700 1 -- 172.21.15.7:0/1015563468 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88e409f8c0 con 0x5561fc1baf60 2024-06-10T01:06:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.678+0000 7f88db7fe700 1 -- 172.21.15.7:0/1015563468 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f88e401f020 con 0x5561fc1baf60 2024-06-10T01:06:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.678+0000 7f890cbc9b80 10 client.15516 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:06:40.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.678+0000 7f890cbc9b80 10 client.15516 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:06:40.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.678+0000 7f890cbc9b80 20 client.15516 populate_metadata read hostname 'smithi007' 2024-06-10T01:06:40.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.678+0000 7f890cbc9b80 1 --2- 172.21.15.7:0/1015563468 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5561fc1cf0e0 0x5561fc1d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:40.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.678+0000 7f890cbc9b80 4 client.15516 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:06:40.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.678+0000 7f890cbc9b80 1 -- 172.21.15.7:0/1015563468 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f88f40148e0 con 0x5561fc1cf0e0 2024-06-10T01:06:40.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.678+0000 7f88fbfff700 1 --2- 172.21.15.7:0/1015563468 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5561fc1cf0e0 0x5561fc1d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:40.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.679+0000 7f88fbfff700 1 --2- 172.21.15.7:0/1015563468 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5561fc1cf0e0 0x5561fc1d14d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cdef7cd5141e784c server_cookie=6f904421929290dd in_seq=0 out_seq=0 2024-06-10T01:06:40.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.679+0000 7f88db7fe700 10 client.15516 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:06:40.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.680+0000 7f88db7fe700 1 -- 172.21.15.7:0/1015563468 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f88f40148e0 con 0x5561fc1cf0e0 2024-06-10T01:06:40.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.680+0000 7f88db7fe700 10 client.15516 handle_command_reply: tid=0 2024-06-10T01:06:40.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.680+0000 7f890cbc9b80 1 client.15516 shutdown 2024-06-10T01:06:40.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.680+0000 7f890cbc9b80 1 -- 172.21.15.7:0/1015563468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5561fc1baf60 msgr2=0x5561fc1be8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:40.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.680+0000 7f890cbc9b80 1 --2- 172.21.15.7:0/1015563468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5561fc1baf60 0x5561fc1be8c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f88e4005060 tx=0x7f88e409afa0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.680+0000 7f890cbc9b80 1 -- 172.21.15.7:0/1015563468 shutdown_connections 2024-06-10T01:06:40.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.680+0000 7f890cbc9b80 1 --2- 172.21.15.7:0/1015563468 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5561fc1cf0e0 0x5561fc1d14d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.680+0000 7f890cbc9b80 1 --2- 172.21.15.7:0/1015563468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561fc1bb880 0x5561fc1bee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.680+0000 7f890cbc9b80 1 --2- 172.21.15.7:0/1015563468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561fc1b57a0 0x5561fc1c15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.680+0000 7f890cbc9b80 1 --2- 172.21.15.7:0/1015563468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5561fc1baf60 0x5561fc1be8c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.680+0000 7f890cbc9b80 1 -- 172.21.15.7:0/1015563468 >> 172.21.15.7:0/1015563468 conn(0x5561fc0e2080 msgr2=0x5561fc1af700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:40.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.680+0000 7f88db7fe700 0 client.15516 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:06:40.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.680+0000 7f890cbc9b80 1 -- 172.21.15.7:0/1015563468 shutdown_connections 2024-06-10T01:06:40.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.680+0000 7f890cbc9b80 1 -- 172.21.15.7:0/1015563468 wait complete. 2024-06-10T01:06:40.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.681+0000 7f890cbc9b80 20 client.15516 trim_cache size 0 max 16384 2024-06-10T01:06:40.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.700+0000 7f890cbc9b80 1 Processor -- start 2024-06-10T01:06:40.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.700+0000 7f890cbc9b80 1 -- start start 2024-06-10T01:06:40.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.700+0000 7f890cbc9b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561fc1b57a0 0x5561fc1b5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:40.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.700+0000 7f890cbc9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5561fc1baf60 0x5561fc1bc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:40.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.700+0000 7f890cbc9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561fc1bb880 0x5561fc1bd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.700+0000 7f890cbc9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc1b2cf0 con 0x5561fc1baf60 2024-06-10T01:06:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.700+0000 7f890cbc9b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc1b0580 con 0x5561fc1b57a0 2024-06-10T01:06:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.700+0000 7f890cbc9b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc1ae3d0 con 0x5561fc1bb880 2024-06-10T01:06:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.701+0000 7f88f3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5561fc1baf60 0x5561fc1bc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.701+0000 7f88f3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5561fc1baf60 0x5561fc1bc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38200/0 (socket says 172.21.15.7:38200) 2024-06-10T01:06:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.701+0000 7f88f3fff700 1 -- 172.21.15.7:0/292029383 learned_addr learned my addr 172.21.15.7:0/292029383 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.701+0000 7f88fbfff700 1 --2- 172.21.15.7:0/292029383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561fc1b57a0 0x5561fc1b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.701+0000 7f89008e3700 1 --2- 172.21.15.7:0/292029383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561fc1bb880 0x5561fc1bd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.701+0000 7f88f3fff700 1 -- 172.21.15.7:0/292029383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561fc1bb880 msgr2=0x5561fc1bd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.701+0000 7f88f3fff700 1 --2- 172.21.15.7:0/292029383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561fc1bb880 0x5561fc1bd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.701+0000 7f88f3fff700 1 -- 172.21.15.7:0/292029383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561fc1b57a0 msgr2=0x5561fc1b5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.701+0000 7f88f3fff700 1 --2- 172.21.15.7:0/292029383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561fc1b57a0 0x5561fc1b5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.701+0000 7f88f3fff700 1 -- 172.21.15.7:0/292029383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561fc1bce30 con 0x5561fc1baf60 2024-06-10T01:06:40.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.701+0000 7f88fbfff700 1 --2- 172.21.15.7:0/292029383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561fc1b57a0 0x5561fc1b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:06:40.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.701+0000 7f88f3fff700 1 --2- 172.21.15.7:0/292029383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5561fc1baf60 0x5561fc1bc4f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f88e409b4a0 tx=0x7f88e400a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935950e84c3abf65 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:40.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.701+0000 7f88f1ffb700 1 -- 172.21.15.7:0/292029383 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e409f8c0 con 0x5561fc1baf60 2024-06-10T01:06:40.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.701+0000 7f88f1ffb700 1 -- 172.21.15.7:0/292029383 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88e409b510 con 0x5561fc1baf60 2024-06-10T01:06:40.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.701+0000 7f88f1ffb700 1 -- 172.21.15.7:0/292029383 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e409a3d0 con 0x5561fc1baf60 2024-06-10T01:06:40.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.702+0000 7f890cbc9b80 1 -- 172.21.15.7:0/292029383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5561fc1baf60 msgr2=0x5561fc1bc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:40.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.702+0000 7f890cbc9b80 1 --2- 172.21.15.7:0/292029383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5561fc1baf60 0x5561fc1bc4f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f88e409b4a0 tx=0x7f88e400a5f0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.702+0000 7f890cbc9b80 1 -- 172.21.15.7:0/292029383 shutdown_connections 2024-06-10T01:06:40.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.702+0000 7f890cbc9b80 1 --2- 172.21.15.7:0/292029383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561fc1bb880 0x5561fc1bd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.702+0000 7f890cbc9b80 1 --2- 172.21.15.7:0/292029383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561fc1b57a0 0x5561fc1b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.702+0000 7f890cbc9b80 1 --2- 172.21.15.7:0/292029383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5561fc1baf60 0x5561fc1bc4f0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.702+0000 7f890cbc9b80 1 -- 172.21.15.7:0/292029383 >> 172.21.15.7:0/292029383 conn(0x5561fc0e2080 msgr2=0x5561fc1bd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:40.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.702+0000 7f890cbc9b80 1 -- 172.21.15.7:0/292029383 shutdown_connections 2024-06-10T01:06:40.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.702+0000 7f890cbc9b80 1 -- 172.21.15.7:0/292029383 wait complete. 2024-06-10T01:06:40.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.703+0000 7f890cbc9b80 1 -- start start 2024-06-10T01:06:40.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.703+0000 7f890cbc9b80 1 Processor -- start 2024-06-10T01:06:40.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.703+0000 7f890cbc9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561fc1b57a0 0x5561fc1c7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:40.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.703+0000 7f890cbc9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5561fc1baf60 0x5561fc1c7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:40.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.704+0000 7f890cbc9b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561fc1bb880 0x5561fc1d5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:40.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.704+0000 7f890cbc9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc1aea40 con 0x5561fc1baf60 2024-06-10T01:06:40.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.704+0000 7f890cbc9b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc1afeb0 con 0x5561fc1bb880 2024-06-10T01:06:40.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.704+0000 7f890cbc9b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc1af110 con 0x5561fc1b57a0 2024-06-10T01:06:40.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.704+0000 7f890cbc9b80 10 client.0 _ll_register_callbacks cb 0x5561fc09c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:40.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.704+0000 7f88f3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5561fc1baf60 0x5561fc1c7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:40.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.704+0000 7f88f3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5561fc1baf60 0x5561fc1c7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38206/0 (socket says 172.21.15.7:38206) 2024-06-10T01:06:40.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.704+0000 7f88f3fff700 1 -- 172.21.15.7:0/2596871605 learned_addr learned my addr 172.21.15.7:0/2596871605 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:40.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.704+0000 7f88fbfff700 1 --2- 172.21.15.7:0/2596871605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561fc1b57a0 0x5561fc1c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:40.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.704+0000 7f88f3fff700 1 -- 172.21.15.7:0/2596871605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561fc1b57a0 msgr2=0x5561fc1c7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:40.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.704+0000 7f88f3fff700 1 --2- 172.21.15.7:0/2596871605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561fc1b57a0 0x5561fc1c7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.704+0000 7f88f3fff700 1 -- 172.21.15.7:0/2596871605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561fc1bb880 msgr2=0x5561fc1d5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:40.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.704+0000 7f88f3fff700 1 --2- 172.21.15.7:0/2596871605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561fc1bb880 0x5561fc1d5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.704+0000 7f88f3fff700 1 -- 172.21.15.7:0/2596871605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561fc1c8190 con 0x5561fc1baf60 2024-06-10T01:06:40.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.704+0000 7f88f3fff700 1 --2- 172.21.15.7:0/2596871605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5561fc1baf60 0x5561fc1c7a90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f88e40cc800 tx=0x7f88e409ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27b9e1779a51146 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:40.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.704+0000 7f88db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:06:40.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.705+0000 7f88db7fe700 1 -- 172.21.15.7:0/2596871605 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e409f8c0 con 0x5561fc1baf60 2024-06-10T01:06:40.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.705+0000 7f890cbc9b80 1 -- 172.21.15.7:0/2596871605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88e409b510 con 0x5561fc1baf60 2024-06-10T01:06:40.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.705+0000 7f88db7fe700 1 -- 172.21.15.7:0/2596871605 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f88e4097260 con 0x5561fc1baf60 2024-06-10T01:06:40.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.705+0000 7f88db7fe700 1 -- 172.21.15.7:0/2596871605 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f88e4097460 con 0x5561fc1baf60 2024-06-10T01:06:40.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.705+0000 7f890cbc9b80 10 client.15522 fetch_fsmap learned FSMap version 27 2024-06-10T01:06:40.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.705+0000 7f890cbc9b80 1 -- 172.21.15.7:0/2596871605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5561fc239340 con 0x5561fc1baf60 2024-06-10T01:06:40.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.705+0000 7f88db7fe700 1 -- 172.21.15.7:0/2596871605 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88e40af400 con 0x5561fc1baf60 2024-06-10T01:06:40.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.705+0000 7f88db7fe700 1 -- 172.21.15.7:0/2596871605 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f88e40a0a50 con 0x5561fc1baf60 2024-06-10T01:06:40.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.705+0000 7f890cbc9b80 10 client.15522 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:06:40.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.705+0000 7f890cbc9b80 10 client.15522 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:06:40.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.705+0000 7f890cbc9b80 20 client.15522 populate_metadata read hostname 'smithi007' 2024-06-10T01:06:40.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.705+0000 7f890cbc9b80 1 --2- 172.21.15.7:0/2596871605 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5561fc239690 0x5561fc23bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:40.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.705+0000 7f890cbc9b80 4 client.15522 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:06:40.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.705+0000 7f890cbc9b80 1 -- 172.21.15.7:0/2596871605 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f88e409b510 con 0x5561fc239690 2024-06-10T01:06:40.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.707+0000 7f88fbfff700 1 --2- 172.21.15.7:0/2596871605 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5561fc239690 0x5561fc23bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:40.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.708+0000 7f88fbfff700 1 --2- 172.21.15.7:0/2596871605 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5561fc239690 0x5561fc23bab0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=14e73f3860361e1 server_cookie=5e4c4fa44ed19a60 in_seq=0 out_seq=0 2024-06-10T01:06:40.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.708+0000 7f88db7fe700 10 client.15522 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:06:40.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.708+0000 7f88db7fe700 1 -- 172.21.15.7:0/2596871605 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f88e409b510 con 0x5561fc239690 2024-06-10T01:06:40.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.708+0000 7f88db7fe700 10 client.15522 handle_command_reply: tid=0 2024-06-10T01:06:40.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.708+0000 7f890cbc9b80 1 client.15522 shutdown 2024-06-10T01:06:40.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.708+0000 7f890cbc9b80 1 -- 172.21.15.7:0/2596871605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5561fc1baf60 msgr2=0x5561fc1c7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:40.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.708+0000 7f890cbc9b80 1 --2- 172.21.15.7:0/2596871605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5561fc1baf60 0x5561fc1c7a90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f88e40cc800 tx=0x7f88e409ada0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.708+0000 7f890cbc9b80 1 -- 172.21.15.7:0/2596871605 shutdown_connections 2024-06-10T01:06:40.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.708+0000 7f890cbc9b80 1 --2- 172.21.15.7:0/2596871605 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5561fc239690 0x5561fc23bab0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.708+0000 7f890cbc9b80 1 --2- 172.21.15.7:0/2596871605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561fc1bb880 0x5561fc1d5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.708+0000 7f890cbc9b80 1 --2- 172.21.15.7:0/2596871605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5561fc1b57a0 0x5561fc1c7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.708+0000 7f88db7fe700 0 client.15522 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:06:40.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.708+0000 7f890cbc9b80 1 --2- 172.21.15.7:0/2596871605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5561fc1baf60 0x5561fc1c7a90 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.708+0000 7f890cbc9b80 1 -- 172.21.15.7:0/2596871605 >> 172.21.15.7:0/2596871605 conn(0x5561fc0e2080 msgr2=0x5561fc1be9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:40.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.708+0000 7f890cbc9b80 1 -- 172.21.15.7:0/2596871605 shutdown_connections 2024-06-10T01:06:40.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.708+0000 7f890cbc9b80 1 -- 172.21.15.7:0/2596871605 wait complete. 2024-06-10T01:06:40.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.709+0000 7f890cbc9b80 20 client.15522 trim_cache size 0 max 16384 2024-06-10T01:06:40.707 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:06:40.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.709+0000 7f8902346700 1 -- 172.21.15.7:0/2088198301 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f88e80a2ca0 msgr2=0x7f88e80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:40.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.709+0000 7f8902346700 1 --2- 172.21.15.7:0/2088198301 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f88e80a2ca0 0x7f88e80a5160 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f88fc199ab0 tx=0x7f88ec005960 comp rx=0 tx=0).stop 2024-06-10T01:06:40.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.709+0000 7f8902346700 1 -- 172.21.15.7:0/2088198301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88fc101440 msgr2=0x7f88fc19ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:40.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.709+0000 7f8902346700 1 --2- 172.21.15.7:0/2088198301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88fc101440 0x7f88fc19ecf0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f88e400c9b0 tx=0x7f88e400ccc0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.709+0000 7f8902346700 1 -- 172.21.15.7:0/2088198301 shutdown_connections 2024-06-10T01:06:40.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.709+0000 7f8902346700 1 --2- 172.21.15.7:0/2088198301 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f88e80a2ca0 0x7f88e80a5160 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.709+0000 7f8902346700 1 --2- 172.21.15.7:0/2088198301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f88fc108510 0x7f88fc198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.709+0000 7f8902346700 1 --2- 172.21.15.7:0/2088198301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88fc100a80 0x7f88fc19e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.709+0000 7f8902346700 1 --2- 172.21.15.7:0/2088198301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88fc101440 0x7f88fc19ecf0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:40.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.709+0000 7f8902346700 1 -- 172.21.15.7:0/2088198301 >> 172.21.15.7:0/2088198301 conn(0x7f88fc077a00 msgr2=0x7f88fc0fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:40.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.710+0000 7f8902346700 1 -- 172.21.15.7:0/2088198301 shutdown_connections 2024-06-10T01:06:40.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:40.710+0000 7f8902346700 1 -- 172.21.15.7:0/2088198301 wait complete. 2024-06-10T01:06:41.522 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-06-10T01:06:41.522 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-06-10T01:06:41.522 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:06:41.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:41 smithi007 ceph-mon[32398]: pgmap v451: 129 pgs: 129 active+clean; 661 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:06:41.576 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:06:41.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:41 smithi175 ceph-mon[36636]: pgmap v451: 129 pgs: 129 active+clean; 661 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:06:41.816 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:06:41.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:41 smithi078 ceph-mon[36683]: pgmap v451: 129 pgs: 129 active+clean; 661 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:06:42.534 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:06:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.234+0000 7f51441a2640 1 -- 172.21.15.7:0/2459411949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f513c1000c0 msgr2=0x7f513c1004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.234+0000 7f51441a2640 1 --2- 172.21.15.7:0/2459411949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f513c1000c0 0x7f513c1004e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5138009900 tx=0x7f513802a3f0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.234+0000 7f51441a2640 1 -- 172.21.15.7:0/2459411949 shutdown_connections 2024-06-10T01:06:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.234+0000 7f51441a2640 1 --2- 172.21.15.7:0/2459411949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f513c101c70 0x7f513c108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.234+0000 7f51441a2640 1 --2- 172.21.15.7:0/2459411949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f513c1012b0 0x7f513c101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.234+0000 7f51441a2640 1 --2- 172.21.15.7:0/2459411949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f513c1000c0 0x7f513c1004e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.234+0000 7f51441a2640 1 -- 172.21.15.7:0/2459411949 >> 172.21.15.7:0/2459411949 conn(0x7f513c0fb610 msgr2=0x7f513c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.234+0000 7f51441a2640 1 -- 172.21.15.7:0/2459411949 shutdown_connections 2024-06-10T01:06:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.234+0000 7f51441a2640 1 -- 172.21.15.7:0/2459411949 wait complete. 2024-06-10T01:06:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.235+0000 7f51441a2640 1 Processor -- start 2024-06-10T01:06:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.235+0000 7f51441a2640 1 -- start start 2024-06-10T01:06:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.235+0000 7f51441a2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f513c1000c0 0x7f513c0fd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.235+0000 7f51441a2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f513c1012b0 0x7f513c19a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.235+0000 7f51441a2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f513c101c70 0x7f513c19aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.235+0000 7f514173d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f513c1012b0 0x7f513c19a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.235+0000 7f51441a2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f513c10bd70 con 0x7f513c1012b0 2024-06-10T01:06:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.235+0000 7f51441a2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f513c10bbf0 con 0x7f513c1000c0 2024-06-10T01:06:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.235+0000 7f514173d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f513c1012b0 0x7f513c19a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41122/0 (socket says 172.21.15.7:41122) 2024-06-10T01:06:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.235+0000 7f51441a2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f513c10bef0 con 0x7f513c101c70 2024-06-10T01:06:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.235+0000 7f514173d640 1 -- 172.21.15.7:0/3389601960 learned_addr learned my addr 172.21.15.7:0/3389601960 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.235+0000 7f5141f3e640 1 --2- 172.21.15.7:0/3389601960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f513c1000c0 0x7f513c0fd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.236+0000 7f514273f640 1 --2- 172.21.15.7:0/3389601960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f513c101c70 0x7f513c19aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.236+0000 7f514173d640 1 -- 172.21.15.7:0/3389601960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f513c101c70 msgr2=0x7f513c19aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.236+0000 7f514173d640 1 --2- 172.21.15.7:0/3389601960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f513c101c70 0x7f513c19aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.236+0000 7f514173d640 1 -- 172.21.15.7:0/3389601960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f513c1000c0 msgr2=0x7f513c0fd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.236+0000 7f514173d640 1 --2- 172.21.15.7:0/3389601960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f513c1000c0 0x7f513c0fd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.236+0000 7f514173d640 1 -- 172.21.15.7:0/3389601960 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f513c19b780 con 0x7f513c1012b0 2024-06-10T01:06:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.236+0000 7f514273f640 1 --2- 172.21.15.7:0/3389601960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f513c101c70 0x7f513c19aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:06:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.236+0000 7f5141f3e640 1 --2- 172.21.15.7:0/3389601960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f513c1000c0 0x7f513c0fd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.236+0000 7f514173d640 1 --2- 172.21.15.7:0/3389601960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f513c1012b0 0x7f513c19a860 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f512c00ea60 tx=0x7f512c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.236+0000 7f5132ffd640 1 -- 172.21.15.7:0/3389601960 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512c00cde0 con 0x7f513c1012b0 2024-06-10T01:06:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.236+0000 7f5132ffd640 1 -- 172.21.15.7:0/3389601960 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f512c0035f0 con 0x7f513c1012b0 2024-06-10T01:06:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.236+0000 7f5132ffd640 1 -- 172.21.15.7:0/3389601960 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512c010670 con 0x7f513c1012b0 2024-06-10T01:06:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.236+0000 7f51441a2640 1 -- 172.21.15.7:0/3389601960 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f513c19ba70 con 0x7f513c1012b0 2024-06-10T01:06:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.237+0000 7f51441a2640 1 -- 172.21.15.7:0/3389601960 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f513c19c010 con 0x7f513c1012b0 2024-06-10T01:06:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.240+0000 7f5132ffd640 1 -- 172.21.15.7:0/3389601960 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f512c0037a0 con 0x7f513c1012b0 2024-06-10T01:06:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.240+0000 7f5146f90740 1 Processor -- start 2024-06-10T01:06:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.240+0000 7f5146f90740 1 -- start start 2024-06-10T01:06:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.240+0000 7f5146f90740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564f8a2e0010 0x564f8a2e0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.240+0000 7f5146f90740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f8a2e0a00 0x564f8a2e5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.240+0000 7f5146f90740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f8a2e5b70 0x564f8a2e8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.240+0000 7f5146f90740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564f8a2434f0 con 0x564f8a2e5b70 2024-06-10T01:06:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.240+0000 7f5146f90740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564f8a1e1870 con 0x564f8a2e0a00 2024-06-10T01:06:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.240+0000 7f5146f90740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564f8a23f4b0 con 0x564f8a2e0010 2024-06-10T01:06:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.241+0000 7f514173d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f8a2e5b70 0x564f8a2e8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.241+0000 7f5141f3e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f8a2e0a00 0x564f8a2e5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.241+0000 7f5141f3e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f8a2e0a00 0x564f8a2e5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:47892/0 (socket says 172.21.15.7:47892) 2024-06-10T01:06:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.241+0000 7f5141f3e640 1 -- 172.21.15.7:0/573777445 learned_addr learned my addr 172.21.15.7:0/573777445 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.241+0000 7f514273f640 1 --2- 172.21.15.7:0/573777445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564f8a2e0010 0x564f8a2e0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.241+0000 7f5141f3e640 1 -- 172.21.15.7:0/573777445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564f8a2e0010 msgr2=0x564f8a2e0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.241+0000 7f5141f3e640 1 --2- 172.21.15.7:0/573777445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564f8a2e0010 0x564f8a2e0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.241+0000 7f5141f3e640 1 -- 172.21.15.7:0/573777445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f8a2e5b70 msgr2=0x564f8a2e8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.241+0000 7f5141f3e640 1 --2- 172.21.15.7:0/573777445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f8a2e5b70 0x564f8a2e8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.241+0000 7f5141f3e640 1 -- 172.21.15.7:0/573777445 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f8a1fd9b0 con 0x564f8a2e0a00 2024-06-10T01:06:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.241+0000 7f5132ffd640 1 --2- 172.21.15.7:0/3389601960 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f51280a2c00 0x7f51280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.241+0000 7f514173d640 1 --2- 172.21.15.7:0/573777445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f8a2e5b70 0x564f8a2e8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:06:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.242+0000 7f514273f640 1 --2- 172.21.15.7:0/3389601960 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f51280a2c00 0x7f51280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.242+0000 7f5132ffd640 1 -- 172.21.15.7:0/3389601960 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f512c0cf730 con 0x7f513c1012b0 2024-06-10T01:06:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.242+0000 7f5141f3e640 1 --2- 172.21.15.7:0/573777445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f8a2e0a00 0x564f8a2e5500 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f513c101110 tx=0x7f51380316b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=985e8fcbeeed5cce server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.242+0000 7f5127fff640 1 -- 172.21.15.7:0/573777445 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5138040070 con 0x564f8a2e0a00 2024-06-10T01:06:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.242+0000 7f5127fff640 1 -- 172.21.15.7:0/573777445 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5138005040 con 0x564f8a2e0a00 2024-06-10T01:06:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.242+0000 7f5127fff640 1 -- 172.21.15.7:0/573777445 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5138035040 con 0x564f8a2e0a00 2024-06-10T01:06:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.243+0000 7f5146f90740 1 -- 172.21.15.7:0/573777445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f8a2e0a00 msgr2=0x564f8a2e5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.243+0000 7f5146f90740 1 --2- 172.21.15.7:0/573777445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f8a2e0a00 0x564f8a2e5500 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f513c101110 tx=0x7f51380316b0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.243+0000 7f514273f640 1 --2- 172.21.15.7:0/3389601960 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f51280a2c00 0x7f51280a50c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x564f8a2e1260 tx=0x7f5134009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.243+0000 7f5146f90740 1 -- 172.21.15.7:0/573777445 shutdown_connections 2024-06-10T01:06:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.243+0000 7f5146f90740 1 --2- 172.21.15.7:0/573777445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f8a2e5b70 0x564f8a2e8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.243+0000 7f5146f90740 1 --2- 172.21.15.7:0/573777445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f8a2e0a00 0x564f8a2e5500 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.243+0000 7f5146f90740 1 --2- 172.21.15.7:0/573777445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564f8a2e0010 0x564f8a2e0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.243+0000 7f5146f90740 1 -- 172.21.15.7:0/573777445 >> 172.21.15.7:0/573777445 conn(0x564f8a2db680 msgr2=0x564f8a2dba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.243+0000 7f5146f90740 1 -- 172.21.15.7:0/573777445 shutdown_connections 2024-06-10T01:06:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.243+0000 7f5146f90740 1 -- 172.21.15.7:0/573777445 wait complete. 2024-06-10T01:06:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.243+0000 7f5146f90740 1 -- start start 2024-06-10T01:06:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.243+0000 7f5146f90740 1 Processor -- start 2024-06-10T01:06:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.244+0000 7f5146f90740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f8a2e0010 0x564f8a2ea2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.244+0000 7f5146f90740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f8a2e0a00 0x564f8a2ea820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.244+0000 7f5146f90740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564f8a2e5b70 0x564f8a2f1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.244+0000 7f5146f90740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564f8a2e8ac0 con 0x564f8a2e0010 2024-06-10T01:06:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.244+0000 7f5146f90740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564f8a1ccae0 con 0x564f8a2e0a00 2024-06-10T01:06:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.244+0000 7f5146f90740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564f8a2e8c40 con 0x564f8a2e5b70 2024-06-10T01:06:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.244+0000 7f5146f90740 10 client.0 _ll_register_callbacks cb 0x564f8a2059b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.244+0000 7f5141f3e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f8a2e0010 0x564f8a2ea2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.244+0000 7f5141f3e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f8a2e0010 0x564f8a2ea2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41140/0 (socket says 172.21.15.7:41140) 2024-06-10T01:06:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.244+0000 7f5141f3e640 1 -- 172.21.15.7:0/3432165667 learned_addr learned my addr 172.21.15.7:0/3432165667 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.244+0000 7f514173d640 1 --2- 172.21.15.7:0/3432165667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f8a2e0a00 0x564f8a2ea820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.244+0000 7f514273f640 1 --2- 172.21.15.7:0/3432165667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564f8a2e5b70 0x564f8a2f1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.244+0000 7f5141f3e640 1 -- 172.21.15.7:0/3432165667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564f8a2e5b70 msgr2=0x564f8a2f1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.244+0000 7f5141f3e640 1 --2- 172.21.15.7:0/3432165667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564f8a2e5b70 0x564f8a2f1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.244+0000 7f5141f3e640 1 -- 172.21.15.7:0/3432165667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f8a2e0a00 msgr2=0x564f8a2ea820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.244+0000 7f5141f3e640 1 --2- 172.21.15.7:0/3432165667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f8a2e0a00 0x564f8a2ea820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.244+0000 7f5141f3e640 1 -- 172.21.15.7:0/3432165667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f89f57210 con 0x564f8a2e0010 2024-06-10T01:06:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.244+0000 7f514273f640 1 --2- 172.21.15.7:0/3432165667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564f8a2e5b70 0x564f8a2f1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:06:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.244+0000 7f514173d640 1 --2- 172.21.15.7:0/3432165667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f8a2e0a00 0x564f8a2ea820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:06:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.244+0000 7f5141f3e640 1 --2- 172.21.15.7:0/3432165667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f8a2e0010 0x564f8a2ea2e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f513802a910 tx=0x7f5138009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c290e569634a636a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.244+0000 7f51257fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:06:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.245+0000 7f51257fa640 1 -- 172.21.15.7:0/3432165667 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513804c020 con 0x564f8a2e0010 2024-06-10T01:06:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.245+0000 7f51257fa640 1 -- 172.21.15.7:0/3432165667 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f5138004900 con 0x564f8a2e0010 2024-06-10T01:06:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.245+0000 7f5146f90740 1 -- 172.21.15.7:0/3432165667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5138005040 con 0x564f8a2e0010 2024-06-10T01:06:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.245+0000 7f51257fa640 1 -- 172.21.15.7:0/3432165667 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5138057050 con 0x564f8a2e0010 2024-06-10T01:06:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.245+0000 7f5146f90740 10 client.15534 fetch_fsmap learned FSMap version 27 2024-06-10T01:06:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.245+0000 7f5146f90740 1 -- 172.21.15.7:0/3432165667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564f8a023620 con 0x564f8a2e0010 2024-06-10T01:06:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.245+0000 7f51257fa640 1 -- 172.21.15.7:0/3432165667 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5138005040 con 0x564f8a2e0010 2024-06-10T01:06:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.246+0000 7f51257fa640 1 -- 172.21.15.7:0/3432165667 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5138044400 con 0x564f8a2e0010 2024-06-10T01:06:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.246+0000 7f5146f90740 10 client.15534 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:06:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.246+0000 7f5146f90740 10 client.15534 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:06:43.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.246+0000 7f5146f90740 20 client.15534 populate_metadata read hostname 'smithi007' 2024-06-10T01:06:43.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.246+0000 7f5146f90740 1 --2- 172.21.15.7:0/3432165667 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564f8a2f5d50 0x564f8a2f8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:43.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.246+0000 7f5146f90740 4 client.15534 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:06:43.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.246+0000 7f5146f90740 1 -- 172.21.15.7:0/3432165667 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f8a256f10 con 0x564f8a2f5d50 2024-06-10T01:06:43.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.246+0000 7f514173d640 1 --2- 172.21.15.7:0/3432165667 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564f8a2f5d50 0x564f8a2f8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:43.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.247+0000 7f514173d640 1 --2- 172.21.15.7:0/3432165667 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564f8a2f5d50 0x564f8a2f8140 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49c745134bf402c4 server_cookie=255485f88afd8cec in_seq=0 out_seq=0 2024-06-10T01:06:43.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.247+0000 7f51257fa640 10 client.15534 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:43.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.247+0000 7f51257fa640 1 -- 172.21.15.7:0/3432165667 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564f8a256f10 con 0x564f8a2f5d50 2024-06-10T01:06:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.247+0000 7f51257fa640 10 client.15534 handle_command_reply: tid=0 2024-06-10T01:06:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.247+0000 7f5146f90740 1 client.15534 shutdown 2024-06-10T01:06:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.248+0000 7f5146f90740 1 -- 172.21.15.7:0/3432165667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f8a2e0010 msgr2=0x564f8a2ea2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.248+0000 7f5146f90740 1 --2- 172.21.15.7:0/3432165667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f8a2e0010 0x564f8a2ea2e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f513802a910 tx=0x7f5138009660 comp rx=0 tx=0).stop 2024-06-10T01:06:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.248+0000 7f5146f90740 1 -- 172.21.15.7:0/3432165667 shutdown_connections 2024-06-10T01:06:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.248+0000 7f5146f90740 1 --2- 172.21.15.7:0/3432165667 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564f8a2f5d50 0x564f8a2f8140 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.248+0000 7f5146f90740 1 --2- 172.21.15.7:0/3432165667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564f8a2e5b70 0x564f8a2f1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.248+0000 7f5146f90740 1 --2- 172.21.15.7:0/3432165667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f8a2e0a00 0x564f8a2ea820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.248+0000 7f51257fa640 0 client.15534 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.248+0000 7f5146f90740 1 --2- 172.21.15.7:0/3432165667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f8a2e0010 0x564f8a2ea2e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.248+0000 7f5146f90740 1 -- 172.21.15.7:0/3432165667 >> 172.21.15.7:0/3432165667 conn(0x564f8a2db680 msgr2=0x564f8a2da7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.248+0000 7f5146f90740 1 -- 172.21.15.7:0/3432165667 shutdown_connections 2024-06-10T01:06:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.248+0000 7f5146f90740 1 -- 172.21.15.7:0/3432165667 wait complete. 2024-06-10T01:06:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.248+0000 7f5146f90740 20 client.15534 trim_cache size 0 max 16384 2024-06-10T01:06:43.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.275+0000 7f5146f90740 1 Processor -- start 2024-06-10T01:06:43.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.275+0000 7f5146f90740 1 -- start start 2024-06-10T01:06:43.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.275+0000 7f5146f90740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564f8a2db680 0x564f8a2db120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.275+0000 7f5146f90740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f8a2e0010 0x564f8a2f1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.276+0000 7f5146f90740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f8a2e0a00 0x564f8a2e95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.276+0000 7f5146f90740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f513c1b3470 con 0x564f8a2e0a00 2024-06-10T01:06:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.276+0000 7f5146f90740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f513c1b0d40 con 0x564f8a2e0010 2024-06-10T01:06:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.276+0000 7f5146f90740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f513c1aeb90 con 0x564f8a2db680 2024-06-10T01:06:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.276+0000 7f514273f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f8a2e0a00 0x564f8a2e95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.276+0000 7f514273f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f8a2e0a00 0x564f8a2e95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41146/0 (socket says 172.21.15.7:41146) 2024-06-10T01:06:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.276+0000 7f514273f640 1 -- 172.21.15.7:0/753828757 learned_addr learned my addr 172.21.15.7:0/753828757 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.276+0000 7f5141f3e640 1 --2- 172.21.15.7:0/753828757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564f8a2db680 0x564f8a2db120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.276+0000 7f514173d640 1 --2- 172.21.15.7:0/753828757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f8a2e0010 0x564f8a2f1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.276+0000 7f514273f640 1 -- 172.21.15.7:0/753828757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564f8a2db680 msgr2=0x564f8a2db120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.276+0000 7f514273f640 1 --2- 172.21.15.7:0/753828757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564f8a2db680 0x564f8a2db120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.276+0000 7f514273f640 1 -- 172.21.15.7:0/753828757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f8a2e0010 msgr2=0x564f8a2f1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.276+0000 7f514273f640 1 --2- 172.21.15.7:0/753828757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f8a2e0010 0x564f8a2f1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.276+0000 7f514273f640 1 -- 172.21.15.7:0/753828757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f8a2e83f0 con 0x564f8a2e0a00 2024-06-10T01:06:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.276+0000 7f514273f640 1 --2- 172.21.15.7:0/753828757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f8a2e0a00 0x564f8a2e95b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5134012670 tx=0x7f5134013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31efa819124c18bf server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.277+0000 7f51277fe640 1 -- 172.21.15.7:0/753828757 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5134011720 con 0x564f8a2e0a00 2024-06-10T01:06:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.277+0000 7f51277fe640 1 -- 172.21.15.7:0/753828757 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5134011da0 con 0x564f8a2e0a00 2024-06-10T01:06:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.277+0000 7f51277fe640 1 -- 172.21.15.7:0/753828757 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513401e3b0 con 0x564f8a2e0a00 2024-06-10T01:06:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.278+0000 7f5146f90740 1 -- 172.21.15.7:0/753828757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f8a2e0a00 msgr2=0x564f8a2e95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.278+0000 7f5146f90740 1 --2- 172.21.15.7:0/753828757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f8a2e0a00 0x564f8a2e95b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5134012670 tx=0x7f5134013000 comp rx=0 tx=0).stop 2024-06-10T01:06:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.278+0000 7f5146f90740 1 -- 172.21.15.7:0/753828757 shutdown_connections 2024-06-10T01:06:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.278+0000 7f5146f90740 1 --2- 172.21.15.7:0/753828757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f8a2e0a00 0x564f8a2e95b0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.278+0000 7f5146f90740 1 --2- 172.21.15.7:0/753828757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f8a2e0010 0x564f8a2f1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.278+0000 7f5146f90740 1 --2- 172.21.15.7:0/753828757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564f8a2db680 0x564f8a2db120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.278+0000 7f5146f90740 1 -- 172.21.15.7:0/753828757 >> 172.21.15.7:0/753828757 conn(0x564f8a2e56b0 msgr2=0x564f8a2dedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.278+0000 7f5146f90740 1 -- 172.21.15.7:0/753828757 shutdown_connections 2024-06-10T01:06:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.278+0000 7f5146f90740 1 -- 172.21.15.7:0/753828757 wait complete. 2024-06-10T01:06:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.278+0000 7f5146f90740 1 -- start start 2024-06-10T01:06:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.278+0000 7f5146f90740 1 Processor -- start 2024-06-10T01:06:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.279+0000 7f5146f90740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f8a2db680 0x564f8a2eed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.279+0000 7f5141f3e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f8a2db680 0x564f8a2eed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.279+0000 7f5141f3e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f8a2db680 0x564f8a2eed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41150/0 (socket says 172.21.15.7:41150) 2024-06-10T01:06:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.279+0000 7f5146f90740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f8a2e0010 0x564f8a2ef270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.279+0000 7f5141f3e640 1 -- 172.21.15.7:0/551502134 learned_addr learned my addr 172.21.15.7:0/551502134 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.279+0000 7f5146f90740 1 --2- 172.21.15.7:0/551502134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564f8a2e0a00 0x564f8a2f0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.279+0000 7f5146f90740 1 -- 172.21.15.7:0/551502134 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f513c1af200 con 0x564f8a2db680 2024-06-10T01:06:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.279+0000 7f5146f90740 1 -- 172.21.15.7:0/551502134 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f513c1b0670 con 0x564f8a2e0010 2024-06-10T01:06:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.279+0000 7f5146f90740 1 -- 172.21.15.7:0/551502134 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f513c1af8d0 con 0x564f8a2e0a00 2024-06-10T01:06:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.279+0000 7f514173d640 1 --2- 172.21.15.7:0/551502134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f8a2e0010 0x564f8a2ef270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.279+0000 7f514273f640 1 --2- 172.21.15.7:0/551502134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564f8a2e0a00 0x564f8a2f0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.279+0000 7f5141f3e640 1 -- 172.21.15.7:0/551502134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564f8a2e0a00 msgr2=0x564f8a2f0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.279+0000 7f5141f3e640 1 --2- 172.21.15.7:0/551502134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564f8a2e0a00 0x564f8a2f0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.279+0000 7f5141f3e640 1 -- 172.21.15.7:0/551502134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f8a2e0010 msgr2=0x564f8a2ef270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.279+0000 7f5141f3e640 1 --2- 172.21.15.7:0/551502134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f8a2e0010 0x564f8a2ef270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.280+0000 7f5141f3e640 1 -- 172.21.15.7:0/551502134 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f8a2f0870 con 0x564f8a2db680 2024-06-10T01:06:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.280+0000 7f514173d640 1 --2- 172.21.15.7:0/551502134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f8a2e0010 0x564f8a2ef270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:06:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.280+0000 7f5146f90740 10 client.0 _ll_register_callbacks cb 0x7f5138005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.280+0000 7f5141f3e640 1 --2- 172.21.15.7:0/551502134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f8a2db680 0x564f8a2eed30 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5138031610 tx=0x7f5138031800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a03ed9207c83be server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.280+0000 7f51257fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:06:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.280+0000 7f5146f90740 1 -- 172.21.15.7:0/551502134 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5134011da0 con 0x564f8a2db680 2024-06-10T01:06:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.280+0000 7f51257fa640 1 -- 172.21.15.7:0/551502134 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5138060060 con 0x564f8a2db680 2024-06-10T01:06:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.280+0000 7f51257fa640 1 -- 172.21.15.7:0/551502134 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f5138031ab0 con 0x564f8a2db680 2024-06-10T01:06:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.280+0000 7f51257fa640 1 -- 172.21.15.7:0/551502134 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5138067050 con 0x564f8a2db680 2024-06-10T01:06:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.280+0000 7f5146f90740 10 client.15540 fetch_fsmap learned FSMap version 27 2024-06-10T01:06:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.280+0000 7f5146f90740 1 -- 172.21.15.7:0/551502134 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=27}) -- 0x564f8a2f1a20 con 0x564f8a2db680 2024-06-10T01:06:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.281+0000 7f51257fa640 1 -- 172.21.15.7:0/551502134 <== mon.0 v2:172.21.15.7:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5138031cd0 con 0x564f8a2db680 2024-06-10T01:06:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.281+0000 7f5146f90740 10 client.15540 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:06:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.281+0000 7f5146f90740 10 client.15540 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:06:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.281+0000 7f5146f90740 20 client.15540 populate_metadata read hostname 'smithi007' 2024-06-10T01:06:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.281+0000 7f5146f90740 1 --2- 172.21.15.7:0/551502134 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564f8a2e5b70 0x564f8a362d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.281+0000 7f5146f90740 4 client.15540 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:06:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.281+0000 7f5146f90740 1 -- 172.21.15.7:0/551502134 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564f8a250c20 con 0x564f8a2e5b70 2024-06-10T01:06:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.281+0000 7f514173d640 1 --2- 172.21.15.7:0/551502134 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564f8a2e5b70 0x564f8a362d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.281+0000 7f514173d640 1 --2- 172.21.15.7:0/551502134 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564f8a2e5b70 0x564f8a362d60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81032e806bb940e0 server_cookie=e038a8324e75ec95 in_seq=0 out_seq=0 2024-06-10T01:06:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.281+0000 7f51257fa640 10 client.15540 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.285+0000 7f51257fa640 1 -- 172.21.15.7:0/551502134 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564f8a250c20 con 0x564f8a2e5b70 2024-06-10T01:06:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.285+0000 7f51257fa640 10 client.15540 handle_command_reply: tid=0 2024-06-10T01:06:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.285+0000 7f5146f90740 1 client.15540 shutdown 2024-06-10T01:06:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.285+0000 7f5146f90740 1 -- 172.21.15.7:0/551502134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f8a2db680 msgr2=0x564f8a2eed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.285+0000 7f5146f90740 1 --2- 172.21.15.7:0/551502134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f8a2db680 0x564f8a2eed30 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5138031610 tx=0x7f5138031800 comp rx=0 tx=0).stop 2024-06-10T01:06:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.285+0000 7f5146f90740 1 -- 172.21.15.7:0/551502134 shutdown_connections 2024-06-10T01:06:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.285+0000 7f5146f90740 1 --2- 172.21.15.7:0/551502134 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564f8a2e5b70 0x564f8a362d60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.285+0000 7f5146f90740 1 --2- 172.21.15.7:0/551502134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564f8a2e0a00 0x564f8a2f0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.285+0000 7f5146f90740 1 --2- 172.21.15.7:0/551502134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f8a2e0010 0x564f8a2ef270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.285+0000 7f51257fa640 0 client.15540 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.285+0000 7f5146f90740 1 --2- 172.21.15.7:0/551502134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f8a2db680 0x564f8a2eed30 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.285+0000 7f5146f90740 1 -- 172.21.15.7:0/551502134 >> 172.21.15.7:0/551502134 conn(0x564f8a2e56b0 msgr2=0x564f8a2d9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.285+0000 7f5146f90740 1 -- 172.21.15.7:0/551502134 shutdown_connections 2024-06-10T01:06:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.285+0000 7f5146f90740 1 -- 172.21.15.7:0/551502134 wait complete. 2024-06-10T01:06:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.286+0000 7f5146f90740 20 client.15540 trim_cache size 0 max 16384 2024-06-10T01:06:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:06:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:06:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:06:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.286+0000 7f51441a2640 1 -- 172.21.15.7:0/3389601960 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f51280a2c00 msgr2=0x7f51280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.286+0000 7f51441a2640 1 --2- 172.21.15.7:0/3389601960 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f51280a2c00 0x7f51280a50c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x564f8a2e1260 tx=0x7f5134009920 comp rx=0 tx=0).stop 2024-06-10T01:06:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.286+0000 7f51441a2640 1 -- 172.21.15.7:0/3389601960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f513c1012b0 msgr2=0x7f513c19a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.286+0000 7f51441a2640 1 --2- 172.21.15.7:0/3389601960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f513c1012b0 0x7f513c19a860 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f512c00ea60 tx=0x7f512c00ef20 comp rx=0 tx=0).stop 2024-06-10T01:06:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.286+0000 7f51441a2640 1 -- 172.21.15.7:0/3389601960 shutdown_connections 2024-06-10T01:06:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.286+0000 7f51441a2640 1 --2- 172.21.15.7:0/3389601960 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f51280a2c00 0x7f51280a50c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.286+0000 7f51441a2640 1 --2- 172.21.15.7:0/3389601960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f513c101c70 0x7f513c19aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.286+0000 7f51441a2640 1 --2- 172.21.15.7:0/3389601960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f513c1012b0 0x7f513c19a860 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.286+0000 7f51441a2640 1 --2- 172.21.15.7:0/3389601960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f513c1000c0 0x7f513c0fd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.286+0000 7f51441a2640 1 -- 172.21.15.7:0/3389601960 >> 172.21.15.7:0/3389601960 conn(0x7f513c0fb610 msgr2=0x7f513c104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.286+0000 7f51441a2640 1 -- 172.21.15.7:0/3389601960 shutdown_connections 2024-06-10T01:06:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:43.286+0000 7f51441a2640 1 -- 172.21.15.7:0/3389601960 wait complete. 2024-06-10T01:06:43.425 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:43 smithi007 ceph-mon[32398]: pgmap v452: 129 pgs: 129 active+clean; 661 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:06:43.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:43 smithi175 ceph-mon[36636]: pgmap v452: 129 pgs: 129 active+clean; 661 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:06:43.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:43 smithi078 ceph-mon[36683]: pgmap v452: 129 pgs: 129 active+clean; 661 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:06:44.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.375+0000 7f4a63bfc700 1 -- 172.21.15.7:0/3954676752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a5c108500 msgr2=0x7f4a5c1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:44.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.375+0000 7f4a63bfc700 1 --2- 172.21.15.7:0/3954676752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a5c108500 0x7f4a5c1088e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009990 tx=0x7f4a58009ca0 comp rx=0 tx=0).stop 2024-06-10T01:06:44.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.376+0000 7f4a63bfc700 1 -- 172.21.15.7:0/3954676752 shutdown_connections 2024-06-10T01:06:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.376+0000 7f4a63bfc700 1 --2- 172.21.15.7:0/3954676752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a5c06ab30 0x7f4a5c10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.376+0000 7f4a63bfc700 1 --2- 172.21.15.7:0/3954676752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a5c108500 0x7f4a5c1088e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.376+0000 7f4a63bfc700 1 --2- 172.21.15.7:0/3954676752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a5c06a170 0x7f4a5c06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.376+0000 7f4a63bfc700 1 -- 172.21.15.7:0/3954676752 >> 172.21.15.7:0/3954676752 conn(0x7f4a5c078340 msgr2=0x7f4a5c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.376+0000 7f4a63bfc700 1 -- 172.21.15.7:0/3954676752 shutdown_connections 2024-06-10T01:06:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.376+0000 7f4a63bfc700 1 -- 172.21.15.7:0/3954676752 wait complete. 2024-06-10T01:06:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.377+0000 7f4a63bfc700 1 Processor -- start 2024-06-10T01:06:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.377+0000 7f4a63bfc700 1 -- start start 2024-06-10T01:06:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.377+0000 7f4a63bfc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a5c06a170 0x7f4a5c19aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.377+0000 7f4a63bfc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a5c06ab30 0x7f4a5c19b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.377+0000 7f4a63bfc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a5c108500 0x7f4a5c19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.377+0000 7f4a63bfc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c112760 con 0x7f4a5c06ab30 2024-06-10T01:06:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.377+0000 7f4a63bfc700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c1125e0 con 0x7f4a5c108500 2024-06-10T01:06:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.377+0000 7f4a63bfc700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c1128e0 con 0x7f4a5c06a170 2024-06-10T01:06:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.377+0000 7f4a61197700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a5c06ab30 0x7f4a5c19b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:44.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.377+0000 7f4a61197700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a5c06ab30 0x7f4a5c19b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41182/0 (socket says 172.21.15.7:41182) 2024-06-10T01:06:44.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.377+0000 7f4a61197700 1 -- 172.21.15.7:0/3055586946 learned_addr learned my addr 172.21.15.7:0/3055586946 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:44.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.377+0000 7f4a62199700 1 --2- 172.21.15.7:0/3055586946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a5c108500 0x7f4a5c19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:44.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.377+0000 7f4a61998700 1 --2- 172.21.15.7:0/3055586946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a5c06a170 0x7f4a5c19aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:44.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.377+0000 7f4a61197700 1 -- 172.21.15.7:0/3055586946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a5c06a170 msgr2=0x7f4a5c19aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:44.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.377+0000 7f4a61197700 1 --2- 172.21.15.7:0/3055586946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a5c06a170 0x7f4a5c19aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:44.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.377+0000 7f4a61197700 1 -- 172.21.15.7:0/3055586946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a5c108500 msgr2=0x7f4a5c19f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:44.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.377+0000 7f4a61197700 1 --2- 172.21.15.7:0/3055586946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a5c108500 0x7f4a5c19f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:44.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.377+0000 7f4a61197700 1 -- 172.21.15.7:0/3055586946 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5c19fee0 con 0x7f4a5c06ab30 2024-06-10T01:06:44.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.378+0000 7f4a61197700 1 --2- 172.21.15.7:0/3055586946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a5c06ab30 0x7f4a5c19b410 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c00e9a0 tx=0x7f4a4c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:44.854 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:06:44.854 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:05:39.175358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/2784910673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2784910673},{"type":"v1","addr":"172.21.15.175:6835","nonce":2784910673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24434,24436,24451]},"id":1}]} 2024-06-10T01:06:44.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.378+0000 7f4a52ffd700 1 -- 172.21.15.7:0/3055586946 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c00cbd0 con 0x7f4a5c06ab30 2024-06-10T01:06:44.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.378+0000 7f4a63bfc700 1 -- 172.21.15.7:0/3055586946 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a5c1a01d0 con 0x7f4a5c06ab30 2024-06-10T01:06:44.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.378+0000 7f4a52ffd700 1 -- 172.21.15.7:0/3055586946 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a4c00cd70 con 0x7f4a5c06ab30 2024-06-10T01:06:44.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.378+0000 7f4a52ffd700 1 -- 172.21.15.7:0/3055586946 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c018750 con 0x7f4a5c06ab30 2024-06-10T01:06:44.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.378+0000 7f4a63bfc700 1 -- 172.21.15.7:0/3055586946 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a5c1a7a50 con 0x7f4a5c06ab30 2024-06-10T01:06:44.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.379+0000 7f4a63bfc700 1 -- 172.21.15.7:0/3055586946 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a5c069180 con 0x7f4a5c06ab30 2024-06-10T01:06:44.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.382+0000 7f4a52ffd700 1 -- 172.21.15.7:0/3055586946 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f4a4c010430 con 0x7f4a5c06ab30 2024-06-10T01:06:44.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.382+0000 7f4a52ffd700 1 --2- 172.21.15.7:0/3055586946 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4a480a2ca0 0x7f4a480a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:44.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.383+0000 7f4a52ffd700 1 -- 172.21.15.7:0/3055586946 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f4a4c0cfb80 con 0x7f4a5c06ab30 2024-06-10T01:06:44.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.383+0000 7f4a61998700 1 --2- 172.21.15.7:0/3055586946 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4a480a2ca0 0x7f4a480a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:44.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.383+0000 7f4a61998700 1 --2- 172.21.15.7:0/3055586946 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4a480a2ca0 0x7f4a480a5160 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4a58006010 tx=0x7f4a5801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:44.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.386+0000 7f4a52ffd700 1 -- 172.21.15.7:0/3055586946 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4a4c09c700 con 0x7f4a5c06ab30 2024-06-10T01:06:44.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.779+0000 7f4a63bfc700 1 -- 172.21.15.7:0/3055586946 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a5c19c230 con 0x7f4a5c06ab30 2024-06-10T01:06:44.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.780+0000 7f4a52ffd700 1 -- 172.21.15.7:0/3055586946 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11855 (secure 0 0 0) 0x7f4a5c19c230 con 0x7f4a5c06ab30 2024-06-10T01:06:44.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.783+0000 7f4a63bfc700 1 -- 172.21.15.7:0/3055586946 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4a480a2ca0 msgr2=0x7f4a480a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:44.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.783+0000 7f4a63bfc700 1 --2- 172.21.15.7:0/3055586946 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4a480a2ca0 0x7f4a480a5160 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4a58006010 tx=0x7f4a5801a040 comp rx=0 tx=0).stop 2024-06-10T01:06:44.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.783+0000 7f4a63bfc700 1 -- 172.21.15.7:0/3055586946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a5c06ab30 msgr2=0x7f4a5c19b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:44.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.783+0000 7f4a63bfc700 1 --2- 172.21.15.7:0/3055586946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a5c06ab30 0x7f4a5c19b410 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c00e9a0 tx=0x7f4a4c00ecb0 comp rx=0 tx=0).stop 2024-06-10T01:06:44.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.784+0000 7f4a63bfc700 1 -- 172.21.15.7:0/3055586946 shutdown_connections 2024-06-10T01:06:44.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.784+0000 7f4a63bfc700 1 --2- 172.21.15.7:0/3055586946 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4a480a2ca0 0x7f4a480a5160 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:44.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.784+0000 7f4a63bfc700 1 --2- 172.21.15.7:0/3055586946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a5c108500 0x7f4a5c19f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:44.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.784+0000 7f4a63bfc700 1 --2- 172.21.15.7:0/3055586946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a5c06a170 0x7f4a5c19aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:44.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.784+0000 7f4a63bfc700 1 --2- 172.21.15.7:0/3055586946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a5c06ab30 0x7f4a5c19b410 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:44.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.784+0000 7f4a63bfc700 1 -- 172.21.15.7:0/3055586946 >> 172.21.15.7:0/3055586946 conn(0x7f4a5c078340 msgr2=0x7f4a5c10d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:44.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.784+0000 7f4a63bfc700 1 -- 172.21.15.7:0/3055586946 shutdown_connections 2024-06-10T01:06:44.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:44.784+0000 7f4a63bfc700 1 -- 172.21.15.7:0/3055586946 wait complete. 2024-06-10T01:06:44.856 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-06-10T01:06:45.556 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:45 smithi007 ceph-mon[32398]: pgmap v453: 129 pgs: 129 active+clean; 661 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-06-10T01:06:45.556 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:45 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3055586946' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:06:45.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:45 smithi175 ceph-mon[36636]: pgmap v453: 129 pgs: 129 active+clean; 661 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-06-10T01:06:46.526 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:45 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3055586946' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:06:46.527 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:45 smithi078 ceph-mon[36683]: pgmap v453: 129 pgs: 129 active+clean; 661 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-06-10T01:06:46.527 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:45 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3055586946' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:06:46.528 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-06-10T01:06:46.529 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-06-10T01:06:46.529 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:06:46.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.894+0000 7fe8bd64a700 1 -- 172.21.15.7:0/3878193364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b8108510 msgr2=0x7fe8b81088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:46.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.894+0000 7fe8bd64a700 1 --2- 172.21.15.7:0/3878193364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b8108510 0x7fe8b81088f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8009990 tx=0x7fe8a8009ca0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.894+0000 7fe8bd64a700 1 -- 172.21.15.7:0/3878193364 shutdown_connections 2024-06-10T01:06:46.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.894+0000 7fe8bd64a700 1 --2- 172.21.15.7:0/3878193364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8b8101440 0x7fe8b810f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.894+0000 7fe8bd64a700 1 --2- 172.21.15.7:0/3878193364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b8108510 0x7fe8b81088f0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.894+0000 7fe8bd64a700 1 --2- 172.21.15.7:0/3878193364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8b8100a80 0x7fe8b8100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.894+0000 7fe8bd64a700 1 -- 172.21.15.7:0/3878193364 >> 172.21.15.7:0/3878193364 conn(0x7fe8b8077a00 msgr2=0x7fe8b80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:46.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.894+0000 7fe8bd64a700 1 -- 172.21.15.7:0/3878193364 shutdown_connections 2024-06-10T01:06:46.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.894+0000 7fe8bd64a700 1 -- 172.21.15.7:0/3878193364 wait complete. 2024-06-10T01:06:46.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.895+0000 7fe8bd64a700 1 Processor -- start 2024-06-10T01:06:46.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.895+0000 7fe8bd64a700 1 -- start start 2024-06-10T01:06:46.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.896+0000 7fe8bd64a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b8100a80 0x7fe8b8101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:46.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.896+0000 7fe8bd64a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8b8101440 0x7fe8b81021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:46.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.896+0000 7fe8bd64a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8b8108510 0x7fe8b81026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:46.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.896+0000 7fe8bd64a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b8112790 con 0x7fe8b8100a80 2024-06-10T01:06:46.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.896+0000 7fe8bd64a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b8112610 con 0x7fe8b8101440 2024-06-10T01:06:46.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.896+0000 7fe8bd64a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b8112910 con 0x7fe8b8108510 2024-06-10T01:06:46.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.896+0000 7fe8b77fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8b8108510 0x7fe8b81026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:46.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.896+0000 7fe8b77fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8b8108510 0x7fe8b81026e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52730/0 (socket says 172.21.15.7:52730) 2024-06-10T01:06:46.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.896+0000 7fe8b77fe700 1 -- 172.21.15.7:0/2732878910 learned_addr learned my addr 172.21.15.7:0/2732878910 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:46.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.896+0000 7fe8b6ffd700 1 --2- 172.21.15.7:0/2732878910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b8100a80 0x7fe8b8101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:46.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.896+0000 7fe8b67fc700 1 --2- 172.21.15.7:0/2732878910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8b8101440 0x7fe8b81021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:46.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.896+0000 7fe8b6ffd700 1 -- 172.21.15.7:0/2732878910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8b8108510 msgr2=0x7fe8b81026e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:46.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.896+0000 7fe8b6ffd700 1 --2- 172.21.15.7:0/2732878910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8b8108510 0x7fe8b81026e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.897+0000 7fe8b6ffd700 1 -- 172.21.15.7:0/2732878910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8b8101440 msgr2=0x7fe8b81021a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:46.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.897+0000 7fe8b6ffd700 1 --2- 172.21.15.7:0/2732878910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8b8101440 0x7fe8b81021a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.897+0000 7fe8b6ffd700 1 -- 172.21.15.7:0/2732878910 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b81a70d0 con 0x7fe8b8100a80 2024-06-10T01:06:46.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.897+0000 7fe8b77fe700 1 --2- 172.21.15.7:0/2732878910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8b8108510 0x7fe8b81026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:46.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.897+0000 7fe8b6ffd700 1 --2- 172.21.15.7:0/2732878910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b8100a80 0x7fe8b8101c60 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80059d0 tx=0x7fe8a80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:46.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.897+0000 7fe89ffff700 1 -- 172.21.15.7:0/2732878910 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8027070 con 0x7fe8b8100a80 2024-06-10T01:06:46.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.897+0000 7fe8bd64a700 1 -- 172.21.15.7:0/2732878910 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8b81a7360 con 0x7fe8b8100a80 2024-06-10T01:06:46.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.897+0000 7fe89ffff700 1 -- 172.21.15.7:0/2732878910 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8a8004050 con 0x7fe8b8100a80 2024-06-10T01:06:46.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.897+0000 7fe89ffff700 1 -- 172.21.15.7:0/2732878910 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8022de0 con 0x7fe8b8100a80 2024-06-10T01:06:46.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.897+0000 7fe8bd64a700 1 -- 172.21.15.7:0/2732878910 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8b81a7810 con 0x7fe8b8100a80 2024-06-10T01:06:46.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.901+0000 7fe89ffff700 1 -- 172.21.15.7:0/2732878910 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fe8a802a070 con 0x7fe8b8100a80 2024-06-10T01:06:46.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.901+0000 7fe89ffff700 1 --2- 172.21.15.7:0/2732878910 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe8a00a2ca0 0x7fe8a00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:46.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.902+0000 7fe89ffff700 1 -- 172.21.15.7:0/2732878910 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fe8a80d0200 con 0x7fe8b8100a80 2024-06-10T01:06:46.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.902+0000 7fe8b67fc700 1 --2- 172.21.15.7:0/2732878910 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe8a00a2ca0 0x7fe8a00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:46.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.902+0000 7fe8b67fc700 1 --2- 172.21.15.7:0/2732878910 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe8a00a2ca0 0x7fe8a00a5160 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8107b20 tx=0x7fe8ac00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:46.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.902+0000 7fe8c7ecdb80 1 Processor -- start 2024-06-10T01:06:46.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.902+0000 7fe8c7ecdb80 1 -- start start 2024-06-10T01:06:46.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.903+0000 7fe8c7ecdb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce09b0b640 0x55ce09b0ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:46.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.903+0000 7fe8c7ecdb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce09b0bf60 0x55ce09b059e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:46.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.903+0000 7fe8c7ecdb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce09b05f20 0x55ce09b06340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:46.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.903+0000 7fe8b6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce09b0b640 0x55ce09b0ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:46.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.903+0000 7fe8b6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce09b0b640 0x55ce09b0ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41198/0 (socket says 172.21.15.7:41198) 2024-06-10T01:06:46.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.903+0000 7fe8b6ffd700 1 -- 172.21.15.7:0/907264854 learned_addr learned my addr 172.21.15.7:0/907264854 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:46.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.903+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/907264854 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ce09a57810 con 0x55ce09b0b640 2024-06-10T01:06:46.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.903+0000 7fe8b67fc700 1 --2- 172.21.15.7:0/907264854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce09b0bf60 0x55ce09b059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:46.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.903+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/907264854 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ce099ff720 con 0x55ce09b05f20 2024-06-10T01:06:46.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.903+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/907264854 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ce09a537d0 con 0x55ce09b0bf60 2024-06-10T01:06:46.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.903+0000 7fe8b77fe700 1 --2- 172.21.15.7:0/907264854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce09b05f20 0x55ce09b06340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.903+0000 7fe8b6ffd700 1 -- 172.21.15.7:0/907264854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce09b0bf60 msgr2=0x55ce09b059e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.903+0000 7fe8b6ffd700 1 --2- 172.21.15.7:0/907264854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce09b0bf60 0x55ce09b059e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.903+0000 7fe8b6ffd700 1 -- 172.21.15.7:0/907264854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce09b05f20 msgr2=0x55ce09b06340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.903+0000 7fe8b6ffd700 1 --2- 172.21.15.7:0/907264854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce09b05f20 0x55ce09b06340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.903+0000 7fe8b6ffd700 1 -- 172.21.15.7:0/907264854 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce099f5f50 con 0x55ce09b0b640 2024-06-10T01:06:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.903+0000 7fe8b77fe700 1 --2- 172.21.15.7:0/907264854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce09b05f20 0x55ce09b06340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:06:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.904+0000 7fe8b6ffd700 1 --2- 172.21.15.7:0/907264854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce09b0b640 0x55ce09b0ba20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe8a809c470 tx=0x7fe8a8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f87b7d5504f2f46 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.904+0000 7fe89d7fa700 1 -- 172.21.15.7:0/907264854 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a80226e0 con 0x55ce09b0b640 2024-06-10T01:06:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.904+0000 7fe89d7fa700 1 -- 172.21.15.7:0/907264854 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8a809ce70 con 0x55ce09b0b640 2024-06-10T01:06:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.905+0000 7fe89d7fa700 1 -- 172.21.15.7:0/907264854 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a80a4ec0 con 0x55ce09b0b640 2024-06-10T01:06:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.905+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/907264854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce09b0b640 msgr2=0x55ce09b0ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.905+0000 7fe8c7ecdb80 1 --2- 172.21.15.7:0/907264854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce09b0b640 0x55ce09b0ba20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe8a809c470 tx=0x7fe8a8009ca0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.905+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/907264854 shutdown_connections 2024-06-10T01:06:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.906+0000 7fe8c7ecdb80 1 --2- 172.21.15.7:0/907264854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce09b05f20 0x55ce09b06340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.906+0000 7fe8c7ecdb80 1 --2- 172.21.15.7:0/907264854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce09b0b640 0x55ce09b0ba20 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.906+0000 7fe8c7ecdb80 1 --2- 172.21.15.7:0/907264854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce09b0bf60 0x55ce09b059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.906+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/907264854 >> 172.21.15.7:0/907264854 conn(0x55ce09a30cc0 msgr2=0x55ce09b00850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:46.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.906+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/907264854 shutdown_connections 2024-06-10T01:06:46.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.906+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/907264854 wait complete. 2024-06-10T01:06:46.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.907+0000 7fe8c7ecdb80 1 -- start start 2024-06-10T01:06:46.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.907+0000 7fe8c7ecdb80 1 Processor -- start 2024-06-10T01:06:46.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.907+0000 7fe8c7ecdb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce09b05f20 0x55ce09b11c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:46.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.907+0000 7fe8c7ecdb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce09b0b640 0x55ce09b0ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:46.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.908+0000 7fe8c7ecdb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce09b0bf60 0x55ce09b0f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:46.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.908+0000 7fe8b6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce09b05f20 0x55ce09b11c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:46.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.908+0000 7fe8b6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce09b05f20 0x55ce09b11c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41208/0 (socket says 172.21.15.7:41208) 2024-06-10T01:06:46.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.908+0000 7fe8b6ffd700 1 -- 172.21.15.7:0/3023643497 learned_addr learned my addr 172.21.15.7:0/3023643497 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:46.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.908+0000 7fe8b67fc700 1 --2- 172.21.15.7:0/3023643497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce09b0b640 0x55ce09b0ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:46.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.908+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/3023643497 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ce09b12790 con 0x55ce09b05f20 2024-06-10T01:06:46.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.908+0000 7fe8b77fe700 1 --2- 172.21.15.7:0/3023643497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce09b0bf60 0x55ce09b0f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:46.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.908+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/3023643497 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ce099f7bf0 con 0x55ce09b0bf60 2024-06-10T01:06:46.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.908+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/3023643497 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ce09b12910 con 0x55ce09b0b640 2024-06-10T01:06:46.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.909+0000 7fe8b6ffd700 1 -- 172.21.15.7:0/3023643497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce09b0b640 msgr2=0x55ce09b0ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:46.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.909+0000 7fe8b6ffd700 1 --2- 172.21.15.7:0/3023643497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce09b0b640 0x55ce09b0ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.909+0000 7fe8b6ffd700 1 -- 172.21.15.7:0/3023643497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce09b0bf60 msgr2=0x55ce09b0f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:46.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.909+0000 7fe8b6ffd700 1 --2- 172.21.15.7:0/3023643497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce09b0bf60 0x55ce09b0f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.909+0000 7fe8b6ffd700 1 -- 172.21.15.7:0/3023643497 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce099a68d0 con 0x55ce09b05f20 2024-06-10T01:06:46.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.909+0000 7fe8c7ecdb80 10 client.0 _ll_register_callbacks cb 0x55ce099f1920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:46.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.909+0000 7fe8b6ffd700 1 --2- 172.21.15.7:0/3023643497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce09b05f20 0x55ce09b11c30 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80cd9d0 tx=0x7fe8a80042b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf921548a12153cc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:46.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.909+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/3023643497 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8a809ce70 con 0x55ce09b05f20 2024-06-10T01:06:46.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.909+0000 7fe89aff5700 10 client.15558 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:06:46.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.909+0000 7fe89aff5700 1 -- 172.21.15.7:0/3023643497 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a80b9020 con 0x55ce09b05f20 2024-06-10T01:06:46.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.909+0000 7fe89aff5700 1 -- 172.21.15.7:0/3023643497 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fe8a80a4ec0 con 0x55ce09b05f20 2024-06-10T01:06:46.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.909+0000 7fe89aff5700 1 -- 172.21.15.7:0/3023643497 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe8a80a50c0 con 0x55ce09b05f20 2024-06-10T01:06:46.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.910+0000 7fe8c7ecdb80 10 client.15558 fetch_fsmap learned FSMap version 27 2024-06-10T01:06:46.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.910+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/3023643497 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55ce09b12790 con 0x55ce09b05f20 2024-06-10T01:06:46.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.910+0000 7fe89aff5700 1 -- 172.21.15.7:0/3023643497 <== mon.0 v2:172.21.15.7:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe8a809bbc0 con 0x55ce09b05f20 2024-06-10T01:06:46.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.910+0000 7fe8c7ecdb80 10 client.15558 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:06:46.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.910+0000 7fe8c7ecdb80 10 client.15558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:06:46.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.910+0000 7fe8c7ecdb80 20 client.15558 populate_metadata read hostname 'smithi007' 2024-06-10T01:06:46.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.910+0000 7fe8c7ecdb80 1 --2- 172.21.15.7:0/3023643497 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ce09b1f190 0x55ce09b21580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:46.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.910+0000 7fe8c7ecdb80 4 client.15558 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:06:46.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.910+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/3023643497 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce09a64d10 con 0x55ce09b1f190 2024-06-10T01:06:46.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.910+0000 7fe8b67fc700 1 --2- 172.21.15.7:0/3023643497 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ce09b1f190 0x55ce09b21580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:46.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.911+0000 7fe89aff5700 10 client.15558 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:46.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.911+0000 7fe8b67fc700 1 --2- 172.21.15.7:0/3023643497 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ce09b1f190 0x55ce09b21580 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b352aa992336f0e server_cookie=9a249efa4bf28d91 in_seq=0 out_seq=0 2024-06-10T01:06:46.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.911+0000 7fe89aff5700 1 -- 172.21.15.7:0/3023643497 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ce09a64d10 con 0x55ce09b1f190 2024-06-10T01:06:46.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.911+0000 7fe89aff5700 10 client.15558 handle_command_reply: tid=0 2024-06-10T01:06:46.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.911+0000 7fe8c7ecdb80 1 client.15558 shutdown 2024-06-10T01:06:46.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.912+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/3023643497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce09b05f20 msgr2=0x55ce09b11c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:46.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.912+0000 7fe8c7ecdb80 1 --2- 172.21.15.7:0/3023643497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce09b05f20 0x55ce09b11c30 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80cd9d0 tx=0x7fe8a80042b0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.912+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/3023643497 shutdown_connections 2024-06-10T01:06:46.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.912+0000 7fe8c7ecdb80 1 --2- 172.21.15.7:0/3023643497 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ce09b1f190 0x55ce09b21580 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.912+0000 7fe8c7ecdb80 1 --2- 172.21.15.7:0/3023643497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce09b0bf60 0x55ce09b0f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.912+0000 7fe8c7ecdb80 1 --2- 172.21.15.7:0/3023643497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce09b05f20 0x55ce09b11c30 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.912+0000 7fe89aff5700 0 client.15558 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:46.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.912+0000 7fe8c7ecdb80 1 --2- 172.21.15.7:0/3023643497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce09b0b640 0x55ce09b0ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.912+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/3023643497 >> 172.21.15.7:0/3023643497 conn(0x55ce09a30cc0 msgr2=0x55ce09b049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.912+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/3023643497 shutdown_connections 2024-06-10T01:06:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.912+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/3023643497 wait complete. 2024-06-10T01:06:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.912+0000 7fe8c7ecdb80 20 client.15558 trim_cache size 0 max 16384 2024-06-10T01:06:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.935+0000 7fe8c7ecdb80 1 Processor -- start 2024-06-10T01:06:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.936+0000 7fe8c7ecdb80 1 -- start start 2024-06-10T01:06:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.936+0000 7fe8c7ecdb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce09b05f20 0x55ce09b059e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.936+0000 7fe8c7ecdb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce09b0b640 0x55ce09b009f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.936+0000 7fe8c7ecdb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce09b0bf60 0x55ce09b0ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.936+0000 7fe8c7ecdb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b81b2ba0 con 0x55ce09b0b640 2024-06-10T01:06:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.936+0000 7fe8c7ecdb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b81b0430 con 0x55ce09b05f20 2024-06-10T01:06:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.936+0000 7fe8c7ecdb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b81ae280 con 0x55ce09b0bf60 2024-06-10T01:06:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.936+0000 7fe8b67fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce09b0b640 0x55ce09b009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.936+0000 7fe8b67fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce09b0b640 0x55ce09b009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41212/0 (socket says 172.21.15.7:41212) 2024-06-10T01:06:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.936+0000 7fe8b67fc700 1 -- 172.21.15.7:0/1570198590 learned_addr learned my addr 172.21.15.7:0/1570198590 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:46.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.936+0000 7fe8b6ffd700 1 --2- 172.21.15.7:0/1570198590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce09b05f20 0x55ce09b059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:46.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.936+0000 7fe8b77fe700 1 --2- 172.21.15.7:0/1570198590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce09b0bf60 0x55ce09b0ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:46.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.936+0000 7fe8b67fc700 1 -- 172.21.15.7:0/1570198590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce09b0bf60 msgr2=0x55ce09b0ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:46.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.936+0000 7fe8b67fc700 1 --2- 172.21.15.7:0/1570198590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce09b0bf60 0x55ce09b0ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.936+0000 7fe8b67fc700 1 -- 172.21.15.7:0/1570198590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce09b05f20 msgr2=0x55ce09b059e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:46.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.936+0000 7fe8b67fc700 1 --2- 172.21.15.7:0/1570198590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce09b05f20 0x55ce09b059e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.936+0000 7fe8b67fc700 1 -- 172.21.15.7:0/1570198590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce09b21bb0 con 0x55ce09b0b640 2024-06-10T01:06:46.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.937+0000 7fe8b67fc700 1 --2- 172.21.15.7:0/1570198590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce09b0b640 0x55ce09b009f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe8b81036f0 tx=0x7fe8ac012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35cab692c806b9f8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:46.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.937+0000 7fe89cff9700 1 -- 172.21.15.7:0/1570198590 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac016070 con 0x55ce09b0b640 2024-06-10T01:06:46.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.937+0000 7fe89cff9700 1 -- 172.21.15.7:0/1570198590 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8ac028430 con 0x55ce09b0b640 2024-06-10T01:06:46.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.937+0000 7fe89cff9700 1 -- 172.21.15.7:0/1570198590 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac01c680 con 0x55ce09b0b640 2024-06-10T01:06:46.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.938+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/1570198590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce09b0b640 msgr2=0x55ce09b009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:46.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.938+0000 7fe8c7ecdb80 1 --2- 172.21.15.7:0/1570198590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce09b0b640 0x55ce09b009f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe8b81036f0 tx=0x7fe8ac012000 comp rx=0 tx=0).stop 2024-06-10T01:06:46.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.938+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/1570198590 shutdown_connections 2024-06-10T01:06:46.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.938+0000 7fe8c7ecdb80 1 --2- 172.21.15.7:0/1570198590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce09b0bf60 0x55ce09b0ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.938+0000 7fe8c7ecdb80 1 --2- 172.21.15.7:0/1570198590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce09b05f20 0x55ce09b059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.938+0000 7fe8c7ecdb80 1 --2- 172.21.15.7:0/1570198590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce09b0b640 0x55ce09b009f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.938+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/1570198590 >> 172.21.15.7:0/1570198590 conn(0x55ce09a30cc0 msgr2=0x55ce09b0c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:46.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.938+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/1570198590 shutdown_connections 2024-06-10T01:06:46.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.938+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/1570198590 wait complete. 2024-06-10T01:06:46.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.938+0000 7fe8c7ecdb80 1 -- start start 2024-06-10T01:06:46.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.939+0000 7fe8c7ecdb80 1 Processor -- start 2024-06-10T01:06:46.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.939+0000 7fe8c7ecdb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce09b05f20 0x55ce09b17c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:46.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.939+0000 7fe8c7ecdb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce09b0b640 0x55ce09b18140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:46.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.939+0000 7fe8c7ecdb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce09b0bf60 0x55ce09b14bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:46.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.939+0000 7fe8c7ecdb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b81ae8f0 con 0x55ce09b05f20 2024-06-10T01:06:46.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.939+0000 7fe8c7ecdb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b81afd60 con 0x55ce09b0bf60 2024-06-10T01:06:46.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.939+0000 7fe8c7ecdb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b81aefc0 con 0x55ce09b0b640 2024-06-10T01:06:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.939+0000 7fe8b6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce09b05f20 0x55ce09b17c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.939+0000 7fe8c7ecdb80 10 client.0 _ll_register_callbacks cb 0x55ce099f1920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.939+0000 7fe8b6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce09b05f20 0x55ce09b17c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41220/0 (socket says 172.21.15.7:41220) 2024-06-10T01:06:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.939+0000 7fe8b6ffd700 1 -- 172.21.15.7:0/3835792215 learned_addr learned my addr 172.21.15.7:0/3835792215 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.939+0000 7fe8b77fe700 1 --2- 172.21.15.7:0/3835792215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce09b0bf60 0x55ce09b14bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.939+0000 7fe8b67fc700 1 --2- 172.21.15.7:0/3835792215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce09b0b640 0x55ce09b18140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.939+0000 7fe8b6ffd700 1 -- 172.21.15.7:0/3835792215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce09b0b640 msgr2=0x55ce09b18140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.939+0000 7fe8b6ffd700 1 --2- 172.21.15.7:0/3835792215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce09b0b640 0x55ce09b18140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.939+0000 7fe8b6ffd700 1 -- 172.21.15.7:0/3835792215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce09b0bf60 msgr2=0x55ce09b14bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.939+0000 7fe8b6ffd700 1 --2- 172.21.15.7:0/3835792215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce09b0bf60 0x55ce09b14bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.939+0000 7fe8b6ffd700 1 -- 172.21.15.7:0/3835792215 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce09b18780 con 0x55ce09b05f20 2024-06-10T01:06:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.939+0000 7fe8b77fe700 1 --2- 172.21.15.7:0/3835792215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce09b0bf60 0x55ce09b14bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:06:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.940+0000 7fe8b6ffd700 1 --2- 172.21.15.7:0/3835792215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce09b05f20 0x55ce09b17c00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8009350 tx=0x7fe8a8003840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5dd2835fda8110 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.940+0000 7fe89aff5700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:06:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.940+0000 7fe89aff5700 1 -- 172.21.15.7:0/3835792215 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a80b9040 con 0x55ce09b05f20 2024-06-10T01:06:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.940+0000 7fe89aff5700 1 -- 172.21.15.7:0/3835792215 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fe8a8003d80 con 0x55ce09b05f20 2024-06-10T01:06:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.940+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/3835792215 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8ac028430 con 0x55ce09b05f20 2024-06-10T01:06:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.940+0000 7fe89aff5700 1 -- 172.21.15.7:0/3835792215 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe8a80226e0 con 0x55ce09b05f20 2024-06-10T01:06:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.940+0000 7fe8c7ecdb80 10 client.15564 fetch_fsmap learned FSMap version 27 2024-06-10T01:06:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.940+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/3835792215 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ce09b18780 con 0x55ce09b05f20 2024-06-10T01:06:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.940+0000 7fe89aff5700 1 -- 172.21.15.7:0/3835792215 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8a809ce70 con 0x55ce09b05f20 2024-06-10T01:06:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.941+0000 7fe89aff5700 1 -- 172.21.15.7:0/3835792215 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe8a809b630 con 0x55ce09b05f20 2024-06-10T01:06:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.941+0000 7fe8c7ecdb80 10 client.15564 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:06:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.941+0000 7fe8c7ecdb80 10 client.15564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:06:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.941+0000 7fe8c7ecdb80 20 client.15564 populate_metadata read hostname 'smithi007' 2024-06-10T01:06:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.941+0000 7fe8c7ecdb80 1 --2- 172.21.15.7:0/3835792215 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ce09b8a6b0 0x55ce09b8caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.941+0000 7fe8c7ecdb80 4 client.15564 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:06:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.941+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/3835792215 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fe8ac028430 con 0x55ce09b8a6b0 2024-06-10T01:06:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.941+0000 7fe8b67fc700 1 --2- 172.21.15.7:0/3835792215 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ce09b8a6b0 0x55ce09b8caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.942+0000 7fe8b67fc700 1 --2- 172.21.15.7:0/3835792215 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ce09b8a6b0 0x55ce09b8caa0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=187118de706b668a server_cookie=1fe8150e7bdce412 in_seq=0 out_seq=0 2024-06-10T01:06:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.942+0000 7fe89aff5700 10 client.15564 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.942+0000 7fe89aff5700 1 -- 172.21.15.7:0/3835792215 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fe8ac028430 con 0x55ce09b8a6b0 2024-06-10T01:06:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.942+0000 7fe89aff5700 10 client.15564 handle_command_reply: tid=0 2024-06-10T01:06:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.942+0000 7fe8c7ecdb80 1 client.15564 shutdown 2024-06-10T01:06:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.942+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/3835792215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce09b05f20 msgr2=0x55ce09b17c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.942+0000 7fe8c7ecdb80 1 --2- 172.21.15.7:0/3835792215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce09b05f20 0x55ce09b17c00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8009350 tx=0x7fe8a8003840 comp rx=0 tx=0).stop 2024-06-10T01:06:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.942+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/3835792215 shutdown_connections 2024-06-10T01:06:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.942+0000 7fe8c7ecdb80 1 --2- 172.21.15.7:0/3835792215 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ce09b8a6b0 0x55ce09b8caa0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.942+0000 7fe8c7ecdb80 1 --2- 172.21.15.7:0/3835792215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce09b0bf60 0x55ce09b14bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.942+0000 7fe8c7ecdb80 1 --2- 172.21.15.7:0/3835792215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce09b05f20 0x55ce09b17c00 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.942+0000 7fe8c7ecdb80 1 --2- 172.21.15.7:0/3835792215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce09b0b640 0x55ce09b18140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.942+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/3835792215 >> 172.21.15.7:0/3835792215 conn(0x55ce09a30cc0 msgr2=0x55ce09af9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:46.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.942+0000 7fe89aff5700 0 client.15564 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:46.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.942+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/3835792215 shutdown_connections 2024-06-10T01:06:46.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.943+0000 7fe8c7ecdb80 1 -- 172.21.15.7:0/3835792215 wait complete. 2024-06-10T01:06:46.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.943+0000 7fe8c7ecdb80 20 client.15564 trim_cache size 0 max 16384 2024-06-10T01:06:46.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.943+0000 7fe8bd64a700 1 -- 172.21.15.7:0/2732878910 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe8a00a2ca0 msgr2=0x7fe8a00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:46.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.943+0000 7fe8bd64a700 1 --2- 172.21.15.7:0/2732878910 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe8a00a2ca0 0x7fe8a00a5160 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8107b20 tx=0x7fe8ac00a300 comp rx=0 tx=0).stop 2024-06-10T01:06:46.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.943+0000 7fe8bd64a700 1 -- 172.21.15.7:0/2732878910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b8100a80 msgr2=0x7fe8b8101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:46.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.943+0000 7fe8bd64a700 1 --2- 172.21.15.7:0/2732878910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b8100a80 0x7fe8b8101c60 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80059d0 tx=0x7fe8a80047b0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.944+0000 7fe8bd64a700 1 -- 172.21.15.7:0/2732878910 shutdown_connections 2024-06-10T01:06:46.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.944+0000 7fe8bd64a700 1 --2- 172.21.15.7:0/2732878910 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe8a00a2ca0 0x7fe8a00a5160 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.944+0000 7fe8bd64a700 1 --2- 172.21.15.7:0/2732878910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8b8108510 0x7fe8b81026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.944+0000 7fe8bd64a700 1 --2- 172.21.15.7:0/2732878910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b8100a80 0x7fe8b8101c60 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.944+0000 7fe8bd64a700 1 --2- 172.21.15.7:0/2732878910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8b8101440 0x7fe8b81021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.944+0000 7fe8bd64a700 1 -- 172.21.15.7:0/2732878910 >> 172.21.15.7:0/2732878910 conn(0x7fe8b8077a00 msgr2=0x7fe8b80ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:46.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.944+0000 7fe8bd64a700 1 -- 172.21.15.7:0/2732878910 shutdown_connections 2024-06-10T01:06:46.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:45.944+0000 7fe8bd64a700 1 -- 172.21.15.7:0/2732878910 wait complete. 2024-06-10T01:06:47.034 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:06:47.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:47 smithi175 ceph-mon[36636]: pgmap v454: 129 pgs: 129 active+clean; 661 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-10T01:06:47.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:47 smithi007 ceph-mon[32398]: pgmap v454: 129 pgs: 129 active+clean; 661 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-10T01:06:47.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:47 smithi078 ceph-mon[36683]: pgmap v454: 129 pgs: 129 active+clean; 661 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-10T01:06:47.976 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:06:47.977 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:06:49.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:49 smithi175 ceph-mon[36636]: pgmap v455: 129 pgs: 129 active+clean; 662 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-10T01:06:49.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:49 smithi007 ceph-mon[32398]: pgmap v455: 129 pgs: 129 active+clean; 662 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-10T01:06:49.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:49 smithi078 ceph-mon[36683]: pgmap v455: 129 pgs: 129 active+clean; 662 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-10T01:06:50.837 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:06:51.633 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:51 smithi078 ceph-mon[36683]: pgmap v456: 129 pgs: 129 active+clean; 662 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:06:51.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:51 smithi175 ceph-mon[36636]: pgmap v456: 129 pgs: 129 active+clean; 662 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:06:51.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:51 smithi007 ceph-mon[32398]: pgmap v456: 129 pgs: 129 active+clean; 662 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:06:53.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.140+0000 7f2d3afc7700 1 -- 172.21.15.7:0/1241485242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d34102f00 msgr2=0x7f2d3410f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:53.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.140+0000 7f2d3afc7700 1 --2- 172.21.15.7:0/1241485242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d34102f00 0x7f2d3410f2c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c00b0a0 tx=0x7f2d2c00b3b0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.140+0000 7f2d3afc7700 1 -- 172.21.15.7:0/1241485242 shutdown_connections 2024-06-10T01:06:53.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.140+0000 7f2d3afc7700 1 --2- 172.21.15.7:0/1241485242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d34102f00 0x7f2d3410f2c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.140+0000 7f2d3afc7700 1 --2- 172.21.15.7:0/1241485242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d341085a0 0x7f2d34108980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.140+0000 7f2d3afc7700 1 --2- 172.21.15.7:0/1241485242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d34102540 0x7f2d341029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.140+0000 7f2d3afc7700 1 -- 172.21.15.7:0/1241485242 >> 172.21.15.7:0/1241485242 conn(0x7f2d34079560 msgr2=0x7f2d34079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:53.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.141+0000 7f2d3afc7700 1 -- 172.21.15.7:0/1241485242 shutdown_connections 2024-06-10T01:06:53.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.141+0000 7f2d3afc7700 1 -- 172.21.15.7:0/1241485242 wait complete. 2024-06-10T01:06:53.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.142+0000 7f2d3afc7700 1 Processor -- start 2024-06-10T01:06:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.142+0000 7f2d3afc7700 1 -- start start 2024-06-10T01:06:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.142+0000 7f2d3afc7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d34102540 0x7f2d3419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.142+0000 7f2d3afc7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d34102f00 0x7f2d3419ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.142+0000 7f2d3afc7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d341085a0 0x7f2d341988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.143+0000 7f2d3afc7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2d34112830 con 0x7f2d341085a0 2024-06-10T01:06:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.143+0000 7f2d3afc7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2d341126b0 con 0x7f2d34102f00 2024-06-10T01:06:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.143+0000 7f2d3afc7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2d341129b0 con 0x7f2d34102540 2024-06-10T01:06:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.143+0000 7f2d38d63700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d34102540 0x7f2d3419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.143+0000 7f2d39564700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d341085a0 0x7f2d341988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.143+0000 7f2d38d63700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d34102540 0x7f2d3419e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:41678/0 (socket says 172.21.15.7:41678) 2024-06-10T01:06:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.143+0000 7f2d38d63700 1 -- 172.21.15.7:0/3229304714 learned_addr learned my addr 172.21.15.7:0/3229304714 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.143+0000 7f2d38d63700 1 -- 172.21.15.7:0/3229304714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d34102f00 msgr2=0x7f2d3419ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.143+0000 7f2d38d63700 1 --2- 172.21.15.7:0/3229304714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d34102f00 0x7f2d3419ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.143+0000 7f2d38d63700 1 -- 172.21.15.7:0/3229304714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d341085a0 msgr2=0x7f2d341988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.143+0000 7f2d38d63700 1 --2- 172.21.15.7:0/3229304714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d341085a0 0x7f2d341988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.143+0000 7f2d38d63700 1 -- 172.21.15.7:0/3229304714 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d34199050 con 0x7f2d34102540 2024-06-10T01:06:53.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.143+0000 7f2d38d63700 1 --2- 172.21.15.7:0/3229304714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d34102540 0x7f2d3419e730 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2d2800d950 tx=0x7f2d2800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:53.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.144+0000 7f2d31ffb700 1 -- 172.21.15.7:0/3229304714 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d28016070 con 0x7f2d34102540 2024-06-10T01:06:53.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.144+0000 7f2d3afc7700 1 -- 172.21.15.7:0/3229304714 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d34199340 con 0x7f2d34102540 2024-06-10T01:06:53.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.144+0000 7f2d39564700 1 --2- 172.21.15.7:0/3229304714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d341085a0 0x7f2d341988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:53.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.144+0000 7f2d31ffb700 1 -- 172.21.15.7:0/3229304714 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d28009e20 con 0x7f2d34102540 2024-06-10T01:06:53.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.144+0000 7f2d3afc7700 1 -- 172.21.15.7:0/3229304714 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d341a5600 con 0x7f2d34102540 2024-06-10T01:06:53.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.144+0000 7f2d31ffb700 1 -- 172.21.15.7:0/3229304714 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2800f620 con 0x7f2d34102540 2024-06-10T01:06:53.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.147+0000 7f2d31ffb700 1 -- 172.21.15.7:0/3229304714 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f2d28019070 con 0x7f2d34102540 2024-06-10T01:06:53.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.148+0000 7f2d31ffb700 1 --2- 172.21.15.7:0/3229304714 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2d240a2cc0 0x7f2d240a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:53.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.148+0000 7f2d31ffb700 1 -- 172.21.15.7:0/3229304714 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f2d280cf2c0 con 0x7f2d34102540 2024-06-10T01:06:53.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.148+0000 7f2d33fff700 1 --2- 172.21.15.7:0/3229304714 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2d240a2cc0 0x7f2d240a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:53.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.148+0000 7f2d33fff700 1 --2- 172.21.15.7:0/3229304714 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2d240a2cc0 0x7f2d240a5180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2d20005950 tx=0x7f2d200058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:53.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.149+0000 7f2d4584ab80 1 Processor -- start 2024-06-10T01:06:53.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.149+0000 7f2d4584ab80 1 -- start start 2024-06-10T01:06:53.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.150+0000 7f2d4584ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d31a5dacd0 0x55d31a5db0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:53.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.150+0000 7f2d4584ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31a5db5f0 0x55d31a5d4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:53.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.150+0000 7f2d4584ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31a5d5460 0x55d31a5d58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:53.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.150+0000 7f2d39564700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31a5d5460 0x55d31a5d58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:53.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.150+0000 7f2d39564700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31a5d5460 0x55d31a5d58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35578/0 (socket says 172.21.15.7:35578) 2024-06-10T01:06:53.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.150+0000 7f2d39564700 1 -- 172.21.15.7:0/4005749941 learned_addr learned my addr 172.21.15.7:0/4005749941 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:53.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.150+0000 7f2d38d63700 1 --2- 172.21.15.7:0/4005749941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d31a5dacd0 0x55d31a5db0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:53.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.150+0000 7f2d33fff700 1 --2- 172.21.15.7:0/4005749941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31a5db5f0 0x55d31a5d4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:53.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.150+0000 7f2d4584ab80 1 -- 172.21.15.7:0/4005749941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d31a5269a0 con 0x55d31a5d5460 2024-06-10T01:06:53.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.151+0000 7f2d4584ab80 1 -- 172.21.15.7:0/4005749941 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d31a4ce8b0 con 0x55d31a5dacd0 2024-06-10T01:06:53.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.151+0000 7f2d4584ab80 1 -- 172.21.15.7:0/4005749941 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d31a522960 con 0x55d31a5db5f0 2024-06-10T01:06:53.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.151+0000 7f2d39564700 1 -- 172.21.15.7:0/4005749941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31a5db5f0 msgr2=0x55d31a5d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:53.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.151+0000 7f2d39564700 1 --2- 172.21.15.7:0/4005749941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31a5db5f0 0x55d31a5d4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.151+0000 7f2d39564700 1 -- 172.21.15.7:0/4005749941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d31a5dacd0 msgr2=0x55d31a5db0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:53.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.151+0000 7f2d39564700 1 --2- 172.21.15.7:0/4005749941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d31a5dacd0 0x55d31a5db0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.151+0000 7f2d39564700 1 -- 172.21.15.7:0/4005749941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d31a4c50e0 con 0x55d31a5d5460 2024-06-10T01:06:53.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.151+0000 7f2d39564700 1 --2- 172.21.15.7:0/4005749941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31a5d5460 0x55d31a5d58a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c009e40 tx=0x7f2d2c0095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad82f1ae2988a577 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:53.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.152+0000 7f2d1effd700 1 -- 172.21.15.7:0/4005749941 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c004d90 con 0x55d31a5d5460 2024-06-10T01:06:53.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.152+0000 7f2d1effd700 1 -- 172.21.15.7:0/4005749941 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d2c0079c0 con 0x55d31a5d5460 2024-06-10T01:06:53.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.152+0000 7f2d1effd700 1 -- 172.21.15.7:0/4005749941 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c005700 con 0x55d31a5d5460 2024-06-10T01:06:53.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.153+0000 7f2d4584ab80 1 -- 172.21.15.7:0/4005749941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31a5d5460 msgr2=0x55d31a5d58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:53.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.153+0000 7f2d4584ab80 1 --2- 172.21.15.7:0/4005749941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31a5d5460 0x55d31a5d58a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c009e40 tx=0x7f2d2c0095a0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.153+0000 7f2d4584ab80 1 -- 172.21.15.7:0/4005749941 shutdown_connections 2024-06-10T01:06:53.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.153+0000 7f2d4584ab80 1 --2- 172.21.15.7:0/4005749941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31a5d5460 0x55d31a5d58a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.153+0000 7f2d4584ab80 1 --2- 172.21.15.7:0/4005749941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d31a5dacd0 0x55d31a5db0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.153+0000 7f2d4584ab80 1 --2- 172.21.15.7:0/4005749941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31a5db5f0 0x55d31a5d4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.153+0000 7f2d4584ab80 1 -- 172.21.15.7:0/4005749941 >> 172.21.15.7:0/4005749941 conn(0x55d31a4ffe50 msgr2=0x55d31a5d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:53.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.153+0000 7f2d4584ab80 1 -- 172.21.15.7:0/4005749941 shutdown_connections 2024-06-10T01:06:53.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.153+0000 7f2d4584ab80 1 -- 172.21.15.7:0/4005749941 wait complete. 2024-06-10T01:06:53.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.154+0000 7f2d4584ab80 1 -- start start 2024-06-10T01:06:53.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.154+0000 7f2d4584ab80 1 Processor -- start 2024-06-10T01:06:53.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.154+0000 7f2d4584ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31a5d5460 0x55d31a5e1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:53.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.154+0000 7f2d4584ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31a5dacd0 0x55d31a5de580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:53.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.154+0000 7f2d4584ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d31a5db5f0 0x55d31a5deac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:53.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.155+0000 7f2d4584ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d31a5e2000 con 0x55d31a5dacd0 2024-06-10T01:06:53.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.155+0000 7f2d4584ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d31a4c6d80 con 0x55d31a5db5f0 2024-06-10T01:06:53.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.155+0000 7f2d38d63700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31a5d5460 0x55d31a5e1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:53.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.155+0000 7f2d4584ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d31a5e2180 con 0x55d31a5d5460 2024-06-10T01:06:53.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.155+0000 7f2d39564700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d31a5db5f0 0x55d31a5deac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:53.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.155+0000 7f2d39564700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d31a5db5f0 0x55d31a5deac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:56706/0 (socket says 172.21.15.7:56706) 2024-06-10T01:06:53.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.155+0000 7f2d39564700 1 -- 172.21.15.7:0/3296869241 learned_addr learned my addr 172.21.15.7:0/3296869241 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:53.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.155+0000 7f2d33fff700 1 --2- 172.21.15.7:0/3296869241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31a5dacd0 0x55d31a5de580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:53.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.155+0000 7f2d4584ab80 10 client.0 _ll_register_callbacks cb 0x55d31a4c0ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:53.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.155+0000 7f2d39564700 1 -- 172.21.15.7:0/3296869241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31a5d5460 msgr2=0x55d31a5e1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:53.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.155+0000 7f2d39564700 1 --2- 172.21.15.7:0/3296869241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31a5d5460 0x55d31a5e1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.155+0000 7f2d39564700 1 -- 172.21.15.7:0/3296869241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31a5dacd0 msgr2=0x55d31a5de580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:53.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.155+0000 7f2d39564700 1 --2- 172.21.15.7:0/3296869241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31a5dacd0 0x55d31a5de580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.155+0000 7f2d39564700 1 -- 172.21.15.7:0/3296869241 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d31a475a60 con 0x55d31a5db5f0 2024-06-10T01:06:53.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.155+0000 7f2d33fff700 1 --2- 172.21.15.7:0/3296869241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31a5dacd0 0x55d31a5de580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:53.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.156+0000 7f2d39564700 1 --2- 172.21.15.7:0/3296869241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d31a5db5f0 0x55d31a5deac0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c00b830 tx=0x7f2d2c006120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=984856a0093f51ff server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:53.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.156+0000 7f2d13fff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:06:53.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.156+0000 7f2d13fff700 1 -- 172.21.15.7:0/3296869241 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c005300 con 0x55d31a5db5f0 2024-06-10T01:06:53.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.156+0000 7f2d4584ab80 1 -- 172.21.15.7:0/3296869241 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d2c0079c0 con 0x55d31a5db5f0 2024-06-10T01:06:53.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.156+0000 7f2d13fff700 1 -- 172.21.15.7:0/3296869241 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f2d2c0054a0 con 0x55d31a5db5f0 2024-06-10T01:06:53.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.156+0000 7f2d13fff700 1 -- 172.21.15.7:0/3296869241 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2d2c039050 con 0x55d31a5db5f0 2024-06-10T01:06:53.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.156+0000 7f2d4584ab80 10 client.25199 fetch_fsmap learned FSMap version 27 2024-06-10T01:06:53.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.156+0000 7f2d4584ab80 1 -- 172.21.15.7:0/3296869241 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d31a5f5060 con 0x55d31a5db5f0 2024-06-10T01:06:53.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.156+0000 7f2d13fff700 1 -- 172.21.15.7:0/3296869241 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d2c0079c0 con 0x55d31a5db5f0 2024-06-10T01:06:53.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.157+0000 7f2d13fff700 1 -- 172.21.15.7:0/3296869241 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2d2c011070 con 0x55d31a5db5f0 2024-06-10T01:06:53.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.157+0000 7f2d4584ab80 10 client.25199 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:06:53.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.157+0000 7f2d4584ab80 10 client.25199 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:06:53.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.157+0000 7f2d4584ab80 20 client.25199 populate_metadata read hostname 'smithi007' 2024-06-10T01:06:53.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.157+0000 7f2d4584ab80 1 --2- 172.21.15.7:0/3296869241 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d31a5f7530 0x55d31a5f9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:53.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.157+0000 7f2d4584ab80 4 client.25199 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:06:53.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.157+0000 7f2d4584ab80 1 -- 172.21.15.7:0/3296869241 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d31a533ea0 con 0x55d31a5f7530 2024-06-10T01:06:53.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.158+0000 7f2d38d63700 1 --2- 172.21.15.7:0/3296869241 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d31a5f7530 0x55d31a5f9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:53.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.158+0000 7f2d13fff700 10 client.25199 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:53.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.158+0000 7f2d38d63700 1 --2- 172.21.15.7:0/3296869241 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d31a5f7530 0x55d31a5f9920 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfece7b975125e4d server_cookie=ffb4c65eebc7cce3 in_seq=0 out_seq=0 2024-06-10T01:06:53.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.159+0000 7f2d13fff700 1 -- 172.21.15.7:0/3296869241 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d31a533ea0 con 0x55d31a5f7530 2024-06-10T01:06:53.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.159+0000 7f2d13fff700 10 client.25199 handle_command_reply: tid=0 2024-06-10T01:06:53.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.159+0000 7f2d4584ab80 1 client.25199 shutdown 2024-06-10T01:06:53.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.159+0000 7f2d4584ab80 1 -- 172.21.15.7:0/3296869241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d31a5db5f0 msgr2=0x55d31a5deac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:53.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.159+0000 7f2d4584ab80 1 --2- 172.21.15.7:0/3296869241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d31a5db5f0 0x55d31a5deac0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c00b830 tx=0x7f2d2c006120 comp rx=0 tx=0).stop 2024-06-10T01:06:53.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.159+0000 7f2d4584ab80 1 -- 172.21.15.7:0/3296869241 shutdown_connections 2024-06-10T01:06:53.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.159+0000 7f2d4584ab80 1 --2- 172.21.15.7:0/3296869241 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d31a5f7530 0x55d31a5f9920 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.159+0000 7f2d13fff700 0 client.25199 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:53.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.159+0000 7f2d4584ab80 1 --2- 172.21.15.7:0/3296869241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d31a5db5f0 0x55d31a5deac0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.159+0000 7f2d4584ab80 1 --2- 172.21.15.7:0/3296869241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31a5d5460 0x55d31a5e1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.159+0000 7f2d4584ab80 1 --2- 172.21.15.7:0/3296869241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31a5dacd0 0x55d31a5de580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.159+0000 7f2d4584ab80 1 -- 172.21.15.7:0/3296869241 >> 172.21.15.7:0/3296869241 conn(0x55d31a4ffe50 msgr2=0x55d31a5cfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:53.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.160+0000 7f2d4584ab80 1 -- 172.21.15.7:0/3296869241 shutdown_connections 2024-06-10T01:06:53.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.160+0000 7f2d4584ab80 1 -- 172.21.15.7:0/3296869241 wait complete. 2024-06-10T01:06:53.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.160+0000 7f2d4584ab80 20 client.25199 trim_cache size 0 max 16384 2024-06-10T01:06:53.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.181+0000 7f2d4584ab80 1 Processor -- start 2024-06-10T01:06:53.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.181+0000 7f2d4584ab80 1 -- start start 2024-06-10T01:06:53.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.181+0000 7f2d4584ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d31a5d5460 0x55d31a5d4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:53.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.181+0000 7f2d4584ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31a5dacd0 0x55d31a5cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:53.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.181+0000 7f2d4584ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31a5db5f0 0x55d31a5feec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:53.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.181+0000 7f2d4584ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2d180112b0 con 0x55d31a5db5f0 2024-06-10T01:06:53.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.181+0000 7f2d4584ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1800eb40 con 0x55d31a5d5460 2024-06-10T01:06:53.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.181+0000 7f2d4584ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1800c990 con 0x55d31a5dacd0 2024-06-10T01:06:53.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.181+0000 7f2d39564700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31a5db5f0 0x55d31a5feec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:53.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.181+0000 7f2d39564700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31a5db5f0 0x55d31a5feec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35604/0 (socket says 172.21.15.7:35604) 2024-06-10T01:06:53.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.181+0000 7f2d39564700 1 -- 172.21.15.7:0/125151836 learned_addr learned my addr 172.21.15.7:0/125151836 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:53.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.181+0000 7f2d33fff700 1 --2- 172.21.15.7:0/125151836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31a5dacd0 0x55d31a5cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:53.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.181+0000 7f2d38d63700 1 --2- 172.21.15.7:0/125151836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d31a5d5460 0x55d31a5d4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:53.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.182+0000 7f2d39564700 1 -- 172.21.15.7:0/125151836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31a5dacd0 msgr2=0x55d31a5cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:53.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.182+0000 7f2d39564700 1 --2- 172.21.15.7:0/125151836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31a5dacd0 0x55d31a5cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.182+0000 7f2d39564700 1 -- 172.21.15.7:0/125151836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d31a5d5460 msgr2=0x55d31a5d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:53.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.182+0000 7f2d39564700 1 --2- 172.21.15.7:0/125151836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d31a5d5460 0x55d31a5d4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.182+0000 7f2d39564700 1 -- 172.21.15.7:0/125151836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d31a5dcb10 con 0x55d31a5db5f0 2024-06-10T01:06:53.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.182+0000 7f2d39564700 1 --2- 172.21.15.7:0/125151836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31a5db5f0 0x55d31a5feec0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c0062a0 tx=0x7f2d2c006190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7744d368b250537 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:53.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.182+0000 7f2d1e7fc700 1 -- 172.21.15.7:0/125151836 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c0076b0 con 0x55d31a5db5f0 2024-06-10T01:06:53.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.182+0000 7f2d1e7fc700 1 -- 172.21.15.7:0/125151836 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d2c007850 con 0x55d31a5db5f0 2024-06-10T01:06:53.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.182+0000 7f2d1e7fc700 1 -- 172.21.15.7:0/125151836 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c005530 con 0x55d31a5db5f0 2024-06-10T01:06:53.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.183+0000 7f2d4584ab80 1 -- 172.21.15.7:0/125151836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31a5db5f0 msgr2=0x55d31a5feec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:53.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.183+0000 7f2d4584ab80 1 --2- 172.21.15.7:0/125151836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31a5db5f0 0x55d31a5feec0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c0062a0 tx=0x7f2d2c006190 comp rx=0 tx=0).stop 2024-06-10T01:06:53.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.183+0000 7f2d4584ab80 1 -- 172.21.15.7:0/125151836 shutdown_connections 2024-06-10T01:06:53.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.183+0000 7f2d4584ab80 1 --2- 172.21.15.7:0/125151836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31a5db5f0 0x55d31a5feec0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.183+0000 7f2d4584ab80 1 --2- 172.21.15.7:0/125151836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d31a5d5460 0x55d31a5d4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.183+0000 7f2d4584ab80 1 --2- 172.21.15.7:0/125151836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31a5dacd0 0x55d31a5cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.183+0000 7f2d4584ab80 1 -- 172.21.15.7:0/125151836 >> 172.21.15.7:0/125151836 conn(0x55d31a4ffe50 msgr2=0x55d31a5cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:53.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.183+0000 7f2d4584ab80 1 -- 172.21.15.7:0/125151836 shutdown_connections 2024-06-10T01:06:53.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.184+0000 7f2d4584ab80 1 -- 172.21.15.7:0/125151836 wait complete. 2024-06-10T01:06:53.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.184+0000 7f2d4584ab80 1 -- start start 2024-06-10T01:06:53.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.184+0000 7f2d4584ab80 1 Processor -- start 2024-06-10T01:06:53.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.184+0000 7f2d4584ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31a5d5460 0x55d31a5f01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:53.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.184+0000 7f2d4584ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31a5dacd0 0x55d31a5ecd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:53.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.184+0000 7f2d4584ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d31a5db5f0 0x55d31a5ed240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:53.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.184+0000 7f2d4584ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1800d000 con 0x55d31a5d5460 2024-06-10T01:06:53.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.184+0000 7f2d4584ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1800e470 con 0x55d31a5db5f0 2024-06-10T01:06:53.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.184+0000 7f2d4584ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1800d6d0 con 0x55d31a5dacd0 2024-06-10T01:06:53.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.184+0000 7f2d4584ab80 10 client.0 _ll_register_callbacks cb 0x55d31a4c0ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:53.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.185+0000 7f2d33fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31a5dacd0 0x55d31a5ecd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:53.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.185+0000 7f2d33fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31a5dacd0 0x55d31a5ecd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:41706/0 (socket says 172.21.15.7:41706) 2024-06-10T01:06:53.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.185+0000 7f2d33fff700 1 -- 172.21.15.7:0/274611045 learned_addr learned my addr 172.21.15.7:0/274611045 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:53.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.185+0000 7f2d39564700 1 --2- 172.21.15.7:0/274611045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d31a5db5f0 0x55d31a5ed240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:53.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.185+0000 7f2d38d63700 1 --2- 172.21.15.7:0/274611045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31a5d5460 0x55d31a5f01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:53.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.185+0000 7f2d33fff700 1 -- 172.21.15.7:0/274611045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d31a5db5f0 msgr2=0x55d31a5ed240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:53.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.185+0000 7f2d33fff700 1 --2- 172.21.15.7:0/274611045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d31a5db5f0 0x55d31a5ed240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.185+0000 7f2d33fff700 1 -- 172.21.15.7:0/274611045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31a5d5460 msgr2=0x55d31a5f01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:53.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.185+0000 7f2d33fff700 1 --2- 172.21.15.7:0/274611045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31a5d5460 0x55d31a5f01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.185+0000 7f2d33fff700 1 -- 172.21.15.7:0/274611045 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d31a5ed780 con 0x55d31a5dacd0 2024-06-10T01:06:53.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.185+0000 7f2d33fff700 1 --2- 172.21.15.7:0/274611045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31a5dacd0 0x55d31a5ecd00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2d20005d90 tx=0x7f2d20005d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de02a1ff05abc4ed server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:53.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.185+0000 7f2d13fff700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:06:53.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.186+0000 7f2d13fff700 1 -- 172.21.15.7:0/274611045 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2000f7d0 con 0x55d31a5dacd0 2024-06-10T01:06:53.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.186+0000 7f2d4584ab80 1 -- 172.21.15.7:0/274611045 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d2c007850 con 0x55d31a5dacd0 2024-06-10T01:06:53.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.186+0000 7f2d13fff700 1 -- 172.21.15.7:0/274611045 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f2d2000f970 con 0x55d31a5dacd0 2024-06-10T01:06:53.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.186+0000 7f2d13fff700 1 -- 172.21.15.7:0/274611045 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2d20034050 con 0x55d31a5dacd0 2024-06-10T01:06:53.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.186+0000 7f2d4584ab80 10 client.25261 fetch_fsmap learned FSMap version 27 2024-06-10T01:06:53.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.186+0000 7f2d4584ab80 1 -- 172.21.15.7:0/274611045 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d31a662bd0 con 0x55d31a5dacd0 2024-06-10T01:06:53.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.186+0000 7f2d13fff700 1 -- 172.21.15.7:0/274611045 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d2c007850 con 0x55d31a5dacd0 2024-06-10T01:06:53.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.186+0000 7f2d13fff700 1 -- 172.21.15.7:0/274611045 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2d2002f3b0 con 0x55d31a5dacd0 2024-06-10T01:06:53.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.186+0000 7f2d4584ab80 10 client.25261 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:06:53.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.187+0000 7f2d4584ab80 10 client.25261 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:06:53.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.187+0000 7f2d4584ab80 20 client.25261 populate_metadata read hostname 'smithi007' 2024-06-10T01:06:53.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.187+0000 7f2d4584ab80 1 --2- 172.21.15.7:0/274611045 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d31a5f7530 0x55d31a6653f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:53.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.187+0000 7f2d4584ab80 4 client.25261 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-06-10T01:06:53.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.187+0000 7f2d4584ab80 1 -- 172.21.15.7:0/274611045 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55d31a52e420 con 0x55d31a5f7530 2024-06-10T01:06:53.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.187+0000 7f2d38d63700 1 --2- 172.21.15.7:0/274611045 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d31a5f7530 0x55d31a6653f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:53.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.188+0000 7f2d13fff700 10 client.25261 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:53.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.188+0000 7f2d38d63700 1 --2- 172.21.15.7:0/274611045 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d31a5f7530 0x55d31a6653f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cba311254ef6870 server_cookie=f895b3b5f63fe5bb in_seq=0 out_seq=0 2024-06-10T01:06:53.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.196+0000 7f2d13fff700 1 -- 172.21.15.7:0/274611045 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55d31a52e420 con 0x55d31a5f7530 2024-06-10T01:06:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.196+0000 7f2d13fff700 10 client.25261 handle_command_reply: tid=0 2024-06-10T01:06:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.196+0000 7f2d4584ab80 1 client.25261 shutdown 2024-06-10T01:06:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.196+0000 7f2d4584ab80 1 -- 172.21.15.7:0/274611045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31a5dacd0 msgr2=0x55d31a5ecd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.196+0000 7f2d4584ab80 1 --2- 172.21.15.7:0/274611045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31a5dacd0 0x55d31a5ecd00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2d20005d90 tx=0x7f2d20005d20 comp rx=0 tx=0).stop 2024-06-10T01:06:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.196+0000 7f2d4584ab80 1 -- 172.21.15.7:0/274611045 shutdown_connections 2024-06-10T01:06:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.196+0000 7f2d4584ab80 1 --2- 172.21.15.7:0/274611045 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d31a5f7530 0x55d31a6653f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.196+0000 7f2d4584ab80 1 --2- 172.21.15.7:0/274611045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d31a5db5f0 0x55d31a5ed240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.196+0000 7f2d13fff700 0 client.25261 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.196+0000 7f2d4584ab80 1 --2- 172.21.15.7:0/274611045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31a5d5460 0x55d31a5f01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.196+0000 7f2d4584ab80 1 --2- 172.21.15.7:0/274611045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31a5dacd0 0x55d31a5ecd00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.196+0000 7f2d4584ab80 1 -- 172.21.15.7:0/274611045 >> 172.21.15.7:0/274611045 conn(0x55d31a4ffe50 msgr2=0x55d31a5d0150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.196+0000 7f2d4584ab80 1 -- 172.21.15.7:0/274611045 shutdown_connections 2024-06-10T01:06:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.196+0000 7f2d4584ab80 1 -- 172.21.15.7:0/274611045 wait complete. 2024-06-10T01:06:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.197+0000 7f2d4584ab80 20 client.25261 trim_cache size 0 max 16384 2024-06-10T01:06:53.194 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:06:53.194 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 27, 2024-06-10T01:06:53.194 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:06:53.194 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 2, 2024-06-10T01:06:53.194 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:06:53.194 INFO:teuthology.orchestra.run.smithi007.stdout: "96f9e47b": { 2024-06-10T01:06:53.194 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 2, 2024-06-10T01:06:53.194 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:06:53.195 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:06:53.195 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:06:53.195 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:06:53.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:06:53.195 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:06:53.195 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 150.0, 2024-06-10T01:06:53.195 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:06:53.195 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:06:53.195 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:06:53.195 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:06:53.195 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:06:53.195 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:06:53.195 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:06:53.195 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:06:53.195 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:06:53.196 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:06:53.196 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:06:53.196 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:06:53.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.197+0000 7f2d3afc7700 1 -- 172.21.15.7:0/3229304714 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2d240a2cc0 msgr2=0x7f2d240a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:53.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.197+0000 7f2d3afc7700 1 --2- 172.21.15.7:0/3229304714 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2d240a2cc0 0x7f2d240a5180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2d20005950 tx=0x7f2d200058e0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.197+0000 7f2d3afc7700 1 -- 172.21.15.7:0/3229304714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d34102540 msgr2=0x7f2d3419e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:53.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.197+0000 7f2d3afc7700 1 --2- 172.21.15.7:0/3229304714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d34102540 0x7f2d3419e730 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2d2800d950 tx=0x7f2d2800dc60 comp rx=0 tx=0).stop 2024-06-10T01:06:53.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.197+0000 7f2d3afc7700 1 -- 172.21.15.7:0/3229304714 shutdown_connections 2024-06-10T01:06:53.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.197+0000 7f2d3afc7700 1 --2- 172.21.15.7:0/3229304714 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2d240a2cc0 0x7f2d240a5180 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.198+0000 7f2d3afc7700 1 --2- 172.21.15.7:0/3229304714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d341085a0 0x7f2d341988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.198+0000 7f2d3afc7700 1 --2- 172.21.15.7:0/3229304714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d34102540 0x7f2d3419e730 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.198+0000 7f2d3afc7700 1 --2- 172.21.15.7:0/3229304714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d34102f00 0x7f2d3419ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:53.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.198+0000 7f2d3afc7700 1 -- 172.21.15.7:0/3229304714 >> 172.21.15.7:0/3229304714 conn(0x7f2d34079560 msgr2=0x7f2d34100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:53.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.198+0000 7f2d3afc7700 1 -- 172.21.15.7:0/3229304714 shutdown_connections 2024-06-10T01:06:53.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:53.198+0000 7f2d3afc7700 1 -- 172.21.15.7:0/3229304714 wait complete. 2024-06-10T01:06:53.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:53 smithi175 ceph-mon[36636]: pgmap v457: 129 pgs: 129 active+clean; 671 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-06-10T01:06:53.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:53 smithi007 ceph-mon[32398]: pgmap v457: 129 pgs: 129 active+clean; 671 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-06-10T01:06:53.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:53 smithi078 ceph-mon[36683]: pgmap v457: 129 pgs: 129 active+clean; 671 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-06-10T01:06:54.195 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '96f9e47b', quiesce took 12.7 seconds. Will release after: 17.3 2024-06-10T01:06:55.570 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:55 smithi007 ceph-mon[32398]: pgmap v458: 129 pgs: 129 active+clean; 671 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-06-10T01:06:55.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:55 smithi175 ceph-mon[36636]: pgmap v458: 129 pgs: 129 active+clean; 671 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-06-10T01:06:55.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.748+0000 7f2df41b7700 1 -- 172.21.15.7:0/3306654857 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dec107400 msgr2=0x7f2dec1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:55.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.748+0000 7f2df41b7700 1 --2- 172.21.15.7:0/3306654857 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dec107400 0x7f2dec1077e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2de8009990 tx=0x7f2de8009ca0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.748+0000 7f2df41b7700 1 -- 172.21.15.7:0/3306654857 shutdown_connections 2024-06-10T01:06:55.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.748+0000 7f2df41b7700 1 --2- 172.21.15.7:0/3306654857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dec101d50 0x7f2dec10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.748+0000 7f2df41b7700 1 --2- 172.21.15.7:0/3306654857 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dec107400 0x7f2dec1077e0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.748+0000 7f2df41b7700 1 --2- 172.21.15.7:0/3306654857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dec101390 0x7f2dec101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.748+0000 7f2df41b7700 1 -- 172.21.15.7:0/3306654857 >> 172.21.15.7:0/3306654857 conn(0x7f2dec078340 msgr2=0x7f2dec078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:55.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.748+0000 7f2df41b7700 1 -- 172.21.15.7:0/3306654857 shutdown_connections 2024-06-10T01:06:55.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.748+0000 7f2df41b7700 1 -- 172.21.15.7:0/3306654857 wait complete. 2024-06-10T01:06:55.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.749+0000 7f2df41b7700 1 Processor -- start 2024-06-10T01:06:55.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.749+0000 7f2df41b7700 1 -- start start 2024-06-10T01:06:55.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.750+0000 7f2df41b7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dec101390 0x7f2dec19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:55.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.750+0000 7f2df41b7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dec101d50 0x7f2dec19ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:55.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.750+0000 7f2df41b7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dec107400 0x7f2dec198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:55.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.750+0000 7f2df41b7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec111670 con 0x7f2dec107400 2024-06-10T01:06:55.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.750+0000 7f2df41b7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1114f0 con 0x7f2dec101d50 2024-06-10T01:06:55.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.750+0000 7f2df41b7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1117f0 con 0x7f2dec101390 2024-06-10T01:06:55.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.750+0000 7f2df1752700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dec101d50 0x7f2dec19ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:55.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.750+0000 7f2df2754700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dec107400 0x7f2dec198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:55.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.750+0000 7f2df2754700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dec107400 0x7f2dec198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35640/0 (socket says 172.21.15.7:35640) 2024-06-10T01:06:55.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.750+0000 7f2df1752700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dec101d50 0x7f2dec19ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:56736/0 (socket says 172.21.15.7:56736) 2024-06-10T01:06:55.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.750+0000 7f2df2754700 1 -- 172.21.15.7:0/1216364689 learned_addr learned my addr 172.21.15.7:0/1216364689 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:55.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.750+0000 7f2df2754700 1 -- 172.21.15.7:0/1216364689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dec101390 msgr2=0x7f2dec19e930 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-10T01:06:55.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.751+0000 7f2df2754700 1 --2- 172.21.15.7:0/1216364689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dec101390 0x7f2dec19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.751+0000 7f2df2754700 1 -- 172.21.15.7:0/1216364689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dec101d50 msgr2=0x7f2dec19ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:55.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.751+0000 7f2df2754700 1 --2- 172.21.15.7:0/1216364689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dec101d50 0x7f2dec19ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.751+0000 7f2df2754700 1 -- 172.21.15.7:0/1216364689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dec199650 con 0x7f2dec107400 2024-06-10T01:06:55.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.751+0000 7f2df1752700 1 --2- 172.21.15.7:0/1216364689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dec101d50 0x7f2dec19ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:55.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.751+0000 7f2df2754700 1 --2- 172.21.15.7:0/1216364689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dec107400 0x7f2dec198d50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2de4007ee0 tx=0x7f2de400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:55.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.751+0000 7f2de2ffd700 1 -- 172.21.15.7:0/1216364689 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de400dbd0 con 0x7f2dec107400 2024-06-10T01:06:55.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.751+0000 7f2de2ffd700 1 -- 172.21.15.7:0/1216364689 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de400dd70 con 0x7f2dec107400 2024-06-10T01:06:55.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.751+0000 7f2de2ffd700 1 -- 172.21.15.7:0/1216364689 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de4005a60 con 0x7f2dec107400 2024-06-10T01:06:55.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.751+0000 7f2df41b7700 1 -- 172.21.15.7:0/1216364689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dec1998e0 con 0x7f2dec107400 2024-06-10T01:06:55.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.751+0000 7f2df41b7700 1 -- 172.21.15.7:0/1216364689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dec1a5720 con 0x7f2dec107400 2024-06-10T01:06:55.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.755+0000 7f2de2ffd700 1 -- 172.21.15.7:0/1216364689 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f2de401a020 con 0x7f2dec107400 2024-06-10T01:06:55.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.756+0000 7f2de2ffd700 1 --2- 172.21.15.7:0/1216364689 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2dd80a2df0 0x7f2dd80a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.757+0000 7f2dfea3ab80 1 Processor -- start 2024-06-10T01:06:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.757+0000 7f2de2ffd700 1 -- 172.21.15.7:0/1216364689 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f2de40cea10 con 0x7f2dec107400 2024-06-10T01:06:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.757+0000 7f2df1f53700 1 --2- 172.21.15.7:0/1216364689 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2dd80a2df0 0x7f2dd80a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.757+0000 7f2dfea3ab80 1 -- start start 2024-06-10T01:06:55.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.757+0000 7f2dfea3ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586d116b080 0x5586d11672e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:55.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.757+0000 7f2dfea3ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586d1167820 0x5586d1167ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:55.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.757+0000 7f2dfea3ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d1179f40 0x5586d117c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:55.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.757+0000 7f2dfea3ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5586d1059fe0 con 0x5586d1167820 2024-06-10T01:06:55.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.757+0000 7f2dfea3ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5586d10b88d0 con 0x5586d1179f40 2024-06-10T01:06:55.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.757+0000 7f2dfea3ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5586d10204e0 con 0x5586d116b080 2024-06-10T01:06:55.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.758+0000 7f2df1752700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586d1167820 0x5586d1167ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:55.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.758+0000 7f2df1752700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586d1167820 0x5586d1167ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35650/0 (socket says 172.21.15.7:35650) 2024-06-10T01:06:55.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.758+0000 7f2df1752700 1 -- 172.21.15.7:0/1304214120 learned_addr learned my addr 172.21.15.7:0/1304214120 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:55.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.758+0000 7f2df2754700 1 --2- 172.21.15.7:0/1304214120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d1179f40 0x5586d117c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:55.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.758+0000 7f2df1f53700 1 --2- 172.21.15.7:0/1216364689 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2dd80a2df0 0x7f2dd80a52b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2de80098e0 tx=0x7f2de8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:55.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.759+0000 7f2df1f53700 1 --2- 172.21.15.7:0/1304214120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586d116b080 0x5586d11672e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:55.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.759+0000 7f2df1752700 1 -- 172.21.15.7:0/1304214120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586d116b080 msgr2=0x5586d11672e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:55.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.759+0000 7f2df1752700 1 --2- 172.21.15.7:0/1304214120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586d116b080 0x5586d11672e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.759+0000 7f2df1752700 1 -- 172.21.15.7:0/1304214120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d1179f40 msgr2=0x5586d117c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:55.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.759+0000 7f2df1752700 1 --2- 172.21.15.7:0/1304214120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d1179f40 0x5586d117c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.759+0000 7f2df1752700 1 -- 172.21.15.7:0/1304214120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586d1007950 con 0x5586d1167820 2024-06-10T01:06:55.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.759+0000 7f2df1752700 1 --2- 172.21.15.7:0/1304214120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586d1167820 0x5586d1167ca0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc0104d0 tx=0x7f2ddc0107e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2d8b503f7794ff server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:55.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.759+0000 7f2dd7fff700 1 -- 172.21.15.7:0/1304214120 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc01c680 con 0x5586d1167820 2024-06-10T01:06:55.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.760+0000 7f2dd7fff700 1 -- 172.21.15.7:0/1304214120 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ddc010e60 con 0x5586d1167820 2024-06-10T01:06:55.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.760+0000 7f2dd7fff700 1 -- 172.21.15.7:0/1304214120 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc011cd0 con 0x5586d1167820 2024-06-10T01:06:55.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.760+0000 7f2dfea3ab80 1 -- 172.21.15.7:0/1304214120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586d1167820 msgr2=0x5586d1167ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:55.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.761+0000 7f2dfea3ab80 1 --2- 172.21.15.7:0/1304214120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586d1167820 0x5586d1167ca0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc0104d0 tx=0x7f2ddc0107e0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.761+0000 7f2dfea3ab80 1 -- 172.21.15.7:0/1304214120 shutdown_connections 2024-06-10T01:06:55.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.761+0000 7f2dfea3ab80 1 --2- 172.21.15.7:0/1304214120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d1179f40 0x5586d117c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.761+0000 7f2dfea3ab80 1 --2- 172.21.15.7:0/1304214120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586d116b080 0x5586d11672e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.761+0000 7f2dfea3ab80 1 --2- 172.21.15.7:0/1304214120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586d1167820 0x5586d1167ca0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.761+0000 7f2dfea3ab80 1 -- 172.21.15.7:0/1304214120 >> 172.21.15.7:0/1304214120 conn(0x5586d1094100 msgr2=0x5586d11622a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:55.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.761+0000 7f2dfea3ab80 1 -- 172.21.15.7:0/1304214120 shutdown_connections 2024-06-10T01:06:55.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.761+0000 7f2dfea3ab80 1 -- 172.21.15.7:0/1304214120 wait complete. 2024-06-10T01:06:55.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.761+0000 7f2dfea3ab80 1 -- start start 2024-06-10T01:06:55.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.761+0000 7f2dfea3ab80 1 Processor -- start 2024-06-10T01:06:55.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.762+0000 7f2dfea3ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d116b080 0x5586d1179920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:55.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.762+0000 7f2dfea3ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586d1179f40 0x5586d1176c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:55.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.762+0000 7f2dfea3ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586d11771b0 0x5586d11775f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:55.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.762+0000 7f2dfea3ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5586d1059040 con 0x5586d1179f40 2024-06-10T01:06:55.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.762+0000 7f2dfea3ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5586d10b4890 con 0x5586d116b080 2024-06-10T01:06:55.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.762+0000 7f2dfea3ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5586d117cb20 con 0x5586d11771b0 2024-06-10T01:06:55.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.762+0000 7f2dfea3ab80 10 client.0 _ll_register_callbacks cb 0x5586d104e630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:55.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.762+0000 7f2df2754700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586d11771b0 0x5586d11775f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:55.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.762+0000 7f2df2754700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586d11771b0 0x5586d11775f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:41760/0 (socket says 172.21.15.7:41760) 2024-06-10T01:06:55.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.762+0000 7f2df2754700 1 -- 172.21.15.7:0/2238221672 learned_addr learned my addr 172.21.15.7:0/2238221672 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:55.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.763+0000 7f2df2754700 1 -- 172.21.15.7:0/2238221672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d116b080 msgr2=0x5586d1179920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:55.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.763+0000 7f2df2754700 1 --2- 172.21.15.7:0/2238221672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d116b080 0x5586d1179920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.763+0000 7f2df2754700 1 -- 172.21.15.7:0/2238221672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586d1179f40 msgr2=0x5586d1176c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:55.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.763+0000 7f2df2754700 1 --2- 172.21.15.7:0/2238221672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586d1179f40 0x5586d1176c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.763+0000 7f2df2754700 1 -- 172.21.15.7:0/2238221672 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586d107e7d0 con 0x5586d11771b0 2024-06-10T01:06:55.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.763+0000 7f2df2754700 1 --2- 172.21.15.7:0/2238221672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586d11771b0 0x5586d11775f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2de40cc130 tx=0x7f2de409af40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=876ac406c72e53d3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:55.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.763+0000 7f2dd57fa700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:06:55.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.763+0000 7f2dd57fa700 1 -- 172.21.15.7:0/2238221672 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de4022250 con 0x5586d11771b0 2024-06-10T01:06:55.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.763+0000 7f2dd57fa700 1 -- 172.21.15.7:0/2238221672 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f2de40223f0 con 0x5586d11771b0 2024-06-10T01:06:55.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.763+0000 7f2dfea3ab80 1 -- 172.21.15.7:0/2238221672 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ddc010e60 con 0x5586d11771b0 2024-06-10T01:06:55.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.764+0000 7f2dd57fa700 1 -- 172.21.15.7:0/2238221672 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2de409b0f0 con 0x5586d11771b0 2024-06-10T01:06:55.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.764+0000 7f2dfea3ab80 10 client.25270 fetch_fsmap learned FSMap version 27 2024-06-10T01:06:55.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.764+0000 7f2dfea3ab80 1 -- 172.21.15.7:0/2238221672 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5586d11878a0 con 0x5586d11771b0 2024-06-10T01:06:55.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.764+0000 7f2dd57fa700 1 -- 172.21.15.7:0/2238221672 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ddc010e60 con 0x5586d11771b0 2024-06-10T01:06:55.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.764+0000 7f2dd57fa700 1 -- 172.21.15.7:0/2238221672 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2de409a2e0 con 0x5586d11771b0 2024-06-10T01:06:55.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.764+0000 7f2dfea3ab80 10 client.25270 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:06:55.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.764+0000 7f2dfea3ab80 10 client.25270 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:06:55.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.764+0000 7f2dfea3ab80 20 client.25270 populate_metadata read hostname 'smithi007' 2024-06-10T01:06:55.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.764+0000 7f2dfea3ab80 1 --2- 172.21.15.7:0/2238221672 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5586d1189e30 0x5586d118c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:55.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.764+0000 7f2dfea3ab80 4 client.25270 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:06:55.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.764+0000 7f2dfea3ab80 1 -- 172.21.15.7:0/2238221672 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586d10c5dd0 con 0x5586d1189e30 2024-06-10T01:06:55.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.765+0000 7f2df1f53700 1 --2- 172.21.15.7:0/2238221672 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5586d1189e30 0x5586d118c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:55.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.765+0000 7f2df1f53700 1 --2- 172.21.15.7:0/2238221672 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5586d1189e30 0x5586d118c2f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=993d1b0d90842321 server_cookie=1284d072510f2d3f in_seq=0 out_seq=0 2024-06-10T01:06:55.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.765+0000 7f2dd57fa700 10 client.25270 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:55.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.766+0000 7f2dd57fa700 1 -- 172.21.15.7:0/2238221672 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5586d10c5dd0 con 0x5586d1189e30 2024-06-10T01:06:55.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.766+0000 7f2dd57fa700 10 client.25270 handle_command_reply: tid=0 2024-06-10T01:06:55.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.766+0000 7f2dfea3ab80 1 client.25270 shutdown 2024-06-10T01:06:55.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.766+0000 7f2dfea3ab80 1 -- 172.21.15.7:0/2238221672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586d11771b0 msgr2=0x5586d11775f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:55.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.766+0000 7f2dfea3ab80 1 --2- 172.21.15.7:0/2238221672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586d11771b0 0x5586d11775f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2de40cc130 tx=0x7f2de409af40 comp rx=0 tx=0).stop 2024-06-10T01:06:55.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.766+0000 7f2dfea3ab80 1 -- 172.21.15.7:0/2238221672 shutdown_connections 2024-06-10T01:06:55.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.766+0000 7f2dfea3ab80 1 --2- 172.21.15.7:0/2238221672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586d11771b0 0x5586d11775f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.766+0000 7f2dfea3ab80 1 --2- 172.21.15.7:0/2238221672 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5586d1189e30 0x5586d118c2f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.766+0000 7f2dfea3ab80 1 --2- 172.21.15.7:0/2238221672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d116b080 0x5586d1179920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.766+0000 7f2dfea3ab80 1 --2- 172.21.15.7:0/2238221672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586d1179f40 0x5586d1176c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.766+0000 7f2dfea3ab80 1 -- 172.21.15.7:0/2238221672 >> 172.21.15.7:0/2238221672 conn(0x5586d1094100 msgr2=0x5586d1161780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:55.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.766+0000 7f2dd57fa700 0 client.25270 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:55.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.766+0000 7f2dfea3ab80 1 -- 172.21.15.7:0/2238221672 shutdown_connections 2024-06-10T01:06:55.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.767+0000 7f2dfea3ab80 1 -- 172.21.15.7:0/2238221672 wait complete. 2024-06-10T01:06:55.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.767+0000 7f2dfea3ab80 20 client.25270 trim_cache size 0 max 16384 2024-06-10T01:06:55.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.808+0000 7f2dfea3ab80 1 Processor -- start 2024-06-10T01:06:55.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.808+0000 7f2dfea3ab80 1 -- start start 2024-06-10T01:06:55.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.808+0000 7f2dfea3ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586d116b080 0x5586d1174790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:55.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.808+0000 7f2dfea3ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586d11771b0 0x5586d1175c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:55.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.808+0000 7f2dfea3ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d1179f40 0x5586d1191c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:55.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.808+0000 7f2dfea3ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1aea80 con 0x5586d116b080 2024-06-10T01:06:55.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.809+0000 7f2dfea3ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1ac310 con 0x5586d1179f40 2024-06-10T01:06:55.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.809+0000 7f2dfea3ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1aa160 con 0x5586d11771b0 2024-06-10T01:06:55.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.809+0000 7f2df1752700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586d11771b0 0x5586d1175c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:55.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.809+0000 7f2df1752700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586d11771b0 0x5586d1175c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:41768/0 (socket says 172.21.15.7:41768) 2024-06-10T01:06:55.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.809+0000 7f2df1f53700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586d116b080 0x5586d1174790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:55.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.809+0000 7f2df1752700 1 -- 172.21.15.7:0/2371370277 learned_addr learned my addr 172.21.15.7:0/2371370277 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:55.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.809+0000 7f2df2754700 1 --2- 172.21.15.7:0/2371370277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d1179f40 0x5586d1191c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:55.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.809+0000 7f2df1752700 1 -- 172.21.15.7:0/2371370277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d1179f40 msgr2=0x5586d1191c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:55.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.809+0000 7f2df1752700 1 --2- 172.21.15.7:0/2371370277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d1179f40 0x5586d1191c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.809+0000 7f2df1752700 1 -- 172.21.15.7:0/2371370277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586d116b080 msgr2=0x5586d1174790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:55.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.809+0000 7f2df1752700 1 --2- 172.21.15.7:0/2371370277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586d116b080 0x5586d1174790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.809+0000 7f2df1752700 1 -- 172.21.15.7:0/2371370277 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586d1175190 con 0x5586d11771b0 2024-06-10T01:06:55.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.809+0000 7f2df2754700 1 --2- 172.21.15.7:0/2371370277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d1179f40 0x5586d1191c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:06:55.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.810+0000 7f2df1f53700 1 --2- 172.21.15.7:0/2371370277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586d116b080 0x5586d1174790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:55.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.810+0000 7f2df1752700 1 --2- 172.21.15.7:0/2371370277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586d11771b0 0x5586d1175c20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc00b4f0 tx=0x7f2ddc00bb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b87e069151cd24fe server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:55.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.810+0000 7f2dd77fe700 1 -- 172.21.15.7:0/2371370277 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc004220 con 0x5586d11771b0 2024-06-10T01:06:55.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.810+0000 7f2dd77fe700 1 -- 172.21.15.7:0/2371370277 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ddc0043c0 con 0x5586d11771b0 2024-06-10T01:06:55.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.810+0000 7f2dd77fe700 1 -- 172.21.15.7:0/2371370277 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc004c50 con 0x5586d11771b0 2024-06-10T01:06:55.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.811+0000 7f2dfea3ab80 1 -- 172.21.15.7:0/2371370277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586d11771b0 msgr2=0x5586d1175c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:55.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.811+0000 7f2dfea3ab80 1 --2- 172.21.15.7:0/2371370277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586d11771b0 0x5586d1175c20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc00b4f0 tx=0x7f2ddc00bb00 comp rx=0 tx=0).stop 2024-06-10T01:06:55.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.811+0000 7f2dfea3ab80 1 -- 172.21.15.7:0/2371370277 shutdown_connections 2024-06-10T01:06:55.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.811+0000 7f2dfea3ab80 1 --2- 172.21.15.7:0/2371370277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d1179f40 0x5586d1191c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.811+0000 7f2dfea3ab80 1 --2- 172.21.15.7:0/2371370277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586d116b080 0x5586d1174790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.811+0000 7f2dfea3ab80 1 --2- 172.21.15.7:0/2371370277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586d11771b0 0x5586d1175c20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.811+0000 7f2dfea3ab80 1 -- 172.21.15.7:0/2371370277 >> 172.21.15.7:0/2371370277 conn(0x5586d1094100 msgr2=0x5586d1175510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:55.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.812+0000 7f2dfea3ab80 1 -- 172.21.15.7:0/2371370277 shutdown_connections 2024-06-10T01:06:55.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.812+0000 7f2dfea3ab80 1 -- 172.21.15.7:0/2371370277 wait complete. 2024-06-10T01:06:55.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.812+0000 7f2dfea3ab80 1 -- start start 2024-06-10T01:06:55.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.812+0000 7f2dfea3ab80 1 Processor -- start 2024-06-10T01:06:55.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.813+0000 7f2dfea3ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d116b080 0x5586d1182580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:55.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.813+0000 7f2dfea3ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586d11771b0 0x5586d117ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:55.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.813+0000 7f2dfea3ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586d1179f40 0x5586d117f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:55.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.814+0000 7f2df1f53700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d116b080 0x5586d1182580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:55.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.814+0000 7f2df1f53700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d116b080 0x5586d1182580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:56766/0 (socket says 172.21.15.7:56766) 2024-06-10T01:06:55.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.814+0000 7f2df1752700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586d11771b0 0x5586d117ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:55.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.814+0000 7f2df1752700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586d11771b0 0x5586d117ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:41782/0 (socket says 172.21.15.7:41782) 2024-06-10T01:06:55.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.814+0000 7f2df2754700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586d1179f40 0x5586d117f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:55.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.814+0000 7f2df2754700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586d1179f40 0x5586d117f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35670/0 (socket says 172.21.15.7:35670) 2024-06-10T01:06:55.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.814+0000 7f2df1f53700 1 -- 172.21.15.7:0/2776881070 learned_addr learned my addr 172.21.15.7:0/2776881070 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:55.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.814+0000 7f2dfea3ab80 1 -- 172.21.15.7:0/2776881070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1aa7d0 con 0x5586d1179f40 2024-06-10T01:06:55.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.814+0000 7f2dfea3ab80 1 -- 172.21.15.7:0/2776881070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1abc40 con 0x5586d116b080 2024-06-10T01:06:55.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.814+0000 7f2dfea3ab80 1 -- 172.21.15.7:0/2776881070 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1aaea0 con 0x5586d11771b0 2024-06-10T01:06:55.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.814+0000 7f2df1f53700 1 -- 172.21.15.7:0/2776881070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586d11771b0 msgr2=0x5586d117ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:55.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.814+0000 7f2df1f53700 1 --2- 172.21.15.7:0/2776881070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586d11771b0 0x5586d117ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.814+0000 7f2df1f53700 1 -- 172.21.15.7:0/2776881070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586d1179f40 msgr2=0x5586d117f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:55.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.814+0000 7f2df1f53700 1 --2- 172.21.15.7:0/2776881070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586d1179f40 0x5586d117f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.814+0000 7f2df1f53700 1 -- 172.21.15.7:0/2776881070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586d117fb40 con 0x5586d116b080 2024-06-10T01:06:55.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.814+0000 7f2df2754700 1 --2- 172.21.15.7:0/2776881070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586d1179f40 0x5586d117f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:55.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.814+0000 7f2dfea3ab80 10 client.0 _ll_register_callbacks cb 0x5586d104e630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:55.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.815+0000 7f2dfea3ab80 1 -- 172.21.15.7:0/2776881070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ddc0043c0 con 0x5586d116b080 2024-06-10T01:06:55.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.815+0000 7f2df1f53700 1 --2- 172.21.15.7:0/2776881070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d116b080 0x5586d1182580 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2de8009990 tx=0x7f2de8003c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=223165382913908f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:55.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.815+0000 7f2dd57fa700 10 client.25214 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:06:55.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.815+0000 7f2dd57fa700 1 -- 172.21.15.7:0/2776881070 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de803d050 con 0x5586d116b080 2024-06-10T01:06:55.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.815+0000 7f2dd57fa700 1 -- 172.21.15.7:0/2776881070 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f2de8005400 con 0x5586d116b080 2024-06-10T01:06:55.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.815+0000 7f2dd57fa700 1 -- 172.21.15.7:0/2776881070 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2de8005600 con 0x5586d116b080 2024-06-10T01:06:55.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.815+0000 7f2dfea3ab80 10 client.25214 fetch_fsmap learned FSMap version 27 2024-06-10T01:06:55.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.815+0000 7f2dfea3ab80 1 -- 172.21.15.7:0/2776881070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=27}) -- 0x5586d11f5a90 con 0x5586d116b080 2024-06-10T01:06:55.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.816+0000 7f2dd57fa700 1 -- 172.21.15.7:0/2776881070 <== mon.2 v2:172.21.15.78:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2de800b7d0 con 0x5586d116b080 2024-06-10T01:06:55.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.816+0000 7f2dfea3ab80 10 client.25214 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:06:55.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.816+0000 7f2dfea3ab80 10 client.25214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:06:55.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.816+0000 7f2dfea3ab80 20 client.25214 populate_metadata read hostname 'smithi007' 2024-06-10T01:06:55.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.816+0000 7f2dfea3ab80 1 --2- 172.21.15.7:0/2776881070 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5586d11f6050 0x5586d11f8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:55.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.816+0000 7f2dfea3ab80 4 client.25214 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:06:55.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.816+0000 7f2dfea3ab80 1 -- 172.21.15.7:0/2776881070 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5586d10c0350 con 0x5586d11f6050 2024-06-10T01:06:55.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.817+0000 7f2df1752700 1 --2- 172.21.15.7:0/2776881070 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5586d11f6050 0x5586d11f8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:55.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.817+0000 7f2df1752700 1 --2- 172.21.15.7:0/2776881070 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5586d11f6050 0x5586d11f8510 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9df6e6778150329e server_cookie=10cdb2a79de1adce in_seq=0 out_seq=0 2024-06-10T01:06:55.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.817+0000 7f2dd57fa700 10 client.25214 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:55.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.818+0000 7f2dd57fa700 1 -- 172.21.15.7:0/2776881070 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5586d10c0350 con 0x5586d11f6050 2024-06-10T01:06:55.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.818+0000 7f2dd57fa700 10 client.25214 handle_command_reply: tid=0 2024-06-10T01:06:55.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.818+0000 7f2dfea3ab80 1 client.25214 shutdown 2024-06-10T01:06:55.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.818+0000 7f2dfea3ab80 1 -- 172.21.15.7:0/2776881070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d116b080 msgr2=0x5586d1182580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:55.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.818+0000 7f2dfea3ab80 1 --2- 172.21.15.7:0/2776881070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d116b080 0x5586d1182580 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2de8009990 tx=0x7f2de8003c40 comp rx=0 tx=0).stop 2024-06-10T01:06:55.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.819+0000 7f2dfea3ab80 1 -- 172.21.15.7:0/2776881070 shutdown_connections 2024-06-10T01:06:55.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.819+0000 7f2dfea3ab80 1 --2- 172.21.15.7:0/2776881070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586d1179f40 0x5586d117f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.819+0000 7f2dfea3ab80 1 --2- 172.21.15.7:0/2776881070 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5586d11f6050 0x5586d11f8510 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.819+0000 7f2dfea3ab80 1 --2- 172.21.15.7:0/2776881070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d116b080 0x5586d1182580 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.819+0000 7f2dfea3ab80 1 --2- 172.21.15.7:0/2776881070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586d11771b0 0x5586d117ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.819+0000 7f2dfea3ab80 1 -- 172.21.15.7:0/2776881070 >> 172.21.15.7:0/2776881070 conn(0x5586d1094100 msgr2=0x5586d1160ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:55.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.819+0000 7f2dd57fa700 0 client.25214 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:55.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.819+0000 7f2dfea3ab80 1 -- 172.21.15.7:0/2776881070 shutdown_connections 2024-06-10T01:06:55.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.819+0000 7f2dfea3ab80 1 -- 172.21.15.7:0/2776881070 wait complete. 2024-06-10T01:06:55.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.819+0000 7f2dfea3ab80 20 client.25214 trim_cache size 0 max 16384 2024-06-10T01:06:55.818 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:06:55.819 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-10T01:06:55.819 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "a3b085a6-ffc2-42b1-ba02-b56b65e7bb46", 2024-06-10T01:06:55.819 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-10T01:06:55.819 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:06:55.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.820+0000 7f2df41b7700 1 -- 172.21.15.7:0/1216364689 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2dd80a2df0 msgr2=0x7f2dd80a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:55.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.820+0000 7f2df41b7700 1 --2- 172.21.15.7:0/1216364689 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2dd80a2df0 0x7f2dd80a52b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2de80098e0 tx=0x7f2de8005fb0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.820+0000 7f2df41b7700 1 -- 172.21.15.7:0/1216364689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dec107400 msgr2=0x7f2dec198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:55.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.820+0000 7f2df41b7700 1 --2- 172.21.15.7:0/1216364689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dec107400 0x7f2dec198d50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2de4007ee0 tx=0x7f2de400d3b0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.820+0000 7f2df41b7700 1 -- 172.21.15.7:0/1216364689 shutdown_connections 2024-06-10T01:06:55.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.820+0000 7f2df41b7700 1 --2- 172.21.15.7:0/1216364689 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2dd80a2df0 0x7f2dd80a52b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.820+0000 7f2df41b7700 1 --2- 172.21.15.7:0/1216364689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dec107400 0x7f2dec198d50 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.820+0000 7f2df41b7700 1 --2- 172.21.15.7:0/1216364689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dec101390 0x7f2dec19e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.820+0000 7f2df41b7700 1 --2- 172.21.15.7:0/1216364689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dec101d50 0x7f2dec19ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:55.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.820+0000 7f2df41b7700 1 -- 172.21.15.7:0/1216364689 >> 172.21.15.7:0/1216364689 conn(0x7f2dec078340 msgr2=0x7f2dec0fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:55.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.820+0000 7f2df41b7700 1 -- 172.21.15.7:0/1216364689 shutdown_connections 2024-06-10T01:06:55.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:55.821+0000 7f2df41b7700 1 -- 172.21.15.7:0/1216364689 wait complete. 2024-06-10T01:06:55.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:55 smithi078 ceph-mon[36683]: pgmap v458: 129 pgs: 129 active+clean; 671 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-06-10T01:06:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.385+0000 7f4d02eba640 1 -- 172.21.15.7:0/3048165638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cfc1080a0 msgr2=0x7f4cfc1084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.385+0000 7f4d02eba640 1 --2- 172.21.15.7:0/3048165638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cfc1080a0 0x7f4cfc1084c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009860 tx=0x7f4cf002a340 comp rx=0 tx=0).stop 2024-06-10T01:06:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.385+0000 7f4d02eba640 1 -- 172.21.15.7:0/3048165638 shutdown_connections 2024-06-10T01:06:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.385+0000 7f4d02eba640 1 --2- 172.21.15.7:0/3048165638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cfc109c00 0x7f4cfc110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.385+0000 7f4d02eba640 1 --2- 172.21.15.7:0/3048165638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cfc109240 0x7f4cfc1096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.385+0000 7f4d02eba640 1 --2- 172.21.15.7:0/3048165638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cfc1080a0 0x7f4cfc1084c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.385+0000 7f4d02eba640 1 -- 172.21.15.7:0/3048165638 >> 172.21.15.7:0/3048165638 conn(0x7f4cfc076ff0 msgr2=0x7f4cfc079430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.385+0000 7f4d02eba640 1 -- 172.21.15.7:0/3048165638 shutdown_connections 2024-06-10T01:06:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.385+0000 7f4d02eba640 1 -- 172.21.15.7:0/3048165638 wait complete. 2024-06-10T01:06:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.386+0000 7f4d02eba640 1 Processor -- start 2024-06-10T01:06:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.386+0000 7f4d02eba640 1 -- start start 2024-06-10T01:06:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.386+0000 7f4d02eba640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cfc1080a0 0x7f4cfc19e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.386+0000 7f4d02eba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cfc109240 0x7f4cfc19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.386+0000 7f4d02eba640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cfc109c00 0x7f4cfc1a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.387+0000 7f4d02eba640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc113ce0 con 0x7f4cfc109c00 2024-06-10T01:06:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.387+0000 7f4d02eba640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc113b60 con 0x7f4cfc109240 2024-06-10T01:06:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.387+0000 7f4d02eba640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc113e60 con 0x7f4cfc1080a0 2024-06-10T01:06:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.387+0000 7f4d00c56640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cfc1080a0 0x7f4cfc19e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.387+0000 7f4cfbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cfc109240 0x7f4cfc19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.387+0000 7f4d00c56640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cfc1080a0 0x7f4cfc19e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:41800/0 (socket says 172.21.15.7:41800) 2024-06-10T01:06:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.387+0000 7f4d00c56640 1 -- 172.21.15.7:0/1060015922 learned_addr learned my addr 172.21.15.7:0/1060015922 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.387+0000 7f4d01457640 1 --2- 172.21.15.7:0/1060015922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cfc109c00 0x7f4cfc1a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.387+0000 7f4d00c56640 1 -- 172.21.15.7:0/1060015922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cfc109240 msgr2=0x7f4cfc19e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.387+0000 7f4d00c56640 1 --2- 172.21.15.7:0/1060015922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cfc109240 0x7f4cfc19e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.387+0000 7f4d00c56640 1 -- 172.21.15.7:0/1060015922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cfc109c00 msgr2=0x7f4cfc1a58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.387+0000 7f4d00c56640 1 --2- 172.21.15.7:0/1060015922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cfc109c00 0x7f4cfc1a58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.387+0000 7f4d00c56640 1 -- 172.21.15.7:0/1060015922 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cfc1a5f60 con 0x7f4cfc1080a0 2024-06-10T01:06:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.387+0000 7f4cfbfff640 1 --2- 172.21.15.7:0/1060015922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cfc109240 0x7f4cfc19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:06:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.387+0000 7f4d01457640 1 --2- 172.21.15.7:0/1060015922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cfc109c00 0x7f4cfc1a58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:06:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.387+0000 7f4d00c56640 1 --2- 172.21.15.7:0/1060015922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cfc1080a0 0x7f4cfc19e340 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0035040 tx=0x7f4cf0033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.388+0000 7f4cf9ffb640 1 -- 172.21.15.7:0/1060015922 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0037040 con 0x7f4cfc1080a0 2024-06-10T01:06:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.388+0000 7f4d02eba640 1 -- 172.21.15.7:0/1060015922 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cfc1a61f0 con 0x7f4cfc1080a0 2024-06-10T01:06:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.388+0000 7f4cf9ffb640 1 -- 172.21.15.7:0/1060015922 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cf0030850 con 0x7f4cfc1080a0 2024-06-10T01:06:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.388+0000 7f4cf9ffb640 1 -- 172.21.15.7:0/1060015922 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0004920 con 0x7f4cfc1080a0 2024-06-10T01:06:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.388+0000 7f4d02eba640 1 -- 172.21.15.7:0/1060015922 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cfc1a66e0 con 0x7f4cfc1080a0 2024-06-10T01:06:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.390+0000 7f4cf9ffb640 1 -- 172.21.15.7:0/1060015922 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f4cf0004050 con 0x7f4cfc1080a0 2024-06-10T01:06:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.392+0000 7f4d05bca740 1 Processor -- start 2024-06-10T01:06:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.392+0000 7f4d05bca740 1 -- start start 2024-06-10T01:06:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.392+0000 7f4d05bca740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564010bf4010 0x564010bf4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.392+0000 7f4d05bca740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564010bf4a00 0x564010bf9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.392+0000 7f4d05bca740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564010bf9b70 0x564010bfc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.392+0000 7f4d05bca740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564010b574f0 con 0x564010bf9b70 2024-06-10T01:06:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.392+0000 7f4d05bca740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564010af5870 con 0x564010bf4a00 2024-06-10T01:06:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.392+0000 7f4d05bca740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564010b534b0 con 0x564010bf4010 2024-06-10T01:06:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.392+0000 7f4cf9ffb640 1 --2- 172.21.15.7:0/1060015922 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4cec0a2b80 0x7f4cec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.393+0000 7f4cfbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564010bf9b70 0x564010bfc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.393+0000 7f4cf9ffb640 1 -- 172.21.15.7:0/1060015922 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f4cf00ef3e0 con 0x7f4cfc1080a0 2024-06-10T01:06:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.393+0000 7f4d01457640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564010bf4010 0x564010bf4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.393+0000 7f4d00c56640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564010bf4a00 0x564010bf9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.393+0000 7f4d00c56640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564010bf4a00 0x564010bf9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:56782/0 (socket says 172.21.15.7:56782) 2024-06-10T01:06:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.393+0000 7f4d00c56640 1 -- 172.21.15.7:0/886347026 learned_addr learned my addr 172.21.15.7:0/886347026 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.393+0000 7f4d01457640 1 --2- 172.21.15.7:0/1060015922 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4cec0a2b80 0x7f4cec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.393+0000 7f4d00c56640 1 -- 172.21.15.7:0/886347026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564010bf4010 msgr2=0x564010bf4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.393+0000 7f4d00c56640 1 --2- 172.21.15.7:0/886347026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564010bf4010 0x564010bf4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.393+0000 7f4d00c56640 1 -- 172.21.15.7:0/886347026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564010bf9b70 msgr2=0x564010bfc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.393+0000 7f4d00c56640 1 --2- 172.21.15.7:0/886347026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564010bf9b70 0x564010bfc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.393+0000 7f4d00c56640 1 -- 172.21.15.7:0/886347026 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564010b119b0 con 0x564010bf4a00 2024-06-10T01:06:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.393+0000 7f4cfbfff640 1 --2- 172.21.15.7:0/886347026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564010bf9b70 0x564010bfc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:06:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.393+0000 7f4d00c56640 1 --2- 172.21.15.7:0/886347026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564010bf4a00 0x564010bf9500 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4cf00b7cc0 tx=0x7f4cf002a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f6a94c61ef1fd5a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.393+0000 7f4ce6ffd640 1 -- 172.21.15.7:0/886347026 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf00bb1c0 con 0x564010bf4a00 2024-06-10T01:06:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.394+0000 7f4ce6ffd640 1 -- 172.21.15.7:0/886347026 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cf0030a00 con 0x564010bf4a00 2024-06-10T01:06:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.394+0000 7f4d05bca740 1 -- 172.21.15.7:0/886347026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564010bf4a00 msgr2=0x564010bf9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.394+0000 7f4d01457640 1 --2- 172.21.15.7:0/1060015922 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4cec0a2b80 0x7f4cec0a5040 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4009fd0 tx=0x7f4cf4004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.394+0000 7f4d05bca740 1 --2- 172.21.15.7:0/886347026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564010bf4a00 0x564010bf9500 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4cf00b7cc0 tx=0x7f4cf002a340 comp rx=0 tx=0).stop 2024-06-10T01:06:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.395+0000 7f4d05bca740 1 -- 172.21.15.7:0/886347026 shutdown_connections 2024-06-10T01:06:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.395+0000 7f4d05bca740 1 --2- 172.21.15.7:0/886347026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564010bf9b70 0x564010bfc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.395+0000 7f4d05bca740 1 --2- 172.21.15.7:0/886347026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564010bf4a00 0x564010bf9500 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.395+0000 7f4d05bca740 1 --2- 172.21.15.7:0/886347026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564010bf4010 0x564010bf4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.395+0000 7f4d05bca740 1 -- 172.21.15.7:0/886347026 >> 172.21.15.7:0/886347026 conn(0x564010bef680 msgr2=0x564010befa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.395+0000 7f4d05bca740 1 -- 172.21.15.7:0/886347026 shutdown_connections 2024-06-10T01:06:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.395+0000 7f4d05bca740 1 -- 172.21.15.7:0/886347026 wait complete. 2024-06-10T01:06:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.395+0000 7f4d05bca740 1 -- start start 2024-06-10T01:06:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.395+0000 7f4d05bca740 1 Processor -- start 2024-06-10T01:06:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.396+0000 7f4d05bca740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564010bf4010 0x564010bfdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.396+0000 7f4d05bca740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564010bf4a00 0x564010bfe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.396+0000 7f4d05bca740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564010bf9b70 0x564010c05300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.396+0000 7f4d05bca740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564010af5870 con 0x564010bf4a00 2024-06-10T01:06:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.396+0000 7f4d05bca740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564010ae0ae0 con 0x564010bf4010 2024-06-10T01:06:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.396+0000 7f4d05bca740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564010bfc8e0 con 0x564010bf9b70 2024-06-10T01:06:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.396+0000 7f4d05bca740 10 client.0 _ll_register_callbacks cb 0x564010b199b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.396+0000 7f4cfbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564010bf4a00 0x564010bfe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.396+0000 7f4d00c56640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564010bf4010 0x564010bfdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.396+0000 7f4d00c56640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564010bf4010 0x564010bfdd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:56786/0 (socket says 172.21.15.7:56786) 2024-06-10T01:06:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.396+0000 7f4d00c56640 1 -- 172.21.15.7:0/3438736093 learned_addr learned my addr 172.21.15.7:0/3438736093 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.396+0000 7f4d01457640 1 --2- 172.21.15.7:0/3438736093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564010bf9b70 0x564010c05300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.397+0000 7f4d00c56640 1 -- 172.21.15.7:0/3438736093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564010bf9b70 msgr2=0x564010c05300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.397+0000 7f4d00c56640 1 --2- 172.21.15.7:0/3438736093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564010bf9b70 0x564010c05300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.397+0000 7f4d00c56640 1 -- 172.21.15.7:0/3438736093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564010bf4a00 msgr2=0x564010bfe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.397+0000 7f4d00c56640 1 --2- 172.21.15.7:0/3438736093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564010bf4a00 0x564010bfe2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.397+0000 7f4d00c56640 1 -- 172.21.15.7:0/3438736093 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56401086b210 con 0x564010bf4010 2024-06-10T01:06:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.397+0000 7f4cfbfff640 1 --2- 172.21.15.7:0/3438736093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564010bf4a00 0x564010bfe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.397+0000 7f4d00c56640 1 --2- 172.21.15.7:0/3438736093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564010bf4010 0x564010bfdd80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4cf00095f0 tx=0x7f4cf00b74c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d8e6643497f2e84 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.397+0000 7f4cdbfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:06:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.397+0000 7f4cdbfff640 1 -- 172.21.15.7:0/3438736093 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf00bbf60 con 0x564010bf4010 2024-06-10T01:06:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.397+0000 7f4d05bca740 1 -- 172.21.15.7:0/3438736093 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cf0030a00 con 0x564010bf4010 2024-06-10T01:06:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.397+0000 7f4cdbfff640 1 -- 172.21.15.7:0/3438736093 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f4cf00a60b0 con 0x564010bf4010 2024-06-10T01:06:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.397+0000 7f4cdbfff640 1 -- 172.21.15.7:0/3438736093 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4cf00a62b0 con 0x564010bf4010 2024-06-10T01:06:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.397+0000 7f4d05bca740 10 client.25226 fetch_fsmap learned FSMap version 27 2024-06-10T01:06:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.397+0000 7f4d05bca740 1 -- 172.21.15.7:0/3438736093 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564010937620 con 0x564010bf4010 2024-06-10T01:06:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.398+0000 7f4cdbfff640 1 -- 172.21.15.7:0/3438736093 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cf0030a00 con 0x564010bf4010 2024-06-10T01:06:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.398+0000 7f4cdbfff640 1 -- 172.21.15.7:0/3438736093 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4cf003a070 con 0x564010bf4010 2024-06-10T01:06:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.398+0000 7f4d05bca740 10 client.25226 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:06:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.398+0000 7f4d05bca740 10 client.25226 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:06:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.398+0000 7f4d05bca740 20 client.25226 populate_metadata read hostname 'smithi007' 2024-06-10T01:06:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.398+0000 7f4d05bca740 1 --2- 172.21.15.7:0/3438736093 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564010c09a30 0x564010c0be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.398+0000 7f4d05bca740 4 client.25226 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:06:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.398+0000 7f4d05bca740 1 -- 172.21.15.7:0/3438736093 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564010b6af10 con 0x564010c09a30 2024-06-10T01:06:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.398+0000 7f4cfbfff640 1 --2- 172.21.15.7:0/3438736093 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564010c09a30 0x564010c0be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.399+0000 7f4cdbfff640 10 client.25226 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.399+0000 7f4cfbfff640 1 --2- 172.21.15.7:0/3438736093 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564010c09a30 0x564010c0be20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b00c9f65594342a server_cookie=352fb32ac8798549 in_seq=0 out_seq=0 2024-06-10T01:06:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.400+0000 7f4cdbfff640 1 -- 172.21.15.7:0/3438736093 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564010b6af10 con 0x564010c09a30 2024-06-10T01:06:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.400+0000 7f4cdbfff640 10 client.25226 handle_command_reply: tid=0 2024-06-10T01:06:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.400+0000 7f4d05bca740 1 client.25226 shutdown 2024-06-10T01:06:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.400+0000 7f4d05bca740 1 -- 172.21.15.7:0/3438736093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564010bf4010 msgr2=0x564010bfdd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.400+0000 7f4d05bca740 1 --2- 172.21.15.7:0/3438736093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564010bf4010 0x564010bfdd80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4cf00095f0 tx=0x7f4cf00b74c0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.400+0000 7f4d05bca740 1 -- 172.21.15.7:0/3438736093 shutdown_connections 2024-06-10T01:06:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.400+0000 7f4d05bca740 1 --2- 172.21.15.7:0/3438736093 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564010c09a30 0x564010c0be20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.400+0000 7f4d05bca740 1 --2- 172.21.15.7:0/3438736093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564010bf9b70 0x564010c05300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.400+0000 7f4d05bca740 1 --2- 172.21.15.7:0/3438736093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564010bf4a00 0x564010bfe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.400+0000 7f4d05bca740 1 --2- 172.21.15.7:0/3438736093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564010bf4010 0x564010bfdd80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.400+0000 7f4d05bca740 1 -- 172.21.15.7:0/3438736093 >> 172.21.15.7:0/3438736093 conn(0x564010bef680 msgr2=0x564010bee7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.400+0000 7f4cdbfff640 0 client.25226 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.400+0000 7f4d05bca740 1 -- 172.21.15.7:0/3438736093 shutdown_connections 2024-06-10T01:06:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.400+0000 7f4d05bca740 1 -- 172.21.15.7:0/3438736093 wait complete. 2024-06-10T01:06:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.401+0000 7f4d05bca740 20 client.25226 trim_cache size 0 max 16384 2024-06-10T01:06:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.432+0000 7f4d05bca740 1 Processor -- start 2024-06-10T01:06:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.432+0000 7f4d05bca740 1 -- start start 2024-06-10T01:06:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.432+0000 7f4d05bca740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564010bf4010 0x564010bfe0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.432+0000 7f4d05bca740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564010bf4a00 0x564010c05290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.432+0000 7f4d05bca740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564010bf9b70 0x564010bf2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.432+0000 7f4d05bca740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc1b34e0 con 0x564010bf4010 2024-06-10T01:06:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.432+0000 7f4d05bca740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc1b0db0 con 0x564010bf9b70 2024-06-10T01:06:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.432+0000 7f4d00c56640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564010bf4010 0x564010bfe0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.432+0000 7f4d05bca740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc1aec00 con 0x564010bf4a00 2024-06-10T01:06:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.432+0000 7f4d00c56640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564010bf4010 0x564010bfe0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35710/0 (socket says 172.21.15.7:35710) 2024-06-10T01:06:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.432+0000 7f4d00c56640 1 -- 172.21.15.7:0/918459176 learned_addr learned my addr 172.21.15.7:0/918459176 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.432+0000 7f4d01457640 1 --2- 172.21.15.7:0/918459176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564010bf9b70 0x564010bf2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.432+0000 7f4d00c56640 1 -- 172.21.15.7:0/918459176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564010bf4a00 msgr2=0x564010c05290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.432+0000 7f4d00c56640 1 --2- 172.21.15.7:0/918459176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564010bf4a00 0x564010c05290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.432+0000 7f4d00c56640 1 -- 172.21.15.7:0/918459176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564010bf9b70 msgr2=0x564010bf2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.432+0000 7f4d00c56640 1 --2- 172.21.15.7:0/918459176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564010bf9b70 0x564010bf2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.432+0000 7f4d00c56640 1 -- 172.21.15.7:0/918459176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564010c0c400 con 0x564010bf4010 2024-06-10T01:06:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.432+0000 7f4d01457640 1 --2- 172.21.15.7:0/918459176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564010bf9b70 0x564010bf2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:06:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.433+0000 7f4d00c56640 1 --2- 172.21.15.7:0/918459176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564010bf4010 0x564010bfe0a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4cf00b7540 tx=0x7f4cf00b9d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534158fb016d858d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.433+0000 7f4ce67fc640 1 -- 172.21.15.7:0/918459176 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0030a00 con 0x564010bf4010 2024-06-10T01:06:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.433+0000 7f4ce67fc640 1 -- 172.21.15.7:0/918459176 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cf00bbf60 con 0x564010bf4010 2024-06-10T01:06:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.433+0000 7f4ce67fc640 1 -- 172.21.15.7:0/918459176 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf00ecb80 con 0x564010bf4010 2024-06-10T01:06:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.434+0000 7f4d05bca740 1 -- 172.21.15.7:0/918459176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564010bf4010 msgr2=0x564010bfe0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.434+0000 7f4d05bca740 1 --2- 172.21.15.7:0/918459176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564010bf4010 0x564010bfe0a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4cf00b7540 tx=0x7f4cf00b9d70 comp rx=0 tx=0).stop 2024-06-10T01:06:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.434+0000 7f4d05bca740 1 -- 172.21.15.7:0/918459176 shutdown_connections 2024-06-10T01:06:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.434+0000 7f4d05bca740 1 --2- 172.21.15.7:0/918459176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564010bf9b70 0x564010bf2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.434+0000 7f4d05bca740 1 --2- 172.21.15.7:0/918459176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564010bf4a00 0x564010c05290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.434+0000 7f4d05bca740 1 --2- 172.21.15.7:0/918459176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564010bf4010 0x564010bfe0a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.434+0000 7f4d05bca740 1 -- 172.21.15.7:0/918459176 >> 172.21.15.7:0/918459176 conn(0x564010bef680 msgr2=0x564010bfd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.434+0000 7f4d05bca740 1 -- 172.21.15.7:0/918459176 shutdown_connections 2024-06-10T01:06:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.434+0000 7f4d05bca740 1 -- 172.21.15.7:0/918459176 wait complete. 2024-06-10T01:06:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.434+0000 7f4d05bca740 1 -- start start 2024-06-10T01:06:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.434+0000 7f4d05bca740 1 Processor -- start 2024-06-10T01:06:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.435+0000 7f4d05bca740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564010bf4a00 0x564010c02680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.435+0000 7f4d05bca740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564010bf9b70 0x564010c02bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.435+0000 7f4d05bca740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564010c03b60 0x564010c03fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.435+0000 7f4d05bca740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc1af270 con 0x564010c03b60 2024-06-10T01:06:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.435+0000 7f4d05bca740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc1b06e0 con 0x564010bf9b70 2024-06-10T01:06:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.435+0000 7f4d05bca740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc1af940 con 0x564010bf4a00 2024-06-10T01:06:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.435+0000 7f4d05bca740 10 client.0 _ll_register_callbacks cb 0x7f4cf00b7b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.435+0000 7f4d01457640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564010c03b60 0x564010c03fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.435+0000 7f4d01457640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564010c03b60 0x564010c03fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35720/0 (socket says 172.21.15.7:35720) 2024-06-10T01:06:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.435+0000 7f4d01457640 1 -- 172.21.15.7:0/27783520 learned_addr learned my addr 172.21.15.7:0/27783520 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.435+0000 7f4cfbfff640 1 --2- 172.21.15.7:0/27783520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564010bf9b70 0x564010c02bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.435+0000 7f4d01457640 1 -- 172.21.15.7:0/27783520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564010bf4a00 msgr2=0x564010c02680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.435+0000 7f4d00c56640 1 --2- 172.21.15.7:0/27783520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564010bf4a00 0x564010c02680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.435+0000 7f4d01457640 1 --2- 172.21.15.7:0/27783520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564010bf4a00 0x564010c02680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.435+0000 7f4d01457640 1 -- 172.21.15.7:0/27783520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564010bf9b70 msgr2=0x564010c02bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.435+0000 7f4d01457640 1 --2- 172.21.15.7:0/27783520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564010bf9b70 0x564010c02bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.435+0000 7f4d01457640 1 -- 172.21.15.7:0/27783520 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564010c04520 con 0x564010c03b60 2024-06-10T01:06:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.435+0000 7f4d00c56640 1 --2- 172.21.15.7:0/27783520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564010bf4a00 0x564010c02680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:06:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.436+0000 7f4d01457640 1 --2- 172.21.15.7:0/27783520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564010c03b60 0x564010c03fe0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4cf400fdf0 tx=0x7f4cf4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9f7a1e0d986cb2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.436+0000 7f4cdbfff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:06:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.436+0000 7f4cdbfff640 1 -- 172.21.15.7:0/27783520 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4020bb0 con 0x564010c03b60 2024-06-10T01:06:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.436+0000 7f4d05bca740 1 -- 172.21.15.7:0/27783520 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cf00bbf60 con 0x564010c03b60 2024-06-10T01:06:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.436+0000 7f4cdbfff640 1 -- 172.21.15.7:0/27783520 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f4cf4020d50 con 0x564010c03b60 2024-06-10T01:06:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.436+0000 7f4cdbfff640 1 -- 172.21.15.7:0/27783520 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4cf4004b20 con 0x564010c03b60 2024-06-10T01:06:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.436+0000 7f4d05bca740 10 client.15615 fetch_fsmap learned FSMap version 27 2024-06-10T01:06:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.436+0000 7f4d05bca740 1 -- 172.21.15.7:0/27783520 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564010c74790 con 0x564010c03b60 2024-06-10T01:06:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.436+0000 7f4cdbfff640 1 -- 172.21.15.7:0/27783520 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cf00bbf60 con 0x564010c03b60 2024-06-10T01:06:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.437+0000 7f4cdbfff640 1 -- 172.21.15.7:0/27783520 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4cf400b040 con 0x564010c03b60 2024-06-10T01:06:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.437+0000 7f4d05bca740 10 client.15615 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:06:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.437+0000 7f4d05bca740 10 client.15615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:06:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.437+0000 7f4d05bca740 20 client.15615 populate_metadata read hostname 'smithi007' 2024-06-10T01:06:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.437+0000 7f4d05bca740 1 --2- 172.21.15.7:0/27783520 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564010c751a0 0x564010c775c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.437+0000 7f4d05bca740 4 client.15615 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:06:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.437+0000 7f4d05bca740 1 -- 172.21.15.7:0/27783520 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564010b64c20 con 0x564010c751a0 2024-06-10T01:06:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.437+0000 7f4d00c56640 1 --2- 172.21.15.7:0/27783520 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564010c751a0 0x564010c775c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.437+0000 7f4cdbfff640 10 client.15615 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.437+0000 7f4d00c56640 1 --2- 172.21.15.7:0/27783520 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564010c751a0 0x564010c775c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5f1e505e0221426 server_cookie=de7488aaefa2771a in_seq=0 out_seq=0 2024-06-10T01:06:56.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.444+0000 7f4cdbfff640 1 -- 172.21.15.7:0/27783520 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564010b64c20 con 0x564010c751a0 2024-06-10T01:06:56.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.444+0000 7f4cdbfff640 10 client.15615 handle_command_reply: tid=0 2024-06-10T01:06:56.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.445+0000 7f4d05bca740 1 client.15615 shutdown 2024-06-10T01:06:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.445+0000 7f4d05bca740 1 -- 172.21.15.7:0/27783520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564010c03b60 msgr2=0x564010c03fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.445+0000 7f4d05bca740 1 --2- 172.21.15.7:0/27783520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564010c03b60 0x564010c03fe0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4cf400fdf0 tx=0x7f4cf4004a40 comp rx=0 tx=0).stop 2024-06-10T01:06:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.445+0000 7f4d05bca740 1 -- 172.21.15.7:0/27783520 shutdown_connections 2024-06-10T01:06:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.445+0000 7f4d05bca740 1 --2- 172.21.15.7:0/27783520 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564010c751a0 0x564010c775c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.445+0000 7f4d05bca740 1 --2- 172.21.15.7:0/27783520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564010c03b60 0x564010c03fe0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.445+0000 7f4cdbfff640 0 client.15615 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.445+0000 7f4d05bca740 1 --2- 172.21.15.7:0/27783520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564010bf9b70 0x564010c02bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.445+0000 7f4d05bca740 1 --2- 172.21.15.7:0/27783520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564010bf4a00 0x564010c02680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.445+0000 7f4d05bca740 1 -- 172.21.15.7:0/27783520 >> 172.21.15.7:0/27783520 conn(0x564010bef680 msgr2=0x564010bf2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.445+0000 7f4d05bca740 1 -- 172.21.15.7:0/27783520 shutdown_connections 2024-06-10T01:06:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.445+0000 7f4d05bca740 1 -- 172.21.15.7:0/27783520 wait complete. 2024-06-10T01:06:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.445+0000 7f4d05bca740 20 client.15615 trim_cache size 0 max 16384 2024-06-10T01:06:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:06:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:06:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:06:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.446+0000 7f4d02eba640 1 -- 172.21.15.7:0/1060015922 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4cec0a2b80 msgr2=0x7f4cec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.446+0000 7f4d02eba640 1 --2- 172.21.15.7:0/1060015922 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4cec0a2b80 0x7f4cec0a5040 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4009fd0 tx=0x7f4cf4004ab0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.446+0000 7f4d02eba640 1 -- 172.21.15.7:0/1060015922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cfc1080a0 msgr2=0x7f4cfc19e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.446+0000 7f4d02eba640 1 --2- 172.21.15.7:0/1060015922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cfc1080a0 0x7f4cfc19e340 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0035040 tx=0x7f4cf0033d90 comp rx=0 tx=0).stop 2024-06-10T01:06:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.446+0000 7f4d02eba640 1 -- 172.21.15.7:0/1060015922 shutdown_connections 2024-06-10T01:06:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.446+0000 7f4d02eba640 1 --2- 172.21.15.7:0/1060015922 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4cec0a2b80 0x7f4cec0a5040 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.446+0000 7f4d02eba640 1 --2- 172.21.15.7:0/1060015922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cfc109c00 0x7f4cfc1a58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.446+0000 7f4d02eba640 1 --2- 172.21.15.7:0/1060015922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cfc109240 0x7f4cfc19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.446+0000 7f4d02eba640 1 --2- 172.21.15.7:0/1060015922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cfc1080a0 0x7f4cfc19e340 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.446+0000 7f4d02eba640 1 -- 172.21.15.7:0/1060015922 >> 172.21.15.7:0/1060015922 conn(0x7f4cfc076ff0 msgr2=0x7f4cfc10c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.446+0000 7f4d02eba640 1 -- 172.21.15.7:0/1060015922 shutdown_connections 2024-06-10T01:06:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:06:56.446+0000 7f4d02eba640 1 -- 172.21.15.7:0/1060015922 wait complete. 2024-06-10T01:06:56.635 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:56 smithi007 ceph-mon[32398]: scrub queued for path: / 2024-06-10T01:06:56.635 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:56 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:06:56.635 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:56 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:06:56.635 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:56 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:06:56.677 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:06:56.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:56 smithi175 ceph-mon[36636]: scrub queued for path: / 2024-06-10T01:06:56.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:56 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:06:56.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:56 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:06:56.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:56 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:06:56.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:56 smithi078 ceph-mon[36683]: scrub queued for path: / 2024-06-10T01:06:56.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:56 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:06:56.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:56 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:06:56.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:56 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:06:57.043 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:06:57.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:57 smithi007 ceph-mon[32398]: pgmap v459: 129 pgs: 129 active+clean; 671 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-10T01:06:57.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:57 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:06:57.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:57 smithi175 ceph-mon[36636]: pgmap v459: 129 pgs: 129 active+clean; 671 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-10T01:06:57.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:57 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:06:57.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:57 smithi078 ceph-mon[36683]: pgmap v459: 129 pgs: 129 active+clean; 671 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-10T01:06:57.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:57 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:06:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.894+0000 7f54f5778700 1 -- 172.21.15.7:0/3529016557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f006b3c0 msgr2=0x7f54f010f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.894+0000 7f54f5778700 1 --2- 172.21.15.7:0/3529016557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f006b3c0 0x7f54f010f310 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f54e800b0a0 tx=0x7f54e800b3b0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.894+0000 7f54f5778700 1 -- 172.21.15.7:0/3529016557 shutdown_connections 2024-06-10T01:06:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.894+0000 7f54f5778700 1 --2- 172.21.15.7:0/3529016557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f006b3c0 0x7f54f010f310 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.894+0000 7f54f5778700 1 --2- 172.21.15.7:0/3529016557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54f01084f0 0x7f54f01088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.894+0000 7f54f5778700 1 --2- 172.21.15.7:0/3529016557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f006aa00 0x7f54f006ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.895+0000 7f54f5778700 1 -- 172.21.15.7:0/3529016557 >> 172.21.15.7:0/3529016557 conn(0x7f54f0078300 msgr2=0x7f54f0078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:58.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.895+0000 7f54f5778700 1 -- 172.21.15.7:0/3529016557 shutdown_connections 2024-06-10T01:06:58.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.895+0000 7f54f5778700 1 -- 172.21.15.7:0/3529016557 wait complete. 2024-06-10T01:06:58.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.896+0000 7f54f5778700 1 Processor -- start 2024-06-10T01:06:58.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.896+0000 7f54f5778700 1 -- start start 2024-06-10T01:06:58.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.896+0000 7f54f5778700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f006aa00 0x7f54f019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:58.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.896+0000 7f54f5778700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54f006b3c0 0x7f54f019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:58.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.896+0000 7f54f5778700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f01084f0 0x7f54f019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:58.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.896+0000 7f54f5778700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f54f01127a0 con 0x7f54f006aa00 2024-06-10T01:06:58.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.896+0000 7f54f5778700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f54f0112620 con 0x7f54f006b3c0 2024-06-10T01:06:58.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.896+0000 7f54f5778700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f54f0112920 con 0x7f54f01084f0 2024-06-10T01:06:58.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.896+0000 7f54eeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f006aa00 0x7f54f019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:58.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.896+0000 7f54eeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f006aa00 0x7f54f019ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35740/0 (socket says 172.21.15.7:35740) 2024-06-10T01:06:58.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.896+0000 7f54eeffd700 1 -- 172.21.15.7:0/2731455971 learned_addr learned my addr 172.21.15.7:0/2731455971 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:58.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.896+0000 7f54ee7fc700 1 --2- 172.21.15.7:0/2731455971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54f006b3c0 0x7f54f019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:58.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.897+0000 7f54ee7fc700 1 -- 172.21.15.7:0/2731455971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f01084f0 msgr2=0x7f54f019f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:58.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.897+0000 7f54ee7fc700 1 --2- 172.21.15.7:0/2731455971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f01084f0 0x7f54f019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.897+0000 7f54ee7fc700 1 -- 172.21.15.7:0/2731455971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f006aa00 msgr2=0x7f54f019ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:58.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.897+0000 7f54ee7fc700 1 --2- 172.21.15.7:0/2731455971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f006aa00 0x7f54f019ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.897+0000 7f54ee7fc700 1 -- 172.21.15.7:0/2731455971 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f019fe80 con 0x7f54f006b3c0 2024-06-10T01:06:58.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.897+0000 7f54eeffd700 1 --2- 172.21.15.7:0/2731455971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f006aa00 0x7f54f019ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:58.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.897+0000 7f54ee7fc700 1 --2- 172.21.15.7:0/2731455971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54f006b3c0 0x7f54f019b3b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f54e400d750 tx=0x7f54e400da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:58.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.897+0000 7f54d7fff700 1 -- 172.21.15.7:0/2731455971 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4016070 con 0x7f54f006b3c0 2024-06-10T01:06:58.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.898+0000 7f54f5778700 1 -- 172.21.15.7:0/2731455971 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f01a0170 con 0x7f54f006b3c0 2024-06-10T01:06:58.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.898+0000 7f54d7fff700 1 -- 172.21.15.7:0/2731455971 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54e4004d10 con 0x7f54f006b3c0 2024-06-10T01:06:58.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.898+0000 7f54d7fff700 1 -- 172.21.15.7:0/2731455971 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e400f690 con 0x7f54f006b3c0 2024-06-10T01:06:58.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.898+0000 7f54f5778700 1 -- 172.21.15.7:0/2731455971 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f01a79f0 con 0x7f54f006b3c0 2024-06-10T01:06:58.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.901+0000 7f54d7fff700 1 -- 172.21.15.7:0/2731455971 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f54e4004750 con 0x7f54f006b3c0 2024-06-10T01:06:58.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.901+0000 7f54ffffbb80 1 Processor -- start 2024-06-10T01:06:58.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.902+0000 7f54ffffbb80 1 -- start start 2024-06-10T01:06:58.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.902+0000 7f54ffffbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f8fb7e770 0x559f8fb7eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:58.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.902+0000 7f54ffffbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f8fb7f090 0x559f8fb78a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:58.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.902+0000 7f54ffffbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f8fb78fb0 0x559f8fb793d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:58.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.902+0000 7f54ffffbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559f8fa62f60 con 0x559f8fb7f090 2024-06-10T01:06:58.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.902+0000 7f54ffffbb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559f8fac1850 con 0x559f8fb78fb0 2024-06-10T01:06:58.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.902+0000 7f54ffffbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559f8fa29460 con 0x559f8fb7e770 2024-06-10T01:06:58.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.902+0000 7f54eeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f8fb7f090 0x559f8fb78a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:58.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.902+0000 7f54eeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f8fb7f090 0x559f8fb78a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35750/0 (socket says 172.21.15.7:35750) 2024-06-10T01:06:58.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.902+0000 7f54eeffd700 1 -- 172.21.15.7:0/20471734 learned_addr learned my addr 172.21.15.7:0/20471734 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:58.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.903+0000 7f54d7fff700 1 --2- 172.21.15.7:0/2731455971 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f54d80a2d20 0x7f54d80a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:58.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.903+0000 7f54eeffd700 1 -- 172.21.15.7:0/20471734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f8fb7e770 msgr2=0x559f8fb7eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:58.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.903+0000 7f54eeffd700 1 --2- 172.21.15.7:0/20471734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f8fb7e770 0x559f8fb7eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.903+0000 7f54eeffd700 1 -- 172.21.15.7:0/20471734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f8fb78fb0 msgr2=0x559f8fb793d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T01:06:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.903+0000 7f54eeffd700 1 --2- 172.21.15.7:0/20471734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f8fb78fb0 0x559f8fb793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.903+0000 7f54eeffd700 1 -- 172.21.15.7:0/20471734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f8fa108d0 con 0x559f8fb7f090 2024-06-10T01:06:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.903+0000 7f54d7fff700 1 -- 172.21.15.7:0/2731455971 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f54e40cf080 con 0x7f54f006b3c0 2024-06-10T01:06:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.903+0000 7f54ef7fe700 1 --2- 172.21.15.7:0/2731455971 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f54d80a2d20 0x7f54d80a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.903+0000 7f54eeffd700 1 --2- 172.21.15.7:0/20471734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f8fb7f090 0x559f8fb78a70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f54e00105e0 tx=0x7f54e00108f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93be746b2aa2cb05 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.903+0000 7f54d57fa700 1 -- 172.21.15.7:0/20471734 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0011700 con 0x559f8fb7f090 2024-06-10T01:06:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.903+0000 7f54d57fa700 1 -- 172.21.15.7:0/20471734 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54e0011d80 con 0x559f8fb7f090 2024-06-10T01:06:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.904+0000 7f54d57fa700 1 -- 172.21.15.7:0/20471734 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0019e00 con 0x559f8fb7f090 2024-06-10T01:06:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.905+0000 7f54ef7fe700 1 --2- 172.21.15.7:0/2731455971 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f54d80a2d20 0x7f54d80a51e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009850 tx=0x7f54e8009780 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.905+0000 7f54ffffbb80 1 -- 172.21.15.7:0/20471734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f8fb7f090 msgr2=0x559f8fb78a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.905+0000 7f54ffffbb80 1 --2- 172.21.15.7:0/20471734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f8fb7f090 0x559f8fb78a70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f54e00105e0 tx=0x7f54e00108f0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.905+0000 7f54ffffbb80 1 -- 172.21.15.7:0/20471734 shutdown_connections 2024-06-10T01:06:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.905+0000 7f54ffffbb80 1 --2- 172.21.15.7:0/20471734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f8fb78fb0 0x559f8fb793d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.905+0000 7f54ffffbb80 1 --2- 172.21.15.7:0/20471734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f8fb7e770 0x559f8fb7eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.905+0000 7f54ffffbb80 1 --2- 172.21.15.7:0/20471734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f8fb7f090 0x559f8fb78a70 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.905+0000 7f54ffffbb80 1 -- 172.21.15.7:0/20471734 >> 172.21.15.7:0/20471734 conn(0x559f8fa9d080 msgr2=0x559f8fb6b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:58.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.905+0000 7f54ffffbb80 1 -- 172.21.15.7:0/20471734 shutdown_connections 2024-06-10T01:06:58.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.905+0000 7f54ffffbb80 1 -- 172.21.15.7:0/20471734 wait complete. 2024-06-10T01:06:58.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.905+0000 7f54ffffbb80 1 -- start start 2024-06-10T01:06:58.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.905+0000 7f54ffffbb80 1 Processor -- start 2024-06-10T01:06:58.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.906+0000 7f54ffffbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f8fb78fb0 0x559f8fb84e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:58.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.906+0000 7f54ffffbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f8fb7e770 0x559f8fb82140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:58.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.906+0000 7f54ffffbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f8fb7f090 0x559f8fb82680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:58.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.906+0000 7f54ef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f8fb7f090 0x559f8fb82680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:58.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.906+0000 7f54ef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f8fb7f090 0x559f8fb82680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35752/0 (socket says 172.21.15.7:35752) 2024-06-10T01:06:58.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.906+0000 7f54ef7fe700 1 -- 172.21.15.7:0/3026568309 learned_addr learned my addr 172.21.15.7:0/3026568309 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:58.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.906+0000 7f54ee7fc700 1 --2- 172.21.15.7:0/3026568309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f8fb7e770 0x559f8fb82140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:58.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.906+0000 7f54ffffbb80 1 -- 172.21.15.7:0/3026568309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559f8fa61fc0 con 0x559f8fb7f090 2024-06-10T01:06:58.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.906+0000 7f54ffffbb80 1 -- 172.21.15.7:0/3026568309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559f8fabd810 con 0x559f8fb78fb0 2024-06-10T01:06:58.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.906+0000 7f54ffffbb80 1 -- 172.21.15.7:0/3026568309 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559f8fb85a70 con 0x559f8fb7e770 2024-06-10T01:06:58.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.906+0000 7f54eeffd700 1 --2- 172.21.15.7:0/3026568309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f8fb78fb0 0x559f8fb84e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:58.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.906+0000 7f54eeffd700 1 -- 172.21.15.7:0/3026568309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f8fb7e770 msgr2=0x559f8fb82140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:58.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.906+0000 7f54eeffd700 1 --2- 172.21.15.7:0/3026568309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f8fb7e770 0x559f8fb82140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.906+0000 7f54eeffd700 1 -- 172.21.15.7:0/3026568309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f8fb7f090 msgr2=0x559f8fb82680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:58.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.906+0000 7f54eeffd700 1 --2- 172.21.15.7:0/3026568309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f8fb7f090 0x559f8fb82680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.906+0000 7f54eeffd700 1 -- 172.21.15.7:0/3026568309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f8fa87750 con 0x559f8fb78fb0 2024-06-10T01:06:58.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.906+0000 7f54ef7fe700 1 --2- 172.21.15.7:0/3026568309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f8fb7f090 0x559f8fb82680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:58.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.907+0000 7f54ffffbb80 10 client.0 _ll_register_callbacks cb 0x559f8fa575b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:58.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.907+0000 7f54ffffbb80 1 -- 172.21.15.7:0/3026568309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54e0011d80 con 0x559f8fb78fb0 2024-06-10T01:06:58.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.907+0000 7f54eeffd700 1 --2- 172.21.15.7:0/3026568309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f8fb78fb0 0x559f8fb84e40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f54e000c010 tx=0x7f54e0019870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ce1a23caf40370a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:58.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.907+0000 7f54d2ff5700 10 client.25241 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:06:58.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.907+0000 7f54d2ff5700 1 -- 172.21.15.7:0/3026568309 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0038030 con 0x559f8fb78fb0 2024-06-10T01:06:58.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.907+0000 7f54d2ff5700 1 -- 172.21.15.7:0/3026568309 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f54e0019bf0 con 0x559f8fb78fb0 2024-06-10T01:06:58.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.907+0000 7f54d2ff5700 1 -- 172.21.15.7:0/3026568309 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f54e001f040 con 0x559f8fb78fb0 2024-06-10T01:06:58.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.907+0000 7f54ffffbb80 10 client.25241 fetch_fsmap learned FSMap version 27 2024-06-10T01:06:58.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.907+0000 7f54ffffbb80 1 -- 172.21.15.7:0/3026568309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=27}) -- 0x559f8fb6df20 con 0x559f8fb78fb0 2024-06-10T01:06:58.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.908+0000 7f54d2ff5700 1 -- 172.21.15.7:0/3026568309 <== mon.2 v2:172.21.15.78:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f54e0005040 con 0x559f8fb78fb0 2024-06-10T01:06:58.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.908+0000 7f54ffffbb80 10 client.25241 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:06:58.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.908+0000 7f54ffffbb80 10 client.25241 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:06:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.908+0000 7f54ffffbb80 20 client.25241 populate_metadata read hostname 'smithi007' 2024-06-10T01:06:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.908+0000 7f54ffffbb80 1 --2- 172.21.15.7:0/3026568309 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559f8fb70780 0x559f8fb72c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.908+0000 7f54ffffbb80 4 client.25241 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:06:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.908+0000 7f54ffffbb80 1 -- 172.21.15.7:0/3026568309 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f8faced50 con 0x559f8fb70780 2024-06-10T01:06:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.910+0000 7f54ee7fc700 1 --2- 172.21.15.7:0/3026568309 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559f8fb70780 0x559f8fb72c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:58.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.911+0000 7f54ee7fc700 1 --2- 172.21.15.7:0/3026568309 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559f8fb70780 0x559f8fb72c40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b1d13bed7877dd server_cookie=fa84baaed5e956b1 in_seq=0 out_seq=0 2024-06-10T01:06:58.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.911+0000 7f54d2ff5700 10 client.25241 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:58.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.912+0000 7f54d2ff5700 1 -- 172.21.15.7:0/3026568309 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559f8faced50 con 0x559f8fb70780 2024-06-10T01:06:58.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.912+0000 7f54d2ff5700 10 client.25241 handle_command_reply: tid=0 2024-06-10T01:06:58.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.912+0000 7f54ffffbb80 1 client.25241 shutdown 2024-06-10T01:06:58.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.912+0000 7f54ffffbb80 1 -- 172.21.15.7:0/3026568309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f8fb78fb0 msgr2=0x559f8fb84e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:58.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.912+0000 7f54ffffbb80 1 --2- 172.21.15.7:0/3026568309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f8fb78fb0 0x559f8fb84e40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f54e000c010 tx=0x7f54e0019870 comp rx=0 tx=0).stop 2024-06-10T01:06:58.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.912+0000 7f54ffffbb80 1 -- 172.21.15.7:0/3026568309 shutdown_connections 2024-06-10T01:06:58.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.912+0000 7f54ffffbb80 1 --2- 172.21.15.7:0/3026568309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f8fb7f090 0x559f8fb82680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.912+0000 7f54ffffbb80 1 --2- 172.21.15.7:0/3026568309 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559f8fb70780 0x559f8fb72c40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.912+0000 7f54ffffbb80 1 --2- 172.21.15.7:0/3026568309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f8fb78fb0 0x559f8fb84e40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.912+0000 7f54ffffbb80 1 --2- 172.21.15.7:0/3026568309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f8fb7e770 0x559f8fb82140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.912+0000 7f54ffffbb80 1 -- 172.21.15.7:0/3026568309 >> 172.21.15.7:0/3026568309 conn(0x559f8fa9d080 msgr2=0x559f8fb6a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:58.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.912+0000 7f54d2ff5700 0 client.25241 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:58.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.912+0000 7f54ffffbb80 1 -- 172.21.15.7:0/3026568309 shutdown_connections 2024-06-10T01:06:58.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.912+0000 7f54ffffbb80 1 -- 172.21.15.7:0/3026568309 wait complete. 2024-06-10T01:06:58.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.912+0000 7f54ffffbb80 20 client.25241 trim_cache size 0 max 16384 2024-06-10T01:06:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.953+0000 7f54ffffbb80 1 Processor -- start 2024-06-10T01:06:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.954+0000 7f54ffffbb80 1 -- start start 2024-06-10T01:06:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.954+0000 7f54ffffbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f8fb78fb0 0x559f8fb7fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.954+0000 7f54ffffbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f8fb7e770 0x559f8fb78550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.954+0000 7f54ffffbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f8fb7f090 0x559f8fb84980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.954+0000 7f54ffffbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f54f01aea70 con 0x559f8fb7f090 2024-06-10T01:06:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.954+0000 7f54ffffbb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f54f01ac300 con 0x559f8fb78fb0 2024-06-10T01:06:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.954+0000 7f54ffffbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f54f01aa150 con 0x559f8fb7e770 2024-06-10T01:06:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.954+0000 7f54ee7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f8fb7e770 0x559f8fb78550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.954+0000 7f54ef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f8fb7f090 0x559f8fb84980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.954+0000 7f54ef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f8fb7f090 0x559f8fb84980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35760/0 (socket says 172.21.15.7:35760) 2024-06-10T01:06:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.954+0000 7f54ee7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f8fb7e770 0x559f8fb78550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:41876/0 (socket says 172.21.15.7:41876) 2024-06-10T01:06:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.954+0000 7f54ef7fe700 1 -- 172.21.15.7:0/871703 learned_addr learned my addr 172.21.15.7:0/871703 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.955+0000 7f54eeffd700 1 --2- 172.21.15.7:0/871703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f8fb78fb0 0x559f8fb7fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.955+0000 7f54ee7fc700 1 -- 172.21.15.7:0/871703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f8fb78fb0 msgr2=0x559f8fb7fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.955+0000 7f54ee7fc700 1 --2- 172.21.15.7:0/871703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f8fb78fb0 0x559f8fb7fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.955+0000 7f54ee7fc700 1 -- 172.21.15.7:0/871703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f8fb7f090 msgr2=0x559f8fb84980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.955+0000 7f54ee7fc700 1 --2- 172.21.15.7:0/871703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f8fb7f090 0x559f8fb84980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.955+0000 7f54ee7fc700 1 -- 172.21.15.7:0/871703 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f8fb83080 con 0x559f8fb7e770 2024-06-10T01:06:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.955+0000 7f54ee7fc700 1 --2- 172.21.15.7:0/871703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f8fb7e770 0x559f8fb78550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f54e40cc7a0 tx=0x7f54e4004620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adfb3dcadb417d55 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:58.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.955+0000 7f54d4ff9700 1 -- 172.21.15.7:0/871703 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e409f8b0 con 0x559f8fb7e770 2024-06-10T01:06:58.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.955+0000 7f54d4ff9700 1 -- 172.21.15.7:0/871703 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54e40a4170 con 0x559f8fb7e770 2024-06-10T01:06:58.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.955+0000 7f54d4ff9700 1 -- 172.21.15.7:0/871703 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e40cc8b0 con 0x559f8fb7e770 2024-06-10T01:06:58.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.956+0000 7f54ffffbb80 1 -- 172.21.15.7:0/871703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f8fb7e770 msgr2=0x559f8fb78550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:58.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.956+0000 7f54ffffbb80 1 --2- 172.21.15.7:0/871703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f8fb7e770 0x559f8fb78550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f54e40cc7a0 tx=0x7f54e4004620 comp rx=0 tx=0).stop 2024-06-10T01:06:58.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.957+0000 7f54ffffbb80 1 -- 172.21.15.7:0/871703 shutdown_connections 2024-06-10T01:06:58.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.957+0000 7f54ffffbb80 1 --2- 172.21.15.7:0/871703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f8fb7f090 0x559f8fb84980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.957+0000 7f54ffffbb80 1 --2- 172.21.15.7:0/871703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f8fb78fb0 0x559f8fb7fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.957+0000 7f54ffffbb80 1 --2- 172.21.15.7:0/871703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f8fb7e770 0x559f8fb78550 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.957+0000 7f54ffffbb80 1 -- 172.21.15.7:0/871703 >> 172.21.15.7:0/871703 conn(0x559f8fa9d080 msgr2=0x559f8fb7d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:58.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.957+0000 7f54ffffbb80 1 -- 172.21.15.7:0/871703 shutdown_connections 2024-06-10T01:06:58.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.957+0000 7f54ffffbb80 1 -- 172.21.15.7:0/871703 wait complete. 2024-06-10T01:06:58.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.957+0000 7f54ffffbb80 1 -- start start 2024-06-10T01:06:58.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.957+0000 7f54ffffbb80 1 Processor -- start 2024-06-10T01:06:58.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.958+0000 7f54ffffbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f8fb78fb0 0x559f8fb94b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:58.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.958+0000 7f54ffffbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f8fb7e770 0x559f8fb90ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:58.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.958+0000 7f54ffffbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f8fb7f090 0x559f8fb91440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:58.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.958+0000 7f54ffffbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f54f01aa7c0 con 0x559f8fb78fb0 2024-06-10T01:06:58.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.958+0000 7f54ffffbb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f54f01abc30 con 0x559f8fb7f090 2024-06-10T01:06:58.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.958+0000 7f54ffffbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f54f01aae90 con 0x559f8fb7e770 2024-06-10T01:06:58.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.958+0000 7f54ffffbb80 10 client.0 _ll_register_callbacks cb 0x559f8fa575b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:58.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.958+0000 7f54ef7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f8fb7f090 0x559f8fb91440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:58.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.958+0000 7f54eeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f8fb78fb0 0x559f8fb94b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:58.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.958+0000 7f54eeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f8fb78fb0 0x559f8fb94b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35764/0 (socket says 172.21.15.7:35764) 2024-06-10T01:06:58.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.958+0000 7f54eeffd700 1 -- 172.21.15.7:0/1391884595 learned_addr learned my addr 172.21.15.7:0/1391884595 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:06:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.959+0000 7f54ee7fc700 1 --2- 172.21.15.7:0/1391884595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f8fb7e770 0x559f8fb90ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.959+0000 7f54eeffd700 1 -- 172.21.15.7:0/1391884595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f8fb7e770 msgr2=0x559f8fb90ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.959+0000 7f54eeffd700 1 --2- 172.21.15.7:0/1391884595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f8fb7e770 0x559f8fb90ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.959+0000 7f54eeffd700 1 -- 172.21.15.7:0/1391884595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f8fb7f090 msgr2=0x559f8fb91440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.959+0000 7f54eeffd700 1 --2- 172.21.15.7:0/1391884595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f8fb7f090 0x559f8fb91440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.959+0000 7f54eeffd700 1 -- 172.21.15.7:0/1391884595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f8fb91c80 con 0x559f8fb78fb0 2024-06-10T01:06:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.960+0000 7f54eeffd700 1 --2- 172.21.15.7:0/1391884595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f8fb78fb0 0x559f8fb94b10 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f54e0019500 tx=0x7f54e0032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1517e51f56ca0e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.960+0000 7f54d2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:06:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.960+0000 7f54d2ff5700 1 -- 172.21.15.7:0/1391884595 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0038040 con 0x559f8fb78fb0 2024-06-10T01:06:58.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.960+0000 7f54d2ff5700 1 -- 172.21.15.7:0/1391884595 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f54e00196c0 con 0x559f8fb78fb0 2024-06-10T01:06:58.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.960+0000 7f54ffffbb80 1 -- 172.21.15.7:0/1391884595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54e40a4170 con 0x559f8fb78fb0 2024-06-10T01:06:58.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.960+0000 7f54d2ff5700 1 -- 172.21.15.7:0/1391884595 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f54e0005660 con 0x559f8fb78fb0 2024-06-10T01:06:58.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.960+0000 7f54ffffbb80 10 client.15633 fetch_fsmap learned FSMap version 27 2024-06-10T01:06:58.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.961+0000 7f54ffffbb80 1 -- 172.21.15.7:0/1391884595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559f8fbf43e0 con 0x559f8fb78fb0 2024-06-10T01:06:58.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.961+0000 7f54d2ff5700 1 -- 172.21.15.7:0/1391884595 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54e40a4170 con 0x559f8fb78fb0 2024-06-10T01:06:58.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.961+0000 7f54d2ff5700 1 -- 172.21.15.7:0/1391884595 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f54e0003790 con 0x559f8fb78fb0 2024-06-10T01:06:58.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.961+0000 7f54ffffbb80 10 client.15633 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:06:58.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.961+0000 7f54ffffbb80 10 client.15633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:06:58.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.961+0000 7f54ffffbb80 20 client.15633 populate_metadata read hostname 'smithi007' 2024-06-10T01:06:58.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.962+0000 7f54ffffbb80 1 --2- 172.21.15.7:0/1391884595 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559f8fbf4a10 0x559f8fbf6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:58.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.962+0000 7f54ffffbb80 4 client.15633 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:06:58.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.962+0000 7f54ffffbb80 1 -- 172.21.15.7:0/1391884595 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559f8fac92d0 con 0x559f8fbf4a10 2024-06-10T01:06:58.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.962+0000 7f54ee7fc700 1 --2- 172.21.15.7:0/1391884595 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559f8fbf4a10 0x559f8fbf6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:58.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.962+0000 7f54d2ff5700 10 client.15633 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:58.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.962+0000 7f54ee7fc700 1 --2- 172.21.15.7:0/1391884595 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559f8fbf4a10 0x559f8fbf6e30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bdcd65f8f8b4186 server_cookie=ac46111226ffe31a in_seq=0 out_seq=0 2024-06-10T01:06:58.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.963+0000 7f54d2ff5700 1 -- 172.21.15.7:0/1391884595 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559f8fac92d0 con 0x559f8fbf4a10 2024-06-10T01:06:58.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.963+0000 7f54d2ff5700 10 client.15633 handle_command_reply: tid=0 2024-06-10T01:06:58.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.963+0000 7f54ffffbb80 1 client.15633 shutdown 2024-06-10T01:06:58.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.963+0000 7f54ffffbb80 1 -- 172.21.15.7:0/1391884595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f8fb78fb0 msgr2=0x559f8fb94b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:58.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.963+0000 7f54ffffbb80 1 --2- 172.21.15.7:0/1391884595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f8fb78fb0 0x559f8fb94b10 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f54e0019500 tx=0x7f54e0032000 comp rx=0 tx=0).stop 2024-06-10T01:06:58.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.963+0000 7f54ffffbb80 1 -- 172.21.15.7:0/1391884595 shutdown_connections 2024-06-10T01:06:58.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.963+0000 7f54ffffbb80 1 --2- 172.21.15.7:0/1391884595 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559f8fbf4a10 0x559f8fbf6e30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.963+0000 7f54d2ff5700 0 client.15633 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:06:58.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.963+0000 7f54ffffbb80 1 --2- 172.21.15.7:0/1391884595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f8fb7f090 0x559f8fb91440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.963+0000 7f54ffffbb80 1 --2- 172.21.15.7:0/1391884595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f8fb78fb0 0x559f8fb94b10 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.963+0000 7f54ffffbb80 1 --2- 172.21.15.7:0/1391884595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f8fb7e770 0x559f8fb90ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.963+0000 7f54ffffbb80 1 -- 172.21.15.7:0/1391884595 >> 172.21.15.7:0/1391884595 conn(0x559f8fa9d080 msgr2=0x559f8fb7fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:58.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.964+0000 7f54ffffbb80 1 -- 172.21.15.7:0/1391884595 shutdown_connections 2024-06-10T01:06:58.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.964+0000 7f54ffffbb80 1 -- 172.21.15.7:0/1391884595 wait complete. 2024-06-10T01:06:58.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.964+0000 7f54ffffbb80 20 client.15633 trim_cache size 0 max 16384 2024-06-10T01:06:58.963 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:06:58.963 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-10T01:06:58.963 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-10T01:06:58.964 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:06:58.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.965+0000 7f54f5778700 1 -- 172.21.15.7:0/2731455971 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f54d80a2d20 msgr2=0x7f54d80a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:58.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.965+0000 7f54f5778700 1 --2- 172.21.15.7:0/2731455971 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f54d80a2d20 0x7f54d80a51e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009850 tx=0x7f54e8009780 comp rx=0 tx=0).stop 2024-06-10T01:06:58.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.965+0000 7f54f5778700 1 -- 172.21.15.7:0/2731455971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54f006b3c0 msgr2=0x7f54f019b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:58.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.965+0000 7f54f5778700 1 --2- 172.21.15.7:0/2731455971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54f006b3c0 0x7f54f019b3b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f54e400d750 tx=0x7f54e400da60 comp rx=0 tx=0).stop 2024-06-10T01:06:58.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.965+0000 7f54f5778700 1 -- 172.21.15.7:0/2731455971 shutdown_connections 2024-06-10T01:06:58.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.965+0000 7f54f5778700 1 --2- 172.21.15.7:0/2731455971 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f54d80a2d20 0x7f54d80a51e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.965+0000 7f54f5778700 1 --2- 172.21.15.7:0/2731455971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f01084f0 0x7f54f019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.965+0000 7f54f5778700 1 --2- 172.21.15.7:0/2731455971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f006aa00 0x7f54f019ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:58.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.965+0000 7f54f5778700 1 --2- 172.21.15.7:0/2731455971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54f006b3c0 0x7f54f019b3b0 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f54e400d750 tx=0x7f54e400da60 comp rx=0 tx=0).stop 2024-06-10T01:06:58.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.965+0000 7f54f5778700 1 -- 172.21.15.7:0/2731455971 >> 172.21.15.7:0/2731455971 conn(0x7f54f0078300 msgr2=0x7f54f010d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:58.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.966+0000 7f54f5778700 1 -- 172.21.15.7:0/2731455971 shutdown_connections 2024-06-10T01:06:58.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:06:58.966+0000 7f54f5778700 1 -- 172.21.15.7:0/2731455971 wait complete. 2024-06-10T01:06:59.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:06:59 smithi007 ceph-mon[32398]: pgmap v460: 129 pgs: 129 active+clean; 671 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-06-10T01:06:59.700 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:06:59.701 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:06:59.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:06:59 smithi175 ceph-mon[36636]: pgmap v460: 129 pgs: 129 active+clean; 671 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-06-10T01:06:59.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:06:59 smithi078 ceph-mon[36683]: pgmap v460: 129 pgs: 129 active+clean; 671 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-06-10T01:06:59.956 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:01.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:01 smithi007 ceph-mon[32398]: pgmap v461: 129 pgs: 129 active+clean; 671 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-06-10T01:07:01.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:01 smithi175 ceph-mon[36636]: pgmap v461: 129 pgs: 129 active+clean; 671 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-06-10T01:07:01.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:01 smithi078 ceph-mon[36683]: pgmap v461: 129 pgs: 129 active+clean; 671 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-06-10T01:07:01.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.843+0000 7f035c4ce700 1 -- 172.21.15.7:0/2111261476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0354101d30 msgr2=0x7f035410e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:01.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.843+0000 7f035c4ce700 1 --2- 172.21.15.7:0/2111261476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0354101d30 0x7f035410e210 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f034c00b0a0 tx=0x7f034c00b3b0 comp rx=0 tx=0).stop 2024-06-10T01:07:01.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.843+0000 7f035c4ce700 1 -- 172.21.15.7:0/2111261476 shutdown_connections 2024-06-10T01:07:01.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.843+0000 7f035c4ce700 1 --2- 172.21.15.7:0/2111261476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0354101d30 0x7f035410e210 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:01.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.843+0000 7f035c4ce700 1 --2- 172.21.15.7:0/2111261476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03541073d0 0x7f03541077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:01.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.843+0000 7f035c4ce700 1 --2- 172.21.15.7:0/2111261476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0354101390 0x7f03541017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:01.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.843+0000 7f035c4ce700 1 -- 172.21.15.7:0/2111261476 >> 172.21.15.7:0/2111261476 conn(0x7f03540782c0 msgr2=0x7f03540786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:01.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.844+0000 7f035c4ce700 1 -- 172.21.15.7:0/2111261476 shutdown_connections 2024-06-10T01:07:01.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.844+0000 7f035c4ce700 1 -- 172.21.15.7:0/2111261476 wait complete. 2024-06-10T01:07:01.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.845+0000 7f035c4ce700 1 Processor -- start 2024-06-10T01:07:01.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.845+0000 7f035c4ce700 1 -- start start 2024-06-10T01:07:01.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.845+0000 7f035c4ce700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0354101390 0x7f035419d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:01.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.845+0000 7f035c4ce700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0354101d30 0x7f035419dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:01.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.845+0000 7f035c4ce700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03541073d0 0x7f0354197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:01.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.846+0000 7f035c4ce700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0354111670 con 0x7f0354101d30 2024-06-10T01:07:01.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.846+0000 7f035c4ce700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f03541114f0 con 0x7f0354101390 2024-06-10T01:07:01.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.846+0000 7f0359a69700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0354101d30 0x7f035419dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:01.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.846+0000 7f0359a69700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0354101d30 0x7f035419dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51412/0 (socket says 172.21.15.7:51412) 2024-06-10T01:07:01.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.846+0000 7f0359a69700 1 -- 172.21.15.7:0/547368228 learned_addr learned my addr 172.21.15.7:0/547368228 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:01.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.846+0000 7f035c4ce700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f03541117f0 con 0x7f03541073d0 2024-06-10T01:07:01.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.846+0000 7f035a26a700 1 --2- 172.21.15.7:0/547368228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0354101390 0x7f035419d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:01.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.846+0000 7f0359a69700 1 -- 172.21.15.7:0/547368228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03541073d0 msgr2=0x7f0354197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:01.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.846+0000 7f0359a69700 1 --2- 172.21.15.7:0/547368228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03541073d0 0x7f0354197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:01.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.846+0000 7f0359a69700 1 -- 172.21.15.7:0/547368228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0354101390 msgr2=0x7f035419d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:01.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.846+0000 7f0359a69700 1 --2- 172.21.15.7:0/547368228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0354101390 0x7f035419d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:01.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.846+0000 7f0359a69700 1 -- 172.21.15.7:0/547368228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0354197fd0 con 0x7f0354101d30 2024-06-10T01:07:01.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.846+0000 7f035a26a700 1 --2- 172.21.15.7:0/547368228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0354101390 0x7f035419d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:01.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.846+0000 7f0359a69700 1 --2- 172.21.15.7:0/547368228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0354101d30 0x7f035419dbf0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f034400d950 tx=0x7f034400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:01.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.847+0000 7f034b7fe700 1 -- 172.21.15.7:0/547368228 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0344016070 con 0x7f0354101d30 2024-06-10T01:07:01.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.847+0000 7f034b7fe700 1 -- 172.21.15.7:0/547368228 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0344009e20 con 0x7f0354101d30 2024-06-10T01:07:01.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.847+0000 7f034b7fe700 1 -- 172.21.15.7:0/547368228 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034400f620 con 0x7f0354101d30 2024-06-10T01:07:01.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.847+0000 7f035c4ce700 1 -- 172.21.15.7:0/547368228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03541982c0 con 0x7f0354101d30 2024-06-10T01:07:01.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.847+0000 7f035c4ce700 1 -- 172.21.15.7:0/547368228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03541a4580 con 0x7f0354101d30 2024-06-10T01:07:01.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.848+0000 7f035c4ce700 1 -- 172.21.15.7:0/547368228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0354069180 con 0x7f0354101d30 2024-06-10T01:07:01.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.850+0000 7f034b7fe700 1 -- 172.21.15.7:0/547368228 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0344019070 con 0x7f0354101d30 2024-06-10T01:07:01.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.851+0000 7f034b7fe700 1 --2- 172.21.15.7:0/547368228 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f03400a2ca0 0x7f03400a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:01.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.851+0000 7f035a26a700 1 --2- 172.21.15.7:0/547368228 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f03400a2ca0 0x7f03400a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:01.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.851+0000 7f034b7fe700 1 -- 172.21.15.7:0/547368228 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f03440cf060 con 0x7f0354101d30 2024-06-10T01:07:01.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.852+0000 7f035a26a700 1 --2- 172.21.15.7:0/547368228 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f03400a2ca0 0x7f03400a5160 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f03541989b0 tx=0x7f035000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:01.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:01.855+0000 7f034b7fe700 1 -- 172.21.15.7:0/547368228 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f034409bbe0 con 0x7f0354101d30 2024-06-10T01:07:02.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:02.109+0000 7f035c4ce700 1 -- 172.21.15.7:0/547368228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f035402dca0 con 0x7f0354101d30 2024-06-10T01:07:02.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:02.110+0000 7f034b7fe700 1 -- 172.21.15.7:0/547368228 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11855 (secure 0 0 0) 0x7f034409b4e0 con 0x7f0354101d30 2024-06-10T01:07:02.108 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:07:02.109 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:05:39.175358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/2784910673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2784910673},{"type":"v1","addr":"172.21.15.175:6835","nonce":2784910673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24434,24436,24451]},"id":1}]} 2024-06-10T01:07:02.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:02.113+0000 7f035c4ce700 1 -- 172.21.15.7:0/547368228 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f03400a2ca0 msgr2=0x7f03400a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:02.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:02.113+0000 7f035c4ce700 1 --2- 172.21.15.7:0/547368228 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f03400a2ca0 0x7f03400a5160 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f03541989b0 tx=0x7f035000a380 comp rx=0 tx=0).stop 2024-06-10T01:07:02.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:02.113+0000 7f035c4ce700 1 -- 172.21.15.7:0/547368228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0354101d30 msgr2=0x7f035419dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:02.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:02.113+0000 7f035c4ce700 1 --2- 172.21.15.7:0/547368228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0354101d30 0x7f035419dbf0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f034400d950 tx=0x7f034400dc60 comp rx=0 tx=0).stop 2024-06-10T01:07:02.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:02.113+0000 7f035c4ce700 1 -- 172.21.15.7:0/547368228 shutdown_connections 2024-06-10T01:07:02.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:02.113+0000 7f035c4ce700 1 --2- 172.21.15.7:0/547368228 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f03400a2ca0 0x7f03400a5160 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:02.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:02.113+0000 7f035c4ce700 1 --2- 172.21.15.7:0/547368228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03541073d0 0x7f0354197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:02.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:02.113+0000 7f035c4ce700 1 --2- 172.21.15.7:0/547368228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0354101390 0x7f035419d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:02.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:02.113+0000 7f035c4ce700 1 --2- 172.21.15.7:0/547368228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0354101d30 0x7f035419dbf0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:02.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:02.113+0000 7f035c4ce700 1 -- 172.21.15.7:0/547368228 >> 172.21.15.7:0/547368228 conn(0x7f03540782c0 msgr2=0x7f03540ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:02.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:02.114+0000 7f035c4ce700 1 -- 172.21.15.7:0/547368228 shutdown_connections 2024-06-10T01:07:02.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:02.114+0000 7f035c4ce700 1 -- 172.21.15.7:0/547368228 wait complete. 2024-06-10T01:07:02.112 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-06-10T01:07:02.392 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:02 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/547368228' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:07:02.707 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:02 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/547368228' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:07:02.737 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:07:02.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:02 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/547368228' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:07:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:03.485 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:03 smithi007 ceph-mon[32398]: pgmap v462: 129 pgs: 129 active+clean; 672 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-06-10T01:07:03.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:03 smithi175 ceph-mon[36636]: pgmap v462: 129 pgs: 129 active+clean; 672 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-06-10T01:07:03.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:03 smithi078 ceph-mon[36683]: pgmap v462: 129 pgs: 129 active+clean; 672 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-06-10T01:07:05.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.039+0000 7fd838912700 1 -- 172.21.15.7:0/3419825592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd834101390 msgr2=0x7fd8341017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:05.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.039+0000 7fd838912700 1 --2- 172.21.15.7:0/3419825592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd834101390 0x7fd8341017f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd828009990 tx=0x7fd828009ca0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.039+0000 7fd838912700 1 -- 172.21.15.7:0/3419825592 shutdown_connections 2024-06-10T01:07:05.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.039+0000 7fd838912700 1 --2- 172.21.15.7:0/3419825592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd834101d30 0x7fd83410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.039+0000 7fd838912700 1 --2- 172.21.15.7:0/3419825592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8341073d0 0x7fd8341077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.039+0000 7fd838912700 1 --2- 172.21.15.7:0/3419825592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd834101390 0x7fd8341017f0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.039+0000 7fd838912700 1 -- 172.21.15.7:0/3419825592 >> 172.21.15.7:0/3419825592 conn(0x7fd8340782c0 msgr2=0x7fd8340786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:05.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.040+0000 7fd838912700 1 -- 172.21.15.7:0/3419825592 shutdown_connections 2024-06-10T01:07:05.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.040+0000 7fd838912700 1 -- 172.21.15.7:0/3419825592 wait complete. 2024-06-10T01:07:05.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.041+0000 7fd838912700 1 Processor -- start 2024-06-10T01:07:05.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.041+0000 7fd838912700 1 -- start start 2024-06-10T01:07:05.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.041+0000 7fd838912700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd834101390 0x7fd834103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:05.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.041+0000 7fd838912700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd834101d30 0x7fd834103580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:05.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.041+0000 7fd838912700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8341073d0 0x7fd834103ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:05.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.041+0000 7fd838912700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd834111630 con 0x7fd8341073d0 2024-06-10T01:07:05.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.041+0000 7fd838912700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd8341114b0 con 0x7fd834101390 2024-06-10T01:07:05.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.041+0000 7fd838912700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd8341117b0 con 0x7fd834101d30 2024-06-10T01:07:05.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.041+0000 7fd832d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8341073d0 0x7fd834103ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:05.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.042+0000 7fd832d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8341073d0 0x7fd834103ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51436/0 (socket says 172.21.15.7:51436) 2024-06-10T01:07:05.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.042+0000 7fd832d9d700 1 -- 172.21.15.7:0/2205818503 learned_addr learned my addr 172.21.15.7:0/2205818503 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:05.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.042+0000 7fd831d9b700 1 --2- 172.21.15.7:0/2205818503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd834101d30 0x7fd834103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:05.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.042+0000 7fd83259c700 1 --2- 172.21.15.7:0/2205818503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd834101390 0x7fd834103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:05.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.042+0000 7fd831d9b700 1 -- 172.21.15.7:0/2205818503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd834101390 msgr2=0x7fd834103040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:05.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.042+0000 7fd831d9b700 1 --2- 172.21.15.7:0/2205818503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd834101390 0x7fd834103040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.042+0000 7fd831d9b700 1 -- 172.21.15.7:0/2205818503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8341073d0 msgr2=0x7fd834103ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:05.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.042+0000 7fd831d9b700 1 --2- 172.21.15.7:0/2205818503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8341073d0 0x7fd834103ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.042+0000 7fd831d9b700 1 -- 172.21.15.7:0/2205818503 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8341a70b0 con 0x7fd834101d30 2024-06-10T01:07:05.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.042+0000 7fd832d9d700 1 --2- 172.21.15.7:0/2205818503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8341073d0 0x7fd834103ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:05.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.042+0000 7fd831d9b700 1 --2- 172.21.15.7:0/2205818503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd834101d30 0x7fd834103580 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd8280059d0 tx=0x7fd8280047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:05.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.043+0000 7fd8237fe700 1 -- 172.21.15.7:0/2205818503 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd828027070 con 0x7fd834101d30 2024-06-10T01:07:05.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.043+0000 7fd838912700 1 -- 172.21.15.7:0/2205818503 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8341a7340 con 0x7fd834101d30 2024-06-10T01:07:05.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.043+0000 7fd8237fe700 1 -- 172.21.15.7:0/2205818503 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd828003740 con 0x7fd834101d30 2024-06-10T01:07:05.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.043+0000 7fd8237fe700 1 -- 172.21.15.7:0/2205818503 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd828022b50 con 0x7fd834101d30 2024-06-10T01:07:05.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.043+0000 7fd838912700 1 -- 172.21.15.7:0/2205818503 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8341a77f0 con 0x7fd834101d30 2024-06-10T01:07:05.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.045+0000 7fd8237fe700 1 -- 172.21.15.7:0/2205818503 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fd828022d10 con 0x7fd834101d30 2024-06-10T01:07:05.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.046+0000 7fd8237fe700 1 --2- 172.21.15.7:0/2205818503 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd81c0a2ca0 0x7fd81c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:05.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.046+0000 7fd8237fe700 1 -- 172.21.15.7:0/2205818503 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fd8280d0430 con 0x7fd834101d30 2024-06-10T01:07:05.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.047+0000 7fd83259c700 1 --2- 172.21.15.7:0/2205818503 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd81c0a2ca0 0x7fd81c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:05.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.047+0000 7fd83259c700 1 --2- 172.21.15.7:0/2205818503 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd81c0a2ca0 0x7fd81c0a5160 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd824007b60 tx=0x7fd8240073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:05.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.048+0000 7fd843195b80 1 Processor -- start 2024-06-10T01:07:05.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.048+0000 7fd843195b80 1 -- start start 2024-06-10T01:07:05.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.049+0000 7fd843195b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577263bbf60 0x5577263bc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:05.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.049+0000 7fd843195b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577263bc880 0x5577263b6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:05.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.049+0000 7fd843195b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577263b67a0 0x5577263b6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:05.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.049+0000 7fd843195b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5577262a8f60 con 0x5577263b67a0 2024-06-10T01:07:05.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.049+0000 7fd843195b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557726307850 con 0x5577263bc880 2024-06-10T01:07:05.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.049+0000 7fd843195b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55772626f460 con 0x5577263bbf60 2024-06-10T01:07:05.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.049+0000 7fd831d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577263bc880 0x5577263b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:05.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.049+0000 7fd831d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577263bc880 0x5577263b6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:36590/0 (socket says 172.21.15.7:36590) 2024-06-10T01:07:05.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.049+0000 7fd831d9b700 1 -- 172.21.15.7:0/2770447481 learned_addr learned my addr 172.21.15.7:0/2770447481 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:05.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.049+0000 7fd83259c700 1 --2- 172.21.15.7:0/2770447481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577263bbf60 0x5577263bc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:05.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.049+0000 7fd832d9d700 1 --2- 172.21.15.7:0/2770447481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577263b67a0 0x5577263b6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:05.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.049+0000 7fd831d9b700 1 -- 172.21.15.7:0/2770447481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577263bbf60 msgr2=0x5577263bc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:05.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.049+0000 7fd831d9b700 1 --2- 172.21.15.7:0/2770447481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577263bbf60 0x5577263bc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.049+0000 7fd831d9b700 1 -- 172.21.15.7:0/2770447481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577263b67a0 msgr2=0x5577263b6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:05.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.049+0000 7fd831d9b700 1 --2- 172.21.15.7:0/2770447481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577263b67a0 0x5577263b6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.049+0000 7fd831d9b700 1 -- 172.21.15.7:0/2770447481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577262568d0 con 0x5577263bc880 2024-06-10T01:07:05.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.049+0000 7fd832d9d700 1 --2- 172.21.15.7:0/2770447481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577263b67a0 0x5577263b6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:05.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.050+0000 7fd831d9b700 1 --2- 172.21.15.7:0/2770447481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577263bc880 0x5577263b6260 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd828098da0 tx=0x7fd828005a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bcb715d23d1fbb1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:05.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.050+0000 7fd820ff9700 1 -- 172.21.15.7:0/2770447481 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8280043d0 con 0x5577263bc880 2024-06-10T01:07:05.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.050+0000 7fd820ff9700 1 -- 172.21.15.7:0/2770447481 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd82809d040 con 0x5577263bc880 2024-06-10T01:07:05.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.050+0000 7fd820ff9700 1 -- 172.21.15.7:0/2770447481 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8280d0110 con 0x5577263bc880 2024-06-10T01:07:05.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.051+0000 7fd843195b80 1 -- 172.21.15.7:0/2770447481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577263bc880 msgr2=0x5577263b6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:05.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.051+0000 7fd843195b80 1 --2- 172.21.15.7:0/2770447481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577263bc880 0x5577263b6260 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd828098da0 tx=0x7fd828005a00 comp rx=0 tx=0).stop 2024-06-10T01:07:05.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.051+0000 7fd843195b80 1 -- 172.21.15.7:0/2770447481 shutdown_connections 2024-06-10T01:07:05.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.051+0000 7fd843195b80 1 --2- 172.21.15.7:0/2770447481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577263b67a0 0x5577263b6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.051+0000 7fd843195b80 1 --2- 172.21.15.7:0/2770447481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577263bbf60 0x5577263bc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.051+0000 7fd843195b80 1 --2- 172.21.15.7:0/2770447481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577263bc880 0x5577263b6260 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.051+0000 7fd843195b80 1 -- 172.21.15.7:0/2770447481 >> 172.21.15.7:0/2770447481 conn(0x5577262e3080 msgr2=0x5577263b1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:05.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.052+0000 7fd843195b80 1 -- 172.21.15.7:0/2770447481 shutdown_connections 2024-06-10T01:07:05.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.052+0000 7fd843195b80 1 -- 172.21.15.7:0/2770447481 wait complete. 2024-06-10T01:07:05.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.052+0000 7fd843195b80 1 -- start start 2024-06-10T01:07:05.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.052+0000 7fd843195b80 1 Processor -- start 2024-06-10T01:07:05.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.053+0000 7fd843195b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577263b67a0 0x5577263c2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:05.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.053+0000 7fd843195b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577263bbf60 0x5577263bf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:05.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.053+0000 7fd843195b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577263bc880 0x5577263bfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:05.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.053+0000 7fd843195b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5577262a7fc0 con 0x5577263bc880 2024-06-10T01:07:05.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.053+0000 7fd843195b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557726303810 con 0x5577263bbf60 2024-06-10T01:07:05.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.053+0000 7fd843195b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557726307850 con 0x5577263b67a0 2024-06-10T01:07:05.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.053+0000 7fd83259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577263b67a0 0x5577263c2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:05.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.053+0000 7fd83259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577263b67a0 0x5577263c2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:42110/0 (socket says 172.21.15.7:42110) 2024-06-10T01:07:05.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.053+0000 7fd83259c700 1 -- 172.21.15.7:0/4282538848 learned_addr learned my addr 172.21.15.7:0/4282538848 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:05.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.053+0000 7fd832d9d700 1 --2- 172.21.15.7:0/4282538848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577263bc880 0x5577263bfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:05.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.053+0000 7fd843195b80 10 client.0 _ll_register_callbacks cb 0x55772629d5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:05.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.053+0000 7fd831d9b700 1 --2- 172.21.15.7:0/4282538848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577263bbf60 0x5577263bf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:05.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.054+0000 7fd83259c700 1 -- 172.21.15.7:0/4282538848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577263bbf60 msgr2=0x5577263bf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:05.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.054+0000 7fd83259c700 1 --2- 172.21.15.7:0/4282538848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577263bbf60 0x5577263bf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.054+0000 7fd83259c700 1 -- 172.21.15.7:0/4282538848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577263bc880 msgr2=0x5577263bfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:05.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.054+0000 7fd83259c700 1 --2- 172.21.15.7:0/4282538848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577263bc880 0x5577263bfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.054+0000 7fd83259c700 1 -- 172.21.15.7:0/4282538848 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577262cd750 con 0x5577263b67a0 2024-06-10T01:07:05.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.054+0000 7fd83259c700 1 --2- 172.21.15.7:0/4282538848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577263b67a0 0x5577263c2600 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd8240114c0 tx=0x7fd8240047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed55ce84837f4562 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:05.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.054+0000 7fd819ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:07:05.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.054+0000 7fd819ffb700 1 -- 172.21.15.7:0/4282538848 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd824028670 con 0x5577263b67a0 2024-06-10T01:07:05.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.054+0000 7fd843195b80 1 -- 172.21.15.7:0/4282538848 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd82809d040 con 0x5577263b67a0 2024-06-10T01:07:05.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.054+0000 7fd819ffb700 1 -- 172.21.15.7:0/4282538848 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fd824028810 con 0x5577263b67a0 2024-06-10T01:07:05.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.055+0000 7fd819ffb700 1 -- 172.21.15.7:0/4282538848 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd824031050 con 0x5577263b67a0 2024-06-10T01:07:05.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.055+0000 7fd843195b80 10 client.25312 fetch_fsmap learned FSMap version 27 2024-06-10T01:07:05.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.055+0000 7fd843195b80 1 -- 172.21.15.7:0/4282538848 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5577263cd470 con 0x5577263b67a0 2024-06-10T01:07:05.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.055+0000 7fd819ffb700 1 -- 172.21.15.7:0/4282538848 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd82809d040 con 0x5577263b67a0 2024-06-10T01:07:05.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.055+0000 7fd819ffb700 1 -- 172.21.15.7:0/4282538848 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd82401e550 con 0x5577263b67a0 2024-06-10T01:07:05.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.055+0000 7fd843195b80 10 client.25312 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:07:05.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.055+0000 7fd843195b80 10 client.25312 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:07:05.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.055+0000 7fd843195b80 20 client.25312 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:05.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.055+0000 7fd843195b80 1 --2- 172.21.15.7:0/4282538848 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5577263cfa00 0x5577263d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:05.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.055+0000 7fd843195b80 4 client.25312 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:07:05.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.055+0000 7fd843195b80 1 -- 172.21.15.7:0/4282538848 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557726314d50 con 0x5577263cfa00 2024-06-10T01:07:05.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.056+0000 7fd831d9b700 1 --2- 172.21.15.7:0/4282538848 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5577263cfa00 0x5577263d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:05.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.056+0000 7fd831d9b700 1 --2- 172.21.15.7:0/4282538848 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5577263cfa00 0x5577263d1df0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2bc49b46c24aa612 server_cookie=191226ea89299aa7 in_seq=0 out_seq=0 2024-06-10T01:07:05.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.056+0000 7fd819ffb700 10 client.25312 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:07:05.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.057+0000 7fd819ffb700 1 -- 172.21.15.7:0/4282538848 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557726314d50 con 0x5577263cfa00 2024-06-10T01:07:05.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.057+0000 7fd819ffb700 10 client.25312 handle_command_reply: tid=0 2024-06-10T01:07:05.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.057+0000 7fd843195b80 1 client.25312 shutdown 2024-06-10T01:07:05.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.057+0000 7fd843195b80 1 -- 172.21.15.7:0/4282538848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577263b67a0 msgr2=0x5577263c2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:05.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.057+0000 7fd843195b80 1 --2- 172.21.15.7:0/4282538848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577263b67a0 0x5577263c2600 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd8240114c0 tx=0x7fd8240047c0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.057+0000 7fd843195b80 1 -- 172.21.15.7:0/4282538848 shutdown_connections 2024-06-10T01:07:05.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.057+0000 7fd843195b80 1 --2- 172.21.15.7:0/4282538848 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5577263cfa00 0x5577263d1df0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.057+0000 7fd843195b80 1 --2- 172.21.15.7:0/4282538848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577263bc880 0x5577263bfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.057+0000 7fd819ffb700 0 client.25312 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:07:05.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.057+0000 7fd843195b80 1 --2- 172.21.15.7:0/4282538848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577263b67a0 0x5577263c2600 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.057+0000 7fd843195b80 1 --2- 172.21.15.7:0/4282538848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577263bbf60 0x5577263bf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.057+0000 7fd843195b80 1 -- 172.21.15.7:0/4282538848 >> 172.21.15.7:0/4282538848 conn(0x5577262e3080 msgr2=0x5577263b0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:05.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.057+0000 7fd843195b80 1 -- 172.21.15.7:0/4282538848 shutdown_connections 2024-06-10T01:07:05.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.057+0000 7fd843195b80 1 -- 172.21.15.7:0/4282538848 wait complete. 2024-06-10T01:07:05.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.057+0000 7fd843195b80 20 client.25312 trim_cache size 0 max 16384 2024-06-10T01:07:05.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.073+0000 7fd843195b80 1 Processor -- start 2024-06-10T01:07:05.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.073+0000 7fd843195b80 1 -- start start 2024-06-10T01:07:05.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.074+0000 7fd843195b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577263b67a0 0x5577263b6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:05.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.074+0000 7fd843195b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577263bbf60 0x5577263c1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:05.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.074+0000 7fd843195b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577263bc880 0x5577263b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:05.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.074+0000 7fd843195b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd8341b2b80 con 0x5577263bc880 2024-06-10T01:07:05.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.074+0000 7fd843195b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd8341b0410 con 0x5577263bbf60 2024-06-10T01:07:05.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.074+0000 7fd843195b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd8341ae260 con 0x5577263b67a0 2024-06-10T01:07:05.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.074+0000 7fd832d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577263bc880 0x5577263b0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:05.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.074+0000 7fd83259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577263b67a0 0x5577263b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:05.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.074+0000 7fd832d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577263bc880 0x5577263b0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51466/0 (socket says 172.21.15.7:51466) 2024-06-10T01:07:05.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.074+0000 7fd83259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577263b67a0 0x5577263b6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:42120/0 (socket says 172.21.15.7:42120) 2024-06-10T01:07:05.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.074+0000 7fd832d9d700 1 -- 172.21.15.7:0/923166743 learned_addr learned my addr 172.21.15.7:0/923166743 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:05.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.074+0000 7fd831d9b700 1 --2- 172.21.15.7:0/923166743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577263bbf60 0x5577263c1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:05.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.074+0000 7fd831d9b700 1 -- 172.21.15.7:0/923166743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577263b67a0 msgr2=0x5577263b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:05.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.074+0000 7fd831d9b700 1 --2- 172.21.15.7:0/923166743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577263b67a0 0x5577263b6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.074+0000 7fd831d9b700 1 -- 172.21.15.7:0/923166743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577263bc880 msgr2=0x5577263b0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:05.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.074+0000 7fd831d9b700 1 --2- 172.21.15.7:0/923166743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577263bc880 0x5577263b0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.074+0000 7fd831d9b700 1 -- 172.21.15.7:0/923166743 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577263bde70 con 0x5577263bbf60 2024-06-10T01:07:05.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.075+0000 7fd831d9b700 1 --2- 172.21.15.7:0/923166743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577263bbf60 0x5577263c1840 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd82809c8d0 tx=0x7fd828087040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d168cd7729862e0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:05.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.075+0000 7fd81bfff700 1 -- 172.21.15.7:0/923166743 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82809d040 con 0x5577263bbf60 2024-06-10T01:07:05.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.075+0000 7fd81bfff700 1 -- 172.21.15.7:0/923166743 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd82809c940 con 0x5577263bbf60 2024-06-10T01:07:05.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.075+0000 7fd81bfff700 1 -- 172.21.15.7:0/923166743 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82809cba0 con 0x5577263bbf60 2024-06-10T01:07:05.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.076+0000 7fd843195b80 1 -- 172.21.15.7:0/923166743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577263bbf60 msgr2=0x5577263c1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:05.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.076+0000 7fd843195b80 1 --2- 172.21.15.7:0/923166743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577263bbf60 0x5577263c1840 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd82809c8d0 tx=0x7fd828087040 comp rx=0 tx=0).stop 2024-06-10T01:07:05.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.076+0000 7fd843195b80 1 -- 172.21.15.7:0/923166743 shutdown_connections 2024-06-10T01:07:05.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.076+0000 7fd843195b80 1 --2- 172.21.15.7:0/923166743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577263bc880 0x5577263b0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.076+0000 7fd843195b80 1 --2- 172.21.15.7:0/923166743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577263b67a0 0x5577263b6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.076+0000 7fd843195b80 1 --2- 172.21.15.7:0/923166743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577263bbf60 0x5577263c1840 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.076+0000 7fd843195b80 1 -- 172.21.15.7:0/923166743 >> 172.21.15.7:0/923166743 conn(0x5577262e3080 msgr2=0x5577263c1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:05.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.076+0000 7fd843195b80 1 -- 172.21.15.7:0/923166743 shutdown_connections 2024-06-10T01:07:05.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.076+0000 7fd843195b80 1 -- 172.21.15.7:0/923166743 wait complete. 2024-06-10T01:07:05.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.076+0000 7fd843195b80 1 -- start start 2024-06-10T01:07:05.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.077+0000 7fd843195b80 1 Processor -- start 2024-06-10T01:07:05.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.077+0000 7fd843195b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577263b67a0 0x5577263c5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:05.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.077+0000 7fd843195b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577263bbf60 0x5577263c5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:05.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.077+0000 7fd843195b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577263bc880 0x557726440e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:05.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.077+0000 7fd843195b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd8341ae8d0 con 0x5577263bbf60 2024-06-10T01:07:05.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.077+0000 7fd843195b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd8341afd40 con 0x5577263bc880 2024-06-10T01:07:05.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.077+0000 7fd843195b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd8341aefa0 con 0x5577263b67a0 2024-06-10T01:07:05.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.077+0000 7fd843195b80 10 client.0 _ll_register_callbacks cb 0x55772629d5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:05.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.077+0000 7fd831d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577263bbf60 0x5577263c5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:05.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.077+0000 7fd831d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577263bbf60 0x5577263c5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51470/0 (socket says 172.21.15.7:51470) 2024-06-10T01:07:05.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.077+0000 7fd832d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577263bc880 0x557726440e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:05.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.077+0000 7fd831d9b700 1 -- 172.21.15.7:0/3807582554 learned_addr learned my addr 172.21.15.7:0/3807582554 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:05.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.077+0000 7fd83259c700 1 --2- 172.21.15.7:0/3807582554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577263b67a0 0x5577263c5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:05.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.077+0000 7fd832d9d700 1 -- 172.21.15.7:0/3807582554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577263b67a0 msgr2=0x5577263c5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:05.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.077+0000 7fd832d9d700 1 --2- 172.21.15.7:0/3807582554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577263b67a0 0x5577263c5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.077+0000 7fd832d9d700 1 -- 172.21.15.7:0/3807582554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577263bbf60 msgr2=0x5577263c5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:05.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.077+0000 7fd832d9d700 1 --2- 172.21.15.7:0/3807582554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577263bbf60 0x5577263c5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.078+0000 7fd832d9d700 1 -- 172.21.15.7:0/3807582554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557726441390 con 0x5577263bc880 2024-06-10T01:07:05.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.078+0000 7fd832d9d700 1 --2- 172.21.15.7:0/3807582554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577263bc880 0x557726440e50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd82c003da0 tx=0x7fd82c004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=644dcdb4cec282ce server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:05.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.078+0000 7fd819ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:07:05.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.078+0000 7fd819ffb700 1 -- 172.21.15.7:0/3807582554 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c020420 con 0x5577263bc880 2024-06-10T01:07:05.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.078+0000 7fd819ffb700 1 -- 172.21.15.7:0/3807582554 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fd82c010040 con 0x5577263bc880 2024-06-10T01:07:05.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.078+0000 7fd843195b80 1 -- 172.21.15.7:0/3807582554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd82809c940 con 0x5577263bc880 2024-06-10T01:07:05.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.078+0000 7fd819ffb700 1 -- 172.21.15.7:0/3807582554 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd82c0205c0 con 0x5577263bc880 2024-06-10T01:07:05.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.078+0000 7fd843195b80 10 client.25265 fetch_fsmap learned FSMap version 27 2024-06-10T01:07:05.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.078+0000 7fd843195b80 1 -- 172.21.15.7:0/3807582554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557726442720 con 0x5577263bc880 2024-06-10T01:07:05.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.078+0000 7fd819ffb700 1 -- 172.21.15.7:0/3807582554 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd82809c940 con 0x5577263bc880 2024-06-10T01:07:05.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.079+0000 7fd819ffb700 1 -- 172.21.15.7:0/3807582554 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd82c016920 con 0x5577263bc880 2024-06-10T01:07:05.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.079+0000 7fd843195b80 10 client.25265 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:07:05.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.079+0000 7fd843195b80 10 client.25265 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:07:05.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.079+0000 7fd843195b80 20 client.25265 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:05.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.079+0000 7fd843195b80 1 --2- 172.21.15.7:0/3807582554 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5577263cfa00 0x557726444f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:05.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.079+0000 7fd843195b80 4 client.25265 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:07:05.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.079+0000 7fd843195b80 1 -- 172.21.15.7:0/3807582554 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55772630f2d0 con 0x5577263cfa00 2024-06-10T01:07:05.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.079+0000 7fd83259c700 1 --2- 172.21.15.7:0/3807582554 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5577263cfa00 0x557726444f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:05.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.079+0000 7fd819ffb700 10 client.25265 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:07:05.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.079+0000 7fd83259c700 1 --2- 172.21.15.7:0/3807582554 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5577263cfa00 0x557726444f20 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=201e5aac512cddf6 server_cookie=f777782a2b0a4bc3 in_seq=0 out_seq=0 2024-06-10T01:07:05.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.080+0000 7fd819ffb700 1 -- 172.21.15.7:0/3807582554 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55772630f2d0 con 0x5577263cfa00 2024-06-10T01:07:05.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.080+0000 7fd819ffb700 10 client.25265 handle_command_reply: tid=0 2024-06-10T01:07:05.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.080+0000 7fd843195b80 1 client.25265 shutdown 2024-06-10T01:07:05.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.080+0000 7fd843195b80 1 -- 172.21.15.7:0/3807582554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577263bc880 msgr2=0x557726440e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:05.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.080+0000 7fd843195b80 1 --2- 172.21.15.7:0/3807582554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577263bc880 0x557726440e50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd82c003da0 tx=0x7fd82c004320 comp rx=0 tx=0).stop 2024-06-10T01:07:05.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.080+0000 7fd843195b80 1 -- 172.21.15.7:0/3807582554 shutdown_connections 2024-06-10T01:07:05.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.080+0000 7fd843195b80 1 --2- 172.21.15.7:0/3807582554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577263bc880 0x557726440e50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.080+0000 7fd843195b80 1 --2- 172.21.15.7:0/3807582554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577263b67a0 0x5577263c5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.080+0000 7fd843195b80 1 --2- 172.21.15.7:0/3807582554 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5577263cfa00 0x557726444f20 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.080+0000 7fd819ffb700 0 client.25265 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:07:05.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.080+0000 7fd843195b80 1 --2- 172.21.15.7:0/3807582554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577263bbf60 0x5577263c5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.080+0000 7fd843195b80 1 -- 172.21.15.7:0/3807582554 >> 172.21.15.7:0/3807582554 conn(0x5577262e3080 msgr2=0x5577263bb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:05.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.080+0000 7fd843195b80 1 -- 172.21.15.7:0/3807582554 shutdown_connections 2024-06-10T01:07:05.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.080+0000 7fd843195b80 1 -- 172.21.15.7:0/3807582554 wait complete. 2024-06-10T01:07:05.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.081+0000 7fd843195b80 20 client.25265 trim_cache size 0 max 16384 2024-06-10T01:07:05.083 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:07:05.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.081+0000 7fd838912700 1 -- 172.21.15.7:0/2205818503 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd81c0a2ca0 msgr2=0x7fd81c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:05.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.081+0000 7fd838912700 1 --2- 172.21.15.7:0/2205818503 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd81c0a2ca0 0x7fd81c0a5160 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd824007b60 tx=0x7fd8240073d0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.081+0000 7fd838912700 1 -- 172.21.15.7:0/2205818503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd834101d30 msgr2=0x7fd834103580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:05.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.081+0000 7fd838912700 1 --2- 172.21.15.7:0/2205818503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd834101d30 0x7fd834103580 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd8280059d0 tx=0x7fd8280047b0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.081+0000 7fd838912700 1 -- 172.21.15.7:0/2205818503 shutdown_connections 2024-06-10T01:07:05.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.081+0000 7fd838912700 1 --2- 172.21.15.7:0/2205818503 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd81c0a2ca0 0x7fd81c0a5160 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.081+0000 7fd838912700 1 --2- 172.21.15.7:0/2205818503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8341073d0 0x7fd834103ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.081+0000 7fd838912700 1 --2- 172.21.15.7:0/2205818503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd834101390 0x7fd834103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.081+0000 7fd838912700 1 --2- 172.21.15.7:0/2205818503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd834101d30 0x7fd834103580 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:05.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.081+0000 7fd838912700 1 -- 172.21.15.7:0/2205818503 >> 172.21.15.7:0/2205818503 conn(0x7fd8340782c0 msgr2=0x7fd834100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:05.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.082+0000 7fd838912700 1 -- 172.21.15.7:0/2205818503 shutdown_connections 2024-06-10T01:07:05.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:05.082+0000 7fd838912700 1 -- 172.21.15.7:0/2205818503 wait complete. 2024-06-10T01:07:05.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:05 smithi175 ceph-mon[36636]: pgmap v463: 129 pgs: 129 active+clean; 672 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 852 B/s wr, 8 op/s 2024-06-10T01:07:05.748 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:05 smithi007 ceph-mon[32398]: pgmap v463: 129 pgs: 129 active+clean; 672 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 852 B/s wr, 8 op/s 2024-06-10T01:07:05.789 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:07:05.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:05 smithi078 ceph-mon[36683]: pgmap v463: 129 pgs: 129 active+clean; 672 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 852 B/s wr, 8 op/s 2024-06-10T01:07:06.055 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:07.608 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:07 smithi007 ceph-mon[32398]: pgmap v464: 129 pgs: 129 active+clean; 672 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 937 B/s wr, 9 op/s 2024-06-10T01:07:07.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:07 smithi175 ceph-mon[36636]: pgmap v464: 129 pgs: 129 active+clean; 672 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 937 B/s wr, 9 op/s 2024-06-10T01:07:07.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:07 smithi078 ceph-mon[36683]: pgmap v464: 129 pgs: 129 active+clean; 672 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 937 B/s wr, 9 op/s 2024-06-10T01:07:07.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.913+0000 7f99e0fc6700 1 -- 172.21.15.7:0/3396455106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99dc101d50 msgr2=0x7f99dc10e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:07.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.913+0000 7f99e0fc6700 1 --2- 172.21.15.7:0/3396455106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99dc101d50 0x7f99dc10e240 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f99d400b0a0 tx=0x7f99d400b3b0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.914+0000 7f99e0fc6700 1 -- 172.21.15.7:0/3396455106 shutdown_connections 2024-06-10T01:07:07.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.914+0000 7f99e0fc6700 1 --2- 172.21.15.7:0/3396455106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99dc101d50 0x7f99dc10e240 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.914+0000 7f99e0fc6700 1 --2- 172.21.15.7:0/3396455106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99dc107400 0x7f99dc1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.914+0000 7f99e0fc6700 1 --2- 172.21.15.7:0/3396455106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99dc101390 0x7f99dc101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.914+0000 7f99e0fc6700 1 -- 172.21.15.7:0/3396455106 >> 172.21.15.7:0/3396455106 conn(0x7f99dc078340 msgr2=0x7f99dc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:07.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.914+0000 7f99e0fc6700 1 -- 172.21.15.7:0/3396455106 shutdown_connections 2024-06-10T01:07:07.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.914+0000 7f99e0fc6700 1 -- 172.21.15.7:0/3396455106 wait complete. 2024-06-10T01:07:07.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.915+0000 7f99e0fc6700 1 Processor -- start 2024-06-10T01:07:07.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.915+0000 7f99e0fc6700 1 -- start start 2024-06-10T01:07:07.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.915+0000 7f99e0fc6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99dc101390 0x7f99dc19d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:07.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.916+0000 7f99e0fc6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99dc101d50 0x7f99dc19dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:07.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.916+0000 7f99e0fc6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99dc107400 0x7f99dc196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:07.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.916+0000 7f99e0fc6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc110580 con 0x7f99dc101d50 2024-06-10T01:07:07.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.916+0000 7f99e0fc6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc110400 con 0x7f99dc107400 2024-06-10T01:07:07.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.916+0000 7f99e0fc6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc110700 con 0x7f99dc101390 2024-06-10T01:07:07.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.916+0000 7f99da59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99dc101390 0x7f99dc19d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:07.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.916+0000 7f99da59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99dc101390 0x7f99dc19d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:42152/0 (socket says 172.21.15.7:42152) 2024-06-10T01:07:07.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.916+0000 7f99da59c700 1 -- 172.21.15.7:0/2919149797 learned_addr learned my addr 172.21.15.7:0/2919149797 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:07.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.916+0000 7f99d9d9b700 1 --2- 172.21.15.7:0/2919149797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99dc101d50 0x7f99dc19dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:07.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.916+0000 7f99dad9d700 1 --2- 172.21.15.7:0/2919149797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99dc107400 0x7f99dc196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:07.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.916+0000 7f99da59c700 1 -- 172.21.15.7:0/2919149797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99dc107400 msgr2=0x7f99dc196910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:07.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.916+0000 7f99da59c700 1 --2- 172.21.15.7:0/2919149797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99dc107400 0x7f99dc196910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.916+0000 7f99da59c700 1 -- 172.21.15.7:0/2919149797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99dc101d50 msgr2=0x7f99dc19dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:07.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.916+0000 7f99da59c700 1 --2- 172.21.15.7:0/2919149797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99dc101d50 0x7f99dc19dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.916+0000 7f99da59c700 1 -- 172.21.15.7:0/2919149797 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99dc197210 con 0x7f99dc101390 2024-06-10T01:07:07.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.916+0000 7f99d9d9b700 1 --2- 172.21.15.7:0/2919149797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99dc101d50 0x7f99dc19dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:07.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.917+0000 7f99da59c700 1 --2- 172.21.15.7:0/2919149797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99dc101390 0x7f99dc19d710 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00e9a0 tx=0x7f99cc00ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:07.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.917+0000 7f99cb7fe700 1 -- 172.21.15.7:0/2919149797 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc00cbd0 con 0x7f99dc101390 2024-06-10T01:07:07.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.917+0000 7f99cb7fe700 1 -- 172.21.15.7:0/2919149797 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99cc00cd70 con 0x7f99dc101390 2024-06-10T01:07:07.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.917+0000 7f99e0fc6700 1 -- 172.21.15.7:0/2919149797 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99dc197500 con 0x7f99dc101390 2024-06-10T01:07:07.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.917+0000 7f99cb7fe700 1 -- 172.21.15.7:0/2919149797 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc0187c0 con 0x7f99dc101390 2024-06-10T01:07:07.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.917+0000 7f99e0fc6700 1 -- 172.21.15.7:0/2919149797 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99dc19b460 con 0x7f99dc101390 2024-06-10T01:07:07.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.920+0000 7f99cb7fe700 1 -- 172.21.15.7:0/2919149797 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f99cc010430 con 0x7f99dc101390 2024-06-10T01:07:07.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.920+0000 7f99cb7fe700 1 --2- 172.21.15.7:0/2919149797 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f99c40a2c80 0x7f99c40a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:07.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.920+0000 7f99cb7fe700 1 -- 172.21.15.7:0/2919149797 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f99cc0cf5b0 con 0x7f99dc101390 2024-06-10T01:07:07.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.921+0000 7f99d9d9b700 1 --2- 172.21.15.7:0/2919149797 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f99c40a2c80 0x7f99c40a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:07.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.921+0000 7f99d9d9b700 1 --2- 172.21.15.7:0/2919149797 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f99c40a2c80 0x7f99c40a5140 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f99dc197f60 tx=0x7f99d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:07.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.921+0000 7f99eb849b80 1 Processor -- start 2024-06-10T01:07:07.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.922+0000 7f99eb849b80 1 -- start start 2024-06-10T01:07:07.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.922+0000 7f99eb849b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586aa49c640 0x5586aa49ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:07.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.922+0000 7f99eb849b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586aa49cf60 0x5586aa4969e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:07.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.922+0000 7f99eb849b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aa496f20 0x5586aa497340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:07.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.922+0000 7f99eb849b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5586aa3e8810 con 0x5586aa49c640 2024-06-10T01:07:07.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.922+0000 7f99eb849b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5586aa390720 con 0x5586aa496f20 2024-06-10T01:07:07.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.922+0000 7f99eb849b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5586aa3e47d0 con 0x5586aa49cf60 2024-06-10T01:07:07.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.922+0000 7f99da59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586aa49c640 0x5586aa49ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:07.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.922+0000 7f99da59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586aa49c640 0x5586aa49ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51494/0 (socket says 172.21.15.7:51494) 2024-06-10T01:07:07.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.922+0000 7f99da59c700 1 -- 172.21.15.7:0/1331711382 learned_addr learned my addr 172.21.15.7:0/1331711382 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:07.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.922+0000 7f99d9d9b700 1 --2- 172.21.15.7:0/1331711382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586aa49cf60 0x5586aa4969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:07.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.923+0000 7f99dad9d700 1 --2- 172.21.15.7:0/1331711382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aa496f20 0x5586aa497340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:07.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.923+0000 7f99da59c700 1 -- 172.21.15.7:0/1331711382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586aa49cf60 msgr2=0x5586aa4969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:07.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.923+0000 7f99da59c700 1 --2- 172.21.15.7:0/1331711382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586aa49cf60 0x5586aa4969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.923+0000 7f99da59c700 1 -- 172.21.15.7:0/1331711382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aa496f20 msgr2=0x5586aa497340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:07.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.923+0000 7f99da59c700 1 --2- 172.21.15.7:0/1331711382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aa496f20 0x5586aa497340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.923+0000 7f99da59c700 1 -- 172.21.15.7:0/1331711382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586aa386f50 con 0x5586aa49c640 2024-06-10T01:07:07.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.923+0000 7f99dad9d700 1 --2- 172.21.15.7:0/1331711382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aa496f20 0x5586aa497340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:07.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.923+0000 7f99da59c700 1 --2- 172.21.15.7:0/1331711382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586aa49c640 0x5586aa49ca20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f99cc097ef0 tx=0x7f99cc09c190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27c248f9e9bcad8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:07.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.923+0000 7f99c8ff9700 1 -- 172.21.15.7:0/1331711382 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc0a4700 con 0x5586aa49c640 2024-06-10T01:07:07.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.923+0000 7f99c8ff9700 1 -- 172.21.15.7:0/1331711382 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99cc099d90 con 0x5586aa49c640 2024-06-10T01:07:07.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.923+0000 7f99c8ff9700 1 -- 172.21.15.7:0/1331711382 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc0a4270 con 0x5586aa49c640 2024-06-10T01:07:07.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.924+0000 7f99eb849b80 1 -- 172.21.15.7:0/1331711382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586aa49c640 msgr2=0x5586aa49ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:07.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.924+0000 7f99eb849b80 1 --2- 172.21.15.7:0/1331711382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586aa49c640 0x5586aa49ca20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f99cc097ef0 tx=0x7f99cc09c190 comp rx=0 tx=0).stop 2024-06-10T01:07:07.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.924+0000 7f99eb849b80 1 -- 172.21.15.7:0/1331711382 shutdown_connections 2024-06-10T01:07:07.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.924+0000 7f99eb849b80 1 --2- 172.21.15.7:0/1331711382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aa496f20 0x5586aa497340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.924+0000 7f99eb849b80 1 --2- 172.21.15.7:0/1331711382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586aa49c640 0x5586aa49ca20 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.924+0000 7f99eb849b80 1 --2- 172.21.15.7:0/1331711382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586aa49cf60 0x5586aa4969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.924+0000 7f99eb849b80 1 -- 172.21.15.7:0/1331711382 >> 172.21.15.7:0/1331711382 conn(0x5586aa3c1cc0 msgr2=0x5586aa491850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:07.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.924+0000 7f99eb849b80 1 -- 172.21.15.7:0/1331711382 shutdown_connections 2024-06-10T01:07:07.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.924+0000 7f99eb849b80 1 -- 172.21.15.7:0/1331711382 wait complete. 2024-06-10T01:07:07.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.925+0000 7f99eb849b80 1 -- start start 2024-06-10T01:07:07.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.925+0000 7f99eb849b80 1 Processor -- start 2024-06-10T01:07:07.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.926+0000 7f99eb849b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586aa496f20 0x5586aa4a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:07.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.926+0000 7f99eb849b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aa49c640 0x5586aa4a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:07.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.926+0000 7f99eb849b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586aa49cf60 0x5586aa4a0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:07.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.926+0000 7f99eb849b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5586aa4a3970 con 0x5586aa496f20 2024-06-10T01:07:07.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.926+0000 7f99da59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586aa496f20 0x5586aa4a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:07.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.926+0000 7f99da59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586aa496f20 0x5586aa4a2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51496/0 (socket says 172.21.15.7:51496) 2024-06-10T01:07:07.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.926+0000 7f99da59c700 1 -- 172.21.15.7:0/619395418 learned_addr learned my addr 172.21.15.7:0/619395418 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:07.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.926+0000 7f99eb849b80 1 -- 172.21.15.7:0/619395418 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5586aa388bf0 con 0x5586aa49c640 2024-06-10T01:07:07.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.926+0000 7f99d9d9b700 1 --2- 172.21.15.7:0/619395418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aa49c640 0x5586aa4a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:07.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.926+0000 7f99eb849b80 1 -- 172.21.15.7:0/619395418 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5586aa4a3af0 con 0x5586aa49cf60 2024-06-10T01:07:07.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.926+0000 7f99eb849b80 10 client.0 _ll_register_callbacks cb 0x5586aa382920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:07.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.927+0000 7f99da59c700 1 -- 172.21.15.7:0/619395418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586aa49cf60 msgr2=0x5586aa4a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:07.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.927+0000 7f99da59c700 1 --2- 172.21.15.7:0/619395418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586aa49cf60 0x5586aa4a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.927+0000 7f99da59c700 1 -- 172.21.15.7:0/619395418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aa49c640 msgr2=0x5586aa4a00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:07.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.927+0000 7f99da59c700 1 --2- 172.21.15.7:0/619395418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aa49c640 0x5586aa4a00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.927+0000 7f99da59c700 1 -- 172.21.15.7:0/619395418 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586aa3378d0 con 0x5586aa496f20 2024-06-10T01:07:07.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.927+0000 7f99da59c700 1 --2- 172.21.15.7:0/619395418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586aa496f20 0x5586aa4a2dc0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f99cc0cd0a0 tx=0x7f99cc00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5037602986731ae server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:07.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.927+0000 7f99c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:07:07.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.927+0000 7f99c1ffb700 1 -- 172.21.15.7:0/619395418 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc0b8030 con 0x5586aa496f20 2024-06-10T01:07:07.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.927+0000 7f99c1ffb700 1 -- 172.21.15.7:0/619395418 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f99cc0cf300 con 0x5586aa496f20 2024-06-10T01:07:07.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.927+0000 7f99eb849b80 1 -- 172.21.15.7:0/619395418 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99cc099d90 con 0x5586aa496f20 2024-06-10T01:07:07.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.928+0000 7f99c1ffb700 1 -- 172.21.15.7:0/619395418 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f99cc0ba2c0 con 0x5586aa496f20 2024-06-10T01:07:07.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.928+0000 7f99eb849b80 10 client.15669 fetch_fsmap learned FSMap version 27 2024-06-10T01:07:07.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.928+0000 7f99eb849b80 1 -- 172.21.15.7:0/619395418 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5586aa4ae360 con 0x5586aa496f20 2024-06-10T01:07:07.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.928+0000 7f99c1ffb700 1 -- 172.21.15.7:0/619395418 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99cc099d90 con 0x5586aa496f20 2024-06-10T01:07:07.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.928+0000 7f99c1ffb700 1 -- 172.21.15.7:0/619395418 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f99cc09a950 con 0x5586aa496f20 2024-06-10T01:07:07.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.928+0000 7f99eb849b80 10 client.15669 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:07:07.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.928+0000 7f99eb849b80 10 client.15669 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:07:07.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.928+0000 7f99eb849b80 20 client.15669 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:07.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.928+0000 7f99eb849b80 1 --2- 172.21.15.7:0/619395418 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5586aa4b0830 0x5586aa4b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:07.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.929+0000 7f99eb849b80 4 client.15669 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:07:07.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.929+0000 7f99eb849b80 1 -- 172.21.15.7:0/619395418 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586aa3f5d10 con 0x5586aa4b0830 2024-06-10T01:07:07.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.929+0000 7f99d9d9b700 1 --2- 172.21.15.7:0/619395418 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5586aa4b0830 0x5586aa4b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:07.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.929+0000 7f99d9d9b700 1 --2- 172.21.15.7:0/619395418 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5586aa4b0830 0x5586aa4b2c20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c31a9e148499880b server_cookie=5b796e4d148cef55 in_seq=0 out_seq=0 2024-06-10T01:07:07.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.929+0000 7f99c1ffb700 10 client.15669 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:07:07.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.930+0000 7f99c1ffb700 1 -- 172.21.15.7:0/619395418 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5586aa3f5d10 con 0x5586aa4b0830 2024-06-10T01:07:07.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.930+0000 7f99c1ffb700 10 client.15669 handle_command_reply: tid=0 2024-06-10T01:07:07.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.930+0000 7f99eb849b80 1 client.15669 shutdown 2024-06-10T01:07:07.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.930+0000 7f99eb849b80 1 -- 172.21.15.7:0/619395418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586aa496f20 msgr2=0x5586aa4a2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:07.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.930+0000 7f99eb849b80 1 --2- 172.21.15.7:0/619395418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586aa496f20 0x5586aa4a2dc0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f99cc0cd0a0 tx=0x7f99cc00cf20 comp rx=0 tx=0).stop 2024-06-10T01:07:07.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.930+0000 7f99eb849b80 1 -- 172.21.15.7:0/619395418 shutdown_connections 2024-06-10T01:07:07.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.930+0000 7f99eb849b80 1 --2- 172.21.15.7:0/619395418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586aa49cf60 0x5586aa4a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.930+0000 7f99eb849b80 1 --2- 172.21.15.7:0/619395418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586aa496f20 0x5586aa4a2dc0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.930+0000 7f99eb849b80 1 --2- 172.21.15.7:0/619395418 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5586aa4b0830 0x5586aa4b2c20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.930+0000 7f99c1ffb700 0 client.15669 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:07:07.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.930+0000 7f99eb849b80 1 --2- 172.21.15.7:0/619395418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aa49c640 0x5586aa4a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.930+0000 7f99eb849b80 1 -- 172.21.15.7:0/619395418 >> 172.21.15.7:0/619395418 conn(0x5586aa3c1cc0 msgr2=0x5586aa4959d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:07.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.931+0000 7f99eb849b80 1 -- 172.21.15.7:0/619395418 shutdown_connections 2024-06-10T01:07:07.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.931+0000 7f99eb849b80 1 -- 172.21.15.7:0/619395418 wait complete. 2024-06-10T01:07:07.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.931+0000 7f99eb849b80 20 client.15669 trim_cache size 0 max 16384 2024-06-10T01:07:07.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.946+0000 7f99eb849b80 1 Processor -- start 2024-06-10T01:07:07.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.946+0000 7f99eb849b80 1 -- start start 2024-06-10T01:07:07.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.947+0000 7f99eb849b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586aa496f20 0x5586aa4969e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:07.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.947+0000 7f99eb849b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aa49c640 0x5586aa49a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:07.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.947+0000 7f99eb849b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586aa49cf60 0x5586aa48b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:07.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.947+0000 7f99eb849b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc1bf9f0 con 0x5586aa49cf60 2024-06-10T01:07:07.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.947+0000 7f99eb849b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc1bd280 con 0x5586aa49c640 2024-06-10T01:07:07.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.947+0000 7f99eb849b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc1bb0d0 con 0x5586aa496f20 2024-06-10T01:07:07.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.947+0000 7f99d9d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aa49c640 0x5586aa49a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:07.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.947+0000 7f99da59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586aa496f20 0x5586aa4969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:07.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.947+0000 7f99da59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586aa496f20 0x5586aa4969e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:42194/0 (socket says 172.21.15.7:42194) 2024-06-10T01:07:07.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.947+0000 7f99da59c700 1 -- 172.21.15.7:0/3352540008 learned_addr learned my addr 172.21.15.7:0/3352540008 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:07.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.947+0000 7f99dad9d700 1 --2- 172.21.15.7:0/3352540008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586aa49cf60 0x5586aa48b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:07.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.947+0000 7f99d9d9b700 1 -- 172.21.15.7:0/3352540008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586aa496f20 msgr2=0x5586aa4969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:07.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.947+0000 7f99d9d9b700 1 --2- 172.21.15.7:0/3352540008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586aa496f20 0x5586aa4969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.947+0000 7f99d9d9b700 1 -- 172.21.15.7:0/3352540008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586aa49cf60 msgr2=0x5586aa48b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:07.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.947+0000 7f99d9d9b700 1 --2- 172.21.15.7:0/3352540008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586aa49cf60 0x5586aa48b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.947+0000 7f99d9d9b700 1 -- 172.21.15.7:0/3352540008 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586aa492320 con 0x5586aa49c640 2024-06-10T01:07:07.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.948+0000 7f99d9d9b700 1 --2- 172.21.15.7:0/3352540008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aa49c640 0x5586aa49a730 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x5586aa4a1690 tx=0x7f99d0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be803b32af8478 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:07.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.948+0000 7f99c3fff700 1 -- 172.21.15.7:0/3352540008 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d002a3b0 con 0x5586aa49c640 2024-06-10T01:07:07.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.948+0000 7f99c3fff700 1 -- 172.21.15.7:0/3352540008 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99d002aa30 con 0x5586aa49c640 2024-06-10T01:07:07.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.948+0000 7f99c3fff700 1 -- 172.21.15.7:0/3352540008 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d0029a70 con 0x5586aa49c640 2024-06-10T01:07:07.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.949+0000 7f99eb849b80 1 -- 172.21.15.7:0/3352540008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aa49c640 msgr2=0x5586aa49a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:07.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.949+0000 7f99eb849b80 1 --2- 172.21.15.7:0/3352540008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aa49c640 0x5586aa49a730 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x5586aa4a1690 tx=0x7f99d0014000 comp rx=0 tx=0).stop 2024-06-10T01:07:07.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.949+0000 7f99eb849b80 1 -- 172.21.15.7:0/3352540008 shutdown_connections 2024-06-10T01:07:07.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.949+0000 7f99eb849b80 1 --2- 172.21.15.7:0/3352540008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586aa49cf60 0x5586aa48b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.949+0000 7f99eb849b80 1 --2- 172.21.15.7:0/3352540008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586aa496f20 0x5586aa4969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.949+0000 7f99eb849b80 1 --2- 172.21.15.7:0/3352540008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aa49c640 0x5586aa49a730 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.949+0000 7f99eb849b80 1 -- 172.21.15.7:0/3352540008 >> 172.21.15.7:0/3352540008 conn(0x5586aa3c1cc0 msgr2=0x5586aa49d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:07.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.949+0000 7f99eb849b80 1 -- 172.21.15.7:0/3352540008 shutdown_connections 2024-06-10T01:07:07.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.949+0000 7f99eb849b80 1 -- 172.21.15.7:0/3352540008 wait complete. 2024-06-10T01:07:07.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.949+0000 7f99eb849b80 1 -- start start 2024-06-10T01:07:07.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.949+0000 7f99eb849b80 1 Processor -- start 2024-06-10T01:07:07.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.950+0000 7f99eb849b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aa496f20 0x5586aa4b60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:07.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.950+0000 7f99eb849b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586aa49c640 0x5586aa4b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:07.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.950+0000 7f99eb849b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586aa49cf60 0x5586aa4b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:07.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.950+0000 7f99d9d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586aa49c640 0x5586aa4b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:07.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.950+0000 7f99da59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aa496f20 0x5586aa4b60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:07.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.950+0000 7f99d9d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586aa49c640 0x5586aa4b34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51520/0 (socket says 172.21.15.7:51520) 2024-06-10T01:07:07.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.950+0000 7f99da59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aa496f20 0x5586aa4b60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:36690/0 (socket says 172.21.15.7:36690) 2024-06-10T01:07:07.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.950+0000 7f99d9d9b700 1 -- 172.21.15.7:0/1920189660 learned_addr learned my addr 172.21.15.7:0/1920189660 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:07.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.950+0000 7f99eb849b80 1 -- 172.21.15.7:0/1920189660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc1bb740 con 0x5586aa49c640 2024-06-10T01:07:07.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.950+0000 7f99eb849b80 1 -- 172.21.15.7:0/1920189660 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc1bcbb0 con 0x5586aa496f20 2024-06-10T01:07:07.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.950+0000 7f99eb849b80 1 -- 172.21.15.7:0/1920189660 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc1bbe10 con 0x5586aa49cf60 2024-06-10T01:07:07.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.950+0000 7f99eb849b80 10 client.0 _ll_register_callbacks cb 0x5586aa382920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:07.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.950+0000 7f99d9d9b700 1 -- 172.21.15.7:0/1920189660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586aa49cf60 msgr2=0x5586aa4b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:07.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.950+0000 7f99d9d9b700 1 --2- 172.21.15.7:0/1920189660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586aa49cf60 0x5586aa4b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.950+0000 7f99d9d9b700 1 -- 172.21.15.7:0/1920189660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aa496f20 msgr2=0x5586aa4b60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:07.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.950+0000 7f99d9d9b700 1 --2- 172.21.15.7:0/1920189660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aa496f20 0x5586aa4b60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.951+0000 7f99d9d9b700 1 -- 172.21.15.7:0/1920189660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586aa4b3f20 con 0x5586aa49c640 2024-06-10T01:07:07.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.951+0000 7f99d9d9b700 1 --2- 172.21.15.7:0/1920189660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586aa49c640 0x5586aa4b34a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f99d0000c00 tx=0x7f99d0012de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1958fb61dc91414 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:07.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.951+0000 7f99c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:07:07.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.951+0000 7f99c1ffb700 1 -- 172.21.15.7:0/1920189660 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d002a3b0 con 0x5586aa49c640 2024-06-10T01:07:07.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.951+0000 7f99c1ffb700 1 -- 172.21.15.7:0/1920189660 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f99d002a550 con 0x5586aa49c640 2024-06-10T01:07:07.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.951+0000 7f99eb849b80 1 -- 172.21.15.7:0/1920189660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99d002aa30 con 0x5586aa49c640 2024-06-10T01:07:07.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.951+0000 7f99c1ffb700 1 -- 172.21.15.7:0/1920189660 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f99d0044050 con 0x5586aa49c640 2024-06-10T01:07:07.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.951+0000 7f99eb849b80 10 client.15675 fetch_fsmap learned FSMap version 27 2024-06-10T01:07:07.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.951+0000 7f99eb849b80 1 -- 172.21.15.7:0/1920189660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5586aa51aea0 con 0x5586aa49c640 2024-06-10T01:07:07.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.952+0000 7f99c1ffb700 1 -- 172.21.15.7:0/1920189660 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99d002aa30 con 0x5586aa49c640 2024-06-10T01:07:07.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.952+0000 7f99c1ffb700 1 -- 172.21.15.7:0/1920189660 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f99d00296f0 con 0x5586aa49c640 2024-06-10T01:07:07.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.952+0000 7f99eb849b80 10 client.15675 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:07:07.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.952+0000 7f99eb849b80 10 client.15675 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:07:07.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.952+0000 7f99eb849b80 20 client.15675 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:07.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.952+0000 7f99eb849b80 1 --2- 172.21.15.7:0/1920189660 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5586aa51b520 0x5586aa51d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:07.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.952+0000 7f99eb849b80 4 client.15675 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:07:07.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.952+0000 7f99eb849b80 1 -- 172.21.15.7:0/1920189660 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5586aa3f0290 con 0x5586aa51b520 2024-06-10T01:07:07.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.952+0000 7f99da59c700 1 --2- 172.21.15.7:0/1920189660 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5586aa51b520 0x5586aa51d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:07.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.953+0000 7f99c1ffb700 10 client.15675 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:07:07.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.953+0000 7f99da59c700 1 --2- 172.21.15.7:0/1920189660 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5586aa51b520 0x5586aa51d940 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b8b2221f0af6849a server_cookie=8ffec13cfc8871e1 in_seq=0 out_seq=0 2024-06-10T01:07:07.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.953+0000 7f99c1ffb700 1 -- 172.21.15.7:0/1920189660 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5586aa3f0290 con 0x5586aa51b520 2024-06-10T01:07:07.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.953+0000 7f99c1ffb700 10 client.15675 handle_command_reply: tid=0 2024-06-10T01:07:07.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.953+0000 7f99eb849b80 1 client.15675 shutdown 2024-06-10T01:07:07.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.953+0000 7f99eb849b80 1 -- 172.21.15.7:0/1920189660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586aa49c640 msgr2=0x5586aa4b34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:07.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.953+0000 7f99eb849b80 1 --2- 172.21.15.7:0/1920189660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586aa49c640 0x5586aa4b34a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f99d0000c00 tx=0x7f99d0012de0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.953+0000 7f99eb849b80 1 -- 172.21.15.7:0/1920189660 shutdown_connections 2024-06-10T01:07:07.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.953+0000 7f99eb849b80 1 --2- 172.21.15.7:0/1920189660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586aa49cf60 0x5586aa4b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.953+0000 7f99eb849b80 1 --2- 172.21.15.7:0/1920189660 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5586aa51b520 0x5586aa51d940 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.953+0000 7f99eb849b80 1 --2- 172.21.15.7:0/1920189660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aa496f20 0x5586aa4b60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.953+0000 7f99eb849b80 1 --2- 172.21.15.7:0/1920189660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586aa49c640 0x5586aa4b34a0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.953+0000 7f99eb849b80 1 -- 172.21.15.7:0/1920189660 >> 172.21.15.7:0/1920189660 conn(0x5586aa3c1cc0 msgr2=0x5586aa49dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:07.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.953+0000 7f99c1ffb700 0 client.15675 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:07:07.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.953+0000 7f99eb849b80 1 -- 172.21.15.7:0/1920189660 shutdown_connections 2024-06-10T01:07:07.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.953+0000 7f99eb849b80 1 -- 172.21.15.7:0/1920189660 wait complete. 2024-06-10T01:07:07.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.954+0000 7f99eb849b80 20 client.15675 trim_cache size 0 max 16384 2024-06-10T01:07:07.955 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:07:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.954+0000 7f99e0fc6700 1 -- 172.21.15.7:0/2919149797 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f99c40a2c80 msgr2=0x7f99c40a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.954+0000 7f99e0fc6700 1 --2- 172.21.15.7:0/2919149797 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f99c40a2c80 0x7f99c40a5140 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f99dc197f60 tx=0x7f99d0009260 comp rx=0 tx=0).stop 2024-06-10T01:07:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.954+0000 7f99e0fc6700 1 -- 172.21.15.7:0/2919149797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99dc101390 msgr2=0x7f99dc19d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.954+0000 7f99e0fc6700 1 --2- 172.21.15.7:0/2919149797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99dc101390 0x7f99dc19d710 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00e9a0 tx=0x7f99cc00ed60 comp rx=0 tx=0).stop 2024-06-10T01:07:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.954+0000 7f99e0fc6700 1 -- 172.21.15.7:0/2919149797 shutdown_connections 2024-06-10T01:07:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.955+0000 7f99e0fc6700 1 --2- 172.21.15.7:0/2919149797 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f99c40a2c80 0x7f99c40a5140 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.955+0000 7f99e0fc6700 1 --2- 172.21.15.7:0/2919149797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99dc107400 0x7f99dc196910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.955+0000 7f99e0fc6700 1 --2- 172.21.15.7:0/2919149797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99dc101390 0x7f99dc19d710 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.955+0000 7f99e0fc6700 1 --2- 172.21.15.7:0/2919149797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99dc101d50 0x7f99dc19dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.955+0000 7f99e0fc6700 1 -- 172.21.15.7:0/2919149797 >> 172.21.15.7:0/2919149797 conn(0x7f99dc078340 msgr2=0x7f99dc0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:07.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.955+0000 7f99e0fc6700 1 -- 172.21.15.7:0/2919149797 shutdown_connections 2024-06-10T01:07:07.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:07.955+0000 7f99e0fc6700 1 -- 172.21.15.7:0/2919149797 wait complete. 2024-06-10T01:07:08.764 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:07:09.028 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:09.405 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:09 smithi007 ceph-mon[32398]: pgmap v465: 129 pgs: 129 active+clean; 672 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 852 B/s wr, 9 op/s 2024-06-10T01:07:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.543+0000 7f5e990fc640 1 -- 172.21.15.7:0/3948544192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e941035c0 msgr2=0x7f5e94109e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.543+0000 7f5e990fc640 1 --2- 172.21.15.7:0/3948544192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e941035c0 0x7f5e94109e50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c009ca0 tx=0x7f5e8c02a8c0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.543+0000 7f5e990fc640 1 -- 172.21.15.7:0/3948544192 shutdown_connections 2024-06-10T01:07:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.543+0000 7f5e990fc640 1 --2- 172.21.15.7:0/3948544192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e941035c0 0x7f5e94109e50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.543+0000 7f5e990fc640 1 --2- 172.21.15.7:0/3948544192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e94102c00 0x7f5e94103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.543+0000 7f5e990fc640 1 --2- 172.21.15.7:0/3948544192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e94101a10 0x7f5e94101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.543+0000 7f5e990fc640 1 -- 172.21.15.7:0/3948544192 >> 172.21.15.7:0/3948544192 conn(0x7f5e940fcfc0 msgr2=0x7f5e940ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.543+0000 7f5e990fc640 1 -- 172.21.15.7:0/3948544192 shutdown_connections 2024-06-10T01:07:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.543+0000 7f5e990fc640 1 -- 172.21.15.7:0/3948544192 wait complete. 2024-06-10T01:07:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.544+0000 7f5e990fc640 1 Processor -- start 2024-06-10T01:07:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.544+0000 7f5e990fc640 1 -- start start 2024-06-10T01:07:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.544+0000 7f5e990fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e94101a10 0x7f5e94075750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.545+0000 7f5e990fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e94102c00 0x7f5e94073d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.545+0000 7f5e990fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e941035c0 0x7f5e940742d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.545+0000 7f5e990fc640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9410d690 con 0x7f5e94101a10 2024-06-10T01:07:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.545+0000 7f5e990fc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9410d510 con 0x7f5e94102c00 2024-06-10T01:07:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.545+0000 7f5e990fc640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9410d810 con 0x7f5e941035c0 2024-06-10T01:07:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.545+0000 7f5e92d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e94101a10 0x7f5e94075750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.545+0000 7f5e92d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e94101a10 0x7f5e94075750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51540/0 (socket says 172.21.15.7:51540) 2024-06-10T01:07:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.545+0000 7f5e92d9d640 1 -- 172.21.15.7:0/2856632734 learned_addr learned my addr 172.21.15.7:0/2856632734 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.545+0000 7f5e9259c640 1 --2- 172.21.15.7:0/2856632734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e94102c00 0x7f5e94073d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.545+0000 7f5e9359e640 1 --2- 172.21.15.7:0/2856632734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e941035c0 0x7f5e940742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.545+0000 7f5e9259c640 1 -- 172.21.15.7:0/2856632734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e941035c0 msgr2=0x7f5e940742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.545+0000 7f5e9259c640 1 --2- 172.21.15.7:0/2856632734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e941035c0 0x7f5e940742d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.545+0000 7f5e9259c640 1 -- 172.21.15.7:0/2856632734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e94101a10 msgr2=0x7f5e94075750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.545+0000 7f5e9259c640 1 --2- 172.21.15.7:0/2856632734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e94101a10 0x7f5e94075750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.545+0000 7f5e9259c640 1 -- 172.21.15.7:0/2856632734 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e94074b60 con 0x7f5e94102c00 2024-06-10T01:07:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.545+0000 7f5e92d9d640 1 --2- 172.21.15.7:0/2856632734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e94101a10 0x7f5e94075750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.545+0000 7f5e9259c640 1 --2- 172.21.15.7:0/2856632734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e94102c00 0x7f5e94073d90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800e9f0 tx=0x7f5e8800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.546+0000 7f5e7bfff640 1 -- 172.21.15.7:0/2856632734 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8800ccc0 con 0x7f5e94102c00 2024-06-10T01:07:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.546+0000 7f5e7bfff640 1 -- 172.21.15.7:0/2856632734 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e88005020 con 0x7f5e94102c00 2024-06-10T01:07:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.546+0000 7f5e7bfff640 1 -- 172.21.15.7:0/2856632734 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e880049e0 con 0x7f5e94102c00 2024-06-10T01:07:09.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.546+0000 7f5e990fc640 1 -- 172.21.15.7:0/2856632734 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e941a1df0 con 0x7f5e94102c00 2024-06-10T01:07:09.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.546+0000 7f5e990fc640 1 -- 172.21.15.7:0/2856632734 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e941a2200 con 0x7f5e94102c00 2024-06-10T01:07:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.548+0000 7f5e9be09740 1 Processor -- start 2024-06-10T01:07:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.548+0000 7f5e9be09740 1 -- start start 2024-06-10T01:07:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.548+0000 7f5e9be09740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3ab267010 0x55e3ab267430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.548+0000 7f5e9be09740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3ab267a00 0x55e3ab26c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.548+0000 7f5e9be09740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3ab26cb70 0x55e3ab26f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.548+0000 7f5e9be09740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e3ab1ca4f0 con 0x55e3ab267010 2024-06-10T01:07:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.548+0000 7f5e9be09740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e3ab168870 con 0x55e3ab26cb70 2024-06-10T01:07:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.548+0000 7f5e9be09740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e3ab1c64b0 con 0x55e3ab267a00 2024-06-10T01:07:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.549+0000 7f5e9359e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3ab267010 0x55e3ab267430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.549+0000 7f5e9359e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3ab267010 0x55e3ab267430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51556/0 (socket says 172.21.15.7:51556) 2024-06-10T01:07:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.549+0000 7f5e9359e640 1 -- 172.21.15.7:0/173908341 learned_addr learned my addr 172.21.15.7:0/173908341 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.549+0000 7f5e92d9d640 1 --2- 172.21.15.7:0/173908341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3ab267a00 0x55e3ab26c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.549+0000 7f5e9359e640 1 -- 172.21.15.7:0/173908341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3ab267a00 msgr2=0x55e3ab26c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.549+0000 7f5e9359e640 1 --2- 172.21.15.7:0/173908341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3ab267a00 0x55e3ab26c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.549+0000 7f5e9359e640 1 -- 172.21.15.7:0/173908341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3ab26cb70 msgr2=0x55e3ab26f000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T01:07:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.549+0000 7f5e9359e640 1 --2- 172.21.15.7:0/173908341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3ab26cb70 0x55e3ab26f000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.549+0000 7f5e9359e640 1 -- 172.21.15.7:0/173908341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3ab1849b0 con 0x55e3ab267010 2024-06-10T01:07:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.549+0000 7f5e7bfff640 1 -- 172.21.15.7:0/2856632734 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f5e88005560 con 0x7f5e94102c00 2024-06-10T01:07:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.549+0000 7f5e9359e640 1 --2- 172.21.15.7:0/173908341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3ab267010 0x55e3ab267430 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c035000 tx=0x7f5e8c009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8e14a1c9a4b42 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.549+0000 7f5e797fa640 1 -- 172.21.15.7:0/173908341 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c031cb0 con 0x55e3ab267010 2024-06-10T01:07:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.550+0000 7f5e797fa640 1 -- 172.21.15.7:0/173908341 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e8c036050 con 0x55e3ab267010 2024-06-10T01:07:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.550+0000 7f5e797fa640 1 -- 172.21.15.7:0/173908341 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c0043c0 con 0x55e3ab267010 2024-06-10T01:07:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.550+0000 7f5e7bfff640 1 --2- 172.21.15.7:0/2856632734 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5e7c0a2b80 0x7f5e7c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.550+0000 7f5e92d9d640 1 --2- 172.21.15.7:0/2856632734 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5e7c0a2b80 0x7f5e7c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.550+0000 7f5e7bfff640 1 -- 172.21.15.7:0/2856632734 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f5e880cf320 con 0x7f5e94102c00 2024-06-10T01:07:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.550+0000 7f5e92d9d640 1 --2- 172.21.15.7:0/2856632734 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5e7c0a2b80 0x7f5e7c0a5040 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55e3ab267e00 tx=0x7f5e84005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.550+0000 7f5e9be09740 1 -- 172.21.15.7:0/173908341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3ab267010 msgr2=0x55e3ab267430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.550+0000 7f5e9be09740 1 --2- 172.21.15.7:0/173908341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3ab267010 0x55e3ab267430 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c035000 tx=0x7f5e8c009360 comp rx=0 tx=0).stop 2024-06-10T01:07:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.550+0000 7f5e9be09740 1 -- 172.21.15.7:0/173908341 shutdown_connections 2024-06-10T01:07:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.550+0000 7f5e9be09740 1 --2- 172.21.15.7:0/173908341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3ab26cb70 0x55e3ab26f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.550+0000 7f5e9be09740 1 --2- 172.21.15.7:0/173908341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3ab267a00 0x55e3ab26c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.550+0000 7f5e9be09740 1 --2- 172.21.15.7:0/173908341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3ab267010 0x55e3ab267430 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.551+0000 7f5e9be09740 1 -- 172.21.15.7:0/173908341 >> 172.21.15.7:0/173908341 conn(0x55e3ab262680 msgr2=0x55e3ab262a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.551+0000 7f5e9be09740 1 -- 172.21.15.7:0/173908341 shutdown_connections 2024-06-10T01:07:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.551+0000 7f5e9be09740 1 -- 172.21.15.7:0/173908341 wait complete. 2024-06-10T01:07:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.551+0000 7f5e9be09740 1 -- start start 2024-06-10T01:07:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.551+0000 7f5e9be09740 1 Processor -- start 2024-06-10T01:07:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.551+0000 7f5e9be09740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3ab267010 0x55e3ab271100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.551+0000 7f5e9be09740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3ab267a00 0x55e3ab271640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.551+0000 7f5e9be09740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3ab26cb70 0x55e3ab278680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.551+0000 7f5e9be09740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e3ab26f940 con 0x55e3ab26cb70 2024-06-10T01:07:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.551+0000 7f5e9be09740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e3ab153ae0 con 0x55e3ab267a00 2024-06-10T01:07:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.551+0000 7f5e9be09740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e3ab26fac0 con 0x55e3ab267010 2024-06-10T01:07:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.551+0000 7f5e9be09740 10 client.0 _ll_register_callbacks cb 0x55e3ab18c9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.551+0000 7f5e9359e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3ab26cb70 0x55e3ab278680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.552+0000 7f5e9359e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3ab26cb70 0x55e3ab278680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51568/0 (socket says 172.21.15.7:51568) 2024-06-10T01:07:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.552+0000 7f5e9359e640 1 -- 172.21.15.7:0/3649925217 learned_addr learned my addr 172.21.15.7:0/3649925217 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.552+0000 7f5e92d9d640 1 --2- 172.21.15.7:0/3649925217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3ab267010 0x55e3ab271100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.552+0000 7f5e9259c640 1 --2- 172.21.15.7:0/3649925217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3ab267a00 0x55e3ab271640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.552+0000 7f5e9359e640 1 -- 172.21.15.7:0/3649925217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3ab267010 msgr2=0x55e3ab271100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.552+0000 7f5e9359e640 1 --2- 172.21.15.7:0/3649925217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3ab267010 0x55e3ab271100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.552+0000 7f5e9359e640 1 -- 172.21.15.7:0/3649925217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3ab267a00 msgr2=0x55e3ab271640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.552+0000 7f5e9359e640 1 --2- 172.21.15.7:0/3649925217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3ab267a00 0x55e3ab271640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.552+0000 7f5e9359e640 1 -- 172.21.15.7:0/3649925217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3aaede210 con 0x55e3ab26cb70 2024-06-10T01:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.552+0000 7f5e9259c640 1 --2- 172.21.15.7:0/3649925217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3ab267a00 0x55e3ab271640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.552+0000 7f5e9359e640 1 --2- 172.21.15.7:0/3649925217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3ab26cb70 0x55e3ab278680 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c02ade0 tx=0x7f5e8c031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b56d553166c260 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.552+0000 7f5e76ff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.552+0000 7f5e76ff5640 1 -- 172.21.15.7:0/3649925217 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c004140 con 0x55e3ab26cb70 2024-06-10T01:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.552+0000 7f5e76ff5640 1 -- 172.21.15.7:0/3649925217 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f5e8c0042e0 con 0x55e3ab26cb70 2024-06-10T01:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.552+0000 7f5e9be09740 1 -- 172.21.15.7:0/3649925217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e8c036050 con 0x55e3ab26cb70 2024-06-10T01:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.552+0000 7f5e76ff5640 1 -- 172.21.15.7:0/3649925217 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5e8c05a050 con 0x55e3ab26cb70 2024-06-10T01:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.552+0000 7f5e9be09740 10 client.15687 fetch_fsmap learned FSMap version 27 2024-06-10T01:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.552+0000 7f5e9be09740 1 -- 172.21.15.7:0/3649925217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e3aafaa620 con 0x55e3ab26cb70 2024-06-10T01:07:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.553+0000 7f5e76ff5640 1 -- 172.21.15.7:0/3649925217 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e8c036050 con 0x55e3ab26cb70 2024-06-10T01:07:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.553+0000 7f5e76ff5640 1 -- 172.21.15.7:0/3649925217 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5e8c04b070 con 0x55e3ab26cb70 2024-06-10T01:07:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.553+0000 7f5e9be09740 10 client.15687 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:07:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.553+0000 7f5e9be09740 10 client.15687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.553+0000 7f5e9be09740 20 client.15687 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.553+0000 7f5e9be09740 1 --2- 172.21.15.7:0/3649925217 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e3ab27cd40 0x55e3ab27f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.553+0000 7f5e9be09740 4 client.15687 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:07:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.553+0000 7f5e9be09740 1 -- 172.21.15.7:0/3649925217 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3ab1ddf10 con 0x55e3ab27cd40 2024-06-10T01:07:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.553+0000 7f5e92d9d640 1 --2- 172.21.15.7:0/3649925217 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e3ab27cd40 0x55e3ab27f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.554+0000 7f5e92d9d640 1 --2- 172.21.15.7:0/3649925217 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e3ab27cd40 0x55e3ab27f130 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b178570c61575258 server_cookie=53c5054388a66c89 in_seq=0 out_seq=0 2024-06-10T01:07:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.554+0000 7f5e76ff5640 10 client.15687 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.554+0000 7f5e76ff5640 1 -- 172.21.15.7:0/3649925217 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e3ab1ddf10 con 0x55e3ab27cd40 2024-06-10T01:07:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.554+0000 7f5e76ff5640 10 client.15687 handle_command_reply: tid=0 2024-06-10T01:07:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.554+0000 7f5e9be09740 1 client.15687 shutdown 2024-06-10T01:07:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.554+0000 7f5e9be09740 1 -- 172.21.15.7:0/3649925217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3ab26cb70 msgr2=0x55e3ab278680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.554+0000 7f5e9be09740 1 --2- 172.21.15.7:0/3649925217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3ab26cb70 0x55e3ab278680 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c02ade0 tx=0x7f5e8c031920 comp rx=0 tx=0).stop 2024-06-10T01:07:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.555+0000 7f5e9be09740 1 -- 172.21.15.7:0/3649925217 shutdown_connections 2024-06-10T01:07:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.555+0000 7f5e9be09740 1 --2- 172.21.15.7:0/3649925217 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e3ab27cd40 0x55e3ab27f130 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.555+0000 7f5e9be09740 1 --2- 172.21.15.7:0/3649925217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3ab26cb70 0x55e3ab278680 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.555+0000 7f5e76ff5640 0 client.15687 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.555+0000 7f5e9be09740 1 --2- 172.21.15.7:0/3649925217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3ab267a00 0x55e3ab271640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.555+0000 7f5e9be09740 1 --2- 172.21.15.7:0/3649925217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3ab267010 0x55e3ab271100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.555+0000 7f5e9be09740 1 -- 172.21.15.7:0/3649925217 >> 172.21.15.7:0/3649925217 conn(0x55e3ab262680 msgr2=0x55e3ab2617e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.555+0000 7f5e9be09740 1 -- 172.21.15.7:0/3649925217 shutdown_connections 2024-06-10T01:07:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.555+0000 7f5e9be09740 1 -- 172.21.15.7:0/3649925217 wait complete. 2024-06-10T01:07:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.555+0000 7f5e9be09740 20 client.15687 trim_cache size 0 max 16384 2024-06-10T01:07:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.570+0000 7f5e9be09740 1 Processor -- start 2024-06-10T01:07:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.570+0000 7f5e9be09740 1 -- start start 2024-06-10T01:07:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.570+0000 7f5e9be09740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3ab262680 0x55e3ab271100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.570+0000 7f5e9be09740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3ab267010 0x55e3ab278640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.570+0000 7f5e9be09740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3ab267a00 0x55e3ab271640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.570+0000 7f5e9be09740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5e941af010 con 0x55e3ab267010 2024-06-10T01:07:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.570+0000 7f5e9be09740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5e941ac8e0 con 0x55e3ab267a00 2024-06-10T01:07:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.570+0000 7f5e9be09740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5e941aa730 con 0x55e3ab262680 2024-06-10T01:07:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.570+0000 7f5e9259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3ab267010 0x55e3ab278640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.571+0000 7f5e9259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3ab267010 0x55e3ab278640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51574/0 (socket says 172.21.15.7:51574) 2024-06-10T01:07:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.571+0000 7f5e9259c640 1 -- 172.21.15.7:0/3342859555 learned_addr learned my addr 172.21.15.7:0/3342859555 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.571+0000 7f5e9359e640 1 --2- 172.21.15.7:0/3342859555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3ab267a00 0x55e3ab271640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.571+0000 7f5e92d9d640 1 --2- 172.21.15.7:0/3342859555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3ab262680 0x55e3ab271100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.571+0000 7f5e9259c640 1 -- 172.21.15.7:0/3342859555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3ab262680 msgr2=0x55e3ab271100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.571+0000 7f5e9259c640 1 --2- 172.21.15.7:0/3342859555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3ab262680 0x55e3ab271100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.571+0000 7f5e9259c640 1 -- 172.21.15.7:0/3342859555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3ab267a00 msgr2=0x55e3ab271640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.571+0000 7f5e9259c640 1 --2- 172.21.15.7:0/3342859555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3ab267a00 0x55e3ab271640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.571+0000 7f5e9259c640 1 -- 172.21.15.7:0/3342859555 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3ab27f4a0 con 0x55e3ab267010 2024-06-10T01:07:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.571+0000 7f5e92d9d640 1 --2- 172.21.15.7:0/3342859555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3ab262680 0x55e3ab271100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:07:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.571+0000 7f5e9359e640 1 --2- 172.21.15.7:0/3342859555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3ab267a00 0x55e3ab271640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.571+0000 7f5e9259c640 1 --2- 172.21.15.7:0/3342859555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3ab267010 0x55e3ab278640 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5e880a3fe0 tx=0x7f5e88005290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03960434cdb1ff7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.571+0000 7f5e78ff9640 1 -- 172.21.15.7:0/3342859555 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8801c030 con 0x55e3ab267010 2024-06-10T01:07:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.571+0000 7f5e78ff9640 1 -- 172.21.15.7:0/3342859555 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e8809fc50 con 0x55e3ab267010 2024-06-10T01:07:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.571+0000 7f5e78ff9640 1 -- 172.21.15.7:0/3342859555 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8809bb00 con 0x55e3ab267010 2024-06-10T01:07:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.572+0000 7f5e9be09740 1 -- 172.21.15.7:0/3342859555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3ab267010 msgr2=0x55e3ab278640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.572+0000 7f5e9be09740 1 --2- 172.21.15.7:0/3342859555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3ab267010 0x55e3ab278640 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5e880a3fe0 tx=0x7f5e88005290 comp rx=0 tx=0).stop 2024-06-10T01:07:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.572+0000 7f5e9be09740 1 -- 172.21.15.7:0/3342859555 shutdown_connections 2024-06-10T01:07:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.572+0000 7f5e9be09740 1 --2- 172.21.15.7:0/3342859555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3ab267a00 0x55e3ab271640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.572+0000 7f5e9be09740 1 --2- 172.21.15.7:0/3342859555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3ab267010 0x55e3ab278640 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.572+0000 7f5e9be09740 1 --2- 172.21.15.7:0/3342859555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3ab262680 0x55e3ab271100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.572+0000 7f5e9be09740 1 -- 172.21.15.7:0/3342859555 >> 172.21.15.7:0/3342859555 conn(0x55e3ab26c640 msgr2=0x55e3ab268c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.572+0000 7f5e9be09740 1 -- 172.21.15.7:0/3342859555 shutdown_connections 2024-06-10T01:07:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.572+0000 7f5e9be09740 1 -- 172.21.15.7:0/3342859555 wait complete. 2024-06-10T01:07:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.573+0000 7f5e9be09740 1 -- start start 2024-06-10T01:07:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.573+0000 7f5e9be09740 1 Processor -- start 2024-06-10T01:07:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.573+0000 7f5e9be09740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3ab262680 0x55e3ab2755a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.573+0000 7f5e9be09740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3ab267010 0x55e3ab275ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.573+0000 7f5e9be09740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3ab267a00 0x55e3ab2e59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.573+0000 7f5e9be09740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5e941aada0 con 0x55e3ab267010 2024-06-10T01:07:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.573+0000 7f5e9be09740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5e941ac210 con 0x55e3ab267a00 2024-06-10T01:07:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.573+0000 7f5e9be09740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5e941ab470 con 0x55e3ab262680 2024-06-10T01:07:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.573+0000 7f5e9be09740 10 client.0 _ll_register_callbacks cb 0x7f5e84009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.573+0000 7f5e9259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3ab267010 0x55e3ab275ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.573+0000 7f5e9259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3ab267010 0x55e3ab275ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51584/0 (socket says 172.21.15.7:51584) 2024-06-10T01:07:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.573+0000 7f5e9259c640 1 -- 172.21.15.7:0/1711247813 learned_addr learned my addr 172.21.15.7:0/1711247813 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.573+0000 7f5e92d9d640 1 --2- 172.21.15.7:0/1711247813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3ab262680 0x55e3ab2755a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.573+0000 7f5e9359e640 1 --2- 172.21.15.7:0/1711247813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3ab267a00 0x55e3ab2e59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.573+0000 7f5e9259c640 1 -- 172.21.15.7:0/1711247813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3ab262680 msgr2=0x55e3ab2755a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.573+0000 7f5e9259c640 1 --2- 172.21.15.7:0/1711247813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3ab262680 0x55e3ab2755a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.573+0000 7f5e9259c640 1 -- 172.21.15.7:0/1711247813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3ab267a00 msgr2=0x55e3ab2e59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.573+0000 7f5e9259c640 1 --2- 172.21.15.7:0/1711247813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3ab267a00 0x55e3ab2e59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.573+0000 7f5e9259c640 1 -- 172.21.15.7:0/1711247813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3ab2e5ef0 con 0x55e3ab267010 2024-06-10T01:07:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.574+0000 7f5e9259c640 1 --2- 172.21.15.7:0/1711247813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3ab267010 0x55e3ab275ae0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5e88004fd0 tx=0x7f5e88005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31317d128b51a38 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.574+0000 7f5e76ff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:07:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.574+0000 7f5e76ff5640 1 -- 172.21.15.7:0/1711247813 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e880b3070 con 0x55e3ab267010 2024-06-10T01:07:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.574+0000 7f5e76ff5640 1 -- 172.21.15.7:0/1711247813 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f5e8809bd00 con 0x55e3ab267010 2024-06-10T01:07:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.574+0000 7f5e9be09740 1 -- 172.21.15.7:0/1711247813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e8809fc50 con 0x55e3ab267010 2024-06-10T01:07:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.574+0000 7f5e76ff5640 1 -- 172.21.15.7:0/1711247813 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5e8801c070 con 0x55e3ab267010 2024-06-10T01:07:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.574+0000 7f5e9be09740 10 client.15693 fetch_fsmap learned FSMap version 27 2024-06-10T01:07:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.574+0000 7f5e9be09740 1 -- 172.21.15.7:0/1711247813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e3ab2e70f0 con 0x55e3ab267010 2024-06-10T01:07:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.574+0000 7f5e76ff5640 1 -- 172.21.15.7:0/1711247813 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e8809fc50 con 0x55e3ab267010 2024-06-10T01:07:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.574+0000 7f5e76ff5640 1 -- 172.21.15.7:0/1711247813 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5e880a4110 con 0x55e3ab267010 2024-06-10T01:07:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.574+0000 7f5e9be09740 10 client.15693 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:07:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.574+0000 7f5e9be09740 10 client.15693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.574+0000 7f5e9be09740 20 client.15693 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.574+0000 7f5e9be09740 1 --2- 172.21.15.7:0/1711247813 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e3ab26cb70 0x55e3ab2e9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.574+0000 7f5e9be09740 4 client.15693 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:07:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.575+0000 7f5e9be09740 1 -- 172.21.15.7:0/1711247813 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e3ab1d7c20 con 0x55e3ab26cb70 2024-06-10T01:07:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.575+0000 7f5e92d9d640 1 --2- 172.21.15.7:0/1711247813 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e3ab26cb70 0x55e3ab2e9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.575+0000 7f5e92d9d640 1 --2- 172.21.15.7:0/1711247813 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e3ab26cb70 0x55e3ab2e9a90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5b2e0800e992c91 server_cookie=3586a2a8de283a0e in_seq=0 out_seq=0 2024-06-10T01:07:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.575+0000 7f5e76ff5640 10 client.15693 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.579+0000 7f5e76ff5640 1 -- 172.21.15.7:0/1711247813 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e3ab1d7c20 con 0x55e3ab26cb70 2024-06-10T01:07:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.579+0000 7f5e76ff5640 10 client.15693 handle_command_reply: tid=0 2024-06-10T01:07:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.579+0000 7f5e9be09740 1 client.15693 shutdown 2024-06-10T01:07:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.579+0000 7f5e9be09740 1 -- 172.21.15.7:0/1711247813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3ab267010 msgr2=0x55e3ab275ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.579+0000 7f5e9be09740 1 --2- 172.21.15.7:0/1711247813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3ab267010 0x55e3ab275ae0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5e88004fd0 tx=0x7f5e88005bd0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.579+0000 7f5e9be09740 1 -- 172.21.15.7:0/1711247813 shutdown_connections 2024-06-10T01:07:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.579+0000 7f5e9be09740 1 --2- 172.21.15.7:0/1711247813 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e3ab26cb70 0x55e3ab2e9a90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.579+0000 7f5e9be09740 1 --2- 172.21.15.7:0/1711247813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3ab267a00 0x55e3ab2e59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.579+0000 7f5e9be09740 1 --2- 172.21.15.7:0/1711247813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3ab267010 0x55e3ab275ae0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.579+0000 7f5e9be09740 1 --2- 172.21.15.7:0/1711247813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3ab262680 0x55e3ab2755a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.579+0000 7f5e76ff5640 0 client.15693 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.579+0000 7f5e9be09740 1 -- 172.21.15.7:0/1711247813 >> 172.21.15.7:0/1711247813 conn(0x55e3ab26c640 msgr2=0x55e3ab268c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.579+0000 7f5e9be09740 1 -- 172.21.15.7:0/1711247813 shutdown_connections 2024-06-10T01:07:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.579+0000 7f5e9be09740 1 -- 172.21.15.7:0/1711247813 wait complete. 2024-06-10T01:07:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.579+0000 7f5e9be09740 20 client.15693 trim_cache size 0 max 16384 2024-06-10T01:07:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:07:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:07:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:07:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.580+0000 7f5e990fc640 1 -- 172.21.15.7:0/2856632734 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5e7c0a2b80 msgr2=0x7f5e7c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.580+0000 7f5e990fc640 1 --2- 172.21.15.7:0/2856632734 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5e7c0a2b80 0x7f5e7c0a5040 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55e3ab267e00 tx=0x7f5e84005f90 comp rx=0 tx=0).stop 2024-06-10T01:07:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.580+0000 7f5e990fc640 1 -- 172.21.15.7:0/2856632734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e94102c00 msgr2=0x7f5e94073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.580+0000 7f5e990fc640 1 --2- 172.21.15.7:0/2856632734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e94102c00 0x7f5e94073d90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800e9f0 tx=0x7f5e8800eeb0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.580+0000 7f5e990fc640 1 -- 172.21.15.7:0/2856632734 shutdown_connections 2024-06-10T01:07:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.580+0000 7f5e990fc640 1 --2- 172.21.15.7:0/2856632734 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5e7c0a2b80 0x7f5e7c0a5040 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.580+0000 7f5e990fc640 1 --2- 172.21.15.7:0/2856632734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e941035c0 0x7f5e940742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.580+0000 7f5e990fc640 1 --2- 172.21.15.7:0/2856632734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e94102c00 0x7f5e94073d90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.580+0000 7f5e990fc640 1 --2- 172.21.15.7:0/2856632734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e94101a10 0x7f5e94075750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.580+0000 7f5e990fc640 1 -- 172.21.15.7:0/2856632734 >> 172.21.15.7:0/2856632734 conn(0x7f5e940fcfc0 msgr2=0x7f5e94105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.580+0000 7f5e990fc640 1 -- 172.21.15.7:0/2856632734 shutdown_connections 2024-06-10T01:07:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:09.580+0000 7f5e990fc640 1 -- 172.21.15.7:0/2856632734 wait complete. 2024-06-10T01:07:09.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:09 smithi175 ceph-mon[36636]: pgmap v465: 129 pgs: 129 active+clean; 672 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 852 B/s wr, 9 op/s 2024-06-10T01:07:09.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:09 smithi078 ceph-mon[36683]: pgmap v465: 129 pgs: 129 active+clean; 672 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 852 B/s wr, 9 op/s 2024-06-10T01:07:11.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.133+0000 7f60b59b7700 1 -- 172.21.15.7:0/1995311611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60b0107840 msgr2=0x7f60b010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:11.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.133+0000 7f60b59b7700 1 --2- 172.21.15.7:0/1995311611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60b0107840 0x7f60b010f3c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f60a800b0d0 tx=0x7f60a800b3e0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.133+0000 7f60b59b7700 1 -- 172.21.15.7:0/1995311611 shutdown_connections 2024-06-10T01:07:11.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.133+0000 7f60b59b7700 1 --2- 172.21.15.7:0/1995311611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60b0107840 0x7f60b010f3c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.133+0000 7f60b59b7700 1 --2- 172.21.15.7:0/1995311611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60b0102740 0x7f60b0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.133+0000 7f60b59b7700 1 --2- 172.21.15.7:0/1995311611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60b01030f0 0x7f60b0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.133+0000 7f60b59b7700 1 -- 172.21.15.7:0/1995311611 >> 172.21.15.7:0/1995311611 conn(0x7f60b00795f0 msgr2=0x7f60b0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:11.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.133+0000 7f60b59b7700 1 -- 172.21.15.7:0/1995311611 shutdown_connections 2024-06-10T01:07:11.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.133+0000 7f60b59b7700 1 -- 172.21.15.7:0/1995311611 wait complete. 2024-06-10T01:07:11.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.133+0000 7f60b59b7700 1 Processor -- start 2024-06-10T01:07:11.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.134+0000 7f60b59b7700 1 -- start start 2024-06-10T01:07:11.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.134+0000 7f60b59b7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60b0102740 0x7f60b0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:11.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.134+0000 7f60b59b7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60b01030f0 0x7f60b0074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:11.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.134+0000 7f60b59b7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60b0107840 0x7f60b0075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:11.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.134+0000 7f60b59b7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f60b0112780 con 0x7f60b01030f0 2024-06-10T01:07:11.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.134+0000 7f60b59b7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f60b0112600 con 0x7f60b0102740 2024-06-10T01:07:11.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.134+0000 7f60b59b7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f60b0112900 con 0x7f60b0107840 2024-06-10T01:07:11.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.134+0000 7f60ae7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60b01030f0 0x7f60b0074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:11.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.134+0000 7f60ae7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60b01030f0 0x7f60b0074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51606/0 (socket says 172.21.15.7:51606) 2024-06-10T01:07:11.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.134+0000 7f60ae7fc700 1 -- 172.21.15.7:0/2428098121 learned_addr learned my addr 172.21.15.7:0/2428098121 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:11.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.134+0000 7f60aeffd700 1 --2- 172.21.15.7:0/2428098121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60b0102740 0x7f60b0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:11.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.134+0000 7f60af7fe700 1 --2- 172.21.15.7:0/2428098121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60b0107840 0x7f60b0075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:11.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.135+0000 7f60ae7fc700 1 -- 172.21.15.7:0/2428098121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60b0107840 msgr2=0x7f60b0075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:11.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.135+0000 7f60ae7fc700 1 --2- 172.21.15.7:0/2428098121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60b0107840 0x7f60b0075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.135+0000 7f60ae7fc700 1 -- 172.21.15.7:0/2428098121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60b0102740 msgr2=0x7f60b0074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:11.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.135+0000 7f60ae7fc700 1 --2- 172.21.15.7:0/2428098121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60b0102740 0x7f60b0074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.135+0000 7f60ae7fc700 1 -- 172.21.15.7:0/2428098121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b01a7060 con 0x7f60b01030f0 2024-06-10T01:07:11.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.135+0000 7f60ae7fc700 1 --2- 172.21.15.7:0/2428098121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60b01030f0 0x7f60b0074cd0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f60a400e9c0 tx=0x7f60a400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:11.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.135+0000 7f60b49b5700 1 -- 172.21.15.7:0/2428098121 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a400cbd0 con 0x7f60b01030f0 2024-06-10T01:07:11.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.135+0000 7f60b49b5700 1 -- 172.21.15.7:0/2428098121 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60a400ee60 con 0x7f60b01030f0 2024-06-10T01:07:11.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.135+0000 7f60b59b7700 1 -- 172.21.15.7:0/2428098121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b01a7350 con 0x7f60b01030f0 2024-06-10T01:07:11.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.135+0000 7f60b49b5700 1 -- 172.21.15.7:0/2428098121 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4005630 con 0x7f60b01030f0 2024-06-10T01:07:11.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.135+0000 7f60b59b7700 1 -- 172.21.15.7:0/2428098121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b01a7840 con 0x7f60b01030f0 2024-06-10T01:07:11.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.138+0000 7f60b49b5700 1 -- 172.21.15.7:0/2428098121 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f60a4021440 con 0x7f60b01030f0 2024-06-10T01:07:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.138+0000 7f60b49b5700 1 --2- 172.21.15.7:0/2428098121 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f60980a2c20 0x7f60980a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.138+0000 7f60aeffd700 1 --2- 172.21.15.7:0/2428098121 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f60980a2c20 0x7f60980a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.138+0000 7f60b49b5700 1 -- 172.21.15.7:0/2428098121 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f60a40cf600 con 0x7f60b01030f0 2024-06-10T01:07:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.139+0000 7f60aeffd700 1 --2- 172.21.15.7:0/2428098121 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f60980a2c20 0x7f60980a50e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f60a0009680 tx=0x7f60a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.139+0000 7f60c023ab80 1 Processor -- start 2024-06-10T01:07:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.139+0000 7f60c023ab80 1 -- start start 2024-06-10T01:07:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.139+0000 7f60c023ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b480e5f60 0x563b480e6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.139+0000 7f60c023ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b480e6880 0x563b480e0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.139+0000 7f60c023ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b480e07a0 0x563b480e0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.139+0000 7f60c023ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563b47fd2f60 con 0x563b480e5f60 2024-06-10T01:07:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.139+0000 7f60c023ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563b48031850 con 0x563b480e07a0 2024-06-10T01:07:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.139+0000 7f60c023ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563b47f99460 con 0x563b480e6880 2024-06-10T01:07:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.140+0000 7f60aeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b480e5f60 0x563b480e6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.140+0000 7f60aeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b480e5f60 0x563b480e6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51616/0 (socket says 172.21.15.7:51616) 2024-06-10T01:07:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.140+0000 7f60aeffd700 1 -- 172.21.15.7:0/25856986 learned_addr learned my addr 172.21.15.7:0/25856986 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.140+0000 7f60af7fe700 1 --2- 172.21.15.7:0/25856986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b480e07a0 0x563b480e0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.140+0000 7f60ae7fc700 1 --2- 172.21.15.7:0/25856986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b480e6880 0x563b480e0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.140+0000 7f60aeffd700 1 -- 172.21.15.7:0/25856986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b480e6880 msgr2=0x563b480e0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.140+0000 7f60aeffd700 1 --2- 172.21.15.7:0/25856986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b480e6880 0x563b480e0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.140+0000 7f60aeffd700 1 -- 172.21.15.7:0/25856986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b480e07a0 msgr2=0x563b480e0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.140+0000 7f60aeffd700 1 --2- 172.21.15.7:0/25856986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b480e07a0 0x563b480e0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.140+0000 7f60aeffd700 1 -- 172.21.15.7:0/25856986 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b47f808d0 con 0x563b480e5f60 2024-06-10T01:07:11.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.140+0000 7f60aeffd700 1 --2- 172.21.15.7:0/25856986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b480e5f60 0x563b480e6340 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f60a0011d70 tx=0x7f60a0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37c7da5364b19b4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:11.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.140+0000 7f6095ffb700 1 -- 172.21.15.7:0/25856986 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a000ebd0 con 0x563b480e5f60 2024-06-10T01:07:11.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.140+0000 7f6095ffb700 1 -- 172.21.15.7:0/25856986 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60a000ed70 con 0x563b480e5f60 2024-06-10T01:07:11.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.140+0000 7f6095ffb700 1 -- 172.21.15.7:0/25856986 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a001d7a0 con 0x563b480e5f60 2024-06-10T01:07:11.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.141+0000 7f60c023ab80 1 -- 172.21.15.7:0/25856986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b480e5f60 msgr2=0x563b480e6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:11.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.141+0000 7f60c023ab80 1 --2- 172.21.15.7:0/25856986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b480e5f60 0x563b480e6340 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f60a0011d70 tx=0x7f60a0014000 comp rx=0 tx=0).stop 2024-06-10T01:07:11.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.141+0000 7f60c023ab80 1 -- 172.21.15.7:0/25856986 shutdown_connections 2024-06-10T01:07:11.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.141+0000 7f60c023ab80 1 --2- 172.21.15.7:0/25856986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b480e07a0 0x563b480e0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.141+0000 7f60c023ab80 1 --2- 172.21.15.7:0/25856986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b480e5f60 0x563b480e6340 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.141+0000 7f60c023ab80 1 --2- 172.21.15.7:0/25856986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b480e6880 0x563b480e0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.141+0000 7f60c023ab80 1 -- 172.21.15.7:0/25856986 >> 172.21.15.7:0/25856986 conn(0x563b4800d080 msgr2=0x563b480db220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:11.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.141+0000 7f60c023ab80 1 -- 172.21.15.7:0/25856986 shutdown_connections 2024-06-10T01:07:11.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.141+0000 7f60c023ab80 1 -- 172.21.15.7:0/25856986 wait complete. 2024-06-10T01:07:11.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.141+0000 7f60c023ab80 1 -- start start 2024-06-10T01:07:11.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.142+0000 7f60c023ab80 1 Processor -- start 2024-06-10T01:07:11.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.142+0000 7f60c023ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b480e07a0 0x563b480ec590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:11.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.142+0000 7f60c023ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b480e5f60 0x563b480e98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:11.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.142+0000 7f60c023ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b480e6880 0x563b480e9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:11.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.142+0000 7f60c023ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563b47fd1fc0 con 0x563b480e5f60 2024-06-10T01:07:11.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.142+0000 7f60c023ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563b4802d810 con 0x563b480e07a0 2024-06-10T01:07:11.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.142+0000 7f60c023ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563b480ed260 con 0x563b480e6880 2024-06-10T01:07:11.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.142+0000 7f60c023ab80 10 client.0 _ll_register_callbacks cb 0x563b47fc75b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:11.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.142+0000 7f60ae7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b480e5f60 0x563b480e98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:11.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.142+0000 7f60ae7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b480e5f60 0x563b480e98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51622/0 (socket says 172.21.15.7:51622) 2024-06-10T01:07:11.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.142+0000 7f60ae7fc700 1 -- 172.21.15.7:0/938610483 learned_addr learned my addr 172.21.15.7:0/938610483 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:11.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.142+0000 7f60aeffd700 1 --2- 172.21.15.7:0/938610483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b480e07a0 0x563b480ec590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:11.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.142+0000 7f60af7fe700 1 --2- 172.21.15.7:0/938610483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b480e6880 0x563b480e9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:11.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.142+0000 7f60ae7fc700 1 -- 172.21.15.7:0/938610483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b480e6880 msgr2=0x563b480e9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:11.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.142+0000 7f60ae7fc700 1 --2- 172.21.15.7:0/938610483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b480e6880 0x563b480e9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.142+0000 7f60ae7fc700 1 -- 172.21.15.7:0/938610483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b480e07a0 msgr2=0x563b480ec590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:11.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.142+0000 7f60ae7fc700 1 --2- 172.21.15.7:0/938610483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b480e07a0 0x563b480ec590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.142+0000 7f60ae7fc700 1 -- 172.21.15.7:0/938610483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b47ff7750 con 0x563b480e5f60 2024-06-10T01:07:11.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.143+0000 7f60ae7fc700 1 --2- 172.21.15.7:0/938610483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b480e5f60 0x563b480e98e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f60a40984b0 tx=0x7f60a409c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6928658e11d7b41 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:11.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.143+0000 7f608effd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:07:11.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.143+0000 7f608effd700 1 -- 172.21.15.7:0/938610483 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a40b3070 con 0x563b480e5f60 2024-06-10T01:07:11.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.143+0000 7f608effd700 1 -- 172.21.15.7:0/938610483 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f60a4016df0 con 0x563b480e5f60 2024-06-10T01:07:11.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.143+0000 7f60c023ab80 1 -- 172.21.15.7:0/938610483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60a000ed70 con 0x563b480e5f60 2024-06-10T01:07:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.143+0000 7f608effd700 1 -- 172.21.15.7:0/938610483 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f60a40a4750 con 0x563b480e5f60 2024-06-10T01:07:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.143+0000 7f60c023ab80 10 client.15705 fetch_fsmap learned FSMap version 27 2024-06-10T01:07:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.143+0000 7f60c023ab80 1 -- 172.21.15.7:0/938610483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563b480f7a10 con 0x563b480e5f60 2024-06-10T01:07:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.143+0000 7f608effd700 1 -- 172.21.15.7:0/938610483 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60a40cf320 con 0x563b480e5f60 2024-06-10T01:07:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.143+0000 7f608effd700 1 -- 172.21.15.7:0/938610483 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f60a40b4ce0 con 0x563b480e5f60 2024-06-10T01:07:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.143+0000 7f60c023ab80 10 client.15705 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:07:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.143+0000 7f60c023ab80 10 client.15705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.143+0000 7f60c023ab80 20 client.15705 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.143+0000 7f60c023ab80 1 --2- 172.21.15.7:0/938610483 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563b480f9fa0 0x563b480fc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.143+0000 7f60c023ab80 4 client.15705 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:07:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.143+0000 7f60c023ab80 1 -- 172.21.15.7:0/938610483 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f60a000ed70 con 0x563b480f9fa0 2024-06-10T01:07:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.144+0000 7f60aeffd700 1 --2- 172.21.15.7:0/938610483 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563b480f9fa0 0x563b480fc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.144+0000 7f60aeffd700 1 --2- 172.21.15.7:0/938610483 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563b480f9fa0 0x563b480fc390 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b10ea43fac4abba9 server_cookie=461e2dddb6332ac in_seq=0 out_seq=0 2024-06-10T01:07:11.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.144+0000 7f608effd700 10 client.15705 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:11.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.145+0000 7f608effd700 1 -- 172.21.15.7:0/938610483 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f60a000ed70 con 0x563b480f9fa0 2024-06-10T01:07:11.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.145+0000 7f608effd700 10 client.15705 handle_command_reply: tid=0 2024-06-10T01:07:11.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.145+0000 7f60c023ab80 1 client.15705 shutdown 2024-06-10T01:07:11.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.145+0000 7f60c023ab80 1 -- 172.21.15.7:0/938610483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b480e5f60 msgr2=0x563b480e98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:11.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.145+0000 7f60c023ab80 1 --2- 172.21.15.7:0/938610483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b480e5f60 0x563b480e98e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f60a40984b0 tx=0x7f60a409c260 comp rx=0 tx=0).stop 2024-06-10T01:07:11.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.145+0000 7f60c023ab80 1 -- 172.21.15.7:0/938610483 shutdown_connections 2024-06-10T01:07:11.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.145+0000 7f60c023ab80 1 --2- 172.21.15.7:0/938610483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b480e6880 0x563b480e9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.145+0000 7f60c023ab80 1 --2- 172.21.15.7:0/938610483 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563b480f9fa0 0x563b480fc390 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.145+0000 7f60c023ab80 1 --2- 172.21.15.7:0/938610483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b480e07a0 0x563b480ec590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.145+0000 7f60c023ab80 1 --2- 172.21.15.7:0/938610483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b480e5f60 0x563b480e98e0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.145+0000 7f60c023ab80 1 -- 172.21.15.7:0/938610483 >> 172.21.15.7:0/938610483 conn(0x563b4800d080 msgr2=0x563b480da700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:11.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.145+0000 7f608effd700 0 client.15705 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:11.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.145+0000 7f60c023ab80 1 -- 172.21.15.7:0/938610483 shutdown_connections 2024-06-10T01:07:11.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.145+0000 7f60c023ab80 1 -- 172.21.15.7:0/938610483 wait complete. 2024-06-10T01:07:11.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.145+0000 7f60c023ab80 20 client.15705 trim_cache size 0 max 16384 2024-06-10T01:07:11.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.161+0000 7f60c023ab80 1 Processor -- start 2024-06-10T01:07:11.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.161+0000 7f60c023ab80 1 -- start start 2024-06-10T01:07:11.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.161+0000 7f60c023ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b480e07a0 0x563b480e0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:11.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.161+0000 7f60c023ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b480e5f60 0x563b480dae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:11.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.161+0000 7f60c023ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b480e6880 0x563b480db3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.161+0000 7f60c023ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01b2de0 con 0x563b480e5f60 2024-06-10T01:07:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.161+0000 7f60c023ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01b0670 con 0x563b480e07a0 2024-06-10T01:07:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.161+0000 7f60c023ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01ae4c0 con 0x563b480e6880 2024-06-10T01:07:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.161+0000 7f60ae7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b480e5f60 0x563b480dae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.161+0000 7f60ae7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b480e5f60 0x563b480dae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51638/0 (socket says 172.21.15.7:51638) 2024-06-10T01:07:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.161+0000 7f60ae7fc700 1 -- 172.21.15.7:0/1384749312 learned_addr learned my addr 172.21.15.7:0/1384749312 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.162+0000 7f60af7fe700 1 --2- 172.21.15.7:0/1384749312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b480e6880 0x563b480db3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.162+0000 7f60aeffd700 1 --2- 172.21.15.7:0/1384749312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b480e07a0 0x563b480e0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:11.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.162+0000 7f60ae7fc700 1 -- 172.21.15.7:0/1384749312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b480e6880 msgr2=0x563b480db3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:11.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.162+0000 7f60ae7fc700 1 --2- 172.21.15.7:0/1384749312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b480e6880 0x563b480db3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.162+0000 7f60ae7fc700 1 -- 172.21.15.7:0/1384749312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b480e07a0 msgr2=0x563b480e0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:11.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.162+0000 7f60ae7fc700 1 --2- 172.21.15.7:0/1384749312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b480e07a0 0x563b480e0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.162+0000 7f60ae7fc700 1 -- 172.21.15.7:0/1384749312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b480e7e50 con 0x563b480e5f60 2024-06-10T01:07:11.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.162+0000 7f60aeffd700 1 --2- 172.21.15.7:0/1384749312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b480e07a0 0x563b480e0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:11.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.162+0000 7f60ae7fc700 1 --2- 172.21.15.7:0/1384749312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b480e5f60 0x563b480dae80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f60a40ccdd0 tx=0x7f60a4099ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349047b58667542e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:11.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.162+0000 7f60957fa700 1 -- 172.21.15.7:0/1384749312 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a40b3070 con 0x563b480e5f60 2024-06-10T01:07:11.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.162+0000 7f60957fa700 1 -- 172.21.15.7:0/1384749312 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60a4016df0 con 0x563b480e5f60 2024-06-10T01:07:11.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.162+0000 7f60957fa700 1 -- 172.21.15.7:0/1384749312 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a40a42c0 con 0x563b480e5f60 2024-06-10T01:07:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.163+0000 7f60c023ab80 1 -- 172.21.15.7:0/1384749312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b480e5f60 msgr2=0x563b480dae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.163+0000 7f60c023ab80 1 --2- 172.21.15.7:0/1384749312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b480e5f60 0x563b480dae80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f60a40ccdd0 tx=0x7f60a4099ff0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.163+0000 7f60c023ab80 1 -- 172.21.15.7:0/1384749312 shutdown_connections 2024-06-10T01:07:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.163+0000 7f60c023ab80 1 --2- 172.21.15.7:0/1384749312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b480e6880 0x563b480db3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.163+0000 7f60c023ab80 1 --2- 172.21.15.7:0/1384749312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b480e07a0 0x563b480e0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.163+0000 7f60c023ab80 1 --2- 172.21.15.7:0/1384749312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b480e5f60 0x563b480dae80 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.163+0000 7f60c023ab80 1 -- 172.21.15.7:0/1384749312 >> 172.21.15.7:0/1384749312 conn(0x563b4800d080 msgr2=0x563b480e81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:11.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.163+0000 7f60c023ab80 1 -- 172.21.15.7:0/1384749312 shutdown_connections 2024-06-10T01:07:11.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.163+0000 7f60c023ab80 1 -- 172.21.15.7:0/1384749312 wait complete. 2024-06-10T01:07:11.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.163+0000 7f60c023ab80 1 -- start start 2024-06-10T01:07:11.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.164+0000 7f60c023ab80 1 Processor -- start 2024-06-10T01:07:11.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.164+0000 7f60c023ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b480e07a0 0x563b480f2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:11.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.164+0000 7f60c023ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b480e5f60 0x563b480f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:11.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.164+0000 7f60c023ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b480e6880 0x563b48101050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:11.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.164+0000 7f60c023ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01aeb30 con 0x563b480e07a0 2024-06-10T01:07:11.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.164+0000 7f60c023ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01affa0 con 0x563b480e6880 2024-06-10T01:07:11.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.164+0000 7f60c023ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01af200 con 0x563b480e5f60 2024-06-10T01:07:11.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.164+0000 7f60c023ab80 10 client.0 _ll_register_callbacks cb 0x563b47fc75b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:11.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.164+0000 7f60aeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b480e07a0 0x563b480f2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:11.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.164+0000 7f60aeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b480e07a0 0x563b480f2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51652/0 (socket says 172.21.15.7:51652) 2024-06-10T01:07:11.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.164+0000 7f60aeffd700 1 -- 172.21.15.7:0/3000265545 learned_addr learned my addr 172.21.15.7:0/3000265545 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:11.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.164+0000 7f60ae7fc700 1 --2- 172.21.15.7:0/3000265545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b480e5f60 0x563b480f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:11.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.164+0000 7f60af7fe700 1 --2- 172.21.15.7:0/3000265545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b480e6880 0x563b48101050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:11.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.165+0000 7f60aeffd700 1 -- 172.21.15.7:0/3000265545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b480e5f60 msgr2=0x563b480f2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:11.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.165+0000 7f60aeffd700 1 --2- 172.21.15.7:0/3000265545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b480e5f60 0x563b480f2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.165+0000 7f60aeffd700 1 -- 172.21.15.7:0/3000265545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b480e6880 msgr2=0x563b48101050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:11.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.165+0000 7f60aeffd700 1 --2- 172.21.15.7:0/3000265545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b480e6880 0x563b48101050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.165+0000 7f60aeffd700 1 -- 172.21.15.7:0/3000265545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b481016c0 con 0x563b480e07a0 2024-06-10T01:07:11.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.165+0000 7f60aeffd700 1 --2- 172.21.15.7:0/3000265545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b480e07a0 0x563b480f2730 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f60a0011d10 tx=0x7f60a001cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adab7ef6f58dd7bb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:11.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.165+0000 7f608effd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:07:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.165+0000 7f608effd700 1 -- 172.21.15.7:0/3000265545 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a000ec10 con 0x563b480e07a0 2024-06-10T01:07:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.165+0000 7f608effd700 1 -- 172.21.15.7:0/3000265545 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f60a000edb0 con 0x563b480e07a0 2024-06-10T01:07:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.165+0000 7f60c023ab80 1 -- 172.21.15.7:0/3000265545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60a4016df0 con 0x563b480e07a0 2024-06-10T01:07:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.165+0000 7f608effd700 1 -- 172.21.15.7:0/3000265545 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f60a0005040 con 0x563b480e07a0 2024-06-10T01:07:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.165+0000 7f60c023ab80 10 client.15711 fetch_fsmap learned FSMap version 27 2024-06-10T01:07:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.165+0000 7f60c023ab80 1 -- 172.21.15.7:0/3000265545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563b48164570 con 0x563b480e07a0 2024-06-10T01:07:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.166+0000 7f608effd700 1 -- 172.21.15.7:0/3000265545 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60a4016df0 con 0x563b480e07a0 2024-06-10T01:07:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.166+0000 7f608effd700 1 -- 172.21.15.7:0/3000265545 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f60a0038070 con 0x563b480e07a0 2024-06-10T01:07:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.166+0000 7f60c023ab80 10 client.15711 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:07:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.166+0000 7f60c023ab80 10 client.15711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.166+0000 7f60c023ab80 20 client.15711 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.166+0000 7f60c023ab80 1 --2- 172.21.15.7:0/3000265545 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563b481648a0 0x563b48166c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.166+0000 7f60c023ab80 4 client.15711 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:07:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.166+0000 7f60c023ab80 1 -- 172.21.15.7:0/3000265545 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563b4803ed50 con 0x563b481648a0 2024-06-10T01:07:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.166+0000 7f60ae7fc700 1 --2- 172.21.15.7:0/3000265545 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563b481648a0 0x563b48166c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.167+0000 7f60ae7fc700 1 --2- 172.21.15.7:0/3000265545 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563b481648a0 0x563b48166c90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59a88aba29b6ebfa server_cookie=6b682a2e59b3b35c in_seq=0 out_seq=0 2024-06-10T01:07:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.167+0000 7f608effd700 10 client.15711 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.167+0000 7f608effd700 1 -- 172.21.15.7:0/3000265545 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563b4803ed50 con 0x563b481648a0 2024-06-10T01:07:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.167+0000 7f608effd700 10 client.15711 handle_command_reply: tid=0 2024-06-10T01:07:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.167+0000 7f60c023ab80 1 client.15711 shutdown 2024-06-10T01:07:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.167+0000 7f60c023ab80 1 -- 172.21.15.7:0/3000265545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b480e07a0 msgr2=0x563b480f2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.167+0000 7f60c023ab80 1 --2- 172.21.15.7:0/3000265545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b480e07a0 0x563b480f2730 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f60a0011d10 tx=0x7f60a001cdf0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.167+0000 7f60c023ab80 1 -- 172.21.15.7:0/3000265545 shutdown_connections 2024-06-10T01:07:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.167+0000 7f60c023ab80 1 --2- 172.21.15.7:0/3000265545 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563b481648a0 0x563b48166c90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.167+0000 7f60c023ab80 1 --2- 172.21.15.7:0/3000265545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b480e6880 0x563b48101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.167+0000 7f60c023ab80 1 --2- 172.21.15.7:0/3000265545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b480e07a0 0x563b480f2730 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.167+0000 7f60c023ab80 1 --2- 172.21.15.7:0/3000265545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b480e5f60 0x563b480f2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.167+0000 7f60c023ab80 1 -- 172.21.15.7:0/3000265545 >> 172.21.15.7:0/3000265545 conn(0x563b4800d080 msgr2=0x563b480e6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:11.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.167+0000 7f608effd700 0 client.15711 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:11.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.167+0000 7f60c023ab80 1 -- 172.21.15.7:0/3000265545 shutdown_connections 2024-06-10T01:07:11.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.167+0000 7f60c023ab80 1 -- 172.21.15.7:0/3000265545 wait complete. 2024-06-10T01:07:11.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.167+0000 7f60c023ab80 20 client.15711 trim_cache size 0 max 16384 2024-06-10T01:07:11.169 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:07:11.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.168+0000 7f60b59b7700 1 -- 172.21.15.7:0/2428098121 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f60980a2c20 msgr2=0x7f60980a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:11.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.168+0000 7f60b59b7700 1 --2- 172.21.15.7:0/2428098121 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f60980a2c20 0x7f60980a50e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f60a0009680 tx=0x7f60a0009260 comp rx=0 tx=0).stop 2024-06-10T01:07:11.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.168+0000 7f60b59b7700 1 -- 172.21.15.7:0/2428098121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60b01030f0 msgr2=0x7f60b0074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:11.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.168+0000 7f60b59b7700 1 --2- 172.21.15.7:0/2428098121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60b01030f0 0x7f60b0074cd0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f60a400e9c0 tx=0x7f60a400ecd0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.168+0000 7f60b59b7700 1 -- 172.21.15.7:0/2428098121 shutdown_connections 2024-06-10T01:07:11.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.168+0000 7f60b59b7700 1 --2- 172.21.15.7:0/2428098121 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f60980a2c20 0x7f60980a50e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.168+0000 7f60b59b7700 1 --2- 172.21.15.7:0/2428098121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60b0107840 0x7f60b0075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.168+0000 7f60b59b7700 1 --2- 172.21.15.7:0/2428098121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60b0102740 0x7f60b0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.168+0000 7f60b59b7700 1 --2- 172.21.15.7:0/2428098121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60b01030f0 0x7f60b0074cd0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:11.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.168+0000 7f60b59b7700 1 -- 172.21.15.7:0/2428098121 >> 172.21.15.7:0/2428098121 conn(0x7f60b00795f0 msgr2=0x7f60b0100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:11.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.168+0000 7f60b59b7700 1 -- 172.21.15.7:0/2428098121 shutdown_connections 2024-06-10T01:07:11.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:11.168+0000 7f60b59b7700 1 -- 172.21.15.7:0/2428098121 wait complete. 2024-06-10T01:07:11.496 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:07:11.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:11 smithi175 ceph-mon[36636]: pgmap v466: 129 pgs: 129 active+clean; 672 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 852 B/s wr, 8 op/s 2024-06-10T01:07:11.732 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:11 smithi007 ceph-mon[32398]: pgmap v466: 129 pgs: 129 active+clean; 672 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 852 B/s wr, 8 op/s 2024-06-10T01:07:11.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:11 smithi078 ceph-mon[36683]: pgmap v466: 129 pgs: 129 active+clean; 672 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 852 B/s wr, 8 op/s 2024-06-10T01:07:11.852 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 91% (11569/12576) Updating files: 92% (11570/12576) Updating files: 93% (11696/12576) Updating files: 94% (11822/12576) Updating files: 95% (11948/12576) Updating files: 96% (12073/12576) Updating files: 97% (12199/12576) Updating files: 98% (12325/12576) Updating files: 99% (12451/12576) Updating files: 100% (12576/12576) Updating files: 100% (12576/12576), done. 2024-06-10T01:07:11.918 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:12.696 INFO:tasks.workunit.client.0.smithi007.stderr:Note: switching to '7b7a5d375cf4e4afab96b9d4d229b699ae8304bb'. 2024-06-10T01:07:12.696 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-06-10T01:07:12.696 INFO:tasks.workunit.client.0.smithi007.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-06-10T01:07:12.696 INFO:tasks.workunit.client.0.smithi007.stderr:changes and commit them, and you can discard any commits you make in this 2024-06-10T01:07:12.696 INFO:tasks.workunit.client.0.smithi007.stderr:state without impacting any branches by switching back to a branch. 2024-06-10T01:07:12.697 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-06-10T01:07:12.697 INFO:tasks.workunit.client.0.smithi007.stderr:If you want to create a new branch to retain commits you create, you may 2024-06-10T01:07:12.697 INFO:tasks.workunit.client.0.smithi007.stderr:do so (now or later) by using -c with the switch command. Example: 2024-06-10T01:07:12.697 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-06-10T01:07:12.697 INFO:tasks.workunit.client.0.smithi007.stderr: git switch -c 2024-06-10T01:07:12.697 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-06-10T01:07:12.697 INFO:tasks.workunit.client.0.smithi007.stderr:Or undo this operation with: 2024-06-10T01:07:12.697 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-06-10T01:07:12.697 INFO:tasks.workunit.client.0.smithi007.stderr: git switch - 2024-06-10T01:07:12.697 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-06-10T01:07:12.697 INFO:tasks.workunit.client.0.smithi007.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-06-10T01:07:12.697 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-06-10T01:07:12.697 INFO:tasks.workunit.client.0.smithi007.stderr:HEAD is now at 7b7a5d375cf Merge pull request #57941 from zdover23/wip-doc-2024-06-09-backport-57939-to-squid 2024-06-10T01:07:12.701 DEBUG:teuthology.orchestra.run.smithi007:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-06-10T01:07:12.801 INFO:tasks.workunit.client.0.smithi007.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-06-10T01:07:12.804 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-10T01:07:12.804 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-06-10T01:07:12.899 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:07:12.899 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:07:13.696 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:13 smithi007 ceph-mon[32398]: pgmap v467: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 937 B/s wr, 8 op/s 2024-06-10T01:07:13.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:13 smithi175 ceph-mon[36636]: pgmap v467: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 937 B/s wr, 8 op/s 2024-06-10T01:07:13.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:13 smithi078 ceph-mon[36683]: pgmap v467: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 937 B/s wr, 8 op/s 2024-06-10T01:07:14.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.297+0000 7fe39f8d7700 1 -- 172.21.15.7:0/1234193748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe398108500 msgr2=0x7fe3981088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:14.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.297+0000 7fe39f8d7700 1 --2- 172.21.15.7:0/1234193748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe398108500 0x7fe3981088e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe39400b0a0 tx=0x7fe39400b3b0 comp rx=0 tx=0).stop 2024-06-10T01:07:14.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.297+0000 7fe39f8d7700 1 -- 172.21.15.7:0/1234193748 shutdown_connections 2024-06-10T01:07:14.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.297+0000 7fe39f8d7700 1 --2- 172.21.15.7:0/1234193748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe39806ab30 0x7fe39810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:14.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.297+0000 7fe39f8d7700 1 --2- 172.21.15.7:0/1234193748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe398108500 0x7fe3981088e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:14.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.297+0000 7fe39f8d7700 1 --2- 172.21.15.7:0/1234193748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe39806a170 0x7fe39806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:14.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.297+0000 7fe39f8d7700 1 -- 172.21.15.7:0/1234193748 >> 172.21.15.7:0/1234193748 conn(0x7fe398078340 msgr2=0x7fe398078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:14.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.297+0000 7fe39f8d7700 1 -- 172.21.15.7:0/1234193748 shutdown_connections 2024-06-10T01:07:14.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.297+0000 7fe39f8d7700 1 -- 172.21.15.7:0/1234193748 wait complete. 2024-06-10T01:07:14.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.298+0000 7fe39f8d7700 1 Processor -- start 2024-06-10T01:07:14.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.298+0000 7fe39f8d7700 1 -- start start 2024-06-10T01:07:14.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.298+0000 7fe39f8d7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe39806a170 0x7fe39819e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.298+0000 7fe39f8d7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe39806ab30 0x7fe39819ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.299+0000 7fe39f8d7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe398108500 0x7fe398198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.299+0000 7fe39f8d7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe3981127d0 con 0x7fe398108500 2024-06-10T01:07:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.299+0000 7fe39f8d7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe398112650 con 0x7fe39806a170 2024-06-10T01:07:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.299+0000 7fe39f8d7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe398112950 con 0x7fe39806ab30 2024-06-10T01:07:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.299+0000 7fe39de74700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe398108500 0x7fe398198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.299+0000 7fe39de74700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe398108500 0x7fe398198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32832/0 (socket says 172.21.15.7:32832) 2024-06-10T01:07:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.299+0000 7fe39de74700 1 -- 172.21.15.7:0/64770557 learned_addr learned my addr 172.21.15.7:0/64770557 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.299+0000 7fe39d673700 1 --2- 172.21.15.7:0/64770557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe39806a170 0x7fe39819e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.299+0000 7fe39ce72700 1 --2- 172.21.15.7:0/64770557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe39806ab30 0x7fe39819ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.299+0000 7fe39de74700 1 -- 172.21.15.7:0/64770557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe39806ab30 msgr2=0x7fe39819ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:14.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.299+0000 7fe39de74700 1 --2- 172.21.15.7:0/64770557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe39806ab30 0x7fe39819ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:14.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.299+0000 7fe39de74700 1 -- 172.21.15.7:0/64770557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe39806a170 msgr2=0x7fe39819e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:14.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.299+0000 7fe39de74700 1 --2- 172.21.15.7:0/64770557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe39806a170 0x7fe39819e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:14.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.299+0000 7fe39de74700 1 -- 172.21.15.7:0/64770557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe398199270 con 0x7fe398108500 2024-06-10T01:07:14.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.299+0000 7fe39de74700 1 --2- 172.21.15.7:0/64770557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe398108500 0x7fe398198b10 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe39000c900 tx=0x7fe39000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:14.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.299+0000 7fe38e7fc700 1 -- 172.21.15.7:0/64770557 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3900078d0 con 0x7fe398108500 2024-06-10T01:07:14.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.299+0000 7fe38e7fc700 1 -- 172.21.15.7:0/64770557 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe39000f450 con 0x7fe398108500 2024-06-10T01:07:14.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.299+0000 7fe39f8d7700 1 -- 172.21.15.7:0/64770557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe398199560 con 0x7fe398108500 2024-06-10T01:07:14.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.299+0000 7fe38e7fc700 1 -- 172.21.15.7:0/64770557 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe390018660 con 0x7fe398108500 2024-06-10T01:07:14.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.299+0000 7fe39f8d7700 1 -- 172.21.15.7:0/64770557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3981a57d0 con 0x7fe398108500 2024-06-10T01:07:14.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.300+0000 7fe39f8d7700 1 -- 172.21.15.7:0/64770557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe39810a9e0 con 0x7fe398108500 2024-06-10T01:07:14.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.302+0000 7fe38e7fc700 1 -- 172.21.15.7:0/64770557 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fe390007a70 con 0x7fe398108500 2024-06-10T01:07:14.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.302+0000 7fe38e7fc700 1 --2- 172.21.15.7:0/64770557 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe3840a2c20 0x7fe3840a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:14.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.303+0000 7fe39d673700 1 --2- 172.21.15.7:0/64770557 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe3840a2c20 0x7fe3840a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:14.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.303+0000 7fe38e7fc700 1 -- 172.21.15.7:0/64770557 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fe3900cf410 con 0x7fe398108500 2024-06-10T01:07:14.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.303+0000 7fe39d673700 1 --2- 172.21.15.7:0/64770557 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe3840a2c20 0x7fe3840a50e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe398107e10 tx=0x7fe394006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:14.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.306+0000 7fe38e7fc700 1 -- 172.21.15.7:0/64770557 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe390014030 con 0x7fe398108500 2024-06-10T01:07:14.308 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:14.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.574+0000 7fe39f8d7700 1 -- 172.21.15.7:0/64770557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe398199ff0 con 0x7fe398108500 2024-06-10T01:07:14.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.575+0000 7fe38e7fc700 1 -- 172.21.15.7:0/64770557 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11855 (secure 0 0 0) 0x7fe39009bf90 con 0x7fe398108500 2024-06-10T01:07:14.572 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:07:14.572 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:05:39.175358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/2784910673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2784910673},{"type":"v1","addr":"172.21.15.175:6835","nonce":2784910673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24434,24436,24451]},"id":1}]} 2024-06-10T01:07:14.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.577+0000 7fe39f8d7700 1 -- 172.21.15.7:0/64770557 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe3840a2c20 msgr2=0x7fe3840a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.577+0000 7fe39f8d7700 1 --2- 172.21.15.7:0/64770557 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe3840a2c20 0x7fe3840a50e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe398107e10 tx=0x7fe394006210 comp rx=0 tx=0).stop 2024-06-10T01:07:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.577+0000 7fe39f8d7700 1 -- 172.21.15.7:0/64770557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe398108500 msgr2=0x7fe398198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.577+0000 7fe39f8d7700 1 --2- 172.21.15.7:0/64770557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe398108500 0x7fe398198b10 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe39000c900 tx=0x7fe39000cc10 comp rx=0 tx=0).stop 2024-06-10T01:07:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.577+0000 7fe39f8d7700 1 -- 172.21.15.7:0/64770557 shutdown_connections 2024-06-10T01:07:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.577+0000 7fe39f8d7700 1 --2- 172.21.15.7:0/64770557 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe3840a2c20 0x7fe3840a50e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.577+0000 7fe39f8d7700 1 --2- 172.21.15.7:0/64770557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe398108500 0x7fe398198b10 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.577+0000 7fe39f8d7700 1 --2- 172.21.15.7:0/64770557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe39806a170 0x7fe39819e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.577+0000 7fe39f8d7700 1 --2- 172.21.15.7:0/64770557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe39806ab30 0x7fe39819ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.577+0000 7fe39f8d7700 1 -- 172.21.15.7:0/64770557 >> 172.21.15.7:0/64770557 conn(0x7fe398078340 msgr2=0x7fe3980fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.577+0000 7fe39f8d7700 1 -- 172.21.15.7:0/64770557 shutdown_connections 2024-06-10T01:07:14.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:14.577+0000 7fe39f8d7700 1 -- 172.21.15.7:0/64770557 wait complete. 2024-06-10T01:07:14.575 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-06-10T01:07:15.233 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-06-10T01:07:15.339 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-06-10T01:07:15.379 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-10T01:07:15.381 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-10T01:07:15.381 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-06-10T01:07:15.421 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-10T01:07:15.425 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-10T01:07:15.425 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-06-10T01:07:15.442 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-06-10T01:07:15.443 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-06-10T01:07:15.443 DEBUG:teuthology.orchestra.run.smithi007:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7b7a5d375cf4e4afab96b9d4d229b699ae8304bb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-06-10T01:07:15.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:15 smithi175 ceph-mon[36636]: pgmap v468: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-06-10T01:07:15.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:15 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/64770557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:07:15.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:15 smithi007 ceph-mon[32398]: pgmap v468: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-06-10T01:07:15.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:15 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/64770557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:07:15.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:15 smithi078 ceph-mon[36683]: pgmap v468: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-06-10T01:07:15.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:15 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/64770557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:07:15.986 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db --set-id 96f9e47b --release --await' 2024-06-10T01:07:15.987 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db --set-id 96f9e47b --release --await 2024-06-10T01:07:17.108 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:17.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.124+0000 7ff4e2dc3700 1 -- 172.21.15.7:0/2703276268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4dc1085d0 msgr2=0x7ff4dc1089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:17.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.124+0000 7ff4e2dc3700 1 --2- 172.21.15.7:0/2703276268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4dc1085d0 0x7ff4dc1089b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00b0d0 tx=0x7ff4cc00b3e0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.124+0000 7ff4e2dc3700 1 -- 172.21.15.7:0/2703276268 shutdown_connections 2024-06-10T01:07:17.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.124+0000 7ff4e2dc3700 1 --2- 172.21.15.7:0/2703276268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4dc102ef0 0x7ff4dc10f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.124+0000 7ff4e2dc3700 1 --2- 172.21.15.7:0/2703276268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4dc1085d0 0x7ff4dc1089b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.124+0000 7ff4e2dc3700 1 --2- 172.21.15.7:0/2703276268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4dc102530 0x7ff4dc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.124+0000 7ff4e2dc3700 1 -- 172.21.15.7:0/2703276268 >> 172.21.15.7:0/2703276268 conn(0x7ff4dc079640 msgr2=0x7ff4dc079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:17.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.124+0000 7ff4e2dc3700 1 -- 172.21.15.7:0/2703276268 shutdown_connections 2024-06-10T01:07:17.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.124+0000 7ff4e2dc3700 1 -- 172.21.15.7:0/2703276268 wait complete. 2024-06-10T01:07:17.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.124+0000 7ff4e2dc3700 1 Processor -- start 2024-06-10T01:07:17.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.125+0000 7ff4e2dc3700 1 -- start start 2024-06-10T01:07:17.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.125+0000 7ff4e2dc3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4dc102530 0x7ff4dc19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.125+0000 7ff4e2dc3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4dc102ef0 0x7ff4dc19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.125+0000 7ff4e2dc3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4dc1085d0 0x7ff4dc198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.125+0000 7ff4e2dc3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc112860 con 0x7ff4dc1085d0 2024-06-10T01:07:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.125+0000 7ff4e2dc3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc1126e0 con 0x7ff4dc102ef0 2024-06-10T01:07:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.125+0000 7ff4e2dc3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc1129e0 con 0x7ff4dc102530 2024-06-10T01:07:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.125+0000 7ff4e1360700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4dc1085d0 0x7ff4dc198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.125+0000 7ff4e0b5f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4dc102530 0x7ff4dc19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.125+0000 7ff4e0b5f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4dc102530 0x7ff4dc19e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43522/0 (socket says 172.21.15.7:43522) 2024-06-10T01:07:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.125+0000 7ff4dbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4dc102ef0 0x7ff4dc19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.125+0000 7ff4e1360700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4dc1085d0 0x7ff4dc198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32846/0 (socket says 172.21.15.7:32846) 2024-06-10T01:07:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.125+0000 7ff4e0b5f700 1 -- 172.21.15.7:0/2571635347 learned_addr learned my addr 172.21.15.7:0/2571635347 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.126+0000 7ff4e1360700 1 -- 172.21.15.7:0/2571635347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4dc102530 msgr2=0x7ff4dc19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.126+0000 7ff4e1360700 1 --2- 172.21.15.7:0/2571635347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4dc102530 0x7ff4dc19e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.126+0000 7ff4e1360700 1 -- 172.21.15.7:0/2571635347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4dc102ef0 msgr2=0x7ff4dc19ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.126+0000 7ff4e1360700 1 --2- 172.21.15.7:0/2571635347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4dc102ef0 0x7ff4dc19ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.126+0000 7ff4e1360700 1 -- 172.21.15.7:0/2571635347 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4dc1991c0 con 0x7ff4dc1085d0 2024-06-10T01:07:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.126+0000 7ff4dbfff700 1 --2- 172.21.15.7:0/2571635347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4dc102ef0 0x7ff4dc19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.126+0000 7ff4e0b5f700 1 --2- 172.21.15.7:0/2571635347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4dc102530 0x7ff4dc19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.126+0000 7ff4e1360700 1 --2- 172.21.15.7:0/2571635347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4dc1085d0 0x7ff4dc198a60 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff4d400d750 tx=0x7ff4d400da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.126+0000 7ff4d9ffb700 1 -- 172.21.15.7:0/2571635347 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4014030 con 0x7ff4dc1085d0 2024-06-10T01:07:17.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.126+0000 7ff4d9ffb700 1 -- 172.21.15.7:0/2571635347 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4d4004500 con 0x7ff4dc1085d0 2024-06-10T01:07:17.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.126+0000 7ff4e2dc3700 1 -- 172.21.15.7:0/2571635347 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4dc1994b0 con 0x7ff4dc1085d0 2024-06-10T01:07:17.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.126+0000 7ff4d9ffb700 1 -- 172.21.15.7:0/2571635347 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4005490 con 0x7ff4dc1085d0 2024-06-10T01:07:17.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.127+0000 7ff4e2dc3700 1 -- 172.21.15.7:0/2571635347 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4dc0746c0 con 0x7ff4dc1085d0 2024-06-10T01:07:17.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.129+0000 7ff4d9ffb700 1 -- 172.21.15.7:0/2571635347 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7ff4d4004750 con 0x7ff4dc1085d0 2024-06-10T01:07:17.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.130+0000 7ff4d9ffb700 1 --2- 172.21.15.7:0/2571635347 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff4c80a2ca0 0x7ff4c80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:17.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.130+0000 7ff4e0b5f700 1 --2- 172.21.15.7:0/2571635347 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff4c80a2ca0 0x7ff4c80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:17.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.130+0000 7ff4d9ffb700 1 -- 172.21.15.7:0/2571635347 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7ff4d40ceb90 con 0x7ff4dc1085d0 2024-06-10T01:07:17.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.131+0000 7ff4e0b5f700 1 --2- 172.21.15.7:0/2571635347 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff4c80a2ca0 0x7ff4c80a5160 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009fd0 tx=0x7ff4cc006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:17.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.131+0000 7ff4ed646b80 1 Processor -- start 2024-06-10T01:07:17.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.131+0000 7ff4ed646b80 1 -- start start 2024-06-10T01:07:17.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.132+0000 7ff4ed646b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567488ed6c0 0x5567488edaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:17.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.132+0000 7ff4ed646b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567488edfe0 0x5567488e7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:17.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.132+0000 7ff4ed646b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567488e7fa0 0x5567488e83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:17.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.132+0000 7ff4ed646b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556748839890 con 0x5567488e7fa0 2024-06-10T01:07:17.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.132+0000 7ff4ed646b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5567487e17a0 con 0x5567488edfe0 2024-06-10T01:07:17.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.132+0000 7ff4ed646b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556748835850 con 0x5567488ed6c0 2024-06-10T01:07:17.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.132+0000 7ff4e1360700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567488e7fa0 0x5567488e83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:17.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.132+0000 7ff4e0b5f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567488ed6c0 0x5567488edaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:17.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.132+0000 7ff4e1360700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567488e7fa0 0x5567488e83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32848/0 (socket says 172.21.15.7:32848) 2024-06-10T01:07:17.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.132+0000 7ff4e1360700 1 -- 172.21.15.7:0/3291935321 learned_addr learned my addr 172.21.15.7:0/3291935321 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:17.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.132+0000 7ff4dbfff700 1 --2- 172.21.15.7:0/3291935321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567488edfe0 0x5567488e7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:17.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.132+0000 7ff4dbfff700 1 -- 172.21.15.7:0/3291935321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567488ed6c0 msgr2=0x5567488edaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:17.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.133+0000 7ff4dbfff700 1 --2- 172.21.15.7:0/3291935321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567488ed6c0 0x5567488edaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.133+0000 7ff4dbfff700 1 -- 172.21.15.7:0/3291935321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567488e7fa0 msgr2=0x5567488e83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:17.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.133+0000 7ff4dbfff700 1 --2- 172.21.15.7:0/3291935321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567488e7fa0 0x5567488e83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.133+0000 7ff4dbfff700 1 -- 172.21.15.7:0/3291935321 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567487d7fd0 con 0x5567488edfe0 2024-06-10T01:07:17.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.133+0000 7ff4e0b5f700 1 --2- 172.21.15.7:0/3291935321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567488ed6c0 0x5567488edaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:17.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.133+0000 7ff4e1360700 1 --2- 172.21.15.7:0/3291935321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567488e7fa0 0x5567488e83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:17.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.133+0000 7ff4dbfff700 1 --2- 172.21.15.7:0/3291935321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567488edfe0 0x5567488e7a60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400f3c0 tx=0x7ff4c400f6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d616d0ad0f644519 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:17.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.133+0000 7ff4d2ffd700 1 -- 172.21.15.7:0/3291935321 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4004a30 con 0x5567488edfe0 2024-06-10T01:07:17.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.133+0000 7ff4d2ffd700 1 -- 172.21.15.7:0/3291935321 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4c4004bd0 con 0x5567488edfe0 2024-06-10T01:07:17.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.133+0000 7ff4d2ffd700 1 -- 172.21.15.7:0/3291935321 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c40188b0 con 0x5567488edfe0 2024-06-10T01:07:17.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.134+0000 7ff4ed646b80 1 -- 172.21.15.7:0/3291935321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567488edfe0 msgr2=0x5567488e7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:17.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.134+0000 7ff4ed646b80 1 --2- 172.21.15.7:0/3291935321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567488edfe0 0x5567488e7a60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400f3c0 tx=0x7ff4c400f6d0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.134+0000 7ff4ed646b80 1 -- 172.21.15.7:0/3291935321 shutdown_connections 2024-06-10T01:07:17.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.134+0000 7ff4ed646b80 1 --2- 172.21.15.7:0/3291935321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567488e7fa0 0x5567488e83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.134+0000 7ff4ed646b80 1 --2- 172.21.15.7:0/3291935321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567488ed6c0 0x5567488edaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.134+0000 7ff4ed646b80 1 --2- 172.21.15.7:0/3291935321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567488edfe0 0x5567488e7a60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.134+0000 7ff4ed646b80 1 -- 172.21.15.7:0/3291935321 >> 172.21.15.7:0/3291935321 conn(0x556748812d40 msgr2=0x5567488e28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:17.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.134+0000 7ff4ed646b80 1 -- 172.21.15.7:0/3291935321 shutdown_connections 2024-06-10T01:07:17.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.134+0000 7ff4ed646b80 1 -- 172.21.15.7:0/3291935321 wait complete. 2024-06-10T01:07:17.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.135+0000 7ff4ed646b80 1 -- start start 2024-06-10T01:07:17.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.135+0000 7ff4ed646b80 1 Processor -- start 2024-06-10T01:07:17.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.135+0000 7ff4ed646b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567488e7fa0 0x5567488f3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:17.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.135+0000 7ff4ed646b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567488ed6c0 0x5567488f1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:17.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.135+0000 7ff4ed646b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567488edfe0 0x5567488f15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:17.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.135+0000 7ff4ed646b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5567488f4aa0 con 0x5567488e7fa0 2024-06-10T01:07:17.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.135+0000 7ff4ed646b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5567487d9c70 con 0x5567488edfe0 2024-06-10T01:07:17.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.135+0000 7ff4ed646b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5567488f4c20 con 0x5567488ed6c0 2024-06-10T01:07:17.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.135+0000 7ff4ed646b80 10 client.0 _ll_register_callbacks cb 0x5567487d39a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:17.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.135+0000 7ff4e0b5f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567488e7fa0 0x5567488f3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:17.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.135+0000 7ff4e0b5f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567488e7fa0 0x5567488f3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32864/0 (socket says 172.21.15.7:32864) 2024-06-10T01:07:17.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.136+0000 7ff4e0b5f700 1 -- 172.21.15.7:0/721088938 learned_addr learned my addr 172.21.15.7:0/721088938 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:17.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.136+0000 7ff4dbfff700 1 --2- 172.21.15.7:0/721088938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567488ed6c0 0x5567488f1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:17.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.136+0000 7ff4e1360700 1 --2- 172.21.15.7:0/721088938 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567488edfe0 0x5567488f15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:17.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.136+0000 7ff4e0b5f700 1 -- 172.21.15.7:0/721088938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567488ed6c0 msgr2=0x5567488f1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:17.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.136+0000 7ff4e0b5f700 1 --2- 172.21.15.7:0/721088938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567488ed6c0 0x5567488f1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.136+0000 7ff4e0b5f700 1 -- 172.21.15.7:0/721088938 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567488edfe0 msgr2=0x5567488f15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:17.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.136+0000 7ff4e0b5f700 1 --2- 172.21.15.7:0/721088938 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567488edfe0 0x5567488f15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.136+0000 7ff4e0b5f700 1 -- 172.21.15.7:0/721088938 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556748788950 con 0x5567488e7fa0 2024-06-10T01:07:17.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.136+0000 7ff4e1360700 1 --2- 172.21.15.7:0/721088938 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567488edfe0 0x5567488f15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:17.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.136+0000 7ff4e0b5f700 1 --2- 172.21.15.7:0/721088938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567488e7fa0 0x5567488f3d10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc007c40 tx=0x7ff4cc0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fafd8eca4d60b9a0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:17.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.136+0000 7ff4bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:07:17.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.136+0000 7ff4bbfff700 1 -- 172.21.15.7:0/721088938 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc009ce0 con 0x5567488e7fa0 2024-06-10T01:07:17.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.136+0000 7ff4bbfff700 1 -- 172.21.15.7:0/721088938 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7ff4cc005250 con 0x5567488e7fa0 2024-06-10T01:07:17.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.136+0000 7ff4ed646b80 1 -- 172.21.15.7:0/721088938 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4c4004bd0 con 0x5567488e7fa0 2024-06-10T01:07:17.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.137+0000 7ff4bbfff700 1 -- 172.21.15.7:0/721088938 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff4cc03b050 con 0x5567488e7fa0 2024-06-10T01:07:17.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.137+0000 7ff4ed646b80 10 client.15729 fetch_fsmap learned FSMap version 27 2024-06-10T01:07:17.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.137+0000 7ff4ed646b80 1 -- 172.21.15.7:0/721088938 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5567488ff320 con 0x5567488e7fa0 2024-06-10T01:07:17.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.137+0000 7ff4bbfff700 1 -- 172.21.15.7:0/721088938 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4c4004bd0 con 0x5567488e7fa0 2024-06-10T01:07:17.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.137+0000 7ff4bbfff700 1 -- 172.21.15.7:0/721088938 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff4cc011070 con 0x5567488e7fa0 2024-06-10T01:07:17.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.137+0000 7ff4ed646b80 10 client.15729 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:07:17.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.137+0000 7ff4ed646b80 10 client.15729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:17.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.137+0000 7ff4ed646b80 20 client.15729 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:17.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.137+0000 7ff4ed646b80 1 --2- 172.21.15.7:0/721088938 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556748901960 0x556748903d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:17.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.137+0000 7ff4ed646b80 4 client.15729 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:07:17.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.137+0000 7ff4ed646b80 1 -- 172.21.15.7:0/721088938 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556748846d90 con 0x556748901960 2024-06-10T01:07:17.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.137+0000 7ff4dbfff700 1 --2- 172.21.15.7:0/721088938 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556748901960 0x556748903d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:17.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.138+0000 7ff4dbfff700 1 --2- 172.21.15.7:0/721088938 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556748901960 0x556748903d80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa9713aebec210c4 server_cookie=a1d3ef1fbaa2f0ca in_seq=0 out_seq=0 2024-06-10T01:07:17.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.138+0000 7ff4bbfff700 10 client.15729 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:17.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.139+0000 7ff4bbfff700 1 -- 172.21.15.7:0/721088938 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556748846d90 con 0x556748901960 2024-06-10T01:07:17.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.139+0000 7ff4bbfff700 10 client.15729 handle_command_reply: tid=0 2024-06-10T01:07:17.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.139+0000 7ff4ed646b80 1 client.15729 shutdown 2024-06-10T01:07:17.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.139+0000 7ff4ed646b80 1 -- 172.21.15.7:0/721088938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567488e7fa0 msgr2=0x5567488f3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:17.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.139+0000 7ff4ed646b80 1 --2- 172.21.15.7:0/721088938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567488e7fa0 0x5567488f3d10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc007c40 tx=0x7ff4cc0099e0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.139+0000 7ff4ed646b80 1 -- 172.21.15.7:0/721088938 shutdown_connections 2024-06-10T01:07:17.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.139+0000 7ff4ed646b80 1 --2- 172.21.15.7:0/721088938 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556748901960 0x556748903d80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.139+0000 7ff4ed646b80 1 --2- 172.21.15.7:0/721088938 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567488edfe0 0x5567488f15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.139+0000 7ff4ed646b80 1 --2- 172.21.15.7:0/721088938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567488e7fa0 0x5567488f3d10 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.139+0000 7ff4ed646b80 1 --2- 172.21.15.7:0/721088938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567488ed6c0 0x5567488f1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.139+0000 7ff4ed646b80 1 -- 172.21.15.7:0/721088938 >> 172.21.15.7:0/721088938 conn(0x556748812d40 msgr2=0x5567488e6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:17.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.139+0000 7ff4bbfff700 0 client.15729 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:17.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.139+0000 7ff4ed646b80 1 -- 172.21.15.7:0/721088938 shutdown_connections 2024-06-10T01:07:17.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.139+0000 7ff4ed646b80 1 -- 172.21.15.7:0/721088938 wait complete. 2024-06-10T01:07:17.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.139+0000 7ff4ed646b80 20 client.15729 trim_cache size 0 max 16384 2024-06-10T01:07:17.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.161+0000 7ff4ed646b80 1 Processor -- start 2024-06-10T01:07:17.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.161+0000 7ff4ed646b80 1 -- start start 2024-06-10T01:07:17.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.161+0000 7ff4ed646b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567488e7fa0 0x5567488e7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:17.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.161+0000 7ff4ed646b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567488ed6c0 0x5567488eec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:17.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.161+0000 7ff4ed646b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567488edfe0 0x5567488effc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:17.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.161+0000 7ff4ed646b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc1b2d10 con 0x5567488e7fa0 2024-06-10T01:07:17.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.161+0000 7ff4ed646b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc1b05a0 con 0x5567488edfe0 2024-06-10T01:07:17.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.161+0000 7ff4ed646b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc1ae3f0 con 0x5567488ed6c0 2024-06-10T01:07:17.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.161+0000 7ff4e0b5f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567488e7fa0 0x5567488e7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:17.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.161+0000 7ff4e0b5f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567488e7fa0 0x5567488e7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32870/0 (socket says 172.21.15.7:32870) 2024-06-10T01:07:17.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.161+0000 7ff4e0b5f700 1 -- 172.21.15.7:0/2184218064 learned_addr learned my addr 172.21.15.7:0/2184218064 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:17.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.162+0000 7ff4dbfff700 1 --2- 172.21.15.7:0/2184218064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567488ed6c0 0x5567488eec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:17.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.162+0000 7ff4e0b5f700 1 -- 172.21.15.7:0/2184218064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567488ed6c0 msgr2=0x5567488eec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:17.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.162+0000 7ff4e0b5f700 1 --2- 172.21.15.7:0/2184218064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567488ed6c0 0x5567488eec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.162+0000 7ff4e0b5f700 1 -- 172.21.15.7:0/2184218064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567488edfe0 msgr2=0x5567488effc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:17.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.162+0000 7ff4e1360700 1 --2- 172.21.15.7:0/2184218064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567488edfe0 0x5567488effc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:17.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.162+0000 7ff4e0b5f700 1 --2- 172.21.15.7:0/2184218064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567488edfe0 0x5567488effc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.162+0000 7ff4e0b5f700 1 -- 172.21.15.7:0/2184218064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567488e86f0 con 0x5567488e7fa0 2024-06-10T01:07:17.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.162+0000 7ff4dbfff700 1 --2- 172.21.15.7:0/2184218064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567488ed6c0 0x5567488eec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:07:17.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.162+0000 7ff4e0b5f700 1 --2- 172.21.15.7:0/2184218064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567488e7fa0 0x5567488e7a60 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc027c10 tx=0x7ff4cc004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0c7b88bed59eaf server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:17.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.162+0000 7ff4d27fc700 1 -- 172.21.15.7:0/2184218064 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc027410 con 0x5567488e7fa0 2024-06-10T01:07:17.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.162+0000 7ff4d27fc700 1 -- 172.21.15.7:0/2184218064 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4cc005250 con 0x5567488e7fa0 2024-06-10T01:07:17.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.162+0000 7ff4d27fc700 1 -- 172.21.15.7:0/2184218064 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc030570 con 0x5567488e7fa0 2024-06-10T01:07:17.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.163+0000 7ff4ed646b80 1 -- 172.21.15.7:0/2184218064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567488e7fa0 msgr2=0x5567488e7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:17.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.163+0000 7ff4ed646b80 1 --2- 172.21.15.7:0/2184218064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567488e7fa0 0x5567488e7a60 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc027c10 tx=0x7ff4cc004ed0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.163+0000 7ff4ed646b80 1 -- 172.21.15.7:0/2184218064 shutdown_connections 2024-06-10T01:07:17.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.163+0000 7ff4ed646b80 1 --2- 172.21.15.7:0/2184218064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567488edfe0 0x5567488effc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.163+0000 7ff4ed646b80 1 --2- 172.21.15.7:0/2184218064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567488e7fa0 0x5567488e7a60 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.163+0000 7ff4ed646b80 1 --2- 172.21.15.7:0/2184218064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567488ed6c0 0x5567488eec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.163+0000 7ff4ed646b80 1 -- 172.21.15.7:0/2184218064 >> 172.21.15.7:0/2184218064 conn(0x556748812d40 msgr2=0x5567488dbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:17.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.163+0000 7ff4ed646b80 1 -- 172.21.15.7:0/2184218064 shutdown_connections 2024-06-10T01:07:17.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.163+0000 7ff4ed646b80 1 -- 172.21.15.7:0/2184218064 wait complete. 2024-06-10T01:07:17.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.163+0000 7ff4ed646b80 1 -- start start 2024-06-10T01:07:17.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.163+0000 7ff4ed646b80 1 Processor -- start 2024-06-10T01:07:17.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.164+0000 7ff4ed646b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567488e7fa0 0x5567488f9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:17.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.164+0000 7ff4ed646b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567488ed6c0 0x5567488fa450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:17.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.164+0000 7ff4ed646b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567488edfe0 0x5567488f6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:17.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.164+0000 7ff4ed646b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc1aea60 con 0x5567488e7fa0 2024-06-10T01:07:17.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.164+0000 7ff4ed646b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc1afed0 con 0x5567488edfe0 2024-06-10T01:07:17.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.164+0000 7ff4ed646b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc1af130 con 0x5567488ed6c0 2024-06-10T01:07:17.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.164+0000 7ff4e0b5f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567488e7fa0 0x5567488f9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:17.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.164+0000 7ff4e0b5f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567488e7fa0 0x5567488f9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32880/0 (socket says 172.21.15.7:32880) 2024-06-10T01:07:17.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.164+0000 7ff4e0b5f700 1 -- 172.21.15.7:0/3706805339 learned_addr learned my addr 172.21.15.7:0/3706805339 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:17.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.164+0000 7ff4ed646b80 10 client.0 _ll_register_callbacks cb 0x5567487d39a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:17.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.164+0000 7ff4dbfff700 1 --2- 172.21.15.7:0/3706805339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567488ed6c0 0x5567488fa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:17.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.164+0000 7ff4e1360700 1 --2- 172.21.15.7:0/3706805339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567488edfe0 0x5567488f6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:17.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.164+0000 7ff4e0b5f700 1 -- 172.21.15.7:0/3706805339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567488ed6c0 msgr2=0x5567488fa450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:17.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.164+0000 7ff4e0b5f700 1 --2- 172.21.15.7:0/3706805339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567488ed6c0 0x5567488fa450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.164+0000 7ff4e0b5f700 1 -- 172.21.15.7:0/3706805339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567488edfe0 msgr2=0x5567488f6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:17.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.164+0000 7ff4e0b5f700 1 --2- 172.21.15.7:0/3706805339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567488edfe0 0x5567488f6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.164+0000 7ff4e0b5f700 1 -- 172.21.15.7:0/3706805339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567488f74d0 con 0x5567488e7fa0 2024-06-10T01:07:17.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.164+0000 7ff4e1360700 1 --2- 172.21.15.7:0/3706805339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567488edfe0 0x5567488f6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:17.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.164+0000 7ff4dbfff700 1 --2- 172.21.15.7:0/3706805339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567488ed6c0 0x5567488fa450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:17.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.164+0000 7ff4e0b5f700 1 --2- 172.21.15.7:0/3706805339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567488e7fa0 0x5567488f9f10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc027d00 tx=0x7ff4cc009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5716529998ed583 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:17.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.164+0000 7ff4bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:07:17.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.165+0000 7ff4bbfff700 1 -- 172.21.15.7:0/3706805339 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc027850 con 0x5567488e7fa0 2024-06-10T01:07:17.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.165+0000 7ff4ed646b80 1 -- 172.21.15.7:0/3706805339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4cc005250 con 0x5567488e7fa0 2024-06-10T01:07:17.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.165+0000 7ff4bbfff700 1 -- 172.21.15.7:0/3706805339 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7ff4cc005400 con 0x5567488e7fa0 2024-06-10T01:07:17.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.165+0000 7ff4bbfff700 1 -- 172.21.15.7:0/3706805339 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff4cc04e050 con 0x5567488e7fa0 2024-06-10T01:07:17.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.165+0000 7ff4ed646b80 10 client.15735 fetch_fsmap learned FSMap version 27 2024-06-10T01:07:17.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.165+0000 7ff4ed646b80 1 -- 172.21.15.7:0/3706805339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55674896ce80 con 0x5567488e7fa0 2024-06-10T01:07:17.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.165+0000 7ff4bbfff700 1 -- 172.21.15.7:0/3706805339 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4cc0304a0 con 0x5567488e7fa0 2024-06-10T01:07:17.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.165+0000 7ff4bbfff700 1 -- 172.21.15.7:0/3706805339 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff4cc011040 con 0x5567488e7fa0 2024-06-10T01:07:17.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.165+0000 7ff4ed646b80 10 client.15735 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:07:17.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.165+0000 7ff4ed646b80 10 client.15735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:17.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.165+0000 7ff4ed646b80 20 client.15735 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:17.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.165+0000 7ff4ed646b80 1 --2- 172.21.15.7:0/3706805339 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55674896d4b0 0x55674896f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:17.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.165+0000 7ff4ed646b80 4 client.15735 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:07:17.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.165+0000 7ff4ed646b80 1 -- 172.21.15.7:0/3706805339 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7ff4cc005250 con 0x55674896d4b0 2024-06-10T01:07:17.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.166+0000 7ff4dbfff700 1 --2- 172.21.15.7:0/3706805339 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55674896d4b0 0x55674896f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:17.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.166+0000 7ff4bbfff700 10 client.15735 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:17.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.166+0000 7ff4dbfff700 1 --2- 172.21.15.7:0/3706805339 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55674896d4b0 0x55674896f8d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ead5f2a35d0b351 server_cookie=4df939f792b5be9f in_seq=0 out_seq=0 2024-06-10T01:07:17.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.167+0000 7ff4bbfff700 1 -- 172.21.15.7:0/3706805339 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7ff4cc005250 con 0x55674896d4b0 2024-06-10T01:07:17.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.167+0000 7ff4bbfff700 10 client.15735 handle_command_reply: tid=0 2024-06-10T01:07:17.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.168+0000 7ff4ed646b80 1 client.15735 shutdown 2024-06-10T01:07:17.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.168+0000 7ff4ed646b80 1 -- 172.21.15.7:0/3706805339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567488e7fa0 msgr2=0x5567488f9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:17.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.168+0000 7ff4ed646b80 1 --2- 172.21.15.7:0/3706805339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567488e7fa0 0x5567488f9f10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc027d00 tx=0x7ff4cc009e10 comp rx=0 tx=0).stop 2024-06-10T01:07:17.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.168+0000 7ff4ed646b80 1 -- 172.21.15.7:0/3706805339 shutdown_connections 2024-06-10T01:07:17.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.168+0000 7ff4ed646b80 1 --2- 172.21.15.7:0/3706805339 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55674896d4b0 0x55674896f8d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.168+0000 7ff4ed646b80 1 --2- 172.21.15.7:0/3706805339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567488edfe0 0x5567488f6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.168+0000 7ff4ed646b80 1 --2- 172.21.15.7:0/3706805339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567488e7fa0 0x5567488f9f10 secure :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc027d00 tx=0x7ff4cc009e10 comp rx=0 tx=0).stop 2024-06-10T01:07:17.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.168+0000 7ff4ed646b80 1 --2- 172.21.15.7:0/3706805339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567488ed6c0 0x5567488fa450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.168+0000 7ff4bbfff700 0 client.15735 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:17.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.168+0000 7ff4ed646b80 1 -- 172.21.15.7:0/3706805339 >> 172.21.15.7:0/3706805339 conn(0x556748812d40 msgr2=0x5567488f2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:17.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.168+0000 7ff4ed646b80 1 -- 172.21.15.7:0/3706805339 shutdown_connections 2024-06-10T01:07:17.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.168+0000 7ff4ed646b80 1 -- 172.21.15.7:0/3706805339 wait complete. 2024-06-10T01:07:17.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.168+0000 7ff4ed646b80 20 client.15735 trim_cache size 0 max 16384 2024-06-10T01:07:17.178 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:07:17.178 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-10T01:07:17.178 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "520a8dc9-1ff8-44e4-8206-07484831053b", 2024-06-10T01:07:17.178 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-10T01:07:17.179 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:07:17.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.169+0000 7ff4e2dc3700 1 -- 172.21.15.7:0/2571635347 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff4c80a2ca0 msgr2=0x7ff4c80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:17.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.169+0000 7ff4e2dc3700 1 --2- 172.21.15.7:0/2571635347 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff4c80a2ca0 0x7ff4c80a5160 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009fd0 tx=0x7ff4cc006210 comp rx=0 tx=0).stop 2024-06-10T01:07:17.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.169+0000 7ff4e2dc3700 1 -- 172.21.15.7:0/2571635347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4dc1085d0 msgr2=0x7ff4dc198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:17.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.169+0000 7ff4e2dc3700 1 --2- 172.21.15.7:0/2571635347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4dc1085d0 0x7ff4dc198a60 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff4d400d750 tx=0x7ff4d400da60 comp rx=0 tx=0).stop 2024-06-10T01:07:17.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.169+0000 7ff4e2dc3700 1 -- 172.21.15.7:0/2571635347 shutdown_connections 2024-06-10T01:07:17.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.169+0000 7ff4e2dc3700 1 --2- 172.21.15.7:0/2571635347 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff4c80a2ca0 0x7ff4c80a5160 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.169+0000 7ff4e2dc3700 1 --2- 172.21.15.7:0/2571635347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4dc1085d0 0x7ff4dc198a60 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.169+0000 7ff4e2dc3700 1 --2- 172.21.15.7:0/2571635347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4dc102530 0x7ff4dc19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.169+0000 7ff4e2dc3700 1 --2- 172.21.15.7:0/2571635347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4dc102ef0 0x7ff4dc19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:17.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.169+0000 7ff4e2dc3700 1 -- 172.21.15.7:0/2571635347 >> 172.21.15.7:0/2571635347 conn(0x7ff4dc079640 msgr2=0x7ff4dc100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:17.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.169+0000 7ff4e2dc3700 1 -- 172.21.15.7:0/2571635347 shutdown_connections 2024-06-10T01:07:17.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:17.169+0000 7ff4e2dc3700 1 -- 172.21.15.7:0/2571635347 wait complete. 2024-06-10T01:07:17.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:17 smithi175 ceph-mon[36636]: pgmap v469: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-06-10T01:07:17.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:17 smithi175 ceph-mon[36636]: scrub queued for path: / 2024-06-10T01:07:17.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:17 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:07:17.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:17 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:07:17.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:17 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:07:17.735 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:17 smithi007 ceph-mon[32398]: pgmap v469: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-06-10T01:07:17.735 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:17 smithi007 ceph-mon[32398]: scrub queued for path: / 2024-06-10T01:07:17.735 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:17 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:07:17.736 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:17 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:07:17.736 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:17 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:07:17.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:17 smithi078 ceph-mon[36683]: pgmap v469: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-06-10T01:07:17.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:17 smithi078 ceph-mon[36683]: scrub queued for path: / 2024-06-10T01:07:17.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:17 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:07:17.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:17 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:07:17.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:17 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:07:18.023 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:07:19.292 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:19.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.533+0000 7f380bebb700 1 -- 172.21.15.7:0/898889310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3804101380 msgr2=0x7f3804101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:19.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.533+0000 7f380bebb700 1 --2- 172.21.15.7:0/898889310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3804101380 0x7f3804101800 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f37f4009990 tx=0x7f37f4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.533+0000 7f380bebb700 1 -- 172.21.15.7:0/898889310 shutdown_connections 2024-06-10T01:07:19.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.533+0000 7f380bebb700 1 --2- 172.21.15.7:0/898889310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3804101d40 0x7f380410e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.533+0000 7f380bebb700 1 --2- 172.21.15.7:0/898889310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38041073f0 0x7f38041077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.533+0000 7f380bebb700 1 --2- 172.21.15.7:0/898889310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3804101380 0x7f3804101800 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.533+0000 7f380bebb700 1 -- 172.21.15.7:0/898889310 >> 172.21.15.7:0/898889310 conn(0x7f3804078300 msgr2=0x7f3804078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:19.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.533+0000 7f380bebb700 1 -- 172.21.15.7:0/898889310 shutdown_connections 2024-06-10T01:07:19.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.533+0000 7f380bebb700 1 -- 172.21.15.7:0/898889310 wait complete. 2024-06-10T01:07:19.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.534+0000 7f380bebb700 1 Processor -- start 2024-06-10T01:07:19.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.534+0000 7f380bebb700 1 -- start start 2024-06-10T01:07:19.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.534+0000 7f380bebb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3804101380 0x7f3804198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:19.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.534+0000 7f380bebb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3804101d40 0x7f38041991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:19.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.534+0000 7f380bebb700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38041073f0 0x7f380419d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:19.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.534+0000 7f380bebb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3804111690 con 0x7f3804101380 2024-06-10T01:07:19.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.534+0000 7f380bebb700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3804111510 con 0x7f3804101d40 2024-06-10T01:07:19.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.534+0000 7f380bebb700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3804111810 con 0x7f38041073f0 2024-06-10T01:07:19.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.534+0000 7f3809c57700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3804101380 0x7f3804198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:19.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.535+0000 7f3809c57700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3804101380 0x7f3804198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32908/0 (socket says 172.21.15.7:32908) 2024-06-10T01:07:19.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.535+0000 7f3809c57700 1 -- 172.21.15.7:0/1641095828 learned_addr learned my addr 172.21.15.7:0/1641095828 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:19.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.535+0000 7f380a458700 1 --2- 172.21.15.7:0/1641095828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38041073f0 0x7f380419d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:19.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.535+0000 7f3809456700 1 --2- 172.21.15.7:0/1641095828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3804101d40 0x7f38041991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:19.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.535+0000 7f3809c57700 1 -- 172.21.15.7:0/1641095828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38041073f0 msgr2=0x7f380419d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:19.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.535+0000 7f3809c57700 1 --2- 172.21.15.7:0/1641095828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38041073f0 0x7f380419d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.535+0000 7f3809c57700 1 -- 172.21.15.7:0/1641095828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3804101d40 msgr2=0x7f38041991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:19.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.535+0000 7f3809c57700 1 --2- 172.21.15.7:0/1641095828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3804101d40 0x7f38041991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.535+0000 7f3809c57700 1 -- 172.21.15.7:0/1641095828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f380419dc80 con 0x7f3804101380 2024-06-10T01:07:19.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.535+0000 7f3809c57700 1 --2- 172.21.15.7:0/1641095828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3804101380 0x7f3804198c80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f380000e9a0 tx=0x7f380000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:19.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.535+0000 7f37faffd700 1 -- 172.21.15.7:0/1641095828 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380000cbd0 con 0x7f3804101380 2024-06-10T01:07:19.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.535+0000 7f37faffd700 1 -- 172.21.15.7:0/1641095828 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f380000cd70 con 0x7f3804101380 2024-06-10T01:07:19.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.535+0000 7f380bebb700 1 -- 172.21.15.7:0/1641095828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f380419df70 con 0x7f3804101380 2024-06-10T01:07:19.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.535+0000 7f37faffd700 1 -- 172.21.15.7:0/1641095828 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38000187c0 con 0x7f3804101380 2024-06-10T01:07:19.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.535+0000 7f380bebb700 1 -- 172.21.15.7:0/1641095828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38041a5830 con 0x7f3804101380 2024-06-10T01:07:19.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.538+0000 7f37faffd700 1 -- 172.21.15.7:0/1641095828 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f3800010430 con 0x7f3804101380 2024-06-10T01:07:19.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.538+0000 7f37faffd700 1 --2- 172.21.15.7:0/1641095828 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f37f00a2c20 0x7f37f00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:19.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.538+0000 7f37faffd700 1 -- 172.21.15.7:0/1641095828 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f38000cf610 con 0x7f3804101380 2024-06-10T01:07:19.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.538+0000 7f3809456700 1 --2- 172.21.15.7:0/1641095828 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f37f00a2c20 0x7f37f00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:19.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.539+0000 7f3809456700 1 --2- 172.21.15.7:0/1641095828 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f37f00a2c20 0x7f37f00a50e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f37f4006010 tx=0x7f37f401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:19.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:19 smithi007 ceph-mon[32398]: pgmap v470: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-10T01:07:19.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.539+0000 7f381673eb80 1 Processor -- start 2024-06-10T01:07:19.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.539+0000 7f381673eb80 1 -- start start 2024-06-10T01:07:19.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.540+0000 7f381673eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a75f09fc40 0x55a75f0a0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:19.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.540+0000 7f381673eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a75f0a0560 0x55a75f099e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:19.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.540+0000 7f381673eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a75f09a3d0 0x55a75f09a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:19.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.540+0000 7f381673eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a75efeb910 con 0x55a75f09fc40 2024-06-10T01:07:19.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.540+0000 7f381673eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a75ef93820 con 0x55a75f0a0560 2024-06-10T01:07:19.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.540+0000 7f381673eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a75efe78d0 con 0x55a75f09a3d0 2024-06-10T01:07:19.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.540+0000 7f3809c57700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a75f09fc40 0x55a75f0a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:19.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.540+0000 7f3809c57700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a75f09fc40 0x55a75f0a0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32916/0 (socket says 172.21.15.7:32916) 2024-06-10T01:07:19.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.540+0000 7f3809c57700 1 -- 172.21.15.7:0/3177013354 learned_addr learned my addr 172.21.15.7:0/3177013354 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:19.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.540+0000 7f380a458700 1 --2- 172.21.15.7:0/3177013354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a75f09a3d0 0x55a75f09a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:19.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.540+0000 7f3809456700 1 --2- 172.21.15.7:0/3177013354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a75f0a0560 0x55a75f099e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:19.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.540+0000 7f3809c57700 1 -- 172.21.15.7:0/3177013354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a75f09a3d0 msgr2=0x55a75f09a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:19.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.540+0000 7f3809c57700 1 --2- 172.21.15.7:0/3177013354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a75f09a3d0 0x55a75f09a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.540+0000 7f3809c57700 1 -- 172.21.15.7:0/3177013354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a75f0a0560 msgr2=0x55a75f099e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:19.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.540+0000 7f3809c57700 1 --2- 172.21.15.7:0/3177013354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a75f0a0560 0x55a75f099e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.540+0000 7f3809c57700 1 -- 172.21.15.7:0/3177013354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a75ef8a050 con 0x55a75f09fc40 2024-06-10T01:07:19.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.540+0000 7f3809456700 1 --2- 172.21.15.7:0/3177013354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a75f0a0560 0x55a75f099e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:19.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.540+0000 7f3809c57700 1 --2- 172.21.15.7:0/3177013354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a75f09fc40 0x55a75f0a0020 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3800097f50 tx=0x7f380009b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c42db59c1b6697 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:19.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.541+0000 7f37effff700 1 -- 172.21.15.7:0/3177013354 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380009fea0 con 0x55a75f09fc40 2024-06-10T01:07:19.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.541+0000 7f37effff700 1 -- 172.21.15.7:0/3177013354 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38000a4760 con 0x55a75f09fc40 2024-06-10T01:07:19.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.541+0000 7f37effff700 1 -- 172.21.15.7:0/3177013354 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38000a42d0 con 0x55a75f09fc40 2024-06-10T01:07:19.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.541+0000 7f381673eb80 1 -- 172.21.15.7:0/3177013354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a75f09fc40 msgr2=0x55a75f0a0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:19.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.541+0000 7f381673eb80 1 --2- 172.21.15.7:0/3177013354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a75f09fc40 0x55a75f0a0020 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3800097f50 tx=0x7f380009b520 comp rx=0 tx=0).stop 2024-06-10T01:07:19.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.542+0000 7f381673eb80 1 -- 172.21.15.7:0/3177013354 shutdown_connections 2024-06-10T01:07:19.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.542+0000 7f381673eb80 1 --2- 172.21.15.7:0/3177013354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a75f09a3d0 0x55a75f09a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.542+0000 7f381673eb80 1 --2- 172.21.15.7:0/3177013354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a75f09fc40 0x55a75f0a0020 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.542+0000 7f381673eb80 1 --2- 172.21.15.7:0/3177013354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a75f0a0560 0x55a75f099e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.542+0000 7f381673eb80 1 -- 172.21.15.7:0/3177013354 >> 172.21.15.7:0/3177013354 conn(0x55a75efc4dc0 msgr2=0x55a75f095120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:19.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.542+0000 7f381673eb80 1 -- 172.21.15.7:0/3177013354 shutdown_connections 2024-06-10T01:07:19.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.542+0000 7f381673eb80 1 -- 172.21.15.7:0/3177013354 wait complete. 2024-06-10T01:07:19.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.542+0000 7f381673eb80 1 -- start start 2024-06-10T01:07:19.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.542+0000 7f381673eb80 1 Processor -- start 2024-06-10T01:07:19.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.543+0000 7f381673eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a75f09a3d0 0x55a75f0a6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:19.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.543+0000 7f381673eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a75f09fc40 0x55a75f0a3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:19.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.543+0000 7f381673eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a75f0a0560 0x55a75f0a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:19.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.543+0000 7f381673eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a75f0a6fd0 con 0x55a75f09fc40 2024-06-10T01:07:19.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.543+0000 7f381673eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a75ef8bcf0 con 0x55a75f0a0560 2024-06-10T01:07:19.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.543+0000 7f381673eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a75f0a7150 con 0x55a75f09a3d0 2024-06-10T01:07:19.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.543+0000 7f381673eb80 10 client.0 _ll_register_callbacks cb 0x55a75ef85a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:19.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.543+0000 7f3809456700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a75f09fc40 0x55a75f0a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:19.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.543+0000 7f3809456700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a75f09fc40 0x55a75f0a3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32920/0 (socket says 172.21.15.7:32920) 2024-06-10T01:07:19.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.543+0000 7f3809c57700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a75f09a3d0 0x55a75f0a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:19.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.543+0000 7f3809456700 1 -- 172.21.15.7:0/979000244 learned_addr learned my addr 172.21.15.7:0/979000244 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:19.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.543+0000 7f380a458700 1 --2- 172.21.15.7:0/979000244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a75f0a0560 0x55a75f0a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:19.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.543+0000 7f3809c57700 1 -- 172.21.15.7:0/979000244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a75f0a0560 msgr2=0x55a75f0a3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:19.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.543+0000 7f3809c57700 1 --2- 172.21.15.7:0/979000244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a75f0a0560 0x55a75f0a3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.543+0000 7f3809c57700 1 -- 172.21.15.7:0/979000244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a75f09fc40 msgr2=0x55a75f0a3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:19.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.543+0000 7f3809c57700 1 --2- 172.21.15.7:0/979000244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a75f09fc40 0x55a75f0a3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.543+0000 7f3809c57700 1 -- 172.21.15.7:0/979000244 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a75ef3a9d0 con 0x55a75f09a3d0 2024-06-10T01:07:19.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.543+0000 7f380a458700 1 --2- 172.21.15.7:0/979000244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a75f0a0560 0x55a75f0a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:19.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.544+0000 7f3809c57700 1 --2- 172.21.15.7:0/979000244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a75f09a3d0 0x55a75f0a6240 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f38000cd100 tx=0x7f3800010d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12bb11d1a431afec server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:19.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.544+0000 7f37ed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:07:19.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.544+0000 7f37ed7fa700 1 -- 172.21.15.7:0/979000244 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800066060 con 0x55a75f09a3d0 2024-06-10T01:07:19.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.544+0000 7f381673eb80 1 -- 172.21.15.7:0/979000244 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38000a4760 con 0x55a75f09a3d0 2024-06-10T01:07:19.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.544+0000 7f37ed7fa700 1 -- 172.21.15.7:0/979000244 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f38000cf360 con 0x55a75f09a3d0 2024-06-10T01:07:19.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.544+0000 7f37ed7fa700 1 -- 172.21.15.7:0/979000244 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3800081050 con 0x55a75f09a3d0 2024-06-10T01:07:19.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.544+0000 7f381673eb80 10 client.25393 fetch_fsmap learned FSMap version 27 2024-06-10T01:07:19.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.544+0000 7f381673eb80 1 -- 172.21.15.7:0/979000244 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a75f0a6fd0 con 0x55a75f09a3d0 2024-06-10T01:07:19.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.544+0000 7f37ed7fa700 1 -- 172.21.15.7:0/979000244 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38000a4760 con 0x55a75f09a3d0 2024-06-10T01:07:19.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.544+0000 7f37ed7fa700 1 -- 172.21.15.7:0/979000244 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f380009a9b0 con 0x55a75f09a3d0 2024-06-10T01:07:19.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.545+0000 7f381673eb80 10 client.25393 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:07:19.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.545+0000 7f381673eb80 10 client.25393 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:07:19.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.545+0000 7f381673eb80 20 client.25393 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:19.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.545+0000 7f381673eb80 1 --2- 172.21.15.7:0/979000244 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a75f0b3b70 0x55a75f0b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:19.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.545+0000 7f381673eb80 4 client.25393 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:07:19.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.545+0000 7f381673eb80 1 -- 172.21.15.7:0/979000244 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a75eff8e10 con 0x55a75f0b3b70 2024-06-10T01:07:19.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.545+0000 7f3809456700 1 --2- 172.21.15.7:0/979000244 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a75f0b3b70 0x55a75f0b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:19.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.545+0000 7f3809456700 1 --2- 172.21.15.7:0/979000244 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a75f0b3b70 0x55a75f0b5f60 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2500150abccb1dc9 server_cookie=894cfb2d14cafa63 in_seq=0 out_seq=0 2024-06-10T01:07:19.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.545+0000 7f37ed7fa700 10 client.25393 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:19.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.546+0000 7f37ed7fa700 1 -- 172.21.15.7:0/979000244 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a75eff8e10 con 0x55a75f0b3b70 2024-06-10T01:07:19.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.546+0000 7f37ed7fa700 10 client.25393 handle_command_reply: tid=0 2024-06-10T01:07:19.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.546+0000 7f381673eb80 1 client.25393 shutdown 2024-06-10T01:07:19.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.546+0000 7f381673eb80 1 -- 172.21.15.7:0/979000244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a75f09a3d0 msgr2=0x55a75f0a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:19.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.546+0000 7f381673eb80 1 --2- 172.21.15.7:0/979000244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a75f09a3d0 0x55a75f0a6240 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f38000cd100 tx=0x7f3800010d00 comp rx=0 tx=0).stop 2024-06-10T01:07:19.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.546+0000 7f381673eb80 1 -- 172.21.15.7:0/979000244 shutdown_connections 2024-06-10T01:07:19.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.546+0000 7f381673eb80 1 --2- 172.21.15.7:0/979000244 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a75f0b3b70 0x55a75f0b5f60 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.546+0000 7f381673eb80 1 --2- 172.21.15.7:0/979000244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a75f0a0560 0x55a75f0a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.546+0000 7f381673eb80 1 --2- 172.21.15.7:0/979000244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a75f09a3d0 0x55a75f0a6240 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.546+0000 7f381673eb80 1 --2- 172.21.15.7:0/979000244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a75f09fc40 0x55a75f0a3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.546+0000 7f37ed7fa700 0 client.25393 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:19.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.546+0000 7f381673eb80 1 -- 172.21.15.7:0/979000244 >> 172.21.15.7:0/979000244 conn(0x55a75efc4dc0 msgr2=0x55a75f094d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:19.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.546+0000 7f381673eb80 1 -- 172.21.15.7:0/979000244 shutdown_connections 2024-06-10T01:07:19.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.546+0000 7f381673eb80 1 -- 172.21.15.7:0/979000244 wait complete. 2024-06-10T01:07:19.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.547+0000 7f381673eb80 20 client.25393 trim_cache size 0 max 16384 2024-06-10T01:07:19.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.568+0000 7f381673eb80 1 Processor -- start 2024-06-10T01:07:19.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.568+0000 7f381673eb80 1 -- start start 2024-06-10T01:07:19.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.568+0000 7f381673eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a75f09a3d0 0x55a75f099e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:19.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.568+0000 7f381673eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a75f09fc40 0x55a75f0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:19.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.568+0000 7f381673eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a75f0a0560 0x55a75f0bb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:19.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.568+0000 7f381673eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f38041b0c10 con 0x55a75f09fc40 2024-06-10T01:07:19.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.568+0000 7f381673eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f38041ae4a0 con 0x55a75f0a0560 2024-06-10T01:07:19.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.568+0000 7f381673eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f38041ac2f0 con 0x55a75f09a3d0 2024-06-10T01:07:19.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.568+0000 7f3809456700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a75f09fc40 0x55a75f0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:19.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.568+0000 7f3809456700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a75f09fc40 0x55a75f0a5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32926/0 (socket says 172.21.15.7:32926) 2024-06-10T01:07:19.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.568+0000 7f3809456700 1 -- 172.21.15.7:0/3237114469 learned_addr learned my addr 172.21.15.7:0/3237114469 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:19.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.568+0000 7f380a458700 1 --2- 172.21.15.7:0/3237114469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a75f0a0560 0x55a75f0bb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:19.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.568+0000 7f3809c57700 1 --2- 172.21.15.7:0/3237114469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a75f09a3d0 0x55a75f099e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:19.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.568+0000 7f3809456700 1 -- 172.21.15.7:0/3237114469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a75f09a3d0 msgr2=0x55a75f099e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:19.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.568+0000 7f3809456700 1 --2- 172.21.15.7:0/3237114469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a75f09a3d0 0x55a75f099e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.568+0000 7f3809456700 1 -- 172.21.15.7:0/3237114469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a75f0a0560 msgr2=0x55a75f0bb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:19.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.568+0000 7f3809456700 1 --2- 172.21.15.7:0/3237114469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a75f0a0560 0x55a75f0bb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.568+0000 7f3809456700 1 -- 172.21.15.7:0/3237114469 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a75f0a1ab0 con 0x55a75f09fc40 2024-06-10T01:07:19.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.568+0000 7f380a458700 1 --2- 172.21.15.7:0/3237114469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a75f0a0560 0x55a75f0bb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:07:19.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.569+0000 7f3809456700 1 --2- 172.21.15.7:0/3237114469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a75f09fc40 0x55a75f0a5480 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f37f4000c00 tx=0x7f37f401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b3bd55dfd8ea3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:19.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.569+0000 7f37ef7fe700 1 -- 172.21.15.7:0/3237114469 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f403e070 con 0x55a75f09fc40 2024-06-10T01:07:19.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.569+0000 7f37ef7fe700 1 -- 172.21.15.7:0/3237114469 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37f4005900 con 0x55a75f09fc40 2024-06-10T01:07:19.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.569+0000 7f37ef7fe700 1 -- 172.21.15.7:0/3237114469 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f4023d70 con 0x55a75f09fc40 2024-06-10T01:07:19.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.569+0000 7f381673eb80 1 -- 172.21.15.7:0/3237114469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a75f09fc40 msgr2=0x55a75f0a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:19.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.569+0000 7f381673eb80 1 --2- 172.21.15.7:0/3237114469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a75f09fc40 0x55a75f0a5480 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f37f4000c00 tx=0x7f37f401e000 comp rx=0 tx=0).stop 2024-06-10T01:07:19.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.570+0000 7f381673eb80 1 -- 172.21.15.7:0/3237114469 shutdown_connections 2024-06-10T01:07:19.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.570+0000 7f381673eb80 1 --2- 172.21.15.7:0/3237114469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a75f0a0560 0x55a75f0bb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.570+0000 7f381673eb80 1 --2- 172.21.15.7:0/3237114469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a75f09a3d0 0x55a75f099e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.570+0000 7f381673eb80 1 --2- 172.21.15.7:0/3237114469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a75f09fc40 0x55a75f0a5480 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.570+0000 7f381673eb80 1 -- 172.21.15.7:0/3237114469 >> 172.21.15.7:0/3237114469 conn(0x55a75efc4dc0 msgr2=0x55a75f0a4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:19.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.570+0000 7f381673eb80 1 -- 172.21.15.7:0/3237114469 shutdown_connections 2024-06-10T01:07:19.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.570+0000 7f381673eb80 1 -- 172.21.15.7:0/3237114469 wait complete. 2024-06-10T01:07:19.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.570+0000 7f381673eb80 1 -- start start 2024-06-10T01:07:19.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.570+0000 7f381673eb80 1 Processor -- start 2024-06-10T01:07:19.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.570+0000 7f381673eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a75f09a3d0 0x55a75f0acac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:19.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.570+0000 7f381673eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a75f09fc40 0x55a75f0a8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:19.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.570+0000 7f381673eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a75f0a0560 0x55a75f0a9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:19.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.571+0000 7f381673eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f38041ac960 con 0x55a75f0a0560 2024-06-10T01:07:19.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.571+0000 7f381673eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f38041addd0 con 0x55a75f09fc40 2024-06-10T01:07:19.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.571+0000 7f381673eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f38041ad030 con 0x55a75f09a3d0 2024-06-10T01:07:19.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.571+0000 7f381673eb80 10 client.0 _ll_register_callbacks cb 0x55a75ef85a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:19.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.571+0000 7f380a458700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a75f0a0560 0x55a75f0a9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:19.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.571+0000 7f3809c57700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a75f09a3d0 0x55a75f0acac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:19.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.571+0000 7f380a458700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a75f0a0560 0x55a75f0a9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32928/0 (socket says 172.21.15.7:32928) 2024-06-10T01:07:19.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.571+0000 7f380a458700 1 -- 172.21.15.7:0/662558009 learned_addr learned my addr 172.21.15.7:0/662558009 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:19.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.571+0000 7f3809456700 1 --2- 172.21.15.7:0/662558009 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a75f09fc40 0x55a75f0a8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:19.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.571+0000 7f3809c57700 1 -- 172.21.15.7:0/662558009 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a75f09fc40 msgr2=0x55a75f0a8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:19.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.571+0000 7f3809c57700 1 --2- 172.21.15.7:0/662558009 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a75f09fc40 0x55a75f0a8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.571+0000 7f3809c57700 1 -- 172.21.15.7:0/662558009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a75f0a0560 msgr2=0x55a75f0a9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:19.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.571+0000 7f3809c57700 1 --2- 172.21.15.7:0/662558009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a75f0a0560 0x55a75f0a9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.571+0000 7f3809c57700 1 -- 172.21.15.7:0/662558009 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a75f0a9c70 con 0x55a75f09a3d0 2024-06-10T01:07:19.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.571+0000 7f380a458700 1 --2- 172.21.15.7:0/662558009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a75f0a0560 0x55a75f0a9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:19.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.571+0000 7f3809c57700 1 --2- 172.21.15.7:0/662558009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a75f09a3d0 0x55a75f0acac0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f380009b7f0 tx=0x7f3800097ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72cd4475b95fea1d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:19.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.571+0000 7f37ed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:07:19.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.571+0000 7f37ed7fa700 1 -- 172.21.15.7:0/662558009 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800066040 con 0x55a75f09a3d0 2024-06-10T01:07:19.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.571+0000 7f381673eb80 1 -- 172.21.15.7:0/662558009 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37f4005900 con 0x55a75f09a3d0 2024-06-10T01:07:19.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.571+0000 7f37ed7fa700 1 -- 172.21.15.7:0/662558009 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f380009a9b0 con 0x55a75f09a3d0 2024-06-10T01:07:19.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.572+0000 7f37ed7fa700 1 -- 172.21.15.7:0/662558009 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f38000ba070 con 0x55a75f09a3d0 2024-06-10T01:07:19.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.572+0000 7f381673eb80 10 client.25396 fetch_fsmap learned FSMap version 27 2024-06-10T01:07:19.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.572+0000 7f381673eb80 1 -- 172.21.15.7:0/662558009 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a75f11f570 con 0x55a75f09a3d0 2024-06-10T01:07:19.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.572+0000 7f37ed7fa700 1 -- 172.21.15.7:0/662558009 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37f4005900 con 0x55a75f09a3d0 2024-06-10T01:07:19.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.572+0000 7f37ed7fa700 1 -- 172.21.15.7:0/662558009 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f38000a11c0 con 0x55a75f09a3d0 2024-06-10T01:07:19.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.572+0000 7f381673eb80 10 client.25396 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:07:19.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.572+0000 7f381673eb80 10 client.25396 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:07:19.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.572+0000 7f381673eb80 20 client.25396 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:19.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.572+0000 7f381673eb80 1 --2- 172.21.15.7:0/662558009 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a75f11f930 0x55a75f121d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:19.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.572+0000 7f381673eb80 4 client.25396 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "96f9e47b", "release": true, "await": true}] 2024-06-10T01:07:19.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.572+0000 7f381673eb80 1 -- 172.21.15.7:0/662558009 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "set_id": "96f9e47b", "release": true, "await": true}) -- 0x55a75eff3390 con 0x55a75f11f930 2024-06-10T01:07:19.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.572+0000 7f3809456700 1 --2- 172.21.15.7:0/662558009 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a75f11f930 0x55a75f121d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:19.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.573+0000 7f37ed7fa700 10 client.25396 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:19.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.573+0000 7f3809456700 1 --2- 172.21.15.7:0/662558009 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a75f11f930 0x55a75f121d50 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49bfa32de1e7078b server_cookie=26fc3f8e597137b in_seq=0 out_seq=0 2024-06-10T01:07:19.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.576+0000 7f37ed7fa700 1 -- 172.21.15.7:0/662558009 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55a75eff3390 con 0x55a75f11f930 2024-06-10T01:07:19.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.576+0000 7f37ed7fa700 10 client.25396 handle_command_reply: tid=0 2024-06-10T01:07:19.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.576+0000 7f381673eb80 1 client.25396 shutdown 2024-06-10T01:07:19.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.576+0000 7f381673eb80 1 -- 172.21.15.7:0/662558009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a75f09a3d0 msgr2=0x55a75f0acac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:19.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.576+0000 7f381673eb80 1 --2- 172.21.15.7:0/662558009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a75f09a3d0 0x55a75f0acac0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f380009b7f0 tx=0x7f3800097ec0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.576+0000 7f381673eb80 1 -- 172.21.15.7:0/662558009 shutdown_connections 2024-06-10T01:07:19.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.576+0000 7f381673eb80 1 --2- 172.21.15.7:0/662558009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a75f0a0560 0x55a75f0a9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.576+0000 7f381673eb80 1 --2- 172.21.15.7:0/662558009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a75f09a3d0 0x55a75f0acac0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.576+0000 7f381673eb80 1 --2- 172.21.15.7:0/662558009 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a75f11f930 0x55a75f121d50 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.576+0000 7f381673eb80 1 --2- 172.21.15.7:0/662558009 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a75f09fc40 0x55a75f0a8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.576+0000 7f381673eb80 1 -- 172.21.15.7:0/662558009 >> 172.21.15.7:0/662558009 conn(0x55a75efc4dc0 msgr2=0x55a75f0a5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:19.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.576+0000 7f37ed7fa700 0 client.25396 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:19.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.577+0000 7f381673eb80 1 -- 172.21.15.7:0/662558009 shutdown_connections 2024-06-10T01:07:19.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.577+0000 7f381673eb80 1 -- 172.21.15.7:0/662558009 wait complete. 2024-06-10T01:07:19.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.577+0000 7f381673eb80 20 client.25396 trim_cache size 0 max 16384 2024-06-10T01:07:19.575 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:07:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 27, 2024-06-10T01:07:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:07:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 4, 2024-06-10T01:07:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:07:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: "96f9e47b": { 2024-06-10T01:07:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 4, 2024-06-10T01:07:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:07:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:07:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:07:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:07:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:07:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:07:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 150.0, 2024-06-10T01:07:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:07:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:07:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:07:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:07:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:07:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:07:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:07:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:07:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:07:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:07:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:07:19.577 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:07:19.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.578+0000 7f380bebb700 1 -- 172.21.15.7:0/1641095828 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f37f00a2c20 msgr2=0x7f37f00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:19.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.578+0000 7f380bebb700 1 --2- 172.21.15.7:0/1641095828 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f37f00a2c20 0x7f37f00a50e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f37f4006010 tx=0x7f37f401a040 comp rx=0 tx=0).stop 2024-06-10T01:07:19.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.578+0000 7f380bebb700 1 -- 172.21.15.7:0/1641095828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3804101380 msgr2=0x7f3804198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:19.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.578+0000 7f380bebb700 1 --2- 172.21.15.7:0/1641095828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3804101380 0x7f3804198c80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f380000e9a0 tx=0x7f380000ed60 comp rx=0 tx=0).stop 2024-06-10T01:07:19.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.578+0000 7f380bebb700 1 -- 172.21.15.7:0/1641095828 shutdown_connections 2024-06-10T01:07:19.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.578+0000 7f380bebb700 1 --2- 172.21.15.7:0/1641095828 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f37f00a2c20 0x7f37f00a50e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.578+0000 7f380bebb700 1 --2- 172.21.15.7:0/1641095828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38041073f0 0x7f380419d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.578+0000 7f380bebb700 1 --2- 172.21.15.7:0/1641095828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3804101380 0x7f3804198c80 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.578+0000 7f380bebb700 1 --2- 172.21.15.7:0/1641095828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3804101d40 0x7f38041991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:19.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.578+0000 7f380bebb700 1 -- 172.21.15.7:0/1641095828 >> 172.21.15.7:0/1641095828 conn(0x7f3804078300 msgr2=0x7f380406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:19.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.578+0000 7f380bebb700 1 -- 172.21.15.7:0/1641095828 shutdown_connections 2024-06-10T01:07:19.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:19.578+0000 7f380bebb700 1 -- 172.21.15.7:0/1641095828 wait complete. 2024-06-10T01:07:19.578 INFO:tasks.workunit.client.0.smithi007.stderr:+ dbench 1 2024-06-10T01:07:19.591 INFO:tasks.workunit.client.0.smithi007.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-06-10T01:07:19.591 INFO:tasks.workunit.client.0.smithi007.stdout: 2024-06-10T01:07:19.591 INFO:tasks.workunit.client.0.smithi007.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-06-10T01:07:19.606 INFO:tasks.workunit.client.0.smithi007.stdout:0 of 1 processes prepared for launch 0 sec 2024-06-10T01:07:19.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:19 smithi175 ceph-mon[36636]: pgmap v470: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-10T01:07:19.707 INFO:tasks.workunit.client.0.smithi007.stdout:1 of 1 processes prepared for launch 0 sec 2024-06-10T01:07:19.707 INFO:tasks.workunit.client.0.smithi007.stdout:releasing clients 2024-06-10T01:07:19.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:19 smithi078 ceph-mon[36683]: pgmap v470: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-10T01:07:20.707 INFO:tasks.workunit.client.0.smithi007.stdout: 1 3671 38.72 MB/sec warmup 1 sec latency 6.572 ms 2024-06-10T01:07:20.988 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '96f9e47b', total seconds elapsed: 39.5 2024-06-10T01:07:20.988 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-06-10T01:07:20.988 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:07:21.488 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:21.693 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:21 smithi007 ceph-mon[32398]: pgmap v471: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:07:21.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:21 smithi175 ceph-mon[36636]: pgmap v471: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:07:21.707 INFO:tasks.workunit.client.0.smithi007.stdout: 1 6745 26.69 MB/sec warmup 2 sec latency 15.142 ms 2024-06-10T01:07:21.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:21 smithi078 ceph-mon[36683]: pgmap v471: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T01:07:22.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.592+0000 7fead68a6700 1 -- 172.21.15.7:0/3826963052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fead0102740 msgr2=0x7fead0102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:22.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.592+0000 7fead68a6700 1 --2- 172.21.15.7:0/3826963052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fead0102740 0x7fead0102b20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7feac00099c0 tx=0x7feac0009cd0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.592+0000 7fead68a6700 1 -- 172.21.15.7:0/3826963052 shutdown_connections 2024-06-10T01:07:22.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.592+0000 7fead68a6700 1 --2- 172.21.15.7:0/3826963052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fead0107840 0x7fead010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.592+0000 7fead68a6700 1 --2- 172.21.15.7:0/3826963052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fead0102740 0x7fead0102b20 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.592+0000 7fead68a6700 1 --2- 172.21.15.7:0/3826963052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fead01030f0 0x7fead0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.592+0000 7fead68a6700 1 -- 172.21.15.7:0/3826963052 >> 172.21.15.7:0/3826963052 conn(0x7fead00795f0 msgr2=0x7fead0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:22.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.592+0000 7fead68a6700 1 -- 172.21.15.7:0/3826963052 shutdown_connections 2024-06-10T01:07:22.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.592+0000 7fead68a6700 1 -- 172.21.15.7:0/3826963052 wait complete. 2024-06-10T01:07:22.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.593+0000 7fead68a6700 1 Processor -- start 2024-06-10T01:07:22.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.593+0000 7fead68a6700 1 -- start start 2024-06-10T01:07:22.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.593+0000 7fead68a6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fead0102740 0x7fead019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.593+0000 7fead68a6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fead01030f0 0x7fead019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.593+0000 7fead68a6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fead0107840 0x7fead0198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.593+0000 7fead68a6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fead0112760 con 0x7fead0107840 2024-06-10T01:07:22.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.593+0000 7fead68a6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fead01125e0 con 0x7fead0102740 2024-06-10T01:07:22.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.593+0000 7fead68a6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fead01128e0 con 0x7fead01030f0 2024-06-10T01:07:22.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.593+0000 7feacffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fead0102740 0x7fead019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.593+0000 7feac7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fead01030f0 0x7fead019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.593+0000 7feacffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fead0102740 0x7fead019e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:55518/0 (socket says 172.21.15.7:55518) 2024-06-10T01:07:22.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.593+0000 7fead4e43700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fead0107840 0x7fead0198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.593+0000 7feacffff700 1 -- 172.21.15.7:0/1959635557 learned_addr learned my addr 172.21.15.7:0/1959635557 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:22.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.593+0000 7feacffff700 1 -- 172.21.15.7:0/1959635557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fead01030f0 msgr2=0x7fead019ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:22.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.593+0000 7feacffff700 1 --2- 172.21.15.7:0/1959635557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fead01030f0 0x7fead019ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.593+0000 7feacffff700 1 -- 172.21.15.7:0/1959635557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fead0107840 msgr2=0x7fead0198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:22.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.593+0000 7feacffff700 1 --2- 172.21.15.7:0/1959635557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fead0107840 0x7fead0198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.593+0000 7feacffff700 1 -- 172.21.15.7:0/1959635557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead0199160 con 0x7fead0102740 2024-06-10T01:07:22.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.593+0000 7feac7fff700 1 --2- 172.21.15.7:0/1959635557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fead01030f0 0x7fead019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:22.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.593+0000 7fead4e43700 1 --2- 172.21.15.7:0/1959635557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fead0107840 0x7fead0198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:22.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.594+0000 7feacffff700 1 --2- 172.21.15.7:0/1959635557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fead0102740 0x7fead019e7f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7feac0006010 tx=0x7feac000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:22.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.594+0000 7feacdffb700 1 -- 172.21.15.7:0/1959635557 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac0003680 con 0x7fead0102740 2024-06-10T01:07:22.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.594+0000 7feacdffb700 1 -- 172.21.15.7:0/1959635557 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feac00050d0 con 0x7fead0102740 2024-06-10T01:07:22.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.594+0000 7feacdffb700 1 -- 172.21.15.7:0/1959635557 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac0003db0 con 0x7fead0102740 2024-06-10T01:07:22.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.594+0000 7fead68a6700 1 -- 172.21.15.7:0/1959635557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fead01993f0 con 0x7fead0102740 2024-06-10T01:07:22.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.594+0000 7fead68a6700 1 -- 172.21.15.7:0/1959635557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fead01a56c0 con 0x7fead0102740 2024-06-10T01:07:22.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.598+0000 7feacdffb700 1 -- 172.21.15.7:0/1959635557 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7feac0005280 con 0x7fead0102740 2024-06-10T01:07:22.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.598+0000 7feacdffb700 1 --2- 172.21.15.7:0/1959635557 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7feabc0a2e40 0x7feabc0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.599+0000 7feac7fff700 1 --2- 172.21.15.7:0/1959635557 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7feabc0a2e40 0x7feabc0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.599+0000 7feacdffb700 1 -- 172.21.15.7:0/1959635557 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7feac00cffc0 con 0x7fead0102740 2024-06-10T01:07:22.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.599+0000 7feac7fff700 1 --2- 172.21.15.7:0/1959635557 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7feabc0a2e40 0x7feabc0a5300 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fead0100f40 tx=0x7feab8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:22.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.600+0000 7feae1129b80 1 Processor -- start 2024-06-10T01:07:22.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.600+0000 7feae1129b80 1 -- start start 2024-06-10T01:07:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.600+0000 7feae1129b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559494695640 0x559494695a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.600+0000 7feae1129b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559494695f60 0x55949468f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.600+0000 7feae1129b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55949468ff20 0x559494690340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.600+0000 7feae1129b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5594945e1810 con 0x559494695640 2024-06-10T01:07:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.601+0000 7feae1129b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559494589720 con 0x559494695f60 2024-06-10T01:07:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.601+0000 7feae1129b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5594945dd7d0 con 0x55949468ff20 2024-06-10T01:07:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.601+0000 7fead4e43700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55949468ff20 0x559494690340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.601+0000 7feac7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559494695f60 0x55949468f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.601+0000 7feacffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559494695640 0x559494695a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.601+0000 7fead4e43700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55949468ff20 0x559494690340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:40984/0 (socket says 172.21.15.7:40984) 2024-06-10T01:07:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.601+0000 7feac7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559494695f60 0x55949468f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:55520/0 (socket says 172.21.15.7:55520) 2024-06-10T01:07:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.601+0000 7fead4e43700 1 -- 172.21.15.7:0/594682727 learned_addr learned my addr 172.21.15.7:0/594682727 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:22.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.601+0000 7fead4e43700 1 -- 172.21.15.7:0/594682727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559494695f60 msgr2=0x55949468f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:22.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.601+0000 7fead4e43700 1 --2- 172.21.15.7:0/594682727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559494695f60 0x55949468f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.601+0000 7fead4e43700 1 -- 172.21.15.7:0/594682727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559494695640 msgr2=0x559494695a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:22.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.601+0000 7fead4e43700 1 --2- 172.21.15.7:0/594682727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559494695640 0x559494695a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.601+0000 7fead4e43700 1 -- 172.21.15.7:0/594682727 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55949457ff50 con 0x55949468ff20 2024-06-10T01:07:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.601+0000 7fead4e43700 1 --2- 172.21.15.7:0/594682727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55949468ff20 0x559494690340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feac8003eb0 tx=0x7feac8009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3d52a68f05470f2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.601+0000 7feac67fc700 1 -- 172.21.15.7:0/594682727 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac80090d0 con 0x55949468ff20 2024-06-10T01:07:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.602+0000 7feac67fc700 1 -- 172.21.15.7:0/594682727 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feac8009270 con 0x55949468ff20 2024-06-10T01:07:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.602+0000 7feac67fc700 1 -- 172.21.15.7:0/594682727 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac8007880 con 0x55949468ff20 2024-06-10T01:07:22.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.602+0000 7feae1129b80 1 -- 172.21.15.7:0/594682727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55949468ff20 msgr2=0x559494690340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:22.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.602+0000 7feae1129b80 1 --2- 172.21.15.7:0/594682727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55949468ff20 0x559494690340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feac8003eb0 tx=0x7feac8009920 comp rx=0 tx=0).stop 2024-06-10T01:07:22.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.603+0000 7feae1129b80 1 -- 172.21.15.7:0/594682727 shutdown_connections 2024-06-10T01:07:22.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.603+0000 7feae1129b80 1 --2- 172.21.15.7:0/594682727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55949468ff20 0x559494690340 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.603+0000 7feae1129b80 1 --2- 172.21.15.7:0/594682727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559494695640 0x559494695a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.603+0000 7feae1129b80 1 --2- 172.21.15.7:0/594682727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559494695f60 0x55949468f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.603+0000 7feae1129b80 1 -- 172.21.15.7:0/594682727 >> 172.21.15.7:0/594682727 conn(0x5594945bacc0 msgr2=0x55949468a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:22.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.603+0000 7feae1129b80 1 -- 172.21.15.7:0/594682727 shutdown_connections 2024-06-10T01:07:22.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.603+0000 7feae1129b80 1 -- 172.21.15.7:0/594682727 wait complete. 2024-06-10T01:07:22.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.604+0000 7feae1129b80 1 -- start start 2024-06-10T01:07:22.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.604+0000 7feae1129b80 1 Processor -- start 2024-06-10T01:07:22.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.604+0000 7feae1129b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55949468ff20 0x55949469bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.604+0000 7feae1129b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559494695640 0x559494698f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.604+0000 7feae1129b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559494695f60 0x5594946994a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.604+0000 7feae1129b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55949469c9d0 con 0x559494695f60 2024-06-10T01:07:22.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.604+0000 7feae1129b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559494581bf0 con 0x55949468ff20 2024-06-10T01:07:22.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.604+0000 7feae1129b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55949469cb50 con 0x559494695640 2024-06-10T01:07:22.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.604+0000 7feae1129b80 10 client.0 _ll_register_callbacks cb 0x55949457b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:22.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.605+0000 7feac7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559494695640 0x559494698f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.605+0000 7fead4e43700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559494695f60 0x5594946994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.605+0000 7feacffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55949468ff20 0x55949469bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.605+0000 7feac7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559494695640 0x559494698f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:40998/0 (socket says 172.21.15.7:40998) 2024-06-10T01:07:22.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.605+0000 7feacffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55949468ff20 0x55949469bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:55534/0 (socket says 172.21.15.7:55534) 2024-06-10T01:07:22.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.605+0000 7fead4e43700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559494695f60 0x5594946994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48718/0 (socket says 172.21.15.7:48718) 2024-06-10T01:07:22.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.605+0000 7feac7fff700 1 -- 172.21.15.7:0/3596071181 learned_addr learned my addr 172.21.15.7:0/3596071181 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:22.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.605+0000 7feacffff700 1 -- 172.21.15.7:0/3596071181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559494695640 msgr2=0x559494698f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:22.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.605+0000 7feacffff700 1 --2- 172.21.15.7:0/3596071181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559494695640 0x559494698f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.605+0000 7feacffff700 1 -- 172.21.15.7:0/3596071181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559494695f60 msgr2=0x5594946994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:22.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.605+0000 7feacffff700 1 --2- 172.21.15.7:0/3596071181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559494695f60 0x5594946994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.605+0000 7feacffff700 1 -- 172.21.15.7:0/3596071181 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594945308d0 con 0x55949468ff20 2024-06-10T01:07:22.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.605+0000 7fead4e43700 1 --2- 172.21.15.7:0/3596071181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559494695f60 0x5594946994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:22.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.605+0000 7feacffff700 1 --2- 172.21.15.7:0/3596071181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55949468ff20 0x55949469bc60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7feac00cd790 tx=0x7feac009c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61c189dbcb7ed877 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:22.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.605+0000 7feaaf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:07:22.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.606+0000 7feaaf7fe700 1 -- 172.21.15.7:0/3596071181 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac00b5060 con 0x55949468ff20 2024-06-10T01:07:22.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.606+0000 7feaaf7fe700 1 -- 172.21.15.7:0/3596071181 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7feac009a830 con 0x55949468ff20 2024-06-10T01:07:22.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.606+0000 7feae1129b80 1 -- 172.21.15.7:0/3596071181 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feac8009270 con 0x55949468ff20 2024-06-10T01:07:22.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.606+0000 7feaaf7fe700 1 -- 172.21.15.7:0/3596071181 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feac009cc30 con 0x55949468ff20 2024-06-10T01:07:22.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.606+0000 7feae1129b80 10 client.25358 fetch_fsmap learned FSMap version 27 2024-06-10T01:07:22.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.606+0000 7feae1129b80 1 -- 172.21.15.7:0/3596071181 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5594946a7080 con 0x55949468ff20 2024-06-10T01:07:22.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.606+0000 7feaaf7fe700 1 -- 172.21.15.7:0/3596071181 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feac8009270 con 0x55949468ff20 2024-06-10T01:07:22.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.607+0000 7feaaf7fe700 1 -- 172.21.15.7:0/3596071181 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7feac00a4c80 con 0x55949468ff20 2024-06-10T01:07:22.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.607+0000 7feae1129b80 10 client.25358 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:07:22.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.607+0000 7feae1129b80 10 client.25358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:22.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.607+0000 7feae1129b80 20 client.25358 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:22.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.607+0000 7feae1129b80 1 --2- 172.21.15.7:0/3596071181 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5594946a9640 0x5594946aba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.607+0000 7feae1129b80 4 client.25358 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:07:22.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.607+0000 7feae1129b80 1 -- 172.21.15.7:0/3596071181 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594945eed10 con 0x5594946a9640 2024-06-10T01:07:22.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.607+0000 7feac7fff700 1 --2- 172.21.15.7:0/3596071181 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5594946a9640 0x5594946aba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.607+0000 7feac7fff700 1 --2- 172.21.15.7:0/3596071181 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5594946a9640 0x5594946aba80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=662662c2aba3c4bf server_cookie=213ebc1f3666e35c in_seq=0 out_seq=0 2024-06-10T01:07:22.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.607+0000 7feaaf7fe700 10 client.25358 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:22.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.609+0000 7feaaf7fe700 1 -- 172.21.15.7:0/3596071181 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5594945eed10 con 0x5594946a9640 2024-06-10T01:07:22.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.609+0000 7feaaf7fe700 10 client.25358 handle_command_reply: tid=0 2024-06-10T01:07:22.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.609+0000 7feae1129b80 1 client.25358 shutdown 2024-06-10T01:07:22.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.610+0000 7feae1129b80 1 -- 172.21.15.7:0/3596071181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55949468ff20 msgr2=0x55949469bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:22.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.610+0000 7feae1129b80 1 --2- 172.21.15.7:0/3596071181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55949468ff20 0x55949469bc60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7feac00cd790 tx=0x7feac009c3d0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.610+0000 7feae1129b80 1 -- 172.21.15.7:0/3596071181 shutdown_connections 2024-06-10T01:07:22.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.610+0000 7feae1129b80 1 --2- 172.21.15.7:0/3596071181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559494695f60 0x5594946994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.610+0000 7feae1129b80 1 --2- 172.21.15.7:0/3596071181 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5594946a9640 0x5594946aba80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.610+0000 7feae1129b80 1 --2- 172.21.15.7:0/3596071181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55949468ff20 0x55949469bc60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.610+0000 7feae1129b80 1 --2- 172.21.15.7:0/3596071181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559494695640 0x559494698f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.610+0000 7feae1129b80 1 -- 172.21.15.7:0/3596071181 >> 172.21.15.7:0/3596071181 conn(0x5594945bacc0 msgr2=0x55949468e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:22.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.610+0000 7feaaf7fe700 0 client.25358 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:22.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.610+0000 7feae1129b80 1 -- 172.21.15.7:0/3596071181 shutdown_connections 2024-06-10T01:07:22.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.610+0000 7feae1129b80 1 -- 172.21.15.7:0/3596071181 wait complete. 2024-06-10T01:07:22.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.610+0000 7feae1129b80 20 client.25358 trim_cache size 0 max 16384 2024-06-10T01:07:22.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.635+0000 7feae1129b80 1 Processor -- start 2024-06-10T01:07:22.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.635+0000 7feae1129b80 1 -- start start 2024-06-10T01:07:22.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.635+0000 7feae1129b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594945bacc0 0x55949468f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.635+0000 7feae1129b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55949468ff20 0x559494684690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.635+0000 7feae1129b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559494695640 0x559494697f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.635+0000 7feae1129b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fead01ae8d0 con 0x55949468ff20 2024-06-10T01:07:22.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.635+0000 7feae1129b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fead01ac160 con 0x5594945bacc0 2024-06-10T01:07:22.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.635+0000 7feae1129b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fead01a9fb0 con 0x559494695640 2024-06-10T01:07:22.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.635+0000 7feac7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55949468ff20 0x559494684690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.635+0000 7feacffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594945bacc0 0x55949468f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.635+0000 7feac7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55949468ff20 0x559494684690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48734/0 (socket says 172.21.15.7:48734) 2024-06-10T01:07:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.635+0000 7feac7fff700 1 -- 172.21.15.7:0/2718915087 learned_addr learned my addr 172.21.15.7:0/2718915087 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.635+0000 7fead4e43700 1 --2- 172.21.15.7:0/2718915087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559494695640 0x559494697f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.636+0000 7feacffff700 1 -- 172.21.15.7:0/2718915087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559494695640 msgr2=0x559494697f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.636+0000 7feacffff700 1 --2- 172.21.15.7:0/2718915087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559494695640 0x559494697f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.636+0000 7feacffff700 1 -- 172.21.15.7:0/2718915087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55949468ff20 msgr2=0x559494684690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.636+0000 7feacffff700 1 --2- 172.21.15.7:0/2718915087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55949468ff20 0x559494684690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.636+0000 7feacffff700 1 -- 172.21.15.7:0/2718915087 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55949469c460 con 0x5594945bacc0 2024-06-10T01:07:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.636+0000 7feac7fff700 1 --2- 172.21.15.7:0/2718915087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55949468ff20 0x559494684690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.636+0000 7feacffff700 1 --2- 172.21.15.7:0/2718915087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594945bacc0 0x55949468f9e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7feac000b580 tx=0x7feac0016fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=473653bf629041da server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.636+0000 7feac5ffb700 1 -- 172.21.15.7:0/2718915087 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac00b5060 con 0x5594945bacc0 2024-06-10T01:07:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.636+0000 7feac5ffb700 1 -- 172.21.15.7:0/2718915087 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feac00a08e0 con 0x5594945bacc0 2024-06-10T01:07:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.636+0000 7feac5ffb700 1 -- 172.21.15.7:0/2718915087 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac009c070 con 0x5594945bacc0 2024-06-10T01:07:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.637+0000 7feae1129b80 1 -- 172.21.15.7:0/2718915087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594945bacc0 msgr2=0x55949468f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.637+0000 7feae1129b80 1 --2- 172.21.15.7:0/2718915087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594945bacc0 0x55949468f9e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7feac000b580 tx=0x7feac0016fb0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.637+0000 7feae1129b80 1 -- 172.21.15.7:0/2718915087 shutdown_connections 2024-06-10T01:07:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.637+0000 7feae1129b80 1 --2- 172.21.15.7:0/2718915087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559494695640 0x559494697f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.637+0000 7feae1129b80 1 --2- 172.21.15.7:0/2718915087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594945bacc0 0x55949468f9e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.637+0000 7feae1129b80 1 --2- 172.21.15.7:0/2718915087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55949468ff20 0x559494684690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.637+0000 7feae1129b80 1 -- 172.21.15.7:0/2718915087 >> 172.21.15.7:0/2718915087 conn(0x55949468aaa0 msgr2=0x559494697850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.637+0000 7feae1129b80 1 -- 172.21.15.7:0/2718915087 shutdown_connections 2024-06-10T01:07:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.637+0000 7feae1129b80 1 -- 172.21.15.7:0/2718915087 wait complete. 2024-06-10T01:07:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.637+0000 7feae1129b80 1 -- start start 2024-06-10T01:07:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.638+0000 7feae1129b80 1 Processor -- start 2024-06-10T01:07:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.638+0000 7feae1129b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5594945bacc0 0x5594946a2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.638+0000 7feae1129b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55949468ff20 0x55949469d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.638+0000 7feae1129b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559494695640 0x55949469dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.638+0000 7feae1129b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fead01aa620 con 0x55949468ff20 2024-06-10T01:07:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.638+0000 7feae1129b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fead01aba90 con 0x559494695640 2024-06-10T01:07:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.638+0000 7feae1129b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fead01aacf0 con 0x5594945bacc0 2024-06-10T01:07:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.638+0000 7feae1129b80 10 client.0 _ll_register_callbacks cb 0x55949457b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.638+0000 7feac7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55949468ff20 0x55949469d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.638+0000 7feac7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55949468ff20 0x55949469d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48750/0 (socket says 172.21.15.7:48750) 2024-06-10T01:07:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.638+0000 7feac7fff700 1 -- 172.21.15.7:0/571891934 learned_addr learned my addr 172.21.15.7:0/571891934 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.638+0000 7feacffff700 1 --2- 172.21.15.7:0/571891934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5594945bacc0 0x5594946a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.638+0000 7fead4e43700 1 --2- 172.21.15.7:0/571891934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559494695640 0x55949469dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.638+0000 7feac7fff700 1 -- 172.21.15.7:0/571891934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5594945bacc0 msgr2=0x5594946a2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.638+0000 7feac7fff700 1 --2- 172.21.15.7:0/571891934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5594945bacc0 0x5594946a2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.638+0000 7feac7fff700 1 -- 172.21.15.7:0/571891934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559494695640 msgr2=0x55949469dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.638+0000 7feac7fff700 1 --2- 172.21.15.7:0/571891934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559494695640 0x55949469dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.638+0000 7feac7fff700 1 -- 172.21.15.7:0/571891934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55949469e5d0 con 0x55949468ff20 2024-06-10T01:07:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.639+0000 7feac7fff700 1 --2- 172.21.15.7:0/571891934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55949468ff20 0x55949469d8a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7feab8013010 tx=0x7feab80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c90e1812547355c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.639+0000 7feaaf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:07:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.639+0000 7feaaf7fe700 1 -- 172.21.15.7:0/571891934 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab802a830 con 0x55949468ff20 2024-06-10T01:07:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.639+0000 7feae1129b80 1 -- 172.21.15.7:0/571891934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feac00a08e0 con 0x55949468ff20 2024-06-10T01:07:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.639+0000 7feaaf7fe700 1 -- 172.21.15.7:0/571891934 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7feab802a9d0 con 0x55949468ff20 2024-06-10T01:07:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.639+0000 7feaaf7fe700 1 -- 172.21.15.7:0/571891934 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feab803b050 con 0x55949468ff20 2024-06-10T01:07:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.639+0000 7feae1129b80 10 client.15771 fetch_fsmap learned FSMap version 27 2024-06-10T01:07:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.639+0000 7feae1129b80 1 -- 172.21.15.7:0/571891934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559494714390 con 0x55949468ff20 2024-06-10T01:07:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.639+0000 7feaaf7fe700 1 -- 172.21.15.7:0/571891934 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feac00a08e0 con 0x55949468ff20 2024-06-10T01:07:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.639+0000 7feaaf7fe700 1 -- 172.21.15.7:0/571891934 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7feab8031030 con 0x55949468ff20 2024-06-10T01:07:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.640+0000 7feae1129b80 10 client.15771 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:07:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.640+0000 7feae1129b80 10 client.15771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.640+0000 7feae1129b80 20 client.15771 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.640+0000 7feae1129b80 1 --2- 172.21.15.7:0/571891934 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559494695f60 0x559494716af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.640+0000 7feae1129b80 4 client.15771 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:07:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.640+0000 7feae1129b80 1 -- 172.21.15.7:0/571891934 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5594945e9290 con 0x559494695f60 2024-06-10T01:07:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.640+0000 7feacffff700 1 --2- 172.21.15.7:0/571891934 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559494695f60 0x559494716af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.640+0000 7feaaf7fe700 10 client.15771 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.640+0000 7feacffff700 1 --2- 172.21.15.7:0/571891934 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559494695f60 0x559494716af0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82d5cf80baff0e64 server_cookie=2891b8cccbb0bd86 in_seq=0 out_seq=0 2024-06-10T01:07:22.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.642+0000 7feaaf7fe700 1 -- 172.21.15.7:0/571891934 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5594945e9290 con 0x559494695f60 2024-06-10T01:07:22.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.642+0000 7feaaf7fe700 10 client.15771 handle_command_reply: tid=0 2024-06-10T01:07:22.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.642+0000 7feae1129b80 1 client.15771 shutdown 2024-06-10T01:07:22.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.643+0000 7feae1129b80 1 -- 172.21.15.7:0/571891934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55949468ff20 msgr2=0x55949469d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:22.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.643+0000 7feae1129b80 1 --2- 172.21.15.7:0/571891934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55949468ff20 0x55949469d8a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7feab8013010 tx=0x7feab80047c0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.643+0000 7feae1129b80 1 -- 172.21.15.7:0/571891934 shutdown_connections 2024-06-10T01:07:22.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.643+0000 7feae1129b80 1 --2- 172.21.15.7:0/571891934 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559494695f60 0x559494716af0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.643+0000 7feae1129b80 1 --2- 172.21.15.7:0/571891934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559494695640 0x55949469dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.643+0000 7feae1129b80 1 --2- 172.21.15.7:0/571891934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5594945bacc0 0x5594946a2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.643+0000 7feaaf7fe700 0 client.15771 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:22.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.643+0000 7feae1129b80 1 --2- 172.21.15.7:0/571891934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55949468ff20 0x55949469d8a0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.643+0000 7feae1129b80 1 -- 172.21.15.7:0/571891934 >> 172.21.15.7:0/571891934 conn(0x55949468aaa0 msgr2=0x55949469b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:22.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.643+0000 7feae1129b80 1 -- 172.21.15.7:0/571891934 shutdown_connections 2024-06-10T01:07:22.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.643+0000 7feae1129b80 1 -- 172.21.15.7:0/571891934 wait complete. 2024-06-10T01:07:22.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.643+0000 7feae1129b80 20 client.15771 trim_cache size 0 max 16384 2024-06-10T01:07:22.642 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:07:22.642 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-10T01:07:22.642 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-10T01:07:22.642 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:07:22.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.643+0000 7fead68a6700 1 -- 172.21.15.7:0/1959635557 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7feabc0a2e40 msgr2=0x7feabc0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:22.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.643+0000 7fead68a6700 1 --2- 172.21.15.7:0/1959635557 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7feabc0a2e40 0x7feabc0a5300 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fead0100f40 tx=0x7feab8009260 comp rx=0 tx=0).stop 2024-06-10T01:07:22.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.643+0000 7fead68a6700 1 -- 172.21.15.7:0/1959635557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fead0102740 msgr2=0x7fead019e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:22.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.643+0000 7fead68a6700 1 --2- 172.21.15.7:0/1959635557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fead0102740 0x7fead019e7f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7feac0006010 tx=0x7feac000f690 comp rx=0 tx=0).stop 2024-06-10T01:07:22.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.644+0000 7fead68a6700 1 -- 172.21.15.7:0/1959635557 shutdown_connections 2024-06-10T01:07:22.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.644+0000 7fead68a6700 1 --2- 172.21.15.7:0/1959635557 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7feabc0a2e40 0x7feabc0a5300 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.644+0000 7fead68a6700 1 --2- 172.21.15.7:0/1959635557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fead0107840 0x7fead0198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.644+0000 7fead68a6700 1 --2- 172.21.15.7:0/1959635557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fead0102740 0x7fead019e7f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.644+0000 7fead68a6700 1 --2- 172.21.15.7:0/1959635557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fead01030f0 0x7fead019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.644+0000 7fead68a6700 1 -- 172.21.15.7:0/1959635557 >> 172.21.15.7:0/1959635557 conn(0x7fead00795f0 msgr2=0x7fead0101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:22.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.644+0000 7fead68a6700 1 -- 172.21.15.7:0/1959635557 shutdown_connections 2024-06-10T01:07:22.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:22.644+0000 7fead68a6700 1 -- 172.21.15.7:0/1959635557 wait complete. 2024-06-10T01:07:22.679 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:22 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:07:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.686+0000 7fd111e10640 1 -- 172.21.15.7:0/274859899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd10c104e90 msgr2=0x7fd10c107280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.686+0000 7fd111e10640 1 --2- 172.21.15.7:0/274859899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd10c104e90 0x7fd10c107280 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd100009900 tx=0x7fd10002a410 comp rx=0 tx=0).stop 2024-06-10T01:07:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.686+0000 7fd111e10640 1 -- 172.21.15.7:0/274859899 shutdown_connections 2024-06-10T01:07:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.686+0000 7fd111e10640 1 --2- 172.21.15.7:0/274859899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd10c1077c0 0x7fd10c109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.686+0000 7fd111e10640 1 --2- 172.21.15.7:0/274859899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd10c104e90 0x7fd10c107280 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.686+0000 7fd111e10640 1 --2- 172.21.15.7:0/274859899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd10c102560 0x7fd10c104950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.686+0000 7fd111e10640 1 -- 172.21.15.7:0/274859899 >> 172.21.15.7:0/274859899 conn(0x7fd10c0fbeb0 msgr2=0x7fd10c0fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.686+0000 7fd111e10640 1 -- 172.21.15.7:0/274859899 shutdown_connections 2024-06-10T01:07:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.686+0000 7fd111e10640 1 -- 172.21.15.7:0/274859899 wait complete. 2024-06-10T01:07:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.687+0000 7fd111e10640 1 Processor -- start 2024-06-10T01:07:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.687+0000 7fd111e10640 1 -- start start 2024-06-10T01:07:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.687+0000 7fd111e10640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd10c102560 0x7fd10c197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.687+0000 7fd111e10640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd10c104e90 0x7fd10c1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.687+0000 7fd111e10640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd10c1077c0 0x7fd10c19f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.687+0000 7fd111e10640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c10d6b0 con 0x7fd10c104e90 2024-06-10T01:07:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.687+0000 7fd111e10640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c10d530 con 0x7fd10c102560 2024-06-10T01:07:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.687+0000 7fd111e10640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c10d830 con 0x7fd10c1077c0 2024-06-10T01:07:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.688+0000 7fd10affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd10c104e90 0x7fd10c1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.688+0000 7fd10affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd10c104e90 0x7fd10c1982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48760/0 (socket says 172.21.15.7:48760) 2024-06-10T01:07:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.688+0000 7fd10affd640 1 -- 172.21.15.7:0/2214439283 learned_addr learned my addr 172.21.15.7:0/2214439283 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.688+0000 7fd10b7fe640 1 --2- 172.21.15.7:0/2214439283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd10c102560 0x7fd10c197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.688+0000 7fd10bfff640 1 --2- 172.21.15.7:0/2214439283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd10c1077c0 0x7fd10c19f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.688+0000 7fd10affd640 1 -- 172.21.15.7:0/2214439283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd10c1077c0 msgr2=0x7fd10c19f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.688+0000 7fd10affd640 1 --2- 172.21.15.7:0/2214439283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd10c1077c0 0x7fd10c19f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.688+0000 7fd10affd640 1 -- 172.21.15.7:0/2214439283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd10c102560 msgr2=0x7fd10c197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.688+0000 7fd10affd640 1 --2- 172.21.15.7:0/2214439283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd10c102560 0x7fd10c197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.688+0000 7fd10affd640 1 -- 172.21.15.7:0/2214439283 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd10c19f980 con 0x7fd10c104e90 2024-06-10T01:07:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.688+0000 7fd10b7fe640 1 --2- 172.21.15.7:0/2214439283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd10c102560 0x7fd10c197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.688+0000 7fd10affd640 1 --2- 172.21.15.7:0/2214439283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd10c104e90 0x7fd10c1982a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd100031610 tx=0x7fd100004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.688+0000 7fd108ff9640 1 -- 172.21.15.7:0/2214439283 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10003f030 con 0x7fd10c104e90 2024-06-10T01:07:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.688+0000 7fd108ff9640 1 -- 172.21.15.7:0/2214439283 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd100003ad0 con 0x7fd10c104e90 2024-06-10T01:07:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.688+0000 7fd111e10640 1 -- 172.21.15.7:0/2214439283 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd10c19fc10 con 0x7fd10c104e90 2024-06-10T01:07:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.688+0000 7fd108ff9640 1 -- 172.21.15.7:0/2214439283 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd100035070 con 0x7fd10c104e90 2024-06-10T01:07:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.688+0000 7fd111e10640 1 -- 172.21.15.7:0/2214439283 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd10c1a0100 con 0x7fd10c104e90 2024-06-10T01:07:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.691+0000 7fd108ff9640 1 -- 172.21.15.7:0/2214439283 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fd100042030 con 0x7fd10c104e90 2024-06-10T01:07:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.691+0000 7fd114bf3740 1 Processor -- start 2024-06-10T01:07:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.691+0000 7fd114bf3740 1 -- start start 2024-06-10T01:07:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.691+0000 7fd114bf3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56390b1a5420 0x56390b1a5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.691+0000 7fd114bf3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56390b1a5e10 0x56390b1aa910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.691+0000 7fd114bf3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56390b1aaf80 0x56390b1ad410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.691+0000 7fd114bf3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56390b1044f0 con 0x56390b1aaf80 2024-06-10T01:07:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.691+0000 7fd114bf3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56390b0a2870 con 0x56390b1a5e10 2024-06-10T01:07:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.691+0000 7fd114bf3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56390b1004b0 con 0x56390b1a5420 2024-06-10T01:07:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.691+0000 7fd10affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56390b1aaf80 0x56390b1ad410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.691+0000 7fd10affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56390b1aaf80 0x56390b1ad410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48762/0 (socket says 172.21.15.7:48762) 2024-06-10T01:07:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.691+0000 7fd10affd640 1 -- 172.21.15.7:0/2597841133 learned_addr learned my addr 172.21.15.7:0/2597841133 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.691+0000 7fd10bfff640 1 --2- 172.21.15.7:0/2597841133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56390b1a5420 0x56390b1a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.691+0000 7fd10b7fe640 1 --2- 172.21.15.7:0/2597841133 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56390b1a5e10 0x56390b1aa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.691+0000 7fd108ff9640 1 --2- 172.21.15.7:0/2214439283 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd0f40a2b80 0x7fd0f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.692+0000 7fd10affd640 1 -- 172.21.15.7:0/2597841133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56390b1a5420 msgr2=0x56390b1a5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.692+0000 7fd10affd640 1 --2- 172.21.15.7:0/2597841133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56390b1a5420 0x56390b1a5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.692+0000 7fd10affd640 1 -- 172.21.15.7:0/2597841133 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56390b1a5e10 msgr2=0x56390b1aa910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.692+0000 7fd10affd640 1 --2- 172.21.15.7:0/2597841133 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56390b1a5e10 0x56390b1aa910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.692+0000 7fd10affd640 1 -- 172.21.15.7:0/2597841133 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56390b0be9b0 con 0x56390b1aaf80 2024-06-10T01:07:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.692+0000 7fd10bfff640 1 --2- 172.21.15.7:0/2214439283 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd0f40a2b80 0x7fd0f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.692+0000 7fd108ff9640 1 -- 172.21.15.7:0/2214439283 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fd1000ef9b0 con 0x7fd10c104e90 2024-06-10T01:07:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.692+0000 7fd10affd640 1 --2- 172.21.15.7:0/2597841133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56390b1aaf80 0x56390b1ad410 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd1000a08d0 tx=0x7fd100086b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f8980a59acbf88 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.692+0000 7fd0f1ffb640 1 -- 172.21.15.7:0/2597841133 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1000a0710 con 0x56390b1aaf80 2024-06-10T01:07:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.692+0000 7fd10bfff640 1 --2- 172.21.15.7:0/2214439283 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd0f40a2b80 0x7fd0f40a5040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x56390b1a6670 tx=0x7fd104014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.692+0000 7fd0f1ffb640 1 -- 172.21.15.7:0/2597841133 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd10003be40 con 0x56390b1aaf80 2024-06-10T01:07:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.692+0000 7fd0f1ffb640 1 -- 172.21.15.7:0/2597841133 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10008f4a0 con 0x56390b1aaf80 2024-06-10T01:07:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.693+0000 7fd114bf3740 1 -- 172.21.15.7:0/2597841133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56390b1aaf80 msgr2=0x56390b1ad410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.693+0000 7fd114bf3740 1 --2- 172.21.15.7:0/2597841133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56390b1aaf80 0x56390b1ad410 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd1000a08d0 tx=0x7fd100086b60 comp rx=0 tx=0).stop 2024-06-10T01:07:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.693+0000 7fd114bf3740 1 -- 172.21.15.7:0/2597841133 shutdown_connections 2024-06-10T01:07:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.693+0000 7fd114bf3740 1 --2- 172.21.15.7:0/2597841133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56390b1aaf80 0x56390b1ad410 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.693+0000 7fd114bf3740 1 --2- 172.21.15.7:0/2597841133 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56390b1a5e10 0x56390b1aa910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.693+0000 7fd114bf3740 1 --2- 172.21.15.7:0/2597841133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56390b1a5420 0x56390b1a5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.693+0000 7fd114bf3740 1 -- 172.21.15.7:0/2597841133 >> 172.21.15.7:0/2597841133 conn(0x56390b19c680 msgr2=0x56390b19ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.693+0000 7fd114bf3740 1 -- 172.21.15.7:0/2597841133 shutdown_connections 2024-06-10T01:07:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.693+0000 7fd114bf3740 1 -- 172.21.15.7:0/2597841133 wait complete. 2024-06-10T01:07:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.693+0000 7fd114bf3740 1 -- start start 2024-06-10T01:07:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.693+0000 7fd114bf3740 1 Processor -- start 2024-06-10T01:07:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.694+0000 7fd114bf3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56390b1a5420 0x56390b1af330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.694+0000 7fd114bf3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56390b1a5e10 0x56390b1af870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.694+0000 7fd114bf3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56390b1aaf80 0x56390b1b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.694+0000 7fd114bf3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56390b1adcf0 con 0x56390b1aaf80 2024-06-10T01:07:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.694+0000 7fd114bf3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56390b08dae0 con 0x56390b1a5e10 2024-06-10T01:07:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.694+0000 7fd114bf3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56390b1ade70 con 0x56390b1a5420 2024-06-10T01:07:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.694+0000 7fd114bf3740 10 client.0 _ll_register_callbacks cb 0x56390b0c69b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.694+0000 7fd10bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56390b1aaf80 0x56390b1b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.694+0000 7fd10affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56390b1a5e10 0x56390b1af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.694+0000 7fd10bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56390b1aaf80 0x56390b1b68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48766/0 (socket says 172.21.15.7:48766) 2024-06-10T01:07:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.694+0000 7fd10bfff640 1 -- 172.21.15.7:0/4247497890 learned_addr learned my addr 172.21.15.7:0/4247497890 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.694+0000 7fd10b7fe640 1 --2- 172.21.15.7:0/4247497890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56390b1a5420 0x56390b1af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.694+0000 7fd10b7fe640 1 -- 172.21.15.7:0/4247497890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56390b1a5e10 msgr2=0x56390b1af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.694+0000 7fd10b7fe640 1 --2- 172.21.15.7:0/4247497890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56390b1a5e10 0x56390b1af870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.694+0000 7fd10b7fe640 1 -- 172.21.15.7:0/4247497890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56390b1aaf80 msgr2=0x56390b1b68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.694+0000 7fd10b7fe640 1 --2- 172.21.15.7:0/4247497890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56390b1aaf80 0x56390b1b68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.694+0000 7fd10b7fe640 1 -- 172.21.15.7:0/4247497890 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56390ae18210 con 0x56390b1a5420 2024-06-10T01:07:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.694+0000 7fd10bfff640 1 --2- 172.21.15.7:0/4247497890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56390b1aaf80 0x56390b1b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.694+0000 7fd10affd640 1 --2- 172.21.15.7:0/4247497890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56390b1a5e10 0x56390b1af870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.694+0000 7fd10b7fe640 1 --2- 172.21.15.7:0/4247497890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56390b1a5420 0x56390b1af330 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x56390b1a6210 tx=0x7fd0fc00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e88f7a8c2532d127 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.694+0000 7fd0eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:07:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.695+0000 7fd0eaffd640 1 -- 172.21.15.7:0/4247497890 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc00fe50 con 0x56390b1a5420 2024-06-10T01:07:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.695+0000 7fd114bf3740 1 -- 172.21.15.7:0/4247497890 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56390b117f10 con 0x56390b1a5420 2024-06-10T01:07:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.695+0000 7fd0eaffd640 1 -- 172.21.15.7:0/4247497890 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fd0fc012110 con 0x56390b1a5420 2024-06-10T01:07:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.695+0000 7fd0eaffd640 1 -- 172.21.15.7:0/4247497890 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd0fc004050 con 0x56390b1a5420 2024-06-10T01:07:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.695+0000 7fd114bf3740 10 client.25426 fetch_fsmap learned FSMap version 27 2024-06-10T01:07:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.695+0000 7fd114bf3740 1 -- 172.21.15.7:0/4247497890 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56390aee4620 con 0x56390b1a5420 2024-06-10T01:07:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.695+0000 7fd0eaffd640 1 -- 172.21.15.7:0/4247497890 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56390b117f10 con 0x56390b1a5420 2024-06-10T01:07:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.695+0000 7fd0eaffd640 1 -- 172.21.15.7:0/4247497890 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd0fc010430 con 0x56390b1a5420 2024-06-10T01:07:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.695+0000 7fd114bf3740 10 client.25426 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:07:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.695+0000 7fd114bf3740 10 client.25426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.695+0000 7fd114bf3740 20 client.25426 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.695+0000 7fd114bf3740 1 --2- 172.21.15.7:0/4247497890 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56390b1bb1f0 0x56390b1bd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.695+0000 7fd114bf3740 4 client.25426 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:07:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.695+0000 7fd114bf3740 1 -- 172.21.15.7:0/4247497890 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56390b111c20 con 0x56390b1bb1f0 2024-06-10T01:07:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.695+0000 7fd10affd640 1 --2- 172.21.15.7:0/4247497890 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56390b1bb1f0 0x56390b1bd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.696+0000 7fd0eaffd640 10 client.25426 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.696+0000 7fd10affd640 1 --2- 172.21.15.7:0/4247497890 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56390b1bb1f0 0x56390b1bd660 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1522e375767b37cc server_cookie=553db9e7b9fad319 in_seq=0 out_seq=0 2024-06-10T01:07:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.697+0000 7fd0eaffd640 1 -- 172.21.15.7:0/4247497890 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56390b111c20 con 0x56390b1bb1f0 2024-06-10T01:07:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.697+0000 7fd0eaffd640 10 client.25426 handle_command_reply: tid=0 2024-06-10T01:07:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.697+0000 7fd114bf3740 1 client.25426 shutdown 2024-06-10T01:07:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.698+0000 7fd114bf3740 1 -- 172.21.15.7:0/4247497890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56390b1a5420 msgr2=0x56390b1af330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.698+0000 7fd114bf3740 1 --2- 172.21.15.7:0/4247497890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56390b1a5420 0x56390b1af330 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x56390b1a6210 tx=0x7fd0fc00f790 comp rx=0 tx=0).stop 2024-06-10T01:07:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.698+0000 7fd114bf3740 1 -- 172.21.15.7:0/4247497890 shutdown_connections 2024-06-10T01:07:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.698+0000 7fd114bf3740 1 --2- 172.21.15.7:0/4247497890 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56390b1bb1f0 0x56390b1bd660 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.698+0000 7fd114bf3740 1 --2- 172.21.15.7:0/4247497890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56390b1aaf80 0x56390b1b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.698+0000 7fd114bf3740 1 --2- 172.21.15.7:0/4247497890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56390b1a5e10 0x56390b1af870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.698+0000 7fd114bf3740 1 --2- 172.21.15.7:0/4247497890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56390b1a5420 0x56390b1af330 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.698+0000 7fd114bf3740 1 -- 172.21.15.7:0/4247497890 >> 172.21.15.7:0/4247497890 conn(0x56390b19c680 msgr2=0x56390b19b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.698+0000 7fd0eaffd640 0 client.25426 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:22.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.698+0000 7fd114bf3740 1 -- 172.21.15.7:0/4247497890 shutdown_connections 2024-06-10T01:07:22.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.698+0000 7fd114bf3740 1 -- 172.21.15.7:0/4247497890 wait complete. 2024-06-10T01:07:22.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.698+0000 7fd114bf3740 20 client.25426 trim_cache size 0 max 16384 2024-06-10T01:07:22.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:22 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:07:22.707 INFO:tasks.workunit.client.0.smithi007.stdout: 1 9492 22.07 MB/sec warmup 3 sec latency 38.004 ms 2024-06-10T01:07:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.713+0000 7fd114bf3740 1 Processor -- start 2024-06-10T01:07:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.713+0000 7fd114bf3740 1 -- start start 2024-06-10T01:07:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.713+0000 7fd114bf3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56390b1a5420 0x56390b19fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.713+0000 7fd114bf3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56390b1a5e10 0x56390b1b6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.713+0000 7fd114bf3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56390b1aaf80 0x56390b1ae950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.714+0000 7fd114bf3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c1aacb0 con 0x56390b1a5e10 2024-06-10T01:07:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.714+0000 7fd114bf3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c1a8580 con 0x56390b1aaf80 2024-06-10T01:07:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.714+0000 7fd114bf3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c1a63d0 con 0x56390b1a5420 2024-06-10T01:07:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.714+0000 7fd10affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56390b1a5e10 0x56390b1b6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.714+0000 7fd10b7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56390b1a5420 0x56390b19fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.714+0000 7fd10affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56390b1a5e10 0x56390b1b6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48772/0 (socket says 172.21.15.7:48772) 2024-06-10T01:07:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.714+0000 7fd10affd640 1 -- 172.21.15.7:0/2233269541 learned_addr learned my addr 172.21.15.7:0/2233269541 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.714+0000 7fd10bfff640 1 --2- 172.21.15.7:0/2233269541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56390b1aaf80 0x56390b1ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.714+0000 7fd10affd640 1 -- 172.21.15.7:0/2233269541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56390b1a5420 msgr2=0x56390b19fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.714+0000 7fd10affd640 1 --2- 172.21.15.7:0/2233269541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56390b1a5420 0x56390b19fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.714+0000 7fd10affd640 1 -- 172.21.15.7:0/2233269541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56390b1aaf80 msgr2=0x56390b1ae950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.714+0000 7fd10affd640 1 --2- 172.21.15.7:0/2233269541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56390b1aaf80 0x56390b1ae950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.714+0000 7fd10affd640 1 -- 172.21.15.7:0/2233269541 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56390b19c4b0 con 0x56390b1a5e10 2024-06-10T01:07:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.714+0000 7fd10b7fe640 1 --2- 172.21.15.7:0/2233269541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56390b1a5420 0x56390b19fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.714+0000 7fd10bfff640 1 --2- 172.21.15.7:0/2233269541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56390b1aaf80 0x56390b1ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.714+0000 7fd10affd640 1 --2- 172.21.15.7:0/2233269541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56390b1a5e10 0x56390b1b6870 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd1000bbbe0 tx=0x7fd10002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a58f3b1626c0b8d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.714+0000 7fd0f17fa640 1 -- 172.21.15.7:0/2233269541 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10008ae60 con 0x56390b1a5e10 2024-06-10T01:07:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.714+0000 7fd0f17fa640 1 -- 172.21.15.7:0/2233269541 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1000bbee0 con 0x56390b1a5e10 2024-06-10T01:07:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.714+0000 7fd0f17fa640 1 -- 172.21.15.7:0/2233269541 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd100049040 con 0x56390b1a5e10 2024-06-10T01:07:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.715+0000 7fd114bf3740 1 -- 172.21.15.7:0/2233269541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56390b1a5e10 msgr2=0x56390b1b6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.715+0000 7fd114bf3740 1 --2- 172.21.15.7:0/2233269541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56390b1a5e10 0x56390b1b6870 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd1000bbbe0 tx=0x7fd10002a410 comp rx=0 tx=0).stop 2024-06-10T01:07:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.715+0000 7fd114bf3740 1 -- 172.21.15.7:0/2233269541 shutdown_connections 2024-06-10T01:07:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.715+0000 7fd114bf3740 1 --2- 172.21.15.7:0/2233269541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56390b1aaf80 0x56390b1ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.715+0000 7fd114bf3740 1 --2- 172.21.15.7:0/2233269541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56390b1a5e10 0x56390b1b6870 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.715+0000 7fd114bf3740 1 --2- 172.21.15.7:0/2233269541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56390b1a5420 0x56390b19fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.715+0000 7fd114bf3740 1 -- 172.21.15.7:0/2233269541 >> 172.21.15.7:0/2233269541 conn(0x56390b19c680 msgr2=0x56390b19f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.715+0000 7fd114bf3740 1 -- 172.21.15.7:0/2233269541 shutdown_connections 2024-06-10T01:07:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.715+0000 7fd114bf3740 1 -- 172.21.15.7:0/2233269541 wait complete. 2024-06-10T01:07:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.715+0000 7fd114bf3740 1 -- start start 2024-06-10T01:07:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.716+0000 7fd114bf3740 1 Processor -- start 2024-06-10T01:07:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.716+0000 7fd114bf3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56390b1a5420 0x56390b1b63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.716+0000 7fd114bf3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56390b1a5e10 0x56390b1b6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.716+0000 7fd114bf3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56390b1aaf80 0x56390b1b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.716+0000 7fd114bf3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c1a6a40 con 0x56390b1a5420 2024-06-10T01:07:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.716+0000 7fd114bf3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c1a7eb0 con 0x56390b1a5e10 2024-06-10T01:07:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.716+0000 7fd114bf3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c1a7110 con 0x56390b1aaf80 2024-06-10T01:07:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.716+0000 7fd114bf3740 10 client.0 _ll_register_callbacks cb 0x7fd0fc009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.716+0000 7fd10b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56390b1a5420 0x56390b1b63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.716+0000 7fd10affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56390b1a5e10 0x56390b1b6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.716+0000 7fd10b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56390b1a5420 0x56390b1b63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48782/0 (socket says 172.21.15.7:48782) 2024-06-10T01:07:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.716+0000 7fd10b7fe640 1 -- 172.21.15.7:0/1592529071 learned_addr learned my addr 172.21.15.7:0/1592529071 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.716+0000 7fd10bfff640 1 --2- 172.21.15.7:0/1592529071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56390b1aaf80 0x56390b1b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.716+0000 7fd10b7fe640 1 -- 172.21.15.7:0/1592529071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56390b1aaf80 msgr2=0x56390b1b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.716+0000 7fd10b7fe640 1 --2- 172.21.15.7:0/1592529071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56390b1aaf80 0x56390b1b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.717+0000 7fd10b7fe640 1 -- 172.21.15.7:0/1592529071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56390b1a5e10 msgr2=0x56390b1b6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.717+0000 7fd10b7fe640 1 --2- 172.21.15.7:0/1592529071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56390b1a5e10 0x56390b1b6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.717+0000 7fd10b7fe640 1 -- 172.21.15.7:0/1592529071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56390b1b3780 con 0x56390b1a5420 2024-06-10T01:07:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.717+0000 7fd10affd640 1 --2- 172.21.15.7:0/1592529071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56390b1a5e10 0x56390b1b6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.717+0000 7fd10b7fe640 1 --2- 172.21.15.7:0/1592529071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56390b1a5420 0x56390b1b63c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x56390b1981b0 tx=0x7fd0fc010d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc24872a33bcf207 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.717+0000 7fd0eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:07:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.717+0000 7fd0eaffd640 1 -- 172.21.15.7:0/1592529071 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc02d070 con 0x56390b1a5420 2024-06-10T01:07:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.717+0000 7fd114bf3740 1 -- 172.21.15.7:0/1592529071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1000bbee0 con 0x56390b1a5420 2024-06-10T01:07:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.717+0000 7fd0eaffd640 1 -- 172.21.15.7:0/1592529071 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fd0fc010640 con 0x56390b1a5420 2024-06-10T01:07:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.717+0000 7fd0eaffd640 1 -- 172.21.15.7:0/1592529071 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd0fc000e60 con 0x56390b1a5420 2024-06-10T01:07:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.717+0000 7fd114bf3740 10 client.15789 fetch_fsmap learned FSMap version 27 2024-06-10T01:07:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.717+0000 7fd114bf3740 1 -- 172.21.15.7:0/1592529071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56390b1a0d30 con 0x56390b1a5420 2024-06-10T01:07:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.717+0000 7fd0eaffd640 1 -- 172.21.15.7:0/1592529071 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1000bbee0 con 0x56390b1a5420 2024-06-10T01:07:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.717+0000 7fd0eaffd640 1 -- 172.21.15.7:0/1592529071 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd0fc003b20 con 0x56390b1a5420 2024-06-10T01:07:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.718+0000 7fd114bf3740 10 client.15789 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:07:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.718+0000 7fd114bf3740 10 client.15789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.718+0000 7fd114bf3740 20 client.15789 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.718+0000 7fd114bf3740 1 --2- 172.21.15.7:0/1592529071 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56390b1a1680 0x56390b1a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.718+0000 7fd114bf3740 4 client.15789 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:07:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.718+0000 7fd114bf3740 1 -- 172.21.15.7:0/1592529071 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56390b10bf70 con 0x56390b1a1680 2024-06-10T01:07:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.718+0000 7fd10affd640 1 --2- 172.21.15.7:0/1592529071 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56390b1a1680 0x56390b1a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.718+0000 7fd0eaffd640 10 client.15789 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.718+0000 7fd10affd640 1 --2- 172.21.15.7:0/1592529071 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56390b1a1680 0x56390b1a3aa0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d607af32cce329c6 server_cookie=60ea6c22791a537b in_seq=0 out_seq=0 2024-06-10T01:07:22.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:22 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:07:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.859+0000 7fd0eaffd640 1 -- 172.21.15.7:0/1592529071 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56390b10bf70 con 0x56390b1a1680 2024-06-10T01:07:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.859+0000 7fd0eaffd640 10 client.15789 handle_command_reply: tid=0 2024-06-10T01:07:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.859+0000 7fd114bf3740 1 client.15789 shutdown 2024-06-10T01:07:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.859+0000 7fd114bf3740 1 -- 172.21.15.7:0/1592529071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56390b1a5420 msgr2=0x56390b1b63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.859+0000 7fd114bf3740 1 --2- 172.21.15.7:0/1592529071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56390b1a5420 0x56390b1b63c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x56390b1981b0 tx=0x7fd0fc010d40 comp rx=0 tx=0).stop 2024-06-10T01:07:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.859+0000 7fd114bf3740 1 -- 172.21.15.7:0/1592529071 shutdown_connections 2024-06-10T01:07:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.859+0000 7fd114bf3740 1 --2- 172.21.15.7:0/1592529071 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56390b1a1680 0x56390b1a3aa0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.859+0000 7fd114bf3740 1 --2- 172.21.15.7:0/1592529071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56390b1aaf80 0x56390b1b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.859+0000 7fd0eaffd640 0 client.15789 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.859+0000 7fd114bf3740 1 --2- 172.21.15.7:0/1592529071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56390b1a5e10 0x56390b1b6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.859+0000 7fd114bf3740 1 --2- 172.21.15.7:0/1592529071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56390b1a5420 0x56390b1b63c0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.859+0000 7fd114bf3740 1 -- 172.21.15.7:0/1592529071 >> 172.21.15.7:0/1592529071 conn(0x56390b19c680 msgr2=0x56390b19af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.860+0000 7fd114bf3740 1 -- 172.21.15.7:0/1592529071 shutdown_connections 2024-06-10T01:07:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.860+0000 7fd114bf3740 1 -- 172.21.15.7:0/1592529071 wait complete. 2024-06-10T01:07:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.860+0000 7fd114bf3740 20 client.15789 trim_cache size 0 max 16384 2024-06-10T01:07:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:07:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:07:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:07:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.860+0000 7fd111e10640 1 -- 172.21.15.7:0/2214439283 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd0f40a2b80 msgr2=0x7fd0f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.860+0000 7fd111e10640 1 --2- 172.21.15.7:0/2214439283 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd0f40a2b80 0x7fd0f40a5040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x56390b1a6670 tx=0x7fd104014040 comp rx=0 tx=0).stop 2024-06-10T01:07:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.860+0000 7fd111e10640 1 -- 172.21.15.7:0/2214439283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd10c104e90 msgr2=0x7fd10c1982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.860+0000 7fd111e10640 1 --2- 172.21.15.7:0/2214439283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd10c104e90 0x7fd10c1982a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd100031610 tx=0x7fd100004270 comp rx=0 tx=0).stop 2024-06-10T01:07:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.860+0000 7fd111e10640 1 -- 172.21.15.7:0/2214439283 shutdown_connections 2024-06-10T01:07:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.860+0000 7fd111e10640 1 --2- 172.21.15.7:0/2214439283 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd0f40a2b80 0x7fd0f40a5040 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.860+0000 7fd111e10640 1 --2- 172.21.15.7:0/2214439283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd10c1077c0 0x7fd10c19f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.860+0000 7fd111e10640 1 --2- 172.21.15.7:0/2214439283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd10c104e90 0x7fd10c1982a0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.860+0000 7fd111e10640 1 --2- 172.21.15.7:0/2214439283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd10c102560 0x7fd10c197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.860+0000 7fd111e10640 1 -- 172.21.15.7:0/2214439283 >> 172.21.15.7:0/2214439283 conn(0x7fd10c0fbeb0 msgr2=0x7fd10c102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.860+0000 7fd111e10640 1 -- 172.21.15.7:0/2214439283 shutdown_connections 2024-06-10T01:07:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:22.860+0000 7fd111e10640 1 -- 172.21.15.7:0/2214439283 wait complete. 2024-06-10T01:07:23.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:23 smithi175 ceph-mon[36636]: pgmap v472: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 93 KiB/s wr, 27 op/s 2024-06-10T01:07:23.707 INFO:tasks.workunit.client.0.smithi007.stdout: 1 11458 19.71 MB/sec warmup 4 sec latency 69.520 ms 2024-06-10T01:07:23.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:23 smithi007 ceph-mon[32398]: pgmap v472: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 93 KiB/s wr, 27 op/s 2024-06-10T01:07:23.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:23 smithi078 ceph-mon[36683]: pgmap v472: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 93 KiB/s wr, 27 op/s 2024-06-10T01:07:23.923 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:07:23.923 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:07:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:24.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.467+0000 7f21d759e700 1 -- 172.21.15.7:0/3433043974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21d009b1d0 msgr2=0x7f21d00a2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:24.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.467+0000 7f21d759e700 1 --2- 172.21.15.7:0/3433043974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21d009b1d0 0x7f21d00a2d50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f21d8067f10 tx=0x7f21d8068aa0 comp rx=0 tx=0).stop 2024-06-10T01:07:24.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.467+0000 7f21d759e700 1 -- 172.21.15.7:0/3433043974 shutdown_connections 2024-06-10T01:07:24.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.467+0000 7f21d759e700 1 --2- 172.21.15.7:0/3433043974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21d009b1d0 0x7f21d00a2d50 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:24.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.467+0000 7f21d759e700 1 --2- 172.21.15.7:0/3433043974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21d00962f0 0x7f21d00966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:24.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.467+0000 7f21d759e700 1 --2- 172.21.15.7:0/3433043974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21d0096ca0 0x7f21d009aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:24.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.467+0000 7f21d759e700 1 -- 172.21.15.7:0/3433043974 >> 172.21.15.7:0/3433043974 conn(0x7f21d000b960 msgr2=0x7f21d000bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:24.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.467+0000 7f21d759e700 1 -- 172.21.15.7:0/3433043974 shutdown_connections 2024-06-10T01:07:24.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.467+0000 7f21d759e700 1 -- 172.21.15.7:0/3433043974 wait complete. 2024-06-10T01:07:24.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.468+0000 7f21d759e700 1 Processor -- start 2024-06-10T01:07:24.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.468+0000 7f21d759e700 1 -- start start 2024-06-10T01:07:24.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.468+0000 7f21d759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21d00962f0 0x7f21d0007840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:24.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.468+0000 7f21d759e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21d0096ca0 0x7f21d0007d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:24.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.468+0000 7f21d759e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21d009b1d0 0x7f21d00082c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:24.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.468+0000 7f21d759e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f21d00a60a0 con 0x7f21d00962f0 2024-06-10T01:07:24.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.468+0000 7f21d759e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f21d00a5f20 con 0x7f21d0096ca0 2024-06-10T01:07:24.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.468+0000 7f21d759e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f21d00a6220 con 0x7f21d009b1d0 2024-06-10T01:07:24.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.469+0000 7f21d5d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21d0096ca0 0x7f21d0007d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:24.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.469+0000 7f21d5d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21d0096ca0 0x7f21d0007d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:55636/0 (socket says 172.21.15.7:55636) 2024-06-10T01:07:24.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.469+0000 7f21d5d9b700 1 -- 172.21.15.7:0/3806046977 learned_addr learned my addr 172.21.15.7:0/3806046977 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:24.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.469+0000 7f21d659c700 1 --2- 172.21.15.7:0/3806046977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21d00962f0 0x7f21d0007840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:24.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.469+0000 7f21d6d9d700 1 --2- 172.21.15.7:0/3806046977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21d009b1d0 0x7f21d00082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:24.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.469+0000 7f21d5d9b700 1 -- 172.21.15.7:0/3806046977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21d009b1d0 msgr2=0x7f21d00082c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:24.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.469+0000 7f21d5d9b700 1 --2- 172.21.15.7:0/3806046977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21d009b1d0 0x7f21d00082c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:24.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.469+0000 7f21d5d9b700 1 -- 172.21.15.7:0/3806046977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21d00962f0 msgr2=0x7f21d0007840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:24.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.469+0000 7f21d5d9b700 1 --2- 172.21.15.7:0/3806046977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21d00962f0 0x7f21d0007840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:24.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.469+0000 7f21d5d9b700 1 -- 172.21.15.7:0/3806046977 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d013a9f0 con 0x7f21d0096ca0 2024-06-10T01:07:24.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.469+0000 7f21d5d9b700 1 --2- 172.21.15.7:0/3806046977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21d0096ca0 0x7f21d0007d80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00e9a0 tx=0x7f21cc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:24.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.469+0000 7f21c77fe700 1 -- 172.21.15.7:0/3806046977 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc00cbd0 con 0x7f21d0096ca0 2024-06-10T01:07:24.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.470+0000 7f21d759e700 1 -- 172.21.15.7:0/3806046977 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21d013ace0 con 0x7f21d0096ca0 2024-06-10T01:07:24.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.470+0000 7f21c77fe700 1 -- 172.21.15.7:0/3806046977 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21cc00cd70 con 0x7f21d0096ca0 2024-06-10T01:07:24.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.470+0000 7f21c77fe700 1 -- 172.21.15.7:0/3806046977 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc018750 con 0x7f21d0096ca0 2024-06-10T01:07:24.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.470+0000 7f21d759e700 1 -- 172.21.15.7:0/3806046977 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21d013b1c0 con 0x7f21d0096ca0 2024-06-10T01:07:24.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.471+0000 7f21d759e700 1 -- 172.21.15.7:0/3806046977 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21d0005a90 con 0x7f21d0096ca0 2024-06-10T01:07:24.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.472+0000 7f21c77fe700 1 -- 172.21.15.7:0/3806046977 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f21cc010430 con 0x7f21d0096ca0 2024-06-10T01:07:24.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.473+0000 7f21c77fe700 1 --2- 172.21.15.7:0/3806046977 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21c00a2c20 0x7f21c00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:24.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.473+0000 7f21c77fe700 1 -- 172.21.15.7:0/3806046977 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f21cc0cf600 con 0x7f21d0096ca0 2024-06-10T01:07:24.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.473+0000 7f21d659c700 1 --2- 172.21.15.7:0/3806046977 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21c00a2c20 0x7f21c00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:24.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.473+0000 7f21d659c700 1 --2- 172.21.15.7:0/3806046977 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21c00a2c20 0x7f21c00a50e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f21c8005950 tx=0x7f21c800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:24.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.478+0000 7f21c77fe700 1 -- 172.21.15.7:0/3806046977 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f21cc09c1e0 con 0x7f21d0096ca0 2024-06-10T01:07:24.707 INFO:tasks.workunit.client.0.smithi007.stdout: 1 14750 19.49 MB/sec warmup 5 sec latency 24.746 ms 2024-06-10T01:07:24.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.748+0000 7f21d759e700 1 -- 172.21.15.7:0/3806046977 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21d013b4c0 con 0x7f21d0096ca0 2024-06-10T01:07:24.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.749+0000 7f21c77fe700 1 -- 172.21.15.7:0/3806046977 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11855 (secure 0 0 0) 0x7f21cc09bae0 con 0x7f21d0096ca0 2024-06-10T01:07:24.745 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:07:24.746 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:05:39.175358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/2784910673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2784910673},{"type":"v1","addr":"172.21.15.175:6835","nonce":2784910673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24434,24436,24451]},"id":1}]} 2024-06-10T01:07:24.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.751+0000 7f21d759e700 1 -- 172.21.15.7:0/3806046977 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21c00a2c20 msgr2=0x7f21c00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:24.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.751+0000 7f21d759e700 1 --2- 172.21.15.7:0/3806046977 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21c00a2c20 0x7f21c00a50e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f21c8005950 tx=0x7f21c800b410 comp rx=0 tx=0).stop 2024-06-10T01:07:24.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.751+0000 7f21d759e700 1 -- 172.21.15.7:0/3806046977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21d0096ca0 msgr2=0x7f21d0007d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:24.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.751+0000 7f21d759e700 1 --2- 172.21.15.7:0/3806046977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21d0096ca0 0x7f21d0007d80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00e9a0 tx=0x7f21cc00ecb0 comp rx=0 tx=0).stop 2024-06-10T01:07:24.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.751+0000 7f21d759e700 1 -- 172.21.15.7:0/3806046977 shutdown_connections 2024-06-10T01:07:24.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.751+0000 7f21d759e700 1 --2- 172.21.15.7:0/3806046977 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21c00a2c20 0x7f21c00a50e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:24.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.751+0000 7f21d759e700 1 --2- 172.21.15.7:0/3806046977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21d009b1d0 0x7f21d00082c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:24.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.751+0000 7f21d759e700 1 --2- 172.21.15.7:0/3806046977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21d00962f0 0x7f21d0007840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:24.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.751+0000 7f21d759e700 1 --2- 172.21.15.7:0/3806046977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21d0096ca0 0x7f21d0007d80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:24.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.751+0000 7f21d759e700 1 -- 172.21.15.7:0/3806046977 >> 172.21.15.7:0/3806046977 conn(0x7f21d000b960 msgr2=0x7f21d0094240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:24.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.752+0000 7f21d759e700 1 -- 172.21.15.7:0/3806046977 shutdown_connections 2024-06-10T01:07:24.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:24.752+0000 7f21d759e700 1 -- 172.21.15.7:0/3806046977 wait complete. 2024-06-10T01:07:24.749 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-06-10T01:07:25.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:25 smithi175 ceph-mon[36636]: pgmap v473: 129 pgs: 129 active+clean; 2.0 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 132 KiB/s wr, 35 op/s 2024-06-10T01:07:25.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:25 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3806046977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:07:25.707 INFO:tasks.workunit.client.0.smithi007.stdout: 1 18103 19.42 MB/sec warmup 6 sec latency 14.019 ms 2024-06-10T01:07:25.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:25 smithi007 ceph-mon[32398]: pgmap v473: 129 pgs: 129 active+clean; 2.0 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 132 KiB/s wr, 35 op/s 2024-06-10T01:07:25.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:25 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3806046977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:07:25.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:25 smithi078 ceph-mon[36683]: pgmap v473: 129 pgs: 129 active+clean; 2.0 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 132 KiB/s wr, 35 op/s 2024-06-10T01:07:25.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:25 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3806046977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:07:26.034 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-06-10T01:07:26.035 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-06-10T01:07:26.451 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:26.699 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:26 smithi007 ceph-mon[32398]: pgmap v474: 129 pgs: 129 active+clean; 3.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 381 KiB/s wr, 124 op/s 2024-06-10T01:07:26.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:26 smithi175 ceph-mon[36636]: pgmap v474: 129 pgs: 129 active+clean; 3.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 381 KiB/s wr, 124 op/s 2024-06-10T01:07:26.707 INFO:tasks.workunit.client.0.smithi007.stdout: 1 21398 19.18 MB/sec warmup 7 sec latency 15.182 ms 2024-06-10T01:07:26.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:26 smithi078 ceph-mon[36683]: pgmap v474: 129 pgs: 129 active+clean; 3.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 381 KiB/s wr, 124 op/s 2024-06-10T01:07:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.603+0000 7fede8207700 1 -- 172.21.15.7:0/2352058154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede0100f20 msgr2=0x7fede01013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.603+0000 7fede8207700 1 --2- 172.21.15.7:0/2352058154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede0100f20 0x7fede01013a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fedd0009990 tx=0x7fedd0009ca0 comp rx=0 tx=0).stop 2024-06-10T01:07:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.603+0000 7fede8207700 1 -- 172.21.15.7:0/2352058154 shutdown_connections 2024-06-10T01:07:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.603+0000 7fede8207700 1 --2- 172.21.15.7:0/2352058154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fede010f160 0x7fede0111550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.603+0000 7fede8207700 1 --2- 172.21.15.7:0/2352058154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fede01035d0 0x7fede01009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.603+0000 7fede8207700 1 --2- 172.21.15.7:0/2352058154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede0100f20 0x7fede01013a0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:27.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.603+0000 7fede8207700 1 -- 172.21.15.7:0/2352058154 >> 172.21.15.7:0/2352058154 conn(0x7fede0077990 msgr2=0x7fede00fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:27.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.603+0000 7fede8207700 1 -- 172.21.15.7:0/2352058154 shutdown_connections 2024-06-10T01:07:27.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.603+0000 7fede8207700 1 -- 172.21.15.7:0/2352058154 wait complete. 2024-06-10T01:07:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.604+0000 7fede8207700 1 Processor -- start 2024-06-10T01:07:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.604+0000 7fede8207700 1 -- start start 2024-06-10T01:07:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.604+0000 7fede8207700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fede0100f20 0x7fede019b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.604+0000 7fede8207700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fede01035d0 0x7fede019b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.604+0000 7fede8207700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede010f160 0x7fede019f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.604+0000 7fede8207700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fede0114c70 con 0x7fede010f160 2024-06-10T01:07:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.604+0000 7fede8207700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fede0114af0 con 0x7fede01035d0 2024-06-10T01:07:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.604+0000 7fede8207700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fede0114df0 con 0x7fede0100f20 2024-06-10T01:07:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.604+0000 7fede67a4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede010f160 0x7fede019f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.604+0000 7fede67a4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede010f160 0x7fede019f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48802/0 (socket says 172.21.15.7:48802) 2024-06-10T01:07:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.604+0000 7fede57a2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fede01035d0 0x7fede019b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.604+0000 7fede5fa3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fede0100f20 0x7fede019b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.604+0000 7fede67a4700 1 -- 172.21.15.7:0/1639420031 learned_addr learned my addr 172.21.15.7:0/1639420031 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.604+0000 7fede67a4700 1 -- 172.21.15.7:0/1639420031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fede0100f20 msgr2=0x7fede019b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.604+0000 7fede67a4700 1 --2- 172.21.15.7:0/1639420031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fede0100f20 0x7fede019b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.604+0000 7fede67a4700 1 -- 172.21.15.7:0/1639420031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fede01035d0 msgr2=0x7fede019b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.604+0000 7fede67a4700 1 --2- 172.21.15.7:0/1639420031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fede01035d0 0x7fede019b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.604+0000 7fede67a4700 1 -- 172.21.15.7:0/1639420031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede01a00b0 con 0x7fede010f160 2024-06-10T01:07:27.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.604+0000 7fede57a2700 1 --2- 172.21.15.7:0/1639420031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fede01035d0 0x7fede019b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:27.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.605+0000 7fede67a4700 1 --2- 172.21.15.7:0/1639420031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede010f160 0x7fede019f970 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fedd800efc0 tx=0x7fedd800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:27.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.605+0000 7fedd6ffd700 1 -- 172.21.15.7:0/1639420031 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd80092e0 con 0x7fede010f160 2024-06-10T01:07:27.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.605+0000 7fedd6ffd700 1 -- 172.21.15.7:0/1639420031 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedd8004bb0 con 0x7fede010f160 2024-06-10T01:07:27.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.605+0000 7fedd6ffd700 1 -- 172.21.15.7:0/1639420031 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8004280 con 0x7fede010f160 2024-06-10T01:07:27.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.605+0000 7fede8207700 1 -- 172.21.15.7:0/1639420031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede01a0340 con 0x7fede010f160 2024-06-10T01:07:27.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.605+0000 7fede8207700 1 -- 172.21.15.7:0/1639420031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede01a7c20 con 0x7fede010f160 2024-06-10T01:07:27.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.606+0000 7fede8207700 1 -- 172.21.15.7:0/1639420031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fede0069180 con 0x7fede010f160 2024-06-10T01:07:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.608+0000 7fedd6ffd700 1 -- 172.21.15.7:0/1639420031 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fedd801b030 con 0x7fede010f160 2024-06-10T01:07:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.609+0000 7fedd6ffd700 1 --2- 172.21.15.7:0/1639420031 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fedcc0a2c20 0x7fedcc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.609+0000 7fede5fa3700 1 --2- 172.21.15.7:0/1639420031 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fedcc0a2c20 0x7fedcc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.609+0000 7fedd6ffd700 1 -- 172.21.15.7:0/1639420031 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fedd80cf750 con 0x7fede010f160 2024-06-10T01:07:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.609+0000 7fede5fa3700 1 --2- 172.21.15.7:0/1639420031 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fedcc0a2c20 0x7fedcc0a50e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7feddc009c60 tx=0x7feddc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:27.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.613+0000 7fedd6ffd700 1 -- 172.21.15.7:0/1639420031 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fedd8022020 con 0x7fede010f160 2024-06-10T01:07:27.707 INFO:tasks.workunit.client.0.smithi007.stdout: 1 23822 17.94 MB/sec warmup 8 sec latency 19.215 ms 2024-06-10T01:07:27.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.885+0000 7fede8207700 1 -- 172.21.15.7:0/1639420031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fede006a9e0 con 0x7fede010f160 2024-06-10T01:07:27.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.886+0000 7fedd6ffd700 1 -- 172.21.15.7:0/1639420031 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11855 (secure 0 0 0) 0x7fedd8010040 con 0x7fede010f160 2024-06-10T01:07:27.882 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:07:27.882 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:05:39.175358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/2784910673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2784910673},{"type":"v1","addr":"172.21.15.175:6835","nonce":2784910673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24434,24436,24451]},"id":1}]} 2024-06-10T01:07:27.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.888+0000 7fede8207700 1 -- 172.21.15.7:0/1639420031 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fedcc0a2c20 msgr2=0x7fedcc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:27.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.888+0000 7fede8207700 1 --2- 172.21.15.7:0/1639420031 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fedcc0a2c20 0x7fedcc0a50e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7feddc009c60 tx=0x7feddc009260 comp rx=0 tx=0).stop 2024-06-10T01:07:27.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.888+0000 7fede8207700 1 -- 172.21.15.7:0/1639420031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede010f160 msgr2=0x7fede019f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:27.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.888+0000 7fede8207700 1 --2- 172.21.15.7:0/1639420031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede010f160 0x7fede019f970 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fedd800efc0 tx=0x7fedd800c3a0 comp rx=0 tx=0).stop 2024-06-10T01:07:27.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.888+0000 7fede8207700 1 -- 172.21.15.7:0/1639420031 shutdown_connections 2024-06-10T01:07:27.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.888+0000 7fede8207700 1 --2- 172.21.15.7:0/1639420031 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fedcc0a2c20 0x7fedcc0a50e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:27.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.888+0000 7fede8207700 1 --2- 172.21.15.7:0/1639420031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede010f160 0x7fede019f970 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:27.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.888+0000 7fede8207700 1 --2- 172.21.15.7:0/1639420031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fede0100f20 0x7fede019b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:27.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.888+0000 7fede8207700 1 --2- 172.21.15.7:0/1639420031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fede01035d0 0x7fede019b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:27.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.888+0000 7fede8207700 1 -- 172.21.15.7:0/1639420031 >> 172.21.15.7:0/1639420031 conn(0x7fede0077990 msgr2=0x7fede0110540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:27.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.888+0000 7fede8207700 1 -- 172.21.15.7:0/1639420031 shutdown_connections 2024-06-10T01:07:27.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:27.888+0000 7fede8207700 1 -- 172.21.15.7:0/1639420031 wait complete. 2024-06-10T01:07:27.885 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-06-10T01:07:28.707 INFO:tasks.workunit.client.0.smithi007.stdout: 1 26854 17.82 MB/sec warmup 9 sec latency 15.025 ms 2024-06-10T01:07:28.724 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:07:28.893 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:28 smithi007 ceph-mon[32398]: pgmap v475: 129 pgs: 129 active+clean; 9.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 966 KiB/s wr, 251 op/s 2024-06-10T01:07:28.893 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:28 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1639420031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:07:28.954 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:28 smithi175 ceph-mon[36636]: pgmap v475: 129 pgs: 129 active+clean; 9.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 966 KiB/s wr, 251 op/s 2024-06-10T01:07:28.954 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:28 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1639420031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:07:29.081 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:28 smithi078 ceph-mon[36683]: pgmap v475: 129 pgs: 129 active+clean; 9.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 966 KiB/s wr, 251 op/s 2024-06-10T01:07:29.081 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:28 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1639420031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:07:29.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.308+0000 7fe5d69c3700 1 -- 172.21.15.7:0/2521938030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d0101d80 msgr2=0x7fe5d010e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:29.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.308+0000 7fe5d69c3700 1 --2- 172.21.15.7:0/2521938030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d0101d80 0x7fe5d010e220 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe5c800b0a0 tx=0x7fe5c800b3b0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.308+0000 7fe5d69c3700 1 -- 172.21.15.7:0/2521938030 shutdown_connections 2024-06-10T01:07:29.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.308+0000 7fe5d69c3700 1 --2- 172.21.15.7:0/2521938030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d0101d80 0x7fe5d010e220 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.308+0000 7fe5d69c3700 1 --2- 172.21.15.7:0/2521938030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5d0107470 0x7fe5d0107850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.308+0000 7fe5d69c3700 1 --2- 172.21.15.7:0/2521938030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5d01013c0 0x7fe5d0101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.308+0000 7fe5d69c3700 1 -- 172.21.15.7:0/2521938030 >> 172.21.15.7:0/2521938030 conn(0x7fe5d00783d0 msgr2=0x7fe5d00787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:29.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.308+0000 7fe5d69c3700 1 -- 172.21.15.7:0/2521938030 shutdown_connections 2024-06-10T01:07:29.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.308+0000 7fe5d69c3700 1 -- 172.21.15.7:0/2521938030 wait complete. 2024-06-10T01:07:29.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.309+0000 7fe5d69c3700 1 Processor -- start 2024-06-10T01:07:29.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.309+0000 7fe5d69c3700 1 -- start start 2024-06-10T01:07:29.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.309+0000 7fe5d69c3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5d01013c0 0x7fe5d019e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:29.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.309+0000 7fe5d69c3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5d0101d80 0x7fe5d019ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:29.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.309+0000 7fe5d69c3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d0107470 0x7fe5d01988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:29.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.309+0000 7fe5d69c3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d0111700 con 0x7fe5d0107470 2024-06-10T01:07:29.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.309+0000 7fe5d69c3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d0111580 con 0x7fe5d01013c0 2024-06-10T01:07:29.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.309+0000 7fe5d69c3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d0111880 con 0x7fe5d0101d80 2024-06-10T01:07:29.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.309+0000 7fe5d4f60700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d0107470 0x7fe5d01988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:29.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.309+0000 7fe5cffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5d01013c0 0x7fe5d019e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:29.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.309+0000 7fe5c7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5d0101d80 0x7fe5d019ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:29.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.309+0000 7fe5cffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5d01013c0 0x7fe5d019e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:55672/0 (socket says 172.21.15.7:55672) 2024-06-10T01:07:29.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.309+0000 7fe5c7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5d0101d80 0x7fe5d019ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:41122/0 (socket says 172.21.15.7:41122) 2024-06-10T01:07:29.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.309+0000 7fe5cffff700 1 -- 172.21.15.7:0/3011628563 learned_addr learned my addr 172.21.15.7:0/3011628563 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:29.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.309+0000 7fe5d4f60700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d0107470 0x7fe5d01988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48812/0 (socket says 172.21.15.7:48812) 2024-06-10T01:07:29.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.310+0000 7fe5cffff700 1 -- 172.21.15.7:0/3011628563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5d0101d80 msgr2=0x7fe5d019ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:29.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.310+0000 7fe5cffff700 1 --2- 172.21.15.7:0/3011628563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5d0101d80 0x7fe5d019ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.310+0000 7fe5cffff700 1 -- 172.21.15.7:0/3011628563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d0107470 msgr2=0x7fe5d01988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:29.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.310+0000 7fe5cffff700 1 --2- 172.21.15.7:0/3011628563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d0107470 0x7fe5d01988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.310+0000 7fe5cffff700 1 -- 172.21.15.7:0/3011628563 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5d0199010 con 0x7fe5d01013c0 2024-06-10T01:07:29.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.310+0000 7fe5d4f60700 1 --2- 172.21.15.7:0/3011628563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d0107470 0x7fe5d01988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:29.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.310+0000 7fe5cffff700 1 --2- 172.21.15.7:0/3011628563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5d01013c0 0x7fe5d019e6f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe5c000d950 tx=0x7fe5c000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:29.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.310+0000 7fe5cdffb700 1 -- 172.21.15.7:0/3011628563 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0016070 con 0x7fe5d01013c0 2024-06-10T01:07:29.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.310+0000 7fe5cdffb700 1 -- 172.21.15.7:0/3011628563 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5c000fd70 con 0x7fe5d01013c0 2024-06-10T01:07:29.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.310+0000 7fe5d69c3700 1 -- 172.21.15.7:0/3011628563 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5d0199300 con 0x7fe5d01013c0 2024-06-10T01:07:29.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.310+0000 7fe5cdffb700 1 -- 172.21.15.7:0/3011628563 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c000e610 con 0x7fe5d01013c0 2024-06-10T01:07:29.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.311+0000 7fe5d69c3700 1 -- 172.21.15.7:0/3011628563 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5d01a55c0 con 0x7fe5d01013c0 2024-06-10T01:07:29.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.313+0000 7fe5cdffb700 1 -- 172.21.15.7:0/3011628563 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fe5c0019070 con 0x7fe5d01013c0 2024-06-10T01:07:29.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.313+0000 7fe5cdffb700 1 --2- 172.21.15.7:0/3011628563 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe5bc0a2d40 0x7fe5bc0a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:29.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.314+0000 7fe5cdffb700 1 -- 172.21.15.7:0/3011628563 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fe5c00cedf0 con 0x7fe5d01013c0 2024-06-10T01:07:29.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.314+0000 7fe5c7fff700 1 --2- 172.21.15.7:0/3011628563 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe5bc0a2d40 0x7fe5bc0a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:29.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.314+0000 7fe5c7fff700 1 --2- 172.21.15.7:0/3011628563 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe5bc0a2d40 0x7fe5bc0a5200 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0078940 tx=0x7fe5b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:29.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.314+0000 7fe5e1246b80 1 Processor -- start 2024-06-10T01:07:29.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.315+0000 7fe5e1246b80 1 -- start start 2024-06-10T01:07:29.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.315+0000 7fe5e1246b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620478ac590 0x5620478ac970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:29.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.315+0000 7fe5e1246b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620478aceb0 0x5620478a67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:29.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.315+0000 7fe5e1246b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620478a6ce0 0x5620478a7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:29.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.315+0000 7fe5e1246b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5620477990f0 con 0x5620478a6ce0 2024-06-10T01:07:29.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.315+0000 7fe5e1246b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5620477f79e0 con 0x5620478aceb0 2024-06-10T01:07:29.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.315+0000 7fe5e1246b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56204775f5f0 con 0x5620478ac590 2024-06-10T01:07:29.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.315+0000 7fe5d4f60700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620478a6ce0 0x5620478a7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:29.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.315+0000 7fe5c7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620478aceb0 0x5620478a67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:29.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.315+0000 7fe5c7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620478aceb0 0x5620478a67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:55688/0 (socket says 172.21.15.7:55688) 2024-06-10T01:07:29.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.315+0000 7fe5c7fff700 1 -- 172.21.15.7:0/1096524103 learned_addr learned my addr 172.21.15.7:0/1096524103 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:29.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.315+0000 7fe5cffff700 1 --2- 172.21.15.7:0/1096524103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620478ac590 0x5620478ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:29.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.315+0000 7fe5c7fff700 1 -- 172.21.15.7:0/1096524103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620478ac590 msgr2=0x5620478ac970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:29.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.315+0000 7fe5c7fff700 1 --2- 172.21.15.7:0/1096524103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620478ac590 0x5620478ac970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.315+0000 7fe5c7fff700 1 -- 172.21.15.7:0/1096524103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620478a6ce0 msgr2=0x5620478a7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:29.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.315+0000 7fe5c7fff700 1 --2- 172.21.15.7:0/1096524103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620478a6ce0 0x5620478a7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.315+0000 7fe5c7fff700 1 -- 172.21.15.7:0/1096524103 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562047746a60 con 0x5620478aceb0 2024-06-10T01:07:29.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.315+0000 7fe5d4f60700 1 --2- 172.21.15.7:0/1096524103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620478a6ce0 0x5620478a7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:29.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.316+0000 7fe5c7fff700 1 --2- 172.21.15.7:0/1096524103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620478aceb0 0x5620478a67a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8011da0 tx=0x7fe5b8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acdcd8edfadb3d4d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:29.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.316+0000 7fe5c67fc700 1 -- 172.21.15.7:0/1096524103 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8004a30 con 0x5620478aceb0 2024-06-10T01:07:29.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.316+0000 7fe5c67fc700 1 -- 172.21.15.7:0/1096524103 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b8005500 con 0x5620478aceb0 2024-06-10T01:07:29.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.316+0000 7fe5c67fc700 1 -- 172.21.15.7:0/1096524103 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8003c60 con 0x5620478aceb0 2024-06-10T01:07:29.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.317+0000 7fe5e1246b80 1 -- 172.21.15.7:0/1096524103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620478aceb0 msgr2=0x5620478a67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:29.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.317+0000 7fe5e1246b80 1 --2- 172.21.15.7:0/1096524103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620478aceb0 0x5620478a67a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8011da0 tx=0x7fe5b8014000 comp rx=0 tx=0).stop 2024-06-10T01:07:29.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.317+0000 7fe5e1246b80 1 -- 172.21.15.7:0/1096524103 shutdown_connections 2024-06-10T01:07:29.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.317+0000 7fe5e1246b80 1 --2- 172.21.15.7:0/1096524103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620478a6ce0 0x5620478a7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.317+0000 7fe5e1246b80 1 --2- 172.21.15.7:0/1096524103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620478ac590 0x5620478ac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.317+0000 7fe5e1246b80 1 --2- 172.21.15.7:0/1096524103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620478aceb0 0x5620478a67a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.317+0000 7fe5e1246b80 1 -- 172.21.15.7:0/1096524103 >> 172.21.15.7:0/1096524103 conn(0x5620477d3210 msgr2=0x5620478a1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:29.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.317+0000 7fe5e1246b80 1 -- 172.21.15.7:0/1096524103 shutdown_connections 2024-06-10T01:07:29.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.317+0000 7fe5e1246b80 1 -- 172.21.15.7:0/1096524103 wait complete. 2024-06-10T01:07:29.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.317+0000 7fe5e1246b80 1 -- start start 2024-06-10T01:07:29.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.317+0000 7fe5e1246b80 1 Processor -- start 2024-06-10T01:07:29.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.318+0000 7fe5e1246b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620478a6ce0 0x5620478b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:29.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.318+0000 7fe5e1246b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620478ac590 0x5620478aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:29.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.318+0000 7fe5e1246b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620478aceb0 0x5620478b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:29.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.318+0000 7fe5cffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620478a6ce0 0x5620478b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:29.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.318+0000 7fe5cffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620478a6ce0 0x5620478b2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48828/0 (socket says 172.21.15.7:48828) 2024-06-10T01:07:29.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.318+0000 7fe5cffff700 1 -- 172.21.15.7:0/1879303396 learned_addr learned my addr 172.21.15.7:0/1879303396 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:29.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.318+0000 7fe5e1246b80 1 -- 172.21.15.7:0/1879303396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562047798150 con 0x5620478a6ce0 2024-06-10T01:07:29.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.318+0000 7fe5e1246b80 1 -- 172.21.15.7:0/1879303396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5620477f39a0 con 0x5620478ac590 2024-06-10T01:07:29.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.318+0000 7fe5e1246b80 1 -- 172.21.15.7:0/1879303396 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5620478b38e0 con 0x5620478aceb0 2024-06-10T01:07:29.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.318+0000 7fe5e1246b80 10 client.0 _ll_register_callbacks cb 0x56204778d740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:29.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.318+0000 7fe5c7fff700 1 --2- 172.21.15.7:0/1879303396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620478ac590 0x5620478aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:29.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.318+0000 7fe5d4f60700 1 --2- 172.21.15.7:0/1879303396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620478aceb0 0x5620478b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:29.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.318+0000 7fe5cffff700 1 -- 172.21.15.7:0/1879303396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620478aceb0 msgr2=0x5620478b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:29.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.318+0000 7fe5cffff700 1 --2- 172.21.15.7:0/1879303396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620478aceb0 0x5620478b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.318+0000 7fe5cffff700 1 -- 172.21.15.7:0/1879303396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620478ac590 msgr2=0x5620478aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:29.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.318+0000 7fe5cffff700 1 --2- 172.21.15.7:0/1879303396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620478ac590 0x5620478aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.318+0000 7fe5cffff700 1 -- 172.21.15.7:0/1879303396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620477bd8e0 con 0x5620478a6ce0 2024-06-10T01:07:29.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.318+0000 7fe5cffff700 1 --2- 172.21.15.7:0/1879303396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620478a6ce0 0x5620478b2c70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe5c00cc570 tx=0x7fe5c009bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=631ddfd2702cd69d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:29.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.318+0000 7fe5af7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:07:29.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.319+0000 7fe5af7fe700 1 -- 172.21.15.7:0/1879303396 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0020070 con 0x5620478a6ce0 2024-06-10T01:07:29.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.319+0000 7fe5af7fe700 1 -- 172.21.15.7:0/1879303396 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fe5c0010710 con 0x5620478a6ce0 2024-06-10T01:07:29.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.319+0000 7fe5e1246b80 1 -- 172.21.15.7:0/1879303396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5b8005500 con 0x5620478a6ce0 2024-06-10T01:07:29.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.319+0000 7fe5af7fe700 1 -- 172.21.15.7:0/1879303396 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe5c00a3f40 con 0x5620478a6ce0 2024-06-10T01:07:29.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.319+0000 7fe5e1246b80 10 client.15813 fetch_fsmap learned FSMap version 27 2024-06-10T01:07:29.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.319+0000 7fe5e1246b80 1 -- 172.21.15.7:0/1879303396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5620478be0c0 con 0x5620478a6ce0 2024-06-10T01:07:29.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.319+0000 7fe5af7fe700 1 -- 172.21.15.7:0/1879303396 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5c0010e10 con 0x5620478a6ce0 2024-06-10T01:07:29.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.319+0000 7fe5af7fe700 1 -- 172.21.15.7:0/1879303396 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe5c009f170 con 0x5620478a6ce0 2024-06-10T01:07:29.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.319+0000 7fe5e1246b80 10 client.15813 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:07:29.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.319+0000 7fe5e1246b80 10 client.15813 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:07:29.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.319+0000 7fe5e1246b80 20 client.15813 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:29.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.319+0000 7fe5e1246b80 1 --2- 172.21.15.7:0/1879303396 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5620478c0650 0x5620478c2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:29.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.319+0000 7fe5e1246b80 4 client.15813 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:07:29.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.319+0000 7fe5e1246b80 1 -- 172.21.15.7:0/1879303396 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe5b8005500 con 0x5620478c0650 2024-06-10T01:07:29.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.320+0000 7fe5c7fff700 1 --2- 172.21.15.7:0/1879303396 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5620478c0650 0x5620478c2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:29.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.320+0000 7fe5af7fe700 10 client.15813 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:29.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.320+0000 7fe5c7fff700 1 --2- 172.21.15.7:0/1879303396 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5620478c0650 0x5620478c2a40 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40922d02774a5168 server_cookie=6eb0940edad0331f in_seq=0 out_seq=0 2024-06-10T01:07:29.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.321+0000 7fe5af7fe700 1 -- 172.21.15.7:0/1879303396 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fe5b8005500 con 0x5620478c0650 2024-06-10T01:07:29.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.321+0000 7fe5af7fe700 10 client.15813 handle_command_reply: tid=0 2024-06-10T01:07:29.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.321+0000 7fe5e1246b80 1 client.15813 shutdown 2024-06-10T01:07:29.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.321+0000 7fe5e1246b80 1 -- 172.21.15.7:0/1879303396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620478a6ce0 msgr2=0x5620478b2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:29.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.321+0000 7fe5e1246b80 1 --2- 172.21.15.7:0/1879303396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620478a6ce0 0x5620478b2c70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe5c00cc570 tx=0x7fe5c009bb00 comp rx=0 tx=0).stop 2024-06-10T01:07:29.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.322+0000 7fe5e1246b80 1 -- 172.21.15.7:0/1879303396 shutdown_connections 2024-06-10T01:07:29.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.322+0000 7fe5e1246b80 1 --2- 172.21.15.7:0/1879303396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620478aceb0 0x5620478b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.322+0000 7fe5e1246b80 1 --2- 172.21.15.7:0/1879303396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620478a6ce0 0x5620478b2c70 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.322+0000 7fe5e1246b80 1 --2- 172.21.15.7:0/1879303396 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5620478c0650 0x5620478c2a40 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.322+0000 7fe5e1246b80 1 --2- 172.21.15.7:0/1879303396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620478ac590 0x5620478aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.322+0000 7fe5e1246b80 1 -- 172.21.15.7:0/1879303396 >> 172.21.15.7:0/1879303396 conn(0x5620477d3210 msgr2=0x5620478a0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:29.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.322+0000 7fe5af7fe700 0 client.15813 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:29.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.322+0000 7fe5e1246b80 1 -- 172.21.15.7:0/1879303396 shutdown_connections 2024-06-10T01:07:29.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.322+0000 7fe5e1246b80 1 -- 172.21.15.7:0/1879303396 wait complete. 2024-06-10T01:07:29.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.322+0000 7fe5e1246b80 20 client.15813 trim_cache size 0 max 16384 2024-06-10T01:07:29.337 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:29.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.345+0000 7fe5e1246b80 1 Processor -- start 2024-06-10T01:07:29.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.345+0000 7fe5e1246b80 1 -- start start 2024-06-10T01:07:29.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.345+0000 7fe5e1246b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620478a6ce0 0x5620478a67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:29.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.345+0000 7fe5e1246b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620478ac590 0x5620478b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:29.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.345+0000 7fe5e1246b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620478aceb0 0x5620478a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:29.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.345+0000 7fe5e1246b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d01b2cd0 con 0x5620478a6ce0 2024-06-10T01:07:29.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.345+0000 7fe5e1246b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d01b0560 con 0x5620478aceb0 2024-06-10T01:07:29.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.345+0000 7fe5e1246b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d01ae3d0 con 0x5620478ac590 2024-06-10T01:07:29.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.345+0000 7fe5cffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620478a6ce0 0x5620478a67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:29.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.345+0000 7fe5cffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620478a6ce0 0x5620478a67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48830/0 (socket says 172.21.15.7:48830) 2024-06-10T01:07:29.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.345+0000 7fe5cffff700 1 -- 172.21.15.7:0/1557229675 learned_addr learned my addr 172.21.15.7:0/1557229675 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:29.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.345+0000 7fe5d4f60700 1 --2- 172.21.15.7:0/1557229675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620478aceb0 0x5620478a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:29.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.345+0000 7fe5c7fff700 1 --2- 172.21.15.7:0/1557229675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620478ac590 0x5620478b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:29.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.346+0000 7fe5d4f60700 1 -- 172.21.15.7:0/1557229675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620478ac590 msgr2=0x5620478b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:29.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.346+0000 7fe5d4f60700 1 --2- 172.21.15.7:0/1557229675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620478ac590 0x5620478b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.346+0000 7fe5d4f60700 1 -- 172.21.15.7:0/1557229675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620478a6ce0 msgr2=0x5620478a67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:29.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.346+0000 7fe5d4f60700 1 --2- 172.21.15.7:0/1557229675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620478a6ce0 0x5620478a67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.346+0000 7fe5d4f60700 1 -- 172.21.15.7:0/1557229675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620478c3020 con 0x5620478aceb0 2024-06-10T01:07:29.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.346+0000 7fe5cffff700 1 --2- 172.21.15.7:0/1557229675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620478a6ce0 0x5620478a67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:29.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.346+0000 7fe5d4f60700 1 --2- 172.21.15.7:0/1557229675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620478aceb0 0x5620478a13c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8009fd0 tx=0x7fe5c8006210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2a1eeb85ff9dd32 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:29.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.346+0000 7fe5c5ffb700 1 -- 172.21.15.7:0/1557229675 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c802a020 con 0x5620478aceb0 2024-06-10T01:07:29.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.346+0000 7fe5c5ffb700 1 -- 172.21.15.7:0/1557229675 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5c8009cc0 con 0x5620478aceb0 2024-06-10T01:07:29.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.346+0000 7fe5c5ffb700 1 -- 172.21.15.7:0/1557229675 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c80052a0 con 0x5620478aceb0 2024-06-10T01:07:29.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.347+0000 7fe5e1246b80 1 -- 172.21.15.7:0/1557229675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620478aceb0 msgr2=0x5620478a13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:29.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.347+0000 7fe5e1246b80 1 --2- 172.21.15.7:0/1557229675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620478aceb0 0x5620478a13c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8009fd0 tx=0x7fe5c8006210 comp rx=0 tx=0).stop 2024-06-10T01:07:29.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.347+0000 7fe5e1246b80 1 -- 172.21.15.7:0/1557229675 shutdown_connections 2024-06-10T01:07:29.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.347+0000 7fe5e1246b80 1 --2- 172.21.15.7:0/1557229675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620478aceb0 0x5620478a13c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.347+0000 7fe5e1246b80 1 --2- 172.21.15.7:0/1557229675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620478a6ce0 0x5620478a67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.347+0000 7fe5e1246b80 1 --2- 172.21.15.7:0/1557229675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620478ac590 0x5620478b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.347+0000 7fe5e1246b80 1 -- 172.21.15.7:0/1557229675 >> 172.21.15.7:0/1557229675 conn(0x5620477d3210 msgr2=0x5620478b00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:29.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.347+0000 7fe5e1246b80 1 -- 172.21.15.7:0/1557229675 shutdown_connections 2024-06-10T01:07:29.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.347+0000 7fe5e1246b80 1 -- 172.21.15.7:0/1557229675 wait complete. 2024-06-10T01:07:29.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.347+0000 7fe5e1246b80 1 -- start start 2024-06-10T01:07:29.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.348+0000 7fe5e1246b80 1 Processor -- start 2024-06-10T01:07:29.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.348+0000 7fe5e1246b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620478a6ce0 0x5620478c8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:29.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.348+0000 7fe5e1246b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620478ac590 0x5620478c8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:29.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.348+0000 7fe5e1246b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620478aceb0 0x5620478c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:29.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.348+0000 7fe5e1246b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d01aea20 con 0x5620478aceb0 2024-06-10T01:07:29.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.348+0000 7fe5e1246b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d01afe90 con 0x5620478ac590 2024-06-10T01:07:29.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.348+0000 7fe5e1246b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d01af0f0 con 0x5620478a6ce0 2024-06-10T01:07:29.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.348+0000 7fe5e1246b80 10 client.0 _ll_register_callbacks cb 0x56204778d740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:29.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.348+0000 7fe5cffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620478a6ce0 0x5620478c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:29.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.348+0000 7fe5d4f60700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620478aceb0 0x5620478c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:29.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.348+0000 7fe5cffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620478a6ce0 0x5620478c8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:41146/0 (socket says 172.21.15.7:41146) 2024-06-10T01:07:29.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.348+0000 7fe5cffff700 1 -- 172.21.15.7:0/92826945 learned_addr learned my addr 172.21.15.7:0/92826945 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:29.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.348+0000 7fe5c7fff700 1 --2- 172.21.15.7:0/92826945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620478ac590 0x5620478c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:29.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.349+0000 7fe5cffff700 1 -- 172.21.15.7:0/92826945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620478ac590 msgr2=0x5620478c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:29.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.349+0000 7fe5cffff700 1 --2- 172.21.15.7:0/92826945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620478ac590 0x5620478c8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.349+0000 7fe5cffff700 1 -- 172.21.15.7:0/92826945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620478aceb0 msgr2=0x5620478c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:29.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.349+0000 7fe5cffff700 1 --2- 172.21.15.7:0/92826945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620478aceb0 0x5620478c8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.349+0000 7fe5cffff700 1 -- 172.21.15.7:0/92826945 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56204792a450 con 0x5620478a6ce0 2024-06-10T01:07:29.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.349+0000 7fe5cffff700 1 --2- 172.21.15.7:0/92826945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620478a6ce0 0x5620478c8050 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe5c00bb180 tx=0x7fe5c0010b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f379cff7b6f1fc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:29.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.349+0000 7fe5af7fe700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:07:29.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.349+0000 7fe5af7fe700 1 -- 172.21.15.7:0/92826945 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0020040 con 0x5620478a6ce0 2024-06-10T01:07:29.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.349+0000 7fe5af7fe700 1 -- 172.21.15.7:0/92826945 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fe5c00a3ab0 con 0x5620478a6ce0 2024-06-10T01:07:29.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.349+0000 7fe5e1246b80 1 -- 172.21.15.7:0/92826945 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5c8009cc0 con 0x5620478a6ce0 2024-06-10T01:07:29.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.349+0000 7fe5af7fe700 1 -- 172.21.15.7:0/92826945 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe5c009b0a0 con 0x5620478a6ce0 2024-06-10T01:07:29.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.349+0000 7fe5e1246b80 10 client.25462 fetch_fsmap learned FSMap version 27 2024-06-10T01:07:29.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.349+0000 7fe5e1246b80 1 -- 172.21.15.7:0/92826945 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56204792b6d0 con 0x5620478a6ce0 2024-06-10T01:07:29.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.350+0000 7fe5af7fe700 1 -- 172.21.15.7:0/92826945 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5c8009cc0 con 0x5620478a6ce0 2024-06-10T01:07:29.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.350+0000 7fe5af7fe700 1 -- 172.21.15.7:0/92826945 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe5c0097020 con 0x5620478a6ce0 2024-06-10T01:07:29.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.350+0000 7fe5e1246b80 10 client.25462 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:07:29.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.350+0000 7fe5e1246b80 10 client.25462 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:07:29.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.350+0000 7fe5e1246b80 20 client.25462 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:29.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.350+0000 7fe5e1246b80 1 --2- 172.21.15.7:0/92826945 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56204792be20 0x56204792e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:29.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.350+0000 7fe5e1246b80 4 client.25462 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-06-10T01:07:29.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.350+0000 7fe5e1246b80 1 -- 172.21.15.7:0/92826945 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x562047804ee0 con 0x56204792be20 2024-06-10T01:07:29.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.350+0000 7fe5c7fff700 1 --2- 172.21.15.7:0/92826945 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56204792be20 0x56204792e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:29.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.351+0000 7fe5af7fe700 10 client.25462 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:29.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.351+0000 7fe5c7fff700 1 --2- 172.21.15.7:0/92826945 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56204792be20 0x56204792e240 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ca075b0e29e045e server_cookie=b34f09ca46de3927 in_seq=0 out_seq=0 2024-06-10T01:07:29.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.553+0000 7fe5af7fe700 1 -- 172.21.15.7:0/92826945 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x562047804ee0 con 0x56204792be20 2024-06-10T01:07:29.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.554+0000 7fe5af7fe700 10 client.25462 handle_command_reply: tid=0 2024-06-10T01:07:29.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.554+0000 7fe5e1246b80 1 client.25462 shutdown 2024-06-10T01:07:29.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.554+0000 7fe5e1246b80 1 -- 172.21.15.7:0/92826945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620478a6ce0 msgr2=0x5620478c8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:29.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.554+0000 7fe5e1246b80 1 --2- 172.21.15.7:0/92826945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620478a6ce0 0x5620478c8050 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe5c00bb180 tx=0x7fe5c0010b10 comp rx=0 tx=0).stop 2024-06-10T01:07:29.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.554+0000 7fe5e1246b80 1 -- 172.21.15.7:0/92826945 shutdown_connections 2024-06-10T01:07:29.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.554+0000 7fe5e1246b80 1 --2- 172.21.15.7:0/92826945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620478aceb0 0x5620478c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.554+0000 7fe5e1246b80 1 --2- 172.21.15.7:0/92826945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620478a6ce0 0x5620478c8050 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.554+0000 7fe5e1246b80 1 --2- 172.21.15.7:0/92826945 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56204792be20 0x56204792e240 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.554+0000 7fe5af7fe700 0 client.25462 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:29.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.554+0000 7fe5e1246b80 1 --2- 172.21.15.7:0/92826945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620478ac590 0x5620478c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.554+0000 7fe5e1246b80 1 -- 172.21.15.7:0/92826945 >> 172.21.15.7:0/92826945 conn(0x5620477d3210 msgr2=0x5620478a5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:29.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.554+0000 7fe5e1246b80 1 -- 172.21.15.7:0/92826945 shutdown_connections 2024-06-10T01:07:29.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.554+0000 7fe5e1246b80 1 -- 172.21.15.7:0/92826945 wait complete. 2024-06-10T01:07:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.554+0000 7fe5e1246b80 20 client.25462 trim_cache size 0 max 16384 2024-06-10T01:07:29.552 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:07:29.552 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 27, 2024-06-10T01:07:29.552 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:07:29.552 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 6, 2024-06-10T01:07:29.552 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:07:29.552 INFO:teuthology.orchestra.run.smithi007.stdout: "19da4be8": { 2024-06-10T01:07:29.552 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 6, 2024-06-10T01:07:29.552 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:07:29.552 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:07:29.553 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:07:29.553 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:07:29.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:07:29.553 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:07:29.553 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 150.0, 2024-06-10T01:07:29.553 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:07:29.553 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:07:29.553 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:07:29.553 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:07:29.553 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:07:29.553 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:07:29.553 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:07:29.553 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:07:29.553 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:07:29.554 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:07:29.554 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:07:29.554 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:07:29.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.555+0000 7fe5d69c3700 1 -- 172.21.15.7:0/3011628563 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe5bc0a2d40 msgr2=0x7fe5bc0a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:29.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.555+0000 7fe5d69c3700 1 --2- 172.21.15.7:0/3011628563 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe5bc0a2d40 0x7fe5bc0a5200 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0078940 tx=0x7fe5b8009260 comp rx=0 tx=0).stop 2024-06-10T01:07:29.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.555+0000 7fe5d69c3700 1 -- 172.21.15.7:0/3011628563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5d01013c0 msgr2=0x7fe5d019e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:29.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.555+0000 7fe5d69c3700 1 --2- 172.21.15.7:0/3011628563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5d01013c0 0x7fe5d019e6f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe5c000d950 tx=0x7fe5c000dc60 comp rx=0 tx=0).stop 2024-06-10T01:07:29.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.556+0000 7fe5d69c3700 1 -- 172.21.15.7:0/3011628563 shutdown_connections 2024-06-10T01:07:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.556+0000 7fe5d69c3700 1 --2- 172.21.15.7:0/3011628563 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe5bc0a2d40 0x7fe5bc0a5200 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.556+0000 7fe5d69c3700 1 --2- 172.21.15.7:0/3011628563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d0107470 0x7fe5d01988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.556+0000 7fe5d69c3700 1 --2- 172.21.15.7:0/3011628563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5d01013c0 0x7fe5d019e6f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.556+0000 7fe5d69c3700 1 --2- 172.21.15.7:0/3011628563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5d0101d80 0x7fe5d019ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.556+0000 7fe5d69c3700 1 -- 172.21.15.7:0/3011628563 >> 172.21.15.7:0/3011628563 conn(0x7fe5d00783d0 msgr2=0x7fe5d0100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.556+0000 7fe5d69c3700 1 -- 172.21.15.7:0/3011628563 shutdown_connections 2024-06-10T01:07:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:29.556+0000 7fe5d69c3700 1 -- 172.21.15.7:0/3011628563 wait complete. 2024-06-10T01:07:29.707 INFO:tasks.workunit.client.0.smithi007.stdout: 1 28947 17.40 MB/sec warmup 10 sec latency 162.983 ms 2024-06-10T01:07:30.396 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '19da4be8', quiesce took 9.4 seconds. Will release after: 20.6 2024-06-10T01:07:30.707 INFO:tasks.workunit.client.0.smithi007.stdout: 1 28947 15.82 MB/sec warmup 11 sec latency 1163.077 ms 2024-06-10T01:07:30.954 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:30 smithi175 ceph-mon[36636]: pgmap v476: 129 pgs: 129 active+clean; 12 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.2 MiB/s wr, 262 op/s 2024-06-10T01:07:30.976 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:30 smithi007 ceph-mon[32398]: pgmap v476: 129 pgs: 129 active+clean; 12 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.2 MiB/s wr, 262 op/s 2024-06-10T01:07:31.080 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:30 smithi078 ceph-mon[36683]: pgmap v476: 129 pgs: 129 active+clean; 12 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.2 MiB/s wr, 262 op/s 2024-06-10T01:07:31.707 INFO:tasks.workunit.client.0.smithi007.stdout: 1 28947 14.50 MB/sec warmup 12 sec latency 2163.160 ms 2024-06-10T01:07:31.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.906+0000 7f6e5b228700 1 -- 172.21.15.7:0/1410069472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e541015d0 msgr2=0x7f6e541019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:31.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.906+0000 7f6e5b228700 1 --2- 172.21.15.7:0/1410069472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e541015d0 0x7f6e541019b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c0099c0 tx=0x7f6e3c009cd0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.906+0000 7f6e5b228700 1 -- 172.21.15.7:0/1410069472 shutdown_connections 2024-06-10T01:07:31.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.906+0000 7f6e5b228700 1 --2- 172.21.15.7:0/1410069472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e541066b0 0x7f6e5410e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.906+0000 7f6e5b228700 1 --2- 172.21.15.7:0/1410069472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e541015d0 0x7f6e541019b0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.906+0000 7f6e5b228700 1 --2- 172.21.15.7:0/1410069472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e54101f80 0x7f6e54105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.906+0000 7f6e5b228700 1 -- 172.21.15.7:0/1410069472 >> 172.21.15.7:0/1410069472 conn(0x7f6e54078400 msgr2=0x7f6e54078810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:31.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.906+0000 7f6e5b228700 1 -- 172.21.15.7:0/1410069472 shutdown_connections 2024-06-10T01:07:31.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.906+0000 7f6e5b228700 1 -- 172.21.15.7:0/1410069472 wait complete. 2024-06-10T01:07:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.907+0000 7f6e5b228700 1 Processor -- start 2024-06-10T01:07:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.907+0000 7f6e5b228700 1 -- start start 2024-06-10T01:07:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.907+0000 7f6e5b228700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e541015d0 0x7f6e5419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.907+0000 7f6e5b228700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e54101f80 0x7f6e5419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.907+0000 7f6e5b228700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e541066b0 0x7f6e54198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.907+0000 7f6e5b228700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6e54112810 con 0x7f6e541015d0 2024-06-10T01:07:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.908+0000 7f6e5b228700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6e54112690 con 0x7f6e541066b0 2024-06-10T01:07:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.908+0000 7f6e5b228700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6e54112990 con 0x7f6e54101f80 2024-06-10T01:07:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.908+0000 7f6e58fc4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e541015d0 0x7f6e5419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.908+0000 7f6e58fc4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e541015d0 0x7f6e5419e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34930/0 (socket says 172.21.15.7:34930) 2024-06-10T01:07:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.908+0000 7f6e58fc4700 1 -- 172.21.15.7:0/422473899 learned_addr learned my addr 172.21.15.7:0/422473899 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.908+0000 7f6e53fff700 1 --2- 172.21.15.7:0/422473899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e54101f80 0x7f6e5419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.908+0000 7f6e597c5700 1 --2- 172.21.15.7:0/422473899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e541066b0 0x7f6e54198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.908+0000 7f6e58fc4700 1 -- 172.21.15.7:0/422473899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e54101f80 msgr2=0x7f6e5419edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.908+0000 7f6e58fc4700 1 --2- 172.21.15.7:0/422473899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e54101f80 0x7f6e5419edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.908+0000 7f6e58fc4700 1 -- 172.21.15.7:0/422473899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e541066b0 msgr2=0x7f6e54198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.908+0000 7f6e58fc4700 1 --2- 172.21.15.7:0/422473899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e541066b0 0x7f6e54198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.908+0000 7f6e58fc4700 1 -- 172.21.15.7:0/422473899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e54199190 con 0x7f6e541015d0 2024-06-10T01:07:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.908+0000 7f6e58fc4700 1 --2- 172.21.15.7:0/422473899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e541015d0 0x7f6e5419e870 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c006010 tx=0x7f6e3c0038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.908+0000 7f6e51ffb700 1 -- 172.21.15.7:0/422473899 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c027070 con 0x7f6e541015d0 2024-06-10T01:07:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.909+0000 7f6e5b228700 1 -- 172.21.15.7:0/422473899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e54199420 con 0x7f6e541015d0 2024-06-10T01:07:31.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.908+0000 7f6e51ffb700 1 -- 172.21.15.7:0/422473899 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e3c004050 con 0x7f6e541015d0 2024-06-10T01:07:31.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.909+0000 7f6e51ffb700 1 -- 172.21.15.7:0/422473899 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c022d40 con 0x7f6e541015d0 2024-06-10T01:07:31.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.909+0000 7f6e5b228700 1 -- 172.21.15.7:0/422473899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e541a5740 con 0x7f6e541015d0 2024-06-10T01:07:31.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.911+0000 7f6e51ffb700 1 -- 172.21.15.7:0/422473899 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f6e3c02a070 con 0x7f6e541015d0 2024-06-10T01:07:31.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.912+0000 7f6e51ffb700 1 --2- 172.21.15.7:0/422473899 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6e400a2c20 0x7f6e400a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:31.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.912+0000 7f6e53fff700 1 --2- 172.21.15.7:0/422473899 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6e400a2c20 0x7f6e400a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:31.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.912+0000 7f6e51ffb700 1 -- 172.21.15.7:0/422473899 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f6e3c0d0250 con 0x7f6e541015d0 2024-06-10T01:07:31.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.912+0000 7f6e53fff700 1 --2- 172.21.15.7:0/422473899 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6e400a2c20 0x7f6e400a50e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6e54107a80 tx=0x7f6e440079e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:31.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.914+0000 7f6e65aabb80 1 Processor -- start 2024-06-10T01:07:31.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.914+0000 7f6e65aabb80 1 -- start start 2024-06-10T01:07:31.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.914+0000 7f6e65aabb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e934c7f60 0x560e934c8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:31.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.914+0000 7f6e65aabb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e934c8880 0x560e934c2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:31.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.914+0000 7f6e65aabb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e934c27a0 0x560e934c2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:31.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.914+0000 7f6e65aabb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560e933b4f60 con 0x560e934c7f60 2024-06-10T01:07:31.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.914+0000 7f6e65aabb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560e93413850 con 0x560e934c27a0 2024-06-10T01:07:31.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.914+0000 7f6e65aabb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560e9337b460 con 0x560e934c8880 2024-06-10T01:07:31.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.914+0000 7f6e58fc4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e934c7f60 0x560e934c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:31.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.914+0000 7f6e58fc4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e934c7f60 0x560e934c8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34936/0 (socket says 172.21.15.7:34936) 2024-06-10T01:07:31.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.914+0000 7f6e58fc4700 1 -- 172.21.15.7:0/3007012366 learned_addr learned my addr 172.21.15.7:0/3007012366 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:31.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.914+0000 7f6e53fff700 1 --2- 172.21.15.7:0/3007012366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e934c8880 0x560e934c2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:31.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.914+0000 7f6e597c5700 1 --2- 172.21.15.7:0/3007012366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e934c27a0 0x560e934c2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:31.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.914+0000 7f6e58fc4700 1 -- 172.21.15.7:0/3007012366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e934c8880 msgr2=0x560e934c2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:31.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.914+0000 7f6e58fc4700 1 --2- 172.21.15.7:0/3007012366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e934c8880 0x560e934c2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.914+0000 7f6e58fc4700 1 -- 172.21.15.7:0/3007012366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e934c27a0 msgr2=0x560e934c2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:31.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.914+0000 7f6e58fc4700 1 --2- 172.21.15.7:0/3007012366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e934c27a0 0x560e934c2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.914+0000 7f6e58fc4700 1 -- 172.21.15.7:0/3007012366 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e933628d0 con 0x560e934c7f60 2024-06-10T01:07:31.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.915+0000 7f6e58fc4700 1 --2- 172.21.15.7:0/3007012366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e934c7f60 0x560e934c8340 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c09c750 tx=0x7f6e3c022810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1e0187db786b2d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:31.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.915+0000 7f6e4affd700 1 -- 172.21.15.7:0/3007012366 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c0a30c0 con 0x560e934c7f60 2024-06-10T01:07:31.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.915+0000 7f6e4affd700 1 -- 172.21.15.7:0/3007012366 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e3c09cec0 con 0x560e934c7f60 2024-06-10T01:07:31.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.915+0000 7f6e4affd700 1 -- 172.21.15.7:0/3007012366 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c09ca20 con 0x560e934c7f60 2024-06-10T01:07:31.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.915+0000 7f6e65aabb80 1 -- 172.21.15.7:0/3007012366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e934c7f60 msgr2=0x560e934c8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:31.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.915+0000 7f6e65aabb80 1 --2- 172.21.15.7:0/3007012366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e934c7f60 0x560e934c8340 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c09c750 tx=0x7f6e3c022810 comp rx=0 tx=0).stop 2024-06-10T01:07:31.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.915+0000 7f6e65aabb80 1 -- 172.21.15.7:0/3007012366 shutdown_connections 2024-06-10T01:07:31.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.915+0000 7f6e65aabb80 1 --2- 172.21.15.7:0/3007012366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e934c27a0 0x560e934c2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.915+0000 7f6e65aabb80 1 --2- 172.21.15.7:0/3007012366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e934c7f60 0x560e934c8340 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.915+0000 7f6e65aabb80 1 --2- 172.21.15.7:0/3007012366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e934c8880 0x560e934c2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.916+0000 7f6e65aabb80 1 -- 172.21.15.7:0/3007012366 >> 172.21.15.7:0/3007012366 conn(0x560e933ef080 msgr2=0x560e934bd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:31.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.916+0000 7f6e65aabb80 1 -- 172.21.15.7:0/3007012366 shutdown_connections 2024-06-10T01:07:31.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.916+0000 7f6e65aabb80 1 -- 172.21.15.7:0/3007012366 wait complete. 2024-06-10T01:07:31.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.916+0000 7f6e65aabb80 1 -- start start 2024-06-10T01:07:31.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.916+0000 7f6e65aabb80 1 Processor -- start 2024-06-10T01:07:31.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.916+0000 7f6e65aabb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e934c27a0 0x560e934ce610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:31.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.916+0000 7f6e65aabb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e934c7f60 0x560e934cb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:31.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.916+0000 7f6e65aabb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e934c8880 0x560e934cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:31.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.916+0000 7f6e65aabb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560e933b3fc0 con 0x560e934c8880 2024-06-10T01:07:31.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.916+0000 7f6e65aabb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560e9340f810 con 0x560e934c27a0 2024-06-10T01:07:31.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.916+0000 7f6e65aabb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560e934cf010 con 0x560e934c7f60 2024-06-10T01:07:31.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.917+0000 7f6e65aabb80 10 client.0 _ll_register_callbacks cb 0x560e933a95b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:31.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.917+0000 7f6e597c5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e934c8880 0x560e934cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:31.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.917+0000 7f6e58fc4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e934c27a0 0x560e934ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:31.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.917+0000 7f6e597c5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e934c8880 0x560e934cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34944/0 (socket says 172.21.15.7:34944) 2024-06-10T01:07:31.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.917+0000 7f6e597c5700 1 -- 172.21.15.7:0/835506890 learned_addr learned my addr 172.21.15.7:0/835506890 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:31.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.917+0000 7f6e53fff700 1 --2- 172.21.15.7:0/835506890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e934c7f60 0x560e934cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:31.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.917+0000 7f6e58fc4700 1 -- 172.21.15.7:0/835506890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e934c7f60 msgr2=0x560e934cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:31.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.917+0000 7f6e58fc4700 1 --2- 172.21.15.7:0/835506890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e934c7f60 0x560e934cb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.917+0000 7f6e58fc4700 1 -- 172.21.15.7:0/835506890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e934c8880 msgr2=0x560e934cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:31.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.917+0000 7f6e58fc4700 1 --2- 172.21.15.7:0/835506890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e934c8880 0x560e934cbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.917+0000 7f6e58fc4700 1 -- 172.21.15.7:0/835506890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e933d9750 con 0x560e934c27a0 2024-06-10T01:07:31.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.917+0000 7f6e597c5700 1 --2- 172.21.15.7:0/835506890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e934c8880 0x560e934cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:31.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.917+0000 7f6e58fc4700 1 --2- 172.21.15.7:0/835506890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e934c27a0 0x560e934ce610 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c0a5560 tx=0x7f6e3c09c220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26c2c6c8600678de server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:31.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.917+0000 7f6e33fff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:07:31.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.917+0000 7f6e33fff700 1 -- 172.21.15.7:0/835506890 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c0b4060 con 0x560e934c27a0 2024-06-10T01:07:31.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.917+0000 7f6e65aabb80 1 -- 172.21.15.7:0/835506890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e3c09cec0 con 0x560e934c27a0 2024-06-10T01:07:31.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.917+0000 7f6e33fff700 1 -- 172.21.15.7:0/835506890 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f6e3c0870b0 con 0x560e934c27a0 2024-06-10T01:07:31.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.918+0000 7f6e33fff700 1 -- 172.21.15.7:0/835506890 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6e3c0a0b70 con 0x560e934c27a0 2024-06-10T01:07:31.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.918+0000 7f6e65aabb80 10 client.25421 fetch_fsmap learned FSMap version 27 2024-06-10T01:07:31.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.918+0000 7f6e65aabb80 1 -- 172.21.15.7:0/835506890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560e934d9740 con 0x560e934c27a0 2024-06-10T01:07:31.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.918+0000 7f6e33fff700 1 -- 172.21.15.7:0/835506890 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e3c09cec0 con 0x560e934c27a0 2024-06-10T01:07:31.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.918+0000 7f6e33fff700 1 -- 172.21.15.7:0/835506890 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6e3c09bda0 con 0x560e934c27a0 2024-06-10T01:07:31.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.918+0000 7f6e65aabb80 10 client.25421 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:07:31.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.918+0000 7f6e65aabb80 10 client.25421 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:07:31.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.918+0000 7f6e65aabb80 20 client.25421 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:31.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.918+0000 7f6e65aabb80 1 --2- 172.21.15.7:0/835506890 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x560e934dbc10 0x560e934de000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:31.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.918+0000 7f6e65aabb80 4 client.25421 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:07:31.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.918+0000 7f6e65aabb80 1 -- 172.21.15.7:0/835506890 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e93420d50 con 0x560e934dbc10 2024-06-10T01:07:31.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.918+0000 7f6e53fff700 1 --2- 172.21.15.7:0/835506890 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x560e934dbc10 0x560e934de000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:31.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.919+0000 7f6e33fff700 10 client.25421 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:07:31.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.919+0000 7f6e53fff700 1 --2- 172.21.15.7:0/835506890 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x560e934dbc10 0x560e934de000 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=45eef5027e74688e server_cookie=ae51d5ca4410ea63 in_seq=0 out_seq=0 2024-06-10T01:07:31.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.919+0000 7f6e33fff700 1 -- 172.21.15.7:0/835506890 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560e93420d50 con 0x560e934dbc10 2024-06-10T01:07:31.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.919+0000 7f6e33fff700 10 client.25421 handle_command_reply: tid=0 2024-06-10T01:07:31.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.919+0000 7f6e65aabb80 1 client.25421 shutdown 2024-06-10T01:07:31.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.920+0000 7f6e65aabb80 1 -- 172.21.15.7:0/835506890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e934c27a0 msgr2=0x560e934ce610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:31.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.920+0000 7f6e65aabb80 1 --2- 172.21.15.7:0/835506890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e934c27a0 0x560e934ce610 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c0a5560 tx=0x7f6e3c09c220 comp rx=0 tx=0).stop 2024-06-10T01:07:31.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.920+0000 7f6e65aabb80 1 -- 172.21.15.7:0/835506890 shutdown_connections 2024-06-10T01:07:31.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.920+0000 7f6e65aabb80 1 --2- 172.21.15.7:0/835506890 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x560e934dbc10 0x560e934de000 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.920+0000 7f6e65aabb80 1 --2- 172.21.15.7:0/835506890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e934c8880 0x560e934cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.920+0000 7f6e65aabb80 1 --2- 172.21.15.7:0/835506890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e934c27a0 0x560e934ce610 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.920+0000 7f6e65aabb80 1 --2- 172.21.15.7:0/835506890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e934c7f60 0x560e934cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.920+0000 7f6e33fff700 0 client.25421 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:07:31.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.920+0000 7f6e65aabb80 1 -- 172.21.15.7:0/835506890 >> 172.21.15.7:0/835506890 conn(0x560e933ef080 msgr2=0x560e934bc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:31.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.920+0000 7f6e65aabb80 1 -- 172.21.15.7:0/835506890 shutdown_connections 2024-06-10T01:07:31.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.920+0000 7f6e65aabb80 1 -- 172.21.15.7:0/835506890 wait complete. 2024-06-10T01:07:31.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.920+0000 7f6e65aabb80 20 client.25421 trim_cache size 0 max 16384 2024-06-10T01:07:31.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.935+0000 7f6e65aabb80 1 Processor -- start 2024-06-10T01:07:31.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.935+0000 7f6e65aabb80 1 -- start start 2024-06-10T01:07:31.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.935+0000 7f6e65aabb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e934c27a0 0x560e934c2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:31.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.935+0000 7f6e65aabb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e934c7f60 0x560e934ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:31.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.935+0000 7f6e65aabb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e934c8880 0x560e934bce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:31.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.935+0000 7f6e65aabb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6e541b09e0 con 0x560e934c27a0 2024-06-10T01:07:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.935+0000 7f6e65aabb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6e541ae270 con 0x560e934c7f60 2024-06-10T01:07:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.936+0000 7f6e65aabb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6e541ac0c0 con 0x560e934c8880 2024-06-10T01:07:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.936+0000 7f6e58fc4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e934c27a0 0x560e934c2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.936+0000 7f6e58fc4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e934c27a0 0x560e934c2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34948/0 (socket says 172.21.15.7:34948) 2024-06-10T01:07:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.936+0000 7f6e58fc4700 1 -- 172.21.15.7:0/603417686 learned_addr learned my addr 172.21.15.7:0/603417686 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.936+0000 7f6e53fff700 1 --2- 172.21.15.7:0/603417686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e934c7f60 0x560e934ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.936+0000 7f6e58fc4700 1 -- 172.21.15.7:0/603417686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e934c8880 msgr2=0x560e934bce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.936+0000 7f6e58fc4700 1 --2- 172.21.15.7:0/603417686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e934c8880 0x560e934bce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.936+0000 7f6e58fc4700 1 -- 172.21.15.7:0/603417686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e934c7f60 msgr2=0x560e934ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.936+0000 7f6e58fc4700 1 --2- 172.21.15.7:0/603417686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e934c7f60 0x560e934ce150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.936+0000 7f6e58fc4700 1 -- 172.21.15.7:0/603417686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e934c9e80 con 0x560e934c27a0 2024-06-10T01:07:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.936+0000 7f6e58fc4700 1 --2- 172.21.15.7:0/603417686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e934c27a0 0x560e934c2260 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009fd0 tx=0x7f6e3c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a525dee6217805 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.936+0000 7f6e4a7fc700 1 -- 172.21.15.7:0/603417686 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c0b4050 con 0x560e934c27a0 2024-06-10T01:07:31.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.936+0000 7f6e4a7fc700 1 -- 172.21.15.7:0/603417686 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e3c09cec0 con 0x560e934c27a0 2024-06-10T01:07:31.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.936+0000 7f6e4a7fc700 1 -- 172.21.15.7:0/603417686 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c0a4f10 con 0x560e934c27a0 2024-06-10T01:07:31.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.937+0000 7f6e65aabb80 1 -- 172.21.15.7:0/603417686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e934c27a0 msgr2=0x560e934c2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:31.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.937+0000 7f6e65aabb80 1 --2- 172.21.15.7:0/603417686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e934c27a0 0x560e934c2260 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009fd0 tx=0x7f6e3c009310 comp rx=0 tx=0).stop 2024-06-10T01:07:31.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.937+0000 7f6e65aabb80 1 -- 172.21.15.7:0/603417686 shutdown_connections 2024-06-10T01:07:31.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.937+0000 7f6e65aabb80 1 --2- 172.21.15.7:0/603417686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e934c8880 0x560e934bce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.937+0000 7f6e65aabb80 1 --2- 172.21.15.7:0/603417686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e934c27a0 0x560e934c2260 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.937+0000 7f6e65aabb80 1 --2- 172.21.15.7:0/603417686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e934c7f60 0x560e934ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.937+0000 7f6e65aabb80 1 -- 172.21.15.7:0/603417686 >> 172.21.15.7:0/603417686 conn(0x560e933ef080 msgr2=0x560e934c72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:31.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.937+0000 7f6e65aabb80 1 -- 172.21.15.7:0/603417686 shutdown_connections 2024-06-10T01:07:31.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.937+0000 7f6e65aabb80 1 -- 172.21.15.7:0/603417686 wait complete. 2024-06-10T01:07:31.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.937+0000 7f6e65aabb80 1 -- start start 2024-06-10T01:07:31.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.937+0000 7f6e65aabb80 1 Processor -- start 2024-06-10T01:07:31.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.938+0000 7f6e65aabb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e934c27a0 0x560e934d4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:31.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.938+0000 7f6e65aabb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e934c7f60 0x560e934e2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:31.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.938+0000 7f6e65aabb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e934c8880 0x560e934e34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:31.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.938+0000 7f6e65aabb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6e541ac730 con 0x560e934c7f60 2024-06-10T01:07:31.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.938+0000 7f6e65aabb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6e541adba0 con 0x560e934c8880 2024-06-10T01:07:31.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.938+0000 7f6e65aabb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6e541ace00 con 0x560e934c27a0 2024-06-10T01:07:31.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.938+0000 7f6e65aabb80 10 client.0 _ll_register_callbacks cb 0x560e933a95b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:31.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.938+0000 7f6e53fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e934c7f60 0x560e934e2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:31.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.938+0000 7f6e53fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e934c7f60 0x560e934e2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34964/0 (socket says 172.21.15.7:34964) 2024-06-10T01:07:31.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.938+0000 7f6e53fff700 1 -- 172.21.15.7:0/1056901621 learned_addr learned my addr 172.21.15.7:0/1056901621 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:31.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.938+0000 7f6e597c5700 1 --2- 172.21.15.7:0/1056901621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e934c8880 0x560e934e34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:31.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.938+0000 7f6e58fc4700 1 --2- 172.21.15.7:0/1056901621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e934c27a0 0x560e934d4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:31.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.938+0000 7f6e53fff700 1 -- 172.21.15.7:0/1056901621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e934c27a0 msgr2=0x560e934d4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:31.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.938+0000 7f6e53fff700 1 --2- 172.21.15.7:0/1056901621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e934c27a0 0x560e934d4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.938+0000 7f6e53fff700 1 -- 172.21.15.7:0/1056901621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e934c8880 msgr2=0x560e934e34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:31.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.938+0000 7f6e53fff700 1 --2- 172.21.15.7:0/1056901621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e934c8880 0x560e934e34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.938+0000 7f6e53fff700 1 -- 172.21.15.7:0/1056901621 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e934e3ce0 con 0x560e934c7f60 2024-06-10T01:07:31.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.938+0000 7f6e53fff700 1 --2- 172.21.15.7:0/1056901621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e934c7f60 0x560e934e2f60 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6e44007fd0 tx=0x7f6e44004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac044d5becedbf5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:31.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.938+0000 7f6e33fff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:07:31.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.939+0000 7f6e33fff700 1 -- 172.21.15.7:0/1056901621 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e44030060 con 0x560e934c7f60 2024-06-10T01:07:31.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.939+0000 7f6e33fff700 1 -- 172.21.15.7:0/1056901621 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f6e44010040 con 0x560e934c7f60 2024-06-10T01:07:31.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.939+0000 7f6e65aabb80 1 -- 172.21.15.7:0/1056901621 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e3c09cec0 con 0x560e934c7f60 2024-06-10T01:07:31.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.939+0000 7f6e33fff700 1 -- 172.21.15.7:0/1056901621 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6e44037050 con 0x560e934c7f60 2024-06-10T01:07:31.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.939+0000 7f6e65aabb80 10 client.15837 fetch_fsmap learned FSMap version 27 2024-06-10T01:07:31.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.939+0000 7f6e65aabb80 1 -- 172.21.15.7:0/1056901621 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560e93546650 con 0x560e934c7f60 2024-06-10T01:07:31.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.939+0000 7f6e33fff700 1 -- 172.21.15.7:0/1056901621 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e3c09cec0 con 0x560e934c7f60 2024-06-10T01:07:31.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.939+0000 7f6e33fff700 1 -- 172.21.15.7:0/1056901621 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6e44021040 con 0x560e934c7f60 2024-06-10T01:07:31.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.939+0000 7f6e65aabb80 10 client.15837 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:07:31.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.939+0000 7f6e65aabb80 10 client.15837 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:07:31.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.939+0000 7f6e65aabb80 20 client.15837 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:31.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.939+0000 7f6e65aabb80 1 --2- 172.21.15.7:0/1056901621 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x560e93546ce0 0x560e93549100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:31.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.939+0000 7f6e65aabb80 4 client.15837 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:07:31.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.939+0000 7f6e65aabb80 1 -- 172.21.15.7:0/1056901621 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560e9341b2d0 con 0x560e93546ce0 2024-06-10T01:07:31.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.940+0000 7f6e58fc4700 1 --2- 172.21.15.7:0/1056901621 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x560e93546ce0 0x560e93549100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:31.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.940+0000 7f6e58fc4700 1 --2- 172.21.15.7:0/1056901621 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x560e93546ce0 0x560e93549100 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=34c12a31186dfe93 server_cookie=58fe285bc820207a in_seq=0 out_seq=0 2024-06-10T01:07:31.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.940+0000 7f6e33fff700 10 client.15837 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:07:31.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.940+0000 7f6e33fff700 1 -- 172.21.15.7:0/1056901621 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560e9341b2d0 con 0x560e93546ce0 2024-06-10T01:07:31.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.940+0000 7f6e33fff700 10 client.15837 handle_command_reply: tid=0 2024-06-10T01:07:31.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.940+0000 7f6e65aabb80 1 client.15837 shutdown 2024-06-10T01:07:31.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.940+0000 7f6e65aabb80 1 -- 172.21.15.7:0/1056901621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e934c7f60 msgr2=0x560e934e2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:31.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.940+0000 7f6e65aabb80 1 --2- 172.21.15.7:0/1056901621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e934c7f60 0x560e934e2f60 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6e44007fd0 tx=0x7f6e44004b40 comp rx=0 tx=0).stop 2024-06-10T01:07:31.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.940+0000 7f6e65aabb80 1 -- 172.21.15.7:0/1056901621 shutdown_connections 2024-06-10T01:07:31.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.940+0000 7f6e65aabb80 1 --2- 172.21.15.7:0/1056901621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e934c8880 0x560e934e34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.940+0000 7f6e65aabb80 1 --2- 172.21.15.7:0/1056901621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e934c27a0 0x560e934d4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.940+0000 7f6e65aabb80 1 --2- 172.21.15.7:0/1056901621 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x560e93546ce0 0x560e93549100 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.940+0000 7f6e65aabb80 1 --2- 172.21.15.7:0/1056901621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e934c7f60 0x560e934e2f60 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.940+0000 7f6e65aabb80 1 -- 172.21.15.7:0/1056901621 >> 172.21.15.7:0/1056901621 conn(0x560e933ef080 msgr2=0x560e934cb170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:31.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.941+0000 7f6e33fff700 0 client.15837 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:07:31.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.941+0000 7f6e65aabb80 1 -- 172.21.15.7:0/1056901621 shutdown_connections 2024-06-10T01:07:31.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.941+0000 7f6e65aabb80 1 -- 172.21.15.7:0/1056901621 wait complete. 2024-06-10T01:07:31.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.941+0000 7f6e65aabb80 20 client.15837 trim_cache size 0 max 16384 2024-06-10T01:07:31.942 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:07:31.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.941+0000 7f6e5b228700 1 -- 172.21.15.7:0/422473899 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6e400a2c20 msgr2=0x7f6e400a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:31.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.941+0000 7f6e5b228700 1 --2- 172.21.15.7:0/422473899 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6e400a2c20 0x7f6e400a50e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6e54107a80 tx=0x7f6e440079e0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.941+0000 7f6e5b228700 1 -- 172.21.15.7:0/422473899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e541015d0 msgr2=0x7f6e5419e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:31.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.941+0000 7f6e5b228700 1 --2- 172.21.15.7:0/422473899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e541015d0 0x7f6e5419e870 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c006010 tx=0x7f6e3c0038b0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.941+0000 7f6e5b228700 1 -- 172.21.15.7:0/422473899 shutdown_connections 2024-06-10T01:07:31.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.941+0000 7f6e5b228700 1 --2- 172.21.15.7:0/422473899 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6e400a2c20 0x7f6e400a50e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.941+0000 7f6e5b228700 1 --2- 172.21.15.7:0/422473899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e541066b0 0x7f6e54198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.941+0000 7f6e5b228700 1 --2- 172.21.15.7:0/422473899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e541015d0 0x7f6e5419e870 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.941+0000 7f6e5b228700 1 --2- 172.21.15.7:0/422473899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e54101f80 0x7f6e5419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:31.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.942+0000 7f6e5b228700 1 -- 172.21.15.7:0/422473899 >> 172.21.15.7:0/422473899 conn(0x7f6e54078400 msgr2=0x7f6e540ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:31.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.942+0000 7f6e5b228700 1 -- 172.21.15.7:0/422473899 shutdown_connections 2024-06-10T01:07:31.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:31.942+0000 7f6e5b228700 1 -- 172.21.15.7:0/422473899 wait complete. 2024-06-10T01:07:32.708 INFO:tasks.workunit.client.0.smithi007.stdout: 1 28947 13.38 MB/sec warmup 13 sec latency 3163.209 ms 2024-06-10T01:07:32.779 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:07:32.954 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:32 smithi175 ceph-mon[36636]: pgmap v477: 129 pgs: 129 active+clean; 12 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.2 MiB/s wr, 265 op/s 2024-06-10T01:07:32.955 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:32 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:07:33.014 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:32 smithi007 ceph-mon[32398]: pgmap v477: 129 pgs: 129 active+clean; 12 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.2 MiB/s wr, 265 op/s 2024-06-10T01:07:33.014 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:32 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:07:33.045 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:33.080 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:32 smithi078 ceph-mon[36683]: pgmap v477: 129 pgs: 129 active+clean; 12 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.2 MiB/s wr, 265 op/s 2024-06-10T01:07:33.080 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:32 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:07:33.708 INFO:tasks.workunit.client.0.smithi007.stdout: 1 28947 12.43 MB/sec warmup 14 sec latency 4163.248 ms 2024-06-10T01:07:33.954 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:33 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:07:33.954 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:33 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:07:33.954 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:33 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:07:34.014 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:33 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:07:34.014 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:33 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:07:34.014 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:33 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:07:34.080 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:33 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:07:34.081 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:33 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:07:34.081 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:33 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:07:34.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.709+0000 7fea6b940700 1 -- 172.21.15.7:0/2845009182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea6406b620 msgr2=0x7fea6410f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:34.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.709+0000 7fea6b940700 1 --2- 172.21.15.7:0/2845009182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea6406b620 0x7fea6410f350 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fea5c00b0d0 tx=0x7fea5c00b3e0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.709+0000 7fea6b940700 1 -- 172.21.15.7:0/2845009182 shutdown_connections 2024-06-10T01:07:34.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.709+0000 7fea6b940700 1 --2- 172.21.15.7:0/2845009182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea6406b620 0x7fea6410f350 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.709+0000 7fea6b940700 1 --2- 172.21.15.7:0/2845009182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea64108580 0x7fea64108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.709+0000 7fea6b940700 1 --2- 172.21.15.7:0/2845009182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea6406ac60 0x7fea6406b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.709+0000 7fea6b940700 1 -- 172.21.15.7:0/2845009182 >> 172.21.15.7:0/2845009182 conn(0x7fea64078430 msgr2=0x7fea64078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:34.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.709+0000 7fea6b940700 1 -- 172.21.15.7:0/2845009182 shutdown_connections 2024-06-10T01:07:34.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.709+0000 7fea6b940700 1 -- 172.21.15.7:0/2845009182 wait complete. 2024-06-10T01:07:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.710+0000 7fea6b940700 1 Processor -- start 2024-06-10T01:07:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.710+0000 7fea6b940700 1 -- start start 2024-06-10T01:07:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.710+0000 7fea6b940700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea6406ac60 0x7fea6419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.710+0000 7fea6b940700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea6406b620 0x7fea6419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.710+0000 7fea6b940700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea64108580 0x7fea641989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.710+0000 7fea6b940700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fea641127c0 con 0x7fea6406b620 2024-06-10T01:07:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.710+0000 7fea6b940700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fea64112640 con 0x7fea64108580 2024-06-10T01:07:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.710+0000 7fea6b940700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fea64112940 con 0x7fea6406ac60 2024-06-10T01:07:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.710+0000 7fea68edb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea6406b620 0x7fea6419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.710+0000 7fea68edb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea6406b620 0x7fea6419ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34972/0 (socket says 172.21.15.7:34972) 2024-06-10T01:07:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.710+0000 7fea68edb700 1 -- 172.21.15.7:0/3890405894 learned_addr learned my addr 172.21.15.7:0/3890405894 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.710+0000 7fea69edd700 1 --2- 172.21.15.7:0/3890405894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea64108580 0x7fea641989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.711+0000 7fea696dc700 1 --2- 172.21.15.7:0/3890405894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea6406ac60 0x7fea6419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.711+0000 7fea68edb700 1 -- 172.21.15.7:0/3890405894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea6406ac60 msgr2=0x7fea6419e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.711+0000 7fea68edb700 1 --2- 172.21.15.7:0/3890405894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea6406ac60 0x7fea6419e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.711+0000 7fea68edb700 1 -- 172.21.15.7:0/3890405894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea64108580 msgr2=0x7fea641989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.711+0000 7fea68edb700 1 --2- 172.21.15.7:0/3890405894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea64108580 0x7fea641989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.711+0000 7fea68edb700 1 -- 172.21.15.7:0/3890405894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea64199120 con 0x7fea6406b620 2024-06-10T01:07:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.711+0000 7fea696dc700 1 --2- 172.21.15.7:0/3890405894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea6406ac60 0x7fea6419e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.711+0000 7fea68edb700 1 --2- 172.21.15.7:0/3890405894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea6406b620 0x7fea6419ecf0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fea5400d980 tx=0x7fea5400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.711+0000 7fea5a7fc700 1 -- 172.21.15.7:0/3890405894 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea54016070 con 0x7fea6406b620 2024-06-10T01:07:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.711+0000 7fea5a7fc700 1 -- 172.21.15.7:0/3890405894 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea54004d10 con 0x7fea6406b620 2024-06-10T01:07:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.711+0000 7fea6b940700 1 -- 172.21.15.7:0/3890405894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea64199410 con 0x7fea6406b620 2024-06-10T01:07:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.711+0000 7fea5a7fc700 1 -- 172.21.15.7:0/3890405894 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea540055f0 con 0x7fea6406b620 2024-06-10T01:07:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.711+0000 7fea6b940700 1 -- 172.21.15.7:0/3890405894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea641a5680 con 0x7fea6406b620 2024-06-10T01:07:34.710 INFO:tasks.workunit.client.0.smithi007.stdout: 1 28947 11.60 MB/sec warmup 15 sec latency 5163.291 ms 2024-06-10T01:07:34.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.713+0000 7fea5a7fc700 1 -- 172.21.15.7:0/3890405894 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fea54004750 con 0x7fea6406b620 2024-06-10T01:07:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.714+0000 7fea5a7fc700 1 --2- 172.21.15.7:0/3890405894 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fea500a2ca0 0x7fea500a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.714+0000 7fea696dc700 1 --2- 172.21.15.7:0/3890405894 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fea500a2ca0 0x7fea500a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.714+0000 7fea5a7fc700 1 -- 172.21.15.7:0/3890405894 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fea540cf260 con 0x7fea6406b620 2024-06-10T01:07:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.715+0000 7fea696dc700 1 --2- 172.21.15.7:0/3890405894 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fea500a2ca0 0x7fea500a5160 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fea60005e10 tx=0x7fea60005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.715+0000 7fea761c3b80 1 Processor -- start 2024-06-10T01:07:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.715+0000 7fea761c3b80 1 -- start start 2024-06-10T01:07:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.715+0000 7fea761c3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a84630af60 0x55a84630b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.715+0000 7fea761c3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a84630b880 0x55a846305260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.715+0000 7fea761c3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a8463057a0 0x55a846305bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.715+0000 7fea761c3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a8461f7f60 con 0x55a8463057a0 2024-06-10T01:07:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.715+0000 7fea761c3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a846256850 con 0x55a84630b880 2024-06-10T01:07:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.715+0000 7fea761c3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a8461be460 con 0x55a84630af60 2024-06-10T01:07:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.715+0000 7fea69edd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a8463057a0 0x55a846305bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.715+0000 7fea69edd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a8463057a0 0x55a846305bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34978/0 (socket says 172.21.15.7:34978) 2024-06-10T01:07:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.715+0000 7fea69edd700 1 -- 172.21.15.7:0/595659366 learned_addr learned my addr 172.21.15.7:0/595659366 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.715+0000 7fea696dc700 1 --2- 172.21.15.7:0/595659366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a84630af60 0x55a84630b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.716+0000 7fea68edb700 1 --2- 172.21.15.7:0/595659366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a84630b880 0x55a846305260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.716+0000 7fea69edd700 1 -- 172.21.15.7:0/595659366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a84630af60 msgr2=0x55a84630b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.716+0000 7fea69edd700 1 --2- 172.21.15.7:0/595659366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a84630af60 0x55a84630b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.716+0000 7fea69edd700 1 -- 172.21.15.7:0/595659366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a84630b880 msgr2=0x55a846305260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.716+0000 7fea69edd700 1 --2- 172.21.15.7:0/595659366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a84630b880 0x55a846305260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.716+0000 7fea69edd700 1 -- 172.21.15.7:0/595659366 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8461a58d0 con 0x55a8463057a0 2024-06-10T01:07:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.716+0000 7fea69edd700 1 --2- 172.21.15.7:0/595659366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a8463057a0 0x55a846305bc0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009cb0 tx=0x7fea5c006210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e995a7b51a3c86d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.716+0000 7fea4f7fe700 1 -- 172.21.15.7:0/595659366 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c0092d0 con 0x55a8463057a0 2024-06-10T01:07:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.716+0000 7fea4f7fe700 1 -- 172.21.15.7:0/595659366 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea5c009470 con 0x55a8463057a0 2024-06-10T01:07:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.716+0000 7fea4f7fe700 1 -- 172.21.15.7:0/595659366 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c005040 con 0x55a8463057a0 2024-06-10T01:07:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.717+0000 7fea761c3b80 1 -- 172.21.15.7:0/595659366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a8463057a0 msgr2=0x55a846305bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.717+0000 7fea761c3b80 1 --2- 172.21.15.7:0/595659366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a8463057a0 0x55a846305bc0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009cb0 tx=0x7fea5c006210 comp rx=0 tx=0).stop 2024-06-10T01:07:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.717+0000 7fea761c3b80 1 -- 172.21.15.7:0/595659366 shutdown_connections 2024-06-10T01:07:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.717+0000 7fea761c3b80 1 --2- 172.21.15.7:0/595659366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a8463057a0 0x55a846305bc0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.717+0000 7fea761c3b80 1 --2- 172.21.15.7:0/595659366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a84630af60 0x55a84630b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.717+0000 7fea761c3b80 1 --2- 172.21.15.7:0/595659366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a84630b880 0x55a846305260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.717+0000 7fea761c3b80 1 -- 172.21.15.7:0/595659366 >> 172.21.15.7:0/595659366 conn(0x55a846232080 msgr2=0x55a846300220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.717+0000 7fea761c3b80 1 -- 172.21.15.7:0/595659366 shutdown_connections 2024-06-10T01:07:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.717+0000 7fea761c3b80 1 -- 172.21.15.7:0/595659366 wait complete. 2024-06-10T01:07:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.717+0000 7fea761c3b80 1 -- start start 2024-06-10T01:07:34.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.718+0000 7fea761c3b80 1 Processor -- start 2024-06-10T01:07:34.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.718+0000 7fea761c3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a8463057a0 0x55a8463115f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.718+0000 7fea761c3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a84630af60 0x55a84630e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.718+0000 7fea761c3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a84630b880 0x55a84630ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.718+0000 7fea761c3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a8461f6fc0 con 0x55a84630af60 2024-06-10T01:07:34.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.718+0000 7fea761c3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a846252810 con 0x55a84630b880 2024-06-10T01:07:34.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.718+0000 7fea761c3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a846312200 con 0x55a8463057a0 2024-06-10T01:07:34.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.718+0000 7fea761c3b80 10 client.0 _ll_register_callbacks cb 0x55a8461ec5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:34.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.718+0000 7fea68edb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a84630af60 0x55a84630e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.718+0000 7fea68edb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a84630af60 0x55a84630e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34980/0 (socket says 172.21.15.7:34980) 2024-06-10T01:07:34.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.718+0000 7fea68edb700 1 -- 172.21.15.7:0/3503162784 learned_addr learned my addr 172.21.15.7:0/3503162784 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:34.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.718+0000 7fea69edd700 1 --2- 172.21.15.7:0/3503162784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a84630b880 0x55a84630ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.718+0000 7fea696dc700 1 --2- 172.21.15.7:0/3503162784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a8463057a0 0x55a8463115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.718+0000 7fea68edb700 1 -- 172.21.15.7:0/3503162784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a8463057a0 msgr2=0x55a8463115f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:34.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.718+0000 7fea68edb700 1 --2- 172.21.15.7:0/3503162784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a8463057a0 0x55a8463115f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.718+0000 7fea68edb700 1 -- 172.21.15.7:0/3503162784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a84630b880 msgr2=0x55a84630ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:34.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.718+0000 7fea68edb700 1 --2- 172.21.15.7:0/3503162784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a84630b880 0x55a84630ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.719+0000 7fea68edb700 1 -- 172.21.15.7:0/3503162784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a84621c750 con 0x55a84630af60 2024-06-10T01:07:34.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.719+0000 7fea68edb700 1 --2- 172.21.15.7:0/3503162784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a84630af60 0x55a84630e8f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fea5400d950 tx=0x7fea5409b2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336021628adfc5f8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:34.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.719+0000 7fea4cff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:07:34.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.719+0000 7fea4cff9700 1 -- 172.21.15.7:0/3503162784 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea540b4060 con 0x55a84630af60 2024-06-10T01:07:34.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.719+0000 7fea4cff9700 1 -- 172.21.15.7:0/3503162784 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fea540999e0 con 0x55a84630af60 2024-06-10T01:07:34.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.719+0000 7fea761c3b80 1 -- 172.21.15.7:0/3503162784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea5c009470 con 0x55a84630af60 2024-06-10T01:07:34.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.719+0000 7fea4cff9700 1 -- 172.21.15.7:0/3503162784 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fea5409b6e0 con 0x55a84630af60 2024-06-10T01:07:34.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.719+0000 7fea761c3b80 10 client.15849 fetch_fsmap learned FSMap version 27 2024-06-10T01:07:34.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.719+0000 7fea761c3b80 1 -- 172.21.15.7:0/3503162784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a84631ca30 con 0x55a84630af60 2024-06-10T01:07:34.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.719+0000 7fea4cff9700 1 -- 172.21.15.7:0/3503162784 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea5c009470 con 0x55a84630af60 2024-06-10T01:07:34.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.719+0000 7fea4cff9700 1 -- 172.21.15.7:0/3503162784 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fea540a3ec0 con 0x55a84630af60 2024-06-10T01:07:34.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.719+0000 7fea761c3b80 10 client.15849 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:07:34.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.719+0000 7fea761c3b80 10 client.15849 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:07:34.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.720+0000 7fea761c3b80 20 client.15849 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:34.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.720+0000 7fea761c3b80 1 --2- 172.21.15.7:0/3503162784 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a84631efc0 0x55a8463213b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.720+0000 7fea761c3b80 4 client.15849 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:07:34.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.720+0000 7fea761c3b80 1 -- 172.21.15.7:0/3503162784 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a846263d50 con 0x55a84631efc0 2024-06-10T01:07:34.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.720+0000 7fea696dc700 1 --2- 172.21.15.7:0/3503162784 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a84631efc0 0x55a8463213b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.720+0000 7fea4cff9700 10 client.15849 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:07:34.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.720+0000 7fea696dc700 1 --2- 172.21.15.7:0/3503162784 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a84631efc0 0x55a8463213b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9a97b08b4f32d4f1 server_cookie=fab25740ba374f14 in_seq=0 out_seq=0 2024-06-10T01:07:34.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.721+0000 7fea4cff9700 1 -- 172.21.15.7:0/3503162784 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a846263d50 con 0x55a84631efc0 2024-06-10T01:07:34.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.721+0000 7fea4cff9700 10 client.15849 handle_command_reply: tid=0 2024-06-10T01:07:34.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.721+0000 7fea761c3b80 1 client.15849 shutdown 2024-06-10T01:07:34.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.721+0000 7fea761c3b80 1 -- 172.21.15.7:0/3503162784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a84630af60 msgr2=0x55a84630e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.721+0000 7fea761c3b80 1 --2- 172.21.15.7:0/3503162784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a84630af60 0x55a84630e8f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fea5400d950 tx=0x7fea5409b2f0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.722+0000 7fea761c3b80 1 -- 172.21.15.7:0/3503162784 shutdown_connections 2024-06-10T01:07:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.722+0000 7fea761c3b80 1 --2- 172.21.15.7:0/3503162784 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a84631efc0 0x55a8463213b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.722+0000 7fea761c3b80 1 --2- 172.21.15.7:0/3503162784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a84630b880 0x55a84630ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.722+0000 7fea761c3b80 1 --2- 172.21.15.7:0/3503162784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a8463057a0 0x55a8463115f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.722+0000 7fea761c3b80 1 --2- 172.21.15.7:0/3503162784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a84630af60 0x55a84630e8f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.722+0000 7fea4cff9700 0 client.15849 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:07:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.722+0000 7fea761c3b80 1 -- 172.21.15.7:0/3503162784 >> 172.21.15.7:0/3503162784 conn(0x55a846232080 msgr2=0x55a8462ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.722+0000 7fea761c3b80 1 -- 172.21.15.7:0/3503162784 shutdown_connections 2024-06-10T01:07:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.722+0000 7fea761c3b80 1 -- 172.21.15.7:0/3503162784 wait complete. 2024-06-10T01:07:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.722+0000 7fea761c3b80 20 client.15849 trim_cache size 0 max 16384 2024-06-10T01:07:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.737+0000 7fea761c3b80 1 Processor -- start 2024-06-10T01:07:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.737+0000 7fea761c3b80 1 -- start start 2024-06-10T01:07:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.737+0000 7fea761c3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a8463057a0 0x55a846305260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.737+0000 7fea761c3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a84630af60 0x55a846310830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.737+0000 7fea761c3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a84630b880 0x55a8463178a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.737+0000 7fea761c3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fea641b0a50 con 0x55a84630af60 2024-06-10T01:07:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.737+0000 7fea761c3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fea641ae2e0 con 0x55a8463057a0 2024-06-10T01:07:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.737+0000 7fea761c3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fea641ac130 con 0x55a84630b880 2024-06-10T01:07:34.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.737+0000 7fea68edb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a84630af60 0x55a846310830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.738+0000 7fea68edb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a84630af60 0x55a846310830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34990/0 (socket says 172.21.15.7:34990) 2024-06-10T01:07:34.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.738+0000 7fea68edb700 1 -- 172.21.15.7:0/3814712801 learned_addr learned my addr 172.21.15.7:0/3814712801 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:34.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.738+0000 7fea68edb700 1 -- 172.21.15.7:0/3814712801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a84630b880 msgr2=0x55a8463178a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:07:34.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.738+0000 7fea68edb700 1 --2- 172.21.15.7:0/3814712801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a84630b880 0x55a8463178a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.738+0000 7fea68edb700 1 -- 172.21.15.7:0/3814712801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a8463057a0 msgr2=0x55a846305260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:34.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.738+0000 7fea68edb700 1 --2- 172.21.15.7:0/3814712801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a8463057a0 0x55a846305260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.738+0000 7fea68edb700 1 -- 172.21.15.7:0/3814712801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a84630ce60 con 0x55a84630af60 2024-06-10T01:07:34.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.738+0000 7fea68edb700 1 --2- 172.21.15.7:0/3814712801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a84630af60 0x55a846310830 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fea5400b540 tx=0x7fea54004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c86fa4a01186d456 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:34.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.738+0000 7fea4effd700 1 -- 172.21.15.7:0/3814712801 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea540a4350 con 0x55a84630af60 2024-06-10T01:07:34.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.738+0000 7fea4effd700 1 -- 172.21.15.7:0/3814712801 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea540999e0 con 0x55a84630af60 2024-06-10T01:07:34.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.738+0000 7fea4effd700 1 -- 172.21.15.7:0/3814712801 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5409ab30 con 0x55a84630af60 2024-06-10T01:07:34.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.739+0000 7fea761c3b80 1 -- 172.21.15.7:0/3814712801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a84630af60 msgr2=0x55a846310830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:34.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.739+0000 7fea761c3b80 1 --2- 172.21.15.7:0/3814712801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a84630af60 0x55a846310830 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fea5400b540 tx=0x7fea54004640 comp rx=0 tx=0).stop 2024-06-10T01:07:34.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.739+0000 7fea761c3b80 1 -- 172.21.15.7:0/3814712801 shutdown_connections 2024-06-10T01:07:34.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.739+0000 7fea761c3b80 1 --2- 172.21.15.7:0/3814712801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a84630b880 0x55a8463178a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.739+0000 7fea761c3b80 1 --2- 172.21.15.7:0/3814712801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a8463057a0 0x55a846305260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.739+0000 7fea761c3b80 1 --2- 172.21.15.7:0/3814712801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a84630af60 0x55a846310830 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.739+0000 7fea761c3b80 1 -- 172.21.15.7:0/3814712801 >> 172.21.15.7:0/3814712801 conn(0x55a846232080 msgr2=0x55a846310120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:34.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.739+0000 7fea761c3b80 1 -- 172.21.15.7:0/3814712801 shutdown_connections 2024-06-10T01:07:34.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.739+0000 7fea761c3b80 1 -- 172.21.15.7:0/3814712801 wait complete. 2024-06-10T01:07:34.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.739+0000 7fea761c3b80 1 -- start start 2024-06-10T01:07:34.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.739+0000 7fea761c3b80 1 Processor -- start 2024-06-10T01:07:34.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.740+0000 7fea761c3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a8463057a0 0x55a846325e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.740+0000 7fea761c3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a84630af60 0x55a846326350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.740+0000 7fea761c3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a84630b880 0x55a8463268b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.740+0000 7fea761c3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fea641ac7a0 con 0x55a84630b880 2024-06-10T01:07:34.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.740+0000 7fea761c3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fea641adc10 con 0x55a84630af60 2024-06-10T01:07:34.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.740+0000 7fea761c3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fea641ace70 con 0x55a8463057a0 2024-06-10T01:07:34.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.740+0000 7fea761c3b80 10 client.0 _ll_register_callbacks cb 0x55a8461ec5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:34.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.740+0000 7fea69edd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a84630b880 0x55a8463268b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.740+0000 7fea69edd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a84630b880 0x55a8463268b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34994/0 (socket says 172.21.15.7:34994) 2024-06-10T01:07:34.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.740+0000 7fea69edd700 1 -- 172.21.15.7:0/77910915 learned_addr learned my addr 172.21.15.7:0/77910915 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:34.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.740+0000 7fea68edb700 1 --2- 172.21.15.7:0/77910915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a84630af60 0x55a846326350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.740+0000 7fea69edd700 1 -- 172.21.15.7:0/77910915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a8463057a0 msgr2=0x55a846325e10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:07:34.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.740+0000 7fea69edd700 1 --2- 172.21.15.7:0/77910915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a8463057a0 0x55a846325e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.740+0000 7fea69edd700 1 -- 172.21.15.7:0/77910915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a84630af60 msgr2=0x55a846326350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:34.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.740+0000 7fea69edd700 1 --2- 172.21.15.7:0/77910915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a84630af60 0x55a846326350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.740+0000 7fea69edd700 1 -- 172.21.15.7:0/77910915 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8463270f0 con 0x55a84630b880 2024-06-10T01:07:34.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.741+0000 7fea69edd700 1 --2- 172.21.15.7:0/77910915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a84630b880 0x55a8463268b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fea5c023000 tx=0x7fea5c0060d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e27bccb4d197861 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:34.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.741+0000 7fea4cff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:07:34.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.741+0000 7fea4cff9700 1 -- 172.21.15.7:0/77910915 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c0092e0 con 0x55a84630b880 2024-06-10T01:07:34.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.741+0000 7fea761c3b80 1 -- 172.21.15.7:0/77910915 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea540999e0 con 0x55a84630b880 2024-06-10T01:07:34.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.741+0000 7fea4cff9700 1 -- 172.21.15.7:0/77910915 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7fea5c003a00 con 0x55a84630b880 2024-06-10T01:07:34.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.741+0000 7fea4cff9700 1 -- 172.21.15.7:0/77910915 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fea5c037050 con 0x55a84630b880 2024-06-10T01:07:34.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.741+0000 7fea761c3b80 10 client.15855 fetch_fsmap learned FSMap version 27 2024-06-10T01:07:34.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.741+0000 7fea761c3b80 1 -- 172.21.15.7:0/77910915 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a84638e5b0 con 0x55a84630b880 2024-06-10T01:07:34.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.741+0000 7fea4cff9700 1 -- 172.21.15.7:0/77910915 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea540999e0 con 0x55a84630b880 2024-06-10T01:07:34.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.741+0000 7fea4cff9700 1 -- 172.21.15.7:0/77910915 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fea5c011070 con 0x55a84630b880 2024-06-10T01:07:34.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.741+0000 7fea761c3b80 10 client.15855 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:07:34.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.741+0000 7fea761c3b80 10 client.15855 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:07:34.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.742+0000 7fea761c3b80 20 client.15855 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:34.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.742+0000 7fea761c3b80 1 --2- 172.21.15.7:0/77910915 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a84638e7e0 0x55a846390bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.742+0000 7fea761c3b80 4 client.15855 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:07:34.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.742+0000 7fea761c3b80 1 -- 172.21.15.7:0/77910915 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a84625e2d0 con 0x55a84638e7e0 2024-06-10T01:07:34.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.743+0000 7fea696dc700 1 --2- 172.21.15.7:0/77910915 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a84638e7e0 0x55a846390bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.743+0000 7fea696dc700 1 --2- 172.21.15.7:0/77910915 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a84638e7e0 0x55a846390bd0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3d57437562c443 server_cookie=e4a257b329d1af66 in_seq=0 out_seq=0 2024-06-10T01:07:34.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.743+0000 7fea4cff9700 10 client.15855 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:07:34.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.744+0000 7fea4cff9700 1 -- 172.21.15.7:0/77910915 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a84625e2d0 con 0x55a84638e7e0 2024-06-10T01:07:34.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.744+0000 7fea4cff9700 10 client.15855 handle_command_reply: tid=0 2024-06-10T01:07:34.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.744+0000 7fea761c3b80 1 client.15855 shutdown 2024-06-10T01:07:34.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.744+0000 7fea761c3b80 1 -- 172.21.15.7:0/77910915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a84630b880 msgr2=0x55a8463268b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:34.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.744+0000 7fea761c3b80 1 --2- 172.21.15.7:0/77910915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a84630b880 0x55a8463268b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fea5c023000 tx=0x7fea5c0060d0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.744+0000 7fea761c3b80 1 -- 172.21.15.7:0/77910915 shutdown_connections 2024-06-10T01:07:34.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.744+0000 7fea761c3b80 1 --2- 172.21.15.7:0/77910915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a84630b880 0x55a8463268b0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.744+0000 7fea761c3b80 1 --2- 172.21.15.7:0/77910915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a8463057a0 0x55a846325e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.744+0000 7fea761c3b80 1 --2- 172.21.15.7:0/77910915 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a84638e7e0 0x55a846390bd0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.744+0000 7fea4cff9700 0 client.15855 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:07:34.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.744+0000 7fea761c3b80 1 --2- 172.21.15.7:0/77910915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a84630af60 0x55a846326350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.744+0000 7fea761c3b80 1 -- 172.21.15.7:0/77910915 >> 172.21.15.7:0/77910915 conn(0x55a846232080 msgr2=0x55a846309a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:34.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.744+0000 7fea761c3b80 1 -- 172.21.15.7:0/77910915 shutdown_connections 2024-06-10T01:07:34.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.744+0000 7fea761c3b80 1 -- 172.21.15.7:0/77910915 wait complete. 2024-06-10T01:07:34.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.744+0000 7fea761c3b80 20 client.15855 trim_cache size 0 max 16384 2024-06-10T01:07:34.744 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:07:34.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.745+0000 7fea6b940700 1 -- 172.21.15.7:0/3890405894 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fea500a2ca0 msgr2=0x7fea500a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:34.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.745+0000 7fea6b940700 1 --2- 172.21.15.7:0/3890405894 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fea500a2ca0 0x7fea500a5160 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fea60005e10 tx=0x7fea60005d60 comp rx=0 tx=0).stop 2024-06-10T01:07:34.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.745+0000 7fea6b940700 1 -- 172.21.15.7:0/3890405894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea6406b620 msgr2=0x7fea6419ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:34.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.745+0000 7fea6b940700 1 --2- 172.21.15.7:0/3890405894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea6406b620 0x7fea6419ecf0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fea5400d980 tx=0x7fea5400dc90 comp rx=0 tx=0).stop 2024-06-10T01:07:34.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.745+0000 7fea6b940700 1 -- 172.21.15.7:0/3890405894 shutdown_connections 2024-06-10T01:07:34.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.745+0000 7fea6b940700 1 --2- 172.21.15.7:0/3890405894 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fea500a2ca0 0x7fea500a5160 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.745+0000 7fea6b940700 1 --2- 172.21.15.7:0/3890405894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea64108580 0x7fea641989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.745+0000 7fea6b940700 1 --2- 172.21.15.7:0/3890405894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea6406ac60 0x7fea6419e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.745+0000 7fea6b940700 1 --2- 172.21.15.7:0/3890405894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea6406b620 0x7fea6419ecf0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.745+0000 7fea6b940700 1 -- 172.21.15.7:0/3890405894 >> 172.21.15.7:0/3890405894 conn(0x7fea64078430 msgr2=0x7fea640ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:34.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.745+0000 7fea6b940700 1 -- 172.21.15.7:0/3890405894 shutdown_connections 2024-06-10T01:07:34.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:34.745+0000 7fea6b940700 1 -- 172.21.15.7:0/3890405894 wait complete. 2024-06-10T01:07:35.014 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:34 smithi007 ceph-mon[32398]: pgmap v478: 129 pgs: 129 active+clean; 15 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.4 MiB/s wr, 278 op/s 2024-06-10T01:07:35.204 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:34 smithi175 ceph-mon[36636]: pgmap v478: 129 pgs: 129 active+clean; 15 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.4 MiB/s wr, 278 op/s 2024-06-10T01:07:35.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:34 smithi078 ceph-mon[36683]: pgmap v478: 129 pgs: 129 active+clean; 15 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.4 MiB/s wr, 278 op/s 2024-06-10T01:07:35.385 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:07:35.641 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:35.708 INFO:tasks.workunit.client.0.smithi007.stdout: 1 28947 10.87 MB/sec warmup 16 sec latency 6163.328 ms 2024-06-10T01:07:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.953+0000 7f079aa09640 1 -- 172.21.15.7:0/102281584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07941035c0 msgr2=0x7f0794109e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.953+0000 7f079aa09640 1 --2- 172.21.15.7:0/102281584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07941035c0 0x7f0794109e50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f078c009ca0 tx=0x7f078c02a8c0 comp rx=0 tx=0).stop 2024-06-10T01:07:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.953+0000 7f079aa09640 1 -- 172.21.15.7:0/102281584 shutdown_connections 2024-06-10T01:07:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.953+0000 7f079aa09640 1 --2- 172.21.15.7:0/102281584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07941035c0 0x7f0794109e50 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.953+0000 7f079aa09640 1 --2- 172.21.15.7:0/102281584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0794102c00 0x7f0794103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.953+0000 7f079aa09640 1 --2- 172.21.15.7:0/102281584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0794101a10 0x7f0794101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.953+0000 7f079aa09640 1 -- 172.21.15.7:0/102281584 >> 172.21.15.7:0/102281584 conn(0x7f07940fcf80 msgr2=0x7f07940ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.953+0000 7f079aa09640 1 -- 172.21.15.7:0/102281584 shutdown_connections 2024-06-10T01:07:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.953+0000 7f079aa09640 1 -- 172.21.15.7:0/102281584 wait complete. 2024-06-10T01:07:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.954+0000 7f079aa09640 1 Processor -- start 2024-06-10T01:07:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.954+0000 7f079aa09640 1 -- start start 2024-06-10T01:07:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.954+0000 7f079aa09640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0794101a10 0x7f0794197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.954+0000 7f079aa09640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0794102c00 0x7f0794198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.954+0000 7f079aa09640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07941035c0 0x7f079419f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.954+0000 7f079aa09640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f079410d6f0 con 0x7f0794101a10 2024-06-10T01:07:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.954+0000 7f079aa09640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f079410d570 con 0x7f0794102c00 2024-06-10T01:07:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.954+0000 7f079aa09640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f079410d870 con 0x7f07941035c0 2024-06-10T01:07:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.954+0000 7f0793fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0794101a10 0x7f0794197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.954+0000 7f0793fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0794101a10 0x7f0794197d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35012/0 (socket says 172.21.15.7:35012) 2024-06-10T01:07:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.954+0000 7f0793fff640 1 -- 172.21.15.7:0/3552934067 learned_addr learned my addr 172.21.15.7:0/3552934067 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.955+0000 7f0798fa6640 1 --2- 172.21.15.7:0/3552934067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07941035c0 0x7f079419f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.955+0000 7f0793fff640 1 -- 172.21.15.7:0/3552934067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07941035c0 msgr2=0x7f079419f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.955+0000 7f0793fff640 1 --2- 172.21.15.7:0/3552934067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07941035c0 0x7f079419f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.955+0000 7f0793fff640 1 -- 172.21.15.7:0/3552934067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0794102c00 msgr2=0x7f0794198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.955+0000 7f07937fe640 1 --2- 172.21.15.7:0/3552934067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0794102c00 0x7f0794198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.955+0000 7f0793fff640 1 --2- 172.21.15.7:0/3552934067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0794102c00 0x7f0794198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.955+0000 7f0793fff640 1 -- 172.21.15.7:0/3552934067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f079419f940 con 0x7f0794101a10 2024-06-10T01:07:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.955+0000 7f0793fff640 1 --2- 172.21.15.7:0/3552934067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0794101a10 0x7f0794197d30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f078400c970 tx=0x7f078400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.955+0000 7f07917fa640 1 -- 172.21.15.7:0/3552934067 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0784004220 con 0x7f0794101a10 2024-06-10T01:07:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.955+0000 7f079aa09640 1 -- 172.21.15.7:0/3552934067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f079419fc30 con 0x7f0794101a10 2024-06-10T01:07:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.955+0000 7f07917fa640 1 -- 172.21.15.7:0/3552934067 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07840043c0 con 0x7f0794101a10 2024-06-10T01:07:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.955+0000 7f07917fa640 1 -- 172.21.15.7:0/3552934067 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078400f450 con 0x7f0794101a10 2024-06-10T01:07:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.955+0000 7f079aa09640 1 -- 172.21.15.7:0/3552934067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07941a0180 con 0x7f0794101a10 2024-06-10T01:07:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.957+0000 7f079d7f1740 1 Processor -- start 2024-06-10T01:07:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.957+0000 7f07917fa640 1 -- 172.21.15.7:0/3552934067 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f07840079b0 con 0x7f0794101a10 2024-06-10T01:07:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.957+0000 7f079d7f1740 1 -- start start 2024-06-10T01:07:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.958+0000 7f079d7f1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638f2dcd420 0x5638f2dcd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.958+0000 7f079d7f1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638f2dcde10 0x5638f2dd2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.958+0000 7f079d7f1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f2dd2f80 0x5638f2dd5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.958+0000 7f079d7f1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5638f2d2c4f0 con 0x5638f2dd2f80 2024-06-10T01:07:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.958+0000 7f079d7f1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5638f2cca870 con 0x5638f2dcd420 2024-06-10T01:07:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.958+0000 7f079d7f1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5638f2d284b0 con 0x5638f2dcde10 2024-06-10T01:07:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.958+0000 7f07937fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f2dd2f80 0x5638f2dd5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.958+0000 7f07937fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f2dd2f80 0x5638f2dd5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35024/0 (socket says 172.21.15.7:35024) 2024-06-10T01:07:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.958+0000 7f07937fe640 1 -- 172.21.15.7:0/2681553566 learned_addr learned my addr 172.21.15.7:0/2681553566 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.958+0000 7f0798fa6640 1 --2- 172.21.15.7:0/2681553566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638f2dcd420 0x5638f2dcd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.958+0000 7f0793fff640 1 --2- 172.21.15.7:0/2681553566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638f2dcde10 0x5638f2dd2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.958+0000 7f07937fe640 1 -- 172.21.15.7:0/2681553566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638f2dcde10 msgr2=0x5638f2dd2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.958+0000 7f07937fe640 1 --2- 172.21.15.7:0/2681553566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638f2dcde10 0x5638f2dd2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.958+0000 7f07937fe640 1 -- 172.21.15.7:0/2681553566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638f2dcd420 msgr2=0x5638f2dcd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.958+0000 7f07937fe640 1 --2- 172.21.15.7:0/2681553566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638f2dcd420 0x5638f2dcd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.958+0000 7f07937fe640 1 -- 172.21.15.7:0/2681553566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638f2ce69b0 con 0x5638f2dd2f80 2024-06-10T01:07:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.958+0000 7f0798fa6640 1 --2- 172.21.15.7:0/2681553566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638f2dcd420 0x5638f2dcd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.958+0000 7f07917fa640 1 --2- 172.21.15.7:0/3552934067 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f077c0a2b80 0x7f077c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.958+0000 7f07937fe640 1 --2- 172.21.15.7:0/2681553566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f2dd2f80 0x5638f2dd5410 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f078800a920 tx=0x7f078800ade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8bf38b121640d1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.958+0000 7f0798fa6640 1 --2- 172.21.15.7:0/3552934067 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f077c0a2b80 0x7f077c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.958+0000 7f07917fa640 1 -- 172.21.15.7:0/3552934067 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f07840cf290 con 0x7f0794101a10 2024-06-10T01:07:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.958+0000 7f077a7fc640 1 -- 172.21.15.7:0/2681553566 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0788015070 con 0x5638f2dd2f80 2024-06-10T01:07:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.959+0000 7f077a7fc640 1 -- 172.21.15.7:0/2681553566 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f078800cc00 con 0x5638f2dd2f80 2024-06-10T01:07:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.959+0000 7f077a7fc640 1 -- 172.21.15.7:0/2681553566 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07880185c0 con 0x5638f2dd2f80 2024-06-10T01:07:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.959+0000 7f079d7f1740 1 -- 172.21.15.7:0/2681553566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f2dd2f80 msgr2=0x5638f2dd5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.959+0000 7f0798fa6640 1 --2- 172.21.15.7:0/3552934067 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f077c0a2b80 0x7f077c0a5040 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f078c031c10 tx=0x7f078c02f310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.959+0000 7f079d7f1740 1 --2- 172.21.15.7:0/2681553566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f2dd2f80 0x5638f2dd5410 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f078800a920 tx=0x7f078800ade0 comp rx=0 tx=0).stop 2024-06-10T01:07:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.959+0000 7f079d7f1740 1 -- 172.21.15.7:0/2681553566 shutdown_connections 2024-06-10T01:07:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.959+0000 7f079d7f1740 1 --2- 172.21.15.7:0/2681553566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f2dd2f80 0x5638f2dd5410 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.959+0000 7f079d7f1740 1 --2- 172.21.15.7:0/2681553566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638f2dcde10 0x5638f2dd2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.959+0000 7f079d7f1740 1 --2- 172.21.15.7:0/2681553566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638f2dcd420 0x5638f2dcd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.959+0000 7f079d7f1740 1 -- 172.21.15.7:0/2681553566 >> 172.21.15.7:0/2681553566 conn(0x5638f2dc4680 msgr2=0x5638f2dc4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.959+0000 7f079d7f1740 1 -- 172.21.15.7:0/2681553566 shutdown_connections 2024-06-10T01:07:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.959+0000 7f079d7f1740 1 -- 172.21.15.7:0/2681553566 wait complete. 2024-06-10T01:07:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.960+0000 7f079d7f1740 1 -- start start 2024-06-10T01:07:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.960+0000 7f079d7f1740 1 Processor -- start 2024-06-10T01:07:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.960+0000 7f079d7f1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638f2dcd420 0x5638f2dd7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.960+0000 7f079d7f1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638f2dcde10 0x5638f2dd7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.960+0000 7f079d7f1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f2dd2f80 0x5638f2ddebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.960+0000 7f079d7f1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5638f2dd5f30 con 0x5638f2dd2f80 2024-06-10T01:07:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.960+0000 7f079d7f1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5638f2cb5ae0 con 0x5638f2dcde10 2024-06-10T01:07:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.960+0000 7f079d7f1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5638f2dd60b0 con 0x5638f2dcd420 2024-06-10T01:07:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.960+0000 7f079d7f1740 10 client.0 _ll_register_callbacks cb 0x5638f2cee9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.960+0000 7f0798fa6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f2dd2f80 0x5638f2ddebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.960+0000 7f0798fa6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f2dd2f80 0x5638f2ddebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35036/0 (socket says 172.21.15.7:35036) 2024-06-10T01:07:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.960+0000 7f0798fa6640 1 -- 172.21.15.7:0/1475403662 learned_addr learned my addr 172.21.15.7:0/1475403662 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.960+0000 7f07937fe640 1 --2- 172.21.15.7:0/1475403662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638f2dcde10 0x5638f2dd7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.961+0000 7f0798fa6640 1 -- 172.21.15.7:0/1475403662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638f2dcd420 msgr2=0x5638f2dd7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.961+0000 7f0798fa6640 1 --2- 172.21.15.7:0/1475403662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638f2dcd420 0x5638f2dd7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.961+0000 7f0798fa6640 1 -- 172.21.15.7:0/1475403662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638f2dcde10 msgr2=0x5638f2dd7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.961+0000 7f0798fa6640 1 --2- 172.21.15.7:0/1475403662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638f2dcde10 0x5638f2dd7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.961+0000 7f0798fa6640 1 -- 172.21.15.7:0/1475403662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638f2a40210 con 0x5638f2dd2f80 2024-06-10T01:07:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.961+0000 7f0798fa6640 1 --2- 172.21.15.7:0/1475403662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f2dd2f80 0x5638f2ddebb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f078c009ca0 tx=0x7f078c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f33619aafd4cc3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.961+0000 7f07737fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:07:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.961+0000 7f07737fe640 1 -- 172.21.15.7:0/1475403662 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c04e070 con 0x5638f2dd2f80 2024-06-10T01:07:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.961+0000 7f07737fe640 1 -- 172.21.15.7:0/1475403662 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f078c004790 con 0x5638f2dd2f80 2024-06-10T01:07:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.961+0000 7f079d7f1740 1 -- 172.21.15.7:0/1475403662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f078800cc00 con 0x5638f2dd2f80 2024-06-10T01:07:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.961+0000 7f07737fe640 1 -- 172.21.15.7:0/1475403662 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f078c05a050 con 0x5638f2dd2f80 2024-06-10T01:07:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.961+0000 7f079d7f1740 10 client.15867 fetch_fsmap learned FSMap version 27 2024-06-10T01:07:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.961+0000 7f079d7f1740 1 -- 172.21.15.7:0/1475403662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5638f2b0c620 con 0x5638f2dd2f80 2024-06-10T01:07:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.961+0000 7f07737fe640 1 -- 172.21.15.7:0/1475403662 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f078800cc00 con 0x5638f2dd2f80 2024-06-10T01:07:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.961+0000 7f07737fe640 1 -- 172.21.15.7:0/1475403662 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f078c043040 con 0x5638f2dd2f80 2024-06-10T01:07:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.962+0000 7f079d7f1740 10 client.15867 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:07:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.962+0000 7f079d7f1740 10 client.15867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.962+0000 7f079d7f1740 20 client.15867 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.962+0000 7f079d7f1740 1 --2- 172.21.15.7:0/1475403662 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5638f2de3510 0x5638f2de59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.962+0000 7f079d7f1740 4 client.15867 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:07:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.962+0000 7f079d7f1740 1 -- 172.21.15.7:0/1475403662 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638f2d3ff10 con 0x5638f2de3510 2024-06-10T01:07:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.962+0000 7f0793fff640 1 --2- 172.21.15.7:0/1475403662 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5638f2de3510 0x5638f2de59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.962+0000 7f07737fe640 10 client.15867 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.962+0000 7f0793fff640 1 --2- 172.21.15.7:0/1475403662 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5638f2de3510 0x5638f2de59b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92ab8517aab1e148 server_cookie=d2b022820895e82e in_seq=0 out_seq=0 2024-06-10T01:07:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.963+0000 7f07737fe640 1 -- 172.21.15.7:0/1475403662 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5638f2d3ff10 con 0x5638f2de3510 2024-06-10T01:07:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.963+0000 7f07737fe640 10 client.15867 handle_command_reply: tid=0 2024-06-10T01:07:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.963+0000 7f079d7f1740 1 client.15867 shutdown 2024-06-10T01:07:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.963+0000 7f079d7f1740 1 -- 172.21.15.7:0/1475403662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f2dd2f80 msgr2=0x5638f2ddebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.963+0000 7f079d7f1740 1 --2- 172.21.15.7:0/1475403662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f2dd2f80 0x5638f2ddebb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f078c009ca0 tx=0x7f078c0095c0 comp rx=0 tx=0).stop 2024-06-10T01:07:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.963+0000 7f079d7f1740 1 -- 172.21.15.7:0/1475403662 shutdown_connections 2024-06-10T01:07:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.963+0000 7f079d7f1740 1 --2- 172.21.15.7:0/1475403662 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5638f2de3510 0x5638f2de59b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.963+0000 7f079d7f1740 1 --2- 172.21.15.7:0/1475403662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f2dd2f80 0x5638f2ddebb0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.963+0000 7f079d7f1740 1 --2- 172.21.15.7:0/1475403662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638f2dcde10 0x5638f2dd7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.963+0000 7f07737fe640 0 client.15867 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.963+0000 7f079d7f1740 1 --2- 172.21.15.7:0/1475403662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638f2dcd420 0x5638f2dd7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.963+0000 7f079d7f1740 1 -- 172.21.15.7:0/1475403662 >> 172.21.15.7:0/1475403662 conn(0x5638f2dc4680 msgr2=0x5638f2dc37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.963+0000 7f079d7f1740 1 -- 172.21.15.7:0/1475403662 shutdown_connections 2024-06-10T01:07:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.963+0000 7f079d7f1740 1 -- 172.21.15.7:0/1475403662 wait complete. 2024-06-10T01:07:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.964+0000 7f079d7f1740 20 client.15867 trim_cache size 0 max 16384 2024-06-10T01:07:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.982+0000 7f079d7f1740 1 Processor -- start 2024-06-10T01:07:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.982+0000 7f079d7f1740 1 -- start start 2024-06-10T01:07:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.982+0000 7f079d7f1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638f2dcd420 0x5638f2dc7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.982+0000 7f079d7f1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638f2dcde10 0x5638f2ddeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.982+0000 7f079d7f1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f2dd2f80 0x5638f2dc7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.982+0000 7f079d7f1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f07941aacc0 con 0x5638f2dd2f80 2024-06-10T01:07:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.982+0000 7f079d7f1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f07941a8590 con 0x5638f2dcde10 2024-06-10T01:07:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.982+0000 7f079d7f1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f07941a63e0 con 0x5638f2dcd420 2024-06-10T01:07:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.982+0000 7f0798fa6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f2dd2f80 0x5638f2dc7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.982+0000 7f0798fa6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f2dd2f80 0x5638f2dc7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35038/0 (socket says 172.21.15.7:35038) 2024-06-10T01:07:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.982+0000 7f0798fa6640 1 -- 172.21.15.7:0/3420499087 learned_addr learned my addr 172.21.15.7:0/3420499087 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.982+0000 7f07937fe640 1 --2- 172.21.15.7:0/3420499087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638f2dcde10 0x5638f2ddeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.982+0000 7f0798fa6640 1 -- 172.21.15.7:0/3420499087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638f2dcd420 msgr2=0x5638f2dc7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.982+0000 7f0798fa6640 1 --2- 172.21.15.7:0/3420499087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638f2dcd420 0x5638f2dc7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.982+0000 7f0798fa6640 1 -- 172.21.15.7:0/3420499087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638f2dcde10 msgr2=0x5638f2ddeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.982+0000 7f0798fa6640 1 --2- 172.21.15.7:0/3420499087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638f2dcde10 0x5638f2ddeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.982+0000 7f0798fa6640 1 -- 172.21.15.7:0/3420499087 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638f2dc44b0 con 0x5638f2dd2f80 2024-06-10T01:07:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.983+0000 7f07937fe640 1 --2- 172.21.15.7:0/3420499087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638f2dcde10 0x5638f2ddeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:07:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.983+0000 7f0798fa6640 1 --2- 172.21.15.7:0/3420499087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f2dd2f80 0x5638f2dc7bb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f078c02ade0 tx=0x7f078c031f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abdb2a30d3f402a3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.983+0000 7f0779ffb640 1 -- 172.21.15.7:0/3420499087 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c057070 con 0x5638f2dd2f80 2024-06-10T01:07:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.983+0000 7f0779ffb640 1 -- 172.21.15.7:0/3420499087 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f078c036040 con 0x5638f2dd2f80 2024-06-10T01:07:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.983+0000 7f0779ffb640 1 -- 172.21.15.7:0/3420499087 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c04e070 con 0x5638f2dd2f80 2024-06-10T01:07:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.984+0000 7f079d7f1740 1 -- 172.21.15.7:0/3420499087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f2dd2f80 msgr2=0x5638f2dc7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.984+0000 7f079d7f1740 1 --2- 172.21.15.7:0/3420499087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f2dd2f80 0x5638f2dc7bb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f078c02ade0 tx=0x7f078c031f80 comp rx=0 tx=0).stop 2024-06-10T01:07:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.984+0000 7f079d7f1740 1 -- 172.21.15.7:0/3420499087 shutdown_connections 2024-06-10T01:07:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.984+0000 7f079d7f1740 1 --2- 172.21.15.7:0/3420499087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f2dd2f80 0x5638f2dc7bb0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.984+0000 7f079d7f1740 1 --2- 172.21.15.7:0/3420499087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638f2dcde10 0x5638f2ddeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.984+0000 7f079d7f1740 1 --2- 172.21.15.7:0/3420499087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638f2dcd420 0x5638f2dc7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.984+0000 7f079d7f1740 1 -- 172.21.15.7:0/3420499087 >> 172.21.15.7:0/3420499087 conn(0x5638f2dc4680 msgr2=0x5638f2dceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.984+0000 7f079d7f1740 1 -- 172.21.15.7:0/3420499087 shutdown_connections 2024-06-10T01:07:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.984+0000 7f079d7f1740 1 -- 172.21.15.7:0/3420499087 wait complete. 2024-06-10T01:07:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.984+0000 7f079d7f1740 1 -- start start 2024-06-10T01:07:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.984+0000 7f079d7f1740 1 Processor -- start 2024-06-10T01:07:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.984+0000 7f079d7f1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f2dcd420 0x5638f2ddc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.984+0000 7f079d7f1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638f2dcde10 0x5638f2ddc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.984+0000 7f079d7f1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638f2dd2f80 0x5638f2ddd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.984+0000 7f079d7f1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f07941a6a50 con 0x5638f2dcd420 2024-06-10T01:07:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.984+0000 7f079d7f1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f07941a7ec0 con 0x5638f2dcde10 2024-06-10T01:07:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.984+0000 7f079d7f1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f07941a7120 con 0x5638f2dd2f80 2024-06-10T01:07:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.984+0000 7f079d7f1740 10 client.0 _ll_register_callbacks cb 0x7f078c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.984+0000 7f0793fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f2dcd420 0x5638f2ddc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.985+0000 7f0793fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f2dcd420 0x5638f2ddc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35040/0 (socket says 172.21.15.7:35040) 2024-06-10T01:07:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.985+0000 7f0793fff640 1 -- 172.21.15.7:0/2934261877 learned_addr learned my addr 172.21.15.7:0/2934261877 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.985+0000 7f0798fa6640 1 --2- 172.21.15.7:0/2934261877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638f2dd2f80 0x5638f2ddd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.985+0000 7f07937fe640 1 --2- 172.21.15.7:0/2934261877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638f2dcde10 0x5638f2ddc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.985+0000 7f0793fff640 1 -- 172.21.15.7:0/2934261877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638f2dd2f80 msgr2=0x5638f2ddd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.985+0000 7f0793fff640 1 --2- 172.21.15.7:0/2934261877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638f2dd2f80 0x5638f2ddd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.985+0000 7f0793fff640 1 -- 172.21.15.7:0/2934261877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638f2dcde10 msgr2=0x5638f2ddc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.985+0000 7f0793fff640 1 --2- 172.21.15.7:0/2934261877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638f2dcde10 0x5638f2ddc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.985+0000 7f0793fff640 1 -- 172.21.15.7:0/2934261877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638f2dceae0 con 0x5638f2dcd420 2024-06-10T01:07:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.985+0000 7f0798fa6640 1 --2- 172.21.15.7:0/2934261877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638f2dd2f80 0x5638f2ddd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.985+0000 7f07937fe640 1 --2- 172.21.15.7:0/2934261877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638f2dcde10 0x5638f2ddc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:07:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.985+0000 7f0793fff640 1 --2- 172.21.15.7:0/2934261877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f2dcd420 0x5638f2ddc290 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f078409b700 tx=0x7f0784003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d494b3818ada927a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.985+0000 7f07737fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:07:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.985+0000 7f07737fe640 1 -- 172.21.15.7:0/2934261877 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078401d040 con 0x5638f2dcd420 2024-06-10T01:07:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.985+0000 7f07737fe640 1 -- 172.21.15.7:0/2934261877 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f0784003610 con 0x5638f2dcd420 2024-06-10T01:07:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.985+0000 7f079d7f1740 1 -- 172.21.15.7:0/2934261877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f078c036040 con 0x5638f2dcd420 2024-06-10T01:07:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.985+0000 7f07737fe640 1 -- 172.21.15.7:0/2934261877 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f07840998b0 con 0x5638f2dcd420 2024-06-10T01:07:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.985+0000 7f079d7f1740 10 client.15873 fetch_fsmap learned FSMap version 27 2024-06-10T01:07:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.985+0000 7f079d7f1740 1 -- 172.21.15.7:0/2934261877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5638f2ddeea0 con 0x5638f2dcd420 2024-06-10T01:07:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.986+0000 7f07737fe640 1 -- 172.21.15.7:0/2934261877 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f078c036040 con 0x5638f2dcd420 2024-06-10T01:07:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.986+0000 7f07737fe640 1 -- 172.21.15.7:0/2934261877 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0784097d50 con 0x5638f2dcd420 2024-06-10T01:07:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.986+0000 7f079d7f1740 10 client.15873 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:07:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.986+0000 7f079d7f1740 10 client.15873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.986+0000 7f079d7f1740 20 client.15873 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.986+0000 7f079d7f1740 1 --2- 172.21.15.7:0/2934261877 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5638f2e4e920 0x5638f2e50d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.986+0000 7f079d7f1740 4 client.15873 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:07:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.986+0000 7f079d7f1740 1 -- 172.21.15.7:0/2934261877 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5638f2d39c20 con 0x5638f2e4e920 2024-06-10T01:07:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.986+0000 7f07937fe640 1 --2- 172.21.15.7:0/2934261877 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5638f2e4e920 0x5638f2e50d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.987+0000 7f07937fe640 1 --2- 172.21.15.7:0/2934261877 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5638f2e4e920 0x5638f2e50d40 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92424e51354077fb server_cookie=dc3366a1f4640330 in_seq=0 out_seq=0 2024-06-10T01:07:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:35.987+0000 7f07737fe640 10 client.15873 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:36.120+0000 7f07737fe640 1 -- 172.21.15.7:0/2934261877 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5638f2d39c20 con 0x5638f2e4e920 2024-06-10T01:07:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:36.120+0000 7f07737fe640 10 client.15873 handle_command_reply: tid=0 2024-06-10T01:07:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:36.120+0000 7f079d7f1740 1 client.15873 shutdown 2024-06-10T01:07:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:36.120+0000 7f079d7f1740 1 -- 172.21.15.7:0/2934261877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f2dcd420 msgr2=0x5638f2ddc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:36.120+0000 7f079d7f1740 1 --2- 172.21.15.7:0/2934261877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f2dcd420 0x5638f2ddc290 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f078409b700 tx=0x7f0784003f80 comp rx=0 tx=0).stop 2024-06-10T01:07:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:36.120+0000 7f079d7f1740 1 -- 172.21.15.7:0/2934261877 shutdown_connections 2024-06-10T01:07:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:36.120+0000 7f079d7f1740 1 --2- 172.21.15.7:0/2934261877 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5638f2e4e920 0x5638f2e50d40 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:36.120+0000 7f07737fe640 0 client.15873 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:36.120+0000 7f079d7f1740 1 --2- 172.21.15.7:0/2934261877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638f2dd2f80 0x5638f2ddd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:36.120+0000 7f079d7f1740 1 --2- 172.21.15.7:0/2934261877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638f2dcde10 0x5638f2ddc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:36.120+0000 7f079d7f1740 1 --2- 172.21.15.7:0/2934261877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f2dcd420 0x5638f2ddc290 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:36.120+0000 7f079d7f1740 1 -- 172.21.15.7:0/2934261877 >> 172.21.15.7:0/2934261877 conn(0x5638f2dc4680 msgr2=0x5638f2dd7630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:36.120+0000 7f079d7f1740 1 -- 172.21.15.7:0/2934261877 shutdown_connections 2024-06-10T01:07:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:36.121+0000 7f079d7f1740 1 -- 172.21.15.7:0/2934261877 wait complete. 2024-06-10T01:07:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:36.121+0000 7f079d7f1740 20 client.15873 trim_cache size 0 max 16384 2024-06-10T01:07:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:07:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:07:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:07:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:36.121+0000 7f079aa09640 1 -- 172.21.15.7:0/3552934067 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f077c0a2b80 msgr2=0x7f077c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:36.121+0000 7f079aa09640 1 --2- 172.21.15.7:0/3552934067 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f077c0a2b80 0x7f077c0a5040 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f078c031c10 tx=0x7f078c02f310 comp rx=0 tx=0).stop 2024-06-10T01:07:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:36.121+0000 7f079aa09640 1 -- 172.21.15.7:0/3552934067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0794101a10 msgr2=0x7f0794197d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:36.121+0000 7f079aa09640 1 --2- 172.21.15.7:0/3552934067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0794101a10 0x7f0794197d30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f078400c970 tx=0x7f078400ce30 comp rx=0 tx=0).stop 2024-06-10T01:07:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:36.122+0000 7f079aa09640 1 -- 172.21.15.7:0/3552934067 shutdown_connections 2024-06-10T01:07:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:36.122+0000 7f079aa09640 1 --2- 172.21.15.7:0/3552934067 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f077c0a2b80 0x7f077c0a5040 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:36.122+0000 7f079aa09640 1 --2- 172.21.15.7:0/3552934067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07941035c0 0x7f079419f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:36.122+0000 7f079aa09640 1 --2- 172.21.15.7:0/3552934067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0794102c00 0x7f0794198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:36.122+0000 7f079aa09640 1 --2- 172.21.15.7:0/3552934067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0794101a10 0x7f0794197d30 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:36.122+0000 7f079aa09640 1 -- 172.21.15.7:0/3552934067 >> 172.21.15.7:0/3552934067 conn(0x7f07940fcf80 msgr2=0x7f0794105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:36.122+0000 7f079aa09640 1 -- 172.21.15.7:0/3552934067 shutdown_connections 2024-06-10T01:07:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:36.122+0000 7f079aa09640 1 -- 172.21.15.7:0/3552934067 wait complete. 2024-06-10T01:07:36.708 INFO:tasks.workunit.client.0.smithi007.stdout: 1 28947 10.23 MB/sec warmup 17 sec latency 7163.383 ms 2024-06-10T01:07:36.881 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:07:36 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-f[66185]: ignoring --setuser ceph since I am not root 2024-06-10T01:07:36.881 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:07:36 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-f[66185]: ignoring --setgroup ceph since I am not root 2024-06-10T01:07:36.881 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:07:36 smithi175 ceph-mds[66189]: -- 172.21.15.175:0/1099313598 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564891e21a40 con 0x564891df4800 2024-06-10T01:07:36.881 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:07:36 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-f[66185]: starting mds.f at 2024-06-10T01:07:36.938 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:36 smithi007 ceph-mon[32398]: pgmap v479: 129 pgs: 129 active+clean; 19 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 1.7 MiB/s wr, 271 op/s 2024-06-10T01:07:36.938 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:36 smithi007 ceph-mon[32398]: daemon mds.f restarted 2024-06-10T01:07:37.204 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:36 smithi175 ceph-mon[36636]: pgmap v479: 129 pgs: 129 active+clean; 19 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 1.7 MiB/s wr, 271 op/s 2024-06-10T01:07:37.204 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:36 smithi175 ceph-mon[36636]: daemon mds.f restarted 2024-06-10T01:07:37.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:36 smithi078 ceph-mon[36683]: pgmap v479: 129 pgs: 129 active+clean; 19 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 1.7 MiB/s wr, 271 op/s 2024-06-10T01:07:37.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:36 smithi078 ceph-mon[36683]: daemon mds.f restarted 2024-06-10T01:07:37.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.463+0000 7f00ccd2c700 1 -- 172.21.15.7:0/2664043919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00c8101b50 msgr2=0x7f00c8101fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:37.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.463+0000 7f00ccd2c700 1 --2- 172.21.15.7:0/2664043919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00c8101b50 0x7f00c8101fd0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f00b0009990 tx=0x7f00b0009ca0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.464+0000 7f00ccd2c700 1 -- 172.21.15.7:0/2664043919 shutdown_connections 2024-06-10T01:07:37.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.464+0000 7f00ccd2c700 1 --2- 172.21.15.7:0/2664043919 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c8102510 0x7f00c810e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.464+0000 7f00ccd2c700 1 --2- 172.21.15.7:0/2664043919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00c81063a0 0x7f00c8106780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.464+0000 7f00ccd2c700 1 --2- 172.21.15.7:0/2664043919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00c8101b50 0x7f00c8101fd0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.464+0000 7f00ccd2c700 1 -- 172.21.15.7:0/2664043919 >> 172.21.15.7:0/2664043919 conn(0x7f00c8077990 msgr2=0x7f00c80fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:37.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.464+0000 7f00ccd2c700 1 -- 172.21.15.7:0/2664043919 shutdown_connections 2024-06-10T01:07:37.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.464+0000 7f00ccd2c700 1 -- 172.21.15.7:0/2664043919 wait complete. 2024-06-10T01:07:37.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.464+0000 7f00ccd2c700 1 Processor -- start 2024-06-10T01:07:37.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.464+0000 7f00ccd2c700 1 -- start start 2024-06-10T01:07:37.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.465+0000 7f00ccd2c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c8101b50 0x7f00c8198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:37.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.465+0000 7f00ccd2c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00c8102510 0x7f00c8199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:37.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.465+0000 7f00ccd2c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00c81063a0 0x7f00c819d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:37.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.465+0000 7f00ccd2c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f00c8112940 con 0x7f00c8102510 2024-06-10T01:07:37.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.465+0000 7f00ccd2c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f00c81127c0 con 0x7f00c81063a0 2024-06-10T01:07:37.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.465+0000 7f00ccd2c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f00c8112ac0 con 0x7f00c8101b50 2024-06-10T01:07:37.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.465+0000 7f00c5d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00c8102510 0x7f00c8199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:37.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.465+0000 7f00c5d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00c8102510 0x7f00c8199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35060/0 (socket says 172.21.15.7:35060) 2024-06-10T01:07:37.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.465+0000 7f00c5d9b700 1 -- 172.21.15.7:0/2316727152 learned_addr learned my addr 172.21.15.7:0/2316727152 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:37.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.465+0000 7f00c6d9d700 1 --2- 172.21.15.7:0/2316727152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00c81063a0 0x7f00c819d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:37.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.465+0000 7f00c659c700 1 --2- 172.21.15.7:0/2316727152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c8101b50 0x7f00c8198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:37.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.465+0000 7f00c5d9b700 1 -- 172.21.15.7:0/2316727152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c8101b50 msgr2=0x7f00c8198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:37.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.465+0000 7f00c5d9b700 1 --2- 172.21.15.7:0/2316727152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c8101b50 0x7f00c8198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.465+0000 7f00c5d9b700 1 -- 172.21.15.7:0/2316727152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00c81063a0 msgr2=0x7f00c819d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:37.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.465+0000 7f00c5d9b700 1 --2- 172.21.15.7:0/2316727152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00c81063a0 0x7f00c819d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.465+0000 7f00c5d9b700 1 -- 172.21.15.7:0/2316727152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c819dd60 con 0x7f00c8102510 2024-06-10T01:07:37.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.465+0000 7f00c659c700 1 --2- 172.21.15.7:0/2316727152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c8101b50 0x7f00c8198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:07:37.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.465+0000 7f00c5d9b700 1 --2- 172.21.15.7:0/2316727152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00c8102510 0x7f00c8199290 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f00b000bff0 tx=0x7f00b0005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:37.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.466+0000 7f00bf7fe700 1 -- 172.21.15.7:0/2316727152 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b0027070 con 0x7f00c8102510 2024-06-10T01:07:37.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.466+0000 7f00ccd2c700 1 -- 172.21.15.7:0/2316727152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00c819dff0 con 0x7f00c8102510 2024-06-10T01:07:37.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.466+0000 7f00bf7fe700 1 -- 172.21.15.7:0/2316727152 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00b00040d0 con 0x7f00c8102510 2024-06-10T01:07:37.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.466+0000 7f00bf7fe700 1 -- 172.21.15.7:0/2316727152 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b0022e00 con 0x7f00c8102510 2024-06-10T01:07:37.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.466+0000 7f00ccd2c700 1 -- 172.21.15.7:0/2316727152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00c81a58d0 con 0x7f00c8102510 2024-06-10T01:07:37.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.468+0000 7f00bf7fe700 1 -- 172.21.15.7:0/2316727152 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f00b002a070 con 0x7f00c8102510 2024-06-10T01:07:37.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.469+0000 7f00bf7fe700 1 --2- 172.21.15.7:0/2316727152 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f00b40a2c20 0x7f00b40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:37.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.469+0000 7f00bf7fe700 1 -- 172.21.15.7:0/2316727152 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f00b00d0180 con 0x7f00c8102510 2024-06-10T01:07:37.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.469+0000 7f00c659c700 1 --2- 172.21.15.7:0/2316727152 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f00b40a2c20 0x7f00b40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:37.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.469+0000 7f00c659c700 1 --2- 172.21.15.7:0/2316727152 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f00b40a2c20 0x7f00b40a50e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f00b80060b0 tx=0x7f00b80076a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:37.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.469+0000 7f00d75afb80 1 Processor -- start 2024-06-10T01:07:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.470+0000 7f00d75afb80 1 -- start start 2024-06-10T01:07:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.470+0000 7f00d75afb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc49ab3640 0x55cc49ab3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.470+0000 7f00d75afb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc49ab3f60 0x55cc49aad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.470+0000 7f00d75afb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc49aadf20 0x55cc49aae340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.470+0000 7f00d75afb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cc499ff810 con 0x55cc49ab3f60 2024-06-10T01:07:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.470+0000 7f00d75afb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cc499a7720 con 0x55cc49ab3640 2024-06-10T01:07:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.470+0000 7f00d75afb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55cc499fb7d0 con 0x55cc49aadf20 2024-06-10T01:07:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.470+0000 7f00c5d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc49ab3f60 0x55cc49aad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.470+0000 7f00c5d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc49ab3f60 0x55cc49aad9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35070/0 (socket says 172.21.15.7:35070) 2024-06-10T01:07:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.470+0000 7f00c5d9b700 1 -- 172.21.15.7:0/3557331618 learned_addr learned my addr 172.21.15.7:0/3557331618 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.470+0000 7f00c659c700 1 --2- 172.21.15.7:0/3557331618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc49ab3640 0x55cc49ab3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.470+0000 7f00c6d9d700 1 --2- 172.21.15.7:0/3557331618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc49aadf20 0x55cc49aae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.470+0000 7f00c5d9b700 1 -- 172.21.15.7:0/3557331618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc49aadf20 msgr2=0x55cc49aae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.470+0000 7f00c5d9b700 1 --2- 172.21.15.7:0/3557331618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc49aadf20 0x55cc49aae340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.470+0000 7f00c5d9b700 1 -- 172.21.15.7:0/3557331618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc49ab3640 msgr2=0x55cc49ab3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.470+0000 7f00c5d9b700 1 --2- 172.21.15.7:0/3557331618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc49ab3640 0x55cc49ab3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.470+0000 7f00c5d9b700 1 -- 172.21.15.7:0/3557331618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc4999df50 con 0x55cc49ab3f60 2024-06-10T01:07:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.470+0000 7f00c6d9d700 1 --2- 172.21.15.7:0/3557331618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc49aadf20 0x55cc49aae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:07:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.471+0000 7f00c5d9b700 1 --2- 172.21.15.7:0/3557331618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc49ab3f60 0x55cc49aad9e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f00b0098b50 tx=0x7f00b0022470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c24f97d6e894a5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.471+0000 7f00bcff9700 1 -- 172.21.15.7:0/3557331618 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b009cdf0 con 0x55cc49ab3f60 2024-06-10T01:07:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.471+0000 7f00bcff9700 1 -- 172.21.15.7:0/3557331618 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00b009c6f0 con 0x55cc49ab3f60 2024-06-10T01:07:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.471+0000 7f00bcff9700 1 -- 172.21.15.7:0/3557331618 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b009c950 con 0x55cc49ab3f60 2024-06-10T01:07:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.471+0000 7f00d75afb80 1 -- 172.21.15.7:0/3557331618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc49ab3f60 msgr2=0x55cc49aad9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.471+0000 7f00d75afb80 1 --2- 172.21.15.7:0/3557331618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc49ab3f60 0x55cc49aad9e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f00b0098b50 tx=0x7f00b0022470 comp rx=0 tx=0).stop 2024-06-10T01:07:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.471+0000 7f00d75afb80 1 -- 172.21.15.7:0/3557331618 shutdown_connections 2024-06-10T01:07:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.471+0000 7f00d75afb80 1 --2- 172.21.15.7:0/3557331618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc49aadf20 0x55cc49aae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.471+0000 7f00d75afb80 1 --2- 172.21.15.7:0/3557331618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc49ab3640 0x55cc49ab3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.471+0000 7f00d75afb80 1 --2- 172.21.15.7:0/3557331618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc49ab3f60 0x55cc49aad9e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.472+0000 7f00d75afb80 1 -- 172.21.15.7:0/3557331618 >> 172.21.15.7:0/3557331618 conn(0x55cc499d8cc0 msgr2=0x55cc49aa8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.472+0000 7f00d75afb80 1 -- 172.21.15.7:0/3557331618 shutdown_connections 2024-06-10T01:07:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.472+0000 7f00d75afb80 1 -- 172.21.15.7:0/3557331618 wait complete. 2024-06-10T01:07:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.472+0000 7f00d75afb80 1 -- start start 2024-06-10T01:07:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.472+0000 7f00d75afb80 1 Processor -- start 2024-06-10T01:07:37.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.472+0000 7f00d75afb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc49aadf20 0x55cc49ab9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:37.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.472+0000 7f00d75afb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc49ab3640 0x55cc49ab70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:37.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.472+0000 7f00d75afb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc49ab3f60 0x55cc49ab75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:37.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.472+0000 7f00d75afb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cc49aba6f0 con 0x55cc49aadf20 2024-06-10T01:07:37.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.472+0000 7f00d75afb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cc4999fbf0 con 0x55cc49ab3640 2024-06-10T01:07:37.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.472+0000 7f00d75afb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55cc49aba870 con 0x55cc49ab3f60 2024-06-10T01:07:37.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.472+0000 7f00d75afb80 10 client.0 _ll_register_callbacks cb 0x55cc49999920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:37.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.472+0000 7f00c659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc49aadf20 0x55cc49ab9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:37.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.473+0000 7f00c659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc49aadf20 0x55cc49ab9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35084/0 (socket says 172.21.15.7:35084) 2024-06-10T01:07:37.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.473+0000 7f00c659c700 1 -- 172.21.15.7:0/2280220308 learned_addr learned my addr 172.21.15.7:0/2280220308 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:37.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.473+0000 7f00c5d9b700 1 --2- 172.21.15.7:0/2280220308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc49ab3640 0x55cc49ab70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:37.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.473+0000 7f00c6d9d700 1 --2- 172.21.15.7:0/2280220308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc49ab3f60 0x55cc49ab75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:37.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.473+0000 7f00c659c700 1 -- 172.21.15.7:0/2280220308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc49ab3f60 msgr2=0x55cc49ab75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:37.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.473+0000 7f00c659c700 1 --2- 172.21.15.7:0/2280220308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc49ab3f60 0x55cc49ab75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.473+0000 7f00c659c700 1 -- 172.21.15.7:0/2280220308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc49ab3640 msgr2=0x55cc49ab70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:37.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.473+0000 7f00c659c700 1 --2- 172.21.15.7:0/2280220308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc49ab3640 0x55cc49ab70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.473+0000 7f00c659c700 1 -- 172.21.15.7:0/2280220308 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc4994e8d0 con 0x55cc49aadf20 2024-06-10T01:07:37.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.473+0000 7f00c6d9d700 1 --2- 172.21.15.7:0/2280220308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc49ab3f60 0x55cc49ab75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:37.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.473+0000 7f00c659c700 1 --2- 172.21.15.7:0/2280220308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc49aadf20 0x55cc49ab9d10 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f00b8014430 tx=0x7f00b8004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a468dad12e4ae93f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:37.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.473+0000 7f00adffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:07:37.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.473+0000 7f00adffb700 1 -- 172.21.15.7:0/2280220308 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b8014e00 con 0x55cc49aadf20 2024-06-10T01:07:37.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.473+0000 7f00d75afb80 1 -- 172.21.15.7:0/2280220308 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00b009c6f0 con 0x55cc49aadf20 2024-06-10T01:07:37.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.473+0000 7f00adffb700 1 -- 172.21.15.7:0/2280220308 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f00b8015420 con 0x55cc49aadf20 2024-06-10T01:07:37.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.473+0000 7f00adffb700 1 -- 172.21.15.7:0/2280220308 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f00b8015620 con 0x55cc49aadf20 2024-06-10T01:07:37.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.474+0000 7f00d75afb80 10 client.15888 fetch_fsmap learned FSMap version 29 2024-06-10T01:07:37.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.474+0000 7f00d75afb80 1 -- 172.21.15.7:0/2280220308 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cc49ac4fc0 con 0x55cc49aadf20 2024-06-10T01:07:37.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.474+0000 7f00adffb700 1 -- 172.21.15.7:0/2280220308 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00b009c6f0 con 0x55cc49aadf20 2024-06-10T01:07:37.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.474+0000 7f00adffb700 1 -- 172.21.15.7:0/2280220308 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f00b8021920 con 0x55cc49aadf20 2024-06-10T01:07:37.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.474+0000 7f00d75afb80 10 client.15888 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:07:37.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.474+0000 7f00d75afb80 10 client.15888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:37.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.474+0000 7f00d75afb80 20 client.15888 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:37.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.474+0000 7f00d75afb80 1 --2- 172.21.15.7:0/2280220308 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55cc49ac7550 0x55cc49ac9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:37.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.474+0000 7f00d75afb80 4 client.15888 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:07:37.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.474+0000 7f00d75afb80 1 -- 172.21.15.7:0/2280220308 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc49a0cd10 con 0x55cc49ac7550 2024-06-10T01:07:37.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.474+0000 7f00c5d9b700 1 --2- 172.21.15.7:0/2280220308 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55cc49ac7550 0x55cc49ac9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:37.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.475+0000 7f00adffb700 10 client.15888 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:37.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.475+0000 7f00c5d9b700 1 --2- 172.21.15.7:0/2280220308 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55cc49ac7550 0x55cc49ac9980 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25b57030bedbe97f server_cookie=584dbf61cf69d981 in_seq=0 out_seq=0 2024-06-10T01:07:37.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.476+0000 7f00adffb700 1 -- 172.21.15.7:0/2280220308 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cc49a0cd10 con 0x55cc49ac7550 2024-06-10T01:07:37.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.476+0000 7f00adffb700 10 client.15888 handle_command_reply: tid=0 2024-06-10T01:07:37.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.476+0000 7f00d75afb80 1 client.15888 shutdown 2024-06-10T01:07:37.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.476+0000 7f00d75afb80 1 -- 172.21.15.7:0/2280220308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc49aadf20 msgr2=0x55cc49ab9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:37.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.476+0000 7f00d75afb80 1 --2- 172.21.15.7:0/2280220308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc49aadf20 0x55cc49ab9d10 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f00b8014430 tx=0x7f00b8004b40 comp rx=0 tx=0).stop 2024-06-10T01:07:37.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.476+0000 7f00d75afb80 1 -- 172.21.15.7:0/2280220308 shutdown_connections 2024-06-10T01:07:37.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.476+0000 7f00d75afb80 1 --2- 172.21.15.7:0/2280220308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc49ab3f60 0x55cc49ab75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.476+0000 7f00d75afb80 1 --2- 172.21.15.7:0/2280220308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc49aadf20 0x55cc49ab9d10 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.476+0000 7f00d75afb80 1 --2- 172.21.15.7:0/2280220308 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55cc49ac7550 0x55cc49ac9980 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.476+0000 7f00d75afb80 1 --2- 172.21.15.7:0/2280220308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc49ab3640 0x55cc49ab70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.476+0000 7f00adffb700 0 client.15888 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:37.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.476+0000 7f00d75afb80 1 -- 172.21.15.7:0/2280220308 >> 172.21.15.7:0/2280220308 conn(0x55cc499d8cc0 msgr2=0x55cc49aac9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:37.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.476+0000 7f00d75afb80 1 -- 172.21.15.7:0/2280220308 shutdown_connections 2024-06-10T01:07:37.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.476+0000 7f00d75afb80 1 -- 172.21.15.7:0/2280220308 wait complete. 2024-06-10T01:07:37.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.476+0000 7f00d75afb80 20 client.15888 trim_cache size 0 max 16384 2024-06-10T01:07:37.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.496+0000 7f00d75afb80 1 Processor -- start 2024-06-10T01:07:37.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.496+0000 7f00d75afb80 1 -- start start 2024-06-10T01:07:37.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.497+0000 7f00d75afb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc49aadf20 0x55cc49aad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:37.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.497+0000 7f00d75afb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc49ab3640 0x55cc49ab9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:37.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.497+0000 7f00d75afb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc49ab3f60 0x55cc49aa2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:37.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.497+0000 7f00d75afb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f00c81b0bd0 con 0x55cc49ab3f60 2024-06-10T01:07:37.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.497+0000 7f00d75afb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f00c81ae460 con 0x55cc49aadf20 2024-06-10T01:07:37.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.497+0000 7f00d75afb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f00c81ac2b0 con 0x55cc49ab3640 2024-06-10T01:07:37.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.497+0000 7f00c6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc49ab3f60 0x55cc49aa2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:37.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.497+0000 7f00c5d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc49ab3640 0x55cc49ab9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:37.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.497+0000 7f00c6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc49ab3f60 0x55cc49aa2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35100/0 (socket says 172.21.15.7:35100) 2024-06-10T01:07:37.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.497+0000 7f00c659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc49aadf20 0x55cc49aad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:37.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.497+0000 7f00c6d9d700 1 -- 172.21.15.7:0/413985594 learned_addr learned my addr 172.21.15.7:0/413985594 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:37.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.497+0000 7f00c5d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc49ab3640 0x55cc49ab9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:55058/0 (socket says 172.21.15.7:55058) 2024-06-10T01:07:37.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.497+0000 7f00c6d9d700 1 -- 172.21.15.7:0/413985594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc49ab3640 msgr2=0x55cc49ab9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:37.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.497+0000 7f00c6d9d700 1 --2- 172.21.15.7:0/413985594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc49ab3640 0x55cc49ab9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.497+0000 7f00c6d9d700 1 -- 172.21.15.7:0/413985594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc49aadf20 msgr2=0x55cc49aad9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:37.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.497+0000 7f00c6d9d700 1 --2- 172.21.15.7:0/413985594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc49aadf20 0x55cc49aad9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.497+0000 7f00c6d9d700 1 -- 172.21.15.7:0/413985594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc49aca000 con 0x55cc49ab3f60 2024-06-10T01:07:37.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.497+0000 7f00c659c700 1 --2- 172.21.15.7:0/413985594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc49aadf20 0x55cc49aad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:37.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.498+0000 7f00c6d9d700 1 --2- 172.21.15.7:0/413985594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc49ab3f60 0x55cc49aa2690 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f00c00099e0 tx=0x7f00c0009060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7bfb1a9b5572968 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:37.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.498+0000 7f00affff700 1 -- 172.21.15.7:0/413985594 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0019030 con 0x55cc49ab3f60 2024-06-10T01:07:37.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.498+0000 7f00affff700 1 -- 172.21.15.7:0/413985594 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00c0012a50 con 0x55cc49ab3f60 2024-06-10T01:07:37.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.498+0000 7f00affff700 1 -- 172.21.15.7:0/413985594 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c000f040 con 0x55cc49ab3f60 2024-06-10T01:07:37.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.498+0000 7f00d75afb80 1 -- 172.21.15.7:0/413985594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc49ab3f60 msgr2=0x55cc49aa2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:37.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.498+0000 7f00d75afb80 1 --2- 172.21.15.7:0/413985594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc49ab3f60 0x55cc49aa2690 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f00c00099e0 tx=0x7f00c0009060 comp rx=0 tx=0).stop 2024-06-10T01:07:37.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.499+0000 7f00d75afb80 1 -- 172.21.15.7:0/413985594 shutdown_connections 2024-06-10T01:07:37.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.499+0000 7f00d75afb80 1 --2- 172.21.15.7:0/413985594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc49ab3f60 0x55cc49aa2690 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.499+0000 7f00d75afb80 1 --2- 172.21.15.7:0/413985594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc49aadf20 0x55cc49aad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.499+0000 7f00d75afb80 1 --2- 172.21.15.7:0/413985594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc49ab3640 0x55cc49ab9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.499+0000 7f00d75afb80 1 -- 172.21.15.7:0/413985594 >> 172.21.15.7:0/413985594 conn(0x55cc499d8cc0 msgr2=0x55cc49ab49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:37.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.499+0000 7f00d75afb80 1 -- 172.21.15.7:0/413985594 shutdown_connections 2024-06-10T01:07:37.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.499+0000 7f00d75afb80 1 -- 172.21.15.7:0/413985594 wait complete. 2024-06-10T01:07:37.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.499+0000 7f00d75afb80 1 -- start start 2024-06-10T01:07:37.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.499+0000 7f00d75afb80 1 Processor -- start 2024-06-10T01:07:37.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.499+0000 7f00d75afb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc49aadf20 0x55cc49ac0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:37.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.499+0000 7f00d75afb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc49ab3640 0x55cc49abb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:37.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.499+0000 7f00d75afb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc49ab3f60 0x55cc49abbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:37.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.499+0000 7f00d75afb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f00c81ac920 con 0x55cc49aadf20 2024-06-10T01:07:37.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.499+0000 7f00d75afb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f00c81add90 con 0x55cc49ab3f60 2024-06-10T01:07:37.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.499+0000 7f00d75afb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f00c81acff0 con 0x55cc49ab3640 2024-06-10T01:07:37.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.499+0000 7f00d75afb80 10 client.0 _ll_register_callbacks cb 0x55cc49999920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:37.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.499+0000 7f00c659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc49aadf20 0x55cc49ac0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:37.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.499+0000 7f00c659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc49aadf20 0x55cc49ac0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35110/0 (socket says 172.21.15.7:35110) 2024-06-10T01:07:37.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.499+0000 7f00c659c700 1 -- 172.21.15.7:0/2248871676 learned_addr learned my addr 172.21.15.7:0/2248871676 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:37.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.500+0000 7f00c5d9b700 1 --2- 172.21.15.7:0/2248871676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc49ab3640 0x55cc49abb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:37.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.500+0000 7f00c6d9d700 1 --2- 172.21.15.7:0/2248871676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc49ab3f60 0x55cc49abbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:37.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.500+0000 7f00c659c700 1 -- 172.21.15.7:0/2248871676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc49ab3640 msgr2=0x55cc49abb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:37.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.500+0000 7f00c659c700 1 --2- 172.21.15.7:0/2248871676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc49ab3640 0x55cc49abb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.500+0000 7f00c659c700 1 -- 172.21.15.7:0/2248871676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc49ab3f60 msgr2=0x55cc49abbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:37.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.500+0000 7f00c659c700 1 --2- 172.21.15.7:0/2248871676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc49ab3f60 0x55cc49abbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.500+0000 7f00c659c700 1 -- 172.21.15.7:0/2248871676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc49abc2f0 con 0x55cc49aadf20 2024-06-10T01:07:37.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.500+0000 7f00c6d9d700 1 --2- 172.21.15.7:0/2248871676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc49ab3f60 0x55cc49abbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:07:37.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.500+0000 7f00c659c700 1 --2- 172.21.15.7:0/2248871676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc49aadf20 0x55cc49ac0220 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f00b8007a80 tx=0x7f00b8014d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d42163426775648 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:37.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.500+0000 7f00adffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:07:37.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.500+0000 7f00adffb700 1 -- 172.21.15.7:0/2248871676 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b8029be0 con 0x55cc49aadf20 2024-06-10T01:07:37.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.500+0000 7f00adffb700 1 -- 172.21.15.7:0/2248871676 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f00b8007cd0 con 0x55cc49aadf20 2024-06-10T01:07:37.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.500+0000 7f00d75afb80 1 -- 172.21.15.7:0/2248871676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00c0012a50 con 0x55cc49aadf20 2024-06-10T01:07:37.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.500+0000 7f00adffb700 1 -- 172.21.15.7:0/2248871676 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f00b8044050 con 0x55cc49aadf20 2024-06-10T01:07:37.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.500+0000 7f00d75afb80 10 client.15894 fetch_fsmap learned FSMap version 29 2024-06-10T01:07:37.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.500+0000 7f00d75afb80 1 -- 172.21.15.7:0/2248871676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cc49b36670 con 0x55cc49aadf20 2024-06-10T01:07:37.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.501+0000 7f00adffb700 1 -- 172.21.15.7:0/2248871676 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00c0012a50 con 0x55cc49aadf20 2024-06-10T01:07:37.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.501+0000 7f00adffb700 1 -- 172.21.15.7:0/2248871676 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f00b8015ac0 con 0x55cc49aadf20 2024-06-10T01:07:37.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.501+0000 7f00d75afb80 10 client.15894 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:07:37.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.501+0000 7f00d75afb80 10 client.15894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:37.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.501+0000 7f00d75afb80 20 client.15894 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:37.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.501+0000 7f00d75afb80 1 --2- 172.21.15.7:0/2248871676 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55cc49b36dd0 0x55cc49b39210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:37.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.501+0000 7f00d75afb80 4 client.15894 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:07:37.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.501+0000 7f00d75afb80 1 -- 172.21.15.7:0/2248871676 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cc49a07290 con 0x55cc49b36dd0 2024-06-10T01:07:37.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.501+0000 7f00c5d9b700 1 --2- 172.21.15.7:0/2248871676 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55cc49b36dd0 0x55cc49b39210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:37.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.501+0000 7f00c5d9b700 1 --2- 172.21.15.7:0/2248871676 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55cc49b36dd0 0x55cc49b39210 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7af7f3046d30d11 server_cookie=29b371b3e10ea81b in_seq=0 out_seq=0 2024-06-10T01:07:37.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.501+0000 7f00adffb700 10 client.15894 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:37.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.502+0000 7f00adffb700 1 -- 172.21.15.7:0/2248871676 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cc49a07290 con 0x55cc49b36dd0 2024-06-10T01:07:37.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.502+0000 7f00adffb700 10 client.15894 handle_command_reply: tid=0 2024-06-10T01:07:37.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.502+0000 7f00d75afb80 1 client.15894 shutdown 2024-06-10T01:07:37.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.502+0000 7f00d75afb80 1 -- 172.21.15.7:0/2248871676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc49aadf20 msgr2=0x55cc49ac0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:37.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.502+0000 7f00d75afb80 1 --2- 172.21.15.7:0/2248871676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc49aadf20 0x55cc49ac0220 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f00b8007a80 tx=0x7f00b8014d60 comp rx=0 tx=0).stop 2024-06-10T01:07:37.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.502+0000 7f00d75afb80 1 -- 172.21.15.7:0/2248871676 shutdown_connections 2024-06-10T01:07:37.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.502+0000 7f00d75afb80 1 --2- 172.21.15.7:0/2248871676 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55cc49b36dd0 0x55cc49b39210 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.502+0000 7f00d75afb80 1 --2- 172.21.15.7:0/2248871676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc49ab3f60 0x55cc49abbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.502+0000 7f00d75afb80 1 --2- 172.21.15.7:0/2248871676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc49aadf20 0x55cc49ac0220 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.502+0000 7f00d75afb80 1 --2- 172.21.15.7:0/2248871676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc49ab3640 0x55cc49abb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.502+0000 7f00adffb700 0 client.15894 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:37.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.502+0000 7f00d75afb80 1 -- 172.21.15.7:0/2248871676 >> 172.21.15.7:0/2248871676 conn(0x55cc499d8cc0 msgr2=0x55cc49ab2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:37.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.502+0000 7f00d75afb80 1 -- 172.21.15.7:0/2248871676 shutdown_connections 2024-06-10T01:07:37.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.502+0000 7f00d75afb80 1 -- 172.21.15.7:0/2248871676 wait complete. 2024-06-10T01:07:37.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.502+0000 7f00d75afb80 20 client.15894 trim_cache size 0 max 16384 2024-06-10T01:07:37.504 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:07:37.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.503+0000 7f00ccd2c700 1 -- 172.21.15.7:0/2316727152 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f00b40a2c20 msgr2=0x7f00b40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:37.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.503+0000 7f00ccd2c700 1 --2- 172.21.15.7:0/2316727152 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f00b40a2c20 0x7f00b40a50e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f00b80060b0 tx=0x7f00b80076a0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.503+0000 7f00ccd2c700 1 -- 172.21.15.7:0/2316727152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00c8102510 msgr2=0x7f00c8199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:37.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.503+0000 7f00ccd2c700 1 --2- 172.21.15.7:0/2316727152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00c8102510 0x7f00c8199290 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f00b000bff0 tx=0x7f00b0005590 comp rx=0 tx=0).stop 2024-06-10T01:07:37.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.503+0000 7f00ccd2c700 1 -- 172.21.15.7:0/2316727152 shutdown_connections 2024-06-10T01:07:37.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.503+0000 7f00ccd2c700 1 --2- 172.21.15.7:0/2316727152 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f00b40a2c20 0x7f00b40a50e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.503+0000 7f00ccd2c700 1 --2- 172.21.15.7:0/2316727152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00c81063a0 0x7f00c819d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.503+0000 7f00ccd2c700 1 --2- 172.21.15.7:0/2316727152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c8101b50 0x7f00c8198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.503+0000 7f00ccd2c700 1 --2- 172.21.15.7:0/2316727152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00c8102510 0x7f00c8199290 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:37.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.503+0000 7f00ccd2c700 1 -- 172.21.15.7:0/2316727152 >> 172.21.15.7:0/2316727152 conn(0x7f00c8077990 msgr2=0x7f00c81043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:37.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.503+0000 7f00ccd2c700 1 -- 172.21.15.7:0/2316727152 shutdown_connections 2024-06-10T01:07:37.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:37.503+0000 7f00ccd2c700 1 -- 172.21.15.7:0/2316727152 wait complete. 2024-06-10T01:07:37.708 INFO:tasks.workunit.client.0.smithi007.stdout: 1 28947 9.67 MB/sec warmup 18 sec latency 8163.480 ms 2024-06-10T01:07:38.165 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:37 smithi007 ceph-mon[32398]: osdmap e100: 12 total, 12 up, 12 in 2024-06-10T01:07:38.165 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:37 smithi007 ceph-mon[32398]: mds.? [v2:172.21.15.175:6834/4278336693,v1:172.21.15.175:6835/4278336693] up:boot 2024-06-10T01:07:38.165 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:37 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 2 up:standby-replay 7 up:standby 2024-06-10T01:07:38.165 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T01:07:38.165 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:37 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:07:38.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:37 smithi175 ceph-mon[36636]: osdmap e100: 12 total, 12 up, 12 in 2024-06-10T01:07:38.204 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:37 smithi175 ceph-mon[36636]: mds.? [v2:172.21.15.175:6834/4278336693,v1:172.21.15.175:6835/4278336693] up:boot 2024-06-10T01:07:38.204 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:37 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 2 up:standby-replay 7 up:standby 2024-06-10T01:07:38.204 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T01:07:38.204 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:37 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:07:38.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:37 smithi078 ceph-mon[36683]: osdmap e100: 12 total, 12 up, 12 in 2024-06-10T01:07:38.331 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:37 smithi078 ceph-mon[36683]: mds.? [v2:172.21.15.175:6834/4278336693,v1:172.21.15.175:6835/4278336693] up:boot 2024-06-10T01:07:38.331 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:37 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 2 up:standby-replay 7 up:standby 2024-06-10T01:07:38.331 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T01:07:38.331 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:37 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:07:38.708 INFO:tasks.workunit.client.0.smithi007.stdout: 1 28947 9.16 MB/sec warmup 19 sec latency 9163.559 ms 2024-06-10T01:07:39.197 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:07:39.198 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:07:39.204 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:38 smithi175 ceph-mon[36636]: pgmap v481: 129 pgs: 129 active+clean; 15 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.1 MiB/s wr, 85 op/s 2024-06-10T01:07:39.227 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:38 smithi007 ceph-mon[32398]: pgmap v481: 129 pgs: 129 active+clean; 15 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.1 MiB/s wr, 85 op/s 2024-06-10T01:07:39.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:38 smithi078 ceph-mon[36683]: pgmap v481: 129 pgs: 129 active+clean; 15 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.1 MiB/s wr, 85 op/s 2024-06-10T01:07:39.465 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:39.708 INFO:tasks.workunit.client.0.smithi007.stdout: 1 28947 8.70 MB/sec warmup 20 sec latency 10163.652 ms 2024-06-10T01:07:40.708 INFO:tasks.workunit.client.0.smithi007.stdout: 1 28947 8.29 MB/sec warmup 21 sec latency 11163.728 ms 2024-06-10T01:07:41.009 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:40 smithi007 ceph-mon[32398]: pgmap v482: 129 pgs: 129 active+clean; 15 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 822 KiB/s wr, 125 op/s 2024-06-10T01:07:41.204 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:40 smithi175 ceph-mon[36636]: pgmap v482: 129 pgs: 129 active+clean; 15 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 822 KiB/s wr, 125 op/s 2024-06-10T01:07:41.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.250+0000 7f71be28b700 1 -- 172.21.15.7:0/2792119788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b81073f0 msgr2=0x7f71b81077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:41.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.250+0000 7f71be28b700 1 --2- 172.21.15.7:0/2792119788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b81073f0 0x7f71b81077d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f71a8009990 tx=0x7f71a8009ca0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.250+0000 7f71be28b700 1 -- 172.21.15.7:0/2792119788 shutdown_connections 2024-06-10T01:07:41.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.250+0000 7f71be28b700 1 --2- 172.21.15.7:0/2792119788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71b8101d40 0x7f71b810e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.250+0000 7f71be28b700 1 --2- 172.21.15.7:0/2792119788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b81073f0 0x7f71b81077d0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.250+0000 7f71be28b700 1 --2- 172.21.15.7:0/2792119788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71b8101380 0x7f71b8101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.250+0000 7f71be28b700 1 -- 172.21.15.7:0/2792119788 >> 172.21.15.7:0/2792119788 conn(0x7f71b8078300 msgr2=0x7f71b8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:41.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.250+0000 7f71be28b700 1 -- 172.21.15.7:0/2792119788 shutdown_connections 2024-06-10T01:07:41.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.250+0000 7f71be28b700 1 -- 172.21.15.7:0/2792119788 wait complete. 2024-06-10T01:07:41.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.251+0000 7f71be28b700 1 Processor -- start 2024-06-10T01:07:41.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.251+0000 7f71be28b700 1 -- start start 2024-06-10T01:07:41.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.251+0000 7f71be28b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71b8101380 0x7f71b8103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:41.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.251+0000 7f71be28b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71b8101d40 0x7f71b81035a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:41.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.251+0000 7f71be28b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b81073f0 0x7f71b8103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:41.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.251+0000 7f71be28b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f71b8111690 con 0x7f71b81073f0 2024-06-10T01:07:41.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.251+0000 7f71be28b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71b8111510 con 0x7f71b8101d40 2024-06-10T01:07:41.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.251+0000 7f71be28b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f71b8111810 con 0x7f71b8101380 2024-06-10T01:07:41.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.251+0000 7f71bc828700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b81073f0 0x7f71b8103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:41.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.251+0000 7f71bc828700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b81073f0 0x7f71b8103ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35132/0 (socket says 172.21.15.7:35132) 2024-06-10T01:07:41.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.251+0000 7f71bc828700 1 -- 172.21.15.7:0/1042036239 learned_addr learned my addr 172.21.15.7:0/1042036239 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:41.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.251+0000 7f71b7fff700 1 --2- 172.21.15.7:0/1042036239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71b8101380 0x7f71b8103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:41.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.251+0000 7f71b77fe700 1 --2- 172.21.15.7:0/1042036239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71b8101d40 0x7f71b81035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:41.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.251+0000 7f71bc828700 1 -- 172.21.15.7:0/1042036239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71b8101380 msgr2=0x7f71b8103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:41.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.251+0000 7f71bc828700 1 --2- 172.21.15.7:0/1042036239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71b8101380 0x7f71b8103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.251+0000 7f71bc828700 1 -- 172.21.15.7:0/1042036239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71b8101d40 msgr2=0x7f71b81035a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:41.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.251+0000 7f71bc828700 1 --2- 172.21.15.7:0/1042036239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71b8101d40 0x7f71b81035a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.251+0000 7f71bc828700 1 -- 172.21.15.7:0/1042036239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b81a7190 con 0x7f71b81073f0 2024-06-10T01:07:41.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.251+0000 7f71b77fe700 1 --2- 172.21.15.7:0/1042036239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71b8101d40 0x7f71b81035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:07:41.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.252+0000 7f71bc828700 1 --2- 172.21.15.7:0/1042036239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b81073f0 0x7f71b8103ae0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f71b0007ee0 tx=0x7f71b000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:41.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.252+0000 7f71b57fa700 1 -- 172.21.15.7:0/1042036239 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b000dbd0 con 0x7f71b81073f0 2024-06-10T01:07:41.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.252+0000 7f71b57fa700 1 -- 172.21.15.7:0/1042036239 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71b000dd70 con 0x7f71b81073f0 2024-06-10T01:07:41.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.252+0000 7f71b57fa700 1 -- 172.21.15.7:0/1042036239 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b00133f0 con 0x7f71b81073f0 2024-06-10T01:07:41.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.252+0000 7f71be28b700 1 -- 172.21.15.7:0/1042036239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71b81a7480 con 0x7f71b81073f0 2024-06-10T01:07:41.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.252+0000 7f71be28b700 1 -- 172.21.15.7:0/1042036239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71b81a7970 con 0x7f71b81073f0 2024-06-10T01:07:41.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.254+0000 7f71b57fa700 1 -- 172.21.15.7:0/1042036239 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f71b001a020 con 0x7f71b81073f0 2024-06-10T01:07:41.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.255+0000 7f71b57fa700 1 --2- 172.21.15.7:0/1042036239 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f71a00a2ca0 0x7f71a00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:41.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.255+0000 7f71b7fff700 1 --2- 172.21.15.7:0/1042036239 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f71a00a2ca0 0x7f71a00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:41.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.255+0000 7f71b57fa700 1 -- 172.21.15.7:0/1042036239 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f71b00ceae0 con 0x7f71b81073f0 2024-06-10T01:07:41.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.255+0000 7f71b7fff700 1 --2- 172.21.15.7:0/1042036239 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f71a00a2ca0 0x7f71a00a5160 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f71a8005cd0 tx=0x7f71a8005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:41.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.256+0000 7f71c8b0eb80 1 Processor -- start 2024-06-10T01:07:41.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.256+0000 7f71c8b0eb80 1 -- start start 2024-06-10T01:07:41.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.256+0000 7f71c8b0eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cbac47fe0 0x558cbac483c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:41.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.256+0000 7f71c8b0eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cbac48900 0x558cbac422e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:41.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.256+0000 7f71c8b0eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558cbac42820 0x558cbac42c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:41.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.256+0000 7f71c8b0eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558cbab34fe0 con 0x558cbac48900 2024-06-10T01:07:41.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.256+0000 7f71c8b0eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558cbab938d0 con 0x558cbac47fe0 2024-06-10T01:07:41.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.256+0000 7f71c8b0eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558cbaafb4e0 con 0x558cbac42820 2024-06-10T01:07:41.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.256+0000 7f71b77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cbac48900 0x558cbac422e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:41.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.256+0000 7f71b77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cbac48900 0x558cbac422e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35142/0 (socket says 172.21.15.7:35142) 2024-06-10T01:07:41.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.256+0000 7f71b77fe700 1 -- 172.21.15.7:0/477922848 learned_addr learned my addr 172.21.15.7:0/477922848 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:41.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.256+0000 7f71b7fff700 1 --2- 172.21.15.7:0/477922848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cbac47fe0 0x558cbac483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:41.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.256+0000 7f71bc828700 1 --2- 172.21.15.7:0/477922848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558cbac42820 0x558cbac42c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:41.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.256+0000 7f71b77fe700 1 -- 172.21.15.7:0/477922848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558cbac42820 msgr2=0x558cbac42c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:41.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.256+0000 7f71b77fe700 1 --2- 172.21.15.7:0/477922848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558cbac42820 0x558cbac42c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.256+0000 7f71b77fe700 1 -- 172.21.15.7:0/477922848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cbac47fe0 msgr2=0x558cbac483c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:41.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.256+0000 7f71b77fe700 1 --2- 172.21.15.7:0/477922848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cbac47fe0 0x558cbac483c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.256+0000 7f71b77fe700 1 -- 172.21.15.7:0/477922848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cbaae2950 con 0x558cbac48900 2024-06-10T01:07:41.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.256+0000 7f71b7fff700 1 --2- 172.21.15.7:0/477922848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cbac47fe0 0x558cbac483c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:41.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.257+0000 7f71b77fe700 1 --2- 172.21.15.7:0/477922848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cbac48900 0x558cbac422e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00bba0 tx=0x7f71ac00beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50af3610fb543dd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:41.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.257+0000 7f719e7fc700 1 -- 172.21.15.7:0/477922848 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac00dd30 con 0x558cbac48900 2024-06-10T01:07:41.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.257+0000 7f719e7fc700 1 -- 172.21.15.7:0/477922848 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71ac017900 con 0x558cbac48900 2024-06-10T01:07:41.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.257+0000 7f719e7fc700 1 -- 172.21.15.7:0/477922848 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac00f960 con 0x558cbac48900 2024-06-10T01:07:41.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.257+0000 7f71c8b0eb80 1 -- 172.21.15.7:0/477922848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cbac48900 msgr2=0x558cbac422e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:41.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.257+0000 7f71c8b0eb80 1 --2- 172.21.15.7:0/477922848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cbac48900 0x558cbac422e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00bba0 tx=0x7f71ac00beb0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.257+0000 7f71c8b0eb80 1 -- 172.21.15.7:0/477922848 shutdown_connections 2024-06-10T01:07:41.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.257+0000 7f71c8b0eb80 1 --2- 172.21.15.7:0/477922848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558cbac42820 0x558cbac42c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.257+0000 7f71c8b0eb80 1 --2- 172.21.15.7:0/477922848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cbac47fe0 0x558cbac483c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.257+0000 7f71c8b0eb80 1 --2- 172.21.15.7:0/477922848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cbac48900 0x558cbac422e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.257+0000 7f71c8b0eb80 1 -- 172.21.15.7:0/477922848 >> 172.21.15.7:0/477922848 conn(0x558cbab6f100 msgr2=0x558cbac3d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:41.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.258+0000 7f71c8b0eb80 1 -- 172.21.15.7:0/477922848 shutdown_connections 2024-06-10T01:07:41.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.258+0000 7f71c8b0eb80 1 -- 172.21.15.7:0/477922848 wait complete. 2024-06-10T01:07:41.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.258+0000 7f71c8b0eb80 1 -- start start 2024-06-10T01:07:41.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.258+0000 7f71c8b0eb80 1 Processor -- start 2024-06-10T01:07:41.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.258+0000 7f71c8b0eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558cbac42820 0x558cbac4e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:41.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.258+0000 7f71c8b0eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cbac47fe0 0x558cbac4b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:41.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.258+0000 7f71c8b0eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cbac48900 0x558cbac4be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:41.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.258+0000 7f71c8b0eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558cbab34040 con 0x558cbac47fe0 2024-06-10T01:07:41.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.258+0000 7f71c8b0eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558cbab8f890 con 0x558cbac48900 2024-06-10T01:07:41.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.259+0000 7f71c8b0eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558cbac4f2e0 con 0x558cbac42820 2024-06-10T01:07:41.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.259+0000 7f71c8b0eb80 10 client.0 _ll_register_callbacks cb 0x558cbab29630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:41.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.259+0000 7f71b77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cbac47fe0 0x558cbac4b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:41.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.259+0000 7f71b77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cbac47fe0 0x558cbac4b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35156/0 (socket says 172.21.15.7:35156) 2024-06-10T01:07:41.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.259+0000 7f71b77fe700 1 -- 172.21.15.7:0/3799717372 learned_addr learned my addr 172.21.15.7:0/3799717372 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:41.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.259+0000 7f71bc828700 1 --2- 172.21.15.7:0/3799717372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cbac48900 0x558cbac4be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:41.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.259+0000 7f71b7fff700 1 --2- 172.21.15.7:0/3799717372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558cbac42820 0x558cbac4e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:41.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.259+0000 7f71b77fe700 1 -- 172.21.15.7:0/3799717372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558cbac42820 msgr2=0x558cbac4e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:41.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.259+0000 7f71b77fe700 1 --2- 172.21.15.7:0/3799717372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558cbac42820 0x558cbac4e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.259+0000 7f71b77fe700 1 -- 172.21.15.7:0/3799717372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cbac48900 msgr2=0x558cbac4be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:41.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.259+0000 7f71b77fe700 1 --2- 172.21.15.7:0/3799717372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cbac48900 0x558cbac4be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.259+0000 7f71b77fe700 1 -- 172.21.15.7:0/3799717372 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cbab597d0 con 0x558cbac47fe0 2024-06-10T01:07:41.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.259+0000 7f71b77fe700 1 --2- 172.21.15.7:0/3799717372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cbac47fe0 0x558cbac4b900 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00bb70 tx=0x7f71ac0176d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a332d5862ff7ff53 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:41.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.259+0000 7f71977fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:07:41.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.259+0000 7f71977fe700 1 -- 172.21.15.7:0/3799717372 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac026020 con 0x558cbac47fe0 2024-06-10T01:07:41.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.259+0000 7f71c8b0eb80 1 -- 172.21.15.7:0/3799717372 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71ac017900 con 0x558cbac47fe0 2024-06-10T01:07:41.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.259+0000 7f71977fe700 1 -- 172.21.15.7:0/3799717372 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f71ac017cc0 con 0x558cbac47fe0 2024-06-10T01:07:41.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.260+0000 7f71977fe700 1 -- 172.21.15.7:0/3799717372 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f71ac031050 con 0x558cbac47fe0 2024-06-10T01:07:41.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.260+0000 7f71c8b0eb80 10 client.15906 fetch_fsmap learned FSMap version 29 2024-06-10T01:07:41.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.260+0000 7f71c8b0eb80 1 -- 172.21.15.7:0/3799717372 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558cbac59a20 con 0x558cbac47fe0 2024-06-10T01:07:41.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.260+0000 7f71977fe700 1 -- 172.21.15.7:0/3799717372 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71ac017900 con 0x558cbac47fe0 2024-06-10T01:07:41.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.260+0000 7f71977fe700 1 -- 172.21.15.7:0/3799717372 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f71ac0216f0 con 0x558cbac47fe0 2024-06-10T01:07:41.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.260+0000 7f71c8b0eb80 10 client.15906 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:07:41.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.260+0000 7f71c8b0eb80 10 client.15906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:41.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.260+0000 7f71c8b0eb80 20 client.15906 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:41.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.260+0000 7f71c8b0eb80 1 --2- 172.21.15.7:0/3799717372 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558cbac5c000 0x558cbac5e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:41.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.260+0000 7f71c8b0eb80 4 client.15906 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:07:41.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.260+0000 7f71c8b0eb80 1 -- 172.21.15.7:0/3799717372 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cbaba0dd0 con 0x558cbac5c000 2024-06-10T01:07:41.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.260+0000 7f71b7fff700 1 --2- 172.21.15.7:0/3799717372 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558cbac5c000 0x558cbac5e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:41.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.261+0000 7f71b7fff700 1 --2- 172.21.15.7:0/3799717372 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558cbac5c000 0x558cbac5e3f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3060307cb49e6e68 server_cookie=31666a9030233398 in_seq=0 out_seq=0 2024-06-10T01:07:41.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.261+0000 7f71977fe700 10 client.15906 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:41.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.261+0000 7f71977fe700 1 -- 172.21.15.7:0/3799717372 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558cbaba0dd0 con 0x558cbac5c000 2024-06-10T01:07:41.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.261+0000 7f71977fe700 10 client.15906 handle_command_reply: tid=0 2024-06-10T01:07:41.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.262+0000 7f71c8b0eb80 1 client.15906 shutdown 2024-06-10T01:07:41.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.262+0000 7f71c8b0eb80 1 -- 172.21.15.7:0/3799717372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cbac47fe0 msgr2=0x558cbac4b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:41.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.262+0000 7f71c8b0eb80 1 --2- 172.21.15.7:0/3799717372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cbac47fe0 0x558cbac4b900 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00bb70 tx=0x7f71ac0176d0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.262+0000 7f71c8b0eb80 1 -- 172.21.15.7:0/3799717372 shutdown_connections 2024-06-10T01:07:41.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.262+0000 7f71c8b0eb80 1 --2- 172.21.15.7:0/3799717372 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558cbac5c000 0x558cbac5e3f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.262+0000 7f71c8b0eb80 1 --2- 172.21.15.7:0/3799717372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cbac48900 0x558cbac4be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.262+0000 7f71977fe700 0 client.15906 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:41.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.262+0000 7f71c8b0eb80 1 --2- 172.21.15.7:0/3799717372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558cbac42820 0x558cbac4e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.262+0000 7f71c8b0eb80 1 --2- 172.21.15.7:0/3799717372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cbac47fe0 0x558cbac4b900 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.262+0000 7f71c8b0eb80 1 -- 172.21.15.7:0/3799717372 >> 172.21.15.7:0/3799717372 conn(0x558cbab6f100 msgr2=0x558cbac3c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:41.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.262+0000 7f71c8b0eb80 1 -- 172.21.15.7:0/3799717372 shutdown_connections 2024-06-10T01:07:41.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.262+0000 7f71c8b0eb80 1 -- 172.21.15.7:0/3799717372 wait complete. 2024-06-10T01:07:41.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.262+0000 7f71c8b0eb80 20 client.15906 trim_cache size 0 max 16384 2024-06-10T01:07:41.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.283+0000 7f71c8b0eb80 1 Processor -- start 2024-06-10T01:07:41.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.283+0000 7f71c8b0eb80 1 -- start start 2024-06-10T01:07:41.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.283+0000 7f71c8b0eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cbac42820 0x558cbac4a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:41.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.283+0000 7f71c8b0eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558cbac47fe0 0x558cbac3cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:41.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.283+0000 7f71c8b0eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cbac48900 0x558cbac3d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:41.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.283+0000 7f71c8b0eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f71b81b2d60 con 0x558cbac42820 2024-06-10T01:07:41.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.283+0000 7f71c8b0eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71b81b05f0 con 0x558cbac48900 2024-06-10T01:07:41.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.283+0000 7f71c8b0eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f71b81ae440 con 0x558cbac47fe0 2024-06-10T01:07:41.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.283+0000 7f71b7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cbac42820 0x558cbac4a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:41.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.283+0000 7f71b7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cbac42820 0x558cbac4a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35164/0 (socket says 172.21.15.7:35164) 2024-06-10T01:07:41.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.283+0000 7f71b7fff700 1 -- 172.21.15.7:0/3931801338 learned_addr learned my addr 172.21.15.7:0/3931801338 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:41.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.283+0000 7f71b77fe700 1 --2- 172.21.15.7:0/3931801338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558cbac47fe0 0x558cbac3cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:41.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.283+0000 7f71b7fff700 1 -- 172.21.15.7:0/3931801338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558cbac47fe0 msgr2=0x558cbac3cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:41.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.283+0000 7f71b7fff700 1 --2- 172.21.15.7:0/3931801338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558cbac47fe0 0x558cbac3cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.283+0000 7f71bc828700 1 --2- 172.21.15.7:0/3931801338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cbac48900 0x558cbac3d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:41.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.283+0000 7f71b7fff700 1 -- 172.21.15.7:0/3931801338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cbac48900 msgr2=0x558cbac3d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:41.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.283+0000 7f71b7fff700 1 --2- 172.21.15.7:0/3931801338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cbac48900 0x558cbac3d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.283+0000 7f71b7fff700 1 -- 172.21.15.7:0/3931801338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cbac4ee00 con 0x558cbac42820 2024-06-10T01:07:41.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.283+0000 7f71b77fe700 1 --2- 172.21.15.7:0/3931801338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558cbac47fe0 0x558cbac3cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:07:41.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.284+0000 7f71b7fff700 1 --2- 172.21.15.7:0/3931801338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cbac42820 0x558cbac4a900 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f71a8000c00 tx=0x7f71a801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1cdf5adaa437fd0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:41.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.284+0000 7f719dffb700 1 -- 172.21.15.7:0/3931801338 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a800b7d0 con 0x558cbac42820 2024-06-10T01:07:41.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.284+0000 7f719dffb700 1 -- 172.21.15.7:0/3931801338 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71a800be50 con 0x558cbac42820 2024-06-10T01:07:41.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.284+0000 7f719dffb700 1 -- 172.21.15.7:0/3931801338 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a8024460 con 0x558cbac42820 2024-06-10T01:07:41.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.285+0000 7f71c8b0eb80 1 -- 172.21.15.7:0/3931801338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cbac42820 msgr2=0x558cbac4a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:41.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.285+0000 7f71c8b0eb80 1 --2- 172.21.15.7:0/3931801338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cbac42820 0x558cbac4a900 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f71a8000c00 tx=0x7f71a801e000 comp rx=0 tx=0).stop 2024-06-10T01:07:41.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.285+0000 7f71c8b0eb80 1 -- 172.21.15.7:0/3931801338 shutdown_connections 2024-06-10T01:07:41.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.285+0000 7f71c8b0eb80 1 --2- 172.21.15.7:0/3931801338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cbac48900 0x558cbac3d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.285+0000 7f71c8b0eb80 1 --2- 172.21.15.7:0/3931801338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cbac42820 0x558cbac4a900 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.285+0000 7f71c8b0eb80 1 --2- 172.21.15.7:0/3931801338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558cbac47fe0 0x558cbac3cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.285+0000 7f71c8b0eb80 1 -- 172.21.15.7:0/3931801338 >> 172.21.15.7:0/3931801338 conn(0x558cbab6f100 msgr2=0x558cbac4a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:41.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.285+0000 7f71c8b0eb80 1 -- 172.21.15.7:0/3931801338 shutdown_connections 2024-06-10T01:07:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.285+0000 7f71c8b0eb80 1 -- 172.21.15.7:0/3931801338 wait complete. 2024-06-10T01:07:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.285+0000 7f71c8b0eb80 1 -- start start 2024-06-10T01:07:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.285+0000 7f71c8b0eb80 1 Processor -- start 2024-06-10T01:07:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.286+0000 7f71c8b0eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cbac42820 0x558cbac54890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.286+0000 7f71c8b0eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558cbac47fe0 0x558cbac54dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.286+0000 7f71c8b0eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cbac48900 0x558cbac515a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.286+0000 7f71c8b0eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f71b81aeab0 con 0x558cbac48900 2024-06-10T01:07:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.286+0000 7f71c8b0eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71b81aff20 con 0x558cbac42820 2024-06-10T01:07:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.286+0000 7f71c8b0eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f71b81af180 con 0x558cbac47fe0 2024-06-10T01:07:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.286+0000 7f71c8b0eb80 10 client.0 _ll_register_callbacks cb 0x558cbab29630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.286+0000 7f71bc828700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cbac48900 0x558cbac515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.286+0000 7f71bc828700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cbac48900 0x558cbac515a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35166/0 (socket says 172.21.15.7:35166) 2024-06-10T01:07:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.286+0000 7f71bc828700 1 -- 172.21.15.7:0/3257980696 learned_addr learned my addr 172.21.15.7:0/3257980696 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.286+0000 7f71b7fff700 1 --2- 172.21.15.7:0/3257980696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cbac42820 0x558cbac54890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.286+0000 7f71b77fe700 1 --2- 172.21.15.7:0/3257980696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558cbac47fe0 0x558cbac54dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.286+0000 7f71bc828700 1 -- 172.21.15.7:0/3257980696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558cbac47fe0 msgr2=0x558cbac54dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.286+0000 7f71bc828700 1 --2- 172.21.15.7:0/3257980696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558cbac47fe0 0x558cbac54dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.286+0000 7f71bc828700 1 -- 172.21.15.7:0/3257980696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cbac42820 msgr2=0x558cbac54890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.286+0000 7f71bc828700 1 --2- 172.21.15.7:0/3257980696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cbac42820 0x558cbac54890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.286+0000 7f71bc828700 1 -- 172.21.15.7:0/3257980696 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cbac51dc0 con 0x558cbac48900 2024-06-10T01:07:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.286+0000 7f71b7fff700 1 --2- 172.21.15.7:0/3257980696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cbac42820 0x558cbac54890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.286+0000 7f71bc828700 1 --2- 172.21.15.7:0/3257980696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cbac48900 0x558cbac515a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f71b000d7b0 tx=0x7f71b00bb0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673aa453d90becc4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.286+0000 7f71977fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:07:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.286+0000 7f71977fe700 1 -- 172.21.15.7:0/3257980696 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b009afc0 con 0x558cbac48900 2024-06-10T01:07:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.287+0000 7f71977fe700 1 -- 172.21.15.7:0/3257980696 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f71b009b220 con 0x558cbac48900 2024-06-10T01:07:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.287+0000 7f71c8b0eb80 1 -- 172.21.15.7:0/3257980696 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71a800be50 con 0x558cbac48900 2024-06-10T01:07:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.287+0000 7f71977fe700 1 -- 172.21.15.7:0/3257980696 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f71b009f370 con 0x558cbac48900 2024-06-10T01:07:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.287+0000 7f71c8b0eb80 10 client.15912 fetch_fsmap learned FSMap version 29 2024-06-10T01:07:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.287+0000 7f71c8b0eb80 1 -- 172.21.15.7:0/3257980696 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558cbacc7590 con 0x558cbac48900 2024-06-10T01:07:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.287+0000 7f71977fe700 1 -- 172.21.15.7:0/3257980696 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71a800be50 con 0x558cbac48900 2024-06-10T01:07:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.287+0000 7f71977fe700 1 -- 172.21.15.7:0/3257980696 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f71b009d070 con 0x558cbac48900 2024-06-10T01:07:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.287+0000 7f71c8b0eb80 10 client.15912 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:07:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.287+0000 7f71c8b0eb80 10 client.15912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.287+0000 7f71c8b0eb80 20 client.15912 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.287+0000 7f71c8b0eb80 1 --2- 172.21.15.7:0/3257980696 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558cbac5c000 0x558cbacc9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.287+0000 7f71c8b0eb80 4 client.15912 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:07:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.287+0000 7f71c8b0eb80 1 -- 172.21.15.7:0/3257980696 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558cbab9b350 con 0x558cbac5c000 2024-06-10T01:07:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.287+0000 7f71b7fff700 1 --2- 172.21.15.7:0/3257980696 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558cbac5c000 0x558cbacc9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.288+0000 7f71b7fff700 1 --2- 172.21.15.7:0/3257980696 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558cbac5c000 0x558cbacc9ce0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbb7d19b13689b51 server_cookie=a26473f2d8614f5c in_seq=0 out_seq=0 2024-06-10T01:07:41.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.288+0000 7f71977fe700 10 client.15912 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:41.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.289+0000 7f71977fe700 1 -- 172.21.15.7:0/3257980696 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558cbab9b350 con 0x558cbac5c000 2024-06-10T01:07:41.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.289+0000 7f71977fe700 10 client.15912 handle_command_reply: tid=0 2024-06-10T01:07:41.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.289+0000 7f71c8b0eb80 1 client.15912 shutdown 2024-06-10T01:07:41.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.290+0000 7f71c8b0eb80 1 -- 172.21.15.7:0/3257980696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cbac48900 msgr2=0x558cbac515a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:41.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.290+0000 7f71c8b0eb80 1 --2- 172.21.15.7:0/3257980696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cbac48900 0x558cbac515a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f71b000d7b0 tx=0x7f71b00bb0b0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.290+0000 7f71c8b0eb80 1 -- 172.21.15.7:0/3257980696 shutdown_connections 2024-06-10T01:07:41.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.290+0000 7f71c8b0eb80 1 --2- 172.21.15.7:0/3257980696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cbac48900 0x558cbac515a0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.290+0000 7f71c8b0eb80 1 --2- 172.21.15.7:0/3257980696 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558cbac5c000 0x558cbacc9ce0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.290+0000 7f71c8b0eb80 1 --2- 172.21.15.7:0/3257980696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cbac42820 0x558cbac54890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.290+0000 7f71c8b0eb80 1 --2- 172.21.15.7:0/3257980696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558cbac47fe0 0x558cbac54dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.290+0000 7f71c8b0eb80 1 -- 172.21.15.7:0/3257980696 >> 172.21.15.7:0/3257980696 conn(0x558cbab6f100 msgr2=0x558cbac493f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:41.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.290+0000 7f71977fe700 0 client.15912 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:41.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.291+0000 7f71c8b0eb80 1 -- 172.21.15.7:0/3257980696 shutdown_connections 2024-06-10T01:07:41.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.291+0000 7f71c8b0eb80 1 -- 172.21.15.7:0/3257980696 wait complete. 2024-06-10T01:07:41.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.291+0000 7f71c8b0eb80 20 client.15912 trim_cache size 0 max 16384 2024-06-10T01:07:41.290 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:07:41.291 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-10T01:07:41.291 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "8b64351a-aeb5-47c0-91da-ec2a284d70f8", 2024-06-10T01:07:41.291 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-10T01:07:41.291 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:07:41.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.292+0000 7f71be28b700 1 -- 172.21.15.7:0/1042036239 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f71a00a2ca0 msgr2=0x7f71a00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:41.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.292+0000 7f71be28b700 1 --2- 172.21.15.7:0/1042036239 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f71a00a2ca0 0x7f71a00a5160 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f71a8005cd0 tx=0x7f71a8005c40 comp rx=0 tx=0).stop 2024-06-10T01:07:41.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.292+0000 7f71be28b700 1 -- 172.21.15.7:0/1042036239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b81073f0 msgr2=0x7f71b8103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:41.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.292+0000 7f71be28b700 1 --2- 172.21.15.7:0/1042036239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b81073f0 0x7f71b8103ae0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f71b0007ee0 tx=0x7f71b000d3b0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.292+0000 7f71be28b700 1 -- 172.21.15.7:0/1042036239 shutdown_connections 2024-06-10T01:07:41.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.292+0000 7f71be28b700 1 --2- 172.21.15.7:0/1042036239 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f71a00a2ca0 0x7f71a00a5160 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.292+0000 7f71be28b700 1 --2- 172.21.15.7:0/1042036239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b81073f0 0x7f71b8103ae0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.292+0000 7f71be28b700 1 --2- 172.21.15.7:0/1042036239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71b8101380 0x7f71b8103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.292+0000 7f71be28b700 1 --2- 172.21.15.7:0/1042036239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71b8101d40 0x7f71b81035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:41.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.292+0000 7f71be28b700 1 -- 172.21.15.7:0/1042036239 >> 172.21.15.7:0/1042036239 conn(0x7f71b8078300 msgr2=0x7f71b81006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:41.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.293+0000 7f71be28b700 1 -- 172.21.15.7:0/1042036239 shutdown_connections 2024-06-10T01:07:41.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:41.293+0000 7f71be28b700 1 -- 172.21.15.7:0/1042036239 wait complete. 2024-06-10T01:07:41.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:40 smithi078 ceph-mon[36683]: pgmap v482: 129 pgs: 129 active+clean; 15 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 822 KiB/s wr, 125 op/s 2024-06-10T01:07:41.708 INFO:tasks.workunit.client.0.smithi007.stdout: 1 28947 7.91 MB/sec warmup 22 sec latency 12163.797 ms 2024-06-10T01:07:42.134 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:07:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:42.708 INFO:tasks.workunit.client.0.smithi007.stdout: 1 28947 7.56 MB/sec warmup 23 sec latency 13163.879 ms 2024-06-10T01:07:43.204 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:42 smithi175 ceph-mon[36636]: scrub queued for path: / 2024-06-10T01:07:43.204 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:42 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:07:43.204 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:42 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:07:43.204 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:42 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:07:43.204 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:42 smithi175 ceph-mon[36636]: pgmap v483: 129 pgs: 129 active+clean; 11 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 774 KiB/s wr, 146 op/s 2024-06-10T01:07:43.204 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:42 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:07:43.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:42 smithi007 ceph-mon[32398]: scrub queued for path: / 2024-06-10T01:07:43.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:42 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:07:43.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:42 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:07:43.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:42 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:07:43.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:42 smithi007 ceph-mon[32398]: pgmap v483: 129 pgs: 129 active+clean; 11 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 774 KiB/s wr, 146 op/s 2024-06-10T01:07:43.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:42 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:07:43.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:42 smithi078 ceph-mon[36683]: scrub queued for path: / 2024-06-10T01:07:43.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:42 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:07:43.331 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:42 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:07:43.331 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:42 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:07:43.331 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:42 smithi078 ceph-mon[36683]: pgmap v483: 129 pgs: 129 active+clean; 11 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 774 KiB/s wr, 146 op/s 2024-06-10T01:07:43.331 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:42 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:07:43.708 INFO:tasks.workunit.client.0.smithi007.stdout: 1 28947 7.25 MB/sec warmup 24 sec latency 14163.935 ms 2024-06-10T01:07:44.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.198+0000 7f1e05c96700 1 -- 172.21.15.7:0/2711622319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e00107550 msgr2=0x7f1e00107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:44.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.198+0000 7f1e05c96700 1 --2- 172.21.15.7:0/2711622319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e00107550 0x7f1e00107930 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009990 tx=0x7f1df0009ca0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.198+0000 7f1e05c96700 1 -- 172.21.15.7:0/2711622319 shutdown_connections 2024-06-10T01:07:44.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.198+0000 7f1e05c96700 1 --2- 172.21.15.7:0/2711622319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e001017f0 0x7f1e0010f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.198+0000 7f1e05c96700 1 --2- 172.21.15.7:0/2711622319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e00107550 0x7f1e00107930 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.198+0000 7f1e05c96700 1 --2- 172.21.15.7:0/2711622319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e00100e30 0x7f1e001012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.198+0000 7f1e05c96700 1 -- 172.21.15.7:0/2711622319 >> 172.21.15.7:0/2711622319 conn(0x7f1e000766b0 msgr2=0x7f1e00076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:44.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.198+0000 7f1e05c96700 1 -- 172.21.15.7:0/2711622319 shutdown_connections 2024-06-10T01:07:44.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.198+0000 7f1e05c96700 1 -- 172.21.15.7:0/2711622319 wait complete. 2024-06-10T01:07:44.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.199+0000 7f1e05c96700 1 Processor -- start 2024-06-10T01:07:44.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.199+0000 7f1e05c96700 1 -- start start 2024-06-10T01:07:44.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.199+0000 7f1e05c96700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e00100e30 0x7f1e00102010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:44.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.199+0000 7f1e05c96700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e001017f0 0x7f1e00102550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:44.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.199+0000 7f1e05c96700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e00107550 0x7f1e00102a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:44.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.199+0000 7f1e05c96700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1e00112790 con 0x7f1e00107550 2024-06-10T01:07:44.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.199+0000 7f1e05c96700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1e00112610 con 0x7f1e00100e30 2024-06-10T01:07:44.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.199+0000 7f1e05c96700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1e00112910 con 0x7f1e001017f0 2024-06-10T01:07:44.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.199+0000 7f1dfffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e00107550 0x7f1e00102a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:44.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.199+0000 7f1dfffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e00107550 0x7f1e00102a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46598/0 (socket says 172.21.15.7:46598) 2024-06-10T01:07:44.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.199+0000 7f1dfffff700 1 -- 172.21.15.7:0/1567665033 learned_addr learned my addr 172.21.15.7:0/1567665033 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:44.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.200+0000 7f1dfeffd700 1 --2- 172.21.15.7:0/1567665033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e001017f0 0x7f1e00102550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:44.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.200+0000 7f1dff7fe700 1 --2- 172.21.15.7:0/1567665033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e00100e30 0x7f1e00102010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:44.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.200+0000 7f1dfffff700 1 -- 172.21.15.7:0/1567665033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e001017f0 msgr2=0x7f1e00102550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:44.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.200+0000 7f1dfffff700 1 --2- 172.21.15.7:0/1567665033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e001017f0 0x7f1e00102550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.200+0000 7f1dfffff700 1 -- 172.21.15.7:0/1567665033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e00100e30 msgr2=0x7f1e00102010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:44.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.200+0000 7f1dfffff700 1 --2- 172.21.15.7:0/1567665033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e00100e30 0x7f1e00102010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.200+0000 7f1dfffff700 1 -- 172.21.15.7:0/1567665033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e001a7150 con 0x7f1e00107550 2024-06-10T01:07:44.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.200+0000 7f1dff7fe700 1 --2- 172.21.15.7:0/1567665033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e00100e30 0x7f1e00102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:07:44.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.200+0000 7f1dfeffd700 1 --2- 172.21.15.7:0/1567665033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e001017f0 0x7f1e00102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:07:44.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.200+0000 7f1dfffff700 1 --2- 172.21.15.7:0/1567665033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e00107550 0x7f1e00102a90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1df8007ee0 tx=0x7f1df800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:44.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.200+0000 7f1dfcff9700 1 -- 172.21.15.7:0/1567665033 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df800dbd0 con 0x7f1e00107550 2024-06-10T01:07:44.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.200+0000 7f1dfcff9700 1 -- 172.21.15.7:0/1567665033 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1df800dd70 con 0x7f1e00107550 2024-06-10T01:07:44.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.200+0000 7f1e05c96700 1 -- 172.21.15.7:0/1567665033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e001a7440 con 0x7f1e00107550 2024-06-10T01:07:44.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.200+0000 7f1dfcff9700 1 -- 172.21.15.7:0/1567665033 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df80133f0 con 0x7f1e00107550 2024-06-10T01:07:44.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.200+0000 7f1e05c96700 1 -- 172.21.15.7:0/1567665033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e001a7930 con 0x7f1e00107550 2024-06-10T01:07:44.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.203+0000 7f1dfcff9700 1 -- 172.21.15.7:0/1567665033 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f1df801a020 con 0x7f1e00107550 2024-06-10T01:07:44.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.203+0000 7f1dfcff9700 1 --2- 172.21.15.7:0/1567665033 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1de80a2c20 0x7f1de80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:44.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.203+0000 7f1dfcff9700 1 -- 172.21.15.7:0/1567665033 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f1df80cebb0 con 0x7f1e00107550 2024-06-10T01:07:44.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.203+0000 7f1dff7fe700 1 --2- 172.21.15.7:0/1567665033 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1de80a2c20 0x7f1de80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:44.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.204+0000 7f1dff7fe700 1 --2- 172.21.15.7:0/1567665033 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1de80a2c20 0x7f1de80a50e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1df0000c00 tx=0x7f1df0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:44.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.204+0000 7f1e10519b80 1 Processor -- start 2024-06-10T01:07:44.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.204+0000 7f1e10519b80 1 -- start start 2024-06-10T01:07:44.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.204+0000 7f1e10519b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f28cc81640 0x55f28cc81a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:44.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.204+0000 7f1e10519b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f28cc81f60 0x55f28cc7b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:44.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.204+0000 7f1e10519b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f28cc7bf20 0x55f28cc7c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:44.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.204+0000 7f1e10519b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f28cbcd810 con 0x55f28cc81f60 2024-06-10T01:07:44.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.204+0000 7f1e10519b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f28cb75720 con 0x55f28cc7bf20 2024-06-10T01:07:44.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.204+0000 7f1e10519b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f28cbc97d0 con 0x55f28cc81640 2024-06-10T01:07:44.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.204+0000 7f1dfeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f28cc81f60 0x55f28cc7b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:44.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.204+0000 7f1dfeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f28cc81f60 0x55f28cc7b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46614/0 (socket says 172.21.15.7:46614) 2024-06-10T01:07:44.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.204+0000 7f1dfeffd700 1 -- 172.21.15.7:0/1010718383 learned_addr learned my addr 172.21.15.7:0/1010718383 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:44.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.205+0000 7f1dff7fe700 1 --2- 172.21.15.7:0/1010718383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f28cc81640 0x55f28cc81a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:44.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.205+0000 7f1dfffff700 1 --2- 172.21.15.7:0/1010718383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f28cc7bf20 0x55f28cc7c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:44.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.205+0000 7f1dfeffd700 1 -- 172.21.15.7:0/1010718383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f28cc81640 msgr2=0x55f28cc81a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:44.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.205+0000 7f1dfeffd700 1 --2- 172.21.15.7:0/1010718383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f28cc81640 0x55f28cc81a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.205+0000 7f1dfeffd700 1 -- 172.21.15.7:0/1010718383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f28cc7bf20 msgr2=0x55f28cc7c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:44.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.205+0000 7f1dfeffd700 1 --2- 172.21.15.7:0/1010718383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f28cc7bf20 0x55f28cc7c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.205+0000 7f1dfeffd700 1 -- 172.21.15.7:0/1010718383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f28cb6bf50 con 0x55f28cc81f60 2024-06-10T01:07:44.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.205+0000 7f1dff7fe700 1 --2- 172.21.15.7:0/1010718383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f28cc81640 0x55f28cc81a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:44.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.205+0000 7f1dfffff700 1 --2- 172.21.15.7:0/1010718383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f28cc7bf20 0x55f28cc7c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:44.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.205+0000 7f1dfeffd700 1 --2- 172.21.15.7:0/1010718383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f28cc81f60 0x55f28cc7b9e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1df4009c70 tx=0x7f1df4009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0dfb3f8f9c6ad2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:44.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.205+0000 7f1de5ffb700 1 -- 172.21.15.7:0/1010718383 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4016070 con 0x55f28cc81f60 2024-06-10T01:07:44.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.205+0000 7f1de5ffb700 1 -- 172.21.15.7:0/1010718383 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1df4004d10 con 0x55f28cc81f60 2024-06-10T01:07:44.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.205+0000 7f1de5ffb700 1 -- 172.21.15.7:0/1010718383 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4005620 con 0x55f28cc81f60 2024-06-10T01:07:44.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.206+0000 7f1e10519b80 1 -- 172.21.15.7:0/1010718383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f28cc81f60 msgr2=0x55f28cc7b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:44.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.206+0000 7f1e10519b80 1 --2- 172.21.15.7:0/1010718383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f28cc81f60 0x55f28cc7b9e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1df4009c70 tx=0x7f1df4009f80 comp rx=0 tx=0).stop 2024-06-10T01:07:44.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.206+0000 7f1e10519b80 1 -- 172.21.15.7:0/1010718383 shutdown_connections 2024-06-10T01:07:44.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.206+0000 7f1e10519b80 1 --2- 172.21.15.7:0/1010718383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f28cc7bf20 0x55f28cc7c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.206+0000 7f1e10519b80 1 --2- 172.21.15.7:0/1010718383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f28cc81640 0x55f28cc81a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.206+0000 7f1e10519b80 1 --2- 172.21.15.7:0/1010718383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f28cc81f60 0x55f28cc7b9e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.206+0000 7f1e10519b80 1 -- 172.21.15.7:0/1010718383 >> 172.21.15.7:0/1010718383 conn(0x55f28cba6cc0 msgr2=0x55f28cc76850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:44.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.206+0000 7f1e10519b80 1 -- 172.21.15.7:0/1010718383 shutdown_connections 2024-06-10T01:07:44.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.206+0000 7f1e10519b80 1 -- 172.21.15.7:0/1010718383 wait complete. 2024-06-10T01:07:44.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.206+0000 7f1e10519b80 1 -- start start 2024-06-10T01:07:44.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.207+0000 7f1e10519b80 1 Processor -- start 2024-06-10T01:07:44.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.207+0000 7f1e10519b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f28cc7bf20 0x55f28cc87cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:44.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.207+0000 7f1e10519b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f28cc81640 0x55f28cc84fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:44.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.207+0000 7f1e10519b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f28cc81f60 0x55f28cc854f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:44.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.207+0000 7f1e10519b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f28cc889d0 con 0x55f28cc81f60 2024-06-10T01:07:44.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.207+0000 7f1e10519b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f28cb6dbf0 con 0x55f28cc7bf20 2024-06-10T01:07:44.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.207+0000 7f1e10519b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f28cc88b50 con 0x55f28cc81640 2024-06-10T01:07:44.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.207+0000 7f1e10519b80 10 client.0 _ll_register_callbacks cb 0x55f28cb67920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:44.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.207+0000 7f1dfffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f28cc81f60 0x55f28cc854f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:44.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.207+0000 7f1dfffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f28cc81f60 0x55f28cc854f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46618/0 (socket says 172.21.15.7:46618) 2024-06-10T01:07:44.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.207+0000 7f1dfffff700 1 -- 172.21.15.7:0/3537056548 learned_addr learned my addr 172.21.15.7:0/3537056548 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:44.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.207+0000 7f1dff7fe700 1 --2- 172.21.15.7:0/3537056548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f28cc7bf20 0x55f28cc87cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:44.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.207+0000 7f1dfeffd700 1 --2- 172.21.15.7:0/3537056548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f28cc81640 0x55f28cc84fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:44.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.207+0000 7f1dfffff700 1 -- 172.21.15.7:0/3537056548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f28cc81640 msgr2=0x55f28cc84fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:44.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.207+0000 7f1dfffff700 1 --2- 172.21.15.7:0/3537056548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f28cc81640 0x55f28cc84fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.207+0000 7f1dfffff700 1 -- 172.21.15.7:0/3537056548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f28cc7bf20 msgr2=0x55f28cc87cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:44.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.207+0000 7f1dfffff700 1 --2- 172.21.15.7:0/3537056548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f28cc7bf20 0x55f28cc87cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.207+0000 7f1dfffff700 1 -- 172.21.15.7:0/3537056548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f28cb1c8d0 con 0x55f28cc81f60 2024-06-10T01:07:44.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.208+0000 7f1dfffff700 1 --2- 172.21.15.7:0/3537056548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f28cc81f60 0x55f28cc854f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1df809ad90 tx=0x7f1df8009410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=514790ac9db97155 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:44.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.208+0000 7f1ddeffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:07:44.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.208+0000 7f1ddeffd700 1 -- 172.21.15.7:0/3537056548 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df80b4060 con 0x55f28cc81f60 2024-06-10T01:07:44.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.208+0000 7f1ddeffd700 1 -- 172.21.15.7:0/3537056548 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f1df8004c40 con 0x55f28cc81f60 2024-06-10T01:07:44.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.208+0000 7f1e10519b80 1 -- 172.21.15.7:0/3537056548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1df4004d10 con 0x55f28cc81f60 2024-06-10T01:07:44.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.208+0000 7f1ddeffd700 1 -- 172.21.15.7:0/3537056548 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1df809f440 con 0x55f28cc81f60 2024-06-10T01:07:44.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.208+0000 7f1e10519b80 10 client.15924 fetch_fsmap learned FSMap version 29 2024-06-10T01:07:44.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.208+0000 7f1e10519b80 1 -- 172.21.15.7:0/3537056548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f28cc93310 con 0x55f28cc81f60 2024-06-10T01:07:44.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.208+0000 7f1ddeffd700 1 -- 172.21.15.7:0/3537056548 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1df4004d10 con 0x55f28cc81f60 2024-06-10T01:07:44.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.208+0000 7f1ddeffd700 1 -- 172.21.15.7:0/3537056548 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f1df80b6070 con 0x55f28cc81f60 2024-06-10T01:07:44.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.208+0000 7f1e10519b80 10 client.15924 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:07:44.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.208+0000 7f1e10519b80 10 client.15924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:44.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.208+0000 7f1e10519b80 20 client.15924 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:44.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.208+0000 7f1e10519b80 1 --2- 172.21.15.7:0/3537056548 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f28cc95950 0x55f28cc97d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:44.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.208+0000 7f1e10519b80 4 client.15924 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:07:44.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.208+0000 7f1e10519b80 1 -- 172.21.15.7:0/3537056548 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f28cbdad10 con 0x55f28cc95950 2024-06-10T01:07:44.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.209+0000 7f1dff7fe700 1 --2- 172.21.15.7:0/3537056548 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f28cc95950 0x55f28cc97d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:44.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.209+0000 7f1dff7fe700 1 --2- 172.21.15.7:0/3537056548 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f28cc95950 0x55f28cc97d40 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6d3ff6d34ff54d2 server_cookie=74f9c7d1217575a5 in_seq=0 out_seq=0 2024-06-10T01:07:44.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.209+0000 7f1ddeffd700 10 client.15924 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:44.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.210+0000 7f1ddeffd700 1 -- 172.21.15.7:0/3537056548 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f28cbdad10 con 0x55f28cc95950 2024-06-10T01:07:44.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.210+0000 7f1ddeffd700 10 client.15924 handle_command_reply: tid=0 2024-06-10T01:07:44.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.210+0000 7f1e10519b80 1 client.15924 shutdown 2024-06-10T01:07:44.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.210+0000 7f1e10519b80 1 -- 172.21.15.7:0/3537056548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f28cc81f60 msgr2=0x55f28cc854f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:44.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.210+0000 7f1e10519b80 1 --2- 172.21.15.7:0/3537056548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f28cc81f60 0x55f28cc854f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1df809ad90 tx=0x7f1df8009410 comp rx=0 tx=0).stop 2024-06-10T01:07:44.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.210+0000 7f1e10519b80 1 -- 172.21.15.7:0/3537056548 shutdown_connections 2024-06-10T01:07:44.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.210+0000 7f1e10519b80 1 --2- 172.21.15.7:0/3537056548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f28cc81f60 0x55f28cc854f0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.210+0000 7f1e10519b80 1 --2- 172.21.15.7:0/3537056548 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f28cc95950 0x55f28cc97d40 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.210+0000 7f1ddeffd700 0 client.15924 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:44.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.210+0000 7f1e10519b80 1 --2- 172.21.15.7:0/3537056548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f28cc7bf20 0x55f28cc87cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.210+0000 7f1e10519b80 1 --2- 172.21.15.7:0/3537056548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f28cc81640 0x55f28cc84fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.210+0000 7f1e10519b80 1 -- 172.21.15.7:0/3537056548 >> 172.21.15.7:0/3537056548 conn(0x55f28cba6cc0 msgr2=0x55f28cc7a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:44.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.210+0000 7f1e10519b80 1 -- 172.21.15.7:0/3537056548 shutdown_connections 2024-06-10T01:07:44.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.210+0000 7f1e10519b80 1 -- 172.21.15.7:0/3537056548 wait complete. 2024-06-10T01:07:44.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.210+0000 7f1e10519b80 20 client.15924 trim_cache size 0 max 16384 2024-06-10T01:07:44.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.231+0000 7f1e10519b80 1 Processor -- start 2024-06-10T01:07:44.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.231+0000 7f1e10519b80 1 -- start start 2024-06-10T01:07:44.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.231+0000 7f1e10519b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f28cc7bf20 0x55f28cc7b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:44.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.231+0000 7f1e10519b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f28cc81640 0x55f28cc769f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:44.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.231+0000 7f1e10519b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f28cc81f60 0x55f28cc82bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:44.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.231+0000 7f1e10519b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1e001b2e50 con 0x55f28cc81640 2024-06-10T01:07:44.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.231+0000 7f1e10519b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1e001b06e0 con 0x55f28cc7bf20 2024-06-10T01:07:44.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.231+0000 7f1e10519b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1e001ae530 con 0x55f28cc81f60 2024-06-10T01:07:44.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.231+0000 7f1dfeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f28cc81640 0x55f28cc769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.231+0000 7f1dfeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f28cc81640 0x55f28cc769f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46624/0 (socket says 172.21.15.7:46624) 2024-06-10T01:07:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.231+0000 7f1dfeffd700 1 -- 172.21.15.7:0/1183630114 learned_addr learned my addr 172.21.15.7:0/1183630114 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.232+0000 7f1dff7fe700 1 --2- 172.21.15.7:0/1183630114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f28cc7bf20 0x55f28cc7b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.232+0000 7f1dfeffd700 1 -- 172.21.15.7:0/1183630114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f28cc81f60 msgr2=0x55f28cc82bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.232+0000 7f1dfeffd700 1 --2- 172.21.15.7:0/1183630114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f28cc81f60 0x55f28cc82bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.232+0000 7f1dfeffd700 1 -- 172.21.15.7:0/1183630114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f28cc7bf20 msgr2=0x55f28cc7b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.232+0000 7f1dfeffd700 1 --2- 172.21.15.7:0/1183630114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f28cc7bf20 0x55f28cc7b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.232+0000 7f1dfeffd700 1 -- 172.21.15.7:0/1183630114 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f28cc85e90 con 0x55f28cc81640 2024-06-10T01:07:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.232+0000 7f1dff7fe700 1 --2- 172.21.15.7:0/1183630114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f28cc7bf20 0x55f28cc7b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:07:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.232+0000 7f1dfeffd700 1 --2- 172.21.15.7:0/1183630114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f28cc81640 0x55f28cc769f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1df4017000 tx=0x7f1df40053c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec7ec15f39c513e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.232+0000 7f1de57fa700 1 -- 172.21.15.7:0/1183630114 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4029070 con 0x55f28cc81640 2024-06-10T01:07:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.232+0000 7f1de57fa700 1 -- 172.21.15.7:0/1183630114 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1df400ee90 con 0x55f28cc81640 2024-06-10T01:07:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.232+0000 7f1de57fa700 1 -- 172.21.15.7:0/1183630114 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4016040 con 0x55f28cc81640 2024-06-10T01:07:44.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.233+0000 7f1e10519b80 1 -- 172.21.15.7:0/1183630114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f28cc81640 msgr2=0x55f28cc769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.233+0000 7f1e10519b80 1 --2- 172.21.15.7:0/1183630114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f28cc81640 0x55f28cc769f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1df4017000 tx=0x7f1df40053c0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.233+0000 7f1e10519b80 1 -- 172.21.15.7:0/1183630114 shutdown_connections 2024-06-10T01:07:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.233+0000 7f1e10519b80 1 --2- 172.21.15.7:0/1183630114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f28cc81f60 0x55f28cc82bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.233+0000 7f1e10519b80 1 --2- 172.21.15.7:0/1183630114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f28cc7bf20 0x55f28cc7b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.233+0000 7f1e10519b80 1 --2- 172.21.15.7:0/1183630114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f28cc81640 0x55f28cc769f0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.233+0000 7f1e10519b80 1 -- 172.21.15.7:0/1183630114 >> 172.21.15.7:0/1183630114 conn(0x55f28cba6cc0 msgr2=0x55f28cc6fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.233+0000 7f1e10519b80 1 -- 172.21.15.7:0/1183630114 shutdown_connections 2024-06-10T01:07:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.233+0000 7f1e10519b80 1 -- 172.21.15.7:0/1183630114 wait complete. 2024-06-10T01:07:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.234+0000 7f1e10519b80 1 -- start start 2024-06-10T01:07:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.234+0000 7f1e10519b80 1 Processor -- start 2024-06-10T01:07:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.234+0000 7f1e10519b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f28cc7bf20 0x55f28cc8de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.234+0000 7f1e10519b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f28cc81640 0x55f28cc8e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:44.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.234+0000 7f1e10519b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f28cc81f60 0x55f28cc8ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:44.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.234+0000 7f1e10519b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1e001aeba0 con 0x55f28cc81f60 2024-06-10T01:07:44.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.234+0000 7f1e10519b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1e001b0010 con 0x55f28cc7bf20 2024-06-10T01:07:44.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.234+0000 7f1e10519b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1e001af270 con 0x55f28cc81640 2024-06-10T01:07:44.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.234+0000 7f1e10519b80 10 client.0 _ll_register_callbacks cb 0x55f28cb67920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:44.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.234+0000 7f1dfffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f28cc81f60 0x55f28cc8ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:44.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.235+0000 7f1dfffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f28cc81f60 0x55f28cc8ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46626/0 (socket says 172.21.15.7:46626) 2024-06-10T01:07:44.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.235+0000 7f1dfffff700 1 -- 172.21.15.7:0/2769813102 learned_addr learned my addr 172.21.15.7:0/2769813102 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:44.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.235+0000 7f1dfeffd700 1 --2- 172.21.15.7:0/2769813102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f28cc81640 0x55f28cc8e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:44.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.235+0000 7f1dff7fe700 1 --2- 172.21.15.7:0/2769813102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f28cc7bf20 0x55f28cc8de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:44.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.235+0000 7f1dfffff700 1 -- 172.21.15.7:0/2769813102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f28cc81640 msgr2=0x55f28cc8e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:44.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.235+0000 7f1dfffff700 1 --2- 172.21.15.7:0/2769813102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f28cc81640 0x55f28cc8e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.235+0000 7f1dfffff700 1 -- 172.21.15.7:0/2769813102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f28cc7bf20 msgr2=0x55f28cc8de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:44.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.235+0000 7f1dfffff700 1 --2- 172.21.15.7:0/2769813102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f28cc7bf20 0x55f28cc8de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.235+0000 7f1dfffff700 1 -- 172.21.15.7:0/2769813102 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f28cc8ea50 con 0x55f28cc81f60 2024-06-10T01:07:44.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.235+0000 7f1dff7fe700 1 --2- 172.21.15.7:0/2769813102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f28cc7bf20 0x55f28cc8de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:44.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.235+0000 7f1dfffff700 1 --2- 172.21.15.7:0/2769813102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f28cc81f60 0x55f28cc8ade0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1df80cc6a0 tx=0x7f1df80a3910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd00a33c75b9f40c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:44.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.235+0000 7f1ddeffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:07:44.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.235+0000 7f1ddeffd700 1 -- 172.21.15.7:0/2769813102 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8097330 con 0x55f28cc81f60 2024-06-10T01:07:44.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.235+0000 7f1ddeffd700 1 -- 172.21.15.7:0/2769813102 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f1df809a6e0 con 0x55f28cc81f60 2024-06-10T01:07:44.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.235+0000 7f1e10519b80 1 -- 172.21.15.7:0/2769813102 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1df400ee90 con 0x55f28cc81f60 2024-06-10T01:07:44.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.236+0000 7f1ddeffd700 1 -- 172.21.15.7:0/2769813102 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1df807f050 con 0x55f28cc81f60 2024-06-10T01:07:44.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.236+0000 7f1e10519b80 10 client.15930 fetch_fsmap learned FSMap version 29 2024-06-10T01:07:44.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.236+0000 7f1e10519b80 1 -- 172.21.15.7:0/2769813102 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f28cd00b60 con 0x55f28cc81f60 2024-06-10T01:07:44.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.236+0000 7f1ddeffd700 1 -- 172.21.15.7:0/2769813102 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1df400ee90 con 0x55f28cc81f60 2024-06-10T01:07:44.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.236+0000 7f1ddeffd700 1 -- 172.21.15.7:0/2769813102 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f1df80b6070 con 0x55f28cc81f60 2024-06-10T01:07:44.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.236+0000 7f1e10519b80 10 client.15930 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:07:44.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.236+0000 7f1e10519b80 10 client.15930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:44.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.236+0000 7f1e10519b80 20 client.15930 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:44.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.236+0000 7f1e10519b80 1 --2- 172.21.15.7:0/2769813102 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f28cc95950 0x55f28cd03030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:44.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.236+0000 7f1e10519b80 4 client.15930 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:07:44.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.236+0000 7f1e10519b80 1 -- 172.21.15.7:0/2769813102 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f28cbd5290 con 0x55f28cc95950 2024-06-10T01:07:44.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.236+0000 7f1dff7fe700 1 --2- 172.21.15.7:0/2769813102 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f28cc95950 0x55f28cd03030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:44.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.237+0000 7f1dff7fe700 1 --2- 172.21.15.7:0/2769813102 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f28cc95950 0x55f28cd03030 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1128d6bcdbb11ce3 server_cookie=3a902d36b4d0f645 in_seq=0 out_seq=0 2024-06-10T01:07:44.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.237+0000 7f1ddeffd700 10 client.15930 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:44.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.237+0000 7f1ddeffd700 1 -- 172.21.15.7:0/2769813102 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f28cbd5290 con 0x55f28cc95950 2024-06-10T01:07:44.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.237+0000 7f1ddeffd700 10 client.15930 handle_command_reply: tid=0 2024-06-10T01:07:44.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.237+0000 7f1e10519b80 1 client.15930 shutdown 2024-06-10T01:07:44.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.237+0000 7f1e10519b80 1 -- 172.21.15.7:0/2769813102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f28cc81f60 msgr2=0x55f28cc8ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:44.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.237+0000 7f1e10519b80 1 --2- 172.21.15.7:0/2769813102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f28cc81f60 0x55f28cc8ade0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1df80cc6a0 tx=0x7f1df80a3910 comp rx=0 tx=0).stop 2024-06-10T01:07:44.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.237+0000 7f1e10519b80 1 -- 172.21.15.7:0/2769813102 shutdown_connections 2024-06-10T01:07:44.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.237+0000 7f1e10519b80 1 --2- 172.21.15.7:0/2769813102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f28cc81f60 0x55f28cc8ade0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.237+0000 7f1e10519b80 1 --2- 172.21.15.7:0/2769813102 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f28cc95950 0x55f28cd03030 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.237+0000 7f1e10519b80 1 --2- 172.21.15.7:0/2769813102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f28cc7bf20 0x55f28cc8de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.237+0000 7f1e10519b80 1 --2- 172.21.15.7:0/2769813102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f28cc81640 0x55f28cc8e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.237+0000 7f1e10519b80 1 -- 172.21.15.7:0/2769813102 >> 172.21.15.7:0/2769813102 conn(0x55f28cba6cc0 msgr2=0x55f28cc70510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:44.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.237+0000 7f1ddeffd700 0 client.15930 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:44.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.237+0000 7f1e10519b80 1 -- 172.21.15.7:0/2769813102 shutdown_connections 2024-06-10T01:07:44.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.237+0000 7f1e10519b80 1 -- 172.21.15.7:0/2769813102 wait complete. 2024-06-10T01:07:44.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.238+0000 7f1e10519b80 20 client.15930 trim_cache size 0 max 16384 2024-06-10T01:07:44.239 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:07:44.239 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-10T01:07:44.239 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-10T01:07:44.239 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:07:44.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.238+0000 7f1e05c96700 1 -- 172.21.15.7:0/1567665033 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1de80a2c20 msgr2=0x7f1de80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:44.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.238+0000 7f1e05c96700 1 --2- 172.21.15.7:0/1567665033 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1de80a2c20 0x7f1de80a50e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1df0000c00 tx=0x7f1df0005fb0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.238+0000 7f1e05c96700 1 -- 172.21.15.7:0/1567665033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e00107550 msgr2=0x7f1e00102a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:44.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.238+0000 7f1e05c96700 1 --2- 172.21.15.7:0/1567665033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e00107550 0x7f1e00102a90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1df8007ee0 tx=0x7f1df800d3b0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.238+0000 7f1e05c96700 1 -- 172.21.15.7:0/1567665033 shutdown_connections 2024-06-10T01:07:44.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.238+0000 7f1e05c96700 1 --2- 172.21.15.7:0/1567665033 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1de80a2c20 0x7f1de80a50e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.238+0000 7f1e05c96700 1 --2- 172.21.15.7:0/1567665033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e00107550 0x7f1e00102a90 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.238+0000 7f1e05c96700 1 --2- 172.21.15.7:0/1567665033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e00100e30 0x7f1e00102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.238+0000 7f1e05c96700 1 --2- 172.21.15.7:0/1567665033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e001017f0 0x7f1e00102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:44.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.238+0000 7f1e05c96700 1 -- 172.21.15.7:0/1567665033 >> 172.21.15.7:0/1567665033 conn(0x7f1e000766b0 msgr2=0x7f1e000ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:44.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.238+0000 7f1e05c96700 1 -- 172.21.15.7:0/1567665033 shutdown_connections 2024-06-10T01:07:44.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:44.238+0000 7f1e05c96700 1 -- 172.21.15.7:0/1567665033 wait complete. 2024-06-10T01:07:44.708 INFO:tasks.workunit.client.0.smithi007.stdout: 1 28947 6.96 MB/sec warmup 25 sec latency 15164.001 ms 2024-06-10T01:07:45.204 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:44 smithi175 ceph-mon[36636]: pgmap v484: 129 pgs: 129 active+clean; 11 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 448 KiB/s wr, 108 op/s 2024-06-10T01:07:45.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:44 smithi007 ceph-mon[32398]: pgmap v484: 129 pgs: 129 active+clean; 11 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 448 KiB/s wr, 108 op/s 2024-06-10T01:07:45.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:44 smithi078 ceph-mon[36683]: pgmap v484: 129 pgs: 129 active+clean; 11 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 448 KiB/s wr, 108 op/s 2024-06-10T01:07:45.708 INFO:tasks.workunit.client.0.smithi007.stdout: 1 28947 6.69 MB/sec warmup 26 sec latency 16164.076 ms 2024-06-10T01:07:46.708 INFO:tasks.workunit.client.0.smithi007.stdout: 1 28947 6.44 MB/sec warmup 27 sec latency 17164.165 ms 2024-06-10T01:07:46.825 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:07:46.825 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:07:47.084 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:47.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:46 smithi175 ceph-mon[36636]: pgmap v485: 129 pgs: 129 active+clean; 11 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 34 KiB/s wr, 113 op/s 2024-06-10T01:07:47.206 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:46 smithi007 ceph-mon[32398]: pgmap v485: 129 pgs: 129 active+clean; 11 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 34 KiB/s wr, 113 op/s 2024-06-10T01:07:47.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:46 smithi078 ceph-mon[36683]: pgmap v485: 129 pgs: 129 active+clean; 11 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 34 KiB/s wr, 113 op/s 2024-06-10T01:07:47.709 INFO:tasks.workunit.client.0.smithi007.stdout: 1 28947 6.21 MB/sec warmup 28 sec latency 18164.253 ms 2024-06-10T01:07:48.709 INFO:tasks.workunit.client.0.smithi007.stdout: 1 28947 6.00 MB/sec warmup 29 sec latency 19164.316 ms 2024-06-10T01:07:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.763+0000 7f2166ff4700 1 -- 172.21.15.7:0/2605786833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2160102740 msgr2=0x7f2160102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.763+0000 7f2166ff4700 1 --2- 172.21.15.7:0/2605786833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2160102740 0x7f2160102b20 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f21540099c0 tx=0x7f2154009cd0 comp rx=0 tx=0).stop 2024-06-10T01:07:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.763+0000 7f2166ff4700 1 -- 172.21.15.7:0/2605786833 shutdown_connections 2024-06-10T01:07:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.763+0000 7f2166ff4700 1 --2- 172.21.15.7:0/2605786833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2160107840 0x7f216010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.763+0000 7f2166ff4700 1 --2- 172.21.15.7:0/2605786833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2160102740 0x7f2160102b20 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.763+0000 7f2166ff4700 1 --2- 172.21.15.7:0/2605786833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21601030f0 0x7f2160107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.763+0000 7f2166ff4700 1 -- 172.21.15.7:0/2605786833 >> 172.21.15.7:0/2605786833 conn(0x7f21600795f0 msgr2=0x7f2160079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.763+0000 7f2166ff4700 1 -- 172.21.15.7:0/2605786833 shutdown_connections 2024-06-10T01:07:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.763+0000 7f2166ff4700 1 -- 172.21.15.7:0/2605786833 wait complete. 2024-06-10T01:07:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.763+0000 7f2166ff4700 1 Processor -- start 2024-06-10T01:07:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.764+0000 7f2166ff4700 1 -- start start 2024-06-10T01:07:48.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.764+0000 7f2166ff4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2160102740 0x7f216019e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:48.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.764+0000 7f2166ff4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21601030f0 0x7f216019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:48.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.764+0000 7f2166ff4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2160107840 0x7f2160198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:48.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.764+0000 7f2166ff4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f21601127e0 con 0x7f2160107840 2024-06-10T01:07:48.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.764+0000 7f2166ff4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2160112660 con 0x7f21601030f0 2024-06-10T01:07:48.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.764+0000 7f2166ff4700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2160112960 con 0x7f2160102740 2024-06-10T01:07:48.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.764+0000 7f2165591700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2160107840 0x7f2160198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:48.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.764+0000 7f2165591700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2160107840 0x7f2160198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46646/0 (socket says 172.21.15.7:46646) 2024-06-10T01:07:48.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.764+0000 7f2165591700 1 -- 172.21.15.7:0/4218703757 learned_addr learned my addr 172.21.15.7:0/4218703757 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:48.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.764+0000 7f215ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21601030f0 0x7f216019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:48.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.764+0000 7f2164d90700 1 --2- 172.21.15.7:0/4218703757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2160102740 0x7f216019e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:48.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.764+0000 7f2165591700 1 -- 172.21.15.7:0/4218703757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2160102740 msgr2=0x7f216019e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:48.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.764+0000 7f2165591700 1 --2- 172.21.15.7:0/4218703757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2160102740 0x7f216019e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:48.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.764+0000 7f2165591700 1 -- 172.21.15.7:0/4218703757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21601030f0 msgr2=0x7f216019ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:48.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.764+0000 7f2165591700 1 --2- 172.21.15.7:0/4218703757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21601030f0 0x7f216019ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:48.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.764+0000 7f2165591700 1 -- 172.21.15.7:0/4218703757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21601991c0 con 0x7f2160107840 2024-06-10T01:07:48.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.764+0000 7f2165591700 1 --2- 172.21.15.7:0/4218703757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2160107840 0x7f2160198a60 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f215800b970 tx=0x7f215800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:48.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.765+0000 7f215dffb700 1 -- 172.21.15.7:0/4218703757 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2158019070 con 0x7f2160107840 2024-06-10T01:07:48.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.765+0000 7f215dffb700 1 -- 172.21.15.7:0/4218703757 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f215800ce10 con 0x7f2160107840 2024-06-10T01:07:48.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.765+0000 7f2166ff4700 1 -- 172.21.15.7:0/4218703757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21601994b0 con 0x7f2160107840 2024-06-10T01:07:48.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.765+0000 7f215dffb700 1 -- 172.21.15.7:0/4218703757 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2158007370 con 0x7f2160107840 2024-06-10T01:07:48.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.765+0000 7f2166ff4700 1 -- 172.21.15.7:0/4218703757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21601a5720 con 0x7f2160107840 2024-06-10T01:07:48.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.766+0000 7f2166ff4700 1 -- 172.21.15.7:0/4218703757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f216010aa80 con 0x7f2160107840 2024-06-10T01:07:48.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.767+0000 7f215dffb700 1 -- 172.21.15.7:0/4218703757 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f215801c070 con 0x7f2160107840 2024-06-10T01:07:48.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.768+0000 7f215dffb700 1 --2- 172.21.15.7:0/4218703757 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21500a2ca0 0x7f21500a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:48.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.768+0000 7f215dffb700 1 -- 172.21.15.7:0/4218703757 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f21580ce720 con 0x7f2160107840 2024-06-10T01:07:48.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.768+0000 7f2164d90700 1 --2- 172.21.15.7:0/4218703757 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21500a2ca0 0x7f21500a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:48.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.768+0000 7f2164d90700 1 --2- 172.21.15.7:0/4218703757 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21500a2ca0 0x7f21500a5160 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2154000c00 tx=0x7f2154005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:48.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:48.772+0000 7f215dffb700 1 -- 172.21.15.7:0/4218703757 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f215809b300 con 0x7f2160107840 2024-06-10T01:07:49.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:49.026+0000 7f2166ff4700 1 -- 172.21.15.7:0/4218703757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2160199f40 con 0x7f2160107840 2024-06-10T01:07:49.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:49.027+0000 7f215dffb700 1 -- 172.21.15.7:0/4218703757 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11855 (secure 0 0 0) 0x7f215809ac00 con 0x7f2160107840 2024-06-10T01:07:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:07:49.023 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:07:36.881518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:07:49.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:49.029+0000 7f2166ff4700 1 -- 172.21.15.7:0/4218703757 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21500a2ca0 msgr2=0x7f21500a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:49.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:49.029+0000 7f2166ff4700 1 --2- 172.21.15.7:0/4218703757 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21500a2ca0 0x7f21500a5160 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2154000c00 tx=0x7f2154005fb0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:49.029+0000 7f2166ff4700 1 -- 172.21.15.7:0/4218703757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2160107840 msgr2=0x7f2160198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:49.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:49.029+0000 7f2166ff4700 1 --2- 172.21.15.7:0/4218703757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2160107840 0x7f2160198a60 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f215800b970 tx=0x7f215800bc80 comp rx=0 tx=0).stop 2024-06-10T01:07:49.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:49.029+0000 7f2166ff4700 1 -- 172.21.15.7:0/4218703757 shutdown_connections 2024-06-10T01:07:49.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:49.029+0000 7f2166ff4700 1 --2- 172.21.15.7:0/4218703757 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21500a2ca0 0x7f21500a5160 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:49.029+0000 7f2166ff4700 1 --2- 172.21.15.7:0/4218703757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2160107840 0x7f2160198a60 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:49.029+0000 7f2166ff4700 1 --2- 172.21.15.7:0/4218703757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2160102740 0x7f216019e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:49.029+0000 7f2166ff4700 1 --2- 172.21.15.7:0/4218703757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21601030f0 0x7f216019ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:49.029+0000 7f2166ff4700 1 -- 172.21.15.7:0/4218703757 >> 172.21.15.7:0/4218703757 conn(0x7f21600795f0 msgr2=0x7f2160100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:49.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:49.029+0000 7f2166ff4700 1 -- 172.21.15.7:0/4218703757 shutdown_connections 2024-06-10T01:07:49.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:49.029+0000 7f2166ff4700 1 -- 172.21.15.7:0/4218703757 wait complete. 2024-06-10T01:07:49.026 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 29 2024-06-10T01:07:49.204 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:48 smithi175 ceph-mon[36636]: pgmap v486: 129 pgs: 129 active+clean; 18 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 845 KiB/s wr, 115 op/s 2024-06-10T01:07:49.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.214+0000 7fc7aaa38640 1 -- 172.21.15.7:0/334099950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7a4107760 msgr2=0x7fc7a4109b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:49.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.214+0000 7fc7aaa38640 1 --2- 172.21.15.7:0/334099950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7a4107760 0x7fc7a4109b50 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009ca0 tx=0x7fc79c02a8c0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.214+0000 7fc7aaa38640 1 -- 172.21.15.7:0/334099950 shutdown_connections 2024-06-10T01:07:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.214+0000 7fc7aaa38640 1 --2- 172.21.15.7:0/334099950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7a4107760 0x7fc7a4109b50 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.214+0000 7fc7aaa38640 1 --2- 172.21.15.7:0/334099950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7a4104e30 0x7fc7a4107220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.214+0000 7fc7aaa38640 1 --2- 172.21.15.7:0/334099950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7a4102500 0x7fc7a41048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.214+0000 7fc7aaa38640 1 -- 172.21.15.7:0/334099950 >> 172.21.15.7:0/334099950 conn(0x7fc7a40fbe70 msgr2=0x7fc7a40fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.214+0000 7fc7aaa38640 1 -- 172.21.15.7:0/334099950 shutdown_connections 2024-06-10T01:07:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.214+0000 7fc7aaa38640 1 -- 172.21.15.7:0/334099950 wait complete. 2024-06-10T01:07:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.214+0000 7fc7aaa38640 1 Processor -- start 2024-06-10T01:07:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.214+0000 7fc7aaa38640 1 -- start start 2024-06-10T01:07:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.215+0000 7fc7aaa38640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7a4102500 0x7fc7a4197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.215+0000 7fc7aaa38640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7a4104e30 0x7fc7a4198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.215+0000 7fc7aaa38640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7a4107760 0x7fc7a419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.215+0000 7fc7aaa38640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a410d680 con 0x7fc7a4102500 2024-06-10T01:07:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.215+0000 7fc7aaa38640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a410d500 con 0x7fc7a4104e30 2024-06-10T01:07:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.215+0000 7fc7aaa38640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a410d800 con 0x7fc7a4107760 2024-06-10T01:07:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.215+0000 7fc7a3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7a4102500 0x7fc7a4197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.215+0000 7fc7a3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7a4102500 0x7fc7a4197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46656/0 (socket says 172.21.15.7:46656) 2024-06-10T01:07:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.215+0000 7fc7a3fff640 1 -- 172.21.15.7:0/1308868038 learned_addr learned my addr 172.21.15.7:0/1308868038 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.215+0000 7fc7a37fe640 1 --2- 172.21.15.7:0/1308868038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7a4104e30 0x7fc7a4198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.215+0000 7fc7a8fd5640 1 --2- 172.21.15.7:0/1308868038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7a4107760 0x7fc7a419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.215+0000 7fc7a3fff640 1 -- 172.21.15.7:0/1308868038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7a4107760 msgr2=0x7fc7a419f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.215+0000 7fc7a3fff640 1 --2- 172.21.15.7:0/1308868038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7a4107760 0x7fc7a419f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.215+0000 7fc7a3fff640 1 -- 172.21.15.7:0/1308868038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7a4104e30 msgr2=0x7fc7a4198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.215+0000 7fc7a3fff640 1 --2- 172.21.15.7:0/1308868038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7a4104e30 0x7fc7a4198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.215+0000 7fc7a3fff640 1 -- 172.21.15.7:0/1308868038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a419f840 con 0x7fc7a4102500 2024-06-10T01:07:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.215+0000 7fc7a8fd5640 1 --2- 172.21.15.7:0/1308868038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7a4107760 0x7fc7a419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:07:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.215+0000 7fc7a37fe640 1 --2- 172.21.15.7:0/1308868038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7a4104e30 0x7fc7a4198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.215+0000 7fc7a3fff640 1 --2- 172.21.15.7:0/1308868038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7a4102500 0x7fc7a4197c30 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc79400c920 tx=0x7fc79400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.216+0000 7fc7a17fa640 1 -- 172.21.15.7:0/1308868038 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc794007c70 con 0x7fc7a4102500 2024-06-10T01:07:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.216+0000 7fc7a17fa640 1 -- 172.21.15.7:0/1308868038 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc794007e10 con 0x7fc7a4102500 2024-06-10T01:07:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.216+0000 7fc7aaa38640 1 -- 172.21.15.7:0/1308868038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7a419fb30 con 0x7fc7a4102500 2024-06-10T01:07:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.216+0000 7fc7a17fa640 1 -- 172.21.15.7:0/1308868038 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc794018890 con 0x7fc7a4102500 2024-06-10T01:07:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.216+0000 7fc7aaa38640 1 -- 172.21.15.7:0/1308868038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7a41a0080 con 0x7fc7a4102500 2024-06-10T01:07:49.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.218+0000 7fc7a17fa640 1 -- 172.21.15.7:0/1308868038 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fc794018a70 con 0x7fc7a4102500 2024-06-10T01:07:49.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.218+0000 7fc7ad821740 1 Processor -- start 2024-06-10T01:07:49.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.218+0000 7fc7ad821740 1 -- start start 2024-06-10T01:07:49.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.218+0000 7fc7ad821740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb0d3ca010 0x55bb0d3ca430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:49.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.218+0000 7fc7ad821740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0d3caa00 0x55bb0d3cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:49.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.218+0000 7fc7ad821740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb0d3cfb70 0x55bb0d3d2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:49.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.218+0000 7fc7ad821740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bb0d32d4f0 con 0x55bb0d3ca010 2024-06-10T01:07:49.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.218+0000 7fc7ad821740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bb0d2cb870 con 0x55bb0d3cfb70 2024-06-10T01:07:49.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.218+0000 7fc7ad821740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bb0d3294b0 con 0x55bb0d3caa00 2024-06-10T01:07:49.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.218+0000 7fc7a8fd5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb0d3ca010 0x55bb0d3ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.218+0000 7fc7a8fd5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb0d3ca010 0x55bb0d3ca430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46660/0 (socket says 172.21.15.7:46660) 2024-06-10T01:07:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.218+0000 7fc7a8fd5640 1 -- 172.21.15.7:0/1962403495 learned_addr learned my addr 172.21.15.7:0/1962403495 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.219+0000 7fc7a17fa640 1 --2- 172.21.15.7:0/1308868038 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc78c0a2b80 0x7fc78c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.219+0000 7fc7a37fe640 1 --2- 172.21.15.7:0/1962403495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb0d3cfb70 0x55bb0d3d2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.219+0000 7fc7a3fff640 1 --2- 172.21.15.7:0/1962403495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0d3caa00 0x55bb0d3cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.219+0000 7fc7a8fd5640 1 --2- 172.21.15.7:0/1308868038 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc78c0a2b80 0x7fc78c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.219+0000 7fc7a17fa640 1 -- 172.21.15.7:0/1308868038 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc7940cf6b0 con 0x7fc7a4102500 2024-06-10T01:07:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.219+0000 7fc7a8fd5640 1 -- 172.21.15.7:0/1962403495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0d3caa00 msgr2=0x55bb0d3cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.219+0000 7fc7a8fd5640 1 --2- 172.21.15.7:0/1962403495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0d3caa00 0x55bb0d3cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.219+0000 7fc7a8fd5640 1 -- 172.21.15.7:0/1962403495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb0d3cfb70 msgr2=0x55bb0d3d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.219+0000 7fc7a8fd5640 1 --2- 172.21.15.7:0/1962403495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb0d3cfb70 0x55bb0d3d2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.219+0000 7fc7a8fd5640 1 -- 172.21.15.7:0/1962403495 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb0d2e79b0 con 0x55bb0d3ca010 2024-06-10T01:07:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.219+0000 7fc7a37fe640 1 --2- 172.21.15.7:0/1962403495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb0d3cfb70 0x55bb0d3d2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.219+0000 7fc7a8fd5640 1 --2- 172.21.15.7:0/1962403495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb0d3ca010 0x55bb0d3ca430 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009430 tx=0x7fc79c031450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e37b5083dbf1533 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:49.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.219+0000 7fc78a7fc640 1 -- 172.21.15.7:0/1962403495 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c004b40 con 0x55bb0d3ca010 2024-06-10T01:07:49.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.219+0000 7fc78a7fc640 1 -- 172.21.15.7:0/1962403495 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc79c04d070 con 0x55bb0d3ca010 2024-06-10T01:07:49.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.219+0000 7fc78a7fc640 1 -- 172.21.15.7:0/1962403495 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c02f870 con 0x55bb0d3ca010 2024-06-10T01:07:49.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.219+0000 7fc7a8fd5640 1 --2- 172.21.15.7:0/1308868038 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc78c0a2b80 0x7fc78c0a5040 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc79c04c000 tx=0x7fc79c04a000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:49.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.220+0000 7fc7ad821740 1 -- 172.21.15.7:0/1962403495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb0d3ca010 msgr2=0x55bb0d3ca430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:49.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.220+0000 7fc7ad821740 1 --2- 172.21.15.7:0/1962403495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb0d3ca010 0x55bb0d3ca430 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009430 tx=0x7fc79c031450 comp rx=0 tx=0).stop 2024-06-10T01:07:49.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.220+0000 7fc7ad821740 1 -- 172.21.15.7:0/1962403495 shutdown_connections 2024-06-10T01:07:49.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.220+0000 7fc7ad821740 1 --2- 172.21.15.7:0/1962403495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb0d3cfb70 0x55bb0d3d2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.220+0000 7fc7ad821740 1 --2- 172.21.15.7:0/1962403495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0d3caa00 0x55bb0d3cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.220+0000 7fc7ad821740 1 --2- 172.21.15.7:0/1962403495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb0d3ca010 0x55bb0d3ca430 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.220+0000 7fc7ad821740 1 -- 172.21.15.7:0/1962403495 >> 172.21.15.7:0/1962403495 conn(0x55bb0d3c5680 msgr2=0x55bb0d3c5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.220+0000 7fc7ad821740 1 -- 172.21.15.7:0/1962403495 shutdown_connections 2024-06-10T01:07:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.220+0000 7fc7ad821740 1 -- 172.21.15.7:0/1962403495 wait complete. 2024-06-10T01:07:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.220+0000 7fc7ad821740 1 -- start start 2024-06-10T01:07:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.221+0000 7fc7ad821740 1 Processor -- start 2024-06-10T01:07:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.221+0000 7fc7ad821740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb0d3ca010 0x55bb0d3d4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.221+0000 7fc7ad821740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0d3caa00 0x55bb0d3d47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.221+0000 7fc7ad821740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb0d3cfb70 0x55bb0d3db7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.221+0000 7fc7ad821740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bb0d3d2b20 con 0x55bb0d3cfb70 2024-06-10T01:07:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.221+0000 7fc7ad821740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bb0d2b6ae0 con 0x55bb0d3ca010 2024-06-10T01:07:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.221+0000 7fc7ad821740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bb0d3d2ca0 con 0x55bb0d3caa00 2024-06-10T01:07:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.221+0000 7fc7ad821740 10 client.0 _ll_register_callbacks cb 0x55bb0d2ef9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.221+0000 7fc7a8fd5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb0d3cfb70 0x55bb0d3db7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.221+0000 7fc7a8fd5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb0d3cfb70 0x55bb0d3db7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46676/0 (socket says 172.21.15.7:46676) 2024-06-10T01:07:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.221+0000 7fc7a8fd5640 1 -- 172.21.15.7:0/1250914260 learned_addr learned my addr 172.21.15.7:0/1250914260 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.221+0000 7fc7a37fe640 1 --2- 172.21.15.7:0/1250914260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0d3caa00 0x55bb0d3d47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.221+0000 7fc7a3fff640 1 --2- 172.21.15.7:0/1250914260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb0d3ca010 0x55bb0d3d4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.221+0000 7fc7a8fd5640 1 -- 172.21.15.7:0/1250914260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0d3caa00 msgr2=0x55bb0d3d47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.221+0000 7fc7a8fd5640 1 --2- 172.21.15.7:0/1250914260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0d3caa00 0x55bb0d3d47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.221+0000 7fc7a8fd5640 1 -- 172.21.15.7:0/1250914260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb0d3ca010 msgr2=0x55bb0d3d4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.221+0000 7fc7a8fd5640 1 --2- 172.21.15.7:0/1250914260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb0d3ca010 0x55bb0d3d4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.221+0000 7fc7a8fd5640 1 -- 172.21.15.7:0/1250914260 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb0d041210 con 0x55bb0d3cfb70 2024-06-10T01:07:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.221+0000 7fc7a3fff640 1 --2- 172.21.15.7:0/1250914260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb0d3ca010 0x55bb0d3d4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.221+0000 7fc7a37fe640 1 --2- 172.21.15.7:0/1250914260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0d3caa00 0x55bb0d3d47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.221+0000 7fc7a8fd5640 1 --2- 172.21.15.7:0/1250914260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb0d3cfb70 0x55bb0d3db7e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc79c02a900 tx=0x7fc79c005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44909c72d1dd1d86 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.222+0000 7fc7837fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:07:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.222+0000 7fc7837fe640 1 -- 172.21.15.7:0/1250914260 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c059070 con 0x55bb0d3cfb70 2024-06-10T01:07:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.222+0000 7fc7837fe640 1 -- 172.21.15.7:0/1250914260 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc79c004d40 con 0x55bb0d3cfb70 2024-06-10T01:07:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.222+0000 7fc7ad821740 1 -- 172.21.15.7:0/1250914260 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc79c04d070 con 0x55bb0d3cfb70 2024-06-10T01:07:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.222+0000 7fc7837fe640 1 -- 172.21.15.7:0/1250914260 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc79c065050 con 0x55bb0d3cfb70 2024-06-10T01:07:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.222+0000 7fc7ad821740 10 client.15948 fetch_fsmap learned FSMap version 29 2024-06-10T01:07:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.222+0000 7fc7ad821740 1 -- 172.21.15.7:0/1250914260 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bb0d10d620 con 0x55bb0d3cfb70 2024-06-10T01:07:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.222+0000 7fc7837fe640 1 -- 172.21.15.7:0/1250914260 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc79c04d070 con 0x55bb0d3cfb70 2024-06-10T01:07:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.222+0000 7fc7837fe640 1 -- 172.21.15.7:0/1250914260 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fc79c0043b0 con 0x55bb0d3cfb70 2024-06-10T01:07:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.222+0000 7fc7ad821740 10 client.15948 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:07:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.222+0000 7fc7ad821740 10 client.15948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.222+0000 7fc7ad821740 20 client.15948 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.222+0000 7fc7ad821740 1 --2- 172.21.15.7:0/1250914260 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55bb0d3dfd20 0x55bb0d3e2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.222+0000 7fc7ad821740 4 client.15948 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:07:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.222+0000 7fc7ad821740 1 -- 172.21.15.7:0/1250914260 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb0d340f10 con 0x55bb0d3dfd20 2024-06-10T01:07:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.223+0000 7fc7a3fff640 1 --2- 172.21.15.7:0/1250914260 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55bb0d3dfd20 0x55bb0d3e2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.223+0000 7fc7837fe640 10 client.15948 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.223+0000 7fc7a3fff640 1 --2- 172.21.15.7:0/1250914260 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55bb0d3dfd20 0x55bb0d3e2110 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58078027c8c0e475 server_cookie=81e01f93baf3ebe6 in_seq=0 out_seq=0 2024-06-10T01:07:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.224+0000 7fc7837fe640 1 -- 172.21.15.7:0/1250914260 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bb0d340f10 con 0x55bb0d3dfd20 2024-06-10T01:07:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.224+0000 7fc7837fe640 10 client.15948 handle_command_reply: tid=0 2024-06-10T01:07:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.224+0000 7fc7ad821740 1 client.15948 shutdown 2024-06-10T01:07:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.224+0000 7fc7ad821740 1 -- 172.21.15.7:0/1250914260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb0d3cfb70 msgr2=0x55bb0d3db7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.224+0000 7fc7ad821740 1 --2- 172.21.15.7:0/1250914260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb0d3cfb70 0x55bb0d3db7e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc79c02a900 tx=0x7fc79c005f80 comp rx=0 tx=0).stop 2024-06-10T01:07:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.224+0000 7fc7ad821740 1 -- 172.21.15.7:0/1250914260 shutdown_connections 2024-06-10T01:07:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.224+0000 7fc7ad821740 1 --2- 172.21.15.7:0/1250914260 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55bb0d3dfd20 0x55bb0d3e2110 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.224+0000 7fc7ad821740 1 --2- 172.21.15.7:0/1250914260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb0d3cfb70 0x55bb0d3db7e0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.224+0000 7fc7ad821740 1 --2- 172.21.15.7:0/1250914260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0d3caa00 0x55bb0d3d47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.224+0000 7fc7ad821740 1 --2- 172.21.15.7:0/1250914260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb0d3ca010 0x55bb0d3d4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.224+0000 7fc7ad821740 1 -- 172.21.15.7:0/1250914260 >> 172.21.15.7:0/1250914260 conn(0x55bb0d3c5680 msgr2=0x55bb0d3c47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.224+0000 7fc7837fe640 0 client.15948 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.224+0000 7fc7ad821740 1 -- 172.21.15.7:0/1250914260 shutdown_connections 2024-06-10T01:07:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.224+0000 7fc7ad821740 1 -- 172.21.15.7:0/1250914260 wait complete. 2024-06-10T01:07:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.224+0000 7fc7ad821740 20 client.15948 trim_cache size 0 max 16384 2024-06-10T01:07:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.239+0000 7fc7ad821740 1 Processor -- start 2024-06-10T01:07:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.239+0000 7fc7ad821740 1 -- start start 2024-06-10T01:07:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.239+0000 7fc7ad821740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb0d3ca010 0x55bb0d3c09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:49.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.239+0000 7fc7ad821740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb0d3caa00 0x55bb0d3db7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:49.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.239+0000 7fc7ad821740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0d3cfb70 0x55bb0d3c8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:49.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.239+0000 7fc7ad821740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a41accd0 con 0x55bb0d3caa00 2024-06-10T01:07:49.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.239+0000 7fc7ad821740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a41aa5a0 con 0x55bb0d3ca010 2024-06-10T01:07:49.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.239+0000 7fc7ad821740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a41a83f0 con 0x55bb0d3cfb70 2024-06-10T01:07:49.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.240+0000 7fc7a37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb0d3caa00 0x55bb0d3db7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:49.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.240+0000 7fc7a37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb0d3caa00 0x55bb0d3db7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46680/0 (socket says 172.21.15.7:46680) 2024-06-10T01:07:49.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.240+0000 7fc7a37fe640 1 -- 172.21.15.7:0/2574501324 learned_addr learned my addr 172.21.15.7:0/2574501324 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:49.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.240+0000 7fc7a3fff640 1 --2- 172.21.15.7:0/2574501324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb0d3ca010 0x55bb0d3c09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:49.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.240+0000 7fc7a37fe640 1 -- 172.21.15.7:0/2574501324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0d3cfb70 msgr2=0x55bb0d3c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.240+0000 7fc7a37fe640 1 --2- 172.21.15.7:0/2574501324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0d3cfb70 0x55bb0d3c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.240+0000 7fc7a37fe640 1 -- 172.21.15.7:0/2574501324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb0d3ca010 msgr2=0x55bb0d3c09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.240+0000 7fc7a37fe640 1 --2- 172.21.15.7:0/2574501324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb0d3ca010 0x55bb0d3c09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.240+0000 7fc7a37fe640 1 -- 172.21.15.7:0/2574501324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb0d3c6350 con 0x55bb0d3caa00 2024-06-10T01:07:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.240+0000 7fc7a3fff640 1 --2- 172.21.15.7:0/2574501324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb0d3ca010 0x55bb0d3c09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.240+0000 7fc7a37fe640 1 --2- 172.21.15.7:0/2574501324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb0d3caa00 0x55bb0d3db7a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc798004490 tx=0x7fc798009c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4d97ba2d13b1b7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.240+0000 7fc789ffb640 1 -- 172.21.15.7:0/2574501324 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc798009e00 con 0x55bb0d3caa00 2024-06-10T01:07:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.240+0000 7fc789ffb640 1 -- 172.21.15.7:0/2574501324 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc798003b50 con 0x55bb0d3caa00 2024-06-10T01:07:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.240+0000 7fc789ffb640 1 -- 172.21.15.7:0/2574501324 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc798012740 con 0x55bb0d3caa00 2024-06-10T01:07:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.241+0000 7fc7ad821740 1 -- 172.21.15.7:0/2574501324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb0d3caa00 msgr2=0x55bb0d3db7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.241+0000 7fc7ad821740 1 --2- 172.21.15.7:0/2574501324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb0d3caa00 0x55bb0d3db7a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc798004490 tx=0x7fc798009c10 comp rx=0 tx=0).stop 2024-06-10T01:07:49.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.241+0000 7fc7ad821740 1 -- 172.21.15.7:0/2574501324 shutdown_connections 2024-06-10T01:07:49.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.241+0000 7fc7ad821740 1 --2- 172.21.15.7:0/2574501324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0d3cfb70 0x55bb0d3c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.241+0000 7fc7ad821740 1 --2- 172.21.15.7:0/2574501324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb0d3caa00 0x55bb0d3db7a0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.241+0000 7fc7ad821740 1 --2- 172.21.15.7:0/2574501324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb0d3ca010 0x55bb0d3c09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.241+0000 7fc7ad821740 1 -- 172.21.15.7:0/2574501324 >> 172.21.15.7:0/2574501324 conn(0x55bb0d3c5680 msgr2=0x55bb0d3cbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:49.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.241+0000 7fc7ad821740 1 -- 172.21.15.7:0/2574501324 shutdown_connections 2024-06-10T01:07:49.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.241+0000 7fc7ad821740 1 -- 172.21.15.7:0/2574501324 wait complete. 2024-06-10T01:07:49.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.241+0000 7fc7ad821740 1 -- start start 2024-06-10T01:07:49.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.241+0000 7fc7ad821740 1 Processor -- start 2024-06-10T01:07:49.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.242+0000 7fc7ad821740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb0d3ca010 0x55bb0d3d85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:49.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.242+0000 7fc7ad821740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb0d3caa00 0x55bb0d3d8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:49.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.242+0000 7fc7ad821740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0d3cfb70 0x55bb0d448990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:49.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.242+0000 7fc7ad821740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a41a8a60 con 0x55bb0d3ca010 2024-06-10T01:07:49.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.242+0000 7fc7ad821740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a41a9ed0 con 0x55bb0d3caa00 2024-06-10T01:07:49.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.242+0000 7fc7ad821740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a41a9130 con 0x55bb0d3cfb70 2024-06-10T01:07:49.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.242+0000 7fc7ad821740 10 client.0 _ll_register_callbacks cb 0x7fc7980095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:49.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.242+0000 7fc7a3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb0d3ca010 0x55bb0d3d85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:49.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.242+0000 7fc7a3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb0d3ca010 0x55bb0d3d85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46696/0 (socket says 172.21.15.7:46696) 2024-06-10T01:07:49.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.242+0000 7fc7a3fff640 1 -- 172.21.15.7:0/4034966512 learned_addr learned my addr 172.21.15.7:0/4034966512 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:49.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.242+0000 7fc7a8fd5640 1 --2- 172.21.15.7:0/4034966512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0d3cfb70 0x55bb0d448990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.242+0000 7fc7a37fe640 1 --2- 172.21.15.7:0/4034966512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb0d3caa00 0x55bb0d3d8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.242+0000 7fc7a3fff640 1 -- 172.21.15.7:0/4034966512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0d3cfb70 msgr2=0x55bb0d448990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.242+0000 7fc7a3fff640 1 --2- 172.21.15.7:0/4034966512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0d3cfb70 0x55bb0d448990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.242+0000 7fc7a3fff640 1 -- 172.21.15.7:0/4034966512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb0d3caa00 msgr2=0x55bb0d3d8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.242+0000 7fc7a3fff640 1 --2- 172.21.15.7:0/4034966512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb0d3caa00 0x55bb0d3d8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.242+0000 7fc7a3fff640 1 -- 172.21.15.7:0/4034966512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb0d448ed0 con 0x55bb0d3ca010 2024-06-10T01:07:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.242+0000 7fc7a37fe640 1 --2- 172.21.15.7:0/4034966512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb0d3caa00 0x55bb0d3d8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.242+0000 7fc7a3fff640 1 --2- 172.21.15.7:0/4034966512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb0d3ca010 0x55bb0d3d85b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc794097f00 tx=0x7fc794014400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5423fa30bba395b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.242+0000 7fc7837fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:07:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.243+0000 7fc7837fe640 1 -- 172.21.15.7:0/4034966512 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79409ce60 con 0x55bb0d3ca010 2024-06-10T01:07:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.243+0000 7fc7837fe640 1 -- 172.21.15.7:0/4034966512 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc79400fb30 con 0x55bb0d3ca010 2024-06-10T01:07:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.243+0000 7fc7ad821740 1 -- 172.21.15.7:0/4034966512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc798003b50 con 0x55bb0d3ca010 2024-06-10T01:07:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.243+0000 7fc7837fe640 1 -- 172.21.15.7:0/4034966512 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc79409bb30 con 0x55bb0d3ca010 2024-06-10T01:07:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.243+0000 7fc7ad821740 10 client.15954 fetch_fsmap learned FSMap version 29 2024-06-10T01:07:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.243+0000 7fc7ad821740 1 -- 172.21.15.7:0/4034966512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bb0d44a190 con 0x55bb0d3ca010 2024-06-10T01:07:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.243+0000 7fc7837fe640 1 -- 172.21.15.7:0/4034966512 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc798003b50 con 0x55bb0d3ca010 2024-06-10T01:07:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.243+0000 7fc7837fe640 1 -- 172.21.15.7:0/4034966512 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fc794098110 con 0x55bb0d3ca010 2024-06-10T01:07:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.243+0000 7fc7ad821740 10 client.15954 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:07:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.243+0000 7fc7ad821740 10 client.15954 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.243+0000 7fc7ad821740 20 client.15954 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.243+0000 7fc7ad821740 1 --2- 172.21.15.7:0/4034966512 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55bb0d44aaa0 0x55bb0d44cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.243+0000 7fc7ad821740 4 client.15954 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:07:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.243+0000 7fc7ad821740 1 -- 172.21.15.7:0/4034966512 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bb0d33ac20 con 0x55bb0d44aaa0 2024-06-10T01:07:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.243+0000 7fc7a37fe640 1 --2- 172.21.15.7:0/4034966512 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55bb0d44aaa0 0x55bb0d44cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.244+0000 7fc7837fe640 10 client.15954 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.244+0000 7fc7a37fe640 1 --2- 172.21.15.7:0/4034966512 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55bb0d44aaa0 0x55bb0d44cec0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66f7d98097f35282 server_cookie=577d7e7281fc84f0 in_seq=0 out_seq=0 2024-06-10T01:07:49.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:48 smithi007 ceph-mon[32398]: pgmap v486: 129 pgs: 129 active+clean; 18 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 845 KiB/s wr, 115 op/s 2024-06-10T01:07:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.292+0000 7fc7837fe640 1 -- 172.21.15.7:0/4034966512 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bb0d33ac20 con 0x55bb0d44aaa0 2024-06-10T01:07:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.292+0000 7fc7837fe640 10 client.15954 handle_command_reply: tid=0 2024-06-10T01:07:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.292+0000 7fc7ad821740 1 client.15954 shutdown 2024-06-10T01:07:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.292+0000 7fc7ad821740 1 -- 172.21.15.7:0/4034966512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb0d3ca010 msgr2=0x55bb0d3d85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.292+0000 7fc7ad821740 1 --2- 172.21.15.7:0/4034966512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb0d3ca010 0x55bb0d3d85b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc794097f00 tx=0x7fc794014400 comp rx=0 tx=0).stop 2024-06-10T01:07:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.292+0000 7fc7ad821740 1 -- 172.21.15.7:0/4034966512 shutdown_connections 2024-06-10T01:07:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.292+0000 7fc7ad821740 1 --2- 172.21.15.7:0/4034966512 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55bb0d44aaa0 0x55bb0d44cec0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.292+0000 7fc7ad821740 1 --2- 172.21.15.7:0/4034966512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0d3cfb70 0x55bb0d448990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.292+0000 7fc7837fe640 0 client.15954 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.292+0000 7fc7ad821740 1 --2- 172.21.15.7:0/4034966512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb0d3caa00 0x55bb0d3d8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.292+0000 7fc7ad821740 1 --2- 172.21.15.7:0/4034966512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb0d3ca010 0x55bb0d3d85b0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.292+0000 7fc7ad821740 1 -- 172.21.15.7:0/4034966512 >> 172.21.15.7:0/4034966512 conn(0x55bb0d3c5680 msgr2=0x55bb0d3c8670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.292+0000 7fc7ad821740 1 -- 172.21.15.7:0/4034966512 shutdown_connections 2024-06-10T01:07:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.292+0000 7fc7ad821740 1 -- 172.21.15.7:0/4034966512 wait complete. 2024-06-10T01:07:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.293+0000 7fc7ad821740 20 client.15954 trim_cache size 0 max 16384 2024-06-10T01:07:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:07:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:07:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:07:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.293+0000 7fc7aaa38640 1 -- 172.21.15.7:0/1308868038 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc78c0a2b80 msgr2=0x7fc78c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.293+0000 7fc7aaa38640 1 --2- 172.21.15.7:0/1308868038 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc78c0a2b80 0x7fc78c0a5040 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc79c04c000 tx=0x7fc79c04a000 comp rx=0 tx=0).stop 2024-06-10T01:07:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.293+0000 7fc7aaa38640 1 -- 172.21.15.7:0/1308868038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7a4102500 msgr2=0x7fc7a4197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.293+0000 7fc7aaa38640 1 --2- 172.21.15.7:0/1308868038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7a4102500 0x7fc7a4197c30 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc79400c920 tx=0x7fc79400cde0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.294+0000 7fc7aaa38640 1 -- 172.21.15.7:0/1308868038 shutdown_connections 2024-06-10T01:07:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.294+0000 7fc7aaa38640 1 --2- 172.21.15.7:0/1308868038 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc78c0a2b80 0x7fc78c0a5040 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.294+0000 7fc7aaa38640 1 --2- 172.21.15.7:0/1308868038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7a4107760 0x7fc7a419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.294+0000 7fc7aaa38640 1 --2- 172.21.15.7:0/1308868038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7a4104e30 0x7fc7a4198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.294+0000 7fc7aaa38640 1 --2- 172.21.15.7:0/1308868038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7a4102500 0x7fc7a4197c30 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.294+0000 7fc7aaa38640 1 -- 172.21.15.7:0/1308868038 >> 172.21.15.7:0/1308868038 conn(0x7fc7a40fbe70 msgr2=0x7fc7a4102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.294+0000 7fc7aaa38640 1 -- 172.21.15.7:0/1308868038 shutdown_connections 2024-06-10T01:07:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:07:49.294+0000 7fc7aaa38640 1 -- 172.21.15.7:0/1308868038 wait complete. 2024-06-10T01:07:49.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:48 smithi078 ceph-mon[36683]: pgmap v486: 129 pgs: 129 active+clean; 18 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 845 KiB/s wr, 115 op/s 2024-06-10T01:07:49.708 INFO:tasks.workunit.client.0.smithi007.stdout: 1 28947 5.80 MB/sec warmup 30 sec latency 20164.359 ms 2024-06-10T01:07:49.747 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:07:49.996 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:50.014 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:49 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/4218703757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:07:50.204 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:49 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/4218703757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:07:50.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:49 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/4218703757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:07:50.709 INFO:tasks.workunit.client.0.smithi007.stdout: 1 28947 5.61 MB/sec warmup 31 sec latency 21164.389 ms 2024-06-10T01:07:50.998 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:07:51.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:50 smithi175 ceph-mon[36636]: pgmap v487: 129 pgs: 129 active+clean; 29 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 2.8 MiB/s wr, 156 op/s 2024-06-10T01:07:51.232 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:50 smithi078 ceph-mon[36683]: pgmap v487: 129 pgs: 129 active+clean; 29 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 2.8 MiB/s wr, 156 op/s 2024-06-10T01:07:51.236 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:50 smithi007 ceph-mon[32398]: pgmap v487: 129 pgs: 129 active+clean; 29 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 2.8 MiB/s wr, 156 op/s 2024-06-10T01:07:51.311 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:51.709 INFO:tasks.workunit.client.0.smithi007.stdout: 1 28947 5.44 MB/sec warmup 32 sec latency 22164.427 ms 2024-06-10T01:07:51.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.981+0000 7f3abf9c4700 1 -- 172.21.15.7:0/3750405277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ab810ede0 msgr2=0x7f3ab81111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:51.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.981+0000 7f3abf9c4700 1 --2- 172.21.15.7:0/3750405277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ab810ede0 0x7f3ab81111d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3ab000b0a0 tx=0x7f3ab000b3b0 comp rx=0 tx=0).stop 2024-06-10T01:07:51.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.981+0000 7f3abf9c4700 1 -- 172.21.15.7:0/3750405277 shutdown_connections 2024-06-10T01:07:51.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.981+0000 7f3abf9c4700 1 --2- 172.21.15.7:0/3750405277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ab810ede0 0x7f3ab81111d0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:51.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.981+0000 7f3abf9c4700 1 --2- 172.21.15.7:0/3750405277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ab81033a0 0x7f3ab806ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:51.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.981+0000 7f3abf9c4700 1 --2- 172.21.15.7:0/3750405277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ab806b0a0 0x7f3ab806b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:51.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.981+0000 7f3abf9c4700 1 -- 172.21.15.7:0/3750405277 >> 172.21.15.7:0/3750405277 conn(0x7f3ab8078340 msgr2=0x7f3ab8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:51.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.981+0000 7f3abf9c4700 1 -- 172.21.15.7:0/3750405277 shutdown_connections 2024-06-10T01:07:51.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.981+0000 7f3abf9c4700 1 -- 172.21.15.7:0/3750405277 wait complete. 2024-06-10T01:07:51.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.982+0000 7f3abf9c4700 1 Processor -- start 2024-06-10T01:07:51.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.982+0000 7f3abf9c4700 1 -- start start 2024-06-10T01:07:51.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.982+0000 7f3abf9c4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ab806b0a0 0x7f3ab819ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:51.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.982+0000 7f3abf9c4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ab81033a0 0x7f3ab819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:51.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.982+0000 7f3abf9c4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ab810ede0 0x7f3ab819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:51.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.982+0000 7f3abf9c4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81149c0 con 0x7f3ab81033a0 2024-06-10T01:07:51.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.982+0000 7f3abf9c4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab8114840 con 0x7f3ab810ede0 2024-06-10T01:07:51.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.982+0000 7f3abf9c4700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab8114b40 con 0x7f3ab806b0a0 2024-06-10T01:07:51.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.982+0000 7f3abcf5f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ab81033a0 0x7f3ab819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:51.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.982+0000 7f3abcf5f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ab81033a0 0x7f3ab819b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36170/0 (socket says 172.21.15.7:36170) 2024-06-10T01:07:51.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.982+0000 7f3abcf5f700 1 -- 172.21.15.7:0/811489612 learned_addr learned my addr 172.21.15.7:0/811489612 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:51.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.983+0000 7f3abd760700 1 --2- 172.21.15.7:0/811489612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ab806b0a0 0x7f3ab819ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:51.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.983+0000 7f3abdf61700 1 --2- 172.21.15.7:0/811489612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ab810ede0 0x7f3ab819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:51.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.983+0000 7f3abcf5f700 1 -- 172.21.15.7:0/811489612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ab806b0a0 msgr2=0x7f3ab819ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:51.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.983+0000 7f3abcf5f700 1 --2- 172.21.15.7:0/811489612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ab806b0a0 0x7f3ab819ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:51.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.983+0000 7f3abcf5f700 1 -- 172.21.15.7:0/811489612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ab810ede0 msgr2=0x7f3ab819f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:51.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.983+0000 7f3abcf5f700 1 --2- 172.21.15.7:0/811489612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ab810ede0 0x7f3ab819f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:51.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.983+0000 7f3abcf5f700 1 -- 172.21.15.7:0/811489612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab819fe60 con 0x7f3ab81033a0 2024-06-10T01:07:51.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.983+0000 7f3abdf61700 1 --2- 172.21.15.7:0/811489612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ab810ede0 0x7f3ab819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:51.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.983+0000 7f3abcf5f700 1 --2- 172.21.15.7:0/811489612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ab81033a0 0x7f3ab819b390 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800d980 tx=0x7f3aa800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:51.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.983+0000 7f3aae7fc700 1 -- 172.21.15.7:0/811489612 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa8017070 con 0x7f3ab81033a0 2024-06-10T01:07:51.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.983+0000 7f3abf9c4700 1 -- 172.21.15.7:0/811489612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab81a0150 con 0x7f3ab81033a0 2024-06-10T01:07:51.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.983+0000 7f3aae7fc700 1 -- 172.21.15.7:0/811489612 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aa8004d10 con 0x7f3ab81033a0 2024-06-10T01:07:51.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.983+0000 7f3aae7fc700 1 -- 172.21.15.7:0/811489612 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa80055c0 con 0x7f3ab81033a0 2024-06-10T01:07:51.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.983+0000 7f3abf9c4700 1 -- 172.21.15.7:0/811489612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab81a79d0 con 0x7f3ab81033a0 2024-06-10T01:07:51.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.986+0000 7f3aae7fc700 1 -- 172.21.15.7:0/811489612 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f3aa8004750 con 0x7f3ab81033a0 2024-06-10T01:07:51.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.987+0000 7f3aae7fc700 1 --2- 172.21.15.7:0/811489612 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3aa40a2ca0 0x7f3aa40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:51.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.987+0000 7f3abd760700 1 --2- 172.21.15.7:0/811489612 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3aa40a2ca0 0x7f3aa40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:51.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.987+0000 7f3aae7fc700 1 -- 172.21.15.7:0/811489612 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f3aa80cf2f0 con 0x7f3ab81033a0 2024-06-10T01:07:51.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.987+0000 7f3abd760700 1 --2- 172.21.15.7:0/811489612 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3aa40a2ca0 0x7f3aa40a5160 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4009bd0 tx=0x7f3ab4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:51.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.988+0000 7f3aca247b80 1 Processor -- start 2024-06-10T01:07:51.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.988+0000 7f3aca247b80 1 -- start start 2024-06-10T01:07:51.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.988+0000 7f3aca247b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55947f10df60 0x55947f10e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:51.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.988+0000 7f3aca247b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947f10e880 0x55947f108260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:51.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.988+0000 7f3aca247b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55947f1087a0 0x55947f108bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:51.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.988+0000 7f3aca247b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55947effaf60 con 0x55947f10df60 2024-06-10T01:07:51.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.988+0000 7f3aca247b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55947f059850 con 0x55947f1087a0 2024-06-10T01:07:51.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.988+0000 7f3aca247b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55947efc1460 con 0x55947f10e880 2024-06-10T01:07:51.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.988+0000 7f3abd760700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55947f10df60 0x55947f10e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:51.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.988+0000 7f3abd760700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55947f10df60 0x55947f10e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36172/0 (socket says 172.21.15.7:36172) 2024-06-10T01:07:51.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.988+0000 7f3abd760700 1 -- 172.21.15.7:0/1299962225 learned_addr learned my addr 172.21.15.7:0/1299962225 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:51.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.988+0000 7f3abcf5f700 1 --2- 172.21.15.7:0/1299962225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947f10e880 0x55947f108260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:51.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.988+0000 7f3abdf61700 1 --2- 172.21.15.7:0/1299962225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55947f1087a0 0x55947f108bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:51.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.988+0000 7f3abd760700 1 -- 172.21.15.7:0/1299962225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947f10e880 msgr2=0x55947f108260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:51.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.988+0000 7f3abd760700 1 --2- 172.21.15.7:0/1299962225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947f10e880 0x55947f108260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:51.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.988+0000 7f3abd760700 1 -- 172.21.15.7:0/1299962225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55947f1087a0 msgr2=0x55947f108bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:51.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.988+0000 7f3abd760700 1 --2- 172.21.15.7:0/1299962225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55947f1087a0 0x55947f108bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:51.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.988+0000 7f3abd760700 1 -- 172.21.15.7:0/1299962225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55947efa88d0 con 0x55947f10df60 2024-06-10T01:07:51.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.989+0000 7f3abdf61700 1 --2- 172.21.15.7:0/1299962225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55947f1087a0 0x55947f108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:07:51.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.989+0000 7f3abd760700 1 --2- 172.21.15.7:0/1299962225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55947f10df60 0x55947f10e340 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4005910 tx=0x7f3ab4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=696c68f6f30e70c8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:51.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.989+0000 7f3aa37fe700 1 -- 172.21.15.7:0/1299962225 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab4011db0 con 0x55947f10df60 2024-06-10T01:07:51.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.989+0000 7f3aa37fe700 1 -- 172.21.15.7:0/1299962225 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ab401e900 con 0x55947f10df60 2024-06-10T01:07:51.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.989+0000 7f3aa37fe700 1 -- 172.21.15.7:0/1299962225 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab401c870 con 0x55947f10df60 2024-06-10T01:07:51.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.990+0000 7f3aca247b80 1 -- 172.21.15.7:0/1299962225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55947f10df60 msgr2=0x55947f10e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:51.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.990+0000 7f3aca247b80 1 --2- 172.21.15.7:0/1299962225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55947f10df60 0x55947f10e340 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4005910 tx=0x7f3ab4013000 comp rx=0 tx=0).stop 2024-06-10T01:07:51.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.990+0000 7f3aca247b80 1 -- 172.21.15.7:0/1299962225 shutdown_connections 2024-06-10T01:07:51.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.990+0000 7f3aca247b80 1 --2- 172.21.15.7:0/1299962225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55947f1087a0 0x55947f108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:51.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.990+0000 7f3aca247b80 1 --2- 172.21.15.7:0/1299962225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55947f10df60 0x55947f10e340 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:51.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.990+0000 7f3aca247b80 1 --2- 172.21.15.7:0/1299962225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947f10e880 0x55947f108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:51.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.990+0000 7f3aca247b80 1 -- 172.21.15.7:0/1299962225 >> 172.21.15.7:0/1299962225 conn(0x55947f035080 msgr2=0x55947f103220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:51.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.990+0000 7f3aca247b80 1 -- 172.21.15.7:0/1299962225 shutdown_connections 2024-06-10T01:07:51.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.990+0000 7f3aca247b80 1 -- 172.21.15.7:0/1299962225 wait complete. 2024-06-10T01:07:51.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.990+0000 7f3aca247b80 1 -- start start 2024-06-10T01:07:51.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.990+0000 7f3aca247b80 1 Processor -- start 2024-06-10T01:07:51.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.991+0000 7f3aca247b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55947f1087a0 0x55947f1145a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:51.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.991+0000 7f3aca247b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947f10df60 0x55947f1118a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:51.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.991+0000 7f3aca247b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55947f10e880 0x55947f111de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:51.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.991+0000 7f3aca247b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55947eff9fc0 con 0x55947f1087a0 2024-06-10T01:07:51.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.991+0000 7f3aca247b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55947f055810 con 0x55947f10e880 2024-06-10T01:07:51.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.991+0000 7f3aca247b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55947f115260 con 0x55947f10df60 2024-06-10T01:07:51.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.991+0000 7f3abd760700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55947f1087a0 0x55947f1145a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:51.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.991+0000 7f3aca247b80 10 client.0 _ll_register_callbacks cb 0x55947efef5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:51.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.991+0000 7f3abd760700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55947f1087a0 0x55947f1145a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36174/0 (socket says 172.21.15.7:36174) 2024-06-10T01:07:51.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.991+0000 7f3abd760700 1 -- 172.21.15.7:0/3330400835 learned_addr learned my addr 172.21.15.7:0/3330400835 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:51.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.991+0000 7f3abcf5f700 1 --2- 172.21.15.7:0/3330400835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947f10df60 0x55947f1118a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:51.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.991+0000 7f3abdf61700 1 --2- 172.21.15.7:0/3330400835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55947f10e880 0x55947f111de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:51.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.991+0000 7f3abd760700 1 -- 172.21.15.7:0/3330400835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947f10df60 msgr2=0x55947f1118a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:51.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.991+0000 7f3abd760700 1 --2- 172.21.15.7:0/3330400835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947f10df60 0x55947f1118a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:51.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.991+0000 7f3abd760700 1 -- 172.21.15.7:0/3330400835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55947f10e880 msgr2=0x55947f111de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:51.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.991+0000 7f3abd760700 1 --2- 172.21.15.7:0/3330400835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55947f10e880 0x55947f111de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:51.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.991+0000 7f3abd760700 1 -- 172.21.15.7:0/3330400835 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55947f01f750 con 0x55947f1087a0 2024-06-10T01:07:51.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.991+0000 7f3abd760700 1 --2- 172.21.15.7:0/3330400835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55947f1087a0 0x55947f1145a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3ab40058e0 tx=0x7f3ab4010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2901e7c2d4263315 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:51.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.991+0000 7f3aa0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:07:51.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.992+0000 7f3aa0ff9700 1 -- 172.21.15.7:0/3330400835 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab4035060 con 0x55947f1087a0 2024-06-10T01:07:51.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.992+0000 7f3aa0ff9700 1 -- 172.21.15.7:0/3330400835 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f3ab401ecc0 con 0x55947f1087a0 2024-06-10T01:07:51.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.992+0000 7f3aca247b80 1 -- 172.21.15.7:0/3330400835 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ab401e900 con 0x55947f1087a0 2024-06-10T01:07:51.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.992+0000 7f3aa0ff9700 1 -- 172.21.15.7:0/3330400835 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3ab403c050 con 0x55947f1087a0 2024-06-10T01:07:51.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.992+0000 7f3aca247b80 10 client.15966 fetch_fsmap learned FSMap version 29 2024-06-10T01:07:51.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.992+0000 7f3aca247b80 1 -- 172.21.15.7:0/3330400835 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55947f11f930 con 0x55947f1087a0 2024-06-10T01:07:51.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.992+0000 7f3aa0ff9700 1 -- 172.21.15.7:0/3330400835 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ab401e900 con 0x55947f1087a0 2024-06-10T01:07:51.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.992+0000 7f3aa0ff9700 1 -- 172.21.15.7:0/3330400835 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f3ab4026480 con 0x55947f1087a0 2024-06-10T01:07:51.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.992+0000 7f3aca247b80 10 client.15966 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:07:51.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.992+0000 7f3aca247b80 10 client.15966 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:07:51.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.992+0000 7f3aca247b80 20 client.15966 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:51.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.992+0000 7f3aca247b80 1 --2- 172.21.15.7:0/3330400835 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55947f121f10 0x55947f124300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:51.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.992+0000 7f3aca247b80 4 client.15966 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:07:51.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.992+0000 7f3aca247b80 1 -- 172.21.15.7:0/3330400835 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55947f066d50 con 0x55947f121f10 2024-06-10T01:07:51.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.993+0000 7f3abcf5f700 1 --2- 172.21.15.7:0/3330400835 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55947f121f10 0x55947f124300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:51.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.993+0000 7f3abcf5f700 1 --2- 172.21.15.7:0/3330400835 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55947f121f10 0x55947f124300 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8434fcb454a37b2f server_cookie=10ffb17e41bd5d28 in_seq=0 out_seq=0 2024-06-10T01:07:51.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.993+0000 7f3aa0ff9700 10 client.15966 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:07:51.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.994+0000 7f3aa0ff9700 1 -- 172.21.15.7:0/3330400835 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55947f066d50 con 0x55947f121f10 2024-06-10T01:07:51.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.994+0000 7f3aa0ff9700 10 client.15966 handle_command_reply: tid=0 2024-06-10T01:07:51.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.994+0000 7f3aca247b80 1 client.15966 shutdown 2024-06-10T01:07:51.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.994+0000 7f3aca247b80 1 -- 172.21.15.7:0/3330400835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55947f1087a0 msgr2=0x55947f1145a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:51.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.994+0000 7f3aca247b80 1 --2- 172.21.15.7:0/3330400835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55947f1087a0 0x55947f1145a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3ab40058e0 tx=0x7f3ab4010f50 comp rx=0 tx=0).stop 2024-06-10T01:07:51.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.994+0000 7f3aca247b80 1 -- 172.21.15.7:0/3330400835 shutdown_connections 2024-06-10T01:07:51.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.994+0000 7f3aca247b80 1 --2- 172.21.15.7:0/3330400835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55947f10e880 0x55947f111de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:51.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.994+0000 7f3aca247b80 1 --2- 172.21.15.7:0/3330400835 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55947f121f10 0x55947f124300 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:51.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.994+0000 7f3aca247b80 1 --2- 172.21.15.7:0/3330400835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55947f1087a0 0x55947f1145a0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:51.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.994+0000 7f3aca247b80 1 --2- 172.21.15.7:0/3330400835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947f10df60 0x55947f1118a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:51.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.994+0000 7f3aa0ff9700 0 client.15966 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:07:51.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.994+0000 7f3aca247b80 1 -- 172.21.15.7:0/3330400835 >> 172.21.15.7:0/3330400835 conn(0x55947f035080 msgr2=0x55947f102700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:51.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.994+0000 7f3aca247b80 1 -- 172.21.15.7:0/3330400835 shutdown_connections 2024-06-10T01:07:51.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.994+0000 7f3aca247b80 1 -- 172.21.15.7:0/3330400835 wait complete. 2024-06-10T01:07:51.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:51.994+0000 7f3aca247b80 20 client.15966 trim_cache size 0 max 16384 2024-06-10T01:07:52.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.009+0000 7f3aca247b80 1 Processor -- start 2024-06-10T01:07:52.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.009+0000 7f3aca247b80 1 -- start start 2024-06-10T01:07:52.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.010+0000 7f3aca247b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55947f1087a0 0x55947f10f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:52.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.010+0000 7f3aca247b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55947f10df60 0x55947f1108a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:52.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.010+0000 7f3aca247b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947f10e880 0x55947f102e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:52.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.010+0000 7f3aca247b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81b2d00 con 0x55947f10df60 2024-06-10T01:07:52.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.010+0000 7f3aca247b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81b0590 con 0x55947f1087a0 2024-06-10T01:07:52.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.010+0000 7f3aca247b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81ae3e0 con 0x55947f10e880 2024-06-10T01:07:52.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.010+0000 7f3abcf5f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55947f10df60 0x55947f1108a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:52.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.010+0000 7f3abcf5f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55947f10df60 0x55947f1108a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36180/0 (socket says 172.21.15.7:36180) 2024-06-10T01:07:52.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.010+0000 7f3abcf5f700 1 -- 172.21.15.7:0/1363014022 learned_addr learned my addr 172.21.15.7:0/1363014022 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:52.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.010+0000 7f3abdf61700 1 --2- 172.21.15.7:0/1363014022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947f10e880 0x55947f102e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:52.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.010+0000 7f3abd760700 1 --2- 172.21.15.7:0/1363014022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55947f1087a0 0x55947f10f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:52.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.010+0000 7f3abcf5f700 1 -- 172.21.15.7:0/1363014022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947f10e880 msgr2=0x55947f102e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:52.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.010+0000 7f3abcf5f700 1 --2- 172.21.15.7:0/1363014022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947f10e880 0x55947f102e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:52.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.010+0000 7f3abcf5f700 1 -- 172.21.15.7:0/1363014022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55947f1087a0 msgr2=0x55947f10f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:52.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.010+0000 7f3abcf5f700 1 --2- 172.21.15.7:0/1363014022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55947f1087a0 0x55947f10f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:52.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.010+0000 7f3abcf5f700 1 -- 172.21.15.7:0/1363014022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55947f10fe10 con 0x55947f10df60 2024-06-10T01:07:52.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.010+0000 7f3abd760700 1 --2- 172.21.15.7:0/1363014022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55947f1087a0 0x55947f10f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:07:52.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.010+0000 7f3abcf5f700 1 --2- 172.21.15.7:0/1363014022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55947f10df60 0x55947f1108a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3aa80981a0 tx=0x7f3aa80cc940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d95e12d7fc023a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:52.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.010+0000 7f3aa2ffd700 1 -- 172.21.15.7:0/1363014022 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa80b3070 con 0x55947f10df60 2024-06-10T01:07:52.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.010+0000 7f3aa2ffd700 1 -- 172.21.15.7:0/1363014022 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aa809b7d0 con 0x55947f10df60 2024-06-10T01:07:52.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.010+0000 7f3aa2ffd700 1 -- 172.21.15.7:0/1363014022 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa809ba30 con 0x55947f10df60 2024-06-10T01:07:52.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.011+0000 7f3aca247b80 1 -- 172.21.15.7:0/1363014022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55947f10df60 msgr2=0x55947f1108a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:52.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.011+0000 7f3aca247b80 1 --2- 172.21.15.7:0/1363014022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55947f10df60 0x55947f1108a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3aa80981a0 tx=0x7f3aa80cc940 comp rx=0 tx=0).stop 2024-06-10T01:07:52.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.011+0000 7f3aca247b80 1 -- 172.21.15.7:0/1363014022 shutdown_connections 2024-06-10T01:07:52.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.011+0000 7f3aca247b80 1 --2- 172.21.15.7:0/1363014022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947f10e880 0x55947f102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:52.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.011+0000 7f3aca247b80 1 --2- 172.21.15.7:0/1363014022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55947f1087a0 0x55947f10f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:52.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.011+0000 7f3aca247b80 1 --2- 172.21.15.7:0/1363014022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55947f10df60 0x55947f1108a0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:52.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.011+0000 7f3aca247b80 1 -- 172.21.15.7:0/1363014022 >> 172.21.15.7:0/1363014022 conn(0x55947f035080 msgr2=0x55947f110190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:52.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.011+0000 7f3aca247b80 1 -- 172.21.15.7:0/1363014022 shutdown_connections 2024-06-10T01:07:52.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.011+0000 7f3aca247b80 1 -- 172.21.15.7:0/1363014022 wait complete. 2024-06-10T01:07:52.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.011+0000 7f3aca247b80 1 -- start start 2024-06-10T01:07:52.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.012+0000 7f3aca247b80 1 Processor -- start 2024-06-10T01:07:52.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.012+0000 7f3aca247b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55947f1087a0 0x55947f11aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:52.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.012+0000 7f3aca247b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55947f10df60 0x55947f115dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:52.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.012+0000 7f3aca247b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947f10e880 0x55947f116310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:52.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.012+0000 7f3aca247b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81aea50 con 0x55947f1087a0 2024-06-10T01:07:52.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.012+0000 7f3aca247b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81afec0 con 0x55947f10df60 2024-06-10T01:07:52.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.012+0000 7f3aca247b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81af120 con 0x55947f10e880 2024-06-10T01:07:52.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.012+0000 7f3aca247b80 10 client.0 _ll_register_callbacks cb 0x55947efef5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:52.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.012+0000 7f3abd760700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55947f1087a0 0x55947f11aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:52.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.012+0000 7f3abd760700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55947f1087a0 0x55947f11aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36188/0 (socket says 172.21.15.7:36188) 2024-06-10T01:07:52.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.012+0000 7f3abd760700 1 -- 172.21.15.7:0/1489505151 learned_addr learned my addr 172.21.15.7:0/1489505151 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:52.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.012+0000 7f3abcf5f700 1 --2- 172.21.15.7:0/1489505151 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55947f10df60 0x55947f115dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:52.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.012+0000 7f3abd760700 1 -- 172.21.15.7:0/1489505151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947f10e880 msgr2=0x55947f116310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:52.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.012+0000 7f3abd760700 1 --2- 172.21.15.7:0/1489505151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947f10e880 0x55947f116310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:52.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.012+0000 7f3abd760700 1 -- 172.21.15.7:0/1489505151 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55947f10df60 msgr2=0x55947f115dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:52.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.012+0000 7f3abd760700 1 --2- 172.21.15.7:0/1489505151 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55947f10df60 0x55947f115dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:52.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.012+0000 7f3abd760700 1 -- 172.21.15.7:0/1489505151 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55947f1168d0 con 0x55947f1087a0 2024-06-10T01:07:52.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.013+0000 7f3abd760700 1 --2- 172.21.15.7:0/1489505151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55947f1087a0 0x55947f11aa50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4009fd0 tx=0x7f3ab401e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4269289a822e16e1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:52.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.013+0000 7f3aa0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:07:52.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.013+0000 7f3aa0ff9700 1 -- 172.21.15.7:0/1489505151 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab402a030 con 0x55947f1087a0 2024-06-10T01:07:52.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.013+0000 7f3aa0ff9700 1 -- 172.21.15.7:0/1489505151 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f3ab40263f0 con 0x55947f1087a0 2024-06-10T01:07:52.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.013+0000 7f3aca247b80 1 -- 172.21.15.7:0/1489505151 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aa809b7d0 con 0x55947f1087a0 2024-06-10T01:07:52.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.013+0000 7f3aa0ff9700 1 -- 172.21.15.7:0/1489505151 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3ab4049050 con 0x55947f1087a0 2024-06-10T01:07:52.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.013+0000 7f3aca247b80 10 client.15972 fetch_fsmap learned FSMap version 29 2024-06-10T01:07:52.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.013+0000 7f3aca247b80 1 -- 172.21.15.7:0/1489505151 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55947f18c640 con 0x55947f1087a0 2024-06-10T01:07:52.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.013+0000 7f3aa0ff9700 1 -- 172.21.15.7:0/1489505151 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aa809b7d0 con 0x55947f1087a0 2024-06-10T01:07:52.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.013+0000 7f3aa0ff9700 1 -- 172.21.15.7:0/1489505151 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f3ab4040910 con 0x55947f1087a0 2024-06-10T01:07:52.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.013+0000 7f3aca247b80 10 client.15972 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:07:52.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.013+0000 7f3aca247b80 10 client.15972 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:07:52.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.013+0000 7f3aca247b80 20 client.15972 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:52.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.013+0000 7f3aca247b80 1 --2- 172.21.15.7:0/1489505151 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55947f18cd20 0x55947f18f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:52.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.014+0000 7f3aca247b80 4 client.15972 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:07:52.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.014+0000 7f3aca247b80 1 -- 172.21.15.7:0/1489505151 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55947f0612d0 con 0x55947f18cd20 2024-06-10T01:07:52.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.014+0000 7f3abcf5f700 1 --2- 172.21.15.7:0/1489505151 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55947f18cd20 0x55947f18f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:52.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.014+0000 7f3abcf5f700 1 --2- 172.21.15.7:0/1489505151 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55947f18cd20 0x55947f18f180 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=75411bb73602aa10 server_cookie=29ff24a1b382bbc in_seq=0 out_seq=0 2024-06-10T01:07:52.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.014+0000 7f3aa0ff9700 10 client.15972 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:07:52.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.014+0000 7f3aa0ff9700 1 -- 172.21.15.7:0/1489505151 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55947f0612d0 con 0x55947f18cd20 2024-06-10T01:07:52.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.014+0000 7f3aa0ff9700 10 client.15972 handle_command_reply: tid=0 2024-06-10T01:07:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.014+0000 7f3aca247b80 1 client.15972 shutdown 2024-06-10T01:07:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.014+0000 7f3aca247b80 1 -- 172.21.15.7:0/1489505151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55947f1087a0 msgr2=0x55947f11aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.014+0000 7f3aca247b80 1 --2- 172.21.15.7:0/1489505151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55947f1087a0 0x55947f11aa50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4009fd0 tx=0x7f3ab401e9e0 comp rx=0 tx=0).stop 2024-06-10T01:07:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.014+0000 7f3aca247b80 1 -- 172.21.15.7:0/1489505151 shutdown_connections 2024-06-10T01:07:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.014+0000 7f3aca247b80 1 --2- 172.21.15.7:0/1489505151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55947f10e880 0x55947f116310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.014+0000 7f3aca247b80 1 --2- 172.21.15.7:0/1489505151 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55947f18cd20 0x55947f18f180 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.014+0000 7f3aca247b80 1 --2- 172.21.15.7:0/1489505151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55947f1087a0 0x55947f11aa50 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.015+0000 7f3aca247b80 1 --2- 172.21.15.7:0/1489505151 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55947f10df60 0x55947f115dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.015+0000 7f3aca247b80 1 -- 172.21.15.7:0/1489505151 >> 172.21.15.7:0/1489505151 conn(0x55947f035080 msgr2=0x55947f107ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.015+0000 7f3aa0ff9700 0 client.15972 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:07:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.015+0000 7f3aca247b80 1 -- 172.21.15.7:0/1489505151 shutdown_connections 2024-06-10T01:07:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.015+0000 7f3aca247b80 1 -- 172.21.15.7:0/1489505151 wait complete. 2024-06-10T01:07:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.015+0000 7f3aca247b80 20 client.15972 trim_cache size 0 max 16384 2024-06-10T01:07:52.017 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:07:52.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.015+0000 7f3abf9c4700 1 -- 172.21.15.7:0/811489612 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3aa40a2ca0 msgr2=0x7f3aa40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:52.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.015+0000 7f3abf9c4700 1 --2- 172.21.15.7:0/811489612 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3aa40a2ca0 0x7f3aa40a5160 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4009bd0 tx=0x7f3ab4009260 comp rx=0 tx=0).stop 2024-06-10T01:07:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.015+0000 7f3abf9c4700 1 -- 172.21.15.7:0/811489612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ab81033a0 msgr2=0x7f3ab819b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.015+0000 7f3abf9c4700 1 --2- 172.21.15.7:0/811489612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ab81033a0 0x7f3ab819b390 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800d980 tx=0x7f3aa800dd40 comp rx=0 tx=0).stop 2024-06-10T01:07:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.015+0000 7f3abf9c4700 1 -- 172.21.15.7:0/811489612 shutdown_connections 2024-06-10T01:07:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.015+0000 7f3abf9c4700 1 --2- 172.21.15.7:0/811489612 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3aa40a2ca0 0x7f3aa40a5160 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.015+0000 7f3abf9c4700 1 --2- 172.21.15.7:0/811489612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ab810ede0 0x7f3ab819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.015+0000 7f3abf9c4700 1 --2- 172.21.15.7:0/811489612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ab806b0a0 0x7f3ab819ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.015+0000 7f3abf9c4700 1 --2- 172.21.15.7:0/811489612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ab81033a0 0x7f3ab819b390 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.015+0000 7f3abf9c4700 1 -- 172.21.15.7:0/811489612 >> 172.21.15.7:0/811489612 conn(0x7f3ab8078340 msgr2=0x7f3ab8100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.016+0000 7f3abf9c4700 1 -- 172.21.15.7:0/811489612 shutdown_connections 2024-06-10T01:07:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:52.016+0000 7f3abf9c4700 1 -- 172.21.15.7:0/811489612 wait complete. 2024-06-10T01:07:52.709 INFO:tasks.workunit.client.0.smithi007.stdout: 1 28947 5.27 MB/sec warmup 33 sec latency 23164.538 ms 2024-06-10T01:07:53.204 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:52 smithi175 ceph-mon[36636]: pgmap v488: 129 pgs: 129 active+clean; 27 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 3.2 MiB/s wr, 133 op/s 2024-06-10T01:07:53.246 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:52 smithi007 ceph-mon[32398]: pgmap v488: 129 pgs: 129 active+clean; 27 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 3.2 MiB/s wr, 133 op/s 2024-06-10T01:07:53.285 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:07:53.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:52 smithi078 ceph-mon[36683]: pgmap v488: 129 pgs: 129 active+clean; 27 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 3.2 MiB/s wr, 133 op/s 2024-06-10T01:07:53.711 INFO:tasks.workunit.client.0.smithi007.stdout: 1 28947 5.12 MB/sec warmup 34 sec latency 24164.564 ms 2024-06-10T01:07:54.226 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.239+0000 7efeef0e7700 1 -- 172.21.15.7:0/1293584554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efee81019d0 msgr2=0x7efee8101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.239+0000 7efeef0e7700 1 --2- 172.21.15.7:0/1293584554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efee81019d0 0x7efee8101db0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7efedc0099c0 tx=0x7efedc009cd0 comp rx=0 tx=0).stop 2024-06-10T01:07:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.239+0000 7efeef0e7700 1 -- 172.21.15.7:0/1293584554 shutdown_connections 2024-06-10T01:07:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.239+0000 7efeef0e7700 1 --2- 172.21.15.7:0/1293584554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efee8106720 0x7efee810e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.239+0000 7efeef0e7700 1 --2- 172.21.15.7:0/1293584554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efee81019d0 0x7efee8101db0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.239+0000 7efeef0e7700 1 --2- 172.21.15.7:0/1293584554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efee8102380 0x7efee8105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.239+0000 7efeef0e7700 1 -- 172.21.15.7:0/1293584554 >> 172.21.15.7:0/1293584554 conn(0x7efee8078340 msgr2=0x7efee8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.240+0000 7efeef0e7700 1 -- 172.21.15.7:0/1293584554 shutdown_connections 2024-06-10T01:07:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.240+0000 7efeef0e7700 1 -- 172.21.15.7:0/1293584554 wait complete. 2024-06-10T01:07:54.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.240+0000 7efeef0e7700 1 Processor -- start 2024-06-10T01:07:54.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.240+0000 7efeef0e7700 1 -- start start 2024-06-10T01:07:54.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.241+0000 7efeef0e7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efee81019d0 0x7efee8199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:54.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.241+0000 7efeef0e7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efee8102380 0x7efee819a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:54.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.241+0000 7efeef0e7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efee8106720 0x7efee81a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:54.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.241+0000 7efeef0e7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efee81116f0 con 0x7efee8106720 2024-06-10T01:07:54.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.241+0000 7efeef0e7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efee8111570 con 0x7efee8102380 2024-06-10T01:07:54.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.241+0000 7efeef0e7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efee8111870 con 0x7efee81019d0 2024-06-10T01:07:54.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.241+0000 7efeed684700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efee8106720 0x7efee81a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:54.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.241+0000 7efeed684700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efee8106720 0x7efee81a4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36206/0 (socket says 172.21.15.7:36206) 2024-06-10T01:07:54.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.241+0000 7efeed684700 1 -- 172.21.15.7:0/334223923 learned_addr learned my addr 172.21.15.7:0/334223923 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:54.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.241+0000 7efee7fff700 1 --2- 172.21.15.7:0/334223923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efee8102380 0x7efee819a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:54.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.241+0000 7efeece83700 1 --2- 172.21.15.7:0/334223923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efee81019d0 0x7efee8199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:54.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.241+0000 7efeed684700 1 -- 172.21.15.7:0/334223923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efee81019d0 msgr2=0x7efee8199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:54.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.241+0000 7efeed684700 1 --2- 172.21.15.7:0/334223923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efee81019d0 0x7efee8199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:54.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.241+0000 7efeed684700 1 -- 172.21.15.7:0/334223923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efee8102380 msgr2=0x7efee819a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:54.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.241+0000 7efeed684700 1 --2- 172.21.15.7:0/334223923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efee8102380 0x7efee819a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:54.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.241+0000 7efeed684700 1 -- 172.21.15.7:0/334223923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee81a50b0 con 0x7efee8106720 2024-06-10T01:07:54.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.242+0000 7efeed684700 1 --2- 172.21.15.7:0/334223923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efee8106720 0x7efee81a4b70 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7efee000ef30 tx=0x7efee000c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:54.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.242+0000 7efee5ffb700 1 -- 172.21.15.7:0/334223923 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee00092e0 con 0x7efee8106720 2024-06-10T01:07:54.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.242+0000 7efeef0e7700 1 -- 172.21.15.7:0/334223923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee81a5280 con 0x7efee8106720 2024-06-10T01:07:54.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.242+0000 7efee5ffb700 1 -- 172.21.15.7:0/334223923 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efee0004bb0 con 0x7efee8106720 2024-06-10T01:07:54.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.242+0000 7efee5ffb700 1 -- 172.21.15.7:0/334223923 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee0004280 con 0x7efee8106720 2024-06-10T01:07:54.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.242+0000 7efeef0e7700 1 -- 172.21.15.7:0/334223923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee81a5810 con 0x7efee8106720 2024-06-10T01:07:54.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.243+0000 7efeef0e7700 1 -- 172.21.15.7:0/334223923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efee8102ad0 con 0x7efee8106720 2024-06-10T01:07:54.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.244+0000 7efee5ffb700 1 -- 172.21.15.7:0/334223923 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7efee001b030 con 0x7efee8106720 2024-06-10T01:07:54.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.245+0000 7efee5ffb700 1 --2- 172.21.15.7:0/334223923 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efed80a2c20 0x7efed80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:54.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.245+0000 7efee5ffb700 1 -- 172.21.15.7:0/334223923 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7efee00cef30 con 0x7efee8106720 2024-06-10T01:07:54.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.245+0000 7efeece83700 1 --2- 172.21.15.7:0/334223923 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efed80a2c20 0x7efed80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:54.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.245+0000 7efeece83700 1 --2- 172.21.15.7:0/334223923 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efed80a2c20 0x7efed80a50e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efedc000c00 tx=0x7efedc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:54.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.250+0000 7efee5ffb700 1 -- 172.21.15.7:0/334223923 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efee0022020 con 0x7efee8106720 2024-06-10T01:07:54.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.505+0000 7efeef0e7700 1 -- 172.21.15.7:0/334223923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efee802dca0 con 0x7efee8106720 2024-06-10T01:07:54.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.506+0000 7efee5ffb700 1 -- 172.21.15.7:0/334223923 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11855 (secure 0 0 0) 0x7efee0010040 con 0x7efee8106720 2024-06-10T01:07:54.502 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:07:54.503 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:07:36.881518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:07:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.509+0000 7efeef0e7700 1 -- 172.21.15.7:0/334223923 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efed80a2c20 msgr2=0x7efed80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.509+0000 7efeef0e7700 1 --2- 172.21.15.7:0/334223923 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efed80a2c20 0x7efed80a50e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efedc000c00 tx=0x7efedc01a040 comp rx=0 tx=0).stop 2024-06-10T01:07:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.509+0000 7efeef0e7700 1 -- 172.21.15.7:0/334223923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efee8106720 msgr2=0x7efee81a4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.509+0000 7efeef0e7700 1 --2- 172.21.15.7:0/334223923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efee8106720 0x7efee81a4b70 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7efee000ef30 tx=0x7efee000c450 comp rx=0 tx=0).stop 2024-06-10T01:07:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.509+0000 7efeef0e7700 1 -- 172.21.15.7:0/334223923 shutdown_connections 2024-06-10T01:07:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.509+0000 7efeef0e7700 1 --2- 172.21.15.7:0/334223923 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efed80a2c20 0x7efed80a50e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.509+0000 7efeef0e7700 1 --2- 172.21.15.7:0/334223923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efee8106720 0x7efee81a4b70 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.509+0000 7efeef0e7700 1 --2- 172.21.15.7:0/334223923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efee81019d0 0x7efee8199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.509+0000 7efeef0e7700 1 --2- 172.21.15.7:0/334223923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efee8102380 0x7efee819a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.509+0000 7efeef0e7700 1 -- 172.21.15.7:0/334223923 >> 172.21.15.7:0/334223923 conn(0x7efee8078340 msgr2=0x7efee8100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.509+0000 7efeef0e7700 1 -- 172.21.15.7:0/334223923 shutdown_connections 2024-06-10T01:07:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:54.509+0000 7efeef0e7700 1 -- 172.21.15.7:0/334223923 wait complete. 2024-06-10T01:07:54.506 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 29 2024-06-10T01:07:54.709 INFO:tasks.workunit.client.0.smithi007.stdout: 1 28947 4.97 MB/sec warmup 35 sec latency 25164.591 ms 2024-06-10T01:07:55.204 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:54 smithi175 ceph-mon[36636]: pgmap v489: 129 pgs: 129 active+clean; 25 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 3.2 MiB/s wr, 120 op/s 2024-06-10T01:07:55.204 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:54 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/334223923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:07:55.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:54 smithi007 ceph-mon[32398]: pgmap v489: 129 pgs: 129 active+clean; 25 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 3.2 MiB/s wr, 120 op/s 2024-06-10T01:07:55.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:54 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/334223923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:07:55.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:54 smithi078 ceph-mon[36683]: pgmap v489: 129 pgs: 129 active+clean; 25 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 3.2 MiB/s wr, 120 op/s 2024-06-10T01:07:55.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:54 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/334223923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:07:55.709 INFO:tasks.workunit.client.0.smithi007.stdout: 1 28947 4.83 MB/sec warmup 36 sec latency 26164.640 ms 2024-06-10T01:07:55.710 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db --set-id 19da4be8 --release --await' 2024-06-10T01:07:55.710 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db --set-id 19da4be8 --release --await 2024-06-10T01:07:56.614 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:56.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.654+0000 7f39c34f7700 1 -- 172.21.15.7:0/3437358486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39bc101380 msgr2=0x7f39bc101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:56.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.654+0000 7f39c34f7700 1 --2- 172.21.15.7:0/3437358486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39bc101380 0x7f39bc101800 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009990 tx=0x7f39ac009ca0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.654+0000 7f39c34f7700 1 -- 172.21.15.7:0/3437358486 shutdown_connections 2024-06-10T01:07:56.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.654+0000 7f39c34f7700 1 --2- 172.21.15.7:0/3437358486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39bc101d40 0x7f39bc10e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.654+0000 7f39c34f7700 1 --2- 172.21.15.7:0/3437358486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39bc1073f0 0x7f39bc1077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.654+0000 7f39c34f7700 1 --2- 172.21.15.7:0/3437358486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39bc101380 0x7f39bc101800 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.654+0000 7f39c34f7700 1 -- 172.21.15.7:0/3437358486 >> 172.21.15.7:0/3437358486 conn(0x7f39bc078300 msgr2=0x7f39bc078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:56.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.655+0000 7f39c34f7700 1 -- 172.21.15.7:0/3437358486 shutdown_connections 2024-06-10T01:07:56.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.655+0000 7f39c34f7700 1 -- 172.21.15.7:0/3437358486 wait complete. 2024-06-10T01:07:56.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.655+0000 7f39c34f7700 1 Processor -- start 2024-06-10T01:07:56.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.655+0000 7f39c34f7700 1 -- start start 2024-06-10T01:07:56.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.655+0000 7f39c34f7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39bc101380 0x7f39bc103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.656+0000 7f39c34f7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39bc101d40 0x7f39bc1035a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.656+0000 7f39c34f7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39bc1073f0 0x7f39bc103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.656+0000 7f39c34f7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc111660 con 0x7f39bc1073f0 2024-06-10T01:07:56.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.656+0000 7f39c34f7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc1114e0 con 0x7f39bc101380 2024-06-10T01:07:56.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.656+0000 7f39c34f7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc1117e0 con 0x7f39bc101d40 2024-06-10T01:07:56.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.656+0000 7f39c1a94700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39bc1073f0 0x7f39bc103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.656+0000 7f39c1293700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39bc101380 0x7f39bc103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.656+0000 7f39c1a94700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39bc1073f0 0x7f39bc103ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36216/0 (socket says 172.21.15.7:36216) 2024-06-10T01:07:56.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.656+0000 7f39c1a94700 1 -- 172.21.15.7:0/2152214698 learned_addr learned my addr 172.21.15.7:0/2152214698 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:56.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.656+0000 7f39c0a92700 1 --2- 172.21.15.7:0/2152214698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39bc101d40 0x7f39bc1035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.656+0000 7f39c1293700 1 -- 172.21.15.7:0/2152214698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39bc101d40 msgr2=0x7f39bc1035a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:56.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.656+0000 7f39c1293700 1 --2- 172.21.15.7:0/2152214698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39bc101d40 0x7f39bc1035a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.656+0000 7f39c1293700 1 -- 172.21.15.7:0/2152214698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39bc1073f0 msgr2=0x7f39bc103ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:56.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.656+0000 7f39c1293700 1 --2- 172.21.15.7:0/2152214698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39bc1073f0 0x7f39bc103ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.656+0000 7f39c1293700 1 -- 172.21.15.7:0/2152214698 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39bc1a7160 con 0x7f39bc101380 2024-06-10T01:07:56.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.656+0000 7f39c1a94700 1 --2- 172.21.15.7:0/2152214698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39bc1073f0 0x7f39bc103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:56.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.656+0000 7f39c1293700 1 --2- 172.21.15.7:0/2152214698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39bc101380 0x7f39bc103060 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f39b800c970 tx=0x7f39b800cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:56.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.657+0000 7f39b27fc700 1 -- 172.21.15.7:0/2152214698 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b8004a30 con 0x7f39bc101380 2024-06-10T01:07:56.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.657+0000 7f39c34f7700 1 -- 172.21.15.7:0/2152214698 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39bc1a7450 con 0x7f39bc101380 2024-06-10T01:07:56.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.657+0000 7f39b27fc700 1 -- 172.21.15.7:0/2152214698 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39b8005500 con 0x7f39bc101380 2024-06-10T01:07:56.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.657+0000 7f39b27fc700 1 -- 172.21.15.7:0/2152214698 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b800ec30 con 0x7f39bc101380 2024-06-10T01:07:56.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.657+0000 7f39c34f7700 1 -- 172.21.15.7:0/2152214698 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39bc1a79b0 con 0x7f39bc101380 2024-06-10T01:07:56.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.660+0000 7f39b27fc700 1 -- 172.21.15.7:0/2152214698 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f39b8004bd0 con 0x7f39bc101380 2024-06-10T01:07:56.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.661+0000 7f39b27fc700 1 --2- 172.21.15.7:0/2152214698 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f39a80a2c20 0x7f39a80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.661+0000 7f39c0a92700 1 --2- 172.21.15.7:0/2152214698 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f39a80a2c20 0x7f39a80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.661+0000 7f39b27fc700 1 -- 172.21.15.7:0/2152214698 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f39b80cedc0 con 0x7f39bc101380 2024-06-10T01:07:56.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.661+0000 7f39c0a92700 1 --2- 172.21.15.7:0/2152214698 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f39a80a2c20 0x7f39a80a50e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00b5c0 tx=0x7f39ac005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:56.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.662+0000 7f39cdd7ab80 1 Processor -- start 2024-06-10T01:07:56.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.662+0000 7f39cdd7ab80 1 -- start start 2024-06-10T01:07:56.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.662+0000 7f39cdd7ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564411016640 0x564411016a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.662+0000 7f39cdd7ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564411016f60 0x5644110109e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.662+0000 7f39cdd7ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564411010f20 0x564411011340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.662+0000 7f39cdd7ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564410f62810 con 0x564411016f60 2024-06-10T01:07:56.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.662+0000 7f39cdd7ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564410f0a720 con 0x564411010f20 2024-06-10T01:07:56.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.662+0000 7f39cdd7ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564410f5e7d0 con 0x564411016640 2024-06-10T01:07:56.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.662+0000 7f39c0a92700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564411016f60 0x5644110109e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.662+0000 7f39c0a92700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564411016f60 0x5644110109e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36230/0 (socket says 172.21.15.7:36230) 2024-06-10T01:07:56.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.662+0000 7f39c0a92700 1 -- 172.21.15.7:0/3771803260 learned_addr learned my addr 172.21.15.7:0/3771803260 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:56.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.662+0000 7f39c1a94700 1 --2- 172.21.15.7:0/3771803260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564411010f20 0x564411011340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.662+0000 7f39c1293700 1 --2- 172.21.15.7:0/3771803260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564411016640 0x564411016a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.662+0000 7f39c0a92700 1 -- 172.21.15.7:0/3771803260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564411016640 msgr2=0x564411016a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:56.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.662+0000 7f39c0a92700 1 --2- 172.21.15.7:0/3771803260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564411016640 0x564411016a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.662+0000 7f39c0a92700 1 -- 172.21.15.7:0/3771803260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564411010f20 msgr2=0x564411011340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:56.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.662+0000 7f39c0a92700 1 --2- 172.21.15.7:0/3771803260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564411010f20 0x564411011340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.662+0000 7f39c0a92700 1 -- 172.21.15.7:0/3771803260 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564410f00f50 con 0x564411016f60 2024-06-10T01:07:56.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.662+0000 7f39c1a94700 1 --2- 172.21.15.7:0/3771803260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564411010f20 0x564411011340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:56.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.663+0000 7f39c0a92700 1 --2- 172.21.15.7:0/3771803260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564411016f60 0x5644110109e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f39ac015000 tx=0x7f39ac020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa5642c2812d103 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:56.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.663+0000 7f39a77fe700 1 -- 172.21.15.7:0/3771803260 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ac00be50 con 0x564411016f60 2024-06-10T01:07:56.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.663+0000 7f39a77fe700 1 -- 172.21.15.7:0/3771803260 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39ac00fd50 con 0x564411016f60 2024-06-10T01:07:56.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.663+0000 7f39a77fe700 1 -- 172.21.15.7:0/3771803260 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ac02f400 con 0x564411016f60 2024-06-10T01:07:56.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.664+0000 7f39cdd7ab80 1 -- 172.21.15.7:0/3771803260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564411016f60 msgr2=0x5644110109e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:56.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.664+0000 7f39cdd7ab80 1 --2- 172.21.15.7:0/3771803260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564411016f60 0x5644110109e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f39ac015000 tx=0x7f39ac020000 comp rx=0 tx=0).stop 2024-06-10T01:07:56.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.664+0000 7f39cdd7ab80 1 -- 172.21.15.7:0/3771803260 shutdown_connections 2024-06-10T01:07:56.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.664+0000 7f39cdd7ab80 1 --2- 172.21.15.7:0/3771803260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564411010f20 0x564411011340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.664+0000 7f39cdd7ab80 1 --2- 172.21.15.7:0/3771803260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564411016640 0x564411016a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.664+0000 7f39cdd7ab80 1 --2- 172.21.15.7:0/3771803260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564411016f60 0x5644110109e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.664+0000 7f39cdd7ab80 1 -- 172.21.15.7:0/3771803260 >> 172.21.15.7:0/3771803260 conn(0x564410f3bcc0 msgr2=0x56441100b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:56.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.664+0000 7f39cdd7ab80 1 -- 172.21.15.7:0/3771803260 shutdown_connections 2024-06-10T01:07:56.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.664+0000 7f39cdd7ab80 1 -- 172.21.15.7:0/3771803260 wait complete. 2024-06-10T01:07:56.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.664+0000 7f39cdd7ab80 1 -- start start 2024-06-10T01:07:56.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.664+0000 7f39cdd7ab80 1 Processor -- start 2024-06-10T01:07:56.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.665+0000 7f39cdd7ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564411010f20 0x56441101cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.665+0000 7f39cdd7ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564411016640 0x56441101a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.665+0000 7f39cdd7ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564411016f60 0x56441101a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.665+0000 7f39cdd7ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56441101d910 con 0x564411016f60 2024-06-10T01:07:56.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.665+0000 7f39cdd7ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564410f02bf0 con 0x564411010f20 2024-06-10T01:07:56.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.665+0000 7f39cdd7ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56441101da90 con 0x564411016640 2024-06-10T01:07:56.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.665+0000 7f39cdd7ab80 10 client.0 _ll_register_callbacks cb 0x564410efc920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:56.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.665+0000 7f39c1a94700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564411016f60 0x56441101a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.665+0000 7f39c1a94700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564411016f60 0x56441101a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36234/0 (socket says 172.21.15.7:36234) 2024-06-10T01:07:56.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.665+0000 7f39c1a94700 1 -- 172.21.15.7:0/3500990908 learned_addr learned my addr 172.21.15.7:0/3500990908 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:56.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.665+0000 7f39c1293700 1 --2- 172.21.15.7:0/3500990908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564411010f20 0x56441101cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.665+0000 7f39c0a92700 1 --2- 172.21.15.7:0/3500990908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564411016640 0x56441101a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.665+0000 7f39c1a94700 1 -- 172.21.15.7:0/3500990908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564411016640 msgr2=0x56441101a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:56.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.665+0000 7f39c1a94700 1 --2- 172.21.15.7:0/3500990908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564411016640 0x56441101a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.665+0000 7f39c1a94700 1 -- 172.21.15.7:0/3500990908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564411010f20 msgr2=0x56441101cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:56.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.665+0000 7f39c1a94700 1 --2- 172.21.15.7:0/3500990908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564411010f20 0x56441101cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.665+0000 7f39c1a94700 1 -- 172.21.15.7:0/3500990908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564410eb18d0 con 0x564411016f60 2024-06-10T01:07:56.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.666+0000 7f39c1a94700 1 --2- 172.21.15.7:0/3500990908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564411016f60 0x56441101a550 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f39b4008ec0 tx=0x7f39b40114a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ece3ae86ffd5d2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:56.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.666+0000 7f39a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:07:56.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.666+0000 7f39a4ff9700 1 -- 172.21.15.7:0/3500990908 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b4016480 con 0x564411016f60 2024-06-10T01:07:56.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.666+0000 7f39a4ff9700 1 -- 172.21.15.7:0/3500990908 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f39b4016620 con 0x564411016f60 2024-06-10T01:07:56.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.666+0000 7f39cdd7ab80 1 -- 172.21.15.7:0/3500990908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39ac00fd50 con 0x564411016f60 2024-06-10T01:07:56.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.666+0000 7f39a4ff9700 1 -- 172.21.15.7:0/3500990908 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f39b4024050 con 0x564411016f60 2024-06-10T01:07:56.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.666+0000 7f39cdd7ab80 10 client.15990 fetch_fsmap learned FSMap version 29 2024-06-10T01:07:56.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.666+0000 7f39cdd7ab80 1 -- 172.21.15.7:0/3500990908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564411028250 con 0x564411016f60 2024-06-10T01:07:56.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.666+0000 7f39a4ff9700 1 -- 172.21.15.7:0/3500990908 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39ac00fd50 con 0x564411016f60 2024-06-10T01:07:56.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.666+0000 7f39a4ff9700 1 -- 172.21.15.7:0/3500990908 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f39b4015900 con 0x564411016f60 2024-06-10T01:07:56.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.666+0000 7f39cdd7ab80 10 client.15990 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:07:56.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.666+0000 7f39cdd7ab80 10 client.15990 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:07:56.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.666+0000 7f39cdd7ab80 20 client.15990 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:56.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.666+0000 7f39cdd7ab80 1 --2- 172.21.15.7:0/3500990908 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x56441102a890 0x56441102cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.667+0000 7f39cdd7ab80 4 client.15990 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:07:56.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.667+0000 7f39cdd7ab80 1 -- 172.21.15.7:0/3500990908 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564410f6fd10 con 0x56441102a890 2024-06-10T01:07:56.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.667+0000 7f39c1293700 1 --2- 172.21.15.7:0/3500990908 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x56441102a890 0x56441102cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.667+0000 7f39c1293700 1 --2- 172.21.15.7:0/3500990908 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x56441102a890 0x56441102cca0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8c18bc32a7769958 server_cookie=ad44fbb462c0de99 in_seq=0 out_seq=0 2024-06-10T01:07:56.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.667+0000 7f39a4ff9700 10 client.15990 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:07:56.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.668+0000 7f39a4ff9700 1 -- 172.21.15.7:0/3500990908 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564410f6fd10 con 0x56441102a890 2024-06-10T01:07:56.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.668+0000 7f39a4ff9700 10 client.15990 handle_command_reply: tid=0 2024-06-10T01:07:56.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.668+0000 7f39cdd7ab80 1 client.15990 shutdown 2024-06-10T01:07:56.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.669+0000 7f39cdd7ab80 1 -- 172.21.15.7:0/3500990908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564411016f60 msgr2=0x56441101a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:56.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.669+0000 7f39cdd7ab80 1 --2- 172.21.15.7:0/3500990908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564411016f60 0x56441101a550 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f39b4008ec0 tx=0x7f39b40114a0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.669+0000 7f39cdd7ab80 1 -- 172.21.15.7:0/3500990908 shutdown_connections 2024-06-10T01:07:56.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.669+0000 7f39cdd7ab80 1 --2- 172.21.15.7:0/3500990908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564411016f60 0x56441101a550 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.669+0000 7f39cdd7ab80 1 --2- 172.21.15.7:0/3500990908 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x56441102a890 0x56441102cca0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.669+0000 7f39cdd7ab80 1 --2- 172.21.15.7:0/3500990908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564411010f20 0x56441101cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.669+0000 7f39a4ff9700 0 client.15990 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:07:56.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.669+0000 7f39cdd7ab80 1 --2- 172.21.15.7:0/3500990908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564411016640 0x56441101a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.669+0000 7f39cdd7ab80 1 -- 172.21.15.7:0/3500990908 >> 172.21.15.7:0/3500990908 conn(0x564410f3bcc0 msgr2=0x56441100f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:56.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.669+0000 7f39cdd7ab80 1 -- 172.21.15.7:0/3500990908 shutdown_connections 2024-06-10T01:07:56.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.669+0000 7f39cdd7ab80 1 -- 172.21.15.7:0/3500990908 wait complete. 2024-06-10T01:07:56.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.669+0000 7f39cdd7ab80 20 client.15990 trim_cache size 0 max 16384 2024-06-10T01:07:56.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.684+0000 7f39cdd7ab80 1 Processor -- start 2024-06-10T01:07:56.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.684+0000 7f39cdd7ab80 1 -- start start 2024-06-10T01:07:56.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.684+0000 7f39cdd7ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564411010f20 0x5644110109e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.684+0000 7f39cdd7ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564411016640 0x564411019910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.685+0000 7f39cdd7ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564411016f60 0x564411017b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.685+0000 7f39cdd7ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc1b2cb0 con 0x564411016640 2024-06-10T01:07:56.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.685+0000 7f39cdd7ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc1b0540 con 0x564411010f20 2024-06-10T01:07:56.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.685+0000 7f39cdd7ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc1ae390 con 0x564411016f60 2024-06-10T01:07:56.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.685+0000 7f39c0a92700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564411016640 0x564411019910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.685+0000 7f39c0a92700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564411016640 0x564411019910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36250/0 (socket says 172.21.15.7:36250) 2024-06-10T01:07:56.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.685+0000 7f39c0a92700 1 -- 172.21.15.7:0/3762358455 learned_addr learned my addr 172.21.15.7:0/3762358455 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:56.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.685+0000 7f39c0a92700 1 -- 172.21.15.7:0/3762358455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564411016f60 msgr2=0x564411017b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:56.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.685+0000 7f39c0a92700 1 --2- 172.21.15.7:0/3762358455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564411016f60 0x564411017b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.685+0000 7f39c1293700 1 --2- 172.21.15.7:0/3762358455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564411010f20 0x5644110109e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.685+0000 7f39c0a92700 1 -- 172.21.15.7:0/3762358455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564411010f20 msgr2=0x5644110109e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:56.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.685+0000 7f39c0a92700 1 --2- 172.21.15.7:0/3762358455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564411010f20 0x5644110109e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.685+0000 7f39c0a92700 1 -- 172.21.15.7:0/3762358455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564411018050 con 0x564411016640 2024-06-10T01:07:56.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.685+0000 7f39c1293700 1 --2- 172.21.15.7:0/3762358455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564411010f20 0x5644110109e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:07:56.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.685+0000 7f39c0a92700 1 --2- 172.21.15.7:0/3762358455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564411016640 0x564411019910 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009990 tx=0x7f39ac005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=227f31028ed918e8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:56.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.685+0000 7f39a6ffd700 1 -- 172.21.15.7:0/3762358455 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ac024cf0 con 0x564411016640 2024-06-10T01:07:56.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.685+0000 7f39a6ffd700 1 -- 172.21.15.7:0/3762358455 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39ac03d070 con 0x564411016640 2024-06-10T01:07:56.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.685+0000 7f39a6ffd700 1 -- 172.21.15.7:0/3762358455 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ac00fe50 con 0x564411016640 2024-06-10T01:07:56.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.686+0000 7f39cdd7ab80 1 -- 172.21.15.7:0/3762358455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564411016640 msgr2=0x564411019910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:56.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.686+0000 7f39cdd7ab80 1 --2- 172.21.15.7:0/3762358455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564411016640 0x564411019910 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009990 tx=0x7f39ac005bc0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.686+0000 7f39cdd7ab80 1 -- 172.21.15.7:0/3762358455 shutdown_connections 2024-06-10T01:07:56.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.686+0000 7f39cdd7ab80 1 --2- 172.21.15.7:0/3762358455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564411016f60 0x564411017b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.686+0000 7f39cdd7ab80 1 --2- 172.21.15.7:0/3762358455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564411010f20 0x5644110109e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.686+0000 7f39cdd7ab80 1 --2- 172.21.15.7:0/3762358455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564411016640 0x564411019910 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.686+0000 7f39cdd7ab80 1 -- 172.21.15.7:0/3762358455 >> 172.21.15.7:0/3762358455 conn(0x564410f3bcc0 msgr2=0x564411017400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:56.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.686+0000 7f39cdd7ab80 1 -- 172.21.15.7:0/3762358455 shutdown_connections 2024-06-10T01:07:56.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.686+0000 7f39cdd7ab80 1 -- 172.21.15.7:0/3762358455 wait complete. 2024-06-10T01:07:56.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.686+0000 7f39cdd7ab80 1 -- start start 2024-06-10T01:07:56.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.687+0000 7f39cdd7ab80 1 Processor -- start 2024-06-10T01:07:56.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.687+0000 7f39cdd7ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564411010f20 0x564411022d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.687+0000 7f39cdd7ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564411016640 0x564411023290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.687+0000 7f39cdd7ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564411016f60 0x564411031740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.687+0000 7f39cdd7ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc1aea00 con 0x564411016640 2024-06-10T01:07:56.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.687+0000 7f39cdd7ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc1afe70 con 0x564411016f60 2024-06-10T01:07:56.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.687+0000 7f39cdd7ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc1af0d0 con 0x564411010f20 2024-06-10T01:07:56.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.687+0000 7f39cdd7ab80 10 client.0 _ll_register_callbacks cb 0x564410efc920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:56.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.687+0000 7f39c0a92700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564411016640 0x564411023290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.687+0000 7f39c0a92700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564411016640 0x564411023290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36258/0 (socket says 172.21.15.7:36258) 2024-06-10T01:07:56.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.687+0000 7f39c0a92700 1 -- 172.21.15.7:0/53704081 learned_addr learned my addr 172.21.15.7:0/53704081 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:56.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.687+0000 7f39c1a94700 1 --2- 172.21.15.7:0/53704081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564411016f60 0x564411031740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.687+0000 7f39c1293700 1 --2- 172.21.15.7:0/53704081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564411010f20 0x564411022d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.687+0000 7f39c0a92700 1 -- 172.21.15.7:0/53704081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564411010f20 msgr2=0x564411022d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:56.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.687+0000 7f39c0a92700 1 --2- 172.21.15.7:0/53704081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564411010f20 0x564411022d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.687+0000 7f39c0a92700 1 -- 172.21.15.7:0/53704081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564411016f60 msgr2=0x564411031740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:56.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.687+0000 7f39c0a92700 1 --2- 172.21.15.7:0/53704081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564411016f60 0x564411031740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.687+0000 7f39c0a92700 1 -- 172.21.15.7:0/53704081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564411031d80 con 0x564411016640 2024-06-10T01:07:56.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.687+0000 7f39c1a94700 1 --2- 172.21.15.7:0/53704081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564411016f60 0x564411031740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:56.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.687+0000 7f39c1293700 1 --2- 172.21.15.7:0/53704081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564411010f20 0x564411022d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:07:56.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.688+0000 7f39c0a92700 1 --2- 172.21.15.7:0/53704081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564411016640 0x564411023290 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00f740 tx=0x7f39ac00f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52bdf15f1d6cbdee server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:56.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.688+0000 7f39a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:07:56.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.688+0000 7f39a4ff9700 1 -- 172.21.15.7:0/53704081 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ac046060 con 0x564411016640 2024-06-10T01:07:56.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.688+0000 7f39a4ff9700 1 -- 172.21.15.7:0/53704081 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f39ac0248a0 con 0x564411016640 2024-06-10T01:07:56.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.688+0000 7f39cdd7ab80 1 -- 172.21.15.7:0/53704081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39ac03d070 con 0x564411016640 2024-06-10T01:07:56.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.688+0000 7f39a4ff9700 1 -- 172.21.15.7:0/53704081 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f39ac038470 con 0x564411016640 2024-06-10T01:07:56.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.688+0000 7f39cdd7ab80 10 client.15996 fetch_fsmap learned FSMap version 29 2024-06-10T01:07:56.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.688+0000 7f39cdd7ab80 1 -- 172.21.15.7:0/53704081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564411094dd0 con 0x564411016640 2024-06-10T01:07:56.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.688+0000 7f39a4ff9700 1 -- 172.21.15.7:0/53704081 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39ac03d070 con 0x564411016640 2024-06-10T01:07:56.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.688+0000 7f39a4ff9700 1 -- 172.21.15.7:0/53704081 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f39ac02f8a0 con 0x564411016640 2024-06-10T01:07:56.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.688+0000 7f39cdd7ab80 10 client.15996 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:07:56.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.688+0000 7f39cdd7ab80 10 client.15996 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:07:56.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.688+0000 7f39cdd7ab80 20 client.15996 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:56.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.688+0000 7f39cdd7ab80 1 --2- 172.21.15.7:0/53704081 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x56441102a890 0x564411097060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.688+0000 7f39cdd7ab80 4 client.15996 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:07:56.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.688+0000 7f39cdd7ab80 1 -- 172.21.15.7:0/53704081 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564410f6a290 con 0x56441102a890 2024-06-10T01:07:56.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.689+0000 7f39c1293700 1 --2- 172.21.15.7:0/53704081 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x56441102a890 0x564411097060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.689+0000 7f39c1293700 1 --2- 172.21.15.7:0/53704081 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x56441102a890 0x564411097060 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=13e1677a971beef4 server_cookie=1c6934c98c6cbd8a in_seq=0 out_seq=0 2024-06-10T01:07:56.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.689+0000 7f39a4ff9700 10 client.15996 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:07:56.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.689+0000 7f39a4ff9700 1 -- 172.21.15.7:0/53704081 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564410f6a290 con 0x56441102a890 2024-06-10T01:07:56.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.689+0000 7f39a4ff9700 10 client.15996 handle_command_reply: tid=0 2024-06-10T01:07:56.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.689+0000 7f39cdd7ab80 1 client.15996 shutdown 2024-06-10T01:07:56.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.689+0000 7f39cdd7ab80 1 -- 172.21.15.7:0/53704081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564411016640 msgr2=0x564411023290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:56.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.689+0000 7f39cdd7ab80 1 --2- 172.21.15.7:0/53704081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564411016640 0x564411023290 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00f740 tx=0x7f39ac00f870 comp rx=0 tx=0).stop 2024-06-10T01:07:56.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.689+0000 7f39cdd7ab80 1 -- 172.21.15.7:0/53704081 shutdown_connections 2024-06-10T01:07:56.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.689+0000 7f39cdd7ab80 1 --2- 172.21.15.7:0/53704081 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x56441102a890 0x564411097060 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.690+0000 7f39cdd7ab80 1 --2- 172.21.15.7:0/53704081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564411016f60 0x564411031740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.690+0000 7f39a4ff9700 0 client.15996 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:07:56.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.690+0000 7f39cdd7ab80 1 --2- 172.21.15.7:0/53704081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564411010f20 0x564411022d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.690+0000 7f39cdd7ab80 1 --2- 172.21.15.7:0/53704081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564411016640 0x564411023290 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.690+0000 7f39cdd7ab80 1 -- 172.21.15.7:0/53704081 >> 172.21.15.7:0/53704081 conn(0x564410f3bcc0 msgr2=0x564411004e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:56.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.690+0000 7f39cdd7ab80 1 -- 172.21.15.7:0/53704081 shutdown_connections 2024-06-10T01:07:56.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.690+0000 7f39cdd7ab80 1 -- 172.21.15.7:0/53704081 wait complete. 2024-06-10T01:07:56.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.690+0000 7f39cdd7ab80 20 client.15996 trim_cache size 0 max 16384 2024-06-10T01:07:56.693 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:07:56.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.690+0000 7f39c34f7700 1 -- 172.21.15.7:0/2152214698 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f39a80a2c20 msgr2=0x7f39a80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:56.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.690+0000 7f39c34f7700 1 --2- 172.21.15.7:0/2152214698 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f39a80a2c20 0x7f39a80a50e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00b5c0 tx=0x7f39ac005960 comp rx=0 tx=0).stop 2024-06-10T01:07:56.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.690+0000 7f39c34f7700 1 -- 172.21.15.7:0/2152214698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39bc101380 msgr2=0x7f39bc103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:56.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.690+0000 7f39c34f7700 1 --2- 172.21.15.7:0/2152214698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39bc101380 0x7f39bc103060 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f39b800c970 tx=0x7f39b800cc80 comp rx=0 tx=0).stop 2024-06-10T01:07:56.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.690+0000 7f39c34f7700 1 -- 172.21.15.7:0/2152214698 shutdown_connections 2024-06-10T01:07:56.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.690+0000 7f39c34f7700 1 --2- 172.21.15.7:0/2152214698 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f39a80a2c20 0x7f39a80a50e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.690+0000 7f39c34f7700 1 --2- 172.21.15.7:0/2152214698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39bc1073f0 0x7f39bc103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.690+0000 7f39c34f7700 1 --2- 172.21.15.7:0/2152214698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39bc101380 0x7f39bc103060 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.690+0000 7f39c34f7700 1 --2- 172.21.15.7:0/2152214698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39bc101d40 0x7f39bc1035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.690+0000 7f39c34f7700 1 -- 172.21.15.7:0/2152214698 >> 172.21.15.7:0/2152214698 conn(0x7f39bc078300 msgr2=0x7f39bc100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:56.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.691+0000 7f39c34f7700 1 -- 172.21.15.7:0/2152214698 shutdown_connections 2024-06-10T01:07:56.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:56.691+0000 7f39c34f7700 1 -- 172.21.15.7:0/2152214698 wait complete. 2024-06-10T01:07:56.709 INFO:tasks.workunit.client.0.smithi007.stdout: 1 28947 4.70 MB/sec warmup 37 sec latency 27164.690 ms 2024-06-10T01:07:57.204 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:56 smithi175 ceph-mon[36636]: pgmap v490: 129 pgs: 129 active+clean; 25 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 3.2 MiB/s wr, 122 op/s 2024-06-10T01:07:57.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:56 smithi007 ceph-mon[32398]: pgmap v490: 129 pgs: 129 active+clean; 25 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 3.2 MiB/s wr, 122 op/s 2024-06-10T01:07:57.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:56 smithi078 ceph-mon[36683]: pgmap v490: 129 pgs: 129 active+clean; 25 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 3.2 MiB/s wr, 122 op/s 2024-06-10T01:07:57.709 INFO:tasks.workunit.client.0.smithi007.stdout: 1 28947 4.58 MB/sec warmup 38 sec latency 28164.795 ms 2024-06-10T01:07:58.680 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:07:58.709 INFO:tasks.workunit.client.0.smithi007.stdout: 1 28947 4.46 MB/sec warmup 39 sec latency 29164.838 ms 2024-06-10T01:07:59.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.136+0000 7f213da27700 1 -- 172.21.15.7:0/327796705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f213806aa00 msgr2=0x7f213806ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:59.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.136+0000 7f213da27700 1 --2- 172.21.15.7:0/327796705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f213806aa00 0x7f213806ae80 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f212c009990 tx=0x7f212c009ca0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.136+0000 7f213da27700 1 -- 172.21.15.7:0/327796705 shutdown_connections 2024-06-10T01:07:59.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.136+0000 7f213da27700 1 --2- 172.21.15.7:0/327796705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f213806b3c0 0x7f213810f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.136+0000 7f213da27700 1 --2- 172.21.15.7:0/327796705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21381084f0 0x7f21381088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.136+0000 7f213da27700 1 --2- 172.21.15.7:0/327796705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f213806aa00 0x7f213806ae80 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.136+0000 7f213da27700 1 -- 172.21.15.7:0/327796705 >> 172.21.15.7:0/327796705 conn(0x7f2138078300 msgr2=0x7f2138078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:59.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.136+0000 7f213da27700 1 -- 172.21.15.7:0/327796705 shutdown_connections 2024-06-10T01:07:59.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.136+0000 7f213da27700 1 -- 172.21.15.7:0/327796705 wait complete. 2024-06-10T01:07:59.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.137+0000 7f213da27700 1 Processor -- start 2024-06-10T01:07:59.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.137+0000 7f213da27700 1 -- start start 2024-06-10T01:07:59.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.138+0000 7f213da27700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f213806aa00 0x7f213819e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:59.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.138+0000 7f213da27700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f213806b3c0 0x7f213819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:59.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.138+0000 7f213da27700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21381084f0 0x7f2138198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:59.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.138+0000 7f213da27700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2138112770 con 0x7f213806b3c0 2024-06-10T01:07:59.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.138+0000 7f213da27700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f21381125f0 con 0x7f21381084f0 2024-06-10T01:07:59.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.138+0000 7f213da27700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f21381128f0 con 0x7f213806aa00 2024-06-10T01:07:59.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.138+0000 7f21367fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f213806b3c0 0x7f213819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:59.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.138+0000 7f21367fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f213806b3c0 0x7f213819edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36272/0 (socket says 172.21.15.7:36272) 2024-06-10T01:07:59.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.138+0000 7f21367fc700 1 -- 172.21.15.7:0/2829354266 learned_addr learned my addr 172.21.15.7:0/2829354266 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:59.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.138+0000 7f21377fe700 1 --2- 172.21.15.7:0/2829354266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21381084f0 0x7f2138198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:59.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.138+0000 7f2136ffd700 1 --2- 172.21.15.7:0/2829354266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f213806aa00 0x7f213819e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:59.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.138+0000 7f21367fc700 1 -- 172.21.15.7:0/2829354266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f213806aa00 msgr2=0x7f213819e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:59.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.138+0000 7f21367fc700 1 --2- 172.21.15.7:0/2829354266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f213806aa00 0x7f213819e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.138+0000 7f21367fc700 1 -- 172.21.15.7:0/2829354266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21381084f0 msgr2=0x7f2138198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:59.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.138+0000 7f21367fc700 1 --2- 172.21.15.7:0/2829354266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21381084f0 0x7f2138198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.139+0000 7f21367fc700 1 -- 172.21.15.7:0/2829354266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21381991b0 con 0x7f213806b3c0 2024-06-10T01:07:59.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.139+0000 7f21367fc700 1 --2- 172.21.15.7:0/2829354266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f213806b3c0 0x7f213819edd0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f212c000c00 tx=0x7f212c00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:59.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.139+0000 7f213ca25700 1 -- 172.21.15.7:0/2829354266 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212c027070 con 0x7f213806b3c0 2024-06-10T01:07:59.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.139+0000 7f213ca25700 1 -- 172.21.15.7:0/2829354266 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f212c003d70 con 0x7f213806b3c0 2024-06-10T01:07:59.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.139+0000 7f213da27700 1 -- 172.21.15.7:0/2829354266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2138199440 con 0x7f213806b3c0 2024-06-10T01:07:59.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.139+0000 7f213ca25700 1 -- 172.21.15.7:0/2829354266 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212c00f460 con 0x7f213806b3c0 2024-06-10T01:07:59.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.139+0000 7f213da27700 1 -- 172.21.15.7:0/2829354266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21381a5760 con 0x7f213806b3c0 2024-06-10T01:07:59.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.142+0000 7f213ca25700 1 -- 172.21.15.7:0/2829354266 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f212c02a070 con 0x7f213806b3c0 2024-06-10T01:07:59.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.143+0000 7f21482aab80 1 Processor -- start 2024-06-10T01:07:59.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.143+0000 7f21482aab80 1 -- start start 2024-06-10T01:07:59.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.143+0000 7f213ca25700 1 --2- 172.21.15.7:0/2829354266 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21200a2ca0 0x7f21200a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:59.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.143+0000 7f21482aab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574cdc3ac40 0x5574cdc3b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:59.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.143+0000 7f21482aab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574cdc3b560 0x5574cdc34e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:59.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.143+0000 7f21482aab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574cdc353d0 0x5574cdc35810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.143+0000 7f21482aab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5574cdb86910 con 0x5574cdc3ac40 2024-06-10T01:07:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.143+0000 7f21482aab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5574cdb2e820 con 0x5574cdc353d0 2024-06-10T01:07:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.143+0000 7f21482aab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5574cdb828d0 con 0x5574cdc3b560 2024-06-10T01:07:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.143+0000 7f21377fe700 1 --2- 172.21.15.7:0/2829354266 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21200a2ca0 0x7f21200a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.143+0000 7f213ca25700 1 -- 172.21.15.7:0/2829354266 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f212c0d01c0 con 0x7f213806b3c0 2024-06-10T01:07:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.144+0000 7f2136ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574cdc3ac40 0x5574cdc3b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.144+0000 7f2136ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574cdc3ac40 0x5574cdc3b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36282/0 (socket says 172.21.15.7:36282) 2024-06-10T01:07:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.144+0000 7f2136ffd700 1 -- 172.21.15.7:0/2075888603 learned_addr learned my addr 172.21.15.7:0/2075888603 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.144+0000 7f21377fe700 1 --2- 172.21.15.7:0/2075888603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574cdc353d0 0x5574cdc35810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.144+0000 7f2136ffd700 1 -- 172.21.15.7:0/2075888603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574cdc3b560 msgr2=0x5574cdc34e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.144+0000 7f21367fc700 1 --2- 172.21.15.7:0/2075888603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574cdc3b560 0x5574cdc34e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.144+0000 7f2136ffd700 1 --2- 172.21.15.7:0/2075888603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574cdc3b560 0x5574cdc34e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.144+0000 7f2136ffd700 1 -- 172.21.15.7:0/2075888603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574cdc353d0 msgr2=0x5574cdc35810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:59.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.144+0000 7f2136ffd700 1 --2- 172.21.15.7:0/2075888603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574cdc353d0 0x5574cdc35810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.144+0000 7f2136ffd700 1 -- 172.21.15.7:0/2075888603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574cdb25050 con 0x5574cdc3ac40 2024-06-10T01:07:59.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.144+0000 7f21377fe700 1 --2- 172.21.15.7:0/2829354266 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21200a2ca0 0x7f21200a5160 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2130009ec0 tx=0x7f213000fc40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:59.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.144+0000 7f21367fc700 1 --2- 172.21.15.7:0/2075888603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574cdc3b560 0x5574cdc34e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:07:59.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.144+0000 7f2136ffd700 1 --2- 172.21.15.7:0/2075888603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574cdc3ac40 0x5574cdc3b020 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2128007c10 tx=0x7f2128007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e3652a327fdfed1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:59.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.144+0000 7f211dffb700 1 -- 172.21.15.7:0/2075888603 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2128010bc0 con 0x5574cdc3ac40 2024-06-10T01:07:59.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.145+0000 7f211dffb700 1 -- 172.21.15.7:0/2075888603 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2128010d60 con 0x5574cdc3ac40 2024-06-10T01:07:59.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.145+0000 7f211dffb700 1 -- 172.21.15.7:0/2075888603 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2128019730 con 0x5574cdc3ac40 2024-06-10T01:07:59.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.146+0000 7f21482aab80 1 -- 172.21.15.7:0/2075888603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574cdc3ac40 msgr2=0x5574cdc3b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:59.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.146+0000 7f21482aab80 1 --2- 172.21.15.7:0/2075888603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574cdc3ac40 0x5574cdc3b020 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2128007c10 tx=0x7f2128007f20 comp rx=0 tx=0).stop 2024-06-10T01:07:59.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.146+0000 7f21482aab80 1 -- 172.21.15.7:0/2075888603 shutdown_connections 2024-06-10T01:07:59.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.146+0000 7f21482aab80 1 --2- 172.21.15.7:0/2075888603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574cdc353d0 0x5574cdc35810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.146+0000 7f21482aab80 1 --2- 172.21.15.7:0/2075888603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574cdc3ac40 0x5574cdc3b020 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.146+0000 7f21482aab80 1 --2- 172.21.15.7:0/2075888603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574cdc3b560 0x5574cdc34e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.146+0000 7f21482aab80 1 -- 172.21.15.7:0/2075888603 >> 172.21.15.7:0/2075888603 conn(0x5574cdb5fdc0 msgr2=0x5574cdc30120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:59.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.146+0000 7f21482aab80 1 -- 172.21.15.7:0/2075888603 shutdown_connections 2024-06-10T01:07:59.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.146+0000 7f21482aab80 1 -- 172.21.15.7:0/2075888603 wait complete. 2024-06-10T01:07:59.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.146+0000 7f21482aab80 1 -- start start 2024-06-10T01:07:59.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.146+0000 7f21482aab80 1 Processor -- start 2024-06-10T01:07:59.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.147+0000 7f21482aab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574cdc353d0 0x5574cdc41490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:59.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.147+0000 7f21482aab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574cdc3b560 0x5574cdc3e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:59.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.147+0000 7f21482aab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574cdc3ecd0 0x5574cdc3f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:59.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.147+0000 7f21482aab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5574cdc41fd0 con 0x5574cdc3b560 2024-06-10T01:07:59.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.147+0000 7f21482aab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5574cdb26cf0 con 0x5574cdc3ecd0 2024-06-10T01:07:59.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.147+0000 7f21482aab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5574cdc42150 con 0x5574cdc353d0 2024-06-10T01:07:59.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.147+0000 7f21482aab80 10 client.0 _ll_register_callbacks cb 0x5574cdb20a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:59.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.147+0000 7f21367fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574cdc3b560 0x5574cdc3e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:59.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.147+0000 7f21367fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574cdc3b560 0x5574cdc3e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36296/0 (socket says 172.21.15.7:36296) 2024-06-10T01:07:59.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.147+0000 7f21367fc700 1 -- 172.21.15.7:0/2479826504 learned_addr learned my addr 172.21.15.7:0/2479826504 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:59.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.147+0000 7f2136ffd700 1 --2- 172.21.15.7:0/2479826504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574cdc353d0 0x5574cdc41490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:59.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.147+0000 7f21367fc700 1 -- 172.21.15.7:0/2479826504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574cdc353d0 msgr2=0x5574cdc41490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:59.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.147+0000 7f21367fc700 1 --2- 172.21.15.7:0/2479826504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574cdc353d0 0x5574cdc41490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.147+0000 7f21367fc700 1 -- 172.21.15.7:0/2479826504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574cdc3ecd0 msgr2=0x5574cdc3f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:59.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.147+0000 7f21377fe700 1 --2- 172.21.15.7:0/2479826504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574cdc3ecd0 0x5574cdc3f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:59.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.147+0000 7f21367fc700 1 --2- 172.21.15.7:0/2479826504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574cdc3ecd0 0x5574cdc3f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.147+0000 7f21367fc700 1 -- 172.21.15.7:0/2479826504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574cdad59d0 con 0x5574cdc3b560 2024-06-10T01:07:59.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.147+0000 7f2136ffd700 1 --2- 172.21.15.7:0/2479826504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574cdc353d0 0x5574cdc41490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:59.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.147+0000 7f21377fe700 1 --2- 172.21.15.7:0/2479826504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574cdc3ecd0 0x5574cdc3f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:07:59.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.148+0000 7f21367fc700 1 --2- 172.21.15.7:0/2479826504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574cdc3b560 0x5574cdc3e790 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f212c0a4d80 tx=0x7f212c0229e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0cdb178bd49af0f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:59.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.148+0000 7f2116ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:07:59.150 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:07:58 smithi007 ceph-mon[32398]: pgmap v491: 129 pgs: 129 active+clean; 25 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 3.2 MiB/s wr, 115 op/s 2024-06-10T01:07:59.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.148+0000 7f2116ffd700 1 -- 172.21.15.7:0/2479826504 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212c087110 con 0x5574cdc3b560 2024-06-10T01:07:59.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.148+0000 7f21482aab80 1 -- 172.21.15.7:0/2479826504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2128010d60 con 0x5574cdc3b560 2024-06-10T01:07:59.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.148+0000 7f2116ffd700 1 -- 172.21.15.7:0/2479826504 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f212c0094b0 con 0x5574cdc3b560 2024-06-10T01:07:59.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.149+0000 7f2116ffd700 1 -- 172.21.15.7:0/2479826504 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f212c0a0a80 con 0x5574cdc3b560 2024-06-10T01:07:59.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.149+0000 7f21482aab80 10 client.16008 fetch_fsmap learned FSMap version 29 2024-06-10T01:07:59.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.149+0000 7f21482aab80 1 -- 172.21.15.7:0/2479826504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5574cdc4cde0 con 0x5574cdc3b560 2024-06-10T01:07:59.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.149+0000 7f2116ffd700 1 -- 172.21.15.7:0/2479826504 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2128010d60 con 0x5574cdc3b560 2024-06-10T01:07:59.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.149+0000 7f2116ffd700 1 -- 172.21.15.7:0/2479826504 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f212c09bbb0 con 0x5574cdc3b560 2024-06-10T01:07:59.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.149+0000 7f21482aab80 10 client.16008 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:07:59.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.149+0000 7f21482aab80 10 client.16008 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:07:59.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.149+0000 7f21482aab80 20 client.16008 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:59.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.149+0000 7f21482aab80 1 --2- 172.21.15.7:0/2479826504 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5574cdc4f2b0 0x5574cdc516a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:59.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.149+0000 7f21482aab80 4 client.16008 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:07:59.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.149+0000 7f21482aab80 1 -- 172.21.15.7:0/2479826504 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574cdb93e10 con 0x5574cdc4f2b0 2024-06-10T01:07:59.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.150+0000 7f2136ffd700 1 --2- 172.21.15.7:0/2479826504 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5574cdc4f2b0 0x5574cdc516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:59.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.150+0000 7f2136ffd700 1 --2- 172.21.15.7:0/2479826504 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5574cdc4f2b0 0x5574cdc516a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1b94b0990f946eb server_cookie=51b9a1259d201fc4 in_seq=0 out_seq=0 2024-06-10T01:07:59.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.150+0000 7f2116ffd700 10 client.16008 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:59.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.151+0000 7f2116ffd700 1 -- 172.21.15.7:0/2479826504 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5574cdb93e10 con 0x5574cdc4f2b0 2024-06-10T01:07:59.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.151+0000 7f2116ffd700 10 client.16008 handle_command_reply: tid=0 2024-06-10T01:07:59.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.151+0000 7f21482aab80 1 client.16008 shutdown 2024-06-10T01:07:59.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.151+0000 7f21482aab80 1 -- 172.21.15.7:0/2479826504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574cdc3b560 msgr2=0x5574cdc3e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:59.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.151+0000 7f21482aab80 1 --2- 172.21.15.7:0/2479826504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574cdc3b560 0x5574cdc3e790 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f212c0a4d80 tx=0x7f212c0229e0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.152+0000 7f21482aab80 1 -- 172.21.15.7:0/2479826504 shutdown_connections 2024-06-10T01:07:59.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.152+0000 7f21482aab80 1 --2- 172.21.15.7:0/2479826504 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5574cdc4f2b0 0x5574cdc516a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.152+0000 7f21482aab80 1 --2- 172.21.15.7:0/2479826504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574cdc3ecd0 0x5574cdc3f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.152+0000 7f2116ffd700 0 client.16008 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:59.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.152+0000 7f21482aab80 1 --2- 172.21.15.7:0/2479826504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574cdc353d0 0x5574cdc41490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.152+0000 7f21482aab80 1 --2- 172.21.15.7:0/2479826504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574cdc3b560 0x5574cdc3e790 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.152+0000 7f21482aab80 1 -- 172.21.15.7:0/2479826504 >> 172.21.15.7:0/2479826504 conn(0x5574cdb5fdc0 msgr2=0x5574cdc33e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:59.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.152+0000 7f21482aab80 1 -- 172.21.15.7:0/2479826504 shutdown_connections 2024-06-10T01:07:59.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.152+0000 7f21482aab80 1 -- 172.21.15.7:0/2479826504 wait complete. 2024-06-10T01:07:59.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.152+0000 7f21482aab80 20 client.16008 trim_cache size 0 max 16384 2024-06-10T01:07:59.155 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.173+0000 7f21482aab80 1 Processor -- start 2024-06-10T01:07:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.173+0000 7f21482aab80 1 -- start start 2024-06-10T01:07:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.173+0000 7f21482aab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574cdc353d0 0x5574cdc3d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.173+0000 7f21482aab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574cdc3b560 0x5574cdc3e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.173+0000 7f21482aab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574cdc3ecd0 0x5574cdc56f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.173+0000 7f21482aab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f21381b0aa0 con 0x5574cdc3b560 2024-06-10T01:07:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.173+0000 7f21482aab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f21381ae330 con 0x5574cdc3ecd0 2024-06-10T01:07:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.173+0000 7f21482aab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f21381ac180 con 0x5574cdc353d0 2024-06-10T01:07:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.174+0000 7f21367fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574cdc3b560 0x5574cdc3e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.174+0000 7f21367fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574cdc3b560 0x5574cdc3e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36300/0 (socket says 172.21.15.7:36300) 2024-06-10T01:07:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.174+0000 7f21367fc700 1 -- 172.21.15.7:0/4159929222 learned_addr learned my addr 172.21.15.7:0/4159929222 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.174+0000 7f2136ffd700 1 --2- 172.21.15.7:0/4159929222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574cdc353d0 0x5574cdc3d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.174+0000 7f21367fc700 1 -- 172.21.15.7:0/4159929222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574cdc353d0 msgr2=0x5574cdc3d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.174+0000 7f21367fc700 1 --2- 172.21.15.7:0/4159929222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574cdc353d0 0x5574cdc3d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.174+0000 7f21367fc700 1 -- 172.21.15.7:0/4159929222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574cdc3ecd0 msgr2=0x5574cdc56f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.174+0000 7f21367fc700 1 --2- 172.21.15.7:0/4159929222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574cdc3ecd0 0x5574cdc56f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.174+0000 7f21367fc700 1 -- 172.21.15.7:0/4159929222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574cdc3cd00 con 0x5574cdc3b560 2024-06-10T01:07:59.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.174+0000 7f21367fc700 1 --2- 172.21.15.7:0/4159929222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574cdc3b560 0x5574cdc3e750 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f212c09c660 tx=0x7f212c022aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac04ff20c8021e88 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:59.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.175+0000 7f211d7fa700 1 -- 172.21.15.7:0/4159929222 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212c0b9030 con 0x5574cdc3b560 2024-06-10T01:07:59.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.175+0000 7f211d7fa700 1 -- 172.21.15.7:0/4159929222 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f212c087110 con 0x5574cdc3b560 2024-06-10T01:07:59.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.175+0000 7f211d7fa700 1 -- 172.21.15.7:0/4159929222 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212c0a5080 con 0x5574cdc3b560 2024-06-10T01:07:59.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.175+0000 7f21482aab80 1 -- 172.21.15.7:0/4159929222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574cdc3b560 msgr2=0x5574cdc3e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:59.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.175+0000 7f21482aab80 1 --2- 172.21.15.7:0/4159929222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574cdc3b560 0x5574cdc3e750 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f212c09c660 tx=0x7f212c022aa0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.176+0000 7f21482aab80 1 -- 172.21.15.7:0/4159929222 shutdown_connections 2024-06-10T01:07:59.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.176+0000 7f21482aab80 1 --2- 172.21.15.7:0/4159929222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574cdc3ecd0 0x5574cdc56f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.176+0000 7f21482aab80 1 --2- 172.21.15.7:0/4159929222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574cdc353d0 0x5574cdc3d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.176+0000 7f21482aab80 1 --2- 172.21.15.7:0/4159929222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574cdc3b560 0x5574cdc3e750 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.176+0000 7f21482aab80 1 -- 172.21.15.7:0/4159929222 >> 172.21.15.7:0/4159929222 conn(0x5574cdb5fdc0 msgr2=0x5574cdc3e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:59.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.176+0000 7f21482aab80 1 -- 172.21.15.7:0/4159929222 shutdown_connections 2024-06-10T01:07:59.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.176+0000 7f21482aab80 1 -- 172.21.15.7:0/4159929222 wait complete. 2024-06-10T01:07:59.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.176+0000 7f21482aab80 1 -- start start 2024-06-10T01:07:59.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.176+0000 7f21482aab80 1 Processor -- start 2024-06-10T01:07:59.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.176+0000 7f21482aab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574cdc353d0 0x5574cdc475f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:59.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.176+0000 7f21482aab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574cdc3b560 0x5574cdc47b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:59.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.177+0000 7f21482aab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574cdc3ecd0 0x5574cdc443e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:59.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.177+0000 7f21482aab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f21381ac7f0 con 0x5574cdc353d0 2024-06-10T01:07:59.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.177+0000 7f21482aab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f21381adc60 con 0x5574cdc3b560 2024-06-10T01:07:59.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.177+0000 7f21482aab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f21381acec0 con 0x5574cdc3ecd0 2024-06-10T01:07:59.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.177+0000 7f2136ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574cdc353d0 0x5574cdc475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:59.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.177+0000 7f2136ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574cdc353d0 0x5574cdc475f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36314/0 (socket says 172.21.15.7:36314) 2024-06-10T01:07:59.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.177+0000 7f2136ffd700 1 -- 172.21.15.7:0/113819498 learned_addr learned my addr 172.21.15.7:0/113819498 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:07:59.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.177+0000 7f21482aab80 10 client.0 _ll_register_callbacks cb 0x5574cdb20a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:59.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.177+0000 7f21367fc700 1 --2- 172.21.15.7:0/113819498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574cdc3b560 0x5574cdc47b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:59.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.177+0000 7f2136ffd700 1 -- 172.21.15.7:0/113819498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574cdc3ecd0 msgr2=0x5574cdc443e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:59.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.177+0000 7f2136ffd700 1 --2- 172.21.15.7:0/113819498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574cdc3ecd0 0x5574cdc443e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.177+0000 7f2136ffd700 1 -- 172.21.15.7:0/113819498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574cdc3b560 msgr2=0x5574cdc47b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:59.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.177+0000 7f2136ffd700 1 --2- 172.21.15.7:0/113819498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574cdc3b560 0x5574cdc47b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.177+0000 7f2136ffd700 1 -- 172.21.15.7:0/113819498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574cdc449f0 con 0x5574cdc353d0 2024-06-10T01:07:59.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.177+0000 7f2136ffd700 1 --2- 172.21.15.7:0/113819498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574cdc353d0 0x5574cdc475f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2128000c00 tx=0x7f2128010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa74089bc5f62422 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:59.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.177+0000 7f2116ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:07:59.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.178+0000 7f2116ffd700 1 -- 172.21.15.7:0/113819498 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21280050c0 con 0x5574cdc353d0 2024-06-10T01:07:59.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.178+0000 7f2116ffd700 1 -- 172.21.15.7:0/113819498 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f2128005260 con 0x5574cdc353d0 2024-06-10T01:07:59.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.178+0000 7f21482aab80 1 -- 172.21.15.7:0/113819498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f212c087110 con 0x5574cdc353d0 2024-06-10T01:07:59.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.178+0000 7f2116ffd700 1 -- 172.21.15.7:0/113819498 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2128036050 con 0x5574cdc353d0 2024-06-10T01:07:59.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.178+0000 7f21482aab80 10 client.16014 fetch_fsmap learned FSMap version 29 2024-06-10T01:07:59.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.178+0000 7f21482aab80 1 -- 172.21.15.7:0/113819498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5574cdcba880 con 0x5574cdc353d0 2024-06-10T01:07:59.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.178+0000 7f2116ffd700 1 -- 172.21.15.7:0/113819498 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f212c087110 con 0x5574cdc353d0 2024-06-10T01:07:59.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.178+0000 7f2116ffd700 1 -- 172.21.15.7:0/113819498 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f2128003680 con 0x5574cdc353d0 2024-06-10T01:07:59.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.178+0000 7f21482aab80 10 client.16014 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:07:59.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.178+0000 7f21482aab80 10 client.16014 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:07:59.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.178+0000 7f21482aab80 20 client.16014 populate_metadata read hostname 'smithi007' 2024-06-10T01:07:59.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.178+0000 7f21482aab80 1 --2- 172.21.15.7:0/113819498 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5574cdc4f2b0 0x5574cdcbcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:59.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.178+0000 7f21482aab80 4 client.16014 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "19da4be8", "release": true, "await": true}] 2024-06-10T01:07:59.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.178+0000 7f21482aab80 1 -- 172.21.15.7:0/113819498 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "set_id": "19da4be8", "release": true, "await": true}) -- 0x5574cdb8e390 con 0x5574cdc4f2b0 2024-06-10T01:07:59.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.179+0000 7f21367fc700 1 --2- 172.21.15.7:0/113819498 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5574cdc4f2b0 0x5574cdcbcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:59.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.179+0000 7f21367fc700 1 --2- 172.21.15.7:0/113819498 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5574cdc4f2b0 0x5574cdcbcc20 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d08ea015c528eec7 server_cookie=b490ee60dbe35172 in_seq=0 out_seq=0 2024-06-10T01:07:59.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.179+0000 7f2116ffd700 10 client.16014 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:59.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.194+0000 7f2116ffd700 1 -- 172.21.15.7:0/113819498 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5574cdb8e390 con 0x5574cdc4f2b0 2024-06-10T01:07:59.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.194+0000 7f2116ffd700 10 client.16014 handle_command_reply: tid=0 2024-06-10T01:07:59.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.194+0000 7f21482aab80 1 client.16014 shutdown 2024-06-10T01:07:59.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.194+0000 7f21482aab80 1 -- 172.21.15.7:0/113819498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574cdc353d0 msgr2=0x5574cdc475f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:59.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.194+0000 7f21482aab80 1 --2- 172.21.15.7:0/113819498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574cdc353d0 0x5574cdc475f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2128000c00 tx=0x7f2128010cb0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.194+0000 7f21482aab80 1 -- 172.21.15.7:0/113819498 shutdown_connections 2024-06-10T01:07:59.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.194+0000 7f21482aab80 1 --2- 172.21.15.7:0/113819498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574cdc3ecd0 0x5574cdc443e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.194+0000 7f21482aab80 1 --2- 172.21.15.7:0/113819498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574cdc353d0 0x5574cdc475f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.194+0000 7f21482aab80 1 --2- 172.21.15.7:0/113819498 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5574cdc4f2b0 0x5574cdcbcc20 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.194+0000 7f21482aab80 1 --2- 172.21.15.7:0/113819498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574cdc3b560 0x5574cdc47b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.194+0000 7f2116ffd700 0 client.16014 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:07:59.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.194+0000 7f21482aab80 1 -- 172.21.15.7:0/113819498 >> 172.21.15.7:0/113819498 conn(0x5574cdb5fdc0 msgr2=0x5574cdc384b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:59.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.194+0000 7f21482aab80 1 -- 172.21.15.7:0/113819498 shutdown_connections 2024-06-10T01:07:59.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.194+0000 7f21482aab80 1 -- 172.21.15.7:0/113819498 wait complete. 2024-06-10T01:07:59.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.194+0000 7f21482aab80 20 client.16014 trim_cache size 0 max 16384 2024-06-10T01:07:59.191 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:07:59.192 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 29, 2024-06-10T01:07:59.192 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:07:59.192 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 8, 2024-06-10T01:07:59.192 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:07:59.192 INFO:teuthology.orchestra.run.smithi007.stdout: "19da4be8": { 2024-06-10T01:07:59.192 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 8, 2024-06-10T01:07:59.192 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:07:59.192 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:07:59.192 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:07:59.192 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:07:59.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:07:59.192 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:07:59.193 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 150.0, 2024-06-10T01:07:59.193 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:07:59.193 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:07:59.193 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:07:59.193 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:07:59.193 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:07:59.193 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:07:59.193 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:07:59.193 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:07:59.193 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:07:59.193 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:07:59.193 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:07:59.193 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:07:59.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.195+0000 7f213da27700 1 -- 172.21.15.7:0/2829354266 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21200a2ca0 msgr2=0x7f21200a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:59.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.195+0000 7f213da27700 1 --2- 172.21.15.7:0/2829354266 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21200a2ca0 0x7f21200a5160 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2130009ec0 tx=0x7f213000fc40 comp rx=0 tx=0).stop 2024-06-10T01:07:59.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.195+0000 7f213da27700 1 -- 172.21.15.7:0/2829354266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f213806b3c0 msgr2=0x7f213819edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:59.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.195+0000 7f213da27700 1 --2- 172.21.15.7:0/2829354266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f213806b3c0 0x7f213819edd0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f212c000c00 tx=0x7f212c00bab0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.195+0000 7f213da27700 1 -- 172.21.15.7:0/2829354266 shutdown_connections 2024-06-10T01:07:59.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.195+0000 7f213da27700 1 --2- 172.21.15.7:0/2829354266 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21200a2ca0 0x7f21200a5160 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.195+0000 7f213da27700 1 --2- 172.21.15.7:0/2829354266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21381084f0 0x7f2138198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.195+0000 7f213da27700 1 --2- 172.21.15.7:0/2829354266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f213806aa00 0x7f213819e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.195+0000 7f213da27700 1 --2- 172.21.15.7:0/2829354266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f213806b3c0 0x7f213819edd0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:59.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.195+0000 7f213da27700 1 -- 172.21.15.7:0/2829354266 >> 172.21.15.7:0/2829354266 conn(0x7f2138078300 msgr2=0x7f2138100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:59.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.195+0000 7f213da27700 1 -- 172.21.15.7:0/2829354266 shutdown_connections 2024-06-10T01:07:59.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:07:59.195+0000 7f213da27700 1 -- 172.21.15.7:0/2829354266 wait complete. 2024-06-10T01:07:59.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:07:58 smithi175 ceph-mon[36636]: pgmap v491: 129 pgs: 129 active+clean; 25 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 3.2 MiB/s wr, 115 op/s 2024-06-10T01:07:59.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:07:58 smithi078 ceph-mon[36683]: pgmap v491: 129 pgs: 129 active+clean; 25 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 3.2 MiB/s wr, 115 op/s 2024-06-10T01:07:59.709 INFO:tasks.workunit.client.0.smithi007.stdout: 1 30278 4.48 MB/sec warmup 40 sec latency 29641.853 ms 2024-06-10T01:08:00.544 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '19da4be8', total seconds elapsed: 39.6 2024-06-10T01:08:00.709 INFO:tasks.workunit.client.0.smithi007.stdout: 1 32839 4.72 MB/sec warmup 41 sec latency 15.650 ms 2024-06-10T01:08:01.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:00 smithi175 ceph-mon[36636]: pgmap v492: 129 pgs: 129 active+clean; 25 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 2.5 MiB/s wr, 102 op/s 2024-06-10T01:08:01.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:00 smithi007 ceph-mon[32398]: pgmap v492: 129 pgs: 129 active+clean; 25 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 2.5 MiB/s wr, 102 op/s 2024-06-10T01:08:01.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:00 smithi078 ceph-mon[36683]: pgmap v492: 129 pgs: 129 active+clean; 25 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 2.5 MiB/s wr, 102 op/s 2024-06-10T01:08:01.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.591+0000 7f7167f2d700 1 -- 172.21.15.7:0/4077766239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f716006a170 msgr2=0x7f716006a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:01.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.591+0000 7f7167f2d700 1 --2- 172.21.15.7:0/4077766239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f716006a170 0x7f716006a5f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7154009990 tx=0x7f7154009ca0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.591+0000 7f7167f2d700 1 -- 172.21.15.7:0/4077766239 shutdown_connections 2024-06-10T01:08:01.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.591+0000 7f7167f2d700 1 --2- 172.21.15.7:0/4077766239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716006ab30 0x7f716010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.591+0000 7f7167f2d700 1 --2- 172.21.15.7:0/4077766239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7160108500 0x7f71601088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.591+0000 7f7167f2d700 1 --2- 172.21.15.7:0/4077766239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f716006a170 0x7f716006a5f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.591+0000 7f7167f2d700 1 -- 172.21.15.7:0/4077766239 >> 172.21.15.7:0/4077766239 conn(0x7f7160078340 msgr2=0x7f7160078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:01.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.591+0000 7f7167f2d700 1 -- 172.21.15.7:0/4077766239 shutdown_connections 2024-06-10T01:08:01.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.591+0000 7f7167f2d700 1 -- 172.21.15.7:0/4077766239 wait complete. 2024-06-10T01:08:01.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.592+0000 7f7167f2d700 1 Processor -- start 2024-06-10T01:08:01.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.592+0000 7f7167f2d700 1 -- start start 2024-06-10T01:08:01.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.592+0000 7f7167f2d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716006a170 0x7f716019e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:01.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.592+0000 7f7167f2d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f716006ab30 0x7f716019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:01.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.592+0000 7f7167f2d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7160108500 0x7f7160198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:01.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.592+0000 7f7167f2d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7160112780 con 0x7f716006ab30 2024-06-10T01:08:01.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.592+0000 7f7167f2d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7160112600 con 0x7f7160108500 2024-06-10T01:08:01.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.592+0000 7f7167f2d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7160112900 con 0x7f716006a170 2024-06-10T01:08:01.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.593+0000 7f7165cc9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716006a170 0x7f716019e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:01.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.593+0000 7f7165cc9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716006a170 0x7f716019e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:58582/0 (socket says 172.21.15.7:58582) 2024-06-10T01:08:01.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.593+0000 7f71654c8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f716006ab30 0x7f716019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:01.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.593+0000 7f71664ca700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7160108500 0x7f7160198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:01.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.593+0000 7f7165cc9700 1 -- 172.21.15.7:0/4237270456 learned_addr learned my addr 172.21.15.7:0/4237270456 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:01.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.593+0000 7f71664ca700 1 -- 172.21.15.7:0/4237270456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716006a170 msgr2=0x7f716019e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:01.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.593+0000 7f71664ca700 1 --2- 172.21.15.7:0/4237270456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716006a170 0x7f716019e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.593+0000 7f71664ca700 1 -- 172.21.15.7:0/4237270456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f716006ab30 msgr2=0x7f716019ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:01.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.593+0000 7f71664ca700 1 --2- 172.21.15.7:0/4237270456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f716006ab30 0x7f716019ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.593+0000 7f71664ca700 1 -- 172.21.15.7:0/4237270456 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7160199200 con 0x7f7160108500 2024-06-10T01:08:01.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.593+0000 7f71654c8700 1 --2- 172.21.15.7:0/4237270456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f716006ab30 0x7f716019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:01.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.593+0000 7f7165cc9700 1 --2- 172.21.15.7:0/4237270456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716006a170 0x7f716019e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:01.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.593+0000 7f71664ca700 1 --2- 172.21.15.7:0/4237270456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7160108500 0x7f7160198aa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f715800efc0 tx=0x7f715800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:01.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.593+0000 7f7152ffd700 1 -- 172.21.15.7:0/4237270456 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71580092e0 con 0x7f7160108500 2024-06-10T01:08:01.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.593+0000 7f7152ffd700 1 -- 172.21.15.7:0/4237270456 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f715800ce20 con 0x7f7160108500 2024-06-10T01:08:01.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.594+0000 7f7167f2d700 1 -- 172.21.15.7:0/4237270456 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7160199490 con 0x7f7160108500 2024-06-10T01:08:01.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.594+0000 7f7152ffd700 1 -- 172.21.15.7:0/4237270456 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7158007c00 con 0x7f7160108500 2024-06-10T01:08:01.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.594+0000 7f7167f2d700 1 -- 172.21.15.7:0/4237270456 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71601a57b0 con 0x7f7160108500 2024-06-10T01:08:01.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.596+0000 7f7152ffd700 1 -- 172.21.15.7:0/4237270456 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f715801b020 con 0x7f7160108500 2024-06-10T01:08:01.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.597+0000 7f7152ffd700 1 --2- 172.21.15.7:0/4237270456 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f714c0a2ca0 0x7f714c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:01.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.597+0000 7f7165cc9700 1 --2- 172.21.15.7:0/4237270456 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f714c0a2ca0 0x7f714c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:01.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.597+0000 7f7152ffd700 1 -- 172.21.15.7:0/4237270456 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f71580cef30 con 0x7f7160108500 2024-06-10T01:08:01.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.598+0000 7f7165cc9700 1 --2- 172.21.15.7:0/4237270456 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f714c0a2ca0 0x7f714c0a5160 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7160199be0 tx=0x7f715c00b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:01.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.598+0000 7f71727b0b80 1 Processor -- start 2024-06-10T01:08:01.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.598+0000 7f71727b0b80 1 -- start start 2024-06-10T01:08:01.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.598+0000 7f71727b0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea6c2640 0x5629ea6c2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:01.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.598+0000 7f71727b0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea6c2f60 0x5629ea6bc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:01.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.598+0000 7f71727b0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea6bcf20 0x5629ea6bd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:01.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.598+0000 7f71727b0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5629ea60e810 con 0x5629ea6c2f60 2024-06-10T01:08:01.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.598+0000 7f71727b0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5629ea5b6720 con 0x5629ea6c2640 2024-06-10T01:08:01.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.598+0000 7f71727b0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5629ea60a7d0 con 0x5629ea6bcf20 2024-06-10T01:08:01.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.599+0000 7f7165cc9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea6c2640 0x5629ea6c2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:01.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.599+0000 7f71654c8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea6c2f60 0x5629ea6bc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:01.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.599+0000 7f7165cc9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea6c2640 0x5629ea6c2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:45314/0 (socket says 172.21.15.7:45314) 2024-06-10T01:08:01.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.599+0000 7f7165cc9700 1 -- 172.21.15.7:0/488984902 learned_addr learned my addr 172.21.15.7:0/488984902 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:01.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.599+0000 7f71664ca700 1 --2- 172.21.15.7:0/488984902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea6bcf20 0x5629ea6bd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:01.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.599+0000 7f71654c8700 1 -- 172.21.15.7:0/488984902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea6bcf20 msgr2=0x5629ea6bd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:01.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.599+0000 7f71654c8700 1 --2- 172.21.15.7:0/488984902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea6bcf20 0x5629ea6bd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.599+0000 7f71654c8700 1 -- 172.21.15.7:0/488984902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea6c2640 msgr2=0x5629ea6c2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:01.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.599+0000 7f71654c8700 1 --2- 172.21.15.7:0/488984902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea6c2640 0x5629ea6c2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.599+0000 7f71654c8700 1 -- 172.21.15.7:0/488984902 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629ea5acf50 con 0x5629ea6c2f60 2024-06-10T01:08:01.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.599+0000 7f71664ca700 1 --2- 172.21.15.7:0/488984902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea6bcf20 0x5629ea6bd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:01.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.599+0000 7f71654c8700 1 --2- 172.21.15.7:0/488984902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea6c2f60 0x5629ea6bc9e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f71540059d0 tx=0x7f7154005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf45916d43bf6bb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:01.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.599+0000 7f714bfff700 1 -- 172.21.15.7:0/488984902 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7154028070 con 0x5629ea6c2f60 2024-06-10T01:08:01.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.599+0000 7f714bfff700 1 -- 172.21.15.7:0/488984902 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f715400bd60 con 0x5629ea6c2f60 2024-06-10T01:08:01.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.599+0000 7f714bfff700 1 -- 172.21.15.7:0/488984902 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71540049f0 con 0x5629ea6c2f60 2024-06-10T01:08:01.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.600+0000 7f71727b0b80 1 -- 172.21.15.7:0/488984902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea6c2f60 msgr2=0x5629ea6bc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:01.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.600+0000 7f71727b0b80 1 --2- 172.21.15.7:0/488984902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea6c2f60 0x5629ea6bc9e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f71540059d0 tx=0x7f7154005c90 comp rx=0 tx=0).stop 2024-06-10T01:08:01.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.600+0000 7f71727b0b80 1 -- 172.21.15.7:0/488984902 shutdown_connections 2024-06-10T01:08:01.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.600+0000 7f71727b0b80 1 --2- 172.21.15.7:0/488984902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea6bcf20 0x5629ea6bd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.600+0000 7f71727b0b80 1 --2- 172.21.15.7:0/488984902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea6c2640 0x5629ea6c2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.600+0000 7f71727b0b80 1 --2- 172.21.15.7:0/488984902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea6c2f60 0x5629ea6bc9e0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.600+0000 7f71727b0b80 1 -- 172.21.15.7:0/488984902 >> 172.21.15.7:0/488984902 conn(0x5629ea5e7cc0 msgr2=0x5629ea6b7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:01.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.600+0000 7f71727b0b80 1 -- 172.21.15.7:0/488984902 shutdown_connections 2024-06-10T01:08:01.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.601+0000 7f71727b0b80 1 -- 172.21.15.7:0/488984902 wait complete. 2024-06-10T01:08:01.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.601+0000 7f71727b0b80 1 -- start start 2024-06-10T01:08:01.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.601+0000 7f71727b0b80 1 Processor -- start 2024-06-10T01:08:01.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.601+0000 7f71727b0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea6bcf20 0x5629ea6c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:01.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.601+0000 7f71727b0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea6c2640 0x5629ea6c6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:01.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.601+0000 7f71727b0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea6c2f60 0x5629ea6c65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:01.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.601+0000 7f71727b0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5629ea6c9970 con 0x5629ea6c2f60 2024-06-10T01:08:01.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.601+0000 7f71727b0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5629ea5aebf0 con 0x5629ea6bcf20 2024-06-10T01:08:01.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.601+0000 7f71727b0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5629ea6c9af0 con 0x5629ea6c2640 2024-06-10T01:08:01.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.601+0000 7f71727b0b80 10 client.0 _ll_register_callbacks cb 0x5629ea5a8920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:01.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.602+0000 7f7165cc9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea6bcf20 0x5629ea6c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:01.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.602+0000 7f71664ca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea6c2f60 0x5629ea6c65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:01.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.602+0000 7f71664ca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea6c2f60 0x5629ea6c65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36358/0 (socket says 172.21.15.7:36358) 2024-06-10T01:08:01.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.602+0000 7f71664ca700 1 -- 172.21.15.7:0/3794604499 learned_addr learned my addr 172.21.15.7:0/3794604499 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:01.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.602+0000 7f7165cc9700 1 -- 172.21.15.7:0/3794604499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea6c2640 msgr2=0x5629ea6c6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:01.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.602+0000 7f7165cc9700 1 --2- 172.21.15.7:0/3794604499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea6c2640 0x5629ea6c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.602+0000 7f7165cc9700 1 -- 172.21.15.7:0/3794604499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea6c2f60 msgr2=0x5629ea6c65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:01.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.602+0000 7f7165cc9700 1 --2- 172.21.15.7:0/3794604499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea6c2f60 0x5629ea6c65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.602+0000 7f7165cc9700 1 -- 172.21.15.7:0/3794604499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629ea55d8d0 con 0x5629ea6bcf20 2024-06-10T01:08:01.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.602+0000 7f71664ca700 1 --2- 172.21.15.7:0/3794604499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea6c2f60 0x5629ea6c65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:01.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.602+0000 7f7165cc9700 1 --2- 172.21.15.7:0/3794604499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea6bcf20 0x5629ea6c8d80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f715c005c20 tx=0x7f715c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0beef884bc6d972 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:01.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.602+0000 7f71497fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:08:01.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.602+0000 7f71497fa700 1 -- 172.21.15.7:0/3794604499 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c01f840 con 0x5629ea6bcf20 2024-06-10T01:08:01.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.602+0000 7f71497fa700 1 -- 172.21.15.7:0/3794604499 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f715c01f9e0 con 0x5629ea6bcf20 2024-06-10T01:08:01.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.602+0000 7f71727b0b80 1 -- 172.21.15.7:0/3794604499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f715400bd60 con 0x5629ea6bcf20 2024-06-10T01:08:01.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.603+0000 7f71497fa700 1 -- 172.21.15.7:0/3794604499 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f715c031050 con 0x5629ea6bcf20 2024-06-10T01:08:01.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.603+0000 7f71727b0b80 10 client.25565 fetch_fsmap learned FSMap version 29 2024-06-10T01:08:01.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.603+0000 7f71727b0b80 1 -- 172.21.15.7:0/3794604499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5629ea6d43b0 con 0x5629ea6bcf20 2024-06-10T01:08:01.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.603+0000 7f71497fa700 1 -- 172.21.15.7:0/3794604499 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f715400bd60 con 0x5629ea6bcf20 2024-06-10T01:08:01.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.603+0000 7f71497fa700 1 -- 172.21.15.7:0/3794604499 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f715c01e430 con 0x5629ea6bcf20 2024-06-10T01:08:01.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.603+0000 7f71727b0b80 10 client.25565 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:08:01.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.603+0000 7f71727b0b80 10 client.25565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:01.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.603+0000 7f71727b0b80 20 client.25565 populate_metadata read hostname 'smithi007' 2024-06-10T01:08:01.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.603+0000 7f71727b0b80 1 --2- 172.21.15.7:0/3794604499 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5629ea6d69f0 0x5629ea6d8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:01.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.603+0000 7f71727b0b80 4 client.25565 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:08:01.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.603+0000 7f71727b0b80 1 -- 172.21.15.7:0/3794604499 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629ea61bd10 con 0x5629ea6d69f0 2024-06-10T01:08:01.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.603+0000 7f71654c8700 1 --2- 172.21.15.7:0/3794604499 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5629ea6d69f0 0x5629ea6d8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:01.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.604+0000 7f71497fa700 10 client.25565 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:01.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.604+0000 7f71654c8700 1 --2- 172.21.15.7:0/3794604499 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5629ea6d69f0 0x5629ea6d8e60 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d73cc924529804e server_cookie=1a8bd7353b8df98 in_seq=0 out_seq=0 2024-06-10T01:08:01.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.605+0000 7f71497fa700 1 -- 172.21.15.7:0/3794604499 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5629ea61bd10 con 0x5629ea6d69f0 2024-06-10T01:08:01.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.605+0000 7f71497fa700 10 client.25565 handle_command_reply: tid=0 2024-06-10T01:08:01.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.605+0000 7f71727b0b80 1 client.25565 shutdown 2024-06-10T01:08:01.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.605+0000 7f71727b0b80 1 -- 172.21.15.7:0/3794604499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea6bcf20 msgr2=0x5629ea6c8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:01.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.605+0000 7f71727b0b80 1 --2- 172.21.15.7:0/3794604499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea6bcf20 0x5629ea6c8d80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f715c005c20 tx=0x7f715c0047c0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.605+0000 7f71727b0b80 1 -- 172.21.15.7:0/3794604499 shutdown_connections 2024-06-10T01:08:01.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.605+0000 7f71727b0b80 1 --2- 172.21.15.7:0/3794604499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea6c2f60 0x5629ea6c65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.605+0000 7f71727b0b80 1 --2- 172.21.15.7:0/3794604499 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5629ea6d69f0 0x5629ea6d8e60 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.605+0000 7f71727b0b80 1 --2- 172.21.15.7:0/3794604499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea6bcf20 0x5629ea6c8d80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.605+0000 7f71727b0b80 1 --2- 172.21.15.7:0/3794604499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea6c2640 0x5629ea6c6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.605+0000 7f71497fa700 0 client.25565 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:01.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.605+0000 7f71727b0b80 1 -- 172.21.15.7:0/3794604499 >> 172.21.15.7:0/3794604499 conn(0x5629ea5e7cc0 msgr2=0x5629ea6bb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:01.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.605+0000 7f71727b0b80 1 -- 172.21.15.7:0/3794604499 shutdown_connections 2024-06-10T01:08:01.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.605+0000 7f71727b0b80 1 -- 172.21.15.7:0/3794604499 wait complete. 2024-06-10T01:08:01.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.605+0000 7f71727b0b80 20 client.25565 trim_cache size 0 max 16384 2024-06-10T01:08:01.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.620+0000 7f71727b0b80 1 Processor -- start 2024-06-10T01:08:01.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.621+0000 7f71727b0b80 1 -- start start 2024-06-10T01:08:01.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.621+0000 7f71727b0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea6bcf20 0x5629ea6d2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.621+0000 7f71727b0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea6c2640 0x5629ea6bc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.621+0000 7f71727b0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea6c2f60 0x5629ea6c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.621+0000 7f71727b0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f71601b2d70 con 0x5629ea6c2f60 2024-06-10T01:08:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.621+0000 7f71727b0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71601b0600 con 0x5629ea6bcf20 2024-06-10T01:08:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.621+0000 7f71727b0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f71601ae450 con 0x5629ea6c2640 2024-06-10T01:08:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.621+0000 7f71664ca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea6c2f60 0x5629ea6c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.621+0000 7f71664ca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea6c2f60 0x5629ea6c4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36374/0 (socket says 172.21.15.7:36374) 2024-06-10T01:08:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.621+0000 7f71664ca700 1 -- 172.21.15.7:0/1102871203 learned_addr learned my addr 172.21.15.7:0/1102871203 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.621+0000 7f71654c8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea6c2640 0x5629ea6bc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.621+0000 7f7165cc9700 1 --2- 172.21.15.7:0/1102871203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea6bcf20 0x5629ea6d2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:01.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.621+0000 7f71664ca700 1 -- 172.21.15.7:0/1102871203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea6c2640 msgr2=0x5629ea6bc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:01.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.621+0000 7f71664ca700 1 --2- 172.21.15.7:0/1102871203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea6c2640 0x5629ea6bc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.621+0000 7f71664ca700 1 -- 172.21.15.7:0/1102871203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea6bcf20 msgr2=0x5629ea6d2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:01.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.621+0000 7f71664ca700 1 --2- 172.21.15.7:0/1102871203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea6bcf20 0x5629ea6d2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.621+0000 7f71664ca700 1 -- 172.21.15.7:0/1102871203 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629ea6c45f0 con 0x5629ea6c2f60 2024-06-10T01:08:01.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.621+0000 7f71654c8700 1 --2- 172.21.15.7:0/1102871203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea6c2640 0x5629ea6bc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:01.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.622+0000 7f71664ca700 1 --2- 172.21.15.7:0/1102871203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea6c2f60 0x5629ea6c4fe0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f715800ef90 tx=0x7f7158016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f9aa3f5449c74b9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:01.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.622+0000 7f714b7fe700 1 -- 172.21.15.7:0/1102871203 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7158004d90 con 0x5629ea6c2f60 2024-06-10T01:08:01.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.622+0000 7f714b7fe700 1 -- 172.21.15.7:0/1102871203 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f715809bab0 con 0x5629ea6c2f60 2024-06-10T01:08:01.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.622+0000 7f714b7fe700 1 -- 172.21.15.7:0/1102871203 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7158004ad0 con 0x5629ea6c2f60 2024-06-10T01:08:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.623+0000 7f71727b0b80 1 -- 172.21.15.7:0/1102871203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea6c2f60 msgr2=0x5629ea6c4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.623+0000 7f71727b0b80 1 --2- 172.21.15.7:0/1102871203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea6c2f60 0x5629ea6c4fe0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f715800ef90 tx=0x7f7158016040 comp rx=0 tx=0).stop 2024-06-10T01:08:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.623+0000 7f71727b0b80 1 -- 172.21.15.7:0/1102871203 shutdown_connections 2024-06-10T01:08:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.623+0000 7f71727b0b80 1 --2- 172.21.15.7:0/1102871203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea6c2f60 0x5629ea6c4fe0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.623+0000 7f71727b0b80 1 --2- 172.21.15.7:0/1102871203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea6bcf20 0x5629ea6d2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.623+0000 7f71727b0b80 1 --2- 172.21.15.7:0/1102871203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea6c2640 0x5629ea6bc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.623+0000 7f71727b0b80 1 -- 172.21.15.7:0/1102871203 >> 172.21.15.7:0/1102871203 conn(0x5629ea5e7cc0 msgr2=0x5629ea6b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.623+0000 7f71727b0b80 1 -- 172.21.15.7:0/1102871203 shutdown_connections 2024-06-10T01:08:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.623+0000 7f71727b0b80 1 -- 172.21.15.7:0/1102871203 wait complete. 2024-06-10T01:08:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.623+0000 7f71727b0b80 1 -- start start 2024-06-10T01:08:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.623+0000 7f71727b0b80 1 Processor -- start 2024-06-10T01:08:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.624+0000 7f71727b0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea6bcf20 0x5629ea6cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:01.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.624+0000 7f71727b0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea6c2640 0x5629ea6ddfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:01.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.624+0000 7f71727b0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea6c2f60 0x5629ea6de4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:01.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.624+0000 7f71727b0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f71601aeac0 con 0x5629ea6bcf20 2024-06-10T01:08:01.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.624+0000 7f71727b0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71601aff30 con 0x5629ea6c2f60 2024-06-10T01:08:01.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.624+0000 7f71727b0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f71601af190 con 0x5629ea6c2640 2024-06-10T01:08:01.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.624+0000 7f71727b0b80 10 client.0 _ll_register_callbacks cb 0x7f715c011660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:01.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.624+0000 7f7165cc9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea6bcf20 0x5629ea6cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:01.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.624+0000 7f7165cc9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea6bcf20 0x5629ea6cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36384/0 (socket says 172.21.15.7:36384) 2024-06-10T01:08:01.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.624+0000 7f7165cc9700 1 -- 172.21.15.7:0/2895579052 learned_addr learned my addr 172.21.15.7:0/2895579052 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:01.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.624+0000 7f71654c8700 1 --2- 172.21.15.7:0/2895579052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea6c2640 0x5629ea6ddfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:01.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.624+0000 7f71664ca700 1 --2- 172.21.15.7:0/2895579052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea6c2f60 0x5629ea6de4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:01.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.624+0000 7f7165cc9700 1 -- 172.21.15.7:0/2895579052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea6c2640 msgr2=0x5629ea6ddfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:01.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.624+0000 7f7165cc9700 1 --2- 172.21.15.7:0/2895579052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea6c2640 0x5629ea6ddfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.624+0000 7f7165cc9700 1 -- 172.21.15.7:0/2895579052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea6c2f60 msgr2=0x5629ea6de4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:01.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.624+0000 7f7165cc9700 1 --2- 172.21.15.7:0/2895579052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea6c2f60 0x5629ea6de4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.624+0000 7f7165cc9700 1 -- 172.21.15.7:0/2895579052 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629ea6debe0 con 0x5629ea6bcf20 2024-06-10T01:08:01.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.624+0000 7f7165cc9700 1 --2- 172.21.15.7:0/2895579052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea6bcf20 0x5629ea6cf3c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f715c01e800 tx=0x7f715c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f270636d9764218 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:01.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.624+0000 7f71497fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:08:01.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.624+0000 7f71497fa700 1 -- 172.21.15.7:0/2895579052 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c01ee30 con 0x5629ea6bcf20 2024-06-10T01:08:01.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.624+0000 7f71497fa700 1 -- 172.21.15.7:0/2895579052 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f715c004020 con 0x5629ea6bcf20 2024-06-10T01:08:01.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.624+0000 7f71727b0b80 1 -- 172.21.15.7:0/2895579052 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f715809bab0 con 0x5629ea6bcf20 2024-06-10T01:08:01.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.625+0000 7f71497fa700 1 -- 172.21.15.7:0/2895579052 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f715c004220 con 0x5629ea6bcf20 2024-06-10T01:08:01.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.625+0000 7f71727b0b80 10 client.16032 fetch_fsmap learned FSMap version 29 2024-06-10T01:08:01.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.625+0000 7f71727b0b80 1 -- 172.21.15.7:0/2895579052 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5629ea741630 con 0x5629ea6bcf20 2024-06-10T01:08:01.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.625+0000 7f71497fa700 1 -- 172.21.15.7:0/2895579052 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f715809bab0 con 0x5629ea6bcf20 2024-06-10T01:08:01.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.625+0000 7f71497fa700 1 -- 172.21.15.7:0/2895579052 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f715c01f640 con 0x5629ea6bcf20 2024-06-10T01:08:01.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.625+0000 7f71727b0b80 10 client.16032 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:08:01.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.625+0000 7f71727b0b80 10 client.16032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:01.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.625+0000 7f71727b0b80 20 client.16032 populate_metadata read hostname 'smithi007' 2024-06-10T01:08:01.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.625+0000 7f71727b0b80 1 --2- 172.21.15.7:0/2895579052 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5629ea741ce0 0x5629ea7440d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:01.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.625+0000 7f71727b0b80 4 client.16032 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:08:01.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.625+0000 7f71727b0b80 1 -- 172.21.15.7:0/2895579052 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5629ea616290 con 0x5629ea741ce0 2024-06-10T01:08:01.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.625+0000 7f71654c8700 1 --2- 172.21.15.7:0/2895579052 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5629ea741ce0 0x5629ea7440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:01.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.626+0000 7f71654c8700 1 --2- 172.21.15.7:0/2895579052 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5629ea741ce0 0x5629ea7440d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb4ea82304c9e426 server_cookie=8623a076016b193f in_seq=0 out_seq=0 2024-06-10T01:08:01.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.626+0000 7f71497fa700 10 client.16032 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:01.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.627+0000 7f71497fa700 1 -- 172.21.15.7:0/2895579052 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5629ea616290 con 0x5629ea741ce0 2024-06-10T01:08:01.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.627+0000 7f71497fa700 10 client.16032 handle_command_reply: tid=0 2024-06-10T01:08:01.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.627+0000 7f71727b0b80 1 client.16032 shutdown 2024-06-10T01:08:01.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.627+0000 7f71727b0b80 1 -- 172.21.15.7:0/2895579052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea6bcf20 msgr2=0x5629ea6cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:01.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.627+0000 7f71727b0b80 1 --2- 172.21.15.7:0/2895579052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea6bcf20 0x5629ea6cf3c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f715c01e800 tx=0x7f715c013000 comp rx=0 tx=0).stop 2024-06-10T01:08:01.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.627+0000 7f71727b0b80 1 -- 172.21.15.7:0/2895579052 shutdown_connections 2024-06-10T01:08:01.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.627+0000 7f71727b0b80 1 --2- 172.21.15.7:0/2895579052 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5629ea741ce0 0x5629ea7440d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.627+0000 7f71727b0b80 1 --2- 172.21.15.7:0/2895579052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea6c2f60 0x5629ea6de4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.627+0000 7f71727b0b80 1 --2- 172.21.15.7:0/2895579052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea6bcf20 0x5629ea6cf3c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.627+0000 7f71727b0b80 1 --2- 172.21.15.7:0/2895579052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea6c2640 0x5629ea6ddfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.627+0000 7f71727b0b80 1 -- 172.21.15.7:0/2895579052 >> 172.21.15.7:0/2895579052 conn(0x5629ea5e7cc0 msgr2=0x5629ea6c3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:01.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.627+0000 7f71497fa700 0 client.16032 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:01.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.627+0000 7f71727b0b80 1 -- 172.21.15.7:0/2895579052 shutdown_connections 2024-06-10T01:08:01.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.627+0000 7f71727b0b80 1 -- 172.21.15.7:0/2895579052 wait complete. 2024-06-10T01:08:01.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.628+0000 7f71727b0b80 20 client.16032 trim_cache size 0 max 16384 2024-06-10T01:08:01.627 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:08:01.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.628+0000 7f7167f2d700 1 -- 172.21.15.7:0/4237270456 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f714c0a2ca0 msgr2=0x7f714c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:01.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.628+0000 7f7167f2d700 1 --2- 172.21.15.7:0/4237270456 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f714c0a2ca0 0x7f714c0a5160 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7160199be0 tx=0x7f715c00b500 comp rx=0 tx=0).stop 2024-06-10T01:08:01.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.628+0000 7f7167f2d700 1 -- 172.21.15.7:0/4237270456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7160108500 msgr2=0x7f7160198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:01.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.628+0000 7f7167f2d700 1 --2- 172.21.15.7:0/4237270456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7160108500 0x7f7160198aa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f715800efc0 tx=0x7f715800c3a0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.628+0000 7f7167f2d700 1 -- 172.21.15.7:0/4237270456 shutdown_connections 2024-06-10T01:08:01.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.628+0000 7f7167f2d700 1 --2- 172.21.15.7:0/4237270456 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f714c0a2ca0 0x7f714c0a5160 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.628+0000 7f7167f2d700 1 --2- 172.21.15.7:0/4237270456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7160108500 0x7f7160198aa0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.628+0000 7f7167f2d700 1 --2- 172.21.15.7:0/4237270456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716006a170 0x7f716019e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.628+0000 7f7167f2d700 1 --2- 172.21.15.7:0/4237270456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f716006ab30 0x7f716019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.628+0000 7f7167f2d700 1 -- 172.21.15.7:0/4237270456 >> 172.21.15.7:0/4237270456 conn(0x7f7160078340 msgr2=0x7f71601000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.628+0000 7f7167f2d700 1 -- 172.21.15.7:0/4237270456 shutdown_connections 2024-06-10T01:08:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:01.628+0000 7f7167f2d700 1 -- 172.21.15.7:0/4237270456 wait complete. 2024-06-10T01:08:01.709 INFO:tasks.workunit.client.0.smithi007.stdout: 1 35926 5.05 MB/sec warmup 42 sec latency 18.463 ms 2024-06-10T01:08:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.391+0000 7ff7a00c4640 1 -- 172.21.15.7:0/4139662909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7981024b0 msgr2=0x7ff798108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.391+0000 7ff7a00c4640 1 --2- 172.21.15.7:0/4139662909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7981024b0 0x7ff798108d40 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff790009ca0 tx=0x7ff79002a8c0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.391+0000 7ff7a00c4640 1 -- 172.21.15.7:0/4139662909 shutdown_connections 2024-06-10T01:08:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.391+0000 7ff7a00c4640 1 --2- 172.21.15.7:0/4139662909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7981024b0 0x7ff798108d40 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.391+0000 7ff7a00c4640 1 --2- 172.21.15.7:0/4139662909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff798101af0 0x7ff798101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.391+0000 7ff7a00c4640 1 --2- 172.21.15.7:0/4139662909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff798100900 0x7ff798100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.391+0000 7ff7a00c4640 1 -- 172.21.15.7:0/4139662909 >> 172.21.15.7:0/4139662909 conn(0x7ff7980fbe70 msgr2=0x7ff7980fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.391+0000 7ff7a00c4640 1 -- 172.21.15.7:0/4139662909 shutdown_connections 2024-06-10T01:08:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.391+0000 7ff7a00c4640 1 -- 172.21.15.7:0/4139662909 wait complete. 2024-06-10T01:08:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.392+0000 7ff7a00c4640 1 Processor -- start 2024-06-10T01:08:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.392+0000 7ff7a00c4640 1 -- start start 2024-06-10T01:08:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.392+0000 7ff7a00c4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff798100900 0x7ff798199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.392+0000 7ff7a00c4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff798101af0 0x7ff79819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.392+0000 7ff7a00c4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7981024b0 0x7ff7981a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.392+0000 7ff7a00c4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff79810d6a0 con 0x7ff798100900 2024-06-10T01:08:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.392+0000 7ff7a00c4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff79810d520 con 0x7ff798101af0 2024-06-10T01:08:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.392+0000 7ff7a00c4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff79810d820 con 0x7ff7981024b0 2024-06-10T01:08:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.392+0000 7ff79de60640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff798100900 0x7ff798199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.392+0000 7ff79de60640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff798100900 0x7ff798199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55752/0 (socket says 172.21.15.7:55752) 2024-06-10T01:08:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.392+0000 7ff79de60640 1 -- 172.21.15.7:0/239127085 learned_addr learned my addr 172.21.15.7:0/239127085 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.393+0000 7ff79e661640 1 --2- 172.21.15.7:0/239127085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7981024b0 0x7ff7981a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.393+0000 7ff79d65f640 1 --2- 172.21.15.7:0/239127085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff798101af0 0x7ff79819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.393+0000 7ff79de60640 1 -- 172.21.15.7:0/239127085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7981024b0 msgr2=0x7ff7981a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.393+0000 7ff79de60640 1 --2- 172.21.15.7:0/239127085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7981024b0 0x7ff7981a14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.393+0000 7ff79de60640 1 -- 172.21.15.7:0/239127085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff798101af0 msgr2=0x7ff79819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.393+0000 7ff79de60640 1 --2- 172.21.15.7:0/239127085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff798101af0 0x7ff79819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.393+0000 7ff79de60640 1 -- 172.21.15.7:0/239127085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7981a1b60 con 0x7ff798100900 2024-06-10T01:08:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.393+0000 7ff79e661640 1 --2- 172.21.15.7:0/239127085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7981024b0 0x7ff7981a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.393+0000 7ff79d65f640 1 --2- 172.21.15.7:0/239127085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff798101af0 0x7ff79819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.393+0000 7ff79de60640 1 --2- 172.21.15.7:0/239127085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff798100900 0x7ff798199f40 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff79400e9c0 tx=0x7ff79400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.393+0000 7ff78effd640 1 -- 172.21.15.7:0/239127085 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79400cd80 con 0x7ff798100900 2024-06-10T01:08:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.393+0000 7ff7a00c4640 1 -- 172.21.15.7:0/239127085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7981a1e50 con 0x7ff798100900 2024-06-10T01:08:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.393+0000 7ff78effd640 1 -- 172.21.15.7:0/239127085 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7940035f0 con 0x7ff798100900 2024-06-10T01:08:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.393+0000 7ff78effd640 1 -- 172.21.15.7:0/239127085 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff794010690 con 0x7ff798100900 2024-06-10T01:08:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.393+0000 7ff7a00c4640 1 -- 172.21.15.7:0/239127085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7981a22b0 con 0x7ff798100900 2024-06-10T01:08:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.396+0000 7ff78effd640 1 -- 172.21.15.7:0/239127085 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7ff7940037a0 con 0x7ff798100900 2024-06-10T01:08:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.396+0000 7ff7a2dd8740 1 Processor -- start 2024-06-10T01:08:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.396+0000 7ff7a2dd8740 1 -- start start 2024-06-10T01:08:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.396+0000 7ff7a2dd8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c2d7dd010 0x562c2d7dd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.396+0000 7ff7a2dd8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c2d7dda00 0x562c2d7e2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.396+0000 7ff7a2dd8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c2d7e2b70 0x562c2d7e5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.396+0000 7ff7a2dd8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562c2d7404f0 con 0x562c2d7dd010 2024-06-10T01:08:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.396+0000 7ff7a2dd8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562c2d6de870 con 0x562c2d7e2b70 2024-06-10T01:08:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.396+0000 7ff7a2dd8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562c2d73c4b0 con 0x562c2d7dda00 2024-06-10T01:08:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.397+0000 7ff79e661640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c2d7dd010 0x562c2d7dd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.397+0000 7ff79d65f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c2d7e2b70 0x562c2d7e5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.397+0000 7ff79de60640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c2d7dda00 0x562c2d7e2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.397+0000 7ff79e661640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c2d7dd010 0x562c2d7dd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55754/0 (socket says 172.21.15.7:55754) 2024-06-10T01:08:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.397+0000 7ff79e661640 1 -- 172.21.15.7:0/4013457552 learned_addr learned my addr 172.21.15.7:0/4013457552 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.397+0000 7ff78effd640 1 --2- 172.21.15.7:0/239127085 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff7840a2b80 0x7ff7840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.397+0000 7ff79d65f640 1 -- 172.21.15.7:0/4013457552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c2d7dda00 msgr2=0x562c2d7e2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.397+0000 7ff79d65f640 1 --2- 172.21.15.7:0/4013457552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c2d7dda00 0x562c2d7e2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.397+0000 7ff79e661640 1 --2- 172.21.15.7:0/239127085 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff7840a2b80 0x7ff7840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.397+0000 7ff79d65f640 1 -- 172.21.15.7:0/4013457552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c2d7dd010 msgr2=0x562c2d7dd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.397+0000 7ff79d65f640 1 --2- 172.21.15.7:0/4013457552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c2d7dd010 0x562c2d7dd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.397+0000 7ff79d65f640 1 -- 172.21.15.7:0/4013457552 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c2d6fa9b0 con 0x562c2d7e2b70 2024-06-10T01:08:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.397+0000 7ff78effd640 1 -- 172.21.15.7:0/239127085 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7ff7940cf740 con 0x7ff798100900 2024-06-10T01:08:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.397+0000 7ff79d65f640 1 --2- 172.21.15.7:0/4013457552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c2d7e2b70 0x562c2d7e5000 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff788009fa0 tx=0x7ff7880118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea7c640a6f2a0d32 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.397+0000 7ff783fff640 1 -- 172.21.15.7:0/4013457552 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788004030 con 0x562c2d7e2b70 2024-06-10T01:08:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.397+0000 7ff79e661640 1 --2- 172.21.15.7:0/239127085 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff7840a2b80 0x7ff7840a5040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x562c2d7de260 tx=0x7ff79002a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.397+0000 7ff783fff640 1 -- 172.21.15.7:0/4013457552 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff788005500 con 0x562c2d7e2b70 2024-06-10T01:08:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.397+0000 7ff783fff640 1 -- 172.21.15.7:0/4013457552 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788004d50 con 0x562c2d7e2b70 2024-06-10T01:08:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.398+0000 7ff7a2dd8740 1 -- 172.21.15.7:0/4013457552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c2d7e2b70 msgr2=0x562c2d7e5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.398+0000 7ff7a2dd8740 1 --2- 172.21.15.7:0/4013457552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c2d7e2b70 0x562c2d7e5000 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff788009fa0 tx=0x7ff7880118d0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.398+0000 7ff7a2dd8740 1 -- 172.21.15.7:0/4013457552 shutdown_connections 2024-06-10T01:08:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.398+0000 7ff7a2dd8740 1 --2- 172.21.15.7:0/4013457552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c2d7e2b70 0x562c2d7e5000 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.398+0000 7ff7a2dd8740 1 --2- 172.21.15.7:0/4013457552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c2d7dda00 0x562c2d7e2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.398+0000 7ff7a2dd8740 1 --2- 172.21.15.7:0/4013457552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c2d7dd010 0x562c2d7dd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.398+0000 7ff7a2dd8740 1 -- 172.21.15.7:0/4013457552 >> 172.21.15.7:0/4013457552 conn(0x562c2d7d8680 msgr2=0x562c2d7d8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.398+0000 7ff7a2dd8740 1 -- 172.21.15.7:0/4013457552 shutdown_connections 2024-06-10T01:08:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.398+0000 7ff7a2dd8740 1 -- 172.21.15.7:0/4013457552 wait complete. 2024-06-10T01:08:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.398+0000 7ff7a2dd8740 1 -- start start 2024-06-10T01:08:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.399+0000 7ff7a2dd8740 1 Processor -- start 2024-06-10T01:08:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.399+0000 7ff7a2dd8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c2d7dd010 0x562c2d7e7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.399+0000 7ff7a2dd8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c2d7dda00 0x562c2d7e7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.399+0000 7ff7a2dd8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c2d7e2b70 0x562c2d7ee7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.399+0000 7ff7a2dd8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562c2d7e5b20 con 0x562c2d7dda00 2024-06-10T01:08:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.399+0000 7ff7a2dd8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562c2d6c9ae0 con 0x562c2d7dd010 2024-06-10T01:08:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.399+0000 7ff7a2dd8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562c2d7e5ca0 con 0x562c2d7e2b70 2024-06-10T01:08:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.399+0000 7ff7a2dd8740 10 client.0 _ll_register_callbacks cb 0x562c2d7029b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.399+0000 7ff79d65f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c2d7dda00 0x562c2d7e7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.399+0000 7ff79de60640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c2d7dd010 0x562c2d7e7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.399+0000 7ff79d65f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c2d7dda00 0x562c2d7e7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55768/0 (socket says 172.21.15.7:55768) 2024-06-10T01:08:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.399+0000 7ff79de60640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c2d7dd010 0x562c2d7e7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:41358/0 (socket says 172.21.15.7:41358) 2024-06-10T01:08:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.399+0000 7ff79d65f640 1 -- 172.21.15.7:0/1348159007 learned_addr learned my addr 172.21.15.7:0/1348159007 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.399+0000 7ff79e661640 1 --2- 172.21.15.7:0/1348159007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c2d7e2b70 0x562c2d7ee7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.399+0000 7ff79de60640 1 -- 172.21.15.7:0/1348159007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c2d7e2b70 msgr2=0x562c2d7ee7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.399+0000 7ff79de60640 1 --2- 172.21.15.7:0/1348159007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c2d7e2b70 0x562c2d7ee7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.399+0000 7ff79de60640 1 -- 172.21.15.7:0/1348159007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c2d7dda00 msgr2=0x562c2d7e7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.399+0000 7ff79de60640 1 --2- 172.21.15.7:0/1348159007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c2d7dda00 0x562c2d7e7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.399+0000 7ff79de60640 1 -- 172.21.15.7:0/1348159007 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c2d454210 con 0x562c2d7dd010 2024-06-10T01:08:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.400+0000 7ff79d65f640 1 --2- 172.21.15.7:0/1348159007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c2d7dda00 0x562c2d7e7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.400+0000 7ff79de60640 1 --2- 172.21.15.7:0/1348159007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c2d7dd010 0x562c2d7e7230 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff7940ccf10 tx=0x7ff79409bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=598f32802a3dbd1c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.400+0000 7ff7817fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:08:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.400+0000 7ff7817fa640 1 -- 172.21.15.7:0/1348159007 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79409c320 con 0x562c2d7dd010 2024-06-10T01:08:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.400+0000 7ff7817fa640 1 -- 172.21.15.7:0/1348159007 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7ff794097970 con 0x562c2d7dd010 2024-06-10T01:08:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.400+0000 7ff7a2dd8740 1 -- 172.21.15.7:0/1348159007 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff788005500 con 0x562c2d7dd010 2024-06-10T01:08:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.400+0000 7ff7817fa640 1 -- 172.21.15.7:0/1348159007 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff79409bc20 con 0x562c2d7dd010 2024-06-10T01:08:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.400+0000 7ff7a2dd8740 10 client.25583 fetch_fsmap learned FSMap version 29 2024-06-10T01:08:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.400+0000 7ff7a2dd8740 1 -- 172.21.15.7:0/1348159007 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562c2d520620 con 0x562c2d7dd010 2024-06-10T01:08:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.400+0000 7ff7817fa640 1 -- 172.21.15.7:0/1348159007 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff788005500 con 0x562c2d7dd010 2024-06-10T01:08:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.401+0000 7ff7817fa640 1 -- 172.21.15.7:0/1348159007 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7ff79409b070 con 0x562c2d7dd010 2024-06-10T01:08:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.401+0000 7ff7a2dd8740 10 client.25583 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:08:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.401+0000 7ff7a2dd8740 10 client.25583 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.401+0000 7ff7a2dd8740 20 client.25583 populate_metadata read hostname 'smithi007' 2024-06-10T01:08:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.401+0000 7ff7a2dd8740 1 --2- 172.21.15.7:0/1348159007 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562c2d7f30f0 0x562c2d7f5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.401+0000 7ff7a2dd8740 4 client.25583 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:08:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.401+0000 7ff7a2dd8740 1 -- 172.21.15.7:0/1348159007 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c2d753f10 con 0x562c2d7f30f0 2024-06-10T01:08:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.401+0000 7ff79d65f640 1 --2- 172.21.15.7:0/1348159007 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562c2d7f30f0 0x562c2d7f5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.401+0000 7ff7817fa640 10 client.25583 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.401+0000 7ff79d65f640 1 --2- 172.21.15.7:0/1348159007 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562c2d7f30f0 0x562c2d7f5520 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa1b90fdc72c5d74 server_cookie=d03a5d658ead55eb in_seq=0 out_seq=0 2024-06-10T01:08:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.402+0000 7ff7817fa640 1 -- 172.21.15.7:0/1348159007 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562c2d753f10 con 0x562c2d7f30f0 2024-06-10T01:08:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.402+0000 7ff7817fa640 10 client.25583 handle_command_reply: tid=0 2024-06-10T01:08:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.402+0000 7ff7a2dd8740 1 client.25583 shutdown 2024-06-10T01:08:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.402+0000 7ff7a2dd8740 1 -- 172.21.15.7:0/1348159007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c2d7dd010 msgr2=0x562c2d7e7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.402+0000 7ff7a2dd8740 1 --2- 172.21.15.7:0/1348159007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c2d7dd010 0x562c2d7e7230 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff7940ccf10 tx=0x7ff79409bab0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.402+0000 7ff7a2dd8740 1 -- 172.21.15.7:0/1348159007 shutdown_connections 2024-06-10T01:08:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.402+0000 7ff7a2dd8740 1 --2- 172.21.15.7:0/1348159007 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562c2d7f30f0 0x562c2d7f5520 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.402+0000 7ff7a2dd8740 1 --2- 172.21.15.7:0/1348159007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c2d7e2b70 0x562c2d7ee7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.402+0000 7ff7a2dd8740 1 --2- 172.21.15.7:0/1348159007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c2d7dda00 0x562c2d7e7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.402+0000 7ff7817fa640 0 client.25583 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.402+0000 7ff7a2dd8740 1 --2- 172.21.15.7:0/1348159007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c2d7dd010 0x562c2d7e7230 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.402+0000 7ff7a2dd8740 1 -- 172.21.15.7:0/1348159007 >> 172.21.15.7:0/1348159007 conn(0x562c2d7d8680 msgr2=0x562c2d7d77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.402+0000 7ff7a2dd8740 1 -- 172.21.15.7:0/1348159007 shutdown_connections 2024-06-10T01:08:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.402+0000 7ff7a2dd8740 1 -- 172.21.15.7:0/1348159007 wait complete. 2024-06-10T01:08:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.402+0000 7ff7a2dd8740 20 client.25583 trim_cache size 0 max 16384 2024-06-10T01:08:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.418+0000 7ff7a2dd8740 1 Processor -- start 2024-06-10T01:08:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.418+0000 7ff7a2dd8740 1 -- start start 2024-06-10T01:08:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.419+0000 7ff7a2dd8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c2d7dd010 0x562c2d7dbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.419+0000 7ff7a2dd8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c2d7dda00 0x562c2d7ee770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.419+0000 7ff7a2dd8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c2d7e2b70 0x562c2d7e4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.419+0000 7ff7a2dd8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981af1b0 con 0x562c2d7e2b70 2024-06-10T01:08:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.419+0000 7ff7a2dd8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981aca80 con 0x562c2d7dda00 2024-06-10T01:08:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.419+0000 7ff7a2dd8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981aa8d0 con 0x562c2d7dd010 2024-06-10T01:08:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.419+0000 7ff79d65f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c2d7dda00 0x562c2d7ee770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.419+0000 7ff79de60640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c2d7dd010 0x562c2d7dbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.419+0000 7ff79d65f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c2d7dda00 0x562c2d7ee770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:41370/0 (socket says 172.21.15.7:41370) 2024-06-10T01:08:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.419+0000 7ff79d65f640 1 -- 172.21.15.7:0/688782027 learned_addr learned my addr 172.21.15.7:0/688782027 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.419+0000 7ff79e661640 1 --2- 172.21.15.7:0/688782027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c2d7e2b70 0x562c2d7e4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.419+0000 7ff79d65f640 1 -- 172.21.15.7:0/688782027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c2d7dd010 msgr2=0x562c2d7dbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.419+0000 7ff79d65f640 1 --2- 172.21.15.7:0/688782027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c2d7dd010 0x562c2d7dbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.419+0000 7ff79d65f640 1 -- 172.21.15.7:0/688782027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c2d7e2b70 msgr2=0x562c2d7e4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.419+0000 7ff79d65f640 1 --2- 172.21.15.7:0/688782027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c2d7e2b70 0x562c2d7e4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.419+0000 7ff79d65f640 1 -- 172.21.15.7:0/688782027 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c2d7d84b0 con 0x562c2d7dda00 2024-06-10T01:08:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.419+0000 7ff79d65f640 1 --2- 172.21.15.7:0/688782027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c2d7dda00 0x562c2d7ee770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff788009d60 tx=0x7ff788004840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=544cb3ed9e3fe838 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.420+0000 7ff7837fe640 1 -- 172.21.15.7:0/688782027 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78801d040 con 0x562c2d7dda00 2024-06-10T01:08:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.420+0000 7ff7837fe640 1 -- 172.21.15.7:0/688782027 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff78800fc40 con 0x562c2d7dda00 2024-06-10T01:08:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.420+0000 7ff7837fe640 1 -- 172.21.15.7:0/688782027 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788003a70 con 0x562c2d7dda00 2024-06-10T01:08:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.420+0000 7ff7a2dd8740 1 -- 172.21.15.7:0/688782027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c2d7dda00 msgr2=0x562c2d7ee770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.420+0000 7ff7a2dd8740 1 --2- 172.21.15.7:0/688782027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c2d7dda00 0x562c2d7ee770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff788009d60 tx=0x7ff788004840 comp rx=0 tx=0).stop 2024-06-10T01:08:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.420+0000 7ff7a2dd8740 1 -- 172.21.15.7:0/688782027 shutdown_connections 2024-06-10T01:08:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.420+0000 7ff7a2dd8740 1 --2- 172.21.15.7:0/688782027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c2d7e2b70 0x562c2d7e4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.420+0000 7ff7a2dd8740 1 --2- 172.21.15.7:0/688782027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c2d7dda00 0x562c2d7ee770 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.420+0000 7ff7a2dd8740 1 --2- 172.21.15.7:0/688782027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c2d7dd010 0x562c2d7dbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.420+0000 7ff7a2dd8740 1 -- 172.21.15.7:0/688782027 >> 172.21.15.7:0/688782027 conn(0x562c2d7d8680 msgr2=0x562c2d7db640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.420+0000 7ff7a2dd8740 1 -- 172.21.15.7:0/688782027 shutdown_connections 2024-06-10T01:08:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.420+0000 7ff7a2dd8740 1 -- 172.21.15.7:0/688782027 wait complete. 2024-06-10T01:08:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.421+0000 7ff7a2dd8740 1 -- start start 2024-06-10T01:08:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.421+0000 7ff7a2dd8740 1 Processor -- start 2024-06-10T01:08:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.421+0000 7ff7a2dd8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c2d7dd010 0x562c2d7eb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.421+0000 7ff7a2dd8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c2d7dda00 0x562c2d7ebbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.421+0000 7ff7a2dd8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c2d7e2b70 0x562c2d85bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.421+0000 7ff7a2dd8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981aaf40 con 0x562c2d7e2b70 2024-06-10T01:08:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.421+0000 7ff7a2dd8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981ac3b0 con 0x562c2d7dda00 2024-06-10T01:08:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.421+0000 7ff7a2dd8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981ab610 con 0x562c2d7dd010 2024-06-10T01:08:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.421+0000 7ff7a2dd8740 10 client.0 _ll_register_callbacks cb 0x7ff794003e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.421+0000 7ff79e661640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c2d7e2b70 0x562c2d85bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.421+0000 7ff79de60640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c2d7dd010 0x562c2d7eb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.421+0000 7ff79de60640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c2d7dd010 0x562c2d7eb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47134/0 (socket says 172.21.15.7:47134) 2024-06-10T01:08:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.421+0000 7ff79de60640 1 -- 172.21.15.7:0/1932415624 learned_addr learned my addr 172.21.15.7:0/1932415624 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.421+0000 7ff79e661640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c2d7e2b70 0x562c2d85bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55782/0 (socket says 172.21.15.7:55782) 2024-06-10T01:08:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.421+0000 7ff79d65f640 1 --2- 172.21.15.7:0/1932415624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c2d7dda00 0x562c2d7ebbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.422+0000 7ff79de60640 1 -- 172.21.15.7:0/1932415624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c2d7dda00 msgr2=0x562c2d7ebbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.422+0000 7ff79de60640 1 --2- 172.21.15.7:0/1932415624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c2d7dda00 0x562c2d7ebbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.422+0000 7ff79de60640 1 -- 172.21.15.7:0/1932415624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c2d7e2b70 msgr2=0x562c2d85bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.422+0000 7ff79de60640 1 --2- 172.21.15.7:0/1932415624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c2d7e2b70 0x562c2d85bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.422+0000 7ff79de60640 1 -- 172.21.15.7:0/1932415624 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c2d85c330 con 0x562c2d7dd010 2024-06-10T01:08:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.422+0000 7ff79de60640 1 --2- 172.21.15.7:0/1932415624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c2d7dd010 0x562c2d7eb670 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff794003ec0 tx=0x7ff794003d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0760a7c89801365 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.422+0000 7ff7817fa640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:08:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.422+0000 7ff7817fa640 1 -- 172.21.15.7:0/1932415624 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79409c320 con 0x562c2d7dd010 2024-06-10T01:08:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.422+0000 7ff7a2dd8740 1 -- 172.21.15.7:0/1932415624 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff78800fc40 con 0x562c2d7dd010 2024-06-10T01:08:02.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.422+0000 7ff7817fa640 1 -- 172.21.15.7:0/1932415624 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7ff794097970 con 0x562c2d7dd010 2024-06-10T01:08:02.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.422+0000 7ff7817fa640 1 -- 172.21.15.7:0/1932415624 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff79409ffd0 con 0x562c2d7dd010 2024-06-10T01:08:02.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.422+0000 7ff7a2dd8740 10 client.25618 fetch_fsmap learned FSMap version 29 2024-06-10T01:08:02.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.422+0000 7ff7a2dd8740 1 -- 172.21.15.7:0/1932415624 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562c2d85d680 con 0x562c2d7dd010 2024-06-10T01:08:02.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.422+0000 7ff7817fa640 1 -- 172.21.15.7:0/1932415624 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff78800fc40 con 0x562c2d7dd010 2024-06-10T01:08:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.423+0000 7ff7817fa640 1 -- 172.21.15.7:0/1932415624 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7ff79409aae0 con 0x562c2d7dd010 2024-06-10T01:08:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.423+0000 7ff7a2dd8740 10 client.25618 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:08:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.423+0000 7ff7a2dd8740 10 client.25618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.423+0000 7ff7a2dd8740 20 client.25618 populate_metadata read hostname 'smithi007' 2024-06-10T01:08:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.423+0000 7ff7a2dd8740 1 --2- 172.21.15.7:0/1932415624 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562c2d85dfd0 0x562c2d8603f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.423+0000 7ff7a2dd8740 4 client.25618 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:08:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.423+0000 7ff7a2dd8740 1 -- 172.21.15.7:0/1932415624 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562c2d74dc20 con 0x562c2d85dfd0 2024-06-10T01:08:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.423+0000 7ff79d65f640 1 --2- 172.21.15.7:0/1932415624 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562c2d85dfd0 0x562c2d8603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.423+0000 7ff79d65f640 1 --2- 172.21.15.7:0/1932415624 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562c2d85dfd0 0x562c2d8603f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a938d121fd739499 server_cookie=8cb4cff432b05467 in_seq=0 out_seq=0 2024-06-10T01:08:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.423+0000 7ff7817fa640 10 client.25618 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.531+0000 7ff7817fa640 1 -- 172.21.15.7:0/1932415624 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562c2d74dc20 con 0x562c2d85dfd0 2024-06-10T01:08:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.531+0000 7ff7817fa640 10 client.25618 handle_command_reply: tid=0 2024-06-10T01:08:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.531+0000 7ff7a2dd8740 1 client.25618 shutdown 2024-06-10T01:08:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.532+0000 7ff7a2dd8740 1 -- 172.21.15.7:0/1932415624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c2d7dd010 msgr2=0x562c2d7eb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.532+0000 7ff7a2dd8740 1 --2- 172.21.15.7:0/1932415624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c2d7dd010 0x562c2d7eb670 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff794003ec0 tx=0x7ff794003d40 comp rx=0 tx=0).stop 2024-06-10T01:08:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.532+0000 7ff7a2dd8740 1 -- 172.21.15.7:0/1932415624 shutdown_connections 2024-06-10T01:08:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.532+0000 7ff7a2dd8740 1 --2- 172.21.15.7:0/1932415624 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562c2d85dfd0 0x562c2d8603f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.532+0000 7ff7a2dd8740 1 --2- 172.21.15.7:0/1932415624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c2d7e2b70 0x562c2d85bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.532+0000 7ff7817fa640 0 client.25618 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.532+0000 7ff7a2dd8740 1 --2- 172.21.15.7:0/1932415624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c2d7dda00 0x562c2d7ebbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.532+0000 7ff7a2dd8740 1 --2- 172.21.15.7:0/1932415624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c2d7dd010 0x562c2d7eb670 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.532+0000 7ff7a2dd8740 1 -- 172.21.15.7:0/1932415624 >> 172.21.15.7:0/1932415624 conn(0x562c2d7d8680 msgr2=0x562c2d7e6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.532+0000 7ff7a2dd8740 1 -- 172.21.15.7:0/1932415624 shutdown_connections 2024-06-10T01:08:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.532+0000 7ff7a2dd8740 1 -- 172.21.15.7:0/1932415624 wait complete. 2024-06-10T01:08:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.532+0000 7ff7a2dd8740 20 client.25618 trim_cache size 0 max 16384 2024-06-10T01:08:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:08:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:08:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:08:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.532+0000 7ff7a00c4640 1 -- 172.21.15.7:0/239127085 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff7840a2b80 msgr2=0x7ff7840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.532+0000 7ff7a00c4640 1 --2- 172.21.15.7:0/239127085 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff7840a2b80 0x7ff7840a5040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x562c2d7de260 tx=0x7ff79002a8c0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.532+0000 7ff7a00c4640 1 -- 172.21.15.7:0/239127085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff798100900 msgr2=0x7ff798199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.532+0000 7ff7a00c4640 1 --2- 172.21.15.7:0/239127085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff798100900 0x7ff798199f40 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff79400e9c0 tx=0x7ff79400ee80 comp rx=0 tx=0).stop 2024-06-10T01:08:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.532+0000 7ff7a00c4640 1 -- 172.21.15.7:0/239127085 shutdown_connections 2024-06-10T01:08:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.532+0000 7ff7a00c4640 1 --2- 172.21.15.7:0/239127085 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff7840a2b80 0x7ff7840a5040 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.533+0000 7ff7a00c4640 1 --2- 172.21.15.7:0/239127085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7981024b0 0x7ff7981a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.533+0000 7ff7a00c4640 1 --2- 172.21.15.7:0/239127085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff798101af0 0x7ff79819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.533+0000 7ff7a00c4640 1 --2- 172.21.15.7:0/239127085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff798100900 0x7ff798199f40 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.533+0000 7ff7a00c4640 1 -- 172.21.15.7:0/239127085 >> 172.21.15.7:0/239127085 conn(0x7ff7980fbe70 msgr2=0x7ff798109700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.533+0000 7ff7a00c4640 1 -- 172.21.15.7:0/239127085 shutdown_connections 2024-06-10T01:08:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:02.533+0000 7ff7a00c4640 1 -- 172.21.15.7:0/239127085 wait complete. 2024-06-10T01:08:02.709 INFO:tasks.workunit.client.0.smithi007.stdout: 1 38856 5.27 MB/sec warmup 43 sec latency 35.439 ms 2024-06-10T01:08:03.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:02 smithi175 ceph-mon[36636]: pgmap v493: 129 pgs: 129 active+clean; 25 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 447 KiB/s wr, 43 op/s 2024-06-10T01:08:03.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:02 smithi007 ceph-mon[32398]: pgmap v493: 129 pgs: 129 active+clean; 25 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 447 KiB/s wr, 43 op/s 2024-06-10T01:08:03.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:02 smithi078 ceph-mon[36683]: pgmap v493: 129 pgs: 129 active+clean; 25 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 447 KiB/s wr, 43 op/s 2024-06-10T01:08:03.650 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:08:03.654 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:08:03.709 INFO:tasks.workunit.client.0.smithi007.stdout: 1 41199 5.39 MB/sec warmup 44 sec latency 34.768 ms 2024-06-10T01:08:03.923 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:08:04.709 INFO:tasks.workunit.client.0.smithi007.stdout: 1 43524 5.58 MB/sec warmup 45 sec latency 17.774 ms 2024-06-10T01:08:05.152 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:04 smithi007 ceph-mon[32398]: pgmap v494: 129 pgs: 129 active+clean; 25 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 101 KiB/s wr, 34 op/s 2024-06-10T01:08:05.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:04 smithi175 ceph-mon[36636]: pgmap v494: 129 pgs: 129 active+clean; 25 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 101 KiB/s wr, 34 op/s 2024-06-10T01:08:05.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:04 smithi078 ceph-mon[36683]: pgmap v494: 129 pgs: 129 active+clean; 25 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 101 KiB/s wr, 34 op/s 2024-06-10T01:08:05.710 INFO:tasks.workunit.client.0.smithi007.stdout: 1 46559 5.86 MB/sec warmup 46 sec latency 69.061 ms 2024-06-10T01:08:05.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.726+0000 7f36092c0700 1 -- 172.21.15.7:0/814180947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3604103150 msgr2=0x7f3604107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:05.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.726+0000 7f36092c0700 1 --2- 172.21.15.7:0/814180947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3604103150 0x7f3604107140 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f35ec0099c0 tx=0x7f35ec009cd0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.726+0000 7f36092c0700 1 -- 172.21.15.7:0/814180947 shutdown_connections 2024-06-10T01:08:05.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.726+0000 7f36092c0700 1 --2- 172.21.15.7:0/814180947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3604107870 0x7f360410f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.726+0000 7f36092c0700 1 --2- 172.21.15.7:0/814180947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36041027a0 0x7f3604102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.726+0000 7f36092c0700 1 --2- 172.21.15.7:0/814180947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3604103150 0x7f3604107140 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.726+0000 7f36092c0700 1 -- 172.21.15.7:0/814180947 >> 172.21.15.7:0/814180947 conn(0x7f3604079570 msgr2=0x7f3604079980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:05.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.726+0000 7f36092c0700 1 -- 172.21.15.7:0/814180947 shutdown_connections 2024-06-10T01:08:05.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.726+0000 7f36092c0700 1 -- 172.21.15.7:0/814180947 wait complete. 2024-06-10T01:08:05.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.727+0000 7f36092c0700 1 Processor -- start 2024-06-10T01:08:05.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.728+0000 7f36092c0700 1 -- start start 2024-06-10T01:08:05.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.728+0000 7f36092c0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36041027a0 0x7f360419ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:05.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.728+0000 7f36092c0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3604103150 0x7f360419b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:05.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.728+0000 7f36092c0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3604107870 0x7f360419f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:05.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.728+0000 7f36027fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3604103150 0x7f360419b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:05.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.728+0000 7f3602ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36041027a0 0x7f360419ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:05.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.728+0000 7f36092c0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f36041127a0 con 0x7f3604107870 2024-06-10T01:08:05.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.728+0000 7f36092c0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3604112620 con 0x7f36041027a0 2024-06-10T01:08:05.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.728+0000 7f36027fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3604103150 0x7f360419b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47154/0 (socket says 172.21.15.7:47154) 2024-06-10T01:08:05.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.728+0000 7f36027fc700 1 -- 172.21.15.7:0/2206296687 learned_addr learned my addr 172.21.15.7:0/2206296687 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:05.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.728+0000 7f36092c0700 1 -- 172.21.15.7:0/2206296687 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3604112920 con 0x7f3604103150 2024-06-10T01:08:05.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.728+0000 7f36037fe700 1 --2- 172.21.15.7:0/2206296687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3604107870 0x7f360419f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:05.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.729+0000 7f3602ffd700 1 -- 172.21.15.7:0/2206296687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3604103150 msgr2=0x7f360419b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:05.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.729+0000 7f3602ffd700 1 --2- 172.21.15.7:0/2206296687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3604103150 0x7f360419b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.729+0000 7f3602ffd700 1 -- 172.21.15.7:0/2206296687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3604107870 msgr2=0x7f360419f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:05.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.729+0000 7f3602ffd700 1 --2- 172.21.15.7:0/2206296687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3604107870 0x7f360419f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.729+0000 7f3602ffd700 1 -- 172.21.15.7:0/2206296687 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f360419fdb0 con 0x7f36041027a0 2024-06-10T01:08:05.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.729+0000 7f36027fc700 1 --2- 172.21.15.7:0/2206296687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3604103150 0x7f360419b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:05.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.729+0000 7f36037fe700 1 --2- 172.21.15.7:0/2206296687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3604107870 0x7f360419f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:05.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.729+0000 7f3602ffd700 1 --2- 172.21.15.7:0/2206296687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36041027a0 0x7f360419ada0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f35f400e9f0 tx=0x7f35f400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:05.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.729+0000 7f35fbfff700 1 -- 172.21.15.7:0/2206296687 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f400c9c0 con 0x7f36041027a0 2024-06-10T01:08:05.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.729+0000 7f36092c0700 1 -- 172.21.15.7:0/2206296687 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36041a00a0 con 0x7f36041027a0 2024-06-10T01:08:05.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.729+0000 7f36092c0700 1 -- 172.21.15.7:0/2206296687 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36041a7920 con 0x7f36041027a0 2024-06-10T01:08:05.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.729+0000 7f35fbfff700 1 -- 172.21.15.7:0/2206296687 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35f400cb60 con 0x7f36041027a0 2024-06-10T01:08:05.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.729+0000 7f35fbfff700 1 -- 172.21.15.7:0/2206296687 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f4018680 con 0x7f36041027a0 2024-06-10T01:08:05.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.731+0000 7f35fbfff700 1 -- 172.21.15.7:0/2206296687 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f35f4018820 con 0x7f36041027a0 2024-06-10T01:08:05.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.732+0000 7f35fbfff700 1 --2- 172.21.15.7:0/2206296687 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35f00a2c20 0x7f35f00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:05.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.732+0000 7f36027fc700 1 --2- 172.21.15.7:0/2206296687 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35f00a2c20 0x7f35f00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:05.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.732+0000 7f35fbfff700 1 -- 172.21.15.7:0/2206296687 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f35f40cf670 con 0x7f36041027a0 2024-06-10T01:08:05.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.733+0000 7f36027fc700 1 --2- 172.21.15.7:0/2206296687 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35f00a2c20 0x7f35f00a50e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f360410f7a0 tx=0x7f35ec005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:05.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.733+0000 7f3613b43b80 1 Processor -- start 2024-06-10T01:08:05.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.733+0000 7f3613b43b80 1 -- start start 2024-06-10T01:08:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.733+0000 7f3613b43b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb8bd5dfe0 0x55eb8bd5e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.734+0000 7f3613b43b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb8bd5e900 0x55eb8bd582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.734+0000 7f3613b43b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb8bd58820 0x55eb8bd58c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.734+0000 7f3613b43b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55eb8bc4afe0 con 0x55eb8bd5e900 2024-06-10T01:08:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.734+0000 7f3613b43b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55eb8bca98d0 con 0x55eb8bd5dfe0 2024-06-10T01:08:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.734+0000 7f3613b43b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55eb8bc114e0 con 0x55eb8bd58820 2024-06-10T01:08:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.734+0000 7f36027fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb8bd5e900 0x55eb8bd582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.734+0000 7f3602ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb8bd5dfe0 0x55eb8bd5e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.734+0000 7f36027fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb8bd5e900 0x55eb8bd582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55820/0 (socket says 172.21.15.7:55820) 2024-06-10T01:08:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.734+0000 7f36027fc700 1 -- 172.21.15.7:0/1338786645 learned_addr learned my addr 172.21.15.7:0/1338786645 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.734+0000 7f36037fe700 1 --2- 172.21.15.7:0/1338786645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb8bd58820 0x55eb8bd58c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.734+0000 7f3602ffd700 1 -- 172.21.15.7:0/1338786645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb8bd58820 msgr2=0x55eb8bd58c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.734+0000 7f3602ffd700 1 --2- 172.21.15.7:0/1338786645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb8bd58820 0x55eb8bd58c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.734+0000 7f3602ffd700 1 -- 172.21.15.7:0/1338786645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb8bd5e900 msgr2=0x55eb8bd582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.734+0000 7f3602ffd700 1 --2- 172.21.15.7:0/1338786645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb8bd5e900 0x55eb8bd582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.734+0000 7f3602ffd700 1 -- 172.21.15.7:0/1338786645 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb8bbf8950 con 0x55eb8bd5dfe0 2024-06-10T01:08:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.734+0000 7f36027fc700 1 --2- 172.21.15.7:0/1338786645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb8bd5e900 0x55eb8bd582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.734+0000 7f36037fe700 1 --2- 172.21.15.7:0/1338786645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb8bd58820 0x55eb8bd58c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:05.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.734+0000 7f3602ffd700 1 --2- 172.21.15.7:0/1338786645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb8bd5dfe0 0x55eb8bd5e3c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f35f4000c00 tx=0x7f35f40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=426ffe03035c409d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:05.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.735+0000 7f35f97fa700 1 -- 172.21.15.7:0/1338786645 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f409ca10 con 0x55eb8bd5dfe0 2024-06-10T01:08:05.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.735+0000 7f35f97fa700 1 -- 172.21.15.7:0/1338786645 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35f40a0dc0 con 0x55eb8bd5dfe0 2024-06-10T01:08:05.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.735+0000 7f35f97fa700 1 -- 172.21.15.7:0/1338786645 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f409c0c0 con 0x55eb8bd5dfe0 2024-06-10T01:08:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.735+0000 7f3613b43b80 1 -- 172.21.15.7:0/1338786645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb8bd5dfe0 msgr2=0x55eb8bd5e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.735+0000 7f3613b43b80 1 --2- 172.21.15.7:0/1338786645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb8bd5dfe0 0x55eb8bd5e3c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f35f4000c00 tx=0x7f35f40047c0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.736+0000 7f3613b43b80 1 -- 172.21.15.7:0/1338786645 shutdown_connections 2024-06-10T01:08:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.736+0000 7f3613b43b80 1 --2- 172.21.15.7:0/1338786645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb8bd58820 0x55eb8bd58c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.736+0000 7f3613b43b80 1 --2- 172.21.15.7:0/1338786645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb8bd5dfe0 0x55eb8bd5e3c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.736+0000 7f3613b43b80 1 --2- 172.21.15.7:0/1338786645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb8bd5e900 0x55eb8bd582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.736+0000 7f3613b43b80 1 -- 172.21.15.7:0/1338786645 >> 172.21.15.7:0/1338786645 conn(0x55eb8bc85100 msgr2=0x55eb8bd532a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.736+0000 7f3613b43b80 1 -- 172.21.15.7:0/1338786645 shutdown_connections 2024-06-10T01:08:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.736+0000 7f3613b43b80 1 -- 172.21.15.7:0/1338786645 wait complete. 2024-06-10T01:08:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.736+0000 7f3613b43b80 1 -- start start 2024-06-10T01:08:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.736+0000 7f3613b43b80 1 Processor -- start 2024-06-10T01:08:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.737+0000 7f3613b43b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb8bd58820 0x55eb8bd64690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.737+0000 7f3613b43b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb8bd5dfe0 0x55eb8bd61990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.737+0000 7f3613b43b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb8bd5e900 0x55eb8bd61ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.737+0000 7f3613b43b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55eb8bc4a040 con 0x55eb8bd5dfe0 2024-06-10T01:08:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.737+0000 7f3613b43b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55eb8bca5890 con 0x55eb8bd5e900 2024-06-10T01:08:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.737+0000 7f3613b43b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55eb8bd652e0 con 0x55eb8bd58820 2024-06-10T01:08:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.737+0000 7f3613b43b80 10 client.0 _ll_register_callbacks cb 0x55eb8bc3f630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.737+0000 7f3602ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb8bd58820 0x55eb8bd64690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.737+0000 7f36037fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb8bd5e900 0x55eb8bd61ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.737+0000 7f36027fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb8bd5dfe0 0x55eb8bd61990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.737+0000 7f3602ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb8bd58820 0x55eb8bd64690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47174/0 (socket says 172.21.15.7:47174) 2024-06-10T01:08:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.737+0000 7f3602ffd700 1 -- 172.21.15.7:0/1480860379 learned_addr learned my addr 172.21.15.7:0/1480860379 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.737+0000 7f3602ffd700 1 -- 172.21.15.7:0/1480860379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb8bd5e900 msgr2=0x55eb8bd61ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.737+0000 7f3602ffd700 1 --2- 172.21.15.7:0/1480860379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb8bd5e900 0x55eb8bd61ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.737+0000 7f3602ffd700 1 -- 172.21.15.7:0/1480860379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb8bd5dfe0 msgr2=0x55eb8bd61990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.737+0000 7f3602ffd700 1 --2- 172.21.15.7:0/1480860379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb8bd5dfe0 0x55eb8bd61990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.737+0000 7f3602ffd700 1 -- 172.21.15.7:0/1480860379 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb8bc6f7d0 con 0x55eb8bd58820 2024-06-10T01:08:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.737+0000 7f3602ffd700 1 --2- 172.21.15.7:0/1480860379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb8bd58820 0x55eb8bd64690 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f35f4098e70 tx=0x7f35f4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9478c442b7596835 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.737+0000 7f35ea7fc700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:08:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.737+0000 7f35ea7fc700 1 -- 172.21.15.7:0/1480860379 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f40cf370 con 0x55eb8bd58820 2024-06-10T01:08:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.737+0000 7f3613b43b80 1 -- 172.21.15.7:0/1480860379 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35f40a0dc0 con 0x55eb8bd58820 2024-06-10T01:08:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.737+0000 7f35ea7fc700 1 -- 172.21.15.7:0/1480860379 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f35f409be60 con 0x55eb8bd58820 2024-06-10T01:08:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.738+0000 7f35ea7fc700 1 -- 172.21.15.7:0/1480860379 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f35f409c060 con 0x55eb8bd58820 2024-06-10T01:08:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.738+0000 7f3613b43b80 10 client.25630 fetch_fsmap learned FSMap version 29 2024-06-10T01:08:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.738+0000 7f3613b43b80 1 -- 172.21.15.7:0/1480860379 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55eb8bd6fac0 con 0x55eb8bd58820 2024-06-10T01:08:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.738+0000 7f35ea7fc700 1 -- 172.21.15.7:0/1480860379 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35f40a0dc0 con 0x55eb8bd58820 2024-06-10T01:08:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.738+0000 7f35ea7fc700 1 -- 172.21.15.7:0/1480860379 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f35f40cf050 con 0x55eb8bd58820 2024-06-10T01:08:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.738+0000 7f3613b43b80 10 client.25630 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:08:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.738+0000 7f3613b43b80 10 client.25630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.738+0000 7f3613b43b80 20 client.25630 populate_metadata read hostname 'smithi007' 2024-06-10T01:08:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.738+0000 7f3613b43b80 1 --2- 172.21.15.7:0/1480860379 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55eb8bd71f90 0x55eb8bd74380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.738+0000 7f3613b43b80 4 client.25630 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:08:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.738+0000 7f3613b43b80 1 -- 172.21.15.7:0/1480860379 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb8bcb6dd0 con 0x55eb8bd71f90 2024-06-10T01:08:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.738+0000 7f36027fc700 1 --2- 172.21.15.7:0/1480860379 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55eb8bd71f90 0x55eb8bd74380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.739+0000 7f36027fc700 1 --2- 172.21.15.7:0/1480860379 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55eb8bd71f90 0x55eb8bd74380 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cb451bfb1f6579a server_cookie=653c9745c4edfaad in_seq=0 out_seq=0 2024-06-10T01:08:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.739+0000 7f35ea7fc700 10 client.25630 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.740+0000 7f35ea7fc700 1 -- 172.21.15.7:0/1480860379 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55eb8bcb6dd0 con 0x55eb8bd71f90 2024-06-10T01:08:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.740+0000 7f35ea7fc700 10 client.25630 handle_command_reply: tid=0 2024-06-10T01:08:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.740+0000 7f3613b43b80 1 client.25630 shutdown 2024-06-10T01:08:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.740+0000 7f3613b43b80 1 -- 172.21.15.7:0/1480860379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb8bd58820 msgr2=0x55eb8bd64690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.740+0000 7f3613b43b80 1 --2- 172.21.15.7:0/1480860379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb8bd58820 0x55eb8bd64690 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f35f4098e70 tx=0x7f35f4004750 comp rx=0 tx=0).stop 2024-06-10T01:08:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.740+0000 7f3613b43b80 1 -- 172.21.15.7:0/1480860379 shutdown_connections 2024-06-10T01:08:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.740+0000 7f3613b43b80 1 --2- 172.21.15.7:0/1480860379 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55eb8bd71f90 0x55eb8bd74380 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.740+0000 7f3613b43b80 1 --2- 172.21.15.7:0/1480860379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb8bd5e900 0x55eb8bd61ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.740+0000 7f3613b43b80 1 --2- 172.21.15.7:0/1480860379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb8bd58820 0x55eb8bd64690 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.740+0000 7f3613b43b80 1 --2- 172.21.15.7:0/1480860379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb8bd5dfe0 0x55eb8bd61990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.740+0000 7f35ea7fc700 0 client.25630 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.740+0000 7f3613b43b80 1 -- 172.21.15.7:0/1480860379 >> 172.21.15.7:0/1480860379 conn(0x55eb8bc85100 msgr2=0x55eb8bd52780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.740+0000 7f3613b43b80 1 -- 172.21.15.7:0/1480860379 shutdown_connections 2024-06-10T01:08:05.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.740+0000 7f3613b43b80 1 -- 172.21.15.7:0/1480860379 wait complete. 2024-06-10T01:08:05.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.741+0000 7f3613b43b80 20 client.25630 trim_cache size 0 max 16384 2024-06-10T01:08:05.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.762+0000 7f3613b43b80 1 Processor -- start 2024-06-10T01:08:05.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.762+0000 7f3613b43b80 1 -- start start 2024-06-10T01:08:05.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.763+0000 7f3613b43b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb8bc85100 0x55eb8bd5d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:05.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.763+0000 7f3613b43b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb8bd58820 0x55eb8bd52f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:05.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.763+0000 7f3613b43b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb8bd5dfe0 0x55eb8bd53440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:05.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.763+0000 7f3613b43b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f35e4011300 con 0x55eb8bd5dfe0 2024-06-10T01:08:05.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.763+0000 7f3613b43b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f35e400eb90 con 0x55eb8bc85100 2024-06-10T01:08:05.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.763+0000 7f3613b43b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f35e400c9e0 con 0x55eb8bd58820 2024-06-10T01:08:05.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.763+0000 7f36027fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb8bd58820 0x55eb8bd52f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:05.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.763+0000 7f3602ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb8bc85100 0x55eb8bd5d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:05.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.763+0000 7f36037fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb8bd5dfe0 0x55eb8bd53440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:05.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.763+0000 7f3602ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb8bc85100 0x55eb8bd5d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:41428/0 (socket says 172.21.15.7:41428) 2024-06-10T01:08:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.763+0000 7f36027fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb8bd58820 0x55eb8bd52f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47178/0 (socket says 172.21.15.7:47178) 2024-06-10T01:08:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.763+0000 7f3602ffd700 1 -- 172.21.15.7:0/2719835470 learned_addr learned my addr 172.21.15.7:0/2719835470 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.763+0000 7f36027fc700 1 -- 172.21.15.7:0/2719835470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb8bc85100 msgr2=0x55eb8bd5d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.763+0000 7f36027fc700 1 --2- 172.21.15.7:0/2719835470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb8bc85100 0x55eb8bd5d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.763+0000 7f36027fc700 1 -- 172.21.15.7:0/2719835470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb8bd5dfe0 msgr2=0x55eb8bd53440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.763+0000 7f36027fc700 1 --2- 172.21.15.7:0/2719835470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb8bd5dfe0 0x55eb8bd53440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.763+0000 7f36027fc700 1 -- 172.21.15.7:0/2719835470 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb8bd58620 con 0x55eb8bd58820 2024-06-10T01:08:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.763+0000 7f36037fe700 1 --2- 172.21.15.7:0/2719835470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb8bd5dfe0 0x55eb8bd53440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.764+0000 7f36027fc700 1 --2- 172.21.15.7:0/2719835470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb8bd58820 0x55eb8bd52f00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f35ec009990 tx=0x7f35ec023000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1771976be790b38 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.764+0000 7f35f8ff9700 1 -- 172.21.15.7:0/2719835470 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec005690 con 0x55eb8bd58820 2024-06-10T01:08:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.764+0000 7f35f8ff9700 1 -- 172.21.15.7:0/2719835470 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35ec00f940 con 0x55eb8bd58820 2024-06-10T01:08:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.764+0000 7f35f8ff9700 1 -- 172.21.15.7:0/2719835470 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec03b700 con 0x55eb8bd58820 2024-06-10T01:08:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.765+0000 7f3613b43b80 1 -- 172.21.15.7:0/2719835470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb8bd58820 msgr2=0x55eb8bd52f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:05.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.765+0000 7f3613b43b80 1 --2- 172.21.15.7:0/2719835470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb8bd58820 0x55eb8bd52f00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f35ec009990 tx=0x7f35ec023000 comp rx=0 tx=0).stop 2024-06-10T01:08:05.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.765+0000 7f3613b43b80 1 -- 172.21.15.7:0/2719835470 shutdown_connections 2024-06-10T01:08:05.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.765+0000 7f3613b43b80 1 --2- 172.21.15.7:0/2719835470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb8bd5dfe0 0x55eb8bd53440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.765+0000 7f3613b43b80 1 --2- 172.21.15.7:0/2719835470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb8bc85100 0x55eb8bd5d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.765+0000 7f3613b43b80 1 --2- 172.21.15.7:0/2719835470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb8bd58820 0x55eb8bd52f00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.765+0000 7f3613b43b80 1 -- 172.21.15.7:0/2719835470 >> 172.21.15.7:0/2719835470 conn(0x55eb8bd5c2f0 msgr2=0x55eb8bd5cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:05.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.765+0000 7f3613b43b80 1 -- 172.21.15.7:0/2719835470 shutdown_connections 2024-06-10T01:08:05.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.765+0000 7f3613b43b80 1 -- 172.21.15.7:0/2719835470 wait complete. 2024-06-10T01:08:05.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.765+0000 7f3613b43b80 1 -- start start 2024-06-10T01:08:05.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.765+0000 7f3613b43b80 1 Processor -- start 2024-06-10T01:08:05.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.765+0000 7f3613b43b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb8bc85100 0x55eb8bdd5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:05.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.765+0000 7f3613b43b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb8bd58820 0x55eb8bdd6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:05.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.765+0000 7f3613b43b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb8bd5dfe0 0x55eb8bdd48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.765+0000 7f3613b43b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f35e400d050 con 0x55eb8bd5dfe0 2024-06-10T01:08:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.766+0000 7f3613b43b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f35e400e4c0 con 0x55eb8bc85100 2024-06-10T01:08:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.766+0000 7f3613b43b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f35e400d720 con 0x55eb8bd58820 2024-06-10T01:08:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.766+0000 7f3613b43b80 10 client.0 _ll_register_callbacks cb 0x55eb8bd591b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.766+0000 7f36037fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb8bd5dfe0 0x55eb8bdd48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.766+0000 7f3602ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb8bc85100 0x55eb8bdd5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.766+0000 7f36027fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb8bd58820 0x55eb8bdd6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.766+0000 7f3602ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb8bc85100 0x55eb8bdd5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:41438/0 (socket says 172.21.15.7:41438) 2024-06-10T01:08:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.766+0000 7f3602ffd700 1 -- 172.21.15.7:0/724974115 learned_addr learned my addr 172.21.15.7:0/724974115 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.766+0000 7f36037fe700 1 -- 172.21.15.7:0/724974115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb8bd58820 msgr2=0x55eb8bdd6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.766+0000 7f36037fe700 1 --2- 172.21.15.7:0/724974115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb8bd58820 0x55eb8bdd6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.766+0000 7f36037fe700 1 -- 172.21.15.7:0/724974115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb8bc85100 msgr2=0x55eb8bdd5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:05.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.766+0000 7f36037fe700 1 --2- 172.21.15.7:0/724974115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb8bc85100 0x55eb8bdd5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.766+0000 7f36037fe700 1 -- 172.21.15.7:0/724974115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb8bdd4f10 con 0x55eb8bd5dfe0 2024-06-10T01:08:05.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.766+0000 7f3602ffd700 1 --2- 172.21.15.7:0/724974115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb8bc85100 0x55eb8bdd5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:05.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.766+0000 7f36037fe700 1 --2- 172.21.15.7:0/724974115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb8bd5dfe0 0x55eb8bdd48a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x55eb8bd65140 tx=0x7f35fc018980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d879b18459078c57 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:05.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.766+0000 7f35ea7fc700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:08:05.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.766+0000 7f35ea7fc700 1 -- 172.21.15.7:0/724974115 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc004bb0 con 0x55eb8bd5dfe0 2024-06-10T01:08:05.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.766+0000 7f35ea7fc700 1 -- 172.21.15.7:0/724974115 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f35fc004d50 con 0x55eb8bd5dfe0 2024-06-10T01:08:05.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.766+0000 7f3613b43b80 1 -- 172.21.15.7:0/724974115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35ec00f940 con 0x55eb8bd5dfe0 2024-06-10T01:08:05.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.767+0000 7f35ea7fc700 1 -- 172.21.15.7:0/724974115 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f35fc02b050 con 0x55eb8bd5dfe0 2024-06-10T01:08:05.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.767+0000 7f3613b43b80 10 client.16068 fetch_fsmap learned FSMap version 29 2024-06-10T01:08:05.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.767+0000 7f3613b43b80 1 -- 172.21.15.7:0/724974115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55eb8bddd4f0 con 0x55eb8bd5dfe0 2024-06-10T01:08:05.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.767+0000 7f35ea7fc700 1 -- 172.21.15.7:0/724974115 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35ec00f940 con 0x55eb8bd5dfe0 2024-06-10T01:08:05.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.767+0000 7f35ea7fc700 1 -- 172.21.15.7:0/724974115 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f35fc016630 con 0x55eb8bd5dfe0 2024-06-10T01:08:05.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.767+0000 7f3613b43b80 10 client.16068 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:08:05.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.767+0000 7f3613b43b80 10 client.16068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:05.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.767+0000 7f3613b43b80 20 client.16068 populate_metadata read hostname 'smithi007' 2024-06-10T01:08:05.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.767+0000 7f3613b43b80 1 --2- 172.21.15.7:0/724974115 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55eb8bd5e900 0x55eb8bddfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:05.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.767+0000 7f3613b43b80 4 client.16068 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:08:05.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.767+0000 7f3613b43b80 1 -- 172.21.15.7:0/724974115 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55eb8bcb1350 con 0x55eb8bd5e900 2024-06-10T01:08:05.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.767+0000 7f3602ffd700 1 --2- 172.21.15.7:0/724974115 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55eb8bd5e900 0x55eb8bddfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:05.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.768+0000 7f3602ffd700 1 --2- 172.21.15.7:0/724974115 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55eb8bd5e900 0x55eb8bddfc40 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71bdd6bd85c4c831 server_cookie=36f752d272e06407 in_seq=0 out_seq=0 2024-06-10T01:08:05.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.768+0000 7f35ea7fc700 10 client.16068 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:05.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.769+0000 7f35ea7fc700 1 -- 172.21.15.7:0/724974115 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55eb8bcb1350 con 0x55eb8bd5e900 2024-06-10T01:08:05.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.769+0000 7f35ea7fc700 10 client.16068 handle_command_reply: tid=0 2024-06-10T01:08:05.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.769+0000 7f3613b43b80 1 client.16068 shutdown 2024-06-10T01:08:05.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.769+0000 7f3613b43b80 1 -- 172.21.15.7:0/724974115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb8bd5dfe0 msgr2=0x55eb8bdd48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:05.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.769+0000 7f3613b43b80 1 --2- 172.21.15.7:0/724974115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb8bd5dfe0 0x55eb8bdd48a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x55eb8bd65140 tx=0x7f35fc018980 comp rx=0 tx=0).stop 2024-06-10T01:08:05.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.769+0000 7f3613b43b80 1 -- 172.21.15.7:0/724974115 shutdown_connections 2024-06-10T01:08:05.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.769+0000 7f3613b43b80 1 --2- 172.21.15.7:0/724974115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb8bd5dfe0 0x55eb8bdd48a0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.769+0000 7f3613b43b80 1 --2- 172.21.15.7:0/724974115 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55eb8bd5e900 0x55eb8bddfc40 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.769+0000 7f3613b43b80 1 --2- 172.21.15.7:0/724974115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb8bc85100 0x55eb8bdd5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.769+0000 7f3613b43b80 1 --2- 172.21.15.7:0/724974115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eb8bd58820 0x55eb8bdd6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.769+0000 7f3613b43b80 1 -- 172.21.15.7:0/724974115 >> 172.21.15.7:0/724974115 conn(0x55eb8bd5c2f0 msgr2=0x55eb8bd5d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:05.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.769+0000 7f35ea7fc700 0 client.16068 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:05.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.770+0000 7f3613b43b80 1 -- 172.21.15.7:0/724974115 shutdown_connections 2024-06-10T01:08:05.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.770+0000 7f3613b43b80 1 -- 172.21.15.7:0/724974115 wait complete. 2024-06-10T01:08:05.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.770+0000 7f3613b43b80 20 client.16068 trim_cache size 0 max 16384 2024-06-10T01:08:05.770 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:08:05.770 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-10T01:08:05.770 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "53d03cc4-f491-42ac-9575-68cbb7982f9c", 2024-06-10T01:08:05.770 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-10T01:08:05.771 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:08:05.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.770+0000 7f36092c0700 1 -- 172.21.15.7:0/2206296687 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35f00a2c20 msgr2=0x7f35f00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:05.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.770+0000 7f36092c0700 1 --2- 172.21.15.7:0/2206296687 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35f00a2c20 0x7f35f00a50e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f360410f7a0 tx=0x7f35ec005960 comp rx=0 tx=0).stop 2024-06-10T01:08:05.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.770+0000 7f36092c0700 1 -- 172.21.15.7:0/2206296687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36041027a0 msgr2=0x7f360419ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:05.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.770+0000 7f36092c0700 1 --2- 172.21.15.7:0/2206296687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36041027a0 0x7f360419ada0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f35f400e9f0 tx=0x7f35f400ed00 comp rx=0 tx=0).stop 2024-06-10T01:08:05.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.771+0000 7f36092c0700 1 -- 172.21.15.7:0/2206296687 shutdown_connections 2024-06-10T01:08:05.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.771+0000 7f36092c0700 1 --2- 172.21.15.7:0/2206296687 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35f00a2c20 0x7f35f00a50e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.771+0000 7f36092c0700 1 --2- 172.21.15.7:0/2206296687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3604107870 0x7f360419f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.771+0000 7f36092c0700 1 --2- 172.21.15.7:0/2206296687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36041027a0 0x7f360419ada0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.771+0000 7f36092c0700 1 --2- 172.21.15.7:0/2206296687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3604103150 0x7f360419b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:05.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.771+0000 7f36092c0700 1 -- 172.21.15.7:0/2206296687 >> 172.21.15.7:0/2206296687 conn(0x7f3604079570 msgr2=0x7f36041006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:05.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.771+0000 7f36092c0700 1 -- 172.21.15.7:0/2206296687 shutdown_connections 2024-06-10T01:08:05.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:05.771+0000 7f36092c0700 1 -- 172.21.15.7:0/2206296687 wait complete. 2024-06-10T01:08:06.595 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:08:06.710 INFO:tasks.workunit.client.0.smithi007.stdout: 1 49796 6.11 MB/sec warmup 47 sec latency 14.473 ms 2024-06-10T01:08:06.859 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:08:07.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:06 smithi175 ceph-mon[36636]: pgmap v495: 129 pgs: 129 active+clean; 29 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 746 KiB/s wr, 172 op/s 2024-06-10T01:08:07.204 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:06 smithi175 ceph-mon[36636]: scrub queued for path: / 2024-06-10T01:08:07.204 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:06 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:08:07.204 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:06 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:08:07.204 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:06 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:08:07.232 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:06 smithi007 ceph-mon[32398]: pgmap v495: 129 pgs: 129 active+clean; 29 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 746 KiB/s wr, 172 op/s 2024-06-10T01:08:07.232 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:06 smithi007 ceph-mon[32398]: scrub queued for path: / 2024-06-10T01:08:07.232 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:06 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:08:07.232 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:06 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:08:07.232 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:06 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:08:07.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:06 smithi078 ceph-mon[36683]: pgmap v495: 129 pgs: 129 active+clean; 29 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 746 KiB/s wr, 172 op/s 2024-06-10T01:08:07.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:06 smithi078 ceph-mon[36683]: scrub queued for path: / 2024-06-10T01:08:07.331 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:06 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:08:07.331 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:06 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:08:07.331 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:06 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:08:07.710 INFO:tasks.workunit.client.0.smithi007.stdout: 1 52266 6.18 MB/sec warmup 48 sec latency 18.470 ms 2024-06-10T01:08:08.084 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:07 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:08:08.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:07 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:08:08.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:07 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:08:08.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.627+0000 7f71dac68700 1 -- 172.21.15.7:0/955620576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71d4100a80 msgr2=0x7f71d4100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:08.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.627+0000 7f71dac68700 1 --2- 172.21.15.7:0/955620576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71d4100a80 0x7f71d4100f00 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f71c0009990 tx=0x7f71c0009ca0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.627+0000 7f71dac68700 1 -- 172.21.15.7:0/955620576 shutdown_connections 2024-06-10T01:08:08.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.627+0000 7f71dac68700 1 --2- 172.21.15.7:0/955620576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71d4101440 0x7f71d410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.627+0000 7f71dac68700 1 --2- 172.21.15.7:0/955620576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71d41086f0 0x7f71d4108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.627+0000 7f71dac68700 1 --2- 172.21.15.7:0/955620576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71d4100a80 0x7f71d4100f00 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.627+0000 7f71dac68700 1 -- 172.21.15.7:0/955620576 >> 172.21.15.7:0/955620576 conn(0x7f71d4077a00 msgr2=0x7f71d40fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.627+0000 7f71dac68700 1 -- 172.21.15.7:0/955620576 shutdown_connections 2024-06-10T01:08:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.627+0000 7f71dac68700 1 -- 172.21.15.7:0/955620576 wait complete. 2024-06-10T01:08:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.628+0000 7f71dac68700 1 Processor -- start 2024-06-10T01:08:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.628+0000 7f71dac68700 1 -- start start 2024-06-10T01:08:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.628+0000 7f71dac68700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71d4100a80 0x7f71d4101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.628+0000 7f71dac68700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71d4101440 0x7f71d41021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.628+0000 7f71dac68700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71d41086f0 0x7f71d41026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.628+0000 7f71dac68700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f71d41129b0 con 0x7f71d41086f0 2024-06-10T01:08:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.628+0000 7f71dac68700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71d4112830 con 0x7f71d4100a80 2024-06-10T01:08:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.628+0000 7f71dac68700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f71d4112b30 con 0x7f71d4101440 2024-06-10T01:08:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.628+0000 7f71d3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71d4101440 0x7f71d41021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.629+0000 7f71d3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71d4101440 0x7f71d41021a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47200/0 (socket says 172.21.15.7:47200) 2024-06-10T01:08:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.629+0000 7f71d3fff700 1 -- 172.21.15.7:0/1007337157 learned_addr learned my addr 172.21.15.7:0/1007337157 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.629+0000 7f71d8a04700 1 --2- 172.21.15.7:0/1007337157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71d4100a80 0x7f71d4101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.629+0000 7f71d9205700 1 --2- 172.21.15.7:0/1007337157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71d41086f0 0x7f71d41026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.629+0000 7f71d3fff700 1 -- 172.21.15.7:0/1007337157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71d4100a80 msgr2=0x7f71d4101c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.629+0000 7f71d3fff700 1 --2- 172.21.15.7:0/1007337157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71d4100a80 0x7f71d4101c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.629+0000 7f71d3fff700 1 -- 172.21.15.7:0/1007337157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71d41086f0 msgr2=0x7f71d41026e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.629+0000 7f71d3fff700 1 --2- 172.21.15.7:0/1007337157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71d41086f0 0x7f71d41026e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.629+0000 7f71d3fff700 1 -- 172.21.15.7:0/1007337157 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d41a72f0 con 0x7f71d4101440 2024-06-10T01:08:08.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.629+0000 7f71d3fff700 1 --2- 172.21.15.7:0/1007337157 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71d4101440 0x7f71d41021a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f71c0000c00 tx=0x7f71c000bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:08.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.629+0000 7f71d1ffb700 1 -- 172.21.15.7:0/1007337157 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c0027070 con 0x7f71d4101440 2024-06-10T01:08:08.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.629+0000 7f71d1ffb700 1 -- 172.21.15.7:0/1007337157 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71c0003d70 con 0x7f71d4101440 2024-06-10T01:08:08.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.629+0000 7f71d1ffb700 1 -- 172.21.15.7:0/1007337157 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c000f6c0 con 0x7f71d4101440 2024-06-10T01:08:08.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.630+0000 7f71dac68700 1 -- 172.21.15.7:0/1007337157 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d41a7580 con 0x7f71d4101440 2024-06-10T01:08:08.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.630+0000 7f71dac68700 1 -- 172.21.15.7:0/1007337157 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d41a7980 con 0x7f71d4101440 2024-06-10T01:08:08.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.633+0000 7f71d1ffb700 1 -- 172.21.15.7:0/1007337157 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f71c002a070 con 0x7f71d4101440 2024-06-10T01:08:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.633+0000 7f71d1ffb700 1 --2- 172.21.15.7:0/1007337157 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f71c40a2c20 0x7f71c40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.634+0000 7f71e54ebb80 1 Processor -- start 2024-06-10T01:08:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.634+0000 7f71d8a04700 1 --2- 172.21.15.7:0/1007337157 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f71c40a2c20 0x7f71c40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.634+0000 7f71d1ffb700 1 -- 172.21.15.7:0/1007337157 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f71c00d04a0 con 0x7f71d4101440 2024-06-10T01:08:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.634+0000 7f71e54ebb80 1 -- start start 2024-06-10T01:08:08.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.634+0000 7f71e54ebb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619c4652640 0x5619c4652a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.634+0000 7f71e54ebb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619c4652f60 0x5619c464c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.634+0000 7f71e54ebb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c464cf20 0x5619c464d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.634+0000 7f71e54ebb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5619c459e810 con 0x5619c4652f60 2024-06-10T01:08:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.634+0000 7f71e54ebb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5619c4546720 con 0x5619c464cf20 2024-06-10T01:08:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.634+0000 7f71e54ebb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5619c459a7d0 con 0x5619c4652640 2024-06-10T01:08:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.634+0000 7f71d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619c4652f60 0x5619c464c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.634+0000 7f71d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619c4652f60 0x5619c464c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55866/0 (socket says 172.21.15.7:55866) 2024-06-10T01:08:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.634+0000 7f71d8a04700 1 --2- 172.21.15.7:0/1007337157 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f71c40a2c20 0x7f71c40a50e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f71d4069990 tx=0x7f71c800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.634+0000 7f71d3fff700 1 -- 172.21.15.7:0/2663894326 learned_addr learned my addr 172.21.15.7:0/2663894326 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.634+0000 7f71d8a04700 1 --2- 172.21.15.7:0/2663894326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619c4652640 0x5619c4652a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.634+0000 7f71d9205700 1 --2- 172.21.15.7:0/2663894326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c464cf20 0x5619c464d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.634+0000 7f71d3fff700 1 -- 172.21.15.7:0/2663894326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619c4652640 msgr2=0x5619c4652a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.634+0000 7f71d3fff700 1 --2- 172.21.15.7:0/2663894326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619c4652640 0x5619c4652a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.634+0000 7f71d3fff700 1 -- 172.21.15.7:0/2663894326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c464cf20 msgr2=0x5619c464d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.634+0000 7f71d3fff700 1 --2- 172.21.15.7:0/2663894326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c464cf20 0x5619c464d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.634+0000 7f71d3fff700 1 -- 172.21.15.7:0/2663894326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619c453cf50 con 0x5619c4652f60 2024-06-10T01:08:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.635+0000 7f71d3fff700 1 --2- 172.21.15.7:0/2663894326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619c4652f60 0x5619c464c9e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f71c009c9a0 tx=0x7f71c0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df1a90ab15b9096a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.635+0000 7f71beffd700 1 -- 172.21.15.7:0/2663894326 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c009ca10 con 0x5619c4652f60 2024-06-10T01:08:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.635+0000 7f71beffd700 1 -- 172.21.15.7:0/2663894326 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71c00d0070 con 0x5619c4652f60 2024-06-10T01:08:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.635+0000 7f71beffd700 1 -- 172.21.15.7:0/2663894326 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c00a30c0 con 0x5619c4652f60 2024-06-10T01:08:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.635+0000 7f71e54ebb80 1 -- 172.21.15.7:0/2663894326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619c4652f60 msgr2=0x5619c464c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.635+0000 7f71e54ebb80 1 --2- 172.21.15.7:0/2663894326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619c4652f60 0x5619c464c9e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f71c009c9a0 tx=0x7f71c0009ca0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.636+0000 7f71e54ebb80 1 -- 172.21.15.7:0/2663894326 shutdown_connections 2024-06-10T01:08:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.636+0000 7f71e54ebb80 1 --2- 172.21.15.7:0/2663894326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c464cf20 0x5619c464d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.636+0000 7f71e54ebb80 1 --2- 172.21.15.7:0/2663894326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619c4652640 0x5619c4652a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.636+0000 7f71e54ebb80 1 --2- 172.21.15.7:0/2663894326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619c4652f60 0x5619c464c9e0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.636+0000 7f71e54ebb80 1 -- 172.21.15.7:0/2663894326 >> 172.21.15.7:0/2663894326 conn(0x5619c4577cc0 msgr2=0x5619c4647850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.636+0000 7f71e54ebb80 1 -- 172.21.15.7:0/2663894326 shutdown_connections 2024-06-10T01:08:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.636+0000 7f71e54ebb80 1 -- 172.21.15.7:0/2663894326 wait complete. 2024-06-10T01:08:08.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.636+0000 7f71e54ebb80 1 -- start start 2024-06-10T01:08:08.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.636+0000 7f71e54ebb80 1 Processor -- start 2024-06-10T01:08:08.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.637+0000 7f71e54ebb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619c464cf20 0x5619c4658d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.637+0000 7f71e54ebb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619c4652640 0x5619c4656000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.637+0000 7f71e54ebb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c4652f60 0x5619c4656540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.637+0000 7f71e54ebb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5619c459e810 con 0x5619c4652640 2024-06-10T01:08:08.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.637+0000 7f71e54ebb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5619c453ebf0 con 0x5619c4652f60 2024-06-10T01:08:08.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.637+0000 7f71e54ebb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5619c46596f0 con 0x5619c464cf20 2024-06-10T01:08:08.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.637+0000 7f71e54ebb80 10 client.0 _ll_register_callbacks cb 0x5619c4538920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:08.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.637+0000 7f71d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619c4652640 0x5619c4656000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.637+0000 7f71d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619c4652640 0x5619c4656000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55880/0 (socket says 172.21.15.7:55880) 2024-06-10T01:08:08.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.637+0000 7f71d3fff700 1 -- 172.21.15.7:0/2628380222 learned_addr learned my addr 172.21.15.7:0/2628380222 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:08.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.637+0000 7f71d8a04700 1 --2- 172.21.15.7:0/2628380222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619c464cf20 0x5619c4658d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.637+0000 7f71d3fff700 1 -- 172.21.15.7:0/2628380222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619c464cf20 msgr2=0x5619c4658d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:08.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.637+0000 7f71d3fff700 1 --2- 172.21.15.7:0/2628380222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619c464cf20 0x5619c4658d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.637+0000 7f71d3fff700 1 -- 172.21.15.7:0/2628380222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c4652f60 msgr2=0x5619c4656540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:08.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.637+0000 7f71d3fff700 1 --2- 172.21.15.7:0/2628380222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c4652f60 0x5619c4656540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.637+0000 7f71d3fff700 1 -- 172.21.15.7:0/2628380222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619c44ed8d0 con 0x5619c4652640 2024-06-10T01:08:08.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.637+0000 7f71d3fff700 1 --2- 172.21.15.7:0/2628380222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619c4652640 0x5619c4656000 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f71c000b820 tx=0x7f71c00a55f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef4191d3b6a70f7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:08.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.637+0000 7f71b3fff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:08:08.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.637+0000 7f71b3fff700 1 -- 172.21.15.7:0/2628380222 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c009ca10 con 0x5619c4652640 2024-06-10T01:08:08.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.637+0000 7f71b3fff700 1 -- 172.21.15.7:0/2628380222 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f71c00d0220 con 0x5619c4652640 2024-06-10T01:08:08.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.637+0000 7f71e54ebb80 1 -- 172.21.15.7:0/2628380222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71c00d0070 con 0x5619c4652640 2024-06-10T01:08:08.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.638+0000 7f71b3fff700 1 -- 172.21.15.7:0/2628380222 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f71c00bb120 con 0x5619c4652640 2024-06-10T01:08:08.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.638+0000 7f71e54ebb80 10 client.16080 fetch_fsmap learned FSMap version 29 2024-06-10T01:08:08.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.638+0000 7f71e54ebb80 1 -- 172.21.15.7:0/2628380222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5619c4663e20 con 0x5619c4652640 2024-06-10T01:08:08.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.638+0000 7f71b3fff700 1 -- 172.21.15.7:0/2628380222 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71c00d0070 con 0x5619c4652640 2024-06-10T01:08:08.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.638+0000 7f71b3fff700 1 -- 172.21.15.7:0/2628380222 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f71c00a3050 con 0x5619c4652640 2024-06-10T01:08:08.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.638+0000 7f71e54ebb80 10 client.16080 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:08:08.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.638+0000 7f71e54ebb80 10 client.16080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:08.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.638+0000 7f71e54ebb80 20 client.16080 populate_metadata read hostname 'smithi007' 2024-06-10T01:08:08.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.638+0000 7f71e54ebb80 1 --2- 172.21.15.7:0/2628380222 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5619c46663e0 0x5619c4668800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.638+0000 7f71e54ebb80 4 client.16080 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:08:08.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.638+0000 7f71e54ebb80 1 -- 172.21.15.7:0/2628380222 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619c45abd10 con 0x5619c46663e0 2024-06-10T01:08:08.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.638+0000 7f71d8a04700 1 --2- 172.21.15.7:0/2628380222 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5619c46663e0 0x5619c4668800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.639+0000 7f71d8a04700 1 --2- 172.21.15.7:0/2628380222 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5619c46663e0 0x5619c4668800 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84ab163465d35aa8 server_cookie=32042d371858ed80 in_seq=0 out_seq=0 2024-06-10T01:08:08.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.639+0000 7f71b3fff700 10 client.16080 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:08.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.639+0000 7f71b3fff700 1 -- 172.21.15.7:0/2628380222 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5619c45abd10 con 0x5619c46663e0 2024-06-10T01:08:08.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.639+0000 7f71b3fff700 10 client.16080 handle_command_reply: tid=0 2024-06-10T01:08:08.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.639+0000 7f71e54ebb80 1 client.16080 shutdown 2024-06-10T01:08:08.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.640+0000 7f71e54ebb80 1 -- 172.21.15.7:0/2628380222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619c4652640 msgr2=0x5619c4656000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:08.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.640+0000 7f71e54ebb80 1 --2- 172.21.15.7:0/2628380222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619c4652640 0x5619c4656000 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f71c000b820 tx=0x7f71c00a55f0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.640+0000 7f71e54ebb80 1 -- 172.21.15.7:0/2628380222 shutdown_connections 2024-06-10T01:08:08.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.640+0000 7f71e54ebb80 1 --2- 172.21.15.7:0/2628380222 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5619c46663e0 0x5619c4668800 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.640+0000 7f71e54ebb80 1 --2- 172.21.15.7:0/2628380222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c4652f60 0x5619c4656540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.640+0000 7f71e54ebb80 1 --2- 172.21.15.7:0/2628380222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619c464cf20 0x5619c4658d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.640+0000 7f71e54ebb80 1 --2- 172.21.15.7:0/2628380222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619c4652640 0x5619c4656000 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.640+0000 7f71b3fff700 0 client.16080 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:08.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.640+0000 7f71e54ebb80 1 -- 172.21.15.7:0/2628380222 >> 172.21.15.7:0/2628380222 conn(0x5619c4577cc0 msgr2=0x5619c46475f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:08.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.640+0000 7f71e54ebb80 1 -- 172.21.15.7:0/2628380222 shutdown_connections 2024-06-10T01:08:08.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.640+0000 7f71e54ebb80 1 -- 172.21.15.7:0/2628380222 wait complete. 2024-06-10T01:08:08.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.640+0000 7f71e54ebb80 20 client.16080 trim_cache size 0 max 16384 2024-06-10T01:08:08.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.661+0000 7f71e54ebb80 1 Processor -- start 2024-06-10T01:08:08.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.661+0000 7f71e54ebb80 1 -- start start 2024-06-10T01:08:08.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.662+0000 7f71e54ebb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619c464cf20 0x5619c464c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.662+0000 7f71e54ebb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619c4652640 0x5619c4650740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.662+0000 7f71e54ebb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c4652f60 0x5619c466e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.662+0000 7f71e54ebb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f71d41b2ca0 con 0x5619c4652640 2024-06-10T01:08:08.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.662+0000 7f71e54ebb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71d41b0530 con 0x5619c4652f60 2024-06-10T01:08:08.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.662+0000 7f71e54ebb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f71d41ae380 con 0x5619c464cf20 2024-06-10T01:08:08.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.662+0000 7f71d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619c4652640 0x5619c4650740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.662+0000 7f71d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619c4652640 0x5619c4650740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55882/0 (socket says 172.21.15.7:55882) 2024-06-10T01:08:08.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.662+0000 7f71d3fff700 1 -- 172.21.15.7:0/3066819455 learned_addr learned my addr 172.21.15.7:0/3066819455 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:08.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.662+0000 7f71d8a04700 1 --2- 172.21.15.7:0/3066819455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619c464cf20 0x5619c464c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.662+0000 7f71d9205700 1 --2- 172.21.15.7:0/3066819455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c4652f60 0x5619c466e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.662+0000 7f71d3fff700 1 -- 172.21.15.7:0/3066819455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619c464cf20 msgr2=0x5619c464c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:08.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.662+0000 7f71d3fff700 1 --2- 172.21.15.7:0/3066819455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619c464cf20 0x5619c464c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.662+0000 7f71d3fff700 1 -- 172.21.15.7:0/3066819455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c4652f60 msgr2=0x5619c466e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:08.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.662+0000 7f71d3fff700 1 --2- 172.21.15.7:0/3066819455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c4652f60 0x5619c466e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.662+0000 7f71d3fff700 1 -- 172.21.15.7:0/3066819455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619c4654570 con 0x5619c4652640 2024-06-10T01:08:08.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.662+0000 7f71d9205700 1 --2- 172.21.15.7:0/3066819455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c4652f60 0x5619c466e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:08.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.662+0000 7f71d3fff700 1 --2- 172.21.15.7:0/3066819455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619c4652640 0x5619c4650740 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f71c00993e0 tx=0x7f71c0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181ba61a9dfded8a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:08.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.663+0000 7f71be7fc700 1 -- 172.21.15.7:0/3066819455 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c009c130 con 0x5619c4652640 2024-06-10T01:08:08.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.663+0000 7f71be7fc700 1 -- 172.21.15.7:0/3066819455 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71c00d0070 con 0x5619c4652640 2024-06-10T01:08:08.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.663+0000 7f71be7fc700 1 -- 172.21.15.7:0/3066819455 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c009cc70 con 0x5619c4652640 2024-06-10T01:08:08.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.663+0000 7f71e54ebb80 1 -- 172.21.15.7:0/3066819455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619c4652640 msgr2=0x5619c4650740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:08.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.663+0000 7f71e54ebb80 1 --2- 172.21.15.7:0/3066819455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619c4652640 0x5619c4650740 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f71c00993e0 tx=0x7f71c0009310 comp rx=0 tx=0).stop 2024-06-10T01:08:08.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.663+0000 7f71e54ebb80 1 -- 172.21.15.7:0/3066819455 shutdown_connections 2024-06-10T01:08:08.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.663+0000 7f71e54ebb80 1 --2- 172.21.15.7:0/3066819455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c4652f60 0x5619c466e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.663+0000 7f71e54ebb80 1 --2- 172.21.15.7:0/3066819455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619c464cf20 0x5619c464c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.663+0000 7f71e54ebb80 1 --2- 172.21.15.7:0/3066819455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619c4652640 0x5619c4650740 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.663+0000 7f71e54ebb80 1 -- 172.21.15.7:0/3066819455 >> 172.21.15.7:0/3066819455 conn(0x5619c4577cc0 msgr2=0x5619c4653a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:08.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.663+0000 7f71e54ebb80 1 -- 172.21.15.7:0/3066819455 shutdown_connections 2024-06-10T01:08:08.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.663+0000 7f71e54ebb80 1 -- 172.21.15.7:0/3066819455 wait complete. 2024-06-10T01:08:08.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.664+0000 7f71e54ebb80 1 -- start start 2024-06-10T01:08:08.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.664+0000 7f71e54ebb80 1 Processor -- start 2024-06-10T01:08:08.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.665+0000 7f71e54ebb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c464cf20 0x5619c466dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.665+0000 7f71e54ebb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619c4652640 0x5619c466e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.665+0000 7f71e54ebb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619c4652f60 0x5619c466e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.665+0000 7f71e54ebb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f71d41ae9f0 con 0x5619c4652f60 2024-06-10T01:08:08.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.665+0000 7f71e54ebb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71d41afe60 con 0x5619c464cf20 2024-06-10T01:08:08.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.665+0000 7f71e54ebb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f71d41af0c0 con 0x5619c4652640 2024-06-10T01:08:08.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.665+0000 7f71e54ebb80 10 client.0 _ll_register_callbacks cb 0x5619c4538920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:08.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.665+0000 7f71d9205700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619c4652f60 0x5619c466e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.665+0000 7f71d8a04700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c464cf20 0x5619c466dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.665+0000 7f71d8a04700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c464cf20 0x5619c466dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:41488/0 (socket says 172.21.15.7:41488) 2024-06-10T01:08:08.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.665+0000 7f71d3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619c4652640 0x5619c466e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.665+0000 7f71d9205700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619c4652f60 0x5619c466e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55898/0 (socket says 172.21.15.7:55898) 2024-06-10T01:08:08.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.665+0000 7f71d8a04700 1 -- 172.21.15.7:0/305920310 learned_addr learned my addr 172.21.15.7:0/305920310 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:08.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.665+0000 7f71d8a04700 1 -- 172.21.15.7:0/305920310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619c4652640 msgr2=0x5619c466e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:08.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.665+0000 7f71d8a04700 1 --2- 172.21.15.7:0/305920310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619c4652640 0x5619c466e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.665+0000 7f71d8a04700 1 -- 172.21.15.7:0/305920310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619c4652f60 msgr2=0x5619c466e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:08.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.665+0000 7f71d8a04700 1 --2- 172.21.15.7:0/305920310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619c4652f60 0x5619c466e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.665+0000 7f71d8a04700 1 -- 172.21.15.7:0/305920310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619c46d0060 con 0x5619c464cf20 2024-06-10T01:08:08.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.665+0000 7f71d9205700 1 --2- 172.21.15.7:0/305920310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619c4652f60 0x5619c466e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:08.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.665+0000 7f71d8a04700 1 --2- 172.21.15.7:0/305920310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c464cf20 0x5619c466dba0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f71c80059d0 tx=0x7f71c80047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37a881bb51192f17 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:08.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.665+0000 7f71b3fff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:08:08.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.665+0000 7f71b3fff700 1 -- 172.21.15.7:0/305920310 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8036060 con 0x5619c464cf20 2024-06-10T01:08:08.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.666+0000 7f71e54ebb80 1 -- 172.21.15.7:0/305920310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71c00d0070 con 0x5619c464cf20 2024-06-10T01:08:08.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.666+0000 7f71b3fff700 1 -- 172.21.15.7:0/305920310 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f71c8031460 con 0x5619c464cf20 2024-06-10T01:08:08.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.666+0000 7f71b3fff700 1 -- 172.21.15.7:0/305920310 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f71c803d050 con 0x5619c464cf20 2024-06-10T01:08:08.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.666+0000 7f71e54ebb80 10 client.25619 fetch_fsmap learned FSMap version 29 2024-06-10T01:08:08.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.666+0000 7f71e54ebb80 1 -- 172.21.15.7:0/305920310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5619c46d12d0 con 0x5619c464cf20 2024-06-10T01:08:08.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.666+0000 7f71b3fff700 1 -- 172.21.15.7:0/305920310 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71c00d0070 con 0x5619c464cf20 2024-06-10T01:08:08.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.666+0000 7f71b3fff700 1 -- 172.21.15.7:0/305920310 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f71c801c7e0 con 0x5619c464cf20 2024-06-10T01:08:08.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.666+0000 7f71e54ebb80 10 client.25619 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:08:08.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.666+0000 7f71e54ebb80 10 client.25619 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:08.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.666+0000 7f71e54ebb80 20 client.25619 populate_metadata read hostname 'smithi007' 2024-06-10T01:08:08.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.666+0000 7f71e54ebb80 1 --2- 172.21.15.7:0/305920310 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5619c46d1930 0x5619c46d3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.666+0000 7f71e54ebb80 4 client.25619 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:08:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.666+0000 7f71e54ebb80 1 -- 172.21.15.7:0/305920310 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5619c45a6290 con 0x5619c46d1930 2024-06-10T01:08:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.666+0000 7f71d3fff700 1 --2- 172.21.15.7:0/305920310 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5619c46d1930 0x5619c46d3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.667+0000 7f71b3fff700 10 client.25619 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.667+0000 7f71d3fff700 1 --2- 172.21.15.7:0/305920310 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5619c46d1930 0x5619c46d3d20 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ef619eca2688964 server_cookie=acbedbe019b1ed22 in_seq=0 out_seq=0 2024-06-10T01:08:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.668+0000 7f71b3fff700 1 -- 172.21.15.7:0/305920310 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5619c45a6290 con 0x5619c46d1930 2024-06-10T01:08:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.668+0000 7f71b3fff700 10 client.25619 handle_command_reply: tid=0 2024-06-10T01:08:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.668+0000 7f71e54ebb80 1 client.25619 shutdown 2024-06-10T01:08:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.668+0000 7f71e54ebb80 1 -- 172.21.15.7:0/305920310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c464cf20 msgr2=0x5619c466dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.668+0000 7f71e54ebb80 1 --2- 172.21.15.7:0/305920310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c464cf20 0x5619c466dba0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f71c80059d0 tx=0x7f71c80047c0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.668+0000 7f71e54ebb80 1 -- 172.21.15.7:0/305920310 shutdown_connections 2024-06-10T01:08:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.668+0000 7f71e54ebb80 1 --2- 172.21.15.7:0/305920310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619c4652f60 0x5619c466e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.668+0000 7f71e54ebb80 1 --2- 172.21.15.7:0/305920310 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5619c46d1930 0x5619c46d3d20 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.668+0000 7f71e54ebb80 1 --2- 172.21.15.7:0/305920310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c464cf20 0x5619c466dba0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.668+0000 7f71e54ebb80 1 --2- 172.21.15.7:0/305920310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619c4652640 0x5619c466e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.668+0000 7f71b3fff700 0 client.25619 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:08.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.668+0000 7f71e54ebb80 1 -- 172.21.15.7:0/305920310 >> 172.21.15.7:0/305920310 conn(0x5619c4577cc0 msgr2=0x5619c4640e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:08.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.668+0000 7f71e54ebb80 1 -- 172.21.15.7:0/305920310 shutdown_connections 2024-06-10T01:08:08.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.668+0000 7f71e54ebb80 1 -- 172.21.15.7:0/305920310 wait complete. 2024-06-10T01:08:08.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.668+0000 7f71e54ebb80 20 client.25619 trim_cache size 0 max 16384 2024-06-10T01:08:08.669 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:08:08.669 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-10T01:08:08.669 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-10T01:08:08.670 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:08:08.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.669+0000 7f71dac68700 1 -- 172.21.15.7:0/1007337157 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f71c40a2c20 msgr2=0x7f71c40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:08.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.669+0000 7f71dac68700 1 --2- 172.21.15.7:0/1007337157 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f71c40a2c20 0x7f71c40a50e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f71d4069990 tx=0x7f71c800a380 comp rx=0 tx=0).stop 2024-06-10T01:08:08.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.669+0000 7f71dac68700 1 -- 172.21.15.7:0/1007337157 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71d4101440 msgr2=0x7f71d41021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:08.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.669+0000 7f71dac68700 1 --2- 172.21.15.7:0/1007337157 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71d4101440 0x7f71d41021a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f71c0000c00 tx=0x7f71c000bab0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.669+0000 7f71dac68700 1 -- 172.21.15.7:0/1007337157 shutdown_connections 2024-06-10T01:08:08.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.669+0000 7f71dac68700 1 --2- 172.21.15.7:0/1007337157 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f71c40a2c20 0x7f71c40a50e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.669+0000 7f71dac68700 1 --2- 172.21.15.7:0/1007337157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71d41086f0 0x7f71d41026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.669+0000 7f71dac68700 1 --2- 172.21.15.7:0/1007337157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71d4100a80 0x7f71d4101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.669+0000 7f71dac68700 1 --2- 172.21.15.7:0/1007337157 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71d4101440 0x7f71d41021a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.669+0000 7f71dac68700 1 -- 172.21.15.7:0/1007337157 >> 172.21.15.7:0/1007337157 conn(0x7f71d4077a00 msgr2=0x7f71d40ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:08.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.669+0000 7f71dac68700 1 -- 172.21.15.7:0/1007337157 shutdown_connections 2024-06-10T01:08:08.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:08.669+0000 7f71dac68700 1 -- 172.21.15.7:0/1007337157 wait complete. 2024-06-10T01:08:08.710 INFO:tasks.workunit.client.0.smithi007.stdout: 1 54621 6.38 MB/sec warmup 49 sec latency 19.145 ms 2024-06-10T01:08:09.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:08 smithi175 ceph-mon[36636]: pgmap v496: 129 pgs: 129 active+clean; 30 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 844 KiB/s wr, 194 op/s 2024-06-10T01:08:09.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:08 smithi007 ceph-mon[32398]: pgmap v496: 129 pgs: 129 active+clean; 30 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 844 KiB/s wr, 194 op/s 2024-06-10T01:08:09.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:08 smithi078 ceph-mon[36683]: pgmap v496: 129 pgs: 129 active+clean; 30 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 844 KiB/s wr, 194 op/s 2024-06-10T01:08:09.496 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:08:09.496 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:08:09.710 INFO:tasks.workunit.client.0.smithi007.stdout: 1 57447 6.54 MB/sec warmup 50 sec latency 13.894 ms 2024-06-10T01:08:09.758 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:08:10.710 INFO:tasks.workunit.client.0.smithi007.stdout: 1 60619 6.78 MB/sec warmup 51 sec latency 15.614 ms 2024-06-10T01:08:11.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:10 smithi175 ceph-mon[36636]: pgmap v497: 129 pgs: 129 active+clean; 31 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.0 MiB/s wr, 215 op/s 2024-06-10T01:08:11.236 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:10 smithi007 ceph-mon[32398]: pgmap v497: 129 pgs: 129 active+clean; 31 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.0 MiB/s wr, 215 op/s 2024-06-10T01:08:11.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:10 smithi078 ceph-mon[36683]: pgmap v497: 129 pgs: 129 active+clean; 31 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.0 MiB/s wr, 215 op/s 2024-06-10T01:08:11.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.474+0000 7f9f57976700 1 -- 172.21.15.7:0/2564285845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f50101d50 msgr2=0x7f9f5010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:11.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.474+0000 7f9f57976700 1 --2- 172.21.15.7:0/2564285845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f50101d50 0x7f9f5010e240 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800b0a0 tx=0x7f9f4800b3b0 comp rx=0 tx=0).stop 2024-06-10T01:08:11.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.474+0000 7f9f57976700 1 -- 172.21.15.7:0/2564285845 shutdown_connections 2024-06-10T01:08:11.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.474+0000 7f9f57976700 1 --2- 172.21.15.7:0/2564285845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f50101d50 0x7f9f5010e240 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:11.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.474+0000 7f9f57976700 1 --2- 172.21.15.7:0/2564285845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f50107400 0x7f9f501077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:11.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.474+0000 7f9f57976700 1 --2- 172.21.15.7:0/2564285845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f50101390 0x7f9f50101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:11.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.474+0000 7f9f57976700 1 -- 172.21.15.7:0/2564285845 >> 172.21.15.7:0/2564285845 conn(0x7f9f50078340 msgr2=0x7f9f50078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:11.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.474+0000 7f9f57976700 1 -- 172.21.15.7:0/2564285845 shutdown_connections 2024-06-10T01:08:11.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.474+0000 7f9f57976700 1 -- 172.21.15.7:0/2564285845 wait complete. 2024-06-10T01:08:11.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.475+0000 7f9f57976700 1 Processor -- start 2024-06-10T01:08:11.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.475+0000 7f9f57976700 1 -- start start 2024-06-10T01:08:11.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.475+0000 7f9f57976700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f50101390 0x7f9f50103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:11.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.475+0000 7f9f57976700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f50101d50 0x7f9f501035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:11.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.475+0000 7f9f57976700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f50107400 0x7f9f50103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:11.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.475+0000 7f9f57976700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9f50111690 con 0x7f9f50101390 2024-06-10T01:08:11.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.475+0000 7f9f57976700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9f50111510 con 0x7f9f50107400 2024-06-10T01:08:11.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.475+0000 7f9f57976700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9f50111810 con 0x7f9f50101d50 2024-06-10T01:08:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.475+0000 7f9f54f11700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f50101d50 0x7f9f501035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.476+0000 7f9f54f11700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f50101d50 0x7f9f501035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47252/0 (socket says 172.21.15.7:47252) 2024-06-10T01:08:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.476+0000 7f9f54f11700 1 -- 172.21.15.7:0/2242239581 learned_addr learned my addr 172.21.15.7:0/2242239581 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.476+0000 7f9f55712700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f50101390 0x7f9f50103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.476+0000 7f9f55f13700 1 --2- 172.21.15.7:0/2242239581 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f50107400 0x7f9f50103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.476+0000 7f9f54f11700 1 -- 172.21.15.7:0/2242239581 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f50107400 msgr2=0x7f9f50103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.476+0000 7f9f54f11700 1 --2- 172.21.15.7:0/2242239581 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f50107400 0x7f9f50103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.476+0000 7f9f54f11700 1 -- 172.21.15.7:0/2242239581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f50101390 msgr2=0x7f9f50103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.476+0000 7f9f54f11700 1 --2- 172.21.15.7:0/2242239581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f50101390 0x7f9f50103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.476+0000 7f9f54f11700 1 -- 172.21.15.7:0/2242239581 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f501a70e0 con 0x7f9f50101d50 2024-06-10T01:08:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.476+0000 7f9f55f13700 1 --2- 172.21.15.7:0/2242239581 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f50107400 0x7f9f50103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:08:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.476+0000 7f9f54f11700 1 --2- 172.21.15.7:0/2242239581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f50101d50 0x7f9f501035b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000e9a0 tx=0x7f9f4000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.476+0000 7f9f467fc700 1 -- 172.21.15.7:0/2242239581 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4000cbd0 con 0x7f9f50101d50 2024-06-10T01:08:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.476+0000 7f9f467fc700 1 -- 172.21.15.7:0/2242239581 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f4000cd70 con 0x7f9f50101d50 2024-06-10T01:08:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.476+0000 7f9f57976700 1 -- 172.21.15.7:0/2242239581 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f501a73d0 con 0x7f9f50101d50 2024-06-10T01:08:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.476+0000 7f9f467fc700 1 -- 172.21.15.7:0/2242239581 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f40018750 con 0x7f9f50101d50 2024-06-10T01:08:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.476+0000 7f9f57976700 1 -- 172.21.15.7:0/2242239581 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f501a7910 con 0x7f9f50101d50 2024-06-10T01:08:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.477+0000 7f9f57976700 1 -- 172.21.15.7:0/2242239581 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f50069180 con 0x7f9f50101d50 2024-06-10T01:08:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.479+0000 7f9f467fc700 1 -- 172.21.15.7:0/2242239581 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9f40010430 con 0x7f9f50101d50 2024-06-10T01:08:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.479+0000 7f9f467fc700 1 --2- 172.21.15.7:0/2242239581 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9f3c0a2c20 0x7f9f3c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.479+0000 7f9f55712700 1 --2- 172.21.15.7:0/2242239581 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9f3c0a2c20 0x7f9f3c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.479+0000 7f9f467fc700 1 -- 172.21.15.7:0/2242239581 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f9f400cfba0 con 0x7f9f50101d50 2024-06-10T01:08:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.480+0000 7f9f55712700 1 --2- 172.21.15.7:0/2242239581 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9f3c0a2c20 0x7f9f3c0a50e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c009be0 tx=0x7f9f4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.483+0000 7f9f467fc700 1 -- 172.21.15.7:0/2242239581 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9f4009c720 con 0x7f9f50101d50 2024-06-10T01:08:11.710 INFO:tasks.workunit.client.0.smithi007.stdout: 1 63515 6.91 MB/sec warmup 52 sec latency 19.473 ms 2024-06-10T01:08:11.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.781+0000 7f9f57976700 1 -- 172.21.15.7:0/2242239581 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f501a7b70 con 0x7f9f50101d50 2024-06-10T01:08:11.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.782+0000 7f9f467fc700 1 -- 172.21.15.7:0/2242239581 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11855 (secure 0 0 0) 0x7f9f400bc0e0 con 0x7f9f50101d50 2024-06-10T01:08:11.779 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:08:11.779 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:07:36.881518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:08:11.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.784+0000 7f9f57976700 1 -- 172.21.15.7:0/2242239581 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9f3c0a2c20 msgr2=0x7f9f3c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:11.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.784+0000 7f9f57976700 1 --2- 172.21.15.7:0/2242239581 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9f3c0a2c20 0x7f9f3c0a50e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c009be0 tx=0x7f9f4c009260 comp rx=0 tx=0).stop 2024-06-10T01:08:11.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.784+0000 7f9f57976700 1 -- 172.21.15.7:0/2242239581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f50101d50 msgr2=0x7f9f501035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:11.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.784+0000 7f9f57976700 1 --2- 172.21.15.7:0/2242239581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f50101d50 0x7f9f501035b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000e9a0 tx=0x7f9f4000ecb0 comp rx=0 tx=0).stop 2024-06-10T01:08:11.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.784+0000 7f9f57976700 1 -- 172.21.15.7:0/2242239581 shutdown_connections 2024-06-10T01:08:11.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.784+0000 7f9f57976700 1 --2- 172.21.15.7:0/2242239581 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9f3c0a2c20 0x7f9f3c0a50e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:11.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.784+0000 7f9f57976700 1 --2- 172.21.15.7:0/2242239581 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f50107400 0x7f9f50103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:11.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.784+0000 7f9f57976700 1 --2- 172.21.15.7:0/2242239581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f50101390 0x7f9f50103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:11.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.784+0000 7f9f57976700 1 --2- 172.21.15.7:0/2242239581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f50101d50 0x7f9f501035b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:11.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.784+0000 7f9f57976700 1 -- 172.21.15.7:0/2242239581 >> 172.21.15.7:0/2242239581 conn(0x7f9f50078340 msgr2=0x7f9f501005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:11.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.784+0000 7f9f57976700 1 -- 172.21.15.7:0/2242239581 shutdown_connections 2024-06-10T01:08:11.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:11.784+0000 7f9f57976700 1 -- 172.21.15.7:0/2242239581 wait complete. 2024-06-10T01:08:11.782 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 29 2024-06-10T01:08:12.038 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:11 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2242239581' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:08:12.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:11 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2242239581' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:08:12.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:11 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2242239581' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:08:12.657 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:08:12.710 INFO:tasks.workunit.client.0.smithi007.stdout: 1 65401 6.99 MB/sec warmup 53 sec latency 79.267 ms 2024-06-10T01:08:12.931 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:08:13.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:12 smithi007 ceph-mon[32398]: pgmap v498: 129 pgs: 129 active+clean; 36 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.5 MiB/s wr, 290 op/s 2024-06-10T01:08:13.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:12 smithi078 ceph-mon[36683]: pgmap v498: 129 pgs: 129 active+clean; 36 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.5 MiB/s wr, 290 op/s 2024-06-10T01:08:13.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:12 smithi175 ceph-mon[36636]: pgmap v498: 129 pgs: 129 active+clean; 36 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.5 MiB/s wr, 290 op/s 2024-06-10T01:08:13.710 INFO:tasks.workunit.client.0.smithi007.stdout: 1 68333 7.13 MB/sec warmup 54 sec latency 14.252 ms 2024-06-10T01:08:14.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.596+0000 7fe914473700 1 -- 172.21.15.7:0/2842845239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe90c1084f0 msgr2=0x7fe90c1088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:14.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.596+0000 7fe914473700 1 --2- 172.21.15.7:0/2842845239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe90c1084f0 0x7fe90c1088d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe908009990 tx=0x7fe908009ca0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.596+0000 7fe914473700 1 -- 172.21.15.7:0/2842845239 shutdown_connections 2024-06-10T01:08:14.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.596+0000 7fe914473700 1 --2- 172.21.15.7:0/2842845239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe90c102f10 0x7fe90c10f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.596+0000 7fe914473700 1 --2- 172.21.15.7:0/2842845239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe90c1084f0 0x7fe90c1088d0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.596+0000 7fe914473700 1 --2- 172.21.15.7:0/2842845239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe90c102550 0x7fe90c1029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.596+0000 7fe914473700 1 -- 172.21.15.7:0/2842845239 >> 172.21.15.7:0/2842845239 conn(0x7fe90c079650 msgr2=0x7fe90c079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:14.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.596+0000 7fe914473700 1 -- 172.21.15.7:0/2842845239 shutdown_connections 2024-06-10T01:08:14.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.596+0000 7fe914473700 1 -- 172.21.15.7:0/2842845239 wait complete. 2024-06-10T01:08:14.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.597+0000 7fe914473700 1 Processor -- start 2024-06-10T01:08:14.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.597+0000 7fe914473700 1 -- start start 2024-06-10T01:08:14.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.597+0000 7fe914473700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe90c102550 0x7fe90c19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:14.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.597+0000 7fe914473700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe90c102f10 0x7fe90c19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:14.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.597+0000 7fe914473700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe90c1084f0 0x7fe90c198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:14.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.597+0000 7fe914473700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe90c112750 con 0x7fe90c102f10 2024-06-10T01:08:14.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.597+0000 7fe914473700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe90c1125d0 con 0x7fe90c1084f0 2024-06-10T01:08:14.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.597+0000 7fe914473700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe90c1128d0 con 0x7fe90c102550 2024-06-10T01:08:14.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.597+0000 7fe91220f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe90c102550 0x7fe90c19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:14.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.597+0000 7fe911a0e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe90c102f10 0x7fe90c19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:14.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.597+0000 7fe91220f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe90c102550 0x7fe90c19e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:55282/0 (socket says 172.21.15.7:55282) 2024-06-10T01:08:14.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.597+0000 7fe91220f700 1 -- 172.21.15.7:0/1070295700 learned_addr learned my addr 172.21.15.7:0/1070295700 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:14.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.597+0000 7fe912a10700 1 --2- 172.21.15.7:0/1070295700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe90c1084f0 0x7fe90c198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:14.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.597+0000 7fe91220f700 1 -- 172.21.15.7:0/1070295700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe90c1084f0 msgr2=0x7fe90c198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:14.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.598+0000 7fe91220f700 1 --2- 172.21.15.7:0/1070295700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe90c1084f0 0x7fe90c198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.598+0000 7fe91220f700 1 -- 172.21.15.7:0/1070295700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe90c102f10 msgr2=0x7fe90c19edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:14.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.598+0000 7fe91220f700 1 --2- 172.21.15.7:0/1070295700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe90c102f10 0x7fe90c19edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.598+0000 7fe91220f700 1 -- 172.21.15.7:0/1070295700 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe90c199190 con 0x7fe90c102550 2024-06-10T01:08:14.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.598+0000 7fe912a10700 1 --2- 172.21.15.7:0/1070295700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe90c1084f0 0x7fe90c198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:14.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.598+0000 7fe911a0e700 1 --2- 172.21.15.7:0/1070295700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe90c102f10 0x7fe90c19edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:14.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.598+0000 7fe91220f700 1 --2- 172.21.15.7:0/1070295700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe90c102550 0x7fe90c19e870 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe908000c00 tx=0x7fe90800bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:14.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.598+0000 7fe8ff7fe700 1 -- 172.21.15.7:0/1070295700 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe908027070 con 0x7fe90c102550 2024-06-10T01:08:14.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.598+0000 7fe914473700 1 -- 172.21.15.7:0/1070295700 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe90c199420 con 0x7fe90c102550 2024-06-10T01:08:14.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.598+0000 7fe8ff7fe700 1 -- 172.21.15.7:0/1070295700 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe908003d70 con 0x7fe90c102550 2024-06-10T01:08:14.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.598+0000 7fe8ff7fe700 1 -- 172.21.15.7:0/1070295700 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90800f460 con 0x7fe90c102550 2024-06-10T01:08:14.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.598+0000 7fe914473700 1 -- 172.21.15.7:0/1070295700 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe90c1a5740 con 0x7fe90c102550 2024-06-10T01:08:14.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.600+0000 7fe8ff7fe700 1 -- 172.21.15.7:0/1070295700 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fe90802a070 con 0x7fe90c102550 2024-06-10T01:08:14.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.601+0000 7fe8ff7fe700 1 --2- 172.21.15.7:0/1070295700 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe8f80a2c20 0x7fe8f80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:14.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.601+0000 7fe911a0e700 1 --2- 172.21.15.7:0/1070295700 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe8f80a2c20 0x7fe8f80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:14.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.602+0000 7fe8ff7fe700 1 -- 172.21.15.7:0/1070295700 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fe9080d01f0 con 0x7fe90c102550 2024-06-10T01:08:14.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.602+0000 7fe911a0e700 1 --2- 172.21.15.7:0/1070295700 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe8f80a2c20 0x7fe8f80a50e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe90c107980 tx=0x7fe900009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:14.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.602+0000 7fe91ecf6b80 1 Processor -- start 2024-06-10T01:08:14.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.603+0000 7fe91ecf6b80 1 -- start start 2024-06-10T01:08:14.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.603+0000 7fe91ecf6b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56089d92e370 0x56089d92e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:14.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.603+0000 7fe91ecf6b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56089d92ec90 0x56089d928670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:14.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.603+0000 7fe91ecf6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56089d928bb0 0x56089d928ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:14.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.603+0000 7fe91ecf6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56089d816f60 con 0x56089d928bb0 2024-06-10T01:08:14.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.603+0000 7fe91ecf6b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56089d875850 con 0x56089d92ec90 2024-06-10T01:08:14.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.603+0000 7fe91ecf6b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56089d7dd460 con 0x56089d92e370 2024-06-10T01:08:14.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.603+0000 7fe912a10700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56089d928bb0 0x56089d928ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:14.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.603+0000 7fe91220f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56089d92e370 0x56089d92e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:14.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.603+0000 7fe911a0e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56089d92ec90 0x56089d928670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:14.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.603+0000 7fe912a10700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56089d928bb0 0x56089d928ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36420/0 (socket says 172.21.15.7:36420) 2024-06-10T01:08:14.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.603+0000 7fe91220f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56089d92e370 0x56089d92e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:55294/0 (socket says 172.21.15.7:55294) 2024-06-10T01:08:14.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.603+0000 7fe912a10700 1 -- 172.21.15.7:0/3162405566 learned_addr learned my addr 172.21.15.7:0/3162405566 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:14.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.604+0000 7fe912a10700 1 -- 172.21.15.7:0/3162405566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56089d92e370 msgr2=0x56089d92e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:14.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.604+0000 7fe912a10700 1 --2- 172.21.15.7:0/3162405566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56089d92e370 0x56089d92e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.604+0000 7fe912a10700 1 -- 172.21.15.7:0/3162405566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56089d92ec90 msgr2=0x56089d928670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:14.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.604+0000 7fe912a10700 1 --2- 172.21.15.7:0/3162405566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56089d92ec90 0x56089d928670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.604+0000 7fe912a10700 1 -- 172.21.15.7:0/3162405566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56089d7c48d0 con 0x56089d928bb0 2024-06-10T01:08:14.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.604+0000 7fe911a0e700 1 --2- 172.21.15.7:0/3162405566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56089d92ec90 0x56089d928670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:14.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.604+0000 7fe912a10700 1 --2- 172.21.15.7:0/3162405566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56089d928bb0 0x56089d928ff0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe904007740 tx=0x7fe904009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a3590a8bcad5ae server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:14.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.604+0000 7fe8fcd70700 1 -- 172.21.15.7:0/3162405566 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe904019020 con 0x56089d928bb0 2024-06-10T01:08:14.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.604+0000 7fe8fcd70700 1 -- 172.21.15.7:0/3162405566 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe904015d30 con 0x56089d928bb0 2024-06-10T01:08:14.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.604+0000 7fe8fcd70700 1 -- 172.21.15.7:0/3162405566 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe904012360 con 0x56089d928bb0 2024-06-10T01:08:14.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.605+0000 7fe91ecf6b80 1 -- 172.21.15.7:0/3162405566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56089d928bb0 msgr2=0x56089d928ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:14.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.605+0000 7fe91ecf6b80 1 --2- 172.21.15.7:0/3162405566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56089d928bb0 0x56089d928ff0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe904007740 tx=0x7fe904009f90 comp rx=0 tx=0).stop 2024-06-10T01:08:14.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.605+0000 7fe91ecf6b80 1 -- 172.21.15.7:0/3162405566 shutdown_connections 2024-06-10T01:08:14.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.605+0000 7fe91ecf6b80 1 --2- 172.21.15.7:0/3162405566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56089d928bb0 0x56089d928ff0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.605+0000 7fe91ecf6b80 1 --2- 172.21.15.7:0/3162405566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56089d92e370 0x56089d92e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.605+0000 7fe91ecf6b80 1 --2- 172.21.15.7:0/3162405566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56089d92ec90 0x56089d928670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.605+0000 7fe91ecf6b80 1 -- 172.21.15.7:0/3162405566 >> 172.21.15.7:0/3162405566 conn(0x56089d851080 msgr2=0x56089d91f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:14.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.605+0000 7fe91ecf6b80 1 -- 172.21.15.7:0/3162405566 shutdown_connections 2024-06-10T01:08:14.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.605+0000 7fe91ecf6b80 1 -- 172.21.15.7:0/3162405566 wait complete. 2024-06-10T01:08:14.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.605+0000 7fe91ecf6b80 1 -- start start 2024-06-10T01:08:14.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.606+0000 7fe91ecf6b80 1 Processor -- start 2024-06-10T01:08:14.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.606+0000 7fe91ecf6b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56089d928bb0 0x56089d934950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:14.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.606+0000 7fe91ecf6b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56089d92e370 0x56089d931c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:14.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.606+0000 7fe91ecf6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56089d92ec90 0x56089d932190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:14.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.606+0000 7fe91ecf6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56089d815fc0 con 0x56089d92ec90 2024-06-10T01:08:14.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.606+0000 7fe91ecf6b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56089d871810 con 0x56089d92e370 2024-06-10T01:08:14.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.606+0000 7fe91ecf6b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56089d935680 con 0x56089d928bb0 2024-06-10T01:08:14.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.606+0000 7fe91ecf6b80 10 client.0 _ll_register_callbacks cb 0x56089d80b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:14.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.606+0000 7fe91220f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56089d928bb0 0x56089d934950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:14.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.606+0000 7fe91220f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56089d928bb0 0x56089d934950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:55298/0 (socket says 172.21.15.7:55298) 2024-06-10T01:08:14.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.606+0000 7fe91220f700 1 -- 172.21.15.7:0/589949370 learned_addr learned my addr 172.21.15.7:0/589949370 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:14.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.606+0000 7fe911a0e700 1 --2- 172.21.15.7:0/589949370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56089d92e370 0x56089d931c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:14.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.606+0000 7fe912a10700 1 --2- 172.21.15.7:0/589949370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56089d92ec90 0x56089d932190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:14.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.606+0000 7fe91220f700 1 -- 172.21.15.7:0/589949370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56089d92e370 msgr2=0x56089d931c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:14.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.606+0000 7fe91220f700 1 --2- 172.21.15.7:0/589949370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56089d92e370 0x56089d931c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.606+0000 7fe91220f700 1 -- 172.21.15.7:0/589949370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56089d92ec90 msgr2=0x56089d932190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:14.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.606+0000 7fe91220f700 1 --2- 172.21.15.7:0/589949370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56089d92ec90 0x56089d932190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.606+0000 7fe91220f700 1 -- 172.21.15.7:0/589949370 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56089d83b750 con 0x56089d928bb0 2024-06-10T01:08:14.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.606+0000 7fe912a10700 1 --2- 172.21.15.7:0/589949370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56089d92ec90 0x56089d932190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:08:14.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.607+0000 7fe91220f700 1 --2- 172.21.15.7:0/589949370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56089d928bb0 0x56089d934950 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe908098b60 tx=0x7fe90809c240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aac418a9b56d942 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:14.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.607+0000 7fe8f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:08:14.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.607+0000 7fe8f5ffb700 1 -- 172.21.15.7:0/589949370 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9080b5060 con 0x56089d928bb0 2024-06-10T01:08:14.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.607+0000 7fe91ecf6b80 1 -- 172.21.15.7:0/589949370 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe904015d30 con 0x56089d928bb0 2024-06-10T01:08:14.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.607+0000 7fe8f5ffb700 1 -- 172.21.15.7:0/589949370 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fe90809aa00 con 0x56089d928bb0 2024-06-10T01:08:14.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.607+0000 7fe8f5ffb700 1 -- 172.21.15.7:0/589949370 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe90809ce00 con 0x56089d928bb0 2024-06-10T01:08:14.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.607+0000 7fe91ecf6b80 10 client.25672 fetch_fsmap learned FSMap version 29 2024-06-10T01:08:14.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.607+0000 7fe91ecf6b80 1 -- 172.21.15.7:0/589949370 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56089d93fda0 con 0x56089d928bb0 2024-06-10T01:08:14.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.607+0000 7fe8f5ffb700 1 -- 172.21.15.7:0/589949370 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe904015d30 con 0x56089d928bb0 2024-06-10T01:08:14.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.607+0000 7fe8f5ffb700 1 -- 172.21.15.7:0/589949370 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fe90809c960 con 0x56089d928bb0 2024-06-10T01:08:14.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.607+0000 7fe91ecf6b80 10 client.25672 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:08:14.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.608+0000 7fe91ecf6b80 10 client.25672 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:08:14.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.608+0000 7fe91ecf6b80 20 client.25672 populate_metadata read hostname 'smithi007' 2024-06-10T01:08:14.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.608+0000 7fe91ecf6b80 1 --2- 172.21.15.7:0/589949370 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x56089d942310 0x56089d944730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:14.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.608+0000 7fe91ecf6b80 4 client.25672 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:08:14.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.608+0000 7fe91ecf6b80 1 -- 172.21.15.7:0/589949370 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56089d882d50 con 0x56089d942310 2024-06-10T01:08:14.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.608+0000 7fe911a0e700 1 --2- 172.21.15.7:0/589949370 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x56089d942310 0x56089d944730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:14.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.608+0000 7fe8f5ffb700 10 client.25672 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:08:14.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.608+0000 7fe911a0e700 1 --2- 172.21.15.7:0/589949370 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x56089d942310 0x56089d944730 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cd23b42a1186f50d server_cookie=d59ba90f66853894 in_seq=0 out_seq=0 2024-06-10T01:08:14.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.609+0000 7fe8f5ffb700 1 -- 172.21.15.7:0/589949370 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56089d882d50 con 0x56089d942310 2024-06-10T01:08:14.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.609+0000 7fe8f5ffb700 10 client.25672 handle_command_reply: tid=0 2024-06-10T01:08:14.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.609+0000 7fe91ecf6b80 1 client.25672 shutdown 2024-06-10T01:08:14.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.609+0000 7fe91ecf6b80 1 -- 172.21.15.7:0/589949370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56089d928bb0 msgr2=0x56089d934950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:14.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.609+0000 7fe91ecf6b80 1 --2- 172.21.15.7:0/589949370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56089d928bb0 0x56089d934950 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe908098b60 tx=0x7fe90809c240 comp rx=0 tx=0).stop 2024-06-10T01:08:14.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.609+0000 7fe91ecf6b80 1 -- 172.21.15.7:0/589949370 shutdown_connections 2024-06-10T01:08:14.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.609+0000 7fe91ecf6b80 1 --2- 172.21.15.7:0/589949370 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x56089d942310 0x56089d944730 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.609+0000 7fe91ecf6b80 1 --2- 172.21.15.7:0/589949370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56089d92ec90 0x56089d932190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.609+0000 7fe91ecf6b80 1 --2- 172.21.15.7:0/589949370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56089d928bb0 0x56089d934950 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.609+0000 7fe91ecf6b80 1 --2- 172.21.15.7:0/589949370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56089d92e370 0x56089d931c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.609+0000 7fe91ecf6b80 1 -- 172.21.15.7:0/589949370 >> 172.21.15.7:0/589949370 conn(0x56089d851080 msgr2=0x56089d91e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:14.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.610+0000 7fe8f5ffb700 0 client.25672 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:08:14.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.610+0000 7fe91ecf6b80 1 -- 172.21.15.7:0/589949370 shutdown_connections 2024-06-10T01:08:14.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.610+0000 7fe91ecf6b80 1 -- 172.21.15.7:0/589949370 wait complete. 2024-06-10T01:08:14.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.610+0000 7fe91ecf6b80 20 client.25672 trim_cache size 0 max 16384 2024-06-10T01:08:14.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.625+0000 7fe91ecf6b80 1 Processor -- start 2024-06-10T01:08:14.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.626+0000 7fe91ecf6b80 1 -- start start 2024-06-10T01:08:14.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.626+0000 7fe91ecf6b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56089d928bb0 0x56089d92f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:14.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.626+0000 7fe91ecf6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56089d92e370 0x56089d91ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:14.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.626+0000 7fe91ecf6b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56089d92ec90 0x56089d91f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:14.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.626+0000 7fe91ecf6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe90c1ae880 con 0x56089d92e370 2024-06-10T01:08:14.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.626+0000 7fe91ecf6b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe90c1ac110 con 0x56089d928bb0 2024-06-10T01:08:14.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.626+0000 7fe91ecf6b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe90c1a9f80 con 0x56089d92ec90 2024-06-10T01:08:14.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.626+0000 7fe911a0e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56089d92e370 0x56089d91ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:14.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.626+0000 7fe911a0e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56089d92e370 0x56089d91ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36440/0 (socket says 172.21.15.7:36440) 2024-06-10T01:08:14.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.626+0000 7fe911a0e700 1 -- 172.21.15.7:0/2152670225 learned_addr learned my addr 172.21.15.7:0/2152670225 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:14.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.626+0000 7fe912a10700 1 --2- 172.21.15.7:0/2152670225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56089d92ec90 0x56089d91f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:14.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.626+0000 7fe91220f700 1 --2- 172.21.15.7:0/2152670225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56089d928bb0 0x56089d92f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:14.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.626+0000 7fe911a0e700 1 -- 172.21.15.7:0/2152670225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56089d92ec90 msgr2=0x56089d91f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:14.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.626+0000 7fe911a0e700 1 --2- 172.21.15.7:0/2152670225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56089d92ec90 0x56089d91f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.626+0000 7fe911a0e700 1 -- 172.21.15.7:0/2152670225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56089d928bb0 msgr2=0x56089d92f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:14.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.626+0000 7fe911a0e700 1 --2- 172.21.15.7:0/2152670225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56089d928bb0 0x56089d92f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.626+0000 7fe911a0e700 1 -- 172.21.15.7:0/2152670225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56089d92eac0 con 0x56089d92e370 2024-06-10T01:08:14.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.626+0000 7fe911a0e700 1 --2- 172.21.15.7:0/2152670225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56089d92e370 0x56089d91ee80 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe900013010 tx=0x7fe900014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7559966cc3e10320 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:14.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.627+0000 7fe8f7fff700 1 -- 172.21.15.7:0/2152670225 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe900029650 con 0x56089d92e370 2024-06-10T01:08:14.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.627+0000 7fe8f7fff700 1 -- 172.21.15.7:0/2152670225 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe900029cd0 con 0x56089d92e370 2024-06-10T01:08:14.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.627+0000 7fe8f7fff700 1 -- 172.21.15.7:0/2152670225 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90001ec10 con 0x56089d92e370 2024-06-10T01:08:14.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.627+0000 7fe91ecf6b80 1 -- 172.21.15.7:0/2152670225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56089d92e370 msgr2=0x56089d91ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:14.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.627+0000 7fe91ecf6b80 1 --2- 172.21.15.7:0/2152670225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56089d92e370 0x56089d91ee80 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe900013010 tx=0x7fe900014000 comp rx=0 tx=0).stop 2024-06-10T01:08:14.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.627+0000 7fe91ecf6b80 1 -- 172.21.15.7:0/2152670225 shutdown_connections 2024-06-10T01:08:14.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.627+0000 7fe91ecf6b80 1 --2- 172.21.15.7:0/2152670225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56089d92ec90 0x56089d91f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.627+0000 7fe91ecf6b80 1 --2- 172.21.15.7:0/2152670225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56089d928bb0 0x56089d92f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.627+0000 7fe91ecf6b80 1 --2- 172.21.15.7:0/2152670225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56089d92e370 0x56089d91ee80 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.627+0000 7fe91ecf6b80 1 -- 172.21.15.7:0/2152670225 >> 172.21.15.7:0/2152670225 conn(0x56089d851080 msgr2=0x56089d92f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:14.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.627+0000 7fe91ecf6b80 1 -- 172.21.15.7:0/2152670225 shutdown_connections 2024-06-10T01:08:14.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.628+0000 7fe91ecf6b80 1 -- 172.21.15.7:0/2152670225 wait complete. 2024-06-10T01:08:14.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.628+0000 7fe91ecf6b80 1 -- start start 2024-06-10T01:08:14.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.628+0000 7fe91ecf6b80 1 Processor -- start 2024-06-10T01:08:14.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.628+0000 7fe91ecf6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56089d928bb0 0x56089d93abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:14.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.628+0000 7fe91ecf6b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56089d92e370 0x56089d93b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:14.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.628+0000 7fe91ecf6b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56089d92ec90 0x56089d9495f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:14.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.628+0000 7fe91ecf6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe90c1aa5d0 con 0x56089d928bb0 2024-06-10T01:08:14.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.628+0000 7fe91ecf6b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe90c1aba40 con 0x56089d92e370 2024-06-10T01:08:14.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.628+0000 7fe91ecf6b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe90c1aaca0 con 0x56089d92ec90 2024-06-10T01:08:14.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.628+0000 7fe91ecf6b80 10 client.0 _ll_register_callbacks cb 0x56089d80b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:14.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.628+0000 7fe91220f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56089d928bb0 0x56089d93abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:14.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.628+0000 7fe911a0e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56089d92e370 0x56089d93b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:14.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.628+0000 7fe91220f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56089d928bb0 0x56089d93abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36442/0 (socket says 172.21.15.7:36442) 2024-06-10T01:08:14.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.628+0000 7fe91220f700 1 -- 172.21.15.7:0/3594809684 learned_addr learned my addr 172.21.15.7:0/3594809684 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:14.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.629+0000 7fe912a10700 1 --2- 172.21.15.7:0/3594809684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56089d92ec90 0x56089d9495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:14.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.629+0000 7fe91220f700 1 -- 172.21.15.7:0/3594809684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56089d92ec90 msgr2=0x56089d9495f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:14.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.629+0000 7fe91220f700 1 --2- 172.21.15.7:0/3594809684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56089d92ec90 0x56089d9495f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.629+0000 7fe91220f700 1 -- 172.21.15.7:0/3594809684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56089d92e370 msgr2=0x56089d93b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:14.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.629+0000 7fe91220f700 1 --2- 172.21.15.7:0/3594809684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56089d92e370 0x56089d93b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.629+0000 7fe91220f700 1 -- 172.21.15.7:0/3594809684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56089d949df0 con 0x56089d928bb0 2024-06-10T01:08:14.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.629+0000 7fe911a0e700 1 --2- 172.21.15.7:0/3594809684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56089d92e370 0x56089d93b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:14.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.629+0000 7fe91220f700 1 --2- 172.21.15.7:0/3594809684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56089d928bb0 0x56089d93abd0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe9080990d0 tx=0x7fe908098a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c511c1ab0d3fd710 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:14.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.629+0000 7fe8f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:08:14.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.629+0000 7fe8f5ffb700 1 -- 172.21.15.7:0/3594809684 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90809c700 con 0x56089d928bb0 2024-06-10T01:08:14.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.629+0000 7fe8f5ffb700 1 -- 172.21.15.7:0/3594809684 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fe9080b0b90 con 0x56089d928bb0 2024-06-10T01:08:14.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.629+0000 7fe91ecf6b80 1 -- 172.21.15.7:0/3594809684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe900029cd0 con 0x56089d928bb0 2024-06-10T01:08:14.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.629+0000 7fe8f5ffb700 1 -- 172.21.15.7:0/3594809684 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe9080a52e0 con 0x56089d928bb0 2024-06-10T01:08:14.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.629+0000 7fe91ecf6b80 10 client.16110 fetch_fsmap learned FSMap version 29 2024-06-10T01:08:14.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.629+0000 7fe91ecf6b80 1 -- 172.21.15.7:0/3594809684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56089d9acaf0 con 0x56089d928bb0 2024-06-10T01:08:14.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.630+0000 7fe8f5ffb700 1 -- 172.21.15.7:0/3594809684 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe900029cd0 con 0x56089d928bb0 2024-06-10T01:08:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.630+0000 7fe8f5ffb700 1 -- 172.21.15.7:0/3594809684 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fe90809bbe0 con 0x56089d928bb0 2024-06-10T01:08:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.630+0000 7fe91ecf6b80 10 client.16110 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:08:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.630+0000 7fe91ecf6b80 10 client.16110 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:08:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.630+0000 7fe91ecf6b80 20 client.16110 populate_metadata read hostname 'smithi007' 2024-06-10T01:08:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.630+0000 7fe91ecf6b80 1 --2- 172.21.15.7:0/3594809684 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x56089d9ad000 0x56089d9af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.630+0000 7fe91ecf6b80 4 client.16110 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:08:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.630+0000 7fe91ecf6b80 1 -- 172.21.15.7:0/3594809684 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56089d87d2d0 con 0x56089d9ad000 2024-06-10T01:08:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.630+0000 7fe911a0e700 1 --2- 172.21.15.7:0/3594809684 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x56089d9ad000 0x56089d9af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.630+0000 7fe8f5ffb700 10 client.16110 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:08:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.630+0000 7fe911a0e700 1 --2- 172.21.15.7:0/3594809684 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x56089d9ad000 0x56089d9af3f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a675bcae5d9b601 server_cookie=559adc2d7d704c0f in_seq=0 out_seq=0 2024-06-10T01:08:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.631+0000 7fe8f5ffb700 1 -- 172.21.15.7:0/3594809684 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56089d87d2d0 con 0x56089d9ad000 2024-06-10T01:08:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.631+0000 7fe8f5ffb700 10 client.16110 handle_command_reply: tid=0 2024-06-10T01:08:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.631+0000 7fe91ecf6b80 1 client.16110 shutdown 2024-06-10T01:08:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.631+0000 7fe91ecf6b80 1 -- 172.21.15.7:0/3594809684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56089d928bb0 msgr2=0x56089d93abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.631+0000 7fe91ecf6b80 1 --2- 172.21.15.7:0/3594809684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56089d928bb0 0x56089d93abd0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe9080990d0 tx=0x7fe908098a90 comp rx=0 tx=0).stop 2024-06-10T01:08:14.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.631+0000 7fe91ecf6b80 1 -- 172.21.15.7:0/3594809684 shutdown_connections 2024-06-10T01:08:14.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.631+0000 7fe91ecf6b80 1 --2- 172.21.15.7:0/3594809684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56089d92ec90 0x56089d9495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.631+0000 7fe91ecf6b80 1 --2- 172.21.15.7:0/3594809684 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x56089d9ad000 0x56089d9af3f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.631+0000 7fe8f5ffb700 0 client.16110 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:08:14.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.631+0000 7fe91ecf6b80 1 --2- 172.21.15.7:0/3594809684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56089d928bb0 0x56089d93abd0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.631+0000 7fe91ecf6b80 1 --2- 172.21.15.7:0/3594809684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56089d92e370 0x56089d93b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.631+0000 7fe91ecf6b80 1 -- 172.21.15.7:0/3594809684 >> 172.21.15.7:0/3594809684 conn(0x56089d851080 msgr2=0x56089d923250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:14.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.631+0000 7fe91ecf6b80 1 -- 172.21.15.7:0/3594809684 shutdown_connections 2024-06-10T01:08:14.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.631+0000 7fe91ecf6b80 1 -- 172.21.15.7:0/3594809684 wait complete. 2024-06-10T01:08:14.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.631+0000 7fe91ecf6b80 20 client.16110 trim_cache size 0 max 16384 2024-06-10T01:08:14.632 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:08:14.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.631+0000 7fe914473700 1 -- 172.21.15.7:0/1070295700 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe8f80a2c20 msgr2=0x7fe8f80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:14.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.631+0000 7fe914473700 1 --2- 172.21.15.7:0/1070295700 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe8f80a2c20 0x7fe8f80a50e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe90c107980 tx=0x7fe900009260 comp rx=0 tx=0).stop 2024-06-10T01:08:14.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.631+0000 7fe914473700 1 -- 172.21.15.7:0/1070295700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe90c102550 msgr2=0x7fe90c19e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:14.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.631+0000 7fe914473700 1 --2- 172.21.15.7:0/1070295700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe90c102550 0x7fe90c19e870 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe908000c00 tx=0x7fe90800bab0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.632+0000 7fe914473700 1 -- 172.21.15.7:0/1070295700 shutdown_connections 2024-06-10T01:08:14.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.632+0000 7fe914473700 1 --2- 172.21.15.7:0/1070295700 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe8f80a2c20 0x7fe8f80a50e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.632+0000 7fe914473700 1 --2- 172.21.15.7:0/1070295700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe90c1084f0 0x7fe90c198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.632+0000 7fe914473700 1 --2- 172.21.15.7:0/1070295700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe90c102550 0x7fe90c19e870 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.632+0000 7fe914473700 1 --2- 172.21.15.7:0/1070295700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe90c102f10 0x7fe90c19edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.632+0000 7fe914473700 1 -- 172.21.15.7:0/1070295700 >> 172.21.15.7:0/1070295700 conn(0x7fe90c079650 msgr2=0x7fe90c1005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:14.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.632+0000 7fe914473700 1 -- 172.21.15.7:0/1070295700 shutdown_connections 2024-06-10T01:08:14.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:14.632+0000 7fe914473700 1 -- 172.21.15.7:0/1070295700 wait complete. 2024-06-10T01:08:14.710 INFO:tasks.workunit.client.0.smithi007.stdout: 1 71344 7.33 MB/sec warmup 55 sec latency 58.174 ms 2024-06-10T01:08:15.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:14 smithi007 ceph-mon[32398]: pgmap v499: 129 pgs: 129 active+clean; 37 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 291 op/s 2024-06-10T01:08:15.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:14 smithi078 ceph-mon[36683]: pgmap v499: 129 pgs: 129 active+clean; 37 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 291 op/s 2024-06-10T01:08:15.378 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:08:15.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:14 smithi175 ceph-mon[36636]: pgmap v499: 129 pgs: 129 active+clean; 37 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 291 op/s 2024-06-10T01:08:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.649+0000 7eff67dce640 1 -- 172.21.15.7:0/4037362345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff601015d0 msgr2=0x7eff60101a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.649+0000 7eff67dce640 1 --2- 172.21.15.7:0/4037362345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff601015d0 0x7eff60101a50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7eff58009be0 tx=0x7eff5802a6c0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.649+0000 7eff67dce640 1 -- 172.21.15.7:0/4037362345 shutdown_connections 2024-06-10T01:08:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.649+0000 7eff67dce640 1 --2- 172.21.15.7:0/4037362345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff601015d0 0x7eff60101a50 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.649+0000 7eff67dce640 1 --2- 172.21.15.7:0/4037362345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff60100c30 0x7eff60101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.649+0000 7eff67dce640 1 --2- 172.21.15.7:0/4037362345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff601002d0 0x7eff601006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.649+0000 7eff67dce640 1 -- 172.21.15.7:0/4037362345 >> 172.21.15.7:0/4037362345 conn(0x7eff600fbe70 msgr2=0x7eff600fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.649+0000 7eff67dce640 1 -- 172.21.15.7:0/4037362345 shutdown_connections 2024-06-10T01:08:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.649+0000 7eff67dce640 1 -- 172.21.15.7:0/4037362345 wait complete. 2024-06-10T01:08:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.650+0000 7eff67dce640 1 Processor -- start 2024-06-10T01:08:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.651+0000 7eff67dce640 1 -- start start 2024-06-10T01:08:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.651+0000 7eff67dce640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff601002d0 0x7eff60199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.651+0000 7eff67dce640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff60100c30 0x7eff6019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.651+0000 7eff67dce640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff601015d0 0x7eff601a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.651+0000 7eff65b6a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff601002d0 0x7eff60199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.651+0000 7eff67dce640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7eff6010f870 con 0x7eff601015d0 2024-06-10T01:08:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.651+0000 7eff65b6a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff601002d0 0x7eff60199eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:55350/0 (socket says 172.21.15.7:55350) 2024-06-10T01:08:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.651+0000 7eff65b6a640 1 -- 172.21.15.7:0/906527466 learned_addr learned my addr 172.21.15.7:0/906527466 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.651+0000 7eff67dce640 1 -- 172.21.15.7:0/906527466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7eff6010f6f0 con 0x7eff60100c30 2024-06-10T01:08:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.651+0000 7eff67dce640 1 -- 172.21.15.7:0/906527466 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7eff6010f9f0 con 0x7eff601002d0 2024-06-10T01:08:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.651+0000 7eff6636b640 1 --2- 172.21.15.7:0/906527466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff601015d0 0x7eff601a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.652+0000 7eff65369640 1 --2- 172.21.15.7:0/906527466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff60100c30 0x7eff6019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.652+0000 7eff65b6a640 1 -- 172.21.15.7:0/906527466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff60100c30 msgr2=0x7eff6019a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.652+0000 7eff65b6a640 1 --2- 172.21.15.7:0/906527466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff60100c30 0x7eff6019a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.652+0000 7eff65b6a640 1 -- 172.21.15.7:0/906527466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff601015d0 msgr2=0x7eff601a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.652+0000 7eff65b6a640 1 --2- 172.21.15.7:0/906527466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff601015d0 0x7eff601a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.652+0000 7eff65b6a640 1 -- 172.21.15.7:0/906527466 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff601a1ad0 con 0x7eff601002d0 2024-06-10T01:08:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.652+0000 7eff65369640 1 --2- 172.21.15.7:0/906527466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff60100c30 0x7eff6019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:08:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.652+0000 7eff6636b640 1 --2- 172.21.15.7:0/906527466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff601015d0 0x7eff601a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.652+0000 7eff65b6a640 1 --2- 172.21.15.7:0/906527466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff601002d0 0x7eff60199eb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00e9f0 tx=0x7eff5c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.652+0000 7eff56ffd640 1 -- 172.21.15.7:0/906527466 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c00cd80 con 0x7eff601002d0 2024-06-10T01:08:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.652+0000 7eff67dce640 1 -- 172.21.15.7:0/906527466 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff601a1dc0 con 0x7eff601002d0 2024-06-10T01:08:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.652+0000 7eff67dce640 1 -- 172.21.15.7:0/906527466 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff601a2280 con 0x7eff601002d0 2024-06-10T01:08:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.652+0000 7eff56ffd640 1 -- 172.21.15.7:0/906527466 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff5c010910 con 0x7eff601002d0 2024-06-10T01:08:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.652+0000 7eff56ffd640 1 -- 172.21.15.7:0/906527466 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c0188f0 con 0x7eff601002d0 2024-06-10T01:08:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.655+0000 7eff6abb0740 1 Processor -- start 2024-06-10T01:08:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.655+0000 7eff6abb0740 1 -- start start 2024-06-10T01:08:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.655+0000 7eff56ffd640 1 -- 172.21.15.7:0/906527466 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7eff5c018bd0 con 0x7eff601002d0 2024-06-10T01:08:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.655+0000 7eff6abb0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55648d893010 0x55648d893430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.655+0000 7eff6abb0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55648d893a00 0x55648d898500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.655+0000 7eff6abb0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55648d898b70 0x55648d89b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.655+0000 7eff6abb0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55648d7f64f0 con 0x55648d893010 2024-06-10T01:08:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.655+0000 7eff6abb0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55648d794870 con 0x55648d893a00 2024-06-10T01:08:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.655+0000 7eff6abb0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55648d7f24b0 con 0x55648d898b70 2024-06-10T01:08:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.656+0000 7eff6636b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55648d893010 0x55648d893430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.656+0000 7eff6636b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55648d893010 0x55648d893430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36464/0 (socket says 172.21.15.7:36464) 2024-06-10T01:08:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.656+0000 7eff6636b640 1 -- 172.21.15.7:0/234897149 learned_addr learned my addr 172.21.15.7:0/234897149 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.656+0000 7eff65369640 1 --2- 172.21.15.7:0/234897149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55648d898b70 0x55648d89b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.656+0000 7eff65b6a640 1 --2- 172.21.15.7:0/234897149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55648d893a00 0x55648d898500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.656+0000 7eff6636b640 1 -- 172.21.15.7:0/234897149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55648d898b70 msgr2=0x55648d89b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.656+0000 7eff6636b640 1 --2- 172.21.15.7:0/234897149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55648d898b70 0x55648d89b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.656+0000 7eff6636b640 1 -- 172.21.15.7:0/234897149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55648d893a00 msgr2=0x55648d898500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.656+0000 7eff6636b640 1 --2- 172.21.15.7:0/234897149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55648d893a00 0x55648d898500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.656+0000 7eff6636b640 1 -- 172.21.15.7:0/234897149 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55648d7b09b0 con 0x55648d893010 2024-06-10T01:08:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.656+0000 7eff6636b640 1 --2- 172.21.15.7:0/234897149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55648d893010 0x55648d893430 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7eff58009be0 tx=0x7eff580303a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f181011452db9749 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.657+0000 7eff4bfff640 1 -- 172.21.15.7:0/234897149 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58041030 con 0x55648d893010 2024-06-10T01:08:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.657+0000 7eff56ffd640 1 --2- 172.21.15.7:0/906527466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7eff4c0a2c00 0x7eff4c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.657+0000 7eff4bfff640 1 -- 172.21.15.7:0/234897149 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff58003e50 con 0x55648d893010 2024-06-10T01:08:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.657+0000 7eff4bfff640 1 -- 172.21.15.7:0/234897149 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58036040 con 0x55648d893010 2024-06-10T01:08:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.657+0000 7eff56ffd640 1 -- 172.21.15.7:0/906527466 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7eff5c0cf700 con 0x7eff601002d0 2024-06-10T01:08:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.657+0000 7eff65b6a640 1 --2- 172.21.15.7:0/906527466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7eff4c0a2c00 0x7eff4c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.658+0000 7eff6abb0740 1 -- 172.21.15.7:0/234897149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55648d893010 msgr2=0x55648d893430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.658+0000 7eff6abb0740 1 --2- 172.21.15.7:0/234897149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55648d893010 0x55648d893430 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7eff58009be0 tx=0x7eff580303a0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.658+0000 7eff65b6a640 1 --2- 172.21.15.7:0/906527466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7eff4c0a2c00 0x7eff4c0a50c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0cd1f0 tx=0x7eff5c0d3040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.658+0000 7eff6abb0740 1 -- 172.21.15.7:0/234897149 shutdown_connections 2024-06-10T01:08:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.658+0000 7eff6abb0740 1 --2- 172.21.15.7:0/234897149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55648d898b70 0x55648d89b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.658+0000 7eff6abb0740 1 --2- 172.21.15.7:0/234897149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55648d893a00 0x55648d898500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.658+0000 7eff6abb0740 1 --2- 172.21.15.7:0/234897149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55648d893010 0x55648d893430 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.658+0000 7eff6abb0740 1 -- 172.21.15.7:0/234897149 >> 172.21.15.7:0/234897149 conn(0x55648d88e680 msgr2=0x55648d88ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.658+0000 7eff6abb0740 1 -- 172.21.15.7:0/234897149 shutdown_connections 2024-06-10T01:08:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.658+0000 7eff6abb0740 1 -- 172.21.15.7:0/234897149 wait complete. 2024-06-10T01:08:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.658+0000 7eff6abb0740 1 -- start start 2024-06-10T01:08:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.658+0000 7eff6abb0740 1 Processor -- start 2024-06-10T01:08:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.659+0000 7eff6abb0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55648d893010 0x55648d89d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.659+0000 7eff6abb0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55648d893a00 0x55648d89d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.659+0000 7eff6abb0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55648d898b70 0x55648d8a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.659+0000 7eff6abb0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55648d89bac0 con 0x55648d893010 2024-06-10T01:08:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.659+0000 7eff6abb0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55648d77fae0 con 0x55648d898b70 2024-06-10T01:08:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.659+0000 7eff6abb0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55648d89bc40 con 0x55648d893a00 2024-06-10T01:08:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.659+0000 7eff6abb0740 10 client.0 _ll_register_callbacks cb 0x55648d7b89b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.659+0000 7eff6636b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55648d893010 0x55648d89d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.659+0000 7eff65369640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55648d893a00 0x55648d89d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.659+0000 7eff6636b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55648d893010 0x55648d89d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36472/0 (socket says 172.21.15.7:36472) 2024-06-10T01:08:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.659+0000 7eff6636b640 1 -- 172.21.15.7:0/948391235 learned_addr learned my addr 172.21.15.7:0/948391235 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.659+0000 7eff6636b640 1 --2- 172.21.15.7:0/948391235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55648d898b70 0x55648d8a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.659+0000 7eff65369640 1 -- 172.21.15.7:0/948391235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55648d898b70 msgr2=0x55648d8a47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.659+0000 7eff65369640 1 --2- 172.21.15.7:0/948391235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55648d898b70 0x55648d8a47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.659+0000 7eff65369640 1 -- 172.21.15.7:0/948391235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55648d893010 msgr2=0x55648d89d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.659+0000 7eff65369640 1 --2- 172.21.15.7:0/948391235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55648d893010 0x55648d89d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.659+0000 7eff65369640 1 -- 172.21.15.7:0/948391235 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55648d50a210 con 0x55648d893a00 2024-06-10T01:08:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.659+0000 7eff6636b640 1 --2- 172.21.15.7:0/948391235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55648d893010 0x55648d89d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.660+0000 7eff65369640 1 --2- 172.21.15.7:0/948391235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55648d893a00 0x55648d89d760 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7eff6019b3c0 tx=0x7eff50012b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5864b18f93f9233f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.660+0000 7eff497fa640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:08:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.660+0000 7eff497fa640 1 -- 172.21.15.7:0/948391235 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50011b20 con 0x55648d893a00 2024-06-10T01:08:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.660+0000 7eff6abb0740 1 -- 172.21.15.7:0/948391235 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff58003e50 con 0x55648d893a00 2024-06-10T01:08:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.660+0000 7eff497fa640 1 -- 172.21.15.7:0/948391235 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7eff50011cc0 con 0x55648d893a00 2024-06-10T01:08:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.660+0000 7eff497fa640 1 -- 172.21.15.7:0/948391235 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7eff50005020 con 0x55648d893a00 2024-06-10T01:08:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.660+0000 7eff6abb0740 10 client.25690 fetch_fsmap learned FSMap version 29 2024-06-10T01:08:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.660+0000 7eff6abb0740 1 -- 172.21.15.7:0/948391235 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55648d5d6620 con 0x55648d893a00 2024-06-10T01:08:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.660+0000 7eff497fa640 1 -- 172.21.15.7:0/948391235 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff50025070 con 0x55648d893a00 2024-06-10T01:08:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.660+0000 7eff497fa640 1 -- 172.21.15.7:0/948391235 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7eff5001c470 con 0x55648d893a00 2024-06-10T01:08:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.660+0000 7eff6abb0740 10 client.25690 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:08:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.660+0000 7eff6abb0740 10 client.25690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.661+0000 7eff6abb0740 20 client.25690 populate_metadata read hostname 'smithi007' 2024-06-10T01:08:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.661+0000 7eff6abb0740 1 --2- 172.21.15.7:0/948391235 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55648d8a9180 0x55648d8ab5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.661+0000 7eff6abb0740 4 client.25690 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:08:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.661+0000 7eff6abb0740 1 -- 172.21.15.7:0/948391235 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55648d809f10 con 0x55648d8a9180 2024-06-10T01:08:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.661+0000 7eff6636b640 1 --2- 172.21.15.7:0/948391235 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55648d8a9180 0x55648d8ab5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.661+0000 7eff6636b640 1 --2- 172.21.15.7:0/948391235 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55648d8a9180 0x55648d8ab5c0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33dd230926dd87f server_cookie=1c089b5301f8bd66 in_seq=0 out_seq=0 2024-06-10T01:08:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.661+0000 7eff497fa640 10 client.25690 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.662+0000 7eff497fa640 1 -- 172.21.15.7:0/948391235 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55648d809f10 con 0x55648d8a9180 2024-06-10T01:08:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.662+0000 7eff497fa640 10 client.25690 handle_command_reply: tid=0 2024-06-10T01:08:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.662+0000 7eff6abb0740 1 client.25690 shutdown 2024-06-10T01:08:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.662+0000 7eff6abb0740 1 -- 172.21.15.7:0/948391235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55648d893a00 msgr2=0x55648d89d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.662+0000 7eff6abb0740 1 --2- 172.21.15.7:0/948391235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55648d893a00 0x55648d89d760 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7eff6019b3c0 tx=0x7eff50012b40 comp rx=0 tx=0).stop 2024-06-10T01:08:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.662+0000 7eff6abb0740 1 -- 172.21.15.7:0/948391235 shutdown_connections 2024-06-10T01:08:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.662+0000 7eff6abb0740 1 --2- 172.21.15.7:0/948391235 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55648d8a9180 0x55648d8ab5c0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.662+0000 7eff6abb0740 1 --2- 172.21.15.7:0/948391235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55648d898b70 0x55648d8a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.662+0000 7eff6abb0740 1 --2- 172.21.15.7:0/948391235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55648d893a00 0x55648d89d760 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.662+0000 7eff497fa640 0 client.25690 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.662+0000 7eff6abb0740 1 --2- 172.21.15.7:0/948391235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55648d893010 0x55648d89d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.662+0000 7eff6abb0740 1 -- 172.21.15.7:0/948391235 >> 172.21.15.7:0/948391235 conn(0x55648d88e680 msgr2=0x55648d88d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.662+0000 7eff6abb0740 1 -- 172.21.15.7:0/948391235 shutdown_connections 2024-06-10T01:08:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.662+0000 7eff6abb0740 1 -- 172.21.15.7:0/948391235 wait complete. 2024-06-10T01:08:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.662+0000 7eff6abb0740 20 client.25690 trim_cache size 0 max 16384 2024-06-10T01:08:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.681+0000 7eff6abb0740 1 Processor -- start 2024-06-10T01:08:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.681+0000 7eff6abb0740 1 -- start start 2024-06-10T01:08:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.681+0000 7eff6abb0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55648d893010 0x55648d894be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.681+0000 7eff6abb0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55648d893a00 0x55648d895120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.681+0000 7eff6abb0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55648d898b70 0x55648d8a4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.681+0000 7eff6abb0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7eff44012410 con 0x55648d893010 2024-06-10T01:08:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.681+0000 7eff6abb0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7eff4400fce0 con 0x55648d898b70 2024-06-10T01:08:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.681+0000 7eff6abb0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7eff4400db30 con 0x55648d893a00 2024-06-10T01:08:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.681+0000 7eff6636b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55648d893010 0x55648d894be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.681+0000 7eff65369640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55648d893a00 0x55648d895120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.681+0000 7eff6636b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55648d893010 0x55648d894be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36474/0 (socket says 172.21.15.7:36474) 2024-06-10T01:08:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.681+0000 7eff6636b640 1 -- 172.21.15.7:0/382058454 learned_addr learned my addr 172.21.15.7:0/382058454 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.681+0000 7eff6636b640 1 --2- 172.21.15.7:0/382058454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55648d898b70 0x55648d8a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.682+0000 7eff65369640 1 -- 172.21.15.7:0/382058454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55648d898b70 msgr2=0x55648d8a4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.682+0000 7eff65369640 1 --2- 172.21.15.7:0/382058454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55648d898b70 0x55648d8a4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.682+0000 7eff65369640 1 -- 172.21.15.7:0/382058454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55648d893010 msgr2=0x55648d894be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.682+0000 7eff65369640 1 --2- 172.21.15.7:0/382058454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55648d893010 0x55648d894be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.682+0000 7eff65369640 1 -- 172.21.15.7:0/382058454 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55648d88e4b0 con 0x55648d893a00 2024-06-10T01:08:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.682+0000 7eff6636b640 1 --2- 172.21.15.7:0/382058454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55648d893010 0x55648d894be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.682+0000 7eff65369640 1 --2- 172.21.15.7:0/382058454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55648d893a00 0x55648d895120 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7eff5001caa0 tx=0x7eff5001cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4a4634a5657a01d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.682+0000 7eff4b7fe640 1 -- 172.21.15.7:0/382058454 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5002d020 con 0x55648d893a00 2024-06-10T01:08:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.682+0000 7eff4b7fe640 1 -- 172.21.15.7:0/382058454 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff50021070 con 0x55648d893a00 2024-06-10T01:08:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.682+0000 7eff4b7fe640 1 -- 172.21.15.7:0/382058454 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50004660 con 0x55648d893a00 2024-06-10T01:08:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.683+0000 7eff6abb0740 1 -- 172.21.15.7:0/382058454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55648d893a00 msgr2=0x55648d895120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.683+0000 7eff6abb0740 1 --2- 172.21.15.7:0/382058454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55648d893a00 0x55648d895120 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7eff5001caa0 tx=0x7eff5001cde0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.683+0000 7eff6abb0740 1 -- 172.21.15.7:0/382058454 shutdown_connections 2024-06-10T01:08:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.683+0000 7eff6abb0740 1 --2- 172.21.15.7:0/382058454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55648d898b70 0x55648d8a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.683+0000 7eff6abb0740 1 --2- 172.21.15.7:0/382058454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55648d893a00 0x55648d895120 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.683+0000 7eff6abb0740 1 --2- 172.21.15.7:0/382058454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55648d893010 0x55648d894be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.683+0000 7eff6abb0740 1 -- 172.21.15.7:0/382058454 >> 172.21.15.7:0/382058454 conn(0x55648d88e680 msgr2=0x55648d898250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.683+0000 7eff6abb0740 1 -- 172.21.15.7:0/382058454 shutdown_connections 2024-06-10T01:08:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.683+0000 7eff6abb0740 1 -- 172.21.15.7:0/382058454 wait complete. 2024-06-10T01:08:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.684+0000 7eff6abb0740 1 -- start start 2024-06-10T01:08:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.684+0000 7eff6abb0740 1 Processor -- start 2024-06-10T01:08:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.684+0000 7eff6abb0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55648d893010 0x55648d8a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.684+0000 7eff6abb0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55648d893a00 0x55648d8a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.684+0000 7eff6abb0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55648d898b70 0x55648d911e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.684+0000 7eff6abb0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7eff4400e1a0 con 0x55648d893010 2024-06-10T01:08:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.684+0000 7eff6abb0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7eff4400f610 con 0x55648d893a00 2024-06-10T01:08:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.684+0000 7eff6abb0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7eff4400e870 con 0x55648d898b70 2024-06-10T01:08:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.685+0000 7eff6abb0740 10 client.0 _ll_register_callbacks cb 0x7eff500097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.685+0000 7eff6636b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55648d893010 0x55648d8a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.685+0000 7eff6636b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55648d893010 0x55648d8a15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36480/0 (socket says 172.21.15.7:36480) 2024-06-10T01:08:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.685+0000 7eff6636b640 1 -- 172.21.15.7:0/1499747502 learned_addr learned my addr 172.21.15.7:0/1499747502 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.685+0000 7eff6636b640 1 --2- 172.21.15.7:0/1499747502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55648d898b70 0x55648d911e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.685+0000 7eff65369640 1 --2- 172.21.15.7:0/1499747502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55648d893a00 0x55648d8a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.685+0000 7eff6636b640 1 -- 172.21.15.7:0/1499747502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55648d898b70 msgr2=0x55648d911e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.685+0000 7eff6636b640 1 --2- 172.21.15.7:0/1499747502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55648d898b70 0x55648d911e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.685+0000 7eff6636b640 1 -- 172.21.15.7:0/1499747502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55648d893a00 msgr2=0x55648d8a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.685+0000 7eff6636b640 1 --2- 172.21.15.7:0/1499747502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55648d893a00 0x55648d8a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.685+0000 7eff6636b640 1 -- 172.21.15.7:0/1499747502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55648d8946d0 con 0x55648d893010 2024-06-10T01:08:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.685+0000 7eff6636b640 1 --2- 172.21.15.7:0/1499747502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55648d893010 0x55648d8a15e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7eff58031040 tx=0x7eff58009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44354c0e7d54b441 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.685+0000 7eff497fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:08:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.685+0000 7eff497fa640 1 -- 172.21.15.7:0/1499747502 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58036050 con 0x55648d893010 2024-06-10T01:08:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.685+0000 7eff497fa640 1 -- 172.21.15.7:0/1499747502 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7eff58003b70 con 0x55648d893010 2024-06-10T01:08:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.685+0000 7eff6abb0740 1 -- 172.21.15.7:0/1499747502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff50021070 con 0x55648d893010 2024-06-10T01:08:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.686+0000 7eff497fa640 1 -- 172.21.15.7:0/1499747502 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7eff58041040 con 0x55648d893010 2024-06-10T01:08:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.686+0000 7eff6abb0740 10 client.16128 fetch_fsmap learned FSMap version 29 2024-06-10T01:08:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.686+0000 7eff6abb0740 1 -- 172.21.15.7:0/1499747502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55648d913450 con 0x55648d893010 2024-06-10T01:08:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.686+0000 7eff497fa640 1 -- 172.21.15.7:0/1499747502 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff50021070 con 0x55648d893010 2024-06-10T01:08:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.686+0000 7eff497fa640 1 -- 172.21.15.7:0/1499747502 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7eff5803d460 con 0x55648d893010 2024-06-10T01:08:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.686+0000 7eff6abb0740 10 client.16128 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:08:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.686+0000 7eff6abb0740 10 client.16128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.686+0000 7eff6abb0740 20 client.16128 populate_metadata read hostname 'smithi007' 2024-06-10T01:08:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.686+0000 7eff6abb0740 1 --2- 172.21.15.7:0/1499747502 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55648d913a20 0x55648d915e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.686+0000 7eff6abb0740 4 client.16128 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:08:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.686+0000 7eff6abb0740 1 -- 172.21.15.7:0/1499747502 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55648d803c20 con 0x55648d913a20 2024-06-10T01:08:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.686+0000 7eff65369640 1 --2- 172.21.15.7:0/1499747502 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55648d913a20 0x55648d915e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.687+0000 7eff497fa640 10 client.16128 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.687+0000 7eff65369640 1 --2- 172.21.15.7:0/1499747502 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55648d913a20 0x55648d915e40 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2670c613d1f5e6dd server_cookie=f986f267af111c27 in_seq=0 out_seq=0 2024-06-10T01:08:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:08:15.710 INFO:tasks.workunit.client.0.smithi007.stdout: 1 74334 7.46 MB/sec warmup 56 sec latency 20.598 ms 2024-06-10T01:08:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.908+0000 7eff497fa640 1 -- 172.21.15.7:0/1499747502 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55648d803c20 con 0x55648d913a20 2024-06-10T01:08:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.908+0000 7eff497fa640 10 client.16128 handle_command_reply: tid=0 2024-06-10T01:08:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.908+0000 7eff6abb0740 1 client.16128 shutdown 2024-06-10T01:08:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.909+0000 7eff6abb0740 1 -- 172.21.15.7:0/1499747502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55648d893010 msgr2=0x55648d8a15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.909+0000 7eff6abb0740 1 --2- 172.21.15.7:0/1499747502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55648d893010 0x55648d8a15e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7eff58031040 tx=0x7eff58009310 comp rx=0 tx=0).stop 2024-06-10T01:08:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.909+0000 7eff6abb0740 1 -- 172.21.15.7:0/1499747502 shutdown_connections 2024-06-10T01:08:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.909+0000 7eff6abb0740 1 --2- 172.21.15.7:0/1499747502 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55648d913a20 0x55648d915e40 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.909+0000 7eff6abb0740 1 --2- 172.21.15.7:0/1499747502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55648d898b70 0x55648d911e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.909+0000 7eff6abb0740 1 --2- 172.21.15.7:0/1499747502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55648d893a00 0x55648d8a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.909+0000 7eff6abb0740 1 --2- 172.21.15.7:0/1499747502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55648d893010 0x55648d8a15e0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.909+0000 7eff6abb0740 1 -- 172.21.15.7:0/1499747502 >> 172.21.15.7:0/1499747502 conn(0x55648d88e680 msgr2=0x55648d89d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.909+0000 7eff497fa640 0 client.16128 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.909+0000 7eff6abb0740 1 -- 172.21.15.7:0/1499747502 shutdown_connections 2024-06-10T01:08:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.909+0000 7eff6abb0740 1 -- 172.21.15.7:0/1499747502 wait complete. 2024-06-10T01:08:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.909+0000 7eff6abb0740 20 client.16128 trim_cache size 0 max 16384 2024-06-10T01:08:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:08:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:08:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:08:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.910+0000 7eff67dce640 1 -- 172.21.15.7:0/906527466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7eff4c0a2c00 msgr2=0x7eff4c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.910+0000 7eff67dce640 1 --2- 172.21.15.7:0/906527466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7eff4c0a2c00 0x7eff4c0a50c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0cd1f0 tx=0x7eff5c0d3040 comp rx=0 tx=0).stop 2024-06-10T01:08:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.910+0000 7eff67dce640 1 -- 172.21.15.7:0/906527466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff601002d0 msgr2=0x7eff60199eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.910+0000 7eff67dce640 1 --2- 172.21.15.7:0/906527466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff601002d0 0x7eff60199eb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00e9f0 tx=0x7eff5c00eeb0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.910+0000 7eff67dce640 1 -- 172.21.15.7:0/906527466 shutdown_connections 2024-06-10T01:08:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.910+0000 7eff67dce640 1 --2- 172.21.15.7:0/906527466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7eff4c0a2c00 0x7eff4c0a50c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.910+0000 7eff67dce640 1 --2- 172.21.15.7:0/906527466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff601015d0 0x7eff601a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.910+0000 7eff67dce640 1 --2- 172.21.15.7:0/906527466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff60100c30 0x7eff6019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.910+0000 7eff67dce640 1 --2- 172.21.15.7:0/906527466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff601002d0 0x7eff60199eb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.910+0000 7eff67dce640 1 -- 172.21.15.7:0/906527466 >> 172.21.15.7:0/906527466 conn(0x7eff600fbe70 msgr2=0x7eff600fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.910+0000 7eff67dce640 1 -- 172.21.15.7:0/906527466 shutdown_connections 2024-06-10T01:08:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:15.910+0000 7eff67dce640 1 -- 172.21.15.7:0/906527466 wait complete. 2024-06-10T01:08:16.710 INFO:tasks.workunit.client.0.smithi007.stdout: 1 76124 7.50 MB/sec warmup 57 sec latency 157.688 ms 2024-06-10T01:08:17.179 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:16 smithi007 ceph-mon[32398]: pgmap v500: 129 pgs: 129 active+clean; 40 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.8 MiB/s wr, 312 op/s 2024-06-10T01:08:17.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:16 smithi078 ceph-mon[36683]: pgmap v500: 129 pgs: 129 active+clean; 40 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.8 MiB/s wr, 312 op/s 2024-06-10T01:08:17.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:16 smithi175 ceph-mon[36636]: pgmap v500: 129 pgs: 129 active+clean; 40 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.8 MiB/s wr, 312 op/s 2024-06-10T01:08:17.710 INFO:tasks.workunit.client.0.smithi007.stdout: 1 78913 7.62 MB/sec warmup 58 sec latency 14.040 ms 2024-06-10T01:08:18.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.705+0000 7f5e56a92700 1 -- 172.21.15.7:0/3618939999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e50101d50 msgr2=0x7f5e5010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:18.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.705+0000 7f5e56a92700 1 --2- 172.21.15.7:0/3618939999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e50101d50 0x7f5e5010e240 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5e4800b0a0 tx=0x7f5e4800b3b0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.705+0000 7f5e56a92700 1 -- 172.21.15.7:0/3618939999 shutdown_connections 2024-06-10T01:08:18.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.705+0000 7f5e56a92700 1 --2- 172.21.15.7:0/3618939999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e50101d50 0x7f5e5010e240 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.705+0000 7f5e56a92700 1 --2- 172.21.15.7:0/3618939999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e50107400 0x7f5e501077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.705+0000 7f5e56a92700 1 --2- 172.21.15.7:0/3618939999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e50101390 0x7f5e50101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.705+0000 7f5e56a92700 1 -- 172.21.15.7:0/3618939999 >> 172.21.15.7:0/3618939999 conn(0x7f5e50078340 msgr2=0x7f5e50078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:18.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.705+0000 7f5e56a92700 1 -- 172.21.15.7:0/3618939999 shutdown_connections 2024-06-10T01:08:18.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.705+0000 7f5e56a92700 1 -- 172.21.15.7:0/3618939999 wait complete. 2024-06-10T01:08:18.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.706+0000 7f5e56a92700 1 Processor -- start 2024-06-10T01:08:18.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.706+0000 7f5e56a92700 1 -- start start 2024-06-10T01:08:18.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.706+0000 7f5e56a92700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e50101390 0x7f5e50103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:18.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.706+0000 7f5e56a92700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e50101d50 0x7f5e501035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:18.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.706+0000 7f5e56a92700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e50107400 0x7f5e50103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:18.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.706+0000 7f5e56a92700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5e50111690 con 0x7f5e50101390 2024-06-10T01:08:18.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.706+0000 7f5e56a92700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5e50111510 con 0x7f5e50101d50 2024-06-10T01:08:18.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.706+0000 7f5e56a92700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5e50111810 con 0x7f5e50107400 2024-06-10T01:08:18.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.706+0000 7f5e47fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e50101d50 0x7f5e501035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:18.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.706+0000 7f5e47fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e50101d50 0x7f5e501035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:45662/0 (socket says 172.21.15.7:45662) 2024-06-10T01:08:18.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.706+0000 7f5e5502f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e50107400 0x7f5e50103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:18.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.706+0000 7f5e5482e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e50101390 0x7f5e50103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:18.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.706+0000 7f5e47fff700 1 -- 172.21.15.7:0/1941423258 learned_addr learned my addr 172.21.15.7:0/1941423258 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:18.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.707+0000 7f5e47fff700 1 -- 172.21.15.7:0/1941423258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e50107400 msgr2=0x7f5e50103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:18.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.707+0000 7f5e47fff700 1 --2- 172.21.15.7:0/1941423258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e50107400 0x7f5e50103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.707+0000 7f5e47fff700 1 -- 172.21.15.7:0/1941423258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e50101390 msgr2=0x7f5e50103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:18.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.707+0000 7f5e47fff700 1 --2- 172.21.15.7:0/1941423258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e50101390 0x7f5e50103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.707+0000 7f5e47fff700 1 -- 172.21.15.7:0/1941423258 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e501a70e0 con 0x7f5e50101d50 2024-06-10T01:08:18.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.707+0000 7f5e47fff700 1 --2- 172.21.15.7:0/1941423258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e50101d50 0x7f5e501035b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000e9a0 tx=0x7f5e4000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:18.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.707+0000 7f5e45ffb700 1 -- 172.21.15.7:0/1941423258 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4000ca10 con 0x7f5e50101d50 2024-06-10T01:08:18.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.707+0000 7f5e56a92700 1 -- 172.21.15.7:0/1941423258 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e501a73d0 con 0x7f5e50101d50 2024-06-10T01:08:18.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.707+0000 7f5e56a92700 1 -- 172.21.15.7:0/1941423258 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e501a7960 con 0x7f5e50101d50 2024-06-10T01:08:18.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.707+0000 7f5e45ffb700 1 -- 172.21.15.7:0/1941423258 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e4000cbb0 con 0x7f5e50101d50 2024-06-10T01:08:18.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.707+0000 7f5e45ffb700 1 -- 172.21.15.7:0/1941423258 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e40018680 con 0x7f5e50101d50 2024-06-10T01:08:18.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.710+0000 7f5e45ffb700 1 -- 172.21.15.7:0/1941423258 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f5e40010430 con 0x7f5e50101d50 2024-06-10T01:08:18.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.711+0000 7f5e45ffb700 1 --2- 172.21.15.7:0/1941423258 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5e380a2c20 0x7f5e380a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:18.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.711+0000 7f5e45ffb700 1 -- 172.21.15.7:0/1941423258 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f5e400cf4a0 con 0x7f5e50101d50 2024-06-10T01:08:18.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.711+0000 7f5e5482e700 1 --2- 172.21.15.7:0/1941423258 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5e380a2c20 0x7f5e380a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:18.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.712+0000 7f5e5482e700 1 --2- 172.21.15.7:0/1941423258 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5e380a2c20 0x7f5e380a50e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c0045b0 tx=0x7f5e4c00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:18.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.712+0000 7f5e61315b80 1 Processor -- start 2024-06-10T01:08:18.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.712+0000 7f5e61315b80 1 -- start start 2024-06-10T01:08:18.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.712+0000 7f5e61315b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591fa14ff60 0x5591fa150340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:18.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.712+0000 7f5e61315b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591fa150880 0x5591fa14a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:18.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.712+0000 7f5e61315b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591fa14a7a0 0x5591fa14abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:18.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.712+0000 7f5e61315b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5591fa03cf60 con 0x5591fa14a7a0 2024-06-10T01:08:18.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.712+0000 7f5e61315b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5591fa09b850 con 0x5591fa14ff60 2024-06-10T01:08:18.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.712+0000 7f5e61315b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5591fa003460 con 0x5591fa150880 2024-06-10T01:08:18.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.712+0000 7f5e5502f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591fa14a7a0 0x5591fa14abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:18.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.712+0000 7f5e5502f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591fa14a7a0 0x5591fa14abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36514/0 (socket says 172.21.15.7:36514) 2024-06-10T01:08:18.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.712+0000 7f5e5502f700 1 -- 172.21.15.7:0/2797542315 learned_addr learned my addr 172.21.15.7:0/2797542315 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:18.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.712+0000 7f5e5482e700 1 --2- 172.21.15.7:0/2797542315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591fa14ff60 0x5591fa150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:18.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.712+0000 7f5e47fff700 1 --2- 172.21.15.7:0/2797542315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591fa150880 0x5591fa14a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:18.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.713+0000 7f5e5502f700 1 -- 172.21.15.7:0/2797542315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591fa150880 msgr2=0x5591fa14a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:18.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.713+0000 7f5e5502f700 1 --2- 172.21.15.7:0/2797542315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591fa150880 0x5591fa14a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.713+0000 7f5e5502f700 1 -- 172.21.15.7:0/2797542315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591fa14ff60 msgr2=0x5591fa150340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:18.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.713+0000 7f5e5502f700 1 --2- 172.21.15.7:0/2797542315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591fa14ff60 0x5591fa150340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.713+0000 7f5e5502f700 1 -- 172.21.15.7:0/2797542315 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591f9fea8d0 con 0x5591fa14a7a0 2024-06-10T01:08:18.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.713+0000 7f5e5502f700 1 --2- 172.21.15.7:0/2797542315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591fa14a7a0 0x5591fa14abc0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5e48009360 tx=0x7f5e48009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dead5e4b156117a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:18.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.713+0000 7f5e36ffd700 1 -- 172.21.15.7:0/2797542315 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e48020070 con 0x5591fa14a7a0 2024-06-10T01:08:18.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.713+0000 7f5e36ffd700 1 -- 172.21.15.7:0/2797542315 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e48006020 con 0x5591fa14a7a0 2024-06-10T01:08:18.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.713+0000 7f5e36ffd700 1 -- 172.21.15.7:0/2797542315 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e48007cf0 con 0x5591fa14a7a0 2024-06-10T01:08:18.712 INFO:tasks.workunit.client.0.smithi007.stdout: 1 81638 7.78 MB/sec warmup 59 sec latency 99.869 ms 2024-06-10T01:08:18.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.714+0000 7f5e61315b80 1 -- 172.21.15.7:0/2797542315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591fa14a7a0 msgr2=0x5591fa14abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:18.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.714+0000 7f5e61315b80 1 --2- 172.21.15.7:0/2797542315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591fa14a7a0 0x5591fa14abc0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5e48009360 tx=0x7f5e48009390 comp rx=0 tx=0).stop 2024-06-10T01:08:18.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.714+0000 7f5e61315b80 1 -- 172.21.15.7:0/2797542315 shutdown_connections 2024-06-10T01:08:18.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.714+0000 7f5e61315b80 1 --2- 172.21.15.7:0/2797542315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591fa14a7a0 0x5591fa14abc0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.714+0000 7f5e61315b80 1 --2- 172.21.15.7:0/2797542315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591fa14ff60 0x5591fa150340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.714+0000 7f5e61315b80 1 --2- 172.21.15.7:0/2797542315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591fa150880 0x5591fa14a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.714+0000 7f5e61315b80 1 -- 172.21.15.7:0/2797542315 >> 172.21.15.7:0/2797542315 conn(0x5591fa077080 msgr2=0x5591fa145220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:18.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.714+0000 7f5e61315b80 1 -- 172.21.15.7:0/2797542315 shutdown_connections 2024-06-10T01:08:18.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.714+0000 7f5e61315b80 1 -- 172.21.15.7:0/2797542315 wait complete. 2024-06-10T01:08:18.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.714+0000 7f5e61315b80 1 -- start start 2024-06-10T01:08:18.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.714+0000 7f5e61315b80 1 Processor -- start 2024-06-10T01:08:18.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.715+0000 7f5e61315b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591fa14a7a0 0x5591fa1565d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:18.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.715+0000 7f5e61315b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591fa14ff60 0x5591fa1538d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:18.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.715+0000 7f5e61315b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591fa150880 0x5591fa153e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:18.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.715+0000 7f5e61315b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5591fa03bfc0 con 0x5591fa14a7a0 2024-06-10T01:08:18.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.715+0000 7f5e61315b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5591fa097810 con 0x5591fa14ff60 2024-06-10T01:08:18.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.715+0000 7f5e61315b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5591fa157200 con 0x5591fa150880 2024-06-10T01:08:18.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.715+0000 7f5e61315b80 10 client.0 _ll_register_callbacks cb 0x5591fa0315b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:18.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.715+0000 7f5e5482e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591fa14a7a0 0x5591fa1565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:18.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.715+0000 7f5e5482e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591fa14a7a0 0x5591fa1565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36518/0 (socket says 172.21.15.7:36518) 2024-06-10T01:08:18.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.715+0000 7f5e5482e700 1 -- 172.21.15.7:0/1554264733 learned_addr learned my addr 172.21.15.7:0/1554264733 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:18.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.715+0000 7f5e47fff700 1 --2- 172.21.15.7:0/1554264733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591fa14ff60 0x5591fa1538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:18.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.715+0000 7f5e5502f700 1 --2- 172.21.15.7:0/1554264733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591fa150880 0x5591fa153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:18.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.715+0000 7f5e5482e700 1 -- 172.21.15.7:0/1554264733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591fa150880 msgr2=0x5591fa153e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:18.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.715+0000 7f5e5482e700 1 --2- 172.21.15.7:0/1554264733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591fa150880 0x5591fa153e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.715+0000 7f5e5482e700 1 -- 172.21.15.7:0/1554264733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591fa14ff60 msgr2=0x5591fa1538d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:18.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.715+0000 7f5e5482e700 1 --2- 172.21.15.7:0/1554264733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591fa14ff60 0x5591fa1538d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.715+0000 7f5e5482e700 1 -- 172.21.15.7:0/1554264733 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591fa061750 con 0x5591fa14a7a0 2024-06-10T01:08:18.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.715+0000 7f5e47fff700 1 --2- 172.21.15.7:0/1554264733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591fa14ff60 0x5591fa1538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:18.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.715+0000 7f5e5482e700 1 --2- 172.21.15.7:0/1554264733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591fa14a7a0 0x5591fa1565d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c003e20 tx=0x7f5e4c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a0876423b2ad30 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:18.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.715+0000 7f5e2ffff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:08:18.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.716+0000 7f5e2ffff700 1 -- 172.21.15.7:0/1554264733 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c011ad0 con 0x5591fa14a7a0 2024-06-10T01:08:18.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.716+0000 7f5e2ffff700 1 -- 172.21.15.7:0/1554264733 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f5e4c011c70 con 0x5591fa14a7a0 2024-06-10T01:08:18.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.716+0000 7f5e61315b80 1 -- 172.21.15.7:0/1554264733 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e48006020 con 0x5591fa14a7a0 2024-06-10T01:08:18.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.716+0000 7f5e2ffff700 1 -- 172.21.15.7:0/1554264733 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5e4c031050 con 0x5591fa14a7a0 2024-06-10T01:08:18.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.716+0000 7f5e61315b80 10 client.16140 fetch_fsmap learned FSMap version 30 2024-06-10T01:08:18.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.716+0000 7f5e61315b80 1 -- 172.21.15.7:0/1554264733 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5591fa161a20 con 0x5591fa14a7a0 2024-06-10T01:08:18.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.716+0000 7f5e2ffff700 1 -- 172.21.15.7:0/1554264733 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e48006020 con 0x5591fa14a7a0 2024-06-10T01:08:18.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.716+0000 7f5e2ffff700 1 -- 172.21.15.7:0/1554264733 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f5e4c01f550 con 0x5591fa14a7a0 2024-06-10T01:08:18.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.716+0000 7f5e61315b80 10 client.16140 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:08:18.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.716+0000 7f5e61315b80 10 client.16140 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:08:18.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.716+0000 7f5e61315b80 20 client.16140 populate_metadata read hostname 'smithi007' 2024-06-10T01:08:18.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.716+0000 7f5e61315b80 1 --2- 172.21.15.7:0/1554264733 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5591fa1640b0 0x5591fa166500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:18.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.716+0000 7f5e61315b80 4 client.16140 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:08:18.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.716+0000 7f5e61315b80 1 -- 172.21.15.7:0/1554264733 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591fa0a8d50 con 0x5591fa1640b0 2024-06-10T01:08:18.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.717+0000 7f5e47fff700 1 --2- 172.21.15.7:0/1554264733 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5591fa1640b0 0x5591fa166500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:18.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.717+0000 7f5e47fff700 1 --2- 172.21.15.7:0/1554264733 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5591fa1640b0 0x5591fa166500 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=af401138f157e66e server_cookie=568323961d1f6ab2 in_seq=0 out_seq=0 2024-06-10T01:08:18.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.717+0000 7f5e2ffff700 10 client.16140 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:08:18.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.718+0000 7f5e2ffff700 1 -- 172.21.15.7:0/1554264733 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5591fa0a8d50 con 0x5591fa1640b0 2024-06-10T01:08:18.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.718+0000 7f5e2ffff700 10 client.16140 handle_command_reply: tid=0 2024-06-10T01:08:18.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.718+0000 7f5e61315b80 1 client.16140 shutdown 2024-06-10T01:08:18.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.718+0000 7f5e61315b80 1 -- 172.21.15.7:0/1554264733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591fa14a7a0 msgr2=0x5591fa1565d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:18.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.718+0000 7f5e61315b80 1 --2- 172.21.15.7:0/1554264733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591fa14a7a0 0x5591fa1565d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c003e20 tx=0x7f5e4c0047c0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.718+0000 7f5e61315b80 1 -- 172.21.15.7:0/1554264733 shutdown_connections 2024-06-10T01:08:18.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.718+0000 7f5e61315b80 1 --2- 172.21.15.7:0/1554264733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591fa150880 0x5591fa153e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.718+0000 7f5e61315b80 1 --2- 172.21.15.7:0/1554264733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591fa14a7a0 0x5591fa1565d0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.718+0000 7f5e61315b80 1 --2- 172.21.15.7:0/1554264733 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5591fa1640b0 0x5591fa166500 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.718+0000 7f5e61315b80 1 --2- 172.21.15.7:0/1554264733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591fa14ff60 0x5591fa1538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.718+0000 7f5e2ffff700 0 client.16140 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:08:18.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.718+0000 7f5e61315b80 1 -- 172.21.15.7:0/1554264733 >> 172.21.15.7:0/1554264733 conn(0x5591fa077080 msgr2=0x5591fa144700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:18.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.718+0000 7f5e61315b80 1 -- 172.21.15.7:0/1554264733 shutdown_connections 2024-06-10T01:08:18.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.718+0000 7f5e61315b80 1 -- 172.21.15.7:0/1554264733 wait complete. 2024-06-10T01:08:18.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.718+0000 7f5e61315b80 20 client.16140 trim_cache size 0 max 16384 2024-06-10T01:08:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.734+0000 7f5e61315b80 1 Processor -- start 2024-06-10T01:08:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.734+0000 7f5e61315b80 1 -- start start 2024-06-10T01:08:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.734+0000 7f5e61315b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591fa14a7a0 0x5591fa1603b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.734+0000 7f5e61315b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591fa14ff60 0x5591fa14a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.734+0000 7f5e61315b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591fa150880 0x5591fa153e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.734+0000 7f5e61315b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c011360 con 0x5591fa14ff60 2024-06-10T01:08:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.734+0000 7f5e61315b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c00eb90 con 0x5591fa150880 2024-06-10T01:08:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.734+0000 7f5e61315b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c00c9e0 con 0x5591fa14a7a0 2024-06-10T01:08:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.734+0000 7f5e47fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591fa14ff60 0x5591fa14a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.734+0000 7f5e47fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591fa14ff60 0x5591fa14a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36522/0 (socket says 172.21.15.7:36522) 2024-06-10T01:08:18.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.734+0000 7f5e47fff700 1 -- 172.21.15.7:0/1763535828 learned_addr learned my addr 172.21.15.7:0/1763535828 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:18.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.734+0000 7f5e5482e700 1 --2- 172.21.15.7:0/1763535828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591fa14a7a0 0x5591fa1603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:18.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.734+0000 7f5e5502f700 1 --2- 172.21.15.7:0/1763535828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591fa150880 0x5591fa153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:18.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.735+0000 7f5e5482e700 1 -- 172.21.15.7:0/1763535828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591fa150880 msgr2=0x5591fa153e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:18.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.735+0000 7f5e5482e700 1 --2- 172.21.15.7:0/1763535828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591fa150880 0x5591fa153e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.735+0000 7f5e5482e700 1 -- 172.21.15.7:0/1763535828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591fa14ff60 msgr2=0x5591fa14a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:18.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.735+0000 7f5e5482e700 1 --2- 172.21.15.7:0/1763535828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591fa14ff60 0x5591fa14a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.735+0000 7f5e5482e700 1 -- 172.21.15.7:0/1763535828 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591fa14ab80 con 0x5591fa14a7a0 2024-06-10T01:08:18.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.735+0000 7f5e47fff700 1 --2- 172.21.15.7:0/1763535828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591fa14ff60 0x5591fa14a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:18.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.735+0000 7f5e5482e700 1 --2- 172.21.15.7:0/1763535828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591fa14a7a0 0x5591fa1603b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c01e020 tx=0x7f5e4c005bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=851d101459a335dd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:18.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.735+0000 7f5e367fc700 1 -- 172.21.15.7:0/1763535828 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c039030 con 0x5591fa14a7a0 2024-06-10T01:08:18.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.735+0000 7f5e367fc700 1 -- 172.21.15.7:0/1763535828 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e4c023070 con 0x5591fa14a7a0 2024-06-10T01:08:18.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.735+0000 7f5e367fc700 1 -- 172.21.15.7:0/1763535828 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c01f650 con 0x5591fa14a7a0 2024-06-10T01:08:18.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.736+0000 7f5e61315b80 1 -- 172.21.15.7:0/1763535828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591fa14a7a0 msgr2=0x5591fa1603b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:18.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.736+0000 7f5e61315b80 1 --2- 172.21.15.7:0/1763535828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591fa14a7a0 0x5591fa1603b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c01e020 tx=0x7f5e4c005bc0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.736+0000 7f5e61315b80 1 -- 172.21.15.7:0/1763535828 shutdown_connections 2024-06-10T01:08:18.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.736+0000 7f5e61315b80 1 --2- 172.21.15.7:0/1763535828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591fa150880 0x5591fa153e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.736+0000 7f5e61315b80 1 --2- 172.21.15.7:0/1763535828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591fa14a7a0 0x5591fa1603b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.736+0000 7f5e61315b80 1 --2- 172.21.15.7:0/1763535828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591fa14ff60 0x5591fa14a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.736+0000 7f5e61315b80 1 -- 172.21.15.7:0/1763535828 >> 172.21.15.7:0/1763535828 conn(0x5591fa077080 msgr2=0x5591fa14dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:18.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.736+0000 7f5e61315b80 1 -- 172.21.15.7:0/1763535828 shutdown_connections 2024-06-10T01:08:18.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.736+0000 7f5e61315b80 1 -- 172.21.15.7:0/1763535828 wait complete. 2024-06-10T01:08:18.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.736+0000 7f5e61315b80 1 -- start start 2024-06-10T01:08:18.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.736+0000 7f5e61315b80 1 Processor -- start 2024-06-10T01:08:18.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.737+0000 7f5e61315b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591fa14a7a0 0x5591fa16c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:18.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.737+0000 7f5e61315b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591fa14ff60 0x5591fa15c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:18.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.737+0000 7f5e61315b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591fa150880 0x5591fa15ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:18.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.737+0000 7f5e61315b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c00d050 con 0x5591fa14a7a0 2024-06-10T01:08:18.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.737+0000 7f5e61315b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c00e4c0 con 0x5591fa150880 2024-06-10T01:08:18.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.737+0000 7f5e61315b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c00d720 con 0x5591fa14ff60 2024-06-10T01:08:18.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.737+0000 7f5e61315b80 10 client.0 _ll_register_callbacks cb 0x5591fa0315b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:18.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.737+0000 7f5e5482e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591fa14a7a0 0x5591fa16c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:18.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.737+0000 7f5e5482e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591fa14a7a0 0x5591fa16c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36532/0 (socket says 172.21.15.7:36532) 2024-06-10T01:08:18.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.737+0000 7f5e5482e700 1 -- 172.21.15.7:0/2066677025 learned_addr learned my addr 172.21.15.7:0/2066677025 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:18.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.737+0000 7f5e47fff700 1 --2- 172.21.15.7:0/2066677025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591fa14ff60 0x5591fa15c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:18.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.737+0000 7f5e5502f700 1 --2- 172.21.15.7:0/2066677025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591fa150880 0x5591fa15ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:18.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.737+0000 7f5e5482e700 1 -- 172.21.15.7:0/2066677025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591fa14ff60 msgr2=0x5591fa15c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:18.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.737+0000 7f5e5482e700 1 --2- 172.21.15.7:0/2066677025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591fa14ff60 0x5591fa15c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.737+0000 7f5e5482e700 1 -- 172.21.15.7:0/2066677025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591fa150880 msgr2=0x5591fa15ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:18.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.737+0000 7f5e5482e700 1 --2- 172.21.15.7:0/2066677025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591fa150880 0x5591fa15ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.737+0000 7f5e5482e700 1 -- 172.21.15.7:0/2066677025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591fa15d130 con 0x5591fa14a7a0 2024-06-10T01:08:18.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.737+0000 7f5e47fff700 1 --2- 172.21.15.7:0/2066677025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591fa14ff60 0x5591fa15c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:18.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.737+0000 7f5e5482e700 1 --2- 172.21.15.7:0/2066677025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591fa14a7a0 0x5591fa16c420 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c005c30 tx=0x7f5e4c01f430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cca0200f16cdf89 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:18.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.737+0000 7f5e2ffff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:08:18.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.737+0000 7f5e2ffff700 1 -- 172.21.15.7:0/2066677025 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c042060 con 0x5591fa14a7a0 2024-06-10T01:08:18.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.737+0000 7f5e61315b80 1 -- 172.21.15.7:0/2066677025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e4c023070 con 0x5591fa14a7a0 2024-06-10T01:08:18.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.737+0000 7f5e2ffff700 1 -- 172.21.15.7:0/2066677025 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f5e4c004710 con 0x5591fa14a7a0 2024-06-10T01:08:18.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.738+0000 7f5e2ffff700 1 -- 172.21.15.7:0/2066677025 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5e4c049050 con 0x5591fa14a7a0 2024-06-10T01:08:18.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.738+0000 7f5e61315b80 10 client.16146 fetch_fsmap learned FSMap version 30 2024-06-10T01:08:18.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.738+0000 7f5e61315b80 1 -- 172.21.15.7:0/2066677025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5591fa1cdea0 con 0x5591fa14a7a0 2024-06-10T01:08:18.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.738+0000 7f5e2ffff700 1 -- 172.21.15.7:0/2066677025 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e4c023070 con 0x5591fa14a7a0 2024-06-10T01:08:18.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.738+0000 7f5e2ffff700 1 -- 172.21.15.7:0/2066677025 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f5e4c005000 con 0x5591fa14a7a0 2024-06-10T01:08:18.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.738+0000 7f5e61315b80 10 client.16146 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:08:18.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.738+0000 7f5e61315b80 10 client.16146 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:08:18.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.738+0000 7f5e61315b80 20 client.16146 populate_metadata read hostname 'smithi007' 2024-06-10T01:08:18.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.738+0000 7f5e61315b80 1 --2- 172.21.15.7:0/2066677025 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5591fa1ce440 0x5591fa1d0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:18.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.738+0000 7f5e61315b80 4 client.16146 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:08:18.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.738+0000 7f5e61315b80 1 -- 172.21.15.7:0/2066677025 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5591fa0a32d0 con 0x5591fa1ce440 2024-06-10T01:08:18.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.738+0000 7f5e47fff700 1 --2- 172.21.15.7:0/2066677025 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5591fa1ce440 0x5591fa1d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:18.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.739+0000 7f5e47fff700 1 --2- 172.21.15.7:0/2066677025 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5591fa1ce440 0x5591fa1d0830 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=32e1b5e13551af2d server_cookie=481cdb589c9701a6 in_seq=0 out_seq=0 2024-06-10T01:08:18.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.739+0000 7f5e2ffff700 10 client.16146 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:08:18.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.739+0000 7f5e2ffff700 1 -- 172.21.15.7:0/2066677025 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5591fa0a32d0 con 0x5591fa1ce440 2024-06-10T01:08:18.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.739+0000 7f5e2ffff700 10 client.16146 handle_command_reply: tid=0 2024-06-10T01:08:18.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.739+0000 7f5e61315b80 1 client.16146 shutdown 2024-06-10T01:08:18.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.739+0000 7f5e61315b80 1 -- 172.21.15.7:0/2066677025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591fa14a7a0 msgr2=0x5591fa16c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:18.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.739+0000 7f5e61315b80 1 --2- 172.21.15.7:0/2066677025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591fa14a7a0 0x5591fa16c420 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c005c30 tx=0x7f5e4c01f430 comp rx=0 tx=0).stop 2024-06-10T01:08:18.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.739+0000 7f5e61315b80 1 -- 172.21.15.7:0/2066677025 shutdown_connections 2024-06-10T01:08:18.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.739+0000 7f5e61315b80 1 --2- 172.21.15.7:0/2066677025 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5591fa1ce440 0x5591fa1d0830 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.739+0000 7f5e61315b80 1 --2- 172.21.15.7:0/2066677025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591fa150880 0x5591fa15ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.739+0000 7f5e61315b80 1 --2- 172.21.15.7:0/2066677025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591fa14a7a0 0x5591fa16c420 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.739+0000 7f5e61315b80 1 --2- 172.21.15.7:0/2066677025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591fa14ff60 0x5591fa15c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.739+0000 7f5e61315b80 1 -- 172.21.15.7:0/2066677025 >> 172.21.15.7:0/2066677025 conn(0x5591fa077080 msgr2=0x5591fa150c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:18.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.739+0000 7f5e2ffff700 0 client.16146 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:08:18.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.739+0000 7f5e61315b80 1 -- 172.21.15.7:0/2066677025 shutdown_connections 2024-06-10T01:08:18.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.739+0000 7f5e61315b80 1 -- 172.21.15.7:0/2066677025 wait complete. 2024-06-10T01:08:18.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.739+0000 7f5e61315b80 20 client.16146 trim_cache size 0 max 16384 2024-06-10T01:08:18.742 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:08:18.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.740+0000 7f5e56a92700 1 -- 172.21.15.7:0/1941423258 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5e380a2c20 msgr2=0x7f5e380a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:18.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.740+0000 7f5e56a92700 1 --2- 172.21.15.7:0/1941423258 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5e380a2c20 0x7f5e380a50e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c0045b0 tx=0x7f5e4c00b3c0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.740+0000 7f5e56a92700 1 -- 172.21.15.7:0/1941423258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e50101d50 msgr2=0x7f5e501035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:18.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.740+0000 7f5e56a92700 1 --2- 172.21.15.7:0/1941423258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e50101d50 0x7f5e501035b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000e9a0 tx=0x7f5e4000ecb0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.740+0000 7f5e56a92700 1 -- 172.21.15.7:0/1941423258 shutdown_connections 2024-06-10T01:08:18.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.740+0000 7f5e56a92700 1 --2- 172.21.15.7:0/1941423258 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5e380a2c20 0x7f5e380a50e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.740+0000 7f5e56a92700 1 --2- 172.21.15.7:0/1941423258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e50107400 0x7f5e50103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.740+0000 7f5e56a92700 1 --2- 172.21.15.7:0/1941423258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e50101390 0x7f5e50103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.740+0000 7f5e56a92700 1 --2- 172.21.15.7:0/1941423258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e50101d50 0x7f5e501035b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.740+0000 7f5e56a92700 1 -- 172.21.15.7:0/1941423258 >> 172.21.15.7:0/1941423258 conn(0x7f5e50078340 msgr2=0x7f5e501005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:18.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.740+0000 7f5e56a92700 1 -- 172.21.15.7:0/1941423258 shutdown_connections 2024-06-10T01:08:18.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:18.740+0000 7f5e56a92700 1 -- 172.21.15.7:0/1941423258 wait complete. 2024-06-10T01:08:19.256 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:18 smithi175 ceph-mon[36636]: pgmap v501: 129 pgs: 129 active+clean; 38 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.4 MiB/s wr, 236 op/s 2024-06-10T01:08:19.256 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:18 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:08:19.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:18 smithi007 ceph-mon[32398]: pgmap v501: 129 pgs: 129 active+clean; 38 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.4 MiB/s wr, 236 op/s 2024-06-10T01:08:19.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:18 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:08:19.331 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:18 smithi078 ceph-mon[36683]: pgmap v501: 129 pgs: 129 active+clean; 38 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.4 MiB/s wr, 236 op/s 2024-06-10T01:08:19.331 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:18 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:08:19.574 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:08:19.710 INFO:tasks.workunit.client.0.smithi007.stdout: 1 81770 7.66 MB/sec warmup 60 sec latency 856.627 ms 2024-06-10T01:08:19.826 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:08:20.710 INFO:tasks.workunit.client.0.smithi007.stdout: 1 81770 7.53 MB/sec warmup 61 sec latency 1856.672 ms 2024-06-10T01:08:21.257 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:20 smithi007 ceph-mon[32398]: pgmap v502: 129 pgs: 129 active+clean; 38 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 377 op/s 2024-06-10T01:08:21.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:20 smithi078 ceph-mon[36683]: pgmap v502: 129 pgs: 129 active+clean; 38 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 377 op/s 2024-06-10T01:08:21.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:20 smithi175 ceph-mon[36636]: pgmap v502: 129 pgs: 129 active+clean; 38 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 377 op/s 2024-06-10T01:08:21.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.538+0000 7f17a4ad2700 1 -- 172.21.15.7:0/977973341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17a0102510 msgr2=0x7f17a0102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:21.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.538+0000 7f17a4ad2700 1 --2- 172.21.15.7:0/977973341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17a0102510 0x7f17a0102990 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1794009990 tx=0x7f1794009ca0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.538+0000 7f17a4ad2700 1 -- 172.21.15.7:0/977973341 shutdown_connections 2024-06-10T01:08:21.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.538+0000 7f17a4ad2700 1 --2- 172.21.15.7:0/977973341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17a0102ed0 0x7f17a010f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.538+0000 7f17a4ad2700 1 --2- 172.21.15.7:0/977973341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17a0108520 0x7f17a0108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.538+0000 7f17a4ad2700 1 --2- 172.21.15.7:0/977973341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17a0102510 0x7f17a0102990 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.538+0000 7f17a4ad2700 1 -- 172.21.15.7:0/977973341 >> 172.21.15.7:0/977973341 conn(0x7f17a0079640 msgr2=0x7f17a0079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:21.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.538+0000 7f17a4ad2700 1 -- 172.21.15.7:0/977973341 shutdown_connections 2024-06-10T01:08:21.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.538+0000 7f17a4ad2700 1 -- 172.21.15.7:0/977973341 wait complete. 2024-06-10T01:08:21.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.539+0000 7f17a4ad2700 1 Processor -- start 2024-06-10T01:08:21.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.539+0000 7f17a4ad2700 1 -- start start 2024-06-10T01:08:21.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.539+0000 7f17a4ad2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17a0102510 0x7f17a0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.539+0000 7f17a4ad2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17a0102ed0 0x7f17a0074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.539+0000 7f17a4ad2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17a0108520 0x7f17a0075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.539+0000 7f17a4ad2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f17a0112870 con 0x7f17a0102ed0 2024-06-10T01:08:21.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.539+0000 7f17a4ad2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f17a01126f0 con 0x7f17a0102510 2024-06-10T01:08:21.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.539+0000 7f17a4ad2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f17a01129f0 con 0x7f17a0108520 2024-06-10T01:08:21.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.540+0000 7f179dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17a0102ed0 0x7f17a0074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.540+0000 7f179dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17a0102ed0 0x7f17a0074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36548/0 (socket says 172.21.15.7:36548) 2024-06-10T01:08:21.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.540+0000 7f179dd9b700 1 -- 172.21.15.7:0/3763163268 learned_addr learned my addr 172.21.15.7:0/3763163268 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:21.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.540+0000 7f179ed9d700 1 --2- 172.21.15.7:0/3763163268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17a0108520 0x7f17a0075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.540+0000 7f179e59c700 1 --2- 172.21.15.7:0/3763163268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17a0102510 0x7f17a0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.540+0000 7f179dd9b700 1 -- 172.21.15.7:0/3763163268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17a0108520 msgr2=0x7f17a0075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:21.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.540+0000 7f179dd9b700 1 --2- 172.21.15.7:0/3763163268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17a0108520 0x7f17a0075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.540+0000 7f179dd9b700 1 -- 172.21.15.7:0/3763163268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17a0102510 msgr2=0x7f17a0074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:21.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.540+0000 7f179dd9b700 1 --2- 172.21.15.7:0/3763163268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17a0102510 0x7f17a0074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.540+0000 7f179dd9b700 1 -- 172.21.15.7:0/3763163268 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a01a7200 con 0x7f17a0102ed0 2024-06-10T01:08:21.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.540+0000 7f179e59c700 1 --2- 172.21.15.7:0/3763163268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17a0102510 0x7f17a0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:08:21.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.540+0000 7f179dd9b700 1 --2- 172.21.15.7:0/3763163268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17a0102ed0 0x7f17a0074cd0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1794005910 tx=0x7f17940047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:21.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.540+0000 7f178f7fe700 1 -- 172.21.15.7:0/3763163268 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1794027070 con 0x7f17a0102ed0 2024-06-10T01:08:21.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.540+0000 7f178f7fe700 1 -- 172.21.15.7:0/3763163268 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1794003780 con 0x7f17a0102ed0 2024-06-10T01:08:21.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.540+0000 7f17a4ad2700 1 -- 172.21.15.7:0/3763163268 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17a01a7490 con 0x7f17a0102ed0 2024-06-10T01:08:21.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.540+0000 7f178f7fe700 1 -- 172.21.15.7:0/3763163268 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1794022da0 con 0x7f17a0102ed0 2024-06-10T01:08:21.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.541+0000 7f17a4ad2700 1 -- 172.21.15.7:0/3763163268 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17a01a78e0 con 0x7f17a0102ed0 2024-06-10T01:08:21.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.544+0000 7f178f7fe700 1 -- 172.21.15.7:0/3763163268 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f179402a070 con 0x7f17a0102ed0 2024-06-10T01:08:21.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.545+0000 7f178f7fe700 1 --2- 172.21.15.7:0/3763163268 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f17880a2c20 0x7f17880a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.545+0000 7f17af355b80 1 Processor -- start 2024-06-10T01:08:21.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.545+0000 7f179e59c700 1 --2- 172.21.15.7:0/3763163268 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f17880a2c20 0x7f17880a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.545+0000 7f178f7fe700 1 -- 172.21.15.7:0/3763163268 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f17940d00c0 con 0x7f17a0102ed0 2024-06-10T01:08:21.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.545+0000 7f17af355b80 1 -- start start 2024-06-10T01:08:21.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.545+0000 7f17af355b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f14942f60 0x555f14943340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.545+0000 7f17af355b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f14943880 0x555f1493d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.545+0000 7f17af355b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555f1493d7a0 0x555f1493dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.545+0000 7f17af355b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555f1482ff60 con 0x555f14942f60 2024-06-10T01:08:21.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.545+0000 7f17af355b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555f1488e850 con 0x555f1493d7a0 2024-06-10T01:08:21.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.545+0000 7f17af355b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555f147f6460 con 0x555f14943880 2024-06-10T01:08:21.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.545+0000 7f179e59c700 1 --2- 172.21.15.7:0/3763163268 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f17880a2c20 0x7f17880a50e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f17900099d0 tx=0x7f1790008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:21.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.545+0000 7f179e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f14942f60 0x555f14943340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.545+0000 7f179e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f14942f60 0x555f14943340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36562/0 (socket says 172.21.15.7:36562) 2024-06-10T01:08:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.545+0000 7f179e59c700 1 -- 172.21.15.7:0/3028605677 learned_addr learned my addr 172.21.15.7:0/3028605677 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.546+0000 7f179ed9d700 1 --2- 172.21.15.7:0/3028605677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555f1493d7a0 0x555f1493dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.546+0000 7f179dd9b700 1 --2- 172.21.15.7:0/3028605677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f14943880 0x555f1493d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.546+0000 7f179e59c700 1 -- 172.21.15.7:0/3028605677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f14943880 msgr2=0x555f1493d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.546+0000 7f179e59c700 1 --2- 172.21.15.7:0/3028605677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f14943880 0x555f1493d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.546+0000 7f179e59c700 1 -- 172.21.15.7:0/3028605677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555f1493d7a0 msgr2=0x555f1493dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.546+0000 7f179e59c700 1 --2- 172.21.15.7:0/3028605677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555f1493d7a0 0x555f1493dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.546+0000 7f179e59c700 1 -- 172.21.15.7:0/3028605677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f147dd8d0 con 0x555f14942f60 2024-06-10T01:08:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.546+0000 7f179e59c700 1 --2- 172.21.15.7:0/3028605677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f14942f60 0x555f14943340 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f179000d7e0 tx=0x7f1790010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1471b06f162623b5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.546+0000 7f178cff9700 1 -- 172.21.15.7:0/3028605677 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1790024070 con 0x555f14942f60 2024-06-10T01:08:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.546+0000 7f178cff9700 1 -- 172.21.15.7:0/3028605677 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1790014ba0 con 0x555f14942f60 2024-06-10T01:08:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.546+0000 7f178cff9700 1 -- 172.21.15.7:0/3028605677 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179001fd30 con 0x555f14942f60 2024-06-10T01:08:21.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.547+0000 7f17af355b80 1 -- 172.21.15.7:0/3028605677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f14942f60 msgr2=0x555f14943340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:21.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.547+0000 7f17af355b80 1 --2- 172.21.15.7:0/3028605677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f14942f60 0x555f14943340 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f179000d7e0 tx=0x7f1790010000 comp rx=0 tx=0).stop 2024-06-10T01:08:21.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.547+0000 7f17af355b80 1 -- 172.21.15.7:0/3028605677 shutdown_connections 2024-06-10T01:08:21.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.547+0000 7f17af355b80 1 --2- 172.21.15.7:0/3028605677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555f1493d7a0 0x555f1493dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.547+0000 7f17af355b80 1 --2- 172.21.15.7:0/3028605677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f14942f60 0x555f14943340 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.547+0000 7f17af355b80 1 --2- 172.21.15.7:0/3028605677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f14943880 0x555f1493d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.547+0000 7f17af355b80 1 -- 172.21.15.7:0/3028605677 >> 172.21.15.7:0/3028605677 conn(0x555f1486a080 msgr2=0x555f14938220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:21.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.547+0000 7f17af355b80 1 -- 172.21.15.7:0/3028605677 shutdown_connections 2024-06-10T01:08:21.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.547+0000 7f17af355b80 1 -- 172.21.15.7:0/3028605677 wait complete. 2024-06-10T01:08:21.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.547+0000 7f17af355b80 1 -- start start 2024-06-10T01:08:21.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.548+0000 7f17af355b80 1 Processor -- start 2024-06-10T01:08:21.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.548+0000 7f17af355b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555f1493d7a0 0x555f149496a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.548+0000 7f17af355b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f14942f60 0x555f149469a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.548+0000 7f17af355b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f14943880 0x555f14946ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.548+0000 7f17af355b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555f1482efc0 con 0x555f14942f60 2024-06-10T01:08:21.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.548+0000 7f17af355b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555f1488a810 con 0x555f1493d7a0 2024-06-10T01:08:21.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.548+0000 7f17af355b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555f1494a260 con 0x555f14943880 2024-06-10T01:08:21.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.548+0000 7f17af355b80 10 client.0 _ll_register_callbacks cb 0x555f148245b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:21.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.548+0000 7f179dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f14942f60 0x555f149469a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.548+0000 7f179dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f14942f60 0x555f149469a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36568/0 (socket says 172.21.15.7:36568) 2024-06-10T01:08:21.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.548+0000 7f179dd9b700 1 -- 172.21.15.7:0/2323694310 learned_addr learned my addr 172.21.15.7:0/2323694310 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:21.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.548+0000 7f179e59c700 1 --2- 172.21.15.7:0/2323694310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555f1493d7a0 0x555f149496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.548+0000 7f179ed9d700 1 --2- 172.21.15.7:0/2323694310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f14943880 0x555f14946ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.548+0000 7f179dd9b700 1 -- 172.21.15.7:0/2323694310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f14943880 msgr2=0x555f14946ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:21.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.548+0000 7f179dd9b700 1 --2- 172.21.15.7:0/2323694310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f14943880 0x555f14946ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.548+0000 7f179dd9b700 1 -- 172.21.15.7:0/2323694310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555f1493d7a0 msgr2=0x555f149496a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:21.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.548+0000 7f179dd9b700 1 --2- 172.21.15.7:0/2323694310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555f1493d7a0 0x555f149496a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.548+0000 7f179dd9b700 1 -- 172.21.15.7:0/2323694310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f14854750 con 0x555f14942f60 2024-06-10T01:08:21.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.548+0000 7f179ed9d700 1 --2- 172.21.15.7:0/2323694310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f14943880 0x555f14946ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:08:21.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.549+0000 7f179dd9b700 1 --2- 172.21.15.7:0/2323694310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f14942f60 0x555f149469a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f179409c560 tx=0x7f179409c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3f754eed60bc15 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:21.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.549+0000 7f1785ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:08:21.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.549+0000 7f1785ffb700 1 -- 172.21.15.7:0/2323694310 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179409ccd0 con 0x555f14942f60 2024-06-10T01:08:21.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.549+0000 7f1785ffb700 1 -- 172.21.15.7:0/2323694310 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f1794022470 con 0x555f14942f60 2024-06-10T01:08:21.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.549+0000 7f17af355b80 1 -- 172.21.15.7:0/2323694310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1790014ba0 con 0x555f14942f60 2024-06-10T01:08:21.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.549+0000 7f1785ffb700 1 -- 172.21.15.7:0/2323694310 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f179409c5d0 con 0x555f14942f60 2024-06-10T01:08:21.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.549+0000 7f17af355b80 10 client.16158 fetch_fsmap learned FSMap version 30 2024-06-10T01:08:21.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.549+0000 7f17af355b80 1 -- 172.21.15.7:0/2323694310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555f14954ac0 con 0x555f14942f60 2024-06-10T01:08:21.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.549+0000 7f1785ffb700 1 -- 172.21.15.7:0/2323694310 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1790014ba0 con 0x555f14942f60 2024-06-10T01:08:21.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.549+0000 7f1785ffb700 1 -- 172.21.15.7:0/2323694310 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f1794085c80 con 0x555f14942f60 2024-06-10T01:08:21.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.549+0000 7f17af355b80 10 client.16158 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:08:21.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.549+0000 7f17af355b80 10 client.16158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:21.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.549+0000 7f17af355b80 20 client.16158 populate_metadata read hostname 'smithi007' 2024-06-10T01:08:21.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.549+0000 7f17af355b80 1 --2- 172.21.15.7:0/2323694310 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555f14957030 0x555f14959420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.549+0000 7f17af355b80 4 client.16158 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:08:21.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.549+0000 7f17af355b80 1 -- 172.21.15.7:0/2323694310 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f1489bd50 con 0x555f14957030 2024-06-10T01:08:21.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.550+0000 7f179e59c700 1 --2- 172.21.15.7:0/2323694310 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555f14957030 0x555f14959420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.550+0000 7f179e59c700 1 --2- 172.21.15.7:0/2323694310 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555f14957030 0x555f14959420 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fee6e95bd8528f4 server_cookie=8716d683bc61a29f in_seq=0 out_seq=0 2024-06-10T01:08:21.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.550+0000 7f1785ffb700 10 client.16158 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:21.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.551+0000 7f1785ffb700 1 -- 172.21.15.7:0/2323694310 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555f1489bd50 con 0x555f14957030 2024-06-10T01:08:21.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.551+0000 7f1785ffb700 10 client.16158 handle_command_reply: tid=0 2024-06-10T01:08:21.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.551+0000 7f17af355b80 1 client.16158 shutdown 2024-06-10T01:08:21.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.551+0000 7f17af355b80 1 -- 172.21.15.7:0/2323694310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f14942f60 msgr2=0x555f149469a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:21.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.551+0000 7f17af355b80 1 --2- 172.21.15.7:0/2323694310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f14942f60 0x555f149469a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f179409c560 tx=0x7f179409c240 comp rx=0 tx=0).stop 2024-06-10T01:08:21.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.551+0000 7f17af355b80 1 -- 172.21.15.7:0/2323694310 shutdown_connections 2024-06-10T01:08:21.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.551+0000 7f17af355b80 1 --2- 172.21.15.7:0/2323694310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f14943880 0x555f14946ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.551+0000 7f17af355b80 1 --2- 172.21.15.7:0/2323694310 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555f14957030 0x555f14959420 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.551+0000 7f17af355b80 1 --2- 172.21.15.7:0/2323694310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555f1493d7a0 0x555f149496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.551+0000 7f17af355b80 1 --2- 172.21.15.7:0/2323694310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f14942f60 0x555f149469a0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.551+0000 7f17af355b80 1 -- 172.21.15.7:0/2323694310 >> 172.21.15.7:0/2323694310 conn(0x555f1486a080 msgr2=0x555f14937700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:21.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.551+0000 7f1785ffb700 0 client.16158 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:21.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.551+0000 7f17af355b80 1 -- 172.21.15.7:0/2323694310 shutdown_connections 2024-06-10T01:08:21.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.551+0000 7f17af355b80 1 -- 172.21.15.7:0/2323694310 wait complete. 2024-06-10T01:08:21.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.551+0000 7f17af355b80 20 client.16158 trim_cache size 0 max 16384 2024-06-10T01:08:21.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.566+0000 7f17af355b80 1 Processor -- start 2024-06-10T01:08:21.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.566+0000 7f17af355b80 1 -- start start 2024-06-10T01:08:21.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.567+0000 7f17af355b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555f1486a080 0x555f149533b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.567+0000 7f17af355b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f1493d7a0 0x555f1493d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.567+0000 7f17af355b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f14942f60 0x555f14944430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.567+0000 7f17af355b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f17a01b2d90 con 0x555f14942f60 2024-06-10T01:08:21.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.567+0000 7f17af355b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f17a01b05c0 con 0x555f1486a080 2024-06-10T01:08:21.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.567+0000 7f17af355b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f17a01ae410 con 0x555f1493d7a0 2024-06-10T01:08:21.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.567+0000 7f179ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f14942f60 0x555f14944430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.567+0000 7f179ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f14942f60 0x555f14944430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36584/0 (socket says 172.21.15.7:36584) 2024-06-10T01:08:21.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.567+0000 7f179ed9d700 1 -- 172.21.15.7:0/454924899 learned_addr learned my addr 172.21.15.7:0/454924899 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:21.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.567+0000 7f179dd9b700 1 --2- 172.21.15.7:0/454924899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f1493d7a0 0x555f1493d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.567+0000 7f179e59c700 1 --2- 172.21.15.7:0/454924899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555f1486a080 0x555f149533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.567+0000 7f179ed9d700 1 -- 172.21.15.7:0/454924899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f1493d7a0 msgr2=0x555f1493d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:21.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.567+0000 7f179ed9d700 1 --2- 172.21.15.7:0/454924899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f1493d7a0 0x555f1493d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.567+0000 7f179ed9d700 1 -- 172.21.15.7:0/454924899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555f1486a080 msgr2=0x555f149533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:21.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.567+0000 7f179ed9d700 1 --2- 172.21.15.7:0/454924899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555f1486a080 0x555f149533b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.567+0000 7f179ed9d700 1 -- 172.21.15.7:0/454924899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f14934810 con 0x555f14942f60 2024-06-10T01:08:21.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.567+0000 7f179e59c700 1 --2- 172.21.15.7:0/454924899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555f1486a080 0x555f149533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:08:21.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.567+0000 7f179dd9b700 1 --2- 172.21.15.7:0/454924899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f1493d7a0 0x555f1493d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:08:21.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.567+0000 7f179ed9d700 1 --2- 172.21.15.7:0/454924899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f14942f60 0x555f14944430 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1798009940 tx=0x7f1798009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c37175db93e3a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:21.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.568+0000 7f1787fff700 1 -- 172.21.15.7:0/454924899 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179801a020 con 0x555f14942f60 2024-06-10T01:08:21.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.568+0000 7f1787fff700 1 -- 172.21.15.7:0/454924899 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1798004860 con 0x555f14942f60 2024-06-10T01:08:21.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.568+0000 7f1787fff700 1 -- 172.21.15.7:0/454924899 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1798016900 con 0x555f14942f60 2024-06-10T01:08:21.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.568+0000 7f17af355b80 1 -- 172.21.15.7:0/454924899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f14942f60 msgr2=0x555f14944430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:21.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.568+0000 7f17af355b80 1 --2- 172.21.15.7:0/454924899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f14942f60 0x555f14944430 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1798009940 tx=0x7f1798009c50 comp rx=0 tx=0).stop 2024-06-10T01:08:21.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.568+0000 7f17af355b80 1 -- 172.21.15.7:0/454924899 shutdown_connections 2024-06-10T01:08:21.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.568+0000 7f17af355b80 1 --2- 172.21.15.7:0/454924899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f14942f60 0x555f14944430 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.568+0000 7f17af355b80 1 --2- 172.21.15.7:0/454924899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555f1486a080 0x555f149533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.568+0000 7f17af355b80 1 --2- 172.21.15.7:0/454924899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f1493d7a0 0x555f1493d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.568+0000 7f17af355b80 1 -- 172.21.15.7:0/454924899 >> 172.21.15.7:0/454924899 conn(0x555f149411a0 msgr2=0x555f14943d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:21.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.568+0000 7f17af355b80 1 -- 172.21.15.7:0/454924899 shutdown_connections 2024-06-10T01:08:21.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.568+0000 7f17af355b80 1 -- 172.21.15.7:0/454924899 wait complete. 2024-06-10T01:08:21.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.569+0000 7f17af355b80 1 -- start start 2024-06-10T01:08:21.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.569+0000 7f17af355b80 1 Processor -- start 2024-06-10T01:08:21.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.569+0000 7f17af355b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f1486a080 0x555f1494f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.569+0000 7f17af355b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555f1493d7a0 0x555f1494fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.569+0000 7f17af355b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f14942f60 0x555f1495dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.569+0000 7f17af355b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f17a01aea80 con 0x555f14942f60 2024-06-10T01:08:21.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.569+0000 7f17af355b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f17a01afef0 con 0x555f1493d7a0 2024-06-10T01:08:21.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.569+0000 7f17af355b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f17a01af150 con 0x555f1486a080 2024-06-10T01:08:21.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.569+0000 7f17af355b80 10 client.0 _ll_register_callbacks cb 0x555f148245b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:21.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.569+0000 7f179ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f14942f60 0x555f1495dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.569+0000 7f179ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f14942f60 0x555f1495dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36594/0 (socket says 172.21.15.7:36594) 2024-06-10T01:08:21.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.569+0000 7f179ed9d700 1 -- 172.21.15.7:0/2850537006 learned_addr learned my addr 172.21.15.7:0/2850537006 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:21.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.569+0000 7f179e59c700 1 --2- 172.21.15.7:0/2850537006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f1486a080 0x555f1494f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.569+0000 7f179dd9b700 1 --2- 172.21.15.7:0/2850537006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555f1493d7a0 0x555f1494fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.569+0000 7f179ed9d700 1 -- 172.21.15.7:0/2850537006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f1486a080 msgr2=0x555f1494f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:21.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.569+0000 7f179ed9d700 1 --2- 172.21.15.7:0/2850537006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f1486a080 0x555f1494f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.569+0000 7f179ed9d700 1 -- 172.21.15.7:0/2850537006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555f1493d7a0 msgr2=0x555f1494fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:21.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.569+0000 7f179ed9d700 1 --2- 172.21.15.7:0/2850537006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555f1493d7a0 0x555f1494fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.569+0000 7f179ed9d700 1 -- 172.21.15.7:0/2850537006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f1495f7d0 con 0x555f14942f60 2024-06-10T01:08:21.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.569+0000 7f179dd9b700 1 --2- 172.21.15.7:0/2850537006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555f1493d7a0 0x555f1494fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:08:21.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.570+0000 7f179ed9d700 1 --2- 172.21.15.7:0/2850537006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f14942f60 0x555f1495dd00 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f179800d010 tx=0x7f1798007d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2500a2d58582fb78 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:21.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.570+0000 7f1785ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:08:21.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.570+0000 7f1785ffb700 1 -- 172.21.15.7:0/2850537006 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1798003c80 con 0x555f14942f60 2024-06-10T01:08:21.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.570+0000 7f17af355b80 1 -- 172.21.15.7:0/2850537006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1798004860 con 0x555f14942f60 2024-06-10T01:08:21.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.570+0000 7f1785ffb700 1 -- 172.21.15.7:0/2850537006 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f17980167b0 con 0x555f14942f60 2024-06-10T01:08:21.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.570+0000 7f1785ffb700 1 -- 172.21.15.7:0/2850537006 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1798031050 con 0x555f14942f60 2024-06-10T01:08:21.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.570+0000 7f17af355b80 10 client.16164 fetch_fsmap learned FSMap version 30 2024-06-10T01:08:21.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.570+0000 7f17af355b80 1 -- 172.21.15.7:0/2850537006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555f149c10b0 con 0x555f14942f60 2024-06-10T01:08:21.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.570+0000 7f1785ffb700 1 -- 172.21.15.7:0/2850537006 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1798004860 con 0x555f14942f60 2024-06-10T01:08:21.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.570+0000 7f1785ffb700 1 -- 172.21.15.7:0/2850537006 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f1798026020 con 0x555f14942f60 2024-06-10T01:08:21.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.570+0000 7f17af355b80 10 client.16164 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:08:21.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.570+0000 7f17af355b80 10 client.16164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:21.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.570+0000 7f17af355b80 20 client.16164 populate_metadata read hostname 'smithi007' 2024-06-10T01:08:21.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.570+0000 7f17af355b80 1 --2- 172.21.15.7:0/2850537006 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555f14943880 0x555f149c37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.570+0000 7f17af355b80 4 client.16164 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:08:21.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.570+0000 7f17af355b80 1 -- 172.21.15.7:0/2850537006 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555f148962d0 con 0x555f14943880 2024-06-10T01:08:21.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.571+0000 7f179e59c700 1 --2- 172.21.15.7:0/2850537006 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555f14943880 0x555f149c37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.571+0000 7f179e59c700 1 --2- 172.21.15.7:0/2850537006 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555f14943880 0x555f149c37a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eeba2dbd473cf84a server_cookie=fb1286600e21cb6c in_seq=0 out_seq=0 2024-06-10T01:08:21.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.571+0000 7f1785ffb700 10 client.16164 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:21.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.571+0000 7f1785ffb700 1 -- 172.21.15.7:0/2850537006 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555f148962d0 con 0x555f14943880 2024-06-10T01:08:21.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.571+0000 7f1785ffb700 10 client.16164 handle_command_reply: tid=0 2024-06-10T01:08:21.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.571+0000 7f17af355b80 1 client.16164 shutdown 2024-06-10T01:08:21.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.571+0000 7f17af355b80 1 -- 172.21.15.7:0/2850537006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f14942f60 msgr2=0x555f1495dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:21.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.571+0000 7f17af355b80 1 --2- 172.21.15.7:0/2850537006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f14942f60 0x555f1495dd00 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f179800d010 tx=0x7f1798007d40 comp rx=0 tx=0).stop 2024-06-10T01:08:21.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.571+0000 7f17af355b80 1 -- 172.21.15.7:0/2850537006 shutdown_connections 2024-06-10T01:08:21.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.571+0000 7f17af355b80 1 --2- 172.21.15.7:0/2850537006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f14942f60 0x555f1495dd00 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.571+0000 7f17af355b80 1 --2- 172.21.15.7:0/2850537006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f1486a080 0x555f1494f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.571+0000 7f17af355b80 1 --2- 172.21.15.7:0/2850537006 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555f14943880 0x555f149c37a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.571+0000 7f17af355b80 1 --2- 172.21.15.7:0/2850537006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555f1493d7a0 0x555f1494fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.571+0000 7f1785ffb700 0 client.16164 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:21.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.571+0000 7f17af355b80 1 -- 172.21.15.7:0/2850537006 >> 172.21.15.7:0/2850537006 conn(0x555f149411a0 msgr2=0x555f14946250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:21.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.572+0000 7f17af355b80 1 -- 172.21.15.7:0/2850537006 shutdown_connections 2024-06-10T01:08:21.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.572+0000 7f17af355b80 1 -- 172.21.15.7:0/2850537006 wait complete. 2024-06-10T01:08:21.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.572+0000 7f17af355b80 20 client.16164 trim_cache size 0 max 16384 2024-06-10T01:08:21.573 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:08:21.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.572+0000 7f17a4ad2700 1 -- 172.21.15.7:0/3763163268 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f17880a2c20 msgr2=0x7f17880a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:21.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.572+0000 7f17a4ad2700 1 --2- 172.21.15.7:0/3763163268 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f17880a2c20 0x7f17880a50e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f17900099d0 tx=0x7f1790008040 comp rx=0 tx=0).stop 2024-06-10T01:08:21.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.572+0000 7f17a4ad2700 1 -- 172.21.15.7:0/3763163268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17a0102ed0 msgr2=0x7f17a0074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:21.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.572+0000 7f17a4ad2700 1 --2- 172.21.15.7:0/3763163268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17a0102ed0 0x7f17a0074cd0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1794005910 tx=0x7f17940047b0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.572+0000 7f17a4ad2700 1 -- 172.21.15.7:0/3763163268 shutdown_connections 2024-06-10T01:08:21.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.572+0000 7f17a4ad2700 1 --2- 172.21.15.7:0/3763163268 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f17880a2c20 0x7f17880a50e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.572+0000 7f17a4ad2700 1 --2- 172.21.15.7:0/3763163268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17a0108520 0x7f17a0075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.572+0000 7f17a4ad2700 1 --2- 172.21.15.7:0/3763163268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17a0102510 0x7f17a0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.572+0000 7f17a4ad2700 1 --2- 172.21.15.7:0/3763163268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17a0102ed0 0x7f17a0074cd0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.572+0000 7f17a4ad2700 1 -- 172.21.15.7:0/3763163268 >> 172.21.15.7:0/3763163268 conn(0x7f17a0079640 msgr2=0x7f17a0101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:21.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.572+0000 7f17a4ad2700 1 -- 172.21.15.7:0/3763163268 shutdown_connections 2024-06-10T01:08:21.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:21.572+0000 7f17a4ad2700 1 -- 172.21.15.7:0/3763163268 wait complete. 2024-06-10T01:08:21.710 INFO:tasks.workunit.client.0.smithi007.stdout: 1 81770 7.41 MB/sec warmup 62 sec latency 2856.762 ms 2024-06-10T01:08:22.710 INFO:tasks.workunit.client.0.smithi007.stdout: 1 81789 7.29 MB/sec warmup 63 sec latency 3346.718 ms 2024-06-10T01:08:23.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:22 smithi007 ceph-mon[32398]: pgmap v503: 129 pgs: 129 active+clean; 34 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 375 op/s 2024-06-10T01:08:23.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:22 smithi078 ceph-mon[36683]: pgmap v503: 129 pgs: 129 active+clean; 34 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 375 op/s 2024-06-10T01:08:23.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:22 smithi175 ceph-mon[36636]: pgmap v503: 129 pgs: 129 active+clean; 34 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 375 op/s 2024-06-10T01:08:23.648 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:08:23.648 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:08:23.710 INFO:tasks.workunit.client.0.smithi007.stdout: 1 81789 7.18 MB/sec warmup 64 sec latency 1489.400 ms 2024-06-10T01:08:23.914 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:08:24.710 INFO:tasks.workunit.client.0.smithi007.stdout: 1 83083 7.17 MB/sec warmup 65 sec latency 1561.178 ms 2024-06-10T01:08:25.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:24 smithi007 ceph-mon[32398]: pgmap v504: 129 pgs: 129 active+clean; 34 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.1 MiB/s wr, 302 op/s 2024-06-10T01:08:25.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:24 smithi078 ceph-mon[36683]: pgmap v504: 129 pgs: 129 active+clean; 34 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.1 MiB/s wr, 302 op/s 2024-06-10T01:08:25.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:24 smithi175 ceph-mon[36636]: pgmap v504: 129 pgs: 129 active+clean; 34 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.1 MiB/s wr, 302 op/s 2024-06-10T01:08:25.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.712+0000 7f6bef098700 1 -- 172.21.15.7:0/3387621373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6be8102740 msgr2=0x7f6be8102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:25.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.712+0000 7f6bef098700 1 --2- 172.21.15.7:0/3387621373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6be8102740 0x7f6be8102b20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc0099c0 tx=0x7f6bdc009cd0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.712+0000 7f6bef098700 1 -- 172.21.15.7:0/3387621373 shutdown_connections 2024-06-10T01:08:25.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.712+0000 7f6bef098700 1 --2- 172.21.15.7:0/3387621373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6be8107840 0x7f6be810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.712+0000 7f6bef098700 1 --2- 172.21.15.7:0/3387621373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6be8102740 0x7f6be8102b20 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.712+0000 7f6bef098700 1 --2- 172.21.15.7:0/3387621373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6be81030f0 0x7f6be8107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.712+0000 7f6bef098700 1 -- 172.21.15.7:0/3387621373 >> 172.21.15.7:0/3387621373 conn(0x7f6be80795f0 msgr2=0x7f6be8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:25.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.712+0000 7f6bef098700 1 -- 172.21.15.7:0/3387621373 shutdown_connections 2024-06-10T01:08:25.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.712+0000 7f6bef098700 1 -- 172.21.15.7:0/3387621373 wait complete. 2024-06-10T01:08:25.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.712+0000 7f6bef098700 1 Processor -- start 2024-06-10T01:08:25.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.712+0000 7f6bef098700 1 -- start start 2024-06-10T01:08:25.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.713+0000 7f6bef098700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6be8102740 0x7f6be819e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:25.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.713+0000 7f6bef098700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6be81030f0 0x7f6be819ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:25.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.713+0000 7f6bef098700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6be8107840 0x7f6be8198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:25.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.713+0000 7f6bef098700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6be81127b0 con 0x7f6be8102740 2024-06-10T01:08:25.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.713+0000 7f6bef098700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6be8112630 con 0x7f6be8107840 2024-06-10T01:08:25.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.713+0000 7f6bef098700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6be8112930 con 0x7f6be81030f0 2024-06-10T01:08:25.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.713+0000 7f6bece34700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6be8102740 0x7f6be819e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:25.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.713+0000 7f6bece34700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6be8102740 0x7f6be819e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54072/0 (socket says 172.21.15.7:54072) 2024-06-10T01:08:25.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.713+0000 7f6bece34700 1 -- 172.21.15.7:0/1473959780 learned_addr learned my addr 172.21.15.7:0/1473959780 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:25.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.713+0000 7f6bed635700 1 --2- 172.21.15.7:0/1473959780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6be8107840 0x7f6be8198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:25.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.713+0000 7f6be7fff700 1 --2- 172.21.15.7:0/1473959780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6be81030f0 0x7f6be819ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:25.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.713+0000 7f6bece34700 1 -- 172.21.15.7:0/1473959780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6be81030f0 msgr2=0x7f6be819ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:25.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.713+0000 7f6bece34700 1 --2- 172.21.15.7:0/1473959780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6be81030f0 0x7f6be819ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.713+0000 7f6bece34700 1 -- 172.21.15.7:0/1473959780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6be8107840 msgr2=0x7f6be8198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:25.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.713+0000 7f6bece34700 1 --2- 172.21.15.7:0/1473959780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6be8107840 0x7f6be8198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.713+0000 7f6bece34700 1 -- 172.21.15.7:0/1473959780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be8199210 con 0x7f6be8102740 2024-06-10T01:08:25.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.713+0000 7f6bece34700 1 --2- 172.21.15.7:0/1473959780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6be8102740 0x7f6be819e8a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc005910 tx=0x7f6bdc00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:25.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.713+0000 7f6be5ffb700 1 -- 172.21.15.7:0/1473959780 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc027070 con 0x7f6be8102740 2024-06-10T01:08:25.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.713+0000 7f6bef098700 1 -- 172.21.15.7:0/1473959780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6be81994a0 con 0x7f6be8102740 2024-06-10T01:08:25.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.714+0000 7f6be5ffb700 1 -- 172.21.15.7:0/1473959780 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bdc003680 con 0x7f6be8102740 2024-06-10T01:08:25.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.714+0000 7f6be5ffb700 1 -- 172.21.15.7:0/1473959780 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc0173f0 con 0x7f6be8102740 2024-06-10T01:08:25.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.714+0000 7f6bef098700 1 -- 172.21.15.7:0/1473959780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6be81a5770 con 0x7f6be8102740 2024-06-10T01:08:25.712 INFO:tasks.workunit.client.0.smithi007.stdout: 1 85318 7.25 MB/sec warmup 66 sec latency 325.029 ms 2024-06-10T01:08:25.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.716+0000 7f6be5ffb700 1 -- 172.21.15.7:0/1473959780 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f6bdc022470 con 0x7f6be8102740 2024-06-10T01:08:25.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.717+0000 7f6be5ffb700 1 --2- 172.21.15.7:0/1473959780 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6bd80a2ca0 0x7f6bd80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:25.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.717+0000 7f6be5ffb700 1 -- 172.21.15.7:0/1473959780 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f6bdc0cfff0 con 0x7f6be8102740 2024-06-10T01:08:25.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.717+0000 7f6be7fff700 1 --2- 172.21.15.7:0/1473959780 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6bd80a2ca0 0x7f6bd80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:25.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.717+0000 7f6be7fff700 1 --2- 172.21.15.7:0/1473959780 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6bd80a2ca0 0x7f6bd80a5160 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4005950 tx=0x7f6bd40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:25.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.718+0000 7f6bf991bb80 1 Processor -- start 2024-06-10T01:08:25.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.718+0000 7f6bf991bb80 1 -- start start 2024-06-10T01:08:25.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.718+0000 7f6bf991bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561a8c07d4f0 0x561a8c07d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:25.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.718+0000 7f6bf991bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a8c0776f0 0x561a8c077b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:25.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.718+0000 7f6bf991bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a8c0780b0 0x561a8c084290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:25.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.718+0000 7f6bf991bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561a8bf65fe0 con 0x561a8c07d4f0 2024-06-10T01:08:25.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.718+0000 7f6bf991bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561a8bfc48d0 con 0x561a8c0780b0 2024-06-10T01:08:25.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.718+0000 7f6bf991bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561a8bf2c4e0 con 0x561a8c0776f0 2024-06-10T01:08:25.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.718+0000 7f6bece34700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561a8c07d4f0 0x561a8c07d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:25.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.718+0000 7f6bece34700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561a8c07d4f0 0x561a8c07d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54086/0 (socket says 172.21.15.7:54086) 2024-06-10T01:08:25.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.718+0000 7f6bece34700 1 -- 172.21.15.7:0/3500618927 learned_addr learned my addr 172.21.15.7:0/3500618927 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:25.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.719+0000 7f6bed635700 1 --2- 172.21.15.7:0/3500618927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a8c0780b0 0x561a8c084290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:25.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.719+0000 7f6be7fff700 1 --2- 172.21.15.7:0/3500618927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a8c0776f0 0x561a8c077b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:25.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.719+0000 7f6bece34700 1 -- 172.21.15.7:0/3500618927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a8c0776f0 msgr2=0x561a8c077b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:25.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.719+0000 7f6bece34700 1 --2- 172.21.15.7:0/3500618927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a8c0776f0 0x561a8c077b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.719+0000 7f6bece34700 1 -- 172.21.15.7:0/3500618927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a8c0780b0 msgr2=0x561a8c084290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:25.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.719+0000 7f6bece34700 1 --2- 172.21.15.7:0/3500618927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a8c0780b0 0x561a8c084290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.719+0000 7f6bece34700 1 -- 172.21.15.7:0/3500618927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a8bf13950 con 0x561a8c07d4f0 2024-06-10T01:08:25.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.719+0000 7f6bece34700 1 --2- 172.21.15.7:0/3500618927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561a8c07d4f0 0x561a8c07d8d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc024000 tx=0x7f6bdc09c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cea2d7010f2c7de server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:25.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.719+0000 7f6bd2ffd700 1 -- 172.21.15.7:0/3500618927 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc09c560 con 0x561a8c07d4f0 2024-06-10T01:08:25.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.719+0000 7f6bd2ffd700 1 -- 172.21.15.7:0/3500618927 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bdc0a0910 con 0x561a8c07d4f0 2024-06-10T01:08:25.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.719+0000 7f6bd2ffd700 1 -- 172.21.15.7:0/3500618927 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc09c9c0 con 0x561a8c07d4f0 2024-06-10T01:08:25.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.720+0000 7f6bf991bb80 1 -- 172.21.15.7:0/3500618927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561a8c07d4f0 msgr2=0x561a8c07d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:25.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.720+0000 7f6bf991bb80 1 --2- 172.21.15.7:0/3500618927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561a8c07d4f0 0x561a8c07d8d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc024000 tx=0x7f6bdc09c400 comp rx=0 tx=0).stop 2024-06-10T01:08:25.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.720+0000 7f6bf991bb80 1 -- 172.21.15.7:0/3500618927 shutdown_connections 2024-06-10T01:08:25.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.720+0000 7f6bf991bb80 1 --2- 172.21.15.7:0/3500618927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a8c0780b0 0x561a8c084290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.720+0000 7f6bf991bb80 1 --2- 172.21.15.7:0/3500618927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561a8c07d4f0 0x561a8c07d8d0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.720+0000 7f6bf991bb80 1 --2- 172.21.15.7:0/3500618927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a8c0776f0 0x561a8c077b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.720+0000 7f6bf991bb80 1 -- 172.21.15.7:0/3500618927 >> 172.21.15.7:0/3500618927 conn(0x561a8bfa0100 msgr2=0x561a8c06e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:25.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.720+0000 7f6bf991bb80 1 -- 172.21.15.7:0/3500618927 shutdown_connections 2024-06-10T01:08:25.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.720+0000 7f6bf991bb80 1 -- 172.21.15.7:0/3500618927 wait complete. 2024-06-10T01:08:25.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.720+0000 7f6bf991bb80 1 -- start start 2024-06-10T01:08:25.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.720+0000 7f6bf991bb80 1 Processor -- start 2024-06-10T01:08:25.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.721+0000 7f6bf991bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a8c0776f0 0x561a8c086ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:25.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.721+0000 7f6bf991bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561a8c0780b0 0x561a8c087400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:25.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.721+0000 7f6bf991bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a8c07d4f0 0x561a8c081240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:25.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.721+0000 7f6bf991bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561a8bf65040 con 0x561a8c0780b0 2024-06-10T01:08:25.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.721+0000 7f6bf991bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561a8bfc0890 con 0x561a8c07d4f0 2024-06-10T01:08:25.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.721+0000 7f6bf991bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561a8c07ded0 con 0x561a8c0776f0 2024-06-10T01:08:25.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.721+0000 7f6bf991bb80 10 client.0 _ll_register_callbacks cb 0x561a8bf5a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:25.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.721+0000 7f6be7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561a8c0780b0 0x561a8c087400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:25.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.721+0000 7f6be7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561a8c0780b0 0x561a8c087400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54098/0 (socket says 172.21.15.7:54098) 2024-06-10T01:08:25.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.721+0000 7f6be7fff700 1 -- 172.21.15.7:0/3387301604 learned_addr learned my addr 172.21.15.7:0/3387301604 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:25.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.721+0000 7f6bece34700 1 --2- 172.21.15.7:0/3387301604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a8c0776f0 0x561a8c086ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:25.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.721+0000 7f6bed635700 1 --2- 172.21.15.7:0/3387301604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a8c07d4f0 0x561a8c081240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:25.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.721+0000 7f6be7fff700 1 -- 172.21.15.7:0/3387301604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a8c0776f0 msgr2=0x561a8c086ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:25.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.721+0000 7f6be7fff700 1 --2- 172.21.15.7:0/3387301604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a8c0776f0 0x561a8c086ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.721+0000 7f6be7fff700 1 -- 172.21.15.7:0/3387301604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a8c07d4f0 msgr2=0x561a8c081240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:25.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.721+0000 7f6be7fff700 1 --2- 172.21.15.7:0/3387301604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a8c07d4f0 0x561a8c081240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.721+0000 7f6be7fff700 1 -- 172.21.15.7:0/3387301604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a8bf8a7d0 con 0x561a8c0780b0 2024-06-10T01:08:25.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.721+0000 7f6be7fff700 1 --2- 172.21.15.7:0/3387301604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561a8c0780b0 0x561a8c087400 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400fe60 tx=0x7f6bd4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f583cfd36ef1fb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:25.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.721+0000 7f6bc7fff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:08:25.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.722+0000 7f6bc7fff700 1 -- 172.21.15.7:0/3387301604 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd401f430 con 0x561a8c0780b0 2024-06-10T01:08:25.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.722+0000 7f6bf991bb80 1 -- 172.21.15.7:0/3387301604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bdc0a0910 con 0x561a8c0780b0 2024-06-10T01:08:25.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.722+0000 7f6bc7fff700 1 -- 172.21.15.7:0/3387301604 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f6bd401f5d0 con 0x561a8c0780b0 2024-06-10T01:08:25.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.722+0000 7f6bc7fff700 1 -- 172.21.15.7:0/3387301604 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6bd4030050 con 0x561a8c0780b0 2024-06-10T01:08:25.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.722+0000 7f6bf991bb80 10 client.16176 fetch_fsmap learned FSMap version 30 2024-06-10T01:08:25.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.722+0000 7f6bf991bb80 1 -- 172.21.15.7:0/3387301604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561a8c08ec80 con 0x561a8c0780b0 2024-06-10T01:08:25.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.722+0000 7f6bc7fff700 1 -- 172.21.15.7:0/3387301604 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bdc0a0910 con 0x561a8c0780b0 2024-06-10T01:08:25.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.722+0000 7f6bc7fff700 1 -- 172.21.15.7:0/3387301604 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f6bd4027840 con 0x561a8c0780b0 2024-06-10T01:08:25.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.722+0000 7f6bf991bb80 10 client.16176 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:08:25.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.722+0000 7f6bf991bb80 10 client.16176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:25.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.722+0000 7f6bf991bb80 20 client.16176 populate_metadata read hostname 'smithi007' 2024-06-10T01:08:25.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.722+0000 7f6bf991bb80 1 --2- 172.21.15.7:0/3387301604 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x561a8c091310 0x561a8c093730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:25.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.722+0000 7f6bf991bb80 4 client.16176 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:08:25.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.722+0000 7f6bf991bb80 1 -- 172.21.15.7:0/3387301604 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a8bfd1dd0 con 0x561a8c091310 2024-06-10T01:08:25.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.723+0000 7f6bece34700 1 --2- 172.21.15.7:0/3387301604 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x561a8c091310 0x561a8c093730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:25.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.723+0000 7f6bece34700 1 --2- 172.21.15.7:0/3387301604 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x561a8c091310 0x561a8c093730 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b29ed6f4ea014d7 server_cookie=a544ca5879707dcc in_seq=0 out_seq=0 2024-06-10T01:08:25.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.723+0000 7f6bc7fff700 10 client.16176 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:25.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.724+0000 7f6bc7fff700 1 -- 172.21.15.7:0/3387301604 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561a8bfd1dd0 con 0x561a8c091310 2024-06-10T01:08:25.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.724+0000 7f6bc7fff700 10 client.16176 handle_command_reply: tid=0 2024-06-10T01:08:25.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.724+0000 7f6bf991bb80 1 client.16176 shutdown 2024-06-10T01:08:25.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.724+0000 7f6bf991bb80 1 -- 172.21.15.7:0/3387301604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561a8c0780b0 msgr2=0x561a8c087400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:25.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.724+0000 7f6bf991bb80 1 --2- 172.21.15.7:0/3387301604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561a8c0780b0 0x561a8c087400 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400fe60 tx=0x7f6bd4004750 comp rx=0 tx=0).stop 2024-06-10T01:08:25.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.724+0000 7f6bf991bb80 1 -- 172.21.15.7:0/3387301604 shutdown_connections 2024-06-10T01:08:25.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.724+0000 7f6bf991bb80 1 --2- 172.21.15.7:0/3387301604 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x561a8c091310 0x561a8c093730 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.724+0000 7f6bf991bb80 1 --2- 172.21.15.7:0/3387301604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a8c07d4f0 0x561a8c081240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.724+0000 7f6bf991bb80 1 --2- 172.21.15.7:0/3387301604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a8c0776f0 0x561a8c086ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.724+0000 7f6bf991bb80 1 --2- 172.21.15.7:0/3387301604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561a8c0780b0 0x561a8c087400 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.724+0000 7f6bc7fff700 0 client.16176 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:25.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.724+0000 7f6bf991bb80 1 -- 172.21.15.7:0/3387301604 >> 172.21.15.7:0/3387301604 conn(0x561a8bfa0100 msgr2=0x561a8c06d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:25.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.724+0000 7f6bf991bb80 1 -- 172.21.15.7:0/3387301604 shutdown_connections 2024-06-10T01:08:25.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.724+0000 7f6bf991bb80 1 -- 172.21.15.7:0/3387301604 wait complete. 2024-06-10T01:08:25.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.724+0000 7f6bf991bb80 20 client.16176 trim_cache size 0 max 16384 2024-06-10T01:08:25.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.745+0000 7f6bf991bb80 1 Processor -- start 2024-06-10T01:08:25.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.745+0000 7f6bf991bb80 1 -- start start 2024-06-10T01:08:25.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.745+0000 7f6bf991bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a8c0776f0 0x561a8c072b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:25.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.745+0000 7f6bf991bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561a8c0780b0 0x561a8c06df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:25.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.745+0000 7f6bf991bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a8c07d4f0 0x561a8c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:25.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.745+0000 7f6bf991bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6be81ae940 con 0x561a8c0780b0 2024-06-10T01:08:25.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.745+0000 7f6bf991bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6be81ac170 con 0x561a8c07d4f0 2024-06-10T01:08:25.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.745+0000 7f6bf991bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6be81a9fe0 con 0x561a8c0776f0 2024-06-10T01:08:25.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.745+0000 7f6be7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561a8c0780b0 0x561a8c06df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:25.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.745+0000 7f6be7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561a8c0780b0 0x561a8c06df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54104/0 (socket says 172.21.15.7:54104) 2024-06-10T01:08:25.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.745+0000 7f6be7fff700 1 -- 172.21.15.7:0/1278519184 learned_addr learned my addr 172.21.15.7:0/1278519184 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:25.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.745+0000 7f6bed635700 1 --2- 172.21.15.7:0/1278519184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a8c07d4f0 0x561a8c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:25.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.745+0000 7f6be7fff700 1 -- 172.21.15.7:0/1278519184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a8c0776f0 msgr2=0x561a8c072b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:25.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.745+0000 7f6be7fff700 1 --2- 172.21.15.7:0/1278519184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a8c0776f0 0x561a8c072b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.745+0000 7f6be7fff700 1 -- 172.21.15.7:0/1278519184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a8c07d4f0 msgr2=0x561a8c06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:25.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.745+0000 7f6be7fff700 1 --2- 172.21.15.7:0/1278519184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a8c07d4f0 0x561a8c06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.745+0000 7f6be7fff700 1 -- 172.21.15.7:0/1278519184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a8c093d90 con 0x561a8c0780b0 2024-06-10T01:08:25.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.746+0000 7f6be7fff700 1 --2- 172.21.15.7:0/1278519184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561a8c0780b0 0x561a8c06df00 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400b570 tx=0x7f6bd400f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab09539a367cb525 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:25.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.746+0000 7f6bd27fc700 1 -- 172.21.15.7:0/1278519184 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4011a00 con 0x561a8c0780b0 2024-06-10T01:08:25.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.746+0000 7f6bd27fc700 1 -- 172.21.15.7:0/1278519184 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bd4038030 con 0x561a8c0780b0 2024-06-10T01:08:25.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.746+0000 7f6bd27fc700 1 -- 172.21.15.7:0/1278519184 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd401f9f0 con 0x561a8c0780b0 2024-06-10T01:08:25.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.746+0000 7f6bf991bb80 1 -- 172.21.15.7:0/1278519184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561a8c0780b0 msgr2=0x561a8c06df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:25.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.746+0000 7f6bf991bb80 1 --2- 172.21.15.7:0/1278519184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561a8c0780b0 0x561a8c06df00 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400b570 tx=0x7f6bd400f770 comp rx=0 tx=0).stop 2024-06-10T01:08:25.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.747+0000 7f6bf991bb80 1 -- 172.21.15.7:0/1278519184 shutdown_connections 2024-06-10T01:08:25.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.747+0000 7f6bf991bb80 1 --2- 172.21.15.7:0/1278519184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a8c07d4f0 0x561a8c06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.747+0000 7f6bf991bb80 1 --2- 172.21.15.7:0/1278519184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a8c0776f0 0x561a8c072b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.747+0000 7f6bf991bb80 1 --2- 172.21.15.7:0/1278519184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561a8c0780b0 0x561a8c06df00 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.747+0000 7f6bf991bb80 1 -- 172.21.15.7:0/1278519184 >> 172.21.15.7:0/1278519184 conn(0x561a8bfa0100 msgr2=0x561a8c081160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.747+0000 7f6bf991bb80 1 -- 172.21.15.7:0/1278519184 shutdown_connections 2024-06-10T01:08:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.747+0000 7f6bf991bb80 1 -- 172.21.15.7:0/1278519184 wait complete. 2024-06-10T01:08:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.747+0000 7f6bf991bb80 1 -- start start 2024-06-10T01:08:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.747+0000 7f6bf991bb80 1 Processor -- start 2024-06-10T01:08:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.747+0000 7f6bf991bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a8c0776f0 0x561a8c08d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.747+0000 7f6bf991bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a8c0780b0 0x561a8c08bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.747+0000 7f6bf991bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561a8c07d4f0 0x561a8c08c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.747+0000 7f6bf991bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6be81aa630 con 0x561a8c07d4f0 2024-06-10T01:08:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.748+0000 7f6bf991bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6be81abaa0 con 0x561a8c0780b0 2024-06-10T01:08:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.748+0000 7f6bf991bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6be81aad00 con 0x561a8c0776f0 2024-06-10T01:08:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.748+0000 7f6bece34700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a8c0776f0 0x561a8c08d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.748+0000 7f6bf991bb80 10 client.0 _ll_register_callbacks cb 0x561a8bf5a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.748+0000 7f6bece34700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a8c0776f0 0x561a8c08d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34544/0 (socket says 172.21.15.7:34544) 2024-06-10T01:08:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.748+0000 7f6bece34700 1 -- 172.21.15.7:0/666726017 learned_addr learned my addr 172.21.15.7:0/666726017 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.748+0000 7f6be7fff700 1 --2- 172.21.15.7:0/666726017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a8c0780b0 0x561a8c08bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.748+0000 7f6bed635700 1 --2- 172.21.15.7:0/666726017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561a8c07d4f0 0x561a8c08c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.748+0000 7f6bece34700 1 -- 172.21.15.7:0/666726017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a8c0780b0 msgr2=0x561a8c08bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.748+0000 7f6bece34700 1 --2- 172.21.15.7:0/666726017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a8c0780b0 0x561a8c08bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.748+0000 7f6bece34700 1 -- 172.21.15.7:0/666726017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561a8c07d4f0 msgr2=0x561a8c08c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.748+0000 7f6bece34700 1 --2- 172.21.15.7:0/666726017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561a8c07d4f0 0x561a8c08c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.748+0000 7f6bece34700 1 -- 172.21.15.7:0/666726017 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a8c08c740 con 0x561a8c0776f0 2024-06-10T01:08:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.748+0000 7f6bece34700 1 --2- 172.21.15.7:0/666726017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a8c0776f0 0x561a8c08d810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc0cdae0 tx=0x7f6bdc09c150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b9335628ce09cf9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.748+0000 7f6bc7fff700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:08:25.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.748+0000 7f6bc7fff700 1 -- 172.21.15.7:0/666726017 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc0b4070 con 0x561a8c0776f0 2024-06-10T01:08:25.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.748+0000 7f6bf991bb80 1 -- 172.21.15.7:0/666726017 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bd4038030 con 0x561a8c0776f0 2024-06-10T01:08:25.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.748+0000 7f6bc7fff700 1 -- 172.21.15.7:0/666726017 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f6bdc0227e0 con 0x561a8c0776f0 2024-06-10T01:08:25.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.748+0000 7f6bc7fff700 1 -- 172.21.15.7:0/666726017 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6bdc0a0910 con 0x561a8c0776f0 2024-06-10T01:08:25.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.748+0000 7f6bf991bb80 10 client.25735 fetch_fsmap learned FSMap version 30 2024-06-10T01:08:25.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.749+0000 7f6bf991bb80 1 -- 172.21.15.7:0/666726017 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561a8c0fce60 con 0x561a8c0776f0 2024-06-10T01:08:25.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.749+0000 7f6bc7fff700 1 -- 172.21.15.7:0/666726017 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bd4038030 con 0x561a8c0776f0 2024-06-10T01:08:25.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.749+0000 7f6bc7fff700 1 -- 172.21.15.7:0/666726017 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f6bdc084030 con 0x561a8c0776f0 2024-06-10T01:08:25.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.749+0000 7f6bf991bb80 10 client.25735 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:08:25.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.749+0000 7f6bf991bb80 10 client.25735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:25.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.749+0000 7f6bf991bb80 20 client.25735 populate_metadata read hostname 'smithi007' 2024-06-10T01:08:25.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.749+0000 7f6bf991bb80 1 --2- 172.21.15.7:0/666726017 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x561a8c0fd390 0x561a8c0ff7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:25.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.749+0000 7f6bf991bb80 4 client.25735 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:08:25.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.749+0000 7f6bf991bb80 1 -- 172.21.15.7:0/666726017 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561a8bfcc350 con 0x561a8c0fd390 2024-06-10T01:08:25.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.749+0000 7f6be7fff700 1 --2- 172.21.15.7:0/666726017 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x561a8c0fd390 0x561a8c0ff7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:25.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.750+0000 7f6be7fff700 1 --2- 172.21.15.7:0/666726017 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x561a8c0fd390 0x561a8c0ff7b0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f109c3a5abf6984 server_cookie=e445edadc4da38f0 in_seq=0 out_seq=0 2024-06-10T01:08:25.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.750+0000 7f6bc7fff700 10 client.25735 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:25.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.750+0000 7f6bc7fff700 1 -- 172.21.15.7:0/666726017 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561a8bfcc350 con 0x561a8c0fd390 2024-06-10T01:08:25.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.750+0000 7f6bc7fff700 10 client.25735 handle_command_reply: tid=0 2024-06-10T01:08:25.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.750+0000 7f6bf991bb80 1 client.25735 shutdown 2024-06-10T01:08:25.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.750+0000 7f6bf991bb80 1 -- 172.21.15.7:0/666726017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a8c0776f0 msgr2=0x561a8c08d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:25.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.750+0000 7f6bf991bb80 1 --2- 172.21.15.7:0/666726017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a8c0776f0 0x561a8c08d810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc0cdae0 tx=0x7f6bdc09c150 comp rx=0 tx=0).stop 2024-06-10T01:08:25.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.750+0000 7f6bf991bb80 1 -- 172.21.15.7:0/666726017 shutdown_connections 2024-06-10T01:08:25.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.750+0000 7f6bf991bb80 1 --2- 172.21.15.7:0/666726017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561a8c07d4f0 0x561a8c08c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.750+0000 7f6bf991bb80 1 --2- 172.21.15.7:0/666726017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a8c0776f0 0x561a8c08d810 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.750+0000 7f6bf991bb80 1 --2- 172.21.15.7:0/666726017 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x561a8c0fd390 0x561a8c0ff7b0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.750+0000 7f6bf991bb80 1 --2- 172.21.15.7:0/666726017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a8c0780b0 0x561a8c08bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.750+0000 7f6bc7fff700 0 client.25735 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:25.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.750+0000 7f6bf991bb80 1 -- 172.21.15.7:0/666726017 >> 172.21.15.7:0/666726017 conn(0x561a8bfa0100 msgr2=0x561a8c072b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:25.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.750+0000 7f6bf991bb80 1 -- 172.21.15.7:0/666726017 shutdown_connections 2024-06-10T01:08:25.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.751+0000 7f6bf991bb80 1 -- 172.21.15.7:0/666726017 wait complete. 2024-06-10T01:08:25.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.751+0000 7f6bf991bb80 20 client.25735 trim_cache size 0 max 16384 2024-06-10T01:08:25.753 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:08:25.753 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-10T01:08:25.753 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "9b79200e-fdca-426f-a620-5d560c26f15c", 2024-06-10T01:08:25.753 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-10T01:08:25.753 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:08:25.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.751+0000 7f6bef098700 1 -- 172.21.15.7:0/1473959780 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6bd80a2ca0 msgr2=0x7f6bd80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:25.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.751+0000 7f6bef098700 1 --2- 172.21.15.7:0/1473959780 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6bd80a2ca0 0x7f6bd80a5160 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4005950 tx=0x7f6bd40058e0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.751+0000 7f6bef098700 1 -- 172.21.15.7:0/1473959780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6be8102740 msgr2=0x7f6be819e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:25.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.751+0000 7f6bef098700 1 --2- 172.21.15.7:0/1473959780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6be8102740 0x7f6be819e8a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc005910 tx=0x7f6bdc00fa00 comp rx=0 tx=0).stop 2024-06-10T01:08:25.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.751+0000 7f6bef098700 1 -- 172.21.15.7:0/1473959780 shutdown_connections 2024-06-10T01:08:25.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.751+0000 7f6bef098700 1 --2- 172.21.15.7:0/1473959780 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6bd80a2ca0 0x7f6bd80a5160 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.751+0000 7f6bef098700 1 --2- 172.21.15.7:0/1473959780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6be8107840 0x7f6be8198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.751+0000 7f6bef098700 1 --2- 172.21.15.7:0/1473959780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6be8102740 0x7f6be819e8a0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.751+0000 7f6bef098700 1 --2- 172.21.15.7:0/1473959780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6be81030f0 0x7f6be819ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:25.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.751+0000 7f6bef098700 1 -- 172.21.15.7:0/1473959780 >> 172.21.15.7:0/1473959780 conn(0x7f6be80795f0 msgr2=0x7f6be8108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:25.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.752+0000 7f6bef098700 1 -- 172.21.15.7:0/1473959780 shutdown_connections 2024-06-10T01:08:25.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:25.752+0000 7f6bef098700 1 -- 172.21.15.7:0/1473959780 wait complete. 2024-06-10T01:08:26.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:25 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:08:26.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:25 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:08:26.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:25 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:08:26.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:25 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:08:26.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:25 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:08:26.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:25 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:08:26.592 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:08:26.710 INFO:tasks.workunit.client.0.smithi007.stdout: 1 85337 7.14 MB/sec warmup 67 sec latency 928.988 ms 2024-06-10T01:08:26.845 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:08:27.218 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:26 smithi007 ceph-mon[32398]: pgmap v505: 129 pgs: 129 active+clean; 35 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 307 op/s 2024-06-10T01:08:27.219 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:26 smithi007 ceph-mon[32398]: scrub queued for path: / 2024-06-10T01:08:27.219 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:26 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:08:27.219 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:26 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:08:27.219 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:26 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:08:27.219 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:26 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:08:27.219 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:26 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:08:27.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:26 smithi078 ceph-mon[36683]: pgmap v505: 129 pgs: 129 active+clean; 35 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 307 op/s 2024-06-10T01:08:27.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:26 smithi078 ceph-mon[36683]: scrub queued for path: / 2024-06-10T01:08:27.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:26 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:08:27.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:26 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:08:27.331 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:26 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:08:27.331 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:26 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:08:27.331 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:26 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:08:27.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:26 smithi175 ceph-mon[36636]: pgmap v505: 129 pgs: 129 active+clean; 35 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 307 op/s 2024-06-10T01:08:27.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:26 smithi175 ceph-mon[36636]: scrub queued for path: / 2024-06-10T01:08:27.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:26 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:08:27.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:26 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:08:27.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:26 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:08:27.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:26 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:08:27.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:26 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:08:27.710 INFO:tasks.workunit.client.0.smithi007.stdout: 1 85337 7.04 MB/sec warmup 68 sec latency 1929.016 ms 2024-06-10T01:08:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.639+0000 7fe2e050d700 1 -- 172.21.15.7:0/3003095050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2d8101d50 msgr2=0x7fe2d810e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.639+0000 7fe2e050d700 1 --2- 172.21.15.7:0/3003095050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2d8101d50 0x7fe2d810e240 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe2d000b0a0 tx=0x7fe2d000b3b0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.639+0000 7fe2e050d700 1 -- 172.21.15.7:0/3003095050 shutdown_connections 2024-06-10T01:08:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.639+0000 7fe2e050d700 1 --2- 172.21.15.7:0/3003095050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2d8101d50 0x7fe2d810e240 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.639+0000 7fe2e050d700 1 --2- 172.21.15.7:0/3003095050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2d8107400 0x7fe2d81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.639+0000 7fe2e050d700 1 --2- 172.21.15.7:0/3003095050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2d8101390 0x7fe2d8101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.639+0000 7fe2e050d700 1 -- 172.21.15.7:0/3003095050 >> 172.21.15.7:0/3003095050 conn(0x7fe2d8078340 msgr2=0x7fe2d8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.639+0000 7fe2e050d700 1 -- 172.21.15.7:0/3003095050 shutdown_connections 2024-06-10T01:08:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.639+0000 7fe2e050d700 1 -- 172.21.15.7:0/3003095050 wait complete. 2024-06-10T01:08:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.640+0000 7fe2e050d700 1 Processor -- start 2024-06-10T01:08:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.640+0000 7fe2e050d700 1 -- start start 2024-06-10T01:08:28.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.640+0000 7fe2e050d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2d8101390 0x7fe2d8199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:28.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.640+0000 7fe2e050d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2d8101d50 0x7fe2d8199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:28.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.640+0000 7fe2e050d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2d8107400 0x7fe2d81a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:28.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.640+0000 7fe2e050d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d8111670 con 0x7fe2d8107400 2024-06-10T01:08:28.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.640+0000 7fe2e050d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d81114f0 con 0x7fe2d8101d50 2024-06-10T01:08:28.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.640+0000 7fe2e050d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d81117f0 con 0x7fe2d8101390 2024-06-10T01:08:28.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.640+0000 7fe2deaaa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2d8107400 0x7fe2d81a4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:28.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.641+0000 7fe2deaaa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2d8107400 0x7fe2d81a4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54122/0 (socket says 172.21.15.7:54122) 2024-06-10T01:08:28.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.641+0000 7fe2deaaa700 1 -- 172.21.15.7:0/3228697841 learned_addr learned my addr 172.21.15.7:0/3228697841 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:28.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.641+0000 7fe2ddaa8700 1 --2- 172.21.15.7:0/3228697841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2d8101d50 0x7fe2d8199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:28.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.641+0000 7fe2de2a9700 1 --2- 172.21.15.7:0/3228697841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2d8101390 0x7fe2d8199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:28.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.641+0000 7fe2deaaa700 1 -- 172.21.15.7:0/3228697841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2d8101390 msgr2=0x7fe2d8199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:28.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.641+0000 7fe2deaaa700 1 --2- 172.21.15.7:0/3228697841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2d8101390 0x7fe2d8199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.641+0000 7fe2deaaa700 1 -- 172.21.15.7:0/3228697841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2d8101d50 msgr2=0x7fe2d8199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:28.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.641+0000 7fe2deaaa700 1 --2- 172.21.15.7:0/3228697841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2d8101d50 0x7fe2d8199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.641+0000 7fe2deaaa700 1 -- 172.21.15.7:0/3228697841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d81a4ff0 con 0x7fe2d8107400 2024-06-10T01:08:28.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.641+0000 7fe2deaaa700 1 --2- 172.21.15.7:0/3228697841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2d8107400 0x7fe2d81a4ab0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fe2d00062a0 tx=0x7fe2d0004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:28.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.641+0000 7fe2cf7fe700 1 -- 172.21.15.7:0/3228697841 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0021070 con 0x7fe2d8107400 2024-06-10T01:08:28.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.641+0000 7fe2cf7fe700 1 -- 172.21.15.7:0/3228697841 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2d0003c00 con 0x7fe2d8107400 2024-06-10T01:08:28.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.641+0000 7fe2e050d700 1 -- 172.21.15.7:0/3228697841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2d81a51c0 con 0x7fe2d8107400 2024-06-10T01:08:28.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.641+0000 7fe2cf7fe700 1 -- 172.21.15.7:0/3228697841 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0004880 con 0x7fe2d8107400 2024-06-10T01:08:28.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.641+0000 7fe2e050d700 1 -- 172.21.15.7:0/3228697841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2d81a5670 con 0x7fe2d8107400 2024-06-10T01:08:28.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.643+0000 7fe2cf7fe700 1 -- 172.21.15.7:0/3228697841 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fe2d0003db0 con 0x7fe2d8107400 2024-06-10T01:08:28.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.644+0000 7fe2cf7fe700 1 --2- 172.21.15.7:0/3228697841 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe2c40a2ca0 0x7fe2c40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:28.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.644+0000 7fe2cf7fe700 1 -- 172.21.15.7:0/3228697841 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fe2d000f020 con 0x7fe2d8107400 2024-06-10T01:08:28.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.644+0000 7fe2de2a9700 1 --2- 172.21.15.7:0/3228697841 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe2c40a2ca0 0x7fe2c40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:28.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.645+0000 7fe2de2a9700 1 --2- 172.21.15.7:0/3228697841 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe2c40a2ca0 0x7fe2c40a5160 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4005950 tx=0x7fe2d400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:28.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.645+0000 7fe2ead90b80 1 Processor -- start 2024-06-10T01:08:28.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.645+0000 7fe2ead90b80 1 -- start start 2024-06-10T01:08:28.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.645+0000 7fe2ead90b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620293e5f60 0x5620293e6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:28.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.645+0000 7fe2ead90b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620293e6880 0x5620293e0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:28.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.645+0000 7fe2ead90b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620293e07a0 0x5620293e0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:28.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.645+0000 7fe2ead90b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5620292d2f60 con 0x5620293e5f60 2024-06-10T01:08:28.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.645+0000 7fe2ead90b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562029331850 con 0x5620293e07a0 2024-06-10T01:08:28.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.645+0000 7fe2ead90b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562029299460 con 0x5620293e6880 2024-06-10T01:08:28.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.645+0000 7fe2de2a9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620293e5f60 0x5620293e6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:28.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.646+0000 7fe2de2a9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620293e5f60 0x5620293e6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54130/0 (socket says 172.21.15.7:54130) 2024-06-10T01:08:28.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.646+0000 7fe2de2a9700 1 -- 172.21.15.7:0/4187318630 learned_addr learned my addr 172.21.15.7:0/4187318630 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:28.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.646+0000 7fe2deaaa700 1 --2- 172.21.15.7:0/4187318630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620293e07a0 0x5620293e0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:28.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.646+0000 7fe2ddaa8700 1 --2- 172.21.15.7:0/4187318630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620293e6880 0x5620293e0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:28.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.646+0000 7fe2de2a9700 1 -- 172.21.15.7:0/4187318630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620293e6880 msgr2=0x5620293e0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:28.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.646+0000 7fe2de2a9700 1 --2- 172.21.15.7:0/4187318630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620293e6880 0x5620293e0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.646+0000 7fe2de2a9700 1 -- 172.21.15.7:0/4187318630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620293e07a0 msgr2=0x5620293e0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:28.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.646+0000 7fe2de2a9700 1 --2- 172.21.15.7:0/4187318630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620293e07a0 0x5620293e0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.646+0000 7fe2de2a9700 1 -- 172.21.15.7:0/4187318630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620292808d0 con 0x5620293e5f60 2024-06-10T01:08:28.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.646+0000 7fe2de2a9700 1 --2- 172.21.15.7:0/4187318630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620293e5f60 0x5620293e6340 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4010e60 tx=0x7fe2d4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca17841c736a2586 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:28.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.646+0000 7fe2ccff9700 1 -- 172.21.15.7:0/4187318630 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d4011db0 con 0x5620293e5f60 2024-06-10T01:08:28.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.646+0000 7fe2ccff9700 1 -- 172.21.15.7:0/4187318630 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2d4010e90 con 0x5620293e5f60 2024-06-10T01:08:28.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.646+0000 7fe2ccff9700 1 -- 172.21.15.7:0/4187318630 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d401c790 con 0x5620293e5f60 2024-06-10T01:08:28.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.647+0000 7fe2ead90b80 1 -- 172.21.15.7:0/4187318630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620293e5f60 msgr2=0x5620293e6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:28.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.647+0000 7fe2ead90b80 1 --2- 172.21.15.7:0/4187318630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620293e5f60 0x5620293e6340 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4010e60 tx=0x7fe2d4013000 comp rx=0 tx=0).stop 2024-06-10T01:08:28.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.647+0000 7fe2ead90b80 1 -- 172.21.15.7:0/4187318630 shutdown_connections 2024-06-10T01:08:28.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.647+0000 7fe2ead90b80 1 --2- 172.21.15.7:0/4187318630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620293e07a0 0x5620293e0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.647+0000 7fe2ead90b80 1 --2- 172.21.15.7:0/4187318630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620293e5f60 0x5620293e6340 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.647+0000 7fe2ead90b80 1 --2- 172.21.15.7:0/4187318630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620293e6880 0x5620293e0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.647+0000 7fe2ead90b80 1 -- 172.21.15.7:0/4187318630 >> 172.21.15.7:0/4187318630 conn(0x56202930d080 msgr2=0x5620293db220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.647+0000 7fe2ead90b80 1 -- 172.21.15.7:0/4187318630 shutdown_connections 2024-06-10T01:08:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.647+0000 7fe2ead90b80 1 -- 172.21.15.7:0/4187318630 wait complete. 2024-06-10T01:08:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.647+0000 7fe2ead90b80 1 -- start start 2024-06-10T01:08:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.648+0000 7fe2ead90b80 1 Processor -- start 2024-06-10T01:08:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.648+0000 7fe2ead90b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620293e07a0 0x5620293ec610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.648+0000 7fe2ead90b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620293e5f60 0x5620293e9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.648+0000 7fe2ead90b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620293e6880 0x5620293e9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.648+0000 7fe2ead90b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5620292d1fc0 con 0x5620293e6880 2024-06-10T01:08:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.648+0000 7fe2ead90b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56202932d810 con 0x5620293e5f60 2024-06-10T01:08:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.648+0000 7fe2ead90b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5620293ed260 con 0x5620293e07a0 2024-06-10T01:08:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.648+0000 7fe2ead90b80 10 client.0 _ll_register_callbacks cb 0x5620292c75b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.648+0000 7fe2deaaa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620293e6880 0x5620293e9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:28.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.648+0000 7fe2deaaa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620293e6880 0x5620293e9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54146/0 (socket says 172.21.15.7:54146) 2024-06-10T01:08:28.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.648+0000 7fe2deaaa700 1 -- 172.21.15.7:0/3289931720 learned_addr learned my addr 172.21.15.7:0/3289931720 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:28.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.648+0000 7fe2ddaa8700 1 --2- 172.21.15.7:0/3289931720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620293e5f60 0x5620293e9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:28.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.648+0000 7fe2de2a9700 1 --2- 172.21.15.7:0/3289931720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620293e07a0 0x5620293ec610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:28.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.648+0000 7fe2deaaa700 1 -- 172.21.15.7:0/3289931720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620293e07a0 msgr2=0x5620293ec610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:28.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.648+0000 7fe2deaaa700 1 --2- 172.21.15.7:0/3289931720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620293e07a0 0x5620293ec610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.648+0000 7fe2deaaa700 1 -- 172.21.15.7:0/3289931720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620293e5f60 msgr2=0x5620293e9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:28.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.648+0000 7fe2deaaa700 1 --2- 172.21.15.7:0/3289931720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620293e5f60 0x5620293e9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.648+0000 7fe2deaaa700 1 -- 172.21.15.7:0/3289931720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620292f7750 con 0x5620293e6880 2024-06-10T01:08:28.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.648+0000 7fe2ddaa8700 1 --2- 172.21.15.7:0/3289931720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620293e5f60 0x5620293e9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:08:28.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.649+0000 7fe2deaaa700 1 --2- 172.21.15.7:0/3289931720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620293e6880 0x5620293e9e50 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe2d009ca10 tx=0x7fe2d009c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88351f368e90b86 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:28.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.649+0000 7fe2c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:08:28.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.649+0000 7fe2c1ffb700 1 -- 172.21.15.7:0/3289931720 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d009cd10 con 0x5620293e6880 2024-06-10T01:08:28.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.649+0000 7fe2c1ffb700 1 -- 172.21.15.7:0/3289931720 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fe2d009b010 con 0x5620293e6880 2024-06-10T01:08:28.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.649+0000 7fe2ead90b80 1 -- 172.21.15.7:0/3289931720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2d4010e90 con 0x5620293e6880 2024-06-10T01:08:28.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.649+0000 7fe2c1ffb700 1 -- 172.21.15.7:0/3289931720 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe2d00a10c0 con 0x5620293e6880 2024-06-10T01:08:28.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.649+0000 7fe2ead90b80 10 client.16194 fetch_fsmap learned FSMap version 30 2024-06-10T01:08:28.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.649+0000 7fe2ead90b80 1 -- 172.21.15.7:0/3289931720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5620293f7a40 con 0x5620293e6880 2024-06-10T01:08:28.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.649+0000 7fe2c1ffb700 1 -- 172.21.15.7:0/3289931720 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2d4010e90 con 0x5620293e6880 2024-06-10T01:08:28.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.649+0000 7fe2c1ffb700 1 -- 172.21.15.7:0/3289931720 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe2d0011070 con 0x5620293e6880 2024-06-10T01:08:28.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.649+0000 7fe2ead90b80 10 client.16194 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:08:28.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.649+0000 7fe2ead90b80 10 client.16194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:28.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.649+0000 7fe2ead90b80 20 client.16194 populate_metadata read hostname 'smithi007' 2024-06-10T01:08:28.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.649+0000 7fe2ead90b80 1 --2- 172.21.15.7:0/3289931720 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5620293fa0d0 0x5620293fc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:28.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.649+0000 7fe2ead90b80 4 client.16194 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:08:28.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.649+0000 7fe2ead90b80 1 -- 172.21.15.7:0/3289931720 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56202933ed50 con 0x5620293fa0d0 2024-06-10T01:08:28.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.650+0000 7fe2de2a9700 1 --2- 172.21.15.7:0/3289931720 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5620293fa0d0 0x5620293fc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:28.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.650+0000 7fe2de2a9700 1 --2- 172.21.15.7:0/3289931720 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5620293fa0d0 0x5620293fc4f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbfe15d09c9fb390 server_cookie=2d4e7a35c13965bc in_seq=0 out_seq=0 2024-06-10T01:08:28.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.650+0000 7fe2c1ffb700 10 client.16194 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:28.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.651+0000 7fe2c1ffb700 1 -- 172.21.15.7:0/3289931720 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56202933ed50 con 0x5620293fa0d0 2024-06-10T01:08:28.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.651+0000 7fe2c1ffb700 10 client.16194 handle_command_reply: tid=0 2024-06-10T01:08:28.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.651+0000 7fe2ead90b80 1 client.16194 shutdown 2024-06-10T01:08:28.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.651+0000 7fe2ead90b80 1 -- 172.21.15.7:0/3289931720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620293e6880 msgr2=0x5620293e9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:28.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.651+0000 7fe2ead90b80 1 --2- 172.21.15.7:0/3289931720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620293e6880 0x5620293e9e50 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe2d009ca10 tx=0x7fe2d009c670 comp rx=0 tx=0).stop 2024-06-10T01:08:28.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.651+0000 7fe2ead90b80 1 -- 172.21.15.7:0/3289931720 shutdown_connections 2024-06-10T01:08:28.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.651+0000 7fe2ead90b80 1 --2- 172.21.15.7:0/3289931720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620293e6880 0x5620293e9e50 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.651+0000 7fe2ead90b80 1 --2- 172.21.15.7:0/3289931720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620293e07a0 0x5620293ec610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.651+0000 7fe2ead90b80 1 --2- 172.21.15.7:0/3289931720 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5620293fa0d0 0x5620293fc4f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.651+0000 7fe2ead90b80 1 --2- 172.21.15.7:0/3289931720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620293e5f60 0x5620293e9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.651+0000 7fe2c1ffb700 0 client.16194 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:28.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.651+0000 7fe2ead90b80 1 -- 172.21.15.7:0/3289931720 >> 172.21.15.7:0/3289931720 conn(0x56202930d080 msgr2=0x5620293da700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:28.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.651+0000 7fe2ead90b80 1 -- 172.21.15.7:0/3289931720 shutdown_connections 2024-06-10T01:08:28.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.651+0000 7fe2ead90b80 1 -- 172.21.15.7:0/3289931720 wait complete. 2024-06-10T01:08:28.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.651+0000 7fe2ead90b80 20 client.16194 trim_cache size 0 max 16384 2024-06-10T01:08:28.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.672+0000 7fe2ead90b80 1 Processor -- start 2024-06-10T01:08:28.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.672+0000 7fe2ead90b80 1 -- start start 2024-06-10T01:08:28.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.672+0000 7fe2ead90b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620293e07a0 0x5620293ec150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:28.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.673+0000 7fe2ead90b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620293e5f60 0x5620293dae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:28.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.673+0000 7fe2ead90b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620293e6880 0x5620293db3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:28.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.673+0000 7fe2ead90b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d81b0ac0 con 0x5620293e5f60 2024-06-10T01:08:28.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.673+0000 7fe2ead90b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d81ae2f0 con 0x5620293e07a0 2024-06-10T01:08:28.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.673+0000 7fe2ead90b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d81ac140 con 0x5620293e6880 2024-06-10T01:08:28.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.673+0000 7fe2ddaa8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620293e5f60 0x5620293dae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:28.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.673+0000 7fe2ddaa8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620293e5f60 0x5620293dae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54162/0 (socket says 172.21.15.7:54162) 2024-06-10T01:08:28.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.673+0000 7fe2ddaa8700 1 -- 172.21.15.7:0/2674513210 learned_addr learned my addr 172.21.15.7:0/2674513210 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:28.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.673+0000 7fe2de2a9700 1 --2- 172.21.15.7:0/2674513210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620293e07a0 0x5620293ec150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:28.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.673+0000 7fe2ddaa8700 1 -- 172.21.15.7:0/2674513210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620293e6880 msgr2=0x5620293db3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:28.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.673+0000 7fe2ddaa8700 1 --2- 172.21.15.7:0/2674513210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620293e6880 0x5620293db3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.673+0000 7fe2ddaa8700 1 -- 172.21.15.7:0/2674513210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620293e07a0 msgr2=0x5620293ec150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:28.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.673+0000 7fe2ddaa8700 1 --2- 172.21.15.7:0/2674513210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620293e07a0 0x5620293ec150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.673+0000 7fe2ddaa8700 1 -- 172.21.15.7:0/2674513210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620293e05a0 con 0x5620293e5f60 2024-06-10T01:08:28.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.673+0000 7fe2ddaa8700 1 --2- 172.21.15.7:0/2674513210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620293e5f60 0x5620293dae80 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800ad20 tx=0x7fe2c8010710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8e7db0e98446f6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:28.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.673+0000 7fe2c3fff700 1 -- 172.21.15.7:0/2674513210 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8020070 con 0x5620293e5f60 2024-06-10T01:08:28.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.673+0000 7fe2c3fff700 1 -- 172.21.15.7:0/2674513210 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2c8003b60 con 0x5620293e5f60 2024-06-10T01:08:28.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.673+0000 7fe2c3fff700 1 -- 172.21.15.7:0/2674513210 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c801b6f0 con 0x5620293e5f60 2024-06-10T01:08:28.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.674+0000 7fe2ead90b80 1 -- 172.21.15.7:0/2674513210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620293e5f60 msgr2=0x5620293dae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:28.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.674+0000 7fe2ead90b80 1 --2- 172.21.15.7:0/2674513210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620293e5f60 0x5620293dae80 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800ad20 tx=0x7fe2c8010710 comp rx=0 tx=0).stop 2024-06-10T01:08:28.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.674+0000 7fe2ead90b80 1 -- 172.21.15.7:0/2674513210 shutdown_connections 2024-06-10T01:08:28.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.674+0000 7fe2ead90b80 1 --2- 172.21.15.7:0/2674513210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620293e6880 0x5620293db3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.674+0000 7fe2ead90b80 1 --2- 172.21.15.7:0/2674513210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620293e07a0 0x5620293ec150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.674+0000 7fe2ead90b80 1 --2- 172.21.15.7:0/2674513210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620293e5f60 0x5620293dae80 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.674+0000 7fe2ead90b80 1 -- 172.21.15.7:0/2674513210 >> 172.21.15.7:0/2674513210 conn(0x56202930d080 msgr2=0x5620293e49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:28.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.674+0000 7fe2ead90b80 1 -- 172.21.15.7:0/2674513210 shutdown_connections 2024-06-10T01:08:28.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.674+0000 7fe2ead90b80 1 -- 172.21.15.7:0/2674513210 wait complete. 2024-06-10T01:08:28.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.674+0000 7fe2ead90b80 1 -- start start 2024-06-10T01:08:28.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.675+0000 7fe2ead90b80 1 Processor -- start 2024-06-10T01:08:28.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.675+0000 7fe2ead90b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620293e07a0 0x5620293f2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:28.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.675+0000 7fe2ead90b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620293e5f60 0x5620294016d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:28.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.675+0000 7fe2ead90b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620293e6880 0x562029401c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:28.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.675+0000 7fe2ead90b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d81ac7b0 con 0x5620293e6880 2024-06-10T01:08:28.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.675+0000 7fe2ead90b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d81adc20 con 0x5620293e07a0 2024-06-10T01:08:28.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.675+0000 7fe2ead90b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d81ace80 con 0x5620293e5f60 2024-06-10T01:08:28.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.675+0000 7fe2ead90b80 10 client.0 _ll_register_callbacks cb 0x5620292c75b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:28.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.675+0000 7fe2deaaa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620293e6880 0x562029401c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:28.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.675+0000 7fe2deaaa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620293e6880 0x562029401c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54168/0 (socket says 172.21.15.7:54168) 2024-06-10T01:08:28.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.675+0000 7fe2deaaa700 1 -- 172.21.15.7:0/2540438467 learned_addr learned my addr 172.21.15.7:0/2540438467 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:28.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.675+0000 7fe2de2a9700 1 --2- 172.21.15.7:0/2540438467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620293e07a0 0x5620293f2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:28.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.675+0000 7fe2ddaa8700 1 --2- 172.21.15.7:0/2540438467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620293e5f60 0x5620294016d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:28.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.675+0000 7fe2deaaa700 1 -- 172.21.15.7:0/2540438467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620293e5f60 msgr2=0x5620294016d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:28.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.675+0000 7fe2deaaa700 1 --2- 172.21.15.7:0/2540438467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620293e5f60 0x5620294016d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.675+0000 7fe2deaaa700 1 -- 172.21.15.7:0/2540438467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620293e07a0 msgr2=0x5620293f2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:28.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.675+0000 7fe2deaaa700 1 --2- 172.21.15.7:0/2540438467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620293e07a0 0x5620293f2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.675+0000 7fe2deaaa700 1 -- 172.21.15.7:0/2540438467 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562029402450 con 0x5620293e6880 2024-06-10T01:08:28.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.675+0000 7fe2deaaa700 1 --2- 172.21.15.7:0/2540438467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620293e6880 0x562029401c10 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0009fd0 tx=0x7fe2d000b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15254208ad9008f2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:28.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.675+0000 7fe2c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:08:28.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.676+0000 7fe2c1ffb700 1 -- 172.21.15.7:0/2540438467 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d009cd10 con 0x5620293e6880 2024-06-10T01:08:28.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.676+0000 7fe2c1ffb700 1 -- 172.21.15.7:0/2540438467 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fe2d00074e0 con 0x5620293e6880 2024-06-10T01:08:28.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.676+0000 7fe2ead90b80 1 -- 172.21.15.7:0/2540438467 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2c8003b60 con 0x5620293e6880 2024-06-10T01:08:28.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.676+0000 7fe2c1ffb700 1 -- 172.21.15.7:0/2540438467 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe2d00ae050 con 0x5620293e6880 2024-06-10T01:08:28.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.676+0000 7fe2ead90b80 10 client.16200 fetch_fsmap learned FSMap version 30 2024-06-10T01:08:28.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.676+0000 7fe2ead90b80 1 -- 172.21.15.7:0/2540438467 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562029402450 con 0x5620293e6880 2024-06-10T01:08:28.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.676+0000 7fe2c1ffb700 1 -- 172.21.15.7:0/2540438467 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2c8003b60 con 0x5620293e6880 2024-06-10T01:08:28.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.676+0000 7fe2c1ffb700 1 -- 172.21.15.7:0/2540438467 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe2d0011070 con 0x5620293e6880 2024-06-10T01:08:28.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.676+0000 7fe2ead90b80 10 client.16200 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:08:28.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.676+0000 7fe2ead90b80 10 client.16200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:28.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.676+0000 7fe2ead90b80 20 client.16200 populate_metadata read hostname 'smithi007' 2024-06-10T01:08:28.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.676+0000 7fe2ead90b80 1 --2- 172.21.15.7:0/2540438467 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5620294650d0 0x5620294674f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:28.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.676+0000 7fe2ead90b80 4 client.16200 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:08:28.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.676+0000 7fe2ead90b80 1 -- 172.21.15.7:0/2540438467 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5620293392d0 con 0x5620294650d0 2024-06-10T01:08:28.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.677+0000 7fe2de2a9700 1 --2- 172.21.15.7:0/2540438467 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5620294650d0 0x5620294674f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:28.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.677+0000 7fe2de2a9700 1 --2- 172.21.15.7:0/2540438467 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5620294650d0 0x5620294674f0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5848a7e1b2ea8e7c server_cookie=10d3953c22872a41 in_seq=0 out_seq=0 2024-06-10T01:08:28.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.677+0000 7fe2c1ffb700 10 client.16200 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:28.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.677+0000 7fe2c1ffb700 1 -- 172.21.15.7:0/2540438467 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5620293392d0 con 0x5620294650d0 2024-06-10T01:08:28.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.677+0000 7fe2c1ffb700 10 client.16200 handle_command_reply: tid=0 2024-06-10T01:08:28.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.677+0000 7fe2ead90b80 1 client.16200 shutdown 2024-06-10T01:08:28.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.677+0000 7fe2ead90b80 1 -- 172.21.15.7:0/2540438467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620293e6880 msgr2=0x562029401c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:28.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.677+0000 7fe2ead90b80 1 --2- 172.21.15.7:0/2540438467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620293e6880 0x562029401c10 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0009fd0 tx=0x7fe2d000b9e0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.677+0000 7fe2ead90b80 1 -- 172.21.15.7:0/2540438467 shutdown_connections 2024-06-10T01:08:28.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.677+0000 7fe2ead90b80 1 --2- 172.21.15.7:0/2540438467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620293e6880 0x562029401c10 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.677+0000 7fe2ead90b80 1 --2- 172.21.15.7:0/2540438467 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5620294650d0 0x5620294674f0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.677+0000 7fe2c1ffb700 0 client.16200 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:28.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.677+0000 7fe2ead90b80 1 --2- 172.21.15.7:0/2540438467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620293e07a0 0x5620293f2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.677+0000 7fe2ead90b80 1 --2- 172.21.15.7:0/2540438467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620293e5f60 0x5620294016d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.677+0000 7fe2ead90b80 1 -- 172.21.15.7:0/2540438467 >> 172.21.15.7:0/2540438467 conn(0x56202930d080 msgr2=0x5620293e5280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:28.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.677+0000 7fe2ead90b80 1 -- 172.21.15.7:0/2540438467 shutdown_connections 2024-06-10T01:08:28.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.677+0000 7fe2ead90b80 1 -- 172.21.15.7:0/2540438467 wait complete. 2024-06-10T01:08:28.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.678+0000 7fe2ead90b80 20 client.16200 trim_cache size 0 max 16384 2024-06-10T01:08:28.679 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:08:28.679 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T01:08:28.679 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-10T01:08:28.680 INFO:teuthology.orchestra.run.smithi007.stdout: "9b79200e-fdca-426f-a620-5d560c26f15c": { 2024-06-10T01:08:28.680 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-10T01:08:28.680 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "9b79200e-fdca-426f-a620-5d560c26f15c", 2024-06-10T01:08:28.680 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-10T01:08:28.680 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:08:28.680 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:08:28.680 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:08:28.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.678+0000 7fe2e050d700 1 -- 172.21.15.7:0/3228697841 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe2c40a2ca0 msgr2=0x7fe2c40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:28.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.678+0000 7fe2e050d700 1 --2- 172.21.15.7:0/3228697841 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe2c40a2ca0 0x7fe2c40a5160 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4005950 tx=0x7fe2d400b410 comp rx=0 tx=0).stop 2024-06-10T01:08:28.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.678+0000 7fe2e050d700 1 -- 172.21.15.7:0/3228697841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2d8107400 msgr2=0x7fe2d81a4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:28.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.678+0000 7fe2e050d700 1 --2- 172.21.15.7:0/3228697841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2d8107400 0x7fe2d81a4ab0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fe2d00062a0 tx=0x7fe2d0004d70 comp rx=0 tx=0).stop 2024-06-10T01:08:28.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.678+0000 7fe2e050d700 1 -- 172.21.15.7:0/3228697841 shutdown_connections 2024-06-10T01:08:28.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.678+0000 7fe2e050d700 1 --2- 172.21.15.7:0/3228697841 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe2c40a2ca0 0x7fe2c40a5160 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.678+0000 7fe2e050d700 1 --2- 172.21.15.7:0/3228697841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2d8107400 0x7fe2d81a4ab0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.678+0000 7fe2e050d700 1 --2- 172.21.15.7:0/3228697841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2d8101390 0x7fe2d8199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.678+0000 7fe2e050d700 1 --2- 172.21.15.7:0/3228697841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2d8101d50 0x7fe2d8199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:28.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.678+0000 7fe2e050d700 1 -- 172.21.15.7:0/3228697841 >> 172.21.15.7:0/3228697841 conn(0x7fe2d8078340 msgr2=0x7fe2d80ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:28.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.678+0000 7fe2e050d700 1 -- 172.21.15.7:0/3228697841 shutdown_connections 2024-06-10T01:08:28.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:08:28.678+0000 7fe2e050d700 1 -- 172.21.15.7:0/3228697841 wait complete. 2024-06-10T01:08:28.710 INFO:tasks.workunit.client.0.smithi007.stdout: 1 85337 6.94 MB/sec warmup 69 sec latency 2929.070 ms 2024-06-10T01:08:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.007+0000 7f4303238640 1 -- 172.21.15.7:0/2001927433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42fc100900 msgr2=0x7f42fc100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.007+0000 7f4303238640 1 --2- 172.21.15.7:0/2001927433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42fc100900 0x7f42fc100d20 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f42f8009ca0 tx=0x7f42f802a8c0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.007+0000 7f4303238640 1 -- 172.21.15.7:0/2001927433 shutdown_connections 2024-06-10T01:08:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.007+0000 7f4303238640 1 --2- 172.21.15.7:0/2001927433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42fc1024b0 0x7f42fc108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.007+0000 7f4303238640 1 --2- 172.21.15.7:0/2001927433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42fc101af0 0x7f42fc101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.007+0000 7f4303238640 1 --2- 172.21.15.7:0/2001927433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42fc100900 0x7f42fc100d20 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.007+0000 7f4303238640 1 -- 172.21.15.7:0/2001927433 >> 172.21.15.7:0/2001927433 conn(0x7f42fc0fbe70 msgr2=0x7f42fc0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:29.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.007+0000 7f4303238640 1 -- 172.21.15.7:0/2001927433 shutdown_connections 2024-06-10T01:08:29.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.007+0000 7f4303238640 1 -- 172.21.15.7:0/2001927433 wait complete. 2024-06-10T01:08:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.008+0000 7f4303238640 1 Processor -- start 2024-06-10T01:08:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.008+0000 7f4303238640 1 -- start start 2024-06-10T01:08:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.008+0000 7f4303238640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42fc100900 0x7f42fc196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.008+0000 7f4303238640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42fc101af0 0x7f42fc197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.008+0000 7f4303238640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42fc1024b0 0x7f42fc19e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.008+0000 7f4303238640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc10c580 con 0x7f42fc100900 2024-06-10T01:08:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.008+0000 7f4303238640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc10c400 con 0x7f42fc101af0 2024-06-10T01:08:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.008+0000 7f4303238640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc10c700 con 0x7f42fc1024b0 2024-06-10T01:08:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.008+0000 7f4300fd4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42fc100900 0x7f42fc196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.008+0000 7f4300fd4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42fc100900 0x7f42fc196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54186/0 (socket says 172.21.15.7:54186) 2024-06-10T01:08:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.008+0000 7f4300fd4640 1 -- 172.21.15.7:0/1398813121 learned_addr learned my addr 172.21.15.7:0/1398813121 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.008+0000 7f42f3fff640 1 --2- 172.21.15.7:0/1398813121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42fc101af0 0x7f42fc197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.009+0000 7f43017d5640 1 --2- 172.21.15.7:0/1398813121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42fc1024b0 0x7f42fc19e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.009+0000 7f4300fd4640 1 -- 172.21.15.7:0/1398813121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42fc1024b0 msgr2=0x7f42fc19e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.009+0000 7f4300fd4640 1 --2- 172.21.15.7:0/1398813121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42fc1024b0 0x7f42fc19e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.009+0000 7f4300fd4640 1 -- 172.21.15.7:0/1398813121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42fc101af0 msgr2=0x7f42fc197180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.009+0000 7f4300fd4640 1 --2- 172.21.15.7:0/1398813121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42fc101af0 0x7f42fc197180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.009+0000 7f4300fd4640 1 -- 172.21.15.7:0/1398813121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42fc19e850 con 0x7f42fc100900 2024-06-10T01:08:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.009+0000 7f42f3fff640 1 --2- 172.21.15.7:0/1398813121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42fc101af0 0x7f42fc197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.009+0000 7f4300fd4640 1 --2- 172.21.15.7:0/1398813121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42fc100900 0x7f42fc196c40 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f42f8009bf0 tx=0x7f42f8031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.009+0000 7f42f1ffb640 1 -- 172.21.15.7:0/1398813121 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f802fa20 con 0x7f42fc100900 2024-06-10T01:08:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.009+0000 7f42f1ffb640 1 -- 172.21.15.7:0/1398813121 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42f802fbc0 con 0x7f42fc100900 2024-06-10T01:08:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.009+0000 7f4303238640 1 -- 172.21.15.7:0/1398813121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42fc19eae0 con 0x7f42fc100900 2024-06-10T01:08:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.009+0000 7f42f1ffb640 1 -- 172.21.15.7:0/1398813121 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f8046750 con 0x7f42fc100900 2024-06-10T01:08:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.009+0000 7f4303238640 1 -- 172.21.15.7:0/1398813121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42fc19f080 con 0x7f42fc100900 2024-06-10T01:08:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.012+0000 7f42f1ffb640 1 -- 172.21.15.7:0/1398813121 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f42f802fd70 con 0x7f42fc100900 2024-06-10T01:08:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.012+0000 7f4306022740 1 Processor -- start 2024-06-10T01:08:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.012+0000 7f4306022740 1 -- start start 2024-06-10T01:08:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.012+0000 7f4306022740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621317ae220 0x5621317ae640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.012+0000 7f4306022740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621317aec10 0x5621317b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.012+0000 7f4306022740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621317b3d80 0x5621317b6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.012+0000 7f4306022740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56213170f4f0 con 0x5621317ae220 2024-06-10T01:08:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.012+0000 7f4306022740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5621316ad870 con 0x5621317aec10 2024-06-10T01:08:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.012+0000 7f4306022740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56213170b4b0 con 0x5621317b3d80 2024-06-10T01:08:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.012+0000 7f42f1ffb640 1 --2- 172.21.15.7:0/1398813121 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f42e40a2c00 0x7f42e40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.013+0000 7f4300fd4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621317aec10 0x5621317b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.013+0000 7f43017d5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621317ae220 0x5621317ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.013+0000 7f42f3fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621317b3d80 0x5621317b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.013+0000 7f43017d5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621317ae220 0x5621317ae640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54192/0 (socket says 172.21.15.7:54192) 2024-06-10T01:08:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.013+0000 7f43017d5640 1 -- 172.21.15.7:0/2672288993 learned_addr learned my addr 172.21.15.7:0/2672288993 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.013+0000 7f43017d5640 1 --2- 172.21.15.7:0/1398813121 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f42e40a2c00 0x7f42e40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.013+0000 7f42f1ffb640 1 -- 172.21.15.7:0/1398813121 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f42f80f06d0 con 0x7f42fc100900 2024-06-10T01:08:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.013+0000 7f42f3fff640 1 -- 172.21.15.7:0/2672288993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621317aec10 msgr2=0x5621317b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.013+0000 7f42f3fff640 1 --2- 172.21.15.7:0/2672288993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621317aec10 0x5621317b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.013+0000 7f42f3fff640 1 -- 172.21.15.7:0/2672288993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621317ae220 msgr2=0x5621317ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.013+0000 7f42f3fff640 1 --2- 172.21.15.7:0/2672288993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621317ae220 0x5621317ae640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.013+0000 7f42f3fff640 1 -- 172.21.15.7:0/2672288993 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621316c99b0 con 0x5621317b3d80 2024-06-10T01:08:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.013+0000 7f4300fd4640 1 --2- 172.21.15.7:0/2672288993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621317aec10 0x5621317b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:08:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.013+0000 7f43017d5640 1 --2- 172.21.15.7:0/2672288993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621317ae220 0x5621317ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.013+0000 7f42f3fff640 1 --2- 172.21.15.7:0/2672288993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621317b3d80 0x5621317b6210 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009fd0 tx=0x7f42ec0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b6292adbb8e8293 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.013+0000 7f43017d5640 1 --2- 172.21.15.7:0/1398813121 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f42e40a2c00 0x7f42e40a50c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x5621317af470 tx=0x7f42f4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.013+0000 7f42e2ffd640 1 -- 172.21.15.7:0/2672288993 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec004030 con 0x5621317b3d80 2024-06-10T01:08:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.013+0000 7f42e2ffd640 1 -- 172.21.15.7:0/2672288993 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42ec005500 con 0x5621317b3d80 2024-06-10T01:08:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.013+0000 7f42e2ffd640 1 -- 172.21.15.7:0/2672288993 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec004d80 con 0x5621317b3d80 2024-06-10T01:08:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.014+0000 7f4306022740 1 -- 172.21.15.7:0/2672288993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621317b3d80 msgr2=0x5621317b6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.014+0000 7f4306022740 1 --2- 172.21.15.7:0/2672288993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621317b3d80 0x5621317b6210 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009fd0 tx=0x7f42ec0118d0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.014+0000 7f4306022740 1 -- 172.21.15.7:0/2672288993 shutdown_connections 2024-06-10T01:08:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.014+0000 7f4306022740 1 --2- 172.21.15.7:0/2672288993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621317b3d80 0x5621317b6210 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.014+0000 7f4306022740 1 --2- 172.21.15.7:0/2672288993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621317aec10 0x5621317b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.014+0000 7f4306022740 1 --2- 172.21.15.7:0/2672288993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621317ae220 0x5621317ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.014+0000 7f4306022740 1 -- 172.21.15.7:0/2672288993 >> 172.21.15.7:0/2672288993 conn(0x5621317a9890 msgr2=0x5621317a9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.014+0000 7f4306022740 1 -- 172.21.15.7:0/2672288993 shutdown_connections 2024-06-10T01:08:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.014+0000 7f4306022740 1 -- 172.21.15.7:0/2672288993 wait complete. 2024-06-10T01:08:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.015+0000 7f4306022740 1 -- start start 2024-06-10T01:08:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.015+0000 7f4306022740 1 Processor -- start 2024-06-10T01:08:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.015+0000 7f4306022740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621317ae220 0x5621317b8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.015+0000 7f4306022740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621317aec10 0x5621317b8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.015+0000 7f4306022740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621317b3d80 0x5621317bf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.015+0000 7f4306022740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5621317b6d30 con 0x5621317b3d80 2024-06-10T01:08:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.015+0000 7f4306022740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562131698ae0 con 0x5621317ae220 2024-06-10T01:08:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.015+0000 7f4306022740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5621317b6eb0 con 0x5621317aec10 2024-06-10T01:08:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.015+0000 7f4306022740 10 client.0 _ll_register_callbacks cb 0x5621316d19b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.015+0000 7f43017d5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621317b3d80 0x5621317bf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.015+0000 7f4300fd4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621317ae220 0x5621317b8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.015+0000 7f42f3fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621317aec10 0x5621317b8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.015+0000 7f43017d5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621317b3d80 0x5621317bf9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54206/0 (socket says 172.21.15.7:54206) 2024-06-10T01:08:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.015+0000 7f4300fd4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621317ae220 0x5621317b8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:37650/0 (socket says 172.21.15.7:37650) 2024-06-10T01:08:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.015+0000 7f43017d5640 1 -- 172.21.15.7:0/568586320 learned_addr learned my addr 172.21.15.7:0/568586320 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.016+0000 7f43017d5640 1 -- 172.21.15.7:0/568586320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621317aec10 msgr2=0x5621317b8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.016+0000 7f43017d5640 1 --2- 172.21.15.7:0/568586320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621317aec10 0x5621317b8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.016+0000 7f43017d5640 1 -- 172.21.15.7:0/568586320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621317ae220 msgr2=0x5621317b8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.016+0000 7f43017d5640 1 --2- 172.21.15.7:0/568586320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621317ae220 0x5621317b8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.016+0000 7f43017d5640 1 -- 172.21.15.7:0/568586320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562131423210 con 0x5621317b3d80 2024-06-10T01:08:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.016+0000 7f42f3fff640 1 --2- 172.21.15.7:0/568586320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621317aec10 0x5621317b8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.016+0000 7f43017d5640 1 --2- 172.21.15.7:0/568586320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621317b3d80 0x5621317bf9c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f42f4004600 tx=0x7f42f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d6fb09ca9b1341 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.016+0000 7f42dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:08:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.016+0000 7f42dbfff640 1 -- 172.21.15.7:0/568586320 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f400fe10 con 0x5621317b3d80 2024-06-10T01:08:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.016+0000 7f42dbfff640 1 -- 172.21.15.7:0/568586320 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f42f401ec90 con 0x5621317b3d80 2024-06-10T01:08:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.016+0000 7f4306022740 1 -- 172.21.15.7:0/568586320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42ec005500 con 0x5621317b3d80 2024-06-10T01:08:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.016+0000 7f42dbfff640 1 -- 172.21.15.7:0/568586320 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f42f4034050 con 0x5621317b3d80 2024-06-10T01:08:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.016+0000 7f4306022740 10 client.16212 fetch_fsmap learned FSMap version 30 2024-06-10T01:08:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.016+0000 7f4306022740 1 -- 172.21.15.7:0/568586320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5621314ef620 con 0x5621317b3d80 2024-06-10T01:08:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.017+0000 7f42dbfff640 1 -- 172.21.15.7:0/568586320 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42ec005500 con 0x5621317b3d80 2024-06-10T01:08:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.017+0000 7f42dbfff640 1 -- 172.21.15.7:0/568586320 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f42f401d070 con 0x5621317b3d80 2024-06-10T01:08:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.017+0000 7f4306022740 10 client.16212 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:08:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.017+0000 7f4306022740 10 client.16212 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.017+0000 7f4306022740 20 client.16212 populate_metadata read hostname 'smithi007' 2024-06-10T01:08:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.017+0000 7f4306022740 1 --2- 172.21.15.7:0/568586320 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5621317c4390 0x5621317c67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.017+0000 7f4306022740 4 client.16212 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:08:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.017+0000 7f4306022740 1 -- 172.21.15.7:0/568586320 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562131722f10 con 0x5621317c4390 2024-06-10T01:08:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.017+0000 7f4300fd4640 1 --2- 172.21.15.7:0/568586320 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5621317c4390 0x5621317c67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.017+0000 7f4300fd4640 1 --2- 172.21.15.7:0/568586320 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5621317c4390 0x5621317c67c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bf493dcdc5d8545 server_cookie=56947b6b82150efb in_seq=0 out_seq=0 2024-06-10T01:08:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.017+0000 7f42dbfff640 10 client.16212 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.018+0000 7f42dbfff640 1 -- 172.21.15.7:0/568586320 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562131722f10 con 0x5621317c4390 2024-06-10T01:08:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.018+0000 7f42dbfff640 10 client.16212 handle_command_reply: tid=0 2024-06-10T01:08:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.018+0000 7f4306022740 1 client.16212 shutdown 2024-06-10T01:08:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.018+0000 7f4306022740 1 -- 172.21.15.7:0/568586320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621317b3d80 msgr2=0x5621317bf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.018+0000 7f4306022740 1 --2- 172.21.15.7:0/568586320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621317b3d80 0x5621317bf9c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f42f4004600 tx=0x7f42f40046c0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.018+0000 7f4306022740 1 -- 172.21.15.7:0/568586320 shutdown_connections 2024-06-10T01:08:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.018+0000 7f4306022740 1 --2- 172.21.15.7:0/568586320 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5621317c4390 0x5621317c67c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.018+0000 7f4306022740 1 --2- 172.21.15.7:0/568586320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621317b3d80 0x5621317bf9c0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.018+0000 7f42dbfff640 0 client.16212 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.018+0000 7f4306022740 1 --2- 172.21.15.7:0/568586320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621317aec10 0x5621317b8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.018+0000 7f4306022740 1 --2- 172.21.15.7:0/568586320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621317ae220 0x5621317b8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.018+0000 7f4306022740 1 -- 172.21.15.7:0/568586320 >> 172.21.15.7:0/568586320 conn(0x5621317a9890 msgr2=0x5621317a67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.018+0000 7f4306022740 1 -- 172.21.15.7:0/568586320 shutdown_connections 2024-06-10T01:08:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.018+0000 7f4306022740 1 -- 172.21.15.7:0/568586320 wait complete. 2024-06-10T01:08:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.018+0000 7f4306022740 20 client.16212 trim_cache size 0 max 16384 2024-06-10T01:08:29.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.034+0000 7f4306022740 1 Processor -- start 2024-06-10T01:08:29.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.034+0000 7f4306022740 1 -- start start 2024-06-10T01:08:29.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.034+0000 7f4306022740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621317ae220 0x5621317b8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:29.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.034+0000 7f4306022740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621317aec10 0x5621317b8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:29.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.034+0000 7f4306022740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621317b3d80 0x5621317bf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:29.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.034+0000 7f4306022740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1aac50 con 0x5621317b3d80 2024-06-10T01:08:29.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.034+0000 7f4306022740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1a84c0 con 0x5621317ae220 2024-06-10T01:08:29.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.034+0000 7f4306022740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1a6310 con 0x5621317aec10 2024-06-10T01:08:29.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.034+0000 7f43017d5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621317b3d80 0x5621317bf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:29.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.034+0000 7f43017d5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621317b3d80 0x5621317bf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54220/0 (socket says 172.21.15.7:54220) 2024-06-10T01:08:29.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.034+0000 7f43017d5640 1 -- 172.21.15.7:0/1870924576 learned_addr learned my addr 172.21.15.7:0/1870924576 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:29.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.034+0000 7f4300fd4640 1 --2- 172.21.15.7:0/1870924576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621317ae220 0x5621317b8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:29.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.034+0000 7f42f3fff640 1 --2- 172.21.15.7:0/1870924576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621317aec10 0x5621317b8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:29.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.034+0000 7f43017d5640 1 -- 172.21.15.7:0/1870924576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621317aec10 msgr2=0x5621317b8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:29.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.034+0000 7f43017d5640 1 --2- 172.21.15.7:0/1870924576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621317aec10 0x5621317b8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.034+0000 7f43017d5640 1 -- 172.21.15.7:0/1870924576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621317ae220 msgr2=0x5621317b8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:29.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.034+0000 7f43017d5640 1 --2- 172.21.15.7:0/1870924576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621317ae220 0x5621317b8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.034+0000 7f43017d5640 1 -- 172.21.15.7:0/1870924576 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621317a96c0 con 0x5621317b3d80 2024-06-10T01:08:29.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.034+0000 7f43017d5640 1 --2- 172.21.15.7:0/1870924576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621317b3d80 0x5621317bf980 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f42f4004bf0 tx=0x7f42f400fb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1162e808a353bf71 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:29.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.035+0000 7f42e27fc640 1 -- 172.21.15.7:0/1870924576 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f403d070 con 0x5621317b3d80 2024-06-10T01:08:29.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.035+0000 7f42e27fc640 1 -- 172.21.15.7:0/1870924576 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42f401ec60 con 0x5621317b3d80 2024-06-10T01:08:29.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.035+0000 7f42e27fc640 1 -- 172.21.15.7:0/1870924576 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f40052f0 con 0x5621317b3d80 2024-06-10T01:08:29.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.035+0000 7f4306022740 1 -- 172.21.15.7:0/1870924576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621317b3d80 msgr2=0x5621317bf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:29.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.035+0000 7f4306022740 1 --2- 172.21.15.7:0/1870924576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621317b3d80 0x5621317bf980 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f42f4004bf0 tx=0x7f42f400fb40 comp rx=0 tx=0).stop 2024-06-10T01:08:29.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.035+0000 7f4306022740 1 -- 172.21.15.7:0/1870924576 shutdown_connections 2024-06-10T01:08:29.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.035+0000 7f4306022740 1 --2- 172.21.15.7:0/1870924576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621317b3d80 0x5621317bf980 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.035+0000 7f4306022740 1 --2- 172.21.15.7:0/1870924576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621317aec10 0x5621317b8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.035+0000 7f4306022740 1 --2- 172.21.15.7:0/1870924576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621317ae220 0x5621317b8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.035+0000 7f4306022740 1 -- 172.21.15.7:0/1870924576 >> 172.21.15.7:0/1870924576 conn(0x5621317a9890 msgr2=0x5621317b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:29.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.035+0000 7f4306022740 1 -- 172.21.15.7:0/1870924576 shutdown_connections 2024-06-10T01:08:29.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.035+0000 7f4306022740 1 -- 172.21.15.7:0/1870924576 wait complete. 2024-06-10T01:08:29.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.036+0000 7f4306022740 1 -- start start 2024-06-10T01:08:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.036+0000 7f4306022740 1 Processor -- start 2024-06-10T01:08:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.036+0000 7f4306022740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621317ae220 0x5621317bc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.036+0000 7f4306022740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621317aec10 0x5621317bc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.036+0000 7f4306022740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621317b3d80 0x56213182d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.036+0000 7f4306022740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1a6980 con 0x5621317ae220 2024-06-10T01:08:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.036+0000 7f4306022740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1a7df0 con 0x5621317aec10 2024-06-10T01:08:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.036+0000 7f4306022740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1a7050 con 0x5621317b3d80 2024-06-10T01:08:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.036+0000 7f4306022740 10 client.0 _ll_register_callbacks cb 0x7f42f400fc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.036+0000 7f4300fd4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621317ae220 0x5621317bc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.036+0000 7f42f3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621317aec10 0x5621317bc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.036+0000 7f42f3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621317aec10 0x5621317bc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:37678/0 (socket says 172.21.15.7:37678) 2024-06-10T01:08:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.036+0000 7f42f3fff640 1 -- 172.21.15.7:0/3831690571 learned_addr learned my addr 172.21.15.7:0/3831690571 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.036+0000 7f43017d5640 1 --2- 172.21.15.7:0/3831690571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621317b3d80 0x56213182d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.036+0000 7f4300fd4640 1 -- 172.21.15.7:0/3831690571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621317b3d80 msgr2=0x56213182d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.036+0000 7f4300fd4640 1 --2- 172.21.15.7:0/3831690571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621317b3d80 0x56213182d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.036+0000 7f4300fd4640 1 -- 172.21.15.7:0/3831690571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621317aec10 msgr2=0x5621317bc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.036+0000 7f4300fd4640 1 --2- 172.21.15.7:0/3831690571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621317aec10 0x5621317bc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.037+0000 7f4300fd4640 1 -- 172.21.15.7:0/3831690571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621317bfa80 con 0x5621317ae220 2024-06-10T01:08:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.037+0000 7f42f3fff640 1 --2- 172.21.15.7:0/3831690571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621317aec10 0x5621317bc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.037+0000 7f4300fd4640 1 --2- 172.21.15.7:0/3831690571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621317ae220 0x5621317bc470 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f42f80bc8b0 tx=0x7f42f80eb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5be4af769622040 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.037+0000 7f42dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:08:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.037+0000 7f42dbfff640 1 -- 172.21.15.7:0/3831690571 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f80c0f60 con 0x5621317ae220 2024-06-10T01:08:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.037+0000 7f42dbfff640 1 -- 172.21.15.7:0/3831690571 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f42f80f03f0 con 0x5621317ae220 2024-06-10T01:08:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.037+0000 7f4306022740 1 -- 172.21.15.7:0/3831690571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42f401ec60 con 0x5621317ae220 2024-06-10T01:08:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.037+0000 7f42dbfff640 1 -- 172.21.15.7:0/3831690571 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f42f80a4050 con 0x5621317ae220 2024-06-10T01:08:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.037+0000 7f4306022740 10 client.16218 fetch_fsmap learned FSMap version 30 2024-06-10T01:08:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.037+0000 7f4306022740 1 -- 172.21.15.7:0/3831690571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56213182e570 con 0x5621317ae220 2024-06-10T01:08:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.037+0000 7f42dbfff640 1 -- 172.21.15.7:0/3831690571 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42f401ec60 con 0x5621317ae220 2024-06-10T01:08:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.037+0000 7f42dbfff640 1 -- 172.21.15.7:0/3831690571 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f42f804e030 con 0x5621317ae220 2024-06-10T01:08:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.037+0000 7f4306022740 10 client.16218 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:08:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.037+0000 7f4306022740 10 client.16218 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.037+0000 7f4306022740 20 client.16218 populate_metadata read hostname 'smithi007' 2024-06-10T01:08:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.037+0000 7f4306022740 1 --2- 172.21.15.7:0/3831690571 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56213182ef30 0x562131831350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.037+0000 7f4306022740 4 client.16218 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:08:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.037+0000 7f4306022740 1 -- 172.21.15.7:0/3831690571 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56213171cc20 con 0x56213182ef30 2024-06-10T01:08:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.038+0000 7f42f3fff640 1 --2- 172.21.15.7:0/3831690571 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56213182ef30 0x562131831350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.038+0000 7f42f3fff640 1 --2- 172.21.15.7:0/3831690571 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56213182ef30 0x562131831350 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5545d4ebe225ae49 server_cookie=bfb7f69f0e63ba74 in_seq=0 out_seq=0 2024-06-10T01:08:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.038+0000 7f42dbfff640 10 client.16218 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.110+0000 7f42dbfff640 1 -- 172.21.15.7:0/3831690571 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56213171cc20 con 0x56213182ef30 2024-06-10T01:08:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.110+0000 7f42dbfff640 10 client.16218 handle_command_reply: tid=0 2024-06-10T01:08:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.110+0000 7f4306022740 1 client.16218 shutdown 2024-06-10T01:08:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.110+0000 7f4306022740 1 -- 172.21.15.7:0/3831690571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621317ae220 msgr2=0x5621317bc470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.110+0000 7f4306022740 1 --2- 172.21.15.7:0/3831690571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621317ae220 0x5621317bc470 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f42f80bc8b0 tx=0x7f42f80eb040 comp rx=0 tx=0).stop 2024-06-10T01:08:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.110+0000 7f4306022740 1 -- 172.21.15.7:0/3831690571 shutdown_connections 2024-06-10T01:08:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.110+0000 7f4306022740 1 --2- 172.21.15.7:0/3831690571 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56213182ef30 0x562131831350 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.110+0000 7f4306022740 1 --2- 172.21.15.7:0/3831690571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5621317b3d80 0x56213182d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.110+0000 7f4306022740 1 --2- 172.21.15.7:0/3831690571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621317aec10 0x5621317bc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.110+0000 7f4306022740 1 --2- 172.21.15.7:0/3831690571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5621317ae220 0x5621317bc470 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.110+0000 7f4306022740 1 -- 172.21.15.7:0/3831690571 >> 172.21.15.7:0/3831690571 conn(0x5621317a9890 msgr2=0x5621317b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.110+0000 7f42dbfff640 0 client.16218 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.110+0000 7f4306022740 1 -- 172.21.15.7:0/3831690571 shutdown_connections 2024-06-10T01:08:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.110+0000 7f4306022740 1 -- 172.21.15.7:0/3831690571 wait complete. 2024-06-10T01:08:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.110+0000 7f4306022740 20 client.16218 trim_cache size 0 max 16384 2024-06-10T01:08:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:08:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:08:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:08:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.111+0000 7f4303238640 1 -- 172.21.15.7:0/1398813121 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f42e40a2c00 msgr2=0x7f42e40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.111+0000 7f4303238640 1 --2- 172.21.15.7:0/1398813121 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f42e40a2c00 0x7f42e40a50c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x5621317af470 tx=0x7f42f4005f90 comp rx=0 tx=0).stop 2024-06-10T01:08:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.111+0000 7f4303238640 1 -- 172.21.15.7:0/1398813121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42fc100900 msgr2=0x7f42fc196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.111+0000 7f4303238640 1 --2- 172.21.15.7:0/1398813121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42fc100900 0x7f42fc196c40 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f42f8009bf0 tx=0x7f42f8031ee0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.111+0000 7f4303238640 1 -- 172.21.15.7:0/1398813121 shutdown_connections 2024-06-10T01:08:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.111+0000 7f4303238640 1 --2- 172.21.15.7:0/1398813121 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f42e40a2c00 0x7f42e40a50c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.111+0000 7f4303238640 1 --2- 172.21.15.7:0/1398813121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42fc1024b0 0x7f42fc19e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.111+0000 7f4303238640 1 --2- 172.21.15.7:0/1398813121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42fc101af0 0x7f42fc197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.111+0000 7f4303238640 1 --2- 172.21.15.7:0/1398813121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42fc100900 0x7f42fc196c40 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.111+0000 7f4303238640 1 -- 172.21.15.7:0/1398813121 >> 172.21.15.7:0/1398813121 conn(0x7f42fc0fbe70 msgr2=0x7f42fc104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.111+0000 7f4303238640 1 -- 172.21.15.7:0/1398813121 shutdown_connections 2024-06-10T01:08:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:29.111+0000 7f4303238640 1 -- 172.21.15.7:0/1398813121 wait complete. 2024-06-10T01:08:29.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:28 smithi007 ceph-mon[32398]: pgmap v506: 129 pgs: 129 active+clean; 35 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 893 KiB/s wr, 283 op/s 2024-06-10T01:08:29.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:28 smithi078 ceph-mon[36683]: pgmap v506: 129 pgs: 129 active+clean; 35 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 893 KiB/s wr, 283 op/s 2024-06-10T01:08:29.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:28 smithi175 ceph-mon[36636]: pgmap v506: 129 pgs: 129 active+clean; 35 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 893 KiB/s wr, 283 op/s 2024-06-10T01:08:29.543 INFO:tasks.cephfs.filesystem:scrub status for tag:9b79200e-fdca-426f-a620-5d560c26f15c - {'path': '/', 'tag': '9b79200e-fdca-426f-a620-5d560c26f15c', 'options': 'recursive,force'} 2024-06-10T01:08:29.710 INFO:tasks.workunit.client.0.smithi007.stdout: 1 86561 6.93 MB/sec warmup 70 sec latency 2999.954 ms 2024-06-10T01:08:30.710 INFO:tasks.workunit.client.0.smithi007.stdout: 1 88840 7.01 MB/sec warmup 71 sec latency 254.821 ms 2024-06-10T01:08:31.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:30 smithi007 ceph-mon[32398]: pgmap v507: 129 pgs: 129 active+clean; 36 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 773 KiB/s wr, 254 op/s 2024-06-10T01:08:31.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:30 smithi078 ceph-mon[36683]: pgmap v507: 129 pgs: 129 active+clean; 36 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 773 KiB/s wr, 254 op/s 2024-06-10T01:08:31.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:30 smithi175 ceph-mon[36636]: pgmap v507: 129 pgs: 129 active+clean; 36 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 773 KiB/s wr, 254 op/s 2024-06-10T01:08:31.711 INFO:tasks.workunit.client.0.smithi007.stdout: 1 88840 6.91 MB/sec warmup 72 sec latency 1254.863 ms 2024-06-10T01:08:32.711 INFO:tasks.workunit.client.0.smithi007.stdout: 1 88840 6.82 MB/sec warmup 73 sec latency 2254.943 ms 2024-06-10T01:08:33.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:32 smithi007 ceph-mon[32398]: pgmap v508: 129 pgs: 129 active+clean; 36 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 282 KiB/s wr, 103 op/s 2024-06-10T01:08:33.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:32 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:08:33.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:32 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:08:33.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:32 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:08:33.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:32 smithi078 ceph-mon[36683]: pgmap v508: 129 pgs: 129 active+clean; 36 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 282 KiB/s wr, 103 op/s 2024-06-10T01:08:33.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:32 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:08:33.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:32 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:08:33.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:32 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:08:33.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:32 smithi175 ceph-mon[36636]: pgmap v508: 129 pgs: 129 active+clean; 36 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 282 KiB/s wr, 103 op/s 2024-06-10T01:08:33.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:32 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:08:33.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:32 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:08:33.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:32 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:08:33.711 INFO:tasks.workunit.client.0.smithi007.stdout: 1 88840 6.73 MB/sec warmup 74 sec latency 3254.982 ms 2024-06-10T01:08:34.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:33 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:08:34.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:33 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:08:34.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:33 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:08:34.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:33 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:08:34.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:33 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:08:34.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:33 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:08:34.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:33 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:08:34.331 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:33 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:08:34.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:33 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:08:34.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:33 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:08:34.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:33 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:08:34.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:33 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:08:34.711 INFO:tasks.workunit.client.0.smithi007.stdout: 1 88881 6.64 MB/sec warmup 75 sec latency 3326.166 ms 2024-06-10T01:08:35.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:34 smithi007 ceph-mon[32398]: pgmap v509: 129 pgs: 129 active+clean; 36 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 343 KiB/s wr, 103 op/s 2024-06-10T01:08:35.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:34 smithi078 ceph-mon[36683]: pgmap v509: 129 pgs: 129 active+clean; 36 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 343 KiB/s wr, 103 op/s 2024-06-10T01:08:35.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:34 smithi175 ceph-mon[36636]: pgmap v509: 129 pgs: 129 active+clean; 36 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 343 KiB/s wr, 103 op/s 2024-06-10T01:08:35.711 INFO:tasks.workunit.client.0.smithi007.stdout: 1 88881 6.55 MB/sec warmup 76 sec latency 1900.517 ms 2024-06-10T01:08:36.711 INFO:tasks.workunit.client.0.smithi007.stdout: 1 88881 6.46 MB/sec warmup 77 sec latency 2900.633 ms 2024-06-10T01:08:37.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:37 smithi007 ceph-mon[32398]: pgmap v510: 129 pgs: 129 active+clean; 34 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 592 KiB/s wr, 147 op/s 2024-06-10T01:08:37.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:37 smithi078 ceph-mon[36683]: pgmap v510: 129 pgs: 129 active+clean; 34 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 592 KiB/s wr, 147 op/s 2024-06-10T01:08:37.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:37 smithi175 ceph-mon[36636]: pgmap v510: 129 pgs: 129 active+clean; 34 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 592 KiB/s wr, 147 op/s 2024-06-10T01:08:37.711 INFO:tasks.workunit.client.0.smithi007.stdout: 1 88881 6.38 MB/sec warmup 78 sec latency 3900.755 ms 2024-06-10T01:08:38.711 INFO:tasks.workunit.client.0.smithi007.stdout: 1 88881 6.30 MB/sec warmup 79 sec latency 4900.838 ms 2024-06-10T01:08:39.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:39 smithi007 ceph-mon[32398]: pgmap v511: 129 pgs: 129 active+clean; 34 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 829 KiB/s wr, 152 op/s 2024-06-10T01:08:39.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:39 smithi078 ceph-mon[36683]: pgmap v511: 129 pgs: 129 active+clean; 34 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 829 KiB/s wr, 152 op/s 2024-06-10T01:08:39.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:39 smithi175 ceph-mon[36636]: pgmap v511: 129 pgs: 129 active+clean; 34 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 829 KiB/s wr, 152 op/s 2024-06-10T01:08:39.711 INFO:tasks.workunit.client.0.smithi007.stdout: 1 90179 6.30 MB/sec warmup 80 sec latency 4973.365 ms 2024-06-10T01:08:40.711 INFO:tasks.workunit.client.0.smithi007.stdout: 1 92508 6.39 MB/sec warmup 81 sec latency 61.302 ms 2024-06-10T01:08:41.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:41 smithi007 ceph-mon[32398]: pgmap v512: 129 pgs: 129 active+clean; 33 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-06-10T01:08:41.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:41 smithi078 ceph-mon[36683]: pgmap v512: 129 pgs: 129 active+clean; 33 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-06-10T01:08:41.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:41 smithi175 ceph-mon[36636]: pgmap v512: 129 pgs: 129 active+clean; 33 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-06-10T01:08:41.711 INFO:tasks.workunit.client.0.smithi007.stdout: 1 93973 6.38 MB/sec warmup 82 sec latency 13.919 ms 2024-06-10T01:08:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.205+0000 7f301195a640 1 -- 172.21.15.7:0/1415916914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f300c104e10 msgr2=0x7f300c107200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.205+0000 7f301195a640 1 --2- 172.21.15.7:0/1415916914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f300c104e10 0x7f300c107200 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3000009900 tx=0x7f300002a410 comp rx=0 tx=0).stop 2024-06-10T01:08:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.205+0000 7f301195a640 1 -- 172.21.15.7:0/1415916914 shutdown_connections 2024-06-10T01:08:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.205+0000 7f301195a640 1 --2- 172.21.15.7:0/1415916914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f300c107740 0x7f300c109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.205+0000 7f301195a640 1 --2- 172.21.15.7:0/1415916914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f300c104e10 0x7f300c107200 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.205+0000 7f301195a640 1 --2- 172.21.15.7:0/1415916914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f300c1024e0 0x7f300c1048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.205+0000 7f301195a640 1 -- 172.21.15.7:0/1415916914 >> 172.21.15.7:0/1415916914 conn(0x7f300c0fbe70 msgr2=0x7f300c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.205+0000 7f301195a640 1 -- 172.21.15.7:0/1415916914 shutdown_connections 2024-06-10T01:08:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.205+0000 7f301195a640 1 -- 172.21.15.7:0/1415916914 wait complete. 2024-06-10T01:08:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.205+0000 7f301195a640 1 Processor -- start 2024-06-10T01:08:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.206+0000 7f301195a640 1 -- start start 2024-06-10T01:08:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.206+0000 7f301195a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f300c1024e0 0x7f300c101e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.206+0000 7f301195a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f300c104e10 0x7f300c100490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.206+0000 7f301195a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f300c107740 0x7f300c1009d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.206+0000 7f301195a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f300c10d680 con 0x7f300c104e10 2024-06-10T01:08:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.206+0000 7f301195a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f300c10d500 con 0x7f300c1024e0 2024-06-10T01:08:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.206+0000 7f301195a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f300c10d800 con 0x7f300c107740 2024-06-10T01:08:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.206+0000 7f300a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f300c104e10 0x7f300c100490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.206+0000 7f300affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f300c1024e0 0x7f300c101e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.206+0000 7f300a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f300c104e10 0x7f300c100490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37500/0 (socket says 172.21.15.7:37500) 2024-06-10T01:08:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.206+0000 7f300affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f300c1024e0 0x7f300c101e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:54718/0 (socket says 172.21.15.7:54718) 2024-06-10T01:08:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.206+0000 7f300b7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f300c107740 0x7f300c1009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.206+0000 7f300a7fc640 1 -- 172.21.15.7:0/3635064064 learned_addr learned my addr 172.21.15.7:0/3635064064 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.206+0000 7f300a7fc640 1 -- 172.21.15.7:0/3635064064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f300c107740 msgr2=0x7f300c1009d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.206+0000 7f300a7fc640 1 --2- 172.21.15.7:0/3635064064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f300c107740 0x7f300c1009d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.206+0000 7f300a7fc640 1 -- 172.21.15.7:0/3635064064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f300c1024e0 msgr2=0x7f300c101e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.206+0000 7f300a7fc640 1 --2- 172.21.15.7:0/3635064064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f300c1024e0 0x7f300c101e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.206+0000 7f300a7fc640 1 -- 172.21.15.7:0/3635064064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300c101260 con 0x7f300c104e10 2024-06-10T01:08:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.206+0000 7f300affd640 1 --2- 172.21.15.7:0/3635064064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f300c1024e0 0x7f300c101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.206+0000 7f300a7fc640 1 --2- 172.21.15.7:0/3635064064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f300c104e10 0x7f300c100490 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3000031610 tx=0x7f3000004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.207+0000 7f3010958640 1 -- 172.21.15.7:0/3635064064 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300003f030 con 0x7f300c104e10 2024-06-10T01:08:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.207+0000 7f3010958640 1 -- 172.21.15.7:0/3635064064 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3000003ad0 con 0x7f300c104e10 2024-06-10T01:08:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.207+0000 7f301195a640 1 -- 172.21.15.7:0/3635064064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f300c1a1e10 con 0x7f300c104e10 2024-06-10T01:08:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.207+0000 7f3010958640 1 -- 172.21.15.7:0/3635064064 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000035070 con 0x7f300c104e10 2024-06-10T01:08:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.207+0000 7f301195a640 1 -- 172.21.15.7:0/3635064064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f300c1a2270 con 0x7f300c104e10 2024-06-10T01:08:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.209+0000 7f3010958640 1 -- 172.21.15.7:0/3635064064 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f3000042030 con 0x7f300c104e10 2024-06-10T01:08:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.209+0000 7f3014666740 1 Processor -- start 2024-06-10T01:08:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.209+0000 7f3014666740 1 -- start start 2024-06-10T01:08:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.209+0000 7f3014666740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b34fc4010 0x555b34fc4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.209+0000 7f3014666740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b34fc4a00 0x555b34fc9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.209+0000 7f3014666740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b34fc9b70 0x555b34fcc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.209+0000 7f3014666740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555b34f274f0 con 0x555b34fc4010 2024-06-10T01:08:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.209+0000 7f3014666740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555b34ec5870 con 0x555b34fc4a00 2024-06-10T01:08:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.209+0000 7f3014666740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555b34f234b0 con 0x555b34fc9b70 2024-06-10T01:08:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.210+0000 7f300b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b34fc4010 0x555b34fc4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.210+0000 7f300b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b34fc4010 0x555b34fc4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37512/0 (socket says 172.21.15.7:37512) 2024-06-10T01:08:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.210+0000 7f300b7fe640 1 -- 172.21.15.7:0/3872748454 learned_addr learned my addr 172.21.15.7:0/3872748454 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.210+0000 7f300affd640 1 --2- 172.21.15.7:0/3872748454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b34fc4a00 0x555b34fc9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.210+0000 7f300a7fc640 1 --2- 172.21.15.7:0/3872748454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b34fc9b70 0x555b34fcc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.210+0000 7f3010958640 1 --2- 172.21.15.7:0/3635064064 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2ff40a2b80 0x7f2ff40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.210+0000 7f300affd640 1 -- 172.21.15.7:0/3872748454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b34fc9b70 msgr2=0x555b34fcc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.210+0000 7f300affd640 1 --2- 172.21.15.7:0/3872748454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b34fc9b70 0x555b34fcc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.210+0000 7f300affd640 1 -- 172.21.15.7:0/3872748454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b34fc4010 msgr2=0x555b34fc4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.210+0000 7f300affd640 1 --2- 172.21.15.7:0/3872748454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b34fc4010 0x555b34fc4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.210+0000 7f300affd640 1 -- 172.21.15.7:0/3872748454 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b34ee19b0 con 0x555b34fc4a00 2024-06-10T01:08:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.210+0000 7f300b7fe640 1 --2- 172.21.15.7:0/3635064064 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2ff40a2b80 0x7f2ff40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.210+0000 7f3010958640 1 -- 172.21.15.7:0/3635064064 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f30000ef920 con 0x7f300c104e10 2024-06-10T01:08:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.210+0000 7f300affd640 1 --2- 172.21.15.7:0/3872748454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b34fc4a00 0x555b34fc9500 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009fd0 tx=0x7f2ffc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8166140323c1a224 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.210+0000 7f2ff1ffb640 1 -- 172.21.15.7:0/3872748454 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc004030 con 0x555b34fc4a00 2024-06-10T01:08:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.210+0000 7f2ff1ffb640 1 -- 172.21.15.7:0/3872748454 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ffc005500 con 0x555b34fc4a00 2024-06-10T01:08:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.210+0000 7f2ff1ffb640 1 -- 172.21.15.7:0/3872748454 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc004d50 con 0x555b34fc4a00 2024-06-10T01:08:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.211+0000 7f3014666740 1 -- 172.21.15.7:0/3872748454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b34fc4a00 msgr2=0x555b34fc9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.211+0000 7f300b7fe640 1 --2- 172.21.15.7:0/3635064064 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2ff40a2b80 0x7f2ff40a5040 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3004009fd0 tx=0x7f3004004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.211+0000 7f3014666740 1 --2- 172.21.15.7:0/3872748454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b34fc4a00 0x555b34fc9500 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009fd0 tx=0x7f2ffc0118d0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.211+0000 7f3014666740 1 -- 172.21.15.7:0/3872748454 shutdown_connections 2024-06-10T01:08:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.211+0000 7f3014666740 1 --2- 172.21.15.7:0/3872748454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b34fc9b70 0x555b34fcc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.211+0000 7f3014666740 1 --2- 172.21.15.7:0/3872748454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b34fc4a00 0x555b34fc9500 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.211+0000 7f3014666740 1 --2- 172.21.15.7:0/3872748454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b34fc4010 0x555b34fc4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.211+0000 7f3014666740 1 -- 172.21.15.7:0/3872748454 >> 172.21.15.7:0/3872748454 conn(0x555b34fbf680 msgr2=0x555b34fbfa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.211+0000 7f3014666740 1 -- 172.21.15.7:0/3872748454 shutdown_connections 2024-06-10T01:08:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.211+0000 7f3014666740 1 -- 172.21.15.7:0/3872748454 wait complete. 2024-06-10T01:08:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.211+0000 7f3014666740 1 -- start start 2024-06-10T01:08:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.212+0000 7f3014666740 1 Processor -- start 2024-06-10T01:08:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.212+0000 7f3014666740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b34fc4010 0x555b34fce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.212+0000 7f3014666740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b34fc4a00 0x555b34fce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.212+0000 7f3014666740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b34fc9b70 0x555b34fd57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.212+0000 7f3014666740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555b34fccb20 con 0x555b34fc4010 2024-06-10T01:08:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.212+0000 7f3014666740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555b34eb0ae0 con 0x555b34fc9b70 2024-06-10T01:08:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.212+0000 7f3014666740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555b34fccca0 con 0x555b34fc4a00 2024-06-10T01:08:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.212+0000 7f3014666740 10 client.0 _ll_register_callbacks cb 0x555b34ee99b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.212+0000 7f300affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b34fc4010 0x555b34fce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.212+0000 7f300affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b34fc4010 0x555b34fce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37528/0 (socket says 172.21.15.7:37528) 2024-06-10T01:08:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.212+0000 7f300a7fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b34fc4a00 0x555b34fce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.212+0000 7f300affd640 1 -- 172.21.15.7:0/2834988180 learned_addr learned my addr 172.21.15.7:0/2834988180 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.212+0000 7f300b7fe640 1 --2- 172.21.15.7:0/2834988180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b34fc9b70 0x555b34fd57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.212+0000 7f300affd640 1 -- 172.21.15.7:0/2834988180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b34fc4a00 msgr2=0x555b34fce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.212+0000 7f300affd640 1 --2- 172.21.15.7:0/2834988180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b34fc4a00 0x555b34fce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.212+0000 7f300affd640 1 -- 172.21.15.7:0/2834988180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b34fc9b70 msgr2=0x555b34fd57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.212+0000 7f300affd640 1 --2- 172.21.15.7:0/2834988180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b34fc9b70 0x555b34fd57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.212+0000 7f300affd640 1 -- 172.21.15.7:0/2834988180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b34c3b210 con 0x555b34fc4010 2024-06-10T01:08:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.213+0000 7f300affd640 1 --2- 172.21.15.7:0/2834988180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b34fc4010 0x555b34fce340 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc005890 tx=0x7f2ffc00fd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f497069ffc10df9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.213+0000 7f2feaffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:08:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.213+0000 7f2feaffd640 1 -- 172.21.15.7:0/2834988180 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc028070 con 0x555b34fc4010 2024-06-10T01:08:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.213+0000 7f2feaffd640 1 -- 172.21.15.7:0/2834988180 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f2ffc004c60 con 0x555b34fc4010 2024-06-10T01:08:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.213+0000 7f3014666740 1 -- 172.21.15.7:0/2834988180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ffc005500 con 0x555b34fc4010 2024-06-10T01:08:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.213+0000 7f2feaffd640 1 -- 172.21.15.7:0/2834988180 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2ffc034050 con 0x555b34fc4010 2024-06-10T01:08:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.213+0000 7f3014666740 10 client.16230 fetch_fsmap learned FSMap version 30 2024-06-10T01:08:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.213+0000 7f3014666740 1 -- 172.21.15.7:0/2834988180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555b34d07620 con 0x555b34fc4010 2024-06-10T01:08:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.213+0000 7f2feaffd640 1 -- 172.21.15.7:0/2834988180 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ffc005500 con 0x555b34fc4010 2024-06-10T01:08:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.213+0000 7f2feaffd640 1 -- 172.21.15.7:0/2834988180 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f2ffc019830 con 0x555b34fc4010 2024-06-10T01:08:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.213+0000 7f3014666740 10 client.16230 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:08:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.214+0000 7f3014666740 10 client.16230 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.214+0000 7f3014666740 20 client.16230 populate_metadata read hostname 'smithi007' 2024-06-10T01:08:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.214+0000 7f3014666740 1 --2- 172.21.15.7:0/2834988180 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555b34fd9ee0 0x555b34fdc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.214+0000 7f3014666740 4 client.16230 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:08:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.214+0000 7f3014666740 1 -- 172.21.15.7:0/2834988180 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b34f3af10 con 0x555b34fd9ee0 2024-06-10T01:08:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.214+0000 7f300a7fc640 1 --2- 172.21.15.7:0/2834988180 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555b34fd9ee0 0x555b34fdc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.214+0000 7f2feaffd640 10 client.16230 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.214+0000 7f300a7fc640 1 --2- 172.21.15.7:0/2834988180 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555b34fd9ee0 0x555b34fdc380 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80a1d139fcd9369b server_cookie=cc637f737efe5407 in_seq=0 out_seq=0 2024-06-10T01:08:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.215+0000 7f2feaffd640 1 -- 172.21.15.7:0/2834988180 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555b34f3af10 con 0x555b34fd9ee0 2024-06-10T01:08:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.215+0000 7f2feaffd640 10 client.16230 handle_command_reply: tid=0 2024-06-10T01:08:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.215+0000 7f3014666740 1 client.16230 shutdown 2024-06-10T01:08:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.215+0000 7f3014666740 1 -- 172.21.15.7:0/2834988180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b34fc4010 msgr2=0x555b34fce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.215+0000 7f3014666740 1 --2- 172.21.15.7:0/2834988180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b34fc4010 0x555b34fce340 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc005890 tx=0x7f2ffc00fd30 comp rx=0 tx=0).stop 2024-06-10T01:08:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.215+0000 7f3014666740 1 -- 172.21.15.7:0/2834988180 shutdown_connections 2024-06-10T01:08:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.215+0000 7f3014666740 1 --2- 172.21.15.7:0/2834988180 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555b34fd9ee0 0x555b34fdc380 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.215+0000 7f3014666740 1 --2- 172.21.15.7:0/2834988180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b34fc9b70 0x555b34fd57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.215+0000 7f3014666740 1 --2- 172.21.15.7:0/2834988180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b34fc4a00 0x555b34fce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.215+0000 7f2feaffd640 0 client.16230 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.215+0000 7f3014666740 1 --2- 172.21.15.7:0/2834988180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b34fc4010 0x555b34fce340 secure :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc005890 tx=0x7f2ffc00fd30 comp rx=0 tx=0).stop 2024-06-10T01:08:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.215+0000 7f3014666740 1 -- 172.21.15.7:0/2834988180 >> 172.21.15.7:0/2834988180 conn(0x555b34fbf680 msgr2=0x555b34fbe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.215+0000 7f3014666740 1 -- 172.21.15.7:0/2834988180 shutdown_connections 2024-06-10T01:08:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.215+0000 7f3014666740 1 -- 172.21.15.7:0/2834988180 wait complete. 2024-06-10T01:08:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.215+0000 7f3014666740 20 client.16230 trim_cache size 0 max 16384 2024-06-10T01:08:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.230+0000 7f3014666740 1 Processor -- start 2024-06-10T01:08:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.230+0000 7f3014666740 1 -- start start 2024-06-10T01:08:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.230+0000 7f3014666740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b34fc4a00 0x555b34fcd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.230+0000 7f3014666740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b34fc9b70 0x555b34fd57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.230+0000 7f3014666740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b34fcd500 0x555b34fcded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.230+0000 7f3014666740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1af120 con 0x555b34fcd500 2024-06-10T01:08:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.230+0000 7f3014666740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1ac990 con 0x555b34fc9b70 2024-06-10T01:08:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.230+0000 7f3014666740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1aa7e0 con 0x555b34fc4a00 2024-06-10T01:08:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.231+0000 7f300b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b34fcd500 0x555b34fcded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.231+0000 7f300b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b34fcd500 0x555b34fcded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37530/0 (socket says 172.21.15.7:37530) 2024-06-10T01:08:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.231+0000 7f300b7fe640 1 -- 172.21.15.7:0/3413503307 learned_addr learned my addr 172.21.15.7:0/3413503307 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.231+0000 7f300affd640 1 --2- 172.21.15.7:0/3413503307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b34fc4a00 0x555b34fcd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.231+0000 7f300a7fc640 1 --2- 172.21.15.7:0/3413503307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b34fc9b70 0x555b34fd57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.231+0000 7f300b7fe640 1 -- 172.21.15.7:0/3413503307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b34fc4a00 msgr2=0x555b34fcd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.231+0000 7f300b7fe640 1 --2- 172.21.15.7:0/3413503307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b34fc4a00 0x555b34fcd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.231+0000 7f300b7fe640 1 -- 172.21.15.7:0/3413503307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b34fc9b70 msgr2=0x555b34fd57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.231+0000 7f300b7fe640 1 --2- 172.21.15.7:0/3413503307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b34fc9b70 0x555b34fd57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.231+0000 7f300b7fe640 1 -- 172.21.15.7:0/3413503307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b34fc3300 con 0x555b34fcd500 2024-06-10T01:08:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.231+0000 7f300b7fe640 1 --2- 172.21.15.7:0/3413503307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b34fcd500 0x555b34fcded0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3004020710 tx=0x7f3004013030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15634aeffe62ba server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.231+0000 7f2ff17fa640 1 -- 172.21.15.7:0/3413503307 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3004004d20 con 0x555b34fcd500 2024-06-10T01:08:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.231+0000 7f2ff17fa640 1 -- 172.21.15.7:0/3413503307 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f300401ec10 con 0x555b34fcd500 2024-06-10T01:08:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.231+0000 7f2ff17fa640 1 -- 172.21.15.7:0/3413503307 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30040058b0 con 0x555b34fcd500 2024-06-10T01:08:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.232+0000 7f3014666740 1 -- 172.21.15.7:0/3413503307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b34fcd500 msgr2=0x555b34fcded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.232+0000 7f3014666740 1 --2- 172.21.15.7:0/3413503307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b34fcd500 0x555b34fcded0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3004020710 tx=0x7f3004013030 comp rx=0 tx=0).stop 2024-06-10T01:08:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.232+0000 7f3014666740 1 -- 172.21.15.7:0/3413503307 shutdown_connections 2024-06-10T01:08:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.232+0000 7f3014666740 1 --2- 172.21.15.7:0/3413503307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b34fcd500 0x555b34fcded0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.232+0000 7f3014666740 1 --2- 172.21.15.7:0/3413503307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b34fc9b70 0x555b34fd57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.232+0000 7f3014666740 1 --2- 172.21.15.7:0/3413503307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b34fc4a00 0x555b34fcd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.232+0000 7f3014666740 1 -- 172.21.15.7:0/3413503307 >> 172.21.15.7:0/3413503307 conn(0x555b34fbf680 msgr2=0x555b34fc2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.232+0000 7f3014666740 1 -- 172.21.15.7:0/3413503307 shutdown_connections 2024-06-10T01:08:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.232+0000 7f3014666740 1 -- 172.21.15.7:0/3413503307 wait complete. 2024-06-10T01:08:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.232+0000 7f3014666740 1 -- start start 2024-06-10T01:08:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.232+0000 7f3014666740 1 Processor -- start 2024-06-10T01:08:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.233+0000 7f3014666740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b34fc4a00 0x555b34fd2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.233+0000 7f3014666740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b34fc9b70 0x555b34fd2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.233+0000 7f3014666740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b34fcd500 0x555b35042c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.233+0000 7f3014666740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1aae50 con 0x555b34fcd500 2024-06-10T01:08:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.233+0000 7f3014666740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1ac2c0 con 0x555b34fc9b70 2024-06-10T01:08:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.233+0000 7f3014666740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1ab520 con 0x555b34fc4a00 2024-06-10T01:08:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.233+0000 7f3014666740 10 client.0 _ll_register_callbacks cb 0x7f3004009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.233+0000 7f300b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b34fcd500 0x555b35042c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.233+0000 7f300affd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b34fc4a00 0x555b34fd2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.233+0000 7f300a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b34fc9b70 0x555b34fd2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.233+0000 7f300affd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b34fc4a00 0x555b34fd2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:45054/0 (socket says 172.21.15.7:45054) 2024-06-10T01:08:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.233+0000 7f300a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b34fc9b70 0x555b34fd2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:54756/0 (socket says 172.21.15.7:54756) 2024-06-10T01:08:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.233+0000 7f300affd640 1 -- 172.21.15.7:0/165624972 learned_addr learned my addr 172.21.15.7:0/165624972 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.233+0000 7f300b7fe640 1 -- 172.21.15.7:0/165624972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b34fc4a00 msgr2=0x555b34fd2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.233+0000 7f300b7fe640 1 --2- 172.21.15.7:0/165624972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b34fc4a00 0x555b34fd2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.233+0000 7f300b7fe640 1 -- 172.21.15.7:0/165624972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b34fc9b70 msgr2=0x555b34fd2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.233+0000 7f300b7fe640 1 --2- 172.21.15.7:0/165624972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b34fc9b70 0x555b34fd2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.233+0000 7f300b7fe640 1 -- 172.21.15.7:0/165624972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b35043220 con 0x555b34fcd500 2024-06-10T01:08:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.233+0000 7f300affd640 1 --2- 172.21.15.7:0/165624972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b34fc4a00 0x555b34fd2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.233+0000 7f300b7fe640 1 --2- 172.21.15.7:0/165624972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b34fcd500 0x555b35042c60 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x555b34fd5da0 tx=0x7f300401edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3573aa53a398b0c3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.233+0000 7f2feaffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:08:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.233+0000 7f2feaffd640 1 -- 172.21.15.7:0/165624972 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3004005630 con 0x555b34fcd500 2024-06-10T01:08:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.233+0000 7f2feaffd640 1 -- 172.21.15.7:0/165624972 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f30040057d0 con 0x555b34fcd500 2024-06-10T01:08:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.233+0000 7f3014666740 1 -- 172.21.15.7:0/165624972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f300401ec10 con 0x555b34fcd500 2024-06-10T01:08:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.234+0000 7f2feaffd640 1 -- 172.21.15.7:0/165624972 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f300403e050 con 0x555b34fcd500 2024-06-10T01:08:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.234+0000 7f3014666740 10 client.16236 fetch_fsmap learned FSMap version 30 2024-06-10T01:08:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.234+0000 7f3014666740 1 -- 172.21.15.7:0/165624972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555b350445a0 con 0x555b34fcd500 2024-06-10T01:08:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.234+0000 7f2feaffd640 1 -- 172.21.15.7:0/165624972 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f300401ec10 con 0x555b34fcd500 2024-06-10T01:08:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.234+0000 7f2feaffd640 1 -- 172.21.15.7:0/165624972 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f300402a720 con 0x555b34fcd500 2024-06-10T01:08:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.234+0000 7f3014666740 10 client.16236 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:08:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.234+0000 7f3014666740 10 client.16236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.234+0000 7f3014666740 20 client.16236 populate_metadata read hostname 'smithi007' 2024-06-10T01:08:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.234+0000 7f3014666740 1 --2- 172.21.15.7:0/165624972 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555b35044f00 0x555b35047320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.234+0000 7f3014666740 4 client.16236 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:08:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.234+0000 7f3014666740 1 -- 172.21.15.7:0/165624972 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555b34f34c20 con 0x555b35044f00 2024-06-10T01:08:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.234+0000 7f300affd640 1 --2- 172.21.15.7:0/165624972 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555b35044f00 0x555b35047320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.235+0000 7f2feaffd640 10 client.16236 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.235+0000 7f300affd640 1 --2- 172.21.15.7:0/165624972 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555b35044f00 0x555b35047320 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8255dbf31c613e7a server_cookie=838f6cfc6c3cc755 in_seq=0 out_seq=0 2024-06-10T01:08:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.301+0000 7f2feaffd640 1 -- 172.21.15.7:0/165624972 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555b34f34c20 con 0x555b35044f00 2024-06-10T01:08:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.301+0000 7f2feaffd640 10 client.16236 handle_command_reply: tid=0 2024-06-10T01:08:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.301+0000 7f3014666740 1 client.16236 shutdown 2024-06-10T01:08:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.302+0000 7f3014666740 1 -- 172.21.15.7:0/165624972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b34fcd500 msgr2=0x555b35042c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.302+0000 7f3014666740 1 --2- 172.21.15.7:0/165624972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b34fcd500 0x555b35042c60 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x555b34fd5da0 tx=0x7f300401edc0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.302+0000 7f3014666740 1 -- 172.21.15.7:0/165624972 shutdown_connections 2024-06-10T01:08:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.302+0000 7f3014666740 1 --2- 172.21.15.7:0/165624972 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555b35044f00 0x555b35047320 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.302+0000 7f3014666740 1 --2- 172.21.15.7:0/165624972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b34fcd500 0x555b35042c60 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.302+0000 7f3014666740 1 --2- 172.21.15.7:0/165624972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b34fc9b70 0x555b34fd2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.302+0000 7f2feaffd640 0 client.16236 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.302+0000 7f3014666740 1 --2- 172.21.15.7:0/165624972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b34fc4a00 0x555b34fd2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.302+0000 7f3014666740 1 -- 172.21.15.7:0/165624972 >> 172.21.15.7:0/165624972 conn(0x555b34fbf680 msgr2=0x555b34fbdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.302+0000 7f3014666740 1 -- 172.21.15.7:0/165624972 shutdown_connections 2024-06-10T01:08:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.302+0000 7f3014666740 1 -- 172.21.15.7:0/165624972 wait complete. 2024-06-10T01:08:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.302+0000 7f3014666740 20 client.16236 trim_cache size 0 max 16384 2024-06-10T01:08:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:08:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:08:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:08:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.302+0000 7f301195a640 1 -- 172.21.15.7:0/3635064064 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2ff40a2b80 msgr2=0x7f2ff40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.302+0000 7f301195a640 1 --2- 172.21.15.7:0/3635064064 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2ff40a2b80 0x7f2ff40a5040 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3004009fd0 tx=0x7f3004004a40 comp rx=0 tx=0).stop 2024-06-10T01:08:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.302+0000 7f301195a640 1 -- 172.21.15.7:0/3635064064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f300c104e10 msgr2=0x7f300c100490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.302+0000 7f301195a640 1 --2- 172.21.15.7:0/3635064064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f300c104e10 0x7f300c100490 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3000031610 tx=0x7f3000004270 comp rx=0 tx=0).stop 2024-06-10T01:08:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.302+0000 7f301195a640 1 -- 172.21.15.7:0/3635064064 shutdown_connections 2024-06-10T01:08:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.302+0000 7f301195a640 1 --2- 172.21.15.7:0/3635064064 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2ff40a2b80 0x7f2ff40a5040 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.303+0000 7f301195a640 1 --2- 172.21.15.7:0/3635064064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f300c107740 0x7f300c1009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.303+0000 7f301195a640 1 --2- 172.21.15.7:0/3635064064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f300c104e10 0x7f300c100490 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.303+0000 7f301195a640 1 --2- 172.21.15.7:0/3635064064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f300c1024e0 0x7f300c101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.303+0000 7f301195a640 1 -- 172.21.15.7:0/3635064064 >> 172.21.15.7:0/3635064064 conn(0x7f300c0fbe70 msgr2=0x7f300c102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.303+0000 7f301195a640 1 -- 172.21.15.7:0/3635064064 shutdown_connections 2024-06-10T01:08:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:42.303+0000 7f301195a640 1 -- 172.21.15.7:0/3635064064 wait complete. 2024-06-10T01:08:42.711 INFO:tasks.workunit.client.0.smithi007.stdout: 1 96276 6.47 MB/sec warmup 83 sec latency 36.976 ms 2024-06-10T01:08:43.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:43 smithi007 ceph-mon[32398]: pgmap v513: 129 pgs: 129 active+clean; 33 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.4 MiB/s wr, 153 op/s 2024-06-10T01:08:43.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:43 smithi078 ceph-mon[36683]: pgmap v513: 129 pgs: 129 active+clean; 33 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.4 MiB/s wr, 153 op/s 2024-06-10T01:08:43.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:43 smithi175 ceph-mon[36636]: pgmap v513: 129 pgs: 129 active+clean; 33 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.4 MiB/s wr, 153 op/s 2024-06-10T01:08:43.711 INFO:tasks.workunit.client.0.smithi007.stdout: 1 98815 6.55 MB/sec warmup 84 sec latency 16.395 ms 2024-06-10T01:08:44.711 INFO:tasks.workunit.client.0.smithi007.stdout: 1 100497 6.57 MB/sec warmup 85 sec latency 11.050 ms 2024-06-10T01:08:45.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:45 smithi007 ceph-mon[32398]: pgmap v514: 129 pgs: 129 active+clean; 33 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 1.8 MiB/s wr, 164 op/s 2024-06-10T01:08:45.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:45 smithi078 ceph-mon[36683]: pgmap v514: 129 pgs: 129 active+clean; 33 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 1.8 MiB/s wr, 164 op/s 2024-06-10T01:08:45.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:45 smithi175 ceph-mon[36636]: pgmap v514: 129 pgs: 129 active+clean; 33 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 1.8 MiB/s wr, 164 op/s 2024-06-10T01:08:45.711 INFO:tasks.workunit.client.0.smithi007.stdout: 1 102967 6.68 MB/sec warmup 86 sec latency 64.481 ms 2024-06-10T01:08:46.711 INFO:tasks.workunit.client.0.smithi007.stdout: 1 104320 6.67 MB/sec warmup 87 sec latency 13.350 ms 2024-06-10T01:08:47.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:47 smithi007 ceph-mon[32398]: pgmap v515: 129 pgs: 129 active+clean; 31 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 281 op/s 2024-06-10T01:08:47.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:47 smithi078 ceph-mon[36683]: pgmap v515: 129 pgs: 129 active+clean; 31 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 281 op/s 2024-06-10T01:08:47.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:47 smithi175 ceph-mon[36636]: pgmap v515: 129 pgs: 129 active+clean; 31 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 281 op/s 2024-06-10T01:08:47.711 INFO:tasks.workunit.client.0.smithi007.stdout: 1 106773 6.75 MB/sec warmup 88 sec latency 36.660 ms 2024-06-10T01:08:48.712 INFO:tasks.workunit.client.0.smithi007.stdout: 1 108835 6.76 MB/sec warmup 89 sec latency 13.123 ms 2024-06-10T01:08:49.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:49 smithi007 ceph-mon[32398]: pgmap v516: 129 pgs: 129 active+clean; 31 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 272 op/s 2024-06-10T01:08:49.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:49 smithi078 ceph-mon[36683]: pgmap v516: 129 pgs: 129 active+clean; 31 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 272 op/s 2024-06-10T01:08:49.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:49 smithi175 ceph-mon[36636]: pgmap v516: 129 pgs: 129 active+clean; 31 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 272 op/s 2024-06-10T01:08:49.712 INFO:tasks.workunit.client.0.smithi007.stdout: 1 110667 6.82 MB/sec warmup 90 sec latency 12.964 ms 2024-06-10T01:08:50.712 INFO:tasks.workunit.client.0.smithi007.stdout: 1 112927 6.88 MB/sec warmup 91 sec latency 30.088 ms 2024-06-10T01:08:51.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:51 smithi007 ceph-mon[32398]: pgmap v517: 129 pgs: 129 active+clean; 34 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 366 op/s 2024-06-10T01:08:51.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:51 smithi078 ceph-mon[36683]: pgmap v517: 129 pgs: 129 active+clean; 34 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 366 op/s 2024-06-10T01:08:51.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:51 smithi175 ceph-mon[36636]: pgmap v517: 129 pgs: 129 active+clean; 34 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 366 op/s 2024-06-10T01:08:51.712 INFO:tasks.workunit.client.0.smithi007.stdout: 1 114582 6.90 MB/sec warmup 92 sec latency 12.784 ms 2024-06-10T01:08:52.712 INFO:tasks.workunit.client.0.smithi007.stdout: 1 117367 7.01 MB/sec warmup 93 sec latency 13.069 ms 2024-06-10T01:08:53.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:53 smithi007 ceph-mon[32398]: pgmap v518: 129 pgs: 129 active+clean; 34 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 345 op/s 2024-06-10T01:08:53.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:53 smithi078 ceph-mon[36683]: pgmap v518: 129 pgs: 129 active+clean; 34 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 345 op/s 2024-06-10T01:08:53.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:53 smithi175 ceph-mon[36636]: pgmap v518: 129 pgs: 129 active+clean; 34 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 345 op/s 2024-06-10T01:08:53.712 INFO:tasks.workunit.client.0.smithi007.stdout: 1 119230 7.00 MB/sec warmup 94 sec latency 17.223 ms 2024-06-10T01:08:54.712 INFO:tasks.workunit.client.0.smithi007.stdout: 1 121227 7.07 MB/sec warmup 95 sec latency 11.275 ms 2024-06-10T01:08:55.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:55 smithi078 ceph-mon[36683]: pgmap v519: 129 pgs: 129 active+clean; 36 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 383 op/s 2024-06-10T01:08:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.393+0000 7f09caf89640 1 -- 172.21.15.7:0/2248613046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09c4100910 msgr2=0x7f09c4100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.393+0000 7f09caf89640 1 --2- 172.21.15.7:0/2248613046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09c4100910 0x7f09c4100d30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f09c0009900 tx=0x7f09c002a410 comp rx=0 tx=0).stop 2024-06-10T01:08:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.393+0000 7f09caf89640 1 -- 172.21.15.7:0/2248613046 shutdown_connections 2024-06-10T01:08:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.393+0000 7f09caf89640 1 --2- 172.21.15.7:0/2248613046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09c41024c0 0x7f09c4108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.393+0000 7f09caf89640 1 --2- 172.21.15.7:0/2248613046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09c4101b00 0x7f09c4101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.393+0000 7f09caf89640 1 --2- 172.21.15.7:0/2248613046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09c4100910 0x7f09c4100d30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.393+0000 7f09caf89640 1 -- 172.21.15.7:0/2248613046 >> 172.21.15.7:0/2248613046 conn(0x7f09c40fbe80 msgr2=0x7f09c40fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.393+0000 7f09caf89640 1 -- 172.21.15.7:0/2248613046 shutdown_connections 2024-06-10T01:08:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.393+0000 7f09caf89640 1 -- 172.21.15.7:0/2248613046 wait complete. 2024-06-10T01:08:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.394+0000 7f09caf89640 1 Processor -- start 2024-06-10T01:08:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.394+0000 7f09caf89640 1 -- start start 2024-06-10T01:08:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.394+0000 7f09caf89640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09c4100910 0x7f09c406ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.394+0000 7f09caf89640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09c4101b00 0x7f09c419ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.394+0000 7f09caf89640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09c41024c0 0x7f09c419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.394+0000 7f09caf89640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f09c410c560 con 0x7f09c4100910 2024-06-10T01:08:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.394+0000 7f09caf89640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f09c410c3e0 con 0x7f09c41024c0 2024-06-10T01:08:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.394+0000 7f09caf89640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f09c410c6e0 con 0x7f09c4101b00 2024-06-10T01:08:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.394+0000 7f09c8d25640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09c4100910 0x7f09c406ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.394+0000 7f09c8d25640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09c4100910 0x7f09c406ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47038/0 (socket says 172.21.15.7:47038) 2024-06-10T01:08:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.394+0000 7f09c8d25640 1 -- 172.21.15.7:0/1676551735 learned_addr learned my addr 172.21.15.7:0/1676551735 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.394+0000 7f09bbfff640 1 --2- 172.21.15.7:0/1676551735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09c4101b00 0x7f09c419ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.394+0000 7f09c9526640 1 --2- 172.21.15.7:0/1676551735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09c41024c0 0x7f09c419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.395+0000 7f09c8d25640 1 -- 172.21.15.7:0/1676551735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09c4101b00 msgr2=0x7f09c419ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.395+0000 7f09c8d25640 1 --2- 172.21.15.7:0/1676551735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09c4101b00 0x7f09c419ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.395+0000 7f09c8d25640 1 -- 172.21.15.7:0/1676551735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09c41024c0 msgr2=0x7f09c419f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.395+0000 7f09c8d25640 1 --2- 172.21.15.7:0/1676551735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09c41024c0 0x7f09c419f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.395+0000 7f09c8d25640 1 -- 172.21.15.7:0/1676551735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c419f9b0 con 0x7f09c4100910 2024-06-10T01:08:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.395+0000 7f09c8d25640 1 --2- 172.21.15.7:0/1676551735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09c4100910 0x7f09c406ab50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f09c0031610 tx=0x7f09c0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.395+0000 7f09b9ffb640 1 -- 172.21.15.7:0/1676551735 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c003f030 con 0x7f09c4100910 2024-06-10T01:08:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.395+0000 7f09caf89640 1 -- 172.21.15.7:0/1676551735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c419fbe0 con 0x7f09c4100910 2024-06-10T01:08:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.395+0000 7f09b9ffb640 1 -- 172.21.15.7:0/1676551735 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09c0003ad0 con 0x7f09c4100910 2024-06-10T01:08:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.395+0000 7f09b9ffb640 1 -- 172.21.15.7:0/1676551735 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c0035070 con 0x7f09c4100910 2024-06-10T01:08:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.395+0000 7f09caf89640 1 -- 172.21.15.7:0/1676551735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c41a0120 con 0x7f09c4100910 2024-06-10T01:08:55.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.398+0000 7f09cdd75740 1 Processor -- start 2024-06-10T01:08:55.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.398+0000 7f09b9ffb640 1 -- 172.21.15.7:0/1676551735 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f09c0042030 con 0x7f09c4100910 2024-06-10T01:08:55.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.398+0000 7f09cdd75740 1 -- start start 2024-06-10T01:08:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.398+0000 7f09cdd75740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651ab580010 0x5651ab580430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.398+0000 7f09cdd75740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651ab580a00 0x5651ab585500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.398+0000 7f09cdd75740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ab585b70 0x5651ab588000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.398+0000 7f09cdd75740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5651ab4e34f0 con 0x5651ab585b70 2024-06-10T01:08:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.398+0000 7f09cdd75740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5651ab481870 con 0x5651ab580a00 2024-06-10T01:08:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.398+0000 7f09cdd75740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5651ab4df4b0 con 0x5651ab580010 2024-06-10T01:08:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.398+0000 7f09bbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ab585b70 0x5651ab588000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.398+0000 7f09bbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ab585b70 0x5651ab588000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47044/0 (socket says 172.21.15.7:47044) 2024-06-10T01:08:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.398+0000 7f09c9526640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651ab580010 0x5651ab580430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.398+0000 7f09bbfff640 1 -- 172.21.15.7:0/3713168199 learned_addr learned my addr 172.21.15.7:0/3713168199 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.398+0000 7f09c8d25640 1 --2- 172.21.15.7:0/3713168199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651ab580a00 0x5651ab585500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.398+0000 7f09bbfff640 1 -- 172.21.15.7:0/3713168199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651ab580010 msgr2=0x5651ab580430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.398+0000 7f09bbfff640 1 --2- 172.21.15.7:0/3713168199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651ab580010 0x5651ab580430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.398+0000 7f09b9ffb640 1 --2- 172.21.15.7:0/1676551735 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f09ac0a2b80 0x7f09ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.398+0000 7f09bbfff640 1 -- 172.21.15.7:0/3713168199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651ab580a00 msgr2=0x5651ab585500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.398+0000 7f09bbfff640 1 --2- 172.21.15.7:0/3713168199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651ab580a00 0x5651ab585500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.398+0000 7f09bbfff640 1 -- 172.21.15.7:0/3713168199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651ab49d9b0 con 0x5651ab585b70 2024-06-10T01:08:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.398+0000 7f09c9526640 1 --2- 172.21.15.7:0/3713168199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651ab580010 0x5651ab580430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.398+0000 7f09c8d25640 1 --2- 172.21.15.7:0/3713168199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651ab580a00 0x5651ab585500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:08:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.399+0000 7f09c9526640 1 --2- 172.21.15.7:0/1676551735 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f09ac0a2b80 0x7f09ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.399+0000 7f09b9ffb640 1 -- 172.21.15.7:0/1676551735 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f09c00efa90 con 0x7f09c4100910 2024-06-10T01:08:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.399+0000 7f09bbfff640 1 --2- 172.21.15.7:0/3713168199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ab585b70 0x5651ab588000 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f09b400bd10 tx=0x7f09b40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3638f0fcd015d1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.399+0000 7f09aaffd640 1 -- 172.21.15.7:0/3713168199 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b4005220 con 0x5651ab585b70 2024-06-10T01:08:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.399+0000 7f09aaffd640 1 -- 172.21.15.7:0/3713168199 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09b4004c80 con 0x5651ab585b70 2024-06-10T01:08:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.399+0000 7f09aaffd640 1 -- 172.21.15.7:0/3713168199 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b4011840 con 0x5651ab585b70 2024-06-10T01:08:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.400+0000 7f09cdd75740 1 -- 172.21.15.7:0/3713168199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ab585b70 msgr2=0x5651ab588000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.400+0000 7f09c9526640 1 --2- 172.21.15.7:0/1676551735 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f09ac0a2b80 0x7f09ac0a5040 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f09bc009b80 tx=0x7f09bc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.400+0000 7f09cdd75740 1 --2- 172.21.15.7:0/3713168199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ab585b70 0x5651ab588000 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f09b400bd10 tx=0x7f09b40107a0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.400+0000 7f09cdd75740 1 -- 172.21.15.7:0/3713168199 shutdown_connections 2024-06-10T01:08:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.400+0000 7f09cdd75740 1 --2- 172.21.15.7:0/3713168199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ab585b70 0x5651ab588000 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.400+0000 7f09cdd75740 1 --2- 172.21.15.7:0/3713168199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651ab580a00 0x5651ab585500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.400+0000 7f09cdd75740 1 --2- 172.21.15.7:0/3713168199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651ab580010 0x5651ab580430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.400+0000 7f09cdd75740 1 -- 172.21.15.7:0/3713168199 >> 172.21.15.7:0/3713168199 conn(0x5651ab57b680 msgr2=0x5651ab57ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.400+0000 7f09cdd75740 1 -- 172.21.15.7:0/3713168199 shutdown_connections 2024-06-10T01:08:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.400+0000 7f09cdd75740 1 -- 172.21.15.7:0/3713168199 wait complete. 2024-06-10T01:08:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.400+0000 7f09cdd75740 1 -- start start 2024-06-10T01:08:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.400+0000 7f09cdd75740 1 Processor -- start 2024-06-10T01:08:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.400+0000 7f09cdd75740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651ab580010 0x5651ab58a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.400+0000 7f09cdd75740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ab580a00 0x5651ab58a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.400+0000 7f09cdd75740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651ab585b70 0x5651ab591830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.400+0000 7f09cdd75740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5651ab588b20 con 0x5651ab580a00 2024-06-10T01:08:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.400+0000 7f09cdd75740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5651ab46cae0 con 0x5651ab585b70 2024-06-10T01:08:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.400+0000 7f09cdd75740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5651ab588ca0 con 0x5651ab580010 2024-06-10T01:08:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.401+0000 7f09cdd75740 10 client.0 _ll_register_callbacks cb 0x5651ab4a59b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.401+0000 7f09bbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ab580a00 0x5651ab58a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.401+0000 7f09bbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ab580a00 0x5651ab58a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47050/0 (socket says 172.21.15.7:47050) 2024-06-10T01:08:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.401+0000 7f09bbfff640 1 -- 172.21.15.7:0/2368524761 learned_addr learned my addr 172.21.15.7:0/2368524761 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.401+0000 7f09c8d25640 1 --2- 172.21.15.7:0/2368524761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651ab580010 0x5651ab58a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.401+0000 7f09c9526640 1 --2- 172.21.15.7:0/2368524761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651ab585b70 0x5651ab591830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.401+0000 7f09c9526640 1 -- 172.21.15.7:0/2368524761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651ab580010 msgr2=0x5651ab58a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.401+0000 7f09c9526640 1 --2- 172.21.15.7:0/2368524761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651ab580010 0x5651ab58a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.401+0000 7f09c9526640 1 -- 172.21.15.7:0/2368524761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ab580a00 msgr2=0x5651ab58a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.401+0000 7f09c9526640 1 --2- 172.21.15.7:0/2368524761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ab580a00 0x5651ab58a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.401+0000 7f09c9526640 1 -- 172.21.15.7:0/2368524761 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651ab1f7210 con 0x5651ab585b70 2024-06-10T01:08:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.401+0000 7f09c9526640 1 --2- 172.21.15.7:0/2368524761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651ab585b70 0x5651ab591830 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f09bc005fd0 tx=0x7f09bc004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9fcf1dfb2e656b8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.401+0000 7f09a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:08:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.401+0000 7f09a3fff640 1 -- 172.21.15.7:0/2368524761 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09bc003b80 con 0x5651ab585b70 2024-06-10T01:08:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.401+0000 7f09a3fff640 1 -- 172.21.15.7:0/2368524761 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f09bc003d20 con 0x5651ab585b70 2024-06-10T01:08:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.401+0000 7f09cdd75740 1 -- 172.21.15.7:0/2368524761 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09b4004c80 con 0x5651ab585b70 2024-06-10T01:08:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.402+0000 7f09a3fff640 1 -- 172.21.15.7:0/2368524761 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f09bc004050 con 0x5651ab585b70 2024-06-10T01:08:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.402+0000 7f09cdd75740 10 client.25772 fetch_fsmap learned FSMap version 30 2024-06-10T01:08:55.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.402+0000 7f09cdd75740 1 -- 172.21.15.7:0/2368524761 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5651ab2c3620 con 0x5651ab585b70 2024-06-10T01:08:55.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.402+0000 7f09a3fff640 1 -- 172.21.15.7:0/2368524761 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09b4004c80 con 0x5651ab585b70 2024-06-10T01:08:55.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.402+0000 7f09a3fff640 1 -- 172.21.15.7:0/2368524761 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f09bc027020 con 0x5651ab585b70 2024-06-10T01:08:55.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.402+0000 7f09cdd75740 10 client.25772 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:08:55.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.402+0000 7f09cdd75740 10 client.25772 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:55.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.402+0000 7f09cdd75740 20 client.25772 populate_metadata read hostname 'smithi007' 2024-06-10T01:08:55.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.402+0000 7f09cdd75740 1 --2- 172.21.15.7:0/2368524761 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5651ab5961f0 0x5651ab598670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:55.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.402+0000 7f09cdd75740 4 client.25772 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:08:55.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.402+0000 7f09cdd75740 1 -- 172.21.15.7:0/2368524761 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651ab4f6f10 con 0x5651ab5961f0 2024-06-10T01:08:55.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.402+0000 7f09c8d25640 1 --2- 172.21.15.7:0/2368524761 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5651ab5961f0 0x5651ab598670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:55.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.402+0000 7f09c8d25640 1 --2- 172.21.15.7:0/2368524761 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5651ab5961f0 0x5651ab598670 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42e8b3bb6b3df048 server_cookie=33a7a019b5624dcd in_seq=0 out_seq=0 2024-06-10T01:08:55.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.402+0000 7f09a3fff640 10 client.25772 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:55.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.403+0000 7f09a3fff640 1 -- 172.21.15.7:0/2368524761 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5651ab4f6f10 con 0x5651ab5961f0 2024-06-10T01:08:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.403+0000 7f09a3fff640 10 client.25772 handle_command_reply: tid=0 2024-06-10T01:08:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.403+0000 7f09cdd75740 1 client.25772 shutdown 2024-06-10T01:08:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.403+0000 7f09cdd75740 1 -- 172.21.15.7:0/2368524761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651ab585b70 msgr2=0x5651ab591830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.403+0000 7f09cdd75740 1 --2- 172.21.15.7:0/2368524761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651ab585b70 0x5651ab591830 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f09bc005fd0 tx=0x7f09bc004ab0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.403+0000 7f09cdd75740 1 -- 172.21.15.7:0/2368524761 shutdown_connections 2024-06-10T01:08:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.403+0000 7f09cdd75740 1 --2- 172.21.15.7:0/2368524761 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5651ab5961f0 0x5651ab598670 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.403+0000 7f09cdd75740 1 --2- 172.21.15.7:0/2368524761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651ab585b70 0x5651ab591830 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.403+0000 7f09cdd75740 1 --2- 172.21.15.7:0/2368524761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ab580a00 0x5651ab58a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.403+0000 7f09cdd75740 1 --2- 172.21.15.7:0/2368524761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651ab580010 0x5651ab58a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.403+0000 7f09a3fff640 0 client.25772 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.403+0000 7f09cdd75740 1 -- 172.21.15.7:0/2368524761 >> 172.21.15.7:0/2368524761 conn(0x5651ab57b680 msgr2=0x5651ab57a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.404+0000 7f09cdd75740 1 -- 172.21.15.7:0/2368524761 shutdown_connections 2024-06-10T01:08:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.404+0000 7f09cdd75740 1 -- 172.21.15.7:0/2368524761 wait complete. 2024-06-10T01:08:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.404+0000 7f09cdd75740 20 client.25772 trim_cache size 0 max 16384 2024-06-10T01:08:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.419+0000 7f09cdd75740 1 Processor -- start 2024-06-10T01:08:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.419+0000 7f09cdd75740 1 -- start start 2024-06-10T01:08:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.419+0000 7f09cdd75740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ab57b680 0x5651ab589d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.419+0000 7f09cdd75740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651ab580010 0x5651ab5917f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.419+0000 7f09cdd75740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651ab580a00 0x5651ab57ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.419+0000 7f09cdd75740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f09c41acf30 con 0x5651ab57b680 2024-06-10T01:08:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.419+0000 7f09cdd75740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f09c41aa7a0 con 0x5651ab580010 2024-06-10T01:08:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.419+0000 7f09cdd75740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f09c41a85f0 con 0x5651ab580a00 2024-06-10T01:08:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.420+0000 7f09c8d25640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ab57b680 0x5651ab589d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.420+0000 7f09bbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651ab580010 0x5651ab5917f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.420+0000 7f09bbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651ab580010 0x5651ab5917f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:47154/0 (socket says 172.21.15.7:47154) 2024-06-10T01:08:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.420+0000 7f09bbfff640 1 -- 172.21.15.7:0/1923078276 learned_addr learned my addr 172.21.15.7:0/1923078276 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.420+0000 7f09c8d25640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ab57b680 0x5651ab589d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47060/0 (socket says 172.21.15.7:47060) 2024-06-10T01:08:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.420+0000 7f09c9526640 1 --2- 172.21.15.7:0/1923078276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651ab580a00 0x5651ab57ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.420+0000 7f09bbfff640 1 -- 172.21.15.7:0/1923078276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651ab580a00 msgr2=0x5651ab57ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.420+0000 7f09bbfff640 1 --2- 172.21.15.7:0/1923078276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651ab580a00 0x5651ab57ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.420+0000 7f09bbfff640 1 -- 172.21.15.7:0/1923078276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ab57b680 msgr2=0x5651ab589d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.420+0000 7f09bbfff640 1 --2- 172.21.15.7:0/1923078276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ab57b680 0x5651ab589d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.420+0000 7f09bbfff640 1 -- 172.21.15.7:0/1923078276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651ab57b4b0 con 0x5651ab580010 2024-06-10T01:08:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.420+0000 7f09c8d25640 1 --2- 172.21.15.7:0/1923078276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ab57b680 0x5651ab589d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.420+0000 7f09bbfff640 1 --2- 172.21.15.7:0/1923078276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651ab580010 0x5651ab5917f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f09b4010ca0 tx=0x7f09b4005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=997f790619c976ba server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.420+0000 7f09aa7fc640 1 -- 172.21.15.7:0/1923078276 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b4004220 con 0x5651ab580010 2024-06-10T01:08:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.420+0000 7f09aa7fc640 1 -- 172.21.15.7:0/1923078276 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09b40043c0 con 0x5651ab580010 2024-06-10T01:08:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.420+0000 7f09aa7fc640 1 -- 172.21.15.7:0/1923078276 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b4004d60 con 0x5651ab580010 2024-06-10T01:08:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.421+0000 7f09cdd75740 1 -- 172.21.15.7:0/1923078276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651ab580010 msgr2=0x5651ab5917f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.421+0000 7f09cdd75740 1 --2- 172.21.15.7:0/1923078276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651ab580010 0x5651ab5917f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f09b4010ca0 tx=0x7f09b4005fb0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.421+0000 7f09cdd75740 1 -- 172.21.15.7:0/1923078276 shutdown_connections 2024-06-10T01:08:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.421+0000 7f09cdd75740 1 --2- 172.21.15.7:0/1923078276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651ab580a00 0x5651ab57ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.421+0000 7f09cdd75740 1 --2- 172.21.15.7:0/1923078276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651ab580010 0x5651ab5917f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.421+0000 7f09cdd75740 1 --2- 172.21.15.7:0/1923078276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ab57b680 0x5651ab589d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.421+0000 7f09cdd75740 1 -- 172.21.15.7:0/1923078276 >> 172.21.15.7:0/1923078276 conn(0x5651ab585640 msgr2=0x5651ab57e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.421+0000 7f09cdd75740 1 -- 172.21.15.7:0/1923078276 shutdown_connections 2024-06-10T01:08:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.421+0000 7f09cdd75740 1 -- 172.21.15.7:0/1923078276 wait complete. 2024-06-10T01:08:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.421+0000 7f09cdd75740 1 -- start start 2024-06-10T01:08:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.421+0000 7f09cdd75740 1 Processor -- start 2024-06-10T01:08:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.422+0000 7f09cdd75740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ab57b680 0x5651ab58ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.422+0000 7f09cdd75740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651ab580a00 0x5651ab58ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.422+0000 7f09cdd75740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651ab585b70 0x5651ab58ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.422+0000 7f09cdd75740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f09c41a8c60 con 0x5651ab57b680 2024-06-10T01:08:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.422+0000 7f09cdd75740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f09c41aa0d0 con 0x5651ab580a00 2024-06-10T01:08:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.422+0000 7f09cdd75740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f09c41a9330 con 0x5651ab585b70 2024-06-10T01:08:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.422+0000 7f09cdd75740 10 client.0 _ll_register_callbacks cb 0x7f09bc009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.422+0000 7f09c8d25640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ab57b680 0x5651ab58ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.422+0000 7f09c8d25640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ab57b680 0x5651ab58ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47072/0 (socket says 172.21.15.7:47072) 2024-06-10T01:08:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.422+0000 7f09c8d25640 1 -- 172.21.15.7:0/2652730003 learned_addr learned my addr 172.21.15.7:0/2652730003 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:08:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.422+0000 7f09c9526640 1 --2- 172.21.15.7:0/2652730003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651ab585b70 0x5651ab58ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.422+0000 7f09bbfff640 1 --2- 172.21.15.7:0/2652730003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651ab580a00 0x5651ab58ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.422+0000 7f09c8d25640 1 -- 172.21.15.7:0/2652730003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651ab585b70 msgr2=0x5651ab58ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.422+0000 7f09c8d25640 1 --2- 172.21.15.7:0/2652730003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651ab585b70 0x5651ab58ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.422+0000 7f09c8d25640 1 -- 172.21.15.7:0/2652730003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651ab580a00 msgr2=0x5651ab58ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.422+0000 7f09c8d25640 1 --2- 172.21.15.7:0/2652730003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651ab580a00 0x5651ab58ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.422+0000 7f09c8d25640 1 -- 172.21.15.7:0/2652730003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651ab590560 con 0x5651ab57b680 2024-06-10T01:08:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.422+0000 7f09c8d25640 1 --2- 172.21.15.7:0/2652730003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ab57b680 0x5651ab58ea20 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f09c00ed580 tx=0x7f09c002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c134b373d15be4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.422+0000 7f09a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:08:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.423+0000 7f09a3fff640 1 -- 172.21.15.7:0/2652730003 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c0049040 con 0x5651ab57b680 2024-06-10T01:08:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.423+0000 7f09a3fff640 1 -- 172.21.15.7:0/2652730003 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f09c003b460 con 0x5651ab57b680 2024-06-10T01:08:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.423+0000 7f09cdd75740 1 -- 172.21.15.7:0/2652730003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09b40043c0 con 0x5651ab57b680 2024-06-10T01:08:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.423+0000 7f09a3fff640 1 -- 172.21.15.7:0/2652730003 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f09c00d6070 con 0x5651ab57b680 2024-06-10T01:08:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.423+0000 7f09cdd75740 10 client.16254 fetch_fsmap learned FSMap version 30 2024-06-10T01:08:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.423+0000 7f09cdd75740 1 -- 172.21.15.7:0/2652730003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5651ab591940 con 0x5651ab57b680 2024-06-10T01:08:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.423+0000 7f09a3fff640 1 -- 172.21.15.7:0/2652730003 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09b40043c0 con 0x5651ab57b680 2024-06-10T01:08:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.423+0000 7f09a3fff640 1 -- 172.21.15.7:0/2652730003 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f09c00b85e0 con 0x5651ab57b680 2024-06-10T01:08:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.423+0000 7f09cdd75740 10 client.16254 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:08:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.423+0000 7f09cdd75740 10 client.16254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.423+0000 7f09cdd75740 20 client.16254 populate_metadata read hostname 'smithi007' 2024-06-10T01:08:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.423+0000 7f09cdd75740 1 --2- 172.21.15.7:0/2652730003 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5651ab601500 0x5651ab603920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.423+0000 7f09cdd75740 4 client.16254 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:08:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.423+0000 7f09cdd75740 1 -- 172.21.15.7:0/2652730003 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5651ab4f0c20 con 0x5651ab601500 2024-06-10T01:08:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.423+0000 7f09bbfff640 1 --2- 172.21.15.7:0/2652730003 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5651ab601500 0x5651ab603920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.424+0000 7f09bbfff640 1 --2- 172.21.15.7:0/2652730003 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5651ab601500 0x5651ab603920 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4da1876a1932093 server_cookie=e2fdb2586f9c5068 in_seq=0 out_seq=0 2024-06-10T01:08:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.424+0000 7f09a3fff640 10 client.16254 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:55.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:55 smithi175 ceph-mon[36636]: pgmap v519: 129 pgs: 129 active+clean; 36 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 383 op/s 2024-06-10T01:08:55.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:55 smithi007 ceph-mon[32398]: pgmap v519: 129 pgs: 129 active+clean; 36 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 383 op/s 2024-06-10T01:08:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.565+0000 7f09a3fff640 1 -- 172.21.15.7:0/2652730003 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5651ab4f0c20 con 0x5651ab601500 2024-06-10T01:08:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.565+0000 7f09a3fff640 10 client.16254 handle_command_reply: tid=0 2024-06-10T01:08:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.565+0000 7f09cdd75740 1 client.16254 shutdown 2024-06-10T01:08:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.566+0000 7f09cdd75740 1 -- 172.21.15.7:0/2652730003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ab57b680 msgr2=0x5651ab58ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.566+0000 7f09cdd75740 1 --2- 172.21.15.7:0/2652730003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ab57b680 0x5651ab58ea20 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f09c00ed580 tx=0x7f09c002a410 comp rx=0 tx=0).stop 2024-06-10T01:08:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.566+0000 7f09cdd75740 1 -- 172.21.15.7:0/2652730003 shutdown_connections 2024-06-10T01:08:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.566+0000 7f09cdd75740 1 --2- 172.21.15.7:0/2652730003 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5651ab601500 0x5651ab603920 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.566+0000 7f09cdd75740 1 --2- 172.21.15.7:0/2652730003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651ab585b70 0x5651ab58ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.566+0000 7f09cdd75740 1 --2- 172.21.15.7:0/2652730003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651ab580a00 0x5651ab58ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.566+0000 7f09a3fff640 0 client.16254 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:08:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.566+0000 7f09cdd75740 1 --2- 172.21.15.7:0/2652730003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ab57b680 0x5651ab58ea20 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.566+0000 7f09cdd75740 1 -- 172.21.15.7:0/2652730003 >> 172.21.15.7:0/2652730003 conn(0x5651ab585640 msgr2=0x5651ab57ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.566+0000 7f09cdd75740 1 -- 172.21.15.7:0/2652730003 shutdown_connections 2024-06-10T01:08:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.566+0000 7f09cdd75740 1 -- 172.21.15.7:0/2652730003 wait complete. 2024-06-10T01:08:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.566+0000 7f09cdd75740 20 client.16254 trim_cache size 0 max 16384 2024-06-10T01:08:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:08:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:08:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:08:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.567+0000 7f09caf89640 1 -- 172.21.15.7:0/1676551735 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f09ac0a2b80 msgr2=0x7f09ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.567+0000 7f09caf89640 1 --2- 172.21.15.7:0/1676551735 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f09ac0a2b80 0x7f09ac0a5040 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f09bc009b80 tx=0x7f09bc009500 comp rx=0 tx=0).stop 2024-06-10T01:08:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.567+0000 7f09caf89640 1 -- 172.21.15.7:0/1676551735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09c4100910 msgr2=0x7f09c406ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.567+0000 7f09caf89640 1 --2- 172.21.15.7:0/1676551735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09c4100910 0x7f09c406ab50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f09c0031610 tx=0x7f09c0004270 comp rx=0 tx=0).stop 2024-06-10T01:08:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.567+0000 7f09caf89640 1 -- 172.21.15.7:0/1676551735 shutdown_connections 2024-06-10T01:08:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.567+0000 7f09caf89640 1 --2- 172.21.15.7:0/1676551735 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f09ac0a2b80 0x7f09ac0a5040 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.567+0000 7f09caf89640 1 --2- 172.21.15.7:0/1676551735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09c41024c0 0x7f09c419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.567+0000 7f09caf89640 1 --2- 172.21.15.7:0/1676551735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09c4101b00 0x7f09c419ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.567+0000 7f09caf89640 1 --2- 172.21.15.7:0/1676551735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09c4100910 0x7f09c406ab50 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.567+0000 7f09caf89640 1 -- 172.21.15.7:0/1676551735 >> 172.21.15.7:0/1676551735 conn(0x7f09c40fbe80 msgr2=0x7f09c4104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.567+0000 7f09caf89640 1 -- 172.21.15.7:0/1676551735 shutdown_connections 2024-06-10T01:08:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:08:55.567+0000 7f09caf89640 1 -- 172.21.15.7:0/1676551735 wait complete. 2024-06-10T01:08:55.712 INFO:tasks.workunit.client.0.smithi007.stdout: 1 123547 7.12 MB/sec warmup 96 sec latency 57.824 ms 2024-06-10T01:08:56.712 INFO:tasks.workunit.client.0.smithi007.stdout: 1 125251 7.14 MB/sec warmup 97 sec latency 17.210 ms 2024-06-10T01:08:57.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:57 smithi078 ceph-mon[36683]: pgmap v520: 129 pgs: 129 active+clean; 38 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 418 op/s 2024-06-10T01:08:57.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:57 smithi175 ceph-mon[36636]: pgmap v520: 129 pgs: 129 active+clean; 38 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 418 op/s 2024-06-10T01:08:57.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:57 smithi007 ceph-mon[32398]: pgmap v520: 129 pgs: 129 active+clean; 38 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 418 op/s 2024-06-10T01:08:57.712 INFO:tasks.workunit.client.0.smithi007.stdout: 1 128007 7.24 MB/sec warmup 98 sec latency 12.856 ms 2024-06-10T01:08:58.712 INFO:tasks.workunit.client.0.smithi007.stdout: 1 129790 7.22 MB/sec warmup 99 sec latency 20.720 ms 2024-06-10T01:08:59.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:08:59 smithi078 ceph-mon[36683]: pgmap v521: 129 pgs: 129 active+clean; 39 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 331 op/s 2024-06-10T01:08:59.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:08:59 smithi175 ceph-mon[36636]: pgmap v521: 129 pgs: 129 active+clean; 39 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 331 op/s 2024-06-10T01:08:59.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:08:59 smithi007 ceph-mon[32398]: pgmap v521: 129 pgs: 129 active+clean; 39 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 331 op/s 2024-06-10T01:08:59.544 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:08:59.712 INFO:tasks.workunit.client.0.smithi007.stdout: 1 131869 7.29 MB/sec warmup 100 sec latency 11.631 ms 2024-06-10T01:08:59.804 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:09:00.712 INFO:tasks.workunit.client.0.smithi007.stdout: 1 134574 7.36 MB/sec warmup 101 sec latency 20.712 ms 2024-06-10T01:09:01.217 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:01 smithi007 ceph-mon[32398]: pgmap v522: 129 pgs: 129 active+clean; 38 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 422 op/s 2024-06-10T01:09:01.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:01 smithi078 ceph-mon[36683]: pgmap v522: 129 pgs: 129 active+clean; 38 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 422 op/s 2024-06-10T01:09:01.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:01 smithi175 ceph-mon[36636]: pgmap v522: 129 pgs: 129 active+clean; 38 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 422 op/s 2024-06-10T01:09:01.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.617+0000 7fe6cadba700 1 -- 172.21.15.7:0/41311030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6c4107840 msgr2=0x7fe6c410f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:01.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.617+0000 7fe6cadba700 1 --2- 172.21.15.7:0/41311030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6c4107840 0x7fe6c410f3c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009fc0 tx=0x7fe6bc00b330 comp rx=0 tx=0).stop 2024-06-10T01:09:01.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.618+0000 7fe6cadba700 1 -- 172.21.15.7:0/41311030 shutdown_connections 2024-06-10T01:09:01.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.618+0000 7fe6cadba700 1 --2- 172.21.15.7:0/41311030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6c4107840 0x7fe6c410f3c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.618+0000 7fe6cadba700 1 --2- 172.21.15.7:0/41311030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c4102740 0x7fe6c4102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.618+0000 7fe6cadba700 1 --2- 172.21.15.7:0/41311030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6c41030f0 0x7fe6c4107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.618+0000 7fe6cadba700 1 -- 172.21.15.7:0/41311030 >> 172.21.15.7:0/41311030 conn(0x7fe6c40795f0 msgr2=0x7fe6c4079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:01.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.618+0000 7fe6cadba700 1 -- 172.21.15.7:0/41311030 shutdown_connections 2024-06-10T01:09:01.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.618+0000 7fe6cadba700 1 -- 172.21.15.7:0/41311030 wait complete. 2024-06-10T01:09:01.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.618+0000 7fe6cadba700 1 Processor -- start 2024-06-10T01:09:01.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.618+0000 7fe6cadba700 1 -- start start 2024-06-10T01:09:01.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.619+0000 7fe6cadba700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6c4102740 0x7fe6c4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:01.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.619+0000 7fe6cadba700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6c41030f0 0x7fe6c4074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:01.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.619+0000 7fe6cadba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c4107840 0x7fe6c4075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:01.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.619+0000 7fe6cadba700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c4112760 con 0x7fe6c4107840 2024-06-10T01:09:01.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.619+0000 7fe6cadba700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c41125e0 con 0x7fe6c4102740 2024-06-10T01:09:01.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.619+0000 7fe6cadba700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c41128e0 con 0x7fe6c41030f0 2024-06-10T01:09:01.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.619+0000 7fe6c8b56700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6c4102740 0x7fe6c4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:01.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.619+0000 7fe6c9357700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c4107840 0x7fe6c4075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:01.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.619+0000 7fe6c8b56700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6c4102740 0x7fe6c4074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:47182/0 (socket says 172.21.15.7:47182) 2024-06-10T01:09:01.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.619+0000 7fe6c3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6c41030f0 0x7fe6c4074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:01.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.619+0000 7fe6c8b56700 1 -- 172.21.15.7:0/2814254771 learned_addr learned my addr 172.21.15.7:0/2814254771 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.619+0000 7fe6c8b56700 1 -- 172.21.15.7:0/2814254771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6c41030f0 msgr2=0x7fe6c4074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.619+0000 7fe6c8b56700 1 --2- 172.21.15.7:0/2814254771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6c41030f0 0x7fe6c4074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.619+0000 7fe6c8b56700 1 -- 172.21.15.7:0/2814254771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c4107840 msgr2=0x7fe6c4075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.619+0000 7fe6c8b56700 1 --2- 172.21.15.7:0/2814254771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c4107840 0x7fe6c4075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.619+0000 7fe6c8b56700 1 -- 172.21.15.7:0/2814254771 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c41a7040 con 0x7fe6c4102740 2024-06-10T01:09:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.619+0000 7fe6c8b56700 1 --2- 172.21.15.7:0/2814254771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6c4102740 0x7fe6c4074790 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe6b800e9a0 tx=0x7fe6b800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.619+0000 7fe6c1ffb700 1 -- 172.21.15.7:0/2814254771 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b800c9c0 con 0x7fe6c4102740 2024-06-10T01:09:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.619+0000 7fe6c1ffb700 1 -- 172.21.15.7:0/2814254771 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6b800cb60 con 0x7fe6c4102740 2024-06-10T01:09:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.619+0000 7fe6cadba700 1 -- 172.21.15.7:0/2814254771 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6c41a7330 con 0x7fe6c4102740 2024-06-10T01:09:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.619+0000 7fe6c1ffb700 1 -- 172.21.15.7:0/2814254771 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8018660 con 0x7fe6c4102740 2024-06-10T01:09:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.620+0000 7fe6cadba700 1 -- 172.21.15.7:0/2814254771 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6c41a7870 con 0x7fe6c4102740 2024-06-10T01:09:01.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.622+0000 7fe6c1ffb700 1 -- 172.21.15.7:0/2814254771 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fe6b8010430 con 0x7fe6c4102740 2024-06-10T01:09:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.622+0000 7fe6c1ffb700 1 --2- 172.21.15.7:0/2814254771 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe6b40a2c20 0x7fe6b40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.623+0000 7fe6c3fff700 1 --2- 172.21.15.7:0/2814254771 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe6b40a2c20 0x7fe6b40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.623+0000 7fe6c1ffb700 1 -- 172.21.15.7:0/2814254771 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fe6b80cf5c0 con 0x7fe6c4102740 2024-06-10T01:09:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.623+0000 7fe6c3fff700 1 --2- 172.21.15.7:0/2814254771 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe6b40a2c20 0x7fe6b40a50e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4108b90 tx=0x7fe6b0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.623+0000 7fe6d563db80 1 Processor -- start 2024-06-10T01:09:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.624+0000 7fe6d563db80 1 -- start start 2024-06-10T01:09:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.624+0000 7fe6d563db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627de0e6640 0x5627de0e6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.624+0000 7fe6d563db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627de0e6f60 0x5627de0e09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.624+0000 7fe6d563db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627de0e0f20 0x5627de0e1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.624+0000 7fe6d563db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5627de032810 con 0x5627de0e6f60 2024-06-10T01:09:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.624+0000 7fe6d563db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5627ddfda720 con 0x5627de0e6640 2024-06-10T01:09:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.624+0000 7fe6d563db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5627de02e7d0 con 0x5627de0e0f20 2024-06-10T01:09:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.624+0000 7fe6c3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627de0e6f60 0x5627de0e09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.624+0000 7fe6c3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627de0e6f60 0x5627de0e09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47110/0 (socket says 172.21.15.7:47110) 2024-06-10T01:09:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.624+0000 7fe6c8b56700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627de0e6640 0x5627de0e6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.624+0000 7fe6c3fff700 1 -- 172.21.15.7:0/1953188048 learned_addr learned my addr 172.21.15.7:0/1953188048 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.624+0000 7fe6c9357700 1 --2- 172.21.15.7:0/1953188048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627de0e0f20 0x5627de0e1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.624+0000 7fe6c8b56700 1 -- 172.21.15.7:0/1953188048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627de0e0f20 msgr2=0x5627de0e1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:01.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.624+0000 7fe6c8b56700 1 --2- 172.21.15.7:0/1953188048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627de0e0f20 0x5627de0e1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.624+0000 7fe6c8b56700 1 -- 172.21.15.7:0/1953188048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627de0e6f60 msgr2=0x5627de0e09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:01.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.624+0000 7fe6c8b56700 1 --2- 172.21.15.7:0/1953188048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627de0e6f60 0x5627de0e09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.624+0000 7fe6c8b56700 1 -- 172.21.15.7:0/1953188048 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627ddfd0f50 con 0x5627de0e6640 2024-06-10T01:09:01.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.624+0000 7fe6c8b56700 1 --2- 172.21.15.7:0/1953188048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627de0e6640 0x5627de0e6a20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe6b800e970 tx=0x7fe6b8009820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4905e0e11e0270b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:01.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.625+0000 7fe6aeffd700 1 -- 172.21.15.7:0/1953188048 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b80a46b0 con 0x5627de0e6640 2024-06-10T01:09:01.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.625+0000 7fe6aeffd700 1 -- 172.21.15.7:0/1953188048 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6b8099d40 con 0x5627de0e6640 2024-06-10T01:09:01.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.625+0000 7fe6aeffd700 1 -- 172.21.15.7:0/1953188048 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8010720 con 0x5627de0e6640 2024-06-10T01:09:01.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.626+0000 7fe6d563db80 1 -- 172.21.15.7:0/1953188048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627de0e6640 msgr2=0x5627de0e6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:01.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.626+0000 7fe6d563db80 1 --2- 172.21.15.7:0/1953188048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627de0e6640 0x5627de0e6a20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe6b800e970 tx=0x7fe6b8009820 comp rx=0 tx=0).stop 2024-06-10T01:09:01.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.626+0000 7fe6d563db80 1 -- 172.21.15.7:0/1953188048 shutdown_connections 2024-06-10T01:09:01.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.626+0000 7fe6d563db80 1 --2- 172.21.15.7:0/1953188048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627de0e0f20 0x5627de0e1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.626+0000 7fe6d563db80 1 --2- 172.21.15.7:0/1953188048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627de0e6640 0x5627de0e6a20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.626+0000 7fe6d563db80 1 --2- 172.21.15.7:0/1953188048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627de0e6f60 0x5627de0e09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.626+0000 7fe6d563db80 1 -- 172.21.15.7:0/1953188048 >> 172.21.15.7:0/1953188048 conn(0x5627de00bcc0 msgr2=0x5627de0db850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:01.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.626+0000 7fe6d563db80 1 -- 172.21.15.7:0/1953188048 shutdown_connections 2024-06-10T01:09:01.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.626+0000 7fe6d563db80 1 -- 172.21.15.7:0/1953188048 wait complete. 2024-06-10T01:09:01.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.626+0000 7fe6d563db80 1 -- start start 2024-06-10T01:09:01.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.626+0000 7fe6d563db80 1 Processor -- start 2024-06-10T01:09:01.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.627+0000 7fe6d563db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627de0e0f20 0x5627de0ecd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:01.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.627+0000 7fe6d563db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627de0e6640 0x5627de0ea0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:01.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.627+0000 7fe6d563db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627de0e6f60 0x5627de0ea600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:01.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.627+0000 7fe6d563db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5627de0ed970 con 0x5627de0e0f20 2024-06-10T01:09:01.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.627+0000 7fe6d563db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5627ddfd2bf0 con 0x5627de0e6640 2024-06-10T01:09:01.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.627+0000 7fe6d563db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5627de0edaf0 con 0x5627de0e6f60 2024-06-10T01:09:01.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.627+0000 7fe6d563db80 10 client.0 _ll_register_callbacks cb 0x5627ddfcc920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:01.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.627+0000 7fe6c8b56700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627de0e0f20 0x5627de0ecd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:01.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.627+0000 7fe6c3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627de0e6640 0x5627de0ea0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:01.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.627+0000 7fe6c8b56700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627de0e0f20 0x5627de0ecd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47116/0 (socket says 172.21.15.7:47116) 2024-06-10T01:09:01.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.627+0000 7fe6c8b56700 1 -- 172.21.15.7:0/1633107204 learned_addr learned my addr 172.21.15.7:0/1633107204 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:01.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.627+0000 7fe6c3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627de0e6640 0x5627de0ea0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:47198/0 (socket says 172.21.15.7:47198) 2024-06-10T01:09:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.627+0000 7fe6c9357700 1 --2- 172.21.15.7:0/1633107204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627de0e6f60 0x5627de0ea600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.627+0000 7fe6c8b56700 1 -- 172.21.15.7:0/1633107204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627de0e6f60 msgr2=0x5627de0ea600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.627+0000 7fe6c8b56700 1 --2- 172.21.15.7:0/1633107204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627de0e6f60 0x5627de0ea600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.627+0000 7fe6c8b56700 1 -- 172.21.15.7:0/1633107204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627de0e6640 msgr2=0x5627de0ea0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.627+0000 7fe6c8b56700 1 --2- 172.21.15.7:0/1633107204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627de0e6640 0x5627de0ea0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.627+0000 7fe6c8b56700 1 -- 172.21.15.7:0/1633107204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627ddf818d0 con 0x5627de0e0f20 2024-06-10T01:09:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.627+0000 7fe6c3fff700 1 --2- 172.21.15.7:0/1633107204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627de0e6640 0x5627de0ea0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.627+0000 7fe6c8b56700 1 --2- 172.21.15.7:0/1633107204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627de0e0f20 0x5627de0ecd70 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8098410 tx=0x7fe6b8005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce7c79656230a7d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.627+0000 7fe6a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:09:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.627+0000 7fe6a3fff700 1 -- 172.21.15.7:0/1633107204 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b80b3070 con 0x5627de0e0f20 2024-06-10T01:09:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.627+0000 7fe6a3fff700 1 -- 172.21.15.7:0/1633107204 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fe6b8010920 con 0x5627de0e0f20 2024-06-10T01:09:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.628+0000 7fe6d563db80 1 -- 172.21.15.7:0/1633107204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6b8099d40 con 0x5627de0e0f20 2024-06-10T01:09:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.628+0000 7fe6a3fff700 1 -- 172.21.15.7:0/1633107204 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe6b80a46b0 con 0x5627de0e0f20 2024-06-10T01:09:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.628+0000 7fe6d563db80 10 client.16266 fetch_fsmap learned FSMap version 30 2024-06-10T01:09:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.628+0000 7fe6d563db80 1 -- 172.21.15.7:0/1633107204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5627de0f8300 con 0x5627de0e0f20 2024-06-10T01:09:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.628+0000 7fe6a3fff700 1 -- 172.21.15.7:0/1633107204 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6b8099d40 con 0x5627de0e0f20 2024-06-10T01:09:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.628+0000 7fe6a3fff700 1 -- 172.21.15.7:0/1633107204 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe6b809aa00 con 0x5627de0e0f20 2024-06-10T01:09:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.628+0000 7fe6d563db80 10 client.16266 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:09:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.628+0000 7fe6d563db80 10 client.16266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.628+0000 7fe6d563db80 20 client.16266 populate_metadata read hostname 'smithi007' 2024-06-10T01:09:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.628+0000 7fe6d563db80 1 --2- 172.21.15.7:0/1633107204 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5627de0fa7d0 0x5627de0fcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.628+0000 7fe6d563db80 4 client.16266 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:09:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.628+0000 7fe6d563db80 1 -- 172.21.15.7:0/1633107204 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627de03fd10 con 0x5627de0fa7d0 2024-06-10T01:09:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.629+0000 7fe6c3fff700 1 --2- 172.21.15.7:0/1633107204 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5627de0fa7d0 0x5627de0fcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.629+0000 7fe6c3fff700 1 --2- 172.21.15.7:0/1633107204 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5627de0fa7d0 0x5627de0fcbc0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c571fed6e85ec885 server_cookie=5229db46ab4fd0c1 in_seq=0 out_seq=0 2024-06-10T01:09:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.629+0000 7fe6a3fff700 10 client.16266 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.631+0000 7fe6a3fff700 1 -- 172.21.15.7:0/1633107204 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5627de03fd10 con 0x5627de0fa7d0 2024-06-10T01:09:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.631+0000 7fe6a3fff700 10 client.16266 handle_command_reply: tid=0 2024-06-10T01:09:01.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.631+0000 7fe6d563db80 1 client.16266 shutdown 2024-06-10T01:09:01.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.631+0000 7fe6d563db80 1 -- 172.21.15.7:0/1633107204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627de0e0f20 msgr2=0x5627de0ecd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:01.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.631+0000 7fe6d563db80 1 --2- 172.21.15.7:0/1633107204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627de0e0f20 0x5627de0ecd70 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8098410 tx=0x7fe6b8005b40 comp rx=0 tx=0).stop 2024-06-10T01:09:01.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.631+0000 7fe6d563db80 1 -- 172.21.15.7:0/1633107204 shutdown_connections 2024-06-10T01:09:01.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.631+0000 7fe6d563db80 1 --2- 172.21.15.7:0/1633107204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627de0e6f60 0x5627de0ea600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.631+0000 7fe6d563db80 1 --2- 172.21.15.7:0/1633107204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627de0e0f20 0x5627de0ecd70 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.631+0000 7fe6d563db80 1 --2- 172.21.15.7:0/1633107204 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5627de0fa7d0 0x5627de0fcbc0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.631+0000 7fe6d563db80 1 --2- 172.21.15.7:0/1633107204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627de0e6640 0x5627de0ea0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.631+0000 7fe6d563db80 1 -- 172.21.15.7:0/1633107204 >> 172.21.15.7:0/1633107204 conn(0x5627de00bcc0 msgr2=0x5627de0db5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:01.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.631+0000 7fe6a3fff700 0 client.16266 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:01.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.631+0000 7fe6d563db80 1 -- 172.21.15.7:0/1633107204 shutdown_connections 2024-06-10T01:09:01.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.631+0000 7fe6d563db80 1 -- 172.21.15.7:0/1633107204 wait complete. 2024-06-10T01:09:01.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.631+0000 7fe6d563db80 20 client.16266 trim_cache size 0 max 16384 2024-06-10T01:09:01.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.653+0000 7fe6d563db80 1 Processor -- start 2024-06-10T01:09:01.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.653+0000 7fe6d563db80 1 -- start start 2024-06-10T01:09:01.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.653+0000 7fe6d563db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627de0e0f20 0x5627de0e09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:01.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.653+0000 7fe6d563db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627de0e6640 0x5627de0d5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:01.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.653+0000 7fe6d563db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627de0e6f60 0x5627de0ff800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:01.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.653+0000 7fe6d563db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c41b2cf0 con 0x5627de0e0f20 2024-06-10T01:09:01.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.653+0000 7fe6d563db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c41b0520 con 0x5627de0e6640 2024-06-10T01:09:01.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.653+0000 7fe6d563db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c41ae370 con 0x5627de0e6f60 2024-06-10T01:09:01.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.653+0000 7fe6c3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627de0e6640 0x5627de0d5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:01.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.653+0000 7fe6c8b56700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627de0e0f20 0x5627de0e09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:01.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.653+0000 7fe6c8b56700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627de0e0f20 0x5627de0e09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47128/0 (socket says 172.21.15.7:47128) 2024-06-10T01:09:01.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.653+0000 7fe6c8b56700 1 -- 172.21.15.7:0/875008106 learned_addr learned my addr 172.21.15.7:0/875008106 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:01.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.653+0000 7fe6c3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627de0e6640 0x5627de0d5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:47200/0 (socket says 172.21.15.7:47200) 2024-06-10T01:09:01.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.653+0000 7fe6c9357700 1 --2- 172.21.15.7:0/875008106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627de0e6f60 0x5627de0ff800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:01.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.653+0000 7fe6c8b56700 1 -- 172.21.15.7:0/875008106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627de0e6f60 msgr2=0x5627de0ff800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:01.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.653+0000 7fe6c8b56700 1 --2- 172.21.15.7:0/875008106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627de0e6f60 0x5627de0ff800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.653+0000 7fe6c8b56700 1 -- 172.21.15.7:0/875008106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627de0e6640 msgr2=0x5627de0d5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:01.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.653+0000 7fe6c8b56700 1 --2- 172.21.15.7:0/875008106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627de0e6640 0x5627de0d5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.653+0000 7fe6c8b56700 1 -- 172.21.15.7:0/875008106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627de0e8630 con 0x5627de0e0f20 2024-06-10T01:09:01.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.653+0000 7fe6c3fff700 1 --2- 172.21.15.7:0/875008106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627de0e6640 0x5627de0d5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:01.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.654+0000 7fe6c8b56700 1 --2- 172.21.15.7:0/875008106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627de0e0f20 0x5627de0e09e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe6b80a4870 tx=0x7fe6b8005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1759e18dbe8ef63 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:01.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.654+0000 7fe6ae7fc700 1 -- 172.21.15.7:0/875008106 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b80b3070 con 0x5627de0e0f20 2024-06-10T01:09:01.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.654+0000 7fe6ae7fc700 1 -- 172.21.15.7:0/875008106 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6b8010720 con 0x5627de0e0f20 2024-06-10T01:09:01.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.654+0000 7fe6ae7fc700 1 -- 172.21.15.7:0/875008106 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b809ab60 con 0x5627de0e0f20 2024-06-10T01:09:01.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.655+0000 7fe6d563db80 1 -- 172.21.15.7:0/875008106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627de0e0f20 msgr2=0x5627de0e09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:01.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.655+0000 7fe6d563db80 1 --2- 172.21.15.7:0/875008106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627de0e0f20 0x5627de0e09e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe6b80a4870 tx=0x7fe6b8005bb0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.655+0000 7fe6d563db80 1 -- 172.21.15.7:0/875008106 shutdown_connections 2024-06-10T01:09:01.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.655+0000 7fe6d563db80 1 --2- 172.21.15.7:0/875008106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627de0e6f60 0x5627de0ff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.655+0000 7fe6d563db80 1 --2- 172.21.15.7:0/875008106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627de0e0f20 0x5627de0e09e0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.655+0000 7fe6d563db80 1 --2- 172.21.15.7:0/875008106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627de0e6640 0x5627de0d5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.655+0000 7fe6d563db80 1 -- 172.21.15.7:0/875008106 >> 172.21.15.7:0/875008106 conn(0x5627de00bcc0 msgr2=0x5627de0d4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.655+0000 7fe6d563db80 1 -- 172.21.15.7:0/875008106 shutdown_connections 2024-06-10T01:09:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.655+0000 7fe6d563db80 1 -- 172.21.15.7:0/875008106 wait complete. 2024-06-10T01:09:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.655+0000 7fe6d563db80 1 -- start start 2024-06-10T01:09:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.655+0000 7fe6d563db80 1 Processor -- start 2024-06-10T01:09:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.656+0000 7fe6d563db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627de0e0f20 0x5627de0ffc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.656+0000 7fe6d563db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627de0e6640 0x5627de0fd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.656+0000 7fe6d563db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627de0e6f60 0x5627de0fd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.656+0000 7fe6d563db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c41ae9e0 con 0x5627de0e6f60 2024-06-10T01:09:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.656+0000 7fe6d563db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c41afe50 con 0x5627de0e6640 2024-06-10T01:09:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.656+0000 7fe6d563db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c41af0b0 con 0x5627de0e0f20 2024-06-10T01:09:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.656+0000 7fe6d563db80 10 client.0 _ll_register_callbacks cb 0x5627ddfcc920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.656+0000 7fe6c8b56700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627de0e0f20 0x5627de0ffc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.656+0000 7fe6c8b56700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627de0e0f20 0x5627de0ffc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49788/0 (socket says 172.21.15.7:49788) 2024-06-10T01:09:01.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.656+0000 7fe6c8b56700 1 -- 172.21.15.7:0/2794611880 learned_addr learned my addr 172.21.15.7:0/2794611880 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:01.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.656+0000 7fe6c3fff700 1 --2- 172.21.15.7:0/2794611880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627de0e6640 0x5627de0fd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:01.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.656+0000 7fe6c9357700 1 --2- 172.21.15.7:0/2794611880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627de0e6f60 0x5627de0fd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:01.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.656+0000 7fe6c9357700 1 -- 172.21.15.7:0/2794611880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627de0e0f20 msgr2=0x5627de0ffc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:01.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.656+0000 7fe6c9357700 1 --2- 172.21.15.7:0/2794611880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627de0e0f20 0x5627de0ffc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.657+0000 7fe6c9357700 1 -- 172.21.15.7:0/2794611880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627de0e6640 msgr2=0x5627de0fd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:01.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.657+0000 7fe6c9357700 1 --2- 172.21.15.7:0/2794611880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627de0e6640 0x5627de0fd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.657+0000 7fe6c9357700 1 -- 172.21.15.7:0/2794611880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627de0fdaa0 con 0x5627de0e6f60 2024-06-10T01:09:01.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.657+0000 7fe6c8b56700 1 --2- 172.21.15.7:0/2794611880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627de0e0f20 0x5627de0ffc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:01.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.657+0000 7fe6c3fff700 1 --2- 172.21.15.7:0/2794611880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627de0e6640 0x5627de0fd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:01.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.657+0000 7fe6c9357700 1 --2- 172.21.15.7:0/2794611880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627de0e6f60 0x5627de0fd560 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009550 tx=0x7fe6bc00bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72dc8beb0f5b70fe server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:01.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.657+0000 7fe6a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:09:01.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.657+0000 7fe6a3fff700 1 -- 172.21.15.7:0/2794611880 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc005450 con 0x5627de0e6f60 2024-06-10T01:09:01.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.657+0000 7fe6a3fff700 1 -- 172.21.15.7:0/2794611880 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fe6bc0055f0 con 0x5627de0e6f60 2024-06-10T01:09:01.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.657+0000 7fe6d563db80 1 -- 172.21.15.7:0/2794611880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6b8010720 con 0x5627de0e6f60 2024-06-10T01:09:01.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.657+0000 7fe6a3fff700 1 -- 172.21.15.7:0/2794611880 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe6bc0057f0 con 0x5627de0e6f60 2024-06-10T01:09:01.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.657+0000 7fe6d563db80 10 client.16272 fetch_fsmap learned FSMap version 30 2024-06-10T01:09:01.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.657+0000 7fe6d563db80 1 -- 172.21.15.7:0/2794611880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5627de16d680 con 0x5627de0e6f60 2024-06-10T01:09:01.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.657+0000 7fe6a3fff700 1 -- 172.21.15.7:0/2794611880 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6b8010720 con 0x5627de0e6f60 2024-06-10T01:09:01.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.658+0000 7fe6a3fff700 1 -- 172.21.15.7:0/2794611880 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe6bc011070 con 0x5627de0e6f60 2024-06-10T01:09:01.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.658+0000 7fe6d563db80 10 client.16272 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:09:01.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.658+0000 7fe6d563db80 10 client.16272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:01.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.658+0000 7fe6d563db80 20 client.16272 populate_metadata read hostname 'smithi007' 2024-06-10T01:09:01.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.658+0000 7fe6d563db80 1 --2- 172.21.15.7:0/2794611880 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5627de16de40 0x5627de170260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:01.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.658+0000 7fe6d563db80 4 client.16272 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:09:01.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.658+0000 7fe6d563db80 1 -- 172.21.15.7:0/2794611880 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5627de03a290 con 0x5627de16de40 2024-06-10T01:09:01.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.658+0000 7fe6c8b56700 1 --2- 172.21.15.7:0/2794611880 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5627de16de40 0x5627de170260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:01.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.658+0000 7fe6c8b56700 1 --2- 172.21.15.7:0/2794611880 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5627de16de40 0x5627de170260 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55c500a49a4880a server_cookie=4ba8d49ec39be585 in_seq=0 out_seq=0 2024-06-10T01:09:01.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.658+0000 7fe6a3fff700 10 client.16272 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:01.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.659+0000 7fe6a3fff700 1 -- 172.21.15.7:0/2794611880 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5627de03a290 con 0x5627de16de40 2024-06-10T01:09:01.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.659+0000 7fe6a3fff700 10 client.16272 handle_command_reply: tid=0 2024-06-10T01:09:01.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.659+0000 7fe6d563db80 1 client.16272 shutdown 2024-06-10T01:09:01.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.659+0000 7fe6d563db80 1 -- 172.21.15.7:0/2794611880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627de0e6f60 msgr2=0x5627de0fd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:01.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.659+0000 7fe6d563db80 1 --2- 172.21.15.7:0/2794611880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627de0e6f60 0x5627de0fd560 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009550 tx=0x7fe6bc00bc20 comp rx=0 tx=0).stop 2024-06-10T01:09:01.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.659+0000 7fe6d563db80 1 -- 172.21.15.7:0/2794611880 shutdown_connections 2024-06-10T01:09:01.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.659+0000 7fe6d563db80 1 --2- 172.21.15.7:0/2794611880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627de0e6f60 0x5627de0fd560 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.659+0000 7fe6d563db80 1 --2- 172.21.15.7:0/2794611880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627de0e0f20 0x5627de0ffc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.659+0000 7fe6d563db80 1 --2- 172.21.15.7:0/2794611880 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5627de16de40 0x5627de170260 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.659+0000 7fe6d563db80 1 --2- 172.21.15.7:0/2794611880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627de0e6640 0x5627de0fd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.659+0000 7fe6d563db80 1 -- 172.21.15.7:0/2794611880 >> 172.21.15.7:0/2794611880 conn(0x5627de00bcc0 msgr2=0x5627de0e7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:01.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.659+0000 7fe6a3fff700 0 client.16272 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:01.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.659+0000 7fe6d563db80 1 -- 172.21.15.7:0/2794611880 shutdown_connections 2024-06-10T01:09:01.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.659+0000 7fe6d563db80 1 -- 172.21.15.7:0/2794611880 wait complete. 2024-06-10T01:09:01.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.659+0000 7fe6d563db80 20 client.16272 trim_cache size 0 max 16384 2024-06-10T01:09:01.661 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:09:01.661 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-10T01:09:01.661 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-10T01:09:01.662 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:09:01.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.660+0000 7fe6cadba700 1 -- 172.21.15.7:0/2814254771 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe6b40a2c20 msgr2=0x7fe6b40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:01.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.660+0000 7fe6cadba700 1 --2- 172.21.15.7:0/2814254771 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe6b40a2c20 0x7fe6b40a50e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4108b90 tx=0x7fe6b0009310 comp rx=0 tx=0).stop 2024-06-10T01:09:01.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.660+0000 7fe6cadba700 1 -- 172.21.15.7:0/2814254771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6c4102740 msgr2=0x7fe6c4074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:01.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.660+0000 7fe6cadba700 1 --2- 172.21.15.7:0/2814254771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6c4102740 0x7fe6c4074790 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe6b800e9a0 tx=0x7fe6b800ecb0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.660+0000 7fe6cadba700 1 -- 172.21.15.7:0/2814254771 shutdown_connections 2024-06-10T01:09:01.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.660+0000 7fe6cadba700 1 --2- 172.21.15.7:0/2814254771 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe6b40a2c20 0x7fe6b40a50e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.660+0000 7fe6cadba700 1 --2- 172.21.15.7:0/2814254771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c4107840 0x7fe6c4075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.660+0000 7fe6cadba700 1 --2- 172.21.15.7:0/2814254771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6c4102740 0x7fe6c4074790 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.660+0000 7fe6cadba700 1 --2- 172.21.15.7:0/2814254771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6c41030f0 0x7fe6c4074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:01.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.660+0000 7fe6cadba700 1 -- 172.21.15.7:0/2814254771 >> 172.21.15.7:0/2814254771 conn(0x7fe6c40795f0 msgr2=0x7fe6c4100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:01.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.660+0000 7fe6cadba700 1 -- 172.21.15.7:0/2814254771 shutdown_connections 2024-06-10T01:09:01.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:01.660+0000 7fe6cadba700 1 -- 172.21.15.7:0/2814254771 wait complete. 2024-06-10T01:09:01.712 INFO:tasks.workunit.client.0.smithi007.stdout: 1 136004 7.35 MB/sec warmup 102 sec latency 15.587 ms 2024-06-10T01:09:02.597 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:09:02.598 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:09:02.712 INFO:tasks.workunit.client.0.smithi007.stdout: 1 138687 7.44 MB/sec warmup 103 sec latency 14.779 ms 2024-06-10T01:09:02.877 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:09:03.257 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:03 smithi007 ceph-mon[32398]: pgmap v523: 129 pgs: 129 active+clean; 38 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 330 op/s 2024-06-10T01:09:03.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:03 smithi078 ceph-mon[36683]: pgmap v523: 129 pgs: 129 active+clean; 38 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 330 op/s 2024-06-10T01:09:03.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:03 smithi175 ceph-mon[36636]: pgmap v523: 129 pgs: 129 active+clean; 38 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 330 op/s 2024-06-10T01:09:03.712 INFO:tasks.workunit.client.0.smithi007.stdout: 1 140785 7.44 MB/sec warmup 104 sec latency 16.041 ms 2024-06-10T01:09:04.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.566+0000 7fab0bec9700 1 -- 172.21.15.7:0/506624509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab04101d60 msgr2=0x7fab0410e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:04.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.566+0000 7fab0bec9700 1 --2- 172.21.15.7:0/506624509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab04101d60 0x7fab0410e240 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7faafc00b0a0 tx=0x7faafc00b3b0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.566+0000 7fab0bec9700 1 -- 172.21.15.7:0/506624509 shutdown_connections 2024-06-10T01:09:04.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.566+0000 7fab0bec9700 1 --2- 172.21.15.7:0/506624509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab04101d60 0x7fab0410e240 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.566+0000 7fab0bec9700 1 --2- 172.21.15.7:0/506624509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab04107400 0x7fab041077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.566+0000 7fab0bec9700 1 --2- 172.21.15.7:0/506624509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab041013c0 0x7fab04101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.566+0000 7fab0bec9700 1 -- 172.21.15.7:0/506624509 >> 172.21.15.7:0/506624509 conn(0x7fab04078330 msgr2=0x7fab04078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:04.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.566+0000 7fab0bec9700 1 -- 172.21.15.7:0/506624509 shutdown_connections 2024-06-10T01:09:04.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.566+0000 7fab0bec9700 1 -- 172.21.15.7:0/506624509 wait complete. 2024-06-10T01:09:04.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.567+0000 7fab0bec9700 1 Processor -- start 2024-06-10T01:09:04.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.567+0000 7fab0bec9700 1 -- start start 2024-06-10T01:09:04.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.567+0000 7fab0bec9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab041013c0 0x7fab04103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:04.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.567+0000 7fab0bec9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab04101d60 0x7fab041035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:04.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.567+0000 7fab0bec9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab04107400 0x7fab04103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:04.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.567+0000 7fab0bec9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fab041116a0 con 0x7fab04101d60 2024-06-10T01:09:04.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.567+0000 7fab0bec9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fab04111520 con 0x7fab04107400 2024-06-10T01:09:04.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.567+0000 7fab0bec9700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fab04111820 con 0x7fab041013c0 2024-06-10T01:09:04.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.567+0000 7fab09464700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab04101d60 0x7fab041035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:04.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.567+0000 7fab09c65700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab041013c0 0x7fab04103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:04.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.567+0000 7fab09464700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab04101d60 0x7fab041035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57118/0 (socket says 172.21.15.7:57118) 2024-06-10T01:09:04.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.567+0000 7fab09464700 1 -- 172.21.15.7:0/1142410456 learned_addr learned my addr 172.21.15.7:0/1142410456 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:04.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.567+0000 7fab0a466700 1 --2- 172.21.15.7:0/1142410456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab04107400 0x7fab04103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:04.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.568+0000 7fab09c65700 1 -- 172.21.15.7:0/1142410456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab04107400 msgr2=0x7fab04103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:04.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.568+0000 7fab09c65700 1 --2- 172.21.15.7:0/1142410456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab04107400 0x7fab04103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.568+0000 7fab09c65700 1 -- 172.21.15.7:0/1142410456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab04101d60 msgr2=0x7fab041035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:04.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.568+0000 7fab09c65700 1 --2- 172.21.15.7:0/1142410456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab04101d60 0x7fab041035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.568+0000 7fab09c65700 1 -- 172.21.15.7:0/1142410456 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab041a7170 con 0x7fab041013c0 2024-06-10T01:09:04.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.568+0000 7fab09464700 1 --2- 172.21.15.7:0/1142410456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab04101d60 0x7fab041035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:04.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.568+0000 7fab0a466700 1 --2- 172.21.15.7:0/1142410456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab04107400 0x7fab04103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:04.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.568+0000 7fab09c65700 1 --2- 172.21.15.7:0/1142410456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab041013c0 0x7fab04103070 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fab0000d750 tx=0x7fab0000da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:04.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.568+0000 7faafaffd700 1 -- 172.21.15.7:0/1142410456 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab00014030 con 0x7fab041013c0 2024-06-10T01:09:04.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.568+0000 7faafaffd700 1 -- 172.21.15.7:0/1142410456 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab00004500 con 0x7fab041013c0 2024-06-10T01:09:04.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.568+0000 7fab0bec9700 1 -- 172.21.15.7:0/1142410456 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab041a7460 con 0x7fab041013c0 2024-06-10T01:09:04.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.568+0000 7faafaffd700 1 -- 172.21.15.7:0/1142410456 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab00005420 con 0x7fab041013c0 2024-06-10T01:09:04.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.568+0000 7fab0bec9700 1 -- 172.21.15.7:0/1142410456 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab041a79a0 con 0x7fab041013c0 2024-06-10T01:09:04.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.569+0000 7fab0bec9700 1 -- 172.21.15.7:0/1142410456 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab04102ab0 con 0x7fab041013c0 2024-06-10T01:09:04.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.571+0000 7faafaffd700 1 -- 172.21.15.7:0/1142410456 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fab00004750 con 0x7fab041013c0 2024-06-10T01:09:04.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.571+0000 7faafaffd700 1 --2- 172.21.15.7:0/1142410456 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7faaf00a2c20 0x7faaf00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:04.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.572+0000 7fab09464700 1 --2- 172.21.15.7:0/1142410456 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7faaf00a2c20 0x7faaf00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:04.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.572+0000 7faafaffd700 1 -- 172.21.15.7:0/1142410456 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fab000ceff0 con 0x7fab041013c0 2024-06-10T01:09:04.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.572+0000 7fab09464700 1 --2- 172.21.15.7:0/1142410456 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7faaf00a2c20 0x7faaf00a50e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fab04078930 tx=0x7faaf4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:04.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.579+0000 7faafaffd700 1 -- 172.21.15.7:0/1142410456 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fab0009bbb0 con 0x7fab041013c0 2024-06-10T01:09:04.712 INFO:tasks.workunit.client.0.smithi007.stdout: 1 142592 7.49 MB/sec warmup 105 sec latency 19.360 ms 2024-06-10T01:09:04.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.860+0000 7fab0bec9700 1 -- 172.21.15.7:0/1142410456 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab04069180 con 0x7fab041013c0 2024-06-10T01:09:04.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.861+0000 7faafaffd700 1 -- 172.21.15.7:0/1142410456 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11858 (secure 0 0 0) 0x7fab0009b4b0 con 0x7fab041013c0 2024-06-10T01:09:04.857 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:09:04.857 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:08:17.181785+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24434,24436,24451]},"id":1}]} 2024-06-10T01:09:04.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.864+0000 7fab0bec9700 1 -- 172.21.15.7:0/1142410456 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7faaf00a2c20 msgr2=0x7faaf00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:04.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.864+0000 7fab0bec9700 1 --2- 172.21.15.7:0/1142410456 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7faaf00a2c20 0x7faaf00a50e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fab04078930 tx=0x7faaf4009260 comp rx=0 tx=0).stop 2024-06-10T01:09:04.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.864+0000 7fab0bec9700 1 -- 172.21.15.7:0/1142410456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab041013c0 msgr2=0x7fab04103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:04.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.864+0000 7fab0bec9700 1 --2- 172.21.15.7:0/1142410456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab041013c0 0x7fab04103070 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fab0000d750 tx=0x7fab0000da60 comp rx=0 tx=0).stop 2024-06-10T01:09:04.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.864+0000 7fab0bec9700 1 -- 172.21.15.7:0/1142410456 shutdown_connections 2024-06-10T01:09:04.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.864+0000 7fab0bec9700 1 --2- 172.21.15.7:0/1142410456 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7faaf00a2c20 0x7faaf00a50e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.864+0000 7fab0bec9700 1 --2- 172.21.15.7:0/1142410456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab04107400 0x7fab04103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.864+0000 7fab0bec9700 1 --2- 172.21.15.7:0/1142410456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab041013c0 0x7fab04103070 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.864+0000 7fab0bec9700 1 --2- 172.21.15.7:0/1142410456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab04101d60 0x7fab041035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.864+0000 7fab0bec9700 1 -- 172.21.15.7:0/1142410456 >> 172.21.15.7:0/1142410456 conn(0x7fab04078330 msgr2=0x7fab04100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:04.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.864+0000 7fab0bec9700 1 -- 172.21.15.7:0/1142410456 shutdown_connections 2024-06-10T01:09:04.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:04.864+0000 7fab0bec9700 1 -- 172.21.15.7:0/1142410456 wait complete. 2024-06-10T01:09:04.864 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-06-10T01:09:05.188 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:05 smithi007 ceph-mon[32398]: pgmap v524: 129 pgs: 129 active+clean; 40 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 344 op/s 2024-06-10T01:09:05.188 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:05 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1142410456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:09:05.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:05 smithi078 ceph-mon[36683]: pgmap v524: 129 pgs: 129 active+clean; 40 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 344 op/s 2024-06-10T01:09:05.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:05 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1142410456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:09:05.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:05 smithi175 ceph-mon[36636]: pgmap v524: 129 pgs: 129 active+clean; 40 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 344 op/s 2024-06-10T01:09:05.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:05 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1142410456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:09:05.712 INFO:tasks.workunit.client.0.smithi007.stdout: 1 145452 7.58 MB/sec warmup 106 sec latency 17.528 ms 2024-06-10T01:09:05.962 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:09:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:09:06.712 INFO:tasks.workunit.client.0.smithi007.stdout: 1 146895 7.57 MB/sec warmup 107 sec latency 12.812 ms 2024-06-10T01:09:07.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:07 smithi078 ceph-mon[36683]: pgmap v525: 129 pgs: 129 active+clean; 43 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.3 MiB/s wr, 369 op/s 2024-06-10T01:09:07.377 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:07 smithi007 ceph-mon[32398]: pgmap v525: 129 pgs: 129 active+clean; 43 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.3 MiB/s wr, 369 op/s 2024-06-10T01:09:07.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:07 smithi175 ceph-mon[36636]: pgmap v525: 129 pgs: 129 active+clean; 43 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.3 MiB/s wr, 369 op/s 2024-06-10T01:09:07.712 INFO:tasks.workunit.client.0.smithi007.stdout: 1 149412 7.63 MB/sec warmup 108 sec latency 9.370 ms 2024-06-10T01:09:08.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.048+0000 7f1fb9712700 1 -- 172.21.15.7:0/2682556690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fb4106b60 msgr2=0x7f1fb4106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:08.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.048+0000 7f1fb9712700 1 --2- 172.21.15.7:0/2682556690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fb4106b60 0x7f1fb4106f40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4009990 tx=0x7f1fa4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.048+0000 7f1fb9712700 1 -- 172.21.15.7:0/2682556690 shutdown_connections 2024-06-10T01:09:08.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.048+0000 7f1fb9712700 1 --2- 172.21.15.7:0/2682556690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fb4101490 0x7f1fb410d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.048+0000 7f1fb9712700 1 --2- 172.21.15.7:0/2682556690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fb4106b60 0x7f1fb4106f40 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.048+0000 7f1fb9712700 1 --2- 172.21.15.7:0/2682556690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fb4100ad0 0x7f1fb4100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.048+0000 7f1fb9712700 1 -- 172.21.15.7:0/2682556690 >> 172.21.15.7:0/2682556690 conn(0x7f1fb4077990 msgr2=0x7f1fb40fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:08.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.048+0000 7f1fb9712700 1 -- 172.21.15.7:0/2682556690 shutdown_connections 2024-06-10T01:09:08.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.048+0000 7f1fb9712700 1 -- 172.21.15.7:0/2682556690 wait complete. 2024-06-10T01:09:08.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.049+0000 7f1fb9712700 1 Processor -- start 2024-06-10T01:09:08.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.049+0000 7f1fb9712700 1 -- start start 2024-06-10T01:09:08.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.049+0000 7f1fb9712700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fb4100ad0 0x7f1fb419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.049+0000 7f1fb9712700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fb4101490 0x7f1fb419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.049+0000 7f1fb9712700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fb4106b60 0x7f1fb4198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.049+0000 7f1fb9712700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb4111680 con 0x7f1fb4100ad0 2024-06-10T01:09:08.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.049+0000 7f1fb9712700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb4111500 con 0x7f1fb4101490 2024-06-10T01:09:08.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.049+0000 7f1fb9712700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb4111800 con 0x7f1fb4106b60 2024-06-10T01:09:08.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.049+0000 7f1fb2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fb4100ad0 0x7f1fb419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.049+0000 7f1fb2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fb4100ad0 0x7f1fb419e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57136/0 (socket says 172.21.15.7:57136) 2024-06-10T01:09:08.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.049+0000 7f1fb2ffd700 1 -- 172.21.15.7:0/3168434650 learned_addr learned my addr 172.21.15.7:0/3168434650 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:08.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.049+0000 7f1fb37fe700 1 --2- 172.21.15.7:0/3168434650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fb4106b60 0x7f1fb4198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.049+0000 7f1fb27fc700 1 --2- 172.21.15.7:0/3168434650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fb4101490 0x7f1fb419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.050+0000 7f1fb37fe700 1 -- 172.21.15.7:0/3168434650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fb4101490 msgr2=0x7f1fb419edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:08.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.050+0000 7f1fb37fe700 1 --2- 172.21.15.7:0/3168434650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fb4101490 0x7f1fb419edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.050+0000 7f1fb37fe700 1 -- 172.21.15.7:0/3168434650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fb4100ad0 msgr2=0x7f1fb419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:08.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.050+0000 7f1fb37fe700 1 --2- 172.21.15.7:0/3168434650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fb4100ad0 0x7f1fb419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.050+0000 7f1fb37fe700 1 -- 172.21.15.7:0/3168434650 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb41991b0 con 0x7f1fb4106b60 2024-06-10T01:09:08.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.050+0000 7f1fb2ffd700 1 --2- 172.21.15.7:0/3168434650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fb4100ad0 0x7f1fb419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:08.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.050+0000 7f1fb37fe700 1 --2- 172.21.15.7:0/3168434650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fb4106b60 0x7f1fb4198a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00efb0 tx=0x7f1fac00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:08.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.050+0000 7f1fabfff700 1 -- 172.21.15.7:0/3168434650 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac003a00 con 0x7f1fb4106b60 2024-06-10T01:09:08.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.050+0000 7f1fb9712700 1 -- 172.21.15.7:0/3168434650 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fb41994a0 con 0x7f1fb4106b60 2024-06-10T01:09:08.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.050+0000 7f1fabfff700 1 -- 172.21.15.7:0/3168434650 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fac003ba0 con 0x7f1fb4106b60 2024-06-10T01:09:08.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.050+0000 7f1fabfff700 1 -- 172.21.15.7:0/3168434650 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac004850 con 0x7f1fb4106b60 2024-06-10T01:09:08.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.050+0000 7f1fb9712700 1 -- 172.21.15.7:0/3168434650 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fb41a5760 con 0x7f1fb4106b60 2024-06-10T01:09:08.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.053+0000 7f1fabfff700 1 -- 172.21.15.7:0/3168434650 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f1fac012050 con 0x7f1fb4106b60 2024-06-10T01:09:08.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.054+0000 7f1fabfff700 1 --2- 172.21.15.7:0/3168434650 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1fa00a2ca0 0x7f1fa00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.054+0000 7f1fb2ffd700 1 --2- 172.21.15.7:0/3168434650 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1fa00a2ca0 0x7f1fa00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.054+0000 7f1fabfff700 1 -- 172.21.15.7:0/3168434650 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f1fac0cebf0 con 0x7f1fb4106b60 2024-06-10T01:09:08.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.054+0000 7f1fb2ffd700 1 --2- 172.21.15.7:0/3168434650 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1fa00a2ca0 0x7f1fa00a5160 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4009750 tx=0x7f1fa4005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:08.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.055+0000 7f1fc3f95b80 1 Processor -- start 2024-06-10T01:09:08.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.055+0000 7f1fc3f95b80 1 -- start start 2024-06-10T01:09:08.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.055+0000 7f1fc3f95b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac67713fd0 0x55ac677143b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.055+0000 7f1fc3f95b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac677148f0 0x55ac677186c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.055+0000 7f1fc3f95b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac67718c00 0x55ac67719060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.055+0000 7f1fc3f95b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ac67661810 con 0x55ac677148f0 2024-06-10T01:09:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.055+0000 7f1fc3f95b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ac67609720 con 0x55ac67713fd0 2024-06-10T01:09:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.055+0000 7f1fc3f95b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ac6765d7d0 con 0x55ac67718c00 2024-06-10T01:09:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.055+0000 7f1fb27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac677148f0 0x55ac677186c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.055+0000 7f1fb27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac677148f0 0x55ac677186c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57152/0 (socket says 172.21.15.7:57152) 2024-06-10T01:09:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.055+0000 7f1fb27fc700 1 -- 172.21.15.7:0/2948697693 learned_addr learned my addr 172.21.15.7:0/2948697693 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.055+0000 7f1fb2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac67713fd0 0x55ac677143b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.055+0000 7f1fb37fe700 1 --2- 172.21.15.7:0/2948697693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac67718c00 0x55ac67719060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.056+0000 7f1fb27fc700 1 -- 172.21.15.7:0/2948697693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac67718c00 msgr2=0x55ac67719060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.056+0000 7f1fb27fc700 1 --2- 172.21.15.7:0/2948697693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac67718c00 0x55ac67719060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.056+0000 7f1fb27fc700 1 -- 172.21.15.7:0/2948697693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac67713fd0 msgr2=0x55ac677143b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.056+0000 7f1fb27fc700 1 --2- 172.21.15.7:0/2948697693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac67713fd0 0x55ac677143b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.056+0000 7f1fb27fc700 1 -- 172.21.15.7:0/2948697693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac675fff50 con 0x55ac677148f0 2024-06-10T01:09:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.056+0000 7f1fb2ffd700 1 --2- 172.21.15.7:0/2948697693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac67713fd0 0x55ac677143b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.056+0000 7f1fb27fc700 1 --2- 172.21.15.7:0/2948697693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac677148f0 0x55ac677186c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c009d30 tx=0x7f1f9c011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b04811d7d6662e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.056+0000 7f1fa97fa700 1 -- 172.21.15.7:0/2948697693 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c01e070 con 0x55ac677148f0 2024-06-10T01:09:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.056+0000 7f1fa97fa700 1 -- 172.21.15.7:0/2948697693 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f9c019b10 con 0x55ac677148f0 2024-06-10T01:09:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.056+0000 7f1fa97fa700 1 -- 172.21.15.7:0/2948697693 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c00eb70 con 0x55ac677148f0 2024-06-10T01:09:08.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.057+0000 7f1fc3f95b80 1 -- 172.21.15.7:0/2948697693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac677148f0 msgr2=0x55ac677186c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:08.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.057+0000 7f1fc3f95b80 1 --2- 172.21.15.7:0/2948697693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac677148f0 0x55ac677186c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c009d30 tx=0x7f1f9c011720 comp rx=0 tx=0).stop 2024-06-10T01:09:08.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.057+0000 7f1fc3f95b80 1 -- 172.21.15.7:0/2948697693 shutdown_connections 2024-06-10T01:09:08.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.057+0000 7f1fc3f95b80 1 --2- 172.21.15.7:0/2948697693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac67718c00 0x55ac67719060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.057+0000 7f1fc3f95b80 1 --2- 172.21.15.7:0/2948697693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac67713fd0 0x55ac677143b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.057+0000 7f1fc3f95b80 1 --2- 172.21.15.7:0/2948697693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac677148f0 0x55ac677186c0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.057+0000 7f1fc3f95b80 1 -- 172.21.15.7:0/2948697693 >> 172.21.15.7:0/2948697693 conn(0x55ac6763acc0 msgr2=0x55ac6770a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:08.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.057+0000 7f1fc3f95b80 1 -- 172.21.15.7:0/2948697693 shutdown_connections 2024-06-10T01:09:08.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.057+0000 7f1fc3f95b80 1 -- 172.21.15.7:0/2948697693 wait complete. 2024-06-10T01:09:08.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.057+0000 7f1fc3f95b80 1 -- start start 2024-06-10T01:09:08.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.057+0000 7f1fc3f95b80 1 Processor -- start 2024-06-10T01:09:08.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.058+0000 7f1fc3f95b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac67713fd0 0x55ac67720140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.058+0000 7f1fc3f95b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac677148f0 0x55ac6771d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.058+0000 7f1fc3f95b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac67718c00 0x55ac6771d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.058+0000 7f1fc3f95b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ac67720dc0 con 0x55ac67713fd0 2024-06-10T01:09:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.058+0000 7f1fc3f95b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ac67601bf0 con 0x55ac67718c00 2024-06-10T01:09:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.058+0000 7f1fc3f95b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ac67720f40 con 0x55ac677148f0 2024-06-10T01:09:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.058+0000 7f1fc3f95b80 10 client.0 _ll_register_callbacks cb 0x55ac675fb920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.058+0000 7f1fb2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac67713fd0 0x55ac67720140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.058+0000 7f1fb37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac67718c00 0x55ac6771d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.058+0000 7f1fb2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac67713fd0 0x55ac67720140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57158/0 (socket says 172.21.15.7:57158) 2024-06-10T01:09:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.058+0000 7f1fb2ffd700 1 -- 172.21.15.7:0/2379841823 learned_addr learned my addr 172.21.15.7:0/2379841823 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.058+0000 7f1fb27fc700 1 --2- 172.21.15.7:0/2379841823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac677148f0 0x55ac6771d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.058+0000 7f1fb37fe700 1 -- 172.21.15.7:0/2379841823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac677148f0 msgr2=0x55ac6771d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.058+0000 7f1fb37fe700 1 --2- 172.21.15.7:0/2379841823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac677148f0 0x55ac6771d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.058+0000 7f1fb37fe700 1 -- 172.21.15.7:0/2379841823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac67713fd0 msgr2=0x55ac67720140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.058+0000 7f1fb37fe700 1 --2- 172.21.15.7:0/2379841823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac67713fd0 0x55ac67720140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.058+0000 7f1fb37fe700 1 -- 172.21.15.7:0/2379841823 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac675b08d0 con 0x55ac67718c00 2024-06-10T01:09:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.058+0000 7f1fb37fe700 1 --2- 172.21.15.7:0/2379841823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac67718c00 0x55ac6771d980 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1fac09ae00 tx=0x7f1fac09af50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4630b2f735ad062 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.058+0000 7f1f9a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:09:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.058+0000 7f1f9a7fc700 1 -- 172.21.15.7:0/2379841823 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac09aa60 con 0x55ac67718c00 2024-06-10T01:09:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.059+0000 7f1f9a7fc700 1 -- 172.21.15.7:0/2379841823 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f1fac0bb070 con 0x55ac67718c00 2024-06-10T01:09:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.059+0000 7f1fc3f95b80 1 -- 172.21.15.7:0/2379841823 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f9c019b10 con 0x55ac67718c00 2024-06-10T01:09:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.059+0000 7f1f9a7fc700 1 -- 172.21.15.7:0/2379841823 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1fac09b800 con 0x55ac67718c00 2024-06-10T01:09:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.059+0000 7f1fc3f95b80 10 client.25811 fetch_fsmap learned FSMap version 30 2024-06-10T01:09:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.059+0000 7f1fc3f95b80 1 -- 172.21.15.7:0/2379841823 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ac6772b7d0 con 0x55ac67718c00 2024-06-10T01:09:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.059+0000 7f1f9a7fc700 1 -- 172.21.15.7:0/2379841823 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fac0b4060 con 0x55ac67718c00 2024-06-10T01:09:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.059+0000 7f1f9a7fc700 1 -- 172.21.15.7:0/2379841823 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f1fac004bd0 con 0x55ac67718c00 2024-06-10T01:09:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.059+0000 7f1fc3f95b80 10 client.25811 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:09:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.059+0000 7f1fc3f95b80 10 client.25811 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:09:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.059+0000 7f1fc3f95b80 20 client.25811 populate_metadata read hostname 'smithi007' 2024-06-10T01:09:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.059+0000 7f1fc3f95b80 1 --2- 172.21.15.7:0/2379841823 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55ac6772de10 0x55ac67730200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.059+0000 7f1fc3f95b80 4 client.25811 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:09:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.059+0000 7f1fc3f95b80 1 -- 172.21.15.7:0/2379841823 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1f9c019b10 con 0x55ac6772de10 2024-06-10T01:09:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.059+0000 7f1fb2ffd700 1 --2- 172.21.15.7:0/2379841823 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55ac6772de10 0x55ac67730200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.060+0000 7f1f9a7fc700 10 client.25811 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:09:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.060+0000 7f1fb2ffd700 1 --2- 172.21.15.7:0/2379841823 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55ac6772de10 0x55ac67730200 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bcc81417f420e5c5 server_cookie=2a944340ad46211e in_seq=0 out_seq=0 2024-06-10T01:09:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.060+0000 7f1f9a7fc700 1 -- 172.21.15.7:0/2379841823 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f1f9c019b10 con 0x55ac6772de10 2024-06-10T01:09:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.060+0000 7f1f9a7fc700 10 client.25811 handle_command_reply: tid=0 2024-06-10T01:09:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.060+0000 7f1fc3f95b80 1 client.25811 shutdown 2024-06-10T01:09:08.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.061+0000 7f1fc3f95b80 1 -- 172.21.15.7:0/2379841823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac67718c00 msgr2=0x55ac6771d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:08.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.061+0000 7f1fc3f95b80 1 --2- 172.21.15.7:0/2379841823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac67718c00 0x55ac6771d980 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1fac09ae00 tx=0x7f1fac09af50 comp rx=0 tx=0).stop 2024-06-10T01:09:08.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.061+0000 7f1fc3f95b80 1 -- 172.21.15.7:0/2379841823 shutdown_connections 2024-06-10T01:09:08.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.061+0000 7f1fc3f95b80 1 --2- 172.21.15.7:0/2379841823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac67718c00 0x55ac6771d980 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.061+0000 7f1fc3f95b80 1 --2- 172.21.15.7:0/2379841823 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55ac6772de10 0x55ac67730200 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.061+0000 7f1fc3f95b80 1 --2- 172.21.15.7:0/2379841823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac67713fd0 0x55ac67720140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.061+0000 7f1fc3f95b80 1 --2- 172.21.15.7:0/2379841823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac677148f0 0x55ac6771d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.061+0000 7f1f9a7fc700 0 client.25811 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:09:08.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.061+0000 7f1fc3f95b80 1 -- 172.21.15.7:0/2379841823 >> 172.21.15.7:0/2379841823 conn(0x55ac6763acc0 msgr2=0x55ac6770a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:08.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.061+0000 7f1fc3f95b80 1 -- 172.21.15.7:0/2379841823 shutdown_connections 2024-06-10T01:09:08.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.061+0000 7f1fc3f95b80 1 -- 172.21.15.7:0/2379841823 wait complete. 2024-06-10T01:09:08.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.061+0000 7f1fc3f95b80 20 client.25811 trim_cache size 0 max 16384 2024-06-10T01:09:08.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.076+0000 7f1fc3f95b80 1 Processor -- start 2024-06-10T01:09:08.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.076+0000 7f1fc3f95b80 1 -- start start 2024-06-10T01:09:08.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.077+0000 7f1fc3f95b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac67713fd0 0x55ac67720140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.077+0000 7f1fc3f95b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac677148f0 0x55ac6771da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.077+0000 7f1fc3f95b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac67718c00 0x55ac6771aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.077+0000 7f1fc3f95b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb41ae950 con 0x55ac67718c00 2024-06-10T01:09:08.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.077+0000 7f1fc3f95b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb41ac180 con 0x55ac677148f0 2024-06-10T01:09:08.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.077+0000 7f1fc3f95b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb41a9fd0 con 0x55ac67713fd0 2024-06-10T01:09:08.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.077+0000 7f1fb37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac67718c00 0x55ac6771aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.077+0000 7f1fb2ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac67713fd0 0x55ac67720140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.077+0000 7f1fb37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac67718c00 0x55ac6771aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57172/0 (socket says 172.21.15.7:57172) 2024-06-10T01:09:08.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.077+0000 7f1fb37fe700 1 -- 172.21.15.7:0/4144470776 learned_addr learned my addr 172.21.15.7:0/4144470776 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:08.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.077+0000 7f1fb2ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac67713fd0 0x55ac67720140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:51744/0 (socket says 172.21.15.7:51744) 2024-06-10T01:09:08.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.077+0000 7f1fb27fc700 1 --2- 172.21.15.7:0/4144470776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac677148f0 0x55ac6771da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.077+0000 7f1fb37fe700 1 -- 172.21.15.7:0/4144470776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac67713fd0 msgr2=0x55ac67720140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:08.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.077+0000 7f1fb37fe700 1 --2- 172.21.15.7:0/4144470776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac67713fd0 0x55ac67720140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.077+0000 7f1fb37fe700 1 -- 172.21.15.7:0/4144470776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac677148f0 msgr2=0x55ac6771da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:08.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.077+0000 7f1fb37fe700 1 --2- 172.21.15.7:0/4144470776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac677148f0 0x55ac6771da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.077+0000 7f1fb37fe700 1 -- 172.21.15.7:0/4144470776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac6771b9b0 con 0x55ac67718c00 2024-06-10T01:09:08.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.077+0000 7f1fb27fc700 1 --2- 172.21.15.7:0/4144470776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac677148f0 0x55ac6771da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:08.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.077+0000 7f1fb37fe700 1 --2- 172.21.15.7:0/4144470776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac67718c00 0x55ac6771aff0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1fac0090a0 tx=0x7f1fac099610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8fd841053dfbb47 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:08.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.078+0000 7f1fa8ff9700 1 -- 172.21.15.7:0/4144470776 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac09aa60 con 0x55ac67718c00 2024-06-10T01:09:08.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.078+0000 7f1fa8ff9700 1 -- 172.21.15.7:0/4144470776 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fac0b4060 con 0x55ac67718c00 2024-06-10T01:09:08.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.078+0000 7f1fa8ff9700 1 -- 172.21.15.7:0/4144470776 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac09f4b0 con 0x55ac67718c00 2024-06-10T01:09:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.078+0000 7f1fc3f95b80 1 -- 172.21.15.7:0/4144470776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac67718c00 msgr2=0x55ac6771aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.078+0000 7f1fc3f95b80 1 --2- 172.21.15.7:0/4144470776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac67718c00 0x55ac6771aff0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1fac0090a0 tx=0x7f1fac099610 comp rx=0 tx=0).stop 2024-06-10T01:09:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.079+0000 7f1fc3f95b80 1 -- 172.21.15.7:0/4144470776 shutdown_connections 2024-06-10T01:09:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.079+0000 7f1fc3f95b80 1 --2- 172.21.15.7:0/4144470776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac67718c00 0x55ac6771aff0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.079+0000 7f1fc3f95b80 1 --2- 172.21.15.7:0/4144470776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac67713fd0 0x55ac67720140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.079+0000 7f1fc3f95b80 1 --2- 172.21.15.7:0/4144470776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac677148f0 0x55ac6771da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.079+0000 7f1fc3f95b80 1 -- 172.21.15.7:0/4144470776 >> 172.21.15.7:0/4144470776 conn(0x55ac6763acc0 msgr2=0x55ac6771a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.079+0000 7f1fc3f95b80 1 -- 172.21.15.7:0/4144470776 shutdown_connections 2024-06-10T01:09:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.079+0000 7f1fc3f95b80 1 -- 172.21.15.7:0/4144470776 wait complete. 2024-06-10T01:09:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.079+0000 7f1fc3f95b80 1 -- start start 2024-06-10T01:09:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.079+0000 7f1fc3f95b80 1 Processor -- start 2024-06-10T01:09:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.079+0000 7f1fc3f95b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac67713fd0 0x55ac677265c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.079+0000 7f1fc3f95b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac677148f0 0x55ac67726b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.079+0000 7f1fc3f95b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac67718c00 0x55ac67735200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.079+0000 7f1fc3f95b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb41aa640 con 0x55ac677148f0 2024-06-10T01:09:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.079+0000 7f1fc3f95b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb41abab0 con 0x55ac67713fd0 2024-06-10T01:09:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.079+0000 7f1fc3f95b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb41aad10 con 0x55ac67718c00 2024-06-10T01:09:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.080+0000 7f1fc3f95b80 10 client.0 _ll_register_callbacks cb 0x55ac675fb920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.080+0000 7f1fb27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac677148f0 0x55ac67726b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.080+0000 7f1fb27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac677148f0 0x55ac67726b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57178/0 (socket says 172.21.15.7:57178) 2024-06-10T01:09:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.080+0000 7f1fb37fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac67718c00 0x55ac67735200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.080+0000 7f1fb27fc700 1 -- 172.21.15.7:0/3684482447 learned_addr learned my addr 172.21.15.7:0/3684482447 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.080+0000 7f1fb2ffd700 1 --2- 172.21.15.7:0/3684482447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac67713fd0 0x55ac677265c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.080+0000 7f1fb27fc700 1 -- 172.21.15.7:0/3684482447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac67718c00 msgr2=0x55ac67735200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.080+0000 7f1fb27fc700 1 --2- 172.21.15.7:0/3684482447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac67718c00 0x55ac67735200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.080+0000 7f1fb27fc700 1 -- 172.21.15.7:0/3684482447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac67713fd0 msgr2=0x55ac677265c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.080+0000 7f1fb27fc700 1 --2- 172.21.15.7:0/3684482447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac67713fd0 0x55ac677265c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.080+0000 7f1fb27fc700 1 -- 172.21.15.7:0/3684482447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac67735a10 con 0x55ac677148f0 2024-06-10T01:09:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.080+0000 7f1fb27fc700 1 --2- 172.21.15.7:0/3684482447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac677148f0 0x55ac67726b00 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c009c50 tx=0x7f1f9c0052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e351fb32a95ce17 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.080+0000 7f1f9a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:09:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.080+0000 7f1f9a7fc700 1 -- 172.21.15.7:0/3684482447 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c01e040 con 0x55ac677148f0 2024-06-10T01:09:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.080+0000 7f1f9a7fc700 1 -- 172.21.15.7:0/3684482447 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f1f9c003680 con 0x55ac677148f0 2024-06-10T01:09:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.080+0000 7f1fc3f95b80 1 -- 172.21.15.7:0/3684482447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fac0b4060 con 0x55ac677148f0 2024-06-10T01:09:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.081+0000 7f1f9a7fc700 1 -- 172.21.15.7:0/3684482447 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1f9c034050 con 0x55ac677148f0 2024-06-10T01:09:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.081+0000 7f1fc3f95b80 10 client.16296 fetch_fsmap learned FSMap version 30 2024-06-10T01:09:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.081+0000 7f1fc3f95b80 1 -- 172.21.15.7:0/3684482447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ac67724e70 con 0x55ac677148f0 2024-06-10T01:09:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.081+0000 7f1f9a7fc700 1 -- 172.21.15.7:0/3684482447 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fac0b4060 con 0x55ac677148f0 2024-06-10T01:09:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.081+0000 7f1f9a7fc700 1 -- 172.21.15.7:0/3684482447 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f1f9c0259e0 con 0x55ac677148f0 2024-06-10T01:09:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.081+0000 7f1fc3f95b80 10 client.16296 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:09:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.081+0000 7f1fc3f95b80 10 client.16296 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:09:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.081+0000 7f1fc3f95b80 20 client.16296 populate_metadata read hostname 'smithi007' 2024-06-10T01:09:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.081+0000 7f1fc3f95b80 1 --2- 172.21.15.7:0/3684482447 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55ac67798bf0 0x55ac6779b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.081+0000 7f1fc3f95b80 4 client.16296 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:09:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.081+0000 7f1fc3f95b80 1 -- 172.21.15.7:0/3684482447 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ac6766ed10 con 0x55ac67798bf0 2024-06-10T01:09:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.081+0000 7f1fb2ffd700 1 --2- 172.21.15.7:0/3684482447 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55ac67798bf0 0x55ac6779b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.082+0000 7f1fb2ffd700 1 --2- 172.21.15.7:0/3684482447 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55ac67798bf0 0x55ac6779b010 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d1dfecaac0d95233 server_cookie=38dbd06ceb32c421 in_seq=0 out_seq=0 2024-06-10T01:09:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.082+0000 7f1f9a7fc700 10 client.16296 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:09:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.082+0000 7f1f9a7fc700 1 -- 172.21.15.7:0/3684482447 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ac6766ed10 con 0x55ac67798bf0 2024-06-10T01:09:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.082+0000 7f1f9a7fc700 10 client.16296 handle_command_reply: tid=0 2024-06-10T01:09:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.082+0000 7f1fc3f95b80 1 client.16296 shutdown 2024-06-10T01:09:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.082+0000 7f1fc3f95b80 1 -- 172.21.15.7:0/3684482447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac677148f0 msgr2=0x55ac67726b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.082+0000 7f1fc3f95b80 1 --2- 172.21.15.7:0/3684482447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac677148f0 0x55ac67726b00 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c009c50 tx=0x7f1f9c0052e0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.082+0000 7f1fc3f95b80 1 -- 172.21.15.7:0/3684482447 shutdown_connections 2024-06-10T01:09:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.082+0000 7f1fc3f95b80 1 --2- 172.21.15.7:0/3684482447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac67718c00 0x55ac67735200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.082+0000 7f1fc3f95b80 1 --2- 172.21.15.7:0/3684482447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac67713fd0 0x55ac677265c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.082+0000 7f1fc3f95b80 1 --2- 172.21.15.7:0/3684482447 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55ac67798bf0 0x55ac6779b010 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.082+0000 7f1fc3f95b80 1 --2- 172.21.15.7:0/3684482447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac677148f0 0x55ac67726b00 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.082+0000 7f1f9a7fc700 0 client.16296 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:09:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.082+0000 7f1fc3f95b80 1 -- 172.21.15.7:0/3684482447 >> 172.21.15.7:0/3684482447 conn(0x55ac6763acc0 msgr2=0x55ac67703e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.082+0000 7f1fc3f95b80 1 -- 172.21.15.7:0/3684482447 shutdown_connections 2024-06-10T01:09:08.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.082+0000 7f1fc3f95b80 1 -- 172.21.15.7:0/3684482447 wait complete. 2024-06-10T01:09:08.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.082+0000 7f1fc3f95b80 20 client.16296 trim_cache size 0 max 16384 2024-06-10T01:09:08.085 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:09:08.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.083+0000 7f1fb9712700 1 -- 172.21.15.7:0/3168434650 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1fa00a2ca0 msgr2=0x7f1fa00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:08.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.083+0000 7f1fb9712700 1 --2- 172.21.15.7:0/3168434650 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1fa00a2ca0 0x7f1fa00a5160 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4009750 tx=0x7f1fa4005e80 comp rx=0 tx=0).stop 2024-06-10T01:09:08.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.083+0000 7f1fb9712700 1 -- 172.21.15.7:0/3168434650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fb4106b60 msgr2=0x7f1fb4198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:08.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.083+0000 7f1fb9712700 1 --2- 172.21.15.7:0/3168434650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fb4106b60 0x7f1fb4198a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00efb0 tx=0x7f1fac00c3a0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.083+0000 7f1fb9712700 1 -- 172.21.15.7:0/3168434650 shutdown_connections 2024-06-10T01:09:08.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.083+0000 7f1fb9712700 1 --2- 172.21.15.7:0/3168434650 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1fa00a2ca0 0x7f1fa00a5160 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.083+0000 7f1fb9712700 1 --2- 172.21.15.7:0/3168434650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fb4106b60 0x7f1fb4198a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.083+0000 7f1fb9712700 1 --2- 172.21.15.7:0/3168434650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fb4100ad0 0x7f1fb419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.083+0000 7f1fb9712700 1 --2- 172.21.15.7:0/3168434650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fb4101490 0x7f1fb419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.083+0000 7f1fb9712700 1 -- 172.21.15.7:0/3168434650 >> 172.21.15.7:0/3168434650 conn(0x7f1fb4077990 msgr2=0x7f1fb410e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:08.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.083+0000 7f1fb9712700 1 -- 172.21.15.7:0/3168434650 shutdown_connections 2024-06-10T01:09:08.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:08.083+0000 7f1fb9712700 1 -- 172.21.15.7:0/3168434650 wait complete. 2024-06-10T01:09:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.669+0000 7f5b5f441640 1 -- 172.21.15.7:0/1640500463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b58101b20 msgr2=0x7f5b58101fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.669+0000 7f5b5f441640 1 --2- 172.21.15.7:0/1640500463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b58101b20 0x7f5b58101fa0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5b48009900 tx=0x7f5b4802a410 comp rx=0 tx=0).stop 2024-06-10T01:09:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.669+0000 7f5b5f441640 1 -- 172.21.15.7:0/1640500463 shutdown_connections 2024-06-10T01:09:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.669+0000 7f5b5f441640 1 --2- 172.21.15.7:0/1640500463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b581024e0 0x7f5b58108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.669+0000 7f5b5f441640 1 --2- 172.21.15.7:0/1640500463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b58101b20 0x7f5b58101fa0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.669+0000 7f5b5f441640 1 --2- 172.21.15.7:0/1640500463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b58100930 0x7f5b58100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.669+0000 7f5b5f441640 1 -- 172.21.15.7:0/1640500463 >> 172.21.15.7:0/1640500463 conn(0x7f5b580fbea0 msgr2=0x7f5b580fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.669+0000 7f5b5f441640 1 -- 172.21.15.7:0/1640500463 shutdown_connections 2024-06-10T01:09:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.669+0000 7f5b5f441640 1 -- 172.21.15.7:0/1640500463 wait complete. 2024-06-10T01:09:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.670+0000 7f5b5f441640 1 Processor -- start 2024-06-10T01:09:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.670+0000 7f5b5f441640 1 -- start start 2024-06-10T01:09:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.670+0000 7f5b5f441640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b58100930 0x7f5b58197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.670+0000 7f5b5f441640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b58101b20 0x7f5b58198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.670+0000 7f5b5f441640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b581024e0 0x7f5b5819f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.670+0000 7f5b5f441640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5810c600 con 0x7f5b58101b20 2024-06-10T01:09:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.670+0000 7f5b5f441640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5810c480 con 0x7f5b581024e0 2024-06-10T01:09:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.670+0000 7f5b5f441640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5810c780 con 0x7f5b58100930 2024-06-10T01:09:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.670+0000 7f5b5c9dc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b58101b20 0x7f5b58198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.670+0000 7f5b5d1dd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b58100930 0x7f5b58197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.670+0000 7f5b5c9dc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b58101b20 0x7f5b58198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57202/0 (socket says 172.21.15.7:57202) 2024-06-10T01:09:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.670+0000 7f5b5c9dc640 1 -- 172.21.15.7:0/3472435813 learned_addr learned my addr 172.21.15.7:0/3472435813 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.670+0000 7f5b5d9de640 1 --2- 172.21.15.7:0/3472435813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b581024e0 0x7f5b5819f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.671+0000 7f5b5d1dd640 1 -- 172.21.15.7:0/3472435813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b581024e0 msgr2=0x7f5b5819f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:08.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.671+0000 7f5b5d1dd640 1 --2- 172.21.15.7:0/3472435813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b581024e0 0x7f5b5819f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.671+0000 7f5b5d1dd640 1 -- 172.21.15.7:0/3472435813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b58101b20 msgr2=0x7f5b58198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:08.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.671+0000 7f5b5d1dd640 1 --2- 172.21.15.7:0/3472435813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b58101b20 0x7f5b58198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.671+0000 7f5b5d1dd640 1 -- 172.21.15.7:0/3472435813 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b5819f910 con 0x7f5b58100930 2024-06-10T01:09:08.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.671+0000 7f5b5c9dc640 1 --2- 172.21.15.7:0/3472435813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b58101b20 0x7f5b58198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:08.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.671+0000 7f5b5d9de640 1 --2- 172.21.15.7:0/3472435813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b581024e0 0x7f5b5819f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:08.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.671+0000 7f5b5d1dd640 1 --2- 172.21.15.7:0/3472435813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b58100930 0x7f5b58197d00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5b5400e9c0 tx=0x7f5b5400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:08.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.671+0000 7f5b4e7fc640 1 -- 172.21.15.7:0/3472435813 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5400ccc0 con 0x7f5b58100930 2024-06-10T01:09:08.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.671+0000 7f5b5f441640 1 -- 172.21.15.7:0/3472435813 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b5819fc00 con 0x7f5b58100930 2024-06-10T01:09:08.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.671+0000 7f5b4e7fc640 1 -- 172.21.15.7:0/3472435813 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b540035f0 con 0x7f5b58100930 2024-06-10T01:09:08.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.671+0000 7f5b4e7fc640 1 -- 172.21.15.7:0/3472435813 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5400ce60 con 0x7f5b58100930 2024-06-10T01:09:08.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.671+0000 7f5b5f441640 1 -- 172.21.15.7:0/3472435813 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b581a01a0 con 0x7f5b58100930 2024-06-10T01:09:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.674+0000 7f5b62220740 1 Processor -- start 2024-06-10T01:09:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.674+0000 7f5b4e7fc640 1 -- 172.21.15.7:0/3472435813 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f5b540037a0 con 0x7f5b58100930 2024-06-10T01:09:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.674+0000 7f5b62220740 1 -- start start 2024-06-10T01:09:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.675+0000 7f5b62220740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdf7acd010 0x55fdf7acd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.675+0000 7f5b62220740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdf7acda00 0x55fdf7ad2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.675+0000 7f5b62220740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdf7ad2b70 0x55fdf7ad5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.675+0000 7f5b62220740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fdf7a304f0 con 0x55fdf7acd010 2024-06-10T01:09:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.675+0000 7f5b62220740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fdf79ce870 con 0x55fdf7ad2b70 2024-06-10T01:09:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.675+0000 7f5b62220740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fdf7a2c4b0 con 0x55fdf7acda00 2024-06-10T01:09:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.675+0000 7f5b5d1dd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdf7acda00 0x55fdf7ad2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.675+0000 7f5b5d9de640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdf7acd010 0x55fdf7acd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.675+0000 7f5b5d1dd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdf7acda00 0x55fdf7ad2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:51798/0 (socket says 172.21.15.7:51798) 2024-06-10T01:09:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.675+0000 7f5b5d9de640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdf7acd010 0x55fdf7acd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57214/0 (socket says 172.21.15.7:57214) 2024-06-10T01:09:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.675+0000 7f5b5d1dd640 1 -- 172.21.15.7:0/4152795481 learned_addr learned my addr 172.21.15.7:0/4152795481 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.675+0000 7f5b5c9dc640 1 --2- 172.21.15.7:0/4152795481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdf7ad2b70 0x55fdf7ad5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.675+0000 7f5b5d1dd640 1 -- 172.21.15.7:0/4152795481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdf7ad2b70 msgr2=0x55fdf7ad5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.675+0000 7f5b5d1dd640 1 --2- 172.21.15.7:0/4152795481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdf7ad2b70 0x55fdf7ad5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.675+0000 7f5b5d1dd640 1 -- 172.21.15.7:0/4152795481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdf7acd010 msgr2=0x55fdf7acd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.675+0000 7f5b5d1dd640 1 --2- 172.21.15.7:0/4152795481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdf7acd010 0x55fdf7acd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.675+0000 7f5b5d1dd640 1 -- 172.21.15.7:0/4152795481 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdf79ea9b0 con 0x55fdf7acda00 2024-06-10T01:09:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.675+0000 7f5b4e7fc640 1 --2- 172.21.15.7:0/3472435813 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5b440a2b80 0x7f5b440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.675+0000 7f5b5d9de640 1 --2- 172.21.15.7:0/3472435813 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5b440a2b80 0x7f5b440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.675+0000 7f5b5d1dd640 1 --2- 172.21.15.7:0/4152795481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdf7acda00 0x55fdf7ad2500 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5b540d5a80 tx=0x7f5b540d5f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed28cef7efb88117 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.675+0000 7f5b4e7fc640 1 -- 172.21.15.7:0/3472435813 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f5b540cf870 con 0x7f5b58100930 2024-06-10T01:09:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.676+0000 7f5b437fe640 1 -- 172.21.15.7:0/4152795481 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b540d2e60 con 0x55fdf7acda00 2024-06-10T01:09:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.676+0000 7f5b437fe640 1 -- 172.21.15.7:0/4152795481 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b5409c450 con 0x55fdf7acda00 2024-06-10T01:09:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.676+0000 7f5b437fe640 1 -- 172.21.15.7:0/4152795481 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b54099e90 con 0x55fdf7acda00 2024-06-10T01:09:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.676+0000 7f5b5d9de640 1 --2- 172.21.15.7:0/3472435813 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5b440a2b80 0x7f5b440a5040 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55fdf7ace260 tx=0x7f5b50009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.676+0000 7f5b62220740 1 -- 172.21.15.7:0/4152795481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdf7acda00 msgr2=0x55fdf7ad2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.676+0000 7f5b62220740 1 --2- 172.21.15.7:0/4152795481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdf7acda00 0x55fdf7ad2500 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5b540d5a80 tx=0x7f5b540d5f40 comp rx=0 tx=0).stop 2024-06-10T01:09:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.677+0000 7f5b62220740 1 -- 172.21.15.7:0/4152795481 shutdown_connections 2024-06-10T01:09:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.677+0000 7f5b62220740 1 --2- 172.21.15.7:0/4152795481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdf7ad2b70 0x55fdf7ad5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.677+0000 7f5b62220740 1 --2- 172.21.15.7:0/4152795481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdf7acda00 0x55fdf7ad2500 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.677+0000 7f5b62220740 1 --2- 172.21.15.7:0/4152795481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdf7acd010 0x55fdf7acd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.677+0000 7f5b62220740 1 -- 172.21.15.7:0/4152795481 >> 172.21.15.7:0/4152795481 conn(0x55fdf7ac8680 msgr2=0x55fdf7ac8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.677+0000 7f5b62220740 1 -- 172.21.15.7:0/4152795481 shutdown_connections 2024-06-10T01:09:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.677+0000 7f5b62220740 1 -- 172.21.15.7:0/4152795481 wait complete. 2024-06-10T01:09:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.677+0000 7f5b62220740 1 -- start start 2024-06-10T01:09:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.677+0000 7f5b62220740 1 Processor -- start 2024-06-10T01:09:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.677+0000 7f5b62220740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdf7acd010 0x55fdf7ad72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.677+0000 7f5b62220740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdf7acda00 0x55fdf7ad7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.677+0000 7f5b62220740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdf7ad2b70 0x55fdf7ade870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.677+0000 7f5b62220740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fdf7ad5b20 con 0x55fdf7acda00 2024-06-10T01:09:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.677+0000 7f5b62220740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fdf79b9ae0 con 0x55fdf7ad2b70 2024-06-10T01:09:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.677+0000 7f5b62220740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fdf7ad5ca0 con 0x55fdf7acd010 2024-06-10T01:09:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.677+0000 7f5b62220740 10 client.0 _ll_register_callbacks cb 0x55fdf79f29b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.677+0000 7f5b5c9dc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdf7acda00 0x55fdf7ad7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.677+0000 7f5b5d1dd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdf7acd010 0x55fdf7ad72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.677+0000 7f5b5c9dc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdf7acda00 0x55fdf7ad7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57226/0 (socket says 172.21.15.7:57226) 2024-06-10T01:09:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.678+0000 7f5b5c9dc640 1 -- 172.21.15.7:0/2452095045 learned_addr learned my addr 172.21.15.7:0/2452095045 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.678+0000 7f5b5d9de640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdf7ad2b70 0x55fdf7ade870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.678+0000 7f5b5d1dd640 1 -- 172.21.15.7:0/2452095045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdf7ad2b70 msgr2=0x55fdf7ade870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.678+0000 7f5b5d1dd640 1 --2- 172.21.15.7:0/2452095045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdf7ad2b70 0x55fdf7ade870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.678+0000 7f5b5d1dd640 1 -- 172.21.15.7:0/2452095045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdf7acda00 msgr2=0x55fdf7ad7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.678+0000 7f5b5d1dd640 1 --2- 172.21.15.7:0/2452095045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdf7acda00 0x55fdf7ad7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.678+0000 7f5b5d1dd640 1 -- 172.21.15.7:0/2452095045 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdf7744210 con 0x55fdf7acd010 2024-06-10T01:09:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.678+0000 7f5b5c9dc640 1 --2- 172.21.15.7:0/2452095045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdf7acda00 0x55fdf7ad7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.678+0000 7f5b5d1dd640 1 --2- 172.21.15.7:0/2452095045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdf7acd010 0x55fdf7ad72f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55fdf7ace470 tx=0x7f5b54099d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50043596c8a51b53 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.678+0000 7f5b40ff9640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:09:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.678+0000 7f5b40ff9640 1 -- 172.21.15.7:0/2452095045 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b540d2e60 con 0x55fdf7acd010 2024-06-10T01:09:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.678+0000 7f5b40ff9640 1 -- 172.21.15.7:0/2452095045 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f5b5409c1b0 con 0x55fdf7acd010 2024-06-10T01:09:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.678+0000 7f5b62220740 1 -- 172.21.15.7:0/2452095045 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b5409c450 con 0x55fdf7acd010 2024-06-10T01:09:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.678+0000 7f5b40ff9640 1 -- 172.21.15.7:0/2452095045 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5b54099e90 con 0x55fdf7acd010 2024-06-10T01:09:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.678+0000 7f5b62220740 10 client.25855 fetch_fsmap learned FSMap version 30 2024-06-10T01:09:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.678+0000 7f5b62220740 1 -- 172.21.15.7:0/2452095045 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fdf7810620 con 0x55fdf7acd010 2024-06-10T01:09:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.679+0000 7f5b40ff9640 1 -- 172.21.15.7:0/2452095045 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b5409c450 con 0x55fdf7acd010 2024-06-10T01:09:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.679+0000 7f5b40ff9640 1 -- 172.21.15.7:0/2452095045 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f5b5409b1a0 con 0x55fdf7acd010 2024-06-10T01:09:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.679+0000 7f5b62220740 10 client.25855 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:09:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.679+0000 7f5b62220740 10 client.25855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.679+0000 7f5b62220740 20 client.25855 populate_metadata read hostname 'smithi007' 2024-06-10T01:09:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.679+0000 7f5b62220740 1 --2- 172.21.15.7:0/2452095045 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fdf7ae2d60 0x55fdf7ae5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.679+0000 7f5b62220740 4 client.25855 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:09:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.679+0000 7f5b62220740 1 -- 172.21.15.7:0/2452095045 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdf7a43f10 con 0x55fdf7ae2d60 2024-06-10T01:09:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.679+0000 7f5b5c9dc640 1 --2- 172.21.15.7:0/2452095045 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fdf7ae2d60 0x55fdf7ae5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.679+0000 7f5b40ff9640 10 client.25855 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.679+0000 7f5b5c9dc640 1 --2- 172.21.15.7:0/2452095045 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fdf7ae2d60 0x55fdf7ae5150 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dde770189346a656 server_cookie=38dec30eee35ccf8 in_seq=0 out_seq=0 2024-06-10T01:09:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.680+0000 7f5b40ff9640 1 -- 172.21.15.7:0/2452095045 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fdf7a43f10 con 0x55fdf7ae2d60 2024-06-10T01:09:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.680+0000 7f5b40ff9640 10 client.25855 handle_command_reply: tid=0 2024-06-10T01:09:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.680+0000 7f5b62220740 1 client.25855 shutdown 2024-06-10T01:09:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.680+0000 7f5b62220740 1 -- 172.21.15.7:0/2452095045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdf7acd010 msgr2=0x55fdf7ad72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.680+0000 7f5b62220740 1 --2- 172.21.15.7:0/2452095045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdf7acd010 0x55fdf7ad72f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55fdf7ace470 tx=0x7f5b54099d60 comp rx=0 tx=0).stop 2024-06-10T01:09:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.680+0000 7f5b62220740 1 -- 172.21.15.7:0/2452095045 shutdown_connections 2024-06-10T01:09:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.680+0000 7f5b62220740 1 --2- 172.21.15.7:0/2452095045 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fdf7ae2d60 0x55fdf7ae5150 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.680+0000 7f5b62220740 1 --2- 172.21.15.7:0/2452095045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdf7ad2b70 0x55fdf7ade870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.680+0000 7f5b40ff9640 0 client.25855 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.680+0000 7f5b62220740 1 --2- 172.21.15.7:0/2452095045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdf7acda00 0x55fdf7ad7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.680+0000 7f5b62220740 1 --2- 172.21.15.7:0/2452095045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdf7acd010 0x55fdf7ad72f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.680+0000 7f5b62220740 1 -- 172.21.15.7:0/2452095045 >> 172.21.15.7:0/2452095045 conn(0x55fdf7ac8680 msgr2=0x55fdf7ac77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.680+0000 7f5b62220740 1 -- 172.21.15.7:0/2452095045 shutdown_connections 2024-06-10T01:09:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.680+0000 7f5b62220740 1 -- 172.21.15.7:0/2452095045 wait complete. 2024-06-10T01:09:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.680+0000 7f5b62220740 20 client.25855 trim_cache size 0 max 16384 2024-06-10T01:09:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.696+0000 7f5b62220740 1 Processor -- start 2024-06-10T01:09:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.696+0000 7f5b62220740 1 -- start start 2024-06-10T01:09:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.696+0000 7f5b62220740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdf7acd010 0x55fdf7acb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.696+0000 7f5b62220740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdf7acda00 0x55fdf7ade830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.696+0000 7f5b62220740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdf7ad2b70 0x55fdf7acbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.696+0000 7f5b62220740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5b581aced0 con 0x55fdf7acd010 2024-06-10T01:09:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.696+0000 7f5b62220740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b581aa740 con 0x55fdf7acda00 2024-06-10T01:09:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.696+0000 7f5b62220740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5b581a8590 con 0x55fdf7ad2b70 2024-06-10T01:09:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.696+0000 7f5b5d1dd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdf7acd010 0x55fdf7acb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.696+0000 7f5b5d1dd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdf7acd010 0x55fdf7acb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57230/0 (socket says 172.21.15.7:57230) 2024-06-10T01:09:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.696+0000 7f5b5d1dd640 1 -- 172.21.15.7:0/4239167687 learned_addr learned my addr 172.21.15.7:0/4239167687 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.696+0000 7f5b5d9de640 1 --2- 172.21.15.7:0/4239167687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdf7ad2b70 0x55fdf7acbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.696+0000 7f5b5c9dc640 1 --2- 172.21.15.7:0/4239167687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdf7acda00 0x55fdf7ade830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.696+0000 7f5b5d1dd640 1 -- 172.21.15.7:0/4239167687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdf7ad2b70 msgr2=0x55fdf7acbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.696+0000 7f5b5d1dd640 1 --2- 172.21.15.7:0/4239167687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdf7ad2b70 0x55fdf7acbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.696+0000 7f5b5d1dd640 1 -- 172.21.15.7:0/4239167687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdf7acda00 msgr2=0x55fdf7ade830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.696+0000 7f5b5d1dd640 1 --2- 172.21.15.7:0/4239167687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdf7acda00 0x55fdf7ade830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.696+0000 7f5b5d1dd640 1 -- 172.21.15.7:0/4239167687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdf7ac84b0 con 0x55fdf7acd010 2024-06-10T01:09:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.696+0000 7f5b5c9dc640 1 --2- 172.21.15.7:0/4239167687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdf7acda00 0x55fdf7ade830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:08.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.697+0000 7f5b5d1dd640 1 --2- 172.21.15.7:0/4239167687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdf7acd010 0x55fdf7acb670 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5b5409ba50 tx=0x7f5b540d7800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa17d3a47c38482 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:08.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.697+0000 7f5b42ffd640 1 -- 172.21.15.7:0/4239167687 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b540d2e60 con 0x55fdf7acd010 2024-06-10T01:09:08.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.697+0000 7f5b42ffd640 1 -- 172.21.15.7:0/4239167687 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b5409c450 con 0x55fdf7acd010 2024-06-10T01:09:08.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.697+0000 7f5b42ffd640 1 -- 172.21.15.7:0/4239167687 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5409bb20 con 0x55fdf7acd010 2024-06-10T01:09:08.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.697+0000 7f5b62220740 1 -- 172.21.15.7:0/4239167687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdf7acd010 msgr2=0x55fdf7acb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:08.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.697+0000 7f5b62220740 1 --2- 172.21.15.7:0/4239167687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdf7acd010 0x55fdf7acb670 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5b5409ba50 tx=0x7f5b540d7800 comp rx=0 tx=0).stop 2024-06-10T01:09:08.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.697+0000 7f5b62220740 1 -- 172.21.15.7:0/4239167687 shutdown_connections 2024-06-10T01:09:08.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.697+0000 7f5b62220740 1 --2- 172.21.15.7:0/4239167687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdf7ad2b70 0x55fdf7acbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.697+0000 7f5b62220740 1 --2- 172.21.15.7:0/4239167687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdf7acda00 0x55fdf7ade830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.697+0000 7f5b62220740 1 --2- 172.21.15.7:0/4239167687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdf7acd010 0x55fdf7acb670 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.697+0000 7f5b62220740 1 -- 172.21.15.7:0/4239167687 >> 172.21.15.7:0/4239167687 conn(0x55fdf7ac8680 msgr2=0x55fdf7ac8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:08.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.697+0000 7f5b62220740 1 -- 172.21.15.7:0/4239167687 shutdown_connections 2024-06-10T01:09:08.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.697+0000 7f5b62220740 1 -- 172.21.15.7:0/4239167687 wait complete. 2024-06-10T01:09:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.698+0000 7f5b62220740 1 -- start start 2024-06-10T01:09:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.698+0000 7f5b62220740 1 Processor -- start 2024-06-10T01:09:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.698+0000 7f5b62220740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdf7acd010 0x55fdf7adb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.698+0000 7f5b62220740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdf7acda00 0x55fdf7adbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.698+0000 7f5b62220740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdf7ad2b70 0x55fdf7b4b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.698+0000 7f5b62220740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5b581a8c00 con 0x55fdf7acda00 2024-06-10T01:09:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.698+0000 7f5b62220740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b581aa070 con 0x55fdf7ad2b70 2024-06-10T01:09:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.698+0000 7f5b62220740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5b581a92d0 con 0x55fdf7acd010 2024-06-10T01:09:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.698+0000 7f5b62220740 10 client.0 _ll_register_callbacks cb 0x7f5b5409e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.698+0000 7f5b5c9dc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdf7acda00 0x55fdf7adbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.698+0000 7f5b5d1dd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdf7acd010 0x55fdf7adb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.698+0000 7f5b5c9dc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdf7acda00 0x55fdf7adbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57238/0 (socket says 172.21.15.7:57238) 2024-06-10T01:09:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.698+0000 7f5b5c9dc640 1 -- 172.21.15.7:0/3585193142 learned_addr learned my addr 172.21.15.7:0/3585193142 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.698+0000 7f5b5d9de640 1 --2- 172.21.15.7:0/3585193142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdf7ad2b70 0x55fdf7b4b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.699+0000 7f5b5c9dc640 1 -- 172.21.15.7:0/3585193142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdf7acd010 msgr2=0x55fdf7adb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.699+0000 7f5b5c9dc640 1 --2- 172.21.15.7:0/3585193142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdf7acd010 0x55fdf7adb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.699+0000 7f5b5c9dc640 1 -- 172.21.15.7:0/3585193142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdf7ad2b70 msgr2=0x55fdf7b4b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.699+0000 7f5b5c9dc640 1 --2- 172.21.15.7:0/3585193142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdf7ad2b70 0x55fdf7b4b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.699+0000 7f5b5c9dc640 1 -- 172.21.15.7:0/3585193142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdf7b4beb0 con 0x55fdf7acda00 2024-06-10T01:09:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.699+0000 7f5b5d9de640 1 --2- 172.21.15.7:0/3585193142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdf7ad2b70 0x55fdf7b4b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:09:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.699+0000 7f5b5d1dd640 1 --2- 172.21.15.7:0/3585193142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdf7acd010 0x55fdf7adb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.699+0000 7f5b5c9dc640 1 --2- 172.21.15.7:0/3585193142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdf7acda00 0x55fdf7adbdd0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5b48009850 tx=0x7f5b48005130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285ef07c4081a48d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.699+0000 7f5b40ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:09:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.699+0000 7f5b40ff9640 1 -- 172.21.15.7:0/3585193142 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b48035040 con 0x55fdf7acda00 2024-06-10T01:09:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.699+0000 7f5b40ff9640 1 -- 172.21.15.7:0/3585193142 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f5b480317b0 con 0x55fdf7acda00 2024-06-10T01:09:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.699+0000 7f5b62220740 1 -- 172.21.15.7:0/3585193142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b5409c450 con 0x55fdf7acda00 2024-06-10T01:09:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.699+0000 7f5b40ff9640 1 -- 172.21.15.7:0/3585193142 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5b48056050 con 0x55fdf7acda00 2024-06-10T01:09:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.699+0000 7f5b62220740 10 client.16314 fetch_fsmap learned FSMap version 30 2024-06-10T01:09:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.699+0000 7f5b62220740 1 -- 172.21.15.7:0/3585193142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fdf7b4d2c0 con 0x55fdf7acda00 2024-06-10T01:09:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.699+0000 7f5b40ff9640 1 -- 172.21.15.7:0/3585193142 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b5409c450 con 0x55fdf7acda00 2024-06-10T01:09:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.700+0000 7f5b40ff9640 1 -- 172.21.15.7:0/3585193142 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f5b480035f0 con 0x55fdf7acda00 2024-06-10T01:09:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.700+0000 7f5b62220740 10 client.16314 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:09:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.700+0000 7f5b62220740 10 client.16314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.700+0000 7f5b62220740 20 client.16314 populate_metadata read hostname 'smithi007' 2024-06-10T01:09:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.700+0000 7f5b62220740 1 --2- 172.21.15.7:0/3585193142 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fdf7b4dcd0 0x55fdf7b500f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.700+0000 7f5b62220740 4 client.16314 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:09:08.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.700+0000 7f5b62220740 1 -- 172.21.15.7:0/3585193142 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fdf7a3dc20 con 0x55fdf7b4dcd0 2024-06-10T01:09:08.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.700+0000 7f5b5d1dd640 1 --2- 172.21.15.7:0/3585193142 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fdf7b4dcd0 0x55fdf7b500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:08.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.700+0000 7f5b40ff9640 10 client.16314 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:08.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.700+0000 7f5b5d1dd640 1 --2- 172.21.15.7:0/3585193142 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fdf7b4dcd0 0x55fdf7b500f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ce16d0c73f8b374 server_cookie=7b4ac61f67012a51 in_seq=0 out_seq=0 2024-06-10T01:09:08.712 INFO:tasks.workunit.client.0.smithi007.stdout: 1 151355 7.62 MB/sec warmup 109 sec latency 19.923 ms 2024-06-10T01:09:08.780 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:09:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.794+0000 7f5b40ff9640 1 -- 172.21.15.7:0/3585193142 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fdf7a3dc20 con 0x55fdf7b4dcd0 2024-06-10T01:09:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.794+0000 7f5b40ff9640 10 client.16314 handle_command_reply: tid=0 2024-06-10T01:09:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.794+0000 7f5b62220740 1 client.16314 shutdown 2024-06-10T01:09:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.794+0000 7f5b62220740 1 -- 172.21.15.7:0/3585193142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdf7acda00 msgr2=0x55fdf7adbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.794+0000 7f5b62220740 1 --2- 172.21.15.7:0/3585193142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdf7acda00 0x55fdf7adbdd0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5b48009850 tx=0x7f5b48005130 comp rx=0 tx=0).stop 2024-06-10T01:09:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.795+0000 7f5b62220740 1 -- 172.21.15.7:0/3585193142 shutdown_connections 2024-06-10T01:09:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.795+0000 7f5b62220740 1 --2- 172.21.15.7:0/3585193142 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fdf7b4dcd0 0x55fdf7b500f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.795+0000 7f5b62220740 1 --2- 172.21.15.7:0/3585193142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdf7ad2b70 0x55fdf7b4b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.795+0000 7f5b40ff9640 0 client.16314 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.795+0000 7f5b62220740 1 --2- 172.21.15.7:0/3585193142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdf7acda00 0x55fdf7adbdd0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.795+0000 7f5b62220740 1 --2- 172.21.15.7:0/3585193142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdf7acd010 0x55fdf7adb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.795+0000 7f5b62220740 1 -- 172.21.15.7:0/3585193142 >> 172.21.15.7:0/3585193142 conn(0x55fdf7ac8680 msgr2=0x55fdf7acb670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.795+0000 7f5b62220740 1 -- 172.21.15.7:0/3585193142 shutdown_connections 2024-06-10T01:09:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.795+0000 7f5b62220740 1 -- 172.21.15.7:0/3585193142 wait complete. 2024-06-10T01:09:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.795+0000 7f5b62220740 20 client.16314 trim_cache size 0 max 16384 2024-06-10T01:09:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:09:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:09:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:09:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.796+0000 7f5b5f441640 1 -- 172.21.15.7:0/3472435813 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5b440a2b80 msgr2=0x7f5b440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.796+0000 7f5b5f441640 1 --2- 172.21.15.7:0/3472435813 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5b440a2b80 0x7f5b440a5040 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55fdf7ace260 tx=0x7f5b50009500 comp rx=0 tx=0).stop 2024-06-10T01:09:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.796+0000 7f5b5f441640 1 -- 172.21.15.7:0/3472435813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b58100930 msgr2=0x7f5b58197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.796+0000 7f5b5f441640 1 --2- 172.21.15.7:0/3472435813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b58100930 0x7f5b58197d00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5b5400e9c0 tx=0x7f5b5400ee80 comp rx=0 tx=0).stop 2024-06-10T01:09:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.796+0000 7f5b5f441640 1 -- 172.21.15.7:0/3472435813 shutdown_connections 2024-06-10T01:09:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.796+0000 7f5b5f441640 1 --2- 172.21.15.7:0/3472435813 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5b440a2b80 0x7f5b440a5040 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.796+0000 7f5b5f441640 1 --2- 172.21.15.7:0/3472435813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b581024e0 0x7f5b5819f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.796+0000 7f5b5f441640 1 --2- 172.21.15.7:0/3472435813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b58101b20 0x7f5b58198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.796+0000 7f5b5f441640 1 --2- 172.21.15.7:0/3472435813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b58100930 0x7f5b58197d00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.796+0000 7f5b5f441640 1 -- 172.21.15.7:0/3472435813 >> 172.21.15.7:0/3472435813 conn(0x7f5b580fbea0 msgr2=0x7f5b580fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.796+0000 7f5b5f441640 1 -- 172.21.15.7:0/3472435813 shutdown_connections 2024-06-10T01:09:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:08.796+0000 7f5b5f441640 1 -- 172.21.15.7:0/3472435813 wait complete. 2024-06-10T01:09:09.053 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:09:09.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:09 smithi007 ceph-mon[32398]: pgmap v526: 129 pgs: 129 active+clean; 44 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 328 op/s 2024-06-10T01:09:09.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:09 smithi078 ceph-mon[36683]: pgmap v526: 129 pgs: 129 active+clean; 44 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 328 op/s 2024-06-10T01:09:09.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:09 smithi175 ceph-mon[36636]: pgmap v526: 129 pgs: 129 active+clean; 44 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 328 op/s 2024-06-10T01:09:09.712 INFO:tasks.workunit.client.0.smithi007.stdout: 1 153126 7.67 MB/sec warmup 110 sec latency 55.692 ms 2024-06-10T01:09:10.712 INFO:tasks.workunit.client.0.smithi007.stdout: 1 155923 7.73 MB/sec warmup 111 sec latency 11.358 ms 2024-06-10T01:09:10.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.865+0000 7fc70519d700 1 -- 172.21.15.7:0/3842150826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc700104f00 msgr2=0x7fc700105360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:10.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.865+0000 7fc70519d700 1 --2- 172.21.15.7:0/3842150826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc700104f00 0x7fc700105360 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc6f800b0a0 tx=0x7fc6f800b3b0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.865+0000 7fc70519d700 1 -- 172.21.15.7:0/3842150826 shutdown_connections 2024-06-10T01:09:10.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.865+0000 7fc70519d700 1 --2- 172.21.15.7:0/3842150826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc700104f00 0x7fc700105360 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.865+0000 7fc70519d700 1 --2- 172.21.15.7:0/3842150826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7001073f0 0x7fc70006ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.865+0000 7fc70519d700 1 --2- 172.21.15.7:0/3842150826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc70006b180 0x7fc70006b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.865+0000 7fc70519d700 1 -- 172.21.15.7:0/3842150826 >> 172.21.15.7:0/3842150826 conn(0x7fc700078430 msgr2=0x7fc700078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:10.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.866+0000 7fc70519d700 1 -- 172.21.15.7:0/3842150826 shutdown_connections 2024-06-10T01:09:10.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.866+0000 7fc70519d700 1 -- 172.21.15.7:0/3842150826 wait complete. 2024-06-10T01:09:10.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.868+0000 7fc70519d700 1 Processor -- start 2024-06-10T01:09:10.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.868+0000 7fc70519d700 1 -- start start 2024-06-10T01:09:10.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.869+0000 7fc70519d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc70006b180 0x7fc70010ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:10.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.869+0000 7fc70519d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc700104f00 0x7fc70010b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:10.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.869+0000 7fc70519d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7001073f0 0x7fc700072ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:10.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.869+0000 7fc6fed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc70006b180 0x7fc70010ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:10.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.869+0000 7fc6fed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc70006b180 0x7fc70010ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57242/0 (socket says 172.21.15.7:57242) 2024-06-10T01:09:10.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.869+0000 7fc70519d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc700118eb0 con 0x7fc70006b180 2024-06-10T01:09:10.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.869+0000 7fc70519d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc700118d30 con 0x7fc7001073f0 2024-06-10T01:09:10.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.870+0000 7fc70519d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc700119030 con 0x7fc700104f00 2024-06-10T01:09:10.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.870+0000 7fc6ff59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7001073f0 0x7fc700072ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:10.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.870+0000 7fc6ff59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7001073f0 0x7fc700072ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:42756/0 (socket says 172.21.15.7:42756) 2024-06-10T01:09:10.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.870+0000 7fc6ff59e700 1 -- 172.21.15.7:0/2723033145 learned_addr learned my addr 172.21.15.7:0/2723033145 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:10.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.870+0000 7fc6ff59e700 1 -- 172.21.15.7:0/2723033145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc700104f00 msgr2=0x7fc70010b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:10.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.870+0000 7fc6ff59e700 1 --2- 172.21.15.7:0/2723033145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc700104f00 0x7fc70010b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.870+0000 7fc6ff59e700 1 -- 172.21.15.7:0/2723033145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc70006b180 msgr2=0x7fc70010ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:10.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.870+0000 7fc6ff59e700 1 --2- 172.21.15.7:0/2723033145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc70006b180 0x7fc70010ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.870+0000 7fc6ff59e700 1 -- 172.21.15.7:0/2723033145 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7000735b0 con 0x7fc7001073f0 2024-06-10T01:09:10.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.870+0000 7fc6fed9d700 1 --2- 172.21.15.7:0/2723033145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc70006b180 0x7fc70010ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:09:10.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.871+0000 7fc6ff59e700 1 --2- 172.21.15.7:0/2723033145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7001073f0 0x7fc700072ce0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc6f800b070 tx=0x7fc6f800cf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:10.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.871+0000 7fc6fcd99700 1 -- 172.21.15.7:0/2723033145 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8021070 con 0x7fc7001073f0 2024-06-10T01:09:10.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.871+0000 7fc6fcd99700 1 -- 172.21.15.7:0/2723033145 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6f8007d60 con 0x7fc7001073f0 2024-06-10T01:09:10.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.871+0000 7fc6fcd99700 1 -- 172.21.15.7:0/2723033145 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f80054e0 con 0x7fc7001073f0 2024-06-10T01:09:10.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.871+0000 7fc70519d700 1 -- 172.21.15.7:0/2723033145 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc700073840 con 0x7fc7001073f0 2024-06-10T01:09:10.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.871+0000 7fc70519d700 1 -- 172.21.15.7:0/2723033145 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc700112c30 con 0x7fc7001073f0 2024-06-10T01:09:10.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.875+0000 7fc6fcd99700 1 -- 172.21.15.7:0/2723033145 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fc6f8024070 con 0x7fc7001073f0 2024-06-10T01:09:10.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.876+0000 7fc6fcd99700 1 --2- 172.21.15.7:0/2723033145 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc6ec0a2cf0 0x7fc6ec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:10.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.876+0000 7fc6fcd99700 1 -- 172.21.15.7:0/2723033145 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc6f80d0750 con 0x7fc7001073f0 2024-06-10T01:09:10.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.876+0000 7fc6fed9d700 1 --2- 172.21.15.7:0/2723033145 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc6ec0a2cf0 0x7fc6ec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:10.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.876+0000 7fc6fed9d700 1 --2- 172.21.15.7:0/2723033145 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc6ec0a2cf0 0x7fc6ec0a51b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0005950 tx=0x7fc6f0005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:10.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.877+0000 7fc70fa20b80 1 Processor -- start 2024-06-10T01:09:10.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.877+0000 7fc70fa20b80 1 -- start start 2024-06-10T01:09:10.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.878+0000 7fc70fa20b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ccd35f60 0x55a1ccd36340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:10.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.878+0000 7fc70fa20b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ccd36880 0x55a1ccd30260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:10.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.878+0000 7fc70fa20b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ccd307a0 0x55a1ccd30bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:10.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.878+0000 7fc70fa20b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a1ccc22f60 con 0x55a1ccd35f60 2024-06-10T01:09:10.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.878+0000 7fc70fa20b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a1ccc81850 con 0x55a1ccd36880 2024-06-10T01:09:10.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.878+0000 7fc70fa20b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a1ccbe9460 con 0x55a1ccd307a0 2024-06-10T01:09:10.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.878+0000 7fc6fed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ccd35f60 0x55a1ccd36340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:10.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.878+0000 7fc6f7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ccd36880 0x55a1ccd30260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:10.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.878+0000 7fc6ff59e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ccd307a0 0x55a1ccd30bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:10.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.878+0000 7fc6fed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ccd35f60 0x55a1ccd36340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57246/0 (socket says 172.21.15.7:57246) 2024-06-10T01:09:10.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.878+0000 7fc6fed9d700 1 -- 172.21.15.7:0/2469633995 learned_addr learned my addr 172.21.15.7:0/2469633995 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.878+0000 7fc6fed9d700 1 -- 172.21.15.7:0/2469633995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ccd307a0 msgr2=0x55a1ccd30bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.878+0000 7fc6fed9d700 1 --2- 172.21.15.7:0/2469633995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ccd307a0 0x55a1ccd30bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.878+0000 7fc6fed9d700 1 -- 172.21.15.7:0/2469633995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ccd36880 msgr2=0x55a1ccd30260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.878+0000 7fc6fed9d700 1 --2- 172.21.15.7:0/2469633995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ccd36880 0x55a1ccd30260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.878+0000 7fc6fed9d700 1 -- 172.21.15.7:0/2469633995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1ccbd08d0 con 0x55a1ccd35f60 2024-06-10T01:09:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.878+0000 7fc6ff59e700 1 --2- 172.21.15.7:0/2469633995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ccd307a0 0x55a1ccd30bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.878+0000 7fc6fed9d700 1 --2- 172.21.15.7:0/2469633995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ccd35f60 0x55a1ccd36340 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000ed30 tx=0x7fc6f0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3443f6da62f7e392 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:10.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.879+0000 7fc6f57fa700 1 -- 172.21.15.7:0/2469633995 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0021070 con 0x55a1ccd35f60 2024-06-10T01:09:10.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.879+0000 7fc6f57fa700 1 -- 172.21.15.7:0/2469633995 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6f0004d10 con 0x55a1ccd35f60 2024-06-10T01:09:10.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.879+0000 7fc6f57fa700 1 -- 172.21.15.7:0/2469633995 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f001d450 con 0x55a1ccd35f60 2024-06-10T01:09:10.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.879+0000 7fc70fa20b80 1 -- 172.21.15.7:0/2469633995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ccd35f60 msgr2=0x55a1ccd36340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:10.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.879+0000 7fc70fa20b80 1 --2- 172.21.15.7:0/2469633995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ccd35f60 0x55a1ccd36340 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000ed30 tx=0x7fc6f0011000 comp rx=0 tx=0).stop 2024-06-10T01:09:10.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.880+0000 7fc70fa20b80 1 -- 172.21.15.7:0/2469633995 shutdown_connections 2024-06-10T01:09:10.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.880+0000 7fc70fa20b80 1 --2- 172.21.15.7:0/2469633995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ccd307a0 0x55a1ccd30bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.880+0000 7fc70fa20b80 1 --2- 172.21.15.7:0/2469633995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ccd35f60 0x55a1ccd36340 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.880+0000 7fc70fa20b80 1 --2- 172.21.15.7:0/2469633995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ccd36880 0x55a1ccd30260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.880+0000 7fc70fa20b80 1 -- 172.21.15.7:0/2469633995 >> 172.21.15.7:0/2469633995 conn(0x55a1ccc5d080 msgr2=0x55a1ccd2b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:10.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.880+0000 7fc70fa20b80 1 -- 172.21.15.7:0/2469633995 shutdown_connections 2024-06-10T01:09:10.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.880+0000 7fc70fa20b80 1 -- 172.21.15.7:0/2469633995 wait complete. 2024-06-10T01:09:10.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.880+0000 7fc70fa20b80 1 -- start start 2024-06-10T01:09:10.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.880+0000 7fc70fa20b80 1 Processor -- start 2024-06-10T01:09:10.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.880+0000 7fc70fa20b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ccd307a0 0x55a1ccd3c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:10.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.880+0000 7fc70fa20b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ccd35f60 0x55a1ccd39910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:10.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.880+0000 7fc70fa20b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ccd36880 0x55a1ccd39e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:10.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.880+0000 7fc70fa20b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a1ccc21fc0 con 0x55a1ccd36880 2024-06-10T01:09:10.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.880+0000 7fc70fa20b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a1ccc7d810 con 0x55a1ccd307a0 2024-06-10T01:09:10.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.880+0000 7fc70fa20b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a1ccd3d260 con 0x55a1ccd35f60 2024-06-10T01:09:10.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.880+0000 7fc70fa20b80 10 client.0 _ll_register_callbacks cb 0x55a1ccc175b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:10.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.880+0000 7fc6ff59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ccd36880 0x55a1ccd39e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:10.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.881+0000 7fc6ff59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ccd36880 0x55a1ccd39e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57258/0 (socket says 172.21.15.7:57258) 2024-06-10T01:09:10.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.881+0000 7fc6ff59e700 1 -- 172.21.15.7:0/968385210 learned_addr learned my addr 172.21.15.7:0/968385210 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:10.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.881+0000 7fc6fed9d700 1 --2- 172.21.15.7:0/968385210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ccd307a0 0x55a1ccd3c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:10.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.881+0000 7fc6f7fff700 1 --2- 172.21.15.7:0/968385210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ccd35f60 0x55a1ccd39910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:10.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.881+0000 7fc6ff59e700 1 -- 172.21.15.7:0/968385210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ccd35f60 msgr2=0x55a1ccd39910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:10.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.881+0000 7fc6ff59e700 1 --2- 172.21.15.7:0/968385210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ccd35f60 0x55a1ccd39910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.881+0000 7fc6ff59e700 1 -- 172.21.15.7:0/968385210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ccd307a0 msgr2=0x55a1ccd3c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:10.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.881+0000 7fc6ff59e700 1 --2- 172.21.15.7:0/968385210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ccd307a0 0x55a1ccd3c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.881+0000 7fc6ff59e700 1 -- 172.21.15.7:0/968385210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1ccc47750 con 0x55a1ccd36880 2024-06-10T01:09:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.881+0000 7fc6ff59e700 1 --2- 172.21.15.7:0/968385210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ccd36880 0x55a1ccd39e50 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc6f80cdf20 tx=0x7fc6f801dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a89777883df8f34 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.881+0000 7fc6e67fc700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:09:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.881+0000 7fc6e67fc700 1 -- 172.21.15.7:0/968385210 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f802b040 con 0x55a1ccd36880 2024-06-10T01:09:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.881+0000 7fc6e67fc700 1 -- 172.21.15.7:0/968385210 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc6f80074e0 con 0x55a1ccd36880 2024-06-10T01:09:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.881+0000 7fc70fa20b80 1 -- 172.21.15.7:0/968385210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6f0004d10 con 0x55a1ccd36880 2024-06-10T01:09:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.881+0000 7fc6e67fc700 1 -- 172.21.15.7:0/968385210 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc6f80a1070 con 0x55a1ccd36880 2024-06-10T01:09:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.881+0000 7fc70fa20b80 10 client.16323 fetch_fsmap learned FSMap version 30 2024-06-10T01:09:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.881+0000 7fc70fa20b80 1 -- 172.21.15.7:0/968385210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a1ccd47a10 con 0x55a1ccd36880 2024-06-10T01:09:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.882+0000 7fc6e67fc700 1 -- 172.21.15.7:0/968385210 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6f0004d10 con 0x55a1ccd36880 2024-06-10T01:09:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.882+0000 7fc6e67fc700 1 -- 172.21.15.7:0/968385210 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc6f8011070 con 0x55a1ccd36880 2024-06-10T01:09:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.882+0000 7fc70fa20b80 10 client.16323 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:09:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.882+0000 7fc70fa20b80 10 client.16323 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:09:10.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.882+0000 7fc70fa20b80 20 client.16323 populate_metadata read hostname 'smithi007' 2024-06-10T01:09:10.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.882+0000 7fc70fa20b80 1 --2- 172.21.15.7:0/968385210 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a1ccd4a020 0x55a1ccd4c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:10.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.882+0000 7fc70fa20b80 4 client.16323 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:09:10.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.882+0000 7fc70fa20b80 1 -- 172.21.15.7:0/968385210 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1ccc8ed50 con 0x55a1ccd4a020 2024-06-10T01:09:10.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.882+0000 7fc6fed9d700 1 --2- 172.21.15.7:0/968385210 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a1ccd4a020 0x55a1ccd4c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:10.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.883+0000 7fc6e67fc700 10 client.16323 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:09:10.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.883+0000 7fc6fed9d700 1 --2- 172.21.15.7:0/968385210 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a1ccd4a020 0x55a1ccd4c410 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=31f154cf6e296902 server_cookie=4b39005169a51fd1 in_seq=0 out_seq=0 2024-06-10T01:09:10.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.883+0000 7fc6e67fc700 1 -- 172.21.15.7:0/968385210 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a1ccc8ed50 con 0x55a1ccd4a020 2024-06-10T01:09:10.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.883+0000 7fc6e67fc700 10 client.16323 handle_command_reply: tid=0 2024-06-10T01:09:10.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.884+0000 7fc70fa20b80 1 client.16323 shutdown 2024-06-10T01:09:10.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.884+0000 7fc70fa20b80 1 -- 172.21.15.7:0/968385210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ccd36880 msgr2=0x55a1ccd39e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:10.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.884+0000 7fc70fa20b80 1 --2- 172.21.15.7:0/968385210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ccd36880 0x55a1ccd39e50 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc6f80cdf20 tx=0x7fc6f801dea0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.884+0000 7fc70fa20b80 1 -- 172.21.15.7:0/968385210 shutdown_connections 2024-06-10T01:09:10.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.884+0000 7fc70fa20b80 1 --2- 172.21.15.7:0/968385210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ccd36880 0x55a1ccd39e50 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.884+0000 7fc70fa20b80 1 --2- 172.21.15.7:0/968385210 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a1ccd4a020 0x55a1ccd4c410 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.884+0000 7fc70fa20b80 1 --2- 172.21.15.7:0/968385210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ccd307a0 0x55a1ccd3c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.884+0000 7fc70fa20b80 1 --2- 172.21.15.7:0/968385210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ccd35f60 0x55a1ccd39910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.884+0000 7fc6e67fc700 0 client.16323 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:09:10.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.884+0000 7fc70fa20b80 1 -- 172.21.15.7:0/968385210 >> 172.21.15.7:0/968385210 conn(0x55a1ccc5d080 msgr2=0x55a1ccd2a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:10.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.884+0000 7fc70fa20b80 1 -- 172.21.15.7:0/968385210 shutdown_connections 2024-06-10T01:09:10.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.884+0000 7fc70fa20b80 1 -- 172.21.15.7:0/968385210 wait complete. 2024-06-10T01:09:10.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.884+0000 7fc70fa20b80 20 client.16323 trim_cache size 0 max 16384 2024-06-10T01:09:10.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.900+0000 7fc70fa20b80 1 Processor -- start 2024-06-10T01:09:10.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.900+0000 7fc70fa20b80 1 -- start start 2024-06-10T01:09:10.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.900+0000 7fc70fa20b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ccd307a0 0x55a1ccd37370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:10.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.900+0000 7fc70fa20b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ccd35f60 0x55a1ccd2ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:10.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.900+0000 7fc70fa20b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ccd36880 0x55a1ccd2b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:10.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.900+0000 7fc70fa20b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc7001c8070 con 0x55a1ccd307a0 2024-06-10T01:09:10.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.900+0000 7fc70fa20b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc7001c58a0 con 0x55a1ccd36880 2024-06-10T01:09:10.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.900+0000 7fc70fa20b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc7001c36f0 con 0x55a1ccd35f60 2024-06-10T01:09:10.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.901+0000 7fc6fed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ccd307a0 0x55a1ccd37370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:10.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.901+0000 7fc6f7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ccd35f60 0x55a1ccd2ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:10.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.901+0000 7fc6ff59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ccd36880 0x55a1ccd2b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:10.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.901+0000 7fc6ff59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ccd36880 0x55a1ccd2b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:42782/0 (socket says 172.21.15.7:42782) 2024-06-10T01:09:10.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.901+0000 7fc6f7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ccd35f60 0x55a1ccd2ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:51860/0 (socket says 172.21.15.7:51860) 2024-06-10T01:09:10.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.901+0000 7fc6fed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ccd307a0 0x55a1ccd37370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57270/0 (socket says 172.21.15.7:57270) 2024-06-10T01:09:10.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.901+0000 7fc6ff59e700 1 -- 172.21.15.7:0/690883413 learned_addr learned my addr 172.21.15.7:0/690883413 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:10.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.901+0000 7fc6ff59e700 1 -- 172.21.15.7:0/690883413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ccd35f60 msgr2=0x55a1ccd2ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:10.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.901+0000 7fc6ff59e700 1 --2- 172.21.15.7:0/690883413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ccd35f60 0x55a1ccd2ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.901+0000 7fc6ff59e700 1 -- 172.21.15.7:0/690883413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ccd307a0 msgr2=0x55a1ccd37370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:10.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.901+0000 7fc6ff59e700 1 --2- 172.21.15.7:0/690883413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ccd307a0 0x55a1ccd37370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.901+0000 7fc6ff59e700 1 -- 172.21.15.7:0/690883413 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1ccd378b0 con 0x55a1ccd36880 2024-06-10T01:09:10.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.901+0000 7fc6fed9d700 1 --2- 172.21.15.7:0/690883413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ccd307a0 0x55a1ccd37370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:10.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.901+0000 7fc6f7fff700 1 --2- 172.21.15.7:0/690883413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ccd35f60 0x55a1ccd2ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:10.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.901+0000 7fc6ff59e700 1 --2- 172.21.15.7:0/690883413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ccd36880 0x55a1ccd2b3c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc6f809cc50 tx=0x7fc6f80099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59c7de74ba9d8f83 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:10.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.901+0000 7fc6f4ff9700 1 -- 172.21.15.7:0/690883413 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f802b070 con 0x55a1ccd36880 2024-06-10T01:09:10.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.901+0000 7fc6f4ff9700 1 -- 172.21.15.7:0/690883413 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6f80074e0 con 0x55a1ccd36880 2024-06-10T01:09:10.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.901+0000 7fc6f4ff9700 1 -- 172.21.15.7:0/690883413 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f80a5410 con 0x55a1ccd36880 2024-06-10T01:09:10.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.902+0000 7fc70fa20b80 1 -- 172.21.15.7:0/690883413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ccd36880 msgr2=0x55a1ccd2b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:10.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.902+0000 7fc70fa20b80 1 --2- 172.21.15.7:0/690883413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ccd36880 0x55a1ccd2b3c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc6f809cc50 tx=0x7fc6f80099e0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.902+0000 7fc70fa20b80 1 -- 172.21.15.7:0/690883413 shutdown_connections 2024-06-10T01:09:10.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.902+0000 7fc70fa20b80 1 --2- 172.21.15.7:0/690883413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ccd36880 0x55a1ccd2b3c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.902+0000 7fc70fa20b80 1 --2- 172.21.15.7:0/690883413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ccd307a0 0x55a1ccd37370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.902+0000 7fc70fa20b80 1 --2- 172.21.15.7:0/690883413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ccd35f60 0x55a1ccd2ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.902+0000 7fc70fa20b80 1 -- 172.21.15.7:0/690883413 >> 172.21.15.7:0/690883413 conn(0x55a1ccc5d080 msgr2=0x55a1ccd36c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:10.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.902+0000 7fc70fa20b80 1 -- 172.21.15.7:0/690883413 shutdown_connections 2024-06-10T01:09:10.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.902+0000 7fc70fa20b80 1 -- 172.21.15.7:0/690883413 wait complete. 2024-06-10T01:09:10.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.903+0000 7fc70fa20b80 1 -- start start 2024-06-10T01:09:10.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.903+0000 7fc70fa20b80 1 Processor -- start 2024-06-10T01:09:10.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.903+0000 7fc70fa20b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ccd307a0 0x55a1ccd42af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:10.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.903+0000 7fc70fa20b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ccd35f60 0x55a1ccd514e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:10.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.903+0000 7fc70fa20b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ccd36880 0x55a1ccd51a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:10.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.903+0000 7fc70fa20b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc7001c3d60 con 0x55a1ccd307a0 2024-06-10T01:09:10.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.903+0000 7fc70fa20b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc7001c51d0 con 0x55a1ccd35f60 2024-06-10T01:09:10.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.903+0000 7fc70fa20b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc7001c4430 con 0x55a1ccd36880 2024-06-10T01:09:10.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.903+0000 7fc70fa20b80 10 client.0 _ll_register_callbacks cb 0x55a1ccc175b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:10.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.903+0000 7fc6fed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ccd307a0 0x55a1ccd42af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:10.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.903+0000 7fc6fed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ccd307a0 0x55a1ccd42af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57284/0 (socket says 172.21.15.7:57284) 2024-06-10T01:09:10.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.903+0000 7fc6fed9d700 1 -- 172.21.15.7:0/1404592974 learned_addr learned my addr 172.21.15.7:0/1404592974 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:10.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.903+0000 7fc6f7fff700 1 --2- 172.21.15.7:0/1404592974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ccd35f60 0x55a1ccd514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:10.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.903+0000 7fc6ff59e700 1 --2- 172.21.15.7:0/1404592974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ccd36880 0x55a1ccd51a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:10.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.904+0000 7fc6ff59e700 1 -- 172.21.15.7:0/1404592974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ccd35f60 msgr2=0x55a1ccd514e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:10.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.904+0000 7fc6ff59e700 1 --2- 172.21.15.7:0/1404592974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ccd35f60 0x55a1ccd514e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.904+0000 7fc6ff59e700 1 -- 172.21.15.7:0/1404592974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ccd307a0 msgr2=0x55a1ccd42af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:10.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.904+0000 7fc6ff59e700 1 --2- 172.21.15.7:0/1404592974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ccd307a0 0x55a1ccd42af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.904+0000 7fc6ff59e700 1 -- 172.21.15.7:0/1404592974 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1ccd52090 con 0x55a1ccd36880 2024-06-10T01:09:10.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.904+0000 7fc6fed9d700 1 --2- 172.21.15.7:0/1404592974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ccd307a0 0x55a1ccd42af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:10.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.904+0000 7fc6f7fff700 1 --2- 172.21.15.7:0/1404592974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ccd35f60 0x55a1ccd514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.904+0000 7fc6ff59e700 1 --2- 172.21.15.7:0/1404592974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ccd36880 0x55a1ccd51a20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc6f809af70 tx=0x7fc6f8098a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6609aa71f1054bdd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.904+0000 7fc6e67fc700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:09:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.904+0000 7fc6e67fc700 1 -- 172.21.15.7:0/1404592974 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f802b070 con 0x55a1ccd36880 2024-06-10T01:09:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.904+0000 7fc6e67fc700 1 -- 172.21.15.7:0/1404592974 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc6f807bd30 con 0x55a1ccd36880 2024-06-10T01:09:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.904+0000 7fc70fa20b80 1 -- 172.21.15.7:0/1404592974 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6f80074e0 con 0x55a1ccd36880 2024-06-10T01:09:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.904+0000 7fc6e67fc700 1 -- 172.21.15.7:0/1404592974 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc6f809c320 con 0x55a1ccd36880 2024-06-10T01:09:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.904+0000 7fc70fa20b80 10 client.25873 fetch_fsmap learned FSMap version 30 2024-06-10T01:09:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.904+0000 7fc70fa20b80 1 -- 172.21.15.7:0/1404592974 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a1ccd52090 con 0x55a1ccd36880 2024-06-10T01:09:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.904+0000 7fc6e67fc700 1 -- 172.21.15.7:0/1404592974 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6f80074e0 con 0x55a1ccd36880 2024-06-10T01:09:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.905+0000 7fc6e67fc700 1 -- 172.21.15.7:0/1404592974 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc6f8011050 con 0x55a1ccd36880 2024-06-10T01:09:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.905+0000 7fc70fa20b80 10 client.25873 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:09:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.905+0000 7fc70fa20b80 10 client.25873 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:09:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.905+0000 7fc70fa20b80 20 client.25873 populate_metadata read hostname 'smithi007' 2024-06-10T01:09:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.905+0000 7fc70fa20b80 1 --2- 172.21.15.7:0/1404592974 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a1ccdb4db0 0x55a1ccdb71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:10.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.905+0000 7fc70fa20b80 4 client.25873 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:09:10.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.905+0000 7fc70fa20b80 1 -- 172.21.15.7:0/1404592974 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a1ccc892d0 con 0x55a1ccdb4db0 2024-06-10T01:09:10.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.905+0000 7fc6fed9d700 1 --2- 172.21.15.7:0/1404592974 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a1ccdb4db0 0x55a1ccdb71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:10.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.905+0000 7fc6e67fc700 10 client.25873 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:09:10.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.905+0000 7fc6fed9d700 1 --2- 172.21.15.7:0/1404592974 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a1ccdb4db0 0x55a1ccdb71a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f9201a3370c3a55 server_cookie=14109218478d1ad1 in_seq=0 out_seq=0 2024-06-10T01:09:10.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.905+0000 7fc6e67fc700 1 -- 172.21.15.7:0/1404592974 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a1ccc892d0 con 0x55a1ccdb4db0 2024-06-10T01:09:10.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.905+0000 7fc6e67fc700 10 client.25873 handle_command_reply: tid=0 2024-06-10T01:09:10.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.905+0000 7fc70fa20b80 1 client.25873 shutdown 2024-06-10T01:09:10.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.905+0000 7fc70fa20b80 1 -- 172.21.15.7:0/1404592974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ccd36880 msgr2=0x55a1ccd51a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:10.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.905+0000 7fc70fa20b80 1 --2- 172.21.15.7:0/1404592974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ccd36880 0x55a1ccd51a20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc6f809af70 tx=0x7fc6f8098a10 comp rx=0 tx=0).stop 2024-06-10T01:09:10.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.906+0000 7fc70fa20b80 1 -- 172.21.15.7:0/1404592974 shutdown_connections 2024-06-10T01:09:10.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.906+0000 7fc70fa20b80 1 --2- 172.21.15.7:0/1404592974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ccd36880 0x55a1ccd51a20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.906+0000 7fc70fa20b80 1 --2- 172.21.15.7:0/1404592974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ccd307a0 0x55a1ccd42af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.906+0000 7fc70fa20b80 1 --2- 172.21.15.7:0/1404592974 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a1ccdb4db0 0x55a1ccdb71a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.906+0000 7fc70fa20b80 1 --2- 172.21.15.7:0/1404592974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ccd35f60 0x55a1ccd514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.906+0000 7fc70fa20b80 1 -- 172.21.15.7:0/1404592974 >> 172.21.15.7:0/1404592974 conn(0x55a1ccc5d080 msgr2=0x55a1ccd38200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:10.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.906+0000 7fc6e67fc700 0 client.25873 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:09:10.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.906+0000 7fc70fa20b80 1 -- 172.21.15.7:0/1404592974 shutdown_connections 2024-06-10T01:09:10.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.906+0000 7fc70fa20b80 1 -- 172.21.15.7:0/1404592974 wait complete. 2024-06-10T01:09:10.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.906+0000 7fc70fa20b80 20 client.25873 trim_cache size 0 max 16384 2024-06-10T01:09:10.907 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:09:10.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.906+0000 7fc70519d700 1 -- 172.21.15.7:0/2723033145 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc6ec0a2cf0 msgr2=0x7fc6ec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:10.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.906+0000 7fc70519d700 1 --2- 172.21.15.7:0/2723033145 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc6ec0a2cf0 0x7fc6ec0a51b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0005950 tx=0x7fc6f0005f50 comp rx=0 tx=0).stop 2024-06-10T01:09:10.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.906+0000 7fc70519d700 1 -- 172.21.15.7:0/2723033145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7001073f0 msgr2=0x7fc700072ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:10.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.906+0000 7fc70519d700 1 --2- 172.21.15.7:0/2723033145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7001073f0 0x7fc700072ce0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc6f800b070 tx=0x7fc6f800cf80 comp rx=0 tx=0).stop 2024-06-10T01:09:10.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.907+0000 7fc70519d700 1 -- 172.21.15.7:0/2723033145 shutdown_connections 2024-06-10T01:09:10.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.907+0000 7fc70519d700 1 --2- 172.21.15.7:0/2723033145 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc6ec0a2cf0 0x7fc6ec0a51b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.907+0000 7fc70519d700 1 --2- 172.21.15.7:0/2723033145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7001073f0 0x7fc700072ce0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.907+0000 7fc70519d700 1 --2- 172.21.15.7:0/2723033145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc70006b180 0x7fc70010ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.907+0000 7fc70519d700 1 --2- 172.21.15.7:0/2723033145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc700104f00 0x7fc70010b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:10.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.907+0000 7fc70519d700 1 -- 172.21.15.7:0/2723033145 >> 172.21.15.7:0/2723033145 conn(0x7fc700078430 msgr2=0x7fc700103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:10.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.907+0000 7fc70519d700 1 -- 172.21.15.7:0/2723033145 shutdown_connections 2024-06-10T01:09:10.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:10.907+0000 7fc70519d700 1 -- 172.21.15.7:0/2723033145 wait complete. 2024-06-10T01:09:11.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:11 smithi007 ceph-mon[32398]: pgmap v527: 129 pgs: 129 active+clean; 48 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 372 op/s 2024-06-10T01:09:11.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:11 smithi078 ceph-mon[36683]: pgmap v527: 129 pgs: 129 active+clean; 48 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 372 op/s 2024-06-10T01:09:11.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:11 smithi175 ceph-mon[36636]: pgmap v527: 129 pgs: 129 active+clean; 48 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 372 op/s 2024-06-10T01:09:11.695 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:09:11.713 INFO:tasks.workunit.client.0.smithi007.stdout: 1 157300 7.72 MB/sec warmup 112 sec latency 27.404 ms 2024-06-10T01:09:11.970 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:09:12.713 INFO:tasks.workunit.client.0.smithi007.stdout: 1 159982 7.80 MB/sec warmup 113 sec latency 10.077 ms 2024-06-10T01:09:13.201 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:13 smithi007 ceph-mon[32398]: pgmap v528: 129 pgs: 129 active+clean; 48 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.5 MiB/s wr, 266 op/s 2024-06-10T01:09:13.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:13 smithi078 ceph-mon[36683]: pgmap v528: 129 pgs: 129 active+clean; 48 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.5 MiB/s wr, 266 op/s 2024-06-10T01:09:13.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:13 smithi175 ceph-mon[36636]: pgmap v528: 129 pgs: 129 active+clean; 48 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.5 MiB/s wr, 266 op/s 2024-06-10T01:09:13.712 INFO:tasks.workunit.client.0.smithi007.stdout: 1 161839 7.78 MB/sec warmup 114 sec latency 64.670 ms 2024-06-10T01:09:13.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.767+0000 7f69c0d41700 1 -- 172.21.15.7:0/1464065059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69bc103300 msgr2=0x7f69bc1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:13.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.767+0000 7f69c0d41700 1 --2- 172.21.15.7:0/1464065059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69bc103300 0x7f69bc1036e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f69ac0099c0 tx=0x7f69ac009cd0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.767+0000 7f69c0d41700 1 -- 172.21.15.7:0/1464065059 shutdown_connections 2024-06-10T01:09:13.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.767+0000 7f69c0d41700 1 --2- 172.21.15.7:0/1464065059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69bc1017f0 0x7f69bc1115e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.767+0000 7f69c0d41700 1 --2- 172.21.15.7:0/1464065059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69bc103300 0x7f69bc1036e0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.767+0000 7f69c0d41700 1 --2- 172.21.15.7:0/1464065059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69bc100e50 0x7f69bc1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.767+0000 7f69c0d41700 1 -- 172.21.15.7:0/1464065059 >> 172.21.15.7:0/1464065059 conn(0x7f69bc0766b0 msgr2=0x7f69bc076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:13.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.767+0000 7f69c0d41700 1 -- 172.21.15.7:0/1464065059 shutdown_connections 2024-06-10T01:09:13.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.767+0000 7f69c0d41700 1 -- 172.21.15.7:0/1464065059 wait complete. 2024-06-10T01:09:13.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.768+0000 7f69c0d41700 1 Processor -- start 2024-06-10T01:09:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.768+0000 7f69c0d41700 1 -- start start 2024-06-10T01:09:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.768+0000 7f69c0d41700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69bc100e50 0x7f69bc19af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.768+0000 7f69c0d41700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69bc1017f0 0x7f69bc19b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.768+0000 7f69c0d41700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69bc103300 0x7f69bc19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.768+0000 7f69c0d41700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc114af0 con 0x7f69bc103300 2024-06-10T01:09:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.768+0000 7f69c0d41700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc114970 con 0x7f69bc100e50 2024-06-10T01:09:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.768+0000 7f69c0d41700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc114c70 con 0x7f69bc1017f0 2024-06-10T01:09:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.768+0000 7f69ba59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69bc100e50 0x7f69bc19af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.768+0000 7f69ba59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69bc100e50 0x7f69bc19af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:32782/0 (socket says 172.21.15.7:32782) 2024-06-10T01:09:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.768+0000 7f69bad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69bc103300 0x7f69bc19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.768+0000 7f69b9d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69bc1017f0 0x7f69bc19b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.768+0000 7f69ba59c700 1 -- 172.21.15.7:0/2333355660 learned_addr learned my addr 172.21.15.7:0/2333355660 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.769+0000 7f69b9d9b700 1 -- 172.21.15.7:0/2333355660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69bc100e50 msgr2=0x7f69bc19af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:13.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.769+0000 7f69b9d9b700 1 --2- 172.21.15.7:0/2333355660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69bc100e50 0x7f69bc19af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.769+0000 7f69b9d9b700 1 -- 172.21.15.7:0/2333355660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69bc103300 msgr2=0x7f69bc19f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:13.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.769+0000 7f69b9d9b700 1 --2- 172.21.15.7:0/2333355660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69bc103300 0x7f69bc19f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.769+0000 7f69b9d9b700 1 -- 172.21.15.7:0/2333355660 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69bc19ff40 con 0x7f69bc1017f0 2024-06-10T01:09:13.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.769+0000 7f69bad9d700 1 --2- 172.21.15.7:0/2333355660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69bc103300 0x7f69bc19f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:13.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.769+0000 7f69ba59c700 1 --2- 172.21.15.7:0/2333355660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69bc100e50 0x7f69bc19af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:13.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.769+0000 7f69b9d9b700 1 --2- 172.21.15.7:0/2333355660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69bc1017f0 0x7f69bc19b4c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f69b000e9f0 tx=0x7f69b000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:13.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.769+0000 7f69ab7fe700 1 -- 172.21.15.7:0/2333355660 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b0004220 con 0x7f69bc1017f0 2024-06-10T01:09:13.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.769+0000 7f69ab7fe700 1 -- 172.21.15.7:0/2333355660 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69b0004d10 con 0x7f69bc1017f0 2024-06-10T01:09:13.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.769+0000 7f69c0d41700 1 -- 172.21.15.7:0/2333355660 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69bc1a0230 con 0x7f69bc1017f0 2024-06-10T01:09:13.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.769+0000 7f69ab7fe700 1 -- 172.21.15.7:0/2333355660 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b0005020 con 0x7f69bc1017f0 2024-06-10T01:09:13.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.769+0000 7f69c0d41700 1 -- 172.21.15.7:0/2333355660 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69bc1a7ab0 con 0x7f69bc1017f0 2024-06-10T01:09:13.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.772+0000 7f69ab7fe700 1 -- 172.21.15.7:0/2333355660 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f69b0004750 con 0x7f69bc1017f0 2024-06-10T01:09:13.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.773+0000 7f69ab7fe700 1 --2- 172.21.15.7:0/2333355660 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f69a40a2c20 0x7f69a40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:13.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.773+0000 7f69cb5c4b80 1 Processor -- start 2024-06-10T01:09:13.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.773+0000 7f69ba59c700 1 --2- 172.21.15.7:0/2333355660 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f69a40a2c20 0x7f69a40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:13.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.773+0000 7f69ab7fe700 1 -- 172.21.15.7:0/2333355660 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f69b00cf2e0 con 0x7f69bc1017f0 2024-06-10T01:09:13.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.774+0000 7f69cb5c4b80 1 -- start start 2024-06-10T01:09:13.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.774+0000 7f69cb5c4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abbb91c640 0x55abbb91ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:13.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.774+0000 7f69cb5c4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbb91cf60 0x55abbb9169e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:13.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.774+0000 7f69cb5c4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbb916f20 0x55abbb917340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:13.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.774+0000 7f69cb5c4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55abbb868810 con 0x55abbb91c640 2024-06-10T01:09:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.774+0000 7f69cb5c4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55abbb810720 con 0x55abbb91cf60 2024-06-10T01:09:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.774+0000 7f69cb5c4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55abbb8647d0 con 0x55abbb916f20 2024-06-10T01:09:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.774+0000 7f69ba59c700 1 --2- 172.21.15.7:0/2333355660 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f69a40a2c20 0x7f69a40a50e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009990 tx=0x7f69ac005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.774+0000 7f69bad9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbb916f20 0x55abbb917340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.774+0000 7f69ba59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abbb91c640 0x55abbb91ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.774+0000 7f69bad9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbb916f20 0x55abbb917340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43068/0 (socket says 172.21.15.7:43068) 2024-06-10T01:09:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.774+0000 7f69bad9d700 1 -- 172.21.15.7:0/3368546765 learned_addr learned my addr 172.21.15.7:0/3368546765 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.774+0000 7f69b9d9b700 1 --2- 172.21.15.7:0/3368546765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbb91cf60 0x55abbb9169e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.774+0000 7f69b9d9b700 1 -- 172.21.15.7:0/3368546765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbb916f20 msgr2=0x55abbb917340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.774+0000 7f69b9d9b700 1 --2- 172.21.15.7:0/3368546765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbb916f20 0x55abbb917340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.774+0000 7f69b9d9b700 1 -- 172.21.15.7:0/3368546765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abbb91c640 msgr2=0x55abbb91ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.774+0000 7f69b9d9b700 1 --2- 172.21.15.7:0/3368546765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abbb91c640 0x55abbb91ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.774+0000 7f69b9d9b700 1 -- 172.21.15.7:0/3368546765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abbb806f50 con 0x55abbb91cf60 2024-06-10T01:09:13.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.774+0000 7f69bad9d700 1 --2- 172.21.15.7:0/3368546765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbb916f20 0x55abbb917340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:13.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.775+0000 7f69b9d9b700 1 --2- 172.21.15.7:0/3368546765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbb91cf60 0x55abbb9169e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f69b000c5b0 tx=0x7f69b009b6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4667e6f1858f18 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:13.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.775+0000 7f69a8ff9700 1 -- 172.21.15.7:0/3368546765 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b001c040 con 0x55abbb91cf60 2024-06-10T01:09:13.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.775+0000 7f69a8ff9700 1 -- 172.21.15.7:0/3368546765 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69b009b7c0 con 0x55abbb91cf60 2024-06-10T01:09:13.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.775+0000 7f69a8ff9700 1 -- 172.21.15.7:0/3368546765 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b0086070 con 0x55abbb91cf60 2024-06-10T01:09:13.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.776+0000 7f69cb5c4b80 1 -- 172.21.15.7:0/3368546765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbb91cf60 msgr2=0x55abbb9169e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:13.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.776+0000 7f69cb5c4b80 1 --2- 172.21.15.7:0/3368546765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbb91cf60 0x55abbb9169e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f69b000c5b0 tx=0x7f69b009b6a0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.776+0000 7f69cb5c4b80 1 -- 172.21.15.7:0/3368546765 shutdown_connections 2024-06-10T01:09:13.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.776+0000 7f69cb5c4b80 1 --2- 172.21.15.7:0/3368546765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbb916f20 0x55abbb917340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.776+0000 7f69cb5c4b80 1 --2- 172.21.15.7:0/3368546765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abbb91c640 0x55abbb91ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.776+0000 7f69cb5c4b80 1 --2- 172.21.15.7:0/3368546765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbb91cf60 0x55abbb9169e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.776+0000 7f69cb5c4b80 1 -- 172.21.15.7:0/3368546765 >> 172.21.15.7:0/3368546765 conn(0x55abbb841cc0 msgr2=0x55abbb911850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:13.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.776+0000 7f69cb5c4b80 1 -- 172.21.15.7:0/3368546765 shutdown_connections 2024-06-10T01:09:13.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.776+0000 7f69cb5c4b80 1 -- 172.21.15.7:0/3368546765 wait complete. 2024-06-10T01:09:13.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.776+0000 7f69cb5c4b80 1 -- start start 2024-06-10T01:09:13.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.776+0000 7f69cb5c4b80 1 Processor -- start 2024-06-10T01:09:13.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.777+0000 7f69cb5c4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbb916f20 0x55abbb922d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:13.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.777+0000 7f69cb5c4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbb91c640 0x55abbb920060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:13.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.777+0000 7f69cb5c4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abbb91cf60 0x55abbb9205a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:13.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.777+0000 7f69cb5c4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55abbb9239d0 con 0x55abbb91cf60 2024-06-10T01:09:13.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.777+0000 7f69cb5c4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55abbb808bf0 con 0x55abbb91c640 2024-06-10T01:09:13.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.777+0000 7f69cb5c4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55abbb923b50 con 0x55abbb916f20 2024-06-10T01:09:13.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.777+0000 7f69cb5c4b80 10 client.0 _ll_register_callbacks cb 0x55abbb802920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:13.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.777+0000 7f69ba59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbb916f20 0x55abbb922d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:13.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.777+0000 7f69b9d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbb91c640 0x55abbb920060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:13.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.777+0000 7f69bad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abbb91cf60 0x55abbb9205a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:13.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.777+0000 7f69b9d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbb91c640 0x55abbb920060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:32794/0 (socket says 172.21.15.7:32794) 2024-06-10T01:09:13.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.777+0000 7f69b9d9b700 1 -- 172.21.15.7:0/2906359144 learned_addr learned my addr 172.21.15.7:0/2906359144 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:13.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.777+0000 7f69ba59c700 1 -- 172.21.15.7:0/2906359144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbb91c640 msgr2=0x55abbb920060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:13.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.777+0000 7f69ba59c700 1 --2- 172.21.15.7:0/2906359144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbb91c640 0x55abbb920060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.777+0000 7f69ba59c700 1 -- 172.21.15.7:0/2906359144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abbb91cf60 msgr2=0x55abbb9205a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:13.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.777+0000 7f69ba59c700 1 --2- 172.21.15.7:0/2906359144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abbb91cf60 0x55abbb9205a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.777+0000 7f69ba59c700 1 -- 172.21.15.7:0/2906359144 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abbb7b78d0 con 0x55abbb916f20 2024-06-10T01:09:13.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.777+0000 7f69bad9d700 1 --2- 172.21.15.7:0/2906359144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abbb91cf60 0x55abbb9205a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:13.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.778+0000 7f69ba59c700 1 --2- 172.21.15.7:0/2906359144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbb916f20 0x55abbb922d60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f69ac006010 tx=0x7f69ac005050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f85883fb1523686e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:13.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.778+0000 7f69a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:09:13.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.778+0000 7f69a1ffb700 1 -- 172.21.15.7:0/2906359144 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac033060 con 0x55abbb916f20 2024-06-10T01:09:13.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.778+0000 7f69a1ffb700 1 -- 172.21.15.7:0/2906359144 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f69ac023660 con 0x55abbb916f20 2024-06-10T01:09:13.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.778+0000 7f69cb5c4b80 1 -- 172.21.15.7:0/2906359144 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69b009b7c0 con 0x55abbb916f20 2024-06-10T01:09:13.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.778+0000 7f69a1ffb700 1 -- 172.21.15.7:0/2906359144 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f69ac03a050 con 0x55abbb916f20 2024-06-10T01:09:13.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.778+0000 7f69cb5c4b80 10 client.25885 fetch_fsmap learned FSMap version 30 2024-06-10T01:09:13.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.778+0000 7f69cb5c4b80 1 -- 172.21.15.7:0/2906359144 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55abbb936b60 con 0x55abbb916f20 2024-06-10T01:09:13.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.779+0000 7f69a1ffb700 1 -- 172.21.15.7:0/2906359144 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69b009b7c0 con 0x55abbb916f20 2024-06-10T01:09:13.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.779+0000 7f69a1ffb700 1 -- 172.21.15.7:0/2906359144 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f69ac02e9a0 con 0x55abbb916f20 2024-06-10T01:09:13.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.779+0000 7f69cb5c4b80 10 client.25885 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:09:13.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.779+0000 7f69cb5c4b80 10 client.25885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:13.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.779+0000 7f69cb5c4b80 20 client.25885 populate_metadata read hostname 'smithi007' 2024-06-10T01:09:13.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.779+0000 7f69cb5c4b80 1 --2- 172.21.15.7:0/2906359144 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55abbb9391a0 0x55abbb93b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:13.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.779+0000 7f69cb5c4b80 4 client.25885 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:09:13.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.779+0000 7f69cb5c4b80 1 -- 172.21.15.7:0/2906359144 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abbb875d10 con 0x55abbb9391a0 2024-06-10T01:09:13.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.779+0000 7f69b9d9b700 1 --2- 172.21.15.7:0/2906359144 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55abbb9391a0 0x55abbb93b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:13.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.780+0000 7f69b9d9b700 1 --2- 172.21.15.7:0/2906359144 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55abbb9391a0 0x55abbb93b590 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2f65e22ad6591d2 server_cookie=7ef7cacd80699262 in_seq=0 out_seq=0 2024-06-10T01:09:13.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.780+0000 7f69a1ffb700 10 client.25885 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:13.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.781+0000 7f69a1ffb700 1 -- 172.21.15.7:0/2906359144 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55abbb875d10 con 0x55abbb9391a0 2024-06-10T01:09:13.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.781+0000 7f69a1ffb700 10 client.25885 handle_command_reply: tid=0 2024-06-10T01:09:13.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.781+0000 7f69cb5c4b80 1 client.25885 shutdown 2024-06-10T01:09:13.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.781+0000 7f69cb5c4b80 1 -- 172.21.15.7:0/2906359144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbb916f20 msgr2=0x55abbb922d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:13.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.781+0000 7f69cb5c4b80 1 --2- 172.21.15.7:0/2906359144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbb916f20 0x55abbb922d60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f69ac006010 tx=0x7f69ac005050 comp rx=0 tx=0).stop 2024-06-10T01:09:13.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.781+0000 7f69cb5c4b80 1 -- 172.21.15.7:0/2906359144 shutdown_connections 2024-06-10T01:09:13.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.781+0000 7f69cb5c4b80 1 --2- 172.21.15.7:0/2906359144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abbb91cf60 0x55abbb9205a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.781+0000 7f69cb5c4b80 1 --2- 172.21.15.7:0/2906359144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbb916f20 0x55abbb922d60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.781+0000 7f69cb5c4b80 1 --2- 172.21.15.7:0/2906359144 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55abbb9391a0 0x55abbb93b590 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.781+0000 7f69cb5c4b80 1 --2- 172.21.15.7:0/2906359144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbb91c640 0x55abbb920060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.781+0000 7f69cb5c4b80 1 -- 172.21.15.7:0/2906359144 >> 172.21.15.7:0/2906359144 conn(0x55abbb841cc0 msgr2=0x55abbb9159d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:13.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.781+0000 7f69a1ffb700 0 client.25885 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:13.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.781+0000 7f69cb5c4b80 1 -- 172.21.15.7:0/2906359144 shutdown_connections 2024-06-10T01:09:13.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.781+0000 7f69cb5c4b80 1 -- 172.21.15.7:0/2906359144 wait complete. 2024-06-10T01:09:13.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.781+0000 7f69cb5c4b80 20 client.25885 trim_cache size 0 max 16384 2024-06-10T01:09:13.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.803+0000 7f69cb5c4b80 1 Processor -- start 2024-06-10T01:09:13.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.803+0000 7f69cb5c4b80 1 -- start start 2024-06-10T01:09:13.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.803+0000 7f69cb5c4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbb916f20 0x55abbb9169e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:13.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.803+0000 7f69cb5c4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abbb91c640 0x55abbb91da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:13.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.803+0000 7f69cb5c4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbb91cf60 0x55abbb940e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:13.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.803+0000 7f69cb5c4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc1aeaa0 con 0x55abbb91c640 2024-06-10T01:09:13.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.803+0000 7f69cb5c4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc1ac2d0 con 0x55abbb916f20 2024-06-10T01:09:13.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.803+0000 7f69cb5c4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc1aa150 con 0x55abbb91cf60 2024-06-10T01:09:13.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.803+0000 7f69ba59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbb916f20 0x55abbb9169e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:13.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.803+0000 7f69b9d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abbb91c640 0x55abbb91da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:13.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.803+0000 7f69ba59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbb916f20 0x55abbb9169e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:32806/0 (socket says 172.21.15.7:32806) 2024-06-10T01:09:13.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.803+0000 7f69bad9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbb91cf60 0x55abbb940e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:13.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.803+0000 7f69ba59c700 1 -- 172.21.15.7:0/2892888647 learned_addr learned my addr 172.21.15.7:0/2892888647 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:13.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.804+0000 7f69ba59c700 1 -- 172.21.15.7:0/2892888647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbb91cf60 msgr2=0x55abbb940e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:13.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.804+0000 7f69ba59c700 1 --2- 172.21.15.7:0/2892888647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbb91cf60 0x55abbb940e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.804+0000 7f69ba59c700 1 -- 172.21.15.7:0/2892888647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abbb91c640 msgr2=0x55abbb91da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:13.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.804+0000 7f69ba59c700 1 --2- 172.21.15.7:0/2892888647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abbb91c640 0x55abbb91da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.804+0000 7f69ba59c700 1 -- 172.21.15.7:0/2892888647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abbb917820 con 0x55abbb916f20 2024-06-10T01:09:13.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.804+0000 7f69ba59c700 1 --2- 172.21.15.7:0/2892888647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbb916f20 0x55abbb9169e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009d10 tx=0x7f69ac005810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41f336e3056550dd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:13.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.804+0000 7f69a3fff700 1 -- 172.21.15.7:0/2892888647 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac033050 con 0x55abbb916f20 2024-06-10T01:09:13.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.804+0000 7f69a3fff700 1 -- 172.21.15.7:0/2892888647 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69ac043070 con 0x55abbb916f20 2024-06-10T01:09:13.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.804+0000 7f69a3fff700 1 -- 172.21.15.7:0/2892888647 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac023b90 con 0x55abbb916f20 2024-06-10T01:09:13.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.805+0000 7f69cb5c4b80 1 -- 172.21.15.7:0/2892888647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbb916f20 msgr2=0x55abbb9169e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:13.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.805+0000 7f69cb5c4b80 1 --2- 172.21.15.7:0/2892888647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbb916f20 0x55abbb9169e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009d10 tx=0x7f69ac005810 comp rx=0 tx=0).stop 2024-06-10T01:09:13.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.805+0000 7f69cb5c4b80 1 -- 172.21.15.7:0/2892888647 shutdown_connections 2024-06-10T01:09:13.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.805+0000 7f69cb5c4b80 1 --2- 172.21.15.7:0/2892888647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbb91cf60 0x55abbb940e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.805+0000 7f69cb5c4b80 1 --2- 172.21.15.7:0/2892888647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbb916f20 0x55abbb9169e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.805+0000 7f69cb5c4b80 1 --2- 172.21.15.7:0/2892888647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abbb91c640 0x55abbb91da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.805+0000 7f69cb5c4b80 1 -- 172.21.15.7:0/2892888647 >> 172.21.15.7:0/2892888647 conn(0x55abbb841cc0 msgr2=0x55abbb91d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:13.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.805+0000 7f69cb5c4b80 1 -- 172.21.15.7:0/2892888647 shutdown_connections 2024-06-10T01:09:13.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.805+0000 7f69cb5c4b80 1 -- 172.21.15.7:0/2892888647 wait complete. 2024-06-10T01:09:13.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.805+0000 7f69cb5c4b80 1 -- start start 2024-06-10T01:09:13.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.805+0000 7f69cb5c4b80 1 Processor -- start 2024-06-10T01:09:13.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.805+0000 7f69cb5c4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbb916f20 0x55abbb931c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:13.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.805+0000 7f69cb5c4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abbb91c640 0x55abbb92c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:13.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.805+0000 7f69cb5c4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbb91cf60 0x55abbb92cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:13.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.805+0000 7f69cb5c4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc1aa790 con 0x55abbb91c640 2024-06-10T01:09:13.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.805+0000 7f69cb5c4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc1abc00 con 0x55abbb916f20 2024-06-10T01:09:13.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.805+0000 7f69cb5c4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc1aae60 con 0x55abbb91cf60 2024-06-10T01:09:13.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.806+0000 7f69cb5c4b80 10 client.0 _ll_register_callbacks cb 0x55abbb802920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:13.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.806+0000 7f69b9d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abbb91c640 0x55abbb92c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:13.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.806+0000 7f69b9d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abbb91c640 0x55abbb92c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43204/0 (socket says 172.21.15.7:43204) 2024-06-10T01:09:13.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.806+0000 7f69b9d9b700 1 -- 172.21.15.7:0/1411233504 learned_addr learned my addr 172.21.15.7:0/1411233504 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:13.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.806+0000 7f69bad9d700 1 --2- 172.21.15.7:0/1411233504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbb91cf60 0x55abbb92cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:13.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.806+0000 7f69ba59c700 1 --2- 172.21.15.7:0/1411233504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbb916f20 0x55abbb931c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:13.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.806+0000 7f69b9d9b700 1 -- 172.21.15.7:0/1411233504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbb91cf60 msgr2=0x55abbb92cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:13.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.806+0000 7f69b9d9b700 1 --2- 172.21.15.7:0/1411233504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbb91cf60 0x55abbb92cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.806+0000 7f69b9d9b700 1 -- 172.21.15.7:0/1411233504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbb916f20 msgr2=0x55abbb931c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:13.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.806+0000 7f69b9d9b700 1 --2- 172.21.15.7:0/1411233504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbb916f20 0x55abbb931c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.806+0000 7f69b9d9b700 1 -- 172.21.15.7:0/1411233504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abbb92d250 con 0x55abbb91c640 2024-06-10T01:09:13.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.806+0000 7f69b9d9b700 1 --2- 172.21.15.7:0/1411233504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abbb91c640 0x55abbb92c630 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f69b000c5b0 tx=0x7f69b0097b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7cd77df25f9b99 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:13.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.806+0000 7f69a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:09:13.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.806+0000 7f69a1ffb700 1 -- 172.21.15.7:0/1411233504 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b001c040 con 0x55abbb91c640 2024-06-10T01:09:13.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.806+0000 7f69a1ffb700 1 -- 172.21.15.7:0/1411233504 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f69b000c6e0 con 0x55abbb91c640 2024-06-10T01:09:13.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.806+0000 7f69cb5c4b80 1 -- 172.21.15.7:0/1411233504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69ac043070 con 0x55abbb91c640 2024-06-10T01:09:13.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.806+0000 7f69a1ffb700 1 -- 172.21.15.7:0/1411233504 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f69b00a4430 con 0x55abbb91c640 2024-06-10T01:09:13.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.806+0000 7f69cb5c4b80 10 client.16347 fetch_fsmap learned FSMap version 30 2024-06-10T01:09:13.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.806+0000 7f69cb5c4b80 1 -- 172.21.15.7:0/1411233504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55abbb9a2cd0 con 0x55abbb91c640 2024-06-10T01:09:13.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.807+0000 7f69a1ffb700 1 -- 172.21.15.7:0/1411233504 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69ac043070 con 0x55abbb91c640 2024-06-10T01:09:13.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.807+0000 7f69a1ffb700 1 -- 172.21.15.7:0/1411233504 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f69b00afce0 con 0x55abbb91c640 2024-06-10T01:09:13.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.807+0000 7f69cb5c4b80 10 client.16347 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:09:13.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.807+0000 7f69cb5c4b80 10 client.16347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:13.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.807+0000 7f69cb5c4b80 20 client.16347 populate_metadata read hostname 'smithi007' 2024-06-10T01:09:13.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.807+0000 7f69cb5c4b80 1 --2- 172.21.15.7:0/1411233504 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55abbb9a3010 0x55abbb9a5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:13.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.807+0000 7f69cb5c4b80 4 client.16347 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:09:13.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.807+0000 7f69cb5c4b80 1 -- 172.21.15.7:0/1411233504 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55abbb870290 con 0x55abbb9a3010 2024-06-10T01:09:13.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.807+0000 7f69ba59c700 1 --2- 172.21.15.7:0/1411233504 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55abbb9a3010 0x55abbb9a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:13.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.807+0000 7f69a1ffb700 10 client.16347 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:13.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.807+0000 7f69ba59c700 1 --2- 172.21.15.7:0/1411233504 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55abbb9a3010 0x55abbb9a5400 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4690a1076836b1a6 server_cookie=36eeb423872ef868 in_seq=0 out_seq=0 2024-06-10T01:09:13.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.808+0000 7f69a1ffb700 1 -- 172.21.15.7:0/1411233504 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55abbb870290 con 0x55abbb9a3010 2024-06-10T01:09:13.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.808+0000 7f69a1ffb700 10 client.16347 handle_command_reply: tid=0 2024-06-10T01:09:13.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.808+0000 7f69cb5c4b80 1 client.16347 shutdown 2024-06-10T01:09:13.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.808+0000 7f69cb5c4b80 1 -- 172.21.15.7:0/1411233504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abbb91c640 msgr2=0x55abbb92c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:13.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.808+0000 7f69cb5c4b80 1 --2- 172.21.15.7:0/1411233504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abbb91c640 0x55abbb92c630 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f69b000c5b0 tx=0x7f69b0097b90 comp rx=0 tx=0).stop 2024-06-10T01:09:13.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.808+0000 7f69cb5c4b80 1 -- 172.21.15.7:0/1411233504 shutdown_connections 2024-06-10T01:09:13.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.808+0000 7f69cb5c4b80 1 --2- 172.21.15.7:0/1411233504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbb91cf60 0x55abbb92cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.808+0000 7f69cb5c4b80 1 --2- 172.21.15.7:0/1411233504 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55abbb9a3010 0x55abbb9a5400 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.808+0000 7f69cb5c4b80 1 --2- 172.21.15.7:0/1411233504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbb916f20 0x55abbb931c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.808+0000 7f69cb5c4b80 1 --2- 172.21.15.7:0/1411233504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abbb91c640 0x55abbb92c630 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.808+0000 7f69cb5c4b80 1 -- 172.21.15.7:0/1411233504 >> 172.21.15.7:0/1411233504 conn(0x55abbb841cc0 msgr2=0x55abbb91d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:13.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.808+0000 7f69a1ffb700 0 client.16347 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:13.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.808+0000 7f69cb5c4b80 1 -- 172.21.15.7:0/1411233504 shutdown_connections 2024-06-10T01:09:13.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.808+0000 7f69cb5c4b80 1 -- 172.21.15.7:0/1411233504 wait complete. 2024-06-10T01:09:13.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.809+0000 7f69cb5c4b80 20 client.16347 trim_cache size 0 max 16384 2024-06-10T01:09:13.809 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:09:13.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.809+0000 7f69c0d41700 1 -- 172.21.15.7:0/2333355660 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f69a40a2c20 msgr2=0x7f69a40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:13.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.809+0000 7f69c0d41700 1 --2- 172.21.15.7:0/2333355660 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f69a40a2c20 0x7f69a40a50e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009990 tx=0x7f69ac005960 comp rx=0 tx=0).stop 2024-06-10T01:09:13.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.809+0000 7f69c0d41700 1 -- 172.21.15.7:0/2333355660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69bc1017f0 msgr2=0x7f69bc19b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:13.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.809+0000 7f69c0d41700 1 --2- 172.21.15.7:0/2333355660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69bc1017f0 0x7f69bc19b4c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f69b000e9f0 tx=0x7f69b000ed00 comp rx=0 tx=0).stop 2024-06-10T01:09:13.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.809+0000 7f69c0d41700 1 -- 172.21.15.7:0/2333355660 shutdown_connections 2024-06-10T01:09:13.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.809+0000 7f69c0d41700 1 --2- 172.21.15.7:0/2333355660 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f69a40a2c20 0x7f69a40a50e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.809+0000 7f69c0d41700 1 --2- 172.21.15.7:0/2333355660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69bc103300 0x7f69bc19f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.809+0000 7f69c0d41700 1 --2- 172.21.15.7:0/2333355660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69bc100e50 0x7f69bc19af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.809+0000 7f69c0d41700 1 --2- 172.21.15.7:0/2333355660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69bc1017f0 0x7f69bc19b4c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:13.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.809+0000 7f69c0d41700 1 -- 172.21.15.7:0/2333355660 >> 172.21.15.7:0/2333355660 conn(0x7f69bc0766b0 msgr2=0x7f69bc0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:13.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.809+0000 7f69c0d41700 1 -- 172.21.15.7:0/2333355660 shutdown_connections 2024-06-10T01:09:13.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:13.809+0000 7f69c0d41700 1 -- 172.21.15.7:0/2333355660 wait complete. 2024-06-10T01:09:14.712 INFO:tasks.workunit.client.0.smithi007.stdout: 1 163798 7.83 MB/sec warmup 115 sec latency 12.398 ms 2024-06-10T01:09:15.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:15 smithi007 ceph-mon[32398]: pgmap v529: 129 pgs: 129 active+clean; 46 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.7 MiB/s wr, 310 op/s 2024-06-10T01:09:15.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:15 smithi078 ceph-mon[36683]: pgmap v529: 129 pgs: 129 active+clean; 46 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.7 MiB/s wr, 310 op/s 2024-06-10T01:09:15.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:15 smithi175 ceph-mon[36636]: pgmap v529: 129 pgs: 129 active+clean; 46 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.7 MiB/s wr, 310 op/s 2024-06-10T01:09:15.713 INFO:tasks.workunit.client.0.smithi007.stdout: 1 166725 7.91 MB/sec warmup 116 sec latency 10.812 ms 2024-06-10T01:09:15.825 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:09:15.826 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:09:16.121 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:09:16.713 INFO:tasks.workunit.client.0.smithi007.stdout: 1 167916 7.88 MB/sec warmup 117 sec latency 30.271 ms 2024-06-10T01:09:17.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:17 smithi007 ceph-mon[32398]: pgmap v530: 129 pgs: 129 active+clean; 48 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.0 MiB/s wr, 345 op/s 2024-06-10T01:09:17.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:17 smithi078 ceph-mon[36683]: pgmap v530: 129 pgs: 129 active+clean; 48 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.0 MiB/s wr, 345 op/s 2024-06-10T01:09:17.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:17 smithi175 ceph-mon[36636]: pgmap v530: 129 pgs: 129 active+clean; 48 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.0 MiB/s wr, 345 op/s 2024-06-10T01:09:17.713 INFO:tasks.workunit.client.0.smithi007.stdout: 1 170490 7.95 MB/sec warmup 118 sec latency 32.582 ms 2024-06-10T01:09:17.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.873+0000 7fa48ffe4700 1 -- 172.21.15.7:0/1872457927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa48806ab30 msgr2=0x7fa48810f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:17.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.873+0000 7fa48ffe4700 1 --2- 172.21.15.7:0/1872457927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa48806ab30 0x7fa48810f320 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa48000b0a0 tx=0x7fa48000b3b0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.873+0000 7fa48ffe4700 1 -- 172.21.15.7:0/1872457927 shutdown_connections 2024-06-10T01:09:17.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.873+0000 7fa48ffe4700 1 --2- 172.21.15.7:0/1872457927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa48806ab30 0x7fa48810f320 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.873+0000 7fa48ffe4700 1 --2- 172.21.15.7:0/1872457927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa488108500 0x7fa4881088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.873+0000 7fa48ffe4700 1 --2- 172.21.15.7:0/1872457927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa48806a170 0x7fa48806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.873+0000 7fa48ffe4700 1 -- 172.21.15.7:0/1872457927 >> 172.21.15.7:0/1872457927 conn(0x7fa488078340 msgr2=0x7fa488078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:17.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.873+0000 7fa48ffe4700 1 -- 172.21.15.7:0/1872457927 shutdown_connections 2024-06-10T01:09:17.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.873+0000 7fa48ffe4700 1 -- 172.21.15.7:0/1872457927 wait complete. 2024-06-10T01:09:17.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.874+0000 7fa48ffe4700 1 Processor -- start 2024-06-10T01:09:17.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.874+0000 7fa48ffe4700 1 -- start start 2024-06-10T01:09:17.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.874+0000 7fa48ffe4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa48806a170 0x7fa48819e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:17.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.874+0000 7fa48ffe4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa48806ab30 0x7fa48819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:17.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.874+0000 7fa48ffe4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa488108500 0x7fa4881989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:17.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.874+0000 7fa48ffe4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa488112760 con 0x7fa48806ab30 2024-06-10T01:09:17.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.874+0000 7fa48ffe4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa4881125e0 con 0x7fa48806a170 2024-06-10T01:09:17.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.874+0000 7fa48ffe4700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa4881128e0 con 0x7fa488108500 2024-06-10T01:09:17.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.874+0000 7fa48d57f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa48806ab30 0x7fa48819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:17.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.874+0000 7fa48dd80700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa48806a170 0x7fa48819e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:17.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.874+0000 7fa48d57f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa48806ab30 0x7fa48819ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43230/0 (socket says 172.21.15.7:43230) 2024-06-10T01:09:17.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.874+0000 7fa48d57f700 1 -- 172.21.15.7:0/3016376625 learned_addr learned my addr 172.21.15.7:0/3016376625 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:17.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.874+0000 7fa48e581700 1 --2- 172.21.15.7:0/3016376625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa488108500 0x7fa4881989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:17.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.874+0000 7fa48dd80700 1 -- 172.21.15.7:0/3016376625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa488108500 msgr2=0x7fa4881989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:17.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.874+0000 7fa48dd80700 1 --2- 172.21.15.7:0/3016376625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa488108500 0x7fa4881989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.874+0000 7fa48dd80700 1 -- 172.21.15.7:0/3016376625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa48806ab30 msgr2=0x7fa48819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.874+0000 7fa48dd80700 1 --2- 172.21.15.7:0/3016376625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa48806ab30 0x7fa48819ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.874+0000 7fa48dd80700 1 -- 172.21.15.7:0/3016376625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa488199120 con 0x7fa48806a170 2024-06-10T01:09:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.874+0000 7fa48d57f700 1 --2- 172.21.15.7:0/3016376625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa48806ab30 0x7fa48819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.875+0000 7fa48dd80700 1 --2- 172.21.15.7:0/3016376625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa48806a170 0x7fa48819e800 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa48400e9f0 tx=0x7fa48400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.875+0000 7fa47effd700 1 -- 172.21.15.7:0/3016376625 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48400c9c0 con 0x7fa48806a170 2024-06-10T01:09:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.875+0000 7fa48ffe4700 1 -- 172.21.15.7:0/3016376625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa488199410 con 0x7fa48806a170 2024-06-10T01:09:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.875+0000 7fa47effd700 1 -- 172.21.15.7:0/3016376625 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa48400cb60 con 0x7fa48806a170 2024-06-10T01:09:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.875+0000 7fa47effd700 1 -- 172.21.15.7:0/3016376625 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa484018680 con 0x7fa48806a170 2024-06-10T01:09:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.875+0000 7fa48ffe4700 1 -- 172.21.15.7:0/3016376625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4881a56d0 con 0x7fa48806a170 2024-06-10T01:09:17.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.877+0000 7fa47effd700 1 -- 172.21.15.7:0/3016376625 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa484010430 con 0x7fa48806a170 2024-06-10T01:09:17.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.878+0000 7fa47effd700 1 --2- 172.21.15.7:0/3016376625 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa4740a2ca0 0x7fa4740a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:17.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.878+0000 7fa48d57f700 1 --2- 172.21.15.7:0/3016376625 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa4740a2ca0 0x7fa4740a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:17.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.878+0000 7fa47effd700 1 -- 172.21.15.7:0/3016376625 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa4840cf410 con 0x7fa48806a170 2024-06-10T01:09:17.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.879+0000 7fa49a867b80 1 Processor -- start 2024-06-10T01:09:17.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.879+0000 7fa48d57f700 1 --2- 172.21.15.7:0/3016376625 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa4740a2ca0 0x7fa4740a5160 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa488078940 tx=0x7fa478009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:17.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.879+0000 7fa49a867b80 1 -- start start 2024-06-10T01:09:17.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.879+0000 7fa49a867b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa0cbf1af0 0x55fa0cbf1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:17.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.879+0000 7fa49a867b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa0cbf2410 0x55fa0cbebe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:17.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.879+0000 7fa49a867b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa0cbec3b0 0x55fa0cbec7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:17.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.879+0000 7fa49a867b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fa0cb39890 con 0x55fa0cbec3b0 2024-06-10T01:09:17.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.879+0000 7fa49a867b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fa0cae17a0 con 0x55fa0cbf1af0 2024-06-10T01:09:17.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.879+0000 7fa49a867b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fa0cb35850 con 0x55fa0cbf2410 2024-06-10T01:09:17.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.879+0000 7fa48d57f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa0cbf2410 0x55fa0cbebe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:17.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.879+0000 7fa48dd80700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa0cbf1af0 0x55fa0cbf1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:17.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.879+0000 7fa48d57f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa0cbf2410 0x55fa0cbebe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43118/0 (socket says 172.21.15.7:43118) 2024-06-10T01:09:17.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.879+0000 7fa48d57f700 1 -- 172.21.15.7:0/267300654 learned_addr learned my addr 172.21.15.7:0/267300654 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:17.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.879+0000 7fa48e581700 1 --2- 172.21.15.7:0/267300654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa0cbec3b0 0x55fa0cbec7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:17.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.880+0000 7fa48d57f700 1 -- 172.21.15.7:0/267300654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa0cbf1af0 msgr2=0x55fa0cbf1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:17.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.880+0000 7fa48d57f700 1 --2- 172.21.15.7:0/267300654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa0cbf1af0 0x55fa0cbf1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.880+0000 7fa48d57f700 1 -- 172.21.15.7:0/267300654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa0cbec3b0 msgr2=0x55fa0cbec7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:17.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.880+0000 7fa48d57f700 1 --2- 172.21.15.7:0/267300654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa0cbec3b0 0x55fa0cbec7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.880+0000 7fa48d57f700 1 -- 172.21.15.7:0/267300654 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa0cad7fd0 con 0x55fa0cbf2410 2024-06-10T01:09:17.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.880+0000 7fa48dd80700 1 --2- 172.21.15.7:0/267300654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa0cbf1af0 0x55fa0cbf1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:17.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.880+0000 7fa48d57f700 1 --2- 172.21.15.7:0/267300654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa0cbf2410 0x55fa0cbebe70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa47800e6d0 tx=0x7fa478014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bac275e17e8b4471 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:17.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.880+0000 7fa473fff700 1 -- 172.21.15.7:0/267300654 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa478026630 con 0x55fa0cbf2410 2024-06-10T01:09:17.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.880+0000 7fa473fff700 1 -- 172.21.15.7:0/267300654 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa47801d8f0 con 0x55fa0cbf2410 2024-06-10T01:09:17.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.880+0000 7fa473fff700 1 -- 172.21.15.7:0/267300654 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4780258b0 con 0x55fa0cbf2410 2024-06-10T01:09:17.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.881+0000 7fa49a867b80 1 -- 172.21.15.7:0/267300654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa0cbf2410 msgr2=0x55fa0cbebe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:17.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.881+0000 7fa49a867b80 1 --2- 172.21.15.7:0/267300654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa0cbf2410 0x55fa0cbebe70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa47800e6d0 tx=0x7fa478014000 comp rx=0 tx=0).stop 2024-06-10T01:09:17.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.881+0000 7fa49a867b80 1 -- 172.21.15.7:0/267300654 shutdown_connections 2024-06-10T01:09:17.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.881+0000 7fa49a867b80 1 --2- 172.21.15.7:0/267300654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa0cbec3b0 0x55fa0cbec7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.881+0000 7fa49a867b80 1 --2- 172.21.15.7:0/267300654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa0cbf1af0 0x55fa0cbf1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.881+0000 7fa49a867b80 1 --2- 172.21.15.7:0/267300654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa0cbf2410 0x55fa0cbebe70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.881+0000 7fa49a867b80 1 -- 172.21.15.7:0/267300654 >> 172.21.15.7:0/267300654 conn(0x55fa0cb12d40 msgr2=0x55fa0cbe28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:17.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.881+0000 7fa49a867b80 1 -- 172.21.15.7:0/267300654 shutdown_connections 2024-06-10T01:09:17.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.881+0000 7fa49a867b80 1 -- 172.21.15.7:0/267300654 wait complete. 2024-06-10T01:09:17.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.882+0000 7fa49a867b80 1 -- start start 2024-06-10T01:09:17.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.882+0000 7fa49a867b80 1 Processor -- start 2024-06-10T01:09:17.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.882+0000 7fa49a867b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa0cbec3b0 0x55fa0cbf81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:17.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.882+0000 7fa49a867b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa0cbf1af0 0x55fa0cbf54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:17.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.882+0000 7fa49a867b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa0cbf2410 0x55fa0cbf5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:17.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.882+0000 7fa49a867b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fa0cbf8e50 con 0x55fa0cbec3b0 2024-06-10T01:09:17.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.882+0000 7fa49a867b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fa0cad9c70 con 0x55fa0cbf1af0 2024-06-10T01:09:17.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.882+0000 7fa49a867b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fa0cbf8fd0 con 0x55fa0cbf2410 2024-06-10T01:09:17.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.882+0000 7fa49a867b80 10 client.0 _ll_register_callbacks cb 0x55fa0cad39a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:17.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.882+0000 7fa48d57f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa0cbf1af0 0x55fa0cbf54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:17.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.882+0000 7fa48d57f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa0cbf1af0 0x55fa0cbf54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:32854/0 (socket says 172.21.15.7:32854) 2024-06-10T01:09:17.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.882+0000 7fa48dd80700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa0cbec3b0 0x55fa0cbf81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:17.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.882+0000 7fa48d57f700 1 -- 172.21.15.7:0/2218367307 learned_addr learned my addr 172.21.15.7:0/2218367307 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:17.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.882+0000 7fa48e581700 1 --2- 172.21.15.7:0/2218367307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa0cbf2410 0x55fa0cbf5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:17.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.883+0000 7fa48dd80700 1 -- 172.21.15.7:0/2218367307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa0cbf2410 msgr2=0x55fa0cbf5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:17.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.883+0000 7fa48dd80700 1 --2- 172.21.15.7:0/2218367307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa0cbf2410 0x55fa0cbf5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.883+0000 7fa48dd80700 1 -- 172.21.15.7:0/2218367307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa0cbf1af0 msgr2=0x55fa0cbf54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:17.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.883+0000 7fa48dd80700 1 --2- 172.21.15.7:0/2218367307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa0cbf1af0 0x55fa0cbf54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.883+0000 7fa48dd80700 1 -- 172.21.15.7:0/2218367307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa0ca88950 con 0x55fa0cbec3b0 2024-06-10T01:09:17.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.883+0000 7fa48d57f700 1 --2- 172.21.15.7:0/2218367307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa0cbf1af0 0x55fa0cbf54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:17.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.883+0000 7fa48dd80700 1 --2- 172.21.15.7:0/2218367307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa0cbec3b0 0x55fa0cbf81d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fa48409b590 tx=0x7fa484086390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa6d12ec3454fc5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:17.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.883+0000 7fa4717fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:09:17.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.883+0000 7fa4717fa700 1 -- 172.21.15.7:0/2218367307 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48409fc40 con 0x55fa0cbec3b0 2024-06-10T01:09:17.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.883+0000 7fa49a867b80 1 -- 172.21.15.7:0/2218367307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa47801d8f0 con 0x55fa0cbec3b0 2024-06-10T01:09:17.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.883+0000 7fa4717fa700 1 -- 172.21.15.7:0/2218367307 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa4840a4070 con 0x55fa0cbec3b0 2024-06-10T01:09:17.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.883+0000 7fa4717fa700 1 -- 172.21.15.7:0/2218367307 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa4840a4500 con 0x55fa0cbec3b0 2024-06-10T01:09:17.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.883+0000 7fa49a867b80 10 client.16359 fetch_fsmap learned FSMap version 30 2024-06-10T01:09:17.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.883+0000 7fa49a867b80 1 -- 172.21.15.7:0/2218367307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fa0cc037f0 con 0x55fa0cbec3b0 2024-06-10T01:09:17.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.884+0000 7fa4717fa700 1 -- 172.21.15.7:0/2218367307 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa47801d8f0 con 0x55fa0cbec3b0 2024-06-10T01:09:17.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.884+0000 7fa4717fa700 1 -- 172.21.15.7:0/2218367307 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fa484010720 con 0x55fa0cbec3b0 2024-06-10T01:09:17.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.884+0000 7fa49a867b80 10 client.16359 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:09:17.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.884+0000 7fa49a867b80 10 client.16359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:17.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.884+0000 7fa49a867b80 20 client.16359 populate_metadata read hostname 'smithi007' 2024-06-10T01:09:17.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.884+0000 7fa49a867b80 1 --2- 172.21.15.7:0/2218367307 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fa0cc05d80 0x55fa0cc08190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:17.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.884+0000 7fa49a867b80 4 client.16359 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:09:17.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.884+0000 7fa49a867b80 1 -- 172.21.15.7:0/2218367307 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa0cb46d90 con 0x55fa0cc05d80 2024-06-10T01:09:17.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.884+0000 7fa48d57f700 1 --2- 172.21.15.7:0/2218367307 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fa0cc05d80 0x55fa0cc08190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:17.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.885+0000 7fa4717fa700 10 client.16359 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:17.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.885+0000 7fa48d57f700 1 --2- 172.21.15.7:0/2218367307 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fa0cc05d80 0x55fa0cc08190 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=697146ab71eaeb2 server_cookie=d0f893791b54f2d1 in_seq=0 out_seq=0 2024-06-10T01:09:17.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.885+0000 7fa4717fa700 1 -- 172.21.15.7:0/2218367307 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fa0cb46d90 con 0x55fa0cc05d80 2024-06-10T01:09:17.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.885+0000 7fa4717fa700 10 client.16359 handle_command_reply: tid=0 2024-06-10T01:09:17.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.885+0000 7fa49a867b80 1 client.16359 shutdown 2024-06-10T01:09:17.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.885+0000 7fa49a867b80 1 -- 172.21.15.7:0/2218367307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa0cbec3b0 msgr2=0x55fa0cbf81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:17.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.885+0000 7fa49a867b80 1 --2- 172.21.15.7:0/2218367307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa0cbec3b0 0x55fa0cbf81d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fa48409b590 tx=0x7fa484086390 comp rx=0 tx=0).stop 2024-06-10T01:09:17.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.886+0000 7fa49a867b80 1 -- 172.21.15.7:0/2218367307 shutdown_connections 2024-06-10T01:09:17.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.886+0000 7fa49a867b80 1 --2- 172.21.15.7:0/2218367307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa0cbf2410 0x55fa0cbf5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.886+0000 7fa49a867b80 1 --2- 172.21.15.7:0/2218367307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa0cbec3b0 0x55fa0cbf81d0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.886+0000 7fa49a867b80 1 --2- 172.21.15.7:0/2218367307 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fa0cc05d80 0x55fa0cc08190 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.886+0000 7fa49a867b80 1 --2- 172.21.15.7:0/2218367307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa0cbf1af0 0x55fa0cbf54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.886+0000 7fa4717fa700 0 client.16359 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:17.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.886+0000 7fa49a867b80 1 -- 172.21.15.7:0/2218367307 >> 172.21.15.7:0/2218367307 conn(0x55fa0cb12d40 msgr2=0x55fa0cbe6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:17.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.886+0000 7fa49a867b80 1 -- 172.21.15.7:0/2218367307 shutdown_connections 2024-06-10T01:09:17.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.886+0000 7fa49a867b80 1 -- 172.21.15.7:0/2218367307 wait complete. 2024-06-10T01:09:17.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.886+0000 7fa49a867b80 20 client.16359 trim_cache size 0 max 16384 2024-06-10T01:09:17.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.910+0000 7fa49a867b80 1 Processor -- start 2024-06-10T01:09:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.910+0000 7fa49a867b80 1 -- start start 2024-06-10T01:09:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.910+0000 7fa49a867b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa0cb12d40 0x55fa0cbdc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.910+0000 7fa49a867b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa0cbec3b0 0x55fa0cc0daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.910+0000 7fa49a867b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa0cbf1af0 0x55fa0cc0dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.910+0000 7fa49a867b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa4881aea70 con 0x55fa0cbec3b0 2024-06-10T01:09:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.910+0000 7fa49a867b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa4881ac2a0 con 0x55fa0cb12d40 2024-06-10T01:09:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.910+0000 7fa49a867b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa4881aa0f0 con 0x55fa0cbf1af0 2024-06-10T01:09:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.910+0000 7fa48d57f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa0cbec3b0 0x55fa0cc0daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.910+0000 7fa48dd80700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa0cb12d40 0x55fa0cbdc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.910+0000 7fa48dd80700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa0cb12d40 0x55fa0cbdc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:32864/0 (socket says 172.21.15.7:32864) 2024-06-10T01:09:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.910+0000 7fa48d57f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa0cbec3b0 0x55fa0cc0daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43272/0 (socket says 172.21.15.7:43272) 2024-06-10T01:09:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.910+0000 7fa48dd80700 1 -- 172.21.15.7:0/3190832623 learned_addr learned my addr 172.21.15.7:0/3190832623 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.910+0000 7fa48e581700 1 --2- 172.21.15.7:0/3190832623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa0cbf1af0 0x55fa0cc0dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.910+0000 7fa48dd80700 1 -- 172.21.15.7:0/3190832623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa0cbf1af0 msgr2=0x55fa0cc0dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.910+0000 7fa48dd80700 1 --2- 172.21.15.7:0/3190832623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa0cbf1af0 0x55fa0cc0dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.910+0000 7fa48dd80700 1 -- 172.21.15.7:0/3190832623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa0cbec3b0 msgr2=0x55fa0cc0daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.910+0000 7fa48dd80700 1 --2- 172.21.15.7:0/3190832623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa0cbec3b0 0x55fa0cc0daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.911+0000 7fa48dd80700 1 -- 172.21.15.7:0/3190832623 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa0cc08810 con 0x55fa0cb12d40 2024-06-10T01:09:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.911+0000 7fa48d57f700 1 --2- 172.21.15.7:0/3190832623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa0cbec3b0 0x55fa0cc0daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.911+0000 7fa48dd80700 1 --2- 172.21.15.7:0/3190832623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa0cb12d40 0x55fa0cbdc590 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa484000c00 tx=0x7fa484099da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dce4b932e4eb35d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.911+0000 7fa4737fe700 1 -- 172.21.15.7:0/3190832623 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4840ba2a0 con 0x55fa0cb12d40 2024-06-10T01:09:17.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.911+0000 7fa4737fe700 1 -- 172.21.15.7:0/3190832623 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa484010720 con 0x55fa0cb12d40 2024-06-10T01:09:17.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.911+0000 7fa4737fe700 1 -- 172.21.15.7:0/3190832623 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48409cbc0 con 0x55fa0cb12d40 2024-06-10T01:09:17.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.912+0000 7fa49a867b80 1 -- 172.21.15.7:0/3190832623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa0cb12d40 msgr2=0x55fa0cbdc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:17.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.912+0000 7fa49a867b80 1 --2- 172.21.15.7:0/3190832623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa0cb12d40 0x55fa0cbdc590 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa484000c00 tx=0x7fa484099da0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.912+0000 7fa49a867b80 1 -- 172.21.15.7:0/3190832623 shutdown_connections 2024-06-10T01:09:17.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.912+0000 7fa49a867b80 1 --2- 172.21.15.7:0/3190832623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa0cbf1af0 0x55fa0cc0dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.912+0000 7fa49a867b80 1 --2- 172.21.15.7:0/3190832623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa0cb12d40 0x55fa0cbdc590 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.912+0000 7fa49a867b80 1 --2- 172.21.15.7:0/3190832623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa0cbec3b0 0x55fa0cc0daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.912+0000 7fa49a867b80 1 -- 172.21.15.7:0/3190832623 >> 172.21.15.7:0/3190832623 conn(0x55fa0cbecaf0 msgr2=0x55fa0cbdbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:17.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.912+0000 7fa49a867b80 1 -- 172.21.15.7:0/3190832623 shutdown_connections 2024-06-10T01:09:17.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.912+0000 7fa49a867b80 1 -- 172.21.15.7:0/3190832623 wait complete. 2024-06-10T01:09:17.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.912+0000 7fa49a867b80 1 -- start start 2024-06-10T01:09:17.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.912+0000 7fa49a867b80 1 Processor -- start 2024-06-10T01:09:17.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.913+0000 7fa49a867b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa0cb12d40 0x55fa0cbfe880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:17.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.913+0000 7fa49a867b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa0cbec3b0 0x55fa0cbfb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:17.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.913+0000 7fa49a867b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa0cbf1af0 0x55fa0cbfb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:17.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.913+0000 7fa49a867b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa4881aa760 con 0x55fa0cbf1af0 2024-06-10T01:09:17.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.913+0000 7fa49a867b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa4881abbd0 con 0x55fa0cb12d40 2024-06-10T01:09:17.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.913+0000 7fa49a867b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa4881aae30 con 0x55fa0cbec3b0 2024-06-10T01:09:17.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.913+0000 7fa48dd80700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa0cb12d40 0x55fa0cbfe880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:17.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.913+0000 7fa49a867b80 10 client.0 _ll_register_callbacks cb 0x55fa0cad39a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:17.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.913+0000 7fa48dd80700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa0cb12d40 0x55fa0cbfe880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:32876/0 (socket says 172.21.15.7:32876) 2024-06-10T01:09:17.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.913+0000 7fa48dd80700 1 -- 172.21.15.7:0/3736398293 learned_addr learned my addr 172.21.15.7:0/3736398293 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:17.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.913+0000 7fa48d57f700 1 --2- 172.21.15.7:0/3736398293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa0cbec3b0 0x55fa0cbfb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:17.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.913+0000 7fa48e581700 1 --2- 172.21.15.7:0/3736398293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa0cbf1af0 0x55fa0cbfb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:17.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.913+0000 7fa48dd80700 1 -- 172.21.15.7:0/3736398293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa0cbec3b0 msgr2=0x55fa0cbfb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:17.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.913+0000 7fa48dd80700 1 --2- 172.21.15.7:0/3736398293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa0cbec3b0 0x55fa0cbfb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.913+0000 7fa48dd80700 1 -- 172.21.15.7:0/3736398293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa0cbf1af0 msgr2=0x55fa0cbfb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:17.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.913+0000 7fa48dd80700 1 --2- 172.21.15.7:0/3736398293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa0cbf1af0 0x55fa0cbfb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.913+0000 7fa48dd80700 1 -- 172.21.15.7:0/3736398293 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa0cbfbe10 con 0x55fa0cb12d40 2024-06-10T01:09:17.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.913+0000 7fa48e581700 1 --2- 172.21.15.7:0/3736398293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa0cbf1af0 0x55fa0cbfb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:09:17.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.914+0000 7fa48dd80700 1 --2- 172.21.15.7:0/3736398293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa0cb12d40 0x55fa0cbfe880 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa48409fe00 tx=0x7fa4840a4300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3af721050155da3a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:17.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.914+0000 7fa4717fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:09:17.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.914+0000 7fa4717fa700 1 -- 172.21.15.7:0/3736398293 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4840ba2a0 con 0x55fa0cb12d40 2024-06-10T01:09:17.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.914+0000 7fa4717fa700 1 -- 172.21.15.7:0/3736398293 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa4840cf0f0 con 0x55fa0cb12d40 2024-06-10T01:09:17.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.914+0000 7fa49a867b80 1 -- 172.21.15.7:0/3736398293 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa484010720 con 0x55fa0cb12d40 2024-06-10T01:09:17.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.914+0000 7fa4717fa700 1 -- 172.21.15.7:0/3736398293 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa4840108d0 con 0x55fa0cb12d40 2024-06-10T01:09:17.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.914+0000 7fa49a867b80 10 client.25889 fetch_fsmap learned FSMap version 30 2024-06-10T01:09:17.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.914+0000 7fa49a867b80 1 -- 172.21.15.7:0/3736398293 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fa0cbfbe10 con 0x55fa0cb12d40 2024-06-10T01:09:17.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.914+0000 7fa4717fa700 1 -- 172.21.15.7:0/3736398293 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa484097070 con 0x55fa0cb12d40 2024-06-10T01:09:17.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.914+0000 7fa4717fa700 1 -- 172.21.15.7:0/3736398293 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fa48409a800 con 0x55fa0cb12d40 2024-06-10T01:09:17.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.914+0000 7fa49a867b80 10 client.25889 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:09:17.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.914+0000 7fa49a867b80 10 client.25889 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:17.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.914+0000 7fa49a867b80 20 client.25889 populate_metadata read hostname 'smithi007' 2024-06-10T01:09:17.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.914+0000 7fa49a867b80 1 --2- 172.21.15.7:0/3736398293 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fa0cbf2410 0x55fa0cc73bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.914+0000 7fa49a867b80 4 client.25889 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:09:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.914+0000 7fa49a867b80 1 -- 172.21.15.7:0/3736398293 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fa484010720 con 0x55fa0cbf2410 2024-06-10T01:09:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.915+0000 7fa48d57f700 1 --2- 172.21.15.7:0/3736398293 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fa0cbf2410 0x55fa0cc73bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.915+0000 7fa4717fa700 10 client.25889 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.915+0000 7fa48d57f700 1 --2- 172.21.15.7:0/3736398293 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fa0cbf2410 0x55fa0cc73bd0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e5503f6259b5e0e server_cookie=72c5f9fd7c28b9c0 in_seq=0 out_seq=0 2024-06-10T01:09:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.916+0000 7fa4717fa700 1 -- 172.21.15.7:0/3736398293 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fa484010720 con 0x55fa0cbf2410 2024-06-10T01:09:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.916+0000 7fa4717fa700 10 client.25889 handle_command_reply: tid=0 2024-06-10T01:09:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.916+0000 7fa49a867b80 1 client.25889 shutdown 2024-06-10T01:09:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.916+0000 7fa49a867b80 1 -- 172.21.15.7:0/3736398293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa0cb12d40 msgr2=0x55fa0cbfe880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.916+0000 7fa49a867b80 1 --2- 172.21.15.7:0/3736398293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa0cb12d40 0x55fa0cbfe880 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa48409fe00 tx=0x7fa4840a4300 comp rx=0 tx=0).stop 2024-06-10T01:09:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.917+0000 7fa49a867b80 1 -- 172.21.15.7:0/3736398293 shutdown_connections 2024-06-10T01:09:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.917+0000 7fa49a867b80 1 --2- 172.21.15.7:0/3736398293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa0cbf1af0 0x55fa0cbfb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.917+0000 7fa49a867b80 1 --2- 172.21.15.7:0/3736398293 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fa0cbf2410 0x55fa0cc73bd0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.917+0000 7fa49a867b80 1 --2- 172.21.15.7:0/3736398293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa0cb12d40 0x55fa0cbfe880 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.917+0000 7fa49a867b80 1 --2- 172.21.15.7:0/3736398293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa0cbec3b0 0x55fa0cbfb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.917+0000 7fa49a867b80 1 -- 172.21.15.7:0/3736398293 >> 172.21.15.7:0/3736398293 conn(0x55fa0cbecaf0 msgr2=0x55fa0cbf3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:17.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.917+0000 7fa4717fa700 0 client.25889 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:17.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.917+0000 7fa49a867b80 1 -- 172.21.15.7:0/3736398293 shutdown_connections 2024-06-10T01:09:17.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.917+0000 7fa49a867b80 1 -- 172.21.15.7:0/3736398293 wait complete. 2024-06-10T01:09:17.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.917+0000 7fa49a867b80 20 client.25889 trim_cache size 0 max 16384 2024-06-10T01:09:17.917 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:09:17.917 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-10T01:09:17.917 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "76fc0745-2b6a-4b68-a0c2-9e866ba0b0da", 2024-06-10T01:09:17.917 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-10T01:09:17.918 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:09:17.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.917+0000 7fa48ffe4700 1 -- 172.21.15.7:0/3016376625 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa4740a2ca0 msgr2=0x7fa4740a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:17.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.917+0000 7fa48ffe4700 1 --2- 172.21.15.7:0/3016376625 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa4740a2ca0 0x7fa4740a5160 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa488078940 tx=0x7fa478009260 comp rx=0 tx=0).stop 2024-06-10T01:09:17.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.917+0000 7fa48ffe4700 1 -- 172.21.15.7:0/3016376625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa48806a170 msgr2=0x7fa48819e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:17.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.917+0000 7fa48ffe4700 1 --2- 172.21.15.7:0/3016376625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa48806a170 0x7fa48819e800 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa48400e9f0 tx=0x7fa48400ed00 comp rx=0 tx=0).stop 2024-06-10T01:09:17.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.917+0000 7fa48ffe4700 1 -- 172.21.15.7:0/3016376625 shutdown_connections 2024-06-10T01:09:17.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.917+0000 7fa48ffe4700 1 --2- 172.21.15.7:0/3016376625 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa4740a2ca0 0x7fa4740a5160 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.917+0000 7fa48ffe4700 1 --2- 172.21.15.7:0/3016376625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa488108500 0x7fa4881989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.917+0000 7fa48ffe4700 1 --2- 172.21.15.7:0/3016376625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa48806a170 0x7fa48819e800 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.917+0000 7fa48ffe4700 1 --2- 172.21.15.7:0/3016376625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa48806ab30 0x7fa48819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:17.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.917+0000 7fa48ffe4700 1 -- 172.21.15.7:0/3016376625 >> 172.21.15.7:0/3016376625 conn(0x7fa488078340 msgr2=0x7fa4880ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:17.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.918+0000 7fa48ffe4700 1 -- 172.21.15.7:0/3016376625 shutdown_connections 2024-06-10T01:09:17.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:17.918+0000 7fa48ffe4700 1 -- 172.21.15.7:0/3016376625 wait complete. 2024-06-10T01:09:18.713 INFO:tasks.workunit.client.0.smithi007.stdout: 1 171965 7.93 MB/sec warmup 119 sec latency 14.505 ms 2024-06-10T01:09:18.745 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:09:19.012 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:09:19.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:19 smithi078 ceph-mon[36683]: pgmap v531: 129 pgs: 129 active+clean; 49 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.6 MiB/s wr, 292 op/s 2024-06-10T01:09:19.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:19 smithi078 ceph-mon[36683]: scrub queued for path: / 2024-06-10T01:09:19.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:19 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:09:19.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:19 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:09:19.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:19 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:09:19.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:19 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:09:19.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:19 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:09:19.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:19 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:09:19.331 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:19 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:09:19.363 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:19 smithi007 ceph-mon[32398]: pgmap v531: 129 pgs: 129 active+clean; 49 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.6 MiB/s wr, 292 op/s 2024-06-10T01:09:19.364 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:19 smithi007 ceph-mon[32398]: scrub queued for path: / 2024-06-10T01:09:19.364 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:19 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:09:19.364 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:19 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:09:19.364 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:19 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:09:19.364 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:19 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:09:19.364 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:19 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:09:19.364 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:19 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:09:19.364 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:19 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:09:19.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:19 smithi175 ceph-mon[36636]: pgmap v531: 129 pgs: 129 active+clean; 49 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.6 MiB/s wr, 292 op/s 2024-06-10T01:09:19.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:19 smithi175 ceph-mon[36636]: scrub queued for path: / 2024-06-10T01:09:19.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:19 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:09:19.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:19 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:09:19.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:19 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:09:19.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:19 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:09:19.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:19 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:09:19.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:19 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:09:19.454 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:19 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:09:20.713 INFO:tasks.workunit.client.0.smithi007.stdout: 1 176572 11.01 MB/sec execute 1 sec latency 27.559 ms 2024-06-10T01:09:21.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:21 smithi007 ceph-mon[32398]: pgmap v532: 129 pgs: 129 active+clean; 53 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 376 op/s 2024-06-10T01:09:21.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:21 smithi078 ceph-mon[36683]: pgmap v532: 129 pgs: 129 active+clean; 53 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 376 op/s 2024-06-10T01:09:21.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:21 smithi175 ceph-mon[36636]: pgmap v532: 129 pgs: 129 active+clean; 53 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 376 op/s 2024-06-10T01:09:21.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.690+0000 7fb01d730700 1 -- 172.21.15.7:0/2186381986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb018103120 msgr2=0x7fb018107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:22.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.690+0000 7fb01d730700 1 --2- 172.21.15.7:0/2186381986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb018103120 0x7fb018107130 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0099c0 tx=0x7fb00c009cd0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.690+0000 7fb01d730700 1 -- 172.21.15.7:0/2186381986 shutdown_connections 2024-06-10T01:09:22.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.690+0000 7fb01d730700 1 --2- 172.21.15.7:0/2186381986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb018107860 0x7fb01810f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.690+0000 7fb01d730700 1 --2- 172.21.15.7:0/2186381986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb018102770 0x7fb018102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.690+0000 7fb01d730700 1 --2- 172.21.15.7:0/2186381986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb018103120 0x7fb018107130 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.690+0000 7fb01d730700 1 -- 172.21.15.7:0/2186381986 >> 172.21.15.7:0/2186381986 conn(0x7fb018079600 msgr2=0x7fb018079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:22.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.690+0000 7fb01d730700 1 -- 172.21.15.7:0/2186381986 shutdown_connections 2024-06-10T01:09:22.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.690+0000 7fb01d730700 1 -- 172.21.15.7:0/2186381986 wait complete. 2024-06-10T01:09:22.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.691+0000 7fb01d730700 1 Processor -- start 2024-06-10T01:09:22.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.691+0000 7fb01d730700 1 -- start start 2024-06-10T01:09:22.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.691+0000 7fb01d730700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb018102770 0x7fb01810e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.691+0000 7fb01d730700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb018103120 0x7fb01810eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.691+0000 7fb01d730700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb018107860 0x7fb0181a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.691+0000 7fb01d730700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb018112770 con 0x7fb018102770 2024-06-10T01:09:22.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.691+0000 7fb01d730700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb0181125f0 con 0x7fb018103120 2024-06-10T01:09:22.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.691+0000 7fb01d730700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb0181128f0 con 0x7fb018107860 2024-06-10T01:09:22.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.691+0000 7fb016ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb018102770 0x7fb01810e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.691+0000 7fb0177fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb018107860 0x7fb0181a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.691+0000 7fb016ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb018102770 0x7fb01810e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43308/0 (socket says 172.21.15.7:43308) 2024-06-10T01:09:22.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.691+0000 7fb016ffd700 1 -- 172.21.15.7:0/3042755423 learned_addr learned my addr 172.21.15.7:0/3042755423 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:22.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.691+0000 7fb0167fc700 1 --2- 172.21.15.7:0/3042755423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb018103120 0x7fb01810eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.692+0000 7fb0177fe700 1 -- 172.21.15.7:0/3042755423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb018103120 msgr2=0x7fb01810eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:22.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.692+0000 7fb0177fe700 1 --2- 172.21.15.7:0/3042755423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb018103120 0x7fb01810eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.692+0000 7fb0177fe700 1 -- 172.21.15.7:0/3042755423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb018102770 msgr2=0x7fb01810e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:22.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.692+0000 7fb0177fe700 1 --2- 172.21.15.7:0/3042755423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb018102770 0x7fb01810e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.692+0000 7fb0177fe700 1 -- 172.21.15.7:0/3042755423 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0181a4190 con 0x7fb018107860 2024-06-10T01:09:22.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.692+0000 7fb0177fe700 1 --2- 172.21.15.7:0/3042755423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb018107860 0x7fb0181a3a50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb01000efc0 tx=0x7fb01000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:22.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.692+0000 7faffffff700 1 -- 172.21.15.7:0/3042755423 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0100092e0 con 0x7fb018107860 2024-06-10T01:09:22.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.692+0000 7fb01d730700 1 -- 172.21.15.7:0/3042755423 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0181a4420 con 0x7fb018107860 2024-06-10T01:09:22.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.692+0000 7faffffff700 1 -- 172.21.15.7:0/3042755423 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb010004bb0 con 0x7fb018107860 2024-06-10T01:09:22.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.692+0000 7faffffff700 1 -- 172.21.15.7:0/3042755423 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb010004280 con 0x7fb018107860 2024-06-10T01:09:22.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.692+0000 7fb01d730700 1 -- 172.21.15.7:0/3042755423 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0181abd00 con 0x7fb018107860 2024-06-10T01:09:22.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.695+0000 7faffffff700 1 -- 172.21.15.7:0/3042755423 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb01001b030 con 0x7fb018107860 2024-06-10T01:09:22.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.696+0000 7faffffff700 1 --2- 172.21.15.7:0/3042755423 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb0000a2c20 0x7fb0000a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.696+0000 7fb016ffd700 1 --2- 172.21.15.7:0/3042755423 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb0000a2c20 0x7fb0000a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.696+0000 7faffffff700 1 -- 172.21.15.7:0/3042755423 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb0100cef80 con 0x7fb018107860 2024-06-10T01:09:22.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.696+0000 7fb016ffd700 1 --2- 172.21.15.7:0/3042755423 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb0000a2c20 0x7fb0000a50e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb008009d20 tx=0x7fb008009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:22.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.696+0000 7fb027fb3b80 1 Processor -- start 2024-06-10T01:09:22.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.696+0000 7fb027fb3b80 1 -- start start 2024-06-10T01:09:22.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.697+0000 7fb027fb3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dca256640 0x558dca256a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.697+0000 7fb027fb3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558dca256f60 0x558dca2509e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.697+0000 7fb027fb3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558dca250f20 0x558dca251340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.697+0000 7fb027fb3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558dca1a2810 con 0x558dca256f60 2024-06-10T01:09:22.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.697+0000 7fb027fb3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558dca14a720 con 0x558dca256640 2024-06-10T01:09:22.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.697+0000 7fb027fb3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558dca19e7d0 con 0x558dca250f20 2024-06-10T01:09:22.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.697+0000 7fb0167fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558dca256f60 0x558dca2509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.697+0000 7fb0167fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558dca256f60 0x558dca2509e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43318/0 (socket says 172.21.15.7:43318) 2024-06-10T01:09:22.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.697+0000 7fb0167fc700 1 -- 172.21.15.7:0/1524079752 learned_addr learned my addr 172.21.15.7:0/1524079752 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:22.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.697+0000 7fb0177fe700 1 --2- 172.21.15.7:0/1524079752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558dca250f20 0x558dca251340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.697+0000 7fb016ffd700 1 --2- 172.21.15.7:0/1524079752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dca256640 0x558dca256a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.697+0000 7fb0167fc700 1 -- 172.21.15.7:0/1524079752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558dca250f20 msgr2=0x558dca251340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:22.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.697+0000 7fb0167fc700 1 --2- 172.21.15.7:0/1524079752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558dca250f20 0x558dca251340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.697+0000 7fb0167fc700 1 -- 172.21.15.7:0/1524079752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dca256640 msgr2=0x558dca256a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:22.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.697+0000 7fb0167fc700 1 --2- 172.21.15.7:0/1524079752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dca256640 0x558dca256a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.697+0000 7fb0167fc700 1 -- 172.21.15.7:0/1524079752 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dca140f50 con 0x558dca256f60 2024-06-10T01:09:22.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.697+0000 7fb0167fc700 1 --2- 172.21.15.7:0/1524079752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558dca256f60 0x558dca2509e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0099c0 tx=0x7fb00c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630c214d568db519 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:22.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.697+0000 7faffd7fa700 1 -- 172.21.15.7:0/1524079752 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c003860 con 0x558dca256f60 2024-06-10T01:09:22.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.697+0000 7faffd7fa700 1 -- 172.21.15.7:0/1524079752 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb00c026070 con 0x558dca256f60 2024-06-10T01:09:22.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.698+0000 7faffd7fa700 1 -- 172.21.15.7:0/1524079752 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c00b9d0 con 0x558dca256f60 2024-06-10T01:09:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.698+0000 7fb027fb3b80 1 -- 172.21.15.7:0/1524079752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558dca256f60 msgr2=0x558dca2509e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.698+0000 7fb027fb3b80 1 --2- 172.21.15.7:0/1524079752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558dca256f60 0x558dca2509e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0099c0 tx=0x7fb00c005c00 comp rx=0 tx=0).stop 2024-06-10T01:09:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.698+0000 7fb027fb3b80 1 -- 172.21.15.7:0/1524079752 shutdown_connections 2024-06-10T01:09:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.698+0000 7fb027fb3b80 1 --2- 172.21.15.7:0/1524079752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558dca250f20 0x558dca251340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.698+0000 7fb027fb3b80 1 --2- 172.21.15.7:0/1524079752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dca256640 0x558dca256a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.698+0000 7fb027fb3b80 1 --2- 172.21.15.7:0/1524079752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558dca256f60 0x558dca2509e0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.698+0000 7fb027fb3b80 1 -- 172.21.15.7:0/1524079752 >> 172.21.15.7:0/1524079752 conn(0x558dca17bcc0 msgr2=0x558dca24b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.699+0000 7fb027fb3b80 1 -- 172.21.15.7:0/1524079752 shutdown_connections 2024-06-10T01:09:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.699+0000 7fb027fb3b80 1 -- 172.21.15.7:0/1524079752 wait complete. 2024-06-10T01:09:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.699+0000 7fb027fb3b80 1 -- start start 2024-06-10T01:09:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.699+0000 7fb027fb3b80 1 Processor -- start 2024-06-10T01:09:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.699+0000 7fb027fb3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558dca250f20 0x558dca25ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.699+0000 7fb027fb3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dca256640 0x558dca259ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.699+0000 7fb027fb3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558dca256f60 0x558dca25a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.699+0000 7fb027fb3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558dca25d970 con 0x558dca256f60 2024-06-10T01:09:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.699+0000 7fb027fb3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558dca142bf0 con 0x558dca256640 2024-06-10T01:09:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.699+0000 7fb027fb3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558dca25daf0 con 0x558dca250f20 2024-06-10T01:09:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.700+0000 7fb016ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558dca250f20 0x558dca25ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.700+0000 7fb016ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558dca250f20 0x558dca25ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43182/0 (socket says 172.21.15.7:43182) 2024-06-10T01:09:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.700+0000 7fb0167fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dca256640 0x558dca259ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.700+0000 7fb016ffd700 1 -- 172.21.15.7:0/1035971520 learned_addr learned my addr 172.21.15.7:0/1035971520 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.700+0000 7fb0177fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558dca256f60 0x558dca25a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.700+0000 7fb027fb3b80 10 client.0 _ll_register_callbacks cb 0x558dca13c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.700+0000 7fb016ffd700 1 -- 172.21.15.7:0/1035971520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dca256640 msgr2=0x558dca259ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.700+0000 7fb016ffd700 1 --2- 172.21.15.7:0/1035971520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dca256640 0x558dca259ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.700+0000 7fb016ffd700 1 -- 172.21.15.7:0/1035971520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558dca256f60 msgr2=0x558dca25a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.700+0000 7fb016ffd700 1 --2- 172.21.15.7:0/1035971520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558dca256f60 0x558dca25a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.700+0000 7fb016ffd700 1 -- 172.21.15.7:0/1035971520 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dca0f18d0 con 0x558dca250f20 2024-06-10T01:09:22.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.700+0000 7fb0177fe700 1 --2- 172.21.15.7:0/1035971520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558dca256f60 0x558dca25a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:22.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.700+0000 7fb0167fc700 1 --2- 172.21.15.7:0/1035971520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dca256640 0x558dca259ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:22.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.700+0000 7fb016ffd700 1 --2- 172.21.15.7:0/1035971520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558dca250f20 0x558dca25ccf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb00801dcb0 tx=0x7fb0080047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d2fa3b6708c67e7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:22.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.700+0000 7faffaff5700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:09:22.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.700+0000 7faffaff5700 1 -- 172.21.15.7:0/1035971520 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00802b060 con 0x558dca250f20 2024-06-10T01:09:22.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.701+0000 7faffaff5700 1 -- 172.21.15.7:0/1035971520 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb00801ca20 con 0x558dca250f20 2024-06-10T01:09:22.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.701+0000 7fb027fb3b80 1 -- 172.21.15.7:0/1035971520 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb00c026070 con 0x558dca250f20 2024-06-10T01:09:22.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.701+0000 7faffaff5700 1 -- 172.21.15.7:0/1035971520 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb008032050 con 0x558dca250f20 2024-06-10T01:09:22.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.701+0000 7fb027fb3b80 10 client.25921 fetch_fsmap learned FSMap version 30 2024-06-10T01:09:22.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.701+0000 7fb027fb3b80 1 -- 172.21.15.7:0/1035971520 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558dca2682d0 con 0x558dca250f20 2024-06-10T01:09:22.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.701+0000 7faffaff5700 1 -- 172.21.15.7:0/1035971520 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb00c026070 con 0x558dca250f20 2024-06-10T01:09:22.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.701+0000 7faffaff5700 1 -- 172.21.15.7:0/1035971520 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fb00800e3b0 con 0x558dca250f20 2024-06-10T01:09:22.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.701+0000 7fb027fb3b80 10 client.25921 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:09:22.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.701+0000 7fb027fb3b80 10 client.25921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:22.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.701+0000 7fb027fb3b80 20 client.25921 populate_metadata read hostname 'smithi007' 2024-06-10T01:09:22.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.701+0000 7fb027fb3b80 1 --2- 172.21.15.7:0/1035971520 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558dca26a910 0x558dca26cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.701+0000 7fb027fb3b80 4 client.25921 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:09:22.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.701+0000 7fb027fb3b80 1 -- 172.21.15.7:0/1035971520 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dca1afd10 con 0x558dca26a910 2024-06-10T01:09:22.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.702+0000 7fb0167fc700 1 --2- 172.21.15.7:0/1035971520 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558dca26a910 0x558dca26cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.702+0000 7fb0167fc700 1 --2- 172.21.15.7:0/1035971520 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558dca26a910 0x558dca26cd20 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c564ae3c14466c4f server_cookie=da821781bba5c234 in_seq=0 out_seq=0 2024-06-10T01:09:22.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.702+0000 7faffaff5700 10 client.25921 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:22.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.704+0000 7faffaff5700 1 -- 172.21.15.7:0/1035971520 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558dca1afd10 con 0x558dca26a910 2024-06-10T01:09:22.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.704+0000 7faffaff5700 10 client.25921 handle_command_reply: tid=0 2024-06-10T01:09:22.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.704+0000 7fb027fb3b80 1 client.25921 shutdown 2024-06-10T01:09:22.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.704+0000 7fb027fb3b80 1 -- 172.21.15.7:0/1035971520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558dca250f20 msgr2=0x558dca25ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:22.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.704+0000 7fb027fb3b80 1 --2- 172.21.15.7:0/1035971520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558dca250f20 0x558dca25ccf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb00801dcb0 tx=0x7fb0080047c0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.704+0000 7fb027fb3b80 1 -- 172.21.15.7:0/1035971520 shutdown_connections 2024-06-10T01:09:22.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.704+0000 7fb027fb3b80 1 --2- 172.21.15.7:0/1035971520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558dca256f60 0x558dca25a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.704+0000 7fb027fb3b80 1 --2- 172.21.15.7:0/1035971520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558dca250f20 0x558dca25ccf0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.704+0000 7fb027fb3b80 1 --2- 172.21.15.7:0/1035971520 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558dca26a910 0x558dca26cd20 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.704+0000 7fb027fb3b80 1 --2- 172.21.15.7:0/1035971520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dca256640 0x558dca259ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.704+0000 7faffaff5700 0 client.25921 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:22.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.704+0000 7fb027fb3b80 1 -- 172.21.15.7:0/1035971520 >> 172.21.15.7:0/1035971520 conn(0x558dca17bcc0 msgr2=0x558dca24f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:22.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.705+0000 7fb027fb3b80 1 -- 172.21.15.7:0/1035971520 shutdown_connections 2024-06-10T01:09:22.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.705+0000 7fb027fb3b80 1 -- 172.21.15.7:0/1035971520 wait complete. 2024-06-10T01:09:22.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.705+0000 7fb027fb3b80 20 client.25921 trim_cache size 0 max 16384 2024-06-10T01:09:22.102 INFO:tasks.workunit.client.0.smithi007.stdout: 1 178052 9.67 MB/sec execute 2 sec latency 41.683 ms 2024-06-10T01:09:22.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.729+0000 7fb027fb3b80 1 Processor -- start 2024-06-10T01:09:22.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.729+0000 7fb027fb3b80 1 -- start start 2024-06-10T01:09:22.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.730+0000 7fb027fb3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558dca250f20 0x558dca2509e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.730+0000 7fb027fb3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dca256640 0x558dca24b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.730+0000 7fb027fb3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558dca256f60 0x558dca25a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.730+0000 7fb027fb3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb0181b6fe0 con 0x558dca256f60 2024-06-10T01:09:22.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.730+0000 7fb027fb3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb0181b4810 con 0x558dca256640 2024-06-10T01:09:22.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.730+0000 7fb027fb3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb0181b2660 con 0x558dca250f20 2024-06-10T01:09:22.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.730+0000 7fb0177fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558dca256f60 0x558dca25a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.730+0000 7fb0177fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558dca256f60 0x558dca25a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43346/0 (socket says 172.21.15.7:43346) 2024-06-10T01:09:22.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.730+0000 7fb0177fe700 1 -- 172.21.15.7:0/712125176 learned_addr learned my addr 172.21.15.7:0/712125176 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:22.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.730+0000 7fb016ffd700 1 --2- 172.21.15.7:0/712125176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558dca250f20 0x558dca2509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.730+0000 7fb0167fc700 1 --2- 172.21.15.7:0/712125176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dca256640 0x558dca24b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.730+0000 7fb0177fe700 1 -- 172.21.15.7:0/712125176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558dca250f20 msgr2=0x558dca2509e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:22.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.730+0000 7fb0177fe700 1 --2- 172.21.15.7:0/712125176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558dca250f20 0x558dca2509e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.730+0000 7fb0177fe700 1 -- 172.21.15.7:0/712125176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dca256640 msgr2=0x558dca24b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:22.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.730+0000 7fb0177fe700 1 --2- 172.21.15.7:0/712125176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dca256640 0x558dca24b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.730+0000 7fb0177fe700 1 -- 172.21.15.7:0/712125176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dca24c720 con 0x558dca256f60 2024-06-10T01:09:22.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.730+0000 7fb0167fc700 1 --2- 172.21.15.7:0/712125176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dca256640 0x558dca24b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:22.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.730+0000 7fb0177fe700 1 --2- 172.21.15.7:0/712125176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558dca256f60 0x558dca25a560 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb010097e60 tx=0x7fb0100bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c465a23ac76d910 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:22.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.731+0000 7faffcff9700 1 -- 172.21.15.7:0/712125176 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01009adf0 con 0x558dca256f60 2024-06-10T01:09:22.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.731+0000 7faffcff9700 1 -- 172.21.15.7:0/712125176 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb01009bb90 con 0x558dca256f60 2024-06-10T01:09:22.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.731+0000 7faffcff9700 1 -- 172.21.15.7:0/712125176 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0100b42a0 con 0x558dca256f60 2024-06-10T01:09:22.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.731+0000 7fb027fb3b80 1 -- 172.21.15.7:0/712125176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558dca256f60 msgr2=0x558dca25a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:22.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.731+0000 7fb027fb3b80 1 --2- 172.21.15.7:0/712125176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558dca256f60 0x558dca25a560 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb010097e60 tx=0x7fb0100bb040 comp rx=0 tx=0).stop 2024-06-10T01:09:22.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.731+0000 7fb027fb3b80 1 -- 172.21.15.7:0/712125176 shutdown_connections 2024-06-10T01:09:22.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.731+0000 7fb027fb3b80 1 --2- 172.21.15.7:0/712125176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558dca256f60 0x558dca25a560 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.731+0000 7fb027fb3b80 1 --2- 172.21.15.7:0/712125176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558dca250f20 0x558dca2509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.731+0000 7fb027fb3b80 1 --2- 172.21.15.7:0/712125176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dca256640 0x558dca24b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.731+0000 7fb027fb3b80 1 -- 172.21.15.7:0/712125176 >> 172.21.15.7:0/712125176 conn(0x558dca17bcc0 msgr2=0x558dca244e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:22.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.732+0000 7fb027fb3b80 1 -- 172.21.15.7:0/712125176 shutdown_connections 2024-06-10T01:09:22.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.732+0000 7fb027fb3b80 1 -- 172.21.15.7:0/712125176 wait complete. 2024-06-10T01:09:22.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.732+0000 7fb027fb3b80 1 -- start start 2024-06-10T01:09:22.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.732+0000 7fb027fb3b80 1 Processor -- start 2024-06-10T01:09:22.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.732+0000 7fb027fb3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dca250f20 0x558dca2632d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.732+0000 7fb027fb3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558dca256640 0x558dca271dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.732+0000 7fb027fb3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558dca256f60 0x558dca272300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.732+0000 7fb027fb3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb0181b2cd0 con 0x558dca256640 2024-06-10T01:09:22.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.732+0000 7fb027fb3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb0181b4140 con 0x558dca250f20 2024-06-10T01:09:22.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.732+0000 7fb027fb3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb0181b33a0 con 0x558dca256f60 2024-06-10T01:09:22.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.732+0000 7fb027fb3b80 10 client.0 _ll_register_callbacks cb 0x558dca13c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:22.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.732+0000 7fb016ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dca250f20 0x558dca2632d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.732+0000 7fb0167fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558dca256640 0x558dca271dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.732+0000 7fb0167fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558dca256640 0x558dca271dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43348/0 (socket says 172.21.15.7:43348) 2024-06-10T01:09:22.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.732+0000 7fb016ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dca250f20 0x558dca2632d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:32954/0 (socket says 172.21.15.7:32954) 2024-06-10T01:09:22.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.732+0000 7fb0177fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558dca256f60 0x558dca272300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.732+0000 7fb016ffd700 1 -- 172.21.15.7:0/497798644 learned_addr learned my addr 172.21.15.7:0/497798644 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:22.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.733+0000 7fb0167fc700 1 -- 172.21.15.7:0/497798644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558dca256f60 msgr2=0x558dca272300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:22.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.733+0000 7fb0167fc700 1 --2- 172.21.15.7:0/497798644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558dca256f60 0x558dca272300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.733+0000 7fb0167fc700 1 -- 172.21.15.7:0/497798644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dca250f20 msgr2=0x558dca2632d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:22.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.733+0000 7fb0167fc700 1 --2- 172.21.15.7:0/497798644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dca250f20 0x558dca2632d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.733+0000 7fb0167fc700 1 -- 172.21.15.7:0/497798644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dca272910 con 0x558dca256640 2024-06-10T01:09:22.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.733+0000 7fb0177fe700 1 --2- 172.21.15.7:0/497798644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558dca256f60 0x558dca272300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:22.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.733+0000 7fb016ffd700 1 --2- 172.21.15.7:0/497798644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dca250f20 0x558dca2632d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:22.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.733+0000 7fb0167fc700 1 --2- 172.21.15.7:0/497798644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558dca256640 0x558dca271dc0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fb00c003800 tx=0x7fb00c005600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=318771ec20736852 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.733+0000 7faffaff5700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:09:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.733+0000 7faffaff5700 1 -- 172.21.15.7:0/497798644 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c00bca0 con 0x558dca256640 2024-06-10T01:09:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.733+0000 7faffaff5700 1 -- 172.21.15.7:0/497798644 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb00c00be40 con 0x558dca256640 2024-06-10T01:09:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.733+0000 7fb027fb3b80 1 -- 172.21.15.7:0/497798644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb01009bb90 con 0x558dca256640 2024-06-10T01:09:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.733+0000 7faffaff5700 1 -- 172.21.15.7:0/497798644 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb00c03d050 con 0x558dca256640 2024-06-10T01:09:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.733+0000 7fb027fb3b80 10 client.16383 fetch_fsmap learned FSMap version 30 2024-06-10T01:09:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.733+0000 7fb027fb3b80 1 -- 172.21.15.7:0/497798644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558dca2d5490 con 0x558dca256640 2024-06-10T01:09:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.733+0000 7faffaff5700 1 -- 172.21.15.7:0/497798644 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb01009bb90 con 0x558dca256640 2024-06-10T01:09:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.734+0000 7faffaff5700 1 -- 172.21.15.7:0/497798644 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fb00c00f920 con 0x558dca256640 2024-06-10T01:09:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.734+0000 7fb027fb3b80 10 client.16383 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:09:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.734+0000 7fb027fb3b80 10 client.16383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.734+0000 7fb027fb3b80 20 client.16383 populate_metadata read hostname 'smithi007' 2024-06-10T01:09:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.734+0000 7fb027fb3b80 1 --2- 172.21.15.7:0/497798644 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558dca2d5760 0x558dca2d7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.734+0000 7fb027fb3b80 4 client.16383 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:09:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.734+0000 7fb027fb3b80 1 -- 172.21.15.7:0/497798644 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558dca1aa290 con 0x558dca2d5760 2024-06-10T01:09:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.734+0000 7fb016ffd700 1 --2- 172.21.15.7:0/497798644 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558dca2d5760 0x558dca2d7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.734+0000 7fb016ffd700 1 --2- 172.21.15.7:0/497798644 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558dca2d5760 0x558dca2d7b50 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0175fa7bcb814ea server_cookie=b6b0c41ec377568c in_seq=0 out_seq=0 2024-06-10T01:09:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.734+0000 7faffaff5700 10 client.16383 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.737+0000 7faffaff5700 1 -- 172.21.15.7:0/497798644 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558dca1aa290 con 0x558dca2d5760 2024-06-10T01:09:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.737+0000 7faffaff5700 10 client.16383 handle_command_reply: tid=0 2024-06-10T01:09:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.737+0000 7fb027fb3b80 1 client.16383 shutdown 2024-06-10T01:09:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.737+0000 7fb027fb3b80 1 -- 172.21.15.7:0/497798644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558dca256640 msgr2=0x558dca271dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.737+0000 7fb027fb3b80 1 --2- 172.21.15.7:0/497798644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558dca256640 0x558dca271dc0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fb00c003800 tx=0x7fb00c005600 comp rx=0 tx=0).stop 2024-06-10T01:09:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.737+0000 7fb027fb3b80 1 -- 172.21.15.7:0/497798644 shutdown_connections 2024-06-10T01:09:22.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.737+0000 7fb027fb3b80 1 --2- 172.21.15.7:0/497798644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558dca256f60 0x558dca272300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.737+0000 7fb027fb3b80 1 --2- 172.21.15.7:0/497798644 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558dca2d5760 0x558dca2d7b50 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.737+0000 7fb027fb3b80 1 --2- 172.21.15.7:0/497798644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dca250f20 0x558dca2632d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.737+0000 7faffaff5700 0 client.16383 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:22.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.737+0000 7fb027fb3b80 1 --2- 172.21.15.7:0/497798644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558dca256640 0x558dca271dc0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.737+0000 7fb027fb3b80 1 -- 172.21.15.7:0/497798644 >> 172.21.15.7:0/497798644 conn(0x558dca17bcc0 msgr2=0x558dca257340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:22.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.737+0000 7fb027fb3b80 1 -- 172.21.15.7:0/497798644 shutdown_connections 2024-06-10T01:09:22.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.737+0000 7fb027fb3b80 1 -- 172.21.15.7:0/497798644 wait complete. 2024-06-10T01:09:22.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.737+0000 7fb027fb3b80 20 client.16383 trim_cache size 0 max 16384 2024-06-10T01:09:22.115 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:09:22.115 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T01:09:22.115 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-10T01:09:22.115 INFO:teuthology.orchestra.run.smithi007.stdout: "76fc0745-2b6a-4b68-a0c2-9e866ba0b0da": { 2024-06-10T01:09:22.115 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-10T01:09:22.115 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "76fc0745-2b6a-4b68-a0c2-9e866ba0b0da", 2024-06-10T01:09:22.115 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-10T01:09:22.115 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:09:22.116 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:09:22.116 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:09:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.738+0000 7fb01d730700 1 -- 172.21.15.7:0/3042755423 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb0000a2c20 msgr2=0x7fb0000a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.738+0000 7fb01d730700 1 --2- 172.21.15.7:0/3042755423 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb0000a2c20 0x7fb0000a50e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb008009d20 tx=0x7fb008009260 comp rx=0 tx=0).stop 2024-06-10T01:09:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.738+0000 7fb01d730700 1 -- 172.21.15.7:0/3042755423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb018107860 msgr2=0x7fb0181a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.738+0000 7fb01d730700 1 --2- 172.21.15.7:0/3042755423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb018107860 0x7fb0181a3a50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb01000efc0 tx=0x7fb01000c3a0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.738+0000 7fb01d730700 1 -- 172.21.15.7:0/3042755423 shutdown_connections 2024-06-10T01:09:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.738+0000 7fb01d730700 1 --2- 172.21.15.7:0/3042755423 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb0000a2c20 0x7fb0000a50e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.738+0000 7fb01d730700 1 --2- 172.21.15.7:0/3042755423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb018107860 0x7fb0181a3a50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.738+0000 7fb01d730700 1 --2- 172.21.15.7:0/3042755423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb018102770 0x7fb01810e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.738+0000 7fb01d730700 1 --2- 172.21.15.7:0/3042755423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb018103120 0x7fb01810eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.738+0000 7fb01d730700 1 -- 172.21.15.7:0/3042755423 >> 172.21.15.7:0/3042755423 conn(0x7fb018079600 msgr2=0x7fb0181003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.738+0000 7fb01d730700 1 -- 172.21.15.7:0/3042755423 shutdown_connections 2024-06-10T01:09:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:21.738+0000 7fb01d730700 1 -- 172.21.15.7:0/3042755423 wait complete. 2024-06-10T01:09:22.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.900+0000 7fb2827fc640 1 -- 172.21.15.7:0/3181252874 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2740319b0 con 0x7fb2841035d0 2024-06-10T01:09:22.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.900+0000 7fb289ebd640 1 -- 172.21.15.7:0/3181252874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2841035d0 msgr2=0x7fb284109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:22.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.900+0000 7fb289ebd640 1 --2- 172.21.15.7:0/3181252874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2841035d0 0x7fb284109e60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb274009900 tx=0x7fb27402a3f0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.900+0000 7fb289ebd640 1 -- 172.21.15.7:0/3181252874 shutdown_connections 2024-06-10T01:09:22.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.900+0000 7fb289ebd640 1 --2- 172.21.15.7:0/3181252874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2841035d0 0x7fb284109e60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.900+0000 7fb289ebd640 1 --2- 172.21.15.7:0/3181252874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb284102c10 0x7fb284103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.900+0000 7fb289ebd640 1 --2- 172.21.15.7:0/3181252874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb284101a20 0x7fb284101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.900+0000 7fb289ebd640 1 -- 172.21.15.7:0/3181252874 >> 172.21.15.7:0/3181252874 conn(0x7fb2840fcf90 msgr2=0x7fb2840ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:22.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.901+0000 7fb289ebd640 1 -- 172.21.15.7:0/3181252874 shutdown_connections 2024-06-10T01:09:22.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.901+0000 7fb289ebd640 1 -- 172.21.15.7:0/3181252874 wait complete. 2024-06-10T01:09:22.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.902+0000 7fb289ebd640 1 Processor -- start 2024-06-10T01:09:22.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.902+0000 7fb289ebd640 1 -- start start 2024-06-10T01:09:22.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.902+0000 7fb289ebd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb284101a20 0x7fb284110de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.902+0000 7fb289ebd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb284102c10 0x7fb284111320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.902+0000 7fb289ebd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2841035d0 0x7fb284116380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.902+0000 7fb289ebd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb28410b4f0 con 0x7fb284101a20 2024-06-10T01:09:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.902+0000 7fb289ebd640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb28410b370 con 0x7fb284102c10 2024-06-10T01:09:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.902+0000 7fb289ebd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb28410b670 con 0x7fb2841035d0 2024-06-10T01:09:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.903+0000 7fb283fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2841035d0 0x7fb284116380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.903+0000 7fb283fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2841035d0 0x7fb284116380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:42904/0 (socket says 172.21.15.7:42904) 2024-06-10T01:09:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.903+0000 7fb283fff640 1 -- 172.21.15.7:0/1713268612 learned_addr learned my addr 172.21.15.7:0/1713268612 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.903+0000 7fb2837fe640 1 --2- 172.21.15.7:0/1713268612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb284101a20 0x7fb284110de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.903+0000 7fb283fff640 1 -- 172.21.15.7:0/1713268612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb284102c10 msgr2=0x7fb284111320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.903+0000 7fb282ffd640 1 --2- 172.21.15.7:0/1713268612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb284102c10 0x7fb284111320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.903+0000 7fb283fff640 1 --2- 172.21.15.7:0/1713268612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb284102c10 0x7fb284111320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.903+0000 7fb283fff640 1 -- 172.21.15.7:0/1713268612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb284101a20 msgr2=0x7fb284110de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.903+0000 7fb283fff640 1 --2- 172.21.15.7:0/1713268612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb284101a20 0x7fb284110de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.903+0000 7fb283fff640 1 -- 172.21.15.7:0/1713268612 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2841169f0 con 0x7fb2841035d0 2024-06-10T01:09:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.903+0000 7fb2837fe640 1 --2- 172.21.15.7:0/1713268612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb284101a20 0x7fb284110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.904+0000 7fb282ffd640 1 --2- 172.21.15.7:0/1713268612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb284102c10 0x7fb284111320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:09:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.904+0000 7fb283fff640 1 --2- 172.21.15.7:0/1713268612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2841035d0 0x7fb284116380 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb27402a910 tx=0x7fb274032c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.905+0000 7fb280ff9640 1 -- 172.21.15.7:0/1713268612 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27403f030 con 0x7fb2841035d0 2024-06-10T01:09:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.905+0000 7fb280ff9640 1 -- 172.21.15.7:0/1713268612 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb274032e50 con 0x7fb2841035d0 2024-06-10T01:09:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.905+0000 7fb280ff9640 1 -- 172.21.15.7:0/1713268612 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb274035070 con 0x7fb2841035d0 2024-06-10T01:09:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.905+0000 7fb289ebd640 1 -- 172.21.15.7:0/1713268612 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb284116c80 con 0x7fb2841035d0 2024-06-10T01:09:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.906+0000 7fb289ebd640 1 -- 172.21.15.7:0/1713268612 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2841aeed0 con 0x7fb2841035d0 2024-06-10T01:09:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.910+0000 7fb28cbcf740 1 Processor -- start 2024-06-10T01:09:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.911+0000 7fb28cbcf740 1 -- start start 2024-06-10T01:09:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.911+0000 7fb280ff9640 1 -- 172.21.15.7:0/1713268612 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb274042030 con 0x7fb2841035d0 2024-06-10T01:09:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.911+0000 7fb28cbcf740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572d109f010 0x5572d109f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.911+0000 7fb28cbcf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572d109fa00 0x5572d10a4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.911+0000 7fb28cbcf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572d10a4b70 0x5572d10a7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.911+0000 7fb28cbcf740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5572d10024f0 con 0x5572d109f010 2024-06-10T01:09:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.911+0000 7fb28cbcf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5572d0fa0870 con 0x5572d10a4b70 2024-06-10T01:09:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.911+0000 7fb28cbcf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5572d0ffe4b0 con 0x5572d109fa00 2024-06-10T01:09:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.911+0000 7fb282ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572d109f010 0x5572d109f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.911+0000 7fb282ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572d109f010 0x5572d109f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51862/0 (socket says 172.21.15.7:51862) 2024-06-10T01:09:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.911+0000 7fb282ffd640 1 -- 172.21.15.7:0/4066493091 learned_addr learned my addr 172.21.15.7:0/4066493091 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.911+0000 7fb2837fe640 1 --2- 172.21.15.7:0/4066493091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572d109fa00 0x5572d10a4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.911+0000 7fb282ffd640 1 --2- 172.21.15.7:0/4066493091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572d10a4b70 0x5572d10a7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.912+0000 7fb282ffd640 1 -- 172.21.15.7:0/4066493091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572d109fa00 msgr2=0x5572d10a4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.912+0000 7fb282ffd640 1 --2- 172.21.15.7:0/4066493091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572d109fa00 0x5572d10a4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.912+0000 7fb282ffd640 1 -- 172.21.15.7:0/4066493091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572d10a4b70 msgr2=0x5572d10a7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.912+0000 7fb282ffd640 1 --2- 172.21.15.7:0/4066493091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572d10a4b70 0x5572d10a7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.912+0000 7fb282ffd640 1 -- 172.21.15.7:0/4066493091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572d0fbc9b0 con 0x5572d109f010 2024-06-10T01:09:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.912+0000 7fb2837fe640 1 --2- 172.21.15.7:0/4066493091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572d109fa00 0x5572d10a4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.912+0000 7fb280ff9640 1 --2- 172.21.15.7:0/1713268612 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb26c0a2d50 0x7fb26c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.912+0000 7fb282ffd640 1 --2- 172.21.15.7:0/4066493091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572d109f010 0x5572d109f430 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb278015870 tx=0x7fb278009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a8be449a25c0e6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.912+0000 7fb280ff9640 1 -- 172.21.15.7:0/1713268612 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb2740ef5e0 con 0x7fb2841035d0 2024-06-10T01:09:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.912+0000 7fb269ffb640 1 -- 172.21.15.7:0/4066493091 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb278004020 con 0x5572d109f010 2024-06-10T01:09:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.912+0000 7fb2837fe640 1 --2- 172.21.15.7:0/1713268612 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb26c0a2d50 0x7fb26c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.913+0000 7fb269ffb640 1 -- 172.21.15.7:0/4066493091 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb27800ebd0 con 0x5572d109f010 2024-06-10T01:09:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.913+0000 7fb269ffb640 1 -- 172.21.15.7:0/4066493091 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2780035f0 con 0x5572d109f010 2024-06-10T01:09:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.913+0000 7fb28cbcf740 1 -- 172.21.15.7:0/4066493091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572d109f010 msgr2=0x5572d109f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.913+0000 7fb28cbcf740 1 --2- 172.21.15.7:0/4066493091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572d109f010 0x5572d109f430 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb278015870 tx=0x7fb278009670 comp rx=0 tx=0).stop 2024-06-10T01:09:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.913+0000 7fb2837fe640 1 --2- 172.21.15.7:0/1713268612 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb26c0a2d50 0x7fb26c0a5210 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x5572d109fe00 tx=0x7fb27c009720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.913+0000 7fb28cbcf740 1 -- 172.21.15.7:0/4066493091 shutdown_connections 2024-06-10T01:09:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.913+0000 7fb28cbcf740 1 --2- 172.21.15.7:0/4066493091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572d10a4b70 0x5572d10a7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.913+0000 7fb28cbcf740 1 --2- 172.21.15.7:0/4066493091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572d109fa00 0x5572d10a4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.913+0000 7fb28cbcf740 1 --2- 172.21.15.7:0/4066493091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572d109f010 0x5572d109f430 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.913+0000 7fb28cbcf740 1 -- 172.21.15.7:0/4066493091 >> 172.21.15.7:0/4066493091 conn(0x5572d109a680 msgr2=0x5572d109aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.913+0000 7fb28cbcf740 1 -- 172.21.15.7:0/4066493091 shutdown_connections 2024-06-10T01:09:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.913+0000 7fb28cbcf740 1 -- 172.21.15.7:0/4066493091 wait complete. 2024-06-10T01:09:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.914+0000 7fb28cbcf740 1 -- start start 2024-06-10T01:09:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.914+0000 7fb28cbcf740 1 Processor -- start 2024-06-10T01:09:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.914+0000 7fb28cbcf740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572d109f010 0x5572d10a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.915+0000 7fb28cbcf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572d109fa00 0x5572d10a97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.915+0000 7fb28cbcf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572d10a4b70 0x5572d10b0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.915+0000 7fb28cbcf740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5572d10a7ac0 con 0x5572d109f010 2024-06-10T01:09:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.915+0000 7fb28cbcf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5572d0f8bae0 con 0x5572d10a4b70 2024-06-10T01:09:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.915+0000 7fb28cbcf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5572d10a7c40 con 0x5572d109fa00 2024-06-10T01:09:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.915+0000 7fb2837fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572d109f010 0x5572d10a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.915+0000 7fb28cbcf740 10 client.0 _ll_register_callbacks cb 0x5572d0fc49b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.915+0000 7fb283fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572d10a4b70 0x5572d10b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.915+0000 7fb2837fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572d109f010 0x5572d10a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51866/0 (socket says 172.21.15.7:51866) 2024-06-10T01:09:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.915+0000 7fb2837fe640 1 -- 172.21.15.7:0/4165461842 learned_addr learned my addr 172.21.15.7:0/4165461842 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.915+0000 7fb283fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572d10a4b70 0x5572d10b0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:35964/0 (socket says 172.21.15.7:35964) 2024-06-10T01:09:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.915+0000 7fb282ffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572d109fa00 0x5572d10a97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.915+0000 7fb283fff640 1 -- 172.21.15.7:0/4165461842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572d109fa00 msgr2=0x5572d10a97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.915+0000 7fb283fff640 1 --2- 172.21.15.7:0/4165461842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572d109fa00 0x5572d10a97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.915+0000 7fb283fff640 1 -- 172.21.15.7:0/4165461842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572d109f010 msgr2=0x5572d10a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.915+0000 7fb283fff640 1 --2- 172.21.15.7:0/4165461842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572d109f010 0x5572d10a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.915+0000 7fb283fff640 1 -- 172.21.15.7:0/4165461842 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572d0d16210 con 0x5572d10a4b70 2024-06-10T01:09:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.915+0000 7fb2837fe640 1 --2- 172.21.15.7:0/4165461842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572d109f010 0x5572d10a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.915+0000 7fb283fff640 1 --2- 172.21.15.7:0/4165461842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572d10a4b70 0x5572d10b0800 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb2740ecd00 tx=0x7fb2740bb3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c3133bc93b36cad server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.915+0000 7fb262ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:09:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.915+0000 7fb262ffd640 1 -- 172.21.15.7:0/4165461842 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2740d3020 con 0x5572d10a4b70 2024-06-10T01:09:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.915+0000 7fb262ffd640 1 -- 172.21.15.7:0/4165461842 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb2740ef2d0 con 0x5572d10a4b70 2024-06-10T01:09:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.915+0000 7fb28cbcf740 1 -- 172.21.15.7:0/4165461842 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb27800ebd0 con 0x5572d10a4b70 2024-06-10T01:09:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.916+0000 7fb262ffd640 1 -- 172.21.15.7:0/4165461842 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb2740bba60 con 0x5572d10a4b70 2024-06-10T01:09:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.916+0000 7fb28cbcf740 10 client.25916 fetch_fsmap learned FSMap version 30 2024-06-10T01:09:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.916+0000 7fb28cbcf740 1 -- 172.21.15.7:0/4165461842 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5572d0de2620 con 0x5572d10a4b70 2024-06-10T01:09:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.916+0000 7fb262ffd640 1 -- 172.21.15.7:0/4165461842 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb27800ebd0 con 0x5572d10a4b70 2024-06-10T01:09:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.916+0000 7fb262ffd640 1 -- 172.21.15.7:0/4165461842 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fb2740c4240 con 0x5572d10a4b70 2024-06-10T01:09:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.916+0000 7fb28cbcf740 10 client.25916 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:09:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.916+0000 7fb28cbcf740 10 client.25916 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.916+0000 7fb28cbcf740 20 client.25916 populate_metadata read hostname 'smithi007' 2024-06-10T01:09:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.916+0000 7fb28cbcf740 1 --2- 172.21.15.7:0/4165461842 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5572d10b50b0 0x5572d10b7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.916+0000 7fb28cbcf740 4 client.25916 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:09:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.916+0000 7fb28cbcf740 1 -- 172.21.15.7:0/4165461842 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572d1015f10 con 0x5572d10b50b0 2024-06-10T01:09:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.916+0000 7fb2837fe640 1 --2- 172.21.15.7:0/4165461842 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5572d10b50b0 0x5572d10b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.917+0000 7fb2837fe640 1 --2- 172.21.15.7:0/4165461842 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5572d10b50b0 0x5572d10b7530 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32a101ba066d64ff server_cookie=3ead803066c0a31c in_seq=0 out_seq=0 2024-06-10T01:09:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.917+0000 7fb262ffd640 10 client.25916 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.917+0000 7fb262ffd640 1 -- 172.21.15.7:0/4165461842 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5572d1015f10 con 0x5572d10b50b0 2024-06-10T01:09:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.917+0000 7fb262ffd640 10 client.25916 handle_command_reply: tid=0 2024-06-10T01:09:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.917+0000 7fb28cbcf740 1 client.25916 shutdown 2024-06-10T01:09:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.918+0000 7fb28cbcf740 1 -- 172.21.15.7:0/4165461842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572d10a4b70 msgr2=0x5572d10b0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.918+0000 7fb28cbcf740 1 --2- 172.21.15.7:0/4165461842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572d10a4b70 0x5572d10b0800 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb2740ecd00 tx=0x7fb2740bb3c0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.918+0000 7fb28cbcf740 1 -- 172.21.15.7:0/4165461842 shutdown_connections 2024-06-10T01:09:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.918+0000 7fb28cbcf740 1 --2- 172.21.15.7:0/4165461842 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5572d10b50b0 0x5572d10b7530 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.918+0000 7fb28cbcf740 1 --2- 172.21.15.7:0/4165461842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572d10a4b70 0x5572d10b0800 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.918+0000 7fb262ffd640 0 client.25916 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.918+0000 7fb28cbcf740 1 --2- 172.21.15.7:0/4165461842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572d109fa00 0x5572d10a97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.918+0000 7fb28cbcf740 1 --2- 172.21.15.7:0/4165461842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572d109f010 0x5572d10a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.918+0000 7fb28cbcf740 1 -- 172.21.15.7:0/4165461842 >> 172.21.15.7:0/4165461842 conn(0x5572d109a680 msgr2=0x5572d10997e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.918+0000 7fb28cbcf740 1 -- 172.21.15.7:0/4165461842 shutdown_connections 2024-06-10T01:09:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.918+0000 7fb28cbcf740 1 -- 172.21.15.7:0/4165461842 wait complete. 2024-06-10T01:09:22.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.918+0000 7fb28cbcf740 20 client.25916 trim_cache size 0 max 16384 2024-06-10T01:09:22.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.934+0000 7fb28cbcf740 1 Processor -- start 2024-06-10T01:09:22.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.934+0000 7fb28cbcf740 1 -- start start 2024-06-10T01:09:22.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.934+0000 7fb28cbcf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572d109f010 0x5572d10a6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.934+0000 7fb28cbcf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572d109fa00 0x5572d10b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.934+0000 7fb28cbcf740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572d10a4b70 0x5572d109d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.934+0000 7fb28cbcf740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb2841bbbb0 con 0x5572d10a4b70 2024-06-10T01:09:22.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.934+0000 7fb28cbcf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb2841b9420 con 0x5572d109fa00 2024-06-10T01:09:22.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.934+0000 7fb28cbcf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb2841b7270 con 0x5572d109f010 2024-06-10T01:09:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.934+0000 7fb283fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572d10a4b70 0x5572d109d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.934+0000 7fb2837fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572d109f010 0x5572d10a6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.934+0000 7fb283fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572d10a4b70 0x5572d109d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51882/0 (socket says 172.21.15.7:51882) 2024-06-10T01:09:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.934+0000 7fb283fff640 1 -- 172.21.15.7:0/1892892511 learned_addr learned my addr 172.21.15.7:0/1892892511 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.934+0000 7fb282ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572d109fa00 0x5572d10b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.934+0000 7fb283fff640 1 -- 172.21.15.7:0/1892892511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572d109f010 msgr2=0x5572d10a6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.934+0000 7fb283fff640 1 --2- 172.21.15.7:0/1892892511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572d109f010 0x5572d10a6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.934+0000 7fb283fff640 1 -- 172.21.15.7:0/1892892511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572d109fa00 msgr2=0x5572d10b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.934+0000 7fb283fff640 1 --2- 172.21.15.7:0/1892892511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572d109fa00 0x5572d10b07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.934+0000 7fb283fff640 1 -- 172.21.15.7:0/1892892511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572d10a3f90 con 0x5572d10a4b70 2024-06-10T01:09:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.934+0000 7fb283fff640 1 --2- 172.21.15.7:0/1892892511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572d10a4b70 0x5572d109d670 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fb274005ab0 tx=0x7fb2740b9f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7975c7fde1648e41 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:22.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.935+0000 7fb2697fa640 1 -- 172.21.15.7:0/1892892511 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2740d3070 con 0x5572d10a4b70 2024-06-10T01:09:22.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.935+0000 7fb2697fa640 1 -- 172.21.15.7:0/1892892511 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2740bfe10 con 0x5572d10a4b70 2024-06-10T01:09:22.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.935+0000 7fb2697fa640 1 -- 172.21.15.7:0/1892892511 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2740b80d0 con 0x5572d10a4b70 2024-06-10T01:09:22.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.935+0000 7fb28cbcf740 1 -- 172.21.15.7:0/1892892511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572d10a4b70 msgr2=0x5572d109d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:22.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.935+0000 7fb28cbcf740 1 --2- 172.21.15.7:0/1892892511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572d10a4b70 0x5572d109d670 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fb274005ab0 tx=0x7fb2740b9f70 comp rx=0 tx=0).stop 2024-06-10T01:09:22.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.936+0000 7fb28cbcf740 1 -- 172.21.15.7:0/1892892511 shutdown_connections 2024-06-10T01:09:22.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.936+0000 7fb28cbcf740 1 --2- 172.21.15.7:0/1892892511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572d10a4b70 0x5572d109d670 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.936+0000 7fb28cbcf740 1 --2- 172.21.15.7:0/1892892511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572d109fa00 0x5572d10b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.936+0000 7fb28cbcf740 1 --2- 172.21.15.7:0/1892892511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572d109f010 0x5572d10a6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.936+0000 7fb28cbcf740 1 -- 172.21.15.7:0/1892892511 >> 172.21.15.7:0/1892892511 conn(0x5572d109a680 msgr2=0x5572d10a0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:22.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.936+0000 7fb28cbcf740 1 -- 172.21.15.7:0/1892892511 shutdown_connections 2024-06-10T01:09:22.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.936+0000 7fb28cbcf740 1 -- 172.21.15.7:0/1892892511 wait complete. 2024-06-10T01:09:22.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.936+0000 7fb28cbcf740 1 -- start start 2024-06-10T01:09:22.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.936+0000 7fb28cbcf740 1 Processor -- start 2024-06-10T01:09:22.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.936+0000 7fb28cbcf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572d109f010 0x5572d10ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.936+0000 7fb28cbcf740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572d109fa00 0x5572d10addf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.936+0000 7fb28cbcf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572d10a4b70 0x5572d10aee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.936+0000 7fb28cbcf740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb2841b78e0 con 0x5572d109fa00 2024-06-10T01:09:22.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.936+0000 7fb28cbcf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb2841b8d50 con 0x5572d10a4b70 2024-06-10T01:09:22.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.936+0000 7fb28cbcf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb2841b7fb0 con 0x5572d109f010 2024-06-10T01:09:22.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.936+0000 7fb28cbcf740 10 client.0 _ll_register_callbacks cb 0x7fb274009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:22.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.936+0000 7fb282ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572d109fa00 0x5572d10addf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.936+0000 7fb282ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572d109fa00 0x5572d10addf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51884/0 (socket says 172.21.15.7:51884) 2024-06-10T01:09:22.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.937+0000 7fb282ffd640 1 -- 172.21.15.7:0/2836054202 learned_addr learned my addr 172.21.15.7:0/2836054202 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:22.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.937+0000 7fb283fff640 1 --2- 172.21.15.7:0/2836054202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572d10a4b70 0x5572d10aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.937+0000 7fb2837fe640 1 --2- 172.21.15.7:0/2836054202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572d109f010 0x5572d10ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.937+0000 7fb282ffd640 1 -- 172.21.15.7:0/2836054202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572d109f010 msgr2=0x5572d10ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:22.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.937+0000 7fb282ffd640 1 --2- 172.21.15.7:0/2836054202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572d109f010 0x5572d10ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.937+0000 7fb282ffd640 1 -- 172.21.15.7:0/2836054202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572d10a4b70 msgr2=0x5572d10aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:22.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.937+0000 7fb282ffd640 1 --2- 172.21.15.7:0/2836054202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572d10a4b70 0x5572d10aee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.937+0000 7fb282ffd640 1 -- 172.21.15.7:0/2836054202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572d10af3f0 con 0x5572d109fa00 2024-06-10T01:09:22.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.937+0000 7fb2837fe640 1 --2- 172.21.15.7:0/2836054202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572d109f010 0x5572d10ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:22.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.937+0000 7fb283fff640 1 --2- 172.21.15.7:0/2836054202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572d10a4b70 0x5572d10aee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:22.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.937+0000 7fb282ffd640 1 --2- 172.21.15.7:0/2836054202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572d109fa00 0x5572d10addf0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x5572d10a0260 tx=0x7fb2780154f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6068757f562673 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:22.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.937+0000 7fb262ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:09:22.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.937+0000 7fb262ffd640 1 -- 172.21.15.7:0/2836054202 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb278005580 con 0x5572d109fa00 2024-06-10T01:09:22.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.937+0000 7fb262ffd640 1 -- 172.21.15.7:0/2836054202 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb27800e6c0 con 0x5572d109fa00 2024-06-10T01:09:22.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.937+0000 7fb28cbcf740 1 -- 172.21.15.7:0/2836054202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2740bfe10 con 0x5572d109fa00 2024-06-10T01:09:22.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.937+0000 7fb262ffd640 1 -- 172.21.15.7:0/2836054202 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb278030050 con 0x5572d109fa00 2024-06-10T01:09:22.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.937+0000 7fb28cbcf740 10 client.16401 fetch_fsmap learned FSMap version 30 2024-06-10T01:09:22.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.937+0000 7fb28cbcf740 1 -- 172.21.15.7:0/2836054202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5572d10b0700 con 0x5572d109fa00 2024-06-10T01:09:22.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.938+0000 7fb262ffd640 1 -- 172.21.15.7:0/2836054202 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb27800eda0 con 0x5572d109fa00 2024-06-10T01:09:22.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.938+0000 7fb262ffd640 1 -- 172.21.15.7:0/2836054202 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fb278027070 con 0x5572d109fa00 2024-06-10T01:09:22.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.938+0000 7fb28cbcf740 10 client.16401 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:09:22.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.938+0000 7fb28cbcf740 10 client.16401 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:22.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.938+0000 7fb28cbcf740 20 client.16401 populate_metadata read hostname 'smithi007' 2024-06-10T01:09:22.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.938+0000 7fb28cbcf740 1 --2- 172.21.15.7:0/2836054202 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5572d10b50b0 0x5572d1121ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:22.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.938+0000 7fb28cbcf740 4 client.16401 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:09:22.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.938+0000 7fb28cbcf740 1 -- 172.21.15.7:0/2836054202 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fb2740bfe10 con 0x5572d10b50b0 2024-06-10T01:09:22.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.938+0000 7fb2837fe640 1 --2- 172.21.15.7:0/2836054202 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5572d10b50b0 0x5572d1121ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:22.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.938+0000 7fb2837fe640 1 --2- 172.21.15.7:0/2836054202 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5572d10b50b0 0x5572d1121ec0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a85a2670cbbd67e5 server_cookie=95f17cc6eaaa4bdf in_seq=0 out_seq=0 2024-06-10T01:09:22.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:21.938+0000 7fb262ffd640 10 client.16401 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:22.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:22.063+0000 7fb262ffd640 1 -- 172.21.15.7:0/2836054202 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fb2740bfe10 con 0x5572d10b50b0 2024-06-10T01:09:22.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:22.063+0000 7fb262ffd640 10 client.16401 handle_command_reply: tid=0 2024-06-10T01:09:22.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:22.063+0000 7fb28cbcf740 1 client.16401 shutdown 2024-06-10T01:09:22.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:22.064+0000 7fb28cbcf740 1 -- 172.21.15.7:0/2836054202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572d109fa00 msgr2=0x5572d10addf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:22.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:22.064+0000 7fb28cbcf740 1 --2- 172.21.15.7:0/2836054202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572d109fa00 0x5572d10addf0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x5572d10a0260 tx=0x7fb2780154f0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:22.064+0000 7fb28cbcf740 1 -- 172.21.15.7:0/2836054202 shutdown_connections 2024-06-10T01:09:22.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:22.064+0000 7fb28cbcf740 1 --2- 172.21.15.7:0/2836054202 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5572d10b50b0 0x5572d1121ec0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:22.064+0000 7fb28cbcf740 1 --2- 172.21.15.7:0/2836054202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572d10a4b70 0x5572d10aee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:22.064+0000 7fb28cbcf740 1 --2- 172.21.15.7:0/2836054202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572d109fa00 0x5572d10addf0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:22.064+0000 7fb262ffd640 0 client.16401 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:22.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:22.064+0000 7fb28cbcf740 1 --2- 172.21.15.7:0/2836054202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572d109f010 0x5572d10ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:22.064+0000 7fb28cbcf740 1 -- 172.21.15.7:0/2836054202 >> 172.21.15.7:0/2836054202 conn(0x5572d109a680 msgr2=0x5572d10a9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:22.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:22.064+0000 7fb28cbcf740 1 -- 172.21.15.7:0/2836054202 shutdown_connections 2024-06-10T01:09:22.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:22.064+0000 7fb28cbcf740 1 -- 172.21.15.7:0/2836054202 wait complete. 2024-06-10T01:09:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:22.064+0000 7fb28cbcf740 20 client.16401 trim_cache size 0 max 16384 2024-06-10T01:09:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:09:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:09:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:09:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:22.065+0000 7fb289ebd640 1 -- 172.21.15.7:0/1713268612 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb26c0a2d50 msgr2=0x7fb26c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:22.065+0000 7fb289ebd640 1 --2- 172.21.15.7:0/1713268612 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb26c0a2d50 0x7fb26c0a5210 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x5572d109fe00 tx=0x7fb27c009720 comp rx=0 tx=0).stop 2024-06-10T01:09:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:22.065+0000 7fb289ebd640 1 -- 172.21.15.7:0/1713268612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2841035d0 msgr2=0x7fb284116380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:22.065+0000 7fb289ebd640 1 --2- 172.21.15.7:0/1713268612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2841035d0 0x7fb284116380 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb27402a910 tx=0x7fb274032c60 comp rx=0 tx=0).stop 2024-06-10T01:09:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:22.065+0000 7fb289ebd640 1 -- 172.21.15.7:0/1713268612 shutdown_connections 2024-06-10T01:09:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:22.065+0000 7fb289ebd640 1 --2- 172.21.15.7:0/1713268612 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb26c0a2d50 0x7fb26c0a5210 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:22.065+0000 7fb289ebd640 1 --2- 172.21.15.7:0/1713268612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2841035d0 0x7fb284116380 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:22.065+0000 7fb289ebd640 1 --2- 172.21.15.7:0/1713268612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb284102c10 0x7fb284111320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:22.065+0000 7fb289ebd640 1 --2- 172.21.15.7:0/1713268612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb284101a20 0x7fb284110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:22.065+0000 7fb289ebd640 1 -- 172.21.15.7:0/1713268612 >> 172.21.15.7:0/1713268612 conn(0x7fb2840fcf90 msgr2=0x7fb284105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:22.065+0000 7fb289ebd640 1 -- 172.21.15.7:0/1713268612 shutdown_connections 2024-06-10T01:09:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:22.065+0000 7fb289ebd640 1 -- 172.21.15.7:0/1713268612 wait complete. 2024-06-10T01:09:22.527 INFO:tasks.cephfs.filesystem:scrub status for tag:76fc0745-2b6a-4b68-a0c2-9e866ba0b0da - {'path': '/', 'tag': '76fc0745-2b6a-4b68-a0c2-9e866ba0b0da', 'options': 'recursive,force'} 2024-06-10T01:09:22.713 INFO:tasks.workunit.client.0.smithi007.stdout: 1 180369 10.57 MB/sec execute 3 sec latency 101.458 ms 2024-06-10T01:09:23.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:23 smithi078 ceph-mon[36683]: pgmap v533: 129 pgs: 129 active+clean; 53 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 300 op/s 2024-06-10T01:09:23.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:23 smithi175 ceph-mon[36636]: pgmap v533: 129 pgs: 129 active+clean; 53 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 300 op/s 2024-06-10T01:09:23.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:23 smithi007 ceph-mon[32398]: pgmap v533: 129 pgs: 129 active+clean; 53 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 300 op/s 2024-06-10T01:09:23.713 INFO:tasks.workunit.client.0.smithi007.stdout: 1 181980 10.05 MB/sec execute 4 sec latency 13.329 ms 2024-06-10T01:09:24.713 INFO:tasks.workunit.client.0.smithi007.stdout: 1 184672 11.41 MB/sec execute 5 sec latency 22.627 ms 2024-06-10T01:09:25.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:25 smithi078 ceph-mon[36683]: pgmap v534: 129 pgs: 129 active+clean; 52 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 364 op/s 2024-06-10T01:09:25.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:25 smithi175 ceph-mon[36636]: pgmap v534: 129 pgs: 129 active+clean; 52 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 364 op/s 2024-06-10T01:09:25.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:25 smithi007 ceph-mon[32398]: pgmap v534: 129 pgs: 129 active+clean; 52 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 364 op/s 2024-06-10T01:09:25.713 INFO:tasks.workunit.client.0.smithi007.stdout: 1 186105 10.44 MB/sec execute 6 sec latency 14.706 ms 2024-06-10T01:09:26.713 INFO:tasks.workunit.client.0.smithi007.stdout: 1 188377 10.93 MB/sec execute 7 sec latency 43.408 ms 2024-06-10T01:09:27.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:27 smithi078 ceph-mon[36683]: pgmap v535: 129 pgs: 129 active+clean; 48 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 376 op/s 2024-06-10T01:09:27.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:27 smithi175 ceph-mon[36636]: pgmap v535: 129 pgs: 129 active+clean; 48 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 376 op/s 2024-06-10T01:09:27.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:27 smithi007 ceph-mon[32398]: pgmap v535: 129 pgs: 129 active+clean; 48 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 376 op/s 2024-06-10T01:09:27.713 INFO:tasks.workunit.client.0.smithi007.stdout: 1 190361 10.43 MB/sec execute 8 sec latency 14.138 ms 2024-06-10T01:09:28.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:28 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:09:28.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:28 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:09:28.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:28 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:09:28.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:28 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:09:28.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:28 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:09:28.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:28 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:09:28.713 INFO:tasks.workunit.client.0.smithi007.stdout: 1 192122 10.62 MB/sec execute 9 sec latency 11.202 ms 2024-06-10T01:09:29.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:29 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:09:29.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:29 smithi078 ceph-mon[36683]: pgmap v536: 129 pgs: 129 active+clean; 48 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 352 op/s 2024-06-10T01:09:29.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:29 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:09:29.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:29 smithi175 ceph-mon[36636]: pgmap v536: 129 pgs: 129 active+clean; 48 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 352 op/s 2024-06-10T01:09:29.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:29 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:09:29.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:29 smithi007 ceph-mon[32398]: pgmap v536: 129 pgs: 129 active+clean; 48 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 352 op/s 2024-06-10T01:09:29.713 INFO:tasks.workunit.client.0.smithi007.stdout: 1 194754 10.89 MB/sec execute 10 sec latency 25.321 ms 2024-06-10T01:09:30.713 INFO:tasks.workunit.client.0.smithi007.stdout: 1 196239 10.61 MB/sec execute 11 sec latency 11.949 ms 2024-06-10T01:09:31.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:31 smithi007 ceph-mon[32398]: pgmap v537: 129 pgs: 129 active+clean; 51 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 424 op/s 2024-06-10T01:09:31.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:31 smithi078 ceph-mon[36683]: pgmap v537: 129 pgs: 129 active+clean; 51 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 424 op/s 2024-06-10T01:09:31.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:31 smithi175 ceph-mon[36636]: pgmap v537: 129 pgs: 129 active+clean; 51 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 424 op/s 2024-06-10T01:09:31.713 INFO:tasks.workunit.client.0.smithi007.stdout: 1 198636 11.02 MB/sec execute 12 sec latency 52.608 ms 2024-06-10T01:09:32.713 INFO:tasks.workunit.client.0.smithi007.stdout: 1 199847 10.46 MB/sec execute 13 sec latency 50.074 ms 2024-06-10T01:09:33.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:33 smithi078 ceph-mon[36683]: pgmap v538: 129 pgs: 129 active+clean; 51 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 336 op/s 2024-06-10T01:09:33.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:33 smithi175 ceph-mon[36636]: pgmap v538: 129 pgs: 129 active+clean; 51 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 336 op/s 2024-06-10T01:09:33.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:33 smithi007 ceph-mon[32398]: pgmap v538: 129 pgs: 129 active+clean; 51 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 336 op/s 2024-06-10T01:09:33.713 INFO:tasks.workunit.client.0.smithi007.stdout: 1 202507 10.90 MB/sec execute 14 sec latency 12.262 ms 2024-06-10T01:09:34.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:34 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:09:34.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:34 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:09:34.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:34 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:09:34.713 INFO:tasks.workunit.client.0.smithi007.stdout: 1 204012 10.56 MB/sec execute 15 sec latency 17.311 ms 2024-06-10T01:09:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.166+0000 7f9fcb7ed640 1 -- 172.21.15.7:0/741764446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fc4101550 msgr2=0x7f9fc4108df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.166+0000 7f9fcb7ed640 1 --2- 172.21.15.7:0/741764446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fc4101550 0x7f9fc4108df0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc009ca0 tx=0x7f9fbc02a8c0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.166+0000 7f9fcb7ed640 1 -- 172.21.15.7:0/741764446 shutdown_connections 2024-06-10T01:09:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.166+0000 7f9fcb7ed640 1 --2- 172.21.15.7:0/741764446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fc4101550 0x7f9fc4108df0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.166+0000 7f9fcb7ed640 1 --2- 172.21.15.7:0/741764446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fc4100b90 0x7f9fc4101010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.166+0000 7f9fcb7ed640 1 --2- 172.21.15.7:0/741764446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fc4101f40 0x7f9fc4102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.166+0000 7f9fcb7ed640 1 -- 172.21.15.7:0/741764446 >> 172.21.15.7:0/741764446 conn(0x7f9fc40fb620 msgr2=0x7f9fc40fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.166+0000 7f9fcb7ed640 1 -- 172.21.15.7:0/741764446 shutdown_connections 2024-06-10T01:09:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.166+0000 7f9fcb7ed640 1 -- 172.21.15.7:0/741764446 wait complete. 2024-06-10T01:09:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.167+0000 7f9fcb7ed640 1 Processor -- start 2024-06-10T01:09:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.167+0000 7f9fcb7ed640 1 -- start start 2024-06-10T01:09:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.167+0000 7f9fcb7ed640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fc4100b90 0x7f9fc4104940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.167+0000 7f9fcb7ed640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fc4101550 0x7f9fc4102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.167+0000 7f9fcb7ed640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fc4101f40 0x7f9fc41034c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.167+0000 7f9fcb7ed640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc410c5a0 con 0x7f9fc4101550 2024-06-10T01:09:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.167+0000 7f9fcb7ed640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc410c420 con 0x7f9fc4100b90 2024-06-10T01:09:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.167+0000 7f9fcb7ed640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc410c720 con 0x7f9fc4101f40 2024-06-10T01:09:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.167+0000 7f9fc8d88640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fc4101550 0x7f9fc4102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.167+0000 7f9fc8d88640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fc4101550 0x7f9fc4102f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35152/0 (socket says 172.21.15.7:35152) 2024-06-10T01:09:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.167+0000 7f9fc8d88640 1 -- 172.21.15.7:0/3605664556 learned_addr learned my addr 172.21.15.7:0/3605664556 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.167+0000 7f9fc9589640 1 --2- 172.21.15.7:0/3605664556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fc4100b90 0x7f9fc4104940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.167+0000 7f9fc9d8a640 1 --2- 172.21.15.7:0/3605664556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fc4101f40 0x7f9fc41034c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.167+0000 7f9fc8d88640 1 -- 172.21.15.7:0/3605664556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fc4101f40 msgr2=0x7f9fc41034c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.167+0000 7f9fc8d88640 1 --2- 172.21.15.7:0/3605664556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fc4101f40 0x7f9fc41034c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.167+0000 7f9fc8d88640 1 -- 172.21.15.7:0/3605664556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fc4100b90 msgr2=0x7f9fc4104940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.167+0000 7f9fc8d88640 1 --2- 172.21.15.7:0/3605664556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fc4100b90 0x7f9fc4104940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.167+0000 7f9fc8d88640 1 -- 172.21.15.7:0/3605664556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc4103d50 con 0x7f9fc4101550 2024-06-10T01:09:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.167+0000 7f9fc8d88640 1 --2- 172.21.15.7:0/3605664556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fc4101550 0x7f9fc4102f80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f9fb400e9c0 tx=0x7f9fb400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.168+0000 7f9fba7fc640 1 -- 172.21.15.7:0/3605664556 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb400cd80 con 0x7f9fc4101550 2024-06-10T01:09:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.168+0000 7f9fba7fc640 1 -- 172.21.15.7:0/3605664556 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb40035f0 con 0x7f9fc4101550 2024-06-10T01:09:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.168+0000 7f9fcb7ed640 1 -- 172.21.15.7:0/3605664556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc41a1e90 con 0x7f9fc4101550 2024-06-10T01:09:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.168+0000 7f9fba7fc640 1 -- 172.21.15.7:0/3605664556 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4010690 con 0x7f9fc4101550 2024-06-10T01:09:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.168+0000 7f9fcb7ed640 1 -- 172.21.15.7:0/3605664556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc41a22f0 con 0x7f9fc4101550 2024-06-10T01:09:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.170+0000 7f9fba7fc640 1 -- 172.21.15.7:0/3605664556 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9fb40037a0 con 0x7f9fc4101550 2024-06-10T01:09:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.171+0000 7f9fce5ce740 1 Processor -- start 2024-06-10T01:09:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.171+0000 7f9fce5ce740 1 -- start start 2024-06-10T01:09:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.171+0000 7f9fce5ce740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5555876a2010 0x5555876a2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.171+0000 7f9fce5ce740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5555876a2a00 0x5555876a7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.171+0000 7f9fce5ce740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5555876a7b70 0x5555876aa000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.171+0000 7f9fce5ce740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5555876054f0 con 0x5555876a2010 2024-06-10T01:09:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.171+0000 7f9fce5ce740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5555875a3870 con 0x5555876a2a00 2024-06-10T01:09:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.171+0000 7f9fce5ce740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5555876014b0 con 0x5555876a7b70 2024-06-10T01:09:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.171+0000 7f9fba7fc640 1 --2- 172.21.15.7:0/3605664556 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9fb00a2b80 0x7f9fb00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.171+0000 7f9fc9d8a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5555876a2010 0x5555876a2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.171+0000 7f9fc9589640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5555876a2a00 0x5555876a7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.171+0000 7f9fc9589640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5555876a2a00 0x5555876a7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:47942/0 (socket says 172.21.15.7:47942) 2024-06-10T01:09:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.171+0000 7f9fc9589640 1 -- 172.21.15.7:0/707173557 learned_addr learned my addr 172.21.15.7:0/707173557 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.171+0000 7f9fc8d88640 1 --2- 172.21.15.7:0/707173557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5555876a7b70 0x5555876aa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.171+0000 7f9fc9d8a640 1 --2- 172.21.15.7:0/3605664556 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9fb00a2b80 0x7f9fb00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.171+0000 7f9fba7fc640 1 -- 172.21.15.7:0/3605664556 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f9fb40cf720 con 0x7f9fc4101550 2024-06-10T01:09:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.171+0000 7f9fc9589640 1 -- 172.21.15.7:0/707173557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5555876a7b70 msgr2=0x5555876aa000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.171+0000 7f9fc9589640 1 --2- 172.21.15.7:0/707173557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5555876a7b70 0x5555876aa000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.171+0000 7f9fc9589640 1 -- 172.21.15.7:0/707173557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5555876a2010 msgr2=0x5555876a2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.171+0000 7f9fc9589640 1 --2- 172.21.15.7:0/707173557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5555876a2010 0x5555876a2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.171+0000 7f9fc9589640 1 -- 172.21.15.7:0/707173557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555875bf9b0 con 0x5555876a2a00 2024-06-10T01:09:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.171+0000 7f9fc9d8a640 1 --2- 172.21.15.7:0/707173557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5555876a2010 0x5555876a2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:09:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.171+0000 7f9fc8d88640 1 --2- 172.21.15.7:0/707173557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5555876a7b70 0x5555876aa000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:09:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.172+0000 7f9fc9589640 1 --2- 172.21.15.7:0/707173557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5555876a2a00 0x5555876a7500 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0009c80 tx=0x7f9fc0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de52b4cf967228aa server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.172+0000 7f9fc9d8a640 1 --2- 172.21.15.7:0/3605664556 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9fb00a2b80 0x7f9fb00a5040 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc005cc0 tx=0x7f9fbc02a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.172+0000 7f9faf7fe640 1 -- 172.21.15.7:0/707173557 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc001e070 con 0x5555876a2a00 2024-06-10T01:09:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.172+0000 7f9faf7fe640 1 -- 172.21.15.7:0/707173557 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fc0009e30 con 0x5555876a2a00 2024-06-10T01:09:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.172+0000 7f9faf7fe640 1 -- 172.21.15.7:0/707173557 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc000ea20 con 0x5555876a2a00 2024-06-10T01:09:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.173+0000 7f9fce5ce740 1 -- 172.21.15.7:0/707173557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5555876a2a00 msgr2=0x5555876a7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.173+0000 7f9fce5ce740 1 --2- 172.21.15.7:0/707173557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5555876a2a00 0x5555876a7500 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0009c80 tx=0x7f9fc0011810 comp rx=0 tx=0).stop 2024-06-10T01:09:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.173+0000 7f9fce5ce740 1 -- 172.21.15.7:0/707173557 shutdown_connections 2024-06-10T01:09:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.173+0000 7f9fce5ce740 1 --2- 172.21.15.7:0/707173557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5555876a7b70 0x5555876aa000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.173+0000 7f9fce5ce740 1 --2- 172.21.15.7:0/707173557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5555876a2a00 0x5555876a7500 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.173+0000 7f9fce5ce740 1 --2- 172.21.15.7:0/707173557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5555876a2010 0x5555876a2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.173+0000 7f9fce5ce740 1 -- 172.21.15.7:0/707173557 >> 172.21.15.7:0/707173557 conn(0x55558769d680 msgr2=0x55558769da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.173+0000 7f9fce5ce740 1 -- 172.21.15.7:0/707173557 shutdown_connections 2024-06-10T01:09:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.173+0000 7f9fce5ce740 1 -- 172.21.15.7:0/707173557 wait complete. 2024-06-10T01:09:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.173+0000 7f9fce5ce740 1 -- start start 2024-06-10T01:09:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.173+0000 7f9fce5ce740 1 Processor -- start 2024-06-10T01:09:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.173+0000 7f9fce5ce740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5555876a2010 0x5555876ac2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.173+0000 7f9fce5ce740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5555876a2a00 0x5555876ac7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.173+0000 7f9fce5ce740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5555876a7b70 0x5555876b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.173+0000 7f9fce5ce740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5555876aab20 con 0x5555876a7b70 2024-06-10T01:09:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.173+0000 7f9fce5ce740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55558758eae0 con 0x5555876a2a00 2024-06-10T01:09:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.173+0000 7f9fce5ce740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5555876aaca0 con 0x5555876a2010 2024-06-10T01:09:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.174+0000 7f9fce5ce740 10 client.0 _ll_register_callbacks cb 0x5555875c79b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.174+0000 7f9fc9d8a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5555876a7b70 0x5555876b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.174+0000 7f9fc8d88640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5555876a2a00 0x5555876ac7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.174+0000 7f9fc8d88640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5555876a2a00 0x5555876ac7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:47946/0 (socket says 172.21.15.7:47946) 2024-06-10T01:09:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.174+0000 7f9fc9d8a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5555876a7b70 0x5555876b3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35178/0 (socket says 172.21.15.7:35178) 2024-06-10T01:09:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.174+0000 7f9fc8d88640 1 -- 172.21.15.7:0/1779704473 learned_addr learned my addr 172.21.15.7:0/1779704473 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.174+0000 7f9fc9589640 1 --2- 172.21.15.7:0/1779704473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5555876a2010 0x5555876ac2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.174+0000 7f9fc8d88640 1 -- 172.21.15.7:0/1779704473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5555876a2010 msgr2=0x5555876ac2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.174+0000 7f9fc8d88640 1 --2- 172.21.15.7:0/1779704473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5555876a2010 0x5555876ac2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.174+0000 7f9fc8d88640 1 -- 172.21.15.7:0/1779704473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5555876a7b70 msgr2=0x5555876b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.174+0000 7f9fc8d88640 1 --2- 172.21.15.7:0/1779704473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5555876a7b70 0x5555876b3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.174+0000 7f9fc8d88640 1 -- 172.21.15.7:0/1779704473 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555587319210 con 0x5555876a2a00 2024-06-10T01:09:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.174+0000 7f9fc9d8a640 1 --2- 172.21.15.7:0/1779704473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5555876a7b70 0x5555876b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.174+0000 7f9fc8d88640 1 --2- 172.21.15.7:0/1779704473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5555876a2a00 0x5555876ac7f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4098060 tx=0x7f9fb409b560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5c0fdd60bb14d9b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.174+0000 7f9facff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:09:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.174+0000 7f9facff9640 1 -- 172.21.15.7:0/1779704473 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb40b2030 con 0x5555876a2a00 2024-06-10T01:09:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.174+0000 7f9facff9640 1 -- 172.21.15.7:0/1779704473 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f9fb40b3070 con 0x5555876a2a00 2024-06-10T01:09:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.174+0000 7f9fce5ce740 1 -- 172.21.15.7:0/1779704473 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fc0009e30 con 0x5555876a2a00 2024-06-10T01:09:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.175+0000 7f9facff9640 1 -- 172.21.15.7:0/1779704473 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9fb409ffb0 con 0x5555876a2a00 2024-06-10T01:09:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.175+0000 7f9fce5ce740 10 client.25934 fetch_fsmap learned FSMap version 30 2024-06-10T01:09:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.175+0000 7f9fce5ce740 1 -- 172.21.15.7:0/1779704473 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5555873e5620 con 0x5555876a2a00 2024-06-10T01:09:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.175+0000 7f9facff9640 1 -- 172.21.15.7:0/1779704473 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fc0009e30 con 0x5555876a2a00 2024-06-10T01:09:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.175+0000 7f9facff9640 1 -- 172.21.15.7:0/1779704473 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f9fb40cfbd0 con 0x5555876a2a00 2024-06-10T01:09:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.175+0000 7f9fce5ce740 10 client.25934 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:09:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.175+0000 7f9fce5ce740 10 client.25934 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.175+0000 7f9fce5ce740 20 client.25934 populate_metadata read hostname 'smithi007' 2024-06-10T01:09:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.175+0000 7f9fce5ce740 1 --2- 172.21.15.7:0/1779704473 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5555876b8110 0x5555876ba540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.175+0000 7f9fce5ce740 4 client.25934 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:09:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.175+0000 7f9fce5ce740 1 -- 172.21.15.7:0/1779704473 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555587618f10 con 0x5555876b8110 2024-06-10T01:09:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.175+0000 7f9fc9589640 1 --2- 172.21.15.7:0/1779704473 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5555876b8110 0x5555876ba540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.175+0000 7f9facff9640 10 client.25934 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.175+0000 7f9fc9589640 1 --2- 172.21.15.7:0/1779704473 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5555876b8110 0x5555876ba540 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d846a7d0ac4fe3c0 server_cookie=ef989cf2da717f25 in_seq=0 out_seq=0 2024-06-10T01:09:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.176+0000 7f9facff9640 1 -- 172.21.15.7:0/1779704473 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555587618f10 con 0x5555876b8110 2024-06-10T01:09:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.176+0000 7f9facff9640 10 client.25934 handle_command_reply: tid=0 2024-06-10T01:09:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.176+0000 7f9fce5ce740 1 client.25934 shutdown 2024-06-10T01:09:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.176+0000 7f9fce5ce740 1 -- 172.21.15.7:0/1779704473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5555876a2a00 msgr2=0x5555876ac7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.176+0000 7f9fce5ce740 1 --2- 172.21.15.7:0/1779704473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5555876a2a00 0x5555876ac7f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4098060 tx=0x7f9fb409b560 comp rx=0 tx=0).stop 2024-06-10T01:09:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.176+0000 7f9fce5ce740 1 -- 172.21.15.7:0/1779704473 shutdown_connections 2024-06-10T01:09:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.176+0000 7f9fce5ce740 1 --2- 172.21.15.7:0/1779704473 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5555876b8110 0x5555876ba540 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.176+0000 7f9fce5ce740 1 --2- 172.21.15.7:0/1779704473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5555876a7b70 0x5555876b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.176+0000 7f9fce5ce740 1 --2- 172.21.15.7:0/1779704473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5555876a2a00 0x5555876ac7f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.176+0000 7f9fce5ce740 1 --2- 172.21.15.7:0/1779704473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5555876a2010 0x5555876ac2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.176+0000 7f9facff9640 0 client.25934 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.176+0000 7f9fce5ce740 1 -- 172.21.15.7:0/1779704473 >> 172.21.15.7:0/1779704473 conn(0x55558769d680 msgr2=0x55558769c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.176+0000 7f9fce5ce740 1 -- 172.21.15.7:0/1779704473 shutdown_connections 2024-06-10T01:09:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.177+0000 7f9fce5ce740 1 -- 172.21.15.7:0/1779704473 wait complete. 2024-06-10T01:09:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.177+0000 7f9fce5ce740 20 client.25934 trim_cache size 0 max 16384 2024-06-10T01:09:35.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.192+0000 7f9fce5ce740 1 Processor -- start 2024-06-10T01:09:35.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.192+0000 7f9fce5ce740 1 -- start start 2024-06-10T01:09:35.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.192+0000 7f9fce5ce740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5555876a2010 0x5555876a3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:35.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.192+0000 7f9fce5ce740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5555876a2a00 0x5555876b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:35.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.192+0000 7f9fce5ce740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5555876a7b70 0x5555876a4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:35.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.192+0000 7f9fce5ce740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc41af210 con 0x5555876a7b70 2024-06-10T01:09:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.192+0000 7f9fce5ce740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc41aca80 con 0x5555876a2a00 2024-06-10T01:09:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.192+0000 7f9fce5ce740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc41aa8d0 con 0x5555876a2010 2024-06-10T01:09:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.193+0000 7f9fc9d8a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5555876a7b70 0x5555876a4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.193+0000 7f9fc8d88640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5555876a2a00 0x5555876b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.193+0000 7f9fc9d8a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5555876a7b70 0x5555876a4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35186/0 (socket says 172.21.15.7:35186) 2024-06-10T01:09:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.193+0000 7f9fc9d8a640 1 -- 172.21.15.7:0/953202617 learned_addr learned my addr 172.21.15.7:0/953202617 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.193+0000 7f9fc9589640 1 --2- 172.21.15.7:0/953202617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5555876a2010 0x5555876a3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.193+0000 7f9fc9d8a640 1 -- 172.21.15.7:0/953202617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5555876a2010 msgr2=0x5555876a3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.193+0000 7f9fc9d8a640 1 --2- 172.21.15.7:0/953202617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5555876a2010 0x5555876a3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.193+0000 7f9fc9d8a640 1 -- 172.21.15.7:0/953202617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5555876a2a00 msgr2=0x5555876b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.193+0000 7f9fc9d8a640 1 --2- 172.21.15.7:0/953202617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5555876a2a00 0x5555876b37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.193+0000 7f9fc9d8a640 1 -- 172.21.15.7:0/953202617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555876a36d0 con 0x5555876a7b70 2024-06-10T01:09:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.193+0000 7f9fc8d88640 1 --2- 172.21.15.7:0/953202617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5555876a2a00 0x5555876b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.193+0000 7f9fc9d8a640 1 --2- 172.21.15.7:0/953202617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5555876a7b70 0x5555876a4120 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc035000 tx=0x7f9fbc005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ec1612d98d98e0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.193+0000 7f9faeffd640 1 -- 172.21.15.7:0/953202617 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc031760 con 0x5555876a7b70 2024-06-10T01:09:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.193+0000 7f9faeffd640 1 -- 172.21.15.7:0/953202617 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fbc036040 con 0x5555876a7b70 2024-06-10T01:09:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.193+0000 7f9faeffd640 1 -- 172.21.15.7:0/953202617 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc02fe60 con 0x5555876a7b70 2024-06-10T01:09:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.194+0000 7f9fce5ce740 1 -- 172.21.15.7:0/953202617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5555876a7b70 msgr2=0x5555876a4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.194+0000 7f9fce5ce740 1 --2- 172.21.15.7:0/953202617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5555876a7b70 0x5555876a4120 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc035000 tx=0x7f9fbc005d80 comp rx=0 tx=0).stop 2024-06-10T01:09:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.194+0000 7f9fce5ce740 1 -- 172.21.15.7:0/953202617 shutdown_connections 2024-06-10T01:09:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.194+0000 7f9fce5ce740 1 --2- 172.21.15.7:0/953202617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5555876a7b70 0x5555876a4120 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.194+0000 7f9fce5ce740 1 --2- 172.21.15.7:0/953202617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5555876a2a00 0x5555876b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.194+0000 7f9fce5ce740 1 --2- 172.21.15.7:0/953202617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5555876a2010 0x5555876a3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.194+0000 7f9fce5ce740 1 -- 172.21.15.7:0/953202617 >> 172.21.15.7:0/953202617 conn(0x55558769d680 msgr2=0x5555876ac2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.194+0000 7f9fce5ce740 1 -- 172.21.15.7:0/953202617 shutdown_connections 2024-06-10T01:09:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.194+0000 7f9fce5ce740 1 -- 172.21.15.7:0/953202617 wait complete. 2024-06-10T01:09:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.194+0000 7f9fce5ce740 1 -- start start 2024-06-10T01:09:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.194+0000 7f9fce5ce740 1 Processor -- start 2024-06-10T01:09:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.195+0000 7f9fce5ce740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5555876a2010 0x5555876b3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.195+0000 7f9fce5ce740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5555876a2a00 0x5555876b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.195+0000 7f9fce5ce740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5555876a7b70 0x5555876b0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.195+0000 7f9fce5ce740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc41aaf40 con 0x5555876a2a00 2024-06-10T01:09:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.195+0000 7f9fce5ce740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc41ac3b0 con 0x5555876a2010 2024-06-10T01:09:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.195+0000 7f9fce5ce740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc41ab610 con 0x5555876a7b70 2024-06-10T01:09:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.195+0000 7f9fce5ce740 10 client.0 _ll_register_callbacks cb 0x7f9fb40046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.195+0000 7f9fc8d88640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5555876a2a00 0x5555876b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.195+0000 7f9fc8d88640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5555876a2a00 0x5555876b02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35196/0 (socket says 172.21.15.7:35196) 2024-06-10T01:09:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.195+0000 7f9fc8d88640 1 -- 172.21.15.7:0/3822152684 learned_addr learned my addr 172.21.15.7:0/3822152684 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.195+0000 7f9fc9589640 1 --2- 172.21.15.7:0/3822152684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5555876a2010 0x5555876b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.195+0000 7f9fc9d8a640 1 --2- 172.21.15.7:0/3822152684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5555876a7b70 0x5555876b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.195+0000 7f9fc8d88640 1 -- 172.21.15.7:0/3822152684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5555876a7b70 msgr2=0x5555876b0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.195+0000 7f9fc8d88640 1 --2- 172.21.15.7:0/3822152684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5555876a7b70 0x5555876b0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.195+0000 7f9fc8d88640 1 -- 172.21.15.7:0/3822152684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5555876a2010 msgr2=0x5555876b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.195+0000 7f9fc8d88640 1 --2- 172.21.15.7:0/3822152684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5555876a2010 0x5555876b3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.195+0000 7f9fc8d88640 1 -- 172.21.15.7:0/3822152684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555587720e40 con 0x5555876a2a00 2024-06-10T01:09:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.195+0000 7f9fc8d88640 1 --2- 172.21.15.7:0/3822152684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5555876a2a00 0x5555876b02e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40058c0 tx=0x7f9fb409b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c5f3462ebb3d4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.195+0000 7f9facff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:09:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.195+0000 7f9facff9640 1 -- 172.21.15.7:0/3822152684 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb40b2070 con 0x5555876a2a00 2024-06-10T01:09:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.196+0000 7f9facff9640 1 -- 172.21.15.7:0/3822152684 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f9fb4003b30 con 0x5555876a2a00 2024-06-10T01:09:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.196+0000 7f9fce5ce740 1 -- 172.21.15.7:0/3822152684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fbc036040 con 0x5555876a2a00 2024-06-10T01:09:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.196+0000 7f9facff9640 1 -- 172.21.15.7:0/3822152684 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9fb409b9d0 con 0x5555876a2a00 2024-06-10T01:09:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.196+0000 7f9fce5ce740 10 client.16419 fetch_fsmap learned FSMap version 30 2024-06-10T01:09:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.196+0000 7f9fce5ce740 1 -- 172.21.15.7:0/3822152684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5555877221a0 con 0x5555876a2a00 2024-06-10T01:09:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.196+0000 7f9facff9640 1 -- 172.21.15.7:0/3822152684 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fbc036040 con 0x5555876a2a00 2024-06-10T01:09:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.196+0000 7f9facff9640 1 -- 172.21.15.7:0/3822152684 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f9fb409b170 con 0x5555876a2a00 2024-06-10T01:09:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.196+0000 7f9fce5ce740 10 client.16419 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:09:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.196+0000 7f9fce5ce740 10 client.16419 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.196+0000 7f9fce5ce740 20 client.16419 populate_metadata read hostname 'smithi007' 2024-06-10T01:09:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.196+0000 7f9fce5ce740 1 --2- 172.21.15.7:0/3822152684 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5555877229f0 0x555587724e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.196+0000 7f9fce5ce740 4 client.16419 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:09:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.196+0000 7f9fce5ce740 1 -- 172.21.15.7:0/3822152684 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555587612c20 con 0x5555877229f0 2024-06-10T01:09:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.196+0000 7f9fc9589640 1 --2- 172.21.15.7:0/3822152684 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5555877229f0 0x555587724e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.197+0000 7f9facff9640 10 client.16419 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.197+0000 7f9fc9589640 1 --2- 172.21.15.7:0/3822152684 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5555877229f0 0x555587724e10 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da5a95bdf8d2f363 server_cookie=3c81ca1a67f0bc86 in_seq=0 out_seq=0 2024-06-10T01:09:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.299+0000 7f9facff9640 1 -- 172.21.15.7:0/3822152684 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555587612c20 con 0x5555877229f0 2024-06-10T01:09:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.299+0000 7f9facff9640 10 client.16419 handle_command_reply: tid=0 2024-06-10T01:09:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.299+0000 7f9fce5ce740 1 client.16419 shutdown 2024-06-10T01:09:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.299+0000 7f9fce5ce740 1 -- 172.21.15.7:0/3822152684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5555876a2a00 msgr2=0x5555876b02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.299+0000 7f9fce5ce740 1 --2- 172.21.15.7:0/3822152684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5555876a2a00 0x5555876b02e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40058c0 tx=0x7f9fb409b690 comp rx=0 tx=0).stop 2024-06-10T01:09:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.299+0000 7f9fce5ce740 1 -- 172.21.15.7:0/3822152684 shutdown_connections 2024-06-10T01:09:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.299+0000 7f9fce5ce740 1 --2- 172.21.15.7:0/3822152684 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5555877229f0 0x555587724e10 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.299+0000 7f9fce5ce740 1 --2- 172.21.15.7:0/3822152684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5555876a7b70 0x5555876b0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.299+0000 7f9fce5ce740 1 --2- 172.21.15.7:0/3822152684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5555876a2a00 0x5555876b02e0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.299+0000 7f9fce5ce740 1 --2- 172.21.15.7:0/3822152684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5555876a2010 0x5555876b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.299+0000 7f9facff9640 0 client.16419 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.299+0000 7f9fce5ce740 1 -- 172.21.15.7:0/3822152684 >> 172.21.15.7:0/3822152684 conn(0x55558769d680 msgr2=0x5555876ac2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.299+0000 7f9fce5ce740 1 -- 172.21.15.7:0/3822152684 shutdown_connections 2024-06-10T01:09:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.299+0000 7f9fce5ce740 1 -- 172.21.15.7:0/3822152684 wait complete. 2024-06-10T01:09:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.299+0000 7f9fce5ce740 20 client.16419 trim_cache size 0 max 16384 2024-06-10T01:09:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:09:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:09:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:09:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.300+0000 7f9fcb7ed640 1 -- 172.21.15.7:0/3605664556 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9fb00a2b80 msgr2=0x7f9fb00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.300+0000 7f9fcb7ed640 1 --2- 172.21.15.7:0/3605664556 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9fb00a2b80 0x7f9fb00a5040 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc005cc0 tx=0x7f9fbc02a8c0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.300+0000 7f9fcb7ed640 1 -- 172.21.15.7:0/3605664556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fc4101550 msgr2=0x7f9fc4102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:35.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.300+0000 7f9fcb7ed640 1 --2- 172.21.15.7:0/3605664556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fc4101550 0x7f9fc4102f80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f9fb400e9c0 tx=0x7f9fb400ee80 comp rx=0 tx=0).stop 2024-06-10T01:09:35.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.300+0000 7f9fcb7ed640 1 -- 172.21.15.7:0/3605664556 shutdown_connections 2024-06-10T01:09:35.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.300+0000 7f9fcb7ed640 1 --2- 172.21.15.7:0/3605664556 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9fb00a2b80 0x7f9fb00a5040 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.300+0000 7f9fcb7ed640 1 --2- 172.21.15.7:0/3605664556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fc4101f40 0x7f9fc41034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.300+0000 7f9fcb7ed640 1 --2- 172.21.15.7:0/3605664556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fc4101550 0x7f9fc4102f80 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.300+0000 7f9fcb7ed640 1 --2- 172.21.15.7:0/3605664556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fc4100b90 0x7f9fc4104940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:35.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.300+0000 7f9fcb7ed640 1 -- 172.21.15.7:0/3605664556 >> 172.21.15.7:0/3605664556 conn(0x7f9fc40fb620 msgr2=0x7f9fc40fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:35.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.300+0000 7f9fcb7ed640 1 -- 172.21.15.7:0/3605664556 shutdown_connections 2024-06-10T01:09:35.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:35.300+0000 7f9fcb7ed640 1 -- 172.21.15.7:0/3605664556 wait complete. 2024-06-10T01:09:35.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:35 smithi078 ceph-mon[36683]: pgmap v539: 129 pgs: 129 active+clean; 53 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 356 op/s 2024-06-10T01:09:35.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:35 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:09:35.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:35 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:09:35.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:35 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:09:35.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:35 smithi175 ceph-mon[36636]: pgmap v539: 129 pgs: 129 active+clean; 53 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 356 op/s 2024-06-10T01:09:35.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:35 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:09:35.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:35 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:09:35.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:35 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:09:35.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:35 smithi007 ceph-mon[32398]: pgmap v539: 129 pgs: 129 active+clean; 53 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 356 op/s 2024-06-10T01:09:35.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:35 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:09:35.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:35 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:09:35.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:35 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:09:35.713 INFO:tasks.workunit.client.0.smithi007.stdout: 1 206139 10.75 MB/sec execute 16 sec latency 89.690 ms 2024-06-10T01:09:36.713 INFO:tasks.workunit.client.0.smithi007.stdout: 1 207708 10.45 MB/sec execute 17 sec latency 78.786 ms 2024-06-10T01:09:37.579 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:37 smithi078 ceph-mon[36683]: pgmap v540: 129 pgs: 129 active+clean; 54 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 346 op/s 2024-06-10T01:09:37.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:37 smithi175 ceph-mon[36636]: pgmap v540: 129 pgs: 129 active+clean; 54 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 346 op/s 2024-06-10T01:09:37.713 INFO:tasks.workunit.client.0.smithi007.stdout: 1 209802 10.62 MB/sec execute 18 sec latency 20.239 ms 2024-06-10T01:09:37.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:37 smithi007 ceph-mon[32398]: pgmap v540: 129 pgs: 129 active+clean; 54 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 346 op/s 2024-06-10T01:09:38.713 INFO:tasks.workunit.client.0.smithi007.stdout: 1 212298 10.73 MB/sec execute 19 sec latency 15.147 ms 2024-06-10T01:09:39.579 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:39 smithi078 ceph-mon[36683]: pgmap v541: 129 pgs: 129 active+clean; 55 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 320 op/s 2024-06-10T01:09:39.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:39 smithi175 ceph-mon[36636]: pgmap v541: 129 pgs: 129 active+clean; 55 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 320 op/s 2024-06-10T01:09:39.713 INFO:tasks.workunit.client.0.smithi007.stdout: 1 213974 10.62 MB/sec execute 20 sec latency 13.190 ms 2024-06-10T01:09:39.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:39 smithi007 ceph-mon[32398]: pgmap v541: 129 pgs: 129 active+clean; 55 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 320 op/s 2024-06-10T01:09:40.713 INFO:tasks.workunit.client.0.smithi007.stdout: 1 216683 10.91 MB/sec execute 21 sec latency 13.314 ms 2024-06-10T01:09:41.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:41 smithi078 ceph-mon[36683]: pgmap v542: 129 pgs: 129 active+clean; 54 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 408 op/s 2024-06-10T01:09:41.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:41 smithi175 ceph-mon[36636]: pgmap v542: 129 pgs: 129 active+clean; 54 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 408 op/s 2024-06-10T01:09:41.713 INFO:tasks.workunit.client.0.smithi007.stdout: 1 218060 10.67 MB/sec execute 22 sec latency 82.389 ms 2024-06-10T01:09:41.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:41 smithi007 ceph-mon[32398]: pgmap v542: 129 pgs: 129 active+clean; 54 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 408 op/s 2024-06-10T01:09:42.714 INFO:tasks.workunit.client.0.smithi007.stdout: 1 220395 10.81 MB/sec execute 23 sec latency 18.102 ms 2024-06-10T01:09:43.516 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:43 smithi007 ceph-mon[32398]: pgmap v543: 129 pgs: 129 active+clean; 54 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 334 op/s 2024-06-10T01:09:43.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:43 smithi078 ceph-mon[36683]: pgmap v543: 129 pgs: 129 active+clean; 54 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 334 op/s 2024-06-10T01:09:43.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:43 smithi175 ceph-mon[36636]: pgmap v543: 129 pgs: 129 active+clean; 54 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 334 op/s 2024-06-10T01:09:43.714 INFO:tasks.workunit.client.0.smithi007.stdout: 1 222749 10.82 MB/sec execute 24 sec latency 14.316 ms 2024-06-10T01:09:44.714 INFO:tasks.workunit.client.0.smithi007.stdout: 1 224387 10.73 MB/sec execute 25 sec latency 13.456 ms 2024-06-10T01:09:45.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:45 smithi078 ceph-mon[36683]: pgmap v544: 129 pgs: 129 active+clean; 58 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 381 op/s 2024-06-10T01:09:45.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:45 smithi175 ceph-mon[36636]: pgmap v544: 129 pgs: 129 active+clean; 58 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 381 op/s 2024-06-10T01:09:45.714 INFO:tasks.workunit.client.0.smithi007.stdout: 1 227174 10.98 MB/sec execute 26 sec latency 15.316 ms 2024-06-10T01:09:45.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:45 smithi007 ceph-mon[32398]: pgmap v544: 129 pgs: 129 active+clean; 58 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 381 op/s 2024-06-10T01:09:46.714 INFO:tasks.workunit.client.0.smithi007.stdout: 1 228509 10.81 MB/sec execute 27 sec latency 32.145 ms 2024-06-10T01:09:47.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:47 smithi078 ceph-mon[36683]: pgmap v545: 129 pgs: 129 active+clean; 58 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 383 op/s 2024-06-10T01:09:47.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:47 smithi175 ceph-mon[36636]: pgmap v545: 129 pgs: 129 active+clean; 58 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 383 op/s 2024-06-10T01:09:47.714 INFO:tasks.workunit.client.0.smithi007.stdout: 1 230924 10.92 MB/sec execute 28 sec latency 44.337 ms 2024-06-10T01:09:47.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:47 smithi007 ceph-mon[32398]: pgmap v545: 129 pgs: 129 active+clean; 58 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 383 op/s 2024-06-10T01:09:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.394+0000 7f169757e640 1 -- 172.21.15.7:0/7105726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16901024b0 msgr2=0x7f1690108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.394+0000 7f169757e640 1 --2- 172.21.15.7:0/7105726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16901024b0 0x7f1690108d40 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1688009ca0 tx=0x7f168802a8c0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.395+0000 7f169757e640 1 -- 172.21.15.7:0/7105726 shutdown_connections 2024-06-10T01:09:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.395+0000 7f169757e640 1 --2- 172.21.15.7:0/7105726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16901024b0 0x7f1690108d40 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.395+0000 7f169757e640 1 --2- 172.21.15.7:0/7105726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1690101af0 0x7f1690101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.395+0000 7f169757e640 1 --2- 172.21.15.7:0/7105726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1690100900 0x7f1690100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.395+0000 7f169757e640 1 -- 172.21.15.7:0/7105726 >> 172.21.15.7:0/7105726 conn(0x7f16900fbe70 msgr2=0x7f16900fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.395+0000 7f169757e640 1 -- 172.21.15.7:0/7105726 shutdown_connections 2024-06-10T01:09:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.395+0000 7f169757e640 1 -- 172.21.15.7:0/7105726 wait complete. 2024-06-10T01:09:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.395+0000 7f169757e640 1 Processor -- start 2024-06-10T01:09:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.395+0000 7f169757e640 1 -- start start 2024-06-10T01:09:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.395+0000 7f169757e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1690100900 0x7f16901a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.395+0000 7f169757e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1690101af0 0x7f16901a2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.395+0000 7f169757e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16901024b0 0x7f16901a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.395+0000 7f169757e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f169010c580 con 0x7f1690101af0 2024-06-10T01:09:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.395+0000 7f169757e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f169010c400 con 0x7f1690100900 2024-06-10T01:09:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.395+0000 7f169757e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f169010c700 con 0x7f16901024b0 2024-06-10T01:09:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.396+0000 7f1694b19640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1690101af0 0x7f16901a2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.396+0000 7f169531a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1690100900 0x7f16901a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.396+0000 7f1694b19640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1690101af0 0x7f16901a2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44708/0 (socket says 172.21.15.7:44708) 2024-06-10T01:09:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.396+0000 7f1694b19640 1 -- 172.21.15.7:0/1252437649 learned_addr learned my addr 172.21.15.7:0/1252437649 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.396+0000 7f1695b1b640 1 --2- 172.21.15.7:0/1252437649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16901024b0 0x7f16901a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.396+0000 7f169531a640 1 -- 172.21.15.7:0/1252437649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16901024b0 msgr2=0x7f16901a5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.396+0000 7f169531a640 1 --2- 172.21.15.7:0/1252437649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16901024b0 0x7f16901a5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.396+0000 7f169531a640 1 -- 172.21.15.7:0/1252437649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1690101af0 msgr2=0x7f16901a2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.396+0000 7f169531a640 1 --2- 172.21.15.7:0/1252437649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1690101af0 0x7f16901a2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.396+0000 7f169531a640 1 -- 172.21.15.7:0/1252437649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16901a5f10 con 0x7f1690100900 2024-06-10T01:09:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.396+0000 7f1694b19640 1 --2- 172.21.15.7:0/1252437649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1690101af0 0x7f16901a2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.396+0000 7f169531a640 1 --2- 172.21.15.7:0/1252437649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1690100900 0x7f16901a0a70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f168c00e9c0 tx=0x7f168c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.396+0000 7f16867fc640 1 -- 172.21.15.7:0/1252437649 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c00ccc0 con 0x7f1690100900 2024-06-10T01:09:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.396+0000 7f169757e640 1 -- 172.21.15.7:0/1252437649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16901a6200 con 0x7f1690100900 2024-06-10T01:09:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.396+0000 7f16867fc640 1 -- 172.21.15.7:0/1252437649 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f168c0035f0 con 0x7f1690100900 2024-06-10T01:09:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.396+0000 7f16867fc640 1 -- 172.21.15.7:0/1252437649 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c00ce60 con 0x7f1690100900 2024-06-10T01:09:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.396+0000 7f169757e640 1 -- 172.21.15.7:0/1252437649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16901a6750 con 0x7f1690100900 2024-06-10T01:09:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.398+0000 7f16867fc640 1 -- 172.21.15.7:0/1252437649 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f168c0037a0 con 0x7f1690100900 2024-06-10T01:09:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.399+0000 7f169a36b740 1 Processor -- start 2024-06-10T01:09:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.399+0000 7f169a36b740 1 -- start start 2024-06-10T01:09:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.399+0000 7f169a36b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cad3e23010 0x55cad3e23430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.399+0000 7f169a36b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cad3e23a00 0x55cad3e28500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.399+0000 7f169a36b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad3e28b70 0x55cad3e2b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.399+0000 7f169a36b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cad3d864f0 con 0x55cad3e28b70 2024-06-10T01:09:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.399+0000 7f169a36b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cad3d24870 con 0x55cad3e23010 2024-06-10T01:09:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.399+0000 7f169a36b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55cad3d824b0 con 0x55cad3e23a00 2024-06-10T01:09:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.399+0000 7f169531a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cad3e23a00 0x55cad3e28500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.399+0000 7f1695b1b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cad3e23010 0x55cad3e23430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.399+0000 7f1694b19640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad3e28b70 0x55cad3e2b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.400+0000 7f1694b19640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad3e28b70 0x55cad3e2b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44718/0 (socket says 172.21.15.7:44718) 2024-06-10T01:09:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.400+0000 7f1695b1b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cad3e23010 0x55cad3e23430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:54700/0 (socket says 172.21.15.7:54700) 2024-06-10T01:09:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.400+0000 7f1694b19640 1 -- 172.21.15.7:0/2356144295 learned_addr learned my addr 172.21.15.7:0/2356144295 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.400+0000 7f16867fc640 1 --2- 172.21.15.7:0/1252437649 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f167c0a2b80 0x7f167c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.400+0000 7f1695b1b640 1 --2- 172.21.15.7:0/1252437649 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f167c0a2b80 0x7f167c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.400+0000 7f1694b19640 1 -- 172.21.15.7:0/2356144295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cad3e23a00 msgr2=0x55cad3e28500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.400+0000 7f1694b19640 1 --2- 172.21.15.7:0/2356144295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cad3e23a00 0x55cad3e28500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.400+0000 7f1694b19640 1 -- 172.21.15.7:0/2356144295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cad3e23010 msgr2=0x55cad3e23430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.400+0000 7f1694b19640 1 --2- 172.21.15.7:0/2356144295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cad3e23010 0x55cad3e23430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.400+0000 7f1694b19640 1 -- 172.21.15.7:0/2356144295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cad3d409b0 con 0x55cad3e28b70 2024-06-10T01:09:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.400+0000 7f16867fc640 1 -- 172.21.15.7:0/1252437649 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f168c0cf8e0 con 0x7f1690100900 2024-06-10T01:09:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.400+0000 7f1694b19640 1 --2- 172.21.15.7:0/2356144295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad3e28b70 0x55cad3e2b000 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f1680011630 tx=0x7f1680011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af9173c42e7177f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.400+0000 7f167b7fe640 1 -- 172.21.15.7:0/2356144295 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168001e070 con 0x55cad3e28b70 2024-06-10T01:09:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.400+0000 7f167b7fe640 1 -- 172.21.15.7:0/2356144295 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f168000fbd0 con 0x55cad3e28b70 2024-06-10T01:09:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.400+0000 7f1695b1b640 1 --2- 172.21.15.7:0/1252437649 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f167c0a2b80 0x7f167c0a5040 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1688005e30 tx=0x7f168802a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.400+0000 7f167b7fe640 1 -- 172.21.15.7:0/2356144295 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1680019d00 con 0x55cad3e28b70 2024-06-10T01:09:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.401+0000 7f169a36b740 1 -- 172.21.15.7:0/2356144295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad3e28b70 msgr2=0x55cad3e2b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.401+0000 7f169a36b740 1 --2- 172.21.15.7:0/2356144295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad3e28b70 0x55cad3e2b000 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f1680011630 tx=0x7f1680011af0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.401+0000 7f169a36b740 1 -- 172.21.15.7:0/2356144295 shutdown_connections 2024-06-10T01:09:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.401+0000 7f169a36b740 1 --2- 172.21.15.7:0/2356144295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad3e28b70 0x55cad3e2b000 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.401+0000 7f169a36b740 1 --2- 172.21.15.7:0/2356144295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cad3e23a00 0x55cad3e28500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.401+0000 7f169a36b740 1 --2- 172.21.15.7:0/2356144295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cad3e23010 0x55cad3e23430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.401+0000 7f169a36b740 1 -- 172.21.15.7:0/2356144295 >> 172.21.15.7:0/2356144295 conn(0x55cad3e1e680 msgr2=0x55cad3e1ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.401+0000 7f169a36b740 1 -- 172.21.15.7:0/2356144295 shutdown_connections 2024-06-10T01:09:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.401+0000 7f169a36b740 1 -- 172.21.15.7:0/2356144295 wait complete. 2024-06-10T01:09:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.401+0000 7f169a36b740 1 -- start start 2024-06-10T01:09:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.401+0000 7f169a36b740 1 Processor -- start 2024-06-10T01:09:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.402+0000 7f169a36b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cad3e23010 0x55cad3e2d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.402+0000 7f169a36b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cad3e23a00 0x55cad3e2d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.402+0000 7f169a36b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad3e28b70 0x55cad3e347d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.402+0000 7f169a36b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cad3e2bb20 con 0x55cad3e28b70 2024-06-10T01:09:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.402+0000 7f169a36b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cad3d0fae0 con 0x55cad3e23010 2024-06-10T01:09:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.402+0000 7f169a36b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55cad3e2bca0 con 0x55cad3e23a00 2024-06-10T01:09:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.402+0000 7f169a36b740 10 client.0 _ll_register_callbacks cb 0x55cad3d489b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.402+0000 7f1695b1b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad3e28b70 0x55cad3e347d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.402+0000 7f1695b1b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad3e28b70 0x55cad3e347d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44724/0 (socket says 172.21.15.7:44724) 2024-06-10T01:09:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.402+0000 7f1695b1b640 1 -- 172.21.15.7:0/4020577221 learned_addr learned my addr 172.21.15.7:0/4020577221 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.402+0000 7f1694b19640 1 --2- 172.21.15.7:0/4020577221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cad3e23a00 0x55cad3e2d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.402+0000 7f169531a640 1 --2- 172.21.15.7:0/4020577221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cad3e23010 0x55cad3e2d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.402+0000 7f1695b1b640 1 -- 172.21.15.7:0/4020577221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cad3e23a00 msgr2=0x55cad3e2d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.402+0000 7f1695b1b640 1 --2- 172.21.15.7:0/4020577221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cad3e23a00 0x55cad3e2d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.402+0000 7f1695b1b640 1 -- 172.21.15.7:0/4020577221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cad3e23010 msgr2=0x55cad3e2d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.402+0000 7f1695b1b640 1 --2- 172.21.15.7:0/4020577221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cad3e23010 0x55cad3e2d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.402+0000 7f1695b1b640 1 -- 172.21.15.7:0/4020577221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cad3a9a210 con 0x55cad3e28b70 2024-06-10T01:09:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.402+0000 7f1694b19640 1 --2- 172.21.15.7:0/4020577221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cad3e23a00 0x55cad3e2d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.402+0000 7f169531a640 1 --2- 172.21.15.7:0/4020577221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cad3e23010 0x55cad3e2d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.402+0000 7f1695b1b640 1 --2- 172.21.15.7:0/4020577221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad3e28b70 0x55cad3e347d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f1688035000 tx=0x7f16880095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3062a27075d7ccf server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.402+0000 7f1678ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:09:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.403+0000 7f1678ff9640 1 -- 172.21.15.7:0/4020577221 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168804e020 con 0x55cad3e28b70 2024-06-10T01:09:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.403+0000 7f1678ff9640 1 -- 172.21.15.7:0/4020577221 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f16880047c0 con 0x55cad3e28b70 2024-06-10T01:09:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.403+0000 7f169a36b740 1 -- 172.21.15.7:0/4020577221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f168000fbd0 con 0x55cad3e28b70 2024-06-10T01:09:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.403+0000 7f1678ff9640 1 -- 172.21.15.7:0/4020577221 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1688004b40 con 0x55cad3e28b70 2024-06-10T01:09:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.403+0000 7f169a36b740 10 client.16431 fetch_fsmap learned FSMap version 30 2024-06-10T01:09:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.403+0000 7f169a36b740 1 -- 172.21.15.7:0/4020577221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cad3b66620 con 0x55cad3e28b70 2024-06-10T01:09:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.403+0000 7f1678ff9640 1 -- 172.21.15.7:0/4020577221 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f168000fbd0 con 0x55cad3e28b70 2024-06-10T01:09:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.403+0000 7f1678ff9640 1 -- 172.21.15.7:0/4020577221 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f1688049040 con 0x55cad3e28b70 2024-06-10T01:09:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.403+0000 7f169a36b740 10 client.16431 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:09:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.403+0000 7f169a36b740 10 client.16431 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.403+0000 7f169a36b740 20 client.16431 populate_metadata read hostname 'smithi007' 2024-06-10T01:09:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.403+0000 7f169a36b740 1 --2- 172.21.15.7:0/4020577221 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55cad3e39280 0x55cad3e3b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.403+0000 7f169a36b740 4 client.16431 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:09:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.403+0000 7f169a36b740 1 -- 172.21.15.7:0/4020577221 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cad3d99f10 con 0x55cad3e39280 2024-06-10T01:09:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.404+0000 7f169531a640 1 --2- 172.21.15.7:0/4020577221 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55cad3e39280 0x55cad3e3b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.404+0000 7f1678ff9640 10 client.16431 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.404+0000 7f169531a640 1 --2- 172.21.15.7:0/4020577221 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55cad3e39280 0x55cad3e3b740 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb36294d34ce1aa9 server_cookie=752f7625292ed624 in_seq=0 out_seq=0 2024-06-10T01:09:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.405+0000 7f1678ff9640 1 -- 172.21.15.7:0/4020577221 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cad3d99f10 con 0x55cad3e39280 2024-06-10T01:09:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.405+0000 7f1678ff9640 10 client.16431 handle_command_reply: tid=0 2024-06-10T01:09:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.405+0000 7f169a36b740 1 client.16431 shutdown 2024-06-10T01:09:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.405+0000 7f169a36b740 1 -- 172.21.15.7:0/4020577221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad3e28b70 msgr2=0x55cad3e347d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.405+0000 7f169a36b740 1 --2- 172.21.15.7:0/4020577221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad3e28b70 0x55cad3e347d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f1688035000 tx=0x7f16880095c0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.405+0000 7f169a36b740 1 -- 172.21.15.7:0/4020577221 shutdown_connections 2024-06-10T01:09:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.405+0000 7f169a36b740 1 --2- 172.21.15.7:0/4020577221 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55cad3e39280 0x55cad3e3b740 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.405+0000 7f169a36b740 1 --2- 172.21.15.7:0/4020577221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad3e28b70 0x55cad3e347d0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.405+0000 7f169a36b740 1 --2- 172.21.15.7:0/4020577221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cad3e23a00 0x55cad3e2d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.405+0000 7f169a36b740 1 --2- 172.21.15.7:0/4020577221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cad3e23010 0x55cad3e2d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.405+0000 7f1678ff9640 0 client.16431 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.405+0000 7f169a36b740 1 -- 172.21.15.7:0/4020577221 >> 172.21.15.7:0/4020577221 conn(0x55cad3e1e680 msgr2=0x55cad3e1d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.405+0000 7f169a36b740 1 -- 172.21.15.7:0/4020577221 shutdown_connections 2024-06-10T01:09:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.405+0000 7f169a36b740 1 -- 172.21.15.7:0/4020577221 wait complete. 2024-06-10T01:09:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.405+0000 7f169a36b740 20 client.16431 trim_cache size 0 max 16384 2024-06-10T01:09:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.421+0000 7f169a36b740 1 Processor -- start 2024-06-10T01:09:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.421+0000 7f169a36b740 1 -- start start 2024-06-10T01:09:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.421+0000 7f169a36b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad3e23010 0x55cad3e24be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.421+0000 7f169a36b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cad3e23a00 0x55cad3e34790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.421+0000 7f169a36b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cad3e28b70 0x55cad3e25120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.421+0000 7f169a36b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f16901b3550 con 0x55cad3e23010 2024-06-10T01:09:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.421+0000 7f169a36b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16901b0dc0 con 0x55cad3e28b70 2024-06-10T01:09:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.421+0000 7f169a36b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f16901aec10 con 0x55cad3e23a00 2024-06-10T01:09:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.421+0000 7f169531a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad3e23010 0x55cad3e24be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.421+0000 7f169531a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad3e23010 0x55cad3e24be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44726/0 (socket says 172.21.15.7:44726) 2024-06-10T01:09:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.421+0000 7f169531a640 1 -- 172.21.15.7:0/2516974898 learned_addr learned my addr 172.21.15.7:0/2516974898 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.421+0000 7f1694b19640 1 --2- 172.21.15.7:0/2516974898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cad3e23a00 0x55cad3e34790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.421+0000 7f1695b1b640 1 --2- 172.21.15.7:0/2516974898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cad3e28b70 0x55cad3e25120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.422+0000 7f169531a640 1 -- 172.21.15.7:0/2516974898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cad3e23a00 msgr2=0x55cad3e34790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.422+0000 7f169531a640 1 --2- 172.21.15.7:0/2516974898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cad3e23a00 0x55cad3e34790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.422+0000 7f169531a640 1 -- 172.21.15.7:0/2516974898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cad3e28b70 msgr2=0x55cad3e25120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.422+0000 7f169531a640 1 --2- 172.21.15.7:0/2516974898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cad3e28b70 0x55cad3e25120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.422+0000 7f169531a640 1 -- 172.21.15.7:0/2516974898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cad3e1e4b0 con 0x55cad3e23010 2024-06-10T01:09:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.422+0000 7f1695b1b640 1 --2- 172.21.15.7:0/2516974898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cad3e28b70 0x55cad3e25120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.422+0000 7f1694b19640 1 --2- 172.21.15.7:0/2516974898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cad3e23a00 0x55cad3e34790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.422+0000 7f169531a640 1 --2- 172.21.15.7:0/2516974898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad3e23010 0x55cad3e24be0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f168c0986e0 tx=0x7f168c098130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14aaab614f08c589 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.422+0000 7f167affd640 1 -- 172.21.15.7:0/2516974898 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c0d5020 con 0x55cad3e23010 2024-06-10T01:09:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.422+0000 7f167affd640 1 -- 172.21.15.7:0/2516974898 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f168c0d2e50 con 0x55cad3e23010 2024-06-10T01:09:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.422+0000 7f167affd640 1 -- 172.21.15.7:0/2516974898 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c0a4540 con 0x55cad3e23010 2024-06-10T01:09:48.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.423+0000 7f169a36b740 1 -- 172.21.15.7:0/2516974898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad3e23010 msgr2=0x55cad3e24be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:48.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.423+0000 7f169a36b740 1 --2- 172.21.15.7:0/2516974898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad3e23010 0x55cad3e24be0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f168c0986e0 tx=0x7f168c098130 comp rx=0 tx=0).stop 2024-06-10T01:09:48.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.423+0000 7f169a36b740 1 -- 172.21.15.7:0/2516974898 shutdown_connections 2024-06-10T01:09:48.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.423+0000 7f169a36b740 1 --2- 172.21.15.7:0/2516974898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cad3e28b70 0x55cad3e25120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.423+0000 7f169a36b740 1 --2- 172.21.15.7:0/2516974898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cad3e23a00 0x55cad3e34790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.423+0000 7f169a36b740 1 --2- 172.21.15.7:0/2516974898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad3e23010 0x55cad3e24be0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.423+0000 7f169a36b740 1 -- 172.21.15.7:0/2516974898 >> 172.21.15.7:0/2516974898 conn(0x55cad3e1e680 msgr2=0x55cad3e2c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:48.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.423+0000 7f169a36b740 1 -- 172.21.15.7:0/2516974898 shutdown_connections 2024-06-10T01:09:48.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.423+0000 7f169a36b740 1 -- 172.21.15.7:0/2516974898 wait complete. 2024-06-10T01:09:48.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.423+0000 7f169a36b740 1 -- start start 2024-06-10T01:09:48.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.423+0000 7f169a36b740 1 Processor -- start 2024-06-10T01:09:48.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.424+0000 7f169a36b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cad3e23010 0x55cad3e34550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.424+0000 7f169a36b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad3e23a00 0x55cad3e30ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.424+0000 7f169a36b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cad3e28b70 0x55cad3e31530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.424+0000 7f169a36b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f16901af280 con 0x55cad3e23a00 2024-06-10T01:09:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.424+0000 7f169a36b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16901b06f0 con 0x55cad3e28b70 2024-06-10T01:09:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.424+0000 7f169a36b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f16901af950 con 0x55cad3e23010 2024-06-10T01:09:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.424+0000 7f169a36b740 10 client.0 _ll_register_callbacks cb 0x7f1680005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.424+0000 7f1694b19640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad3e23a00 0x55cad3e30ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.424+0000 7f1694b19640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad3e23a00 0x55cad3e30ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44732/0 (socket says 172.21.15.7:44732) 2024-06-10T01:09:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.424+0000 7f169531a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cad3e23010 0x55cad3e34550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.424+0000 7f1694b19640 1 -- 172.21.15.7:0/3158655612 learned_addr learned my addr 172.21.15.7:0/3158655612 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.424+0000 7f1695b1b640 1 --2- 172.21.15.7:0/3158655612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cad3e28b70 0x55cad3e31530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.424+0000 7f1694b19640 1 -- 172.21.15.7:0/3158655612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cad3e23010 msgr2=0x55cad3e34550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.424+0000 7f1694b19640 1 --2- 172.21.15.7:0/3158655612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cad3e23010 0x55cad3e34550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.424+0000 7f1694b19640 1 -- 172.21.15.7:0/3158655612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cad3e28b70 msgr2=0x55cad3e31530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.424+0000 7f1694b19640 1 --2- 172.21.15.7:0/3158655612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cad3e28b70 0x55cad3e31530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.424+0000 7f1694b19640 1 -- 172.21.15.7:0/3158655612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cad3ea2010 con 0x55cad3e23a00 2024-06-10T01:09:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.424+0000 7f1694b19640 1 --2- 172.21.15.7:0/3158655612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad3e23a00 0x55cad3e30ff0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f168001c040 tx=0x7f16800195b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef6b0ac0fb75a682 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.424+0000 7f1678ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:09:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.424+0000 7f1678ff9640 1 -- 172.21.15.7:0/3158655612 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168001e040 con 0x55cad3e23a00 2024-06-10T01:09:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.424+0000 7f1678ff9640 1 -- 172.21.15.7:0/3158655612 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f168000fa00 con 0x55cad3e23a00 2024-06-10T01:09:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.424+0000 7f169a36b740 1 -- 172.21.15.7:0/3158655612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f168c0d2e50 con 0x55cad3e23a00 2024-06-10T01:09:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.425+0000 7f1678ff9640 1 -- 172.21.15.7:0/3158655612 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1680035050 con 0x55cad3e23a00 2024-06-10T01:09:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.425+0000 7f169a36b740 10 client.16437 fetch_fsmap learned FSMap version 30 2024-06-10T01:09:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.425+0000 7f169a36b740 1 -- 172.21.15.7:0/3158655612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cad3ea31c0 con 0x55cad3e23a00 2024-06-10T01:09:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.425+0000 7f1678ff9640 1 -- 172.21.15.7:0/3158655612 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f168c0d2e50 con 0x55cad3e23a00 2024-06-10T01:09:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.425+0000 7f1678ff9640 1 -- 172.21.15.7:0/3158655612 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f1680004a10 con 0x55cad3e23a00 2024-06-10T01:09:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.425+0000 7f169a36b740 10 client.16437 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:09:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.425+0000 7f169a36b740 10 client.16437 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.425+0000 7f169a36b740 20 client.16437 populate_metadata read hostname 'smithi007' 2024-06-10T01:09:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.425+0000 7f169a36b740 1 --2- 172.21.15.7:0/3158655612 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55cad3ea39d0 0x55cad3ea5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.425+0000 7f169a36b740 4 client.16437 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:09:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.425+0000 7f169a36b740 1 -- 172.21.15.7:0/3158655612 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cad3d93c20 con 0x55cad3ea39d0 2024-06-10T01:09:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.425+0000 7f169531a640 1 --2- 172.21.15.7:0/3158655612 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55cad3ea39d0 0x55cad3ea5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.425+0000 7f1678ff9640 10 client.16437 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.425+0000 7f169531a640 1 --2- 172.21.15.7:0/3158655612 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55cad3ea39d0 0x55cad3ea5df0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc19c95a91809371 server_cookie=9765f22e6e6833c8 in_seq=0 out_seq=0 2024-06-10T01:09:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.522+0000 7f1678ff9640 1 -- 172.21.15.7:0/3158655612 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55cad3d93c20 con 0x55cad3ea39d0 2024-06-10T01:09:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.522+0000 7f1678ff9640 10 client.16437 handle_command_reply: tid=0 2024-06-10T01:09:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.522+0000 7f169a36b740 1 client.16437 shutdown 2024-06-10T01:09:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.522+0000 7f169a36b740 1 -- 172.21.15.7:0/3158655612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad3e23a00 msgr2=0x55cad3e30ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.522+0000 7f169a36b740 1 --2- 172.21.15.7:0/3158655612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad3e23a00 0x55cad3e30ff0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f168001c040 tx=0x7f16800195b0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.522+0000 7f169a36b740 1 -- 172.21.15.7:0/3158655612 shutdown_connections 2024-06-10T01:09:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.522+0000 7f169a36b740 1 --2- 172.21.15.7:0/3158655612 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55cad3ea39d0 0x55cad3ea5df0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.522+0000 7f169a36b740 1 --2- 172.21.15.7:0/3158655612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cad3e28b70 0x55cad3e31530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.522+0000 7f169a36b740 1 --2- 172.21.15.7:0/3158655612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad3e23a00 0x55cad3e30ff0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.522+0000 7f1678ff9640 0 client.16437 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.522+0000 7f169a36b740 1 --2- 172.21.15.7:0/3158655612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cad3e23010 0x55cad3e34550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.522+0000 7f169a36b740 1 -- 172.21.15.7:0/3158655612 >> 172.21.15.7:0/3158655612 conn(0x55cad3e1e680 msgr2=0x55cad3e2c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.522+0000 7f169a36b740 1 -- 172.21.15.7:0/3158655612 shutdown_connections 2024-06-10T01:09:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.522+0000 7f169a36b740 1 -- 172.21.15.7:0/3158655612 wait complete. 2024-06-10T01:09:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.522+0000 7f169a36b740 20 client.16437 trim_cache size 0 max 16384 2024-06-10T01:09:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:09:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:09:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:09:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.523+0000 7f169757e640 1 -- 172.21.15.7:0/1252437649 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f167c0a2b80 msgr2=0x7f167c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.523+0000 7f169757e640 1 --2- 172.21.15.7:0/1252437649 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f167c0a2b80 0x7f167c0a5040 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1688005e30 tx=0x7f168802a8c0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.523+0000 7f169757e640 1 -- 172.21.15.7:0/1252437649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1690100900 msgr2=0x7f16901a0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.523+0000 7f169757e640 1 --2- 172.21.15.7:0/1252437649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1690100900 0x7f16901a0a70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f168c00e9c0 tx=0x7f168c00ee80 comp rx=0 tx=0).stop 2024-06-10T01:09:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.523+0000 7f169757e640 1 -- 172.21.15.7:0/1252437649 shutdown_connections 2024-06-10T01:09:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.523+0000 7f169757e640 1 --2- 172.21.15.7:0/1252437649 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f167c0a2b80 0x7f167c0a5040 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.523+0000 7f169757e640 1 --2- 172.21.15.7:0/1252437649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16901024b0 0x7f16901a5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.523+0000 7f169757e640 1 --2- 172.21.15.7:0/1252437649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1690101af0 0x7f16901a2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.523+0000 7f169757e640 1 --2- 172.21.15.7:0/1252437649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1690100900 0x7f16901a0a70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.523+0000 7f169757e640 1 -- 172.21.15.7:0/1252437649 >> 172.21.15.7:0/1252437649 conn(0x7f16900fbe70 msgr2=0x7f16900691c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.523+0000 7f169757e640 1 -- 172.21.15.7:0/1252437649 shutdown_connections 2024-06-10T01:09:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:09:48.523+0000 7f169757e640 1 -- 172.21.15.7:0/1252437649 wait complete. 2024-06-10T01:09:48.714 INFO:tasks.workunit.client.0.smithi007.stdout: 1 232499 10.74 MB/sec execute 29 sec latency 46.309 ms 2024-06-10T01:09:49.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:49 smithi078 ceph-mon[36683]: pgmap v546: 129 pgs: 129 active+clean; 62 MiB data, 929 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 392 op/s 2024-06-10T01:09:49.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:49 smithi175 ceph-mon[36636]: pgmap v546: 129 pgs: 129 active+clean; 62 MiB data, 929 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 392 op/s 2024-06-10T01:09:49.714 INFO:tasks.workunit.client.0.smithi007.stdout: 1 234682 10.84 MB/sec execute 30 sec latency 19.805 ms 2024-06-10T01:09:49.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:49 smithi007 ceph-mon[32398]: pgmap v546: 129 pgs: 129 active+clean; 62 MiB data, 929 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 392 op/s 2024-06-10T01:09:50.714 INFO:tasks.workunit.client.0.smithi007.stdout: 1 237145 10.89 MB/sec execute 31 sec latency 16.215 ms 2024-06-10T01:09:51.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:51 smithi078 ceph-mon[36683]: pgmap v547: 129 pgs: 129 active+clean; 58 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 416 op/s 2024-06-10T01:09:51.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:51 smithi175 ceph-mon[36636]: pgmap v547: 129 pgs: 129 active+clean; 58 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 416 op/s 2024-06-10T01:09:51.714 INFO:tasks.workunit.client.0.smithi007.stdout: 1 238800 10.82 MB/sec execute 32 sec latency 11.214 ms 2024-06-10T01:09:51.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:51 smithi007 ceph-mon[32398]: pgmap v547: 129 pgs: 129 active+clean; 58 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 416 op/s 2024-06-10T01:09:51.973 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.2 2024-06-10T01:09:51.974 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:09:52.246 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:09:52.528 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:09:52.714 INFO:tasks.workunit.client.0.smithi007.stdout: 1 241465 11.00 MB/sec execute 33 sec latency 34.232 ms 2024-06-10T01:09:53.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:53 smithi078 ceph-mon[36683]: pgmap v548: 129 pgs: 129 active+clean; 59 MiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 338 op/s 2024-06-10T01:09:53.608 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:53 smithi007 ceph-mon[32398]: pgmap v548: 129 pgs: 129 active+clean; 59 MiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 338 op/s 2024-06-10T01:09:53.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:53 smithi175 ceph-mon[36636]: pgmap v548: 129 pgs: 129 active+clean; 59 MiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 338 op/s 2024-06-10T01:09:53.714 INFO:tasks.workunit.client.0.smithi007.stdout: 1 242888 10.84 MB/sec execute 34 sec latency 71.937 ms 2024-06-10T01:09:54.084 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:09:54.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.547+0000 7f1b79e92700 1 -- 172.21.15.7:0/2648894690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b74102e80 msgr2=0x7f1b7410f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:54.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.547+0000 7f1b79e92700 1 --2- 172.21.15.7:0/2648894690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b74102e80 0x7f1b7410f330 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00b0a0 tx=0x7f1b6c00b3b0 comp rx=0 tx=0).stop 2024-06-10T01:09:54.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.547+0000 7f1b79e92700 1 -- 172.21.15.7:0/2648894690 shutdown_connections 2024-06-10T01:09:54.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.547+0000 7f1b79e92700 1 --2- 172.21.15.7:0/2648894690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b74102e80 0x7f1b7410f330 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:54.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.547+0000 7f1b79e92700 1 --2- 172.21.15.7:0/2648894690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b741084f0 0x7f1b741088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:54.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.548+0000 7f1b79e92700 1 --2- 172.21.15.7:0/2648894690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b741024e0 0x7f1b74102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:54.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.548+0000 7f1b79e92700 1 -- 172.21.15.7:0/2648894690 >> 172.21.15.7:0/2648894690 conn(0x7f1b74079530 msgr2=0x7f1b74079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:54.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.548+0000 7f1b79e92700 1 -- 172.21.15.7:0/2648894690 shutdown_connections 2024-06-10T01:09:54.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.548+0000 7f1b79e92700 1 -- 172.21.15.7:0/2648894690 wait complete. 2024-06-10T01:09:54.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.548+0000 7f1b79e92700 1 Processor -- start 2024-06-10T01:09:54.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.548+0000 7f1b79e92700 1 -- start start 2024-06-10T01:09:54.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.549+0000 7f1b79e92700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b741024e0 0x7f1b7419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:54.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.549+0000 7f1b79e92700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b74102e80 0x7f1b7419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:54.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.549+0000 7f1b79e92700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b741084f0 0x7f1b74198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:54.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.549+0000 7f1b79e92700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1b74112760 con 0x7f1b741084f0 2024-06-10T01:09:54.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.549+0000 7f1b79e92700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1b741125e0 con 0x7f1b74102e80 2024-06-10T01:09:54.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.549+0000 7f1b79e92700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1b741128e0 con 0x7f1b741024e0 2024-06-10T01:09:54.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.549+0000 7f1b737fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b741024e0 0x7f1b7419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:54.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.549+0000 7f1b73fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b741084f0 0x7f1b74198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:54.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.549+0000 7f1b737fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b741024e0 0x7f1b7419e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43258/0 (socket says 172.21.15.7:43258) 2024-06-10T01:09:54.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.549+0000 7f1b737fe700 1 -- 172.21.15.7:0/2500810694 learned_addr learned my addr 172.21.15.7:0/2500810694 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:54.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.549+0000 7f1b72ffd700 1 --2- 172.21.15.7:0/2500810694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b74102e80 0x7f1b7419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:54.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.549+0000 7f1b737fe700 1 -- 172.21.15.7:0/2500810694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b74102e80 msgr2=0x7f1b7419edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:54.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.549+0000 7f1b737fe700 1 --2- 172.21.15.7:0/2500810694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b74102e80 0x7f1b7419edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:54.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.549+0000 7f1b737fe700 1 -- 172.21.15.7:0/2500810694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b741084f0 msgr2=0x7f1b74198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:54.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.549+0000 7f1b737fe700 1 --2- 172.21.15.7:0/2500810694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b741084f0 0x7f1b74198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:54.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.549+0000 7f1b737fe700 1 -- 172.21.15.7:0/2500810694 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b74199220 con 0x7f1b741024e0 2024-06-10T01:09:54.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.549+0000 7f1b737fe700 1 --2- 172.21.15.7:0/2500810694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b741024e0 0x7f1b7419e8b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400e9c0 tx=0x7f1b6400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:54.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.550+0000 7f1b70ff9700 1 -- 172.21.15.7:0/2500810694 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6400cbd0 con 0x7f1b741024e0 2024-06-10T01:09:54.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.550+0000 7f1b79e92700 1 -- 172.21.15.7:0/2500810694 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b74199510 con 0x7f1b741024e0 2024-06-10T01:09:54.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.550+0000 7f1b70ff9700 1 -- 172.21.15.7:0/2500810694 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b6400ee60 con 0x7f1b741024e0 2024-06-10T01:09:54.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.550+0000 7f1b70ff9700 1 -- 172.21.15.7:0/2500810694 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64005630 con 0x7f1b741024e0 2024-06-10T01:09:54.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.550+0000 7f1b79e92700 1 -- 172.21.15.7:0/2500810694 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b741a5780 con 0x7f1b741024e0 2024-06-10T01:09:54.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.551+0000 7f1b79e92700 1 -- 172.21.15.7:0/2500810694 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b74069180 con 0x7f1b741024e0 2024-06-10T01:09:54.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.552+0000 7f1b70ff9700 1 -- 172.21.15.7:0/2500810694 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f1b64004750 con 0x7f1b741024e0 2024-06-10T01:09:54.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.553+0000 7f1b70ff9700 1 --2- 172.21.15.7:0/2500810694 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1b5c0a2c20 0x7f1b5c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:54.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.553+0000 7f1b72ffd700 1 --2- 172.21.15.7:0/2500810694 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1b5c0a2c20 0x7f1b5c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:54.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.553+0000 7f1b70ff9700 1 -- 172.21.15.7:0/2500810694 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f1b640cf800 con 0x7f1b741024e0 2024-06-10T01:09:54.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.553+0000 7f1b72ffd700 1 --2- 172.21.15.7:0/2500810694 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1b5c0a2c20 0x7f1b5c0a50e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1b74079b30 tx=0x7f1b68009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:54.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.557+0000 7f1b70ff9700 1 -- 172.21.15.7:0/2500810694 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1b64053850 con 0x7f1b741024e0 2024-06-10T01:09:54.715 INFO:tasks.workunit.client.0.smithi007.stdout: 1 245210 10.93 MB/sec execute 35 sec latency 61.417 ms 2024-06-10T01:09:54.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.827+0000 7f1b79e92700 1 -- 172.21.15.7:0/2500810694 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b74199fa0 con 0x7f1b741024e0 2024-06-10T01:09:54.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.828+0000 7f1b70ff9700 1 -- 172.21.15.7:0/2500810694 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11858 (secure 0 0 0) 0x7f1b6409c380 con 0x7f1b741024e0 2024-06-10T01:09:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:09:54.824 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:08:17.181785+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24434,24436,24451]},"id":1}]} 2024-06-10T01:09:54.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.831+0000 7f1b79e92700 1 -- 172.21.15.7:0/2500810694 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1b5c0a2c20 msgr2=0x7f1b5c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:54.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.831+0000 7f1b79e92700 1 --2- 172.21.15.7:0/2500810694 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1b5c0a2c20 0x7f1b5c0a50e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1b74079b30 tx=0x7f1b68009260 comp rx=0 tx=0).stop 2024-06-10T01:09:54.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.831+0000 7f1b79e92700 1 -- 172.21.15.7:0/2500810694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b741024e0 msgr2=0x7f1b7419e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:54.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.831+0000 7f1b79e92700 1 --2- 172.21.15.7:0/2500810694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b741024e0 0x7f1b7419e8b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400e9c0 tx=0x7f1b6400ecd0 comp rx=0 tx=0).stop 2024-06-10T01:09:54.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.831+0000 7f1b79e92700 1 -- 172.21.15.7:0/2500810694 shutdown_connections 2024-06-10T01:09:54.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.831+0000 7f1b79e92700 1 --2- 172.21.15.7:0/2500810694 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1b5c0a2c20 0x7f1b5c0a50e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:54.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.831+0000 7f1b79e92700 1 --2- 172.21.15.7:0/2500810694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b741084f0 0x7f1b74198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:54.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.831+0000 7f1b79e92700 1 --2- 172.21.15.7:0/2500810694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b741024e0 0x7f1b7419e8b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:54.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.831+0000 7f1b79e92700 1 --2- 172.21.15.7:0/2500810694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b74102e80 0x7f1b7419edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:54.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.831+0000 7f1b79e92700 1 -- 172.21.15.7:0/2500810694 >> 172.21.15.7:0/2500810694 conn(0x7f1b74079530 msgr2=0x7f1b741005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:54.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.831+0000 7f1b79e92700 1 -- 172.21.15.7:0/2500810694 shutdown_connections 2024-06-10T01:09:54.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:54.831+0000 7f1b79e92700 1 -- 172.21.15.7:0/2500810694 wait complete. 2024-06-10T01:09:54.828 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-06-10T01:09:55.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:55 smithi078 ceph-mon[36683]: pgmap v549: 129 pgs: 129 active+clean; 61 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 353 op/s 2024-06-10T01:09:55.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:55 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2500810694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:09:55.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:55 smithi175 ceph-mon[36636]: pgmap v549: 129 pgs: 129 active+clean; 61 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 353 op/s 2024-06-10T01:09:55.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:55 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2500810694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:09:55.714 INFO:tasks.workunit.client.0.smithi007.stdout: 1 247357 10.85 MB/sec execute 36 sec latency 14.533 ms 2024-06-10T01:09:55.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:55 smithi007 ceph-mon[32398]: pgmap v549: 129 pgs: 129 active+clean; 61 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 353 op/s 2024-06-10T01:09:55.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:55 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2500810694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:09:56.714 INFO:tasks.workunit.client.0.smithi007.stdout: 1 249139 10.87 MB/sec execute 37 sec latency 15.773 ms 2024-06-10T01:09:56.871 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db / --timeout 90 --expiration 138.2 --await-for 45.0' 2024-06-10T01:09:56.872 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db / --timeout 90 --expiration 138.2 --await-for 45.0 2024-06-10T01:09:57.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.425+0000 7fd557597700 1 -- 172.21.15.7:0/1813327800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5501073f0 msgr2=0x7fd5501077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:57.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.425+0000 7fd557597700 1 --2- 172.21.15.7:0/1813327800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5501073f0 0x7fd5501077d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd54c009990 tx=0x7fd54c009ca0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.425+0000 7fd557597700 1 -- 172.21.15.7:0/1813327800 shutdown_connections 2024-06-10T01:09:57.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.425+0000 7fd557597700 1 --2- 172.21.15.7:0/1813327800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd550101d40 0x7fd55010e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.425+0000 7fd557597700 1 --2- 172.21.15.7:0/1813327800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5501073f0 0x7fd5501077d0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.425+0000 7fd557597700 1 --2- 172.21.15.7:0/1813327800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd550101380 0x7fd550101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.425+0000 7fd557597700 1 -- 172.21.15.7:0/1813327800 >> 172.21.15.7:0/1813327800 conn(0x7fd550078300 msgr2=0x7fd550078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:57.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.425+0000 7fd557597700 1 -- 172.21.15.7:0/1813327800 shutdown_connections 2024-06-10T01:09:57.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.425+0000 7fd557597700 1 -- 172.21.15.7:0/1813327800 wait complete. 2024-06-10T01:09:57.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.426+0000 7fd557597700 1 Processor -- start 2024-06-10T01:09:57.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.426+0000 7fd557597700 1 -- start start 2024-06-10T01:09:57.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.426+0000 7fd557597700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd550101380 0x7fd5501082d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.426+0000 7fd557597700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd550101d40 0x7fd550108810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.426+0000 7fd557597700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5501073f0 0x7fd550108d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.426+0000 7fd557597700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd550111660 con 0x7fd550101380 2024-06-10T01:09:57.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.426+0000 7fd557597700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd5501114e0 con 0x7fd550101d40 2024-06-10T01:09:57.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.426+0000 7fd557597700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd5501117e0 con 0x7fd5501073f0 2024-06-10T01:09:57.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.426+0000 7fd555333700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd550101380 0x7fd5501082d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.426+0000 7fd554b32700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd550101d40 0x7fd550108810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.426+0000 7fd555333700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd550101380 0x7fd5501082d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48568/0 (socket says 172.21.15.7:48568) 2024-06-10T01:09:57.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.426+0000 7fd555333700 1 -- 172.21.15.7:0/17518212 learned_addr learned my addr 172.21.15.7:0/17518212 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:57.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.426+0000 7fd555b34700 1 --2- 172.21.15.7:0/17518212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5501073f0 0x7fd550108d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.427+0000 7fd555b34700 1 -- 172.21.15.7:0/17518212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd550101d40 msgr2=0x7fd550108810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:57.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.427+0000 7fd555b34700 1 --2- 172.21.15.7:0/17518212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd550101d40 0x7fd550108810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.427+0000 7fd555b34700 1 -- 172.21.15.7:0/17518212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd550101380 msgr2=0x7fd5501082d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:57.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.427+0000 7fd555b34700 1 --2- 172.21.15.7:0/17518212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd550101380 0x7fd5501082d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.427+0000 7fd555b34700 1 -- 172.21.15.7:0/17518212 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5501a7130 con 0x7fd5501073f0 2024-06-10T01:09:57.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.427+0000 7fd555333700 1 --2- 172.21.15.7:0/17518212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd550101380 0x7fd5501082d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:57.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.427+0000 7fd554b32700 1 --2- 172.21.15.7:0/17518212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd550101d40 0x7fd550108810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:57.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.427+0000 7fd555b34700 1 --2- 172.21.15.7:0/17518212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5501073f0 0x7fd550108d50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd548007f10 tx=0x7fd54800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:57.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.427+0000 7fd5467fc700 1 -- 172.21.15.7:0/17518212 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548004db0 con 0x7fd5501073f0 2024-06-10T01:09:57.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.427+0000 7fd557597700 1 -- 172.21.15.7:0/17518212 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5501a7420 con 0x7fd5501073f0 2024-06-10T01:09:57.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.428+0000 7fd5467fc700 1 -- 172.21.15.7:0/17518212 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd548009e20 con 0x7fd5501073f0 2024-06-10T01:09:57.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.428+0000 7fd557597700 1 -- 172.21.15.7:0/17518212 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5501a7960 con 0x7fd5501073f0 2024-06-10T01:09:57.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.428+0000 7fd5467fc700 1 -- 172.21.15.7:0/17518212 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548015400 con 0x7fd5501073f0 2024-06-10T01:09:57.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.430+0000 7fd5467fc700 1 -- 172.21.15.7:0/17518212 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fd54800db20 con 0x7fd5501073f0 2024-06-10T01:09:57.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.431+0000 7fd5467fc700 1 --2- 172.21.15.7:0/17518212 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd53c0a2c20 0x7fd53c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.431+0000 7fd555333700 1 --2- 172.21.15.7:0/17518212 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd53c0a2c20 0x7fd53c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.431+0000 7fd5467fc700 1 -- 172.21.15.7:0/17518212 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fd5480cec50 con 0x7fd5501073f0 2024-06-10T01:09:57.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.432+0000 7fd555333700 1 --2- 172.21.15.7:0/17518212 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd53c0a2c20 0x7fd53c0a50e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd54c00b2e0 tx=0x7fd54c009590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:57.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.433+0000 7fd561e1ab80 1 Processor -- start 2024-06-10T01:09:57.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.433+0000 7fd561e1ab80 1 -- start start 2024-06-10T01:09:57.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.434+0000 7fd561e1ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558987719f60 0x55898771a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.434+0000 7fd561e1ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55898771a880 0x558987714260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.434+0000 7fd561e1ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589877147a0 0x558987714bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.434+0000 7fd561e1ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558987606f60 con 0x558987719f60 2024-06-10T01:09:57.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.434+0000 7fd561e1ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558987665850 con 0x55898771a880 2024-06-10T01:09:57.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.434+0000 7fd561e1ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5589875cd460 con 0x5589877147a0 2024-06-10T01:09:57.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.434+0000 7fd555b34700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589877147a0 0x558987714bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.434+0000 7fd555333700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558987719f60 0x55898771a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.434+0000 7fd555b34700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589877147a0 0x558987714bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43290/0 (socket says 172.21.15.7:43290) 2024-06-10T01:09:57.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.434+0000 7fd555b34700 1 -- 172.21.15.7:0/4139941042 learned_addr learned my addr 172.21.15.7:0/4139941042 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:57.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.434+0000 7fd554b32700 1 --2- 172.21.15.7:0/4139941042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55898771a880 0x558987714260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.434+0000 7fd555b34700 1 -- 172.21.15.7:0/4139941042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55898771a880 msgr2=0x558987714260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:57.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.434+0000 7fd555b34700 1 --2- 172.21.15.7:0/4139941042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55898771a880 0x558987714260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.434+0000 7fd555b34700 1 -- 172.21.15.7:0/4139941042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558987719f60 msgr2=0x55898771a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:57.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.434+0000 7fd555b34700 1 --2- 172.21.15.7:0/4139941042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558987719f60 0x55898771a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.434+0000 7fd555b34700 1 -- 172.21.15.7:0/4139941042 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589875b48d0 con 0x5589877147a0 2024-06-10T01:09:57.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.434+0000 7fd555b34700 1 --2- 172.21.15.7:0/4139941042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589877147a0 0x558987714bc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd54809b060 tx=0x7fd54809ad20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8db028b419c23ce9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:57.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.435+0000 7fd53b7fe700 1 -- 172.21.15.7:0/4139941042 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5480a3d40 con 0x5589877147a0 2024-06-10T01:09:57.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.435+0000 7fd53b7fe700 1 -- 172.21.15.7:0/4139941042 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd54800de10 con 0x5589877147a0 2024-06-10T01:09:57.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.435+0000 7fd53b7fe700 1 -- 172.21.15.7:0/4139941042 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54809b590 con 0x5589877147a0 2024-06-10T01:09:57.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.436+0000 7fd561e1ab80 1 -- 172.21.15.7:0/4139941042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589877147a0 msgr2=0x558987714bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:57.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.436+0000 7fd561e1ab80 1 --2- 172.21.15.7:0/4139941042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589877147a0 0x558987714bc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd54809b060 tx=0x7fd54809ad20 comp rx=0 tx=0).stop 2024-06-10T01:09:57.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.436+0000 7fd561e1ab80 1 -- 172.21.15.7:0/4139941042 shutdown_connections 2024-06-10T01:09:57.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.436+0000 7fd561e1ab80 1 --2- 172.21.15.7:0/4139941042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589877147a0 0x558987714bc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.436+0000 7fd561e1ab80 1 --2- 172.21.15.7:0/4139941042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558987719f60 0x55898771a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.436+0000 7fd561e1ab80 1 --2- 172.21.15.7:0/4139941042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55898771a880 0x558987714260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.436+0000 7fd561e1ab80 1 -- 172.21.15.7:0/4139941042 >> 172.21.15.7:0/4139941042 conn(0x558987641080 msgr2=0x55898770f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:57.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.436+0000 7fd561e1ab80 1 -- 172.21.15.7:0/4139941042 shutdown_connections 2024-06-10T01:09:57.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.436+0000 7fd561e1ab80 1 -- 172.21.15.7:0/4139941042 wait complete. 2024-06-10T01:09:57.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.436+0000 7fd561e1ab80 1 -- start start 2024-06-10T01:09:57.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.436+0000 7fd561e1ab80 1 Processor -- start 2024-06-10T01:09:57.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.437+0000 7fd561e1ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589877147a0 0x5589877205b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.437+0000 7fd561e1ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558987719f60 0x55898771d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.437+0000 7fd561e1ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55898771a880 0x55898771ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.437+0000 7fd561e1ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558987605fc0 con 0x55898771a880 2024-06-10T01:09:57.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.437+0000 7fd561e1ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558987661810 con 0x5589877147a0 2024-06-10T01:09:57.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.437+0000 7fd561e1ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558987721200 con 0x558987719f60 2024-06-10T01:09:57.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.437+0000 7fd555b34700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55898771a880 0x55898771ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.437+0000 7fd554b32700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558987719f60 0x55898771d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.437+0000 7fd554b32700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558987719f60 0x55898771d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43304/0 (socket says 172.21.15.7:43304) 2024-06-10T01:09:57.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.437+0000 7fd554b32700 1 -- 172.21.15.7:0/3906630177 learned_addr learned my addr 172.21.15.7:0/3906630177 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:57.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.437+0000 7fd555333700 1 --2- 172.21.15.7:0/3906630177 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589877147a0 0x5589877205b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.437+0000 7fd555b34700 1 -- 172.21.15.7:0/3906630177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558987719f60 msgr2=0x55898771d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:57.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.437+0000 7fd555b34700 1 --2- 172.21.15.7:0/3906630177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558987719f60 0x55898771d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.437+0000 7fd555b34700 1 -- 172.21.15.7:0/3906630177 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589877147a0 msgr2=0x5589877205b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:57.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.437+0000 7fd555b34700 1 --2- 172.21.15.7:0/3906630177 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589877147a0 0x5589877205b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.437+0000 7fd555b34700 1 -- 172.21.15.7:0/3906630177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55898762b750 con 0x55898771a880 2024-06-10T01:09:57.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.438+0000 7fd561e1ab80 10 client.0 _ll_register_callbacks cb 0x5589875fb5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:57.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.438+0000 7fd561e1ab80 1 -- 172.21.15.7:0/3906630177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd54800de10 con 0x55898771a880 2024-06-10T01:09:57.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.438+0000 7fd555b34700 1 --2- 172.21.15.7:0/3906630177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55898771a880 0x55898771ddf0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd5480974e0 tx=0x7fd54800d830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78c3d1fd1d3d4b7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:57.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.438+0000 7fd538ff9700 10 client.16455 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:09:57.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.438+0000 7fd538ff9700 1 -- 172.21.15.7:0/3906630177 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54809b530 con 0x55898771a880 2024-06-10T01:09:57.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.438+0000 7fd538ff9700 1 -- 172.21.15.7:0/3906630177 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fd5480a38b0 con 0x55898771a880 2024-06-10T01:09:57.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.438+0000 7fd538ff9700 1 -- 172.21.15.7:0/3906630177 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd5480a3ab0 con 0x55898771a880 2024-06-10T01:09:57.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.438+0000 7fd561e1ab80 10 client.16455 fetch_fsmap learned FSMap version 30 2024-06-10T01:09:57.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.438+0000 7fd561e1ab80 1 -- 172.21.15.7:0/3906630177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=30}) -- 0x558987733ec0 con 0x55898771a880 2024-06-10T01:09:57.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.439+0000 7fd538ff9700 1 -- 172.21.15.7:0/3906630177 <== mon.0 v2:172.21.15.7:3300/0 4 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fd54809a520 con 0x55898771a880 2024-06-10T01:09:57.437 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:57 smithi007 ceph-mon[32398]: pgmap v550: 129 pgs: 129 active+clean; 63 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 364 op/s 2024-06-10T01:09:57.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.439+0000 7fd561e1ab80 10 client.16455 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:09:57.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.439+0000 7fd561e1ab80 10 client.16455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:57.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.439+0000 7fd561e1ab80 20 client.16455 populate_metadata read hostname 'smithi007' 2024-06-10T01:09:57.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.439+0000 7fd561e1ab80 1 --2- 172.21.15.7:0/3906630177 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558987736390 0x558987738780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.439+0000 7fd561e1ab80 4 client.16455 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:09:57.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.439+0000 7fd561e1ab80 1 -- 172.21.15.7:0/3906630177 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558987672d50 con 0x558987736390 2024-06-10T01:09:57.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.439+0000 7fd555333700 1 --2- 172.21.15.7:0/3906630177 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558987736390 0x558987738780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.440+0000 7fd555333700 1 --2- 172.21.15.7:0/3906630177 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558987736390 0x558987738780 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e14b12628a1f714 server_cookie=28479a39489e6c17 in_seq=0 out_seq=0 2024-06-10T01:09:57.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.440+0000 7fd538ff9700 10 client.16455 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:57.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.441+0000 7fd538ff9700 1 -- 172.21.15.7:0/3906630177 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558987672d50 con 0x558987736390 2024-06-10T01:09:57.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.441+0000 7fd538ff9700 10 client.16455 handle_command_reply: tid=0 2024-06-10T01:09:57.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.441+0000 7fd561e1ab80 1 client.16455 shutdown 2024-06-10T01:09:57.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.441+0000 7fd561e1ab80 1 -- 172.21.15.7:0/3906630177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55898771a880 msgr2=0x55898771ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:57.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.441+0000 7fd561e1ab80 1 --2- 172.21.15.7:0/3906630177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55898771a880 0x55898771ddf0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd5480974e0 tx=0x7fd54800d830 comp rx=0 tx=0).stop 2024-06-10T01:09:57.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.441+0000 7fd561e1ab80 1 -- 172.21.15.7:0/3906630177 shutdown_connections 2024-06-10T01:09:57.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.442+0000 7fd561e1ab80 1 --2- 172.21.15.7:0/3906630177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55898771a880 0x55898771ddf0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.442+0000 7fd561e1ab80 1 --2- 172.21.15.7:0/3906630177 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558987736390 0x558987738780 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.442+0000 7fd561e1ab80 1 --2- 172.21.15.7:0/3906630177 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589877147a0 0x5589877205b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.442+0000 7fd561e1ab80 1 --2- 172.21.15.7:0/3906630177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558987719f60 0x55898771d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.442+0000 7fd561e1ab80 1 -- 172.21.15.7:0/3906630177 >> 172.21.15.7:0/3906630177 conn(0x558987641080 msgr2=0x55898770e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:57.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.442+0000 7fd538ff9700 0 client.16455 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:57.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.442+0000 7fd561e1ab80 1 -- 172.21.15.7:0/3906630177 shutdown_connections 2024-06-10T01:09:57.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.442+0000 7fd561e1ab80 1 -- 172.21.15.7:0/3906630177 wait complete. 2024-06-10T01:09:57.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.442+0000 7fd561e1ab80 20 client.16455 trim_cache size 0 max 16384 2024-06-10T01:09:57.442 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:09:57.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.466+0000 7fd561e1ab80 1 Processor -- start 2024-06-10T01:09:57.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.466+0000 7fd561e1ab80 1 -- start start 2024-06-10T01:09:57.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.466+0000 7fd561e1ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589877147a0 0x558987714260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.466+0000 7fd561e1ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558987719f60 0x55898770ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.466+0000 7fd561e1ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55898771a880 0x55898770f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.466+0000 7fd561e1ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd534011360 con 0x5589877147a0 2024-06-10T01:09:57.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.466+0000 7fd561e1ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd53400eb90 con 0x55898771a880 2024-06-10T01:09:57.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.466+0000 7fd561e1ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd53400c9e0 con 0x558987719f60 2024-06-10T01:09:57.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.467+0000 7fd555333700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589877147a0 0x558987714260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.467+0000 7fd555333700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589877147a0 0x558987714260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48600/0 (socket says 172.21.15.7:48600) 2024-06-10T01:09:57.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.467+0000 7fd555333700 1 -- 172.21.15.7:0/3219592379 learned_addr learned my addr 172.21.15.7:0/3219592379 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:57.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.467+0000 7fd554b32700 1 --2- 172.21.15.7:0/3219592379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558987719f60 0x55898770ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.467+0000 7fd555b34700 1 --2- 172.21.15.7:0/3219592379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55898771a880 0x55898770f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.467+0000 7fd555333700 1 -- 172.21.15.7:0/3219592379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558987719f60 msgr2=0x55898770ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:57.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.467+0000 7fd555333700 1 --2- 172.21.15.7:0/3219592379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558987719f60 0x55898770ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.467+0000 7fd555333700 1 -- 172.21.15.7:0/3219592379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55898771a880 msgr2=0x55898770f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:57.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.467+0000 7fd555333700 1 --2- 172.21.15.7:0/3219592379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55898771a880 0x55898770f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.467+0000 7fd555333700 1 -- 172.21.15.7:0/3219592379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55898771be20 con 0x5589877147a0 2024-06-10T01:09:57.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.467+0000 7fd555333700 1 --2- 172.21.15.7:0/3219592379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589877147a0 0x558987714260 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd54c006010 tx=0x7fd54c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0176d8763ff532 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:57.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.468+0000 7fd53affd700 1 -- 172.21.15.7:0/3219592379 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c00bd80 con 0x5589877147a0 2024-06-10T01:09:57.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.468+0000 7fd53affd700 1 -- 172.21.15.7:0/3219592379 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd54c005510 con 0x5589877147a0 2024-06-10T01:09:57.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.468+0000 7fd53affd700 1 -- 172.21.15.7:0/3219592379 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c03c430 con 0x5589877147a0 2024-06-10T01:09:57.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.468+0000 7fd561e1ab80 1 -- 172.21.15.7:0/3219592379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589877147a0 msgr2=0x558987714260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:57.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.468+0000 7fd561e1ab80 1 --2- 172.21.15.7:0/3219592379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589877147a0 0x558987714260 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd54c006010 tx=0x7fd54c020000 comp rx=0 tx=0).stop 2024-06-10T01:09:57.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.469+0000 7fd561e1ab80 1 -- 172.21.15.7:0/3219592379 shutdown_connections 2024-06-10T01:09:57.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.469+0000 7fd561e1ab80 1 --2- 172.21.15.7:0/3219592379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55898771a880 0x55898770f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.469+0000 7fd561e1ab80 1 --2- 172.21.15.7:0/3219592379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589877147a0 0x558987714260 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.469+0000 7fd561e1ab80 1 --2- 172.21.15.7:0/3219592379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558987719f60 0x55898770ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.469+0000 7fd561e1ab80 1 -- 172.21.15.7:0/3219592379 >> 172.21.15.7:0/3219592379 conn(0x558987641080 msgr2=0x5589877179e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:57.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.469+0000 7fd561e1ab80 1 -- 172.21.15.7:0/3219592379 shutdown_connections 2024-06-10T01:09:57.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.469+0000 7fd561e1ab80 1 -- 172.21.15.7:0/3219592379 wait complete. 2024-06-10T01:09:57.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.469+0000 7fd561e1ab80 1 -- start start 2024-06-10T01:09:57.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.469+0000 7fd561e1ab80 1 Processor -- start 2024-06-10T01:09:57.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.470+0000 7fd561e1ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589877147a0 0x558987726df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.470+0000 7fd561e1ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558987719f60 0x55898773d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.470+0000 7fd561e1ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55898771a880 0x55898773df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.470+0000 7fd561e1ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd53400d050 con 0x55898771a880 2024-06-10T01:09:57.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.470+0000 7fd561e1ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd53400e4c0 con 0x5589877147a0 2024-06-10T01:09:57.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.470+0000 7fd561e1ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd53400d720 con 0x558987719f60 2024-06-10T01:09:57.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.470+0000 7fd561e1ab80 10 client.0 _ll_register_callbacks cb 0x5589875fb5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:57.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.470+0000 7fd555b34700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55898771a880 0x55898773df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.470+0000 7fd554b32700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558987719f60 0x55898773d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.470+0000 7fd554b32700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558987719f60 0x55898773d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43328/0 (socket says 172.21.15.7:43328) 2024-06-10T01:09:57.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.470+0000 7fd555333700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589877147a0 0x558987726df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.470+0000 7fd555b34700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55898771a880 0x55898773df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48612/0 (socket says 172.21.15.7:48612) 2024-06-10T01:09:57.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.470+0000 7fd554b32700 1 -- 172.21.15.7:0/724424827 learned_addr learned my addr 172.21.15.7:0/724424827 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:09:57.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.471+0000 7fd555b34700 1 -- 172.21.15.7:0/724424827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558987719f60 msgr2=0x55898773d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:57.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.471+0000 7fd555b34700 1 --2- 172.21.15.7:0/724424827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558987719f60 0x55898773d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.471+0000 7fd555b34700 1 -- 172.21.15.7:0/724424827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589877147a0 msgr2=0x558987726df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:57.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.471+0000 7fd555b34700 1 --2- 172.21.15.7:0/724424827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589877147a0 0x558987726df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.471+0000 7fd555b34700 1 -- 172.21.15.7:0/724424827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55898779fe10 con 0x55898771a880 2024-06-10T01:09:57.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.471+0000 7fd555333700 1 --2- 172.21.15.7:0/724424827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589877147a0 0x558987726df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:57.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.471+0000 7fd555b34700 1 --2- 172.21.15.7:0/724424827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55898771a880 0x55898773df30 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd54809add0 tx=0x7fd5480973f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c88d7fd662d707 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:57.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.471+0000 7fd538ff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:09:57.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.471+0000 7fd538ff9700 1 -- 172.21.15.7:0/724424827 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54809b0d0 con 0x55898771a880 2024-06-10T01:09:57.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.471+0000 7fd538ff9700 1 -- 172.21.15.7:0/724424827 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fd54809b330 con 0x55898771a880 2024-06-10T01:09:57.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.471+0000 7fd561e1ab80 1 -- 172.21.15.7:0/724424827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd54c005510 con 0x55898771a880 2024-06-10T01:09:57.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.472+0000 7fd538ff9700 1 -- 172.21.15.7:0/724424827 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd54809b530 con 0x55898771a880 2024-06-10T01:09:57.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.472+0000 7fd561e1ab80 10 client.16461 fetch_fsmap learned FSMap version 30 2024-06-10T01:09:57.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.472+0000 7fd561e1ab80 1 -- 172.21.15.7:0/724424827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5589877a0f00 con 0x55898771a880 2024-06-10T01:09:57.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.472+0000 7fd538ff9700 1 -- 172.21.15.7:0/724424827 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd54c005510 con 0x55898771a880 2024-06-10T01:09:57.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.472+0000 7fd538ff9700 1 -- 172.21.15.7:0/724424827 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fd548005000 con 0x55898771a880 2024-06-10T01:09:57.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.472+0000 7fd561e1ab80 10 client.16461 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:09:57.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.472+0000 7fd561e1ab80 10 client.16461 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:57.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.472+0000 7fd561e1ab80 20 client.16461 populate_metadata read hostname 'smithi007' 2024-06-10T01:09:57.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.472+0000 7fd561e1ab80 1 --2- 172.21.15.7:0/724424827 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5589877a13e0 0x5589877a3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.472+0000 7fd561e1ab80 4 client.16461 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:09:57.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.472+0000 7fd561e1ab80 1 -- 172.21.15.7:0/724424827 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55898766d2d0 con 0x5589877a13e0 2024-06-10T01:09:57.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.472+0000 7fd555333700 1 --2- 172.21.15.7:0/724424827 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5589877a13e0 0x5589877a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.473+0000 7fd555333700 1 --2- 172.21.15.7:0/724424827 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5589877a13e0 0x5589877a3800 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95a2dced982346f server_cookie=d4fd67e33d6cdd3a in_seq=0 out_seq=0 2024-06-10T01:09:57.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.473+0000 7fd538ff9700 10 client.16461 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:57.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.474+0000 7fd538ff9700 1 -- 172.21.15.7:0/724424827 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55898766d2d0 con 0x5589877a13e0 2024-06-10T01:09:57.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.474+0000 7fd538ff9700 10 client.16461 handle_command_reply: tid=0 2024-06-10T01:09:57.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.474+0000 7fd561e1ab80 1 client.16461 shutdown 2024-06-10T01:09:57.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.474+0000 7fd561e1ab80 1 -- 172.21.15.7:0/724424827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55898771a880 msgr2=0x55898773df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:57.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.474+0000 7fd561e1ab80 1 --2- 172.21.15.7:0/724424827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55898771a880 0x55898773df30 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd54809add0 tx=0x7fd5480973f0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.474+0000 7fd561e1ab80 1 -- 172.21.15.7:0/724424827 shutdown_connections 2024-06-10T01:09:57.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.474+0000 7fd561e1ab80 1 --2- 172.21.15.7:0/724424827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55898771a880 0x55898773df30 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.474+0000 7fd561e1ab80 1 --2- 172.21.15.7:0/724424827 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5589877a13e0 0x5589877a3800 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.474+0000 7fd561e1ab80 1 --2- 172.21.15.7:0/724424827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589877147a0 0x558987726df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.474+0000 7fd561e1ab80 1 --2- 172.21.15.7:0/724424827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558987719f60 0x55898773d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.474+0000 7fd561e1ab80 1 -- 172.21.15.7:0/724424827 >> 172.21.15.7:0/724424827 conn(0x558987641080 msgr2=0x55898771c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:57.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.474+0000 7fd538ff9700 0 client.16461 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:09:57.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.475+0000 7fd561e1ab80 1 -- 172.21.15.7:0/724424827 shutdown_connections 2024-06-10T01:09:57.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.475+0000 7fd561e1ab80 1 -- 172.21.15.7:0/724424827 wait complete. 2024-06-10T01:09:57.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.475+0000 7fd561e1ab80 20 client.16461 trim_cache size 0 max 16384 2024-06-10T01:09:57.473 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:09:57.473 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-10T01:09:57.473 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-10T01:09:57.473 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:09:57.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.475+0000 7fd557597700 1 -- 172.21.15.7:0/17518212 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd53c0a2c20 msgr2=0x7fd53c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:57.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.475+0000 7fd557597700 1 --2- 172.21.15.7:0/17518212 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd53c0a2c20 0x7fd53c0a50e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd54c00b2e0 tx=0x7fd54c009590 comp rx=0 tx=0).stop 2024-06-10T01:09:57.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.476+0000 7fd557597700 1 -- 172.21.15.7:0/17518212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5501073f0 msgr2=0x7fd550108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:57.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.476+0000 7fd557597700 1 --2- 172.21.15.7:0/17518212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5501073f0 0x7fd550108d50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd548007f10 tx=0x7fd54800d3b0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.476+0000 7fd557597700 1 -- 172.21.15.7:0/17518212 shutdown_connections 2024-06-10T01:09:57.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.476+0000 7fd557597700 1 --2- 172.21.15.7:0/17518212 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd53c0a2c20 0x7fd53c0a50e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.476+0000 7fd557597700 1 --2- 172.21.15.7:0/17518212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5501073f0 0x7fd550108d50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.476+0000 7fd557597700 1 --2- 172.21.15.7:0/17518212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd550101380 0x7fd5501082d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.476+0000 7fd557597700 1 --2- 172.21.15.7:0/17518212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd550101d40 0x7fd550108810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.476+0000 7fd557597700 1 -- 172.21.15.7:0/17518212 >> 172.21.15.7:0/17518212 conn(0x7fd550078300 msgr2=0x7fd5500ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:57.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.476+0000 7fd557597700 1 -- 172.21.15.7:0/17518212 shutdown_connections 2024-06-10T01:09:57.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:09:57.476+0000 7fd557597700 1 -- 172.21.15.7:0/17518212 wait complete. 2024-06-10T01:09:57.579 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:57 smithi078 ceph-mon[36683]: pgmap v550: 129 pgs: 129 active+clean; 63 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 364 op/s 2024-06-10T01:09:57.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:57 smithi175 ceph-mon[36636]: pgmap v550: 129 pgs: 129 active+clean; 63 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 364 op/s 2024-06-10T01:09:57.714 INFO:tasks.workunit.client.0.smithi007.stdout: 1 251349 10.90 MB/sec execute 38 sec latency 67.383 ms 2024-06-10T01:09:58.714 INFO:tasks.workunit.client.0.smithi007.stdout: 1 252782 10.81 MB/sec execute 39 sec latency 36.243 ms 2024-06-10T01:09:59.259 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:09:59.259 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:09:59.485 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:09:59 smithi007 ceph-mon[32398]: pgmap v551: 129 pgs: 129 active+clean; 63 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 367 op/s 2024-06-10T01:09:59.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:09:59 smithi078 ceph-mon[36683]: pgmap v551: 129 pgs: 129 active+clean; 63 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 367 op/s 2024-06-10T01:09:59.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:09:59 smithi175 ceph-mon[36636]: pgmap v551: 129 pgs: 129 active+clean; 63 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 367 op/s 2024-06-10T01:09:59.714 INFO:tasks.workunit.client.0.smithi007.stdout: 1 255712 10.99 MB/sec execute 40 sec latency 9.566 ms 2024-06-10T01:09:59.898 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:10:00.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:00 smithi078 ceph-mon[36683]: overall HEALTH_OK 2024-06-10T01:10:00.677 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:00 smithi007 ceph-mon[32398]: overall HEALTH_OK 2024-06-10T01:10:00.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:00 smithi175 ceph-mon[36636]: overall HEALTH_OK 2024-06-10T01:10:00.714 INFO:tasks.workunit.client.0.smithi007.stdout: 1 257416 10.86 MB/sec execute 41 sec latency 13.966 ms 2024-06-10T01:10:01.578 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:01 smithi007 ceph-mon[32398]: pgmap v552: 129 pgs: 129 active+clean; 64 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 344 op/s 2024-06-10T01:10:01.579 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:01 smithi078 ceph-mon[36683]: pgmap v552: 129 pgs: 129 active+clean; 64 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 344 op/s 2024-06-10T01:10:01.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.600+0000 7f2df162f700 1 -- 172.21.15.7:0/158636744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dec103150 msgr2=0x7f2dec107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:01.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.600+0000 7f2df162f700 1 --2- 172.21.15.7:0/158636744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dec103150 0x7f2dec107140 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2de00099c0 tx=0x7f2de0009cd0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.600+0000 7f2df162f700 1 -- 172.21.15.7:0/158636744 shutdown_connections 2024-06-10T01:10:01.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.600+0000 7f2df162f700 1 --2- 172.21.15.7:0/158636744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dec107870 0x7f2dec10f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.600+0000 7f2df162f700 1 --2- 172.21.15.7:0/158636744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dec1027a0 0x7f2dec102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.600+0000 7f2df162f700 1 --2- 172.21.15.7:0/158636744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dec103150 0x7f2dec107140 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.600+0000 7f2df162f700 1 -- 172.21.15.7:0/158636744 >> 172.21.15.7:0/158636744 conn(0x7f2dec079570 msgr2=0x7f2dec079980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:01.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.600+0000 7f2df162f700 1 -- 172.21.15.7:0/158636744 shutdown_connections 2024-06-10T01:10:01.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.600+0000 7f2df162f700 1 -- 172.21.15.7:0/158636744 wait complete. 2024-06-10T01:10:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.601+0000 7f2df162f700 1 Processor -- start 2024-06-10T01:10:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.601+0000 7f2df162f700 1 -- start start 2024-06-10T01:10:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.601+0000 7f2df162f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dec1027a0 0x7f2dec19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.601+0000 7f2df162f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dec103150 0x7f2dec19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.601+0000 7f2df162f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dec107870 0x7f2dec198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.602+0000 7f2df162f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1127f0 con 0x7f2dec107870 2024-06-10T01:10:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.602+0000 7f2df162f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec112670 con 0x7f2dec1027a0 2024-06-10T01:10:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.602+0000 7f2df162f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec112970 con 0x7f2dec103150 2024-06-10T01:10:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.602+0000 7f2deaffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dec1027a0 0x7f2dec19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.602+0000 7f2deb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dec107870 0x7f2dec198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.602+0000 7f2dea7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dec103150 0x7f2dec19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.602+0000 7f2deaffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dec1027a0 0x7f2dec19e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:41810/0 (socket says 172.21.15.7:41810) 2024-06-10T01:10:01.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.602+0000 7f2deaffd700 1 -- 172.21.15.7:0/2709357946 learned_addr learned my addr 172.21.15.7:0/2709357946 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:01.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.602+0000 7f2dea7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dec103150 0x7f2dec19ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43342/0 (socket says 172.21.15.7:43342) 2024-06-10T01:10:01.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.602+0000 7f2deaffd700 1 -- 172.21.15.7:0/2709357946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dec103150 msgr2=0x7f2dec19ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:01.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.602+0000 7f2deaffd700 1 --2- 172.21.15.7:0/2709357946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dec103150 0x7f2dec19ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.602+0000 7f2deaffd700 1 -- 172.21.15.7:0/2709357946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dec107870 msgr2=0x7f2dec198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:01.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.602+0000 7f2deaffd700 1 --2- 172.21.15.7:0/2709357946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dec107870 0x7f2dec198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.602+0000 7f2deaffd700 1 -- 172.21.15.7:0/2709357946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dec199170 con 0x7f2dec1027a0 2024-06-10T01:10:01.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.602+0000 7f2dea7fc700 1 --2- 172.21.15.7:0/2709357946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dec103150 0x7f2dec19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:01.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.602+0000 7f2deaffd700 1 --2- 172.21.15.7:0/2709357946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dec1027a0 0x7f2dec19e800 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc00e9c0 tx=0x7f2ddc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:01.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.602+0000 7f2dd3fff700 1 -- 172.21.15.7:0/2709357946 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc00cbd0 con 0x7f2dec1027a0 2024-06-10T01:10:01.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.602+0000 7f2dd3fff700 1 -- 172.21.15.7:0/2709357946 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ddc00cd70 con 0x7f2dec1027a0 2024-06-10T01:10:01.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.603+0000 7f2dd3fff700 1 -- 172.21.15.7:0/2709357946 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc010430 con 0x7f2dec1027a0 2024-06-10T01:10:01.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.603+0000 7f2df162f700 1 -- 172.21.15.7:0/2709357946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dec199460 con 0x7f2dec1027a0 2024-06-10T01:10:01.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.603+0000 7f2df162f700 1 -- 172.21.15.7:0/2709357946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dec0746c0 con 0x7f2dec1027a0 2024-06-10T01:10:01.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.605+0000 7f2dd3fff700 1 -- 172.21.15.7:0/2709357946 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f2ddc004750 con 0x7f2dec1027a0 2024-06-10T01:10:01.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.606+0000 7f2dd3fff700 1 --2- 172.21.15.7:0/2709357946 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2dd40a2d40 0x7f2dd40a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.606+0000 7f2dea7fc700 1 --2- 172.21.15.7:0/2709357946 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2dd40a2d40 0x7f2dd40a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.606+0000 7f2dd3fff700 1 -- 172.21.15.7:0/2709357946 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f2ddc0cf620 con 0x7f2dec1027a0 2024-06-10T01:10:01.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.606+0000 7f2dea7fc700 1 --2- 172.21.15.7:0/2709357946 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2dd40a2d40 0x7f2dd40a5200 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2dec108c50 tx=0x7f2de0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:01.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.607+0000 7f2dfbeb2b80 1 Processor -- start 2024-06-10T01:10:01.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.607+0000 7f2dfbeb2b80 1 -- start start 2024-06-10T01:10:01.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.607+0000 7f2dfbeb2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d44f71ccd0 0x55d44f71d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.607+0000 7f2dfbeb2b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d44f71d5f0 0x55d44f716f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.607+0000 7f2dfbeb2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d44f717460 0x55d44f7178a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.607+0000 7f2dfbeb2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d44f6689a0 con 0x55d44f71ccd0 2024-06-10T01:10:01.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.607+0000 7f2dfbeb2b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d44f6108b0 con 0x55d44f71d5f0 2024-06-10T01:10:01.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.607+0000 7f2dfbeb2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d44f664960 con 0x55d44f717460 2024-06-10T01:10:01.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.607+0000 7f2dea7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d44f71d5f0 0x55d44f716f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.607+0000 7f2dea7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d44f71d5f0 0x55d44f716f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:41826/0 (socket says 172.21.15.7:41826) 2024-06-10T01:10:01.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.607+0000 7f2dea7fc700 1 -- 172.21.15.7:0/128026994 learned_addr learned my addr 172.21.15.7:0/128026994 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:01.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.607+0000 7f2deb7fe700 1 --2- 172.21.15.7:0/128026994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d44f717460 0x55d44f7178a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.607+0000 7f2deaffd700 1 --2- 172.21.15.7:0/128026994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d44f71ccd0 0x55d44f71d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.607+0000 7f2dea7fc700 1 -- 172.21.15.7:0/128026994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d44f717460 msgr2=0x55d44f7178a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:01.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.607+0000 7f2dea7fc700 1 --2- 172.21.15.7:0/128026994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d44f717460 0x55d44f7178a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.607+0000 7f2dea7fc700 1 -- 172.21.15.7:0/128026994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d44f71ccd0 msgr2=0x55d44f71d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:01.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.607+0000 7f2dea7fc700 1 --2- 172.21.15.7:0/128026994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d44f71ccd0 0x55d44f71d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.607+0000 7f2dea7fc700 1 -- 172.21.15.7:0/128026994 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d44f6070e0 con 0x55d44f71d5f0 2024-06-10T01:10:01.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.607+0000 7f2deaffd700 1 --2- 172.21.15.7:0/128026994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d44f71ccd0 0x55d44f71d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:01.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.608+0000 7f2dea7fc700 1 --2- 172.21.15.7:0/128026994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d44f71d5f0 0x55d44f716f20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2de00099c0 tx=0x7f2de0020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc9f431845626160 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:01.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.608+0000 7f2dd17fa700 1 -- 172.21.15.7:0/128026994 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de0003c40 con 0x55d44f71d5f0 2024-06-10T01:10:01.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.608+0000 7f2dd17fa700 1 -- 172.21.15.7:0/128026994 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de0003de0 con 0x55d44f71d5f0 2024-06-10T01:10:01.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.608+0000 7f2dd17fa700 1 -- 172.21.15.7:0/128026994 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de000fc60 con 0x55d44f71d5f0 2024-06-10T01:10:01.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.609+0000 7f2dfbeb2b80 1 -- 172.21.15.7:0/128026994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d44f71d5f0 msgr2=0x55d44f716f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:01.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.609+0000 7f2dfbeb2b80 1 --2- 172.21.15.7:0/128026994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d44f71d5f0 0x55d44f716f20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2de00099c0 tx=0x7f2de0020000 comp rx=0 tx=0).stop 2024-06-10T01:10:01.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.609+0000 7f2dfbeb2b80 1 -- 172.21.15.7:0/128026994 shutdown_connections 2024-06-10T01:10:01.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.609+0000 7f2dfbeb2b80 1 --2- 172.21.15.7:0/128026994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d44f717460 0x55d44f7178a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.609+0000 7f2dfbeb2b80 1 --2- 172.21.15.7:0/128026994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d44f71ccd0 0x55d44f71d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.609+0000 7f2dfbeb2b80 1 --2- 172.21.15.7:0/128026994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d44f71d5f0 0x55d44f716f20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.609+0000 7f2dfbeb2b80 1 -- 172.21.15.7:0/128026994 >> 172.21.15.7:0/128026994 conn(0x55d44f641e50 msgr2=0x55d44f7121b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:01.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.609+0000 7f2dfbeb2b80 1 -- 172.21.15.7:0/128026994 shutdown_connections 2024-06-10T01:10:01.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.609+0000 7f2dfbeb2b80 1 -- 172.21.15.7:0/128026994 wait complete. 2024-06-10T01:10:01.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.609+0000 7f2dfbeb2b80 1 -- start start 2024-06-10T01:10:01.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.609+0000 7f2dfbeb2b80 1 Processor -- start 2024-06-10T01:10:01.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.610+0000 7f2dfbeb2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d44f717460 0x55d44f7233a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.610+0000 7f2dfbeb2b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d44f71ccd0 0x55d44f7206a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.610+0000 7f2dfbeb2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d44f71d5f0 0x55d44f720be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.610+0000 7f2dfbeb2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d44f724000 con 0x55d44f717460 2024-06-10T01:10:01.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.610+0000 7f2dfbeb2b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d44f608d80 con 0x55d44f71ccd0 2024-06-10T01:10:01.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.610+0000 7f2dfbeb2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d44f724180 con 0x55d44f71d5f0 2024-06-10T01:10:01.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.610+0000 7f2deaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d44f717460 0x55d44f7233a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.610+0000 7f2deaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d44f717460 0x55d44f7233a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48632/0 (socket says 172.21.15.7:48632) 2024-06-10T01:10:01.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.610+0000 7f2deaffd700 1 -- 172.21.15.7:0/399956607 learned_addr learned my addr 172.21.15.7:0/399956607 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:01.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.610+0000 7f2dea7fc700 1 --2- 172.21.15.7:0/399956607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d44f71ccd0 0x55d44f7206a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.610+0000 7f2deb7fe700 1 --2- 172.21.15.7:0/399956607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d44f71d5f0 0x55d44f720be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.610+0000 7f2dfbeb2b80 10 client.0 _ll_register_callbacks cb 0x55d44f602ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:01.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.610+0000 7f2dea7fc700 1 -- 172.21.15.7:0/399956607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d44f71d5f0 msgr2=0x55d44f720be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:01.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.610+0000 7f2dea7fc700 1 --2- 172.21.15.7:0/399956607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d44f71d5f0 0x55d44f720be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.610+0000 7f2dea7fc700 1 -- 172.21.15.7:0/399956607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d44f717460 msgr2=0x55d44f7233a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:01.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.610+0000 7f2dea7fc700 1 --2- 172.21.15.7:0/399956607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d44f717460 0x55d44f7233a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.610+0000 7f2dea7fc700 1 -- 172.21.15.7:0/399956607 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d44f5b7a60 con 0x55d44f71ccd0 2024-06-10T01:10:01.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.610+0000 7f2deaffd700 1 --2- 172.21.15.7:0/399956607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d44f717460 0x55d44f7233a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:01.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.610+0000 7f2dea7fc700 1 --2- 172.21.15.7:0/399956607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d44f71ccd0 0x55d44f7206a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2de00098e0 tx=0x7f2de000fb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=389335482ed23dca server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:01.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.610+0000 7f2dceff5700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:10:01.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.611+0000 7f2dceff5700 1 -- 172.21.15.7:0/399956607 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de0003c40 con 0x55d44f71ccd0 2024-06-10T01:10:01.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.611+0000 7f2dceff5700 1 -- 172.21.15.7:0/399956607 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f2de0005460 con 0x55d44f71ccd0 2024-06-10T01:10:01.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.611+0000 7f2dfbeb2b80 1 -- 172.21.15.7:0/399956607 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2de0003de0 con 0x55d44f71ccd0 2024-06-10T01:10:01.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.611+0000 7f2dceff5700 1 -- 172.21.15.7:0/399956607 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2de0040050 con 0x55d44f71ccd0 2024-06-10T01:10:01.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.611+0000 7f2dfbeb2b80 10 client.25994 fetch_fsmap learned FSMap version 30 2024-06-10T01:10:01.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.611+0000 7f2dfbeb2b80 1 -- 172.21.15.7:0/399956607 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d44f72e880 con 0x55d44f71ccd0 2024-06-10T01:10:01.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.611+0000 7f2dceff5700 1 -- 172.21.15.7:0/399956607 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de0003de0 con 0x55d44f71ccd0 2024-06-10T01:10:01.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.611+0000 7f2dceff5700 1 -- 172.21.15.7:0/399956607 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f2de0024410 con 0x55d44f71ccd0 2024-06-10T01:10:01.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.611+0000 7f2dfbeb2b80 10 client.25994 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:10:01.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.611+0000 7f2dfbeb2b80 10 client.25994 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:10:01.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.611+0000 7f2dfbeb2b80 20 client.25994 populate_metadata read hostname 'smithi007' 2024-06-10T01:10:01.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.611+0000 7f2dfbeb2b80 1 --2- 172.21.15.7:0/399956607 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d44f730d50 0x55d44f733140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.611+0000 7f2dfbeb2b80 4 client.25994 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:10:01.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.612+0000 7f2dfbeb2b80 1 -- 172.21.15.7:0/399956607 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d44f675ea0 con 0x55d44f730d50 2024-06-10T01:10:01.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.612+0000 7f2deaffd700 1 --2- 172.21.15.7:0/399956607 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d44f730d50 0x55d44f733140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.612+0000 7f2deaffd700 1 --2- 172.21.15.7:0/399956607 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d44f730d50 0x55d44f733140 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6130211938db0829 server_cookie=e4c079e51e0727f9 in_seq=0 out_seq=0 2024-06-10T01:10:01.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.612+0000 7f2dceff5700 10 client.25994 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:01.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.613+0000 7f2dceff5700 1 -- 172.21.15.7:0/399956607 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d44f675ea0 con 0x55d44f730d50 2024-06-10T01:10:01.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.613+0000 7f2dceff5700 10 client.25994 handle_command_reply: tid=0 2024-06-10T01:10:01.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.613+0000 7f2dfbeb2b80 1 client.25994 shutdown 2024-06-10T01:10:01.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.613+0000 7f2dfbeb2b80 1 -- 172.21.15.7:0/399956607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d44f71ccd0 msgr2=0x55d44f7206a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:01.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.613+0000 7f2dfbeb2b80 1 --2- 172.21.15.7:0/399956607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d44f71ccd0 0x55d44f7206a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2de00098e0 tx=0x7f2de000fb10 comp rx=0 tx=0).stop 2024-06-10T01:10:01.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.613+0000 7f2dfbeb2b80 1 -- 172.21.15.7:0/399956607 shutdown_connections 2024-06-10T01:10:01.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.613+0000 7f2dfbeb2b80 1 --2- 172.21.15.7:0/399956607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d44f71d5f0 0x55d44f720be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.613+0000 7f2dfbeb2b80 1 --2- 172.21.15.7:0/399956607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d44f717460 0x55d44f7233a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.613+0000 7f2dfbeb2b80 1 --2- 172.21.15.7:0/399956607 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d44f730d50 0x55d44f733140 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.613+0000 7f2dfbeb2b80 1 --2- 172.21.15.7:0/399956607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d44f71ccd0 0x55d44f7206a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.613+0000 7f2dceff5700 0 client.25994 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:01.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.613+0000 7f2dfbeb2b80 1 -- 172.21.15.7:0/399956607 >> 172.21.15.7:0/399956607 conn(0x55d44f641e50 msgr2=0x55d44f711da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:01.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.613+0000 7f2dfbeb2b80 1 -- 172.21.15.7:0/399956607 shutdown_connections 2024-06-10T01:10:01.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.613+0000 7f2dfbeb2b80 1 -- 172.21.15.7:0/399956607 wait complete. 2024-06-10T01:10:01.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.613+0000 7f2dfbeb2b80 20 client.25994 trim_cache size 0 max 16384 2024-06-10T01:10:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.624+0000 7f5bcd4ae640 1 -- 172.21.15.7:0/1640593757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bc81024c0 msgr2=0x7f5bc81048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.624+0000 7f5bcd4ae640 1 --2- 172.21.15.7:0/1640593757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bc81024c0 0x7f5bc81048b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009be0 tx=0x7f5bc002a6c0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.624+0000 7f5bcd4ae640 1 -- 172.21.15.7:0/1640593757 shutdown_connections 2024-06-10T01:10:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.624+0000 7f5bcd4ae640 1 --2- 172.21.15.7:0/1640593757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bc8107720 0x7f5bc8109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.624+0000 7f5bcd4ae640 1 --2- 172.21.15.7:0/1640593757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bc8104df0 0x7f5bc81071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.624+0000 7f5bcd4ae640 1 --2- 172.21.15.7:0/1640593757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bc81024c0 0x7f5bc81048b0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.624+0000 7f5bcd4ae640 1 -- 172.21.15.7:0/1640593757 >> 172.21.15.7:0/1640593757 conn(0x7f5bc80fbed0 msgr2=0x7f5bc80fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:01.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.624+0000 7f5bcd4ae640 1 -- 172.21.15.7:0/1640593757 shutdown_connections 2024-06-10T01:10:01.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.624+0000 7f5bcd4ae640 1 -- 172.21.15.7:0/1640593757 wait complete. 2024-06-10T01:10:01.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.625+0000 7f5bcd4ae640 1 Processor -- start 2024-06-10T01:10:01.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.625+0000 7f5bcd4ae640 1 -- start start 2024-06-10T01:10:01.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.625+0000 7f5bcd4ae640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bc81024c0 0x7f5bc8101e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.625+0000 7f5bcd4ae640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bc8104df0 0x7f5bc8100470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.625+0000 7f5bcd4ae640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bc8107720 0x7f5bc81009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.625+0000 7f5bcd4ae640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc810d6a0 con 0x7f5bc81024c0 2024-06-10T01:10:01.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.625+0000 7f5bcd4ae640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc810d520 con 0x7f5bc8107720 2024-06-10T01:10:01.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.625+0000 7f5bcd4ae640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc810d820 con 0x7f5bc8104df0 2024-06-10T01:10:01.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.625+0000 7f5bc6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bc81024c0 0x7f5bc8101e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.625+0000 7f5bc6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bc81024c0 0x7f5bc8101e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48636/0 (socket says 172.21.15.7:48636) 2024-06-10T01:10:01.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.625+0000 7f5bc6ffd640 1 -- 172.21.15.7:0/3802486035 learned_addr learned my addr 172.21.15.7:0/3802486035 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:01.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.625+0000 7f5bc67fc640 1 --2- 172.21.15.7:0/3802486035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bc8104df0 0x7f5bc8100470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.625+0000 7f5bc77fe640 1 --2- 172.21.15.7:0/3802486035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bc8107720 0x7f5bc81009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.625+0000 7f5bc6ffd640 1 -- 172.21.15.7:0/3802486035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bc8104df0 msgr2=0x7f5bc8100470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:01.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.625+0000 7f5bc6ffd640 1 --2- 172.21.15.7:0/3802486035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bc8104df0 0x7f5bc8100470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.625+0000 7f5bc6ffd640 1 -- 172.21.15.7:0/3802486035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bc8107720 msgr2=0x7f5bc81009b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:01.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.625+0000 7f5bc6ffd640 1 --2- 172.21.15.7:0/3802486035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bc8107720 0x7f5bc81009b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.625+0000 7f5bc6ffd640 1 -- 172.21.15.7:0/3802486035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc8101240 con 0x7f5bc81024c0 2024-06-10T01:10:01.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.625+0000 7f5bc77fe640 1 --2- 172.21.15.7:0/3802486035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bc8107720 0x7f5bc81009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:01.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.626+0000 7f5bc6ffd640 1 --2- 172.21.15.7:0/3802486035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bc81024c0 0x7f5bc8101e30 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009d00 tx=0x7f5bc0030630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:01.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.626+0000 7f5bbffff640 1 -- 172.21.15.7:0/3802486035 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc0041020 con 0x7f5bc81024c0 2024-06-10T01:10:01.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.626+0000 7f5bbffff640 1 -- 172.21.15.7:0/3802486035 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bc0037040 con 0x7f5bc81024c0 2024-06-10T01:10:01.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.626+0000 7f5bcd4ae640 1 -- 172.21.15.7:0/3802486035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bc81a1e40 con 0x7f5bc81024c0 2024-06-10T01:10:01.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.626+0000 7f5bbffff640 1 -- 172.21.15.7:0/3802486035 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc0036070 con 0x7f5bc81024c0 2024-06-10T01:10:01.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.626+0000 7f5bcd4ae640 1 -- 172.21.15.7:0/3802486035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bc81a2250 con 0x7f5bc81024c0 2024-06-10T01:10:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.628+0000 7f5bbffff640 1 -- 172.21.15.7:0/3802486035 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f5bc0044020 con 0x7f5bc81024c0 2024-06-10T01:10:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.629+0000 7f5bd01b6740 1 Processor -- start 2024-06-10T01:10:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.629+0000 7f5bd01b6740 1 -- start start 2024-06-10T01:10:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.629+0000 7f5bd01b6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617f636f010 0x5617f636f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.629+0000 7f5bd01b6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617f636fa00 0x5617f6374500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.629+0000 7f5bd01b6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617f6374b70 0x5617f6377000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.629+0000 7f5bd01b6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5617f62d24f0 con 0x5617f636fa00 2024-06-10T01:10:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.629+0000 7f5bd01b6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5617f6270870 con 0x5617f636f010 2024-06-10T01:10:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.629+0000 7f5bd01b6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5617f62ce4b0 con 0x5617f6374b70 2024-06-10T01:10:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.629+0000 7f5bc6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617f636fa00 0x5617f6374500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.629+0000 7f5bc77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617f636f010 0x5617f636f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.629+0000 7f5bc6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617f636fa00 0x5617f6374500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48650/0 (socket says 172.21.15.7:48650) 2024-06-10T01:10:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.629+0000 7f5bc6ffd640 1 -- 172.21.15.7:0/2025964390 learned_addr learned my addr 172.21.15.7:0/2025964390 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.629+0000 7f5bc67fc640 1 --2- 172.21.15.7:0/2025964390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617f6374b70 0x5617f6377000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.630+0000 7f5bbffff640 1 --2- 172.21.15.7:0/3802486035 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5bb40a2b80 0x7f5bb40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.630+0000 7f5bc6ffd640 1 -- 172.21.15.7:0/2025964390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617f6374b70 msgr2=0x5617f6377000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.630+0000 7f5bc6ffd640 1 --2- 172.21.15.7:0/2025964390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617f6374b70 0x5617f6377000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.630+0000 7f5bc6ffd640 1 -- 172.21.15.7:0/2025964390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617f636f010 msgr2=0x5617f636f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.630+0000 7f5bc6ffd640 1 --2- 172.21.15.7:0/2025964390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617f636f010 0x5617f636f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.630+0000 7f5bc6ffd640 1 -- 172.21.15.7:0/2025964390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617f628c9b0 con 0x5617f636fa00 2024-06-10T01:10:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.630+0000 7f5bc67fc640 1 --2- 172.21.15.7:0/2025964390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617f6374b70 0x5617f6377000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.630+0000 7f5bc77fe640 1 --2- 172.21.15.7:0/3802486035 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5bb40a2b80 0x7f5bb40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.630+0000 7f5bbffff640 1 -- 172.21.15.7:0/3802486035 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f5bc00efe70 con 0x7f5bc81024c0 2024-06-10T01:10:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.630+0000 7f5bc6ffd640 1 --2- 172.21.15.7:0/2025964390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617f636fa00 0x5617f6374500 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5bc008cb80 tx=0x7f5bc008b2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=addcb9d5b4e8b7be server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.630+0000 7f5bbd7fa640 1 -- 172.21.15.7:0/2025964390 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc00bbcb0 con 0x5617f636fa00 2024-06-10T01:10:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.630+0000 7f5bbd7fa640 1 -- 172.21.15.7:0/2025964390 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bc008d840 con 0x5617f636fa00 2024-06-10T01:10:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.630+0000 7f5bbd7fa640 1 -- 172.21.15.7:0/2025964390 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc008d130 con 0x5617f636fa00 2024-06-10T01:10:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.631+0000 7f5bd01b6740 1 -- 172.21.15.7:0/2025964390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617f636fa00 msgr2=0x5617f6374500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.631+0000 7f5bd01b6740 1 --2- 172.21.15.7:0/2025964390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617f636fa00 0x5617f6374500 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5bc008cb80 tx=0x7f5bc008b2d0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.631+0000 7f5bc77fe640 1 --2- 172.21.15.7:0/3802486035 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5bb40a2b80 0x7f5bb40a5040 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x5617f6370260 tx=0x7f5bb8004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.631+0000 7f5bd01b6740 1 -- 172.21.15.7:0/2025964390 shutdown_connections 2024-06-10T01:10:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.631+0000 7f5bd01b6740 1 --2- 172.21.15.7:0/2025964390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617f6374b70 0x5617f6377000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.631+0000 7f5bd01b6740 1 --2- 172.21.15.7:0/2025964390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617f636fa00 0x5617f6374500 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.631+0000 7f5bd01b6740 1 --2- 172.21.15.7:0/2025964390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617f636f010 0x5617f636f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.631+0000 7f5bd01b6740 1 -- 172.21.15.7:0/2025964390 >> 172.21.15.7:0/2025964390 conn(0x5617f636a680 msgr2=0x5617f636aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.631+0000 7f5bd01b6740 1 -- 172.21.15.7:0/2025964390 shutdown_connections 2024-06-10T01:10:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.631+0000 7f5bd01b6740 1 -- 172.21.15.7:0/2025964390 wait complete. 2024-06-10T01:10:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.632+0000 7f5bd01b6740 1 -- start start 2024-06-10T01:10:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.632+0000 7f5bd01b6740 1 Processor -- start 2024-06-10T01:10:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.632+0000 7f5bd01b6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617f636f010 0x5617f6378ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.632+0000 7f5bd01b6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617f636fa00 0x5617f6379410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.632+0000 7f5bd01b6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617f6374b70 0x5617f6380450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.632+0000 7f5bd01b6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5617f63778e0 con 0x5617f6374b70 2024-06-10T01:10:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.632+0000 7f5bd01b6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5617f625bae0 con 0x5617f636fa00 2024-06-10T01:10:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.632+0000 7f5bd01b6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5617f6377a60 con 0x5617f636f010 2024-06-10T01:10:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.632+0000 7f5bd01b6740 10 client.0 _ll_register_callbacks cb 0x5617f62949b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.632+0000 7f5bc77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617f6374b70 0x5617f6380450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.632+0000 7f5bc67fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617f636fa00 0x5617f6379410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.632+0000 7f5bc6ffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617f636f010 0x5617f6378ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.632+0000 7f5bc67fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617f636fa00 0x5617f6379410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:41880/0 (socket says 172.21.15.7:41880) 2024-06-10T01:10:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.632+0000 7f5bc67fc640 1 -- 172.21.15.7:0/799672883 learned_addr learned my addr 172.21.15.7:0/799672883 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.632+0000 7f5bc6ffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617f636f010 0x5617f6378ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43410/0 (socket says 172.21.15.7:43410) 2024-06-10T01:10:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.633+0000 7f5bc67fc640 1 -- 172.21.15.7:0/799672883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617f636f010 msgr2=0x5617f6378ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.633+0000 7f5bc67fc640 1 --2- 172.21.15.7:0/799672883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617f636f010 0x5617f6378ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.633+0000 7f5bc67fc640 1 -- 172.21.15.7:0/799672883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617f6374b70 msgr2=0x5617f6380450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.633+0000 7f5bc67fc640 1 --2- 172.21.15.7:0/799672883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617f6374b70 0x5617f6380450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.633+0000 7f5bc67fc640 1 -- 172.21.15.7:0/799672883 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617f5fe6210 con 0x5617f636fa00 2024-06-10T01:10:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.633+0000 7f5bc67fc640 1 --2- 172.21.15.7:0/799672883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617f636fa00 0x5617f6379410 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0012700 tx=0x7f5bb0012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2a8eab114ef2e9d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.633+0000 7f5bae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:10:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.633+0000 7f5bae7fc640 1 -- 172.21.15.7:0/799672883 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0004050 con 0x5617f636fa00 2024-06-10T01:10:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.633+0000 7f5bd01b6740 1 -- 172.21.15.7:0/799672883 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5617f62e5f10 con 0x5617f636fa00 2024-06-10T01:10:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.633+0000 7f5bae7fc640 1 -- 172.21.15.7:0/799672883 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f5bb00041f0 con 0x5617f636fa00 2024-06-10T01:10:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.633+0000 7f5bae7fc640 1 -- 172.21.15.7:0/799672883 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5bb0025050 con 0x5617f636fa00 2024-06-10T01:10:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.633+0000 7f5bd01b6740 10 client.26006 fetch_fsmap learned FSMap version 30 2024-06-10T01:10:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.633+0000 7f5bd01b6740 1 -- 172.21.15.7:0/799672883 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5617f60b2620 con 0x5617f636fa00 2024-06-10T01:10:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.633+0000 7f5bae7fc640 1 -- 172.21.15.7:0/799672883 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5617f62e5f10 con 0x5617f636fa00 2024-06-10T01:10:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.634+0000 7f5bae7fc640 1 -- 172.21.15.7:0/799672883 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f5bb0003a40 con 0x5617f636fa00 2024-06-10T01:10:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.634+0000 7f5bd01b6740 10 client.26006 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:10:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.634+0000 7f5bd01b6740 10 client.26006 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:10:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.634+0000 7f5bd01b6740 20 client.26006 populate_metadata read hostname 'smithi007' 2024-06-10T01:10:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.634+0000 7f5bd01b6740 1 --2- 172.21.15.7:0/799672883 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5617f6384da0 0x5617f63871e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.634+0000 7f5bd01b6740 4 client.26006 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:10:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.634+0000 7f5bd01b6740 1 -- 172.21.15.7:0/799672883 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617f62dfc20 con 0x5617f6384da0 2024-06-10T01:10:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.634+0000 7f5bc6ffd640 1 --2- 172.21.15.7:0/799672883 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5617f6384da0 0x5617f63871e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.634+0000 7f5bc6ffd640 1 --2- 172.21.15.7:0/799672883 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5617f6384da0 0x5617f63871e0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a1ff29ec665968a server_cookie=bc8b0146a1d8821d in_seq=0 out_seq=0 2024-06-10T01:10:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.634+0000 7f5bae7fc640 10 client.26006 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.635+0000 7f5bae7fc640 1 -- 172.21.15.7:0/799672883 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5617f62dfc20 con 0x5617f6384da0 2024-06-10T01:10:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.635+0000 7f5bae7fc640 10 client.26006 handle_command_reply: tid=0 2024-06-10T01:10:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.635+0000 7f5bd01b6740 1 client.26006 shutdown 2024-06-10T01:10:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.635+0000 7f5bd01b6740 1 -- 172.21.15.7:0/799672883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617f636fa00 msgr2=0x5617f6379410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.635+0000 7f5bd01b6740 1 --2- 172.21.15.7:0/799672883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617f636fa00 0x5617f6379410 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0012700 tx=0x7f5bb0012bc0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.635+0000 7f5bd01b6740 1 -- 172.21.15.7:0/799672883 shutdown_connections 2024-06-10T01:10:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.635+0000 7f5bd01b6740 1 --2- 172.21.15.7:0/799672883 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5617f6384da0 0x5617f63871e0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.635+0000 7f5bd01b6740 1 --2- 172.21.15.7:0/799672883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617f6374b70 0x5617f6380450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.635+0000 7f5bd01b6740 1 --2- 172.21.15.7:0/799672883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617f636fa00 0x5617f6379410 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.635+0000 7f5bd01b6740 1 --2- 172.21.15.7:0/799672883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617f636f010 0x5617f6378ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.635+0000 7f5bd01b6740 1 -- 172.21.15.7:0/799672883 >> 172.21.15.7:0/799672883 conn(0x5617f636a680 msgr2=0x5617f63697e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.635+0000 7f5bae7fc640 0 client.26006 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.635+0000 7f5bd01b6740 1 -- 172.21.15.7:0/799672883 shutdown_connections 2024-06-10T01:10:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.635+0000 7f5bd01b6740 1 -- 172.21.15.7:0/799672883 wait complete. 2024-06-10T01:10:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.636+0000 7f5bd01b6740 20 client.26006 trim_cache size 0 max 16384 2024-06-10T01:10:01.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.638+0000 7f2dfbeb2b80 1 Processor -- start 2024-06-10T01:10:01.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.638+0000 7f2dfbeb2b80 1 -- start start 2024-06-10T01:10:01.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.638+0000 7f2dfbeb2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d44f717460 0x55d44f716f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.638+0000 7f2dfbeb2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d44f71ccd0 0x55d44f712150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.638+0000 7f2dfbeb2b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d44f71d5f0 0x55d44f71f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.638+0000 7f2dfbeb2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1ae8c0 con 0x55d44f71ccd0 2024-06-10T01:10:01.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.638+0000 7f2dfbeb2b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1ac0f0 con 0x55d44f71d5f0 2024-06-10T01:10:01.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.638+0000 7f2dfbeb2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1a9f40 con 0x55d44f717460 2024-06-10T01:10:01.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.638+0000 7f2dea7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d44f71ccd0 0x55d44f712150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.638+0000 7f2deaffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d44f717460 0x55d44f716f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.638+0000 7f2dea7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d44f71ccd0 0x55d44f712150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48654/0 (socket says 172.21.15.7:48654) 2024-06-10T01:10:01.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.638+0000 7f2dea7fc700 1 -- 172.21.15.7:0/3499220060 learned_addr learned my addr 172.21.15.7:0/3499220060 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:01.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.638+0000 7f2deb7fe700 1 --2- 172.21.15.7:0/3499220060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d44f71d5f0 0x55d44f71f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.638+0000 7f2dea7fc700 1 -- 172.21.15.7:0/3499220060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d44f717460 msgr2=0x55d44f716f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.638+0000 7f2dea7fc700 1 --2- 172.21.15.7:0/3499220060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d44f717460 0x55d44f716f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.638+0000 7f2dea7fc700 1 -- 172.21.15.7:0/3499220060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d44f71d5f0 msgr2=0x55d44f71f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.638+0000 7f2dea7fc700 1 --2- 172.21.15.7:0/3499220060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d44f71d5f0 0x55d44f71f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.638+0000 7f2dea7fc700 1 -- 172.21.15.7:0/3499220060 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d44f724060 con 0x55d44f71ccd0 2024-06-10T01:10:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.638+0000 7f2deaffd700 1 --2- 172.21.15.7:0/3499220060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d44f717460 0x55d44f716f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.638+0000 7f2deb7fe700 1 --2- 172.21.15.7:0/3499220060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d44f71d5f0 0x55d44f71f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:10:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.639+0000 7f2dea7fc700 1 --2- 172.21.15.7:0/3499220060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d44f71ccd0 0x55d44f712150 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2de00249a0 tx=0x7f2de000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=647e73aac1bf5e1c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.639+0000 7f2dd0ff9700 1 -- 172.21.15.7:0/3499220060 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de000fea0 con 0x55d44f71ccd0 2024-06-10T01:10:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.639+0000 7f2dd0ff9700 1 -- 172.21.15.7:0/3499220060 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de000fc30 con 0x55d44f71ccd0 2024-06-10T01:10:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.639+0000 7f2dd0ff9700 1 -- 172.21.15.7:0/3499220060 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de00054c0 con 0x55d44f71ccd0 2024-06-10T01:10:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.640+0000 7f2dfbeb2b80 1 -- 172.21.15.7:0/3499220060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d44f71ccd0 msgr2=0x55d44f712150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.640+0000 7f2dfbeb2b80 1 --2- 172.21.15.7:0/3499220060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d44f71ccd0 0x55d44f712150 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2de00249a0 tx=0x7f2de000bfd0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.640+0000 7f2dfbeb2b80 1 -- 172.21.15.7:0/3499220060 shutdown_connections 2024-06-10T01:10:01.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.640+0000 7f2dfbeb2b80 1 --2- 172.21.15.7:0/3499220060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d44f71d5f0 0x55d44f71f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.640+0000 7f2dfbeb2b80 1 --2- 172.21.15.7:0/3499220060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d44f717460 0x55d44f716f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.640+0000 7f2dfbeb2b80 1 --2- 172.21.15.7:0/3499220060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d44f71ccd0 0x55d44f712150 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.640+0000 7f2dfbeb2b80 1 -- 172.21.15.7:0/3499220060 >> 172.21.15.7:0/3499220060 conn(0x55d44f641e50 msgr2=0x55d44f71da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:01.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.640+0000 7f2dfbeb2b80 1 -- 172.21.15.7:0/3499220060 shutdown_connections 2024-06-10T01:10:01.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.640+0000 7f2dfbeb2b80 1 -- 172.21.15.7:0/3499220060 wait complete. 2024-06-10T01:10:01.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.640+0000 7f2dfbeb2b80 1 -- start start 2024-06-10T01:10:01.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.641+0000 7f2dfbeb2b80 1 Processor -- start 2024-06-10T01:10:01.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.641+0000 7f2dfbeb2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d44f717460 0x55d44f736690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.641+0000 7f2dfbeb2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d44f71ccd0 0x55d44f734a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.641+0000 7f2dfbeb2b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d44f71d5f0 0x55d44f734fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.641+0000 7f2dfbeb2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1aa5b0 con 0x55d44f71ccd0 2024-06-10T01:10:01.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.641+0000 7f2dfbeb2b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1aba20 con 0x55d44f71d5f0 2024-06-10T01:10:01.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.641+0000 7f2dfbeb2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1aac80 con 0x55d44f717460 2024-06-10T01:10:01.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.641+0000 7f2dea7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d44f71ccd0 0x55d44f734a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.641+0000 7f2dfbeb2b80 10 client.0 _ll_register_callbacks cb 0x55d44f602ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:01.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.641+0000 7f2deb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d44f71d5f0 0x55d44f734fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.641+0000 7f2dea7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d44f71ccd0 0x55d44f734a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48662/0 (socket says 172.21.15.7:48662) 2024-06-10T01:10:01.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.641+0000 7f2dea7fc700 1 -- 172.21.15.7:0/1242075005 learned_addr learned my addr 172.21.15.7:0/1242075005 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:01.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.641+0000 7f2deaffd700 1 --2- 172.21.15.7:0/1242075005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d44f717460 0x55d44f736690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.641+0000 7f2deb7fe700 1 -- 172.21.15.7:0/1242075005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d44f717460 msgr2=0x55d44f736690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:01.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.641+0000 7f2deb7fe700 1 --2- 172.21.15.7:0/1242075005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d44f717460 0x55d44f736690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.641+0000 7f2deb7fe700 1 -- 172.21.15.7:0/1242075005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d44f71ccd0 msgr2=0x55d44f734a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:01.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.641+0000 7f2deb7fe700 1 --2- 172.21.15.7:0/1242075005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d44f71ccd0 0x55d44f734a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.642+0000 7f2deb7fe700 1 -- 172.21.15.7:0/1242075005 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d44f7126f0 con 0x55d44f71d5f0 2024-06-10T01:10:01.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.642+0000 7f2dea7fc700 1 --2- 172.21.15.7:0/1242075005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d44f71ccd0 0x55d44f734a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:01.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.642+0000 7f2deb7fe700 1 --2- 172.21.15.7:0/1242075005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d44f71d5f0 0x55d44f734fd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2de4014fa0 tx=0x7f2de40077e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9f595764c7c583a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:01.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.642+0000 7f2dceff5700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:10:01.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.642+0000 7f2dceff5700 1 -- 172.21.15.7:0/1242075005 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de4004bb0 con 0x55d44f71d5f0 2024-06-10T01:10:01.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.642+0000 7f2dceff5700 1 -- 172.21.15.7:0/1242075005 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f2de4003a00 con 0x55d44f71d5f0 2024-06-10T01:10:01.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.642+0000 7f2dfbeb2b80 1 -- 172.21.15.7:0/1242075005 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2de000fc30 con 0x55d44f71d5f0 2024-06-10T01:10:01.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.642+0000 7f2dceff5700 1 -- 172.21.15.7:0/1242075005 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2de402d050 con 0x55d44f71d5f0 2024-06-10T01:10:01.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.642+0000 7f2dfbeb2b80 10 client.26012 fetch_fsmap learned FSMap version 30 2024-06-10T01:10:01.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.642+0000 7f2dfbeb2b80 1 -- 172.21.15.7:0/1242075005 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d44f79c300 con 0x55d44f71d5f0 2024-06-10T01:10:01.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.643+0000 7f2dceff5700 1 -- 172.21.15.7:0/1242075005 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de4003c40 con 0x55d44f71d5f0 2024-06-10T01:10:01.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.643+0000 7f2dceff5700 1 -- 172.21.15.7:0/1242075005 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f2de4004320 con 0x55d44f71d5f0 2024-06-10T01:10:01.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.643+0000 7f2dfbeb2b80 10 client.26012 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:10:01.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.643+0000 7f2dfbeb2b80 10 client.26012 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:10:01.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.643+0000 7f2dfbeb2b80 20 client.26012 populate_metadata read hostname 'smithi007' 2024-06-10T01:10:01.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.643+0000 7f2dfbeb2b80 1 --2- 172.21.15.7:0/1242075005 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d44f79cb60 0x55d44f79ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.643+0000 7f2dfbeb2b80 4 client.26012 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.2, "await_for": 45.0}] 2024-06-10T01:10:01.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.643+0000 7f2dfbeb2b80 1 -- 172.21.15.7:0/1242075005 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.2, "await_for": 45.0}) -- 0x7f2de000fc30 con 0x55d44f79cb60 2024-06-10T01:10:01.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.643+0000 7f2deaffd700 1 --2- 172.21.15.7:0/1242075005 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d44f79cb60 0x55d44f79ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.644+0000 7f2deaffd700 1 --2- 172.21.15.7:0/1242075005 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d44f79cb60 0x55d44f79ef80 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=495e9120999ae6c0 server_cookie=91c64ade1ba39f3e in_seq=0 out_seq=0 2024-06-10T01:10:01.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.644+0000 7f2dceff5700 10 client.26012 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.655+0000 7f5bd01b6740 1 Processor -- start 2024-06-10T01:10:01.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.655+0000 7f5bd01b6740 1 -- start start 2024-06-10T01:10:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.656+0000 7f5bd01b6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617f636f010 0x5617f6365560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.656+0000 7f5bd01b6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617f636fa00 0x5617f6380410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.656+0000 7f5bd01b6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617f6374b70 0x5617f6378ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.656+0000 7f5bd01b6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81af180 con 0x5617f636fa00 2024-06-10T01:10:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.656+0000 7f5bd01b6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81ac9f0 con 0x5617f6374b70 2024-06-10T01:10:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.656+0000 7f5bd01b6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81aa840 con 0x5617f636f010 2024-06-10T01:10:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.656+0000 7f5bc67fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617f636fa00 0x5617f6380410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.656+0000 7f5bc67fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617f636fa00 0x5617f6380410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48674/0 (socket says 172.21.15.7:48674) 2024-06-10T01:10:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.656+0000 7f5bc67fc640 1 -- 172.21.15.7:0/3932840575 learned_addr learned my addr 172.21.15.7:0/3932840575 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.656+0000 7f5bc6ffd640 1 --2- 172.21.15.7:0/3932840575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617f636f010 0x5617f6365560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.656+0000 7f5bc77fe640 1 --2- 172.21.15.7:0/3932840575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617f6374b70 0x5617f6378ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.657+0000 7f5bc67fc640 1 -- 172.21.15.7:0/3932840575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617f636f010 msgr2=0x5617f6365560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.657+0000 7f5bc67fc640 1 --2- 172.21.15.7:0/3932840575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617f636f010 0x5617f6365560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.657+0000 7f5bc67fc640 1 -- 172.21.15.7:0/3932840575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617f6374b70 msgr2=0x5617f6378ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.657+0000 7f5bc67fc640 1 --2- 172.21.15.7:0/3932840575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617f6374b70 0x5617f6378ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.657+0000 7f5bc67fc640 1 -- 172.21.15.7:0/3932840575 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617f636fde0 con 0x5617f636fa00 2024-06-10T01:10:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.657+0000 7f5bc6ffd640 1 --2- 172.21.15.7:0/3932840575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617f636f010 0x5617f6365560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.657+0000 7f5bc67fc640 1 --2- 172.21.15.7:0/3932840575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617f636fa00 0x5617f6380410 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0003de0 tx=0x7f5bb0005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3204069c862745 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.657+0000 7f5bbcff9640 1 -- 172.21.15.7:0/3932840575 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0004000 con 0x5617f636fa00 2024-06-10T01:10:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.658+0000 7f5bbcff9640 1 -- 172.21.15.7:0/3932840575 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bb002e070 con 0x5617f636fa00 2024-06-10T01:10:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.658+0000 7f5bbcff9640 1 -- 172.21.15.7:0/3932840575 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0003770 con 0x5617f636fa00 2024-06-10T01:10:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.658+0000 7f5bd01b6740 1 -- 172.21.15.7:0/3932840575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617f636fa00 msgr2=0x5617f6380410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.658+0000 7f5bd01b6740 1 --2- 172.21.15.7:0/3932840575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617f636fa00 0x5617f6380410 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0003de0 tx=0x7f5bb0005db0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.659+0000 7f5bd01b6740 1 -- 172.21.15.7:0/3932840575 shutdown_connections 2024-06-10T01:10:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.659+0000 7f5bd01b6740 1 --2- 172.21.15.7:0/3932840575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617f6374b70 0x5617f6378ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.659+0000 7f5bd01b6740 1 --2- 172.21.15.7:0/3932840575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617f636fa00 0x5617f6380410 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.659+0000 7f5bd01b6740 1 --2- 172.21.15.7:0/3932840575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617f636f010 0x5617f6365560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.659+0000 7f5bd01b6740 1 -- 172.21.15.7:0/3932840575 >> 172.21.15.7:0/3932840575 conn(0x5617f636a680 msgr2=0x5617f6374260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.659+0000 7f5bd01b6740 1 -- 172.21.15.7:0/3932840575 shutdown_connections 2024-06-10T01:10:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.659+0000 7f5bd01b6740 1 -- 172.21.15.7:0/3932840575 wait complete. 2024-06-10T01:10:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.659+0000 7f5bd01b6740 1 -- start start 2024-06-10T01:10:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.660+0000 7f5bd01b6740 1 Processor -- start 2024-06-10T01:10:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.660+0000 7f5bd01b6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617f636f010 0x5617f637fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.660+0000 7f5bd01b6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617f636fa00 0x5617f6380110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.660+0000 7f5bc6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617f636f010 0x5617f637fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.660+0000 7f5bc6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617f636f010 0x5617f637fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48684/0 (socket says 172.21.15.7:48684) 2024-06-10T01:10:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.660+0000 7f5bc6ffd640 1 -- 172.21.15.7:0/3704172178 learned_addr learned my addr 172.21.15.7:0/3704172178 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.660+0000 7f5bd01b6740 1 --2- 172.21.15.7:0/3704172178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617f6374b70 0x5617f637c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.660+0000 7f5bd01b6740 1 -- 172.21.15.7:0/3704172178 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81aaeb0 con 0x5617f636f010 2024-06-10T01:10:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.660+0000 7f5bd01b6740 1 -- 172.21.15.7:0/3704172178 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81ac320 con 0x5617f6374b70 2024-06-10T01:10:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.660+0000 7f5bd01b6740 1 -- 172.21.15.7:0/3704172178 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81ab580 con 0x5617f636fa00 2024-06-10T01:10:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.660+0000 7f5bd01b6740 10 client.0 _ll_register_callbacks cb 0x7f5bb0009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.660+0000 7f5bc67fc640 1 --2- 172.21.15.7:0/3704172178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617f636fa00 0x5617f6380110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.660+0000 7f5bc77fe640 1 --2- 172.21.15.7:0/3704172178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617f6374b70 0x5617f637c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.660+0000 7f5bc6ffd640 1 -- 172.21.15.7:0/3704172178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617f636fa00 msgr2=0x5617f6380110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.660+0000 7f5bc6ffd640 1 --2- 172.21.15.7:0/3704172178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617f636fa00 0x5617f6380110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.660+0000 7f5bc6ffd640 1 -- 172.21.15.7:0/3704172178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617f6374b70 msgr2=0x5617f637c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.660+0000 7f5bc6ffd640 1 --2- 172.21.15.7:0/3704172178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617f6374b70 0x5617f637c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.661+0000 7f5bc6ffd640 1 -- 172.21.15.7:0/3704172178 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617f637ccb0 con 0x5617f636f010 2024-06-10T01:10:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.661+0000 7f5bc77fe640 1 --2- 172.21.15.7:0/3704172178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617f6374b70 0x5617f637c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:10:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.661+0000 7f5bc6ffd640 1 --2- 172.21.15.7:0/3704172178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617f636f010 0x5617f637fbd0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f5bc00b8d20 tx=0x7f5bc00daf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=becdd39a7acad01 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.661+0000 7f5bae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:10:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.661+0000 7f5bae7fc640 1 -- 172.21.15.7:0/3704172178 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc00bca50 con 0x5617f636f010 2024-06-10T01:10:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.661+0000 7f5bae7fc640 1 -- 172.21.15.7:0/3704172178 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f5bc0003d80 con 0x5617f636f010 2024-06-10T01:10:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.661+0000 7f5bd01b6740 1 -- 172.21.15.7:0/3704172178 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bb002e070 con 0x5617f636f010 2024-06-10T01:10:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.661+0000 7f5bae7fc640 1 -- 172.21.15.7:0/3704172178 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5bc00a5020 con 0x5617f636f010 2024-06-10T01:10:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.661+0000 7f5bd01b6740 10 client.16497 fetch_fsmap learned FSMap version 30 2024-06-10T01:10:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.661+0000 7f5bd01b6740 1 -- 172.21.15.7:0/3704172178 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5617f63ee6f0 con 0x5617f636f010 2024-06-10T01:10:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.662+0000 7f5bae7fc640 1 -- 172.21.15.7:0/3704172178 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bb002e070 con 0x5617f636f010 2024-06-10T01:10:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.662+0000 7f5bae7fc640 1 -- 172.21.15.7:0/3704172178 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f5bc00b89c0 con 0x5617f636f010 2024-06-10T01:10:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.662+0000 7f5bd01b6740 10 client.16497 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:10:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.662+0000 7f5bd01b6740 10 client.16497 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:10:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.662+0000 7f5bd01b6740 20 client.16497 populate_metadata read hostname 'smithi007' 2024-06-10T01:10:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.662+0000 7f5bd01b6740 1 --2- 172.21.15.7:0/3704172178 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5617f63eef40 0x5617f63f1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.662+0000 7f5bd01b6740 4 client.16497 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:10:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.662+0000 7f5bd01b6740 1 -- 172.21.15.7:0/3704172178 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5617f62d9f70 con 0x5617f63eef40 2024-06-10T01:10:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.663+0000 7f5bc67fc640 1 --2- 172.21.15.7:0/3704172178 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5617f63eef40 0x5617f63f1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.668+0000 7f5bc67fc640 1 --2- 172.21.15.7:0/3704172178 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5617f63eef40 0x5617f63f1360 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98714389864f4a28 server_cookie=c363dea995d27268 in_seq=0 out_seq=0 2024-06-10T01:10:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.668+0000 7f5bae7fc640 10 client.16497 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:01.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:01 smithi175 ceph-mon[36636]: pgmap v552: 129 pgs: 129 active+clean; 64 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 344 op/s 2024-06-10T01:10:01.714 INFO:tasks.workunit.client.0.smithi007.stdout: 1 259415 10.93 MB/sec execute 42 sec latency 54.466 ms 2024-06-10T01:10:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.814+0000 7f5bae7fc640 1 -- 172.21.15.7:0/3704172178 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5617f62d9f70 con 0x5617f63eef40 2024-06-10T01:10:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.814+0000 7f5bae7fc640 10 client.16497 handle_command_reply: tid=0 2024-06-10T01:10:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.814+0000 7f5bd01b6740 1 client.16497 shutdown 2024-06-10T01:10:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.814+0000 7f5bd01b6740 1 -- 172.21.15.7:0/3704172178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617f636f010 msgr2=0x5617f637fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.814+0000 7f5bd01b6740 1 --2- 172.21.15.7:0/3704172178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617f636f010 0x5617f637fbd0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f5bc00b8d20 tx=0x7f5bc00daf90 comp rx=0 tx=0).stop 2024-06-10T01:10:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.814+0000 7f5bd01b6740 1 -- 172.21.15.7:0/3704172178 shutdown_connections 2024-06-10T01:10:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.814+0000 7f5bd01b6740 1 --2- 172.21.15.7:0/3704172178 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5617f63eef40 0x5617f63f1360 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.814+0000 7f5bd01b6740 1 --2- 172.21.15.7:0/3704172178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617f6374b70 0x5617f637c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.814+0000 7f5bd01b6740 1 --2- 172.21.15.7:0/3704172178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617f636fa00 0x5617f6380110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.814+0000 7f5bae7fc640 0 client.16497 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.814+0000 7f5bd01b6740 1 --2- 172.21.15.7:0/3704172178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617f636f010 0x5617f637fbd0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.814+0000 7f5bd01b6740 1 -- 172.21.15.7:0/3704172178 >> 172.21.15.7:0/3704172178 conn(0x5617f636a680 msgr2=0x5617f6370bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.814+0000 7f5bd01b6740 1 -- 172.21.15.7:0/3704172178 shutdown_connections 2024-06-10T01:10:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.815+0000 7f5bd01b6740 1 -- 172.21.15.7:0/3704172178 wait complete. 2024-06-10T01:10:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.815+0000 7f5bd01b6740 20 client.16497 trim_cache size 0 max 16384 2024-06-10T01:10:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:10:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:10:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:10:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.815+0000 7f5bcd4ae640 1 -- 172.21.15.7:0/3802486035 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5bb40a2b80 msgr2=0x7f5bb40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.815+0000 7f5bcd4ae640 1 --2- 172.21.15.7:0/3802486035 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5bb40a2b80 0x7f5bb40a5040 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x5617f6370260 tx=0x7f5bb8004730 comp rx=0 tx=0).stop 2024-06-10T01:10:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.815+0000 7f5bcd4ae640 1 -- 172.21.15.7:0/3802486035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bc81024c0 msgr2=0x7f5bc8101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.815+0000 7f5bcd4ae640 1 --2- 172.21.15.7:0/3802486035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bc81024c0 0x7f5bc8101e30 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009d00 tx=0x7f5bc0030630 comp rx=0 tx=0).stop 2024-06-10T01:10:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.815+0000 7f5bcd4ae640 1 -- 172.21.15.7:0/3802486035 shutdown_connections 2024-06-10T01:10:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.815+0000 7f5bcd4ae640 1 --2- 172.21.15.7:0/3802486035 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5bb40a2b80 0x7f5bb40a5040 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.815+0000 7f5bcd4ae640 1 --2- 172.21.15.7:0/3802486035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bc8107720 0x7f5bc81009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.815+0000 7f5bcd4ae640 1 --2- 172.21.15.7:0/3802486035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bc8104df0 0x7f5bc8100470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.815+0000 7f5bcd4ae640 1 --2- 172.21.15.7:0/3802486035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bc81024c0 0x7f5bc8101e30 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.815+0000 7f5bcd4ae640 1 -- 172.21.15.7:0/3802486035 >> 172.21.15.7:0/3802486035 conn(0x7f5bc80fbed0 msgr2=0x7f5bc8102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.816+0000 7f5bcd4ae640 1 -- 172.21.15.7:0/3802486035 shutdown_connections 2024-06-10T01:10:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:01.816+0000 7f5bcd4ae640 1 -- 172.21.15.7:0/3802486035 wait complete. 2024-06-10T01:10:01.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.820+0000 7f2dceff5700 1 -- 172.21.15.7:0/1242075005 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f2de000fc30 con 0x55d44f79cb60 2024-06-10T01:10:01.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.820+0000 7f2dceff5700 10 client.26012 handle_command_reply: tid=0 2024-06-10T01:10:01.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.820+0000 7f2dfbeb2b80 1 client.26012 shutdown 2024-06-10T01:10:01.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.820+0000 7f2dfbeb2b80 1 -- 172.21.15.7:0/1242075005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d44f71d5f0 msgr2=0x55d44f734fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:01.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.820+0000 7f2dfbeb2b80 1 --2- 172.21.15.7:0/1242075005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d44f71d5f0 0x55d44f734fd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2de4014fa0 tx=0x7f2de40077e0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.820+0000 7f2dfbeb2b80 1 -- 172.21.15.7:0/1242075005 shutdown_connections 2024-06-10T01:10:01.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.820+0000 7f2dfbeb2b80 1 --2- 172.21.15.7:0/1242075005 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d44f79cb60 0x55d44f79ef80 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.820+0000 7f2dfbeb2b80 1 --2- 172.21.15.7:0/1242075005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d44f71d5f0 0x55d44f734fd0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.820+0000 7f2dfbeb2b80 1 --2- 172.21.15.7:0/1242075005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d44f717460 0x55d44f736690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.820+0000 7f2dceff5700 0 client.26012 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:01.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.820+0000 7f2dfbeb2b80 1 --2- 172.21.15.7:0/1242075005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d44f71ccd0 0x55d44f734a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.820+0000 7f2dfbeb2b80 1 -- 172.21.15.7:0/1242075005 >> 172.21.15.7:0/1242075005 conn(0x55d44f641e50 msgr2=0x55d44f711fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:01.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.821+0000 7f2dfbeb2b80 1 -- 172.21.15.7:0/1242075005 shutdown_connections 2024-06-10T01:10:01.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.821+0000 7f2dfbeb2b80 1 -- 172.21.15.7:0/1242075005 wait complete. 2024-06-10T01:10:01.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.821+0000 7f2dfbeb2b80 20 client.26012 trim_cache size 0 max 16384 2024-06-10T01:10:01.817 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:10:01.817 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 30, 2024-06-10T01:10:01.817 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:10:01.817 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 10, 2024-06-10T01:10:01.818 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:10:01.818 INFO:teuthology.orchestra.run.smithi007.stdout: "3d7bfe3b": { 2024-06-10T01:10:01.818 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 10, 2024-06-10T01:10:01.818 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:10:01.818 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:10:01.818 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:10:01.818 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:10:01.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:10:01.818 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:10:01.818 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 138.2, 2024-06-10T01:10:01.818 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:10:01.818 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:10:01.818 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:10:01.818 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:10:01.819 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:10:01.819 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:10:01.819 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:10:01.819 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:10:01.819 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:10:01.819 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:10:01.819 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:10:01.819 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:10:01.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.821+0000 7f2df162f700 1 -- 172.21.15.7:0/2709357946 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2dd40a2d40 msgr2=0x7f2dd40a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:01.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.821+0000 7f2df162f700 1 --2- 172.21.15.7:0/2709357946 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2dd40a2d40 0x7f2dd40a5200 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2dec108c50 tx=0x7f2de0005fb0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.821+0000 7f2df162f700 1 -- 172.21.15.7:0/2709357946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dec1027a0 msgr2=0x7f2dec19e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:01.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.821+0000 7f2df162f700 1 --2- 172.21.15.7:0/2709357946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dec1027a0 0x7f2dec19e800 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc00e9c0 tx=0x7f2ddc00ecd0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.821+0000 7f2df162f700 1 -- 172.21.15.7:0/2709357946 shutdown_connections 2024-06-10T01:10:01.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.821+0000 7f2df162f700 1 --2- 172.21.15.7:0/2709357946 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2dd40a2d40 0x7f2dd40a5200 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.821+0000 7f2df162f700 1 --2- 172.21.15.7:0/2709357946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dec107870 0x7f2dec198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.821+0000 7f2df162f700 1 --2- 172.21.15.7:0/2709357946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dec1027a0 0x7f2dec19e800 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.821+0000 7f2df162f700 1 --2- 172.21.15.7:0/2709357946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dec103150 0x7f2dec19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.821+0000 7f2df162f700 1 -- 172.21.15.7:0/2709357946 >> 172.21.15.7:0/2709357946 conn(0x7f2dec079570 msgr2=0x7f2dec100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:01.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.822+0000 7f2df162f700 1 -- 172.21.15.7:0/2709357946 shutdown_connections 2024-06-10T01:10:01.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:01.822+0000 7f2df162f700 1 -- 172.21.15.7:0/2709357946 wait complete. 2024-06-10T01:10:02.714 INFO:tasks.workunit.client.0.smithi007.stdout: 1 259415 10.67 MB/sec execute 43 sec latency 1054.505 ms 2024-06-10T01:10:03.508 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:03 smithi007 ceph-mon[32398]: pgmap v553: 129 pgs: 129 active+clean; 66 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 307 op/s 2024-06-10T01:10:03.539 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3d7bfe3b', quiesce took 11.6 seconds. Will release after: 6.6 2024-06-10T01:10:03.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:03 smithi078 ceph-mon[36683]: pgmap v553: 129 pgs: 129 active+clean; 66 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 307 op/s 2024-06-10T01:10:03.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:03 smithi175 ceph-mon[36636]: pgmap v553: 129 pgs: 129 active+clean; 66 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 307 op/s 2024-06-10T01:10:03.714 INFO:tasks.workunit.client.0.smithi007.stdout: 1 259415 10.43 MB/sec execute 44 sec latency 2054.581 ms 2024-06-10T01:10:04.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.095+0000 7efd9bd65700 1 -- 172.21.15.7:0/532391249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd9406ac00 msgr2=0x7efd9406afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:04.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.095+0000 7efd9bd65700 1 --2- 172.21.15.7:0/532391249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd9406ac00 0x7efd9406afe0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7efd90009990 tx=0x7efd90009ca0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.095+0000 7efd9bd65700 1 -- 172.21.15.7:0/532391249 shutdown_connections 2024-06-10T01:10:04.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.095+0000 7efd9bd65700 1 --2- 172.21.15.7:0/532391249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd9410ef70 0x7efd94111360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.095+0000 7efd9bd65700 1 --2- 172.21.15.7:0/532391249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd9406ac00 0x7efd9406afe0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.095+0000 7efd9bd65700 1 --2- 172.21.15.7:0/532391249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd9406b520 0x7efd9410ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.095+0000 7efd9bd65700 1 -- 172.21.15.7:0/532391249 >> 172.21.15.7:0/532391249 conn(0x7efd940773b0 msgr2=0x7efd940777c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:04.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.095+0000 7efd9bd65700 1 -- 172.21.15.7:0/532391249 shutdown_connections 2024-06-10T01:10:04.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.095+0000 7efd9bd65700 1 -- 172.21.15.7:0/532391249 wait complete. 2024-06-10T01:10:04.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.096+0000 7efd9bd65700 1 Processor -- start 2024-06-10T01:10:04.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.096+0000 7efd9bd65700 1 -- start start 2024-06-10T01:10:04.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.096+0000 7efd9bd65700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd9406ac00 0x7efd9419aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:04.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.096+0000 7efd9bd65700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd9406b520 0x7efd9419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:04.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.096+0000 7efd9bd65700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd9410ef70 0x7efd9419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:04.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.096+0000 7efd9bd65700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efd94114a10 con 0x7efd9406ac00 2024-06-10T01:10:04.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.096+0000 7efd9bd65700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efd94114890 con 0x7efd9410ef70 2024-06-10T01:10:04.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.096+0000 7efd9bd65700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efd94114b90 con 0x7efd9406b520 2024-06-10T01:10:04.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.096+0000 7efd99b01700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd9406ac00 0x7efd9419aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:04.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.096+0000 7efd99b01700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd9406ac00 0x7efd9419aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39596/0 (socket says 172.21.15.7:39596) 2024-06-10T01:10:04.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.096+0000 7efd99b01700 1 -- 172.21.15.7:0/3700652316 learned_addr learned my addr 172.21.15.7:0/3700652316 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:04.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.096+0000 7efd9a302700 1 --2- 172.21.15.7:0/3700652316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd9410ef70 0x7efd9419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:04.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.096+0000 7efd99300700 1 --2- 172.21.15.7:0/3700652316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd9406b520 0x7efd9419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:04.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.096+0000 7efd99b01700 1 -- 172.21.15.7:0/3700652316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd9406b520 msgr2=0x7efd9419b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:04.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.096+0000 7efd99b01700 1 --2- 172.21.15.7:0/3700652316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd9406b520 0x7efd9419b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.096+0000 7efd99b01700 1 -- 172.21.15.7:0/3700652316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd9410ef70 msgr2=0x7efd9419f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:04.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.096+0000 7efd99b01700 1 --2- 172.21.15.7:0/3700652316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd9410ef70 0x7efd9419f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.097+0000 7efd99b01700 1 -- 172.21.15.7:0/3700652316 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd9419fe60 con 0x7efd9406ac00 2024-06-10T01:10:04.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.097+0000 7efd99300700 1 --2- 172.21.15.7:0/3700652316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd9406b520 0x7efd9419b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:10:04.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.097+0000 7efd99b01700 1 --2- 172.21.15.7:0/3700652316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd9406ac00 0x7efd9419aea0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7efd900038b0 tx=0x7efd90009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:04.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.097+0000 7efd8affd700 1 -- 172.21.15.7:0/3700652316 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90027070 con 0x7efd9406ac00 2024-06-10T01:10:04.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.097+0000 7efd9bd65700 1 -- 172.21.15.7:0/3700652316 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd941a00f0 con 0x7efd9406ac00 2024-06-10T01:10:04.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.097+0000 7efd8affd700 1 -- 172.21.15.7:0/3700652316 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd90004050 con 0x7efd9406ac00 2024-06-10T01:10:04.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.097+0000 7efd8affd700 1 -- 172.21.15.7:0/3700652316 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90022d40 con 0x7efd9406ac00 2024-06-10T01:10:04.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.097+0000 7efd9bd65700 1 -- 172.21.15.7:0/3700652316 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd941a79d0 con 0x7efd9406ac00 2024-06-10T01:10:04.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.098+0000 7efd9bd65700 1 -- 172.21.15.7:0/3700652316 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd941019a0 con 0x7efd9406ac00 2024-06-10T01:10:04.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.099+0000 7efd8affd700 1 -- 172.21.15.7:0/3700652316 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7efd9002a070 con 0x7efd9406ac00 2024-06-10T01:10:04.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.100+0000 7efd8affd700 1 --2- 172.21.15.7:0/3700652316 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efd800a2c20 0x7efd800a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:04.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.100+0000 7efd99300700 1 --2- 172.21.15.7:0/3700652316 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efd800a2c20 0x7efd800a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:04.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.100+0000 7efd8affd700 1 -- 172.21.15.7:0/3700652316 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7efd900d00c0 con 0x7efd9406ac00 2024-06-10T01:10:04.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.100+0000 7efd99300700 1 --2- 172.21.15.7:0/3700652316 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efd800a2c20 0x7efd800a50e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efd941028e0 tx=0x7efd8400b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:04.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.104+0000 7efd8affd700 1 -- 172.21.15.7:0/3700652316 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efd9005e850 con 0x7efd9406ac00 2024-06-10T01:10:04.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.357+0000 7efd9bd65700 1 -- 172.21.15.7:0/3700652316 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd9419c1b0 con 0x7efd9406ac00 2024-06-10T01:10:04.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.358+0000 7efd8affd700 1 -- 172.21.15.7:0/3700652316 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11858 (secure 0 0 0) 0x7efd90013070 con 0x7efd9406ac00 2024-06-10T01:10:04.353 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:10:04.354 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:08:17.181785+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24434,24436,24451]},"id":1}]} 2024-06-10T01:10:04.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.360+0000 7efd9bd65700 1 -- 172.21.15.7:0/3700652316 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efd800a2c20 msgr2=0x7efd800a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:04.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.360+0000 7efd9bd65700 1 --2- 172.21.15.7:0/3700652316 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efd800a2c20 0x7efd800a50e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efd941028e0 tx=0x7efd8400b5f0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.360+0000 7efd9bd65700 1 -- 172.21.15.7:0/3700652316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd9406ac00 msgr2=0x7efd9419aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:04.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.360+0000 7efd9bd65700 1 --2- 172.21.15.7:0/3700652316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd9406ac00 0x7efd9419aea0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7efd900038b0 tx=0x7efd90009f50 comp rx=0 tx=0).stop 2024-06-10T01:10:04.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.361+0000 7efd9bd65700 1 -- 172.21.15.7:0/3700652316 shutdown_connections 2024-06-10T01:10:04.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.361+0000 7efd9bd65700 1 --2- 172.21.15.7:0/3700652316 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efd800a2c20 0x7efd800a50e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.361+0000 7efd9bd65700 1 --2- 172.21.15.7:0/3700652316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd9410ef70 0x7efd9419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.361+0000 7efd9bd65700 1 --2- 172.21.15.7:0/3700652316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd9406ac00 0x7efd9419aea0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.361+0000 7efd9bd65700 1 --2- 172.21.15.7:0/3700652316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd9406b520 0x7efd9419b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.361+0000 7efd9bd65700 1 -- 172.21.15.7:0/3700652316 >> 172.21.15.7:0/3700652316 conn(0x7efd940773b0 msgr2=0x7efd941105a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:04.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.361+0000 7efd9bd65700 1 -- 172.21.15.7:0/3700652316 shutdown_connections 2024-06-10T01:10:04.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:04.361+0000 7efd9bd65700 1 -- 172.21.15.7:0/3700652316 wait complete. 2024-06-10T01:10:04.357 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-06-10T01:10:04.715 INFO:tasks.workunit.client.0.smithi007.stdout: 1 259415 10.20 MB/sec execute 45 sec latency 3054.679 ms 2024-06-10T01:10:05.099 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:10:05.357 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:10:05.480 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:05 smithi007 ceph-mon[32398]: pgmap v554: 129 pgs: 129 active+clean; 63 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 327 op/s 2024-06-10T01:10:05.481 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:05 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3700652316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:10:05.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:05 smithi078 ceph-mon[36683]: pgmap v554: 129 pgs: 129 active+clean; 63 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 327 op/s 2024-06-10T01:10:05.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:05 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3700652316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:10:05.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:05 smithi175 ceph-mon[36636]: pgmap v554: 129 pgs: 129 active+clean; 63 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 327 op/s 2024-06-10T01:10:05.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:05 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3700652316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:10:05.715 INFO:tasks.workunit.client.0.smithi007.stdout: 1 259415 9.98 MB/sec execute 46 sec latency 4054.761 ms 2024-06-10T01:10:06.715 INFO:tasks.workunit.client.0.smithi007.stdout: 1 259415 9.76 MB/sec execute 47 sec latency 5054.870 ms 2024-06-10T01:10:07.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.155+0000 7ff8c21a3700 1 -- 172.21.15.7:0/2463041338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8bc107840 msgr2=0x7ff8bc10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:07.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.155+0000 7ff8c21a3700 1 --2- 172.21.15.7:0/2463041338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8bc107840 0x7ff8bc10f3c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400b0d0 tx=0x7ff8b400b3e0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.155+0000 7ff8c21a3700 1 -- 172.21.15.7:0/2463041338 shutdown_connections 2024-06-10T01:10:07.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.155+0000 7ff8c21a3700 1 --2- 172.21.15.7:0/2463041338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8bc107840 0x7ff8bc10f3c0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.155+0000 7ff8c21a3700 1 --2- 172.21.15.7:0/2463041338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8bc102740 0x7ff8bc102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.155+0000 7ff8c21a3700 1 --2- 172.21.15.7:0/2463041338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8bc1030f0 0x7ff8bc107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.155+0000 7ff8c21a3700 1 -- 172.21.15.7:0/2463041338 >> 172.21.15.7:0/2463041338 conn(0x7ff8bc0795f0 msgr2=0x7ff8bc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:07.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.155+0000 7ff8c21a3700 1 -- 172.21.15.7:0/2463041338 shutdown_connections 2024-06-10T01:10:07.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.156+0000 7ff8c21a3700 1 -- 172.21.15.7:0/2463041338 wait complete. 2024-06-10T01:10:07.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.156+0000 7ff8c21a3700 1 Processor -- start 2024-06-10T01:10:07.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.156+0000 7ff8c21a3700 1 -- start start 2024-06-10T01:10:07.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.157+0000 7ff8c21a3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8bc102740 0x7ff8bc19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:07.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.157+0000 7ff8c21a3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8bc1030f0 0x7ff8bc19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:07.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.157+0000 7ff8c21a3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8bc107840 0x7ff8bc198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:07.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.157+0000 7ff8c21a3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc112760 con 0x7ff8bc107840 2024-06-10T01:10:07.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.157+0000 7ff8c21a3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc1125e0 con 0x7ff8bc1030f0 2024-06-10T01:10:07.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.157+0000 7ff8c21a3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc1128e0 con 0x7ff8bc102740 2024-06-10T01:10:07.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.157+0000 7ff8bbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8bc107840 0x7ff8bc198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:07.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.157+0000 7ff8bbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8bc107840 0x7ff8bc198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39614/0 (socket says 172.21.15.7:39614) 2024-06-10T01:10:07.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.157+0000 7ff8bbfff700 1 -- 172.21.15.7:0/3103319826 learned_addr learned my addr 172.21.15.7:0/3103319826 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:07.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.157+0000 7ff8baffd700 1 --2- 172.21.15.7:0/3103319826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8bc1030f0 0x7ff8bc19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:07.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.157+0000 7ff8bb7fe700 1 --2- 172.21.15.7:0/3103319826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8bc102740 0x7ff8bc19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:07.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.157+0000 7ff8bbfff700 1 -- 172.21.15.7:0/3103319826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8bc102740 msgr2=0x7ff8bc19e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:07.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.157+0000 7ff8bbfff700 1 --2- 172.21.15.7:0/3103319826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8bc102740 0x7ff8bc19e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.157+0000 7ff8bbfff700 1 -- 172.21.15.7:0/3103319826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8bc1030f0 msgr2=0x7ff8bc19ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:07.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.157+0000 7ff8bbfff700 1 --2- 172.21.15.7:0/3103319826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8bc1030f0 0x7ff8bc19ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.157+0000 7ff8bbfff700 1 -- 172.21.15.7:0/3103319826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8bc1990a0 con 0x7ff8bc107840 2024-06-10T01:10:07.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.157+0000 7ff8bb7fe700 1 --2- 172.21.15.7:0/3103319826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8bc102740 0x7ff8bc19e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:10:07.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.157+0000 7ff8bbfff700 1 --2- 172.21.15.7:0/3103319826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8bc107840 0x7ff8bc198940 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4006070 tx=0x7ff8b4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:07.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.157+0000 7ff8b8ff9700 1 -- 172.21.15.7:0/3103319826 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4021070 con 0x7ff8bc107840 2024-06-10T01:10:07.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.157+0000 7ff8b8ff9700 1 -- 172.21.15.7:0/3103319826 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8b4007b80 con 0x7ff8bc107840 2024-06-10T01:10:07.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.157+0000 7ff8c21a3700 1 -- 172.21.15.7:0/3103319826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8bc199330 con 0x7ff8bc107840 2024-06-10T01:10:07.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.157+0000 7ff8b8ff9700 1 -- 172.21.15.7:0/3103319826 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4005330 con 0x7ff8bc107840 2024-06-10T01:10:07.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.158+0000 7ff8c21a3700 1 -- 172.21.15.7:0/3103319826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8bc1a5650 con 0x7ff8bc107840 2024-06-10T01:10:07.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.160+0000 7ff8b8ff9700 1 -- 172.21.15.7:0/3103319826 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7ff8b4024070 con 0x7ff8bc107840 2024-06-10T01:10:07.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.160+0000 7ff8b8ff9700 1 --2- 172.21.15.7:0/3103319826 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff8a40a2c20 0x7ff8a40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:07.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.161+0000 7ff8b8ff9700 1 -- 172.21.15.7:0/3103319826 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7ff8b400f020 con 0x7ff8bc107840 2024-06-10T01:10:07.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.161+0000 7ff8bb7fe700 1 --2- 172.21.15.7:0/3103319826 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff8a40a2c20 0x7ff8a40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:07.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.161+0000 7ff8bb7fe700 1 --2- 172.21.15.7:0/3103319826 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff8a40a2c20 0x7ff8a40a50e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac005950 tx=0x7ff8ac0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:07.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.161+0000 7ff8cca26b80 1 Processor -- start 2024-06-10T01:10:07.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.161+0000 7ff8cca26b80 1 -- start start 2024-06-10T01:10:07.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.162+0000 7ff8cca26b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d15d9a7f60 0x55d15d9a8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:07.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.162+0000 7ff8cca26b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d15d9a8880 0x55d15d9a2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:07.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.162+0000 7ff8cca26b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d15d9a27a0 0x55d15d9a2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:07.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.162+0000 7ff8cca26b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d15d894f60 con 0x55d15d9a27a0 2024-06-10T01:10:07.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.162+0000 7ff8cca26b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d15d8f3850 con 0x55d15d9a8880 2024-06-10T01:10:07.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.162+0000 7ff8cca26b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d15d85b460 con 0x55d15d9a7f60 2024-06-10T01:10:07.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.162+0000 7ff8bbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d15d9a27a0 0x55d15d9a2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:07.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.162+0000 7ff8bbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d15d9a27a0 0x55d15d9a2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39624/0 (socket says 172.21.15.7:39624) 2024-06-10T01:10:07.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.162+0000 7ff8bbfff700 1 -- 172.21.15.7:0/1682922911 learned_addr learned my addr 172.21.15.7:0/1682922911 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:07.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.162+0000 7ff8bb7fe700 1 --2- 172.21.15.7:0/1682922911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d15d9a7f60 0x55d15d9a8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:07.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.162+0000 7ff8baffd700 1 --2- 172.21.15.7:0/1682922911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d15d9a8880 0x55d15d9a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:07.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.162+0000 7ff8bbfff700 1 -- 172.21.15.7:0/1682922911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d15d9a7f60 msgr2=0x55d15d9a8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:07.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.162+0000 7ff8bbfff700 1 --2- 172.21.15.7:0/1682922911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d15d9a7f60 0x55d15d9a8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.162+0000 7ff8bbfff700 1 -- 172.21.15.7:0/1682922911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d15d9a8880 msgr2=0x55d15d9a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:07.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.162+0000 7ff8bbfff700 1 --2- 172.21.15.7:0/1682922911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d15d9a8880 0x55d15d9a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.162+0000 7ff8bbfff700 1 -- 172.21.15.7:0/1682922911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d15d8428d0 con 0x55d15d9a27a0 2024-06-10T01:10:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.162+0000 7ff8bbfff700 1 --2- 172.21.15.7:0/1682922911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d15d9a27a0 0x55d15d9a2bc0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40062a0 tx=0x7ff8b401dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd4a892c5bdbedc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.162+0000 7ff8a1ffb700 1 -- 172.21.15.7:0/1682922911 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b409cef0 con 0x55d15d9a27a0 2024-06-10T01:10:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.163+0000 7ff8a1ffb700 1 -- 172.21.15.7:0/1682922911 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8b40a12a0 con 0x55d15d9a27a0 2024-06-10T01:10:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.163+0000 7ff8a1ffb700 1 -- 172.21.15.7:0/1682922911 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b409d3b0 con 0x55d15d9a27a0 2024-06-10T01:10:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.163+0000 7ff8cca26b80 1 -- 172.21.15.7:0/1682922911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d15d9a27a0 msgr2=0x55d15d9a2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.163+0000 7ff8cca26b80 1 --2- 172.21.15.7:0/1682922911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d15d9a27a0 0x55d15d9a2bc0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40062a0 tx=0x7ff8b401dd60 comp rx=0 tx=0).stop 2024-06-10T01:10:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.163+0000 7ff8cca26b80 1 -- 172.21.15.7:0/1682922911 shutdown_connections 2024-06-10T01:10:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.163+0000 7ff8cca26b80 1 --2- 172.21.15.7:0/1682922911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d15d9a27a0 0x55d15d9a2bc0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.163+0000 7ff8cca26b80 1 --2- 172.21.15.7:0/1682922911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d15d9a7f60 0x55d15d9a8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.163+0000 7ff8cca26b80 1 --2- 172.21.15.7:0/1682922911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d15d9a8880 0x55d15d9a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.163+0000 7ff8cca26b80 1 -- 172.21.15.7:0/1682922911 >> 172.21.15.7:0/1682922911 conn(0x55d15d8cf080 msgr2=0x55d15d99d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.163+0000 7ff8cca26b80 1 -- 172.21.15.7:0/1682922911 shutdown_connections 2024-06-10T01:10:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.163+0000 7ff8cca26b80 1 -- 172.21.15.7:0/1682922911 wait complete. 2024-06-10T01:10:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.164+0000 7ff8cca26b80 1 -- start start 2024-06-10T01:10:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.164+0000 7ff8cca26b80 1 Processor -- start 2024-06-10T01:10:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.164+0000 7ff8cca26b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d15d9a27a0 0x55d15d9ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.164+0000 7ff8cca26b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d15d9a7f60 0x55d15d9ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.164+0000 7ff8cca26b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d15d9a8880 0x55d15d9abe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.164+0000 7ff8cca26b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d15d893fc0 con 0x55d15d9a27a0 2024-06-10T01:10:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.164+0000 7ff8cca26b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d15d8ef810 con 0x55d15d9a8880 2024-06-10T01:10:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.164+0000 7ff8cca26b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d15d9af010 con 0x55d15d9a7f60 2024-06-10T01:10:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.164+0000 7ff8cca26b80 10 client.0 _ll_register_callbacks cb 0x55d15d8895b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.164+0000 7ff8bb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d15d9a27a0 0x55d15d9ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.164+0000 7ff8bb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d15d9a27a0 0x55d15d9ae540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39640/0 (socket says 172.21.15.7:39640) 2024-06-10T01:10:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.164+0000 7ff8bb7fe700 1 -- 172.21.15.7:0/874626602 learned_addr learned my addr 172.21.15.7:0/874626602 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.165+0000 7ff8baffd700 1 --2- 172.21.15.7:0/874626602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d15d9a7f60 0x55d15d9ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.165+0000 7ff8bbfff700 1 --2- 172.21.15.7:0/874626602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d15d9a8880 0x55d15d9abe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.165+0000 7ff8bb7fe700 1 -- 172.21.15.7:0/874626602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d15d9a7f60 msgr2=0x55d15d9ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.165+0000 7ff8bb7fe700 1 --2- 172.21.15.7:0/874626602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d15d9a7f60 0x55d15d9ab8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.165+0000 7ff8bb7fe700 1 -- 172.21.15.7:0/874626602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d15d9a8880 msgr2=0x55d15d9abe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.165+0000 7ff8bb7fe700 1 --2- 172.21.15.7:0/874626602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d15d9a8880 0x55d15d9abe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.165+0000 7ff8bb7fe700 1 -- 172.21.15.7:0/874626602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d15d8b9750 con 0x55d15d9a27a0 2024-06-10T01:10:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.165+0000 7ff8bbfff700 1 --2- 172.21.15.7:0/874626602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d15d9a8880 0x55d15d9abe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:10:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.165+0000 7ff8bb7fe700 1 --2- 172.21.15.7:0/874626602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d15d9a27a0 0x55d15d9ae540 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac005d10 tx=0x7ff8ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78fd9c3ec07ac42 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.165+0000 7ff89affd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:10:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.165+0000 7ff89affd700 1 -- 172.21.15.7:0/874626602 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac01f840 con 0x55d15d9a27a0 2024-06-10T01:10:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.165+0000 7ff89affd700 1 -- 172.21.15.7:0/874626602 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7ff8ac01f9e0 con 0x55d15d9a27a0 2024-06-10T01:10:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.165+0000 7ff8cca26b80 1 -- 172.21.15.7:0/874626602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8b40a12a0 con 0x55d15d9a27a0 2024-06-10T01:10:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.165+0000 7ff89affd700 1 -- 172.21.15.7:0/874626602 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff8ac01fc20 con 0x55d15d9a27a0 2024-06-10T01:10:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.165+0000 7ff8cca26b80 10 client.16515 fetch_fsmap learned FSMap version 30 2024-06-10T01:10:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.165+0000 7ff8cca26b80 1 -- 172.21.15.7:0/874626602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d15d9b9760 con 0x55d15d9a27a0 2024-06-10T01:10:07.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.165+0000 7ff89affd700 1 -- 172.21.15.7:0/874626602 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8b40a12a0 con 0x55d15d9a27a0 2024-06-10T01:10:07.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.166+0000 7ff89affd700 1 -- 172.21.15.7:0/874626602 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ff8ac01e430 con 0x55d15d9a27a0 2024-06-10T01:10:07.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.166+0000 7ff8cca26b80 10 client.16515 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:10:07.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.166+0000 7ff8cca26b80 10 client.16515 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:10:07.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.166+0000 7ff8cca26b80 20 client.16515 populate_metadata read hostname 'smithi007' 2024-06-10T01:10:07.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.166+0000 7ff8cca26b80 1 --2- 172.21.15.7:0/874626602 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55d15d9bbdf0 0x55d15d9be220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:07.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.166+0000 7ff8cca26b80 4 client.16515 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:10:07.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.166+0000 7ff8cca26b80 1 -- 172.21.15.7:0/874626602 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d15d900d50 con 0x55d15d9bbdf0 2024-06-10T01:10:07.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.166+0000 7ff8baffd700 1 --2- 172.21.15.7:0/874626602 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55d15d9bbdf0 0x55d15d9be220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:07.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.166+0000 7ff8baffd700 1 --2- 172.21.15.7:0/874626602 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55d15d9bbdf0 0x55d15d9be220 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=325cd0308d5db67a server_cookie=3cf7563eb0a1ed0c in_seq=0 out_seq=0 2024-06-10T01:10:07.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.166+0000 7ff89affd700 10 client.16515 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:10:07.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.167+0000 7ff89affd700 1 -- 172.21.15.7:0/874626602 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d15d900d50 con 0x55d15d9bbdf0 2024-06-10T01:10:07.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.167+0000 7ff89affd700 10 client.16515 handle_command_reply: tid=0 2024-06-10T01:10:07.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.167+0000 7ff8cca26b80 1 client.16515 shutdown 2024-06-10T01:10:07.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.167+0000 7ff8cca26b80 1 -- 172.21.15.7:0/874626602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d15d9a27a0 msgr2=0x55d15d9ae540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:07.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.167+0000 7ff8cca26b80 1 --2- 172.21.15.7:0/874626602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d15d9a27a0 0x55d15d9ae540 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac005d10 tx=0x7ff8ac004750 comp rx=0 tx=0).stop 2024-06-10T01:10:07.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.167+0000 7ff8cca26b80 1 -- 172.21.15.7:0/874626602 shutdown_connections 2024-06-10T01:10:07.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.167+0000 7ff8cca26b80 1 --2- 172.21.15.7:0/874626602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d15d9a8880 0x55d15d9abe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.167+0000 7ff8cca26b80 1 --2- 172.21.15.7:0/874626602 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55d15d9bbdf0 0x55d15d9be220 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.167+0000 7ff89affd700 0 client.16515 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:10:07.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.167+0000 7ff8cca26b80 1 --2- 172.21.15.7:0/874626602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d15d9a27a0 0x55d15d9ae540 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.167+0000 7ff8cca26b80 1 --2- 172.21.15.7:0/874626602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d15d9a7f60 0x55d15d9ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.167+0000 7ff8cca26b80 1 -- 172.21.15.7:0/874626602 >> 172.21.15.7:0/874626602 conn(0x55d15d8cf080 msgr2=0x55d15d99c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:07.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.167+0000 7ff8cca26b80 1 -- 172.21.15.7:0/874626602 shutdown_connections 2024-06-10T01:10:07.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.168+0000 7ff8cca26b80 1 -- 172.21.15.7:0/874626602 wait complete. 2024-06-10T01:10:07.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.168+0000 7ff8cca26b80 20 client.16515 trim_cache size 0 max 16384 2024-06-10T01:10:07.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.201+0000 7ff8cca26b80 1 Processor -- start 2024-06-10T01:10:07.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.201+0000 7ff8cca26b80 1 -- start start 2024-06-10T01:10:07.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.202+0000 7ff8cca26b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d15d9a27a0 0x55d15d9a2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:07.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.202+0000 7ff8cca26b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d15d9a7f60 0x55d15d9aaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:07.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.202+0000 7ff8cca26b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d15d9a8880 0x55d15d99ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:07.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.202+0000 7ff8cca26b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc1b0a10 con 0x55d15d9a7f60 2024-06-10T01:10:07.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.202+0000 7ff8cca26b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc1ae240 con 0x55d15d9a8880 2024-06-10T01:10:07.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.202+0000 7ff8cca26b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc1ac090 con 0x55d15d9a27a0 2024-06-10T01:10:07.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.202+0000 7ff8bb7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d15d9a27a0 0x55d15d9a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:07.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.203+0000 7ff8bb7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d15d9a27a0 0x55d15d9a2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43264/0 (socket says 172.21.15.7:43264) 2024-06-10T01:10:07.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.203+0000 7ff8bb7fe700 1 -- 172.21.15.7:0/2346981776 learned_addr learned my addr 172.21.15.7:0/2346981776 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:07.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.203+0000 7ff8bbfff700 1 --2- 172.21.15.7:0/2346981776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d15d9a8880 0x55d15d99ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:07.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.203+0000 7ff8baffd700 1 --2- 172.21.15.7:0/2346981776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d15d9a7f60 0x55d15d9aaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:07.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.203+0000 7ff8bb7fe700 1 -- 172.21.15.7:0/2346981776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d15d9a8880 msgr2=0x55d15d99ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:07.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.203+0000 7ff8bb7fe700 1 --2- 172.21.15.7:0/2346981776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d15d9a8880 0x55d15d99ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.203+0000 7ff8bb7fe700 1 -- 172.21.15.7:0/2346981776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d15d9a7f60 msgr2=0x55d15d9aaa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:07.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.203+0000 7ff8bb7fe700 1 --2- 172.21.15.7:0/2346981776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d15d9a7f60 0x55d15d9aaa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.203+0000 7ff8bb7fe700 1 -- 172.21.15.7:0/2346981776 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d15d9a8680 con 0x55d15d9a27a0 2024-06-10T01:10:07.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.203+0000 7ff8bbfff700 1 --2- 172.21.15.7:0/2346981776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d15d9a8880 0x55d15d99ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:10:07.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.204+0000 7ff8bb7fe700 1 --2- 172.21.15.7:0/2346981776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d15d9a27a0 0x55d15d9a2260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac014040 tx=0x7ff8ac00fa50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dc051d9237b6c8c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:07.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.205+0000 7ff8a17fa700 1 -- 172.21.15.7:0/2346981776 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac03b020 con 0x55d15d9a27a0 2024-06-10T01:10:07.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.207+0000 7ff8a17fa700 1 -- 172.21.15.7:0/2346981776 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8ac023070 con 0x55d15d9a27a0 2024-06-10T01:10:07.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.208+0000 7ff8cca26b80 1 -- 172.21.15.7:0/2346981776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d15d9a27a0 msgr2=0x55d15d9a2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:07.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.208+0000 7ff8cca26b80 1 --2- 172.21.15.7:0/2346981776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d15d9a27a0 0x55d15d9a2260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac014040 tx=0x7ff8ac00fa50 comp rx=0 tx=0).stop 2024-06-10T01:10:07.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.208+0000 7ff8cca26b80 1 -- 172.21.15.7:0/2346981776 shutdown_connections 2024-06-10T01:10:07.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.208+0000 7ff8cca26b80 1 --2- 172.21.15.7:0/2346981776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d15d9a8880 0x55d15d99ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.208+0000 7ff8cca26b80 1 --2- 172.21.15.7:0/2346981776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d15d9a27a0 0x55d15d9a2260 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac014040 tx=0x7ff8ac00fa50 comp rx=0 tx=0).stop 2024-06-10T01:10:07.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.208+0000 7ff8cca26b80 1 --2- 172.21.15.7:0/2346981776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d15d9a7f60 0x55d15d9aaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.208+0000 7ff8cca26b80 1 -- 172.21.15.7:0/2346981776 >> 172.21.15.7:0/2346981776 conn(0x55d15d8cf080 msgr2=0x55d15d9a92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:07.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.210+0000 7ff8cca26b80 1 -- 172.21.15.7:0/2346981776 shutdown_connections 2024-06-10T01:10:07.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.210+0000 7ff8cca26b80 1 -- 172.21.15.7:0/2346981776 wait complete. 2024-06-10T01:10:07.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.211+0000 7ff8cca26b80 1 -- start start 2024-06-10T01:10:07.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.211+0000 7ff8cca26b80 1 Processor -- start 2024-06-10T01:10:07.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.212+0000 7ff8cca26b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d15d9a7f60 0x55d15d9b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:07.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.212+0000 7ff8cca26b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d15d9a8880 0x55d15d9c3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:07.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.212+0000 7ff8cca26b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d15d9bbdf0 0x55d15d9c38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:07.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.212+0000 7ff8cca26b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc1ac700 con 0x55d15d9bbdf0 2024-06-10T01:10:07.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.212+0000 7ff8cca26b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc1adb70 con 0x55d15d9a7f60 2024-06-10T01:10:07.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.212+0000 7ff8cca26b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc1acdd0 con 0x55d15d9a8880 2024-06-10T01:10:07.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.212+0000 7ff8cca26b80 10 client.0 _ll_register_callbacks cb 0x55d15d8895b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:07.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.213+0000 7ff8bbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d15d9bbdf0 0x55d15d9c38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:07.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.213+0000 7ff8bbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d15d9bbdf0 0x55d15d9c38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39646/0 (socket says 172.21.15.7:39646) 2024-06-10T01:10:07.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.213+0000 7ff8bbfff700 1 -- 172.21.15.7:0/1516706445 learned_addr learned my addr 172.21.15.7:0/1516706445 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:07.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.213+0000 7ff8bbfff700 1 -- 172.21.15.7:0/1516706445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d15d9a8880 msgr2=0x55d15d9c3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:07.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.213+0000 7ff8bbfff700 1 --2- 172.21.15.7:0/1516706445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d15d9a8880 0x55d15d9c3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.213+0000 7ff8bbfff700 1 -- 172.21.15.7:0/1516706445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d15d9a7f60 msgr2=0x55d15d9b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:07.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.213+0000 7ff8bbfff700 1 --2- 172.21.15.7:0/1516706445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d15d9a7f60 0x55d15d9b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.213+0000 7ff8bbfff700 1 -- 172.21.15.7:0/1516706445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d15d9c4030 con 0x55d15d9bbdf0 2024-06-10T01:10:07.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.213+0000 7ff8bbfff700 1 --2- 172.21.15.7:0/1516706445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d15d9bbdf0 0x55d15d9c38a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4007fa0 tx=0x7ff8b4099190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df31611bc8fadb86 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:07.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.213+0000 7ff89affd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:10:07.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.215+0000 7ff89affd700 1 -- 172.21.15.7:0/1516706445 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b40a12a0 con 0x55d15d9bbdf0 2024-06-10T01:10:07.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.215+0000 7ff89affd700 1 -- 172.21.15.7:0/1516706445 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7ff8b409d150 con 0x55d15d9bbdf0 2024-06-10T01:10:07.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.215+0000 7ff8cca26b80 1 -- 172.21.15.7:0/1516706445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8ac023070 con 0x55d15d9bbdf0 2024-06-10T01:10:07.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.215+0000 7ff89affd700 1 -- 172.21.15.7:0/1516706445 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff8b40a5ad0 con 0x55d15d9bbdf0 2024-06-10T01:10:07.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.215+0000 7ff8cca26b80 10 client.16518 fetch_fsmap learned FSMap version 30 2024-06-10T01:10:07.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.215+0000 7ff8cca26b80 1 -- 172.21.15.7:0/1516706445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d15da26a00 con 0x55d15d9bbdf0 2024-06-10T01:10:07.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.216+0000 7ff89affd700 1 -- 172.21.15.7:0/1516706445 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8ac023070 con 0x55d15d9bbdf0 2024-06-10T01:10:07.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.216+0000 7ff89affd700 1 -- 172.21.15.7:0/1516706445 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ff8b4083020 con 0x55d15d9bbdf0 2024-06-10T01:10:07.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.216+0000 7ff8cca26b80 10 client.16518 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:10:07.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.216+0000 7ff8cca26b80 10 client.16518 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:10:07.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.216+0000 7ff8cca26b80 20 client.16518 populate_metadata read hostname 'smithi007' 2024-06-10T01:10:07.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.216+0000 7ff8cca26b80 1 --2- 172.21.15.7:0/1516706445 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55d15da492b0 0x55d15da4b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:07.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.216+0000 7ff8cca26b80 4 client.16518 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:10:07.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.216+0000 7ff8cca26b80 1 -- 172.21.15.7:0/1516706445 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d15d8fb2d0 con 0x55d15da492b0 2024-06-10T01:10:07.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.218+0000 7ff8bb7fe700 1 --2- 172.21.15.7:0/1516706445 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55d15da492b0 0x55d15da4b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:07.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.220+0000 7ff8bb7fe700 1 --2- 172.21.15.7:0/1516706445 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55d15da492b0 0x55d15da4b6a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=65db179b0e6ad487 server_cookie=716c18de19fa86c2 in_seq=0 out_seq=0 2024-06-10T01:10:07.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.220+0000 7ff89affd700 10 client.16518 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:10:07.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.223+0000 7ff89affd700 1 -- 172.21.15.7:0/1516706445 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d15d8fb2d0 con 0x55d15da492b0 2024-06-10T01:10:07.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.223+0000 7ff89affd700 10 client.16518 handle_command_reply: tid=0 2024-06-10T01:10:07.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.223+0000 7ff8cca26b80 1 client.16518 shutdown 2024-06-10T01:10:07.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.224+0000 7ff8cca26b80 1 -- 172.21.15.7:0/1516706445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d15d9bbdf0 msgr2=0x55d15d9c38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:07.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.224+0000 7ff8cca26b80 1 --2- 172.21.15.7:0/1516706445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d15d9bbdf0 0x55d15d9c38a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4007fa0 tx=0x7ff8b4099190 comp rx=0 tx=0).stop 2024-06-10T01:10:07.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.224+0000 7ff8cca26b80 1 -- 172.21.15.7:0/1516706445 shutdown_connections 2024-06-10T01:10:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.224+0000 7ff8cca26b80 1 --2- 172.21.15.7:0/1516706445 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55d15da492b0 0x55d15da4b6a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.224+0000 7ff8cca26b80 1 --2- 172.21.15.7:0/1516706445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d15d9bbdf0 0x55d15d9c38a0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.224+0000 7ff8cca26b80 1 --2- 172.21.15.7:0/1516706445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d15d9a7f60 0x55d15d9b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.224+0000 7ff8cca26b80 1 --2- 172.21.15.7:0/1516706445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d15d9a8880 0x55d15d9c3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.224+0000 7ff8cca26b80 1 -- 172.21.15.7:0/1516706445 >> 172.21.15.7:0/1516706445 conn(0x55d15d8cf080 msgr2=0x55d15d9acee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.224+0000 7ff89affd700 0 client.16518 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:10:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.225+0000 7ff8cca26b80 1 -- 172.21.15.7:0/1516706445 shutdown_connections 2024-06-10T01:10:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.225+0000 7ff8cca26b80 1 -- 172.21.15.7:0/1516706445 wait complete. 2024-06-10T01:10:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.225+0000 7ff8cca26b80 20 client.16518 trim_cache size 0 max 16384 2024-06-10T01:10:07.221 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:10:07.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.226+0000 7ff8c21a3700 1 -- 172.21.15.7:0/3103319826 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff8a40a2c20 msgr2=0x7ff8a40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:07.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.226+0000 7ff8c21a3700 1 --2- 172.21.15.7:0/3103319826 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff8a40a2c20 0x7ff8a40a50e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac005950 tx=0x7ff8ac0058e0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.226+0000 7ff8c21a3700 1 -- 172.21.15.7:0/3103319826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8bc107840 msgr2=0x7ff8bc198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:07.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.226+0000 7ff8c21a3700 1 --2- 172.21.15.7:0/3103319826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8bc107840 0x7ff8bc198940 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4006070 tx=0x7ff8b4004d00 comp rx=0 tx=0).stop 2024-06-10T01:10:07.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.227+0000 7ff8c21a3700 1 -- 172.21.15.7:0/3103319826 shutdown_connections 2024-06-10T01:10:07.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.227+0000 7ff8c21a3700 1 --2- 172.21.15.7:0/3103319826 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff8a40a2c20 0x7ff8a40a50e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.227+0000 7ff8c21a3700 1 --2- 172.21.15.7:0/3103319826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8bc107840 0x7ff8bc198940 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.227+0000 7ff8c21a3700 1 --2- 172.21.15.7:0/3103319826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8bc102740 0x7ff8bc19e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.227+0000 7ff8c21a3700 1 --2- 172.21.15.7:0/3103319826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8bc1030f0 0x7ff8bc19ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:07.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.227+0000 7ff8c21a3700 1 -- 172.21.15.7:0/3103319826 >> 172.21.15.7:0/3103319826 conn(0x7ff8bc0795f0 msgr2=0x7ff8bc1005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:07.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.227+0000 7ff8c21a3700 1 -- 172.21.15.7:0/3103319826 shutdown_connections 2024-06-10T01:10:07.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:07.227+0000 7ff8c21a3700 1 -- 172.21.15.7:0/3103319826 wait complete. 2024-06-10T01:10:07.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:07 smithi007 ceph-mon[32398]: pgmap v555: 129 pgs: 129 active+clean; 59 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 332 op/s 2024-06-10T01:10:07.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:07 smithi078 ceph-mon[36683]: pgmap v555: 129 pgs: 129 active+clean; 59 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 332 op/s 2024-06-10T01:10:07.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:07 smithi175 ceph-mon[36636]: pgmap v555: 129 pgs: 129 active+clean; 59 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 332 op/s 2024-06-10T01:10:07.715 INFO:tasks.workunit.client.0.smithi007.stdout: 1 259415 9.56 MB/sec execute 48 sec latency 6054.915 ms 2024-06-10T01:10:08.060 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:10:08.311 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:10:08.715 INFO:tasks.workunit.client.0.smithi007.stdout: 1 259415 9.37 MB/sec execute 49 sec latency 7054.999 ms 2024-06-10T01:10:09.450 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:09 smithi007 ceph-mon[32398]: pgmap v556: 129 pgs: 129 active+clean; 59 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 290 op/s 2024-06-10T01:10:09.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:09 smithi078 ceph-mon[36683]: pgmap v556: 129 pgs: 129 active+clean; 59 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 290 op/s 2024-06-10T01:10:09.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:09 smithi175 ceph-mon[36636]: pgmap v556: 129 pgs: 129 active+clean; 59 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 290 op/s 2024-06-10T01:10:09.715 INFO:tasks.workunit.client.0.smithi007.stdout: 1 259415 9.18 MB/sec execute 50 sec latency 8055.053 ms 2024-06-10T01:10:09.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.001+0000 7f3d89fa5700 1 -- 172.21.15.7:0/2604367997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d84101fd0 msgr2=0x7f3d84105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:09.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.001+0000 7f3d89fa5700 1 --2- 172.21.15.7:0/2604367997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d84101fd0 0x7f3d84105fd0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3d780099c0 tx=0x7f3d78009cd0 comp rx=0 tx=0).stop 2024-06-10T01:10:09.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.002+0000 7f3d89fa5700 1 -- 172.21.15.7:0/2604367997 shutdown_connections 2024-06-10T01:10:09.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.002+0000 7f3d89fa5700 1 --2- 172.21.15.7:0/2604367997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d841066b0 0x7f3d8410e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:09.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.002+0000 7f3d89fa5700 1 --2- 172.21.15.7:0/2604367997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d84101620 0x7f3d84101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:09.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.002+0000 7f3d89fa5700 1 --2- 172.21.15.7:0/2604367997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d84101fd0 0x7f3d84105fd0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:09.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.002+0000 7f3d89fa5700 1 -- 172.21.15.7:0/2604367997 >> 172.21.15.7:0/2604367997 conn(0x7f3d84078340 msgr2=0x7f3d84078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:10.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.002+0000 7f3d89fa5700 1 -- 172.21.15.7:0/2604367997 shutdown_connections 2024-06-10T01:10:10.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.002+0000 7f3d89fa5700 1 -- 172.21.15.7:0/2604367997 wait complete. 2024-06-10T01:10:10.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.002+0000 7f3d89fa5700 1 Processor -- start 2024-06-10T01:10:10.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.003+0000 7f3d89fa5700 1 -- start start 2024-06-10T01:10:10.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.003+0000 7f3d89fa5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d84101620 0x7f3d84197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:10.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.003+0000 7f3d89fa5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d84101fd0 0x7f3d84198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:10.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.003+0000 7f3d89fa5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d841066b0 0x7f3d84198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:10.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.003+0000 7f3d89fa5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3d84111770 con 0x7f3d84101620 2024-06-10T01:10:10.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.003+0000 7f3d89fa5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3d841115f0 con 0x7f3d841066b0 2024-06-10T01:10:10.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.003+0000 7f3d89fa5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3d841118f0 con 0x7f3d84101fd0 2024-06-10T01:10:10.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.003+0000 7f3d837fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d84101620 0x7f3d84197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:10.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.003+0000 7f3d837fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d84101620 0x7f3d84197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39668/0 (socket says 172.21.15.7:39668) 2024-06-10T01:10:10.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.003+0000 7f3d837fe700 1 -- 172.21.15.7:0/377870766 learned_addr learned my addr 172.21.15.7:0/377870766 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:10.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.003+0000 7f3d82ffd700 1 --2- 172.21.15.7:0/377870766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d84101fd0 0x7f3d84198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:10.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.003+0000 7f3d83fff700 1 --2- 172.21.15.7:0/377870766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d841066b0 0x7f3d84198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:10.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.003+0000 7f3d837fe700 1 -- 172.21.15.7:0/377870766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d84101fd0 msgr2=0x7f3d84198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:10.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.003+0000 7f3d837fe700 1 --2- 172.21.15.7:0/377870766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d84101fd0 0x7f3d84198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:10.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.003+0000 7f3d837fe700 1 -- 172.21.15.7:0/377870766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d841066b0 msgr2=0x7f3d84198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:10.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.003+0000 7f3d837fe700 1 --2- 172.21.15.7:0/377870766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d841066b0 0x7f3d84198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:10.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.003+0000 7f3d837fe700 1 -- 172.21.15.7:0/377870766 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d841a7340 con 0x7f3d84101620 2024-06-10T01:10:10.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.004+0000 7f3d837fe700 1 --2- 172.21.15.7:0/377870766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d84101620 0x7f3d84197ad0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400b530 tx=0x7f3d7400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:10.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.004+0000 7f3d80ff9700 1 -- 172.21.15.7:0/377870766 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d74015020 con 0x7f3d84101620 2024-06-10T01:10:10.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.004+0000 7f3d80ff9700 1 -- 172.21.15.7:0/377870766 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d74011d10 con 0x7f3d84101620 2024-06-10T01:10:10.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.004+0000 7f3d89fa5700 1 -- 172.21.15.7:0/377870766 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d841a7630 con 0x7f3d84101620 2024-06-10T01:10:10.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.004+0000 7f3d80ff9700 1 -- 172.21.15.7:0/377870766 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7400f2e0 con 0x7f3d84101620 2024-06-10T01:10:10.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.004+0000 7f3d89fa5700 1 -- 172.21.15.7:0/377870766 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d841a7b40 con 0x7f3d84101620 2024-06-10T01:10:10.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.006+0000 7f3d80ff9700 1 -- 172.21.15.7:0/377870766 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f3d74018020 con 0x7f3d84101620 2024-06-10T01:10:10.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.007+0000 7f3d80ff9700 1 --2- 172.21.15.7:0/377870766 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3d6c0a2c20 0x7f3d6c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:10.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.007+0000 7f3d82ffd700 1 --2- 172.21.15.7:0/377870766 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3d6c0a2c20 0x7f3d6c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:10.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.007+0000 7f3d80ff9700 1 -- 172.21.15.7:0/377870766 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f3d740ceac0 con 0x7f3d84101620 2024-06-10T01:10:10.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.007+0000 7f3d82ffd700 1 --2- 172.21.15.7:0/377870766 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3d6c0a2c20 0x7f3d6c0a50e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3d78000c00 tx=0x7f3d78005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:10.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.007+0000 7f3d94828b80 1 Processor -- start 2024-06-10T01:10:10.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.007+0000 7f3d94828b80 1 -- start start 2024-06-10T01:10:10.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.008+0000 7f3d94828b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650a510e640 0x5650a510ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:10.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.008+0000 7f3d94828b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650a510ef60 0x5650a51089e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:10.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.008+0000 7f3d94828b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650a5108f20 0x5650a5109340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:10.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.008+0000 7f3d94828b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5650a505a810 con 0x5650a510ef60 2024-06-10T01:10:10.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.008+0000 7f3d94828b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5650a5002720 con 0x5650a510e640 2024-06-10T01:10:10.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.008+0000 7f3d94828b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5650a50567d0 con 0x5650a5108f20 2024-06-10T01:10:10.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.008+0000 7f3d82ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650a510ef60 0x5650a51089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:10.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.008+0000 7f3d82ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650a510ef60 0x5650a51089e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39670/0 (socket says 172.21.15.7:39670) 2024-06-10T01:10:10.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.008+0000 7f3d82ffd700 1 -- 172.21.15.7:0/3633433040 learned_addr learned my addr 172.21.15.7:0/3633433040 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:10.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.008+0000 7f3d83fff700 1 --2- 172.21.15.7:0/3633433040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650a5108f20 0x5650a5109340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:10.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.008+0000 7f3d837fe700 1 --2- 172.21.15.7:0/3633433040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650a510e640 0x5650a510ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:10.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.008+0000 7f3d82ffd700 1 -- 172.21.15.7:0/3633433040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650a5108f20 msgr2=0x5650a5109340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:10.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.008+0000 7f3d82ffd700 1 --2- 172.21.15.7:0/3633433040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650a5108f20 0x5650a5109340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:10.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.008+0000 7f3d82ffd700 1 -- 172.21.15.7:0/3633433040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650a510e640 msgr2=0x5650a510ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:10.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.008+0000 7f3d82ffd700 1 --2- 172.21.15.7:0/3633433040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650a510e640 0x5650a510ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:10.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.008+0000 7f3d82ffd700 1 -- 172.21.15.7:0/3633433040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650a4ff8f50 con 0x5650a510ef60 2024-06-10T01:10:10.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.008+0000 7f3d82ffd700 1 --2- 172.21.15.7:0/3633433040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650a510ef60 0x5650a51089e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3d78015000 tx=0x7f3d78020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f25c4e454ae4657 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:10.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.009+0000 7f3d69ffb700 1 -- 172.21.15.7:0/3633433040 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7800f590 con 0x5650a510ef60 2024-06-10T01:10:10.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.009+0000 7f3d69ffb700 1 -- 172.21.15.7:0/3633433040 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d7800fc10 con 0x5650a510ef60 2024-06-10T01:10:10.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.009+0000 7f3d69ffb700 1 -- 172.21.15.7:0/3633433040 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d78024e30 con 0x5650a510ef60 2024-06-10T01:10:10.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.009+0000 7f3d94828b80 1 -- 172.21.15.7:0/3633433040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650a510ef60 msgr2=0x5650a51089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:10.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.009+0000 7f3d94828b80 1 --2- 172.21.15.7:0/3633433040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650a510ef60 0x5650a51089e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3d78015000 tx=0x7f3d78020000 comp rx=0 tx=0).stop 2024-06-10T01:10:10.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.009+0000 7f3d94828b80 1 -- 172.21.15.7:0/3633433040 shutdown_connections 2024-06-10T01:10:10.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.009+0000 7f3d94828b80 1 --2- 172.21.15.7:0/3633433040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650a5108f20 0x5650a5109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:10.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.009+0000 7f3d94828b80 1 --2- 172.21.15.7:0/3633433040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650a510e640 0x5650a510ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:10.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.009+0000 7f3d94828b80 1 --2- 172.21.15.7:0/3633433040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650a510ef60 0x5650a51089e0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:10.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.009+0000 7f3d94828b80 1 -- 172.21.15.7:0/3633433040 >> 172.21.15.7:0/3633433040 conn(0x5650a5033cc0 msgr2=0x5650a5103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:10.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.010+0000 7f3d94828b80 1 -- 172.21.15.7:0/3633433040 shutdown_connections 2024-06-10T01:10:10.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.010+0000 7f3d94828b80 1 -- 172.21.15.7:0/3633433040 wait complete. 2024-06-10T01:10:10.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.010+0000 7f3d94828b80 1 -- start start 2024-06-10T01:10:10.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.010+0000 7f3d94828b80 1 Processor -- start 2024-06-10T01:10:10.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.010+0000 7f3d94828b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650a5108f20 0x5650a5114cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:10.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.010+0000 7f3d94828b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650a510e640 0x5650a5111fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:10.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.010+0000 7f3d94828b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650a510ef60 0x5650a5112500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:10.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.010+0000 7f3d94828b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5650a5115910 con 0x5650a510e640 2024-06-10T01:10:10.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.010+0000 7f3d94828b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5650a4ffabf0 con 0x5650a5108f20 2024-06-10T01:10:10.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.010+0000 7f3d94828b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5650a5115a90 con 0x5650a510ef60 2024-06-10T01:10:10.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.010+0000 7f3d94828b80 10 client.0 _ll_register_callbacks cb 0x5650a4ff4920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:10.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.011+0000 7f3d82ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650a510e640 0x5650a5111fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:10.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.011+0000 7f3d82ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650a510e640 0x5650a5111fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39678/0 (socket says 172.21.15.7:39678) 2024-06-10T01:10:10.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.011+0000 7f3d82ffd700 1 -- 172.21.15.7:0/1709429595 learned_addr learned my addr 172.21.15.7:0/1709429595 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:10.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.011+0000 7f3d837fe700 1 --2- 172.21.15.7:0/1709429595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650a5108f20 0x5650a5114cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:10.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.011+0000 7f3d83fff700 1 --2- 172.21.15.7:0/1709429595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650a510ef60 0x5650a5112500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:10.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.011+0000 7f3d82ffd700 1 -- 172.21.15.7:0/1709429595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650a510ef60 msgr2=0x5650a5112500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:10.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.011+0000 7f3d82ffd700 1 --2- 172.21.15.7:0/1709429595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650a510ef60 0x5650a5112500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:10.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.011+0000 7f3d82ffd700 1 -- 172.21.15.7:0/1709429595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650a5108f20 msgr2=0x5650a5114cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:10.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.011+0000 7f3d82ffd700 1 --2- 172.21.15.7:0/1709429595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650a5108f20 0x5650a5114cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:10.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.011+0000 7f3d82ffd700 1 -- 172.21.15.7:0/1709429595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650a4fa98d0 con 0x5650a510e640 2024-06-10T01:10:10.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.011+0000 7f3d82ffd700 1 --2- 172.21.15.7:0/1709429595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650a510e640 0x5650a5111fc0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f3d78009d10 tx=0x7f3d780092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af7ac79c8db090d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:10.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.011+0000 7f3d62ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:10:10.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.011+0000 7f3d62ffd700 1 -- 172.21.15.7:0/1709429595 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7803c060 con 0x5650a510e640 2024-06-10T01:10:10.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.011+0000 7f3d62ffd700 1 -- 172.21.15.7:0/1709429595 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f3d7800f590 con 0x5650a510e640 2024-06-10T01:10:10.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.011+0000 7f3d94828b80 1 -- 172.21.15.7:0/1709429595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d7800fc10 con 0x5650a510e640 2024-06-10T01:10:10.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.012+0000 7f3d62ffd700 1 -- 172.21.15.7:0/1709429595 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3d7800f790 con 0x5650a510e640 2024-06-10T01:10:10.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.012+0000 7f3d94828b80 10 client.16530 fetch_fsmap learned FSMap version 30 2024-06-10T01:10:10.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.012+0000 7f3d94828b80 1 -- 172.21.15.7:0/1709429595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5650a5120210 con 0x5650a510e640 2024-06-10T01:10:10.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.012+0000 7f3d62ffd700 1 -- 172.21.15.7:0/1709429595 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d7800fc10 con 0x5650a510e640 2024-06-10T01:10:10.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.012+0000 7f3d62ffd700 1 -- 172.21.15.7:0/1709429595 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f3d780174f0 con 0x5650a510e640 2024-06-10T01:10:10.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.012+0000 7f3d94828b80 10 client.16530 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:10:10.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.012+0000 7f3d94828b80 10 client.16530 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:10:10.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.012+0000 7f3d94828b80 20 client.16530 populate_metadata read hostname 'smithi007' 2024-06-10T01:10:10.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.012+0000 7f3d94828b80 1 --2- 172.21.15.7:0/1709429595 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5650a51226e0 0x5650a5124ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:10.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.012+0000 7f3d94828b80 4 client.16530 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:10:10.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.012+0000 7f3d94828b80 1 -- 172.21.15.7:0/1709429595 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650a5067d10 con 0x5650a51226e0 2024-06-10T01:10:10.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.013+0000 7f3d837fe700 1 --2- 172.21.15.7:0/1709429595 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5650a51226e0 0x5650a5124ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:10.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.013+0000 7f3d837fe700 1 --2- 172.21.15.7:0/1709429595 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5650a51226e0 0x5650a5124ad0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7a2b65e905b375e9 server_cookie=433cdb8a93f0a11a in_seq=0 out_seq=0 2024-06-10T01:10:10.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.013+0000 7f3d62ffd700 10 client.16530 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:10:10.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.014+0000 7f3d62ffd700 1 -- 172.21.15.7:0/1709429595 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5650a5067d10 con 0x5650a51226e0 2024-06-10T01:10:10.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.014+0000 7f3d62ffd700 10 client.16530 handle_command_reply: tid=0 2024-06-10T01:10:10.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.014+0000 7f3d94828b80 1 client.16530 shutdown 2024-06-10T01:10:10.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.014+0000 7f3d94828b80 1 -- 172.21.15.7:0/1709429595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650a510e640 msgr2=0x5650a5111fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:10.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.014+0000 7f3d94828b80 1 --2- 172.21.15.7:0/1709429595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650a510e640 0x5650a5111fc0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f3d78009d10 tx=0x7f3d780092f0 comp rx=0 tx=0).stop 2024-06-10T01:10:10.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.014+0000 7f3d94828b80 1 -- 172.21.15.7:0/1709429595 shutdown_connections 2024-06-10T01:10:10.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.014+0000 7f3d94828b80 1 --2- 172.21.15.7:0/1709429595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650a510ef60 0x5650a5112500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:10.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.014+0000 7f3d94828b80 1 --2- 172.21.15.7:0/1709429595 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5650a51226e0 0x5650a5124ad0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:10.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.014+0000 7f3d62ffd700 0 client.16530 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:10:10.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.014+0000 7f3d94828b80 1 --2- 172.21.15.7:0/1709429595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650a5108f20 0x5650a5114cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:10.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.014+0000 7f3d94828b80 1 --2- 172.21.15.7:0/1709429595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650a510e640 0x5650a5111fc0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:10.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.014+0000 7f3d94828b80 1 -- 172.21.15.7:0/1709429595 >> 172.21.15.7:0/1709429595 conn(0x5650a5033cc0 msgr2=0x5650a51035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:10.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.014+0000 7f3d94828b80 1 -- 172.21.15.7:0/1709429595 shutdown_connections 2024-06-10T01:10:10.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.014+0000 7f3d94828b80 1 -- 172.21.15.7:0/1709429595 wait complete. 2024-06-10T01:10:10.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.014+0000 7f3d94828b80 20 client.16530 trim_cache size 0 max 16384 2024-06-10T01:10:10.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.029+0000 7f3d94828b80 1 Processor -- start 2024-06-10T01:10:10.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.030+0000 7f3d94828b80 1 -- start start 2024-06-10T01:10:10.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.030+0000 7f3d94828b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650a5108f20 0x5650a51089e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:10.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.030+0000 7f3d94828b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650a510e640 0x5650a5111f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:10.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.030+0000 7f3d94828b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650a510ef60 0x5650a51124c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:10.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.030+0000 7f3d94828b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3d841b2f30 con 0x5650a510e640 2024-06-10T01:10:10.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.030+0000 7f3d94828b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3d841b0760 con 0x5650a510ef60 2024-06-10T01:10:10.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.030+0000 7f3d94828b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3d841ae5b0 con 0x5650a5108f20 2024-06-10T01:10:10.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.030+0000 7f3d82ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650a510e640 0x5650a5111f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:10.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.030+0000 7f3d82ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650a510e640 0x5650a5111f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39694/0 (socket says 172.21.15.7:39694) 2024-06-10T01:10:10.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.030+0000 7f3d82ffd700 1 -- 172.21.15.7:0/1746374088 learned_addr learned my addr 172.21.15.7:0/1746374088 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:10.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.030+0000 7f3d837fe700 1 --2- 172.21.15.7:0/1746374088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650a5108f20 0x5650a51089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:10.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.030+0000 7f3d82ffd700 1 -- 172.21.15.7:0/1746374088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650a5108f20 msgr2=0x5650a51089e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:10.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.030+0000 7f3d82ffd700 1 --2- 172.21.15.7:0/1746374088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650a5108f20 0x5650a51089e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:10.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.030+0000 7f3d83fff700 1 --2- 172.21.15.7:0/1746374088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650a510ef60 0x5650a51124c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:10.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.030+0000 7f3d82ffd700 1 -- 172.21.15.7:0/1746374088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650a510ef60 msgr2=0x5650a51124c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:10.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.030+0000 7f3d82ffd700 1 --2- 172.21.15.7:0/1746374088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650a510ef60 0x5650a51124c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:10.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.030+0000 7f3d82ffd700 1 -- 172.21.15.7:0/1746374088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650a510e140 con 0x5650a510e640 2024-06-10T01:10:10.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.030+0000 7f3d83fff700 1 --2- 172.21.15.7:0/1746374088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650a510ef60 0x5650a51124c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:10:10.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.030+0000 7f3d82ffd700 1 --2- 172.21.15.7:0/1746374088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650a510e640 0x5650a5111f80 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3d780098e0 tx=0x7f3d78017f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462ad436b2fa89d3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:10.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.031+0000 7f3d697fa700 1 -- 172.21.15.7:0/1746374088 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7803c040 con 0x5650a510e640 2024-06-10T01:10:10.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.031+0000 7f3d697fa700 1 -- 172.21.15.7:0/1746374088 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d7800f460 con 0x5650a510e640 2024-06-10T01:10:10.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.031+0000 7f3d697fa700 1 -- 172.21.15.7:0/1746374088 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d780173f0 con 0x5650a510e640 2024-06-10T01:10:10.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.031+0000 7f3d94828b80 1 -- 172.21.15.7:0/1746374088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650a510e640 msgr2=0x5650a5111f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:10.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.031+0000 7f3d94828b80 1 --2- 172.21.15.7:0/1746374088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650a510e640 0x5650a5111f80 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3d780098e0 tx=0x7f3d78017f60 comp rx=0 tx=0).stop 2024-06-10T01:10:10.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.031+0000 7f3d94828b80 1 -- 172.21.15.7:0/1746374088 shutdown_connections 2024-06-10T01:10:10.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.031+0000 7f3d94828b80 1 --2- 172.21.15.7:0/1746374088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650a510ef60 0x5650a51124c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:10.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.031+0000 7f3d94828b80 1 --2- 172.21.15.7:0/1746374088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650a5108f20 0x5650a51089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:10.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.031+0000 7f3d94828b80 1 --2- 172.21.15.7:0/1746374088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650a510e640 0x5650a5111f80 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:10.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.031+0000 7f3d94828b80 1 -- 172.21.15.7:0/1746374088 >> 172.21.15.7:0/1746374088 conn(0x5650a5033cc0 msgr2=0x5650a51108f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:10.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.032+0000 7f3d94828b80 1 -- 172.21.15.7:0/1746374088 shutdown_connections 2024-06-10T01:10:10.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.032+0000 7f3d94828b80 1 -- 172.21.15.7:0/1746374088 wait complete. 2024-06-10T01:10:10.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.032+0000 7f3d94828b80 1 -- start start 2024-06-10T01:10:10.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.032+0000 7f3d94828b80 1 Processor -- start 2024-06-10T01:10:10.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.032+0000 7f3d94828b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650a5108f20 0x5650a5127bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:10.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.032+0000 7f3d94828b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650a510e640 0x5650a51280f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:10.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.032+0000 7f3d94828b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650a510ef60 0x5650a51250b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:10.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.033+0000 7f3d94828b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3d841aec20 con 0x5650a510e640 2024-06-10T01:10:10.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.033+0000 7f3d94828b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3d841b0090 con 0x5650a510ef60 2024-06-10T01:10:10.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.033+0000 7f3d94828b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3d841af2f0 con 0x5650a5108f20 2024-06-10T01:10:10.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.033+0000 7f3d82ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650a510e640 0x5650a51280f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:10.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.033+0000 7f3d94828b80 10 client.0 _ll_register_callbacks cb 0x5650a4ff4920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:10.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.033+0000 7f3d82ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650a510e640 0x5650a51280f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39696/0 (socket says 172.21.15.7:39696) 2024-06-10T01:10:10.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.033+0000 7f3d82ffd700 1 -- 172.21.15.7:0/3561109219 learned_addr learned my addr 172.21.15.7:0/3561109219 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:10.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.033+0000 7f3d837fe700 1 --2- 172.21.15.7:0/3561109219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650a5108f20 0x5650a5127bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:10.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.033+0000 7f3d83fff700 1 --2- 172.21.15.7:0/3561109219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650a510ef60 0x5650a51250b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:10.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.033+0000 7f3d82ffd700 1 -- 172.21.15.7:0/3561109219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650a5108f20 msgr2=0x5650a5127bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:10.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.033+0000 7f3d82ffd700 1 --2- 172.21.15.7:0/3561109219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650a5108f20 0x5650a5127bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:10.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.033+0000 7f3d82ffd700 1 -- 172.21.15.7:0/3561109219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650a510ef60 msgr2=0x5650a51250b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:10.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.033+0000 7f3d82ffd700 1 --2- 172.21.15.7:0/3561109219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650a510ef60 0x5650a51250b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:10.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.033+0000 7f3d82ffd700 1 -- 172.21.15.7:0/3561109219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650a5125850 con 0x5650a510e640 2024-06-10T01:10:10.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.033+0000 7f3d837fe700 1 --2- 172.21.15.7:0/3561109219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650a5108f20 0x5650a5127bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:10.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.033+0000 7f3d82ffd700 1 --2- 172.21.15.7:0/3561109219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650a510e640 0x5650a51280f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3d78017960 tx=0x7f3d780177d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7a7183817ce584 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:10.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.033+0000 7f3d62ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:10:10.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.033+0000 7f3d62ffd700 1 -- 172.21.15.7:0/3561109219 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7804b060 con 0x5650a510e640 2024-06-10T01:10:10.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.033+0000 7f3d62ffd700 1 -- 172.21.15.7:0/3561109219 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f3d78029040 con 0x5650a510e640 2024-06-10T01:10:10.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.033+0000 7f3d94828b80 1 -- 172.21.15.7:0/3561109219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d7800f460 con 0x5650a510e640 2024-06-10T01:10:10.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.034+0000 7f3d62ffd700 1 -- 172.21.15.7:0/3561109219 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3d78052050 con 0x5650a510e640 2024-06-10T01:10:10.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.034+0000 7f3d94828b80 10 client.16536 fetch_fsmap learned FSMap version 30 2024-06-10T01:10:10.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.034+0000 7f3d94828b80 1 -- 172.21.15.7:0/3561109219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5650a518ca30 con 0x5650a510e640 2024-06-10T01:10:10.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.034+0000 7f3d62ffd700 1 -- 172.21.15.7:0/3561109219 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d7800f460 con 0x5650a510e640 2024-06-10T01:10:10.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.034+0000 7f3d62ffd700 1 -- 172.21.15.7:0/3561109219 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f3d78045070 con 0x5650a510e640 2024-06-10T01:10:10.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.034+0000 7f3d94828b80 10 client.16536 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:10:10.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.034+0000 7f3d94828b80 10 client.16536 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:10:10.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.034+0000 7f3d94828b80 20 client.16536 populate_metadata read hostname 'smithi007' 2024-06-10T01:10:10.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.034+0000 7f3d94828b80 1 --2- 172.21.15.7:0/3561109219 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5650a51226e0 0x5650a518eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:10.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.034+0000 7f3d94828b80 4 client.16536 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:10:10.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.034+0000 7f3d94828b80 1 -- 172.21.15.7:0/3561109219 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5650a5062290 con 0x5650a51226e0 2024-06-10T01:10:10.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.035+0000 7f3d837fe700 1 --2- 172.21.15.7:0/3561109219 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5650a51226e0 0x5650a518eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:10.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.035+0000 7f3d62ffd700 10 client.16536 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:10:10.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.035+0000 7f3d837fe700 1 --2- 172.21.15.7:0/3561109219 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5650a51226e0 0x5650a518eef0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5f99ad19c601c02d server_cookie=f6562088478f0fc6 in_seq=0 out_seq=0 2024-06-10T01:10:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.035+0000 7f3d62ffd700 1 -- 172.21.15.7:0/3561109219 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5650a5062290 con 0x5650a51226e0 2024-06-10T01:10:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.035+0000 7f3d62ffd700 10 client.16536 handle_command_reply: tid=0 2024-06-10T01:10:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.035+0000 7f3d94828b80 1 client.16536 shutdown 2024-06-10T01:10:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.035+0000 7f3d94828b80 1 -- 172.21.15.7:0/3561109219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650a510e640 msgr2=0x5650a51280f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.035+0000 7f3d94828b80 1 --2- 172.21.15.7:0/3561109219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650a510e640 0x5650a51280f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3d78017960 tx=0x7f3d780177d0 comp rx=0 tx=0).stop 2024-06-10T01:10:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.035+0000 7f3d94828b80 1 -- 172.21.15.7:0/3561109219 shutdown_connections 2024-06-10T01:10:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.035+0000 7f3d94828b80 1 --2- 172.21.15.7:0/3561109219 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5650a51226e0 0x5650a518eef0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.035+0000 7f3d94828b80 1 --2- 172.21.15.7:0/3561109219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650a510ef60 0x5650a51250b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.035+0000 7f3d62ffd700 0 client.16536 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:10:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.035+0000 7f3d94828b80 1 --2- 172.21.15.7:0/3561109219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650a5108f20 0x5650a5127bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.035+0000 7f3d94828b80 1 --2- 172.21.15.7:0/3561109219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650a510e640 0x5650a51280f0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.035+0000 7f3d94828b80 1 -- 172.21.15.7:0/3561109219 >> 172.21.15.7:0/3561109219 conn(0x5650a5033cc0 msgr2=0x5650a50fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.036+0000 7f3d94828b80 1 -- 172.21.15.7:0/3561109219 shutdown_connections 2024-06-10T01:10:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.036+0000 7f3d94828b80 1 -- 172.21.15.7:0/3561109219 wait complete. 2024-06-10T01:10:10.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.036+0000 7f3d94828b80 20 client.16536 trim_cache size 0 max 16384 2024-06-10T01:10:10.036 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:10:10.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.036+0000 7f3d89fa5700 1 -- 172.21.15.7:0/377870766 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3d6c0a2c20 msgr2=0x7f3d6c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:10.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.036+0000 7f3d89fa5700 1 --2- 172.21.15.7:0/377870766 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3d6c0a2c20 0x7f3d6c0a50e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3d78000c00 tx=0x7f3d78005fb0 comp rx=0 tx=0).stop 2024-06-10T01:10:10.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.036+0000 7f3d89fa5700 1 -- 172.21.15.7:0/377870766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d84101620 msgr2=0x7f3d84197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:10.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.036+0000 7f3d89fa5700 1 --2- 172.21.15.7:0/377870766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d84101620 0x7f3d84197ad0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400b530 tx=0x7f3d7400b840 comp rx=0 tx=0).stop 2024-06-10T01:10:10.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.036+0000 7f3d89fa5700 1 -- 172.21.15.7:0/377870766 shutdown_connections 2024-06-10T01:10:10.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.036+0000 7f3d89fa5700 1 --2- 172.21.15.7:0/377870766 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3d6c0a2c20 0x7f3d6c0a50e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:10.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.036+0000 7f3d89fa5700 1 --2- 172.21.15.7:0/377870766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d841066b0 0x7f3d84198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:10.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.036+0000 7f3d89fa5700 1 --2- 172.21.15.7:0/377870766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d84101620 0x7f3d84197ad0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:10.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.036+0000 7f3d89fa5700 1 --2- 172.21.15.7:0/377870766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d84101fd0 0x7f3d84198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:10.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.036+0000 7f3d89fa5700 1 -- 172.21.15.7:0/377870766 >> 172.21.15.7:0/377870766 conn(0x7f3d84078340 msgr2=0x7f3d840ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:10.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.037+0000 7f3d89fa5700 1 -- 172.21.15.7:0/377870766 shutdown_connections 2024-06-10T01:10:10.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:10.037+0000 7f3d89fa5700 1 -- 172.21.15.7:0/377870766 wait complete. 2024-06-10T01:10:10.140 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:10:10.715 INFO:tasks.workunit.client.0.smithi007.stdout: 1 259415 9.00 MB/sec execute 51 sec latency 9055.138 ms 2024-06-10T01:10:10.823 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:10:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:10:11.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:11 smithi078 ceph-mon[36683]: pgmap v557: 129 pgs: 129 active+clean; 57 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 2.6 MiB/s wr, 267 op/s 2024-06-10T01:10:11.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:11 smithi175 ceph-mon[36636]: pgmap v557: 129 pgs: 129 active+clean; 57 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 2.6 MiB/s wr, 267 op/s 2024-06-10T01:10:11.715 INFO:tasks.workunit.client.0.smithi007.stdout: 1 259415 8.83 MB/sec execute 52 sec latency 10055.255 ms 2024-06-10T01:10:11.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:11 smithi007 ceph-mon[32398]: pgmap v557: 129 pgs: 129 active+clean; 57 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 2.6 MiB/s wr, 267 op/s 2024-06-10T01:10:12.229 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:10:12.715 INFO:tasks.workunit.client.0.smithi007.stdout: 1 259415 8.66 MB/sec execute 53 sec latency 11055.298 ms 2024-06-10T01:10:12.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.953+0000 7fb4f7efc700 1 -- 172.21.15.7:0/312899842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4f006ab30 msgr2=0x7fb4f010f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:12.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.953+0000 7fb4f7efc700 1 --2- 172.21.15.7:0/312899842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4f006ab30 0x7fb4f010f320 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb4e800b0a0 tx=0x7fb4e800b3b0 comp rx=0 tx=0).stop 2024-06-10T01:10:12.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.953+0000 7fb4f7efc700 1 -- 172.21.15.7:0/312899842 shutdown_connections 2024-06-10T01:10:12.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.953+0000 7fb4f7efc700 1 --2- 172.21.15.7:0/312899842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4f006ab30 0x7fb4f010f320 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:12.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.953+0000 7fb4f7efc700 1 --2- 172.21.15.7:0/312899842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4f0108500 0x7fb4f01088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:12.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.953+0000 7fb4f7efc700 1 --2- 172.21.15.7:0/312899842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f006a170 0x7fb4f006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:12.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.953+0000 7fb4f7efc700 1 -- 172.21.15.7:0/312899842 >> 172.21.15.7:0/312899842 conn(0x7fb4f0078340 msgr2=0x7fb4f0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:12.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.953+0000 7fb4f7efc700 1 -- 172.21.15.7:0/312899842 shutdown_connections 2024-06-10T01:10:12.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.953+0000 7fb4f7efc700 1 -- 172.21.15.7:0/312899842 wait complete. 2024-06-10T01:10:12.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.954+0000 7fb4f7efc700 1 Processor -- start 2024-06-10T01:10:12.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.954+0000 7fb4f7efc700 1 -- start start 2024-06-10T01:10:12.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.954+0000 7fb4f7efc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f006a170 0x7fb4f0101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:12.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.954+0000 7fb4f7efc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4f006ab30 0x7fb4f01019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:12.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.954+0000 7fb4f7efc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4f0108500 0x7fb4f0101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:12.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.954+0000 7fb4f7efc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f0112780 con 0x7fb4f0108500 2024-06-10T01:10:12.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.954+0000 7fb4f7efc700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f0112600 con 0x7fb4f006a170 2024-06-10T01:10:12.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.954+0000 7fb4f7efc700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f0112900 con 0x7fb4f006ab30 2024-06-10T01:10:12.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.954+0000 7fb4f6499700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4f0108500 0x7fb4f0101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:12.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.954+0000 7fb4f6499700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4f0108500 0x7fb4f0101ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60484/0 (socket says 172.21.15.7:60484) 2024-06-10T01:10:12.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.954+0000 7fb4f6499700 1 -- 172.21.15.7:0/2900628669 learned_addr learned my addr 172.21.15.7:0/2900628669 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:12.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.954+0000 7fb4f5c98700 1 --2- 172.21.15.7:0/2900628669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f006a170 0x7fb4f0101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:12.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.954+0000 7fb4f5497700 1 --2- 172.21.15.7:0/2900628669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4f006ab30 0x7fb4f01019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:12.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.954+0000 7fb4f6499700 1 -- 172.21.15.7:0/2900628669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4f006ab30 msgr2=0x7fb4f01019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:12.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.954+0000 7fb4f6499700 1 --2- 172.21.15.7:0/2900628669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4f006ab30 0x7fb4f01019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:12.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.954+0000 7fb4f6499700 1 -- 172.21.15.7:0/2900628669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f006a170 msgr2=0x7fb4f0101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:12.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.954+0000 7fb4f6499700 1 --2- 172.21.15.7:0/2900628669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f006a170 0x7fb4f0101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:12.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.954+0000 7fb4f6499700 1 -- 172.21.15.7:0/2900628669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f01a70f0 con 0x7fb4f0108500 2024-06-10T01:10:12.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.954+0000 7fb4f5497700 1 --2- 172.21.15.7:0/2900628669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4f006ab30 0x7fb4f01019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:10:12.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.954+0000 7fb4f6499700 1 --2- 172.21.15.7:0/2900628669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4f0108500 0x7fb4f0101ee0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80062a0 tx=0x7fb4e8004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:12.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.955+0000 7fb4e2ffd700 1 -- 172.21.15.7:0/2900628669 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8021070 con 0x7fb4f0108500 2024-06-10T01:10:12.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.955+0000 7fb4e2ffd700 1 -- 172.21.15.7:0/2900628669 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4e8007c00 con 0x7fb4f0108500 2024-06-10T01:10:12.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.955+0000 7fb4f7efc700 1 -- 172.21.15.7:0/2900628669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4f01a7380 con 0x7fb4f0108500 2024-06-10T01:10:12.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.955+0000 7fb4e2ffd700 1 -- 172.21.15.7:0/2900628669 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e801d450 con 0x7fb4f0108500 2024-06-10T01:10:12.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.955+0000 7fb4f7efc700 1 -- 172.21.15.7:0/2900628669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4f01a7830 con 0x7fb4f0108500 2024-06-10T01:10:12.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.956+0000 7fb4f7efc700 1 -- 172.21.15.7:0/2900628669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4f010a9e0 con 0x7fb4f0108500 2024-06-10T01:10:12.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.957+0000 7fb4e2ffd700 1 -- 172.21.15.7:0/2900628669 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb4e8004000 con 0x7fb4f0108500 2024-06-10T01:10:12.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.958+0000 7fb4e2ffd700 1 --2- 172.21.15.7:0/2900628669 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb4dc0a2c20 0x7fb4dc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:12.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.958+0000 7fb4f5c98700 1 --2- 172.21.15.7:0/2900628669 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb4dc0a2c20 0x7fb4dc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:12.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.958+0000 7fb4e2ffd700 1 -- 172.21.15.7:0/2900628669 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb4e800f020 con 0x7fb4f0108500 2024-06-10T01:10:12.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.958+0000 7fb4f5c98700 1 --2- 172.21.15.7:0/2900628669 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb4dc0a2c20 0x7fb4dc0a50e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb4e40045b0 tx=0x7fb4e400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:12.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:12.962+0000 7fb4e2ffd700 1 -- 172.21.15.7:0/2900628669 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb4e802a020 con 0x7fb4f0108500 2024-06-10T01:10:13.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:13.219+0000 7fb4f7efc700 1 -- 172.21.15.7:0/2900628669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4f0069180 con 0x7fb4f0108500 2024-06-10T01:10:13.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:13.220+0000 7fb4e2ffd700 1 -- 172.21.15.7:0/2900628669 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11858 (secure 0 0 0) 0x7fb4e809d570 con 0x7fb4f0108500 2024-06-10T01:10:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:10:13.219 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:08:17.181785+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24434,24436,24451]},"id":1}]} 2024-06-10T01:10:13.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:13.222+0000 7fb4f7efc700 1 -- 172.21.15.7:0/2900628669 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb4dc0a2c20 msgr2=0x7fb4dc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:13.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:13.222+0000 7fb4f7efc700 1 --2- 172.21.15.7:0/2900628669 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb4dc0a2c20 0x7fb4dc0a50e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb4e40045b0 tx=0x7fb4e400b3c0 comp rx=0 tx=0).stop 2024-06-10T01:10:13.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:13.222+0000 7fb4f7efc700 1 -- 172.21.15.7:0/2900628669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4f0108500 msgr2=0x7fb4f0101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:13.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:13.222+0000 7fb4f7efc700 1 --2- 172.21.15.7:0/2900628669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4f0108500 0x7fb4f0101ee0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80062a0 tx=0x7fb4e8004d70 comp rx=0 tx=0).stop 2024-06-10T01:10:13.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:13.222+0000 7fb4f7efc700 1 -- 172.21.15.7:0/2900628669 shutdown_connections 2024-06-10T01:10:13.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:13.222+0000 7fb4f7efc700 1 --2- 172.21.15.7:0/2900628669 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb4dc0a2c20 0x7fb4dc0a50e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:13.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:13.222+0000 7fb4f7efc700 1 --2- 172.21.15.7:0/2900628669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4f0108500 0x7fb4f0101ee0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:13.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:13.222+0000 7fb4f7efc700 1 --2- 172.21.15.7:0/2900628669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f006a170 0x7fb4f0101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:13.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:13.222+0000 7fb4f7efc700 1 --2- 172.21.15.7:0/2900628669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4f006ab30 0x7fb4f01019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:13.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:13.222+0000 7fb4f7efc700 1 -- 172.21.15.7:0/2900628669 >> 172.21.15.7:0/2900628669 conn(0x7fb4f0078340 msgr2=0x7fb4f0100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:13.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:13.222+0000 7fb4f7efc700 1 -- 172.21.15.7:0/2900628669 shutdown_connections 2024-06-10T01:10:13.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:13.222+0000 7fb4f7efc700 1 -- 172.21.15.7:0/2900628669 wait complete. 2024-06-10T01:10:13.221 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-06-10T01:10:13.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:13 smithi007 ceph-mon[32398]: pgmap v558: 129 pgs: 129 active+clean; 58 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 2.1 MiB/s wr, 213 op/s 2024-06-10T01:10:13.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:13 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2900628669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:10:13.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:13 smithi078 ceph-mon[36683]: pgmap v558: 129 pgs: 129 active+clean; 58 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 2.1 MiB/s wr, 213 op/s 2024-06-10T01:10:13.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:13 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2900628669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:10:13.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:13 smithi175 ceph-mon[36636]: pgmap v558: 129 pgs: 129 active+clean; 58 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 2.1 MiB/s wr, 213 op/s 2024-06-10T01:10:13.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:13 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2900628669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:10:13.715 INFO:tasks.workunit.client.0.smithi007.stdout: 1 259415 8.50 MB/sec execute 54 sec latency 12055.371 ms 2024-06-10T01:10:14.113 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db --set-id 3d7bfe3b --release --await' 2024-06-10T01:10:14.113 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db --set-id 3d7bfe3b --release --await 2024-06-10T01:10:14.715 INFO:tasks.workunit.client.0.smithi007.stdout: 1 259415 8.34 MB/sec execute 55 sec latency 13055.467 ms 2024-06-10T01:10:14.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.910+0000 7fc382aba640 1 -- 172.21.15.7:0/3440274584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc37c104e70 msgr2=0x7fc37c107260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:14.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.910+0000 7fc382aba640 1 --2- 172.21.15.7:0/3440274584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc37c104e70 0x7fc37c107260 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc368009900 tx=0x7fc36802a410 comp rx=0 tx=0).stop 2024-06-10T01:10:14.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.910+0000 7fc382aba640 1 -- 172.21.15.7:0/3440274584 shutdown_connections 2024-06-10T01:10:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.910+0000 7fc382aba640 1 --2- 172.21.15.7:0/3440274584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc37c1077a0 0x7fc37c109b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.910+0000 7fc382aba640 1 --2- 172.21.15.7:0/3440274584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc37c104e70 0x7fc37c107260 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.910+0000 7fc382aba640 1 --2- 172.21.15.7:0/3440274584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc37c102540 0x7fc37c104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.910+0000 7fc382aba640 1 -- 172.21.15.7:0/3440274584 >> 172.21.15.7:0/3440274584 conn(0x7fc37c0fbed0 msgr2=0x7fc37c0fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.910+0000 7fc382aba640 1 -- 172.21.15.7:0/3440274584 shutdown_connections 2024-06-10T01:10:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.910+0000 7fc382aba640 1 -- 172.21.15.7:0/3440274584 wait complete. 2024-06-10T01:10:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.911+0000 7fc382aba640 1 Processor -- start 2024-06-10T01:10:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.911+0000 7fc382aba640 1 -- start start 2024-06-10T01:10:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.911+0000 7fc382aba640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc37c102540 0x7fc37c101eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.911+0000 7fc382aba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc37c104e70 0x7fc37c1004f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.911+0000 7fc382aba640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc37c1077a0 0x7fc37c100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.911+0000 7fc382aba640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c10d6b0 con 0x7fc37c102540 2024-06-10T01:10:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.911+0000 7fc382aba640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c10d530 con 0x7fc37c104e70 2024-06-10T01:10:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.911+0000 7fc382aba640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c10d830 con 0x7fc37c1077a0 2024-06-10T01:10:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.912+0000 7fc381057640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc37c1077a0 0x7fc37c100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.912+0000 7fc380856640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc37c102540 0x7fc37c101eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.912+0000 7fc381057640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc37c1077a0 0x7fc37c100a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34874/0 (socket says 172.21.15.7:34874) 2024-06-10T01:10:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.912+0000 7fc381057640 1 -- 172.21.15.7:0/892150697 learned_addr learned my addr 172.21.15.7:0/892150697 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.912+0000 7fc37bfff640 1 --2- 172.21.15.7:0/892150697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc37c104e70 0x7fc37c1004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.912+0000 7fc381057640 1 -- 172.21.15.7:0/892150697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc37c104e70 msgr2=0x7fc37c1004f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.912+0000 7fc381057640 1 --2- 172.21.15.7:0/892150697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc37c104e70 0x7fc37c1004f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.912+0000 7fc381057640 1 -- 172.21.15.7:0/892150697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc37c102540 msgr2=0x7fc37c101eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.912+0000 7fc381057640 1 --2- 172.21.15.7:0/892150697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc37c102540 0x7fc37c101eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.912+0000 7fc381057640 1 -- 172.21.15.7:0/892150697 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc37c1012c0 con 0x7fc37c1077a0 2024-06-10T01:10:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.912+0000 7fc380856640 1 --2- 172.21.15.7:0/892150697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc37c102540 0x7fc37c101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.913+0000 7fc381057640 1 --2- 172.21.15.7:0/892150697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc37c1077a0 0x7fc37c100a30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc37400eb10 tx=0x7fc37400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.913+0000 7fc379ffb640 1 -- 172.21.15.7:0/892150697 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc374004b20 con 0x7fc37c1077a0 2024-06-10T01:10:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.913+0000 7fc379ffb640 1 -- 172.21.15.7:0/892150697 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc374004cc0 con 0x7fc37c1077a0 2024-06-10T01:10:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.913+0000 7fc382aba640 1 -- 172.21.15.7:0/892150697 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc37c1a1e80 con 0x7fc37c1077a0 2024-06-10T01:10:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.913+0000 7fc379ffb640 1 -- 172.21.15.7:0/892150697 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc374010760 con 0x7fc37c1077a0 2024-06-10T01:10:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.913+0000 7fc382aba640 1 -- 172.21.15.7:0/892150697 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc37c1a2340 con 0x7fc37c1077a0 2024-06-10T01:10:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.916+0000 7fc379ffb640 1 -- 172.21.15.7:0/892150697 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fc374005020 con 0x7fc37c1077a0 2024-06-10T01:10:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.917+0000 7fc379ffb640 1 --2- 172.21.15.7:0/892150697 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc36c0a2c00 0x7fc36c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.917+0000 7fc3857c9740 1 Processor -- start 2024-06-10T01:10:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.917+0000 7fc3857c9740 1 -- start start 2024-06-10T01:10:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.917+0000 7fc379ffb640 1 -- 172.21.15.7:0/892150697 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc3740cfab0 con 0x7fc37c1077a0 2024-06-10T01:10:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.917+0000 7fc380856640 1 --2- 172.21.15.7:0/892150697 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc36c0a2c00 0x7fc36c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.917+0000 7fc3857c9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4300c4010 0x55f4300c4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.917+0000 7fc3857c9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4300c4a00 0x55f4300c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.917+0000 7fc3857c9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4300c9b70 0x55f4300cc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.917+0000 7fc3857c9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f4300274f0 con 0x55f4300c4a00 2024-06-10T01:10:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.917+0000 7fc3857c9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f42ffc5870 con 0x55f4300c4010 2024-06-10T01:10:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.917+0000 7fc3857c9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f4300234b0 con 0x55f4300c9b70 2024-06-10T01:10:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.917+0000 7fc37bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4300c4a00 0x55f4300c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.918+0000 7fc37bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4300c4a00 0x55f4300c9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60514/0 (socket says 172.21.15.7:60514) 2024-06-10T01:10:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.918+0000 7fc37bfff640 1 -- 172.21.15.7:0/2353922152 learned_addr learned my addr 172.21.15.7:0/2353922152 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.918+0000 7fc381057640 1 --2- 172.21.15.7:0/2353922152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4300c9b70 0x55f4300cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.918+0000 7fc380856640 1 --2- 172.21.15.7:0/892150697 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc36c0a2c00 0x7fc36c0a50c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc370005fd0 tx=0x7fc37000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:14.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.918+0000 7fc37bfff640 1 -- 172.21.15.7:0/2353922152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4300c9b70 msgr2=0x55f4300cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:14.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.918+0000 7fc37bfff640 1 --2- 172.21.15.7:0/2353922152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4300c9b70 0x55f4300cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.918+0000 7fc37bfff640 1 -- 172.21.15.7:0/2353922152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4300c4010 msgr2=0x55f4300c4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:14.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.918+0000 7fc37bfff640 1 --2- 172.21.15.7:0/2353922152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4300c4010 0x55f4300c4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.918+0000 7fc37bfff640 1 -- 172.21.15.7:0/2353922152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f42ffe19b0 con 0x55f4300c4a00 2024-06-10T01:10:14.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.918+0000 7fc381057640 1 --2- 172.21.15.7:0/2353922152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4300c9b70 0x55f4300cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:14.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.918+0000 7fc37bfff640 1 --2- 172.21.15.7:0/2353922152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4300c4a00 0x55f4300c9500 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc368031440 tx=0x7fc368005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927c6644804cff1a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:14.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.918+0000 7fc366ffd640 1 -- 172.21.15.7:0/2353922152 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36803f070 con 0x55f4300c4a00 2024-06-10T01:10:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.919+0000 7fc366ffd640 1 -- 172.21.15.7:0/2353922152 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc368004030 con 0x55f4300c4a00 2024-06-10T01:10:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.919+0000 7fc366ffd640 1 -- 172.21.15.7:0/2353922152 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc368035070 con 0x55f4300c4a00 2024-06-10T01:10:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.919+0000 7fc3857c9740 1 -- 172.21.15.7:0/2353922152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4300c4a00 msgr2=0x55f4300c9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.919+0000 7fc3857c9740 1 --2- 172.21.15.7:0/2353922152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4300c4a00 0x55f4300c9500 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc368031440 tx=0x7fc368005e60 comp rx=0 tx=0).stop 2024-06-10T01:10:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.919+0000 7fc3857c9740 1 -- 172.21.15.7:0/2353922152 shutdown_connections 2024-06-10T01:10:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.919+0000 7fc3857c9740 1 --2- 172.21.15.7:0/2353922152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4300c9b70 0x55f4300cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.919+0000 7fc3857c9740 1 --2- 172.21.15.7:0/2353922152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4300c4a00 0x55f4300c9500 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.919+0000 7fc3857c9740 1 --2- 172.21.15.7:0/2353922152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4300c4010 0x55f4300c4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.919+0000 7fc3857c9740 1 -- 172.21.15.7:0/2353922152 >> 172.21.15.7:0/2353922152 conn(0x55f4300bf680 msgr2=0x55f4300bfa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.919+0000 7fc3857c9740 1 -- 172.21.15.7:0/2353922152 shutdown_connections 2024-06-10T01:10:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.919+0000 7fc3857c9740 1 -- 172.21.15.7:0/2353922152 wait complete. 2024-06-10T01:10:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.920+0000 7fc3857c9740 1 -- start start 2024-06-10T01:10:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.920+0000 7fc3857c9740 1 Processor -- start 2024-06-10T01:10:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.920+0000 7fc3857c9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4300c4010 0x55f4300ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.920+0000 7fc3857c9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4300c4a00 0x55f4300ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.920+0000 7fc3857c9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4300c9b70 0x55f4300d5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.920+0000 7fc3857c9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f4300ccac0 con 0x55f4300c4010 2024-06-10T01:10:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.920+0000 7fc3857c9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f42ffb0ae0 con 0x55f4300c4a00 2024-06-10T01:10:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.920+0000 7fc3857c9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f4300ccc40 con 0x55f4300c9b70 2024-06-10T01:10:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.921+0000 7fc3857c9740 10 client.0 _ll_register_callbacks cb 0x55f42ffe99b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.921+0000 7fc380856640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4300c4010 0x55f4300ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.921+0000 7fc381057640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4300c9b70 0x55f4300d5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.921+0000 7fc380856640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4300c4010 0x55f4300ce280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60516/0 (socket says 172.21.15.7:60516) 2024-06-10T01:10:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.921+0000 7fc380856640 1 -- 172.21.15.7:0/1230025673 learned_addr learned my addr 172.21.15.7:0/1230025673 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.921+0000 7fc37bfff640 1 --2- 172.21.15.7:0/1230025673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4300c4a00 0x55f4300ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.921+0000 7fc381057640 1 -- 172.21.15.7:0/1230025673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4300c4a00 msgr2=0x55f4300ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.921+0000 7fc381057640 1 --2- 172.21.15.7:0/1230025673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4300c4a00 0x55f4300ce7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.921+0000 7fc381057640 1 -- 172.21.15.7:0/1230025673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4300c4010 msgr2=0x55f4300ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.921+0000 7fc381057640 1 --2- 172.21.15.7:0/1230025673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4300c4010 0x55f4300ce280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.921+0000 7fc381057640 1 -- 172.21.15.7:0/1230025673 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f42fd3b210 con 0x55f4300c9b70 2024-06-10T01:10:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.921+0000 7fc380856640 1 --2- 172.21.15.7:0/1230025673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4300c4010 0x55f4300ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.921+0000 7fc381057640 1 --2- 172.21.15.7:0/1230025673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4300c9b70 0x55f4300d5800 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc3740049d0 tx=0x7fc37409bba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f8d5a8f74360b2f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.921+0000 7fc35bfff640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:10:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.922+0000 7fc35bfff640 1 -- 172.21.15.7:0/1230025673 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3740b4020 con 0x55f4300c9b70 2024-06-10T01:10:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.922+0000 7fc35bfff640 1 -- 172.21.15.7:0/1230025673 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc3740badb0 con 0x55f4300c9b70 2024-06-10T01:10:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.922+0000 7fc3857c9740 1 -- 172.21.15.7:0/1230025673 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc368004030 con 0x55f4300c9b70 2024-06-10T01:10:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.922+0000 7fc35bfff640 1 -- 172.21.15.7:0/1230025673 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc37409bf70 con 0x55f4300c9b70 2024-06-10T01:10:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.922+0000 7fc3857c9740 10 client.26089 fetch_fsmap learned FSMap version 30 2024-06-10T01:10:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.922+0000 7fc3857c9740 1 -- 172.21.15.7:0/1230025673 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f42fe07620 con 0x55f4300c9b70 2024-06-10T01:10:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.922+0000 7fc35bfff640 1 -- 172.21.15.7:0/1230025673 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc368004030 con 0x55f4300c9b70 2024-06-10T01:10:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.923+0000 7fc35bfff640 1 -- 172.21.15.7:0/1230025673 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc37409b3c0 con 0x55f4300c9b70 2024-06-10T01:10:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.923+0000 7fc3857c9740 10 client.26089 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:10:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.923+0000 7fc3857c9740 10 client.26089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:10:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.923+0000 7fc3857c9740 20 client.26089 populate_metadata read hostname 'smithi007' 2024-06-10T01:10:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.923+0000 7fc3857c9740 1 --2- 172.21.15.7:0/1230025673 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f4300da150 0x55f4300dc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.923+0000 7fc3857c9740 4 client.26089 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:10:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.923+0000 7fc3857c9740 1 -- 172.21.15.7:0/1230025673 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f43003af10 con 0x55f4300da150 2024-06-10T01:10:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.923+0000 7fc380856640 1 --2- 172.21.15.7:0/1230025673 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f4300da150 0x55f4300dc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.924+0000 7fc35bfff640 10 client.26089 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.924+0000 7fc380856640 1 --2- 172.21.15.7:0/1230025673 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f4300da150 0x55f4300dc540 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb4b8a462b9b8db9 server_cookie=d0bd61a13fb743ca in_seq=0 out_seq=0 2024-06-10T01:10:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.924+0000 7fc35bfff640 1 -- 172.21.15.7:0/1230025673 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f43003af10 con 0x55f4300da150 2024-06-10T01:10:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.924+0000 7fc35bfff640 10 client.26089 handle_command_reply: tid=0 2024-06-10T01:10:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.924+0000 7fc3857c9740 1 client.26089 shutdown 2024-06-10T01:10:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.925+0000 7fc3857c9740 1 -- 172.21.15.7:0/1230025673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4300c9b70 msgr2=0x55f4300d5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.925+0000 7fc3857c9740 1 --2- 172.21.15.7:0/1230025673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4300c9b70 0x55f4300d5800 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc3740049d0 tx=0x7fc37409bba0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.925+0000 7fc3857c9740 1 -- 172.21.15.7:0/1230025673 shutdown_connections 2024-06-10T01:10:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.925+0000 7fc3857c9740 1 --2- 172.21.15.7:0/1230025673 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f4300da150 0x55f4300dc540 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.925+0000 7fc35bfff640 0 client.26089 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.925+0000 7fc3857c9740 1 --2- 172.21.15.7:0/1230025673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4300c9b70 0x55f4300d5800 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.925+0000 7fc3857c9740 1 --2- 172.21.15.7:0/1230025673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4300c4a00 0x55f4300ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.925+0000 7fc3857c9740 1 --2- 172.21.15.7:0/1230025673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4300c4010 0x55f4300ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.925+0000 7fc3857c9740 1 -- 172.21.15.7:0/1230025673 >> 172.21.15.7:0/1230025673 conn(0x55f4300bf680 msgr2=0x55f4300be7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.925+0000 7fc3857c9740 1 -- 172.21.15.7:0/1230025673 shutdown_connections 2024-06-10T01:10:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.925+0000 7fc3857c9740 1 -- 172.21.15.7:0/1230025673 wait complete. 2024-06-10T01:10:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.925+0000 7fc3857c9740 20 client.26089 trim_cache size 0 max 16384 2024-06-10T01:10:14.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.941+0000 7fc3857c9740 1 Processor -- start 2024-06-10T01:10:14.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.941+0000 7fc3857c9740 1 -- start start 2024-06-10T01:10:14.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.941+0000 7fc3857c9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4300c4010 0x55f4300c2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.941+0000 7fc3857c9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4300c4a00 0x55f4300d57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.941+0000 7fc3857c9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4300c9b70 0x55f4300c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.941+0000 7fc3857c9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c1aefe0 con 0x55f4300c9b70 2024-06-10T01:10:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.941+0000 7fc3857c9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c1ac850 con 0x55f4300c4a00 2024-06-10T01:10:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.941+0000 7fc3857c9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c1aa6a0 con 0x55f4300c4010 2024-06-10T01:10:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.941+0000 7fc381057640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4300c9b70 0x55f4300c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.941+0000 7fc381057640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4300c9b70 0x55f4300c2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60528/0 (socket says 172.21.15.7:60528) 2024-06-10T01:10:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.941+0000 7fc381057640 1 -- 172.21.15.7:0/1382268748 learned_addr learned my addr 172.21.15.7:0/1382268748 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.941+0000 7fc37bfff640 1 --2- 172.21.15.7:0/1382268748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4300c4a00 0x55f4300d57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.941+0000 7fc380856640 1 --2- 172.21.15.7:0/1382268748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4300c4010 0x55f4300c2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.942+0000 7fc381057640 1 -- 172.21.15.7:0/1382268748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4300c4010 msgr2=0x55f4300c2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.942+0000 7fc381057640 1 --2- 172.21.15.7:0/1382268748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4300c4010 0x55f4300c2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.942+0000 7fc381057640 1 -- 172.21.15.7:0/1382268748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4300c4a00 msgr2=0x55f4300d57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.942+0000 7fc381057640 1 --2- 172.21.15.7:0/1382268748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4300c4a00 0x55f4300d57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.942+0000 7fc381057640 1 -- 172.21.15.7:0/1382268748 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4300c0350 con 0x55f4300c9b70 2024-06-10T01:10:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.942+0000 7fc381057640 1 --2- 172.21.15.7:0/1382268748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4300c9b70 0x55f4300c2b90 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc3740a46b0 tx=0x7fc37400efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3e810d921b92f9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.942+0000 7fc3667fc640 1 -- 172.21.15.7:0/1382268748 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3740b4070 con 0x55f4300c9b70 2024-06-10T01:10:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.942+0000 7fc3667fc640 1 -- 172.21.15.7:0/1382268748 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3740a4be0 con 0x55f4300c9b70 2024-06-10T01:10:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.942+0000 7fc3667fc640 1 -- 172.21.15.7:0/1382268748 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3740b6020 con 0x55f4300c9b70 2024-06-10T01:10:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.943+0000 7fc3857c9740 1 -- 172.21.15.7:0/1382268748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4300c9b70 msgr2=0x55f4300c2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.943+0000 7fc3857c9740 1 --2- 172.21.15.7:0/1382268748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4300c9b70 0x55f4300c2b90 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc3740a46b0 tx=0x7fc37400efd0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.943+0000 7fc3857c9740 1 -- 172.21.15.7:0/1382268748 shutdown_connections 2024-06-10T01:10:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.943+0000 7fc3857c9740 1 --2- 172.21.15.7:0/1382268748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4300c9b70 0x55f4300c2b90 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.943+0000 7fc3857c9740 1 --2- 172.21.15.7:0/1382268748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4300c4a00 0x55f4300d57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.943+0000 7fc3857c9740 1 --2- 172.21.15.7:0/1382268748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4300c4010 0x55f4300c2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.943+0000 7fc3857c9740 1 -- 172.21.15.7:0/1382268748 >> 172.21.15.7:0/1382268748 conn(0x55f4300bf680 msgr2=0x55f4300ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.943+0000 7fc3857c9740 1 -- 172.21.15.7:0/1382268748 shutdown_connections 2024-06-10T01:10:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.943+0000 7fc3857c9740 1 -- 172.21.15.7:0/1382268748 wait complete. 2024-06-10T01:10:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.943+0000 7fc3857c9740 1 -- start start 2024-06-10T01:10:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.943+0000 7fc3857c9740 1 Processor -- start 2024-06-10T01:10:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.944+0000 7fc3857c9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4300c4010 0x55f4300d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.944+0000 7fc3857c9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4300c4a00 0x55f4300d2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.944+0000 7fc3857c9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4300c9b70 0x55f4300d3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.944+0000 7fc3857c9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c1aad10 con 0x55f4300c4010 2024-06-10T01:10:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.944+0000 7fc3857c9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c1ac180 con 0x55f4300c4a00 2024-06-10T01:10:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.944+0000 7fc3857c9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c1ab3e0 con 0x55f4300c9b70 2024-06-10T01:10:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.944+0000 7fc3857c9740 10 client.0 _ll_register_callbacks cb 0x7fc374004eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.944+0000 7fc380856640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4300c4010 0x55f4300d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.944+0000 7fc380856640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4300c4010 0x55f4300d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60534/0 (socket says 172.21.15.7:60534) 2024-06-10T01:10:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.944+0000 7fc380856640 1 -- 172.21.15.7:0/2392706271 learned_addr learned my addr 172.21.15.7:0/2392706271 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.944+0000 7fc37bfff640 1 --2- 172.21.15.7:0/2392706271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4300c4a00 0x55f4300d2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.944+0000 7fc381057640 1 --2- 172.21.15.7:0/2392706271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4300c9b70 0x55f4300d3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.944+0000 7fc380856640 1 -- 172.21.15.7:0/2392706271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4300c9b70 msgr2=0x55f4300d3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.944+0000 7fc380856640 1 --2- 172.21.15.7:0/2392706271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4300c9b70 0x55f4300d3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.944+0000 7fc380856640 1 -- 172.21.15.7:0/2392706271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4300c4a00 msgr2=0x55f4300d2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.944+0000 7fc380856640 1 --2- 172.21.15.7:0/2392706271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4300c4a00 0x55f4300d2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.944+0000 7fc380856640 1 -- 172.21.15.7:0/2392706271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4300d4480 con 0x55f4300c4010 2024-06-10T01:10:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.944+0000 7fc380856640 1 --2- 172.21.15.7:0/2392706271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4300c4010 0x55f4300d2940 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x55f4300dcdf0 tx=0x7fc370004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1275f2fbb1cda3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.944+0000 7fc35bfff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:10:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.944+0000 7fc35bfff640 1 -- 172.21.15.7:0/2392706271 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37001ecb0 con 0x55f4300c4010 2024-06-10T01:10:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.944+0000 7fc35bfff640 1 -- 172.21.15.7:0/2392706271 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc370030470 con 0x55f4300c4010 2024-06-10T01:10:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.944+0000 7fc3857c9740 1 -- 172.21.15.7:0/2392706271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3740a4be0 con 0x55f4300c4010 2024-06-10T01:10:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.945+0000 7fc35bfff640 1 -- 172.21.15.7:0/2392706271 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc370039050 con 0x55f4300c4010 2024-06-10T01:10:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.945+0000 7fc3857c9740 10 client.16560 fetch_fsmap learned FSMap version 30 2024-06-10T01:10:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.945+0000 7fc3857c9740 1 -- 172.21.15.7:0/2392706271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f4300d5860 con 0x55f4300c4010 2024-06-10T01:10:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.945+0000 7fc35bfff640 1 -- 172.21.15.7:0/2392706271 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3740a4be0 con 0x55f4300c4010 2024-06-10T01:10:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.945+0000 7fc35bfff640 1 -- 172.21.15.7:0/2392706271 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc37001d3b0 con 0x55f4300c4010 2024-06-10T01:10:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.945+0000 7fc3857c9740 10 client.16560 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:10:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.945+0000 7fc3857c9740 10 client.16560 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:10:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.945+0000 7fc3857c9740 20 client.16560 populate_metadata read hostname 'smithi007' 2024-06-10T01:10:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.945+0000 7fc3857c9740 1 --2- 172.21.15.7:0/2392706271 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f4300da150 0x55f4301473c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.945+0000 7fc3857c9740 4 client.16560 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:10:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.945+0000 7fc3857c9740 1 -- 172.21.15.7:0/2392706271 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f430034c20 con 0x55f4300da150 2024-06-10T01:10:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.945+0000 7fc37bfff640 1 --2- 172.21.15.7:0/2392706271 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f4300da150 0x55f4301473c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.946+0000 7fc37bfff640 1 --2- 172.21.15.7:0/2392706271 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f4300da150 0x55f4301473c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b23d282ed1406e9b server_cookie=49e2b5ffdeabee63 in_seq=0 out_seq=0 2024-06-10T01:10:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.946+0000 7fc35bfff640 10 client.16560 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.980+0000 7fc35bfff640 1 -- 172.21.15.7:0/2392706271 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f430034c20 con 0x55f4300da150 2024-06-10T01:10:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.980+0000 7fc35bfff640 10 client.16560 handle_command_reply: tid=0 2024-06-10T01:10:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.980+0000 7fc3857c9740 1 client.16560 shutdown 2024-06-10T01:10:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.980+0000 7fc3857c9740 1 -- 172.21.15.7:0/2392706271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4300c4010 msgr2=0x55f4300d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.980+0000 7fc3857c9740 1 --2- 172.21.15.7:0/2392706271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4300c4010 0x55f4300d2940 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x55f4300dcdf0 tx=0x7fc370004730 comp rx=0 tx=0).stop 2024-06-10T01:10:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.980+0000 7fc3857c9740 1 -- 172.21.15.7:0/2392706271 shutdown_connections 2024-06-10T01:10:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.980+0000 7fc3857c9740 1 --2- 172.21.15.7:0/2392706271 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f4300da150 0x55f4301473c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.980+0000 7fc3857c9740 1 --2- 172.21.15.7:0/2392706271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4300c9b70 0x55f4300d3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.980+0000 7fc35bfff640 0 client.16560 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.980+0000 7fc3857c9740 1 --2- 172.21.15.7:0/2392706271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4300c4a00 0x55f4300d2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.980+0000 7fc3857c9740 1 --2- 172.21.15.7:0/2392706271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4300c4010 0x55f4300d2940 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.980+0000 7fc3857c9740 1 -- 172.21.15.7:0/2392706271 >> 172.21.15.7:0/2392706271 conn(0x55f4300bf680 msgr2=0x55f4300ce260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.980+0000 7fc3857c9740 1 -- 172.21.15.7:0/2392706271 shutdown_connections 2024-06-10T01:10:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.980+0000 7fc3857c9740 1 -- 172.21.15.7:0/2392706271 wait complete. 2024-06-10T01:10:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.980+0000 7fc3857c9740 20 client.16560 trim_cache size 0 max 16384 2024-06-10T01:10:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:10:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:10:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:10:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.981+0000 7fc382aba640 1 -- 172.21.15.7:0/892150697 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc36c0a2c00 msgr2=0x7fc36c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.981+0000 7fc382aba640 1 --2- 172.21.15.7:0/892150697 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc36c0a2c00 0x7fc36c0a50c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc370005fd0 tx=0x7fc37000b390 comp rx=0 tx=0).stop 2024-06-10T01:10:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.981+0000 7fc382aba640 1 -- 172.21.15.7:0/892150697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc37c1077a0 msgr2=0x7fc37c100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.981+0000 7fc382aba640 1 --2- 172.21.15.7:0/892150697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc37c1077a0 0x7fc37c100a30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc37400eb10 tx=0x7fc37400c2d0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.981+0000 7fc382aba640 1 -- 172.21.15.7:0/892150697 shutdown_connections 2024-06-10T01:10:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.981+0000 7fc382aba640 1 --2- 172.21.15.7:0/892150697 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc36c0a2c00 0x7fc36c0a50c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.981+0000 7fc382aba640 1 --2- 172.21.15.7:0/892150697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc37c1077a0 0x7fc37c100a30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.981+0000 7fc382aba640 1 --2- 172.21.15.7:0/892150697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc37c104e70 0x7fc37c1004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.981+0000 7fc382aba640 1 --2- 172.21.15.7:0/892150697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc37c102540 0x7fc37c101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.981+0000 7fc382aba640 1 -- 172.21.15.7:0/892150697 >> 172.21.15.7:0/892150697 conn(0x7fc37c0fbed0 msgr2=0x7fc37c102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.981+0000 7fc382aba640 1 -- 172.21.15.7:0/892150697 shutdown_connections 2024-06-10T01:10:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:14.981+0000 7fc382aba640 1 -- 172.21.15.7:0/892150697 wait complete. 2024-06-10T01:10:15.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.120+0000 7f7d6a286700 1 -- 172.21.15.7:0/957902762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d64107400 msgr2=0x7f7d641077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:15.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.120+0000 7f7d6a286700 1 --2- 172.21.15.7:0/957902762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d64107400 0x7f7d641077e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009990 tx=0x7f7d54009ca0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.120+0000 7f7d6a286700 1 -- 172.21.15.7:0/957902762 shutdown_connections 2024-06-10T01:10:15.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.120+0000 7f7d6a286700 1 --2- 172.21.15.7:0/957902762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d64101d50 0x7f7d6410e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.120+0000 7f7d6a286700 1 --2- 172.21.15.7:0/957902762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d64107400 0x7f7d641077e0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.120+0000 7f7d6a286700 1 --2- 172.21.15.7:0/957902762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d64101390 0x7f7d64101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.120+0000 7f7d6a286700 1 -- 172.21.15.7:0/957902762 >> 172.21.15.7:0/957902762 conn(0x7f7d64078340 msgr2=0x7f7d64078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:15.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.120+0000 7f7d6a286700 1 -- 172.21.15.7:0/957902762 shutdown_connections 2024-06-10T01:10:15.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.120+0000 7f7d6a286700 1 -- 172.21.15.7:0/957902762 wait complete. 2024-06-10T01:10:15.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.121+0000 7f7d6a286700 1 Processor -- start 2024-06-10T01:10:15.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.121+0000 7f7d6a286700 1 -- start start 2024-06-10T01:10:15.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.121+0000 7f7d6a286700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d64101390 0x7f7d641978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:15.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.121+0000 7f7d6a286700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d64101d50 0x7f7d64197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:15.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.121+0000 7f7d6a286700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d64107400 0x7f7d64198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:15.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.121+0000 7f7d6a286700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7d641116a0 con 0x7f7d64101d50 2024-06-10T01:10:15.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.121+0000 7f7d6a286700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7d64111520 con 0x7f7d64101390 2024-06-10T01:10:15.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.121+0000 7f7d6a286700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7d64111820 con 0x7f7d64107400 2024-06-10T01:10:15.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.121+0000 7f7d637fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d64101d50 0x7f7d64197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:15.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.121+0000 7f7d637fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d64101d50 0x7f7d64197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60550/0 (socket says 172.21.15.7:60550) 2024-06-10T01:10:15.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.121+0000 7f7d637fe700 1 -- 172.21.15.7:0/3915474680 learned_addr learned my addr 172.21.15.7:0/3915474680 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:15.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.122+0000 7f7d63fff700 1 --2- 172.21.15.7:0/3915474680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d64101390 0x7f7d641978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:15.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.122+0000 7f7d68823700 1 --2- 172.21.15.7:0/3915474680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d64107400 0x7f7d64198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:15.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.122+0000 7f7d637fe700 1 -- 172.21.15.7:0/3915474680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d64107400 msgr2=0x7f7d64198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:15.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.122+0000 7f7d637fe700 1 --2- 172.21.15.7:0/3915474680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d64107400 0x7f7d64198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.122+0000 7f7d637fe700 1 -- 172.21.15.7:0/3915474680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d64101390 msgr2=0x7f7d641978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:15.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.122+0000 7f7d637fe700 1 --2- 172.21.15.7:0/3915474680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d64101390 0x7f7d641978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.122+0000 7f7d637fe700 1 -- 172.21.15.7:0/3915474680 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d641a7100 con 0x7f7d64101d50 2024-06-10T01:10:15.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.122+0000 7f7d63fff700 1 --2- 172.21.15.7:0/3915474680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d64101390 0x7f7d641978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:10:15.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.122+0000 7f7d637fe700 1 --2- 172.21.15.7:0/3915474680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d64101d50 0x7f7d64197e30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800c970 tx=0x7f7d5800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:15.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.122+0000 7f7d617fa700 1 -- 172.21.15.7:0/3915474680 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d58007aa0 con 0x7f7d64101d50 2024-06-10T01:10:15.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.122+0000 7f7d617fa700 1 -- 172.21.15.7:0/3915474680 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d58004d10 con 0x7f7d64101d50 2024-06-10T01:10:15.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.122+0000 7f7d6a286700 1 -- 172.21.15.7:0/3915474680 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d641a73f0 con 0x7f7d64101d50 2024-06-10T01:10:15.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.122+0000 7f7d617fa700 1 -- 172.21.15.7:0/3915474680 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5800f450 con 0x7f7d64101d50 2024-06-10T01:10:15.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.122+0000 7f7d6a286700 1 -- 172.21.15.7:0/3915474680 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d641a7930 con 0x7f7d64101d50 2024-06-10T01:10:15.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.124+0000 7f7d617fa700 1 -- 172.21.15.7:0/3915474680 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f7d58004750 con 0x7f7d64101d50 2024-06-10T01:10:15.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.125+0000 7f7d617fa700 1 --2- 172.21.15.7:0/3915474680 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7d4c0a2ca0 0x7f7d4c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:15.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.125+0000 7f7d617fa700 1 -- 172.21.15.7:0/3915474680 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f7d580cf230 con 0x7f7d64101d50 2024-06-10T01:10:15.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.125+0000 7f7d63fff700 1 --2- 172.21.15.7:0/3915474680 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7d4c0a2ca0 0x7f7d4c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:15.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.126+0000 7f7d63fff700 1 --2- 172.21.15.7:0/3915474680 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7d4c0a2ca0 0x7f7d4c0a5160 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7d54006010 tx=0x7f7d5400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:15.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.126+0000 7f7d74b09b80 1 Processor -- start 2024-06-10T01:10:15.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.126+0000 7f7d74b09b80 1 -- start start 2024-06-10T01:10:15.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.126+0000 7f7d74b09b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39f6af640 0x55b39f6afa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:15.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.126+0000 7f7d74b09b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b39f6aff60 0x55b39f6a99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:15.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.126+0000 7f7d74b09b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b39f6a9f20 0x55b39f6aa340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:15.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.126+0000 7f7d74b09b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b39f5fb810 con 0x55b39f6aff60 2024-06-10T01:10:15.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.126+0000 7f7d74b09b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b39f5a3720 con 0x55b39f6af640 2024-06-10T01:10:15.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.126+0000 7f7d74b09b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b39f5f77d0 con 0x55b39f6a9f20 2024-06-10T01:10:15.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.126+0000 7f7d637fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b39f6aff60 0x55b39f6a99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:15.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.126+0000 7f7d637fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b39f6aff60 0x55b39f6a99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60552/0 (socket says 172.21.15.7:60552) 2024-06-10T01:10:15.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.126+0000 7f7d637fe700 1 -- 172.21.15.7:0/989021454 learned_addr learned my addr 172.21.15.7:0/989021454 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:15.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.126+0000 7f7d637fe700 1 -- 172.21.15.7:0/989021454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b39f6a9f20 msgr2=0x55b39f6aa340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:15.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.126+0000 7f7d68823700 1 --2- 172.21.15.7:0/989021454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b39f6a9f20 0x55b39f6aa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:15.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.126+0000 7f7d63fff700 1 --2- 172.21.15.7:0/989021454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39f6af640 0x55b39f6afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:15.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.126+0000 7f7d637fe700 1 --2- 172.21.15.7:0/989021454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b39f6a9f20 0x55b39f6aa340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.126+0000 7f7d637fe700 1 -- 172.21.15.7:0/989021454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39f6af640 msgr2=0x55b39f6afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:15.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.126+0000 7f7d637fe700 1 --2- 172.21.15.7:0/989021454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39f6af640 0x55b39f6afa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.127+0000 7f7d637fe700 1 -- 172.21.15.7:0/989021454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b39f599f50 con 0x55b39f6aff60 2024-06-10T01:10:15.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.127+0000 7f7d63fff700 1 --2- 172.21.15.7:0/989021454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39f6af640 0x55b39f6afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:10:15.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.127+0000 7f7d637fe700 1 --2- 172.21.15.7:0/989021454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b39f6aff60 0x55b39f6a99e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f7d58097ba0 tx=0x7f7d5809b350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b59cda7fc3420ec server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:15.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.127+0000 7f7d4a7fc700 1 -- 172.21.15.7:0/989021454 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d580b4070 con 0x55b39f6aff60 2024-06-10T01:10:15.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.127+0000 7f7d4a7fc700 1 -- 172.21.15.7:0/989021454 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d5809be40 con 0x55b39f6aff60 2024-06-10T01:10:15.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.127+0000 7f7d4a7fc700 1 -- 172.21.15.7:0/989021454 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d58099a40 con 0x55b39f6aff60 2024-06-10T01:10:15.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.127+0000 7f7d74b09b80 1 -- 172.21.15.7:0/989021454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b39f6aff60 msgr2=0x55b39f6a99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:15.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.127+0000 7f7d74b09b80 1 --2- 172.21.15.7:0/989021454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b39f6aff60 0x55b39f6a99e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f7d58097ba0 tx=0x7f7d5809b350 comp rx=0 tx=0).stop 2024-06-10T01:10:15.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.128+0000 7f7d74b09b80 1 -- 172.21.15.7:0/989021454 shutdown_connections 2024-06-10T01:10:15.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.128+0000 7f7d74b09b80 1 --2- 172.21.15.7:0/989021454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b39f6a9f20 0x55b39f6aa340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.128+0000 7f7d74b09b80 1 --2- 172.21.15.7:0/989021454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39f6af640 0x55b39f6afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.128+0000 7f7d74b09b80 1 --2- 172.21.15.7:0/989021454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b39f6aff60 0x55b39f6a99e0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.128+0000 7f7d74b09b80 1 -- 172.21.15.7:0/989021454 >> 172.21.15.7:0/989021454 conn(0x55b39f5d4cc0 msgr2=0x55b39f6a4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:15.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.128+0000 7f7d74b09b80 1 -- 172.21.15.7:0/989021454 shutdown_connections 2024-06-10T01:10:15.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.128+0000 7f7d74b09b80 1 -- 172.21.15.7:0/989021454 wait complete. 2024-06-10T01:10:15.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.128+0000 7f7d74b09b80 1 -- start start 2024-06-10T01:10:15.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.128+0000 7f7d74b09b80 1 Processor -- start 2024-06-10T01:10:15.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.129+0000 7f7d74b09b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b39f6a9f20 0x55b39f6b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:15.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.129+0000 7f7d74b09b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b39f6af640 0x55b39f6b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:15.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.129+0000 7f7d74b09b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39f6aff60 0x55b39f6b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:15.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.129+0000 7f7d74b09b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b39f6b69d0 con 0x55b39f6af640 2024-06-10T01:10:15.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.129+0000 7f7d74b09b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b39f59bbf0 con 0x55b39f6aff60 2024-06-10T01:10:15.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.129+0000 7f7d74b09b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b39f6b6b50 con 0x55b39f6a9f20 2024-06-10T01:10:15.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.129+0000 7f7d74b09b80 10 client.0 _ll_register_callbacks cb 0x55b39f595920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:15.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.129+0000 7f7d637fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b39f6af640 0x55b39f6b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:15.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.129+0000 7f7d637fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b39f6af640 0x55b39f6b30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60562/0 (socket says 172.21.15.7:60562) 2024-06-10T01:10:15.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.129+0000 7f7d637fe700 1 -- 172.21.15.7:0/3547352888 learned_addr learned my addr 172.21.15.7:0/3547352888 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:15.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.129+0000 7f7d68823700 1 --2- 172.21.15.7:0/3547352888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39f6aff60 0x55b39f6b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:15.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.129+0000 7f7d63fff700 1 --2- 172.21.15.7:0/3547352888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b39f6a9f20 0x55b39f6b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:15.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.129+0000 7f7d637fe700 1 -- 172.21.15.7:0/3547352888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b39f6a9f20 msgr2=0x55b39f6b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:15.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.129+0000 7f7d637fe700 1 --2- 172.21.15.7:0/3547352888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b39f6a9f20 0x55b39f6b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.129+0000 7f7d637fe700 1 -- 172.21.15.7:0/3547352888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39f6aff60 msgr2=0x55b39f6b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:15.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.129+0000 7f7d637fe700 1 --2- 172.21.15.7:0/3547352888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39f6aff60 0x55b39f6b35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.129+0000 7f7d637fe700 1 -- 172.21.15.7:0/3547352888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b39f54a8d0 con 0x55b39f6af640 2024-06-10T01:10:15.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.129+0000 7f7d637fe700 1 --2- 172.21.15.7:0/3547352888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b39f6af640 0x55b39f6b30a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f7d580074a0 tx=0x7f7d5809bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66cc42e0bd45e815 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:15.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.129+0000 7f7d437fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:10:15.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.129+0000 7f7d437fe700 1 -- 172.21.15.7:0/3547352888 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d580b9070 con 0x55b39f6af640 2024-06-10T01:10:15.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.129+0000 7f7d437fe700 1 -- 172.21.15.7:0/3547352888 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f7d580860b0 con 0x55b39f6af640 2024-06-10T01:10:15.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.129+0000 7f7d74b09b80 1 -- 172.21.15.7:0/3547352888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d5809be40 con 0x55b39f6af640 2024-06-10T01:10:15.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.130+0000 7f7d437fe700 1 -- 172.21.15.7:0/3547352888 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7d580b4070 con 0x55b39f6af640 2024-06-10T01:10:15.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.130+0000 7f7d74b09b80 10 client.16572 fetch_fsmap learned FSMap version 30 2024-06-10T01:10:15.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.130+0000 7f7d74b09b80 1 -- 172.21.15.7:0/3547352888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b39f6c1350 con 0x55b39f6af640 2024-06-10T01:10:15.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.130+0000 7f7d437fe700 1 -- 172.21.15.7:0/3547352888 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d5809be40 con 0x55b39f6af640 2024-06-10T01:10:15.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.130+0000 7f7d437fe700 1 -- 172.21.15.7:0/3547352888 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7d580bb040 con 0x55b39f6af640 2024-06-10T01:10:15.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.130+0000 7f7d74b09b80 10 client.16572 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:10:15.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.130+0000 7f7d74b09b80 10 client.16572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:10:15.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.130+0000 7f7d74b09b80 20 client.16572 populate_metadata read hostname 'smithi007' 2024-06-10T01:10:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.130+0000 7f7d74b09b80 1 --2- 172.21.15.7:0/3547352888 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b39f6c3820 0x55b39f6c5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.130+0000 7f7d74b09b80 4 client.16572 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:10:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.130+0000 7f7d74b09b80 1 -- 172.21.15.7:0/3547352888 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b39f608d10 con 0x55b39f6c3820 2024-06-10T01:10:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.130+0000 7f7d63fff700 1 --2- 172.21.15.7:0/3547352888 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b39f6c3820 0x55b39f6c5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.131+0000 7f7d63fff700 1 --2- 172.21.15.7:0/3547352888 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b39f6c3820 0x55b39f6c5c10 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7203df41705ecaa1 server_cookie=1afafe6b3cbeeae7 in_seq=0 out_seq=0 2024-06-10T01:10:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.131+0000 7f7d437fe700 10 client.16572 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.132+0000 7f7d437fe700 1 -- 172.21.15.7:0/3547352888 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b39f608d10 con 0x55b39f6c3820 2024-06-10T01:10:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.132+0000 7f7d437fe700 10 client.16572 handle_command_reply: tid=0 2024-06-10T01:10:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.132+0000 7f7d74b09b80 1 client.16572 shutdown 2024-06-10T01:10:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.132+0000 7f7d74b09b80 1 -- 172.21.15.7:0/3547352888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b39f6af640 msgr2=0x55b39f6b30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.132+0000 7f7d74b09b80 1 --2- 172.21.15.7:0/3547352888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b39f6af640 0x55b39f6b30a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f7d580074a0 tx=0x7f7d5809bcf0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.132+0000 7f7d74b09b80 1 -- 172.21.15.7:0/3547352888 shutdown_connections 2024-06-10T01:10:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.132+0000 7f7d74b09b80 1 --2- 172.21.15.7:0/3547352888 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b39f6c3820 0x55b39f6c5c10 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.132+0000 7f7d74b09b80 1 --2- 172.21.15.7:0/3547352888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39f6aff60 0x55b39f6b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.132+0000 7f7d74b09b80 1 --2- 172.21.15.7:0/3547352888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b39f6a9f20 0x55b39f6b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.132+0000 7f7d74b09b80 1 --2- 172.21.15.7:0/3547352888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b39f6af640 0x55b39f6b30a0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.132+0000 7f7d437fe700 0 client.16572 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.132+0000 7f7d74b09b80 1 -- 172.21.15.7:0/3547352888 >> 172.21.15.7:0/3547352888 conn(0x55b39f5d4cc0 msgr2=0x55b39f6a89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.132+0000 7f7d74b09b80 1 -- 172.21.15.7:0/3547352888 shutdown_connections 2024-06-10T01:10:15.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.132+0000 7f7d74b09b80 1 -- 172.21.15.7:0/3547352888 wait complete. 2024-06-10T01:10:15.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.132+0000 7f7d74b09b80 20 client.16572 trim_cache size 0 max 16384 2024-06-10T01:10:15.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.154+0000 7f7d74b09b80 1 Processor -- start 2024-06-10T01:10:15.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.154+0000 7f7d74b09b80 1 -- start start 2024-06-10T01:10:15.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.154+0000 7f7d74b09b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b39f6a9f20 0x55b39f6a99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:15.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.154+0000 7f7d74b09b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39f6af640 0x55b39f69e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.154+0000 7f7d74b09b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b39f6aff60 0x55b39f6c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.154+0000 7f7d74b09b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7d641b2dc0 con 0x55b39f6a9f20 2024-06-10T01:10:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.154+0000 7f7d74b09b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7d641b05f0 con 0x55b39f6af640 2024-06-10T01:10:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.154+0000 7f7d74b09b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7d641ae440 con 0x55b39f6aff60 2024-06-10T01:10:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.154+0000 7f7d63fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b39f6a9f20 0x55b39f6a99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.154+0000 7f7d63fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b39f6a9f20 0x55b39f6a99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60568/0 (socket says 172.21.15.7:60568) 2024-06-10T01:10:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.154+0000 7f7d63fff700 1 -- 172.21.15.7:0/3457246069 learned_addr learned my addr 172.21.15.7:0/3457246069 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.155+0000 7f7d68823700 1 --2- 172.21.15.7:0/3457246069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b39f6aff60 0x55b39f6c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.155+0000 7f7d637fe700 1 --2- 172.21.15.7:0/3457246069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39f6af640 0x55b39f69e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.155+0000 7f7d63fff700 1 -- 172.21.15.7:0/3457246069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b39f6aff60 msgr2=0x55b39f6c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.155+0000 7f7d63fff700 1 --2- 172.21.15.7:0/3457246069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b39f6aff60 0x55b39f6c8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.155+0000 7f7d63fff700 1 -- 172.21.15.7:0/3457246069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39f6af640 msgr2=0x55b39f69e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.155+0000 7f7d63fff700 1 --2- 172.21.15.7:0/3457246069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39f6af640 0x55b39f69e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.155+0000 7f7d63fff700 1 -- 172.21.15.7:0/3457246069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b39f6b6a50 con 0x55b39f6a9f20 2024-06-10T01:10:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.155+0000 7f7d637fe700 1 --2- 172.21.15.7:0/3457246069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39f6af640 0x55b39f69e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:10:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.155+0000 7f7d68823700 1 --2- 172.21.15.7:0/3457246069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b39f6aff60 0x55b39f6c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:10:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.155+0000 7f7d63fff700 1 --2- 172.21.15.7:0/3457246069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b39f6a9f20 0x55b39f6a99e0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7d54000c00 tx=0x7f7d5401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5feb874b40522750 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.155+0000 7f7d49ffb700 1 -- 172.21.15.7:0/3457246069 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d54003960 con 0x55b39f6a9f20 2024-06-10T01:10:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.155+0000 7f7d49ffb700 1 -- 172.21.15.7:0/3457246069 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d540050b0 con 0x55b39f6a9f20 2024-06-10T01:10:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.155+0000 7f7d49ffb700 1 -- 172.21.15.7:0/3457246069 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d540326f0 con 0x55b39f6a9f20 2024-06-10T01:10:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.156+0000 7f7d74b09b80 1 -- 172.21.15.7:0/3457246069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b39f6a9f20 msgr2=0x55b39f6a99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.156+0000 7f7d74b09b80 1 --2- 172.21.15.7:0/3457246069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b39f6a9f20 0x55b39f6a99e0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7d54000c00 tx=0x7f7d5401f000 comp rx=0 tx=0).stop 2024-06-10T01:10:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.156+0000 7f7d74b09b80 1 -- 172.21.15.7:0/3457246069 shutdown_connections 2024-06-10T01:10:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.156+0000 7f7d74b09b80 1 --2- 172.21.15.7:0/3457246069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b39f6aff60 0x55b39f6c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.156+0000 7f7d74b09b80 1 --2- 172.21.15.7:0/3457246069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b39f6a9f20 0x55b39f6a99e0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.156+0000 7f7d74b09b80 1 --2- 172.21.15.7:0/3457246069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39f6af640 0x55b39f69e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.156+0000 7f7d74b09b80 1 -- 172.21.15.7:0/3457246069 >> 172.21.15.7:0/3457246069 conn(0x55b39f5d4cc0 msgr2=0x55b39f6b0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:15.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.156+0000 7f7d74b09b80 1 -- 172.21.15.7:0/3457246069 shutdown_connections 2024-06-10T01:10:15.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.156+0000 7f7d74b09b80 1 -- 172.21.15.7:0/3457246069 wait complete. 2024-06-10T01:10:15.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.156+0000 7f7d74b09b80 1 -- start start 2024-06-10T01:10:15.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.157+0000 7f7d74b09b80 1 Processor -- start 2024-06-10T01:10:15.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.157+0000 7f7d74b09b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b39f6a9f20 0x55b39f6c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:15.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.157+0000 7f7d74b09b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b39f6af640 0x55b39f6c6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:15.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.157+0000 7f7d74b09b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39f6aff60 0x55b39f6c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:15.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.157+0000 7f7d74b09b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7d641aeab0 con 0x55b39f6af640 2024-06-10T01:10:15.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.157+0000 7f7d74b09b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7d641aff20 con 0x55b39f6aff60 2024-06-10T01:10:15.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.157+0000 7f7d74b09b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7d641af180 con 0x55b39f6a9f20 2024-06-10T01:10:15.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.157+0000 7f7d74b09b80 10 client.0 _ll_register_callbacks cb 0x55b39f595920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:15.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.157+0000 7f7d637fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b39f6af640 0x55b39f6c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:15.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.157+0000 7f7d637fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b39f6af640 0x55b39f6c6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60584/0 (socket says 172.21.15.7:60584) 2024-06-10T01:10:15.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.157+0000 7f7d637fe700 1 -- 172.21.15.7:0/2830821231 learned_addr learned my addr 172.21.15.7:0/2830821231 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:15.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.157+0000 7f7d63fff700 1 --2- 172.21.15.7:0/2830821231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b39f6a9f20 0x55b39f6c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:15.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.157+0000 7f7d68823700 1 --2- 172.21.15.7:0/2830821231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39f6aff60 0x55b39f6c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:15.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.157+0000 7f7d637fe700 1 -- 172.21.15.7:0/2830821231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b39f6a9f20 msgr2=0x55b39f6c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:15.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.157+0000 7f7d637fe700 1 --2- 172.21.15.7:0/2830821231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b39f6a9f20 0x55b39f6c90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.157+0000 7f7d637fe700 1 -- 172.21.15.7:0/2830821231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39f6aff60 msgr2=0x55b39f6c69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:15.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.157+0000 7f7d637fe700 1 --2- 172.21.15.7:0/2830821231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39f6aff60 0x55b39f6c69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.157+0000 7f7d637fe700 1 -- 172.21.15.7:0/2830821231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b39f6c6f30 con 0x55b39f6af640 2024-06-10T01:10:15.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.157+0000 7f7d68823700 1 --2- 172.21.15.7:0/2830821231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39f6aff60 0x55b39f6c69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:10:15.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.158+0000 7f7d637fe700 1 --2- 172.21.15.7:0/2830821231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b39f6af640 0x55b39f6c6480 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f7d5809b440 tx=0x7f7d580cc9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9dc869ab2a357b5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:15.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.158+0000 7f7d437fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:10:15.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.158+0000 7f7d437fe700 1 -- 172.21.15.7:0/2830821231 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d580b9070 con 0x55b39f6af640 2024-06-10T01:10:15.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.158+0000 7f7d437fe700 1 -- 172.21.15.7:0/2830821231 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f7d580afbd0 con 0x55b39f6af640 2024-06-10T01:10:15.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.158+0000 7f7d74b09b80 1 -- 172.21.15.7:0/2830821231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d540050b0 con 0x55b39f6af640 2024-06-10T01:10:15.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.158+0000 7f7d437fe700 1 -- 172.21.15.7:0/2830821231 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7d5809b280 con 0x55b39f6af640 2024-06-10T01:10:15.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.158+0000 7f7d74b09b80 10 client.16578 fetch_fsmap learned FSMap version 30 2024-06-10T01:10:15.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.158+0000 7f7d74b09b80 1 -- 172.21.15.7:0/2830821231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b39f72de50 con 0x55b39f6af640 2024-06-10T01:10:15.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.158+0000 7f7d437fe700 1 -- 172.21.15.7:0/2830821231 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d540050b0 con 0x55b39f6af640 2024-06-10T01:10:15.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.158+0000 7f7d437fe700 1 -- 172.21.15.7:0/2830821231 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7d580bb040 con 0x55b39f6af640 2024-06-10T01:10:15.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.158+0000 7f7d74b09b80 10 client.16578 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:10:15.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.158+0000 7f7d74b09b80 10 client.16578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:10:15.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.158+0000 7f7d74b09b80 20 client.16578 populate_metadata read hostname 'smithi007' 2024-06-10T01:10:15.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.158+0000 7f7d74b09b80 1 --2- 172.21.15.7:0/2830821231 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b39f72e300 0x55b39f7306f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:15.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.158+0000 7f7d74b09b80 4 client.16578 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:10:15.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.158+0000 7f7d74b09b80 1 -- 172.21.15.7:0/2830821231 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b39f603290 con 0x55b39f72e300 2024-06-10T01:10:15.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.159+0000 7f7d63fff700 1 --2- 172.21.15.7:0/2830821231 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b39f72e300 0x55b39f7306f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:15.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.159+0000 7f7d63fff700 1 --2- 172.21.15.7:0/2830821231 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b39f72e300 0x55b39f7306f0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de6e8c67ddd142b6 server_cookie=506038c278dd48c4 in_seq=0 out_seq=0 2024-06-10T01:10:15.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.159+0000 7f7d437fe700 10 client.16578 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:15.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.159+0000 7f7d437fe700 1 -- 172.21.15.7:0/2830821231 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b39f603290 con 0x55b39f72e300 2024-06-10T01:10:15.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.159+0000 7f7d437fe700 10 client.16578 handle_command_reply: tid=0 2024-06-10T01:10:15.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.159+0000 7f7d74b09b80 1 client.16578 shutdown 2024-06-10T01:10:15.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.160+0000 7f7d74b09b80 1 -- 172.21.15.7:0/2830821231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b39f6af640 msgr2=0x55b39f6c6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:15.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.160+0000 7f7d74b09b80 1 --2- 172.21.15.7:0/2830821231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b39f6af640 0x55b39f6c6480 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f7d5809b440 tx=0x7f7d580cc9d0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.160+0000 7f7d74b09b80 1 -- 172.21.15.7:0/2830821231 shutdown_connections 2024-06-10T01:10:15.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.160+0000 7f7d74b09b80 1 --2- 172.21.15.7:0/2830821231 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b39f72e300 0x55b39f7306f0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.160+0000 7f7d74b09b80 1 --2- 172.21.15.7:0/2830821231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39f6aff60 0x55b39f6c69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.160+0000 7f7d74b09b80 1 --2- 172.21.15.7:0/2830821231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b39f6a9f20 0x55b39f6c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.160+0000 7f7d74b09b80 1 --2- 172.21.15.7:0/2830821231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b39f6af640 0x55b39f6c6480 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.160+0000 7f7d74b09b80 1 -- 172.21.15.7:0/2830821231 >> 172.21.15.7:0/2830821231 conn(0x55b39f5d4cc0 msgr2=0x55b39f6b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:15.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.160+0000 7f7d437fe700 0 client.16578 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:15.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.160+0000 7f7d74b09b80 1 -- 172.21.15.7:0/2830821231 shutdown_connections 2024-06-10T01:10:15.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.160+0000 7f7d74b09b80 1 -- 172.21.15.7:0/2830821231 wait complete. 2024-06-10T01:10:15.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.160+0000 7f7d74b09b80 20 client.16578 trim_cache size 0 max 16384 2024-06-10T01:10:15.163 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:10:15.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.160+0000 7f7d6a286700 1 -- 172.21.15.7:0/3915474680 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7d4c0a2ca0 msgr2=0x7f7d4c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:15.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.160+0000 7f7d6a286700 1 --2- 172.21.15.7:0/3915474680 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7d4c0a2ca0 0x7f7d4c0a5160 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7d54006010 tx=0x7f7d5400b540 comp rx=0 tx=0).stop 2024-06-10T01:10:15.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.160+0000 7f7d6a286700 1 -- 172.21.15.7:0/3915474680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d64101d50 msgr2=0x7f7d64197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:15.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.160+0000 7f7d6a286700 1 --2- 172.21.15.7:0/3915474680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d64101d50 0x7f7d64197e30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800c970 tx=0x7f7d5800cd30 comp rx=0 tx=0).stop 2024-06-10T01:10:15.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.161+0000 7f7d6a286700 1 -- 172.21.15.7:0/3915474680 shutdown_connections 2024-06-10T01:10:15.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.161+0000 7f7d6a286700 1 --2- 172.21.15.7:0/3915474680 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7d4c0a2ca0 0x7f7d4c0a5160 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.161+0000 7f7d6a286700 1 --2- 172.21.15.7:0/3915474680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d64107400 0x7f7d64198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.161+0000 7f7d6a286700 1 --2- 172.21.15.7:0/3915474680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d64101390 0x7f7d641978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.161+0000 7f7d6a286700 1 --2- 172.21.15.7:0/3915474680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d64101d50 0x7f7d64197e30 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:15.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.161+0000 7f7d6a286700 1 -- 172.21.15.7:0/3915474680 >> 172.21.15.7:0/3915474680 conn(0x7f7d64078340 msgr2=0x7f7d64100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:15.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.161+0000 7f7d6a286700 1 -- 172.21.15.7:0/3915474680 shutdown_connections 2024-06-10T01:10:15.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:15.161+0000 7f7d6a286700 1 -- 172.21.15.7:0/3915474680 wait complete. 2024-06-10T01:10:15.166 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:10:15.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:15 smithi007 ceph-mon[32398]: pgmap v559: 129 pgs: 129 active+clean; 61 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 2.2 MiB/s wr, 188 op/s 2024-06-10T01:10:15.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:15 smithi078 ceph-mon[36683]: pgmap v559: 129 pgs: 129 active+clean; 61 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 2.2 MiB/s wr, 188 op/s 2024-06-10T01:10:15.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:15 smithi175 ceph-mon[36636]: pgmap v559: 129 pgs: 129 active+clean; 61 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 2.2 MiB/s wr, 188 op/s 2024-06-10T01:10:15.715 INFO:tasks.workunit.client.0.smithi007.stdout: 1 259415 8.20 MB/sec execute 56 sec latency 14055.569 ms 2024-06-10T01:10:16.716 INFO:tasks.workunit.client.0.smithi007.stdout: 1 259415 8.05 MB/sec execute 57 sec latency 15055.684 ms 2024-06-10T01:10:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.570+0000 7fd2eaf58700 1 -- 172.21.15.7:0/3815120707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2e4102ef0 msgr2=0x7fd2e410f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.570+0000 7fd2eaf58700 1 --2- 172.21.15.7:0/3815120707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2e4102ef0 0x7fd2e410f3a0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc0099c0 tx=0x7fd2dc009cd0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.570+0000 7fd2eaf58700 1 -- 172.21.15.7:0/3815120707 shutdown_connections 2024-06-10T01:10:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.570+0000 7fd2eaf58700 1 --2- 172.21.15.7:0/3815120707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2e4102ef0 0x7fd2e410f3a0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.570+0000 7fd2eaf58700 1 --2- 172.21.15.7:0/3815120707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2e41085d0 0x7fd2e41089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.570+0000 7fd2eaf58700 1 --2- 172.21.15.7:0/3815120707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2e4102530 0x7fd2e41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.570+0000 7fd2eaf58700 1 -- 172.21.15.7:0/3815120707 >> 172.21.15.7:0/3815120707 conn(0x7fd2e4079640 msgr2=0x7fd2e4079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.570+0000 7fd2eaf58700 1 -- 172.21.15.7:0/3815120707 shutdown_connections 2024-06-10T01:10:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.570+0000 7fd2eaf58700 1 -- 172.21.15.7:0/3815120707 wait complete. 2024-06-10T01:10:17.567 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:17 smithi007 ceph-mon[32398]: pgmap v560: 129 pgs: 129 active+clean; 63 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 1.9 MiB/s wr, 129 op/s 2024-06-10T01:10:17.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.571+0000 7fd2eaf58700 1 Processor -- start 2024-06-10T01:10:17.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.571+0000 7fd2eaf58700 1 -- start start 2024-06-10T01:10:17.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.571+0000 7fd2eaf58700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2e4102530 0x7fd2e4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:17.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.571+0000 7fd2eaf58700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2e4102ef0 0x7fd2e4074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:17.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.571+0000 7fd2eaf58700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2e41085d0 0x7fd2e4075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:17.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.571+0000 7fd2eaf58700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e4112810 con 0x7fd2e41085d0 2024-06-10T01:10:17.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.571+0000 7fd2eaf58700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e4112690 con 0x7fd2e4102530 2024-06-10T01:10:17.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.571+0000 7fd2eaf58700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e4112990 con 0x7fd2e4102ef0 2024-06-10T01:10:17.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.571+0000 7fd2e94f5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2e41085d0 0x7fd2e4075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:17.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.572+0000 7fd2e94f5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2e41085d0 0x7fd2e4075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60592/0 (socket says 172.21.15.7:60592) 2024-06-10T01:10:17.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.572+0000 7fd2e3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2e4102ef0 0x7fd2e4074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:17.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.572+0000 7fd2e94f5700 1 -- 172.21.15.7:0/1533345020 learned_addr learned my addr 172.21.15.7:0/1533345020 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:17.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.572+0000 7fd2e8cf4700 1 --2- 172.21.15.7:0/1533345020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2e4102530 0x7fd2e4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:17.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.572+0000 7fd2e3fff700 1 -- 172.21.15.7:0/1533345020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2e4102530 msgr2=0x7fd2e4074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:17.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.572+0000 7fd2e3fff700 1 --2- 172.21.15.7:0/1533345020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2e4102530 0x7fd2e4074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.572+0000 7fd2e3fff700 1 -- 172.21.15.7:0/1533345020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2e41085d0 msgr2=0x7fd2e4075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:17.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.572+0000 7fd2e3fff700 1 --2- 172.21.15.7:0/1533345020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2e41085d0 0x7fd2e4075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.572+0000 7fd2e3fff700 1 -- 172.21.15.7:0/1533345020 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e41a7090 con 0x7fd2e4102ef0 2024-06-10T01:10:17.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.572+0000 7fd2e94f5700 1 --2- 172.21.15.7:0/1533345020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2e41085d0 0x7fd2e4075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:17.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.572+0000 7fd2e3fff700 1 --2- 172.21.15.7:0/1533345020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2e4102ef0 0x7fd2e4074cd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc00e9f0 tx=0x7fd2cc00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:17.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.572+0000 7fd2e1ffb700 1 -- 172.21.15.7:0/1533345020 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc00c9c0 con 0x7fd2e4102ef0 2024-06-10T01:10:17.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.572+0000 7fd2eaf58700 1 -- 172.21.15.7:0/1533345020 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2e41a7380 con 0x7fd2e4102ef0 2024-06-10T01:10:17.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.572+0000 7fd2e1ffb700 1 -- 172.21.15.7:0/1533345020 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2cc00cb60 con 0x7fd2e4102ef0 2024-06-10T01:10:17.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.572+0000 7fd2e1ffb700 1 -- 172.21.15.7:0/1533345020 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc018680 con 0x7fd2e4102ef0 2024-06-10T01:10:17.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.573+0000 7fd2eaf58700 1 -- 172.21.15.7:0/1533345020 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2e41a7910 con 0x7fd2e4102ef0 2024-06-10T01:10:17.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.575+0000 7fd2e1ffb700 1 -- 172.21.15.7:0/1533345020 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fd2cc010430 con 0x7fd2e4102ef0 2024-06-10T01:10:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.576+0000 7fd2e1ffb700 1 --2- 172.21.15.7:0/1533345020 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd2d00a2c20 0x7fd2d00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.576+0000 7fd2e8cf4700 1 --2- 172.21.15.7:0/1533345020 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd2d00a2c20 0x7fd2d00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.576+0000 7fd2e1ffb700 1 -- 172.21.15.7:0/1533345020 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fd2cc0cf530 con 0x7fd2e4102ef0 2024-06-10T01:10:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.576+0000 7fd2e8cf4700 1 --2- 172.21.15.7:0/1533345020 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd2d00a2c20 0x7fd2d00a50e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4009550 tx=0x7fd2d40091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.577+0000 7fd2f57dbb80 1 Processor -- start 2024-06-10T01:10:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.577+0000 7fd2f57dbb80 1 -- start start 2024-06-10T01:10:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.577+0000 7fd2f57dbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac03941500 0x55ac039418e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.577+0000 7fd2f57dbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac03941e20 0x55ac0393b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.577+0000 7fd2f57dbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac0393bc50 0x55ac0393c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.577+0000 7fd2f57dbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ac0382e060 con 0x55ac0393bc50 2024-06-10T01:10:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.577+0000 7fd2f57dbb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ac0388c950 con 0x55ac03941500 2024-06-10T01:10:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.577+0000 7fd2f57dbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ac037f4560 con 0x55ac03941e20 2024-06-10T01:10:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.577+0000 7fd2e94f5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac0393bc50 0x55ac0393c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.577+0000 7fd2e3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac03941e20 0x55ac0393b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.577+0000 7fd2e94f5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac0393bc50 0x55ac0393c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60596/0 (socket says 172.21.15.7:60596) 2024-06-10T01:10:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.577+0000 7fd2e94f5700 1 -- 172.21.15.7:0/3719152965 learned_addr learned my addr 172.21.15.7:0/3719152965 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.577+0000 7fd2e8cf4700 1 --2- 172.21.15.7:0/3719152965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac03941500 0x55ac039418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.578+0000 7fd2e94f5700 1 -- 172.21.15.7:0/3719152965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac03941e20 msgr2=0x55ac0393b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.578+0000 7fd2e94f5700 1 --2- 172.21.15.7:0/3719152965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac03941e20 0x55ac0393b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.578+0000 7fd2e94f5700 1 -- 172.21.15.7:0/3719152965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac03941500 msgr2=0x55ac039418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.578+0000 7fd2e94f5700 1 --2- 172.21.15.7:0/3719152965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac03941500 0x55ac039418e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.578+0000 7fd2e94f5700 1 -- 172.21.15.7:0/3719152965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac037db9d0 con 0x55ac0393bc50 2024-06-10T01:10:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.578+0000 7fd2e3fff700 1 --2- 172.21.15.7:0/3719152965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac03941e20 0x55ac0393b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.578+0000 7fd2e94f5700 1 --2- 172.21.15.7:0/3719152965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac0393bc50 0x55ac0393c070 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc005cb0 tx=0x7fd2dc003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97485cb9426020d8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.578+0000 7fd2daffd700 1 -- 172.21.15.7:0/3719152965 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc028070 con 0x55ac0393bc50 2024-06-10T01:10:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.578+0000 7fd2daffd700 1 -- 172.21.15.7:0/3719152965 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2dc009590 con 0x55ac0393bc50 2024-06-10T01:10:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.578+0000 7fd2daffd700 1 -- 172.21.15.7:0/3719152965 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc023460 con 0x55ac0393bc50 2024-06-10T01:10:17.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.579+0000 7fd2f57dbb80 1 -- 172.21.15.7:0/3719152965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac0393bc50 msgr2=0x55ac0393c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:17.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.579+0000 7fd2f57dbb80 1 --2- 172.21.15.7:0/3719152965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac0393bc50 0x55ac0393c070 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc005cb0 tx=0x7fd2dc003680 comp rx=0 tx=0).stop 2024-06-10T01:10:17.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.579+0000 7fd2f57dbb80 1 -- 172.21.15.7:0/3719152965 shutdown_connections 2024-06-10T01:10:17.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.579+0000 7fd2f57dbb80 1 --2- 172.21.15.7:0/3719152965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac0393bc50 0x55ac0393c070 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.579+0000 7fd2f57dbb80 1 --2- 172.21.15.7:0/3719152965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac03941500 0x55ac039418e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.579+0000 7fd2f57dbb80 1 --2- 172.21.15.7:0/3719152965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac03941e20 0x55ac0393b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.579+0000 7fd2f57dbb80 1 -- 172.21.15.7:0/3719152965 >> 172.21.15.7:0/3719152965 conn(0x55ac03868180 msgr2=0x55ac03936a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:17.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.579+0000 7fd2f57dbb80 1 -- 172.21.15.7:0/3719152965 shutdown_connections 2024-06-10T01:10:17.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.579+0000 7fd2f57dbb80 1 -- 172.21.15.7:0/3719152965 wait complete. 2024-06-10T01:10:17.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.579+0000 7fd2f57dbb80 1 -- start start 2024-06-10T01:10:17.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.579+0000 7fd2f57dbb80 1 Processor -- start 2024-06-10T01:10:17.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.580+0000 7fd2f57dbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac0393bc50 0x55ac03947b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:17.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.580+0000 7fd2f57dbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac03941500 0x55ac03944e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:17.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.580+0000 7fd2f57dbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac03941e20 0x55ac039453b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:17.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.580+0000 7fd2f57dbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ac0382d0c0 con 0x55ac0393bc50 2024-06-10T01:10:17.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.580+0000 7fd2f57dbb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ac03888910 con 0x55ac03941e20 2024-06-10T01:10:17.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.580+0000 7fd2f57dbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ac039487a0 con 0x55ac03941500 2024-06-10T01:10:17.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.580+0000 7fd2f57dbb80 10 client.0 _ll_register_callbacks cb 0x55ac038226b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:17.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.580+0000 7fd2e8cf4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac0393bc50 0x55ac03947b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:17.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.580+0000 7fd2e8cf4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac0393bc50 0x55ac03947b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60612/0 (socket says 172.21.15.7:60612) 2024-06-10T01:10:17.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.580+0000 7fd2e8cf4700 1 -- 172.21.15.7:0/2033575592 learned_addr learned my addr 172.21.15.7:0/2033575592 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:17.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.580+0000 7fd2e3fff700 1 --2- 172.21.15.7:0/2033575592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac03941500 0x55ac03944e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:17.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.580+0000 7fd2e94f5700 1 --2- 172.21.15.7:0/2033575592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac03941e20 0x55ac039453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:17.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.580+0000 7fd2e8cf4700 1 -- 172.21.15.7:0/2033575592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac03941500 msgr2=0x55ac03944e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:17.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.580+0000 7fd2e8cf4700 1 --2- 172.21.15.7:0/2033575592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac03941500 0x55ac03944e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.580+0000 7fd2e8cf4700 1 -- 172.21.15.7:0/2033575592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac03941e20 msgr2=0x55ac039453b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:17.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.580+0000 7fd2e8cf4700 1 --2- 172.21.15.7:0/2033575592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac03941e20 0x55ac039453b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.580+0000 7fd2e8cf4700 1 -- 172.21.15.7:0/2033575592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac03852850 con 0x55ac0393bc50 2024-06-10T01:10:17.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.580+0000 7fd2e8cf4700 1 --2- 172.21.15.7:0/2033575592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac0393bc50 0x55ac03947b70 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4007ab0 tx=0x7fd2d4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b824f9f626e7d6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:17.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.580+0000 7fd2c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:10:17.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.580+0000 7fd2c3fff700 1 -- 172.21.15.7:0/2033575592 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d4026070 con 0x55ac0393bc50 2024-06-10T01:10:17.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.581+0000 7fd2c3fff700 1 -- 172.21.15.7:0/2033575592 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fd2d4015ce0 con 0x55ac0393bc50 2024-06-10T01:10:17.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.581+0000 7fd2f57dbb80 1 -- 172.21.15.7:0/2033575592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2dc009590 con 0x55ac0393bc50 2024-06-10T01:10:17.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.581+0000 7fd2c3fff700 1 -- 172.21.15.7:0/2033575592 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd2d4032050 con 0x55ac0393bc50 2024-06-10T01:10:17.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.581+0000 7fd2f57dbb80 10 client.16590 fetch_fsmap learned FSMap version 30 2024-06-10T01:10:17.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.581+0000 7fd2f57dbb80 1 -- 172.21.15.7:0/2033575592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ac03952fb0 con 0x55ac0393bc50 2024-06-10T01:10:17.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.581+0000 7fd2c3fff700 1 -- 172.21.15.7:0/2033575592 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2dc009590 con 0x55ac0393bc50 2024-06-10T01:10:17.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.581+0000 7fd2c3fff700 1 -- 172.21.15.7:0/2033575592 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fd2d40163f0 con 0x55ac0393bc50 2024-06-10T01:10:17.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.581+0000 7fd2f57dbb80 10 client.16590 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:10:17.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.581+0000 7fd2f57dbb80 10 client.16590 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:10:17.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.581+0000 7fd2f57dbb80 20 client.16590 populate_metadata read hostname 'smithi007' 2024-06-10T01:10:17.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.581+0000 7fd2f57dbb80 1 --2- 172.21.15.7:0/2033575592 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ac03955590 0x55ac03957a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:17.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.581+0000 7fd2f57dbb80 4 client.16590 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:10:17.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.581+0000 7fd2f57dbb80 1 -- 172.21.15.7:0/2033575592 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac03899e50 con 0x55ac03955590 2024-06-10T01:10:17.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.582+0000 7fd2e3fff700 1 --2- 172.21.15.7:0/2033575592 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ac03955590 0x55ac03957a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:17.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.582+0000 7fd2e3fff700 1 --2- 172.21.15.7:0/2033575592 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ac03955590 0x55ac03957a10 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62af69a949c7b352 server_cookie=ab4b64e4986ed472 in_seq=0 out_seq=0 2024-06-10T01:10:17.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.582+0000 7fd2c3fff700 10 client.16590 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:17.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.583+0000 7fd2c3fff700 1 -- 172.21.15.7:0/2033575592 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ac03899e50 con 0x55ac03955590 2024-06-10T01:10:17.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.583+0000 7fd2c3fff700 10 client.16590 handle_command_reply: tid=0 2024-06-10T01:10:17.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.583+0000 7fd2f57dbb80 1 client.16590 shutdown 2024-06-10T01:10:17.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.583+0000 7fd2f57dbb80 1 -- 172.21.15.7:0/2033575592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac0393bc50 msgr2=0x55ac03947b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:17.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.583+0000 7fd2f57dbb80 1 --2- 172.21.15.7:0/2033575592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac0393bc50 0x55ac03947b70 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4007ab0 tx=0x7fd2d4004b40 comp rx=0 tx=0).stop 2024-06-10T01:10:17.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.583+0000 7fd2f57dbb80 1 -- 172.21.15.7:0/2033575592 shutdown_connections 2024-06-10T01:10:17.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.583+0000 7fd2f57dbb80 1 --2- 172.21.15.7:0/2033575592 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ac03955590 0x55ac03957a10 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.583+0000 7fd2f57dbb80 1 --2- 172.21.15.7:0/2033575592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac03941e20 0x55ac039453b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.583+0000 7fd2f57dbb80 1 --2- 172.21.15.7:0/2033575592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac0393bc50 0x55ac03947b70 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.583+0000 7fd2f57dbb80 1 --2- 172.21.15.7:0/2033575592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac03941500 0x55ac03944e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.583+0000 7fd2c3fff700 0 client.16590 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:17.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.583+0000 7fd2f57dbb80 1 -- 172.21.15.7:0/2033575592 >> 172.21.15.7:0/2033575592 conn(0x55ac03868180 msgr2=0x55ac03935bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:17.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.583+0000 7fd2f57dbb80 1 -- 172.21.15.7:0/2033575592 shutdown_connections 2024-06-10T01:10:17.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.583+0000 7fd2f57dbb80 1 -- 172.21.15.7:0/2033575592 wait complete. 2024-06-10T01:10:17.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.583+0000 7fd2f57dbb80 20 client.16590 trim_cache size 0 max 16384 2024-06-10T01:10:17.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:17 smithi078 ceph-mon[36683]: pgmap v560: 129 pgs: 129 active+clean; 63 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 1.9 MiB/s wr, 129 op/s 2024-06-10T01:10:17.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.604+0000 7fd2f57dbb80 1 Processor -- start 2024-06-10T01:10:17.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.604+0000 7fd2f57dbb80 1 -- start start 2024-06-10T01:10:17.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.604+0000 7fd2f57dbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac03868180 0x55ac0393b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:17.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.604+0000 7fd2f57dbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac0393bc50 0x55ac03940740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:17.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.604+0000 7fd2f57dbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac03941500 0x55ac03940c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:17.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.604+0000 7fd2f57dbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e41b2be0 con 0x55ac03868180 2024-06-10T01:10:17.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.604+0000 7fd2f57dbb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e41b0410 con 0x55ac0393bc50 2024-06-10T01:10:17.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.604+0000 7fd2f57dbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e41ae260 con 0x55ac03941500 2024-06-10T01:10:17.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.604+0000 7fd2e8cf4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac03868180 0x55ac0393b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:17.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.604+0000 7fd2e8cf4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac03868180 0x55ac0393b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60622/0 (socket says 172.21.15.7:60622) 2024-06-10T01:10:17.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.604+0000 7fd2e8cf4700 1 -- 172.21.15.7:0/1631905577 learned_addr learned my addr 172.21.15.7:0/1631905577 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:17.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.604+0000 7fd2e8cf4700 1 -- 172.21.15.7:0/1631905577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac03941500 msgr2=0x55ac03940c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:17.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.604+0000 7fd2e8cf4700 1 --2- 172.21.15.7:0/1631905577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac03941500 0x55ac03940c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.604+0000 7fd2e8cf4700 1 -- 172.21.15.7:0/1631905577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac0393bc50 msgr2=0x55ac03940740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:17.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.604+0000 7fd2e3fff700 1 --2- 172.21.15.7:0/1631905577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac0393bc50 0x55ac03940740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:17.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.604+0000 7fd2e8cf4700 1 --2- 172.21.15.7:0/1631905577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac0393bc50 0x55ac03940740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.604+0000 7fd2e8cf4700 1 -- 172.21.15.7:0/1631905577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac03958090 con 0x55ac03868180 2024-06-10T01:10:17.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.605+0000 7fd2e8cf4700 1 --2- 172.21.15.7:0/1631905577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac03868180 0x55ac0393b710 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd2d40167c0 tx=0x7fd2d4004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce9e6970c9fa163 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:17.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.605+0000 7fd2da7fc700 1 -- 172.21.15.7:0/1631905577 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d4026040 con 0x55ac03868180 2024-06-10T01:10:17.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.605+0000 7fd2da7fc700 1 -- 172.21.15.7:0/1631905577 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2d403b070 con 0x55ac03868180 2024-06-10T01:10:17.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.605+0000 7fd2da7fc700 1 -- 172.21.15.7:0/1631905577 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d401c040 con 0x55ac03868180 2024-06-10T01:10:17.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.605+0000 7fd2f57dbb80 1 -- 172.21.15.7:0/1631905577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac03868180 msgr2=0x55ac0393b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:17.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.605+0000 7fd2f57dbb80 1 --2- 172.21.15.7:0/1631905577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac03868180 0x55ac0393b710 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd2d40167c0 tx=0x7fd2d4004390 comp rx=0 tx=0).stop 2024-06-10T01:10:17.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.605+0000 7fd2f57dbb80 1 -- 172.21.15.7:0/1631905577 shutdown_connections 2024-06-10T01:10:17.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.605+0000 7fd2f57dbb80 1 --2- 172.21.15.7:0/1631905577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac03941500 0x55ac03940c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.605+0000 7fd2f57dbb80 1 --2- 172.21.15.7:0/1631905577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac03868180 0x55ac0393b710 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.605+0000 7fd2f57dbb80 1 --2- 172.21.15.7:0/1631905577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac0393bc50 0x55ac03940740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.605+0000 7fd2f57dbb80 1 -- 172.21.15.7:0/1631905577 >> 172.21.15.7:0/1631905577 conn(0x55ac03944310 msgr2=0x55ac03940030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:17.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.606+0000 7fd2f57dbb80 1 -- 172.21.15.7:0/1631905577 shutdown_connections 2024-06-10T01:10:17.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.606+0000 7fd2f57dbb80 1 -- 172.21.15.7:0/1631905577 wait complete. 2024-06-10T01:10:17.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.606+0000 7fd2f57dbb80 1 -- start start 2024-06-10T01:10:17.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.606+0000 7fd2f57dbb80 1 Processor -- start 2024-06-10T01:10:17.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.606+0000 7fd2f57dbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac03868180 0x55ac0394e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:17.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.606+0000 7fd2f57dbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac0393bc50 0x55ac0394aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:17.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.606+0000 7fd2f57dbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac03941500 0x55ac0394b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:17.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.606+0000 7fd2f57dbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e41ae8d0 con 0x55ac03941500 2024-06-10T01:10:17.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.606+0000 7fd2f57dbb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e41afd40 con 0x55ac0393bc50 2024-06-10T01:10:17.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.606+0000 7fd2f57dbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e41aefa0 con 0x55ac03868180 2024-06-10T01:10:17.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.606+0000 7fd2f57dbb80 10 client.0 _ll_register_callbacks cb 0x55ac038226b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:17.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.607+0000 7fd2e94f5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac03941500 0x55ac0394b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:17.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.607+0000 7fd2e94f5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac03941500 0x55ac0394b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60636/0 (socket says 172.21.15.7:60636) 2024-06-10T01:10:17.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.607+0000 7fd2e94f5700 1 -- 172.21.15.7:0/4271985062 learned_addr learned my addr 172.21.15.7:0/4271985062 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:17.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.607+0000 7fd2e8cf4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac03868180 0x55ac0394e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:17.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.607+0000 7fd2e3fff700 1 --2- 172.21.15.7:0/4271985062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac0393bc50 0x55ac0394aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:17.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.607+0000 7fd2e94f5700 1 -- 172.21.15.7:0/4271985062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac03868180 msgr2=0x55ac0394e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:17.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.607+0000 7fd2e94f5700 1 --2- 172.21.15.7:0/4271985062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac03868180 0x55ac0394e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.607+0000 7fd2e94f5700 1 -- 172.21.15.7:0/4271985062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac0393bc50 msgr2=0x55ac0394aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:17.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.607+0000 7fd2e94f5700 1 --2- 172.21.15.7:0/4271985062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac0393bc50 0x55ac0394aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.607+0000 7fd2e94f5700 1 -- 172.21.15.7:0/4271985062 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac039c02c0 con 0x55ac03941500 2024-06-10T01:10:17.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.607+0000 7fd2e8cf4700 1 --2- 172.21.15.7:0/4271985062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac03868180 0x55ac0394e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:17.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.607+0000 7fd2e94f5700 1 --2- 172.21.15.7:0/4271985062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac03941500 0x55ac0394b120 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc026040 tx=0x7fd2dc003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed2fa171ef81ba1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:17.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.607+0000 7fd2c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:10:17.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.607+0000 7fd2c3fff700 1 -- 172.21.15.7:0/4271985062 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc031060 con 0x55ac03941500 2024-06-10T01:10:17.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.607+0000 7fd2c3fff700 1 -- 172.21.15.7:0/4271985062 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fd2dc003e00 con 0x55ac03941500 2024-06-10T01:10:17.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.607+0000 7fd2f57dbb80 1 -- 172.21.15.7:0/4271985062 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2d403b070 con 0x55ac03941500 2024-06-10T01:10:17.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.607+0000 7fd2c3fff700 1 -- 172.21.15.7:0/4271985062 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd2dc038050 con 0x55ac03941500 2024-06-10T01:10:17.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.607+0000 7fd2f57dbb80 10 client.16596 fetch_fsmap learned FSMap version 30 2024-06-10T01:10:17.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.607+0000 7fd2f57dbb80 1 -- 172.21.15.7:0/4271985062 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ac039c14e0 con 0x55ac03941500 2024-06-10T01:10:17.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.608+0000 7fd2c3fff700 1 -- 172.21.15.7:0/4271985062 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2d403b070 con 0x55ac03941500 2024-06-10T01:10:17.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.608+0000 7fd2c3fff700 1 -- 172.21.15.7:0/4271985062 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fd2dc02b070 con 0x55ac03941500 2024-06-10T01:10:17.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.608+0000 7fd2f57dbb80 10 client.16596 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:10:17.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.608+0000 7fd2f57dbb80 10 client.16596 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:10:17.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.608+0000 7fd2f57dbb80 20 client.16596 populate_metadata read hostname 'smithi007' 2024-06-10T01:10:17.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.608+0000 7fd2f57dbb80 1 --2- 172.21.15.7:0/4271985062 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ac03941e20 0x55ac039c3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:17.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.608+0000 7fd2f57dbb80 4 client.16596 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3d7bfe3b", "release": true, "await": true}] 2024-06-10T01:10:17.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.608+0000 7fd2f57dbb80 1 -- 172.21.15.7:0/4271985062 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3d7bfe3b", "release": true, "await": true}) -- 0x55ac038943d0 con 0x55ac03941e20 2024-06-10T01:10:17.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.608+0000 7fd2e8cf4700 1 --2- 172.21.15.7:0/4271985062 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ac03941e20 0x55ac039c3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:17.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.609+0000 7fd2e8cf4700 1 --2- 172.21.15.7:0/4271985062 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ac03941e20 0x55ac039c3770 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b276e4146d32a01 server_cookie=6e5697dab4be6e15 in_seq=0 out_seq=0 2024-06-10T01:10:17.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.609+0000 7fd2c3fff700 10 client.16596 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:17.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.621+0000 7fd2c3fff700 1 -- 172.21.15.7:0/4271985062 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ac038943d0 con 0x55ac03941e20 2024-06-10T01:10:17.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.621+0000 7fd2c3fff700 10 client.16596 handle_command_reply: tid=0 2024-06-10T01:10:17.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.621+0000 7fd2f57dbb80 1 client.16596 shutdown 2024-06-10T01:10:17.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.621+0000 7fd2f57dbb80 1 -- 172.21.15.7:0/4271985062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac03941500 msgr2=0x55ac0394b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:17.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.621+0000 7fd2f57dbb80 1 --2- 172.21.15.7:0/4271985062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac03941500 0x55ac0394b120 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc026040 tx=0x7fd2dc003a90 comp rx=0 tx=0).stop 2024-06-10T01:10:17.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.621+0000 7fd2f57dbb80 1 -- 172.21.15.7:0/4271985062 shutdown_connections 2024-06-10T01:10:17.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.621+0000 7fd2f57dbb80 1 --2- 172.21.15.7:0/4271985062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac03941500 0x55ac0394b120 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.621+0000 7fd2f57dbb80 1 --2- 172.21.15.7:0/4271985062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac03868180 0x55ac0394e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.621+0000 7fd2f57dbb80 1 --2- 172.21.15.7:0/4271985062 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ac03941e20 0x55ac039c3770 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.621+0000 7fd2c3fff700 0 client.16596 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:17.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.621+0000 7fd2f57dbb80 1 --2- 172.21.15.7:0/4271985062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac0393bc50 0x55ac0394aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.621+0000 7fd2f57dbb80 1 -- 172.21.15.7:0/4271985062 >> 172.21.15.7:0/4271985062 conn(0x55ac03944310 msgr2=0x55ac039408c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:17.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.621+0000 7fd2f57dbb80 1 -- 172.21.15.7:0/4271985062 shutdown_connections 2024-06-10T01:10:17.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.621+0000 7fd2f57dbb80 1 -- 172.21.15.7:0/4271985062 wait complete. 2024-06-10T01:10:17.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.621+0000 7fd2f57dbb80 20 client.16596 trim_cache size 0 max 16384 2024-06-10T01:10:17.618 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:10:17.618 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 30, 2024-06-10T01:10:17.618 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:10:17.618 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 12, 2024-06-10T01:10:17.618 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:10:17.618 INFO:teuthology.orchestra.run.smithi007.stdout: "3d7bfe3b": { 2024-06-10T01:10:17.618 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 12, 2024-06-10T01:10:17.619 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:10:17.619 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:10:17.619 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:10:17.619 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:10:17.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:10:17.619 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:10:17.619 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 138.2, 2024-06-10T01:10:17.619 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:10:17.619 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:10:17.619 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:10:17.619 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:10:17.619 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:10:17.619 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:10:17.620 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:10:17.620 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:10:17.620 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:10:17.620 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:10:17.620 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:10:17.620 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:10:17.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.622+0000 7fd2eaf58700 1 -- 172.21.15.7:0/1533345020 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd2d00a2c20 msgr2=0x7fd2d00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:17.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.622+0000 7fd2eaf58700 1 --2- 172.21.15.7:0/1533345020 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd2d00a2c20 0x7fd2d00a50e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4009550 tx=0x7fd2d40091b0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.622+0000 7fd2eaf58700 1 -- 172.21.15.7:0/1533345020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2e4102ef0 msgr2=0x7fd2e4074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:17.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.622+0000 7fd2eaf58700 1 --2- 172.21.15.7:0/1533345020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2e4102ef0 0x7fd2e4074cd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc00e9f0 tx=0x7fd2cc00ed00 comp rx=0 tx=0).stop 2024-06-10T01:10:17.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.622+0000 7fd2eaf58700 1 -- 172.21.15.7:0/1533345020 shutdown_connections 2024-06-10T01:10:17.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.622+0000 7fd2eaf58700 1 --2- 172.21.15.7:0/1533345020 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd2d00a2c20 0x7fd2d00a50e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.622+0000 7fd2eaf58700 1 --2- 172.21.15.7:0/1533345020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2e41085d0 0x7fd2e4075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.622+0000 7fd2eaf58700 1 --2- 172.21.15.7:0/1533345020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2e4102530 0x7fd2e4074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.622+0000 7fd2eaf58700 1 --2- 172.21.15.7:0/1533345020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2e4102ef0 0x7fd2e4074cd0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:17.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.622+0000 7fd2eaf58700 1 -- 172.21.15.7:0/1533345020 >> 172.21.15.7:0/1533345020 conn(0x7fd2e4079640 msgr2=0x7fd2e4100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:17.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.622+0000 7fd2eaf58700 1 -- 172.21.15.7:0/1533345020 shutdown_connections 2024-06-10T01:10:17.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:17.622+0000 7fd2eaf58700 1 -- 172.21.15.7:0/1533345020 wait complete. 2024-06-10T01:10:17.630 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:10:17.630 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:10:17.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:17 smithi175 ceph-mon[36636]: pgmap v560: 129 pgs: 129 active+clean; 63 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 1.9 MiB/s wr, 129 op/s 2024-06-10T01:10:17.715 INFO:tasks.workunit.client.0.smithi007.stdout: 1 259438 7.91 MB/sec execute 58 sec latency 15959.802 ms 2024-06-10T01:10:18.410 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3d7bfe3b', total seconds elapsed: 26.4 2024-06-10T01:10:18.427 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:10:18.716 INFO:tasks.workunit.client.0.smithi007.stdout: 1 261949 7.99 MB/sec execute 59 sec latency 40.720 ms 2024-06-10T01:10:19.579 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:19 smithi078 ceph-mon[36683]: pgmap v561: 129 pgs: 129 active+clean; 62 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2024-06-10T01:10:19.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:19 smithi175 ceph-mon[36636]: pgmap v561: 129 pgs: 129 active+clean; 62 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2024-06-10T01:10:19.716 INFO:tasks.workunit.client.0.smithi007.stdout: 1 263620 8.00 MB/sec execute 60 sec latency 16.195 ms 2024-06-10T01:10:19.746 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:19 smithi007 ceph-mon[32398]: pgmap v561: 129 pgs: 129 active+clean; 62 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2024-06-10T01:10:20.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.294+0000 7f883ec06700 1 -- 172.21.15.7:0/2856714988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8838107570 msgr2=0x7f8838107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:20.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.294+0000 7f883ec06700 1 --2- 172.21.15.7:0/2856714988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8838107570 0x7f8838107950 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8820009990 tx=0x7f8820009ca0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.294+0000 7f883ec06700 1 -- 172.21.15.7:0/2856714988 shutdown_connections 2024-06-10T01:10:20.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.294+0000 7f883ec06700 1 --2- 172.21.15.7:0/2856714988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8838101810 0x7f883810f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.294+0000 7f883ec06700 1 --2- 172.21.15.7:0/2856714988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8838107570 0x7f8838107950 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.294+0000 7f883ec06700 1 --2- 172.21.15.7:0/2856714988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8838100e50 0x7f88381012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.294+0000 7f883ec06700 1 -- 172.21.15.7:0/2856714988 >> 172.21.15.7:0/2856714988 conn(0x7f88380766b0 msgr2=0x7f8838076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:20.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.294+0000 7f883ec06700 1 -- 172.21.15.7:0/2856714988 shutdown_connections 2024-06-10T01:10:20.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.295+0000 7f883ec06700 1 -- 172.21.15.7:0/2856714988 wait complete. 2024-06-10T01:10:20.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.296+0000 7f883ec06700 1 Processor -- start 2024-06-10T01:10:20.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.296+0000 7f883ec06700 1 -- start start 2024-06-10T01:10:20.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.296+0000 7f883ec06700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8838100e50 0x7f8838102030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:20.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.296+0000 7f883ec06700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8838101810 0x7f8838102570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:20.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.296+0000 7f883ec06700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8838107570 0x7f8838102ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:20.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.296+0000 7f883ec06700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8838112800 con 0x7f8838100e50 2024-06-10T01:10:20.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.296+0000 7f883ec06700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8838112680 con 0x7f8838107570 2024-06-10T01:10:20.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.296+0000 7f883ec06700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8838112980 con 0x7f8838101810 2024-06-10T01:10:20.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.296+0000 7f883c9a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8838100e50 0x7f8838102030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:20.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.296+0000 7f883c9a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8838100e50 0x7f8838102030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60660/0 (socket says 172.21.15.7:60660) 2024-06-10T01:10:20.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.296+0000 7f883c9a2700 1 -- 172.21.15.7:0/213952076 learned_addr learned my addr 172.21.15.7:0/213952076 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.296+0000 7f8837fff700 1 --2- 172.21.15.7:0/213952076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8838101810 0x7f8838102570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.296+0000 7f883d1a3700 1 --2- 172.21.15.7:0/213952076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8838107570 0x7f8838102ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.297+0000 7f883c9a2700 1 -- 172.21.15.7:0/213952076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8838101810 msgr2=0x7f8838102570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.297+0000 7f883c9a2700 1 --2- 172.21.15.7:0/213952076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8838101810 0x7f8838102570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.297+0000 7f883c9a2700 1 -- 172.21.15.7:0/213952076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8838107570 msgr2=0x7f8838102ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.297+0000 7f883c9a2700 1 --2- 172.21.15.7:0/213952076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8838107570 0x7f8838102ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.297+0000 7f883c9a2700 1 -- 172.21.15.7:0/213952076 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88381a7220 con 0x7f8838100e50 2024-06-10T01:10:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.297+0000 7f883d1a3700 1 --2- 172.21.15.7:0/213952076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8838107570 0x7f8838102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:10:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.297+0000 7f8837fff700 1 --2- 172.21.15.7:0/213952076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8838101810 0x7f8838102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.297+0000 7f883c9a2700 1 --2- 172.21.15.7:0/213952076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8838100e50 0x7f8838102030 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8820000c00 tx=0x7f882000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.297+0000 7f8835ffb700 1 -- 172.21.15.7:0/213952076 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8820026070 con 0x7f8838100e50 2024-06-10T01:10:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.297+0000 7f8835ffb700 1 -- 172.21.15.7:0/213952076 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8820003d70 con 0x7f8838100e50 2024-06-10T01:10:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.297+0000 7f883ec06700 1 -- 172.21.15.7:0/213952076 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88381a74b0 con 0x7f8838100e50 2024-06-10T01:10:20.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.297+0000 7f8835ffb700 1 -- 172.21.15.7:0/213952076 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8820021410 con 0x7f8838100e50 2024-06-10T01:10:20.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.297+0000 7f883ec06700 1 -- 172.21.15.7:0/213952076 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88381a7960 con 0x7f8838100e50 2024-06-10T01:10:20.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.300+0000 7f8835ffb700 1 -- 172.21.15.7:0/213952076 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f8820029070 con 0x7f8838100e50 2024-06-10T01:10:20.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.300+0000 7f8835ffb700 1 --2- 172.21.15.7:0/213952076 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f88240a2ca0 0x7f88240a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:20.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.300+0000 7f8835ffb700 1 -- 172.21.15.7:0/213952076 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f88200d01b0 con 0x7f8838100e50 2024-06-10T01:10:20.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.300+0000 7f8837fff700 1 --2- 172.21.15.7:0/213952076 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f88240a2ca0 0x7f88240a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:20.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.301+0000 7f8837fff700 1 --2- 172.21.15.7:0/213952076 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f88240a2ca0 0x7f88240a5160 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8838106490 tx=0x7f88280091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:20.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.302+0000 7f8849489b80 1 Processor -- start 2024-06-10T01:10:20.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.302+0000 7f8849489b80 1 -- start start 2024-06-10T01:10:20.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.302+0000 7f8849489b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed7bf956c0 0x55ed7bf95aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:20.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.302+0000 7f8849489b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed7bf95fe0 0x55ed7bf8fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:20.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.302+0000 7f8849489b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7bf8ffa0 0x55ed7bf903c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:20.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.302+0000 7f8849489b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ed7bee1890 con 0x55ed7bf956c0 2024-06-10T01:10:20.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.302+0000 7f8849489b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ed7be897a0 con 0x55ed7bf8ffa0 2024-06-10T01:10:20.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.302+0000 7f8849489b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ed7bedd850 con 0x55ed7bf95fe0 2024-06-10T01:10:20.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.302+0000 7f883c9a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed7bf956c0 0x55ed7bf95aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:20.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.302+0000 7f883c9a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed7bf956c0 0x55ed7bf95aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60672/0 (socket says 172.21.15.7:60672) 2024-06-10T01:10:20.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.302+0000 7f883c9a2700 1 -- 172.21.15.7:0/143677513 learned_addr learned my addr 172.21.15.7:0/143677513 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:20.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.302+0000 7f8837fff700 1 --2- 172.21.15.7:0/143677513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed7bf95fe0 0x55ed7bf8fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:20.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.302+0000 7f883d1a3700 1 --2- 172.21.15.7:0/143677513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7bf8ffa0 0x55ed7bf903c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:20.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.302+0000 7f883c9a2700 1 -- 172.21.15.7:0/143677513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed7bf95fe0 msgr2=0x55ed7bf8fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:20.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.302+0000 7f883c9a2700 1 --2- 172.21.15.7:0/143677513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed7bf95fe0 0x55ed7bf8fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.302+0000 7f883c9a2700 1 -- 172.21.15.7:0/143677513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7bf8ffa0 msgr2=0x55ed7bf903c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:20.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.302+0000 7f883c9a2700 1 --2- 172.21.15.7:0/143677513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7bf8ffa0 0x55ed7bf903c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.303+0000 7f883c9a2700 1 -- 172.21.15.7:0/143677513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed7be7ffd0 con 0x55ed7bf956c0 2024-06-10T01:10:20.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.303+0000 7f883c9a2700 1 --2- 172.21.15.7:0/143677513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed7bf956c0 0x55ed7bf95aa0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f88200a4dd0 tx=0x7f8820009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f057d3746e5ba2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:20.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.303+0000 7f882effd700 1 -- 172.21.15.7:0/143677513 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88200043b0 con 0x55ed7bf956c0 2024-06-10T01:10:20.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.303+0000 7f882effd700 1 -- 172.21.15.7:0/143677513 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f882009ce20 con 0x55ed7bf956c0 2024-06-10T01:10:20.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.303+0000 7f882effd700 1 -- 172.21.15.7:0/143677513 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882009a850 con 0x55ed7bf956c0 2024-06-10T01:10:20.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.304+0000 7f8849489b80 1 -- 172.21.15.7:0/143677513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed7bf956c0 msgr2=0x55ed7bf95aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:20.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.304+0000 7f8849489b80 1 --2- 172.21.15.7:0/143677513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed7bf956c0 0x55ed7bf95aa0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f88200a4dd0 tx=0x7f8820009310 comp rx=0 tx=0).stop 2024-06-10T01:10:20.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.304+0000 7f8849489b80 1 -- 172.21.15.7:0/143677513 shutdown_connections 2024-06-10T01:10:20.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.304+0000 7f8849489b80 1 --2- 172.21.15.7:0/143677513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7bf8ffa0 0x55ed7bf903c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.304+0000 7f8849489b80 1 --2- 172.21.15.7:0/143677513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed7bf956c0 0x55ed7bf95aa0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.304+0000 7f8849489b80 1 --2- 172.21.15.7:0/143677513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed7bf95fe0 0x55ed7bf8fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.304+0000 7f8849489b80 1 -- 172.21.15.7:0/143677513 >> 172.21.15.7:0/143677513 conn(0x55ed7bebad40 msgr2=0x55ed7bf8a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:20.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.304+0000 7f8849489b80 1 -- 172.21.15.7:0/143677513 shutdown_connections 2024-06-10T01:10:20.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.304+0000 7f8849489b80 1 -- 172.21.15.7:0/143677513 wait complete. 2024-06-10T01:10:20.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.304+0000 7f8849489b80 1 -- start start 2024-06-10T01:10:20.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.304+0000 7f8849489b80 1 Processor -- start 2024-06-10T01:10:20.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.305+0000 7f8849489b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7bf8ffa0 0x55ed7bf9bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:20.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.305+0000 7f8849489b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed7bf956c0 0x55ed7bf99090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:20.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.305+0000 7f8849489b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed7bf95fe0 0x55ed7bf995d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:20.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.305+0000 7f8849489b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ed7bf9c770 con 0x55ed7bf95fe0 2024-06-10T01:10:20.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.305+0000 7f8849489b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ed7be81c70 con 0x55ed7bf8ffa0 2024-06-10T01:10:20.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.305+0000 7f8849489b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ed7bf9c8f0 con 0x55ed7bf956c0 2024-06-10T01:10:20.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.305+0000 7f8849489b80 10 client.0 _ll_register_callbacks cb 0x55ed7be7b9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:20.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.305+0000 7f883d1a3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed7bf95fe0 0x55ed7bf995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:20.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.305+0000 7f883c9a2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7bf8ffa0 0x55ed7bf9bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:20.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.305+0000 7f883d1a3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed7bf95fe0 0x55ed7bf995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60682/0 (socket says 172.21.15.7:60682) 2024-06-10T01:10:20.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.305+0000 7f883d1a3700 1 -- 172.21.15.7:0/1507744357 learned_addr learned my addr 172.21.15.7:0/1507744357 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:20.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.305+0000 7f8837fff700 1 --2- 172.21.15.7:0/1507744357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed7bf956c0 0x55ed7bf99090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:20.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.305+0000 7f883d1a3700 1 -- 172.21.15.7:0/1507744357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed7bf956c0 msgr2=0x55ed7bf99090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:20.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.305+0000 7f883d1a3700 1 --2- 172.21.15.7:0/1507744357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed7bf956c0 0x55ed7bf99090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.305+0000 7f883d1a3700 1 -- 172.21.15.7:0/1507744357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7bf8ffa0 msgr2=0x55ed7bf9bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:20.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.305+0000 7f883d1a3700 1 --2- 172.21.15.7:0/1507744357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7bf8ffa0 0x55ed7bf9bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.305+0000 7f883d1a3700 1 -- 172.21.15.7:0/1507744357 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed7be30950 con 0x55ed7bf95fe0 2024-06-10T01:10:20.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.305+0000 7f883c9a2700 1 --2- 172.21.15.7:0/1507744357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7bf8ffa0 0x55ed7bf9bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:20.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.305+0000 7f883d1a3700 1 --2- 172.21.15.7:0/1507744357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed7bf95fe0 0x55ed7bf995d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f8830009f80 tx=0x7f8830012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b1001c2638963e9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:20.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.305+0000 7f8817fff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:10:20.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.306+0000 7f8817fff700 1 -- 172.21.15.7:0/1507744357 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8830004220 con 0x55ed7bf95fe0 2024-06-10T01:10:20.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.306+0000 7f8817fff700 1 -- 172.21.15.7:0/1507744357 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f88300043c0 con 0x55ed7bf95fe0 2024-06-10T01:10:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.306+0000 7f8849489b80 1 -- 172.21.15.7:0/1507744357 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f882009ce20 con 0x55ed7bf95fe0 2024-06-10T01:10:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.306+0000 7f8817fff700 1 -- 172.21.15.7:0/1507744357 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8830024050 con 0x55ed7bf95fe0 2024-06-10T01:10:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.306+0000 7f8849489b80 10 client.16608 fetch_fsmap learned FSMap version 30 2024-06-10T01:10:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.306+0000 7f8849489b80 1 -- 172.21.15.7:0/1507744357 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ed7bfa7120 con 0x55ed7bf95fe0 2024-06-10T01:10:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.306+0000 7f8817fff700 1 -- 172.21.15.7:0/1507744357 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f882009ce20 con 0x55ed7bf95fe0 2024-06-10T01:10:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.306+0000 7f8817fff700 1 -- 172.21.15.7:0/1507744357 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f8830019030 con 0x55ed7bf95fe0 2024-06-10T01:10:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.306+0000 7f8849489b80 10 client.16608 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:10:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.306+0000 7f8849489b80 10 client.16608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:10:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.306+0000 7f8849489b80 20 client.16608 populate_metadata read hostname 'smithi007' 2024-06-10T01:10:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.306+0000 7f8849489b80 1 --2- 172.21.15.7:0/1507744357 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ed7bfa9800 0x55ed7bfabc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.306+0000 7f8849489b80 4 client.16608 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:10:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.306+0000 7f8849489b80 1 -- 172.21.15.7:0/1507744357 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed7beeed90 con 0x55ed7bfa9800 2024-06-10T01:10:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.307+0000 7f883c9a2700 1 --2- 172.21.15.7:0/1507744357 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ed7bfa9800 0x55ed7bfabc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.307+0000 7f8817fff700 10 client.16608 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:20.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.307+0000 7f883c9a2700 1 --2- 172.21.15.7:0/1507744357 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ed7bfa9800 0x55ed7bfabc10 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c024f08271c3e48 server_cookie=77c97d043b3933eb in_seq=0 out_seq=0 2024-06-10T01:10:20.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.309+0000 7f8817fff700 1 -- 172.21.15.7:0/1507744357 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ed7beeed90 con 0x55ed7bfa9800 2024-06-10T01:10:20.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.309+0000 7f8817fff700 10 client.16608 handle_command_reply: tid=0 2024-06-10T01:10:20.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.309+0000 7f8849489b80 1 client.16608 shutdown 2024-06-10T01:10:20.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.309+0000 7f8849489b80 1 -- 172.21.15.7:0/1507744357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed7bf95fe0 msgr2=0x55ed7bf995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:20.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.309+0000 7f8849489b80 1 --2- 172.21.15.7:0/1507744357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed7bf95fe0 0x55ed7bf995d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f8830009f80 tx=0x7f8830012730 comp rx=0 tx=0).stop 2024-06-10T01:10:20.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.310+0000 7f8849489b80 1 -- 172.21.15.7:0/1507744357 shutdown_connections 2024-06-10T01:10:20.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.310+0000 7f8849489b80 1 --2- 172.21.15.7:0/1507744357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed7bf95fe0 0x55ed7bf995d0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.310+0000 7f8849489b80 1 --2- 172.21.15.7:0/1507744357 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ed7bfa9800 0x55ed7bfabc10 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.310+0000 7f8849489b80 1 --2- 172.21.15.7:0/1507744357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7bf8ffa0 0x55ed7bf9bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.310+0000 7f8849489b80 1 --2- 172.21.15.7:0/1507744357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed7bf956c0 0x55ed7bf99090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.310+0000 7f8849489b80 1 -- 172.21.15.7:0/1507744357 >> 172.21.15.7:0/1507744357 conn(0x55ed7bebad40 msgr2=0x55ed7bf8a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:20.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.310+0000 7f8817fff700 0 client.16608 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:20.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.310+0000 7f8849489b80 1 -- 172.21.15.7:0/1507744357 shutdown_connections 2024-06-10T01:10:20.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.310+0000 7f8849489b80 1 -- 172.21.15.7:0/1507744357 wait complete. 2024-06-10T01:10:20.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.310+0000 7f8849489b80 20 client.16608 trim_cache size 0 max 16384 2024-06-10T01:10:20.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.333+0000 7f8849489b80 1 Processor -- start 2024-06-10T01:10:20.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.333+0000 7f8849489b80 1 -- start start 2024-06-10T01:10:20.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.333+0000 7f8849489b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed7bf8ffa0 0x55ed7bf8fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:20.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.333+0000 7f8849489b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7bf956c0 0x55ed7bf84590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:20.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.333+0000 7f8849489b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed7bf95fe0 0x55ed7bfb14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:20.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.333+0000 7f8849489b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f88381b2d90 con 0x55ed7bf8ffa0 2024-06-10T01:10:20.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.333+0000 7f8849489b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f88381b05c0 con 0x55ed7bf956c0 2024-06-10T01:10:20.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.333+0000 7f8849489b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f88381ae410 con 0x55ed7bf95fe0 2024-06-10T01:10:20.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.333+0000 7f883c9a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed7bf8ffa0 0x55ed7bf8fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:20.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.333+0000 7f883c9a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed7bf8ffa0 0x55ed7bf8fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60696/0 (socket says 172.21.15.7:60696) 2024-06-10T01:10:20.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.333+0000 7f883c9a2700 1 -- 172.21.15.7:0/2940959426 learned_addr learned my addr 172.21.15.7:0/2940959426 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:20.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.333+0000 7f883d1a3700 1 --2- 172.21.15.7:0/2940959426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed7bf95fe0 0x55ed7bfb14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:20.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.333+0000 7f8837fff700 1 --2- 172.21.15.7:0/2940959426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7bf956c0 0x55ed7bf84590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:20.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.334+0000 7f883c9a2700 1 -- 172.21.15.7:0/2940959426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed7bf95fe0 msgr2=0x55ed7bfb14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:20.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.334+0000 7f883c9a2700 1 --2- 172.21.15.7:0/2940959426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed7bf95fe0 0x55ed7bfb14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.334+0000 7f883c9a2700 1 -- 172.21.15.7:0/2940959426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7bf956c0 msgr2=0x55ed7bf84590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:20.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.334+0000 7f883c9a2700 1 --2- 172.21.15.7:0/2940959426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7bf956c0 0x55ed7bf84590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.334+0000 7f883c9a2700 1 -- 172.21.15.7:0/2940959426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed7bf8c0d0 con 0x55ed7bf8ffa0 2024-06-10T01:10:20.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.334+0000 7f8837fff700 1 --2- 172.21.15.7:0/2940959426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7bf956c0 0x55ed7bf84590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:20.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.334+0000 7f883c9a2700 1 --2- 172.21.15.7:0/2940959426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed7bf8ffa0 0x55ed7bf8fa60 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f882009c6b0 tx=0x7f882001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ef2189414c888a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:20.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.334+0000 7f882e7fc700 1 -- 172.21.15.7:0/2940959426 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88200a0ad0 con 0x55ed7bf8ffa0 2024-06-10T01:10:20.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.334+0000 7f882e7fc700 1 -- 172.21.15.7:0/2940959426 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88200095a0 con 0x55ed7bf8ffa0 2024-06-10T01:10:20.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.334+0000 7f882e7fc700 1 -- 172.21.15.7:0/2940959426 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88200870c0 con 0x55ed7bf8ffa0 2024-06-10T01:10:20.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.335+0000 7f8849489b80 1 -- 172.21.15.7:0/2940959426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed7bf8ffa0 msgr2=0x55ed7bf8fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:20.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.335+0000 7f8849489b80 1 --2- 172.21.15.7:0/2940959426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed7bf8ffa0 0x55ed7bf8fa60 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f882009c6b0 tx=0x7f882001a040 comp rx=0 tx=0).stop 2024-06-10T01:10:20.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.335+0000 7f8849489b80 1 -- 172.21.15.7:0/2940959426 shutdown_connections 2024-06-10T01:10:20.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.335+0000 7f8849489b80 1 --2- 172.21.15.7:0/2940959426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed7bf95fe0 0x55ed7bfb14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.335+0000 7f8849489b80 1 --2- 172.21.15.7:0/2940959426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed7bf8ffa0 0x55ed7bf8fa60 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.335+0000 7f8849489b80 1 --2- 172.21.15.7:0/2940959426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7bf956c0 0x55ed7bf84590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.335+0000 7f8849489b80 1 -- 172.21.15.7:0/2940959426 >> 172.21.15.7:0/2940959426 conn(0x55ed7bebad40 msgr2=0x55ed7bf83e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:20.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.335+0000 7f8849489b80 1 -- 172.21.15.7:0/2940959426 shutdown_connections 2024-06-10T01:10:20.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.335+0000 7f8849489b80 1 -- 172.21.15.7:0/2940959426 wait complete. 2024-06-10T01:10:20.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.335+0000 7f8849489b80 1 -- start start 2024-06-10T01:10:20.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.335+0000 7f8849489b80 1 Processor -- start 2024-06-10T01:10:20.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.336+0000 7f8849489b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed7bf8ffa0 0x55ed7bfb1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:20.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.336+0000 7f8849489b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed7bf956c0 0x55ed7bfb1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:20.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.336+0000 7f8849489b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7bf95fe0 0x55ed7bfb1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:20.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.336+0000 7f8849489b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f88381aea80 con 0x55ed7bf8ffa0 2024-06-10T01:10:20.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.336+0000 7f8849489b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f88381afef0 con 0x55ed7bf95fe0 2024-06-10T01:10:20.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.336+0000 7f8849489b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f88381af150 con 0x55ed7bf956c0 2024-06-10T01:10:20.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.336+0000 7f8849489b80 10 client.0 _ll_register_callbacks cb 0x55ed7be7b9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:20.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.336+0000 7f883c9a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed7bf8ffa0 0x55ed7bfb1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:20.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.336+0000 7f883c9a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed7bf8ffa0 0x55ed7bfb1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60704/0 (socket says 172.21.15.7:60704) 2024-06-10T01:10:20.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.336+0000 7f883c9a2700 1 -- 172.21.15.7:0/2290882877 learned_addr learned my addr 172.21.15.7:0/2290882877 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:20.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.336+0000 7f883d1a3700 1 --2- 172.21.15.7:0/2290882877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7bf95fe0 0x55ed7bfb1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:20.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.336+0000 7f8837fff700 1 --2- 172.21.15.7:0/2290882877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed7bf956c0 0x55ed7bfb1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:20.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.336+0000 7f883c9a2700 1 -- 172.21.15.7:0/2290882877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed7bf956c0 msgr2=0x55ed7bfb1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:20.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.336+0000 7f883c9a2700 1 --2- 172.21.15.7:0/2290882877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed7bf956c0 0x55ed7bfb1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.336+0000 7f883c9a2700 1 -- 172.21.15.7:0/2290882877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7bf95fe0 msgr2=0x55ed7bfb1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:20.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.336+0000 7f883c9a2700 1 --2- 172.21.15.7:0/2290882877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7bf95fe0 0x55ed7bfb1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.336+0000 7f883c9a2700 1 -- 172.21.15.7:0/2290882877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed7c013580 con 0x55ed7bf8ffa0 2024-06-10T01:10:20.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.336+0000 7f883c9a2700 1 --2- 172.21.15.7:0/2290882877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed7bf8ffa0 0x55ed7bfb1210 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f882000c010 tx=0x7f8820009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a5147265319b33 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:20.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.336+0000 7f8817fff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:10:20.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.336+0000 7f8817fff700 1 -- 172.21.15.7:0/2290882877 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882009ce20 con 0x55ed7bf8ffa0 2024-06-10T01:10:20.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.336+0000 7f8849489b80 1 -- 172.21.15.7:0/2290882877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88200095a0 con 0x55ed7bf8ffa0 2024-06-10T01:10:20.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.336+0000 7f8817fff700 1 -- 172.21.15.7:0/2290882877 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f882000f460 con 0x55ed7bf8ffa0 2024-06-10T01:10:20.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.337+0000 7f8817fff700 1 -- 172.21.15.7:0/2290882877 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f88200a0ad0 con 0x55ed7bf8ffa0 2024-06-10T01:10:20.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.337+0000 7f8849489b80 10 client.16614 fetch_fsmap learned FSMap version 30 2024-06-10T01:10:20.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.337+0000 7f8849489b80 1 -- 172.21.15.7:0/2290882877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ed7c0148c0 con 0x55ed7bf8ffa0 2024-06-10T01:10:20.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.337+0000 7f8817fff700 1 -- 172.21.15.7:0/2290882877 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88200095a0 con 0x55ed7bf8ffa0 2024-06-10T01:10:20.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.337+0000 7f8817fff700 1 -- 172.21.15.7:0/2290882877 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f88200b0bb0 con 0x55ed7bf8ffa0 2024-06-10T01:10:20.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.337+0000 7f8849489b80 10 client.16614 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:10:20.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.337+0000 7f8849489b80 10 client.16614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:10:20.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.337+0000 7f8849489b80 20 client.16614 populate_metadata read hostname 'smithi007' 2024-06-10T01:10:20.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.337+0000 7f8849489b80 1 --2- 172.21.15.7:0/2290882877 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ed7c014b90 0x55ed7c016f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:20.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.337+0000 7f8849489b80 4 client.16614 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:10:20.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.337+0000 7f8849489b80 1 -- 172.21.15.7:0/2290882877 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ed7bee9310 con 0x55ed7c014b90 2024-06-10T01:10:20.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.337+0000 7f8837fff700 1 --2- 172.21.15.7:0/2290882877 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ed7c014b90 0x55ed7c016f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:20.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.338+0000 7f8837fff700 1 --2- 172.21.15.7:0/2290882877 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ed7c014b90 0x55ed7c016f80 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e46b231bd9f2f4e2 server_cookie=c44fb081065e6225 in_seq=0 out_seq=0 2024-06-10T01:10:20.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.338+0000 7f8817fff700 10 client.16614 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:20.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.339+0000 7f8817fff700 1 -- 172.21.15.7:0/2290882877 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ed7bee9310 con 0x55ed7c014b90 2024-06-10T01:10:20.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.339+0000 7f8817fff700 10 client.16614 handle_command_reply: tid=0 2024-06-10T01:10:20.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.339+0000 7f8849489b80 1 client.16614 shutdown 2024-06-10T01:10:20.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.339+0000 7f8849489b80 1 -- 172.21.15.7:0/2290882877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed7bf8ffa0 msgr2=0x55ed7bfb1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:20.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.339+0000 7f8849489b80 1 --2- 172.21.15.7:0/2290882877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed7bf8ffa0 0x55ed7bfb1210 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f882000c010 tx=0x7f8820009350 comp rx=0 tx=0).stop 2024-06-10T01:10:20.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.339+0000 7f8849489b80 1 -- 172.21.15.7:0/2290882877 shutdown_connections 2024-06-10T01:10:20.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.339+0000 7f8849489b80 1 --2- 172.21.15.7:0/2290882877 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ed7c014b90 0x55ed7c016f80 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.339+0000 7f8817fff700 0 client.16614 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:20.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.339+0000 7f8849489b80 1 --2- 172.21.15.7:0/2290882877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7bf95fe0 0x55ed7bfb1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.339+0000 7f8849489b80 1 --2- 172.21.15.7:0/2290882877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed7bf8ffa0 0x55ed7bfb1210 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.339+0000 7f8849489b80 1 --2- 172.21.15.7:0/2290882877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed7bf956c0 0x55ed7bfb1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.339+0000 7f8849489b80 1 -- 172.21.15.7:0/2290882877 >> 172.21.15.7:0/2290882877 conn(0x55ed7bebad40 msgr2=0x55ed7bf963c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:20.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.339+0000 7f8849489b80 1 -- 172.21.15.7:0/2290882877 shutdown_connections 2024-06-10T01:10:20.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.339+0000 7f8849489b80 1 -- 172.21.15.7:0/2290882877 wait complete. 2024-06-10T01:10:20.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.340+0000 7f8849489b80 20 client.16614 trim_cache size 0 max 16384 2024-06-10T01:10:20.339 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:10:20.339 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-10T01:10:20.340 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "38a67e87-c1f0-4060-8281-6b1d9d70af9a", 2024-06-10T01:10:20.340 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-10T01:10:20.340 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:10:20.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.340+0000 7f883ec06700 1 -- 172.21.15.7:0/213952076 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f88240a2ca0 msgr2=0x7f88240a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:20.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.340+0000 7f883ec06700 1 --2- 172.21.15.7:0/213952076 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f88240a2ca0 0x7f88240a5160 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8838106490 tx=0x7f88280091b0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.340+0000 7f883ec06700 1 -- 172.21.15.7:0/213952076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8838100e50 msgr2=0x7f8838102030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:20.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.340+0000 7f883ec06700 1 --2- 172.21.15.7:0/213952076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8838100e50 0x7f8838102030 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8820000c00 tx=0x7f882000bab0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.340+0000 7f883ec06700 1 -- 172.21.15.7:0/213952076 shutdown_connections 2024-06-10T01:10:20.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.340+0000 7f883ec06700 1 --2- 172.21.15.7:0/213952076 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f88240a2ca0 0x7f88240a5160 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.340+0000 7f883ec06700 1 --2- 172.21.15.7:0/213952076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8838107570 0x7f8838102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.340+0000 7f883ec06700 1 --2- 172.21.15.7:0/213952076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8838100e50 0x7f8838102030 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.340+0000 7f883ec06700 1 --2- 172.21.15.7:0/213952076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8838101810 0x7f8838102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:20.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.340+0000 7f883ec06700 1 -- 172.21.15.7:0/213952076 >> 172.21.15.7:0/213952076 conn(0x7f88380766b0 msgr2=0x7f88380ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:20.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.340+0000 7f883ec06700 1 -- 172.21.15.7:0/213952076 shutdown_connections 2024-06-10T01:10:20.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:20.340+0000 7f883ec06700 1 -- 172.21.15.7:0/213952076 wait complete. 2024-06-10T01:10:20.716 INFO:tasks.workunit.client.0.smithi007.stdout: 1 266292 8.15 MB/sec execute 61 sec latency 20.202 ms 2024-06-10T01:10:21.146 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:10:21.459 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:10:21.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:21 smithi007 ceph-mon[32398]: pgmap v562: 129 pgs: 129 active+clean; 55 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.8 MiB/s wr, 106 op/s 2024-06-10T01:10:21.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:21 smithi007 ceph-mon[32398]: scrub queued for path: / 2024-06-10T01:10:21.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:21 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:10:21.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:21 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:10:21.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:21 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:10:21.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:21 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:10:21.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:21 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:10:21.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:21 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:10:21.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:21 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:10:21.579 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:21 smithi078 ceph-mon[36683]: pgmap v562: 129 pgs: 129 active+clean; 55 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.8 MiB/s wr, 106 op/s 2024-06-10T01:10:21.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:21 smithi078 ceph-mon[36683]: scrub queued for path: / 2024-06-10T01:10:21.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:21 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:10:21.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:21 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:10:21.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:21 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:10:21.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:21 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:10:21.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:21 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:10:21.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:21 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:10:21.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:21 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:10:21.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:21 smithi175 ceph-mon[36636]: pgmap v562: 129 pgs: 129 active+clean; 55 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.8 MiB/s wr, 106 op/s 2024-06-10T01:10:21.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:21 smithi175 ceph-mon[36636]: scrub queued for path: / 2024-06-10T01:10:21.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:21 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:10:21.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:21 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:10:21.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:21 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:10:21.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:21 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:10:21.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:21 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:10:21.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:21 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:10:21.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:21 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:10:21.716 INFO:tasks.workunit.client.0.smithi007.stdout: 1 267930 8.11 MB/sec execute 62 sec latency 11.215 ms 2024-06-10T01:10:22.717 INFO:tasks.workunit.client.0.smithi007.stdout: 1 270013 8.20 MB/sec execute 63 sec latency 46.943 ms 2024-06-10T01:10:23.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.285+0000 7f7b33af2700 1 -- 172.21.15.7:0/2583585928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b2c06a170 msgr2=0x7f7b2c06a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:23.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.285+0000 7f7b33af2700 1 --2- 172.21.15.7:0/2583585928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b2c06a170 0x7f7b2c06a5f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c009990 tx=0x7f7b1c009ca0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.285+0000 7f7b33af2700 1 -- 172.21.15.7:0/2583585928 shutdown_connections 2024-06-10T01:10:23.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.285+0000 7f7b33af2700 1 --2- 172.21.15.7:0/2583585928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b2c06ab30 0x7f7b2c10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.285+0000 7f7b33af2700 1 --2- 172.21.15.7:0/2583585928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b2c108500 0x7f7b2c1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.285+0000 7f7b33af2700 1 --2- 172.21.15.7:0/2583585928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b2c06a170 0x7f7b2c06a5f0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.285+0000 7f7b33af2700 1 -- 172.21.15.7:0/2583585928 >> 172.21.15.7:0/2583585928 conn(0x7f7b2c078340 msgr2=0x7f7b2c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:23.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.285+0000 7f7b33af2700 1 -- 172.21.15.7:0/2583585928 shutdown_connections 2024-06-10T01:10:23.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.285+0000 7f7b33af2700 1 -- 172.21.15.7:0/2583585928 wait complete. 2024-06-10T01:10:23.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.285+0000 7f7b33af2700 1 Processor -- start 2024-06-10T01:10:23.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.286+0000 7f7b33af2700 1 -- start start 2024-06-10T01:10:23.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.286+0000 7f7b33af2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b2c06a170 0x7f7b2c19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:23.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.286+0000 7f7b33af2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b2c06ab30 0x7f7b2c19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:23.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.286+0000 7f7b33af2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b2c108500 0x7f7b2c198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:23.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.286+0000 7f7b33af2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c112760 con 0x7f7b2c06a170 2024-06-10T01:10:23.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.286+0000 7f7b33af2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c1125e0 con 0x7f7b2c06ab30 2024-06-10T01:10:23.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.286+0000 7f7b33af2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c1128e0 con 0x7f7b2c108500 2024-06-10T01:10:23.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.286+0000 7f7b3188e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b2c06a170 0x7f7b2c19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:23.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.286+0000 7f7b3108d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b2c06ab30 0x7f7b2c19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:23.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.286+0000 7f7b3188e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b2c06a170 0x7f7b2c19e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57540/0 (socket says 172.21.15.7:57540) 2024-06-10T01:10:23.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.286+0000 7f7b3188e700 1 -- 172.21.15.7:0/3221550476 learned_addr learned my addr 172.21.15.7:0/3221550476 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:23.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.286+0000 7f7b3208f700 1 --2- 172.21.15.7:0/3221550476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b2c108500 0x7f7b2c198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:23.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.286+0000 7f7b3108d700 1 -- 172.21.15.7:0/3221550476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b2c108500 msgr2=0x7f7b2c198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:23.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.286+0000 7f7b3108d700 1 --2- 172.21.15.7:0/3221550476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b2c108500 0x7f7b2c198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.286+0000 7f7b3108d700 1 -- 172.21.15.7:0/3221550476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b2c06a170 msgr2=0x7f7b2c19e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:23.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.286+0000 7f7b3108d700 1 --2- 172.21.15.7:0/3221550476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b2c06a170 0x7f7b2c19e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.286+0000 7f7b3108d700 1 -- 172.21.15.7:0/3221550476 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b2c199180 con 0x7f7b2c06ab30 2024-06-10T01:10:23.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.287+0000 7f7b3108d700 1 --2- 172.21.15.7:0/3221550476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b2c06ab30 0x7f7b2c19eda0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c0059d0 tx=0x7f7b1c00bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:23.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.287+0000 7f7b22ffd700 1 -- 172.21.15.7:0/3221550476 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c027070 con 0x7f7b2c06ab30 2024-06-10T01:10:23.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.287+0000 7f7b33af2700 1 -- 172.21.15.7:0/3221550476 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b2c199410 con 0x7f7b2c06ab30 2024-06-10T01:10:23.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.287+0000 7f7b22ffd700 1 -- 172.21.15.7:0/3221550476 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b1c003d70 con 0x7f7b2c06ab30 2024-06-10T01:10:23.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.287+0000 7f7b22ffd700 1 -- 172.21.15.7:0/3221550476 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c022e30 con 0x7f7b2c06ab30 2024-06-10T01:10:23.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.287+0000 7f7b33af2700 1 -- 172.21.15.7:0/3221550476 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b2c1a5730 con 0x7f7b2c06ab30 2024-06-10T01:10:23.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.290+0000 7f7b22ffd700 1 -- 172.21.15.7:0/3221550476 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f7b1c02a070 con 0x7f7b2c06ab30 2024-06-10T01:10:23.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.291+0000 7f7b22ffd700 1 --2- 172.21.15.7:0/3221550476 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7b180a2c20 0x7f7b180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:23.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.291+0000 7f7b3188e700 1 --2- 172.21.15.7:0/3221550476 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7b180a2c20 0x7f7b180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:23.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.291+0000 7f7b22ffd700 1 -- 172.21.15.7:0/3221550476 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f7b1c0d0120 con 0x7f7b2c06ab30 2024-06-10T01:10:23.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.291+0000 7f7b3e375b80 1 Processor -- start 2024-06-10T01:10:23.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.291+0000 7f7b3188e700 1 --2- 172.21.15.7:0/3221550476 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7b180a2c20 0x7f7b180a50e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c199b60 tx=0x7f7b28009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:23.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.292+0000 7f7b3e375b80 1 -- start start 2024-06-10T01:10:23.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.292+0000 7f7b3e375b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56456dd9fa70 0x56456dd9fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:23.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.292+0000 7f7b3e375b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56456dda0390 0x56456dd99df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:23.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.292+0000 7f7b3e375b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56456dd9a330 0x56456dd9a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:23.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.292+0000 7f7b3e375b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56456dce7810 con 0x56456dd9fa70 2024-06-10T01:10:23.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.292+0000 7f7b3e375b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56456dc8f720 con 0x56456dd9a330 2024-06-10T01:10:23.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.292+0000 7f7b3e375b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56456dce37d0 con 0x56456dda0390 2024-06-10T01:10:23.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.292+0000 7f7b3188e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56456dd9fa70 0x56456dd9fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:23.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.292+0000 7f7b3188e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56456dd9fa70 0x56456dd9fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57556/0 (socket says 172.21.15.7:57556) 2024-06-10T01:10:23.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.292+0000 7f7b3188e700 1 -- 172.21.15.7:0/1630704413 learned_addr learned my addr 172.21.15.7:0/1630704413 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:23.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.292+0000 7f7b3208f700 1 --2- 172.21.15.7:0/1630704413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56456dd9a330 0x56456dd9a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:23.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.292+0000 7f7b3108d700 1 --2- 172.21.15.7:0/1630704413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56456dda0390 0x56456dd99df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:23.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.292+0000 7f7b3188e700 1 -- 172.21.15.7:0/1630704413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56456dda0390 msgr2=0x56456dd99df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:23.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.292+0000 7f7b3188e700 1 --2- 172.21.15.7:0/1630704413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56456dda0390 0x56456dd99df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.292+0000 7f7b3188e700 1 -- 172.21.15.7:0/1630704413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56456dd9a330 msgr2=0x56456dd9a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:23.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.292+0000 7f7b3188e700 1 --2- 172.21.15.7:0/1630704413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56456dd9a330 0x56456dd9a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.292+0000 7f7b3188e700 1 -- 172.21.15.7:0/1630704413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56456dc85f50 con 0x56456dd9fa70 2024-06-10T01:10:23.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.292+0000 7f7b3208f700 1 --2- 172.21.15.7:0/1630704413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56456dd9a330 0x56456dd9a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:23.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.293+0000 7f7b3188e700 1 --2- 172.21.15.7:0/1630704413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56456dd9fa70 0x56456dd9fe50 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800e570 tx=0x7f7b28014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c53b593a62fdd0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:23.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.293+0000 7f7b17fff700 1 -- 172.21.15.7:0/1630704413 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2800ed10 con 0x56456dd9fa70 2024-06-10T01:10:23.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.293+0000 7f7b17fff700 1 -- 172.21.15.7:0/1630704413 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b2801e900 con 0x56456dd9fa70 2024-06-10T01:10:23.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.293+0000 7f7b17fff700 1 -- 172.21.15.7:0/1630704413 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28026900 con 0x56456dd9fa70 2024-06-10T01:10:23.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.294+0000 7f7b3e375b80 1 -- 172.21.15.7:0/1630704413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56456dd9fa70 msgr2=0x56456dd9fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:23.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.294+0000 7f7b3e375b80 1 --2- 172.21.15.7:0/1630704413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56456dd9fa70 0x56456dd9fe50 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800e570 tx=0x7f7b28014000 comp rx=0 tx=0).stop 2024-06-10T01:10:23.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.294+0000 7f7b3e375b80 1 -- 172.21.15.7:0/1630704413 shutdown_connections 2024-06-10T01:10:23.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.294+0000 7f7b3e375b80 1 --2- 172.21.15.7:0/1630704413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56456dd9a330 0x56456dd9a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.294+0000 7f7b3e375b80 1 --2- 172.21.15.7:0/1630704413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56456dd9fa70 0x56456dd9fe50 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.294+0000 7f7b3e375b80 1 --2- 172.21.15.7:0/1630704413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56456dda0390 0x56456dd99df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.294+0000 7f7b3e375b80 1 -- 172.21.15.7:0/1630704413 >> 172.21.15.7:0/1630704413 conn(0x56456dcc0cc0 msgr2=0x56456dd90850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:23.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.294+0000 7f7b3e375b80 1 -- 172.21.15.7:0/1630704413 shutdown_connections 2024-06-10T01:10:23.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.294+0000 7f7b3e375b80 1 -- 172.21.15.7:0/1630704413 wait complete. 2024-06-10T01:10:23.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.294+0000 7f7b3e375b80 1 -- start start 2024-06-10T01:10:23.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.294+0000 7f7b3e375b80 1 Processor -- start 2024-06-10T01:10:23.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.295+0000 7f7b3e375b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56456dd9a330 0x56456dda6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:23.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.295+0000 7f7b3e375b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56456dd9fa70 0x56456dda3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:23.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.295+0000 7f7b3e375b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56456dda0390 0x56456dda3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:23.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.295+0000 7f7b3e375b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56456dda6dd0 con 0x56456dd9fa70 2024-06-10T01:10:23.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.295+0000 7f7b3e375b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56456dc87bf0 con 0x56456dda0390 2024-06-10T01:10:23.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.295+0000 7f7b3e375b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56456dda6f50 con 0x56456dd9a330 2024-06-10T01:10:23.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.295+0000 7f7b3108d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56456dd9fa70 0x56456dda3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:23.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.295+0000 7f7b3e375b80 10 client.0 _ll_register_callbacks cb 0x56456dc81920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:23.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.295+0000 7f7b3188e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56456dd9a330 0x56456dda6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:23.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.295+0000 7f7b3108d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56456dd9fa70 0x56456dda3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57570/0 (socket says 172.21.15.7:57570) 2024-06-10T01:10:23.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.295+0000 7f7b3208f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56456dda0390 0x56456dda3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:23.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.295+0000 7f7b3108d700 1 -- 172.21.15.7:0/3889636243 learned_addr learned my addr 172.21.15.7:0/3889636243 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:23.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.295+0000 7f7b3108d700 1 -- 172.21.15.7:0/3889636243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56456dd9a330 msgr2=0x56456dda6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:23.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.295+0000 7f7b3108d700 1 --2- 172.21.15.7:0/3889636243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56456dd9a330 0x56456dda6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.295+0000 7f7b3108d700 1 -- 172.21.15.7:0/3889636243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56456dda0390 msgr2=0x56456dda3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:23.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.295+0000 7f7b3108d700 1 --2- 172.21.15.7:0/3889636243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56456dda0390 0x56456dda3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.295+0000 7f7b3108d700 1 -- 172.21.15.7:0/3889636243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56456dc368d0 con 0x56456dd9fa70 2024-06-10T01:10:23.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.295+0000 7f7b3188e700 1 --2- 172.21.15.7:0/3889636243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56456dd9a330 0x56456dda6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:23.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.295+0000 7f7b3108d700 1 --2- 172.21.15.7:0/3889636243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56456dd9fa70 0x56456dda3500 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c0a3110 tx=0x7f7b1c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a207dc28dd8d2a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:23.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.295+0000 7f7b157fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:10:23.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.295+0000 7f7b157fa700 1 -- 172.21.15.7:0/3889636243 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c09bff0 con 0x56456dd9fa70 2024-06-10T01:10:23.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.295+0000 7f7b157fa700 1 -- 172.21.15.7:0/3889636243 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f7b1c0b5510 con 0x56456dd9fa70 2024-06-10T01:10:23.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.296+0000 7f7b3e375b80 1 -- 172.21.15.7:0/3889636243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b2801e900 con 0x56456dd9fa70 2024-06-10T01:10:23.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.296+0000 7f7b157fa700 1 -- 172.21.15.7:0/3889636243 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7b1c0095d0 con 0x56456dd9fa70 2024-06-10T01:10:23.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.296+0000 7f7b3e375b80 10 client.16626 fetch_fsmap learned FSMap version 30 2024-06-10T01:10:23.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.296+0000 7f7b3e375b80 1 -- 172.21.15.7:0/3889636243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56456ddb17e0 con 0x56456dd9fa70 2024-06-10T01:10:23.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.296+0000 7f7b157fa700 1 -- 172.21.15.7:0/3889636243 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b2801e900 con 0x56456dd9fa70 2024-06-10T01:10:23.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.296+0000 7f7b157fa700 1 -- 172.21.15.7:0/3889636243 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7b1c0b50d0 con 0x56456dd9fa70 2024-06-10T01:10:23.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.296+0000 7f7b3e375b80 10 client.16626 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:10:23.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.296+0000 7f7b3e375b80 10 client.16626 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:10:23.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.296+0000 7f7b3e375b80 20 client.16626 populate_metadata read hostname 'smithi007' 2024-06-10T01:10:23.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.296+0000 7f7b3e375b80 1 --2- 172.21.15.7:0/3889636243 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56456ddb3cb0 0x56456ddb60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:23.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.296+0000 7f7b3e375b80 4 client.16626 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:10:23.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.296+0000 7f7b3e375b80 1 -- 172.21.15.7:0/3889636243 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56456dcf4d10 con 0x56456ddb3cb0 2024-06-10T01:10:23.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.296+0000 7f7b3188e700 1 --2- 172.21.15.7:0/3889636243 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56456ddb3cb0 0x56456ddb60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:23.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.297+0000 7f7b3188e700 1 --2- 172.21.15.7:0/3889636243 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56456ddb3cb0 0x56456ddb60a0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4562957080f911ca server_cookie=f64620fa932986a9 in_seq=0 out_seq=0 2024-06-10T01:10:23.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.297+0000 7f7b157fa700 10 client.16626 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:23.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.298+0000 7f7b157fa700 1 -- 172.21.15.7:0/3889636243 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56456dcf4d10 con 0x56456ddb3cb0 2024-06-10T01:10:23.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.298+0000 7f7b157fa700 10 client.16626 handle_command_reply: tid=0 2024-06-10T01:10:23.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.298+0000 7f7b3e375b80 1 client.16626 shutdown 2024-06-10T01:10:23.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.298+0000 7f7b3e375b80 1 -- 172.21.15.7:0/3889636243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56456dd9fa70 msgr2=0x56456dda3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:23.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.298+0000 7f7b3e375b80 1 --2- 172.21.15.7:0/3889636243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56456dd9fa70 0x56456dda3500 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c0a3110 tx=0x7f7b1c009ca0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.298+0000 7f7b3e375b80 1 -- 172.21.15.7:0/3889636243 shutdown_connections 2024-06-10T01:10:23.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.298+0000 7f7b3e375b80 1 --2- 172.21.15.7:0/3889636243 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56456ddb3cb0 0x56456ddb60a0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.298+0000 7f7b3e375b80 1 --2- 172.21.15.7:0/3889636243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56456dda0390 0x56456dda3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.298+0000 7f7b3e375b80 1 --2- 172.21.15.7:0/3889636243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56456dd9a330 0x56456dda6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.298+0000 7f7b3e375b80 1 --2- 172.21.15.7:0/3889636243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56456dd9fa70 0x56456dda3500 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.298+0000 7f7b157fa700 0 client.16626 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:23.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.298+0000 7f7b3e375b80 1 -- 172.21.15.7:0/3889636243 >> 172.21.15.7:0/3889636243 conn(0x56456dcc0cc0 msgr2=0x56456dd949d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:23.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.298+0000 7f7b3e375b80 1 -- 172.21.15.7:0/3889636243 shutdown_connections 2024-06-10T01:10:23.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.298+0000 7f7b3e375b80 1 -- 172.21.15.7:0/3889636243 wait complete. 2024-06-10T01:10:23.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.298+0000 7f7b3e375b80 20 client.16626 trim_cache size 0 max 16384 2024-06-10T01:10:23.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.321+0000 7f7b3e375b80 1 Processor -- start 2024-06-10T01:10:23.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.321+0000 7f7b3e375b80 1 -- start start 2024-06-10T01:10:23.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.321+0000 7f7b3e375b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56456dd9a330 0x56456dd9d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:23.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.321+0000 7f7b3e375b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56456dd9fa70 0x56456dd9d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:23.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.321+0000 7f7b3e375b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56456dda0390 0x56456dd949d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:23.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.321+0000 7f7b3e375b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c1ae8a0 con 0x56456dd9a330 2024-06-10T01:10:23.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.321+0000 7f7b3e375b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c1ac0d0 con 0x56456dda0390 2024-06-10T01:10:23.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.321+0000 7f7b3e375b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c1a9f20 con 0x56456dd9fa70 2024-06-10T01:10:23.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.321+0000 7f7b3188e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56456dd9a330 0x56456dd9d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:23.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.321+0000 7f7b3188e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56456dd9a330 0x56456dd9d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57574/0 (socket says 172.21.15.7:57574) 2024-06-10T01:10:23.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.321+0000 7f7b3208f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56456dda0390 0x56456dd949d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:23.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.321+0000 7f7b3188e700 1 -- 172.21.15.7:0/744484465 learned_addr learned my addr 172.21.15.7:0/744484465 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:23.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.321+0000 7f7b3108d700 1 --2- 172.21.15.7:0/744484465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56456dd9fa70 0x56456dd9d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:23.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.321+0000 7f7b3188e700 1 -- 172.21.15.7:0/744484465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56456dd9fa70 msgr2=0x56456dd9d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:23.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.321+0000 7f7b3188e700 1 --2- 172.21.15.7:0/744484465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56456dd9fa70 0x56456dd9d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.321+0000 7f7b3188e700 1 -- 172.21.15.7:0/744484465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56456dda0390 msgr2=0x56456dd949d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:23.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.321+0000 7f7b3188e700 1 --2- 172.21.15.7:0/744484465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56456dda0390 0x56456dd949d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.321+0000 7f7b3188e700 1 -- 172.21.15.7:0/744484465 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56456dd9df30 con 0x56456dd9a330 2024-06-10T01:10:23.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.321+0000 7f7b3208f700 1 --2- 172.21.15.7:0/744484465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56456dda0390 0x56456dd949d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:23.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.322+0000 7f7b3188e700 1 --2- 172.21.15.7:0/744484465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56456dd9a330 0x56456dd9d4b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800eb60 tx=0x7f7b280264a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be288a56b6855265 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:23.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.322+0000 7f7b177fe700 1 -- 172.21.15.7:0/744484465 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28005450 con 0x56456dd9a330 2024-06-10T01:10:23.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.322+0000 7f7b177fe700 1 -- 172.21.15.7:0/744484465 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b280055f0 con 0x56456dd9a330 2024-06-10T01:10:23.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.322+0000 7f7b177fe700 1 -- 172.21.15.7:0/744484465 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28004930 con 0x56456dd9a330 2024-06-10T01:10:23.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.323+0000 7f7b3e375b80 1 -- 172.21.15.7:0/744484465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56456dd9a330 msgr2=0x56456dd9d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:23.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.323+0000 7f7b3e375b80 1 --2- 172.21.15.7:0/744484465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56456dd9a330 0x56456dd9d4b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800eb60 tx=0x7f7b280264a0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.323+0000 7f7b3e375b80 1 -- 172.21.15.7:0/744484465 shutdown_connections 2024-06-10T01:10:23.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.323+0000 7f7b3e375b80 1 --2- 172.21.15.7:0/744484465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56456dda0390 0x56456dd949d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.323+0000 7f7b3e375b80 1 --2- 172.21.15.7:0/744484465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56456dd9a330 0x56456dd9d4b0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.323+0000 7f7b3e375b80 1 --2- 172.21.15.7:0/744484465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56456dd9fa70 0x56456dd9d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.323+0000 7f7b3e375b80 1 -- 172.21.15.7:0/744484465 >> 172.21.15.7:0/744484465 conn(0x56456dcc0cc0 msgr2=0x56456dda34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:23.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.323+0000 7f7b3e375b80 1 -- 172.21.15.7:0/744484465 shutdown_connections 2024-06-10T01:10:23.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.323+0000 7f7b3e375b80 1 -- 172.21.15.7:0/744484465 wait complete. 2024-06-10T01:10:23.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.323+0000 7f7b3e375b80 1 -- start start 2024-06-10T01:10:23.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.323+0000 7f7b3e375b80 1 Processor -- start 2024-06-10T01:10:23.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.324+0000 7f7b3e375b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56456dd9a330 0x56456ddbb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:23.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.324+0000 7f7b3e375b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56456dd9fa70 0x56456ddbbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:23.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.324+0000 7f7b3e375b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56456dda0390 0x56456ddbc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:23.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.324+0000 7f7b3e375b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c1aa590 con 0x56456dd9a330 2024-06-10T01:10:23.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.324+0000 7f7b3e375b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c1aba00 con 0x56456dd9fa70 2024-06-10T01:10:23.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.324+0000 7f7b3e375b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c1aac60 con 0x56456dda0390 2024-06-10T01:10:23.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.324+0000 7f7b3e375b80 10 client.0 _ll_register_callbacks cb 0x56456dc81920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:23.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.324+0000 7f7b3188e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56456dd9a330 0x56456ddbb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:23.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.324+0000 7f7b3108d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56456dd9fa70 0x56456ddbbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:23.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.324+0000 7f7b3188e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56456dd9a330 0x56456ddbb710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57580/0 (socket says 172.21.15.7:57580) 2024-06-10T01:10:23.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.324+0000 7f7b3188e700 1 -- 172.21.15.7:0/478143144 learned_addr learned my addr 172.21.15.7:0/478143144 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:23.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.324+0000 7f7b3208f700 1 --2- 172.21.15.7:0/478143144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56456dda0390 0x56456ddbc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:23.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.324+0000 7f7b3188e700 1 -- 172.21.15.7:0/478143144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56456dda0390 msgr2=0x56456ddbc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:23.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.324+0000 7f7b3188e700 1 --2- 172.21.15.7:0/478143144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56456dda0390 0x56456ddbc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.324+0000 7f7b3188e700 1 -- 172.21.15.7:0/478143144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56456dd9fa70 msgr2=0x56456ddbbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:23.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.324+0000 7f7b3188e700 1 --2- 172.21.15.7:0/478143144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56456dd9fa70 0x56456ddbbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.324+0000 7f7b3188e700 1 -- 172.21.15.7:0/478143144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56456de1dac0 con 0x56456dd9a330 2024-06-10T01:10:23.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.324+0000 7f7b3108d700 1 --2- 172.21.15.7:0/478143144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56456dd9fa70 0x56456ddbbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:23.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.325+0000 7f7b3188e700 1 --2- 172.21.15.7:0/478143144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56456dd9a330 0x56456ddbb710 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f7b28013010 tx=0x7f7b28009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7142a245564e3503 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:23.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.325+0000 7f7b157fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:10:23.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.325+0000 7f7b157fa700 1 -- 172.21.15.7:0/478143144 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28005450 con 0x56456dd9a330 2024-06-10T01:10:23.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.325+0000 7f7b3e375b80 1 -- 172.21.15.7:0/478143144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b280055f0 con 0x56456dd9a330 2024-06-10T01:10:23.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.325+0000 7f7b157fa700 1 -- 172.21.15.7:0/478143144 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f7b2801e730 con 0x56456dd9a330 2024-06-10T01:10:23.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.325+0000 7f7b157fa700 1 -- 172.21.15.7:0/478143144 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7b28011ab0 con 0x56456dd9a330 2024-06-10T01:10:23.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.325+0000 7f7b3e375b80 10 client.16632 fetch_fsmap learned FSMap version 30 2024-06-10T01:10:23.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.325+0000 7f7b3e375b80 1 -- 172.21.15.7:0/478143144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56456de1edb0 con 0x56456dd9a330 2024-06-10T01:10:23.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.325+0000 7f7b157fa700 1 -- 172.21.15.7:0/478143144 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b280055f0 con 0x56456dd9a330 2024-06-10T01:10:23.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.325+0000 7f7b157fa700 1 -- 172.21.15.7:0/478143144 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7b280046d0 con 0x56456dd9a330 2024-06-10T01:10:23.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.325+0000 7f7b3e375b80 10 client.16632 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:10:23.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.325+0000 7f7b3e375b80 10 client.16632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:10:23.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.325+0000 7f7b3e375b80 20 client.16632 populate_metadata read hostname 'smithi007' 2024-06-10T01:10:23.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.325+0000 7f7b3e375b80 1 --2- 172.21.15.7:0/478143144 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56456de1f240 0x56456de21660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:23.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.325+0000 7f7b3e375b80 4 client.16632 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:10:23.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.325+0000 7f7b3e375b80 1 -- 172.21.15.7:0/478143144 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56456dcef290 con 0x56456de1f240 2024-06-10T01:10:23.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.326+0000 7f7b3108d700 1 --2- 172.21.15.7:0/478143144 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56456de1f240 0x56456de21660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:23.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.326+0000 7f7b3108d700 1 --2- 172.21.15.7:0/478143144 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56456de1f240 0x56456de21660 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6021103fb4e0dbb4 server_cookie=f0b3f68d2b3334cb in_seq=0 out_seq=0 2024-06-10T01:10:23.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.326+0000 7f7b157fa700 10 client.16632 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:23.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.327+0000 7f7b157fa700 1 -- 172.21.15.7:0/478143144 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56456dcef290 con 0x56456de1f240 2024-06-10T01:10:23.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.327+0000 7f7b157fa700 10 client.16632 handle_command_reply: tid=0 2024-06-10T01:10:23.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.327+0000 7f7b3e375b80 1 client.16632 shutdown 2024-06-10T01:10:23.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.327+0000 7f7b3e375b80 1 -- 172.21.15.7:0/478143144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56456dd9a330 msgr2=0x56456ddbb710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:23.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.327+0000 7f7b3e375b80 1 --2- 172.21.15.7:0/478143144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56456dd9a330 0x56456ddbb710 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f7b28013010 tx=0x7f7b28009d50 comp rx=0 tx=0).stop 2024-06-10T01:10:23.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.327+0000 7f7b3e375b80 1 -- 172.21.15.7:0/478143144 shutdown_connections 2024-06-10T01:10:23.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.327+0000 7f7b3e375b80 1 --2- 172.21.15.7:0/478143144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56456dda0390 0x56456ddbc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.327+0000 7f7b3e375b80 1 --2- 172.21.15.7:0/478143144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56456dd9a330 0x56456ddbb710 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.327+0000 7f7b3e375b80 1 --2- 172.21.15.7:0/478143144 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56456de1f240 0x56456de21660 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.327+0000 7f7b3e375b80 1 --2- 172.21.15.7:0/478143144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56456dd9fa70 0x56456ddbbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.327+0000 7f7b3e375b80 1 -- 172.21.15.7:0/478143144 >> 172.21.15.7:0/478143144 conn(0x56456dcc0cc0 msgr2=0x56456dda34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:23.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.327+0000 7f7b157fa700 0 client.16632 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:23.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.327+0000 7f7b3e375b80 1 -- 172.21.15.7:0/478143144 shutdown_connections 2024-06-10T01:10:23.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.327+0000 7f7b3e375b80 1 -- 172.21.15.7:0/478143144 wait complete. 2024-06-10T01:10:23.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.327+0000 7f7b3e375b80 20 client.16632 trim_cache size 0 max 16384 2024-06-10T01:10:23.327 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:10:23.327 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T01:10:23.327 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-10T01:10:23.327 INFO:teuthology.orchestra.run.smithi007.stdout: "38a67e87-c1f0-4060-8281-6b1d9d70af9a": { 2024-06-10T01:10:23.327 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-10T01:10:23.327 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "38a67e87-c1f0-4060-8281-6b1d9d70af9a", 2024-06-10T01:10:23.327 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-10T01:10:23.327 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:10:23.328 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:10:23.328 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:10:23.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.328+0000 7f7b33af2700 1 -- 172.21.15.7:0/3221550476 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7b180a2c20 msgr2=0x7f7b180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:23.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.328+0000 7f7b33af2700 1 --2- 172.21.15.7:0/3221550476 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7b180a2c20 0x7f7b180a50e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c199b60 tx=0x7f7b28009260 comp rx=0 tx=0).stop 2024-06-10T01:10:23.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.328+0000 7f7b33af2700 1 -- 172.21.15.7:0/3221550476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b2c06ab30 msgr2=0x7f7b2c19eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:23.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.328+0000 7f7b33af2700 1 --2- 172.21.15.7:0/3221550476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b2c06ab30 0x7f7b2c19eda0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c0059d0 tx=0x7f7b1c00bab0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.328+0000 7f7b33af2700 1 -- 172.21.15.7:0/3221550476 shutdown_connections 2024-06-10T01:10:23.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.328+0000 7f7b33af2700 1 --2- 172.21.15.7:0/3221550476 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7b180a2c20 0x7f7b180a50e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.328+0000 7f7b33af2700 1 --2- 172.21.15.7:0/3221550476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b2c108500 0x7f7b2c198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.328+0000 7f7b33af2700 1 --2- 172.21.15.7:0/3221550476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b2c06a170 0x7f7b2c19e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.328+0000 7f7b33af2700 1 --2- 172.21.15.7:0/3221550476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b2c06ab30 0x7f7b2c19eda0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:23.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.328+0000 7f7b33af2700 1 -- 172.21.15.7:0/3221550476 >> 172.21.15.7:0/3221550476 conn(0x7f7b2c078340 msgr2=0x7f7b2c0fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:23.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.328+0000 7f7b33af2700 1 -- 172.21.15.7:0/3221550476 shutdown_connections 2024-06-10T01:10:23.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:23.328+0000 7f7b33af2700 1 -- 172.21.15.7:0/3221550476 wait complete. 2024-06-10T01:10:23.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:23 smithi007 ceph-mon[32398]: pgmap v563: 129 pgs: 129 active+clean; 55 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 1.7 MiB/s wr, 112 op/s 2024-06-10T01:10:23.579 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:23 smithi078 ceph-mon[36683]: pgmap v563: 129 pgs: 129 active+clean; 55 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 1.7 MiB/s wr, 112 op/s 2024-06-10T01:10:23.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:23 smithi175 ceph-mon[36636]: pgmap v563: 129 pgs: 129 active+clean; 55 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 1.7 MiB/s wr, 112 op/s 2024-06-10T01:10:23.716 INFO:tasks.workunit.client.0.smithi007.stdout: 1 272081 8.19 MB/sec execute 64 sec latency 23.533 ms 2024-06-10T01:10:24.716 INFO:tasks.workunit.client.0.smithi007.stdout: 1 273884 8.24 MB/sec execute 65 sec latency 13.207 ms 2024-06-10T01:10:24.962 INFO:tasks.cephfs.filesystem:scrub status for tag:38a67e87-c1f0-4060-8281-6b1d9d70af9a - {'path': '/', 'tag': '38a67e87-c1f0-4060-8281-6b1d9d70af9a', 'options': 'recursive,force'} 2024-06-10T01:10:25.579 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:25 smithi078 ceph-mon[36683]: pgmap v564: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 187 op/s 2024-06-10T01:10:25.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:25 smithi175 ceph-mon[36636]: pgmap v564: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 187 op/s 2024-06-10T01:10:25.716 INFO:tasks.workunit.client.0.smithi007.stdout: 1 276702 8.38 MB/sec execute 66 sec latency 20.564 ms 2024-06-10T01:10:25.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:25 smithi007 ceph-mon[32398]: pgmap v564: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 187 op/s 2024-06-10T01:10:26.716 INFO:tasks.workunit.client.0.smithi007.stdout: 1 278145 8.35 MB/sec execute 67 sec latency 15.298 ms 2024-06-10T01:10:27.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:27 smithi175 ceph-mon[36636]: pgmap v565: 129 pgs: 129 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 227 op/s 2024-06-10T01:10:27.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:27 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:10:27.716 INFO:tasks.workunit.client.0.smithi007.stdout: 1 280510 8.43 MB/sec execute 68 sec latency 48.015 ms 2024-06-10T01:10:27.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:27 smithi007 ceph-mon[32398]: pgmap v565: 129 pgs: 129 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 227 op/s 2024-06-10T01:10:27.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:27 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:10:27.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:27 smithi078 ceph-mon[36683]: pgmap v565: 129 pgs: 129 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 227 op/s 2024-06-10T01:10:27.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:27 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:10:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.092+0000 7fcb40f9e640 1 -- 172.21.15.7:0/698623564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb3c101b00 msgr2=0x7fcb3c101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.092+0000 7fcb40f9e640 1 --2- 172.21.15.7:0/698623564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb3c101b00 0x7fcb3c101f80 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fcb24009900 tx=0x7fcb2402a410 comp rx=0 tx=0).stop 2024-06-10T01:10:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.092+0000 7fcb40f9e640 1 -- 172.21.15.7:0/698623564 shutdown_connections 2024-06-10T01:10:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.092+0000 7fcb40f9e640 1 --2- 172.21.15.7:0/698623564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb3c1024c0 0x7fcb3c108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.092+0000 7fcb40f9e640 1 --2- 172.21.15.7:0/698623564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb3c101b00 0x7fcb3c101f80 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.092+0000 7fcb40f9e640 1 --2- 172.21.15.7:0/698623564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb3c100910 0x7fcb3c100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.092+0000 7fcb40f9e640 1 -- 172.21.15.7:0/698623564 >> 172.21.15.7:0/698623564 conn(0x7fcb3c0fbe80 msgr2=0x7fcb3c0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.092+0000 7fcb40f9e640 1 -- 172.21.15.7:0/698623564 shutdown_connections 2024-06-10T01:10:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.092+0000 7fcb40f9e640 1 -- 172.21.15.7:0/698623564 wait complete. 2024-06-10T01:10:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.092+0000 7fcb40f9e640 1 Processor -- start 2024-06-10T01:10:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.093+0000 7fcb40f9e640 1 -- start start 2024-06-10T01:10:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.093+0000 7fcb40f9e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb3c100910 0x7fcb3c06abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.093+0000 7fcb40f9e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb3c101b00 0x7fcb3c19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.093+0000 7fcb40f9e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb3c1024c0 0x7fcb3c19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.093+0000 7fcb40f9e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c10c590 con 0x7fcb3c101b00 2024-06-10T01:10:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.093+0000 7fcb40f9e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c10c410 con 0x7fcb3c1024c0 2024-06-10T01:10:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.093+0000 7fcb40f9e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c10c710 con 0x7fcb3c100910 2024-06-10T01:10:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.093+0000 7fcb31d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb3c101b00 0x7fcb3c19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.093+0000 7fcb3ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb3c1024c0 0x7fcb3c19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.093+0000 7fcb31d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb3c101b00 0x7fcb3c19ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57602/0 (socket says 172.21.15.7:57602) 2024-06-10T01:10:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.093+0000 7fcb31d9b640 1 -- 172.21.15.7:0/1841690293 learned_addr learned my addr 172.21.15.7:0/1841690293 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.093+0000 7fcb3ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb3c1024c0 0x7fcb3c19f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:55558/0 (socket says 172.21.15.7:55558) 2024-06-10T01:10:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.093+0000 7fcb3a59c640 1 --2- 172.21.15.7:0/1841690293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb3c100910 0x7fcb3c06abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.093+0000 7fcb31d9b640 1 -- 172.21.15.7:0/1841690293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb3c100910 msgr2=0x7fcb3c06abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.093+0000 7fcb31d9b640 1 --2- 172.21.15.7:0/1841690293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb3c100910 0x7fcb3c06abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.093+0000 7fcb31d9b640 1 -- 172.21.15.7:0/1841690293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb3c1024c0 msgr2=0x7fcb3c19f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.093+0000 7fcb31d9b640 1 --2- 172.21.15.7:0/1841690293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb3c1024c0 0x7fcb3c19f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.093+0000 7fcb31d9b640 1 -- 172.21.15.7:0/1841690293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3c19f930 con 0x7fcb3c101b00 2024-06-10T01:10:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.093+0000 7fcb3ad9d640 1 --2- 172.21.15.7:0/1841690293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb3c1024c0 0x7fcb3c19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.094+0000 7fcb31d9b640 1 --2- 172.21.15.7:0/1841690293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb3c101b00 0x7fcb3c19ca30 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fcb24009230 tx=0x7fcb240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.094+0000 7fcb33fff640 1 -- 172.21.15.7:0/1841690293 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2403f030 con 0x7fcb3c101b00 2024-06-10T01:10:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.094+0000 7fcb40f9e640 1 -- 172.21.15.7:0/1841690293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb3c19fbc0 con 0x7fcb3c101b00 2024-06-10T01:10:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.094+0000 7fcb33fff640 1 -- 172.21.15.7:0/1841690293 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb24003ce0 con 0x7fcb3c101b00 2024-06-10T01:10:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.094+0000 7fcb33fff640 1 -- 172.21.15.7:0/1841690293 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb24035070 con 0x7fcb3c101b00 2024-06-10T01:10:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.094+0000 7fcb40f9e640 1 -- 172.21.15.7:0/1841690293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb3c1a00b0 con 0x7fcb3c101b00 2024-06-10T01:10:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.097+0000 7fcb33fff640 1 -- 172.21.15.7:0/1841690293 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fcb24042030 con 0x7fcb3c101b00 2024-06-10T01:10:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.097+0000 7fcb43d89740 1 Processor -- start 2024-06-10T01:10:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.097+0000 7fcb43d89740 1 -- start start 2024-06-10T01:10:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.097+0000 7fcb43d89740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572e52bf010 0x5572e52bf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.097+0000 7fcb43d89740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572e52bfa00 0x5572e52c4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.097+0000 7fcb43d89740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572e52c4b70 0x5572e52c7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.097+0000 7fcb43d89740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5572e52224f0 con 0x5572e52bf010 2024-06-10T01:10:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.097+0000 7fcb43d89740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5572e51c0870 con 0x5572e52c4b70 2024-06-10T01:10:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.097+0000 7fcb43d89740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5572e521e4b0 con 0x5572e52bfa00 2024-06-10T01:10:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.098+0000 7fcb3ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572e52bf010 0x5572e52bf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.098+0000 7fcb3ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572e52bf010 0x5572e52bf430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57606/0 (socket says 172.21.15.7:57606) 2024-06-10T01:10:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.098+0000 7fcb3ad9d640 1 -- 172.21.15.7:0/1015729517 learned_addr learned my addr 172.21.15.7:0/1015729517 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.098+0000 7fcb3a59c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572e52bfa00 0x5572e52c4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.098+0000 7fcb31d9b640 1 --2- 172.21.15.7:0/1015729517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572e52c4b70 0x5572e52c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.098+0000 7fcb3ad9d640 1 -- 172.21.15.7:0/1015729517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572e52bfa00 msgr2=0x5572e52c4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.098+0000 7fcb3ad9d640 1 --2- 172.21.15.7:0/1015729517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572e52bfa00 0x5572e52c4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.098+0000 7fcb3ad9d640 1 -- 172.21.15.7:0/1015729517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572e52c4b70 msgr2=0x5572e52c7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.098+0000 7fcb3ad9d640 1 --2- 172.21.15.7:0/1015729517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572e52c4b70 0x5572e52c7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.098+0000 7fcb3ad9d640 1 -- 172.21.15.7:0/1015729517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572e51dc9b0 con 0x5572e52bf010 2024-06-10T01:10:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.098+0000 7fcb3a59c640 1 --2- 172.21.15.7:0/1015729517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572e52bfa00 0x5572e52c4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.098+0000 7fcb33fff640 1 --2- 172.21.15.7:0/1841690293 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcb280a2b80 0x7fcb280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.098+0000 7fcb3ad9d640 1 --2- 172.21.15.7:0/1015729517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572e52bf010 0x5572e52bf430 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fcb340117e0 tx=0x7fcb34011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2744440a488d7477 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.098+0000 7fcb33fff640 1 -- 172.21.15.7:0/1841690293 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fcb240ef830 con 0x7fcb3c101b00 2024-06-10T01:10:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.098+0000 7fcb3a59c640 1 --2- 172.21.15.7:0/1841690293 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcb280a2b80 0x7fcb280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.098+0000 7fcb30d99640 1 -- 172.21.15.7:0/1015729517 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb34011310 con 0x5572e52bf010 2024-06-10T01:10:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.098+0000 7fcb30d99640 1 -- 172.21.15.7:0/1015729517 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb34003e50 con 0x5572e52bf010 2024-06-10T01:10:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.098+0000 7fcb30d99640 1 -- 172.21.15.7:0/1015729517 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb34004590 con 0x5572e52bf010 2024-06-10T01:10:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.099+0000 7fcb43d89740 1 -- 172.21.15.7:0/1015729517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572e52bf010 msgr2=0x5572e52bf430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.099+0000 7fcb3a59c640 1 --2- 172.21.15.7:0/1841690293 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcb280a2b80 0x7fcb280a5040 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x5572e52bfe00 tx=0x7fcb2c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.099+0000 7fcb43d89740 1 --2- 172.21.15.7:0/1015729517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572e52bf010 0x5572e52bf430 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fcb340117e0 tx=0x7fcb34011ca0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.099+0000 7fcb43d89740 1 -- 172.21.15.7:0/1015729517 shutdown_connections 2024-06-10T01:10:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.099+0000 7fcb43d89740 1 --2- 172.21.15.7:0/1015729517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572e52c4b70 0x5572e52c7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.099+0000 7fcb43d89740 1 --2- 172.21.15.7:0/1015729517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572e52bfa00 0x5572e52c4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.099+0000 7fcb43d89740 1 --2- 172.21.15.7:0/1015729517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572e52bf010 0x5572e52bf430 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.099+0000 7fcb43d89740 1 -- 172.21.15.7:0/1015729517 >> 172.21.15.7:0/1015729517 conn(0x5572e52ba680 msgr2=0x5572e52baa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.099+0000 7fcb43d89740 1 -- 172.21.15.7:0/1015729517 shutdown_connections 2024-06-10T01:10:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.099+0000 7fcb43d89740 1 -- 172.21.15.7:0/1015729517 wait complete. 2024-06-10T01:10:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.100+0000 7fcb43d89740 1 -- start start 2024-06-10T01:10:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.100+0000 7fcb43d89740 1 Processor -- start 2024-06-10T01:10:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.100+0000 7fcb43d89740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572e52bf010 0x5572e52c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.100+0000 7fcb43d89740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572e52bfa00 0x5572e52c9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.100+0000 7fcb43d89740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572e52c4b70 0x5572e52d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.100+0000 7fcb43d89740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5572e52c7b20 con 0x5572e52bf010 2024-06-10T01:10:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.100+0000 7fcb43d89740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5572e51abae0 con 0x5572e52bfa00 2024-06-10T01:10:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.100+0000 7fcb43d89740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5572e52c7ca0 con 0x5572e52c4b70 2024-06-10T01:10:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.100+0000 7fcb43d89740 10 client.0 _ll_register_callbacks cb 0x5572e51e49b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.100+0000 7fcb3a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572e52bf010 0x5572e52c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.100+0000 7fcb3a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572e52bf010 0x5572e52c92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57622/0 (socket says 172.21.15.7:57622) 2024-06-10T01:10:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.100+0000 7fcb3a59c640 1 -- 172.21.15.7:0/954727927 learned_addr learned my addr 172.21.15.7:0/954727927 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.100+0000 7fcb3ad9d640 1 --2- 172.21.15.7:0/954727927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572e52c4b70 0x5572e52d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.100+0000 7fcb31d9b640 1 --2- 172.21.15.7:0/954727927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572e52bfa00 0x5572e52c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.100+0000 7fcb3a59c640 1 -- 172.21.15.7:0/954727927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572e52c4b70 msgr2=0x5572e52d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.100+0000 7fcb3a59c640 1 --2- 172.21.15.7:0/954727927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572e52c4b70 0x5572e52d0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.100+0000 7fcb3a59c640 1 -- 172.21.15.7:0/954727927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572e52bfa00 msgr2=0x5572e52c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.100+0000 7fcb3a59c640 1 --2- 172.21.15.7:0/954727927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572e52bfa00 0x5572e52c9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.100+0000 7fcb3a59c640 1 -- 172.21.15.7:0/954727927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572e4f36210 con 0x5572e52bf010 2024-06-10T01:10:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.101+0000 7fcb3a59c640 1 --2- 172.21.15.7:0/954727927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572e52bf010 0x5572e52c92f0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009560 tx=0x7fcb2c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76a36905ea38731 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.101+0000 7fcb21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:10:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.101+0000 7fcb21ffb640 1 -- 172.21.15.7:0/954727927 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c009910 con 0x5572e52bf010 2024-06-10T01:10:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.101+0000 7fcb21ffb640 1 -- 172.21.15.7:0/954727927 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fcb2c018030 con 0x5572e52bf010 2024-06-10T01:10:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.101+0000 7fcb43d89740 1 -- 172.21.15.7:0/954727927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb34003e50 con 0x5572e52bf010 2024-06-10T01:10:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.101+0000 7fcb21ffb640 1 -- 172.21.15.7:0/954727927 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcb2c004020 con 0x5572e52bf010 2024-06-10T01:10:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.101+0000 7fcb43d89740 10 client.16644 fetch_fsmap learned FSMap version 30 2024-06-10T01:10:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.101+0000 7fcb43d89740 1 -- 172.21.15.7:0/954727927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5572e5002620 con 0x5572e52bf010 2024-06-10T01:10:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.101+0000 7fcb21ffb640 1 -- 172.21.15.7:0/954727927 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb34003e50 con 0x5572e52bf010 2024-06-10T01:10:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.101+0000 7fcb21ffb640 1 -- 172.21.15.7:0/954727927 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fcb2c004a70 con 0x5572e52bf010 2024-06-10T01:10:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.101+0000 7fcb43d89740 10 client.16644 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:10:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.101+0000 7fcb43d89740 10 client.16644 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:10:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.101+0000 7fcb43d89740 20 client.16644 populate_metadata read hostname 'smithi007' 2024-06-10T01:10:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.101+0000 7fcb43d89740 1 --2- 172.21.15.7:0/954727927 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5572e52d95d0 0x5572e52dba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.102+0000 7fcb43d89740 4 client.16644 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:10:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.102+0000 7fcb43d89740 1 -- 172.21.15.7:0/954727927 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572e5235f10 con 0x5572e52d95d0 2024-06-10T01:10:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.102+0000 7fcb31d9b640 1 --2- 172.21.15.7:0/954727927 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5572e52d95d0 0x5572e52dba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.102+0000 7fcb31d9b640 1 --2- 172.21.15.7:0/954727927 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5572e52d95d0 0x5572e52dba70 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=609e8d0e9004dd0b server_cookie=1ad04f7158de6877 in_seq=0 out_seq=0 2024-06-10T01:10:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.102+0000 7fcb21ffb640 10 client.16644 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.104+0000 7fcb21ffb640 1 -- 172.21.15.7:0/954727927 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5572e5235f10 con 0x5572e52d95d0 2024-06-10T01:10:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.104+0000 7fcb21ffb640 10 client.16644 handle_command_reply: tid=0 2024-06-10T01:10:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.104+0000 7fcb43d89740 1 client.16644 shutdown 2024-06-10T01:10:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.104+0000 7fcb43d89740 1 -- 172.21.15.7:0/954727927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572e52bf010 msgr2=0x5572e52c92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.104+0000 7fcb43d89740 1 --2- 172.21.15.7:0/954727927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572e52bf010 0x5572e52c92f0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009560 tx=0x7fcb2c004730 comp rx=0 tx=0).stop 2024-06-10T01:10:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.104+0000 7fcb43d89740 1 -- 172.21.15.7:0/954727927 shutdown_connections 2024-06-10T01:10:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.104+0000 7fcb43d89740 1 --2- 172.21.15.7:0/954727927 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5572e52d95d0 0x5572e52dba70 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.104+0000 7fcb43d89740 1 --2- 172.21.15.7:0/954727927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572e52c4b70 0x5572e52d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.104+0000 7fcb21ffb640 0 client.16644 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.104+0000 7fcb43d89740 1 --2- 172.21.15.7:0/954727927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572e52bfa00 0x5572e52c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.104+0000 7fcb43d89740 1 --2- 172.21.15.7:0/954727927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572e52bf010 0x5572e52c92f0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.104+0000 7fcb43d89740 1 -- 172.21.15.7:0/954727927 >> 172.21.15.7:0/954727927 conn(0x5572e52ba680 msgr2=0x5572e52b97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.104+0000 7fcb43d89740 1 -- 172.21.15.7:0/954727927 shutdown_connections 2024-06-10T01:10:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.105+0000 7fcb43d89740 1 -- 172.21.15.7:0/954727927 wait complete. 2024-06-10T01:10:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.105+0000 7fcb43d89740 20 client.16644 trim_cache size 0 max 16384 2024-06-10T01:10:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.120+0000 7fcb43d89740 1 Processor -- start 2024-06-10T01:10:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.120+0000 7fcb43d89740 1 -- start start 2024-06-10T01:10:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.121+0000 7fcb43d89740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572e52bf010 0x5572e52c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.121+0000 7fcb43d89740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572e52bfa00 0x5572e52d0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.121+0000 7fcb43d89740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572e52c4b70 0x5572e52c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.121+0000 7fcb43d89740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c1aacd0 con 0x5572e52bfa00 2024-06-10T01:10:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.121+0000 7fcb43d89740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c1a8540 con 0x5572e52bf010 2024-06-10T01:10:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.121+0000 7fcb43d89740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c1a6390 con 0x5572e52c4b70 2024-06-10T01:10:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.121+0000 7fcb31d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572e52bfa00 0x5572e52d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.121+0000 7fcb31d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572e52bfa00 0x5572e52d0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57632/0 (socket says 172.21.15.7:57632) 2024-06-10T01:10:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.121+0000 7fcb31d9b640 1 -- 172.21.15.7:0/287388814 learned_addr learned my addr 172.21.15.7:0/287388814 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.121+0000 7fcb3a59c640 1 --2- 172.21.15.7:0/287388814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572e52bf010 0x5572e52c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.121+0000 7fcb3ad9d640 1 --2- 172.21.15.7:0/287388814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572e52c4b70 0x5572e52c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.121+0000 7fcb31d9b640 1 -- 172.21.15.7:0/287388814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572e52c4b70 msgr2=0x5572e52c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.121+0000 7fcb31d9b640 1 --2- 172.21.15.7:0/287388814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572e52c4b70 0x5572e52c8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.121+0000 7fcb31d9b640 1 -- 172.21.15.7:0/287388814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572e52bf010 msgr2=0x5572e52c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.121+0000 7fcb31d9b640 1 --2- 172.21.15.7:0/287388814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572e52bf010 0x5572e52c6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.121+0000 7fcb31d9b640 1 -- 172.21.15.7:0/287388814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572e52d6100 con 0x5572e52bfa00 2024-06-10T01:10:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.121+0000 7fcb3ad9d640 1 --2- 172.21.15.7:0/287388814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572e52c4b70 0x5572e52c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:10:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.121+0000 7fcb3a59c640 1 --2- 172.21.15.7:0/287388814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572e52bf010 0x5572e52c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:10:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.121+0000 7fcb31d9b640 1 --2- 172.21.15.7:0/287388814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572e52bfa00 0x5572e52d0830 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fcb24009a20 tx=0x7fcb2402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8c8ab1c99bff20 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.121+0000 7fcb23fff640 1 -- 172.21.15.7:0/287388814 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb24049070 con 0x5572e52bfa00 2024-06-10T01:10:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.121+0000 7fcb23fff640 1 -- 172.21.15.7:0/287388814 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb240bbdc0 con 0x5572e52bfa00 2024-06-10T01:10:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.121+0000 7fcb23fff640 1 -- 172.21.15.7:0/287388814 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb240b8430 con 0x5572e52bfa00 2024-06-10T01:10:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.122+0000 7fcb43d89740 1 -- 172.21.15.7:0/287388814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572e52bfa00 msgr2=0x5572e52d0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.122+0000 7fcb43d89740 1 --2- 172.21.15.7:0/287388814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572e52bfa00 0x5572e52d0830 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fcb24009a20 tx=0x7fcb2402a410 comp rx=0 tx=0).stop 2024-06-10T01:10:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.122+0000 7fcb43d89740 1 -- 172.21.15.7:0/287388814 shutdown_connections 2024-06-10T01:10:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.122+0000 7fcb43d89740 1 --2- 172.21.15.7:0/287388814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572e52c4b70 0x5572e52c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.122+0000 7fcb43d89740 1 --2- 172.21.15.7:0/287388814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572e52bfa00 0x5572e52d0830 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.122+0000 7fcb43d89740 1 --2- 172.21.15.7:0/287388814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572e52bf010 0x5572e52c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.122+0000 7fcb43d89740 1 -- 172.21.15.7:0/287388814 >> 172.21.15.7:0/287388814 conn(0x5572e52ba680 msgr2=0x5572e52c42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.122+0000 7fcb43d89740 1 -- 172.21.15.7:0/287388814 shutdown_connections 2024-06-10T01:10:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.122+0000 7fcb43d89740 1 -- 172.21.15.7:0/287388814 wait complete. 2024-06-10T01:10:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.123+0000 7fcb43d89740 1 -- start start 2024-06-10T01:10:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.123+0000 7fcb43d89740 1 Processor -- start 2024-06-10T01:10:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.123+0000 7fcb43d89740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572e52bf010 0x5572e52cd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.123+0000 7fcb43d89740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572e52bfa00 0x5572e52cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.123+0000 7fcb43d89740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572e52c4b70 0x5572e5342340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.123+0000 7fcb43d89740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c1a6a00 con 0x5572e52bf010 2024-06-10T01:10:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.123+0000 7fcb43d89740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c1a7e70 con 0x5572e52c4b70 2024-06-10T01:10:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.123+0000 7fcb43d89740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c1a70d0 con 0x5572e52bfa00 2024-06-10T01:10:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.123+0000 7fcb43d89740 10 client.0 _ll_register_callbacks cb 0x7fcb2c009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.123+0000 7fcb3a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572e52bf010 0x5572e52cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.123+0000 7fcb3a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572e52bf010 0x5572e52cd380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57642/0 (socket says 172.21.15.7:57642) 2024-06-10T01:10:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.123+0000 7fcb3a59c640 1 -- 172.21.15.7:0/1807462375 learned_addr learned my addr 172.21.15.7:0/1807462375 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.123+0000 7fcb31d9b640 1 --2- 172.21.15.7:0/1807462375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572e52bfa00 0x5572e52cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.123+0000 7fcb3ad9d640 1 --2- 172.21.15.7:0/1807462375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572e52c4b70 0x5572e5342340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.123+0000 7fcb3a59c640 1 -- 172.21.15.7:0/1807462375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572e52bfa00 msgr2=0x5572e52cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.123+0000 7fcb3a59c640 1 --2- 172.21.15.7:0/1807462375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572e52bfa00 0x5572e52cd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.123+0000 7fcb3a59c640 1 -- 172.21.15.7:0/1807462375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572e52c4b70 msgr2=0x5572e5342340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.123+0000 7fcb3a59c640 1 --2- 172.21.15.7:0/1807462375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572e52c4b70 0x5572e5342340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.123+0000 7fcb3a59c640 1 -- 172.21.15.7:0/1807462375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572e52d0990 con 0x5572e52bf010 2024-06-10T01:10:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.124+0000 7fcb3a59c640 1 --2- 172.21.15.7:0/1807462375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572e52bf010 0x5572e52cd380 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c004700 tx=0x7fcb2c0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a326dcfa0c5282cf server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.124+0000 7fcb21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:10:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.124+0000 7fcb21ffb640 1 -- 172.21.15.7:0/1807462375 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c009910 con 0x5572e52bf010 2024-06-10T01:10:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.124+0000 7fcb21ffb640 1 -- 172.21.15.7:0/1807462375 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fcb2c005000 con 0x5572e52bf010 2024-06-10T01:10:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.124+0000 7fcb43d89740 1 -- 172.21.15.7:0/1807462375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb240bbdc0 con 0x5572e52bf010 2024-06-10T01:10:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.124+0000 7fcb21ffb640 1 -- 172.21.15.7:0/1807462375 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcb2c047050 con 0x5572e52bf010 2024-06-10T01:10:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.124+0000 7fcb43d89740 10 client.16650 fetch_fsmap learned FSMap version 30 2024-06-10T01:10:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.124+0000 7fcb43d89740 1 -- 172.21.15.7:0/1807462375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5572e53437d0 con 0x5572e52bf010 2024-06-10T01:10:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.124+0000 7fcb21ffb640 1 -- 172.21.15.7:0/1807462375 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb240bbdc0 con 0x5572e52bf010 2024-06-10T01:10:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.124+0000 7fcb21ffb640 1 -- 172.21.15.7:0/1807462375 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fcb2c003b50 con 0x5572e52bf010 2024-06-10T01:10:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.124+0000 7fcb43d89740 10 client.16650 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:10:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.124+0000 7fcb43d89740 10 client.16650 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:10:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.124+0000 7fcb43d89740 20 client.16650 populate_metadata read hostname 'smithi007' 2024-06-10T01:10:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.124+0000 7fcb43d89740 1 --2- 172.21.15.7:0/1807462375 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5572e52d95d0 0x5572e53462f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.124+0000 7fcb43d89740 4 client.16650 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:10:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.124+0000 7fcb43d89740 1 -- 172.21.15.7:0/1807462375 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5572e522fc20 con 0x5572e52d95d0 2024-06-10T01:10:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.124+0000 7fcb31d9b640 1 --2- 172.21.15.7:0/1807462375 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5572e52d95d0 0x5572e53462f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.125+0000 7fcb21ffb640 10 client.16650 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.125+0000 7fcb31d9b640 1 --2- 172.21.15.7:0/1807462375 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5572e52d95d0 0x5572e53462f0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26205a8413ac6090 server_cookie=e65131f1a00ac2b1 in_seq=0 out_seq=0 2024-06-10T01:10:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.212+0000 7fcb21ffb640 1 -- 172.21.15.7:0/1807462375 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5572e522fc20 con 0x5572e52d95d0 2024-06-10T01:10:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.212+0000 7fcb21ffb640 10 client.16650 handle_command_reply: tid=0 2024-06-10T01:10:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.212+0000 7fcb43d89740 1 client.16650 shutdown 2024-06-10T01:10:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.212+0000 7fcb43d89740 1 -- 172.21.15.7:0/1807462375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572e52bf010 msgr2=0x5572e52cd380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.212+0000 7fcb43d89740 1 --2- 172.21.15.7:0/1807462375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572e52bf010 0x5572e52cd380 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c004700 tx=0x7fcb2c0049d0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.212+0000 7fcb43d89740 1 -- 172.21.15.7:0/1807462375 shutdown_connections 2024-06-10T01:10:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.212+0000 7fcb43d89740 1 --2- 172.21.15.7:0/1807462375 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5572e52d95d0 0x5572e53462f0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.212+0000 7fcb43d89740 1 --2- 172.21.15.7:0/1807462375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572e52c4b70 0x5572e5342340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.212+0000 7fcb43d89740 1 --2- 172.21.15.7:0/1807462375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5572e52bfa00 0x5572e52cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.212+0000 7fcb21ffb640 0 client.16650 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.212+0000 7fcb43d89740 1 --2- 172.21.15.7:0/1807462375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572e52bf010 0x5572e52cd380 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.212+0000 7fcb43d89740 1 -- 172.21.15.7:0/1807462375 >> 172.21.15.7:0/1807462375 conn(0x5572e52ba680 msgr2=0x5572e52c8f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.212+0000 7fcb43d89740 1 -- 172.21.15.7:0/1807462375 shutdown_connections 2024-06-10T01:10:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.212+0000 7fcb43d89740 1 -- 172.21.15.7:0/1807462375 wait complete. 2024-06-10T01:10:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.212+0000 7fcb43d89740 20 client.16650 trim_cache size 0 max 16384 2024-06-10T01:10:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:10:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:10:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:10:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.213+0000 7fcb40f9e640 1 -- 172.21.15.7:0/1841690293 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcb280a2b80 msgr2=0x7fcb280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.213+0000 7fcb40f9e640 1 --2- 172.21.15.7:0/1841690293 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcb280a2b80 0x7fcb280a5040 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x5572e52bfe00 tx=0x7fcb2c005f90 comp rx=0 tx=0).stop 2024-06-10T01:10:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.213+0000 7fcb40f9e640 1 -- 172.21.15.7:0/1841690293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb3c101b00 msgr2=0x7fcb3c19ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.213+0000 7fcb40f9e640 1 --2- 172.21.15.7:0/1841690293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb3c101b00 0x7fcb3c19ca30 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fcb24009230 tx=0x7fcb240042f0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.213+0000 7fcb40f9e640 1 -- 172.21.15.7:0/1841690293 shutdown_connections 2024-06-10T01:10:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.213+0000 7fcb40f9e640 1 --2- 172.21.15.7:0/1841690293 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcb280a2b80 0x7fcb280a5040 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.213+0000 7fcb40f9e640 1 --2- 172.21.15.7:0/1841690293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb3c1024c0 0x7fcb3c19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.213+0000 7fcb40f9e640 1 --2- 172.21.15.7:0/1841690293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb3c101b00 0x7fcb3c19ca30 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.213+0000 7fcb40f9e640 1 --2- 172.21.15.7:0/1841690293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb3c100910 0x7fcb3c06abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.213+0000 7fcb40f9e640 1 -- 172.21.15.7:0/1841690293 >> 172.21.15.7:0/1841690293 conn(0x7fcb3c0fbe80 msgr2=0x7fcb3c104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.213+0000 7fcb40f9e640 1 -- 172.21.15.7:0/1841690293 shutdown_connections 2024-06-10T01:10:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:28.213+0000 7fcb40f9e640 1 -- 172.21.15.7:0/1841690293 wait complete. 2024-06-10T01:10:28.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:28 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:10:28.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:28 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:10:28.716 INFO:tasks.workunit.client.0.smithi007.stdout: 1 281891 8.39 MB/sec execute 69 sec latency 61.230 ms 2024-06-10T01:10:28.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:28 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:10:28.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:28 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:10:28.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:28 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:10:28.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:28 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:10:29.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:29 smithi175 ceph-mon[36636]: pgmap v566: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 234 op/s 2024-06-10T01:10:29.716 INFO:tasks.workunit.client.0.smithi007.stdout: 1 284166 8.47 MB/sec execute 70 sec latency 21.360 ms 2024-06-10T01:10:29.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:29 smithi007 ceph-mon[32398]: pgmap v566: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 234 op/s 2024-06-10T01:10:29.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:29 smithi078 ceph-mon[36683]: pgmap v566: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 234 op/s 2024-06-10T01:10:30.716 INFO:tasks.workunit.client.0.smithi007.stdout: 1 285986 8.43 MB/sec execute 71 sec latency 17.358 ms 2024-06-10T01:10:31.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:31 smithi175 ceph-mon[36636]: pgmap v567: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 318 op/s 2024-06-10T01:10:31.716 INFO:tasks.workunit.client.0.smithi007.stdout: 1 287889 8.50 MB/sec execute 72 sec latency 12.330 ms 2024-06-10T01:10:31.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:31 smithi007 ceph-mon[32398]: pgmap v567: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 318 op/s 2024-06-10T01:10:31.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:31 smithi078 ceph-mon[36683]: pgmap v567: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 318 op/s 2024-06-10T01:10:32.716 INFO:tasks.workunit.client.0.smithi007.stdout: 1 290708 8.58 MB/sec execute 73 sec latency 14.779 ms 2024-06-10T01:10:33.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:33 smithi175 ceph-mon[36636]: pgmap v568: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 329 op/s 2024-06-10T01:10:33.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:33 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:10:33.716 INFO:tasks.workunit.client.0.smithi007.stdout: 1 291934 8.55 MB/sec execute 74 sec latency 34.432 ms 2024-06-10T01:10:33.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:33 smithi007 ceph-mon[32398]: pgmap v568: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 329 op/s 2024-06-10T01:10:33.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:33 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:10:33.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:33 smithi078 ceph-mon[36683]: pgmap v568: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 329 op/s 2024-06-10T01:10:33.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:33 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:10:34.716 INFO:tasks.workunit.client.0.smithi007.stdout: 1 294535 8.65 MB/sec execute 75 sec latency 68.013 ms 2024-06-10T01:10:35.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:35 smithi175 ceph-mon[36636]: pgmap v569: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 374 op/s 2024-06-10T01:10:35.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:35 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:10:35.716 INFO:tasks.workunit.client.0.smithi007.stdout: 1 295645 8.58 MB/sec execute 76 sec latency 25.192 ms 2024-06-10T01:10:35.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:35 smithi007 ceph-mon[32398]: pgmap v569: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 374 op/s 2024-06-10T01:10:35.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:35 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:10:35.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:35 smithi078 ceph-mon[36683]: pgmap v569: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 374 op/s 2024-06-10T01:10:35.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:35 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:10:36.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:10:36.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:10:36.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:10:36.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:10:36.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:10:36.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:10:36.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:10:36.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:10:36.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:10:36.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:10:36.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:10:36.716 INFO:tasks.workunit.client.0.smithi007.stdout: 1 298237 8.69 MB/sec execute 77 sec latency 11.456 ms 2024-06-10T01:10:36.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:10:36.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:10:36.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:10:36.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:10:36.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:10:36.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:10:36.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:10:36.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:10:36.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:10:36.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:10:36.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:10:36.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:10:36.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:10:36.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:10:36.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:10:36.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:10:36.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:10:36.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:10:36.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:10:36.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:10:36.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:10:36.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:10:37.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:37 smithi175 ceph-mon[36636]: Adjusting osd_memory_target on smithi078 to 176.2M 2024-06-10T01:10:37.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:37 smithi175 ceph-mon[36636]: Unable to set osd_memory_target on smithi078 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-06-10T01:10:37.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:37 smithi175 ceph-mon[36636]: Adjusting osd_memory_target on smithi007 to 177.2M 2024-06-10T01:10:37.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:37 smithi175 ceph-mon[36636]: Unable to set osd_memory_target on smithi007 to 185842585: error parsing value: Value '185842585' is below minimum 939524096 2024-06-10T01:10:37.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:37 smithi175 ceph-mon[36636]: pgmap v570: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 322 op/s 2024-06-10T01:10:37.716 INFO:tasks.workunit.client.0.smithi007.stdout: 1 300115 8.65 MB/sec execute 78 sec latency 18.427 ms 2024-06-10T01:10:37.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:37 smithi007 ceph-mon[32398]: Adjusting osd_memory_target on smithi078 to 176.2M 2024-06-10T01:10:37.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:37 smithi007 ceph-mon[32398]: Unable to set osd_memory_target on smithi078 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-06-10T01:10:37.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:37 smithi007 ceph-mon[32398]: Adjusting osd_memory_target on smithi007 to 177.2M 2024-06-10T01:10:37.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:37 smithi007 ceph-mon[32398]: Unable to set osd_memory_target on smithi007 to 185842585: error parsing value: Value '185842585' is below minimum 939524096 2024-06-10T01:10:37.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:37 smithi007 ceph-mon[32398]: pgmap v570: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 322 op/s 2024-06-10T01:10:37.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:37 smithi078 ceph-mon[36683]: Adjusting osd_memory_target on smithi078 to 176.2M 2024-06-10T01:10:37.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:37 smithi078 ceph-mon[36683]: Unable to set osd_memory_target on smithi078 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-06-10T01:10:37.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:37 smithi078 ceph-mon[36683]: Adjusting osd_memory_target on smithi007 to 177.2M 2024-06-10T01:10:37.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:37 smithi078 ceph-mon[36683]: Unable to set osd_memory_target on smithi007 to 185842585: error parsing value: Value '185842585' is below minimum 939524096 2024-06-10T01:10:37.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:37 smithi078 ceph-mon[36683]: pgmap v570: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 322 op/s 2024-06-10T01:10:38.718 INFO:tasks.workunit.client.0.smithi007.stdout: 1 302073 8.71 MB/sec execute 79 sec latency 12.411 ms 2024-06-10T01:10:39.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:39 smithi175 ceph-mon[36636]: pgmap v571: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 339 op/s 2024-06-10T01:10:39.716 INFO:tasks.workunit.client.0.smithi007.stdout: 1 304512 8.76 MB/sec execute 80 sec latency 26.136 ms 2024-06-10T01:10:39.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:39 smithi007 ceph-mon[32398]: pgmap v571: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 339 op/s 2024-06-10T01:10:39.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:39 smithi078 ceph-mon[36683]: pgmap v571: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 339 op/s 2024-06-10T01:10:40.716 INFO:tasks.workunit.client.0.smithi007.stdout: 1 305939 8.74 MB/sec execute 81 sec latency 13.941 ms 2024-06-10T01:10:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.307+0000 7f5e4dd90640 1 -- 172.21.15.7:0/3423337561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e481035e0 msgr2=0x7f5e48109e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.307+0000 7f5e4dd90640 1 --2- 172.21.15.7:0/3423337561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e481035e0 0x7f5e48109e70 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f5e40009ca0 tx=0x7f5e4002a8c0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.307+0000 7f5e4dd90640 1 -- 172.21.15.7:0/3423337561 shutdown_connections 2024-06-10T01:10:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.307+0000 7f5e4dd90640 1 --2- 172.21.15.7:0/3423337561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e481035e0 0x7f5e48109e70 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.307+0000 7f5e4dd90640 1 --2- 172.21.15.7:0/3423337561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e48102c20 0x7f5e481030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.307+0000 7f5e4dd90640 1 --2- 172.21.15.7:0/3423337561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e48101a30 0x7f5e48101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.307+0000 7f5e4dd90640 1 -- 172.21.15.7:0/3423337561 >> 172.21.15.7:0/3423337561 conn(0x7f5e480fcfc0 msgr2=0x7f5e480ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.307+0000 7f5e4dd90640 1 -- 172.21.15.7:0/3423337561 shutdown_connections 2024-06-10T01:10:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.307+0000 7f5e4dd90640 1 -- 172.21.15.7:0/3423337561 wait complete. 2024-06-10T01:10:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.307+0000 7f5e4dd90640 1 Processor -- start 2024-06-10T01:10:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.307+0000 7f5e4dd90640 1 -- start start 2024-06-10T01:10:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.308+0000 7f5e4dd90640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e48101a30 0x7f5e48197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.308+0000 7f5e4dd90640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e48102c20 0x7f5e48198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.308+0000 7f5e4dd90640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e481035e0 0x7f5e4819f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.308+0000 7f5e4dd90640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4810d690 con 0x7f5e48102c20 2024-06-10T01:10:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.308+0000 7f5e4dd90640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4810d510 con 0x7f5e48101a30 2024-06-10T01:10:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.308+0000 7f5e4dd90640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4810d810 con 0x7f5e481035e0 2024-06-10T01:10:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.308+0000 7f5e3effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e48102c20 0x7f5e48198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.308+0000 7f5e477fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e48101a30 0x7f5e48197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.308+0000 7f5e477fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e48101a30 0x7f5e48197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:34256/0 (socket says 172.21.15.7:34256) 2024-06-10T01:10:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.308+0000 7f5e477fe640 1 -- 172.21.15.7:0/3638911798 learned_addr learned my addr 172.21.15.7:0/3638911798 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.308+0000 7f5e47fff640 1 --2- 172.21.15.7:0/3638911798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e481035e0 0x7f5e4819f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.308+0000 7f5e477fe640 1 -- 172.21.15.7:0/3638911798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e481035e0 msgr2=0x7f5e4819f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.308+0000 7f5e477fe640 1 --2- 172.21.15.7:0/3638911798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e481035e0 0x7f5e4819f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.308+0000 7f5e477fe640 1 -- 172.21.15.7:0/3638911798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e48102c20 msgr2=0x7f5e48198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.308+0000 7f5e477fe640 1 --2- 172.21.15.7:0/3638911798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e48102c20 0x7f5e48198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.308+0000 7f5e477fe640 1 -- 172.21.15.7:0/3638911798 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4819f930 con 0x7f5e48101a30 2024-06-10T01:10:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.308+0000 7f5e3effd640 1 --2- 172.21.15.7:0/3638911798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e48102c20 0x7f5e48198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.308+0000 7f5e477fe640 1 --2- 172.21.15.7:0/3638911798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e48101a30 0x7f5e48197d20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5e3800e9c0 tx=0x7f5e3800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.308+0000 7f5e457fa640 1 -- 172.21.15.7:0/3638911798 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3800cd80 con 0x7f5e48101a30 2024-06-10T01:10:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.309+0000 7f5e457fa640 1 -- 172.21.15.7:0/3638911798 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e380035f0 con 0x7f5e48101a30 2024-06-10T01:10:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.309+0000 7f5e4dd90640 1 -- 172.21.15.7:0/3638911798 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e4819fc20 con 0x7f5e48101a30 2024-06-10T01:10:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.309+0000 7f5e457fa640 1 -- 172.21.15.7:0/3638911798 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e38010690 con 0x7f5e48101a30 2024-06-10T01:10:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.309+0000 7f5e4dd90640 1 -- 172.21.15.7:0/3638911798 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e481a01c0 con 0x7f5e48101a30 2024-06-10T01:10:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.311+0000 7f5e457fa640 1 -- 172.21.15.7:0/3638911798 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f5e380037a0 con 0x7f5e48101a30 2024-06-10T01:10:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.311+0000 7f5e50b7c740 1 Processor -- start 2024-06-10T01:10:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.311+0000 7f5e50b7c740 1 -- start start 2024-06-10T01:10:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.311+0000 7f5e50b7c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b760160010 0x55b760160430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.311+0000 7f5e50b7c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b760160a00 0x55b760165500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.311+0000 7f5e50b7c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b760165b70 0x55b760168000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.311+0000 7f5e50b7c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b7600c34f0 con 0x55b760165b70 2024-06-10T01:10:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.311+0000 7f5e50b7c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b760061870 con 0x55b760160a00 2024-06-10T01:10:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.311+0000 7f5e50b7c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b7600bf4b0 con 0x55b760160010 2024-06-10T01:10:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.311+0000 7f5e3effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b760165b70 0x55b760168000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.311+0000 7f5e47fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b760160010 0x55b760160430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.311+0000 7f5e477fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b760160a00 0x55b760165500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.312+0000 7f5e3effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b760165b70 0x55b760168000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48488/0 (socket says 172.21.15.7:48488) 2024-06-10T01:10:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.312+0000 7f5e3effd640 1 -- 172.21.15.7:0/675755169 learned_addr learned my addr 172.21.15.7:0/675755169 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.312+0000 7f5e457fa640 1 --2- 172.21.15.7:0/3638911798 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5e340a2c00 0x7f5e340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.312+0000 7f5e3effd640 1 -- 172.21.15.7:0/675755169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b760160010 msgr2=0x55b760160430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.312+0000 7f5e3effd640 1 --2- 172.21.15.7:0/675755169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b760160010 0x55b760160430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.312+0000 7f5e3effd640 1 -- 172.21.15.7:0/675755169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b760160a00 msgr2=0x55b760165500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.312+0000 7f5e3effd640 1 --2- 172.21.15.7:0/675755169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b760160a00 0x55b760165500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.312+0000 7f5e3effd640 1 -- 172.21.15.7:0/675755169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b76007d9b0 con 0x55b760165b70 2024-06-10T01:10:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.312+0000 7f5e47fff640 1 --2- 172.21.15.7:0/675755169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b760160010 0x55b760160430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.312+0000 7f5e47fff640 1 --2- 172.21.15.7:0/3638911798 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5e340a2c00 0x7f5e340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.312+0000 7f5e457fa640 1 -- 172.21.15.7:0/3638911798 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f5e380cf810 con 0x7f5e48101a30 2024-06-10T01:10:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.312+0000 7f5e3effd640 1 --2- 172.21.15.7:0/675755169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b760165b70 0x55b760168000 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5e300115f0 tx=0x7f5e30011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeee7883189c9f7e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.312+0000 7f5e3dffb640 1 -- 172.21.15.7:0/675755169 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3001e070 con 0x55b760165b70 2024-06-10T01:10:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.312+0000 7f5e3dffb640 1 -- 172.21.15.7:0/675755169 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e3000fb20 con 0x55b760165b70 2024-06-10T01:10:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.312+0000 7f5e3dffb640 1 -- 172.21.15.7:0/675755169 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e30019bd0 con 0x55b760165b70 2024-06-10T01:10:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.312+0000 7f5e47fff640 1 --2- 172.21.15.7:0/3638911798 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5e340a2c00 0x7f5e340a50c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5e40005e30 tx=0x7f5e400095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.313+0000 7f5e50b7c740 1 -- 172.21.15.7:0/675755169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b760165b70 msgr2=0x55b760168000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.313+0000 7f5e50b7c740 1 --2- 172.21.15.7:0/675755169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b760165b70 0x55b760168000 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5e300115f0 tx=0x7f5e30011ab0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.313+0000 7f5e50b7c740 1 -- 172.21.15.7:0/675755169 shutdown_connections 2024-06-10T01:10:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.313+0000 7f5e50b7c740 1 --2- 172.21.15.7:0/675755169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b760165b70 0x55b760168000 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.313+0000 7f5e50b7c740 1 --2- 172.21.15.7:0/675755169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b760160a00 0x55b760165500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.313+0000 7f5e50b7c740 1 --2- 172.21.15.7:0/675755169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b760160010 0x55b760160430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.313+0000 7f5e50b7c740 1 -- 172.21.15.7:0/675755169 >> 172.21.15.7:0/675755169 conn(0x55b76015b680 msgr2=0x55b76015ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.313+0000 7f5e50b7c740 1 -- 172.21.15.7:0/675755169 shutdown_connections 2024-06-10T01:10:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.313+0000 7f5e50b7c740 1 -- 172.21.15.7:0/675755169 wait complete. 2024-06-10T01:10:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.313+0000 7f5e50b7c740 1 -- start start 2024-06-10T01:10:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.314+0000 7f5e50b7c740 1 Processor -- start 2024-06-10T01:10:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.314+0000 7f5e50b7c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b760160010 0x55b76016a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.314+0000 7f5e50b7c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b760160a00 0x55b76016a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.314+0000 7f5e50b7c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b760165b70 0x55b7601717a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.314+0000 7f5e50b7c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b760168b20 con 0x55b760160a00 2024-06-10T01:10:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.314+0000 7f5e50b7c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b76004cae0 con 0x55b760165b70 2024-06-10T01:10:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.314+0000 7f5e50b7c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b760168ca0 con 0x55b760160010 2024-06-10T01:10:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.314+0000 7f5e50b7c740 10 client.0 _ll_register_callbacks cb 0x55b7600859b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.314+0000 7f5e3effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b760160a00 0x55b76016a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.314+0000 7f5e3effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b760160a00 0x55b76016a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48492/0 (socket says 172.21.15.7:48492) 2024-06-10T01:10:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.314+0000 7f5e3effd640 1 -- 172.21.15.7:0/3572494091 learned_addr learned my addr 172.21.15.7:0/3572494091 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.314+0000 7f5e477fe640 1 --2- 172.21.15.7:0/3572494091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b760160010 0x55b76016a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.314+0000 7f5e47fff640 1 --2- 172.21.15.7:0/3572494091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b760165b70 0x55b7601717a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.314+0000 7f5e477fe640 1 -- 172.21.15.7:0/3572494091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b760165b70 msgr2=0x55b7601717a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.314+0000 7f5e477fe640 1 --2- 172.21.15.7:0/3572494091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b760165b70 0x55b7601717a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.314+0000 7f5e477fe640 1 -- 172.21.15.7:0/3572494091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b760160a00 msgr2=0x55b76016a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.314+0000 7f5e477fe640 1 --2- 172.21.15.7:0/3572494091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b760160a00 0x55b76016a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.314+0000 7f5e477fe640 1 -- 172.21.15.7:0/3572494091 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b75fdd7210 con 0x55b760160010 2024-06-10T01:10:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.314+0000 7f5e3effd640 1 --2- 172.21.15.7:0/3572494091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b760160a00 0x55b76016a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.314+0000 7f5e47fff640 1 --2- 172.21.15.7:0/3572494091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b760165b70 0x55b7601717a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.314+0000 7f5e477fe640 1 --2- 172.21.15.7:0/3572494091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b760160010 0x55b76016a220 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5e380978e0 tx=0x7f5e380d3040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e4875ded54496c5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.314+0000 7f5e26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:10:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.315+0000 7f5e26ffd640 1 -- 172.21.15.7:0/3572494091 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3809ba60 con 0x55b760160010 2024-06-10T01:10:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.315+0000 7f5e26ffd640 1 -- 172.21.15.7:0/3572494091 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f5e380a4670 con 0x55b760160010 2024-06-10T01:10:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.315+0000 7f5e50b7c740 1 -- 172.21.15.7:0/3572494091 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e3000fb20 con 0x55b760160010 2024-06-10T01:10:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.315+0000 7f5e26ffd640 1 -- 172.21.15.7:0/3572494091 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5e3809c390 con 0x55b760160010 2024-06-10T01:10:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.315+0000 7f5e50b7c740 10 client.26182 fetch_fsmap learned FSMap version 31 2024-06-10T01:10:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.315+0000 7f5e50b7c740 1 -- 172.21.15.7:0/3572494091 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b75fea3620 con 0x55b760160010 2024-06-10T01:10:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.315+0000 7f5e26ffd640 1 -- 172.21.15.7:0/3572494091 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e3000fb20 con 0x55b760160010 2024-06-10T01:10:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.315+0000 7f5e26ffd640 1 -- 172.21.15.7:0/3572494091 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f5e3809ab50 con 0x55b760160010 2024-06-10T01:10:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.315+0000 7f5e50b7c740 10 client.26182 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:10:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.315+0000 7f5e50b7c740 10 client.26182 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:10:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.315+0000 7f5e50b7c740 20 client.26182 populate_metadata read hostname 'smithi007' 2024-06-10T01:10:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.315+0000 7f5e50b7c740 1 --2- 172.21.15.7:0/3572494091 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b7601761a0 0x55b7601785f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.315+0000 7f5e50b7c740 4 client.26182 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:10:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.315+0000 7f5e50b7c740 1 -- 172.21.15.7:0/3572494091 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7600d6f10 con 0x55b7601761a0 2024-06-10T01:10:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.315+0000 7f5e3effd640 1 --2- 172.21.15.7:0/3572494091 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b7601761a0 0x55b7601785f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.316+0000 7f5e3effd640 1 --2- 172.21.15.7:0/3572494091 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b7601761a0 0x55b7601785f0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3ae686bcfdc8ac8 server_cookie=dfdb2f1dde7dff5 in_seq=0 out_seq=0 2024-06-10T01:10:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.316+0000 7f5e26ffd640 10 client.26182 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.317+0000 7f5e26ffd640 1 -- 172.21.15.7:0/3572494091 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b7600d6f10 con 0x55b7601761a0 2024-06-10T01:10:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.317+0000 7f5e26ffd640 10 client.26182 handle_command_reply: tid=0 2024-06-10T01:10:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.317+0000 7f5e50b7c740 1 client.26182 shutdown 2024-06-10T01:10:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.317+0000 7f5e50b7c740 1 -- 172.21.15.7:0/3572494091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b760160010 msgr2=0x55b76016a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.317+0000 7f5e50b7c740 1 --2- 172.21.15.7:0/3572494091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b760160010 0x55b76016a220 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5e380978e0 tx=0x7f5e380d3040 comp rx=0 tx=0).stop 2024-06-10T01:10:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.317+0000 7f5e50b7c740 1 -- 172.21.15.7:0/3572494091 shutdown_connections 2024-06-10T01:10:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.317+0000 7f5e50b7c740 1 --2- 172.21.15.7:0/3572494091 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b7601761a0 0x55b7601785f0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.317+0000 7f5e50b7c740 1 --2- 172.21.15.7:0/3572494091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b760165b70 0x55b7601717a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.317+0000 7f5e50b7c740 1 --2- 172.21.15.7:0/3572494091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b760160a00 0x55b76016a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.317+0000 7f5e50b7c740 1 --2- 172.21.15.7:0/3572494091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b760160010 0x55b76016a220 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.317+0000 7f5e26ffd640 0 client.26182 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.317+0000 7f5e50b7c740 1 -- 172.21.15.7:0/3572494091 >> 172.21.15.7:0/3572494091 conn(0x55b76015b680 msgr2=0x55b76015a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.317+0000 7f5e50b7c740 1 -- 172.21.15.7:0/3572494091 shutdown_connections 2024-06-10T01:10:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.317+0000 7f5e50b7c740 1 -- 172.21.15.7:0/3572494091 wait complete. 2024-06-10T01:10:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.317+0000 7f5e50b7c740 20 client.26182 trim_cache size 0 max 16384 2024-06-10T01:10:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.333+0000 7f5e50b7c740 1 Processor -- start 2024-06-10T01:10:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.333+0000 7f5e50b7c740 1 -- start start 2024-06-10T01:10:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.333+0000 7f5e50b7c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b760160010 0x55b760165240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.333+0000 7f5e50b7c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b760160a00 0x55b760171760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.333+0000 7f5e50b7c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b760165b70 0x55b760169e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.333+0000 7f5e50b7c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5e481acf40 con 0x55b760160010 2024-06-10T01:10:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.333+0000 7f5e50b7c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5e481aa7e0 con 0x55b760160a00 2024-06-10T01:10:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.333+0000 7f5e50b7c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5e481a8630 con 0x55b760165b70 2024-06-10T01:10:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.333+0000 7f5e477fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b760160010 0x55b760165240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.333+0000 7f5e477fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b760160010 0x55b760165240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48500/0 (socket says 172.21.15.7:48500) 2024-06-10T01:10:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.333+0000 7f5e3effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b760160a00 0x55b760171760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.333+0000 7f5e477fe640 1 -- 172.21.15.7:0/3537949386 learned_addr learned my addr 172.21.15.7:0/3537949386 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.333+0000 7f5e47fff640 1 --2- 172.21.15.7:0/3537949386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b760165b70 0x55b760169e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.334+0000 7f5e3effd640 1 -- 172.21.15.7:0/3537949386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b760165b70 msgr2=0x55b760169e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.334+0000 7f5e3effd640 1 --2- 172.21.15.7:0/3537949386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b760165b70 0x55b760169e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.334+0000 7f5e3effd640 1 -- 172.21.15.7:0/3537949386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b760160010 msgr2=0x55b760165240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.334+0000 7f5e3effd640 1 --2- 172.21.15.7:0/3537949386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b760160010 0x55b760165240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.334+0000 7f5e3effd640 1 -- 172.21.15.7:0/3537949386 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b76015b4b0 con 0x55b760160a00 2024-06-10T01:10:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.334+0000 7f5e3effd640 1 --2- 172.21.15.7:0/3537949386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b760160a00 0x55b760171760 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55b76016b700 tx=0x7f5e3000f910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50ad065ddc474a86 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.334+0000 7f5e3d7fa640 1 -- 172.21.15.7:0/3537949386 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3001e040 con 0x55b760160a00 2024-06-10T01:10:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.334+0000 7f5e3d7fa640 1 -- 172.21.15.7:0/3537949386 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e3000fa80 con 0x55b760160a00 2024-06-10T01:10:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.334+0000 7f5e3d7fa640 1 -- 172.21.15.7:0/3537949386 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e30003a30 con 0x55b760160a00 2024-06-10T01:10:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.335+0000 7f5e50b7c740 1 -- 172.21.15.7:0/3537949386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b760160a00 msgr2=0x55b760171760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.335+0000 7f5e50b7c740 1 --2- 172.21.15.7:0/3537949386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b760160a00 0x55b760171760 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55b76016b700 tx=0x7f5e3000f910 comp rx=0 tx=0).stop 2024-06-10T01:10:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.335+0000 7f5e50b7c740 1 -- 172.21.15.7:0/3537949386 shutdown_connections 2024-06-10T01:10:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.335+0000 7f5e50b7c740 1 --2- 172.21.15.7:0/3537949386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b760165b70 0x55b760169e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.335+0000 7f5e50b7c740 1 --2- 172.21.15.7:0/3537949386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b760160a00 0x55b760171760 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.335+0000 7f5e50b7c740 1 --2- 172.21.15.7:0/3537949386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b760160010 0x55b760165240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.335+0000 7f5e50b7c740 1 -- 172.21.15.7:0/3537949386 >> 172.21.15.7:0/3537949386 conn(0x55b76015b680 msgr2=0x55b76015ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.335+0000 7f5e50b7c740 1 -- 172.21.15.7:0/3537949386 shutdown_connections 2024-06-10T01:10:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.335+0000 7f5e50b7c740 1 -- 172.21.15.7:0/3537949386 wait complete. 2024-06-10T01:10:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.335+0000 7f5e50b7c740 1 -- start start 2024-06-10T01:10:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.335+0000 7f5e50b7c740 1 Processor -- start 2024-06-10T01:10:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.336+0000 7f5e50b7c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b760160010 0x55b760171120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.336+0000 7f5e50b7c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b760160a00 0x55b760171660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.336+0000 7f5e50b7c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b760165b70 0x55b76016dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.336+0000 7f5e50b7c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5e481a8ca0 con 0x55b760165b70 2024-06-10T01:10:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.336+0000 7f5e50b7c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5e481aa110 con 0x55b760160010 2024-06-10T01:10:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.336+0000 7f5e50b7c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5e481a9370 con 0x55b760160a00 2024-06-10T01:10:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.336+0000 7f5e50b7c740 10 client.0 _ll_register_callbacks cb 0x7f5e380d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.336+0000 7f5e47fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b760165b70 0x55b76016dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.336+0000 7f5e47fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b760165b70 0x55b76016dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48502/0 (socket says 172.21.15.7:48502) 2024-06-10T01:10:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.336+0000 7f5e477fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b760160010 0x55b760171120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.336+0000 7f5e47fff640 1 -- 172.21.15.7:0/969447168 learned_addr learned my addr 172.21.15.7:0/969447168 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.336+0000 7f5e3effd640 1 --2- 172.21.15.7:0/969447168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b760160a00 0x55b760171660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.336+0000 7f5e47fff640 1 -- 172.21.15.7:0/969447168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b760160a00 msgr2=0x55b760171660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.336+0000 7f5e47fff640 1 --2- 172.21.15.7:0/969447168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b760160a00 0x55b760171660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.336+0000 7f5e47fff640 1 -- 172.21.15.7:0/969447168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b760160010 msgr2=0x55b760171120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.336+0000 7f5e47fff640 1 --2- 172.21.15.7:0/969447168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b760160010 0x55b760171120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.336+0000 7f5e47fff640 1 -- 172.21.15.7:0/969447168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b76016e460 con 0x55b760165b70 2024-06-10T01:10:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.336+0000 7f5e477fe640 1 --2- 172.21.15.7:0/969447168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b760160010 0x55b760171120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.336+0000 7f5e47fff640 1 --2- 172.21.15.7:0/969447168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b760165b70 0x55b76016dc30 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f5e40009bf0 tx=0x7f5e4002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df94139766f1095 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.336+0000 7f5e26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:10:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.336+0000 7f5e26ffd640 1 -- 172.21.15.7:0/969447168 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e40059060 con 0x55b760165b70 2024-06-10T01:10:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.336+0000 7f5e26ffd640 1 -- 172.21.15.7:0/969447168 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f5e400046a0 con 0x55b760165b70 2024-06-10T01:10:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.336+0000 7f5e50b7c740 1 -- 172.21.15.7:0/969447168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e3000fa80 con 0x55b760165b70 2024-06-10T01:10:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.337+0000 7f5e26ffd640 1 -- 172.21.15.7:0/969447168 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5e40060050 con 0x55b760165b70 2024-06-10T01:10:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.337+0000 7f5e50b7c740 10 client.16668 fetch_fsmap learned FSMap version 31 2024-06-10T01:10:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.337+0000 7f5e50b7c740 1 -- 172.21.15.7:0/969447168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b7601dfcb0 con 0x55b760165b70 2024-06-10T01:10:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.337+0000 7f5e26ffd640 1 -- 172.21.15.7:0/969447168 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e3000fa80 con 0x55b760165b70 2024-06-10T01:10:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.337+0000 7f5e26ffd640 1 -- 172.21.15.7:0/969447168 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f5e40053030 con 0x55b760165b70 2024-06-10T01:10:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.337+0000 7f5e50b7c740 10 client.16668 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:10:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.337+0000 7f5e50b7c740 10 client.16668 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:10:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.337+0000 7f5e50b7c740 20 client.16668 populate_metadata read hostname 'smithi007' 2024-06-10T01:10:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.337+0000 7f5e50b7c740 1 --2- 172.21.15.7:0/969447168 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b7601e06c0 0x55b7601e2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.337+0000 7f5e50b7c740 4 client.16668 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:10:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.337+0000 7f5e50b7c740 1 -- 172.21.15.7:0/969447168 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b7600d0c20 con 0x55b7601e06c0 2024-06-10T01:10:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.337+0000 7f5e477fe640 1 --2- 172.21.15.7:0/969447168 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b7601e06c0 0x55b7601e2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.338+0000 7f5e477fe640 1 --2- 172.21.15.7:0/969447168 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b7601e06c0 0x55b7601e2ae0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d781f78bcd41a004 server_cookie=f633775780f61478 in_seq=0 out_seq=0 2024-06-10T01:10:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.338+0000 7f5e26ffd640 10 client.16668 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:41.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.449+0000 7f5e26ffd640 1 -- 172.21.15.7:0/969447168 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b7600d0c20 con 0x55b7601e06c0 2024-06-10T01:10:41.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.449+0000 7f5e26ffd640 10 client.16668 handle_command_reply: tid=0 2024-06-10T01:10:41.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.449+0000 7f5e50b7c740 1 client.16668 shutdown 2024-06-10T01:10:41.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.450+0000 7f5e50b7c740 1 -- 172.21.15.7:0/969447168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b760165b70 msgr2=0x55b76016dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:41.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.450+0000 7f5e50b7c740 1 --2- 172.21.15.7:0/969447168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b760165b70 0x55b76016dc30 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f5e40009bf0 tx=0x7f5e4002a8c0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.450+0000 7f5e50b7c740 1 -- 172.21.15.7:0/969447168 shutdown_connections 2024-06-10T01:10:41.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.450+0000 7f5e50b7c740 1 --2- 172.21.15.7:0/969447168 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b7601e06c0 0x55b7601e2ae0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.450+0000 7f5e50b7c740 1 --2- 172.21.15.7:0/969447168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b760165b70 0x55b76016dc30 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.450+0000 7f5e50b7c740 1 --2- 172.21.15.7:0/969447168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b760160a00 0x55b760171660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.450+0000 7f5e26ffd640 0 client.16668 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:41.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.450+0000 7f5e50b7c740 1 --2- 172.21.15.7:0/969447168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b760160010 0x55b760171120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.450+0000 7f5e50b7c740 1 -- 172.21.15.7:0/969447168 >> 172.21.15.7:0/969447168 conn(0x55b76015b680 msgr2=0x55b760164e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:41.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.450+0000 7f5e50b7c740 1 -- 172.21.15.7:0/969447168 shutdown_connections 2024-06-10T01:10:41.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.450+0000 7f5e50b7c740 1 -- 172.21.15.7:0/969447168 wait complete. 2024-06-10T01:10:41.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.450+0000 7f5e50b7c740 20 client.16668 trim_cache size 0 max 16384 2024-06-10T01:10:41.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:10:41.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:10:41.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:10:41.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.450+0000 7f5e4dd90640 1 -- 172.21.15.7:0/3638911798 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5e340a2c00 msgr2=0x7f5e340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:41.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.450+0000 7f5e4dd90640 1 --2- 172.21.15.7:0/3638911798 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5e340a2c00 0x7f5e340a50c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5e40005e30 tx=0x7f5e400095c0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.450+0000 7f5e4dd90640 1 -- 172.21.15.7:0/3638911798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e48101a30 msgr2=0x7f5e48197d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.450+0000 7f5e4dd90640 1 --2- 172.21.15.7:0/3638911798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e48101a30 0x7f5e48197d20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5e3800e9c0 tx=0x7f5e3800ee80 comp rx=0 tx=0).stop 2024-06-10T01:10:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.451+0000 7f5e4dd90640 1 -- 172.21.15.7:0/3638911798 shutdown_connections 2024-06-10T01:10:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.451+0000 7f5e4dd90640 1 --2- 172.21.15.7:0/3638911798 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5e340a2c00 0x7f5e340a50c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.451+0000 7f5e4dd90640 1 --2- 172.21.15.7:0/3638911798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e481035e0 0x7f5e4819f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.451+0000 7f5e4dd90640 1 --2- 172.21.15.7:0/3638911798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e48102c20 0x7f5e48198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.451+0000 7f5e4dd90640 1 --2- 172.21.15.7:0/3638911798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e48101a30 0x7f5e48197d20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.451+0000 7f5e4dd90640 1 -- 172.21.15.7:0/3638911798 >> 172.21.15.7:0/3638911798 conn(0x7f5e480fcfc0 msgr2=0x7f5e48105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.451+0000 7f5e4dd90640 1 -- 172.21.15.7:0/3638911798 shutdown_connections 2024-06-10T01:10:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:41.451+0000 7f5e4dd90640 1 -- 172.21.15.7:0/3638911798 wait complete. 2024-06-10T01:10:41.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:41 smithi175 ceph-mon[36636]: pgmap v572: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 335 op/s 2024-06-10T01:10:41.716 INFO:tasks.workunit.client.0.smithi007.stdout: 1 308574 8.84 MB/sec execute 82 sec latency 51.952 ms 2024-06-10T01:10:41.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:41 smithi007 ceph-mon[32398]: pgmap v572: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 335 op/s 2024-06-10T01:10:41.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:41 smithi078 ceph-mon[36683]: pgmap v572: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 335 op/s 2024-06-10T01:10:42.717 INFO:tasks.workunit.client.0.smithi007.stdout: 1 309768 8.78 MB/sec execute 83 sec latency 17.908 ms 2024-06-10T01:10:43.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:43 smithi175 ceph-mon[36636]: pgmap v573: 129 pgs: 129 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 257 op/s 2024-06-10T01:10:43.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:43 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:10:43.717 INFO:tasks.workunit.client.0.smithi007.stdout: 1 312447 8.87 MB/sec execute 84 sec latency 17.016 ms 2024-06-10T01:10:43.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:43 smithi007 ceph-mon[32398]: pgmap v573: 129 pgs: 129 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 257 op/s 2024-06-10T01:10:43.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:43 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:10:43.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:43 smithi078 ceph-mon[36683]: pgmap v573: 129 pgs: 129 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 257 op/s 2024-06-10T01:10:43.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:43 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:10:44.717 INFO:tasks.workunit.client.0.smithi007.stdout: 1 314159 8.84 MB/sec execute 85 sec latency 11.561 ms 2024-06-10T01:10:45.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:45 smithi175 ceph-mon[36636]: pgmap v574: 129 pgs: 129 active+clean; 65 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 290 op/s 2024-06-10T01:10:45.717 INFO:tasks.workunit.client.0.smithi007.stdout: 1 316203 8.89 MB/sec execute 86 sec latency 9.326 ms 2024-06-10T01:10:45.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:45 smithi007 ceph-mon[32398]: pgmap v574: 129 pgs: 129 active+clean; 65 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 290 op/s 2024-06-10T01:10:45.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:45 smithi078 ceph-mon[36683]: pgmap v574: 129 pgs: 129 active+clean; 65 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 290 op/s 2024-06-10T01:10:46.717 INFO:tasks.workunit.client.0.smithi007.stdout: 1 318713 8.94 MB/sec execute 87 sec latency 17.697 ms 2024-06-10T01:10:47.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:47 smithi175 ceph-mon[36636]: pgmap v575: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 287 op/s 2024-06-10T01:10:47.718 INFO:tasks.workunit.client.0.smithi007.stdout: 1 320340 8.93 MB/sec execute 88 sec latency 12.876 ms 2024-06-10T01:10:47.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:47 smithi007 ceph-mon[32398]: pgmap v575: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 287 op/s 2024-06-10T01:10:47.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:47 smithi078 ceph-mon[36683]: pgmap v575: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 287 op/s 2024-06-10T01:10:48.717 INFO:tasks.workunit.client.0.smithi007.stdout: 1 323106 9.02 MB/sec execute 89 sec latency 12.269 ms 2024-06-10T01:10:49.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:49 smithi175 ceph-mon[36636]: pgmap v576: 129 pgs: 129 active+clean; 68 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 276 op/s 2024-06-10T01:10:49.717 INFO:tasks.workunit.client.0.smithi007.stdout: 1 324773 8.98 MB/sec execute 90 sec latency 10.865 ms 2024-06-10T01:10:49.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:49 smithi007 ceph-mon[32398]: pgmap v576: 129 pgs: 129 active+clean; 68 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 276 op/s 2024-06-10T01:10:49.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:49 smithi078 ceph-mon[36683]: pgmap v576: 129 pgs: 129 active+clean; 68 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 276 op/s 2024-06-10T01:10:50.717 INFO:tasks.workunit.client.0.smithi007.stdout: 1 326889 9.04 MB/sec execute 91 sec latency 13.314 ms 2024-06-10T01:10:51.631 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:51 smithi078 ceph-mon[36683]: pgmap v577: 129 pgs: 129 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 290 op/s 2024-06-10T01:10:51.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:51 smithi175 ceph-mon[36636]: pgmap v577: 129 pgs: 129 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 290 op/s 2024-06-10T01:10:51.717 INFO:tasks.workunit.client.0.smithi007.stdout: 1 329547 9.08 MB/sec execute 92 sec latency 14.135 ms 2024-06-10T01:10:51.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:51 smithi007 ceph-mon[32398]: pgmap v577: 129 pgs: 129 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 290 op/s 2024-06-10T01:10:52.211 INFO:tasks.quiescer.fs.[cephfs]:Total duration (58.9) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-06-10T01:10:52.211 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.45 2024-06-10T01:10:52.211 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:10:52.470 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:10:52.717 INFO:tasks.workunit.client.0.smithi007.stdout: 1 331021 9.07 MB/sec execute 93 sec latency 14.157 ms 2024-06-10T01:10:53.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:53 smithi007 ceph-mon[32398]: pgmap v578: 129 pgs: 129 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 283 op/s 2024-06-10T01:10:53.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:53 smithi175 ceph-mon[36636]: pgmap v578: 129 pgs: 129 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 283 op/s 2024-06-10T01:10:53.717 INFO:tasks.workunit.client.0.smithi007.stdout: 1 333640 9.15 MB/sec execute 94 sec latency 29.932 ms 2024-06-10T01:10:53.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:53 smithi078 ceph-mon[36683]: pgmap v578: 129 pgs: 129 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 283 op/s 2024-06-10T01:10:54.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.264+0000 7fcd66b38700 1 -- 172.21.15.7:0/4046323416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd60105060 msgr2=0x7fcd60111570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:54.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.264+0000 7fcd66b38700 1 --2- 172.21.15.7:0/4046323416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd60105060 0x7fcd60111570 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fcd5800b0a0 tx=0x7fcd5800b3b0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.264+0000 7fcd66b38700 1 -- 172.21.15.7:0/4046323416 shutdown_connections 2024-06-10T01:10:54.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.264+0000 7fcd66b38700 1 --2- 172.21.15.7:0/4046323416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd60105060 0x7fcd60111570 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.264+0000 7fcd66b38700 1 --2- 172.21.15.7:0/4046323416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd6010a6a0 0x7fcd6010aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.264+0000 7fcd66b38700 1 --2- 172.21.15.7:0/4046323416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd601046a0 0x7fcd60104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.264+0000 7fcd66b38700 1 -- 172.21.15.7:0/4046323416 >> 172.21.15.7:0/4046323416 conn(0x7fcd60100170 msgr2=0x7fcd60102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:54.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.264+0000 7fcd66b38700 1 -- 172.21.15.7:0/4046323416 shutdown_connections 2024-06-10T01:10:54.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.264+0000 7fcd66b38700 1 -- 172.21.15.7:0/4046323416 wait complete. 2024-06-10T01:10:54.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.265+0000 7fcd66b38700 1 Processor -- start 2024-06-10T01:10:54.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.265+0000 7fcd66b38700 1 -- start start 2024-06-10T01:10:54.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.265+0000 7fcd66b38700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd601046a0 0x7fcd6019ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.265+0000 7fcd66b38700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd60105060 0x7fcd6019b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.265+0000 7fcd66b38700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd6010a6a0 0x7fcd6019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.265+0000 7fcd66b38700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcd601149d0 con 0x7fcd60105060 2024-06-10T01:10:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.266+0000 7fcd66b38700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcd60114850 con 0x7fcd6010a6a0 2024-06-10T01:10:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.266+0000 7fcd66b38700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcd60114b50 con 0x7fcd601046a0 2024-06-10T01:10:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.266+0000 7fcd650d5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd6010a6a0 0x7fcd6019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.266+0000 7fcd650d5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd6010a6a0 0x7fcd6019f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:57338/0 (socket says 172.21.15.7:57338) 2024-06-10T01:10:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.266+0000 7fcd650d5700 1 -- 172.21.15.7:0/778038297 learned_addr learned my addr 172.21.15.7:0/778038297 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.266+0000 7fcd5ffff700 1 --2- 172.21.15.7:0/778038297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd60105060 0x7fcd6019b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.266+0000 7fcd648d4700 1 --2- 172.21.15.7:0/778038297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd601046a0 0x7fcd6019ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.266+0000 7fcd650d5700 1 -- 172.21.15.7:0/778038297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd601046a0 msgr2=0x7fcd6019ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.266+0000 7fcd650d5700 1 --2- 172.21.15.7:0/778038297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd601046a0 0x7fcd6019ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.266+0000 7fcd650d5700 1 -- 172.21.15.7:0/778038297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd60105060 msgr2=0x7fcd6019b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.266+0000 7fcd650d5700 1 --2- 172.21.15.7:0/778038297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd60105060 0x7fcd6019b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.266+0000 7fcd650d5700 1 -- 172.21.15.7:0/778038297 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd6019fe90 con 0x7fcd6010a6a0 2024-06-10T01:10:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.266+0000 7fcd650d5700 1 --2- 172.21.15.7:0/778038297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd6010a6a0 0x7fcd6019f750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcd5800b780 tx=0x7fcd58004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.267+0000 7fcd5dffb700 1 -- 172.21.15.7:0/778038297 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd58021070 con 0x7fcd6010a6a0 2024-06-10T01:10:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.267+0000 7fcd5dffb700 1 -- 172.21.15.7:0/778038297 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd58007b80 con 0x7fcd6010a6a0 2024-06-10T01:10:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.267+0000 7fcd66b38700 1 -- 172.21.15.7:0/778038297 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd601a0120 con 0x7fcd6010a6a0 2024-06-10T01:10:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.267+0000 7fcd5dffb700 1 -- 172.21.15.7:0/778038297 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd58005370 con 0x7fcd6010a6a0 2024-06-10T01:10:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.267+0000 7fcd66b38700 1 -- 172.21.15.7:0/778038297 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd601a7a00 con 0x7fcd6010a6a0 2024-06-10T01:10:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.268+0000 7fcd66b38700 1 -- 172.21.15.7:0/778038297 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd60069180 con 0x7fcd6010a6a0 2024-06-10T01:10:54.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.270+0000 7fcd5dffb700 1 -- 172.21.15.7:0/778038297 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fcd58024070 con 0x7fcd6010a6a0 2024-06-10T01:10:54.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.271+0000 7fcd5dffb700 1 --2- 172.21.15.7:0/778038297 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcd500a2c20 0x7fcd500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.271+0000 7fcd5dffb700 1 -- 172.21.15.7:0/778038297 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fcd5800f020 con 0x7fcd6010a6a0 2024-06-10T01:10:54.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.271+0000 7fcd648d4700 1 --2- 172.21.15.7:0/778038297 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcd500a2c20 0x7fcd500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.271+0000 7fcd648d4700 1 --2- 172.21.15.7:0/778038297 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcd500a2c20 0x7fcd500a50e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcd54009b10 tx=0x7fcd54009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.273+0000 7fcd5dffb700 1 -- 172.21.15.7:0/778038297 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcd5802b020 con 0x7fcd6010a6a0 2024-06-10T01:10:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.547+0000 7f731f61c640 1 -- 172.21.15.7:0/2828016160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7318104e30 msgr2=0x7f7318107220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.547+0000 7f731f61c640 1 --2- 172.21.15.7:0/2828016160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7318104e30 0x7f7318107220 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7308009900 tx=0x7f730802a3f0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.548+0000 7f731f61c640 1 -- 172.21.15.7:0/2828016160 shutdown_connections 2024-06-10T01:10:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.548+0000 7f731f61c640 1 --2- 172.21.15.7:0/2828016160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7318107760 0x7f7318109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.548+0000 7f731f61c640 1 --2- 172.21.15.7:0/2828016160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7318104e30 0x7f7318107220 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.548+0000 7f731f61c640 1 --2- 172.21.15.7:0/2828016160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7318102500 0x7f73181048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.548+0000 7f731f61c640 1 -- 172.21.15.7:0/2828016160 >> 172.21.15.7:0/2828016160 conn(0x7f73180fbe70 msgr2=0x7f73180fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.548+0000 7f731f61c640 1 -- 172.21.15.7:0/2828016160 shutdown_connections 2024-06-10T01:10:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.548+0000 7f731f61c640 1 -- 172.21.15.7:0/2828016160 wait complete. 2024-06-10T01:10:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.548+0000 7f731f61c640 1 Processor -- start 2024-06-10T01:10:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.548+0000 7f731f61c640 1 -- start start 2024-06-10T01:10:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.549+0000 7f731f61c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7318102500 0x7f7318101e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.549+0000 7f731f61c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7318104e30 0x7f73181004b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.549+0000 7f731f61c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7318107760 0x7f73181009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.549+0000 7f731f61c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f731810d670 con 0x7f7318104e30 2024-06-10T01:10:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.549+0000 7f731f61c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f731810d4f0 con 0x7f7318107760 2024-06-10T01:10:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.549+0000 7f731f61c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f731810d7f0 con 0x7f7318102500 2024-06-10T01:10:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.549+0000 7f731d3b8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7318102500 0x7f7318101e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.549+0000 7f731cbb7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7318104e30 0x7f73181004b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.549+0000 7f731dbb9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7318107760 0x7f73181009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.549+0000 7f731d3b8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7318102500 0x7f7318101e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:39184/0 (socket says 172.21.15.7:39184) 2024-06-10T01:10:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.549+0000 7f731dbb9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7318107760 0x7f73181009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:57358/0 (socket says 172.21.15.7:57358) 2024-06-10T01:10:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.549+0000 7f731d3b8640 1 -- 172.21.15.7:0/2563037669 learned_addr learned my addr 172.21.15.7:0/2563037669 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.549+0000 7f731cbb7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7318104e30 0x7f73181004b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52714/0 (socket says 172.21.15.7:52714) 2024-06-10T01:10:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.549+0000 7f731d3b8640 1 -- 172.21.15.7:0/2563037669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7318107760 msgr2=0x7f73181009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.549+0000 7f731d3b8640 1 --2- 172.21.15.7:0/2563037669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7318107760 0x7f73181009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.549+0000 7f731d3b8640 1 -- 172.21.15.7:0/2563037669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7318104e30 msgr2=0x7f73181004b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.549+0000 7f731d3b8640 1 --2- 172.21.15.7:0/2563037669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7318104e30 0x7f73181004b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.549+0000 7f731d3b8640 1 -- 172.21.15.7:0/2563037669 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7318101280 con 0x7f7318102500 2024-06-10T01:10:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.549+0000 7f731dbb9640 1 --2- 172.21.15.7:0/2563037669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7318107760 0x7f73181009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.549+0000 7f731d3b8640 1 --2- 172.21.15.7:0/2563037669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7318102500 0x7f7318101e70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f731400e9f0 tx=0x7f731400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.550+0000 7f730e7fc640 1 -- 172.21.15.7:0/2563037669 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731400cd80 con 0x7f7318102500 2024-06-10T01:10:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.550+0000 7f730e7fc640 1 -- 172.21.15.7:0/2563037669 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7314005500 con 0x7f7318102500 2024-06-10T01:10:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.550+0000 7f730e7fc640 1 -- 172.21.15.7:0/2563037669 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314010690 con 0x7f7318102500 2024-06-10T01:10:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.550+0000 7f731f61c640 1 -- 172.21.15.7:0/2563037669 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73181a1e90 con 0x7f7318102500 2024-06-10T01:10:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.550+0000 7f731f61c640 1 -- 172.21.15.7:0/2563037669 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73181a2290 con 0x7f7318102500 2024-06-10T01:10:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.553+0000 7f7322408740 1 Processor -- start 2024-06-10T01:10:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.553+0000 7f7322408740 1 -- start start 2024-06-10T01:10:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.553+0000 7f7322408740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af7a744010 0x55af7a744430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.553+0000 7f7322408740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7a744a00 0x55af7a749500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.553+0000 7f7322408740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af7a749b70 0x55af7a74c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.553+0000 7f7322408740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55af7a6a74f0 con 0x55af7a744a00 2024-06-10T01:10:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.553+0000 7f7322408740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55af7a645870 con 0x55af7a744010 2024-06-10T01:10:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.553+0000 7f7322408740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55af7a6a34b0 con 0x55af7a749b70 2024-06-10T01:10:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.553+0000 7f731dbb9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af7a744010 0x55af7a744430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.553+0000 7f731cbb7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af7a749b70 0x55af7a74c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.553+0000 7f731cbb7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af7a749b70 0x55af7a74c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:39194/0 (socket says 172.21.15.7:39194) 2024-06-10T01:10:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.553+0000 7f731dbb9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af7a744010 0x55af7a744430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:57370/0 (socket says 172.21.15.7:57370) 2024-06-10T01:10:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.553+0000 7f731cbb7640 1 -- 172.21.15.7:0/3427516317 learned_addr learned my addr 172.21.15.7:0/3427516317 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.553+0000 7f730e7fc640 1 -- 172.21.15.7:0/2563037669 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f73140046c0 con 0x7f7318102500 2024-06-10T01:10:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.553+0000 7f731d3b8640 1 --2- 172.21.15.7:0/3427516317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7a744a00 0x55af7a749500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.553+0000 7f731cbb7640 1 -- 172.21.15.7:0/3427516317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af7a744010 msgr2=0x55af7a744430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.553+0000 7f731cbb7640 1 --2- 172.21.15.7:0/3427516317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af7a744010 0x55af7a744430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.553+0000 7f731cbb7640 1 -- 172.21.15.7:0/3427516317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7a744a00 msgr2=0x55af7a749500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.553+0000 7f731cbb7640 1 --2- 172.21.15.7:0/3427516317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7a744a00 0x55af7a749500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.553+0000 7f731cbb7640 1 -- 172.21.15.7:0/3427516317 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af7a6619b0 con 0x55af7a749b70 2024-06-10T01:10:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.553+0000 7f731d3b8640 1 --2- 172.21.15.7:0/3427516317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7a744a00 0x55af7a749500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:10:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.554+0000 7f731cbb7640 1 --2- 172.21.15.7:0/3427516317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af7a749b70 0x55af7a74c000 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7308031610 tx=0x7f7308005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c17766900614a00 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.554+0000 7f73037fe640 1 -- 172.21.15.7:0/3427516317 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308040070 con 0x55af7a749b70 2024-06-10T01:10:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.554+0000 7f73037fe640 1 -- 172.21.15.7:0/3427516317 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7308004030 con 0x55af7a749b70 2024-06-10T01:10:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.554+0000 7f730e7fc640 1 --2- 172.21.15.7:0/2563037669 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f73040a2c00 0x7f73040a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.554+0000 7f73037fe640 1 -- 172.21.15.7:0/3427516317 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308035040 con 0x55af7a749b70 2024-06-10T01:10:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.554+0000 7f730e7fc640 1 -- 172.21.15.7:0/2563037669 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f73140cf700 con 0x7f7318102500 2024-06-10T01:10:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.554+0000 7f731dbb9640 1 --2- 172.21.15.7:0/2563037669 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f73040a2c00 0x7f73040a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.555+0000 7f7322408740 1 -- 172.21.15.7:0/3427516317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af7a749b70 msgr2=0x55af7a74c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.555+0000 7f7322408740 1 --2- 172.21.15.7:0/3427516317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af7a749b70 0x55af7a74c000 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7308031610 tx=0x7f7308005de0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.555+0000 7f7322408740 1 -- 172.21.15.7:0/3427516317 shutdown_connections 2024-06-10T01:10:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.555+0000 7f7322408740 1 --2- 172.21.15.7:0/3427516317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af7a749b70 0x55af7a74c000 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.555+0000 7f7322408740 1 --2- 172.21.15.7:0/3427516317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7a744a00 0x55af7a749500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.555+0000 7f7322408740 1 --2- 172.21.15.7:0/3427516317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af7a744010 0x55af7a744430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.555+0000 7f7322408740 1 -- 172.21.15.7:0/3427516317 >> 172.21.15.7:0/3427516317 conn(0x55af7a73f680 msgr2=0x55af7a73fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.555+0000 7f731dbb9640 1 --2- 172.21.15.7:0/2563037669 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f73040a2c00 0x7f73040a50c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55af7a745260 tx=0x7f7310009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.555+0000 7f7322408740 1 -- 172.21.15.7:0/3427516317 shutdown_connections 2024-06-10T01:10:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.555+0000 7f7322408740 1 -- 172.21.15.7:0/3427516317 wait complete. 2024-06-10T01:10:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.555+0000 7f7322408740 1 -- start start 2024-06-10T01:10:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.555+0000 7f7322408740 1 Processor -- start 2024-06-10T01:10:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.556+0000 7f7322408740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af7a744010 0x55af7a74e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.556+0000 7f7322408740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7a744a00 0x55af7a74e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.556+0000 7f7322408740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af7a749b70 0x55af7a755790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.556+0000 7f7322408740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55af7a74cb10 con 0x55af7a744a00 2024-06-10T01:10:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.556+0000 7f7322408740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55af7a630ae0 con 0x55af7a749b70 2024-06-10T01:10:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.556+0000 7f7322408740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55af7a74cc90 con 0x55af7a744010 2024-06-10T01:10:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.556+0000 7f731d3b8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af7a744010 0x55af7a74e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.556+0000 7f731cbb7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7a744a00 0x55af7a74e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.556+0000 7f731d3b8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af7a744010 0x55af7a74e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:39198/0 (socket says 172.21.15.7:39198) 2024-06-10T01:10:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.556+0000 7f7322408740 10 client.0 _ll_register_callbacks cb 0x55af7a6699b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.556+0000 7f731d3b8640 1 -- 172.21.15.7:0/1602627077 learned_addr learned my addr 172.21.15.7:0/1602627077 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.556+0000 7f731dbb9640 1 --2- 172.21.15.7:0/1602627077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af7a749b70 0x55af7a755790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.556+0000 7f731d3b8640 1 -- 172.21.15.7:0/1602627077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af7a749b70 msgr2=0x55af7a755790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.556+0000 7f731d3b8640 1 --2- 172.21.15.7:0/1602627077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af7a749b70 0x55af7a755790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.556+0000 7f731d3b8640 1 -- 172.21.15.7:0/1602627077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7a744a00 msgr2=0x55af7a74e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.556+0000 7f731d3b8640 1 --2- 172.21.15.7:0/1602627077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7a744a00 0x55af7a74e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.556+0000 7f731d3b8640 1 -- 172.21.15.7:0/1602627077 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af7a3bb210 con 0x55af7a744010 2024-06-10T01:10:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.556+0000 7f731cbb7640 1 --2- 172.21.15.7:0/1602627077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7a744a00 0x55af7a74e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.556+0000 7f731dbb9640 1 --2- 172.21.15.7:0/1602627077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af7a749b70 0x55af7a755790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:10:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.557+0000 7f731d3b8640 1 --2- 172.21.15.7:0/1602627077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af7a744010 0x55af7a74e210 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f731400c6a0 tx=0x7f731409b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bef1c889eb8566f1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.557+0000 7f7300ff9640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:10:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.557+0000 7f7300ff9640 1 -- 172.21.15.7:0/1602627077 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314086050 con 0x55af7a744010 2024-06-10T01:10:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.557+0000 7f7300ff9640 1 -- 172.21.15.7:0/1602627077 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f7314098250 con 0x55af7a744010 2024-06-10T01:10:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.557+0000 7f7322408740 1 -- 172.21.15.7:0/1602627077 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7308004030 con 0x55af7a744010 2024-06-10T01:10:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.557+0000 7f7300ff9640 1 -- 172.21.15.7:0/1602627077 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f731409ff90 con 0x55af7a744010 2024-06-10T01:10:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.557+0000 7f7322408740 10 client.26203 fetch_fsmap learned FSMap version 32 2024-06-10T01:10:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.557+0000 7f7322408740 1 -- 172.21.15.7:0/1602627077 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55af7a487620 con 0x55af7a744010 2024-06-10T01:10:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.557+0000 7f7300ff9640 1 -- 172.21.15.7:0/1602627077 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7308004030 con 0x55af7a744010 2024-06-10T01:10:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.557+0000 7f7300ff9640 1 -- 172.21.15.7:0/1602627077 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f73140cfbb0 con 0x55af7a744010 2024-06-10T01:10:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.557+0000 7f7322408740 10 client.26203 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:10:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.557+0000 7f7322408740 10 client.26203 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:10:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.557+0000 7f7322408740 20 client.26203 populate_metadata read hostname 'smithi007' 2024-06-10T01:10:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.557+0000 7f7322408740 1 --2- 172.21.15.7:0/1602627077 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55af7a75a560 0x55af7a75c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.557+0000 7f7322408740 4 client.26203 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:10:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.557+0000 7f7322408740 1 -- 172.21.15.7:0/1602627077 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af7a6baf10 con 0x55af7a75a560 2024-06-10T01:10:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.558+0000 7f731cbb7640 1 --2- 172.21.15.7:0/1602627077 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55af7a75a560 0x55af7a75c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.558+0000 7f7300ff9640 10 client.26203 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.558+0000 7f731cbb7640 1 --2- 172.21.15.7:0/1602627077 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55af7a75a560 0x55af7a75c9f0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f856d6a038f1fcc4 server_cookie=f3558d601e16c97c in_seq=0 out_seq=0 2024-06-10T01:10:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.559+0000 7f7300ff9640 1 -- 172.21.15.7:0/1602627077 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55af7a6baf10 con 0x55af7a75a560 2024-06-10T01:10:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.559+0000 7f7300ff9640 10 client.26203 handle_command_reply: tid=0 2024-06-10T01:10:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.559+0000 7f7322408740 1 client.26203 shutdown 2024-06-10T01:10:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.559+0000 7f7322408740 1 -- 172.21.15.7:0/1602627077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af7a744010 msgr2=0x55af7a74e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.559+0000 7f7322408740 1 --2- 172.21.15.7:0/1602627077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af7a744010 0x55af7a74e210 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f731400c6a0 tx=0x7f731409b7e0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.559+0000 7f7322408740 1 -- 172.21.15.7:0/1602627077 shutdown_connections 2024-06-10T01:10:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.559+0000 7f7322408740 1 --2- 172.21.15.7:0/1602627077 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55af7a75a560 0x55af7a75c9f0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.559+0000 7f7322408740 1 --2- 172.21.15.7:0/1602627077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af7a749b70 0x55af7a755790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.559+0000 7f7300ff9640 0 client.26203 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.559+0000 7f7322408740 1 --2- 172.21.15.7:0/1602627077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7a744a00 0x55af7a74e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.559+0000 7f7322408740 1 --2- 172.21.15.7:0/1602627077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af7a744010 0x55af7a74e210 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.559+0000 7f7322408740 1 -- 172.21.15.7:0/1602627077 >> 172.21.15.7:0/1602627077 conn(0x55af7a73f680 msgr2=0x55af7a73e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.559+0000 7f7322408740 1 -- 172.21.15.7:0/1602627077 shutdown_connections 2024-06-10T01:10:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.559+0000 7f7322408740 1 -- 172.21.15.7:0/1602627077 wait complete. 2024-06-10T01:10:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.559+0000 7f7322408740 20 client.26203 trim_cache size 0 max 16384 2024-06-10T01:10:54.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.565+0000 7fcd66b38700 1 -- 172.21.15.7:0/778038297 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd6002dca0 con 0x7fcd6010a6a0 2024-06-10T01:10:54.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.565+0000 7fcd5dffb700 1 -- 172.21.15.7:0/778038297 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11855 (secure 0 0 0) 0x7fcd5809d930 con 0x7fcd6010a6a0 2024-06-10T01:10:54.567 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:10:54.567 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:10:42.213996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:10:54.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.569+0000 7fcd66b38700 1 -- 172.21.15.7:0/778038297 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcd500a2c20 msgr2=0x7fcd500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:54.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.569+0000 7fcd66b38700 1 --2- 172.21.15.7:0/778038297 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcd500a2c20 0x7fcd500a50e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcd54009b10 tx=0x7fcd54009260 comp rx=0 tx=0).stop 2024-06-10T01:10:54.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.569+0000 7fcd66b38700 1 -- 172.21.15.7:0/778038297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd6010a6a0 msgr2=0x7fcd6019f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:54.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.569+0000 7fcd66b38700 1 --2- 172.21.15.7:0/778038297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd6010a6a0 0x7fcd6019f750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcd5800b780 tx=0x7fcd58004d00 comp rx=0 tx=0).stop 2024-06-10T01:10:54.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.569+0000 7fcd66b38700 1 -- 172.21.15.7:0/778038297 shutdown_connections 2024-06-10T01:10:54.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.569+0000 7fcd66b38700 1 --2- 172.21.15.7:0/778038297 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcd500a2c20 0x7fcd500a50e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.569+0000 7fcd66b38700 1 --2- 172.21.15.7:0/778038297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd6010a6a0 0x7fcd6019f750 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.569+0000 7fcd66b38700 1 --2- 172.21.15.7:0/778038297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd601046a0 0x7fcd6019ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.569+0000 7fcd66b38700 1 --2- 172.21.15.7:0/778038297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd60105060 0x7fcd6019b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.569+0000 7fcd66b38700 1 -- 172.21.15.7:0/778038297 >> 172.21.15.7:0/778038297 conn(0x7fcd60100170 msgr2=0x7fcd601006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:54.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.569+0000 7fcd66b38700 1 -- 172.21.15.7:0/778038297 shutdown_connections 2024-06-10T01:10:54.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:54.569+0000 7fcd66b38700 1 -- 172.21.15.7:0/778038297 wait complete. 2024-06-10T01:10:54.568 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 32 2024-06-10T01:10:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.580+0000 7f7322408740 1 Processor -- start 2024-06-10T01:10:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.580+0000 7f7322408740 1 -- start start 2024-06-10T01:10:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.580+0000 7f7322408740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7a744010 0x55af7a749240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.581+0000 7f7322408740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af7a744a00 0x55af7a755750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.581+0000 7f7322408740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af7a749b70 0x55af7a742590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.581+0000 7f7322408740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f73181aefb0 con 0x55af7a744010 2024-06-10T01:10:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.581+0000 7f7322408740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f73181ac880 con 0x55af7a744a00 2024-06-10T01:10:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.581+0000 7f7322408740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f73181aa6d0 con 0x55af7a749b70 2024-06-10T01:10:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.581+0000 7f731cbb7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af7a744a00 0x55af7a755750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.581+0000 7f731d3b8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7a744010 0x55af7a749240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.581+0000 7f731cbb7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af7a744a00 0x55af7a755750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:57390/0 (socket says 172.21.15.7:57390) 2024-06-10T01:10:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.581+0000 7f731cbb7640 1 -- 172.21.15.7:0/2928864229 learned_addr learned my addr 172.21.15.7:0/2928864229 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.581+0000 7f731dbb9640 1 --2- 172.21.15.7:0/2928864229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af7a749b70 0x55af7a742590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.581+0000 7f731cbb7640 1 -- 172.21.15.7:0/2928864229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af7a749b70 msgr2=0x55af7a742590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.581+0000 7f731cbb7640 1 --2- 172.21.15.7:0/2928864229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af7a749b70 0x55af7a742590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.581+0000 7f731cbb7640 1 -- 172.21.15.7:0/2928864229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7a744010 msgr2=0x55af7a749240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.581+0000 7f731cbb7640 1 --2- 172.21.15.7:0/2928864229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7a744010 0x55af7a749240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.581+0000 7f731cbb7640 1 -- 172.21.15.7:0/2928864229 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af7a73f4b0 con 0x55af7a744a00 2024-06-10T01:10:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.581+0000 7f731cbb7640 1 --2- 172.21.15.7:0/2928864229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af7a744a00 0x55af7a755750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7308033000 tx=0x7f7308047040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ce55929b6eeb09f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.582+0000 7f7302ffd640 1 -- 172.21.15.7:0/2928864229 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308040040 con 0x55af7a744a00 2024-06-10T01:10:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.582+0000 7f7302ffd640 1 -- 172.21.15.7:0/2928864229 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73080318b0 con 0x55af7a744a00 2024-06-10T01:10:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.582+0000 7f7302ffd640 1 -- 172.21.15.7:0/2928864229 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308035040 con 0x55af7a744a00 2024-06-10T01:10:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.582+0000 7f7322408740 1 -- 172.21.15.7:0/2928864229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af7a744a00 msgr2=0x55af7a755750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.582+0000 7f7322408740 1 --2- 172.21.15.7:0/2928864229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af7a744a00 0x55af7a755750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7308033000 tx=0x7f7308047040 comp rx=0 tx=0).stop 2024-06-10T01:10:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.583+0000 7f7322408740 1 -- 172.21.15.7:0/2928864229 shutdown_connections 2024-06-10T01:10:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.583+0000 7f7322408740 1 --2- 172.21.15.7:0/2928864229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af7a749b70 0x55af7a742590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.583+0000 7f7322408740 1 --2- 172.21.15.7:0/2928864229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af7a744a00 0x55af7a755750 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.583+0000 7f7322408740 1 --2- 172.21.15.7:0/2928864229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7a744010 0x55af7a749240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.583+0000 7f7322408740 1 -- 172.21.15.7:0/2928864229 >> 172.21.15.7:0/2928864229 conn(0x55af7a73f680 msgr2=0x55af7a73fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.583+0000 7f7322408740 1 -- 172.21.15.7:0/2928864229 shutdown_connections 2024-06-10T01:10:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.583+0000 7f7322408740 1 -- 172.21.15.7:0/2928864229 wait complete. 2024-06-10T01:10:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.583+0000 7f7322408740 1 -- start start 2024-06-10T01:10:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.583+0000 7f7322408740 1 Processor -- start 2024-06-10T01:10:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.583+0000 7f7322408740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7a744010 0x55af7a7554c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.583+0000 7f7322408740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af7a744a00 0x55af7a751f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.584+0000 7f7322408740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af7a749b70 0x55af7a7524a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.584+0000 7f7322408740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f73181aad40 con 0x55af7a744010 2024-06-10T01:10:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.584+0000 7f7322408740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f73181ac1b0 con 0x55af7a744a00 2024-06-10T01:10:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.584+0000 7f7322408740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f73181ab410 con 0x55af7a749b70 2024-06-10T01:10:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.584+0000 7f7322408740 10 client.0 _ll_register_callbacks cb 0x7f731400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.584+0000 7f731cbb7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af7a744a00 0x55af7a751f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.584+0000 7f731dbb9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af7a749b70 0x55af7a7524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.584+0000 7f731cbb7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af7a744a00 0x55af7a751f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:57392/0 (socket says 172.21.15.7:57392) 2024-06-10T01:10:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.584+0000 7f731cbb7640 1 -- 172.21.15.7:0/707311174 learned_addr learned my addr 172.21.15.7:0/707311174 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.584+0000 7f731d3b8640 1 --2- 172.21.15.7:0/707311174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7a744010 0x55af7a7554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.584+0000 7f731cbb7640 1 -- 172.21.15.7:0/707311174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af7a749b70 msgr2=0x55af7a7524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.584+0000 7f731cbb7640 1 --2- 172.21.15.7:0/707311174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af7a749b70 0x55af7a7524a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.584+0000 7f731cbb7640 1 -- 172.21.15.7:0/707311174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7a744010 msgr2=0x55af7a7554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.584+0000 7f731cbb7640 1 --2- 172.21.15.7:0/707311174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7a744010 0x55af7a7554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.584+0000 7f731cbb7640 1 -- 172.21.15.7:0/707311174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af7a752b40 con 0x55af7a744a00 2024-06-10T01:10:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.584+0000 7f731dbb9640 1 --2- 172.21.15.7:0/707311174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af7a749b70 0x55af7a7524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.584+0000 7f731cbb7640 1 --2- 172.21.15.7:0/707311174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af7a744a00 0x55af7a751f60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f730802a910 tx=0x7f7308005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9489fcf58c57235f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.584+0000 7f7300ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:10:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.585+0000 7f7300ff9640 1 -- 172.21.15.7:0/707311174 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308035030 con 0x55af7a744a00 2024-06-10T01:10:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.585+0000 7f7300ff9640 1 -- 172.21.15.7:0/707311174 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f7308004250 con 0x55af7a744a00 2024-06-10T01:10:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.585+0000 7f7322408740 1 -- 172.21.15.7:0/707311174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73080318b0 con 0x55af7a744a00 2024-06-10T01:10:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.585+0000 7f7300ff9640 1 -- 172.21.15.7:0/707311174 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7308066050 con 0x55af7a744a00 2024-06-10T01:10:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.585+0000 7f7322408740 10 client.26180 fetch_fsmap learned FSMap version 32 2024-06-10T01:10:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.585+0000 7f7322408740 1 -- 172.21.15.7:0/707311174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55af7a7c4400 con 0x55af7a744a00 2024-06-10T01:10:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.585+0000 7f7300ff9640 1 -- 172.21.15.7:0/707311174 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73080318b0 con 0x55af7a744a00 2024-06-10T01:10:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.585+0000 7f7300ff9640 1 -- 172.21.15.7:0/707311174 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f7308061030 con 0x55af7a744a00 2024-06-10T01:10:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.585+0000 7f7322408740 10 client.26180 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:10:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.585+0000 7f7322408740 10 client.26180 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:10:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.585+0000 7f7322408740 20 client.26180 populate_metadata read hostname 'smithi007' 2024-06-10T01:10:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.585+0000 7f7322408740 1 --2- 172.21.15.7:0/707311174 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55af7a7c49b0 0x55af7a7c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.585+0000 7f7322408740 4 client.26180 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:10:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.585+0000 7f7322408740 1 -- 172.21.15.7:0/707311174 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55af7a6b4c20 con 0x55af7a7c49b0 2024-06-10T01:10:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.586+0000 7f731d3b8640 1 --2- 172.21.15.7:0/707311174 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55af7a7c49b0 0x55af7a7c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.586+0000 7f7300ff9640 10 client.26180 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.586+0000 7f731d3b8640 1 --2- 172.21.15.7:0/707311174 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55af7a7c49b0 0x55af7a7c6dd0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7198c76a8436e10d server_cookie=99f0e3f9c44d3db7 in_seq=0 out_seq=0 2024-06-10T01:10:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.696+0000 7f7300ff9640 1 -- 172.21.15.7:0/707311174 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55af7a6b4c20 con 0x55af7a7c49b0 2024-06-10T01:10:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.696+0000 7f7300ff9640 10 client.26180 handle_command_reply: tid=0 2024-06-10T01:10:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.696+0000 7f7322408740 1 client.26180 shutdown 2024-06-10T01:10:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.696+0000 7f7322408740 1 -- 172.21.15.7:0/707311174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af7a744a00 msgr2=0x55af7a751f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.696+0000 7f7322408740 1 --2- 172.21.15.7:0/707311174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af7a744a00 0x55af7a751f60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f730802a910 tx=0x7f7308005e40 comp rx=0 tx=0).stop 2024-06-10T01:10:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.696+0000 7f7322408740 1 -- 172.21.15.7:0/707311174 shutdown_connections 2024-06-10T01:10:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.696+0000 7f7322408740 1 --2- 172.21.15.7:0/707311174 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55af7a7c49b0 0x55af7a7c6dd0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.696+0000 7f7322408740 1 --2- 172.21.15.7:0/707311174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af7a749b70 0x55af7a7524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.696+0000 7f7322408740 1 --2- 172.21.15.7:0/707311174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af7a744a00 0x55af7a751f60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.696+0000 7f7322408740 1 --2- 172.21.15.7:0/707311174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7a744010 0x55af7a7554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.696+0000 7f7300ff9640 0 client.26180 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.696+0000 7f7322408740 1 -- 172.21.15.7:0/707311174 >> 172.21.15.7:0/707311174 conn(0x55af7a73f680 msgr2=0x55af7a745be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:54.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.697+0000 7f7322408740 1 -- 172.21.15.7:0/707311174 shutdown_connections 2024-06-10T01:10:54.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.697+0000 7f7322408740 1 -- 172.21.15.7:0/707311174 wait complete. 2024-06-10T01:10:54.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.697+0000 7f7322408740 20 client.26180 trim_cache size 0 max 16384 2024-06-10T01:10:54.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:10:54.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:10:54.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:10:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.700+0000 7f731f61c640 1 -- 172.21.15.7:0/2563037669 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f73040a2c00 msgr2=0x7f73040a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.700+0000 7f731f61c640 1 --2- 172.21.15.7:0/2563037669 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f73040a2c00 0x7f73040a50c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55af7a745260 tx=0x7f7310009500 comp rx=0 tx=0).stop 2024-06-10T01:10:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.700+0000 7f731f61c640 1 -- 172.21.15.7:0/2563037669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7318102500 msgr2=0x7f7318101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.700+0000 7f731f61c640 1 --2- 172.21.15.7:0/2563037669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7318102500 0x7f7318101e70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f731400e9f0 tx=0x7f731400edf0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.700+0000 7f731f61c640 1 -- 172.21.15.7:0/2563037669 shutdown_connections 2024-06-10T01:10:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.700+0000 7f731f61c640 1 --2- 172.21.15.7:0/2563037669 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f73040a2c00 0x7f73040a50c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.700+0000 7f731f61c640 1 --2- 172.21.15.7:0/2563037669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7318107760 0x7f73181009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.700+0000 7f731f61c640 1 --2- 172.21.15.7:0/2563037669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7318104e30 0x7f73181004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.700+0000 7f731f61c640 1 --2- 172.21.15.7:0/2563037669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7318102500 0x7f7318101e70 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.700+0000 7f731f61c640 1 -- 172.21.15.7:0/2563037669 >> 172.21.15.7:0/2563037669 conn(0x7f73180fbe70 msgr2=0x7f7318102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.700+0000 7f731f61c640 1 -- 172.21.15.7:0/2563037669 shutdown_connections 2024-06-10T01:10:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:10:54.700+0000 7f731f61c640 1 -- 172.21.15.7:0/2563037669 wait complete. 2024-06-10T01:10:54.717 INFO:tasks.workunit.client.0.smithi007.stdout: 1 334679 9.08 MB/sec execute 95 sec latency 42.295 ms 2024-06-10T01:10:54.963 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:10:55.317 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db / --timeout 90 --expiration 149.45 --await-for 45.0' 2024-06-10T01:10:55.317 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db / --timeout 90 --expiration 149.45 --await-for 45.0 2024-06-10T01:10:55.363 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:10:55.564 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:55 smithi007 ceph-mon[32398]: pgmap v579: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 281 op/s 2024-06-10T01:10:55.565 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:55 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/778038297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:10:55.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:55 smithi175 ceph-mon[36636]: pgmap v579: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 281 op/s 2024-06-10T01:10:55.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:55 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/778038297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:10:55.717 INFO:tasks.workunit.client.0.smithi007.stdout: 1 337358 9.16 MB/sec execute 96 sec latency 14.576 ms 2024-06-10T01:10:55.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:55 smithi078 ceph-mon[36683]: pgmap v579: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 281 op/s 2024-06-10T01:10:55.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:55 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/778038297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:10:56.718 INFO:tasks.workunit.client.0.smithi007.stdout: 1 339413 9.14 MB/sec execute 97 sec latency 13.433 ms 2024-06-10T01:10:57.127 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:10:57.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:57 smithi007 ceph-mon[32398]: pgmap v580: 129 pgs: 129 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 290 op/s 2024-06-10T01:10:57.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:57 smithi175 ceph-mon[36636]: pgmap v580: 129 pgs: 129 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 290 op/s 2024-06-10T01:10:57.718 INFO:tasks.workunit.client.0.smithi007.stdout: 1 341088 9.17 MB/sec execute 98 sec latency 16.853 ms 2024-06-10T01:10:57.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:57 smithi078 ceph-mon[36683]: pgmap v580: 129 pgs: 129 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 290 op/s 2024-06-10T01:10:58.718 INFO:tasks.workunit.client.0.smithi007.stdout: 1 343443 9.20 MB/sec execute 99 sec latency 30.789 ms 2024-06-10T01:10:59.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.449+0000 7f9e0f390700 1 -- 172.21.15.7:0/2482623514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e08107400 msgr2=0x7f9e081077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:59.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.449+0000 7f9e0f390700 1 --2- 172.21.15.7:0/2482623514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e08107400 0x7f9e081077e0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f9e04009990 tx=0x7f9e04009ca0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.449+0000 7f9e0f390700 1 -- 172.21.15.7:0/2482623514 shutdown_connections 2024-06-10T01:10:59.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.449+0000 7f9e0f390700 1 --2- 172.21.15.7:0/2482623514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e08101d50 0x7f9e0810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.449+0000 7f9e0f390700 1 --2- 172.21.15.7:0/2482623514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e08107400 0x7f9e081077e0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.449+0000 7f9e0f390700 1 --2- 172.21.15.7:0/2482623514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e08101390 0x7f9e08101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.449+0000 7f9e0f390700 1 -- 172.21.15.7:0/2482623514 >> 172.21.15.7:0/2482623514 conn(0x7f9e08078340 msgr2=0x7f9e08078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:59.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.449+0000 7f9e0f390700 1 -- 172.21.15.7:0/2482623514 shutdown_connections 2024-06-10T01:10:59.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.449+0000 7f9e0f390700 1 -- 172.21.15.7:0/2482623514 wait complete. 2024-06-10T01:10:59.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.450+0000 7f9e0f390700 1 Processor -- start 2024-06-10T01:10:59.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.450+0000 7f9e0f390700 1 -- start start 2024-06-10T01:10:59.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.450+0000 7f9e0f390700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e08101390 0x7f9e0819e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:59.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.450+0000 7f9e0f390700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e08101d50 0x7f9e0819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:59.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.450+0000 7f9e0f390700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e08107400 0x7f9e08198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:59.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.450+0000 7f9e0f390700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9e08111670 con 0x7f9e08101d50 2024-06-10T01:10:59.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.450+0000 7f9e0f390700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9e081114f0 con 0x7f9e08107400 2024-06-10T01:10:59.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.450+0000 7f9e0f390700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9e081117f0 con 0x7f9e08101390 2024-06-10T01:10:59.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.450+0000 7f9e0c92b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e08101d50 0x7f9e0819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:59.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.450+0000 7f9e0c92b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e08101d50 0x7f9e0819edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52764/0 (socket says 172.21.15.7:52764) 2024-06-10T01:10:59.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.450+0000 7f9e0c92b700 1 -- 172.21.15.7:0/2352032387 learned_addr learned my addr 172.21.15.7:0/2352032387 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:59.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.450+0000 7f9e0d12c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e08101390 0x7f9e0819e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:59.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.450+0000 7f9e0d92d700 1 --2- 172.21.15.7:0/2352032387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e08107400 0x7f9e08198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:59.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.450+0000 7f9e0c92b700 1 -- 172.21.15.7:0/2352032387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e08101390 msgr2=0x7f9e0819e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:59.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.450+0000 7f9e0c92b700 1 --2- 172.21.15.7:0/2352032387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e08101390 0x7f9e0819e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.450+0000 7f9e0c92b700 1 -- 172.21.15.7:0/2352032387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e08107400 msgr2=0x7f9e08198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:59.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.450+0000 7f9e0c92b700 1 --2- 172.21.15.7:0/2352032387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e08107400 0x7f9e08198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.450+0000 7f9e0c92b700 1 -- 172.21.15.7:0/2352032387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e081991b0 con 0x7f9e08101d50 2024-06-10T01:10:59.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.450+0000 7f9e0d92d700 1 --2- 172.21.15.7:0/2352032387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e08107400 0x7f9e08198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:59.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.451+0000 7f9e0c92b700 1 --2- 172.21.15.7:0/2352032387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e08101d50 0x7f9e0819edd0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f9df800d980 tx=0x7f9df800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:59.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.451+0000 7f9dfe7fc700 1 -- 172.21.15.7:0/2352032387 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df8016070 con 0x7f9e08101d50 2024-06-10T01:10:59.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.451+0000 7f9dfe7fc700 1 -- 172.21.15.7:0/2352032387 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9df8004d10 con 0x7f9e08101d50 2024-06-10T01:10:59.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.451+0000 7f9e0f390700 1 -- 172.21.15.7:0/2352032387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e081994a0 con 0x7f9e08101d50 2024-06-10T01:10:59.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.451+0000 7f9e0f390700 1 -- 172.21.15.7:0/2352032387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e081a5760 con 0x7f9e08101d50 2024-06-10T01:10:59.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.451+0000 7f9dfe7fc700 1 -- 172.21.15.7:0/2352032387 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df8005670 con 0x7f9e08101d50 2024-06-10T01:10:59.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.453+0000 7f9dfe7fc700 1 -- 172.21.15.7:0/2352032387 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9df8004750 con 0x7f9e08101d50 2024-06-10T01:10:59.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.454+0000 7f9dfe7fc700 1 --2- 172.21.15.7:0/2352032387 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9df40a2c20 0x7f9df40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:59.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.454+0000 7f9e0d12c700 1 --2- 172.21.15.7:0/2352032387 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9df40a2c20 0x7f9df40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:59.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.454+0000 7f9dfe7fc700 1 -- 172.21.15.7:0/2352032387 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f9df80cf150 con 0x7f9e08101d50 2024-06-10T01:10:59.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.454+0000 7f9e0d12c700 1 --2- 172.21.15.7:0/2352032387 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9df40a2c20 0x7f9df40a50e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9e08199b90 tx=0x7f9e04005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:59.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.455+0000 7f9e19c13b80 1 Processor -- start 2024-06-10T01:10:59.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.455+0000 7f9e19c13b80 1 -- start start 2024-06-10T01:10:59.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.455+0000 7f9e19c13b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a22f78370 0x555a22f78750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:59.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.455+0000 7f9e19c13b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a22f78c90 0x555a22f72670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:59.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.455+0000 7f9e19c13b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a22f72bb0 0x555a22f72ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:59.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.455+0000 7f9e19c13b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555a22e60f60 con 0x555a22f78370 2024-06-10T01:10:59.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.455+0000 7f9e19c13b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555a22ebf850 con 0x555a22f78c90 2024-06-10T01:10:59.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.455+0000 7f9e19c13b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555a22e27460 con 0x555a22f72bb0 2024-06-10T01:10:59.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.455+0000 7f9e0d12c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a22f78370 0x555a22f78750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:59.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.455+0000 7f9e0c92b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a22f78c90 0x555a22f72670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:59.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.455+0000 7f9e0d12c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a22f78370 0x555a22f78750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52766/0 (socket says 172.21.15.7:52766) 2024-06-10T01:10:59.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.455+0000 7f9e0d12c700 1 -- 172.21.15.7:0/1193081262 learned_addr learned my addr 172.21.15.7:0/1193081262 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:59.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.455+0000 7f9e0d92d700 1 --2- 172.21.15.7:0/1193081262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a22f72bb0 0x555a22f72ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:59.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.456+0000 7f9e0c92b700 1 -- 172.21.15.7:0/1193081262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a22f72bb0 msgr2=0x555a22f72ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:59.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.456+0000 7f9e0c92b700 1 --2- 172.21.15.7:0/1193081262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a22f72bb0 0x555a22f72ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.456+0000 7f9e0c92b700 1 -- 172.21.15.7:0/1193081262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a22f78370 msgr2=0x555a22f78750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:59.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.456+0000 7f9e0c92b700 1 --2- 172.21.15.7:0/1193081262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a22f78370 0x555a22f78750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.456+0000 7f9e0c92b700 1 -- 172.21.15.7:0/1193081262 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a22e0e8d0 con 0x555a22f78c90 2024-06-10T01:10:59.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.456+0000 7f9e0c92b700 1 --2- 172.21.15.7:0/1193081262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a22f78c90 0x555a22f72670 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9df800d950 tx=0x7f9df809b1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3178d4e9a1e1be0a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:59.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.456+0000 7f9df37fe700 1 -- 172.21.15.7:0/1193081262 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df809bd30 con 0x555a22f78c90 2024-06-10T01:10:59.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.456+0000 7f9df37fe700 1 -- 172.21.15.7:0/1193081262 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9df809b630 con 0x555a22f78c90 2024-06-10T01:10:59.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.456+0000 7f9df37fe700 1 -- 172.21.15.7:0/1193081262 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df8004340 con 0x555a22f78c90 2024-06-10T01:10:59.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.457+0000 7f9e19c13b80 1 -- 172.21.15.7:0/1193081262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a22f78c90 msgr2=0x555a22f72670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:59.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.457+0000 7f9e19c13b80 1 --2- 172.21.15.7:0/1193081262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a22f78c90 0x555a22f72670 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9df800d950 tx=0x7f9df809b1f0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.457+0000 7f9e19c13b80 1 -- 172.21.15.7:0/1193081262 shutdown_connections 2024-06-10T01:10:59.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.457+0000 7f9e19c13b80 1 --2- 172.21.15.7:0/1193081262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a22f72bb0 0x555a22f72ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.457+0000 7f9e19c13b80 1 --2- 172.21.15.7:0/1193081262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a22f78370 0x555a22f78750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.457+0000 7f9e19c13b80 1 --2- 172.21.15.7:0/1193081262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a22f78c90 0x555a22f72670 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.457+0000 7f9e19c13b80 1 -- 172.21.15.7:0/1193081262 >> 172.21.15.7:0/1193081262 conn(0x555a22e9b080 msgr2=0x555a22f69220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:59.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.457+0000 7f9e19c13b80 1 -- 172.21.15.7:0/1193081262 shutdown_connections 2024-06-10T01:10:59.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.457+0000 7f9e19c13b80 1 -- 172.21.15.7:0/1193081262 wait complete. 2024-06-10T01:10:59.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.457+0000 7f9e19c13b80 1 -- start start 2024-06-10T01:10:59.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.457+0000 7f9e19c13b80 1 Processor -- start 2024-06-10T01:10:59.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.458+0000 7f9e19c13b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a22f72bb0 0x555a22f7eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:59.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.458+0000 7f9e19c13b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a22f78370 0x555a22f7be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:59.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.458+0000 7f9e19c13b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a22f78c90 0x555a22f7c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:59.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.458+0000 7f9e19c13b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555a22e5ffc0 con 0x555a22f78c90 2024-06-10T01:10:59.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.458+0000 7f9e19c13b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555a22ebb810 con 0x555a22f78370 2024-06-10T01:10:59.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.458+0000 7f9e19c13b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555a22f7f680 con 0x555a22f72bb0 2024-06-10T01:10:59.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.458+0000 7f9e19c13b80 10 client.0 _ll_register_callbacks cb 0x555a22e555b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:59.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.458+0000 7f9e0d92d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a22f78c90 0x555a22f7c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:59.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.458+0000 7f9e0d12c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a22f72bb0 0x555a22f7eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:59.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.458+0000 7f9e0c92b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a22f78370 0x555a22f7be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:59.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.458+0000 7f9e0d92d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a22f78c90 0x555a22f7c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52774/0 (socket says 172.21.15.7:52774) 2024-06-10T01:10:59.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.458+0000 7f9e0d92d700 1 -- 172.21.15.7:0/3961538089 learned_addr learned my addr 172.21.15.7:0/3961538089 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:59.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.459+0000 7f9e0c92b700 1 -- 172.21.15.7:0/3961538089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a22f72bb0 msgr2=0x555a22f7eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:59.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.459+0000 7f9e0c92b700 1 --2- 172.21.15.7:0/3961538089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a22f72bb0 0x555a22f7eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.459+0000 7f9e0c92b700 1 -- 172.21.15.7:0/3961538089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a22f78c90 msgr2=0x555a22f7c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:59.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.459+0000 7f9e0c92b700 1 --2- 172.21.15.7:0/3961538089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a22f78c90 0x555a22f7c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.459+0000 7f9e0c92b700 1 -- 172.21.15.7:0/3961538089 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a22e85750 con 0x555a22f78370 2024-06-10T01:10:59.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.459+0000 7f9e0d92d700 1 --2- 172.21.15.7:0/3961538089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a22f78c90 0x555a22f7c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:59.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.459+0000 7f9e0c92b700 1 --2- 172.21.15.7:0/3961538089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a22f78370 0x555a22f7be90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9df809b100 tx=0x7f9df8007640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba0a1cfaf3c39d93 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:59.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.459+0000 7f9df0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:10:59.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.459+0000 7f9df0ff9700 1 -- 172.21.15.7:0/3961538089 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df809bd30 con 0x555a22f78370 2024-06-10T01:10:59.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.459+0000 7f9df0ff9700 1 -- 172.21.15.7:0/3961538089 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f9df8004a40 con 0x555a22f78370 2024-06-10T01:10:59.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.459+0000 7f9e19c13b80 1 -- 172.21.15.7:0/3961538089 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9df809b630 con 0x555a22f78370 2024-06-10T01:10:59.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.459+0000 7f9df0ff9700 1 -- 172.21.15.7:0/3961538089 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9df80a42a0 con 0x555a22f78370 2024-06-10T01:10:59.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.459+0000 7f9e19c13b80 10 client.26192 fetch_fsmap learned FSMap version 32 2024-06-10T01:10:59.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.459+0000 7f9e19c13b80 1 -- 172.21.15.7:0/3961538089 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555a22f89f70 con 0x555a22f78370 2024-06-10T01:10:59.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.459+0000 7f9df0ff9700 1 -- 172.21.15.7:0/3961538089 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9df809b630 con 0x555a22f78370 2024-06-10T01:10:59.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.459+0000 7f9df0ff9700 1 -- 172.21.15.7:0/3961538089 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f9df80a3e10 con 0x555a22f78370 2024-06-10T01:10:59.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.460+0000 7f9e19c13b80 10 client.26192 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:10:59.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.460+0000 7f9e19c13b80 10 client.26192 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:10:59.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.460+0000 7f9e19c13b80 20 client.26192 populate_metadata read hostname 'smithi007' 2024-06-10T01:10:59.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.460+0000 7f9e19c13b80 1 --2- 172.21.15.7:0/3961538089 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555a22f8c440 0x555a22f8e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:59.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.460+0000 7f9e19c13b80 4 client.26192 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:10:59.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.460+0000 7f9e19c13b80 1 -- 172.21.15.7:0/3961538089 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a22eccd50 con 0x555a22f8c440 2024-06-10T01:10:59.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.460+0000 7f9e0d12c700 1 --2- 172.21.15.7:0/3961538089 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555a22f8c440 0x555a22f8e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:59.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.460+0000 7f9e0d12c700 1 --2- 172.21.15.7:0/3961538089 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555a22f8c440 0x555a22f8e830 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2dfee0beb8e0a34 server_cookie=14e7a5269478c7b9 in_seq=0 out_seq=0 2024-06-10T01:10:59.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.460+0000 7f9df0ff9700 10 client.26192 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:59.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.461+0000 7f9df0ff9700 1 -- 172.21.15.7:0/3961538089 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555a22eccd50 con 0x555a22f8c440 2024-06-10T01:10:59.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.461+0000 7f9df0ff9700 10 client.26192 handle_command_reply: tid=0 2024-06-10T01:10:59.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.461+0000 7f9e19c13b80 1 client.26192 shutdown 2024-06-10T01:10:59.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.461+0000 7f9e19c13b80 1 -- 172.21.15.7:0/3961538089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a22f78370 msgr2=0x555a22f7be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:59.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.461+0000 7f9e19c13b80 1 --2- 172.21.15.7:0/3961538089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a22f78370 0x555a22f7be90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9df809b100 tx=0x7f9df8007640 comp rx=0 tx=0).stop 2024-06-10T01:10:59.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.461+0000 7f9e19c13b80 1 -- 172.21.15.7:0/3961538089 shutdown_connections 2024-06-10T01:10:59.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.461+0000 7f9e19c13b80 1 --2- 172.21.15.7:0/3961538089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a22f78c90 0x555a22f7c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.461+0000 7f9e19c13b80 1 --2- 172.21.15.7:0/3961538089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a22f72bb0 0x555a22f7eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.461+0000 7f9e19c13b80 1 --2- 172.21.15.7:0/3961538089 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555a22f8c440 0x555a22f8e830 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.461+0000 7f9e19c13b80 1 --2- 172.21.15.7:0/3961538089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a22f78370 0x555a22f7be90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.461+0000 7f9df0ff9700 0 client.26192 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:59.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.461+0000 7f9e19c13b80 1 -- 172.21.15.7:0/3961538089 >> 172.21.15.7:0/3961538089 conn(0x555a22e9b080 msgr2=0x555a22f68700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:59.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.461+0000 7f9e19c13b80 1 -- 172.21.15.7:0/3961538089 shutdown_connections 2024-06-10T01:10:59.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.461+0000 7f9e19c13b80 1 -- 172.21.15.7:0/3961538089 wait complete. 2024-06-10T01:10:59.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.462+0000 7f9e19c13b80 20 client.26192 trim_cache size 0 max 16384 2024-06-10T01:10:59.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.485+0000 7f9e19c13b80 1 Processor -- start 2024-06-10T01:10:59.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.485+0000 7f9e19c13b80 1 -- start start 2024-06-10T01:10:59.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.485+0000 7f9e19c13b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a22f72bb0 0x555a22f7e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:59.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.485+0000 7f9e19c13b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a22f78370 0x555a22f68e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:59.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.485+0000 7f9e19c13b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a22f78c90 0x555a22f693c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:59.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.485+0000 7f9e19c13b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9e081ae890 con 0x555a22f72bb0 2024-06-10T01:10:59.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.485+0000 7f9e19c13b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9e081ac120 con 0x555a22f78370 2024-06-10T01:10:59.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.485+0000 7f9e19c13b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9e081a9f70 con 0x555a22f78c90 2024-06-10T01:10:59.481 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:10:59 smithi007 ceph-mon[32398]: pgmap v581: 129 pgs: 129 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 265 op/s 2024-06-10T01:10:59.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.485+0000 7f9e0c92b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a22f78370 0x555a22f68e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:59.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.485+0000 7f9e0d92d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a22f78c90 0x555a22f693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:59.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.485+0000 7f9e0c92b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a22f78370 0x555a22f68e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:57440/0 (socket says 172.21.15.7:57440) 2024-06-10T01:10:59.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.485+0000 7f9e0d92d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a22f78c90 0x555a22f693c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:39258/0 (socket says 172.21.15.7:39258) 2024-06-10T01:10:59.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.485+0000 7f9e0c92b700 1 -- 172.21.15.7:0/2227060946 learned_addr learned my addr 172.21.15.7:0/2227060946 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:59.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.485+0000 7f9e0d12c700 1 --2- 172.21.15.7:0/2227060946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a22f72bb0 0x555a22f7e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:59.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.486+0000 7f9e0c92b700 1 -- 172.21.15.7:0/2227060946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a22f78c90 msgr2=0x555a22f693c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:59.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.486+0000 7f9e0c92b700 1 --2- 172.21.15.7:0/2227060946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a22f78c90 0x555a22f693c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.486+0000 7f9e0c92b700 1 -- 172.21.15.7:0/2227060946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a22f72bb0 msgr2=0x555a22f7e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:59.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.486+0000 7f9e0c92b700 1 --2- 172.21.15.7:0/2227060946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a22f72bb0 0x555a22f7e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.486+0000 7f9e0c92b700 1 -- 172.21.15.7:0/2227060946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a22f6abf0 con 0x555a22f78370 2024-06-10T01:10:59.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.486+0000 7f9e0d92d700 1 --2- 172.21.15.7:0/2227060946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a22f78c90 0x555a22f693c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:59.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.486+0000 7f9e0c92b700 1 --2- 172.21.15.7:0/2227060946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a22f78370 0x555a22f68e80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9df809b330 tx=0x7f9df8004640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9746c15cef99dd8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:59.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.486+0000 7f9df2ffd700 1 -- 172.21.15.7:0/2227060946 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df809b630 con 0x555a22f78370 2024-06-10T01:10:59.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.486+0000 7f9df2ffd700 1 -- 172.21.15.7:0/2227060946 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9df809f9e0 con 0x555a22f78370 2024-06-10T01:10:59.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.486+0000 7f9df2ffd700 1 -- 172.21.15.7:0/2227060946 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df809baf0 con 0x555a22f78370 2024-06-10T01:10:59.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.487+0000 7f9e19c13b80 1 -- 172.21.15.7:0/2227060946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a22f78370 msgr2=0x555a22f68e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:59.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.487+0000 7f9e19c13b80 1 --2- 172.21.15.7:0/2227060946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a22f78370 0x555a22f68e80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9df809b330 tx=0x7f9df8004640 comp rx=0 tx=0).stop 2024-06-10T01:10:59.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.487+0000 7f9e19c13b80 1 -- 172.21.15.7:0/2227060946 shutdown_connections 2024-06-10T01:10:59.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.487+0000 7f9e19c13b80 1 --2- 172.21.15.7:0/2227060946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a22f78c90 0x555a22f693c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.487+0000 7f9e19c13b80 1 --2- 172.21.15.7:0/2227060946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a22f72bb0 0x555a22f7e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.487+0000 7f9e19c13b80 1 --2- 172.21.15.7:0/2227060946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a22f78370 0x555a22f68e80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.487+0000 7f9e19c13b80 1 -- 172.21.15.7:0/2227060946 >> 172.21.15.7:0/2227060946 conn(0x555a22e9b080 msgr2=0x555a22f7b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:59.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.487+0000 7f9e19c13b80 1 -- 172.21.15.7:0/2227060946 shutdown_connections 2024-06-10T01:10:59.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.487+0000 7f9e19c13b80 1 -- 172.21.15.7:0/2227060946 wait complete. 2024-06-10T01:10:59.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.488+0000 7f9e19c13b80 1 -- start start 2024-06-10T01:10:59.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.488+0000 7f9e19c13b80 1 Processor -- start 2024-06-10T01:10:59.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.488+0000 7f9e19c13b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a22f72bb0 0x555a22f8f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:59.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.488+0000 7f9e19c13b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a22f78370 0x555a22f8fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:59.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.488+0000 7f9e19c13b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a22f78c90 0x555a22f90350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:59.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.488+0000 7f9e19c13b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9e081aa5e0 con 0x555a22f72bb0 2024-06-10T01:10:59.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.488+0000 7f9e19c13b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9e081aba50 con 0x555a22f78370 2024-06-10T01:10:59.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.488+0000 7f9e19c13b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9e081aacb0 con 0x555a22f78c90 2024-06-10T01:10:59.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.488+0000 7f9e0d12c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a22f72bb0 0x555a22f8f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:59.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.488+0000 7f9e19c13b80 10 client.0 _ll_register_callbacks cb 0x555a22e555b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:59.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.488+0000 7f9e0d12c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a22f72bb0 0x555a22f8f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52794/0 (socket says 172.21.15.7:52794) 2024-06-10T01:10:59.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.488+0000 7f9e0d12c700 1 -- 172.21.15.7:0/1787169789 learned_addr learned my addr 172.21.15.7:0/1787169789 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:10:59.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.488+0000 7f9e0c92b700 1 --2- 172.21.15.7:0/1787169789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a22f78370 0x555a22f8fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:59.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.488+0000 7f9e0d92d700 1 --2- 172.21.15.7:0/1787169789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a22f78c90 0x555a22f90350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:59.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.489+0000 7f9e0c92b700 1 -- 172.21.15.7:0/1787169789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a22f78c90 msgr2=0x555a22f90350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:59.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.489+0000 7f9e0c92b700 1 --2- 172.21.15.7:0/1787169789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a22f78c90 0x555a22f90350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.489+0000 7f9e0c92b700 1 -- 172.21.15.7:0/1787169789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a22f72bb0 msgr2=0x555a22f8f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:59.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.489+0000 7f9e0c92b700 1 --2- 172.21.15.7:0/1787169789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a22f72bb0 0x555a22f8f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.489+0000 7f9e0c92b700 1 -- 172.21.15.7:0/1787169789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a22ff5fe0 con 0x555a22f78370 2024-06-10T01:10:59.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.489+0000 7f9e0d92d700 1 --2- 172.21.15.7:0/1787169789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a22f78c90 0x555a22f90350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:10:59.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.489+0000 7f9e0d12c700 1 --2- 172.21.15.7:0/1787169789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a22f72bb0 0x555a22f8f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:59.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.489+0000 7f9e0c92b700 1 --2- 172.21.15.7:0/1787169789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a22f78370 0x555a22f8fe10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x555a22f7a5e0 tx=0x7f9df80cc940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1be471d1f681f13 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:59.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.489+0000 7f9df0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:10:59.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.489+0000 7f9df0ff9700 1 -- 172.21.15.7:0/1787169789 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df809af90 con 0x555a22f78370 2024-06-10T01:10:59.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.489+0000 7f9e19c13b80 1 -- 172.21.15.7:0/1787169789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9df809f9e0 con 0x555a22f78370 2024-06-10T01:10:59.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.489+0000 7f9df0ff9700 1 -- 172.21.15.7:0/1787169789 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f9df809ba90 con 0x555a22f78370 2024-06-10T01:10:59.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.489+0000 7f9df0ff9700 1 -- 172.21.15.7:0/1787169789 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9df809bd30 con 0x555a22f78370 2024-06-10T01:10:59.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.489+0000 7f9e19c13b80 10 client.26198 fetch_fsmap learned FSMap version 32 2024-06-10T01:10:59.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.489+0000 7f9e19c13b80 1 -- 172.21.15.7:0/1787169789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555a22ff5fe0 con 0x555a22f78370 2024-06-10T01:10:59.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.489+0000 7f9df0ff9700 1 -- 172.21.15.7:0/1787169789 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9df809f9e0 con 0x555a22f78370 2024-06-10T01:10:59.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.489+0000 7f9df0ff9700 1 -- 172.21.15.7:0/1787169789 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f9df809a4f0 con 0x555a22f78370 2024-06-10T01:10:59.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.490+0000 7f9e19c13b80 10 client.26198 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:10:59.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.490+0000 7f9e19c13b80 10 client.26198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:10:59.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.490+0000 7f9e19c13b80 20 client.26198 populate_metadata read hostname 'smithi007' 2024-06-10T01:10:59.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.490+0000 7f9e19c13b80 1 --2- 172.21.15.7:0/1787169789 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555a22ff74d0 0x555a22ff98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:59.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.490+0000 7f9e19c13b80 4 client.26198 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:10:59.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.490+0000 7f9e19c13b80 1 -- 172.21.15.7:0/1787169789 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555a22ec72d0 con 0x555a22ff74d0 2024-06-10T01:10:59.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.490+0000 7f9e0d12c700 1 --2- 172.21.15.7:0/1787169789 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555a22ff74d0 0x555a22ff98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:59.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.490+0000 7f9e0d12c700 1 --2- 172.21.15.7:0/1787169789 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555a22ff74d0 0x555a22ff98c0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12f645bc80dff317 server_cookie=d9fded4e73d93449 in_seq=0 out_seq=0 2024-06-10T01:10:59.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.490+0000 7f9df0ff9700 10 client.26198 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:59.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.490+0000 7f9df0ff9700 1 -- 172.21.15.7:0/1787169789 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555a22ec72d0 con 0x555a22ff74d0 2024-06-10T01:10:59.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.490+0000 7f9df0ff9700 10 client.26198 handle_command_reply: tid=0 2024-06-10T01:10:59.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.490+0000 7f9e19c13b80 1 client.26198 shutdown 2024-06-10T01:10:59.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.491+0000 7f9e19c13b80 1 -- 172.21.15.7:0/1787169789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a22f78370 msgr2=0x555a22f8fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:59.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.491+0000 7f9e19c13b80 1 --2- 172.21.15.7:0/1787169789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a22f78370 0x555a22f8fe10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x555a22f7a5e0 tx=0x7f9df80cc940 comp rx=0 tx=0).stop 2024-06-10T01:10:59.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.491+0000 7f9e19c13b80 1 -- 172.21.15.7:0/1787169789 shutdown_connections 2024-06-10T01:10:59.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.491+0000 7f9e19c13b80 1 --2- 172.21.15.7:0/1787169789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a22f78c90 0x555a22f90350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.491+0000 7f9e19c13b80 1 --2- 172.21.15.7:0/1787169789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a22f72bb0 0x555a22f8f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.491+0000 7f9e19c13b80 1 --2- 172.21.15.7:0/1787169789 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555a22ff74d0 0x555a22ff98c0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.491+0000 7f9e19c13b80 1 --2- 172.21.15.7:0/1787169789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a22f78370 0x555a22f8fe10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.491+0000 7f9df0ff9700 0 client.26198 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:10:59.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.491+0000 7f9e19c13b80 1 -- 172.21.15.7:0/1787169789 >> 172.21.15.7:0/1787169789 conn(0x555a22e9b080 msgr2=0x555a22f79960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:59.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.491+0000 7f9e19c13b80 1 -- 172.21.15.7:0/1787169789 shutdown_connections 2024-06-10T01:10:59.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.491+0000 7f9e19c13b80 1 -- 172.21.15.7:0/1787169789 wait complete. 2024-06-10T01:10:59.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.491+0000 7f9e19c13b80 20 client.26198 trim_cache size 0 max 16384 2024-06-10T01:10:59.491 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:10:59.491 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-10T01:10:59.491 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-10T01:10:59.491 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:10:59.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.491+0000 7f9e0f390700 1 -- 172.21.15.7:0/2352032387 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9df40a2c20 msgr2=0x7f9df40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:59.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.491+0000 7f9e0f390700 1 --2- 172.21.15.7:0/2352032387 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9df40a2c20 0x7f9df40a50e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9e08199b90 tx=0x7f9e04005960 comp rx=0 tx=0).stop 2024-06-10T01:10:59.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.491+0000 7f9e0f390700 1 -- 172.21.15.7:0/2352032387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e08101d50 msgr2=0x7f9e0819edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:59.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.491+0000 7f9e0f390700 1 --2- 172.21.15.7:0/2352032387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e08101d50 0x7f9e0819edd0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f9df800d980 tx=0x7f9df800dc90 comp rx=0 tx=0).stop 2024-06-10T01:10:59.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.491+0000 7f9e0f390700 1 -- 172.21.15.7:0/2352032387 shutdown_connections 2024-06-10T01:10:59.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.491+0000 7f9e0f390700 1 --2- 172.21.15.7:0/2352032387 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9df40a2c20 0x7f9df40a50e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.491+0000 7f9e0f390700 1 --2- 172.21.15.7:0/2352032387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e08107400 0x7f9e08198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.492+0000 7f9e0f390700 1 --2- 172.21.15.7:0/2352032387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e08101390 0x7f9e0819e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.492+0000 7f9e0f390700 1 --2- 172.21.15.7:0/2352032387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e08101d50 0x7f9e0819edd0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.492+0000 7f9e0f390700 1 -- 172.21.15.7:0/2352032387 >> 172.21.15.7:0/2352032387 conn(0x7f9e08078340 msgr2=0x7f9e081006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.492+0000 7f9e0f390700 1 -- 172.21.15.7:0/2352032387 shutdown_connections 2024-06-10T01:10:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:10:59.492+0000 7f9e0f390700 1 -- 172.21.15.7:0/2352032387 wait complete. 2024-06-10T01:10:59.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:10:59 smithi175 ceph-mon[36636]: pgmap v581: 129 pgs: 129 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 265 op/s 2024-06-10T01:10:59.718 INFO:tasks.workunit.client.0.smithi007.stdout: 1 344970 9.19 MB/sec execute 100 sec latency 20.922 ms 2024-06-10T01:10:59.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:10:59 smithi078 ceph-mon[36683]: pgmap v581: 129 pgs: 129 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 265 op/s 2024-06-10T01:11:00.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.013+0000 7f0c9507e700 1 -- 172.21.15.7:0/4281449327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c90103150 msgr2=0x7f0c901070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:00.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.013+0000 7f0c9507e700 1 --2- 172.21.15.7:0/4281449327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c90103150 0x7f0c901070b0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f0c780099c0 tx=0x7f0c78009cd0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.013+0000 7f0c9507e700 1 -- 172.21.15.7:0/4281449327 shutdown_connections 2024-06-10T01:11:00.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.013+0000 7f0c9507e700 1 --2- 172.21.15.7:0/4281449327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c90107790 0x7f0c9010f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.013+0000 7f0c9507e700 1 --2- 172.21.15.7:0/4281449327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c901027a0 0x7f0c90102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.013+0000 7f0c9507e700 1 --2- 172.21.15.7:0/4281449327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c90103150 0x7f0c901070b0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.013+0000 7f0c9507e700 1 -- 172.21.15.7:0/4281449327 >> 172.21.15.7:0/4281449327 conn(0x7f0c90079560 msgr2=0x7f0c90079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:00.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.013+0000 7f0c9507e700 1 -- 172.21.15.7:0/4281449327 shutdown_connections 2024-06-10T01:11:00.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.013+0000 7f0c9507e700 1 -- 172.21.15.7:0/4281449327 wait complete. 2024-06-10T01:11:00.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.014+0000 7f0c9507e700 1 Processor -- start 2024-06-10T01:11:00.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.014+0000 7f0c9507e700 1 -- start start 2024-06-10T01:11:00.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.015+0000 7f0c9507e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c901027a0 0x7f0c9019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:00.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.015+0000 7f0c9507e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c90103150 0x7f0c9019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:00.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.015+0000 7f0c9507e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c90107790 0x7f0c901989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:00.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.015+0000 7f0c9507e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0c901127b0 con 0x7f0c90103150 2024-06-10T01:11:00.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.015+0000 7f0c9507e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0c90112630 con 0x7f0c90107790 2024-06-10T01:11:00.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.015+0000 7f0c9507e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0c90112930 con 0x7f0c901027a0 2024-06-10T01:11:00.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.015+0000 7f0c8659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c90103150 0x7f0c9019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:00.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.015+0000 7f0c8ed9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c901027a0 0x7f0c9019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:00.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.015+0000 7f0c8659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c90103150 0x7f0c9019ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52816/0 (socket says 172.21.15.7:52816) 2024-06-10T01:11:00.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.015+0000 7f0c8659c700 1 -- 172.21.15.7:0/81716764 learned_addr learned my addr 172.21.15.7:0/81716764 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:00.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.015+0000 7f0c8f59e700 1 --2- 172.21.15.7:0/81716764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c90107790 0x7f0c901989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:00.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.015+0000 7f0c8ed9d700 1 -- 172.21.15.7:0/81716764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c90107790 msgr2=0x7f0c901989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:00.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.015+0000 7f0c8ed9d700 1 --2- 172.21.15.7:0/81716764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c90107790 0x7f0c901989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.015+0000 7f0c8ed9d700 1 -- 172.21.15.7:0/81716764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c90103150 msgr2=0x7f0c9019ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:00.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.015+0000 7f0c8ed9d700 1 --2- 172.21.15.7:0/81716764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c90103150 0x7f0c9019ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.015+0000 7f0c8ed9d700 1 -- 172.21.15.7:0/81716764 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c90199110 con 0x7f0c901027a0 2024-06-10T01:11:00.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.015+0000 7f0c8659c700 1 --2- 172.21.15.7:0/81716764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c90103150 0x7f0c9019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:11:00.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.015+0000 7f0c8ed9d700 1 --2- 172.21.15.7:0/81716764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c901027a0 0x7f0c9019e7f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000e9f0 tx=0x7f0c8000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:00.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.016+0000 7f0c8cd99700 1 -- 172.21.15.7:0/81716764 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8000c9c0 con 0x7f0c901027a0 2024-06-10T01:11:00.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.016+0000 7f0c9507e700 1 -- 172.21.15.7:0/81716764 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c90199400 con 0x7f0c901027a0 2024-06-10T01:11:00.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.016+0000 7f0c8cd99700 1 -- 172.21.15.7:0/81716764 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c8000cb60 con 0x7f0c901027a0 2024-06-10T01:11:00.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.016+0000 7f0c8cd99700 1 -- 172.21.15.7:0/81716764 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80018680 con 0x7f0c901027a0 2024-06-10T01:11:00.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.016+0000 7f0c9507e700 1 -- 172.21.15.7:0/81716764 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c901a56c0 con 0x7f0c901027a0 2024-06-10T01:11:00.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.018+0000 7f0c8cd99700 1 -- 172.21.15.7:0/81716764 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0c80018820 con 0x7f0c901027a0 2024-06-10T01:11:00.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.019+0000 7f0c8cd99700 1 --2- 172.21.15.7:0/81716764 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0c7c0a2cc0 0x7f0c7c0a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:00.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.019+0000 7f0c8659c700 1 --2- 172.21.15.7:0/81716764 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0c7c0a2cc0 0x7f0c7c0a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:00.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.019+0000 7f0c8cd99700 1 -- 172.21.15.7:0/81716764 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f0c800cf520 con 0x7f0c901027a0 2024-06-10T01:11:00.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.020+0000 7f0c9f901b80 1 Processor -- start 2024-06-10T01:11:00.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.020+0000 7f0c9f901b80 1 -- start start 2024-06-10T01:11:00.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.020+0000 7f0c8659c700 1 --2- 172.21.15.7:0/81716764 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0c7c0a2cc0 0x7f0c7c0a5180 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0c90107110 tx=0x7f0c78005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:00.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.020+0000 7f0c9f901b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a8a90f0 0x557b3a8a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:00.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.020+0000 7f0c9f901b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b3a8a9a10 0x557b3a8a3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:00.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.020+0000 7f0c9f901b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b3a8a3880 0x557b3a8a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:00.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.020+0000 7f0c9f901b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557b3a7f09b0 con 0x557b3a8a9a10 2024-06-10T01:11:00.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.020+0000 7f0c9f901b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557b3a7988c0 con 0x557b3a8a90f0 2024-06-10T01:11:00.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.020+0000 7f0c9f901b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557b3a7ec970 con 0x557b3a8a3880 2024-06-10T01:11:00.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.020+0000 7f0c8659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b3a8a9a10 0x557b3a8a3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:00.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.020+0000 7f0c8659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b3a8a9a10 0x557b3a8a3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52830/0 (socket says 172.21.15.7:52830) 2024-06-10T01:11:00.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.020+0000 7f0c8659c700 1 -- 172.21.15.7:0/1482099878 learned_addr learned my addr 172.21.15.7:0/1482099878 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:00.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.020+0000 7f0c8ed9d700 1 --2- 172.21.15.7:0/1482099878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a8a90f0 0x557b3a8a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:00.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.020+0000 7f0c8f59e700 1 --2- 172.21.15.7:0/1482099878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b3a8a3880 0x557b3a8a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:00.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.021+0000 7f0c8f59e700 1 -- 172.21.15.7:0/1482099878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a8a90f0 msgr2=0x557b3a8a94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:00.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.021+0000 7f0c8f59e700 1 --2- 172.21.15.7:0/1482099878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a8a90f0 0x557b3a8a94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.021+0000 7f0c8f59e700 1 -- 172.21.15.7:0/1482099878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b3a8a9a10 msgr2=0x557b3a8a3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:00.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.021+0000 7f0c8f59e700 1 --2- 172.21.15.7:0/1482099878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b3a8a9a10 0x557b3a8a3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.021+0000 7f0c8f59e700 1 -- 172.21.15.7:0/1482099878 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b3a78f0f0 con 0x557b3a8a3880 2024-06-10T01:11:00.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.021+0000 7f0c8659c700 1 --2- 172.21.15.7:0/1482099878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b3a8a9a10 0x557b3a8a3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:11:00.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.021+0000 7f0c8f59e700 1 --2- 172.21.15.7:0/1482099878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b3a8a3880 0x557b3a8a3cc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0c88011580 tx=0x7f0c88009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=934ef6a3fd0bdbc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:00.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.021+0000 7f0c8559a700 1 -- 172.21.15.7:0/1482099878 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c88011e80 con 0x557b3a8a3880 2024-06-10T01:11:00.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.021+0000 7f0c8559a700 1 -- 172.21.15.7:0/1482099878 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c88004bb0 con 0x557b3a8a3880 2024-06-10T01:11:00.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.021+0000 7f0c8559a700 1 -- 172.21.15.7:0/1482099878 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c88004840 con 0x557b3a8a3880 2024-06-10T01:11:00.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.022+0000 7f0c9f901b80 1 -- 172.21.15.7:0/1482099878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b3a8a3880 msgr2=0x557b3a8a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:00.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.022+0000 7f0c9f901b80 1 --2- 172.21.15.7:0/1482099878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b3a8a3880 0x557b3a8a3cc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0c88011580 tx=0x7f0c88009920 comp rx=0 tx=0).stop 2024-06-10T01:11:00.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.022+0000 7f0c9f901b80 1 -- 172.21.15.7:0/1482099878 shutdown_connections 2024-06-10T01:11:00.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.022+0000 7f0c9f901b80 1 --2- 172.21.15.7:0/1482099878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b3a8a3880 0x557b3a8a3cc0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.022+0000 7f0c9f901b80 1 --2- 172.21.15.7:0/1482099878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a8a90f0 0x557b3a8a94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.022+0000 7f0c9f901b80 1 --2- 172.21.15.7:0/1482099878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b3a8a9a10 0x557b3a8a3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.022+0000 7f0c9f901b80 1 -- 172.21.15.7:0/1482099878 >> 172.21.15.7:0/1482099878 conn(0x557b3a7c9e60 msgr2=0x557b3a89a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:00.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.022+0000 7f0c9f901b80 1 -- 172.21.15.7:0/1482099878 shutdown_connections 2024-06-10T01:11:00.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.022+0000 7f0c9f901b80 1 -- 172.21.15.7:0/1482099878 wait complete. 2024-06-10T01:11:00.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.023+0000 7f0c9f901b80 1 -- start start 2024-06-10T01:11:00.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.023+0000 7f0c9f901b80 1 Processor -- start 2024-06-10T01:11:00.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.023+0000 7f0c9f901b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b3a8a3880 0x557b3a8af700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:00.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.023+0000 7f0c9f901b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a8a90f0 0x557b3a8aca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:00.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.023+0000 7f0c9f901b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b3a8a9a10 0x557b3a8acf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:00.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.023+0000 7f0c9f901b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557b3a8b0480 con 0x557b3a8a9a10 2024-06-10T01:11:00.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.023+0000 7f0c9f901b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557b3a790d90 con 0x557b3a8a90f0 2024-06-10T01:11:00.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.023+0000 7f0c9f901b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557b3a8b0600 con 0x557b3a8a3880 2024-06-10T01:11:00.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.023+0000 7f0c9f901b80 10 client.0 _ll_register_callbacks cb 0x557b3a78aac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:11:00.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.023+0000 7f0c8659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a8a90f0 0x557b3a8aca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:00.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.023+0000 7f0c8ed9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b3a8a3880 0x557b3a8af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:00.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.023+0000 7f0c8659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a8a90f0 0x557b3a8aca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:57482/0 (socket says 172.21.15.7:57482) 2024-06-10T01:11:00.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.023+0000 7f0c8659c700 1 -- 172.21.15.7:0/2235447891 learned_addr learned my addr 172.21.15.7:0/2235447891 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:00.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.023+0000 7f0c8f59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b3a8a9a10 0x557b3a8acf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:00.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.024+0000 7f0c8ed9d700 1 -- 172.21.15.7:0/2235447891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a8a90f0 msgr2=0x557b3a8aca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:00.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.024+0000 7f0c8ed9d700 1 --2- 172.21.15.7:0/2235447891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a8a90f0 0x557b3a8aca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.024+0000 7f0c8ed9d700 1 -- 172.21.15.7:0/2235447891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b3a8a9a10 msgr2=0x557b3a8acf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:00.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.024+0000 7f0c8ed9d700 1 --2- 172.21.15.7:0/2235447891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b3a8a9a10 0x557b3a8acf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.024+0000 7f0c8ed9d700 1 -- 172.21.15.7:0/2235447891 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b3a73fa70 con 0x557b3a8a3880 2024-06-10T01:11:00.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.024+0000 7f0c8f59e700 1 --2- 172.21.15.7:0/2235447891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b3a8a9a10 0x557b3a8acf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:11:00.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.024+0000 7f0c8659c700 1 --2- 172.21.15.7:0/2235447891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a8a90f0 0x557b3a8aca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:11:00.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.024+0000 7f0c8ed9d700 1 --2- 172.21.15.7:0/2235447891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b3a8a3880 0x557b3a8af700 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000c5b0 tx=0x7f0c8000cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71182b010effad8a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:00.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.024+0000 7f0c767fc700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:11:00.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.024+0000 7f0c767fc700 1 -- 172.21.15.7:0/2235447891 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c800b4060 con 0x557b3a8a3880 2024-06-10T01:11:00.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.024+0000 7f0c767fc700 1 -- 172.21.15.7:0/2235447891 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f0c800a41e0 con 0x557b3a8a3880 2024-06-10T01:11:00.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.024+0000 7f0c9f901b80 1 -- 172.21.15.7:0/2235447891 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c88004bb0 con 0x557b3a8a3880 2024-06-10T01:11:00.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.024+0000 7f0c767fc700 1 -- 172.21.15.7:0/2235447891 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0c8009ba00 con 0x557b3a8a3880 2024-06-10T01:11:00.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.025+0000 7f0c9f901b80 10 client.26239 fetch_fsmap learned FSMap version 32 2024-06-10T01:11:00.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.025+0000 7f0c9f901b80 1 -- 172.21.15.7:0/2235447891 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557b3a8bad70 con 0x557b3a8a3880 2024-06-10T01:11:00.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.025+0000 7f0c767fc700 1 -- 172.21.15.7:0/2235447891 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c88004bb0 con 0x557b3a8a3880 2024-06-10T01:11:00.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.025+0000 7f0c767fc700 1 -- 172.21.15.7:0/2235447891 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f0c8009ae50 con 0x557b3a8a3880 2024-06-10T01:11:00.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.025+0000 7f0c9f901b80 10 client.26239 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:11:00.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.025+0000 7f0c9f901b80 10 client.26239 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:11:00.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.025+0000 7f0c9f901b80 20 client.26239 populate_metadata read hostname 'smithi007' 2024-06-10T01:11:00.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.025+0000 7f0c9f901b80 1 --2- 172.21.15.7:0/2235447891 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557b3a8bd300 0x557b3a8bf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:00.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.025+0000 7f0c9f901b80 4 client.26239 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:11:00.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.025+0000 7f0c9f901b80 1 -- 172.21.15.7:0/2235447891 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b3a7fdeb0 con 0x557b3a8bd300 2024-06-10T01:11:00.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.025+0000 7f0c8659c700 1 --2- 172.21.15.7:0/2235447891 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557b3a8bd300 0x557b3a8bf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:00.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.026+0000 7f0c767fc700 10 client.26239 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:00.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.026+0000 7f0c8659c700 1 --2- 172.21.15.7:0/2235447891 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557b3a8bd300 0x557b3a8bf6f0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=221a43ac4c3939ff server_cookie=6b846b73d745e71 in_seq=0 out_seq=0 2024-06-10T01:11:00.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.026+0000 7f0c767fc700 1 -- 172.21.15.7:0/2235447891 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557b3a7fdeb0 con 0x557b3a8bd300 2024-06-10T01:11:00.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.026+0000 7f0c767fc700 10 client.26239 handle_command_reply: tid=0 2024-06-10T01:11:00.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.026+0000 7f0c9f901b80 1 client.26239 shutdown 2024-06-10T01:11:00.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.027+0000 7f0c9f901b80 1 -- 172.21.15.7:0/2235447891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b3a8a3880 msgr2=0x557b3a8af700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:00.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.027+0000 7f0c9f901b80 1 --2- 172.21.15.7:0/2235447891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b3a8a3880 0x557b3a8af700 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000c5b0 tx=0x7f0c8000cd10 comp rx=0 tx=0).stop 2024-06-10T01:11:00.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.027+0000 7f0c9f901b80 1 -- 172.21.15.7:0/2235447891 shutdown_connections 2024-06-10T01:11:00.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.027+0000 7f0c9f901b80 1 --2- 172.21.15.7:0/2235447891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b3a8a9a10 0x557b3a8acf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.027+0000 7f0c9f901b80 1 --2- 172.21.15.7:0/2235447891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b3a8a3880 0x557b3a8af700 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.027+0000 7f0c9f901b80 1 --2- 172.21.15.7:0/2235447891 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557b3a8bd300 0x557b3a8bf6f0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.027+0000 7f0c9f901b80 1 --2- 172.21.15.7:0/2235447891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a8a90f0 0x557b3a8aca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.027+0000 7f0c767fc700 0 client.26239 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:00.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.027+0000 7f0c9f901b80 1 -- 172.21.15.7:0/2235447891 >> 172.21.15.7:0/2235447891 conn(0x557b3a7c9e60 msgr2=0x557b3a899db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:00.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.027+0000 7f0c9f901b80 1 -- 172.21.15.7:0/2235447891 shutdown_connections 2024-06-10T01:11:00.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.027+0000 7f0c9f901b80 1 -- 172.21.15.7:0/2235447891 wait complete. 2024-06-10T01:11:00.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.027+0000 7f0c9f901b80 20 client.26239 trim_cache size 0 max 16384 2024-06-10T01:11:00.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.051+0000 7f0c9f901b80 1 Processor -- start 2024-06-10T01:11:00.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.051+0000 7f0c9f901b80 1 -- start start 2024-06-10T01:11:00.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.051+0000 7f0c9f901b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a8a3880 0x557b3a8ab960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:00.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.051+0000 7f0c9f901b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b3a8a90f0 0x557b3a899fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:00.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.051+0000 7f0c9f901b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b3a8a9a10 0x557b3a8c4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:00.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.051+0000 7f0c9f901b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0c901ae9a0 con 0x557b3a8a9a10 2024-06-10T01:11:00.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.051+0000 7f0c9f901b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0c901ac230 con 0x557b3a8a3880 2024-06-10T01:11:00.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.051+0000 7f0c9f901b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0c901aa080 con 0x557b3a8a90f0 2024-06-10T01:11:00.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.051+0000 7f0c8f59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b3a8a9a10 0x557b3a8c4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:00.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.051+0000 7f0c8659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b3a8a90f0 0x557b3a899fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:00.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.051+0000 7f0c8ed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a8a3880 0x557b3a8ab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:00.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.051+0000 7f0c8f59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b3a8a9a10 0x557b3a8c4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52850/0 (socket says 172.21.15.7:52850) 2024-06-10T01:11:00.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.051+0000 7f0c8f59e700 1 -- 172.21.15.7:0/2948944515 learned_addr learned my addr 172.21.15.7:0/2948944515 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:00.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.052+0000 7f0c8ed9d700 1 -- 172.21.15.7:0/2948944515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b3a8a90f0 msgr2=0x557b3a899fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:00.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.052+0000 7f0c8ed9d700 1 --2- 172.21.15.7:0/2948944515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b3a8a90f0 0x557b3a899fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.052+0000 7f0c8ed9d700 1 -- 172.21.15.7:0/2948944515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b3a8a9a10 msgr2=0x557b3a8c4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:00.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.052+0000 7f0c8ed9d700 1 --2- 172.21.15.7:0/2948944515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b3a8a9a10 0x557b3a8c4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.052+0000 7f0c8ed9d700 1 -- 172.21.15.7:0/2948944515 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b3a8bfa60 con 0x557b3a8a3880 2024-06-10T01:11:00.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.052+0000 7f0c8f59e700 1 --2- 172.21.15.7:0/2948944515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b3a8a9a10 0x557b3a8c4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:11:00.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.052+0000 7f0c8ed9d700 1 --2- 172.21.15.7:0/2948944515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a8a3880 0x557b3a8ab960 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0c80097e60 tx=0x7f0c80099f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac5aed8f59dbd7f7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:00.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.052+0000 7f0c84d99700 1 -- 172.21.15.7:0/2948944515 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c800a4670 con 0x557b3a8a3880 2024-06-10T01:11:00.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.052+0000 7f0c84d99700 1 -- 172.21.15.7:0/2948944515 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c8009fdb0 con 0x557b3a8a3880 2024-06-10T01:11:00.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.052+0000 7f0c84d99700 1 -- 172.21.15.7:0/2948944515 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80086130 con 0x557b3a8a3880 2024-06-10T01:11:00.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.053+0000 7f0c9f901b80 1 -- 172.21.15.7:0/2948944515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a8a3880 msgr2=0x557b3a8ab960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:00.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.053+0000 7f0c9f901b80 1 --2- 172.21.15.7:0/2948944515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a8a3880 0x557b3a8ab960 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0c80097e60 tx=0x7f0c80099f10 comp rx=0 tx=0).stop 2024-06-10T01:11:00.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.053+0000 7f0c9f901b80 1 -- 172.21.15.7:0/2948944515 shutdown_connections 2024-06-10T01:11:00.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.053+0000 7f0c9f901b80 1 --2- 172.21.15.7:0/2948944515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b3a8a9a10 0x557b3a8c4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.053+0000 7f0c9f901b80 1 --2- 172.21.15.7:0/2948944515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a8a3880 0x557b3a8ab960 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.053+0000 7f0c9f901b80 1 --2- 172.21.15.7:0/2948944515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b3a8a90f0 0x557b3a899fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.053+0000 7f0c9f901b80 1 -- 172.21.15.7:0/2948944515 >> 172.21.15.7:0/2948944515 conn(0x557b3a7c9e60 msgr2=0x557b3a8998d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:00.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.053+0000 7f0c9f901b80 1 -- 172.21.15.7:0/2948944515 shutdown_connections 2024-06-10T01:11:00.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.053+0000 7f0c9f901b80 1 -- 172.21.15.7:0/2948944515 wait complete. 2024-06-10T01:11:00.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.053+0000 7f0c9f901b80 1 -- start start 2024-06-10T01:11:00.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.053+0000 7f0c9f901b80 1 Processor -- start 2024-06-10T01:11:00.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.054+0000 7f0c9f901b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b3a8a3880 0x557b3a8c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:00.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.054+0000 7f0c9f901b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b3a8a90f0 0x557b3a8c5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:00.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.054+0000 7f0c9f901b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a8a9a10 0x557b3a8c5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:00.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.054+0000 7f0c9f901b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0c901aa6f0 con 0x557b3a8a90f0 2024-06-10T01:11:00.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.054+0000 7f0c9f901b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0c901abb60 con 0x557b3a8a9a10 2024-06-10T01:11:00.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.054+0000 7f0c9f901b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0c901aadc0 con 0x557b3a8a3880 2024-06-10T01:11:00.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.054+0000 7f0c9f901b80 10 client.0 _ll_register_callbacks cb 0x557b3a78aac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:11:00.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.054+0000 7f0c8ed9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b3a8a3880 0x557b3a8c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:00.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.054+0000 7f0c8659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b3a8a90f0 0x557b3a8c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:00.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.054+0000 7f0c8ed9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b3a8a3880 0x557b3a8c4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:39302/0 (socket says 172.21.15.7:39302) 2024-06-10T01:11:00.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.054+0000 7f0c8ed9d700 1 -- 172.21.15.7:0/2367844978 learned_addr learned my addr 172.21.15.7:0/2367844978 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:00.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.054+0000 7f0c8f59e700 1 --2- 172.21.15.7:0/2367844978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a8a9a10 0x557b3a8c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:00.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.054+0000 7f0c8659c700 1 -- 172.21.15.7:0/2367844978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b3a8a3880 msgr2=0x557b3a8c4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:00.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.054+0000 7f0c8659c700 1 --2- 172.21.15.7:0/2367844978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b3a8a3880 0x557b3a8c4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.054+0000 7f0c8659c700 1 -- 172.21.15.7:0/2367844978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a8a9a10 msgr2=0x557b3a8c5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:00.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.054+0000 7f0c8659c700 1 --2- 172.21.15.7:0/2367844978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a8a9a10 0x557b3a8c5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.054+0000 7f0c8659c700 1 -- 172.21.15.7:0/2367844978 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b3a927090 con 0x557b3a8a90f0 2024-06-10T01:11:00.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.054+0000 7f0c8ed9d700 1 --2- 172.21.15.7:0/2367844978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b3a8a3880 0x557b3a8c4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:11:00.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.054+0000 7f0c8659c700 1 --2- 172.21.15.7:0/2367844978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b3a8a90f0 0x557b3a8c5220 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x557b3a8adfd0 tx=0x7f0c78009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d34eb8a6dd423e83 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:00.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.054+0000 7f0c767fc700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:11:00.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.055+0000 7f0c767fc700 1 -- 172.21.15.7:0/2367844978 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7800fa50 con 0x557b3a8a90f0 2024-06-10T01:11:00.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.055+0000 7f0c767fc700 1 -- 172.21.15.7:0/2367844978 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f0c7800fbf0 con 0x557b3a8a90f0 2024-06-10T01:11:00.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.055+0000 7f0c9f901b80 1 -- 172.21.15.7:0/2367844978 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c8009fdb0 con 0x557b3a8a90f0 2024-06-10T01:11:00.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.055+0000 7f0c767fc700 1 -- 172.21.15.7:0/2367844978 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0c7800fdf0 con 0x557b3a8a90f0 2024-06-10T01:11:00.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.055+0000 7f0c9f901b80 10 client.16725 fetch_fsmap learned FSMap version 32 2024-06-10T01:11:00.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.055+0000 7f0c9f901b80 1 -- 172.21.15.7:0/2367844978 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557b3a9283d0 con 0x557b3a8a90f0 2024-06-10T01:11:00.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.055+0000 7f0c767fc700 1 -- 172.21.15.7:0/2367844978 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c8009fdb0 con 0x557b3a8a90f0 2024-06-10T01:11:00.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.055+0000 7f0c767fc700 1 -- 172.21.15.7:0/2367844978 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f0c7803a400 con 0x557b3a8a90f0 2024-06-10T01:11:00.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.055+0000 7f0c9f901b80 10 client.16725 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:11:00.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.055+0000 7f0c9f901b80 10 client.16725 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:11:00.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.055+0000 7f0c9f901b80 20 client.16725 populate_metadata read hostname 'smithi007' 2024-06-10T01:11:00.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.055+0000 7f0c9f901b80 1 --2- 172.21.15.7:0/2367844978 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557b3a928b20 0x557b3a92af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:00.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.055+0000 7f0c9f901b80 4 client.16725 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.45, "await_for": 45.0}] 2024-06-10T01:11:00.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.055+0000 7f0c9f901b80 1 -- 172.21.15.7:0/2367844978 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.45, "await_for": 45.0}) -- 0x557b3a7f8430 con 0x557b3a928b20 2024-06-10T01:11:00.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.055+0000 7f0c8ed9d700 1 --2- 172.21.15.7:0/2367844978 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557b3a928b20 0x557b3a92af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:00.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.056+0000 7f0c8ed9d700 1 --2- 172.21.15.7:0/2367844978 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557b3a928b20 0x557b3a92af40 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e8b550ec0cdb4bb server_cookie=e3f7951616df1655 in_seq=0 out_seq=0 2024-06-10T01:11:00.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.056+0000 7f0c767fc700 10 client.16725 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:00.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.136+0000 7f0c767fc700 1 -- 172.21.15.7:0/2367844978 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557b3a7f8430 con 0x557b3a928b20 2024-06-10T01:11:00.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.136+0000 7f0c767fc700 10 client.16725 handle_command_reply: tid=0 2024-06-10T01:11:00.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.136+0000 7f0c9f901b80 1 client.16725 shutdown 2024-06-10T01:11:00.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.136+0000 7f0c9f901b80 1 -- 172.21.15.7:0/2367844978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b3a8a90f0 msgr2=0x557b3a8c5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:00.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.136+0000 7f0c9f901b80 1 --2- 172.21.15.7:0/2367844978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b3a8a90f0 0x557b3a8c5220 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x557b3a8adfd0 tx=0x7f0c78009cd0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.136+0000 7f0c9f901b80 1 -- 172.21.15.7:0/2367844978 shutdown_connections 2024-06-10T01:11:00.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.136+0000 7f0c9f901b80 1 --2- 172.21.15.7:0/2367844978 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557b3a928b20 0x557b3a92af40 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.136+0000 7f0c9f901b80 1 --2- 172.21.15.7:0/2367844978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a8a9a10 0x557b3a8c5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.136+0000 7f0c9f901b80 1 --2- 172.21.15.7:0/2367844978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b3a8a3880 0x557b3a8c4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.136+0000 7f0c767fc700 0 client.16725 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.136+0000 7f0c9f901b80 1 --2- 172.21.15.7:0/2367844978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b3a8a90f0 0x557b3a8c5220 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.136+0000 7f0c9f901b80 1 -- 172.21.15.7:0/2367844978 >> 172.21.15.7:0/2367844978 conn(0x557b3a7c9e60 msgr2=0x557b3a89a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.137+0000 7f0c9f901b80 1 -- 172.21.15.7:0/2367844978 shutdown_connections 2024-06-10T01:11:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.137+0000 7f0c9f901b80 1 -- 172.21.15.7:0/2367844978 wait complete. 2024-06-10T01:11:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.137+0000 7f0c9f901b80 20 client.16725 trim_cache size 0 max 16384 2024-06-10T01:11:00.133 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:11:00.133 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 32, 2024-06-10T01:11:00.133 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:11:00.133 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 14, 2024-06-10T01:11:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:11:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: "24a573d7": { 2024-06-10T01:11:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 14, 2024-06-10T01:11:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:11:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:11:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:11:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:11:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:11:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:11:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 149.4, 2024-06-10T01:11:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:11:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:11:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:11:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:11:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:11:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:11:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:11:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:11:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:11:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:11:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:11:00.135 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:11:00.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.137+0000 7f0c9507e700 1 -- 172.21.15.7:0/81716764 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0c7c0a2cc0 msgr2=0x7f0c7c0a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:00.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.137+0000 7f0c9507e700 1 --2- 172.21.15.7:0/81716764 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0c7c0a2cc0 0x7f0c7c0a5180 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0c90107110 tx=0x7f0c78005fb0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.137+0000 7f0c9507e700 1 -- 172.21.15.7:0/81716764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c901027a0 msgr2=0x7f0c9019e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:00.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.137+0000 7f0c9507e700 1 --2- 172.21.15.7:0/81716764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c901027a0 0x7f0c9019e7f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000e9f0 tx=0x7f0c8000ed00 comp rx=0 tx=0).stop 2024-06-10T01:11:00.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.137+0000 7f0c9507e700 1 -- 172.21.15.7:0/81716764 shutdown_connections 2024-06-10T01:11:00.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.137+0000 7f0c9507e700 1 --2- 172.21.15.7:0/81716764 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0c7c0a2cc0 0x7f0c7c0a5180 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.137+0000 7f0c9507e700 1 --2- 172.21.15.7:0/81716764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c90107790 0x7f0c901989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.137+0000 7f0c9507e700 1 --2- 172.21.15.7:0/81716764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c901027a0 0x7f0c9019e7f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.137+0000 7f0c9507e700 1 --2- 172.21.15.7:0/81716764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c90103150 0x7f0c9019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:00.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.137+0000 7f0c9507e700 1 -- 172.21.15.7:0/81716764 >> 172.21.15.7:0/81716764 conn(0x7f0c90079560 msgr2=0x7f0c90101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:00.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.137+0000 7f0c9507e700 1 -- 172.21.15.7:0/81716764 shutdown_connections 2024-06-10T01:11:00.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:00.137+0000 7f0c9507e700 1 -- 172.21.15.7:0/81716764 wait complete. 2024-06-10T01:11:00.718 INFO:tasks.workunit.client.0.smithi007.stdout: 1 345759 9.14 MB/sec execute 101 sec latency 665.226 ms 2024-06-10T01:11:00.791 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:11:00.791 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:11:01.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:01 smithi175 ceph-mon[36636]: pgmap v582: 129 pgs: 129 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 372 op/s 2024-06-10T01:11:01.718 INFO:tasks.workunit.client.0.smithi007.stdout: 1 345759 9.05 MB/sec execute 102 sec latency 1665.288 ms 2024-06-10T01:11:01.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:01 smithi007 ceph-mon[32398]: pgmap v582: 129 pgs: 129 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 372 op/s 2024-06-10T01:11:01.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:01 smithi078 ceph-mon[36683]: pgmap v582: 129 pgs: 129 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 372 op/s 2024-06-10T01:11:02.059 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '24a573d7', quiesce took 9.8 seconds. Will release after: 19.65 2024-06-10T01:11:02.109 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:11:02.718 INFO:tasks.workunit.client.0.smithi007.stdout: 1 345759 8.96 MB/sec execute 103 sec latency 2665.392 ms 2024-06-10T01:11:03.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:03 smithi007 ceph-mon[32398]: pgmap v583: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 307 op/s 2024-06-10T01:11:03.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:03 smithi175 ceph-mon[36636]: pgmap v583: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 307 op/s 2024-06-10T01:11:03.718 INFO:tasks.workunit.client.0.smithi007.stdout: 1 345759 8.88 MB/sec execute 104 sec latency 3665.454 ms 2024-06-10T01:11:03.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:03 smithi078 ceph-mon[36683]: pgmap v583: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 307 op/s 2024-06-10T01:11:03.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.819+0000 7f65e93f1700 1 -- 172.21.15.7:0/2582351949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e4107840 msgr2=0x7f65e410f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:03.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.819+0000 7f65e93f1700 1 --2- 172.21.15.7:0/2582351949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e4107840 0x7f65e410f3c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00b0d0 tx=0x7f65dc00b3e0 comp rx=0 tx=0).stop 2024-06-10T01:11:03.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.819+0000 7f65e93f1700 1 -- 172.21.15.7:0/2582351949 shutdown_connections 2024-06-10T01:11:03.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.819+0000 7f65e93f1700 1 --2- 172.21.15.7:0/2582351949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e4107840 0x7f65e410f3c0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:03.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.819+0000 7f65e93f1700 1 --2- 172.21.15.7:0/2582351949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65e4102740 0x7f65e4102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:03.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.819+0000 7f65e93f1700 1 --2- 172.21.15.7:0/2582351949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65e41030f0 0x7f65e4107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:03.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.819+0000 7f65e93f1700 1 -- 172.21.15.7:0/2582351949 >> 172.21.15.7:0/2582351949 conn(0x7f65e40795f0 msgr2=0x7f65e4079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:03.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.819+0000 7f65e93f1700 1 -- 172.21.15.7:0/2582351949 shutdown_connections 2024-06-10T01:11:03.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.819+0000 7f65e93f1700 1 -- 172.21.15.7:0/2582351949 wait complete. 2024-06-10T01:11:03.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.819+0000 7f65e93f1700 1 Processor -- start 2024-06-10T01:11:03.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.820+0000 7f65e93f1700 1 -- start start 2024-06-10T01:11:03.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.820+0000 7f65e93f1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e4102740 0x7f65e41a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:03.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.820+0000 7f65e93f1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65e41030f0 0x7f65e41a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:03.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.820+0000 7f65e93f1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65e4107840 0x7f65e419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:03.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.820+0000 7f65e93f1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f65e4112760 con 0x7f65e4102740 2024-06-10T01:11:03.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.820+0000 7f65e93f1700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f65e41125e0 con 0x7f65e4107840 2024-06-10T01:11:03.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.820+0000 7f65e93f1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f65e41128e0 con 0x7f65e41030f0 2024-06-10T01:11:03.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.820+0000 7f65e2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e4102740 0x7f65e41a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:03.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.820+0000 7f65e2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e4102740 0x7f65e41a0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48594/0 (socket says 172.21.15.7:48594) 2024-06-10T01:11:03.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.820+0000 7f65e2ffd700 1 -- 172.21.15.7:0/2482232890 learned_addr learned my addr 172.21.15.7:0/2482232890 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:03.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.820+0000 7f65e27fc700 1 --2- 172.21.15.7:0/2482232890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65e41030f0 0x7f65e41a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:03.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.820+0000 7f65e37fe700 1 --2- 172.21.15.7:0/2482232890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65e4107840 0x7f65e419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:03.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.820+0000 7f65e2ffd700 1 -- 172.21.15.7:0/2482232890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65e41030f0 msgr2=0x7f65e41a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:03.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.820+0000 7f65e2ffd700 1 --2- 172.21.15.7:0/2482232890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65e41030f0 0x7f65e41a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:03.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.820+0000 7f65e2ffd700 1 -- 172.21.15.7:0/2482232890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65e4107840 msgr2=0x7f65e419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:03.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.820+0000 7f65e2ffd700 1 --2- 172.21.15.7:0/2482232890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65e4107840 0x7f65e419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:03.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.820+0000 7f65e2ffd700 1 -- 172.21.15.7:0/2482232890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e419b1d0 con 0x7f65e4102740 2024-06-10T01:11:03.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.821+0000 7f65e2ffd700 1 --2- 172.21.15.7:0/2482232890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e4102740 0x7f65e41a0860 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f65d400c9b0 tx=0x7f65d400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:03.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.821+0000 7f65cbfff700 1 -- 172.21.15.7:0/2482232890 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d4007b00 con 0x7f65e4102740 2024-06-10T01:11:03.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.821+0000 7f65cbfff700 1 -- 172.21.15.7:0/2482232890 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65d4007ca0 con 0x7f65e4102740 2024-06-10T01:11:03.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.821+0000 7f65cbfff700 1 -- 172.21.15.7:0/2482232890 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d4018780 con 0x7f65e4102740 2024-06-10T01:11:03.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.821+0000 7f65e93f1700 1 -- 172.21.15.7:0/2482232890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65e419b4c0 con 0x7f65e4102740 2024-06-10T01:11:03.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.821+0000 7f65e93f1700 1 -- 172.21.15.7:0/2482232890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65e41a7730 con 0x7f65e4102740 2024-06-10T01:11:03.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.822+0000 7f65e93f1700 1 -- 172.21.15.7:0/2482232890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65e4069180 con 0x7f65e4102740 2024-06-10T01:11:03.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.823+0000 7f65cbfff700 1 -- 172.21.15.7:0/2482232890 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f65d400f450 con 0x7f65e4102740 2024-06-10T01:11:03.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.824+0000 7f65cbfff700 1 --2- 172.21.15.7:0/2482232890 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f65cc0a2c20 0x7f65cc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:03.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.824+0000 7f65e27fc700 1 --2- 172.21.15.7:0/2482232890 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f65cc0a2c20 0x7f65cc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:03.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.824+0000 7f65cbfff700 1 -- 172.21.15.7:0/2482232890 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f65d40cf6f0 con 0x7f65e4102740 2024-06-10T01:11:03.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.824+0000 7f65e27fc700 1 --2- 172.21.15.7:0/2482232890 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f65cc0a2c20 0x7f65cc0a50e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f65d8005ea0 tx=0x7f65d8005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:03.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:03.827+0000 7f65cbfff700 1 -- 172.21.15.7:0/2482232890 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f65d4014030 con 0x7f65e4102740 2024-06-10T01:11:04.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:04.095+0000 7f65e93f1700 1 -- 172.21.15.7:0/2482232890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65e419bf50 con 0x7f65e4102740 2024-06-10T01:11:04.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:04.096+0000 7f65cbfff700 1 -- 172.21.15.7:0/2482232890 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11855 (secure 0 0 0) 0x7f65d409c360 con 0x7f65e4102740 2024-06-10T01:11:04.094 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:11:04.094 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:10:42.213996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:11:04.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:04.098+0000 7f65e93f1700 1 -- 172.21.15.7:0/2482232890 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f65cc0a2c20 msgr2=0x7f65cc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:04.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:04.098+0000 7f65e93f1700 1 --2- 172.21.15.7:0/2482232890 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f65cc0a2c20 0x7f65cc0a50e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f65d8005ea0 tx=0x7f65d8005e30 comp rx=0 tx=0).stop 2024-06-10T01:11:04.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:04.098+0000 7f65e93f1700 1 -- 172.21.15.7:0/2482232890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e4102740 msgr2=0x7f65e41a0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:04.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:04.099+0000 7f65e93f1700 1 --2- 172.21.15.7:0/2482232890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e4102740 0x7f65e41a0860 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f65d400c9b0 tx=0x7f65d400cd70 comp rx=0 tx=0).stop 2024-06-10T01:11:04.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:04.099+0000 7f65e93f1700 1 -- 172.21.15.7:0/2482232890 shutdown_connections 2024-06-10T01:11:04.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:04.099+0000 7f65e93f1700 1 --2- 172.21.15.7:0/2482232890 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f65cc0a2c20 0x7f65cc0a50e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:04.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:04.099+0000 7f65e93f1700 1 --2- 172.21.15.7:0/2482232890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65e4107840 0x7f65e419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:04.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:04.099+0000 7f65e93f1700 1 --2- 172.21.15.7:0/2482232890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e4102740 0x7f65e41a0860 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:04.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:04.099+0000 7f65e93f1700 1 --2- 172.21.15.7:0/2482232890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65e41030f0 0x7f65e41a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:04.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:04.099+0000 7f65e93f1700 1 -- 172.21.15.7:0/2482232890 >> 172.21.15.7:0/2482232890 conn(0x7f65e40795f0 msgr2=0x7f65e41005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:04.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:04.099+0000 7f65e93f1700 1 -- 172.21.15.7:0/2482232890 shutdown_connections 2024-06-10T01:11:04.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:04.099+0000 7f65e93f1700 1 -- 172.21.15.7:0/2482232890 wait complete. 2024-06-10T01:11:04.095 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 32 2024-06-10T01:11:04.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:04 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2482232890' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:11:04.718 INFO:tasks.workunit.client.0.smithi007.stdout: 1 345759 8.79 MB/sec execute 105 sec latency 4665.550 ms 2024-06-10T01:11:04.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:04 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2482232890' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:11:04.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:04 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2482232890' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:11:04.880 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:11:05.135 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:11:05.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:05 smithi007 ceph-mon[32398]: pgmap v584: 129 pgs: 129 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 322 op/s 2024-06-10T01:11:05.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:05 smithi175 ceph-mon[36636]: pgmap v584: 129 pgs: 129 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 322 op/s 2024-06-10T01:11:05.719 INFO:tasks.workunit.client.0.smithi007.stdout: 1 345759 8.71 MB/sec execute 106 sec latency 5665.662 ms 2024-06-10T01:11:05.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:05 smithi078 ceph-mon[36683]: pgmap v584: 129 pgs: 129 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 322 op/s 2024-06-10T01:11:06.729 INFO:tasks.workunit.client.0.smithi007.stdout: 1 345759 8.63 MB/sec execute 107 sec latency 6676.321 ms 2024-06-10T01:11:06.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.903+0000 7fbaf31c6700 1 -- 172.21.15.7:0/2442993373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaec06b620 msgr2=0x7fbaec10f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:06.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.903+0000 7fbaf31c6700 1 --2- 172.21.15.7:0/2442993373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaec06b620 0x7fbaec10f350 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fbae400b0d0 tx=0x7fbae400b3e0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.903+0000 7fbaf31c6700 1 -- 172.21.15.7:0/2442993373 shutdown_connections 2024-06-10T01:11:06.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.903+0000 7fbaf31c6700 1 --2- 172.21.15.7:0/2442993373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaec06b620 0x7fbaec10f350 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.903+0000 7fbaf31c6700 1 --2- 172.21.15.7:0/2442993373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbaec108580 0x7fbaec108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.903+0000 7fbaf31c6700 1 --2- 172.21.15.7:0/2442993373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaec06ac60 0x7fbaec06b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.903+0000 7fbaf31c6700 1 -- 172.21.15.7:0/2442993373 >> 172.21.15.7:0/2442993373 conn(0x7fbaec078430 msgr2=0x7fbaec078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:06.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.903+0000 7fbaf31c6700 1 -- 172.21.15.7:0/2442993373 shutdown_connections 2024-06-10T01:11:06.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.903+0000 7fbaf31c6700 1 -- 172.21.15.7:0/2442993373 wait complete. 2024-06-10T01:11:06.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.904+0000 7fbaf31c6700 1 Processor -- start 2024-06-10T01:11:06.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.904+0000 7fbaf31c6700 1 -- start start 2024-06-10T01:11:06.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.904+0000 7fbaf31c6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbaec06ac60 0x7fbaec19e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:06.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.904+0000 7fbaf31c6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaec06b620 0x7fbaec19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:06.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.904+0000 7fbaf31c6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaec108580 0x7fbaec1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:06.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.904+0000 7fbaf31c6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec1127f0 con 0x7fbaec108580 2024-06-10T01:11:06.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.904+0000 7fbaf31c6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec112670 con 0x7fbaec06ac60 2024-06-10T01:11:06.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.904+0000 7fbaf31c6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec112970 con 0x7fbaec06b620 2024-06-10T01:11:06.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.904+0000 7fbaf1763700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaec108580 0x7fbaec1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:06.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.904+0000 7fbaf1763700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaec108580 0x7fbaec1989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48600/0 (socket says 172.21.15.7:48600) 2024-06-10T01:11:06.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.904+0000 7fbaf1763700 1 -- 172.21.15.7:0/2475712903 learned_addr learned my addr 172.21.15.7:0/2475712903 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:06.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.904+0000 7fbaf0f62700 1 --2- 172.21.15.7:0/2475712903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbaec06ac60 0x7fbaec19e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:06.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.904+0000 7fbaebfff700 1 --2- 172.21.15.7:0/2475712903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaec06b620 0x7fbaec19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:06.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.904+0000 7fbaf1763700 1 -- 172.21.15.7:0/2475712903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaec06b620 msgr2=0x7fbaec19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:06.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.904+0000 7fbaf1763700 1 --2- 172.21.15.7:0/2475712903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaec06b620 0x7fbaec19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.904+0000 7fbaf1763700 1 -- 172.21.15.7:0/2475712903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbaec06ac60 msgr2=0x7fbaec19e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:06.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.904+0000 7fbaf1763700 1 --2- 172.21.15.7:0/2475712903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbaec06ac60 0x7fbaec19e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.904+0000 7fbaf1763700 1 -- 172.21.15.7:0/2475712903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaec199110 con 0x7fbaec108580 2024-06-10T01:11:06.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.905+0000 7fbaf1763700 1 --2- 172.21.15.7:0/2475712903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaec108580 0x7fbaec1989b0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fbae4006070 tx=0x7fbae4007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:06.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.905+0000 7fbae9ffb700 1 -- 172.21.15.7:0/2475712903 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae4003c00 con 0x7fbaec108580 2024-06-10T01:11:06.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.905+0000 7fbae9ffb700 1 -- 172.21.15.7:0/2475712903 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbae4022070 con 0x7fbaec108580 2024-06-10T01:11:06.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.905+0000 7fbaf31c6700 1 -- 172.21.15.7:0/2475712903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaec1993a0 con 0x7fbaec108580 2024-06-10T01:11:06.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.905+0000 7fbae9ffb700 1 -- 172.21.15.7:0/2475712903 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae401b6c0 con 0x7fbaec108580 2024-06-10T01:11:06.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.905+0000 7fbaf31c6700 1 -- 172.21.15.7:0/2475712903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaec1a56c0 con 0x7fbaec108580 2024-06-10T01:11:06.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.907+0000 7fbae9ffb700 1 -- 172.21.15.7:0/2475712903 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fbae401d470 con 0x7fbaec108580 2024-06-10T01:11:06.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.908+0000 7fbae9ffb700 1 --2- 172.21.15.7:0/2475712903 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fbadc0a2c20 0x7fbadc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:06.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.909+0000 7fbaf0f62700 1 --2- 172.21.15.7:0/2475712903 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fbadc0a2c20 0x7fbadc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:06.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.909+0000 7fbae9ffb700 1 -- 172.21.15.7:0/2475712903 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fbae400f020 con 0x7fbaec108580 2024-06-10T01:11:06.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.909+0000 7fbaf0f62700 1 --2- 172.21.15.7:0/2475712903 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fbadc0a2c20 0x7fbadc0a50e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbaec199af0 tx=0x7fbae0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:06.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.909+0000 7fbafda49b80 1 Processor -- start 2024-06-10T01:11:06.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.910+0000 7fbafda49b80 1 -- start start 2024-06-10T01:11:06.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.910+0000 7fbafda49b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556475e84370 0x556475e84750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:06.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.910+0000 7fbafda49b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556475e84c90 0x556475e7e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:06.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.910+0000 7fbafda49b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556475e7ebb0 0x556475e7eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:06.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.910+0000 7fbafda49b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556475d6cf60 con 0x556475e7ebb0 2024-06-10T01:11:06.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.910+0000 7fbafda49b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556475dcb850 con 0x556475e84370 2024-06-10T01:11:06.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.910+0000 7fbafda49b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556475d33460 con 0x556475e84c90 2024-06-10T01:11:06.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.910+0000 7fbaf1763700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556475e7ebb0 0x556475e7eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:06.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.910+0000 7fbaf0f62700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556475e84370 0x556475e84750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:06.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.910+0000 7fbaf1763700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556475e7ebb0 0x556475e7eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48604/0 (socket says 172.21.15.7:48604) 2024-06-10T01:11:06.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.910+0000 7fbaf1763700 1 -- 172.21.15.7:0/376939088 learned_addr learned my addr 172.21.15.7:0/376939088 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:06.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.910+0000 7fbaebfff700 1 --2- 172.21.15.7:0/376939088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556475e84c90 0x556475e7e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:06.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.910+0000 7fbaf0f62700 1 -- 172.21.15.7:0/376939088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556475e84c90 msgr2=0x556475e7e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:06.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.910+0000 7fbaf0f62700 1 --2- 172.21.15.7:0/376939088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556475e84c90 0x556475e7e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.910+0000 7fbaf0f62700 1 -- 172.21.15.7:0/376939088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556475e7ebb0 msgr2=0x556475e7eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:06.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.910+0000 7fbaf0f62700 1 --2- 172.21.15.7:0/376939088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556475e7ebb0 0x556475e7eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.910+0000 7fbaf0f62700 1 -- 172.21.15.7:0/376939088 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556475d1a8d0 con 0x556475e84370 2024-06-10T01:11:06.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.911+0000 7fbaf0f62700 1 --2- 172.21.15.7:0/376939088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556475e84370 0x556475e84750 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbae0011c90 tx=0x7fbae0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99a72ce876505ddc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:06.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.911+0000 7fbad6ffd700 1 -- 172.21.15.7:0/376939088 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae0004230 con 0x556475e84370 2024-06-10T01:11:06.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.911+0000 7fbad6ffd700 1 -- 172.21.15.7:0/376939088 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbae000ed20 con 0x556475e84370 2024-06-10T01:11:06.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.911+0000 7fbad6ffd700 1 -- 172.21.15.7:0/376939088 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae0005020 con 0x556475e84370 2024-06-10T01:11:06.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.911+0000 7fbafda49b80 1 -- 172.21.15.7:0/376939088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556475e84370 msgr2=0x556475e84750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:06.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.911+0000 7fbafda49b80 1 --2- 172.21.15.7:0/376939088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556475e84370 0x556475e84750 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbae0011c90 tx=0x7fbae0014000 comp rx=0 tx=0).stop 2024-06-10T01:11:06.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.912+0000 7fbafda49b80 1 -- 172.21.15.7:0/376939088 shutdown_connections 2024-06-10T01:11:06.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.912+0000 7fbafda49b80 1 --2- 172.21.15.7:0/376939088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556475e7ebb0 0x556475e7eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.912+0000 7fbafda49b80 1 --2- 172.21.15.7:0/376939088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556475e84370 0x556475e84750 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.912+0000 7fbafda49b80 1 --2- 172.21.15.7:0/376939088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556475e84c90 0x556475e7e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.912+0000 7fbafda49b80 1 -- 172.21.15.7:0/376939088 >> 172.21.15.7:0/376939088 conn(0x556475da7080 msgr2=0x556475e75220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:06.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.912+0000 7fbafda49b80 1 -- 172.21.15.7:0/376939088 shutdown_connections 2024-06-10T01:11:06.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.912+0000 7fbafda49b80 1 -- 172.21.15.7:0/376939088 wait complete. 2024-06-10T01:11:06.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.912+0000 7fbafda49b80 1 -- start start 2024-06-10T01:11:06.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.912+0000 7fbafda49b80 1 Processor -- start 2024-06-10T01:11:06.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.912+0000 7fbafda49b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556475e7ebb0 0x556475e8aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:06.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.912+0000 7fbafda49b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556475e84370 0x556475e87d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:06.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.912+0000 7fbafda49b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556475e84c90 0x556475e88260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:06.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.912+0000 7fbafda49b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556475d6bfc0 con 0x556475e84370 2024-06-10T01:11:06.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.912+0000 7fbafda49b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556475dc7810 con 0x556475e7ebb0 2024-06-10T01:11:06.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.912+0000 7fbafda49b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556475e8b680 con 0x556475e84c90 2024-06-10T01:11:06.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.913+0000 7fbafda49b80 10 client.0 _ll_register_callbacks cb 0x556475d615b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:11:06.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.913+0000 7fbaebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556475e84370 0x556475e87d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:06.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.913+0000 7fbaf0f62700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556475e7ebb0 0x556475e8aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:06.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.913+0000 7fbaebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556475e84370 0x556475e87d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48618/0 (socket says 172.21.15.7:48618) 2024-06-10T01:11:06.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.913+0000 7fbaebfff700 1 -- 172.21.15.7:0/3178466421 learned_addr learned my addr 172.21.15.7:0/3178466421 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:06.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.913+0000 7fbaf0f62700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556475e7ebb0 0x556475e8aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:55834/0 (socket says 172.21.15.7:55834) 2024-06-10T01:11:06.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.913+0000 7fbaf1763700 1 --2- 172.21.15.7:0/3178466421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556475e84c90 0x556475e88260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:06.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.913+0000 7fbaf0f62700 1 -- 172.21.15.7:0/3178466421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556475e84c90 msgr2=0x556475e88260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:06.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.913+0000 7fbaf0f62700 1 --2- 172.21.15.7:0/3178466421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556475e84c90 0x556475e88260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.913+0000 7fbaf0f62700 1 -- 172.21.15.7:0/3178466421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556475e84370 msgr2=0x556475e87d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:06.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.913+0000 7fbaf0f62700 1 --2- 172.21.15.7:0/3178466421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556475e84370 0x556475e87d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.913+0000 7fbaf0f62700 1 -- 172.21.15.7:0/3178466421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556475d91750 con 0x556475e7ebb0 2024-06-10T01:11:06.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.913+0000 7fbaf1763700 1 --2- 172.21.15.7:0/3178466421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556475e84c90 0x556475e88260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:11:06.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.913+0000 7fbaebfff700 1 --2- 172.21.15.7:0/3178466421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556475e84370 0x556475e87d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:11:06.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.913+0000 7fbaf0f62700 1 --2- 172.21.15.7:0/3178466421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556475e7ebb0 0x556475e8aa20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbae0005920 tx=0x7fbae0004990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a608dca5877b2fd7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:06.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.913+0000 7fbacbfff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:11:06.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.913+0000 7fbacbfff700 1 -- 172.21.15.7:0/3178466421 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae0005200 con 0x556475e7ebb0 2024-06-10T01:11:06.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.913+0000 7fbacbfff700 1 -- 172.21.15.7:0/3178466421 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fbae00053a0 con 0x556475e7ebb0 2024-06-10T01:11:06.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.913+0000 7fbafda49b80 1 -- 172.21.15.7:0/3178466421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbae000ed20 con 0x556475e7ebb0 2024-06-10T01:11:06.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.914+0000 7fbacbfff700 1 -- 172.21.15.7:0/3178466421 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbae00055a0 con 0x556475e7ebb0 2024-06-10T01:11:06.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.914+0000 7fbafda49b80 10 client.26231 fetch_fsmap learned FSMap version 32 2024-06-10T01:11:06.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.914+0000 7fbafda49b80 1 -- 172.21.15.7:0/3178466421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556475e95e70 con 0x556475e7ebb0 2024-06-10T01:11:06.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.914+0000 7fbacbfff700 1 -- 172.21.15.7:0/3178466421 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbae000ed20 con 0x556475e7ebb0 2024-06-10T01:11:06.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.914+0000 7fbacbfff700 1 -- 172.21.15.7:0/3178466421 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fbae002d020 con 0x556475e7ebb0 2024-06-10T01:11:06.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.914+0000 7fbafda49b80 10 client.26231 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:11:06.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.914+0000 7fbafda49b80 10 client.26231 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:11:06.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.914+0000 7fbafda49b80 20 client.26231 populate_metadata read hostname 'smithi007' 2024-06-10T01:11:06.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.914+0000 7fbafda49b80 1 --2- 172.21.15.7:0/3178466421 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x556475e98340 0x556475e9a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:06.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.914+0000 7fbafda49b80 4 client.26231 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:11:06.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.914+0000 7fbafda49b80 1 -- 172.21.15.7:0/3178466421 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556475dd8d50 con 0x556475e98340 2024-06-10T01:11:06.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.914+0000 7fbaebfff700 1 --2- 172.21.15.7:0/3178466421 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x556475e98340 0x556475e9a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:06.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.915+0000 7fbaebfff700 1 --2- 172.21.15.7:0/3178466421 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x556475e98340 0x556475e9a730 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5de93132f996b8ab server_cookie=d38740a3db3004df in_seq=0 out_seq=0 2024-06-10T01:11:06.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.915+0000 7fbacbfff700 10 client.26231 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:11:06.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.915+0000 7fbacbfff700 1 -- 172.21.15.7:0/3178466421 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556475dd8d50 con 0x556475e98340 2024-06-10T01:11:06.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.915+0000 7fbacbfff700 10 client.26231 handle_command_reply: tid=0 2024-06-10T01:11:06.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.915+0000 7fbafda49b80 1 client.26231 shutdown 2024-06-10T01:11:06.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.916+0000 7fbafda49b80 1 -- 172.21.15.7:0/3178466421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556475e7ebb0 msgr2=0x556475e8aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:06.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.916+0000 7fbafda49b80 1 --2- 172.21.15.7:0/3178466421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556475e7ebb0 0x556475e8aa20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbae0005920 tx=0x7fbae0004990 comp rx=0 tx=0).stop 2024-06-10T01:11:06.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.916+0000 7fbafda49b80 1 -- 172.21.15.7:0/3178466421 shutdown_connections 2024-06-10T01:11:06.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.916+0000 7fbafda49b80 1 --2- 172.21.15.7:0/3178466421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556475e84c90 0x556475e88260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.916+0000 7fbafda49b80 1 --2- 172.21.15.7:0/3178466421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556475e7ebb0 0x556475e8aa20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.916+0000 7fbafda49b80 1 --2- 172.21.15.7:0/3178466421 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x556475e98340 0x556475e9a730 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.916+0000 7fbafda49b80 1 --2- 172.21.15.7:0/3178466421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556475e84370 0x556475e87d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.916+0000 7fbafda49b80 1 -- 172.21.15.7:0/3178466421 >> 172.21.15.7:0/3178466421 conn(0x556475da7080 msgr2=0x556475e74700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:06.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.916+0000 7fbacbfff700 0 client.26231 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:11:06.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.916+0000 7fbafda49b80 1 -- 172.21.15.7:0/3178466421 shutdown_connections 2024-06-10T01:11:06.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.916+0000 7fbafda49b80 1 -- 172.21.15.7:0/3178466421 wait complete. 2024-06-10T01:11:06.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.916+0000 7fbafda49b80 20 client.26231 trim_cache size 0 max 16384 2024-06-10T01:11:06.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.931+0000 7fbafda49b80 1 Processor -- start 2024-06-10T01:11:06.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.931+0000 7fbafda49b80 1 -- start start 2024-06-10T01:11:06.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.931+0000 7fbafda49b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556475e7ebb0 0x556475e85780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:06.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.931+0000 7fbafda49b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556475e84370 0x556475e74e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:06.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.932+0000 7fbafda49b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556475e84c90 0x556475e753c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:06.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.932+0000 7fbafda49b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec1ae8a0 con 0x556475e84c90 2024-06-10T01:11:06.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.932+0000 7fbafda49b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec1ac130 con 0x556475e7ebb0 2024-06-10T01:11:06.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.932+0000 7fbafda49b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec1a9f80 con 0x556475e84370 2024-06-10T01:11:06.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.932+0000 7fbaf1763700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556475e84c90 0x556475e753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:06.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.932+0000 7fbaf1763700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556475e84c90 0x556475e753c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48626/0 (socket says 172.21.15.7:48626) 2024-06-10T01:11:06.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.932+0000 7fbaf1763700 1 -- 172.21.15.7:0/1748353595 learned_addr learned my addr 172.21.15.7:0/1748353595 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:06.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.932+0000 7fbaf0f62700 1 --2- 172.21.15.7:0/1748353595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556475e7ebb0 0x556475e85780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:06.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.932+0000 7fbaebfff700 1 --2- 172.21.15.7:0/1748353595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556475e84370 0x556475e74e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:06.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.932+0000 7fbaf1763700 1 -- 172.21.15.7:0/1748353595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556475e84370 msgr2=0x556475e74e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:06.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.932+0000 7fbaf1763700 1 --2- 172.21.15.7:0/1748353595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556475e84370 0x556475e74e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.932+0000 7fbaf1763700 1 -- 172.21.15.7:0/1748353595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556475e7ebb0 msgr2=0x556475e85780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:06.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.932+0000 7fbaf1763700 1 --2- 172.21.15.7:0/1748353595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556475e7ebb0 0x556475e85780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.932+0000 7fbaf1763700 1 -- 172.21.15.7:0/1748353595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556475e862b0 con 0x556475e84c90 2024-06-10T01:11:06.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.932+0000 7fbaebfff700 1 --2- 172.21.15.7:0/1748353595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556475e84370 0x556475e74e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:11:06.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.932+0000 7fbaf0f62700 1 --2- 172.21.15.7:0/1748353595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556475e7ebb0 0x556475e85780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:11:06.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.932+0000 7fbaf1763700 1 --2- 172.21.15.7:0/1748353595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556475e84c90 0x556475e753c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fbae40cd950 tx=0x7fbae40a3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed86f57fb8a6fb22 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:06.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.932+0000 7fbad67fc700 1 -- 172.21.15.7:0/1748353595 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae4003da0 con 0x556475e84c90 2024-06-10T01:11:06.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.932+0000 7fbad67fc700 1 -- 172.21.15.7:0/1748353595 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbae409c6f0 con 0x556475e84c90 2024-06-10T01:11:06.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.932+0000 7fbad67fc700 1 -- 172.21.15.7:0/1748353595 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae40074e0 con 0x556475e84c90 2024-06-10T01:11:06.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.933+0000 7fbafda49b80 1 -- 172.21.15.7:0/1748353595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556475e84c90 msgr2=0x556475e753c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:06.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.933+0000 7fbafda49b80 1 --2- 172.21.15.7:0/1748353595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556475e84c90 0x556475e753c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fbae40cd950 tx=0x7fbae40a3040 comp rx=0 tx=0).stop 2024-06-10T01:11:06.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.933+0000 7fbafda49b80 1 -- 172.21.15.7:0/1748353595 shutdown_connections 2024-06-10T01:11:06.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.933+0000 7fbafda49b80 1 --2- 172.21.15.7:0/1748353595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556475e84c90 0x556475e753c0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.933+0000 7fbafda49b80 1 --2- 172.21.15.7:0/1748353595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556475e7ebb0 0x556475e85780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.933+0000 7fbafda49b80 1 --2- 172.21.15.7:0/1748353595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556475e84370 0x556475e74e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.933+0000 7fbafda49b80 1 -- 172.21.15.7:0/1748353595 >> 172.21.15.7:0/1748353595 conn(0x556475da7080 msgr2=0x556475e85070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:06.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.933+0000 7fbafda49b80 1 -- 172.21.15.7:0/1748353595 shutdown_connections 2024-06-10T01:11:06.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.933+0000 7fbafda49b80 1 -- 172.21.15.7:0/1748353595 wait complete. 2024-06-10T01:11:06.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.933+0000 7fbafda49b80 1 -- start start 2024-06-10T01:11:06.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.934+0000 7fbafda49b80 1 Processor -- start 2024-06-10T01:11:06.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.934+0000 7fbafda49b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556475e7ebb0 0x556475e90db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:06.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.934+0000 7fbafda49b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556475e84370 0x556475e9f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:06.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.934+0000 7fbafda49b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556475e84c90 0x556475e9f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:06.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.934+0000 7fbafda49b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec1aa5f0 con 0x556475e84370 2024-06-10T01:11:06.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.934+0000 7fbafda49b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec1aba60 con 0x556475e7ebb0 2024-06-10T01:11:06.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.934+0000 7fbafda49b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec1aacc0 con 0x556475e84c90 2024-06-10T01:11:06.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.934+0000 7fbafda49b80 10 client.0 _ll_register_callbacks cb 0x556475d615b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:11:06.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.934+0000 7fbaebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556475e84370 0x556475e9f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:06.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.934+0000 7fbaebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556475e84370 0x556475e9f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48640/0 (socket says 172.21.15.7:48640) 2024-06-10T01:11:06.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.934+0000 7fbaebfff700 1 -- 172.21.15.7:0/4090454728 learned_addr learned my addr 172.21.15.7:0/4090454728 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:06.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.934+0000 7fbaf0f62700 1 --2- 172.21.15.7:0/4090454728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556475e7ebb0 0x556475e90db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:06.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.934+0000 7fbaf1763700 1 --2- 172.21.15.7:0/4090454728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556475e84c90 0x556475e9f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:06.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.934+0000 7fbaebfff700 1 -- 172.21.15.7:0/4090454728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556475e84c90 msgr2=0x556475e9f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:06.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.934+0000 7fbaebfff700 1 --2- 172.21.15.7:0/4090454728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556475e84c90 0x556475e9f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.934+0000 7fbaebfff700 1 -- 172.21.15.7:0/4090454728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556475e7ebb0 msgr2=0x556475e90db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:06.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.934+0000 7fbaebfff700 1 --2- 172.21.15.7:0/4090454728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556475e7ebb0 0x556475e90db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.934+0000 7fbaebfff700 1 -- 172.21.15.7:0/4090454728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556475e9fde0 con 0x556475e84370 2024-06-10T01:11:06.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.934+0000 7fbaf1763700 1 --2- 172.21.15.7:0/4090454728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556475e84c90 0x556475e9f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:11:06.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.934+0000 7fbaf0f62700 1 --2- 172.21.15.7:0/4090454728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556475e7ebb0 0x556475e90db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:11:06.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.935+0000 7fbaebfff700 1 --2- 172.21.15.7:0/4090454728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556475e84370 0x556475e9f360 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x556475e9ac70 tx=0x7fbad8015960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=724367298547cbf2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:06.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.935+0000 7fbacbfff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:11:06.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.935+0000 7fbacbfff700 1 -- 172.21.15.7:0/4090454728 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad80154e0 con 0x556475e84370 2024-06-10T01:11:06.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.935+0000 7fbacbfff700 1 -- 172.21.15.7:0/4090454728 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fbad8004830 con 0x556475e84370 2024-06-10T01:11:06.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.935+0000 7fbafda49b80 1 -- 172.21.15.7:0/4090454728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbae409c6f0 con 0x556475e84370 2024-06-10T01:11:06.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.935+0000 7fbacbfff700 1 -- 172.21.15.7:0/4090454728 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbad802c050 con 0x556475e84370 2024-06-10T01:11:06.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.935+0000 7fbafda49b80 10 client.16749 fetch_fsmap learned FSMap version 32 2024-06-10T01:11:06.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.935+0000 7fbafda49b80 1 -- 172.21.15.7:0/4090454728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556475f02960 con 0x556475e84370 2024-06-10T01:11:06.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.935+0000 7fbacbfff700 1 -- 172.21.15.7:0/4090454728 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbae409c6f0 con 0x556475e84370 2024-06-10T01:11:06.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.935+0000 7fbacbfff700 1 -- 172.21.15.7:0/4090454728 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fbad801e730 con 0x556475e84370 2024-06-10T01:11:06.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.935+0000 7fbafda49b80 10 client.16749 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:11:06.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.935+0000 7fbafda49b80 10 client.16749 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:11:06.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.935+0000 7fbafda49b80 20 client.16749 populate_metadata read hostname 'smithi007' 2024-06-10T01:11:06.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.935+0000 7fbafda49b80 1 --2- 172.21.15.7:0/4090454728 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x556475f03040 0x556475f05460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:06.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.935+0000 7fbafda49b80 4 client.16749 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:11:06.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.935+0000 7fbafda49b80 1 -- 172.21.15.7:0/4090454728 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556475dd32d0 con 0x556475f03040 2024-06-10T01:11:06.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.936+0000 7fbaf0f62700 1 --2- 172.21.15.7:0/4090454728 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x556475f03040 0x556475f05460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:06.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.936+0000 7fbaf0f62700 1 --2- 172.21.15.7:0/4090454728 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x556475f03040 0x556475f05460 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8123eb3735c172e8 server_cookie=25bb54c2c486b1b9 in_seq=0 out_seq=0 2024-06-10T01:11:06.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.936+0000 7fbacbfff700 10 client.16749 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:11:06.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.936+0000 7fbacbfff700 1 -- 172.21.15.7:0/4090454728 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556475dd32d0 con 0x556475f03040 2024-06-10T01:11:06.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.936+0000 7fbacbfff700 10 client.16749 handle_command_reply: tid=0 2024-06-10T01:11:06.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.936+0000 7fbafda49b80 1 client.16749 shutdown 2024-06-10T01:11:06.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.936+0000 7fbafda49b80 1 -- 172.21.15.7:0/4090454728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556475e84370 msgr2=0x556475e9f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:06.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.936+0000 7fbafda49b80 1 --2- 172.21.15.7:0/4090454728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556475e84370 0x556475e9f360 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x556475e9ac70 tx=0x7fbad8015960 comp rx=0 tx=0).stop 2024-06-10T01:11:06.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.936+0000 7fbafda49b80 1 -- 172.21.15.7:0/4090454728 shutdown_connections 2024-06-10T01:11:06.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.936+0000 7fbafda49b80 1 --2- 172.21.15.7:0/4090454728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556475e84c90 0x556475e9f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.936+0000 7fbafda49b80 1 --2- 172.21.15.7:0/4090454728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556475e7ebb0 0x556475e90db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.936+0000 7fbafda49b80 1 --2- 172.21.15.7:0/4090454728 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x556475f03040 0x556475f05460 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.936+0000 7fbafda49b80 1 --2- 172.21.15.7:0/4090454728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556475e84370 0x556475e9f360 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.936+0000 7fbacbfff700 0 client.16749 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:11:06.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.936+0000 7fbafda49b80 1 -- 172.21.15.7:0/4090454728 >> 172.21.15.7:0/4090454728 conn(0x556475da7080 msgr2=0x556475e82ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:06.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.937+0000 7fbafda49b80 1 -- 172.21.15.7:0/4090454728 shutdown_connections 2024-06-10T01:11:06.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.937+0000 7fbafda49b80 1 -- 172.21.15.7:0/4090454728 wait complete. 2024-06-10T01:11:06.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.937+0000 7fbafda49b80 20 client.16749 trim_cache size 0 max 16384 2024-06-10T01:11:06.943 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:11:06.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.937+0000 7fbaf31c6700 1 -- 172.21.15.7:0/2475712903 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fbadc0a2c20 msgr2=0x7fbadc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:06.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.937+0000 7fbaf31c6700 1 --2- 172.21.15.7:0/2475712903 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fbadc0a2c20 0x7fbadc0a50e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbaec199af0 tx=0x7fbae0009260 comp rx=0 tx=0).stop 2024-06-10T01:11:06.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.937+0000 7fbaf31c6700 1 -- 172.21.15.7:0/2475712903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaec108580 msgr2=0x7fbaec1989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:06.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.937+0000 7fbaf31c6700 1 --2- 172.21.15.7:0/2475712903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaec108580 0x7fbaec1989b0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fbae4006070 tx=0x7fbae4007a20 comp rx=0 tx=0).stop 2024-06-10T01:11:06.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.937+0000 7fbaf31c6700 1 -- 172.21.15.7:0/2475712903 shutdown_connections 2024-06-10T01:11:06.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.937+0000 7fbaf31c6700 1 --2- 172.21.15.7:0/2475712903 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fbadc0a2c20 0x7fbadc0a50e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.937+0000 7fbaf31c6700 1 --2- 172.21.15.7:0/2475712903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaec108580 0x7fbaec1989b0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.937+0000 7fbaf31c6700 1 --2- 172.21.15.7:0/2475712903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbaec06ac60 0x7fbaec19e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.937+0000 7fbaf31c6700 1 --2- 172.21.15.7:0/2475712903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaec06b620 0x7fbaec19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:06.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.937+0000 7fbaf31c6700 1 -- 172.21.15.7:0/2475712903 >> 172.21.15.7:0/2475712903 conn(0x7fbaec078430 msgr2=0x7fbaec0ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:06.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.937+0000 7fbaf31c6700 1 -- 172.21.15.7:0/2475712903 shutdown_connections 2024-06-10T01:11:06.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:06.938+0000 7fbaf31c6700 1 -- 172.21.15.7:0/2475712903 wait complete. 2024-06-10T01:11:07.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:07 smithi175 ceph-mon[36636]: pgmap v585: 129 pgs: 129 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 323 op/s 2024-06-10T01:11:07.729 INFO:tasks.workunit.client.0.smithi007.stdout: 1 345759 8.55 MB/sec execute 108 sec latency 7676.371 ms 2024-06-10T01:11:07.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:07 smithi007 ceph-mon[32398]: pgmap v585: 129 pgs: 129 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 323 op/s 2024-06-10T01:11:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.798+0000 7f0e8a7e7640 1 -- 172.21.15.7:0/209811247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e84100900 msgr2=0x7f0e84100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.798+0000 7f0e8a7e7640 1 --2- 172.21.15.7:0/209811247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e84100900 0x7f0e84100d20 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f0e74009900 tx=0x7f0e7402a410 comp rx=0 tx=0).stop 2024-06-10T01:11:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.799+0000 7f0e8a7e7640 1 -- 172.21.15.7:0/209811247 shutdown_connections 2024-06-10T01:11:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.799+0000 7f0e8a7e7640 1 --2- 172.21.15.7:0/209811247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e841024b0 0x7f0e84108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.799+0000 7f0e8a7e7640 1 --2- 172.21.15.7:0/209811247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e84101af0 0x7f0e84101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.799+0000 7f0e8a7e7640 1 --2- 172.21.15.7:0/209811247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e84100900 0x7f0e84100d20 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.799+0000 7f0e8a7e7640 1 -- 172.21.15.7:0/209811247 >> 172.21.15.7:0/209811247 conn(0x7f0e840fbe70 msgr2=0x7f0e840fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.799+0000 7f0e8a7e7640 1 -- 172.21.15.7:0/209811247 shutdown_connections 2024-06-10T01:11:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.799+0000 7f0e8a7e7640 1 -- 172.21.15.7:0/209811247 wait complete. 2024-06-10T01:11:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.799+0000 7f0e8a7e7640 1 Processor -- start 2024-06-10T01:11:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.799+0000 7f0e8a7e7640 1 -- start start 2024-06-10T01:11:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.800+0000 7f0e8a7e7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e84100900 0x7f0e84196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.800+0000 7f0e8a7e7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e84101af0 0x7f0e841971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.800+0000 7f0e8a7e7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e841024b0 0x7f0e8419e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.800+0000 7f0e8a7e7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8410c550 con 0x7f0e841024b0 2024-06-10T01:11:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.800+0000 7f0e8a7e7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8410c3d0 con 0x7f0e84100900 2024-06-10T01:11:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.800+0000 7f0e8a7e7640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8410c6d0 con 0x7f0e84101af0 2024-06-10T01:11:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.800+0000 7f0e88d84640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e841024b0 0x7f0e8419e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.800+0000 7f0e88d84640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e841024b0 0x7f0e8419e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48664/0 (socket says 172.21.15.7:48664) 2024-06-10T01:11:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.800+0000 7f0e88d84640 1 -- 172.21.15.7:0/191402647 learned_addr learned my addr 172.21.15.7:0/191402647 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.800+0000 7f0e837fe640 1 --2- 172.21.15.7:0/191402647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e84101af0 0x7f0e841971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.800+0000 7f0e83fff640 1 --2- 172.21.15.7:0/191402647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e84100900 0x7f0e84196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.800+0000 7f0e88d84640 1 -- 172.21.15.7:0/191402647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e84101af0 msgr2=0x7f0e841971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.800+0000 7f0e88d84640 1 --2- 172.21.15.7:0/191402647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e84101af0 0x7f0e841971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.800+0000 7f0e88d84640 1 -- 172.21.15.7:0/191402647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e84100900 msgr2=0x7f0e84196c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.800+0000 7f0e88d84640 1 --2- 172.21.15.7:0/191402647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e84100900 0x7f0e84196c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.800+0000 7f0e88d84640 1 -- 172.21.15.7:0/191402647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e8419e880 con 0x7f0e841024b0 2024-06-10T01:11:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.801+0000 7f0e88d84640 1 --2- 172.21.15.7:0/191402647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e841024b0 0x7f0e8419e1e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00eaa0 tx=0x7f0e7c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.801+0000 7f0e817fa640 1 -- 172.21.15.7:0/191402647 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c00cdb0 con 0x7f0e841024b0 2024-06-10T01:11:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.801+0000 7f0e8a7e7640 1 -- 172.21.15.7:0/191402647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e8419eb70 con 0x7f0e841024b0 2024-06-10T01:11:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.801+0000 7f0e817fa640 1 -- 172.21.15.7:0/191402647 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e7c005020 con 0x7f0e841024b0 2024-06-10T01:11:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.801+0000 7f0e817fa640 1 -- 172.21.15.7:0/191402647 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c0116b0 con 0x7f0e841024b0 2024-06-10T01:11:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.801+0000 7f0e8a7e7640 1 -- 172.21.15.7:0/191402647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e8419f090 con 0x7f0e841024b0 2024-06-10T01:11:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.803+0000 7f0e817fa640 1 -- 172.21.15.7:0/191402647 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0e7c005560 con 0x7f0e841024b0 2024-06-10T01:11:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.804+0000 7f0e817fa640 1 --2- 172.21.15.7:0/191402647 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0e6c0a2b80 0x7f0e6c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.804+0000 7f0e8d5d2740 1 Processor -- start 2024-06-10T01:11:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.804+0000 7f0e83fff640 1 --2- 172.21.15.7:0/191402647 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0e6c0a2b80 0x7f0e6c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.804+0000 7f0e8d5d2740 1 -- start start 2024-06-10T01:11:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.804+0000 7f0e817fa640 1 -- 172.21.15.7:0/191402647 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f0e7c0cf660 con 0x7f0e841024b0 2024-06-10T01:11:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.804+0000 7f0e8d5d2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1c538b670 0x55e1c5389dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.804+0000 7f0e8d5d2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1c538a300 0x55e1c538a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.804+0000 7f0e8d5d2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1c538fec0 0x55e1c53922b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.804+0000 7f0e8d5d2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e1c52ed4f0 con 0x55e1c538fec0 2024-06-10T01:11:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.804+0000 7f0e8d5d2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e1c528b870 con 0x55e1c538a300 2024-06-10T01:11:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.804+0000 7f0e8d5d2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e1c52e94b0 con 0x55e1c538b670 2024-06-10T01:11:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.805+0000 7f0e88d84640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1c538fec0 0x55e1c53922b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.805+0000 7f0e88d84640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1c538fec0 0x55e1c53922b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48674/0 (socket says 172.21.15.7:48674) 2024-06-10T01:11:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.805+0000 7f0e88d84640 1 -- 172.21.15.7:0/1376092419 learned_addr learned my addr 172.21.15.7:0/1376092419 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.805+0000 7f0e83fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1c538b670 0x55e1c5389dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.805+0000 7f0e837fe640 1 --2- 172.21.15.7:0/1376092419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1c538a300 0x55e1c538a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.805+0000 7f0e83fff640 1 --2- 172.21.15.7:0/191402647 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0e6c0a2b80 0x7f0e6c0a5040 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0e74005ab0 tx=0x7f0e74000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.805+0000 7f0e88d84640 1 -- 172.21.15.7:0/1376092419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1c538b670 msgr2=0x55e1c5389dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.805+0000 7f0e88d84640 1 --2- 172.21.15.7:0/1376092419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1c538b670 0x55e1c5389dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.805+0000 7f0e88d84640 1 -- 172.21.15.7:0/1376092419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1c538a300 msgr2=0x55e1c538a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.805+0000 7f0e88d84640 1 --2- 172.21.15.7:0/1376092419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1c538a300 0x55e1c538a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.805+0000 7f0e88d84640 1 -- 172.21.15.7:0/1376092419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1c52a79b0 con 0x55e1c538fec0 2024-06-10T01:11:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.805+0000 7f0e88d84640 1 --2- 172.21.15.7:0/1376092419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1c538fec0 0x55e1c53922b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c09bb20 tx=0x7f0e7c09bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2a9230811f814d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:07.805 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:11:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.805+0000 7f0e6a7fc640 1 -- 172.21.15.7:0/1376092419 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c01f040 con 0x55e1c538fec0 2024-06-10T01:11:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.805+0000 7f0e6a7fc640 1 -- 172.21.15.7:0/1376092419 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e7c0cf350 con 0x55e1c538fec0 2024-06-10T01:11:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.805+0000 7f0e6a7fc640 1 -- 172.21.15.7:0/1376092419 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c09b2b0 con 0x55e1c538fec0 2024-06-10T01:11:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.806+0000 7f0e8d5d2740 1 -- 172.21.15.7:0/1376092419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1c538fec0 msgr2=0x55e1c53922b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.806+0000 7f0e8d5d2740 1 --2- 172.21.15.7:0/1376092419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1c538fec0 0x55e1c53922b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c09bb20 tx=0x7f0e7c09bc10 comp rx=0 tx=0).stop 2024-06-10T01:11:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.806+0000 7f0e8d5d2740 1 -- 172.21.15.7:0/1376092419 shutdown_connections 2024-06-10T01:11:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.806+0000 7f0e8d5d2740 1 --2- 172.21.15.7:0/1376092419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1c538fec0 0x55e1c53922b0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.806+0000 7f0e8d5d2740 1 --2- 172.21.15.7:0/1376092419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1c538a300 0x55e1c538a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.806+0000 7f0e8d5d2740 1 --2- 172.21.15.7:0/1376092419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1c538b670 0x55e1c5389dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.806+0000 7f0e8d5d2740 1 -- 172.21.15.7:0/1376092419 >> 172.21.15.7:0/1376092419 conn(0x55e1c5385680 msgr2=0x55e1c5385a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.806+0000 7f0e8d5d2740 1 -- 172.21.15.7:0/1376092419 shutdown_connections 2024-06-10T01:11:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.806+0000 7f0e8d5d2740 1 -- 172.21.15.7:0/1376092419 wait complete. 2024-06-10T01:11:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.807+0000 7f0e8d5d2740 1 -- start start 2024-06-10T01:11:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.807+0000 7f0e8d5d2740 1 Processor -- start 2024-06-10T01:11:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.807+0000 7f0e8d5d2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1c538a300 0x55e1c53942e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.807+0000 7f0e8d5d2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1c538b670 0x55e1c5394820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.807+0000 7f0e8d5d2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1c538fec0 0x55e1c539b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.807+0000 7f0e8d5d2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e1c5392ac0 con 0x55e1c538fec0 2024-06-10T01:11:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.807+0000 7f0e8d5d2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e1c5276ae0 con 0x55e1c538a300 2024-06-10T01:11:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.807+0000 7f0e8d5d2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e1c5392c40 con 0x55e1c538b670 2024-06-10T01:11:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.807+0000 7f0e8d5d2740 10 client.0 _ll_register_callbacks cb 0x55e1c52af9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:11:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.807+0000 7f0e88d84640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1c538fec0 0x55e1c539b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.807+0000 7f0e88d84640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1c538fec0 0x55e1c539b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48680/0 (socket says 172.21.15.7:48680) 2024-06-10T01:11:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.807+0000 7f0e88d84640 1 -- 172.21.15.7:0/155873903 learned_addr learned my addr 172.21.15.7:0/155873903 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.807+0000 7f0e83fff640 1 --2- 172.21.15.7:0/155873903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1c538a300 0x55e1c53942e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.807+0000 7f0e837fe640 1 --2- 172.21.15.7:0/155873903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1c538b670 0x55e1c5394820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.808+0000 7f0e88d84640 1 -- 172.21.15.7:0/155873903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1c538b670 msgr2=0x55e1c5394820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.808+0000 7f0e88d84640 1 --2- 172.21.15.7:0/155873903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1c538b670 0x55e1c5394820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.808+0000 7f0e88d84640 1 -- 172.21.15.7:0/155873903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1c538a300 msgr2=0x55e1c53942e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.808+0000 7f0e88d84640 1 --2- 172.21.15.7:0/155873903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1c538a300 0x55e1c53942e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.808+0000 7f0e88d84640 1 -- 172.21.15.7:0/155873903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1c5001210 con 0x55e1c538fec0 2024-06-10T01:11:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.808+0000 7f0e88d84640 1 --2- 172.21.15.7:0/155873903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1c538fec0 0x55e1c539b860 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c01c040 tx=0x7f0e7c005400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919c48ed9080fe0e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.808+0000 7f0e637fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:11:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.808+0000 7f0e637fe640 1 -- 172.21.15.7:0/155873903 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c0b9060 con 0x55e1c538fec0 2024-06-10T01:11:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.808+0000 7f0e637fe640 1 -- 172.21.15.7:0/155873903 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f0e7c097a60 con 0x55e1c538fec0 2024-06-10T01:11:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.808+0000 7f0e8d5d2740 1 -- 172.21.15.7:0/155873903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e7c0cf350 con 0x55e1c538fec0 2024-06-10T01:11:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.808+0000 7f0e637fe640 1 -- 172.21.15.7:0/155873903 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0e7c01f040 con 0x55e1c538fec0 2024-06-10T01:11:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.808+0000 7f0e8d5d2740 10 client.16761 fetch_fsmap learned FSMap version 32 2024-06-10T01:11:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.808+0000 7f0e8d5d2740 1 -- 172.21.15.7:0/155873903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e1c50cd620 con 0x55e1c538fec0 2024-06-10T01:11:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.808+0000 7f0e637fe640 1 -- 172.21.15.7:0/155873903 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e7c0cf350 con 0x55e1c538fec0 2024-06-10T01:11:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.808+0000 7f0e637fe640 1 -- 172.21.15.7:0/155873903 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f0e7c0b4ce0 con 0x55e1c538fec0 2024-06-10T01:11:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.809+0000 7f0e8d5d2740 10 client.16761 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:11:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.809+0000 7f0e8d5d2740 10 client.16761 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:11:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.809+0000 7f0e8d5d2740 20 client.16761 populate_metadata read hostname 'smithi007' 2024-06-10T01:11:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.809+0000 7f0e8d5d2740 1 --2- 172.21.15.7:0/155873903 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e1c539fd50 0x55e1c53a2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.809+0000 7f0e8d5d2740 4 client.16761 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:11:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.809+0000 7f0e8d5d2740 1 -- 172.21.15.7:0/155873903 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1c5300f10 con 0x55e1c539fd50 2024-06-10T01:11:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.809+0000 7f0e83fff640 1 --2- 172.21.15.7:0/155873903 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e1c539fd50 0x55e1c53a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.809+0000 7f0e637fe640 10 client.16761 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.809+0000 7f0e83fff640 1 --2- 172.21.15.7:0/155873903 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e1c539fd50 0x55e1c53a2140 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ff6edd4b9163804 server_cookie=e7f52dec83630bee in_seq=0 out_seq=0 2024-06-10T01:11:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.810+0000 7f0e637fe640 1 -- 172.21.15.7:0/155873903 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e1c5300f10 con 0x55e1c539fd50 2024-06-10T01:11:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.810+0000 7f0e637fe640 10 client.16761 handle_command_reply: tid=0 2024-06-10T01:11:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.810+0000 7f0e8d5d2740 1 client.16761 shutdown 2024-06-10T01:11:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.810+0000 7f0e8d5d2740 1 -- 172.21.15.7:0/155873903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1c538fec0 msgr2=0x55e1c539b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.810+0000 7f0e8d5d2740 1 --2- 172.21.15.7:0/155873903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1c538fec0 0x55e1c539b860 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c01c040 tx=0x7f0e7c005400 comp rx=0 tx=0).stop 2024-06-10T01:11:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.810+0000 7f0e8d5d2740 1 -- 172.21.15.7:0/155873903 shutdown_connections 2024-06-10T01:11:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.810+0000 7f0e8d5d2740 1 --2- 172.21.15.7:0/155873903 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e1c539fd50 0x55e1c53a2140 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.810+0000 7f0e8d5d2740 1 --2- 172.21.15.7:0/155873903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1c538fec0 0x55e1c539b860 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.810+0000 7f0e8d5d2740 1 --2- 172.21.15.7:0/155873903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1c538b670 0x55e1c5394820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.810+0000 7f0e8d5d2740 1 --2- 172.21.15.7:0/155873903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1c538a300 0x55e1c53942e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.810+0000 7f0e637fe640 0 client.16761 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.810+0000 7f0e8d5d2740 1 -- 172.21.15.7:0/155873903 >> 172.21.15.7:0/155873903 conn(0x55e1c5385680 msgr2=0x55e1c53847e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.810+0000 7f0e8d5d2740 1 -- 172.21.15.7:0/155873903 shutdown_connections 2024-06-10T01:11:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.810+0000 7f0e8d5d2740 1 -- 172.21.15.7:0/155873903 wait complete. 2024-06-10T01:11:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.810+0000 7f0e8d5d2740 20 client.16761 trim_cache size 0 max 16384 2024-06-10T01:11:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.826+0000 7f0e8d5d2740 1 Processor -- start 2024-06-10T01:11:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.826+0000 7f0e8d5d2740 1 -- start start 2024-06-10T01:11:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.826+0000 7f0e8d5d2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1c538a300 0x55e1c5384f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.826+0000 7f0e8d5d2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1c538b670 0x55e1c539b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.826+0000 7f0e8d5d2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1c538fec0 0x55e1c5388590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.826+0000 7f0e8d5d2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0e841aacf0 con 0x55e1c538fec0 2024-06-10T01:11:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.826+0000 7f0e8d5d2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e841a85c0 con 0x55e1c538a300 2024-06-10T01:11:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.826+0000 7f0e8d5d2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0e841a6410 con 0x55e1c538b670 2024-06-10T01:11:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.826+0000 7f0e88d84640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1c538fec0 0x55e1c5388590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.826+0000 7f0e88d84640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1c538fec0 0x55e1c5388590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48684/0 (socket says 172.21.15.7:48684) 2024-06-10T01:11:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.826+0000 7f0e88d84640 1 -- 172.21.15.7:0/2548852024 learned_addr learned my addr 172.21.15.7:0/2548852024 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.826+0000 7f0e83fff640 1 --2- 172.21.15.7:0/2548852024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1c538a300 0x55e1c5384f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.827+0000 7f0e837fe640 1 --2- 172.21.15.7:0/2548852024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1c538b670 0x55e1c539b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.827+0000 7f0e88d84640 1 -- 172.21.15.7:0/2548852024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1c538b670 msgr2=0x55e1c539b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.827+0000 7f0e88d84640 1 --2- 172.21.15.7:0/2548852024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1c538b670 0x55e1c539b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.827+0000 7f0e88d84640 1 -- 172.21.15.7:0/2548852024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1c538a300 msgr2=0x55e1c5384f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.827+0000 7f0e88d84640 1 --2- 172.21.15.7:0/2548852024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1c538a300 0x55e1c5384f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.827+0000 7f0e88d84640 1 -- 172.21.15.7:0/2548852024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1c5392710 con 0x55e1c538fec0 2024-06-10T01:11:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.827+0000 7f0e83fff640 1 --2- 172.21.15.7:0/2548852024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1c538a300 0x55e1c5384f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:11:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.827+0000 7f0e88d84640 1 --2- 172.21.15.7:0/2548852024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1c538fec0 0x55e1c5388590 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00efd0 tx=0x7f0e7c09bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9be326ba4caf500 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.827+0000 7f0e69ffb640 1 -- 172.21.15.7:0/2548852024 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c0b3070 con 0x55e1c538fec0 2024-06-10T01:11:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.827+0000 7f0e69ffb640 1 -- 172.21.15.7:0/2548852024 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e7c097a60 con 0x55e1c538fec0 2024-06-10T01:11:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.827+0000 7f0e69ffb640 1 -- 172.21.15.7:0/2548852024 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c0b9040 con 0x55e1c538fec0 2024-06-10T01:11:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.828+0000 7f0e8d5d2740 1 -- 172.21.15.7:0/2548852024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1c538fec0 msgr2=0x55e1c5388590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.828+0000 7f0e8d5d2740 1 --2- 172.21.15.7:0/2548852024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1c538fec0 0x55e1c5388590 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00efd0 tx=0x7f0e7c09bc10 comp rx=0 tx=0).stop 2024-06-10T01:11:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.828+0000 7f0e8d5d2740 1 -- 172.21.15.7:0/2548852024 shutdown_connections 2024-06-10T01:11:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.828+0000 7f0e8d5d2740 1 --2- 172.21.15.7:0/2548852024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1c538fec0 0x55e1c5388590 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.828+0000 7f0e8d5d2740 1 --2- 172.21.15.7:0/2548852024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1c538b670 0x55e1c539b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.828+0000 7f0e8d5d2740 1 --2- 172.21.15.7:0/2548852024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1c538a300 0x55e1c5384f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.828+0000 7f0e8d5d2740 1 -- 172.21.15.7:0/2548852024 >> 172.21.15.7:0/2548852024 conn(0x55e1c5385680 msgr2=0x55e1c538c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.828+0000 7f0e8d5d2740 1 -- 172.21.15.7:0/2548852024 shutdown_connections 2024-06-10T01:11:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.828+0000 7f0e8d5d2740 1 -- 172.21.15.7:0/2548852024 wait complete. 2024-06-10T01:11:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.828+0000 7f0e8d5d2740 1 -- start start 2024-06-10T01:11:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.828+0000 7f0e8d5d2740 1 Processor -- start 2024-06-10T01:11:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.829+0000 7f0e8d5d2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1c538a300 0x55e1c5398730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.829+0000 7f0e8d5d2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1c538b670 0x55e1c5398c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.829+0000 7f0e8d5d2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1c538fec0 0x55e1c5408920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.829+0000 7f0e8d5d2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0e841a6a80 con 0x55e1c538fec0 2024-06-10T01:11:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.829+0000 7f0e8d5d2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e841a7ef0 con 0x55e1c538a300 2024-06-10T01:11:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.829+0000 7f0e8d5d2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0e841a7150 con 0x55e1c538b670 2024-06-10T01:11:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.829+0000 7f0e8d5d2740 10 client.0 _ll_register_callbacks cb 0x7f0e7c09f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:11:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.829+0000 7f0e88d84640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1c538fec0 0x55e1c5408920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.829+0000 7f0e88d84640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1c538fec0 0x55e1c5408920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48688/0 (socket says 172.21.15.7:48688) 2024-06-10T01:11:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.829+0000 7f0e88d84640 1 -- 172.21.15.7:0/1437794810 learned_addr learned my addr 172.21.15.7:0/1437794810 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.829+0000 7f0e83fff640 1 --2- 172.21.15.7:0/1437794810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1c538a300 0x55e1c5398730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.829+0000 7f0e837fe640 1 --2- 172.21.15.7:0/1437794810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1c538b670 0x55e1c5398c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.829+0000 7f0e88d84640 1 -- 172.21.15.7:0/1437794810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1c538b670 msgr2=0x55e1c5398c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.829+0000 7f0e88d84640 1 --2- 172.21.15.7:0/1437794810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1c538b670 0x55e1c5398c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.829+0000 7f0e88d84640 1 -- 172.21.15.7:0/1437794810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1c538a300 msgr2=0x55e1c5398730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.829+0000 7f0e88d84640 1 --2- 172.21.15.7:0/1437794810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1c538a300 0x55e1c5398730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.829+0000 7f0e88d84640 1 -- 172.21.15.7:0/1437794810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1c5408e60 con 0x55e1c538fec0 2024-06-10T01:11:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.829+0000 7f0e88d84640 1 --2- 172.21.15.7:0/1437794810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1c538fec0 0x55e1c5408920 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c09bb20 tx=0x7f0e7c09b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc5a9da4d0dfcf17 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.829+0000 7f0e637fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:11:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.829+0000 7f0e637fe640 1 -- 172.21.15.7:0/1437794810 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c0b3070 con 0x55e1c538fec0 2024-06-10T01:11:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.830+0000 7f0e637fe640 1 -- 172.21.15.7:0/1437794810 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f0e7c004bb0 con 0x55e1c538fec0 2024-06-10T01:11:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.830+0000 7f0e8d5d2740 1 -- 172.21.15.7:0/1437794810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e7c097a60 con 0x55e1c538fec0 2024-06-10T01:11:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.830+0000 7f0e637fe640 1 -- 172.21.15.7:0/1437794810 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0e7c0ac050 con 0x55e1c538fec0 2024-06-10T01:11:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.830+0000 7f0e8d5d2740 10 client.16767 fetch_fsmap learned FSMap version 32 2024-06-10T01:11:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.830+0000 7f0e8d5d2740 1 -- 172.21.15.7:0/1437794810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e1c5408e60 con 0x55e1c538fec0 2024-06-10T01:11:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.830+0000 7f0e637fe640 1 -- 172.21.15.7:0/1437794810 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e7c097a60 con 0x55e1c538fec0 2024-06-10T01:11:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.830+0000 7f0e637fe640 1 -- 172.21.15.7:0/1437794810 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f0e7c083070 con 0x55e1c538fec0 2024-06-10T01:11:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.830+0000 7f0e8d5d2740 10 client.16767 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:11:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.830+0000 7f0e8d5d2740 10 client.16767 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:11:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.830+0000 7f0e8d5d2740 20 client.16767 populate_metadata read hostname 'smithi007' 2024-06-10T01:11:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.830+0000 7f0e8d5d2740 1 --2- 172.21.15.7:0/1437794810 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e1c540a6b0 0x55e1c540cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.830+0000 7f0e8d5d2740 4 client.16767 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:11:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.830+0000 7f0e8d5d2740 1 -- 172.21.15.7:0/1437794810 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e1c52fac20 con 0x55e1c540a6b0 2024-06-10T01:11:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.830+0000 7f0e83fff640 1 --2- 172.21.15.7:0/1437794810 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e1c540a6b0 0x55e1c540cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.831+0000 7f0e637fe640 10 client.16767 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.831+0000 7f0e83fff640 1 --2- 172.21.15.7:0/1437794810 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e1c540a6b0 0x55e1c540cad0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cb1ca9da61585e3 server_cookie=60f5465809e70923 in_seq=0 out_seq=0 2024-06-10T01:11:07.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:07 smithi078 ceph-mon[36683]: pgmap v585: 129 pgs: 129 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 323 op/s 2024-06-10T01:11:07.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.890+0000 7f0e637fe640 1 -- 172.21.15.7:0/1437794810 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e1c52fac20 con 0x55e1c540a6b0 2024-06-10T01:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.890+0000 7f0e637fe640 10 client.16767 handle_command_reply: tid=0 2024-06-10T01:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.890+0000 7f0e8d5d2740 1 client.16767 shutdown 2024-06-10T01:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.891+0000 7f0e8d5d2740 1 -- 172.21.15.7:0/1437794810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1c538fec0 msgr2=0x55e1c5408920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.891+0000 7f0e8d5d2740 1 --2- 172.21.15.7:0/1437794810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1c538fec0 0x55e1c5408920 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c09bb20 tx=0x7f0e7c09b6d0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.891+0000 7f0e8d5d2740 1 -- 172.21.15.7:0/1437794810 shutdown_connections 2024-06-10T01:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.891+0000 7f0e8d5d2740 1 --2- 172.21.15.7:0/1437794810 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e1c540a6b0 0x55e1c540cad0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.891+0000 7f0e8d5d2740 1 --2- 172.21.15.7:0/1437794810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1c538fec0 0x55e1c5408920 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.891+0000 7f0e8d5d2740 1 --2- 172.21.15.7:0/1437794810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1c538b670 0x55e1c5398c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.891+0000 7f0e637fe640 0 client.16767 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.891+0000 7f0e8d5d2740 1 --2- 172.21.15.7:0/1437794810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1c538a300 0x55e1c5398730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.891+0000 7f0e8d5d2740 1 -- 172.21.15.7:0/1437794810 >> 172.21.15.7:0/1437794810 conn(0x55e1c5385680 msgr2=0x55e1c53942c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.891+0000 7f0e8d5d2740 1 -- 172.21.15.7:0/1437794810 shutdown_connections 2024-06-10T01:11:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.891+0000 7f0e8d5d2740 1 -- 172.21.15.7:0/1437794810 wait complete. 2024-06-10T01:11:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.891+0000 7f0e8d5d2740 20 client.16767 trim_cache size 0 max 16384 2024-06-10T01:11:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:11:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:11:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:11:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.891+0000 7f0e8a7e7640 1 -- 172.21.15.7:0/191402647 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0e6c0a2b80 msgr2=0x7f0e6c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.891+0000 7f0e8a7e7640 1 --2- 172.21.15.7:0/191402647 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0e6c0a2b80 0x7f0e6c0a5040 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0e74005ab0 tx=0x7f0e74000bc0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.892+0000 7f0e8a7e7640 1 -- 172.21.15.7:0/191402647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e841024b0 msgr2=0x7f0e8419e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.892+0000 7f0e8a7e7640 1 --2- 172.21.15.7:0/191402647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e841024b0 0x7f0e8419e1e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00eaa0 tx=0x7f0e7c00ef60 comp rx=0 tx=0).stop 2024-06-10T01:11:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.892+0000 7f0e8a7e7640 1 -- 172.21.15.7:0/191402647 shutdown_connections 2024-06-10T01:11:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.892+0000 7f0e8a7e7640 1 --2- 172.21.15.7:0/191402647 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0e6c0a2b80 0x7f0e6c0a5040 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.892+0000 7f0e8a7e7640 1 --2- 172.21.15.7:0/191402647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e841024b0 0x7f0e8419e1e0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.892+0000 7f0e8a7e7640 1 --2- 172.21.15.7:0/191402647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e84101af0 0x7f0e841971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.892+0000 7f0e8a7e7640 1 --2- 172.21.15.7:0/191402647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e84100900 0x7f0e84196c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.892+0000 7f0e8a7e7640 1 -- 172.21.15.7:0/191402647 >> 172.21.15.7:0/191402647 conn(0x7f0e840fbe70 msgr2=0x7f0e84104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.892+0000 7f0e8a7e7640 1 -- 172.21.15.7:0/191402647 shutdown_connections 2024-06-10T01:11:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:07.892+0000 7f0e8a7e7640 1 -- 172.21.15.7:0/191402647 wait complete. 2024-06-10T01:11:08.089 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:11:08.729 INFO:tasks.workunit.client.0.smithi007.stdout: 1 345759 8.47 MB/sec execute 109 sec latency 8676.422 ms 2024-06-10T01:11:09.444 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:09 smithi007 ceph-mon[32398]: pgmap v586: 129 pgs: 129 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 2.2 MiB/s wr, 242 op/s 2024-06-10T01:11:09.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.685+0000 7f3633b68700 1 -- 172.21.15.7:0/2274904425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f362c102740 msgr2=0x7f362c102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:09.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.685+0000 7f3633b68700 1 --2- 172.21.15.7:0/2274904425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f362c102740 0x7f362c102b20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f36280099c0 tx=0x7f3628009cd0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.685+0000 7f3633b68700 1 -- 172.21.15.7:0/2274904425 shutdown_connections 2024-06-10T01:11:09.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.685+0000 7f3633b68700 1 --2- 172.21.15.7:0/2274904425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f362c107840 0x7f362c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.685+0000 7f3633b68700 1 --2- 172.21.15.7:0/2274904425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f362c102740 0x7f362c102b20 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.685+0000 7f3633b68700 1 --2- 172.21.15.7:0/2274904425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f362c1030f0 0x7f362c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.685+0000 7f3633b68700 1 -- 172.21.15.7:0/2274904425 >> 172.21.15.7:0/2274904425 conn(0x7f362c0795f0 msgr2=0x7f362c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:09.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.685+0000 7f3633b68700 1 -- 172.21.15.7:0/2274904425 shutdown_connections 2024-06-10T01:11:09.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.685+0000 7f3633b68700 1 -- 172.21.15.7:0/2274904425 wait complete. 2024-06-10T01:11:09.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.685+0000 7f3633b68700 1 Processor -- start 2024-06-10T01:11:09.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.685+0000 7f3633b68700 1 -- start start 2024-06-10T01:11:09.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.686+0000 7f3633b68700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f362c102740 0x7f362c19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:09.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.686+0000 7f3633b68700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f362c1030f0 0x7f362c19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:09.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.686+0000 7f3633b68700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f362c107840 0x7f362c198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:09.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.686+0000 7f3633b68700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f362c112850 con 0x7f362c1030f0 2024-06-10T01:11:09.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.686+0000 7f3633b68700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1126d0 con 0x7f362c107840 2024-06-10T01:11:09.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.686+0000 7f3633b68700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1129d0 con 0x7f362c102740 2024-06-10T01:11:09.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.686+0000 7f3631103700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f362c1030f0 0x7f362c19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:09.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.686+0000 7f3631103700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f362c1030f0 0x7f362c19ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48700/0 (socket says 172.21.15.7:48700) 2024-06-10T01:11:09.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.686+0000 7f3631103700 1 -- 172.21.15.7:0/3229431353 learned_addr learned my addr 172.21.15.7:0/3229431353 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:09.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.686+0000 7f3631904700 1 --2- 172.21.15.7:0/3229431353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f362c102740 0x7f362c19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:09.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.686+0000 7f3632105700 1 --2- 172.21.15.7:0/3229431353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f362c107840 0x7f362c198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:09.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.686+0000 7f3631103700 1 -- 172.21.15.7:0/3229431353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f362c102740 msgr2=0x7f362c19e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:09.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.686+0000 7f3631103700 1 --2- 172.21.15.7:0/3229431353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f362c102740 0x7f362c19e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.686+0000 7f3631103700 1 -- 172.21.15.7:0/3229431353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f362c107840 msgr2=0x7f362c198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:09.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.686+0000 7f3631103700 1 --2- 172.21.15.7:0/3229431353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f362c107840 0x7f362c198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.686+0000 7f3631103700 1 -- 172.21.15.7:0/3229431353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f362c1992b0 con 0x7f362c1030f0 2024-06-10T01:11:09.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.686+0000 7f3632105700 1 --2- 172.21.15.7:0/3229431353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f362c107840 0x7f362c198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:11:09.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.686+0000 7f3631103700 1 --2- 172.21.15.7:0/3229431353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f362c1030f0 0x7f362c19ee80 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f361c00b530 tx=0x7f361c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:09.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.687+0000 7f3622ffd700 1 -- 172.21.15.7:0/3229431353 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c015020 con 0x7f362c1030f0 2024-06-10T01:11:09.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.687+0000 7f3622ffd700 1 -- 172.21.15.7:0/3229431353 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f361c011d10 con 0x7f362c1030f0 2024-06-10T01:11:09.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.687+0000 7f3633b68700 1 -- 172.21.15.7:0/3229431353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f362c1995a0 con 0x7f362c1030f0 2024-06-10T01:11:09.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.687+0000 7f3622ffd700 1 -- 172.21.15.7:0/3229431353 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c010e50 con 0x7f362c1030f0 2024-06-10T01:11:09.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.687+0000 7f3633b68700 1 -- 172.21.15.7:0/3229431353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f362c0746c0 con 0x7f362c1030f0 2024-06-10T01:11:09.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.689+0000 7f3622ffd700 1 -- 172.21.15.7:0/3229431353 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f361c018020 con 0x7f362c1030f0 2024-06-10T01:11:09.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.690+0000 7f3622ffd700 1 --2- 172.21.15.7:0/3229431353 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f36180a2dc0 0x7f36180a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:09.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.690+0000 7f3622ffd700 1 -- 172.21.15.7:0/3229431353 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f361c0cec20 con 0x7f362c1030f0 2024-06-10T01:11:09.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.690+0000 7f3631904700 1 --2- 172.21.15.7:0/3229431353 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f36180a2dc0 0x7f36180a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:09.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.690+0000 7f3631904700 1 --2- 172.21.15.7:0/3229431353 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f36180a2dc0 0x7f36180a5280 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f36280059d0 tx=0x7f3628005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:09.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.690+0000 7f363e3ebb80 1 Processor -- start 2024-06-10T01:11:09.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.691+0000 7f363e3ebb80 1 -- start start 2024-06-10T01:11:09.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.691+0000 7f363e3ebb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aefbb66640 0x55aefbb66a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:09.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.691+0000 7f363e3ebb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aefbb66f60 0x55aefbb609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:09.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.691+0000 7f363e3ebb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aefbb60f20 0x55aefbb61340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:09.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.691+0000 7f363e3ebb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55aefbab2810 con 0x55aefbb66f60 2024-06-10T01:11:09.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.691+0000 7f363e3ebb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55aefba5a720 con 0x55aefbb60f20 2024-06-10T01:11:09.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.691+0000 7f363e3ebb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55aefbaae7d0 con 0x55aefbb66640 2024-06-10T01:11:09.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.691+0000 7f3631103700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aefbb66f60 0x55aefbb609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:09.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.691+0000 7f3631103700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aefbb66f60 0x55aefbb609e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48702/0 (socket says 172.21.15.7:48702) 2024-06-10T01:11:09.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.691+0000 7f3631103700 1 -- 172.21.15.7:0/507142891 learned_addr learned my addr 172.21.15.7:0/507142891 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:09.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.691+0000 7f3632105700 1 --2- 172.21.15.7:0/507142891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aefbb60f20 0x55aefbb61340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:09.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.691+0000 7f3631904700 1 --2- 172.21.15.7:0/507142891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aefbb66640 0x55aefbb66a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:09.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.691+0000 7f3631103700 1 -- 172.21.15.7:0/507142891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aefbb66640 msgr2=0x55aefbb66a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:09.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.691+0000 7f3631103700 1 --2- 172.21.15.7:0/507142891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aefbb66640 0x55aefbb66a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.691+0000 7f3631103700 1 -- 172.21.15.7:0/507142891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aefbb60f20 msgr2=0x55aefbb61340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:09.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.691+0000 7f3631103700 1 --2- 172.21.15.7:0/507142891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aefbb60f20 0x55aefbb61340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.691+0000 7f3631103700 1 -- 172.21.15.7:0/507142891 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aefba50f50 con 0x55aefbb66f60 2024-06-10T01:11:09.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.691+0000 7f3632105700 1 --2- 172.21.15.7:0/507142891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aefbb60f20 0x55aefbb61340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:11:09.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.691+0000 7f3631904700 1 --2- 172.21.15.7:0/507142891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aefbb66640 0x55aefbb66a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:11:09.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.692+0000 7f3631103700 1 --2- 172.21.15.7:0/507142891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aefbb66f60 0x55aefbb609e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f361c09ae00 tx=0x7f361c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c39d8442265ab09 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:09.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.692+0000 7f3617fff700 1 -- 172.21.15.7:0/507142891 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c09f4b0 con 0x55aefbb66f60 2024-06-10T01:11:09.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.692+0000 7f3617fff700 1 -- 172.21.15.7:0/507142891 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f361c0a3d70 con 0x55aefbb66f60 2024-06-10T01:11:09.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.692+0000 7f3617fff700 1 -- 172.21.15.7:0/507142891 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c0a38e0 con 0x55aefbb66f60 2024-06-10T01:11:09.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.692+0000 7f363e3ebb80 1 -- 172.21.15.7:0/507142891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aefbb66f60 msgr2=0x55aefbb609e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:09.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.692+0000 7f363e3ebb80 1 --2- 172.21.15.7:0/507142891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aefbb66f60 0x55aefbb609e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f361c09ae00 tx=0x7f361c004750 comp rx=0 tx=0).stop 2024-06-10T01:11:09.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.693+0000 7f363e3ebb80 1 -- 172.21.15.7:0/507142891 shutdown_connections 2024-06-10T01:11:09.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.693+0000 7f363e3ebb80 1 --2- 172.21.15.7:0/507142891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aefbb60f20 0x55aefbb61340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.693+0000 7f363e3ebb80 1 --2- 172.21.15.7:0/507142891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aefbb66640 0x55aefbb66a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.693+0000 7f363e3ebb80 1 --2- 172.21.15.7:0/507142891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aefbb66f60 0x55aefbb609e0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.693+0000 7f363e3ebb80 1 -- 172.21.15.7:0/507142891 >> 172.21.15.7:0/507142891 conn(0x55aefba8bcc0 msgr2=0x55aefbb5b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:09.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.693+0000 7f363e3ebb80 1 -- 172.21.15.7:0/507142891 shutdown_connections 2024-06-10T01:11:09.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.693+0000 7f363e3ebb80 1 -- 172.21.15.7:0/507142891 wait complete. 2024-06-10T01:11:09.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.693+0000 7f363e3ebb80 1 -- start start 2024-06-10T01:11:09.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.693+0000 7f363e3ebb80 1 Processor -- start 2024-06-10T01:11:09.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.694+0000 7f363e3ebb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aefbb60f20 0x55aefbb6cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:09.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.694+0000 7f363e3ebb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aefbb66640 0x55aefbb69fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:09.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.694+0000 7f363e3ebb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aefbb66f60 0x55aefbb6a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:09.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.694+0000 7f363e3ebb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55aefbb6d9d0 con 0x55aefbb66640 2024-06-10T01:11:09.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.694+0000 7f363e3ebb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55aefba52bf0 con 0x55aefbb60f20 2024-06-10T01:11:09.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.694+0000 7f363e3ebb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55aefbb6db50 con 0x55aefbb66f60 2024-06-10T01:11:09.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.694+0000 7f363e3ebb80 10 client.0 _ll_register_callbacks cb 0x55aefba4c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:11:09.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.694+0000 7f3631103700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aefbb66640 0x55aefbb69fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:09.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.694+0000 7f3631103700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aefbb66640 0x55aefbb69fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48706/0 (socket says 172.21.15.7:48706) 2024-06-10T01:11:09.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.694+0000 7f3631103700 1 -- 172.21.15.7:0/1616040980 learned_addr learned my addr 172.21.15.7:0/1616040980 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:09.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.694+0000 7f3631904700 1 --2- 172.21.15.7:0/1616040980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aefbb60f20 0x55aefbb6cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:09.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.694+0000 7f3632105700 1 --2- 172.21.15.7:0/1616040980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aefbb66f60 0x55aefbb6a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:09.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.694+0000 7f3631103700 1 -- 172.21.15.7:0/1616040980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aefbb66f60 msgr2=0x55aefbb6a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:09.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.694+0000 7f3631103700 1 --2- 172.21.15.7:0/1616040980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aefbb66f60 0x55aefbb6a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.694+0000 7f3631103700 1 -- 172.21.15.7:0/1616040980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aefbb60f20 msgr2=0x55aefbb6cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:09.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.694+0000 7f3631103700 1 --2- 172.21.15.7:0/1616040980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aefbb60f20 0x55aefbb6cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.694+0000 7f3631103700 1 -- 172.21.15.7:0/1616040980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aefba018d0 con 0x55aefbb66640 2024-06-10T01:11:09.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.694+0000 7f3632105700 1 --2- 172.21.15.7:0/1616040980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aefbb66f60 0x55aefbb6a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:11:09.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.694+0000 7f3631103700 1 --2- 172.21.15.7:0/1616040980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aefbb66640 0x55aefbb69fa0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f361c0cc710 tx=0x7f361c004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4626622e6b577b7f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:09.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.694+0000 7f36157fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:11:09.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.694+0000 7f36157fa700 1 -- 172.21.15.7:0/1616040980 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c0b3030 con 0x55aefbb66640 2024-06-10T01:11:09.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.694+0000 7f36157fa700 1 -- 172.21.15.7:0/1616040980 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f361c09f170 con 0x55aefbb66640 2024-06-10T01:11:09.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.694+0000 7f363e3ebb80 1 -- 172.21.15.7:0/1616040980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f361c0a3d70 con 0x55aefbb66640 2024-06-10T01:11:09.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.695+0000 7f36157fa700 1 -- 172.21.15.7:0/1616040980 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f361c0661a0 con 0x55aefbb66640 2024-06-10T01:11:09.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.695+0000 7f363e3ebb80 10 client.16779 fetch_fsmap learned FSMap version 32 2024-06-10T01:11:09.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.695+0000 7f363e3ebb80 1 -- 172.21.15.7:0/1616040980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55aefbb6d9d0 con 0x55aefbb66640 2024-06-10T01:11:09.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.695+0000 7f36157fa700 1 -- 172.21.15.7:0/1616040980 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f361c0a3d70 con 0x55aefbb66640 2024-06-10T01:11:09.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.695+0000 7f36157fa700 1 -- 172.21.15.7:0/1616040980 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f361c018bc0 con 0x55aefbb66640 2024-06-10T01:11:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.695+0000 7f363e3ebb80 10 client.16779 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:11:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.695+0000 7f363e3ebb80 10 client.16779 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:11:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.695+0000 7f363e3ebb80 20 client.16779 populate_metadata read hostname 'smithi007' 2024-06-10T01:11:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.695+0000 7f363e3ebb80 1 --2- 172.21.15.7:0/1616040980 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55aefbb7a570 0x55aefbb7c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.695+0000 7f363e3ebb80 4 client.16779 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:11:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.695+0000 7f363e3ebb80 1 -- 172.21.15.7:0/1616040980 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aefbabfd10 con 0x55aefbb7a570 2024-06-10T01:11:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.695+0000 7f3631904700 1 --2- 172.21.15.7:0/1616040980 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55aefbb7a570 0x55aefbb7c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.696+0000 7f36157fa700 10 client.16779 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:11:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.696+0000 7f3631904700 1 --2- 172.21.15.7:0/1616040980 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55aefbb7a570 0x55aefbb7c960 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e372f37f11d81b27 server_cookie=ac85148c77fe9444 in_seq=0 out_seq=0 2024-06-10T01:11:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.697+0000 7f36157fa700 1 -- 172.21.15.7:0/1616040980 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55aefbabfd10 con 0x55aefbb7a570 2024-06-10T01:11:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.697+0000 7f36157fa700 10 client.16779 handle_command_reply: tid=0 2024-06-10T01:11:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.697+0000 7f363e3ebb80 1 client.16779 shutdown 2024-06-10T01:11:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.697+0000 7f363e3ebb80 1 -- 172.21.15.7:0/1616040980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aefbb66640 msgr2=0x55aefbb69fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.697+0000 7f363e3ebb80 1 --2- 172.21.15.7:0/1616040980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aefbb66640 0x55aefbb69fa0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f361c0cc710 tx=0x7f361c004860 comp rx=0 tx=0).stop 2024-06-10T01:11:09.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.697+0000 7f363e3ebb80 1 -- 172.21.15.7:0/1616040980 shutdown_connections 2024-06-10T01:11:09.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.697+0000 7f363e3ebb80 1 --2- 172.21.15.7:0/1616040980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aefbb66f60 0x55aefbb6a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.697+0000 7f363e3ebb80 1 --2- 172.21.15.7:0/1616040980 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55aefbb7a570 0x55aefbb7c960 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.697+0000 7f363e3ebb80 1 --2- 172.21.15.7:0/1616040980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aefbb60f20 0x55aefbb6cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.697+0000 7f363e3ebb80 1 --2- 172.21.15.7:0/1616040980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aefbb66640 0x55aefbb69fa0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.697+0000 7f363e3ebb80 1 -- 172.21.15.7:0/1616040980 >> 172.21.15.7:0/1616040980 conn(0x55aefba8bcc0 msgr2=0x55aefbb5f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:09.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.697+0000 7f36157fa700 0 client.16779 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:11:09.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.697+0000 7f363e3ebb80 1 -- 172.21.15.7:0/1616040980 shutdown_connections 2024-06-10T01:11:09.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.697+0000 7f363e3ebb80 1 -- 172.21.15.7:0/1616040980 wait complete. 2024-06-10T01:11:09.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.697+0000 7f363e3ebb80 20 client.16779 trim_cache size 0 max 16384 2024-06-10T01:11:09.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:09 smithi175 ceph-mon[36636]: pgmap v586: 129 pgs: 129 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 2.2 MiB/s wr, 242 op/s 2024-06-10T01:11:09.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.712+0000 7f363e3ebb80 1 Processor -- start 2024-06-10T01:11:09.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.713+0000 7f363e3ebb80 1 -- start start 2024-06-10T01:11:09.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.713+0000 7f363e3ebb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aefbb60f20 0x55aefbb609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:09.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.713+0000 7f363e3ebb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aefbb66640 0x55aefbb55510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:09.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.713+0000 7f363e3ebb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aefbb66f60 0x55aefbb65110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:09.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.713+0000 7f363e3ebb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1b2ea0 con 0x55aefbb60f20 2024-06-10T01:11:09.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.713+0000 7f363e3ebb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1b0730 con 0x55aefbb66f60 2024-06-10T01:11:09.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.713+0000 7f363e3ebb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1ae580 con 0x55aefbb66640 2024-06-10T01:11:09.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.713+0000 7f3631904700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aefbb60f20 0x55aefbb609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:09.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.713+0000 7f3631904700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aefbb60f20 0x55aefbb609e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48714/0 (socket says 172.21.15.7:48714) 2024-06-10T01:11:09.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.713+0000 7f3631904700 1 -- 172.21.15.7:0/3854309535 learned_addr learned my addr 172.21.15.7:0/3854309535 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:09.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.713+0000 7f3631103700 1 --2- 172.21.15.7:0/3854309535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aefbb66640 0x55aefbb55510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:09.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.713+0000 7f3631904700 1 -- 172.21.15.7:0/3854309535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aefbb66640 msgr2=0x55aefbb55510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:09.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.713+0000 7f3631904700 1 --2- 172.21.15.7:0/3854309535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aefbb66640 0x55aefbb55510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.713+0000 7f3632105700 1 --2- 172.21.15.7:0/3854309535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aefbb66f60 0x55aefbb65110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:09.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.713+0000 7f3631904700 1 -- 172.21.15.7:0/3854309535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aefbb66f60 msgr2=0x55aefbb65110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:09.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.713+0000 7f3631904700 1 --2- 172.21.15.7:0/3854309535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aefbb66f60 0x55aefbb65110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.713+0000 7f3631904700 1 -- 172.21.15.7:0/3854309535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aefbb6aea0 con 0x55aefbb60f20 2024-06-10T01:11:09.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.713+0000 7f3631904700 1 --2- 172.21.15.7:0/3854309535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aefbb60f20 0x55aefbb609e0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f362800b580 tx=0x7f3628022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698bfefb25f3da10 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:09.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.714+0000 7f36177fe700 1 -- 172.21.15.7:0/3854309535 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362800bc90 con 0x55aefbb60f20 2024-06-10T01:11:09.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.714+0000 7f36177fe700 1 -- 172.21.15.7:0/3854309535 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f362800be30 con 0x55aefbb60f20 2024-06-10T01:11:09.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.714+0000 7f36177fe700 1 -- 172.21.15.7:0/3854309535 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36280315a0 con 0x55aefbb60f20 2024-06-10T01:11:09.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.714+0000 7f363e3ebb80 1 -- 172.21.15.7:0/3854309535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aefbb60f20 msgr2=0x55aefbb609e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:09.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.714+0000 7f363e3ebb80 1 --2- 172.21.15.7:0/3854309535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aefbb60f20 0x55aefbb609e0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f362800b580 tx=0x7f3628022000 comp rx=0 tx=0).stop 2024-06-10T01:11:09.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.714+0000 7f363e3ebb80 1 -- 172.21.15.7:0/3854309535 shutdown_connections 2024-06-10T01:11:09.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.714+0000 7f363e3ebb80 1 --2- 172.21.15.7:0/3854309535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aefbb66f60 0x55aefbb65110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.714+0000 7f363e3ebb80 1 --2- 172.21.15.7:0/3854309535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aefbb60f20 0x55aefbb609e0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.714+0000 7f363e3ebb80 1 --2- 172.21.15.7:0/3854309535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aefbb66640 0x55aefbb55510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.714+0000 7f363e3ebb80 1 -- 172.21.15.7:0/3854309535 >> 172.21.15.7:0/3854309535 conn(0x55aefba8bcc0 msgr2=0x55aefbb54e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:09.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.715+0000 7f363e3ebb80 1 -- 172.21.15.7:0/3854309535 shutdown_connections 2024-06-10T01:11:09.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.715+0000 7f363e3ebb80 1 -- 172.21.15.7:0/3854309535 wait complete. 2024-06-10T01:11:09.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.715+0000 7f363e3ebb80 1 -- start start 2024-06-10T01:11:09.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.715+0000 7f363e3ebb80 1 Processor -- start 2024-06-10T01:11:09.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.715+0000 7f363e3ebb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aefbb60f20 0x55aefbb7fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:09.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.715+0000 7f363e3ebb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aefbb66640 0x55aefbb7cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:09.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.715+0000 7f363e3ebb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aefbb66f60 0x55aefbb7d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:09.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.715+0000 7f363e3ebb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1aebf0 con 0x55aefbb66f60 2024-06-10T01:11:09.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.715+0000 7f363e3ebb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1b0060 con 0x55aefbb60f20 2024-06-10T01:11:09.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.715+0000 7f363e3ebb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1af2c0 con 0x55aefbb66640 2024-06-10T01:11:09.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.716+0000 7f3632105700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aefbb66f60 0x55aefbb7d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:09.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.716+0000 7f363e3ebb80 10 client.0 _ll_register_callbacks cb 0x55aefba4c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:11:09.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.716+0000 7f3632105700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aefbb66f60 0x55aefbb7d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48726/0 (socket says 172.21.15.7:48726) 2024-06-10T01:11:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.716+0000 7f3632105700 1 -- 172.21.15.7:0/2122188704 learned_addr learned my addr 172.21.15.7:0/2122188704 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.716+0000 7f3631904700 1 --2- 172.21.15.7:0/2122188704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aefbb60f20 0x55aefbb7fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.716+0000 7f3631103700 1 --2- 172.21.15.7:0/2122188704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aefbb66640 0x55aefbb7cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.716+0000 7f3632105700 1 -- 172.21.15.7:0/2122188704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aefbb66640 msgr2=0x55aefbb7cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.716+0000 7f3632105700 1 --2- 172.21.15.7:0/2122188704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aefbb66640 0x55aefbb7cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.716+0000 7f3632105700 1 -- 172.21.15.7:0/2122188704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aefbb60f20 msgr2=0x55aefbb7fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.716+0000 7f3632105700 1 --2- 172.21.15.7:0/2122188704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aefbb60f20 0x55aefbb7fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.716+0000 7f3632105700 1 -- 172.21.15.7:0/2122188704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aefbb7da70 con 0x55aefbb66f60 2024-06-10T01:11:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.716+0000 7f3631103700 1 --2- 172.21.15.7:0/2122188704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aefbb66640 0x55aefbb7cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:11:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.716+0000 7f3631904700 1 --2- 172.21.15.7:0/2122188704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aefbb60f20 0x55aefbb7fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:11:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.716+0000 7f3632105700 1 --2- 172.21.15.7:0/2122188704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aefbb66f60 0x55aefbb7d430 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f3624010a80 tx=0x7f3624010ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68203bed97fd8e34 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.716+0000 7f36157fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:11:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.716+0000 7f36157fa700 1 -- 172.21.15.7:0/2122188704 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362400b980 con 0x55aefbb66f60 2024-06-10T01:11:09.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.716+0000 7f363e3ebb80 1 -- 172.21.15.7:0/2122188704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f362800be30 con 0x55aefbb66f60 2024-06-10T01:11:09.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.716+0000 7f36157fa700 1 -- 172.21.15.7:0/2122188704 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f362400bb20 con 0x55aefbb66f60 2024-06-10T01:11:09.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.717+0000 7f36157fa700 1 -- 172.21.15.7:0/2122188704 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3624024050 con 0x55aefbb66f60 2024-06-10T01:11:09.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.717+0000 7f363e3ebb80 10 client.16785 fetch_fsmap learned FSMap version 32 2024-06-10T01:11:09.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.717+0000 7f363e3ebb80 1 -- 172.21.15.7:0/2122188704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55aefbbed5a0 con 0x55aefbb66f60 2024-06-10T01:11:09.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.717+0000 7f36157fa700 1 -- 172.21.15.7:0/2122188704 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f362800be30 con 0x55aefbb66f60 2024-06-10T01:11:09.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.717+0000 7f36157fa700 1 -- 172.21.15.7:0/2122188704 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f36240193f0 con 0x55aefbb66f60 2024-06-10T01:11:09.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.717+0000 7f363e3ebb80 10 client.16785 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:11:09.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.717+0000 7f363e3ebb80 10 client.16785 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:11:09.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.717+0000 7f363e3ebb80 20 client.16785 populate_metadata read hostname 'smithi007' 2024-06-10T01:11:09.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.717+0000 7f363e3ebb80 1 --2- 172.21.15.7:0/2122188704 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55aefbbedce0 0x55aefbbf0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:09.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.717+0000 7f363e3ebb80 4 client.16785 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:11:09.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.717+0000 7f363e3ebb80 1 -- 172.21.15.7:0/2122188704 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aefbaba290 con 0x55aefbbedce0 2024-06-10T01:11:09.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.718+0000 7f3631904700 1 --2- 172.21.15.7:0/2122188704 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55aefbbedce0 0x55aefbbf0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:09.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.718+0000 7f3631904700 1 --2- 172.21.15.7:0/2122188704 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55aefbbedce0 0x55aefbbf0100 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=37905bc96eb3610a server_cookie=8c466320111462ae in_seq=0 out_seq=0 2024-06-10T01:11:09.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.718+0000 7f36157fa700 10 client.16785 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:11:09.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.718+0000 7f36157fa700 1 -- 172.21.15.7:0/2122188704 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aefbaba290 con 0x55aefbbedce0 2024-06-10T01:11:09.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.718+0000 7f36157fa700 10 client.16785 handle_command_reply: tid=0 2024-06-10T01:11:09.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.718+0000 7f363e3ebb80 1 client.16785 shutdown 2024-06-10T01:11:09.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.718+0000 7f363e3ebb80 1 -- 172.21.15.7:0/2122188704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aefbb66f60 msgr2=0x55aefbb7d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:09.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.718+0000 7f363e3ebb80 1 --2- 172.21.15.7:0/2122188704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aefbb66f60 0x55aefbb7d430 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f3624010a80 tx=0x7f3624010ab0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.718+0000 7f363e3ebb80 1 -- 172.21.15.7:0/2122188704 shutdown_connections 2024-06-10T01:11:09.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.718+0000 7f363e3ebb80 1 --2- 172.21.15.7:0/2122188704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aefbb66f60 0x55aefbb7d430 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.718+0000 7f363e3ebb80 1 --2- 172.21.15.7:0/2122188704 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55aefbbedce0 0x55aefbbf0100 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.719+0000 7f363e3ebb80 1 --2- 172.21.15.7:0/2122188704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aefbb60f20 0x55aefbb7fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.719+0000 7f363e3ebb80 1 --2- 172.21.15.7:0/2122188704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aefbb66640 0x55aefbb7cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.719+0000 7f36157fa700 0 client.16785 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:11:09.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.719+0000 7f363e3ebb80 1 -- 172.21.15.7:0/2122188704 >> 172.21.15.7:0/2122188704 conn(0x55aefba8bcc0 msgr2=0x55aefbb68760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:09.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.719+0000 7f363e3ebb80 1 -- 172.21.15.7:0/2122188704 shutdown_connections 2024-06-10T01:11:09.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.719+0000 7f363e3ebb80 1 -- 172.21.15.7:0/2122188704 wait complete. 2024-06-10T01:11:09.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.719+0000 7f363e3ebb80 20 client.16785 trim_cache size 0 max 16384 2024-06-10T01:11:09.719 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:11:09.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.719+0000 7f3633b68700 1 -- 172.21.15.7:0/3229431353 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f36180a2dc0 msgr2=0x7f36180a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:09.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.719+0000 7f3633b68700 1 --2- 172.21.15.7:0/3229431353 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f36180a2dc0 0x7f36180a5280 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f36280059d0 tx=0x7f3628005940 comp rx=0 tx=0).stop 2024-06-10T01:11:09.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.719+0000 7f3633b68700 1 -- 172.21.15.7:0/3229431353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f362c1030f0 msgr2=0x7f362c19ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:09.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.719+0000 7f3633b68700 1 --2- 172.21.15.7:0/3229431353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f362c1030f0 0x7f362c19ee80 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f361c00b530 tx=0x7f361c00b8f0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.719+0000 7f3633b68700 1 -- 172.21.15.7:0/3229431353 shutdown_connections 2024-06-10T01:11:09.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.719+0000 7f3633b68700 1 --2- 172.21.15.7:0/3229431353 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f36180a2dc0 0x7f36180a5280 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.719+0000 7f3633b68700 1 --2- 172.21.15.7:0/3229431353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f362c107840 0x7f362c198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.719+0000 7f3633b68700 1 --2- 172.21.15.7:0/3229431353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f362c102740 0x7f362c19e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.719+0000 7f3633b68700 1 --2- 172.21.15.7:0/3229431353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f362c1030f0 0x7f362c19ee80 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:09.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.719+0000 7f3633b68700 1 -- 172.21.15.7:0/3229431353 >> 172.21.15.7:0/3229431353 conn(0x7f362c0795f0 msgr2=0x7f362c100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:09.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.720+0000 7f3633b68700 1 -- 172.21.15.7:0/3229431353 shutdown_connections 2024-06-10T01:11:09.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:09.720+0000 7f3633b68700 1 -- 172.21.15.7:0/3229431353 wait complete. 2024-06-10T01:11:09.729 INFO:tasks.workunit.client.0.smithi007.stdout: 1 345759 8.39 MB/sec execute 110 sec latency 9676.453 ms 2024-06-10T01:11:09.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:09 smithi078 ceph-mon[36683]: pgmap v586: 129 pgs: 129 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 2.2 MiB/s wr, 242 op/s 2024-06-10T01:11:10.505 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:11:10.729 INFO:tasks.workunit.client.0.smithi007.stdout: 1 345759 8.32 MB/sec execute 111 sec latency 10676.498 ms 2024-06-10T01:11:10.758 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:11:11.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:11 smithi175 ceph-mon[36636]: pgmap v587: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 2.5 MiB/s wr, 238 op/s 2024-06-10T01:11:11.729 INFO:tasks.workunit.client.0.smithi007.stdout: 1 345759 8.24 MB/sec execute 112 sec latency 11676.589 ms 2024-06-10T01:11:11.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:11 smithi007 ceph-mon[32398]: pgmap v587: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 2.5 MiB/s wr, 238 op/s 2024-06-10T01:11:11.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:11 smithi078 ceph-mon[36683]: pgmap v587: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 2.5 MiB/s wr, 238 op/s 2024-06-10T01:11:12.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.633+0000 7f0bf10b8700 1 -- 172.21.15.7:0/2665521062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bec101620 msgr2=0x7f0bec101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:12.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.633+0000 7f0bf10b8700 1 --2- 172.21.15.7:0/2665521062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bec101620 0x7f0bec101a00 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc0099c0 tx=0x7f0bdc009cd0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.633+0000 7f0bf10b8700 1 -- 172.21.15.7:0/2665521062 shutdown_connections 2024-06-10T01:11:12.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.633+0000 7f0bf10b8700 1 --2- 172.21.15.7:0/2665521062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0bec1066b0 0x7f0bec10e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.633+0000 7f0bf10b8700 1 --2- 172.21.15.7:0/2665521062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bec101620 0x7f0bec101a00 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.633+0000 7f0bf10b8700 1 --2- 172.21.15.7:0/2665521062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bec101fd0 0x7f0bec105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.633+0000 7f0bf10b8700 1 -- 172.21.15.7:0/2665521062 >> 172.21.15.7:0/2665521062 conn(0x7f0bec078340 msgr2=0x7f0bec078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:12.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.633+0000 7f0bf10b8700 1 -- 172.21.15.7:0/2665521062 shutdown_connections 2024-06-10T01:11:12.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.633+0000 7f0bf10b8700 1 -- 172.21.15.7:0/2665521062 wait complete. 2024-06-10T01:11:12.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.634+0000 7f0bf10b8700 1 Processor -- start 2024-06-10T01:11:12.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.634+0000 7f0bf10b8700 1 -- start start 2024-06-10T01:11:12.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.634+0000 7f0bf10b8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bec101620 0x7f0bec103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:12.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.634+0000 7f0bf10b8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0bec101fd0 0x7f0bec1035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:12.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.634+0000 7f0bf10b8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bec1066b0 0x7f0bec103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:12.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.635+0000 7f0bf10b8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0bec111690 con 0x7f0bec1066b0 2024-06-10T01:11:12.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.635+0000 7f0bf10b8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0bec111510 con 0x7f0bec101fd0 2024-06-10T01:11:12.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.635+0000 7f0bf10b8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0bec111810 con 0x7f0bec101620 2024-06-10T01:11:12.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.635+0000 7f0beb59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bec1066b0 0x7f0bec103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:12.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.635+0000 7f0bead9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bec101620 0x7f0bec103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:12.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.635+0000 7f0beb59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bec1066b0 0x7f0bec103af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47574/0 (socket says 172.21.15.7:47574) 2024-06-10T01:11:12.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.635+0000 7f0beb59e700 1 -- 172.21.15.7:0/3346850257 learned_addr learned my addr 172.21.15.7:0/3346850257 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:12.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.635+0000 7f0bead9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bec101620 0x7f0bec103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34080/0 (socket says 172.21.15.7:34080) 2024-06-10T01:11:12.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.635+0000 7f0bea59c700 1 --2- 172.21.15.7:0/3346850257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0bec101fd0 0x7f0bec1035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:12.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.635+0000 7f0beb59e700 1 -- 172.21.15.7:0/3346850257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bec101620 msgr2=0x7f0bec103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:12.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.635+0000 7f0beb59e700 1 --2- 172.21.15.7:0/3346850257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bec101620 0x7f0bec103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.635+0000 7f0beb59e700 1 -- 172.21.15.7:0/3346850257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0bec101fd0 msgr2=0x7f0bec1035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:12.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.635+0000 7f0beb59e700 1 --2- 172.21.15.7:0/3346850257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0bec101fd0 0x7f0bec1035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.635+0000 7f0beb59e700 1 -- 172.21.15.7:0/3346850257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bec1a7090 con 0x7f0bec1066b0 2024-06-10T01:11:12.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.635+0000 7f0beb59e700 1 --2- 172.21.15.7:0/3346850257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bec1066b0 0x7f0bec103af0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f0be4007d70 tx=0x7f0be400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:12.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.635+0000 7f0bd3fff700 1 -- 172.21.15.7:0/3346850257 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be4016020 con 0x7f0bec1066b0 2024-06-10T01:11:12.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.636+0000 7f0bd3fff700 1 -- 172.21.15.7:0/3346850257 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0be400ac90 con 0x7f0bec1066b0 2024-06-10T01:11:12.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.636+0000 7f0bf10b8700 1 -- 172.21.15.7:0/3346850257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bec1a7380 con 0x7f0bec1066b0 2024-06-10T01:11:12.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.636+0000 7f0bd3fff700 1 -- 172.21.15.7:0/3346850257 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be40115c0 con 0x7f0bec1066b0 2024-06-10T01:11:12.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.636+0000 7f0bf10b8700 1 -- 172.21.15.7:0/3346850257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bec1a7870 con 0x7f0bec1066b0 2024-06-10T01:11:12.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.638+0000 7f0bd3fff700 1 -- 172.21.15.7:0/3346850257 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0be4019020 con 0x7f0bec1066b0 2024-06-10T01:11:12.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.639+0000 7f0bd3fff700 1 --2- 172.21.15.7:0/3346850257 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0bd40a2c20 0x7f0bd40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:12.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.639+0000 7f0bd3fff700 1 -- 172.21.15.7:0/3346850257 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f0be40cf340 con 0x7f0bec1066b0 2024-06-10T01:11:12.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.639+0000 7f0bead9d700 1 --2- 172.21.15.7:0/3346850257 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0bd40a2c20 0x7f0bd40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:12.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.639+0000 7f0bfb93bb80 1 Processor -- start 2024-06-10T01:11:12.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.639+0000 7f0bfb93bb80 1 -- start start 2024-06-10T01:11:12.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.639+0000 7f0bead9d700 1 --2- 172.21.15.7:0/3346850257 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0bd40a2c20 0x7f0bd40a50e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc009fd0 tx=0x7f0bdc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:12.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.640+0000 7f0bfb93bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564a18781f60 0x564a18782340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:12.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.640+0000 7f0bfb93bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a18782880 0x564a1877c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:12.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.640+0000 7f0bfb93bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1877c7a0 0x564a1877cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:12.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.640+0000 7f0bfb93bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564a1866ef60 con 0x564a1877c7a0 2024-06-10T01:11:12.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.640+0000 7f0bfb93bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564a186cd850 con 0x564a18782880 2024-06-10T01:11:12.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.640+0000 7f0bfb93bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564a18635460 con 0x564a18781f60 2024-06-10T01:11:12.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.640+0000 7f0beb59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1877c7a0 0x564a1877cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:12.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.640+0000 7f0bead9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564a18781f60 0x564a18782340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:12.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.640+0000 7f0bead9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564a18781f60 0x564a18782340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34092/0 (socket says 172.21.15.7:34092) 2024-06-10T01:11:12.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.640+0000 7f0beb59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1877c7a0 0x564a1877cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47580/0 (socket says 172.21.15.7:47580) 2024-06-10T01:11:12.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.640+0000 7f0bead9d700 1 -- 172.21.15.7:0/3657454528 learned_addr learned my addr 172.21.15.7:0/3657454528 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:12.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.640+0000 7f0bea59c700 1 --2- 172.21.15.7:0/3657454528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a18782880 0x564a1877c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:12.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.640+0000 7f0bead9d700 1 -- 172.21.15.7:0/3657454528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a18782880 msgr2=0x564a1877c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:12.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.640+0000 7f0bead9d700 1 --2- 172.21.15.7:0/3657454528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a18782880 0x564a1877c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.640+0000 7f0bead9d700 1 -- 172.21.15.7:0/3657454528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1877c7a0 msgr2=0x564a1877cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:12.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.640+0000 7f0bead9d700 1 --2- 172.21.15.7:0/3657454528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1877c7a0 0x564a1877cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.640+0000 7f0bead9d700 1 -- 172.21.15.7:0/3657454528 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a1861c8d0 con 0x564a18781f60 2024-06-10T01:11:12.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.640+0000 7f0beb59e700 1 --2- 172.21.15.7:0/3657454528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1877c7a0 0x564a1877cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:11:12.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.640+0000 7f0bead9d700 1 --2- 172.21.15.7:0/3657454528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564a18781f60 0x564a18782340 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc015000 tx=0x7f0bdc020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=356c449b5868218c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:12.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.641+0000 7f0bd17fa700 1 -- 172.21.15.7:0/3657454528 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc00bea0 con 0x564a18781f60 2024-06-10T01:11:12.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.641+0000 7f0bd17fa700 1 -- 172.21.15.7:0/3657454528 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bdc00fd50 con 0x564a18781f60 2024-06-10T01:11:12.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.641+0000 7f0bd17fa700 1 -- 172.21.15.7:0/3657454528 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc02f400 con 0x564a18781f60 2024-06-10T01:11:12.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.641+0000 7f0bfb93bb80 1 -- 172.21.15.7:0/3657454528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564a18781f60 msgr2=0x564a18782340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:12.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.641+0000 7f0bfb93bb80 1 --2- 172.21.15.7:0/3657454528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564a18781f60 0x564a18782340 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc015000 tx=0x7f0bdc020000 comp rx=0 tx=0).stop 2024-06-10T01:11:12.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.641+0000 7f0bfb93bb80 1 -- 172.21.15.7:0/3657454528 shutdown_connections 2024-06-10T01:11:12.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.641+0000 7f0bfb93bb80 1 --2- 172.21.15.7:0/3657454528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1877c7a0 0x564a1877cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.641+0000 7f0bfb93bb80 1 --2- 172.21.15.7:0/3657454528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564a18781f60 0x564a18782340 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.641+0000 7f0bfb93bb80 1 --2- 172.21.15.7:0/3657454528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a18782880 0x564a1877c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.641+0000 7f0bfb93bb80 1 -- 172.21.15.7:0/3657454528 >> 172.21.15.7:0/3657454528 conn(0x564a186a9080 msgr2=0x564a18777220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:12.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.642+0000 7f0bfb93bb80 1 -- 172.21.15.7:0/3657454528 shutdown_connections 2024-06-10T01:11:12.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.642+0000 7f0bfb93bb80 1 -- 172.21.15.7:0/3657454528 wait complete. 2024-06-10T01:11:12.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.642+0000 7f0bfb93bb80 1 -- start start 2024-06-10T01:11:12.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.642+0000 7f0bfb93bb80 1 Processor -- start 2024-06-10T01:11:12.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.642+0000 7f0bfb93bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1877c7a0 0x564a187885c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:12.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.642+0000 7f0bfb93bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564a18781f60 0x564a187858c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:12.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.642+0000 7f0bfb93bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a18782880 0x564a18785e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.642+0000 7f0bfb93bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564a1866dfc0 con 0x564a1877c7a0 2024-06-10T01:11:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.642+0000 7f0bfb93bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564a186c9810 con 0x564a18782880 2024-06-10T01:11:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.642+0000 7f0bfb93bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564a187891a0 con 0x564a18781f60 2024-06-10T01:11:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.642+0000 7f0bfb93bb80 10 client.0 _ll_register_callbacks cb 0x564a186635b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:11:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.642+0000 7f0bead9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1877c7a0 0x564a187885c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.642+0000 7f0bead9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1877c7a0 0x564a187885c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47586/0 (socket says 172.21.15.7:47586) 2024-06-10T01:11:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.643+0000 7f0bead9d700 1 -- 172.21.15.7:0/4243125559 learned_addr learned my addr 172.21.15.7:0/4243125559 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.643+0000 7f0bea59c700 1 --2- 172.21.15.7:0/4243125559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564a18781f60 0x564a187858c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.643+0000 7f0beb59e700 1 --2- 172.21.15.7:0/4243125559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a18782880 0x564a18785e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.643+0000 7f0bead9d700 1 -- 172.21.15.7:0/4243125559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564a18781f60 msgr2=0x564a187858c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.643+0000 7f0bead9d700 1 --2- 172.21.15.7:0/4243125559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564a18781f60 0x564a187858c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.643+0000 7f0bead9d700 1 -- 172.21.15.7:0/4243125559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a18782880 msgr2=0x564a18785e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:12.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.643+0000 7f0bead9d700 1 --2- 172.21.15.7:0/4243125559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a18782880 0x564a18785e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.643+0000 7f0bead9d700 1 -- 172.21.15.7:0/4243125559 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a18693750 con 0x564a1877c7a0 2024-06-10T01:11:12.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.643+0000 7f0bead9d700 1 --2- 172.21.15.7:0/4243125559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1877c7a0 0x564a187885c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc031040 tx=0x7f0bdc024de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71869b9cb2b34b5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:12.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.643+0000 7f0bceff5700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:11:12.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.643+0000 7f0bceff5700 1 -- 172.21.15.7:0/4243125559 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc03c060 con 0x564a1877c7a0 2024-06-10T01:11:12.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.643+0000 7f0bceff5700 1 -- 172.21.15.7:0/4243125559 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f0bdc00fa80 con 0x564a1877c7a0 2024-06-10T01:11:12.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.643+0000 7f0bfb93bb80 1 -- 172.21.15.7:0/4243125559 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bdc00fd50 con 0x564a1877c7a0 2024-06-10T01:11:12.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.644+0000 7f0bceff5700 1 -- 172.21.15.7:0/4243125559 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0bdc00bea0 con 0x564a1877c7a0 2024-06-10T01:11:12.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.644+0000 7f0bfb93bb80 10 client.16797 fetch_fsmap learned FSMap version 32 2024-06-10T01:11:12.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.644+0000 7f0bfb93bb80 1 -- 172.21.15.7:0/4243125559 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564a18793900 con 0x564a1877c7a0 2024-06-10T01:11:12.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.644+0000 7f0bceff5700 1 -- 172.21.15.7:0/4243125559 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bdc00fd50 con 0x564a1877c7a0 2024-06-10T01:11:12.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.644+0000 7f0bceff5700 1 -- 172.21.15.7:0/4243125559 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f0bdc017480 con 0x564a1877c7a0 2024-06-10T01:11:12.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.644+0000 7f0bfb93bb80 10 client.16797 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:11:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.644+0000 7f0bfb93bb80 10 client.16797 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:11:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.644+0000 7f0bfb93bb80 20 client.16797 populate_metadata read hostname 'smithi007' 2024-06-10T01:11:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.644+0000 7f0bfb93bb80 1 --2- 172.21.15.7:0/4243125559 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564a18795dd0 0x564a187981c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.644+0000 7f0bfb93bb80 4 client.16797 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:11:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.644+0000 7f0bfb93bb80 1 -- 172.21.15.7:0/4243125559 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a186dad50 con 0x564a18795dd0 2024-06-10T01:11:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.644+0000 7f0bea59c700 1 --2- 172.21.15.7:0/4243125559 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564a18795dd0 0x564a187981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.645+0000 7f0bea59c700 1 --2- 172.21.15.7:0/4243125559 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564a18795dd0 0x564a187981c0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79e39f1ab6389414 server_cookie=bf22070da975054b in_seq=0 out_seq=0 2024-06-10T01:11:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.645+0000 7f0bceff5700 10 client.16797 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:12.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.646+0000 7f0bceff5700 1 -- 172.21.15.7:0/4243125559 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564a186dad50 con 0x564a18795dd0 2024-06-10T01:11:12.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.646+0000 7f0bceff5700 10 client.16797 handle_command_reply: tid=0 2024-06-10T01:11:12.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.646+0000 7f0bfb93bb80 1 client.16797 shutdown 2024-06-10T01:11:12.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.646+0000 7f0bfb93bb80 1 -- 172.21.15.7:0/4243125559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1877c7a0 msgr2=0x564a187885c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.646+0000 7f0bfb93bb80 1 --2- 172.21.15.7:0/4243125559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1877c7a0 0x564a187885c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc031040 tx=0x7f0bdc024de0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.646+0000 7f0bfb93bb80 1 -- 172.21.15.7:0/4243125559 shutdown_connections 2024-06-10T01:11:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.646+0000 7f0bfb93bb80 1 --2- 172.21.15.7:0/4243125559 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564a18795dd0 0x564a187981c0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.646+0000 7f0bfb93bb80 1 --2- 172.21.15.7:0/4243125559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a18782880 0x564a18785e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.646+0000 7f0bfb93bb80 1 --2- 172.21.15.7:0/4243125559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1877c7a0 0x564a187885c0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.646+0000 7f0bfb93bb80 1 --2- 172.21.15.7:0/4243125559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564a18781f60 0x564a187858c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.646+0000 7f0bceff5700 0 client.16797 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.646+0000 7f0bfb93bb80 1 -- 172.21.15.7:0/4243125559 >> 172.21.15.7:0/4243125559 conn(0x564a186a9080 msgr2=0x564a18776700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.646+0000 7f0bfb93bb80 1 -- 172.21.15.7:0/4243125559 shutdown_connections 2024-06-10T01:11:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.646+0000 7f0bfb93bb80 1 -- 172.21.15.7:0/4243125559 wait complete. 2024-06-10T01:11:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.646+0000 7f0bfb93bb80 20 client.16797 trim_cache size 0 max 16384 2024-06-10T01:11:12.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.661+0000 7f0bfb93bb80 1 Processor -- start 2024-06-10T01:11:12.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.661+0000 7f0bfb93bb80 1 -- start start 2024-06-10T01:11:12.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.662+0000 7f0bfb93bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564a1877c7a0 0x564a18787980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:12.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.662+0000 7f0bfb93bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a18781f60 0x564a18776e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:12.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.662+0000 7f0bfb93bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a18782880 0x564a187773c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:12.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.662+0000 7f0bfb93bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0bec1b2c40 con 0x564a18781f60 2024-06-10T01:11:12.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.662+0000 7f0bfb93bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0bec1b04d0 con 0x564a18782880 2024-06-10T01:11:12.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.662+0000 7f0bfb93bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0bec1ae320 con 0x564a1877c7a0 2024-06-10T01:11:12.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.662+0000 7f0bea59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a18781f60 0x564a18776e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:12.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.662+0000 7f0bea59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a18781f60 0x564a18776e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47598/0 (socket says 172.21.15.7:47598) 2024-06-10T01:11:12.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.662+0000 7f0bea59c700 1 -- 172.21.15.7:0/1836870559 learned_addr learned my addr 172.21.15.7:0/1836870559 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:12.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.662+0000 7f0bead9d700 1 --2- 172.21.15.7:0/1836870559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564a1877c7a0 0x564a18787980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:12.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.662+0000 7f0bea59c700 1 -- 172.21.15.7:0/1836870559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564a1877c7a0 msgr2=0x564a18787980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.662+0000 7f0bea59c700 1 --2- 172.21.15.7:0/1836870559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564a1877c7a0 0x564a18787980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.662+0000 7f0bea59c700 1 -- 172.21.15.7:0/1836870559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a18782880 msgr2=0x564a187773c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.662+0000 7f0beb59e700 1 --2- 172.21.15.7:0/1836870559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a18782880 0x564a187773c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.662+0000 7f0bea59c700 1 --2- 172.21.15.7:0/1836870559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a18782880 0x564a187773c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.662+0000 7f0bea59c700 1 -- 172.21.15.7:0/1836870559 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a18783e50 con 0x564a18781f60 2024-06-10T01:11:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.662+0000 7f0beb59e700 1 --2- 172.21.15.7:0/1836870559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a18782880 0x564a187773c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:11:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.662+0000 7f0bea59c700 1 --2- 172.21.15.7:0/1836870559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a18781f60 0x564a18776e80 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0bec107b80 tx=0x7f0be000a380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1927265e4e258e2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.662+0000 7f0bd0ff9700 1 -- 172.21.15.7:0/1836870559 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be0004250 con 0x564a18781f60 2024-06-10T01:11:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.663+0000 7f0bd0ff9700 1 -- 172.21.15.7:0/1836870559 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0be00043f0 con 0x564a18781f60 2024-06-10T01:11:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.663+0000 7f0bd0ff9700 1 -- 172.21.15.7:0/1836870559 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be001d9b0 con 0x564a18781f60 2024-06-10T01:11:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.663+0000 7f0bfb93bb80 1 -- 172.21.15.7:0/1836870559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a18781f60 msgr2=0x564a18776e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.663+0000 7f0bfb93bb80 1 --2- 172.21.15.7:0/1836870559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a18781f60 0x564a18776e80 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0bec107b80 tx=0x7f0be000a380 comp rx=0 tx=0).stop 2024-06-10T01:11:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.663+0000 7f0bfb93bb80 1 -- 172.21.15.7:0/1836870559 shutdown_connections 2024-06-10T01:11:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.663+0000 7f0bfb93bb80 1 --2- 172.21.15.7:0/1836870559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a18782880 0x564a187773c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.663+0000 7f0bfb93bb80 1 --2- 172.21.15.7:0/1836870559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564a1877c7a0 0x564a18787980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.663+0000 7f0bfb93bb80 1 --2- 172.21.15.7:0/1836870559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a18781f60 0x564a18776e80 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.663+0000 7f0bfb93bb80 1 -- 172.21.15.7:0/1836870559 >> 172.21.15.7:0/1836870559 conn(0x564a186a9080 msgr2=0x564a187832d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.663+0000 7f0bfb93bb80 1 -- 172.21.15.7:0/1836870559 shutdown_connections 2024-06-10T01:11:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.663+0000 7f0bfb93bb80 1 -- 172.21.15.7:0/1836870559 wait complete. 2024-06-10T01:11:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.664+0000 7f0bfb93bb80 1 -- start start 2024-06-10T01:11:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.664+0000 7f0bfb93bb80 1 Processor -- start 2024-06-10T01:11:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.664+0000 7f0bfb93bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a1877c7a0 0x564a1879b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.664+0000 7f0bfb93bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a18781f60 0x564a18798b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.664+0000 7f0bfb93bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564a18782880 0x564a187990c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.664+0000 7f0bfb93bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0bec1ae990 con 0x564a18781f60 2024-06-10T01:11:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.664+0000 7f0bfb93bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0bec1afe00 con 0x564a1877c7a0 2024-06-10T01:11:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.664+0000 7f0bfb93bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0bec1af060 con 0x564a18782880 2024-06-10T01:11:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.664+0000 7f0bfb93bb80 10 client.0 _ll_register_callbacks cb 0x564a186635b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:11:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.664+0000 7f0bea59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a18781f60 0x564a18798b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.664+0000 7f0beb59e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564a18782880 0x564a187990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.664+0000 7f0beb59e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564a18782880 0x564a187990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34110/0 (socket says 172.21.15.7:34110) 2024-06-10T01:11:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.664+0000 7f0beb59e700 1 -- 172.21.15.7:0/493246844 learned_addr learned my addr 172.21.15.7:0/493246844 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.664+0000 7f0bea59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a18781f60 0x564a18798b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47610/0 (socket says 172.21.15.7:47610) 2024-06-10T01:11:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.664+0000 7f0bead9d700 1 --2- 172.21.15.7:0/493246844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a1877c7a0 0x564a1879b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.665+0000 7f0beb59e700 1 -- 172.21.15.7:0/493246844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a1877c7a0 msgr2=0x564a1879b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.665+0000 7f0beb59e700 1 --2- 172.21.15.7:0/493246844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a1877c7a0 0x564a1879b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.665+0000 7f0beb59e700 1 -- 172.21.15.7:0/493246844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a18781f60 msgr2=0x564a18798b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.665+0000 7f0beb59e700 1 --2- 172.21.15.7:0/493246844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a18781f60 0x564a18798b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.665+0000 7f0beb59e700 1 -- 172.21.15.7:0/493246844 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a18799700 con 0x564a18782880 2024-06-10T01:11:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.665+0000 7f0bea59c700 1 --2- 172.21.15.7:0/493246844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a18781f60 0x564a18798b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:11:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.665+0000 7f0beb59e700 1 --2- 172.21.15.7:0/493246844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564a18782880 0x564a187990c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0be4098190 tx=0x7f0be400af90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccc904d68f988c0d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.665+0000 7f0bceff5700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:11:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.665+0000 7f0bceff5700 1 -- 172.21.15.7:0/493246844 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be409b7c0 con 0x564a18782880 2024-06-10T01:11:12.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.665+0000 7f0bceff5700 1 -- 172.21.15.7:0/493246844 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f0be40860b0 con 0x564a18782880 2024-06-10T01:11:12.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.665+0000 7f0bfb93bb80 1 -- 172.21.15.7:0/493246844 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0be00043f0 con 0x564a18782880 2024-06-10T01:11:12.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.665+0000 7f0bceff5700 1 -- 172.21.15.7:0/493246844 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0be40862b0 con 0x564a18782880 2024-06-10T01:11:12.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.665+0000 7f0bfb93bb80 10 client.26299 fetch_fsmap learned FSMap version 32 2024-06-10T01:11:12.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.665+0000 7f0bfb93bb80 1 -- 172.21.15.7:0/493246844 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564a18800550 con 0x564a18782880 2024-06-10T01:11:12.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.665+0000 7f0bceff5700 1 -- 172.21.15.7:0/493246844 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0be00043f0 con 0x564a18782880 2024-06-10T01:11:12.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.666+0000 7f0bceff5700 1 -- 172.21.15.7:0/493246844 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f0be40afce0 con 0x564a18782880 2024-06-10T01:11:12.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.666+0000 7f0bfb93bb80 10 client.26299 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:11:12.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.666+0000 7f0bfb93bb80 10 client.26299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:11:12.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.666+0000 7f0bfb93bb80 20 client.26299 populate_metadata read hostname 'smithi007' 2024-06-10T01:11:12.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.666+0000 7f0bfb93bb80 1 --2- 172.21.15.7:0/493246844 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564a18800c30 0x564a18803020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:12.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.666+0000 7f0bfb93bb80 4 client.26299 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:11:12.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.666+0000 7f0bfb93bb80 1 -- 172.21.15.7:0/493246844 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564a186d52d0 con 0x564a18800c30 2024-06-10T01:11:12.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.666+0000 7f0bead9d700 1 --2- 172.21.15.7:0/493246844 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564a18800c30 0x564a18803020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:12.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.667+0000 7f0bead9d700 1 --2- 172.21.15.7:0/493246844 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564a18800c30 0x564a18803020 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51252461f65c0429 server_cookie=3ee1e04ec31407a3 in_seq=0 out_seq=0 2024-06-10T01:11:12.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.667+0000 7f0bceff5700 10 client.26299 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:12.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.667+0000 7f0bceff5700 1 -- 172.21.15.7:0/493246844 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564a186d52d0 con 0x564a18800c30 2024-06-10T01:11:12.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.667+0000 7f0bceff5700 10 client.26299 handle_command_reply: tid=0 2024-06-10T01:11:12.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.667+0000 7f0bfb93bb80 1 client.26299 shutdown 2024-06-10T01:11:12.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.667+0000 7f0bfb93bb80 1 -- 172.21.15.7:0/493246844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564a18782880 msgr2=0x564a187990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:12.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.667+0000 7f0bfb93bb80 1 --2- 172.21.15.7:0/493246844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564a18782880 0x564a187990c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0be4098190 tx=0x7f0be400af90 comp rx=0 tx=0).stop 2024-06-10T01:11:12.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.667+0000 7f0bfb93bb80 1 -- 172.21.15.7:0/493246844 shutdown_connections 2024-06-10T01:11:12.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.667+0000 7f0bfb93bb80 1 --2- 172.21.15.7:0/493246844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564a18782880 0x564a187990c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.667+0000 7f0bfb93bb80 1 --2- 172.21.15.7:0/493246844 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564a18800c30 0x564a18803020 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.667+0000 7f0bfb93bb80 1 --2- 172.21.15.7:0/493246844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a1877c7a0 0x564a1879b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.667+0000 7f0bceff5700 0 client.26299 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:12.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.667+0000 7f0bfb93bb80 1 --2- 172.21.15.7:0/493246844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a18781f60 0x564a18798b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.667+0000 7f0bfb93bb80 1 -- 172.21.15.7:0/493246844 >> 172.21.15.7:0/493246844 conn(0x564a186a9080 msgr2=0x564a1877bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:12.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.667+0000 7f0bfb93bb80 1 -- 172.21.15.7:0/493246844 shutdown_connections 2024-06-10T01:11:12.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.667+0000 7f0bfb93bb80 1 -- 172.21.15.7:0/493246844 wait complete. 2024-06-10T01:11:12.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.668+0000 7f0bfb93bb80 20 client.26299 trim_cache size 0 max 16384 2024-06-10T01:11:12.668 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:11:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.668+0000 7f0bf10b8700 1 -- 172.21.15.7:0/3346850257 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0bd40a2c20 msgr2=0x7f0bd40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.668+0000 7f0bf10b8700 1 --2- 172.21.15.7:0/3346850257 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0bd40a2c20 0x7f0bd40a50e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc009fd0 tx=0x7f0bdc005fb0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.668+0000 7f0bf10b8700 1 -- 172.21.15.7:0/3346850257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bec1066b0 msgr2=0x7f0bec103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.668+0000 7f0bf10b8700 1 --2- 172.21.15.7:0/3346850257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bec1066b0 0x7f0bec103af0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f0be4007d70 tx=0x7f0be400a340 comp rx=0 tx=0).stop 2024-06-10T01:11:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.668+0000 7f0bf10b8700 1 -- 172.21.15.7:0/3346850257 shutdown_connections 2024-06-10T01:11:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.668+0000 7f0bf10b8700 1 --2- 172.21.15.7:0/3346850257 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0bd40a2c20 0x7f0bd40a50e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.668+0000 7f0bf10b8700 1 --2- 172.21.15.7:0/3346850257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bec1066b0 0x7f0bec103af0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.668+0000 7f0bf10b8700 1 --2- 172.21.15.7:0/3346850257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bec101620 0x7f0bec103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.668+0000 7f0bf10b8700 1 --2- 172.21.15.7:0/3346850257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0bec101fd0 0x7f0bec1035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.668+0000 7f0bf10b8700 1 -- 172.21.15.7:0/3346850257 >> 172.21.15.7:0/3346850257 conn(0x7f0bec078340 msgr2=0x7f0bec06ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.668+0000 7f0bf10b8700 1 -- 172.21.15.7:0/3346850257 shutdown_connections 2024-06-10T01:11:12.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:12.668+0000 7f0bf10b8700 1 -- 172.21.15.7:0/3346850257 wait complete. 2024-06-10T01:11:12.729 INFO:tasks.workunit.client.0.smithi007.stdout: 1 345759 8.17 MB/sec execute 113 sec latency 12676.636 ms 2024-06-10T01:11:13.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:13 smithi175 ceph-mon[36636]: pgmap v588: 129 pgs: 129 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 1.7 MiB/s wr, 124 op/s 2024-06-10T01:11:13.730 INFO:tasks.workunit.client.0.smithi007.stdout: 1 345759 8.10 MB/sec execute 114 sec latency 13676.689 ms 2024-06-10T01:11:13.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:13 smithi007 ceph-mon[32398]: pgmap v588: 129 pgs: 129 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 1.7 MiB/s wr, 124 op/s 2024-06-10T01:11:13.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:13 smithi078 ceph-mon[36683]: pgmap v588: 129 pgs: 129 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 1.7 MiB/s wr, 124 op/s 2024-06-10T01:11:14.479 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:11:14.480 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:11:14.729 INFO:tasks.workunit.client.0.smithi007.stdout: 1 345759 8.03 MB/sec execute 115 sec latency 14676.738 ms 2024-06-10T01:11:14.741 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:11:15.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:15 smithi007 ceph-mon[32398]: pgmap v589: 129 pgs: 129 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.6 MiB/s wr, 123 op/s 2024-06-10T01:11:15.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:15 smithi175 ceph-mon[36636]: pgmap v589: 129 pgs: 129 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.6 MiB/s wr, 123 op/s 2024-06-10T01:11:15.730 INFO:tasks.workunit.client.0.smithi007.stdout: 1 345759 7.96 MB/sec execute 116 sec latency 15676.774 ms 2024-06-10T01:11:15.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:15 smithi078 ceph-mon[36683]: pgmap v589: 129 pgs: 129 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.6 MiB/s wr, 123 op/s 2024-06-10T01:11:16.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.520+0000 7f5d86431700 1 -- 172.21.15.7:0/3587193613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d80107840 msgr2=0x7f5d8010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:16.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.520+0000 7f5d86431700 1 --2- 172.21.15.7:0/3587193613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d80107840 0x7f5d8010f3c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5d7800b0d0 tx=0x7f5d7800b3e0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.520+0000 7f5d86431700 1 -- 172.21.15.7:0/3587193613 shutdown_connections 2024-06-10T01:11:16.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.520+0000 7f5d86431700 1 --2- 172.21.15.7:0/3587193613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d80107840 0x7f5d8010f3c0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.520+0000 7f5d86431700 1 --2- 172.21.15.7:0/3587193613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d80102740 0x7f5d80102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.520+0000 7f5d86431700 1 --2- 172.21.15.7:0/3587193613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d801030f0 0x7f5d80107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.520+0000 7f5d86431700 1 -- 172.21.15.7:0/3587193613 >> 172.21.15.7:0/3587193613 conn(0x7f5d800795f0 msgr2=0x7f5d80079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:16.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.520+0000 7f5d86431700 1 -- 172.21.15.7:0/3587193613 shutdown_connections 2024-06-10T01:11:16.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.520+0000 7f5d86431700 1 -- 172.21.15.7:0/3587193613 wait complete. 2024-06-10T01:11:16.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.521+0000 7f5d86431700 1 Processor -- start 2024-06-10T01:11:16.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.521+0000 7f5d86431700 1 -- start start 2024-06-10T01:11:16.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.521+0000 7f5d86431700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d80102740 0x7f5d801a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:16.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.521+0000 7f5d86431700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d801030f0 0x7f5d801a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:16.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.521+0000 7f5d86431700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d80107840 0x7f5d8019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:16.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.521+0000 7f5d86431700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5d80112760 con 0x7f5d80102740 2024-06-10T01:11:16.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.521+0000 7f5d86431700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5d801125e0 con 0x7f5d80107840 2024-06-10T01:11:16.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.521+0000 7f5d86431700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5d801128e0 con 0x7f5d801030f0 2024-06-10T01:11:16.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.521+0000 7f5d7ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d80102740 0x7f5d801a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:16.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.521+0000 7f5d7ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d80102740 0x7f5d801a08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47620/0 (socket says 172.21.15.7:47620) 2024-06-10T01:11:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.521+0000 7f5d7ffff700 1 -- 172.21.15.7:0/1368258865 learned_addr learned my addr 172.21.15.7:0/1368258865 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.521+0000 7f5d77fff700 1 --2- 172.21.15.7:0/1368258865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d801030f0 0x7f5d801a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.521+0000 7f5d849ce700 1 --2- 172.21.15.7:0/1368258865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d80107840 0x7f5d8019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.521+0000 7f5d7ffff700 1 -- 172.21.15.7:0/1368258865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d801030f0 msgr2=0x7f5d801a0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.521+0000 7f5d7ffff700 1 --2- 172.21.15.7:0/1368258865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d801030f0 0x7f5d801a0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.521+0000 7f5d7ffff700 1 -- 172.21.15.7:0/1368258865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d80107840 msgr2=0x7f5d8019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.521+0000 7f5d7ffff700 1 --2- 172.21.15.7:0/1368258865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d80107840 0x7f5d8019aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.521+0000 7f5d7ffff700 1 -- 172.21.15.7:0/1368258865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d8019b210 con 0x7f5d80102740 2024-06-10T01:11:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.521+0000 7f5d77fff700 1 --2- 172.21.15.7:0/1368258865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d801030f0 0x7f5d801a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:11:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.522+0000 7f5d7ffff700 1 --2- 172.21.15.7:0/1368258865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d80102740 0x7f5d801a08f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000c9b0 tx=0x7f5d7000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.522+0000 7f5d7dffb700 1 -- 172.21.15.7:0/1368258865 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d70007b00 con 0x7f5d80102740 2024-06-10T01:11:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.522+0000 7f5d86431700 1 -- 172.21.15.7:0/1368258865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d8019b500 con 0x7f5d80102740 2024-06-10T01:11:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.522+0000 7f5d7dffb700 1 -- 172.21.15.7:0/1368258865 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d70007ca0 con 0x7f5d80102740 2024-06-10T01:11:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.522+0000 7f5d7dffb700 1 -- 172.21.15.7:0/1368258865 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d70018780 con 0x7f5d80102740 2024-06-10T01:11:16.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.522+0000 7f5d86431700 1 -- 172.21.15.7:0/1368258865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d801a77c0 con 0x7f5d80102740 2024-06-10T01:11:16.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.524+0000 7f5d7dffb700 1 -- 172.21.15.7:0/1368258865 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f5d7000f450 con 0x7f5d80102740 2024-06-10T01:11:16.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.525+0000 7f5d7dffb700 1 --2- 172.21.15.7:0/1368258865 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5d6c0a2c20 0x7f5d6c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:16.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.525+0000 7f5d77fff700 1 --2- 172.21.15.7:0/1368258865 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5d6c0a2c20 0x7f5d6c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:16.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.525+0000 7f5d7dffb700 1 -- 172.21.15.7:0/1368258865 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f5d700cf4c0 con 0x7f5d80102740 2024-06-10T01:11:16.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.525+0000 7f5d77fff700 1 --2- 172.21.15.7:0/1368258865 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5d6c0a2c20 0x7f5d6c0a50e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5d8010f770 tx=0x7f5d68009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:16.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.525+0000 7f5d90cb4b80 1 Processor -- start 2024-06-10T01:11:16.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.526+0000 7f5d90cb4b80 1 -- start start 2024-06-10T01:11:16.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.526+0000 7f5d90cb4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3bb0db6c0 0x55d3bb0dbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:16.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.526+0000 7f5d90cb4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3bb0dbfe0 0x55d3bb0d5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:16.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.526+0000 7f5d90cb4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d3bb0d5fa0 0x55d3bb0d63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:16.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.526+0000 7f5d90cb4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d3bb027890 con 0x55d3bb0d5fa0 2024-06-10T01:11:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.526+0000 7f5d90cb4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d3bafcf7a0 con 0x55d3bb0db6c0 2024-06-10T01:11:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.526+0000 7f5d90cb4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d3bb023850 con 0x55d3bb0dbfe0 2024-06-10T01:11:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.526+0000 7f5d849ce700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d3bb0d5fa0 0x55d3bb0d63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.526+0000 7f5d849ce700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d3bb0d5fa0 0x55d3bb0d63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47636/0 (socket says 172.21.15.7:47636) 2024-06-10T01:11:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.526+0000 7f5d849ce700 1 -- 172.21.15.7:0/2711040492 learned_addr learned my addr 172.21.15.7:0/2711040492 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.526+0000 7f5d7ffff700 1 --2- 172.21.15.7:0/2711040492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3bb0db6c0 0x55d3bb0dbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.526+0000 7f5d77fff700 1 --2- 172.21.15.7:0/2711040492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3bb0dbfe0 0x55d3bb0d5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.526+0000 7f5d849ce700 1 -- 172.21.15.7:0/2711040492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3bb0dbfe0 msgr2=0x55d3bb0d5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.526+0000 7f5d849ce700 1 --2- 172.21.15.7:0/2711040492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3bb0dbfe0 0x55d3bb0d5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.526+0000 7f5d849ce700 1 -- 172.21.15.7:0/2711040492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3bb0db6c0 msgr2=0x55d3bb0dbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.526+0000 7f5d849ce700 1 --2- 172.21.15.7:0/2711040492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3bb0db6c0 0x55d3bb0dbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.526+0000 7f5d849ce700 1 -- 172.21.15.7:0/2711040492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3bafc5fd0 con 0x55d3bb0d5fa0 2024-06-10T01:11:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.527+0000 7f5d849ce700 1 --2- 172.21.15.7:0/2711040492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d3bb0d5fa0 0x55d3bb0d63c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f5d780092e0 tx=0x7f5d78009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b10bbb46ae3bf8ac server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.527+0000 7f5d767fc700 1 -- 172.21.15.7:0/2711040492 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d78020030 con 0x55d3bb0d5fa0 2024-06-10T01:11:16.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.527+0000 7f5d767fc700 1 -- 172.21.15.7:0/2711040492 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d78004a50 con 0x55d3bb0d5fa0 2024-06-10T01:11:16.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.527+0000 7f5d767fc700 1 -- 172.21.15.7:0/2711040492 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d78007af0 con 0x55d3bb0d5fa0 2024-06-10T01:11:16.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.527+0000 7f5d90cb4b80 1 -- 172.21.15.7:0/2711040492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d3bb0d5fa0 msgr2=0x55d3bb0d63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:16.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.527+0000 7f5d90cb4b80 1 --2- 172.21.15.7:0/2711040492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d3bb0d5fa0 0x55d3bb0d63c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f5d780092e0 tx=0x7f5d78009310 comp rx=0 tx=0).stop 2024-06-10T01:11:16.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.527+0000 7f5d90cb4b80 1 -- 172.21.15.7:0/2711040492 shutdown_connections 2024-06-10T01:11:16.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.528+0000 7f5d90cb4b80 1 --2- 172.21.15.7:0/2711040492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d3bb0d5fa0 0x55d3bb0d63c0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.528+0000 7f5d90cb4b80 1 --2- 172.21.15.7:0/2711040492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3bb0db6c0 0x55d3bb0dbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.528+0000 7f5d90cb4b80 1 --2- 172.21.15.7:0/2711040492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3bb0dbfe0 0x55d3bb0d5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.528+0000 7f5d90cb4b80 1 -- 172.21.15.7:0/2711040492 >> 172.21.15.7:0/2711040492 conn(0x55d3bb000d40 msgr2=0x55d3bb0d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.528+0000 7f5d90cb4b80 1 -- 172.21.15.7:0/2711040492 shutdown_connections 2024-06-10T01:11:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.528+0000 7f5d90cb4b80 1 -- 172.21.15.7:0/2711040492 wait complete. 2024-06-10T01:11:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.528+0000 7f5d90cb4b80 1 -- start start 2024-06-10T01:11:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.528+0000 7f5d90cb4b80 1 Processor -- start 2024-06-10T01:11:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.528+0000 7f5d90cb4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3bb0d5fa0 0x55d3bb0e1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.528+0000 7f5d90cb4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d3bb0db6c0 0x55d3bb0df0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.528+0000 7f5d90cb4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3bb0dbfe0 0x55d3bb0df5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.528+0000 7f5d90cb4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d3bb0e29f0 con 0x55d3bb0db6c0 2024-06-10T01:11:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.529+0000 7f5d90cb4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d3bafc7c70 con 0x55d3bb0d5fa0 2024-06-10T01:11:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.529+0000 7f5d90cb4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d3bb0e2b70 con 0x55d3bb0dbfe0 2024-06-10T01:11:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.529+0000 7f5d90cb4b80 10 client.0 _ll_register_callbacks cb 0x55d3bafc19a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:11:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.529+0000 7f5d77fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d3bb0db6c0 0x55d3bb0df0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.529+0000 7f5d77fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d3bb0db6c0 0x55d3bb0df0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47642/0 (socket says 172.21.15.7:47642) 2024-06-10T01:11:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.529+0000 7f5d77fff700 1 -- 172.21.15.7:0/2522488925 learned_addr learned my addr 172.21.15.7:0/2522488925 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.529+0000 7f5d7ffff700 1 --2- 172.21.15.7:0/2522488925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3bb0d5fa0 0x55d3bb0e1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.529+0000 7f5d849ce700 1 --2- 172.21.15.7:0/2522488925 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3bb0dbfe0 0x55d3bb0df5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.529+0000 7f5d77fff700 1 -- 172.21.15.7:0/2522488925 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3bb0dbfe0 msgr2=0x55d3bb0df5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.529+0000 7f5d77fff700 1 --2- 172.21.15.7:0/2522488925 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3bb0dbfe0 0x55d3bb0df5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.529+0000 7f5d77fff700 1 -- 172.21.15.7:0/2522488925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3bb0d5fa0 msgr2=0x55d3bb0e1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.529+0000 7f5d77fff700 1 --2- 172.21.15.7:0/2522488925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3bb0d5fa0 0x55d3bb0e1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.529+0000 7f5d77fff700 1 -- 172.21.15.7:0/2522488925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3baf76950 con 0x55d3bb0db6c0 2024-06-10T01:11:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.529+0000 7f5d7ffff700 1 --2- 172.21.15.7:0/2522488925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3bb0d5fa0 0x55d3bb0e1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:11:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.529+0000 7f5d849ce700 1 --2- 172.21.15.7:0/2522488925 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3bb0dbfe0 0x55d3bb0df5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:11:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.529+0000 7f5d77fff700 1 --2- 172.21.15.7:0/2522488925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d3bb0db6c0 0x55d3bb0df0b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f5d68010fa0 tx=0x7f5d68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf9c859d1bcb6a3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.529+0000 7f5d5f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:11:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.529+0000 7f5d5f7fe700 1 -- 172.21.15.7:0/2522488925 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d68020630 con 0x55d3bb0db6c0 2024-06-10T01:11:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.529+0000 7f5d90cb4b80 1 -- 172.21.15.7:0/2522488925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d78004a50 con 0x55d3bb0db6c0 2024-06-10T01:11:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.529+0000 7f5d5f7fe700 1 -- 172.21.15.7:0/2522488925 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f5d680207d0 con 0x55d3bb0db6c0 2024-06-10T01:11:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.530+0000 7f5d5f7fe700 1 -- 172.21.15.7:0/2522488925 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5d68031050 con 0x55d3bb0db6c0 2024-06-10T01:11:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.530+0000 7f5d90cb4b80 10 client.16815 fetch_fsmap learned FSMap version 32 2024-06-10T01:11:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.530+0000 7f5d90cb4b80 1 -- 172.21.15.7:0/2522488925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d3bb0ed400 con 0x55d3bb0db6c0 2024-06-10T01:11:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.530+0000 7f5d5f7fe700 1 -- 172.21.15.7:0/2522488925 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d78004a50 con 0x55d3bb0db6c0 2024-06-10T01:11:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.530+0000 7f5d5f7fe700 1 -- 172.21.15.7:0/2522488925 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f5d6800f040 con 0x55d3bb0db6c0 2024-06-10T01:11:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.530+0000 7f5d90cb4b80 10 client.16815 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:11:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.530+0000 7f5d90cb4b80 10 client.16815 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:11:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.530+0000 7f5d90cb4b80 20 client.16815 populate_metadata read hostname 'smithi007' 2024-06-10T01:11:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.530+0000 7f5d90cb4b80 1 --2- 172.21.15.7:0/2522488925 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d3bb0ef9e0 0x55d3bb0f1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:16.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.530+0000 7f5d90cb4b80 4 client.16815 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:11:16.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.530+0000 7f5d90cb4b80 1 -- 172.21.15.7:0/2522488925 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3bb034d90 con 0x55d3bb0ef9e0 2024-06-10T01:11:16.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.530+0000 7f5d7ffff700 1 --2- 172.21.15.7:0/2522488925 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d3bb0ef9e0 0x55d3bb0f1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:16.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.531+0000 7f5d7ffff700 1 --2- 172.21.15.7:0/2522488925 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d3bb0ef9e0 0x55d3bb0f1e20 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=870e925a7f1cf3bd server_cookie=d8e087de967190c4 in_seq=0 out_seq=0 2024-06-10T01:11:16.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.531+0000 7f5d5f7fe700 10 client.16815 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:16.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.531+0000 7f5d5f7fe700 1 -- 172.21.15.7:0/2522488925 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d3bb034d90 con 0x55d3bb0ef9e0 2024-06-10T01:11:16.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.531+0000 7f5d5f7fe700 10 client.16815 handle_command_reply: tid=0 2024-06-10T01:11:16.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.532+0000 7f5d90cb4b80 1 client.16815 shutdown 2024-06-10T01:11:16.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.532+0000 7f5d90cb4b80 1 -- 172.21.15.7:0/2522488925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d3bb0db6c0 msgr2=0x55d3bb0df0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:16.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.532+0000 7f5d90cb4b80 1 --2- 172.21.15.7:0/2522488925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d3bb0db6c0 0x55d3bb0df0b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f5d68010fa0 tx=0x7f5d68004750 comp rx=0 tx=0).stop 2024-06-10T01:11:16.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.532+0000 7f5d90cb4b80 1 -- 172.21.15.7:0/2522488925 shutdown_connections 2024-06-10T01:11:16.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.532+0000 7f5d90cb4b80 1 --2- 172.21.15.7:0/2522488925 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3bb0dbfe0 0x55d3bb0df5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.532+0000 7f5d90cb4b80 1 --2- 172.21.15.7:0/2522488925 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d3bb0ef9e0 0x55d3bb0f1e20 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.532+0000 7f5d90cb4b80 1 --2- 172.21.15.7:0/2522488925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3bb0d5fa0 0x55d3bb0e1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.532+0000 7f5d90cb4b80 1 --2- 172.21.15.7:0/2522488925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d3bb0db6c0 0x55d3bb0df0b0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.532+0000 7f5d90cb4b80 1 -- 172.21.15.7:0/2522488925 >> 172.21.15.7:0/2522488925 conn(0x55d3bb000d40 msgr2=0x55d3bb0d4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:16.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.532+0000 7f5d5f7fe700 0 client.16815 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:16.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.532+0000 7f5d90cb4b80 1 -- 172.21.15.7:0/2522488925 shutdown_connections 2024-06-10T01:11:16.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.532+0000 7f5d90cb4b80 1 -- 172.21.15.7:0/2522488925 wait complete. 2024-06-10T01:11:16.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.532+0000 7f5d90cb4b80 20 client.16815 trim_cache size 0 max 16384 2024-06-10T01:11:16.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.553+0000 7f5d90cb4b80 1 Processor -- start 2024-06-10T01:11:16.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.553+0000 7f5d90cb4b80 1 -- start start 2024-06-10T01:11:16.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.553+0000 7f5d90cb4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3bb0d5fa0 0x55d3bb0d5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:16.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.553+0000 7f5d90cb4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3bb0db6c0 0x55d3bb0dcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:16.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.553+0000 7f5d90cb4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d3bb0dbfe0 0x55d3bb0ca710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:16.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.553+0000 7f5d90cb4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5d801b2be0 con 0x55d3bb0dbfe0 2024-06-10T01:11:16.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.553+0000 7f5d90cb4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5d801b0470 con 0x55d3bb0db6c0 2024-06-10T01:11:16.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.553+0000 7f5d90cb4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5d801ae2c0 con 0x55d3bb0d5fa0 2024-06-10T01:11:16.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.553+0000 7f5d849ce700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d3bb0dbfe0 0x55d3bb0ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:16.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.553+0000 7f5d849ce700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d3bb0dbfe0 0x55d3bb0ca710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47648/0 (socket says 172.21.15.7:47648) 2024-06-10T01:11:16.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.553+0000 7f5d849ce700 1 -- 172.21.15.7:0/3599713956 learned_addr learned my addr 172.21.15.7:0/3599713956 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:16.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.553+0000 7f5d849ce700 1 -- 172.21.15.7:0/3599713956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3bb0d5fa0 msgr2=0x55d3bb0d5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:16.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.553+0000 7f5d849ce700 1 --2- 172.21.15.7:0/3599713956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3bb0d5fa0 0x55d3bb0d5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.553+0000 7f5d849ce700 1 -- 172.21.15.7:0/3599713956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3bb0db6c0 msgr2=0x55d3bb0dcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:16.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.553+0000 7f5d849ce700 1 --2- 172.21.15.7:0/3599713956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3bb0db6c0 0x55d3bb0dcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.553+0000 7f5d849ce700 1 -- 172.21.15.7:0/3599713956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3bb0d1580 con 0x55d3bb0dbfe0 2024-06-10T01:11:16.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.553+0000 7f5d849ce700 1 --2- 172.21.15.7:0/3599713956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d3bb0dbfe0 0x55d3bb0ca710 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5d78009380 tx=0x7f5d780097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2acf542a78f005 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:16.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.554+0000 7f5d75ffb700 1 -- 172.21.15.7:0/3599713956 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d78020040 con 0x55d3bb0dbfe0 2024-06-10T01:11:16.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.554+0000 7f5d75ffb700 1 -- 172.21.15.7:0/3599713956 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d7800bba0 con 0x55d3bb0dbfe0 2024-06-10T01:11:16.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.554+0000 7f5d75ffb700 1 -- 172.21.15.7:0/3599713956 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d78007d70 con 0x55d3bb0dbfe0 2024-06-10T01:11:16.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.554+0000 7f5d90cb4b80 1 -- 172.21.15.7:0/3599713956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d3bb0dbfe0 msgr2=0x55d3bb0ca710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:16.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.554+0000 7f5d90cb4b80 1 --2- 172.21.15.7:0/3599713956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d3bb0dbfe0 0x55d3bb0ca710 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5d78009380 tx=0x7f5d780097e0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.554+0000 7f5d90cb4b80 1 -- 172.21.15.7:0/3599713956 shutdown_connections 2024-06-10T01:11:16.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.554+0000 7f5d90cb4b80 1 --2- 172.21.15.7:0/3599713956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d3bb0dbfe0 0x55d3bb0ca710 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.554+0000 7f5d90cb4b80 1 --2- 172.21.15.7:0/3599713956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3bb0d5fa0 0x55d3bb0d5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.554+0000 7f5d90cb4b80 1 --2- 172.21.15.7:0/3599713956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3bb0db6c0 0x55d3bb0dcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.554+0000 7f5d90cb4b80 1 -- 172.21.15.7:0/3599713956 >> 172.21.15.7:0/3599713956 conn(0x55d3bb000d40 msgr2=0x55d3bb0dd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:16.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.554+0000 7f5d90cb4b80 1 -- 172.21.15.7:0/3599713956 shutdown_connections 2024-06-10T01:11:16.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.554+0000 7f5d90cb4b80 1 -- 172.21.15.7:0/3599713956 wait complete. 2024-06-10T01:11:16.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.555+0000 7f5d90cb4b80 1 -- start start 2024-06-10T01:11:16.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.555+0000 7f5d90cb4b80 1 Processor -- start 2024-06-10T01:11:16.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.555+0000 7f5d90cb4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3bb0d5fa0 0x55d3bb0e3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:16.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.555+0000 7f5d90cb4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d3bb0db6c0 0x55d3bb0e4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:16.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.555+0000 7f5d90cb4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3bb0dbfe0 0x55d3bb159670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:16.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.555+0000 7f5d90cb4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5d801ae930 con 0x55d3bb0db6c0 2024-06-10T01:11:16.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.555+0000 7f5d90cb4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5d801afda0 con 0x55d3bb0dbfe0 2024-06-10T01:11:16.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.555+0000 7f5d90cb4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5d801af000 con 0x55d3bb0d5fa0 2024-06-10T01:11:16.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.555+0000 7f5d90cb4b80 10 client.0 _ll_register_callbacks cb 0x55d3bafc19a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:11:16.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.555+0000 7f5d77fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d3bb0db6c0 0x55d3bb0e4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:16.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.555+0000 7f5d77fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d3bb0db6c0 0x55d3bb0e4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47652/0 (socket says 172.21.15.7:47652) 2024-06-10T01:11:16.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.555+0000 7f5d77fff700 1 -- 172.21.15.7:0/2719834644 learned_addr learned my addr 172.21.15.7:0/2719834644 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:16.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.555+0000 7f5d7ffff700 1 --2- 172.21.15.7:0/2719834644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3bb0d5fa0 0x55d3bb0e3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:16.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.556+0000 7f5d849ce700 1 --2- 172.21.15.7:0/2719834644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3bb0dbfe0 0x55d3bb159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:16.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.556+0000 7f5d77fff700 1 -- 172.21.15.7:0/2719834644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3bb0d5fa0 msgr2=0x55d3bb0e3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:16.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.556+0000 7f5d77fff700 1 --2- 172.21.15.7:0/2719834644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3bb0d5fa0 0x55d3bb0e3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.556+0000 7f5d77fff700 1 -- 172.21.15.7:0/2719834644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3bb0dbfe0 msgr2=0x55d3bb159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:16.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.556+0000 7f5d77fff700 1 --2- 172.21.15.7:0/2719834644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3bb0dbfe0 0x55d3bb159670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.556+0000 7f5d77fff700 1 -- 172.21.15.7:0/2719834644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3bb0e4d30 con 0x55d3bb0db6c0 2024-06-10T01:11:16.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.556+0000 7f5d849ce700 1 --2- 172.21.15.7:0/2719834644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3bb0dbfe0 0x55d3bb159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:11:16.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.556+0000 7f5d77fff700 1 --2- 172.21.15.7:0/2719834644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d3bb0db6c0 0x55d3bb0e4460 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5d68015040 tx=0x7f5d68005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d87c9e6c5ae47e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:16.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.556+0000 7f5d5f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:11:16.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.556+0000 7f5d5f7fe700 1 -- 172.21.15.7:0/2719834644 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d68039020 con 0x55d3bb0db6c0 2024-06-10T01:11:16.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.556+0000 7f5d5f7fe700 1 -- 172.21.15.7:0/2719834644 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f5d68003ab0 con 0x55d3bb0db6c0 2024-06-10T01:11:16.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.556+0000 7f5d90cb4b80 1 -- 172.21.15.7:0/2719834644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d7800bba0 con 0x55d3bb0db6c0 2024-06-10T01:11:16.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.556+0000 7f5d5f7fe700 1 -- 172.21.15.7:0/2719834644 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5d68044050 con 0x55d3bb0db6c0 2024-06-10T01:11:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.556+0000 7f5d90cb4b80 10 client.16821 fetch_fsmap learned FSMap version 32 2024-06-10T01:11:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.556+0000 7f5d90cb4b80 1 -- 172.21.15.7:0/2719834644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d3bb15acf0 con 0x55d3bb0db6c0 2024-06-10T01:11:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.557+0000 7f5d5f7fe700 1 -- 172.21.15.7:0/2719834644 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d7800bba0 con 0x55d3bb0db6c0 2024-06-10T01:11:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.557+0000 7f5d5f7fe700 1 -- 172.21.15.7:0/2719834644 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f5d68034040 con 0x55d3bb0db6c0 2024-06-10T01:11:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.557+0000 7f5d90cb4b80 10 client.16821 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:11:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.557+0000 7f5d90cb4b80 10 client.16821 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:11:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.557+0000 7f5d90cb4b80 20 client.16821 populate_metadata read hostname 'smithi007' 2024-06-10T01:11:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.557+0000 7f5d90cb4b80 1 --2- 172.21.15.7:0/2719834644 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d3bb15b3c0 0x55d3bb15d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.557+0000 7f5d90cb4b80 4 client.16821 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:11:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.557+0000 7f5d90cb4b80 1 -- 172.21.15.7:0/2719834644 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d3bb02f310 con 0x55d3bb15b3c0 2024-06-10T01:11:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.557+0000 7f5d7ffff700 1 --2- 172.21.15.7:0/2719834644 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d3bb15b3c0 0x55d3bb15d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.557+0000 7f5d7ffff700 1 --2- 172.21.15.7:0/2719834644 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d3bb15b3c0 0x55d3bb15d7e0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a15ffc33c3cff41b server_cookie=873fa715aae8a2fd in_seq=0 out_seq=0 2024-06-10T01:11:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.557+0000 7f5d5f7fe700 10 client.16821 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:16.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.559+0000 7f5d5f7fe700 1 -- 172.21.15.7:0/2719834644 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d3bb02f310 con 0x55d3bb15b3c0 2024-06-10T01:11:16.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.559+0000 7f5d5f7fe700 10 client.16821 handle_command_reply: tid=0 2024-06-10T01:11:16.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.559+0000 7f5d90cb4b80 1 client.16821 shutdown 2024-06-10T01:11:16.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.559+0000 7f5d90cb4b80 1 -- 172.21.15.7:0/2719834644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d3bb0db6c0 msgr2=0x55d3bb0e4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:16.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.559+0000 7f5d90cb4b80 1 --2- 172.21.15.7:0/2719834644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d3bb0db6c0 0x55d3bb0e4460 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5d68015040 tx=0x7f5d68005730 comp rx=0 tx=0).stop 2024-06-10T01:11:16.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.559+0000 7f5d90cb4b80 1 -- 172.21.15.7:0/2719834644 shutdown_connections 2024-06-10T01:11:16.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.559+0000 7f5d90cb4b80 1 --2- 172.21.15.7:0/2719834644 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d3bb15b3c0 0x55d3bb15d7e0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.559+0000 7f5d90cb4b80 1 --2- 172.21.15.7:0/2719834644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3bb0dbfe0 0x55d3bb159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.559+0000 7f5d90cb4b80 1 --2- 172.21.15.7:0/2719834644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d3bb0d5fa0 0x55d3bb0e3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.559+0000 7f5d90cb4b80 1 --2- 172.21.15.7:0/2719834644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d3bb0db6c0 0x55d3bb0e4460 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.559+0000 7f5d90cb4b80 1 -- 172.21.15.7:0/2719834644 >> 172.21.15.7:0/2719834644 conn(0x55d3bb000d40 msgr2=0x55d3bb0e1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:16.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.559+0000 7f5d5f7fe700 0 client.16821 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:16.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.559+0000 7f5d90cb4b80 1 -- 172.21.15.7:0/2719834644 shutdown_connections 2024-06-10T01:11:16.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.559+0000 7f5d90cb4b80 1 -- 172.21.15.7:0/2719834644 wait complete. 2024-06-10T01:11:16.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.559+0000 7f5d90cb4b80 20 client.16821 trim_cache size 0 max 16384 2024-06-10T01:11:16.559 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:11:16.559 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-10T01:11:16.559 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "2380d01b-750e-4985-88c1-5e8d2c6d4c42", 2024-06-10T01:11:16.559 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-10T01:11:16.559 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:11:16.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.560+0000 7f5d86431700 1 -- 172.21.15.7:0/1368258865 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5d6c0a2c20 msgr2=0x7f5d6c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:16.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.560+0000 7f5d86431700 1 --2- 172.21.15.7:0/1368258865 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5d6c0a2c20 0x7f5d6c0a50e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5d8010f770 tx=0x7f5d68009260 comp rx=0 tx=0).stop 2024-06-10T01:11:16.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.560+0000 7f5d86431700 1 -- 172.21.15.7:0/1368258865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d80102740 msgr2=0x7f5d801a08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:16.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.560+0000 7f5d86431700 1 --2- 172.21.15.7:0/1368258865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d80102740 0x7f5d801a08f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000c9b0 tx=0x7f5d7000cd70 comp rx=0 tx=0).stop 2024-06-10T01:11:16.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.560+0000 7f5d86431700 1 -- 172.21.15.7:0/1368258865 shutdown_connections 2024-06-10T01:11:16.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.560+0000 7f5d86431700 1 --2- 172.21.15.7:0/1368258865 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5d6c0a2c20 0x7f5d6c0a50e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.560+0000 7f5d86431700 1 --2- 172.21.15.7:0/1368258865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d80107840 0x7f5d8019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.560+0000 7f5d86431700 1 --2- 172.21.15.7:0/1368258865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d80102740 0x7f5d801a08f0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.560+0000 7f5d86431700 1 --2- 172.21.15.7:0/1368258865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d801030f0 0x7f5d801a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:16.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.560+0000 7f5d86431700 1 -- 172.21.15.7:0/1368258865 >> 172.21.15.7:0/1368258865 conn(0x7f5d800795f0 msgr2=0x7f5d801005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:16.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.560+0000 7f5d86431700 1 -- 172.21.15.7:0/1368258865 shutdown_connections 2024-06-10T01:11:16.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:16.560+0000 7f5d86431700 1 -- 172.21.15.7:0/1368258865 wait complete. 2024-06-10T01:11:16.730 INFO:tasks.workunit.client.0.smithi007.stdout: 1 345759 7.89 MB/sec execute 117 sec latency 16676.811 ms 2024-06-10T01:11:16.953 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jun 10 01:11:16 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-i[66802]: 2024-06-10T01:11:16.583+0000 7fe512307700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000229 [...2,head] /volumes/qa/sv_1/56b71f3b-2893-4c82-ac3a-2d6fa4194c93/client.0/tmp/clients/client0/~dmtmp/PARADOX/ auth{1=1,2=1} v58570 ap=3 f(v1964 m2024-06-10T01:11:00.054695+0000 25=25+0) n(v19 rc2024-06-10T01:11:00.054695+0000 b5218847 26=25+1) (inest sync->mix g=1,2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:260 nref=3)) caps={24973=pAsLsXsFs/pAsLsXsFsxcral@6913} mcw={1=AsLsXsFs} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x55e19b59a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000229:[<0x10000000009/PARADOX v1870>,<0x10000000008/~dmtmp v34>,<0x10000000007/client0 v18>,<0x10000000006/clients v8>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/56b71f3b-2893-4c82-ac3a-2d6fa4194c93 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v37>,<0x1/volumes v37>]//[]","memoryvalue":"(2)0x10000000229:[<0x10000000009/PARADOX v58570>,<0x10000000008/~dmtmp v14774>,<0x10000000007/client0 v461>,<0x10000000006/clients v430>,<0x10000000005/tmp v420>,<0x100000001fb/client.0 v417>,<0x100000001fa/56b71f3b-2893-4c82-ac3a-2d6fa4194c93 v422>,<0x10000000001/sv_1 v431>,<0x10000000000/qa v418>,<0x1/volumes v397>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-10T01:11:00.046695+0000 26=26+0)","ondisk_value.rstat":"n(v0 rc2024-06-10T01:11:00.054695+0000 b5218847 26=25+1)","memory_value.dirstat":"f(v1964 m2024-06-10T01:11:00.054695+0000 25=25+0)","memory_value.rstat":"n(v19 rc2024-06-10T01:11:00.054695+0000 b5218847 26=25+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-10T01:11:17.403 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:11:17.654 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:11:17.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:17 smithi175 ceph-mon[36636]: pgmap v590: 129 pgs: 129 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 1.3 MiB/s wr, 110 op/s 2024-06-10T01:11:17.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:17 smithi175 ceph-mon[36636]: scrub queued for path: / 2024-06-10T01:11:17.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:17 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:11:17.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:17 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:11:17.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:17 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:11:17.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:17 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:11:17.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:17 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:11:17.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:17 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:11:17.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:17 smithi175 ceph-mon[36636]: Scrub error on inode 0x10000000229 (/volumes/qa/sv_1/56b71f3b-2893-4c82-ac3a-2d6fa4194c93/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.i log and `damage ls` output for details 2024-06-10T01:11:17.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:17 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:11:17.729 INFO:tasks.workunit.client.0.smithi007.stdout: 1 345759 7.82 MB/sec execute 118 sec latency 17676.879 ms 2024-06-10T01:11:17.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:17 smithi007 ceph-mon[32398]: pgmap v590: 129 pgs: 129 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 1.3 MiB/s wr, 110 op/s 2024-06-10T01:11:17.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:17 smithi007 ceph-mon[32398]: scrub queued for path: / 2024-06-10T01:11:17.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:17 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:11:17.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:17 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:11:17.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:17 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:11:17.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:17 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:11:17.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:17 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:11:17.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:17 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:11:17.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:17 smithi007 ceph-mon[32398]: Scrub error on inode 0x10000000229 (/volumes/qa/sv_1/56b71f3b-2893-4c82-ac3a-2d6fa4194c93/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.i log and `damage ls` output for details 2024-06-10T01:11:17.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:17 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:11:17.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:17 smithi078 ceph-mon[36683]: pgmap v590: 129 pgs: 129 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 1.3 MiB/s wr, 110 op/s 2024-06-10T01:11:17.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:17 smithi078 ceph-mon[36683]: scrub queued for path: / 2024-06-10T01:11:17.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:17 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:11:17.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:17 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:11:17.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:17 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:11:17.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:17 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:11:17.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:17 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:11:17.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:17 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:11:17.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:17 smithi078 ceph-mon[36683]: Scrub error on inode 0x10000000229 (/volumes/qa/sv_1/56b71f3b-2893-4c82-ac3a-2d6fa4194c93/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.i log and `damage ls` output for details 2024-06-10T01:11:17.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:17 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:11:18.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:18 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:11:18.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:18 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:11:18.730 INFO:tasks.workunit.client.0.smithi007.stdout: 1 345759 7.76 MB/sec execute 119 sec latency 18676.919 ms 2024-06-10T01:11:18.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:18 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:11:18.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:18 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:11:18.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:18 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:11:18.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:18 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:11:19.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.437+0000 7fc0aa427700 1 -- 172.21.15.7:0/1775004183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0a4101d80 msgr2=0x7fc0a410e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:19.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.437+0000 7fc0aa427700 1 --2- 172.21.15.7:0/1775004183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0a4101d80 0x7fc0a410e240 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00b0a0 tx=0x7fc09c00b3b0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.437+0000 7fc0aa427700 1 -- 172.21.15.7:0/1775004183 shutdown_connections 2024-06-10T01:11:19.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.437+0000 7fc0aa427700 1 --2- 172.21.15.7:0/1775004183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0a4101d80 0x7fc0a410e240 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.437+0000 7fc0aa427700 1 --2- 172.21.15.7:0/1775004183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0a4107440 0x7fc0a4107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.437+0000 7fc0aa427700 1 --2- 172.21.15.7:0/1775004183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0a41013c0 0x7fc0a4101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.437+0000 7fc0aa427700 1 -- 172.21.15.7:0/1775004183 >> 172.21.15.7:0/1775004183 conn(0x7fc0a4077400 msgr2=0x7fc0a4077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:19.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.437+0000 7fc0aa427700 1 -- 172.21.15.7:0/1775004183 shutdown_connections 2024-06-10T01:11:19.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.437+0000 7fc0aa427700 1 -- 172.21.15.7:0/1775004183 wait complete. 2024-06-10T01:11:19.433 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:19 smithi007 ceph-mon[32398]: pgmap v591: 129 pgs: 129 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.2 MiB/s wr, 98 op/s 2024-06-10T01:11:19.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.437+0000 7fc0aa427700 1 Processor -- start 2024-06-10T01:11:19.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.437+0000 7fc0aa427700 1 -- start start 2024-06-10T01:11:19.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.438+0000 7fc0aa427700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0a41013c0 0x7fc0a419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:19.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.438+0000 7fc0aa427700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0a4101d80 0x7fc0a419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:19.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.438+0000 7fc0aa427700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0a4107440 0x7fc0a4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:19.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.438+0000 7fc0aa427700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a4111660 con 0x7fc0a4101d80 2024-06-10T01:11:19.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.438+0000 7fc0aa427700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a41114e0 con 0x7fc0a4107440 2024-06-10T01:11:19.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.438+0000 7fc0aa427700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a41117e0 con 0x7fc0a41013c0 2024-06-10T01:11:19.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.438+0000 7fc0a37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0a4101d80 0x7fc0a419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:19.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.438+0000 7fc0a37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0a4101d80 0x7fc0a419ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47670/0 (socket says 172.21.15.7:47670) 2024-06-10T01:11:19.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.438+0000 7fc0a37fe700 1 -- 172.21.15.7:0/556556661 learned_addr learned my addr 172.21.15.7:0/556556661 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:19.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.438+0000 7fc0a3fff700 1 --2- 172.21.15.7:0/556556661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0a41013c0 0x7fc0a419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:19.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.438+0000 7fc0a89c4700 1 --2- 172.21.15.7:0/556556661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0a4107440 0x7fc0a4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:19.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.438+0000 7fc0a37fe700 1 -- 172.21.15.7:0/556556661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0a41013c0 msgr2=0x7fc0a419e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:19.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.438+0000 7fc0a37fe700 1 --2- 172.21.15.7:0/556556661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0a41013c0 0x7fc0a419e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.438+0000 7fc0a37fe700 1 -- 172.21.15.7:0/556556661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0a4107440 msgr2=0x7fc0a4198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:19.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.438+0000 7fc0a37fe700 1 --2- 172.21.15.7:0/556556661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0a4107440 0x7fc0a4198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.438+0000 7fc0a37fe700 1 -- 172.21.15.7:0/556556661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a41990d0 con 0x7fc0a4101d80 2024-06-10T01:11:19.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.439+0000 7fc0a37fe700 1 --2- 172.21.15.7:0/556556661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0a4101d80 0x7fc0a419ecf0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fc09800d950 tx=0x7fc09800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:19.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.439+0000 7fc0a17fa700 1 -- 172.21.15.7:0/556556661 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098016070 con 0x7fc0a4101d80 2024-06-10T01:11:19.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.439+0000 7fc0a17fa700 1 -- 172.21.15.7:0/556556661 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc09800fd70 con 0x7fc0a4101d80 2024-06-10T01:11:19.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.439+0000 7fc0aa427700 1 -- 172.21.15.7:0/556556661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0a41993c0 con 0x7fc0a4101d80 2024-06-10T01:11:19.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.439+0000 7fc0a17fa700 1 -- 172.21.15.7:0/556556661 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09800e610 con 0x7fc0a4101d80 2024-06-10T01:11:19.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.439+0000 7fc0aa427700 1 -- 172.21.15.7:0/556556661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0a41a5680 con 0x7fc0a4101d80 2024-06-10T01:11:19.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.441+0000 7fc0a17fa700 1 -- 172.21.15.7:0/556556661 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fc098019070 con 0x7fc0a4101d80 2024-06-10T01:11:19.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.442+0000 7fc0a17fa700 1 --2- 172.21.15.7:0/556556661 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc08c0a2c20 0x7fc08c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:19.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.442+0000 7fc0a3fff700 1 --2- 172.21.15.7:0/556556661 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc08c0a2c20 0x7fc08c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:19.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.442+0000 7fc0a17fa700 1 -- 172.21.15.7:0/556556661 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc0980ceed0 con 0x7fc0a4101d80 2024-06-10T01:11:19.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.442+0000 7fc0a3fff700 1 --2- 172.21.15.7:0/556556661 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc08c0a2c20 0x7fc08c0a50e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc094005950 tx=0x7fc09400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:19.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.443+0000 7fc0b4caab80 1 Processor -- start 2024-06-10T01:11:19.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.443+0000 7fc0b4caab80 1 -- start start 2024-06-10T01:11:19.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.443+0000 7fc0b4caab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634abb1e640 0x5634abb1ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:19.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.443+0000 7fc0b4caab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634abb1ef60 0x5634abb189e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:19.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.443+0000 7fc0b4caab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634abb18f20 0x5634abb19340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:19.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.443+0000 7fc0b4caab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5634aba6a810 con 0x5634abb1e640 2024-06-10T01:11:19.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.443+0000 7fc0b4caab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5634aba12720 con 0x5634abb18f20 2024-06-10T01:11:19.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.443+0000 7fc0b4caab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5634aba667d0 con 0x5634abb1ef60 2024-06-10T01:11:19.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.443+0000 7fc0a3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634abb1e640 0x5634abb1ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:19.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.443+0000 7fc0a3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634abb1e640 0x5634abb1ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47684/0 (socket says 172.21.15.7:47684) 2024-06-10T01:11:19.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.443+0000 7fc0a3fff700 1 -- 172.21.15.7:0/1167409386 learned_addr learned my addr 172.21.15.7:0/1167409386 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:19.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.443+0000 7fc0a89c4700 1 --2- 172.21.15.7:0/1167409386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634abb18f20 0x5634abb19340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:19.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.443+0000 7fc0a37fe700 1 --2- 172.21.15.7:0/1167409386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634abb1ef60 0x5634abb189e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:19.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.443+0000 7fc0a3fff700 1 -- 172.21.15.7:0/1167409386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634abb1ef60 msgr2=0x5634abb189e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:19.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.443+0000 7fc0a3fff700 1 --2- 172.21.15.7:0/1167409386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634abb1ef60 0x5634abb189e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.443+0000 7fc0a3fff700 1 -- 172.21.15.7:0/1167409386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634abb18f20 msgr2=0x5634abb19340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:19.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.443+0000 7fc0a3fff700 1 --2- 172.21.15.7:0/1167409386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634abb18f20 0x5634abb19340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.444+0000 7fc0a3fff700 1 -- 172.21.15.7:0/1167409386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634aba08f50 con 0x5634abb1e640 2024-06-10T01:11:19.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.444+0000 7fc0a89c4700 1 --2- 172.21.15.7:0/1167409386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634abb18f20 0x5634abb19340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:11:19.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.444+0000 7fc0a37fe700 1 --2- 172.21.15.7:0/1167409386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634abb1ef60 0x5634abb189e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:11:19.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.444+0000 7fc0a3fff700 1 --2- 172.21.15.7:0/1167409386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634abb1e640 0x5634abb1ea20 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fc09400fd30 tx=0x7fc094012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a583fe43e346c0fa server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:19.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.444+0000 7fc08a7fc700 1 -- 172.21.15.7:0/1167409386 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc094016070 con 0x5634abb1e640 2024-06-10T01:11:19.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.444+0000 7fc08a7fc700 1 -- 172.21.15.7:0/1167409386 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc094010da0 con 0x5634abb1e640 2024-06-10T01:11:19.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.444+0000 7fc08a7fc700 1 -- 172.21.15.7:0/1167409386 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09401b420 con 0x5634abb1e640 2024-06-10T01:11:19.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.445+0000 7fc0b4caab80 1 -- 172.21.15.7:0/1167409386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634abb1e640 msgr2=0x5634abb1ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:19.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.445+0000 7fc0b4caab80 1 --2- 172.21.15.7:0/1167409386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634abb1e640 0x5634abb1ea20 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fc09400fd30 tx=0x7fc094012000 comp rx=0 tx=0).stop 2024-06-10T01:11:19.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.445+0000 7fc0b4caab80 1 -- 172.21.15.7:0/1167409386 shutdown_connections 2024-06-10T01:11:19.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.445+0000 7fc0b4caab80 1 --2- 172.21.15.7:0/1167409386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634abb18f20 0x5634abb19340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.445+0000 7fc0b4caab80 1 --2- 172.21.15.7:0/1167409386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634abb1e640 0x5634abb1ea20 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.445+0000 7fc0b4caab80 1 --2- 172.21.15.7:0/1167409386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634abb1ef60 0x5634abb189e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.445+0000 7fc0b4caab80 1 -- 172.21.15.7:0/1167409386 >> 172.21.15.7:0/1167409386 conn(0x5634aba43cc0 msgr2=0x5634abb13850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:19.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.445+0000 7fc0b4caab80 1 -- 172.21.15.7:0/1167409386 shutdown_connections 2024-06-10T01:11:19.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.445+0000 7fc0b4caab80 1 -- 172.21.15.7:0/1167409386 wait complete. 2024-06-10T01:11:19.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.445+0000 7fc0b4caab80 1 -- start start 2024-06-10T01:11:19.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.445+0000 7fc0b4caab80 1 Processor -- start 2024-06-10T01:11:19.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.446+0000 7fc0b4caab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634abb18f20 0x5634abb24ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:19.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.446+0000 7fc0b4caab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634abb1e640 0x5634abb21fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:19.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.446+0000 7fc0b4caab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634abb1ef60 0x5634abb224e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:19.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.446+0000 7fc0b4caab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5634abb259d0 con 0x5634abb1ef60 2024-06-10T01:11:19.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.446+0000 7fc0b4caab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5634aba0abf0 con 0x5634abb18f20 2024-06-10T01:11:19.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.446+0000 7fc0b4caab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5634abb25b50 con 0x5634abb1e640 2024-06-10T01:11:19.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.446+0000 7fc0b4caab80 10 client.0 _ll_register_callbacks cb 0x5634aba04920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:11:19.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.446+0000 7fc0a89c4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634abb1ef60 0x5634abb224e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:19.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.446+0000 7fc0a3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634abb18f20 0x5634abb24ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:19.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.446+0000 7fc0a89c4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634abb1ef60 0x5634abb224e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47700/0 (socket says 172.21.15.7:47700) 2024-06-10T01:11:19.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.446+0000 7fc0a89c4700 1 -- 172.21.15.7:0/950738745 learned_addr learned my addr 172.21.15.7:0/950738745 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:19.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.446+0000 7fc0a37fe700 1 --2- 172.21.15.7:0/950738745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634abb1e640 0x5634abb21fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:19.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.446+0000 7fc0a3fff700 1 -- 172.21.15.7:0/950738745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634abb1e640 msgr2=0x5634abb21fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:19.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.446+0000 7fc0a3fff700 1 --2- 172.21.15.7:0/950738745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634abb1e640 0x5634abb21fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.446+0000 7fc0a3fff700 1 -- 172.21.15.7:0/950738745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634abb1ef60 msgr2=0x5634abb224e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:19.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.446+0000 7fc0a3fff700 1 --2- 172.21.15.7:0/950738745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634abb1ef60 0x5634abb224e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.446+0000 7fc0a3fff700 1 -- 172.21.15.7:0/950738745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634ab9b98d0 con 0x5634abb18f20 2024-06-10T01:11:19.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.447+0000 7fc0a89c4700 1 --2- 172.21.15.7:0/950738745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634abb1ef60 0x5634abb224e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:11:19.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.447+0000 7fc0a3fff700 1 --2- 172.21.15.7:0/950738745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634abb18f20 0x5634abb24ca0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc09400fd00 tx=0x7fc0940256a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e192478c0ee90bfc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:19.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.447+0000 7fc0837fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:11:19.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.447+0000 7fc0837fe700 1 -- 172.21.15.7:0/950738745 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc094033060 con 0x5634abb18f20 2024-06-10T01:11:19.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.447+0000 7fc0837fe700 1 -- 172.21.15.7:0/950738745 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc09401c420 con 0x5634abb18f20 2024-06-10T01:11:19.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.447+0000 7fc0b4caab80 1 -- 172.21.15.7:0/950738745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc094010da0 con 0x5634abb18f20 2024-06-10T01:11:19.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.447+0000 7fc0837fe700 1 -- 172.21.15.7:0/950738745 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc09403a050 con 0x5634abb18f20 2024-06-10T01:11:19.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.447+0000 7fc0b4caab80 10 client.26288 fetch_fsmap learned FSMap version 32 2024-06-10T01:11:19.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.447+0000 7fc0b4caab80 1 -- 172.21.15.7:0/950738745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5634abb30250 con 0x5634abb18f20 2024-06-10T01:11:19.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.448+0000 7fc0837fe700 1 -- 172.21.15.7:0/950738745 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc094010da0 con 0x5634abb18f20 2024-06-10T01:11:19.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.448+0000 7fc0837fe700 1 -- 172.21.15.7:0/950738745 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fc09401b7b0 con 0x5634abb18f20 2024-06-10T01:11:19.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.448+0000 7fc0b4caab80 10 client.26288 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:11:19.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.448+0000 7fc0b4caab80 10 client.26288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:11:19.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.448+0000 7fc0b4caab80 20 client.26288 populate_metadata read hostname 'smithi007' 2024-06-10T01:11:19.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.448+0000 7fc0b4caab80 1 --2- 172.21.15.7:0/950738745 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5634abb327d0 0x5634abb34bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:19.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.448+0000 7fc0b4caab80 4 client.26288 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:11:19.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.448+0000 7fc0b4caab80 1 -- 172.21.15.7:0/950738745 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634aba77d10 con 0x5634abb327d0 2024-06-10T01:11:19.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.448+0000 7fc0a37fe700 1 --2- 172.21.15.7:0/950738745 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5634abb327d0 0x5634abb34bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:19.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.448+0000 7fc0837fe700 10 client.26288 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:19.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.448+0000 7fc0a37fe700 1 --2- 172.21.15.7:0/950738745 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5634abb327d0 0x5634abb34bc0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7032785bc22f99ed server_cookie=624a412568ce70c2 in_seq=0 out_seq=0 2024-06-10T01:11:19.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.449+0000 7fc0837fe700 1 -- 172.21.15.7:0/950738745 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5634aba77d10 con 0x5634abb327d0 2024-06-10T01:11:19.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.449+0000 7fc0837fe700 10 client.26288 handle_command_reply: tid=0 2024-06-10T01:11:19.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.449+0000 7fc0b4caab80 1 client.26288 shutdown 2024-06-10T01:11:19.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.449+0000 7fc0b4caab80 1 -- 172.21.15.7:0/950738745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634abb18f20 msgr2=0x5634abb24ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:19.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.449+0000 7fc0b4caab80 1 --2- 172.21.15.7:0/950738745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634abb18f20 0x5634abb24ca0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc09400fd00 tx=0x7fc0940256a0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.449+0000 7fc0b4caab80 1 -- 172.21.15.7:0/950738745 shutdown_connections 2024-06-10T01:11:19.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.449+0000 7fc0b4caab80 1 --2- 172.21.15.7:0/950738745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634abb1ef60 0x5634abb224e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.450+0000 7fc0b4caab80 1 --2- 172.21.15.7:0/950738745 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5634abb327d0 0x5634abb34bc0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.450+0000 7fc0b4caab80 1 --2- 172.21.15.7:0/950738745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634abb18f20 0x5634abb24ca0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.450+0000 7fc0837fe700 0 client.26288 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:19.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.450+0000 7fc0b4caab80 1 --2- 172.21.15.7:0/950738745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634abb1e640 0x5634abb21fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.450+0000 7fc0b4caab80 1 -- 172.21.15.7:0/950738745 >> 172.21.15.7:0/950738745 conn(0x5634aba43cc0 msgr2=0x5634abb179d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:19.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.450+0000 7fc0b4caab80 1 -- 172.21.15.7:0/950738745 shutdown_connections 2024-06-10T01:11:19.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.450+0000 7fc0b4caab80 1 -- 172.21.15.7:0/950738745 wait complete. 2024-06-10T01:11:19.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.450+0000 7fc0b4caab80 20 client.26288 trim_cache size 0 max 16384 2024-06-10T01:11:19.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.470+0000 7fc0b4caab80 1 Processor -- start 2024-06-10T01:11:19.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.471+0000 7fc0b4caab80 1 -- start start 2024-06-10T01:11:19.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.471+0000 7fc0b4caab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634abb18f20 0x5634abb189e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:19.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.471+0000 7fc0b4caab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634abb1e640 0x5634abb0d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:19.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.471+0000 7fc0b4caab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634abb1ef60 0x5634abb3a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:19.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.471+0000 7fc0b4caab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a41b0bd0 con 0x5634abb1ef60 2024-06-10T01:11:19.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.471+0000 7fc0b4caab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a41ae460 con 0x5634abb18f20 2024-06-10T01:11:19.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.471+0000 7fc0b4caab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a41ac2b0 con 0x5634abb1e640 2024-06-10T01:11:19.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.471+0000 7fc0a89c4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634abb1ef60 0x5634abb3a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:19.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.471+0000 7fc0a89c4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634abb1ef60 0x5634abb3a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47714/0 (socket says 172.21.15.7:47714) 2024-06-10T01:11:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.471+0000 7fc0a89c4700 1 -- 172.21.15.7:0/4163065926 learned_addr learned my addr 172.21.15.7:0/4163065926 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.471+0000 7fc0a3fff700 1 --2- 172.21.15.7:0/4163065926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634abb18f20 0x5634abb189e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.471+0000 7fc0a37fe700 1 --2- 172.21.15.7:0/4163065926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634abb1e640 0x5634abb0d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.471+0000 7fc0a89c4700 1 -- 172.21.15.7:0/4163065926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634abb1e640 msgr2=0x5634abb0d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.471+0000 7fc0a89c4700 1 --2- 172.21.15.7:0/4163065926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634abb1e640 0x5634abb0d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.471+0000 7fc0a89c4700 1 -- 172.21.15.7:0/4163065926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634abb18f20 msgr2=0x5634abb189e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.471+0000 7fc0a89c4700 1 --2- 172.21.15.7:0/4163065926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634abb18f20 0x5634abb189e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.471+0000 7fc0a89c4700 1 -- 172.21.15.7:0/4163065926 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634abb20510 con 0x5634abb1ef60 2024-06-10T01:11:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.472+0000 7fc0a89c4700 1 --2- 172.21.15.7:0/4163065926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634abb1ef60 0x5634abb3a220 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009480 tx=0x7fc09c00b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46edd5b0351a243a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.472+0000 7fc089ffb700 1 -- 172.21.15.7:0/4163065926 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c003c00 con 0x5634abb1ef60 2024-06-10T01:11:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.472+0000 7fc089ffb700 1 -- 172.21.15.7:0/4163065926 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc09c021030 con 0x5634abb1ef60 2024-06-10T01:11:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.472+0000 7fc089ffb700 1 -- 172.21.15.7:0/4163065926 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c005470 con 0x5634abb1ef60 2024-06-10T01:11:19.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.472+0000 7fc0b4caab80 1 -- 172.21.15.7:0/4163065926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634abb1ef60 msgr2=0x5634abb3a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:19.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.472+0000 7fc0b4caab80 1 --2- 172.21.15.7:0/4163065926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634abb1ef60 0x5634abb3a220 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009480 tx=0x7fc09c00b7f0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.472+0000 7fc0b4caab80 1 -- 172.21.15.7:0/4163065926 shutdown_connections 2024-06-10T01:11:19.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.472+0000 7fc0b4caab80 1 --2- 172.21.15.7:0/4163065926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634abb1ef60 0x5634abb3a220 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.472+0000 7fc0b4caab80 1 --2- 172.21.15.7:0/4163065926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634abb18f20 0x5634abb189e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.472+0000 7fc0b4caab80 1 --2- 172.21.15.7:0/4163065926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634abb1e640 0x5634abb0d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.472+0000 7fc0b4caab80 1 -- 172.21.15.7:0/4163065926 >> 172.21.15.7:0/4163065926 conn(0x5634aba43cc0 msgr2=0x5634abb1f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:19.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.473+0000 7fc0b4caab80 1 -- 172.21.15.7:0/4163065926 shutdown_connections 2024-06-10T01:11:19.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.473+0000 7fc0b4caab80 1 -- 172.21.15.7:0/4163065926 wait complete. 2024-06-10T01:11:19.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.473+0000 7fc0b4caab80 1 -- start start 2024-06-10T01:11:19.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.473+0000 7fc0b4caab80 1 Processor -- start 2024-06-10T01:11:19.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.473+0000 7fc0b4caab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634abb18f20 0x5634abb2b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:19.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.473+0000 7fc0b4caab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634abb1e640 0x5634abb269d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:19.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.473+0000 7fc0b4caab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634abb1ef60 0x5634abb26f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:19.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.473+0000 7fc0b4caab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a41ac920 con 0x5634abb1e640 2024-06-10T01:11:19.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.473+0000 7fc0b4caab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a41add90 con 0x5634abb1ef60 2024-06-10T01:11:19.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.473+0000 7fc0b4caab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a41acff0 con 0x5634abb18f20 2024-06-10T01:11:19.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.473+0000 7fc0b4caab80 10 client.0 _ll_register_callbacks cb 0x5634aba04920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:11:19.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.473+0000 7fc0a37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634abb1e640 0x5634abb269d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:19.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.473+0000 7fc0a37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634abb1e640 0x5634abb269d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47724/0 (socket says 172.21.15.7:47724) 2024-06-10T01:11:19.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.474+0000 7fc0a37fe700 1 -- 172.21.15.7:0/96974142 learned_addr learned my addr 172.21.15.7:0/96974142 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:19.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.474+0000 7fc0a3fff700 1 --2- 172.21.15.7:0/96974142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634abb18f20 0x5634abb2b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:19.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.474+0000 7fc0a89c4700 1 --2- 172.21.15.7:0/96974142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634abb1ef60 0x5634abb26f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:19.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.474+0000 7fc0a37fe700 1 -- 172.21.15.7:0/96974142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634abb18f20 msgr2=0x5634abb2b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:19.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.474+0000 7fc0a37fe700 1 --2- 172.21.15.7:0/96974142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634abb18f20 0x5634abb2b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.474+0000 7fc0a37fe700 1 -- 172.21.15.7:0/96974142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634abb1ef60 msgr2=0x5634abb26f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:19.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.474+0000 7fc0a37fe700 1 --2- 172.21.15.7:0/96974142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634abb1ef60 0x5634abb26f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.474+0000 7fc0a37fe700 1 -- 172.21.15.7:0/96974142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634abb277d0 con 0x5634abb1e640 2024-06-10T01:11:19.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.474+0000 7fc0a37fe700 1 --2- 172.21.15.7:0/96974142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634abb1e640 0x5634abb269d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fc09800b2f0 tx=0x7fc0980bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2b68f44f41931fc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:19.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.474+0000 7fc0837fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:11:19.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.474+0000 7fc0837fe700 1 -- 172.21.15.7:0/96974142 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098014030 con 0x5634abb1e640 2024-06-10T01:11:19.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.474+0000 7fc0837fe700 1 -- 172.21.15.7:0/96974142 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc098010460 con 0x5634abb1e640 2024-06-10T01:11:19.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.474+0000 7fc0b4caab80 1 -- 172.21.15.7:0/96974142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc09c021030 con 0x5634abb1e640 2024-06-10T01:11:19.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.474+0000 7fc0837fe700 1 -- 172.21.15.7:0/96974142 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc098020040 con 0x5634abb1e640 2024-06-10T01:11:19.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.474+0000 7fc0b4caab80 10 client.16839 fetch_fsmap learned FSMap version 32 2024-06-10T01:11:19.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.474+0000 7fc0b4caab80 1 -- 172.21.15.7:0/96974142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5634abba1770 con 0x5634abb1e640 2024-06-10T01:11:19.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.475+0000 7fc0837fe700 1 -- 172.21.15.7:0/96974142 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc09c021030 con 0x5634abb1e640 2024-06-10T01:11:19.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.475+0000 7fc0837fe700 1 -- 172.21.15.7:0/96974142 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fc0980a3b30 con 0x5634abb1e640 2024-06-10T01:11:19.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.475+0000 7fc0b4caab80 10 client.16839 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:11:19.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.475+0000 7fc0b4caab80 10 client.16839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:11:19.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.475+0000 7fc0b4caab80 20 client.16839 populate_metadata read hostname 'smithi007' 2024-06-10T01:11:19.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.475+0000 7fc0b4caab80 1 --2- 172.21.15.7:0/96974142 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5634abb327d0 0x5634abba3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:19.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.475+0000 7fc0b4caab80 4 client.16839 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:11:19.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.475+0000 7fc0b4caab80 1 -- 172.21.15.7:0/96974142 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5634aba72290 con 0x5634abb327d0 2024-06-10T01:11:19.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.475+0000 7fc0a3fff700 1 --2- 172.21.15.7:0/96974142 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5634abb327d0 0x5634abba3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:19.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.475+0000 7fc0a3fff700 1 --2- 172.21.15.7:0/96974142 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5634abb327d0 0x5634abba3e70 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdc3c8a2aa468a06 server_cookie=6239003a0f996bc3 in_seq=0 out_seq=0 2024-06-10T01:11:19.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.475+0000 7fc0837fe700 10 client.16839 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:19.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.476+0000 7fc0837fe700 1 -- 172.21.15.7:0/96974142 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5634aba72290 con 0x5634abb327d0 2024-06-10T01:11:19.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.476+0000 7fc0837fe700 10 client.16839 handle_command_reply: tid=0 2024-06-10T01:11:19.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.476+0000 7fc0b4caab80 1 client.16839 shutdown 2024-06-10T01:11:19.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.476+0000 7fc0b4caab80 1 -- 172.21.15.7:0/96974142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634abb1e640 msgr2=0x5634abb269d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:19.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.476+0000 7fc0b4caab80 1 --2- 172.21.15.7:0/96974142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634abb1e640 0x5634abb269d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fc09800b2f0 tx=0x7fc0980bb040 comp rx=0 tx=0).stop 2024-06-10T01:11:19.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.476+0000 7fc0b4caab80 1 -- 172.21.15.7:0/96974142 shutdown_connections 2024-06-10T01:11:19.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.476+0000 7fc0b4caab80 1 --2- 172.21.15.7:0/96974142 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5634abb327d0 0x5634abba3e70 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.476+0000 7fc0b4caab80 1 --2- 172.21.15.7:0/96974142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634abb1ef60 0x5634abb26f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.476+0000 7fc0b4caab80 1 --2- 172.21.15.7:0/96974142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5634abb18f20 0x5634abb2b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.476+0000 7fc0b4caab80 1 --2- 172.21.15.7:0/96974142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634abb1e640 0x5634abb269d0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.476+0000 7fc0b4caab80 1 -- 172.21.15.7:0/96974142 >> 172.21.15.7:0/96974142 conn(0x5634aba43cc0 msgr2=0x5634abb1d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:19.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.476+0000 7fc0837fe700 0 client.16839 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:19.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.476+0000 7fc0b4caab80 1 -- 172.21.15.7:0/96974142 shutdown_connections 2024-06-10T01:11:19.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.476+0000 7fc0b4caab80 1 -- 172.21.15.7:0/96974142 wait complete. 2024-06-10T01:11:19.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.476+0000 7fc0b4caab80 20 client.16839 trim_cache size 0 max 16384 2024-06-10T01:11:19.476 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:11:19.476 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T01:11:19.477 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-10T01:11:19.477 INFO:teuthology.orchestra.run.smithi007.stdout: "2380d01b-750e-4985-88c1-5e8d2c6d4c42": { 2024-06-10T01:11:19.477 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-10T01:11:19.477 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "2380d01b-750e-4985-88c1-5e8d2c6d4c42", 2024-06-10T01:11:19.477 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-10T01:11:19.477 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:11:19.477 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:11:19.477 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:11:19.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.476+0000 7fc0aa427700 1 -- 172.21.15.7:0/556556661 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc08c0a2c20 msgr2=0x7fc08c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:19.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.476+0000 7fc0aa427700 1 --2- 172.21.15.7:0/556556661 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc08c0a2c20 0x7fc08c0a50e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc094005950 tx=0x7fc09400a300 comp rx=0 tx=0).stop 2024-06-10T01:11:19.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.477+0000 7fc0aa427700 1 -- 172.21.15.7:0/556556661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0a4101d80 msgr2=0x7fc0a419ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:19.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.477+0000 7fc0aa427700 1 --2- 172.21.15.7:0/556556661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0a4101d80 0x7fc0a419ecf0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fc09800d950 tx=0x7fc09800dc60 comp rx=0 tx=0).stop 2024-06-10T01:11:19.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.477+0000 7fc0aa427700 1 -- 172.21.15.7:0/556556661 shutdown_connections 2024-06-10T01:11:19.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.477+0000 7fc0aa427700 1 --2- 172.21.15.7:0/556556661 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc08c0a2c20 0x7fc08c0a50e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.477+0000 7fc0aa427700 1 --2- 172.21.15.7:0/556556661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0a4107440 0x7fc0a4198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.477+0000 7fc0aa427700 1 --2- 172.21.15.7:0/556556661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0a41013c0 0x7fc0a419e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.477+0000 7fc0aa427700 1 --2- 172.21.15.7:0/556556661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0a4101d80 0x7fc0a419ecf0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:19.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.477+0000 7fc0aa427700 1 -- 172.21.15.7:0/556556661 >> 172.21.15.7:0/556556661 conn(0x7fc0a4077400 msgr2=0x7fc0a406ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:19.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.477+0000 7fc0aa427700 1 -- 172.21.15.7:0/556556661 shutdown_connections 2024-06-10T01:11:19.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:19.477+0000 7fc0aa427700 1 -- 172.21.15.7:0/556556661 wait complete. 2024-06-10T01:11:19.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:19 smithi175 ceph-mon[36636]: pgmap v591: 129 pgs: 129 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.2 MiB/s wr, 98 op/s 2024-06-10T01:11:19.730 INFO:tasks.workunit.client.0.smithi007.stdout: 1 345759 7.69 MB/sec execute 120 sec latency 19677.046 ms 2024-06-10T01:11:19.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:19 smithi078 ceph-mon[36683]: pgmap v591: 129 pgs: 129 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.2 MiB/s wr, 98 op/s 2024-06-10T01:11:20.125 INFO:tasks.cephfs.filesystem:scrub status for tag:2380d01b-750e-4985-88c1-5e8d2c6d4c42 - {'path': '/', 'tag': '2380d01b-750e-4985-88c1-5e8d2c6d4c42', 'options': 'recursive,force'} 2024-06-10T01:11:20.730 INFO:tasks.workunit.client.0.smithi007.stdout: 1 345759 7.63 MB/sec execute 121 sec latency 20677.125 ms 2024-06-10T01:11:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.997+0000 7f72a957e640 1 -- 172.21.15.7:0/2297352051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72a4104db0 msgr2=0x7f72a41071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.997+0000 7f72a957e640 1 --2- 172.21.15.7:0/2297352051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72a4104db0 0x7f72a41071a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f7298009900 tx=0x7f729802a410 comp rx=0 tx=0).stop 2024-06-10T01:11:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.997+0000 7f72a957e640 1 -- 172.21.15.7:0/2297352051 shutdown_connections 2024-06-10T01:11:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.997+0000 7f72a957e640 1 --2- 172.21.15.7:0/2297352051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72a41076e0 0x7f72a4109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.997+0000 7f72a957e640 1 --2- 172.21.15.7:0/2297352051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72a4104db0 0x7f72a41071a0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.997+0000 7f72a957e640 1 --2- 172.21.15.7:0/2297352051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72a4102480 0x7f72a4104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.997+0000 7f72a957e640 1 -- 172.21.15.7:0/2297352051 >> 172.21.15.7:0/2297352051 conn(0x7f72a40fbe70 msgr2=0x7f72a40fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.997+0000 7f72a957e640 1 -- 172.21.15.7:0/2297352051 shutdown_connections 2024-06-10T01:11:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.997+0000 7f72a957e640 1 -- 172.21.15.7:0/2297352051 wait complete. 2024-06-10T01:11:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.998+0000 7f72a957e640 1 Processor -- start 2024-06-10T01:11:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.998+0000 7f72a957e640 1 -- start start 2024-06-10T01:11:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.998+0000 7f72a957e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72a4102480 0x7f72a4197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.998+0000 7f72a957e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72a4104db0 0x7f72a4198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.998+0000 7f72a957e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72a41076e0 0x7f72a419f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.998+0000 7f72a957e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f72a410d680 con 0x7f72a41076e0 2024-06-10T01:11:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.998+0000 7f72a957e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f72a410d500 con 0x7f72a4104db0 2024-06-10T01:11:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.998+0000 7f72a957e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f72a410d800 con 0x7f72a4102480 2024-06-10T01:11:20.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.999+0000 7f72a37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72a41076e0 0x7f72a419f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:20.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.999+0000 7f72a37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72a41076e0 0x7f72a419f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47748/0 (socket says 172.21.15.7:47748) 2024-06-10T01:11:20.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.999+0000 7f72a37fe640 1 -- 172.21.15.7:0/911519707 learned_addr learned my addr 172.21.15.7:0/911519707 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:20.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.999+0000 7f72a27fc640 1 --2- 172.21.15.7:0/911519707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72a4104db0 0x7f72a4198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:20.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.999+0000 7f72a2ffd640 1 --2- 172.21.15.7:0/911519707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72a4102480 0x7f72a4197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:20.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.999+0000 7f72a37fe640 1 -- 172.21.15.7:0/911519707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72a4102480 msgr2=0x7f72a4197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:20.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.999+0000 7f72a37fe640 1 --2- 172.21.15.7:0/911519707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72a4102480 0x7f72a4197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:20.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.999+0000 7f72a37fe640 1 -- 172.21.15.7:0/911519707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72a4104db0 msgr2=0x7f72a4198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:20.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.999+0000 7f72a37fe640 1 --2- 172.21.15.7:0/911519707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72a4104db0 0x7f72a4198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:20.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.999+0000 7f72a37fe640 1 -- 172.21.15.7:0/911519707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a419f970 con 0x7f72a41076e0 2024-06-10T01:11:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.999+0000 7f72a2ffd640 1 --2- 172.21.15.7:0/911519707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72a4102480 0x7f72a4197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:11:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.999+0000 7f72a37fe640 1 --2- 172.21.15.7:0/911519707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72a41076e0 0x7f72a419f300 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f729c00d750 tx=0x7f729c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.999+0000 7f728bfff640 1 -- 172.21.15.7:0/911519707 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c017070 con 0x7f72a41076e0 2024-06-10T01:11:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.999+0000 7f72a957e640 1 -- 172.21.15.7:0/911519707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a419fc60 con 0x7f72a41076e0 2024-06-10T01:11:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.999+0000 7f728bfff640 1 -- 172.21.15.7:0/911519707 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f729c005500 con 0x7f72a41076e0 2024-06-10T01:11:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.999+0000 7f728bfff640 1 -- 172.21.15.7:0/911519707 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c004d90 con 0x7f72a41076e0 2024-06-10T01:11:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:20.999+0000 7f72a957e640 1 -- 172.21.15.7:0/911519707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a41a0180 con 0x7f72a41076e0 2024-06-10T01:11:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.002+0000 7f728bfff640 1 -- 172.21.15.7:0/911519707 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f729c004f30 con 0x7f72a41076e0 2024-06-10T01:11:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.002+0000 7f72ac36f740 1 Processor -- start 2024-06-10T01:11:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.002+0000 7f72ac36f740 1 -- start start 2024-06-10T01:11:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.002+0000 7f72ac36f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcbec0010 0x560bcbec0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.002+0000 7f72ac36f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcbec0a00 0x560bcbec5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.002+0000 7f72ac36f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcbec5b70 0x560bcbec8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.002+0000 7f72ac36f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560bcbe234f0 con 0x560bcbec0a00 2024-06-10T01:11:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.002+0000 7f72ac36f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560bcbdc1870 con 0x560bcbec0010 2024-06-10T01:11:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.002+0000 7f72ac36f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560bcbe1f4b0 con 0x560bcbec5b70 2024-06-10T01:11:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.003+0000 7f72a2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcbec0a00 0x560bcbec5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.003+0000 7f72a2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcbec0a00 0x560bcbec5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47756/0 (socket says 172.21.15.7:47756) 2024-06-10T01:11:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.003+0000 7f72a2ffd640 1 -- 172.21.15.7:0/1718643923 learned_addr learned my addr 172.21.15.7:0/1718643923 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.003+0000 7f72a27fc640 1 --2- 172.21.15.7:0/1718643923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcbec0010 0x560bcbec0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.003+0000 7f72a27fc640 1 --2- 172.21.15.7:0/1718643923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcbec5b70 0x560bcbec8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.003+0000 7f728bfff640 1 --2- 172.21.15.7:0/911519707 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f728c0a2b80 0x7f728c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.003+0000 7f72a2ffd640 1 -- 172.21.15.7:0/1718643923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcbec5b70 msgr2=0x560bcbec8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.003+0000 7f72a2ffd640 1 --2- 172.21.15.7:0/1718643923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcbec5b70 0x560bcbec8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.003+0000 7f72a2ffd640 1 -- 172.21.15.7:0/1718643923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcbec0010 msgr2=0x560bcbec0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.003+0000 7f72a2ffd640 1 --2- 172.21.15.7:0/1718643923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcbec0010 0x560bcbec0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.003+0000 7f72a2ffd640 1 -- 172.21.15.7:0/1718643923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bcbddd9b0 con 0x560bcbec0a00 2024-06-10T01:11:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.003+0000 7f72a27fc640 1 --2- 172.21.15.7:0/1718643923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcbec0010 0x560bcbec0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:11:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.003+0000 7f72a37fe640 1 --2- 172.21.15.7:0/911519707 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f728c0a2b80 0x7f728c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.003+0000 7f72a2ffd640 1 --2- 172.21.15.7:0/1718643923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcbec0a00 0x560bcbec5500 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f7294010550 tx=0x7f7294010a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f422de6e69eaa400 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.003+0000 7f728bfff640 1 -- 172.21.15.7:0/911519707 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f729c0ced90 con 0x7f72a41076e0 2024-06-10T01:11:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.003+0000 7f72897fa640 1 -- 172.21.15.7:0/1718643923 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7294017070 con 0x560bcbec0a00 2024-06-10T01:11:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.003+0000 7f72897fa640 1 -- 172.21.15.7:0/1718643923 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7294011c40 con 0x560bcbec0a00 2024-06-10T01:11:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.003+0000 7f72897fa640 1 -- 172.21.15.7:0/1718643923 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729400edf0 con 0x560bcbec0a00 2024-06-10T01:11:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.004+0000 7f72ac36f740 1 -- 172.21.15.7:0/1718643923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcbec0a00 msgr2=0x560bcbec5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.004+0000 7f72ac36f740 1 --2- 172.21.15.7:0/1718643923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcbec0a00 0x560bcbec5500 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f7294010550 tx=0x7f7294010a10 comp rx=0 tx=0).stop 2024-06-10T01:11:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.004+0000 7f72a37fe640 1 --2- 172.21.15.7:0/911519707 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f728c0a2b80 0x7f728c0a5040 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f729c0976d0 tx=0x7f729c009bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.004+0000 7f72ac36f740 1 -- 172.21.15.7:0/1718643923 shutdown_connections 2024-06-10T01:11:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.004+0000 7f72ac36f740 1 --2- 172.21.15.7:0/1718643923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcbec5b70 0x560bcbec8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.004+0000 7f72ac36f740 1 --2- 172.21.15.7:0/1718643923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcbec0a00 0x560bcbec5500 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.004+0000 7f72ac36f740 1 --2- 172.21.15.7:0/1718643923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcbec0010 0x560bcbec0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.004+0000 7f72ac36f740 1 -- 172.21.15.7:0/1718643923 >> 172.21.15.7:0/1718643923 conn(0x560bcbebb680 msgr2=0x560bcbebba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.004+0000 7f72ac36f740 1 -- 172.21.15.7:0/1718643923 shutdown_connections 2024-06-10T01:11:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.004+0000 7f72ac36f740 1 -- 172.21.15.7:0/1718643923 wait complete. 2024-06-10T01:11:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.005+0000 7f72ac36f740 1 -- start start 2024-06-10T01:11:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.005+0000 7f72ac36f740 1 Processor -- start 2024-06-10T01:11:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.005+0000 7f72ac36f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcbec0010 0x560bcbeca220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.005+0000 7f72ac36f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcbec0a00 0x560bcbeca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.005+0000 7f72ac36f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcbec5b70 0x560bcbed17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.005+0000 7f72ac36f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560bcbec8b20 con 0x560bcbec0010 2024-06-10T01:11:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.005+0000 7f72ac36f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560bcbdacae0 con 0x560bcbec5b70 2024-06-10T01:11:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.005+0000 7f72ac36f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560bcbec8ca0 con 0x560bcbec0a00 2024-06-10T01:11:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.005+0000 7f72ac36f740 10 client.0 _ll_register_callbacks cb 0x560bcbde59b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:11:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.005+0000 7f72a27fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcbec0010 0x560bcbeca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.005+0000 7f72a27fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcbec0010 0x560bcbeca220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47772/0 (socket says 172.21.15.7:47772) 2024-06-10T01:11:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.005+0000 7f72a27fc640 1 -- 172.21.15.7:0/477927248 learned_addr learned my addr 172.21.15.7:0/477927248 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.005+0000 7f72a2ffd640 1 --2- 172.21.15.7:0/477927248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcbec0a00 0x560bcbeca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.005+0000 7f72a27fc640 1 --2- 172.21.15.7:0/477927248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcbec5b70 0x560bcbed17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.006+0000 7f72a27fc640 1 -- 172.21.15.7:0/477927248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcbec0a00 msgr2=0x560bcbeca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.006+0000 7f72a27fc640 1 --2- 172.21.15.7:0/477927248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcbec0a00 0x560bcbeca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.006+0000 7f72a27fc640 1 -- 172.21.15.7:0/477927248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcbec5b70 msgr2=0x560bcbed17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.006+0000 7f72a27fc640 1 --2- 172.21.15.7:0/477927248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcbec5b70 0x560bcbed17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.006+0000 7f72a27fc640 1 -- 172.21.15.7:0/477927248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bcbb37210 con 0x560bcbec0010 2024-06-10T01:11:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.006+0000 7f72a27fc640 1 --2- 172.21.15.7:0/477927248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcbec0010 0x560bcbeca220 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f72a4069d20 tx=0x7f729802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51d30b4aa5f283b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.006+0000 7f7286ff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:11:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.006+0000 7f7286ff5640 1 -- 172.21.15.7:0/477927248 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7298035070 con 0x560bcbec0010 2024-06-10T01:11:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.006+0000 7f7286ff5640 1 -- 172.21.15.7:0/477927248 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f7298005450 con 0x560bcbec0010 2024-06-10T01:11:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.006+0000 7f72ac36f740 1 -- 172.21.15.7:0/477927248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7294011c40 con 0x560bcbec0010 2024-06-10T01:11:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.006+0000 7f7286ff5640 1 -- 172.21.15.7:0/477927248 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7298050050 con 0x560bcbec0010 2024-06-10T01:11:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.006+0000 7f72ac36f740 10 client.16851 fetch_fsmap learned FSMap version 32 2024-06-10T01:11:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.006+0000 7f72ac36f740 1 -- 172.21.15.7:0/477927248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560bcbc03620 con 0x560bcbec0010 2024-06-10T01:11:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.006+0000 7f7286ff5640 1 -- 172.21.15.7:0/477927248 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7294011c40 con 0x560bcbec0010 2024-06-10T01:11:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.007+0000 7f7286ff5640 1 -- 172.21.15.7:0/477927248 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f7298004cd0 con 0x560bcbec0010 2024-06-10T01:11:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.007+0000 7f72ac36f740 10 client.16851 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:11:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.007+0000 7f72ac36f740 10 client.16851 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:11:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.007+0000 7f72ac36f740 20 client.16851 populate_metadata read hostname 'smithi007' 2024-06-10T01:11:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.007+0000 7f72ac36f740 1 --2- 172.21.15.7:0/477927248 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560bcbed60d0 0x560bcbed8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.007+0000 7f72ac36f740 4 client.16851 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:11:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.007+0000 7f72ac36f740 1 -- 172.21.15.7:0/477927248 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bcbe36f10 con 0x560bcbed60d0 2024-06-10T01:11:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.007+0000 7f72a2ffd640 1 --2- 172.21.15.7:0/477927248 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560bcbed60d0 0x560bcbed8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.007+0000 7f7286ff5640 10 client.16851 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.007+0000 7f72a2ffd640 1 --2- 172.21.15.7:0/477927248 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560bcbed60d0 0x560bcbed8500 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48d94632bdbb736b server_cookie=f3f10b905c9c15a1 in_seq=0 out_seq=0 2024-06-10T01:11:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.008+0000 7f7286ff5640 1 -- 172.21.15.7:0/477927248 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560bcbe36f10 con 0x560bcbed60d0 2024-06-10T01:11:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.008+0000 7f7286ff5640 10 client.16851 handle_command_reply: tid=0 2024-06-10T01:11:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.008+0000 7f72ac36f740 1 client.16851 shutdown 2024-06-10T01:11:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.008+0000 7f72ac36f740 1 -- 172.21.15.7:0/477927248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcbec0010 msgr2=0x560bcbeca220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.008+0000 7f72ac36f740 1 --2- 172.21.15.7:0/477927248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcbec0010 0x560bcbeca220 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f72a4069d20 tx=0x7f729802a410 comp rx=0 tx=0).stop 2024-06-10T01:11:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.008+0000 7f72ac36f740 1 -- 172.21.15.7:0/477927248 shutdown_connections 2024-06-10T01:11:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.008+0000 7f72ac36f740 1 --2- 172.21.15.7:0/477927248 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560bcbed60d0 0x560bcbed8500 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.008+0000 7f72ac36f740 1 --2- 172.21.15.7:0/477927248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcbec5b70 0x560bcbed17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.008+0000 7f7286ff5640 0 client.16851 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.008+0000 7f72ac36f740 1 --2- 172.21.15.7:0/477927248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcbec0a00 0x560bcbeca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.008+0000 7f72ac36f740 1 --2- 172.21.15.7:0/477927248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcbec0010 0x560bcbeca220 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.008+0000 7f72ac36f740 1 -- 172.21.15.7:0/477927248 >> 172.21.15.7:0/477927248 conn(0x560bcbebb680 msgr2=0x560bcbeba7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.008+0000 7f72ac36f740 1 -- 172.21.15.7:0/477927248 shutdown_connections 2024-06-10T01:11:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.008+0000 7f72ac36f740 1 -- 172.21.15.7:0/477927248 wait complete. 2024-06-10T01:11:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.008+0000 7f72ac36f740 20 client.16851 trim_cache size 0 max 16384 2024-06-10T01:11:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.024+0000 7f72ac36f740 1 Processor -- start 2024-06-10T01:11:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.024+0000 7f72ac36f740 1 -- start start 2024-06-10T01:11:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.024+0000 7f72ac36f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcbec0010 0x560bcbebe670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.024+0000 7f72ac36f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcbec0a00 0x560bcbebebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.024+0000 7f72ac36f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcbec5b70 0x560bcbec9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.024+0000 7f72ac36f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f72a41ace40 con 0x560bcbec0010 2024-06-10T01:11:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.024+0000 7f72ac36f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f72a41aa710 con 0x560bcbec5b70 2024-06-10T01:11:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.024+0000 7f72ac36f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f72a41a8560 con 0x560bcbec0a00 2024-06-10T01:11:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.024+0000 7f72a27fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcbec0010 0x560bcbebe670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.024+0000 7f72a27fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcbec0010 0x560bcbebe670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47784/0 (socket says 172.21.15.7:47784) 2024-06-10T01:11:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.024+0000 7f72a27fc640 1 -- 172.21.15.7:0/3076912296 learned_addr learned my addr 172.21.15.7:0/3076912296 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.025+0000 7f72a2ffd640 1 --2- 172.21.15.7:0/3076912296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcbec0a00 0x560bcbebebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.025+0000 7f72a27fc640 1 --2- 172.21.15.7:0/3076912296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcbec5b70 0x560bcbec9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.025+0000 7f72a27fc640 1 -- 172.21.15.7:0/3076912296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcbec0a00 msgr2=0x560bcbebebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.025+0000 7f72a27fc640 1 --2- 172.21.15.7:0/3076912296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcbec0a00 0x560bcbebebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.025+0000 7f72a27fc640 1 -- 172.21.15.7:0/3076912296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcbec5b70 msgr2=0x560bcbec9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.025+0000 7f72a27fc640 1 --2- 172.21.15.7:0/3076912296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcbec5b70 0x560bcbec9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.025+0000 7f72a27fc640 1 -- 172.21.15.7:0/3076912296 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bcbec16d0 con 0x560bcbec0010 2024-06-10T01:11:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.025+0000 7f72a2ffd640 1 --2- 172.21.15.7:0/3076912296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcbec0a00 0x560bcbebebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:11:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.025+0000 7f72a27fc640 1 --2- 172.21.15.7:0/3076912296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcbec0010 0x560bcbebe670 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f7298031440 tx=0x7f7298005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c5675ddcf75b37 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.025+0000 7f7288ff9640 1 -- 172.21.15.7:0/3076912296 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729805d070 con 0x560bcbec0010 2024-06-10T01:11:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.025+0000 7f7288ff9640 1 -- 172.21.15.7:0/3076912296 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7298031930 con 0x560bcbec0010 2024-06-10T01:11:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.025+0000 7f7288ff9640 1 -- 172.21.15.7:0/3076912296 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7298035070 con 0x560bcbec0010 2024-06-10T01:11:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.026+0000 7f72ac36f740 1 -- 172.21.15.7:0/3076912296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcbec0010 msgr2=0x560bcbebe670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.026+0000 7f72ac36f740 1 --2- 172.21.15.7:0/3076912296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcbec0010 0x560bcbebe670 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f7298031440 tx=0x7f7298005fd0 comp rx=0 tx=0).stop 2024-06-10T01:11:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.026+0000 7f72ac36f740 1 -- 172.21.15.7:0/3076912296 shutdown_connections 2024-06-10T01:11:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.026+0000 7f72ac36f740 1 --2- 172.21.15.7:0/3076912296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcbec5b70 0x560bcbec9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.026+0000 7f72ac36f740 1 --2- 172.21.15.7:0/3076912296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcbec0a00 0x560bcbebebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.026+0000 7f72ac36f740 1 --2- 172.21.15.7:0/3076912296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcbec0010 0x560bcbebe670 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.026+0000 7f72ac36f740 1 -- 172.21.15.7:0/3076912296 >> 172.21.15.7:0/3076912296 conn(0x560bcbebb680 msgr2=0x560bcbec1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.026+0000 7f72ac36f740 1 -- 172.21.15.7:0/3076912296 shutdown_connections 2024-06-10T01:11:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.026+0000 7f72ac36f740 1 -- 172.21.15.7:0/3076912296 wait complete. 2024-06-10T01:11:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.026+0000 7f72ac36f740 1 -- start start 2024-06-10T01:11:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.026+0000 7f72ac36f740 1 Processor -- start 2024-06-10T01:11:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.026+0000 7f72ac36f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcbec0010 0x560bcbf3ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.026+0000 7f72ac36f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcbec0a00 0x560bcbf3bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.026+0000 7f72ac36f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcbec5b70 0x560bcbf42f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.026+0000 7f72ac36f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f72a41a8bd0 con 0x560bcbec0a00 2024-06-10T01:11:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.026+0000 7f72ac36f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f72a41aa040 con 0x560bcbec5b70 2024-06-10T01:11:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.026+0000 7f72ac36f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f72a41a92a0 con 0x560bcbec0010 2024-06-10T01:11:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.026+0000 7f72ac36f740 10 client.0 _ll_register_callbacks cb 0x7f7298009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:11:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.027+0000 7f72a2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcbec0a00 0x560bcbf3bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.027+0000 7f72a2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcbec0a00 0x560bcbf3bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47798/0 (socket says 172.21.15.7:47798) 2024-06-10T01:11:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.027+0000 7f72a2ffd640 1 -- 172.21.15.7:0/2687162437 learned_addr learned my addr 172.21.15.7:0/2687162437 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.027+0000 7f72a27fc640 1 --2- 172.21.15.7:0/2687162437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcbec0010 0x560bcbf3ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.027+0000 7f72a27fc640 1 --2- 172.21.15.7:0/2687162437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcbec5b70 0x560bcbf42f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.027+0000 7f72a2ffd640 1 -- 172.21.15.7:0/2687162437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcbec0010 msgr2=0x560bcbf3ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.027+0000 7f72a2ffd640 1 --2- 172.21.15.7:0/2687162437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcbec0010 0x560bcbf3ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.027+0000 7f72a2ffd640 1 -- 172.21.15.7:0/2687162437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcbec5b70 msgr2=0x560bcbf42f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.027+0000 7f72a2ffd640 1 --2- 172.21.15.7:0/2687162437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcbec5b70 0x560bcbf42f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.027+0000 7f72a2ffd640 1 -- 172.21.15.7:0/2687162437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bcbec58c0 con 0x560bcbec0a00 2024-06-10T01:11:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.027+0000 7f72a2ffd640 1 --2- 172.21.15.7:0/2687162437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcbec0a00 0x560bcbf3bf40 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f7294010550 tx=0x7f7294009db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b5fc0fa898787a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.027+0000 7f7286ff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:11:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.027+0000 7f7286ff5640 1 -- 172.21.15.7:0/2687162437 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7294017040 con 0x560bcbec0a00 2024-06-10T01:11:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.027+0000 7f7286ff5640 1 -- 172.21.15.7:0/2687162437 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f729400e3d0 con 0x560bcbec0a00 2024-06-10T01:11:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.027+0000 7f72ac36f740 1 -- 172.21.15.7:0/2687162437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7298031930 con 0x560bcbec0a00 2024-06-10T01:11:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.027+0000 7f7286ff5640 1 -- 172.21.15.7:0/2687162437 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7294037050 con 0x560bcbec0a00 2024-06-10T01:11:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.027+0000 7f72ac36f740 10 client.16857 fetch_fsmap learned FSMap version 32 2024-06-10T01:11:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.027+0000 7f72ac36f740 1 -- 172.21.15.7:0/2687162437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560bcbebb4b0 con 0x560bcbec0a00 2024-06-10T01:11:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.028+0000 7f7286ff5640 1 -- 172.21.15.7:0/2687162437 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7298031930 con 0x560bcbec0a00 2024-06-10T01:11:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.028+0000 7f7286ff5640 1 -- 172.21.15.7:0/2687162437 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f729400e890 con 0x560bcbec0a00 2024-06-10T01:11:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.028+0000 7f72ac36f740 10 client.16857 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:11:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.028+0000 7f72ac36f740 10 client.16857 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:11:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.028+0000 7f72ac36f740 20 client.16857 populate_metadata read hostname 'smithi007' 2024-06-10T01:11:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.028+0000 7f72ac36f740 1 --2- 172.21.15.7:0/2687162437 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560bcbf44c90 0x560bcbf470b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.028+0000 7f72ac36f740 4 client.16857 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:11:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.028+0000 7f72ac36f740 1 -- 172.21.15.7:0/2687162437 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560bcbe30c20 con 0x560bcbf44c90 2024-06-10T01:11:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.028+0000 7f72a27fc640 1 --2- 172.21.15.7:0/2687162437 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560bcbf44c90 0x560bcbf470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.028+0000 7f72a27fc640 1 --2- 172.21.15.7:0/2687162437 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560bcbf44c90 0x560bcbf470b0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e68922aa5064c1bf server_cookie=fba86ed86f6ca85b in_seq=0 out_seq=0 2024-06-10T01:11:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.028+0000 7f7286ff5640 10 client.16857 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.157+0000 7f7286ff5640 1 -- 172.21.15.7:0/2687162437 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560bcbe30c20 con 0x560bcbf44c90 2024-06-10T01:11:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.157+0000 7f7286ff5640 10 client.16857 handle_command_reply: tid=0 2024-06-10T01:11:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.157+0000 7f72ac36f740 1 client.16857 shutdown 2024-06-10T01:11:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.157+0000 7f72ac36f740 1 -- 172.21.15.7:0/2687162437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcbec0a00 msgr2=0x560bcbf3bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.157+0000 7f72ac36f740 1 --2- 172.21.15.7:0/2687162437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcbec0a00 0x560bcbf3bf40 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f7294010550 tx=0x7f7294009db0 comp rx=0 tx=0).stop 2024-06-10T01:11:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.157+0000 7f72ac36f740 1 -- 172.21.15.7:0/2687162437 shutdown_connections 2024-06-10T01:11:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.157+0000 7f72ac36f740 1 --2- 172.21.15.7:0/2687162437 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560bcbf44c90 0x560bcbf470b0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.157+0000 7f72ac36f740 1 --2- 172.21.15.7:0/2687162437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcbec5b70 0x560bcbf42f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.157+0000 7f72ac36f740 1 --2- 172.21.15.7:0/2687162437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcbec0a00 0x560bcbf3bf40 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.157+0000 7f72ac36f740 1 --2- 172.21.15.7:0/2687162437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcbec0010 0x560bcbf3ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.157+0000 7f7286ff5640 0 client.16857 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.157+0000 7f72ac36f740 1 -- 172.21.15.7:0/2687162437 >> 172.21.15.7:0/2687162437 conn(0x560bcbebb680 msgr2=0x560bcbebe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.157+0000 7f72ac36f740 1 -- 172.21.15.7:0/2687162437 shutdown_connections 2024-06-10T01:11:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.157+0000 7f72ac36f740 1 -- 172.21.15.7:0/2687162437 wait complete. 2024-06-10T01:11:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.157+0000 7f72ac36f740 20 client.16857 trim_cache size 0 max 16384 2024-06-10T01:11:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:11:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:11:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:11:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.158+0000 7f72a957e640 1 -- 172.21.15.7:0/911519707 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f728c0a2b80 msgr2=0x7f728c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.158+0000 7f72a957e640 1 --2- 172.21.15.7:0/911519707 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f728c0a2b80 0x7f728c0a5040 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f729c0976d0 tx=0x7f729c009bd0 comp rx=0 tx=0).stop 2024-06-10T01:11:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.158+0000 7f72a957e640 1 -- 172.21.15.7:0/911519707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72a41076e0 msgr2=0x7f72a419f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.158+0000 7f72a957e640 1 --2- 172.21.15.7:0/911519707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72a41076e0 0x7f72a419f300 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f729c00d750 tx=0x7f729c00dc10 comp rx=0 tx=0).stop 2024-06-10T01:11:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.158+0000 7f72a957e640 1 -- 172.21.15.7:0/911519707 shutdown_connections 2024-06-10T01:11:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.158+0000 7f72a957e640 1 --2- 172.21.15.7:0/911519707 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f728c0a2b80 0x7f728c0a5040 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.158+0000 7f72a957e640 1 --2- 172.21.15.7:0/911519707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72a41076e0 0x7f72a419f300 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.158+0000 7f72a957e640 1 --2- 172.21.15.7:0/911519707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72a4104db0 0x7f72a4198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.158+0000 7f72a957e640 1 --2- 172.21.15.7:0/911519707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72a4102480 0x7f72a4197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.158+0000 7f72a957e640 1 -- 172.21.15.7:0/911519707 >> 172.21.15.7:0/911519707 conn(0x7f72a40fbe70 msgr2=0x7f72a4102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.158+0000 7f72a957e640 1 -- 172.21.15.7:0/911519707 shutdown_connections 2024-06-10T01:11:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:21.158+0000 7f72a957e640 1 -- 172.21.15.7:0/911519707 wait complete. 2024-06-10T01:11:21.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:21 smithi175 ceph-mon[36636]: pgmap v592: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-06-10T01:11:21.710 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:11:21.730 INFO:tasks.workunit.client.0.smithi007.stdout: 1 345759 7.57 MB/sec execute 122 sec latency 21677.158 ms 2024-06-10T01:11:21.739 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:21 smithi007 ceph-mon[32398]: pgmap v592: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-06-10T01:11:21.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:21 smithi078 ceph-mon[36683]: pgmap v592: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-06-10T01:11:22.010 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:11:22.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:22 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:11:22.730 INFO:tasks.workunit.client.0.smithi007.stdout: 1 345759 7.50 MB/sec execute 123 sec latency 22677.238 ms 2024-06-10T01:11:22.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:22 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:11:22.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:22 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:11:23.690 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:23 smithi007 ceph-mon[32398]: pgmap v593: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 1.3 MiB/s wr, 91 op/s 2024-06-10T01:11:23.690 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:23 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:11:23.690 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:23 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:11:23.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:23 smithi175 ceph-mon[36636]: pgmap v593: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 1.3 MiB/s wr, 91 op/s 2024-06-10T01:11:23.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:23 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:11:23.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:23 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:11:23.730 INFO:tasks.workunit.client.0.smithi007.stdout: 1 345759 7.44 MB/sec execute 124 sec latency 23677.348 ms 2024-06-10T01:11:23.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:23 smithi078 ceph-mon[36683]: pgmap v593: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 1.3 MiB/s wr, 91 op/s 2024-06-10T01:11:23.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:23 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:11:23.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:23 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:11:24.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.132+0000 7f1894580700 1 -- 172.21.15.7:0/1395696004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f188c101d70 msgr2=0x7f188c10e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:24.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.132+0000 7f1894580700 1 --2- 172.21.15.7:0/1395696004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f188c101d70 0x7f188c10e250 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f188400b0a0 tx=0x7f188400b3b0 comp rx=0 tx=0).stop 2024-06-10T01:11:24.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.132+0000 7f1894580700 1 -- 172.21.15.7:0/1395696004 shutdown_connections 2024-06-10T01:11:24.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.132+0000 7f1894580700 1 --2- 172.21.15.7:0/1395696004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f188c101d70 0x7f188c10e250 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:24.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.132+0000 7f1894580700 1 --2- 172.21.15.7:0/1395696004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f188c107410 0x7f188c1077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:24.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.132+0000 7f1894580700 1 --2- 172.21.15.7:0/1395696004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f188c1013d0 0x7f188c101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:24.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.132+0000 7f1894580700 1 -- 172.21.15.7:0/1395696004 >> 172.21.15.7:0/1395696004 conn(0x7f188c0782c0 msgr2=0x7f188c0786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:24.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.132+0000 7f1894580700 1 -- 172.21.15.7:0/1395696004 shutdown_connections 2024-06-10T01:11:24.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.132+0000 7f1894580700 1 -- 172.21.15.7:0/1395696004 wait complete. 2024-06-10T01:11:24.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.133+0000 7f1894580700 1 Processor -- start 2024-06-10T01:11:24.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.133+0000 7f1894580700 1 -- start start 2024-06-10T01:11:24.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.133+0000 7f1894580700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f188c1013d0 0x7f188c19d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:24.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.133+0000 7f1894580700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f188c101d70 0x7f188c19dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:24.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.133+0000 7f1894580700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f188c107410 0x7f188c197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:24.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.133+0000 7f1894580700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f188c111690 con 0x7f188c107410 2024-06-10T01:11:24.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.133+0000 7f1894580700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f188c111510 con 0x7f188c1013d0 2024-06-10T01:11:24.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.133+0000 7f1894580700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f188c111810 con 0x7f188c101d70 2024-06-10T01:11:24.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.133+0000 7f1892b1d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f188c107410 0x7f188c197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:24.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.133+0000 7f1892b1d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f188c107410 0x7f188c197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37012/0 (socket says 172.21.15.7:37012) 2024-06-10T01:11:24.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.133+0000 7f1892b1d700 1 -- 172.21.15.7:0/3691820848 learned_addr learned my addr 172.21.15.7:0/3691820848 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:24.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.133+0000 7f1891b1b700 1 --2- 172.21.15.7:0/3691820848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f188c101d70 0x7f188c19dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:24.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.133+0000 7f189231c700 1 --2- 172.21.15.7:0/3691820848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f188c1013d0 0x7f188c19d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:24.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.133+0000 7f1892b1d700 1 -- 172.21.15.7:0/3691820848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f188c101d70 msgr2=0x7f188c19dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:24.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.133+0000 7f1892b1d700 1 --2- 172.21.15.7:0/3691820848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f188c101d70 0x7f188c19dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:24.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.133+0000 7f1892b1d700 1 -- 172.21.15.7:0/3691820848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f188c1013d0 msgr2=0x7f188c19d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:24.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.133+0000 7f1892b1d700 1 --2- 172.21.15.7:0/3691820848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f188c1013d0 0x7f188c19d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:24.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.133+0000 7f1892b1d700 1 -- 172.21.15.7:0/3691820848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188c198070 con 0x7f188c107410 2024-06-10T01:11:24.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.134+0000 7f1892b1d700 1 --2- 172.21.15.7:0/3691820848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f188c107410 0x7f188c197910 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f18840062a0 tx=0x7f1884004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:24.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.134+0000 7f18837fe700 1 -- 172.21.15.7:0/3691820848 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884021070 con 0x7f188c107410 2024-06-10T01:11:24.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.134+0000 7f18837fe700 1 -- 172.21.15.7:0/3691820848 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1884007c00 con 0x7f188c107410 2024-06-10T01:11:24.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.134+0000 7f1894580700 1 -- 172.21.15.7:0/3691820848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f188c198300 con 0x7f188c107410 2024-06-10T01:11:24.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.134+0000 7f18837fe700 1 -- 172.21.15.7:0/3691820848 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188401d450 con 0x7f188c107410 2024-06-10T01:11:24.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.134+0000 7f1894580700 1 -- 172.21.15.7:0/3691820848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f188c1a4620 con 0x7f188c107410 2024-06-10T01:11:24.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.135+0000 7f1894580700 1 -- 172.21.15.7:0/3691820848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f188c069180 con 0x7f188c107410 2024-06-10T01:11:24.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.136+0000 7f18837fe700 1 -- 172.21.15.7:0/3691820848 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f1884004020 con 0x7f188c107410 2024-06-10T01:11:24.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.137+0000 7f18837fe700 1 --2- 172.21.15.7:0/3691820848 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f18780a2ca0 0x7f18780a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:24.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.137+0000 7f189231c700 1 --2- 172.21.15.7:0/3691820848 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f18780a2ca0 0x7f18780a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:24.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.137+0000 7f18837fe700 1 -- 172.21.15.7:0/3691820848 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f18840d0420 con 0x7f188c107410 2024-06-10T01:11:24.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.137+0000 7f189231c700 1 --2- 172.21.15.7:0/3691820848 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f18780a2ca0 0x7f18780a5160 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f188c198a50 tx=0x7f188800b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:24.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.141+0000 7f18837fe700 1 -- 172.21.15.7:0/3691820848 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f188409d030 con 0x7f188c107410 2024-06-10T01:11:24.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.394+0000 7f1894580700 1 -- 172.21.15.7:0/3691820848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f188c198e40 con 0x7f188c107410 2024-06-10T01:11:24.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.395+0000 7f18837fe700 1 -- 172.21.15.7:0/3691820848 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11855 (secure 0 0 0) 0x7f188409c930 con 0x7f188c107410 2024-06-10T01:11:24.390 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:11:24.390 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:10:42.213996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:11:24.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.397+0000 7f1894580700 1 -- 172.21.15.7:0/3691820848 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f18780a2ca0 msgr2=0x7f18780a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:24.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.397+0000 7f1894580700 1 --2- 172.21.15.7:0/3691820848 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f18780a2ca0 0x7f18780a5160 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f188c198a50 tx=0x7f188800b5b0 comp rx=0 tx=0).stop 2024-06-10T01:11:24.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.397+0000 7f1894580700 1 -- 172.21.15.7:0/3691820848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f188c107410 msgr2=0x7f188c197910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:24.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.397+0000 7f1894580700 1 --2- 172.21.15.7:0/3691820848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f188c107410 0x7f188c197910 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f18840062a0 tx=0x7f1884004d70 comp rx=0 tx=0).stop 2024-06-10T01:11:24.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.398+0000 7f1894580700 1 -- 172.21.15.7:0/3691820848 shutdown_connections 2024-06-10T01:11:24.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.398+0000 7f1894580700 1 --2- 172.21.15.7:0/3691820848 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f18780a2ca0 0x7f18780a5160 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:24.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.398+0000 7f1894580700 1 --2- 172.21.15.7:0/3691820848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f188c107410 0x7f188c197910 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:24.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.398+0000 7f1894580700 1 --2- 172.21.15.7:0/3691820848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f188c1013d0 0x7f188c19d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:24.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.398+0000 7f1894580700 1 --2- 172.21.15.7:0/3691820848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f188c101d70 0x7f188c19dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:24.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.398+0000 7f1894580700 1 -- 172.21.15.7:0/3691820848 >> 172.21.15.7:0/3691820848 conn(0x7f188c0782c0 msgr2=0x7f188c100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:24.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.398+0000 7f1894580700 1 -- 172.21.15.7:0/3691820848 shutdown_connections 2024-06-10T01:11:24.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:24.398+0000 7f1894580700 1 -- 172.21.15.7:0/3691820848 wait complete. 2024-06-10T01:11:24.394 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 32 2024-06-10T01:11:24.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:24 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3691820848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:11:24.730 INFO:tasks.workunit.client.0.smithi007.stdout: 1 345759 7.38 MB/sec execute 125 sec latency 24677.456 ms 2024-06-10T01:11:24.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:24 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3691820848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:11:24.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:24 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3691820848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:11:25.403 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db --set-id 24a573d7 --release --await' 2024-06-10T01:11:25.403 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db --set-id 24a573d7 --release --await 2024-06-10T01:11:25.701 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:11:25.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:25 smithi175 ceph-mon[36636]: pgmap v594: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 1.6 MiB/s wr, 88 op/s 2024-06-10T01:11:25.730 INFO:tasks.workunit.client.0.smithi007.stdout: 1 345759 7.33 MB/sec execute 126 sec latency 25677.542 ms 2024-06-10T01:11:25.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:25 smithi007 ceph-mon[32398]: pgmap v594: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 1.6 MiB/s wr, 88 op/s 2024-06-10T01:11:25.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:25 smithi078 ceph-mon[36683]: pgmap v594: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 1.6 MiB/s wr, 88 op/s 2024-06-10T01:11:26.730 INFO:tasks.workunit.client.0.smithi007.stdout: 1 345759 7.27 MB/sec execute 127 sec latency 26677.576 ms 2024-06-10T01:11:27.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:27 smithi175 ceph-mon[36636]: pgmap v595: 129 pgs: 129 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 1.7 MiB/s wr, 92 op/s 2024-06-10T01:11:27.730 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:27 smithi007 ceph-mon[32398]: pgmap v595: 129 pgs: 129 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 1.7 MiB/s wr, 92 op/s 2024-06-10T01:11:27.730 INFO:tasks.workunit.client.0.smithi007.stdout: 1 345759 7.21 MB/sec execute 128 sec latency 27677.608 ms 2024-06-10T01:11:27.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.750+0000 7fe687fe2700 1 -- 172.21.15.7:0/1020563303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe68006ab30 msgr2=0x7fe68010f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:27.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.750+0000 7fe687fe2700 1 --2- 172.21.15.7:0/1020563303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe68006ab30 0x7fe68010f320 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fe67800b0a0 tx=0x7fe67800b3b0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.750+0000 7fe687fe2700 1 -- 172.21.15.7:0/1020563303 shutdown_connections 2024-06-10T01:11:27.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.750+0000 7fe687fe2700 1 --2- 172.21.15.7:0/1020563303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe68006ab30 0x7fe68010f320 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.750+0000 7fe687fe2700 1 --2- 172.21.15.7:0/1020563303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe680108500 0x7fe6801088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.750+0000 7fe687fe2700 1 --2- 172.21.15.7:0/1020563303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe68006a170 0x7fe68006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.750+0000 7fe687fe2700 1 -- 172.21.15.7:0/1020563303 >> 172.21.15.7:0/1020563303 conn(0x7fe680078340 msgr2=0x7fe680078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:27.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.750+0000 7fe687fe2700 1 -- 172.21.15.7:0/1020563303 shutdown_connections 2024-06-10T01:11:27.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.750+0000 7fe687fe2700 1 -- 172.21.15.7:0/1020563303 wait complete. 2024-06-10T01:11:27.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.751+0000 7fe687fe2700 1 Processor -- start 2024-06-10T01:11:27.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.751+0000 7fe687fe2700 1 -- start start 2024-06-10T01:11:27.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.751+0000 7fe687fe2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe68006a170 0x7fe680198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:27.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.751+0000 7fe687fe2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe68006ab30 0x7fe680199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:27.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.751+0000 7fe687fe2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe680108500 0x7fe68019d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:27.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.751+0000 7fe687fe2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe680112760 con 0x7fe68006a170 2024-06-10T01:11:27.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.751+0000 7fe687fe2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe6801125e0 con 0x7fe68006ab30 2024-06-10T01:11:27.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.751+0000 7fe687fe2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe6801128e0 con 0x7fe680108500 2024-06-10T01:11:27.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.751+0000 7fe685d7e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe68006a170 0x7fe680198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:27.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.751+0000 7fe685d7e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe68006a170 0x7fe680198cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37018/0 (socket says 172.21.15.7:37018) 2024-06-10T01:11:27.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.752+0000 7fe685d7e700 1 -- 172.21.15.7:0/1873747521 learned_addr learned my addr 172.21.15.7:0/1873747521 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:27.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.752+0000 7fe68557d700 1 --2- 172.21.15.7:0/1873747521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe68006ab30 0x7fe680199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:27.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.752+0000 7fe68657f700 1 --2- 172.21.15.7:0/1873747521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe680108500 0x7fe68019d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:27.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.752+0000 7fe685d7e700 1 -- 172.21.15.7:0/1873747521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe680108500 msgr2=0x7fe68019d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.752+0000 7fe685d7e700 1 --2- 172.21.15.7:0/1873747521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe680108500 0x7fe68019d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.752+0000 7fe685d7e700 1 -- 172.21.15.7:0/1873747521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe68006ab30 msgr2=0x7fe680199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.752+0000 7fe685d7e700 1 --2- 172.21.15.7:0/1873747521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe68006ab30 0x7fe680199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.752+0000 7fe685d7e700 1 -- 172.21.15.7:0/1873747521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68019dc80 con 0x7fe68006a170 2024-06-10T01:11:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.752+0000 7fe68657f700 1 --2- 172.21.15.7:0/1873747521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe680108500 0x7fe68019d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:11:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.752+0000 7fe685d7e700 1 --2- 172.21.15.7:0/1873747521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe68006a170 0x7fe680198cd0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe67c00d950 tx=0x7fe67c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.752+0000 7fe672ffd700 1 -- 172.21.15.7:0/1873747521 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67c016070 con 0x7fe68006a170 2024-06-10T01:11:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.752+0000 7fe672ffd700 1 -- 172.21.15.7:0/1873747521 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe67c009e20 con 0x7fe68006a170 2024-06-10T01:11:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.752+0000 7fe687fe2700 1 -- 172.21.15.7:0/1873747521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe68019df70 con 0x7fe68006a170 2024-06-10T01:11:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.752+0000 7fe672ffd700 1 -- 172.21.15.7:0/1873747521 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67c00f620 con 0x7fe68006a170 2024-06-10T01:11:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.752+0000 7fe687fe2700 1 -- 172.21.15.7:0/1873747521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6801a5830 con 0x7fe68006a170 2024-06-10T01:11:27.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.754+0000 7fe672ffd700 1 -- 172.21.15.7:0/1873747521 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fe67c019070 con 0x7fe68006a170 2024-06-10T01:11:27.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.755+0000 7fe672ffd700 1 --2- 172.21.15.7:0/1873747521 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe66c0a2c20 0x7fe66c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:27.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.755+0000 7fe68557d700 1 --2- 172.21.15.7:0/1873747521 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe66c0a2c20 0x7fe66c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:27.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.755+0000 7fe672ffd700 1 -- 172.21.15.7:0/1873747521 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fe67c0cef40 con 0x7fe68006a170 2024-06-10T01:11:27.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.756+0000 7fe68557d700 1 --2- 172.21.15.7:0/1873747521 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe66c0a2c20 0x7fe66c0a50e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe680078840 tx=0x7fe67400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:27.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.756+0000 7fe692865b80 1 Processor -- start 2024-06-10T01:11:27.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.756+0000 7fe692865b80 1 -- start start 2024-06-10T01:11:27.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.756+0000 7fe692865b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588a620c500 0x5588a620c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:27.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.756+0000 7fe692865b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588a620ce20 0x5588a6206710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:27.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.756+0000 7fe692865b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a6206c50 0x5588a6207070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:27.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.756+0000 7fe692865b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5588a60f9060 con 0x5588a620ce20 2024-06-10T01:11:27.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.756+0000 7fe692865b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5588a6157950 con 0x5588a6206c50 2024-06-10T01:11:27.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.756+0000 7fe692865b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5588a60bf560 con 0x5588a620c500 2024-06-10T01:11:27.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.756+0000 7fe68557d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588a620ce20 0x5588a6206710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:27.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.757+0000 7fe68557d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588a620ce20 0x5588a6206710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37026/0 (socket says 172.21.15.7:37026) 2024-06-10T01:11:27.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.757+0000 7fe68557d700 1 -- 172.21.15.7:0/3424048373 learned_addr learned my addr 172.21.15.7:0/3424048373 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:27.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.757+0000 7fe685d7e700 1 --2- 172.21.15.7:0/3424048373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588a620c500 0x5588a620c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:27.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.757+0000 7fe68657f700 1 --2- 172.21.15.7:0/3424048373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a6206c50 0x5588a6207070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:27.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.757+0000 7fe68557d700 1 -- 172.21.15.7:0/3424048373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588a620c500 msgr2=0x5588a620c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:27.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.757+0000 7fe68557d700 1 --2- 172.21.15.7:0/3424048373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588a620c500 0x5588a620c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.757+0000 7fe68557d700 1 -- 172.21.15.7:0/3424048373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a6206c50 msgr2=0x5588a6207070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:27.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.757+0000 7fe68557d700 1 --2- 172.21.15.7:0/3424048373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a6206c50 0x5588a6207070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.757+0000 7fe68557d700 1 -- 172.21.15.7:0/3424048373 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588a60a69d0 con 0x5588a620ce20 2024-06-10T01:11:27.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.757+0000 7fe68657f700 1 --2- 172.21.15.7:0/3424048373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a6206c50 0x5588a6207070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:11:27.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.757+0000 7fe68557d700 1 --2- 172.21.15.7:0/3424048373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588a620ce20 0x5588a6206710 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fe67400fd70 tx=0x7fe674012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1db1bbb088a7f46 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:27.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.757+0000 7fe66bfff700 1 -- 172.21.15.7:0/3424048373 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe674016070 con 0x5588a620ce20 2024-06-10T01:11:27.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.757+0000 7fe66bfff700 1 -- 172.21.15.7:0/3424048373 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe67400fe50 con 0x5588a620ce20 2024-06-10T01:11:27.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.757+0000 7fe66bfff700 1 -- 172.21.15.7:0/3424048373 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67401b420 con 0x5588a620ce20 2024-06-10T01:11:27.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.758+0000 7fe692865b80 1 -- 172.21.15.7:0/3424048373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588a620ce20 msgr2=0x5588a6206710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:27.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.758+0000 7fe692865b80 1 --2- 172.21.15.7:0/3424048373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588a620ce20 0x5588a6206710 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fe67400fd70 tx=0x7fe674012000 comp rx=0 tx=0).stop 2024-06-10T01:11:27.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.758+0000 7fe692865b80 1 -- 172.21.15.7:0/3424048373 shutdown_connections 2024-06-10T01:11:27.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.758+0000 7fe692865b80 1 --2- 172.21.15.7:0/3424048373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a6206c50 0x5588a6207070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.758+0000 7fe692865b80 1 --2- 172.21.15.7:0/3424048373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588a620c500 0x5588a620c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.758+0000 7fe692865b80 1 --2- 172.21.15.7:0/3424048373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588a620ce20 0x5588a6206710 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.758+0000 7fe692865b80 1 -- 172.21.15.7:0/3424048373 >> 172.21.15.7:0/3424048373 conn(0x5588a6133180 msgr2=0x5588a6201a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:27.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.758+0000 7fe692865b80 1 -- 172.21.15.7:0/3424048373 shutdown_connections 2024-06-10T01:11:27.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.758+0000 7fe692865b80 1 -- 172.21.15.7:0/3424048373 wait complete. 2024-06-10T01:11:27.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.759+0000 7fe692865b80 1 -- start start 2024-06-10T01:11:27.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.759+0000 7fe692865b80 1 Processor -- start 2024-06-10T01:11:27.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.759+0000 7fe692865b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588a6206c50 0x5588a6212c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:27.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.759+0000 7fe692865b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a620c500 0x5588a620ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:27.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.759+0000 7fe692865b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588a620ce20 0x5588a62104e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:27.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.759+0000 7fe692865b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5588a60f80c0 con 0x5588a620ce20 2024-06-10T01:11:27.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.759+0000 7fe692865b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5588a6153910 con 0x5588a620c500 2024-06-10T01:11:27.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.759+0000 7fe692865b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5588a6213800 con 0x5588a6206c50 2024-06-10T01:11:27.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.759+0000 7fe692865b80 10 client.0 _ll_register_callbacks cb 0x5588a60ed6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:11:27.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.759+0000 7fe68657f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588a620ce20 0x5588a62104e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:27.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.759+0000 7fe68657f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588a620ce20 0x5588a62104e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37034/0 (socket says 172.21.15.7:37034) 2024-06-10T01:11:27.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.759+0000 7fe68657f700 1 -- 172.21.15.7:0/1675865673 learned_addr learned my addr 172.21.15.7:0/1675865673 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:27.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.759+0000 7fe685d7e700 1 --2- 172.21.15.7:0/1675865673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588a6206c50 0x5588a6212c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:27.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.759+0000 7fe68557d700 1 --2- 172.21.15.7:0/1675865673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a620c500 0x5588a620ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:27.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.760+0000 7fe68657f700 1 -- 172.21.15.7:0/1675865673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588a6206c50 msgr2=0x5588a6212c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:27.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.760+0000 7fe68657f700 1 --2- 172.21.15.7:0/1675865673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588a6206c50 0x5588a6212c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.760+0000 7fe68657f700 1 -- 172.21.15.7:0/1675865673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a620c500 msgr2=0x5588a620ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:27.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.760+0000 7fe68657f700 1 --2- 172.21.15.7:0/1675865673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a620c500 0x5588a620ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.760+0000 7fe68657f700 1 -- 172.21.15.7:0/1675865673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588a611d850 con 0x5588a620ce20 2024-06-10T01:11:27.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.760+0000 7fe68657f700 1 --2- 172.21.15.7:0/1675865673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588a620ce20 0x5588a62104e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fe67800b070 tx=0x7fe678009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ff1bad86eb86b5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:27.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.760+0000 7fe6697fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:11:27.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.760+0000 7fe6697fa700 1 -- 172.21.15.7:0/1675865673 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe678022070 con 0x5588a620ce20 2024-06-10T01:11:27.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.760+0000 7fe6697fa700 1 -- 172.21.15.7:0/1675865673 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fe6780076e0 con 0x5588a620ce20 2024-06-10T01:11:27.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.760+0000 7fe692865b80 1 -- 172.21.15.7:0/1675865673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe67400fe50 con 0x5588a620ce20 2024-06-10T01:11:27.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.760+0000 7fe6697fa700 1 -- 172.21.15.7:0/1675865673 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe6780078e0 con 0x5588a620ce20 2024-06-10T01:11:27.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.760+0000 7fe692865b80 10 client.16875 fetch_fsmap learned FSMap version 32 2024-06-10T01:11:27.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.760+0000 7fe692865b80 1 -- 172.21.15.7:0/1675865673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5588a621dfd0 con 0x5588a620ce20 2024-06-10T01:11:27.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.760+0000 7fe6697fa700 1 -- 172.21.15.7:0/1675865673 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe67400fe50 con 0x5588a620ce20 2024-06-10T01:11:27.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.761+0000 7fe6697fa700 1 -- 172.21.15.7:0/1675865673 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fe678011070 con 0x5588a620ce20 2024-06-10T01:11:27.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.761+0000 7fe692865b80 10 client.16875 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:11:27.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.761+0000 7fe692865b80 10 client.16875 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:11:27.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.761+0000 7fe692865b80 20 client.16875 populate_metadata read hostname 'smithi007' 2024-06-10T01:11:27.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.761+0000 7fe692865b80 1 --2- 172.21.15.7:0/1675865673 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5588a62205b0 0x5588a62229c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:27.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.761+0000 7fe692865b80 4 client.16875 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:11:27.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.761+0000 7fe692865b80 1 -- 172.21.15.7:0/1675865673 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588a6164e50 con 0x5588a62205b0 2024-06-10T01:11:27.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.761+0000 7fe685d7e700 1 --2- 172.21.15.7:0/1675865673 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5588a62205b0 0x5588a62229c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:27.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.761+0000 7fe685d7e700 1 --2- 172.21.15.7:0/1675865673 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5588a62205b0 0x5588a62229c0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec1d7a8bd63ed4cd server_cookie=3b1c93f7b5fb6d9f in_seq=0 out_seq=0 2024-06-10T01:11:27.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.761+0000 7fe6697fa700 10 client.16875 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:27.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.762+0000 7fe6697fa700 1 -- 172.21.15.7:0/1675865673 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5588a6164e50 con 0x5588a62205b0 2024-06-10T01:11:27.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.762+0000 7fe6697fa700 10 client.16875 handle_command_reply: tid=0 2024-06-10T01:11:27.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.762+0000 7fe692865b80 1 client.16875 shutdown 2024-06-10T01:11:27.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.762+0000 7fe692865b80 1 -- 172.21.15.7:0/1675865673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588a620ce20 msgr2=0x5588a62104e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:27.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.762+0000 7fe692865b80 1 --2- 172.21.15.7:0/1675865673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588a620ce20 0x5588a62104e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fe67800b070 tx=0x7fe678009e10 comp rx=0 tx=0).stop 2024-06-10T01:11:27.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.763+0000 7fe692865b80 1 -- 172.21.15.7:0/1675865673 shutdown_connections 2024-06-10T01:11:27.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.763+0000 7fe692865b80 1 --2- 172.21.15.7:0/1675865673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588a620ce20 0x5588a62104e0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.763+0000 7fe692865b80 1 --2- 172.21.15.7:0/1675865673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588a6206c50 0x5588a6212c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.763+0000 7fe692865b80 1 --2- 172.21.15.7:0/1675865673 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5588a62205b0 0x5588a62229c0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.763+0000 7fe692865b80 1 --2- 172.21.15.7:0/1675865673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a620c500 0x5588a620ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.763+0000 7fe692865b80 1 -- 172.21.15.7:0/1675865673 >> 172.21.15.7:0/1675865673 conn(0x5588a6133180 msgr2=0x5588a6200bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:27.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.763+0000 7fe6697fa700 0 client.16875 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:27.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.763+0000 7fe692865b80 1 -- 172.21.15.7:0/1675865673 shutdown_connections 2024-06-10T01:11:27.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.763+0000 7fe692865b80 1 -- 172.21.15.7:0/1675865673 wait complete. 2024-06-10T01:11:27.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.763+0000 7fe692865b80 20 client.16875 trim_cache size 0 max 16384 2024-06-10T01:11:27.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.783+0000 7fe692865b80 1 Processor -- start 2024-06-10T01:11:27.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.783+0000 7fe692865b80 1 -- start start 2024-06-10T01:11:27.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.783+0000 7fe692865b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588a6206c50 0x5588a620f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:27.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.783+0000 7fe692865b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588a620c500 0x5588a6201330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.783+0000 7fe692865b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a620ce20 0x5588a6201870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.783+0000 7fe692865b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe6801b0bd0 con 0x5588a620c500 2024-06-10T01:11:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.783+0000 7fe692865b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe6801ae460 con 0x5588a620ce20 2024-06-10T01:11:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.783+0000 7fe692865b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe6801ac2b0 con 0x5588a6206c50 2024-06-10T01:11:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.784+0000 7fe68557d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588a620c500 0x5588a6201330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.784+0000 7fe68557d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588a620c500 0x5588a6201330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37048/0 (socket says 172.21.15.7:37048) 2024-06-10T01:11:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.784+0000 7fe68557d700 1 -- 172.21.15.7:0/2773583701 learned_addr learned my addr 172.21.15.7:0/2773583701 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.784+0000 7fe685d7e700 1 --2- 172.21.15.7:0/2773583701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588a6206c50 0x5588a620f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.784+0000 7fe68657f700 1 --2- 172.21.15.7:0/2773583701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a620ce20 0x5588a6201870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.784+0000 7fe68557d700 1 -- 172.21.15.7:0/2773583701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588a6206c50 msgr2=0x5588a620f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.784+0000 7fe68557d700 1 --2- 172.21.15.7:0/2773583701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588a6206c50 0x5588a620f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.784+0000 7fe68557d700 1 -- 172.21.15.7:0/2773583701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a620ce20 msgr2=0x5588a6201870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.784+0000 7fe68557d700 1 --2- 172.21.15.7:0/2773583701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a620ce20 0x5588a6201870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.784+0000 7fe68557d700 1 -- 172.21.15.7:0/2773583701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588a61f94a0 con 0x5588a620c500 2024-06-10T01:11:27.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.784+0000 7fe68557d700 1 --2- 172.21.15.7:0/2773583701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588a620c500 0x5588a6201330 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fe67400fd40 tx=0x7fe674005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2477d199db0f9ee server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:27.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.784+0000 7fe66b7fe700 1 -- 172.21.15.7:0/2773583701 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe674025620 con 0x5588a620c500 2024-06-10T01:11:27.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.784+0000 7fe66b7fe700 1 -- 172.21.15.7:0/2773583701 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe674010da0 con 0x5588a620c500 2024-06-10T01:11:27.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.784+0000 7fe66b7fe700 1 -- 172.21.15.7:0/2773583701 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6740046f0 con 0x5588a620c500 2024-06-10T01:11:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.785+0000 7fe692865b80 1 -- 172.21.15.7:0/2773583701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588a620c500 msgr2=0x5588a6201330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.785+0000 7fe692865b80 1 --2- 172.21.15.7:0/2773583701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588a620c500 0x5588a6201330 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fe67400fd40 tx=0x7fe674005e00 comp rx=0 tx=0).stop 2024-06-10T01:11:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.785+0000 7fe692865b80 1 -- 172.21.15.7:0/2773583701 shutdown_connections 2024-06-10T01:11:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.785+0000 7fe692865b80 1 --2- 172.21.15.7:0/2773583701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a620ce20 0x5588a6201870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.785+0000 7fe692865b80 1 --2- 172.21.15.7:0/2773583701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588a6206c50 0x5588a620f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.785+0000 7fe692865b80 1 --2- 172.21.15.7:0/2773583701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588a620c500 0x5588a6201330 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.785+0000 7fe692865b80 1 -- 172.21.15.7:0/2773583701 >> 172.21.15.7:0/2773583701 conn(0x5588a6133180 msgr2=0x5588a621b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.785+0000 7fe692865b80 1 -- 172.21.15.7:0/2773583701 shutdown_connections 2024-06-10T01:11:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.785+0000 7fe692865b80 1 -- 172.21.15.7:0/2773583701 wait complete. 2024-06-10T01:11:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.785+0000 7fe692865b80 1 -- start start 2024-06-10T01:11:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.785+0000 7fe692865b80 1 Processor -- start 2024-06-10T01:11:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.786+0000 7fe692865b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588a6206c50 0x5588a6218ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.786+0000 7fe692865b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a620c500 0x5588a62191e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:27.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.786+0000 7fe692865b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588a620ce20 0x5588a6215a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:27.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.786+0000 7fe692865b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe6801ac920 con 0x5588a620ce20 2024-06-10T01:11:27.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.786+0000 7fe692865b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe6801add90 con 0x5588a620c500 2024-06-10T01:11:27.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.786+0000 7fe692865b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe6801acff0 con 0x5588a6206c50 2024-06-10T01:11:27.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.786+0000 7fe692865b80 10 client.0 _ll_register_callbacks cb 0x5588a60ed6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:11:27.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.786+0000 7fe68657f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588a620ce20 0x5588a6215a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:27.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.786+0000 7fe68657f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588a620ce20 0x5588a6215a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37058/0 (socket says 172.21.15.7:37058) 2024-06-10T01:11:27.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.786+0000 7fe68657f700 1 -- 172.21.15.7:0/3687957773 learned_addr learned my addr 172.21.15.7:0/3687957773 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:27.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.786+0000 7fe68557d700 1 --2- 172.21.15.7:0/3687957773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a620c500 0x5588a62191e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:27.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.786+0000 7fe685d7e700 1 --2- 172.21.15.7:0/3687957773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588a6206c50 0x5588a6218ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:27.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.786+0000 7fe68657f700 1 -- 172.21.15.7:0/3687957773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588a6206c50 msgr2=0x5588a6218ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:27.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.786+0000 7fe68657f700 1 --2- 172.21.15.7:0/3687957773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588a6206c50 0x5588a6218ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.786+0000 7fe68657f700 1 -- 172.21.15.7:0/3687957773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a620c500 msgr2=0x5588a62191e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:27.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.786+0000 7fe68657f700 1 --2- 172.21.15.7:0/3687957773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a620c500 0x5588a62191e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.786+0000 7fe68657f700 1 -- 172.21.15.7:0/3687957773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588a6216100 con 0x5588a620ce20 2024-06-10T01:11:27.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.786+0000 7fe685d7e700 1 --2- 172.21.15.7:0/3687957773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588a6206c50 0x5588a6218ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:11:27.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.787+0000 7fe68657f700 1 --2- 172.21.15.7:0/3687957773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588a620ce20 0x5588a6215a90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe67800b0a0 tx=0x7fe6780076d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7058b98fe09c0f1c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:27.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.787+0000 7fe6697fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:11:27.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.787+0000 7fe6697fa700 1 -- 172.21.15.7:0/3687957773 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe678022070 con 0x5588a620ce20 2024-06-10T01:11:27.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.787+0000 7fe6697fa700 1 -- 172.21.15.7:0/3687957773 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fe678007a80 con 0x5588a620ce20 2024-06-10T01:11:27.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.787+0000 7fe692865b80 1 -- 172.21.15.7:0/3687957773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe674010da0 con 0x5588a620ce20 2024-06-10T01:11:27.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.787+0000 7fe6697fa700 1 -- 172.21.15.7:0/3687957773 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe67803d050 con 0x5588a620ce20 2024-06-10T01:11:27.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.787+0000 7fe692865b80 10 client.16881 fetch_fsmap learned FSMap version 32 2024-06-10T01:11:27.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.787+0000 7fe692865b80 1 -- 172.21.15.7:0/3687957773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5588a628bbf0 con 0x5588a620ce20 2024-06-10T01:11:27.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.787+0000 7fe6697fa700 1 -- 172.21.15.7:0/3687957773 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe674010da0 con 0x5588a620ce20 2024-06-10T01:11:27.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.787+0000 7fe6697fa700 1 -- 172.21.15.7:0/3687957773 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fe678011040 con 0x5588a620ce20 2024-06-10T01:11:27.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.787+0000 7fe692865b80 10 client.16881 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:11:27.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.787+0000 7fe692865b80 10 client.16881 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:11:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.787+0000 7fe692865b80 20 client.16881 populate_metadata read hostname 'smithi007' 2024-06-10T01:11:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.787+0000 7fe692865b80 1 --2- 172.21.15.7:0/3687957773 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5588a628c2c0 0x5588a628e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.787+0000 7fe692865b80 4 client.16881 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "24a573d7", "release": true, "await": true}] 2024-06-10T01:11:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.787+0000 7fe692865b80 1 -- 172.21.15.7:0/3687957773 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "set_id": "24a573d7", "release": true, "await": true}) -- 0x5588a615f3d0 con 0x5588a628c2c0 2024-06-10T01:11:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.788+0000 7fe685d7e700 1 --2- 172.21.15.7:0/3687957773 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5588a628c2c0 0x5588a628e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.788+0000 7fe6697fa700 10 client.16881 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.788+0000 7fe685d7e700 1 --2- 172.21.15.7:0/3687957773 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5588a628c2c0 0x5588a628e6e0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e83c06e9d8f8293 server_cookie=e24dfbb3525b1502 in_seq=0 out_seq=0 2024-06-10T01:11:27.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.795+0000 7fe6697fa700 1 -- 172.21.15.7:0/3687957773 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5588a615f3d0 con 0x5588a628c2c0 2024-06-10T01:11:27.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.795+0000 7fe6697fa700 10 client.16881 handle_command_reply: tid=0 2024-06-10T01:11:27.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.795+0000 7fe692865b80 1 client.16881 shutdown 2024-06-10T01:11:27.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.795+0000 7fe692865b80 1 -- 172.21.15.7:0/3687957773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588a620ce20 msgr2=0x5588a6215a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:27.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.795+0000 7fe692865b80 1 --2- 172.21.15.7:0/3687957773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588a620ce20 0x5588a6215a90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe67800b0a0 tx=0x7fe6780076d0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.795+0000 7fe692865b80 1 -- 172.21.15.7:0/3687957773 shutdown_connections 2024-06-10T01:11:27.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.795+0000 7fe692865b80 1 --2- 172.21.15.7:0/3687957773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588a620ce20 0x5588a6215a90 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.795+0000 7fe692865b80 1 --2- 172.21.15.7:0/3687957773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588a6206c50 0x5588a6218ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.795+0000 7fe692865b80 1 --2- 172.21.15.7:0/3687957773 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5588a628c2c0 0x5588a628e6e0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.795+0000 7fe692865b80 1 --2- 172.21.15.7:0/3687957773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a620c500 0x5588a62191e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.795+0000 7fe6697fa700 0 client.16881 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:27.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.795+0000 7fe692865b80 1 -- 172.21.15.7:0/3687957773 >> 172.21.15.7:0/3687957773 conn(0x5588a6133180 msgr2=0x5588a6206480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:27.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.795+0000 7fe692865b80 1 -- 172.21.15.7:0/3687957773 shutdown_connections 2024-06-10T01:11:27.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.795+0000 7fe692865b80 1 -- 172.21.15.7:0/3687957773 wait complete. 2024-06-10T01:11:27.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.795+0000 7fe692865b80 20 client.16881 trim_cache size 0 max 16384 2024-06-10T01:11:27.791 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:11:27.791 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 32, 2024-06-10T01:11:27.791 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:11:27.791 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 16, 2024-06-10T01:11:27.792 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:11:27.792 INFO:teuthology.orchestra.run.smithi007.stdout: "24a573d7": { 2024-06-10T01:11:27.792 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 16, 2024-06-10T01:11:27.792 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:11:27.792 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:11:27.792 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:11:27.792 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:11:27.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:11:27.792 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:11:27.792 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 149.4, 2024-06-10T01:11:27.792 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:11:27.792 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:11:27.792 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:11:27.792 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:11:27.792 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:11:27.793 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:11:27.793 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:11:27.793 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:11:27.793 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:11:27.793 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:11:27.793 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:11:27.793 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:11:27.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.796+0000 7fe687fe2700 1 -- 172.21.15.7:0/1873747521 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe66c0a2c20 msgr2=0x7fe66c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:27.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.796+0000 7fe687fe2700 1 --2- 172.21.15.7:0/1873747521 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe66c0a2c20 0x7fe66c0a50e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe680078840 tx=0x7fe67400a300 comp rx=0 tx=0).stop 2024-06-10T01:11:27.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.796+0000 7fe687fe2700 1 -- 172.21.15.7:0/1873747521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe68006a170 msgr2=0x7fe680198cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:27.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.796+0000 7fe687fe2700 1 --2- 172.21.15.7:0/1873747521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe68006a170 0x7fe680198cd0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe67c00d950 tx=0x7fe67c00dc60 comp rx=0 tx=0).stop 2024-06-10T01:11:27.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.796+0000 7fe687fe2700 1 -- 172.21.15.7:0/1873747521 shutdown_connections 2024-06-10T01:11:27.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.796+0000 7fe687fe2700 1 --2- 172.21.15.7:0/1873747521 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe66c0a2c20 0x7fe66c0a50e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.796+0000 7fe687fe2700 1 --2- 172.21.15.7:0/1873747521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe680108500 0x7fe68019d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.796+0000 7fe687fe2700 1 --2- 172.21.15.7:0/1873747521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe68006a170 0x7fe680198cd0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.796+0000 7fe687fe2700 1 --2- 172.21.15.7:0/1873747521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe68006ab30 0x7fe680199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:27.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.796+0000 7fe687fe2700 1 -- 172.21.15.7:0/1873747521 >> 172.21.15.7:0/1873747521 conn(0x7fe680078340 msgr2=0x7fe6800fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:27.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.796+0000 7fe687fe2700 1 -- 172.21.15.7:0/1873747521 shutdown_connections 2024-06-10T01:11:27.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:27.796+0000 7fe687fe2700 1 -- 172.21.15.7:0/1873747521 wait complete. 2024-06-10T01:11:27.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:27 smithi078 ceph-mon[36683]: pgmap v595: 129 pgs: 129 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 1.7 MiB/s wr, 92 op/s 2024-06-10T01:11:28.521 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '24a573d7', total seconds elapsed: 36.3 2024-06-10T01:11:28.521 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.45 2024-06-10T01:11:28.522 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:11:28.730 INFO:tasks.workunit.client.0.smithi007.stdout: 1 347931 7.26 MB/sec execute 129 sec latency 27739.441 ms 2024-06-10T01:11:28.784 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:11:29.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:29 smithi175 ceph-mon[36636]: pgmap v596: 129 pgs: 129 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 1.8 MiB/s wr, 91 op/s 2024-06-10T01:11:29.731 INFO:tasks.workunit.client.0.smithi007.stdout: 1 349303 7.25 MB/sec execute 130 sec latency 23.012 ms 2024-06-10T01:11:29.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:29 smithi007 ceph-mon[32398]: pgmap v596: 129 pgs: 129 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 1.8 MiB/s wr, 91 op/s 2024-06-10T01:11:29.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:29 smithi078 ceph-mon[36683]: pgmap v596: 129 pgs: 129 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 1.8 MiB/s wr, 91 op/s 2024-06-10T01:11:30.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.615+0000 7f7e956a6700 1 -- 172.21.15.7:0/3321961697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e9006b520 msgr2=0x7f7e9010f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:30.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.615+0000 7f7e956a6700 1 --2- 172.21.15.7:0/3321961697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e9006b520 0x7f7e9010f330 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f7e8800b0a0 tx=0x7f7e8800b3b0 comp rx=0 tx=0).stop 2024-06-10T01:11:30.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.615+0000 7f7e956a6700 1 -- 172.21.15.7:0/3321961697 shutdown_connections 2024-06-10T01:11:30.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.615+0000 7f7e956a6700 1 --2- 172.21.15.7:0/3321961697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e9006b520 0x7f7e9010f330 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:30.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.615+0000 7f7e956a6700 1 --2- 172.21.15.7:0/3321961697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e90108510 0x7f7e901088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:30.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.615+0000 7f7e956a6700 1 --2- 172.21.15.7:0/3321961697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e9006ab60 0x7f7e9006afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:30.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.615+0000 7f7e956a6700 1 -- 172.21.15.7:0/3321961697 >> 172.21.15.7:0/3321961697 conn(0x7f7e90078340 msgr2=0x7f7e90078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:30.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.615+0000 7f7e956a6700 1 -- 172.21.15.7:0/3321961697 shutdown_connections 2024-06-10T01:11:30.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.615+0000 7f7e956a6700 1 -- 172.21.15.7:0/3321961697 wait complete. 2024-06-10T01:11:30.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.616+0000 7f7e956a6700 1 Processor -- start 2024-06-10T01:11:30.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.616+0000 7f7e956a6700 1 -- start start 2024-06-10T01:11:30.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.616+0000 7f7e956a6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e9006ab60 0x7f7e9019e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:30.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.616+0000 7f7e956a6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e9006b520 0x7f7e9019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:30.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.616+0000 7f7e956a6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e90108510 0x7f7e90198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:30.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.616+0000 7f7e956a6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7e90112790 con 0x7f7e9006b520 2024-06-10T01:11:30.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.616+0000 7f7e956a6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7e90112610 con 0x7f7e9006ab60 2024-06-10T01:11:30.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.616+0000 7f7e956a6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7e90112910 con 0x7f7e90108510 2024-06-10T01:11:30.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.616+0000 7f7e867fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e9006b520 0x7f7e9019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:30.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.616+0000 7f7e8effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e9006ab60 0x7f7e9019e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:30.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.617+0000 7f7e867fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e9006b520 0x7f7e9019ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37066/0 (socket says 172.21.15.7:37066) 2024-06-10T01:11:30.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.617+0000 7f7e867fc700 1 -- 172.21.15.7:0/197583720 learned_addr learned my addr 172.21.15.7:0/197583720 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:30.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.617+0000 7f7e8f7fe700 1 --2- 172.21.15.7:0/197583720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e90108510 0x7f7e90198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:30.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.617+0000 7f7e867fc700 1 -- 172.21.15.7:0/197583720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e90108510 msgr2=0x7f7e90198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:30.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.617+0000 7f7e867fc700 1 --2- 172.21.15.7:0/197583720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e90108510 0x7f7e90198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:30.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.617+0000 7f7e867fc700 1 -- 172.21.15.7:0/197583720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e9006ab60 msgr2=0x7f7e9019e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:30.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.617+0000 7f7e867fc700 1 --2- 172.21.15.7:0/197583720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e9006ab60 0x7f7e9019e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:30.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.617+0000 7f7e867fc700 1 -- 172.21.15.7:0/197583720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e90199170 con 0x7f7e9006b520 2024-06-10T01:11:30.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.617+0000 7f7e867fc700 1 --2- 172.21.15.7:0/197583720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e9006b520 0x7f7e9019ed90 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800e9c0 tx=0x7f7e7800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:30.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.617+0000 7f7e8cff9700 1 -- 172.21.15.7:0/197583720 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7800cbd0 con 0x7f7e9006b520 2024-06-10T01:11:30.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.617+0000 7f7e956a6700 1 -- 172.21.15.7:0/197583720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e90199460 con 0x7f7e9006b520 2024-06-10T01:11:30.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.617+0000 7f7e8cff9700 1 -- 172.21.15.7:0/197583720 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e7800ee60 con 0x7f7e9006b520 2024-06-10T01:11:30.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.617+0000 7f7e8cff9700 1 -- 172.21.15.7:0/197583720 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78005630 con 0x7f7e9006b520 2024-06-10T01:11:30.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.617+0000 7f7e956a6700 1 -- 172.21.15.7:0/197583720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e901a5720 con 0x7f7e9006b520 2024-06-10T01:11:30.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.619+0000 7f7e956a6700 1 -- 172.21.15.7:0/197583720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e90103be0 con 0x7f7e9006b520 2024-06-10T01:11:30.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.620+0000 7f7e8cff9700 1 -- 172.21.15.7:0/197583720 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f7e78004750 con 0x7f7e9006b520 2024-06-10T01:11:30.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.621+0000 7f7e8cff9700 1 --2- 172.21.15.7:0/197583720 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7e7c0a2c20 0x7f7e7c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:30.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.621+0000 7f7e8effd700 1 --2- 172.21.15.7:0/197583720 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7e7c0a2c20 0x7f7e7c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:30.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.621+0000 7f7e8cff9700 1 -- 172.21.15.7:0/197583720 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f7e780cf810 con 0x7f7e9006b520 2024-06-10T01:11:30.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.621+0000 7f7e8effd700 1 --2- 172.21.15.7:0/197583720 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7e7c0a2c20 0x7f7e7c0a50e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7e90199b50 tx=0x7f7e80009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:30.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.624+0000 7f7e8cff9700 1 -- 172.21.15.7:0/197583720 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7e78053850 con 0x7f7e9006b520 2024-06-10T01:11:30.731 INFO:tasks.workunit.client.0.smithi007.stdout: 1 351565 7.30 MB/sec execute 131 sec latency 25.579 ms 2024-06-10T01:11:30.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.875+0000 7f7e956a6700 1 -- 172.21.15.7:0/197583720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e90199f40 con 0x7f7e9006b520 2024-06-10T01:11:30.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.876+0000 7f7e8cff9700 1 -- 172.21.15.7:0/197583720 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11855 (secure 0 0 0) 0x7f7e7809c3f0 con 0x7f7e9006b520 2024-06-10T01:11:30.872 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:11:30.872 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:10:42.213996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:11:30.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.878+0000 7f7e956a6700 1 -- 172.21.15.7:0/197583720 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7e7c0a2c20 msgr2=0x7f7e7c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:30.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.878+0000 7f7e956a6700 1 --2- 172.21.15.7:0/197583720 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7e7c0a2c20 0x7f7e7c0a50e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7e90199b50 tx=0x7f7e80009260 comp rx=0 tx=0).stop 2024-06-10T01:11:30.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.878+0000 7f7e956a6700 1 -- 172.21.15.7:0/197583720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e9006b520 msgr2=0x7f7e9019ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:30.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.878+0000 7f7e956a6700 1 --2- 172.21.15.7:0/197583720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e9006b520 0x7f7e9019ed90 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800e9c0 tx=0x7f7e7800ecd0 comp rx=0 tx=0).stop 2024-06-10T01:11:30.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.878+0000 7f7e956a6700 1 -- 172.21.15.7:0/197583720 shutdown_connections 2024-06-10T01:11:30.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.878+0000 7f7e956a6700 1 --2- 172.21.15.7:0/197583720 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7e7c0a2c20 0x7f7e7c0a50e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:30.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.878+0000 7f7e956a6700 1 --2- 172.21.15.7:0/197583720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e90108510 0x7f7e90198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:30.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.878+0000 7f7e956a6700 1 --2- 172.21.15.7:0/197583720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e9006ab60 0x7f7e9019e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:30.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.878+0000 7f7e956a6700 1 --2- 172.21.15.7:0/197583720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e9006b520 0x7f7e9019ed90 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:30.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.878+0000 7f7e956a6700 1 -- 172.21.15.7:0/197583720 >> 172.21.15.7:0/197583720 conn(0x7f7e90078340 msgr2=0x7f7e90100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:30.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.879+0000 7f7e956a6700 1 -- 172.21.15.7:0/197583720 shutdown_connections 2024-06-10T01:11:30.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:30.879+0000 7f7e956a6700 1 -- 172.21.15.7:0/197583720 wait complete. 2024-06-10T01:11:30.876 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 32 2024-06-10T01:11:31.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:31 smithi007 ceph-mon[32398]: pgmap v597: 129 pgs: 129 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 2.2 MiB/s wr, 128 op/s 2024-06-10T01:11:31.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:31 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/197583720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:11:31.649 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db / --timeout 90 --expiration 149.45 --await-for 45.0' 2024-06-10T01:11:31.650 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db / --timeout 90 --expiration 149.45 --await-for 45.0 2024-06-10T01:11:31.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:31 smithi175 ceph-mon[36636]: pgmap v597: 129 pgs: 129 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 2.2 MiB/s wr, 128 op/s 2024-06-10T01:11:31.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:31 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/197583720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:11:31.731 INFO:tasks.workunit.client.0.smithi007.stdout: 1 353717 7.30 MB/sec execute 132 sec latency 14.026 ms 2024-06-10T01:11:31.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:31 smithi078 ceph-mon[36683]: pgmap v597: 129 pgs: 129 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 2.2 MiB/s wr, 128 op/s 2024-06-10T01:11:31.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:31 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/197583720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:11:31.918 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:11:32.731 INFO:tasks.workunit.client.0.smithi007.stdout: 1 355377 7.33 MB/sec execute 133 sec latency 17.191 ms 2024-06-10T01:11:33.599 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:33 smithi007 ceph-mon[32398]: pgmap v598: 129 pgs: 129 active+clean; 69 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 118 op/s 2024-06-10T01:11:33.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:33 smithi175 ceph-mon[36636]: pgmap v598: 129 pgs: 129 active+clean; 69 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 118 op/s 2024-06-10T01:11:33.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.726+0000 7f0a4ee23700 1 -- 172.21.15.7:0/2428652570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a4806ac00 msgr2=0x7f0a4806b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:33.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.726+0000 7f0a4ee23700 1 --2- 172.21.15.7:0/2428652570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a4806ac00 0x7f0a4806b080 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f0a340099c0 tx=0x7f0a34009cd0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.726+0000 7f0a4ee23700 1 -- 172.21.15.7:0/2428652570 shutdown_connections 2024-06-10T01:11:33.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.726+0000 7f0a4ee23700 1 --2- 172.21.15.7:0/2428652570 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a4806b5c0 0x7f0a4810f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.726+0000 7f0a4ee23700 1 --2- 172.21.15.7:0/2428652570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a48108560 0x7f0a48108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.726+0000 7f0a4ee23700 1 --2- 172.21.15.7:0/2428652570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a4806ac00 0x7f0a4806b080 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.726+0000 7f0a4ee23700 1 -- 172.21.15.7:0/2428652570 >> 172.21.15.7:0/2428652570 conn(0x7f0a480783b0 msgr2=0x7f0a480787c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:33.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.726+0000 7f0a4ee23700 1 -- 172.21.15.7:0/2428652570 shutdown_connections 2024-06-10T01:11:33.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.726+0000 7f0a4ee23700 1 -- 172.21.15.7:0/2428652570 wait complete. 2024-06-10T01:11:33.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.726+0000 7f0a4ee23700 1 Processor -- start 2024-06-10T01:11:33.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.726+0000 7f0a4ee23700 1 -- start start 2024-06-10T01:11:33.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.727+0000 7f0a4ee23700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a4806ac00 0x7f0a4819e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:33.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.727+0000 7f0a4ee23700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a4806b5c0 0x7f0a4819ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:33.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.727+0000 7f0a4ee23700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a48108560 0x7f0a481989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:33.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.727+0000 7f0a4ee23700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0a48112790 con 0x7f0a4806b5c0 2024-06-10T01:11:33.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.727+0000 7f0a4ee23700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0a48112610 con 0x7f0a4806ac00 2024-06-10T01:11:33.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.727+0000 7f0a4ee23700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0a48112910 con 0x7f0a48108560 2024-06-10T01:11:33.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.727+0000 7f0a47fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a4806b5c0 0x7f0a4819ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:33.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.727+0000 7f0a47fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a4806b5c0 0x7f0a4819ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47722/0 (socket says 172.21.15.7:47722) 2024-06-10T01:11:33.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.727+0000 7f0a47fff700 1 -- 172.21.15.7:0/832857569 learned_addr learned my addr 172.21.15.7:0/832857569 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:33.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.727+0000 7f0a4d3c0700 1 --2- 172.21.15.7:0/832857569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a48108560 0x7f0a481989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:33.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.727+0000 7f0a4cbbf700 1 --2- 172.21.15.7:0/832857569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a4806ac00 0x7f0a4819e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:33.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.727+0000 7f0a47fff700 1 -- 172.21.15.7:0/832857569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a48108560 msgr2=0x7f0a481989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:33.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.727+0000 7f0a47fff700 1 --2- 172.21.15.7:0/832857569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a48108560 0x7f0a481989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.727+0000 7f0a47fff700 1 -- 172.21.15.7:0/832857569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a4806ac00 msgr2=0x7f0a4819e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:33.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.727+0000 7f0a47fff700 1 --2- 172.21.15.7:0/832857569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a4806ac00 0x7f0a4819e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.727+0000 7f0a47fff700 1 -- 172.21.15.7:0/832857569 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a48199150 con 0x7f0a4806b5c0 2024-06-10T01:11:33.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.727+0000 7f0a47fff700 1 --2- 172.21.15.7:0/832857569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a4806b5c0 0x7f0a4819ed20 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f0a34006010 tx=0x7f0a3400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:33.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.728+0000 7f0a45ffb700 1 -- 172.21.15.7:0/832857569 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34003680 con 0x7f0a4806b5c0 2024-06-10T01:11:33.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.728+0000 7f0a45ffb700 1 -- 172.21.15.7:0/832857569 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a340050d0 con 0x7f0a4806b5c0 2024-06-10T01:11:33.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.728+0000 7f0a4ee23700 1 -- 172.21.15.7:0/832857569 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a481993e0 con 0x7f0a4806b5c0 2024-06-10T01:11:33.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.728+0000 7f0a45ffb700 1 -- 172.21.15.7:0/832857569 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34003db0 con 0x7f0a4806b5c0 2024-06-10T01:11:33.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.728+0000 7f0a4ee23700 1 -- 172.21.15.7:0/832857569 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a481a56b0 con 0x7f0a4806b5c0 2024-06-10T01:11:33.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.731+0000 7f0a45ffb700 1 -- 172.21.15.7:0/832857569 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0a34005280 con 0x7f0a4806b5c0 2024-06-10T01:11:33.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.731+0000 7f0a45ffb700 1 --2- 172.21.15.7:0/832857569 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0a380a2ee0 0x7f0a380a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:33.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.732+0000 7f0a4cbbf700 1 --2- 172.21.15.7:0/832857569 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0a380a2ee0 0x7f0a380a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:33.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.732+0000 7f0a45ffb700 1 -- 172.21.15.7:0/832857569 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f0a340cffd0 con 0x7f0a4806b5c0 2024-06-10T01:11:33.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.732+0000 7f0a4cbbf700 1 --2- 172.21.15.7:0/832857569 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0a380a2ee0 0x7f0a380a53a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0a48108d40 tx=0x7f0a3c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:33.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.732+0000 7f0a596a6b80 1 Processor -- start 2024-06-10T01:11:33.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.732+0000 7f0a596a6b80 1 -- start start 2024-06-10T01:11:33.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.732+0000 7f0a596a6b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab7ad91ce0 0x55ab7ad920c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:33.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.732+0000 7f0a596a6b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab7ad92600 0x55ab7ad8bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:33.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.732+0000 7f0a596a6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab7ad8c470 0x55ab7ad8c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:33.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.732+0000 7f0a596a6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ab7acdd9b0 con 0x55ab7ad8c470 2024-06-10T01:11:33.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.732+0000 7f0a596a6b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ab7ac858c0 con 0x55ab7ad91ce0 2024-06-10T01:11:33.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.732+0000 7f0a596a6b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ab7acd9970 con 0x55ab7ad92600 2024-06-10T01:11:33.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.733+0000 7f0a4cbbf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab7ad91ce0 0x55ab7ad920c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:33.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.733+0000 7f0a4d3c0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab7ad8c470 0x55ab7ad8c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:33.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.733+0000 7f0a47fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab7ad92600 0x55ab7ad8bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:33.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.733+0000 7f0a4cbbf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab7ad91ce0 0x55ab7ad920c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:46670/0 (socket says 172.21.15.7:46670) 2024-06-10T01:11:33.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.733+0000 7f0a4cbbf700 1 -- 172.21.15.7:0/2863839756 learned_addr learned my addr 172.21.15.7:0/2863839756 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:33.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.733+0000 7f0a4cbbf700 1 -- 172.21.15.7:0/2863839756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab7ad92600 msgr2=0x55ab7ad8bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:33.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.733+0000 7f0a4cbbf700 1 --2- 172.21.15.7:0/2863839756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab7ad92600 0x55ab7ad8bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.733+0000 7f0a4cbbf700 1 -- 172.21.15.7:0/2863839756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab7ad8c470 msgr2=0x55ab7ad8c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:33.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.733+0000 7f0a4cbbf700 1 --2- 172.21.15.7:0/2863839756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab7ad8c470 0x55ab7ad8c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.733+0000 7f0a4cbbf700 1 -- 172.21.15.7:0/2863839756 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab7ac7c0f0 con 0x55ab7ad91ce0 2024-06-10T01:11:33.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.733+0000 7f0a4cbbf700 1 --2- 172.21.15.7:0/2863839756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab7ad91ce0 0x55ab7ad920c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c011d60 tx=0x7f0a3c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d41c745ddfd2e5e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:33.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.733+0000 7f0a32ffd700 1 -- 172.21.15.7:0/2863839756 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c011d90 con 0x55ab7ad91ce0 2024-06-10T01:11:33.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.733+0000 7f0a32ffd700 1 -- 172.21.15.7:0/2863839756 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a3c004030 con 0x55ab7ad91ce0 2024-06-10T01:11:33.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.733+0000 7f0a32ffd700 1 -- 172.21.15.7:0/2863839756 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c003d40 con 0x55ab7ad91ce0 2024-06-10T01:11:33.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.734+0000 7f0a596a6b80 1 -- 172.21.15.7:0/2863839756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab7ad91ce0 msgr2=0x55ab7ad920c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:33.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.734+0000 7f0a596a6b80 1 --2- 172.21.15.7:0/2863839756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab7ad91ce0 0x55ab7ad920c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c011d60 tx=0x7f0a3c014000 comp rx=0 tx=0).stop 2024-06-10T01:11:33.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.734+0000 7f0a596a6b80 1 -- 172.21.15.7:0/2863839756 shutdown_connections 2024-06-10T01:11:33.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.734+0000 7f0a596a6b80 1 --2- 172.21.15.7:0/2863839756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab7ad8c470 0x55ab7ad8c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.734+0000 7f0a596a6b80 1 --2- 172.21.15.7:0/2863839756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab7ad91ce0 0x55ab7ad920c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.734+0000 7f0a596a6b80 1 --2- 172.21.15.7:0/2863839756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab7ad92600 0x55ab7ad8bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.734+0000 7f0a596a6b80 1 -- 172.21.15.7:0/2863839756 >> 172.21.15.7:0/2863839756 conn(0x55ab7acb6e60 msgr2=0x55ab7ad871c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:33.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.734+0000 7f0a596a6b80 1 -- 172.21.15.7:0/2863839756 shutdown_connections 2024-06-10T01:11:33.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.734+0000 7f0a596a6b80 1 -- 172.21.15.7:0/2863839756 wait complete. 2024-06-10T01:11:33.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.735+0000 7f0a596a6b80 1 -- start start 2024-06-10T01:11:33.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.735+0000 7f0a596a6b80 1 Processor -- start 2024-06-10T01:11:33.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.735+0000 7f0a596a6b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab7ad8c470 0x55ab7ad98420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:33.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.735+0000 7f0a596a6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab7ad91ce0 0x55ab7ad95720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:33.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.735+0000 7f0a596a6b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab7ad92600 0x55ab7ad95c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:33.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.735+0000 7f0a596a6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ab7ad99070 con 0x55ab7ad91ce0 2024-06-10T01:11:33.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.735+0000 7f0a596a6b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ab7ac7dd90 con 0x55ab7ad92600 2024-06-10T01:11:33.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.735+0000 7f0a596a6b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ab7ad991f0 con 0x55ab7ad8c470 2024-06-10T01:11:33.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.735+0000 7f0a596a6b80 10 client.0 _ll_register_callbacks cb 0x55ab7ac77ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:11:33.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.735+0000 7f0a4cbbf700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab7ad8c470 0x55ab7ad98420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:33.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.735+0000 7f0a4cbbf700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab7ad8c470 0x55ab7ad98420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:37762/0 (socket says 172.21.15.7:37762) 2024-06-10T01:11:33.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.735+0000 7f0a47fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab7ad91ce0 0x55ab7ad95720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:33.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.735+0000 7f0a4cbbf700 1 -- 172.21.15.7:0/29406380 learned_addr learned my addr 172.21.15.7:0/29406380 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:33.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.735+0000 7f0a4d3c0700 1 --2- 172.21.15.7:0/29406380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab7ad92600 0x55ab7ad95c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:33.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.736+0000 7f0a4cbbf700 1 -- 172.21.15.7:0/29406380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab7ad92600 msgr2=0x55ab7ad95c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:33.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.736+0000 7f0a4cbbf700 1 --2- 172.21.15.7:0/29406380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab7ad92600 0x55ab7ad95c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.736+0000 7f0a4cbbf700 1 -- 172.21.15.7:0/29406380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab7ad91ce0 msgr2=0x55ab7ad95720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:33.733 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357691 7.37 MB/sec execute 134 sec latency 62.081 ms 2024-06-10T01:11:33.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.736+0000 7f0a4cbbf700 1 --2- 172.21.15.7:0/29406380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab7ad91ce0 0x55ab7ad95720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.736+0000 7f0a4cbbf700 1 -- 172.21.15.7:0/29406380 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab7ac2ca70 con 0x55ab7ad8c470 2024-06-10T01:11:33.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.736+0000 7f0a47fff700 1 --2- 172.21.15.7:0/29406380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab7ad91ce0 0x55ab7ad95720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:11:33.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.736+0000 7f0a4d3c0700 1 --2- 172.21.15.7:0/29406380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab7ad92600 0x55ab7ad95c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:11:33.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.736+0000 7f0a4cbbf700 1 --2- 172.21.15.7:0/29406380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab7ad8c470 0x55ab7ad98420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c003840 tx=0x7f0a3c005420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c29d0d809f544fa server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:33.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.736+0000 7f0a27fff700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:11:33.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.736+0000 7f0a27fff700 1 -- 172.21.15.7:0/29406380 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c034060 con 0x55ab7ad8c470 2024-06-10T01:11:33.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.736+0000 7f0a596a6b80 1 -- 172.21.15.7:0/29406380 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a3c004030 con 0x55ab7ad8c470 2024-06-10T01:11:33.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.736+0000 7f0a27fff700 1 -- 172.21.15.7:0/29406380 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f0a3c003940 con 0x55ab7ad8c470 2024-06-10T01:11:33.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.736+0000 7f0a27fff700 1 -- 172.21.15.7:0/29406380 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0a3c03b050 con 0x55ab7ad8c470 2024-06-10T01:11:33.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.736+0000 7f0a596a6b80 10 client.26374 fetch_fsmap learned FSMap version 32 2024-06-10T01:11:33.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.736+0000 7f0a596a6b80 1 -- 172.21.15.7:0/29406380 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ab7ada39a0 con 0x55ab7ad8c470 2024-06-10T01:11:33.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.737+0000 7f0a27fff700 1 -- 172.21.15.7:0/29406380 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a3c004030 con 0x55ab7ad8c470 2024-06-10T01:11:33.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.737+0000 7f0a27fff700 1 -- 172.21.15.7:0/29406380 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f0a3c01ec20 con 0x55ab7ad8c470 2024-06-10T01:11:33.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.737+0000 7f0a596a6b80 10 client.26374 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:11:33.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.737+0000 7f0a596a6b80 10 client.26374 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:11:33.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.737+0000 7f0a596a6b80 20 client.26374 populate_metadata read hostname 'smithi007' 2024-06-10T01:11:33.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.737+0000 7f0a596a6b80 1 --2- 172.21.15.7:0/29406380 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ab7ada5e70 0x55ab7ada8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:33.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.737+0000 7f0a596a6b80 4 client.26374 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:11:33.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.737+0000 7f0a596a6b80 1 -- 172.21.15.7:0/29406380 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab7aceaeb0 con 0x55ab7ada5e70 2024-06-10T01:11:33.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.737+0000 7f0a47fff700 1 --2- 172.21.15.7:0/29406380 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ab7ada5e70 0x55ab7ada8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:33.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.737+0000 7f0a27fff700 10 client.26374 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:33.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.737+0000 7f0a47fff700 1 --2- 172.21.15.7:0/29406380 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ab7ada5e70 0x55ab7ada8260 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb110864ea3b7cfb server_cookie=51d0f4659378321 in_seq=0 out_seq=0 2024-06-10T01:11:33.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.738+0000 7f0a27fff700 1 -- 172.21.15.7:0/29406380 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ab7aceaeb0 con 0x55ab7ada5e70 2024-06-10T01:11:33.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.738+0000 7f0a27fff700 10 client.26374 handle_command_reply: tid=0 2024-06-10T01:11:33.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.738+0000 7f0a596a6b80 1 client.26374 shutdown 2024-06-10T01:11:33.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.738+0000 7f0a596a6b80 1 -- 172.21.15.7:0/29406380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab7ad8c470 msgr2=0x55ab7ad98420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:33.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.738+0000 7f0a596a6b80 1 --2- 172.21.15.7:0/29406380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab7ad8c470 0x55ab7ad98420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c003840 tx=0x7f0a3c005420 comp rx=0 tx=0).stop 2024-06-10T01:11:33.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.738+0000 7f0a596a6b80 1 -- 172.21.15.7:0/29406380 shutdown_connections 2024-06-10T01:11:33.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.738+0000 7f0a596a6b80 1 --2- 172.21.15.7:0/29406380 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ab7ada5e70 0x55ab7ada8260 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.738+0000 7f0a596a6b80 1 --2- 172.21.15.7:0/29406380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab7ad92600 0x55ab7ad95c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.738+0000 7f0a596a6b80 1 --2- 172.21.15.7:0/29406380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab7ad8c470 0x55ab7ad98420 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.738+0000 7f0a27fff700 0 client.26374 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:33.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.738+0000 7f0a596a6b80 1 --2- 172.21.15.7:0/29406380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab7ad91ce0 0x55ab7ad95720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.738+0000 7f0a596a6b80 1 -- 172.21.15.7:0/29406380 >> 172.21.15.7:0/29406380 conn(0x55ab7acb6e60 msgr2=0x55ab7ad86db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:33.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.739+0000 7f0a596a6b80 1 -- 172.21.15.7:0/29406380 shutdown_connections 2024-06-10T01:11:33.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.739+0000 7f0a596a6b80 1 -- 172.21.15.7:0/29406380 wait complete. 2024-06-10T01:11:33.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.739+0000 7f0a596a6b80 20 client.26374 trim_cache size 0 max 16384 2024-06-10T01:11:33.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.761+0000 7f0a596a6b80 1 Processor -- start 2024-06-10T01:11:33.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.761+0000 7f0a596a6b80 1 -- start start 2024-06-10T01:11:33.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.761+0000 7f0a596a6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab7ad8c470 0x55ab7ad8bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:33.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.761+0000 7f0a596a6b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab7ad91ce0 0x55ab7ad93270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:33.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.761+0000 7f0a596a6b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab7ad92600 0x55ab7ad945f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:33.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.761+0000 7f0a596a6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0a481b09e0 con 0x55ab7ad8c470 2024-06-10T01:11:33.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.761+0000 7f0a596a6b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0a481ae270 con 0x55ab7ad91ce0 2024-06-10T01:11:33.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.761+0000 7f0a596a6b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0a481ac0c0 con 0x55ab7ad92600 2024-06-10T01:11:33.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.761+0000 7f0a4cbbf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab7ad8c470 0x55ab7ad8bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:33.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.761+0000 7f0a4cbbf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab7ad8c470 0x55ab7ad8bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47750/0 (socket says 172.21.15.7:47750) 2024-06-10T01:11:33.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.761+0000 7f0a47fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab7ad91ce0 0x55ab7ad93270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:33.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.761+0000 7f0a4cbbf700 1 -- 172.21.15.7:0/3079581062 learned_addr learned my addr 172.21.15.7:0/3079581062 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:33.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.761+0000 7f0a4d3c0700 1 --2- 172.21.15.7:0/3079581062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab7ad92600 0x55ab7ad945f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:33.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.762+0000 7f0a4cbbf700 1 -- 172.21.15.7:0/3079581062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab7ad92600 msgr2=0x55ab7ad945f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:33.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.762+0000 7f0a4cbbf700 1 --2- 172.21.15.7:0/3079581062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab7ad92600 0x55ab7ad945f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.762+0000 7f0a4cbbf700 1 -- 172.21.15.7:0/3079581062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab7ad91ce0 msgr2=0x55ab7ad93270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:33.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.762+0000 7f0a4cbbf700 1 --2- 172.21.15.7:0/3079581062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab7ad91ce0 0x55ab7ad93270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.762+0000 7f0a4cbbf700 1 -- 172.21.15.7:0/3079581062 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab7ad7ee60 con 0x55ab7ad8c470 2024-06-10T01:11:33.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.762+0000 7f0a47fff700 1 --2- 172.21.15.7:0/3079581062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab7ad91ce0 0x55ab7ad93270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:11:33.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.762+0000 7f0a4cbbf700 1 --2- 172.21.15.7:0/3079581062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab7ad8c470 0x55ab7ad8bf30 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c009aa0 tx=0x7f0a3c0093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899dd09e8961c205 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:33.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.762+0000 7f0a327fc700 1 -- 172.21.15.7:0/3079581062 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c034040 con 0x55ab7ad8c470 2024-06-10T01:11:33.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.762+0000 7f0a327fc700 1 -- 172.21.15.7:0/3079581062 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a3c0119e0 con 0x55ab7ad8c470 2024-06-10T01:11:33.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.762+0000 7f0a327fc700 1 -- 172.21.15.7:0/3079581062 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c01e420 con 0x55ab7ad8c470 2024-06-10T01:11:33.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.763+0000 7f0a596a6b80 1 -- 172.21.15.7:0/3079581062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab7ad8c470 msgr2=0x55ab7ad8bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:33.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.763+0000 7f0a596a6b80 1 --2- 172.21.15.7:0/3079581062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab7ad8c470 0x55ab7ad8bf30 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c009aa0 tx=0x7f0a3c0093d0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.763+0000 7f0a596a6b80 1 -- 172.21.15.7:0/3079581062 shutdown_connections 2024-06-10T01:11:33.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.763+0000 7f0a596a6b80 1 --2- 172.21.15.7:0/3079581062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab7ad92600 0x55ab7ad945f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.763+0000 7f0a596a6b80 1 --2- 172.21.15.7:0/3079581062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab7ad8c470 0x55ab7ad8bf30 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.763+0000 7f0a596a6b80 1 --2- 172.21.15.7:0/3079581062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab7ad91ce0 0x55ab7ad93270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.763+0000 7f0a596a6b80 1 -- 172.21.15.7:0/3079581062 >> 172.21.15.7:0/3079581062 conn(0x55ab7acb6e60 msgr2=0x55ab7ad93ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:33.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.763+0000 7f0a596a6b80 1 -- 172.21.15.7:0/3079581062 shutdown_connections 2024-06-10T01:11:33.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.763+0000 7f0a596a6b80 1 -- 172.21.15.7:0/3079581062 wait complete. 2024-06-10T01:11:33.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.763+0000 7f0a596a6b80 1 -- start start 2024-06-10T01:11:33.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.763+0000 7f0a596a6b80 1 Processor -- start 2024-06-10T01:11:33.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.764+0000 7f0a596a6b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab7ad8c470 0x55ab7ad9a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:33.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.764+0000 7f0a596a6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab7ad91ce0 0x55ab7ad9a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:33.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.764+0000 7f0a596a6b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab7ad92600 0x55ab7ad9ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:33.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.764+0000 7f0a596a6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0a481ac730 con 0x55ab7ad91ce0 2024-06-10T01:11:33.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.764+0000 7f0a596a6b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0a481adba0 con 0x55ab7ad92600 2024-06-10T01:11:33.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.764+0000 7f0a596a6b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0a481ace00 con 0x55ab7ad8c470 2024-06-10T01:11:33.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.764+0000 7f0a596a6b80 10 client.0 _ll_register_callbacks cb 0x55ab7ac77ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:11:33.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.764+0000 7f0a47fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab7ad91ce0 0x55ab7ad9a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:33.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.764+0000 7f0a4cbbf700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab7ad8c470 0x55ab7ad9a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:33.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.764+0000 7f0a4d3c0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab7ad92600 0x55ab7ad9ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.764+0000 7f0a4cbbf700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab7ad8c470 0x55ab7ad9a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:37782/0 (socket says 172.21.15.7:37782) 2024-06-10T01:11:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.764+0000 7f0a4cbbf700 1 -- 172.21.15.7:0/143928548 learned_addr learned my addr 172.21.15.7:0/143928548 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.764+0000 7f0a47fff700 1 -- 172.21.15.7:0/143928548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab7ad8c470 msgr2=0x55ab7ad9a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.764+0000 7f0a47fff700 1 --2- 172.21.15.7:0/143928548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab7ad8c470 0x55ab7ad9a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.764+0000 7f0a47fff700 1 -- 172.21.15.7:0/143928548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab7ad92600 msgr2=0x55ab7ad9ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.764+0000 7f0a47fff700 1 --2- 172.21.15.7:0/143928548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab7ad92600 0x55ab7ad9ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.764+0000 7f0a47fff700 1 -- 172.21.15.7:0/143928548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab7ae0f940 con 0x55ab7ad91ce0 2024-06-10T01:11:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.764+0000 7f0a4d3c0700 1 --2- 172.21.15.7:0/143928548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab7ad92600 0x55ab7ad9ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:11:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.764+0000 7f0a47fff700 1 --2- 172.21.15.7:0/143928548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab7ad91ce0 0x55ab7ad9a7f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f0a340cd720 tx=0x7f0a34087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc1de8c02edb359 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.764+0000 7f0a27fff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:11:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.764+0000 7f0a27fff700 1 -- 172.21.15.7:0/143928548 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a340b4020 con 0x55ab7ad91ce0 2024-06-10T01:11:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.765+0000 7f0a27fff700 1 -- 172.21.15.7:0/143928548 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f0a3409a840 con 0x55ab7ad91ce0 2024-06-10T01:11:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.765+0000 7f0a596a6b80 1 -- 172.21.15.7:0/143928548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a3c0119e0 con 0x55ab7ad91ce0 2024-06-10T01:11:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.765+0000 7f0a27fff700 1 -- 172.21.15.7:0/143928548 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0a3409bea0 con 0x55ab7ad91ce0 2024-06-10T01:11:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.765+0000 7f0a596a6b80 10 client.16905 fetch_fsmap learned FSMap version 32 2024-06-10T01:11:33.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.765+0000 7f0a596a6b80 1 -- 172.21.15.7:0/143928548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ab7ae10c60 con 0x55ab7ad91ce0 2024-06-10T01:11:33.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.765+0000 7f0a27fff700 1 -- 172.21.15.7:0/143928548 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a3c0119e0 con 0x55ab7ad91ce0 2024-06-10T01:11:33.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.765+0000 7f0a27fff700 1 -- 172.21.15.7:0/143928548 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f0a3409c7a0 con 0x55ab7ad91ce0 2024-06-10T01:11:33.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.765+0000 7f0a596a6b80 10 client.16905 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:11:33.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.765+0000 7f0a596a6b80 10 client.16905 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:11:33.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.765+0000 7f0a596a6b80 20 client.16905 populate_metadata read hostname 'smithi007' 2024-06-10T01:11:33.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.765+0000 7f0a596a6b80 1 --2- 172.21.15.7:0/143928548 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ab7ae11330 0x55ab7ae13720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:33.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.765+0000 7f0a596a6b80 4 client.16905 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.45, "await_for": 45.0}] 2024-06-10T01:11:33.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.765+0000 7f0a596a6b80 1 -- 172.21.15.7:0/143928548 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.45, "await_for": 45.0}) -- 0x55ab7ace5430 con 0x55ab7ae11330 2024-06-10T01:11:33.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.765+0000 7f0a4cbbf700 1 --2- 172.21.15.7:0/143928548 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ab7ae11330 0x55ab7ae13720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:33.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.766+0000 7f0a4cbbf700 1 --2- 172.21.15.7:0/143928548 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ab7ae11330 0x55ab7ae13720 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0e2d6ca157f2d34 server_cookie=7b27f2a78ab9a17d in_seq=0 out_seq=0 2024-06-10T01:11:33.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.766+0000 7f0a27fff700 10 client.16905 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:33.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:33 smithi078 ceph-mon[36683]: pgmap v598: 129 pgs: 129 active+clean; 69 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 118 op/s 2024-06-10T01:11:33.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.865+0000 7f0a27fff700 1 -- 172.21.15.7:0/143928548 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ab7ace5430 con 0x55ab7ae11330 2024-06-10T01:11:33.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.865+0000 7f0a27fff700 10 client.16905 handle_command_reply: tid=0 2024-06-10T01:11:33.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.865+0000 7f0a596a6b80 1 client.16905 shutdown 2024-06-10T01:11:33.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.865+0000 7f0a596a6b80 1 -- 172.21.15.7:0/143928548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab7ad91ce0 msgr2=0x55ab7ad9a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:33.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.865+0000 7f0a596a6b80 1 --2- 172.21.15.7:0/143928548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab7ad91ce0 0x55ab7ad9a7f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f0a340cd720 tx=0x7f0a34087040 comp rx=0 tx=0).stop 2024-06-10T01:11:33.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.865+0000 7f0a596a6b80 1 -- 172.21.15.7:0/143928548 shutdown_connections 2024-06-10T01:11:33.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.865+0000 7f0a596a6b80 1 --2- 172.21.15.7:0/143928548 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ab7ae11330 0x55ab7ae13720 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.865+0000 7f0a596a6b80 1 --2- 172.21.15.7:0/143928548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab7ad92600 0x55ab7ad9ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.865+0000 7f0a596a6b80 1 --2- 172.21.15.7:0/143928548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab7ad8c470 0x55ab7ad9a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.865+0000 7f0a27fff700 0 client.16905 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:33.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.865+0000 7f0a596a6b80 1 --2- 172.21.15.7:0/143928548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab7ad91ce0 0x55ab7ad9a7f0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.865+0000 7f0a596a6b80 1 -- 172.21.15.7:0/143928548 >> 172.21.15.7:0/143928548 conn(0x55ab7acb6e60 msgr2=0x55ab7ad90880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:33.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.865+0000 7f0a596a6b80 1 -- 172.21.15.7:0/143928548 shutdown_connections 2024-06-10T01:11:33.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.865+0000 7f0a596a6b80 1 -- 172.21.15.7:0/143928548 wait complete. 2024-06-10T01:11:33.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.865+0000 7f0a596a6b80 20 client.16905 trim_cache size 0 max 16384 2024-06-10T01:11:33.862 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:11:33.862 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 32, 2024-06-10T01:11:33.862 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:11:33.862 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 18, 2024-06-10T01:11:33.862 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:11:33.862 INFO:teuthology.orchestra.run.smithi007.stdout: "55ac2137": { 2024-06-10T01:11:33.862 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 18, 2024-06-10T01:11:33.862 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:11:33.863 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:11:33.863 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:11:33.863 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:11:33.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:11:33.863 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:11:33.863 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 149.4, 2024-06-10T01:11:33.863 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:11:33.863 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:11:33.863 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:11:33.863 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:11:33.863 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:11:33.863 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:11:33.863 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:11:33.863 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:11:33.864 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:11:33.864 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:11:33.864 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:11:33.864 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:11:33.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.866+0000 7f0a4ee23700 1 -- 172.21.15.7:0/832857569 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0a380a2ee0 msgr2=0x7f0a380a53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:33.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.866+0000 7f0a4ee23700 1 --2- 172.21.15.7:0/832857569 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0a380a2ee0 0x7f0a380a53a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0a48108d40 tx=0x7f0a3c009260 comp rx=0 tx=0).stop 2024-06-10T01:11:33.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.866+0000 7f0a4ee23700 1 -- 172.21.15.7:0/832857569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a4806b5c0 msgr2=0x7f0a4819ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:33.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.866+0000 7f0a4ee23700 1 --2- 172.21.15.7:0/832857569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a4806b5c0 0x7f0a4819ed20 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f0a34006010 tx=0x7f0a3400f690 comp rx=0 tx=0).stop 2024-06-10T01:11:33.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.866+0000 7f0a4ee23700 1 -- 172.21.15.7:0/832857569 shutdown_connections 2024-06-10T01:11:33.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.866+0000 7f0a4ee23700 1 --2- 172.21.15.7:0/832857569 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0a380a2ee0 0x7f0a380a53a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.866+0000 7f0a4ee23700 1 --2- 172.21.15.7:0/832857569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a48108560 0x7f0a481989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.866+0000 7f0a4ee23700 1 --2- 172.21.15.7:0/832857569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a4806ac00 0x7f0a4819e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.866+0000 7f0a4ee23700 1 --2- 172.21.15.7:0/832857569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a4806b5c0 0x7f0a4819ed20 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:33.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.866+0000 7f0a4ee23700 1 -- 172.21.15.7:0/832857569 >> 172.21.15.7:0/832857569 conn(0x7f0a480783b0 msgr2=0x7f0a480ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:33.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.866+0000 7f0a4ee23700 1 -- 172.21.15.7:0/832857569 shutdown_connections 2024-06-10T01:11:33.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:33.866+0000 7f0a4ee23700 1 -- 172.21.15.7:0/832857569 wait complete. 2024-06-10T01:11:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.257+0000 7fe21a7c1640 1 -- 172.21.15.7:0/3894154598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2141024c0 msgr2=0x7fe2141048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.257+0000 7fe21a7c1640 1 --2- 172.21.15.7:0/3894154598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2141024c0 0x7fe2141048b0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fe204009900 tx=0x7fe20402a410 comp rx=0 tx=0).stop 2024-06-10T01:11:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.257+0000 7fe21a7c1640 1 -- 172.21.15.7:0/3894154598 shutdown_connections 2024-06-10T01:11:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.257+0000 7fe21a7c1640 1 --2- 172.21.15.7:0/3894154598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe214107720 0x7fe214109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.257+0000 7fe21a7c1640 1 --2- 172.21.15.7:0/3894154598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe214104df0 0x7fe2141071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.257+0000 7fe21a7c1640 1 --2- 172.21.15.7:0/3894154598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2141024c0 0x7fe2141048b0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.257+0000 7fe21a7c1640 1 -- 172.21.15.7:0/3894154598 >> 172.21.15.7:0/3894154598 conn(0x7fe2140fbed0 msgr2=0x7fe2140fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.257+0000 7fe21a7c1640 1 -- 172.21.15.7:0/3894154598 shutdown_connections 2024-06-10T01:11:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.258+0000 7fe21a7c1640 1 -- 172.21.15.7:0/3894154598 wait complete. 2024-06-10T01:11:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.258+0000 7fe21a7c1640 1 Processor -- start 2024-06-10T01:11:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.258+0000 7fe21a7c1640 1 -- start start 2024-06-10T01:11:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.258+0000 7fe21a7c1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2141024c0 0x7fe214101e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.258+0000 7fe21a7c1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe214104df0 0x7fe214100470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.258+0000 7fe21a7c1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe214107720 0x7fe2141009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.258+0000 7fe21a7c1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe21410d6c0 con 0x7fe214104df0 2024-06-10T01:11:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.258+0000 7fe21a7c1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe21410d540 con 0x7fe2141024c0 2024-06-10T01:11:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.258+0000 7fe21a7c1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe21410d840 con 0x7fe214107720 2024-06-10T01:11:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.259+0000 7fe2137fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe214104df0 0x7fe214100470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.259+0000 7fe2137fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe214104df0 0x7fe214100470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47776/0 (socket says 172.21.15.7:47776) 2024-06-10T01:11:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.259+0000 7fe2137fe640 1 -- 172.21.15.7:0/4231011997 learned_addr learned my addr 172.21.15.7:0/4231011997 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.259+0000 7fe213fff640 1 --2- 172.21.15.7:0/4231011997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2141024c0 0x7fe214101e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.259+0000 7fe218d5e640 1 --2- 172.21.15.7:0/4231011997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe214107720 0x7fe2141009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.259+0000 7fe2137fe640 1 -- 172.21.15.7:0/4231011997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe214107720 msgr2=0x7fe2141009b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.259+0000 7fe2137fe640 1 --2- 172.21.15.7:0/4231011997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe214107720 0x7fe2141009b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.259+0000 7fe2137fe640 1 -- 172.21.15.7:0/4231011997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2141024c0 msgr2=0x7fe214101e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.259+0000 7fe2137fe640 1 --2- 172.21.15.7:0/4231011997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2141024c0 0x7fe214101e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.259+0000 7fe2137fe640 1 -- 172.21.15.7:0/4231011997 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe214101240 con 0x7fe214104df0 2024-06-10T01:11:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.259+0000 7fe2137fe640 1 --2- 172.21.15.7:0/4231011997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe214104df0 0x7fe214100470 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fe20800e9c0 tx=0x7fe20800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.259+0000 7fe2117fa640 1 -- 172.21.15.7:0/4231011997 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20800cd80 con 0x7fe214104df0 2024-06-10T01:11:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.259+0000 7fe2117fa640 1 -- 172.21.15.7:0/4231011997 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe208005500 con 0x7fe214104df0 2024-06-10T01:11:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.259+0000 7fe21a7c1640 1 -- 172.21.15.7:0/4231011997 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2141a1e90 con 0x7fe214104df0 2024-06-10T01:11:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.259+0000 7fe2117fa640 1 -- 172.21.15.7:0/4231011997 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe208010690 con 0x7fe214104df0 2024-06-10T01:11:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.259+0000 7fe21a7c1640 1 -- 172.21.15.7:0/4231011997 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2141a22f0 con 0x7fe214104df0 2024-06-10T01:11:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.262+0000 7fe2117fa640 1 -- 172.21.15.7:0/4231011997 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fe2080046c0 con 0x7fe214104df0 2024-06-10T01:11:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.262+0000 7fe21d5a9740 1 Processor -- start 2024-06-10T01:11:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.262+0000 7fe21d5a9740 1 -- start start 2024-06-10T01:11:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.262+0000 7fe21d5a9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b7df737010 0x55b7df737430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.262+0000 7fe21d5a9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7df737a00 0x55b7df73c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.262+0000 7fe21d5a9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7df73cb70 0x55b7df73f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.262+0000 7fe21d5a9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b7df69a4f0 con 0x55b7df73cb70 2024-06-10T01:11:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.262+0000 7fe21d5a9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b7df638870 con 0x55b7df737a00 2024-06-10T01:11:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.262+0000 7fe21d5a9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b7df6964b0 con 0x55b7df737010 2024-06-10T01:11:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.262+0000 7fe2137fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7df73cb70 0x55b7df73f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.262+0000 7fe2137fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7df73cb70 0x55b7df73f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47790/0 (socket says 172.21.15.7:47790) 2024-06-10T01:11:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.262+0000 7fe2137fe640 1 -- 172.21.15.7:0/635102148 learned_addr learned my addr 172.21.15.7:0/635102148 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.262+0000 7fe218d5e640 1 --2- 172.21.15.7:0/635102148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b7df737010 0x55b7df737430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.262+0000 7fe213fff640 1 --2- 172.21.15.7:0/635102148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7df737a00 0x55b7df73c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.262+0000 7fe2137fe640 1 -- 172.21.15.7:0/635102148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b7df737010 msgr2=0x55b7df737430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.262+0000 7fe2137fe640 1 --2- 172.21.15.7:0/635102148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b7df737010 0x55b7df737430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.262+0000 7fe2137fe640 1 -- 172.21.15.7:0/635102148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7df737a00 msgr2=0x55b7df73c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.262+0000 7fe2137fe640 1 --2- 172.21.15.7:0/635102148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7df737a00 0x55b7df73c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.262+0000 7fe2137fe640 1 -- 172.21.15.7:0/635102148 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7df6549b0 con 0x55b7df73cb70 2024-06-10T01:11:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.263+0000 7fe2137fe640 1 --2- 172.21.15.7:0/635102148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7df73cb70 0x55b7df73f000 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe2080d5a90 tx=0x7fe2080d5b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef266a314975ce41 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.263+0000 7fe1fa7fc640 1 -- 172.21.15.7:0/635102148 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2080de070 con 0x55b7df73cb70 2024-06-10T01:11:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.263+0000 7fe2117fa640 1 --2- 172.21.15.7:0/4231011997 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe1fc0a2b80 0x7fe1fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.263+0000 7fe1fa7fc640 1 -- 172.21.15.7:0/635102148 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2080d7d60 con 0x55b7df73cb70 2024-06-10T01:11:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.263+0000 7fe1fa7fc640 1 -- 172.21.15.7:0/635102148 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2080d1970 con 0x55b7df73cb70 2024-06-10T01:11:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.263+0000 7fe218d5e640 1 --2- 172.21.15.7:0/4231011997 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe1fc0a2b80 0x7fe1fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.263+0000 7fe2117fa640 1 -- 172.21.15.7:0/4231011997 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fe2080cf750 con 0x7fe214104df0 2024-06-10T01:11:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.264+0000 7fe21d5a9740 1 -- 172.21.15.7:0/635102148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7df73cb70 msgr2=0x55b7df73f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.264+0000 7fe218d5e640 1 --2- 172.21.15.7:0/4231011997 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe1fc0a2b80 0x7fe1fc0a5040 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x55b7df738260 tx=0x7fe20c0096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.264+0000 7fe21d5a9740 1 --2- 172.21.15.7:0/635102148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7df73cb70 0x55b7df73f000 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe2080d5a90 tx=0x7fe2080d5b80 comp rx=0 tx=0).stop 2024-06-10T01:11:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.264+0000 7fe21d5a9740 1 -- 172.21.15.7:0/635102148 shutdown_connections 2024-06-10T01:11:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.264+0000 7fe21d5a9740 1 --2- 172.21.15.7:0/635102148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7df73cb70 0x55b7df73f000 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.264+0000 7fe21d5a9740 1 --2- 172.21.15.7:0/635102148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7df737a00 0x55b7df73c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.264+0000 7fe21d5a9740 1 --2- 172.21.15.7:0/635102148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b7df737010 0x55b7df737430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.264+0000 7fe21d5a9740 1 -- 172.21.15.7:0/635102148 >> 172.21.15.7:0/635102148 conn(0x55b7df732680 msgr2=0x55b7df732a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.264+0000 7fe21d5a9740 1 -- 172.21.15.7:0/635102148 shutdown_connections 2024-06-10T01:11:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.264+0000 7fe21d5a9740 1 -- 172.21.15.7:0/635102148 wait complete. 2024-06-10T01:11:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.264+0000 7fe21d5a9740 1 -- start start 2024-06-10T01:11:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.264+0000 7fe21d5a9740 1 Processor -- start 2024-06-10T01:11:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.265+0000 7fe21d5a9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7df737010 0x55b7df741230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.265+0000 7fe21d5a9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7df737a00 0x55b7df741770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.265+0000 7fe21d5a9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b7df73cb70 0x55b7df7487b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.265+0000 7fe213fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7df737010 0x55b7df741230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.265+0000 7fe21d5a9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b7df73fb20 con 0x55b7df737010 2024-06-10T01:11:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.265+0000 7fe21d5a9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b7df623ae0 con 0x55b7df737a00 2024-06-10T01:11:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.265+0000 7fe213fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7df737010 0x55b7df741230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47804/0 (socket says 172.21.15.7:47804) 2024-06-10T01:11:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.265+0000 7fe21d5a9740 1 -- 172.21.15.7:0/2884990718 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b7df73fca0 con 0x55b7df73cb70 2024-06-10T01:11:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.265+0000 7fe213fff640 1 -- 172.21.15.7:0/2884990718 learned_addr learned my addr 172.21.15.7:0/2884990718 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.265+0000 7fe2137fe640 1 --2- 172.21.15.7:0/2884990718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7df737a00 0x55b7df741770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.265+0000 7fe21d5a9740 10 client.0 _ll_register_callbacks cb 0x55b7df65c9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:11:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.265+0000 7fe218d5e640 1 --2- 172.21.15.7:0/2884990718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b7df73cb70 0x55b7df7487b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.265+0000 7fe213fff640 1 -- 172.21.15.7:0/2884990718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b7df73cb70 msgr2=0x55b7df7487b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.265+0000 7fe213fff640 1 --2- 172.21.15.7:0/2884990718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b7df73cb70 0x55b7df7487b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.265+0000 7fe213fff640 1 -- 172.21.15.7:0/2884990718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7df737a00 msgr2=0x55b7df741770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.265+0000 7fe213fff640 1 --2- 172.21.15.7:0/2884990718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7df737a00 0x55b7df741770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.265+0000 7fe213fff640 1 -- 172.21.15.7:0/2884990718 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7df3ae210 con 0x55b7df737010 2024-06-10T01:11:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.265+0000 7fe2137fe640 1 --2- 172.21.15.7:0/2884990718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7df737a00 0x55b7df741770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:11:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.265+0000 7fe213fff640 1 --2- 172.21.15.7:0/2884990718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7df737010 0x55b7df741230 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe204009900 tx=0x7fe2040318e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f452054a99e2ce server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.265+0000 7fe1f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:11:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.265+0000 7fe1f37fe640 1 -- 172.21.15.7:0/2884990718 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe204035040 con 0x55b7df737010 2024-06-10T01:11:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.265+0000 7fe1f37fe640 1 -- 172.21.15.7:0/2884990718 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fe204004040 con 0x55b7df737010 2024-06-10T01:11:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.265+0000 7fe21d5a9740 1 -- 172.21.15.7:0/2884990718 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2080d7d60 con 0x55b7df737010 2024-06-10T01:11:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.266+0000 7fe1f37fe640 1 -- 172.21.15.7:0/2884990718 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe20404f050 con 0x55b7df737010 2024-06-10T01:11:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.266+0000 7fe21d5a9740 10 client.16917 fetch_fsmap learned FSMap version 32 2024-06-10T01:11:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.266+0000 7fe21d5a9740 1 -- 172.21.15.7:0/2884990718 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b7df47a620 con 0x55b7df737010 2024-06-10T01:11:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.266+0000 7fe1f37fe640 1 -- 172.21.15.7:0/2884990718 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2080d7d60 con 0x55b7df737010 2024-06-10T01:11:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.266+0000 7fe1f37fe640 1 -- 172.21.15.7:0/2884990718 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fe2040035f0 con 0x55b7df737010 2024-06-10T01:11:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.266+0000 7fe21d5a9740 10 client.16917 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:11:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.266+0000 7fe21d5a9740 10 client.16917 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:11:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.266+0000 7fe21d5a9740 20 client.16917 populate_metadata read hostname 'smithi007' 2024-06-10T01:11:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.266+0000 7fe21d5a9740 1 --2- 172.21.15.7:0/2884990718 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b7df74d090 0x55b7df74f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.267+0000 7fe21d5a9740 4 client.16917 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:11:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.267+0000 7fe21d5a9740 1 -- 172.21.15.7:0/2884990718 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7df6adf10 con 0x55b7df74d090 2024-06-10T01:11:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.267+0000 7fe2137fe640 1 --2- 172.21.15.7:0/2884990718 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b7df74d090 0x55b7df74f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.267+0000 7fe2137fe640 1 --2- 172.21.15.7:0/2884990718 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b7df74d090 0x55b7df74f4e0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=421d06c276da847d server_cookie=107b99850349c201 in_seq=0 out_seq=0 2024-06-10T01:11:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.267+0000 7fe1f37fe640 10 client.16917 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.268+0000 7fe1f37fe640 1 -- 172.21.15.7:0/2884990718 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b7df6adf10 con 0x55b7df74d090 2024-06-10T01:11:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.268+0000 7fe1f37fe640 10 client.16917 handle_command_reply: tid=0 2024-06-10T01:11:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.268+0000 7fe21d5a9740 1 client.16917 shutdown 2024-06-10T01:11:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.268+0000 7fe21d5a9740 1 -- 172.21.15.7:0/2884990718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7df737010 msgr2=0x55b7df741230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.268+0000 7fe21d5a9740 1 --2- 172.21.15.7:0/2884990718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7df737010 0x55b7df741230 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe204009900 tx=0x7fe2040318e0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.268+0000 7fe21d5a9740 1 -- 172.21.15.7:0/2884990718 shutdown_connections 2024-06-10T01:11:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.268+0000 7fe21d5a9740 1 --2- 172.21.15.7:0/2884990718 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b7df74d090 0x55b7df74f4e0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.268+0000 7fe21d5a9740 1 --2- 172.21.15.7:0/2884990718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b7df73cb70 0x55b7df7487b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.268+0000 7fe1f37fe640 0 client.16917 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.268+0000 7fe21d5a9740 1 --2- 172.21.15.7:0/2884990718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7df737a00 0x55b7df741770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.268+0000 7fe21d5a9740 1 --2- 172.21.15.7:0/2884990718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7df737010 0x55b7df741230 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.268+0000 7fe21d5a9740 1 -- 172.21.15.7:0/2884990718 >> 172.21.15.7:0/2884990718 conn(0x55b7df732680 msgr2=0x55b7df7317e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.268+0000 7fe21d5a9740 1 -- 172.21.15.7:0/2884990718 shutdown_connections 2024-06-10T01:11:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.268+0000 7fe21d5a9740 1 -- 172.21.15.7:0/2884990718 wait complete. 2024-06-10T01:11:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.268+0000 7fe21d5a9740 20 client.16917 trim_cache size 0 max 16384 2024-06-10T01:11:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.283+0000 7fe21d5a9740 1 Processor -- start 2024-06-10T01:11:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.283+0000 7fe21d5a9740 1 -- start start 2024-06-10T01:11:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.283+0000 7fe21d5a9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b7df737a00 0x55b7df73c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.283+0000 7fe21d5a9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7df73cb70 0x55b7df7486d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.283+0000 7fe21d5a9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7df735360 0x55b7df7357e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.283+0000 7fe21d5a9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe2141af110 con 0x55b7df735360 2024-06-10T01:11:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.283+0000 7fe21d5a9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe2141ac9e0 con 0x55b7df73cb70 2024-06-10T01:11:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.283+0000 7fe21d5a9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe2141aa830 con 0x55b7df737a00 2024-06-10T01:11:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.284+0000 7fe218d5e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7df735360 0x55b7df7357e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.284+0000 7fe218d5e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7df735360 0x55b7df7357e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47808/0 (socket says 172.21.15.7:47808) 2024-06-10T01:11:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.284+0000 7fe218d5e640 1 -- 172.21.15.7:0/3682575820 learned_addr learned my addr 172.21.15.7:0/3682575820 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.284+0000 7fe213fff640 1 --2- 172.21.15.7:0/3682575820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b7df737a00 0x55b7df73c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.284+0000 7fe2137fe640 1 --2- 172.21.15.7:0/3682575820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7df73cb70 0x55b7df7486d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.284+0000 7fe218d5e640 1 -- 172.21.15.7:0/3682575820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b7df737a00 msgr2=0x55b7df73c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.284+0000 7fe218d5e640 1 --2- 172.21.15.7:0/3682575820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b7df737a00 0x55b7df73c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.284+0000 7fe218d5e640 1 -- 172.21.15.7:0/3682575820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7df73cb70 msgr2=0x55b7df7486d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.284+0000 7fe218d5e640 1 --2- 172.21.15.7:0/3682575820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7df73cb70 0x55b7df7486d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.284+0000 7fe218d5e640 1 -- 172.21.15.7:0/3682575820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7df736b70 con 0x55b7df735360 2024-06-10T01:11:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.284+0000 7fe218d5e640 1 --2- 172.21.15.7:0/3682575820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7df735360 0x55b7df7357e0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe20c01e770 tx=0x7fe20c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85d49bee7b6c81a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.284+0000 7fe1f9ffb640 1 -- 172.21.15.7:0/3682575820 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c011740 con 0x55b7df735360 2024-06-10T01:11:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.284+0000 7fe1f9ffb640 1 -- 172.21.15.7:0/3682575820 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe20c011dc0 con 0x55b7df735360 2024-06-10T01:11:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.284+0000 7fe1f9ffb640 1 -- 172.21.15.7:0/3682575820 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c010430 con 0x55b7df735360 2024-06-10T01:11:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.285+0000 7fe21d5a9740 1 -- 172.21.15.7:0/3682575820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7df735360 msgr2=0x55b7df7357e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.285+0000 7fe21d5a9740 1 --2- 172.21.15.7:0/3682575820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7df735360 0x55b7df7357e0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe20c01e770 tx=0x7fe20c019000 comp rx=0 tx=0).stop 2024-06-10T01:11:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.285+0000 7fe21d5a9740 1 -- 172.21.15.7:0/3682575820 shutdown_connections 2024-06-10T01:11:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.285+0000 7fe21d5a9740 1 --2- 172.21.15.7:0/3682575820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7df735360 0x55b7df7357e0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.285+0000 7fe21d5a9740 1 --2- 172.21.15.7:0/3682575820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7df73cb70 0x55b7df7486d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.285+0000 7fe21d5a9740 1 --2- 172.21.15.7:0/3682575820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b7df737a00 0x55b7df73c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.285+0000 7fe21d5a9740 1 -- 172.21.15.7:0/3682575820 >> 172.21.15.7:0/3682575820 conn(0x55b7df732680 msgr2=0x55b7df732a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.285+0000 7fe21d5a9740 1 -- 172.21.15.7:0/3682575820 shutdown_connections 2024-06-10T01:11:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.285+0000 7fe21d5a9740 1 -- 172.21.15.7:0/3682575820 wait complete. 2024-06-10T01:11:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.285+0000 7fe21d5a9740 1 -- start start 2024-06-10T01:11:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.285+0000 7fe21d5a9740 1 Processor -- start 2024-06-10T01:11:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.286+0000 7fe21d5a9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b7df735360 0x55b7df748700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.286+0000 7fe21d5a9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7df737a00 0x55b7df7451a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.286+0000 7fe21d5a9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7df73cb70 0x55b7df7457d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.286+0000 7fe21d5a9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe2141aaea0 con 0x55b7df73cb70 2024-06-10T01:11:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.286+0000 7fe21d5a9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe2141ac310 con 0x55b7df737a00 2024-06-10T01:11:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.286+0000 7fe21d5a9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe2141ab570 con 0x55b7df735360 2024-06-10T01:11:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.286+0000 7fe21d5a9740 10 client.0 _ll_register_callbacks cb 0x7fe204009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:11:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.286+0000 7fe218d5e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7df73cb70 0x55b7df7457d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.286+0000 7fe218d5e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7df73cb70 0x55b7df7457d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47810/0 (socket says 172.21.15.7:47810) 2024-06-10T01:11:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.286+0000 7fe218d5e640 1 -- 172.21.15.7:0/1384394360 learned_addr learned my addr 172.21.15.7:0/1384394360 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.286+0000 7fe2137fe640 1 --2- 172.21.15.7:0/1384394360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7df737a00 0x55b7df7451a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.286+0000 7fe213fff640 1 --2- 172.21.15.7:0/1384394360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b7df735360 0x55b7df748700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.286+0000 7fe218d5e640 1 -- 172.21.15.7:0/1384394360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b7df735360 msgr2=0x55b7df748700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.286+0000 7fe218d5e640 1 --2- 172.21.15.7:0/1384394360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b7df735360 0x55b7df748700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.286+0000 7fe218d5e640 1 -- 172.21.15.7:0/1384394360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7df737a00 msgr2=0x55b7df7451a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.286+0000 7fe218d5e640 1 --2- 172.21.15.7:0/1384394360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7df737a00 0x55b7df7451a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.286+0000 7fe218d5e640 1 -- 172.21.15.7:0/1384394360 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7df7324b0 con 0x55b7df73cb70 2024-06-10T01:11:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.286+0000 7fe2137fe640 1 --2- 172.21.15.7:0/1384394360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7df737a00 0x55b7df7451a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:11:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.286+0000 7fe218d5e640 1 --2- 172.21.15.7:0/1384394360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7df73cb70 0x55b7df7457d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe20c004820 tx=0x7fe20c01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed211a98fc31c099 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.286+0000 7fe1f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:11:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.286+0000 7fe1f37fe640 1 -- 172.21.15.7:0/1384394360 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c036060 con 0x55b7df73cb70 2024-06-10T01:11:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.286+0000 7fe1f37fe640 1 -- 172.21.15.7:0/1384394360 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fe20c011940 con 0x55b7df73cb70 2024-06-10T01:11:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.286+0000 7fe21d5a9740 1 -- 172.21.15.7:0/1384394360 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe20c011dc0 con 0x55b7df73cb70 2024-06-10T01:11:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.287+0000 7fe1f37fe640 1 -- 172.21.15.7:0/1384394360 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe20c03d050 con 0x55b7df73cb70 2024-06-10T01:11:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.287+0000 7fe21d5a9740 10 client.16923 fetch_fsmap learned FSMap version 32 2024-06-10T01:11:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.287+0000 7fe21d5a9740 1 -- 172.21.15.7:0/1384394360 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b7df7b6fc0 con 0x55b7df73cb70 2024-06-10T01:11:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.287+0000 7fe1f37fe640 1 -- 172.21.15.7:0/1384394360 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe20c011dc0 con 0x55b7df73cb70 2024-06-10T01:11:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.287+0000 7fe1f37fe640 1 -- 172.21.15.7:0/1384394360 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fe20c010430 con 0x55b7df73cb70 2024-06-10T01:11:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.287+0000 7fe21d5a9740 10 client.16923 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:11:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.287+0000 7fe21d5a9740 10 client.16923 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:11:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.287+0000 7fe21d5a9740 20 client.16923 populate_metadata read hostname 'smithi007' 2024-06-10T01:11:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.287+0000 7fe21d5a9740 1 --2- 172.21.15.7:0/1384394360 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b7df7b7990 0x55b7df7b9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.287+0000 7fe21d5a9740 4 client.16923 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:11:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.287+0000 7fe21d5a9740 1 -- 172.21.15.7:0/1384394360 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b7df6a7c20 con 0x55b7df7b7990 2024-06-10T01:11:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.287+0000 7fe213fff640 1 --2- 172.21.15.7:0/1384394360 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b7df7b7990 0x55b7df7b9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.288+0000 7fe213fff640 1 --2- 172.21.15.7:0/1384394360 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b7df7b7990 0x55b7df7b9db0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1ff87b0d64fe200 server_cookie=5ad7f4c944b14287 in_seq=0 out_seq=0 2024-06-10T01:11:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.288+0000 7fe1f37fe640 10 client.16923 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.360+0000 7fe1f37fe640 1 -- 172.21.15.7:0/1384394360 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b7df6a7c20 con 0x55b7df7b7990 2024-06-10T01:11:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.360+0000 7fe1f37fe640 10 client.16923 handle_command_reply: tid=0 2024-06-10T01:11:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.360+0000 7fe21d5a9740 1 client.16923 shutdown 2024-06-10T01:11:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.360+0000 7fe21d5a9740 1 -- 172.21.15.7:0/1384394360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7df73cb70 msgr2=0x55b7df7457d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.360+0000 7fe21d5a9740 1 --2- 172.21.15.7:0/1384394360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7df73cb70 0x55b7df7457d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe20c004820 tx=0x7fe20c01c910 comp rx=0 tx=0).stop 2024-06-10T01:11:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.361+0000 7fe21d5a9740 1 -- 172.21.15.7:0/1384394360 shutdown_connections 2024-06-10T01:11:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.361+0000 7fe21d5a9740 1 --2- 172.21.15.7:0/1384394360 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b7df7b7990 0x55b7df7b9db0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.361+0000 7fe21d5a9740 1 --2- 172.21.15.7:0/1384394360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7df73cb70 0x55b7df7457d0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.361+0000 7fe21d5a9740 1 --2- 172.21.15.7:0/1384394360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7df737a00 0x55b7df7451a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.361+0000 7fe1f37fe640 0 client.16923 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.361+0000 7fe21d5a9740 1 --2- 172.21.15.7:0/1384394360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b7df735360 0x55b7df748700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.361+0000 7fe21d5a9740 1 -- 172.21.15.7:0/1384394360 >> 172.21.15.7:0/1384394360 conn(0x55b7df732680 msgr2=0x55b7df738bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.361+0000 7fe21d5a9740 1 -- 172.21.15.7:0/1384394360 shutdown_connections 2024-06-10T01:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.361+0000 7fe21d5a9740 1 -- 172.21.15.7:0/1384394360 wait complete. 2024-06-10T01:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.361+0000 7fe21d5a9740 20 client.16923 trim_cache size 0 max 16384 2024-06-10T01:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:11:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.361+0000 7fe21a7c1640 1 -- 172.21.15.7:0/4231011997 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe1fc0a2b80 msgr2=0x7fe1fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.361+0000 7fe21a7c1640 1 --2- 172.21.15.7:0/4231011997 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe1fc0a2b80 0x7fe1fc0a5040 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x55b7df738260 tx=0x7fe20c0096a0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.361+0000 7fe21a7c1640 1 -- 172.21.15.7:0/4231011997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe214104df0 msgr2=0x7fe214100470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.361+0000 7fe21a7c1640 1 --2- 172.21.15.7:0/4231011997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe214104df0 0x7fe214100470 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fe20800e9c0 tx=0x7fe20800ee80 comp rx=0 tx=0).stop 2024-06-10T01:11:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.361+0000 7fe21a7c1640 1 -- 172.21.15.7:0/4231011997 shutdown_connections 2024-06-10T01:11:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.361+0000 7fe21a7c1640 1 --2- 172.21.15.7:0/4231011997 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe1fc0a2b80 0x7fe1fc0a5040 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.361+0000 7fe21a7c1640 1 --2- 172.21.15.7:0/4231011997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe214107720 0x7fe2141009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.361+0000 7fe21a7c1640 1 --2- 172.21.15.7:0/4231011997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe214104df0 0x7fe214100470 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.361+0000 7fe21a7c1640 1 --2- 172.21.15.7:0/4231011997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2141024c0 0x7fe214101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.361+0000 7fe21a7c1640 1 -- 172.21.15.7:0/4231011997 >> 172.21.15.7:0/4231011997 conn(0x7fe2140fbed0 msgr2=0x7fe214102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.361+0000 7fe21a7c1640 1 -- 172.21.15.7:0/4231011997 shutdown_connections 2024-06-10T01:11:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:34.361+0000 7fe21a7c1640 1 -- 172.21.15.7:0/4231011997 wait complete. 2024-06-10T01:11:34.666 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '55ac2137', quiesce took 6.1 seconds. Will release after: 23.35 2024-06-10T01:11:34.731 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 7.32 MB/sec execute 135 sec latency 884.719 ms 2024-06-10T01:11:35.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:35 smithi175 ceph-mon[36636]: pgmap v599: 129 pgs: 129 active+clean; 69 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 149 op/s 2024-06-10T01:11:35.731 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 7.27 MB/sec execute 136 sec latency 1884.762 ms 2024-06-10T01:11:35.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:35 smithi007 ceph-mon[32398]: pgmap v599: 129 pgs: 129 active+clean; 69 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 149 op/s 2024-06-10T01:11:35.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:35 smithi078 ceph-mon[36683]: pgmap v599: 129 pgs: 129 active+clean; 69 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 149 op/s 2024-06-10T01:11:36.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:11:36.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:11:36.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:11:36.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:11:36.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:11:36.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:11:36.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:11:36.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:11:36.731 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 7.21 MB/sec execute 137 sec latency 2884.862 ms 2024-06-10T01:11:36.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:11:36.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:11:36.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:11:36.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:11:36.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:11:36.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:11:36.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:11:36.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:11:36.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:11:36.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:11:36.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:11:36.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:11:36.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:11:36.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:11:36.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:11:36.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:11:37.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:37 smithi175 ceph-mon[36636]: pgmap v600: 129 pgs: 129 active+clean; 70 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 215 op/s 2024-06-10T01:11:37.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:37 smithi175 ceph-mon[36636]: Adjusting osd_memory_target on smithi175 to 176.2M 2024-06-10T01:11:37.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:37 smithi175 ceph-mon[36636]: Unable to set osd_memory_target on smithi175 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-10T01:11:37.731 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 7.16 MB/sec execute 138 sec latency 3884.956 ms 2024-06-10T01:11:37.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:37 smithi007 ceph-mon[32398]: pgmap v600: 129 pgs: 129 active+clean; 70 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 215 op/s 2024-06-10T01:11:37.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:37 smithi007 ceph-mon[32398]: Adjusting osd_memory_target on smithi175 to 176.2M 2024-06-10T01:11:37.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:37 smithi007 ceph-mon[32398]: Unable to set osd_memory_target on smithi175 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-10T01:11:37.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:37 smithi078 ceph-mon[36683]: pgmap v600: 129 pgs: 129 active+clean; 70 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 215 op/s 2024-06-10T01:11:37.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:37 smithi078 ceph-mon[36683]: Adjusting osd_memory_target on smithi175 to 176.2M 2024-06-10T01:11:37.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:37 smithi078 ceph-mon[36683]: Unable to set osd_memory_target on smithi175 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-10T01:11:38.731 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 7.11 MB/sec execute 139 sec latency 4885.068 ms 2024-06-10T01:11:39.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:39 smithi175 ceph-mon[36636]: pgmap v601: 129 pgs: 129 active+clean; 70 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 216 op/s 2024-06-10T01:11:39.731 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 7.06 MB/sec execute 140 sec latency 5885.196 ms 2024-06-10T01:11:39.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:39 smithi007 ceph-mon[32398]: pgmap v601: 129 pgs: 129 active+clean; 70 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 216 op/s 2024-06-10T01:11:39.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:39 smithi078 ceph-mon[36683]: pgmap v601: 129 pgs: 129 active+clean; 70 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 216 op/s 2024-06-10T01:11:40.731 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 7.01 MB/sec execute 141 sec latency 6885.299 ms 2024-06-10T01:11:41.731 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 6.96 MB/sec execute 142 sec latency 7885.374 ms 2024-06-10T01:11:41.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:41 smithi007 ceph-mon[32398]: pgmap v602: 129 pgs: 129 active+clean; 66 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 223 op/s 2024-06-10T01:11:41.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:41 smithi078 ceph-mon[36683]: pgmap v602: 129 pgs: 129 active+clean; 66 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 223 op/s 2024-06-10T01:11:41.953 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:41 smithi175 ceph-mon[36636]: pgmap v602: 129 pgs: 129 active+clean; 66 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 223 op/s 2024-06-10T01:11:42.731 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 6.91 MB/sec execute 143 sec latency 8885.474 ms 2024-06-10T01:11:43.732 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 6.86 MB/sec execute 144 sec latency 9885.608 ms 2024-06-10T01:11:43.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:43 smithi007 ceph-mon[32398]: pgmap v603: 129 pgs: 129 active+clean; 65 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 180 op/s 2024-06-10T01:11:43.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:43 smithi078 ceph-mon[36683]: pgmap v603: 129 pgs: 129 active+clean; 65 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 180 op/s 2024-06-10T01:11:43.953 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:43 smithi175 ceph-mon[36636]: pgmap v603: 129 pgs: 129 active+clean; 65 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 180 op/s 2024-06-10T01:11:44.732 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 6.82 MB/sec execute 145 sec latency 10885.683 ms 2024-06-10T01:11:45.732 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 6.77 MB/sec execute 146 sec latency 11885.799 ms 2024-06-10T01:11:45.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:45 smithi007 ceph-mon[32398]: pgmap v604: 129 pgs: 129 active+clean; 65 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 895 KiB/s wr, 163 op/s 2024-06-10T01:11:45.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:45 smithi078 ceph-mon[36683]: pgmap v604: 129 pgs: 129 active+clean; 65 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 895 KiB/s wr, 163 op/s 2024-06-10T01:11:45.953 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:45 smithi175 ceph-mon[36636]: pgmap v604: 129 pgs: 129 active+clean; 65 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 895 KiB/s wr, 163 op/s 2024-06-10T01:11:46.732 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 6.72 MB/sec execute 147 sec latency 12885.898 ms 2024-06-10T01:11:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.462+0000 7f667a9d5640 1 -- 172.21.15.7:0/2833088478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66741098e0 msgr2=0x7f667410bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.462+0000 7f667a9d5640 1 --2- 172.21.15.7:0/2833088478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66741098e0 0x7f667410bd70 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f666c009ca0 tx=0x7f666c02a950 comp rx=0 tx=0).stop 2024-06-10T01:11:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.462+0000 7f667a9d5640 1 -- 172.21.15.7:0/2833088478 shutdown_connections 2024-06-10T01:11:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.462+0000 7f667a9d5640 1 --2- 172.21.15.7:0/2833088478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66741098e0 0x7f667410bd70 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.462+0000 7f667a9d5640 1 --2- 172.21.15.7:0/2833088478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6674074380 0x7f66740747e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.462+0000 7f667a9d5640 1 --2- 172.21.15.7:0/2833088478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6674075960 0x7f6674073db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.462+0000 7f667a9d5640 1 -- 172.21.15.7:0/2833088478 >> 172.21.15.7:0/2833088478 conn(0x7f66740fcfc0 msgr2=0x7f66740ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.462+0000 7f667a9d5640 1 -- 172.21.15.7:0/2833088478 shutdown_connections 2024-06-10T01:11:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.462+0000 7f667a9d5640 1 -- 172.21.15.7:0/2833088478 wait complete. 2024-06-10T01:11:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.462+0000 7f667a9d5640 1 Processor -- start 2024-06-10T01:11:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.462+0000 7f667a9d5640 1 -- start start 2024-06-10T01:11:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.463+0000 7f667a9d5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6674074380 0x7f6674199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.463+0000 7f667a9d5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6674075960 0x7f667419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.463+0000 7f667a9d5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66741098e0 0x7f66741a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.463+0000 7f667a9d5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f667410f8a0 con 0x7f6674074380 2024-06-10T01:11:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.463+0000 7f667a9d5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f667410f720 con 0x7f6674075960 2024-06-10T01:11:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.463+0000 7f667a9d5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f667410fa20 con 0x7f66741098e0 2024-06-10T01:11:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.463+0000 7f6673fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6674074380 0x7f6674199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.463+0000 7f6673fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6674074380 0x7f6674199df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54696/0 (socket says 172.21.15.7:54696) 2024-06-10T01:11:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.463+0000 7f6673fff640 1 -- 172.21.15.7:0/2953005436 learned_addr learned my addr 172.21.15.7:0/2953005436 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.463+0000 7f66737fe640 1 --2- 172.21.15.7:0/2953005436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6674075960 0x7f667419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.463+0000 7f6678f72640 1 --2- 172.21.15.7:0/2953005436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66741098e0 0x7f66741a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.463+0000 7f6673fff640 1 -- 172.21.15.7:0/2953005436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66741098e0 msgr2=0x7f66741a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.463+0000 7f6673fff640 1 --2- 172.21.15.7:0/2953005436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66741098e0 0x7f66741a13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.463+0000 7f6673fff640 1 -- 172.21.15.7:0/2953005436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6674075960 msgr2=0x7f667419a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.463+0000 7f6673fff640 1 --2- 172.21.15.7:0/2953005436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6674075960 0x7f667419a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.463+0000 7f6673fff640 1 -- 172.21.15.7:0/2953005436 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66741a1a10 con 0x7f6674074380 2024-06-10T01:11:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.464+0000 7f6673fff640 1 --2- 172.21.15.7:0/2953005436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6674074380 0x7f6674199df0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f666400c920 tx=0x7f666400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.464+0000 7f66717fa640 1 -- 172.21.15.7:0/2953005436 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664007c70 con 0x7f6674074380 2024-06-10T01:11:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.464+0000 7f66717fa640 1 -- 172.21.15.7:0/2953005436 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6664007e10 con 0x7f6674074380 2024-06-10T01:11:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.464+0000 7f667a9d5640 1 -- 172.21.15.7:0/2953005436 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66741a1d00 con 0x7f6674074380 2024-06-10T01:11:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.464+0000 7f66717fa640 1 -- 172.21.15.7:0/2953005436 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664018890 con 0x7f6674074380 2024-06-10T01:11:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.464+0000 7f667a9d5640 1 -- 172.21.15.7:0/2953005436 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66741a21e0 con 0x7f6674074380 2024-06-10T01:11:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.466+0000 7f66717fa640 1 -- 172.21.15.7:0/2953005436 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f6664018a70 con 0x7f6674074380 2024-06-10T01:11:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.466+0000 7f667d7be740 1 Processor -- start 2024-06-10T01:11:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.466+0000 7f667d7be740 1 -- start start 2024-06-10T01:11:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.467+0000 7f667d7be740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55982a4b4010 0x55982a4b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.467+0000 7f667d7be740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55982a4b4a00 0x55982a4b9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.467+0000 7f667d7be740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982a4b9b70 0x55982a4bc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.467+0000 7f667d7be740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55982a4174f0 con 0x55982a4b4a00 2024-06-10T01:11:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.467+0000 7f667d7be740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55982a3b5870 con 0x55982a4b9b70 2024-06-10T01:11:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.467+0000 7f667d7be740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55982a4134b0 con 0x55982a4b4010 2024-06-10T01:11:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.467+0000 7f6673fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55982a4b4a00 0x55982a4b9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.467+0000 7f6673fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55982a4b4a00 0x55982a4b9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54708/0 (socket says 172.21.15.7:54708) 2024-06-10T01:11:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.467+0000 7f6673fff640 1 -- 172.21.15.7:0/2240723727 learned_addr learned my addr 172.21.15.7:0/2240723727 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.467+0000 7f6678f72640 1 --2- 172.21.15.7:0/2240723727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55982a4b4010 0x55982a4b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.467+0000 7f66737fe640 1 --2- 172.21.15.7:0/2240723727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982a4b9b70 0x55982a4bc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.467+0000 7f66717fa640 1 --2- 172.21.15.7:0/2953005436 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f66600a2c00 0x7f66600a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.467+0000 7f6673fff640 1 -- 172.21.15.7:0/2240723727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55982a4b4010 msgr2=0x55982a4b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.467+0000 7f6673fff640 1 --2- 172.21.15.7:0/2240723727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55982a4b4010 0x55982a4b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.467+0000 7f6673fff640 1 -- 172.21.15.7:0/2240723727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982a4b9b70 msgr2=0x55982a4bc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.467+0000 7f6673fff640 1 --2- 172.21.15.7:0/2240723727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982a4b9b70 0x55982a4bc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.467+0000 7f6673fff640 1 -- 172.21.15.7:0/2240723727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55982a3d19b0 con 0x55982a4b4a00 2024-06-10T01:11:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.467+0000 7f6678f72640 1 --2- 172.21.15.7:0/2953005436 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f66600a2c00 0x7f66600a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.467+0000 7f66717fa640 1 -- 172.21.15.7:0/2953005436 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f66640cf3a0 con 0x7f6674074380 2024-06-10T01:11:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.467+0000 7f6673fff640 1 --2- 172.21.15.7:0/2240723727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55982a4b4a00 0x55982a4b9500 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f66640aee20 tx=0x7f6664097bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62fa136614387b41 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.468+0000 7f666a7fc640 1 -- 172.21.15.7:0/2240723727 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666409b5f0 con 0x55982a4b4a00 2024-06-10T01:11:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.468+0000 7f666a7fc640 1 -- 172.21.15.7:0/2240723727 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66640afd40 con 0x55982a4b4a00 2024-06-10T01:11:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.468+0000 7f666a7fc640 1 -- 172.21.15.7:0/2240723727 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66640a3190 con 0x55982a4b4a00 2024-06-10T01:11:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.468+0000 7f667d7be740 1 -- 172.21.15.7:0/2240723727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55982a4b4a00 msgr2=0x55982a4b9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.468+0000 7f6678f72640 1 --2- 172.21.15.7:0/2953005436 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f66600a2c00 0x7f66600a50c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f666c005fe0 tx=0x7f666c03c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.468+0000 7f667d7be740 1 --2- 172.21.15.7:0/2240723727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55982a4b4a00 0x55982a4b9500 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f66640aee20 tx=0x7f6664097bf0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.469+0000 7f667d7be740 1 -- 172.21.15.7:0/2240723727 shutdown_connections 2024-06-10T01:11:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.469+0000 7f667d7be740 1 --2- 172.21.15.7:0/2240723727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982a4b9b70 0x55982a4bc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.469+0000 7f667d7be740 1 --2- 172.21.15.7:0/2240723727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55982a4b4a00 0x55982a4b9500 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.469+0000 7f667d7be740 1 --2- 172.21.15.7:0/2240723727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55982a4b4010 0x55982a4b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.469+0000 7f667d7be740 1 -- 172.21.15.7:0/2240723727 >> 172.21.15.7:0/2240723727 conn(0x55982a4af680 msgr2=0x55982a4afa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.469+0000 7f667d7be740 1 -- 172.21.15.7:0/2240723727 shutdown_connections 2024-06-10T01:11:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.469+0000 7f667d7be740 1 -- 172.21.15.7:0/2240723727 wait complete. 2024-06-10T01:11:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.469+0000 7f667d7be740 1 -- start start 2024-06-10T01:11:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.469+0000 7f667d7be740 1 Processor -- start 2024-06-10T01:11:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.470+0000 7f667d7be740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982a4b4010 0x55982a4be2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.470+0000 7f667d7be740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55982a4b4a00 0x55982a4be7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.470+0000 7f667d7be740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55982a4b9b70 0x55982a4c5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.470+0000 7f667d7be740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55982a4bcb20 con 0x55982a4b4a00 2024-06-10T01:11:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.470+0000 7f667d7be740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55982a3a0ae0 con 0x55982a4b4010 2024-06-10T01:11:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.470+0000 7f667d7be740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55982a4bcca0 con 0x55982a4b9b70 2024-06-10T01:11:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.470+0000 7f667d7be740 10 client.0 _ll_register_callbacks cb 0x55982a3d99b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:11:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.470+0000 7f66737fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55982a4b4a00 0x55982a4be7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.470+0000 7f6673fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982a4b4010 0x55982a4be2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.470+0000 7f66737fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55982a4b4a00 0x55982a4be7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54710/0 (socket says 172.21.15.7:54710) 2024-06-10T01:11:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.470+0000 7f66737fe640 1 -- 172.21.15.7:0/2911581623 learned_addr learned my addr 172.21.15.7:0/2911581623 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.470+0000 7f6678f72640 1 --2- 172.21.15.7:0/2911581623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55982a4b9b70 0x55982a4c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.470+0000 7f66737fe640 1 -- 172.21.15.7:0/2911581623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55982a4b9b70 msgr2=0x55982a4c5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.470+0000 7f66737fe640 1 --2- 172.21.15.7:0/2911581623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55982a4b9b70 0x55982a4c5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.470+0000 7f66737fe640 1 -- 172.21.15.7:0/2911581623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982a4b4010 msgr2=0x55982a4be2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.470+0000 7f66737fe640 1 --2- 172.21.15.7:0/2911581623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982a4b4010 0x55982a4be2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.470+0000 7f66737fe640 1 -- 172.21.15.7:0/2911581623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55982a12b210 con 0x55982a4b4a00 2024-06-10T01:11:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.470+0000 7f66737fe640 1 --2- 172.21.15.7:0/2911581623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55982a4b4a00 0x55982a4be7f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f665c011440 tx=0x7f665c011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75258c8e93c94d19 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.470+0000 7f66537fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:11:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.471+0000 7f66537fe640 1 -- 172.21.15.7:0/2911581623 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c004020 con 0x55982a4b4a00 2024-06-10T01:11:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.471+0000 7f66537fe640 1 -- 172.21.15.7:0/2911581623 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f665c0041c0 con 0x55982a4b4a00 2024-06-10T01:11:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.471+0000 7f667d7be740 1 -- 172.21.15.7:0/2911581623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66640afd40 con 0x55982a4b4a00 2024-06-10T01:11:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.472+0000 7f66537fe640 1 -- 172.21.15.7:0/2911581623 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f665c024050 con 0x55982a4b4a00 2024-06-10T01:11:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.472+0000 7f667d7be740 10 client.16935 fetch_fsmap learned FSMap version 32 2024-06-10T01:11:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.472+0000 7f667d7be740 1 -- 172.21.15.7:0/2911581623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55982a1f7620 con 0x55982a4b4a00 2024-06-10T01:11:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.472+0000 7f66537fe640 1 -- 172.21.15.7:0/2911581623 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66640afd40 con 0x55982a4b4a00 2024-06-10T01:11:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.472+0000 7f66537fe640 1 -- 172.21.15.7:0/2911581623 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f665c003950 con 0x55982a4b4a00 2024-06-10T01:11:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.472+0000 7f667d7be740 10 client.16935 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:11:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.472+0000 7f667d7be740 10 client.16935 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:11:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.472+0000 7f667d7be740 20 client.16935 populate_metadata read hostname 'smithi007' 2024-06-10T01:11:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.472+0000 7f667d7be740 1 --2- 172.21.15.7:0/2911581623 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55982a4ca180 0x55982a4cc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.472+0000 7f667d7be740 4 client.16935 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:11:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.472+0000 7f667d7be740 1 -- 172.21.15.7:0/2911581623 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55982a42af10 con 0x55982a4ca180 2024-06-10T01:11:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.472+0000 7f6673fff640 1 --2- 172.21.15.7:0/2911581623 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55982a4ca180 0x55982a4cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.473+0000 7f6673fff640 1 --2- 172.21.15.7:0/2911581623 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55982a4ca180 0x55982a4cc600 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79798e56d0fb6eff server_cookie=6ba40d819056e297 in_seq=0 out_seq=0 2024-06-10T01:11:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.473+0000 7f66537fe640 10 client.16935 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.474+0000 7f66537fe640 1 -- 172.21.15.7:0/2911581623 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55982a42af10 con 0x55982a4ca180 2024-06-10T01:11:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.474+0000 7f66537fe640 10 client.16935 handle_command_reply: tid=0 2024-06-10T01:11:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.474+0000 7f667d7be740 1 client.16935 shutdown 2024-06-10T01:11:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.474+0000 7f667d7be740 1 -- 172.21.15.7:0/2911581623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55982a4b4a00 msgr2=0x55982a4be7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.474+0000 7f667d7be740 1 --2- 172.21.15.7:0/2911581623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55982a4b4a00 0x55982a4be7f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f665c011440 tx=0x7f665c011900 comp rx=0 tx=0).stop 2024-06-10T01:11:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.474+0000 7f667d7be740 1 -- 172.21.15.7:0/2911581623 shutdown_connections 2024-06-10T01:11:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.474+0000 7f667d7be740 1 --2- 172.21.15.7:0/2911581623 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55982a4ca180 0x55982a4cc600 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.474+0000 7f667d7be740 1 --2- 172.21.15.7:0/2911581623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55982a4b9b70 0x55982a4c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.474+0000 7f667d7be740 1 --2- 172.21.15.7:0/2911581623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55982a4b4a00 0x55982a4be7f0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.474+0000 7f667d7be740 1 --2- 172.21.15.7:0/2911581623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982a4b4010 0x55982a4be2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.474+0000 7f66537fe640 0 client.16935 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.474+0000 7f667d7be740 1 -- 172.21.15.7:0/2911581623 >> 172.21.15.7:0/2911581623 conn(0x55982a4af680 msgr2=0x55982a4ae7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.474+0000 7f667d7be740 1 -- 172.21.15.7:0/2911581623 shutdown_connections 2024-06-10T01:11:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.474+0000 7f667d7be740 1 -- 172.21.15.7:0/2911581623 wait complete. 2024-06-10T01:11:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.474+0000 7f667d7be740 20 client.16935 trim_cache size 0 max 16384 2024-06-10T01:11:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.489+0000 7f667d7be740 1 Processor -- start 2024-06-10T01:11:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.489+0000 7f667d7be740 1 -- start start 2024-06-10T01:11:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.490+0000 7f667d7be740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55982a4b4010 0x55982a4bd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.490+0000 7f667d7be740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982a4b4a00 0x55982a4c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.490+0000 7f667d7be740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55982a4b9b70 0x55982a4b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.490+0000 7f667d7be740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f66741aefe0 con 0x55982a4b4010 2024-06-10T01:11:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.490+0000 7f667d7be740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f66741ac8b0 con 0x55982a4b4a00 2024-06-10T01:11:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.490+0000 7f667d7be740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f66741aa700 con 0x55982a4b9b70 2024-06-10T01:11:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.490+0000 7f6673fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55982a4b4010 0x55982a4bd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.490+0000 7f6673fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55982a4b4010 0x55982a4bd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54718/0 (socket says 172.21.15.7:54718) 2024-06-10T01:11:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.490+0000 7f6673fff640 1 -- 172.21.15.7:0/702245754 learned_addr learned my addr 172.21.15.7:0/702245754 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.490+0000 7f6673fff640 1 -- 172.21.15.7:0/702245754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55982a4b9b70 msgr2=0x55982a4b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.490+0000 7f6673fff640 1 --2- 172.21.15.7:0/702245754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55982a4b9b70 0x55982a4b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.490+0000 7f6673fff640 1 -- 172.21.15.7:0/702245754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982a4b4a00 msgr2=0x55982a4c57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.490+0000 7f6673fff640 1 --2- 172.21.15.7:0/702245754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982a4b4a00 0x55982a4c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.490+0000 7f6673fff640 1 -- 172.21.15.7:0/702245754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55982a4af4b0 con 0x55982a4b4010 2024-06-10T01:11:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.490+0000 7f6673fff640 1 --2- 172.21.15.7:0/702245754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55982a4b4010 0x55982a4bd8d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f66640ccac0 tx=0x7f666400cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e1855e10725d7d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.490+0000 7f6669ffb640 1 -- 172.21.15.7:0/702245754 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66640a3df0 con 0x55982a4b4010 2024-06-10T01:11:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.490+0000 7f6669ffb640 1 -- 172.21.15.7:0/702245754 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66640b3040 con 0x55982a4b4010 2024-06-10T01:11:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.490+0000 7f6669ffb640 1 -- 172.21.15.7:0/702245754 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666409b180 con 0x55982a4b4010 2024-06-10T01:11:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.491+0000 7f667d7be740 1 -- 172.21.15.7:0/702245754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55982a4b4010 msgr2=0x55982a4bd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.491+0000 7f667d7be740 1 --2- 172.21.15.7:0/702245754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55982a4b4010 0x55982a4bd8d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f66640ccac0 tx=0x7f666400cf10 comp rx=0 tx=0).stop 2024-06-10T01:11:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.491+0000 7f667d7be740 1 -- 172.21.15.7:0/702245754 shutdown_connections 2024-06-10T01:11:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.491+0000 7f667d7be740 1 --2- 172.21.15.7:0/702245754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55982a4b9b70 0x55982a4b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.491+0000 7f667d7be740 1 --2- 172.21.15.7:0/702245754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982a4b4a00 0x55982a4c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.491+0000 7f667d7be740 1 --2- 172.21.15.7:0/702245754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55982a4b4010 0x55982a4bd8d0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.491+0000 7f667d7be740 1 -- 172.21.15.7:0/702245754 >> 172.21.15.7:0/702245754 conn(0x55982a4af680 msgr2=0x55982a4b92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.491+0000 7f667d7be740 1 -- 172.21.15.7:0/702245754 shutdown_connections 2024-06-10T01:11:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.491+0000 7f667d7be740 1 -- 172.21.15.7:0/702245754 wait complete. 2024-06-10T01:11:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.491+0000 7f667d7be740 1 -- start start 2024-06-10T01:11:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.492+0000 7f667d7be740 1 Processor -- start 2024-06-10T01:11:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.492+0000 7f667d7be740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55982a4b4010 0x55982a4c29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.492+0000 7f667d7be740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982a4b4a00 0x55982a4c2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.492+0000 7f667d7be740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55982a4b9b70 0x55982a4c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.492+0000 7f667d7be740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f66741aad70 con 0x55982a4b9b70 2024-06-10T01:11:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.492+0000 7f667d7be740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f66741ac1e0 con 0x55982a4b4a00 2024-06-10T01:11:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.492+0000 7f667d7be740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f66741ab440 con 0x55982a4b4010 2024-06-10T01:11:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.492+0000 7f667d7be740 10 client.0 _ll_register_callbacks cb 0x7f665c00bc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:11:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.492+0000 7f6678f72640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55982a4b9b70 0x55982a4c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.492+0000 7f6678f72640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55982a4b9b70 0x55982a4c3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54730/0 (socket says 172.21.15.7:54730) 2024-06-10T01:11:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.492+0000 7f6678f72640 1 -- 172.21.15.7:0/1358905184 learned_addr learned my addr 172.21.15.7:0/1358905184 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.492+0000 7f6673fff640 1 --2- 172.21.15.7:0/1358905184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55982a4b4010 0x55982a4c29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.492+0000 7f66737fe640 1 --2- 172.21.15.7:0/1358905184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982a4b4a00 0x55982a4c2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.492+0000 7f6678f72640 1 -- 172.21.15.7:0/1358905184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55982a4b4010 msgr2=0x55982a4c29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.492+0000 7f6678f72640 1 --2- 172.21.15.7:0/1358905184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55982a4b4010 0x55982a4c29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.492+0000 7f6678f72640 1 -- 172.21.15.7:0/1358905184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982a4b4a00 msgr2=0x55982a4c2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.492+0000 7f6678f72640 1 --2- 172.21.15.7:0/1358905184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982a4b4a00 0x55982a4c2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.492+0000 7f6678f72640 1 -- 172.21.15.7:0/1358905184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55982a4b0350 con 0x55982a4b9b70 2024-06-10T01:11:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.492+0000 7f6673fff640 1 --2- 172.21.15.7:0/1358905184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55982a4b4010 0x55982a4c29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:11:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.492+0000 7f66737fe640 1 --2- 172.21.15.7:0/1358905184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982a4b4a00 0x55982a4c2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:11:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.492+0000 7f6678f72640 1 --2- 172.21.15.7:0/1358905184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55982a4b9b70 0x55982a4c3f20 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f666c009ca0 tx=0x7f666c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1d26429f02bf81 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.492+0000 7f66537fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:11:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.493+0000 7f66537fe640 1 -- 172.21.15.7:0/1358905184 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c056060 con 0x55982a4b9b70 2024-06-10T01:11:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.493+0000 7f66537fe640 1 -- 172.21.15.7:0/1358905184 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f666c030ba0 con 0x55982a4b9b70 2024-06-10T01:11:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.493+0000 7f667d7be740 1 -- 172.21.15.7:0/1358905184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66640b3040 con 0x55982a4b9b70 2024-06-10T01:11:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.493+0000 7f66537fe640 1 -- 172.21.15.7:0/1358905184 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f666c05d050 con 0x55982a4b9b70 2024-06-10T01:11:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.493+0000 7f667d7be740 10 client.16941 fetch_fsmap learned FSMap version 32 2024-06-10T01:11:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.493+0000 7f667d7be740 1 -- 172.21.15.7:0/1358905184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55982a4c56c0 con 0x55982a4b9b70 2024-06-10T01:11:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.493+0000 7f66537fe640 1 -- 172.21.15.7:0/1358905184 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66640b3040 con 0x55982a4b9b70 2024-06-10T01:11:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.493+0000 7f66537fe640 1 -- 172.21.15.7:0/1358905184 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f666c051420 con 0x55982a4b9b70 2024-06-10T01:11:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.493+0000 7f667d7be740 10 client.16941 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:11:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.493+0000 7f667d7be740 10 client.16941 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:11:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.493+0000 7f667d7be740 20 client.16941 populate_metadata read hostname 'smithi007' 2024-06-10T01:11:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.493+0000 7f667d7be740 1 --2- 172.21.15.7:0/1358905184 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55982a5352f0 0x55982a537710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.493+0000 7f667d7be740 4 client.16941 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:11:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.493+0000 7f667d7be740 1 -- 172.21.15.7:0/1358905184 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55982a424c20 con 0x55982a5352f0 2024-06-10T01:11:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.494+0000 7f6673fff640 1 --2- 172.21.15.7:0/1358905184 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55982a5352f0 0x55982a537710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.494+0000 7f66537fe640 10 client.16941 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.494+0000 7f6673fff640 1 --2- 172.21.15.7:0/1358905184 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55982a5352f0 0x55982a537710 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e6ba52ee0a52d71 server_cookie=b1d5a4e83b276850 in_seq=0 out_seq=0 2024-06-10T01:11:47.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.522+0000 7f66537fe640 1 -- 172.21.15.7:0/1358905184 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55982a424c20 con 0x55982a5352f0 2024-06-10T01:11:47.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.522+0000 7f66537fe640 10 client.16941 handle_command_reply: tid=0 2024-06-10T01:11:47.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.522+0000 7f667d7be740 1 client.16941 shutdown 2024-06-10T01:11:47.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.523+0000 7f667d7be740 1 -- 172.21.15.7:0/1358905184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55982a4b9b70 msgr2=0x55982a4c3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:47.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.523+0000 7f667d7be740 1 --2- 172.21.15.7:0/1358905184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55982a4b9b70 0x55982a4c3f20 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f666c009ca0 tx=0x7f666c0095c0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.523+0000 7f667d7be740 1 -- 172.21.15.7:0/1358905184 shutdown_connections 2024-06-10T01:11:47.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.523+0000 7f667d7be740 1 --2- 172.21.15.7:0/1358905184 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55982a5352f0 0x55982a537710 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.523+0000 7f667d7be740 1 --2- 172.21.15.7:0/1358905184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55982a4b9b70 0x55982a4c3f20 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.523+0000 7f667d7be740 1 --2- 172.21.15.7:0/1358905184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982a4b4a00 0x55982a4c2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.523+0000 7f66537fe640 0 client.16941 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:47.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.523+0000 7f667d7be740 1 --2- 172.21.15.7:0/1358905184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55982a4b4010 0x55982a4c29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.523+0000 7f667d7be740 1 -- 172.21.15.7:0/1358905184 >> 172.21.15.7:0/1358905184 conn(0x55982a4af680 msgr2=0x55982a4be290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:47.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.523+0000 7f667d7be740 1 -- 172.21.15.7:0/1358905184 shutdown_connections 2024-06-10T01:11:47.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.523+0000 7f667d7be740 1 -- 172.21.15.7:0/1358905184 wait complete. 2024-06-10T01:11:47.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.523+0000 7f667d7be740 20 client.16941 trim_cache size 0 max 16384 2024-06-10T01:11:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:11:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:11:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:11:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.523+0000 7f667a9d5640 1 -- 172.21.15.7:0/2953005436 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f66600a2c00 msgr2=0x7f66600a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.523+0000 7f667a9d5640 1 --2- 172.21.15.7:0/2953005436 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f66600a2c00 0x7f66600a50c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f666c005fe0 tx=0x7f666c03c040 comp rx=0 tx=0).stop 2024-06-10T01:11:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.523+0000 7f667a9d5640 1 -- 172.21.15.7:0/2953005436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6674074380 msgr2=0x7f6674199df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.523+0000 7f667a9d5640 1 --2- 172.21.15.7:0/2953005436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6674074380 0x7f6674199df0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f666400c920 tx=0x7f666400cde0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.524+0000 7f667a9d5640 1 -- 172.21.15.7:0/2953005436 shutdown_connections 2024-06-10T01:11:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.524+0000 7f667a9d5640 1 --2- 172.21.15.7:0/2953005436 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f66600a2c00 0x7f66600a50c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.524+0000 7f667a9d5640 1 --2- 172.21.15.7:0/2953005436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66741098e0 0x7f66741a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.524+0000 7f667a9d5640 1 --2- 172.21.15.7:0/2953005436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6674075960 0x7f667419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.524+0000 7f667a9d5640 1 --2- 172.21.15.7:0/2953005436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6674074380 0x7f6674199df0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.524+0000 7f667a9d5640 1 -- 172.21.15.7:0/2953005436 >> 172.21.15.7:0/2953005436 conn(0x7f66740fcfc0 msgr2=0x7f6674105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.524+0000 7f667a9d5640 1 -- 172.21.15.7:0/2953005436 shutdown_connections 2024-06-10T01:11:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:11:47.524+0000 7f667a9d5640 1 -- 172.21.15.7:0/2953005436 wait complete. 2024-06-10T01:11:47.732 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 6.68 MB/sec execute 148 sec latency 13885.996 ms 2024-06-10T01:11:47.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:47 smithi007 ceph-mon[32398]: pgmap v605: 129 pgs: 129 active+clean; 65 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 387 KiB/s wr, 123 op/s 2024-06-10T01:11:47.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:47 smithi078 ceph-mon[36683]: pgmap v605: 129 pgs: 129 active+clean; 65 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 387 KiB/s wr, 123 op/s 2024-06-10T01:11:47.953 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:47 smithi175 ceph-mon[36636]: pgmap v605: 129 pgs: 129 active+clean; 65 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 387 KiB/s wr, 123 op/s 2024-06-10T01:11:48.732 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 6.63 MB/sec execute 149 sec latency 14886.115 ms 2024-06-10T01:11:49.732 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 6.59 MB/sec execute 150 sec latency 15886.230 ms 2024-06-10T01:11:49.755 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:49 smithi175 ceph-mon[36636]: pgmap v606: 129 pgs: 129 active+clean; 65 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 58 KiB/s wr, 43 op/s 2024-06-10T01:11:49.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:49 smithi007 ceph-mon[32398]: pgmap v606: 129 pgs: 129 active+clean; 65 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 58 KiB/s wr, 43 op/s 2024-06-10T01:11:49.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:49 smithi078 ceph-mon[36683]: pgmap v606: 129 pgs: 129 active+clean; 65 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 58 KiB/s wr, 43 op/s 2024-06-10T01:11:50.126 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:11:50.390 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:11:50.732 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 6.55 MB/sec execute 151 sec latency 16886.284 ms 2024-06-10T01:11:51.732 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 6.50 MB/sec execute 152 sec latency 17886.342 ms 2024-06-10T01:11:51.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:51 smithi007 ceph-mon[32398]: pgmap v607: 129 pgs: 129 active+clean; 65 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 29 op/s 2024-06-10T01:11:51.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:51 smithi078 ceph-mon[36683]: pgmap v607: 129 pgs: 129 active+clean; 65 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 29 op/s 2024-06-10T01:11:51.953 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:51 smithi175 ceph-mon[36636]: pgmap v607: 129 pgs: 129 active+clean; 65 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 29 op/s 2024-06-10T01:11:52.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.183+0000 7fa7c00fa700 1 -- 172.21.15.7:0/355574597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b806abe0 msgr2=0x7fa7b806b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:52.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.183+0000 7fa7c00fa700 1 --2- 172.21.15.7:0/355574597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b806abe0 0x7fa7b806b060 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8009990 tx=0x7fa7a8009ca0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.183+0000 7fa7c00fa700 1 -- 172.21.15.7:0/355574597 shutdown_connections 2024-06-10T01:11:52.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.183+0000 7fa7c00fa700 1 --2- 172.21.15.7:0/355574597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7b806b5a0 0x7fa7b810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.183+0000 7fa7c00fa700 1 --2- 172.21.15.7:0/355574597 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7b8108500 0x7fa7b81088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.183+0000 7fa7c00fa700 1 --2- 172.21.15.7:0/355574597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b806abe0 0x7fa7b806b060 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.183+0000 7fa7c00fa700 1 -- 172.21.15.7:0/355574597 >> 172.21.15.7:0/355574597 conn(0x7fa7b8078360 msgr2=0x7fa7b8078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:52.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.183+0000 7fa7c00fa700 1 -- 172.21.15.7:0/355574597 shutdown_connections 2024-06-10T01:11:52.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.183+0000 7fa7c00fa700 1 -- 172.21.15.7:0/355574597 wait complete. 2024-06-10T01:11:52.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.183+0000 7fa7c00fa700 1 Processor -- start 2024-06-10T01:11:52.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.183+0000 7fa7c00fa700 1 -- start start 2024-06-10T01:11:52.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.184+0000 7fa7c00fa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b806abe0 0x7fa7b8198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:52.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.184+0000 7fa7c00fa700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7b806b5a0 0x7fa7b81992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:52.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.184+0000 7fa7c00fa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7b8108500 0x7fa7b819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:52.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.184+0000 7fa7c00fa700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b8112860 con 0x7fa7b806abe0 2024-06-10T01:11:52.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.184+0000 7fa7c00fa700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b81126e0 con 0x7fa7b8108500 2024-06-10T01:11:52.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.184+0000 7fa7c00fa700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b81129e0 con 0x7fa7b806b5a0 2024-06-10T01:11:52.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.184+0000 7fa7bde96700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b806abe0 0x7fa7b8198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:52.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.184+0000 7fa7bde96700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b806abe0 0x7fa7b8198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37526/0 (socket says 172.21.15.7:37526) 2024-06-10T01:11:52.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.184+0000 7fa7bde96700 1 -- 172.21.15.7:0/2475033932 learned_addr learned my addr 172.21.15.7:0/2475033932 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:52.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.184+0000 7fa7bd695700 1 --2- 172.21.15.7:0/2475033932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7b806b5a0 0x7fa7b81992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:52.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.184+0000 7fa7be697700 1 --2- 172.21.15.7:0/2475033932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7b8108500 0x7fa7b819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:52.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.184+0000 7fa7bde96700 1 -- 172.21.15.7:0/2475033932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7b806b5a0 msgr2=0x7fa7b81992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:52.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.184+0000 7fa7bde96700 1 --2- 172.21.15.7:0/2475033932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7b806b5a0 0x7fa7b81992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.184+0000 7fa7bde96700 1 -- 172.21.15.7:0/2475033932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7b8108500 msgr2=0x7fa7b819d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:52.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.184+0000 7fa7bde96700 1 --2- 172.21.15.7:0/2475033932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7b8108500 0x7fa7b819d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.184+0000 7fa7bde96700 1 -- 172.21.15.7:0/2475033932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b819dd80 con 0x7fa7b806abe0 2024-06-10T01:11:52.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.184+0000 7fa7bde96700 1 --2- 172.21.15.7:0/2475033932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b806abe0 0x7fa7b8198d70 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400b530 tx=0x7fa7b400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:52.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.185+0000 7fa7aeffd700 1 -- 172.21.15.7:0/2475033932 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4015020 con 0x7fa7b806abe0 2024-06-10T01:11:52.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.185+0000 7fa7aeffd700 1 -- 172.21.15.7:0/2475033932 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7b4011d10 con 0x7fa7b806abe0 2024-06-10T01:11:52.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.185+0000 7fa7c00fa700 1 -- 172.21.15.7:0/2475033932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b819e070 con 0x7fa7b806abe0 2024-06-10T01:11:52.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.185+0000 7fa7aeffd700 1 -- 172.21.15.7:0/2475033932 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b400f2e0 con 0x7fa7b806abe0 2024-06-10T01:11:52.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.185+0000 7fa7c00fa700 1 -- 172.21.15.7:0/2475033932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b81a58f0 con 0x7fa7b806abe0 2024-06-10T01:11:52.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.188+0000 7fa7aeffd700 1 -- 172.21.15.7:0/2475033932 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa7b4018020 con 0x7fa7b806abe0 2024-06-10T01:11:52.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.188+0000 7fa7aeffd700 1 --2- 172.21.15.7:0/2475033932 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa7a40a2c20 0x7fa7a40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:52.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.188+0000 7fa7aeffd700 1 -- 172.21.15.7:0/2475033932 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa7b40ceac0 con 0x7fa7b806abe0 2024-06-10T01:11:52.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.188+0000 7fa7bd695700 1 --2- 172.21.15.7:0/2475033932 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa7a40a2c20 0x7fa7a40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:52.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.189+0000 7fa7ca97db80 1 Processor -- start 2024-06-10T01:11:52.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.189+0000 7fa7ca97db80 1 -- start start 2024-06-10T01:11:52.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.189+0000 7fa7bd695700 1 --2- 172.21.15.7:0/2475033932 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa7a40a2c20 0x7fa7a40a50e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa7a800b5c0 tx=0x7fa7a8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:52.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.189+0000 7fa7ca97db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a893f2a50 0x558a893f2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:52.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.189+0000 7fa7ca97db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a893f3370 0x558a893ecdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:52.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.189+0000 7fa7ca97db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a893ed330 0x558a893ed750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:52.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.189+0000 7fa7ca97db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558a8933a810 con 0x558a893f2a50 2024-06-10T01:11:52.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.189+0000 7fa7ca97db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558a892e2720 con 0x558a893ed330 2024-06-10T01:11:52.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.189+0000 7fa7ca97db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558a893367d0 con 0x558a893f3370 2024-06-10T01:11:52.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.189+0000 7fa7bde96700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a893f2a50 0x558a893f2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:52.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.189+0000 7fa7bde96700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a893f2a50 0x558a893f2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37530/0 (socket says 172.21.15.7:37530) 2024-06-10T01:11:52.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.190+0000 7fa7bde96700 1 -- 172.21.15.7:0/4035124525 learned_addr learned my addr 172.21.15.7:0/4035124525 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:52.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.190+0000 7fa7be697700 1 --2- 172.21.15.7:0/4035124525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a893ed330 0x558a893ed750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:52.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.190+0000 7fa7bd695700 1 --2- 172.21.15.7:0/4035124525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a893f3370 0x558a893ecdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:52.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.190+0000 7fa7bde96700 1 -- 172.21.15.7:0/4035124525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a893f3370 msgr2=0x558a893ecdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:52.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.190+0000 7fa7bde96700 1 --2- 172.21.15.7:0/4035124525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a893f3370 0x558a893ecdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.190+0000 7fa7bde96700 1 -- 172.21.15.7:0/4035124525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a893ed330 msgr2=0x558a893ed750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:52.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.190+0000 7fa7bde96700 1 --2- 172.21.15.7:0/4035124525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a893ed330 0x558a893ed750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.190+0000 7fa7bde96700 1 -- 172.21.15.7:0/4035124525 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a892d8f50 con 0x558a893f2a50 2024-06-10T01:11:52.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.190+0000 7fa7be697700 1 --2- 172.21.15.7:0/4035124525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a893ed330 0x558a893ed750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:11:52.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.190+0000 7fa7bde96700 1 --2- 172.21.15.7:0/4035124525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a893f2a50 0x558a893f2e30 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa7b409aca0 tx=0x7fa7b409abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6e9b6d4f82aa1b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:52.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.190+0000 7fa7a3fff700 1 -- 172.21.15.7:0/4035124525 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b409afa0 con 0x558a893f2a50 2024-06-10T01:11:52.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.190+0000 7fa7a3fff700 1 -- 172.21.15.7:0/4035124525 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7b409f350 con 0x558a893f2a50 2024-06-10T01:11:52.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.190+0000 7fa7a3fff700 1 -- 172.21.15.7:0/4035124525 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b40b9e40 con 0x558a893f2a50 2024-06-10T01:11:52.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.191+0000 7fa7ca97db80 1 -- 172.21.15.7:0/4035124525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a893f2a50 msgr2=0x558a893f2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:52.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.191+0000 7fa7ca97db80 1 --2- 172.21.15.7:0/4035124525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a893f2a50 0x558a893f2e30 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa7b409aca0 tx=0x7fa7b409abd0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.191+0000 7fa7ca97db80 1 -- 172.21.15.7:0/4035124525 shutdown_connections 2024-06-10T01:11:52.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.191+0000 7fa7ca97db80 1 --2- 172.21.15.7:0/4035124525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a893ed330 0x558a893ed750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.191+0000 7fa7ca97db80 1 --2- 172.21.15.7:0/4035124525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a893f2a50 0x558a893f2e30 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.191+0000 7fa7ca97db80 1 --2- 172.21.15.7:0/4035124525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a893f3370 0x558a893ecdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.191+0000 7fa7ca97db80 1 -- 172.21.15.7:0/4035124525 >> 172.21.15.7:0/4035124525 conn(0x558a89313cc0 msgr2=0x558a893e3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:52.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.191+0000 7fa7ca97db80 1 -- 172.21.15.7:0/4035124525 shutdown_connections 2024-06-10T01:11:52.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.191+0000 7fa7ca97db80 1 -- 172.21.15.7:0/4035124525 wait complete. 2024-06-10T01:11:52.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.191+0000 7fa7ca97db80 1 -- start start 2024-06-10T01:11:52.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.191+0000 7fa7ca97db80 1 Processor -- start 2024-06-10T01:11:52.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.192+0000 7fa7ca97db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a893ed330 0x558a893f9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:52.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.192+0000 7fa7ca97db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a893f2a50 0x558a893f6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:52.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.192+0000 7fa7ca97db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a893f3370 0x558a893f69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:52.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.192+0000 7fa7ca97db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558a893f9de0 con 0x558a893f2a50 2024-06-10T01:11:52.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.192+0000 7fa7ca97db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558a892dabf0 con 0x558a893f3370 2024-06-10T01:11:52.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.192+0000 7fa7ca97db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558a893f9f60 con 0x558a893ed330 2024-06-10T01:11:52.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.192+0000 7fa7ca97db80 10 client.0 _ll_register_callbacks cb 0x558a892d4920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:11:52.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.192+0000 7fa7bd695700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a893f2a50 0x558a893f6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:52.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.192+0000 7fa7bd695700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a893f2a50 0x558a893f6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37540/0 (socket says 172.21.15.7:37540) 2024-06-10T01:11:52.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.192+0000 7fa7bd695700 1 -- 172.21.15.7:0/1069547506 learned_addr learned my addr 172.21.15.7:0/1069547506 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:52.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.192+0000 7fa7bde96700 1 --2- 172.21.15.7:0/1069547506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a893ed330 0x558a893f9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:52.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.192+0000 7fa7be697700 1 --2- 172.21.15.7:0/1069547506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a893f3370 0x558a893f69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:52.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.192+0000 7fa7bd695700 1 -- 172.21.15.7:0/1069547506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a893ed330 msgr2=0x558a893f9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:52.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.192+0000 7fa7bd695700 1 --2- 172.21.15.7:0/1069547506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a893ed330 0x558a893f9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.192+0000 7fa7bd695700 1 -- 172.21.15.7:0/1069547506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a893f3370 msgr2=0x558a893f69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:52.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.192+0000 7fa7bd695700 1 --2- 172.21.15.7:0/1069547506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a893f3370 0x558a893f69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.192+0000 7fa7bd695700 1 -- 172.21.15.7:0/1069547506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a892898d0 con 0x558a893f2a50 2024-06-10T01:11:52.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.192+0000 7fa7be697700 1 --2- 172.21.15.7:0/1069547506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a893f3370 0x558a893f69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:11:52.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.193+0000 7fa7bd695700 1 --2- 172.21.15.7:0/1069547506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a893f2a50 0x558a893f6470 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8015000 tx=0x7fa7a8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7f378ae235f1da server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:52.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.193+0000 7fa7a17fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:11:52.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.193+0000 7fa7a17fa700 1 -- 172.21.15.7:0/1069547506 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8035060 con 0x558a893f2a50 2024-06-10T01:11:52.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.193+0000 7fa7a17fa700 1 -- 172.21.15.7:0/1069547506 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa7a8030630 con 0x558a893f2a50 2024-06-10T01:11:52.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.193+0000 7fa7ca97db80 1 -- 172.21.15.7:0/1069547506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7b409f350 con 0x558a893f2a50 2024-06-10T01:11:52.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.193+0000 7fa7a17fa700 1 -- 172.21.15.7:0/1069547506 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa7a803c050 con 0x558a893f2a50 2024-06-10T01:11:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.193+0000 7fa7ca97db80 10 client.16953 fetch_fsmap learned FSMap version 32 2024-06-10T01:11:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.193+0000 7fa7ca97db80 1 -- 172.21.15.7:0/1069547506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558a89404790 con 0x558a893f2a50 2024-06-10T01:11:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.193+0000 7fa7a17fa700 1 -- 172.21.15.7:0/1069547506 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7b409f350 con 0x558a893f2a50 2024-06-10T01:11:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.193+0000 7fa7a17fa700 1 -- 172.21.15.7:0/1069547506 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fa7a802f070 con 0x558a893f2a50 2024-06-10T01:11:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.193+0000 7fa7ca97db80 10 client.16953 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:11:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.193+0000 7fa7ca97db80 10 client.16953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:11:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.193+0000 7fa7ca97db80 20 client.16953 populate_metadata read hostname 'smithi007' 2024-06-10T01:11:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.193+0000 7fa7ca97db80 1 --2- 172.21.15.7:0/1069547506 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558a89406dd0 0x558a894091c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.193+0000 7fa7ca97db80 4 client.16953 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:11:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.193+0000 7fa7ca97db80 1 -- 172.21.15.7:0/1069547506 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a89347d10 con 0x558a89406dd0 2024-06-10T01:11:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.194+0000 7fa7bde96700 1 --2- 172.21.15.7:0/1069547506 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558a89406dd0 0x558a894091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.194+0000 7fa7bde96700 1 --2- 172.21.15.7:0/1069547506 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558a89406dd0 0x558a894091c0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83bf439174df4f3e server_cookie=64569dbe53c38216 in_seq=0 out_seq=0 2024-06-10T01:11:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.194+0000 7fa7a17fa700 10 client.16953 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.195+0000 7fa7a17fa700 1 -- 172.21.15.7:0/1069547506 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558a89347d10 con 0x558a89406dd0 2024-06-10T01:11:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.195+0000 7fa7a17fa700 10 client.16953 handle_command_reply: tid=0 2024-06-10T01:11:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.195+0000 7fa7ca97db80 1 client.16953 shutdown 2024-06-10T01:11:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.195+0000 7fa7ca97db80 1 -- 172.21.15.7:0/1069547506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a893f2a50 msgr2=0x558a893f6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.195+0000 7fa7ca97db80 1 --2- 172.21.15.7:0/1069547506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a893f2a50 0x558a893f6470 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8015000 tx=0x7fa7a8009ca0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.195+0000 7fa7ca97db80 1 -- 172.21.15.7:0/1069547506 shutdown_connections 2024-06-10T01:11:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.195+0000 7fa7ca97db80 1 --2- 172.21.15.7:0/1069547506 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558a89406dd0 0x558a894091c0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.195+0000 7fa7ca97db80 1 --2- 172.21.15.7:0/1069547506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a893f3370 0x558a893f69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.195+0000 7fa7ca97db80 1 --2- 172.21.15.7:0/1069547506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a893ed330 0x558a893f9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.195+0000 7fa7a17fa700 0 client.16953 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.195+0000 7fa7ca97db80 1 --2- 172.21.15.7:0/1069547506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a893f2a50 0x558a893f6470 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.195+0000 7fa7ca97db80 1 -- 172.21.15.7:0/1069547506 >> 172.21.15.7:0/1069547506 conn(0x558a89313cc0 msgr2=0x558a893ebde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.195+0000 7fa7ca97db80 1 -- 172.21.15.7:0/1069547506 shutdown_connections 2024-06-10T01:11:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.195+0000 7fa7ca97db80 1 -- 172.21.15.7:0/1069547506 wait complete. 2024-06-10T01:11:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.195+0000 7fa7ca97db80 20 client.16953 trim_cache size 0 max 16384 2024-06-10T01:11:52.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.216+0000 7fa7ca97db80 1 Processor -- start 2024-06-10T01:11:52.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.216+0000 7fa7ca97db80 1 -- start start 2024-06-10T01:11:52.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.216+0000 7fa7ca97db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a893ed330 0x558a89403080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:52.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.216+0000 7fa7ca97db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a893f2a50 0x558a893ecdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:52.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.216+0000 7fa7ca97db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a893f3370 0x558a893f53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:52.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.216+0000 7fa7ca97db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b81aea30 con 0x558a893f2a50 2024-06-10T01:11:52.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.216+0000 7fa7ca97db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b81ac2c0 con 0x558a893ed330 2024-06-10T01:11:52.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.216+0000 7fa7ca97db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b81aa110 con 0x558a893f3370 2024-06-10T01:11:52.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.216+0000 7fa7bd695700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a893f2a50 0x558a893ecdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:52.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.216+0000 7fa7bd695700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a893f2a50 0x558a893ecdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37552/0 (socket says 172.21.15.7:37552) 2024-06-10T01:11:52.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.216+0000 7fa7bd695700 1 -- 172.21.15.7:0/2830321210 learned_addr learned my addr 172.21.15.7:0/2830321210 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:52.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.216+0000 7fa7be697700 1 --2- 172.21.15.7:0/2830321210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a893f3370 0x558a893f53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:52.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.216+0000 7fa7bde96700 1 --2- 172.21.15.7:0/2830321210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a893ed330 0x558a89403080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:52.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.217+0000 7fa7bd695700 1 -- 172.21.15.7:0/2830321210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a893f3370 msgr2=0x558a893f53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:52.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.217+0000 7fa7bd695700 1 --2- 172.21.15.7:0/2830321210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a893f3370 0x558a893f53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.217+0000 7fa7bd695700 1 -- 172.21.15.7:0/2830321210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a893ed330 msgr2=0x558a89403080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:52.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.217+0000 7fa7bd695700 1 --2- 172.21.15.7:0/2830321210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a893ed330 0x558a89403080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.217+0000 7fa7bd695700 1 -- 172.21.15.7:0/2830321210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a89409820 con 0x558a893f2a50 2024-06-10T01:11:52.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.217+0000 7fa7bd695700 1 --2- 172.21.15.7:0/2830321210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a893f2a50 0x558a893ecdf0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8024d60 tx=0x7fa7a800fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=176fd8d10247f91b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:52.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.217+0000 7fa7a37fe700 1 -- 172.21.15.7:0/2830321210 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8028070 con 0x558a893f2a50 2024-06-10T01:11:52.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.217+0000 7fa7a37fe700 1 -- 172.21.15.7:0/2830321210 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a8017580 con 0x558a893f2a50 2024-06-10T01:11:52.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.217+0000 7fa7a37fe700 1 -- 172.21.15.7:0/2830321210 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8035030 con 0x558a893f2a50 2024-06-10T01:11:52.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.217+0000 7fa7ca97db80 1 -- 172.21.15.7:0/2830321210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a893f2a50 msgr2=0x558a893ecdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:52.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.217+0000 7fa7ca97db80 1 --2- 172.21.15.7:0/2830321210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a893f2a50 0x558a893ecdf0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8024d60 tx=0x7fa7a800fb20 comp rx=0 tx=0).stop 2024-06-10T01:11:52.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.217+0000 7fa7ca97db80 1 -- 172.21.15.7:0/2830321210 shutdown_connections 2024-06-10T01:11:52.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.217+0000 7fa7ca97db80 1 --2- 172.21.15.7:0/2830321210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a893f3370 0x558a893f53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.217+0000 7fa7ca97db80 1 --2- 172.21.15.7:0/2830321210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a893ed330 0x558a89403080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.217+0000 7fa7ca97db80 1 --2- 172.21.15.7:0/2830321210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a893f2a50 0x558a893ecdf0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.218+0000 7fa7ca97db80 1 -- 172.21.15.7:0/2830321210 >> 172.21.15.7:0/2830321210 conn(0x558a89313cc0 msgr2=0x558a893dce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:52.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.218+0000 7fa7ca97db80 1 -- 172.21.15.7:0/2830321210 shutdown_connections 2024-06-10T01:11:52.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.218+0000 7fa7ca97db80 1 -- 172.21.15.7:0/2830321210 wait complete. 2024-06-10T01:11:52.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.218+0000 7fa7ca97db80 1 -- start start 2024-06-10T01:11:52.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.218+0000 7fa7ca97db80 1 Processor -- start 2024-06-10T01:11:52.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.218+0000 7fa7ca97db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a893ed330 0x558a8940e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:52.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.218+0000 7fa7ca97db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a893f2a50 0x558a8940ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:52.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.218+0000 7fa7ca97db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a893f3370 0x558a8940f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:52.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.218+0000 7fa7ca97db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b81aa780 con 0x558a893f2a50 2024-06-10T01:11:52.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.218+0000 7fa7ca97db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b81abbf0 con 0x558a893f3370 2024-06-10T01:11:52.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.218+0000 7fa7ca97db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b81aae50 con 0x558a893ed330 2024-06-10T01:11:52.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.219+0000 7fa7bd695700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a893f2a50 0x558a8940ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:52.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.219+0000 7fa7ca97db80 10 client.0 _ll_register_callbacks cb 0x558a892d4920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:11:52.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.219+0000 7fa7bd695700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a893f2a50 0x558a8940ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37560/0 (socket says 172.21.15.7:37560) 2024-06-10T01:11:52.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.219+0000 7fa7bd695700 1 -- 172.21.15.7:0/995157824 learned_addr learned my addr 172.21.15.7:0/995157824 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:52.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.219+0000 7fa7bde96700 1 --2- 172.21.15.7:0/995157824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a893ed330 0x558a8940e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:52.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.219+0000 7fa7be697700 1 --2- 172.21.15.7:0/995157824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a893f3370 0x558a8940f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:52.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.219+0000 7fa7bd695700 1 -- 172.21.15.7:0/995157824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a893ed330 msgr2=0x558a8940e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:52.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.219+0000 7fa7bd695700 1 --2- 172.21.15.7:0/995157824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a893ed330 0x558a8940e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.219+0000 7fa7bd695700 1 -- 172.21.15.7:0/995157824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a893f3370 msgr2=0x558a8940f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:52.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.219+0000 7fa7bd695700 1 --2- 172.21.15.7:0/995157824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a893f3370 0x558a8940f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.219+0000 7fa7bd695700 1 -- 172.21.15.7:0/995157824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a89470b00 con 0x558a893f2a50 2024-06-10T01:11:52.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.219+0000 7fa7be697700 1 --2- 172.21.15.7:0/995157824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a893f3370 0x558a8940f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:11:52.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.219+0000 7fa7bd695700 1 --2- 172.21.15.7:0/995157824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a893f2a50 0x558a8940ec80 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8005a60 tx=0x7fa7a8017e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520e8364a56fc351 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:52.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.219+0000 7fa7a17fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:11:52.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.219+0000 7fa7a17fa700 1 -- 172.21.15.7:0/995157824 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8035070 con 0x558a893f2a50 2024-06-10T01:11:52.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.219+0000 7fa7a17fa700 1 -- 172.21.15.7:0/995157824 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa7a80245f0 con 0x558a893f2a50 2024-06-10T01:11:52.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.219+0000 7fa7ca97db80 1 -- 172.21.15.7:0/995157824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7a8017580 con 0x558a893f2a50 2024-06-10T01:11:52.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.219+0000 7fa7a17fa700 1 -- 172.21.15.7:0/995157824 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa7a8028040 con 0x558a893f2a50 2024-06-10T01:11:52.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.219+0000 7fa7ca97db80 10 client.16959 fetch_fsmap learned FSMap version 32 2024-06-10T01:11:52.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.219+0000 7fa7ca97db80 1 -- 172.21.15.7:0/995157824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558a89471e10 con 0x558a893f2a50 2024-06-10T01:11:52.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.220+0000 7fa7a17fa700 1 -- 172.21.15.7:0/995157824 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a8017580 con 0x558a893f2a50 2024-06-10T01:11:52.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.220+0000 7fa7a17fa700 1 -- 172.21.15.7:0/995157824 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fa7a802f040 con 0x558a893f2a50 2024-06-10T01:11:52.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.220+0000 7fa7ca97db80 10 client.16959 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:11:52.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.220+0000 7fa7ca97db80 10 client.16959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:11:52.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.220+0000 7fa7ca97db80 20 client.16959 populate_metadata read hostname 'smithi007' 2024-06-10T01:11:52.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.220+0000 7fa7ca97db80 1 --2- 172.21.15.7:0/995157824 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558a89472430 0x558a89474850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:52.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.220+0000 7fa7ca97db80 4 client.16959 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:11:52.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.220+0000 7fa7ca97db80 1 -- 172.21.15.7:0/995157824 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558a89342290 con 0x558a89472430 2024-06-10T01:11:52.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.220+0000 7fa7bde96700 1 --2- 172.21.15.7:0/995157824 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558a89472430 0x558a89474850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:52.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.221+0000 7fa7a17fa700 10 client.16959 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:52.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.221+0000 7fa7bde96700 1 --2- 172.21.15.7:0/995157824 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558a89472430 0x558a89474850 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cde8ab4194f897a server_cookie=3fd57fcfb41f26e7 in_seq=0 out_seq=0 2024-06-10T01:11:52.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.221+0000 7fa7a17fa700 1 -- 172.21.15.7:0/995157824 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558a89342290 con 0x558a89472430 2024-06-10T01:11:52.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.221+0000 7fa7a17fa700 10 client.16959 handle_command_reply: tid=0 2024-06-10T01:11:52.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.221+0000 7fa7ca97db80 1 client.16959 shutdown 2024-06-10T01:11:52.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.221+0000 7fa7ca97db80 1 -- 172.21.15.7:0/995157824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a893f2a50 msgr2=0x558a8940ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:52.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.221+0000 7fa7ca97db80 1 --2- 172.21.15.7:0/995157824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a893f2a50 0x558a8940ec80 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8005a60 tx=0x7fa7a8017e70 comp rx=0 tx=0).stop 2024-06-10T01:11:52.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.221+0000 7fa7ca97db80 1 -- 172.21.15.7:0/995157824 shutdown_connections 2024-06-10T01:11:52.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.221+0000 7fa7ca97db80 1 --2- 172.21.15.7:0/995157824 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558a89472430 0x558a89474850 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.221+0000 7fa7ca97db80 1 --2- 172.21.15.7:0/995157824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a893f3370 0x558a8940f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.221+0000 7fa7ca97db80 1 --2- 172.21.15.7:0/995157824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a893ed330 0x558a8940e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.221+0000 7fa7a17fa700 0 client.16959 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:11:52.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.221+0000 7fa7ca97db80 1 --2- 172.21.15.7:0/995157824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a893f2a50 0x558a8940ec80 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.221+0000 7fa7ca97db80 1 -- 172.21.15.7:0/995157824 >> 172.21.15.7:0/995157824 conn(0x558a89313cc0 msgr2=0x558a893f3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:52.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.221+0000 7fa7ca97db80 1 -- 172.21.15.7:0/995157824 shutdown_connections 2024-06-10T01:11:52.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.221+0000 7fa7ca97db80 1 -- 172.21.15.7:0/995157824 wait complete. 2024-06-10T01:11:52.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.221+0000 7fa7ca97db80 20 client.16959 trim_cache size 0 max 16384 2024-06-10T01:11:52.225 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:11:52.225 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-10T01:11:52.225 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-10T01:11:52.225 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:11:52.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.222+0000 7fa7c00fa700 1 -- 172.21.15.7:0/2475033932 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa7a40a2c20 msgr2=0x7fa7a40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:52.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.222+0000 7fa7c00fa700 1 --2- 172.21.15.7:0/2475033932 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa7a40a2c20 0x7fa7a40a50e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa7a800b5c0 tx=0x7fa7a8005960 comp rx=0 tx=0).stop 2024-06-10T01:11:52.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.222+0000 7fa7c00fa700 1 -- 172.21.15.7:0/2475033932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b806abe0 msgr2=0x7fa7b8198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:52.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.222+0000 7fa7c00fa700 1 --2- 172.21.15.7:0/2475033932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b806abe0 0x7fa7b8198d70 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400b530 tx=0x7fa7b400b840 comp rx=0 tx=0).stop 2024-06-10T01:11:52.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.222+0000 7fa7c00fa700 1 -- 172.21.15.7:0/2475033932 shutdown_connections 2024-06-10T01:11:52.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.222+0000 7fa7c00fa700 1 --2- 172.21.15.7:0/2475033932 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa7a40a2c20 0x7fa7a40a50e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.222+0000 7fa7c00fa700 1 --2- 172.21.15.7:0/2475033932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7b8108500 0x7fa7b819d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.222+0000 7fa7c00fa700 1 --2- 172.21.15.7:0/2475033932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b806abe0 0x7fa7b8198d70 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.222+0000 7fa7c00fa700 1 --2- 172.21.15.7:0/2475033932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7b806b5a0 0x7fa7b81992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:52.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.222+0000 7fa7c00fa700 1 -- 172.21.15.7:0/2475033932 >> 172.21.15.7:0/2475033932 conn(0x7fa7b8078360 msgr2=0x7fa7b8100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:52.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.222+0000 7fa7c00fa700 1 -- 172.21.15.7:0/2475033932 shutdown_connections 2024-06-10T01:11:52.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:52.222+0000 7fa7c00fa700 1 -- 172.21.15.7:0/2475033932 wait complete. 2024-06-10T01:11:52.732 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 6.46 MB/sec execute 153 sec latency 18886.396 ms 2024-06-10T01:11:52.947 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:11:52.947 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:11:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:11:53.638 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:53 smithi007 ceph-mon[32398]: pgmap v608: 129 pgs: 129 active+clean; 65 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-06-10T01:11:53.732 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 6.42 MB/sec execute 154 sec latency 19886.501 ms 2024-06-10T01:11:53.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:53 smithi078 ceph-mon[36683]: pgmap v608: 129 pgs: 129 active+clean; 65 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-06-10T01:11:53.953 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:53 smithi175 ceph-mon[36636]: pgmap v608: 129 pgs: 129 active+clean; 65 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-06-10T01:11:54.733 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 6.38 MB/sec execute 155 sec latency 20887.133 ms 2024-06-10T01:11:54.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.976+0000 7fb8002c8700 1 -- 172.21.15.7:0/3335993903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7f806b3c0 msgr2=0x7fb7f810f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:54.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.976+0000 7fb8002c8700 1 --2- 172.21.15.7:0/3335993903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7f806b3c0 0x7fb7f810f310 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0009990 tx=0x7fb7f0009ca0 comp rx=0 tx=0).stop 2024-06-10T01:11:54.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.976+0000 7fb8002c8700 1 -- 172.21.15.7:0/3335993903 shutdown_connections 2024-06-10T01:11:54.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.976+0000 7fb8002c8700 1 --2- 172.21.15.7:0/3335993903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7f806b3c0 0x7fb7f810f310 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:54.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.976+0000 7fb8002c8700 1 --2- 172.21.15.7:0/3335993903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7f81084f0 0x7fb7f81088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:54.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.976+0000 7fb8002c8700 1 --2- 172.21.15.7:0/3335993903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7f806aa00 0x7fb7f806ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:54.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.976+0000 7fb8002c8700 1 -- 172.21.15.7:0/3335993903 >> 172.21.15.7:0/3335993903 conn(0x7fb7f8078300 msgr2=0x7fb7f8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:54.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.976+0000 7fb8002c8700 1 -- 172.21.15.7:0/3335993903 shutdown_connections 2024-06-10T01:11:54.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.976+0000 7fb8002c8700 1 -- 172.21.15.7:0/3335993903 wait complete. 2024-06-10T01:11:54.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.977+0000 7fb8002c8700 1 Processor -- start 2024-06-10T01:11:54.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.977+0000 7fb8002c8700 1 -- start start 2024-06-10T01:11:54.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.977+0000 7fb8002c8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7f806aa00 0x7fb7f819e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:54.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.977+0000 7fb8002c8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7f806b3c0 0x7fb7f819ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:54.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.977+0000 7fb8002c8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7f81084f0 0x7fb7f8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:54.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.977+0000 7fb8002c8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f8112750 con 0x7fb7f81084f0 2024-06-10T01:11:54.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.977+0000 7fb8002c8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81125d0 con 0x7fb7f806b3c0 2024-06-10T01:11:54.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.977+0000 7fb8002c8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81128d0 con 0x7fb7f806aa00 2024-06-10T01:11:54.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.977+0000 7fb7fe865700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7f81084f0 0x7fb7f8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:54.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.977+0000 7fb7fd863700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7f806b3c0 0x7fb7f819ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:54.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.977+0000 7fb7fe865700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7f81084f0 0x7fb7f8198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37568/0 (socket says 172.21.15.7:37568) 2024-06-10T01:11:54.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.977+0000 7fb7fe865700 1 -- 172.21.15.7:0/3447404642 learned_addr learned my addr 172.21.15.7:0/3447404642 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:54.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.978+0000 7fb7fe064700 1 --2- 172.21.15.7:0/3447404642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7f806aa00 0x7fb7f819e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:54.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.978+0000 7fb7fe865700 1 -- 172.21.15.7:0/3447404642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7f806aa00 msgr2=0x7fb7f819e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:54.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.978+0000 7fb7fe865700 1 --2- 172.21.15.7:0/3447404642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7f806aa00 0x7fb7f819e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:54.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.978+0000 7fb7fe865700 1 -- 172.21.15.7:0/3447404642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7f806b3c0 msgr2=0x7fb7f819ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:54.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.978+0000 7fb7fe865700 1 --2- 172.21.15.7:0/3447404642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7f806b3c0 0x7fb7f819ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:54.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.978+0000 7fb7fe865700 1 -- 172.21.15.7:0/3447404642 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f8199090 con 0x7fb7f81084f0 2024-06-10T01:11:54.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.978+0000 7fb7fe064700 1 --2- 172.21.15.7:0/3447404642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7f806aa00 0x7fb7f819e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:11:54.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.978+0000 7fb7fd863700 1 --2- 172.21.15.7:0/3447404642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7f806b3c0 0x7fb7f819ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:11:54.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.978+0000 7fb7fe865700 1 --2- 172.21.15.7:0/3447404642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7f81084f0 0x7fb7f8198930 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0005910 tx=0x7fb7f00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:54.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.978+0000 7fb7ef7fe700 1 -- 172.21.15.7:0/3447404642 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0027070 con 0x7fb7f81084f0 2024-06-10T01:11:54.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.978+0000 7fb8002c8700 1 -- 172.21.15.7:0/3447404642 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f8199320 con 0x7fb7f81084f0 2024-06-10T01:11:54.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.978+0000 7fb7ef7fe700 1 -- 172.21.15.7:0/3447404642 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7f0004050 con 0x7fb7f81084f0 2024-06-10T01:11:54.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.978+0000 7fb7ef7fe700 1 -- 172.21.15.7:0/3447404642 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0022e40 con 0x7fb7f81084f0 2024-06-10T01:11:54.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.978+0000 7fb8002c8700 1 -- 172.21.15.7:0/3447404642 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f81a5640 con 0x7fb7f81084f0 2024-06-10T01:11:54.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.979+0000 7fb8002c8700 1 -- 172.21.15.7:0/3447404642 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7f8069180 con 0x7fb7f81084f0 2024-06-10T01:11:54.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.981+0000 7fb7ef7fe700 1 -- 172.21.15.7:0/3447404642 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb7f002a070 con 0x7fb7f81084f0 2024-06-10T01:11:54.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.982+0000 7fb7ef7fe700 1 --2- 172.21.15.7:0/3447404642 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb7e40a2c20 0x7fb7e40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:54.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.982+0000 7fb7fe064700 1 --2- 172.21.15.7:0/3447404642 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb7e40a2c20 0x7fb7e40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:54.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.982+0000 7fb7ef7fe700 1 -- 172.21.15.7:0/3447404642 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb7f00d0750 con 0x7fb7f81084f0 2024-06-10T01:11:54.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.982+0000 7fb7fe064700 1 --2- 172.21.15.7:0/3447404642 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb7e40a2c20 0x7fb7e40a50e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8199a70 tx=0x7fb7f4009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:54.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:54.985+0000 7fb7ef7fe700 1 -- 172.21.15.7:0/3447404642 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb7f0031070 con 0x7fb7f81084f0 2024-06-10T01:11:55.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:55.240+0000 7fb8002c8700 1 -- 172.21.15.7:0/3447404642 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7f8199e60 con 0x7fb7f81084f0 2024-06-10T01:11:55.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:55.241+0000 7fb7ef7fe700 1 -- 172.21.15.7:0/3447404642 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11855 (secure 0 0 0) 0x7fb7f0013070 con 0x7fb7f81084f0 2024-06-10T01:11:55.237 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:11:55.237 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:10:42.213996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:11:55.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:55.243+0000 7fb8002c8700 1 -- 172.21.15.7:0/3447404642 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb7e40a2c20 msgr2=0x7fb7e40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:55.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:55.243+0000 7fb8002c8700 1 --2- 172.21.15.7:0/3447404642 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb7e40a2c20 0x7fb7e40a50e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8199a70 tx=0x7fb7f4009040 comp rx=0 tx=0).stop 2024-06-10T01:11:55.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:55.243+0000 7fb8002c8700 1 -- 172.21.15.7:0/3447404642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7f81084f0 msgr2=0x7fb7f8198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:55.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:55.243+0000 7fb8002c8700 1 --2- 172.21.15.7:0/3447404642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7f81084f0 0x7fb7f8198930 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0005910 tx=0x7fb7f00047b0 comp rx=0 tx=0).stop 2024-06-10T01:11:55.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:55.243+0000 7fb8002c8700 1 -- 172.21.15.7:0/3447404642 shutdown_connections 2024-06-10T01:11:55.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:55.243+0000 7fb8002c8700 1 --2- 172.21.15.7:0/3447404642 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb7e40a2c20 0x7fb7e40a50e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:55.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:55.243+0000 7fb8002c8700 1 --2- 172.21.15.7:0/3447404642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7f81084f0 0x7fb7f8198930 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:55.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:55.243+0000 7fb8002c8700 1 --2- 172.21.15.7:0/3447404642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7f806aa00 0x7fb7f819e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:55.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:55.243+0000 7fb8002c8700 1 --2- 172.21.15.7:0/3447404642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7f806b3c0 0x7fb7f819ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:55.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:55.243+0000 7fb8002c8700 1 -- 172.21.15.7:0/3447404642 >> 172.21.15.7:0/3447404642 conn(0x7fb7f8078300 msgr2=0x7fb7f80ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:55.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:55.244+0000 7fb8002c8700 1 -- 172.21.15.7:0/3447404642 shutdown_connections 2024-06-10T01:11:55.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:55.244+0000 7fb8002c8700 1 -- 172.21.15.7:0/3447404642 wait complete. 2024-06-10T01:11:55.241 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 32 2024-06-10T01:11:55.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:55 smithi007 ceph-mon[32398]: pgmap v609: 129 pgs: 129 active+clean; 65 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-06-10T01:11:55.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:55 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3447404642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:11:55.733 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 6.34 MB/sec execute 156 sec latency 21887.223 ms 2024-06-10T01:11:55.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:55 smithi078 ceph-mon[36683]: pgmap v609: 129 pgs: 129 active+clean; 65 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-06-10T01:11:55.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:55 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3447404642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:11:55.953 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:55 smithi175 ceph-mon[36636]: pgmap v609: 129 pgs: 129 active+clean; 65 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-06-10T01:11:55.953 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:55 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3447404642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:11:55.963 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:11:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:11:56.733 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 6.30 MB/sec execute 157 sec latency 22887.328 ms 2024-06-10T01:11:57.693 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:57 smithi007 ceph-mon[32398]: pgmap v610: 129 pgs: 129 active+clean; 66 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 59 KiB/s wr, 12 op/s 2024-06-10T01:11:57.733 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 6.26 MB/sec execute 158 sec latency 23887.367 ms 2024-06-10T01:11:57.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:57 smithi078 ceph-mon[36683]: pgmap v610: 129 pgs: 129 active+clean; 66 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 59 KiB/s wr, 12 op/s 2024-06-10T01:11:57.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:57 smithi175 ceph-mon[36636]: pgmap v610: 129 pgs: 129 active+clean; 66 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 59 KiB/s wr, 12 op/s 2024-06-10T01:11:57.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.964+0000 7f220b65f700 1 -- 172.21.15.7:0/3461775314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2204107450 msgr2=0x7f2204107830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:57.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.964+0000 7f220b65f700 1 --2- 172.21.15.7:0/3461775314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2204107450 0x7f2204107830 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f2200009990 tx=0x7f2200009ca0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.965+0000 7f220b65f700 1 -- 172.21.15.7:0/3461775314 shutdown_connections 2024-06-10T01:11:57.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.965+0000 7f220b65f700 1 --2- 172.21.15.7:0/3461775314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2204102ee0 0x7f220410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.965+0000 7f220b65f700 1 --2- 172.21.15.7:0/3461775314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2204107450 0x7f2204107830 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.965+0000 7f220b65f700 1 --2- 172.21.15.7:0/3461775314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2204102540 0x7f22041029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.965+0000 7f220b65f700 1 -- 172.21.15.7:0/3461775314 >> 172.21.15.7:0/3461775314 conn(0x7f22040795f0 msgr2=0x7f2204079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:57.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.965+0000 7f220b65f700 1 -- 172.21.15.7:0/3461775314 shutdown_connections 2024-06-10T01:11:57.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.965+0000 7f220b65f700 1 -- 172.21.15.7:0/3461775314 wait complete. 2024-06-10T01:11:57.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.965+0000 7f220b65f700 1 Processor -- start 2024-06-10T01:11:57.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.966+0000 7f220b65f700 1 -- start start 2024-06-10T01:11:57.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.966+0000 7f220b65f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2204102540 0x7f2204074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:57.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.966+0000 7f220b65f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2204102ee0 0x7f2204074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:57.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.966+0000 7f220b65f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2204107450 0x7f2204075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:57.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.966+0000 7f220b65f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f22041127c0 con 0x7f2204102540 2024-06-10T01:11:57.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.966+0000 7f220b65f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2204112640 con 0x7f2204102ee0 2024-06-10T01:11:57.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.966+0000 7f220b65f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2204112940 con 0x7f2204107450 2024-06-10T01:11:57.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.966+0000 7f22093fb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2204102540 0x7f2204074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:57.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.966+0000 7f22093fb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2204102540 0x7f2204074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37588/0 (socket says 172.21.15.7:37588) 2024-06-10T01:11:57.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.966+0000 7f22093fb700 1 -- 172.21.15.7:0/2127236479 learned_addr learned my addr 172.21.15.7:0/2127236479 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:57.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.966+0000 7f2209bfc700 1 --2- 172.21.15.7:0/2127236479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2204107450 0x7f2204075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:57.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.966+0000 7f2208bfa700 1 --2- 172.21.15.7:0/2127236479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2204102ee0 0x7f2204074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:57.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.966+0000 7f22093fb700 1 -- 172.21.15.7:0/2127236479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2204107450 msgr2=0x7f2204075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:57.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.966+0000 7f22093fb700 1 --2- 172.21.15.7:0/2127236479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2204107450 0x7f2204075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.966+0000 7f22093fb700 1 -- 172.21.15.7:0/2127236479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2204102ee0 msgr2=0x7f2204074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:57.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.966+0000 7f22093fb700 1 --2- 172.21.15.7:0/2127236479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2204102ee0 0x7f2204074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.966+0000 7f22093fb700 1 -- 172.21.15.7:0/2127236479 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22041a7100 con 0x7f2204102540 2024-06-10T01:11:57.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.967+0000 7f22093fb700 1 --2- 172.21.15.7:0/2127236479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2204102540 0x7f2204074790 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f2200000c00 tx=0x7f220000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:57.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.967+0000 7f21fa7fc700 1 -- 172.21.15.7:0/2127236479 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2200027070 con 0x7f2204102540 2024-06-10T01:11:57.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.967+0000 7f220b65f700 1 -- 172.21.15.7:0/2127236479 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22041a7390 con 0x7f2204102540 2024-06-10T01:11:57.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.967+0000 7f21fa7fc700 1 -- 172.21.15.7:0/2127236479 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2200003d70 con 0x7f2204102540 2024-06-10T01:11:57.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.967+0000 7f21fa7fc700 1 -- 172.21.15.7:0/2127236479 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220000f460 con 0x7f2204102540 2024-06-10T01:11:57.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.967+0000 7f220b65f700 1 -- 172.21.15.7:0/2127236479 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22041a7840 con 0x7f2204102540 2024-06-10T01:11:57.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.969+0000 7f21fa7fc700 1 -- 172.21.15.7:0/2127236479 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f220002a070 con 0x7f2204102540 2024-06-10T01:11:57.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.970+0000 7f21fa7fc700 1 --2- 172.21.15.7:0/2127236479 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21f00a2c20 0x7f21f00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:57.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.970+0000 7f2208bfa700 1 --2- 172.21.15.7:0/2127236479 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21f00a2c20 0x7f21f00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:57.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.970+0000 7f21fa7fc700 1 -- 172.21.15.7:0/2127236479 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f22000d0080 con 0x7f2204102540 2024-06-10T01:11:57.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.970+0000 7f2208bfa700 1 --2- 172.21.15.7:0/2127236479 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21f00a2c20 0x7f21f00a50e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2204108bb0 tx=0x7f21f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.971+0000 7f2215ee2b80 1 Processor -- start 2024-06-10T01:11:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.971+0000 7f2215ee2b80 1 -- start start 2024-06-10T01:11:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.971+0000 7f2215ee2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bed48a4f60 0x55bed48a5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.971+0000 7f2215ee2b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bed48a5880 0x55bed489f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.971+0000 7f2215ee2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bed489f7a0 0x55bed489fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.972+0000 7f2215ee2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bed4791f60 con 0x55bed489f7a0 2024-06-10T01:11:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.972+0000 7f2215ee2b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bed47f0850 con 0x55bed48a5880 2024-06-10T01:11:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.972+0000 7f2209bfc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bed489f7a0 0x55bed489fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.972+0000 7f2215ee2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bed4758460 con 0x55bed48a4f60 2024-06-10T01:11:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.972+0000 7f2209bfc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bed489f7a0 0x55bed489fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37590/0 (socket says 172.21.15.7:37590) 2024-06-10T01:11:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.972+0000 7f2209bfc700 1 -- 172.21.15.7:0/1859051514 learned_addr learned my addr 172.21.15.7:0/1859051514 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.972+0000 7f2208bfa700 1 --2- 172.21.15.7:0/1859051514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bed48a5880 0x55bed489f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:57.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.972+0000 7f22093fb700 1 --2- 172.21.15.7:0/1859051514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bed48a4f60 0x55bed48a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:57.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.972+0000 7f2209bfc700 1 -- 172.21.15.7:0/1859051514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bed48a4f60 msgr2=0x55bed48a5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:57.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.972+0000 7f2209bfc700 1 --2- 172.21.15.7:0/1859051514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bed48a4f60 0x55bed48a5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.972+0000 7f2209bfc700 1 -- 172.21.15.7:0/1859051514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bed48a5880 msgr2=0x55bed489f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:57.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.972+0000 7f2209bfc700 1 --2- 172.21.15.7:0/1859051514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bed48a5880 0x55bed489f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.972+0000 7f2209bfc700 1 -- 172.21.15.7:0/1859051514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bed473f8d0 con 0x55bed489f7a0 2024-06-10T01:11:57.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.972+0000 7f2208bfa700 1 --2- 172.21.15.7:0/1859051514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bed48a5880 0x55bed489f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:11:57.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.972+0000 7f22093fb700 1 --2- 172.21.15.7:0/1859051514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bed48a4f60 0x55bed48a5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:11:57.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.972+0000 7f2209bfc700 1 --2- 172.21.15.7:0/1859051514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bed489f7a0 0x55bed489fbc0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00f3e0 tx=0x7f21fc00f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf3dbd1bdfb1390 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:57.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.972+0000 7f21ef7fe700 1 -- 172.21.15.7:0/1859051514 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc018030 con 0x55bed489f7a0 2024-06-10T01:11:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.972+0000 7f21ef7fe700 1 -- 172.21.15.7:0/1859051514 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21fc004bb0 con 0x55bed489f7a0 2024-06-10T01:11:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.972+0000 7f21ef7fe700 1 -- 172.21.15.7:0/1859051514 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc005020 con 0x55bed489f7a0 2024-06-10T01:11:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.973+0000 7f2215ee2b80 1 -- 172.21.15.7:0/1859051514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bed489f7a0 msgr2=0x55bed489fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.973+0000 7f2215ee2b80 1 --2- 172.21.15.7:0/1859051514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bed489f7a0 0x55bed489fbc0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00f3e0 tx=0x7f21fc00f6f0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.973+0000 7f2215ee2b80 1 -- 172.21.15.7:0/1859051514 shutdown_connections 2024-06-10T01:11:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.973+0000 7f2215ee2b80 1 --2- 172.21.15.7:0/1859051514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bed489f7a0 0x55bed489fbc0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.973+0000 7f2215ee2b80 1 --2- 172.21.15.7:0/1859051514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bed48a4f60 0x55bed48a5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.973+0000 7f2215ee2b80 1 --2- 172.21.15.7:0/1859051514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bed48a5880 0x55bed489f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.973+0000 7f2215ee2b80 1 -- 172.21.15.7:0/1859051514 >> 172.21.15.7:0/1859051514 conn(0x55bed47cc080 msgr2=0x55bed489a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.973+0000 7f2215ee2b80 1 -- 172.21.15.7:0/1859051514 shutdown_connections 2024-06-10T01:11:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.973+0000 7f2215ee2b80 1 -- 172.21.15.7:0/1859051514 wait complete. 2024-06-10T01:11:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.974+0000 7f2215ee2b80 1 -- start start 2024-06-10T01:11:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.974+0000 7f2215ee2b80 1 Processor -- start 2024-06-10T01:11:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.974+0000 7f2215ee2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bed489f7a0 0x55bed48ab590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.974+0000 7f2215ee2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bed48a4f60 0x55bed48a8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.974+0000 7f2215ee2b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bed48a5880 0x55bed48a8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.974+0000 7f2215ee2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bed4790fc0 con 0x55bed489f7a0 2024-06-10T01:11:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.974+0000 7f2215ee2b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bed47ec810 con 0x55bed48a5880 2024-06-10T01:11:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.974+0000 7f2215ee2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bed48ac260 con 0x55bed48a4f60 2024-06-10T01:11:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.974+0000 7f2215ee2b80 10 client.0 _ll_register_callbacks cb 0x55bed47865b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:11:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.974+0000 7f22093fb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bed489f7a0 0x55bed48ab590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.974+0000 7f22093fb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bed489f7a0 0x55bed48ab590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37596/0 (socket says 172.21.15.7:37596) 2024-06-10T01:11:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.974+0000 7f22093fb700 1 -- 172.21.15.7:0/1507369627 learned_addr learned my addr 172.21.15.7:0/1507369627 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.974+0000 7f2209bfc700 1 --2- 172.21.15.7:0/1507369627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bed48a5880 0x55bed48a8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.975+0000 7f2208bfa700 1 --2- 172.21.15.7:0/1507369627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bed48a4f60 0x55bed48a8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.975+0000 7f22093fb700 1 -- 172.21.15.7:0/1507369627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bed48a4f60 msgr2=0x55bed48a8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.975+0000 7f22093fb700 1 --2- 172.21.15.7:0/1507369627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bed48a4f60 0x55bed48a8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.975+0000 7f22093fb700 1 -- 172.21.15.7:0/1507369627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bed48a5880 msgr2=0x55bed48a8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.975+0000 7f22093fb700 1 --2- 172.21.15.7:0/1507369627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bed48a5880 0x55bed48a8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.975+0000 7f22093fb700 1 -- 172.21.15.7:0/1507369627 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bed47b6750 con 0x55bed489f7a0 2024-06-10T01:11:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.975+0000 7f2209bfc700 1 --2- 172.21.15.7:0/1507369627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bed48a5880 0x55bed48a8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:11:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.975+0000 7f2208bfa700 1 --2- 172.21.15.7:0/1507369627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bed48a4f60 0x55bed48a8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:11:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.975+0000 7f22093fb700 1 --2- 172.21.15.7:0/1507369627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bed489f7a0 0x55bed48ab590 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f220009c2f0 tx=0x7f22000989c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb2b0e0da2b08ad server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.975+0000 7f21ecff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:11:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.975+0000 7f21ecff9700 1 -- 172.21.15.7:0/1507369627 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22000095d0 con 0x55bed489f7a0 2024-06-10T01:11:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.975+0000 7f21ecff9700 1 -- 172.21.15.7:0/1507369627 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f22000a4d40 con 0x55bed489f7a0 2024-06-10T01:11:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.975+0000 7f2215ee2b80 1 -- 172.21.15.7:0/1507369627 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21fc004bb0 con 0x55bed489f7a0 2024-06-10T01:11:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.975+0000 7f21ecff9700 1 -- 172.21.15.7:0/1507369627 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f220009ccf0 con 0x55bed489f7a0 2024-06-10T01:11:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.975+0000 7f2215ee2b80 10 client.16977 fetch_fsmap learned FSMap version 32 2024-06-10T01:11:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.975+0000 7f2215ee2b80 1 -- 172.21.15.7:0/1507369627 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bed48b6980 con 0x55bed489f7a0 2024-06-10T01:11:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.975+0000 7f21ecff9700 1 -- 172.21.15.7:0/1507369627 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21fc004bb0 con 0x55bed489f7a0 2024-06-10T01:11:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.976+0000 7f21ecff9700 1 -- 172.21.15.7:0/1507369627 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f2200085b80 con 0x55bed489f7a0 2024-06-10T01:11:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.976+0000 7f2215ee2b80 10 client.16977 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:11:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.976+0000 7f2215ee2b80 10 client.16977 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:11:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.976+0000 7f2215ee2b80 20 client.16977 populate_metadata read hostname 'smithi007' 2024-06-10T01:11:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.976+0000 7f2215ee2b80 1 --2- 172.21.15.7:0/1507369627 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55bed48b8f90 0x55bed48bb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.976+0000 7f2215ee2b80 4 client.16977 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:11:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.976+0000 7f2215ee2b80 1 -- 172.21.15.7:0/1507369627 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bed47fdd50 con 0x55bed48b8f90 2024-06-10T01:11:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.976+0000 7f2208bfa700 1 --2- 172.21.15.7:0/1507369627 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55bed48b8f90 0x55bed48bb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.976+0000 7f21ecff9700 10 client.16977 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:11:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.976+0000 7f2208bfa700 1 --2- 172.21.15.7:0/1507369627 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55bed48b8f90 0x55bed48bb3e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9ed757a21c19eaf3 server_cookie=5d01580e127445b1 in_seq=0 out_seq=0 2024-06-10T01:11:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.977+0000 7f21ecff9700 1 -- 172.21.15.7:0/1507369627 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bed47fdd50 con 0x55bed48b8f90 2024-06-10T01:11:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.977+0000 7f21ecff9700 10 client.16977 handle_command_reply: tid=0 2024-06-10T01:11:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.977+0000 7f2215ee2b80 1 client.16977 shutdown 2024-06-10T01:11:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.977+0000 7f2215ee2b80 1 -- 172.21.15.7:0/1507369627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bed489f7a0 msgr2=0x55bed48ab590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.977+0000 7f2215ee2b80 1 --2- 172.21.15.7:0/1507369627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bed489f7a0 0x55bed48ab590 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f220009c2f0 tx=0x7f22000989c0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.977+0000 7f2215ee2b80 1 -- 172.21.15.7:0/1507369627 shutdown_connections 2024-06-10T01:11:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.977+0000 7f2215ee2b80 1 --2- 172.21.15.7:0/1507369627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bed48a5880 0x55bed48a8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.977+0000 7f2215ee2b80 1 --2- 172.21.15.7:0/1507369627 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55bed48b8f90 0x55bed48bb3e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.977+0000 7f2215ee2b80 1 --2- 172.21.15.7:0/1507369627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bed489f7a0 0x55bed48ab590 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.977+0000 7f2215ee2b80 1 --2- 172.21.15.7:0/1507369627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bed48a4f60 0x55bed48a8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.977+0000 7f2215ee2b80 1 -- 172.21.15.7:0/1507369627 >> 172.21.15.7:0/1507369627 conn(0x55bed47cc080 msgr2=0x55bed4899700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.977+0000 7f21ecff9700 0 client.16977 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:11:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.977+0000 7f2215ee2b80 1 -- 172.21.15.7:0/1507369627 shutdown_connections 2024-06-10T01:11:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.977+0000 7f2215ee2b80 1 -- 172.21.15.7:0/1507369627 wait complete. 2024-06-10T01:11:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.977+0000 7f2215ee2b80 20 client.16977 trim_cache size 0 max 16384 2024-06-10T01:11:57.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.993+0000 7f2215ee2b80 1 Processor -- start 2024-06-10T01:11:57.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.993+0000 7f2215ee2b80 1 -- start start 2024-06-10T01:11:57.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.993+0000 7f2215ee2b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bed489f7a0 0x55bed489f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:57.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.993+0000 7f2215ee2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bed48a4f60 0x55bed48a64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:57.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.993+0000 7f2215ee2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bed48a5880 0x55bed48a77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:57.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.993+0000 7f2215ee2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f22041b2bd0 con 0x55bed48a5880 2024-06-10T01:11:57.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.993+0000 7f2215ee2b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f22041b0460 con 0x55bed489f7a0 2024-06-10T01:11:57.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.993+0000 7f2215ee2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f22041ae2b0 con 0x55bed48a4f60 2024-06-10T01:11:57.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.993+0000 7f2209bfc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bed48a5880 0x55bed48a77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:57.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.993+0000 7f2209bfc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bed48a5880 0x55bed48a77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37606/0 (socket says 172.21.15.7:37606) 2024-06-10T01:11:57.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.993+0000 7f2209bfc700 1 -- 172.21.15.7:0/879450591 learned_addr learned my addr 172.21.15.7:0/879450591 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:57.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.993+0000 7f22093fb700 1 --2- 172.21.15.7:0/879450591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bed489f7a0 0x55bed489f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:57.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.993+0000 7f2208bfa700 1 --2- 172.21.15.7:0/879450591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bed48a4f60 0x55bed48a64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:57.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.993+0000 7f2209bfc700 1 -- 172.21.15.7:0/879450591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bed48a4f60 msgr2=0x55bed48a64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:57.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.993+0000 7f2209bfc700 1 --2- 172.21.15.7:0/879450591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bed48a4f60 0x55bed48a64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.993+0000 7f2209bfc700 1 -- 172.21.15.7:0/879450591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bed489f7a0 msgr2=0x55bed489f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:57.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.993+0000 7f2209bfc700 1 --2- 172.21.15.7:0/879450591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bed489f7a0 0x55bed489f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.993+0000 7f2209bfc700 1 -- 172.21.15.7:0/879450591 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bed48bba60 con 0x55bed48a5880 2024-06-10T01:11:57.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.994+0000 7f2208bfa700 1 --2- 172.21.15.7:0/879450591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bed48a4f60 0x55bed48a64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:11:57.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.993+0000 7f22093fb700 1 --2- 172.21.15.7:0/879450591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bed489f7a0 0x55bed489f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:11:57.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.994+0000 7f2209bfc700 1 --2- 172.21.15.7:0/879450591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bed48a5880 0x55bed48a77f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00f330 tx=0x7f21fc007910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=594c418fd59905d2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:57.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.994+0000 7f21eeffd700 1 -- 172.21.15.7:0/879450591 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc014880 con 0x55bed48a5880 2024-06-10T01:11:57.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.994+0000 7f21eeffd700 1 -- 172.21.15.7:0/879450591 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21fc014a20 con 0x55bed48a5880 2024-06-10T01:11:57.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.994+0000 7f21eeffd700 1 -- 172.21.15.7:0/879450591 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc0054b0 con 0x55bed48a5880 2024-06-10T01:11:57.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.994+0000 7f2215ee2b80 1 -- 172.21.15.7:0/879450591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bed48a5880 msgr2=0x55bed48a77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:57.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.994+0000 7f2215ee2b80 1 --2- 172.21.15.7:0/879450591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bed48a5880 0x55bed48a77f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00f330 tx=0x7f21fc007910 comp rx=0 tx=0).stop 2024-06-10T01:11:57.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.994+0000 7f2215ee2b80 1 -- 172.21.15.7:0/879450591 shutdown_connections 2024-06-10T01:11:57.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.994+0000 7f2215ee2b80 1 --2- 172.21.15.7:0/879450591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bed48a5880 0x55bed48a77f0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.994+0000 7f2215ee2b80 1 --2- 172.21.15.7:0/879450591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bed489f7a0 0x55bed489f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.994+0000 7f2215ee2b80 1 --2- 172.21.15.7:0/879450591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bed48a4f60 0x55bed48a64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.995+0000 7f2215ee2b80 1 -- 172.21.15.7:0/879450591 >> 172.21.15.7:0/879450591 conn(0x55bed47cc080 msgr2=0x55bed48aa0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:57.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.995+0000 7f2215ee2b80 1 -- 172.21.15.7:0/879450591 shutdown_connections 2024-06-10T01:11:57.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.995+0000 7f2215ee2b80 1 -- 172.21.15.7:0/879450591 wait complete. 2024-06-10T01:11:57.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.995+0000 7f2215ee2b80 1 -- start start 2024-06-10T01:11:57.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.995+0000 7f2215ee2b80 1 Processor -- start 2024-06-10T01:11:57.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.995+0000 7f2215ee2b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bed489f7a0 0x55bed48b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:57.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.995+0000 7f2215ee2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bed48a4f60 0x55bed48c03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:57.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.995+0000 7f2215ee2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bed48a5880 0x55bed48c0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:57.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.995+0000 7f2215ee2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f22041ae920 con 0x55bed48a4f60 2024-06-10T01:11:57.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.995+0000 7f2215ee2b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f22041afd90 con 0x55bed489f7a0 2024-06-10T01:11:57.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.995+0000 7f2215ee2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f22041aeff0 con 0x55bed48a5880 2024-06-10T01:11:57.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.995+0000 7f2215ee2b80 10 client.0 _ll_register_callbacks cb 0x55bed47865b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:11:57.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.996+0000 7f2208bfa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bed48a4f60 0x55bed48c03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:57.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.996+0000 7f2208bfa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bed48a4f60 0x55bed48c03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37610/0 (socket says 172.21.15.7:37610) 2024-06-10T01:11:57.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.996+0000 7f2208bfa700 1 -- 172.21.15.7:0/2935007768 learned_addr learned my addr 172.21.15.7:0/2935007768 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:11:57.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.996+0000 7f22093fb700 1 --2- 172.21.15.7:0/2935007768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bed489f7a0 0x55bed48b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:57.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.996+0000 7f2209bfc700 1 --2- 172.21.15.7:0/2935007768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bed48a5880 0x55bed48c0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:57.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.996+0000 7f2208bfa700 1 -- 172.21.15.7:0/2935007768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bed48a5880 msgr2=0x55bed48c0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:57.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.996+0000 7f2208bfa700 1 --2- 172.21.15.7:0/2935007768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bed48a5880 0x55bed48c0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.996+0000 7f2208bfa700 1 -- 172.21.15.7:0/2935007768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bed489f7a0 msgr2=0x55bed48b1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:11:57.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.996+0000 7f2208bfa700 1 --2- 172.21.15.7:0/2935007768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bed489f7a0 0x55bed48b1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.996+0000 7f2208bfa700 1 -- 172.21.15.7:0/2935007768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bed48c0ef0 con 0x55bed48a4f60 2024-06-10T01:11:57.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.996+0000 7f2208bfa700 1 --2- 172.21.15.7:0/2935007768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bed48a4f60 0x55bed48c03c0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x55bed48a9e60 tx=0x7f21f4013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=118b7272ede041e4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:57.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.996+0000 7f21ecff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:11:57.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.996+0000 7f21ecff9700 1 -- 172.21.15.7:0/2935007768 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f402c060 con 0x55bed48a4f60 2024-06-10T01:11:57.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.996+0000 7f21ecff9700 1 -- 172.21.15.7:0/2935007768 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f21f401edb0 con 0x55bed48a4f60 2024-06-10T01:11:57.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.996+0000 7f2215ee2b80 1 -- 172.21.15.7:0/2935007768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21fc014a20 con 0x55bed48a4f60 2024-06-10T01:11:57.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.996+0000 7f21ecff9700 1 -- 172.21.15.7:0/2935007768 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f21f4033050 con 0x55bed48a4f60 2024-06-10T01:11:57.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.996+0000 7f2215ee2b80 10 client.16983 fetch_fsmap learned FSMap version 32 2024-06-10T01:11:57.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.996+0000 7f2215ee2b80 1 -- 172.21.15.7:0/2935007768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bed4923aa0 con 0x55bed48a4f60 2024-06-10T01:11:57.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.997+0000 7f21ecff9700 1 -- 172.21.15.7:0/2935007768 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21fc014a20 con 0x55bed48a4f60 2024-06-10T01:11:57.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.997+0000 7f21ecff9700 1 -- 172.21.15.7:0/2935007768 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f21f4027340 con 0x55bed48a4f60 2024-06-10T01:11:57.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.997+0000 7f2215ee2b80 10 client.16983 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:11:57.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.997+0000 7f2215ee2b80 10 client.16983 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:11:57.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.997+0000 7f2215ee2b80 20 client.16983 populate_metadata read hostname 'smithi007' 2024-06-10T01:11:57.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.997+0000 7f2215ee2b80 1 --2- 172.21.15.7:0/2935007768 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55bed4924210 0x55bed4926630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:57.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.997+0000 7f2215ee2b80 4 client.16983 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:11:57.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.997+0000 7f2215ee2b80 1 -- 172.21.15.7:0/2935007768 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bed47f82d0 con 0x55bed4924210 2024-06-10T01:11:57.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.997+0000 7f22093fb700 1 --2- 172.21.15.7:0/2935007768 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55bed4924210 0x55bed4926630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:57.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.998+0000 7f22093fb700 1 --2- 172.21.15.7:0/2935007768 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55bed4924210 0x55bed4926630 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c29d3f3638d03fb9 server_cookie=6760b389ddd4f487 in_seq=0 out_seq=0 2024-06-10T01:11:57.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.998+0000 7f21ecff9700 10 client.16983 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:11:57.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.998+0000 7f21ecff9700 1 -- 172.21.15.7:0/2935007768 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bed47f82d0 con 0x55bed4924210 2024-06-10T01:11:57.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.998+0000 7f21ecff9700 10 client.16983 handle_command_reply: tid=0 2024-06-10T01:11:57.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.998+0000 7f2215ee2b80 1 client.16983 shutdown 2024-06-10T01:11:57.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.998+0000 7f2215ee2b80 1 -- 172.21.15.7:0/2935007768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bed48a4f60 msgr2=0x55bed48c03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:57.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.998+0000 7f2215ee2b80 1 --2- 172.21.15.7:0/2935007768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bed48a4f60 0x55bed48c03c0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x55bed48a9e60 tx=0x7f21f4013040 comp rx=0 tx=0).stop 2024-06-10T01:11:57.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.998+0000 7f2215ee2b80 1 -- 172.21.15.7:0/2935007768 shutdown_connections 2024-06-10T01:11:57.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.998+0000 7f2215ee2b80 1 --2- 172.21.15.7:0/2935007768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bed48a5880 0x55bed48c0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.998+0000 7f2215ee2b80 1 --2- 172.21.15.7:0/2935007768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bed489f7a0 0x55bed48b1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.998+0000 7f2215ee2b80 1 --2- 172.21.15.7:0/2935007768 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55bed4924210 0x55bed4926630 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.998+0000 7f2215ee2b80 1 --2- 172.21.15.7:0/2935007768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bed48a4f60 0x55bed48c03c0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.998+0000 7f21ecff9700 0 client.16983 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:11:57.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.998+0000 7f2215ee2b80 1 -- 172.21.15.7:0/2935007768 >> 172.21.15.7:0/2935007768 conn(0x55bed47cc080 msgr2=0x55bed48a6370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:58.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.998+0000 7f2215ee2b80 1 -- 172.21.15.7:0/2935007768 shutdown_connections 2024-06-10T01:11:58.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.998+0000 7f2215ee2b80 1 -- 172.21.15.7:0/2935007768 wait complete. 2024-06-10T01:11:58.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.998+0000 7f2215ee2b80 20 client.16983 trim_cache size 0 max 16384 2024-06-10T01:11:58.000 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:11:58.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.999+0000 7f220b65f700 1 -- 172.21.15.7:0/2127236479 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21f00a2c20 msgr2=0x7f21f00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:58.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.999+0000 7f220b65f700 1 --2- 172.21.15.7:0/2127236479 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21f00a2c20 0x7f21f00a50e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2204108bb0 tx=0x7f21f4009260 comp rx=0 tx=0).stop 2024-06-10T01:11:58.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.999+0000 7f220b65f700 1 -- 172.21.15.7:0/2127236479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2204102540 msgr2=0x7f2204074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:58.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.999+0000 7f220b65f700 1 --2- 172.21.15.7:0/2127236479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2204102540 0x7f2204074790 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f2200000c00 tx=0x7f220000bab0 comp rx=0 tx=0).stop 2024-06-10T01:11:58.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.999+0000 7f220b65f700 1 -- 172.21.15.7:0/2127236479 shutdown_connections 2024-06-10T01:11:58.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.999+0000 7f220b65f700 1 --2- 172.21.15.7:0/2127236479 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21f00a2c20 0x7f21f00a50e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:58.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.999+0000 7f220b65f700 1 --2- 172.21.15.7:0/2127236479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2204107450 0x7f2204075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:58.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.999+0000 7f220b65f700 1 --2- 172.21.15.7:0/2127236479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2204102540 0x7f2204074790 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:58.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.999+0000 7f220b65f700 1 --2- 172.21.15.7:0/2127236479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2204102ee0 0x7f2204074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:58.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.999+0000 7f220b65f700 1 -- 172.21.15.7:0/2127236479 >> 172.21.15.7:0/2127236479 conn(0x7f22040795f0 msgr2=0x7f2204100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:58.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.999+0000 7f220b65f700 1 -- 172.21.15.7:0/2127236479 shutdown_connections 2024-06-10T01:11:58.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:11:57.999+0000 7f220b65f700 1 -- 172.21.15.7:0/2127236479 wait complete. 2024-06-10T01:11:58.017 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:11:58.585 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:11:58 smithi007 ceph-mon[32398]: pgmap v611: 129 pgs: 129 active+clean; 69 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 319 KiB/s wr, 16 op/s 2024-06-10T01:11:58.733 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 6.22 MB/sec execute 159 sec latency 24887.399 ms 2024-06-10T01:11:58.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:11:58 smithi078 ceph-mon[36683]: pgmap v611: 129 pgs: 129 active+clean; 69 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 319 KiB/s wr, 16 op/s 2024-06-10T01:11:58.953 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:11:58 smithi175 ceph-mon[36636]: pgmap v611: 129 pgs: 129 active+clean; 69 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 319 KiB/s wr, 16 op/s 2024-06-10T01:11:59.021 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:11:59.105 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:11:59.733 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 6.18 MB/sec execute 160 sec latency 25887.480 ms 2024-06-10T01:12:00.404 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:12:00.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.617+0000 7feabb064640 1 -- 172.21.15.7:0/243901359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feab4107710 msgr2=0x7feab4109b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:00.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.617+0000 7feabb064640 1 --2- 172.21.15.7:0/243901359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feab4107710 0x7feab4109b60 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7feaac009ca0 tx=0x7feaac02a8c0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.617+0000 7feabb064640 1 -- 172.21.15.7:0/243901359 shutdown_connections 2024-06-10T01:12:00.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.617+0000 7feabb064640 1 --2- 172.21.15.7:0/243901359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feab4107710 0x7feab4109b60 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.617+0000 7feabb064640 1 --2- 172.21.15.7:0/243901359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feab4104de0 0x7feab41071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.617+0000 7feabb064640 1 --2- 172.21.15.7:0/243901359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feab41024b0 0x7feab41048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.617+0000 7feabb064640 1 -- 172.21.15.7:0/243901359 >> 172.21.15.7:0/243901359 conn(0x7feab40fbec0 msgr2=0x7feab40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:00.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.617+0000 7feabb064640 1 -- 172.21.15.7:0/243901359 shutdown_connections 2024-06-10T01:12:00.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.617+0000 7feabb064640 1 -- 172.21.15.7:0/243901359 wait complete. 2024-06-10T01:12:00.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.618+0000 7feabb064640 1 Processor -- start 2024-06-10T01:12:00.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.618+0000 7feabb064640 1 -- start start 2024-06-10T01:12:00.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.618+0000 7feabb064640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feab41024b0 0x7feab4197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:00.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.618+0000 7feabb064640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feab4104de0 0x7feab41981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:00.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.618+0000 7feabb064640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feab4107710 0x7feab419f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:00.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.618+0000 7feabb064640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feab410d6d0 con 0x7feab41024b0 2024-06-10T01:12:00.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.618+0000 7feabb064640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feab410d550 con 0x7feab4107710 2024-06-10T01:12:00.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.618+0000 7feabb064640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feab410d850 con 0x7feab4104de0 2024-06-10T01:12:00.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.618+0000 7feab8e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feab41024b0 0x7feab4197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:00.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.618+0000 7feab8e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feab41024b0 0x7feab4197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37622/0 (socket says 172.21.15.7:37622) 2024-06-10T01:12:00.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.618+0000 7feab8e00640 1 -- 172.21.15.7:0/3515864540 learned_addr learned my addr 172.21.15.7:0/3515864540 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:00.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.618+0000 7feab9601640 1 --2- 172.21.15.7:0/3515864540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feab4107710 0x7feab419f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:00.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.618+0000 7feab3fff640 1 --2- 172.21.15.7:0/3515864540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feab4104de0 0x7feab41981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:00.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.618+0000 7feab8e00640 1 -- 172.21.15.7:0/3515864540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feab4104de0 msgr2=0x7feab41981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:00.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.618+0000 7feab8e00640 1 --2- 172.21.15.7:0/3515864540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feab4104de0 0x7feab41981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.618+0000 7feab8e00640 1 -- 172.21.15.7:0/3515864540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feab4107710 msgr2=0x7feab419f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:00.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.618+0000 7feab8e00640 1 --2- 172.21.15.7:0/3515864540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feab4107710 0x7feab419f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.618+0000 7feab8e00640 1 -- 172.21.15.7:0/3515864540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab419f8d0 con 0x7feab41024b0 2024-06-10T01:12:00.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.619+0000 7feab8e00640 1 --2- 172.21.15.7:0/3515864540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feab41024b0 0x7feab4197c90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7feaa800e9f0 tx=0x7feaa800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:00.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.619+0000 7feab1ffb640 1 -- 172.21.15.7:0/3515864540 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa800cd80 con 0x7feab41024b0 2024-06-10T01:12:00.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.619+0000 7feab1ffb640 1 -- 172.21.15.7:0/3515864540 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaa8010910 con 0x7feab41024b0 2024-06-10T01:12:00.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.619+0000 7feabb064640 1 -- 172.21.15.7:0/3515864540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feab419fbc0 con 0x7feab41024b0 2024-06-10T01:12:00.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.619+0000 7feab1ffb640 1 -- 172.21.15.7:0/3515864540 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8018990 con 0x7feab41024b0 2024-06-10T01:12:00.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.619+0000 7feabb064640 1 -- 172.21.15.7:0/3515864540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feab41a0160 con 0x7feab41024b0 2024-06-10T01:12:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.622+0000 7feab1ffb640 1 -- 172.21.15.7:0/3515864540 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7feaa8010ac0 con 0x7feab41024b0 2024-06-10T01:12:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.622+0000 7feabdd75740 1 Processor -- start 2024-06-10T01:12:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.622+0000 7feabdd75740 1 -- start start 2024-06-10T01:12:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.622+0000 7feabdd75740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b8bab9420 0x557b8bab9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.622+0000 7feabdd75740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b8bab9e10 0x557b8babe910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.622+0000 7feabdd75740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b8babef80 0x557b8bac1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.622+0000 7feabdd75740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557b8ba184f0 con 0x557b8bab9e10 2024-06-10T01:12:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.622+0000 7feabdd75740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557b8b9b6870 con 0x557b8bab9420 2024-06-10T01:12:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.622+0000 7feabdd75740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557b8ba144b0 con 0x557b8babef80 2024-06-10T01:12:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.623+0000 7feab8e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b8bab9e10 0x557b8babe910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.623+0000 7feab8e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b8bab9e10 0x557b8babe910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37634/0 (socket says 172.21.15.7:37634) 2024-06-10T01:12:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.623+0000 7feab8e00640 1 -- 172.21.15.7:0/1232155174 learned_addr learned my addr 172.21.15.7:0/1232155174 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.623+0000 7feab3fff640 1 --2- 172.21.15.7:0/1232155174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b8babef80 0x557b8bac1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.623+0000 7feab9601640 1 --2- 172.21.15.7:0/1232155174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b8bab9420 0x557b8bab9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.623+0000 7feab1ffb640 1 --2- 172.21.15.7:0/3515864540 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7feaa40a2c00 0x7feaa40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.623+0000 7feab8e00640 1 -- 172.21.15.7:0/1232155174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b8babef80 msgr2=0x557b8bac1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.623+0000 7feab8e00640 1 --2- 172.21.15.7:0/1232155174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b8babef80 0x557b8bac1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.623+0000 7feab8e00640 1 -- 172.21.15.7:0/1232155174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b8bab9420 msgr2=0x557b8bab9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.623+0000 7feab8e00640 1 --2- 172.21.15.7:0/1232155174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b8bab9420 0x557b8bab9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.623+0000 7feab8e00640 1 -- 172.21.15.7:0/1232155174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b8b9d29b0 con 0x557b8bab9e10 2024-06-10T01:12:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.623+0000 7feab9601640 1 --2- 172.21.15.7:0/1232155174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b8bab9420 0x557b8bab9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:12:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.623+0000 7feab9601640 1 --2- 172.21.15.7:0/3515864540 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7feaa40a2c00 0x7feaa40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.623+0000 7feab1ffb640 1 -- 172.21.15.7:0/3515864540 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7feaa80cf690 con 0x7feab41024b0 2024-06-10T01:12:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.623+0000 7feab8e00640 1 --2- 172.21.15.7:0/1232155174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b8bab9e10 0x557b8babe910 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7feaa80861a0 tx=0x7feaa801d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b751fdd6f29b2e61 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.623+0000 7fea9effd640 1 -- 172.21.15.7:0/1232155174 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa809b700 con 0x557b8bab9e10 2024-06-10T01:12:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.624+0000 7fea9effd640 1 -- 172.21.15.7:0/1232155174 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaa809bbc0 con 0x557b8bab9e10 2024-06-10T01:12:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.624+0000 7fea9effd640 1 -- 172.21.15.7:0/1232155174 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa809b520 con 0x557b8bab9e10 2024-06-10T01:12:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.624+0000 7feabdd75740 1 -- 172.21.15.7:0/1232155174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b8bab9e10 msgr2=0x557b8babe910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.624+0000 7feabdd75740 1 --2- 172.21.15.7:0/1232155174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b8bab9e10 0x557b8babe910 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7feaa80861a0 tx=0x7feaa801d9c0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.624+0000 7feab9601640 1 --2- 172.21.15.7:0/3515864540 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7feaa40a2c00 0x7feaa40a50c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x557b8baba670 tx=0x7feaac03f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.624+0000 7feabdd75740 1 -- 172.21.15.7:0/1232155174 shutdown_connections 2024-06-10T01:12:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.624+0000 7feabdd75740 1 --2- 172.21.15.7:0/1232155174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b8babef80 0x557b8bac1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.624+0000 7feabdd75740 1 --2- 172.21.15.7:0/1232155174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b8bab9e10 0x557b8babe910 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.624+0000 7feabdd75740 1 --2- 172.21.15.7:0/1232155174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b8bab9420 0x557b8bab9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.624+0000 7feabdd75740 1 -- 172.21.15.7:0/1232155174 >> 172.21.15.7:0/1232155174 conn(0x557b8bab0680 msgr2=0x557b8bab0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:00.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.624+0000 7feabdd75740 1 -- 172.21.15.7:0/1232155174 shutdown_connections 2024-06-10T01:12:00.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.624+0000 7feabdd75740 1 -- 172.21.15.7:0/1232155174 wait complete. 2024-06-10T01:12:00.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.625+0000 7feabdd75740 1 -- start start 2024-06-10T01:12:00.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.625+0000 7feabdd75740 1 Processor -- start 2024-06-10T01:12:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.625+0000 7feabdd75740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b8bab9420 0x557b8bac35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.625+0000 7feabdd75740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b8bab9e10 0x557b8bac3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.625+0000 7feabdd75740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b8babef80 0x557b8bacab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.625+0000 7feabdd75740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557b8bac1ed0 con 0x557b8bab9e10 2024-06-10T01:12:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.625+0000 7feabdd75740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557b8b9a1ae0 con 0x557b8babef80 2024-06-10T01:12:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.625+0000 7feabdd75740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557b8bac2050 con 0x557b8bab9420 2024-06-10T01:12:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.625+0000 7feabdd75740 10 client.0 _ll_register_callbacks cb 0x557b8b9da9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:12:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.625+0000 7feab3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b8bab9e10 0x557b8bac3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.625+0000 7feab3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b8bab9e10 0x557b8bac3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37646/0 (socket says 172.21.15.7:37646) 2024-06-10T01:12:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.625+0000 7feab3fff640 1 -- 172.21.15.7:0/3197478404 learned_addr learned my addr 172.21.15.7:0/3197478404 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.625+0000 7feab9601640 1 --2- 172.21.15.7:0/3197478404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b8babef80 0x557b8bacab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.625+0000 7feab3fff640 1 -- 172.21.15.7:0/3197478404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b8bab9420 msgr2=0x557b8bac35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.625+0000 7feab3fff640 1 --2- 172.21.15.7:0/3197478404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b8bab9420 0x557b8bac35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.625+0000 7feab3fff640 1 -- 172.21.15.7:0/3197478404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b8babef80 msgr2=0x557b8bacab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.625+0000 7feab3fff640 1 --2- 172.21.15.7:0/3197478404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b8babef80 0x557b8bacab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.625+0000 7feab3fff640 1 -- 172.21.15.7:0/3197478404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b8b72c210 con 0x557b8bab9e10 2024-06-10T01:12:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.626+0000 7feab3fff640 1 --2- 172.21.15.7:0/3197478404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b8bab9e10 0x557b8bac3ae0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7feaa00126e0 tx=0x7feaa0012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98a18e14caa67c9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.626+0000 7fea93fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:12:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.626+0000 7fea93fff640 1 -- 172.21.15.7:0/3197478404 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa0012190 con 0x557b8bab9e10 2024-06-10T01:12:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.626+0000 7feabdd75740 1 -- 172.21.15.7:0/3197478404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaa809bbc0 con 0x557b8bab9e10 2024-06-10T01:12:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.626+0000 7fea93fff640 1 -- 172.21.15.7:0/3197478404 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7feaa0004020 con 0x557b8bab9e10 2024-06-10T01:12:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.626+0000 7fea93fff640 1 -- 172.21.15.7:0/3197478404 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feaa0025050 con 0x557b8bab9e10 2024-06-10T01:12:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.626+0000 7feabdd75740 10 client.16995 fetch_fsmap learned FSMap version 32 2024-06-10T01:12:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.626+0000 7feabdd75740 1 -- 172.21.15.7:0/3197478404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557b8b7f8620 con 0x557b8bab9e10 2024-06-10T01:12:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.626+0000 7fea93fff640 1 -- 172.21.15.7:0/3197478404 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaa809bbc0 con 0x557b8bab9e10 2024-06-10T01:12:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.626+0000 7fea93fff640 1 -- 172.21.15.7:0/3197478404 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7feaa00037b0 con 0x557b8bab9e10 2024-06-10T01:12:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.626+0000 7feabdd75740 10 client.16995 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:12:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.626+0000 7feabdd75740 10 client.16995 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:12:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.626+0000 7feabdd75740 20 client.16995 populate_metadata read hostname 'smithi007' 2024-06-10T01:12:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.626+0000 7feabdd75740 1 --2- 172.21.15.7:0/3197478404 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557b8bacf520 0x557b8bad19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.626+0000 7feabdd75740 4 client.16995 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:12:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.626+0000 7feabdd75740 1 -- 172.21.15.7:0/3197478404 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b8ba2bf10 con 0x557b8bacf520 2024-06-10T01:12:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.627+0000 7feab8e00640 1 --2- 172.21.15.7:0/3197478404 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557b8bacf520 0x557b8bad19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.627+0000 7feab8e00640 1 --2- 172.21.15.7:0/3197478404 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557b8bacf520 0x557b8bad19e0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d20dfc93e073977 server_cookie=fc7c7d6e302fa291 in_seq=0 out_seq=0 2024-06-10T01:12:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.627+0000 7fea93fff640 10 client.16995 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.628+0000 7fea93fff640 1 -- 172.21.15.7:0/3197478404 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557b8ba2bf10 con 0x557b8bacf520 2024-06-10T01:12:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.628+0000 7fea93fff640 10 client.16995 handle_command_reply: tid=0 2024-06-10T01:12:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.628+0000 7feabdd75740 1 client.16995 shutdown 2024-06-10T01:12:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.628+0000 7feabdd75740 1 -- 172.21.15.7:0/3197478404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b8bab9e10 msgr2=0x557b8bac3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.628+0000 7feabdd75740 1 --2- 172.21.15.7:0/3197478404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b8bab9e10 0x557b8bac3ae0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7feaa00126e0 tx=0x7feaa0012ba0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.628+0000 7feabdd75740 1 -- 172.21.15.7:0/3197478404 shutdown_connections 2024-06-10T01:12:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.628+0000 7feabdd75740 1 --2- 172.21.15.7:0/3197478404 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557b8bacf520 0x557b8bad19e0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.628+0000 7feabdd75740 1 --2- 172.21.15.7:0/3197478404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b8babef80 0x557b8bacab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.628+0000 7feabdd75740 1 --2- 172.21.15.7:0/3197478404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b8bab9e10 0x557b8bac3ae0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.628+0000 7feabdd75740 1 --2- 172.21.15.7:0/3197478404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b8bab9420 0x557b8bac35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.628+0000 7fea93fff640 0 client.16995 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.628+0000 7feabdd75740 1 -- 172.21.15.7:0/3197478404 >> 172.21.15.7:0/3197478404 conn(0x557b8bab0680 msgr2=0x557b8baaf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.628+0000 7feabdd75740 1 -- 172.21.15.7:0/3197478404 shutdown_connections 2024-06-10T01:12:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.628+0000 7feabdd75740 1 -- 172.21.15.7:0/3197478404 wait complete. 2024-06-10T01:12:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.628+0000 7feabdd75740 20 client.16995 trim_cache size 0 max 16384 2024-06-10T01:12:00.643 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.643+0000 7feabdd75740 1 Processor -- start 2024-06-10T01:12:00.643 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.643+0000 7feabdd75740 1 -- start start 2024-06-10T01:12:00.643 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.643+0000 7feabdd75740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b8bab9420 0x557b8bad1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:00.643 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.643+0000 7feabdd75740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b8bab9e10 0x557b8bacaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:00.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.643+0000 7feabdd75740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b8babef80 0x557b8bac35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:00.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.643+0000 7feabdd75740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feab41aac00 con 0x557b8bab9420 2024-06-10T01:12:00.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.643+0000 7feabdd75740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feab41a84d0 con 0x557b8bab9e10 2024-06-10T01:12:00.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.643+0000 7feabdd75740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feab41a6320 con 0x557b8babef80 2024-06-10T01:12:00.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.644+0000 7feab8e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b8bab9420 0x557b8bad1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:00.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.644+0000 7feab8e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b8bab9420 0x557b8bad1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37648/0 (socket says 172.21.15.7:37648) 2024-06-10T01:12:00.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.644+0000 7feab8e00640 1 -- 172.21.15.7:0/236282052 learned_addr learned my addr 172.21.15.7:0/236282052 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:00.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.644+0000 7feab3fff640 1 --2- 172.21.15.7:0/236282052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b8bab9e10 0x557b8bacaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:00.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.644+0000 7feab9601640 1 --2- 172.21.15.7:0/236282052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b8babef80 0x557b8bac35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:00.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.644+0000 7feab8e00640 1 -- 172.21.15.7:0/236282052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b8babef80 msgr2=0x557b8bac35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:00.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.644+0000 7feab8e00640 1 --2- 172.21.15.7:0/236282052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b8babef80 0x557b8bac35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.644+0000 7feab8e00640 1 -- 172.21.15.7:0/236282052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b8bab9e10 msgr2=0x557b8bacaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:00.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.644+0000 7feab8e00640 1 --2- 172.21.15.7:0/236282052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b8bab9e10 0x557b8bacaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.644+0000 7feab8e00640 1 -- 172.21.15.7:0/236282052 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b8bab04b0 con 0x557b8bab9420 2024-06-10T01:12:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.644+0000 7feab9601640 1 --2- 172.21.15.7:0/236282052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b8babef80 0x557b8bac35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:12:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.644+0000 7feab3fff640 1 --2- 172.21.15.7:0/236282052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b8bab9e10 0x557b8bacaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:12:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.644+0000 7feab8e00640 1 --2- 172.21.15.7:0/236282052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b8bab9420 0x557b8bad1530 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7feaa800c6a0 tx=0x7feaa8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd79053252a7bb5a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.644+0000 7fea9e7fc640 1 -- 172.21.15.7:0/236282052 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa80a4780 con 0x557b8bab9420 2024-06-10T01:12:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.644+0000 7fea9e7fc640 1 -- 172.21.15.7:0/236282052 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaa809bbc0 con 0x557b8bab9420 2024-06-10T01:12:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.644+0000 7fea9e7fc640 1 -- 172.21.15.7:0/236282052 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa80ccec0 con 0x557b8bab9420 2024-06-10T01:12:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.645+0000 7feabdd75740 1 -- 172.21.15.7:0/236282052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b8bab9420 msgr2=0x557b8bad1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.645+0000 7feabdd75740 1 --2- 172.21.15.7:0/236282052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b8bab9420 0x557b8bad1530 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7feaa800c6a0 tx=0x7feaa8009310 comp rx=0 tx=0).stop 2024-06-10T01:12:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.645+0000 7feabdd75740 1 -- 172.21.15.7:0/236282052 shutdown_connections 2024-06-10T01:12:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.645+0000 7feabdd75740 1 --2- 172.21.15.7:0/236282052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b8babef80 0x557b8bac35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.645+0000 7feabdd75740 1 --2- 172.21.15.7:0/236282052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b8bab9e10 0x557b8bacaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.645+0000 7feabdd75740 1 --2- 172.21.15.7:0/236282052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b8bab9420 0x557b8bad1530 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.645+0000 7feabdd75740 1 -- 172.21.15.7:0/236282052 >> 172.21.15.7:0/236282052 conn(0x557b8bab0680 msgr2=0x557b8baab560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.645+0000 7feabdd75740 1 -- 172.21.15.7:0/236282052 shutdown_connections 2024-06-10T01:12:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.645+0000 7feabdd75740 1 -- 172.21.15.7:0/236282052 wait complete. 2024-06-10T01:12:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.645+0000 7feabdd75740 1 -- start start 2024-06-10T01:12:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.645+0000 7feabdd75740 1 Processor -- start 2024-06-10T01:12:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.646+0000 7feabdd75740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b8bab9420 0x557b8baca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.646+0000 7feabdd75740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b8bab9e10 0x557b8bac72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.646+0000 7feabdd75740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b8babef80 0x557b8bac7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.646+0000 7feabdd75740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feab41a6990 con 0x557b8bab9420 2024-06-10T01:12:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.646+0000 7feabdd75740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feab41a7e00 con 0x557b8bab9e10 2024-06-10T01:12:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.646+0000 7feabdd75740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feab41a7060 con 0x557b8babef80 2024-06-10T01:12:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.646+0000 7feabdd75740 10 client.0 _ll_register_callbacks cb 0x7feaa0009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:12:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.646+0000 7feab8e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b8bab9420 0x557b8baca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.646+0000 7feab8e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b8bab9420 0x557b8baca7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37658/0 (socket says 172.21.15.7:37658) 2024-06-10T01:12:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.646+0000 7feab8e00640 1 -- 172.21.15.7:0/3511945786 learned_addr learned my addr 172.21.15.7:0/3511945786 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.646+0000 7feab3fff640 1 --2- 172.21.15.7:0/3511945786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b8bab9e10 0x557b8bac72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.646+0000 7feab8e00640 1 -- 172.21.15.7:0/3511945786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b8babef80 msgr2=0x557b8bac7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.646+0000 7feab8e00640 1 --2- 172.21.15.7:0/3511945786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b8babef80 0x557b8bac7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.646+0000 7feab8e00640 1 -- 172.21.15.7:0/3511945786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b8bab9e10 msgr2=0x557b8bac72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.646+0000 7feab8e00640 1 --2- 172.21.15.7:0/3511945786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b8bab9e10 0x557b8bac72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.646+0000 7feab8e00640 1 -- 172.21.15.7:0/3511945786 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b8bac7e20 con 0x557b8bab9420 2024-06-10T01:12:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.646+0000 7feab3fff640 1 --2- 172.21.15.7:0/3511945786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b8bab9e10 0x557b8bac72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:12:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.646+0000 7feab8e00640 1 --2- 172.21.15.7:0/3511945786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b8bab9420 0x557b8baca7b0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7feaa80978a0 tx=0x7feaa809b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3347d05e92634eb7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.646+0000 7fea93fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:12:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.646+0000 7fea93fff640 1 -- 172.21.15.7:0/3511945786 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa80afd30 con 0x557b8bab9420 2024-06-10T01:12:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.646+0000 7fea93fff640 1 -- 172.21.15.7:0/3511945786 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7feaa80cf3b0 con 0x557b8bab9420 2024-06-10T01:12:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.646+0000 7feabdd75740 1 -- 172.21.15.7:0/3511945786 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaa809bbc0 con 0x557b8bab9420 2024-06-10T01:12:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.646+0000 7fea93fff640 1 -- 172.21.15.7:0/3511945786 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feaa80cce30 con 0x557b8bab9420 2024-06-10T01:12:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.646+0000 7feabdd75740 10 client.17001 fetch_fsmap learned FSMap version 32 2024-06-10T01:12:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.646+0000 7feabdd75740 1 -- 172.21.15.7:0/3511945786 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557b8bb392a0 con 0x557b8bab9420 2024-06-10T01:12:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.647+0000 7fea93fff640 1 -- 172.21.15.7:0/3511945786 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaa809bbc0 con 0x557b8bab9420 2024-06-10T01:12:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.647+0000 7fea93fff640 1 -- 172.21.15.7:0/3511945786 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7feaa809be20 con 0x557b8bab9420 2024-06-10T01:12:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.647+0000 7feabdd75740 10 client.17001 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:12:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.647+0000 7feabdd75740 10 client.17001 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:12:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.647+0000 7feabdd75740 20 client.17001 populate_metadata read hostname 'smithi007' 2024-06-10T01:12:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.647+0000 7feabdd75740 1 --2- 172.21.15.7:0/3511945786 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557b8bb396e0 0x557b8bb3bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.647+0000 7feabdd75740 4 client.17001 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:12:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.647+0000 7feabdd75740 1 -- 172.21.15.7:0/3511945786 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557b8ba25c20 con 0x557b8bb396e0 2024-06-10T01:12:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.647+0000 7feab3fff640 1 --2- 172.21.15.7:0/3511945786 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557b8bb396e0 0x557b8bb3bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.648+0000 7feab3fff640 1 --2- 172.21.15.7:0/3511945786 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557b8bb396e0 0x557b8bb3bad0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7878f6e2394f08cb server_cookie=9683f456882f4a70 in_seq=0 out_seq=0 2024-06-10T01:12:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.648+0000 7fea93fff640 10 client.17001 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.684+0000 7fea93fff640 1 -- 172.21.15.7:0/3511945786 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557b8ba25c20 con 0x557b8bb396e0 2024-06-10T01:12:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.684+0000 7fea93fff640 10 client.17001 handle_command_reply: tid=0 2024-06-10T01:12:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.684+0000 7feabdd75740 1 client.17001 shutdown 2024-06-10T01:12:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.684+0000 7feabdd75740 1 -- 172.21.15.7:0/3511945786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b8bab9420 msgr2=0x557b8baca7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.684+0000 7feabdd75740 1 --2- 172.21.15.7:0/3511945786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b8bab9420 0x557b8baca7b0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7feaa80978a0 tx=0x7feaa809b800 comp rx=0 tx=0).stop 2024-06-10T01:12:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.684+0000 7feabdd75740 1 -- 172.21.15.7:0/3511945786 shutdown_connections 2024-06-10T01:12:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.684+0000 7feabdd75740 1 --2- 172.21.15.7:0/3511945786 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557b8bb396e0 0x557b8bb3bad0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.684+0000 7feabdd75740 1 --2- 172.21.15.7:0/3511945786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b8babef80 0x557b8bac7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.684+0000 7feabdd75740 1 --2- 172.21.15.7:0/3511945786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b8bab9e10 0x557b8bac72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.684+0000 7fea93fff640 0 client.17001 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.684+0000 7feabdd75740 1 --2- 172.21.15.7:0/3511945786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b8bab9420 0x557b8baca7b0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.684+0000 7feabdd75740 1 -- 172.21.15.7:0/3511945786 >> 172.21.15.7:0/3511945786 conn(0x557b8bab0680 msgr2=0x557b8bac3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.684+0000 7feabdd75740 1 -- 172.21.15.7:0/3511945786 shutdown_connections 2024-06-10T01:12:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.684+0000 7feabdd75740 1 -- 172.21.15.7:0/3511945786 wait complete. 2024-06-10T01:12:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.684+0000 7feabdd75740 20 client.17001 trim_cache size 0 max 16384 2024-06-10T01:12:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:12:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:12:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:12:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.684+0000 7feabb064640 1 -- 172.21.15.7:0/3515864540 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7feaa40a2c00 msgr2=0x7feaa40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.684+0000 7feabb064640 1 --2- 172.21.15.7:0/3515864540 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7feaa40a2c00 0x7feaa40a50c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x557b8baba670 tx=0x7feaac03f040 comp rx=0 tx=0).stop 2024-06-10T01:12:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.684+0000 7feabb064640 1 -- 172.21.15.7:0/3515864540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feab41024b0 msgr2=0x7feab4197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.684+0000 7feabb064640 1 --2- 172.21.15.7:0/3515864540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feab41024b0 0x7feab4197c90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7feaa800e9f0 tx=0x7feaa800eeb0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.684+0000 7feabb064640 1 -- 172.21.15.7:0/3515864540 shutdown_connections 2024-06-10T01:12:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.684+0000 7feabb064640 1 --2- 172.21.15.7:0/3515864540 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7feaa40a2c00 0x7feaa40a50c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.684+0000 7feabb064640 1 --2- 172.21.15.7:0/3515864540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feab4107710 0x7feab419f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.684+0000 7feabb064640 1 --2- 172.21.15.7:0/3515864540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feab4104de0 0x7feab41981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.684+0000 7feabb064640 1 --2- 172.21.15.7:0/3515864540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feab41024b0 0x7feab4197c90 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.684+0000 7feabb064640 1 -- 172.21.15.7:0/3515864540 >> 172.21.15.7:0/3515864540 conn(0x7feab40fbec0 msgr2=0x7feab4102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.684+0000 7feabb064640 1 -- 172.21.15.7:0/3515864540 shutdown_connections 2024-06-10T01:12:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:00.685+0000 7feabb064640 1 -- 172.21.15.7:0/3515864540 wait complete. 2024-06-10T01:12:00.733 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 6.14 MB/sec execute 161 sec latency 26887.581 ms 2024-06-10T01:12:00.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:00 smithi007 ceph-mon[32398]: pgmap v612: 129 pgs: 129 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 999 KiB/s wr, 40 op/s 2024-06-10T01:12:01.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:00 smithi078 ceph-mon[36683]: pgmap v612: 129 pgs: 129 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 999 KiB/s wr, 40 op/s 2024-06-10T01:12:01.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:00 smithi175 ceph-mon[36636]: pgmap v612: 129 pgs: 129 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 999 KiB/s wr, 40 op/s 2024-06-10T01:12:01.733 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 6.10 MB/sec execute 162 sec latency 27887.624 ms 2024-06-10T01:12:02.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.263+0000 7efd006c8700 1 -- 172.21.15.7:0/148063132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcf8107840 msgr2=0x7efcf810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.263+0000 7efd006c8700 1 --2- 172.21.15.7:0/148063132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcf8107840 0x7efcf810f3c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7efcf000b0d0 tx=0x7efcf000b3e0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.264+0000 7efd006c8700 1 -- 172.21.15.7:0/148063132 shutdown_connections 2024-06-10T01:12:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.264+0000 7efd006c8700 1 --2- 172.21.15.7:0/148063132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcf8107840 0x7efcf810f3c0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.264+0000 7efd006c8700 1 --2- 172.21.15.7:0/148063132 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcf8102740 0x7efcf8102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.264+0000 7efd006c8700 1 --2- 172.21.15.7:0/148063132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcf81030f0 0x7efcf8107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.264+0000 7efd006c8700 1 -- 172.21.15.7:0/148063132 >> 172.21.15.7:0/148063132 conn(0x7efcf80795f0 msgr2=0x7efcf8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.264+0000 7efd006c8700 1 -- 172.21.15.7:0/148063132 shutdown_connections 2024-06-10T01:12:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.264+0000 7efd006c8700 1 -- 172.21.15.7:0/148063132 wait complete. 2024-06-10T01:12:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.264+0000 7efd006c8700 1 Processor -- start 2024-06-10T01:12:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.264+0000 7efd006c8700 1 -- start start 2024-06-10T01:12:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.265+0000 7efd006c8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcf8102740 0x7efcf819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.265+0000 7efd006c8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcf81030f0 0x7efcf819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.265+0000 7efd006c8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcf8107840 0x7efcf81989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.265+0000 7efd006c8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efcf81127d0 con 0x7efcf81030f0 2024-06-10T01:12:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.265+0000 7efd006c8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efcf8112650 con 0x7efcf8102740 2024-06-10T01:12:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.265+0000 7efd006c8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efcf8112950 con 0x7efcf8107840 2024-06-10T01:12:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.265+0000 7efcfdc63700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcf81030f0 0x7efcf819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.265+0000 7efcfdc63700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcf81030f0 0x7efcf819ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58960/0 (socket says 172.21.15.7:58960) 2024-06-10T01:12:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.265+0000 7efcfdc63700 1 -- 172.21.15.7:0/1583117529 learned_addr learned my addr 172.21.15.7:0/1583117529 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.265+0000 7efcfe464700 1 --2- 172.21.15.7:0/1583117529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcf8102740 0x7efcf819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.265+0000 7efcfec65700 1 --2- 172.21.15.7:0/1583117529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcf8107840 0x7efcf81989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.265+0000 7efcfdc63700 1 -- 172.21.15.7:0/1583117529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcf8107840 msgr2=0x7efcf81989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.265+0000 7efcfdc63700 1 --2- 172.21.15.7:0/1583117529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcf8107840 0x7efcf81989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.265+0000 7efcfdc63700 1 -- 172.21.15.7:0/1583117529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcf8102740 msgr2=0x7efcf819e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.265+0000 7efcfdc63700 1 --2- 172.21.15.7:0/1583117529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcf8102740 0x7efcf819e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.265+0000 7efcfdc63700 1 -- 172.21.15.7:0/1583117529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf8199130 con 0x7efcf81030f0 2024-06-10T01:12:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.265+0000 7efcfec65700 1 --2- 172.21.15.7:0/1583117529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcf8107840 0x7efcf81989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:12:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.265+0000 7efcfe464700 1 --2- 172.21.15.7:0/1583117529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcf8102740 0x7efcf819e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.265+0000 7efcfdc63700 1 --2- 172.21.15.7:0/1583117529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcf81030f0 0x7efcf819ed50 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7efce800d980 tx=0x7efce800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.265+0000 7efcef7fe700 1 -- 172.21.15.7:0/1583117529 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce8017070 con 0x7efcf81030f0 2024-06-10T01:12:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.265+0000 7efd006c8700 1 -- 172.21.15.7:0/1583117529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcf8199420 con 0x7efcf81030f0 2024-06-10T01:12:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.266+0000 7efcef7fe700 1 -- 172.21.15.7:0/1583117529 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efce8005020 con 0x7efcf81030f0 2024-06-10T01:12:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.266+0000 7efd006c8700 1 -- 172.21.15.7:0/1583117529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcf81a56e0 con 0x7efcf81030f0 2024-06-10T01:12:02.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.266+0000 7efcef7fe700 1 -- 172.21.15.7:0/1583117529 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce8009bc0 con 0x7efcf81030f0 2024-06-10T01:12:02.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.267+0000 7efd006c8700 1 -- 172.21.15.7:0/1583117529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcf8103bd0 con 0x7efcf81030f0 2024-06-10T01:12:02.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.268+0000 7efcef7fe700 1 -- 172.21.15.7:0/1583117529 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7efce8004a30 con 0x7efcf81030f0 2024-06-10T01:12:02.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.269+0000 7efcef7fe700 1 --2- 172.21.15.7:0/1583117529 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efce40a2b00 0x7efce40a4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:02.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.269+0000 7efcfe464700 1 --2- 172.21.15.7:0/1583117529 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efce40a2b00 0x7efce40a4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:02.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.269+0000 7efcef7fe700 1 -- 172.21.15.7:0/1583117529 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7efce80cf450 con 0x7efcf81030f0 2024-06-10T01:12:02.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.269+0000 7efcfe464700 1 --2- 172.21.15.7:0/1583117529 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efce40a2b00 0x7efce40a4fc0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7efcf8199b10 tx=0x7efcf4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.273+0000 7efcef7fe700 1 -- 172.21.15.7:0/1583117529 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efce8020030 con 0x7efcf81030f0 2024-06-10T01:12:02.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.467+0000 7f8a58970700 1 -- 172.21.15.7:0/3937938150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a541084e0 msgr2=0x7f8a541088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:02.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.467+0000 7f8a58970700 1 --2- 172.21.15.7:0/3937938150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a541084e0 0x7f8a541088c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8a440098f0 tx=0x7f8a44009c00 comp rx=0 tx=0).stop 2024-06-10T01:12:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.467+0000 7f8a58970700 1 -- 172.21.15.7:0/3937938150 shutdown_connections 2024-06-10T01:12:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.467+0000 7f8a58970700 1 --2- 172.21.15.7:0/3937938150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a5406ab10 0x7f8a5410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.467+0000 7f8a58970700 1 --2- 172.21.15.7:0/3937938150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a541084e0 0x7f8a541088c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.467+0000 7f8a58970700 1 --2- 172.21.15.7:0/3937938150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a5406a170 0x7f8a5406a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.467+0000 7f8a58970700 1 -- 172.21.15.7:0/3937938150 >> 172.21.15.7:0/3937938150 conn(0x7f8a540782c0 msgr2=0x7f8a540786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.467+0000 7f8a58970700 1 -- 172.21.15.7:0/3937938150 shutdown_connections 2024-06-10T01:12:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.467+0000 7f8a58970700 1 -- 172.21.15.7:0/3937938150 wait complete. 2024-06-10T01:12:02.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.468+0000 7f8a58970700 1 Processor -- start 2024-06-10T01:12:02.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.468+0000 7f8a58970700 1 -- start start 2024-06-10T01:12:02.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.468+0000 7f8a58970700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a5406a170 0x7f8a54101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:02.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.468+0000 7f8a58970700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a5406ab10 0x7f8a541019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:02.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.468+0000 7f8a58970700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a541084e0 0x7f8a54101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:02.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.468+0000 7f8a58970700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8a54112740 con 0x7f8a5406ab10 2024-06-10T01:12:02.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.468+0000 7f8a58970700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541125c0 con 0x7f8a541084e0 2024-06-10T01:12:02.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.468+0000 7f8a58970700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541128c0 con 0x7f8a5406a170 2024-06-10T01:12:02.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.468+0000 7f8a51d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a5406ab10 0x7f8a541019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:02.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.468+0000 7f8a51d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a5406ab10 0x7f8a541019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58986/0 (socket says 172.21.15.7:58986) 2024-06-10T01:12:02.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.468+0000 7f8a51d9b700 1 -- 172.21.15.7:0/1175960374 learned_addr learned my addr 172.21.15.7:0/1175960374 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:02.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.468+0000 7f8a5259c700 1 --2- 172.21.15.7:0/1175960374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a5406a170 0x7f8a54101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:02.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.469+0000 7f8a52d9d700 1 --2- 172.21.15.7:0/1175960374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a541084e0 0x7f8a54101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:02.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.469+0000 7f8a51d9b700 1 -- 172.21.15.7:0/1175960374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a5406a170 msgr2=0x7f8a54101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:02.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.469+0000 7f8a51d9b700 1 --2- 172.21.15.7:0/1175960374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a5406a170 0x7f8a54101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.469+0000 7f8a51d9b700 1 -- 172.21.15.7:0/1175960374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a541084e0 msgr2=0x7f8a54101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:02.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.469+0000 7f8a51d9b700 1 --2- 172.21.15.7:0/1175960374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a541084e0 0x7f8a54101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.469+0000 7f8a51d9b700 1 -- 172.21.15.7:0/1175960374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a541a71a0 con 0x7f8a5406ab10 2024-06-10T01:12:02.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.469+0000 7f8a52d9d700 1 --2- 172.21.15.7:0/1175960374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a541084e0 0x7f8a54101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:12:02.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.469+0000 7f8a51d9b700 1 --2- 172.21.15.7:0/1175960374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a5406ab10 0x7f8a541019a0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800d980 tx=0x7f8a4800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:02.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.469+0000 7f8a437fe700 1 -- 172.21.15.7:0/1175960374 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a48017070 con 0x7f8a5406ab10 2024-06-10T01:12:02.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.469+0000 7f8a58970700 1 -- 172.21.15.7:0/1175960374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a541a7490 con 0x7f8a5406ab10 2024-06-10T01:12:02.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.469+0000 7f8a437fe700 1 -- 172.21.15.7:0/1175960374 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a48005020 con 0x7f8a5406ab10 2024-06-10T01:12:02.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.469+0000 7f8a437fe700 1 -- 172.21.15.7:0/1175960374 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a48009bc0 con 0x7f8a5406ab10 2024-06-10T01:12:02.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.469+0000 7f8a58970700 1 -- 172.21.15.7:0/1175960374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a541a7980 con 0x7f8a5406ab10 2024-06-10T01:12:02.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.472+0000 7f8a437fe700 1 -- 172.21.15.7:0/1175960374 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f8a480051d0 con 0x7f8a5406ab10 2024-06-10T01:12:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.473+0000 7f8a437fe700 1 --2- 172.21.15.7:0/1175960374 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8a3c0a2c20 0x7f8a3c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.473+0000 7f8a437fe700 1 -- 172.21.15.7:0/1175960374 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f8a480cf350 con 0x7f8a5406ab10 2024-06-10T01:12:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.473+0000 7f8a5259c700 1 --2- 172.21.15.7:0/1175960374 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8a3c0a2c20 0x7f8a3c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.474+0000 7f8a631f3b80 1 Processor -- start 2024-06-10T01:12:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.474+0000 7f8a5259c700 1 --2- 172.21.15.7:0/1175960374 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8a3c0a2c20 0x7f8a3c0a50e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8a440098c0 tx=0x7f8a4401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.474+0000 7f8a631f3b80 1 -- start start 2024-06-10T01:12:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.474+0000 7f8a631f3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563565a29640 0x563565a29a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.474+0000 7f8a631f3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563565a29f60 0x563565a239e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.474+0000 7f8a631f3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563565a23f20 0x563565a24340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.474+0000 7f8a631f3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563565975810 con 0x563565a29640 2024-06-10T01:12:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.474+0000 7f8a631f3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56356591d720 con 0x563565a23f20 2024-06-10T01:12:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.474+0000 7f8a631f3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5635659717d0 con 0x563565a29f60 2024-06-10T01:12:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.474+0000 7f8a5259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563565a29640 0x563565a29a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.474+0000 7f8a5259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563565a29640 0x563565a29a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58992/0 (socket says 172.21.15.7:58992) 2024-06-10T01:12:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.474+0000 7f8a5259c700 1 -- 172.21.15.7:0/1525183091 learned_addr learned my addr 172.21.15.7:0/1525183091 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.475+0000 7f8a51d9b700 1 --2- 172.21.15.7:0/1525183091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563565a29f60 0x563565a239e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:02.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.475+0000 7f8a52d9d700 1 --2- 172.21.15.7:0/1525183091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563565a23f20 0x563565a24340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:02.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.475+0000 7f8a5259c700 1 -- 172.21.15.7:0/1525183091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563565a29f60 msgr2=0x563565a239e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:02.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.475+0000 7f8a5259c700 1 --2- 172.21.15.7:0/1525183091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563565a29f60 0x563565a239e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.475+0000 7f8a5259c700 1 -- 172.21.15.7:0/1525183091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563565a23f20 msgr2=0x563565a24340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:02.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.475+0000 7f8a5259c700 1 --2- 172.21.15.7:0/1525183091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563565a23f20 0x563565a24340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.475+0000 7f8a5259c700 1 -- 172.21.15.7:0/1525183091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563565913f50 con 0x563565a29640 2024-06-10T01:12:02.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.475+0000 7f8a5259c700 1 --2- 172.21.15.7:0/1525183091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563565a29640 0x563565a29a20 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f8a44016000 tx=0x7f8a4401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e45c57e3477970 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:02.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.475+0000 7f8a40ff9700 1 -- 172.21.15.7:0/1525183091 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a440052c0 con 0x563565a29640 2024-06-10T01:12:02.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.475+0000 7f8a40ff9700 1 -- 172.21.15.7:0/1525183091 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a44005460 con 0x563565a29640 2024-06-10T01:12:02.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.475+0000 7f8a40ff9700 1 -- 172.21.15.7:0/1525183091 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4400b9d0 con 0x563565a29640 2024-06-10T01:12:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.476+0000 7f8a631f3b80 1 -- 172.21.15.7:0/1525183091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563565a29640 msgr2=0x563565a29a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.476+0000 7f8a631f3b80 1 --2- 172.21.15.7:0/1525183091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563565a29640 0x563565a29a20 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f8a44016000 tx=0x7f8a4401e000 comp rx=0 tx=0).stop 2024-06-10T01:12:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.476+0000 7f8a631f3b80 1 -- 172.21.15.7:0/1525183091 shutdown_connections 2024-06-10T01:12:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.476+0000 7f8a631f3b80 1 --2- 172.21.15.7:0/1525183091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563565a23f20 0x563565a24340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.476+0000 7f8a631f3b80 1 --2- 172.21.15.7:0/1525183091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563565a29640 0x563565a29a20 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.476+0000 7f8a631f3b80 1 --2- 172.21.15.7:0/1525183091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563565a29f60 0x563565a239e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.476+0000 7f8a631f3b80 1 -- 172.21.15.7:0/1525183091 >> 172.21.15.7:0/1525183091 conn(0x56356594ecc0 msgr2=0x563565a1e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.476+0000 7f8a631f3b80 1 -- 172.21.15.7:0/1525183091 shutdown_connections 2024-06-10T01:12:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.476+0000 7f8a631f3b80 1 -- 172.21.15.7:0/1525183091 wait complete. 2024-06-10T01:12:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.477+0000 7f8a631f3b80 1 -- start start 2024-06-10T01:12:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.477+0000 7f8a631f3b80 1 Processor -- start 2024-06-10T01:12:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.477+0000 7f8a631f3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563565a23f20 0x563565a2fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.477+0000 7f8a631f3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563565a29640 0x563565a2cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.477+0000 7f8a631f3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563565a29f60 0x563565a2d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.477+0000 7f8a631f3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563565a30970 con 0x563565a29f60 2024-06-10T01:12:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.477+0000 7f8a631f3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563565915bf0 con 0x563565a29640 2024-06-10T01:12:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.477+0000 7f8a631f3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563565a30af0 con 0x563565a23f20 2024-06-10T01:12:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.477+0000 7f8a631f3b80 10 client.0 _ll_register_callbacks cb 0x56356590f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:12:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.477+0000 7f8a52d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563565a29f60 0x563565a2d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.477+0000 7f8a52d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563565a29f60 0x563565a2d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58994/0 (socket says 172.21.15.7:58994) 2024-06-10T01:12:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.477+0000 7f8a52d9d700 1 -- 172.21.15.7:0/733411582 learned_addr learned my addr 172.21.15.7:0/733411582 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.477+0000 7f8a5259c700 1 --2- 172.21.15.7:0/733411582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563565a23f20 0x563565a2fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.477+0000 7f8a51d9b700 1 --2- 172.21.15.7:0/733411582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563565a29640 0x563565a2cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.477+0000 7f8a52d9d700 1 -- 172.21.15.7:0/733411582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563565a23f20 msgr2=0x563565a2fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.477+0000 7f8a52d9d700 1 --2- 172.21.15.7:0/733411582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563565a23f20 0x563565a2fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.477+0000 7f8a52d9d700 1 -- 172.21.15.7:0/733411582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563565a29640 msgr2=0x563565a2cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.477+0000 7f8a52d9d700 1 --2- 172.21.15.7:0/733411582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563565a29640 0x563565a2cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.477+0000 7f8a52d9d700 1 -- 172.21.15.7:0/733411582 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635658c48d0 con 0x563565a29f60 2024-06-10T01:12:02.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.478+0000 7f8a51d9b700 1 --2- 172.21.15.7:0/733411582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563565a29640 0x563565a2cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:12:02.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.478+0000 7f8a52d9d700 1 --2- 172.21.15.7:0/733411582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563565a29f60 0x563565a2d510 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c007c20 tx=0x7f8a4c007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4098c6eedc6951 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:02.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.478+0000 7f8a39ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:12:02.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.478+0000 7f8a39ffb700 1 -- 172.21.15.7:0/733411582 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c01a070 con 0x563565a29f60 2024-06-10T01:12:02.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.478+0000 7f8a39ffb700 1 -- 172.21.15.7:0/733411582 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f8a4c004db0 con 0x563565a29f60 2024-06-10T01:12:02.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.478+0000 7f8a631f3b80 1 -- 172.21.15.7:0/733411582 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a44005460 con 0x563565a29f60 2024-06-10T01:12:02.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.478+0000 7f8a39ffb700 1 -- 172.21.15.7:0/733411582 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8a4c025050 con 0x563565a29f60 2024-06-10T01:12:02.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.478+0000 7f8a631f3b80 10 client.17019 fetch_fsmap learned FSMap version 32 2024-06-10T01:12:02.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.478+0000 7f8a631f3b80 1 -- 172.21.15.7:0/733411582 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563565a3b320 con 0x563565a29f60 2024-06-10T01:12:02.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.478+0000 7f8a39ffb700 1 -- 172.21.15.7:0/733411582 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a44005460 con 0x563565a29f60 2024-06-10T01:12:02.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.479+0000 7f8a39ffb700 1 -- 172.21.15.7:0/733411582 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f8a4c005830 con 0x563565a29f60 2024-06-10T01:12:02.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.479+0000 7f8a631f3b80 10 client.17019 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:12:02.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.479+0000 7f8a631f3b80 10 client.17019 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:12:02.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.479+0000 7f8a631f3b80 20 client.17019 populate_metadata read hostname 'smithi007' 2024-06-10T01:12:02.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.479+0000 7f8a631f3b80 1 --2- 172.21.15.7:0/733411582 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x563565a3d8b0 0x563565a3fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:02.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.479+0000 7f8a631f3b80 4 client.17019 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:12:02.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.479+0000 7f8a631f3b80 1 -- 172.21.15.7:0/733411582 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563565982d10 con 0x563565a3d8b0 2024-06-10T01:12:02.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.479+0000 7f8a5259c700 1 --2- 172.21.15.7:0/733411582 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x563565a3d8b0 0x563565a3fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:02.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.480+0000 7f8a5259c700 1 --2- 172.21.15.7:0/733411582 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x563565a3d8b0 0x563565a3fcd0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=81e2ccf81a747876 server_cookie=caa4777abcb19942 in_seq=0 out_seq=0 2024-06-10T01:12:02.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.480+0000 7f8a39ffb700 10 client.17019 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:12:02.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.480+0000 7f8a39ffb700 1 -- 172.21.15.7:0/733411582 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563565982d10 con 0x563565a3d8b0 2024-06-10T01:12:02.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.480+0000 7f8a39ffb700 10 client.17019 handle_command_reply: tid=0 2024-06-10T01:12:02.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.480+0000 7f8a631f3b80 1 client.17019 shutdown 2024-06-10T01:12:02.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.480+0000 7f8a631f3b80 1 -- 172.21.15.7:0/733411582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563565a29f60 msgr2=0x563565a2d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:02.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.480+0000 7f8a631f3b80 1 --2- 172.21.15.7:0/733411582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563565a29f60 0x563565a2d510 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c007c20 tx=0x7f8a4c007f30 comp rx=0 tx=0).stop 2024-06-10T01:12:02.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.480+0000 7f8a631f3b80 1 -- 172.21.15.7:0/733411582 shutdown_connections 2024-06-10T01:12:02.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.480+0000 7f8a631f3b80 1 --2- 172.21.15.7:0/733411582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563565a29f60 0x563565a2d510 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.480+0000 7f8a631f3b80 1 --2- 172.21.15.7:0/733411582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563565a23f20 0x563565a2fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.480+0000 7f8a631f3b80 1 --2- 172.21.15.7:0/733411582 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x563565a3d8b0 0x563565a3fcd0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.481+0000 7f8a631f3b80 1 --2- 172.21.15.7:0/733411582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563565a29640 0x563565a2cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.481+0000 7f8a39ffb700 0 client.17019 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:12:02.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.481+0000 7f8a631f3b80 1 -- 172.21.15.7:0/733411582 >> 172.21.15.7:0/733411582 conn(0x56356594ecc0 msgr2=0x563565a1e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:02.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.481+0000 7f8a631f3b80 1 -- 172.21.15.7:0/733411582 shutdown_connections 2024-06-10T01:12:02.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.481+0000 7f8a631f3b80 1 -- 172.21.15.7:0/733411582 wait complete. 2024-06-10T01:12:02.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.481+0000 7f8a631f3b80 20 client.17019 trim_cache size 0 max 16384 2024-06-10T01:12:02.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.496+0000 7f8a631f3b80 1 Processor -- start 2024-06-10T01:12:02.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.496+0000 7f8a631f3b80 1 -- start start 2024-06-10T01:12:02.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.496+0000 7f8a631f3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563565a23f20 0x563565a239e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:02.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.496+0000 7f8a631f3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563565a29640 0x563565a2b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:02.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.496+0000 7f8a631f3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563565a29f60 0x563565a2be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:02.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.496+0000 7f8a631f3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541b2c90 con 0x563565a29f60 2024-06-10T01:12:02.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.496+0000 7f8a631f3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541b0520 con 0x563565a23f20 2024-06-10T01:12:02.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.496+0000 7f8a631f3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541ae370 con 0x563565a29640 2024-06-10T01:12:02.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.496+0000 7f8a52d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563565a29f60 0x563565a2be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:02.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.496+0000 7f8a52d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563565a29f60 0x563565a2be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58998/0 (socket says 172.21.15.7:58998) 2024-06-10T01:12:02.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.496+0000 7f8a52d9d700 1 -- 172.21.15.7:0/2593018623 learned_addr learned my addr 172.21.15.7:0/2593018623 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:02.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.496+0000 7f8a51d9b700 1 --2- 172.21.15.7:0/2593018623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563565a29640 0x563565a2b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:02.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.497+0000 7f8a5259c700 1 --2- 172.21.15.7:0/2593018623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563565a23f20 0x563565a239e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:02.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.497+0000 7f8a52d9d700 1 -- 172.21.15.7:0/2593018623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563565a29640 msgr2=0x563565a2b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:02.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.497+0000 7f8a52d9d700 1 --2- 172.21.15.7:0/2593018623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563565a29640 0x563565a2b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.497+0000 7f8a52d9d700 1 -- 172.21.15.7:0/2593018623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563565a23f20 msgr2=0x563565a239e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:02.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.497+0000 7f8a52d9d700 1 --2- 172.21.15.7:0/2593018623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563565a23f20 0x563565a239e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.497+0000 7f8a52d9d700 1 -- 172.21.15.7:0/2593018623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563565a2b560 con 0x563565a29f60 2024-06-10T01:12:02.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.497+0000 7f8a51d9b700 1 --2- 172.21.15.7:0/2593018623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563565a29640 0x563565a2b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:12:02.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.497+0000 7f8a5259c700 1 --2- 172.21.15.7:0/2593018623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563565a23f20 0x563565a239e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:12:02.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.497+0000 7f8a52d9d700 1 --2- 172.21.15.7:0/2593018623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563565a29f60 0x563565a2be20 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c005e10 tx=0x7f8a4c005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95140ff05c20194 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:02.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.497+0000 7f8a3bfff700 1 -- 172.21.15.7:0/2593018623 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c01a040 con 0x563565a29f60 2024-06-10T01:12:02.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.497+0000 7f8a3bfff700 1 -- 172.21.15.7:0/2593018623 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a4c022070 con 0x563565a29f60 2024-06-10T01:12:02.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.497+0000 7f8a3bfff700 1 -- 172.21.15.7:0/2593018623 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c012d30 con 0x563565a29f60 2024-06-10T01:12:02.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.498+0000 7f8a631f3b80 1 -- 172.21.15.7:0/2593018623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563565a29f60 msgr2=0x563565a2be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:02.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.498+0000 7f8a631f3b80 1 --2- 172.21.15.7:0/2593018623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563565a29f60 0x563565a2be20 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c005e10 tx=0x7f8a4c005e40 comp rx=0 tx=0).stop 2024-06-10T01:12:02.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.498+0000 7f8a631f3b80 1 -- 172.21.15.7:0/2593018623 shutdown_connections 2024-06-10T01:12:02.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.498+0000 7f8a631f3b80 1 --2- 172.21.15.7:0/2593018623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563565a29f60 0x563565a2be20 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.498+0000 7f8a631f3b80 1 --2- 172.21.15.7:0/2593018623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563565a23f20 0x563565a239e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.498+0000 7f8a631f3b80 1 --2- 172.21.15.7:0/2593018623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563565a29640 0x563565a2b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.498+0000 7f8a631f3b80 1 -- 172.21.15.7:0/2593018623 >> 172.21.15.7:0/2593018623 conn(0x56356594ecc0 msgr2=0x563565a2cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:02.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.498+0000 7f8a631f3b80 1 -- 172.21.15.7:0/2593018623 shutdown_connections 2024-06-10T01:12:02.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.498+0000 7f8a631f3b80 1 -- 172.21.15.7:0/2593018623 wait complete. 2024-06-10T01:12:02.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.498+0000 7f8a631f3b80 1 -- start start 2024-06-10T01:12:02.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.498+0000 7f8a631f3b80 1 Processor -- start 2024-06-10T01:12:02.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.498+0000 7f8a631f3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563565a23f20 0x563565a36400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:02.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.498+0000 7f8a631f3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563565a29640 0x563565a44e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:02.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.498+0000 7f8a631f3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563565a29f60 0x563565a45390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:02.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.499+0000 7f8a631f3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541ae9e0 con 0x563565a29f60 2024-06-10T01:12:02.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.499+0000 7f8a631f3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541afe50 con 0x563565a29640 2024-06-10T01:12:02.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.499+0000 7f8a631f3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541af0b0 con 0x563565a23f20 2024-06-10T01:12:02.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.499+0000 7f8a52d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563565a29f60 0x563565a45390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:02.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.499+0000 7f8a631f3b80 10 client.0 _ll_register_callbacks cb 0x56356590f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:12:02.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.499+0000 7f8a5259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563565a23f20 0x563565a36400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:02.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.499+0000 7f8a5259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563565a23f20 0x563565a36400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:56034/0 (socket says 172.21.15.7:56034) 2024-06-10T01:12:02.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.499+0000 7f8a52d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563565a29f60 0x563565a45390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59012/0 (socket says 172.21.15.7:59012) 2024-06-10T01:12:02.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.499+0000 7f8a5259c700 1 -- 172.21.15.7:0/1718844080 learned_addr learned my addr 172.21.15.7:0/1718844080 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:02.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.499+0000 7f8a51d9b700 1 --2- 172.21.15.7:0/1718844080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563565a29640 0x563565a44e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:02.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.499+0000 7f8a52d9d700 1 -- 172.21.15.7:0/1718844080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563565a23f20 msgr2=0x563565a36400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:02.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.499+0000 7f8a52d9d700 1 --2- 172.21.15.7:0/1718844080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563565a23f20 0x563565a36400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.499+0000 7f8a52d9d700 1 -- 172.21.15.7:0/1718844080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563565a29640 msgr2=0x563565a44e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:02.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.499+0000 7f8a52d9d700 1 --2- 172.21.15.7:0/1718844080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563565a29640 0x563565a44e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.499+0000 7f8a52d9d700 1 -- 172.21.15.7:0/1718844080 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563565a45ad0 con 0x563565a29f60 2024-06-10T01:12:02.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.499+0000 7f8a5259c700 1 --2- 172.21.15.7:0/1718844080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563565a23f20 0x563565a36400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:02.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.499+0000 7f8a51d9b700 1 --2- 172.21.15.7:0/1718844080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563565a29640 0x563565a44e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:02.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.499+0000 7f8a52d9d700 1 --2- 172.21.15.7:0/1718844080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563565a29f60 0x563565a45390 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c0095b0 tx=0x7f8a4c012640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7fc2fbf3b589c5d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:02.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.499+0000 7f8a39ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:12:02.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.499+0000 7f8a39ffb700 1 -- 172.21.15.7:0/1718844080 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c036060 con 0x563565a29f60 2024-06-10T01:12:02.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.499+0000 7f8a631f3b80 1 -- 172.21.15.7:0/1718844080 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a4c022070 con 0x563565a29f60 2024-06-10T01:12:02.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.499+0000 7f8a39ffb700 1 -- 172.21.15.7:0/1718844080 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f8a4c004ca0 con 0x563565a29f60 2024-06-10T01:12:02.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.500+0000 7f8a39ffb700 1 -- 172.21.15.7:0/1718844080 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8a4c03d050 con 0x563565a29f60 2024-06-10T01:12:02.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.500+0000 7f8a631f3b80 10 client.17025 fetch_fsmap learned FSMap version 32 2024-06-10T01:12:02.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.500+0000 7f8a631f3b80 1 -- 172.21.15.7:0/1718844080 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563565aa8410 con 0x563565a29f60 2024-06-10T01:12:02.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.500+0000 7f8a39ffb700 1 -- 172.21.15.7:0/1718844080 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a4c022070 con 0x563565a29f60 2024-06-10T01:12:02.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.500+0000 7f8a39ffb700 1 -- 172.21.15.7:0/1718844080 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f8a4c016b80 con 0x563565a29f60 2024-06-10T01:12:02.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.500+0000 7f8a631f3b80 10 client.17025 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:12:02.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.500+0000 7f8a631f3b80 10 client.17025 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:12:02.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.500+0000 7f8a631f3b80 20 client.17025 populate_metadata read hostname 'smithi007' 2024-06-10T01:12:02.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.500+0000 7f8a631f3b80 1 --2- 172.21.15.7:0/1718844080 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x563565aa8a50 0x563565aaae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:02.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.500+0000 7f8a631f3b80 4 client.17025 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:12:02.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.500+0000 7f8a631f3b80 1 -- 172.21.15.7:0/1718844080 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56356597d290 con 0x563565aa8a50 2024-06-10T01:12:02.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.500+0000 7f8a5259c700 1 --2- 172.21.15.7:0/1718844080 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x563565aa8a50 0x563565aaae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:02.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.501+0000 7f8a5259c700 1 --2- 172.21.15.7:0/1718844080 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x563565aa8a50 0x563565aaae40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8813e0f181dcf16f server_cookie=e24e682f09fde4db in_seq=0 out_seq=0 2024-06-10T01:12:02.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.501+0000 7f8a39ffb700 10 client.17025 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:12:02.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.501+0000 7f8a39ffb700 1 -- 172.21.15.7:0/1718844080 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56356597d290 con 0x563565aa8a50 2024-06-10T01:12:02.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.501+0000 7f8a39ffb700 10 client.17025 handle_command_reply: tid=0 2024-06-10T01:12:02.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.501+0000 7f8a631f3b80 1 client.17025 shutdown 2024-06-10T01:12:02.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.501+0000 7f8a631f3b80 1 -- 172.21.15.7:0/1718844080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563565a29f60 msgr2=0x563565a45390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:02.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.501+0000 7f8a631f3b80 1 --2- 172.21.15.7:0/1718844080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563565a29f60 0x563565a45390 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c0095b0 tx=0x7f8a4c012640 comp rx=0 tx=0).stop 2024-06-10T01:12:02.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.501+0000 7f8a631f3b80 1 -- 172.21.15.7:0/1718844080 shutdown_connections 2024-06-10T01:12:02.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.501+0000 7f8a631f3b80 1 --2- 172.21.15.7:0/1718844080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563565a29f60 0x563565a45390 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.501+0000 7f8a631f3b80 1 --2- 172.21.15.7:0/1718844080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563565a23f20 0x563565a36400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.501+0000 7f8a631f3b80 1 --2- 172.21.15.7:0/1718844080 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x563565aa8a50 0x563565aaae40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.501+0000 7f8a631f3b80 1 --2- 172.21.15.7:0/1718844080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563565a29640 0x563565a44e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.501+0000 7f8a631f3b80 1 -- 172.21.15.7:0/1718844080 >> 172.21.15.7:0/1718844080 conn(0x56356594ecc0 msgr2=0x563565a2b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:02.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.501+0000 7f8a39ffb700 0 client.17025 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:12:02.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.501+0000 7f8a631f3b80 1 -- 172.21.15.7:0/1718844080 shutdown_connections 2024-06-10T01:12:02.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.501+0000 7f8a631f3b80 1 -- 172.21.15.7:0/1718844080 wait complete. 2024-06-10T01:12:02.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.502+0000 7f8a631f3b80 20 client.17025 trim_cache size 0 max 16384 2024-06-10T01:12:02.503 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:12:02.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.502+0000 7f8a58970700 1 -- 172.21.15.7:0/1175960374 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8a3c0a2c20 msgr2=0x7f8a3c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:02.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.502+0000 7f8a58970700 1 --2- 172.21.15.7:0/1175960374 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8a3c0a2c20 0x7f8a3c0a50e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8a440098c0 tx=0x7f8a4401a040 comp rx=0 tx=0).stop 2024-06-10T01:12:02.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.502+0000 7f8a58970700 1 -- 172.21.15.7:0/1175960374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a5406ab10 msgr2=0x7f8a541019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:02.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.502+0000 7f8a58970700 1 --2- 172.21.15.7:0/1175960374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a5406ab10 0x7f8a541019a0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800d980 tx=0x7f8a4800dc90 comp rx=0 tx=0).stop 2024-06-10T01:12:02.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.502+0000 7f8a58970700 1 -- 172.21.15.7:0/1175960374 shutdown_connections 2024-06-10T01:12:02.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.502+0000 7f8a58970700 1 --2- 172.21.15.7:0/1175960374 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8a3c0a2c20 0x7f8a3c0a50e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.502+0000 7f8a58970700 1 --2- 172.21.15.7:0/1175960374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a541084e0 0x7f8a54101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.502+0000 7f8a58970700 1 --2- 172.21.15.7:0/1175960374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a5406a170 0x7f8a54101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.502+0000 7f8a58970700 1 --2- 172.21.15.7:0/1175960374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a5406ab10 0x7f8a541019a0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.502+0000 7f8a58970700 1 -- 172.21.15.7:0/1175960374 >> 172.21.15.7:0/1175960374 conn(0x7f8a540782c0 msgr2=0x7f8a540ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:02.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.502+0000 7f8a58970700 1 -- 172.21.15.7:0/1175960374 shutdown_connections 2024-06-10T01:12:02.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.502+0000 7f8a58970700 1 -- 172.21.15.7:0/1175960374 wait complete. 2024-06-10T01:12:02.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.537+0000 7efd006c8700 1 -- 172.21.15.7:0/1583117529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efcf8199f00 con 0x7efcf81030f0 2024-06-10T01:12:02.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.538+0000 7efcef7fe700 1 -- 172.21.15.7:0/1583117529 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11855 (secure 0 0 0) 0x7efce809c0b0 con 0x7efcf81030f0 2024-06-10T01:12:02.533 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:12:02.533 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:10:42.213996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:12:02.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.540+0000 7efd006c8700 1 -- 172.21.15.7:0/1583117529 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efce40a2b00 msgr2=0x7efce40a4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:02.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.540+0000 7efd006c8700 1 --2- 172.21.15.7:0/1583117529 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efce40a2b00 0x7efce40a4fc0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7efcf8199b10 tx=0x7efcf4009260 comp rx=0 tx=0).stop 2024-06-10T01:12:02.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.540+0000 7efd006c8700 1 -- 172.21.15.7:0/1583117529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcf81030f0 msgr2=0x7efcf819ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:02.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.540+0000 7efd006c8700 1 --2- 172.21.15.7:0/1583117529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcf81030f0 0x7efcf819ed50 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7efce800d980 tx=0x7efce800dc90 comp rx=0 tx=0).stop 2024-06-10T01:12:02.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.540+0000 7efd006c8700 1 -- 172.21.15.7:0/1583117529 shutdown_connections 2024-06-10T01:12:02.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.540+0000 7efd006c8700 1 --2- 172.21.15.7:0/1583117529 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efce40a2b00 0x7efce40a4fc0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.540+0000 7efd006c8700 1 --2- 172.21.15.7:0/1583117529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcf8107840 0x7efcf81989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.540+0000 7efd006c8700 1 --2- 172.21.15.7:0/1583117529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcf8102740 0x7efcf819e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.540+0000 7efd006c8700 1 --2- 172.21.15.7:0/1583117529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcf81030f0 0x7efcf819ed50 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:02.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.540+0000 7efd006c8700 1 -- 172.21.15.7:0/1583117529 >> 172.21.15.7:0/1583117529 conn(0x7efcf80795f0 msgr2=0x7efcf8100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:02.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.540+0000 7efd006c8700 1 -- 172.21.15.7:0/1583117529 shutdown_connections 2024-06-10T01:12:02.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:02.540+0000 7efd006c8700 1 -- 172.21.15.7:0/1583117529 wait complete. 2024-06-10T01:12:02.536 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 32 2024-06-10T01:12:02.734 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 6.06 MB/sec execute 163 sec latency 28887.650 ms 2024-06-10T01:12:03.013 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:02 smithi007 ceph-mon[32398]: pgmap v613: 129 pgs: 129 active+clean; 67 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 997 KiB/s wr, 43 op/s 2024-06-10T01:12:03.013 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:02 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1583117529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:12:03.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:02 smithi078 ceph-mon[36683]: pgmap v613: 129 pgs: 129 active+clean; 67 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 997 KiB/s wr, 43 op/s 2024-06-10T01:12:03.080 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:02 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1583117529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:12:03.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:02 smithi175 ceph-mon[36636]: pgmap v613: 129 pgs: 129 active+clean; 67 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 997 KiB/s wr, 43 op/s 2024-06-10T01:12:03.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:02 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1583117529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:12:03.571 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:12:03.733 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 6.03 MB/sec execute 164 sec latency 29887.675 ms 2024-06-10T01:12:04.113 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db --set-id 55ac2137 --release --await' 2024-06-10T01:12:04.113 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db --set-id 55ac2137 --release --await 2024-06-10T01:12:04.167 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:12:04.733 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 5.99 MB/sec execute 165 sec latency 30887.711 ms 2024-06-10T01:12:05.013 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:04 smithi007 ceph-mon[32398]: pgmap v614: 129 pgs: 129 active+clean; 66 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 1003 KiB/s wr, 45 op/s 2024-06-10T01:12:05.082 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:04 smithi078 ceph-mon[36683]: pgmap v614: 129 pgs: 129 active+clean; 66 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 1003 KiB/s wr, 45 op/s 2024-06-10T01:12:05.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:04 smithi175 ceph-mon[36636]: pgmap v614: 129 pgs: 129 active+clean; 66 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 1003 KiB/s wr, 45 op/s 2024-06-10T01:12:05.437 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:12:05.733 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 5.95 MB/sec execute 166 sec latency 31887.750 ms 2024-06-10T01:12:05.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.930+0000 7f531b8b1700 1 -- 172.21.15.7:0/3800620112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5314108510 msgr2=0x7f53141088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:05.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.930+0000 7f531b8b1700 1 --2- 172.21.15.7:0/3800620112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5314108510 0x7f53141088f0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f530400b0a0 tx=0x7f530400b3b0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.930+0000 7f531b8b1700 1 -- 172.21.15.7:0/3800620112 shutdown_connections 2024-06-10T01:12:05.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.930+0000 7f531b8b1700 1 --2- 172.21.15.7:0/3800620112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5314102ed0 0x7f531410f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.930+0000 7f531b8b1700 1 --2- 172.21.15.7:0/3800620112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5314108510 0x7f53141088f0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.930+0000 7f531b8b1700 1 --2- 172.21.15.7:0/3800620112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5314102510 0x7f5314102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.930+0000 7f531b8b1700 1 -- 172.21.15.7:0/3800620112 >> 172.21.15.7:0/3800620112 conn(0x7f53140795b0 msgr2=0x7f53140799c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:05.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.930+0000 7f531b8b1700 1 -- 172.21.15.7:0/3800620112 shutdown_connections 2024-06-10T01:12:05.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.930+0000 7f531b8b1700 1 -- 172.21.15.7:0/3800620112 wait complete. 2024-06-10T01:12:05.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.931+0000 7f531b8b1700 1 Processor -- start 2024-06-10T01:12:05.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.931+0000 7f531b8b1700 1 -- start start 2024-06-10T01:12:05.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.931+0000 7f531b8b1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5314102510 0x7f531419e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:05.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.931+0000 7f531b8b1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5314102ed0 0x7f531419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:05.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.931+0000 7f531b8b1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5314108510 0x7f5314198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:05.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.931+0000 7f531b8b1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5314112760 con 0x7f5314102510 2024-06-10T01:12:05.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.931+0000 7f531b8b1700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f53141125e0 con 0x7f5314108510 2024-06-10T01:12:05.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.931+0000 7f531b8b1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f53141128e0 con 0x7f5314102ed0 2024-06-10T01:12:05.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.931+0000 7f531964d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5314102510 0x7f531419e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:05.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.931+0000 7f531964d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5314102510 0x7f531419e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59024/0 (socket says 172.21.15.7:59024) 2024-06-10T01:12:05.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.931+0000 7f531964d700 1 -- 172.21.15.7:0/1142161981 learned_addr learned my addr 172.21.15.7:0/1142161981 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:05.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.931+0000 7f5319e4e700 1 --2- 172.21.15.7:0/1142161981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5314108510 0x7f5314198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:05.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.931+0000 7f5318e4c700 1 --2- 172.21.15.7:0/1142161981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5314102ed0 0x7f531419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:05.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.931+0000 7f531964d700 1 -- 172.21.15.7:0/1142161981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5314102ed0 msgr2=0x7f531419ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:05.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.931+0000 7f531964d700 1 --2- 172.21.15.7:0/1142161981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5314102ed0 0x7f531419ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.931+0000 7f531964d700 1 -- 172.21.15.7:0/1142161981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5314108510 msgr2=0x7f5314198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:05.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.931+0000 7f531964d700 1 --2- 172.21.15.7:0/1142161981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5314108510 0x7f5314198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.931+0000 7f531964d700 1 -- 172.21.15.7:0/1142161981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5314199220 con 0x7f5314102510 2024-06-10T01:12:05.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.932+0000 7f5319e4e700 1 --2- 172.21.15.7:0/1142161981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5314108510 0x7f5314198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:05.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.932+0000 7f531964d700 1 --2- 172.21.15.7:0/1142161981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5314102510 0x7f531419e900 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f53040062a0 tx=0x7f5304004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:05.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.932+0000 7f53127fc700 1 -- 172.21.15.7:0/1142161981 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5304021070 con 0x7f5314102510 2024-06-10T01:12:05.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.932+0000 7f53127fc700 1 -- 172.21.15.7:0/1142161981 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5304007c00 con 0x7f5314102510 2024-06-10T01:12:05.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.932+0000 7f53127fc700 1 -- 172.21.15.7:0/1142161981 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53040052c0 con 0x7f5314102510 2024-06-10T01:12:05.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.932+0000 7f531b8b1700 1 -- 172.21.15.7:0/1142161981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53141994b0 con 0x7f5314102510 2024-06-10T01:12:05.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.932+0000 7f531b8b1700 1 -- 172.21.15.7:0/1142161981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53141a57d0 con 0x7f5314102510 2024-06-10T01:12:05.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.935+0000 7f53127fc700 1 -- 172.21.15.7:0/1142161981 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f5304024070 con 0x7f5314102510 2024-06-10T01:12:05.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.935+0000 7f53127fc700 1 --2- 172.21.15.7:0/1142161981 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f52fc0a2ca0 0x7f52fc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:05.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.935+0000 7f5318e4c700 1 --2- 172.21.15.7:0/1142161981 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f52fc0a2ca0 0x7f52fc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:05.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.935+0000 7f53127fc700 1 -- 172.21.15.7:0/1142161981 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f530400f020 con 0x7f5314102510 2024-06-10T01:12:05.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.936+0000 7f5318e4c700 1 --2- 172.21.15.7:0/1142161981 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f52fc0a2ca0 0x7f52fc0a5160 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5314079cb0 tx=0x7f5308009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:05.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.936+0000 7f5326134b80 1 Processor -- start 2024-06-10T01:12:05.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.936+0000 7f5326134b80 1 -- start start 2024-06-10T01:12:05.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.937+0000 7f5326134b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5ee4cf640 0x55d5ee4cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:05.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.937+0000 7f5326134b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ee4cff60 0x55d5ee4c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:05.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.937+0000 7f5326134b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5ee4c9f20 0x55d5ee4ca340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:05.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.937+0000 7f5326134b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d5ee41b810 con 0x55d5ee4c9f20 2024-06-10T01:12:05.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.937+0000 7f5326134b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d5ee3c3720 con 0x55d5ee4cff60 2024-06-10T01:12:05.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.937+0000 7f5326134b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d5ee4177d0 con 0x55d5ee4cf640 2024-06-10T01:12:05.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.937+0000 7f5319e4e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5ee4c9f20 0x55d5ee4ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:05.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.937+0000 7f5319e4e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5ee4c9f20 0x55d5ee4ca340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59038/0 (socket says 172.21.15.7:59038) 2024-06-10T01:12:05.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.937+0000 7f5319e4e700 1 -- 172.21.15.7:0/2389523400 learned_addr learned my addr 172.21.15.7:0/2389523400 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:05.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.937+0000 7f5318e4c700 1 --2- 172.21.15.7:0/2389523400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ee4cff60 0x55d5ee4c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:05.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.937+0000 7f531964d700 1 --2- 172.21.15.7:0/2389523400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5ee4cf640 0x55d5ee4cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:05.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.937+0000 7f5319e4e700 1 -- 172.21.15.7:0/2389523400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5ee4cf640 msgr2=0x55d5ee4cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:05.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.937+0000 7f5319e4e700 1 --2- 172.21.15.7:0/2389523400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5ee4cf640 0x55d5ee4cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.937+0000 7f5319e4e700 1 -- 172.21.15.7:0/2389523400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ee4cff60 msgr2=0x55d5ee4c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:05.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.937+0000 7f5319e4e700 1 --2- 172.21.15.7:0/2389523400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ee4cff60 0x55d5ee4c99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.937+0000 7f5319e4e700 1 -- 172.21.15.7:0/2389523400 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5ee3b9f50 con 0x55d5ee4c9f20 2024-06-10T01:12:05.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.937+0000 7f5319e4e700 1 --2- 172.21.15.7:0/2389523400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5ee4c9f20 0x55d5ee4ca340 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f530c010400 tx=0x7f530c010710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f1e140de312104 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:05.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.937+0000 7f52fb7fe700 1 -- 172.21.15.7:0/2389523400 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c011440 con 0x55d5ee4c9f20 2024-06-10T01:12:05.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.938+0000 7f52fb7fe700 1 -- 172.21.15.7:0/2389523400 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f530c011ac0 con 0x55d5ee4c9f20 2024-06-10T01:12:05.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.938+0000 7f52fb7fe700 1 -- 172.21.15.7:0/2389523400 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c019b90 con 0x55d5ee4c9f20 2024-06-10T01:12:05.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.938+0000 7f5326134b80 1 -- 172.21.15.7:0/2389523400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5ee4c9f20 msgr2=0x55d5ee4ca340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:05.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.938+0000 7f5326134b80 1 --2- 172.21.15.7:0/2389523400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5ee4c9f20 0x55d5ee4ca340 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f530c010400 tx=0x7f530c010710 comp rx=0 tx=0).stop 2024-06-10T01:12:05.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.938+0000 7f5326134b80 1 -- 172.21.15.7:0/2389523400 shutdown_connections 2024-06-10T01:12:05.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.938+0000 7f5326134b80 1 --2- 172.21.15.7:0/2389523400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5ee4c9f20 0x55d5ee4ca340 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.938+0000 7f5326134b80 1 --2- 172.21.15.7:0/2389523400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5ee4cf640 0x55d5ee4cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.938+0000 7f5326134b80 1 --2- 172.21.15.7:0/2389523400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ee4cff60 0x55d5ee4c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.938+0000 7f5326134b80 1 -- 172.21.15.7:0/2389523400 >> 172.21.15.7:0/2389523400 conn(0x55d5ee3f4cc0 msgr2=0x55d5ee4c4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:05.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.938+0000 7f5326134b80 1 -- 172.21.15.7:0/2389523400 shutdown_connections 2024-06-10T01:12:05.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.938+0000 7f5326134b80 1 -- 172.21.15.7:0/2389523400 wait complete. 2024-06-10T01:12:05.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.939+0000 7f5326134b80 1 -- start start 2024-06-10T01:12:05.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.939+0000 7f5326134b80 1 Processor -- start 2024-06-10T01:12:05.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.939+0000 7f5326134b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5ee4c9f20 0x55d5ee4d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:05.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.939+0000 7f5326134b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5ee4cf640 0x55d5ee4d2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:05.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.939+0000 7f5326134b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ee4cff60 0x55d5ee4d34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:05.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.939+0000 7f5326134b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d5ee4d69d0 con 0x55d5ee4c9f20 2024-06-10T01:12:05.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.939+0000 7f5326134b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d5ee3bbbf0 con 0x55d5ee4cff60 2024-06-10T01:12:05.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.939+0000 7f5326134b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d5ee4d6b50 con 0x55d5ee4cf640 2024-06-10T01:12:05.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.939+0000 7f5326134b80 10 client.0 _ll_register_callbacks cb 0x55d5ee3b5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:12:05.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.939+0000 7f531964d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5ee4c9f20 0x55d5ee4d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:05.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.940+0000 7f531964d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5ee4c9f20 0x55d5ee4d5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59046/0 (socket says 172.21.15.7:59046) 2024-06-10T01:12:05.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.940+0000 7f531964d700 1 -- 172.21.15.7:0/758341710 learned_addr learned my addr 172.21.15.7:0/758341710 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:05.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.940+0000 7f5319e4e700 1 --2- 172.21.15.7:0/758341710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ee4cff60 0x55d5ee4d34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:05.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.940+0000 7f5318e4c700 1 --2- 172.21.15.7:0/758341710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5ee4cf640 0x55d5ee4d2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:05.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.940+0000 7f531964d700 1 -- 172.21.15.7:0/758341710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5ee4cf640 msgr2=0x55d5ee4d2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:05.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.940+0000 7f531964d700 1 --2- 172.21.15.7:0/758341710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5ee4cf640 0x55d5ee4d2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.940+0000 7f531964d700 1 -- 172.21.15.7:0/758341710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ee4cff60 msgr2=0x55d5ee4d34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:05.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.940+0000 7f531964d700 1 --2- 172.21.15.7:0/758341710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ee4cff60 0x55d5ee4d34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.940+0000 7f531964d700 1 -- 172.21.15.7:0/758341710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5ee36a8d0 con 0x55d5ee4c9f20 2024-06-10T01:12:05.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.940+0000 7f5319e4e700 1 --2- 172.21.15.7:0/758341710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ee4cff60 0x55d5ee4d34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:05.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.940+0000 7f531964d700 1 --2- 172.21.15.7:0/758341710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5ee4c9f20 0x55d5ee4d5cb0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f5304004fd0 tx=0x7f530409d600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d5683c5a0717d84 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:05.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.940+0000 7f52f8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:12:05.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.940+0000 7f52f8ff9700 1 -- 172.21.15.7:0/758341710 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53040a1230 con 0x55d5ee4c9f20 2024-06-10T01:12:05.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.940+0000 7f52f8ff9700 1 -- 172.21.15.7:0/758341710 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f530409b180 con 0x55d5ee4c9f20 2024-06-10T01:12:05.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.940+0000 7f5326134b80 1 -- 172.21.15.7:0/758341710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f530c011ac0 con 0x55d5ee4c9f20 2024-06-10T01:12:05.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.940+0000 7f52f8ff9700 1 -- 172.21.15.7:0/758341710 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f53040a5a60 con 0x55d5ee4c9f20 2024-06-10T01:12:05.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.940+0000 7f5326134b80 10 client.17037 fetch_fsmap learned FSMap version 32 2024-06-10T01:12:05.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.940+0000 7f5326134b80 1 -- 172.21.15.7:0/758341710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d5ee4e12b0 con 0x55d5ee4c9f20 2024-06-10T01:12:05.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.941+0000 7f52f8ff9700 1 -- 172.21.15.7:0/758341710 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f530c011ac0 con 0x55d5ee4c9f20 2024-06-10T01:12:05.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.941+0000 7f52f8ff9700 1 -- 172.21.15.7:0/758341710 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f5304011070 con 0x55d5ee4c9f20 2024-06-10T01:12:05.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.941+0000 7f5326134b80 10 client.17037 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:12:05.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.941+0000 7f5326134b80 10 client.17037 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:12:05.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.941+0000 7f5326134b80 20 client.17037 populate_metadata read hostname 'smithi007' 2024-06-10T01:12:05.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.941+0000 7f5326134b80 1 --2- 172.21.15.7:0/758341710 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d5ee4e38b0 0x55d5ee4e5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:05.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.941+0000 7f5326134b80 4 client.17037 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:12:05.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.941+0000 7f5326134b80 1 -- 172.21.15.7:0/758341710 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5ee428d10 con 0x55d5ee4e38b0 2024-06-10T01:12:05.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.941+0000 7f5318e4c700 1 --2- 172.21.15.7:0/758341710 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d5ee4e38b0 0x55d5ee4e5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:05.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.942+0000 7f5318e4c700 1 --2- 172.21.15.7:0/758341710 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d5ee4e38b0 0x55d5ee4e5cd0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46654cdcaf9e5c0b server_cookie=5c178b732b2bcef3 in_seq=0 out_seq=0 2024-06-10T01:12:05.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.942+0000 7f52f8ff9700 10 client.17037 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:05.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.942+0000 7f52f8ff9700 1 -- 172.21.15.7:0/758341710 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d5ee428d10 con 0x55d5ee4e38b0 2024-06-10T01:12:05.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.942+0000 7f52f8ff9700 10 client.17037 handle_command_reply: tid=0 2024-06-10T01:12:05.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.943+0000 7f5326134b80 1 client.17037 shutdown 2024-06-10T01:12:05.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.943+0000 7f5326134b80 1 -- 172.21.15.7:0/758341710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5ee4c9f20 msgr2=0x55d5ee4d5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:05.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.943+0000 7f5326134b80 1 --2- 172.21.15.7:0/758341710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5ee4c9f20 0x55d5ee4d5cb0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f5304004fd0 tx=0x7f530409d600 comp rx=0 tx=0).stop 2024-06-10T01:12:05.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.943+0000 7f5326134b80 1 -- 172.21.15.7:0/758341710 shutdown_connections 2024-06-10T01:12:05.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.943+0000 7f5326134b80 1 --2- 172.21.15.7:0/758341710 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d5ee4e38b0 0x55d5ee4e5cd0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.943+0000 7f52f8ff9700 0 client.17037 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:05.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.943+0000 7f5326134b80 1 --2- 172.21.15.7:0/758341710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ee4cff60 0x55d5ee4d34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.943+0000 7f5326134b80 1 --2- 172.21.15.7:0/758341710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5ee4c9f20 0x55d5ee4d5cb0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.943+0000 7f5326134b80 1 --2- 172.21.15.7:0/758341710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5ee4cf640 0x55d5ee4d2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.943+0000 7f5326134b80 1 -- 172.21.15.7:0/758341710 >> 172.21.15.7:0/758341710 conn(0x55d5ee3f4cc0 msgr2=0x55d5ee4c89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:05.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.943+0000 7f5326134b80 1 -- 172.21.15.7:0/758341710 shutdown_connections 2024-06-10T01:12:05.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.943+0000 7f5326134b80 1 -- 172.21.15.7:0/758341710 wait complete. 2024-06-10T01:12:05.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.943+0000 7f5326134b80 20 client.17037 trim_cache size 0 max 16384 2024-06-10T01:12:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.958+0000 7f5326134b80 1 Processor -- start 2024-06-10T01:12:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.958+0000 7f5326134b80 1 -- start start 2024-06-10T01:12:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.958+0000 7f5326134b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5ee4c9f20 0x55d5ee4c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.958+0000 7f5326134b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5ee4cf640 0x55d5ee4d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.958+0000 7f5326134b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ee4cff60 0x55d5ee4d1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.958+0000 7f5326134b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f53141ae8e0 con 0x55d5ee4cf640 2024-06-10T01:12:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.958+0000 7f5326134b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f53141ac170 con 0x55d5ee4cff60 2024-06-10T01:12:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.959+0000 7f5326134b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f53141a9fc0 con 0x55d5ee4c9f20 2024-06-10T01:12:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.959+0000 7f5318e4c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5ee4cf640 0x55d5ee4d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.959+0000 7f5318e4c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5ee4cf640 0x55d5ee4d5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59056/0 (socket says 172.21.15.7:59056) 2024-06-10T01:12:05.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.959+0000 7f5318e4c700 1 -- 172.21.15.7:0/2268204973 learned_addr learned my addr 172.21.15.7:0/2268204973 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:05.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.959+0000 7f531964d700 1 --2- 172.21.15.7:0/2268204973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5ee4c9f20 0x55d5ee4c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:05.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.959+0000 7f5319e4e700 1 --2- 172.21.15.7:0/2268204973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ee4cff60 0x55d5ee4d1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:05.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.959+0000 7f5318e4c700 1 -- 172.21.15.7:0/2268204973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5ee4c9f20 msgr2=0x55d5ee4c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:05.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.959+0000 7f5318e4c700 1 --2- 172.21.15.7:0/2268204973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5ee4c9f20 0x55d5ee4c99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.959+0000 7f5318e4c700 1 -- 172.21.15.7:0/2268204973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ee4cff60 msgr2=0x55d5ee4d1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:05.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.959+0000 7f5318e4c700 1 --2- 172.21.15.7:0/2268204973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ee4cff60 0x55d5ee4d1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.959+0000 7f5318e4c700 1 -- 172.21.15.7:0/2268204973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5ee4d1540 con 0x55d5ee4cf640 2024-06-10T01:12:05.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.959+0000 7f5319e4e700 1 --2- 172.21.15.7:0/2268204973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ee4cff60 0x55d5ee4d1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:12:05.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.959+0000 7f5318e4c700 1 --2- 172.21.15.7:0/2268204973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5ee4cf640 0x55d5ee4d5cb0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f5308011b50 tx=0x7f5308014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b53a74b14c9f56e5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:05.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.959+0000 7f52faffd700 1 -- 172.21.15.7:0/2268204973 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5308004520 con 0x55d5ee4cf640 2024-06-10T01:12:05.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.959+0000 7f52faffd700 1 -- 172.21.15.7:0/2268204973 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5308004d10 con 0x55d5ee4cf640 2024-06-10T01:12:05.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.959+0000 7f52faffd700 1 -- 172.21.15.7:0/2268204973 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530801f960 con 0x55d5ee4cf640 2024-06-10T01:12:05.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.960+0000 7f5326134b80 1 -- 172.21.15.7:0/2268204973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5ee4cf640 msgr2=0x55d5ee4d5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:05.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.960+0000 7f5326134b80 1 --2- 172.21.15.7:0/2268204973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5ee4cf640 0x55d5ee4d5cb0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f5308011b50 tx=0x7f5308014000 comp rx=0 tx=0).stop 2024-06-10T01:12:05.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.960+0000 7f5326134b80 1 -- 172.21.15.7:0/2268204973 shutdown_connections 2024-06-10T01:12:05.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.960+0000 7f5326134b80 1 --2- 172.21.15.7:0/2268204973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ee4cff60 0x55d5ee4d1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.960+0000 7f5326134b80 1 --2- 172.21.15.7:0/2268204973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5ee4c9f20 0x55d5ee4c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.960+0000 7f5326134b80 1 --2- 172.21.15.7:0/2268204973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5ee4cf640 0x55d5ee4d5cb0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.960+0000 7f5326134b80 1 -- 172.21.15.7:0/2268204973 >> 172.21.15.7:0/2268204973 conn(0x55d5ee3f4cc0 msgr2=0x55d5ee4d0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:05.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.960+0000 7f5326134b80 1 -- 172.21.15.7:0/2268204973 shutdown_connections 2024-06-10T01:12:05.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.960+0000 7f5326134b80 1 -- 172.21.15.7:0/2268204973 wait complete. 2024-06-10T01:12:05.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.960+0000 7f5326134b80 1 -- start start 2024-06-10T01:12:05.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.960+0000 7f5326134b80 1 Processor -- start 2024-06-10T01:12:05.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.961+0000 7f5326134b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ee4c9f20 0x55d5ee4dbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:05.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.961+0000 7f5326134b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5ee4cf640 0x55d5ee4dc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:05.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.961+0000 7f5326134b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5ee4cff60 0x55d5ee4ea9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:05.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.961+0000 7f5326134b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f53141aa630 con 0x55d5ee4cff60 2024-06-10T01:12:05.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.961+0000 7f5326134b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f53141abaa0 con 0x55d5ee4c9f20 2024-06-10T01:12:05.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.961+0000 7f5326134b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f53141aad00 con 0x55d5ee4cf640 2024-06-10T01:12:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.961+0000 7f5326134b80 10 client.0 _ll_register_callbacks cb 0x55d5ee3b5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:12:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.961+0000 7f5319e4e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5ee4cff60 0x55d5ee4ea9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.961+0000 7f5319e4e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5ee4cff60 0x55d5ee4ea9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59064/0 (socket says 172.21.15.7:59064) 2024-06-10T01:12:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.961+0000 7f5319e4e700 1 -- 172.21.15.7:0/298080156 learned_addr learned my addr 172.21.15.7:0/298080156 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.961+0000 7f531964d700 1 --2- 172.21.15.7:0/298080156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ee4c9f20 0x55d5ee4dbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.961+0000 7f5318e4c700 1 --2- 172.21.15.7:0/298080156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5ee4cf640 0x55d5ee4dc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.961+0000 7f5319e4e700 1 -- 172.21.15.7:0/298080156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5ee4cf640 msgr2=0x55d5ee4dc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.961+0000 7f5319e4e700 1 --2- 172.21.15.7:0/298080156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5ee4cf640 0x55d5ee4dc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.961+0000 7f5319e4e700 1 -- 172.21.15.7:0/298080156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ee4c9f20 msgr2=0x55d5ee4dbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.961+0000 7f5319e4e700 1 --2- 172.21.15.7:0/298080156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ee4c9f20 0x55d5ee4dbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.961+0000 7f5319e4e700 1 -- 172.21.15.7:0/298080156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5ee4eb040 con 0x55d5ee4cff60 2024-06-10T01:12:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.961+0000 7f5318e4c700 1 --2- 172.21.15.7:0/298080156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5ee4cf640 0x55d5ee4dc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:12:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.961+0000 7f5319e4e700 1 --2- 172.21.15.7:0/298080156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5ee4cff60 0x55d5ee4ea9d0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f530c007c10 tx=0x7f530c007fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16da7141e319dc2c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:05.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.961+0000 7f52f8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:12:05.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.962+0000 7f52f8ff9700 1 -- 172.21.15.7:0/298080156 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c005040 con 0x55d5ee4cff60 2024-06-10T01:12:05.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.962+0000 7f52f8ff9700 1 -- 172.21.15.7:0/298080156 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f530c0051e0 con 0x55d5ee4cff60 2024-06-10T01:12:05.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.962+0000 7f5326134b80 1 -- 172.21.15.7:0/298080156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5308004d10 con 0x55d5ee4cff60 2024-06-10T01:12:05.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.962+0000 7f52f8ff9700 1 -- 172.21.15.7:0/298080156 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f530c032050 con 0x55d5ee4cff60 2024-06-10T01:12:05.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.962+0000 7f5326134b80 10 client.17043 fetch_fsmap learned FSMap version 32 2024-06-10T01:12:05.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.962+0000 7f5326134b80 1 -- 172.21.15.7:0/298080156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d5ee54de90 con 0x55d5ee4cff60 2024-06-10T01:12:05.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.962+0000 7f52f8ff9700 1 -- 172.21.15.7:0/298080156 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5308004d10 con 0x55d5ee4cff60 2024-06-10T01:12:05.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.962+0000 7f52f8ff9700 1 -- 172.21.15.7:0/298080156 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f530c003680 con 0x55d5ee4cff60 2024-06-10T01:12:05.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.962+0000 7f5326134b80 10 client.17043 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:12:05.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.962+0000 7f5326134b80 10 client.17043 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:12:05.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.962+0000 7f5326134b80 20 client.17043 populate_metadata read hostname 'smithi007' 2024-06-10T01:12:05.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.962+0000 7f5326134b80 1 --2- 172.21.15.7:0/298080156 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d5ee4e38b0 0x55d5ee5502c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:05.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.963+0000 7f5326134b80 4 client.17043 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:12:05.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.963+0000 7f5326134b80 1 -- 172.21.15.7:0/298080156 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d5ee423290 con 0x55d5ee4e38b0 2024-06-10T01:12:05.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.963+0000 7f531964d700 1 --2- 172.21.15.7:0/298080156 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d5ee4e38b0 0x55d5ee5502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:05.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.963+0000 7f531964d700 1 --2- 172.21.15.7:0/298080156 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d5ee4e38b0 0x55d5ee5502c0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=216ec6cbd9c03eea server_cookie=283c7736127bc42e in_seq=0 out_seq=0 2024-06-10T01:12:05.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.963+0000 7f52f8ff9700 10 client.17043 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:05.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.963+0000 7f52f8ff9700 1 -- 172.21.15.7:0/298080156 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d5ee423290 con 0x55d5ee4e38b0 2024-06-10T01:12:05.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.963+0000 7f52f8ff9700 10 client.17043 handle_command_reply: tid=0 2024-06-10T01:12:05.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.963+0000 7f5326134b80 1 client.17043 shutdown 2024-06-10T01:12:05.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.964+0000 7f5326134b80 1 -- 172.21.15.7:0/298080156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5ee4cff60 msgr2=0x55d5ee4ea9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:05.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.964+0000 7f5326134b80 1 --2- 172.21.15.7:0/298080156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5ee4cff60 0x55d5ee4ea9d0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f530c007c10 tx=0x7f530c007fa0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.964+0000 7f5326134b80 1 -- 172.21.15.7:0/298080156 shutdown_connections 2024-06-10T01:12:05.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.964+0000 7f5326134b80 1 --2- 172.21.15.7:0/298080156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5ee4cff60 0x55d5ee4ea9d0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.964+0000 7f5326134b80 1 --2- 172.21.15.7:0/298080156 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d5ee4e38b0 0x55d5ee5502c0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.964+0000 7f52f8ff9700 0 client.17043 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:05.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.964+0000 7f5326134b80 1 --2- 172.21.15.7:0/298080156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ee4c9f20 0x55d5ee4dbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.964+0000 7f5326134b80 1 --2- 172.21.15.7:0/298080156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5ee4cf640 0x55d5ee4dc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.964+0000 7f5326134b80 1 -- 172.21.15.7:0/298080156 >> 172.21.15.7:0/298080156 conn(0x55d5ee3f4cc0 msgr2=0x55d5ee4d0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:05.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.964+0000 7f5326134b80 1 -- 172.21.15.7:0/298080156 shutdown_connections 2024-06-10T01:12:05.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.964+0000 7f5326134b80 1 -- 172.21.15.7:0/298080156 wait complete. 2024-06-10T01:12:05.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.964+0000 7f5326134b80 20 client.17043 trim_cache size 0 max 16384 2024-06-10T01:12:05.964 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:12:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.965+0000 7f531b8b1700 1 -- 172.21.15.7:0/1142161981 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f52fc0a2ca0 msgr2=0x7f52fc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.965+0000 7f531b8b1700 1 --2- 172.21.15.7:0/1142161981 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f52fc0a2ca0 0x7f52fc0a5160 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5314079cb0 tx=0x7f5308009260 comp rx=0 tx=0).stop 2024-06-10T01:12:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.965+0000 7f531b8b1700 1 -- 172.21.15.7:0/1142161981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5314102510 msgr2=0x7f531419e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.965+0000 7f531b8b1700 1 --2- 172.21.15.7:0/1142161981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5314102510 0x7f531419e900 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f53040062a0 tx=0x7f5304004d70 comp rx=0 tx=0).stop 2024-06-10T01:12:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.965+0000 7f531b8b1700 1 -- 172.21.15.7:0/1142161981 shutdown_connections 2024-06-10T01:12:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.965+0000 7f531b8b1700 1 --2- 172.21.15.7:0/1142161981 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f52fc0a2ca0 0x7f52fc0a5160 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.965+0000 7f531b8b1700 1 --2- 172.21.15.7:0/1142161981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5314108510 0x7f5314198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.965+0000 7f531b8b1700 1 --2- 172.21.15.7:0/1142161981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5314102510 0x7f531419e900 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.965+0000 7f531b8b1700 1 --2- 172.21.15.7:0/1142161981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5314102ed0 0x7f531419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.965+0000 7f531b8b1700 1 -- 172.21.15.7:0/1142161981 >> 172.21.15.7:0/1142161981 conn(0x7f53140795b0 msgr2=0x7f53141012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:05.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.965+0000 7f531b8b1700 1 -- 172.21.15.7:0/1142161981 shutdown_connections 2024-06-10T01:12:05.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:05.965+0000 7f531b8b1700 1 -- 172.21.15.7:0/1142161981 wait complete. 2024-06-10T01:12:06.733 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 5.92 MB/sec execute 167 sec latency 32887.842 ms 2024-06-10T01:12:07.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:06 smithi078 ceph-mon[36683]: pgmap v615: 129 pgs: 129 active+clean; 66 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 1014 KiB/s wr, 50 op/s 2024-06-10T01:12:07.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:06 smithi175 ceph-mon[36636]: pgmap v615: 129 pgs: 129 active+clean; 66 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 1014 KiB/s wr, 50 op/s 2024-06-10T01:12:07.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:06 smithi007 ceph-mon[32398]: pgmap v615: 129 pgs: 129 active+clean; 66 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 1014 KiB/s wr, 50 op/s 2024-06-10T01:12:07.734 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357888 5.88 MB/sec execute 168 sec latency 33887.944 ms 2024-06-10T01:12:08.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.356+0000 7fb636a93700 1 -- 172.21.15.7:0/1152183530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb630101fd0 msgr2=0x7fb630105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:08.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.356+0000 7fb636a93700 1 --2- 172.21.15.7:0/1152183530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb630101fd0 0x7fb630105fd0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb6240099c0 tx=0x7fb624009cd0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.357+0000 7fb636a93700 1 -- 172.21.15.7:0/1152183530 shutdown_connections 2024-06-10T01:12:08.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.357+0000 7fb636a93700 1 --2- 172.21.15.7:0/1152183530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6301066b0 0x7fb63010e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.357+0000 7fb636a93700 1 --2- 172.21.15.7:0/1152183530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb630101620 0x7fb630101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.357+0000 7fb636a93700 1 --2- 172.21.15.7:0/1152183530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb630101fd0 0x7fb630105fd0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.357+0000 7fb636a93700 1 -- 172.21.15.7:0/1152183530 >> 172.21.15.7:0/1152183530 conn(0x7fb630078340 msgr2=0x7fb630078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:08.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.357+0000 7fb636a93700 1 -- 172.21.15.7:0/1152183530 shutdown_connections 2024-06-10T01:12:08.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.357+0000 7fb636a93700 1 -- 172.21.15.7:0/1152183530 wait complete. 2024-06-10T01:12:08.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.357+0000 7fb636a93700 1 Processor -- start 2024-06-10T01:12:08.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.357+0000 7fb636a93700 1 -- start start 2024-06-10T01:12:08.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.357+0000 7fb636a93700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb630101620 0x7fb630103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:08.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.357+0000 7fb636a93700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb630101fd0 0x7fb6301035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:08.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.357+0000 7fb636a93700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6301066b0 0x7fb630103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:08.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.358+0000 7fb636a93700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb630111690 con 0x7fb630101620 2024-06-10T01:12:08.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.358+0000 7fb636a93700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb630111510 con 0x7fb630101fd0 2024-06-10T01:12:08.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.358+0000 7fb636a93700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb630111810 con 0x7fb6301066b0 2024-06-10T01:12:08.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.358+0000 7fb63482f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb630101620 0x7fb630103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:08.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.358+0000 7fb63482f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb630101620 0x7fb630103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59076/0 (socket says 172.21.15.7:59076) 2024-06-10T01:12:08.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.358+0000 7fb63482f700 1 -- 172.21.15.7:0/1793119266 learned_addr learned my addr 172.21.15.7:0/1793119266 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:08.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.358+0000 7fb62ffff700 1 --2- 172.21.15.7:0/1793119266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb630101fd0 0x7fb6301035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:08.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.358+0000 7fb635030700 1 --2- 172.21.15.7:0/1793119266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6301066b0 0x7fb630103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:08.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.358+0000 7fb63482f700 1 -- 172.21.15.7:0/1793119266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6301066b0 msgr2=0x7fb630103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:08.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.358+0000 7fb63482f700 1 --2- 172.21.15.7:0/1793119266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6301066b0 0x7fb630103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.358+0000 7fb63482f700 1 -- 172.21.15.7:0/1793119266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb630101fd0 msgr2=0x7fb6301035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:08.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.358+0000 7fb63482f700 1 --2- 172.21.15.7:0/1793119266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb630101fd0 0x7fb6301035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.358+0000 7fb63482f700 1 -- 172.21.15.7:0/1793119266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6301a7130 con 0x7fb630101620 2024-06-10T01:12:08.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.358+0000 7fb635030700 1 --2- 172.21.15.7:0/1793119266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6301066b0 0x7fb630103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:12:08.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.358+0000 7fb63482f700 1 --2- 172.21.15.7:0/1793119266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb630101620 0x7fb630103070 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb62000c970 tx=0x7fb62000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:08.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.358+0000 7fb62dffb700 1 -- 172.21.15.7:0/1793119266 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb620007aa0 con 0x7fb630101620 2024-06-10T01:12:08.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.358+0000 7fb636a93700 1 -- 172.21.15.7:0/1793119266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6301a7420 con 0x7fb630101620 2024-06-10T01:12:08.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.358+0000 7fb62dffb700 1 -- 172.21.15.7:0/1793119266 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb620004d10 con 0x7fb630101620 2024-06-10T01:12:08.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.358+0000 7fb62dffb700 1 -- 172.21.15.7:0/1793119266 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62000f450 con 0x7fb630101620 2024-06-10T01:12:08.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.359+0000 7fb636a93700 1 -- 172.21.15.7:0/1793119266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6301a7930 con 0x7fb630101620 2024-06-10T01:12:08.356 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:12:08.356 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:12:08.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.361+0000 7fb62dffb700 1 -- 172.21.15.7:0/1793119266 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb620004750 con 0x7fb630101620 2024-06-10T01:12:08.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.361+0000 7fb62dffb700 1 --2- 172.21.15.7:0/1793119266 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb6180a2c20 0x7fb6180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:08.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.361+0000 7fb62ffff700 1 --2- 172.21.15.7:0/1793119266 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb6180a2c20 0x7fb6180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:08.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.362+0000 7fb62dffb700 1 -- 172.21.15.7:0/1793119266 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb6200cf200 con 0x7fb630101620 2024-06-10T01:12:08.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.362+0000 7fb62ffff700 1 --2- 172.21.15.7:0/1793119266 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb6180a2c20 0x7fb6180a50e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb624009990 tx=0x7fb624005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:08.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.362+0000 7fb641316b80 1 Processor -- start 2024-06-10T01:12:08.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.362+0000 7fb641316b80 1 -- start start 2024-06-10T01:12:08.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.363+0000 7fb641316b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af89f51500 0x55af89f518e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:08.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.363+0000 7fb641316b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af89f51e20 0x55af89f4b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:08.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.363+0000 7fb641316b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af89f4bc50 0x55af89f4c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:08.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.363+0000 7fb641316b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55af89e3e060 con 0x55af89f51500 2024-06-10T01:12:08.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.363+0000 7fb641316b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55af89e9c950 con 0x55af89f4bc50 2024-06-10T01:12:08.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.363+0000 7fb641316b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55af89e04560 con 0x55af89f51e20 2024-06-10T01:12:08.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.363+0000 7fb63482f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af89f51500 0x55af89f518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:08.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.363+0000 7fb63482f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af89f51500 0x55af89f518e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59078/0 (socket says 172.21.15.7:59078) 2024-06-10T01:12:08.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.363+0000 7fb63482f700 1 -- 172.21.15.7:0/2461998650 learned_addr learned my addr 172.21.15.7:0/2461998650 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:08.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.363+0000 7fb62ffff700 1 --2- 172.21.15.7:0/2461998650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af89f51e20 0x55af89f4b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:08.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.363+0000 7fb635030700 1 --2- 172.21.15.7:0/2461998650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af89f4bc50 0x55af89f4c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:08.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.363+0000 7fb63482f700 1 -- 172.21.15.7:0/2461998650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af89f51e20 msgr2=0x55af89f4b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:08.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.363+0000 7fb63482f700 1 --2- 172.21.15.7:0/2461998650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af89f51e20 0x55af89f4b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.363+0000 7fb63482f700 1 -- 172.21.15.7:0/2461998650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af89f4bc50 msgr2=0x55af89f4c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:08.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.363+0000 7fb63482f700 1 --2- 172.21.15.7:0/2461998650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af89f4bc50 0x55af89f4c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.363+0000 7fb63482f700 1 -- 172.21.15.7:0/2461998650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af89deb9d0 con 0x55af89f51500 2024-06-10T01:12:08.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.363+0000 7fb635030700 1 --2- 172.21.15.7:0/2461998650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af89f4bc50 0x55af89f4c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:12:08.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.363+0000 7fb63482f700 1 --2- 172.21.15.7:0/2461998650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af89f51500 0x55af89f518e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fb6200a3dc0 tx=0x7fb62009b250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b45671a15d25435b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:08.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.364+0000 7fb616ffd700 1 -- 172.21.15.7:0/2461998650 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6200b4070 con 0x55af89f51500 2024-06-10T01:12:08.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.364+0000 7fb616ffd700 1 -- 172.21.15.7:0/2461998650 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb620005700 con 0x55af89f51500 2024-06-10T01:12:08.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.364+0000 7fb616ffd700 1 -- 172.21.15.7:0/2461998650 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb620007cd0 con 0x55af89f51500 2024-06-10T01:12:08.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.364+0000 7fb641316b80 1 -- 172.21.15.7:0/2461998650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af89f51500 msgr2=0x55af89f518e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:08.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.364+0000 7fb641316b80 1 --2- 172.21.15.7:0/2461998650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af89f51500 0x55af89f518e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fb6200a3dc0 tx=0x7fb62009b250 comp rx=0 tx=0).stop 2024-06-10T01:12:08.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.365+0000 7fb641316b80 1 -- 172.21.15.7:0/2461998650 shutdown_connections 2024-06-10T01:12:08.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.365+0000 7fb641316b80 1 --2- 172.21.15.7:0/2461998650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af89f4bc50 0x55af89f4c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.365+0000 7fb641316b80 1 --2- 172.21.15.7:0/2461998650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af89f51500 0x55af89f518e0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.365+0000 7fb641316b80 1 --2- 172.21.15.7:0/2461998650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af89f51e20 0x55af89f4b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.365+0000 7fb641316b80 1 -- 172.21.15.7:0/2461998650 >> 172.21.15.7:0/2461998650 conn(0x55af89e78180 msgr2=0x55af89f46a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:08.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.365+0000 7fb641316b80 1 -- 172.21.15.7:0/2461998650 shutdown_connections 2024-06-10T01:12:08.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.365+0000 7fb641316b80 1 -- 172.21.15.7:0/2461998650 wait complete. 2024-06-10T01:12:08.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.365+0000 7fb641316b80 1 -- start start 2024-06-10T01:12:08.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.365+0000 7fb641316b80 1 Processor -- start 2024-06-10T01:12:08.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.366+0000 7fb641316b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af89f4bc50 0x55af89f57bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:08.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.366+0000 7fb641316b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af89f51500 0x55af89f54eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:08.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.366+0000 7fb641316b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af89f51e20 0x55af89f553f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:08.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.366+0000 7fb641316b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55af89e3d0c0 con 0x55af89f51e20 2024-06-10T01:12:08.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.366+0000 7fb641316b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55af89e98910 con 0x55af89f4bc50 2024-06-10T01:12:08.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.366+0000 7fb641316b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55af89f58800 con 0x55af89f51500 2024-06-10T01:12:08.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.366+0000 7fb641316b80 10 client.0 _ll_register_callbacks cb 0x55af89e326b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:12:08.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.366+0000 7fb63482f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af89f4bc50 0x55af89f57bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:08.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.366+0000 7fb63482f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af89f4bc50 0x55af89f57bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:40054/0 (socket says 172.21.15.7:40054) 2024-06-10T01:12:08.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.366+0000 7fb635030700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af89f51e20 0x55af89f553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:08.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.366+0000 7fb63482f700 1 -- 172.21.15.7:0/1333444805 learned_addr learned my addr 172.21.15.7:0/1333444805 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:08.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.366+0000 7fb62ffff700 1 --2- 172.21.15.7:0/1333444805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af89f51500 0x55af89f54eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:08.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.366+0000 7fb63482f700 1 -- 172.21.15.7:0/1333444805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af89f51500 msgr2=0x55af89f54eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:08.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.366+0000 7fb63482f700 1 --2- 172.21.15.7:0/1333444805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af89f51500 0x55af89f54eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.366+0000 7fb63482f700 1 -- 172.21.15.7:0/1333444805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af89f51e20 msgr2=0x55af89f553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:08.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.366+0000 7fb63482f700 1 --2- 172.21.15.7:0/1333444805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af89f51e20 0x55af89f553f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.366+0000 7fb63482f700 1 -- 172.21.15.7:0/1333444805 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af89e62850 con 0x55af89f4bc50 2024-06-10T01:12:08.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.366+0000 7fb635030700 1 --2- 172.21.15.7:0/1333444805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af89f51e20 0x55af89f553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:08.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.367+0000 7fb63482f700 1 --2- 172.21.15.7:0/1333444805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af89f4bc50 0x55af89f57bb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb62009bd70 tx=0x7fb62009bf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cdc373fc7ee7a3a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:08.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.367+0000 7fb60ffff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:12:08.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.367+0000 7fb60ffff700 1 -- 172.21.15.7:0/1333444805 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb620065060 con 0x55af89f4bc50 2024-06-10T01:12:08.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.367+0000 7fb641316b80 1 -- 172.21.15.7:0/1333444805 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb620005700 con 0x55af89f4bc50 2024-06-10T01:12:08.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.367+0000 7fb60ffff700 1 -- 172.21.15.7:0/1333444805 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb620007cd0 con 0x55af89f4bc50 2024-06-10T01:12:08.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.367+0000 7fb60ffff700 1 -- 172.21.15.7:0/1333444805 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb6200b9070 con 0x55af89f4bc50 2024-06-10T01:12:08.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.367+0000 7fb641316b80 10 client.26480 fetch_fsmap learned FSMap version 32 2024-06-10T01:12:08.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.367+0000 7fb641316b80 1 -- 172.21.15.7:0/1333444805 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55af89f62fe0 con 0x55af89f4bc50 2024-06-10T01:12:08.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.367+0000 7fb60ffff700 1 -- 172.21.15.7:0/1333444805 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb620005700 con 0x55af89f4bc50 2024-06-10T01:12:08.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.367+0000 7fb60ffff700 1 -- 172.21.15.7:0/1333444805 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fb6200afc30 con 0x55af89f4bc50 2024-06-10T01:12:08.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.368+0000 7fb641316b80 10 client.26480 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:12:08.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.368+0000 7fb641316b80 10 client.26480 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:12:08.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.368+0000 7fb641316b80 20 client.26480 populate_metadata read hostname 'smithi007' 2024-06-10T01:12:08.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.368+0000 7fb641316b80 1 --2- 172.21.15.7:0/1333444805 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55af89f656e0 0x55af89f67ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:08.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.368+0000 7fb641316b80 4 client.26480 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:12:08.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.368+0000 7fb641316b80 1 -- 172.21.15.7:0/1333444805 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af89ea9e50 con 0x55af89f656e0 2024-06-10T01:12:08.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.368+0000 7fb62ffff700 1 --2- 172.21.15.7:0/1333444805 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55af89f656e0 0x55af89f67ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:08.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.368+0000 7fb62ffff700 1 --2- 172.21.15.7:0/1333444805 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55af89f656e0 0x55af89f67ad0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c832e0eb67d0e46b server_cookie=f21217c8bb6a858b in_seq=0 out_seq=0 2024-06-10T01:12:08.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.368+0000 7fb60ffff700 10 client.26480 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:08.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.369+0000 7fb60ffff700 1 -- 172.21.15.7:0/1333444805 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55af89ea9e50 con 0x55af89f656e0 2024-06-10T01:12:08.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.369+0000 7fb60ffff700 10 client.26480 handle_command_reply: tid=0 2024-06-10T01:12:08.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.369+0000 7fb641316b80 1 client.26480 shutdown 2024-06-10T01:12:08.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.369+0000 7fb641316b80 1 -- 172.21.15.7:0/1333444805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af89f4bc50 msgr2=0x55af89f57bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:08.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.369+0000 7fb641316b80 1 --2- 172.21.15.7:0/1333444805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af89f4bc50 0x55af89f57bb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb62009bd70 tx=0x7fb62009bf70 comp rx=0 tx=0).stop 2024-06-10T01:12:08.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.369+0000 7fb641316b80 1 -- 172.21.15.7:0/1333444805 shutdown_connections 2024-06-10T01:12:08.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.369+0000 7fb641316b80 1 --2- 172.21.15.7:0/1333444805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af89f51e20 0x55af89f553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.369+0000 7fb641316b80 1 --2- 172.21.15.7:0/1333444805 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55af89f656e0 0x55af89f67ad0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.369+0000 7fb641316b80 1 --2- 172.21.15.7:0/1333444805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af89f4bc50 0x55af89f57bb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.369+0000 7fb641316b80 1 --2- 172.21.15.7:0/1333444805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af89f51500 0x55af89f54eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.369+0000 7fb60ffff700 0 client.26480 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:08.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.369+0000 7fb641316b80 1 -- 172.21.15.7:0/1333444805 >> 172.21.15.7:0/1333444805 conn(0x55af89e78180 msgr2=0x55af89f45bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:08.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.370+0000 7fb641316b80 1 -- 172.21.15.7:0/1333444805 shutdown_connections 2024-06-10T01:12:08.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.370+0000 7fb641316b80 1 -- 172.21.15.7:0/1333444805 wait complete. 2024-06-10T01:12:08.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.370+0000 7fb641316b80 20 client.26480 trim_cache size 0 max 16384 2024-06-10T01:12:08.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.391+0000 7fb641316b80 1 Processor -- start 2024-06-10T01:12:08.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.391+0000 7fb641316b80 1 -- start start 2024-06-10T01:12:08.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.392+0000 7fb641316b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af89f4bc50 0x55af89f506e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:08.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.392+0000 7fb641316b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af89f51500 0x55af89f55420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:08.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.392+0000 7fb641316b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af89f51e20 0x55af89f61050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:08.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.392+0000 7fb641316b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb6301b2cc0 con 0x55af89f4bc50 2024-06-10T01:12:08.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.392+0000 7fb641316b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb6301b0550 con 0x55af89f51e20 2024-06-10T01:12:08.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.392+0000 7fb641316b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb6301ae3a0 con 0x55af89f51500 2024-06-10T01:12:08.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.392+0000 7fb63482f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af89f4bc50 0x55af89f506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:08.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.392+0000 7fb63482f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af89f4bc50 0x55af89f506e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59096/0 (socket says 172.21.15.7:59096) 2024-06-10T01:12:08.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.392+0000 7fb63482f700 1 -- 172.21.15.7:0/1446945104 learned_addr learned my addr 172.21.15.7:0/1446945104 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:08.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.392+0000 7fb62ffff700 1 --2- 172.21.15.7:0/1446945104 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af89f51500 0x55af89f55420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:08.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.392+0000 7fb635030700 1 --2- 172.21.15.7:0/1446945104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af89f51e20 0x55af89f61050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:08.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.392+0000 7fb63482f700 1 -- 172.21.15.7:0/1446945104 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af89f51500 msgr2=0x55af89f55420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:08.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.392+0000 7fb63482f700 1 --2- 172.21.15.7:0/1446945104 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af89f51500 0x55af89f55420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.392+0000 7fb63482f700 1 -- 172.21.15.7:0/1446945104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af89f51e20 msgr2=0x55af89f61050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:08.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.392+0000 7fb63482f700 1 --2- 172.21.15.7:0/1446945104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af89f51e20 0x55af89f61050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.392+0000 7fb63482f700 1 -- 172.21.15.7:0/1446945104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af89f68100 con 0x55af89f4bc50 2024-06-10T01:12:08.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.392+0000 7fb635030700 1 --2- 172.21.15.7:0/1446945104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af89f51e20 0x55af89f61050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:08.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.392+0000 7fb63482f700 1 --2- 172.21.15.7:0/1446945104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af89f4bc50 0x55af89f506e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fb620004fb0 tx=0x7fb62009af60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a05a1b84777bda server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:08.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.393+0000 7fb6167fc700 1 -- 172.21.15.7:0/1446945104 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb620065040 con 0x55af89f4bc50 2024-06-10T01:12:08.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.393+0000 7fb6167fc700 1 -- 172.21.15.7:0/1446945104 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb620005700 con 0x55af89f4bc50 2024-06-10T01:12:08.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.393+0000 7fb6167fc700 1 -- 172.21.15.7:0/1446945104 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6200b4070 con 0x55af89f4bc50 2024-06-10T01:12:08.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.393+0000 7fb641316b80 1 -- 172.21.15.7:0/1446945104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af89f4bc50 msgr2=0x55af89f506e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:08.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.393+0000 7fb641316b80 1 --2- 172.21.15.7:0/1446945104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af89f4bc50 0x55af89f506e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fb620004fb0 tx=0x7fb62009af60 comp rx=0 tx=0).stop 2024-06-10T01:12:08.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.393+0000 7fb641316b80 1 -- 172.21.15.7:0/1446945104 shutdown_connections 2024-06-10T01:12:08.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.393+0000 7fb641316b80 1 --2- 172.21.15.7:0/1446945104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af89f51e20 0x55af89f61050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.393+0000 7fb641316b80 1 --2- 172.21.15.7:0/1446945104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af89f4bc50 0x55af89f506e0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.393+0000 7fb641316b80 1 --2- 172.21.15.7:0/1446945104 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af89f51500 0x55af89f55420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.393+0000 7fb641316b80 1 -- 172.21.15.7:0/1446945104 >> 172.21.15.7:0/1446945104 conn(0x55af89e78180 msgr2=0x55af89f60940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:08.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.394+0000 7fb641316b80 1 -- 172.21.15.7:0/1446945104 shutdown_connections 2024-06-10T01:12:08.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.394+0000 7fb641316b80 1 -- 172.21.15.7:0/1446945104 wait complete. 2024-06-10T01:12:08.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.394+0000 7fb641316b80 1 -- start start 2024-06-10T01:12:08.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.394+0000 7fb641316b80 1 Processor -- start 2024-06-10T01:12:08.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.394+0000 7fb641316b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af89f4bc50 0x55af89f6d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:08.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.394+0000 7fb641316b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af89f51500 0x55af89f6d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:08.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.394+0000 7fb641316b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af89f51e20 0x55af89f6db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:08.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.394+0000 7fb641316b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb6301aea10 con 0x55af89f4bc50 2024-06-10T01:12:08.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.394+0000 7fb641316b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb6301afe80 con 0x55af89f51e20 2024-06-10T01:12:08.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.394+0000 7fb641316b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb6301af0e0 con 0x55af89f51500 2024-06-10T01:12:08.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.395+0000 7fb641316b80 10 client.0 _ll_register_callbacks cb 0x55af89e326b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:12:08.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.395+0000 7fb63482f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af89f4bc50 0x55af89f6d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:08.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.395+0000 7fb63482f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af89f4bc50 0x55af89f6d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59112/0 (socket says 172.21.15.7:59112) 2024-06-10T01:12:08.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.395+0000 7fb63482f700 1 -- 172.21.15.7:0/466451934 learned_addr learned my addr 172.21.15.7:0/466451934 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:08.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.395+0000 7fb62ffff700 1 --2- 172.21.15.7:0/466451934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af89f51500 0x55af89f6d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:08.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.395+0000 7fb635030700 1 --2- 172.21.15.7:0/466451934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af89f51e20 0x55af89f6db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:08.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.395+0000 7fb63482f700 1 -- 172.21.15.7:0/466451934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af89f51500 msgr2=0x55af89f6d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:08.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.395+0000 7fb63482f700 1 --2- 172.21.15.7:0/466451934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af89f51500 0x55af89f6d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.395+0000 7fb63482f700 1 -- 172.21.15.7:0/466451934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af89f51e20 msgr2=0x55af89f6db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:08.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.395+0000 7fb63482f700 1 --2- 172.21.15.7:0/466451934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af89f51e20 0x55af89f6db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.395+0000 7fb63482f700 1 -- 172.21.15.7:0/466451934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af89fcf440 con 0x55af89f4bc50 2024-06-10T01:12:08.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.395+0000 7fb635030700 1 --2- 172.21.15.7:0/466451934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af89f51e20 0x55af89f6db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:12:08.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.395+0000 7fb62ffff700 1 --2- 172.21.15.7:0/466451934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af89f51500 0x55af89f6d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:12:08.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.395+0000 7fb63482f700 1 --2- 172.21.15.7:0/466451934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af89f4bc50 0x55af89f6d0c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fb62000c940 tx=0x7fb62009b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362314f199dc5d23 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:08.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.395+0000 7fb60ffff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:12:08.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.395+0000 7fb60ffff700 1 -- 172.21.15.7:0/466451934 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb620065070 con 0x55af89f4bc50 2024-06-10T01:12:08.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.395+0000 7fb60ffff700 1 -- 172.21.15.7:0/466451934 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb62009a6a0 con 0x55af89f4bc50 2024-06-10T01:12:08.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.395+0000 7fb641316b80 1 -- 172.21.15.7:0/466451934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb620005700 con 0x55af89f4bc50 2024-06-10T01:12:08.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.396+0000 7fb60ffff700 1 -- 172.21.15.7:0/466451934 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb6200b4070 con 0x55af89f4bc50 2024-06-10T01:12:08.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.396+0000 7fb641316b80 10 client.17061 fetch_fsmap learned FSMap version 32 2024-06-10T01:12:08.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.396+0000 7fb641316b80 1 -- 172.21.15.7:0/466451934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55af89fcf440 con 0x55af89f4bc50 2024-06-10T01:12:08.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.396+0000 7fb60ffff700 1 -- 172.21.15.7:0/466451934 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb620005700 con 0x55af89f4bc50 2024-06-10T01:12:08.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.396+0000 7fb60ffff700 1 -- 172.21.15.7:0/466451934 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fb620084060 con 0x55af89f4bc50 2024-06-10T01:12:08.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.396+0000 7fb641316b80 10 client.17061 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:12:08.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.396+0000 7fb641316b80 10 client.17061 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:12:08.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.396+0000 7fb641316b80 20 client.17061 populate_metadata read hostname 'smithi007' 2024-06-10T01:12:08.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.396+0000 7fb641316b80 1 --2- 172.21.15.7:0/466451934 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55af89fd0990 0x55af89fd2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:08.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.396+0000 7fb641316b80 4 client.17061 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "55ac2137", "release": true, "await": true}] 2024-06-10T01:12:08.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.396+0000 7fb641316b80 1 -- 172.21.15.7:0/466451934 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "set_id": "55ac2137", "release": true, "await": true}) -- 0x55af89ea43d0 con 0x55af89fd0990 2024-06-10T01:12:08.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.396+0000 7fb62ffff700 1 --2- 172.21.15.7:0/466451934 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55af89fd0990 0x55af89fd2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:08.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.397+0000 7fb62ffff700 1 --2- 172.21.15.7:0/466451934 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55af89fd0990 0x55af89fd2d80 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bac562a57e8c011c server_cookie=c4db5aa126cbeff4 in_seq=0 out_seq=0 2024-06-10T01:12:08.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.397+0000 7fb60ffff700 10 client.17061 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:08.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.404+0000 7fb60ffff700 1 -- 172.21.15.7:0/466451934 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55af89ea43d0 con 0x55af89fd0990 2024-06-10T01:12:08.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.404+0000 7fb60ffff700 10 client.17061 handle_command_reply: tid=0 2024-06-10T01:12:08.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.404+0000 7fb641316b80 1 client.17061 shutdown 2024-06-10T01:12:08.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.404+0000 7fb641316b80 1 -- 172.21.15.7:0/466451934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af89f4bc50 msgr2=0x55af89f6d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:08.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.405+0000 7fb641316b80 1 --2- 172.21.15.7:0/466451934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af89f4bc50 0x55af89f6d0c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fb62000c940 tx=0x7fb62009b5e0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.405+0000 7fb641316b80 1 -- 172.21.15.7:0/466451934 shutdown_connections 2024-06-10T01:12:08.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.405+0000 7fb641316b80 1 --2- 172.21.15.7:0/466451934 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55af89fd0990 0x55af89fd2d80 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.405+0000 7fb641316b80 1 --2- 172.21.15.7:0/466451934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af89f51e20 0x55af89f6db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.405+0000 7fb641316b80 1 --2- 172.21.15.7:0/466451934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af89f4bc50 0x55af89f6d0c0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.405+0000 7fb641316b80 1 --2- 172.21.15.7:0/466451934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af89f51500 0x55af89f6d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.405+0000 7fb641316b80 1 -- 172.21.15.7:0/466451934 >> 172.21.15.7:0/466451934 conn(0x55af89e78180 msgr2=0x55af89f4b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:08.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.405+0000 7fb60ffff700 0 client.17061 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:08.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.405+0000 7fb641316b80 1 -- 172.21.15.7:0/466451934 shutdown_connections 2024-06-10T01:12:08.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.405+0000 7fb641316b80 1 -- 172.21.15.7:0/466451934 wait complete. 2024-06-10T01:12:08.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.405+0000 7fb641316b80 20 client.17061 trim_cache size 0 max 16384 2024-06-10T01:12:08.401 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:12:08.401 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 32, 2024-06-10T01:12:08.401 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:12:08.401 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 20, 2024-06-10T01:12:08.401 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:12:08.401 INFO:teuthology.orchestra.run.smithi007.stdout: "55ac2137": { 2024-06-10T01:12:08.402 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 20, 2024-06-10T01:12:08.402 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:12:08.402 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:12:08.402 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:12:08.402 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:12:08.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:12:08.402 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:12:08.402 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 149.4, 2024-06-10T01:12:08.402 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:12:08.402 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:12:08.402 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:12:08.402 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:12:08.402 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:12:08.403 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:12:08.403 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:12:08.403 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:12:08.403 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:12:08.403 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:12:08.403 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:12:08.403 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:12:08.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.406+0000 7fb636a93700 1 -- 172.21.15.7:0/1793119266 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb6180a2c20 msgr2=0x7fb6180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:08.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.406+0000 7fb636a93700 1 --2- 172.21.15.7:0/1793119266 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb6180a2c20 0x7fb6180a50e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb624009990 tx=0x7fb624005fb0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.406+0000 7fb636a93700 1 -- 172.21.15.7:0/1793119266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb630101620 msgr2=0x7fb630103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:08.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.406+0000 7fb636a93700 1 --2- 172.21.15.7:0/1793119266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb630101620 0x7fb630103070 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb62000c970 tx=0x7fb62000cc80 comp rx=0 tx=0).stop 2024-06-10T01:12:08.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.406+0000 7fb636a93700 1 -- 172.21.15.7:0/1793119266 shutdown_connections 2024-06-10T01:12:08.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.406+0000 7fb636a93700 1 --2- 172.21.15.7:0/1793119266 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb6180a2c20 0x7fb6180a50e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.406+0000 7fb636a93700 1 --2- 172.21.15.7:0/1793119266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6301066b0 0x7fb630103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.406+0000 7fb636a93700 1 --2- 172.21.15.7:0/1793119266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb630101620 0x7fb630103070 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.406+0000 7fb636a93700 1 --2- 172.21.15.7:0/1793119266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb630101fd0 0x7fb6301035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:08.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.406+0000 7fb636a93700 1 -- 172.21.15.7:0/1793119266 >> 172.21.15.7:0/1793119266 conn(0x7fb630078340 msgr2=0x7fb63006aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:08.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.406+0000 7fb636a93700 1 -- 172.21.15.7:0/1793119266 shutdown_connections 2024-06-10T01:12:08.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:08.406+0000 7fb636a93700 1 -- 172.21.15.7:0/1793119266 wait complete. 2024-06-10T01:12:08.734 INFO:tasks.workunit.client.0.smithi007.stdout: 1 358486 5.88 MB/sec execute 169 sec latency 34556.416 ms 2024-06-10T01:12:09.067 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:08 smithi007 ceph-mon[32398]: pgmap v616: 129 pgs: 129 active+clean; 66 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 962 KiB/s wr, 49 op/s 2024-06-10T01:12:09.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:08 smithi078 ceph-mon[36683]: pgmap v616: 129 pgs: 129 active+clean; 66 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 962 KiB/s wr, 49 op/s 2024-06-10T01:12:09.098 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '55ac2137', total seconds elapsed: 40.6 2024-06-10T01:12:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:12:09.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:08 smithi175 ceph-mon[36636]: pgmap v616: 129 pgs: 129 active+clean; 66 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 962 KiB/s wr, 49 op/s 2024-06-10T01:12:09.734 INFO:tasks.workunit.client.0.smithi007.stdout: 1 359935 5.88 MB/sec execute 170 sec latency 12.769 ms 2024-06-10T01:12:10.734 INFO:tasks.workunit.client.0.smithi007.stdout: 1 362210 5.93 MB/sec execute 171 sec latency 16.680 ms 2024-06-10T01:12:10.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.921+0000 7f63e8146700 1 -- 172.21.15.7:0/3875715404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63e00746c0 msgr2=0x7f63e0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:10.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.921+0000 7f63e8146700 1 --2- 172.21.15.7:0/3875715404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63e00746c0 0x7f63e0074aa0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f63dc009990 tx=0x7f63dc009ca0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.921+0000 7f63e8146700 1 -- 172.21.15.7:0/3875715404 shutdown_connections 2024-06-10T01:12:10.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.921+0000 7f63e8146700 1 --2- 172.21.15.7:0/3875715404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63e010eec0 0x7f63e01112b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.921+0000 7f63e8146700 1 --2- 172.21.15.7:0/3875715404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63e00746c0 0x7f63e0074aa0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.921+0000 7f63e8146700 1 --2- 172.21.15.7:0/3875715404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63e0074fe0 0x7f63e010e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.921+0000 7f63e8146700 1 -- 172.21.15.7:0/3875715404 >> 172.21.15.7:0/3875715404 conn(0x7f63e0079640 msgr2=0x7f63e0079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:10.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.921+0000 7f63e8146700 1 -- 172.21.15.7:0/3875715404 shutdown_connections 2024-06-10T01:12:10.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.921+0000 7f63e8146700 1 -- 172.21.15.7:0/3875715404 wait complete. 2024-06-10T01:12:10.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.922+0000 7f63e8146700 1 Processor -- start 2024-06-10T01:12:10.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.922+0000 7f63e8146700 1 -- start start 2024-06-10T01:12:10.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.922+0000 7f63e8146700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63e00746c0 0x7f63e019ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:10.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.922+0000 7f63e8146700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63e0074fe0 0x7f63e019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:10.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.922+0000 7f63e8146700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63e010eec0 0x7f63e019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:10.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.922+0000 7f63e8146700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f63e0114a20 con 0x7f63e0074fe0 2024-06-10T01:12:10.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.922+0000 7f63e8146700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f63e01148a0 con 0x7f63e010eec0 2024-06-10T01:12:10.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.922+0000 7f63e8146700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f63e0114ba0 con 0x7f63e00746c0 2024-06-10T01:12:10.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.922+0000 7f63e56e1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63e0074fe0 0x7f63e019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:10.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.923+0000 7f63e56e1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63e0074fe0 0x7f63e019b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59118/0 (socket says 172.21.15.7:59118) 2024-06-10T01:12:10.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.923+0000 7f63e56e1700 1 -- 172.21.15.7:0/2762600536 learned_addr learned my addr 172.21.15.7:0/2762600536 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:10.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.923+0000 7f63e5ee2700 1 --2- 172.21.15.7:0/2762600536 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63e00746c0 0x7f63e019ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:10.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.923+0000 7f63e66e3700 1 --2- 172.21.15.7:0/2762600536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63e010eec0 0x7f63e019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:10.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.923+0000 7f63e56e1700 1 -- 172.21.15.7:0/2762600536 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63e00746c0 msgr2=0x7f63e019ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:10.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.923+0000 7f63e56e1700 1 --2- 172.21.15.7:0/2762600536 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63e00746c0 0x7f63e019ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.923+0000 7f63e56e1700 1 -- 172.21.15.7:0/2762600536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63e010eec0 msgr2=0x7f63e019f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:10.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.923+0000 7f63e56e1700 1 --2- 172.21.15.7:0/2762600536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63e010eec0 0x7f63e019f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.923+0000 7f63e56e1700 1 -- 172.21.15.7:0/2762600536 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e019fe50 con 0x7f63e0074fe0 2024-06-10T01:12:10.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.923+0000 7f63e56e1700 1 --2- 172.21.15.7:0/2762600536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63e0074fe0 0x7f63e019b3d0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f63d000c9b0 tx=0x7f63d000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:10.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.923+0000 7f63d6ffd700 1 -- 172.21.15.7:0/2762600536 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d0007b00 con 0x7f63e0074fe0 2024-06-10T01:12:10.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.923+0000 7f63d6ffd700 1 -- 172.21.15.7:0/2762600536 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63d000ce50 con 0x7f63e0074fe0 2024-06-10T01:12:10.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.923+0000 7f63e8146700 1 -- 172.21.15.7:0/2762600536 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e01a0140 con 0x7f63e0074fe0 2024-06-10T01:12:10.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.923+0000 7f63d6ffd700 1 -- 172.21.15.7:0/2762600536 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d0018660 con 0x7f63e0074fe0 2024-06-10T01:12:10.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.923+0000 7f63e8146700 1 -- 172.21.15.7:0/2762600536 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e01a79c0 con 0x7f63e0074fe0 2024-06-10T01:12:10.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.927+0000 7f63d6ffd700 1 -- 172.21.15.7:0/2762600536 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f63d000f450 con 0x7f63e0074fe0 2024-06-10T01:12:10.921 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:10 smithi007 ceph-mon[32398]: pgmap v617: 129 pgs: 129 active+clean; 67 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 745 KiB/s wr, 57 op/s 2024-06-10T01:12:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.928+0000 7f63d6ffd700 1 --2- 172.21.15.7:0/2762600536 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f63cc0a2c20 0x7f63cc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.928+0000 7f63e5ee2700 1 --2- 172.21.15.7:0/2762600536 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f63cc0a2c20 0x7f63cc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.928+0000 7f63d6ffd700 1 -- 172.21.15.7:0/2762600536 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f63d00cf2f0 con 0x7f63e0074fe0 2024-06-10T01:12:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.928+0000 7f63f29c9b80 1 Processor -- start 2024-06-10T01:12:10.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.928+0000 7f63f29c9b80 1 -- start start 2024-06-10T01:12:10.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.928+0000 7f63e5ee2700 1 --2- 172.21.15.7:0/2762600536 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f63cc0a2c20 0x7f63cc0a50e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f63dc0098e0 tx=0x7f63dc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:10.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.929+0000 7f63f29c9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649e5aa3fe0 0x5649e5aa43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:10.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.929+0000 7f63f29c9b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e5aa4900 0x5649e5a9e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:10.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.929+0000 7f63f29c9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649e5a9e820 0x5649e5a9ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:10.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.929+0000 7f63f29c9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5649e5990fe0 con 0x5649e5a9e820 2024-06-10T01:12:10.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.929+0000 7f63f29c9b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5649e59ef8d0 con 0x5649e5aa4900 2024-06-10T01:12:10.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.929+0000 7f63f29c9b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5649e59574e0 con 0x5649e5aa3fe0 2024-06-10T01:12:10.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.929+0000 7f63e5ee2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649e5aa3fe0 0x5649e5aa43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:10.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.929+0000 7f63e56e1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e5aa4900 0x5649e5a9e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:10.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.929+0000 7f63e5ee2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649e5aa3fe0 0x5649e5aa43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:56186/0 (socket says 172.21.15.7:56186) 2024-06-10T01:12:10.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.929+0000 7f63e5ee2700 1 -- 172.21.15.7:0/1418725072 learned_addr learned my addr 172.21.15.7:0/1418725072 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:10.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.929+0000 7f63e66e3700 1 --2- 172.21.15.7:0/1418725072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649e5a9e820 0x5649e5a9ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:10.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.929+0000 7f63e56e1700 1 -- 172.21.15.7:0/1418725072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649e5aa3fe0 msgr2=0x5649e5aa43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:10.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.929+0000 7f63e56e1700 1 --2- 172.21.15.7:0/1418725072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649e5aa3fe0 0x5649e5aa43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.929+0000 7f63e56e1700 1 -- 172.21.15.7:0/1418725072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649e5a9e820 msgr2=0x5649e5a9ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:10.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.929+0000 7f63e56e1700 1 --2- 172.21.15.7:0/1418725072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649e5a9e820 0x5649e5a9ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.929+0000 7f63e56e1700 1 -- 172.21.15.7:0/1418725072 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649e593e950 con 0x5649e5aa4900 2024-06-10T01:12:10.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.929+0000 7f63e66e3700 1 --2- 172.21.15.7:0/1418725072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649e5a9e820 0x5649e5a9ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:12:10.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.929+0000 7f63e56e1700 1 --2- 172.21.15.7:0/1418725072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e5aa4900 0x5649e5a9e2e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f63d00cca70 tx=0x7f63d009bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62a7f47f76ab98cc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:10.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.930+0000 7f63cbfff700 1 -- 172.21.15.7:0/1418725072 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d009fb80 con 0x5649e5aa4900 2024-06-10T01:12:10.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.930+0000 7f63cbfff700 1 -- 172.21.15.7:0/1418725072 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63d00a4440 con 0x5649e5aa4900 2024-06-10T01:12:10.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.930+0000 7f63cbfff700 1 -- 172.21.15.7:0/1418725072 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d009ba30 con 0x5649e5aa4900 2024-06-10T01:12:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.930+0000 7f63f29c9b80 1 -- 172.21.15.7:0/1418725072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e5aa4900 msgr2=0x5649e5a9e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.930+0000 7f63f29c9b80 1 --2- 172.21.15.7:0/1418725072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e5aa4900 0x5649e5a9e2e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f63d00cca70 tx=0x7f63d009bed0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.930+0000 7f63f29c9b80 1 -- 172.21.15.7:0/1418725072 shutdown_connections 2024-06-10T01:12:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.930+0000 7f63f29c9b80 1 --2- 172.21.15.7:0/1418725072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649e5a9e820 0x5649e5a9ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.930+0000 7f63f29c9b80 1 --2- 172.21.15.7:0/1418725072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649e5aa3fe0 0x5649e5aa43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.930+0000 7f63f29c9b80 1 --2- 172.21.15.7:0/1418725072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e5aa4900 0x5649e5a9e2e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.930+0000 7f63f29c9b80 1 -- 172.21.15.7:0/1418725072 >> 172.21.15.7:0/1418725072 conn(0x5649e59cb100 msgr2=0x5649e5a992a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.931+0000 7f63f29c9b80 1 -- 172.21.15.7:0/1418725072 shutdown_connections 2024-06-10T01:12:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.931+0000 7f63f29c9b80 1 -- 172.21.15.7:0/1418725072 wait complete. 2024-06-10T01:12:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.931+0000 7f63f29c9b80 1 -- start start 2024-06-10T01:12:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.931+0000 7f63f29c9b80 1 Processor -- start 2024-06-10T01:12:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.932+0000 7f63f29c9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649e5a9e820 0x5649e5aaa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.932+0000 7f63f29c9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649e5aa3fe0 0x5649e5aa7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.932+0000 7f63f29c9b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e5aa4900 0x5649e5aa7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.932+0000 7f63f29c9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5649e5990040 con 0x5649e5aa3fe0 2024-06-10T01:12:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.932+0000 7f63f29c9b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5649e59eb890 con 0x5649e5aa4900 2024-06-10T01:12:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.932+0000 7f63f29c9b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5649e5aab2e0 con 0x5649e5a9e820 2024-06-10T01:12:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.932+0000 7f63f29c9b80 10 client.0 _ll_register_callbacks cb 0x5649e5985630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:12:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.932+0000 7f63e56e1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649e5aa3fe0 0x5649e5aa7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.932+0000 7f63e5ee2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649e5a9e820 0x5649e5aaa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.932+0000 7f63e5ee2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649e5a9e820 0x5649e5aaa760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:56188/0 (socket says 172.21.15.7:56188) 2024-06-10T01:12:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.932+0000 7f63e56e1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649e5aa3fe0 0x5649e5aa7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59134/0 (socket says 172.21.15.7:59134) 2024-06-10T01:12:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.932+0000 7f63e5ee2700 1 -- 172.21.15.7:0/1339001312 learned_addr learned my addr 172.21.15.7:0/1339001312 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.932+0000 7f63e66e3700 1 --2- 172.21.15.7:0/1339001312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e5aa4900 0x5649e5aa7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:10.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.932+0000 7f63e5ee2700 1 -- 172.21.15.7:0/1339001312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e5aa4900 msgr2=0x5649e5aa7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:10.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.932+0000 7f63e5ee2700 1 --2- 172.21.15.7:0/1339001312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e5aa4900 0x5649e5aa7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.932+0000 7f63e5ee2700 1 -- 172.21.15.7:0/1339001312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649e5aa3fe0 msgr2=0x5649e5aa7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:10.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.932+0000 7f63e5ee2700 1 --2- 172.21.15.7:0/1339001312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649e5aa3fe0 0x5649e5aa7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.932+0000 7f63e5ee2700 1 -- 172.21.15.7:0/1339001312 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649e59b57d0 con 0x5649e5a9e820 2024-06-10T01:12:10.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.932+0000 7f63e5ee2700 1 --2- 172.21.15.7:0/1339001312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649e5a9e820 0x5649e5aaa760 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f63dc00b2e0 tx=0x7f63dc009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5896fd655f51914f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:10.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.932+0000 7f63c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:12:10.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.933+0000 7f63c97fa700 1 -- 172.21.15.7:0/1339001312 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc00f660 con 0x5649e5a9e820 2024-06-10T01:12:10.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.933+0000 7f63c97fa700 1 -- 172.21.15.7:0/1339001312 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f63dc00f800 con 0x5649e5a9e820 2024-06-10T01:12:10.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.933+0000 7f63f29c9b80 1 -- 172.21.15.7:0/1339001312 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63d00a4440 con 0x5649e5a9e820 2024-06-10T01:12:10.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.933+0000 7f63c97fa700 1 -- 172.21.15.7:0/1339001312 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f63dc03c050 con 0x5649e5a9e820 2024-06-10T01:12:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.933+0000 7f63f29c9b80 10 client.26500 fetch_fsmap learned FSMap version 32 2024-06-10T01:12:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.933+0000 7f63f29c9b80 1 -- 172.21.15.7:0/1339001312 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5649e5ab5b30 con 0x5649e5a9e820 2024-06-10T01:12:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.933+0000 7f63c97fa700 1 -- 172.21.15.7:0/1339001312 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63d00a4440 con 0x5649e5a9e820 2024-06-10T01:12:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.933+0000 7f63c97fa700 1 -- 172.21.15.7:0/1339001312 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f63dc0228f0 con 0x5649e5a9e820 2024-06-10T01:12:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.933+0000 7f63f29c9b80 10 client.26500 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:12:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.933+0000 7f63f29c9b80 10 client.26500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:12:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.933+0000 7f63f29c9b80 20 client.26500 populate_metadata read hostname 'smithi007' 2024-06-10T01:12:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.933+0000 7f63f29c9b80 1 --2- 172.21.15.7:0/1339001312 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5649e5ab8170 0x5649e5aba560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.933+0000 7f63f29c9b80 4 client.26500 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:12:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.933+0000 7f63f29c9b80 1 -- 172.21.15.7:0/1339001312 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649e59fcdd0 con 0x5649e5ab8170 2024-06-10T01:12:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.934+0000 7f63e56e1700 1 --2- 172.21.15.7:0/1339001312 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5649e5ab8170 0x5649e5aba560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.934+0000 7f63e56e1700 1 --2- 172.21.15.7:0/1339001312 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5649e5ab8170 0x5649e5aba560 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9d522e5c87b9ae3 server_cookie=20405e22c2c781da in_seq=0 out_seq=0 2024-06-10T01:12:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.934+0000 7f63c97fa700 10 client.26500 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.935+0000 7f63c97fa700 1 -- 172.21.15.7:0/1339001312 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5649e59fcdd0 con 0x5649e5ab8170 2024-06-10T01:12:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.935+0000 7f63c97fa700 10 client.26500 handle_command_reply: tid=0 2024-06-10T01:12:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.935+0000 7f63f29c9b80 1 client.26500 shutdown 2024-06-10T01:12:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.936+0000 7f63f29c9b80 1 -- 172.21.15.7:0/1339001312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649e5a9e820 msgr2=0x5649e5aaa760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.936+0000 7f63f29c9b80 1 --2- 172.21.15.7:0/1339001312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649e5a9e820 0x5649e5aaa760 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f63dc00b2e0 tx=0x7f63dc009ca0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.936+0000 7f63f29c9b80 1 -- 172.21.15.7:0/1339001312 shutdown_connections 2024-06-10T01:12:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.936+0000 7f63f29c9b80 1 --2- 172.21.15.7:0/1339001312 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5649e5ab8170 0x5649e5aba560 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.936+0000 7f63f29c9b80 1 --2- 172.21.15.7:0/1339001312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e5aa4900 0x5649e5aa7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.936+0000 7f63f29c9b80 1 --2- 172.21.15.7:0/1339001312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649e5a9e820 0x5649e5aaa760 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.936+0000 7f63c97fa700 0 client.26500 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.936+0000 7f63f29c9b80 1 --2- 172.21.15.7:0/1339001312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649e5aa3fe0 0x5649e5aa7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.936+0000 7f63f29c9b80 1 -- 172.21.15.7:0/1339001312 >> 172.21.15.7:0/1339001312 conn(0x5649e59cb100 msgr2=0x5649e5a98780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.936+0000 7f63f29c9b80 1 -- 172.21.15.7:0/1339001312 shutdown_connections 2024-06-10T01:12:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.936+0000 7f63f29c9b80 1 -- 172.21.15.7:0/1339001312 wait complete. 2024-06-10T01:12:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.936+0000 7f63f29c9b80 20 client.26500 trim_cache size 0 max 16384 2024-06-10T01:12:10.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.959+0000 7f63f29c9b80 1 Processor -- start 2024-06-10T01:12:10.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.959+0000 7f63f29c9b80 1 -- start start 2024-06-10T01:12:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.959+0000 7f63f29c9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649e59cb100 0x5649e5ab4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.959+0000 7f63f29c9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649e5a9e820 0x5649e5aa3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.959+0000 7f63f29c9b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e5aa3fe0 0x5649e5a98f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.959+0000 7f63f29c9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f63e01b2d90 con 0x5649e59cb100 2024-06-10T01:12:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.959+0000 7f63f29c9b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f63e01b0620 con 0x5649e5aa3fe0 2024-06-10T01:12:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.959+0000 7f63f29c9b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f63e01ae470 con 0x5649e5a9e820 2024-06-10T01:12:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.959+0000 7f63e5ee2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649e59cb100 0x5649e5ab4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.959+0000 7f63e5ee2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649e59cb100 0x5649e5ab4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59142/0 (socket says 172.21.15.7:59142) 2024-06-10T01:12:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.959+0000 7f63e5ee2700 1 -- 172.21.15.7:0/2041103800 learned_addr learned my addr 172.21.15.7:0/2041103800 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.959+0000 7f63e56e1700 1 --2- 172.21.15.7:0/2041103800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649e5a9e820 0x5649e5aa3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.959+0000 7f63e66e3700 1 --2- 172.21.15.7:0/2041103800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e5aa3fe0 0x5649e5a98f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.959+0000 7f63e5ee2700 1 -- 172.21.15.7:0/2041103800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649e5a9e820 msgr2=0x5649e5aa3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.959+0000 7f63e5ee2700 1 --2- 172.21.15.7:0/2041103800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649e5a9e820 0x5649e5aa3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.959+0000 7f63e5ee2700 1 -- 172.21.15.7:0/2041103800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e5aa3fe0 msgr2=0x5649e5a98f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:10.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.959+0000 7f63e5ee2700 1 --2- 172.21.15.7:0/2041103800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e5aa3fe0 0x5649e5a98f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.959+0000 7f63e5ee2700 1 -- 172.21.15.7:0/2041103800 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649e5abab90 con 0x5649e59cb100 2024-06-10T01:12:10.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.959+0000 7f63e66e3700 1 --2- 172.21.15.7:0/2041103800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e5aa3fe0 0x5649e5a98f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:12:10.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.959+0000 7f63e5ee2700 1 --2- 172.21.15.7:0/2041103800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649e59cb100 0x5649e5ab4400 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f63dc022d90 tx=0x7f63dc00bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3718779d7270a76 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:10.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.960+0000 7f63cb7fe700 1 -- 172.21.15.7:0/2041103800 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc0225e0 con 0x5649e59cb100 2024-06-10T01:12:10.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.960+0000 7f63cb7fe700 1 -- 172.21.15.7:0/2041103800 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63dc033070 con 0x5649e59cb100 2024-06-10T01:12:10.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.960+0000 7f63cb7fe700 1 -- 172.21.15.7:0/2041103800 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc00fd80 con 0x5649e59cb100 2024-06-10T01:12:10.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.960+0000 7f63f29c9b80 1 -- 172.21.15.7:0/2041103800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649e59cb100 msgr2=0x5649e5ab4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:10.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.960+0000 7f63f29c9b80 1 --2- 172.21.15.7:0/2041103800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649e59cb100 0x5649e5ab4400 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f63dc022d90 tx=0x7f63dc00bfc0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.960+0000 7f63f29c9b80 1 -- 172.21.15.7:0/2041103800 shutdown_connections 2024-06-10T01:12:10.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.960+0000 7f63f29c9b80 1 --2- 172.21.15.7:0/2041103800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e5aa3fe0 0x5649e5a98f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.960+0000 7f63f29c9b80 1 --2- 172.21.15.7:0/2041103800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649e59cb100 0x5649e5ab4400 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.960+0000 7f63f29c9b80 1 --2- 172.21.15.7:0/2041103800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649e5a9e820 0x5649e5aa3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.960+0000 7f63f29c9b80 1 -- 172.21.15.7:0/2041103800 >> 172.21.15.7:0/2041103800 conn(0x5649e5aa58d0 msgr2=0x5649e5aa2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:10.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.961+0000 7f63f29c9b80 1 -- 172.21.15.7:0/2041103800 shutdown_connections 2024-06-10T01:12:10.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.961+0000 7f63f29c9b80 1 -- 172.21.15.7:0/2041103800 wait complete. 2024-06-10T01:12:10.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.961+0000 7f63f29c9b80 1 -- start start 2024-06-10T01:12:10.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.961+0000 7f63f29c9b80 1 Processor -- start 2024-06-10T01:12:10.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.961+0000 7f63f29c9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649e59cb100 0x5649e5aadd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:10.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.961+0000 7f63f29c9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649e5a9e820 0x5649e5aae240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:10.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.961+0000 7f63f29c9b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e5aa3fe0 0x5649e5b1cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:10.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.961+0000 7f63f29c9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f63e01aeae0 con 0x5649e59cb100 2024-06-10T01:12:10.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.961+0000 7f63f29c9b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f63e01aff50 con 0x5649e5aa3fe0 2024-06-10T01:12:10.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.961+0000 7f63f29c9b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f63e01af1b0 con 0x5649e5a9e820 2024-06-10T01:12:10.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.961+0000 7f63f29c9b80 10 client.0 _ll_register_callbacks cb 0x5649e5aa2620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:12:10.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.962+0000 7f63e5ee2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649e59cb100 0x5649e5aadd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:10.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.962+0000 7f63e5ee2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649e59cb100 0x5649e5aadd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59154/0 (socket says 172.21.15.7:59154) 2024-06-10T01:12:10.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.962+0000 7f63e5ee2700 1 -- 172.21.15.7:0/1978468137 learned_addr learned my addr 172.21.15.7:0/1978468137 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:10.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.962+0000 7f63e56e1700 1 --2- 172.21.15.7:0/1978468137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649e5a9e820 0x5649e5aae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:10.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.962+0000 7f63e66e3700 1 --2- 172.21.15.7:0/1978468137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e5aa3fe0 0x5649e5b1cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:10.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.962+0000 7f63e5ee2700 1 -- 172.21.15.7:0/1978468137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649e5a9e820 msgr2=0x5649e5aae240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:10.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.962+0000 7f63e5ee2700 1 --2- 172.21.15.7:0/1978468137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649e5a9e820 0x5649e5aae240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.962+0000 7f63e5ee2700 1 -- 172.21.15.7:0/1978468137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e5aa3fe0 msgr2=0x5649e5b1cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:10.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.962+0000 7f63e5ee2700 1 --2- 172.21.15.7:0/1978468137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e5aa3fe0 0x5649e5b1cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.962+0000 7f63e5ee2700 1 -- 172.21.15.7:0/1978468137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649e5b1d360 con 0x5649e59cb100 2024-06-10T01:12:10.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.962+0000 7f63e56e1700 1 --2- 172.21.15.7:0/1978468137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649e5a9e820 0x5649e5aae240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:10.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.962+0000 7f63e5ee2700 1 --2- 172.21.15.7:0/1978468137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649e59cb100 0x5649e5aadd00 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f63dc011040 tx=0x7f63dc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28cd4f5ddaf676ad server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:10.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.962+0000 7f63c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:12:10.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.962+0000 7f63c97fa700 1 -- 172.21.15.7:0/1978468137 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc026040 con 0x5649e59cb100 2024-06-10T01:12:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.962+0000 7f63c97fa700 1 -- 172.21.15.7:0/1978468137 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f63dc00fdd0 con 0x5649e59cb100 2024-06-10T01:12:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.962+0000 7f63f29c9b80 1 -- 172.21.15.7:0/1978468137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63dc033070 con 0x5649e59cb100 2024-06-10T01:12:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.962+0000 7f63c97fa700 1 -- 172.21.15.7:0/1978468137 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f63dc057050 con 0x5649e59cb100 2024-06-10T01:12:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.962+0000 7f63f29c9b80 10 client.17079 fetch_fsmap learned FSMap version 32 2024-06-10T01:12:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.962+0000 7f63f29c9b80 1 -- 172.21.15.7:0/1978468137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5649e5b246e0 con 0x5649e59cb100 2024-06-10T01:12:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.963+0000 7f63c97fa700 1 -- 172.21.15.7:0/1978468137 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63dc033070 con 0x5649e59cb100 2024-06-10T01:12:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.963+0000 7f63c97fa700 1 -- 172.21.15.7:0/1978468137 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f63dc0497a0 con 0x5649e59cb100 2024-06-10T01:12:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.963+0000 7f63f29c9b80 10 client.17079 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:12:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.963+0000 7f63f29c9b80 10 client.17079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:12:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.963+0000 7f63f29c9b80 20 client.17079 populate_metadata read hostname 'smithi007' 2024-06-10T01:12:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.963+0000 7f63f29c9b80 1 --2- 172.21.15.7:0/1978468137 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5649e5aa4900 0x5649e5b26d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.963+0000 7f63f29c9b80 4 client.17079 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:12:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.963+0000 7f63f29c9b80 1 -- 172.21.15.7:0/1978468137 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5649e59f7350 con 0x5649e5aa4900 2024-06-10T01:12:10.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.963+0000 7f63e56e1700 1 --2- 172.21.15.7:0/1978468137 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5649e5aa4900 0x5649e5b26d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:10.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.963+0000 7f63c97fa700 10 client.17079 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:10.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.963+0000 7f63e56e1700 1 --2- 172.21.15.7:0/1978468137 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5649e5aa4900 0x5649e5b26d10 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f2b51c42d5d91ed server_cookie=898eb441c97ff813 in_seq=0 out_seq=0 2024-06-10T01:12:10.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.964+0000 7f63c97fa700 1 -- 172.21.15.7:0/1978468137 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5649e59f7350 con 0x5649e5aa4900 2024-06-10T01:12:10.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.964+0000 7f63c97fa700 10 client.17079 handle_command_reply: tid=0 2024-06-10T01:12:10.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.964+0000 7f63f29c9b80 1 client.17079 shutdown 2024-06-10T01:12:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.965+0000 7f63f29c9b80 1 -- 172.21.15.7:0/1978468137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649e59cb100 msgr2=0x5649e5aadd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.965+0000 7f63f29c9b80 1 --2- 172.21.15.7:0/1978468137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649e59cb100 0x5649e5aadd00 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f63dc011040 tx=0x7f63dc00f690 comp rx=0 tx=0).stop 2024-06-10T01:12:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.965+0000 7f63f29c9b80 1 -- 172.21.15.7:0/1978468137 shutdown_connections 2024-06-10T01:12:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.965+0000 7f63f29c9b80 1 --2- 172.21.15.7:0/1978468137 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5649e5aa4900 0x5649e5b26d10 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.965+0000 7f63f29c9b80 1 --2- 172.21.15.7:0/1978468137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e5aa3fe0 0x5649e5b1cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.965+0000 7f63f29c9b80 1 --2- 172.21.15.7:0/1978468137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649e59cb100 0x5649e5aadd00 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.965+0000 7f63c97fa700 0 client.17079 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.965+0000 7f63f29c9b80 1 --2- 172.21.15.7:0/1978468137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649e5a9e820 0x5649e5aae240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.965+0000 7f63f29c9b80 1 -- 172.21.15.7:0/1978468137 >> 172.21.15.7:0/1978468137 conn(0x5649e5aa58d0 msgr2=0x5649e5aa33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.965+0000 7f63f29c9b80 1 -- 172.21.15.7:0/1978468137 shutdown_connections 2024-06-10T01:12:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.965+0000 7f63f29c9b80 1 -- 172.21.15.7:0/1978468137 wait complete. 2024-06-10T01:12:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.965+0000 7f63f29c9b80 20 client.17079 trim_cache size 0 max 16384 2024-06-10T01:12:10.965 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:12:10.965 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-10T01:12:10.965 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "ad99df4e-74fc-4c99-af0a-6b835103ab86", 2024-06-10T01:12:10.965 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-10T01:12:10.965 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:12:10.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.966+0000 7f63e8146700 1 -- 172.21.15.7:0/2762600536 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f63cc0a2c20 msgr2=0x7f63cc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:10.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.966+0000 7f63e8146700 1 --2- 172.21.15.7:0/2762600536 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f63cc0a2c20 0x7f63cc0a50e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f63dc0098e0 tx=0x7f63dc005960 comp rx=0 tx=0).stop 2024-06-10T01:12:10.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.966+0000 7f63e8146700 1 -- 172.21.15.7:0/2762600536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63e0074fe0 msgr2=0x7f63e019b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:10.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.966+0000 7f63e8146700 1 --2- 172.21.15.7:0/2762600536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63e0074fe0 0x7f63e019b3d0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f63d000c9b0 tx=0x7f63d000ccc0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.966+0000 7f63e8146700 1 -- 172.21.15.7:0/2762600536 shutdown_connections 2024-06-10T01:12:10.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.966+0000 7f63e8146700 1 --2- 172.21.15.7:0/2762600536 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f63cc0a2c20 0x7f63cc0a50e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.966+0000 7f63e8146700 1 --2- 172.21.15.7:0/2762600536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63e010eec0 0x7f63e019f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.966+0000 7f63e8146700 1 --2- 172.21.15.7:0/2762600536 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63e00746c0 0x7f63e019ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.966+0000 7f63e8146700 1 --2- 172.21.15.7:0/2762600536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63e0074fe0 0x7f63e019b3d0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:10.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.966+0000 7f63e8146700 1 -- 172.21.15.7:0/2762600536 >> 172.21.15.7:0/2762600536 conn(0x7f63e0079640 msgr2=0x7f63e00ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:10.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.966+0000 7f63e8146700 1 -- 172.21.15.7:0/2762600536 shutdown_connections 2024-06-10T01:12:10.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:10.966+0000 7f63e8146700 1 -- 172.21.15.7:0/2762600536 wait complete. 2024-06-10T01:12:11.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:10 smithi078 ceph-mon[36683]: pgmap v617: 129 pgs: 129 active+clean; 67 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 745 KiB/s wr, 57 op/s 2024-06-10T01:12:11.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:10 smithi175 ceph-mon[36636]: pgmap v617: 129 pgs: 129 active+clean; 67 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 745 KiB/s wr, 57 op/s 2024-06-10T01:12:11.674 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:12:11.734 INFO:tasks.workunit.client.0.smithi007.stdout: 1 364670 5.96 MB/sec execute 172 sec latency 15.856 ms 2024-06-10T01:12:11.932 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:12:12.013 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:11 smithi007 ceph-mon[32398]: scrub queued for path: / 2024-06-10T01:12:12.013 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:11 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:12:12.013 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:11 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:12:12.013 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:11 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:12:12.013 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:11 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:12:12.013 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:11 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:12:12.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:11 smithi078 ceph-mon[36683]: scrub queued for path: / 2024-06-10T01:12:12.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:11 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:12:12.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:11 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:12:12.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:11 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:12:12.080 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:11 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:12:12.080 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:11 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:12:12.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:11 smithi175 ceph-mon[36636]: scrub queued for path: / 2024-06-10T01:12:12.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:11 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:12:12.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:11 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:12:12.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:11 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:12:12.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:11 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:12:12.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:11 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:12:12.734 INFO:tasks.workunit.client.0.smithi007.stdout: 1 366308 5.97 MB/sec execute 173 sec latency 11.022 ms 2024-06-10T01:12:13.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:12 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:12:13.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:12 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:12:13.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:12 smithi078 ceph-mon[36683]: pgmap v618: 129 pgs: 129 active+clean; 67 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 115 KiB/s wr, 38 op/s 2024-06-10T01:12:13.166 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:12 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:12:13.166 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:12 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:12:13.166 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:12 smithi007 ceph-mon[32398]: pgmap v618: 129 pgs: 129 active+clean; 67 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 115 KiB/s wr, 38 op/s 2024-06-10T01:12:13.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:12 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:12:13.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:12 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:12:13.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:12 smithi175 ceph-mon[36636]: pgmap v618: 129 pgs: 129 active+clean; 67 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 115 KiB/s wr, 38 op/s 2024-06-10T01:12:13.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.710+0000 7f3da7df1700 1 -- 172.21.15.7:0/578313804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3da01030f0 msgr2=0x7f3da0107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:13.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.710+0000 7f3da7df1700 1 --2- 172.21.15.7:0/578313804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3da01030f0 0x7f3da0107110 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f3d940099c0 tx=0x7f3d94009cd0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.710+0000 7f3da7df1700 1 -- 172.21.15.7:0/578313804 shutdown_connections 2024-06-10T01:12:13.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.710+0000 7f3da7df1700 1 --2- 172.21.15.7:0/578313804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3da0107840 0x7f3da010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.710+0000 7f3da7df1700 1 --2- 172.21.15.7:0/578313804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3da0102740 0x7f3da0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.710+0000 7f3da7df1700 1 --2- 172.21.15.7:0/578313804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3da01030f0 0x7f3da0107110 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.710+0000 7f3da7df1700 1 -- 172.21.15.7:0/578313804 >> 172.21.15.7:0/578313804 conn(0x7f3da00795f0 msgr2=0x7f3da0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:13.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.710+0000 7f3da7df1700 1 -- 172.21.15.7:0/578313804 shutdown_connections 2024-06-10T01:12:13.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.710+0000 7f3da7df1700 1 -- 172.21.15.7:0/578313804 wait complete. 2024-06-10T01:12:13.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.711+0000 7f3da7df1700 1 Processor -- start 2024-06-10T01:12:13.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.711+0000 7f3da7df1700 1 -- start start 2024-06-10T01:12:13.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.711+0000 7f3da7df1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3da0102740 0x7f3da019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.711+0000 7f3da7df1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3da01030f0 0x7f3da019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.711+0000 7f3da7df1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3da0107840 0x7f3da01989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.711+0000 7f3da7df1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3da0112770 con 0x7f3da01030f0 2024-06-10T01:12:13.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.711+0000 7f3da7df1700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3da01125f0 con 0x7f3da0107840 2024-06-10T01:12:13.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.711+0000 7f3da7df1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3da01128f0 con 0x7f3da0102740 2024-06-10T01:12:13.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.712+0000 7f3da538c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3da01030f0 0x7f3da019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.712+0000 7f3da638e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3da0107840 0x7f3da01989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.712+0000 7f3da5b8d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3da0102740 0x7f3da019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.712+0000 7f3da638e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3da0107840 0x7f3da01989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:50018/0 (socket says 172.21.15.7:50018) 2024-06-10T01:12:13.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.712+0000 7f3da538c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3da01030f0 0x7f3da019ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55730/0 (socket says 172.21.15.7:55730) 2024-06-10T01:12:13.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.712+0000 7f3da638e700 1 -- 172.21.15.7:0/4057878127 learned_addr learned my addr 172.21.15.7:0/4057878127 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:13.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.712+0000 7f3da5b8d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3da0102740 0x7f3da019e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:38908/0 (socket says 172.21.15.7:38908) 2024-06-10T01:12:13.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.712+0000 7f3da5b8d700 1 -- 172.21.15.7:0/4057878127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3da0107840 msgr2=0x7f3da01989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:13.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.712+0000 7f3da5b8d700 1 --2- 172.21.15.7:0/4057878127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3da0107840 0x7f3da01989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.712+0000 7f3da5b8d700 1 -- 172.21.15.7:0/4057878127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3da01030f0 msgr2=0x7f3da019ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:13.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.712+0000 7f3da5b8d700 1 --2- 172.21.15.7:0/4057878127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3da01030f0 0x7f3da019ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.712+0000 7f3da5b8d700 1 -- 172.21.15.7:0/4057878127 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da0199110 con 0x7f3da0102740 2024-06-10T01:12:13.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.712+0000 7f3da638e700 1 --2- 172.21.15.7:0/4057878127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3da0107840 0x7f3da01989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:13.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.712+0000 7f3da5b8d700 1 --2- 172.21.15.7:0/4057878127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3da0102740 0x7f3da019e7f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800ef10 tx=0x7f3d9800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:13.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.712+0000 7f3d9effd700 1 -- 172.21.15.7:0/4057878127 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9800c9c0 con 0x7f3da0102740 2024-06-10T01:12:13.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.712+0000 7f3da7df1700 1 -- 172.21.15.7:0/4057878127 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da01993a0 con 0x7f3da0102740 2024-06-10T01:12:13.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.712+0000 7f3d9effd700 1 -- 172.21.15.7:0/4057878127 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d98019070 con 0x7f3da0102740 2024-06-10T01:12:13.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.712+0000 7f3d9effd700 1 -- 172.21.15.7:0/4057878127 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d980146e0 con 0x7f3da0102740 2024-06-10T01:12:13.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.713+0000 7f3da7df1700 1 -- 172.21.15.7:0/4057878127 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da01a56c0 con 0x7f3da0102740 2024-06-10T01:12:13.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.715+0000 7f3d9effd700 1 -- 172.21.15.7:0/4057878127 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f3d98012050 con 0x7f3da0102740 2024-06-10T01:12:13.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.715+0000 7f3d9effd700 1 --2- 172.21.15.7:0/4057878127 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3d880a2c20 0x7f3d880a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.716+0000 7f3d9effd700 1 -- 172.21.15.7:0/4057878127 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f3d980cf3e0 con 0x7f3da0102740 2024-06-10T01:12:13.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.716+0000 7f3da538c700 1 --2- 172.21.15.7:0/4057878127 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3d880a2c20 0x7f3d880a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.716+0000 7f3da538c700 1 --2- 172.21.15.7:0/4057878127 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3d880a2c20 0x7f3d880a50e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009990 tx=0x7f3d94005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:13.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.716+0000 7f3db2674b80 1 Processor -- start 2024-06-10T01:12:13.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.717+0000 7f3db2674b80 1 -- start start 2024-06-10T01:12:13.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.717+0000 7f3db2674b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccb5fe3a70 0x55ccb5fe3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.717+0000 7f3db2674b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccb5fe4390 0x55ccb5fdddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.717+0000 7f3db2674b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccb5fde330 0x55ccb5fde750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.717+0000 7f3db2674b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ccb5f2b810 con 0x55ccb5fe3a70 2024-06-10T01:12:13.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.717+0000 7f3db2674b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ccb5ed3720 con 0x55ccb5fe4390 2024-06-10T01:12:13.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.717+0000 7f3db2674b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ccb5f277d0 con 0x55ccb5fde330 2024-06-10T01:12:13.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.717+0000 7f3da5b8d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccb5fe3a70 0x55ccb5fe3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.717+0000 7f3da538c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccb5fe4390 0x55ccb5fdddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.717+0000 7f3da5b8d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccb5fe3a70 0x55ccb5fe3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55738/0 (socket says 172.21.15.7:55738) 2024-06-10T01:12:13.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.717+0000 7f3da5b8d700 1 -- 172.21.15.7:0/2333671029 learned_addr learned my addr 172.21.15.7:0/2333671029 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:13.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.717+0000 7f3da638e700 1 --2- 172.21.15.7:0/2333671029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccb5fde330 0x55ccb5fde750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.718+0000 7f3da5b8d700 1 -- 172.21.15.7:0/2333671029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccb5fde330 msgr2=0x55ccb5fde750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:13.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.718+0000 7f3da5b8d700 1 --2- 172.21.15.7:0/2333671029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccb5fde330 0x55ccb5fde750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.718+0000 7f3da5b8d700 1 -- 172.21.15.7:0/2333671029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccb5fe4390 msgr2=0x55ccb5fdddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:13.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.718+0000 7f3da5b8d700 1 --2- 172.21.15.7:0/2333671029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccb5fe4390 0x55ccb5fdddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.718+0000 7f3da5b8d700 1 -- 172.21.15.7:0/2333671029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccb5ec9f50 con 0x55ccb5fe3a70 2024-06-10T01:12:13.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.718+0000 7f3da638e700 1 --2- 172.21.15.7:0/2333671029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccb5fde330 0x55ccb5fde750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:13.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.718+0000 7f3da5b8d700 1 --2- 172.21.15.7:0/2333671029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccb5fe3a70 0x55ccb5fe3e50 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f3d9809b7f0 tx=0x7f3d9809b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7cbf0ef5d40ab64 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:13.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.718+0000 7f3d87fff700 1 -- 172.21.15.7:0/2333671029 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9809b860 con 0x55ccb5fe3a70 2024-06-10T01:12:13.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.718+0000 7f3d87fff700 1 -- 172.21.15.7:0/2333671029 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d9809fc10 con 0x55ccb5fe3a70 2024-06-10T01:12:13.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.718+0000 7f3d87fff700 1 -- 172.21.15.7:0/2333671029 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98099b60 con 0x55ccb5fe3a70 2024-06-10T01:12:13.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.719+0000 7f3db2674b80 1 -- 172.21.15.7:0/2333671029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccb5fe3a70 msgr2=0x55ccb5fe3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:13.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.719+0000 7f3db2674b80 1 --2- 172.21.15.7:0/2333671029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccb5fe3a70 0x55ccb5fe3e50 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f3d9809b7f0 tx=0x7f3d9809b470 comp rx=0 tx=0).stop 2024-06-10T01:12:13.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.719+0000 7f3db2674b80 1 -- 172.21.15.7:0/2333671029 shutdown_connections 2024-06-10T01:12:13.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.719+0000 7f3db2674b80 1 --2- 172.21.15.7:0/2333671029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccb5fde330 0x55ccb5fde750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.719+0000 7f3db2674b80 1 --2- 172.21.15.7:0/2333671029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccb5fe3a70 0x55ccb5fe3e50 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.719+0000 7f3db2674b80 1 --2- 172.21.15.7:0/2333671029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccb5fe4390 0x55ccb5fdddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.719+0000 7f3db2674b80 1 -- 172.21.15.7:0/2333671029 >> 172.21.15.7:0/2333671029 conn(0x55ccb5f04cc0 msgr2=0x55ccb5fd4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:13.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.719+0000 7f3db2674b80 1 -- 172.21.15.7:0/2333671029 shutdown_connections 2024-06-10T01:12:13.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.719+0000 7f3db2674b80 1 -- 172.21.15.7:0/2333671029 wait complete. 2024-06-10T01:12:13.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.719+0000 7f3db2674b80 1 -- start start 2024-06-10T01:12:13.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.720+0000 7f3db2674b80 1 Processor -- start 2024-06-10T01:12:13.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.720+0000 7f3db2674b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccb5fde330 0x55ccb5fea1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.720+0000 7f3db2674b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccb5fe3a70 0x55ccb5fe7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.720+0000 7f3db2674b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccb5fe4390 0x55ccb5fe7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.720+0000 7f3db2674b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ccb5fead70 con 0x55ccb5fe3a70 2024-06-10T01:12:13.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.720+0000 7f3db2674b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ccb5ecbbf0 con 0x55ccb5fde330 2024-06-10T01:12:13.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.720+0000 7f3db2674b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ccb5feaef0 con 0x55ccb5fe4390 2024-06-10T01:12:13.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.720+0000 7f3db2674b80 10 client.0 _ll_register_callbacks cb 0x55ccb5ec5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:12:13.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.720+0000 7f3da5b8d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccb5fde330 0x55ccb5fea1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.720+0000 7f3da538c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccb5fe3a70 0x55ccb5fe7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.720+0000 7f3da5b8d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccb5fde330 0x55ccb5fea1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:50040/0 (socket says 172.21.15.7:50040) 2024-06-10T01:12:13.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.720+0000 7f3da5b8d700 1 -- 172.21.15.7:0/3072361686 learned_addr learned my addr 172.21.15.7:0/3072361686 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:13.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.720+0000 7f3da638e700 1 --2- 172.21.15.7:0/3072361686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccb5fe4390 0x55ccb5fe7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.721+0000 7f3da5b8d700 1 -- 172.21.15.7:0/3072361686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccb5fe4390 msgr2=0x55ccb5fe7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:13.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.721+0000 7f3da5b8d700 1 --2- 172.21.15.7:0/3072361686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccb5fe4390 0x55ccb5fe7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.721+0000 7f3da5b8d700 1 -- 172.21.15.7:0/3072361686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccb5fe3a70 msgr2=0x55ccb5fe7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:13.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.721+0000 7f3da5b8d700 1 --2- 172.21.15.7:0/3072361686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccb5fe3a70 0x55ccb5fe7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.721+0000 7f3da5b8d700 1 -- 172.21.15.7:0/3072361686 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccb5e7a8d0 con 0x55ccb5fde330 2024-06-10T01:12:13.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.721+0000 7f3da5b8d700 1 --2- 172.21.15.7:0/3072361686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccb5fde330 0x55ccb5fea1b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800cfd0 tx=0x7f3d980cca70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20f2c4e940f0cb72 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:13.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.721+0000 7f3d857fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:12:13.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.721+0000 7f3d857fa700 1 -- 172.21.15.7:0/3072361686 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d980ba270 con 0x55ccb5fde330 2024-06-10T01:12:13.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.721+0000 7f3d857fa700 1 -- 172.21.15.7:0/3072361686 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f3d9800cd80 con 0x55ccb5fde330 2024-06-10T01:12:13.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.721+0000 7f3db2674b80 1 -- 172.21.15.7:0/3072361686 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d9809fc10 con 0x55ccb5fde330 2024-06-10T01:12:13.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.721+0000 7f3d857fa700 1 -- 172.21.15.7:0/3072361686 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3d980a44d0 con 0x55ccb5fde330 2024-06-10T01:12:13.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.721+0000 7f3db2674b80 10 client.26513 fetch_fsmap learned FSMap version 32 2024-06-10T01:12:13.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.721+0000 7f3db2674b80 1 -- 172.21.15.7:0/3072361686 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ccb5ff57a0 con 0x55ccb5fde330 2024-06-10T01:12:13.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.722+0000 7f3d857fa700 1 -- 172.21.15.7:0/3072361686 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d9809fc10 con 0x55ccb5fde330 2024-06-10T01:12:13.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.722+0000 7f3d857fa700 1 -- 172.21.15.7:0/3072361686 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f3d9809bb50 con 0x55ccb5fde330 2024-06-10T01:12:13.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.722+0000 7f3db2674b80 10 client.26513 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:12:13.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.722+0000 7f3db2674b80 10 client.26513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:12:13.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.722+0000 7f3db2674b80 20 client.26513 populate_metadata read hostname 'smithi007' 2024-06-10T01:12:13.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.722+0000 7f3db2674b80 1 --2- 172.21.15.7:0/3072361686 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ccb5ff7c70 0x55ccb5ffa060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.722+0000 7f3db2674b80 4 client.26513 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:12:13.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.722+0000 7f3db2674b80 1 -- 172.21.15.7:0/3072361686 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccb5f38d10 con 0x55ccb5ff7c70 2024-06-10T01:12:13.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.722+0000 7f3da538c700 1 --2- 172.21.15.7:0/3072361686 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ccb5ff7c70 0x55ccb5ffa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.723+0000 7f3da538c700 1 --2- 172.21.15.7:0/3072361686 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ccb5ff7c70 0x55ccb5ffa060 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7654cf239a84cc62 server_cookie=e05895712e17bdec in_seq=0 out_seq=0 2024-06-10T01:12:13.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.723+0000 7f3d857fa700 10 client.26513 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:13.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.723+0000 7f3d857fa700 1 -- 172.21.15.7:0/3072361686 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ccb5f38d10 con 0x55ccb5ff7c70 2024-06-10T01:12:13.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.723+0000 7f3d857fa700 10 client.26513 handle_command_reply: tid=0 2024-06-10T01:12:13.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.723+0000 7f3db2674b80 1 client.26513 shutdown 2024-06-10T01:12:13.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.723+0000 7f3db2674b80 1 -- 172.21.15.7:0/3072361686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccb5fde330 msgr2=0x55ccb5fea1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:13.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.723+0000 7f3db2674b80 1 --2- 172.21.15.7:0/3072361686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccb5fde330 0x55ccb5fea1b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800cfd0 tx=0x7f3d980cca70 comp rx=0 tx=0).stop 2024-06-10T01:12:13.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.723+0000 7f3db2674b80 1 -- 172.21.15.7:0/3072361686 shutdown_connections 2024-06-10T01:12:13.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.723+0000 7f3db2674b80 1 --2- 172.21.15.7:0/3072361686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccb5fe4390 0x55ccb5fe7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.723+0000 7f3db2674b80 1 --2- 172.21.15.7:0/3072361686 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ccb5ff7c70 0x55ccb5ffa060 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.723+0000 7f3db2674b80 1 --2- 172.21.15.7:0/3072361686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccb5fde330 0x55ccb5fea1b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.724+0000 7f3db2674b80 1 --2- 172.21.15.7:0/3072361686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccb5fe3a70 0x55ccb5fe7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.724+0000 7f3db2674b80 1 -- 172.21.15.7:0/3072361686 >> 172.21.15.7:0/3072361686 conn(0x55ccb5f04cc0 msgr2=0x55ccb5fd89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:13.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.724+0000 7f3d857fa700 0 client.26513 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:13.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.724+0000 7f3db2674b80 1 -- 172.21.15.7:0/3072361686 shutdown_connections 2024-06-10T01:12:13.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.724+0000 7f3db2674b80 1 -- 172.21.15.7:0/3072361686 wait complete. 2024-06-10T01:12:13.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.724+0000 7f3db2674b80 20 client.26513 trim_cache size 0 max 16384 2024-06-10T01:12:13.734 INFO:tasks.workunit.client.0.smithi007.stdout: 1 369034 6.04 MB/sec execute 174 sec latency 47.654 ms 2024-06-10T01:12:13.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.745+0000 7f3db2674b80 1 Processor -- start 2024-06-10T01:12:13.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.745+0000 7f3db2674b80 1 -- start start 2024-06-10T01:12:13.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.746+0000 7f3db2674b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccb5fde330 0x55ccb5fd90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.746+0000 7f3db2674b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccb5fe3a70 0x55ccb5fe48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.746+0000 7f3db2674b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccb5fe4390 0x55ccb5fe4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.746+0000 7f3db2674b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3da01ae900 con 0x55ccb5fe3a70 2024-06-10T01:12:13.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.746+0000 7f3db2674b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3da01ac190 con 0x55ccb5fde330 2024-06-10T01:12:13.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.746+0000 7f3db2674b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3da01aa000 con 0x55ccb5fe4390 2024-06-10T01:12:13.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.746+0000 7f3da5b8d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccb5fde330 0x55ccb5fd90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.746+0000 7f3da538c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccb5fe3a70 0x55ccb5fe48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.746+0000 7f3da5b8d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccb5fde330 0x55ccb5fd90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:50046/0 (socket says 172.21.15.7:50046) 2024-06-10T01:12:13.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.746+0000 7f3da5b8d700 1 -- 172.21.15.7:0/2969732680 learned_addr learned my addr 172.21.15.7:0/2969732680 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:13.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.746+0000 7f3da638e700 1 --2- 172.21.15.7:0/2969732680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccb5fe4390 0x55ccb5fe4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.746+0000 7f3da5b8d700 1 -- 172.21.15.7:0/2969732680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccb5fe4390 msgr2=0x55ccb5fe4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:13.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.746+0000 7f3da5b8d700 1 --2- 172.21.15.7:0/2969732680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccb5fe4390 0x55ccb5fe4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.746+0000 7f3da5b8d700 1 -- 172.21.15.7:0/2969732680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccb5fe3a70 msgr2=0x55ccb5fe48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:13.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.746+0000 7f3da5b8d700 1 --2- 172.21.15.7:0/2969732680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccb5fe3a70 0x55ccb5fe48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.746+0000 7f3da5b8d700 1 -- 172.21.15.7:0/2969732680 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccb5fe5ab0 con 0x55ccb5fde330 2024-06-10T01:12:13.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.746+0000 7f3da538c700 1 --2- 172.21.15.7:0/2969732680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccb5fe3a70 0x55ccb5fe48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:13.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.747+0000 7f3da5b8d700 1 --2- 172.21.15.7:0/2969732680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccb5fde330 0x55ccb5fd90e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3d98098230 tx=0x7f3d98086050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f60bf98dc02d9b91 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:13.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.747+0000 7f3d877fe700 1 -- 172.21.15.7:0/2969732680 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d980ba270 con 0x55ccb5fde330 2024-06-10T01:12:13.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.747+0000 7f3d877fe700 1 -- 172.21.15.7:0/2969732680 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d980b9070 con 0x55ccb5fde330 2024-06-10T01:12:13.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.747+0000 7f3d877fe700 1 -- 172.21.15.7:0/2969732680 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98099900 con 0x55ccb5fde330 2024-06-10T01:12:13.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.747+0000 7f3db2674b80 1 -- 172.21.15.7:0/2969732680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccb5fde330 msgr2=0x55ccb5fd90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:13.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.747+0000 7f3db2674b80 1 --2- 172.21.15.7:0/2969732680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccb5fde330 0x55ccb5fd90e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3d98098230 tx=0x7f3d98086050 comp rx=0 tx=0).stop 2024-06-10T01:12:13.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.748+0000 7f3db2674b80 1 -- 172.21.15.7:0/2969732680 shutdown_connections 2024-06-10T01:12:13.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.748+0000 7f3db2674b80 1 --2- 172.21.15.7:0/2969732680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccb5fe4390 0x55ccb5fe4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.748+0000 7f3db2674b80 1 --2- 172.21.15.7:0/2969732680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccb5fde330 0x55ccb5fd90e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.748+0000 7f3db2674b80 1 --2- 172.21.15.7:0/2969732680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccb5fe3a70 0x55ccb5fe48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.748+0000 7f3db2674b80 1 -- 172.21.15.7:0/2969732680 >> 172.21.15.7:0/2969732680 conn(0x55ccb5f04cc0 msgr2=0x55ccb5fd89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:13.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.748+0000 7f3db2674b80 1 -- 172.21.15.7:0/2969732680 shutdown_connections 2024-06-10T01:12:13.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.748+0000 7f3db2674b80 1 -- 172.21.15.7:0/2969732680 wait complete. 2024-06-10T01:12:13.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.748+0000 7f3db2674b80 1 -- start start 2024-06-10T01:12:13.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.748+0000 7f3db2674b80 1 Processor -- start 2024-06-10T01:12:13.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.749+0000 7f3db2674b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccb5fde330 0x55ccb5ffd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.749+0000 7f3db2674b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccb5fe3a70 0x55ccb5fdc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.749+0000 7f3db2674b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccb5fe4390 0x55ccb5fdce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.749+0000 7f3db2674b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3da01aa650 con 0x55ccb5fe4390 2024-06-10T01:12:13.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.749+0000 7f3db2674b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3da01abac0 con 0x55ccb5fe3a70 2024-06-10T01:12:13.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.749+0000 7f3db2674b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3da01aad20 con 0x55ccb5fde330 2024-06-10T01:12:13.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.749+0000 7f3db2674b80 10 client.0 _ll_register_callbacks cb 0x55ccb5ec5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:12:13.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.749+0000 7f3da538c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccb5fe3a70 0x55ccb5fdc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.749+0000 7f3da5b8d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccb5fde330 0x55ccb5ffd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.749+0000 7f3da538c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccb5fe3a70 0x55ccb5fdc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:50060/0 (socket says 172.21.15.7:50060) 2024-06-10T01:12:13.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.749+0000 7f3da538c700 1 -- 172.21.15.7:0/462315127 learned_addr learned my addr 172.21.15.7:0/462315127 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:13.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.749+0000 7f3da5b8d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccb5fde330 0x55ccb5ffd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:38952/0 (socket says 172.21.15.7:38952) 2024-06-10T01:12:13.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.749+0000 7f3da638e700 1 --2- 172.21.15.7:0/462315127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccb5fe4390 0x55ccb5fdce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.749+0000 7f3da538c700 1 -- 172.21.15.7:0/462315127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccb5fde330 msgr2=0x55ccb5ffd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:13.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.749+0000 7f3da538c700 1 --2- 172.21.15.7:0/462315127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccb5fde330 0x55ccb5ffd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.749+0000 7f3da538c700 1 -- 172.21.15.7:0/462315127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccb5fe4390 msgr2=0x55ccb5fdce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:13.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.749+0000 7f3da538c700 1 --2- 172.21.15.7:0/462315127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccb5fe4390 0x55ccb5fdce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.749+0000 7f3da538c700 1 -- 172.21.15.7:0/462315127 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccb5fdd660 con 0x55ccb5fe3a70 2024-06-10T01:12:13.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.749+0000 7f3da638e700 1 --2- 172.21.15.7:0/462315127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccb5fe4390 0x55ccb5fdce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:12:13.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.749+0000 7f3da538c700 1 --2- 172.21.15.7:0/462315127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccb5fe3a70 0x55ccb5fdc8e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3d94005850 tx=0x7f3d94009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c143b1758166956 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:13.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.750+0000 7f3d857fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:12:13.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.750+0000 7f3d857fa700 1 -- 172.21.15.7:0/462315127 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9403f060 con 0x55ccb5fe3a70 2024-06-10T01:12:13.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.750+0000 7f3d857fa700 1 -- 172.21.15.7:0/462315127 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f3d94024890 con 0x55ccb5fe3a70 2024-06-10T01:12:13.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.750+0000 7f3db2674b80 1 -- 172.21.15.7:0/462315127 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d980b9070 con 0x55ccb5fe3a70 2024-06-10T01:12:13.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.750+0000 7f3d857fa700 1 -- 172.21.15.7:0/462315127 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3d94046050 con 0x55ccb5fe3a70 2024-06-10T01:12:13.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.750+0000 7f3db2674b80 10 client.26519 fetch_fsmap learned FSMap version 32 2024-06-10T01:12:13.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.750+0000 7f3db2674b80 1 -- 172.21.15.7:0/462315127 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ccb605e640 con 0x55ccb5fe3a70 2024-06-10T01:12:13.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.750+0000 7f3d857fa700 1 -- 172.21.15.7:0/462315127 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d9400f460 con 0x55ccb5fe3a70 2024-06-10T01:12:13.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.750+0000 7f3d857fa700 1 -- 172.21.15.7:0/462315127 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f3d9403a410 con 0x55ccb5fe3a70 2024-06-10T01:12:13.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.750+0000 7f3db2674b80 10 client.26519 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:12:13.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.750+0000 7f3db2674b80 10 client.26519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:12:13.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.750+0000 7f3db2674b80 20 client.26519 populate_metadata read hostname 'smithi007' 2024-06-10T01:12:13.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.750+0000 7f3db2674b80 1 --2- 172.21.15.7:0/462315127 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ccb605ecb0 0x55ccb60610d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.750+0000 7f3db2674b80 4 client.26519 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:12:13.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.750+0000 7f3db2674b80 1 -- 172.21.15.7:0/462315127 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f3d980b9070 con 0x55ccb605ecb0 2024-06-10T01:12:13.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.751+0000 7f3da5b8d700 1 --2- 172.21.15.7:0/462315127 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ccb605ecb0 0x55ccb60610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.752+0000 7f3d857fa700 10 client.26519 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:13.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.752+0000 7f3da5b8d700 1 --2- 172.21.15.7:0/462315127 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ccb605ecb0 0x55ccb60610d0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80a1a026e3ae4e1d server_cookie=8d084e9c04a2e94d in_seq=0 out_seq=0 2024-06-10T01:12:13.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.752+0000 7f3d857fa700 1 -- 172.21.15.7:0/462315127 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f3d980b9070 con 0x55ccb605ecb0 2024-06-10T01:12:13.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.752+0000 7f3d857fa700 10 client.26519 handle_command_reply: tid=0 2024-06-10T01:12:13.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.752+0000 7f3db2674b80 1 client.26519 shutdown 2024-06-10T01:12:13.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.752+0000 7f3db2674b80 1 -- 172.21.15.7:0/462315127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccb5fe3a70 msgr2=0x55ccb5fdc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:13.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.752+0000 7f3db2674b80 1 --2- 172.21.15.7:0/462315127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccb5fe3a70 0x55ccb5fdc8e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3d94005850 tx=0x7f3d94009cd0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.752+0000 7f3db2674b80 1 -- 172.21.15.7:0/462315127 shutdown_connections 2024-06-10T01:12:13.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.752+0000 7f3db2674b80 1 --2- 172.21.15.7:0/462315127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccb5fe4390 0x55ccb5fdce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.752+0000 7f3db2674b80 1 --2- 172.21.15.7:0/462315127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccb5fde330 0x55ccb5ffd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.752+0000 7f3db2674b80 1 --2- 172.21.15.7:0/462315127 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ccb605ecb0 0x55ccb60610d0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.752+0000 7f3db2674b80 1 --2- 172.21.15.7:0/462315127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccb5fe3a70 0x55ccb5fdc8e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.752+0000 7f3d857fa700 0 client.26519 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:13.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.752+0000 7f3db2674b80 1 -- 172.21.15.7:0/462315127 >> 172.21.15.7:0/462315127 conn(0x55ccb5f04cc0 msgr2=0x55ccb5fe48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:13.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.752+0000 7f3db2674b80 1 -- 172.21.15.7:0/462315127 shutdown_connections 2024-06-10T01:12:13.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.753+0000 7f3db2674b80 1 -- 172.21.15.7:0/462315127 wait complete. 2024-06-10T01:12:13.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.753+0000 7f3db2674b80 20 client.26519 trim_cache size 0 max 16384 2024-06-10T01:12:13.753 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:12:13.753 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T01:12:13.753 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-10T01:12:13.753 INFO:teuthology.orchestra.run.smithi007.stdout: "ad99df4e-74fc-4c99-af0a-6b835103ab86": { 2024-06-10T01:12:13.753 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-10T01:12:13.753 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "ad99df4e-74fc-4c99-af0a-6b835103ab86", 2024-06-10T01:12:13.753 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-10T01:12:13.754 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:12:13.754 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:12:13.754 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:12:13.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.753+0000 7f3da7df1700 1 -- 172.21.15.7:0/4057878127 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3d880a2c20 msgr2=0x7f3d880a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:13.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.753+0000 7f3da7df1700 1 --2- 172.21.15.7:0/4057878127 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3d880a2c20 0x7f3d880a50e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009990 tx=0x7f3d94005a90 comp rx=0 tx=0).stop 2024-06-10T01:12:13.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.753+0000 7f3da7df1700 1 -- 172.21.15.7:0/4057878127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3da0102740 msgr2=0x7f3da019e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:13.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.753+0000 7f3da7df1700 1 --2- 172.21.15.7:0/4057878127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3da0102740 0x7f3da019e7f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800ef10 tx=0x7f3d9800c3a0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.753+0000 7f3da7df1700 1 -- 172.21.15.7:0/4057878127 shutdown_connections 2024-06-10T01:12:13.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.753+0000 7f3da7df1700 1 --2- 172.21.15.7:0/4057878127 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3d880a2c20 0x7f3d880a50e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.753+0000 7f3da7df1700 1 --2- 172.21.15.7:0/4057878127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3da0107840 0x7f3da01989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.753+0000 7f3da7df1700 1 --2- 172.21.15.7:0/4057878127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3da0102740 0x7f3da019e7f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.753+0000 7f3da7df1700 1 --2- 172.21.15.7:0/4057878127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3da01030f0 0x7f3da019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.753+0000 7f3da7df1700 1 -- 172.21.15.7:0/4057878127 >> 172.21.15.7:0/4057878127 conn(0x7f3da00795f0 msgr2=0x7f3da0101c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:13.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.754+0000 7f3da7df1700 1 -- 172.21.15.7:0/4057878127 shutdown_connections 2024-06-10T01:12:13.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:13.754+0000 7f3da7df1700 1 -- 172.21.15.7:0/4057878127 wait complete. 2024-06-10T01:12:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.789+0000 7f2b2c5d5640 1 -- 172.21.15.7:0/1880216207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b24100900 msgr2=0x7f2b24100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.789+0000 7f2b2c5d5640 1 --2- 172.21.15.7:0/1880216207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b24100900 0x7f2b24100d20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2b20009860 tx=0x7f2b2002a340 comp rx=0 tx=0).stop 2024-06-10T01:12:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.789+0000 7f2b2c5d5640 1 -- 172.21.15.7:0/1880216207 shutdown_connections 2024-06-10T01:12:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.789+0000 7f2b2c5d5640 1 --2- 172.21.15.7:0/1880216207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b241024b0 0x7f2b24108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.789+0000 7f2b2c5d5640 1 --2- 172.21.15.7:0/1880216207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b24101af0 0x7f2b24101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.789+0000 7f2b2c5d5640 1 --2- 172.21.15.7:0/1880216207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b24100900 0x7f2b24100d20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.789+0000 7f2b2c5d5640 1 -- 172.21.15.7:0/1880216207 >> 172.21.15.7:0/1880216207 conn(0x7f2b240fbe70 msgr2=0x7f2b240fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.789+0000 7f2b2c5d5640 1 -- 172.21.15.7:0/1880216207 shutdown_connections 2024-06-10T01:12:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.789+0000 7f2b2c5d5640 1 -- 172.21.15.7:0/1880216207 wait complete. 2024-06-10T01:12:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.790+0000 7f2b2c5d5640 1 Processor -- start 2024-06-10T01:12:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.790+0000 7f2b2c5d5640 1 -- start start 2024-06-10T01:12:13.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.790+0000 7f2b2c5d5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b24100900 0x7f2b24104840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.791+0000 7f2b2c5d5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b24101af0 0x7f2b24102ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.791+0000 7f2b2a371640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b24100900 0x7f2b24104840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.791+0000 7f2b2a371640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b24100900 0x7f2b24104840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55788/0 (socket says 172.21.15.7:55788) 2024-06-10T01:12:13.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.791+0000 7f2b2c5d5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b241024b0 0x7f2b24103410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.791+0000 7f2b2c5d5640 1 -- 172.21.15.7:0/709754228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2410c570 con 0x7f2b24100900 2024-06-10T01:12:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.791+0000 7f2b2a371640 1 -- 172.21.15.7:0/709754228 learned_addr learned my addr 172.21.15.7:0/709754228 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.791+0000 7f2b2c5d5640 1 -- 172.21.15.7:0/709754228 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2410c3f0 con 0x7f2b241024b0 2024-06-10T01:12:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.791+0000 7f2b2c5d5640 1 -- 172.21.15.7:0/709754228 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2410c6f0 con 0x7f2b24101af0 2024-06-10T01:12:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.791+0000 7f2b2ab72640 1 --2- 172.21.15.7:0/709754228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b241024b0 0x7f2b24103410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.791+0000 7f2b29b70640 1 --2- 172.21.15.7:0/709754228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b24101af0 0x7f2b24102ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.791+0000 7f2b2a371640 1 -- 172.21.15.7:0/709754228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b24101af0 msgr2=0x7f2b24102ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.791+0000 7f2b2a371640 1 --2- 172.21.15.7:0/709754228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b24101af0 0x7f2b24102ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.791+0000 7f2b2a371640 1 -- 172.21.15.7:0/709754228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b241024b0 msgr2=0x7f2b24103410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.791+0000 7f2b2a371640 1 --2- 172.21.15.7:0/709754228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b241024b0 0x7f2b24103410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.791+0000 7f2b2a371640 1 -- 172.21.15.7:0/709754228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b24103cd0 con 0x7f2b24100900 2024-06-10T01:12:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.791+0000 7f2b2a371640 1 --2- 172.21.15.7:0/709754228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b24100900 0x7f2b24104840 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2b20009830 tx=0x7f2b20005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.792+0000 7f2b1b7fe640 1 -- 172.21.15.7:0/709754228 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2003f020 con 0x7f2b24100900 2024-06-10T01:12:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.792+0000 7f2b2c5d5640 1 -- 172.21.15.7:0/709754228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b241a1e50 con 0x7f2b24100900 2024-06-10T01:12:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.792+0000 7f2b1b7fe640 1 -- 172.21.15.7:0/709754228 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b20033c60 con 0x7f2b24100900 2024-06-10T01:12:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.792+0000 7f2b1b7fe640 1 -- 172.21.15.7:0/709754228 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b20037040 con 0x7f2b24100900 2024-06-10T01:12:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.792+0000 7f2b2c5d5640 1 -- 172.21.15.7:0/709754228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b241a2250 con 0x7f2b24100900 2024-06-10T01:12:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.794+0000 7f2b1b7fe640 1 -- 172.21.15.7:0/709754228 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f2b20042020 con 0x7f2b24100900 2024-06-10T01:12:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.794+0000 7f2b2f3b6740 1 Processor -- start 2024-06-10T01:12:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.795+0000 7f2b2f3b6740 1 -- start start 2024-06-10T01:12:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.795+0000 7f2b2f3b6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8f38b010 0x558c8f38b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.795+0000 7f2b2f3b6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8f38ba00 0x558c8f390500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.795+0000 7f2b2f3b6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8f390b70 0x558c8f393000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.795+0000 7f2b2f3b6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558c8f2ee4f0 con 0x558c8f38b010 2024-06-10T01:12:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.795+0000 7f2b2f3b6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558c8f28c870 con 0x558c8f38ba00 2024-06-10T01:12:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.795+0000 7f2b2f3b6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558c8f2ea4b0 con 0x558c8f390b70 2024-06-10T01:12:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.795+0000 7f2b2ab72640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8f38b010 0x558c8f38b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.795+0000 7f2b1b7fe640 1 --2- 172.21.15.7:0/709754228 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2b100a2b80 0x7f2b100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.795+0000 7f2b2ab72640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8f38b010 0x558c8f38b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55790/0 (socket says 172.21.15.7:55790) 2024-06-10T01:12:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.795+0000 7f2b2a371640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8f38ba00 0x558c8f390500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.795+0000 7f2b2ab72640 1 -- 172.21.15.7:0/345064458 learned_addr learned my addr 172.21.15.7:0/345064458 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.795+0000 7f2b29b70640 1 --2- 172.21.15.7:0/345064458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8f390b70 0x558c8f393000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.795+0000 7f2b2ab72640 1 --2- 172.21.15.7:0/709754228 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2b100a2b80 0x7f2b100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.795+0000 7f2b2a371640 1 -- 172.21.15.7:0/345064458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8f390b70 msgr2=0x558c8f393000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.795+0000 7f2b2a371640 1 --2- 172.21.15.7:0/345064458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8f390b70 0x558c8f393000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.795+0000 7f2b1b7fe640 1 -- 172.21.15.7:0/709754228 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f2b200ef750 con 0x7f2b24100900 2024-06-10T01:12:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.795+0000 7f2b2a371640 1 -- 172.21.15.7:0/345064458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8f38b010 msgr2=0x558c8f38b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.796+0000 7f2b2a371640 1 --2- 172.21.15.7:0/345064458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8f38b010 0x558c8f38b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.796+0000 7f2b2a371640 1 -- 172.21.15.7:0/345064458 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c8f2a89b0 con 0x558c8f38ba00 2024-06-10T01:12:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.796+0000 7f2b2ab72640 1 --2- 172.21.15.7:0/345064458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8f38b010 0x558c8f38b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.796+0000 7f2b2a371640 1 --2- 172.21.15.7:0/345064458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8f38ba00 0x558c8f390500 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2b200c47b0 tx=0x7f2b20005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5580f56d34df5875 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.796+0000 7f2b18bfe640 1 -- 172.21.15.7:0/345064458 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b20030950 con 0x558c8f38ba00 2024-06-10T01:12:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.796+0000 7f2b18bfe640 1 -- 172.21.15.7:0/345064458 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b20033e10 con 0x558c8f38ba00 2024-06-10T01:12:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.796+0000 7f2b18bfe640 1 -- 172.21.15.7:0/345064458 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b200035f0 con 0x558c8f38ba00 2024-06-10T01:12:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.796+0000 7f2b2ab72640 1 --2- 172.21.15.7:0/709754228 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2b100a2b80 0x7f2b100a5040 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x558c8f38c260 tx=0x7f2b1c004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.797+0000 7f2b2f3b6740 1 -- 172.21.15.7:0/345064458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8f38ba00 msgr2=0x558c8f390500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.797+0000 7f2b2f3b6740 1 --2- 172.21.15.7:0/345064458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8f38ba00 0x558c8f390500 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2b200c47b0 tx=0x7f2b20005f90 comp rx=0 tx=0).stop 2024-06-10T01:12:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.797+0000 7f2b2f3b6740 1 -- 172.21.15.7:0/345064458 shutdown_connections 2024-06-10T01:12:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.797+0000 7f2b2f3b6740 1 --2- 172.21.15.7:0/345064458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8f390b70 0x558c8f393000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.797+0000 7f2b2f3b6740 1 --2- 172.21.15.7:0/345064458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8f38ba00 0x558c8f390500 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.797+0000 7f2b2f3b6740 1 --2- 172.21.15.7:0/345064458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8f38b010 0x558c8f38b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.797+0000 7f2b2f3b6740 1 -- 172.21.15.7:0/345064458 >> 172.21.15.7:0/345064458 conn(0x558c8f386680 msgr2=0x558c8f386a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.797+0000 7f2b2f3b6740 1 -- 172.21.15.7:0/345064458 shutdown_connections 2024-06-10T01:12:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.797+0000 7f2b2f3b6740 1 -- 172.21.15.7:0/345064458 wait complete. 2024-06-10T01:12:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.797+0000 7f2b2f3b6740 1 -- start start 2024-06-10T01:12:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.797+0000 7f2b2f3b6740 1 Processor -- start 2024-06-10T01:12:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.798+0000 7f2b2f3b6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8f38b010 0x558c8f394e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.798+0000 7f2b2f3b6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8f38ba00 0x558c8f395370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.798+0000 7f2b2f3b6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8f390b70 0x558c8f39c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.798+0000 7f2b2f3b6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558c8f28c870 con 0x558c8f390b70 2024-06-10T01:12:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.798+0000 7f2b2f3b6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558c8f277ae0 con 0x558c8f38ba00 2024-06-10T01:12:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.798+0000 7f2b2f3b6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558c8f3938e0 con 0x558c8f38b010 2024-06-10T01:12:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.798+0000 7f2b2f3b6740 10 client.0 _ll_register_callbacks cb 0x558c8f2b09b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:12:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.798+0000 7f2b29b70640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8f38ba00 0x558c8f395370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.798+0000 7f2b2a371640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8f38b010 0x558c8f394e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.798+0000 7f2b29b70640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8f38ba00 0x558c8f395370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:50096/0 (socket says 172.21.15.7:50096) 2024-06-10T01:12:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.798+0000 7f2b29b70640 1 -- 172.21.15.7:0/3074149858 learned_addr learned my addr 172.21.15.7:0/3074149858 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.798+0000 7f2b2a371640 1 --2- 172.21.15.7:0/3074149858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8f38b010 0x558c8f394e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:38974/0 (socket says 172.21.15.7:38974) 2024-06-10T01:12:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.798+0000 7f2b2ab72640 1 --2- 172.21.15.7:0/3074149858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8f390b70 0x558c8f39c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.798+0000 7f2b29b70640 1 -- 172.21.15.7:0/3074149858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8f38b010 msgr2=0x558c8f394e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.798+0000 7f2b29b70640 1 --2- 172.21.15.7:0/3074149858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8f38b010 0x558c8f394e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.798+0000 7f2b29b70640 1 -- 172.21.15.7:0/3074149858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8f390b70 msgr2=0x558c8f39c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.798+0000 7f2b29b70640 1 --2- 172.21.15.7:0/3074149858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8f390b70 0x558c8f39c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.798+0000 7f2b29b70640 1 -- 172.21.15.7:0/3074149858 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c8f002210 con 0x558c8f38ba00 2024-06-10T01:12:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.798+0000 7f2b29b70640 1 --2- 172.21.15.7:0/3074149858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8f38ba00 0x558c8f395370 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2b14012700 tx=0x7f2b14012b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d16fd25dc2fc507e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.798+0000 7f2b0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:12:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.798+0000 7f2b0dffb640 1 -- 172.21.15.7:0/3074149858 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b14004260 con 0x558c8f38ba00 2024-06-10T01:12:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.799+0000 7f2b2f3b6740 1 -- 172.21.15.7:0/3074149858 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558c8f301f10 con 0x558c8f38ba00 2024-06-10T01:12:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.799+0000 7f2b0dffb640 1 -- 172.21.15.7:0/3074149858 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f2b14004400 con 0x558c8f38ba00 2024-06-10T01:12:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.799+0000 7f2b0dffb640 1 -- 172.21.15.7:0/3074149858 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2b14025050 con 0x558c8f38ba00 2024-06-10T01:12:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.799+0000 7f2b2f3b6740 10 client.26531 fetch_fsmap learned FSMap version 32 2024-06-10T01:12:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.799+0000 7f2b2f3b6740 1 -- 172.21.15.7:0/3074149858 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558c8f0ce620 con 0x558c8f38ba00 2024-06-10T01:12:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.799+0000 7f2b0dffb640 1 -- 172.21.15.7:0/3074149858 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558c8f301f10 con 0x558c8f38ba00 2024-06-10T01:12:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.799+0000 7f2b0dffb640 1 -- 172.21.15.7:0/3074149858 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f2b14003ae0 con 0x558c8f38ba00 2024-06-10T01:12:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.799+0000 7f2b2f3b6740 10 client.26531 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:12:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.799+0000 7f2b2f3b6740 10 client.26531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:12:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.799+0000 7f2b2f3b6740 20 client.26531 populate_metadata read hostname 'smithi007' 2024-06-10T01:12:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.799+0000 7f2b2f3b6740 1 --2- 172.21.15.7:0/3074149858 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c8f3a0d50 0x558c8f3a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.799+0000 7f2b2f3b6740 4 client.26531 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:12:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.799+0000 7f2b2f3b6740 1 -- 172.21.15.7:0/3074149858 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c8f2fbc20 con 0x558c8f3a0d50 2024-06-10T01:12:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.799+0000 7f2b2a371640 1 --2- 172.21.15.7:0/3074149858 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c8f3a0d50 0x558c8f3a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.800+0000 7f2b0dffb640 10 client.26531 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.800+0000 7f2b2a371640 1 --2- 172.21.15.7:0/3074149858 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c8f3a0d50 0x558c8f3a31b0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3167bf825f2722a server_cookie=cf72ea77c53693f8 in_seq=0 out_seq=0 2024-06-10T01:12:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.801+0000 7f2b0dffb640 1 -- 172.21.15.7:0/3074149858 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558c8f2fbc20 con 0x558c8f3a0d50 2024-06-10T01:12:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.801+0000 7f2b0dffb640 10 client.26531 handle_command_reply: tid=0 2024-06-10T01:12:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.801+0000 7f2b2f3b6740 1 client.26531 shutdown 2024-06-10T01:12:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.801+0000 7f2b2f3b6740 1 -- 172.21.15.7:0/3074149858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8f38ba00 msgr2=0x558c8f395370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.801+0000 7f2b2f3b6740 1 --2- 172.21.15.7:0/3074149858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8f38ba00 0x558c8f395370 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2b14012700 tx=0x7f2b14012b00 comp rx=0 tx=0).stop 2024-06-10T01:12:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.801+0000 7f2b2f3b6740 1 -- 172.21.15.7:0/3074149858 shutdown_connections 2024-06-10T01:12:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.801+0000 7f2b2f3b6740 1 --2- 172.21.15.7:0/3074149858 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c8f3a0d50 0x558c8f3a31b0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.801+0000 7f2b2f3b6740 1 --2- 172.21.15.7:0/3074149858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8f390b70 0x558c8f39c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.801+0000 7f2b2f3b6740 1 --2- 172.21.15.7:0/3074149858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8f38ba00 0x558c8f395370 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.801+0000 7f2b2f3b6740 1 --2- 172.21.15.7:0/3074149858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8f38b010 0x558c8f394e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.801+0000 7f2b0dffb640 0 client.26531 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.801+0000 7f2b2f3b6740 1 -- 172.21.15.7:0/3074149858 >> 172.21.15.7:0/3074149858 conn(0x558c8f386680 msgr2=0x558c8f3857e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.801+0000 7f2b2f3b6740 1 -- 172.21.15.7:0/3074149858 shutdown_connections 2024-06-10T01:12:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.801+0000 7f2b2f3b6740 1 -- 172.21.15.7:0/3074149858 wait complete. 2024-06-10T01:12:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.801+0000 7f2b2f3b6740 20 client.26531 trim_cache size 0 max 16384 2024-06-10T01:12:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.817+0000 7f2b2f3b6740 1 Processor -- start 2024-06-10T01:12:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.817+0000 7f2b2f3b6740 1 -- start start 2024-06-10T01:12:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.817+0000 7f2b2f3b6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8f38b010 0x558c8f390210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.817+0000 7f2b2f3b6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8f38ba00 0x558c8f39c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.817+0000 7f2b2f3b6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8f390b70 0x558c8f394e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.817+0000 7f2b2f3b6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2b241af070 con 0x558c8f38ba00 2024-06-10T01:12:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.817+0000 7f2b2f3b6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2b241ac940 con 0x558c8f38b010 2024-06-10T01:12:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.817+0000 7f2b2f3b6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2b241aa790 con 0x558c8f390b70 2024-06-10T01:12:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.817+0000 7f2b29b70640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8f38ba00 0x558c8f39c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.817+0000 7f2b29b70640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8f38ba00 0x558c8f39c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55816/0 (socket says 172.21.15.7:55816) 2024-06-10T01:12:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.817+0000 7f2b29b70640 1 -- 172.21.15.7:0/1375824901 learned_addr learned my addr 172.21.15.7:0/1375824901 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.817+0000 7f2b2a371640 1 --2- 172.21.15.7:0/1375824901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8f38b010 0x558c8f390210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.817+0000 7f2b2ab72640 1 --2- 172.21.15.7:0/1375824901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8f390b70 0x558c8f394e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.817+0000 7f2b29b70640 1 -- 172.21.15.7:0/1375824901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8f390b70 msgr2=0x558c8f394e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.817+0000 7f2b29b70640 1 --2- 172.21.15.7:0/1375824901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8f390b70 0x558c8f394e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.817+0000 7f2b29b70640 1 -- 172.21.15.7:0/1375824901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8f38b010 msgr2=0x558c8f390210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.817+0000 7f2b29b70640 1 --2- 172.21.15.7:0/1375824901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8f38b010 0x558c8f390210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.817+0000 7f2b29b70640 1 -- 172.21.15.7:0/1375824901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c8f3864b0 con 0x558c8f38ba00 2024-06-10T01:12:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.817+0000 7f2b2ab72640 1 --2- 172.21.15.7:0/1375824901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8f390b70 0x558c8f394e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:12:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.817+0000 7f2b29b70640 1 --2- 172.21.15.7:0/1375824901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8f38ba00 0x558c8f39c370 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f2b140126d0 tx=0x7f2b14009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2589546e224a4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.818+0000 7f2b0ffff640 1 -- 172.21.15.7:0/1375824901 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1402d020 con 0x558c8f38ba00 2024-06-10T01:12:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.818+0000 7f2b0ffff640 1 -- 172.21.15.7:0/1375824901 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b1400f6f0 con 0x558c8f38ba00 2024-06-10T01:12:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.818+0000 7f2b0ffff640 1 -- 172.21.15.7:0/1375824901 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b14022040 con 0x558c8f38ba00 2024-06-10T01:12:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.818+0000 7f2b2f3b6740 1 -- 172.21.15.7:0/1375824901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8f38ba00 msgr2=0x558c8f39c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.818+0000 7f2b2f3b6740 1 --2- 172.21.15.7:0/1375824901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8f38ba00 0x558c8f39c370 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f2b140126d0 tx=0x7f2b14009640 comp rx=0 tx=0).stop 2024-06-10T01:12:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.818+0000 7f2b2f3b6740 1 -- 172.21.15.7:0/1375824901 shutdown_connections 2024-06-10T01:12:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.818+0000 7f2b2f3b6740 1 --2- 172.21.15.7:0/1375824901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8f390b70 0x558c8f394e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.818+0000 7f2b2f3b6740 1 --2- 172.21.15.7:0/1375824901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8f38ba00 0x558c8f39c370 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.818+0000 7f2b2f3b6740 1 --2- 172.21.15.7:0/1375824901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8f38b010 0x558c8f390210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.818+0000 7f2b2f3b6740 1 -- 172.21.15.7:0/1375824901 >> 172.21.15.7:0/1375824901 conn(0x558c8f386680 msgr2=0x558c8f386a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.818+0000 7f2b2f3b6740 1 -- 172.21.15.7:0/1375824901 shutdown_connections 2024-06-10T01:12:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.818+0000 7f2b2f3b6740 1 -- 172.21.15.7:0/1375824901 wait complete. 2024-06-10T01:12:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.819+0000 7f2b2f3b6740 1 -- start start 2024-06-10T01:12:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.819+0000 7f2b2f3b6740 1 Processor -- start 2024-06-10T01:12:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.819+0000 7f2b2f3b6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8f38b010 0x558c8f399460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.819+0000 7f2b2f3b6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8f38ba00 0x558c8f3999a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.819+0000 7f2b2f3b6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8f390b70 0x558c8f39a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.819+0000 7f2b2f3b6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2b241aae00 con 0x558c8f38ba00 2024-06-10T01:12:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.819+0000 7f2b2f3b6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2b241ac270 con 0x558c8f38b010 2024-06-10T01:12:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.819+0000 7f2b2f3b6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2b241ab4d0 con 0x558c8f390b70 2024-06-10T01:12:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.819+0000 7f2b2f3b6740 10 client.0 _ll_register_callbacks cb 0x7f2b140097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:12:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.819+0000 7f2b29b70640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8f38ba00 0x558c8f3999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.819+0000 7f2b29b70640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8f38ba00 0x558c8f3999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55826/0 (socket says 172.21.15.7:55826) 2024-06-10T01:12:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.819+0000 7f2b29b70640 1 -- 172.21.15.7:0/2202416672 learned_addr learned my addr 172.21.15.7:0/2202416672 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.819+0000 7f2b2a371640 1 --2- 172.21.15.7:0/2202416672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8f38b010 0x558c8f399460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.819+0000 7f2b2ab72640 1 --2- 172.21.15.7:0/2202416672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8f390b70 0x558c8f39a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.819+0000 7f2b29b70640 1 -- 172.21.15.7:0/2202416672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8f390b70 msgr2=0x558c8f39a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.819+0000 7f2b29b70640 1 --2- 172.21.15.7:0/2202416672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8f390b70 0x558c8f39a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.819+0000 7f2b29b70640 1 -- 172.21.15.7:0/2202416672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8f38b010 msgr2=0x558c8f399460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.819+0000 7f2b29b70640 1 --2- 172.21.15.7:0/2202416672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8f38b010 0x558c8f399460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.819+0000 7f2b29b70640 1 -- 172.21.15.7:0/2202416672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c8f387350 con 0x558c8f38ba00 2024-06-10T01:12:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.819+0000 7f2b2a371640 1 --2- 172.21.15.7:0/2202416672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8f38b010 0x558c8f399460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.820+0000 7f2b29b70640 1 --2- 172.21.15.7:0/2202416672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8f38ba00 0x558c8f3999a0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f2b14000c00 tx=0x7f2b14005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6dee1962a09ef5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.820+0000 7f2b0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:12:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.820+0000 7f2b0dffb640 1 -- 172.21.15.7:0/2202416672 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1402d030 con 0x558c8f38ba00 2024-06-10T01:12:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.820+0000 7f2b0dffb640 1 -- 172.21.15.7:0/2202416672 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f2b1400faf0 con 0x558c8f38ba00 2024-06-10T01:12:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.820+0000 7f2b2f3b6740 1 -- 172.21.15.7:0/2202416672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b1400f6f0 con 0x558c8f38ba00 2024-06-10T01:12:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.820+0000 7f2b0dffb640 1 -- 172.21.15.7:0/2202416672 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2b14042050 con 0x558c8f38ba00 2024-06-10T01:12:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.820+0000 7f2b2f3b6740 10 client.17115 fetch_fsmap learned FSMap version 32 2024-06-10T01:12:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.820+0000 7f2b2f3b6740 1 -- 172.21.15.7:0/2202416672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558c8f39c0e0 con 0x558c8f38ba00 2024-06-10T01:12:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.820+0000 7f2b0dffb640 1 -- 172.21.15.7:0/2202416672 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b1400f6f0 con 0x558c8f38ba00 2024-06-10T01:12:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.820+0000 7f2b0dffb640 1 -- 172.21.15.7:0/2202416672 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f2b14011970 con 0x558c8f38ba00 2024-06-10T01:12:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.820+0000 7f2b2f3b6740 10 client.17115 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:12:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.820+0000 7f2b2f3b6740 10 client.17115 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:12:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.820+0000 7f2b2f3b6740 20 client.17115 populate_metadata read hostname 'smithi007' 2024-06-10T01:12:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.820+0000 7f2b2f3b6740 1 --2- 172.21.15.7:0/2202416672 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c8f40bb70 0x558c8f40df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.820+0000 7f2b2f3b6740 4 client.17115 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:12:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.820+0000 7f2b2f3b6740 1 -- 172.21.15.7:0/2202416672 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558c8f2f5f70 con 0x558c8f40bb70 2024-06-10T01:12:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.821+0000 7f2b2a371640 1 --2- 172.21.15.7:0/2202416672 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c8f40bb70 0x558c8f40df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.821+0000 7f2b0dffb640 10 client.17115 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.821+0000 7f2b2a371640 1 --2- 172.21.15.7:0/2202416672 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c8f40bb70 0x558c8f40df90 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96a2ab24489b8c1c server_cookie=2a0f9e40267f2cf1 in_seq=0 out_seq=0 2024-06-10T01:12:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.877+0000 7f2b0dffb640 1 -- 172.21.15.7:0/2202416672 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558c8f2f5f70 con 0x558c8f40bb70 2024-06-10T01:12:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.877+0000 7f2b0dffb640 10 client.17115 handle_command_reply: tid=0 2024-06-10T01:12:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.877+0000 7f2b2f3b6740 1 client.17115 shutdown 2024-06-10T01:12:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.877+0000 7f2b2f3b6740 1 -- 172.21.15.7:0/2202416672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8f38ba00 msgr2=0x558c8f3999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.877+0000 7f2b2f3b6740 1 --2- 172.21.15.7:0/2202416672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8f38ba00 0x558c8f3999a0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f2b14000c00 tx=0x7f2b14005f90 comp rx=0 tx=0).stop 2024-06-10T01:12:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.877+0000 7f2b2f3b6740 1 -- 172.21.15.7:0/2202416672 shutdown_connections 2024-06-10T01:12:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.877+0000 7f2b2f3b6740 1 --2- 172.21.15.7:0/2202416672 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c8f40bb70 0x558c8f40df90 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.877+0000 7f2b2f3b6740 1 --2- 172.21.15.7:0/2202416672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8f390b70 0x558c8f39a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.877+0000 7f2b2f3b6740 1 --2- 172.21.15.7:0/2202416672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8f38ba00 0x558c8f3999a0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.877+0000 7f2b0dffb640 0 client.17115 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.877+0000 7f2b2f3b6740 1 --2- 172.21.15.7:0/2202416672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8f38b010 0x558c8f399460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.877+0000 7f2b2f3b6740 1 -- 172.21.15.7:0/2202416672 >> 172.21.15.7:0/2202416672 conn(0x558c8f386680 msgr2=0x558c8f381560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.877+0000 7f2b2f3b6740 1 -- 172.21.15.7:0/2202416672 shutdown_connections 2024-06-10T01:12:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.877+0000 7f2b2f3b6740 1 -- 172.21.15.7:0/2202416672 wait complete. 2024-06-10T01:12:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.877+0000 7f2b2f3b6740 20 client.17115 trim_cache size 0 max 16384 2024-06-10T01:12:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:12:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:12:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:12:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.878+0000 7f2b2c5d5640 1 -- 172.21.15.7:0/709754228 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2b100a2b80 msgr2=0x7f2b100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.878+0000 7f2b2c5d5640 1 --2- 172.21.15.7:0/709754228 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2b100a2b80 0x7f2b100a5040 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x558c8f38c260 tx=0x7f2b1c004ab0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.878+0000 7f2b2c5d5640 1 -- 172.21.15.7:0/709754228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b24100900 msgr2=0x7f2b24104840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.878+0000 7f2b2c5d5640 1 --2- 172.21.15.7:0/709754228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b24100900 0x7f2b24104840 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2b20009830 tx=0x7f2b20005dc0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.878+0000 7f2b2c5d5640 1 -- 172.21.15.7:0/709754228 shutdown_connections 2024-06-10T01:12:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.878+0000 7f2b2c5d5640 1 --2- 172.21.15.7:0/709754228 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2b100a2b80 0x7f2b100a5040 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.878+0000 7f2b2c5d5640 1 --2- 172.21.15.7:0/709754228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b241024b0 0x7f2b24103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.878+0000 7f2b2c5d5640 1 --2- 172.21.15.7:0/709754228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b24101af0 0x7f2b24102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.878+0000 7f2b2c5d5640 1 --2- 172.21.15.7:0/709754228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b24100900 0x7f2b24104840 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.878+0000 7f2b2c5d5640 1 -- 172.21.15.7:0/709754228 >> 172.21.15.7:0/709754228 conn(0x7f2b240fbe70 msgr2=0x7f2b24069180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.878+0000 7f2b2c5d5640 1 -- 172.21.15.7:0/709754228 shutdown_connections 2024-06-10T01:12:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:13.878+0000 7f2b2c5d5640 1 -- 172.21.15.7:0/709754228 wait complete. 2024-06-10T01:12:14.488 INFO:tasks.cephfs.filesystem:scrub status for tag:ad99df4e-74fc-4c99-af0a-6b835103ab86 - {'path': '/', 'tag': 'ad99df4e-74fc-4c99-af0a-6b835103ab86', 'options': 'recursive,force'} 2024-06-10T01:12:14.734 INFO:tasks.workunit.client.0.smithi007.stdout: 1 370104 6.02 MB/sec execute 175 sec latency 54.179 ms 2024-06-10T01:12:15.080 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:14 smithi078 ceph-mon[36683]: pgmap v619: 129 pgs: 129 active+clean; 67 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 289 KiB/s wr, 59 op/s 2024-06-10T01:12:15.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:14 smithi175 ceph-mon[36636]: pgmap v619: 129 pgs: 129 active+clean; 67 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 289 KiB/s wr, 59 op/s 2024-06-10T01:12:15.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:14 smithi007 ceph-mon[32398]: pgmap v619: 129 pgs: 129 active+clean; 67 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 289 KiB/s wr, 59 op/s 2024-06-10T01:12:15.734 INFO:tasks.workunit.client.0.smithi007.stdout: 1 372750 6.08 MB/sec execute 176 sec latency 11.143 ms 2024-06-10T01:12:16.734 INFO:tasks.workunit.client.0.smithi007.stdout: 1 374770 6.09 MB/sec execute 177 sec latency 14.323 ms 2024-06-10T01:12:17.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:16 smithi078 ceph-mon[36683]: pgmap v620: 129 pgs: 129 active+clean; 65 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 583 KiB/s wr, 137 op/s 2024-06-10T01:12:17.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:16 smithi175 ceph-mon[36636]: pgmap v620: 129 pgs: 129 active+clean; 65 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 583 KiB/s wr, 137 op/s 2024-06-10T01:12:17.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:16 smithi007 ceph-mon[32398]: pgmap v620: 129 pgs: 129 active+clean; 65 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 583 KiB/s wr, 137 op/s 2024-06-10T01:12:17.734 INFO:tasks.workunit.client.0.smithi007.stdout: 1 376562 6.12 MB/sec execute 178 sec latency 11.421 ms 2024-06-10T01:12:18.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:17 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:12:18.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:17 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:12:18.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:17 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:12:18.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:17 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:12:18.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:17 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:12:18.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:17 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:12:18.734 INFO:tasks.workunit.client.0.smithi007.stdout: 1 379362 6.17 MB/sec execute 179 sec latency 11.329 ms 2024-06-10T01:12:19.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:18 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:12:19.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:18 smithi078 ceph-mon[36683]: pgmap v621: 129 pgs: 129 active+clean; 66 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 633 KiB/s wr, 150 op/s 2024-06-10T01:12:19.104 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:18 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:12:19.104 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:18 smithi175 ceph-mon[36636]: pgmap v621: 129 pgs: 129 active+clean; 66 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 633 KiB/s wr, 150 op/s 2024-06-10T01:12:19.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:18 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:12:19.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:18 smithi007 ceph-mon[32398]: pgmap v621: 129 pgs: 129 active+clean; 66 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 633 KiB/s wr, 150 op/s 2024-06-10T01:12:19.734 INFO:tasks.workunit.client.0.smithi007.stdout: 1 380622 6.17 MB/sec execute 180 sec latency 18.643 ms 2024-06-10T01:12:20.734 INFO:tasks.workunit.client.0.smithi007.stdout: 1 383353 6.23 MB/sec execute 181 sec latency 11.379 ms 2024-06-10T01:12:21.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:20 smithi078 ceph-mon[36683]: pgmap v622: 129 pgs: 129 active+clean; 70 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.0 MiB/s wr, 248 op/s 2024-06-10T01:12:21.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:20 smithi175 ceph-mon[36636]: pgmap v622: 129 pgs: 129 active+clean; 70 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.0 MiB/s wr, 248 op/s 2024-06-10T01:12:21.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:20 smithi007 ceph-mon[32398]: pgmap v622: 129 pgs: 129 active+clean; 70 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.0 MiB/s wr, 248 op/s 2024-06-10T01:12:21.734 INFO:tasks.workunit.client.0.smithi007.stdout: 1 384799 6.23 MB/sec execute 182 sec latency 29.799 ms 2024-06-10T01:12:22.734 INFO:tasks.workunit.client.0.smithi007.stdout: 1 387116 6.27 MB/sec execute 183 sec latency 21.586 ms 2024-06-10T01:12:23.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:22 smithi078 ceph-mon[36683]: pgmap v623: 129 pgs: 129 active+clean; 70 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 244 op/s 2024-06-10T01:12:23.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:22 smithi175 ceph-mon[36636]: pgmap v623: 129 pgs: 129 active+clean; 70 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 244 op/s 2024-06-10T01:12:23.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:22 smithi007 ceph-mon[32398]: pgmap v623: 129 pgs: 129 active+clean; 70 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 244 op/s 2024-06-10T01:12:23.734 INFO:tasks.workunit.client.0.smithi007.stdout: 1 389692 6.30 MB/sec execute 184 sec latency 11.105 ms 2024-06-10T01:12:24.734 INFO:tasks.workunit.client.0.smithi007.stdout: 1 391257 6.32 MB/sec execute 185 sec latency 24.304 ms 2024-06-10T01:12:25.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:24 smithi078 ceph-mon[36683]: pgmap v624: 129 pgs: 129 active+clean; 72 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.2 MiB/s wr, 262 op/s 2024-06-10T01:12:25.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:24 smithi175 ceph-mon[36636]: pgmap v624: 129 pgs: 129 active+clean; 72 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.2 MiB/s wr, 262 op/s 2024-06-10T01:12:25.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:24 smithi007 ceph-mon[32398]: pgmap v624: 129 pgs: 129 active+clean; 72 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.2 MiB/s wr, 262 op/s 2024-06-10T01:12:25.734 INFO:tasks.workunit.client.0.smithi007.stdout: 1 394008 6.37 MB/sec execute 186 sec latency 10.237 ms 2024-06-10T01:12:26.734 INFO:tasks.workunit.client.0.smithi007.stdout: 1 395543 6.37 MB/sec execute 187 sec latency 23.899 ms 2024-06-10T01:12:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.972+0000 7f1c8d288640 1 -- 172.21.15.7:0/3783979696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c881098c0 msgr2=0x7f1c8810bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.972+0000 7f1c8d288640 1 --2- 172.21.15.7:0/3783979696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c881098c0 0x7f1c8810bd50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f1c80009ca0 tx=0x7f1c8002a950 comp rx=0 tx=0).stop 2024-06-10T01:12:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.972+0000 7f1c8d288640 1 -- 172.21.15.7:0/3783979696 shutdown_connections 2024-06-10T01:12:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.972+0000 7f1c8d288640 1 --2- 172.21.15.7:0/3783979696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c881098c0 0x7f1c8810bd50 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.972+0000 7f1c8d288640 1 --2- 172.21.15.7:0/3783979696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c88074380 0x7f1c880747e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.972+0000 7f1c8d288640 1 --2- 172.21.15.7:0/3783979696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c88075960 0x7f1c88073db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.972+0000 7f1c8d288640 1 -- 172.21.15.7:0/3783979696 >> 172.21.15.7:0/3783979696 conn(0x7f1c880fcfc0 msgr2=0x7f1c880ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.972+0000 7f1c8d288640 1 -- 172.21.15.7:0/3783979696 shutdown_connections 2024-06-10T01:12:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.972+0000 7f1c8d288640 1 -- 172.21.15.7:0/3783979696 wait complete. 2024-06-10T01:12:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.972+0000 7f1c8d288640 1 Processor -- start 2024-06-10T01:12:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.973+0000 7f1c8d288640 1 -- start start 2024-06-10T01:12:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.973+0000 7f1c8d288640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c88074380 0x7f1c88199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.973+0000 7f1c8d288640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c88075960 0x7f1c8819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.973+0000 7f1c8d288640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c881098c0 0x7f1c881a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.973+0000 7f1c8d288640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8810f880 con 0x7f1c88075960 2024-06-10T01:12:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.973+0000 7f1c8d288640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8810f700 con 0x7f1c881098c0 2024-06-10T01:12:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.973+0000 7f1c8d288640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8810fa00 con 0x7f1c88074380 2024-06-10T01:12:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.973+0000 7f1c867fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c88075960 0x7f1c8819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.973+0000 7f1c867fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c88075960 0x7f1c8819a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50922/0 (socket says 172.21.15.7:50922) 2024-06-10T01:12:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.973+0000 7f1c867fc640 1 -- 172.21.15.7:0/2809509945 learned_addr learned my addr 172.21.15.7:0/2809509945 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.973+0000 7f1c86ffd640 1 --2- 172.21.15.7:0/2809509945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c88074380 0x7f1c88199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.973+0000 7f1c877fe640 1 --2- 172.21.15.7:0/2809509945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c881098c0 0x7f1c881a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.973+0000 7f1c867fc640 1 -- 172.21.15.7:0/2809509945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c88074380 msgr2=0x7f1c88199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.973+0000 7f1c867fc640 1 --2- 172.21.15.7:0/2809509945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c88074380 0x7f1c88199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.973+0000 7f1c867fc640 1 -- 172.21.15.7:0/2809509945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c881098c0 msgr2=0x7f1c881a1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.973+0000 7f1c867fc640 1 --2- 172.21.15.7:0/2809509945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c881098c0 0x7f1c881a1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.973+0000 7f1c867fc640 1 -- 172.21.15.7:0/2809509945 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c881a1ab0 con 0x7f1c88075960 2024-06-10T01:12:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.973+0000 7f1c877fe640 1 --2- 172.21.15.7:0/2809509945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c881098c0 0x7f1c881a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:12:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.973+0000 7f1c86ffd640 1 --2- 172.21.15.7:0/2809509945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c88074380 0x7f1c88199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.974+0000 7f1c867fc640 1 --2- 172.21.15.7:0/2809509945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c88075960 0x7f1c8819a3d0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c00e9f0 tx=0x7f1c7c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.974+0000 7f1c6ffff640 1 -- 172.21.15.7:0/2809509945 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c00cd80 con 0x7f1c88075960 2024-06-10T01:12:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.974+0000 7f1c8d288640 1 -- 172.21.15.7:0/2809509945 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c881a1da0 con 0x7f1c88075960 2024-06-10T01:12:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.974+0000 7f1c6ffff640 1 -- 172.21.15.7:0/2809509945 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c7c010910 con 0x7f1c88075960 2024-06-10T01:12:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.974+0000 7f1c6ffff640 1 -- 172.21.15.7:0/2809509945 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c0188f0 con 0x7f1c88075960 2024-06-10T01:12:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.974+0000 7f1c8d288640 1 -- 172.21.15.7:0/2809509945 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c881a2270 con 0x7f1c88075960 2024-06-10T01:12:26.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.976+0000 7f1c6ffff640 1 -- 172.21.15.7:0/2809509945 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f1c7c010ac0 con 0x7f1c88075960 2024-06-10T01:12:26.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.976+0000 7f1c8ff94740 1 Processor -- start 2024-06-10T01:12:26.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.976+0000 7f1c8ff94740 1 -- start start 2024-06-10T01:12:26.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.976+0000 7f1c8ff94740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f56f59b010 0x55f56f59b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:26.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.976+0000 7f1c8ff94740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f56f59ba00 0x55f56f5a0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:26.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.976+0000 7f1c8ff94740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f56f5a0b70 0x55f56f5a3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:26.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.976+0000 7f1c8ff94740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f56f4fe4f0 con 0x55f56f59ba00 2024-06-10T01:12:26.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.976+0000 7f1c8ff94740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f56f49c870 con 0x55f56f5a0b70 2024-06-10T01:12:26.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.976+0000 7f1c8ff94740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f56f4fa4b0 con 0x55f56f59b010 2024-06-10T01:12:26.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.977+0000 7f1c86ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f56f59ba00 0x55f56f5a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:26.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.977+0000 7f1c86ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f56f59ba00 0x55f56f5a0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50924/0 (socket says 172.21.15.7:50924) 2024-06-10T01:12:26.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.977+0000 7f1c86ffd640 1 -- 172.21.15.7:0/3737913876 learned_addr learned my addr 172.21.15.7:0/3737913876 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:26.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.977+0000 7f1c877fe640 1 --2- 172.21.15.7:0/3737913876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f56f59b010 0x55f56f59b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:26.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.977+0000 7f1c867fc640 1 --2- 172.21.15.7:0/3737913876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f56f5a0b70 0x55f56f5a3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:26.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.977+0000 7f1c86ffd640 1 -- 172.21.15.7:0/3737913876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f56f59b010 msgr2=0x55f56f59b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:26.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.977+0000 7f1c86ffd640 1 --2- 172.21.15.7:0/3737913876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f56f59b010 0x55f56f59b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:26.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.977+0000 7f1c86ffd640 1 -- 172.21.15.7:0/3737913876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f56f5a0b70 msgr2=0x55f56f5a3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:26.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.977+0000 7f1c86ffd640 1 --2- 172.21.15.7:0/3737913876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f56f5a0b70 0x55f56f5a3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:26.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.977+0000 7f1c86ffd640 1 -- 172.21.15.7:0/3737913876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f56f4b89b0 con 0x55f56f59ba00 2024-06-10T01:12:26.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.977+0000 7f1c6ffff640 1 --2- 172.21.15.7:0/2809509945 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1c700a2b80 0x7f1c700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:26.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.977+0000 7f1c86ffd640 1 --2- 172.21.15.7:0/3737913876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f56f59ba00 0x55f56f5a0500 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f1c780115f0 tx=0x7f1c78011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff3859d61c46936f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:26.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.977+0000 7f1c877fe640 1 --2- 172.21.15.7:0/2809509945 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1c700a2b80 0x7f1c700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:26.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.977+0000 7f1c6ffff640 1 -- 172.21.15.7:0/2809509945 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f1c7c0cf850 con 0x7f1c88075960 2024-06-10T01:12:26.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.977+0000 7f1c6d7fa640 1 -- 172.21.15.7:0/3737913876 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7801e070 con 0x55f56f59ba00 2024-06-10T01:12:26.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.977+0000 7f1c6d7fa640 1 -- 172.21.15.7:0/3737913876 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c7800fb20 con 0x55f56f59ba00 2024-06-10T01:12:26.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.977+0000 7f1c6d7fa640 1 -- 172.21.15.7:0/3737913876 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c78019bf0 con 0x55f56f59ba00 2024-06-10T01:12:26.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.978+0000 7f1c8ff94740 1 -- 172.21.15.7:0/3737913876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f56f59ba00 msgr2=0x55f56f5a0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:26.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.978+0000 7f1c877fe640 1 --2- 172.21.15.7:0/2809509945 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1c700a2b80 0x7f1c700a5040 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1c80004af0 tx=0x7f1c8003c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:26.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.978+0000 7f1c8ff94740 1 --2- 172.21.15.7:0/3737913876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f56f59ba00 0x55f56f5a0500 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f1c780115f0 tx=0x7f1c78011ab0 comp rx=0 tx=0).stop 2024-06-10T01:12:26.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.978+0000 7f1c8ff94740 1 -- 172.21.15.7:0/3737913876 shutdown_connections 2024-06-10T01:12:26.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.978+0000 7f1c8ff94740 1 --2- 172.21.15.7:0/3737913876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f56f5a0b70 0x55f56f5a3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:26.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.978+0000 7f1c8ff94740 1 --2- 172.21.15.7:0/3737913876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f56f59ba00 0x55f56f5a0500 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:26.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.978+0000 7f1c8ff94740 1 --2- 172.21.15.7:0/3737913876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f56f59b010 0x55f56f59b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:26.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.978+0000 7f1c8ff94740 1 -- 172.21.15.7:0/3737913876 >> 172.21.15.7:0/3737913876 conn(0x55f56f596680 msgr2=0x55f56f596a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:26.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.978+0000 7f1c8ff94740 1 -- 172.21.15.7:0/3737913876 shutdown_connections 2024-06-10T01:12:26.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.978+0000 7f1c8ff94740 1 -- 172.21.15.7:0/3737913876 wait complete. 2024-06-10T01:12:26.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.979+0000 7f1c8ff94740 1 -- start start 2024-06-10T01:12:26.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.979+0000 7f1c8ff94740 1 Processor -- start 2024-06-10T01:12:26.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.979+0000 7f1c8ff94740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f56f59b010 0x55f56f5a5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:26.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.979+0000 7f1c8ff94740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f56f59ba00 0x55f56f5a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:26.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.979+0000 7f1c8ff94740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f56f5a0b70 0x55f56f5ac810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:26.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.979+0000 7f1c8ff94740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f56f5a3b20 con 0x55f56f59b010 2024-06-10T01:12:26.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.979+0000 7f1c8ff94740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f56f487ae0 con 0x55f56f5a0b70 2024-06-10T01:12:26.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.979+0000 7f1c8ff94740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f56f5a3ca0 con 0x55f56f59ba00 2024-06-10T01:12:26.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.979+0000 7f1c86ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f56f59b010 0x55f56f5a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:26.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.979+0000 7f1c8ff94740 10 client.0 _ll_register_callbacks cb 0x55f56f4c09b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:12:26.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.979+0000 7f1c86ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f56f59b010 0x55f56f5a5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50938/0 (socket says 172.21.15.7:50938) 2024-06-10T01:12:26.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.979+0000 7f1c86ffd640 1 -- 172.21.15.7:0/693697836 learned_addr learned my addr 172.21.15.7:0/693697836 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.979+0000 7f1c867fc640 1 --2- 172.21.15.7:0/693697836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f56f59ba00 0x55f56f5a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.979+0000 7f1c877fe640 1 --2- 172.21.15.7:0/693697836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f56f5a0b70 0x55f56f5ac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.979+0000 7f1c86ffd640 1 -- 172.21.15.7:0/693697836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f56f59ba00 msgr2=0x55f56f5a57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.979+0000 7f1c86ffd640 1 --2- 172.21.15.7:0/693697836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f56f59ba00 0x55f56f5a57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.979+0000 7f1c86ffd640 1 -- 172.21.15.7:0/693697836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f56f5a0b70 msgr2=0x55f56f5ac810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.979+0000 7f1c86ffd640 1 --2- 172.21.15.7:0/693697836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f56f5a0b70 0x55f56f5ac810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.980+0000 7f1c86ffd640 1 -- 172.21.15.7:0/693697836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f56f212210 con 0x55f56f59b010 2024-06-10T01:12:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.980+0000 7f1c877fe640 1 --2- 172.21.15.7:0/693697836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f56f5a0b70 0x55f56f5ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:12:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.980+0000 7f1c86ffd640 1 --2- 172.21.15.7:0/693697836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f56f59b010 0x55f56f5a5290 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f1c78005890 tx=0x7f1c7800f8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43958a650f0cfc1e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.980+0000 7f1c6aff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:12:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.980+0000 7f1c6aff5640 1 -- 172.21.15.7:0/693697836 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7801e040 con 0x55f56f59b010 2024-06-10T01:12:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.980+0000 7f1c6aff5640 1 -- 172.21.15.7:0/693697836 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f1c78004040 con 0x55f56f59b010 2024-06-10T01:12:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.980+0000 7f1c8ff94740 1 -- 172.21.15.7:0/693697836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c7800fb20 con 0x55f56f59b010 2024-06-10T01:12:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.980+0000 7f1c6aff5640 1 -- 172.21.15.7:0/693697836 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1c7802f050 con 0x55f56f59b010 2024-06-10T01:12:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.980+0000 7f1c8ff94740 10 client.17127 fetch_fsmap learned FSMap version 32 2024-06-10T01:12:26.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.980+0000 7f1c8ff94740 1 -- 172.21.15.7:0/693697836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f56f2de620 con 0x55f56f59b010 2024-06-10T01:12:26.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.980+0000 7f1c6aff5640 1 -- 172.21.15.7:0/693697836 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c7800fb20 con 0x55f56f59b010 2024-06-10T01:12:26.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.980+0000 7f1c6aff5640 1 -- 172.21.15.7:0/693697836 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f1c7802a030 con 0x55f56f59b010 2024-06-10T01:12:26.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.980+0000 7f1c8ff94740 10 client.17127 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:12:26.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.981+0000 7f1c8ff94740 10 client.17127 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:12:26.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.981+0000 7f1c8ff94740 20 client.17127 populate_metadata read hostname 'smithi007' 2024-06-10T01:12:26.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.981+0000 7f1c8ff94740 1 --2- 172.21.15.7:0/693697836 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f56f5b0e20 0x55f56f5b3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:26.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.981+0000 7f1c8ff94740 4 client.17127 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:12:26.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.981+0000 7f1c8ff94740 1 -- 172.21.15.7:0/693697836 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f56f511f10 con 0x55f56f5b0e20 2024-06-10T01:12:26.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.981+0000 7f1c867fc640 1 --2- 172.21.15.7:0/693697836 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f56f5b0e20 0x55f56f5b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:26.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.981+0000 7f1c867fc640 1 --2- 172.21.15.7:0/693697836 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f56f5b0e20 0x55f56f5b3210 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ade7f682f8350a71 server_cookie=5cfe2ea0f359d41 in_seq=0 out_seq=0 2024-06-10T01:12:26.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.981+0000 7f1c6aff5640 10 client.17127 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:26.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.982+0000 7f1c6aff5640 1 -- 172.21.15.7:0/693697836 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f56f511f10 con 0x55f56f5b0e20 2024-06-10T01:12:26.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.982+0000 7f1c6aff5640 10 client.17127 handle_command_reply: tid=0 2024-06-10T01:12:26.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.982+0000 7f1c8ff94740 1 client.17127 shutdown 2024-06-10T01:12:26.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.982+0000 7f1c8ff94740 1 -- 172.21.15.7:0/693697836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f56f59b010 msgr2=0x55f56f5a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:26.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.982+0000 7f1c8ff94740 1 --2- 172.21.15.7:0/693697836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f56f59b010 0x55f56f5a5290 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f1c78005890 tx=0x7f1c7800f8f0 comp rx=0 tx=0).stop 2024-06-10T01:12:26.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.982+0000 7f1c8ff94740 1 -- 172.21.15.7:0/693697836 shutdown_connections 2024-06-10T01:12:26.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.982+0000 7f1c8ff94740 1 --2- 172.21.15.7:0/693697836 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f56f5b0e20 0x55f56f5b3210 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:26.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.982+0000 7f1c6aff5640 0 client.17127 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:26.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.982+0000 7f1c8ff94740 1 --2- 172.21.15.7:0/693697836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f56f5a0b70 0x55f56f5ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:26.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.982+0000 7f1c8ff94740 1 --2- 172.21.15.7:0/693697836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f56f59ba00 0x55f56f5a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:26.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.982+0000 7f1c8ff94740 1 --2- 172.21.15.7:0/693697836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f56f59b010 0x55f56f5a5290 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:26.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.982+0000 7f1c8ff94740 1 -- 172.21.15.7:0/693697836 >> 172.21.15.7:0/693697836 conn(0x55f56f596680 msgr2=0x55f56f5957e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:26.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.982+0000 7f1c8ff94740 1 -- 172.21.15.7:0/693697836 shutdown_connections 2024-06-10T01:12:26.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.982+0000 7f1c8ff94740 1 -- 172.21.15.7:0/693697836 wait complete. 2024-06-10T01:12:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.982+0000 7f1c8ff94740 20 client.17127 trim_cache size 0 max 16384 2024-06-10T01:12:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.999+0000 7f1c8ff94740 1 Processor -- start 2024-06-10T01:12:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.999+0000 7f1c8ff94740 1 -- start start 2024-06-10T01:12:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.999+0000 7f1c8ff94740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f56f59b010 0x55f56f599b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.999+0000 7f1c8ff94740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f56f59ba00 0x55f56f5ac7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.999+0000 7f1c8ff94740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f56f5a0b70 0x55f56f5a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.999+0000 7f1c8ff94740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1c881af170 con 0x55f56f59ba00 2024-06-10T01:12:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.999+0000 7f1c8ff94740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1c881aca40 con 0x55f56f59b010 2024-06-10T01:12:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:26.999+0000 7f1c8ff94740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1c881aa890 con 0x55f56f5a0b70 2024-06-10T01:12:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.000+0000 7f1c867fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f56f59ba00 0x55f56f5ac7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.000+0000 7f1c867fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f56f59ba00 0x55f56f5ac7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50948/0 (socket says 172.21.15.7:50948) 2024-06-10T01:12:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.000+0000 7f1c867fc640 1 -- 172.21.15.7:0/688749765 learned_addr learned my addr 172.21.15.7:0/688749765 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.000+0000 7f1c877fe640 1 --2- 172.21.15.7:0/688749765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f56f5a0b70 0x55f56f5a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.000+0000 7f1c86ffd640 1 --2- 172.21.15.7:0/688749765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f56f59b010 0x55f56f599b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.000+0000 7f1c867fc640 1 -- 172.21.15.7:0/688749765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f56f5a0b70 msgr2=0x55f56f5a48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.000+0000 7f1c867fc640 1 --2- 172.21.15.7:0/688749765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f56f5a0b70 0x55f56f5a48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.000+0000 7f1c867fc640 1 -- 172.21.15.7:0/688749765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f56f59b010 msgr2=0x55f56f599b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.000+0000 7f1c867fc640 1 --2- 172.21.15.7:0/688749765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f56f59b010 0x55f56f599b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.000+0000 7f1c867fc640 1 -- 172.21.15.7:0/688749765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f56f59b7a0 con 0x55f56f59ba00 2024-06-10T01:12:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.000+0000 7f1c86ffd640 1 --2- 172.21.15.7:0/688749765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f56f59b010 0x55f56f599b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:12:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.000+0000 7f1c867fc640 1 --2- 172.21.15.7:0/688749765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f56f59ba00 0x55f56f5ac7d0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c098700 tx=0x7f1c7c0ccea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16659d8378fa2b23 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.000+0000 7f1c6cff9640 1 -- 172.21.15.7:0/688749765 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c0a49a0 con 0x55f56f59ba00 2024-06-10T01:12:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.000+0000 7f1c6cff9640 1 -- 172.21.15.7:0/688749765 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c7c099dd0 con 0x55f56f59ba00 2024-06-10T01:12:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.000+0000 7f1c6cff9640 1 -- 172.21.15.7:0/688749765 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c09b4b0 con 0x55f56f59ba00 2024-06-10T01:12:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.001+0000 7f1c8ff94740 1 -- 172.21.15.7:0/688749765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f56f59ba00 msgr2=0x55f56f5ac7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.001+0000 7f1c8ff94740 1 --2- 172.21.15.7:0/688749765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f56f59ba00 0x55f56f5ac7d0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c098700 tx=0x7f1c7c0ccea0 comp rx=0 tx=0).stop 2024-06-10T01:12:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.001+0000 7f1c8ff94740 1 -- 172.21.15.7:0/688749765 shutdown_connections 2024-06-10T01:12:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.001+0000 7f1c8ff94740 1 --2- 172.21.15.7:0/688749765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f56f5a0b70 0x55f56f5a48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.001+0000 7f1c8ff94740 1 --2- 172.21.15.7:0/688749765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f56f59ba00 0x55f56f5ac7d0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.001+0000 7f1c8ff94740 1 --2- 172.21.15.7:0/688749765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f56f59b010 0x55f56f599b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.001+0000 7f1c8ff94740 1 -- 172.21.15.7:0/688749765 >> 172.21.15.7:0/688749765 conn(0x55f56f596680 msgr2=0x55f56f599420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.001+0000 7f1c8ff94740 1 -- 172.21.15.7:0/688749765 shutdown_connections 2024-06-10T01:12:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.001+0000 7f1c8ff94740 1 -- 172.21.15.7:0/688749765 wait complete. 2024-06-10T01:12:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.001+0000 7f1c8ff94740 1 -- start start 2024-06-10T01:12:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.001+0000 7f1c8ff94740 1 Processor -- start 2024-06-10T01:12:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.002+0000 7f1c8ff94740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f56f59b010 0x55f56f5a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.002+0000 7f1c8ff94740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f56f59ba00 0x55f56f5a99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.002+0000 7f1c8ff94740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f56f5a0b70 0x55f56f619ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.002+0000 7f1c8ff94740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1c881aaf00 con 0x55f56f5a0b70 2024-06-10T01:12:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.002+0000 7f1c8ff94740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1c881ac370 con 0x55f56f59b010 2024-06-10T01:12:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.002+0000 7f1c8ff94740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1c881ab5d0 con 0x55f56f59ba00 2024-06-10T01:12:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.002+0000 7f1c8ff94740 10 client.0 _ll_register_callbacks cb 0x7f1c7801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:12:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.002+0000 7f1c877fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f56f5a0b70 0x55f56f619ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.002+0000 7f1c867fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f56f59ba00 0x55f56f5a99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.002+0000 7f1c867fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f56f59ba00 0x55f56f5a99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43074/0 (socket says 172.21.15.7:43074) 2024-06-10T01:12:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.002+0000 7f1c877fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f56f5a0b70 0x55f56f619ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50954/0 (socket says 172.21.15.7:50954) 2024-06-10T01:12:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.002+0000 7f1c867fc640 1 -- 172.21.15.7:0/1156474057 learned_addr learned my addr 172.21.15.7:0/1156474057 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.002+0000 7f1c86ffd640 1 --2- 172.21.15.7:0/1156474057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f56f59b010 0x55f56f5a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.002+0000 7f1c877fe640 1 -- 172.21.15.7:0/1156474057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f56f59ba00 msgr2=0x55f56f5a99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.002+0000 7f1c877fe640 1 --2- 172.21.15.7:0/1156474057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f56f59ba00 0x55f56f5a99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.002+0000 7f1c877fe640 1 -- 172.21.15.7:0/1156474057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f56f59b010 msgr2=0x55f56f5a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.002+0000 7f1c877fe640 1 --2- 172.21.15.7:0/1156474057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f56f59b010 0x55f56f5a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.002+0000 7f1c877fe640 1 -- 172.21.15.7:0/1156474057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f56f5a9ff0 con 0x55f56f5a0b70 2024-06-10T01:12:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.002+0000 7f1c867fc640 1 --2- 172.21.15.7:0/1156474057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f56f59ba00 0x55f56f5a99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.002+0000 7f1c877fe640 1 --2- 172.21.15.7:0/1156474057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f56f5a0b70 0x55f56f619ae0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x55f56f59c260 tx=0x7f1c8002a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ebba9eeea34cd43 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.002+0000 7f1c6aff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:12:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.002+0000 7f1c6aff5640 1 -- 172.21.15.7:0/1156474057 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c80034050 con 0x55f56f5a0b70 2024-06-10T01:12:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.002+0000 7f1c6aff5640 1 -- 172.21.15.7:0/1156474057 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f1c80004250 con 0x55f56f5a0b70 2024-06-10T01:12:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.002+0000 7f1c8ff94740 1 -- 172.21.15.7:0/1156474057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c7c099dd0 con 0x55f56f5a0b70 2024-06-10T01:12:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.003+0000 7f1c6aff5640 1 -- 172.21.15.7:0/1156474057 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1c80004450 con 0x55f56f5a0b70 2024-06-10T01:12:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.003+0000 7f1c8ff94740 10 client.17133 fetch_fsmap learned FSMap version 32 2024-06-10T01:12:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.003+0000 7f1c8ff94740 1 -- 172.21.15.7:0/1156474057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f56f61b130 con 0x55f56f5a0b70 2024-06-10T01:12:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.003+0000 7f1c6aff5640 1 -- 172.21.15.7:0/1156474057 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c7c099dd0 con 0x55f56f5a0b70 2024-06-10T01:12:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.003+0000 7f1c6aff5640 1 -- 172.21.15.7:0/1156474057 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f1c80044070 con 0x55f56f5a0b70 2024-06-10T01:12:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.003+0000 7f1c8ff94740 10 client.17133 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:12:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.003+0000 7f1c8ff94740 10 client.17133 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:12:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.003+0000 7f1c8ff94740 20 client.17133 populate_metadata read hostname 'smithi007' 2024-06-10T01:12:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.003+0000 7f1c8ff94740 1 --2- 172.21.15.7:0/1156474057 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f56f5b0e20 0x55f56f61dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.003+0000 7f1c8ff94740 4 client.17133 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:12:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.003+0000 7f1c8ff94740 1 -- 172.21.15.7:0/1156474057 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f56f50bc20 con 0x55f56f5b0e20 2024-06-10T01:12:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.003+0000 7f1c86ffd640 1 --2- 172.21.15.7:0/1156474057 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f56f5b0e20 0x55f56f61dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.004+0000 7f1c6aff5640 10 client.17133 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.004+0000 7f1c86ffd640 1 --2- 172.21.15.7:0/1156474057 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f56f5b0e20 0x55f56f61dba0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a64158fa7f6e9fa3 server_cookie=c3b2a7ff7d9fa795 in_seq=0 out_seq=0 2024-06-10T01:12:27.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:26 smithi078 ceph-mon[36683]: pgmap v625: 129 pgs: 129 active+clean; 74 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 281 op/s 2024-06-10T01:12:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.108+0000 7f1c6aff5640 1 -- 172.21.15.7:0/1156474057 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f56f50bc20 con 0x55f56f5b0e20 2024-06-10T01:12:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.108+0000 7f1c6aff5640 10 client.17133 handle_command_reply: tid=0 2024-06-10T01:12:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.108+0000 7f1c8ff94740 1 client.17133 shutdown 2024-06-10T01:12:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.108+0000 7f1c8ff94740 1 -- 172.21.15.7:0/1156474057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f56f5a0b70 msgr2=0x55f56f619ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.108+0000 7f1c8ff94740 1 --2- 172.21.15.7:0/1156474057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f56f5a0b70 0x55f56f619ae0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x55f56f59c260 tx=0x7f1c8002a950 comp rx=0 tx=0).stop 2024-06-10T01:12:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.108+0000 7f1c8ff94740 1 -- 172.21.15.7:0/1156474057 shutdown_connections 2024-06-10T01:12:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.108+0000 7f1c8ff94740 1 --2- 172.21.15.7:0/1156474057 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f56f5b0e20 0x55f56f61dba0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.108+0000 7f1c8ff94740 1 --2- 172.21.15.7:0/1156474057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f56f5a0b70 0x55f56f619ae0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.108+0000 7f1c6aff5640 0 client.17133 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.108+0000 7f1c8ff94740 1 --2- 172.21.15.7:0/1156474057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f56f59ba00 0x55f56f5a99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.108+0000 7f1c8ff94740 1 --2- 172.21.15.7:0/1156474057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f56f59b010 0x55f56f5a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.108+0000 7f1c8ff94740 1 -- 172.21.15.7:0/1156474057 >> 172.21.15.7:0/1156474057 conn(0x55f56f596680 msgr2=0x55f56f599cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.108+0000 7f1c8ff94740 1 -- 172.21.15.7:0/1156474057 shutdown_connections 2024-06-10T01:12:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.108+0000 7f1c8ff94740 1 -- 172.21.15.7:0/1156474057 wait complete. 2024-06-10T01:12:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.108+0000 7f1c8ff94740 20 client.17133 trim_cache size 0 max 16384 2024-06-10T01:12:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:12:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:12:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:12:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.109+0000 7f1c8d288640 1 -- 172.21.15.7:0/2809509945 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1c700a2b80 msgr2=0x7f1c700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.109+0000 7f1c8d288640 1 --2- 172.21.15.7:0/2809509945 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1c700a2b80 0x7f1c700a5040 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1c80004af0 tx=0x7f1c8003c040 comp rx=0 tx=0).stop 2024-06-10T01:12:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.109+0000 7f1c8d288640 1 -- 172.21.15.7:0/2809509945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c88075960 msgr2=0x7f1c8819a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.109+0000 7f1c8d288640 1 --2- 172.21.15.7:0/2809509945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c88075960 0x7f1c8819a3d0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c00e9f0 tx=0x7f1c7c00eeb0 comp rx=0 tx=0).stop 2024-06-10T01:12:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.109+0000 7f1c8d288640 1 -- 172.21.15.7:0/2809509945 shutdown_connections 2024-06-10T01:12:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.109+0000 7f1c8d288640 1 --2- 172.21.15.7:0/2809509945 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1c700a2b80 0x7f1c700a5040 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.109+0000 7f1c8d288640 1 --2- 172.21.15.7:0/2809509945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c881098c0 0x7f1c881a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.109+0000 7f1c8d288640 1 --2- 172.21.15.7:0/2809509945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c88075960 0x7f1c8819a3d0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.109+0000 7f1c8d288640 1 --2- 172.21.15.7:0/2809509945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c88074380 0x7f1c88199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.109+0000 7f1c8d288640 1 -- 172.21.15.7:0/2809509945 >> 172.21.15.7:0/2809509945 conn(0x7f1c880fcfc0 msgr2=0x7f1c88105850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.109+0000 7f1c8d288640 1 -- 172.21.15.7:0/2809509945 shutdown_connections 2024-06-10T01:12:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:27.109+0000 7f1c8d288640 1 -- 172.21.15.7:0/2809509945 wait complete. 2024-06-10T01:12:27.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:26 smithi175 ceph-mon[36636]: pgmap v625: 129 pgs: 129 active+clean; 74 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 281 op/s 2024-06-10T01:12:27.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:26 smithi007 ceph-mon[32398]: pgmap v625: 129 pgs: 129 active+clean; 74 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 281 op/s 2024-06-10T01:12:27.992 INFO:tasks.workunit.client.0.smithi007.stdout: 1 397728 6.41 MB/sec execute 188 sec latency 63.091 ms 2024-06-10T01:12:28.735 INFO:tasks.workunit.client.0.smithi007.stdout: 1 400224 6.44 MB/sec execute 189 sec latency 11.705 ms 2024-06-10T01:12:29.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:28 smithi078 ceph-mon[36683]: pgmap v626: 129 pgs: 129 active+clean; 77 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.2 MiB/s wr, 259 op/s 2024-06-10T01:12:29.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:28 smithi175 ceph-mon[36636]: pgmap v626: 129 pgs: 129 active+clean; 77 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.2 MiB/s wr, 259 op/s 2024-06-10T01:12:29.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:28 smithi007 ceph-mon[32398]: pgmap v626: 129 pgs: 129 active+clean; 77 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.2 MiB/s wr, 259 op/s 2024-06-10T01:12:29.734 INFO:tasks.workunit.client.0.smithi007.stdout: 1 401745 6.44 MB/sec execute 190 sec latency 13.284 ms 2024-06-10T01:12:30.735 INFO:tasks.workunit.client.0.smithi007.stdout: 1 404604 6.51 MB/sec execute 191 sec latency 17.539 ms 2024-06-10T01:12:31.064 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:30 smithi007 ceph-mon[32398]: pgmap v627: 129 pgs: 129 active+clean; 75 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.4 MiB/s wr, 316 op/s 2024-06-10T01:12:31.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:30 smithi078 ceph-mon[36683]: pgmap v627: 129 pgs: 129 active+clean; 75 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.4 MiB/s wr, 316 op/s 2024-06-10T01:12:31.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:30 smithi175 ceph-mon[36636]: pgmap v627: 129 pgs: 129 active+clean; 75 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.4 MiB/s wr, 316 op/s 2024-06-10T01:12:31.735 INFO:tasks.workunit.client.0.smithi007.stdout: 1 405930 6.50 MB/sec execute 192 sec latency 18.935 ms 2024-06-10T01:12:32.735 INFO:tasks.workunit.client.0.smithi007.stdout: 1 408316 6.54 MB/sec execute 193 sec latency 18.636 ms 2024-06-10T01:12:33.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:32 smithi078 ceph-mon[36683]: pgmap v628: 129 pgs: 129 active+clean; 77 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.2 MiB/s wr, 289 op/s 2024-06-10T01:12:33.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:32 smithi175 ceph-mon[36636]: pgmap v628: 129 pgs: 129 active+clean; 77 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.2 MiB/s wr, 289 op/s 2024-06-10T01:12:33.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:32 smithi007 ceph-mon[32398]: pgmap v628: 129 pgs: 129 active+clean; 77 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.2 MiB/s wr, 289 op/s 2024-06-10T01:12:33.735 INFO:tasks.workunit.client.0.smithi007.stdout: 1 410391 6.54 MB/sec execute 194 sec latency 11.806 ms 2024-06-10T01:12:34.735 INFO:tasks.workunit.client.0.smithi007.stdout: 1 412100 6.57 MB/sec execute 195 sec latency 15.260 ms 2024-06-10T01:12:35.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:34 smithi078 ceph-mon[36683]: pgmap v629: 129 pgs: 129 active+clean; 78 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.3 MiB/s wr, 288 op/s 2024-06-10T01:12:35.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:34 smithi175 ceph-mon[36636]: pgmap v629: 129 pgs: 129 active+clean; 78 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.3 MiB/s wr, 288 op/s 2024-06-10T01:12:35.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:34 smithi007 ceph-mon[32398]: pgmap v629: 129 pgs: 129 active+clean; 78 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.3 MiB/s wr, 288 op/s 2024-06-10T01:12:35.735 INFO:tasks.workunit.client.0.smithi007.stdout: 1 414842 6.61 MB/sec execute 196 sec latency 9.573 ms 2024-06-10T01:12:36.735 INFO:tasks.workunit.client.0.smithi007.stdout: 1 416112 6.61 MB/sec execute 197 sec latency 26.681 ms 2024-06-10T01:12:37.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:36 smithi078 ceph-mon[36683]: pgmap v630: 129 pgs: 129 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 304 op/s 2024-06-10T01:12:37.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:12:37.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:36 smithi175 ceph-mon[36636]: pgmap v630: 129 pgs: 129 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 304 op/s 2024-06-10T01:12:37.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:12:37.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:36 smithi007 ceph-mon[32398]: pgmap v630: 129 pgs: 129 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 304 op/s 2024-06-10T01:12:37.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:12:37.735 INFO:tasks.workunit.client.0.smithi007.stdout: 1 418807 6.66 MB/sec execute 198 sec latency 15.802 ms 2024-06-10T01:12:38.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:12:38.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:12:38.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:37 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:12:38.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:12:38.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:12:38.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:37 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:12:38.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:12:38.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:12:38.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:37 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:12:38.735 INFO:tasks.workunit.client.0.smithi007.stdout: 1 420142 6.66 MB/sec execute 199 sec latency 13.443 ms 2024-06-10T01:12:39.579 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:39 smithi078 ceph-mon[36683]: pgmap v631: 129 pgs: 129 active+clean; 83 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.3 MiB/s wr, 313 op/s 2024-06-10T01:12:39.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:39 smithi175 ceph-mon[36636]: pgmap v631: 129 pgs: 129 active+clean; 83 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.3 MiB/s wr, 313 op/s 2024-06-10T01:12:39.735 INFO:tasks.workunit.client.0.smithi007.stdout: 1 422506 6.69 MB/sec execute 200 sec latency 17.022 ms 2024-06-10T01:12:39.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:39 smithi007 ceph-mon[32398]: pgmap v631: 129 pgs: 129 active+clean; 83 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.3 MiB/s wr, 313 op/s 2024-06-10T01:12:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.210+0000 7f155cb89640 1 -- 172.21.15.7:0/1962344176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1558103c30 msgr2=0x7f1558104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.210+0000 7f155cb89640 1 --2- 172.21.15.7:0/1962344176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1558103c30 0x7f1558104050 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f1548009900 tx=0x7f154802a410 comp rx=0 tx=0).stop 2024-06-10T01:12:40.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.210+0000 7f155cb89640 1 -- 172.21.15.7:0/1962344176 shutdown_connections 2024-06-10T01:12:40.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.210+0000 7f155cb89640 1 --2- 172.21.15.7:0/1962344176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15581057e0 0x7f155810c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.211+0000 7f155cb89640 1 --2- 172.21.15.7:0/1962344176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1558104e20 0x7f15581052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.211+0000 7f155cb89640 1 --2- 172.21.15.7:0/1962344176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1558103c30 0x7f1558104050 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.211+0000 7f155cb89640 1 -- 172.21.15.7:0/1962344176 >> 172.21.15.7:0/1962344176 conn(0x7f15580ff1c0 msgr2=0x7f15581015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:40.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.211+0000 7f155cb89640 1 -- 172.21.15.7:0/1962344176 shutdown_connections 2024-06-10T01:12:40.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.211+0000 7f155cb89640 1 -- 172.21.15.7:0/1962344176 wait complete. 2024-06-10T01:12:40.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.211+0000 7f155cb89640 1 Processor -- start 2024-06-10T01:12:40.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.211+0000 7f155cb89640 1 -- start start 2024-06-10T01:12:40.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.211+0000 7f155cb89640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1558103c30 0x7f1558199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:40.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.211+0000 7f155cb89640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1558104e20 0x7f155819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:40.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.211+0000 7f155cb89640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15581057e0 0x7f15581a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.212+0000 7f155cb89640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f155810f880 con 0x7f1558103c30 2024-06-10T01:12:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.212+0000 7f155cb89640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f155810f700 con 0x7f15581057e0 2024-06-10T01:12:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.212+0000 7f155cb89640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f155810fa00 con 0x7f1558104e20 2024-06-10T01:12:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.212+0000 7f155659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1558103c30 0x7f1558199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.212+0000 7f154ffff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1558104e20 0x7f155819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.212+0000 7f155659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1558103c30 0x7f1558199f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37704/0 (socket says 172.21.15.7:37704) 2024-06-10T01:12:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.212+0000 7f155659c640 1 -- 172.21.15.7:0/64311036 learned_addr learned my addr 172.21.15.7:0/64311036 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.212+0000 7f1556d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15581057e0 0x7f15581a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.212+0000 7f155659c640 1 -- 172.21.15.7:0/64311036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1558104e20 msgr2=0x7f155819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.212+0000 7f155659c640 1 --2- 172.21.15.7:0/64311036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1558104e20 0x7f155819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.212+0000 7f155659c640 1 -- 172.21.15.7:0/64311036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15581057e0 msgr2=0x7f15581a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.212+0000 7f155659c640 1 --2- 172.21.15.7:0/64311036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15581057e0 0x7f15581a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.212+0000 7f155659c640 1 -- 172.21.15.7:0/64311036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15581a1ba0 con 0x7f1558103c30 2024-06-10T01:12:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.212+0000 7f1556d9d640 1 --2- 172.21.15.7:0/64311036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15581057e0 0x7f15581a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.212+0000 7f155659c640 1 --2- 172.21.15.7:0/64311036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1558103c30 0x7f1558199f80 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1548031610 tx=0x7f15480043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.212+0000 7f154f7fe640 1 -- 172.21.15.7:0/64311036 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154803f030 con 0x7f1558103c30 2024-06-10T01:12:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.212+0000 7f154f7fe640 1 -- 172.21.15.7:0/64311036 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1548003ce0 con 0x7f1558103c30 2024-06-10T01:12:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.212+0000 7f155cb89640 1 -- 172.21.15.7:0/64311036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15581a1e30 con 0x7f1558103c30 2024-06-10T01:12:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.212+0000 7f154f7fe640 1 -- 172.21.15.7:0/64311036 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1548035070 con 0x7f1558103c30 2024-06-10T01:12:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.213+0000 7f155cb89640 1 -- 172.21.15.7:0/64311036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15581a2240 con 0x7f1558103c30 2024-06-10T01:12:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.215+0000 7f154f7fe640 1 -- 172.21.15.7:0/64311036 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f1548042030 con 0x7f1558103c30 2024-06-10T01:12:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.215+0000 7f155f971740 1 Processor -- start 2024-06-10T01:12:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.216+0000 7f155f971740 1 -- start start 2024-06-10T01:12:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.216+0000 7f155f971740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6dbd27010 0x55e6dbd27430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.216+0000 7f155f971740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6dbd27a00 0x55e6dbd2c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.216+0000 7f155f971740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6dbd2cb70 0x55e6dbd2f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.216+0000 7f155f971740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e6dbc8a4f0 con 0x55e6dbd27010 2024-06-10T01:12:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.216+0000 7f155f971740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e6dbc28870 con 0x55e6dbd2cb70 2024-06-10T01:12:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.216+0000 7f155f971740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e6dbc864b0 con 0x55e6dbd27a00 2024-06-10T01:12:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.216+0000 7f155659c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6dbd27a00 0x55e6dbd2c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.216+0000 7f1556d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6dbd27010 0x55e6dbd27430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.216+0000 7f155659c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6dbd27a00 0x55e6dbd2c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:44288/0 (socket says 172.21.15.7:44288) 2024-06-10T01:12:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.216+0000 7f155659c640 1 -- 172.21.15.7:0/1414787533 learned_addr learned my addr 172.21.15.7:0/1414787533 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.216+0000 7f154ffff640 1 --2- 172.21.15.7:0/1414787533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6dbd2cb70 0x55e6dbd2f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.216+0000 7f154f7fe640 1 --2- 172.21.15.7:0/64311036 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f15440a2b80 0x7f15440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.216+0000 7f155659c640 1 -- 172.21.15.7:0/1414787533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6dbd2cb70 msgr2=0x55e6dbd2f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.216+0000 7f155659c640 1 --2- 172.21.15.7:0/1414787533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6dbd2cb70 0x55e6dbd2f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.216+0000 7f155659c640 1 -- 172.21.15.7:0/1414787533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6dbd27010 msgr2=0x55e6dbd27430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.216+0000 7f155659c640 1 --2- 172.21.15.7:0/1414787533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6dbd27010 0x55e6dbd27430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.217+0000 7f155659c640 1 -- 172.21.15.7:0/1414787533 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6dbc449b0 con 0x55e6dbd27a00 2024-06-10T01:12:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.217+0000 7f154ffff640 1 --2- 172.21.15.7:0/1414787533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6dbd2cb70 0x55e6dbd2f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.216+0000 7f1556d9d640 1 --2- 172.21.15.7:0/64311036 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f15440a2b80 0x7f15440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.217+0000 7f154f7fe640 1 -- 172.21.15.7:0/64311036 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f15480ef8d0 con 0x7f1558103c30 2024-06-10T01:12:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.217+0000 7f155659c640 1 --2- 172.21.15.7:0/1414787533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6dbd27a00 0x55e6dbd2c500 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f15480b8810 tx=0x7f154804b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eec88f92a7753eea server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.217+0000 7f1556d9d640 1 --2- 172.21.15.7:0/64311036 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f15440a2b80 0x7f15440a5040 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1550009fd0 tx=0x7f1550004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.217+0000 7f154cff9640 1 -- 172.21.15.7:0/1414787533 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154804b300 con 0x55e6dbd27a00 2024-06-10T01:12:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.217+0000 7f154cff9640 1 -- 172.21.15.7:0/1414787533 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f154802d1c0 con 0x55e6dbd27a00 2024-06-10T01:12:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.217+0000 7f154cff9640 1 -- 172.21.15.7:0/1414787533 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15480c4a00 con 0x55e6dbd27a00 2024-06-10T01:12:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.218+0000 7f155f971740 1 -- 172.21.15.7:0/1414787533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6dbd27a00 msgr2=0x55e6dbd2c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.218+0000 7f155f971740 1 --2- 172.21.15.7:0/1414787533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6dbd27a00 0x55e6dbd2c500 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f15480b8810 tx=0x7f154804b000 comp rx=0 tx=0).stop 2024-06-10T01:12:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.218+0000 7f155f971740 1 -- 172.21.15.7:0/1414787533 shutdown_connections 2024-06-10T01:12:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.218+0000 7f155f971740 1 --2- 172.21.15.7:0/1414787533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6dbd2cb70 0x55e6dbd2f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.218+0000 7f155f971740 1 --2- 172.21.15.7:0/1414787533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6dbd27a00 0x55e6dbd2c500 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.218+0000 7f155f971740 1 --2- 172.21.15.7:0/1414787533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6dbd27010 0x55e6dbd27430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.218+0000 7f155f971740 1 -- 172.21.15.7:0/1414787533 >> 172.21.15.7:0/1414787533 conn(0x55e6dbd22680 msgr2=0x55e6dbd22a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:40.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.218+0000 7f155f971740 1 -- 172.21.15.7:0/1414787533 shutdown_connections 2024-06-10T01:12:40.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.218+0000 7f155f971740 1 -- 172.21.15.7:0/1414787533 wait complete. 2024-06-10T01:12:40.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.218+0000 7f155f971740 1 -- start start 2024-06-10T01:12:40.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.218+0000 7f155f971740 1 Processor -- start 2024-06-10T01:12:40.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.219+0000 7f155f971740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6dbd27010 0x55e6dbd30e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:40.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.219+0000 7f155f971740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6dbd27a00 0x55e6dbd31340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:40.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.219+0000 7f155f971740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6dbd2cb70 0x55e6dbd38310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:40.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.219+0000 7f155f971740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e6dbc864b0 con 0x55e6dbd27010 2024-06-10T01:12:40.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.219+0000 7f155f971740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e6dbc13ae0 con 0x55e6dbd27a00 2024-06-10T01:12:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.219+0000 7f155f971740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e6dbd2f8e0 con 0x55e6dbd2cb70 2024-06-10T01:12:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.219+0000 7f155f971740 10 client.0 _ll_register_callbacks cb 0x55e6dbc4c9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:12:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.219+0000 7f155659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6dbd27010 0x55e6dbd30e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.219+0000 7f155659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6dbd27010 0x55e6dbd30e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37728/0 (socket says 172.21.15.7:37728) 2024-06-10T01:12:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.219+0000 7f155659c640 1 -- 172.21.15.7:0/330557834 learned_addr learned my addr 172.21.15.7:0/330557834 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.219+0000 7f154ffff640 1 --2- 172.21.15.7:0/330557834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6dbd27a00 0x55e6dbd31340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.219+0000 7f1556d9d640 1 --2- 172.21.15.7:0/330557834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6dbd2cb70 0x55e6dbd38310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.219+0000 7f155659c640 1 -- 172.21.15.7:0/330557834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6dbd2cb70 msgr2=0x55e6dbd38310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.219+0000 7f155659c640 1 --2- 172.21.15.7:0/330557834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6dbd2cb70 0x55e6dbd38310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.219+0000 7f155659c640 1 -- 172.21.15.7:0/330557834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6dbd27a00 msgr2=0x55e6dbd31340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.219+0000 7f155659c640 1 --2- 172.21.15.7:0/330557834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6dbd27a00 0x55e6dbd31340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.219+0000 7f155659c640 1 -- 172.21.15.7:0/330557834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6db99e210 con 0x55e6dbd27010 2024-06-10T01:12:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.219+0000 7f154ffff640 1 --2- 172.21.15.7:0/330557834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6dbd27a00 0x55e6dbd31340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.219+0000 7f155659c640 1 --2- 172.21.15.7:0/330557834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6dbd27010 0x55e6dbd30e00 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f15480b87c0 tx=0x7f154802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1cd6bb2c3a7167 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.219+0000 7f153dffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:12:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.219+0000 7f153dffb640 1 -- 172.21.15.7:0/330557834 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1548049040 con 0x55e6dbd27010 2024-06-10T01:12:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.219+0000 7f153dffb640 1 -- 172.21.15.7:0/330557834 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f154803bd70 con 0x55e6dbd27010 2024-06-10T01:12:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.219+0000 7f155f971740 1 -- 172.21.15.7:0/330557834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f154802d1c0 con 0x55e6dbd27010 2024-06-10T01:12:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.220+0000 7f153dffb640 1 -- 172.21.15.7:0/330557834 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f15480c4a00 con 0x55e6dbd27010 2024-06-10T01:12:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.220+0000 7f155f971740 10 client.17145 fetch_fsmap learned FSMap version 32 2024-06-10T01:12:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.220+0000 7f155f971740 1 -- 172.21.15.7:0/330557834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e6dba6a620 con 0x55e6dbd27010 2024-06-10T01:12:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.220+0000 7f153dffb640 1 -- 172.21.15.7:0/330557834 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f154802d1c0 con 0x55e6dbd27010 2024-06-10T01:12:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.220+0000 7f153dffb640 1 -- 172.21.15.7:0/330557834 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f1548034170 con 0x55e6dbd27010 2024-06-10T01:12:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.220+0000 7f155f971740 10 client.17145 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:12:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.220+0000 7f155f971740 10 client.17145 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:12:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.220+0000 7f155f971740 20 client.17145 populate_metadata read hostname 'smithi007' 2024-06-10T01:12:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.220+0000 7f155f971740 1 --2- 172.21.15.7:0/330557834 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e6dbd3cb40 0x55e6dbd3ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.220+0000 7f155f971740 4 client.17145 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:12:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.220+0000 7f155f971740 1 -- 172.21.15.7:0/330557834 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6dbc9df10 con 0x55e6dbd3cb40 2024-06-10T01:12:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.220+0000 7f154ffff640 1 --2- 172.21.15.7:0/330557834 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e6dbd3cb40 0x55e6dbd3ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.221+0000 7f154ffff640 1 --2- 172.21.15.7:0/330557834 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e6dbd3cb40 0x55e6dbd3ef50 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=700ef5330bcdb4ee server_cookie=a69c184a0990a22 in_seq=0 out_seq=0 2024-06-10T01:12:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.221+0000 7f153dffb640 10 client.17145 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.221+0000 7f153dffb640 1 -- 172.21.15.7:0/330557834 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e6dbc9df10 con 0x55e6dbd3cb40 2024-06-10T01:12:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.222+0000 7f153dffb640 10 client.17145 handle_command_reply: tid=0 2024-06-10T01:12:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.222+0000 7f155f971740 1 client.17145 shutdown 2024-06-10T01:12:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.222+0000 7f155f971740 1 -- 172.21.15.7:0/330557834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6dbd27010 msgr2=0x55e6dbd30e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.222+0000 7f155f971740 1 --2- 172.21.15.7:0/330557834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6dbd27010 0x55e6dbd30e00 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f15480b87c0 tx=0x7f154802a410 comp rx=0 tx=0).stop 2024-06-10T01:12:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.222+0000 7f155f971740 1 -- 172.21.15.7:0/330557834 shutdown_connections 2024-06-10T01:12:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.222+0000 7f155f971740 1 --2- 172.21.15.7:0/330557834 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e6dbd3cb40 0x55e6dbd3ef50 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.222+0000 7f155f971740 1 --2- 172.21.15.7:0/330557834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6dbd2cb70 0x55e6dbd38310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.222+0000 7f153dffb640 0 client.17145 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.222+0000 7f155f971740 1 --2- 172.21.15.7:0/330557834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6dbd27a00 0x55e6dbd31340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.222+0000 7f155f971740 1 --2- 172.21.15.7:0/330557834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6dbd27010 0x55e6dbd30e00 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.222+0000 7f155f971740 1 -- 172.21.15.7:0/330557834 >> 172.21.15.7:0/330557834 conn(0x55e6dbd22680 msgr2=0x55e6dbd217e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.222+0000 7f155f971740 1 -- 172.21.15.7:0/330557834 shutdown_connections 2024-06-10T01:12:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.222+0000 7f155f971740 1 -- 172.21.15.7:0/330557834 wait complete. 2024-06-10T01:12:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.222+0000 7f155f971740 20 client.17145 trim_cache size 0 max 16384 2024-06-10T01:12:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.238+0000 7f155f971740 1 Processor -- start 2024-06-10T01:12:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.238+0000 7f155f971740 1 -- start start 2024-06-10T01:12:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.238+0000 7f155f971740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6dbd27010 0x55e6dbd2ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.239+0000 7f155f971740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6dbd27a00 0x55e6dbd382a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.239+0000 7f155f971740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6dbd2cb70 0x55e6dbd30a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.239+0000 7f155f971740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f15581af0d0 con 0x55e6dbd27010 2024-06-10T01:12:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.239+0000 7f155f971740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f15581ac9a0 con 0x55e6dbd27a00 2024-06-10T01:12:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.239+0000 7f155f971740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f15581aa7f0 con 0x55e6dbd2cb70 2024-06-10T01:12:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.239+0000 7f155659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6dbd27010 0x55e6dbd2ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.239+0000 7f155659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6dbd27010 0x55e6dbd2ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37730/0 (socket says 172.21.15.7:37730) 2024-06-10T01:12:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.239+0000 7f155659c640 1 -- 172.21.15.7:0/1853738843 learned_addr learned my addr 172.21.15.7:0/1853738843 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.239+0000 7f154ffff640 1 --2- 172.21.15.7:0/1853738843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6dbd27a00 0x55e6dbd382a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.239+0000 7f1556d9d640 1 --2- 172.21.15.7:0/1853738843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6dbd2cb70 0x55e6dbd30a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.239+0000 7f155659c640 1 -- 172.21.15.7:0/1853738843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6dbd2cb70 msgr2=0x55e6dbd30a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.239+0000 7f155659c640 1 --2- 172.21.15.7:0/1853738843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6dbd2cb70 0x55e6dbd30a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.239+0000 7f155659c640 1 -- 172.21.15.7:0/1853738843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6dbd27a00 msgr2=0x55e6dbd382a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.239+0000 7f155659c640 1 --2- 172.21.15.7:0/1853738843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6dbd27a00 0x55e6dbd382a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.239+0000 7f155659c640 1 -- 172.21.15.7:0/1853738843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6dbd3f5d0 con 0x55e6dbd27010 2024-06-10T01:12:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.239+0000 7f155659c640 1 --2- 172.21.15.7:0/1853738843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6dbd27010 0x55e6dbd2ef60 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f15480b8810 tx=0x7f15480ba350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ad4aa65d4b1d77 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.239+0000 7f153ffff640 1 -- 172.21.15.7:0/1853738843 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1548049070 con 0x55e6dbd27010 2024-06-10T01:12:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.239+0000 7f153ffff640 1 -- 172.21.15.7:0/1853738843 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1548003800 con 0x55e6dbd27010 2024-06-10T01:12:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.239+0000 7f153ffff640 1 -- 172.21.15.7:0/1853738843 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1548034170 con 0x55e6dbd27010 2024-06-10T01:12:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.240+0000 7f155f971740 1 -- 172.21.15.7:0/1853738843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6dbd27010 msgr2=0x55e6dbd2ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.240+0000 7f155f971740 1 --2- 172.21.15.7:0/1853738843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6dbd27010 0x55e6dbd2ef60 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f15480b8810 tx=0x7f15480ba350 comp rx=0 tx=0).stop 2024-06-10T01:12:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.240+0000 7f155f971740 1 -- 172.21.15.7:0/1853738843 shutdown_connections 2024-06-10T01:12:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.240+0000 7f155f971740 1 --2- 172.21.15.7:0/1853738843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6dbd2cb70 0x55e6dbd30a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.240+0000 7f155f971740 1 --2- 172.21.15.7:0/1853738843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6dbd27a00 0x55e6dbd382a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.240+0000 7f155f971740 1 --2- 172.21.15.7:0/1853738843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6dbd27010 0x55e6dbd2ef60 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.240+0000 7f155f971740 1 -- 172.21.15.7:0/1853738843 >> 172.21.15.7:0/1853738843 conn(0x55e6dbd22680 msgr2=0x55e6dbd22a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.240+0000 7f155f971740 1 -- 172.21.15.7:0/1853738843 shutdown_connections 2024-06-10T01:12:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.240+0000 7f155f971740 1 -- 172.21.15.7:0/1853738843 wait complete. 2024-06-10T01:12:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.240+0000 7f155f971740 1 -- start start 2024-06-10T01:12:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.241+0000 7f155f971740 1 Processor -- start 2024-06-10T01:12:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.241+0000 7f155f971740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6dbd27010 0x55e6dbd37d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.241+0000 7f155f971740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6dbd27a00 0x55e6dbd38260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.241+0000 7f155f971740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6dbd2cb70 0x55e6dbd34870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.241+0000 7f155f971740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f15581aae60 con 0x55e6dbd2cb70 2024-06-10T01:12:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.241+0000 7f155f971740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f15581ac2d0 con 0x55e6dbd27010 2024-06-10T01:12:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.241+0000 7f155f971740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f15581ab530 con 0x55e6dbd27a00 2024-06-10T01:12:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.241+0000 7f155f971740 10 client.0 _ll_register_callbacks cb 0x7f1550009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:12:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.241+0000 7f1556d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6dbd2cb70 0x55e6dbd34870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.241+0000 7f1556d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6dbd2cb70 0x55e6dbd34870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37736/0 (socket says 172.21.15.7:37736) 2024-06-10T01:12:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.241+0000 7f1556d9d640 1 -- 172.21.15.7:0/851226626 learned_addr learned my addr 172.21.15.7:0/851226626 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.241+0000 7f154ffff640 1 --2- 172.21.15.7:0/851226626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6dbd27a00 0x55e6dbd38260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.241+0000 7f155659c640 1 --2- 172.21.15.7:0/851226626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6dbd27010 0x55e6dbd37d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.241+0000 7f1556d9d640 1 -- 172.21.15.7:0/851226626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6dbd27a00 msgr2=0x55e6dbd38260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.241+0000 7f1556d9d640 1 --2- 172.21.15.7:0/851226626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6dbd27a00 0x55e6dbd38260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.241+0000 7f1556d9d640 1 -- 172.21.15.7:0/851226626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6dbd27010 msgr2=0x55e6dbd37d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.241+0000 7f1556d9d640 1 --2- 172.21.15.7:0/851226626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6dbd27010 0x55e6dbd37d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.241+0000 7f1556d9d640 1 -- 172.21.15.7:0/851226626 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6dbd350a0 con 0x55e6dbd2cb70 2024-06-10T01:12:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.241+0000 7f155659c640 1 --2- 172.21.15.7:0/851226626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6dbd27010 0x55e6dbd37d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:12:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.241+0000 7f1556d9d640 1 --2- 172.21.15.7:0/851226626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6dbd2cb70 0x55e6dbd34870 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f155000eed0 tx=0x7f1550004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279814eec4acc45a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.241+0000 7f153dffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:12:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.242+0000 7f153dffb640 1 -- 172.21.15.7:0/851226626 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1550004b20 con 0x55e6dbd2cb70 2024-06-10T01:12:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.242+0000 7f153dffb640 1 -- 172.21.15.7:0/851226626 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f1550004cc0 con 0x55e6dbd2cb70 2024-06-10T01:12:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.242+0000 7f155f971740 1 -- 172.21.15.7:0/851226626 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1548003800 con 0x55e6dbd2cb70 2024-06-10T01:12:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.242+0000 7f153dffb640 1 -- 172.21.15.7:0/851226626 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1550010630 con 0x55e6dbd2cb70 2024-06-10T01:12:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.242+0000 7f155f971740 10 client.17151 fetch_fsmap learned FSMap version 32 2024-06-10T01:12:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.242+0000 7f155f971740 1 -- 172.21.15.7:0/851226626 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e6dbda6720 con 0x55e6dbd2cb70 2024-06-10T01:12:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.242+0000 7f153dffb640 1 -- 172.21.15.7:0/851226626 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1550034070 con 0x55e6dbd2cb70 2024-06-10T01:12:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.242+0000 7f153dffb640 1 -- 172.21.15.7:0/851226626 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f155001e040 con 0x55e6dbd2cb70 2024-06-10T01:12:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.242+0000 7f155f971740 10 client.17151 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:12:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.242+0000 7f155f971740 10 client.17151 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:12:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.242+0000 7f155f971740 20 client.17151 populate_metadata read hostname 'smithi007' 2024-06-10T01:12:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.242+0000 7f155f971740 1 --2- 172.21.15.7:0/851226626 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e6dbda7130 0x55e6dbda9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.242+0000 7f155f971740 4 client.17151 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:12:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.242+0000 7f155f971740 1 -- 172.21.15.7:0/851226626 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e6dbc97c20 con 0x55e6dbda7130 2024-06-10T01:12:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.242+0000 7f155659c640 1 --2- 172.21.15.7:0/851226626 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e6dbda7130 0x55e6dbda9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.243+0000 7f155659c640 1 --2- 172.21.15.7:0/851226626 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e6dbda7130 0x55e6dbda9550 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a84bafd9e0e01cd6 server_cookie=9f9ba81e4a08a394 in_seq=0 out_seq=0 2024-06-10T01:12:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.243+0000 7f153dffb640 10 client.17151 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:40.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.414+0000 7f153dffb640 1 -- 172.21.15.7:0/851226626 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e6dbc97c20 con 0x55e6dbda7130 2024-06-10T01:12:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.414+0000 7f153dffb640 10 client.17151 handle_command_reply: tid=0 2024-06-10T01:12:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.414+0000 7f155f971740 1 client.17151 shutdown 2024-06-10T01:12:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.414+0000 7f155f971740 1 -- 172.21.15.7:0/851226626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6dbd2cb70 msgr2=0x55e6dbd34870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.414+0000 7f155f971740 1 --2- 172.21.15.7:0/851226626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6dbd2cb70 0x55e6dbd34870 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f155000eed0 tx=0x7f1550004a40 comp rx=0 tx=0).stop 2024-06-10T01:12:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.414+0000 7f155f971740 1 -- 172.21.15.7:0/851226626 shutdown_connections 2024-06-10T01:12:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.414+0000 7f155f971740 1 --2- 172.21.15.7:0/851226626 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e6dbda7130 0x55e6dbda9550 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.414+0000 7f155f971740 1 --2- 172.21.15.7:0/851226626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6dbd2cb70 0x55e6dbd34870 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.414+0000 7f155f971740 1 --2- 172.21.15.7:0/851226626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e6dbd27a00 0x55e6dbd38260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.414+0000 7f153dffb640 0 client.17151 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.414+0000 7f155f971740 1 --2- 172.21.15.7:0/851226626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6dbd27010 0x55e6dbd37d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.414+0000 7f155f971740 1 -- 172.21.15.7:0/851226626 >> 172.21.15.7:0/851226626 conn(0x55e6dbd22680 msgr2=0x55e6dbd309f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.415+0000 7f155f971740 1 -- 172.21.15.7:0/851226626 shutdown_connections 2024-06-10T01:12:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.415+0000 7f155f971740 1 -- 172.21.15.7:0/851226626 wait complete. 2024-06-10T01:12:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.415+0000 7f155f971740 20 client.17151 trim_cache size 0 max 16384 2024-06-10T01:12:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:12:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:12:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:12:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.415+0000 7f155cb89640 1 -- 172.21.15.7:0/64311036 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f15440a2b80 msgr2=0x7f15440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.415+0000 7f155cb89640 1 --2- 172.21.15.7:0/64311036 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f15440a2b80 0x7f15440a5040 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1550009fd0 tx=0x7f1550004ab0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.415+0000 7f155cb89640 1 -- 172.21.15.7:0/64311036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1558103c30 msgr2=0x7f1558199f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.415+0000 7f155cb89640 1 --2- 172.21.15.7:0/64311036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1558103c30 0x7f1558199f80 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1548031610 tx=0x7f15480043b0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.415+0000 7f155cb89640 1 -- 172.21.15.7:0/64311036 shutdown_connections 2024-06-10T01:12:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.415+0000 7f155cb89640 1 --2- 172.21.15.7:0/64311036 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f15440a2b80 0x7f15440a5040 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.415+0000 7f155cb89640 1 --2- 172.21.15.7:0/64311036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15581057e0 0x7f15581a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.415+0000 7f155cb89640 1 --2- 172.21.15.7:0/64311036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1558104e20 0x7f155819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.415+0000 7f155cb89640 1 --2- 172.21.15.7:0/64311036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1558103c30 0x7f1558199f80 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.415+0000 7f155cb89640 1 -- 172.21.15.7:0/64311036 >> 172.21.15.7:0/64311036 conn(0x7f15580ff1c0 msgr2=0x7f1558108190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.415+0000 7f155cb89640 1 -- 172.21.15.7:0/64311036 shutdown_connections 2024-06-10T01:12:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:40.415+0000 7f155cb89640 1 -- 172.21.15.7:0/64311036 wait complete. 2024-06-10T01:12:40.735 INFO:tasks.workunit.client.0.smithi007.stdout: 1 423800 6.69 MB/sec execute 201 sec latency 126.702 ms 2024-06-10T01:12:41.579 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:41 smithi078 ceph-mon[36683]: pgmap v632: 129 pgs: 129 active+clean; 85 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.3 MiB/s wr, 292 op/s 2024-06-10T01:12:41.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:41 smithi175 ceph-mon[36636]: pgmap v632: 129 pgs: 129 active+clean; 85 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.3 MiB/s wr, 292 op/s 2024-06-10T01:12:41.735 INFO:tasks.workunit.client.0.smithi007.stdout: 1 426061 6.72 MB/sec execute 202 sec latency 57.383 ms 2024-06-10T01:12:41.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:41 smithi007 ceph-mon[32398]: pgmap v632: 129 pgs: 129 active+clean; 85 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.3 MiB/s wr, 292 op/s 2024-06-10T01:12:42.735 INFO:tasks.workunit.client.0.smithi007.stdout: 1 428110 6.72 MB/sec execute 203 sec latency 11.082 ms 2024-06-10T01:12:43.579 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:43 smithi078 ceph-mon[36683]: pgmap v633: 129 pgs: 129 active+clean; 83 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 281 op/s 2024-06-10T01:12:43.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:43 smithi175 ceph-mon[36636]: pgmap v633: 129 pgs: 129 active+clean; 83 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 281 op/s 2024-06-10T01:12:43.736 INFO:tasks.workunit.client.0.smithi007.stdout: 1 429818 6.75 MB/sec execute 204 sec latency 12.386 ms 2024-06-10T01:12:43.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:43 smithi007 ceph-mon[32398]: pgmap v633: 129 pgs: 129 active+clean; 83 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 281 op/s 2024-06-10T01:12:44.489 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:12:44.735 INFO:tasks.workunit.client.0.smithi007.stdout: 1 432416 6.78 MB/sec execute 205 sec latency 26.102 ms 2024-06-10T01:12:44.744 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:12:45.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:45 smithi007 ceph-mon[32398]: pgmap v634: 129 pgs: 129 active+clean; 84 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.3 MiB/s wr, 247 op/s 2024-06-10T01:12:45.579 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:45 smithi078 ceph-mon[36683]: pgmap v634: 129 pgs: 129 active+clean; 84 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.3 MiB/s wr, 247 op/s 2024-06-10T01:12:45.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:45 smithi175 ceph-mon[36636]: pgmap v634: 129 pgs: 129 active+clean; 84 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.3 MiB/s wr, 247 op/s 2024-06-10T01:12:45.735 INFO:tasks.workunit.client.0.smithi007.stdout: 1 433592 6.78 MB/sec execute 206 sec latency 44.921 ms 2024-06-10T01:12:46.736 INFO:tasks.workunit.client.0.smithi007.stdout: 1 436399 6.83 MB/sec execute 207 sec latency 37.800 ms 2024-06-10T01:12:47.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.260+0000 7f988be78700 1 -- 172.21.15.7:0/3102384230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9884108560 msgr2=0x7f9884108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:47.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.260+0000 7f988be78700 1 --2- 172.21.15.7:0/3102384230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9884108560 0x7f9884108940 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9880009990 tx=0x7f9880009ca0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.260+0000 7f988be78700 1 -- 172.21.15.7:0/3102384230 shutdown_connections 2024-06-10T01:12:47.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.260+0000 7f988be78700 1 --2- 172.21.15.7:0/3102384230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98841013a0 0x7f988410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.260+0000 7f988be78700 1 --2- 172.21.15.7:0/3102384230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9884108560 0x7f9884108940 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.260+0000 7f988be78700 1 --2- 172.21.15.7:0/3102384230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98841009e0 0x7f9884100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.260+0000 7f988be78700 1 -- 172.21.15.7:0/3102384230 >> 172.21.15.7:0/3102384230 conn(0x7f9884077990 msgr2=0x7f98840fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:47.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.260+0000 7f988be78700 1 -- 172.21.15.7:0/3102384230 shutdown_connections 2024-06-10T01:12:47.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.260+0000 7f988be78700 1 -- 172.21.15.7:0/3102384230 wait complete. 2024-06-10T01:12:47.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.261+0000 7f988be78700 1 Processor -- start 2024-06-10T01:12:47.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.261+0000 7f988be78700 1 -- start start 2024-06-10T01:12:47.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.261+0000 7f988be78700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98841009e0 0x7f988419e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:47.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.261+0000 7f988be78700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98841013a0 0x7f988419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:47.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.261+0000 7f988be78700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9884108560 0x7f98841989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:47.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.261+0000 7f988be78700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9884112790 con 0x7f9884108560 2024-06-10T01:12:47.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.261+0000 7f988be78700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9884112610 con 0x7f98841009e0 2024-06-10T01:12:47.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.261+0000 7f988be78700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9884112910 con 0x7f98841013a0 2024-06-10T01:12:47.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.261+0000 7f9889413700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98841013a0 0x7f988419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:47.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.261+0000 7f988a415700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9884108560 0x7f98841989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:47.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.261+0000 7f9889413700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98841013a0 0x7f988419ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52918/0 (socket says 172.21.15.7:52918) 2024-06-10T01:12:47.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.261+0000 7f9889413700 1 -- 172.21.15.7:0/795173532 learned_addr learned my addr 172.21.15.7:0/795173532 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:47.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.261+0000 7f9889c14700 1 --2- 172.21.15.7:0/795173532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98841009e0 0x7f988419e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:47.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.262+0000 7f988a415700 1 -- 172.21.15.7:0/795173532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98841013a0 msgr2=0x7f988419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:47.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.262+0000 7f988a415700 1 --2- 172.21.15.7:0/795173532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98841013a0 0x7f988419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.262+0000 7f988a415700 1 -- 172.21.15.7:0/795173532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98841009e0 msgr2=0x7f988419e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:47.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.262+0000 7f988a415700 1 --2- 172.21.15.7:0/795173532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98841009e0 0x7f988419e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.262+0000 7f988a415700 1 -- 172.21.15.7:0/795173532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9884199150 con 0x7f9884108560 2024-06-10T01:12:47.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.262+0000 7f9889413700 1 --2- 172.21.15.7:0/795173532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98841013a0 0x7f988419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:47.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.262+0000 7f9889c14700 1 --2- 172.21.15.7:0/795173532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98841009e0 0x7f988419e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:47.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.262+0000 7f988a415700 1 --2- 172.21.15.7:0/795173532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9884108560 0x7f98841989f0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f987c00efb0 tx=0x7f987c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:47.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.262+0000 7f987affd700 1 -- 172.21.15.7:0/795173532 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c003a00 con 0x7f9884108560 2024-06-10T01:12:47.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.262+0000 7f987affd700 1 -- 172.21.15.7:0/795173532 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f987c003ba0 con 0x7f9884108560 2024-06-10T01:12:47.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.262+0000 7f988be78700 1 -- 172.21.15.7:0/795173532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9884199440 con 0x7f9884108560 2024-06-10T01:12:47.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.262+0000 7f987affd700 1 -- 172.21.15.7:0/795173532 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c004850 con 0x7f9884108560 2024-06-10T01:12:47.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.262+0000 7f988be78700 1 -- 172.21.15.7:0/795173532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98841a56b0 con 0x7f9884108560 2024-06-10T01:12:47.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.265+0000 7f987affd700 1 -- 172.21.15.7:0/795173532 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f987c012050 con 0x7f9884108560 2024-06-10T01:12:47.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.265+0000 7f987affd700 1 --2- 172.21.15.7:0/795173532 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f98700a2c20 0x7f98700a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:47.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.265+0000 7f9889c14700 1 --2- 172.21.15.7:0/795173532 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f98700a2c20 0x7f98700a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:47.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.265+0000 7f987affd700 1 -- 172.21.15.7:0/795173532 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f987c0cecc0 con 0x7f9884108560 2024-06-10T01:12:47.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.266+0000 7f9889c14700 1 --2- 172.21.15.7:0/795173532 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f98700a2c20 0x7f98700a50e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f988000b2e0 tx=0x7f988001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:47.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.266+0000 7f98966fbb80 1 Processor -- start 2024-06-10T01:12:47.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.266+0000 7f98966fbb80 1 -- start start 2024-06-10T01:12:47.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.267+0000 7f98966fbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d024b1fd0 0x562d024b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:47.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.267+0000 7f98966fbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d024b28f0 0x562d024b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:47.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.267+0000 7f98966fbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d024b6c30 0x562d024b7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:47.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.267+0000 7f98966fbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562d023ff810 con 0x562d024b6c30 2024-06-10T01:12:47.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.267+0000 7f98966fbb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562d023a7720 con 0x562d024b1fd0 2024-06-10T01:12:47.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.267+0000 7f98966fbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562d023fb7d0 con 0x562d024b28f0 2024-06-10T01:12:47.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.267+0000 7f9889413700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d024b28f0 0x562d024b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:47.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.267+0000 7f988a415700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d024b6c30 0x562d024b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:47.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.267+0000 7f9889413700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d024b28f0 0x562d024b66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52932/0 (socket says 172.21.15.7:52932) 2024-06-10T01:12:47.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.267+0000 7f9889413700 1 -- 172.21.15.7:0/3731031488 learned_addr learned my addr 172.21.15.7:0/3731031488 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:47.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.267+0000 7f988a415700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d024b6c30 0x562d024b7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48586/0 (socket says 172.21.15.7:48586) 2024-06-10T01:12:47.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.267+0000 7f9889c14700 1 --2- 172.21.15.7:0/3731031488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d024b1fd0 0x562d024b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:47.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.267+0000 7f9889413700 1 -- 172.21.15.7:0/3731031488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d024b1fd0 msgr2=0x562d024b23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:47.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.267+0000 7f9889413700 1 --2- 172.21.15.7:0/3731031488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d024b1fd0 0x562d024b23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.267+0000 7f9889413700 1 -- 172.21.15.7:0/3731031488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d024b6c30 msgr2=0x562d024b7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:47.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.267+0000 7f9889413700 1 --2- 172.21.15.7:0/3731031488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d024b6c30 0x562d024b7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.267+0000 7f9889413700 1 -- 172.21.15.7:0/3731031488 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d0239df50 con 0x562d024b28f0 2024-06-10T01:12:47.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.267+0000 7f9889413700 1 --2- 172.21.15.7:0/3731031488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d024b28f0 0x562d024b66f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9874011680 tx=0x7f9874011990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87a958025efecd46 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:47.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.268+0000 7f986ffff700 1 -- 172.21.15.7:0/3731031488 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9874004230 con 0x562d024b28f0 2024-06-10T01:12:47.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.268+0000 7f986ffff700 1 -- 172.21.15.7:0/3731031488 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98740043d0 con 0x562d024b28f0 2024-06-10T01:12:47.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.268+0000 7f986ffff700 1 -- 172.21.15.7:0/3731031488 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9874019980 con 0x562d024b28f0 2024-06-10T01:12:47.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.268+0000 7f98966fbb80 1 -- 172.21.15.7:0/3731031488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d024b28f0 msgr2=0x562d024b66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:47.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.268+0000 7f98966fbb80 1 --2- 172.21.15.7:0/3731031488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d024b28f0 0x562d024b66f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9874011680 tx=0x7f9874011990 comp rx=0 tx=0).stop 2024-06-10T01:12:47.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.268+0000 7f98966fbb80 1 -- 172.21.15.7:0/3731031488 shutdown_connections 2024-06-10T01:12:47.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.268+0000 7f98966fbb80 1 --2- 172.21.15.7:0/3731031488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d024b6c30 0x562d024b7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.268+0000 7f98966fbb80 1 --2- 172.21.15.7:0/3731031488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d024b1fd0 0x562d024b23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.268+0000 7f98966fbb80 1 --2- 172.21.15.7:0/3731031488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d024b28f0 0x562d024b66f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.268+0000 7f98966fbb80 1 -- 172.21.15.7:0/3731031488 >> 172.21.15.7:0/3731031488 conn(0x562d023d8cc0 msgr2=0x562d024a8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:47.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.269+0000 7f98966fbb80 1 -- 172.21.15.7:0/3731031488 shutdown_connections 2024-06-10T01:12:47.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.269+0000 7f98966fbb80 1 -- 172.21.15.7:0/3731031488 wait complete. 2024-06-10T01:12:47.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.269+0000 7f98966fbb80 1 -- start start 2024-06-10T01:12:47.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.269+0000 7f98966fbb80 1 Processor -- start 2024-06-10T01:12:47.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.269+0000 7f98966fbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d024b1fd0 0x562d024be1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:47.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.269+0000 7f98966fbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d024b28f0 0x562d024bb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:47.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.270+0000 7f98966fbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d024b6c30 0x562d024bba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:47.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.270+0000 7f98966fbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562d024bedd0 con 0x562d024b6c30 2024-06-10T01:12:47.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.270+0000 7f98966fbb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562d0239fbf0 con 0x562d024b1fd0 2024-06-10T01:12:47.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.270+0000 7f98966fbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562d024bef50 con 0x562d024b28f0 2024-06-10T01:12:47.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.270+0000 7f98966fbb80 10 client.0 _ll_register_callbacks cb 0x562d02399920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:12:47.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.270+0000 7f988a415700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d024b6c30 0x562d024bba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:47.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.270+0000 7f988a415700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d024b6c30 0x562d024bba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48598/0 (socket says 172.21.15.7:48598) 2024-06-10T01:12:47.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.270+0000 7f9889413700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d024b28f0 0x562d024bb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:47.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.270+0000 7f988a415700 1 -- 172.21.15.7:0/2359748672 learned_addr learned my addr 172.21.15.7:0/2359748672 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:47.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.270+0000 7f9889c14700 1 --2- 172.21.15.7:0/2359748672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d024b1fd0 0x562d024be1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:47.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.270+0000 7f9889413700 1 -- 172.21.15.7:0/2359748672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d024b1fd0 msgr2=0x562d024be1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:47.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.270+0000 7f9889413700 1 --2- 172.21.15.7:0/2359748672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d024b1fd0 0x562d024be1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.270+0000 7f9889413700 1 -- 172.21.15.7:0/2359748672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d024b6c30 msgr2=0x562d024bba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:47.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.270+0000 7f9889413700 1 --2- 172.21.15.7:0/2359748672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d024b6c30 0x562d024bba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.270+0000 7f9889413700 1 -- 172.21.15.7:0/2359748672 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d0234e8d0 con 0x562d024b28f0 2024-06-10T01:12:47.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.270+0000 7f988a415700 1 --2- 172.21.15.7:0/2359748672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d024b6c30 0x562d024bba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:47.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.270+0000 7f9889413700 1 --2- 172.21.15.7:0/2359748672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d024b28f0 0x562d024bb540 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9874019460 tx=0x7f98740052c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3916c25c260a4219 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:47.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.270+0000 7f986d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:12:47.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.271+0000 7f986d7fa700 1 -- 172.21.15.7:0/2359748672 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9874027070 con 0x562d024b28f0 2024-06-10T01:12:47.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.271+0000 7f98966fbb80 1 -- 172.21.15.7:0/2359748672 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98740043d0 con 0x562d024b28f0 2024-06-10T01:12:47.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.271+0000 7f986d7fa700 1 -- 172.21.15.7:0/2359748672 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f9874004950 con 0x562d024b28f0 2024-06-10T01:12:47.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.271+0000 7f986d7fa700 1 -- 172.21.15.7:0/2359748672 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9874033050 con 0x562d024b28f0 2024-06-10T01:12:47.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.271+0000 7f98966fbb80 10 client.26590 fetch_fsmap learned FSMap version 32 2024-06-10T01:12:47.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.271+0000 7f98966fbb80 1 -- 172.21.15.7:0/2359748672 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562d024c9720 con 0x562d024b28f0 2024-06-10T01:12:47.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.271+0000 7f986d7fa700 1 -- 172.21.15.7:0/2359748672 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98740043d0 con 0x562d024b28f0 2024-06-10T01:12:47.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.271+0000 7f986d7fa700 1 -- 172.21.15.7:0/2359748672 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f9874019600 con 0x562d024b28f0 2024-06-10T01:12:47.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.271+0000 7f98966fbb80 10 client.26590 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:12:47.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.271+0000 7f98966fbb80 10 client.26590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:12:47.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.271+0000 7f98966fbb80 20 client.26590 populate_metadata read hostname 'smithi007' 2024-06-10T01:12:47.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.271+0000 7f98966fbb80 1 --2- 172.21.15.7:0/2359748672 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562d024cbcb0 0x562d024ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:47.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.271+0000 7f98966fbb80 4 client.26590 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:12:47.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.271+0000 7f98966fbb80 1 -- 172.21.15.7:0/2359748672 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d0240cd10 con 0x562d024cbcb0 2024-06-10T01:12:47.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.272+0000 7f9889c14700 1 --2- 172.21.15.7:0/2359748672 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562d024cbcb0 0x562d024ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:47.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.272+0000 7f986d7fa700 10 client.26590 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:47.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.272+0000 7f9889c14700 1 --2- 172.21.15.7:0/2359748672 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562d024cbcb0 0x562d024ce0a0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=401bd05dc3ab5be2 server_cookie=8d022fe0e12621d in_seq=0 out_seq=0 2024-06-10T01:12:47.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.273+0000 7f986d7fa700 1 -- 172.21.15.7:0/2359748672 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562d0240cd10 con 0x562d024cbcb0 2024-06-10T01:12:47.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.273+0000 7f986d7fa700 10 client.26590 handle_command_reply: tid=0 2024-06-10T01:12:47.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.273+0000 7f98966fbb80 1 client.26590 shutdown 2024-06-10T01:12:47.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.274+0000 7f98966fbb80 1 -- 172.21.15.7:0/2359748672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d024b28f0 msgr2=0x562d024bb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:47.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.274+0000 7f98966fbb80 1 --2- 172.21.15.7:0/2359748672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d024b28f0 0x562d024bb540 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9874019460 tx=0x7f98740052c0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.274+0000 7f98966fbb80 1 -- 172.21.15.7:0/2359748672 shutdown_connections 2024-06-10T01:12:47.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.274+0000 7f98966fbb80 1 --2- 172.21.15.7:0/2359748672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d024b6c30 0x562d024bba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.274+0000 7f98966fbb80 1 --2- 172.21.15.7:0/2359748672 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562d024cbcb0 0x562d024ce0a0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.274+0000 7f98966fbb80 1 --2- 172.21.15.7:0/2359748672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d024b1fd0 0x562d024be1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.274+0000 7f98966fbb80 1 --2- 172.21.15.7:0/2359748672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d024b28f0 0x562d024bb540 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.274+0000 7f98966fbb80 1 -- 172.21.15.7:0/2359748672 >> 172.21.15.7:0/2359748672 conn(0x562d023d8cc0 msgr2=0x562d024ac9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:47.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.274+0000 7f986d7fa700 0 client.26590 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:47.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.274+0000 7f98966fbb80 1 -- 172.21.15.7:0/2359748672 shutdown_connections 2024-06-10T01:12:47.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.274+0000 7f98966fbb80 1 -- 172.21.15.7:0/2359748672 wait complete. 2024-06-10T01:12:47.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.274+0000 7f98966fbb80 20 client.26590 trim_cache size 0 max 16384 2024-06-10T01:12:47.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.300+0000 7f98966fbb80 1 Processor -- start 2024-06-10T01:12:47.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.300+0000 7f98966fbb80 1 -- start start 2024-06-10T01:12:47.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.300+0000 7f98966fbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d024b1fd0 0x562d024a89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:47.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.300+0000 7f98966fbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d024b28f0 0x562d024b90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:47.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.300+0000 7f98966fbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d024b6c30 0x562d024b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:47.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.300+0000 7f98966fbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f98841ae890 con 0x562d024b6c30 2024-06-10T01:12:47.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.300+0000 7f98966fbb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f98841ac120 con 0x562d024b1fd0 2024-06-10T01:12:47.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.300+0000 7f98966fbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f98841a9f70 con 0x562d024b28f0 2024-06-10T01:12:47.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.300+0000 7f988a415700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d024b6c30 0x562d024b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:47.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.300+0000 7f988a415700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d024b6c30 0x562d024b62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48612/0 (socket says 172.21.15.7:48612) 2024-06-10T01:12:47.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.300+0000 7f988a415700 1 -- 172.21.15.7:0/728095360 learned_addr learned my addr 172.21.15.7:0/728095360 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:47.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.300+0000 7f9889413700 1 --2- 172.21.15.7:0/728095360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d024b28f0 0x562d024b90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:47.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.300+0000 7f9889c14700 1 --2- 172.21.15.7:0/728095360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d024b1fd0 0x562d024a89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:47.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.300+0000 7f988a415700 1 -- 172.21.15.7:0/728095360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d024b28f0 msgr2=0x562d024b90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:47.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.300+0000 7f988a415700 1 --2- 172.21.15.7:0/728095360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d024b28f0 0x562d024b90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.300+0000 7f988a415700 1 -- 172.21.15.7:0/728095360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d024b1fd0 msgr2=0x562d024a89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:47.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.300+0000 7f988a415700 1 --2- 172.21.15.7:0/728095360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d024b1fd0 0x562d024a89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.301+0000 7f988a415700 1 -- 172.21.15.7:0/728095360 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d024b9ad0 con 0x562d024b6c30 2024-06-10T01:12:47.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.301+0000 7f988a415700 1 --2- 172.21.15.7:0/728095360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d024b6c30 0x562d024b62b0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f987c09b160 tx=0x7f987c0ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be0e23a6ac57936a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:47.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.301+0000 7f986f7fe700 1 -- 172.21.15.7:0/728095360 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c0b3070 con 0x562d024b6c30 2024-06-10T01:12:47.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.301+0000 7f986f7fe700 1 -- 172.21.15.7:0/728095360 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f987c09b8d0 con 0x562d024b6c30 2024-06-10T01:12:47.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.301+0000 7f986f7fe700 1 -- 172.21.15.7:0/728095360 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c0994d0 con 0x562d024b6c30 2024-06-10T01:12:47.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.302+0000 7f98966fbb80 1 -- 172.21.15.7:0/728095360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d024b6c30 msgr2=0x562d024b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:47.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.302+0000 7f98966fbb80 1 --2- 172.21.15.7:0/728095360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d024b6c30 0x562d024b62b0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f987c09b160 tx=0x7f987c0ce040 comp rx=0 tx=0).stop 2024-06-10T01:12:47.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.302+0000 7f98966fbb80 1 -- 172.21.15.7:0/728095360 shutdown_connections 2024-06-10T01:12:47.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.302+0000 7f98966fbb80 1 --2- 172.21.15.7:0/728095360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d024b6c30 0x562d024b62b0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.302+0000 7f98966fbb80 1 --2- 172.21.15.7:0/728095360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d024b1fd0 0x562d024a89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.302+0000 7f98966fbb80 1 --2- 172.21.15.7:0/728095360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d024b28f0 0x562d024b90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.302+0000 7f98966fbb80 1 -- 172.21.15.7:0/728095360 >> 172.21.15.7:0/728095360 conn(0x562d023d8cc0 msgr2=0x562d024b5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:47.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.302+0000 7f98966fbb80 1 -- 172.21.15.7:0/728095360 shutdown_connections 2024-06-10T01:12:47.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.302+0000 7f98966fbb80 1 -- 172.21.15.7:0/728095360 wait complete. 2024-06-10T01:12:47.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.302+0000 7f98966fbb80 1 -- start start 2024-06-10T01:12:47.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.302+0000 7f98966fbb80 1 Processor -- start 2024-06-10T01:12:47.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.303+0000 7f98966fbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d024b1fd0 0x562d024c0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:47.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.303+0000 7f98966fbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d024b28f0 0x562d024c06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:47.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.303+0000 7f98966fbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d024b6c30 0x562d024c0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:47.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.303+0000 7f98966fbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f98841aa5e0 con 0x562d024b1fd0 2024-06-10T01:12:47.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.303+0000 7f98966fbb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f98841aba50 con 0x562d024b6c30 2024-06-10T01:12:47.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.303+0000 7f98966fbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f98841aacb0 con 0x562d024b28f0 2024-06-10T01:12:47.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.303+0000 7f98966fbb80 10 client.0 _ll_register_callbacks cb 0x562d02399920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:12:47.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.303+0000 7f9889c14700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d024b1fd0 0x562d024c0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:47.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.303+0000 7f9889c14700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d024b1fd0 0x562d024c0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48616/0 (socket says 172.21.15.7:48616) 2024-06-10T01:12:47.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.303+0000 7f9889c14700 1 -- 172.21.15.7:0/1080257719 learned_addr learned my addr 172.21.15.7:0/1080257719 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:47.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.303+0000 7f9889413700 1 --2- 172.21.15.7:0/1080257719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d024b28f0 0x562d024c06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:47.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.303+0000 7f988a415700 1 --2- 172.21.15.7:0/1080257719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d024b6c30 0x562d024c0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:47.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.303+0000 7f9889c14700 1 -- 172.21.15.7:0/1080257719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d024b28f0 msgr2=0x562d024c06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:47.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.303+0000 7f9889c14700 1 --2- 172.21.15.7:0/1080257719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d024b28f0 0x562d024c06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.303+0000 7f9889c14700 1 -- 172.21.15.7:0/1080257719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d024b6c30 msgr2=0x562d024c0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:47.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.303+0000 7f9889c14700 1 --2- 172.21.15.7:0/1080257719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d024b6c30 0x562d024c0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.303+0000 7f9889c14700 1 -- 172.21.15.7:0/1080257719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d025359c0 con 0x562d024b1fd0 2024-06-10T01:12:47.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.304+0000 7f9889c14700 1 --2- 172.21.15.7:0/1080257719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d024b1fd0 0x562d024c0170 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f9880000c00 tx=0x7f9880009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8763ab0724b5f0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:47.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.304+0000 7f986d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:12:47.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.304+0000 7f986d7fa700 1 -- 172.21.15.7:0/1080257719 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880003c60 con 0x562d024b1fd0 2024-06-10T01:12:47.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.304+0000 7f986d7fa700 1 -- 172.21.15.7:0/1080257719 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f9880003e00 con 0x562d024b1fd0 2024-06-10T01:12:47.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.304+0000 7f98966fbb80 1 -- 172.21.15.7:0/1080257719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f987c09b8d0 con 0x562d024b1fd0 2024-06-10T01:12:47.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.304+0000 7f986d7fa700 1 -- 172.21.15.7:0/1080257719 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f98800056d0 con 0x562d024b1fd0 2024-06-10T01:12:47.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.304+0000 7f98966fbb80 10 client.17169 fetch_fsmap learned FSMap version 32 2024-06-10T01:12:47.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.304+0000 7f98966fbb80 1 -- 172.21.15.7:0/1080257719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562d02536cb0 con 0x562d024b1fd0 2024-06-10T01:12:47.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.304+0000 7f986d7fa700 1 -- 172.21.15.7:0/1080257719 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f987c09b8d0 con 0x562d024b1fd0 2024-06-10T01:12:47.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.304+0000 7f986d7fa700 1 -- 172.21.15.7:0/1080257719 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f988003c730 con 0x562d024b1fd0 2024-06-10T01:12:47.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.304+0000 7f98966fbb80 10 client.17169 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:12:47.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.304+0000 7f98966fbb80 10 client.17169 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:12:47.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.304+0000 7f98966fbb80 20 client.17169 populate_metadata read hostname 'smithi007' 2024-06-10T01:12:47.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.304+0000 7f98966fbb80 1 --2- 172.21.15.7:0/1080257719 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562d025373c0 0x562d025397e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:47.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.304+0000 7f98966fbb80 4 client.17169 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:12:47.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.304+0000 7f98966fbb80 1 -- 172.21.15.7:0/1080257719 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562d02407290 con 0x562d025373c0 2024-06-10T01:12:47.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.305+0000 7f9889413700 1 --2- 172.21.15.7:0/1080257719 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562d025373c0 0x562d025397e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:47.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.305+0000 7f986d7fa700 10 client.17169 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:47.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.305+0000 7f9889413700 1 --2- 172.21.15.7:0/1080257719 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562d025373c0 0x562d025397e0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad2c49d3b8975075 server_cookie=8a86e76d936e47ca in_seq=0 out_seq=0 2024-06-10T01:12:47.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.307+0000 7f986d7fa700 1 -- 172.21.15.7:0/1080257719 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562d02407290 con 0x562d025373c0 2024-06-10T01:12:47.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.307+0000 7f986d7fa700 10 client.17169 handle_command_reply: tid=0 2024-06-10T01:12:47.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.307+0000 7f98966fbb80 1 client.17169 shutdown 2024-06-10T01:12:47.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.307+0000 7f98966fbb80 1 -- 172.21.15.7:0/1080257719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d024b1fd0 msgr2=0x562d024c0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:47.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.307+0000 7f98966fbb80 1 --2- 172.21.15.7:0/1080257719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d024b1fd0 0x562d024c0170 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f9880000c00 tx=0x7f9880009310 comp rx=0 tx=0).stop 2024-06-10T01:12:47.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.307+0000 7f98966fbb80 1 -- 172.21.15.7:0/1080257719 shutdown_connections 2024-06-10T01:12:47.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.307+0000 7f98966fbb80 1 --2- 172.21.15.7:0/1080257719 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562d025373c0 0x562d025397e0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.307+0000 7f98966fbb80 1 --2- 172.21.15.7:0/1080257719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d024b6c30 0x562d024c0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.307+0000 7f98966fbb80 1 --2- 172.21.15.7:0/1080257719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d024b1fd0 0x562d024c0170 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.307+0000 7f986d7fa700 0 client.17169 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:47.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.307+0000 7f98966fbb80 1 --2- 172.21.15.7:0/1080257719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d024b28f0 0x562d024c06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.307+0000 7f98966fbb80 1 -- 172.21.15.7:0/1080257719 >> 172.21.15.7:0/1080257719 conn(0x562d023d8cc0 msgr2=0x562d024b5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:47.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.307+0000 7f98966fbb80 1 -- 172.21.15.7:0/1080257719 shutdown_connections 2024-06-10T01:12:47.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.307+0000 7f98966fbb80 1 -- 172.21.15.7:0/1080257719 wait complete. 2024-06-10T01:12:47.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.307+0000 7f98966fbb80 20 client.17169 trim_cache size 0 max 16384 2024-06-10T01:12:47.305 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:12:47.306 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-10T01:12:47.306 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-10T01:12:47.306 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:12:47.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.308+0000 7f988be78700 1 -- 172.21.15.7:0/795173532 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f98700a2c20 msgr2=0x7f98700a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:47.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.308+0000 7f988be78700 1 --2- 172.21.15.7:0/795173532 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f98700a2c20 0x7f98700a50e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f988000b2e0 tx=0x7f988001a040 comp rx=0 tx=0).stop 2024-06-10T01:12:47.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.308+0000 7f988be78700 1 -- 172.21.15.7:0/795173532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9884108560 msgr2=0x7f98841989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:47.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.308+0000 7f988be78700 1 --2- 172.21.15.7:0/795173532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9884108560 0x7f98841989f0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f987c00efb0 tx=0x7f987c00c3a0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.308+0000 7f988be78700 1 -- 172.21.15.7:0/795173532 shutdown_connections 2024-06-10T01:12:47.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.308+0000 7f988be78700 1 --2- 172.21.15.7:0/795173532 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f98700a2c20 0x7f98700a50e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.308+0000 7f988be78700 1 --2- 172.21.15.7:0/795173532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9884108560 0x7f98841989f0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.308+0000 7f988be78700 1 --2- 172.21.15.7:0/795173532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98841009e0 0x7f988419e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.308+0000 7f988be78700 1 --2- 172.21.15.7:0/795173532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98841013a0 0x7f988419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:47.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.308+0000 7f988be78700 1 -- 172.21.15.7:0/795173532 >> 172.21.15.7:0/795173532 conn(0x7f9884077990 msgr2=0x7f98840ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:47.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.308+0000 7f988be78700 1 -- 172.21.15.7:0/795173532 shutdown_connections 2024-06-10T01:12:47.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:47.308+0000 7f988be78700 1 -- 172.21.15.7:0/795173532 wait complete. 2024-06-10T01:12:47.579 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:47 smithi078 ceph-mon[36683]: pgmap v635: 129 pgs: 129 active+clean; 87 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.5 MiB/s wr, 310 op/s 2024-06-10T01:12:47.621 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:47 smithi007 ceph-mon[32398]: pgmap v635: 129 pgs: 129 active+clean; 87 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.5 MiB/s wr, 310 op/s 2024-06-10T01:12:47.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:47 smithi175 ceph-mon[36636]: pgmap v635: 129 pgs: 129 active+clean; 87 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.5 MiB/s wr, 310 op/s 2024-06-10T01:12:47.736 INFO:tasks.workunit.client.0.smithi007.stdout: 1 437748 6.83 MB/sec execute 208 sec latency 28.083 ms 2024-06-10T01:12:48.136 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:12:48.137 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:12:48.405 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:12:48.736 INFO:tasks.workunit.client.0.smithi007.stdout: 1 440247 6.86 MB/sec execute 209 sec latency 20.782 ms 2024-06-10T01:12:49.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:49 smithi007 ceph-mon[32398]: pgmap v636: 129 pgs: 129 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 307 op/s 2024-06-10T01:12:49.579 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:49 smithi078 ceph-mon[36683]: pgmap v636: 129 pgs: 129 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 307 op/s 2024-06-10T01:12:49.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:49 smithi175 ceph-mon[36636]: pgmap v636: 129 pgs: 129 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 307 op/s 2024-06-10T01:12:49.736 INFO:tasks.workunit.client.0.smithi007.stdout: 1 442184 6.86 MB/sec execute 210 sec latency 52.974 ms 2024-06-10T01:12:50.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.192+0000 7f35ef696700 1 -- 172.21.15.7:0/738443937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e81084f0 msgr2=0x7f35e81088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:50.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.192+0000 7f35ef696700 1 --2- 172.21.15.7:0/738443937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e81084f0 0x7f35e81088d0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009990 tx=0x7f35e4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:12:50.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.192+0000 7f35ef696700 1 -- 172.21.15.7:0/738443937 shutdown_connections 2024-06-10T01:12:50.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.192+0000 7f35ef696700 1 --2- 172.21.15.7:0/738443937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35e806b3c0 0x7f35e810f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:50.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.192+0000 7f35ef696700 1 --2- 172.21.15.7:0/738443937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e81084f0 0x7f35e81088d0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:50.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.192+0000 7f35ef696700 1 --2- 172.21.15.7:0/738443937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35e806aa00 0x7f35e806ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:50.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.192+0000 7f35ef696700 1 -- 172.21.15.7:0/738443937 >> 172.21.15.7:0/738443937 conn(0x7f35e8078300 msgr2=0x7f35e8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:50.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.192+0000 7f35ef696700 1 -- 172.21.15.7:0/738443937 shutdown_connections 2024-06-10T01:12:50.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.192+0000 7f35ef696700 1 -- 172.21.15.7:0/738443937 wait complete. 2024-06-10T01:12:50.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.193+0000 7f35ef696700 1 Processor -- start 2024-06-10T01:12:50.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.193+0000 7f35ef696700 1 -- start start 2024-06-10T01:12:50.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.193+0000 7f35ef696700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35e806aa00 0x7f35e819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:50.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.193+0000 7f35ef696700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35e806b3c0 0x7f35e819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:50.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.193+0000 7f35ef696700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e81084f0 0x7f35e81989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:50.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.193+0000 7f35ef696700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f35e8112770 con 0x7f35e81084f0 2024-06-10T01:12:50.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.193+0000 7f35ef696700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f35e81125f0 con 0x7f35e806aa00 2024-06-10T01:12:50.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.193+0000 7f35ef696700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f35e81128f0 con 0x7f35e806b3c0 2024-06-10T01:12:50.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.193+0000 7f35edc33700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e81084f0 0x7f35e81989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:50.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.193+0000 7f35ed432700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35e806aa00 0x7f35e819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:50.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.193+0000 7f35ed432700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35e806aa00 0x7f35e819e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:58412/0 (socket says 172.21.15.7:58412) 2024-06-10T01:12:50.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.193+0000 7f35edc33700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e81084f0 0x7f35e81989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48632/0 (socket says 172.21.15.7:48632) 2024-06-10T01:12:50.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.193+0000 7f35ed432700 1 -- 172.21.15.7:0/920504220 learned_addr learned my addr 172.21.15.7:0/920504220 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:50.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.193+0000 7f35ecc31700 1 --2- 172.21.15.7:0/920504220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35e806b3c0 0x7f35e819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:50.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.194+0000 7f35ed432700 1 -- 172.21.15.7:0/920504220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35e806b3c0 msgr2=0x7f35e819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:50.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.194+0000 7f35ed432700 1 --2- 172.21.15.7:0/920504220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35e806b3c0 0x7f35e819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:50.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.194+0000 7f35ed432700 1 -- 172.21.15.7:0/920504220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e81084f0 msgr2=0x7f35e81989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:50.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.194+0000 7f35ed432700 1 --2- 172.21.15.7:0/920504220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e81084f0 0x7f35e81989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:50.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.194+0000 7f35ed432700 1 -- 172.21.15.7:0/920504220 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e8199150 con 0x7f35e806aa00 2024-06-10T01:12:50.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.194+0000 7f35edc33700 1 --2- 172.21.15.7:0/920504220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e81084f0 0x7f35e81989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:50.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.194+0000 7f35ed432700 1 --2- 172.21.15.7:0/920504220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35e806aa00 0x7f35e819e830 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f35e4000c00 tx=0x7f35e400bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:50.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.194+0000 7f35de7fc700 1 -- 172.21.15.7:0/920504220 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e4027070 con 0x7f35e806aa00 2024-06-10T01:12:50.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.194+0000 7f35ef696700 1 -- 172.21.15.7:0/920504220 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e81993e0 con 0x7f35e806aa00 2024-06-10T01:12:50.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.194+0000 7f35ef696700 1 -- 172.21.15.7:0/920504220 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e8101380 con 0x7f35e806aa00 2024-06-10T01:12:50.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.195+0000 7f35de7fc700 1 -- 172.21.15.7:0/920504220 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35e40043d0 con 0x7f35e806aa00 2024-06-10T01:12:50.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.195+0000 7f35de7fc700 1 -- 172.21.15.7:0/920504220 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e4022ad0 con 0x7f35e806aa00 2024-06-10T01:12:50.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.197+0000 7f35ef696700 1 -- 172.21.15.7:0/920504220 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35e810a9d0 con 0x7f35e806aa00 2024-06-10T01:12:50.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.199+0000 7f35de7fc700 1 -- 172.21.15.7:0/920504220 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f35e402a070 con 0x7f35e806aa00 2024-06-10T01:12:50.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.201+0000 7f35de7fc700 1 --2- 172.21.15.7:0/920504220 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35d40ab530 0x7f35d40ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:50.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.201+0000 7f35de7fc700 1 -- 172.21.15.7:0/920504220 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f35e40d0e90 con 0x7f35e806aa00 2024-06-10T01:12:50.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.201+0000 7f35ecc31700 1 --2- 172.21.15.7:0/920504220 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35d40ab530 0x7f35d40ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:50.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.203+0000 7f35ecc31700 1 --2- 172.21.15.7:0/920504220 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35d40ab530 0x7f35d40ad9f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f35e8078890 tx=0x7f35d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:50.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.206+0000 7f35de7fc700 1 -- 172.21.15.7:0/920504220 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f35e4031030 con 0x7f35e806aa00 2024-06-10T01:12:50.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.469+0000 7f35ef696700 1 -- 172.21.15.7:0/920504220 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35e8199f20 con 0x7f35e806aa00 2024-06-10T01:12:50.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.470+0000 7f35de7fc700 1 -- 172.21.15.7:0/920504220 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11855 (secure 0 0 0) 0x7f35e4013070 con 0x7f35e806aa00 2024-06-10T01:12:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:12:50.465 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:10:42.213996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:12:50.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.472+0000 7f35ef696700 1 -- 172.21.15.7:0/920504220 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35d40ab530 msgr2=0x7f35d40ad9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:50.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.472+0000 7f35ef696700 1 --2- 172.21.15.7:0/920504220 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35d40ab530 0x7f35d40ad9f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f35e8078890 tx=0x7f35d8009260 comp rx=0 tx=0).stop 2024-06-10T01:12:50.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.472+0000 7f35ef696700 1 -- 172.21.15.7:0/920504220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35e806aa00 msgr2=0x7f35e819e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:50.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.472+0000 7f35ef696700 1 --2- 172.21.15.7:0/920504220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35e806aa00 0x7f35e819e830 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f35e4000c00 tx=0x7f35e400bab0 comp rx=0 tx=0).stop 2024-06-10T01:12:50.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.472+0000 7f35ef696700 1 -- 172.21.15.7:0/920504220 shutdown_connections 2024-06-10T01:12:50.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.472+0000 7f35ef696700 1 --2- 172.21.15.7:0/920504220 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35d40ab530 0x7f35d40ad9f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:50.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.472+0000 7f35ef696700 1 --2- 172.21.15.7:0/920504220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e81084f0 0x7f35e81989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:50.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.472+0000 7f35ef696700 1 --2- 172.21.15.7:0/920504220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35e806aa00 0x7f35e819e830 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:50.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.472+0000 7f35ef696700 1 --2- 172.21.15.7:0/920504220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35e806b3c0 0x7f35e819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:50.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.472+0000 7f35ef696700 1 -- 172.21.15.7:0/920504220 >> 172.21.15.7:0/920504220 conn(0x7f35e8078300 msgr2=0x7f35e80fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:50.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.472+0000 7f35ef696700 1 -- 172.21.15.7:0/920504220 shutdown_connections 2024-06-10T01:12:50.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:50.472+0000 7f35ef696700 1 -- 172.21.15.7:0/920504220 wait complete. 2024-06-10T01:12:50.468 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 32 2024-06-10T01:12:50.736 INFO:tasks.workunit.client.0.smithi007.stdout: 1 443968 6.89 MB/sec execute 211 sec latency 17.972 ms 2024-06-10T01:12:51.338 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:12:51.579 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:51 smithi078 ceph-mon[36683]: pgmap v637: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 348 op/s 2024-06-10T01:12:51.579 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:51 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/920504220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:12:51.595 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:12:51.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:51 smithi175 ceph-mon[36636]: pgmap v637: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 348 op/s 2024-06-10T01:12:51.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:51 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/920504220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:12:51.717 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:51 smithi007 ceph-mon[32398]: pgmap v637: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 348 op/s 2024-06-10T01:12:51.717 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:51 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/920504220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:12:51.736 INFO:tasks.workunit.client.0.smithi007.stdout: 1 446611 6.92 MB/sec execute 212 sec latency 15.787 ms 2024-06-10T01:12:52.736 INFO:tasks.workunit.client.0.smithi007.stdout: 1 448125 6.92 MB/sec execute 213 sec latency 31.621 ms 2024-06-10T01:12:53.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.420+0000 7f00a643e700 1 -- 172.21.15.7:0/3429017198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00a0101d50 msgr2=0x7f00a010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:53.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.420+0000 7f00a643e700 1 --2- 172.21.15.7:0/3429017198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00a0101d50 0x7f00a010e240 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f009800b0a0 tx=0x7f009800b3b0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.420+0000 7f00a643e700 1 -- 172.21.15.7:0/3429017198 shutdown_connections 2024-06-10T01:12:53.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.420+0000 7f00a643e700 1 --2- 172.21.15.7:0/3429017198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00a0101d50 0x7f00a010e240 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.420+0000 7f00a643e700 1 --2- 172.21.15.7:0/3429017198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00a0107400 0x7f00a01077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.420+0000 7f00a643e700 1 --2- 172.21.15.7:0/3429017198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00a0101390 0x7f00a0101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.420+0000 7f00a643e700 1 -- 172.21.15.7:0/3429017198 >> 172.21.15.7:0/3429017198 conn(0x7f00a0078340 msgr2=0x7f00a0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:53.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.420+0000 7f00a643e700 1 -- 172.21.15.7:0/3429017198 shutdown_connections 2024-06-10T01:12:53.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.420+0000 7f00a643e700 1 -- 172.21.15.7:0/3429017198 wait complete. 2024-06-10T01:12:53.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.421+0000 7f00a643e700 1 Processor -- start 2024-06-10T01:12:53.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.421+0000 7f00a643e700 1 -- start start 2024-06-10T01:12:53.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.421+0000 7f00a643e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00a0101390 0x7f00a019e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.421+0000 7f00a643e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00a0101d50 0x7f00a019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.421+0000 7f00a643e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00a0107400 0x7f00a0198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.421+0000 7f00a643e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f00a0111690 con 0x7f00a0101390 2024-06-10T01:12:53.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.421+0000 7f00a643e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f00a0111510 con 0x7f00a0101d50 2024-06-10T01:12:53.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.421+0000 7f00a643e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f00a0111810 con 0x7f00a0107400 2024-06-10T01:12:53.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.422+0000 7f009ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00a0101390 0x7f00a019e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.422+0000 7f009ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00a0101390 0x7f00a019e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44928/0 (socket says 172.21.15.7:44928) 2024-06-10T01:12:53.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.422+0000 7f00a49db700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00a0107400 0x7f00a0198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.422+0000 7f009ffff700 1 -- 172.21.15.7:0/945502876 learned_addr learned my addr 172.21.15.7:0/945502876 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:53.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.422+0000 7f009f7fe700 1 --2- 172.21.15.7:0/945502876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00a0101d50 0x7f00a019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.422+0000 7f00a49db700 1 -- 172.21.15.7:0/945502876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00a0101d50 msgr2=0x7f00a019ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:53.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.422+0000 7f00a49db700 1 --2- 172.21.15.7:0/945502876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00a0101d50 0x7f00a019ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.422+0000 7f00a49db700 1 -- 172.21.15.7:0/945502876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00a0101390 msgr2=0x7f00a019e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:53.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.422+0000 7f00a49db700 1 --2- 172.21.15.7:0/945502876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00a0101390 0x7f00a019e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.422+0000 7f00a49db700 1 -- 172.21.15.7:0/945502876 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a0199170 con 0x7f00a0107400 2024-06-10T01:12:53.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.422+0000 7f009ffff700 1 --2- 172.21.15.7:0/945502876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00a0101390 0x7f00a019e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:53.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.422+0000 7f00a49db700 1 --2- 172.21.15.7:0/945502876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00a0107400 0x7f00a0198a10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f009800b0a0 tx=0x7f0098004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:53.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.422+0000 7f009d7fa700 1 -- 172.21.15.7:0/945502876 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0098021070 con 0x7f00a0107400 2024-06-10T01:12:53.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.423+0000 7f00a643e700 1 -- 172.21.15.7:0/945502876 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00a0199400 con 0x7f00a0107400 2024-06-10T01:12:53.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.423+0000 7f009d7fa700 1 -- 172.21.15.7:0/945502876 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0098007b80 con 0x7f00a0107400 2024-06-10T01:12:53.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.423+0000 7f009d7fa700 1 -- 172.21.15.7:0/945502876 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0098005040 con 0x7f00a0107400 2024-06-10T01:12:53.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.423+0000 7f00a643e700 1 -- 172.21.15.7:0/945502876 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00a01a56d0 con 0x7f00a0107400 2024-06-10T01:12:53.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.425+0000 7f009d7fa700 1 -- 172.21.15.7:0/945502876 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0098024070 con 0x7f00a0107400 2024-06-10T01:12:53.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.426+0000 7f009d7fa700 1 --2- 172.21.15.7:0/945502876 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f00880a2c20 0x7f00880a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.426+0000 7f009ffff700 1 --2- 172.21.15.7:0/945502876 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f00880a2c20 0x7f00880a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.426+0000 7f009d7fa700 1 -- 172.21.15.7:0/945502876 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f009800f020 con 0x7f00a0107400 2024-06-10T01:12:53.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.426+0000 7f00b0cc1b80 1 Processor -- start 2024-06-10T01:12:53.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.426+0000 7f00b0cc1b80 1 -- start start 2024-06-10T01:12:53.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.426+0000 7f009ffff700 1 --2- 172.21.15.7:0/945502876 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f00880a2c20 0x7f00880a50e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f00a0106c80 tx=0x7f0090009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:53.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.426+0000 7f00b0cc1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15f54bf60 0x55f15f54c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.427+0000 7f00b0cc1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f15f54c880 0x55f15f546260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.427+0000 7f00b0cc1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f15f5467a0 0x55f15f546bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.427+0000 7f00b0cc1b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f15f438f60 con 0x55f15f54bf60 2024-06-10T01:12:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.427+0000 7f00b0cc1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f15f497850 con 0x55f15f5467a0 2024-06-10T01:12:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.427+0000 7f00b0cc1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f15f3ff460 con 0x55f15f54c880 2024-06-10T01:12:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.427+0000 7f009ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15f54bf60 0x55f15f54c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.427+0000 7f009ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15f54bf60 0x55f15f54c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44930/0 (socket says 172.21.15.7:44930) 2024-06-10T01:12:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.427+0000 7f009ffff700 1 -- 172.21.15.7:0/124141012 learned_addr learned my addr 172.21.15.7:0/124141012 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.427+0000 7f009f7fe700 1 --2- 172.21.15.7:0/124141012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f15f54c880 0x55f15f546260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.427+0000 7f00a49db700 1 --2- 172.21.15.7:0/124141012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f15f5467a0 0x55f15f546bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.427+0000 7f009f7fe700 1 -- 172.21.15.7:0/124141012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f15f5467a0 msgr2=0x55f15f546bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.427+0000 7f009f7fe700 1 --2- 172.21.15.7:0/124141012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f15f5467a0 0x55f15f546bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.427+0000 7f009f7fe700 1 -- 172.21.15.7:0/124141012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15f54bf60 msgr2=0x55f15f54c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.427+0000 7f009f7fe700 1 --2- 172.21.15.7:0/124141012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15f54bf60 0x55f15f54c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.427+0000 7f009f7fe700 1 -- 172.21.15.7:0/124141012 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f15f3e68d0 con 0x55f15f54c880 2024-06-10T01:12:53.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.427+0000 7f009ffff700 1 --2- 172.21.15.7:0/124141012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15f54bf60 0x55f15f54c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:53.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.427+0000 7f00a49db700 1 --2- 172.21.15.7:0/124141012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f15f5467a0 0x55f15f546bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:53.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.427+0000 7f009f7fe700 1 --2- 172.21.15.7:0/124141012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f15f54c880 0x55f15f546260 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0094009c80 tx=0x7f0094009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cceb19808a058354 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:53.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.427+0000 7f00867fc700 1 -- 172.21.15.7:0/124141012 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0094011cb0 con 0x55f15f54c880 2024-06-10T01:12:53.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.427+0000 7f00867fc700 1 -- 172.21.15.7:0/124141012 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0094011e50 con 0x55f15f54c880 2024-06-10T01:12:53.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.427+0000 7f00867fc700 1 -- 172.21.15.7:0/124141012 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0094005310 con 0x55f15f54c880 2024-06-10T01:12:53.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.428+0000 7f00b0cc1b80 1 -- 172.21.15.7:0/124141012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f15f54c880 msgr2=0x55f15f546260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:53.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.428+0000 7f00b0cc1b80 1 --2- 172.21.15.7:0/124141012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f15f54c880 0x55f15f546260 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0094009c80 tx=0x7f0094009f90 comp rx=0 tx=0).stop 2024-06-10T01:12:53.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.428+0000 7f00b0cc1b80 1 -- 172.21.15.7:0/124141012 shutdown_connections 2024-06-10T01:12:53.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.428+0000 7f00b0cc1b80 1 --2- 172.21.15.7:0/124141012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f15f5467a0 0x55f15f546bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.428+0000 7f00b0cc1b80 1 --2- 172.21.15.7:0/124141012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15f54bf60 0x55f15f54c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.428+0000 7f00b0cc1b80 1 --2- 172.21.15.7:0/124141012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f15f54c880 0x55f15f546260 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.428+0000 7f00b0cc1b80 1 -- 172.21.15.7:0/124141012 >> 172.21.15.7:0/124141012 conn(0x55f15f473080 msgr2=0x55f15f541220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:53.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.428+0000 7f00b0cc1b80 1 -- 172.21.15.7:0/124141012 shutdown_connections 2024-06-10T01:12:53.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.428+0000 7f00b0cc1b80 1 -- 172.21.15.7:0/124141012 wait complete. 2024-06-10T01:12:53.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.429+0000 7f00b0cc1b80 1 -- start start 2024-06-10T01:12:53.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.429+0000 7f00b0cc1b80 1 Processor -- start 2024-06-10T01:12:53.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.429+0000 7f00b0cc1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f15f5467a0 0x55f15f552610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.429+0000 7f00b0cc1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15f54bf60 0x55f15f54f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.429+0000 7f00b0cc1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f15f54c880 0x55f15f54fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.429+0000 7f00b0cc1b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f15f437fc0 con 0x55f15f54bf60 2024-06-10T01:12:53.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.429+0000 7f00b0cc1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f15f493810 con 0x55f15f54c880 2024-06-10T01:12:53.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.429+0000 7f00b0cc1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f15f553260 con 0x55f15f5467a0 2024-06-10T01:12:53.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.429+0000 7f00b0cc1b80 10 client.0 _ll_register_callbacks cb 0x55f15f42d5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:12:53.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.429+0000 7f009ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f15f5467a0 0x55f15f552610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.429+0000 7f009f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15f54bf60 0x55f15f54f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.429+0000 7f009ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f15f5467a0 0x55f15f552610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:42030/0 (socket says 172.21.15.7:42030) 2024-06-10T01:12:53.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.429+0000 7f009ffff700 1 -- 172.21.15.7:0/1174691384 learned_addr learned my addr 172.21.15.7:0/1174691384 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:53.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.430+0000 7f00a49db700 1 --2- 172.21.15.7:0/1174691384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f15f54c880 0x55f15f54fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.430+0000 7f009ffff700 1 -- 172.21.15.7:0/1174691384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f15f54c880 msgr2=0x55f15f54fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:53.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.430+0000 7f009ffff700 1 --2- 172.21.15.7:0/1174691384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f15f54c880 0x55f15f54fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.430+0000 7f009ffff700 1 -- 172.21.15.7:0/1174691384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15f54bf60 msgr2=0x55f15f54f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:53.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.430+0000 7f009ffff700 1 --2- 172.21.15.7:0/1174691384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15f54bf60 0x55f15f54f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.430+0000 7f009ffff700 1 -- 172.21.15.7:0/1174691384 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f15f45d750 con 0x55f15f5467a0 2024-06-10T01:12:53.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.430+0000 7f009f7fe700 1 --2- 172.21.15.7:0/1174691384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15f54bf60 0x55f15f54f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:53.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.430+0000 7f009ffff700 1 --2- 172.21.15.7:0/1174691384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f15f5467a0 0x55f15f552610 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f009000fef0 tx=0x7f00900047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fd8adde759c7651 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:53.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.430+0000 7f007f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:12:53.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.430+0000 7f007f7fe700 1 -- 172.21.15.7:0/1174691384 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009001fdb0 con 0x55f15f5467a0 2024-06-10T01:12:53.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.430+0000 7f007f7fe700 1 -- 172.21.15.7:0/1174691384 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f00900273e0 con 0x55f15f5467a0 2024-06-10T01:12:53.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.430+0000 7f00b0cc1b80 1 -- 172.21.15.7:0/1174691384 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0094011e50 con 0x55f15f5467a0 2024-06-10T01:12:53.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.431+0000 7f007f7fe700 1 -- 172.21.15.7:0/1174691384 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f00900275e0 con 0x55f15f5467a0 2024-06-10T01:12:53.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.431+0000 7f00b0cc1b80 10 client.26614 fetch_fsmap learned FSMap version 32 2024-06-10T01:12:53.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.431+0000 7f00b0cc1b80 1 -- 172.21.15.7:0/1174691384 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f15f55da70 con 0x55f15f5467a0 2024-06-10T01:12:53.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.431+0000 7f007f7fe700 1 -- 172.21.15.7:0/1174691384 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0090027c60 con 0x55f15f5467a0 2024-06-10T01:12:53.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.431+0000 7f007f7fe700 1 -- 172.21.15.7:0/1174691384 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f0090026070 con 0x55f15f5467a0 2024-06-10T01:12:53.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.431+0000 7f00b0cc1b80 10 client.26614 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:12:53.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.431+0000 7f00b0cc1b80 10 client.26614 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:12:53.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.431+0000 7f00b0cc1b80 20 client.26614 populate_metadata read hostname 'smithi007' 2024-06-10T01:12:53.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.431+0000 7f00b0cc1b80 1 --2- 172.21.15.7:0/1174691384 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55f15f560100 0x55f15f562510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.431+0000 7f00b0cc1b80 4 client.26614 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:12:53.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.431+0000 7f00b0cc1b80 1 -- 172.21.15.7:0/1174691384 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0094011e50 con 0x55f15f560100 2024-06-10T01:12:53.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.432+0000 7f009f7fe700 1 --2- 172.21.15.7:0/1174691384 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55f15f560100 0x55f15f562510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.432+0000 7f009f7fe700 1 --2- 172.21.15.7:0/1174691384 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55f15f560100 0x55f15f562510 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f6abe8f6c5eb8b8f server_cookie=80cb202b2c2f9d15 in_seq=0 out_seq=0 2024-06-10T01:12:53.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.432+0000 7f007f7fe700 10 client.26614 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:12:53.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.433+0000 7f007f7fe700 1 -- 172.21.15.7:0/1174691384 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f0094011e50 con 0x55f15f560100 2024-06-10T01:12:53.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.433+0000 7f007f7fe700 10 client.26614 handle_command_reply: tid=0 2024-06-10T01:12:53.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.433+0000 7f00b0cc1b80 1 client.26614 shutdown 2024-06-10T01:12:53.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.433+0000 7f00b0cc1b80 1 -- 172.21.15.7:0/1174691384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f15f5467a0 msgr2=0x55f15f552610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:53.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.433+0000 7f00b0cc1b80 1 --2- 172.21.15.7:0/1174691384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f15f5467a0 0x55f15f552610 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f009000fef0 tx=0x7f00900047c0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.433+0000 7f00b0cc1b80 1 -- 172.21.15.7:0/1174691384 shutdown_connections 2024-06-10T01:12:53.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.433+0000 7f00b0cc1b80 1 --2- 172.21.15.7:0/1174691384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f15f54c880 0x55f15f54fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.433+0000 7f00b0cc1b80 1 --2- 172.21.15.7:0/1174691384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f15f5467a0 0x55f15f552610 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.433+0000 7f00b0cc1b80 1 --2- 172.21.15.7:0/1174691384 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55f15f560100 0x55f15f562510 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.433+0000 7f00b0cc1b80 1 --2- 172.21.15.7:0/1174691384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15f54bf60 0x55f15f54f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.433+0000 7f007f7fe700 0 client.26614 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:12:53.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.433+0000 7f00b0cc1b80 1 -- 172.21.15.7:0/1174691384 >> 172.21.15.7:0/1174691384 conn(0x55f15f473080 msgr2=0x55f15f540700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:53.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.433+0000 7f00b0cc1b80 1 -- 172.21.15.7:0/1174691384 shutdown_connections 2024-06-10T01:12:53.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.433+0000 7f00b0cc1b80 1 -- 172.21.15.7:0/1174691384 wait complete. 2024-06-10T01:12:53.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.433+0000 7f00b0cc1b80 20 client.26614 trim_cache size 0 max 16384 2024-06-10T01:12:53.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.449+0000 7f00b0cc1b80 1 Processor -- start 2024-06-10T01:12:53.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.449+0000 7f00b0cc1b80 1 -- start start 2024-06-10T01:12:53.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.449+0000 7f00b0cc1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15f5467a0 0x55f15f55c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.449+0000 7f00b0cc1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f15f54bf60 0x55f15f54d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.449+0000 7f00b0cc1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f15f54c880 0x55f15f540e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.449+0000 7f00b0cc1b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f00a01ae8d0 con 0x55f15f5467a0 2024-06-10T01:12:53.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.449+0000 7f00b0cc1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f00a01ac160 con 0x55f15f54c880 2024-06-10T01:12:53.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.449+0000 7f00b0cc1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f00a01a9fb0 con 0x55f15f54bf60 2024-06-10T01:12:53.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.449+0000 7f009ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15f5467a0 0x55f15f55c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.449+0000 7f009f7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f15f54bf60 0x55f15f54d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.449+0000 7f00a49db700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f15f54c880 0x55f15f540e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.449+0000 7f009f7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f15f54bf60 0x55f15f54d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:42036/0 (socket says 172.21.15.7:42036) 2024-06-10T01:12:53.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.449+0000 7f009f7fe700 1 -- 172.21.15.7:0/281312378 learned_addr learned my addr 172.21.15.7:0/281312378 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:53.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.449+0000 7f009f7fe700 1 -- 172.21.15.7:0/281312378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f15f54c880 msgr2=0x55f15f540e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:53.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.449+0000 7f009f7fe700 1 --2- 172.21.15.7:0/281312378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f15f54c880 0x55f15f540e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.449+0000 7f009f7fe700 1 -- 172.21.15.7:0/281312378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15f5467a0 msgr2=0x55f15f55c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:53.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.449+0000 7f009f7fe700 1 --2- 172.21.15.7:0/281312378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15f5467a0 0x55f15f55c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.449+0000 7f009f7fe700 1 -- 172.21.15.7:0/281312378 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f15f54dec0 con 0x55f15f54bf60 2024-06-10T01:12:53.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.449+0000 7f00a49db700 1 --2- 172.21.15.7:0/281312378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f15f54c880 0x55f15f540e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:53.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.449+0000 7f009ffff700 1 --2- 172.21.15.7:0/281312378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15f5467a0 0x55f15f55c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:53.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.450+0000 7f009f7fe700 1 --2- 172.21.15.7:0/281312378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f15f54bf60 0x55f15f54d370 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0094004e10 tx=0x7f0094011f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64a2dae5ed18664f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:53.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.450+0000 7f0085ffb700 1 -- 172.21.15.7:0/281312378 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0094004040 con 0x55f15f54bf60 2024-06-10T01:12:53.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.450+0000 7f0085ffb700 1 -- 172.21.15.7:0/281312378 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0094003680 con 0x55f15f54bf60 2024-06-10T01:12:53.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.450+0000 7f0085ffb700 1 -- 172.21.15.7:0/281312378 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00940050a0 con 0x55f15f54bf60 2024-06-10T01:12:53.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.451+0000 7f00b0cc1b80 1 -- 172.21.15.7:0/281312378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f15f54bf60 msgr2=0x55f15f54d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:53.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.451+0000 7f00b0cc1b80 1 --2- 172.21.15.7:0/281312378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f15f54bf60 0x55f15f54d370 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0094004e10 tx=0x7f0094011f50 comp rx=0 tx=0).stop 2024-06-10T01:12:53.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.451+0000 7f00b0cc1b80 1 -- 172.21.15.7:0/281312378 shutdown_connections 2024-06-10T01:12:53.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.451+0000 7f00b0cc1b80 1 --2- 172.21.15.7:0/281312378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f15f54c880 0x55f15f540e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.451+0000 7f00b0cc1b80 1 --2- 172.21.15.7:0/281312378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15f5467a0 0x55f15f55c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.451+0000 7f00b0cc1b80 1 --2- 172.21.15.7:0/281312378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f15f54bf60 0x55f15f54d370 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.451+0000 7f00b0cc1b80 1 -- 172.21.15.7:0/281312378 >> 172.21.15.7:0/281312378 conn(0x55f15f473080 msgr2=0x55f15f54cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:53.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.451+0000 7f00b0cc1b80 1 -- 172.21.15.7:0/281312378 shutdown_connections 2024-06-10T01:12:53.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.451+0000 7f00b0cc1b80 1 -- 172.21.15.7:0/281312378 wait complete. 2024-06-10T01:12:53.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.451+0000 7f00b0cc1b80 1 -- start start 2024-06-10T01:12:53.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.451+0000 7f00b0cc1b80 1 Processor -- start 2024-06-10T01:12:53.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.451+0000 7f00b0cc1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f15f5467a0 0x55f15f558860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.451+0000 7f00b0cc1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f15f54bf60 0x55f15f558da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.452+0000 7f00b0cc1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15f54c880 0x55f15f567510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.452+0000 7f00b0cc1b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f00a01aa620 con 0x55f15f54c880 2024-06-10T01:12:53.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.452+0000 7f00b0cc1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f00a01aba90 con 0x55f15f54bf60 2024-06-10T01:12:53.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.452+0000 7f00b0cc1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f00a01aacf0 con 0x55f15f5467a0 2024-06-10T01:12:53.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.452+0000 7f00b0cc1b80 10 client.0 _ll_register_callbacks cb 0x55f15f42d5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:12:53.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.452+0000 7f00a49db700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15f54c880 0x55f15f567510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.452+0000 7f009ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f15f5467a0 0x55f15f558860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.452+0000 7f00a49db700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15f54c880 0x55f15f567510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44942/0 (socket says 172.21.15.7:44942) 2024-06-10T01:12:53.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.452+0000 7f00a49db700 1 -- 172.21.15.7:0/3087365238 learned_addr learned my addr 172.21.15.7:0/3087365238 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:53.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.452+0000 7f009f7fe700 1 --2- 172.21.15.7:0/3087365238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f15f54bf60 0x55f15f558da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.452+0000 7f009ffff700 1 -- 172.21.15.7:0/3087365238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f15f54bf60 msgr2=0x55f15f558da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:53.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.452+0000 7f009ffff700 1 --2- 172.21.15.7:0/3087365238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f15f54bf60 0x55f15f558da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.452+0000 7f009ffff700 1 -- 172.21.15.7:0/3087365238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15f54c880 msgr2=0x55f15f567510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:53.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.452+0000 7f009ffff700 1 --2- 172.21.15.7:0/3087365238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15f54c880 0x55f15f567510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.452+0000 7f009ffff700 1 -- 172.21.15.7:0/3087365238 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f15f567d50 con 0x55f15f5467a0 2024-06-10T01:12:53.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.452+0000 7f00a49db700 1 --2- 172.21.15.7:0/3087365238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15f54c880 0x55f15f567510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:53.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.452+0000 7f009ffff700 1 --2- 172.21.15.7:0/3087365238 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f15f5467a0 0x55f15f558860 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0090016040 tx=0x7f00900289f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1af880e975cd1d50 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:53.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.452+0000 7f007f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:12:53.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.453+0000 7f007f7fe700 1 -- 172.21.15.7:0/3087365238 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009001f9f0 con 0x55f15f5467a0 2024-06-10T01:12:53.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.453+0000 7f007f7fe700 1 -- 172.21.15.7:0/3087365238 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f00900273e0 con 0x55f15f5467a0 2024-06-10T01:12:53.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.453+0000 7f00b0cc1b80 1 -- 172.21.15.7:0/3087365238 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0094003680 con 0x55f15f5467a0 2024-06-10T01:12:53.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.453+0000 7f007f7fe700 1 -- 172.21.15.7:0/3087365238 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0090045050 con 0x55f15f5467a0 2024-06-10T01:12:53.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.453+0000 7f00b0cc1b80 10 client.26620 fetch_fsmap learned FSMap version 32 2024-06-10T01:12:53.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.453+0000 7f00b0cc1b80 1 -- 172.21.15.7:0/3087365238 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f15f5caa10 con 0x55f15f5467a0 2024-06-10T01:12:53.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.453+0000 7f007f7fe700 1 -- 172.21.15.7:0/3087365238 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0094003680 con 0x55f15f5467a0 2024-06-10T01:12:53.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.453+0000 7f007f7fe700 1 -- 172.21.15.7:0/3087365238 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f0090026040 con 0x55f15f5467a0 2024-06-10T01:12:53.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.453+0000 7f00b0cc1b80 10 client.26620 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:12:53.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.453+0000 7f00b0cc1b80 10 client.26620 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:12:53.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.453+0000 7f00b0cc1b80 20 client.26620 populate_metadata read hostname 'smithi007' 2024-06-10T01:12:53.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.453+0000 7f00b0cc1b80 1 --2- 172.21.15.7:0/3087365238 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55f15f5cb0e0 0x55f15f5cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.453+0000 7f00b0cc1b80 4 client.26620 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:12:53.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.453+0000 7f00b0cc1b80 1 -- 172.21.15.7:0/3087365238 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f15f4a4d50 con 0x55f15f5cb0e0 2024-06-10T01:12:53.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.453+0000 7f009f7fe700 1 --2- 172.21.15.7:0/3087365238 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55f15f5cb0e0 0x55f15f5cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.454+0000 7f009f7fe700 1 --2- 172.21.15.7:0/3087365238 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55f15f5cb0e0 0x55f15f5cd500 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=251d98c6dfbc663f server_cookie=b85c2ddbf61f9fe4 in_seq=0 out_seq=0 2024-06-10T01:12:53.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.454+0000 7f007f7fe700 10 client.26620 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:12:53.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.454+0000 7f007f7fe700 1 -- 172.21.15.7:0/3087365238 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f15f4a4d50 con 0x55f15f5cb0e0 2024-06-10T01:12:53.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.454+0000 7f007f7fe700 10 client.26620 handle_command_reply: tid=0 2024-06-10T01:12:53.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.454+0000 7f00b0cc1b80 1 client.26620 shutdown 2024-06-10T01:12:53.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.454+0000 7f00b0cc1b80 1 -- 172.21.15.7:0/3087365238 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f15f5467a0 msgr2=0x55f15f558860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:53.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.454+0000 7f00b0cc1b80 1 --2- 172.21.15.7:0/3087365238 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f15f5467a0 0x55f15f558860 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0090016040 tx=0x7f00900289f0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.454+0000 7f00b0cc1b80 1 -- 172.21.15.7:0/3087365238 shutdown_connections 2024-06-10T01:12:53.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.454+0000 7f00b0cc1b80 1 --2- 172.21.15.7:0/3087365238 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55f15f5cb0e0 0x55f15f5cd500 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.454+0000 7f007f7fe700 0 client.26620 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:12:53.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.454+0000 7f00b0cc1b80 1 --2- 172.21.15.7:0/3087365238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15f54c880 0x55f15f567510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.454+0000 7f00b0cc1b80 1 --2- 172.21.15.7:0/3087365238 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f15f5467a0 0x55f15f558860 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.454+0000 7f00b0cc1b80 1 --2- 172.21.15.7:0/3087365238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f15f54bf60 0x55f15f558da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.454+0000 7f00b0cc1b80 1 -- 172.21.15.7:0/3087365238 >> 172.21.15.7:0/3087365238 conn(0x55f15f473080 msgr2=0x55f15f54aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:53.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.454+0000 7f00b0cc1b80 1 -- 172.21.15.7:0/3087365238 shutdown_connections 2024-06-10T01:12:53.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.455+0000 7f00b0cc1b80 1 -- 172.21.15.7:0/3087365238 wait complete. 2024-06-10T01:12:53.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.455+0000 7f00b0cc1b80 20 client.26620 trim_cache size 0 max 16384 2024-06-10T01:12:53.455 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:12:53.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.455+0000 7f00a643e700 1 -- 172.21.15.7:0/945502876 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f00880a2c20 msgr2=0x7f00880a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:53.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.455+0000 7f00a643e700 1 --2- 172.21.15.7:0/945502876 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f00880a2c20 0x7f00880a50e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f00a0106c80 tx=0x7f0090009260 comp rx=0 tx=0).stop 2024-06-10T01:12:53.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.455+0000 7f00a643e700 1 -- 172.21.15.7:0/945502876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00a0107400 msgr2=0x7f00a0198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:53.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.455+0000 7f00a643e700 1 --2- 172.21.15.7:0/945502876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00a0107400 0x7f00a0198a10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f009800b0a0 tx=0x7f0098004d00 comp rx=0 tx=0).stop 2024-06-10T01:12:53.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.455+0000 7f00a643e700 1 -- 172.21.15.7:0/945502876 shutdown_connections 2024-06-10T01:12:53.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.455+0000 7f00a643e700 1 --2- 172.21.15.7:0/945502876 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f00880a2c20 0x7f00880a50e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.455+0000 7f00a643e700 1 --2- 172.21.15.7:0/945502876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00a0107400 0x7f00a0198a10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.455+0000 7f00a643e700 1 --2- 172.21.15.7:0/945502876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00a0101390 0x7f00a019e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.455+0000 7f00a643e700 1 --2- 172.21.15.7:0/945502876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00a0101d50 0x7f00a019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.455+0000 7f00a643e700 1 -- 172.21.15.7:0/945502876 >> 172.21.15.7:0/945502876 conn(0x7f00a0078340 msgr2=0x7f00a01005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:53.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.455+0000 7f00a643e700 1 -- 172.21.15.7:0/945502876 shutdown_connections 2024-06-10T01:12:53.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:53.456+0000 7f00a643e700 1 -- 172.21.15.7:0/945502876 wait complete. 2024-06-10T01:12:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.515+0000 7fb6a6a93640 1 -- 172.21.15.7:0/3887324191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6a0102c00 msgr2=0x7fb6a0103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.515+0000 7fb6a6a93640 1 --2- 172.21.15.7:0/3887324191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6a0102c00 0x7fb6a0103080 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fb68c009900 tx=0x7fb68c02a410 comp rx=0 tx=0).stop 2024-06-10T01:12:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.515+0000 7fb6a6a93640 1 -- 172.21.15.7:0/3887324191 shutdown_connections 2024-06-10T01:12:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.515+0000 7fb6a6a93640 1 --2- 172.21.15.7:0/3887324191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6a01035c0 0x7fb6a0109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.515+0000 7fb6a6a93640 1 --2- 172.21.15.7:0/3887324191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6a0102c00 0x7fb6a0103080 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.515+0000 7fb6a6a93640 1 --2- 172.21.15.7:0/3887324191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6a0101a10 0x7fb6a0101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.515+0000 7fb6a6a93640 1 -- 172.21.15.7:0/3887324191 >> 172.21.15.7:0/3887324191 conn(0x7fb6a00fcf80 msgr2=0x7fb6a00ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.515+0000 7fb6a6a93640 1 -- 172.21.15.7:0/3887324191 shutdown_connections 2024-06-10T01:12:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.515+0000 7fb6a6a93640 1 -- 172.21.15.7:0/3887324191 wait complete. 2024-06-10T01:12:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.515+0000 7fb6a6a93640 1 Processor -- start 2024-06-10T01:12:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.516+0000 7fb6a6a93640 1 -- start start 2024-06-10T01:12:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.516+0000 7fb6a6a93640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6a0101a10 0x7fb6a0197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.516+0000 7fb6a6a93640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6a0102c00 0x7fb6a0198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.516+0000 7fb6a6a93640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6a01035c0 0x7fb6a019f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.516+0000 7fb6a6a93640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a010d690 con 0x7fb6a0102c00 2024-06-10T01:12:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.516+0000 7fb6a6a93640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a010d510 con 0x7fb6a01035c0 2024-06-10T01:12:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.516+0000 7fb6a6a93640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a010d810 con 0x7fb6a0101a10 2024-06-10T01:12:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.516+0000 7fb69ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6a0102c00 0x7fb6a0198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.516+0000 7fb6a482f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6a0101a10 0x7fb6a0197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.516+0000 7fb6a5030640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6a01035c0 0x7fb6a019f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.516+0000 7fb69ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6a0102c00 0x7fb6a0198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44968/0 (socket says 172.21.15.7:44968) 2024-06-10T01:12:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.516+0000 7fb69ffff640 1 -- 172.21.15.7:0/1706563598 learned_addr learned my addr 172.21.15.7:0/1706563598 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.516+0000 7fb69ffff640 1 -- 172.21.15.7:0/1706563598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6a0101a10 msgr2=0x7fb6a0197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.516+0000 7fb69ffff640 1 --2- 172.21.15.7:0/1706563598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6a0101a10 0x7fb6a0197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.516+0000 7fb69ffff640 1 -- 172.21.15.7:0/1706563598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6a01035c0 msgr2=0x7fb6a019f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.516+0000 7fb69ffff640 1 --2- 172.21.15.7:0/1706563598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6a01035c0 0x7fb6a019f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.516+0000 7fb69ffff640 1 -- 172.21.15.7:0/1706563598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a019f940 con 0x7fb6a0102c00 2024-06-10T01:12:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.516+0000 7fb6a482f640 1 --2- 172.21.15.7:0/1706563598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6a0101a10 0x7fb6a0197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.516+0000 7fb69ffff640 1 --2- 172.21.15.7:0/1706563598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6a0102c00 0x7fb6a0198270 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fb68c009230 tx=0x7fb68c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.517+0000 7fb69dffb640 1 -- 172.21.15.7:0/1706563598 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c03f030 con 0x7fb6a0102c00 2024-06-10T01:12:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.517+0000 7fb6a6a93640 1 -- 172.21.15.7:0/1706563598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6a019fbd0 con 0x7fb6a0102c00 2024-06-10T01:12:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.517+0000 7fb69dffb640 1 -- 172.21.15.7:0/1706563598 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb68c003ce0 con 0x7fb6a0102c00 2024-06-10T01:12:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.517+0000 7fb69dffb640 1 -- 172.21.15.7:0/1706563598 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c035070 con 0x7fb6a0102c00 2024-06-10T01:12:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.517+0000 7fb6a6a93640 1 -- 172.21.15.7:0/1706563598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6a01a00c0 con 0x7fb6a0102c00 2024-06-10T01:12:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.519+0000 7fb69dffb640 1 -- 172.21.15.7:0/1706563598 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb68c042030 con 0x7fb6a0102c00 2024-06-10T01:12:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.519+0000 7fb6a979e740 1 Processor -- start 2024-06-10T01:12:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.520+0000 7fb6a979e740 1 -- start start 2024-06-10T01:12:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.520+0000 7fb6a979e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b26b0a6010 0x55b26b0a6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.520+0000 7fb6a979e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b26b0a6a00 0x55b26b0ab500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.520+0000 7fb6a979e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b26b0abb70 0x55b26b0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.520+0000 7fb6a979e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b26b0094f0 con 0x55b26b0a6a00 2024-06-10T01:12:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.520+0000 7fb6a979e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b26afa7870 con 0x55b26b0abb70 2024-06-10T01:12:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.520+0000 7fb6a979e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b26b0054b0 con 0x55b26b0a6010 2024-06-10T01:12:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.520+0000 7fb6a482f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b26b0a6a00 0x55b26b0ab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.520+0000 7fb6a482f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b26b0a6a00 0x55b26b0ab500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44970/0 (socket says 172.21.15.7:44970) 2024-06-10T01:12:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.520+0000 7fb69dffb640 1 --2- 172.21.15.7:0/1706563598 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb6900a2b80 0x7fb6900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.520+0000 7fb6a482f640 1 -- 172.21.15.7:0/1087000306 learned_addr learned my addr 172.21.15.7:0/1087000306 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.520+0000 7fb6a5030640 1 --2- 172.21.15.7:0/1087000306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b26b0a6010 0x55b26b0a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.520+0000 7fb69ffff640 1 --2- 172.21.15.7:0/1087000306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b26b0abb70 0x55b26b0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.520+0000 7fb6a482f640 1 -- 172.21.15.7:0/1087000306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b26b0a6010 msgr2=0x55b26b0a6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.520+0000 7fb6a482f640 1 --2- 172.21.15.7:0/1087000306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b26b0a6010 0x55b26b0a6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.520+0000 7fb6a5030640 1 --2- 172.21.15.7:0/1706563598 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb6900a2b80 0x7fb6900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.520+0000 7fb6a482f640 1 -- 172.21.15.7:0/1087000306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b26b0abb70 msgr2=0x55b26b0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.520+0000 7fb6a482f640 1 --2- 172.21.15.7:0/1087000306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b26b0abb70 0x55b26b0ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.520+0000 7fb6a482f640 1 -- 172.21.15.7:0/1087000306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b26afc39b0 con 0x55b26b0a6a00 2024-06-10T01:12:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.520+0000 7fb69ffff640 1 --2- 172.21.15.7:0/1087000306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b26b0abb70 0x55b26b0ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:12:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.520+0000 7fb69dffb640 1 -- 172.21.15.7:0/1706563598 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb68c0ef8c0 con 0x7fb6a0102c00 2024-06-10T01:12:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.520+0000 7fb6a482f640 1 --2- 172.21.15.7:0/1087000306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b26b0a6a00 0x55b26b0ab500 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fb6940114f0 tx=0x7fb6940119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944e18b9d53cb76f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.521+0000 7fb6a5030640 1 --2- 172.21.15.7:0/1706563598 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb6900a2b80 0x7fb6900a5040 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x55b26b0a7260 tx=0x7fb698004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.521+0000 7fb68affd640 1 -- 172.21.15.7:0/1087000306 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb694004310 con 0x55b26b0a6a00 2024-06-10T01:12:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.521+0000 7fb68affd640 1 -- 172.21.15.7:0/1087000306 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6940044b0 con 0x55b26b0a6a00 2024-06-10T01:12:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.521+0000 7fb68affd640 1 -- 172.21.15.7:0/1087000306 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb694003850 con 0x55b26b0a6a00 2024-06-10T01:12:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.521+0000 7fb6a979e740 1 -- 172.21.15.7:0/1087000306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b26b0a6a00 msgr2=0x55b26b0ab500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.521+0000 7fb6a979e740 1 --2- 172.21.15.7:0/1087000306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b26b0a6a00 0x55b26b0ab500 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fb6940114f0 tx=0x7fb6940119b0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.521+0000 7fb6a979e740 1 -- 172.21.15.7:0/1087000306 shutdown_connections 2024-06-10T01:12:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.521+0000 7fb6a979e740 1 --2- 172.21.15.7:0/1087000306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b26b0abb70 0x55b26b0ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.521+0000 7fb6a979e740 1 --2- 172.21.15.7:0/1087000306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b26b0a6a00 0x55b26b0ab500 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.521+0000 7fb6a979e740 1 --2- 172.21.15.7:0/1087000306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b26b0a6010 0x55b26b0a6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.521+0000 7fb6a979e740 1 -- 172.21.15.7:0/1087000306 >> 172.21.15.7:0/1087000306 conn(0x55b26b0a1680 msgr2=0x55b26b0a1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.521+0000 7fb6a979e740 1 -- 172.21.15.7:0/1087000306 shutdown_connections 2024-06-10T01:12:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.521+0000 7fb6a979e740 1 -- 172.21.15.7:0/1087000306 wait complete. 2024-06-10T01:12:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.522+0000 7fb6a979e740 1 -- start start 2024-06-10T01:12:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.522+0000 7fb6a979e740 1 Processor -- start 2024-06-10T01:12:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.522+0000 7fb6a979e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b26b0a6010 0x55b26b0b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.522+0000 7fb6a979e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b26b0a6a00 0x55b26b0b0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.522+0000 7fb6a979e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b26b0abb70 0x55b26b0b7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.522+0000 7fb6a979e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b26b0aeb20 con 0x55b26b0a6a00 2024-06-10T01:12:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.522+0000 7fb69ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b26b0a6a00 0x55b26b0b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.522+0000 7fb6a979e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b26af92ae0 con 0x55b26b0a6010 2024-06-10T01:12:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.522+0000 7fb6a979e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b26b0aeca0 con 0x55b26b0abb70 2024-06-10T01:12:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.522+0000 7fb6a482f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b26b0a6010 0x55b26b0b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.522+0000 7fb69ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b26b0a6a00 0x55b26b0b0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44976/0 (socket says 172.21.15.7:44976) 2024-06-10T01:12:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.522+0000 7fb69ffff640 1 -- 172.21.15.7:0/4000236336 learned_addr learned my addr 172.21.15.7:0/4000236336 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.522+0000 7fb6a979e740 10 client.0 _ll_register_callbacks cb 0x55b26afcb9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:12:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.523+0000 7fb6a5030640 1 --2- 172.21.15.7:0/4000236336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b26b0abb70 0x55b26b0b7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.523+0000 7fb69ffff640 1 -- 172.21.15.7:0/4000236336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b26b0abb70 msgr2=0x55b26b0b7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.523+0000 7fb69ffff640 1 --2- 172.21.15.7:0/4000236336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b26b0abb70 0x55b26b0b7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.523+0000 7fb69ffff640 1 -- 172.21.15.7:0/4000236336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b26b0a6010 msgr2=0x55b26b0b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.523+0000 7fb69ffff640 1 --2- 172.21.15.7:0/4000236336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b26b0a6010 0x55b26b0b02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.523+0000 7fb69ffff640 1 -- 172.21.15.7:0/4000236336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b26ad1d210 con 0x55b26b0a6a00 2024-06-10T01:12:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.523+0000 7fb6a5030640 1 --2- 172.21.15.7:0/4000236336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b26b0abb70 0x55b26b0b7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:12:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.523+0000 7fb6a482f640 1 --2- 172.21.15.7:0/4000236336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b26b0a6010 0x55b26b0b02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.523+0000 7fb69ffff640 1 --2- 172.21.15.7:0/4000236336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b26b0a6a00 0x55b26b0b0800 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0bbb50 tx=0x7fb68c08ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d781669a02ebe40f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.523+0000 7fb67ffff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:12:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.523+0000 7fb67ffff640 1 -- 172.21.15.7:0/4000236336 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c049040 con 0x55b26b0a6a00 2024-06-10T01:12:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.523+0000 7fb67ffff640 1 -- 172.21.15.7:0/4000236336 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb68c0b8430 con 0x55b26b0a6a00 2024-06-10T01:12:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.523+0000 7fb6a979e740 1 -- 172.21.15.7:0/4000236336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6940044b0 con 0x55b26b0a6a00 2024-06-10T01:12:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.523+0000 7fb67ffff640 1 -- 172.21.15.7:0/4000236336 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb68c0b9f90 con 0x55b26b0a6a00 2024-06-10T01:12:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.523+0000 7fb6a979e740 10 client.17205 fetch_fsmap learned FSMap version 32 2024-06-10T01:12:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.523+0000 7fb6a979e740 1 -- 172.21.15.7:0/4000236336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b26ade9620 con 0x55b26b0a6a00 2024-06-10T01:12:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.524+0000 7fb67ffff640 1 -- 172.21.15.7:0/4000236336 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6940044b0 con 0x55b26b0a6a00 2024-06-10T01:12:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.524+0000 7fb67ffff640 1 -- 172.21.15.7:0/4000236336 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fb68c08bce0 con 0x55b26b0a6a00 2024-06-10T01:12:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.524+0000 7fb6a979e740 10 client.17205 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:12:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.524+0000 7fb6a979e740 10 client.17205 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:12:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.524+0000 7fb6a979e740 20 client.17205 populate_metadata read hostname 'smithi007' 2024-06-10T01:12:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.524+0000 7fb6a979e740 1 --2- 172.21.15.7:0/4000236336 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b26b0bc010 0x55b26b0be480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.524+0000 7fb6a979e740 4 client.17205 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:12:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.524+0000 7fb6a979e740 1 -- 172.21.15.7:0/4000236336 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b26b01cf10 con 0x55b26b0bc010 2024-06-10T01:12:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.524+0000 7fb6a482f640 1 --2- 172.21.15.7:0/4000236336 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b26b0bc010 0x55b26b0be480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.524+0000 7fb6a482f640 1 --2- 172.21.15.7:0/4000236336 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b26b0bc010 0x55b26b0be480 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfeb99b1a9c46c9 server_cookie=4a4d104ae99e697e in_seq=0 out_seq=0 2024-06-10T01:12:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.524+0000 7fb67ffff640 10 client.17205 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.525+0000 7fb67ffff640 1 -- 172.21.15.7:0/4000236336 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b26b01cf10 con 0x55b26b0bc010 2024-06-10T01:12:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.525+0000 7fb67ffff640 10 client.17205 handle_command_reply: tid=0 2024-06-10T01:12:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.525+0000 7fb6a979e740 1 client.17205 shutdown 2024-06-10T01:12:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.525+0000 7fb6a979e740 1 -- 172.21.15.7:0/4000236336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b26b0a6a00 msgr2=0x55b26b0b0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.525+0000 7fb6a979e740 1 --2- 172.21.15.7:0/4000236336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b26b0a6a00 0x55b26b0b0800 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0bbb50 tx=0x7fb68c08ad40 comp rx=0 tx=0).stop 2024-06-10T01:12:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.525+0000 7fb6a979e740 1 -- 172.21.15.7:0/4000236336 shutdown_connections 2024-06-10T01:12:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.525+0000 7fb6a979e740 1 --2- 172.21.15.7:0/4000236336 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b26b0bc010 0x55b26b0be480 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.525+0000 7fb6a979e740 1 --2- 172.21.15.7:0/4000236336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b26b0abb70 0x55b26b0b7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.525+0000 7fb67ffff640 0 client.17205 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.525+0000 7fb6a979e740 1 --2- 172.21.15.7:0/4000236336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b26b0a6a00 0x55b26b0b0800 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.525+0000 7fb6a979e740 1 --2- 172.21.15.7:0/4000236336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b26b0a6010 0x55b26b0b02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.525+0000 7fb6a979e740 1 -- 172.21.15.7:0/4000236336 >> 172.21.15.7:0/4000236336 conn(0x55b26b0a1680 msgr2=0x55b26b0a07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.525+0000 7fb6a979e740 1 -- 172.21.15.7:0/4000236336 shutdown_connections 2024-06-10T01:12:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.525+0000 7fb6a979e740 1 -- 172.21.15.7:0/4000236336 wait complete. 2024-06-10T01:12:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.525+0000 7fb6a979e740 20 client.17205 trim_cache size 0 max 16384 2024-06-10T01:12:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.541+0000 7fb6a979e740 1 Processor -- start 2024-06-10T01:12:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.541+0000 7fb6a979e740 1 -- start start 2024-06-10T01:12:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.541+0000 7fb6a979e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b26b0a6010 0x55b26b0af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.541+0000 7fb6a979e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b26b0a6a00 0x55b26b0b7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.541+0000 7fb6a979e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b26b0abb70 0x55b26b0a7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.541+0000 7fb6a979e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a01acda0 con 0x55b26b0a6010 2024-06-10T01:12:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.541+0000 7fb6a979e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a01aa670 con 0x55b26b0abb70 2024-06-10T01:12:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.541+0000 7fb6a979e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a01a84c0 con 0x55b26b0a6a00 2024-06-10T01:12:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.541+0000 7fb6a482f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b26b0a6010 0x55b26b0af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.541+0000 7fb6a482f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b26b0a6010 0x55b26b0af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44992/0 (socket says 172.21.15.7:44992) 2024-06-10T01:12:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.541+0000 7fb6a482f640 1 -- 172.21.15.7:0/1019446173 learned_addr learned my addr 172.21.15.7:0/1019446173 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.541+0000 7fb69ffff640 1 --2- 172.21.15.7:0/1019446173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b26b0a6a00 0x55b26b0b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.541+0000 7fb6a5030640 1 --2- 172.21.15.7:0/1019446173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b26b0abb70 0x55b26b0a7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.541+0000 7fb6a482f640 1 -- 172.21.15.7:0/1019446173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b26b0a6a00 msgr2=0x55b26b0b7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.541+0000 7fb6a482f640 1 --2- 172.21.15.7:0/1019446173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b26b0a6a00 0x55b26b0b7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.541+0000 7fb6a482f640 1 -- 172.21.15.7:0/1019446173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b26b0abb70 msgr2=0x55b26b0a7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.541+0000 7fb6a482f640 1 --2- 172.21.15.7:0/1019446173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b26b0abb70 0x55b26b0a7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.541+0000 7fb6a482f640 1 -- 172.21.15.7:0/1019446173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b26b0a7290 con 0x55b26b0a6010 2024-06-10T01:12:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.541+0000 7fb6a5030640 1 --2- 172.21.15.7:0/1019446173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b26b0abb70 0x55b26b0a7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:12:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.542+0000 7fb6a482f640 1 --2- 172.21.15.7:0/1019446173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b26b0a6010 0x55b26b0af8e0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fb6940049a0 tx=0x7fb6940096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8cd73b50da5dc6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.542+0000 7fb68a7fc640 1 -- 172.21.15.7:0/1019446173 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb694003870 con 0x55b26b0a6010 2024-06-10T01:12:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.542+0000 7fb68a7fc640 1 -- 172.21.15.7:0/1019446173 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb694033070 con 0x55b26b0a6010 2024-06-10T01:12:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.542+0000 7fb68a7fc640 1 -- 172.21.15.7:0/1019446173 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69400fce0 con 0x55b26b0a6010 2024-06-10T01:12:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.542+0000 7fb6a979e740 1 -- 172.21.15.7:0/1019446173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b26b0a6010 msgr2=0x55b26b0af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.543+0000 7fb6a979e740 1 --2- 172.21.15.7:0/1019446173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b26b0a6010 0x55b26b0af8e0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fb6940049a0 tx=0x7fb6940096c0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.543+0000 7fb6a979e740 1 -- 172.21.15.7:0/1019446173 shutdown_connections 2024-06-10T01:12:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.543+0000 7fb6a979e740 1 --2- 172.21.15.7:0/1019446173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b26b0abb70 0x55b26b0a7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.543+0000 7fb6a979e740 1 --2- 172.21.15.7:0/1019446173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b26b0a6a00 0x55b26b0b7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.543+0000 7fb6a979e740 1 --2- 172.21.15.7:0/1019446173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b26b0a6010 0x55b26b0af8e0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.543+0000 7fb6a979e740 1 -- 172.21.15.7:0/1019446173 >> 172.21.15.7:0/1019446173 conn(0x55b26b0a1680 msgr2=0x55b26b0ab290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.543+0000 7fb6a979e740 1 -- 172.21.15.7:0/1019446173 shutdown_connections 2024-06-10T01:12:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.543+0000 7fb6a979e740 1 -- 172.21.15.7:0/1019446173 wait complete. 2024-06-10T01:12:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.543+0000 7fb6a979e740 1 -- start start 2024-06-10T01:12:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.543+0000 7fb6a979e740 1 Processor -- start 2024-06-10T01:12:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.543+0000 7fb6a979e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b26b0a6010 0x55b26b0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.543+0000 7fb6a979e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b26b0a6a00 0x55b26b0b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.543+0000 7fb6a979e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b26b0abb70 0x55b26b124d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.543+0000 7fb6a979e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a01a8b30 con 0x55b26b0a6010 2024-06-10T01:12:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.543+0000 7fb6a979e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a01a9fa0 con 0x55b26b0a6a00 2024-06-10T01:12:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.543+0000 7fb6a979e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a01a9200 con 0x55b26b0abb70 2024-06-10T01:12:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.544+0000 7fb6a979e740 10 client.0 _ll_register_callbacks cb 0x7fb68c0b8060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:12:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.544+0000 7fb6a482f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b26b0a6010 0x55b26b0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.544+0000 7fb6a482f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b26b0a6010 0x55b26b0b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44998/0 (socket says 172.21.15.7:44998) 2024-06-10T01:12:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.544+0000 7fb6a482f640 1 -- 172.21.15.7:0/3218966189 learned_addr learned my addr 172.21.15.7:0/3218966189 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.544+0000 7fb69ffff640 1 --2- 172.21.15.7:0/3218966189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b26b0a6a00 0x55b26b0b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.544+0000 7fb6a5030640 1 --2- 172.21.15.7:0/3218966189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b26b0abb70 0x55b26b124d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.544+0000 7fb6a482f640 1 -- 172.21.15.7:0/3218966189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b26b0abb70 msgr2=0x55b26b124d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.544+0000 7fb6a482f640 1 --2- 172.21.15.7:0/3218966189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b26b0abb70 0x55b26b124d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.544+0000 7fb6a482f640 1 -- 172.21.15.7:0/3218966189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b26b0a6a00 msgr2=0x55b26b0b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.544+0000 7fb6a482f640 1 --2- 172.21.15.7:0/3218966189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b26b0a6a00 0x55b26b0b4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.544+0000 7fb6a482f640 1 -- 172.21.15.7:0/3218966189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b26b125290 con 0x55b26b0a6010 2024-06-10T01:12:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.544+0000 7fb6a482f640 1 --2- 172.21.15.7:0/3218966189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b26b0a6010 0x55b26b0b4650 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fb69401b040 tx=0x7fb694004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92607331ab08c8f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.544+0000 7fb67ffff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:12:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.544+0000 7fb67ffff640 1 -- 172.21.15.7:0/3218966189 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69403d020 con 0x55b26b0a6010 2024-06-10T01:12:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.544+0000 7fb67ffff640 1 -- 172.21.15.7:0/3218966189 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb69400fc20 con 0x55b26b0a6010 2024-06-10T01:12:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.544+0000 7fb6a979e740 1 -- 172.21.15.7:0/3218966189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb694033070 con 0x55b26b0a6010 2024-06-10T01:12:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.544+0000 7fb67ffff640 1 -- 172.21.15.7:0/3218966189 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb694048050 con 0x55b26b0a6010 2024-06-10T01:12:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.544+0000 7fb6a979e740 10 client.17211 fetch_fsmap learned FSMap version 32 2024-06-10T01:12:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.544+0000 7fb6a979e740 1 -- 172.21.15.7:0/3218966189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b26b126410 con 0x55b26b0a6010 2024-06-10T01:12:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.544+0000 7fb67ffff640 1 -- 172.21.15.7:0/3218966189 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb694033070 con 0x55b26b0a6010 2024-06-10T01:12:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.545+0000 7fb67ffff640 1 -- 172.21.15.7:0/3218966189 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fb69402e3f0 con 0x55b26b0a6010 2024-06-10T01:12:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.545+0000 7fb6a979e740 10 client.17211 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:12:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.545+0000 7fb6a979e740 10 client.17211 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:12:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.545+0000 7fb6a979e740 20 client.17211 populate_metadata read hostname 'smithi007' 2024-06-10T01:12:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.545+0000 7fb6a979e740 1 --2- 172.21.15.7:0/3218966189 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b26b126c40 0x55b26b129060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.545+0000 7fb6a979e740 4 client.17211 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:12:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.545+0000 7fb6a979e740 1 -- 172.21.15.7:0/3218966189 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b26b016c20 con 0x55b26b126c40 2024-06-10T01:12:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.545+0000 7fb69ffff640 1 --2- 172.21.15.7:0/3218966189 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b26b126c40 0x55b26b129060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.545+0000 7fb67ffff640 10 client.17211 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.545+0000 7fb69ffff640 1 --2- 172.21.15.7:0/3218966189 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b26b126c40 0x55b26b129060 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce066cc55787e569 server_cookie=8175f7fdca02b81d in_seq=0 out_seq=0 2024-06-10T01:12:53.579 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:53 smithi078 ceph-mon[36683]: pgmap v638: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 325 op/s 2024-06-10T01:12:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.655+0000 7fb67ffff640 1 -- 172.21.15.7:0/3218966189 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b26b016c20 con 0x55b26b126c40 2024-06-10T01:12:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.655+0000 7fb67ffff640 10 client.17211 handle_command_reply: tid=0 2024-06-10T01:12:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.655+0000 7fb6a979e740 1 client.17211 shutdown 2024-06-10T01:12:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.655+0000 7fb6a979e740 1 -- 172.21.15.7:0/3218966189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b26b0a6010 msgr2=0x55b26b0b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.655+0000 7fb6a979e740 1 --2- 172.21.15.7:0/3218966189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b26b0a6010 0x55b26b0b4650 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fb69401b040 tx=0x7fb694004500 comp rx=0 tx=0).stop 2024-06-10T01:12:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.655+0000 7fb6a979e740 1 -- 172.21.15.7:0/3218966189 shutdown_connections 2024-06-10T01:12:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.655+0000 7fb6a979e740 1 --2- 172.21.15.7:0/3218966189 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b26b126c40 0x55b26b129060 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.655+0000 7fb6a979e740 1 --2- 172.21.15.7:0/3218966189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b26b0abb70 0x55b26b124d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.655+0000 7fb6a979e740 1 --2- 172.21.15.7:0/3218966189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b26b0a6a00 0x55b26b0b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.655+0000 7fb67ffff640 0 client.17211 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.655+0000 7fb6a979e740 1 --2- 172.21.15.7:0/3218966189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b26b0a6010 0x55b26b0b4650 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.655+0000 7fb6a979e740 1 -- 172.21.15.7:0/3218966189 >> 172.21.15.7:0/3218966189 conn(0x55b26b0a1680 msgr2=0x55b26b0b02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.656+0000 7fb6a979e740 1 -- 172.21.15.7:0/3218966189 shutdown_connections 2024-06-10T01:12:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.656+0000 7fb6a979e740 1 -- 172.21.15.7:0/3218966189 wait complete. 2024-06-10T01:12:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.656+0000 7fb6a979e740 20 client.17211 trim_cache size 0 max 16384 2024-06-10T01:12:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:12:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:12:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:12:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.656+0000 7fb6a6a93640 1 -- 172.21.15.7:0/1706563598 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb6900a2b80 msgr2=0x7fb6900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.656+0000 7fb6a6a93640 1 --2- 172.21.15.7:0/1706563598 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb6900a2b80 0x7fb6900a5040 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x55b26b0a7260 tx=0x7fb698004ab0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.656+0000 7fb6a6a93640 1 -- 172.21.15.7:0/1706563598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6a0102c00 msgr2=0x7fb6a0198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.656+0000 7fb6a6a93640 1 --2- 172.21.15.7:0/1706563598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6a0102c00 0x7fb6a0198270 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fb68c009230 tx=0x7fb68c0042f0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.656+0000 7fb6a6a93640 1 -- 172.21.15.7:0/1706563598 shutdown_connections 2024-06-10T01:12:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.656+0000 7fb6a6a93640 1 --2- 172.21.15.7:0/1706563598 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb6900a2b80 0x7fb6900a5040 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.656+0000 7fb6a6a93640 1 --2- 172.21.15.7:0/1706563598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6a01035c0 0x7fb6a019f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.656+0000 7fb6a6a93640 1 --2- 172.21.15.7:0/1706563598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6a0102c00 0x7fb6a0198270 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.656+0000 7fb6a6a93640 1 --2- 172.21.15.7:0/1706563598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6a0101a10 0x7fb6a0197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.656+0000 7fb6a6a93640 1 -- 172.21.15.7:0/1706563598 >> 172.21.15.7:0/1706563598 conn(0x7fb6a00fcf80 msgr2=0x7fb6a0105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.656+0000 7fb6a6a93640 1 -- 172.21.15.7:0/1706563598 shutdown_connections 2024-06-10T01:12:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:12:53.657+0000 7fb6a6a93640 1 -- 172.21.15.7:0/1706563598 wait complete. 2024-06-10T01:12:53.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:53 smithi175 ceph-mon[36636]: pgmap v638: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 325 op/s 2024-06-10T01:12:53.736 INFO:tasks.workunit.client.0.smithi007.stdout: 1 450716 6.97 MB/sec execute 214 sec latency 69.540 ms 2024-06-10T01:12:53.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:53 smithi007 ceph-mon[32398]: pgmap v638: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 325 op/s 2024-06-10T01:12:54.300 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:12:54.558 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:12:54.736 INFO:tasks.workunit.client.0.smithi007.stdout: 1 451988 6.96 MB/sec execute 215 sec latency 45.943 ms 2024-06-10T01:12:55.579 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:55 smithi078 ceph-mon[36683]: pgmap v639: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 304 op/s 2024-06-10T01:12:55.690 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:55 smithi007 ceph-mon[32398]: pgmap v639: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 304 op/s 2024-06-10T01:12:55.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:55 smithi175 ceph-mon[36636]: pgmap v639: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 304 op/s 2024-06-10T01:12:55.736 INFO:tasks.workunit.client.0.smithi007.stdout: 1 454408 6.99 MB/sec execute 216 sec latency 27.311 ms 2024-06-10T01:12:56.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.214+0000 7f077f3ae700 1 -- 172.21.15.7:0/1700807228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f077811bc70 msgr2=0x7f077811e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.214+0000 7f077f3ae700 1 --2- 172.21.15.7:0/1700807228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f077811bc70 0x7f077811e060 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0770009f90 tx=0x7f077000b330 comp rx=0 tx=0).stop 2024-06-10T01:12:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.215+0000 7f077f3ae700 1 -- 172.21.15.7:0/1700807228 shutdown_connections 2024-06-10T01:12:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.215+0000 7f077f3ae700 1 --2- 172.21.15.7:0/1700807228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f077811bc70 0x7f077811e060 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.215+0000 7f077f3ae700 1 --2- 172.21.15.7:0/1700807228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0778110c70 0x7f0778109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.215+0000 7f077f3ae700 1 --2- 172.21.15.7:0/1700807228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07781099d0 0x7f0778109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.215+0000 7f077f3ae700 1 -- 172.21.15.7:0/1700807228 >> 172.21.15.7:0/1700807228 conn(0x7f077806d600 msgr2=0x7f077806da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.218+0000 7f077f3ae700 1 -- 172.21.15.7:0/1700807228 shutdown_connections 2024-06-10T01:12:56.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.218+0000 7f077f3ae700 1 -- 172.21.15.7:0/1700807228 wait complete. 2024-06-10T01:12:56.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.219+0000 7f077f3ae700 1 Processor -- start 2024-06-10T01:12:56.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.219+0000 7f077f3ae700 1 -- start start 2024-06-10T01:12:56.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.220+0000 7f077f3ae700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07781099d0 0x7f077811b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:56.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.220+0000 7f077f3ae700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0778110c70 0x7f07781154b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:56.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.220+0000 7f077f3ae700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f077811bc70 0x7f07781159f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:56.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.220+0000 7f077f3ae700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0778111e80 con 0x7f077811bc70 2024-06-10T01:12:56.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.220+0000 7f077f3ae700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0778111d00 con 0x7f0778110c70 2024-06-10T01:12:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.220+0000 7f077f3ae700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0778112000 con 0x7f07781099d0 2024-06-10T01:12:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.220+0000 7f077d14a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07781099d0 0x7f077811b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.221+0000 7f077d14a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07781099d0 0x7f077811b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:42116/0 (socket says 172.21.15.7:42116) 2024-06-10T01:12:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.221+0000 7f077d14a700 1 -- 172.21.15.7:0/1220572242 learned_addr learned my addr 172.21.15.7:0/1220572242 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:56.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.221+0000 7f077d14a700 1 -- 172.21.15.7:0/1220572242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0778110c70 msgr2=0x7f07781154b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-10T01:12:56.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.221+0000 7f077d14a700 1 --2- 172.21.15.7:0/1220572242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0778110c70 0x7f07781154b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.221+0000 7f077d14a700 1 -- 172.21.15.7:0/1220572242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f077811bc70 msgr2=0x7f07781159f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:56.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.221+0000 7f077d14a700 1 --2- 172.21.15.7:0/1220572242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f077811bc70 0x7f07781159f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.221+0000 7f077d14a700 1 -- 172.21.15.7:0/1220572242 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0778116180 con 0x7f07781099d0 2024-06-10T01:12:56.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.222+0000 7f077d14a700 1 --2- 172.21.15.7:0/1220572242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07781099d0 0x7f077811b3e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f076c00e9c0 tx=0x7f076c00ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:56.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.223+0000 7f076a7fc700 1 -- 172.21.15.7:0/1220572242 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c00cbd0 con 0x7f07781099d0 2024-06-10T01:12:56.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.223+0000 7f077f3ae700 1 -- 172.21.15.7:0/1220572242 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07781bca10 con 0x7f07781099d0 2024-06-10T01:12:56.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.223+0000 7f077f3ae700 1 -- 172.21.15.7:0/1220572242 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07781bcf20 con 0x7f07781099d0 2024-06-10T01:12:56.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.225+0000 7f076a7fc700 1 -- 172.21.15.7:0/1220572242 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f076c00cd70 con 0x7f07781099d0 2024-06-10T01:12:56.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.225+0000 7f076a7fc700 1 -- 172.21.15.7:0/1220572242 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c010430 con 0x7f07781099d0 2024-06-10T01:12:56.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.228+0000 7f076a7fc700 1 -- 172.21.15.7:0/1220572242 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f076c010690 con 0x7f07781099d0 2024-06-10T01:12:56.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.232+0000 7f0789c31b80 1 Processor -- start 2024-06-10T01:12:56.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.232+0000 7f076a7fc700 1 --2- 172.21.15.7:0/1220572242 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f07640a2df0 0x7f07640a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:56.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.232+0000 7f0789c31b80 1 -- start start 2024-06-10T01:12:56.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.232+0000 7f0789c31b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e7bec41640 0x55e7bec41a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:56.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.232+0000 7f0789c31b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7bec41f60 0x55e7bec3b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:56.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.232+0000 7f0789c31b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e7bec3bf20 0x55e7bec3c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:56.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.232+0000 7f0789c31b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e7beb8d810 con 0x55e7bec3bf20 2024-06-10T01:12:56.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.232+0000 7f0789c31b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e7beb35720 con 0x55e7bec41f60 2024-06-10T01:12:56.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.232+0000 7f0789c31b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e7beb897d0 con 0x55e7bec41640 2024-06-10T01:12:56.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.232+0000 7f077d94b700 1 --2- 172.21.15.7:0/1220572242 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f07640a2df0 0x7f07640a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:56.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.232+0000 7f077d94b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e7bec3bf20 0x55e7bec3c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:56.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.232+0000 7f077c949700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7bec41f60 0x55e7bec3b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:56.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.232+0000 7f077d94b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e7bec3bf20 0x55e7bec3c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45046/0 (socket says 172.21.15.7:45046) 2024-06-10T01:12:56.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.232+0000 7f077d94b700 1 -- 172.21.15.7:0/12212995 learned_addr learned my addr 172.21.15.7:0/12212995 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:56.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.233+0000 7f077d14a700 1 --2- 172.21.15.7:0/12212995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e7bec41640 0x55e7bec41a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:56.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.233+0000 7f076a7fc700 1 -- 172.21.15.7:0/1220572242 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f076c0d05d0 con 0x7f07781099d0 2024-06-10T01:12:56.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.233+0000 7f077c949700 1 -- 172.21.15.7:0/12212995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e7bec41640 msgr2=0x55e7bec41a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:56.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.233+0000 7f077c949700 1 --2- 172.21.15.7:0/12212995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e7bec41640 0x55e7bec41a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.233+0000 7f077c949700 1 -- 172.21.15.7:0/12212995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e7bec3bf20 msgr2=0x55e7bec3c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:56.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.233+0000 7f077c949700 1 --2- 172.21.15.7:0/12212995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e7bec3bf20 0x55e7bec3c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.233+0000 7f077c949700 1 -- 172.21.15.7:0/12212995 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7beb2bf50 con 0x55e7bec41f60 2024-06-10T01:12:56.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.233+0000 7f077d94b700 1 --2- 172.21.15.7:0/12212995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e7bec3bf20 0x55e7bec3c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:56.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.233+0000 7f077c949700 1 --2- 172.21.15.7:0/12212995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7bec41f60 0x55e7bec3b9e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0778116cb0 tx=0x7f07740099f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=325e7e9bea1b240f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:56.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.233+0000 7f07637fe700 1 -- 172.21.15.7:0/12212995 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07740107b0 con 0x55e7bec41f60 2024-06-10T01:12:56.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.234+0000 7f07637fe700 1 -- 172.21.15.7:0/12212995 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0774016070 con 0x55e7bec41f60 2024-06-10T01:12:56.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.234+0000 7f07637fe700 1 -- 172.21.15.7:0/12212995 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0774011da0 con 0x55e7bec41f60 2024-06-10T01:12:56.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.235+0000 7f0789c31b80 1 -- 172.21.15.7:0/12212995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7bec41f60 msgr2=0x55e7bec3b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:56.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.235+0000 7f077d94b700 1 --2- 172.21.15.7:0/1220572242 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f07640a2df0 0x7f07640a52b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f077000bd30 tx=0x7f0770009af0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:56.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.235+0000 7f0789c31b80 1 --2- 172.21.15.7:0/12212995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7bec41f60 0x55e7bec3b9e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0778116cb0 tx=0x7f07740099f0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.235+0000 7f0789c31b80 1 -- 172.21.15.7:0/12212995 shutdown_connections 2024-06-10T01:12:56.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.235+0000 7f0789c31b80 1 --2- 172.21.15.7:0/12212995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e7bec3bf20 0x55e7bec3c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.235+0000 7f0789c31b80 1 --2- 172.21.15.7:0/12212995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e7bec41640 0x55e7bec41a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.235+0000 7f0789c31b80 1 --2- 172.21.15.7:0/12212995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7bec41f60 0x55e7bec3b9e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.235+0000 7f0789c31b80 1 -- 172.21.15.7:0/12212995 >> 172.21.15.7:0/12212995 conn(0x55e7beb66cc0 msgr2=0x55e7bec36850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:56.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.235+0000 7f0789c31b80 1 -- 172.21.15.7:0/12212995 shutdown_connections 2024-06-10T01:12:56.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.235+0000 7f0789c31b80 1 -- 172.21.15.7:0/12212995 wait complete. 2024-06-10T01:12:56.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.235+0000 7f0789c31b80 1 -- start start 2024-06-10T01:12:56.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.235+0000 7f0789c31b80 1 Processor -- start 2024-06-10T01:12:56.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.236+0000 7f0789c31b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e7bec3bf20 0x55e7bec47c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:56.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.236+0000 7f0789c31b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e7bec41640 0x55e7bec44fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:56.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.236+0000 7f0789c31b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7bec41f60 0x55e7bec45500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:56.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.236+0000 7f077d14a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e7bec3bf20 0x55e7bec47c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:56.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.236+0000 7f0789c31b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e7bec489d0 con 0x55e7bec3bf20 2024-06-10T01:12:56.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.236+0000 7f077d14a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e7bec3bf20 0x55e7bec47c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45060/0 (socket says 172.21.15.7:45060) 2024-06-10T01:12:56.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.236+0000 7f077d14a700 1 -- 172.21.15.7:0/1000487099 learned_addr learned my addr 172.21.15.7:0/1000487099 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:56.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.236+0000 7f0789c31b80 1 -- 172.21.15.7:0/1000487099 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e7beb2dbf0 con 0x55e7bec41f60 2024-06-10T01:12:56.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.236+0000 7f0789c31b80 1 -- 172.21.15.7:0/1000487099 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e7bec48b50 con 0x55e7bec41640 2024-06-10T01:12:56.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.236+0000 7f077c949700 1 --2- 172.21.15.7:0/1000487099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e7bec41640 0x55e7bec44fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:56.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.236+0000 7f077d94b700 1 --2- 172.21.15.7:0/1000487099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7bec41f60 0x55e7bec45500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:56.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.236+0000 7f0789c31b80 10 client.0 _ll_register_callbacks cb 0x55e7beb27920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:12:56.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.236+0000 7f077d14a700 1 -- 172.21.15.7:0/1000487099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e7bec41640 msgr2=0x55e7bec44fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:56.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.236+0000 7f077d14a700 1 --2- 172.21.15.7:0/1000487099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e7bec41640 0x55e7bec44fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.236+0000 7f077d14a700 1 -- 172.21.15.7:0/1000487099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7bec41f60 msgr2=0x55e7bec45500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:56.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.236+0000 7f077d14a700 1 --2- 172.21.15.7:0/1000487099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7bec41f60 0x55e7bec45500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.236+0000 7f077d14a700 1 -- 172.21.15.7:0/1000487099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7beadc8d0 con 0x55e7bec3bf20 2024-06-10T01:12:56.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.236+0000 7f077d14a700 1 --2- 172.21.15.7:0/1000487099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e7bec3bf20 0x55e7bec47c70 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f076c005370 tx=0x7f076c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f31977aa4fe658 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:56.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.236+0000 7f0760ff9700 10 client.17220 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:12:56.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.237+0000 7f0789c31b80 1 -- 172.21.15.7:0/1000487099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0774016070 con 0x55e7bec3bf20 2024-06-10T01:12:56.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.237+0000 7f0760ff9700 1 -- 172.21.15.7:0/1000487099 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c0af770 con 0x55e7bec3bf20 2024-06-10T01:12:56.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.237+0000 7f0760ff9700 1 -- 172.21.15.7:0/1000487099 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f076c016df0 con 0x55e7bec3bf20 2024-06-10T01:12:56.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.237+0000 7f0760ff9700 1 -- 172.21.15.7:0/1000487099 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f076c0af590 con 0x55e7bec3bf20 2024-06-10T01:12:56.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.237+0000 7f0789c31b80 10 client.17220 fetch_fsmap learned FSMap version 32 2024-06-10T01:12:56.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.237+0000 7f0789c31b80 1 -- 172.21.15.7:0/1000487099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55e7bec531e0 con 0x55e7bec3bf20 2024-06-10T01:12:56.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.237+0000 7f0760ff9700 1 -- 172.21.15.7:0/1000487099 <== mon.0 v2:172.21.15.7:3300/0 4 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f076c0bb2a0 con 0x55e7bec3bf20 2024-06-10T01:12:56.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.237+0000 7f0789c31b80 10 client.17220 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:12:56.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.237+0000 7f0789c31b80 10 client.17220 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:12:56.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.237+0000 7f0789c31b80 20 client.17220 populate_metadata read hostname 'smithi007' 2024-06-10T01:12:56.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.237+0000 7f0789c31b80 1 --2- 172.21.15.7:0/1000487099 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55e7bec55710 0x55e7bec57b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:56.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.238+0000 7f0789c31b80 4 client.17220 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:12:56.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.238+0000 7f0789c31b80 1 -- 172.21.15.7:0/1000487099 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7beb9ad10 con 0x55e7bec55710 2024-06-10T01:12:56.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.238+0000 7f077c949700 1 --2- 172.21.15.7:0/1000487099 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55e7bec55710 0x55e7bec57b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:56.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.238+0000 7f077c949700 1 --2- 172.21.15.7:0/1000487099 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55e7bec55710 0x55e7bec57b60 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c059fad9a563252a server_cookie=5b00a6582350c307 in_seq=0 out_seq=0 2024-06-10T01:12:56.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.238+0000 7f0760ff9700 10 client.17220 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:12:56.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.239+0000 7f0760ff9700 1 -- 172.21.15.7:0/1000487099 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e7beb9ad10 con 0x55e7bec55710 2024-06-10T01:12:56.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.239+0000 7f0760ff9700 10 client.17220 handle_command_reply: tid=0 2024-06-10T01:12:56.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.239+0000 7f0789c31b80 1 client.17220 shutdown 2024-06-10T01:12:56.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.239+0000 7f0789c31b80 1 -- 172.21.15.7:0/1000487099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e7bec3bf20 msgr2=0x55e7bec47c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:56.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.239+0000 7f0789c31b80 1 --2- 172.21.15.7:0/1000487099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e7bec3bf20 0x55e7bec47c70 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f076c005370 tx=0x7f076c01d000 comp rx=0 tx=0).stop 2024-06-10T01:12:56.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.239+0000 7f0789c31b80 1 -- 172.21.15.7:0/1000487099 shutdown_connections 2024-06-10T01:12:56.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.239+0000 7f0789c31b80 1 --2- 172.21.15.7:0/1000487099 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55e7bec55710 0x55e7bec57b60 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.239+0000 7f0789c31b80 1 --2- 172.21.15.7:0/1000487099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7bec41f60 0x55e7bec45500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.239+0000 7f0789c31b80 1 --2- 172.21.15.7:0/1000487099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e7bec3bf20 0x55e7bec47c70 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.239+0000 7f0789c31b80 1 --2- 172.21.15.7:0/1000487099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e7bec41640 0x55e7bec44fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.239+0000 7f0789c31b80 1 -- 172.21.15.7:0/1000487099 >> 172.21.15.7:0/1000487099 conn(0x55e7beb66cc0 msgr2=0x55e7bec3a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:56.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.239+0000 7f0760ff9700 0 client.17220 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:12:56.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.240+0000 7f0789c31b80 1 -- 172.21.15.7:0/1000487099 shutdown_connections 2024-06-10T01:12:56.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.240+0000 7f0789c31b80 1 -- 172.21.15.7:0/1000487099 wait complete. 2024-06-10T01:12:56.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.240+0000 7f0789c31b80 20 client.17220 trim_cache size 0 max 16384 2024-06-10T01:12:56.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.256+0000 7f0789c31b80 1 Processor -- start 2024-06-10T01:12:56.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.256+0000 7f0789c31b80 1 -- start start 2024-06-10T01:12:56.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.257+0000 7f0789c31b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e7bec3bf20 0x55e7bec3b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:56.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.257+0000 7f0789c31b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7bec41640 0x55e7bec3f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:56.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.257+0000 7f0789c31b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e7bec41f60 0x55e7bec2fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:56.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.257+0000 7f0789c31b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f075c011300 con 0x55e7bec41f60 2024-06-10T01:12:56.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.257+0000 7f0789c31b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f075c00eb90 con 0x55e7bec41640 2024-06-10T01:12:56.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.257+0000 7f0789c31b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f075c00c9e0 con 0x55e7bec3bf20 2024-06-10T01:12:56.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.257+0000 7f077d94b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e7bec41f60 0x55e7bec2fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:56.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.257+0000 7f077d14a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e7bec3bf20 0x55e7bec3b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:56.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.257+0000 7f077d14a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e7bec3bf20 0x55e7bec3b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:42136/0 (socket says 172.21.15.7:42136) 2024-06-10T01:12:56.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.257+0000 7f077d94b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e7bec41f60 0x55e7bec2fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45064/0 (socket says 172.21.15.7:45064) 2024-06-10T01:12:56.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.257+0000 7f077d14a700 1 -- 172.21.15.7:0/3444868779 learned_addr learned my addr 172.21.15.7:0/3444868779 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:56.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.257+0000 7f077c949700 1 --2- 172.21.15.7:0/3444868779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7bec41640 0x55e7bec3f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:56.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.257+0000 7f077c949700 1 -- 172.21.15.7:0/3444868779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e7bec3bf20 msgr2=0x55e7bec3b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:56.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.257+0000 7f077c949700 1 --2- 172.21.15.7:0/3444868779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e7bec3bf20 0x55e7bec3b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.257+0000 7f077c949700 1 -- 172.21.15.7:0/3444868779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e7bec41f60 msgr2=0x55e7bec2fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:56.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.257+0000 7f077c949700 1 --2- 172.21.15.7:0/3444868779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e7bec41f60 0x55e7bec2fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.257+0000 7f077c949700 1 -- 172.21.15.7:0/3444868779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7bec43530 con 0x55e7bec41640 2024-06-10T01:12:56.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.257+0000 7f077d14a700 1 --2- 172.21.15.7:0/3444868779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e7bec3bf20 0x55e7bec3b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:56.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.258+0000 7f077c949700 1 --2- 172.21.15.7:0/3444868779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7bec41640 0x55e7bec3f5c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0774011920 tx=0x7f07740090e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74272b219e6868d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:56.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.258+0000 7f0762ffd700 1 -- 172.21.15.7:0/3444868779 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0774011bf0 con 0x55e7bec41640 2024-06-10T01:12:56.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.258+0000 7f0762ffd700 1 -- 172.21.15.7:0/3444868779 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0774029070 con 0x55e7bec41640 2024-06-10T01:12:56.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.258+0000 7f0762ffd700 1 -- 172.21.15.7:0/3444868779 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07740232c0 con 0x55e7bec41640 2024-06-10T01:12:56.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.259+0000 7f0789c31b80 1 -- 172.21.15.7:0/3444868779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7bec41640 msgr2=0x55e7bec3f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:56.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.259+0000 7f0789c31b80 1 --2- 172.21.15.7:0/3444868779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7bec41640 0x55e7bec3f5c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0774011920 tx=0x7f07740090e0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.259+0000 7f0789c31b80 1 -- 172.21.15.7:0/3444868779 shutdown_connections 2024-06-10T01:12:56.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.259+0000 7f0789c31b80 1 --2- 172.21.15.7:0/3444868779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e7bec41f60 0x55e7bec2fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.259+0000 7f0789c31b80 1 --2- 172.21.15.7:0/3444868779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e7bec3bf20 0x55e7bec3b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.259+0000 7f0789c31b80 1 --2- 172.21.15.7:0/3444868779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7bec41640 0x55e7bec3f5c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.259+0000 7f0789c31b80 1 -- 172.21.15.7:0/3444868779 >> 172.21.15.7:0/3444868779 conn(0x55e7beb66cc0 msgr2=0x55e7bec3eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:56.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.259+0000 7f0789c31b80 1 -- 172.21.15.7:0/3444868779 shutdown_connections 2024-06-10T01:12:56.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.259+0000 7f0789c31b80 1 -- 172.21.15.7:0/3444868779 wait complete. 2024-06-10T01:12:56.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.259+0000 7f0789c31b80 1 -- start start 2024-06-10T01:12:56.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.259+0000 7f0789c31b80 1 Processor -- start 2024-06-10T01:12:56.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.260+0000 7f0789c31b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e7bec3bf20 0x55e7bec4de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:56.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.260+0000 7f0789c31b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e7bec41640 0x55e7bec4e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:56.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.260+0000 7f0789c31b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7bec41f60 0x55e7bec5c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:56.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.260+0000 7f0789c31b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f075c00d050 con 0x55e7bec3bf20 2024-06-10T01:12:56.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.260+0000 7f0789c31b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f075c00e4c0 con 0x55e7bec41f60 2024-06-10T01:12:56.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.260+0000 7f0789c31b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f075c00d720 con 0x55e7bec41640 2024-06-10T01:12:56.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.260+0000 7f0789c31b80 10 client.0 _ll_register_callbacks cb 0x55e7beb27920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:12:56.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.260+0000 7f077d14a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e7bec3bf20 0x55e7bec4de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:56.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.260+0000 7f077c949700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e7bec41640 0x55e7bec4e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:56.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.260+0000 7f077c949700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e7bec41640 0x55e7bec4e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:42148/0 (socket says 172.21.15.7:42148) 2024-06-10T01:12:56.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.260+0000 7f077c949700 1 -- 172.21.15.7:0/2404864915 learned_addr learned my addr 172.21.15.7:0/2404864915 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:56.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.260+0000 7f077d94b700 1 --2- 172.21.15.7:0/2404864915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7bec41f60 0x55e7bec5c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:56.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.260+0000 7f077c949700 1 -- 172.21.15.7:0/2404864915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7bec41f60 msgr2=0x55e7bec5c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:56.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.260+0000 7f077c949700 1 --2- 172.21.15.7:0/2404864915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7bec41f60 0x55e7bec5c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.260+0000 7f077c949700 1 -- 172.21.15.7:0/2404864915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e7bec3bf20 msgr2=0x55e7bec4de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:56.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.260+0000 7f077c949700 1 --2- 172.21.15.7:0/2404864915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e7bec3bf20 0x55e7bec4de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.260+0000 7f077c949700 1 -- 172.21.15.7:0/2404864915 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7bec4ea40 con 0x55e7bec41640 2024-06-10T01:12:56.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.260+0000 7f077d14a700 1 --2- 172.21.15.7:0/2404864915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e7bec3bf20 0x55e7bec4de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:56.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.261+0000 7f077c949700 1 --2- 172.21.15.7:0/2404864915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e7bec41640 0x55e7bec4e3b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f07740047f0 tx=0x7f0774011c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=795b890d6b994b21 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:56.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.261+0000 7f0760ff9700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:12:56.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.261+0000 7f0760ff9700 1 -- 172.21.15.7:0/2404864915 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0774031070 con 0x55e7bec41640 2024-06-10T01:12:56.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.261+0000 7f0789c31b80 1 -- 172.21.15.7:0/2404864915 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0774029070 con 0x55e7bec41640 2024-06-10T01:12:56.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.261+0000 7f0760ff9700 1 -- 172.21.15.7:0/2404864915 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f07740109c0 con 0x55e7bec41640 2024-06-10T01:12:56.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.261+0000 7f0760ff9700 1 -- 172.21.15.7:0/2404864915 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0774011560 con 0x55e7bec41640 2024-06-10T01:12:56.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.261+0000 7f0789c31b80 10 client.26656 fetch_fsmap learned FSMap version 32 2024-06-10T01:12:56.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.261+0000 7f0789c31b80 1 -- 172.21.15.7:0/2404864915 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e7becbfa20 con 0x55e7bec41640 2024-06-10T01:12:56.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.261+0000 7f0760ff9700 1 -- 172.21.15.7:0/2404864915 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0774029070 con 0x55e7bec41640 2024-06-10T01:12:56.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.261+0000 7f0760ff9700 1 -- 172.21.15.7:0/2404864915 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f077401a3f0 con 0x55e7bec41640 2024-06-10T01:12:56.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.262+0000 7f0789c31b80 10 client.26656 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:12:56.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.262+0000 7f0789c31b80 10 client.26656 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:12:56.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.262+0000 7f0789c31b80 20 client.26656 populate_metadata read hostname 'smithi007' 2024-06-10T01:12:56.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.262+0000 7f0789c31b80 1 --2- 172.21.15.7:0/2404864915 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55e7becbff70 0x55e7becc2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:56.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.262+0000 7f0789c31b80 4 client.26656 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:12:56.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.262+0000 7f0789c31b80 1 -- 172.21.15.7:0/2404864915 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e7beb95290 con 0x55e7becbff70 2024-06-10T01:12:56.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.262+0000 7f077d14a700 1 --2- 172.21.15.7:0/2404864915 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55e7becbff70 0x55e7becc2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:56.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.262+0000 7f0760ff9700 10 client.26656 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:12:56.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.262+0000 7f077d14a700 1 --2- 172.21.15.7:0/2404864915 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55e7becbff70 0x55e7becc2390 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=408eab712af9aa7b server_cookie=72c66348b2288a6e in_seq=0 out_seq=0 2024-06-10T01:12:56.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.262+0000 7f0760ff9700 1 -- 172.21.15.7:0/2404864915 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e7beb95290 con 0x55e7becbff70 2024-06-10T01:12:56.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.262+0000 7f0760ff9700 10 client.26656 handle_command_reply: tid=0 2024-06-10T01:12:56.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.263+0000 7f0789c31b80 1 client.26656 shutdown 2024-06-10T01:12:56.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.263+0000 7f0789c31b80 1 -- 172.21.15.7:0/2404864915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e7bec41640 msgr2=0x55e7bec4e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:56.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.263+0000 7f0789c31b80 1 --2- 172.21.15.7:0/2404864915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e7bec41640 0x55e7bec4e3b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f07740047f0 tx=0x7f0774011c00 comp rx=0 tx=0).stop 2024-06-10T01:12:56.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.263+0000 7f0789c31b80 1 -- 172.21.15.7:0/2404864915 shutdown_connections 2024-06-10T01:12:56.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.263+0000 7f0789c31b80 1 --2- 172.21.15.7:0/2404864915 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55e7becbff70 0x55e7becc2390 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.263+0000 7f0789c31b80 1 --2- 172.21.15.7:0/2404864915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7bec41f60 0x55e7bec5c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.263+0000 7f0789c31b80 1 --2- 172.21.15.7:0/2404864915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e7bec3bf20 0x55e7bec4de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.263+0000 7f0789c31b80 1 --2- 172.21.15.7:0/2404864915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e7bec41640 0x55e7bec4e3b0 secure :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f07740047f0 tx=0x7f0774011c00 comp rx=0 tx=0).stop 2024-06-10T01:12:56.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.263+0000 7f0789c31b80 1 -- 172.21.15.7:0/2404864915 >> 172.21.15.7:0/2404864915 conn(0x55e7beb66cc0 msgr2=0x55e7bec2fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:56.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.263+0000 7f0760ff9700 0 client.26656 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:12:56.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.263+0000 7f0789c31b80 1 -- 172.21.15.7:0/2404864915 shutdown_connections 2024-06-10T01:12:56.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.263+0000 7f0789c31b80 1 -- 172.21.15.7:0/2404864915 wait complete. 2024-06-10T01:12:56.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.263+0000 7f0789c31b80 20 client.26656 trim_cache size 0 max 16384 2024-06-10T01:12:56.263 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:12:56.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.264+0000 7f077f3ae700 1 -- 172.21.15.7:0/1220572242 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f07640a2df0 msgr2=0x7f07640a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:56.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.264+0000 7f077f3ae700 1 --2- 172.21.15.7:0/1220572242 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f07640a2df0 0x7f07640a52b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f077000bd30 tx=0x7f0770009af0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.264+0000 7f077f3ae700 1 -- 172.21.15.7:0/1220572242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07781099d0 msgr2=0x7f077811b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:56.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.264+0000 7f077f3ae700 1 --2- 172.21.15.7:0/1220572242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07781099d0 0x7f077811b3e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f076c00e9c0 tx=0x7f076c00ed80 comp rx=0 tx=0).stop 2024-06-10T01:12:56.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.264+0000 7f077f3ae700 1 -- 172.21.15.7:0/1220572242 shutdown_connections 2024-06-10T01:12:56.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.264+0000 7f077f3ae700 1 --2- 172.21.15.7:0/1220572242 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f07640a2df0 0x7f07640a52b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.264+0000 7f077f3ae700 1 --2- 172.21.15.7:0/1220572242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f077811bc70 0x7f07781159f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.264+0000 7f077f3ae700 1 --2- 172.21.15.7:0/1220572242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07781099d0 0x7f077811b3e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.264+0000 7f077f3ae700 1 --2- 172.21.15.7:0/1220572242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0778110c70 0x7f07781154b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:56.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.264+0000 7f077f3ae700 1 -- 172.21.15.7:0/1220572242 >> 172.21.15.7:0/1220572242 conn(0x7f077806d600 msgr2=0x7f077811d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:56.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.264+0000 7f077f3ae700 1 -- 172.21.15.7:0/1220572242 shutdown_connections 2024-06-10T01:12:56.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:56.264+0000 7f077f3ae700 1 -- 172.21.15.7:0/1220572242 wait complete. 2024-06-10T01:12:56.736 INFO:tasks.workunit.client.0.smithi007.stdout: 1 456130 6.99 MB/sec execute 217 sec latency 58.337 ms 2024-06-10T01:12:56.926 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:12:57.186 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:12:57.579 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:57 smithi078 ceph-mon[36683]: pgmap v640: 129 pgs: 129 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 349 op/s 2024-06-10T01:12:57.646 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:57 smithi007 ceph-mon[32398]: pgmap v640: 129 pgs: 129 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 349 op/s 2024-06-10T01:12:57.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:57 smithi175 ceph-mon[36636]: pgmap v640: 129 pgs: 129 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 349 op/s 2024-06-10T01:12:57.736 INFO:tasks.workunit.client.0.smithi007.stdout: 1 458143 7.02 MB/sec execute 218 sec latency 11.864 ms 2024-06-10T01:12:58.736 INFO:tasks.workunit.client.0.smithi007.stdout: 1 459507 7.09 MB/sec execute 219 sec latency 46.786 ms 2024-06-10T01:12:58.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.845+0000 7f207a78a700 1 -- 172.21.15.7:0/3244073757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2074102e80 msgr2=0x7f207410f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:58.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.845+0000 7f207a78a700 1 --2- 172.21.15.7:0/3244073757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2074102e80 0x7f207410f330 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f206c00b0a0 tx=0x7f206c00b3b0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.845+0000 7f207a78a700 1 -- 172.21.15.7:0/3244073757 shutdown_connections 2024-06-10T01:12:58.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.845+0000 7f207a78a700 1 --2- 172.21.15.7:0/3244073757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2074102e80 0x7f207410f330 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.845+0000 7f207a78a700 1 --2- 172.21.15.7:0/3244073757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20741084f0 0x7f20741088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.845+0000 7f207a78a700 1 --2- 172.21.15.7:0/3244073757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20741024e0 0x7f2074102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.845+0000 7f207a78a700 1 -- 172.21.15.7:0/3244073757 >> 172.21.15.7:0/3244073757 conn(0x7f2074079530 msgr2=0x7f2074079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:58.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.845+0000 7f207a78a700 1 -- 172.21.15.7:0/3244073757 shutdown_connections 2024-06-10T01:12:58.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.845+0000 7f207a78a700 1 -- 172.21.15.7:0/3244073757 wait complete. 2024-06-10T01:12:58.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.846+0000 7f207a78a700 1 Processor -- start 2024-06-10T01:12:58.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.846+0000 7f207a78a700 1 -- start start 2024-06-10T01:12:58.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.846+0000 7f207a78a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20741024e0 0x7f207419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:58.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.846+0000 7f207a78a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2074102e80 0x7f207419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:58.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.846+0000 7f207a78a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20741084f0 0x7f20741989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:58.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.846+0000 7f207a78a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2074112770 con 0x7f20741024e0 2024-06-10T01:12:58.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.846+0000 7f207a78a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f20741125f0 con 0x7f2074102e80 2024-06-10T01:12:58.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.846+0000 7f207a78a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f20741128f0 con 0x7f20741084f0 2024-06-10T01:12:58.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.846+0000 7f2073fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20741024e0 0x7f207419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:58.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.846+0000 7f2073fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20741024e0 0x7f207419e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45084/0 (socket says 172.21.15.7:45084) 2024-06-10T01:12:58.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.846+0000 7f2073fff700 1 -- 172.21.15.7:0/773127699 learned_addr learned my addr 172.21.15.7:0/773127699 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:58.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.846+0000 7f2078d27700 1 --2- 172.21.15.7:0/773127699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20741084f0 0x7f20741989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:58.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.846+0000 7f206bfff700 1 --2- 172.21.15.7:0/773127699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2074102e80 0x7f207419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:58.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.847+0000 7f2073fff700 1 -- 172.21.15.7:0/773127699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20741084f0 msgr2=0x7f20741989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:58.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.847+0000 7f2073fff700 1 --2- 172.21.15.7:0/773127699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20741084f0 0x7f20741989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.847+0000 7f2073fff700 1 -- 172.21.15.7:0/773127699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2074102e80 msgr2=0x7f207419ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:58.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.847+0000 7f2073fff700 1 --2- 172.21.15.7:0/773127699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2074102e80 0x7f207419ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.847+0000 7f2073fff700 1 -- 172.21.15.7:0/773127699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2074199150 con 0x7f20741024e0 2024-06-10T01:12:58.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.847+0000 7f2078d27700 1 --2- 172.21.15.7:0/773127699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20741084f0 0x7f20741989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:58.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.847+0000 7f2073fff700 1 --2- 172.21.15.7:0/773127699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20741024e0 0x7f207419e830 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f206400e9a0 tx=0x7f206400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:58.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.847+0000 7f2071ffb700 1 -- 172.21.15.7:0/773127699 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206400ca10 con 0x7f20741024e0 2024-06-10T01:12:58.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.847+0000 7f207a78a700 1 -- 172.21.15.7:0/773127699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2074199440 con 0x7f20741024e0 2024-06-10T01:12:58.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.847+0000 7f2071ffb700 1 -- 172.21.15.7:0/773127699 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f206400cbb0 con 0x7f20741024e0 2024-06-10T01:12:58.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.847+0000 7f2071ffb700 1 -- 172.21.15.7:0/773127699 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064018680 con 0x7f20741024e0 2024-06-10T01:12:58.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.847+0000 7f207a78a700 1 -- 172.21.15.7:0/773127699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20741a5700 con 0x7f20741024e0 2024-06-10T01:12:58.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.849+0000 7f2071ffb700 1 -- 172.21.15.7:0/773127699 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f2064010430 con 0x7f20741024e0 2024-06-10T01:12:58.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.850+0000 7f2071ffb700 1 --2- 172.21.15.7:0/773127699 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f20600a2c20 0x7f20600a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:58.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.850+0000 7f2071ffb700 1 -- 172.21.15.7:0/773127699 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f20640cf570 con 0x7f20741024e0 2024-06-10T01:12:58.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.850+0000 7f206bfff700 1 --2- 172.21.15.7:0/773127699 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f20600a2c20 0x7f20600a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:58.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.851+0000 7f206bfff700 1 --2- 172.21.15.7:0/773127699 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f20600a2c20 0x7f20600a50e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2074079ae0 tx=0x7f205c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:58.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.851+0000 7f208500db80 1 Processor -- start 2024-06-10T01:12:58.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.851+0000 7f208500db80 1 -- start start 2024-06-10T01:12:58.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.851+0000 7f208500db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c504b8a70 0x558c504b8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:58.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.851+0000 7f208500db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c504b9390 0x558c504b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:58.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.851+0000 7f208500db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c504b3330 0x558c504b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:58.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.851+0000 7f208500db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558c50400810 con 0x558c504b8a70 2024-06-10T01:12:58.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.851+0000 7f208500db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558c503a8720 con 0x558c504b3330 2024-06-10T01:12:58.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.851+0000 7f208500db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558c503fc7d0 con 0x558c504b9390 2024-06-10T01:12:58.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.851+0000 7f2073fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c504b8a70 0x558c504b8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:58.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.851+0000 7f2073fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c504b8a70 0x558c504b8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45094/0 (socket says 172.21.15.7:45094) 2024-06-10T01:12:58.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.851+0000 7f2073fff700 1 -- 172.21.15.7:0/1961353596 learned_addr learned my addr 172.21.15.7:0/1961353596 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:58.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.851+0000 7f206bfff700 1 --2- 172.21.15.7:0/1961353596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c504b9390 0x558c504b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:58.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.851+0000 7f2078d27700 1 --2- 172.21.15.7:0/1961353596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c504b3330 0x558c504b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:58.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.852+0000 7f2073fff700 1 -- 172.21.15.7:0/1961353596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c504b9390 msgr2=0x558c504b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:58.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.852+0000 7f2073fff700 1 --2- 172.21.15.7:0/1961353596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c504b9390 0x558c504b2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.852+0000 7f2073fff700 1 -- 172.21.15.7:0/1961353596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c504b3330 msgr2=0x558c504b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:58.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.852+0000 7f2073fff700 1 --2- 172.21.15.7:0/1961353596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c504b3330 0x558c504b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.852+0000 7f2073fff700 1 -- 172.21.15.7:0/1961353596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c5039ef50 con 0x558c504b8a70 2024-06-10T01:12:58.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.852+0000 7f2078d27700 1 --2- 172.21.15.7:0/1961353596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c504b3330 0x558c504b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:58.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.852+0000 7f2073fff700 1 --2- 172.21.15.7:0/1961353596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c504b8a70 0x558c504b8e50 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f206409b9e0 tx=0x7f206409c2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a6323aa5be8eb81 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:58.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.852+0000 7f206a7fc700 1 -- 172.21.15.7:0/1961353596 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20640a46c0 con 0x558c504b8a70 2024-06-10T01:12:58.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.852+0000 7f206a7fc700 1 -- 172.21.15.7:0/1961353596 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f206400cd60 con 0x558c504b8a70 2024-06-10T01:12:58.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.852+0000 7f206a7fc700 1 -- 172.21.15.7:0/1961353596 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064010720 con 0x558c504b8a70 2024-06-10T01:12:58.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.853+0000 7f208500db80 1 -- 172.21.15.7:0/1961353596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c504b8a70 msgr2=0x558c504b8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:58.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.853+0000 7f208500db80 1 --2- 172.21.15.7:0/1961353596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c504b8a70 0x558c504b8e50 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f206409b9e0 tx=0x7f206409c2a0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.853+0000 7f208500db80 1 -- 172.21.15.7:0/1961353596 shutdown_connections 2024-06-10T01:12:58.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.853+0000 7f208500db80 1 --2- 172.21.15.7:0/1961353596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c504b3330 0x558c504b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.853+0000 7f208500db80 1 --2- 172.21.15.7:0/1961353596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c504b8a70 0x558c504b8e50 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.853+0000 7f208500db80 1 --2- 172.21.15.7:0/1961353596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c504b9390 0x558c504b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.853+0000 7f208500db80 1 -- 172.21.15.7:0/1961353596 >> 172.21.15.7:0/1961353596 conn(0x558c503d9cc0 msgr2=0x558c504a9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:58.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.853+0000 7f208500db80 1 -- 172.21.15.7:0/1961353596 shutdown_connections 2024-06-10T01:12:58.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.853+0000 7f208500db80 1 -- 172.21.15.7:0/1961353596 wait complete. 2024-06-10T01:12:58.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.853+0000 7f208500db80 1 -- start start 2024-06-10T01:12:58.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.853+0000 7f208500db80 1 Processor -- start 2024-06-10T01:12:58.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.854+0000 7f208500db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c504b3330 0x558c504bf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:58.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.854+0000 7f208500db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c504b8a70 0x558c504bc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:58.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.854+0000 7f208500db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c504b9390 0x558c504bca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:58.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.854+0000 7f208500db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558c504bfdd0 con 0x558c504b8a70 2024-06-10T01:12:58.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.854+0000 7f208500db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558c503a0bf0 con 0x558c504b3330 2024-06-10T01:12:58.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.854+0000 7f208500db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558c504bff50 con 0x558c504b9390 2024-06-10T01:12:58.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.854+0000 7f208500db80 10 client.0 _ll_register_callbacks cb 0x558c5039a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:12:58.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.854+0000 7f206bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c504b8a70 0x558c504bc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:58.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.854+0000 7f2078d27700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c504b9390 0x558c504bca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:58.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.854+0000 7f206bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c504b8a70 0x558c504bc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45102/0 (socket says 172.21.15.7:45102) 2024-06-10T01:12:58.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.854+0000 7f206bfff700 1 -- 172.21.15.7:0/1010611207 learned_addr learned my addr 172.21.15.7:0/1010611207 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:58.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.854+0000 7f2073fff700 1 --2- 172.21.15.7:0/1010611207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c504b3330 0x558c504bf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:58.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.854+0000 7f206bfff700 1 -- 172.21.15.7:0/1010611207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c504b9390 msgr2=0x558c504bca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:58.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.854+0000 7f206bfff700 1 --2- 172.21.15.7:0/1010611207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c504b9390 0x558c504bca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.854+0000 7f206bfff700 1 -- 172.21.15.7:0/1010611207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c504b3330 msgr2=0x558c504bf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:58.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.854+0000 7f206bfff700 1 --2- 172.21.15.7:0/1010611207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c504b3330 0x558c504bf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.854+0000 7f206bfff700 1 -- 172.21.15.7:0/1010611207 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c5034f8d0 con 0x558c504b8a70 2024-06-10T01:12:58.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.854+0000 7f2073fff700 1 --2- 172.21.15.7:0/1010611207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c504b3330 0x558c504bf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:58.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.854+0000 7f206bfff700 1 --2- 172.21.15.7:0/1010611207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c504b8a70 0x558c504bc510 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f205c004510 tx=0x7f205c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5677b98a428ee0d4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:58.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.854+0000 7f20537fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:12:58.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.855+0000 7f20537fe700 1 -- 172.21.15.7:0/1010611207 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205c004a30 con 0x558c504b8a70 2024-06-10T01:12:58.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.855+0000 7f20537fe700 1 -- 172.21.15.7:0/1010611207 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f205c004bd0 con 0x558c504b8a70 2024-06-10T01:12:58.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.855+0000 7f208500db80 1 -- 172.21.15.7:0/1010611207 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f206400cd60 con 0x558c504b8a70 2024-06-10T01:12:58.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.855+0000 7f20537fe700 1 -- 172.21.15.7:0/1010611207 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f205c004540 con 0x558c504b8a70 2024-06-10T01:12:58.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.855+0000 7f208500db80 10 client.17238 fetch_fsmap learned FSMap version 32 2024-06-10T01:12:58.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.855+0000 7f208500db80 1 -- 172.21.15.7:0/1010611207 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558c504ca850 con 0x558c504b8a70 2024-06-10T01:12:58.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.855+0000 7f20537fe700 1 -- 172.21.15.7:0/1010611207 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f206400cd60 con 0x558c504b8a70 2024-06-10T01:12:58.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.855+0000 7f20537fe700 1 -- 172.21.15.7:0/1010611207 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f205c026030 con 0x558c504b8a70 2024-06-10T01:12:58.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.855+0000 7f208500db80 10 client.17238 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:12:58.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.855+0000 7f208500db80 10 client.17238 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:12:58.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.855+0000 7f208500db80 20 client.17238 populate_metadata read hostname 'smithi007' 2024-06-10T01:12:58.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.855+0000 7f208500db80 1 --2- 172.21.15.7:0/1010611207 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c504cce90 0x558c504cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:58.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.855+0000 7f208500db80 4 client.17238 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:12:58.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.855+0000 7f208500db80 1 -- 172.21.15.7:0/1010611207 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c5040dd10 con 0x558c504cce90 2024-06-10T01:12:58.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.856+0000 7f2073fff700 1 --2- 172.21.15.7:0/1010611207 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c504cce90 0x558c504cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:58.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.856+0000 7f2073fff700 1 --2- 172.21.15.7:0/1010611207 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c504cce90 0x558c504cf280 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=599c9522879525fd server_cookie=a2924547d7c0a00 in_seq=0 out_seq=0 2024-06-10T01:12:58.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.856+0000 7f20537fe700 10 client.17238 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:58.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.857+0000 7f20537fe700 1 -- 172.21.15.7:0/1010611207 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558c5040dd10 con 0x558c504cce90 2024-06-10T01:12:58.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.857+0000 7f20537fe700 10 client.17238 handle_command_reply: tid=0 2024-06-10T01:12:58.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.857+0000 7f208500db80 1 client.17238 shutdown 2024-06-10T01:12:58.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.857+0000 7f208500db80 1 -- 172.21.15.7:0/1010611207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c504b8a70 msgr2=0x558c504bc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:58.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.857+0000 7f208500db80 1 --2- 172.21.15.7:0/1010611207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c504b8a70 0x558c504bc510 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f205c004510 tx=0x7f205c0047c0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.857+0000 7f208500db80 1 -- 172.21.15.7:0/1010611207 shutdown_connections 2024-06-10T01:12:58.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.857+0000 7f208500db80 1 --2- 172.21.15.7:0/1010611207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c504b9390 0x558c504bca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.857+0000 7f208500db80 1 --2- 172.21.15.7:0/1010611207 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c504cce90 0x558c504cf280 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.857+0000 7f208500db80 1 --2- 172.21.15.7:0/1010611207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c504b3330 0x558c504bf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.857+0000 7f208500db80 1 --2- 172.21.15.7:0/1010611207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c504b8a70 0x558c504bc510 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.857+0000 7f20537fe700 0 client.17238 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:58.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.857+0000 7f208500db80 1 -- 172.21.15.7:0/1010611207 >> 172.21.15.7:0/1010611207 conn(0x558c503d9cc0 msgr2=0x558c504ad9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:58.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.857+0000 7f208500db80 1 -- 172.21.15.7:0/1010611207 shutdown_connections 2024-06-10T01:12:58.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.857+0000 7f208500db80 1 -- 172.21.15.7:0/1010611207 wait complete. 2024-06-10T01:12:58.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.857+0000 7f208500db80 20 client.17238 trim_cache size 0 max 16384 2024-06-10T01:12:58.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.873+0000 7f208500db80 1 Processor -- start 2024-06-10T01:12:58.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.873+0000 7f208500db80 1 -- start start 2024-06-10T01:12:58.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.873+0000 7f208500db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c504b3330 0x558c504c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:58.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.873+0000 7f208500db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c504b8a70 0x558c504b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:58.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.873+0000 7f208500db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c504b9390 0x558c504b69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:58.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.873+0000 7f208500db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f20741ae990 con 0x558c504b3330 2024-06-10T01:12:58.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.873+0000 7f208500db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f20741ac220 con 0x558c504b9390 2024-06-10T01:12:58.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.873+0000 7f208500db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f20741aa070 con 0x558c504b8a70 2024-06-10T01:12:58.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.873+0000 7f2073fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c504b3330 0x558c504c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:58.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.873+0000 7f2073fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c504b3330 0x558c504c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45108/0 (socket says 172.21.15.7:45108) 2024-06-10T01:12:58.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.873+0000 7f2073fff700 1 -- 172.21.15.7:0/2411824290 learned_addr learned my addr 172.21.15.7:0/2411824290 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:58.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.873+0000 7f206bfff700 1 --2- 172.21.15.7:0/2411824290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c504b8a70 0x558c504b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:58.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.873+0000 7f2078d27700 1 --2- 172.21.15.7:0/2411824290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c504b9390 0x558c504b69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:58.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.873+0000 7f2073fff700 1 -- 172.21.15.7:0/2411824290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c504b8a70 msgr2=0x558c504b64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:58.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.873+0000 7f2073fff700 1 --2- 172.21.15.7:0/2411824290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c504b8a70 0x558c504b64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.873+0000 7f2073fff700 1 -- 172.21.15.7:0/2411824290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c504b9390 msgr2=0x558c504b69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:58.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.873+0000 7f2073fff700 1 --2- 172.21.15.7:0/2411824290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c504b9390 0x558c504b69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.873+0000 7f2073fff700 1 -- 172.21.15.7:0/2411824290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c504bcd80 con 0x558c504b3330 2024-06-10T01:12:58.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.873+0000 7f2078d27700 1 --2- 172.21.15.7:0/2411824290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c504b9390 0x558c504b69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:12:58.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.874+0000 7f2073fff700 1 --2- 172.21.15.7:0/2411824290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c504b3330 0x558c504c9070 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f2064098420 tx=0x7f206409e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9cfa6a0d84d0744 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:58.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.874+0000 7f2069ffb700 1 -- 172.21.15.7:0/2411824290 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064099d50 con 0x558c504b3330 2024-06-10T01:12:58.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.874+0000 7f2069ffb700 1 -- 172.21.15.7:0/2411824290 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20640cf2d0 con 0x558c504b3330 2024-06-10T01:12:58.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.874+0000 7f2069ffb700 1 -- 172.21.15.7:0/2411824290 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064010720 con 0x558c504b3330 2024-06-10T01:12:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.874+0000 7f208500db80 1 -- 172.21.15.7:0/2411824290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c504b3330 msgr2=0x558c504c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.874+0000 7f208500db80 1 --2- 172.21.15.7:0/2411824290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c504b3330 0x558c504c9070 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f2064098420 tx=0x7f206409e040 comp rx=0 tx=0).stop 2024-06-10T01:12:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.874+0000 7f208500db80 1 -- 172.21.15.7:0/2411824290 shutdown_connections 2024-06-10T01:12:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.874+0000 7f208500db80 1 --2- 172.21.15.7:0/2411824290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c504b9390 0x558c504b69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.874+0000 7f208500db80 1 --2- 172.21.15.7:0/2411824290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c504b3330 0x558c504c9070 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.874+0000 7f208500db80 1 --2- 172.21.15.7:0/2411824290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c504b8a70 0x558c504b64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.874+0000 7f208500db80 1 -- 172.21.15.7:0/2411824290 >> 172.21.15.7:0/2411824290 conn(0x558c503d9cc0 msgr2=0x558c504aaee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.875+0000 7f208500db80 1 -- 172.21.15.7:0/2411824290 shutdown_connections 2024-06-10T01:12:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.875+0000 7f208500db80 1 -- 172.21.15.7:0/2411824290 wait complete. 2024-06-10T01:12:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.875+0000 7f208500db80 1 -- start start 2024-06-10T01:12:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.875+0000 7f208500db80 1 Processor -- start 2024-06-10T01:12:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.875+0000 7f208500db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c504b3330 0x558c504c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.875+0000 7f208500db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c504b8a70 0x558c504c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.875+0000 7f208500db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c504b9390 0x558c504d3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.875+0000 7f208500db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f20741aa6e0 con 0x558c504b3330 2024-06-10T01:12:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.875+0000 7f208500db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f20741abb50 con 0x558c504b9390 2024-06-10T01:12:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.875+0000 7f208500db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f20741aadb0 con 0x558c504b8a70 2024-06-10T01:12:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.876+0000 7f208500db80 10 client.0 _ll_register_callbacks cb 0x558c5039a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:12:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.876+0000 7f2073fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c504b3330 0x558c504c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.876+0000 7f206bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c504b8a70 0x558c504c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.876+0000 7f2073fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c504b3330 0x558c504c51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45114/0 (socket says 172.21.15.7:45114) 2024-06-10T01:12:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.876+0000 7f2073fff700 1 -- 172.21.15.7:0/247729228 learned_addr learned my addr 172.21.15.7:0/247729228 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:12:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.876+0000 7f2078d27700 1 --2- 172.21.15.7:0/247729228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c504b9390 0x558c504d3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.876+0000 7f206bfff700 1 -- 172.21.15.7:0/247729228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c504b9390 msgr2=0x558c504d3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.876+0000 7f206bfff700 1 --2- 172.21.15.7:0/247729228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c504b9390 0x558c504d3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.876+0000 7f206bfff700 1 -- 172.21.15.7:0/247729228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c504b3330 msgr2=0x558c504c51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:12:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.876+0000 7f206bfff700 1 --2- 172.21.15.7:0/247729228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c504b3330 0x558c504c51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.876+0000 7f206bfff700 1 -- 172.21.15.7:0/247729228 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c504d40e0 con 0x558c504b8a70 2024-06-10T01:12:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.876+0000 7f2073fff700 1 --2- 172.21.15.7:0/247729228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c504b3330 0x558c504c51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:12:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.876+0000 7f206bfff700 1 --2- 172.21.15.7:0/247729228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c504b8a70 0x558c504c5720 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f205c01e9d0 tx=0x7f205c011b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1169df02b5b911bc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.876+0000 7f20537fe700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:12:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.876+0000 7f20537fe700 1 -- 172.21.15.7:0/247729228 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205c039070 con 0x558c504b8a70 2024-06-10T01:12:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.876+0000 7f208500db80 1 -- 172.21.15.7:0/247729228 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20640cf2d0 con 0x558c504b8a70 2024-06-10T01:12:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.876+0000 7f20537fe700 1 -- 172.21.15.7:0/247729228 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f205c004a30 con 0x558c504b8a70 2024-06-10T01:12:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.877+0000 7f20537fe700 1 -- 172.21.15.7:0/247729228 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f205c044050 con 0x558c504b8a70 2024-06-10T01:12:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.877+0000 7f208500db80 10 client.26674 fetch_fsmap learned FSMap version 32 2024-06-10T01:12:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.877+0000 7f208500db80 1 -- 172.21.15.7:0/247729228 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558c505372c0 con 0x558c504b8a70 2024-06-10T01:12:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.877+0000 7f20537fe700 1 -- 172.21.15.7:0/247729228 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20640cf2d0 con 0x558c504b8a70 2024-06-10T01:12:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.877+0000 7f20537fe700 1 -- 172.21.15.7:0/247729228 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f205c026040 con 0x558c504b8a70 2024-06-10T01:12:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.877+0000 7f208500db80 10 client.26674 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T01:12:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.877+0000 7f208500db80 10 client.26674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:12:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.877+0000 7f208500db80 20 client.26674 populate_metadata read hostname 'smithi007' 2024-06-10T01:12:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.877+0000 7f208500db80 1 --2- 172.21.15.7:0/247729228 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c50537a00 0x558c50539e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.877+0000 7f208500db80 4 client.26674 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:12:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.877+0000 7f208500db80 1 -- 172.21.15.7:0/247729228 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558c50408290 con 0x558c50537a00 2024-06-10T01:12:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.877+0000 7f2073fff700 1 --2- 172.21.15.7:0/247729228 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c50537a00 0x558c50539e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.878+0000 7f2073fff700 1 --2- 172.21.15.7:0/247729228 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c50537a00 0x558c50539e20 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=892bc9ee8effc2b0 server_cookie=61b957d417868bc4 in_seq=0 out_seq=0 2024-06-10T01:12:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.878+0000 7f20537fe700 10 client.26674 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.878+0000 7f20537fe700 1 -- 172.21.15.7:0/247729228 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558c50408290 con 0x558c50537a00 2024-06-10T01:12:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.878+0000 7f20537fe700 10 client.26674 handle_command_reply: tid=0 2024-06-10T01:12:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.878+0000 7f208500db80 1 client.26674 shutdown 2024-06-10T01:12:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.878+0000 7f208500db80 1 -- 172.21.15.7:0/247729228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c504b8a70 msgr2=0x558c504c5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.878+0000 7f208500db80 1 --2- 172.21.15.7:0/247729228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c504b8a70 0x558c504c5720 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f205c01e9d0 tx=0x7f205c011b60 comp rx=0 tx=0).stop 2024-06-10T01:12:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.878+0000 7f208500db80 1 -- 172.21.15.7:0/247729228 shutdown_connections 2024-06-10T01:12:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.878+0000 7f208500db80 1 --2- 172.21.15.7:0/247729228 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c50537a00 0x558c50539e20 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.878+0000 7f208500db80 1 --2- 172.21.15.7:0/247729228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c504b9390 0x558c504d3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.878+0000 7f20537fe700 0 client.26674 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:12:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.878+0000 7f208500db80 1 --2- 172.21.15.7:0/247729228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c504b3330 0x558c504c51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.878+0000 7f208500db80 1 --2- 172.21.15.7:0/247729228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c504b8a70 0x558c504c5720 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.878+0000 7f208500db80 1 -- 172.21.15.7:0/247729228 >> 172.21.15.7:0/247729228 conn(0x558c503d9cc0 msgr2=0x558c504a9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.878+0000 7f208500db80 1 -- 172.21.15.7:0/247729228 shutdown_connections 2024-06-10T01:12:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.878+0000 7f208500db80 1 -- 172.21.15.7:0/247729228 wait complete. 2024-06-10T01:12:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.878+0000 7f208500db80 20 client.26674 trim_cache size 0 max 16384 2024-06-10T01:12:58.878 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:12:58.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.879+0000 7f207a78a700 1 -- 172.21.15.7:0/773127699 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f20600a2c20 msgr2=0x7f20600a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:58.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.879+0000 7f207a78a700 1 --2- 172.21.15.7:0/773127699 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f20600a2c20 0x7f20600a50e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2074079ae0 tx=0x7f205c009260 comp rx=0 tx=0).stop 2024-06-10T01:12:58.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.879+0000 7f207a78a700 1 -- 172.21.15.7:0/773127699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20741024e0 msgr2=0x7f207419e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:58.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.879+0000 7f207a78a700 1 --2- 172.21.15.7:0/773127699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20741024e0 0x7f207419e830 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f206400e9a0 tx=0x7f206400ecb0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.879+0000 7f207a78a700 1 -- 172.21.15.7:0/773127699 shutdown_connections 2024-06-10T01:12:58.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.879+0000 7f207a78a700 1 --2- 172.21.15.7:0/773127699 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f20600a2c20 0x7f20600a50e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.879+0000 7f207a78a700 1 --2- 172.21.15.7:0/773127699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20741084f0 0x7f20741989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.879+0000 7f207a78a700 1 --2- 172.21.15.7:0/773127699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20741024e0 0x7f207419e830 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.879+0000 7f207a78a700 1 --2- 172.21.15.7:0/773127699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2074102e80 0x7f207419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:58.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.879+0000 7f207a78a700 1 -- 172.21.15.7:0/773127699 >> 172.21.15.7:0/773127699 conn(0x7f2074079530 msgr2=0x7f20741004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:58.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.879+0000 7f207a78a700 1 -- 172.21.15.7:0/773127699 shutdown_connections 2024-06-10T01:12:58.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:12:58.879+0000 7f207a78a700 1 -- 172.21.15.7:0/773127699 wait complete. 2024-06-10T01:12:59.579 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:12:59 smithi078 ceph-mon[36683]: pgmap v641: 129 pgs: 129 active+clean; 87 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 358 op/s 2024-06-10T01:12:59.604 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:12:59 smithi007 ceph-mon[32398]: pgmap v641: 129 pgs: 129 active+clean; 87 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 358 op/s 2024-06-10T01:12:59.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:12:59 smithi175 ceph-mon[36636]: pgmap v641: 129 pgs: 129 active+clean; 87 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 358 op/s 2024-06-10T01:12:59.736 INFO:tasks.workunit.client.0.smithi007.stdout: 1 462186 7.13 MB/sec execute 220 sec latency 20.828 ms 2024-06-10T01:13:00.579 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:00 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:13:00.633 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:13:00.633 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:13:00.657 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:00 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:13:00.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:00 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:13:00.736 INFO:tasks.workunit.client.0.smithi007.stdout: 1 463764 7.13 MB/sec execute 221 sec latency 42.024 ms 2024-06-10T01:13:00.906 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:13:01.579 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:01 smithi078 ceph-mon[36683]: pgmap v642: 129 pgs: 129 active+clean; 86 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 380 op/s 2024-06-10T01:13:01.688 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:01 smithi007 ceph-mon[32398]: pgmap v642: 129 pgs: 129 active+clean; 86 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 380 op/s 2024-06-10T01:13:01.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:01 smithi175 ceph-mon[36636]: pgmap v642: 129 pgs: 129 active+clean; 86 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 380 op/s 2024-06-10T01:13:01.736 INFO:tasks.workunit.client.0.smithi007.stdout: 1 466122 7.16 MB/sec execute 222 sec latency 13.936 ms 2024-06-10T01:13:02.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.636+0000 7fa1c35d9700 1 -- 172.21.15.7:0/3382160191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1bc06b0a0 msgr2=0x7fa1bc06b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:02.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.636+0000 7fa1c35d9700 1 --2- 172.21.15.7:0/3382160191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1bc06b0a0 0x7fa1bc06b500 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac0098f0 tx=0x7fa1ac009c00 comp rx=0 tx=0).stop 2024-06-10T01:13:02.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.636+0000 7fa1c35d9700 1 -- 172.21.15.7:0/3382160191 shutdown_connections 2024-06-10T01:13:02.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.636+0000 7fa1c35d9700 1 --2- 172.21.15.7:0/3382160191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1bc10ede0 0x7fa1bc1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.636+0000 7fa1c35d9700 1 --2- 172.21.15.7:0/3382160191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1bc1033a0 0x7fa1bc06ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.636+0000 7fa1c35d9700 1 --2- 172.21.15.7:0/3382160191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1bc06b0a0 0x7fa1bc06b500 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.636+0000 7fa1c35d9700 1 -- 172.21.15.7:0/3382160191 >> 172.21.15.7:0/3382160191 conn(0x7fa1bc078340 msgr2=0x7fa1bc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:02.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.637+0000 7fa1c35d9700 1 -- 172.21.15.7:0/3382160191 shutdown_connections 2024-06-10T01:13:02.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.637+0000 7fa1c35d9700 1 -- 172.21.15.7:0/3382160191 wait complete. 2024-06-10T01:13:02.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.637+0000 7fa1c35d9700 1 Processor -- start 2024-06-10T01:13:02.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.637+0000 7fa1c35d9700 1 -- start start 2024-06-10T01:13:02.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.637+0000 7fa1c35d9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1bc06b0a0 0x7fa1bc19aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:02.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.637+0000 7fa1c35d9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1bc1033a0 0x7fa1bc19b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:02.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.637+0000 7fa1c35d9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1bc10ede0 0x7fa1bc19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:02.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.637+0000 7fa1c35d9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc1149a0 con 0x7fa1bc1033a0 2024-06-10T01:13:02.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.637+0000 7fa1c35d9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc114820 con 0x7fa1bc06b0a0 2024-06-10T01:13:02.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.637+0000 7fa1c35d9700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc114b20 con 0x7fa1bc10ede0 2024-06-10T01:13:02.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.638+0000 7fa1c1375700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1bc06b0a0 0x7fa1bc19aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:02.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.638+0000 7fa1c1b76700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1bc10ede0 0x7fa1bc19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:02.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.638+0000 7fa1c0b74700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1bc1033a0 0x7fa1bc19b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:02.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.638+0000 7fa1c1375700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1bc06b0a0 0x7fa1bc19aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:41716/0 (socket says 172.21.15.7:41716) 2024-06-10T01:13:02.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.638+0000 7fa1c1375700 1 -- 172.21.15.7:0/1535695496 learned_addr learned my addr 172.21.15.7:0/1535695496 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.638+0000 7fa1c0b74700 1 -- 172.21.15.7:0/1535695496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1bc10ede0 msgr2=0x7fa1bc19f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.638+0000 7fa1c0b74700 1 --2- 172.21.15.7:0/1535695496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1bc10ede0 0x7fa1bc19f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.638+0000 7fa1c0b74700 1 -- 172.21.15.7:0/1535695496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1bc06b0a0 msgr2=0x7fa1bc19aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.638+0000 7fa1c0b74700 1 --2- 172.21.15.7:0/1535695496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1bc06b0a0 0x7fa1bc19aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.638+0000 7fa1c0b74700 1 -- 172.21.15.7:0/1535695496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1bc19fec0 con 0x7fa1bc1033a0 2024-06-10T01:13:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.638+0000 7fa1c1b76700 1 --2- 172.21.15.7:0/1535695496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1bc10ede0 0x7fa1bc19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.638+0000 7fa1c1375700 1 --2- 172.21.15.7:0/1535695496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1bc06b0a0 0x7fa1bc19aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.638+0000 7fa1c0b74700 1 --2- 172.21.15.7:0/1535695496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1bc1033a0 0x7fa1bc19b3f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00c010 tx=0x7fa1ac00fb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.639+0000 7fa1b27fc700 1 -- 172.21.15.7:0/1535695496 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac005280 con 0x7fa1bc1033a0 2024-06-10T01:13:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.639+0000 7fa1b27fc700 1 -- 172.21.15.7:0/1535695496 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1ac00ba40 con 0x7fa1bc1033a0 2024-06-10T01:13:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.639+0000 7fa1c35d9700 1 -- 172.21.15.7:0/1535695496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1bc1a0150 con 0x7fa1bc1033a0 2024-06-10T01:13:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.639+0000 7fa1b27fc700 1 -- 172.21.15.7:0/1535695496 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac003b20 con 0x7fa1bc1033a0 2024-06-10T01:13:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.639+0000 7fa1c35d9700 1 -- 172.21.15.7:0/1535695496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1bc1a7a30 con 0x7fa1bc1033a0 2024-06-10T01:13:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.641+0000 7fa1b27fc700 1 -- 172.21.15.7:0/1535695496 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa1ac00bbf0 con 0x7fa1bc1033a0 2024-06-10T01:13:02.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.642+0000 7fa1b27fc700 1 --2- 172.21.15.7:0/1535695496 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa1a80a2c20 0x7fa1a80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:02.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.643+0000 7fa1c1375700 1 --2- 172.21.15.7:0/1535695496 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa1a80a2c20 0x7fa1a80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:02.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.643+0000 7fa1b27fc700 1 -- 172.21.15.7:0/1535695496 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa1ac0d01e0 con 0x7fa1bc1033a0 2024-06-10T01:13:02.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.643+0000 7fa1cde5cb80 1 Processor -- start 2024-06-10T01:13:02.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.643+0000 7fa1cde5cb80 1 -- start start 2024-06-10T01:13:02.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.643+0000 7fa1c1375700 1 --2- 172.21.15.7:0/1535695496 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa1a80a2c20 0x7fa1a80a50e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8005950 tx=0x7fa1b8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:02.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.643+0000 7fa1cde5cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddd378cfe0 0x55ddd378d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:02.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.643+0000 7fa1cde5cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddd378d900 0x55ddd37872e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:02.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.643+0000 7fa1cde5cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd3787820 0x55ddd3787c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.643+0000 7fa1cde5cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ddd3679fe0 con 0x55ddd3787820 2024-06-10T01:13:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.643+0000 7fa1cde5cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ddd36d88d0 con 0x55ddd378d900 2024-06-10T01:13:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.643+0000 7fa1cde5cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ddd36404e0 con 0x55ddd378cfe0 2024-06-10T01:13:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.644+0000 7fa1c1375700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddd378cfe0 0x55ddd378d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.644+0000 7fa1c0b74700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddd378d900 0x55ddd37872e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.644+0000 7fa1c1375700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddd378cfe0 0x55ddd378d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:56758/0 (socket says 172.21.15.7:56758) 2024-06-10T01:13:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.644+0000 7fa1c1375700 1 -- 172.21.15.7:0/4220668289 learned_addr learned my addr 172.21.15.7:0/4220668289 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.644+0000 7fa1c1b76700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd3787820 0x55ddd3787c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.644+0000 7fa1c1375700 1 -- 172.21.15.7:0/4220668289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddd378d900 msgr2=0x55ddd37872e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.644+0000 7fa1c1375700 1 --2- 172.21.15.7:0/4220668289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddd378d900 0x55ddd37872e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.644+0000 7fa1c1375700 1 -- 172.21.15.7:0/4220668289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd3787820 msgr2=0x55ddd3787c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.644+0000 7fa1c1375700 1 --2- 172.21.15.7:0/4220668289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd3787820 0x55ddd3787c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.644+0000 7fa1c1375700 1 -- 172.21.15.7:0/4220668289 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddd3627950 con 0x55ddd378cfe0 2024-06-10T01:13:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.644+0000 7fa1c1375700 1 --2- 172.21.15.7:0/4220668289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddd378cfe0 0x55ddd378d3c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa1b800e6e0 tx=0x7fa1b8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4b51bc749949941 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.644+0000 7fa1a77fe700 1 -- 172.21.15.7:0/4220668289 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b8026630 con 0x55ddd378cfe0 2024-06-10T01:13:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.644+0000 7fa1a77fe700 1 -- 172.21.15.7:0/4220668289 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1b801c8f0 con 0x55ddd378cfe0 2024-06-10T01:13:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.644+0000 7fa1a77fe700 1 -- 172.21.15.7:0/4220668289 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b801d930 con 0x55ddd378cfe0 2024-06-10T01:13:02.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.645+0000 7fa1cde5cb80 1 -- 172.21.15.7:0/4220668289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddd378cfe0 msgr2=0x55ddd378d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:02.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.645+0000 7fa1cde5cb80 1 --2- 172.21.15.7:0/4220668289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddd378cfe0 0x55ddd378d3c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa1b800e6e0 tx=0x7fa1b8014000 comp rx=0 tx=0).stop 2024-06-10T01:13:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.645+0000 7fa1cde5cb80 1 -- 172.21.15.7:0/4220668289 shutdown_connections 2024-06-10T01:13:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.645+0000 7fa1cde5cb80 1 --2- 172.21.15.7:0/4220668289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd3787820 0x55ddd3787c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.645+0000 7fa1cde5cb80 1 --2- 172.21.15.7:0/4220668289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddd378cfe0 0x55ddd378d3c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.645+0000 7fa1cde5cb80 1 --2- 172.21.15.7:0/4220668289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddd378d900 0x55ddd37872e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.645+0000 7fa1cde5cb80 1 -- 172.21.15.7:0/4220668289 >> 172.21.15.7:0/4220668289 conn(0x55ddd36b4100 msgr2=0x55ddd37822a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.645+0000 7fa1cde5cb80 1 -- 172.21.15.7:0/4220668289 shutdown_connections 2024-06-10T01:13:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.645+0000 7fa1cde5cb80 1 -- 172.21.15.7:0/4220668289 wait complete. 2024-06-10T01:13:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.646+0000 7fa1cde5cb80 1 -- start start 2024-06-10T01:13:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.646+0000 7fa1cde5cb80 1 Processor -- start 2024-06-10T01:13:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.646+0000 7fa1cde5cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddd3787820 0x55ddd37936d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.646+0000 7fa1cde5cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd378cfe0 0x55ddd37909d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.646+0000 7fa1cde5cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddd378d900 0x55ddd3790f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.646+0000 7fa1cde5cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ddd3679040 con 0x55ddd378cfe0 2024-06-10T01:13:02.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.646+0000 7fa1cde5cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ddd36d4890 con 0x55ddd378d900 2024-06-10T01:13:02.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.646+0000 7fa1cde5cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ddd37942e0 con 0x55ddd3787820 2024-06-10T01:13:02.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.646+0000 7fa1cde5cb80 10 client.0 _ll_register_callbacks cb 0x55ddd366e630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:13:02.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.646+0000 7fa1c1375700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddd3787820 0x55ddd37936d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:02.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.647+0000 7fa1c1375700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddd3787820 0x55ddd37936d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:56772/0 (socket says 172.21.15.7:56772) 2024-06-10T01:13:02.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.647+0000 7fa1c1375700 1 -- 172.21.15.7:0/737364375 learned_addr learned my addr 172.21.15.7:0/737364375 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:02.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.647+0000 7fa1c1b76700 1 --2- 172.21.15.7:0/737364375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddd378d900 0x55ddd3790f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:02.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.647+0000 7fa1c0b74700 1 --2- 172.21.15.7:0/737364375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd378cfe0 0x55ddd37909d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:02.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.647+0000 7fa1c1b76700 1 -- 172.21.15.7:0/737364375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddd3787820 msgr2=0x55ddd37936d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:02.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.647+0000 7fa1c1b76700 1 --2- 172.21.15.7:0/737364375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddd3787820 0x55ddd37936d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.647+0000 7fa1c1b76700 1 -- 172.21.15.7:0/737364375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd378cfe0 msgr2=0x55ddd37909d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:02.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.647+0000 7fa1c1b76700 1 --2- 172.21.15.7:0/737364375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd378cfe0 0x55ddd37909d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.647+0000 7fa1c1b76700 1 -- 172.21.15.7:0/737364375 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddd369e7d0 con 0x55ddd378d900 2024-06-10T01:13:02.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.647+0000 7fa1c1b76700 1 --2- 172.21.15.7:0/737364375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddd378d900 0x55ddd3790f10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4009550 tx=0x7fa1b4004630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db4f3e5df1d5929e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:02.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.647+0000 7fa1a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:13:02.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.647+0000 7fa1a4ff9700 1 -- 172.21.15.7:0/737364375 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b401a070 con 0x55ddd378d900 2024-06-10T01:13:02.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.647+0000 7fa1a4ff9700 1 -- 172.21.15.7:0/737364375 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa1b40079d0 con 0x55ddd378d900 2024-06-10T01:13:02.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.647+0000 7fa1cde5cb80 1 -- 172.21.15.7:0/737364375 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1b801c8f0 con 0x55ddd378d900 2024-06-10T01:13:02.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.648+0000 7fa1a4ff9700 1 -- 172.21.15.7:0/737364375 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa1b4026050 con 0x55ddd378d900 2024-06-10T01:13:02.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.648+0000 7fa1cde5cb80 10 client.26669 fetch_fsmap learned FSMap version 33 2024-06-10T01:13:02.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.648+0000 7fa1cde5cb80 1 -- 172.21.15.7:0/737364375 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ddd379eb20 con 0x55ddd378d900 2024-06-10T01:13:02.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.648+0000 7fa1a4ff9700 1 -- 172.21.15.7:0/737364375 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1b4007bd0 con 0x55ddd378d900 2024-06-10T01:13:02.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.648+0000 7fa1a4ff9700 1 -- 172.21.15.7:0/737364375 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7fa1b401e3b0 con 0x55ddd378d900 2024-06-10T01:13:02.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.648+0000 7fa1cde5cb80 10 client.26669 fetch_fsmap finished waiting for FSMap version 33 2024-06-10T01:13:02.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.648+0000 7fa1cde5cb80 10 client.26669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:13:02.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.648+0000 7fa1cde5cb80 20 client.26669 populate_metadata read hostname 'smithi007' 2024-06-10T01:13:02.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.648+0000 7fa1cde5cb80 1 --2- 172.21.15.7:0/737364375 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ddd37a1370 0x55ddd37a3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:02.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.648+0000 7fa1cde5cb80 4 client.26669 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:13:02.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.648+0000 7fa1cde5cb80 1 -- 172.21.15.7:0/737364375 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa1b801c8f0 con 0x55ddd37a1370 2024-06-10T01:13:02.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.649+0000 7fa1c1375700 1 --2- 172.21.15.7:0/737364375 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ddd37a1370 0x55ddd37a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:02.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.649+0000 7fa1a4ff9700 10 client.26669 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:02.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.649+0000 7fa1c1375700 1 --2- 172.21.15.7:0/737364375 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ddd37a1370 0x55ddd37a3790 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75dd31f04280422a server_cookie=a14c4fa12cd11a2e in_seq=0 out_seq=0 2024-06-10T01:13:02.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.651+0000 7fa1a4ff9700 1 -- 172.21.15.7:0/737364375 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fa1b801c8f0 con 0x55ddd37a1370 2024-06-10T01:13:02.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.651+0000 7fa1a4ff9700 10 client.26669 handle_command_reply: tid=0 2024-06-10T01:13:02.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.651+0000 7fa1cde5cb80 1 client.26669 shutdown 2024-06-10T01:13:02.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.651+0000 7fa1cde5cb80 1 -- 172.21.15.7:0/737364375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddd378d900 msgr2=0x55ddd3790f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:02.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.651+0000 7fa1cde5cb80 1 --2- 172.21.15.7:0/737364375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddd378d900 0x55ddd3790f10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4009550 tx=0x7fa1b4004630 comp rx=0 tx=0).stop 2024-06-10T01:13:02.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.651+0000 7fa1cde5cb80 1 -- 172.21.15.7:0/737364375 shutdown_connections 2024-06-10T01:13:02.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.651+0000 7fa1cde5cb80 1 --2- 172.21.15.7:0/737364375 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ddd37a1370 0x55ddd37a3790 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.651+0000 7fa1cde5cb80 1 --2- 172.21.15.7:0/737364375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddd378d900 0x55ddd3790f10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.651+0000 7fa1cde5cb80 1 --2- 172.21.15.7:0/737364375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddd3787820 0x55ddd37936d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.651+0000 7fa1cde5cb80 1 --2- 172.21.15.7:0/737364375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd378cfe0 0x55ddd37909d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.651+0000 7fa1cde5cb80 1 -- 172.21.15.7:0/737364375 >> 172.21.15.7:0/737364375 conn(0x55ddd36b4100 msgr2=0x55ddd3781780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:02.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.651+0000 7fa1a4ff9700 0 client.26669 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:02.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.651+0000 7fa1cde5cb80 1 -- 172.21.15.7:0/737364375 shutdown_connections 2024-06-10T01:13:02.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.651+0000 7fa1cde5cb80 1 -- 172.21.15.7:0/737364375 wait complete. 2024-06-10T01:13:02.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.651+0000 7fa1cde5cb80 20 client.26669 trim_cache size 0 max 16384 2024-06-10T01:13:02.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.677+0000 7fa1cde5cb80 1 Processor -- start 2024-06-10T01:13:02.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.677+0000 7fa1cde5cb80 1 -- start start 2024-06-10T01:13:02.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.678+0000 7fa1cde5cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd3787820 0x55ddd37872e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:02.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.678+0000 7fa1cde5cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddd378cfe0 0x55ddd378f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:02.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.678+0000 7fa1cde5cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddd378d900 0x55ddd378e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:02.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.678+0000 7fa1cde5cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc1b2d60 con 0x55ddd3787820 2024-06-10T01:13:02.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.678+0000 7fa1cde5cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc1b05f0 con 0x55ddd378cfe0 2024-06-10T01:13:02.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.678+0000 7fa1cde5cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc1ae440 con 0x55ddd378d900 2024-06-10T01:13:02.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.678+0000 7fa1c1375700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd3787820 0x55ddd37872e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:02.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.678+0000 7fa1c1b76700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddd378d900 0x55ddd378e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:02.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.678+0000 7fa1c0b74700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddd378cfe0 0x55ddd378f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:02.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.678+0000 7fa1c1375700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd3787820 0x55ddd37872e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52928/0 (socket says 172.21.15.7:52928) 2024-06-10T01:13:02.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.678+0000 7fa1c1375700 1 -- 172.21.15.7:0/3368413570 learned_addr learned my addr 172.21.15.7:0/3368413570 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:02.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.678+0000 7fa1c0b74700 1 -- 172.21.15.7:0/3368413570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddd378d900 msgr2=0x55ddd378e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:02.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.678+0000 7fa1c0b74700 1 --2- 172.21.15.7:0/3368413570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddd378d900 0x55ddd378e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.678+0000 7fa1c0b74700 1 -- 172.21.15.7:0/3368413570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd3787820 msgr2=0x55ddd37872e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:02.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.678+0000 7fa1c0b74700 1 --2- 172.21.15.7:0/3368413570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd3787820 0x55ddd37872e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.678+0000 7fa1c0b74700 1 -- 172.21.15.7:0/3368413570 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddd3791f30 con 0x55ddd378cfe0 2024-06-10T01:13:02.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.678+0000 7fa1c1375700 1 --2- 172.21.15.7:0/3368413570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd3787820 0x55ddd37872e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:02.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.678+0000 7fa1c0b74700 1 --2- 172.21.15.7:0/3368413570 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddd378cfe0 0x55ddd378f950 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac0cd9b0 tx=0x7fa1ac0a3040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1461b64fe4c5368 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:02.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.679+0000 7fa1a6ffd700 1 -- 172.21.15.7:0/3368413570 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac0b5070 con 0x55ddd378cfe0 2024-06-10T01:13:02.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.679+0000 7fa1a6ffd700 1 -- 172.21.15.7:0/3368413570 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1ac005420 con 0x55ddd378cfe0 2024-06-10T01:13:02.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.679+0000 7fa1a6ffd700 1 -- 172.21.15.7:0/3368413570 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac0870b0 con 0x55ddd378cfe0 2024-06-10T01:13:02.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.680+0000 7fa1cde5cb80 1 -- 172.21.15.7:0/3368413570 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddd378cfe0 msgr2=0x55ddd378f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:02.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.680+0000 7fa1cde5cb80 1 --2- 172.21.15.7:0/3368413570 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddd378cfe0 0x55ddd378f950 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac0cd9b0 tx=0x7fa1ac0a3040 comp rx=0 tx=0).stop 2024-06-10T01:13:02.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.680+0000 7fa1cde5cb80 1 -- 172.21.15.7:0/3368413570 shutdown_connections 2024-06-10T01:13:02.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.680+0000 7fa1cde5cb80 1 --2- 172.21.15.7:0/3368413570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddd378d900 0x55ddd378e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.680+0000 7fa1cde5cb80 1 --2- 172.21.15.7:0/3368413570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd3787820 0x55ddd37872e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.680+0000 7fa1cde5cb80 1 --2- 172.21.15.7:0/3368413570 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddd378cfe0 0x55ddd378f950 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.680+0000 7fa1cde5cb80 1 -- 172.21.15.7:0/3368413570 >> 172.21.15.7:0/3368413570 conn(0x55ddd36b4100 msgr2=0x55ddd3792200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:02.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.680+0000 7fa1cde5cb80 1 -- 172.21.15.7:0/3368413570 shutdown_connections 2024-06-10T01:13:02.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.680+0000 7fa1cde5cb80 1 -- 172.21.15.7:0/3368413570 wait complete. 2024-06-10T01:13:02.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.680+0000 7fa1cde5cb80 1 -- start start 2024-06-10T01:13:02.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.680+0000 7fa1cde5cb80 1 Processor -- start 2024-06-10T01:13:02.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.680+0000 7fa1cde5cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddd3787820 0x55ddd3799860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:02.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.681+0000 7fa1cde5cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd378cfe0 0x55ddd3799da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:02.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.681+0000 7fa1cde5cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddd378d900 0x55ddd37965b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:02.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.681+0000 7fa1cde5cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc1aeab0 con 0x55ddd378cfe0 2024-06-10T01:13:02.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.681+0000 7fa1cde5cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc1aff20 con 0x55ddd378d900 2024-06-10T01:13:02.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.681+0000 7fa1cde5cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc1af180 con 0x55ddd3787820 2024-06-10T01:13:02.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.681+0000 7fa1cde5cb80 10 client.0 _ll_register_callbacks cb 0x55ddd366e630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:13:02.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.681+0000 7fa1c1b76700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddd378d900 0x55ddd37965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:02.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.681+0000 7fa1c1375700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddd3787820 0x55ddd3799860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:02.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.681+0000 7fa1c1b76700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddd378d900 0x55ddd37965b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:41762/0 (socket says 172.21.15.7:41762) 2024-06-10T01:13:02.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.681+0000 7fa1c1b76700 1 -- 172.21.15.7:0/1099280302 learned_addr learned my addr 172.21.15.7:0/1099280302 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:02.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.681+0000 7fa1c0b74700 1 --2- 172.21.15.7:0/1099280302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd378cfe0 0x55ddd3799da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:02.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.681+0000 7fa1c1b76700 1 -- 172.21.15.7:0/1099280302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddd3787820 msgr2=0x55ddd3799860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:02.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.681+0000 7fa1c1b76700 1 --2- 172.21.15.7:0/1099280302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddd3787820 0x55ddd3799860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.681+0000 7fa1c1b76700 1 -- 172.21.15.7:0/1099280302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd378cfe0 msgr2=0x55ddd3799da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:02.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.681+0000 7fa1c1b76700 1 --2- 172.21.15.7:0/1099280302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd378cfe0 0x55ddd3799da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.681+0000 7fa1c1b76700 1 -- 172.21.15.7:0/1099280302 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddd3796df0 con 0x55ddd378d900 2024-06-10T01:13:02.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.681+0000 7fa1c1b76700 1 --2- 172.21.15.7:0/1099280302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddd378d900 0x55ddd37965b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4009550 tx=0x7fa1b401e5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fed2b2697a4d745 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:02.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.681+0000 7fa1a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:13:02.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.681+0000 7fa1a4ff9700 1 -- 172.21.15.7:0/1099280302 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b401a050 con 0x55ddd378d900 2024-06-10T01:13:02.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.682+0000 7fa1cde5cb80 1 -- 172.21.15.7:0/1099280302 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ddd36e5dd0 con 0x55ddd378d900 2024-06-10T01:13:02.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.682+0000 7fa1a4ff9700 1 -- 172.21.15.7:0/1099280302 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa1b4004a00 con 0x55ddd378d900 2024-06-10T01:13:02.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.682+0000 7fa1a4ff9700 1 -- 172.21.15.7:0/1099280302 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa1b4038050 con 0x55ddd378d900 2024-06-10T01:13:02.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.682+0000 7fa1cde5cb80 10 client.26675 fetch_fsmap learned FSMap version 33 2024-06-10T01:13:02.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.682+0000 7fa1cde5cb80 1 -- 172.21.15.7:0/1099280302 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ddd380cc00 con 0x55ddd378d900 2024-06-10T01:13:02.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.682+0000 7fa1a4ff9700 1 -- 172.21.15.7:0/1099280302 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ddd36e5dd0 con 0x55ddd378d900 2024-06-10T01:13:02.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.682+0000 7fa1a4ff9700 1 -- 172.21.15.7:0/1099280302 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7fa1b4007aa0 con 0x55ddd378d900 2024-06-10T01:13:02.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.682+0000 7fa1cde5cb80 10 client.26675 fetch_fsmap finished waiting for FSMap version 33 2024-06-10T01:13:02.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.682+0000 7fa1cde5cb80 10 client.26675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:13:02.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.682+0000 7fa1cde5cb80 20 client.26675 populate_metadata read hostname 'smithi007' 2024-06-10T01:13:02.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.682+0000 7fa1cde5cb80 1 --2- 172.21.15.7:0/1099280302 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ddd380d360 0x55ddd380f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:02.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.682+0000 7fa1cde5cb80 4 client.26675 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:13:02.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.682+0000 7fa1cde5cb80 1 -- 172.21.15.7:0/1099280302 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ddd36e0350 con 0x55ddd380d360 2024-06-10T01:13:02.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.682+0000 7fa1c1375700 1 --2- 172.21.15.7:0/1099280302 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ddd380d360 0x55ddd380f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:02.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.683+0000 7fa1c1375700 1 --2- 172.21.15.7:0/1099280302 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ddd380d360 0x55ddd380f780 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50c93a97d3281167 server_cookie=88060f7eebb0cec8 in_seq=0 out_seq=0 2024-06-10T01:13:02.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.683+0000 7fa1a4ff9700 10 client.26675 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:02.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.684+0000 7fa1a4ff9700 1 -- 172.21.15.7:0/1099280302 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ddd36e0350 con 0x55ddd380d360 2024-06-10T01:13:02.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.684+0000 7fa1a4ff9700 10 client.26675 handle_command_reply: tid=0 2024-06-10T01:13:02.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.684+0000 7fa1cde5cb80 1 client.26675 shutdown 2024-06-10T01:13:02.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.684+0000 7fa1cde5cb80 1 -- 172.21.15.7:0/1099280302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddd378d900 msgr2=0x55ddd37965b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:02.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.684+0000 7fa1cde5cb80 1 --2- 172.21.15.7:0/1099280302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddd378d900 0x55ddd37965b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4009550 tx=0x7fa1b401e5e0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.684+0000 7fa1cde5cb80 1 -- 172.21.15.7:0/1099280302 shutdown_connections 2024-06-10T01:13:02.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.685+0000 7fa1cde5cb80 1 --2- 172.21.15.7:0/1099280302 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ddd380d360 0x55ddd380f780 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.685+0000 7fa1cde5cb80 1 --2- 172.21.15.7:0/1099280302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddd378d900 0x55ddd37965b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.685+0000 7fa1cde5cb80 1 --2- 172.21.15.7:0/1099280302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddd3787820 0x55ddd3799860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.685+0000 7fa1cde5cb80 1 --2- 172.21.15.7:0/1099280302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd378cfe0 0x55ddd3799da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.685+0000 7fa1a4ff9700 0 client.26675 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:02.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.685+0000 7fa1cde5cb80 1 -- 172.21.15.7:0/1099280302 >> 172.21.15.7:0/1099280302 conn(0x55ddd36b4100 msgr2=0x55ddd3792200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:02.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.685+0000 7fa1cde5cb80 1 -- 172.21.15.7:0/1099280302 shutdown_connections 2024-06-10T01:13:02.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.685+0000 7fa1cde5cb80 1 -- 172.21.15.7:0/1099280302 wait complete. 2024-06-10T01:13:02.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.685+0000 7fa1cde5cb80 20 client.26675 trim_cache size 0 max 16384 2024-06-10T01:13:02.687 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:13:02.687 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-10T01:13:02.687 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "2ca5eb0c-edf5-4d99-9709-398bbbf3d14a", 2024-06-10T01:13:02.687 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-10T01:13:02.688 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:13:02.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.685+0000 7fa1c35d9700 1 -- 172.21.15.7:0/1535695496 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa1a80a2c20 msgr2=0x7fa1a80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:02.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.685+0000 7fa1c35d9700 1 --2- 172.21.15.7:0/1535695496 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa1a80a2c20 0x7fa1a80a50e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8005950 tx=0x7fa1b8009310 comp rx=0 tx=0).stop 2024-06-10T01:13:02.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.685+0000 7fa1c35d9700 1 -- 172.21.15.7:0/1535695496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1bc1033a0 msgr2=0x7fa1bc19b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:02.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.685+0000 7fa1c35d9700 1 --2- 172.21.15.7:0/1535695496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1bc1033a0 0x7fa1bc19b3f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00c010 tx=0x7fa1ac00fb40 comp rx=0 tx=0).stop 2024-06-10T01:13:02.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.685+0000 7fa1c35d9700 1 -- 172.21.15.7:0/1535695496 shutdown_connections 2024-06-10T01:13:02.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.686+0000 7fa1c35d9700 1 --2- 172.21.15.7:0/1535695496 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa1a80a2c20 0x7fa1a80a50e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.686+0000 7fa1c35d9700 1 --2- 172.21.15.7:0/1535695496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1bc10ede0 0x7fa1bc19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.686+0000 7fa1c35d9700 1 --2- 172.21.15.7:0/1535695496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1bc06b0a0 0x7fa1bc19aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.686+0000 7fa1c35d9700 1 --2- 172.21.15.7:0/1535695496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1bc1033a0 0x7fa1bc19b3f0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:02.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.686+0000 7fa1c35d9700 1 -- 172.21.15.7:0/1535695496 >> 172.21.15.7:0/1535695496 conn(0x7fa1bc078340 msgr2=0x7fa1bc100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:02.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.686+0000 7fa1c35d9700 1 -- 172.21.15.7:0/1535695496 shutdown_connections 2024-06-10T01:13:02.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:02.686+0000 7fa1c35d9700 1 -- 172.21.15.7:0/1535695496 wait complete. 2024-06-10T01:13:02.736 INFO:tasks.workunit.client.0.smithi007.stdout: 1 468973 7.20 MB/sec execute 223 sec latency 62.953 ms 2024-06-10T01:13:03.437 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:13:03.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:03 smithi078 ceph-mon[36683]: pgmap v643: 129 pgs: 129 active+clean; 87 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 345 op/s 2024-06-10T01:13:03.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:03 smithi078 ceph-mon[36683]: scrub queued for path: / 2024-06-10T01:13:03.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:03 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:13:03.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:03 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:13:03.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:03 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:13:03.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:03 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:13:03.580 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:03 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:13:03.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:03 smithi175 ceph-mon[36636]: pgmap v643: 129 pgs: 129 active+clean; 87 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 345 op/s 2024-06-10T01:13:03.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:03 smithi175 ceph-mon[36636]: scrub queued for path: / 2024-06-10T01:13:03.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:03 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:13:03.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:03 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:13:03.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:03 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:13:03.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:03 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:13:03.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:03 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:13:03.736 INFO:tasks.workunit.client.0.smithi007.stdout: 1 470855 7.20 MB/sec execute 224 sec latency 13.737 ms 2024-06-10T01:13:03.748 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:13:03.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:03 smithi007 ceph-mon[32398]: pgmap v643: 129 pgs: 129 active+clean; 87 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 345 op/s 2024-06-10T01:13:03.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:03 smithi007 ceph-mon[32398]: scrub queued for path: / 2024-06-10T01:13:03.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:03 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:13:03.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:03 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:13:03.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:03 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:13:03.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:03 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:13:03.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:03 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:13:04.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:04 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:13:04.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:04 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:13:04.579 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:04 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:13:04.579 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:04 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:13:04.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:04 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:13:04.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:04 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:13:04.736 INFO:tasks.workunit.client.0.smithi007.stdout: 1 473291 7.23 MB/sec execute 225 sec latency 17.305 ms 2024-06-10T01:13:05.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.566+0000 7fc199e50700 1 -- 172.21.15.7:0/4415788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1941073d0 msgr2=0x7fc1941077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:05.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.566+0000 7fc199e50700 1 --2- 172.21.15.7:0/4415788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1941073d0 0x7fc1941077b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc184009990 tx=0x7fc184009ca0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.566+0000 7fc199e50700 1 -- 172.21.15.7:0/4415788 shutdown_connections 2024-06-10T01:13:05.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.566+0000 7fc199e50700 1 --2- 172.21.15.7:0/4415788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc194101d30 0x7fc19410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.566+0000 7fc199e50700 1 --2- 172.21.15.7:0/4415788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1941073d0 0x7fc1941077b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.566+0000 7fc199e50700 1 --2- 172.21.15.7:0/4415788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc194101390 0x7fc1941017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.566+0000 7fc199e50700 1 -- 172.21.15.7:0/4415788 >> 172.21.15.7:0/4415788 conn(0x7fc1940782c0 msgr2=0x7fc1940786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:05.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.566+0000 7fc199e50700 1 -- 172.21.15.7:0/4415788 shutdown_connections 2024-06-10T01:13:05.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.566+0000 7fc199e50700 1 -- 172.21.15.7:0/4415788 wait complete. 2024-06-10T01:13:05.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.567+0000 7fc199e50700 1 Processor -- start 2024-06-10T01:13:05.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.567+0000 7fc199e50700 1 -- start start 2024-06-10T01:13:05.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.567+0000 7fc199e50700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc194101390 0x7fc19419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:05.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.567+0000 7fc199e50700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc194101d30 0x7fc19419ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:05.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.567+0000 7fc199e50700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1941073d0 0x7fc194198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:05.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.567+0000 7fc199e50700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc194111630 con 0x7fc194101d30 2024-06-10T01:13:05.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.567+0000 7fc199e50700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941114b0 con 0x7fc1941073d0 2024-06-10T01:13:05.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.567+0000 7fc199e50700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941117b0 con 0x7fc194101390 2024-06-10T01:13:05.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.568+0000 7fc192ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc194101d30 0x7fc19419ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:05.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.568+0000 7fc1937fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc194101390 0x7fc19419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:05.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.568+0000 7fc192ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc194101d30 0x7fc19419ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52962/0 (socket says 172.21.15.7:52962) 2024-06-10T01:13:05.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.568+0000 7fc192ffd700 1 -- 172.21.15.7:0/3013508659 learned_addr learned my addr 172.21.15.7:0/3013508659 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:05.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.568+0000 7fc193fff700 1 --2- 172.21.15.7:0/3013508659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1941073d0 0x7fc194198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:05.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.568+0000 7fc1937fe700 1 -- 172.21.15.7:0/3013508659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1941073d0 msgr2=0x7fc194198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:05.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.568+0000 7fc1937fe700 1 --2- 172.21.15.7:0/3013508659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1941073d0 0x7fc194198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.568+0000 7fc1937fe700 1 -- 172.21.15.7:0/3013508659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc194101d30 msgr2=0x7fc19419ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:05.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.568+0000 7fc1937fe700 1 --2- 172.21.15.7:0/3013508659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc194101d30 0x7fc19419ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.568+0000 7fc1937fe700 1 -- 172.21.15.7:0/3013508659 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc194199260 con 0x7fc194101390 2024-06-10T01:13:05.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.568+0000 7fc193fff700 1 --2- 172.21.15.7:0/3013508659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1941073d0 0x7fc194198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:05.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.568+0000 7fc192ffd700 1 --2- 172.21.15.7:0/3013508659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc194101d30 0x7fc19419ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:05.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.568+0000 7fc1937fe700 1 --2- 172.21.15.7:0/3013508659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc194101390 0x7fc19419e940 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc1840059d0 tx=0x7fc18400bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:05.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.569+0000 7fc190ff9700 1 -- 172.21.15.7:0/3013508659 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184027070 con 0x7fc194101390 2024-06-10T01:13:05.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.569+0000 7fc199e50700 1 -- 172.21.15.7:0/3013508659 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1941994f0 con 0x7fc194101390 2024-06-10T01:13:05.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.569+0000 7fc190ff9700 1 -- 172.21.15.7:0/3013508659 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc184003d70 con 0x7fc194101390 2024-06-10T01:13:05.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.569+0000 7fc190ff9700 1 -- 172.21.15.7:0/3013508659 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184022e30 con 0x7fc194101390 2024-06-10T01:13:05.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.569+0000 7fc199e50700 1 -- 172.21.15.7:0/3013508659 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1941a5810 con 0x7fc194101390 2024-06-10T01:13:05.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.572+0000 7fc190ff9700 1 -- 172.21.15.7:0/3013508659 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fc18402a070 con 0x7fc194101390 2024-06-10T01:13:05.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.573+0000 7fc190ff9700 1 --2- 172.21.15.7:0/3013508659 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc17c0a2c20 0x7fc17c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:05.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.573+0000 7fc190ff9700 1 -- 172.21.15.7:0/3013508659 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc1840d0110 con 0x7fc194101390 2024-06-10T01:13:05.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.573+0000 7fc192ffd700 1 --2- 172.21.15.7:0/3013508659 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc17c0a2c20 0x7fc17c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:05.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.573+0000 7fc1a46d3b80 1 Processor -- start 2024-06-10T01:13:05.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.573+0000 7fc192ffd700 1 --2- 172.21.15.7:0/3013508659 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc17c0a2c20 0x7fc17c0a50e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc19406b250 tx=0x7fc188009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:05.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.573+0000 7fc1a46d3b80 1 -- start start 2024-06-10T01:13:05.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.573+0000 7fc1a46d3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b125d3bf60 0x55b125d3c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:05.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.574+0000 7fc1a46d3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b125d3c880 0x55b125d36260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:05.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.574+0000 7fc1a46d3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b125d367a0 0x55b125d36bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:05.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.574+0000 7fc1a46d3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b125c28f60 con 0x55b125d3bf60 2024-06-10T01:13:05.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.574+0000 7fc1a46d3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b125c87850 con 0x55b125d3c880 2024-06-10T01:13:05.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.574+0000 7fc1a46d3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b125bef460 con 0x55b125d367a0 2024-06-10T01:13:05.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.574+0000 7fc192ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b125d3c880 0x55b125d36260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:05.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.574+0000 7fc1937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b125d3bf60 0x55b125d3c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:05.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.574+0000 7fc192ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b125d3c880 0x55b125d36260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:41796/0 (socket says 172.21.15.7:41796) 2024-06-10T01:13:05.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.574+0000 7fc192ffd700 1 -- 172.21.15.7:0/971520331 learned_addr learned my addr 172.21.15.7:0/971520331 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:05.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.574+0000 7fc193fff700 1 --2- 172.21.15.7:0/971520331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b125d367a0 0x55b125d36bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.574+0000 7fc192ffd700 1 -- 172.21.15.7:0/971520331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b125d367a0 msgr2=0x55b125d36bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.574+0000 7fc192ffd700 1 --2- 172.21.15.7:0/971520331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b125d367a0 0x55b125d36bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.574+0000 7fc192ffd700 1 -- 172.21.15.7:0/971520331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b125d3bf60 msgr2=0x55b125d3c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.574+0000 7fc192ffd700 1 --2- 172.21.15.7:0/971520331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b125d3bf60 0x55b125d3c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.574+0000 7fc192ffd700 1 -- 172.21.15.7:0/971520331 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b125bd68d0 con 0x55b125d3c880 2024-06-10T01:13:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.574+0000 7fc193fff700 1 --2- 172.21.15.7:0/971520331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b125d367a0 0x55b125d36bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:13:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.574+0000 7fc192ffd700 1 --2- 172.21.15.7:0/971520331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b125d3c880 0x55b125d36260 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc18800f6f0 tx=0x7fc188014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=feedafcc51ae4e3e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.574+0000 7fc179ffb700 1 -- 172.21.15.7:0/971520331 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188026630 con 0x55b125d3c880 2024-06-10T01:13:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.575+0000 7fc179ffb700 1 -- 172.21.15.7:0/971520331 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc18800fe90 con 0x55b125d3c880 2024-06-10T01:13:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.575+0000 7fc179ffb700 1 -- 172.21.15.7:0/971520331 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18801d990 con 0x55b125d3c880 2024-06-10T01:13:05.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.575+0000 7fc1a46d3b80 1 -- 172.21.15.7:0/971520331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b125d3c880 msgr2=0x55b125d36260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:05.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.575+0000 7fc1a46d3b80 1 --2- 172.21.15.7:0/971520331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b125d3c880 0x55b125d36260 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc18800f6f0 tx=0x7fc188014000 comp rx=0 tx=0).stop 2024-06-10T01:13:05.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.575+0000 7fc1a46d3b80 1 -- 172.21.15.7:0/971520331 shutdown_connections 2024-06-10T01:13:05.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.575+0000 7fc1a46d3b80 1 --2- 172.21.15.7:0/971520331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b125d367a0 0x55b125d36bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.575+0000 7fc1a46d3b80 1 --2- 172.21.15.7:0/971520331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b125d3bf60 0x55b125d3c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.575+0000 7fc1a46d3b80 1 --2- 172.21.15.7:0/971520331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b125d3c880 0x55b125d36260 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.575+0000 7fc1a46d3b80 1 -- 172.21.15.7:0/971520331 >> 172.21.15.7:0/971520331 conn(0x55b125c63080 msgr2=0x55b125d31220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:05.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.575+0000 7fc1a46d3b80 1 -- 172.21.15.7:0/971520331 shutdown_connections 2024-06-10T01:13:05.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.575+0000 7fc1a46d3b80 1 -- 172.21.15.7:0/971520331 wait complete. 2024-06-10T01:13:05.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.576+0000 7fc1a46d3b80 1 -- start start 2024-06-10T01:13:05.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.576+0000 7fc1a46d3b80 1 Processor -- start 2024-06-10T01:13:05.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.576+0000 7fc1a46d3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b125d367a0 0x55b125d42630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:05.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.576+0000 7fc1a46d3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b125d3bf60 0x55b125d3f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:05.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.576+0000 7fc1a46d3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b125d3c880 0x55b125d3fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:05.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.576+0000 7fc1a46d3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b125c27fc0 con 0x55b125d367a0 2024-06-10T01:13:05.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.576+0000 7fc1a46d3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b125c83810 con 0x55b125d3bf60 2024-06-10T01:13:05.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.576+0000 7fc1a46d3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b125d43260 con 0x55b125d3c880 2024-06-10T01:13:05.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.576+0000 7fc1a46d3b80 10 client.0 _ll_register_callbacks cb 0x55b125c1d5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:13:05.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.576+0000 7fc1937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b125d367a0 0x55b125d42630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:05.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.576+0000 7fc1937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b125d367a0 0x55b125d42630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52974/0 (socket says 172.21.15.7:52974) 2024-06-10T01:13:05.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.577+0000 7fc1937fe700 1 -- 172.21.15.7:0/2377813218 learned_addr learned my addr 172.21.15.7:0/2377813218 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:05.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.577+0000 7fc193fff700 1 --2- 172.21.15.7:0/2377813218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b125d3c880 0x55b125d3fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:05.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.577+0000 7fc192ffd700 1 --2- 172.21.15.7:0/2377813218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b125d3bf60 0x55b125d3f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:05.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.577+0000 7fc1937fe700 1 -- 172.21.15.7:0/2377813218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b125d3c880 msgr2=0x55b125d3fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:05.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.577+0000 7fc1937fe700 1 --2- 172.21.15.7:0/2377813218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b125d3c880 0x55b125d3fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.577+0000 7fc1937fe700 1 -- 172.21.15.7:0/2377813218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b125d3bf60 msgr2=0x55b125d3f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:05.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.577+0000 7fc1937fe700 1 --2- 172.21.15.7:0/2377813218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b125d3bf60 0x55b125d3f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.577+0000 7fc1937fe700 1 -- 172.21.15.7:0/2377813218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b125c4d750 con 0x55b125d367a0 2024-06-10T01:13:05.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.577+0000 7fc1937fe700 1 --2- 172.21.15.7:0/2377813218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b125d367a0 0x55b125d42630 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fc184000c00 tx=0x7fc184009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc71bd5fcccbfe4d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:05.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.577+0000 7fc172ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:13:05.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.577+0000 7fc172ffd700 1 -- 172.21.15.7:0/2377813218 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18409bfe0 con 0x55b125d367a0 2024-06-10T01:13:05.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.577+0000 7fc172ffd700 1 -- 172.21.15.7:0/2377813218 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc1840a1bc0 con 0x55b125d367a0 2024-06-10T01:13:05.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.577+0000 7fc1a46d3b80 1 -- 172.21.15.7:0/2377813218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc18800fe90 con 0x55b125d367a0 2024-06-10T01:13:05.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.577+0000 7fc172ffd700 1 -- 172.21.15.7:0/2377813218 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc1840095d0 con 0x55b125d367a0 2024-06-10T01:13:05.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.577+0000 7fc1a46d3b80 10 client.17274 fetch_fsmap learned FSMap version 33 2024-06-10T01:13:05.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.577+0000 7fc1a46d3b80 1 -- 172.21.15.7:0/2377813218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b125d4d9d0 con 0x55b125d367a0 2024-06-10T01:13:05.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.578+0000 7fc172ffd700 1 -- 172.21.15.7:0/2377813218 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc18800fe90 con 0x55b125d367a0 2024-06-10T01:13:05.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.578+0000 7fc172ffd700 1 -- 172.21.15.7:0/2377813218 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7fc18409bad0 con 0x55b125d367a0 2024-06-10T01:13:05.578 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:05 smithi007 ceph-mon[32398]: pgmap v644: 129 pgs: 129 active+clean; 81 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 363 op/s 2024-06-10T01:13:05.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.578+0000 7fc1a46d3b80 10 client.17274 fetch_fsmap finished waiting for FSMap version 33 2024-06-10T01:13:05.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.578+0000 7fc1a46d3b80 10 client.17274 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:13:05.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.578+0000 7fc1a46d3b80 20 client.17274 populate_metadata read hostname 'smithi007' 2024-06-10T01:13:05.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.578+0000 7fc1a46d3b80 1 --2- 172.21.15.7:0/2377813218 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b125d4ff40 0x55b125d52330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:05.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.578+0000 7fc1a46d3b80 4 client.17274 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:13:05.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.578+0000 7fc1a46d3b80 1 -- 172.21.15.7:0/2377813218 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b125c94d50 con 0x55b125d4ff40 2024-06-10T01:13:05.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.578+0000 7fc192ffd700 1 --2- 172.21.15.7:0/2377813218 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b125d4ff40 0x55b125d52330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:05.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.579+0000 7fc192ffd700 1 --2- 172.21.15.7:0/2377813218 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b125d4ff40 0x55b125d52330 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6027d562adb6c0d9 server_cookie=8ca00f958e405d81 in_seq=0 out_seq=0 2024-06-10T01:13:05.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.579+0000 7fc172ffd700 10 client.17274 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.579+0000 7fc172ffd700 1 -- 172.21.15.7:0/2377813218 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b125c94d50 con 0x55b125d4ff40 2024-06-10T01:13:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.580+0000 7fc172ffd700 10 client.17274 handle_command_reply: tid=0 2024-06-10T01:13:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.580+0000 7fc1a46d3b80 1 client.17274 shutdown 2024-06-10T01:13:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.580+0000 7fc1a46d3b80 1 -- 172.21.15.7:0/2377813218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b125d367a0 msgr2=0x55b125d42630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.580+0000 7fc1a46d3b80 1 --2- 172.21.15.7:0/2377813218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b125d367a0 0x55b125d42630 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fc184000c00 tx=0x7fc184009310 comp rx=0 tx=0).stop 2024-06-10T01:13:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.580+0000 7fc1a46d3b80 1 -- 172.21.15.7:0/2377813218 shutdown_connections 2024-06-10T01:13:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.580+0000 7fc1a46d3b80 1 --2- 172.21.15.7:0/2377813218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b125d3c880 0x55b125d3fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.580+0000 7fc1a46d3b80 1 --2- 172.21.15.7:0/2377813218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b125d367a0 0x55b125d42630 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.580+0000 7fc1a46d3b80 1 --2- 172.21.15.7:0/2377813218 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b125d4ff40 0x55b125d52330 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.580+0000 7fc1a46d3b80 1 --2- 172.21.15.7:0/2377813218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b125d3bf60 0x55b125d3f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.580+0000 7fc1a46d3b80 1 -- 172.21.15.7:0/2377813218 >> 172.21.15.7:0/2377813218 conn(0x55b125c63080 msgr2=0x55b125d30700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.580+0000 7fc172ffd700 0 client.17274 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.580+0000 7fc1a46d3b80 1 -- 172.21.15.7:0/2377813218 shutdown_connections 2024-06-10T01:13:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.580+0000 7fc1a46d3b80 1 -- 172.21.15.7:0/2377813218 wait complete. 2024-06-10T01:13:05.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.580+0000 7fc1a46d3b80 20 client.17274 trim_cache size 0 max 16384 2024-06-10T01:13:05.581 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:05 smithi078 ceph-mon[36683]: pgmap v644: 129 pgs: 129 active+clean; 81 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 363 op/s 2024-06-10T01:13:05.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.607+0000 7fc1a46d3b80 1 Processor -- start 2024-06-10T01:13:05.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.607+0000 7fc1a46d3b80 1 -- start start 2024-06-10T01:13:05.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.607+0000 7fc1a46d3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b125d367a0 0x55b125d3d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:05.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.607+0000 7fc1a46d3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b125d3bf60 0x55b125d41870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:05.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.607+0000 7fc1a46d3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b125d3c880 0x55b125d30e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:05.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.607+0000 7fc1a46d3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941ae840 con 0x55b125d367a0 2024-06-10T01:13:05.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.607+0000 7fc1a46d3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941ac0d0 con 0x55b125d3c880 2024-06-10T01:13:05.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.607+0000 7fc1a46d3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941a9f20 con 0x55b125d3bf60 2024-06-10T01:13:05.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.607+0000 7fc1937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b125d367a0 0x55b125d3d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:05.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.607+0000 7fc1937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b125d367a0 0x55b125d3d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52978/0 (socket says 172.21.15.7:52978) 2024-06-10T01:13:05.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.607+0000 7fc1937fe700 1 -- 172.21.15.7:0/3616528890 learned_addr learned my addr 172.21.15.7:0/3616528890 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:05.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.608+0000 7fc193fff700 1 --2- 172.21.15.7:0/3616528890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b125d3c880 0x55b125d30e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:05.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.608+0000 7fc192ffd700 1 --2- 172.21.15.7:0/3616528890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b125d3bf60 0x55b125d41870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.608+0000 7fc1937fe700 1 -- 172.21.15.7:0/3616528890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b125d3bf60 msgr2=0x55b125d41870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.608+0000 7fc1937fe700 1 --2- 172.21.15.7:0/3616528890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b125d3bf60 0x55b125d41870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.608+0000 7fc1937fe700 1 -- 172.21.15.7:0/3616528890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b125d3c880 msgr2=0x55b125d30e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.608+0000 7fc1937fe700 1 --2- 172.21.15.7:0/3616528890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b125d3c880 0x55b125d30e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.608+0000 7fc1937fe700 1 -- 172.21.15.7:0/3616528890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b125d370a0 con 0x55b125d367a0 2024-06-10T01:13:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.608+0000 7fc193fff700 1 --2- 172.21.15.7:0/3616528890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b125d3c880 0x55b125d30e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.608+0000 7fc1937fe700 1 --2- 172.21.15.7:0/3616528890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b125d367a0 0x55b125d3d530 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fc18409c610 tx=0x7fc18409cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d13330e3aad3f356 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.608+0000 7fc1797fa700 1 -- 172.21.15.7:0/3616528890 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1840095d0 con 0x55b125d367a0 2024-06-10T01:13:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.608+0000 7fc1797fa700 1 -- 172.21.15.7:0/3616528890 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc18409cd80 con 0x55b125d367a0 2024-06-10T01:13:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.608+0000 7fc1797fa700 1 -- 172.21.15.7:0/3616528890 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18409bad0 con 0x55b125d367a0 2024-06-10T01:13:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.609+0000 7fc1a46d3b80 1 -- 172.21.15.7:0/3616528890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b125d367a0 msgr2=0x55b125d3d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.609+0000 7fc1a46d3b80 1 --2- 172.21.15.7:0/3616528890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b125d367a0 0x55b125d3d530 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fc18409c610 tx=0x7fc18409cca0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.609+0000 7fc1a46d3b80 1 -- 172.21.15.7:0/3616528890 shutdown_connections 2024-06-10T01:13:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.609+0000 7fc1a46d3b80 1 --2- 172.21.15.7:0/3616528890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b125d3c880 0x55b125d30e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.609+0000 7fc1a46d3b80 1 --2- 172.21.15.7:0/3616528890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b125d367a0 0x55b125d3d530 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.609+0000 7fc1a46d3b80 1 --2- 172.21.15.7:0/3616528890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b125d3bf60 0x55b125d41870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.609+0000 7fc1a46d3b80 1 -- 172.21.15.7:0/3616528890 >> 172.21.15.7:0/3616528890 conn(0x55b125c63080 msgr2=0x55b125d41160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.609+0000 7fc1a46d3b80 1 -- 172.21.15.7:0/3616528890 shutdown_connections 2024-06-10T01:13:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.609+0000 7fc1a46d3b80 1 -- 172.21.15.7:0/3616528890 wait complete. 2024-06-10T01:13:05.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.610+0000 7fc1a46d3b80 1 -- start start 2024-06-10T01:13:05.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.610+0000 7fc1a46d3b80 1 Processor -- start 2024-06-10T01:13:05.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.610+0000 7fc1a46d3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b125d367a0 0x55b125d48dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:05.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.610+0000 7fc1a46d3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b125d3bf60 0x55b125d57640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:05.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.610+0000 7fc1a46d3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b125d3c880 0x55b125d57c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:05.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.610+0000 7fc1a46d3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941aa590 con 0x55b125d3c880 2024-06-10T01:13:05.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.610+0000 7fc1a46d3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941aba00 con 0x55b125d367a0 2024-06-10T01:13:05.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.610+0000 7fc1a46d3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941aac60 con 0x55b125d3bf60 2024-06-10T01:13:05.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.610+0000 7fc1a46d3b80 10 client.0 _ll_register_callbacks cb 0x55b125c1d5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:13:05.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.610+0000 7fc193fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b125d3c880 0x55b125d57c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:05.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.610+0000 7fc192ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b125d3bf60 0x55b125d57640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:05.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.610+0000 7fc193fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b125d3c880 0x55b125d57c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52982/0 (socket says 172.21.15.7:52982) 2024-06-10T01:13:05.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.610+0000 7fc193fff700 1 -- 172.21.15.7:0/1240432447 learned_addr learned my addr 172.21.15.7:0/1240432447 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:05.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.610+0000 7fc1937fe700 1 --2- 172.21.15.7:0/1240432447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b125d367a0 0x55b125d48dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:05.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.611+0000 7fc192ffd700 1 -- 172.21.15.7:0/1240432447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b125d367a0 msgr2=0x55b125d48dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:05.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.611+0000 7fc192ffd700 1 --2- 172.21.15.7:0/1240432447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b125d367a0 0x55b125d48dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.611+0000 7fc192ffd700 1 -- 172.21.15.7:0/1240432447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b125d3c880 msgr2=0x55b125d57c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:05.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.611+0000 7fc192ffd700 1 --2- 172.21.15.7:0/1240432447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b125d3c880 0x55b125d57c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.611+0000 7fc192ffd700 1 -- 172.21.15.7:0/1240432447 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b125db9ba0 con 0x55b125d3bf60 2024-06-10T01:13:05.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.611+0000 7fc193fff700 1 --2- 172.21.15.7:0/1240432447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b125d3c880 0x55b125d57c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:05.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.611+0000 7fc192ffd700 1 --2- 172.21.15.7:0/1240432447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b125d3bf60 0x55b125d57640 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc18800fce0 tx=0x7fc18801d410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8cad6e9b69ea862 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:05.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.611+0000 7fc172ffd700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:13:05.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.611+0000 7fc172ffd700 1 -- 172.21.15.7:0/1240432447 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18803e060 con 0x55b125d3bf60 2024-06-10T01:13:05.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.611+0000 7fc172ffd700 1 -- 172.21.15.7:0/1240432447 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc18802b070 con 0x55b125d3bf60 2024-06-10T01:13:05.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.611+0000 7fc1a46d3b80 1 -- 172.21.15.7:0/1240432447 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc18409cd80 con 0x55b125d3bf60 2024-06-10T01:13:05.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.611+0000 7fc172ffd700 1 -- 172.21.15.7:0/1240432447 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc188045050 con 0x55b125d3bf60 2024-06-10T01:13:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.611+0000 7fc1a46d3b80 10 client.26710 fetch_fsmap learned FSMap version 33 2024-06-10T01:13:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.611+0000 7fc1a46d3b80 1 -- 172.21.15.7:0/1240432447 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b125dbacf0 con 0x55b125d3bf60 2024-06-10T01:13:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.612+0000 7fc172ffd700 1 -- 172.21.15.7:0/1240432447 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc18409cd80 con 0x55b125d3bf60 2024-06-10T01:13:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.612+0000 7fc172ffd700 1 -- 172.21.15.7:0/1240432447 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7fc1880049e0 con 0x55b125d3bf60 2024-06-10T01:13:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.612+0000 7fc1a46d3b80 10 client.26710 fetch_fsmap finished waiting for FSMap version 33 2024-06-10T01:13:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.612+0000 7fc1a46d3b80 10 client.26710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:13:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.612+0000 7fc1a46d3b80 20 client.26710 populate_metadata read hostname 'smithi007' 2024-06-10T01:13:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.612+0000 7fc1a46d3b80 1 --2- 172.21.15.7:0/1240432447 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b125dbb130 0x55b125dbd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.612+0000 7fc1a46d3b80 4 client.26710 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:13:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.612+0000 7fc1a46d3b80 1 -- 172.21.15.7:0/1240432447 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b125c8f2d0 con 0x55b125dbb130 2024-06-10T01:13:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.612+0000 7fc1937fe700 1 --2- 172.21.15.7:0/1240432447 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b125dbb130 0x55b125dbd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:05.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.613+0000 7fc1937fe700 1 --2- 172.21.15.7:0/1240432447 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b125dbb130 0x55b125dbd550 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88e5994b0fd208a5 server_cookie=6ec8db1cce30e744 in_seq=0 out_seq=0 2024-06-10T01:13:05.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.613+0000 7fc172ffd700 10 client.26710 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:05.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.613+0000 7fc172ffd700 1 -- 172.21.15.7:0/1240432447 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b125c8f2d0 con 0x55b125dbb130 2024-06-10T01:13:05.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.613+0000 7fc172ffd700 10 client.26710 handle_command_reply: tid=0 2024-06-10T01:13:05.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.613+0000 7fc1a46d3b80 1 client.26710 shutdown 2024-06-10T01:13:05.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.613+0000 7fc1a46d3b80 1 -- 172.21.15.7:0/1240432447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b125d3bf60 msgr2=0x55b125d57640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:05.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.613+0000 7fc1a46d3b80 1 --2- 172.21.15.7:0/1240432447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b125d3bf60 0x55b125d57640 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc18800fce0 tx=0x7fc18801d410 comp rx=0 tx=0).stop 2024-06-10T01:13:05.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.613+0000 7fc1a46d3b80 1 -- 172.21.15.7:0/1240432447 shutdown_connections 2024-06-10T01:13:05.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.613+0000 7fc1a46d3b80 1 --2- 172.21.15.7:0/1240432447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b125d3c880 0x55b125d57c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.613+0000 7fc1a46d3b80 1 --2- 172.21.15.7:0/1240432447 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b125dbb130 0x55b125dbd550 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.613+0000 7fc1a46d3b80 1 --2- 172.21.15.7:0/1240432447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b125d367a0 0x55b125d48dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.613+0000 7fc1a46d3b80 1 --2- 172.21.15.7:0/1240432447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b125d3bf60 0x55b125d57640 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.613+0000 7fc172ffd700 0 client.26710 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:05.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.613+0000 7fc1a46d3b80 1 -- 172.21.15.7:0/1240432447 >> 172.21.15.7:0/1240432447 conn(0x55b125c63080 msgr2=0x55b125d419f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:05.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.614+0000 7fc1a46d3b80 1 -- 172.21.15.7:0/1240432447 shutdown_connections 2024-06-10T01:13:05.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.614+0000 7fc1a46d3b80 1 -- 172.21.15.7:0/1240432447 wait complete. 2024-06-10T01:13:05.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.614+0000 7fc1a46d3b80 20 client.26710 trim_cache size 0 max 16384 2024-06-10T01:13:05.614 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:13:05.614 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T01:13:05.614 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-10T01:13:05.614 INFO:teuthology.orchestra.run.smithi007.stdout: "2ca5eb0c-edf5-4d99-9709-398bbbf3d14a": { 2024-06-10T01:13:05.615 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-10T01:13:05.615 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "2ca5eb0c-edf5-4d99-9709-398bbbf3d14a", 2024-06-10T01:13:05.615 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-10T01:13:05.615 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:13:05.615 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:13:05.615 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:13:05.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.614+0000 7fc199e50700 1 -- 172.21.15.7:0/3013508659 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc17c0a2c20 msgr2=0x7fc17c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:05.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.614+0000 7fc199e50700 1 --2- 172.21.15.7:0/3013508659 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc17c0a2c20 0x7fc17c0a50e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc19406b250 tx=0x7fc188009260 comp rx=0 tx=0).stop 2024-06-10T01:13:05.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.614+0000 7fc199e50700 1 -- 172.21.15.7:0/3013508659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc194101390 msgr2=0x7fc19419e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:05.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.614+0000 7fc199e50700 1 --2- 172.21.15.7:0/3013508659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc194101390 0x7fc19419e940 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc1840059d0 tx=0x7fc18400bab0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.614+0000 7fc199e50700 1 -- 172.21.15.7:0/3013508659 shutdown_connections 2024-06-10T01:13:05.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.614+0000 7fc199e50700 1 --2- 172.21.15.7:0/3013508659 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc17c0a2c20 0x7fc17c0a50e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.614+0000 7fc199e50700 1 --2- 172.21.15.7:0/3013508659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1941073d0 0x7fc194198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.614+0000 7fc199e50700 1 --2- 172.21.15.7:0/3013508659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc194101390 0x7fc19419e940 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.614+0000 7fc199e50700 1 --2- 172.21.15.7:0/3013508659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc194101d30 0x7fc19419ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:05.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.614+0000 7fc199e50700 1 -- 172.21.15.7:0/3013508659 >> 172.21.15.7:0/3013508659 conn(0x7fc1940782c0 msgr2=0x7fc194105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:05.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.615+0000 7fc199e50700 1 -- 172.21.15.7:0/3013508659 shutdown_connections 2024-06-10T01:13:05.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:05.615+0000 7fc199e50700 1 -- 172.21.15.7:0/3013508659 wait complete. 2024-06-10T01:13:05.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:05 smithi175 ceph-mon[36636]: pgmap v644: 129 pgs: 129 active+clean; 81 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 363 op/s 2024-06-10T01:13:05.736 INFO:tasks.workunit.client.0.smithi007.stdout: 1 475520 7.26 MB/sec execute 226 sec latency 68.679 ms 2024-06-10T01:13:06.491 INFO:tasks.cephfs.filesystem:scrub status for tag:2ca5eb0c-edf5-4d99-9709-398bbbf3d14a - {'path': '/', 'tag': '2ca5eb0c-edf5-4d99-9709-398bbbf3d14a', 'options': 'recursive,force'} 2024-06-10T01:13:06.736 INFO:tasks.workunit.client.0.smithi007.stdout: 1 477145 7.26 MB/sec execute 227 sec latency 67.685 ms 2024-06-10T01:13:06.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.753+0000 7fd63936a640 1 -- 172.21.15.7:0/2007611345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd634102500 msgr2=0x7fd6341048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:06.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.753+0000 7fd63936a640 1 --2- 172.21.15.7:0/2007611345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd634102500 0x7fd6341048f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd624009900 tx=0x7fd62402a410 comp rx=0 tx=0).stop 2024-06-10T01:13:06.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.754+0000 7fd63936a640 1 -- 172.21.15.7:0/2007611345 shutdown_connections 2024-06-10T01:13:06.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.754+0000 7fd63936a640 1 --2- 172.21.15.7:0/2007611345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd634107760 0x7fd634109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.754+0000 7fd63936a640 1 --2- 172.21.15.7:0/2007611345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd634104e30 0x7fd634107220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.754+0000 7fd63936a640 1 --2- 172.21.15.7:0/2007611345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd634102500 0x7fd6341048f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.754+0000 7fd63936a640 1 -- 172.21.15.7:0/2007611345 >> 172.21.15.7:0/2007611345 conn(0x7fd6340fbe70 msgr2=0x7fd6340fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:06.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.754+0000 7fd63936a640 1 -- 172.21.15.7:0/2007611345 shutdown_connections 2024-06-10T01:13:06.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.754+0000 7fd63936a640 1 -- 172.21.15.7:0/2007611345 wait complete. 2024-06-10T01:13:06.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.754+0000 7fd63936a640 1 Processor -- start 2024-06-10T01:13:06.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.754+0000 7fd63936a640 1 -- start start 2024-06-10T01:13:06.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.754+0000 7fd63936a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd634102500 0x7fd634199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:06.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.754+0000 7fd63936a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd634104e30 0x7fd63419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.754+0000 7fd63936a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd634107760 0x7fd6341a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.754+0000 7fd63936a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd63410d6a0 con 0x7fd634102500 2024-06-10T01:13:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.754+0000 7fd63936a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd63410d520 con 0x7fd634104e30 2024-06-10T01:13:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.754+0000 7fd63936a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd63410d820 con 0x7fd634107760 2024-06-10T01:13:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.754+0000 7fd632ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd634102500 0x7fd634199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.755+0000 7fd62bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd634104e30 0x7fd63419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.755+0000 7fd632ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd634102500 0x7fd634199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52994/0 (socket says 172.21.15.7:52994) 2024-06-10T01:13:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.755+0000 7fd632ffd640 1 -- 172.21.15.7:0/3966507904 learned_addr learned my addr 172.21.15.7:0/3966507904 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.755+0000 7fd6337fe640 1 --2- 172.21.15.7:0/3966507904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd634107760 0x7fd6341a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.755+0000 7fd632ffd640 1 -- 172.21.15.7:0/3966507904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd634107760 msgr2=0x7fd6341a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:06.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.755+0000 7fd632ffd640 1 --2- 172.21.15.7:0/3966507904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd634107760 0x7fd6341a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.755+0000 7fd632ffd640 1 -- 172.21.15.7:0/3966507904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd634104e30 msgr2=0x7fd63419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:06.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.755+0000 7fd632ffd640 1 --2- 172.21.15.7:0/3966507904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd634104e30 0x7fd63419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.755+0000 7fd632ffd640 1 -- 172.21.15.7:0/3966507904 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6341a1b70 con 0x7fd634102500 2024-06-10T01:13:06.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.755+0000 7fd62bfff640 1 --2- 172.21.15.7:0/3966507904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd634104e30 0x7fd63419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:06.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.755+0000 7fd632ffd640 1 --2- 172.21.15.7:0/3966507904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd634102500 0x7fd634199f50 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fd624009230 tx=0x7fd6240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:06.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.755+0000 7fd630ff9640 1 -- 172.21.15.7:0/3966507904 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62403f030 con 0x7fd634102500 2024-06-10T01:13:06.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.755+0000 7fd63936a640 1 -- 172.21.15.7:0/3966507904 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6341a1e00 con 0x7fd634102500 2024-06-10T01:13:06.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.755+0000 7fd630ff9640 1 -- 172.21.15.7:0/3966507904 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd624003ce0 con 0x7fd634102500 2024-06-10T01:13:06.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.755+0000 7fd630ff9640 1 -- 172.21.15.7:0/3966507904 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd624035070 con 0x7fd634102500 2024-06-10T01:13:06.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.755+0000 7fd63936a640 1 -- 172.21.15.7:0/3966507904 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6341a2260 con 0x7fd634102500 2024-06-10T01:13:06.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.758+0000 7fd630ff9640 1 -- 172.21.15.7:0/3966507904 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fd624042030 con 0x7fd634102500 2024-06-10T01:13:06.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.758+0000 7fd63c07f740 1 Processor -- start 2024-06-10T01:13:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.758+0000 7fd63c07f740 1 -- start start 2024-06-10T01:13:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.758+0000 7fd63c07f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924eef7010 0x55924eef7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.758+0000 7fd63c07f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924eef7a00 0x55924eefc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.758+0000 7fd63c07f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55924eefcb70 0x55924eeff000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.758+0000 7fd63c07f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55924ee5a4f0 con 0x55924eef7a00 2024-06-10T01:13:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.758+0000 7fd63c07f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55924edf8870 con 0x55924eefcb70 2024-06-10T01:13:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.758+0000 7fd63c07f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55924ee564b0 con 0x55924eef7010 2024-06-10T01:13:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.758+0000 7fd632ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924eef7a00 0x55924eefc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.758+0000 7fd632ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924eef7a00 0x55924eefc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52996/0 (socket says 172.21.15.7:52996) 2024-06-10T01:13:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.758+0000 7fd632ffd640 1 -- 172.21.15.7:0/1892145902 learned_addr learned my addr 172.21.15.7:0/1892145902 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.758+0000 7fd6337fe640 1 --2- 172.21.15.7:0/1892145902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924eef7010 0x55924eef7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.758+0000 7fd62bfff640 1 --2- 172.21.15.7:0/1892145902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55924eefcb70 0x55924eeff000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.759+0000 7fd6337fe640 1 -- 172.21.15.7:0/1892145902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55924eefcb70 msgr2=0x55924eeff000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.759+0000 7fd6337fe640 1 --2- 172.21.15.7:0/1892145902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55924eefcb70 0x55924eeff000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.759+0000 7fd6337fe640 1 -- 172.21.15.7:0/1892145902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924eef7a00 msgr2=0x55924eefc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.759+0000 7fd6337fe640 1 --2- 172.21.15.7:0/1892145902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924eef7a00 0x55924eefc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.759+0000 7fd6337fe640 1 -- 172.21.15.7:0/1892145902 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55924ee149b0 con 0x55924eef7010 2024-06-10T01:13:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.759+0000 7fd632ffd640 1 --2- 172.21.15.7:0/1892145902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924eef7a00 0x55924eefc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.759+0000 7fd630ff9640 1 --2- 172.21.15.7:0/3966507904 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd6200a2b80 0x7fd6200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.759+0000 7fd6337fe640 1 --2- 172.21.15.7:0/1892145902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924eef7010 0x55924eef7430 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd62c009f10 tx=0x7fd62c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb6bc95935348258 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.759+0000 7fd632ffd640 1 --2- 172.21.15.7:0/3966507904 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd6200a2b80 0x7fd6200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.759+0000 7fd630ff9640 1 -- 172.21.15.7:0/3966507904 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fd6240ef870 con 0x7fd634102500 2024-06-10T01:13:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.759+0000 7fd6297fa640 1 -- 172.21.15.7:0/1892145902 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c004d20 con 0x55924eef7010 2024-06-10T01:13:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.759+0000 7fd6297fa640 1 -- 172.21.15.7:0/1892145902 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd62c004030 con 0x55924eef7010 2024-06-10T01:13:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.759+0000 7fd6297fa640 1 -- 172.21.15.7:0/1892145902 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c005220 con 0x55924eef7010 2024-06-10T01:13:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.760+0000 7fd63c07f740 1 -- 172.21.15.7:0/1892145902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924eef7010 msgr2=0x55924eef7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.760+0000 7fd632ffd640 1 --2- 172.21.15.7:0/3966507904 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd6200a2b80 0x7fd6200a5040 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd624009a20 tx=0x7fd6240f5530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.760+0000 7fd63c07f740 1 --2- 172.21.15.7:0/1892145902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924eef7010 0x55924eef7430 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd62c009f10 tx=0x7fd62c011810 comp rx=0 tx=0).stop 2024-06-10T01:13:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.760+0000 7fd63c07f740 1 -- 172.21.15.7:0/1892145902 shutdown_connections 2024-06-10T01:13:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.760+0000 7fd63c07f740 1 --2- 172.21.15.7:0/1892145902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55924eefcb70 0x55924eeff000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.760+0000 7fd63c07f740 1 --2- 172.21.15.7:0/1892145902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924eef7a00 0x55924eefc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.760+0000 7fd63c07f740 1 --2- 172.21.15.7:0/1892145902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924eef7010 0x55924eef7430 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.760+0000 7fd63c07f740 1 -- 172.21.15.7:0/1892145902 >> 172.21.15.7:0/1892145902 conn(0x55924eef2680 msgr2=0x55924eef2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.760+0000 7fd63c07f740 1 -- 172.21.15.7:0/1892145902 shutdown_connections 2024-06-10T01:13:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.760+0000 7fd63c07f740 1 -- 172.21.15.7:0/1892145902 wait complete. 2024-06-10T01:13:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.760+0000 7fd63c07f740 1 -- start start 2024-06-10T01:13:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.760+0000 7fd63c07f740 1 Processor -- start 2024-06-10T01:13:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.761+0000 7fd63c07f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924eef7010 0x55924ef01250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:06.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.761+0000 7fd63c07f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924eef7a00 0x55924ef01790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:06.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.761+0000 7fd63c07f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55924eefcb70 0x55924ef087d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:06.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.761+0000 7fd63c07f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55924eeffb20 con 0x55924eef7010 2024-06-10T01:13:06.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.761+0000 7fd63c07f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55924ede3ae0 con 0x55924eefcb70 2024-06-10T01:13:06.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.761+0000 7fd63c07f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55924eeffca0 con 0x55924eef7a00 2024-06-10T01:13:06.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.761+0000 7fd63c07f740 10 client.0 _ll_register_callbacks cb 0x55924ee1c9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:13:06.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.761+0000 7fd6337fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924eef7010 0x55924ef01250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:06.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.761+0000 7fd6337fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924eef7010 0x55924ef01250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52998/0 (socket says 172.21.15.7:52998) 2024-06-10T01:13:06.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.761+0000 7fd62bfff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924eef7a00 0x55924ef01790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:06.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.761+0000 7fd6337fe640 1 -- 172.21.15.7:0/1356102673 learned_addr learned my addr 172.21.15.7:0/1356102673 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:06.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.761+0000 7fd6337fe640 1 --2- 172.21.15.7:0/1356102673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55924eefcb70 0x55924ef087d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:06.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.761+0000 7fd62bfff640 1 -- 172.21.15.7:0/1356102673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55924eefcb70 msgr2=0x55924ef087d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:06.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.761+0000 7fd62bfff640 1 --2- 172.21.15.7:0/1356102673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55924eefcb70 0x55924ef087d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.761+0000 7fd62bfff640 1 -- 172.21.15.7:0/1356102673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924eef7010 msgr2=0x55924ef01250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.761+0000 7fd62bfff640 1 --2- 172.21.15.7:0/1356102673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924eef7010 0x55924ef01250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.761+0000 7fd62bfff640 1 -- 172.21.15.7:0/1356102673 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55924eb6e210 con 0x55924eef7a00 2024-06-10T01:13:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.761+0000 7fd62bfff640 1 --2- 172.21.15.7:0/1356102673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924eef7a00 0x55924ef01790 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd63419b460 tx=0x7fd61c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=289f8b9fe291e938 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.761+0000 7fd61a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:13:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.761+0000 7fd61a7fc640 1 -- 172.21.15.7:0/1356102673 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c009e30 con 0x55924eef7a00 2024-06-10T01:13:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.762+0000 7fd61a7fc640 1 -- 172.21.15.7:0/1356102673 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fd61c012110 con 0x55924eef7a00 2024-06-10T01:13:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.762+0000 7fd63c07f740 1 -- 172.21.15.7:0/1356102673 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd62c004030 con 0x55924eef7a00 2024-06-10T01:13:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.762+0000 7fd61a7fc640 1 -- 172.21.15.7:0/1356102673 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd61c025050 con 0x55924eef7a00 2024-06-10T01:13:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.762+0000 7fd63c07f740 10 client.26722 fetch_fsmap learned FSMap version 33 2024-06-10T01:13:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.762+0000 7fd63c07f740 1 -- 172.21.15.7:0/1356102673 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55924ec3a620 con 0x55924eef7a00 2024-06-10T01:13:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.762+0000 7fd61a7fc640 1 -- 172.21.15.7:0/1356102673 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd62c004030 con 0x55924eef7a00 2024-06-10T01:13:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.762+0000 7fd61a7fc640 1 -- 172.21.15.7:0/1356102673 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7fd61c003c70 con 0x55924eef7a00 2024-06-10T01:13:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.762+0000 7fd63c07f740 10 client.26722 fetch_fsmap finished waiting for FSMap version 33 2024-06-10T01:13:06.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.762+0000 7fd63c07f740 10 client.26722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:13:06.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.762+0000 7fd63c07f740 20 client.26722 populate_metadata read hostname 'smithi007' 2024-06-10T01:13:06.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.762+0000 7fd63c07f740 1 --2- 172.21.15.7:0/1356102673 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55924ef0d110 0x55924ef0f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:06.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.762+0000 7fd63c07f740 4 client.26722 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:13:06.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.762+0000 7fd63c07f740 1 -- 172.21.15.7:0/1356102673 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55924ee6df10 con 0x55924ef0d110 2024-06-10T01:13:06.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.762+0000 7fd6337fe640 1 --2- 172.21.15.7:0/1356102673 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55924ef0d110 0x55924ef0f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:06.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.763+0000 7fd61a7fc640 10 client.26722 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:06.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.763+0000 7fd6337fe640 1 --2- 172.21.15.7:0/1356102673 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55924ef0d110 0x55924ef0f580 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6777e60843631ae server_cookie=a0b0069a8645aa2d in_seq=0 out_seq=0 2024-06-10T01:13:06.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.763+0000 7fd61a7fc640 1 -- 172.21.15.7:0/1356102673 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55924ee6df10 con 0x55924ef0d110 2024-06-10T01:13:06.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.763+0000 7fd61a7fc640 10 client.26722 handle_command_reply: tid=0 2024-06-10T01:13:06.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.763+0000 7fd63c07f740 1 client.26722 shutdown 2024-06-10T01:13:06.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.764+0000 7fd63c07f740 1 -- 172.21.15.7:0/1356102673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924eef7a00 msgr2=0x55924ef01790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:06.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.764+0000 7fd63c07f740 1 --2- 172.21.15.7:0/1356102673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924eef7a00 0x55924ef01790 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd63419b460 tx=0x7fd61c00f790 comp rx=0 tx=0).stop 2024-06-10T01:13:06.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.764+0000 7fd63c07f740 1 -- 172.21.15.7:0/1356102673 shutdown_connections 2024-06-10T01:13:06.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.764+0000 7fd63c07f740 1 --2- 172.21.15.7:0/1356102673 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55924ef0d110 0x55924ef0f580 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.764+0000 7fd63c07f740 1 --2- 172.21.15.7:0/1356102673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55924eefcb70 0x55924ef087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.764+0000 7fd61a7fc640 0 client.26722 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:06.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.764+0000 7fd63c07f740 1 --2- 172.21.15.7:0/1356102673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924eef7a00 0x55924ef01790 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.764+0000 7fd63c07f740 1 --2- 172.21.15.7:0/1356102673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924eef7010 0x55924ef01250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.764+0000 7fd63c07f740 1 -- 172.21.15.7:0/1356102673 >> 172.21.15.7:0/1356102673 conn(0x55924eef2680 msgr2=0x55924eef17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:06.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.764+0000 7fd63c07f740 1 -- 172.21.15.7:0/1356102673 shutdown_connections 2024-06-10T01:13:06.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.764+0000 7fd63c07f740 1 -- 172.21.15.7:0/1356102673 wait complete. 2024-06-10T01:13:06.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.764+0000 7fd63c07f740 20 client.26722 trim_cache size 0 max 16384 2024-06-10T01:13:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.780+0000 7fd63c07f740 1 Processor -- start 2024-06-10T01:13:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.780+0000 7fd63c07f740 1 -- start start 2024-06-10T01:13:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.780+0000 7fd63c07f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55924eef7010 0x55924ef01250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.780+0000 7fd63c07f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924eef7a00 0x55924ef01790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.780+0000 7fd63c07f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924eefcb70 0x55924ef08790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.780+0000 7fd63c07f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd6341af0d0 con 0x55924eef7a00 2024-06-10T01:13:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.780+0000 7fd63c07f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd6341ac9a0 con 0x55924eef7010 2024-06-10T01:13:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.780+0000 7fd63c07f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd6341aa7f0 con 0x55924eefcb70 2024-06-10T01:13:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.780+0000 7fd62bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924eef7a00 0x55924ef01790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.780+0000 7fd62bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924eef7a00 0x55924ef01790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53012/0 (socket says 172.21.15.7:53012) 2024-06-10T01:13:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.780+0000 7fd6337fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55924eef7010 0x55924ef01250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.780+0000 7fd62bfff640 1 -- 172.21.15.7:0/3399974015 learned_addr learned my addr 172.21.15.7:0/3399974015 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.780+0000 7fd6337fe640 1 --2- 172.21.15.7:0/3399974015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924eefcb70 0x55924ef08790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.781+0000 7fd62bfff640 1 -- 172.21.15.7:0/3399974015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924eefcb70 msgr2=0x55924ef08790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.781+0000 7fd62bfff640 1 --2- 172.21.15.7:0/3399974015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924eefcb70 0x55924ef08790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.781+0000 7fd62bfff640 1 -- 172.21.15.7:0/3399974015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55924eef7010 msgr2=0x55924ef01250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.781+0000 7fd62bfff640 1 --2- 172.21.15.7:0/3399974015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55924eef7010 0x55924ef01250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.781+0000 7fd62bfff640 1 -- 172.21.15.7:0/3399974015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55924eef24b0 con 0x55924eef7a00 2024-06-10T01:13:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.781+0000 7fd6337fe640 1 --2- 172.21.15.7:0/3399974015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55924eef7010 0x55924ef01250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.781+0000 7fd62bfff640 1 --2- 172.21.15.7:0/3399974015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924eef7a00 0x55924ef01790 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fd61c010480 tx=0x7fd61c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=437433c7f61aaf7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.781+0000 7fd628ff9640 1 -- 172.21.15.7:0/3399974015 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c012250 con 0x55924eef7a00 2024-06-10T01:13:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.781+0000 7fd628ff9640 1 -- 172.21.15.7:0/3399974015 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd61c0106f0 con 0x55924eef7a00 2024-06-10T01:13:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.781+0000 7fd628ff9640 1 -- 172.21.15.7:0/3399974015 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c0035f0 con 0x55924eef7a00 2024-06-10T01:13:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.782+0000 7fd63c07f740 1 -- 172.21.15.7:0/3399974015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924eef7a00 msgr2=0x55924ef01790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.782+0000 7fd63c07f740 1 --2- 172.21.15.7:0/3399974015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924eef7a00 0x55924ef01790 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fd61c010480 tx=0x7fd61c009c00 comp rx=0 tx=0).stop 2024-06-10T01:13:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.782+0000 7fd63c07f740 1 -- 172.21.15.7:0/3399974015 shutdown_connections 2024-06-10T01:13:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.782+0000 7fd63c07f740 1 --2- 172.21.15.7:0/3399974015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924eefcb70 0x55924ef08790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.782+0000 7fd63c07f740 1 --2- 172.21.15.7:0/3399974015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924eef7a00 0x55924ef01790 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.782+0000 7fd63c07f740 1 --2- 172.21.15.7:0/3399974015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55924eef7010 0x55924ef01250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.782+0000 7fd63c07f740 1 -- 172.21.15.7:0/3399974015 >> 172.21.15.7:0/3399974015 conn(0x55924eef2680 msgr2=0x55924eef2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.782+0000 7fd63c07f740 1 -- 172.21.15.7:0/3399974015 shutdown_connections 2024-06-10T01:13:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.782+0000 7fd63c07f740 1 -- 172.21.15.7:0/3399974015 wait complete. 2024-06-10T01:13:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.782+0000 7fd63c07f740 1 -- start start 2024-06-10T01:13:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.782+0000 7fd63c07f740 1 Processor -- start 2024-06-10T01:13:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.782+0000 7fd63c07f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924eef7010 0x55924ef05690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.782+0000 7fd63c07f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55924eef7a00 0x55924ef05bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.782+0000 7fd63c07f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924eefcb70 0x55924ef75e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.782+0000 7fd63c07f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd6341aae60 con 0x55924eefcb70 2024-06-10T01:13:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.782+0000 7fd63c07f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd6341ac2d0 con 0x55924eef7a00 2024-06-10T01:13:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.782+0000 7fd63c07f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd6341ab530 con 0x55924eef7010 2024-06-10T01:13:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.782+0000 7fd63c07f740 10 client.0 _ll_register_callbacks cb 0x7fd61c009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:13:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.782+0000 7fd6337fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924eef7010 0x55924ef05690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.783+0000 7fd6337fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924eef7010 0x55924ef05690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:56888/0 (socket says 172.21.15.7:56888) 2024-06-10T01:13:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.783+0000 7fd62bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55924eef7a00 0x55924ef05bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.783+0000 7fd6337fe640 1 -- 172.21.15.7:0/3848290200 learned_addr learned my addr 172.21.15.7:0/3848290200 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.783+0000 7fd6337fe640 1 --2- 172.21.15.7:0/3848290200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924eefcb70 0x55924ef75e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.783+0000 7fd62bfff640 1 -- 172.21.15.7:0/3848290200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924eef7010 msgr2=0x55924ef05690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.783+0000 7fd62bfff640 1 --2- 172.21.15.7:0/3848290200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924eef7010 0x55924ef05690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.783+0000 7fd62bfff640 1 -- 172.21.15.7:0/3848290200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924eefcb70 msgr2=0x55924ef75e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.783+0000 7fd62bfff640 1 --2- 172.21.15.7:0/3848290200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924eefcb70 0x55924ef75e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.783+0000 7fd62bfff640 1 -- 172.21.15.7:0/3848290200 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55924ef76390 con 0x55924eef7a00 2024-06-10T01:13:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.783+0000 7fd6337fe640 1 --2- 172.21.15.7:0/3848290200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924eef7010 0x55924ef05690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.783+0000 7fd62bfff640 1 --2- 172.21.15.7:0/3848290200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55924eef7a00 0x55924ef05bd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd61c009c90 tx=0x7fd61c005050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53b0f4cb3e6c41c5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.783+0000 7fd61a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:13:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.783+0000 7fd61a7fc640 1 -- 172.21.15.7:0/3848290200 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c036020 con 0x55924eef7a00 2024-06-10T01:13:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.783+0000 7fd61a7fc640 1 -- 172.21.15.7:0/3848290200 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fd61c012250 con 0x55924eef7a00 2024-06-10T01:13:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.783+0000 7fd63c07f740 1 -- 172.21.15.7:0/3848290200 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd61c0106f0 con 0x55924eef7a00 2024-06-10T01:13:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.783+0000 7fd61a7fc640 1 -- 172.21.15.7:0/3848290200 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd61c041050 con 0x55924eef7a00 2024-06-10T01:13:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.783+0000 7fd63c07f740 10 client.26711 fetch_fsmap learned FSMap version 33 2024-06-10T01:13:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.783+0000 7fd63c07f740 1 -- 172.21.15.7:0/3848290200 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55924ef775b0 con 0x55924eef7a00 2024-06-10T01:13:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.784+0000 7fd61a7fc640 1 -- 172.21.15.7:0/3848290200 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd61c0106f0 con 0x55924eef7a00 2024-06-10T01:13:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.784+0000 7fd61a7fc640 1 -- 172.21.15.7:0/3848290200 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7fd61c003ab0 con 0x55924eef7a00 2024-06-10T01:13:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.784+0000 7fd63c07f740 10 client.26711 fetch_fsmap finished waiting for FSMap version 33 2024-06-10T01:13:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.784+0000 7fd63c07f740 10 client.26711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:13:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.784+0000 7fd63c07f740 20 client.26711 populate_metadata read hostname 'smithi007' 2024-06-10T01:13:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.784+0000 7fd63c07f740 1 --2- 172.21.15.7:0/3848290200 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55924ef0d110 0x55924ef79f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.784+0000 7fd63c07f740 4 client.26711 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:13:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.784+0000 7fd63c07f740 1 -- 172.21.15.7:0/3848290200 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55924ee67c20 con 0x55924ef0d110 2024-06-10T01:13:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.784+0000 7fd6337fe640 1 --2- 172.21.15.7:0/3848290200 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55924ef0d110 0x55924ef79f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.785+0000 7fd6337fe640 1 --2- 172.21.15.7:0/3848290200 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55924ef0d110 0x55924ef79f40 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=711f091794a16392 server_cookie=de6033f202674e8b in_seq=0 out_seq=0 2024-06-10T01:13:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.785+0000 7fd61a7fc640 10 client.26711 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.932+0000 7fd61a7fc640 1 -- 172.21.15.7:0/3848290200 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55924ee67c20 con 0x55924ef0d110 2024-06-10T01:13:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.932+0000 7fd61a7fc640 10 client.26711 handle_command_reply: tid=0 2024-06-10T01:13:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.932+0000 7fd63c07f740 1 client.26711 shutdown 2024-06-10T01:13:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.933+0000 7fd63c07f740 1 -- 172.21.15.7:0/3848290200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55924eef7a00 msgr2=0x55924ef05bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.933+0000 7fd63c07f740 1 --2- 172.21.15.7:0/3848290200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55924eef7a00 0x55924ef05bd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd61c009c90 tx=0x7fd61c005050 comp rx=0 tx=0).stop 2024-06-10T01:13:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.933+0000 7fd63c07f740 1 -- 172.21.15.7:0/3848290200 shutdown_connections 2024-06-10T01:13:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.933+0000 7fd63c07f740 1 --2- 172.21.15.7:0/3848290200 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55924ef0d110 0x55924ef79f40 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.933+0000 7fd63c07f740 1 --2- 172.21.15.7:0/3848290200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924eefcb70 0x55924ef75e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.933+0000 7fd63c07f740 1 --2- 172.21.15.7:0/3848290200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55924eef7a00 0x55924ef05bd0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.933+0000 7fd61a7fc640 0 client.26711 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.933+0000 7fd63c07f740 1 --2- 172.21.15.7:0/3848290200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924eef7010 0x55924ef05690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.933+0000 7fd63c07f740 1 -- 172.21.15.7:0/3848290200 >> 172.21.15.7:0/3848290200 conn(0x55924eef2680 msgr2=0x55924ef01210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.933+0000 7fd63c07f740 1 -- 172.21.15.7:0/3848290200 shutdown_connections 2024-06-10T01:13:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.933+0000 7fd63c07f740 1 -- 172.21.15.7:0/3848290200 wait complete. 2024-06-10T01:13:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.933+0000 7fd63c07f740 20 client.26711 trim_cache size 0 max 16384 2024-06-10T01:13:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:13:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:13:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:13:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.933+0000 7fd63936a640 1 -- 172.21.15.7:0/3966507904 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd6200a2b80 msgr2=0x7fd6200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.933+0000 7fd63936a640 1 --2- 172.21.15.7:0/3966507904 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd6200a2b80 0x7fd6200a5040 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd624009a20 tx=0x7fd6240f5530 comp rx=0 tx=0).stop 2024-06-10T01:13:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.933+0000 7fd63936a640 1 -- 172.21.15.7:0/3966507904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd634102500 msgr2=0x7fd634199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.933+0000 7fd63936a640 1 --2- 172.21.15.7:0/3966507904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd634102500 0x7fd634199f50 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fd624009230 tx=0x7fd6240042f0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.934+0000 7fd63936a640 1 -- 172.21.15.7:0/3966507904 shutdown_connections 2024-06-10T01:13:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.934+0000 7fd63936a640 1 --2- 172.21.15.7:0/3966507904 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd6200a2b80 0x7fd6200a5040 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.934+0000 7fd63936a640 1 --2- 172.21.15.7:0/3966507904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd634107760 0x7fd6341a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.934+0000 7fd63936a640 1 --2- 172.21.15.7:0/3966507904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd634104e30 0x7fd63419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.934+0000 7fd63936a640 1 --2- 172.21.15.7:0/3966507904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd634102500 0x7fd634199f50 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.934+0000 7fd63936a640 1 -- 172.21.15.7:0/3966507904 >> 172.21.15.7:0/3966507904 conn(0x7fd6340fbe70 msgr2=0x7fd634101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.934+0000 7fd63936a640 1 -- 172.21.15.7:0/3966507904 shutdown_connections 2024-06-10T01:13:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:06.934+0000 7fd63936a640 1 -- 172.21.15.7:0/3966507904 wait complete. 2024-06-10T01:13:07.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:07 smithi175 ceph-mon[36636]: pgmap v645: 129 pgs: 129 active+clean; 80 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 385 op/s 2024-06-10T01:13:07.736 INFO:tasks.workunit.client.0.smithi007.stdout: 1 479881 7.30 MB/sec execute 228 sec latency 75.492 ms 2024-06-10T01:13:07.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:07 smithi007 ceph-mon[32398]: pgmap v645: 129 pgs: 129 active+clean; 80 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 385 op/s 2024-06-10T01:13:07.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:07 smithi078 ceph-mon[36683]: pgmap v645: 129 pgs: 129 active+clean; 80 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 385 op/s 2024-06-10T01:13:08.736 INFO:tasks.workunit.client.0.smithi007.stdout: 1 481852 7.30 MB/sec execute 229 sec latency 14.806 ms 2024-06-10T01:13:09.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:09 smithi175 ceph-mon[36636]: pgmap v646: 129 pgs: 129 active+clean; 81 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 336 op/s 2024-06-10T01:13:09.736 INFO:tasks.workunit.client.0.smithi007.stdout: 1 484024 7.33 MB/sec execute 230 sec latency 20.440 ms 2024-06-10T01:13:09.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:09 smithi007 ceph-mon[32398]: pgmap v646: 129 pgs: 129 active+clean; 81 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 336 op/s 2024-06-10T01:13:09.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:09 smithi078 ceph-mon[36683]: pgmap v646: 129 pgs: 129 active+clean; 81 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 336 op/s 2024-06-10T01:13:10.737 INFO:tasks.workunit.client.0.smithi007.stdout: 1 486604 7.36 MB/sec execute 231 sec latency 46.532 ms 2024-06-10T01:13:11.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:11 smithi175 ceph-mon[36636]: pgmap v647: 129 pgs: 129 active+clean; 84 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 380 op/s 2024-06-10T01:13:11.737 INFO:tasks.workunit.client.0.smithi007.stdout: 1 488284 7.37 MB/sec execute 232 sec latency 27.903 ms 2024-06-10T01:13:11.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:11 smithi007 ceph-mon[32398]: pgmap v647: 129 pgs: 129 active+clean; 84 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 380 op/s 2024-06-10T01:13:11.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:11 smithi078 ceph-mon[36683]: pgmap v647: 129 pgs: 129 active+clean; 84 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 380 op/s 2024-06-10T01:13:12.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:12 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:13:12.737 INFO:tasks.workunit.client.0.smithi007.stdout: 1 490864 7.39 MB/sec execute 233 sec latency 13.659 ms 2024-06-10T01:13:12.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:12 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:13:12.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:12 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:13:13.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:13 smithi175 ceph-mon[36636]: pgmap v648: 129 pgs: 129 active+clean; 79 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 2.9 MiB/s wr, 363 op/s 2024-06-10T01:13:13.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:13 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:13:13.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:13 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:13:13.737 INFO:tasks.workunit.client.0.smithi007.stdout: 1 493683 7.43 MB/sec execute 234 sec latency 14.022 ms 2024-06-10T01:13:13.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:13 smithi007 ceph-mon[32398]: pgmap v648: 129 pgs: 129 active+clean; 79 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 2.9 MiB/s wr, 363 op/s 2024-06-10T01:13:13.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:13 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:13:13.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:13 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:13:13.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:13 smithi078 ceph-mon[36683]: pgmap v648: 129 pgs: 129 active+clean; 79 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 2.9 MiB/s wr, 363 op/s 2024-06-10T01:13:13.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:13 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:13:13.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:13 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:13:14.737 INFO:tasks.workunit.client.0.smithi007.stdout: 1 495501 7.44 MB/sec execute 235 sec latency 13.825 ms 2024-06-10T01:13:15.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:15 smithi175 ceph-mon[36636]: pgmap v649: 129 pgs: 129 active+clean; 79 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 3.3 MiB/s wr, 381 op/s 2024-06-10T01:13:15.737 INFO:tasks.workunit.client.0.smithi007.stdout: 1 497766 7.46 MB/sec execute 236 sec latency 42.859 ms 2024-06-10T01:13:15.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:15 smithi007 ceph-mon[32398]: pgmap v649: 129 pgs: 129 active+clean; 79 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 3.3 MiB/s wr, 381 op/s 2024-06-10T01:13:15.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:15 smithi078 ceph-mon[36683]: pgmap v649: 129 pgs: 129 active+clean; 79 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 3.3 MiB/s wr, 381 op/s 2024-06-10T01:13:16.737 INFO:tasks.workunit.client.0.smithi007.stdout: 1 499857 7.46 MB/sec execute 237 sec latency 16.715 ms 2024-06-10T01:13:17.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:17 smithi175 ceph-mon[36636]: pgmap v650: 129 pgs: 129 active+clean; 79 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 3.0 MiB/s wr, 377 op/s 2024-06-10T01:13:17.737 INFO:tasks.workunit.client.0.smithi007.stdout: 1 501865 7.48 MB/sec execute 238 sec latency 19.182 ms 2024-06-10T01:13:17.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:17 smithi007 ceph-mon[32398]: pgmap v650: 129 pgs: 129 active+clean; 79 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 3.0 MiB/s wr, 377 op/s 2024-06-10T01:13:17.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:17 smithi078 ceph-mon[36683]: pgmap v650: 129 pgs: 129 active+clean; 79 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 3.0 MiB/s wr, 377 op/s 2024-06-10T01:13:18.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:18 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:13:18.737 INFO:tasks.workunit.client.0.smithi007.stdout: 1 504634 7.52 MB/sec execute 239 sec latency 12.360 ms 2024-06-10T01:13:18.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:18 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:13:18.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:18 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:13:19.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:19 smithi175 ceph-mon[36636]: pgmap v651: 129 pgs: 129 active+clean; 79 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 2.9 MiB/s wr, 338 op/s 2024-06-10T01:13:19.737 INFO:tasks.workunit.client.0.smithi007.stdout: 1 506582 7.52 MB/sec execute 240 sec latency 11.513 ms 2024-06-10T01:13:19.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:19 smithi007 ceph-mon[32398]: pgmap v651: 129 pgs: 129 active+clean; 79 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 2.9 MiB/s wr, 338 op/s 2024-06-10T01:13:19.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:19 smithi078 ceph-mon[36683]: pgmap v651: 129 pgs: 129 active+clean; 79 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 2.9 MiB/s wr, 338 op/s 2024-06-10T01:13:20.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.032+0000 7f40458a3640 1 -- 172.21.15.7:0/3964377661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4040100900 msgr2=0x7f4040100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:20.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.032+0000 7f40458a3640 1 --2- 172.21.15.7:0/3964377661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4040100900 0x7f4040100d20 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f4030009900 tx=0x7f403002a410 comp rx=0 tx=0).stop 2024-06-10T01:13:20.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.032+0000 7f40458a3640 1 -- 172.21.15.7:0/3964377661 shutdown_connections 2024-06-10T01:13:20.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.032+0000 7f40458a3640 1 --2- 172.21.15.7:0/3964377661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40401024b0 0x7f4040108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.032+0000 7f40458a3640 1 --2- 172.21.15.7:0/3964377661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4040101af0 0x7f4040101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.032+0000 7f40458a3640 1 --2- 172.21.15.7:0/3964377661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4040100900 0x7f4040100d20 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.032+0000 7f40458a3640 1 -- 172.21.15.7:0/3964377661 >> 172.21.15.7:0/3964377661 conn(0x7f40400fbe70 msgr2=0x7f40400fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.032+0000 7f40458a3640 1 -- 172.21.15.7:0/3964377661 shutdown_connections 2024-06-10T01:13:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.033+0000 7f40458a3640 1 -- 172.21.15.7:0/3964377661 wait complete. 2024-06-10T01:13:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.033+0000 7f40458a3640 1 Processor -- start 2024-06-10T01:13:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.033+0000 7f40458a3640 1 -- start start 2024-06-10T01:13:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.033+0000 7f40458a3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4040100900 0x7f4040197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.033+0000 7f40458a3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4040101af0 0x7f40401982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.033+0000 7f40458a3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40401024b0 0x7f404019f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.033+0000 7f40458a3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f404010c570 con 0x7f4040101af0 2024-06-10T01:13:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.033+0000 7f40458a3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f404010c3f0 con 0x7f40401024b0 2024-06-10T01:13:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.033+0000 7f40458a3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f404010c6f0 con 0x7f4040100900 2024-06-10T01:13:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.034+0000 7f403effd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4040100900 0x7f4040197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.034+0000 7f403f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40401024b0 0x7f404019f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.034+0000 7f403effd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4040100900 0x7f4040197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:46706/0 (socket says 172.21.15.7:46706) 2024-06-10T01:13:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.034+0000 7f403effd640 1 -- 172.21.15.7:0/3287880516 learned_addr learned my addr 172.21.15.7:0/3287880516 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.034+0000 7f403e7fc640 1 --2- 172.21.15.7:0/3287880516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4040101af0 0x7f40401982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.034+0000 7f403f7fe640 1 -- 172.21.15.7:0/3287880516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4040100900 msgr2=0x7f4040197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.034+0000 7f403f7fe640 1 --2- 172.21.15.7:0/3287880516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4040100900 0x7f4040197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.034+0000 7f403f7fe640 1 -- 172.21.15.7:0/3287880516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4040101af0 msgr2=0x7f40401982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.034+0000 7f403f7fe640 1 --2- 172.21.15.7:0/3287880516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4040101af0 0x7f40401982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.034+0000 7f403f7fe640 1 -- 172.21.15.7:0/3287880516 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404019f9c0 con 0x7f40401024b0 2024-06-10T01:13:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.034+0000 7f403f7fe640 1 --2- 172.21.15.7:0/3287880516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40401024b0 0x7f404019f320 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4038009fd0 tx=0x7f403800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.034+0000 7f40448a1640 1 -- 172.21.15.7:0/3287880516 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403800ce20 con 0x7f40401024b0 2024-06-10T01:13:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.034+0000 7f40448a1640 1 -- 172.21.15.7:0/3287880516 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4038005020 con 0x7f40401024b0 2024-06-10T01:13:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.034+0000 7f40458a3640 1 -- 172.21.15.7:0/3287880516 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f404019fcb0 con 0x7f40401024b0 2024-06-10T01:13:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.034+0000 7f40448a1640 1 -- 172.21.15.7:0/3287880516 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4038010720 con 0x7f40401024b0 2024-06-10T01:13:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.034+0000 7f40458a3640 1 -- 172.21.15.7:0/3287880516 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40401a01b0 con 0x7f40401024b0 2024-06-10T01:13:20.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.037+0000 7f40448a1640 1 -- 172.21.15.7:0/3287880516 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f4038005540 con 0x7f40401024b0 2024-06-10T01:13:20.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.037+0000 7f40485a8740 1 Processor -- start 2024-06-10T01:13:20.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.037+0000 7f40485a8740 1 -- start start 2024-06-10T01:13:20.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.037+0000 7f40485a8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f846ac010 0x557f846ac430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:20.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.037+0000 7f40485a8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f846aca00 0x557f846b1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:20.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.037+0000 7f40485a8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f846b1b70 0x557f846b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:20.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.037+0000 7f40485a8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557f8460f4f0 con 0x557f846ac010 2024-06-10T01:13:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.037+0000 7f40485a8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557f845ad870 con 0x557f846b1b70 2024-06-10T01:13:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.037+0000 7f40485a8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557f8460b4b0 con 0x557f846aca00 2024-06-10T01:13:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.037+0000 7f403e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f846ac010 0x557f846ac430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.037+0000 7f403e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f846ac010 0x557f846ac430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42188/0 (socket says 172.21.15.7:42188) 2024-06-10T01:13:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.037+0000 7f403e7fc640 1 -- 172.21.15.7:0/4149986350 learned_addr learned my addr 172.21.15.7:0/4149986350 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.037+0000 7f403effd640 1 --2- 172.21.15.7:0/4149986350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f846aca00 0x557f846b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.037+0000 7f40448a1640 1 --2- 172.21.15.7:0/3287880516 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f40280a2b80 0x7f40280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.037+0000 7f403e7fc640 1 --2- 172.21.15.7:0/4149986350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f846b1b70 0x557f846b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.038+0000 7f403f7fe640 1 --2- 172.21.15.7:0/3287880516 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f40280a2b80 0x7f40280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.038+0000 7f403e7fc640 1 -- 172.21.15.7:0/4149986350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f846aca00 msgr2=0x557f846b1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.038+0000 7f403e7fc640 1 --2- 172.21.15.7:0/4149986350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f846aca00 0x557f846b1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.038+0000 7f403e7fc640 1 -- 172.21.15.7:0/4149986350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f846b1b70 msgr2=0x557f846b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.038+0000 7f403e7fc640 1 --2- 172.21.15.7:0/4149986350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f846b1b70 0x557f846b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.038+0000 7f403e7fc640 1 -- 172.21.15.7:0/4149986350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f845c99b0 con 0x557f846ac010 2024-06-10T01:13:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.038+0000 7f40448a1640 1 -- 172.21.15.7:0/3287880516 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f40380cfbb0 con 0x7f40401024b0 2024-06-10T01:13:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.038+0000 7f403effd640 1 --2- 172.21.15.7:0/4149986350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f846aca00 0x557f846b1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.038+0000 7f403e7fc640 1 --2- 172.21.15.7:0/4149986350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f846ac010 0x557f846ac430 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f4034009710 tx=0x7f40340097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce13b199a59646be server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.038+0000 7f403f7fe640 1 --2- 172.21.15.7:0/3287880516 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f40280a2b80 0x7f40280a5040 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4038086b60 tx=0x7f40380670c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.038+0000 7f4025ffb640 1 -- 172.21.15.7:0/4149986350 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034013170 con 0x557f846ac010 2024-06-10T01:13:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.038+0000 7f4025ffb640 1 -- 172.21.15.7:0/4149986350 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f403400fca0 con 0x557f846ac010 2024-06-10T01:13:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.038+0000 7f4025ffb640 1 -- 172.21.15.7:0/4149986350 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034010a70 con 0x557f846ac010 2024-06-10T01:13:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.039+0000 7f40485a8740 1 -- 172.21.15.7:0/4149986350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f846ac010 msgr2=0x557f846ac430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.039+0000 7f40485a8740 1 --2- 172.21.15.7:0/4149986350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f846ac010 0x557f846ac430 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f4034009710 tx=0x7f40340097a0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.039+0000 7f40485a8740 1 -- 172.21.15.7:0/4149986350 shutdown_connections 2024-06-10T01:13:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.039+0000 7f40485a8740 1 --2- 172.21.15.7:0/4149986350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f846b1b70 0x557f846b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.039+0000 7f40485a8740 1 --2- 172.21.15.7:0/4149986350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f846aca00 0x557f846b1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.039+0000 7f40485a8740 1 --2- 172.21.15.7:0/4149986350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f846ac010 0x557f846ac430 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.039+0000 7f40485a8740 1 -- 172.21.15.7:0/4149986350 >> 172.21.15.7:0/4149986350 conn(0x557f846a7680 msgr2=0x557f846a7a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.039+0000 7f40485a8740 1 -- 172.21.15.7:0/4149986350 shutdown_connections 2024-06-10T01:13:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.039+0000 7f40485a8740 1 -- 172.21.15.7:0/4149986350 wait complete. 2024-06-10T01:13:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.039+0000 7f40485a8740 1 -- start start 2024-06-10T01:13:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.040+0000 7f40485a8740 1 Processor -- start 2024-06-10T01:13:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.040+0000 7f40485a8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f846ac010 0x557f846b6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.040+0000 7f40485a8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f846aca00 0x557f846b6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.040+0000 7f40485a8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f846b1b70 0x557f846bd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.040+0000 7f40485a8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557f846b4b20 con 0x557f846aca00 2024-06-10T01:13:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.040+0000 7f40485a8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557f84598ae0 con 0x557f846b1b70 2024-06-10T01:13:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.040+0000 7f40485a8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557f846b4ca0 con 0x557f846ac010 2024-06-10T01:13:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.040+0000 7f40485a8740 10 client.0 _ll_register_callbacks cb 0x557f845d19b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:13:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.040+0000 7f403effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f846aca00 0x557f846b6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.040+0000 7f403effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f846aca00 0x557f846b6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42194/0 (socket says 172.21.15.7:42194) 2024-06-10T01:13:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.040+0000 7f403effd640 1 -- 172.21.15.7:0/2960074799 learned_addr learned my addr 172.21.15.7:0/2960074799 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.040+0000 7f403e7fc640 1 --2- 172.21.15.7:0/2960074799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f846ac010 0x557f846b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.040+0000 7f403e7fc640 1 --2- 172.21.15.7:0/2960074799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f846b1b70 0x557f846bd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.040+0000 7f403effd640 1 -- 172.21.15.7:0/2960074799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f846ac010 msgr2=0x557f846b6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.040+0000 7f403effd640 1 --2- 172.21.15.7:0/2960074799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f846ac010 0x557f846b6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.040+0000 7f403effd640 1 -- 172.21.15.7:0/2960074799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f846b1b70 msgr2=0x557f846bd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.040+0000 7f403effd640 1 --2- 172.21.15.7:0/2960074799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f846b1b70 0x557f846bd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.040+0000 7f403effd640 1 -- 172.21.15.7:0/2960074799 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f84323210 con 0x557f846aca00 2024-06-10T01:13:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.040+0000 7f403e7fc640 1 --2- 172.21.15.7:0/2960074799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f846ac010 0x557f846b6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.041+0000 7f403effd640 1 --2- 172.21.15.7:0/2960074799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f846aca00 0x557f846b6790 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f4030009230 tx=0x7f4030031950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be60e3239c3fe4e1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.041+0000 7f401effd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:13:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.041+0000 7f401effd640 1 -- 172.21.15.7:0/2960074799 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4030035040 con 0x557f846aca00 2024-06-10T01:13:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.041+0000 7f40485a8740 1 -- 172.21.15.7:0/2960074799 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f403400fca0 con 0x557f846aca00 2024-06-10T01:13:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.041+0000 7f401effd640 1 -- 172.21.15.7:0/2960074799 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f4030004240 con 0x557f846aca00 2024-06-10T01:13:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.041+0000 7f401effd640 1 -- 172.21.15.7:0/2960074799 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f403004f050 con 0x557f846aca00 2024-06-10T01:13:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.041+0000 7f40485a8740 10 client.17310 fetch_fsmap learned FSMap version 34 2024-06-10T01:13:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.041+0000 7f40485a8740 1 -- 172.21.15.7:0/2960074799 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557f843ef620 con 0x557f846aca00 2024-06-10T01:13:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.041+0000 7f401effd640 1 -- 172.21.15.7:0/2960074799 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f403400fca0 con 0x557f846aca00 2024-06-10T01:13:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.041+0000 7f401effd640 1 -- 172.21.15.7:0/2960074799 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f4030046640 con 0x557f846aca00 2024-06-10T01:13:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.041+0000 7f40485a8740 10 client.17310 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T01:13:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.041+0000 7f40485a8740 10 client.17310 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:13:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.041+0000 7f40485a8740 20 client.17310 populate_metadata read hostname 'smithi007' 2024-06-10T01:13:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.041+0000 7f40485a8740 1 --2- 172.21.15.7:0/2960074799 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f846c21c0 0x557f846c45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.041+0000 7f40485a8740 4 client.17310 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:13:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.041+0000 7f40485a8740 1 -- 172.21.15.7:0/2960074799 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f84622f10 con 0x557f846c21c0 2024-06-10T01:13:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.042+0000 7f403e7fc640 1 --2- 172.21.15.7:0/2960074799 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f846c21c0 0x557f846c45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.042+0000 7f401effd640 10 client.17310 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.042+0000 7f403e7fc640 1 --2- 172.21.15.7:0/2960074799 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f846c21c0 0x557f846c45e0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=364e7d285cbca236 server_cookie=900e41296f7971ce in_seq=0 out_seq=0 2024-06-10T01:13:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.043+0000 7f401effd640 1 -- 172.21.15.7:0/2960074799 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557f84622f10 con 0x557f846c21c0 2024-06-10T01:13:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.043+0000 7f401effd640 10 client.17310 handle_command_reply: tid=0 2024-06-10T01:13:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.043+0000 7f40485a8740 1 client.17310 shutdown 2024-06-10T01:13:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.043+0000 7f40485a8740 1 -- 172.21.15.7:0/2960074799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f846aca00 msgr2=0x557f846b6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.043+0000 7f40485a8740 1 --2- 172.21.15.7:0/2960074799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f846aca00 0x557f846b6790 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f4030009230 tx=0x7f4030031950 comp rx=0 tx=0).stop 2024-06-10T01:13:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.043+0000 7f40485a8740 1 -- 172.21.15.7:0/2960074799 shutdown_connections 2024-06-10T01:13:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.043+0000 7f40485a8740 1 --2- 172.21.15.7:0/2960074799 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f846c21c0 0x557f846c45e0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.043+0000 7f40485a8740 1 --2- 172.21.15.7:0/2960074799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f846b1b70 0x557f846bd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.043+0000 7f40485a8740 1 --2- 172.21.15.7:0/2960074799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f846aca00 0x557f846b6790 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.043+0000 7f401effd640 0 client.17310 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.043+0000 7f40485a8740 1 --2- 172.21.15.7:0/2960074799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f846ac010 0x557f846b6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.043+0000 7f40485a8740 1 -- 172.21.15.7:0/2960074799 >> 172.21.15.7:0/2960074799 conn(0x557f846a7680 msgr2=0x557f846a67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.043+0000 7f40485a8740 1 -- 172.21.15.7:0/2960074799 shutdown_connections 2024-06-10T01:13:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.043+0000 7f40485a8740 1 -- 172.21.15.7:0/2960074799 wait complete. 2024-06-10T01:13:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.043+0000 7f40485a8740 20 client.17310 trim_cache size 0 max 16384 2024-06-10T01:13:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.060+0000 7f40485a8740 1 Processor -- start 2024-06-10T01:13:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.060+0000 7f40485a8740 1 -- start start 2024-06-10T01:13:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.060+0000 7f40485a8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f846ac010 0x557f846b5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.060+0000 7f40485a8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f846aca00 0x557f846bd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.060+0000 7f40485a8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f846b1b70 0x557f846b6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.060+0000 7f40485a8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f40401acdb0 con 0x557f846aca00 2024-06-10T01:13:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.060+0000 7f40485a8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f40401aa650 con 0x557f846ac010 2024-06-10T01:13:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.060+0000 7f40485a8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f40401a84a0 con 0x557f846b1b70 2024-06-10T01:13:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.060+0000 7f403effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f846aca00 0x557f846bd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.060+0000 7f403effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f846aca00 0x557f846bd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42208/0 (socket says 172.21.15.7:42208) 2024-06-10T01:13:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.060+0000 7f403effd640 1 -- 172.21.15.7:0/1377221439 learned_addr learned my addr 172.21.15.7:0/1377221439 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.060+0000 7f403e7fc640 1 --2- 172.21.15.7:0/1377221439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f846ac010 0x557f846b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.060+0000 7f403e7fc640 1 --2- 172.21.15.7:0/1377221439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f846b1b70 0x557f846b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.060+0000 7f403effd640 1 -- 172.21.15.7:0/1377221439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f846b1b70 msgr2=0x557f846b6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.060+0000 7f403effd640 1 --2- 172.21.15.7:0/1377221439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f846b1b70 0x557f846b6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.060+0000 7f403effd640 1 -- 172.21.15.7:0/1377221439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f846ac010 msgr2=0x557f846b5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.060+0000 7f403effd640 1 --2- 172.21.15.7:0/1377221439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f846ac010 0x557f846b5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.060+0000 7f403effd640 1 -- 172.21.15.7:0/1377221439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f846c0a50 con 0x557f846aca00 2024-06-10T01:13:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.060+0000 7f403e7fc640 1 --2- 172.21.15.7:0/1377221439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f846b1b70 0x557f846b6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:13:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.060+0000 7f403effd640 1 --2- 172.21.15.7:0/1377221439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f846aca00 0x557f846bd790 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f4030006010 tx=0x7f4030009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c0eadb29f5a5cd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.061+0000 7f40257fa640 1 -- 172.21.15.7:0/1377221439 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4030059020 con 0x557f846aca00 2024-06-10T01:13:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.061+0000 7f40257fa640 1 -- 172.21.15.7:0/1377221439 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40300319c0 con 0x557f846aca00 2024-06-10T01:13:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.061+0000 7f40257fa640 1 -- 172.21.15.7:0/1377221439 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4030035070 con 0x557f846aca00 2024-06-10T01:13:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.061+0000 7f40485a8740 1 -- 172.21.15.7:0/1377221439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f846aca00 msgr2=0x557f846bd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.061+0000 7f40485a8740 1 --2- 172.21.15.7:0/1377221439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f846aca00 0x557f846bd790 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f4030006010 tx=0x7f4030009630 comp rx=0 tx=0).stop 2024-06-10T01:13:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.061+0000 7f40485a8740 1 -- 172.21.15.7:0/1377221439 shutdown_connections 2024-06-10T01:13:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.061+0000 7f40485a8740 1 --2- 172.21.15.7:0/1377221439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f846b1b70 0x557f846b6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.061+0000 7f40485a8740 1 --2- 172.21.15.7:0/1377221439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f846aca00 0x557f846bd790 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.061+0000 7f40485a8740 1 --2- 172.21.15.7:0/1377221439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f846ac010 0x557f846b5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.061+0000 7f40485a8740 1 -- 172.21.15.7:0/1377221439 >> 172.21.15.7:0/1377221439 conn(0x557f846a7680 msgr2=0x557f846a7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.061+0000 7f40485a8740 1 -- 172.21.15.7:0/1377221439 shutdown_connections 2024-06-10T01:13:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.062+0000 7f40485a8740 1 -- 172.21.15.7:0/1377221439 wait complete. 2024-06-10T01:13:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.062+0000 7f40485a8740 1 -- start start 2024-06-10T01:13:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.062+0000 7f40485a8740 1 Processor -- start 2024-06-10T01:13:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.062+0000 7f40485a8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f846ac010 0x557f846bd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.062+0000 7f40485a8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f846aca00 0x557f846ba0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.062+0000 7f40485a8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f846b1b70 0x557f846ba5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.062+0000 7f40485a8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f40401a8b10 con 0x557f846aca00 2024-06-10T01:13:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.062+0000 7f40485a8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f40401a9f80 con 0x557f846b1b70 2024-06-10T01:13:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.062+0000 7f40485a8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f40401a91e0 con 0x557f846ac010 2024-06-10T01:13:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.062+0000 7f40485a8740 10 client.0 _ll_register_callbacks cb 0x7f4030009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:13:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.062+0000 7f403effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f846aca00 0x557f846ba0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.062+0000 7f403e7fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f846ac010 0x557f846bd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.062+0000 7f403effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f846aca00 0x557f846ba0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42224/0 (socket says 172.21.15.7:42224) 2024-06-10T01:13:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.062+0000 7f403effd640 1 -- 172.21.15.7:0/845887324 learned_addr learned my addr 172.21.15.7:0/845887324 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.063+0000 7f403e7fc640 1 --2- 172.21.15.7:0/845887324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f846b1b70 0x557f846ba5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.063+0000 7f403effd640 1 -- 172.21.15.7:0/845887324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f846ac010 msgr2=0x557f846bd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.063+0000 7f403effd640 1 --2- 172.21.15.7:0/845887324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f846ac010 0x557f846bd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.063+0000 7f403effd640 1 -- 172.21.15.7:0/845887324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f846b1b70 msgr2=0x557f846ba5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.063+0000 7f403effd640 1 --2- 172.21.15.7:0/845887324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f846b1b70 0x557f846ba5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.063+0000 7f403effd640 1 -- 172.21.15.7:0/845887324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f846bac80 con 0x557f846aca00 2024-06-10T01:13:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.063+0000 7f403effd640 1 --2- 172.21.15.7:0/845887324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f846aca00 0x557f846ba0a0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f4030000c00 tx=0x7f4030005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ab89002ab143f0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.063+0000 7f401effd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:13:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.063+0000 7f401effd640 1 -- 172.21.15.7:0/845887324 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4030035070 con 0x557f846aca00 2024-06-10T01:13:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.063+0000 7f401effd640 1 -- 172.21.15.7:0/845887324 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f4030004780 con 0x557f846aca00 2024-06-10T01:13:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.063+0000 7f40485a8740 1 -- 172.21.15.7:0/845887324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40300319c0 con 0x557f846aca00 2024-06-10T01:13:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.063+0000 7f401effd640 1 -- 172.21.15.7:0/845887324 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4030066050 con 0x557f846aca00 2024-06-10T01:13:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.063+0000 7f40485a8740 10 client.17316 fetch_fsmap learned FSMap version 34 2024-06-10T01:13:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.063+0000 7f40485a8740 1 -- 172.21.15.7:0/845887324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557f8472bff0 con 0x557f846aca00 2024-06-10T01:13:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.064+0000 7f401effd640 1 -- 172.21.15.7:0/845887324 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40300319c0 con 0x557f846aca00 2024-06-10T01:13:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.064+0000 7f401effd640 1 -- 172.21.15.7:0/845887324 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f40300537d0 con 0x557f846aca00 2024-06-10T01:13:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.064+0000 7f40485a8740 10 client.17316 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T01:13:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.064+0000 7f40485a8740 10 client.17316 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:13:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.064+0000 7f40485a8740 20 client.17316 populate_metadata read hostname 'smithi007' 2024-06-10T01:13:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.064+0000 7f40485a8740 1 --2- 172.21.15.7:0/845887324 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f8472c8c0 0x557f8472ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.064+0000 7f40485a8740 4 client.17316 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:13:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.064+0000 7f40485a8740 1 -- 172.21.15.7:0/845887324 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557f8461cc20 con 0x557f8472c8c0 2024-06-10T01:13:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.064+0000 7f403e7fc640 1 --2- 172.21.15.7:0/845887324 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f8472c8c0 0x557f8472ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.064+0000 7f401effd640 10 client.17316 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.064+0000 7f403e7fc640 1 --2- 172.21.15.7:0/845887324 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f8472c8c0 0x557f8472ece0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cb476ee2508d8ba server_cookie=5230ca7339b4facb in_seq=0 out_seq=0 2024-06-10T01:13:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.271+0000 7f401effd640 1 -- 172.21.15.7:0/845887324 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557f8461cc20 con 0x557f8472c8c0 2024-06-10T01:13:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.271+0000 7f401effd640 10 client.17316 handle_command_reply: tid=0 2024-06-10T01:13:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.271+0000 7f40485a8740 1 client.17316 shutdown 2024-06-10T01:13:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.271+0000 7f40485a8740 1 -- 172.21.15.7:0/845887324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f846aca00 msgr2=0x557f846ba0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.271+0000 7f40485a8740 1 --2- 172.21.15.7:0/845887324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f846aca00 0x557f846ba0a0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f4030000c00 tx=0x7f4030005610 comp rx=0 tx=0).stop 2024-06-10T01:13:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.271+0000 7f40485a8740 1 -- 172.21.15.7:0/845887324 shutdown_connections 2024-06-10T01:13:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.271+0000 7f40485a8740 1 --2- 172.21.15.7:0/845887324 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f8472c8c0 0x557f8472ece0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.271+0000 7f40485a8740 1 --2- 172.21.15.7:0/845887324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f846b1b70 0x557f846ba5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.271+0000 7f40485a8740 1 --2- 172.21.15.7:0/845887324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f846aca00 0x557f846ba0a0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.271+0000 7f401effd640 0 client.17316 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.271+0000 7f40485a8740 1 --2- 172.21.15.7:0/845887324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f846ac010 0x557f846bd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.271+0000 7f40485a8740 1 -- 172.21.15.7:0/845887324 >> 172.21.15.7:0/845887324 conn(0x557f846a7680 msgr2=0x557f846b5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.272+0000 7f40485a8740 1 -- 172.21.15.7:0/845887324 shutdown_connections 2024-06-10T01:13:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.272+0000 7f40485a8740 1 -- 172.21.15.7:0/845887324 wait complete. 2024-06-10T01:13:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.272+0000 7f40485a8740 20 client.17316 trim_cache size 0 max 16384 2024-06-10T01:13:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:13:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:13:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:13:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.273+0000 7f40458a3640 1 -- 172.21.15.7:0/3287880516 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f40280a2b80 msgr2=0x7f40280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.273+0000 7f40458a3640 1 --2- 172.21.15.7:0/3287880516 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f40280a2b80 0x7f40280a5040 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4038086b60 tx=0x7f40380670c0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.273+0000 7f40458a3640 1 -- 172.21.15.7:0/3287880516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40401024b0 msgr2=0x7f404019f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.273+0000 7f40458a3640 1 --2- 172.21.15.7:0/3287880516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40401024b0 0x7f404019f320 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4038009fd0 tx=0x7f403800ef10 comp rx=0 tx=0).stop 2024-06-10T01:13:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.273+0000 7f40458a3640 1 -- 172.21.15.7:0/3287880516 shutdown_connections 2024-06-10T01:13:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.273+0000 7f40458a3640 1 --2- 172.21.15.7:0/3287880516 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f40280a2b80 0x7f40280a5040 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.273+0000 7f40458a3640 1 --2- 172.21.15.7:0/3287880516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40401024b0 0x7f404019f320 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.273+0000 7f40458a3640 1 --2- 172.21.15.7:0/3287880516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4040101af0 0x7f40401982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.273+0000 7f40458a3640 1 --2- 172.21.15.7:0/3287880516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4040100900 0x7f4040197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.273+0000 7f40458a3640 1 -- 172.21.15.7:0/3287880516 >> 172.21.15.7:0/3287880516 conn(0x7f40400fbe70 msgr2=0x7f40400fc540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.273+0000 7f40458a3640 1 -- 172.21.15.7:0/3287880516 shutdown_connections 2024-06-10T01:13:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:20.273+0000 7f40458a3640 1 -- 172.21.15.7:0/3287880516 wait complete. 2024-06-10T01:13:20.737 INFO:tasks.workunit.client.0.smithi007.stdout: 1 508423 7.55 MB/sec execute 241 sec latency 95.513 ms 2024-06-10T01:13:21.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:21 smithi175 ceph-mon[36636]: pgmap v652: 129 pgs: 129 active+clean; 81 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 3.4 MiB/s wr, 398 op/s 2024-06-10T01:13:21.737 INFO:tasks.workunit.client.0.smithi007.stdout: 1 510520 7.55 MB/sec execute 242 sec latency 29.904 ms 2024-06-10T01:13:21.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:21 smithi007 ceph-mon[32398]: pgmap v652: 129 pgs: 129 active+clean; 81 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 3.4 MiB/s wr, 398 op/s 2024-06-10T01:13:21.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:21 smithi078 ceph-mon[36683]: pgmap v652: 129 pgs: 129 active+clean; 81 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 3.4 MiB/s wr, 398 op/s 2024-06-10T01:13:22.737 INFO:tasks.workunit.client.0.smithi007.stdout: 1 512493 7.57 MB/sec execute 243 sec latency 20.614 ms 2024-06-10T01:13:23.630 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:23 smithi078 ceph-mon[36683]: pgmap v653: 129 pgs: 129 active+clean; 81 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 2.8 MiB/s wr, 305 op/s 2024-06-10T01:13:23.630 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:23 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:13:23.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:23 smithi175 ceph-mon[36636]: pgmap v653: 129 pgs: 129 active+clean; 81 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 2.8 MiB/s wr, 305 op/s 2024-06-10T01:13:23.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:23 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:13:23.737 INFO:tasks.workunit.client.0.smithi007.stdout: 1 515233 7.60 MB/sec execute 244 sec latency 11.538 ms 2024-06-10T01:13:23.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:23 smithi007 ceph-mon[32398]: pgmap v653: 129 pgs: 129 active+clean; 81 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 2.8 MiB/s wr, 305 op/s 2024-06-10T01:13:23.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:23 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:13:24.737 INFO:tasks.workunit.client.0.smithi007.stdout: 1 516998 7.60 MB/sec execute 245 sec latency 18.212 ms 2024-06-10T01:13:25.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:25 smithi175 ceph-mon[36636]: pgmap v654: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 3.0 MiB/s wr, 350 op/s 2024-06-10T01:13:25.738 INFO:tasks.workunit.client.0.smithi007.stdout: 1 519054 7.63 MB/sec execute 246 sec latency 46.680 ms 2024-06-10T01:13:25.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:25 smithi007 ceph-mon[32398]: pgmap v654: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 3.0 MiB/s wr, 350 op/s 2024-06-10T01:13:25.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:25 smithi078 ceph-mon[36683]: pgmap v654: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 3.0 MiB/s wr, 350 op/s 2024-06-10T01:13:26.738 INFO:tasks.workunit.client.0.smithi007.stdout: 1 521069 7.62 MB/sec execute 247 sec latency 42.130 ms 2024-06-10T01:13:27.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:27 smithi175 ceph-mon[36636]: pgmap v655: 129 pgs: 129 active+clean; 76 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 2.8 MiB/s wr, 364 op/s 2024-06-10T01:13:27.737 INFO:tasks.workunit.client.0.smithi007.stdout: 1 522881 7.64 MB/sec execute 248 sec latency 14.552 ms 2024-06-10T01:13:27.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:27 smithi007 ceph-mon[32398]: pgmap v655: 129 pgs: 129 active+clean; 76 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 2.8 MiB/s wr, 364 op/s 2024-06-10T01:13:27.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:27 smithi078 ceph-mon[36683]: pgmap v655: 129 pgs: 129 active+clean; 76 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 2.8 MiB/s wr, 364 op/s 2024-06-10T01:13:28.737 INFO:tasks.workunit.client.0.smithi007.stdout: 1 525202 7.66 MB/sec execute 249 sec latency 22.574 ms 2024-06-10T01:13:29.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:29 smithi175 ceph-mon[36636]: pgmap v656: 129 pgs: 129 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 2.8 MiB/s wr, 372 op/s 2024-06-10T01:13:29.737 INFO:tasks.workunit.client.0.smithi007.stdout: 1 526837 7.66 MB/sec execute 250 sec latency 15.356 ms 2024-06-10T01:13:29.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:29 smithi007 ceph-mon[32398]: pgmap v656: 129 pgs: 129 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 2.8 MiB/s wr, 372 op/s 2024-06-10T01:13:29.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:29 smithi078 ceph-mon[36683]: pgmap v656: 129 pgs: 129 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 2.8 MiB/s wr, 372 op/s 2024-06-10T01:13:30.737 INFO:tasks.workunit.client.0.smithi007.stdout: 1 529379 7.70 MB/sec execute 251 sec latency 83.189 ms 2024-06-10T01:13:31.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:31 smithi175 ceph-mon[36636]: pgmap v657: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 3.0 MiB/s wr, 388 op/s 2024-06-10T01:13:31.737 INFO:tasks.workunit.client.0.smithi007.stdout: 1 530800 7.69 MB/sec execute 252 sec latency 23.969 ms 2024-06-10T01:13:31.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:31 smithi007 ceph-mon[32398]: pgmap v657: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 3.0 MiB/s wr, 388 op/s 2024-06-10T01:13:31.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:31 smithi078 ceph-mon[36683]: pgmap v657: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 3.0 MiB/s wr, 388 op/s 2024-06-10T01:13:32.737 INFO:tasks.workunit.client.0.smithi007.stdout: 1 533267 7.72 MB/sec execute 253 sec latency 9.664 ms 2024-06-10T01:13:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.376+0000 7f35c95a9640 1 -- 172.21.15.7:0/3560372936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35c4100930 msgr2=0x7f35c4100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.376+0000 7f35c95a9640 1 --2- 172.21.15.7:0/3560372936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35c4100930 0x7f35c4100d50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f35b4009900 tx=0x7f35b402a410 comp rx=0 tx=0).stop 2024-06-10T01:13:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.376+0000 7f35c95a9640 1 -- 172.21.15.7:0/3560372936 shutdown_connections 2024-06-10T01:13:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.376+0000 7f35c95a9640 1 --2- 172.21.15.7:0/3560372936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35c41024e0 0x7f35c4108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.376+0000 7f35c95a9640 1 --2- 172.21.15.7:0/3560372936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35c4101b20 0x7f35c4101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.376+0000 7f35c95a9640 1 --2- 172.21.15.7:0/3560372936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35c4100930 0x7f35c4100d50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.376+0000 7f35c95a9640 1 -- 172.21.15.7:0/3560372936 >> 172.21.15.7:0/3560372936 conn(0x7f35c40fbec0 msgr2=0x7f35c40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.377+0000 7f35c95a9640 1 -- 172.21.15.7:0/3560372936 shutdown_connections 2024-06-10T01:13:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.377+0000 7f35c95a9640 1 -- 172.21.15.7:0/3560372936 wait complete. 2024-06-10T01:13:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.377+0000 7f35c95a9640 1 Processor -- start 2024-06-10T01:13:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.377+0000 7f35c95a9640 1 -- start start 2024-06-10T01:13:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.378+0000 7f35c95a9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35c4100930 0x7f35c4197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.378+0000 7f35c95a9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35c4101b20 0x7f35c41982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.378+0000 7f35c95a9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35c41024e0 0x7f35c419f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.378+0000 7f35c95a9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f35c410c5a0 con 0x7f35c4100930 2024-06-10T01:13:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.378+0000 7f35c95a9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f35c410c420 con 0x7f35c4101b20 2024-06-10T01:13:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.378+0000 7f35c95a9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f35c410c720 con 0x7f35c41024e0 2024-06-10T01:13:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.378+0000 7f35c2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35c4100930 0x7f35c4197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.378+0000 7f35c2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35c4100930 0x7f35c4197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60272/0 (socket says 172.21.15.7:60272) 2024-06-10T01:13:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.378+0000 7f35c2ffd640 1 -- 172.21.15.7:0/1696571752 learned_addr learned my addr 172.21.15.7:0/1696571752 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.378+0000 7f35c37fe640 1 --2- 172.21.15.7:0/1696571752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35c41024e0 0x7f35c419f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.378+0000 7f35c27fc640 1 --2- 172.21.15.7:0/1696571752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35c4101b20 0x7f35c41982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.378+0000 7f35c2ffd640 1 -- 172.21.15.7:0/1696571752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35c41024e0 msgr2=0x7f35c419f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.378+0000 7f35c2ffd640 1 --2- 172.21.15.7:0/1696571752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35c41024e0 0x7f35c419f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.378+0000 7f35c2ffd640 1 -- 172.21.15.7:0/1696571752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35c4101b20 msgr2=0x7f35c41982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.378+0000 7f35c2ffd640 1 --2- 172.21.15.7:0/1696571752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35c4101b20 0x7f35c41982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.378+0000 7f35c2ffd640 1 -- 172.21.15.7:0/1696571752 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c419f970 con 0x7f35c4100930 2024-06-10T01:13:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.378+0000 7f35c2ffd640 1 --2- 172.21.15.7:0/1696571752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35c4100930 0x7f35c4197da0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f35b4031610 tx=0x7f35b4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.381+0000 7f35abfff640 1 -- 172.21.15.7:0/1696571752 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b403f030 con 0x7f35c4100930 2024-06-10T01:13:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.381+0000 7f35c95a9640 1 -- 172.21.15.7:0/1696571752 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35c419fc00 con 0x7f35c4100930 2024-06-10T01:13:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.381+0000 7f35abfff640 1 -- 172.21.15.7:0/1696571752 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35b4003ad0 con 0x7f35c4100930 2024-06-10T01:13:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.381+0000 7f35abfff640 1 -- 172.21.15.7:0/1696571752 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b4035070 con 0x7f35c4100930 2024-06-10T01:13:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.382+0000 7f35c95a9640 1 -- 172.21.15.7:0/1696571752 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35c41a0140 con 0x7f35c4100930 2024-06-10T01:13:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.384+0000 7f35abfff640 1 -- 172.21.15.7:0/1696571752 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f35b4042030 con 0x7f35c4100930 2024-06-10T01:13:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.384+0000 7f35cc399740 1 Processor -- start 2024-06-10T01:13:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.384+0000 7f35cc399740 1 -- start start 2024-06-10T01:13:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.384+0000 7f35cc399740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dffd338010 0x55dffd338430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.384+0000 7f35cc399740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dffd338a00 0x55dffd33d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.384+0000 7f35cc399740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffd33db70 0x55dffd340000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.384+0000 7f35cc399740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55dffd29b4f0 con 0x55dffd338a00 2024-06-10T01:13:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.384+0000 7f35cc399740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55dffd239870 con 0x55dffd33db70 2024-06-10T01:13:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.384+0000 7f35cc399740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dffd2974b0 con 0x55dffd338010 2024-06-10T01:13:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.384+0000 7f35abfff640 1 --2- 172.21.15.7:0/1696571752 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35ac0a2b80 0x7f35ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.385+0000 7f35c37fe640 1 --2- 172.21.15.7:0/1696571752 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35ac0a2b80 0x7f35ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.385+0000 7f35c2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dffd338a00 0x55dffd33d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.385+0000 7f35c37fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dffd338010 0x55dffd338430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.385+0000 7f35c27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffd33db70 0x55dffd340000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.385+0000 7f35c37fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dffd338010 0x55dffd338430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52208/0 (socket says 172.21.15.7:52208) 2024-06-10T01:13:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.385+0000 7f35c27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffd33db70 0x55dffd340000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:34060/0 (socket says 172.21.15.7:34060) 2024-06-10T01:13:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.385+0000 7f35c37fe640 1 -- 172.21.15.7:0/3898684903 learned_addr learned my addr 172.21.15.7:0/3898684903 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.385+0000 7f35c2ffd640 1 --2- 172.21.15.7:0/3898684903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dffd338a00 0x55dffd33d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60278/0 (socket says 172.21.15.7:60278) 2024-06-10T01:13:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.385+0000 7f35abfff640 1 -- 172.21.15.7:0/1696571752 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f35b40ef780 con 0x7f35c4100930 2024-06-10T01:13:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.385+0000 7f35c2ffd640 1 -- 172.21.15.7:0/3898684903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dffd338010 msgr2=0x55dffd338430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.385+0000 7f35c2ffd640 1 --2- 172.21.15.7:0/3898684903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dffd338010 0x55dffd338430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.385+0000 7f35c2ffd640 1 -- 172.21.15.7:0/3898684903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffd33db70 msgr2=0x55dffd340000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.385+0000 7f35c2ffd640 1 --2- 172.21.15.7:0/3898684903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffd33db70 0x55dffd340000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.385+0000 7f35c2ffd640 1 -- 172.21.15.7:0/3898684903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dffd2559b0 con 0x55dffd338a00 2024-06-10T01:13:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.385+0000 7f35c37fe640 1 --2- 172.21.15.7:0/3898684903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dffd338010 0x55dffd338430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.385+0000 7f35c27fc640 1 --2- 172.21.15.7:0/3898684903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffd33db70 0x55dffd340000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.385+0000 7f35c2ffd640 1 --2- 172.21.15.7:0/3898684903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dffd338a00 0x55dffd33d500 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f35b40ecf50 tx=0x7f35b4040fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bbcd2a824638980 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.385+0000 7f35c37fe640 1 --2- 172.21.15.7:0/1696571752 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35ac0a2b80 0x7f35ac0a5040 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x55dffd339260 tx=0x7f35bc004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.385+0000 7f35a97fa640 1 -- 172.21.15.7:0/3898684903 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b4049040 con 0x55dffd338a00 2024-06-10T01:13:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.385+0000 7f35a97fa640 1 -- 172.21.15.7:0/3898684903 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35b40bc360 con 0x55dffd338a00 2024-06-10T01:13:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.385+0000 7f35a97fa640 1 -- 172.21.15.7:0/3898684903 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b4003c80 con 0x55dffd338a00 2024-06-10T01:13:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.386+0000 7f35cc399740 1 -- 172.21.15.7:0/3898684903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dffd338a00 msgr2=0x55dffd33d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.386+0000 7f35cc399740 1 --2- 172.21.15.7:0/3898684903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dffd338a00 0x55dffd33d500 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f35b40ecf50 tx=0x7f35b4040fb0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.386+0000 7f35cc399740 1 -- 172.21.15.7:0/3898684903 shutdown_connections 2024-06-10T01:13:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.386+0000 7f35cc399740 1 --2- 172.21.15.7:0/3898684903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffd33db70 0x55dffd340000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.386+0000 7f35cc399740 1 --2- 172.21.15.7:0/3898684903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dffd338a00 0x55dffd33d500 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.386+0000 7f35cc399740 1 --2- 172.21.15.7:0/3898684903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dffd338010 0x55dffd338430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.386+0000 7f35cc399740 1 -- 172.21.15.7:0/3898684903 >> 172.21.15.7:0/3898684903 conn(0x55dffd333680 msgr2=0x55dffd333a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.386+0000 7f35cc399740 1 -- 172.21.15.7:0/3898684903 shutdown_connections 2024-06-10T01:13:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.386+0000 7f35cc399740 1 -- 172.21.15.7:0/3898684903 wait complete. 2024-06-10T01:13:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.386+0000 7f35cc399740 1 -- start start 2024-06-10T01:13:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.387+0000 7f35cc399740 1 Processor -- start 2024-06-10T01:13:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.387+0000 7f35cc399740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dffd338010 0x55dffd341f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.387+0000 7f35cc399740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dffd338a00 0x55dffd342460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.387+0000 7f35cc399740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffd33db70 0x55dffd3494a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.387+0000 7f35cc399740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55dffd3408e0 con 0x55dffd338a00 2024-06-10T01:13:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.387+0000 7f35cc399740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55dffd224ae0 con 0x55dffd33db70 2024-06-10T01:13:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.387+0000 7f35cc399740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dffd340a60 con 0x55dffd338010 2024-06-10T01:13:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.387+0000 7f35cc399740 10 client.0 _ll_register_callbacks cb 0x55dffd25d9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:13:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.387+0000 7f35c27fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dffd338a00 0x55dffd342460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.387+0000 7f35c2ffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dffd338010 0x55dffd341f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.387+0000 7f35c27fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dffd338a00 0x55dffd342460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60292/0 (socket says 172.21.15.7:60292) 2024-06-10T01:13:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.387+0000 7f35c27fc640 1 -- 172.21.15.7:0/4249847838 learned_addr learned my addr 172.21.15.7:0/4249847838 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.387+0000 7f35c37fe640 1 --2- 172.21.15.7:0/4249847838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffd33db70 0x55dffd3494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.387+0000 7f35c2ffd640 1 -- 172.21.15.7:0/4249847838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffd33db70 msgr2=0x55dffd3494a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.387+0000 7f35c2ffd640 1 --2- 172.21.15.7:0/4249847838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffd33db70 0x55dffd3494a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.387+0000 7f35c2ffd640 1 -- 172.21.15.7:0/4249847838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dffd338a00 msgr2=0x55dffd342460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.387+0000 7f35c2ffd640 1 --2- 172.21.15.7:0/4249847838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dffd338a00 0x55dffd342460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.387+0000 7f35c2ffd640 1 -- 172.21.15.7:0/4249847838 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dffcfaf210 con 0x55dffd338010 2024-06-10T01:13:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.387+0000 7f35c27fc640 1 --2- 172.21.15.7:0/4249847838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dffd338a00 0x55dffd342460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.387+0000 7f35c2ffd640 1 --2- 172.21.15.7:0/4249847838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dffd338010 0x55dffd341f20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f35b40bc510 tx=0x7f35b40bb8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=990cb37bbd86be61 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.387+0000 7f35a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:13:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.388+0000 7f35a6ff5640 1 -- 172.21.15.7:0/4249847838 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b4049070 con 0x55dffd338010 2024-06-10T01:13:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.388+0000 7f35a6ff5640 1 -- 172.21.15.7:0/4249847838 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f35b40b82d0 con 0x55dffd338010 2024-06-10T01:13:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.388+0000 7f35cc399740 1 -- 172.21.15.7:0/4249847838 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35b40bc360 con 0x55dffd338010 2024-06-10T01:13:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.388+0000 7f35a6ff5640 1 -- 172.21.15.7:0/4249847838 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f35b40b9d70 con 0x55dffd338010 2024-06-10T01:13:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.388+0000 7f35cc399740 10 client.26758 fetch_fsmap learned FSMap version 35 2024-06-10T01:13:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.388+0000 7f35cc399740 1 -- 172.21.15.7:0/4249847838 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55dffd07b620 con 0x55dffd338010 2024-06-10T01:13:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.388+0000 7f35a6ff5640 1 -- 172.21.15.7:0/4249847838 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35b40bc360 con 0x55dffd338010 2024-06-10T01:13:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.388+0000 7f35a6ff5640 1 -- 172.21.15.7:0/4249847838 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f35b4031bd0 con 0x55dffd338010 2024-06-10T01:13:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.388+0000 7f35cc399740 10 client.26758 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:13:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.388+0000 7f35cc399740 10 client.26758 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:13:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.388+0000 7f35cc399740 20 client.26758 populate_metadata read hostname 'smithi007' 2024-06-10T01:13:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.388+0000 7f35cc399740 1 --2- 172.21.15.7:0/4249847838 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55dffd34dba0 0x55dffd34ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.388+0000 7f35cc399740 4 client.26758 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:13:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.388+0000 7f35cc399740 1 -- 172.21.15.7:0/4249847838 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dffd2aef10 con 0x55dffd34dba0 2024-06-10T01:13:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.389+0000 7f35c27fc640 1 --2- 172.21.15.7:0/4249847838 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55dffd34dba0 0x55dffd34ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.389+0000 7f35c27fc640 1 --2- 172.21.15.7:0/4249847838 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55dffd34dba0 0x55dffd34ffd0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4f6953c3da883ed server_cookie=7b7b982cf09acaea in_seq=0 out_seq=0 2024-06-10T01:13:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.389+0000 7f35a6ff5640 10 client.26758 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.390+0000 7f35a6ff5640 1 -- 172.21.15.7:0/4249847838 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55dffd2aef10 con 0x55dffd34dba0 2024-06-10T01:13:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.390+0000 7f35a6ff5640 10 client.26758 handle_command_reply: tid=0 2024-06-10T01:13:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.390+0000 7f35cc399740 1 client.26758 shutdown 2024-06-10T01:13:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.390+0000 7f35cc399740 1 -- 172.21.15.7:0/4249847838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dffd338010 msgr2=0x55dffd341f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.390+0000 7f35cc399740 1 --2- 172.21.15.7:0/4249847838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dffd338010 0x55dffd341f20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f35b40bc510 tx=0x7f35b40bb8a0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.390+0000 7f35cc399740 1 -- 172.21.15.7:0/4249847838 shutdown_connections 2024-06-10T01:13:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.390+0000 7f35cc399740 1 --2- 172.21.15.7:0/4249847838 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55dffd34dba0 0x55dffd34ffd0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.390+0000 7f35cc399740 1 --2- 172.21.15.7:0/4249847838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffd33db70 0x55dffd3494a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.390+0000 7f35cc399740 1 --2- 172.21.15.7:0/4249847838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dffd338a00 0x55dffd342460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.390+0000 7f35cc399740 1 --2- 172.21.15.7:0/4249847838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dffd338010 0x55dffd341f20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.390+0000 7f35a6ff5640 0 client.26758 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.390+0000 7f35cc399740 1 -- 172.21.15.7:0/4249847838 >> 172.21.15.7:0/4249847838 conn(0x55dffd333680 msgr2=0x55dffd3327e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.390+0000 7f35cc399740 1 -- 172.21.15.7:0/4249847838 shutdown_connections 2024-06-10T01:13:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.390+0000 7f35cc399740 1 -- 172.21.15.7:0/4249847838 wait complete. 2024-06-10T01:13:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.390+0000 7f35cc399740 20 client.26758 trim_cache size 0 max 16384 2024-06-10T01:13:33.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.407+0000 7f35cc399740 1 Processor -- start 2024-06-10T01:13:33.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.407+0000 7f35cc399740 1 -- start start 2024-06-10T01:13:33.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.407+0000 7f35cc399740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffd338010 0x55dffd341b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.407+0000 7f35cc399740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dffd338a00 0x55dffd349460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.407+0000 7f35cc399740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dffd33db70 0x55dffd342070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.407+0000 7f35cc399740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f35c41acee0 con 0x55dffd33db70 2024-06-10T01:13:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.407+0000 7f35cc399740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f35c41aa750 con 0x55dffd338010 2024-06-10T01:13:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.407+0000 7f35cc399740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f35c41a85a0 con 0x55dffd338a00 2024-06-10T01:13:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.407+0000 7f35c37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dffd33db70 0x55dffd342070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.407+0000 7f35c37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dffd33db70 0x55dffd342070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60308/0 (socket says 172.21.15.7:60308) 2024-06-10T01:13:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.407+0000 7f35c37fe640 1 -- 172.21.15.7:0/1240763534 learned_addr learned my addr 172.21.15.7:0/1240763534 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.407+0000 7f35c27fc640 1 --2- 172.21.15.7:0/1240763534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dffd338a00 0x55dffd349460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.407+0000 7f35c2ffd640 1 --2- 172.21.15.7:0/1240763534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffd338010 0x55dffd341b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.407+0000 7f35c37fe640 1 -- 172.21.15.7:0/1240763534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dffd338a00 msgr2=0x55dffd349460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.407+0000 7f35c37fe640 1 --2- 172.21.15.7:0/1240763534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dffd338a00 0x55dffd349460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.407+0000 7f35c37fe640 1 -- 172.21.15.7:0/1240763534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffd338010 msgr2=0x55dffd341b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.407+0000 7f35c37fe640 1 --2- 172.21.15.7:0/1240763534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffd338010 0x55dffd341b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.407+0000 7f35c37fe640 1 -- 172.21.15.7:0/1240763534 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dffd350650 con 0x55dffd33db70 2024-06-10T01:13:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.407+0000 7f35c37fe640 1 --2- 172.21.15.7:0/1240763534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dffd33db70 0x55dffd342070 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f35bc0109b0 tx=0x7f35bc012070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eee7a1a796c4322 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.408+0000 7f35a8ff9640 1 -- 172.21.15.7:0/1240763534 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35bc02f070 con 0x55dffd33db70 2024-06-10T01:13:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.408+0000 7f35a8ff9640 1 -- 172.21.15.7:0/1240763534 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35bc00fcc0 con 0x55dffd33db70 2024-06-10T01:13:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.408+0000 7f35a8ff9640 1 -- 172.21.15.7:0/1240763534 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35bc019040 con 0x55dffd33db70 2024-06-10T01:13:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.408+0000 7f35cc399740 1 -- 172.21.15.7:0/1240763534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dffd33db70 msgr2=0x55dffd342070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.408+0000 7f35cc399740 1 --2- 172.21.15.7:0/1240763534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dffd33db70 0x55dffd342070 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f35bc0109b0 tx=0x7f35bc012070 comp rx=0 tx=0).stop 2024-06-10T01:13:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.408+0000 7f35cc399740 1 -- 172.21.15.7:0/1240763534 shutdown_connections 2024-06-10T01:13:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.408+0000 7f35cc399740 1 --2- 172.21.15.7:0/1240763534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dffd33db70 0x55dffd342070 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.408+0000 7f35cc399740 1 --2- 172.21.15.7:0/1240763534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dffd338a00 0x55dffd349460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.408+0000 7f35cc399740 1 --2- 172.21.15.7:0/1240763534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffd338010 0x55dffd341b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.408+0000 7f35cc399740 1 -- 172.21.15.7:0/1240763534 >> 172.21.15.7:0/1240763534 conn(0x55dffd333680 msgr2=0x55dffd33d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.408+0000 7f35cc399740 1 -- 172.21.15.7:0/1240763534 shutdown_connections 2024-06-10T01:13:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.408+0000 7f35cc399740 1 -- 172.21.15.7:0/1240763534 wait complete. 2024-06-10T01:13:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.409+0000 7f35cc399740 1 -- start start 2024-06-10T01:13:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.409+0000 7f35cc399740 1 Processor -- start 2024-06-10T01:13:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.409+0000 7f35cc399740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dffd338010 0x55dffd346440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.409+0000 7f35cc399740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dffd338a00 0x55dffd346980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.409+0000 7f35cc399740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffd33db70 0x55dffd3b68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.409+0000 7f35cc399740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f35c41a8c10 con 0x55dffd338010 2024-06-10T01:13:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.409+0000 7f35c2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dffd338010 0x55dffd346440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.409+0000 7f35cc399740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f35c41aa080 con 0x55dffd33db70 2024-06-10T01:13:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.409+0000 7f35cc399740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f35c41a92e0 con 0x55dffd338a00 2024-06-10T01:13:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.410+0000 7f35c2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dffd338010 0x55dffd346440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60316/0 (socket says 172.21.15.7:60316) 2024-06-10T01:13:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.410+0000 7f35cc399740 10 client.0 _ll_register_callbacks cb 0x7f35b8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:13:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.410+0000 7f35c27fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dffd338a00 0x55dffd346980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.410+0000 7f35c2ffd640 1 -- 172.21.15.7:0/202586474 learned_addr learned my addr 172.21.15.7:0/202586474 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.410+0000 7f35c37fe640 1 --2- 172.21.15.7:0/202586474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffd33db70 0x55dffd3b68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.410+0000 7f35c27fc640 1 -- 172.21.15.7:0/202586474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffd33db70 msgr2=0x55dffd3b68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.410+0000 7f35c27fc640 1 --2- 172.21.15.7:0/202586474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffd33db70 0x55dffd3b68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.410+0000 7f35c27fc640 1 -- 172.21.15.7:0/202586474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dffd338010 msgr2=0x55dffd346440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.410+0000 7f35c27fc640 1 --2- 172.21.15.7:0/202586474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dffd338010 0x55dffd346440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.410+0000 7f35c27fc640 1 -- 172.21.15.7:0/202586474 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dffd3387a0 con 0x55dffd338a00 2024-06-10T01:13:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.410+0000 7f35c2ffd640 1 --2- 172.21.15.7:0/202586474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dffd338010 0x55dffd346440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.410+0000 7f35c27fc640 1 --2- 172.21.15.7:0/202586474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dffd338a00 0x55dffd346980 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x55dffd3505d0 tx=0x7f35b8012ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40c6ddac7ceca268 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.410+0000 7f35a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:13:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.410+0000 7f35a6ff5640 1 -- 172.21.15.7:0/202586474 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b8005220 con 0x55dffd338a00 2024-06-10T01:13:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.410+0000 7f35a6ff5640 1 -- 172.21.15.7:0/202586474 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f35b80053c0 con 0x55dffd338a00 2024-06-10T01:13:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.410+0000 7f35cc399740 1 -- 172.21.15.7:0/202586474 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35bc00fcc0 con 0x55dffd338a00 2024-06-10T01:13:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.411+0000 7f35a6ff5640 1 -- 172.21.15.7:0/202586474 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f35b80055c0 con 0x55dffd338a00 2024-06-10T01:13:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.411+0000 7f35cc399740 10 client.26764 fetch_fsmap learned FSMap version 35 2024-06-10T01:13:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.411+0000 7f35cc399740 1 -- 172.21.15.7:0/202586474 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55dffd334350 con 0x55dffd338a00 2024-06-10T01:13:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.411+0000 7f35a6ff5640 1 -- 172.21.15.7:0/202586474 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35bc00fcc0 con 0x55dffd338a00 2024-06-10T01:13:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.411+0000 7f35a6ff5640 1 -- 172.21.15.7:0/202586474 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f35b8025070 con 0x55dffd338a00 2024-06-10T01:13:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.411+0000 7f35cc399740 10 client.26764 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:13:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.411+0000 7f35cc399740 10 client.26764 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:13:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.411+0000 7f35cc399740 20 client.26764 populate_metadata read hostname 'smithi007' 2024-06-10T01:13:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.411+0000 7f35cc399740 1 --2- 172.21.15.7:0/202586474 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55dffd3b8880 0x55dffd3baca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.411+0000 7f35cc399740 4 client.26764 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:13:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.411+0000 7f35cc399740 1 -- 172.21.15.7:0/202586474 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dffd2a8c20 con 0x55dffd3b8880 2024-06-10T01:13:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.411+0000 7f35c2ffd640 1 --2- 172.21.15.7:0/202586474 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55dffd3b8880 0x55dffd3baca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.412+0000 7f35c2ffd640 1 --2- 172.21.15.7:0/202586474 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55dffd3b8880 0x55dffd3baca0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8245321410a306f3 server_cookie=8f6e5bcc6ee28d48 in_seq=0 out_seq=0 2024-06-10T01:13:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.412+0000 7f35a6ff5640 10 client.26764 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.545+0000 7f35a6ff5640 1 -- 172.21.15.7:0/202586474 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55dffd2a8c20 con 0x55dffd3b8880 2024-06-10T01:13:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.545+0000 7f35a6ff5640 10 client.26764 handle_command_reply: tid=0 2024-06-10T01:13:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.545+0000 7f35cc399740 1 client.26764 shutdown 2024-06-10T01:13:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.545+0000 7f35cc399740 1 -- 172.21.15.7:0/202586474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dffd338a00 msgr2=0x55dffd346980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.545+0000 7f35cc399740 1 --2- 172.21.15.7:0/202586474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dffd338a00 0x55dffd346980 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x55dffd3505d0 tx=0x7f35b8012ae0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.546+0000 7f35cc399740 1 -- 172.21.15.7:0/202586474 shutdown_connections 2024-06-10T01:13:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.546+0000 7f35cc399740 1 --2- 172.21.15.7:0/202586474 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55dffd3b8880 0x55dffd3baca0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.546+0000 7f35cc399740 1 --2- 172.21.15.7:0/202586474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffd33db70 0x55dffd3b68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.546+0000 7f35cc399740 1 --2- 172.21.15.7:0/202586474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dffd338a00 0x55dffd346980 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.546+0000 7f35cc399740 1 --2- 172.21.15.7:0/202586474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dffd338010 0x55dffd346440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.546+0000 7f35cc399740 1 -- 172.21.15.7:0/202586474 >> 172.21.15.7:0/202586474 conn(0x55dffd333680 msgr2=0x55dffd341b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.546+0000 7f35a6ff5640 0 client.26764 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.546+0000 7f35cc399740 1 -- 172.21.15.7:0/202586474 shutdown_connections 2024-06-10T01:13:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.546+0000 7f35cc399740 1 -- 172.21.15.7:0/202586474 wait complete. 2024-06-10T01:13:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.546+0000 7f35cc399740 20 client.26764 trim_cache size 0 max 16384 2024-06-10T01:13:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:13:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:13:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:13:33.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.546+0000 7f35c95a9640 1 -- 172.21.15.7:0/1696571752 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35ac0a2b80 msgr2=0x7f35ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:33.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.546+0000 7f35c95a9640 1 --2- 172.21.15.7:0/1696571752 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35ac0a2b80 0x7f35ac0a5040 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x55dffd339260 tx=0x7f35bc004ab0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.546+0000 7f35c95a9640 1 -- 172.21.15.7:0/1696571752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35c4100930 msgr2=0x7f35c4197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:33.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.547+0000 7f35c95a9640 1 --2- 172.21.15.7:0/1696571752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35c4100930 0x7f35c4197da0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f35b4031610 tx=0x7f35b4004270 comp rx=0 tx=0).stop 2024-06-10T01:13:33.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.547+0000 7f35c95a9640 1 -- 172.21.15.7:0/1696571752 shutdown_connections 2024-06-10T01:13:33.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.547+0000 7f35c95a9640 1 --2- 172.21.15.7:0/1696571752 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35ac0a2b80 0x7f35ac0a5040 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.547+0000 7f35c95a9640 1 --2- 172.21.15.7:0/1696571752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35c41024e0 0x7f35c419f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.547+0000 7f35c95a9640 1 --2- 172.21.15.7:0/1696571752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35c4101b20 0x7f35c41982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.547+0000 7f35c95a9640 1 --2- 172.21.15.7:0/1696571752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35c4100930 0x7f35c4197da0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:33.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.547+0000 7f35c95a9640 1 -- 172.21.15.7:0/1696571752 >> 172.21.15.7:0/1696571752 conn(0x7f35c40fbec0 msgr2=0x7f35c40fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:33.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.547+0000 7f35c95a9640 1 -- 172.21.15.7:0/1696571752 shutdown_connections 2024-06-10T01:13:33.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:33.547+0000 7f35c95a9640 1 -- 172.21.15.7:0/1696571752 wait complete. 2024-06-10T01:13:33.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:33 smithi175 ceph-mon[36636]: pgmap v658: 129 pgs: 129 active+clean; 83 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 2.9 MiB/s wr, 354 op/s 2024-06-10T01:13:33.738 INFO:tasks.workunit.client.0.smithi007.stdout: 1 535223 7.71 MB/sec execute 254 sec latency 98.216 ms 2024-06-10T01:13:33.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:33 smithi007 ceph-mon[32398]: pgmap v658: 129 pgs: 129 active+clean; 83 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 2.9 MiB/s wr, 354 op/s 2024-06-10T01:13:33.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:33 smithi078 ceph-mon[36683]: pgmap v658: 129 pgs: 129 active+clean; 83 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 2.9 MiB/s wr, 354 op/s 2024-06-10T01:13:34.738 INFO:tasks.workunit.client.0.smithi007.stdout: 1 536995 7.73 MB/sec execute 255 sec latency 21.512 ms 2024-06-10T01:13:35.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:35 smithi175 ceph-mon[36636]: pgmap v659: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 3.8 MiB/s wr, 386 op/s 2024-06-10T01:13:35.738 INFO:tasks.workunit.client.0.smithi007.stdout: 1 539307 7.75 MB/sec execute 256 sec latency 52.082 ms 2024-06-10T01:13:35.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:35 smithi007 ceph-mon[32398]: pgmap v659: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 3.8 MiB/s wr, 386 op/s 2024-06-10T01:13:35.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:35 smithi078 ceph-mon[36683]: pgmap v659: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 3.8 MiB/s wr, 386 op/s 2024-06-10T01:13:36.492 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:13:36.738 INFO:tasks.workunit.client.0.smithi007.stdout: 1 540521 7.75 MB/sec execute 257 sec latency 141.928 ms 2024-06-10T01:13:36.745 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:13:37.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:37 smithi175 ceph-mon[36636]: pgmap v660: 129 pgs: 129 active+clean; 97 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 4.3 MiB/s wr, 347 op/s 2024-06-10T01:13:37.738 INFO:tasks.workunit.client.0.smithi007.stdout: 1 542545 7.75 MB/sec execute 258 sec latency 89.329 ms 2024-06-10T01:13:37.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:37 smithi007 ceph-mon[32398]: pgmap v660: 129 pgs: 129 active+clean; 97 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 4.3 MiB/s wr, 347 op/s 2024-06-10T01:13:37.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:37 smithi078 ceph-mon[36683]: pgmap v660: 129 pgs: 129 active+clean; 97 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 4.3 MiB/s wr, 347 op/s 2024-06-10T01:13:38.512 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:38 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:13:38.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:38 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:13:38.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:38 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:13:38.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:38 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:13:38.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:38 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:13:38.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:38 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:13:38.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:38 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:13:38.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:38 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:13:38.738 INFO:tasks.workunit.client.0.smithi007.stdout: 1 544280 7.76 MB/sec execute 259 sec latency 16.924 ms 2024-06-10T01:13:38.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:38 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:13:38.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:38 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:13:38.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:38 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:13:38.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:38 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:13:39.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.019+0000 7ff9cda9a700 1 -- 172.21.15.7:0/979747127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9c8101380 msgr2=0x7ff9c8101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:39.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.019+0000 7ff9cda9a700 1 --2- 172.21.15.7:0/979747127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9c8101380 0x7ff9c8101800 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009990 tx=0x7ff9bc009ca0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.020+0000 7ff9cda9a700 1 -- 172.21.15.7:0/979747127 shutdown_connections 2024-06-10T01:13:39.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.020+0000 7ff9cda9a700 1 --2- 172.21.15.7:0/979747127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9c8101d40 0x7ff9c810e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.020+0000 7ff9cda9a700 1 --2- 172.21.15.7:0/979747127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9c81073f0 0x7ff9c81077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.020+0000 7ff9cda9a700 1 --2- 172.21.15.7:0/979747127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9c8101380 0x7ff9c8101800 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.020+0000 7ff9cda9a700 1 -- 172.21.15.7:0/979747127 >> 172.21.15.7:0/979747127 conn(0x7ff9c8078300 msgr2=0x7ff9c8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:39.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.020+0000 7ff9cda9a700 1 -- 172.21.15.7:0/979747127 shutdown_connections 2024-06-10T01:13:39.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.020+0000 7ff9cda9a700 1 -- 172.21.15.7:0/979747127 wait complete. 2024-06-10T01:13:39.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.020+0000 7ff9cda9a700 1 Processor -- start 2024-06-10T01:13:39.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.020+0000 7ff9cda9a700 1 -- start start 2024-06-10T01:13:39.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.021+0000 7ff9cda9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9c8101380 0x7ff9c8103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:39.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.021+0000 7ff9cda9a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9c8101d40 0x7ff9c81035a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:39.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.021+0000 7ff9cda9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9c81073f0 0x7ff9c8103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:39.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.021+0000 7ff9cda9a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c81116e0 con 0x7ff9c8101380 2024-06-10T01:13:39.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.021+0000 7ff9cda9a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c8111560 con 0x7ff9c8101d40 2024-06-10T01:13:39.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.021+0000 7ff9cda9a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c8111860 con 0x7ff9c81073f0 2024-06-10T01:13:39.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.021+0000 7ff9c77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9c8101380 0x7ff9c8103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:39.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.021+0000 7ff9c77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9c8101380 0x7ff9c8103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60336/0 (socket says 172.21.15.7:60336) 2024-06-10T01:13:39.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.021+0000 7ff9c7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9c81073f0 0x7ff9c8103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:39.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.021+0000 7ff9c77fe700 1 -- 172.21.15.7:0/3792553276 learned_addr learned my addr 172.21.15.7:0/3792553276 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:39.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.021+0000 7ff9c6ffd700 1 --2- 172.21.15.7:0/3792553276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9c8101d40 0x7ff9c81035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:39.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.021+0000 7ff9c77fe700 1 -- 172.21.15.7:0/3792553276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9c81073f0 msgr2=0x7ff9c8103ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:39.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.021+0000 7ff9c77fe700 1 --2- 172.21.15.7:0/3792553276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9c81073f0 0x7ff9c8103ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.021+0000 7ff9c77fe700 1 -- 172.21.15.7:0/3792553276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9c8101d40 msgr2=0x7ff9c81035a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:39.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.021+0000 7ff9c77fe700 1 --2- 172.21.15.7:0/3792553276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9c8101d40 0x7ff9c81035a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.021+0000 7ff9c77fe700 1 -- 172.21.15.7:0/3792553276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c81a71f0 con 0x7ff9c8101380 2024-06-10T01:13:39.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.021+0000 7ff9c77fe700 1 --2- 172.21.15.7:0/3792553276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9c8101380 0x7ff9c8103060 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800e9a0 tx=0x7ff9b800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:39.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.021+0000 7ff9c4ff9700 1 -- 172.21.15.7:0/3792553276 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b800ca10 con 0x7ff9c8101380 2024-06-10T01:13:39.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.022+0000 7ff9c4ff9700 1 -- 172.21.15.7:0/3792553276 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9b800cbb0 con 0x7ff9c8101380 2024-06-10T01:13:39.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.022+0000 7ff9cda9a700 1 -- 172.21.15.7:0/3792553276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c81a74e0 con 0x7ff9c8101380 2024-06-10T01:13:39.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.022+0000 7ff9c4ff9700 1 -- 172.21.15.7:0/3792553276 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8018680 con 0x7ff9c8101380 2024-06-10T01:13:39.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.022+0000 7ff9cda9a700 1 -- 172.21.15.7:0/3792553276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c81a7a20 con 0x7ff9c8101380 2024-06-10T01:13:39.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.024+0000 7ff9c4ff9700 1 -- 172.21.15.7:0/3792553276 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7ff9b8010430 con 0x7ff9c8101380 2024-06-10T01:13:39.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.025+0000 7ff9c4ff9700 1 --2- 172.21.15.7:0/3792553276 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff9b00a2c20 0x7ff9b00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:39.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.025+0000 7ff9c6ffd700 1 --2- 172.21.15.7:0/3792553276 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff9b00a2c20 0x7ff9b00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:39.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.025+0000 7ff9c4ff9700 1 -- 172.21.15.7:0/3792553276 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7ff9b80cf5d0 con 0x7ff9c8101380 2024-06-10T01:13:39.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.025+0000 7ff9c6ffd700 1 --2- 172.21.15.7:0/3792553276 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff9b00a2c20 0x7ff9b00a50e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8106210 tx=0x7ff9bc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:39.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.025+0000 7ff9d831db80 1 Processor -- start 2024-06-10T01:13:39.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.025+0000 7ff9d831db80 1 -- start start 2024-06-10T01:13:39.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.026+0000 7ff9d831db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c61c3640 0x5649c61c3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:39.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.026+0000 7ff9d831db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649c61c3f60 0x5649c61bd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:39.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.026+0000 7ff9d831db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649c61bdf20 0x5649c61be340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:39.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.026+0000 7ff9d831db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5649c610f810 con 0x5649c61bdf20 2024-06-10T01:13:39.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.026+0000 7ff9d831db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5649c60b7720 con 0x5649c61c3640 2024-06-10T01:13:39.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.026+0000 7ff9d831db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5649c610b7d0 con 0x5649c61c3f60 2024-06-10T01:13:39.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.026+0000 7ff9c7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649c61bdf20 0x5649c61be340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:39.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.026+0000 7ff9c6ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649c61c3f60 0x5649c61bd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:39.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.026+0000 7ff9c7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649c61bdf20 0x5649c61be340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60340/0 (socket says 172.21.15.7:60340) 2024-06-10T01:13:39.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.026+0000 7ff9c77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c61c3640 0x5649c61c3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:39.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.026+0000 7ff9c7fff700 1 -- 172.21.15.7:0/2101350974 learned_addr learned my addr 172.21.15.7:0/2101350974 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:39.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.026+0000 7ff9c7fff700 1 -- 172.21.15.7:0/2101350974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649c61c3f60 msgr2=0x5649c61bd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:39.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.026+0000 7ff9c7fff700 1 --2- 172.21.15.7:0/2101350974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649c61c3f60 0x5649c61bd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.026+0000 7ff9c7fff700 1 -- 172.21.15.7:0/2101350974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c61c3640 msgr2=0x5649c61c3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:39.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.026+0000 7ff9c7fff700 1 --2- 172.21.15.7:0/2101350974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c61c3640 0x5649c61c3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.026+0000 7ff9c7fff700 1 -- 172.21.15.7:0/2101350974 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649c60adf50 con 0x5649c61bdf20 2024-06-10T01:13:39.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.026+0000 7ff9c77fe700 1 --2- 172.21.15.7:0/2101350974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c61c3640 0x5649c61c3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:13:39.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.026+0000 7ff9c7fff700 1 --2- 172.21.15.7:0/2101350974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649c61bdf20 0x5649c61be340 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0011680 tx=0x7ff9c0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f67302c617c1ad2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:39.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.027+0000 7ff9adffb700 1 -- 172.21.15.7:0/2101350974 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0019030 con 0x5649c61bdf20 2024-06-10T01:13:39.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.027+0000 7ff9adffb700 1 -- 172.21.15.7:0/2101350974 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9c00092e0 con 0x5649c61bdf20 2024-06-10T01:13:39.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.027+0000 7ff9adffb700 1 -- 172.21.15.7:0/2101350974 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0007700 con 0x5649c61bdf20 2024-06-10T01:13:39.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.027+0000 7ff9d831db80 1 -- 172.21.15.7:0/2101350974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649c61bdf20 msgr2=0x5649c61be340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:39.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.027+0000 7ff9d831db80 1 --2- 172.21.15.7:0/2101350974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649c61bdf20 0x5649c61be340 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0011680 tx=0x7ff9c0009920 comp rx=0 tx=0).stop 2024-06-10T01:13:39.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.027+0000 7ff9d831db80 1 -- 172.21.15.7:0/2101350974 shutdown_connections 2024-06-10T01:13:39.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.027+0000 7ff9d831db80 1 --2- 172.21.15.7:0/2101350974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649c61bdf20 0x5649c61be340 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.027+0000 7ff9d831db80 1 --2- 172.21.15.7:0/2101350974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c61c3640 0x5649c61c3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.027+0000 7ff9d831db80 1 --2- 172.21.15.7:0/2101350974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649c61c3f60 0x5649c61bd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.028+0000 7ff9d831db80 1 -- 172.21.15.7:0/2101350974 >> 172.21.15.7:0/2101350974 conn(0x5649c60e8cc0 msgr2=0x5649c61b8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:39.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.028+0000 7ff9d831db80 1 -- 172.21.15.7:0/2101350974 shutdown_connections 2024-06-10T01:13:39.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.028+0000 7ff9d831db80 1 -- 172.21.15.7:0/2101350974 wait complete. 2024-06-10T01:13:39.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.028+0000 7ff9d831db80 1 -- start start 2024-06-10T01:13:39.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.028+0000 7ff9d831db80 1 Processor -- start 2024-06-10T01:13:39.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.029+0000 7ff9d831db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649c61bdf20 0x5649c61c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:39.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.029+0000 7ff9d831db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649c61c3640 0x5649c61c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:39.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.029+0000 7ff9d831db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c61c3f60 0x5649c61c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:39.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.029+0000 7ff9d831db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5649c61ca9d0 con 0x5649c61c3640 2024-06-10T01:13:39.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.029+0000 7ff9d831db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5649c60afbf0 con 0x5649c61c3f60 2024-06-10T01:13:39.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.029+0000 7ff9d831db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5649c61cab50 con 0x5649c61bdf20 2024-06-10T01:13:39.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.029+0000 7ff9d831db80 10 client.0 _ll_register_callbacks cb 0x5649c60a9920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:13:39.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.029+0000 7ff9c6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649c61c3640 0x5649c61c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:39.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.029+0000 7ff9c6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649c61c3640 0x5649c61c6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60356/0 (socket says 172.21.15.7:60356) 2024-06-10T01:13:39.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.029+0000 7ff9c6ffd700 1 -- 172.21.15.7:0/2669120847 learned_addr learned my addr 172.21.15.7:0/2669120847 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:39.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.029+0000 7ff9c77fe700 1 --2- 172.21.15.7:0/2669120847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649c61bdf20 0x5649c61c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:39.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.029+0000 7ff9c7fff700 1 --2- 172.21.15.7:0/2669120847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c61c3f60 0x5649c61c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:39.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.029+0000 7ff9c6ffd700 1 -- 172.21.15.7:0/2669120847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649c61bdf20 msgr2=0x5649c61c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:39.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.029+0000 7ff9c6ffd700 1 --2- 172.21.15.7:0/2669120847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649c61bdf20 0x5649c61c9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.029+0000 7ff9c6ffd700 1 -- 172.21.15.7:0/2669120847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c61c3f60 msgr2=0x5649c61c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:39.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.029+0000 7ff9c6ffd700 1 --2- 172.21.15.7:0/2669120847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c61c3f60 0x5649c61c74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.029+0000 7ff9c6ffd700 1 -- 172.21.15.7:0/2669120847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649c605e8d0 con 0x5649c61c3640 2024-06-10T01:13:39.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.029+0000 7ff9c77fe700 1 --2- 172.21.15.7:0/2669120847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649c61bdf20 0x5649c61c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:39.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.029+0000 7ff9c6ffd700 1 --2- 172.21.15.7:0/2669120847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649c61c3640 0x5649c61c6fb0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009990 tx=0x7ff9bc009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be69d27f315336c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:39.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.029+0000 7ff9a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:13:39.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.030+0000 7ff9a6ffd700 1 -- 172.21.15.7:0/2669120847 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc00f460 con 0x5649c61c3640 2024-06-10T01:13:39.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.030+0000 7ff9a6ffd700 1 -- 172.21.15.7:0/2669120847 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7ff9bc00f600 con 0x5649c61c3640 2024-06-10T01:13:39.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.030+0000 7ff9d831db80 1 -- 172.21.15.7:0/2669120847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9c00092e0 con 0x5649c61c3640 2024-06-10T01:13:39.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.030+0000 7ff9a6ffd700 1 -- 172.21.15.7:0/2669120847 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff9bc03c050 con 0x5649c61c3640 2024-06-10T01:13:39.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.030+0000 7ff9d831db80 10 client.17346 fetch_fsmap learned FSMap version 35 2024-06-10T01:13:39.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.030+0000 7ff9d831db80 1 -- 172.21.15.7:0/2669120847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5649c61d52e0 con 0x5649c61c3640 2024-06-10T01:13:39.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.030+0000 7ff9a6ffd700 1 -- 172.21.15.7:0/2669120847 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9c00092e0 con 0x5649c61c3640 2024-06-10T01:13:39.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.030+0000 7ff9a6ffd700 1 -- 172.21.15.7:0/2669120847 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7ff9bc0226b0 con 0x5649c61c3640 2024-06-10T01:13:39.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.030+0000 7ff9d831db80 10 client.17346 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:13:39.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.030+0000 7ff9d831db80 10 client.17346 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:13:39.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.030+0000 7ff9d831db80 20 client.17346 populate_metadata read hostname 'smithi007' 2024-06-10T01:13:39.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.030+0000 7ff9d831db80 1 --2- 172.21.15.7:0/2669120847 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5649c61d7920 0x5649c61d9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:39.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.030+0000 7ff9d831db80 4 client.17346 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:13:39.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.030+0000 7ff9d831db80 1 -- 172.21.15.7:0/2669120847 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649c611cd10 con 0x5649c61d7920 2024-06-10T01:13:39.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.030+0000 7ff9c77fe700 1 --2- 172.21.15.7:0/2669120847 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5649c61d7920 0x5649c61d9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:39.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.031+0000 7ff9c77fe700 1 --2- 172.21.15.7:0/2669120847 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5649c61d7920 0x5649c61d9d30 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd5e77379452eda0 server_cookie=884c299edf52bc50 in_seq=0 out_seq=0 2024-06-10T01:13:39.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.031+0000 7ff9a6ffd700 10 client.17346 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:39.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.032+0000 7ff9a6ffd700 1 -- 172.21.15.7:0/2669120847 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5649c611cd10 con 0x5649c61d7920 2024-06-10T01:13:39.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.032+0000 7ff9a6ffd700 10 client.17346 handle_command_reply: tid=0 2024-06-10T01:13:39.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.033+0000 7ff9d831db80 1 client.17346 shutdown 2024-06-10T01:13:39.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.033+0000 7ff9d831db80 1 -- 172.21.15.7:0/2669120847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649c61c3640 msgr2=0x5649c61c6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:39.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.033+0000 7ff9d831db80 1 --2- 172.21.15.7:0/2669120847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649c61c3640 0x5649c61c6fb0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009990 tx=0x7ff9bc009ca0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.033+0000 7ff9d831db80 1 -- 172.21.15.7:0/2669120847 shutdown_connections 2024-06-10T01:13:39.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.033+0000 7ff9d831db80 1 --2- 172.21.15.7:0/2669120847 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5649c61d7920 0x5649c61d9d30 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.033+0000 7ff9d831db80 1 --2- 172.21.15.7:0/2669120847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c61c3f60 0x5649c61c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.033+0000 7ff9d831db80 1 --2- 172.21.15.7:0/2669120847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649c61bdf20 0x5649c61c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.033+0000 7ff9d831db80 1 --2- 172.21.15.7:0/2669120847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649c61c3640 0x5649c61c6fb0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.033+0000 7ff9d831db80 1 -- 172.21.15.7:0/2669120847 >> 172.21.15.7:0/2669120847 conn(0x5649c60e8cc0 msgr2=0x5649c61bc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:39.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.033+0000 7ff9a6ffd700 0 client.17346 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:39.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.033+0000 7ff9d831db80 1 -- 172.21.15.7:0/2669120847 shutdown_connections 2024-06-10T01:13:39.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.033+0000 7ff9d831db80 1 -- 172.21.15.7:0/2669120847 wait complete. 2024-06-10T01:13:39.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.033+0000 7ff9d831db80 20 client.17346 trim_cache size 0 max 16384 2024-06-10T01:13:39.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.054+0000 7ff9d831db80 1 Processor -- start 2024-06-10T01:13:39.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.054+0000 7ff9d831db80 1 -- start start 2024-06-10T01:13:39.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.054+0000 7ff9d831db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649c61bdf20 0x5649c61bd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:39.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.054+0000 7ff9d831db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c61c3640 0x5649c61b89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:39.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.054+0000 7ff9d831db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649c61c3f60 0x5649c61b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:39.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.054+0000 7ff9d831db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c81b2e40 con 0x5649c61bdf20 2024-06-10T01:13:39.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.054+0000 7ff9d831db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c81b0670 con 0x5649c61c3640 2024-06-10T01:13:39.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.054+0000 7ff9d831db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c81ae4c0 con 0x5649c61c3f60 2024-06-10T01:13:39.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.054+0000 7ff9c77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649c61bdf20 0x5649c61bd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:39.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.054+0000 7ff9c6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c61c3640 0x5649c61b89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:39.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.054+0000 7ff9c6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c61c3640 0x5649c61b89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:34126/0 (socket says 172.21.15.7:34126) 2024-06-10T01:13:39.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.054+0000 7ff9c6ffd700 1 -- 172.21.15.7:0/2187988980 learned_addr learned my addr 172.21.15.7:0/2187988980 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:39.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.054+0000 7ff9c77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649c61bdf20 0x5649c61bd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60368/0 (socket says 172.21.15.7:60368) 2024-06-10T01:13:39.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.055+0000 7ff9c7fff700 1 --2- 172.21.15.7:0/2187988980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649c61c3f60 0x5649c61b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:39.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.055+0000 7ff9c77fe700 1 -- 172.21.15.7:0/2187988980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649c61c3f60 msgr2=0x5649c61b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:39.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.055+0000 7ff9c77fe700 1 --2- 172.21.15.7:0/2187988980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649c61c3f60 0x5649c61b1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.055+0000 7ff9c77fe700 1 -- 172.21.15.7:0/2187988980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c61c3640 msgr2=0x5649c61b89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:39.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.055+0000 7ff9c77fe700 1 --2- 172.21.15.7:0/2187988980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c61c3640 0x5649c61b89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.055+0000 7ff9c77fe700 1 -- 172.21.15.7:0/2187988980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649c61c5540 con 0x5649c61bdf20 2024-06-10T01:13:39.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.055+0000 7ff9c6ffd700 1 --2- 172.21.15.7:0/2187988980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c61c3640 0x5649c61b89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:39.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.055+0000 7ff9c77fe700 1 --2- 172.21.15.7:0/2187988980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649c61bdf20 0x5649c61bd9e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800cfd0 tx=0x7ff9b80ccbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68350d0cc504de3f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:39.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.055+0000 7ff9ad7fa700 1 -- 172.21.15.7:0/2187988980 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b80b3070 con 0x5649c61bdf20 2024-06-10T01:13:39.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.055+0000 7ff9ad7fa700 1 -- 172.21.15.7:0/2187988980 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9b800cd60 con 0x5649c61bdf20 2024-06-10T01:13:39.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.055+0000 7ff9ad7fa700 1 -- 172.21.15.7:0/2187988980 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8010980 con 0x5649c61bdf20 2024-06-10T01:13:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.056+0000 7ff9d831db80 1 -- 172.21.15.7:0/2187988980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649c61bdf20 msgr2=0x5649c61bd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.056+0000 7ff9d831db80 1 --2- 172.21.15.7:0/2187988980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649c61bdf20 0x5649c61bd9e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800cfd0 tx=0x7ff9b80ccbc0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.056+0000 7ff9d831db80 1 -- 172.21.15.7:0/2187988980 shutdown_connections 2024-06-10T01:13:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.056+0000 7ff9d831db80 1 --2- 172.21.15.7:0/2187988980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649c61c3f60 0x5649c61b1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.056+0000 7ff9d831db80 1 --2- 172.21.15.7:0/2187988980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649c61bdf20 0x5649c61bd9e0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.056+0000 7ff9d831db80 1 --2- 172.21.15.7:0/2187988980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c61c3640 0x5649c61b89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.056+0000 7ff9d831db80 1 -- 172.21.15.7:0/2187988980 >> 172.21.15.7:0/2187988980 conn(0x5649c60e8cc0 msgr2=0x5649c61c6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.056+0000 7ff9d831db80 1 -- 172.21.15.7:0/2187988980 shutdown_connections 2024-06-10T01:13:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.056+0000 7ff9d831db80 1 -- 172.21.15.7:0/2187988980 wait complete. 2024-06-10T01:13:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.056+0000 7ff9d831db80 1 -- start start 2024-06-10T01:13:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.057+0000 7ff9d831db80 1 Processor -- start 2024-06-10T01:13:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.057+0000 7ff9d831db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649c61bdf20 0x5649c61d0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.057+0000 7ff9d831db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649c61c3640 0x5649c61d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.057+0000 7ff9d831db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c61c3f60 0x5649c61dec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.057+0000 7ff9d831db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c81aeb30 con 0x5649c61c3640 2024-06-10T01:13:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.057+0000 7ff9d831db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c81affa0 con 0x5649c61c3f60 2024-06-10T01:13:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.057+0000 7ff9d831db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c81af200 con 0x5649c61bdf20 2024-06-10T01:13:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.057+0000 7ff9d831db80 10 client.0 _ll_register_callbacks cb 0x5649c60a9920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:13:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.057+0000 7ff9c6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649c61c3640 0x5649c61d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.057+0000 7ff9c6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649c61c3640 0x5649c61d0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60374/0 (socket says 172.21.15.7:60374) 2024-06-10T01:13:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.057+0000 7ff9c77fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649c61bdf20 0x5649c61d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.057+0000 7ff9c6ffd700 1 -- 172.21.15.7:0/2786006772 learned_addr learned my addr 172.21.15.7:0/2786006772 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.057+0000 7ff9c7fff700 1 --2- 172.21.15.7:0/2786006772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c61c3f60 0x5649c61dec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.057+0000 7ff9c6ffd700 1 -- 172.21.15.7:0/2786006772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649c61bdf20 msgr2=0x5649c61d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.057+0000 7ff9c6ffd700 1 --2- 172.21.15.7:0/2786006772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649c61bdf20 0x5649c61d0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.057+0000 7ff9c6ffd700 1 -- 172.21.15.7:0/2786006772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c61c3f60 msgr2=0x5649c61dec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:39.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.057+0000 7ff9c6ffd700 1 --2- 172.21.15.7:0/2786006772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c61c3f60 0x5649c61dec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.057+0000 7ff9c6ffd700 1 -- 172.21.15.7:0/2786006772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649c61df490 con 0x5649c61c3640 2024-06-10T01:13:39.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.058+0000 7ff9c77fe700 1 --2- 172.21.15.7:0/2786006772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649c61bdf20 0x5649c61d0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:39.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.058+0000 7ff9c6ffd700 1 --2- 172.21.15.7:0/2786006772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649c61c3640 0x5649c61d0680 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009990 tx=0x7ff9bc00b2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84bc9edfe3d49ecd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:39.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.058+0000 7ff9a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:13:39.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.058+0000 7ff9a6ffd700 1 -- 172.21.15.7:0/2786006772 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc0223e0 con 0x5649c61c3640 2024-06-10T01:13:39.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.058+0000 7ff9d831db80 1 -- 172.21.15.7:0/2786006772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9b800cd60 con 0x5649c61c3640 2024-06-10T01:13:39.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.058+0000 7ff9a6ffd700 1 -- 172.21.15.7:0/2786006772 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7ff9bc022580 con 0x5649c61c3640 2024-06-10T01:13:39.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.058+0000 7ff9a6ffd700 1 -- 172.21.15.7:0/2786006772 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff9bc04e050 con 0x5649c61c3640 2024-06-10T01:13:39.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.058+0000 7ff9d831db80 10 client.17352 fetch_fsmap learned FSMap version 35 2024-06-10T01:13:39.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.058+0000 7ff9d831db80 1 -- 172.21.15.7:0/2786006772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5649c62421a0 con 0x5649c61c3640 2024-06-10T01:13:39.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.058+0000 7ff9a6ffd700 1 -- 172.21.15.7:0/2786006772 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9b800cd60 con 0x5649c61c3640 2024-06-10T01:13:39.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.058+0000 7ff9a6ffd700 1 -- 172.21.15.7:0/2786006772 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7ff9bc00f690 con 0x5649c61c3640 2024-06-10T01:13:39.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.059+0000 7ff9d831db80 10 client.17352 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:13:39.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.059+0000 7ff9d831db80 10 client.17352 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:13:39.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.059+0000 7ff9d831db80 20 client.17352 populate_metadata read hostname 'smithi007' 2024-06-10T01:13:39.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.059+0000 7ff9d831db80 1 --2- 172.21.15.7:0/2786006772 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5649c6242870 0x5649c6244c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:39.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.059+0000 7ff9d831db80 4 client.17352 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:13:39.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.059+0000 7ff9d831db80 1 -- 172.21.15.7:0/2786006772 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5649c6117290 con 0x5649c6242870 2024-06-10T01:13:39.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.059+0000 7ff9c77fe700 1 --2- 172.21.15.7:0/2786006772 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5649c6242870 0x5649c6244c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:39.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.059+0000 7ff9c77fe700 1 --2- 172.21.15.7:0/2786006772 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5649c6242870 0x5649c6244c90 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fbc292c75760e91 server_cookie=4eaeace7a0f9c8b9 in_seq=0 out_seq=0 2024-06-10T01:13:39.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.059+0000 7ff9a6ffd700 10 client.17352 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:39.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.059+0000 7ff9a6ffd700 1 -- 172.21.15.7:0/2786006772 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5649c6117290 con 0x5649c6242870 2024-06-10T01:13:39.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.059+0000 7ff9a6ffd700 10 client.17352 handle_command_reply: tid=0 2024-06-10T01:13:39.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.059+0000 7ff9d831db80 1 client.17352 shutdown 2024-06-10T01:13:39.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.060+0000 7ff9d831db80 1 -- 172.21.15.7:0/2786006772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649c61c3640 msgr2=0x5649c61d0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:39.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.060+0000 7ff9d831db80 1 --2- 172.21.15.7:0/2786006772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649c61c3640 0x5649c61d0680 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009990 tx=0x7ff9bc00b2a0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.060+0000 7ff9d831db80 1 -- 172.21.15.7:0/2786006772 shutdown_connections 2024-06-10T01:13:39.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.060+0000 7ff9d831db80 1 --2- 172.21.15.7:0/2786006772 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5649c6242870 0x5649c6244c90 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.060+0000 7ff9d831db80 1 --2- 172.21.15.7:0/2786006772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c61c3f60 0x5649c61dec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.060+0000 7ff9d831db80 1 --2- 172.21.15.7:0/2786006772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649c61bdf20 0x5649c61d0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.060+0000 7ff9d831db80 1 --2- 172.21.15.7:0/2786006772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649c61c3640 0x5649c61d0680 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.060+0000 7ff9d831db80 1 -- 172.21.15.7:0/2786006772 >> 172.21.15.7:0/2786006772 conn(0x5649c60e8cc0 msgr2=0x5649c61b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:39.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.060+0000 7ff9a6ffd700 0 client.17352 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:39.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.060+0000 7ff9d831db80 1 -- 172.21.15.7:0/2786006772 shutdown_connections 2024-06-10T01:13:39.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.060+0000 7ff9d831db80 1 -- 172.21.15.7:0/2786006772 wait complete. 2024-06-10T01:13:39.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.060+0000 7ff9d831db80 20 client.17352 trim_cache size 0 max 16384 2024-06-10T01:13:39.059 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:13:39.059 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-10T01:13:39.060 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-10T01:13:39.060 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:13:39.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.060+0000 7ff9cda9a700 1 -- 172.21.15.7:0/3792553276 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff9b00a2c20 msgr2=0x7ff9b00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:39.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.060+0000 7ff9cda9a700 1 --2- 172.21.15.7:0/3792553276 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff9b00a2c20 0x7ff9b00a50e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8106210 tx=0x7ff9bc005fd0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.061+0000 7ff9cda9a700 1 -- 172.21.15.7:0/3792553276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9c8101380 msgr2=0x7ff9c8103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:39.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.061+0000 7ff9cda9a700 1 --2- 172.21.15.7:0/3792553276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9c8101380 0x7ff9c8103060 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800e9a0 tx=0x7ff9b800ecb0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.061+0000 7ff9cda9a700 1 -- 172.21.15.7:0/3792553276 shutdown_connections 2024-06-10T01:13:39.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.061+0000 7ff9cda9a700 1 --2- 172.21.15.7:0/3792553276 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff9b00a2c20 0x7ff9b00a50e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.061+0000 7ff9cda9a700 1 --2- 172.21.15.7:0/3792553276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9c81073f0 0x7ff9c8103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.061+0000 7ff9cda9a700 1 --2- 172.21.15.7:0/3792553276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9c8101380 0x7ff9c8103060 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.061+0000 7ff9cda9a700 1 --2- 172.21.15.7:0/3792553276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9c8101d40 0x7ff9c81035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:39.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.061+0000 7ff9cda9a700 1 -- 172.21.15.7:0/3792553276 >> 172.21.15.7:0/3792553276 conn(0x7ff9c8078300 msgr2=0x7ff9c81005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:39.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.061+0000 7ff9cda9a700 1 -- 172.21.15.7:0/3792553276 shutdown_connections 2024-06-10T01:13:39.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:39.061+0000 7ff9cda9a700 1 -- 172.21.15.7:0/3792553276 wait complete. 2024-06-10T01:13:39.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:39 smithi175 ceph-mon[36636]: pgmap v661: 129 pgs: 129 active+clean; 101 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 4.3 MiB/s wr, 347 op/s 2024-06-10T01:13:39.738 INFO:tasks.workunit.client.0.smithi007.stdout: 1 546716 7.78 MB/sec execute 260 sec latency 54.408 ms 2024-06-10T01:13:39.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:39 smithi007 ceph-mon[32398]: pgmap v661: 129 pgs: 129 active+clean; 101 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 4.3 MiB/s wr, 347 op/s 2024-06-10T01:13:39.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:39 smithi078 ceph-mon[36683]: pgmap v661: 129 pgs: 129 active+clean; 101 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 4.3 MiB/s wr, 347 op/s 2024-06-10T01:13:40.018 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:13:40.018 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:13:40.274 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:13:40.738 INFO:tasks.workunit.client.0.smithi007.stdout: 1 548254 7.78 MB/sec execute 261 sec latency 72.845 ms 2024-06-10T01:13:41.649 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:41 smithi007 ceph-mon[32398]: pgmap v662: 129 pgs: 129 active+clean; 100 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 4.5 MiB/s wr, 352 op/s 2024-06-10T01:13:41.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:41 smithi175 ceph-mon[36636]: pgmap v662: 129 pgs: 129 active+clean; 100 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 4.5 MiB/s wr, 352 op/s 2024-06-10T01:13:41.738 INFO:tasks.workunit.client.0.smithi007.stdout: 1 550577 7.81 MB/sec execute 262 sec latency 93.997 ms 2024-06-10T01:13:41.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:41 smithi078 ceph-mon[36683]: pgmap v662: 129 pgs: 129 active+clean; 100 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 4.5 MiB/s wr, 352 op/s 2024-06-10T01:13:42.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.077+0000 7f0041ca0700 1 -- 172.21.15.7:0/568867162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f003c107550 msgr2=0x7f003c107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:42.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.077+0000 7f0041ca0700 1 --2- 172.21.15.7:0/568867162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f003c107550 0x7f003c107930 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f002c009990 tx=0x7f002c009ca0 comp rx=0 tx=0).stop 2024-06-10T01:13:42.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.077+0000 7f0041ca0700 1 -- 172.21.15.7:0/568867162 shutdown_connections 2024-06-10T01:13:42.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.077+0000 7f0041ca0700 1 --2- 172.21.15.7:0/568867162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f003c1017f0 0x7f003c10f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:42.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.077+0000 7f0041ca0700 1 --2- 172.21.15.7:0/568867162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f003c107550 0x7f003c107930 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:42.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.077+0000 7f0041ca0700 1 --2- 172.21.15.7:0/568867162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f003c100e30 0x7f003c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:42.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.077+0000 7f0041ca0700 1 -- 172.21.15.7:0/568867162 >> 172.21.15.7:0/568867162 conn(0x7f003c0766b0 msgr2=0x7f003c076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:42.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.077+0000 7f0041ca0700 1 -- 172.21.15.7:0/568867162 shutdown_connections 2024-06-10T01:13:42.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.077+0000 7f0041ca0700 1 -- 172.21.15.7:0/568867162 wait complete. 2024-06-10T01:13:42.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.078+0000 7f0041ca0700 1 Processor -- start 2024-06-10T01:13:42.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.078+0000 7f0041ca0700 1 -- start start 2024-06-10T01:13:42.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.078+0000 7f0041ca0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f003c100e30 0x7f003c19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:42.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.078+0000 7f0041ca0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f003c1017f0 0x7f003c19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:42.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.078+0000 7f0041ca0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f003c107550 0x7f003c198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:42.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.078+0000 7f0041ca0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f003c112790 con 0x7f003c100e30 2024-06-10T01:13:42.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.078+0000 7f0041ca0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f003c112610 con 0x7f003c1017f0 2024-06-10T01:13:42.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.078+0000 7f0041ca0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f003c112910 con 0x7f003c107550 2024-06-10T01:13:42.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.078+0000 7f003b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f003c100e30 0x7f003c19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:42.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.078+0000 7f003b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f003c100e30 0x7f003c19e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41426/0 (socket says 172.21.15.7:41426) 2024-06-10T01:13:42.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.078+0000 7f003b7fe700 1 -- 172.21.15.7:0/3116337122 learned_addr learned my addr 172.21.15.7:0/3116337122 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:42.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.078+0000 7f003bfff700 1 --2- 172.21.15.7:0/3116337122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f003c107550 0x7f003c198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:42.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.078+0000 7f003affd700 1 --2- 172.21.15.7:0/3116337122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f003c1017f0 0x7f003c19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:42.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.079+0000 7f003b7fe700 1 -- 172.21.15.7:0/3116337122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f003c107550 msgr2=0x7f003c198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:42.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.079+0000 7f003b7fe700 1 --2- 172.21.15.7:0/3116337122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f003c107550 0x7f003c198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:42.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.079+0000 7f003b7fe700 1 -- 172.21.15.7:0/3116337122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f003c1017f0 msgr2=0x7f003c19edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:42.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.079+0000 7f003b7fe700 1 --2- 172.21.15.7:0/3116337122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f003c1017f0 0x7f003c19edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:42.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.079+0000 7f003b7fe700 1 -- 172.21.15.7:0/3116337122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f003c199190 con 0x7f003c100e30 2024-06-10T01:13:42.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.079+0000 7f003bfff700 1 --2- 172.21.15.7:0/3116337122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f003c107550 0x7f003c198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:42.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.079+0000 7f003b7fe700 1 --2- 172.21.15.7:0/3116337122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f003c100e30 0x7f003c19e870 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f002c000c00 tx=0x7f002c00bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:42.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.079+0000 7f0038ff9700 1 -- 172.21.15.7:0/3116337122 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c026070 con 0x7f003c100e30 2024-06-10T01:13:42.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.079+0000 7f0041ca0700 1 -- 172.21.15.7:0/3116337122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f003c199420 con 0x7f003c100e30 2024-06-10T01:13:42.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.079+0000 7f0038ff9700 1 -- 172.21.15.7:0/3116337122 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f002c003d70 con 0x7f003c100e30 2024-06-10T01:13:42.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.079+0000 7f0038ff9700 1 -- 172.21.15.7:0/3116337122 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c00fdf0 con 0x7f003c100e30 2024-06-10T01:13:42.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.079+0000 7f0041ca0700 1 -- 172.21.15.7:0/3116337122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f003c1a5740 con 0x7f003c100e30 2024-06-10T01:13:42.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.080+0000 7f0041ca0700 1 -- 172.21.15.7:0/3116337122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f003c069180 con 0x7f003c100e30 2024-06-10T01:13:42.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.081+0000 7f0038ff9700 1 -- 172.21.15.7:0/3116337122 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f002c029070 con 0x7f003c100e30 2024-06-10T01:13:42.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.082+0000 7f0038ff9700 1 --2- 172.21.15.7:0/3116337122 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f00240a2c20 0x7f00240a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:42.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.082+0000 7f003affd700 1 --2- 172.21.15.7:0/3116337122 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f00240a2c20 0x7f00240a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:42.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.082+0000 7f0038ff9700 1 -- 172.21.15.7:0/3116337122 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f002c0d0670 con 0x7f003c100e30 2024-06-10T01:13:42.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.083+0000 7f003affd700 1 --2- 172.21.15.7:0/3116337122 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f00240a2c20 0x7f00240a50e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f003c076c40 tx=0x7f0030009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:42.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.086+0000 7f0038ff9700 1 -- 172.21.15.7:0/3116337122 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f002c030080 con 0x7f003c100e30 2024-06-10T01:13:42.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.347+0000 7f0041ca0700 1 -- 172.21.15.7:0/3116337122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f003c02dca0 con 0x7f003c100e30 2024-06-10T01:13:42.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.348+0000 7f0038ff9700 1 -- 172.21.15.7:0/3116337122 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11859 (secure 0 0 0) 0x7f002c013070 con 0x7f003c100e30 2024-06-10T01:13:42.343 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:13:42.343 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:13:22.220710+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:13:42.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.351+0000 7f0041ca0700 1 -- 172.21.15.7:0/3116337122 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f00240a2c20 msgr2=0x7f00240a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:42.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.351+0000 7f0041ca0700 1 --2- 172.21.15.7:0/3116337122 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f00240a2c20 0x7f00240a50e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f003c076c40 tx=0x7f0030009260 comp rx=0 tx=0).stop 2024-06-10T01:13:42.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.352+0000 7f0041ca0700 1 -- 172.21.15.7:0/3116337122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f003c100e30 msgr2=0x7f003c19e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:42.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.352+0000 7f0041ca0700 1 --2- 172.21.15.7:0/3116337122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f003c100e30 0x7f003c19e870 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f002c000c00 tx=0x7f002c00bb30 comp rx=0 tx=0).stop 2024-06-10T01:13:42.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.352+0000 7f0041ca0700 1 -- 172.21.15.7:0/3116337122 shutdown_connections 2024-06-10T01:13:42.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.352+0000 7f0041ca0700 1 --2- 172.21.15.7:0/3116337122 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f00240a2c20 0x7f00240a50e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:42.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.352+0000 7f0041ca0700 1 --2- 172.21.15.7:0/3116337122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f003c107550 0x7f003c198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:42.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.352+0000 7f0041ca0700 1 --2- 172.21.15.7:0/3116337122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f003c100e30 0x7f003c19e870 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:42.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.352+0000 7f0041ca0700 1 --2- 172.21.15.7:0/3116337122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f003c1017f0 0x7f003c19edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:42.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.352+0000 7f0041ca0700 1 -- 172.21.15.7:0/3116337122 >> 172.21.15.7:0/3116337122 conn(0x7f003c0766b0 msgr2=0x7f003c0ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:42.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.352+0000 7f0041ca0700 1 -- 172.21.15.7:0/3116337122 shutdown_connections 2024-06-10T01:13:42.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:42.352+0000 7f0041ca0700 1 -- 172.21.15.7:0/3116337122 wait complete. 2024-06-10T01:13:42.347 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 35 2024-06-10T01:13:42.643 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:42 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3116337122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:13:42.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:42 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3116337122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:13:42.738 INFO:tasks.workunit.client.0.smithi007.stdout: 1 552143 7.81 MB/sec execute 263 sec latency 17.533 ms 2024-06-10T01:13:42.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:42 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3116337122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:13:42.898 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:13:43.158 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:13:43.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:43 smithi007 ceph-mon[32398]: pgmap v663: 129 pgs: 129 active+clean; 102 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 370 op/s 2024-06-10T01:13:43.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:43 smithi175 ceph-mon[36636]: pgmap v663: 129 pgs: 129 active+clean; 102 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 370 op/s 2024-06-10T01:13:43.738 INFO:tasks.workunit.client.0.smithi007.stdout: 1 554521 7.83 MB/sec execute 264 sec latency 12.669 ms 2024-06-10T01:13:43.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:43 smithi078 ceph-mon[36683]: pgmap v663: 129 pgs: 129 active+clean; 102 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 370 op/s 2024-06-10T01:13:44.738 INFO:tasks.workunit.client.0.smithi007.stdout: 1 556713 7.83 MB/sec execute 265 sec latency 16.838 ms 2024-06-10T01:13:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.840+0000 7f59c6cef700 1 -- 172.21.15.7:0/3336341512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59c0106720 msgr2=0x7f59c010e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.840+0000 7f59c6cef700 1 --2- 172.21.15.7:0/3336341512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59c0106720 0x7f59c010e2c0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f59b800b0d0 tx=0x7f59b800b3e0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.840+0000 7f59c6cef700 1 -- 172.21.15.7:0/3336341512 shutdown_connections 2024-06-10T01:13:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.840+0000 7f59c6cef700 1 --2- 172.21.15.7:0/3336341512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59c0106720 0x7f59c010e2c0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.840+0000 7f59c6cef700 1 --2- 172.21.15.7:0/3336341512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59c01019d0 0x7f59c0101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.840+0000 7f59c6cef700 1 --2- 172.21.15.7:0/3336341512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c0102380 0x7f59c0105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.840+0000 7f59c6cef700 1 -- 172.21.15.7:0/3336341512 >> 172.21.15.7:0/3336341512 conn(0x7f59c0078340 msgr2=0x7f59c0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.840+0000 7f59c6cef700 1 -- 172.21.15.7:0/3336341512 shutdown_connections 2024-06-10T01:13:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.840+0000 7f59c6cef700 1 -- 172.21.15.7:0/3336341512 wait complete. 2024-06-10T01:13:44.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.841+0000 7f59c6cef700 1 Processor -- start 2024-06-10T01:13:44.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.841+0000 7f59c6cef700 1 -- start start 2024-06-10T01:13:44.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.841+0000 7f59c6cef700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59c01019d0 0x7f59c019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:44.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.841+0000 7f59c6cef700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c0102380 0x7f59c019edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:44.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.841+0000 7f59c6cef700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59c0106720 0x7f59c0198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:44.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.841+0000 7f59c6cef700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f59c0111700 con 0x7f59c0106720 2024-06-10T01:13:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.841+0000 7f59c6cef700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f59c0111580 con 0x7f59c01019d0 2024-06-10T01:13:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.841+0000 7f59c6cef700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f59c0111880 con 0x7f59c0102380 2024-06-10T01:13:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.841+0000 7f59bffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c0102380 0x7f59c019edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.841+0000 7f59c4a8b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59c01019d0 0x7f59c019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.841+0000 7f59bffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c0102380 0x7f59c019edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34312/0 (socket says 172.21.15.7:34312) 2024-06-10T01:13:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.841+0000 7f59bffff700 1 -- 172.21.15.7:0/1178860731 learned_addr learned my addr 172.21.15.7:0/1178860731 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.841+0000 7f59c4a8b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59c01019d0 0x7f59c019e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:47072/0 (socket says 172.21.15.7:47072) 2024-06-10T01:13:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.842+0000 7f59bffff700 1 -- 172.21.15.7:0/1178860731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59c01019d0 msgr2=0x7f59c019e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.842+0000 7f59bffff700 1 --2- 172.21.15.7:0/1178860731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59c01019d0 0x7f59c019e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.842+0000 7f59bffff700 1 -- 172.21.15.7:0/1178860731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59c0106720 msgr2=0x7f59c0198a50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T01:13:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.842+0000 7f59bffff700 1 --2- 172.21.15.7:0/1178860731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59c0106720 0x7f59c0198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.842+0000 7f59bffff700 1 -- 172.21.15.7:0/1178860731 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c01991b0 con 0x7f59c0102380 2024-06-10T01:13:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.842+0000 7f59bffff700 1 --2- 172.21.15.7:0/1178860731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c0102380 0x7f59c019edd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f59ac00e9c0 tx=0x7f59ac00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:44.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.842+0000 7f59bdffb700 1 -- 172.21.15.7:0/1178860731 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac00cbd0 con 0x7f59c0102380 2024-06-10T01:13:44.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.842+0000 7f59bdffb700 1 -- 172.21.15.7:0/1178860731 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59ac00cd70 con 0x7f59c0102380 2024-06-10T01:13:44.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.842+0000 7f59bdffb700 1 -- 172.21.15.7:0/1178860731 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac0056e0 con 0x7f59c0102380 2024-06-10T01:13:44.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.842+0000 7f59c6cef700 1 -- 172.21.15.7:0/1178860731 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c01994a0 con 0x7f59c0102380 2024-06-10T01:13:44.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.843+0000 7f59c6cef700 1 -- 172.21.15.7:0/1178860731 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c01a5760 con 0x7f59c0102380 2024-06-10T01:13:44.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.845+0000 7f59bdffb700 1 -- 172.21.15.7:0/1178860731 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f59ac004750 con 0x7f59c0102380 2024-06-10T01:13:44.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.845+0000 7f59bdffb700 1 --2- 172.21.15.7:0/1178860731 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f59b00a2ca0 0x7f59b00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:44.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.846+0000 7f59c4a8b700 1 --2- 172.21.15.7:0/1178860731 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f59b00a2ca0 0x7f59b00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:44.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.846+0000 7f59bdffb700 1 -- 172.21.15.7:0/1178860731 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f59ac0cf580 con 0x7f59c0102380 2024-06-10T01:13:44.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.846+0000 7f59c4a8b700 1 --2- 172.21.15.7:0/1178860731 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f59b00a2ca0 0x7f59b00a5160 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f59c0199b90 tx=0x7f59b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:44.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.846+0000 7f59d1572b80 1 Processor -- start 2024-06-10T01:13:44.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.847+0000 7f59d1572b80 1 -- start start 2024-06-10T01:13:44.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.847+0000 7f59d1572b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9ee4dcfd0 0x55a9ee4dd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:44.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.847+0000 7f59d1572b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a9ee4dd8f0 0x55a9ee4e16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:44.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.847+0000 7f59d1572b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a9ee4e1c00 0x55a9ee4e2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:44.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.847+0000 7f59d1572b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a9ee42a810 con 0x55a9ee4dcfd0 2024-06-10T01:13:44.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.847+0000 7f59d1572b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a9ee3d2720 con 0x55a9ee4e1c00 2024-06-10T01:13:44.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.847+0000 7f59d1572b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a9ee4267d0 con 0x55a9ee4dd8f0 2024-06-10T01:13:44.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.847+0000 7f59c4a8b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9ee4dcfd0 0x55a9ee4dd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:44.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.847+0000 7f59c4a8b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9ee4dcfd0 0x55a9ee4dd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41452/0 (socket says 172.21.15.7:41452) 2024-06-10T01:13:44.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.847+0000 7f59c528c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a9ee4e1c00 0x55a9ee4e2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:44.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.847+0000 7f59c4a8b700 1 -- 172.21.15.7:0/3765677444 learned_addr learned my addr 172.21.15.7:0/3765677444 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:44.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.847+0000 7f59bffff700 1 --2- 172.21.15.7:0/3765677444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a9ee4dd8f0 0x55a9ee4e16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:44.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.847+0000 7f59c4a8b700 1 -- 172.21.15.7:0/3765677444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a9ee4dd8f0 msgr2=0x55a9ee4e16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:44.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.847+0000 7f59c4a8b700 1 --2- 172.21.15.7:0/3765677444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a9ee4dd8f0 0x55a9ee4e16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.847+0000 7f59c4a8b700 1 -- 172.21.15.7:0/3765677444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a9ee4e1c00 msgr2=0x55a9ee4e2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:44.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.847+0000 7f59c4a8b700 1 --2- 172.21.15.7:0/3765677444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a9ee4e1c00 0x55a9ee4e2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.847+0000 7f59c4a8b700 1 -- 172.21.15.7:0/3765677444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9ee3c8f50 con 0x55a9ee4dcfd0 2024-06-10T01:13:44.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.847+0000 7f59c528c700 1 --2- 172.21.15.7:0/3765677444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a9ee4e1c00 0x55a9ee4e2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:44.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.848+0000 7f59c4a8b700 1 --2- 172.21.15.7:0/3765677444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9ee4dcfd0 0x55a9ee4dd3b0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f59b4011d50 tx=0x7f59b4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca664136d4f472d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:44.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.848+0000 7f59aaffd700 1 -- 172.21.15.7:0/3765677444 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b400eb20 con 0x55a9ee4dcfd0 2024-06-10T01:13:44.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.848+0000 7f59aaffd700 1 -- 172.21.15.7:0/3765677444 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59b4004500 con 0x55a9ee4dcfd0 2024-06-10T01:13:44.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.848+0000 7f59aaffd700 1 -- 172.21.15.7:0/3765677444 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b4026480 con 0x55a9ee4dcfd0 2024-06-10T01:13:44.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.848+0000 7f59d1572b80 1 -- 172.21.15.7:0/3765677444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9ee4dcfd0 msgr2=0x55a9ee4dd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:44.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.848+0000 7f59d1572b80 1 --2- 172.21.15.7:0/3765677444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9ee4dcfd0 0x55a9ee4dd3b0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f59b4011d50 tx=0x7f59b4014000 comp rx=0 tx=0).stop 2024-06-10T01:13:44.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.849+0000 7f59d1572b80 1 -- 172.21.15.7:0/3765677444 shutdown_connections 2024-06-10T01:13:44.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.849+0000 7f59d1572b80 1 --2- 172.21.15.7:0/3765677444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a9ee4e1c00 0x55a9ee4e2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.849+0000 7f59d1572b80 1 --2- 172.21.15.7:0/3765677444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9ee4dcfd0 0x55a9ee4dd3b0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.849+0000 7f59d1572b80 1 --2- 172.21.15.7:0/3765677444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a9ee4dd8f0 0x55a9ee4e16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.849+0000 7f59d1572b80 1 -- 172.21.15.7:0/3765677444 >> 172.21.15.7:0/3765677444 conn(0x55a9ee403cc0 msgr2=0x55a9ee4d3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:44.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.849+0000 7f59d1572b80 1 -- 172.21.15.7:0/3765677444 shutdown_connections 2024-06-10T01:13:44.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.849+0000 7f59d1572b80 1 -- 172.21.15.7:0/3765677444 wait complete. 2024-06-10T01:13:44.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.849+0000 7f59d1572b80 1 -- start start 2024-06-10T01:13:44.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.849+0000 7f59d1572b80 1 Processor -- start 2024-06-10T01:13:44.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.849+0000 7f59d1572b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9ee4dcfd0 0x55a9ee4e9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:44.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.849+0000 7f59d1572b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a9ee4dd8f0 0x55a9ee4e6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:44.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.849+0000 7f59d1572b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a9ee4e1c00 0x55a9ee4e69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:44.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.849+0000 7f59d1572b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a9ee4e9dc0 con 0x55a9ee4dcfd0 2024-06-10T01:13:44.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.849+0000 7f59d1572b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a9ee3cabf0 con 0x55a9ee4dd8f0 2024-06-10T01:13:44.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.849+0000 7f59d1572b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a9ee4e9f40 con 0x55a9ee4e1c00 2024-06-10T01:13:44.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.850+0000 7f59c4a8b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9ee4dcfd0 0x55a9ee4e9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:44.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.850+0000 7f59d1572b80 10 client.0 _ll_register_callbacks cb 0x55a9ee3c4920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:13:44.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.850+0000 7f59bffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a9ee4dd8f0 0x55a9ee4e6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:44.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.850+0000 7f59c4a8b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9ee4dcfd0 0x55a9ee4e9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41456/0 (socket says 172.21.15.7:41456) 2024-06-10T01:13:44.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.850+0000 7f59c4a8b700 1 -- 172.21.15.7:0/2291261843 learned_addr learned my addr 172.21.15.7:0/2291261843 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:44.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.850+0000 7f59c528c700 1 --2- 172.21.15.7:0/2291261843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a9ee4e1c00 0x55a9ee4e69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:44.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.850+0000 7f59c4a8b700 1 -- 172.21.15.7:0/2291261843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a9ee4e1c00 msgr2=0x55a9ee4e69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:44.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.850+0000 7f59c4a8b700 1 --2- 172.21.15.7:0/2291261843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a9ee4e1c00 0x55a9ee4e69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.850+0000 7f59c4a8b700 1 -- 172.21.15.7:0/2291261843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a9ee4dd8f0 msgr2=0x55a9ee4e6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:44.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.850+0000 7f59c4a8b700 1 --2- 172.21.15.7:0/2291261843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a9ee4dd8f0 0x55a9ee4e6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.850+0000 7f59c4a8b700 1 -- 172.21.15.7:0/2291261843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9ee3798d0 con 0x55a9ee4dcfd0 2024-06-10T01:13:44.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.850+0000 7f59bffff700 1 --2- 172.21.15.7:0/2291261843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a9ee4dd8f0 0x55a9ee4e6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:44.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.850+0000 7f59c4a8b700 1 --2- 172.21.15.7:0/2291261843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9ee4dcfd0 0x55a9ee4e9170 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f59b4005760 tx=0x7f59b4005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f5264843748ef1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:44.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.850+0000 7f599ffff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:13:44.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.850+0000 7f599ffff700 1 -- 172.21.15.7:0/2291261843 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b4003700 con 0x55a9ee4dcfd0 2024-06-10T01:13:44.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.850+0000 7f599ffff700 1 -- 172.21.15.7:0/2291261843 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f59b4003bc0 con 0x55a9ee4dcfd0 2024-06-10T01:13:44.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.850+0000 7f59d1572b80 1 -- 172.21.15.7:0/2291261843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59b4004500 con 0x55a9ee4dcfd0 2024-06-10T01:13:44.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.851+0000 7f599ffff700 1 -- 172.21.15.7:0/2291261843 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f59b4038050 con 0x55a9ee4dcfd0 2024-06-10T01:13:44.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.851+0000 7f59d1572b80 10 client.17367 fetch_fsmap learned FSMap version 35 2024-06-10T01:13:44.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.851+0000 7f59d1572b80 1 -- 172.21.15.7:0/2291261843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a9ee4f4710 con 0x55a9ee4dcfd0 2024-06-10T01:13:44.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.851+0000 7f599ffff700 1 -- 172.21.15.7:0/2291261843 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59b4004500 con 0x55a9ee4dcfd0 2024-06-10T01:13:44.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.851+0000 7f599ffff700 1 -- 172.21.15.7:0/2291261843 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f59b4026b30 con 0x55a9ee4dcfd0 2024-06-10T01:13:44.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.851+0000 7f59d1572b80 10 client.17367 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:13:44.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.851+0000 7f59d1572b80 10 client.17367 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:13:44.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.851+0000 7f59d1572b80 20 client.17367 populate_metadata read hostname 'smithi007' 2024-06-10T01:13:44.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.851+0000 7f59d1572b80 1 --2- 172.21.15.7:0/2291261843 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55a9ee4f6be0 0x55a9ee4f8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:44.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.851+0000 7f59d1572b80 4 client.17367 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:13:44.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.851+0000 7f59d1572b80 1 -- 172.21.15.7:0/2291261843 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9ee437d10 con 0x55a9ee4f6be0 2024-06-10T01:13:44.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.851+0000 7f59bffff700 1 --2- 172.21.15.7:0/2291261843 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55a9ee4f6be0 0x55a9ee4f8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:44.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.852+0000 7f59bffff700 1 --2- 172.21.15.7:0/2291261843 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55a9ee4f6be0 0x55a9ee4f8fd0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3bc597b563c07b93 server_cookie=8b6312027cdcd6be in_seq=0 out_seq=0 2024-06-10T01:13:44.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.852+0000 7f599ffff700 10 client.17367 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:13:44.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.853+0000 7f599ffff700 1 -- 172.21.15.7:0/2291261843 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a9ee437d10 con 0x55a9ee4f6be0 2024-06-10T01:13:44.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.853+0000 7f599ffff700 10 client.17367 handle_command_reply: tid=0 2024-06-10T01:13:44.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.853+0000 7f59d1572b80 1 client.17367 shutdown 2024-06-10T01:13:44.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.853+0000 7f59d1572b80 1 -- 172.21.15.7:0/2291261843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9ee4dcfd0 msgr2=0x55a9ee4e9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:44.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.853+0000 7f59d1572b80 1 --2- 172.21.15.7:0/2291261843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9ee4dcfd0 0x55a9ee4e9170 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f59b4005760 tx=0x7f59b4005790 comp rx=0 tx=0).stop 2024-06-10T01:13:44.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.853+0000 7f59d1572b80 1 -- 172.21.15.7:0/2291261843 shutdown_connections 2024-06-10T01:13:44.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.853+0000 7f59d1572b80 1 --2- 172.21.15.7:0/2291261843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a9ee4e1c00 0x55a9ee4e69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.853+0000 7f59d1572b80 1 --2- 172.21.15.7:0/2291261843 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55a9ee4f6be0 0x55a9ee4f8fd0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.853+0000 7f59d1572b80 1 --2- 172.21.15.7:0/2291261843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9ee4dcfd0 0x55a9ee4e9170 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.853+0000 7f599ffff700 0 client.17367 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:13:44.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.853+0000 7f59d1572b80 1 --2- 172.21.15.7:0/2291261843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a9ee4dd8f0 0x55a9ee4e6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.853+0000 7f59d1572b80 1 -- 172.21.15.7:0/2291261843 >> 172.21.15.7:0/2291261843 conn(0x55a9ee403cc0 msgr2=0x55a9ee4d79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.853+0000 7f59d1572b80 1 -- 172.21.15.7:0/2291261843 shutdown_connections 2024-06-10T01:13:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.853+0000 7f59d1572b80 1 -- 172.21.15.7:0/2291261843 wait complete. 2024-06-10T01:13:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.853+0000 7f59d1572b80 20 client.17367 trim_cache size 0 max 16384 2024-06-10T01:13:44.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.870+0000 7f59d1572b80 1 Processor -- start 2024-06-10T01:13:44.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.870+0000 7f59d1572b80 1 -- start start 2024-06-10T01:13:44.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.870+0000 7f59d1572b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9ee4dcfd0 0x55a9ee4e2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:44.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.870+0000 7f59d1572b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a9ee4dd8f0 0x55a9ee4cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:44.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.870+0000 7f59d1572b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a9ee4e1c00 0x55a9ee4cd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:44.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.870+0000 7f59d1572b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f59c01aea90 con 0x55a9ee4dcfd0 2024-06-10T01:13:44.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.870+0000 7f59d1572b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f59c01ac2c0 con 0x55a9ee4e1c00 2024-06-10T01:13:44.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.870+0000 7f59d1572b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f59c01aa110 con 0x55a9ee4dd8f0 2024-06-10T01:13:44.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.870+0000 7f59c4a8b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9ee4dcfd0 0x55a9ee4e2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:44.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.870+0000 7f59c528c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a9ee4e1c00 0x55a9ee4cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:44.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.870+0000 7f59c528c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a9ee4e1c00 0x55a9ee4cd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:47112/0 (socket says 172.21.15.7:47112) 2024-06-10T01:13:44.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.870+0000 7f59c4a8b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9ee4dcfd0 0x55a9ee4e2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41472/0 (socket says 172.21.15.7:41472) 2024-06-10T01:13:44.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.870+0000 7f59c528c700 1 -- 172.21.15.7:0/1244614316 learned_addr learned my addr 172.21.15.7:0/1244614316 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:44.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.870+0000 7f59bffff700 1 --2- 172.21.15.7:0/1244614316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a9ee4dd8f0 0x55a9ee4cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:44.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.871+0000 7f59c528c700 1 -- 172.21.15.7:0/1244614316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a9ee4dd8f0 msgr2=0x55a9ee4cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:44.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.871+0000 7f59c528c700 1 --2- 172.21.15.7:0/1244614316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a9ee4dd8f0 0x55a9ee4cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.871+0000 7f59c528c700 1 -- 172.21.15.7:0/1244614316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9ee4dcfd0 msgr2=0x55a9ee4e2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:44.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.871+0000 7f59c528c700 1 --2- 172.21.15.7:0/1244614316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9ee4dcfd0 0x55a9ee4e2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.871+0000 7f59c528c700 1 -- 172.21.15.7:0/1244614316 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9ee4e49e0 con 0x55a9ee4e1c00 2024-06-10T01:13:44.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.871+0000 7f59c4a8b700 1 --2- 172.21.15.7:0/1244614316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9ee4dcfd0 0x55a9ee4e2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:44.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.871+0000 7f59bffff700 1 --2- 172.21.15.7:0/1244614316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a9ee4dd8f0 0x55a9ee4cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:44.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.871+0000 7f59c528c700 1 --2- 172.21.15.7:0/1244614316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a9ee4e1c00 0x55a9ee4cd340 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f59b8009580 tx=0x7f59b8006210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c78fb009b6a11c6f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:44.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.871+0000 7f59aa7fc700 1 -- 172.21.15.7:0/1244614316 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b800bda0 con 0x55a9ee4e1c00 2024-06-10T01:13:44.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.871+0000 7f59aa7fc700 1 -- 172.21.15.7:0/1244614316 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59b80074e0 con 0x55a9ee4e1c00 2024-06-10T01:13:44.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.871+0000 7f59aa7fc700 1 -- 172.21.15.7:0/1244614316 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b8005260 con 0x55a9ee4e1c00 2024-06-10T01:13:44.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.872+0000 7f59d1572b80 1 -- 172.21.15.7:0/1244614316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a9ee4e1c00 msgr2=0x55a9ee4cd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:44.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.872+0000 7f59d1572b80 1 --2- 172.21.15.7:0/1244614316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a9ee4e1c00 0x55a9ee4cd340 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f59b8009580 tx=0x7f59b8006210 comp rx=0 tx=0).stop 2024-06-10T01:13:44.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.872+0000 7f59d1572b80 1 -- 172.21.15.7:0/1244614316 shutdown_connections 2024-06-10T01:13:44.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.872+0000 7f59d1572b80 1 --2- 172.21.15.7:0/1244614316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a9ee4e1c00 0x55a9ee4cd340 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.872+0000 7f59d1572b80 1 --2- 172.21.15.7:0/1244614316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9ee4dcfd0 0x55a9ee4e2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.872+0000 7f59d1572b80 1 --2- 172.21.15.7:0/1244614316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a9ee4dd8f0 0x55a9ee4cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.872+0000 7f59d1572b80 1 -- 172.21.15.7:0/1244614316 >> 172.21.15.7:0/1244614316 conn(0x55a9ee403cc0 msgr2=0x55a9ee4e3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:44.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.872+0000 7f59d1572b80 1 -- 172.21.15.7:0/1244614316 shutdown_connections 2024-06-10T01:13:44.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.872+0000 7f59d1572b80 1 -- 172.21.15.7:0/1244614316 wait complete. 2024-06-10T01:13:44.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.872+0000 7f59d1572b80 1 -- start start 2024-06-10T01:13:44.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.873+0000 7f59d1572b80 1 Processor -- start 2024-06-10T01:13:44.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.873+0000 7f59d1572b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a9ee4dcfd0 0x55a9ee4f75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:44.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.873+0000 7f59d1572b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a9ee4dd8f0 0x55a9ee4f7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:44.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.873+0000 7f59d1572b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9ee4e1c00 0x55a9ee55f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:44.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.873+0000 7f59d1572b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f59c01aa780 con 0x55a9ee4e1c00 2024-06-10T01:13:44.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.873+0000 7f59d1572b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f59c01abbf0 con 0x55a9ee4dcfd0 2024-06-10T01:13:44.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.873+0000 7f59d1572b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f59c01aae50 con 0x55a9ee4dd8f0 2024-06-10T01:13:44.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.873+0000 7f59d1572b80 10 client.0 _ll_register_callbacks cb 0x55a9ee3c4920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:13:44.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.873+0000 7f59c528c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9ee4e1c00 0x55a9ee55f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:44.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.873+0000 7f59c528c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9ee4e1c00 0x55a9ee55f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41488/0 (socket says 172.21.15.7:41488) 2024-06-10T01:13:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.873+0000 7f59c528c700 1 -- 172.21.15.7:0/1061126606 learned_addr learned my addr 172.21.15.7:0/1061126606 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.873+0000 7f59bffff700 1 --2- 172.21.15.7:0/1061126606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a9ee4dd8f0 0x55a9ee4f7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.873+0000 7f59c4a8b700 1 --2- 172.21.15.7:0/1061126606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a9ee4dcfd0 0x55a9ee4f75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.873+0000 7f59bffff700 1 -- 172.21.15.7:0/1061126606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a9ee4dcfd0 msgr2=0x55a9ee4f75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.873+0000 7f59bffff700 1 --2- 172.21.15.7:0/1061126606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a9ee4dcfd0 0x55a9ee4f75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.874+0000 7f59bffff700 1 -- 172.21.15.7:0/1061126606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9ee4e1c00 msgr2=0x55a9ee55f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.874+0000 7f59bffff700 1 --2- 172.21.15.7:0/1061126606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9ee4e1c00 0x55a9ee55f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.874+0000 7f59bffff700 1 -- 172.21.15.7:0/1061126606 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9ee570b50 con 0x55a9ee4dd8f0 2024-06-10T01:13:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.874+0000 7f59bffff700 1 --2- 172.21.15.7:0/1061126606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a9ee4dd8f0 0x55a9ee4f7ae0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f59ac00c5b0 tx=0x7f59ac086040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e81a1a4a731903c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.874+0000 7f599ffff700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:13:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.874+0000 7f599ffff700 1 -- 172.21.15.7:0/1061126606 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac0a4670 con 0x55a9ee4dd8f0 2024-06-10T01:13:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.874+0000 7f59d1572b80 1 -- 172.21.15.7:0/1061126606 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59b80074e0 con 0x55a9ee4dd8f0 2024-06-10T01:13:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.874+0000 7f599ffff700 1 -- 172.21.15.7:0/1061126606 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f59ac099d00 con 0x55a9ee4dd8f0 2024-06-10T01:13:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.874+0000 7f599ffff700 1 -- 172.21.15.7:0/1061126606 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f59ac09bc60 con 0x55a9ee4dd8f0 2024-06-10T01:13:44.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.874+0000 7f59d1572b80 10 client.26806 fetch_fsmap learned FSMap version 35 2024-06-10T01:13:44.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.874+0000 7f59d1572b80 1 -- 172.21.15.7:0/1061126606 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a9ee571e90 con 0x55a9ee4dd8f0 2024-06-10T01:13:44.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.874+0000 7f599ffff700 1 -- 172.21.15.7:0/1061126606 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59b80074e0 con 0x55a9ee4dd8f0 2024-06-10T01:13:44.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.875+0000 7f599ffff700 1 -- 172.21.15.7:0/1061126606 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f59ac09ae50 con 0x55a9ee4dd8f0 2024-06-10T01:13:44.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.875+0000 7f59d1572b80 10 client.26806 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:13:44.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.875+0000 7f59d1572b80 10 client.26806 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:13:44.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.875+0000 7f59d1572b80 20 client.26806 populate_metadata read hostname 'smithi007' 2024-06-10T01:13:44.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.875+0000 7f59d1572b80 1 --2- 172.21.15.7:0/1061126606 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55a9ee4f6be0 0x55a9ee5745c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:44.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.875+0000 7f59d1572b80 4 client.26806 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:13:44.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.875+0000 7f59d1572b80 1 -- 172.21.15.7:0/1061126606 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a9ee432290 con 0x55a9ee4f6be0 2024-06-10T01:13:44.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.875+0000 7f59c4a8b700 1 --2- 172.21.15.7:0/1061126606 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55a9ee4f6be0 0x55a9ee5745c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:44.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.875+0000 7f599ffff700 10 client.26806 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:13:44.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.875+0000 7f59c4a8b700 1 --2- 172.21.15.7:0/1061126606 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55a9ee4f6be0 0x55a9ee5745c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d98ca5bddb7b9290 server_cookie=c2b2a0722fc00f27 in_seq=0 out_seq=0 2024-06-10T01:13:44.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.875+0000 7f599ffff700 1 -- 172.21.15.7:0/1061126606 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a9ee432290 con 0x55a9ee4f6be0 2024-06-10T01:13:44.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.875+0000 7f599ffff700 10 client.26806 handle_command_reply: tid=0 2024-06-10T01:13:44.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.875+0000 7f59d1572b80 1 client.26806 shutdown 2024-06-10T01:13:44.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.876+0000 7f59d1572b80 1 -- 172.21.15.7:0/1061126606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a9ee4dd8f0 msgr2=0x55a9ee4f7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:44.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.876+0000 7f59d1572b80 1 --2- 172.21.15.7:0/1061126606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a9ee4dd8f0 0x55a9ee4f7ae0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f59ac00c5b0 tx=0x7f59ac086040 comp rx=0 tx=0).stop 2024-06-10T01:13:44.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.876+0000 7f59d1572b80 1 -- 172.21.15.7:0/1061126606 shutdown_connections 2024-06-10T01:13:44.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.876+0000 7f59d1572b80 1 --2- 172.21.15.7:0/1061126606 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55a9ee4f6be0 0x55a9ee5745c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.876+0000 7f59d1572b80 1 --2- 172.21.15.7:0/1061126606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9ee4e1c00 0x55a9ee55f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.876+0000 7f59d1572b80 1 --2- 172.21.15.7:0/1061126606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a9ee4dcfd0 0x55a9ee4f75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.876+0000 7f59d1572b80 1 --2- 172.21.15.7:0/1061126606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a9ee4dd8f0 0x55a9ee4f7ae0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.876+0000 7f599ffff700 0 client.26806 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:13:44.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.876+0000 7f59d1572b80 1 -- 172.21.15.7:0/1061126606 >> 172.21.15.7:0/1061126606 conn(0x55a9ee403cc0 msgr2=0x55a9ee4cce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:44.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.876+0000 7f59d1572b80 1 -- 172.21.15.7:0/1061126606 shutdown_connections 2024-06-10T01:13:44.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.876+0000 7f59d1572b80 1 -- 172.21.15.7:0/1061126606 wait complete. 2024-06-10T01:13:44.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.876+0000 7f59d1572b80 20 client.26806 trim_cache size 0 max 16384 2024-06-10T01:13:44.876 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:13:44.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.876+0000 7f59c6cef700 1 -- 172.21.15.7:0/1178860731 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f59b00a2ca0 msgr2=0x7f59b00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:44.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.876+0000 7f59c6cef700 1 --2- 172.21.15.7:0/1178860731 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f59b00a2ca0 0x7f59b00a5160 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f59c0199b90 tx=0x7f59b4009260 comp rx=0 tx=0).stop 2024-06-10T01:13:44.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.876+0000 7f59c6cef700 1 -- 172.21.15.7:0/1178860731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c0102380 msgr2=0x7f59c019edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:44.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.876+0000 7f59c6cef700 1 --2- 172.21.15.7:0/1178860731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c0102380 0x7f59c019edd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f59ac00e9c0 tx=0x7f59ac00ecd0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.877+0000 7f59c6cef700 1 -- 172.21.15.7:0/1178860731 shutdown_connections 2024-06-10T01:13:44.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.877+0000 7f59c6cef700 1 --2- 172.21.15.7:0/1178860731 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f59b00a2ca0 0x7f59b00a5160 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.877+0000 7f59c6cef700 1 --2- 172.21.15.7:0/1178860731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59c0106720 0x7f59c0198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.877+0000 7f59c6cef700 1 --2- 172.21.15.7:0/1178860731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59c01019d0 0x7f59c019e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.877+0000 7f59c6cef700 1 --2- 172.21.15.7:0/1178860731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c0102380 0x7f59c019edd0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:44.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.877+0000 7f59c6cef700 1 -- 172.21.15.7:0/1178860731 >> 172.21.15.7:0/1178860731 conn(0x7f59c0078340 msgr2=0x7f59c006ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:44.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.877+0000 7f59c6cef700 1 -- 172.21.15.7:0/1178860731 shutdown_connections 2024-06-10T01:13:44.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:44.877+0000 7f59c6cef700 1 -- 172.21.15.7:0/1178860731 wait complete. 2024-06-10T01:13:45.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:45 smithi007 ceph-mon[32398]: pgmap v664: 129 pgs: 129 active+clean; 103 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 358 op/s 2024-06-10T01:13:45.601 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:13:45.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:45 smithi175 ceph-mon[36636]: pgmap v664: 129 pgs: 129 active+clean; 103 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 358 op/s 2024-06-10T01:13:45.739 INFO:tasks.workunit.client.0.smithi007.stdout: 1 558443 7.84 MB/sec execute 266 sec latency 31.480 ms 2024-06-10T01:13:45.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:45 smithi078 ceph-mon[36683]: pgmap v664: 129 pgs: 129 active+clean; 103 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 358 op/s 2024-06-10T01:13:45.916 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:13:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.644+0000 7ff09c4b3640 1 -- 172.21.15.7:0/3120477163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff094107790 msgr2=0x7ff094109b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.644+0000 7ff09c4b3640 1 --2- 172.21.15.7:0/3120477163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff094107790 0x7ff094109b80 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009ca0 tx=0x7ff08c02a8c0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.644+0000 7ff09c4b3640 1 -- 172.21.15.7:0/3120477163 shutdown_connections 2024-06-10T01:13:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.644+0000 7ff09c4b3640 1 --2- 172.21.15.7:0/3120477163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff094107790 0x7ff094109b80 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.644+0000 7ff09c4b3640 1 --2- 172.21.15.7:0/3120477163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff094104e60 0x7ff094107250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.644+0000 7ff09c4b3640 1 --2- 172.21.15.7:0/3120477163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff094102530 0x7ff094104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.644+0000 7ff09c4b3640 1 -- 172.21.15.7:0/3120477163 >> 172.21.15.7:0/3120477163 conn(0x7ff0940fbe80 msgr2=0x7ff0940fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.644+0000 7ff09c4b3640 1 -- 172.21.15.7:0/3120477163 shutdown_connections 2024-06-10T01:13:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.645+0000 7ff09c4b3640 1 -- 172.21.15.7:0/3120477163 wait complete. 2024-06-10T01:13:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.645+0000 7ff09c4b3640 1 Processor -- start 2024-06-10T01:13:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.645+0000 7ff09c4b3640 1 -- start start 2024-06-10T01:13:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.645+0000 7ff09c4b3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff094102530 0x7ff094197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.645+0000 7ff09c4b3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff094104e60 0x7ff094198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.645+0000 7ff09c4b3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff094107790 0x7ff09419f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.645+0000 7ff09c4b3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff09410d6b0 con 0x7ff094107790 2024-06-10T01:13:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.645+0000 7ff09c4b3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff09410d530 con 0x7ff094102530 2024-06-10T01:13:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.645+0000 7ff09c4b3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff09410d830 con 0x7ff094104e60 2024-06-10T01:13:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.645+0000 7ff09aa50640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff094107790 0x7ff09419f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.645+0000 7ff09aa50640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff094107790 0x7ff09419f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41500/0 (socket says 172.21.15.7:41500) 2024-06-10T01:13:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.645+0000 7ff09aa50640 1 -- 172.21.15.7:0/3472967014 learned_addr learned my addr 172.21.15.7:0/3472967014 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.645+0000 7ff099a4e640 1 --2- 172.21.15.7:0/3472967014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff094104e60 0x7ff094198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.645+0000 7ff09a24f640 1 --2- 172.21.15.7:0/3472967014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff094102530 0x7ff094197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.646+0000 7ff099a4e640 1 -- 172.21.15.7:0/3472967014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff094102530 msgr2=0x7ff094197cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.646+0000 7ff099a4e640 1 --2- 172.21.15.7:0/3472967014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff094102530 0x7ff094197cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.646+0000 7ff099a4e640 1 -- 172.21.15.7:0/3472967014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff094107790 msgr2=0x7ff09419f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.646+0000 7ff099a4e640 1 --2- 172.21.15.7:0/3472967014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff094107790 0x7ff09419f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.646+0000 7ff099a4e640 1 -- 172.21.15.7:0/3472967014 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09419f900 con 0x7ff094104e60 2024-06-10T01:13:46.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.646+0000 7ff099a4e640 1 --2- 172.21.15.7:0/3472967014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff094104e60 0x7ff094198230 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff08400e9c0 tx=0x7ff08400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:46.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.646+0000 7ff08b7fe640 1 -- 172.21.15.7:0/3472967014 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08400cd80 con 0x7ff094104e60 2024-06-10T01:13:46.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.646+0000 7ff09c4b3640 1 -- 172.21.15.7:0/3472967014 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff09419fbf0 con 0x7ff094104e60 2024-06-10T01:13:46.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.646+0000 7ff08b7fe640 1 -- 172.21.15.7:0/3472967014 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0840035f0 con 0x7ff094104e60 2024-06-10T01:13:46.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.646+0000 7ff08b7fe640 1 -- 172.21.15.7:0/3472967014 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff084010670 con 0x7ff094104e60 2024-06-10T01:13:46.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.646+0000 7ff09c4b3640 1 -- 172.21.15.7:0/3472967014 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0941a0190 con 0x7ff094104e60 2024-06-10T01:13:46.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.649+0000 7ff08b7fe640 1 -- 172.21.15.7:0/3472967014 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7ff0840037a0 con 0x7ff094104e60 2024-06-10T01:13:46.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.649+0000 7ff09f1c0740 1 Processor -- start 2024-06-10T01:13:46.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.650+0000 7ff09f1c0740 1 -- start start 2024-06-10T01:13:46.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.650+0000 7ff09f1c0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73c651010 0x55a73c651430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:46.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.650+0000 7ff09f1c0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a73c651a00 0x55a73c656500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:46.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.650+0000 7ff09f1c0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73c656b70 0x55a73c659000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:46.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.650+0000 7ff09f1c0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a73c5b44f0 con 0x55a73c656b70 2024-06-10T01:13:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.650+0000 7ff09f1c0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a73c552870 con 0x55a73c651010 2024-06-10T01:13:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.650+0000 7ff09f1c0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a73c5b04b0 con 0x55a73c651a00 2024-06-10T01:13:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.650+0000 7ff09a24f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a73c651a00 0x55a73c656500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.650+0000 7ff09aa50640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73c651010 0x55a73c651430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.650+0000 7ff099a4e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73c656b70 0x55a73c659000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.650+0000 7ff09aa50640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73c651010 0x55a73c651430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:47156/0 (socket says 172.21.15.7:47156) 2024-06-10T01:13:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.650+0000 7ff09a24f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a73c651a00 0x55a73c656500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34378/0 (socket says 172.21.15.7:34378) 2024-06-10T01:13:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.650+0000 7ff09aa50640 1 -- 172.21.15.7:0/2751892398 learned_addr learned my addr 172.21.15.7:0/2751892398 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.650+0000 7ff09aa50640 1 -- 172.21.15.7:0/2751892398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a73c651a00 msgr2=0x55a73c656500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.650+0000 7ff09aa50640 1 --2- 172.21.15.7:0/2751892398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a73c651a00 0x55a73c656500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.650+0000 7ff09aa50640 1 -- 172.21.15.7:0/2751892398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73c656b70 msgr2=0x55a73c659000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.650+0000 7ff09aa50640 1 --2- 172.21.15.7:0/2751892398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73c656b70 0x55a73c659000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.650+0000 7ff09aa50640 1 -- 172.21.15.7:0/2751892398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a73c56e9b0 con 0x55a73c651010 2024-06-10T01:13:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.651+0000 7ff08b7fe640 1 --2- 172.21.15.7:0/3472967014 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff0800a2b80 0x7ff0800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.651+0000 7ff09aa50640 1 --2- 172.21.15.7:0/2751892398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73c651010 0x55a73c651430 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0049d0 tx=0x7ff08c009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2045beed050f5f1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.651+0000 7ff09a24f640 1 --2- 172.21.15.7:0/3472967014 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff0800a2b80 0x7ff0800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.651+0000 7ff08b7fe640 1 -- 172.21.15.7:0/3472967014 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7ff0840cf730 con 0x7ff094104e60 2024-06-10T01:13:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.651+0000 7ff088bfe640 1 -- 172.21.15.7:0/2751892398 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c049070 con 0x55a73c651010 2024-06-10T01:13:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.651+0000 7ff088bfe640 1 -- 172.21.15.7:0/2751892398 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff08c036040 con 0x55a73c651010 2024-06-10T01:13:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.651+0000 7ff088bfe640 1 -- 172.21.15.7:0/2751892398 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c02fd10 con 0x55a73c651010 2024-06-10T01:13:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.652+0000 7ff09f1c0740 1 -- 172.21.15.7:0/2751892398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73c651010 msgr2=0x55a73c651430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.652+0000 7ff09f1c0740 1 --2- 172.21.15.7:0/2751892398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73c651010 0x55a73c651430 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0049d0 tx=0x7ff08c009360 comp rx=0 tx=0).stop 2024-06-10T01:13:46.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.652+0000 7ff09a24f640 1 --2- 172.21.15.7:0/3472967014 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff0800a2b80 0x7ff0800a5040 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x55a73c651e00 tx=0x7ff090005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:46.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.652+0000 7ff09f1c0740 1 -- 172.21.15.7:0/2751892398 shutdown_connections 2024-06-10T01:13:46.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.652+0000 7ff09f1c0740 1 --2- 172.21.15.7:0/2751892398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73c656b70 0x55a73c659000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.652+0000 7ff09f1c0740 1 --2- 172.21.15.7:0/2751892398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a73c651a00 0x55a73c656500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.652+0000 7ff09f1c0740 1 --2- 172.21.15.7:0/2751892398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73c651010 0x55a73c651430 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.652+0000 7ff09f1c0740 1 -- 172.21.15.7:0/2751892398 >> 172.21.15.7:0/2751892398 conn(0x55a73c64c680 msgr2=0x55a73c64ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:46.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.652+0000 7ff09f1c0740 1 -- 172.21.15.7:0/2751892398 shutdown_connections 2024-06-10T01:13:46.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.652+0000 7ff09f1c0740 1 -- 172.21.15.7:0/2751892398 wait complete. 2024-06-10T01:13:46.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.652+0000 7ff09f1c0740 1 -- start start 2024-06-10T01:13:46.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.652+0000 7ff09f1c0740 1 Processor -- start 2024-06-10T01:13:46.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.652+0000 7ff09f1c0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a73c651010 0x55a73c65b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:46.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.652+0000 7ff09f1c0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73c651a00 0x55a73c65b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:46.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.652+0000 7ff09f1c0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73c656b70 0x55a73c662640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:46.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.652+0000 7ff09f1c0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a73c659940 con 0x55a73c656b70 2024-06-10T01:13:46.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.653+0000 7ff09f1c0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a73c53dae0 con 0x55a73c651a00 2024-06-10T01:13:46.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.653+0000 7ff09f1c0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a73c659ac0 con 0x55a73c651010 2024-06-10T01:13:46.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.653+0000 7ff09f1c0740 10 client.0 _ll_register_callbacks cb 0x55a73c5769b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:13:46.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.653+0000 7ff09aa50640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73c656b70 0x55a73c662640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:46.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.653+0000 7ff09a24f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a73c651010 0x55a73c65b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:46.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.653+0000 7ff09aa50640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73c656b70 0x55a73c662640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41520/0 (socket says 172.21.15.7:41520) 2024-06-10T01:13:46.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.653+0000 7ff09aa50640 1 -- 172.21.15.7:0/451695906 learned_addr learned my addr 172.21.15.7:0/451695906 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:46.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.653+0000 7ff099a4e640 1 --2- 172.21.15.7:0/451695906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73c651a00 0x55a73c65b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:46.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.653+0000 7ff09aa50640 1 -- 172.21.15.7:0/451695906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a73c651010 msgr2=0x55a73c65b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:46.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.653+0000 7ff09aa50640 1 --2- 172.21.15.7:0/451695906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a73c651010 0x55a73c65b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.653+0000 7ff09aa50640 1 -- 172.21.15.7:0/451695906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73c651a00 msgr2=0x55a73c65b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:46.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.653+0000 7ff09aa50640 1 --2- 172.21.15.7:0/451695906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73c651a00 0x55a73c65b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.653+0000 7ff09aa50640 1 -- 172.21.15.7:0/451695906 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a73c2c8210 con 0x55a73c656b70 2024-06-10T01:13:46.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.653+0000 7ff09aa50640 1 --2- 172.21.15.7:0/451695906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73c656b70 0x55a73c662640 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff08c047040 tx=0x7ff08c02fef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de58928d98b6a3c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:46.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.653+0000 7ff07dffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:13:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.653+0000 7ff07dffb640 1 -- 172.21.15.7:0/451695906 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c054060 con 0x55a73c656b70 2024-06-10T01:13:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.653+0000 7ff07dffb640 1 -- 172.21.15.7:0/451695906 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7ff08c031450 con 0x55a73c656b70 2024-06-10T01:13:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.653+0000 7ff09f1c0740 1 -- 172.21.15.7:0/451695906 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff08c036040 con 0x55a73c656b70 2024-06-10T01:13:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.653+0000 7ff07dffb640 1 -- 172.21.15.7:0/451695906 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff08c049070 con 0x55a73c656b70 2024-06-10T01:13:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.654+0000 7ff09f1c0740 10 client.17385 fetch_fsmap learned FSMap version 35 2024-06-10T01:13:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.654+0000 7ff09f1c0740 1 -- 172.21.15.7:0/451695906 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a73c394620 con 0x55a73c656b70 2024-06-10T01:13:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.654+0000 7ff07dffb640 1 -- 172.21.15.7:0/451695906 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff08c036040 con 0x55a73c656b70 2024-06-10T01:13:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.654+0000 7ff07dffb640 1 -- 172.21.15.7:0/451695906 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7ff08c030c60 con 0x55a73c656b70 2024-06-10T01:13:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.654+0000 7ff09f1c0740 10 client.17385 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:13:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.654+0000 7ff09f1c0740 10 client.17385 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:13:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.654+0000 7ff09f1c0740 20 client.17385 populate_metadata read hostname 'smithi007' 2024-06-10T01:13:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.654+0000 7ff09f1c0740 1 --2- 172.21.15.7:0/451695906 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a73c666cf0 0x55a73c6690e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.654+0000 7ff09f1c0740 4 client.17385 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:13:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.654+0000 7ff09f1c0740 1 -- 172.21.15.7:0/451695906 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a73c5c7f10 con 0x55a73c666cf0 2024-06-10T01:13:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.654+0000 7ff09a24f640 1 --2- 172.21.15.7:0/451695906 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a73c666cf0 0x55a73c6690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.655+0000 7ff07dffb640 10 client.17385 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.655+0000 7ff09a24f640 1 --2- 172.21.15.7:0/451695906 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a73c666cf0 0x55a73c6690e0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9af41691321d650c server_cookie=749bdd94f7730bed in_seq=0 out_seq=0 2024-06-10T01:13:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.655+0000 7ff07dffb640 1 -- 172.21.15.7:0/451695906 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a73c5c7f10 con 0x55a73c666cf0 2024-06-10T01:13:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.655+0000 7ff07dffb640 10 client.17385 handle_command_reply: tid=0 2024-06-10T01:13:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.655+0000 7ff09f1c0740 1 client.17385 shutdown 2024-06-10T01:13:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.656+0000 7ff09f1c0740 1 -- 172.21.15.7:0/451695906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73c656b70 msgr2=0x55a73c662640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.656+0000 7ff09f1c0740 1 --2- 172.21.15.7:0/451695906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73c656b70 0x55a73c662640 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff08c047040 tx=0x7ff08c02fef0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.656+0000 7ff09f1c0740 1 -- 172.21.15.7:0/451695906 shutdown_connections 2024-06-10T01:13:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.656+0000 7ff09f1c0740 1 --2- 172.21.15.7:0/451695906 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a73c666cf0 0x55a73c6690e0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.656+0000 7ff09f1c0740 1 --2- 172.21.15.7:0/451695906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73c656b70 0x55a73c662640 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.656+0000 7ff09f1c0740 1 --2- 172.21.15.7:0/451695906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73c651a00 0x55a73c65b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.656+0000 7ff07dffb640 0 client.17385 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.656+0000 7ff09f1c0740 1 --2- 172.21.15.7:0/451695906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a73c651010 0x55a73c65b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.656+0000 7ff09f1c0740 1 -- 172.21.15.7:0/451695906 >> 172.21.15.7:0/451695906 conn(0x55a73c64c680 msgr2=0x55a73c64b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.656+0000 7ff09f1c0740 1 -- 172.21.15.7:0/451695906 shutdown_connections 2024-06-10T01:13:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.656+0000 7ff09f1c0740 1 -- 172.21.15.7:0/451695906 wait complete. 2024-06-10T01:13:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.656+0000 7ff09f1c0740 20 client.17385 trim_cache size 0 max 16384 2024-06-10T01:13:46.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.671+0000 7ff09f1c0740 1 Processor -- start 2024-06-10T01:13:46.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.672+0000 7ff09f1c0740 1 -- start start 2024-06-10T01:13:46.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.672+0000 7ff09f1c0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73c651010 0x55a73c65b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:46.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.672+0000 7ff09f1c0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a73c651a00 0x55a73c662600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:46.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.672+0000 7ff09f1c0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73c656b70 0x55a73c65b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:46.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.672+0000 7ff09f1c0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff0941ace90 con 0x55a73c656b70 2024-06-10T01:13:46.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.672+0000 7ff09f1c0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff0941aa700 con 0x55a73c651010 2024-06-10T01:13:46.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.672+0000 7ff09f1c0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff0941a8550 con 0x55a73c651a00 2024-06-10T01:13:46.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.672+0000 7ff09aa50640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73c656b70 0x55a73c65b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:46.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.672+0000 7ff09aa50640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73c656b70 0x55a73c65b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41536/0 (socket says 172.21.15.7:41536) 2024-06-10T01:13:46.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.672+0000 7ff09aa50640 1 -- 172.21.15.7:0/3933550352 learned_addr learned my addr 172.21.15.7:0/3933550352 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:46.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.672+0000 7ff099a4e640 1 --2- 172.21.15.7:0/3933550352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a73c651a00 0x55a73c662600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:46.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.672+0000 7ff09a24f640 1 --2- 172.21.15.7:0/3933550352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73c651010 0x55a73c65b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:46.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.672+0000 7ff09a24f640 1 -- 172.21.15.7:0/3933550352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a73c651a00 msgr2=0x55a73c662600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:46.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.672+0000 7ff09a24f640 1 --2- 172.21.15.7:0/3933550352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a73c651a00 0x55a73c662600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.672+0000 7ff09a24f640 1 -- 172.21.15.7:0/3933550352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73c656b70 msgr2=0x55a73c65b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:46.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.672+0000 7ff09a24f640 1 --2- 172.21.15.7:0/3933550352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73c656b70 0x55a73c65b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.672+0000 7ff09a24f640 1 -- 172.21.15.7:0/3933550352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a73c646ae0 con 0x55a73c651010 2024-06-10T01:13:46.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.672+0000 7ff09aa50640 1 --2- 172.21.15.7:0/3933550352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73c656b70 0x55a73c65b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:46.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.672+0000 7ff09a24f640 1 --2- 172.21.15.7:0/3933550352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73c651010 0x55a73c65b130 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x55a73c669950 tx=0x7ff090019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f58379eb3f8dff4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:46.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.673+0000 7ff07ffff640 1 -- 172.21.15.7:0/3933550352 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09001dd80 con 0x55a73c651010 2024-06-10T01:13:46.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.673+0000 7ff07ffff640 1 -- 172.21.15.7:0/3933550352 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff090005500 con 0x55a73c651010 2024-06-10T01:13:46.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.673+0000 7ff07ffff640 1 -- 172.21.15.7:0/3933550352 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09002a410 con 0x55a73c651010 2024-06-10T01:13:46.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.673+0000 7ff09f1c0740 1 -- 172.21.15.7:0/3933550352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73c651010 msgr2=0x55a73c65b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:46.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.673+0000 7ff09f1c0740 1 --2- 172.21.15.7:0/3933550352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73c651010 0x55a73c65b130 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x55a73c669950 tx=0x7ff090019000 comp rx=0 tx=0).stop 2024-06-10T01:13:46.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.673+0000 7ff09f1c0740 1 -- 172.21.15.7:0/3933550352 shutdown_connections 2024-06-10T01:13:46.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.673+0000 7ff09f1c0740 1 --2- 172.21.15.7:0/3933550352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73c656b70 0x55a73c65b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.673+0000 7ff09f1c0740 1 --2- 172.21.15.7:0/3933550352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a73c651a00 0x55a73c662600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.673+0000 7ff09f1c0740 1 --2- 172.21.15.7:0/3933550352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73c651010 0x55a73c65b130 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.673+0000 7ff09f1c0740 1 -- 172.21.15.7:0/3933550352 >> 172.21.15.7:0/3933550352 conn(0x55a73c64c680 msgr2=0x55a73c652c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:46.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.673+0000 7ff09f1c0740 1 -- 172.21.15.7:0/3933550352 shutdown_connections 2024-06-10T01:13:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.673+0000 7ff09f1c0740 1 -- 172.21.15.7:0/3933550352 wait complete. 2024-06-10T01:13:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.674+0000 7ff09f1c0740 1 -- start start 2024-06-10T01:13:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.674+0000 7ff09f1c0740 1 Processor -- start 2024-06-10T01:13:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.674+0000 7ff09f1c0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a73c651010 0x55a73c662350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.674+0000 7ff09f1c0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73c651a00 0x55a73c65edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.674+0000 7ff09f1c0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73c656b70 0x55a73c65f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.674+0000 7ff09f1c0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff0941a8bc0 con 0x55a73c651a00 2024-06-10T01:13:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.674+0000 7ff09f1c0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff0941aa030 con 0x55a73c656b70 2024-06-10T01:13:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.674+0000 7ff09f1c0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff0941a9290 con 0x55a73c651010 2024-06-10T01:13:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.674+0000 7ff09f1c0740 10 client.0 _ll_register_callbacks cb 0x7ff08c047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:13:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.674+0000 7ff099a4e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73c651a00 0x55a73c65edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.674+0000 7ff099a4e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73c651a00 0x55a73c65edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41540/0 (socket says 172.21.15.7:41540) 2024-06-10T01:13:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.674+0000 7ff099a4e640 1 -- 172.21.15.7:0/1774960518 learned_addr learned my addr 172.21.15.7:0/1774960518 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.674+0000 7ff09a24f640 1 --2- 172.21.15.7:0/1774960518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a73c651010 0x55a73c662350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.674+0000 7ff09aa50640 1 --2- 172.21.15.7:0/1774960518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73c656b70 0x55a73c65f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.674+0000 7ff099a4e640 1 -- 172.21.15.7:0/1774960518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a73c651010 msgr2=0x55a73c662350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.674+0000 7ff099a4e640 1 --2- 172.21.15.7:0/1774960518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a73c651010 0x55a73c662350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.674+0000 7ff099a4e640 1 -- 172.21.15.7:0/1774960518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73c656b70 msgr2=0x55a73c65f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.674+0000 7ff099a4e640 1 --2- 172.21.15.7:0/1774960518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73c656b70 0x55a73c65f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.674+0000 7ff099a4e640 1 -- 172.21.15.7:0/1774960518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a73c65f920 con 0x55a73c651a00 2024-06-10T01:13:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.675+0000 7ff099a4e640 1 --2- 172.21.15.7:0/1774960518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73c651a00 0x55a73c65edf0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff0840cceb0 tx=0x7ff084005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d01e2f6b9bc5ce server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.675+0000 7ff07dffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:13:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.675+0000 7ff07dffb640 1 -- 172.21.15.7:0/1774960518 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08409c310 con 0x55a73c651a00 2024-06-10T01:13:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.675+0000 7ff07dffb640 1 -- 172.21.15.7:0/1774960518 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7ff0840981f0 con 0x55a73c651a00 2024-06-10T01:13:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.675+0000 7ff09f1c0740 1 -- 172.21.15.7:0/1774960518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff090005500 con 0x55a73c651a00 2024-06-10T01:13:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.675+0000 7ff07dffb640 1 -- 172.21.15.7:0/1774960518 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff08409bc10 con 0x55a73c651a00 2024-06-10T01:13:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.675+0000 7ff09f1c0740 10 client.17391 fetch_fsmap learned FSMap version 35 2024-06-10T01:13:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.675+0000 7ff09f1c0740 1 -- 172.21.15.7:0/1774960518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a73c6d0ac0 con 0x55a73c651a00 2024-06-10T01:13:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.675+0000 7ff07dffb640 1 -- 172.21.15.7:0/1774960518 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0840b2030 con 0x55a73c651a00 2024-06-10T01:13:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.675+0000 7ff07dffb640 1 -- 172.21.15.7:0/1774960518 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7ff08409b060 con 0x55a73c651a00 2024-06-10T01:13:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.675+0000 7ff09f1c0740 10 client.17391 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:13:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.675+0000 7ff09f1c0740 10 client.17391 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:13:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.675+0000 7ff09f1c0740 20 client.17391 populate_metadata read hostname 'smithi007' 2024-06-10T01:13:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.675+0000 7ff09f1c0740 1 --2- 172.21.15.7:0/1774960518 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a73c6d1520 0x55a73c6d3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.675+0000 7ff09f1c0740 4 client.17391 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:13:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.675+0000 7ff09f1c0740 1 -- 172.21.15.7:0/1774960518 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a73c656870 con 0x55a73c6d1520 2024-06-10T01:13:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.675+0000 7ff09a24f640 1 --2- 172.21.15.7:0/1774960518 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a73c6d1520 0x55a73c6d3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.676+0000 7ff07dffb640 10 client.17391 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.676+0000 7ff09a24f640 1 --2- 172.21.15.7:0/1774960518 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a73c6d1520 0x55a73c6d3940 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfceb43037f7b975 server_cookie=a06959e6c777a1dc in_seq=0 out_seq=0 2024-06-10T01:13:46.740 INFO:tasks.workunit.client.0.smithi007.stdout: 1 560463 7.86 MB/sec execute 267 sec latency 93.334 ms 2024-06-10T01:13:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.776+0000 7ff07dffb640 1 -- 172.21.15.7:0/1774960518 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a73c656870 con 0x55a73c6d1520 2024-06-10T01:13:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.776+0000 7ff07dffb640 10 client.17391 handle_command_reply: tid=0 2024-06-10T01:13:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.776+0000 7ff09f1c0740 1 client.17391 shutdown 2024-06-10T01:13:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.776+0000 7ff09f1c0740 1 -- 172.21.15.7:0/1774960518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73c651a00 msgr2=0x55a73c65edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.776+0000 7ff09f1c0740 1 --2- 172.21.15.7:0/1774960518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73c651a00 0x55a73c65edf0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff0840cceb0 tx=0x7ff084005bb0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.776+0000 7ff09f1c0740 1 -- 172.21.15.7:0/1774960518 shutdown_connections 2024-06-10T01:13:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.776+0000 7ff09f1c0740 1 --2- 172.21.15.7:0/1774960518 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a73c6d1520 0x55a73c6d3940 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.776+0000 7ff09f1c0740 1 --2- 172.21.15.7:0/1774960518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73c656b70 0x55a73c65f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.776+0000 7ff09f1c0740 1 --2- 172.21.15.7:0/1774960518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73c651a00 0x55a73c65edf0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.776+0000 7ff07dffb640 0 client.17391 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.776+0000 7ff09f1c0740 1 --2- 172.21.15.7:0/1774960518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a73c651010 0x55a73c662350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.776+0000 7ff09f1c0740 1 -- 172.21.15.7:0/1774960518 >> 172.21.15.7:0/1774960518 conn(0x55a73c64c680 msgr2=0x55a73c652c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.776+0000 7ff09f1c0740 1 -- 172.21.15.7:0/1774960518 shutdown_connections 2024-06-10T01:13:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.776+0000 7ff09f1c0740 1 -- 172.21.15.7:0/1774960518 wait complete. 2024-06-10T01:13:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.777+0000 7ff09f1c0740 20 client.17391 trim_cache size 0 max 16384 2024-06-10T01:13:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:13:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:13:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:13:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.777+0000 7ff09c4b3640 1 -- 172.21.15.7:0/3472967014 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff0800a2b80 msgr2=0x7ff0800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.777+0000 7ff09c4b3640 1 --2- 172.21.15.7:0/3472967014 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff0800a2b80 0x7ff0800a5040 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x55a73c651e00 tx=0x7ff090005f90 comp rx=0 tx=0).stop 2024-06-10T01:13:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.777+0000 7ff09c4b3640 1 -- 172.21.15.7:0/3472967014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff094104e60 msgr2=0x7ff094198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.777+0000 7ff09c4b3640 1 --2- 172.21.15.7:0/3472967014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff094104e60 0x7ff094198230 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff08400e9c0 tx=0x7ff08400ee80 comp rx=0 tx=0).stop 2024-06-10T01:13:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.777+0000 7ff09c4b3640 1 -- 172.21.15.7:0/3472967014 shutdown_connections 2024-06-10T01:13:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.777+0000 7ff09c4b3640 1 --2- 172.21.15.7:0/3472967014 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff0800a2b80 0x7ff0800a5040 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.777+0000 7ff09c4b3640 1 --2- 172.21.15.7:0/3472967014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff094107790 0x7ff09419f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.777+0000 7ff09c4b3640 1 --2- 172.21.15.7:0/3472967014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff094104e60 0x7ff094198230 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.777+0000 7ff09c4b3640 1 --2- 172.21.15.7:0/3472967014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff094102530 0x7ff094197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.777+0000 7ff09c4b3640 1 -- 172.21.15.7:0/3472967014 >> 172.21.15.7:0/3472967014 conn(0x7ff0940fbe80 msgr2=0x7ff094102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.777+0000 7ff09c4b3640 1 -- 172.21.15.7:0/3472967014 shutdown_connections 2024-06-10T01:13:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:46.777+0000 7ff09c4b3640 1 -- 172.21.15.7:0/3472967014 wait complete. 2024-06-10T01:13:47.636 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:47 smithi007 ceph-mon[32398]: pgmap v665: 129 pgs: 129 active+clean; 103 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 315 op/s 2024-06-10T01:13:47.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.657+0000 7faa69485700 1 -- 172.21.15.7:0/3256274202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa64103120 msgr2=0x7faa64107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:47.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.657+0000 7faa69485700 1 --2- 172.21.15.7:0/3256274202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa64103120 0x7faa64107130 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7faa580099c0 tx=0x7faa58009cd0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.657+0000 7faa69485700 1 -- 172.21.15.7:0/3256274202 shutdown_connections 2024-06-10T01:13:47.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.657+0000 7faa69485700 1 --2- 172.21.15.7:0/3256274202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa64107860 0x7faa6410f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.657+0000 7faa69485700 1 --2- 172.21.15.7:0/3256274202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa64102770 0x7faa64102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.657+0000 7faa69485700 1 --2- 172.21.15.7:0/3256274202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa64103120 0x7faa64107130 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.657+0000 7faa69485700 1 -- 172.21.15.7:0/3256274202 >> 172.21.15.7:0/3256274202 conn(0x7faa64079600 msgr2=0x7faa64079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:47.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.657+0000 7faa69485700 1 -- 172.21.15.7:0/3256274202 shutdown_connections 2024-06-10T01:13:47.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.657+0000 7faa69485700 1 -- 172.21.15.7:0/3256274202 wait complete. 2024-06-10T01:13:47.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.657+0000 7faa69485700 1 Processor -- start 2024-06-10T01:13:47.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.658+0000 7faa69485700 1 -- start start 2024-06-10T01:13:47.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.658+0000 7faa69485700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa64102770 0x7faa64074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:47.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.658+0000 7faa69485700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa64103120 0x7faa64074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:47.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.658+0000 7faa69485700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa64107860 0x7faa64075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:47.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.658+0000 7faa69485700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faa64112790 con 0x7faa64107860 2024-06-10T01:13:47.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.658+0000 7faa69485700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faa64112610 con 0x7faa64103120 2024-06-10T01:13:47.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.658+0000 7faa69485700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faa64112910 con 0x7faa64102770 2024-06-10T01:13:47.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.658+0000 7faa637fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa64107860 0x7faa64075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:47.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.658+0000 7faa637fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa64107860 0x7faa64075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41556/0 (socket says 172.21.15.7:41556) 2024-06-10T01:13:47.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.658+0000 7faa637fe700 1 -- 172.21.15.7:0/576271780 learned_addr learned my addr 172.21.15.7:0/576271780 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:47.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.658+0000 7faa62ffd700 1 --2- 172.21.15.7:0/576271780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa64102770 0x7faa64074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:47.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.658+0000 7faa627fc700 1 --2- 172.21.15.7:0/576271780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa64103120 0x7faa64074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:47.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.658+0000 7faa637fe700 1 -- 172.21.15.7:0/576271780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa64102770 msgr2=0x7faa64074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:47.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.658+0000 7faa637fe700 1 --2- 172.21.15.7:0/576271780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa64102770 0x7faa64074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.658+0000 7faa637fe700 1 -- 172.21.15.7:0/576271780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa64103120 msgr2=0x7faa64074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:47.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.658+0000 7faa637fe700 1 --2- 172.21.15.7:0/576271780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa64103120 0x7faa64074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.658+0000 7faa637fe700 1 -- 172.21.15.7:0/576271780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa641a70c0 con 0x7faa64107860 2024-06-10T01:13:47.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.658+0000 7faa627fc700 1 --2- 172.21.15.7:0/576271780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa64103120 0x7faa64074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:47.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.659+0000 7faa637fe700 1 --2- 172.21.15.7:0/576271780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa64107860 0x7faa64075210 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7faa5c00ef30 tx=0x7faa5c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:47.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.659+0000 7faa4bfff700 1 -- 172.21.15.7:0/576271780 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c0092e0 con 0x7faa64107860 2024-06-10T01:13:47.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.659+0000 7faa4bfff700 1 -- 172.21.15.7:0/576271780 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa5c004bb0 con 0x7faa64107860 2024-06-10T01:13:47.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.659+0000 7faa69485700 1 -- 172.21.15.7:0/576271780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa641a7350 con 0x7faa64107860 2024-06-10T01:13:47.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.659+0000 7faa4bfff700 1 -- 172.21.15.7:0/576271780 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c004280 con 0x7faa64107860 2024-06-10T01:13:47.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.659+0000 7faa69485700 1 -- 172.21.15.7:0/576271780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa641a7860 con 0x7faa64107860 2024-06-10T01:13:47.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.661+0000 7faa4bfff700 1 -- 172.21.15.7:0/576271780 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7faa5c01b030 con 0x7faa64107860 2024-06-10T01:13:47.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.662+0000 7faa4bfff700 1 --2- 172.21.15.7:0/576271780 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7faa4c0a2c20 0x7faa4c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:47.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.662+0000 7faa4bfff700 1 -- 172.21.15.7:0/576271780 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7faa5c0cee20 con 0x7faa64107860 2024-06-10T01:13:47.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.662+0000 7faa62ffd700 1 --2- 172.21.15.7:0/576271780 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7faa4c0a2c20 0x7faa4c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:47.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.663+0000 7faa62ffd700 1 --2- 172.21.15.7:0/576271780 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7faa4c0a2c20 0x7faa4c0a50e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7faa54009c60 tx=0x7faa54009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:47.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.663+0000 7faa73d08b80 1 Processor -- start 2024-06-10T01:13:47.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.663+0000 7faa73d08b80 1 -- start start 2024-06-10T01:13:47.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.663+0000 7faa73d08b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56202bf02f60 0x56202bf03340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:47.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.663+0000 7faa73d08b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56202bf03880 0x56202befd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:47.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.663+0000 7faa73d08b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56202befd7a0 0x56202befdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:47.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.663+0000 7faa73d08b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56202bdeff60 con 0x56202bf02f60 2024-06-10T01:13:47.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.663+0000 7faa73d08b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56202be4e850 con 0x56202bf03880 2024-06-10T01:13:47.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.663+0000 7faa73d08b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56202bdb6460 con 0x56202befd7a0 2024-06-10T01:13:47.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.663+0000 7faa62ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56202bf02f60 0x56202bf03340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:47.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.663+0000 7faa62ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56202bf02f60 0x56202bf03340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41566/0 (socket says 172.21.15.7:41566) 2024-06-10T01:13:47.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.663+0000 7faa62ffd700 1 -- 172.21.15.7:0/1637293830 learned_addr learned my addr 172.21.15.7:0/1637293830 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:47.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.663+0000 7faa627fc700 1 --2- 172.21.15.7:0/1637293830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56202bf03880 0x56202befd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:47.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.663+0000 7faa637fe700 1 --2- 172.21.15.7:0/1637293830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56202befd7a0 0x56202befdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:47.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.663+0000 7faa62ffd700 1 -- 172.21.15.7:0/1637293830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56202befd7a0 msgr2=0x56202befdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:47.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.663+0000 7faa62ffd700 1 --2- 172.21.15.7:0/1637293830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56202befd7a0 0x56202befdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.664+0000 7faa62ffd700 1 -- 172.21.15.7:0/1637293830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56202bf03880 msgr2=0x56202befd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:47.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.664+0000 7faa62ffd700 1 --2- 172.21.15.7:0/1637293830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56202bf03880 0x56202befd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.664+0000 7faa62ffd700 1 -- 172.21.15.7:0/1637293830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56202bd9d8d0 con 0x56202bf02f60 2024-06-10T01:13:47.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.664+0000 7faa627fc700 1 --2- 172.21.15.7:0/1637293830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56202bf03880 0x56202befd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:47.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.664+0000 7faa62ffd700 1 --2- 172.21.15.7:0/1637293830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56202bf02f60 0x56202bf03340 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7faa5400e4e0 tx=0x7faa54014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653754a0924e47f6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:47.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.664+0000 7faa497fa700 1 -- 172.21.15.7:0/1637293830 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5400ee00 con 0x56202bf02f60 2024-06-10T01:13:47.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.664+0000 7faa497fa700 1 -- 172.21.15.7:0/1637293830 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa5401c8f0 con 0x56202bf02f60 2024-06-10T01:13:47.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.664+0000 7faa497fa700 1 -- 172.21.15.7:0/1637293830 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5401dae0 con 0x56202bf02f60 2024-06-10T01:13:47.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.665+0000 7faa73d08b80 1 -- 172.21.15.7:0/1637293830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56202bf02f60 msgr2=0x56202bf03340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:47.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.665+0000 7faa73d08b80 1 --2- 172.21.15.7:0/1637293830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56202bf02f60 0x56202bf03340 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7faa5400e4e0 tx=0x7faa54014000 comp rx=0 tx=0).stop 2024-06-10T01:13:47.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.665+0000 7faa73d08b80 1 -- 172.21.15.7:0/1637293830 shutdown_connections 2024-06-10T01:13:47.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.665+0000 7faa73d08b80 1 --2- 172.21.15.7:0/1637293830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56202befd7a0 0x56202befdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.665+0000 7faa73d08b80 1 --2- 172.21.15.7:0/1637293830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56202bf02f60 0x56202bf03340 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.665+0000 7faa73d08b80 1 --2- 172.21.15.7:0/1637293830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56202bf03880 0x56202befd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.665+0000 7faa73d08b80 1 -- 172.21.15.7:0/1637293830 >> 172.21.15.7:0/1637293830 conn(0x56202be2a080 msgr2=0x56202bef8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:47.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.665+0000 7faa73d08b80 1 -- 172.21.15.7:0/1637293830 shutdown_connections 2024-06-10T01:13:47.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.665+0000 7faa73d08b80 1 -- 172.21.15.7:0/1637293830 wait complete. 2024-06-10T01:13:47.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.665+0000 7faa73d08b80 1 -- start start 2024-06-10T01:13:47.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.665+0000 7faa73d08b80 1 Processor -- start 2024-06-10T01:13:47.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.665+0000 7faa73d08b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56202befd7a0 0x56202bf095f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:47.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.665+0000 7faa73d08b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56202bf02f60 0x56202bf068f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:47.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.666+0000 7faa73d08b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56202bf03880 0x56202bf06e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:47.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.666+0000 7faa73d08b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56202bdeefc0 con 0x56202bf02f60 2024-06-10T01:13:47.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.666+0000 7faa73d08b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56202be4a810 con 0x56202bf03880 2024-06-10T01:13:47.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.666+0000 7faa73d08b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56202bf0a260 con 0x56202befd7a0 2024-06-10T01:13:47.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.666+0000 7faa73d08b80 10 client.0 _ll_register_callbacks cb 0x56202bde45b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:13:47.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.666+0000 7faa62ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56202befd7a0 0x56202bf095f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:47.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.666+0000 7faa627fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56202bf02f60 0x56202bf068f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:47.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.666+0000 7faa637fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56202bf03880 0x56202bf06e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:47.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.666+0000 7faa62ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56202befd7a0 0x56202bf095f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34424/0 (socket says 172.21.15.7:34424) 2024-06-10T01:13:47.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.666+0000 7faa627fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56202bf02f60 0x56202bf068f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41570/0 (socket says 172.21.15.7:41570) 2024-06-10T01:13:47.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.666+0000 7faa62ffd700 1 -- 172.21.15.7:0/835450042 learned_addr learned my addr 172.21.15.7:0/835450042 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:47.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.666+0000 7faa637fe700 1 -- 172.21.15.7:0/835450042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56202befd7a0 msgr2=0x56202bf095f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:47.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.666+0000 7faa637fe700 1 --2- 172.21.15.7:0/835450042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56202befd7a0 0x56202bf095f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.666+0000 7faa637fe700 1 -- 172.21.15.7:0/835450042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56202bf02f60 msgr2=0x56202bf068f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:47.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.666+0000 7faa637fe700 1 --2- 172.21.15.7:0/835450042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56202bf02f60 0x56202bf068f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.666+0000 7faa637fe700 1 -- 172.21.15.7:0/835450042 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56202be14750 con 0x56202bf03880 2024-06-10T01:13:47.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.666+0000 7faa627fc700 1 --2- 172.21.15.7:0/835450042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56202bf02f60 0x56202bf068f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:47.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.666+0000 7faa637fe700 1 --2- 172.21.15.7:0/835450042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56202bf03880 0x56202bf06e30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0cc5f0 tx=0x7faa5c09b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e35d9b371f7fbe41 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:47.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.666+0000 7faa46ff5700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:13:47.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.666+0000 7faa46ff5700 1 -- 172.21.15.7:0/835450042 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c00ce80 con 0x56202bf03880 2024-06-10T01:13:47.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.666+0000 7faa46ff5700 1 -- 172.21.15.7:0/835450042 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7faa5c004d60 con 0x56202bf03880 2024-06-10T01:13:47.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.666+0000 7faa73d08b80 1 -- 172.21.15.7:0/835450042 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa5401c8f0 con 0x56202bf03880 2024-06-10T01:13:47.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.666+0000 7faa46ff5700 1 -- 172.21.15.7:0/835450042 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7faa5c09ba90 con 0x56202bf03880 2024-06-10T01:13:47.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.667+0000 7faa73d08b80 10 client.26819 fetch_fsmap learned FSMap version 35 2024-06-10T01:13:47.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.667+0000 7faa73d08b80 1 -- 172.21.15.7:0/835450042 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56202bf14a40 con 0x56202bf03880 2024-06-10T01:13:47.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.667+0000 7faa46ff5700 1 -- 172.21.15.7:0/835450042 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa5401c8f0 con 0x56202bf03880 2024-06-10T01:13:47.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.667+0000 7faa46ff5700 1 -- 172.21.15.7:0/835450042 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7faa5c09a870 con 0x56202bf03880 2024-06-10T01:13:47.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.667+0000 7faa73d08b80 10 client.26819 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:13:47.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.667+0000 7faa73d08b80 10 client.26819 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:13:47.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.667+0000 7faa73d08b80 20 client.26819 populate_metadata read hostname 'smithi007' 2024-06-10T01:13:47.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.667+0000 7faa73d08b80 1 --2- 172.21.15.7:0/835450042 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x56202bf17080 0x56202bf19470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:47.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.667+0000 7faa73d08b80 4 client.26819 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:13:47.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.667+0000 7faa73d08b80 1 -- 172.21.15.7:0/835450042 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56202be5bd50 con 0x56202bf17080 2024-06-10T01:13:47.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.667+0000 7faa62ffd700 1 --2- 172.21.15.7:0/835450042 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x56202bf17080 0x56202bf19470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:47.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.668+0000 7faa62ffd700 1 --2- 172.21.15.7:0/835450042 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x56202bf17080 0x56202bf19470 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d748736d4e05d378 server_cookie=43a025467f5ee44 in_seq=0 out_seq=0 2024-06-10T01:13:47.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.668+0000 7faa46ff5700 10 client.26819 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:13:47.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.669+0000 7faa46ff5700 1 -- 172.21.15.7:0/835450042 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56202be5bd50 con 0x56202bf17080 2024-06-10T01:13:47.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.669+0000 7faa46ff5700 10 client.26819 handle_command_reply: tid=0 2024-06-10T01:13:47.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.669+0000 7faa73d08b80 1 client.26819 shutdown 2024-06-10T01:13:47.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.669+0000 7faa73d08b80 1 -- 172.21.15.7:0/835450042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56202bf03880 msgr2=0x56202bf06e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:47.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.669+0000 7faa73d08b80 1 --2- 172.21.15.7:0/835450042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56202bf03880 0x56202bf06e30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0cc5f0 tx=0x7faa5c09b8d0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.669+0000 7faa73d08b80 1 -- 172.21.15.7:0/835450042 shutdown_connections 2024-06-10T01:13:47.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.669+0000 7faa73d08b80 1 --2- 172.21.15.7:0/835450042 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x56202bf17080 0x56202bf19470 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.669+0000 7faa73d08b80 1 --2- 172.21.15.7:0/835450042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56202bf03880 0x56202bf06e30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.669+0000 7faa73d08b80 1 --2- 172.21.15.7:0/835450042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56202befd7a0 0x56202bf095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.669+0000 7faa73d08b80 1 --2- 172.21.15.7:0/835450042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56202bf02f60 0x56202bf068f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.669+0000 7faa46ff5700 0 client.26819 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:13:47.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.669+0000 7faa73d08b80 1 -- 172.21.15.7:0/835450042 >> 172.21.15.7:0/835450042 conn(0x56202be2a080 msgr2=0x56202bef7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:47.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.669+0000 7faa73d08b80 1 -- 172.21.15.7:0/835450042 shutdown_connections 2024-06-10T01:13:47.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.669+0000 7faa73d08b80 1 -- 172.21.15.7:0/835450042 wait complete. 2024-06-10T01:13:47.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.669+0000 7faa73d08b80 20 client.26819 trim_cache size 0 max 16384 2024-06-10T01:13:47.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.686+0000 7faa73d08b80 1 Processor -- start 2024-06-10T01:13:47.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.686+0000 7faa73d08b80 1 -- start start 2024-06-10T01:13:47.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.686+0000 7faa73d08b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56202befd7a0 0x56202bef7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:47.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.686+0000 7faa73d08b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56202bf02f60 0x56202bef83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:47.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.686+0000 7faa73d08b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56202bf03880 0x56202bf01a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:47.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.686+0000 7faa73d08b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faa641b2e90 con 0x56202bf02f60 2024-06-10T01:13:47.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.686+0000 7faa73d08b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faa641b06c0 con 0x56202bf03880 2024-06-10T01:13:47.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.686+0000 7faa73d08b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faa641ae510 con 0x56202befd7a0 2024-06-10T01:13:47.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.686+0000 7faa627fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56202bf02f60 0x56202bef83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:47.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.686+0000 7faa627fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56202bf02f60 0x56202bef83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41580/0 (socket says 172.21.15.7:41580) 2024-06-10T01:13:47.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.686+0000 7faa627fc700 1 -- 172.21.15.7:0/3451458959 learned_addr learned my addr 172.21.15.7:0/3451458959 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:47.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.686+0000 7faa62ffd700 1 --2- 172.21.15.7:0/3451458959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56202befd7a0 0x56202bef7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:47.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.686+0000 7faa637fe700 1 --2- 172.21.15.7:0/3451458959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56202bf03880 0x56202bf01a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:47.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.686+0000 7faa627fc700 1 -- 172.21.15.7:0/3451458959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56202befd7a0 msgr2=0x56202bef7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:47.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.686+0000 7faa627fc700 1 --2- 172.21.15.7:0/3451458959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56202befd7a0 0x56202bef7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.686+0000 7faa627fc700 1 -- 172.21.15.7:0/3451458959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56202bf03880 msgr2=0x56202bf01a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:47.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.686+0000 7faa627fc700 1 --2- 172.21.15.7:0/3451458959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56202bf03880 0x56202bf01a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.686+0000 7faa627fc700 1 -- 172.21.15.7:0/3451458959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56202bf077f0 con 0x56202bf02f60 2024-06-10T01:13:47.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.686+0000 7faa62ffd700 1 --2- 172.21.15.7:0/3451458959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56202befd7a0 0x56202bef7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:47.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.687+0000 7faa627fc700 1 --2- 172.21.15.7:0/3451458959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56202bf02f60 0x56202bef83c0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7faa580099c0 tx=0x7faa580095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cdf8580b9ef0027 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:47.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.687+0000 7faa48ff9700 1 -- 172.21.15.7:0/3451458959 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa58028020 con 0x56202bf02f60 2024-06-10T01:13:47.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.687+0000 7faa48ff9700 1 -- 172.21.15.7:0/3451458959 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa58005070 con 0x56202bf02f60 2024-06-10T01:13:47.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.687+0000 7faa48ff9700 1 -- 172.21.15.7:0/3451458959 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5800f840 con 0x56202bf02f60 2024-06-10T01:13:47.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.688+0000 7faa73d08b80 1 -- 172.21.15.7:0/3451458959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56202bf02f60 msgr2=0x56202bef83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:47.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.688+0000 7faa73d08b80 1 --2- 172.21.15.7:0/3451458959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56202bf02f60 0x56202bef83c0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7faa580099c0 tx=0x7faa580095c0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.688+0000 7faa73d08b80 1 -- 172.21.15.7:0/3451458959 shutdown_connections 2024-06-10T01:13:47.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.688+0000 7faa73d08b80 1 --2- 172.21.15.7:0/3451458959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56202bf03880 0x56202bf01a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.688+0000 7faa73d08b80 1 --2- 172.21.15.7:0/3451458959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56202befd7a0 0x56202bef7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.688+0000 7faa73d08b80 1 --2- 172.21.15.7:0/3451458959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56202bf02f60 0x56202bef83c0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.688+0000 7faa73d08b80 1 -- 172.21.15.7:0/3451458959 >> 172.21.15.7:0/3451458959 conn(0x56202be2a080 msgr2=0x56202bf051e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:47.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.688+0000 7faa73d08b80 1 -- 172.21.15.7:0/3451458959 shutdown_connections 2024-06-10T01:13:47.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.688+0000 7faa73d08b80 1 -- 172.21.15.7:0/3451458959 wait complete. 2024-06-10T01:13:47.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.688+0000 7faa73d08b80 1 -- start start 2024-06-10T01:13:47.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.688+0000 7faa73d08b80 1 Processor -- start 2024-06-10T01:13:47.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.689+0000 7faa73d08b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56202befd7a0 0x56202bf0fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:47.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.689+0000 7faa73d08b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56202bf02f60 0x56202bf1e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:47.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.689+0000 7faa73d08b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56202bf03880 0x56202bf1ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:47.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.689+0000 7faa73d08b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faa641aeb80 con 0x56202befd7a0 2024-06-10T01:13:47.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.689+0000 7faa73d08b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faa641afff0 con 0x56202bf03880 2024-06-10T01:13:47.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.689+0000 7faa73d08b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faa641af250 con 0x56202bf02f60 2024-06-10T01:13:47.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.689+0000 7faa73d08b80 10 client.0 _ll_register_callbacks cb 0x56202bde45b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:13:47.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.689+0000 7faa62ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56202befd7a0 0x56202bf0fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:47.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.689+0000 7faa62ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56202befd7a0 0x56202bf0fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41592/0 (socket says 172.21.15.7:41592) 2024-06-10T01:13:47.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.689+0000 7faa62ffd700 1 -- 172.21.15.7:0/482103745 learned_addr learned my addr 172.21.15.7:0/482103745 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:47.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.689+0000 7faa637fe700 1 --2- 172.21.15.7:0/482103745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56202bf03880 0x56202bf1ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:47.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.689+0000 7faa627fc700 1 --2- 172.21.15.7:0/482103745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56202bf02f60 0x56202bf1e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:47.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.689+0000 7faa62ffd700 1 -- 172.21.15.7:0/482103745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56202bf02f60 msgr2=0x56202bf1e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:47.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.689+0000 7faa62ffd700 1 --2- 172.21.15.7:0/482103745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56202bf02f60 0x56202bf1e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.689+0000 7faa62ffd700 1 -- 172.21.15.7:0/482103745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56202bf03880 msgr2=0x56202bf1ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:47.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.689+0000 7faa62ffd700 1 --2- 172.21.15.7:0/482103745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56202bf03880 0x56202bf1ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.689+0000 7faa62ffd700 1 -- 172.21.15.7:0/482103745 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56202bf1f470 con 0x56202befd7a0 2024-06-10T01:13:47.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.689+0000 7faa637fe700 1 --2- 172.21.15.7:0/482103745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56202bf03880 0x56202bf1ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:47.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.689+0000 7faa62ffd700 1 --2- 172.21.15.7:0/482103745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56202befd7a0 0x56202bf0fcf0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x56202bf19bf0 tx=0x7faa5400ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80774543aaeddf54 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:47.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.689+0000 7faa46ff5700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:13:47.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.689+0000 7faa46ff5700 1 -- 172.21.15.7:0/482103745 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5401d410 con 0x56202befd7a0 2024-06-10T01:13:47.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.689+0000 7faa46ff5700 1 -- 172.21.15.7:0/482103745 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7faa5400ede0 con 0x56202befd7a0 2024-06-10T01:13:47.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.689+0000 7faa73d08b80 1 -- 172.21.15.7:0/482103745 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa58005070 con 0x56202befd7a0 2024-06-10T01:13:47.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.690+0000 7faa46ff5700 1 -- 172.21.15.7:0/482103745 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7faa54041050 con 0x56202befd7a0 2024-06-10T01:13:47.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.690+0000 7faa73d08b80 10 client.17409 fetch_fsmap learned FSMap version 35 2024-06-10T01:13:47.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.690+0000 7faa73d08b80 1 -- 172.21.15.7:0/482103745 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56202bf81d00 con 0x56202befd7a0 2024-06-10T01:13:47.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.690+0000 7faa46ff5700 1 -- 172.21.15.7:0/482103745 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa58005070 con 0x56202befd7a0 2024-06-10T01:13:47.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.690+0000 7faa46ff5700 1 -- 172.21.15.7:0/482103745 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7faa5401dad0 con 0x56202befd7a0 2024-06-10T01:13:47.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.690+0000 7faa73d08b80 10 client.17409 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:13:47.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.690+0000 7faa73d08b80 10 client.17409 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:13:47.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.690+0000 7faa73d08b80 20 client.17409 populate_metadata read hostname 'smithi007' 2024-06-10T01:13:47.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.690+0000 7faa73d08b80 1 --2- 172.21.15.7:0/482103745 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x56202bf82100 0x56202bf84520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:47.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.690+0000 7faa73d08b80 4 client.17409 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:13:47.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.690+0000 7faa73d08b80 1 -- 172.21.15.7:0/482103745 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56202be562d0 con 0x56202bf82100 2024-06-10T01:13:47.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.690+0000 7faa627fc700 1 --2- 172.21.15.7:0/482103745 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x56202bf82100 0x56202bf84520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:47.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.691+0000 7faa46ff5700 10 client.17409 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:13:47.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.691+0000 7faa627fc700 1 --2- 172.21.15.7:0/482103745 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x56202bf82100 0x56202bf84520 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=30b36fd1e6177df5 server_cookie=ac55d1ff3b46eeac in_seq=0 out_seq=0 2024-06-10T01:13:47.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.691+0000 7faa46ff5700 1 -- 172.21.15.7:0/482103745 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56202be562d0 con 0x56202bf82100 2024-06-10T01:13:47.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.691+0000 7faa46ff5700 10 client.17409 handle_command_reply: tid=0 2024-06-10T01:13:47.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.691+0000 7faa73d08b80 1 client.17409 shutdown 2024-06-10T01:13:47.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.691+0000 7faa73d08b80 1 -- 172.21.15.7:0/482103745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56202befd7a0 msgr2=0x56202bf0fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:47.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.691+0000 7faa73d08b80 1 --2- 172.21.15.7:0/482103745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56202befd7a0 0x56202bf0fcf0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x56202bf19bf0 tx=0x7faa5400ead0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.691+0000 7faa73d08b80 1 -- 172.21.15.7:0/482103745 shutdown_connections 2024-06-10T01:13:47.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.691+0000 7faa73d08b80 1 --2- 172.21.15.7:0/482103745 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x56202bf82100 0x56202bf84520 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.691+0000 7faa46ff5700 0 client.17409 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:13:47.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.691+0000 7faa73d08b80 1 --2- 172.21.15.7:0/482103745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56202bf03880 0x56202bf1ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.691+0000 7faa73d08b80 1 --2- 172.21.15.7:0/482103745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56202befd7a0 0x56202bf0fcf0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.691+0000 7faa73d08b80 1 --2- 172.21.15.7:0/482103745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56202bf02f60 0x56202bf1e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.691+0000 7faa73d08b80 1 -- 172.21.15.7:0/482103745 >> 172.21.15.7:0/482103745 conn(0x56202be2a080 msgr2=0x56202bf08120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:47.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.691+0000 7faa73d08b80 1 -- 172.21.15.7:0/482103745 shutdown_connections 2024-06-10T01:13:47.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.691+0000 7faa73d08b80 1 -- 172.21.15.7:0/482103745 wait complete. 2024-06-10T01:13:47.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.691+0000 7faa73d08b80 20 client.17409 trim_cache size 0 max 16384 2024-06-10T01:13:47.691 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:13:47.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.692+0000 7faa69485700 1 -- 172.21.15.7:0/576271780 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7faa4c0a2c20 msgr2=0x7faa4c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:47.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.692+0000 7faa69485700 1 --2- 172.21.15.7:0/576271780 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7faa4c0a2c20 0x7faa4c0a50e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7faa54009c60 tx=0x7faa54009260 comp rx=0 tx=0).stop 2024-06-10T01:13:47.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.692+0000 7faa69485700 1 -- 172.21.15.7:0/576271780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa64107860 msgr2=0x7faa64075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:47.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.692+0000 7faa69485700 1 --2- 172.21.15.7:0/576271780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa64107860 0x7faa64075210 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7faa5c00ef30 tx=0x7faa5c00c3a0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.692+0000 7faa69485700 1 -- 172.21.15.7:0/576271780 shutdown_connections 2024-06-10T01:13:47.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.692+0000 7faa69485700 1 --2- 172.21.15.7:0/576271780 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7faa4c0a2c20 0x7faa4c0a50e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.692+0000 7faa69485700 1 --2- 172.21.15.7:0/576271780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa64107860 0x7faa64075210 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.692+0000 7faa69485700 1 --2- 172.21.15.7:0/576271780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa64102770 0x7faa64074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.692+0000 7faa69485700 1 --2- 172.21.15.7:0/576271780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa64103120 0x7faa64074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:47.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.692+0000 7faa69485700 1 -- 172.21.15.7:0/576271780 >> 172.21.15.7:0/576271780 conn(0x7faa64079600 msgr2=0x7faa64100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:47.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.692+0000 7faa69485700 1 -- 172.21.15.7:0/576271780 shutdown_connections 2024-06-10T01:13:47.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:47.692+0000 7faa69485700 1 -- 172.21.15.7:0/576271780 wait complete. 2024-06-10T01:13:47.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:47 smithi175 ceph-mon[36636]: pgmap v665: 129 pgs: 129 active+clean; 103 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 315 op/s 2024-06-10T01:13:47.739 INFO:tasks.workunit.client.0.smithi007.stdout: 1 561991 7.86 MB/sec execute 268 sec latency 57.342 ms 2024-06-10T01:13:47.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:47 smithi078 ceph-mon[36683]: pgmap v665: 129 pgs: 129 active+clean; 103 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 315 op/s 2024-06-10T01:13:48.337 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:13:48.605 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:13:48.739 INFO:tasks.workunit.client.0.smithi007.stdout: 1 564555 7.88 MB/sec execute 269 sec latency 22.007 ms 2024-06-10T01:13:49.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:49 smithi175 ceph-mon[36636]: pgmap v666: 129 pgs: 129 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 341 op/s 2024-06-10T01:13:49.739 INFO:tasks.workunit.client.0.smithi007.stdout: 1 566052 7.88 MB/sec execute 270 sec latency 15.685 ms 2024-06-10T01:13:49.742 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:49 smithi007 ceph-mon[32398]: pgmap v666: 129 pgs: 129 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 341 op/s 2024-06-10T01:13:49.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:49 smithi078 ceph-mon[36683]: pgmap v666: 129 pgs: 129 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 341 op/s 2024-06-10T01:13:50.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.343+0000 7f4f5cc06700 1 -- 172.21.15.7:0/2203556863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f58103170 msgr2=0x7f4f5810f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:50.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.343+0000 7f4f5cc06700 1 --2- 172.21.15.7:0/2203556863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f58103170 0x7f4f5810f3e0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f4f5000b0a0 tx=0x7f4f5000b3b0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.343+0000 7f4f5cc06700 1 -- 172.21.15.7:0/2203556863 shutdown_connections 2024-06-10T01:13:50.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.343+0000 7f4f5cc06700 1 --2- 172.21.15.7:0/2203556863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f58103170 0x7f4f5810f3e0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.343+0000 7f4f5cc06700 1 --2- 172.21.15.7:0/2203556863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f58108650 0x7f4f58108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.343+0000 7f4f5cc06700 1 --2- 172.21.15.7:0/2203556863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f581027b0 0x7f4f58102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.343+0000 7f4f5cc06700 1 -- 172.21.15.7:0/2203556863 >> 172.21.15.7:0/2203556863 conn(0x7f4f5807a420 msgr2=0x7f4f5807a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:50.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.343+0000 7f4f5cc06700 1 -- 172.21.15.7:0/2203556863 shutdown_connections 2024-06-10T01:13:50.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.343+0000 7f4f5cc06700 1 -- 172.21.15.7:0/2203556863 wait complete. 2024-06-10T01:13:50.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.344+0000 7f4f5cc06700 1 Processor -- start 2024-06-10T01:13:50.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.344+0000 7f4f5cc06700 1 -- start start 2024-06-10T01:13:50.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.344+0000 7f4f5cc06700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f581027b0 0x7f4f58074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:50.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.344+0000 7f4f5cc06700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f58103170 0x7f4f58074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:50.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.344+0000 7f4f5cc06700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f58108650 0x7f4f58075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:50.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.344+0000 7f4f5cc06700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4f581128b0 con 0x7f4f581027b0 2024-06-10T01:13:50.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.344+0000 7f4f5cc06700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4f58112730 con 0x7f4f58103170 2024-06-10T01:13:50.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.344+0000 7f4f5cc06700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4f58112a30 con 0x7f4f58108650 2024-06-10T01:13:50.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.344+0000 7f4f5659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f581027b0 0x7f4f58074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:50.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.344+0000 7f4f5659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f581027b0 0x7f4f58074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41604/0 (socket says 172.21.15.7:41604) 2024-06-10T01:13:50.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.344+0000 7f4f5659c700 1 -- 172.21.15.7:0/328467153 learned_addr learned my addr 172.21.15.7:0/328467153 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:50.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.344+0000 7f4f55d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f58103170 0x7f4f58074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:50.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.344+0000 7f4f56d9d700 1 --2- 172.21.15.7:0/328467153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f58108650 0x7f4f58075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:50.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.345+0000 7f4f5659c700 1 -- 172.21.15.7:0/328467153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f58108650 msgr2=0x7f4f58075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:50.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.345+0000 7f4f5659c700 1 --2- 172.21.15.7:0/328467153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f58108650 0x7f4f58075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.345+0000 7f4f5659c700 1 -- 172.21.15.7:0/328467153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f58103170 msgr2=0x7f4f58074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:50.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.345+0000 7f4f5659c700 1 --2- 172.21.15.7:0/328467153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f58103170 0x7f4f58074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.345+0000 7f4f5659c700 1 -- 172.21.15.7:0/328467153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f581a7190 con 0x7f4f581027b0 2024-06-10T01:13:50.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.345+0000 7f4f56d9d700 1 --2- 172.21.15.7:0/328467153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f58108650 0x7f4f58075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:50.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.345+0000 7f4f55d9b700 1 --2- 172.21.15.7:0/328467153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f58103170 0x7f4f58074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:50.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.345+0000 7f4f5659c700 1 --2- 172.21.15.7:0/328467153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f581027b0 0x7f4f58074790 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f4f4800c920 tx=0x7f4f4800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:50.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.345+0000 7f4f477fe700 1 -- 172.21.15.7:0/328467153 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f48004a30 con 0x7f4f581027b0 2024-06-10T01:13:50.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.345+0000 7f4f477fe700 1 -- 172.21.15.7:0/328467153 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f48005500 con 0x7f4f581027b0 2024-06-10T01:13:50.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.345+0000 7f4f477fe700 1 -- 172.21.15.7:0/328467153 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f48007d50 con 0x7f4f581027b0 2024-06-10T01:13:50.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.345+0000 7f4f5cc06700 1 -- 172.21.15.7:0/328467153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f581a7480 con 0x7f4f581027b0 2024-06-10T01:13:50.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.346+0000 7f4f5cc06700 1 -- 172.21.15.7:0/328467153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f581a7970 con 0x7f4f581027b0 2024-06-10T01:13:50.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.348+0000 7f4f477fe700 1 -- 172.21.15.7:0/328467153 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f4f48004d60 con 0x7f4f581027b0 2024-06-10T01:13:50.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.349+0000 7f4f477fe700 1 --2- 172.21.15.7:0/328467153 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4f400a2ca0 0x7f4f400a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:50.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.349+0000 7f4f477fe700 1 -- 172.21.15.7:0/328467153 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f4f480cedc0 con 0x7f4f581027b0 2024-06-10T01:13:50.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.349+0000 7f4f55d9b700 1 --2- 172.21.15.7:0/328467153 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4f400a2ca0 0x7f4f400a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:50.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.349+0000 7f4f67489b80 1 Processor -- start 2024-06-10T01:13:50.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.349+0000 7f4f55d9b700 1 --2- 172.21.15.7:0/328467153 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4f400a2ca0 0x7f4f400a5160 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4f5807a920 tx=0x7f4f4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:50.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.350+0000 7f4f67489b80 1 -- start start 2024-06-10T01:13:50.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.350+0000 7f4f67489b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f912e3640 0x557f912e3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:50.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.350+0000 7f4f67489b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f912e3f60 0x557f912dd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:50.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.350+0000 7f4f67489b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f912ddf20 0x557f912de340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:50.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.350+0000 7f4f67489b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557f9122f810 con 0x557f912e3640 2024-06-10T01:13:50.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.350+0000 7f4f67489b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557f911d7720 con 0x557f912ddf20 2024-06-10T01:13:50.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.350+0000 7f4f67489b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557f9122b7d0 con 0x557f912e3f60 2024-06-10T01:13:50.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.350+0000 7f4f55d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f912e3f60 0x557f912dd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:50.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.350+0000 7f4f56d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f912ddf20 0x557f912de340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:50.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.350+0000 7f4f55d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f912e3f60 0x557f912dd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34462/0 (socket says 172.21.15.7:34462) 2024-06-10T01:13:50.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.350+0000 7f4f55d9b700 1 -- 172.21.15.7:0/1632271212 learned_addr learned my addr 172.21.15.7:0/1632271212 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:50.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.351+0000 7f4f5659c700 1 --2- 172.21.15.7:0/1632271212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f912e3640 0x557f912e3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:50.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.351+0000 7f4f56d9d700 1 -- 172.21.15.7:0/1632271212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f912e3f60 msgr2=0x557f912dd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:50.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.351+0000 7f4f56d9d700 1 --2- 172.21.15.7:0/1632271212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f912e3f60 0x557f912dd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.351+0000 7f4f56d9d700 1 -- 172.21.15.7:0/1632271212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f912e3640 msgr2=0x557f912e3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:50.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.351+0000 7f4f56d9d700 1 --2- 172.21.15.7:0/1632271212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f912e3640 0x557f912e3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.351+0000 7f4f56d9d700 1 -- 172.21.15.7:0/1632271212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f911cdf50 con 0x557f912ddf20 2024-06-10T01:13:50.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.351+0000 7f4f56d9d700 1 --2- 172.21.15.7:0/1632271212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f912ddf20 0x557f912de340 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4f50009fd0 tx=0x7f4f500092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25f5405ff8d665fc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:50.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.351+0000 7f4f44ff9700 1 -- 172.21.15.7:0/1632271212 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50021070 con 0x557f912ddf20 2024-06-10T01:13:50.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.351+0000 7f4f44ff9700 1 -- 172.21.15.7:0/1632271212 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f50003c00 con 0x557f912ddf20 2024-06-10T01:13:50.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.351+0000 7f4f44ff9700 1 -- 172.21.15.7:0/1632271212 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f500052a0 con 0x557f912ddf20 2024-06-10T01:13:50.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.353+0000 7f4f67489b80 1 -- 172.21.15.7:0/1632271212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f912ddf20 msgr2=0x557f912de340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:50.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.353+0000 7f4f67489b80 1 --2- 172.21.15.7:0/1632271212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f912ddf20 0x557f912de340 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4f50009fd0 tx=0x7f4f500092e0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.353+0000 7f4f67489b80 1 -- 172.21.15.7:0/1632271212 shutdown_connections 2024-06-10T01:13:50.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.353+0000 7f4f67489b80 1 --2- 172.21.15.7:0/1632271212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f912ddf20 0x557f912de340 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.353+0000 7f4f67489b80 1 --2- 172.21.15.7:0/1632271212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f912e3640 0x557f912e3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.353+0000 7f4f67489b80 1 --2- 172.21.15.7:0/1632271212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f912e3f60 0x557f912dd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.353+0000 7f4f67489b80 1 -- 172.21.15.7:0/1632271212 >> 172.21.15.7:0/1632271212 conn(0x557f91208cc0 msgr2=0x557f912d8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:50.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.353+0000 7f4f67489b80 1 -- 172.21.15.7:0/1632271212 shutdown_connections 2024-06-10T01:13:50.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.353+0000 7f4f67489b80 1 -- 172.21.15.7:0/1632271212 wait complete. 2024-06-10T01:13:50.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.354+0000 7f4f67489b80 1 -- start start 2024-06-10T01:13:50.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.354+0000 7f4f67489b80 1 Processor -- start 2024-06-10T01:13:50.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.354+0000 7f4f67489b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f912ddf20 0x557f912e9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:50.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.354+0000 7f4f67489b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f912e3640 0x557f912e6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:50.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.354+0000 7f4f67489b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f912e3f60 0x557f912e74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:50.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.354+0000 7f4f67489b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557f912ea970 con 0x557f912ddf20 2024-06-10T01:13:50.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.354+0000 7f4f67489b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557f911cfbf0 con 0x557f912e3640 2024-06-10T01:13:50.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.354+0000 7f4f67489b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557f912eaaf0 con 0x557f912e3f60 2024-06-10T01:13:50.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.354+0000 7f4f67489b80 10 client.0 _ll_register_callbacks cb 0x557f911c9920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:13:50.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.354+0000 7f4f55d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f912e3640 0x557f912e6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:50.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.354+0000 7f4f55d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f912e3640 0x557f912e6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:47302/0 (socket says 172.21.15.7:47302) 2024-06-10T01:13:50.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.354+0000 7f4f55d9b700 1 -- 172.21.15.7:0/1018901280 learned_addr learned my addr 172.21.15.7:0/1018901280 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:50.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.355+0000 7f4f56d9d700 1 --2- 172.21.15.7:0/1018901280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f912e3f60 0x557f912e74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:50.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.355+0000 7f4f5659c700 1 --2- 172.21.15.7:0/1018901280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f912ddf20 0x557f912e9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:50.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.355+0000 7f4f55d9b700 1 -- 172.21.15.7:0/1018901280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f912e3f60 msgr2=0x557f912e74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:50.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.355+0000 7f4f55d9b700 1 --2- 172.21.15.7:0/1018901280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f912e3f60 0x557f912e74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.355+0000 7f4f55d9b700 1 -- 172.21.15.7:0/1018901280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f912ddf20 msgr2=0x557f912e9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:50.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.355+0000 7f4f55d9b700 1 --2- 172.21.15.7:0/1018901280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f912ddf20 0x557f912e9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.355+0000 7f4f55d9b700 1 -- 172.21.15.7:0/1018901280 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f9117e8d0 con 0x557f912e3640 2024-06-10T01:13:50.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.355+0000 7f4f56d9d700 1 --2- 172.21.15.7:0/1018901280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f912e3f60 0x557f912e74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:50.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.355+0000 7f4f5659c700 1 --2- 172.21.15.7:0/1018901280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f912ddf20 0x557f912e9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:13:50.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.355+0000 7f4f55d9b700 1 --2- 172.21.15.7:0/1018901280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f912e3640 0x557f912e6fa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c005930 tx=0x7f4f4c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b456ffcca62eef server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:50.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.355+0000 7f4f3dffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:13:50.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.355+0000 7f4f3dffb700 1 -- 172.21.15.7:0/1018901280 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c01f780 con 0x557f912e3640 2024-06-10T01:13:50.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.355+0000 7f4f3dffb700 1 -- 172.21.15.7:0/1018901280 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f4f4c01f920 con 0x557f912e3640 2024-06-10T01:13:50.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.355+0000 7f4f67489b80 1 -- 172.21.15.7:0/1018901280 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f50003c00 con 0x557f912e3640 2024-06-10T01:13:50.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.355+0000 7f4f3dffb700 1 -- 172.21.15.7:0/1018901280 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4f4c01fb20 con 0x557f912e3640 2024-06-10T01:13:50.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.355+0000 7f4f67489b80 10 client.26834 fetch_fsmap learned FSMap version 35 2024-06-10T01:13:50.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.355+0000 7f4f67489b80 1 -- 172.21.15.7:0/1018901280 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557f912f5220 con 0x557f912e3640 2024-06-10T01:13:50.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.356+0000 7f4f3dffb700 1 -- 172.21.15.7:0/1018901280 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f50003c00 con 0x557f912e3640 2024-06-10T01:13:50.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.356+0000 7f4f3dffb700 1 -- 172.21.15.7:0/1018901280 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f4f4c026020 con 0x557f912e3640 2024-06-10T01:13:50.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.356+0000 7f4f67489b80 10 client.26834 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:13:50.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.356+0000 7f4f67489b80 10 client.26834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:13:50.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.356+0000 7f4f67489b80 20 client.26834 populate_metadata read hostname 'smithi007' 2024-06-10T01:13:50.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.356+0000 7f4f67489b80 1 --2- 172.21.15.7:0/1018901280 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f912f7860 0x557f912f9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:50.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.356+0000 7f4f67489b80 4 client.26834 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:13:50.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.356+0000 7f4f67489b80 1 -- 172.21.15.7:0/1018901280 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f9123cd10 con 0x557f912f7860 2024-06-10T01:13:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.356+0000 7f4f5659c700 1 --2- 172.21.15.7:0/1018901280 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f912f7860 0x557f912f9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.357+0000 7f4f3dffb700 10 client.26834 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.357+0000 7f4f5659c700 1 --2- 172.21.15.7:0/1018901280 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f912f7860 0x557f912f9c50 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2257c22ad5f53fd server_cookie=318d78723787dd14 in_seq=0 out_seq=0 2024-06-10T01:13:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.357+0000 7f4f3dffb700 1 -- 172.21.15.7:0/1018901280 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557f9123cd10 con 0x557f912f7860 2024-06-10T01:13:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.357+0000 7f4f3dffb700 10 client.26834 handle_command_reply: tid=0 2024-06-10T01:13:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.357+0000 7f4f67489b80 1 client.26834 shutdown 2024-06-10T01:13:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.357+0000 7f4f67489b80 1 -- 172.21.15.7:0/1018901280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f912e3640 msgr2=0x557f912e6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.357+0000 7f4f67489b80 1 --2- 172.21.15.7:0/1018901280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f912e3640 0x557f912e6fa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c005930 tx=0x7f4f4c0047c0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.358+0000 7f4f67489b80 1 -- 172.21.15.7:0/1018901280 shutdown_connections 2024-06-10T01:13:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.358+0000 7f4f67489b80 1 --2- 172.21.15.7:0/1018901280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f912e3f60 0x557f912e74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.358+0000 7f4f67489b80 1 --2- 172.21.15.7:0/1018901280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f912ddf20 0x557f912e9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.358+0000 7f4f67489b80 1 --2- 172.21.15.7:0/1018901280 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f912f7860 0x557f912f9c50 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.358+0000 7f4f3dffb700 0 client.26834 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.358+0000 7f4f67489b80 1 --2- 172.21.15.7:0/1018901280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f912e3640 0x557f912e6fa0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.358+0000 7f4f67489b80 1 -- 172.21.15.7:0/1018901280 >> 172.21.15.7:0/1018901280 conn(0x557f91208cc0 msgr2=0x557f912dc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:50.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.358+0000 7f4f67489b80 1 -- 172.21.15.7:0/1018901280 shutdown_connections 2024-06-10T01:13:50.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.358+0000 7f4f67489b80 1 -- 172.21.15.7:0/1018901280 wait complete. 2024-06-10T01:13:50.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.358+0000 7f4f67489b80 20 client.26834 trim_cache size 0 max 16384 2024-06-10T01:13:50.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.376+0000 7f4f67489b80 1 Processor -- start 2024-06-10T01:13:50.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.376+0000 7f4f67489b80 1 -- start start 2024-06-10T01:13:50.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.376+0000 7f4f67489b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f912ddf20 0x557f912dd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:50.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.376+0000 7f4f67489b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f912e3640 0x557f912e5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:50.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.376+0000 7f4f67489b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f912e3f60 0x557f912e5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:50.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.376+0000 7f4f67489b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4f581b2df0 con 0x557f912ddf20 2024-06-10T01:13:50.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.376+0000 7f4f67489b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4f581b0620 con 0x557f912e3640 2024-06-10T01:13:50.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.376+0000 7f4f67489b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4f581ae470 con 0x557f912e3f60 2024-06-10T01:13:50.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.376+0000 7f4f5659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f912ddf20 0x557f912dd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:50.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.376+0000 7f4f5659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f912ddf20 0x557f912dd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41634/0 (socket says 172.21.15.7:41634) 2024-06-10T01:13:50.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.376+0000 7f4f5659c700 1 -- 172.21.15.7:0/817884736 learned_addr learned my addr 172.21.15.7:0/817884736 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:50.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.376+0000 7f4f55d9b700 1 --2- 172.21.15.7:0/817884736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f912e3640 0x557f912e5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:50.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.376+0000 7f4f56d9d700 1 --2- 172.21.15.7:0/817884736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f912e3f60 0x557f912e5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:50.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.376+0000 7f4f5659c700 1 -- 172.21.15.7:0/817884736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f912e3f60 msgr2=0x557f912e5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:50.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.376+0000 7f4f5659c700 1 --2- 172.21.15.7:0/817884736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f912e3f60 0x557f912e5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.376+0000 7f4f5659c700 1 -- 172.21.15.7:0/817884736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f912e3640 msgr2=0x557f912e5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:50.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.376+0000 7f4f5659c700 1 --2- 172.21.15.7:0/817884736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f912e3640 0x557f912e5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.376+0000 7f4f5659c700 1 -- 172.21.15.7:0/817884736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f912de690 con 0x557f912ddf20 2024-06-10T01:13:50.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.376+0000 7f4f55d9b700 1 --2- 172.21.15.7:0/817884736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f912e3640 0x557f912e5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:50.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.377+0000 7f4f5659c700 1 --2- 172.21.15.7:0/817884736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f912ddf20 0x557f912dd9e0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f4f4809b230 tx=0x7f4f4800a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f83a0629758660 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:50.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.377+0000 7f4f3ffff700 1 -- 172.21.15.7:0/817884736 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4801d040 con 0x557f912ddf20 2024-06-10T01:13:50.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.377+0000 7f4f3ffff700 1 -- 172.21.15.7:0/817884736 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f4809f650 con 0x557f912ddf20 2024-06-10T01:13:50.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.377+0000 7f4f3ffff700 1 -- 172.21.15.7:0/817884736 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f480a3a80 con 0x557f912ddf20 2024-06-10T01:13:50.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.378+0000 7f4f67489b80 1 -- 172.21.15.7:0/817884736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f912ddf20 msgr2=0x557f912dd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:50.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.378+0000 7f4f67489b80 1 --2- 172.21.15.7:0/817884736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f912ddf20 0x557f912dd9e0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f4f4809b230 tx=0x7f4f4800a470 comp rx=0 tx=0).stop 2024-06-10T01:13:50.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.378+0000 7f4f67489b80 1 -- 172.21.15.7:0/817884736 shutdown_connections 2024-06-10T01:13:50.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.378+0000 7f4f67489b80 1 --2- 172.21.15.7:0/817884736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f912e3f60 0x557f912e5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.378+0000 7f4f67489b80 1 --2- 172.21.15.7:0/817884736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f912ddf20 0x557f912dd9e0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.378+0000 7f4f67489b80 1 --2- 172.21.15.7:0/817884736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f912e3640 0x557f912e5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.378+0000 7f4f67489b80 1 -- 172.21.15.7:0/817884736 >> 172.21.15.7:0/817884736 conn(0x557f91208cc0 msgr2=0x557f912d8380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:50.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.378+0000 7f4f67489b80 1 -- 172.21.15.7:0/817884736 shutdown_connections 2024-06-10T01:13:50.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.378+0000 7f4f67489b80 1 -- 172.21.15.7:0/817884736 wait complete. 2024-06-10T01:13:50.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.378+0000 7f4f67489b80 1 -- start start 2024-06-10T01:13:50.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.378+0000 7f4f67489b80 1 Processor -- start 2024-06-10T01:13:50.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.379+0000 7f4f67489b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f912ddf20 0x557f912f02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:50.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.379+0000 7f4f67489b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f912e3640 0x557f912fec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:50.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.379+0000 7f4f67489b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f912e3f60 0x557f912ff1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:50.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.379+0000 7f4f67489b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4f581aeae0 con 0x557f912ddf20 2024-06-10T01:13:50.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.379+0000 7f4f67489b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4f581aff50 con 0x557f912e3640 2024-06-10T01:13:50.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.379+0000 7f4f67489b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4f581af1b0 con 0x557f912e3f60 2024-06-10T01:13:50.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.379+0000 7f4f67489b80 10 client.0 _ll_register_callbacks cb 0x557f911c9920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:13:50.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.379+0000 7f4f5659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f912ddf20 0x557f912f02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:50.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.379+0000 7f4f5659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f912ddf20 0x557f912f02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41650/0 (socket says 172.21.15.7:41650) 2024-06-10T01:13:50.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.379+0000 7f4f5659c700 1 -- 172.21.15.7:0/2473946586 learned_addr learned my addr 172.21.15.7:0/2473946586 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:50.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.379+0000 7f4f56d9d700 1 --2- 172.21.15.7:0/2473946586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f912e3f60 0x557f912ff1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:50.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.379+0000 7f4f55d9b700 1 --2- 172.21.15.7:0/2473946586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f912e3640 0x557f912fec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:50.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.379+0000 7f4f5659c700 1 -- 172.21.15.7:0/2473946586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f912e3f60 msgr2=0x557f912ff1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:50.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.379+0000 7f4f5659c700 1 --2- 172.21.15.7:0/2473946586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f912e3f60 0x557f912ff1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.379+0000 7f4f5659c700 1 -- 172.21.15.7:0/2473946586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f912e3640 msgr2=0x557f912fec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:50.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.379+0000 7f4f5659c700 1 --2- 172.21.15.7:0/2473946586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f912e3640 0x557f912fec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.379+0000 7f4f5659c700 1 -- 172.21.15.7:0/2473946586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f912ff7a0 con 0x557f912ddf20 2024-06-10T01:13:50.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.379+0000 7f4f55d9b700 1 --2- 172.21.15.7:0/2473946586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f912e3640 0x557f912fec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:13:50.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.379+0000 7f4f5659c700 1 --2- 172.21.15.7:0/2473946586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f912ddf20 0x557f912f02a0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f4f48097c70 tx=0x7f4f48005930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8ceec8bcee431e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:50.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.379+0000 7f4f3dffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:13:50.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.380+0000 7f4f3dffb700 1 -- 172.21.15.7:0/2473946586 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f480b4060 con 0x557f912ddf20 2024-06-10T01:13:50.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.380+0000 7f4f3dffb700 1 -- 172.21.15.7:0/2473946586 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f4f4809b500 con 0x557f912ddf20 2024-06-10T01:13:50.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.380+0000 7f4f67489b80 1 -- 172.21.15.7:0/2473946586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f4809f650 con 0x557f912ddf20 2024-06-10T01:13:50.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.380+0000 7f4f3dffb700 1 -- 172.21.15.7:0/2473946586 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4f4801d040 con 0x557f912ddf20 2024-06-10T01:13:50.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.380+0000 7f4f67489b80 10 client.17427 fetch_fsmap learned FSMap version 35 2024-06-10T01:13:50.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.380+0000 7f4f67489b80 1 -- 172.21.15.7:0/2473946586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557f91362310 con 0x557f912ddf20 2024-06-10T01:13:50.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.380+0000 7f4f3dffb700 1 -- 172.21.15.7:0/2473946586 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f4809f650 con 0x557f912ddf20 2024-06-10T01:13:50.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.380+0000 7f4f3dffb700 1 -- 172.21.15.7:0/2473946586 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f4f4809f190 con 0x557f912ddf20 2024-06-10T01:13:50.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.380+0000 7f4f67489b80 10 client.17427 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:13:50.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.380+0000 7f4f67489b80 10 client.17427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:13:50.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.380+0000 7f4f67489b80 20 client.17427 populate_metadata read hostname 'smithi007' 2024-06-10T01:13:50.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.380+0000 7f4f67489b80 1 --2- 172.21.15.7:0/2473946586 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f91362a60 0x557f91364e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:50.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.380+0000 7f4f67489b80 4 client.17427 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:13:50.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.380+0000 7f4f67489b80 1 -- 172.21.15.7:0/2473946586 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f91237290 con 0x557f91362a60 2024-06-10T01:13:50.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.381+0000 7f4f55d9b700 1 --2- 172.21.15.7:0/2473946586 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f91362a60 0x557f91364e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:50.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.381+0000 7f4f55d9b700 1 --2- 172.21.15.7:0/2473946586 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f91362a60 0x557f91364e80 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ae1e25b7c20f03c server_cookie=ad7e92f86bce647c in_seq=0 out_seq=0 2024-06-10T01:13:50.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.381+0000 7f4f3dffb700 10 client.17427 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:50.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.381+0000 7f4f3dffb700 1 -- 172.21.15.7:0/2473946586 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f91237290 con 0x557f91362a60 2024-06-10T01:13:50.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.381+0000 7f4f3dffb700 10 client.17427 handle_command_reply: tid=0 2024-06-10T01:13:50.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.381+0000 7f4f67489b80 1 client.17427 shutdown 2024-06-10T01:13:50.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.381+0000 7f4f67489b80 1 -- 172.21.15.7:0/2473946586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f912ddf20 msgr2=0x557f912f02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:50.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.381+0000 7f4f67489b80 1 --2- 172.21.15.7:0/2473946586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f912ddf20 0x557f912f02a0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f4f48097c70 tx=0x7f4f48005930 comp rx=0 tx=0).stop 2024-06-10T01:13:50.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.381+0000 7f4f67489b80 1 -- 172.21.15.7:0/2473946586 shutdown_connections 2024-06-10T01:13:50.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.381+0000 7f4f67489b80 1 --2- 172.21.15.7:0/2473946586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f912e3f60 0x557f912ff1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.381+0000 7f4f67489b80 1 --2- 172.21.15.7:0/2473946586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f912ddf20 0x557f912f02a0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.381+0000 7f4f67489b80 1 --2- 172.21.15.7:0/2473946586 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f91362a60 0x557f91364e80 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.381+0000 7f4f67489b80 1 --2- 172.21.15.7:0/2473946586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f912e3640 0x557f912fec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.381+0000 7f4f67489b80 1 -- 172.21.15.7:0/2473946586 >> 172.21.15.7:0/2473946586 conn(0x557f91208cc0 msgr2=0x557f912e5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:50.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.381+0000 7f4f3dffb700 0 client.17427 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:50.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.382+0000 7f4f67489b80 1 -- 172.21.15.7:0/2473946586 shutdown_connections 2024-06-10T01:13:50.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.382+0000 7f4f67489b80 1 -- 172.21.15.7:0/2473946586 wait complete. 2024-06-10T01:13:50.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.382+0000 7f4f67489b80 20 client.17427 trim_cache size 0 max 16384 2024-06-10T01:13:50.382 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:13:50.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.382+0000 7f4f5cc06700 1 -- 172.21.15.7:0/328467153 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4f400a2ca0 msgr2=0x7f4f400a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:50.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.382+0000 7f4f5cc06700 1 --2- 172.21.15.7:0/328467153 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4f400a2ca0 0x7f4f400a5160 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4f5807a920 tx=0x7f4f4c009260 comp rx=0 tx=0).stop 2024-06-10T01:13:50.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.382+0000 7f4f5cc06700 1 -- 172.21.15.7:0/328467153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f581027b0 msgr2=0x7f4f58074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:50.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.382+0000 7f4f5cc06700 1 --2- 172.21.15.7:0/328467153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f581027b0 0x7f4f58074790 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f4f4800c920 tx=0x7f4f4800cc30 comp rx=0 tx=0).stop 2024-06-10T01:13:50.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.382+0000 7f4f5cc06700 1 -- 172.21.15.7:0/328467153 shutdown_connections 2024-06-10T01:13:50.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.382+0000 7f4f5cc06700 1 --2- 172.21.15.7:0/328467153 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4f400a2ca0 0x7f4f400a5160 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.382+0000 7f4f5cc06700 1 --2- 172.21.15.7:0/328467153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f58108650 0x7f4f58075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.382+0000 7f4f5cc06700 1 --2- 172.21.15.7:0/328467153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f581027b0 0x7f4f58074790 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.382+0000 7f4f5cc06700 1 --2- 172.21.15.7:0/328467153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f58103170 0x7f4f58074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:50.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.382+0000 7f4f5cc06700 1 -- 172.21.15.7:0/328467153 >> 172.21.15.7:0/328467153 conn(0x7f4f5807a420 msgr2=0x7f4f58101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:50.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.382+0000 7f4f5cc06700 1 -- 172.21.15.7:0/328467153 shutdown_connections 2024-06-10T01:13:50.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:50.382+0000 7f4f5cc06700 1 -- 172.21.15.7:0/328467153 wait complete. 2024-06-10T01:13:50.739 INFO:tasks.workunit.client.0.smithi007.stdout: 1 568597 7.91 MB/sec execute 271 sec latency 22.275 ms 2024-06-10T01:13:51.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:51 smithi175 ceph-mon[36636]: pgmap v667: 129 pgs: 129 active+clean; 107 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 323 op/s 2024-06-10T01:13:51.739 INFO:tasks.workunit.client.0.smithi007.stdout: 1 570003 7.90 MB/sec execute 272 sec latency 78.037 ms 2024-06-10T01:13:51.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:51 smithi007 ceph-mon[32398]: pgmap v667: 129 pgs: 129 active+clean; 107 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 323 op/s 2024-06-10T01:13:51.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:51 smithi078 ceph-mon[36683]: pgmap v667: 129 pgs: 129 active+clean; 107 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 323 op/s 2024-06-10T01:13:52.240 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:13:52.297 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:13:52.562 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:13:52.739 INFO:tasks.workunit.client.0.smithi007.stdout: 1 572256 7.92 MB/sec execute 273 sec latency 17.492 ms 2024-06-10T01:13:53.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:53 smithi175 ceph-mon[36636]: pgmap v668: 129 pgs: 129 active+clean; 101 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 348 op/s 2024-06-10T01:13:53.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:53 smithi007 ceph-mon[32398]: pgmap v668: 129 pgs: 129 active+clean; 101 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 348 op/s 2024-06-10T01:13:53.739 INFO:tasks.workunit.client.0.smithi007.stdout: 1 574386 7.92 MB/sec execute 274 sec latency 15.759 ms 2024-06-10T01:13:53.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:53 smithi078 ceph-mon[36683]: pgmap v668: 129 pgs: 129 active+clean; 101 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 348 op/s 2024-06-10T01:13:54.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.291+0000 7fa29cf41700 1 -- 172.21.15.7:0/2847643926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa298102ea0 msgr2=0x7fa29810f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:54.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.291+0000 7fa29cf41700 1 --2- 172.21.15.7:0/2847643926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa298102ea0 0x7fa29810f380 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fa29000b0a0 tx=0x7fa29000b3b0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.291+0000 7fa29cf41700 1 -- 172.21.15.7:0/2847643926 shutdown_connections 2024-06-10T01:13:54.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.291+0000 7fa29cf41700 1 --2- 172.21.15.7:0/2847643926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa298102ea0 0x7fa29810f380 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.291+0000 7fa29cf41700 1 --2- 172.21.15.7:0/2847643926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2981085f0 0x7fa2981089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.291+0000 7fa29cf41700 1 --2- 172.21.15.7:0/2847643926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2981024e0 0x7fa298102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.291+0000 7fa29cf41700 1 -- 172.21.15.7:0/2847643926 >> 172.21.15.7:0/2847643926 conn(0x7fa298079640 msgr2=0x7fa298079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:54.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.291+0000 7fa29cf41700 1 -- 172.21.15.7:0/2847643926 shutdown_connections 2024-06-10T01:13:54.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.291+0000 7fa29cf41700 1 -- 172.21.15.7:0/2847643926 wait complete. 2024-06-10T01:13:54.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.292+0000 7fa29cf41700 1 Processor -- start 2024-06-10T01:13:54.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.292+0000 7fa29cf41700 1 -- start start 2024-06-10T01:13:54.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.292+0000 7fa29cf41700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2981024e0 0x7fa29819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:54.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.292+0000 7fa29cf41700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa298102ea0 0x7fa29819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:54.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.292+0000 7fa29cf41700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2981085f0 0x7fa2981989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:54.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.292+0000 7fa29cf41700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa298112790 con 0x7fa298102ea0 2024-06-10T01:13:54.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.292+0000 7fa29cf41700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa298112610 con 0x7fa2981085f0 2024-06-10T01:13:54.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.292+0000 7fa29cf41700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa298112910 con 0x7fa2981024e0 2024-06-10T01:13:54.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.292+0000 7fa29659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2981024e0 0x7fa29819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:54.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.292+0000 7fa29659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2981024e0 0x7fa29819e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43802/0 (socket says 172.21.15.7:43802) 2024-06-10T01:13:54.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.292+0000 7fa29659c700 1 -- 172.21.15.7:0/3486407174 learned_addr learned my addr 172.21.15.7:0/3486407174 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:54.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.292+0000 7fa295d9b700 1 --2- 172.21.15.7:0/3486407174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa298102ea0 0x7fa29819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:54.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.292+0000 7fa296d9d700 1 --2- 172.21.15.7:0/3486407174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2981085f0 0x7fa2981989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:54.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.293+0000 7fa29659c700 1 -- 172.21.15.7:0/3486407174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2981085f0 msgr2=0x7fa2981989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:54.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.293+0000 7fa29659c700 1 --2- 172.21.15.7:0/3486407174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2981085f0 0x7fa2981989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.293+0000 7fa29659c700 1 -- 172.21.15.7:0/3486407174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa298102ea0 msgr2=0x7fa29819ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:54.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.293+0000 7fa29659c700 1 --2- 172.21.15.7:0/3486407174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa298102ea0 0x7fa29819ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.293+0000 7fa29659c700 1 -- 172.21.15.7:0/3486407174 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa298199110 con 0x7fa2981024e0 2024-06-10T01:13:54.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.293+0000 7fa29659c700 1 --2- 172.21.15.7:0/3486407174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2981024e0 0x7fa29819e7a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa28800e9a0 tx=0x7fa28800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:54.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.293+0000 7fa2877fe700 1 -- 172.21.15.7:0/3486407174 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28800cbd0 con 0x7fa2981024e0 2024-06-10T01:13:54.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.293+0000 7fa2877fe700 1 -- 172.21.15.7:0/3486407174 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa28800cd70 con 0x7fa2981024e0 2024-06-10T01:13:54.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.293+0000 7fa29cf41700 1 -- 172.21.15.7:0/3486407174 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa298199400 con 0x7fa2981024e0 2024-06-10T01:13:54.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.293+0000 7fa2877fe700 1 -- 172.21.15.7:0/3486407174 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa288018750 con 0x7fa2981024e0 2024-06-10T01:13:54.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.293+0000 7fa29cf41700 1 -- 172.21.15.7:0/3486407174 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2981a5670 con 0x7fa2981024e0 2024-06-10T01:13:54.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.296+0000 7fa2877fe700 1 -- 172.21.15.7:0/3486407174 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa288010430 con 0x7fa2981024e0 2024-06-10T01:13:54.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.296+0000 7fa2877fe700 1 --2- 172.21.15.7:0/3486407174 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa2800a2c20 0x7fa2800a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:54.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.296+0000 7fa295d9b700 1 --2- 172.21.15.7:0/3486407174 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa2800a2c20 0x7fa2800a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:54.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.296+0000 7fa2877fe700 1 -- 172.21.15.7:0/3486407174 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa2880cf6b0 con 0x7fa2981024e0 2024-06-10T01:13:54.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.297+0000 7fa295d9b700 1 --2- 172.21.15.7:0/3486407174 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa2800a2c20 0x7fa2800a50e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa298079b60 tx=0x7fa28c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:54.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.297+0000 7fa2a77c4b80 1 Processor -- start 2024-06-10T01:13:54.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.297+0000 7fa2a77c4b80 1 -- start start 2024-06-10T01:13:54.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.298+0000 7fa2a77c4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9bee14fe0 0x55d9bee153c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:54.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.298+0000 7fa2a77c4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9bee15900 0x55d9bee0f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:54.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.298+0000 7fa2a77c4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9bee0f820 0x55d9bee0fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:54.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.298+0000 7fa2a77c4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d9bed01fe0 con 0x55d9bee15900 2024-06-10T01:13:54.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.298+0000 7fa2a77c4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d9bed608d0 con 0x55d9bee0f820 2024-06-10T01:13:54.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.298+0000 7fa2a77c4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d9becc84e0 con 0x55d9bee14fe0 2024-06-10T01:13:54.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.298+0000 7fa29659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9bee14fe0 0x55d9bee153c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:54.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.298+0000 7fa295d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9bee15900 0x55d9bee0f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:54.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.298+0000 7fa29659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9bee14fe0 0x55d9bee153c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43810/0 (socket says 172.21.15.7:43810) 2024-06-10T01:13:54.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.298+0000 7fa29659c700 1 -- 172.21.15.7:0/3379900008 learned_addr learned my addr 172.21.15.7:0/3379900008 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:54.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.298+0000 7fa296d9d700 1 --2- 172.21.15.7:0/3379900008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9bee0f820 0x55d9bee0fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:54.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.298+0000 7fa29659c700 1 -- 172.21.15.7:0/3379900008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9bee0f820 msgr2=0x55d9bee0fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:54.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.298+0000 7fa29659c700 1 --2- 172.21.15.7:0/3379900008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9bee0f820 0x55d9bee0fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.298+0000 7fa29659c700 1 -- 172.21.15.7:0/3379900008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9bee15900 msgr2=0x55d9bee0f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:54.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.298+0000 7fa29659c700 1 --2- 172.21.15.7:0/3379900008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9bee15900 0x55d9bee0f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.298+0000 7fa29659c700 1 -- 172.21.15.7:0/3379900008 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9becaf950 con 0x55d9bee14fe0 2024-06-10T01:13:54.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.298+0000 7fa29659c700 1 --2- 172.21.15.7:0/3379900008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9bee14fe0 0x55d9bee153c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa288097ff0 tx=0x7fa28809c290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27859e74ccb61703 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:54.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.299+0000 7fa284ff9700 1 -- 172.21.15.7:0/3379900008 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28809ff40 con 0x55d9bee14fe0 2024-06-10T01:13:54.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.299+0000 7fa284ff9700 1 -- 172.21.15.7:0/3379900008 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2880a4800 con 0x55d9bee14fe0 2024-06-10T01:13:54.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.299+0000 7fa284ff9700 1 -- 172.21.15.7:0/3379900008 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2880a4370 con 0x55d9bee14fe0 2024-06-10T01:13:54.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.299+0000 7fa2a77c4b80 1 -- 172.21.15.7:0/3379900008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9bee14fe0 msgr2=0x55d9bee153c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:54.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.299+0000 7fa2a77c4b80 1 --2- 172.21.15.7:0/3379900008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9bee14fe0 0x55d9bee153c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa288097ff0 tx=0x7fa28809c290 comp rx=0 tx=0).stop 2024-06-10T01:13:54.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.299+0000 7fa2a77c4b80 1 -- 172.21.15.7:0/3379900008 shutdown_connections 2024-06-10T01:13:54.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.299+0000 7fa2a77c4b80 1 --2- 172.21.15.7:0/3379900008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9bee0f820 0x55d9bee0fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.299+0000 7fa2a77c4b80 1 --2- 172.21.15.7:0/3379900008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9bee14fe0 0x55d9bee153c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.299+0000 7fa2a77c4b80 1 --2- 172.21.15.7:0/3379900008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9bee15900 0x55d9bee0f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.299+0000 7fa2a77c4b80 1 -- 172.21.15.7:0/3379900008 >> 172.21.15.7:0/3379900008 conn(0x55d9bed3c100 msgr2=0x55d9bee0a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:54.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.300+0000 7fa2a77c4b80 1 -- 172.21.15.7:0/3379900008 shutdown_connections 2024-06-10T01:13:54.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.300+0000 7fa2a77c4b80 1 -- 172.21.15.7:0/3379900008 wait complete. 2024-06-10T01:13:54.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.300+0000 7fa2a77c4b80 1 -- start start 2024-06-10T01:13:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.300+0000 7fa2a77c4b80 1 Processor -- start 2024-06-10T01:13:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.300+0000 7fa2a77c4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9bee0f820 0x55d9bee1b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.300+0000 7fa2a77c4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9bee14fe0 0x55d9bee189a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.300+0000 7fa2a77c4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9bee15900 0x55d9bee18ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.300+0000 7fa2a77c4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d9bed01040 con 0x55d9bee15900 2024-06-10T01:13:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.300+0000 7fa2a77c4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d9bed5c890 con 0x55d9bee14fe0 2024-06-10T01:13:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.300+0000 7fa2a77c4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d9bee1c2e0 con 0x55d9bee0f820 2024-06-10T01:13:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.300+0000 7fa2a77c4b80 10 client.0 _ll_register_callbacks cb 0x55d9becf6630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:13:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.301+0000 7fa296d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9bee15900 0x55d9bee18ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.301+0000 7fa29659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9bee0f820 0x55d9bee1b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.301+0000 7fa29659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9bee0f820 0x55d9bee1b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43826/0 (socket says 172.21.15.7:43826) 2024-06-10T01:13:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.301+0000 7fa296d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9bee15900 0x55d9bee18ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54910/0 (socket says 172.21.15.7:54910) 2024-06-10T01:13:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.301+0000 7fa29659c700 1 -- 172.21.15.7:0/424676810 learned_addr learned my addr 172.21.15.7:0/424676810 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.301+0000 7fa295d9b700 1 --2- 172.21.15.7:0/424676810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9bee14fe0 0x55d9bee189a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.301+0000 7fa29659c700 1 -- 172.21.15.7:0/424676810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9bee14fe0 msgr2=0x55d9bee189a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.301+0000 7fa29659c700 1 --2- 172.21.15.7:0/424676810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9bee14fe0 0x55d9bee189a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.301+0000 7fa29659c700 1 -- 172.21.15.7:0/424676810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9bee15900 msgr2=0x55d9bee18ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.301+0000 7fa29659c700 1 --2- 172.21.15.7:0/424676810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9bee15900 0x55d9bee18ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.301+0000 7fa29659c700 1 -- 172.21.15.7:0/424676810 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9bed267d0 con 0x55d9bee0f820 2024-06-10T01:13:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.301+0000 7fa29659c700 1 --2- 172.21.15.7:0/424676810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9bee0f820 0x55d9bee1b6a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa28809bb20 tx=0x7fa28800cf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c1558abca8d6d3e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.301+0000 7fa27dffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:13:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.301+0000 7fa27dffb700 1 -- 172.21.15.7:0/424676810 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa288099e90 con 0x55d9bee0f820 2024-06-10T01:13:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.301+0000 7fa27dffb700 1 -- 172.21.15.7:0/424676810 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa2880109a0 con 0x55d9bee0f820 2024-06-10T01:13:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.301+0000 7fa2a77c4b80 1 -- 172.21.15.7:0/424676810 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2880a4800 con 0x55d9bee0f820 2024-06-10T01:13:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.301+0000 7fa27dffb700 1 -- 172.21.15.7:0/424676810 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa28809afe0 con 0x55d9bee0f820 2024-06-10T01:13:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.302+0000 7fa2a77c4b80 10 client.26872 fetch_fsmap learned FSMap version 35 2024-06-10T01:13:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.302+0000 7fa2a77c4b80 1 -- 172.21.15.7:0/424676810 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d9bee26a60 con 0x55d9bee0f820 2024-06-10T01:13:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.302+0000 7fa27dffb700 1 -- 172.21.15.7:0/424676810 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2880a4800 con 0x55d9bee0f820 2024-06-10T01:13:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.302+0000 7fa27dffb700 1 -- 172.21.15.7:0/424676810 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7fa2880ce040 con 0x55d9bee0f820 2024-06-10T01:13:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.302+0000 7fa2a77c4b80 10 client.26872 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:13:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.302+0000 7fa2a77c4b80 10 client.26872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:13:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.302+0000 7fa2a77c4b80 20 client.26872 populate_metadata read hostname 'smithi007' 2024-06-10T01:13:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.302+0000 7fa2a77c4b80 1 --2- 172.21.15.7:0/424676810 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d9bee28d60 0x55d9bee2b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.302+0000 7fa2a77c4b80 4 client.26872 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:13:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.302+0000 7fa2a77c4b80 1 -- 172.21.15.7:0/424676810 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9bed6ddd0 con 0x55d9bee28d60 2024-06-10T01:13:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.302+0000 7fa295d9b700 1 --2- 172.21.15.7:0/424676810 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d9bee28d60 0x55d9bee2b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:54.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.303+0000 7fa27dffb700 10 client.26872 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:54.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.303+0000 7fa295d9b700 1 --2- 172.21.15.7:0/424676810 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d9bee28d60 0x55d9bee2b150 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21fd3516655c4085 server_cookie=661c44fbfb7a95d4 in_seq=0 out_seq=0 2024-06-10T01:13:54.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.304+0000 7fa27dffb700 1 -- 172.21.15.7:0/424676810 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d9bed6ddd0 con 0x55d9bee28d60 2024-06-10T01:13:54.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.304+0000 7fa27dffb700 10 client.26872 handle_command_reply: tid=0 2024-06-10T01:13:54.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.304+0000 7fa2a77c4b80 1 client.26872 shutdown 2024-06-10T01:13:54.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.304+0000 7fa2a77c4b80 1 -- 172.21.15.7:0/424676810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9bee0f820 msgr2=0x55d9bee1b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:54.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.304+0000 7fa2a77c4b80 1 --2- 172.21.15.7:0/424676810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9bee0f820 0x55d9bee1b6a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa28809bb20 tx=0x7fa28800cf70 comp rx=0 tx=0).stop 2024-06-10T01:13:54.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.304+0000 7fa2a77c4b80 1 -- 172.21.15.7:0/424676810 shutdown_connections 2024-06-10T01:13:54.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.304+0000 7fa2a77c4b80 1 --2- 172.21.15.7:0/424676810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9bee15900 0x55d9bee18ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.304+0000 7fa2a77c4b80 1 --2- 172.21.15.7:0/424676810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9bee0f820 0x55d9bee1b6a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.304+0000 7fa2a77c4b80 1 --2- 172.21.15.7:0/424676810 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d9bee28d60 0x55d9bee2b150 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.304+0000 7fa2a77c4b80 1 --2- 172.21.15.7:0/424676810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9bee14fe0 0x55d9bee189a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.304+0000 7fa27dffb700 0 client.26872 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:54.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.304+0000 7fa2a77c4b80 1 -- 172.21.15.7:0/424676810 >> 172.21.15.7:0/424676810 conn(0x55d9bed3c100 msgr2=0x55d9bee09780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:54.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.304+0000 7fa2a77c4b80 1 -- 172.21.15.7:0/424676810 shutdown_connections 2024-06-10T01:13:54.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.304+0000 7fa2a77c4b80 1 -- 172.21.15.7:0/424676810 wait complete. 2024-06-10T01:13:54.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.304+0000 7fa2a77c4b80 20 client.26872 trim_cache size 0 max 16384 2024-06-10T01:13:54.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.327+0000 7fa2a77c4b80 1 Processor -- start 2024-06-10T01:13:54.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.327+0000 7fa2a77c4b80 1 -- start start 2024-06-10T01:13:54.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.327+0000 7fa2a77c4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9bee0f820 0x55d9bee0f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:54.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.327+0000 7fa2a77c4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9bee14fe0 0x55d9bee179f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:54.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.327+0000 7fa2a77c4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9bee15900 0x55d9bee09f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:54.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.327+0000 7fa2a77c4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa2981aea30 con 0x55d9bee0f820 2024-06-10T01:13:54.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.327+0000 7fa2a77c4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa2981ac260 con 0x55d9bee15900 2024-06-10T01:13:54.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.327+0000 7fa2a77c4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2981aa0d0 con 0x55d9bee14fe0 2024-06-10T01:13:54.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.327+0000 7fa29659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9bee0f820 0x55d9bee0f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:54.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.327+0000 7fa29659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9bee0f820 0x55d9bee0f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54924/0 (socket says 172.21.15.7:54924) 2024-06-10T01:13:54.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.327+0000 7fa29659c700 1 -- 172.21.15.7:0/2872470414 learned_addr learned my addr 172.21.15.7:0/2872470414 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:54.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.327+0000 7fa295d9b700 1 --2- 172.21.15.7:0/2872470414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9bee14fe0 0x55d9bee179f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:54.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.328+0000 7fa296d9d700 1 --2- 172.21.15.7:0/2872470414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9bee15900 0x55d9bee09f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:54.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.328+0000 7fa29659c700 1 -- 172.21.15.7:0/2872470414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9bee14fe0 msgr2=0x55d9bee179f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:54.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.328+0000 7fa29659c700 1 --2- 172.21.15.7:0/2872470414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9bee14fe0 0x55d9bee179f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.328+0000 7fa29659c700 1 -- 172.21.15.7:0/2872470414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9bee15900 msgr2=0x55d9bee09f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:54.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.328+0000 7fa29659c700 1 --2- 172.21.15.7:0/2872470414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9bee15900 0x55d9bee09f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.328+0000 7fa29659c700 1 -- 172.21.15.7:0/2872470414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9bee14ae0 con 0x55d9bee0f820 2024-06-10T01:13:54.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.328+0000 7fa29659c700 1 --2- 172.21.15.7:0/2872470414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9bee0f820 0x55d9bee0f2e0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fa288098560 tx=0x7fa288010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37431cbb5143c2d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:54.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.328+0000 7fa27ffff700 1 -- 172.21.15.7:0/2872470414 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa288099c30 con 0x55d9bee0f820 2024-06-10T01:13:54.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.328+0000 7fa27ffff700 1 -- 172.21.15.7:0/2872470414 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa288063030 con 0x55d9bee0f820 2024-06-10T01:13:54.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.328+0000 7fa27ffff700 1 -- 172.21.15.7:0/2872470414 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2880978e0 con 0x55d9bee0f820 2024-06-10T01:13:54.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.329+0000 7fa2a77c4b80 1 -- 172.21.15.7:0/2872470414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9bee0f820 msgr2=0x55d9bee0f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:54.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.329+0000 7fa2a77c4b80 1 --2- 172.21.15.7:0/2872470414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9bee0f820 0x55d9bee0f2e0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fa288098560 tx=0x7fa288010ca0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.329+0000 7fa2a77c4b80 1 -- 172.21.15.7:0/2872470414 shutdown_connections 2024-06-10T01:13:54.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.329+0000 7fa2a77c4b80 1 --2- 172.21.15.7:0/2872470414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9bee15900 0x55d9bee09f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.329+0000 7fa2a77c4b80 1 --2- 172.21.15.7:0/2872470414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9bee0f820 0x55d9bee0f2e0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.329+0000 7fa2a77c4b80 1 --2- 172.21.15.7:0/2872470414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9bee14fe0 0x55d9bee179f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.329+0000 7fa2a77c4b80 1 -- 172.21.15.7:0/2872470414 >> 172.21.15.7:0/2872470414 conn(0x55d9bed3c100 msgr2=0x55d9bee1aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:54.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.329+0000 7fa2a77c4b80 1 -- 172.21.15.7:0/2872470414 shutdown_connections 2024-06-10T01:13:54.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.329+0000 7fa2a77c4b80 1 -- 172.21.15.7:0/2872470414 wait complete. 2024-06-10T01:13:54.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.329+0000 7fa2a77c4b80 1 -- start start 2024-06-10T01:13:54.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.329+0000 7fa2a77c4b80 1 Processor -- start 2024-06-10T01:13:54.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.330+0000 7fa2a77c4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9bee0f820 0x55d9bee2c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:54.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.330+0000 7fa2a77c4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9bee14fe0 0x55d9bee2c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:54.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.330+0000 7fa2a77c4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9bee15900 0x55d9bee2cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:54.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.330+0000 7fa2a77c4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa2981aa720 con 0x55d9bee15900 2024-06-10T01:13:54.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.330+0000 7fa2a77c4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa2981abb90 con 0x55d9bee14fe0 2024-06-10T01:13:54.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.330+0000 7fa2a77c4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2981aadf0 con 0x55d9bee0f820 2024-06-10T01:13:54.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.330+0000 7fa2a77c4b80 10 client.0 _ll_register_callbacks cb 0x55d9becf6630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:13:54.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.330+0000 7fa295d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9bee14fe0 0x55d9bee2c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:54.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.330+0000 7fa29659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9bee0f820 0x55d9bee2c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:54.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.330+0000 7fa296d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9bee15900 0x55d9bee2cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:54.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.330+0000 7fa295d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9bee14fe0 0x55d9bee2c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:42302/0 (socket says 172.21.15.7:42302) 2024-06-10T01:13:54.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.330+0000 7fa29659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9bee0f820 0x55d9bee2c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43854/0 (socket says 172.21.15.7:43854) 2024-06-10T01:13:54.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.330+0000 7fa295d9b700 1 -- 172.21.15.7:0/713159253 learned_addr learned my addr 172.21.15.7:0/713159253 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:54.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.330+0000 7fa295d9b700 1 -- 172.21.15.7:0/713159253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9bee0f820 msgr2=0x55d9bee2c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:54.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.330+0000 7fa295d9b700 1 --2- 172.21.15.7:0/713159253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9bee0f820 0x55d9bee2c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.330+0000 7fa295d9b700 1 -- 172.21.15.7:0/713159253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9bee15900 msgr2=0x55d9bee2cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:54.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.330+0000 7fa295d9b700 1 --2- 172.21.15.7:0/713159253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9bee15900 0x55d9bee2cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.330+0000 7fa295d9b700 1 -- 172.21.15.7:0/713159253 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9bee92900 con 0x55d9bee14fe0 2024-06-10T01:13:54.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.330+0000 7fa29659c700 1 --2- 172.21.15.7:0/713159253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9bee0f820 0x55d9bee2c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:54.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.330+0000 7fa295d9b700 1 --2- 172.21.15.7:0/713159253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9bee14fe0 0x55d9bee2c710 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa28c0094f0 tx=0x7fa28c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7efd0e9233d4dfc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:54.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.330+0000 7fa27dffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:13:54.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.331+0000 7fa27dffb700 1 -- 172.21.15.7:0/713159253 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c038060 con 0x55d9bee14fe0 2024-06-10T01:13:54.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.331+0000 7fa2a77c4b80 1 -- 172.21.15.7:0/713159253 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa288063030 con 0x55d9bee14fe0 2024-06-10T01:13:54.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.331+0000 7fa27dffb700 1 -- 172.21.15.7:0/713159253 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa28c01dcd0 con 0x55d9bee14fe0 2024-06-10T01:13:54.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.331+0000 7fa27dffb700 1 -- 172.21.15.7:0/713159253 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa28c03f050 con 0x55d9bee14fe0 2024-06-10T01:13:54.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.331+0000 7fa2a77c4b80 10 client.26858 fetch_fsmap learned FSMap version 35 2024-06-10T01:13:54.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.331+0000 7fa2a77c4b80 1 -- 172.21.15.7:0/713159253 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d9bee93c80 con 0x55d9bee14fe0 2024-06-10T01:13:54.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.331+0000 7fa27dffb700 1 -- 172.21.15.7:0/713159253 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa288063030 con 0x55d9bee14fe0 2024-06-10T01:13:54.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.331+0000 7fa27dffb700 1 -- 172.21.15.7:0/713159253 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7fa28c033800 con 0x55d9bee14fe0 2024-06-10T01:13:54.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.331+0000 7fa2a77c4b80 10 client.26858 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:13:54.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.331+0000 7fa2a77c4b80 10 client.26858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:13:54.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.331+0000 7fa2a77c4b80 20 client.26858 populate_metadata read hostname 'smithi007' 2024-06-10T01:13:54.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.331+0000 7fa2a77c4b80 1 --2- 172.21.15.7:0/713159253 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d9bee94370 0x55d9bee96790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:54.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.331+0000 7fa2a77c4b80 4 client.26858 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:13:54.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.331+0000 7fa2a77c4b80 1 -- 172.21.15.7:0/713159253 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d9bed68350 con 0x55d9bee94370 2024-06-10T01:13:54.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.332+0000 7fa29659c700 1 --2- 172.21.15.7:0/713159253 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d9bee94370 0x55d9bee96790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:54.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.332+0000 7fa29659c700 1 --2- 172.21.15.7:0/713159253 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d9bee94370 0x55d9bee96790 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65d8c8d25f6f336a server_cookie=db66b3c0f9af2a54 in_seq=0 out_seq=0 2024-06-10T01:13:54.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.332+0000 7fa27dffb700 10 client.26858 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:54.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.333+0000 7fa27dffb700 1 -- 172.21.15.7:0/713159253 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d9bed68350 con 0x55d9bee94370 2024-06-10T01:13:54.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.333+0000 7fa27dffb700 10 client.26858 handle_command_reply: tid=0 2024-06-10T01:13:54.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.333+0000 7fa2a77c4b80 1 client.26858 shutdown 2024-06-10T01:13:54.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.333+0000 7fa2a77c4b80 1 -- 172.21.15.7:0/713159253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9bee14fe0 msgr2=0x55d9bee2c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:54.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.333+0000 7fa2a77c4b80 1 --2- 172.21.15.7:0/713159253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9bee14fe0 0x55d9bee2c710 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa28c0094f0 tx=0x7fa28c0047c0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.333+0000 7fa2a77c4b80 1 -- 172.21.15.7:0/713159253 shutdown_connections 2024-06-10T01:13:54.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.333+0000 7fa2a77c4b80 1 --2- 172.21.15.7:0/713159253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9bee15900 0x55d9bee2cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.333+0000 7fa2a77c4b80 1 --2- 172.21.15.7:0/713159253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9bee0f820 0x55d9bee2c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.333+0000 7fa2a77c4b80 1 --2- 172.21.15.7:0/713159253 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d9bee94370 0x55d9bee96790 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.333+0000 7fa2a77c4b80 1 --2- 172.21.15.7:0/713159253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9bee14fe0 0x55d9bee2c710 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.333+0000 7fa27dffb700 0 client.26858 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:54.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.333+0000 7fa2a77c4b80 1 -- 172.21.15.7:0/713159253 >> 172.21.15.7:0/713159253 conn(0x55d9bed3c100 msgr2=0x55d9bee14360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:54.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.333+0000 7fa2a77c4b80 1 -- 172.21.15.7:0/713159253 shutdown_connections 2024-06-10T01:13:54.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.333+0000 7fa2a77c4b80 1 -- 172.21.15.7:0/713159253 wait complete. 2024-06-10T01:13:54.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.333+0000 7fa2a77c4b80 20 client.26858 trim_cache size 0 max 16384 2024-06-10T01:13:54.332 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:13:54.333 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-10T01:13:54.333 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "4da96152-8bc2-4e17-ac3f-c2eeb2a3acdc", 2024-06-10T01:13:54.333 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-10T01:13:54.333 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:13:54.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.333+0000 7fa29cf41700 1 -- 172.21.15.7:0/3486407174 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa2800a2c20 msgr2=0x7fa2800a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:54.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.333+0000 7fa29cf41700 1 --2- 172.21.15.7:0/3486407174 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa2800a2c20 0x7fa2800a50e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa298079b60 tx=0x7fa28c009260 comp rx=0 tx=0).stop 2024-06-10T01:13:54.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.333+0000 7fa29cf41700 1 -- 172.21.15.7:0/3486407174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2981024e0 msgr2=0x7fa29819e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:54.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.333+0000 7fa29cf41700 1 --2- 172.21.15.7:0/3486407174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2981024e0 0x7fa29819e7a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa28800e9a0 tx=0x7fa28800ecb0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.334+0000 7fa29cf41700 1 -- 172.21.15.7:0/3486407174 shutdown_connections 2024-06-10T01:13:54.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.334+0000 7fa29cf41700 1 --2- 172.21.15.7:0/3486407174 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa2800a2c20 0x7fa2800a50e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.334+0000 7fa29cf41700 1 --2- 172.21.15.7:0/3486407174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2981085f0 0x7fa2981989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.334+0000 7fa29cf41700 1 --2- 172.21.15.7:0/3486407174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2981024e0 0x7fa29819e7a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.334+0000 7fa29cf41700 1 --2- 172.21.15.7:0/3486407174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa298102ea0 0x7fa29819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:54.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.334+0000 7fa29cf41700 1 -- 172.21.15.7:0/3486407174 >> 172.21.15.7:0/3486407174 conn(0x7fa298079640 msgr2=0x7fa298100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:54.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.334+0000 7fa29cf41700 1 -- 172.21.15.7:0/3486407174 shutdown_connections 2024-06-10T01:13:54.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:54.334+0000 7fa29cf41700 1 -- 172.21.15.7:0/3486407174 wait complete. 2024-06-10T01:13:54.739 INFO:tasks.workunit.client.0.smithi007.stdout: 1 576234 7.94 MB/sec execute 275 sec latency 60.252 ms 2024-06-10T01:13:55.047 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:13:55.325 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:13:55.667 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:55 smithi007 ceph-mon[32398]: pgmap v669: 129 pgs: 129 active+clean; 102 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 317 op/s 2024-06-10T01:13:55.667 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:55 smithi007 ceph-mon[32398]: scrub queued for path: / 2024-06-10T01:13:55.668 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:55 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:13:55.668 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:55 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:13:55.668 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:55 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:13:55.668 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:55 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:13:55.668 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:55 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:13:55.668 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:55 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:13:55.668 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:55 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:13:55.668 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:55 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:13:55.668 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:55 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:13:55.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:55 smithi175 ceph-mon[36636]: pgmap v669: 129 pgs: 129 active+clean; 102 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 317 op/s 2024-06-10T01:13:55.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:55 smithi175 ceph-mon[36636]: scrub queued for path: / 2024-06-10T01:13:55.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:55 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:13:55.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:55 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:13:55.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:55 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:13:55.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:55 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:13:55.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:55 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:13:55.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:55 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:13:55.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:55 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:13:55.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:55 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:13:55.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:55 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:13:55.739 INFO:tasks.workunit.client.0.smithi007.stdout: 1 578784 7.96 MB/sec execute 276 sec latency 36.358 ms 2024-06-10T01:13:55.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:55 smithi078 ceph-mon[36683]: pgmap v669: 129 pgs: 129 active+clean; 102 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 317 op/s 2024-06-10T01:13:55.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:55 smithi078 ceph-mon[36683]: scrub queued for path: / 2024-06-10T01:13:55.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:55 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:13:55.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:55 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:13:55.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:55 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:13:55.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:55 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:13:55.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:55 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:13:55.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:55 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:13:55.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:55 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:13:55.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:55 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:13:55.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:55 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:13:56.739 INFO:tasks.workunit.client.0.smithi007.stdout: 1 580235 7.95 MB/sec execute 277 sec latency 37.315 ms 2024-06-10T01:13:57.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.206+0000 7f906ab08700 1 -- 172.21.15.7:0/3805932505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90641073f0 msgr2=0x7f90641077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:57.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.206+0000 7f906ab08700 1 --2- 172.21.15.7:0/3805932505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90641073f0 0x7f90641077d0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f9058009990 tx=0x7f9058009ca0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.206+0000 7f906ab08700 1 -- 172.21.15.7:0/3805932505 shutdown_connections 2024-06-10T01:13:57.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.206+0000 7f906ab08700 1 --2- 172.21.15.7:0/3805932505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9064101d40 0x7f906410e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.206+0000 7f906ab08700 1 --2- 172.21.15.7:0/3805932505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90641073f0 0x7f90641077d0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.206+0000 7f906ab08700 1 --2- 172.21.15.7:0/3805932505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9064101380 0x7f9064101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.206+0000 7f906ab08700 1 -- 172.21.15.7:0/3805932505 >> 172.21.15.7:0/3805932505 conn(0x7f9064078300 msgr2=0x7f9064078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:57.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.206+0000 7f906ab08700 1 -- 172.21.15.7:0/3805932505 shutdown_connections 2024-06-10T01:13:57.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.206+0000 7f906ab08700 1 -- 172.21.15.7:0/3805932505 wait complete. 2024-06-10T01:13:57.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.207+0000 7f906ab08700 1 Processor -- start 2024-06-10T01:13:57.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.207+0000 7f906ab08700 1 -- start start 2024-06-10T01:13:57.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.207+0000 7f906ab08700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9064101380 0x7f9064199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:57.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.207+0000 7f906ab08700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9064101d40 0x7f906419a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:57.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.207+0000 7f906ab08700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90641073f0 0x7f90641a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:57.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.207+0000 7f906ab08700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9064111690 con 0x7f9064101d40 2024-06-10T01:13:57.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.207+0000 7f906ab08700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9064111510 con 0x7f9064101380 2024-06-10T01:13:57.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.207+0000 7f906ab08700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9064111810 con 0x7f90641073f0 2024-06-10T01:13:57.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.207+0000 7f9063fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9064101d40 0x7f906419a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:57.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.207+0000 7f9063fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9064101d40 0x7f906419a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54946/0 (socket says 172.21.15.7:54946) 2024-06-10T01:13:57.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.207+0000 7f9063fff700 1 -- 172.21.15.7:0/3539551981 learned_addr learned my addr 172.21.15.7:0/3539551981 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:57.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.207+0000 7f90690a5700 1 --2- 172.21.15.7:0/3539551981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90641073f0 0x7f90641a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:57.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.207+0000 7f90688a4700 1 --2- 172.21.15.7:0/3539551981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9064101380 0x7f9064199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:57.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.208+0000 7f9063fff700 1 -- 172.21.15.7:0/3539551981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90641073f0 msgr2=0x7f90641a4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:57.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.208+0000 7f9063fff700 1 --2- 172.21.15.7:0/3539551981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90641073f0 0x7f90641a4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.208+0000 7f9063fff700 1 -- 172.21.15.7:0/3539551981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9064101380 msgr2=0x7f9064199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:57.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.208+0000 7f9063fff700 1 --2- 172.21.15.7:0/3539551981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9064101380 0x7f9064199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.208+0000 7f9063fff700 1 -- 172.21.15.7:0/3539551981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90641a5090 con 0x7f9064101d40 2024-06-10T01:13:57.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.208+0000 7f90690a5700 1 --2- 172.21.15.7:0/3539551981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90641073f0 0x7f90641a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:13:57.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.208+0000 7f9063fff700 1 --2- 172.21.15.7:0/3539551981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9064101d40 0x7f906419a010 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f905000e9a0 tx=0x7f905000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:57.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.208+0000 7f9061ffb700 1 -- 172.21.15.7:0/3539551981 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905000cbd0 con 0x7f9064101d40 2024-06-10T01:13:57.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.208+0000 7f9061ffb700 1 -- 172.21.15.7:0/3539551981 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f905000cd70 con 0x7f9064101d40 2024-06-10T01:13:57.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.208+0000 7f906ab08700 1 -- 172.21.15.7:0/3539551981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90641a52c0 con 0x7f9064101d40 2024-06-10T01:13:57.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.208+0000 7f9061ffb700 1 -- 172.21.15.7:0/3539551981 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9050018750 con 0x7f9064101d40 2024-06-10T01:13:57.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.208+0000 7f906ab08700 1 -- 172.21.15.7:0/3539551981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90641a5800 con 0x7f9064101d40 2024-06-10T01:13:57.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.211+0000 7f9061ffb700 1 -- 172.21.15.7:0/3539551981 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9050010430 con 0x7f9064101d40 2024-06-10T01:13:57.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.212+0000 7f9061ffb700 1 --2- 172.21.15.7:0/3539551981 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f90540a2c20 0x7f90540a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:57.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.212+0000 7f90688a4700 1 --2- 172.21.15.7:0/3539551981 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f90540a2c20 0x7f90540a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:57.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.212+0000 7f9061ffb700 1 -- 172.21.15.7:0/3539551981 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f90500cf6e0 con 0x7f9064101d40 2024-06-10T01:13:57.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.212+0000 7f907538bb80 1 Processor -- start 2024-06-10T01:13:57.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.212+0000 7f907538bb80 1 -- start start 2024-06-10T01:13:57.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.212+0000 7f90688a4700 1 --2- 172.21.15.7:0/3539551981 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f90540a2c20 0x7f90540a50e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f905800b2e0 tx=0x7f9058005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:57.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.212+0000 7f907538bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebd8f15640 0x55ebd8f15a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:57.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.212+0000 7f907538bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebd8f15f60 0x55ebd8f0f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:57.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.212+0000 7f907538bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebd8f0ff20 0x55ebd8f10340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:57.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.212+0000 7f907538bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ebd8e61810 con 0x55ebd8f15640 2024-06-10T01:13:57.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.212+0000 7f907538bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ebd8e09720 con 0x55ebd8f15f60 2024-06-10T01:13:57.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.212+0000 7f907538bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ebd8e5d7d0 con 0x55ebd8f0ff20 2024-06-10T01:13:57.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.213+0000 7f90688a4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebd8f15640 0x55ebd8f15a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:57.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.213+0000 7f9063fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebd8f15f60 0x55ebd8f0f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:57.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.213+0000 7f90688a4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebd8f15640 0x55ebd8f15a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54956/0 (socket says 172.21.15.7:54956) 2024-06-10T01:13:57.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.213+0000 7f9063fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebd8f15f60 0x55ebd8f0f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:42324/0 (socket says 172.21.15.7:42324) 2024-06-10T01:13:57.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.213+0000 7f90688a4700 1 -- 172.21.15.7:0/3199779564 learned_addr learned my addr 172.21.15.7:0/3199779564 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:57.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.213+0000 7f90690a5700 1 --2- 172.21.15.7:0/3199779564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebd8f0ff20 0x55ebd8f10340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:57.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.213+0000 7f9063fff700 1 -- 172.21.15.7:0/3199779564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebd8f0ff20 msgr2=0x55ebd8f10340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:57.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.213+0000 7f9063fff700 1 --2- 172.21.15.7:0/3199779564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebd8f0ff20 0x55ebd8f10340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.213+0000 7f9063fff700 1 -- 172.21.15.7:0/3199779564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebd8f15640 msgr2=0x55ebd8f15a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:57.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.213+0000 7f9063fff700 1 --2- 172.21.15.7:0/3199779564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebd8f15640 0x55ebd8f15a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.213+0000 7f9063fff700 1 -- 172.21.15.7:0/3199779564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebd8dfff50 con 0x55ebd8f15f60 2024-06-10T01:13:57.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.213+0000 7f90690a5700 1 --2- 172.21.15.7:0/3199779564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebd8f0ff20 0x55ebd8f10340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:57.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.213+0000 7f9063fff700 1 --2- 172.21.15.7:0/3199779564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebd8f15f60 0x55ebd8f0f9e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f90500cce00 tx=0x7f9050097f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57736388d34b40e7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:57.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.213+0000 7f904effd700 1 -- 172.21.15.7:0/3199779564 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90500b3070 con 0x55ebd8f15f60 2024-06-10T01:13:57.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.213+0000 7f904effd700 1 -- 172.21.15.7:0/3199779564 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f905009bb60 con 0x55ebd8f15f60 2024-06-10T01:13:57.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.213+0000 7f904effd700 1 -- 172.21.15.7:0/3199779564 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9050010ac0 con 0x55ebd8f15f60 2024-06-10T01:13:57.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.214+0000 7f907538bb80 1 -- 172.21.15.7:0/3199779564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebd8f15f60 msgr2=0x55ebd8f0f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:57.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.214+0000 7f907538bb80 1 --2- 172.21.15.7:0/3199779564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebd8f15f60 0x55ebd8f0f9e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f90500cce00 tx=0x7f9050097f10 comp rx=0 tx=0).stop 2024-06-10T01:13:57.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.214+0000 7f907538bb80 1 -- 172.21.15.7:0/3199779564 shutdown_connections 2024-06-10T01:13:57.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.214+0000 7f907538bb80 1 --2- 172.21.15.7:0/3199779564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebd8f0ff20 0x55ebd8f10340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.214+0000 7f907538bb80 1 --2- 172.21.15.7:0/3199779564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebd8f15640 0x55ebd8f15a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.214+0000 7f907538bb80 1 --2- 172.21.15.7:0/3199779564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebd8f15f60 0x55ebd8f0f9e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.214+0000 7f907538bb80 1 -- 172.21.15.7:0/3199779564 >> 172.21.15.7:0/3199779564 conn(0x55ebd8e3acc0 msgr2=0x55ebd8f0a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:57.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.214+0000 7f907538bb80 1 -- 172.21.15.7:0/3199779564 shutdown_connections 2024-06-10T01:13:57.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.214+0000 7f907538bb80 1 -- 172.21.15.7:0/3199779564 wait complete. 2024-06-10T01:13:57.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.214+0000 7f907538bb80 1 -- start start 2024-06-10T01:13:57.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.215+0000 7f907538bb80 1 Processor -- start 2024-06-10T01:13:57.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.215+0000 7f907538bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebd8f0ff20 0x55ebd8f1bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:57.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.215+0000 7f907538bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebd8f15640 0x55ebd8f19050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:57.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.215+0000 7f907538bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebd8f15f60 0x55ebd8f19590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:57.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.215+0000 7f907538bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ebd8f1c9d0 con 0x55ebd8f15f60 2024-06-10T01:13:57.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.215+0000 7f907538bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ebd8e01bf0 con 0x55ebd8f0ff20 2024-06-10T01:13:57.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.215+0000 7f907538bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ebd8f1cb50 con 0x55ebd8f15640 2024-06-10T01:13:57.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.215+0000 7f90690a5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebd8f15f60 0x55ebd8f19590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:57.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.215+0000 7f90690a5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebd8f15f60 0x55ebd8f19590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54960/0 (socket says 172.21.15.7:54960) 2024-06-10T01:13:57.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.215+0000 7f90688a4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebd8f0ff20 0x55ebd8f1bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:57.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.215+0000 7f90690a5700 1 -- 172.21.15.7:0/2677887349 learned_addr learned my addr 172.21.15.7:0/2677887349 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:57.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.215+0000 7f9063fff700 1 --2- 172.21.15.7:0/2677887349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebd8f15640 0x55ebd8f19050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:57.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.215+0000 7f907538bb80 10 client.0 _ll_register_callbacks cb 0x55ebd8dfb920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:13:57.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.216+0000 7f90690a5700 1 -- 172.21.15.7:0/2677887349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebd8f15640 msgr2=0x55ebd8f19050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:57.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.216+0000 7f90690a5700 1 --2- 172.21.15.7:0/2677887349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebd8f15640 0x55ebd8f19050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.216+0000 7f90690a5700 1 -- 172.21.15.7:0/2677887349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebd8f0ff20 msgr2=0x55ebd8f1bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:57.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.216+0000 7f90690a5700 1 --2- 172.21.15.7:0/2677887349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebd8f0ff20 0x55ebd8f1bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.216+0000 7f90690a5700 1 -- 172.21.15.7:0/2677887349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebd8db08d0 con 0x55ebd8f15f60 2024-06-10T01:13:57.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.216+0000 7f90688a4700 1 --2- 172.21.15.7:0/2677887349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebd8f0ff20 0x55ebd8f1bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:57.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.216+0000 7f9063fff700 1 --2- 172.21.15.7:0/2677887349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebd8f15640 0x55ebd8f19050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:57.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.216+0000 7f90690a5700 1 --2- 172.21.15.7:0/2677887349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebd8f15f60 0x55ebd8f19590 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f905c007f60 tx=0x7f905c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55edb5907aa0cc02 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:57.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.216+0000 7f9043fff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:13:57.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.216+0000 7f9043fff700 1 -- 172.21.15.7:0/2677887349 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c01c070 con 0x55ebd8f15f60 2024-06-10T01:13:57.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.216+0000 7f9043fff700 1 -- 172.21.15.7:0/2677887349 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f905c013b10 con 0x55ebd8f15f60 2024-06-10T01:13:57.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.216+0000 7f907538bb80 1 -- 172.21.15.7:0/2677887349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f905009bb60 con 0x55ebd8f15f60 2024-06-10T01:13:57.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.216+0000 7f9043fff700 1 -- 172.21.15.7:0/2677887349 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f905c027050 con 0x55ebd8f15f60 2024-06-10T01:13:57.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.216+0000 7f907538bb80 10 client.17457 fetch_fsmap learned FSMap version 35 2024-06-10T01:13:57.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.216+0000 7f907538bb80 1 -- 172.21.15.7:0/2677887349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ebd8f27340 con 0x55ebd8f15f60 2024-06-10T01:13:57.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.216+0000 7f9043fff700 1 -- 172.21.15.7:0/2677887349 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f905009bb60 con 0x55ebd8f15f60 2024-06-10T01:13:57.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.217+0000 7f9043fff700 1 -- 172.21.15.7:0/2677887349 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f905c015390 con 0x55ebd8f15f60 2024-06-10T01:13:57.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.217+0000 7f907538bb80 10 client.17457 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:13:57.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.217+0000 7f907538bb80 10 client.17457 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:13:57.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.217+0000 7f907538bb80 20 client.17457 populate_metadata read hostname 'smithi007' 2024-06-10T01:13:57.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.217+0000 7f907538bb80 1 --2- 172.21.15.7:0/2677887349 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ebd8f299d0 0x55ebd8f2bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:57.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.217+0000 7f907538bb80 4 client.17457 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:13:57.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.217+0000 7f907538bb80 1 -- 172.21.15.7:0/2677887349 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebd8e6ed10 con 0x55ebd8f299d0 2024-06-10T01:13:57.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.217+0000 7f90688a4700 1 --2- 172.21.15.7:0/2677887349 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ebd8f299d0 0x55ebd8f2bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:57.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.217+0000 7f9043fff700 10 client.17457 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:57.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.217+0000 7f90688a4700 1 --2- 172.21.15.7:0/2677887349 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ebd8f299d0 0x55ebd8f2bdc0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8678ce6ab08e7fc3 server_cookie=632946c3c9262cf8 in_seq=0 out_seq=0 2024-06-10T01:13:57.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.218+0000 7f9043fff700 1 -- 172.21.15.7:0/2677887349 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ebd8e6ed10 con 0x55ebd8f299d0 2024-06-10T01:13:57.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.218+0000 7f9043fff700 10 client.17457 handle_command_reply: tid=0 2024-06-10T01:13:57.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.218+0000 7f907538bb80 1 client.17457 shutdown 2024-06-10T01:13:57.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.218+0000 7f907538bb80 1 -- 172.21.15.7:0/2677887349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebd8f15f60 msgr2=0x55ebd8f19590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:57.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.218+0000 7f907538bb80 1 --2- 172.21.15.7:0/2677887349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebd8f15f60 0x55ebd8f19590 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f905c007f60 tx=0x7f905c007f90 comp rx=0 tx=0).stop 2024-06-10T01:13:57.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.218+0000 7f907538bb80 1 -- 172.21.15.7:0/2677887349 shutdown_connections 2024-06-10T01:13:57.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.218+0000 7f907538bb80 1 --2- 172.21.15.7:0/2677887349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebd8f15f60 0x55ebd8f19590 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.218+0000 7f907538bb80 1 --2- 172.21.15.7:0/2677887349 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ebd8f299d0 0x55ebd8f2bdc0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.218+0000 7f907538bb80 1 --2- 172.21.15.7:0/2677887349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebd8f0ff20 0x55ebd8f1bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.218+0000 7f907538bb80 1 --2- 172.21.15.7:0/2677887349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebd8f15640 0x55ebd8f19050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.218+0000 7f9043fff700 0 client.17457 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:57.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.218+0000 7f907538bb80 1 -- 172.21.15.7:0/2677887349 >> 172.21.15.7:0/2677887349 conn(0x55ebd8e3acc0 msgr2=0x55ebd8f0e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:57.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.218+0000 7f907538bb80 1 -- 172.21.15.7:0/2677887349 shutdown_connections 2024-06-10T01:13:57.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.218+0000 7f907538bb80 1 -- 172.21.15.7:0/2677887349 wait complete. 2024-06-10T01:13:57.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.218+0000 7f907538bb80 20 client.17457 trim_cache size 0 max 16384 2024-06-10T01:13:57.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.241+0000 7f907538bb80 1 Processor -- start 2024-06-10T01:13:57.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.241+0000 7f907538bb80 1 -- start start 2024-06-10T01:13:57.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.242+0000 7f907538bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebd8f0ff20 0x55ebd8f0f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:57.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.242+0000 7f907538bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebd8f15640 0x55ebd8f16b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:57.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.242+0000 7f907538bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebd8f15f60 0x55ebd8f31670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:57.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.242+0000 7f907538bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f90641aea80 con 0x55ebd8f15f60 2024-06-10T01:13:57.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.242+0000 7f907538bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f90641ac2b0 con 0x55ebd8f15640 2024-06-10T01:13:57.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.242+0000 7f907538bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f90641aa100 con 0x55ebd8f0ff20 2024-06-10T01:13:57.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.242+0000 7f90688a4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebd8f0ff20 0x55ebd8f0f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:57.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.242+0000 7f9063fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebd8f15640 0x55ebd8f16b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:57.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.242+0000 7f90688a4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebd8f0ff20 0x55ebd8f0f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43884/0 (socket says 172.21.15.7:43884) 2024-06-10T01:13:57.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.242+0000 7f90688a4700 1 -- 172.21.15.7:0/1308926637 learned_addr learned my addr 172.21.15.7:0/1308926637 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:57.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.242+0000 7f90690a5700 1 --2- 172.21.15.7:0/1308926637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebd8f15f60 0x55ebd8f31670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:57.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.242+0000 7f9063fff700 1 -- 172.21.15.7:0/1308926637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebd8f0ff20 msgr2=0x55ebd8f0f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:57.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.242+0000 7f9063fff700 1 --2- 172.21.15.7:0/1308926637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebd8f0ff20 0x55ebd8f0f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.242+0000 7f9063fff700 1 -- 172.21.15.7:0/1308926637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebd8f15f60 msgr2=0x55ebd8f31670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:57.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.242+0000 7f9063fff700 1 --2- 172.21.15.7:0/1308926637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebd8f15f60 0x55ebd8f31670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.242+0000 7f9063fff700 1 -- 172.21.15.7:0/1308926637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebd8f175c0 con 0x55ebd8f15640 2024-06-10T01:13:57.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.242+0000 7f9063fff700 1 --2- 172.21.15.7:0/1308926637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebd8f15640 0x55ebd8f16b10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f905000e970 tx=0x7f905009b5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cd23e9a2acfe9d9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:57.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.243+0000 7f904e7fc700 1 -- 172.21.15.7:0/1308926637 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90500b3070 con 0x55ebd8f15640 2024-06-10T01:13:57.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.243+0000 7f904e7fc700 1 -- 172.21.15.7:0/1308926637 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90500a47d0 con 0x55ebd8f15640 2024-06-10T01:13:57.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.243+0000 7f904e7fc700 1 -- 172.21.15.7:0/1308926637 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90500cf3a0 con 0x55ebd8f15640 2024-06-10T01:13:57.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.243+0000 7f907538bb80 1 -- 172.21.15.7:0/1308926637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebd8f15640 msgr2=0x55ebd8f16b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:57.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.243+0000 7f907538bb80 1 --2- 172.21.15.7:0/1308926637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebd8f15640 0x55ebd8f16b10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f905000e970 tx=0x7f905009b5c0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.243+0000 7f907538bb80 1 -- 172.21.15.7:0/1308926637 shutdown_connections 2024-06-10T01:13:57.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.243+0000 7f907538bb80 1 --2- 172.21.15.7:0/1308926637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebd8f15f60 0x55ebd8f31670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.243+0000 7f907538bb80 1 --2- 172.21.15.7:0/1308926637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebd8f0ff20 0x55ebd8f0f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.243+0000 7f907538bb80 1 --2- 172.21.15.7:0/1308926637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebd8f15640 0x55ebd8f16b10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.243+0000 7f907538bb80 1 -- 172.21.15.7:0/1308926637 >> 172.21.15.7:0/1308926637 conn(0x55ebd8e3acc0 msgr2=0x55ebd8f16400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:57.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.244+0000 7f907538bb80 1 -- 172.21.15.7:0/1308926637 shutdown_connections 2024-06-10T01:13:57.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.244+0000 7f907538bb80 1 -- 172.21.15.7:0/1308926637 wait complete. 2024-06-10T01:13:57.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.244+0000 7f907538bb80 1 -- start start 2024-06-10T01:13:57.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.244+0000 7f907538bb80 1 Processor -- start 2024-06-10T01:13:57.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.244+0000 7f907538bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebd8f0ff20 0x55ebd8f225e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:57.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.244+0000 7f907538bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebd8f15640 0x55ebd8f31040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:57.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.244+0000 7f907538bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebd8f15f60 0x55ebd8f315c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:57.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.244+0000 7f907538bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f90641aa770 con 0x55ebd8f0ff20 2024-06-10T01:13:57.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.244+0000 7f907538bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f90641abbe0 con 0x55ebd8f15640 2024-06-10T01:13:57.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.244+0000 7f907538bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f90641aae40 con 0x55ebd8f15f60 2024-06-10T01:13:57.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.244+0000 7f907538bb80 10 client.0 _ll_register_callbacks cb 0x55ebd8dfb920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:13:57.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.245+0000 7f90688a4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebd8f0ff20 0x55ebd8f225e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:57.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.245+0000 7f90688a4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebd8f0ff20 0x55ebd8f225e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54966/0 (socket says 172.21.15.7:54966) 2024-06-10T01:13:57.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.245+0000 7f90688a4700 1 -- 172.21.15.7:0/3654097607 learned_addr learned my addr 172.21.15.7:0/3654097607 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:57.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.245+0000 7f9063fff700 1 --2- 172.21.15.7:0/3654097607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebd8f15640 0x55ebd8f31040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:57.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.245+0000 7f90690a5700 1 --2- 172.21.15.7:0/3654097607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebd8f15f60 0x55ebd8f315c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:57.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.245+0000 7f90688a4700 1 -- 172.21.15.7:0/3654097607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebd8f15f60 msgr2=0x55ebd8f315c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:57.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.245+0000 7f90688a4700 1 --2- 172.21.15.7:0/3654097607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebd8f15f60 0x55ebd8f315c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.245+0000 7f90688a4700 1 -- 172.21.15.7:0/3654097607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebd8f15640 msgr2=0x55ebd8f31040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:57.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.245+0000 7f90688a4700 1 --2- 172.21.15.7:0/3654097607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebd8f15640 0x55ebd8f31040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.245+0000 7f90688a4700 1 -- 172.21.15.7:0/3654097607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebd8f31e00 con 0x55ebd8f0ff20 2024-06-10T01:13:57.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.245+0000 7f90688a4700 1 --2- 172.21.15.7:0/3654097607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebd8f0ff20 0x55ebd8f225e0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f9058000c00 tx=0x7f905801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4248725bbb772b89 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:57.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.245+0000 7f9043fff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:13:57.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.245+0000 7f9043fff700 1 -- 172.21.15.7:0/3654097607 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90580343a0 con 0x55ebd8f0ff20 2024-06-10T01:13:57.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.245+0000 7f9043fff700 1 -- 172.21.15.7:0/3654097607 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f9058034540 con 0x55ebd8f0ff20 2024-06-10T01:13:57.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.245+0000 7f907538bb80 1 -- 172.21.15.7:0/3654097607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90500a47d0 con 0x55ebd8f0ff20 2024-06-10T01:13:57.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.246+0000 7f9043fff700 1 -- 172.21.15.7:0/3654097607 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9058043050 con 0x55ebd8f0ff20 2024-06-10T01:13:57.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.246+0000 7f907538bb80 10 client.17463 fetch_fsmap learned FSMap version 35 2024-06-10T01:13:57.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.246+0000 7f907538bb80 1 -- 172.21.15.7:0/3654097607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ebd8f94700 con 0x55ebd8f0ff20 2024-06-10T01:13:57.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.246+0000 7f9043fff700 1 -- 172.21.15.7:0/3654097607 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90500a47d0 con 0x55ebd8f0ff20 2024-06-10T01:13:57.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.246+0000 7f9043fff700 1 -- 172.21.15.7:0/3654097607 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f905800f970 con 0x55ebd8f0ff20 2024-06-10T01:13:57.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.246+0000 7f907538bb80 10 client.17463 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:13:57.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.246+0000 7f907538bb80 10 client.17463 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:13:57.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.246+0000 7f907538bb80 20 client.17463 populate_metadata read hostname 'smithi007' 2024-06-10T01:13:57.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.246+0000 7f907538bb80 1 --2- 172.21.15.7:0/3654097607 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ebd8f94e40 0x55ebd8f97260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:57.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.246+0000 7f907538bb80 4 client.17463 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:13:57.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.246+0000 7f907538bb80 1 -- 172.21.15.7:0/3654097607 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ebd8e69290 con 0x55ebd8f94e40 2024-06-10T01:13:57.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.246+0000 7f9063fff700 1 --2- 172.21.15.7:0/3654097607 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ebd8f94e40 0x55ebd8f97260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:57.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.247+0000 7f9063fff700 1 --2- 172.21.15.7:0/3654097607 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ebd8f94e40 0x55ebd8f97260 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64312cba201bcac8 server_cookie=2c0145d45f1a9d5d in_seq=0 out_seq=0 2024-06-10T01:13:57.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.247+0000 7f9043fff700 10 client.17463 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:57.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.248+0000 7f9043fff700 1 -- 172.21.15.7:0/3654097607 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ebd8e69290 con 0x55ebd8f94e40 2024-06-10T01:13:57.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.248+0000 7f9043fff700 10 client.17463 handle_command_reply: tid=0 2024-06-10T01:13:57.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.248+0000 7f907538bb80 1 client.17463 shutdown 2024-06-10T01:13:57.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.248+0000 7f907538bb80 1 -- 172.21.15.7:0/3654097607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebd8f0ff20 msgr2=0x55ebd8f225e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:57.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.248+0000 7f907538bb80 1 --2- 172.21.15.7:0/3654097607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebd8f0ff20 0x55ebd8f225e0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f9058000c00 tx=0x7f905801f000 comp rx=0 tx=0).stop 2024-06-10T01:13:57.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.248+0000 7f907538bb80 1 -- 172.21.15.7:0/3654097607 shutdown_connections 2024-06-10T01:13:57.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.248+0000 7f907538bb80 1 --2- 172.21.15.7:0/3654097607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebd8f15f60 0x55ebd8f315c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.248+0000 7f907538bb80 1 --2- 172.21.15.7:0/3654097607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebd8f0ff20 0x55ebd8f225e0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.248+0000 7f907538bb80 1 --2- 172.21.15.7:0/3654097607 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ebd8f94e40 0x55ebd8f97260 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.248+0000 7f907538bb80 1 --2- 172.21.15.7:0/3654097607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebd8f15640 0x55ebd8f31040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.248+0000 7f907538bb80 1 -- 172.21.15.7:0/3654097607 >> 172.21.15.7:0/3654097607 conn(0x55ebd8e3acc0 msgr2=0x55ebd8f16340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:57.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.248+0000 7f9043fff700 0 client.17463 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:57.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.248+0000 7f907538bb80 1 -- 172.21.15.7:0/3654097607 shutdown_connections 2024-06-10T01:13:57.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.248+0000 7f907538bb80 1 -- 172.21.15.7:0/3654097607 wait complete. 2024-06-10T01:13:57.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.248+0000 7f907538bb80 20 client.17463 trim_cache size 0 max 16384 2024-06-10T01:13:57.247 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:13:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T01:13:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-10T01:13:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: "4da96152-8bc2-4e17-ac3f-c2eeb2a3acdc": { 2024-06-10T01:13:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-10T01:13:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "4da96152-8bc2-4e17-ac3f-c2eeb2a3acdc", 2024-06-10T01:13:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-10T01:13:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:13:57.248 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:13:57.248 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:13:57.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.249+0000 7f906ab08700 1 -- 172.21.15.7:0/3539551981 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f90540a2c20 msgr2=0x7f90540a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:57.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.249+0000 7f906ab08700 1 --2- 172.21.15.7:0/3539551981 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f90540a2c20 0x7f90540a50e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f905800b2e0 tx=0x7f9058005960 comp rx=0 tx=0).stop 2024-06-10T01:13:57.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.249+0000 7f906ab08700 1 -- 172.21.15.7:0/3539551981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9064101d40 msgr2=0x7f906419a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:57.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.249+0000 7f906ab08700 1 --2- 172.21.15.7:0/3539551981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9064101d40 0x7f906419a010 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f905000e9a0 tx=0x7f905000ecb0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.249+0000 7f906ab08700 1 -- 172.21.15.7:0/3539551981 shutdown_connections 2024-06-10T01:13:57.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.249+0000 7f906ab08700 1 --2- 172.21.15.7:0/3539551981 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f90540a2c20 0x7f90540a50e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.249+0000 7f906ab08700 1 --2- 172.21.15.7:0/3539551981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90641073f0 0x7f90641a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.249+0000 7f906ab08700 1 --2- 172.21.15.7:0/3539551981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9064101380 0x7f9064199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.249+0000 7f906ab08700 1 --2- 172.21.15.7:0/3539551981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9064101d40 0x7f906419a010 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:57.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.249+0000 7f906ab08700 1 -- 172.21.15.7:0/3539551981 >> 172.21.15.7:0/3539551981 conn(0x7f9064078300 msgr2=0x7f9064100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:57.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.249+0000 7f906ab08700 1 -- 172.21.15.7:0/3539551981 shutdown_connections 2024-06-10T01:13:57.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:13:57.249+0000 7f906ab08700 1 -- 172.21.15.7:0/3539551981 wait complete. 2024-06-10T01:13:57.512 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:57 smithi007 ceph-mon[32398]: pgmap v670: 129 pgs: 129 active+clean; 103 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 307 op/s 2024-06-10T01:13:57.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:57 smithi175 ceph-mon[36636]: pgmap v670: 129 pgs: 129 active+clean; 103 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 307 op/s 2024-06-10T01:13:57.739 INFO:tasks.workunit.client.0.smithi007.stdout: 1 582519 7.98 MB/sec execute 278 sec latency 95.976 ms 2024-06-10T01:13:57.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:57 smithi078 ceph-mon[36683]: pgmap v670: 129 pgs: 129 active+clean; 103 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 307 op/s 2024-06-10T01:13:57.953 INFO:tasks.cephfs.filesystem:scrub status for tag:4da96152-8bc2-4e17-ac3f-c2eeb2a3acdc - {'path': '/', 'tag': '4da96152-8bc2-4e17-ac3f-c2eeb2a3acdc', 'options': 'recursive,force'} 2024-06-10T01:13:58.485 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 16.0 2024-06-10T01:13:58.485 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:13:58.739 INFO:tasks.workunit.client.0.smithi007.stdout: 1 584225 7.98 MB/sec execute 279 sec latency 15.020 ms 2024-06-10T01:13:58.751 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:13:59.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:13:59 smithi175 ceph-mon[36636]: pgmap v671: 129 pgs: 129 active+clean; 105 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 365 op/s 2024-06-10T01:13:59.739 INFO:tasks.workunit.client.0.smithi007.stdout: 1 586498 8.00 MB/sec execute 280 sec latency 15.021 ms 2024-06-10T01:13:59.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:13:59 smithi007 ceph-mon[32398]: pgmap v671: 129 pgs: 129 active+clean; 105 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 365 op/s 2024-06-10T01:13:59.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:13:59 smithi078 ceph-mon[36683]: pgmap v671: 129 pgs: 129 active+clean; 105 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 365 op/s 2024-06-10T01:13:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.871+0000 7f51ac69e640 1 -- 172.21.15.7:0/1707756530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51a4101b00 msgr2=0x7f51a4101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.871+0000 7f51ac69e640 1 --2- 172.21.15.7:0/1707756530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51a4101b00 0x7f51a4101f80 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f5194009900 tx=0x7f519402a3f0 comp rx=0 tx=0).stop 2024-06-10T01:13:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.871+0000 7f51ac69e640 1 -- 172.21.15.7:0/1707756530 shutdown_connections 2024-06-10T01:13:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.871+0000 7f51ac69e640 1 --2- 172.21.15.7:0/1707756530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51a41024c0 0x7f51a4108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.871+0000 7f51ac69e640 1 --2- 172.21.15.7:0/1707756530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51a4101b00 0x7f51a4101f80 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.871+0000 7f51ac69e640 1 --2- 172.21.15.7:0/1707756530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51a4100910 0x7f51a4100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.871+0000 7f51ac69e640 1 -- 172.21.15.7:0/1707756530 >> 172.21.15.7:0/1707756530 conn(0x7f51a40fbe80 msgr2=0x7f51a40fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.871+0000 7f51ac69e640 1 -- 172.21.15.7:0/1707756530 shutdown_connections 2024-06-10T01:13:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.871+0000 7f51ac69e640 1 -- 172.21.15.7:0/1707756530 wait complete. 2024-06-10T01:13:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.872+0000 7f51ac69e640 1 Processor -- start 2024-06-10T01:13:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.872+0000 7f51ac69e640 1 -- start start 2024-06-10T01:13:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.872+0000 7f51ac69e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51a4100910 0x7f51a4197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.872+0000 7f51ac69e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51a4101b00 0x7f51a4198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.872+0000 7f51ac69e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51a41024c0 0x7f51a419f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.872+0000 7f51ac69e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f51a410c590 con 0x7f51a4100910 2024-06-10T01:13:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.872+0000 7f51ac69e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f51a410c410 con 0x7f51a4101b00 2024-06-10T01:13:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.872+0000 7f51ac69e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f51a410c710 con 0x7f51a41024c0 2024-06-10T01:13:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.872+0000 7f51aa43a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51a4100910 0x7f51a4197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.872+0000 7f51aa43a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51a4100910 0x7f51a4197d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54992/0 (socket says 172.21.15.7:54992) 2024-06-10T01:13:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.872+0000 7f51aa43a640 1 -- 172.21.15.7:0/3800323191 learned_addr learned my addr 172.21.15.7:0/3800323191 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.872+0000 7f51a9c39640 1 --2- 172.21.15.7:0/3800323191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51a4101b00 0x7f51a4198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.872+0000 7f51aac3b640 1 --2- 172.21.15.7:0/3800323191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51a41024c0 0x7f51a419f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.872+0000 7f51aa43a640 1 -- 172.21.15.7:0/3800323191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51a41024c0 msgr2=0x7f51a419f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.872+0000 7f51aa43a640 1 --2- 172.21.15.7:0/3800323191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51a41024c0 0x7f51a419f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.872+0000 7f51aa43a640 1 -- 172.21.15.7:0/3800323191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51a4101b00 msgr2=0x7f51a4198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.872+0000 7f51aa43a640 1 --2- 172.21.15.7:0/3800323191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51a4101b00 0x7f51a4198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.872+0000 7f51aa43a640 1 -- 172.21.15.7:0/3800323191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a419f940 con 0x7f51a4100910 2024-06-10T01:13:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.872+0000 7f51aac3b640 1 --2- 172.21.15.7:0/3800323191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51a41024c0 0x7f51a419f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:13:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.873+0000 7f51aa43a640 1 --2- 172.21.15.7:0/3800323191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51a4100910 0x7f51a4197d30 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f51a000e9c0 tx=0x7f51a000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.873+0000 7f519b7fe640 1 -- 172.21.15.7:0/3800323191 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a000cd80 con 0x7f51a4100910 2024-06-10T01:13:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.873+0000 7f51ac69e640 1 -- 172.21.15.7:0/3800323191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a419fc30 con 0x7f51a4100910 2024-06-10T01:13:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.873+0000 7f519b7fe640 1 -- 172.21.15.7:0/3800323191 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51a00035f0 con 0x7f51a4100910 2024-06-10T01:13:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.873+0000 7f519b7fe640 1 -- 172.21.15.7:0/3800323191 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a0010690 con 0x7f51a4100910 2024-06-10T01:13:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.873+0000 7f51ac69e640 1 -- 172.21.15.7:0/3800323191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a41a0180 con 0x7f51a4100910 2024-06-10T01:13:59.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.875+0000 7f519b7fe640 1 -- 172.21.15.7:0/3800323191 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f51a00037a0 con 0x7f51a4100910 2024-06-10T01:13:59.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.875+0000 7f51af3a9740 1 Processor -- start 2024-06-10T01:13:59.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.875+0000 7f51af3a9740 1 -- start start 2024-06-10T01:13:59.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.876+0000 7f51af3a9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ee6a85010 0x557ee6a85430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:59.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.876+0000 7f51af3a9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557ee6a85a00 0x557ee6a8a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:59.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.876+0000 7f51af3a9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ee6a8ab70 0x557ee6a8d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:59.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.876+0000 7f51af3a9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557ee69e84f0 con 0x557ee6a85010 2024-06-10T01:13:59.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.876+0000 7f51af3a9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557ee6986870 con 0x557ee6a8ab70 2024-06-10T01:13:59.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.876+0000 7f51af3a9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557ee69e44b0 con 0x557ee6a85a00 2024-06-10T01:13:59.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.876+0000 7f51aac3b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ee6a85010 0x557ee6a85430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:59.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.876+0000 7f51aac3b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ee6a85010 0x557ee6a85430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54998/0 (socket says 172.21.15.7:54998) 2024-06-10T01:13:59.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.876+0000 7f51aac3b640 1 -- 172.21.15.7:0/606415865 learned_addr learned my addr 172.21.15.7:0/606415865 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:59.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.876+0000 7f51a9c39640 1 --2- 172.21.15.7:0/606415865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ee6a8ab70 0x557ee6a8d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:59.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.876+0000 7f51aa43a640 1 --2- 172.21.15.7:0/606415865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557ee6a85a00 0x557ee6a8a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:59.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.876+0000 7f519b7fe640 1 --2- 172.21.15.7:0/3800323191 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f51900a2b80 0x7f51900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:59.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.876+0000 7f51aac3b640 1 -- 172.21.15.7:0/606415865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557ee6a85a00 msgr2=0x557ee6a8a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:59.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.876+0000 7f51aac3b640 1 --2- 172.21.15.7:0/606415865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557ee6a85a00 0x557ee6a8a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:59.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.876+0000 7f51aac3b640 1 -- 172.21.15.7:0/606415865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ee6a8ab70 msgr2=0x557ee6a8d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:59.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.876+0000 7f51aac3b640 1 --2- 172.21.15.7:0/606415865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ee6a8ab70 0x557ee6a8d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.876+0000 7f51aac3b640 1 -- 172.21.15.7:0/606415865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ee69a29b0 con 0x557ee6a85010 2024-06-10T01:13:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.876+0000 7f51aa43a640 1 --2- 172.21.15.7:0/606415865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557ee6a85a00 0x557ee6a8a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:13:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.876+0000 7f519b7fe640 1 -- 172.21.15.7:0/3800323191 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f51a00cf8d0 con 0x7f51a4100910 2024-06-10T01:13:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.876+0000 7f51aac3b640 1 --2- 172.21.15.7:0/3800323191 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f51900a2b80 0x7f51900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.876+0000 7f51aac3b640 1 --2- 172.21.15.7:0/606415865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ee6a85010 0x557ee6a85430 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f519c009f90 tx=0x7f519c00f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50bb93465296ff4a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.877+0000 7f5198ff9640 1 -- 172.21.15.7:0/606415865 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c011ac0 con 0x557ee6a85010 2024-06-10T01:13:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.877+0000 7f5198ff9640 1 -- 172.21.15.7:0/606415865 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f519c011ce0 con 0x557ee6a85010 2024-06-10T01:13:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.877+0000 7f5198ff9640 1 -- 172.21.15.7:0/606415865 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c01c3c0 con 0x557ee6a85010 2024-06-10T01:13:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.877+0000 7f51af3a9740 1 -- 172.21.15.7:0/606415865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ee6a85010 msgr2=0x557ee6a85430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.877+0000 7f51af3a9740 1 --2- 172.21.15.7:0/606415865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ee6a85010 0x557ee6a85430 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f519c009f90 tx=0x7f519c00f7f0 comp rx=0 tx=0).stop 2024-06-10T01:13:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.877+0000 7f51aac3b640 1 --2- 172.21.15.7:0/3800323191 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f51900a2b80 0x7f51900a5040 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f519c01e000 tx=0x7f519c021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.877+0000 7f51af3a9740 1 -- 172.21.15.7:0/606415865 shutdown_connections 2024-06-10T01:13:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.877+0000 7f51af3a9740 1 --2- 172.21.15.7:0/606415865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ee6a8ab70 0x557ee6a8d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.877+0000 7f51af3a9740 1 --2- 172.21.15.7:0/606415865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557ee6a85a00 0x557ee6a8a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.877+0000 7f51af3a9740 1 --2- 172.21.15.7:0/606415865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ee6a85010 0x557ee6a85430 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.877+0000 7f51af3a9740 1 -- 172.21.15.7:0/606415865 >> 172.21.15.7:0/606415865 conn(0x557ee6a80680 msgr2=0x557ee6a80a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.878+0000 7f51af3a9740 1 -- 172.21.15.7:0/606415865 shutdown_connections 2024-06-10T01:13:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.878+0000 7f51af3a9740 1 -- 172.21.15.7:0/606415865 wait complete. 2024-06-10T01:13:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.878+0000 7f51af3a9740 1 -- start start 2024-06-10T01:13:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.878+0000 7f51af3a9740 1 Processor -- start 2024-06-10T01:13:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.878+0000 7f51af3a9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557ee6a85010 0x557ee6a8f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.878+0000 7f51af3a9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ee6a85a00 0x557ee6a8f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.878+0000 7f51af3a9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ee6a8ab70 0x557ee6a967a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.878+0000 7f51af3a9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557ee6a8db20 con 0x557ee6a8ab70 2024-06-10T01:13:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.878+0000 7f51af3a9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557ee6971ae0 con 0x557ee6a85a00 2024-06-10T01:13:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.878+0000 7f51af3a9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557ee6a8dca0 con 0x557ee6a85010 2024-06-10T01:13:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.878+0000 7f51af3a9740 10 client.0 _ll_register_callbacks cb 0x557ee69aa9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:13:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.878+0000 7f51aac3b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ee6a8ab70 0x557ee6a967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.878+0000 7f51aa43a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557ee6a85010 0x557ee6a8f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.878+0000 7f51a9c39640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ee6a85a00 0x557ee6a8f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.878+0000 7f51aac3b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ee6a8ab70 0x557ee6a967a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55000/0 (socket says 172.21.15.7:55000) 2024-06-10T01:13:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.878+0000 7f51aac3b640 1 -- 172.21.15.7:0/3176193313 learned_addr learned my addr 172.21.15.7:0/3176193313 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.878+0000 7f51a9c39640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ee6a85a00 0x557ee6a8f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:42394/0 (socket says 172.21.15.7:42394) 2024-06-10T01:13:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.878+0000 7f51aa43a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557ee6a85010 0x557ee6a8f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43930/0 (socket says 172.21.15.7:43930) 2024-06-10T01:13:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.879+0000 7f51a9c39640 1 -- 172.21.15.7:0/3176193313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557ee6a85010 msgr2=0x557ee6a8f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.879+0000 7f51a9c39640 1 --2- 172.21.15.7:0/3176193313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557ee6a85010 0x557ee6a8f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.879+0000 7f51a9c39640 1 -- 172.21.15.7:0/3176193313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ee6a8ab70 msgr2=0x557ee6a967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.879+0000 7f51a9c39640 1 --2- 172.21.15.7:0/3176193313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ee6a8ab70 0x557ee6a967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.879+0000 7f51a9c39640 1 -- 172.21.15.7:0/3176193313 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ee66fc210 con 0x557ee6a85a00 2024-06-10T01:13:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.879+0000 7f51a9c39640 1 --2- 172.21.15.7:0/3176193313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ee6a85a00 0x557ee6a8f760 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5194031440 tx=0x7f5194005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d13cebeaa85f01a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.879+0000 7f518dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:13:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.879+0000 7f518dffb640 1 -- 172.21.15.7:0/3176193313 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194035040 con 0x557ee6a85a00 2024-06-10T01:13:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.879+0000 7f51af3a9740 1 -- 172.21.15.7:0/3176193313 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f519c011ce0 con 0x557ee6a85a00 2024-06-10T01:13:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.879+0000 7f518dffb640 1 -- 172.21.15.7:0/3176193313 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f51940319b0 con 0x557ee6a85a00 2024-06-10T01:13:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.879+0000 7f518dffb640 1 -- 172.21.15.7:0/3176193313 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5194004040 con 0x557ee6a85a00 2024-06-10T01:13:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.879+0000 7f51af3a9740 10 client.26888 fetch_fsmap learned FSMap version 35 2024-06-10T01:13:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.879+0000 7f51af3a9740 1 -- 172.21.15.7:0/3176193313 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557ee67c8620 con 0x557ee6a85a00 2024-06-10T01:13:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.879+0000 7f518dffb640 1 -- 172.21.15.7:0/3176193313 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f519c011ce0 con 0x557ee6a85a00 2024-06-10T01:13:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.880+0000 7f518dffb640 1 -- 172.21.15.7:0/3176193313 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f51940049e0 con 0x557ee6a85a00 2024-06-10T01:13:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.880+0000 7f51af3a9740 10 client.26888 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:13:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.880+0000 7f51af3a9740 10 client.26888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:13:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.880+0000 7f51af3a9740 20 client.26888 populate_metadata read hostname 'smithi007' 2024-06-10T01:13:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.880+0000 7f51af3a9740 1 --2- 172.21.15.7:0/3176193313 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557ee6a9b040 0x557ee6a9d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.880+0000 7f51af3a9740 4 client.26888 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:13:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.880+0000 7f51af3a9740 1 -- 172.21.15.7:0/3176193313 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ee69fbf10 con 0x557ee6a9b040 2024-06-10T01:13:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.880+0000 7f51aa43a640 1 --2- 172.21.15.7:0/3176193313 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557ee6a9b040 0x557ee6a9d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.880+0000 7f51aa43a640 1 --2- 172.21.15.7:0/3176193313 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557ee6a9b040 0x557ee6a9d490 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=183ef82d3b33602d server_cookie=ace05992c6b29a5d in_seq=0 out_seq=0 2024-06-10T01:13:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.880+0000 7f518dffb640 10 client.26888 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.881+0000 7f518dffb640 1 -- 172.21.15.7:0/3176193313 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557ee69fbf10 con 0x557ee6a9b040 2024-06-10T01:13:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.881+0000 7f518dffb640 10 client.26888 handle_command_reply: tid=0 2024-06-10T01:13:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.881+0000 7f51af3a9740 1 client.26888 shutdown 2024-06-10T01:13:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.881+0000 7f51af3a9740 1 -- 172.21.15.7:0/3176193313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ee6a85a00 msgr2=0x557ee6a8f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.881+0000 7f51af3a9740 1 --2- 172.21.15.7:0/3176193313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ee6a85a00 0x557ee6a8f760 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5194031440 tx=0x7f5194005e50 comp rx=0 tx=0).stop 2024-06-10T01:13:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.881+0000 7f51af3a9740 1 -- 172.21.15.7:0/3176193313 shutdown_connections 2024-06-10T01:13:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.881+0000 7f51af3a9740 1 --2- 172.21.15.7:0/3176193313 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557ee6a9b040 0x557ee6a9d490 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.881+0000 7f51af3a9740 1 --2- 172.21.15.7:0/3176193313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ee6a8ab70 0x557ee6a967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.881+0000 7f51af3a9740 1 --2- 172.21.15.7:0/3176193313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ee6a85a00 0x557ee6a8f760 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.881+0000 7f518dffb640 0 client.26888 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.881+0000 7f51af3a9740 1 --2- 172.21.15.7:0/3176193313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557ee6a85010 0x557ee6a8f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.881+0000 7f51af3a9740 1 -- 172.21.15.7:0/3176193313 >> 172.21.15.7:0/3176193313 conn(0x557ee6a80680 msgr2=0x557ee6a7f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.882+0000 7f51af3a9740 1 -- 172.21.15.7:0/3176193313 shutdown_connections 2024-06-10T01:13:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.882+0000 7f51af3a9740 1 -- 172.21.15.7:0/3176193313 wait complete. 2024-06-10T01:13:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.882+0000 7f51af3a9740 20 client.26888 trim_cache size 0 max 16384 2024-06-10T01:13:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.900+0000 7f51af3a9740 1 Processor -- start 2024-06-10T01:13:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.900+0000 7f51af3a9740 1 -- start start 2024-06-10T01:13:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.900+0000 7f51af3a9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ee6a85010 0x557ee6a86b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.900+0000 7f51af3a9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ee6a85a00 0x557ee6a96760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.900+0000 7f51af3a9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557ee6a8ab70 0x557ee6a870c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.900+0000 7f51af3a9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f51a41acf80 con 0x557ee6a85010 2024-06-10T01:13:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.900+0000 7f51af3a9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f51a41aa7f0 con 0x557ee6a85a00 2024-06-10T01:13:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.900+0000 7f51af3a9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f51a41a8640 con 0x557ee6a8ab70 2024-06-10T01:13:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.900+0000 7f51aa43a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ee6a85010 0x557ee6a86b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.900+0000 7f51a9c39640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ee6a85a00 0x557ee6a96760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.900+0000 7f51a9c39640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ee6a85a00 0x557ee6a96760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:42410/0 (socket says 172.21.15.7:42410) 2024-06-10T01:13:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.900+0000 7f51a9c39640 1 -- 172.21.15.7:0/668761798 learned_addr learned my addr 172.21.15.7:0/668761798 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.900+0000 7f51aac3b640 1 --2- 172.21.15.7:0/668761798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557ee6a8ab70 0x557ee6a870c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.900+0000 7f51a9c39640 1 -- 172.21.15.7:0/668761798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557ee6a8ab70 msgr2=0x557ee6a870c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.900+0000 7f51a9c39640 1 --2- 172.21.15.7:0/668761798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557ee6a8ab70 0x557ee6a870c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.900+0000 7f51a9c39640 1 -- 172.21.15.7:0/668761798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ee6a85010 msgr2=0x557ee6a86b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.900+0000 7f51a9c39640 1 --2- 172.21.15.7:0/668761798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ee6a85010 0x557ee6a86b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.900+0000 7f51a9c39640 1 -- 172.21.15.7:0/668761798 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ee6a85de0 con 0x557ee6a85a00 2024-06-10T01:13:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.901+0000 7f51a9c39640 1 --2- 172.21.15.7:0/668761798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ee6a85a00 0x557ee6a96760 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5194009850 tx=0x7f51940315d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf805aae89849536 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.901+0000 7f518ffff640 1 -- 172.21.15.7:0/668761798 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194035030 con 0x557ee6a85a00 2024-06-10T01:13:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.901+0000 7f518ffff640 1 -- 172.21.15.7:0/668761798 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5194009520 con 0x557ee6a85a00 2024-06-10T01:13:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.901+0000 7f518ffff640 1 -- 172.21.15.7:0/668761798 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194003990 con 0x557ee6a85a00 2024-06-10T01:13:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.902+0000 7f51af3a9740 1 -- 172.21.15.7:0/668761798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ee6a85a00 msgr2=0x557ee6a96760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:13:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.902+0000 7f51af3a9740 1 --2- 172.21.15.7:0/668761798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ee6a85a00 0x557ee6a96760 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5194009850 tx=0x7f51940315d0 comp rx=0 tx=0).stop 2024-06-10T01:13:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.902+0000 7f51af3a9740 1 -- 172.21.15.7:0/668761798 shutdown_connections 2024-06-10T01:13:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.902+0000 7f51af3a9740 1 --2- 172.21.15.7:0/668761798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557ee6a8ab70 0x557ee6a870c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.902+0000 7f51af3a9740 1 --2- 172.21.15.7:0/668761798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ee6a85a00 0x557ee6a96760 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.902+0000 7f51af3a9740 1 --2- 172.21.15.7:0/668761798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ee6a85010 0x557ee6a86b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.902+0000 7f51af3a9740 1 -- 172.21.15.7:0/668761798 >> 172.21.15.7:0/668761798 conn(0x557ee6a80680 msgr2=0x557ee6a8f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:13:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.902+0000 7f51af3a9740 1 -- 172.21.15.7:0/668761798 shutdown_connections 2024-06-10T01:13:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.902+0000 7f51af3a9740 1 -- 172.21.15.7:0/668761798 wait complete. 2024-06-10T01:13:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.902+0000 7f51af3a9740 1 -- start start 2024-06-10T01:13:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.902+0000 7f51af3a9740 1 Processor -- start 2024-06-10T01:13:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.903+0000 7f51af3a9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557ee6a85010 0x557ee6a93910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.903+0000 7f51af3a9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ee6a85a00 0x557ee6a93e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.903+0000 7f51af3a9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ee6a8ab70 0x557ee6a94e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.903+0000 7f51af3a9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f51a41a8cb0 con 0x557ee6a8ab70 2024-06-10T01:13:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.903+0000 7f51af3a9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f51a41aa120 con 0x557ee6a85a00 2024-06-10T01:13:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.903+0000 7f51af3a9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f51a41a9380 con 0x557ee6a85010 2024-06-10T01:13:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.903+0000 7f51af3a9740 10 client.0 _ll_register_callbacks cb 0x7f5194009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:13:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.903+0000 7f51aac3b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ee6a8ab70 0x557ee6a94e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.903+0000 7f51a9c39640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ee6a85a00 0x557ee6a93e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.903+0000 7f51aac3b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ee6a8ab70 0x557ee6a94e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55020/0 (socket says 172.21.15.7:55020) 2024-06-10T01:13:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.903+0000 7f51aa43a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557ee6a85010 0x557ee6a93910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.903+0000 7f51aac3b640 1 -- 172.21.15.7:0/2204715251 learned_addr learned my addr 172.21.15.7:0/2204715251 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:13:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.903+0000 7f51aa43a640 1 -- 172.21.15.7:0/2204715251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ee6a85a00 msgr2=0x557ee6a93e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.903+0000 7f51aa43a640 1 --2- 172.21.15.7:0/2204715251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ee6a85a00 0x557ee6a93e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.903+0000 7f51aa43a640 1 -- 172.21.15.7:0/2204715251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ee6a8ab70 msgr2=0x557ee6a94e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:13:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.903+0000 7f51aa43a640 1 --2- 172.21.15.7:0/2204715251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ee6a8ab70 0x557ee6a94e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:13:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.903+0000 7f51aa43a640 1 -- 172.21.15.7:0/2204715251 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ee6a95450 con 0x557ee6a85010 2024-06-10T01:13:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.903+0000 7f51a9c39640 1 --2- 172.21.15.7:0/2204715251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ee6a85a00 0x557ee6a93e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.903+0000 7f51aac3b640 1 --2- 172.21.15.7:0/2204715251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ee6a8ab70 0x557ee6a94e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:13:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.904+0000 7f51aa43a640 1 --2- 172.21.15.7:0/2204715251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557ee6a85010 0x557ee6a93910 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f51a00cd0a0 tx=0x7f51a000e940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dc547f92edf00 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:13:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.904+0000 7f518dffb640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:13:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.904+0000 7f518dffb640 1 -- 172.21.15.7:0/2204715251 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a00a4a20 con 0x557ee6a85010 2024-06-10T01:13:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.904+0000 7f51af3a9740 1 -- 172.21.15.7:0/2204715251 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5194009520 con 0x557ee6a85010 2024-06-10T01:13:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.904+0000 7f518dffb640 1 -- 172.21.15.7:0/2204715251 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f51a0003b30 con 0x557ee6a85010 2024-06-10T01:13:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.904+0000 7f518dffb640 1 -- 172.21.15.7:0/2204715251 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f51a0098390 con 0x557ee6a85010 2024-06-10T01:13:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.904+0000 7f51af3a9740 10 client.26911 fetch_fsmap learned FSMap version 35 2024-06-10T01:13:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.904+0000 7f51af3a9740 1 -- 172.21.15.7:0/2204715251 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557ee6a967d0 con 0x557ee6a85010 2024-06-10T01:13:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.904+0000 7f518dffb640 1 -- 172.21.15.7:0/2204715251 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5194009520 con 0x557ee6a85010 2024-06-10T01:13:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.904+0000 7f518dffb640 1 -- 172.21.15.7:0/2204715251 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f51a009ad00 con 0x557ee6a85010 2024-06-10T01:13:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.904+0000 7f51af3a9740 10 client.26911 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:13:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.904+0000 7f51af3a9740 10 client.26911 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:13:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.904+0000 7f51af3a9740 20 client.26911 populate_metadata read hostname 'smithi007' 2024-06-10T01:13:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.904+0000 7f51af3a9740 1 --2- 172.21.15.7:0/2204715251 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557ee6b061b0 0x557ee6b085d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:13:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.904+0000 7f51af3a9740 4 client.26911 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:13:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.904+0000 7f51af3a9740 1 -- 172.21.15.7:0/2204715251 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557ee69f5c20 con 0x557ee6b061b0 2024-06-10T01:13:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.904+0000 7f51a9c39640 1 --2- 172.21.15.7:0/2204715251 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557ee6b061b0 0x557ee6b085d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:13:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.905+0000 7f518dffb640 10 client.26911 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:13:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:13:59.905+0000 7f51a9c39640 1 --2- 172.21.15.7:0/2204715251 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557ee6b061b0 0x557ee6b085d0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddb0cb070d02fa8f server_cookie=a5e0e93b0a5f5365 in_seq=0 out_seq=0 2024-06-10T01:14:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:00.007+0000 7f518dffb640 1 -- 172.21.15.7:0/2204715251 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557ee69f5c20 con 0x557ee6b061b0 2024-06-10T01:14:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:00.007+0000 7f518dffb640 10 client.26911 handle_command_reply: tid=0 2024-06-10T01:14:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:00.007+0000 7f51af3a9740 1 client.26911 shutdown 2024-06-10T01:14:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:00.007+0000 7f51af3a9740 1 -- 172.21.15.7:0/2204715251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557ee6a85010 msgr2=0x557ee6a93910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:00.007+0000 7f51af3a9740 1 --2- 172.21.15.7:0/2204715251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557ee6a85010 0x557ee6a93910 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f51a00cd0a0 tx=0x7f51a000e940 comp rx=0 tx=0).stop 2024-06-10T01:14:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:00.007+0000 7f51af3a9740 1 -- 172.21.15.7:0/2204715251 shutdown_connections 2024-06-10T01:14:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:00.007+0000 7f51af3a9740 1 --2- 172.21.15.7:0/2204715251 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557ee6b061b0 0x557ee6b085d0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:00.007+0000 7f51af3a9740 1 --2- 172.21.15.7:0/2204715251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ee6a8ab70 0x557ee6a94e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:00.007+0000 7f51af3a9740 1 --2- 172.21.15.7:0/2204715251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ee6a85a00 0x557ee6a93e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:00.007+0000 7f51af3a9740 1 --2- 172.21.15.7:0/2204715251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557ee6a85010 0x557ee6a93910 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:00.007+0000 7f518dffb640 0 client.26911 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:00.007+0000 7f51af3a9740 1 -- 172.21.15.7:0/2204715251 >> 172.21.15.7:0/2204715251 conn(0x557ee6a80680 msgr2=0x557ee6a8f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:00.007+0000 7f51af3a9740 1 -- 172.21.15.7:0/2204715251 shutdown_connections 2024-06-10T01:14:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:00.007+0000 7f51af3a9740 1 -- 172.21.15.7:0/2204715251 wait complete. 2024-06-10T01:14:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:00.008+0000 7f51af3a9740 20 client.26911 trim_cache size 0 max 16384 2024-06-10T01:14:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:14:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:14:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:14:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:00.008+0000 7f51ac69e640 1 -- 172.21.15.7:0/3800323191 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f51900a2b80 msgr2=0x7f51900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:00.008+0000 7f51ac69e640 1 --2- 172.21.15.7:0/3800323191 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f51900a2b80 0x7f51900a5040 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f519c01e000 tx=0x7f519c021000 comp rx=0 tx=0).stop 2024-06-10T01:14:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:00.008+0000 7f51ac69e640 1 -- 172.21.15.7:0/3800323191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51a4100910 msgr2=0x7f51a4197d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:00.008+0000 7f51ac69e640 1 --2- 172.21.15.7:0/3800323191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51a4100910 0x7f51a4197d30 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f51a000e9c0 tx=0x7f51a000ee80 comp rx=0 tx=0).stop 2024-06-10T01:14:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:00.008+0000 7f51ac69e640 1 -- 172.21.15.7:0/3800323191 shutdown_connections 2024-06-10T01:14:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:00.008+0000 7f51ac69e640 1 --2- 172.21.15.7:0/3800323191 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f51900a2b80 0x7f51900a5040 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:00.008+0000 7f51ac69e640 1 --2- 172.21.15.7:0/3800323191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51a41024c0 0x7f51a419f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:00.008+0000 7f51ac69e640 1 --2- 172.21.15.7:0/3800323191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51a4101b00 0x7f51a4198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:00.008+0000 7f51ac69e640 1 --2- 172.21.15.7:0/3800323191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51a4100910 0x7f51a4197d30 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:00.008+0000 7f51ac69e640 1 -- 172.21.15.7:0/3800323191 >> 172.21.15.7:0/3800323191 conn(0x7f51a40fbe80 msgr2=0x7f51a406a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:00.009+0000 7f51ac69e640 1 -- 172.21.15.7:0/3800323191 shutdown_connections 2024-06-10T01:14:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:00.009+0000 7f51ac69e640 1 -- 172.21.15.7:0/3800323191 wait complete. 2024-06-10T01:14:00.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.513+0000 7f29d2c05700 1 -- 172.21.15.7:0/1677878630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29cc108500 msgr2=0x7f29cc1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:00.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.513+0000 7f29d2c05700 1 --2- 172.21.15.7:0/1677878630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29cc108500 0x7f29cc1088e0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f29c0009990 tx=0x7f29c0009ca0 comp rx=0 tx=0).stop 2024-06-10T01:14:00.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.513+0000 7f29d2c05700 1 -- 172.21.15.7:0/1677878630 shutdown_connections 2024-06-10T01:14:00.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.513+0000 7f29d2c05700 1 --2- 172.21.15.7:0/1677878630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29cc06ab30 0x7f29cc10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:00.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.513+0000 7f29d2c05700 1 --2- 172.21.15.7:0/1677878630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29cc108500 0x7f29cc1088e0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:00.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.513+0000 7f29d2c05700 1 --2- 172.21.15.7:0/1677878630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29cc06a170 0x7f29cc06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:00.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.513+0000 7f29d2c05700 1 -- 172.21.15.7:0/1677878630 >> 172.21.15.7:0/1677878630 conn(0x7f29cc078340 msgr2=0x7f29cc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:00.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.513+0000 7f29d2c05700 1 -- 172.21.15.7:0/1677878630 shutdown_connections 2024-06-10T01:14:00.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.513+0000 7f29d2c05700 1 -- 172.21.15.7:0/1677878630 wait complete. 2024-06-10T01:14:00.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.513+0000 7f29d2c05700 1 Processor -- start 2024-06-10T01:14:00.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.513+0000 7f29d2c05700 1 -- start start 2024-06-10T01:14:00.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.514+0000 7f29d2c05700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29cc06a170 0x7f29cc19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:00.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.514+0000 7f29d2c05700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29cc06ab30 0x7f29cc19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:00.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.514+0000 7f29d2c05700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29cc108500 0x7f29cc198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:00.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.514+0000 7f29d2c05700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc112760 con 0x7f29cc06ab30 2024-06-10T01:14:00.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.514+0000 7f29d2c05700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc1125e0 con 0x7f29cc108500 2024-06-10T01:14:00.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.514+0000 7f29d2c05700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc1128e0 con 0x7f29cc06a170 2024-06-10T01:14:00.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.514+0000 7f29cbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29cc06ab30 0x7f29cc19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:00.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.514+0000 7f29cbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29cc06ab30 0x7f29cc19ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55032/0 (socket says 172.21.15.7:55032) 2024-06-10T01:14:00.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.514+0000 7f29d09a1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29cc06a170 0x7f29cc19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:00.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.514+0000 7f29d11a2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29cc108500 0x7f29cc198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:00.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.514+0000 7f29cbfff700 1 -- 172.21.15.7:0/3250543792 learned_addr learned my addr 172.21.15.7:0/3250543792 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:00.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.514+0000 7f29cbfff700 1 -- 172.21.15.7:0/3250543792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29cc06a170 msgr2=0x7f29cc19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:00.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.514+0000 7f29cbfff700 1 --2- 172.21.15.7:0/3250543792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29cc06a170 0x7f29cc19e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:00.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.514+0000 7f29cbfff700 1 -- 172.21.15.7:0/3250543792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29cc108500 msgr2=0x7f29cc198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:00.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.514+0000 7f29cbfff700 1 --2- 172.21.15.7:0/3250543792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29cc108500 0x7f29cc198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:00.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.514+0000 7f29cbfff700 1 -- 172.21.15.7:0/3250543792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29cc1991c0 con 0x7f29cc06ab30 2024-06-10T01:14:00.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.514+0000 7f29cbfff700 1 --2- 172.21.15.7:0/3250543792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29cc06ab30 0x7f29cc19ed90 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f29b800e9a0 tx=0x7f29b800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:00.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.515+0000 7f29c9ffb700 1 -- 172.21.15.7:0/3250543792 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b800ca10 con 0x7f29cc06ab30 2024-06-10T01:14:00.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.515+0000 7f29c9ffb700 1 -- 172.21.15.7:0/3250543792 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29b800cbb0 con 0x7f29cc06ab30 2024-06-10T01:14:00.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.515+0000 7f29d2c05700 1 -- 172.21.15.7:0/3250543792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29cc1994b0 con 0x7f29cc06ab30 2024-06-10T01:14:00.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.515+0000 7f29c9ffb700 1 -- 172.21.15.7:0/3250543792 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b8018680 con 0x7f29cc06ab30 2024-06-10T01:14:00.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.515+0000 7f29d2c05700 1 -- 172.21.15.7:0/3250543792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29cc1a5720 con 0x7f29cc06ab30 2024-06-10T01:14:00.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.516+0000 7f29d2c05700 1 -- 172.21.15.7:0/3250543792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29cc10a9e0 con 0x7f29cc06ab30 2024-06-10T01:14:00.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.517+0000 7f29c9ffb700 1 -- 172.21.15.7:0/3250543792 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f29b8010430 con 0x7f29cc06ab30 2024-06-10T01:14:00.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.518+0000 7f29c9ffb700 1 --2- 172.21.15.7:0/3250543792 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f29bc0a2ca0 0x7f29bc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:00.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.518+0000 7f29d09a1700 1 --2- 172.21.15.7:0/3250543792 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f29bc0a2ca0 0x7f29bc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:00.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.518+0000 7f29c9ffb700 1 -- 172.21.15.7:0/3250543792 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f29b80cfa30 con 0x7f29cc06ab30 2024-06-10T01:14:00.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.518+0000 7f29d09a1700 1 --2- 172.21.15.7:0/3250543792 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f29bc0a2ca0 0x7f29bc0a5160 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f29c000b2e0 tx=0x7f29c001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:00.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.521+0000 7f29c9ffb700 1 -- 172.21.15.7:0/3250543792 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f29b809c610 con 0x7f29cc06ab30 2024-06-10T01:14:00.739 INFO:tasks.workunit.client.0.smithi007.stdout: 1 588550 8.00 MB/sec execute 281 sec latency 34.449 ms 2024-06-10T01:14:00.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.792+0000 7f29d2c05700 1 -- 172.21.15.7:0/3250543792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29cc069180 con 0x7f29cc06ab30 2024-06-10T01:14:00.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.792+0000 7f29c9ffb700 1 -- 172.21.15.7:0/3250543792 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11859 (secure 0 0 0) 0x7f29b809bf10 con 0x7f29cc06ab30 2024-06-10T01:14:00.787 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:14:00.787 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:13:22.220710+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:14:00.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.795+0000 7f29d2c05700 1 -- 172.21.15.7:0/3250543792 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f29bc0a2ca0 msgr2=0x7f29bc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:00.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.795+0000 7f29d2c05700 1 --2- 172.21.15.7:0/3250543792 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f29bc0a2ca0 0x7f29bc0a5160 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f29c000b2e0 tx=0x7f29c001a040 comp rx=0 tx=0).stop 2024-06-10T01:14:00.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.795+0000 7f29d2c05700 1 -- 172.21.15.7:0/3250543792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29cc06ab30 msgr2=0x7f29cc19ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:00.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.795+0000 7f29d2c05700 1 --2- 172.21.15.7:0/3250543792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29cc06ab30 0x7f29cc19ed90 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f29b800e9a0 tx=0x7f29b800ecb0 comp rx=0 tx=0).stop 2024-06-10T01:14:00.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.795+0000 7f29d2c05700 1 -- 172.21.15.7:0/3250543792 shutdown_connections 2024-06-10T01:14:00.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.795+0000 7f29d2c05700 1 --2- 172.21.15.7:0/3250543792 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f29bc0a2ca0 0x7f29bc0a5160 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:00.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.795+0000 7f29d2c05700 1 --2- 172.21.15.7:0/3250543792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29cc108500 0x7f29cc198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:00.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.795+0000 7f29d2c05700 1 --2- 172.21.15.7:0/3250543792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29cc06a170 0x7f29cc19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:00.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.795+0000 7f29d2c05700 1 --2- 172.21.15.7:0/3250543792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29cc06ab30 0x7f29cc19ed90 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:00.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.795+0000 7f29d2c05700 1 -- 172.21.15.7:0/3250543792 >> 172.21.15.7:0/3250543792 conn(0x7f29cc078340 msgr2=0x7f29cc0ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:00.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.795+0000 7f29d2c05700 1 -- 172.21.15.7:0/3250543792 shutdown_connections 2024-06-10T01:14:00.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:00.795+0000 7f29d2c05700 1 -- 172.21.15.7:0/3250543792 wait complete. 2024-06-10T01:14:00.791 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 35 2024-06-10T01:14:01.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:01 smithi175 ceph-mon[36636]: pgmap v672: 129 pgs: 129 active+clean; 106 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 345 op/s 2024-06-10T01:14:01.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:01 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3250543792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:14:01.739 INFO:tasks.workunit.client.0.smithi007.stdout: 1 590190 8.01 MB/sec execute 282 sec latency 50.471 ms 2024-06-10T01:14:01.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:01 smithi007 ceph-mon[32398]: pgmap v672: 129 pgs: 129 active+clean; 106 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 345 op/s 2024-06-10T01:14:01.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:01 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3250543792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:14:01.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:01 smithi078 ceph-mon[36683]: pgmap v672: 129 pgs: 129 active+clean; 106 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 345 op/s 2024-06-10T01:14:01.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:01 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3250543792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:14:02.231 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db / --timeout 90 --expiration 136.0 --await-for 45.0' 2024-06-10T01:14:02.231 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db / --timeout 90 --expiration 136.0 --await-for 45.0 2024-06-10T01:14:02.494 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:14:02.512 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:02 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:14:02.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:02 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:14:02.739 INFO:tasks.workunit.client.0.smithi007.stdout: 1 592380 8.02 MB/sec execute 283 sec latency 125.489 ms 2024-06-10T01:14:02.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:02 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:14:03.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:03 smithi175 ceph-mon[36636]: pgmap v673: 129 pgs: 129 active+clean; 102 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 294 op/s 2024-06-10T01:14:03.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:03 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:14:03.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:03 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:14:03.739 INFO:tasks.workunit.client.0.smithi007.stdout: 1 594067 8.02 MB/sec execute 284 sec latency 17.887 ms 2024-06-10T01:14:03.753 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:03 smithi007 ceph-mon[32398]: pgmap v673: 129 pgs: 129 active+clean; 102 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 294 op/s 2024-06-10T01:14:03.754 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:03 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:14:03.754 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:03 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:14:03.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:03 smithi078 ceph-mon[36683]: pgmap v673: 129 pgs: 129 active+clean; 102 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 294 op/s 2024-06-10T01:14:03.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:03 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:14:03.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:03 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:14:04.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.292+0000 7f5816e3d700 1 -- 172.21.15.7:0/4176090099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5810101fa0 msgr2=0x7f5810105fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:04.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.292+0000 7f5816e3d700 1 --2- 172.21.15.7:0/4176090099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5810101fa0 0x7f5810105fe0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f57fc0099c0 tx=0x7f57fc009cd0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.292+0000 7f5816e3d700 1 -- 172.21.15.7:0/4176090099 shutdown_connections 2024-06-10T01:14:04.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.292+0000 7f5816e3d700 1 --2- 172.21.15.7:0/4176090099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58101066c0 0x7f581010e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.292+0000 7f5816e3d700 1 --2- 172.21.15.7:0/4176090099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58101015f0 0x7f58101019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.292+0000 7f5816e3d700 1 --2- 172.21.15.7:0/4176090099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5810101fa0 0x7f5810105fe0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.292+0000 7f5816e3d700 1 -- 172.21.15.7:0/4176090099 >> 172.21.15.7:0/4176090099 conn(0x7f5810078330 msgr2=0x7f5810078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:04.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.292+0000 7f5816e3d700 1 -- 172.21.15.7:0/4176090099 shutdown_connections 2024-06-10T01:14:04.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.292+0000 7f5816e3d700 1 -- 172.21.15.7:0/4176090099 wait complete. 2024-06-10T01:14:04.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.293+0000 7f5816e3d700 1 Processor -- start 2024-06-10T01:14:04.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.293+0000 7f5816e3d700 1 -- start start 2024-06-10T01:14:04.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.293+0000 7f5816e3d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58101015f0 0x7f581019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:04.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.293+0000 7f5816e3d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5810101fa0 0x7f581019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:04.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.293+0000 7f5816e3d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58101066c0 0x7f5810198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:04.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.293+0000 7f5816e3d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f58101116a0 con 0x7f58101066c0 2024-06-10T01:14:04.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.293+0000 7f5816e3d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5810111520 con 0x7f58101015f0 2024-06-10T01:14:04.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.293+0000 7f5816e3d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5810111820 con 0x7f5810101fa0 2024-06-10T01:14:04.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.293+0000 7f58153da700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58101066c0 0x7f5810198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:04.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.294+0000 7f58153da700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58101066c0 0x7f5810198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43272/0 (socket says 172.21.15.7:43272) 2024-06-10T01:14:04.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.294+0000 7f58153da700 1 -- 172.21.15.7:0/2075337580 learned_addr learned my addr 172.21.15.7:0/2075337580 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:04.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.294+0000 7f580ffff700 1 --2- 172.21.15.7:0/2075337580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5810101fa0 0x7f581019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:04.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.294+0000 7f5814bd9700 1 --2- 172.21.15.7:0/2075337580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58101015f0 0x7f581019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:04.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.294+0000 7f58153da700 1 -- 172.21.15.7:0/2075337580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5810101fa0 msgr2=0x7f581019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:04.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.294+0000 7f58153da700 1 --2- 172.21.15.7:0/2075337580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5810101fa0 0x7f581019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.294+0000 7f58153da700 1 -- 172.21.15.7:0/2075337580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58101015f0 msgr2=0x7f581019e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:04.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.294+0000 7f58153da700 1 --2- 172.21.15.7:0/2075337580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58101015f0 0x7f581019e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.294+0000 7f58153da700 1 -- 172.21.15.7:0/2075337580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5810199180 con 0x7f58101066c0 2024-06-10T01:14:04.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.294+0000 7f5814bd9700 1 --2- 172.21.15.7:0/2075337580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58101015f0 0x7f581019e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:04.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.294+0000 7f58153da700 1 --2- 172.21.15.7:0/2075337580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58101066c0 0x7f5810198a20 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f580800e940 tx=0x7f580800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:04.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.294+0000 7f580dffb700 1 -- 172.21.15.7:0/2075337580 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5808004a30 con 0x7f58101066c0 2024-06-10T01:14:04.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.294+0000 7f580dffb700 1 -- 172.21.15.7:0/2075337580 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5808005500 con 0x7f58101066c0 2024-06-10T01:14:04.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.294+0000 7f5816e3d700 1 -- 172.21.15.7:0/2075337580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5810199470 con 0x7f58101066c0 2024-06-10T01:14:04.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.294+0000 7f580dffb700 1 -- 172.21.15.7:0/2075337580 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580800ce20 con 0x7f58101066c0 2024-06-10T01:14:04.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.294+0000 7f5816e3d700 1 -- 172.21.15.7:0/2075337580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58101a56e0 con 0x7f58101066c0 2024-06-10T01:14:04.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.297+0000 7f580dffb700 1 -- 172.21.15.7:0/2075337580 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f5808005020 con 0x7f58101066c0 2024-06-10T01:14:04.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.297+0000 7f580dffb700 1 --2- 172.21.15.7:0/2075337580 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f58000a2cc0 0x7f58000a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.298+0000 7f580dffb700 1 -- 172.21.15.7:0/2075337580 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f58080cf4a0 con 0x7f58101066c0 2024-06-10T01:14:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.298+0000 7f5814bd9700 1 --2- 172.21.15.7:0/2075337580 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f58000a2cc0 0x7f58000a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.298+0000 7f5814bd9700 1 --2- 172.21.15.7:0/2075337580 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f58000a2cc0 0x7f58000a5180 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5810078b70 tx=0x7f580400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.298+0000 7f58216c0b80 1 Processor -- start 2024-06-10T01:14:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.299+0000 7f58216c0b80 1 -- start start 2024-06-10T01:14:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.299+0000 7f58216c0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d446dc39a0 0x55d446dc3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.299+0000 7f58216c0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d446dc42c0 0x55d446dbdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.299+0000 7f58216c0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d446dbe0f0 0x55d446dbe510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.299+0000 7f58216c0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d446cac0f0 con 0x55d446dbe0f0 2024-06-10T01:14:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.299+0000 7f58216c0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d446d0a9e0 con 0x55d446dc39a0 2024-06-10T01:14:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.299+0000 7f58216c0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d446c725f0 con 0x55d446dc42c0 2024-06-10T01:14:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.299+0000 7f58153da700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d446dbe0f0 0x55d446dbe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.299+0000 7f58153da700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d446dbe0f0 0x55d446dbe510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43286/0 (socket says 172.21.15.7:43286) 2024-06-10T01:14:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.299+0000 7f58153da700 1 -- 172.21.15.7:0/3877599437 learned_addr learned my addr 172.21.15.7:0/3877599437 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.299+0000 7f5814bd9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d446dc39a0 0x55d446dc3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.299+0000 7f580ffff700 1 --2- 172.21.15.7:0/3877599437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d446dc42c0 0x55d446dbdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.299+0000 7f58153da700 1 -- 172.21.15.7:0/3877599437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d446dc42c0 msgr2=0x55d446dbdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.299+0000 7f58153da700 1 --2- 172.21.15.7:0/3877599437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d446dc42c0 0x55d446dbdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.299+0000 7f58153da700 1 -- 172.21.15.7:0/3877599437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d446dc39a0 msgr2=0x55d446dc3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.299+0000 7f58153da700 1 --2- 172.21.15.7:0/3877599437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d446dc39a0 0x55d446dc3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.299+0000 7f58153da700 1 -- 172.21.15.7:0/3877599437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d446c59a60 con 0x55d446dbe0f0 2024-06-10T01:14:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.299+0000 7f58153da700 1 --2- 172.21.15.7:0/3877599437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d446dbe0f0 0x55d446dbe510 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f5808098350 tx=0x7f5808004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da5599efea87bfd2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.299+0000 7f57faffd700 1 -- 172.21.15.7:0/3877599437 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58080b4070 con 0x55d446dbe0f0 2024-06-10T01:14:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.300+0000 7f57faffd700 1 -- 172.21.15.7:0/3877599437 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58080cf050 con 0x55d446dbe0f0 2024-06-10T01:14:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.300+0000 7f57faffd700 1 -- 172.21.15.7:0/3877599437 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58080cf200 con 0x55d446dbe0f0 2024-06-10T01:14:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.300+0000 7f58216c0b80 1 -- 172.21.15.7:0/3877599437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d446dbe0f0 msgr2=0x55d446dbe510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.300+0000 7f58216c0b80 1 --2- 172.21.15.7:0/3877599437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d446dbe0f0 0x55d446dbe510 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f5808098350 tx=0x7f5808004f60 comp rx=0 tx=0).stop 2024-06-10T01:14:04.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.300+0000 7f58216c0b80 1 -- 172.21.15.7:0/3877599437 shutdown_connections 2024-06-10T01:14:04.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.300+0000 7f58216c0b80 1 --2- 172.21.15.7:0/3877599437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d446dbe0f0 0x55d446dbe510 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.300+0000 7f58216c0b80 1 --2- 172.21.15.7:0/3877599437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d446dc39a0 0x55d446dc3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.300+0000 7f58216c0b80 1 --2- 172.21.15.7:0/3877599437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d446dc42c0 0x55d446dbdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.300+0000 7f58216c0b80 1 -- 172.21.15.7:0/3877599437 >> 172.21.15.7:0/3877599437 conn(0x55d446ce6210 msgr2=0x55d446db4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:04.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.300+0000 7f58216c0b80 1 -- 172.21.15.7:0/3877599437 shutdown_connections 2024-06-10T01:14:04.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.301+0000 7f58216c0b80 1 -- 172.21.15.7:0/3877599437 wait complete. 2024-06-10T01:14:04.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.301+0000 7f58216c0b80 1 -- start start 2024-06-10T01:14:04.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.301+0000 7f58216c0b80 1 Processor -- start 2024-06-10T01:14:04.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.301+0000 7f58216c0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d446dbe0f0 0x55d446dca080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:04.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.301+0000 7f58216c0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d446dc39a0 0x55d446dc7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:04.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.301+0000 7f58216c0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d446dc42c0 0x55d446dc78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:04.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.301+0000 7f58216c0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d446cab150 con 0x55d446dc39a0 2024-06-10T01:14:04.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.302+0000 7f58216c0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d446d069a0 con 0x55d446dc42c0 2024-06-10T01:14:04.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.302+0000 7f58216c0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d446dcaca0 con 0x55d446dbe0f0 2024-06-10T01:14:04.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.302+0000 7f58216c0b80 10 client.0 _ll_register_callbacks cb 0x55d446ca0740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:14:04.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.302+0000 7f580ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d446dc39a0 0x55d446dc7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:04.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.302+0000 7f580ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d446dc39a0 0x55d446dc7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43288/0 (socket says 172.21.15.7:43288) 2024-06-10T01:14:04.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.302+0000 7f580ffff700 1 -- 172.21.15.7:0/1821021662 learned_addr learned my addr 172.21.15.7:0/1821021662 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:04.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.302+0000 7f5814bd9700 1 --2- 172.21.15.7:0/1821021662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d446dbe0f0 0x55d446dca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:04.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.302+0000 7f58153da700 1 --2- 172.21.15.7:0/1821021662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d446dc42c0 0x55d446dc78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:04.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.302+0000 7f580ffff700 1 -- 172.21.15.7:0/1821021662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d446dbe0f0 msgr2=0x55d446dca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:04.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.302+0000 7f580ffff700 1 --2- 172.21.15.7:0/1821021662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d446dbe0f0 0x55d446dca080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.302+0000 7f580ffff700 1 -- 172.21.15.7:0/1821021662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d446dc42c0 msgr2=0x55d446dc78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:04.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.302+0000 7f580ffff700 1 --2- 172.21.15.7:0/1821021662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d446dc42c0 0x55d446dc78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.302+0000 7f580ffff700 1 -- 172.21.15.7:0/1821021662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d446cd08e0 con 0x55d446dc39a0 2024-06-10T01:14:04.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.302+0000 7f580ffff700 1 --2- 172.21.15.7:0/1821021662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d446dc39a0 0x55d446dc7380 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f57fc0099c0 tx=0x7f57fc00bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72b95abc0341764 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:04.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.302+0000 7f57effff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:14:04.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.302+0000 7f57effff700 1 -- 172.21.15.7:0/1821021662 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc003a60 con 0x55d446dc39a0 2024-06-10T01:14:04.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.302+0000 7f58216c0b80 1 -- 172.21.15.7:0/1821021662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58080cf050 con 0x55d446dc39a0 2024-06-10T01:14:04.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.302+0000 7f57effff700 1 -- 172.21.15.7:0/1821021662 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f57fc003c00 con 0x55d446dc39a0 2024-06-10T01:14:04.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.302+0000 7f57effff700 1 -- 172.21.15.7:0/1821021662 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f57fc031050 con 0x55d446dc39a0 2024-06-10T01:14:04.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.302+0000 7f58216c0b80 10 client.17499 fetch_fsmap learned FSMap version 35 2024-06-10T01:14:04.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.302+0000 7f58216c0b80 1 -- 172.21.15.7:0/1821021662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d446dd54a0 con 0x55d446dc39a0 2024-06-10T01:14:04.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.303+0000 7f57effff700 1 -- 172.21.15.7:0/1821021662 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58080cf050 con 0x55d446dc39a0 2024-06-10T01:14:04.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.303+0000 7f57effff700 1 -- 172.21.15.7:0/1821021662 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f57fc00f580 con 0x55d446dc39a0 2024-06-10T01:14:04.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.303+0000 7f58216c0b80 10 client.17499 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:14:04.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.303+0000 7f58216c0b80 10 client.17499 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:14:04.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.303+0000 7f58216c0b80 20 client.17499 populate_metadata read hostname 'smithi007' 2024-06-10T01:14:04.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.303+0000 7f58216c0b80 1 --2- 172.21.15.7:0/1821021662 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d446dd7ad0 0x55d446dd9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:04.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.303+0000 7f58216c0b80 4 client.17499 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:14:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.303+0000 7f58216c0b80 1 -- 172.21.15.7:0/1821021662 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d446d17ee0 con 0x55d446dd7ad0 2024-06-10T01:14:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.303+0000 7f5814bd9700 1 --2- 172.21.15.7:0/1821021662 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d446dd7ad0 0x55d446dd9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.304+0000 7f5814bd9700 1 --2- 172.21.15.7:0/1821021662 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d446dd7ad0 0x55d446dd9ef0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fee037326912a3ac server_cookie=91bbc0dfb3fa8022 in_seq=0 out_seq=0 2024-06-10T01:14:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.304+0000 7f57effff700 10 client.17499 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.304+0000 7f57effff700 1 -- 172.21.15.7:0/1821021662 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d446d17ee0 con 0x55d446dd7ad0 2024-06-10T01:14:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.304+0000 7f57effff700 10 client.17499 handle_command_reply: tid=0 2024-06-10T01:14:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.304+0000 7f58216c0b80 1 client.17499 shutdown 2024-06-10T01:14:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.304+0000 7f58216c0b80 1 -- 172.21.15.7:0/1821021662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d446dc39a0 msgr2=0x55d446dc7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.304+0000 7f58216c0b80 1 --2- 172.21.15.7:0/1821021662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d446dc39a0 0x55d446dc7380 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f57fc0099c0 tx=0x7f57fc00bee0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.304+0000 7f58216c0b80 1 -- 172.21.15.7:0/1821021662 shutdown_connections 2024-06-10T01:14:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.305+0000 7f58216c0b80 1 --2- 172.21.15.7:0/1821021662 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d446dd7ad0 0x55d446dd9ef0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.305+0000 7f58216c0b80 1 --2- 172.21.15.7:0/1821021662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d446dc42c0 0x55d446dc78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.305+0000 7f58216c0b80 1 --2- 172.21.15.7:0/1821021662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d446dbe0f0 0x55d446dca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.305+0000 7f57effff700 0 client.17499 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:04.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.305+0000 7f58216c0b80 1 --2- 172.21.15.7:0/1821021662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d446dc39a0 0x55d446dc7380 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.305+0000 7f58216c0b80 1 -- 172.21.15.7:0/1821021662 >> 172.21.15.7:0/1821021662 conn(0x55d446ce6210 msgr2=0x55d446db3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:04.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.305+0000 7f58216c0b80 1 -- 172.21.15.7:0/1821021662 shutdown_connections 2024-06-10T01:14:04.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.305+0000 7f58216c0b80 1 -- 172.21.15.7:0/1821021662 wait complete. 2024-06-10T01:14:04.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.305+0000 7f58216c0b80 20 client.17499 trim_cache size 0 max 16384 2024-06-10T01:14:04.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.326+0000 7f58216c0b80 1 Processor -- start 2024-06-10T01:14:04.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.326+0000 7f58216c0b80 1 -- start start 2024-06-10T01:14:04.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.326+0000 7f58216c0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d446dbe0f0 0x55d446dc9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:04.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.326+0000 7f58216c0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d446dc39a0 0x55d446db43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:04.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.326+0000 7f58216c0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d446dc42c0 0x55d446db4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:04.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.326+0000 7f58216c0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f58101ae8a0 con 0x55d446dbe0f0 2024-06-10T01:14:04.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.326+0000 7f58216c0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f58101ac0d0 con 0x55d446dc42c0 2024-06-10T01:14:04.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.326+0000 7f58216c0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f58101a9f20 con 0x55d446dc39a0 2024-06-10T01:14:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.326+0000 7f5814bd9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d446dbe0f0 0x55d446dc9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.326+0000 7f5814bd9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d446dbe0f0 0x55d446dc9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43300/0 (socket says 172.21.15.7:43300) 2024-06-10T01:14:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.326+0000 7f5814bd9700 1 -- 172.21.15.7:0/2289051145 learned_addr learned my addr 172.21.15.7:0/2289051145 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.326+0000 7f580ffff700 1 --2- 172.21.15.7:0/2289051145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d446dc39a0 0x55d446db43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.326+0000 7f58153da700 1 --2- 172.21.15.7:0/2289051145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d446dc42c0 0x55d446db4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.326+0000 7f5814bd9700 1 -- 172.21.15.7:0/2289051145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d446dc39a0 msgr2=0x55d446db43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.326+0000 7f5814bd9700 1 --2- 172.21.15.7:0/2289051145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d446dc39a0 0x55d446db43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.326+0000 7f5814bd9700 1 -- 172.21.15.7:0/2289051145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d446dc42c0 msgr2=0x55d446db4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.326+0000 7f5814bd9700 1 --2- 172.21.15.7:0/2289051145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d446dc42c0 0x55d446db4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.326+0000 7f5814bd9700 1 -- 172.21.15.7:0/2289051145 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d446dc58f0 con 0x55d446dbe0f0 2024-06-10T01:14:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.326+0000 7f58153da700 1 --2- 172.21.15.7:0/2289051145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d446dc42c0 0x55d446db4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:14:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.326+0000 7f580ffff700 1 --2- 172.21.15.7:0/2289051145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d446dc39a0 0x55d446db43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:14:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.327+0000 7f5814bd9700 1 --2- 172.21.15.7:0/2289051145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d446dbe0f0 0x55d446dc9bc0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f580400b010 tx=0x7f5804014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14f85f3f28c5e31 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.327+0000 7f57fa7fc700 1 -- 172.21.15.7:0/2289051145 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804023450 con 0x55d446dbe0f0 2024-06-10T01:14:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.327+0000 7f57fa7fc700 1 -- 172.21.15.7:0/2289051145 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5804030070 con 0x55d446dbe0f0 2024-06-10T01:14:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.327+0000 7f57fa7fc700 1 -- 172.21.15.7:0/2289051145 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804017a10 con 0x55d446dbe0f0 2024-06-10T01:14:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.328+0000 7f58216c0b80 1 -- 172.21.15.7:0/2289051145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d446dbe0f0 msgr2=0x55d446dc9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.328+0000 7f58216c0b80 1 --2- 172.21.15.7:0/2289051145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d446dbe0f0 0x55d446dc9bc0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f580400b010 tx=0x7f5804014000 comp rx=0 tx=0).stop 2024-06-10T01:14:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.328+0000 7f58216c0b80 1 -- 172.21.15.7:0/2289051145 shutdown_connections 2024-06-10T01:14:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.328+0000 7f58216c0b80 1 --2- 172.21.15.7:0/2289051145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d446dc42c0 0x55d446db4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.328+0000 7f58216c0b80 1 --2- 172.21.15.7:0/2289051145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d446dbe0f0 0x55d446dc9bc0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.328+0000 7f58216c0b80 1 --2- 172.21.15.7:0/2289051145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d446dc39a0 0x55d446db43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.328+0000 7f58216c0b80 1 -- 172.21.15.7:0/2289051145 >> 172.21.15.7:0/2289051145 conn(0x55d446ce6210 msgr2=0x55d446dc2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.328+0000 7f58216c0b80 1 -- 172.21.15.7:0/2289051145 shutdown_connections 2024-06-10T01:14:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.328+0000 7f58216c0b80 1 -- 172.21.15.7:0/2289051145 wait complete. 2024-06-10T01:14:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.328+0000 7f58216c0b80 1 -- start start 2024-06-10T01:14:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.328+0000 7f58216c0b80 1 Processor -- start 2024-06-10T01:14:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.329+0000 7f58216c0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d446dbe0f0 0x55d446dd0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.329+0000 7f58216c0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d446dc39a0 0x55d446dccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.329+0000 7f58216c0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d446dc42c0 0x55d446dcd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.329+0000 7f58216c0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f58101aa590 con 0x55d446dbe0f0 2024-06-10T01:14:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.329+0000 7f58216c0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f58101aba00 con 0x55d446dc39a0 2024-06-10T01:14:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.329+0000 7f58216c0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f58101aac60 con 0x55d446dc42c0 2024-06-10T01:14:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.329+0000 7f58216c0b80 10 client.0 _ll_register_callbacks cb 0x55d446ca0740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:14:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.329+0000 7f5814bd9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d446dbe0f0 0x55d446dd0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.329+0000 7f5814bd9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d446dbe0f0 0x55d446dd0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43316/0 (socket says 172.21.15.7:43316) 2024-06-10T01:14:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.329+0000 7f5814bd9700 1 -- 172.21.15.7:0/3493904746 learned_addr learned my addr 172.21.15.7:0/3493904746 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.329+0000 7f58153da700 1 --2- 172.21.15.7:0/3493904746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d446dc42c0 0x55d446dcd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.329+0000 7f580ffff700 1 --2- 172.21.15.7:0/3493904746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d446dc39a0 0x55d446dccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.329+0000 7f5814bd9700 1 -- 172.21.15.7:0/3493904746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d446dc42c0 msgr2=0x55d446dcd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.329+0000 7f5814bd9700 1 --2- 172.21.15.7:0/3493904746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d446dc42c0 0x55d446dcd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.329+0000 7f5814bd9700 1 -- 172.21.15.7:0/3493904746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d446dc39a0 msgr2=0x55d446dccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.329+0000 7f5814bd9700 1 --2- 172.21.15.7:0/3493904746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d446dc39a0 0x55d446dccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.329+0000 7f5814bd9700 1 -- 172.21.15.7:0/3493904746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d446dcdcc0 con 0x55d446dbe0f0 2024-06-10T01:14:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.329+0000 7f58153da700 1 --2- 172.21.15.7:0/3493904746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d446dc42c0 0x55d446dcd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:14:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.329+0000 7f580ffff700 1 --2- 172.21.15.7:0/3493904746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d446dc39a0 0x55d446dccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:14:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.329+0000 7f5814bd9700 1 --2- 172.21.15.7:0/3493904746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d446dbe0f0 0x55d446dd0700 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f5804017630 tx=0x7f580400e5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e67366efac7ac4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.329+0000 7f57effff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:14:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.329+0000 7f57effff700 1 -- 172.21.15.7:0/3493904746 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804017ad0 con 0x55d446dbe0f0 2024-06-10T01:14:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.329+0000 7f57effff700 1 -- 172.21.15.7:0/3493904746 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f5804017c70 con 0x55d446dbe0f0 2024-06-10T01:14:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.329+0000 7f58216c0b80 1 -- 172.21.15.7:0/3493904746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5804030070 con 0x55d446dbe0f0 2024-06-10T01:14:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.330+0000 7f57effff700 1 -- 172.21.15.7:0/3493904746 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5804044050 con 0x55d446dbe0f0 2024-06-10T01:14:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.330+0000 7f58216c0b80 10 client.17505 fetch_fsmap learned FSMap version 35 2024-06-10T01:14:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.330+0000 7f58216c0b80 1 -- 172.21.15.7:0/3493904746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d446e43780 con 0x55d446dbe0f0 2024-06-10T01:14:04.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.330+0000 7f57effff700 1 -- 172.21.15.7:0/3493904746 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5804030070 con 0x55d446dbe0f0 2024-06-10T01:14:04.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.330+0000 7f57effff700 1 -- 172.21.15.7:0/3493904746 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f5804020040 con 0x55d446dbe0f0 2024-06-10T01:14:04.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.330+0000 7f58216c0b80 10 client.17505 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:14:04.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.330+0000 7f58216c0b80 10 client.17505 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:14:04.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.330+0000 7f58216c0b80 20 client.17505 populate_metadata read hostname 'smithi007' 2024-06-10T01:14:04.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.330+0000 7f58216c0b80 1 --2- 172.21.15.7:0/3493904746 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d446e43e90 0x55d446e462b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:04.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.330+0000 7f58216c0b80 4 client.17505 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.0, "await_for": 45.0}] 2024-06-10T01:14:04.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.330+0000 7f58216c0b80 1 -- 172.21.15.7:0/3493904746 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.0, "await_for": 45.0}) -- 0x55d446d12460 con 0x55d446e43e90 2024-06-10T01:14:04.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.330+0000 7f580ffff700 1 --2- 172.21.15.7:0/3493904746 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d446e43e90 0x55d446e462b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:04.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.331+0000 7f580ffff700 1 --2- 172.21.15.7:0/3493904746 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d446e43e90 0x55d446e462b0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d92b03cac96368ee server_cookie=3d51918bf70f78aa in_seq=0 out_seq=0 2024-06-10T01:14:04.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.331+0000 7f57effff700 10 client.17505 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:04.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.409+0000 7f57effff700 1 -- 172.21.15.7:0/3493904746 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d446d12460 con 0x55d446e43e90 2024-06-10T01:14:04.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.409+0000 7f57effff700 10 client.17505 handle_command_reply: tid=0 2024-06-10T01:14:04.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.409+0000 7f58216c0b80 1 client.17505 shutdown 2024-06-10T01:14:04.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.409+0000 7f58216c0b80 1 -- 172.21.15.7:0/3493904746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d446dbe0f0 msgr2=0x55d446dd0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:04.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.409+0000 7f58216c0b80 1 --2- 172.21.15.7:0/3493904746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d446dbe0f0 0x55d446dd0700 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f5804017630 tx=0x7f580400e5a0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.409+0000 7f58216c0b80 1 -- 172.21.15.7:0/3493904746 shutdown_connections 2024-06-10T01:14:04.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.409+0000 7f58216c0b80 1 --2- 172.21.15.7:0/3493904746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d446dc42c0 0x55d446dcd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.409+0000 7f58216c0b80 1 --2- 172.21.15.7:0/3493904746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d446dbe0f0 0x55d446dd0700 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.409+0000 7f58216c0b80 1 --2- 172.21.15.7:0/3493904746 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d446e43e90 0x55d446e462b0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.409+0000 7f58216c0b80 1 --2- 172.21.15.7:0/3493904746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d446dc39a0 0x55d446dccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.409+0000 7f57effff700 0 client.17505 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:04.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.409+0000 7f58216c0b80 1 -- 172.21.15.7:0/3493904746 >> 172.21.15.7:0/3493904746 conn(0x55d446ce6210 msgr2=0x55d446dc6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:04.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.409+0000 7f58216c0b80 1 -- 172.21.15.7:0/3493904746 shutdown_connections 2024-06-10T01:14:04.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.409+0000 7f58216c0b80 1 -- 172.21.15.7:0/3493904746 wait complete. 2024-06-10T01:14:04.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.410+0000 7f58216c0b80 20 client.17505 trim_cache size 0 max 16384 2024-06-10T01:14:04.496 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:14:04.496 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 35, 2024-06-10T01:14:04.496 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:14:04.496 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 22, 2024-06-10T01:14:04.496 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:14:04.496 INFO:teuthology.orchestra.run.smithi007.stdout: "6ddf71a3": { 2024-06-10T01:14:04.496 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 22, 2024-06-10T01:14:04.496 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:14:04.496 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:14:04.496 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:14:04.496 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:14:04.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:14:04.497 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:14:04.497 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 136.0, 2024-06-10T01:14:04.497 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:14:04.497 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:14:04.497 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:14:04.497 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:14:04.497 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:14:04.497 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:14:04.497 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:14:04.497 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:14:04.497 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:14:04.497 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:14:04.497 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:14:04.498 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:14:04.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.410+0000 7f5816e3d700 1 -- 172.21.15.7:0/2075337580 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f58000a2cc0 msgr2=0x7f58000a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:04.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.410+0000 7f5816e3d700 1 --2- 172.21.15.7:0/2075337580 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f58000a2cc0 0x7f58000a5180 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5810078b70 tx=0x7f580400b040 comp rx=0 tx=0).stop 2024-06-10T01:14:04.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.410+0000 7f5816e3d700 1 -- 172.21.15.7:0/2075337580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58101066c0 msgr2=0x7f5810198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:04.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.410+0000 7f5816e3d700 1 --2- 172.21.15.7:0/2075337580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58101066c0 0x7f5810198a20 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f580800e940 tx=0x7f580800ec50 comp rx=0 tx=0).stop 2024-06-10T01:14:04.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.410+0000 7f5816e3d700 1 -- 172.21.15.7:0/2075337580 shutdown_connections 2024-06-10T01:14:04.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.410+0000 7f5816e3d700 1 --2- 172.21.15.7:0/2075337580 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f58000a2cc0 0x7f58000a5180 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.410+0000 7f5816e3d700 1 --2- 172.21.15.7:0/2075337580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58101066c0 0x7f5810198a20 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.410+0000 7f5816e3d700 1 --2- 172.21.15.7:0/2075337580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58101015f0 0x7f581019e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.410+0000 7f5816e3d700 1 --2- 172.21.15.7:0/2075337580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5810101fa0 0x7f581019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:04.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.410+0000 7f5816e3d700 1 -- 172.21.15.7:0/2075337580 >> 172.21.15.7:0/2075337580 conn(0x7f5810078330 msgr2=0x7f581006a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:04.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.410+0000 7f5816e3d700 1 -- 172.21.15.7:0/2075337580 shutdown_connections 2024-06-10T01:14:04.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:04.410+0000 7f5816e3d700 1 -- 172.21.15.7:0/2075337580 wait complete. 2024-06-10T01:14:04.739 INFO:tasks.workunit.client.0.smithi007.stdout: 1 595615 8.02 MB/sec execute 285 sec latency 341.934 ms 2024-06-10T01:14:05.325 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6ddf71a3', quiesce took 6.8 seconds. Will release after: 9.2 2024-06-10T01:14:05.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:05 smithi175 ceph-mon[36636]: pgmap v674: 129 pgs: 129 active+clean; 103 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 321 op/s 2024-06-10T01:14:05.740 INFO:tasks.workunit.client.0.smithi007.stdout: 1 595615 7.99 MB/sec execute 286 sec latency 1341.995 ms 2024-06-10T01:14:05.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:05 smithi007 ceph-mon[32398]: pgmap v674: 129 pgs: 129 active+clean; 103 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 321 op/s 2024-06-10T01:14:05.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:05 smithi078 ceph-mon[36683]: pgmap v674: 129 pgs: 129 active+clean; 103 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 321 op/s 2024-06-10T01:14:06.740 INFO:tasks.workunit.client.0.smithi007.stdout: 1 595615 7.96 MB/sec execute 287 sec latency 2342.033 ms 2024-06-10T01:14:07.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:07 smithi175 ceph-mon[36636]: pgmap v675: 129 pgs: 129 active+clean; 105 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 340 op/s 2024-06-10T01:14:07.740 INFO:tasks.workunit.client.0.smithi007.stdout: 1 595615 7.93 MB/sec execute 288 sec latency 3342.150 ms 2024-06-10T01:14:07.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:07 smithi007 ceph-mon[32398]: pgmap v675: 129 pgs: 129 active+clean; 105 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 340 op/s 2024-06-10T01:14:07.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:07 smithi078 ceph-mon[36683]: pgmap v675: 129 pgs: 129 active+clean; 105 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 340 op/s 2024-06-10T01:14:08.740 INFO:tasks.workunit.client.0.smithi007.stdout: 1 595615 7.91 MB/sec execute 289 sec latency 4342.245 ms 2024-06-10T01:14:09.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:09 smithi175 ceph-mon[36636]: pgmap v676: 129 pgs: 129 active+clean; 106 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 336 op/s 2024-06-10T01:14:09.740 INFO:tasks.workunit.client.0.smithi007.stdout: 1 595615 7.88 MB/sec execute 290 sec latency 5342.367 ms 2024-06-10T01:14:09.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:09 smithi007 ceph-mon[32398]: pgmap v676: 129 pgs: 129 active+clean; 106 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 336 op/s 2024-06-10T01:14:09.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:09 smithi078 ceph-mon[36683]: pgmap v676: 129 pgs: 129 active+clean; 106 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 336 op/s 2024-06-10T01:14:10.740 INFO:tasks.workunit.client.0.smithi007.stdout: 1 595615 7.85 MB/sec execute 291 sec latency 6342.428 ms 2024-06-10T01:14:11.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:11 smithi175 ceph-mon[36636]: pgmap v677: 129 pgs: 129 active+clean; 107 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 2.7 MiB/s wr, 281 op/s 2024-06-10T01:14:11.740 INFO:tasks.workunit.client.0.smithi007.stdout: 1 595615 7.82 MB/sec execute 292 sec latency 7342.508 ms 2024-06-10T01:14:11.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:11 smithi007 ceph-mon[32398]: pgmap v677: 129 pgs: 129 active+clean; 107 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 2.7 MiB/s wr, 281 op/s 2024-06-10T01:14:11.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:11 smithi078 ceph-mon[36683]: pgmap v677: 129 pgs: 129 active+clean; 107 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 2.7 MiB/s wr, 281 op/s 2024-06-10T01:14:12.740 INFO:tasks.workunit.client.0.smithi007.stdout: 1 595615 7.80 MB/sec execute 293 sec latency 8342.615 ms 2024-06-10T01:14:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.115+0000 7f02aadb0640 1 -- 172.21.15.7:0/3433015686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02a4102c00 msgr2=0x7f02a4103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.115+0000 7f02aadb0640 1 --2- 172.21.15.7:0/3433015686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02a4102c00 0x7f02a4103080 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f028c009ca0 tx=0x7f028c02a8c0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.115+0000 7f02aadb0640 1 -- 172.21.15.7:0/3433015686 shutdown_connections 2024-06-10T01:14:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.115+0000 7f02aadb0640 1 --2- 172.21.15.7:0/3433015686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02a41035c0 0x7f02a4109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.115+0000 7f02aadb0640 1 --2- 172.21.15.7:0/3433015686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02a4102c00 0x7f02a4103080 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.115+0000 7f02aadb0640 1 --2- 172.21.15.7:0/3433015686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02a4101a10 0x7f02a4101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.115+0000 7f02aadb0640 1 -- 172.21.15.7:0/3433015686 >> 172.21.15.7:0/3433015686 conn(0x7f02a40fcf80 msgr2=0x7f02a40ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.115+0000 7f02aadb0640 1 -- 172.21.15.7:0/3433015686 shutdown_connections 2024-06-10T01:14:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.115+0000 7f02aadb0640 1 -- 172.21.15.7:0/3433015686 wait complete. 2024-06-10T01:14:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.116+0000 7f02aadb0640 1 Processor -- start 2024-06-10T01:14:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.116+0000 7f02aadb0640 1 -- start start 2024-06-10T01:14:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.116+0000 7f02aadb0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02a4101a10 0x7f02a4197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.116+0000 7f02aadb0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02a4102c00 0x7f02a4198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.116+0000 7f02aadb0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02a41035c0 0x7f02a419f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.116+0000 7f02aadb0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f02a410d690 con 0x7f02a4102c00 2024-06-10T01:14:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.116+0000 7f02aadb0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f02a410d510 con 0x7f02a41035c0 2024-06-10T01:14:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.116+0000 7f02aadb0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f02a410d810 con 0x7f02a4101a10 2024-06-10T01:14:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.116+0000 7f02a3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02a4102c00 0x7f02a4198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.116+0000 7f02a3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02a4102c00 0x7f02a4198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46282/0 (socket says 172.21.15.7:46282) 2024-06-10T01:14:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.116+0000 7f02a3fff640 1 -- 172.21.15.7:0/1021752196 learned_addr learned my addr 172.21.15.7:0/1021752196 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.116+0000 7f02a934d640 1 --2- 172.21.15.7:0/1021752196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02a41035c0 0x7f02a419f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.116+0000 7f02a8b4c640 1 --2- 172.21.15.7:0/1021752196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02a4101a10 0x7f02a4197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.116+0000 7f02a3fff640 1 -- 172.21.15.7:0/1021752196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02a4101a10 msgr2=0x7f02a4197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.116+0000 7f02a3fff640 1 --2- 172.21.15.7:0/1021752196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02a4101a10 0x7f02a4197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.116+0000 7f02a3fff640 1 -- 172.21.15.7:0/1021752196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02a41035c0 msgr2=0x7f02a419f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.116+0000 7f02a3fff640 1 --2- 172.21.15.7:0/1021752196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02a41035c0 0x7f02a419f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.116+0000 7f02a3fff640 1 -- 172.21.15.7:0/1021752196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a419f930 con 0x7f02a4102c00 2024-06-10T01:14:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.116+0000 7f02a934d640 1 --2- 172.21.15.7:0/1021752196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02a41035c0 0x7f02a419f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.117+0000 7f02a3fff640 1 --2- 172.21.15.7:0/1021752196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02a4102c00 0x7f02a4198260 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f028c031a40 tx=0x7f028c031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.117+0000 7f02a1ffb640 1 -- 172.21.15.7:0/1021752196 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c02fa20 con 0x7f02a4102c00 2024-06-10T01:14:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.117+0000 7f02a1ffb640 1 -- 172.21.15.7:0/1021752196 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f028c02fbc0 con 0x7f02a4102c00 2024-06-10T01:14:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.117+0000 7f02aadb0640 1 -- 172.21.15.7:0/1021752196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02a419fbc0 con 0x7f02a4102c00 2024-06-10T01:14:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.117+0000 7f02a1ffb640 1 -- 172.21.15.7:0/1021752196 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c03a780 con 0x7f02a4102c00 2024-06-10T01:14:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.117+0000 7f02aadb0640 1 -- 172.21.15.7:0/1021752196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02a41a0110 con 0x7f02a4102c00 2024-06-10T01:14:13.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.120+0000 7f02a1ffb640 1 -- 172.21.15.7:0/1021752196 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f028c03a920 con 0x7f02a4102c00 2024-06-10T01:14:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.120+0000 7f02adb9f740 1 Processor -- start 2024-06-10T01:14:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.120+0000 7f02adb9f740 1 -- start start 2024-06-10T01:14:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.120+0000 7f02adb9f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56086da32010 0x56086da32430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.120+0000 7f02adb9f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56086da32a00 0x56086da37500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.120+0000 7f02adb9f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56086da37b70 0x56086da3a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.120+0000 7f02adb9f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56086d9954f0 con 0x56086da37b70 2024-06-10T01:14:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.120+0000 7f02adb9f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56086d933870 con 0x56086da32a00 2024-06-10T01:14:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.120+0000 7f02adb9f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56086d9914b0 con 0x56086da32010 2024-06-10T01:14:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.120+0000 7f02a3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56086da37b70 0x56086da3a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.120+0000 7f02a1ffb640 1 --2- 172.21.15.7:0/1021752196 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f02900a2b80 0x7f02900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.120+0000 7f02a3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56086da37b70 0x56086da3a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46286/0 (socket says 172.21.15.7:46286) 2024-06-10T01:14:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.120+0000 7f02a3fff640 1 -- 172.21.15.7:0/4195819415 learned_addr learned my addr 172.21.15.7:0/4195819415 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.121+0000 7f02a934d640 1 --2- 172.21.15.7:0/4195819415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56086da32010 0x56086da32430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.121+0000 7f02a8b4c640 1 --2- 172.21.15.7:0/4195819415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56086da32a00 0x56086da37500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.121+0000 7f02a934d640 1 --2- 172.21.15.7:0/1021752196 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f02900a2b80 0x7f02900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.121+0000 7f02a1ffb640 1 -- 172.21.15.7:0/1021752196 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f028c0f0a30 con 0x7f02a4102c00 2024-06-10T01:14:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.121+0000 7f02a8b4c640 1 -- 172.21.15.7:0/4195819415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56086da32010 msgr2=0x56086da32430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.121+0000 7f02a8b4c640 1 --2- 172.21.15.7:0/4195819415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56086da32010 0x56086da32430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.121+0000 7f02a8b4c640 1 -- 172.21.15.7:0/4195819415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56086da37b70 msgr2=0x56086da3a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.121+0000 7f02a8b4c640 1 --2- 172.21.15.7:0/4195819415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56086da37b70 0x56086da3a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.121+0000 7f02a8b4c640 1 -- 172.21.15.7:0/4195819415 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56086d94f9b0 con 0x56086da32a00 2024-06-10T01:14:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.121+0000 7f02a3fff640 1 --2- 172.21.15.7:0/4195819415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56086da37b70 0x56086da3a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.121+0000 7f02a934d640 1 --2- 172.21.15.7:0/1021752196 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f02900a2b80 0x7f02900a5040 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f029c009710 tx=0x7f029c009b30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.121+0000 7f02a8b4c640 1 --2- 172.21.15.7:0/4195819415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56086da32a00 0x56086da37500 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f02a4102a60 tx=0x7f029400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae99d52fb091ae84 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.121+0000 7f029affd640 1 -- 172.21.15.7:0/4195819415 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0294015070 con 0x56086da32a00 2024-06-10T01:14:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.121+0000 7f029affd640 1 -- 172.21.15.7:0/4195819415 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02940035f0 con 0x56086da32a00 2024-06-10T01:14:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.121+0000 7f029affd640 1 -- 172.21.15.7:0/4195819415 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02940116a0 con 0x56086da32a00 2024-06-10T01:14:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.122+0000 7f02adb9f740 1 -- 172.21.15.7:0/4195819415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56086da32a00 msgr2=0x56086da37500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.122+0000 7f02adb9f740 1 --2- 172.21.15.7:0/4195819415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56086da32a00 0x56086da37500 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f02a4102a60 tx=0x7f029400f730 comp rx=0 tx=0).stop 2024-06-10T01:14:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.122+0000 7f02adb9f740 1 -- 172.21.15.7:0/4195819415 shutdown_connections 2024-06-10T01:14:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.122+0000 7f02adb9f740 1 --2- 172.21.15.7:0/4195819415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56086da37b70 0x56086da3a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.122+0000 7f02adb9f740 1 --2- 172.21.15.7:0/4195819415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56086da32a00 0x56086da37500 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.122+0000 7f02adb9f740 1 --2- 172.21.15.7:0/4195819415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56086da32010 0x56086da32430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.122+0000 7f02adb9f740 1 -- 172.21.15.7:0/4195819415 >> 172.21.15.7:0/4195819415 conn(0x56086da2d680 msgr2=0x56086da2da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.122+0000 7f02adb9f740 1 -- 172.21.15.7:0/4195819415 shutdown_connections 2024-06-10T01:14:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.122+0000 7f02adb9f740 1 -- 172.21.15.7:0/4195819415 wait complete. 2024-06-10T01:14:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.122+0000 7f02adb9f740 1 -- start start 2024-06-10T01:14:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.122+0000 7f02adb9f740 1 Processor -- start 2024-06-10T01:14:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.123+0000 7f02adb9f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56086da32010 0x56086da3c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.123+0000 7f02adb9f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56086da32a00 0x56086da3c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.123+0000 7f02adb9f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56086da37b70 0x56086da43830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.123+0000 7f02adb9f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56086da3ab20 con 0x56086da32010 2024-06-10T01:14:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.123+0000 7f02adb9f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56086d91eae0 con 0x56086da32a00 2024-06-10T01:14:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.123+0000 7f02adb9f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56086da3aca0 con 0x56086da37b70 2024-06-10T01:14:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.123+0000 7f02adb9f740 10 client.0 _ll_register_callbacks cb 0x56086d9579b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:14:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.123+0000 7f02a8b4c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56086da32010 0x56086da3c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.123+0000 7f02a8b4c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56086da32010 0x56086da3c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46300/0 (socket says 172.21.15.7:46300) 2024-06-10T01:14:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.123+0000 7f02a8b4c640 1 -- 172.21.15.7:0/3503817767 learned_addr learned my addr 172.21.15.7:0/3503817767 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.123+0000 7f02a3fff640 1 --2- 172.21.15.7:0/3503817767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56086da32a00 0x56086da3c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.123+0000 7f02a934d640 1 --2- 172.21.15.7:0/3503817767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56086da37b70 0x56086da43830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.123+0000 7f02a8b4c640 1 -- 172.21.15.7:0/3503817767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56086da37b70 msgr2=0x56086da43830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.123+0000 7f02a8b4c640 1 --2- 172.21.15.7:0/3503817767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56086da37b70 0x56086da43830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.123+0000 7f02a8b4c640 1 -- 172.21.15.7:0/3503817767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56086da32a00 msgr2=0x56086da3c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.123+0000 7f02a8b4c640 1 --2- 172.21.15.7:0/3503817767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56086da32a00 0x56086da3c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.123+0000 7f02a8b4c640 1 -- 172.21.15.7:0/3503817767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56086d6a9210 con 0x56086da32010 2024-06-10T01:14:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.123+0000 7f02a3fff640 1 --2- 172.21.15.7:0/3503817767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56086da32a00 0x56086da3c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.124+0000 7f02a8b4c640 1 --2- 172.21.15.7:0/3503817767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56086da32010 0x56086da3c2b0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f0294003e50 tx=0x7f029400beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb9a0ae28e326bb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.124+0000 7f0283fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:14:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.124+0000 7f0283fff640 1 -- 172.21.15.7:0/3503817767 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0294015070 con 0x56086da32010 2024-06-10T01:14:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.124+0000 7f0283fff640 1 -- 172.21.15.7:0/3503817767 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f0294003a30 con 0x56086da32010 2024-06-10T01:14:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.124+0000 7f02adb9f740 1 -- 172.21.15.7:0/3503817767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02940035f0 con 0x56086da32010 2024-06-10T01:14:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.124+0000 7f0283fff640 1 -- 172.21.15.7:0/3503817767 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0294003fe0 con 0x56086da32010 2024-06-10T01:14:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.124+0000 7f02adb9f740 10 client.17517 fetch_fsmap learned FSMap version 35 2024-06-10T01:14:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.124+0000 7f02adb9f740 1 -- 172.21.15.7:0/3503817767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56086d775620 con 0x56086da32010 2024-06-10T01:14:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.124+0000 7f0283fff640 1 -- 172.21.15.7:0/3503817767 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02940035f0 con 0x56086da32010 2024-06-10T01:14:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.124+0000 7f0283fff640 1 -- 172.21.15.7:0/3503817767 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f0294011b70 con 0x56086da32010 2024-06-10T01:14:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.124+0000 7f02adb9f740 10 client.17517 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:14:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.124+0000 7f02adb9f740 10 client.17517 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:14:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.124+0000 7f02adb9f740 20 client.17517 populate_metadata read hostname 'smithi007' 2024-06-10T01:14:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.124+0000 7f02adb9f740 1 --2- 172.21.15.7:0/3503817767 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56086da48070 0x56086da4a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.124+0000 7f02adb9f740 4 client.17517 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:14:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.124+0000 7f02adb9f740 1 -- 172.21.15.7:0/3503817767 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56086d9a8f10 con 0x56086da48070 2024-06-10T01:14:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.125+0000 7f02a3fff640 1 --2- 172.21.15.7:0/3503817767 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56086da48070 0x56086da4a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.125+0000 7f02a3fff640 1 --2- 172.21.15.7:0/3503817767 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56086da48070 0x56086da4a4c0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1e447d19cbfaa4f server_cookie=631fdb0c744e6509 in_seq=0 out_seq=0 2024-06-10T01:14:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.125+0000 7f0283fff640 10 client.17517 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.126+0000 7f0283fff640 1 -- 172.21.15.7:0/3503817767 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56086d9a8f10 con 0x56086da48070 2024-06-10T01:14:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.126+0000 7f0283fff640 10 client.17517 handle_command_reply: tid=0 2024-06-10T01:14:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.126+0000 7f02adb9f740 1 client.17517 shutdown 2024-06-10T01:14:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.126+0000 7f02adb9f740 1 -- 172.21.15.7:0/3503817767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56086da32010 msgr2=0x56086da3c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.126+0000 7f02adb9f740 1 --2- 172.21.15.7:0/3503817767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56086da32010 0x56086da3c2b0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f0294003e50 tx=0x7f029400beb0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.126+0000 7f02adb9f740 1 -- 172.21.15.7:0/3503817767 shutdown_connections 2024-06-10T01:14:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.126+0000 7f02adb9f740 1 --2- 172.21.15.7:0/3503817767 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56086da48070 0x56086da4a4c0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.126+0000 7f02adb9f740 1 --2- 172.21.15.7:0/3503817767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56086da37b70 0x56086da43830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.126+0000 7f02adb9f740 1 --2- 172.21.15.7:0/3503817767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56086da32a00 0x56086da3c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.126+0000 7f0283fff640 0 client.17517 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.126+0000 7f02adb9f740 1 --2- 172.21.15.7:0/3503817767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56086da32010 0x56086da3c2b0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.126+0000 7f02adb9f740 1 -- 172.21.15.7:0/3503817767 >> 172.21.15.7:0/3503817767 conn(0x56086da2d680 msgr2=0x56086da2c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.126+0000 7f02adb9f740 1 -- 172.21.15.7:0/3503817767 shutdown_connections 2024-06-10T01:14:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.126+0000 7f02adb9f740 1 -- 172.21.15.7:0/3503817767 wait complete. 2024-06-10T01:14:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.126+0000 7f02adb9f740 20 client.17517 trim_cache size 0 max 16384 2024-06-10T01:14:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.141+0000 7f02adb9f740 1 Processor -- start 2024-06-10T01:14:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.142+0000 7f02adb9f740 1 -- start start 2024-06-10T01:14:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.142+0000 7f02adb9f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56086da32010 0x56086da30650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.142+0000 7f02adb9f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56086da32a00 0x56086da437f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.142+0000 7f02adb9f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56086da37b70 0x56086da30b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.142+0000 7f02adb9f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f02a41acd50 con 0x56086da37b70 2024-06-10T01:14:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.142+0000 7f02adb9f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f02a41aa5c0 con 0x56086da32a00 2024-06-10T01:14:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.142+0000 7f02adb9f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f02a41a8410 con 0x56086da32010 2024-06-10T01:14:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.142+0000 7f02a934d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56086da37b70 0x56086da30b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.142+0000 7f02a934d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56086da37b70 0x56086da30b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46316/0 (socket says 172.21.15.7:46316) 2024-06-10T01:14:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.142+0000 7f02a934d640 1 -- 172.21.15.7:0/3606597849 learned_addr learned my addr 172.21.15.7:0/3606597849 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.142+0000 7f02a8b4c640 1 --2- 172.21.15.7:0/3606597849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56086da32010 0x56086da30650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.142+0000 7f02a3fff640 1 --2- 172.21.15.7:0/3606597849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56086da32a00 0x56086da437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.142+0000 7f02a934d640 1 -- 172.21.15.7:0/3606597849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56086da32010 msgr2=0x56086da30650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.142+0000 7f02a934d640 1 --2- 172.21.15.7:0/3606597849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56086da32010 0x56086da30650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.142+0000 7f02a934d640 1 -- 172.21.15.7:0/3606597849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56086da32a00 msgr2=0x56086da437f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.142+0000 7f02a934d640 1 --2- 172.21.15.7:0/3606597849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56086da32a00 0x56086da437f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.142+0000 7f02a934d640 1 -- 172.21.15.7:0/3606597849 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56086da28fe0 con 0x56086da37b70 2024-06-10T01:14:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.142+0000 7f02a3fff640 1 --2- 172.21.15.7:0/3606597849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56086da32a00 0x56086da437f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:14:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.142+0000 7f02a934d640 1 --2- 172.21.15.7:0/3606597849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56086da37b70 0x56086da30b90 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f029c00fb10 tx=0x7f029c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f401659a3f693a96 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.143+0000 7f029a7fc640 1 -- 172.21.15.7:0/3606597849 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c02f070 con 0x56086da37b70 2024-06-10T01:14:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.143+0000 7f029a7fc640 1 -- 172.21.15.7:0/3606597849 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f029c020b80 con 0x56086da37b70 2024-06-10T01:14:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.143+0000 7f029a7fc640 1 -- 172.21.15.7:0/3606597849 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c016040 con 0x56086da37b70 2024-06-10T01:14:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.143+0000 7f02adb9f740 1 -- 172.21.15.7:0/3606597849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56086da37b70 msgr2=0x56086da30b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.143+0000 7f02adb9f740 1 --2- 172.21.15.7:0/3606597849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56086da37b70 0x56086da30b90 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f029c00fb10 tx=0x7f029c005f90 comp rx=0 tx=0).stop 2024-06-10T01:14:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.143+0000 7f02adb9f740 1 -- 172.21.15.7:0/3606597849 shutdown_connections 2024-06-10T01:14:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.143+0000 7f02adb9f740 1 --2- 172.21.15.7:0/3606597849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56086da37b70 0x56086da30b90 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.143+0000 7f02adb9f740 1 --2- 172.21.15.7:0/3606597849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56086da32a00 0x56086da437f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.143+0000 7f02adb9f740 1 --2- 172.21.15.7:0/3606597849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56086da32010 0x56086da30650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.143+0000 7f02adb9f740 1 -- 172.21.15.7:0/3606597849 >> 172.21.15.7:0/3606597849 conn(0x56086da2d680 msgr2=0x56086da2da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.143+0000 7f02adb9f740 1 -- 172.21.15.7:0/3606597849 shutdown_connections 2024-06-10T01:14:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.143+0000 7f02adb9f740 1 -- 172.21.15.7:0/3606597849 wait complete. 2024-06-10T01:14:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.144+0000 7f02adb9f740 1 -- start start 2024-06-10T01:14:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.144+0000 7f02adb9f740 1 Processor -- start 2024-06-10T01:14:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.144+0000 7f02adb9f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56086da32010 0x56086da430d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.144+0000 7f02adb9f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56086da32a00 0x56086da43610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.144+0000 7f02adb9f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56086da37b70 0x56086da3fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.144+0000 7f02adb9f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f02a41a8a80 con 0x56086da32a00 2024-06-10T01:14:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.144+0000 7f02adb9f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f02a41a9ef0 con 0x56086da37b70 2024-06-10T01:14:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.144+0000 7f02adb9f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f02a41a9150 con 0x56086da32010 2024-06-10T01:14:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.144+0000 7f02adb9f740 10 client.0 _ll_register_callbacks cb 0x7f028c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:14:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.144+0000 7f02a3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56086da32a00 0x56086da43610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.144+0000 7f02a3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56086da32a00 0x56086da43610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46328/0 (socket says 172.21.15.7:46328) 2024-06-10T01:14:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.144+0000 7f02a3fff640 1 -- 172.21.15.7:0/442948448 learned_addr learned my addr 172.21.15.7:0/442948448 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.144+0000 7f02a8b4c640 1 --2- 172.21.15.7:0/442948448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56086da32010 0x56086da430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.144+0000 7f02a3fff640 1 -- 172.21.15.7:0/442948448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56086da32010 msgr2=0x56086da430d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.144+0000 7f02a3fff640 1 --2- 172.21.15.7:0/442948448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56086da32010 0x56086da430d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.144+0000 7f02a934d640 1 --2- 172.21.15.7:0/442948448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56086da37b70 0x56086da3fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.144+0000 7f02a3fff640 1 -- 172.21.15.7:0/442948448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56086da37b70 msgr2=0x56086da3fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.144+0000 7f02a3fff640 1 --2- 172.21.15.7:0/442948448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56086da37b70 0x56086da3fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.144+0000 7f02a3fff640 1 -- 172.21.15.7:0/442948448 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56086da40300 con 0x56086da32a00 2024-06-10T01:14:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.144+0000 7f02a934d640 1 --2- 172.21.15.7:0/442948448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56086da37b70 0x56086da3fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:14:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.145+0000 7f02a3fff640 1 --2- 172.21.15.7:0/442948448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56086da32a00 0x56086da43610 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f028c0b9850 tx=0x7f028c030e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf788e5a09e648c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.145+0000 7f0283fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:14:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.145+0000 7f0283fff640 1 -- 172.21.15.7:0/442948448 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c0bd610 con 0x56086da32a00 2024-06-10T01:14:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.145+0000 7f0283fff640 1 -- 172.21.15.7:0/442948448 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f028c0b94f0 con 0x56086da32a00 2024-06-10T01:14:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.145+0000 7f02adb9f740 1 -- 172.21.15.7:0/442948448 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f029c020b80 con 0x56086da32a00 2024-06-10T01:14:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.145+0000 7f0283fff640 1 -- 172.21.15.7:0/442948448 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f028c0bcf10 con 0x56086da32a00 2024-06-10T01:14:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.145+0000 7f02adb9f740 10 client.17523 fetch_fsmap learned FSMap version 35 2024-06-10T01:14:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.145+0000 7f02adb9f740 1 -- 172.21.15.7:0/442948448 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56086dab1a60 con 0x56086da32a00 2024-06-10T01:14:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.145+0000 7f0283fff640 1 -- 172.21.15.7:0/442948448 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f029c020b80 con 0x56086da32a00 2024-06-10T01:14:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.145+0000 7f0283fff640 1 -- 172.21.15.7:0/442948448 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f028c0c56f0 con 0x56086da32a00 2024-06-10T01:14:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.145+0000 7f02adb9f740 10 client.17523 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:14:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.145+0000 7f02adb9f740 10 client.17523 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:14:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.145+0000 7f02adb9f740 20 client.17523 populate_metadata read hostname 'smithi007' 2024-06-10T01:14:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.145+0000 7f02adb9f740 1 --2- 172.21.15.7:0/442948448 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56086dab2400 0x56086dab4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.145+0000 7f02adb9f740 4 client.17523 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:14:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.145+0000 7f02adb9f740 1 -- 172.21.15.7:0/442948448 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56086d9a2c20 con 0x56086dab2400 2024-06-10T01:14:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.146+0000 7f02a8b4c640 1 --2- 172.21.15.7:0/442948448 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56086dab2400 0x56086dab4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.146+0000 7f02a8b4c640 1 --2- 172.21.15.7:0/442948448 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56086dab2400 0x56086dab4820 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35a58b6638307975 server_cookie=afcde563ade8fac8 in_seq=0 out_seq=0 2024-06-10T01:14:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.146+0000 7f0283fff640 10 client.17523 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:13.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.200+0000 7f0283fff640 1 -- 172.21.15.7:0/442948448 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56086d9a2c20 con 0x56086dab2400 2024-06-10T01:14:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.200+0000 7f0283fff640 10 client.17523 handle_command_reply: tid=0 2024-06-10T01:14:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.200+0000 7f02adb9f740 1 client.17523 shutdown 2024-06-10T01:14:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.200+0000 7f02adb9f740 1 -- 172.21.15.7:0/442948448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56086da32a00 msgr2=0x56086da43610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.200+0000 7f02adb9f740 1 --2- 172.21.15.7:0/442948448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56086da32a00 0x56086da43610 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f028c0b9850 tx=0x7f028c030e40 comp rx=0 tx=0).stop 2024-06-10T01:14:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.200+0000 7f02adb9f740 1 -- 172.21.15.7:0/442948448 shutdown_connections 2024-06-10T01:14:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.200+0000 7f02adb9f740 1 --2- 172.21.15.7:0/442948448 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56086dab2400 0x56086dab4820 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.200+0000 7f02adb9f740 1 --2- 172.21.15.7:0/442948448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56086da37b70 0x56086da3fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.200+0000 7f0283fff640 0 client.17523 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.200+0000 7f02adb9f740 1 --2- 172.21.15.7:0/442948448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56086da32a00 0x56086da43610 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.200+0000 7f02adb9f740 1 --2- 172.21.15.7:0/442948448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56086da32010 0x56086da430d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.200+0000 7f02adb9f740 1 -- 172.21.15.7:0/442948448 >> 172.21.15.7:0/442948448 conn(0x56086da2d680 msgr2=0x56086da30650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.201+0000 7f02adb9f740 1 -- 172.21.15.7:0/442948448 shutdown_connections 2024-06-10T01:14:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.201+0000 7f02adb9f740 1 -- 172.21.15.7:0/442948448 wait complete. 2024-06-10T01:14:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.201+0000 7f02adb9f740 20 client.17523 trim_cache size 0 max 16384 2024-06-10T01:14:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:14:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:14:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:14:13.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.201+0000 7f02aadb0640 1 -- 172.21.15.7:0/1021752196 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f02900a2b80 msgr2=0x7f02900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:13.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.201+0000 7f02aadb0640 1 --2- 172.21.15.7:0/1021752196 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f02900a2b80 0x7f02900a5040 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f029c009710 tx=0x7f029c009b30 comp rx=0 tx=0).stop 2024-06-10T01:14:13.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.201+0000 7f02aadb0640 1 -- 172.21.15.7:0/1021752196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02a4102c00 msgr2=0x7f02a4198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:13.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.201+0000 7f02aadb0640 1 --2- 172.21.15.7:0/1021752196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02a4102c00 0x7f02a4198260 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f028c031a40 tx=0x7f028c031ee0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.201+0000 7f02aadb0640 1 -- 172.21.15.7:0/1021752196 shutdown_connections 2024-06-10T01:14:13.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.201+0000 7f02aadb0640 1 --2- 172.21.15.7:0/1021752196 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f02900a2b80 0x7f02900a5040 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.201+0000 7f02aadb0640 1 --2- 172.21.15.7:0/1021752196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02a41035c0 0x7f02a419f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.201+0000 7f02aadb0640 1 --2- 172.21.15.7:0/1021752196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02a4102c00 0x7f02a4198260 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.201+0000 7f02aadb0640 1 --2- 172.21.15.7:0/1021752196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02a4101a10 0x7f02a4197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:13.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.201+0000 7f02aadb0640 1 -- 172.21.15.7:0/1021752196 >> 172.21.15.7:0/1021752196 conn(0x7f02a40fcf80 msgr2=0x7f02a4105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:13.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.201+0000 7f02aadb0640 1 -- 172.21.15.7:0/1021752196 shutdown_connections 2024-06-10T01:14:13.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:13.201+0000 7f02aadb0640 1 -- 172.21.15.7:0/1021752196 wait complete. 2024-06-10T01:14:13.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:13 smithi175 ceph-mon[36636]: pgmap v678: 129 pgs: 129 active+clean; 106 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 2.2 MiB/s wr, 240 op/s 2024-06-10T01:14:13.740 INFO:tasks.workunit.client.0.smithi007.stdout: 1 595615 7.77 MB/sec execute 294 sec latency 9342.687 ms 2024-06-10T01:14:13.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:13 smithi007 ceph-mon[32398]: pgmap v678: 129 pgs: 129 active+clean; 106 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 2.2 MiB/s wr, 240 op/s 2024-06-10T01:14:13.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:13 smithi078 ceph-mon[36683]: pgmap v678: 129 pgs: 129 active+clean; 106 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 2.2 MiB/s wr, 240 op/s 2024-06-10T01:14:14.527 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:14:14.740 INFO:tasks.workunit.client.0.smithi007.stdout: 1 595615 7.74 MB/sec execute 295 sec latency 10342.720 ms 2024-06-10T01:14:14.789 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:14:15.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:15 smithi007 ceph-mon[32398]: pgmap v679: 129 pgs: 129 active+clean; 106 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 2.3 MiB/s wr, 227 op/s 2024-06-10T01:14:15.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:15 smithi175 ceph-mon[36636]: pgmap v679: 129 pgs: 129 active+clean; 106 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 2.3 MiB/s wr, 227 op/s 2024-06-10T01:14:15.740 INFO:tasks.workunit.client.0.smithi007.stdout: 1 595615 7.72 MB/sec execute 296 sec latency 11342.813 ms 2024-06-10T01:14:15.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:15 smithi078 ceph-mon[36683]: pgmap v679: 129 pgs: 129 active+clean; 106 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 2.3 MiB/s wr, 227 op/s 2024-06-10T01:14:16.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.594+0000 7f5b24759700 1 -- 172.21.15.7:0/1596030760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b1c06aa00 msgr2=0x7f5b1c06ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:16.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.594+0000 7f5b24759700 1 --2- 172.21.15.7:0/1596030760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b1c06aa00 0x7f5b1c06ae80 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c009990 tx=0x7f5b0c009ca0 comp rx=0 tx=0).stop 2024-06-10T01:14:16.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.594+0000 7f5b24759700 1 -- 172.21.15.7:0/1596030760 shutdown_connections 2024-06-10T01:14:16.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.594+0000 7f5b24759700 1 --2- 172.21.15.7:0/1596030760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1c06b3c0 0x7f5b1c10f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:16.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.594+0000 7f5b24759700 1 --2- 172.21.15.7:0/1596030760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b1c1084f0 0x7f5b1c1088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:16.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.594+0000 7f5b24759700 1 --2- 172.21.15.7:0/1596030760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b1c06aa00 0x7f5b1c06ae80 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:16.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.594+0000 7f5b24759700 1 -- 172.21.15.7:0/1596030760 >> 172.21.15.7:0/1596030760 conn(0x7f5b1c078300 msgr2=0x7f5b1c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:16.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.594+0000 7f5b24759700 1 -- 172.21.15.7:0/1596030760 shutdown_connections 2024-06-10T01:14:16.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.594+0000 7f5b24759700 1 -- 172.21.15.7:0/1596030760 wait complete. 2024-06-10T01:14:16.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.594+0000 7f5b24759700 1 Processor -- start 2024-06-10T01:14:16.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.595+0000 7f5b24759700 1 -- start start 2024-06-10T01:14:16.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.595+0000 7f5b24759700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b1c06aa00 0x7f5b1c101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:16.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.595+0000 7f5b24759700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b1c06b3c0 0x7f5b1c101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:16.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.595+0000 7f5b24759700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1c1084f0 0x7f5b1c101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:16.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.595+0000 7f5b24759700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c112750 con 0x7f5b1c06b3c0 2024-06-10T01:14:16.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.595+0000 7f5b24759700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c1125d0 con 0x7f5b1c1084f0 2024-06-10T01:14:16.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.595+0000 7f5b24759700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c1128d0 con 0x7f5b1c06aa00 2024-06-10T01:14:16.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.595+0000 7f5b21cf4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b1c06b3c0 0x7f5b1c101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:16.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.595+0000 7f5b21cf4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b1c06b3c0 0x7f5b1c101990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46342/0 (socket says 172.21.15.7:46342) 2024-06-10T01:14:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.595+0000 7f5b21cf4700 1 -- 172.21.15.7:0/807814168 learned_addr learned my addr 172.21.15.7:0/807814168 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.595+0000 7f5b224f5700 1 --2- 172.21.15.7:0/807814168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b1c06aa00 0x7f5b1c101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.595+0000 7f5b22cf6700 1 --2- 172.21.15.7:0/807814168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1c1084f0 0x7f5b1c101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.595+0000 7f5b21cf4700 1 -- 172.21.15.7:0/807814168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b1c06aa00 msgr2=0x7f5b1c101450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.595+0000 7f5b21cf4700 1 --2- 172.21.15.7:0/807814168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b1c06aa00 0x7f5b1c101450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.595+0000 7f5b21cf4700 1 -- 172.21.15.7:0/807814168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1c1084f0 msgr2=0x7f5b1c101ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.595+0000 7f5b21cf4700 1 --2- 172.21.15.7:0/807814168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1c1084f0 0x7f5b1c101ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.595+0000 7f5b21cf4700 1 -- 172.21.15.7:0/807814168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b1c1a71b0 con 0x7f5b1c06b3c0 2024-06-10T01:14:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.595+0000 7f5b224f5700 1 --2- 172.21.15.7:0/807814168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b1c06aa00 0x7f5b1c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:14:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.596+0000 7f5b21cf4700 1 --2- 172.21.15.7:0/807814168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b1c06b3c0 0x7f5b1c101990 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c000c00 tx=0x7f5b0c00bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.596+0000 7f5b137fe700 1 -- 172.21.15.7:0/807814168 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c027070 con 0x7f5b1c06b3c0 2024-06-10T01:14:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.596+0000 7f5b137fe700 1 -- 172.21.15.7:0/807814168 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b0c003d70 con 0x7f5b1c06b3c0 2024-06-10T01:14:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.596+0000 7f5b137fe700 1 -- 172.21.15.7:0/807814168 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c00f460 con 0x7f5b1c06b3c0 2024-06-10T01:14:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.596+0000 7f5b24759700 1 -- 172.21.15.7:0/807814168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b1c1a7440 con 0x7f5b1c06b3c0 2024-06-10T01:14:16.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.596+0000 7f5b24759700 1 -- 172.21.15.7:0/807814168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b1c1a77e0 con 0x7f5b1c06b3c0 2024-06-10T01:14:16.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.597+0000 7f5b24759700 1 -- 172.21.15.7:0/807814168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b1c069180 con 0x7f5b1c06b3c0 2024-06-10T01:14:16.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.598+0000 7f5b137fe700 1 -- 172.21.15.7:0/807814168 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f5b0c02a070 con 0x7f5b1c06b3c0 2024-06-10T01:14:16.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.599+0000 7f5b137fe700 1 --2- 172.21.15.7:0/807814168 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5b080a2d70 0x7f5b080a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:16.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.599+0000 7f5b137fe700 1 -- 172.21.15.7:0/807814168 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f5b0c0d00f0 con 0x7f5b1c06b3c0 2024-06-10T01:14:16.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.599+0000 7f5b224f5700 1 --2- 172.21.15.7:0/807814168 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5b080a2d70 0x7f5b080a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:16.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.600+0000 7f5b224f5700 1 --2- 172.21.15.7:0/807814168 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5b080a2d70 0x7f5b080a5230 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5b18009bc0 tx=0x7f5b18009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:16.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.604+0000 7f5b137fe700 1 -- 172.21.15.7:0/807814168 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5b0c013070 con 0x7f5b1c06b3c0 2024-06-10T01:14:16.740 INFO:tasks.workunit.client.0.smithi007.stdout: 1 595615 7.69 MB/sec execute 297 sec latency 12342.877 ms 2024-06-10T01:14:16.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.858+0000 7f5b24759700 1 -- 172.21.15.7:0/807814168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b1c02dca0 con 0x7f5b1c06b3c0 2024-06-10T01:14:16.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.859+0000 7f5b137fe700 1 -- 172.21.15.7:0/807814168 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11859 (secure 0 0 0) 0x7f5b0c0bc020 con 0x7f5b1c06b3c0 2024-06-10T01:14:16.854 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:14:16.854 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:13:22.220710+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:14:16.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.861+0000 7f5b24759700 1 -- 172.21.15.7:0/807814168 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5b080a2d70 msgr2=0x7f5b080a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:16.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.861+0000 7f5b24759700 1 --2- 172.21.15.7:0/807814168 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5b080a2d70 0x7f5b080a5230 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5b18009bc0 tx=0x7f5b18009260 comp rx=0 tx=0).stop 2024-06-10T01:14:16.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.861+0000 7f5b24759700 1 -- 172.21.15.7:0/807814168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b1c06b3c0 msgr2=0x7f5b1c101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:16.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.861+0000 7f5b24759700 1 --2- 172.21.15.7:0/807814168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b1c06b3c0 0x7f5b1c101990 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c000c00 tx=0x7f5b0c00bb60 comp rx=0 tx=0).stop 2024-06-10T01:14:16.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.861+0000 7f5b24759700 1 -- 172.21.15.7:0/807814168 shutdown_connections 2024-06-10T01:14:16.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.861+0000 7f5b24759700 1 --2- 172.21.15.7:0/807814168 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5b080a2d70 0x7f5b080a5230 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:16.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.861+0000 7f5b24759700 1 --2- 172.21.15.7:0/807814168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1c1084f0 0x7f5b1c101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:16.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.861+0000 7f5b24759700 1 --2- 172.21.15.7:0/807814168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b1c06aa00 0x7f5b1c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.861+0000 7f5b24759700 1 --2- 172.21.15.7:0/807814168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b1c06b3c0 0x7f5b1c101990 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.861+0000 7f5b24759700 1 -- 172.21.15.7:0/807814168 >> 172.21.15.7:0/807814168 conn(0x7f5b1c078300 msgr2=0x7f5b1c100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.862+0000 7f5b24759700 1 -- 172.21.15.7:0/807814168 shutdown_connections 2024-06-10T01:14:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:16.862+0000 7f5b24759700 1 -- 172.21.15.7:0/807814168 wait complete. 2024-06-10T01:14:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 35 2024-06-10T01:14:17.512 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:17 smithi007 ceph-mon[32398]: pgmap v680: 129 pgs: 129 active+clean; 105 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 1.8 MiB/s wr, 149 op/s 2024-06-10T01:14:17.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:17 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/807814168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:14:17.643 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db --set-id 6ddf71a3 --release --await' 2024-06-10T01:14:17.643 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db --set-id 6ddf71a3 --release --await 2024-06-10T01:14:17.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:17 smithi175 ceph-mon[36636]: pgmap v680: 129 pgs: 129 active+clean; 105 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 1.8 MiB/s wr, 149 op/s 2024-06-10T01:14:17.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:17 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/807814168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:14:17.740 INFO:tasks.workunit.client.0.smithi007.stdout: 1 595615 7.67 MB/sec execute 298 sec latency 13342.927 ms 2024-06-10T01:14:17.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:17 smithi078 ceph-mon[36683]: pgmap v680: 129 pgs: 129 active+clean; 105 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 1.8 MiB/s wr, 149 op/s 2024-06-10T01:14:17.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:17 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/807814168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:14:17.896 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:14:18.740 INFO:tasks.workunit.client.0.smithi007.stdout: 1 595615 7.64 MB/sec execute 299 sec latency 14342.978 ms 2024-06-10T01:14:19.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.552+0000 7ff0b2c23700 1 -- 172.21.15.7:0/4020981956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0ac101fc0 msgr2=0x7ff0ac106010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:19.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.552+0000 7ff0b2c23700 1 --2- 172.21.15.7:0/4020981956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0ac101fc0 0x7ff0ac106010 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff0980099c0 tx=0x7ff098009cd0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.552+0000 7ff0b2c23700 1 -- 172.21.15.7:0/4020981956 shutdown_connections 2024-06-10T01:14:19.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.552+0000 7ff0b2c23700 1 --2- 172.21.15.7:0/4020981956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0ac1066f0 0x7ff0ac10e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.552+0000 7ff0b2c23700 1 --2- 172.21.15.7:0/4020981956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0ac101610 0x7ff0ac1019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.552+0000 7ff0b2c23700 1 --2- 172.21.15.7:0/4020981956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0ac101fc0 0x7ff0ac106010 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.552+0000 7ff0b2c23700 1 -- 172.21.15.7:0/4020981956 >> 172.21.15.7:0/4020981956 conn(0x7ff0ac078430 msgr2=0x7ff0ac078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:19.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.552+0000 7ff0b2c23700 1 -- 172.21.15.7:0/4020981956 shutdown_connections 2024-06-10T01:14:19.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:19 smithi007 ceph-mon[32398]: pgmap v681: 129 pgs: 129 active+clean; 102 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.7 MiB/s wr, 117 op/s 2024-06-10T01:14:19.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.552+0000 7ff0b2c23700 1 -- 172.21.15.7:0/4020981956 wait complete. 2024-06-10T01:14:19.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.553+0000 7ff0b2c23700 1 Processor -- start 2024-06-10T01:14:19.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.553+0000 7ff0b2c23700 1 -- start start 2024-06-10T01:14:19.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.553+0000 7ff0b2c23700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0ac101610 0x7ff0ac19ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:19.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.553+0000 7ff0b2c23700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0ac101fc0 0x7ff0ac19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:19.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.553+0000 7ff0b2c23700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0ac1066f0 0x7ff0ac19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:19.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.553+0000 7ff0b2c23700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac111720 con 0x7ff0ac1066f0 2024-06-10T01:14:19.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.553+0000 7ff0b2c23700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac1115a0 con 0x7ff0ac101610 2024-06-10T01:14:19.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.553+0000 7ff0b2c23700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac1118a0 con 0x7ff0ac101fc0 2024-06-10T01:14:19.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.553+0000 7ff0b11c0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0ac1066f0 0x7ff0ac19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:19.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.553+0000 7ff0abfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0ac101fc0 0x7ff0ac19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:19.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.553+0000 7ff0abfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0ac101fc0 0x7ff0ac19b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:60222/0 (socket says 172.21.15.7:60222) 2024-06-10T01:14:19.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.553+0000 7ff0b11c0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0ac1066f0 0x7ff0ac19f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46356/0 (socket says 172.21.15.7:46356) 2024-06-10T01:14:19.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.553+0000 7ff0abfff700 1 -- 172.21.15.7:0/2918327937 learned_addr learned my addr 172.21.15.7:0/2918327937 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:19.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.553+0000 7ff0b09bf700 1 --2- 172.21.15.7:0/2918327937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0ac101610 0x7ff0ac19ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:19.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.554+0000 7ff0b11c0700 1 -- 172.21.15.7:0/2918327937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0ac101fc0 msgr2=0x7ff0ac19b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:19.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.554+0000 7ff0b11c0700 1 --2- 172.21.15.7:0/2918327937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0ac101fc0 0x7ff0ac19b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.554+0000 7ff0b11c0700 1 -- 172.21.15.7:0/2918327937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0ac101610 msgr2=0x7ff0ac19ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:19.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.554+0000 7ff0b11c0700 1 --2- 172.21.15.7:0/2918327937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0ac101610 0x7ff0ac19ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.554+0000 7ff0b11c0700 1 -- 172.21.15.7:0/2918327937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ac19fe80 con 0x7ff0ac1066f0 2024-06-10T01:14:19.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.554+0000 7ff0abfff700 1 --2- 172.21.15.7:0/2918327937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0ac101fc0 0x7ff0ac19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:19.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.554+0000 7ff0b11c0700 1 --2- 172.21.15.7:0/2918327937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0ac1066f0 0x7ff0ac19f740 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4007f10 tx=0x7ff0a400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:19.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.554+0000 7ff0a9ffb700 1 -- 172.21.15.7:0/2918327937 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4004db0 con 0x7ff0ac1066f0 2024-06-10T01:14:19.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.554+0000 7ff0a9ffb700 1 -- 172.21.15.7:0/2918327937 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a4009e20 con 0x7ff0ac1066f0 2024-06-10T01:14:19.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.554+0000 7ff0b2c23700 1 -- 172.21.15.7:0/2918327937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0ac1a0170 con 0x7ff0ac1066f0 2024-06-10T01:14:19.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.554+0000 7ff0a9ffb700 1 -- 172.21.15.7:0/2918327937 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4015400 con 0x7ff0ac1066f0 2024-06-10T01:14:19.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.554+0000 7ff0b2c23700 1 -- 172.21.15.7:0/2918327937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0ac1a79f0 con 0x7ff0ac1066f0 2024-06-10T01:14:19.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.556+0000 7ff0a9ffb700 1 -- 172.21.15.7:0/2918327937 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7ff0a400db20 con 0x7ff0ac1066f0 2024-06-10T01:14:19.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.557+0000 7ff0a9ffb700 1 --2- 172.21.15.7:0/2918327937 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff09c0a2ca0 0x7ff09c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:19.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.557+0000 7ff0a9ffb700 1 -- 172.21.15.7:0/2918327937 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7ff0a40cec40 con 0x7ff0ac1066f0 2024-06-10T01:14:19.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.557+0000 7ff0b09bf700 1 --2- 172.21.15.7:0/2918327937 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff09c0a2ca0 0x7ff09c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:19.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.558+0000 7ff0b09bf700 1 --2- 172.21.15.7:0/2918327937 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff09c0a2ca0 0x7ff09c0a5160 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0005950 tx=0x7ff0a000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:19.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.558+0000 7ff0bd4a6b80 1 Processor -- start 2024-06-10T01:14:19.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.558+0000 7ff0bd4a6b80 1 -- start start 2024-06-10T01:14:19.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.558+0000 7ff0bd4a6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56059d23cc40 0x56059d23d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:19.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.558+0000 7ff0bd4a6b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56059d23d560 0x56059d236e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:19.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.558+0000 7ff0bd4a6b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56059d2373d0 0x56059d237810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:19.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.558+0000 7ff0bd4a6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56059d188910 con 0x56059d23cc40 2024-06-10T01:14:19.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.558+0000 7ff0bd4a6b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56059d130820 con 0x56059d2373d0 2024-06-10T01:14:19.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.558+0000 7ff0bd4a6b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56059d1848d0 con 0x56059d23d560 2024-06-10T01:14:19.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.558+0000 7ff0b09bf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56059d23cc40 0x56059d23d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:19.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.558+0000 7ff0b09bf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56059d23cc40 0x56059d23d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46364/0 (socket says 172.21.15.7:46364) 2024-06-10T01:14:19.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.558+0000 7ff0b09bf700 1 -- 172.21.15.7:0/813466049 learned_addr learned my addr 172.21.15.7:0/813466049 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:19.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.559+0000 7ff0b11c0700 1 --2- 172.21.15.7:0/813466049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56059d2373d0 0x56059d237810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:19.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.559+0000 7ff0abfff700 1 --2- 172.21.15.7:0/813466049 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56059d23d560 0x56059d236e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:19.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.559+0000 7ff0b09bf700 1 -- 172.21.15.7:0/813466049 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56059d23d560 msgr2=0x56059d236e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:19.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.559+0000 7ff0b09bf700 1 --2- 172.21.15.7:0/813466049 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56059d23d560 0x56059d236e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.559+0000 7ff0b09bf700 1 -- 172.21.15.7:0/813466049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56059d2373d0 msgr2=0x56059d237810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:19.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.559+0000 7ff0b09bf700 1 --2- 172.21.15.7:0/813466049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56059d2373d0 0x56059d237810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.559+0000 7ff0b09bf700 1 -- 172.21.15.7:0/813466049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56059d127050 con 0x56059d23cc40 2024-06-10T01:14:19.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.559+0000 7ff0b11c0700 1 --2- 172.21.15.7:0/813466049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56059d2373d0 0x56059d237810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:14:19.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.559+0000 7ff0b09bf700 1 --2- 172.21.15.7:0/813466049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56059d23cc40 0x56059d23d020 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0010fa0 tx=0x7ff0a0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf18cc4aedd269 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:19.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.559+0000 7ff096ffd700 1 -- 172.21.15.7:0/813466049 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a000be60 con 0x56059d23cc40 2024-06-10T01:14:19.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.559+0000 7ff096ffd700 1 -- 172.21.15.7:0/813466049 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a001eb00 con 0x56059d23cc40 2024-06-10T01:14:19.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.559+0000 7ff096ffd700 1 -- 172.21.15.7:0/813466049 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a001cc20 con 0x56059d23cc40 2024-06-10T01:14:19.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.560+0000 7ff0bd4a6b80 1 -- 172.21.15.7:0/813466049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56059d23cc40 msgr2=0x56059d23d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:19.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.560+0000 7ff0bd4a6b80 1 --2- 172.21.15.7:0/813466049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56059d23cc40 0x56059d23d020 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0010fa0 tx=0x7ff0a0013000 comp rx=0 tx=0).stop 2024-06-10T01:14:19.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.560+0000 7ff0bd4a6b80 1 -- 172.21.15.7:0/813466049 shutdown_connections 2024-06-10T01:14:19.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.560+0000 7ff0bd4a6b80 1 --2- 172.21.15.7:0/813466049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56059d2373d0 0x56059d237810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.560+0000 7ff0bd4a6b80 1 --2- 172.21.15.7:0/813466049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56059d23cc40 0x56059d23d020 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.560+0000 7ff0bd4a6b80 1 --2- 172.21.15.7:0/813466049 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56059d23d560 0x56059d236e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.560+0000 7ff0bd4a6b80 1 -- 172.21.15.7:0/813466049 >> 172.21.15.7:0/813466049 conn(0x56059d161dc0 msgr2=0x56059d232120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:19.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.560+0000 7ff0bd4a6b80 1 -- 172.21.15.7:0/813466049 shutdown_connections 2024-06-10T01:14:19.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.560+0000 7ff0bd4a6b80 1 -- 172.21.15.7:0/813466049 wait complete. 2024-06-10T01:14:19.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.560+0000 7ff0bd4a6b80 1 -- start start 2024-06-10T01:14:19.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.560+0000 7ff0bd4a6b80 1 Processor -- start 2024-06-10T01:14:19.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.561+0000 7ff0bd4a6b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56059d2373d0 0x56059d243240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:19.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.561+0000 7ff0bd4a6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56059d23cc40 0x56059d240540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:19.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.561+0000 7ff0bd4a6b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56059d23d560 0x56059d240a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:19.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.561+0000 7ff0bd4a6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56059d243fd0 con 0x56059d23cc40 2024-06-10T01:14:19.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.561+0000 7ff0bd4a6b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56059d128cf0 con 0x56059d23d560 2024-06-10T01:14:19.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.561+0000 7ff0bd4a6b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56059d244150 con 0x56059d2373d0 2024-06-10T01:14:19.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.561+0000 7ff0bd4a6b80 10 client.0 _ll_register_callbacks cb 0x56059d122a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:14:19.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.561+0000 7ff0abfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56059d23cc40 0x56059d240540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:19.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.561+0000 7ff0abfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56059d23cc40 0x56059d240540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46368/0 (socket says 172.21.15.7:46368) 2024-06-10T01:14:19.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.561+0000 7ff0abfff700 1 -- 172.21.15.7:0/3779847035 learned_addr learned my addr 172.21.15.7:0/3779847035 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:19.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.561+0000 7ff0b09bf700 1 --2- 172.21.15.7:0/3779847035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56059d2373d0 0x56059d243240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:19.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.561+0000 7ff0b11c0700 1 --2- 172.21.15.7:0/3779847035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56059d23d560 0x56059d240a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:19.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.561+0000 7ff0abfff700 1 -- 172.21.15.7:0/3779847035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56059d2373d0 msgr2=0x56059d243240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:19.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.561+0000 7ff0abfff700 1 --2- 172.21.15.7:0/3779847035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56059d2373d0 0x56059d243240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.561+0000 7ff0abfff700 1 -- 172.21.15.7:0/3779847035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56059d23d560 msgr2=0x56059d240a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:19.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.561+0000 7ff0abfff700 1 --2- 172.21.15.7:0/3779847035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56059d23d560 0x56059d240a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.561+0000 7ff0abfff700 1 -- 172.21.15.7:0/3779847035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56059d0d79d0 con 0x56059d23cc40 2024-06-10T01:14:19.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.561+0000 7ff0b11c0700 1 --2- 172.21.15.7:0/3779847035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56059d23d560 0x56059d240a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:14:19.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.561+0000 7ff0abfff700 1 --2- 172.21.15.7:0/3779847035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56059d23cc40 0x56059d240540 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7ff098009720 tx=0x7ff098003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e6d83fd367b82 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:19.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.561+0000 7ff08bfff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:14:19.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.562+0000 7ff08bfff700 1 -- 172.21.15.7:0/3779847035 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff098028060 con 0x56059d23cc40 2024-06-10T01:14:19.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.562+0000 7ff08bfff700 1 -- 172.21.15.7:0/3779847035 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7ff098003d70 con 0x56059d23cc40 2024-06-10T01:14:19.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.562+0000 7ff0bd4a6b80 1 -- 172.21.15.7:0/3779847035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0a001eb00 con 0x56059d23cc40 2024-06-10T01:14:19.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.562+0000 7ff08bfff700 1 -- 172.21.15.7:0/3779847035 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff09802f050 con 0x56059d23cc40 2024-06-10T01:14:19.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.562+0000 7ff0bd4a6b80 10 client.17541 fetch_fsmap learned FSMap version 35 2024-06-10T01:14:19.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.562+0000 7ff0bd4a6b80 1 -- 172.21.15.7:0/3779847035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56059d24e870 con 0x56059d23cc40 2024-06-10T01:14:19.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.562+0000 7ff08bfff700 1 -- 172.21.15.7:0/3779847035 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a001eb00 con 0x56059d23cc40 2024-06-10T01:14:19.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.562+0000 7ff08bfff700 1 -- 172.21.15.7:0/3779847035 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7ff0980238e0 con 0x56059d23cc40 2024-06-10T01:14:19.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.562+0000 7ff0bd4a6b80 10 client.17541 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:14:19.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.562+0000 7ff0bd4a6b80 10 client.17541 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:14:19.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.562+0000 7ff0bd4a6b80 20 client.17541 populate_metadata read hostname 'smithi007' 2024-06-10T01:14:19.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.562+0000 7ff0bd4a6b80 1 --2- 172.21.15.7:0/3779847035 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56059d250eb0 0x56059d2532a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:19.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.562+0000 7ff0bd4a6b80 4 client.17541 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:14:19.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.562+0000 7ff0bd4a6b80 1 -- 172.21.15.7:0/3779847035 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56059d195e10 con 0x56059d250eb0 2024-06-10T01:14:19.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.563+0000 7ff0b09bf700 1 --2- 172.21.15.7:0/3779847035 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56059d250eb0 0x56059d2532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:19.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.563+0000 7ff08bfff700 10 client.17541 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:19.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.563+0000 7ff0b09bf700 1 --2- 172.21.15.7:0/3779847035 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56059d250eb0 0x56059d2532a0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1697d014b75d3de6 server_cookie=3b43c14f4d824f92 in_seq=0 out_seq=0 2024-06-10T01:14:19.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.564+0000 7ff08bfff700 1 -- 172.21.15.7:0/3779847035 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56059d195e10 con 0x56059d250eb0 2024-06-10T01:14:19.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.564+0000 7ff08bfff700 10 client.17541 handle_command_reply: tid=0 2024-06-10T01:14:19.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.564+0000 7ff0bd4a6b80 1 client.17541 shutdown 2024-06-10T01:14:19.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.564+0000 7ff0bd4a6b80 1 -- 172.21.15.7:0/3779847035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56059d23cc40 msgr2=0x56059d240540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:19.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.564+0000 7ff0bd4a6b80 1 --2- 172.21.15.7:0/3779847035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56059d23cc40 0x56059d240540 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7ff098009720 tx=0x7ff098003680 comp rx=0 tx=0).stop 2024-06-10T01:14:19.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.564+0000 7ff0bd4a6b80 1 -- 172.21.15.7:0/3779847035 shutdown_connections 2024-06-10T01:14:19.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.564+0000 7ff0bd4a6b80 1 --2- 172.21.15.7:0/3779847035 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56059d250eb0 0x56059d2532a0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.564+0000 7ff0bd4a6b80 1 --2- 172.21.15.7:0/3779847035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56059d23d560 0x56059d240a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.564+0000 7ff0bd4a6b80 1 --2- 172.21.15.7:0/3779847035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56059d2373d0 0x56059d243240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.564+0000 7ff0bd4a6b80 1 --2- 172.21.15.7:0/3779847035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56059d23cc40 0x56059d240540 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.564+0000 7ff0bd4a6b80 1 -- 172.21.15.7:0/3779847035 >> 172.21.15.7:0/3779847035 conn(0x56059d161dc0 msgr2=0x56059d231d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:19.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.564+0000 7ff08bfff700 0 client.17541 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:19.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.564+0000 7ff0bd4a6b80 1 -- 172.21.15.7:0/3779847035 shutdown_connections 2024-06-10T01:14:19.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.564+0000 7ff0bd4a6b80 1 -- 172.21.15.7:0/3779847035 wait complete. 2024-06-10T01:14:19.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.564+0000 7ff0bd4a6b80 20 client.17541 trim_cache size 0 max 16384 2024-06-10T01:14:19.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.585+0000 7ff0bd4a6b80 1 Processor -- start 2024-06-10T01:14:19.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.585+0000 7ff0bd4a6b80 1 -- start start 2024-06-10T01:14:19.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.585+0000 7ff0bd4a6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56059d2373d0 0x56059d236e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:19.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.585+0000 7ff0bd4a6b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56059d23cc40 0x56059d23e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:19.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.585+0000 7ff0bd4a6b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56059d23d560 0x56059d231ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:19.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.585+0000 7ff0bd4a6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac1b2f10 con 0x56059d2373d0 2024-06-10T01:14:19.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.585+0000 7ff0bd4a6b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac1b0740 con 0x56059d23cc40 2024-06-10T01:14:19.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.585+0000 7ff0bd4a6b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac1ae590 con 0x56059d23d560 2024-06-10T01:14:19.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.585+0000 7ff0b09bf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56059d2373d0 0x56059d236e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:19.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.585+0000 7ff0b09bf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56059d2373d0 0x56059d236e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46380/0 (socket says 172.21.15.7:46380) 2024-06-10T01:14:19.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.585+0000 7ff0b09bf700 1 -- 172.21.15.7:0/3568800372 learned_addr learned my addr 172.21.15.7:0/3568800372 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:19.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.585+0000 7ff0abfff700 1 --2- 172.21.15.7:0/3568800372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56059d23cc40 0x56059d23e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:19.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.585+0000 7ff0b11c0700 1 --2- 172.21.15.7:0/3568800372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56059d23d560 0x56059d231ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:19.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.585+0000 7ff0b09bf700 1 -- 172.21.15.7:0/3568800372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56059d23d560 msgr2=0x56059d231ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:19.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.585+0000 7ff0b09bf700 1 --2- 172.21.15.7:0/3568800372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56059d23d560 0x56059d231ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.585+0000 7ff0b09bf700 1 -- 172.21.15.7:0/3568800372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56059d23cc40 msgr2=0x56059d23e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:19.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.585+0000 7ff0b09bf700 1 --2- 172.21.15.7:0/3568800372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56059d23cc40 0x56059d23e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.585+0000 7ff0b09bf700 1 -- 172.21.15.7:0/3568800372 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56059d2336e0 con 0x56059d2373d0 2024-06-10T01:14:19.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.585+0000 7ff0abfff700 1 --2- 172.21.15.7:0/3568800372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56059d23cc40 0x56059d23e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:14:19.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.585+0000 7ff0b11c0700 1 --2- 172.21.15.7:0/3568800372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56059d23d560 0x56059d231ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:14:19.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.585+0000 7ff0b09bf700 1 --2- 172.21.15.7:0/3568800372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56059d2373d0 0x56059d236e90 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0010f40 tx=0x7ff0a001c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8323fdf95518f9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:19.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.586+0000 7ff0967fc700 1 -- 172.21.15.7:0/3568800372 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a001c850 con 0x56059d2373d0 2024-06-10T01:14:19.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.586+0000 7ff0967fc700 1 -- 172.21.15.7:0/3568800372 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a0005040 con 0x56059d2373d0 2024-06-10T01:14:19.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.586+0000 7ff0967fc700 1 -- 172.21.15.7:0/3568800372 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0004a00 con 0x56059d2373d0 2024-06-10T01:14:19.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.586+0000 7ff0bd4a6b80 1 -- 172.21.15.7:0/3568800372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56059d2373d0 msgr2=0x56059d236e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:19.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.586+0000 7ff0bd4a6b80 1 --2- 172.21.15.7:0/3568800372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56059d2373d0 0x56059d236e90 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0010f40 tx=0x7ff0a001c7e0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.586+0000 7ff0bd4a6b80 1 -- 172.21.15.7:0/3568800372 shutdown_connections 2024-06-10T01:14:19.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.586+0000 7ff0bd4a6b80 1 --2- 172.21.15.7:0/3568800372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56059d23d560 0x56059d231ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.586+0000 7ff0bd4a6b80 1 --2- 172.21.15.7:0/3568800372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56059d2373d0 0x56059d236e90 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.586+0000 7ff0bd4a6b80 1 --2- 172.21.15.7:0/3568800372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56059d23cc40 0x56059d23e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.586+0000 7ff0bd4a6b80 1 -- 172.21.15.7:0/3568800372 >> 172.21.15.7:0/3568800372 conn(0x56059d161dc0 msgr2=0x56059d23b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:19.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.586+0000 7ff0bd4a6b80 1 -- 172.21.15.7:0/3568800372 shutdown_connections 2024-06-10T01:14:19.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.586+0000 7ff0bd4a6b80 1 -- 172.21.15.7:0/3568800372 wait complete. 2024-06-10T01:14:19.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.587+0000 7ff0bd4a6b80 1 -- start start 2024-06-10T01:14:19.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.587+0000 7ff0bd4a6b80 1 Processor -- start 2024-06-10T01:14:19.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.587+0000 7ff0bd4a6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56059d2373d0 0x56059d2498a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:19.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.587+0000 7ff0bd4a6b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56059d23cc40 0x56059d246410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:19.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.587+0000 7ff0bd4a6b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56059d23d560 0x56059d246950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:19.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.587+0000 7ff0bd4a6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac1aec00 con 0x56059d2373d0 2024-06-10T01:14:19.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.587+0000 7ff0bd4a6b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac1b0070 con 0x56059d23d560 2024-06-10T01:14:19.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.587+0000 7ff0bd4a6b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac1af2d0 con 0x56059d23cc40 2024-06-10T01:14:19.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.587+0000 7ff0bd4a6b80 10 client.0 _ll_register_callbacks cb 0x56059d122a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:14:19.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.587+0000 7ff0b09bf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56059d2373d0 0x56059d2498a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:19.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.587+0000 7ff0b09bf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56059d2373d0 0x56059d2498a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46394/0 (socket says 172.21.15.7:46394) 2024-06-10T01:14:19.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.587+0000 7ff0b09bf700 1 -- 172.21.15.7:0/3561641185 learned_addr learned my addr 172.21.15.7:0/3561641185 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:19.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.588+0000 7ff0abfff700 1 --2- 172.21.15.7:0/3561641185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56059d23cc40 0x56059d246410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:19.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.588+0000 7ff0b11c0700 1 --2- 172.21.15.7:0/3561641185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56059d23d560 0x56059d246950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:19.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.588+0000 7ff0b09bf700 1 -- 172.21.15.7:0/3561641185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56059d23cc40 msgr2=0x56059d246410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:19.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.588+0000 7ff0b09bf700 1 --2- 172.21.15.7:0/3561641185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56059d23cc40 0x56059d246410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.588+0000 7ff0b09bf700 1 -- 172.21.15.7:0/3561641185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56059d23d560 msgr2=0x56059d246950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:19.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.588+0000 7ff0b09bf700 1 --2- 172.21.15.7:0/3561641185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56059d23d560 0x56059d246950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.588+0000 7ff0b09bf700 1 -- 172.21.15.7:0/3561641185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56059d246e90 con 0x56059d2373d0 2024-06-10T01:14:19.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.588+0000 7ff0b09bf700 1 --2- 172.21.15.7:0/3561641185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56059d2373d0 0x56059d2498a0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0000c00 tx=0x7ff0a00052a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e20cdf4defd324 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:19.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.588+0000 7ff08bfff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:14:19.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.588+0000 7ff08bfff700 1 -- 172.21.15.7:0/3561641185 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0041020 con 0x56059d2373d0 2024-06-10T01:14:19.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.588+0000 7ff0bd4a6b80 1 -- 172.21.15.7:0/3561641185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0a0005040 con 0x56059d2373d0 2024-06-10T01:14:19.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.588+0000 7ff08bfff700 1 -- 172.21.15.7:0/3561641185 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7ff0a001ca50 con 0x56059d2373d0 2024-06-10T01:14:19.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.588+0000 7ff08bfff700 1 -- 172.21.15.7:0/3561641185 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff0a001cc50 con 0x56059d2373d0 2024-06-10T01:14:19.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.588+0000 7ff0bd4a6b80 10 client.17547 fetch_fsmap learned FSMap version 35 2024-06-10T01:14:19.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.588+0000 7ff0bd4a6b80 1 -- 172.21.15.7:0/3561641185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56059d2bc840 con 0x56059d2373d0 2024-06-10T01:14:19.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.588+0000 7ff08bfff700 1 -- 172.21.15.7:0/3561641185 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a0005040 con 0x56059d2373d0 2024-06-10T01:14:19.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.589+0000 7ff08bfff700 1 -- 172.21.15.7:0/3561641185 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7ff0a003d6e0 con 0x56059d2373d0 2024-06-10T01:14:19.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.589+0000 7ff0bd4a6b80 10 client.17547 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:14:19.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.589+0000 7ff0bd4a6b80 10 client.17547 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:14:19.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.589+0000 7ff0bd4a6b80 20 client.17547 populate_metadata read hostname 'smithi007' 2024-06-10T01:14:19.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.589+0000 7ff0bd4a6b80 1 --2- 172.21.15.7:0/3561641185 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56059d2bce00 0x56059d2bf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:19.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.589+0000 7ff0bd4a6b80 4 client.17547 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6ddf71a3", "release": true, "await": true}] 2024-06-10T01:14:19.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.589+0000 7ff0bd4a6b80 1 -- 172.21.15.7:0/3561641185 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6ddf71a3", "release": true, "await": true}) -- 0x56059d190390 con 0x56059d2bce00 2024-06-10T01:14:19.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.589+0000 7ff0abfff700 1 --2- 172.21.15.7:0/3561641185 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56059d2bce00 0x56059d2bf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:19.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.589+0000 7ff0abfff700 1 --2- 172.21.15.7:0/3561641185 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56059d2bce00 0x56059d2bf220 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=653a2692048d8c48 server_cookie=4fa44f1c99fbcc07 in_seq=0 out_seq=0 2024-06-10T01:14:19.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.589+0000 7ff08bfff700 10 client.17547 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:19.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.602+0000 7ff08bfff700 1 -- 172.21.15.7:0/3561641185 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56059d190390 con 0x56059d2bce00 2024-06-10T01:14:19.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.602+0000 7ff08bfff700 10 client.17547 handle_command_reply: tid=0 2024-06-10T01:14:19.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.602+0000 7ff0bd4a6b80 1 client.17547 shutdown 2024-06-10T01:14:19.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.602+0000 7ff0bd4a6b80 1 -- 172.21.15.7:0/3561641185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56059d2373d0 msgr2=0x56059d2498a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:19.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.602+0000 7ff0bd4a6b80 1 --2- 172.21.15.7:0/3561641185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56059d2373d0 0x56059d2498a0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0000c00 tx=0x7ff0a00052a0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.602+0000 7ff0bd4a6b80 1 -- 172.21.15.7:0/3561641185 shutdown_connections 2024-06-10T01:14:19.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.602+0000 7ff0bd4a6b80 1 --2- 172.21.15.7:0/3561641185 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56059d2bce00 0x56059d2bf220 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.602+0000 7ff0bd4a6b80 1 --2- 172.21.15.7:0/3561641185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56059d23d560 0x56059d246950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.602+0000 7ff0bd4a6b80 1 --2- 172.21.15.7:0/3561641185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56059d2373d0 0x56059d2498a0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.602+0000 7ff08bfff700 0 client.17547 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:19.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.602+0000 7ff0bd4a6b80 1 --2- 172.21.15.7:0/3561641185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56059d23cc40 0x56059d246410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.602+0000 7ff0bd4a6b80 1 -- 172.21.15.7:0/3561641185 >> 172.21.15.7:0/3561641185 conn(0x56059d161dc0 msgr2=0x56059d240680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:19.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.602+0000 7ff0bd4a6b80 1 -- 172.21.15.7:0/3561641185 shutdown_connections 2024-06-10T01:14:19.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.602+0000 7ff0bd4a6b80 1 -- 172.21.15.7:0/3561641185 wait complete. 2024-06-10T01:14:19.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.603+0000 7ff0bd4a6b80 20 client.17547 trim_cache size 0 max 16384 2024-06-10T01:14:19.598 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:14:19.598 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 35, 2024-06-10T01:14:19.598 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:14:19.598 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 24, 2024-06-10T01:14:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:14:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "6ddf71a3": { 2024-06-10T01:14:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 24, 2024-06-10T01:14:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:14:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:14:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:14:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:14:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:14:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:14:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 136.0, 2024-06-10T01:14:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:14:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:14:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:14:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:14:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:14:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:14:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:14:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:14:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:14:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:14:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:14:19.600 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:14:19.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.603+0000 7ff0b2c23700 1 -- 172.21.15.7:0/2918327937 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff09c0a2ca0 msgr2=0x7ff09c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:19.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.603+0000 7ff0b2c23700 1 --2- 172.21.15.7:0/2918327937 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff09c0a2ca0 0x7ff09c0a5160 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0005950 tx=0x7ff0a000b410 comp rx=0 tx=0).stop 2024-06-10T01:14:19.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.603+0000 7ff0b2c23700 1 -- 172.21.15.7:0/2918327937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0ac1066f0 msgr2=0x7ff0ac19f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:19.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.603+0000 7ff0b2c23700 1 --2- 172.21.15.7:0/2918327937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0ac1066f0 0x7ff0ac19f740 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4007f10 tx=0x7ff0a400d3b0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.603+0000 7ff0b2c23700 1 -- 172.21.15.7:0/2918327937 shutdown_connections 2024-06-10T01:14:19.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.603+0000 7ff0b2c23700 1 --2- 172.21.15.7:0/2918327937 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff09c0a2ca0 0x7ff09c0a5160 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.603+0000 7ff0b2c23700 1 --2- 172.21.15.7:0/2918327937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0ac1066f0 0x7ff0ac19f740 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.604+0000 7ff0b2c23700 1 --2- 172.21.15.7:0/2918327937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0ac101610 0x7ff0ac19ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.604+0000 7ff0b2c23700 1 --2- 172.21.15.7:0/2918327937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0ac101fc0 0x7ff0ac19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:19.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.604+0000 7ff0b2c23700 1 -- 172.21.15.7:0/2918327937 >> 172.21.15.7:0/2918327937 conn(0x7ff0ac078430 msgr2=0x7ff0ac1079b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:19.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.604+0000 7ff0b2c23700 1 -- 172.21.15.7:0/2918327937 shutdown_connections 2024-06-10T01:14:19.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:19.604+0000 7ff0b2c23700 1 -- 172.21.15.7:0/2918327937 wait complete. 2024-06-10T01:14:19.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:19 smithi175 ceph-mon[36636]: pgmap v681: 129 pgs: 129 active+clean; 102 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.7 MiB/s wr, 117 op/s 2024-06-10T01:14:19.741 INFO:tasks.workunit.client.0.smithi007.stdout: 1 596063 7.63 MB/sec execute 300 sec latency 15192.862 ms 2024-06-10T01:14:19.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:19 smithi078 ceph-mon[36683]: pgmap v681: 129 pgs: 129 active+clean; 102 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.7 MiB/s wr, 117 op/s 2024-06-10T01:14:20.483 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6ddf71a3', total seconds elapsed: 22.0 2024-06-10T01:14:20.741 INFO:tasks.workunit.client.0.smithi007.stdout: 1 597804 7.64 MB/sec execute 301 sec latency 14.833 ms 2024-06-10T01:14:21.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:21 smithi175 ceph-mon[36636]: pgmap v682: 129 pgs: 129 active+clean; 102 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 1.8 MiB/s wr, 102 op/s 2024-06-10T01:14:21.741 INFO:tasks.workunit.client.0.smithi007.stdout: 1 600263 7.66 MB/sec execute 302 sec latency 61.690 ms 2024-06-10T01:14:21.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:21 smithi007 ceph-mon[32398]: pgmap v682: 129 pgs: 129 active+clean; 102 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 1.8 MiB/s wr, 102 op/s 2024-06-10T01:14:21.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:21 smithi078 ceph-mon[36683]: pgmap v682: 129 pgs: 129 active+clean; 102 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 1.8 MiB/s wr, 102 op/s 2024-06-10T01:14:22.741 INFO:tasks.workunit.client.0.smithi007.stdout: 1 601517 7.65 MB/sec execute 303 sec latency 26.138 ms 2024-06-10T01:14:23.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:23 smithi175 ceph-mon[36636]: pgmap v683: 129 pgs: 129 active+clean; 101 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 1.5 MiB/s wr, 101 op/s 2024-06-10T01:14:23.741 INFO:tasks.workunit.client.0.smithi007.stdout: 1 604086 7.68 MB/sec execute 304 sec latency 11.231 ms 2024-06-10T01:14:23.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:23 smithi007 ceph-mon[32398]: pgmap v683: 129 pgs: 129 active+clean; 101 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 1.5 MiB/s wr, 101 op/s 2024-06-10T01:14:23.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:23 smithi078 ceph-mon[36683]: pgmap v683: 129 pgs: 129 active+clean; 101 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 1.5 MiB/s wr, 101 op/s 2024-06-10T01:14:24.741 INFO:tasks.workunit.client.0.smithi007.stdout: 1 606173 7.68 MB/sec execute 305 sec latency 11.598 ms 2024-06-10T01:14:25.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:25 smithi175 ceph-mon[36636]: pgmap v684: 129 pgs: 129 active+clean; 99 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.8 MiB/s wr, 120 op/s 2024-06-10T01:14:25.741 INFO:tasks.workunit.client.0.smithi007.stdout: 1 608011 7.69 MB/sec execute 306 sec latency 12.633 ms 2024-06-10T01:14:25.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:25 smithi007 ceph-mon[32398]: pgmap v684: 129 pgs: 129 active+clean; 99 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.8 MiB/s wr, 120 op/s 2024-06-10T01:14:25.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:25 smithi078 ceph-mon[36683]: pgmap v684: 129 pgs: 129 active+clean; 99 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.8 MiB/s wr, 120 op/s 2024-06-10T01:14:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.293+0000 7f4285f5a640 1 -- 172.21.15.7:0/3500255784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4280104dc0 msgr2=0x7f42801071b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.293+0000 7f4285f5a640 1 --2- 172.21.15.7:0/3500255784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4280104dc0 0x7f42801071b0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f4268009900 tx=0x7f426802a410 comp rx=0 tx=0).stop 2024-06-10T01:14:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.293+0000 7f4285f5a640 1 -- 172.21.15.7:0/3500255784 shutdown_connections 2024-06-10T01:14:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.293+0000 7f4285f5a640 1 --2- 172.21.15.7:0/3500255784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42801076f0 0x7f4280109b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.293+0000 7f4285f5a640 1 --2- 172.21.15.7:0/3500255784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4280104dc0 0x7f42801071b0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.293+0000 7f4285f5a640 1 --2- 172.21.15.7:0/3500255784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4280102490 0x7f4280104880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.293+0000 7f4285f5a640 1 -- 172.21.15.7:0/3500255784 >> 172.21.15.7:0/3500255784 conn(0x7f42800fbe80 msgr2=0x7f42800fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.293+0000 7f4285f5a640 1 -- 172.21.15.7:0/3500255784 shutdown_connections 2024-06-10T01:14:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.293+0000 7f4285f5a640 1 -- 172.21.15.7:0/3500255784 wait complete. 2024-06-10T01:14:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.293+0000 7f4285f5a640 1 Processor -- start 2024-06-10T01:14:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.294+0000 7f4285f5a640 1 -- start start 2024-06-10T01:14:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.294+0000 7f4285f5a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4280102490 0x7f4280101e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.294+0000 7f4285f5a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4280104dc0 0x7f4280100440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.294+0000 7f4285f5a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42801076f0 0x7f4280100980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.294+0000 7f4285f5a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f428010d6d0 con 0x7f42801076f0 2024-06-10T01:14:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.294+0000 7f4285f5a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f428010d550 con 0x7f4280104dc0 2024-06-10T01:14:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.294+0000 7f4285f5a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f428010d850 con 0x7f4280102490 2024-06-10T01:14:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.294+0000 7f427ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42801076f0 0x7f4280100980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.294+0000 7f427f7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4280102490 0x7f4280101e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.294+0000 7f427ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42801076f0 0x7f4280100980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60658/0 (socket says 172.21.15.7:60658) 2024-06-10T01:14:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.294+0000 7f427f7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4280102490 0x7f4280101e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:60652/0 (socket says 172.21.15.7:60652) 2024-06-10T01:14:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.294+0000 7f427ffff640 1 -- 172.21.15.7:0/788308535 learned_addr learned my addr 172.21.15.7:0/788308535 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.294+0000 7f4276ffd640 1 --2- 172.21.15.7:0/788308535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4280104dc0 0x7f4280100440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.294+0000 7f427f7fe640 1 -- 172.21.15.7:0/788308535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4280104dc0 msgr2=0x7f4280100440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.294+0000 7f427f7fe640 1 --2- 172.21.15.7:0/788308535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4280104dc0 0x7f4280100440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.294+0000 7f427f7fe640 1 -- 172.21.15.7:0/788308535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42801076f0 msgr2=0x7f4280100980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.294+0000 7f427f7fe640 1 --2- 172.21.15.7:0/788308535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42801076f0 0x7f4280100980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.294+0000 7f427f7fe640 1 -- 172.21.15.7:0/788308535 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4280101210 con 0x7f4280102490 2024-06-10T01:14:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.294+0000 7f4276ffd640 1 --2- 172.21.15.7:0/788308535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4280104dc0 0x7f4280100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.294+0000 7f427ffff640 1 --2- 172.21.15.7:0/788308535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42801076f0 0x7f4280100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.295+0000 7f427f7fe640 1 --2- 172.21.15.7:0/788308535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4280102490 0x7f4280101e00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f427000d6d0 tx=0x7f427000db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.295+0000 7f427d7fa640 1 -- 172.21.15.7:0/788308535 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4270017070 con 0x7f4280102490 2024-06-10T01:14:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.295+0000 7f427d7fa640 1 -- 172.21.15.7:0/788308535 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4270009820 con 0x7f4280102490 2024-06-10T01:14:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.295+0000 7f4285f5a640 1 -- 172.21.15.7:0/788308535 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42801a1e70 con 0x7f4280102490 2024-06-10T01:14:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.295+0000 7f427d7fa640 1 -- 172.21.15.7:0/788308535 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4270004cd0 con 0x7f4280102490 2024-06-10T01:14:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.295+0000 7f4285f5a640 1 -- 172.21.15.7:0/788308535 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42801a2270 con 0x7f4280102490 2024-06-10T01:14:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.297+0000 7f427d7fa640 1 -- 172.21.15.7:0/788308535 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f42700099d0 con 0x7f4280102490 2024-06-10T01:14:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.298+0000 7f4288c67740 1 Processor -- start 2024-06-10T01:14:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.298+0000 7f4288c67740 1 -- start start 2024-06-10T01:14:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.298+0000 7f4288c67740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e41e479010 0x55e41e479430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.298+0000 7f4288c67740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e41e479a00 0x55e41e47e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.298+0000 7f4288c67740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41e47eb70 0x55e41e481000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.298+0000 7f4288c67740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e41e3dc4f0 con 0x55e41e479a00 2024-06-10T01:14:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.298+0000 7f4288c67740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e41e37a870 con 0x55e41e47eb70 2024-06-10T01:14:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.298+0000 7f4288c67740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e41e3d84b0 con 0x55e41e479010 2024-06-10T01:14:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.298+0000 7f427f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e41e479a00 0x55e41e47e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.298+0000 7f427ffff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e41e479010 0x55e41e479430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.298+0000 7f427ffff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e41e479010 0x55e41e479430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:60660/0 (socket says 172.21.15.7:60660) 2024-06-10T01:14:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.298+0000 7f427ffff640 1 -- 172.21.15.7:0/2167467125 learned_addr learned my addr 172.21.15.7:0/2167467125 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.298+0000 7f427f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e41e479a00 0x55e41e47e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60666/0 (socket says 172.21.15.7:60666) 2024-06-10T01:14:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.298+0000 7f427d7fa640 1 --2- 172.21.15.7:0/788308535 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f426c0a2b80 0x7f426c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.298+0000 7f4276ffd640 1 --2- 172.21.15.7:0/2167467125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41e47eb70 0x55e41e481000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.298+0000 7f427ffff640 1 -- 172.21.15.7:0/2167467125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41e47eb70 msgr2=0x55e41e481000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.298+0000 7f427ffff640 1 --2- 172.21.15.7:0/2167467125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41e47eb70 0x55e41e481000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.298+0000 7f427ffff640 1 -- 172.21.15.7:0/2167467125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e41e479a00 msgr2=0x55e41e47e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.298+0000 7f427ffff640 1 --2- 172.21.15.7:0/2167467125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e41e479a00 0x55e41e47e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.298+0000 7f427ffff640 1 -- 172.21.15.7:0/2167467125 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e41e3969b0 con 0x55e41e479010 2024-06-10T01:14:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.298+0000 7f427f7fe640 1 --2- 172.21.15.7:0/2167467125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e41e479a00 0x55e41e47e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.298+0000 7f427d7fa640 1 -- 172.21.15.7:0/788308535 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f42700cee70 con 0x7f4280102490 2024-06-10T01:14:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.299+0000 7f427ffff640 1 --2- 172.21.15.7:0/788308535 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f426c0a2b80 0x7f426c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.299+0000 7f427ffff640 1 --2- 172.21.15.7:0/2167467125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e41e479010 0x55e41e479430 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f427800e9b0 tx=0x7f427800ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8695717460437d1f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.299+0000 7f4275ffb640 1 -- 172.21.15.7:0/2167467125 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4278017040 con 0x55e41e479010 2024-06-10T01:14:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.299+0000 7f4275ffb640 1 -- 172.21.15.7:0/2167467125 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4278019050 con 0x55e41e479010 2024-06-10T01:14:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.299+0000 7f4275ffb640 1 -- 172.21.15.7:0/2167467125 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427800f620 con 0x55e41e479010 2024-06-10T01:14:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.299+0000 7f4288c67740 1 -- 172.21.15.7:0/2167467125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e41e479010 msgr2=0x55e41e479430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.299+0000 7f4288c67740 1 --2- 172.21.15.7:0/2167467125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e41e479010 0x55e41e479430 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f427800e9b0 tx=0x7f427800ee70 comp rx=0 tx=0).stop 2024-06-10T01:14:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.299+0000 7f427ffff640 1 --2- 172.21.15.7:0/788308535 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f426c0a2b80 0x7f426c0a5040 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4278022000 tx=0x7f4278023000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.300+0000 7f4288c67740 1 -- 172.21.15.7:0/2167467125 shutdown_connections 2024-06-10T01:14:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.300+0000 7f4288c67740 1 --2- 172.21.15.7:0/2167467125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41e47eb70 0x55e41e481000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.300+0000 7f4288c67740 1 --2- 172.21.15.7:0/2167467125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e41e479a00 0x55e41e47e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.300+0000 7f4288c67740 1 --2- 172.21.15.7:0/2167467125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e41e479010 0x55e41e479430 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.300+0000 7f4288c67740 1 -- 172.21.15.7:0/2167467125 >> 172.21.15.7:0/2167467125 conn(0x55e41e474680 msgr2=0x55e41e474a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.300+0000 7f4288c67740 1 -- 172.21.15.7:0/2167467125 shutdown_connections 2024-06-10T01:14:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.300+0000 7f4288c67740 1 -- 172.21.15.7:0/2167467125 wait complete. 2024-06-10T01:14:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.300+0000 7f4288c67740 1 -- start start 2024-06-10T01:14:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.300+0000 7f4288c67740 1 Processor -- start 2024-06-10T01:14:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.300+0000 7f4288c67740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e41e479010 0x55e41e483250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.300+0000 7f4288c67740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e41e479a00 0x55e41e483790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.300+0000 7f4288c67740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41e47eb70 0x55e41e48a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.300+0000 7f4288c67740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e41e481b20 con 0x55e41e479a00 2024-06-10T01:14:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.300+0000 7f4288c67740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e41e365ae0 con 0x55e41e47eb70 2024-06-10T01:14:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.300+0000 7f4288c67740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e41e481ca0 con 0x55e41e479010 2024-06-10T01:14:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.300+0000 7f4288c67740 10 client.0 _ll_register_callbacks cb 0x55e41e39e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:14:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.300+0000 7f4276ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e41e479a00 0x55e41e483790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.301+0000 7f427ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41e47eb70 0x55e41e48a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.301+0000 7f4276ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e41e479a00 0x55e41e483790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60674/0 (socket says 172.21.15.7:60674) 2024-06-10T01:14:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.301+0000 7f4276ffd640 1 -- 172.21.15.7:0/2515440889 learned_addr learned my addr 172.21.15.7:0/2515440889 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.301+0000 7f427f7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e41e479010 0x55e41e483250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.301+0000 7f427ffff640 1 -- 172.21.15.7:0/2515440889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e41e479010 msgr2=0x55e41e483250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.301+0000 7f427ffff640 1 --2- 172.21.15.7:0/2515440889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e41e479010 0x55e41e483250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.301+0000 7f427ffff640 1 -- 172.21.15.7:0/2515440889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e41e479a00 msgr2=0x55e41e483790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.301+0000 7f427ffff640 1 --2- 172.21.15.7:0/2515440889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e41e479a00 0x55e41e483790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.301+0000 7f427ffff640 1 -- 172.21.15.7:0/2515440889 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e41e0f0210 con 0x55e41e47eb70 2024-06-10T01:14:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.301+0000 7f427ffff640 1 --2- 172.21.15.7:0/2515440889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41e47eb70 0x55e41e48a7d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4278005c40 tx=0x7f4278009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffa54e30fccf8587 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.301+0000 7f425effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:14:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.301+0000 7f425effd640 1 -- 172.21.15.7:0/2515440889 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4278017050 con 0x55e41e47eb70 2024-06-10T01:14:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.301+0000 7f425effd640 1 -- 172.21.15.7:0/2515440889 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f4278004c10 con 0x55e41e47eb70 2024-06-10T01:14:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.301+0000 7f4288c67740 1 -- 172.21.15.7:0/2515440889 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4278019050 con 0x55e41e47eb70 2024-06-10T01:14:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.301+0000 7f425effd640 1 -- 172.21.15.7:0/2515440889 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4278026e10 con 0x55e41e47eb70 2024-06-10T01:14:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.301+0000 7f4288c67740 10 client.26957 fetch_fsmap learned FSMap version 35 2024-06-10T01:14:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.301+0000 7f4288c67740 1 -- 172.21.15.7:0/2515440889 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e41e1bc620 con 0x55e41e47eb70 2024-06-10T01:14:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.302+0000 7f425effd640 1 -- 172.21.15.7:0/2515440889 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4278019050 con 0x55e41e47eb70 2024-06-10T01:14:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.302+0000 7f425effd640 1 -- 172.21.15.7:0/2515440889 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f427800faf0 con 0x55e41e47eb70 2024-06-10T01:14:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.302+0000 7f4288c67740 10 client.26957 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:14:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.302+0000 7f4288c67740 10 client.26957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:14:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.302+0000 7f4288c67740 20 client.26957 populate_metadata read hostname 'smithi007' 2024-06-10T01:14:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.302+0000 7f4288c67740 1 --2- 172.21.15.7:0/2515440889 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e41e48f000 0x55e41e4913f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.302+0000 7f4288c67740 4 client.26957 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:14:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.302+0000 7f4288c67740 1 -- 172.21.15.7:0/2515440889 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e41e3eff10 con 0x55e41e48f000 2024-06-10T01:14:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.302+0000 7f427f7fe640 1 --2- 172.21.15.7:0/2515440889 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e41e48f000 0x55e41e4913f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.302+0000 7f425effd640 10 client.26957 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.302+0000 7f427f7fe640 1 --2- 172.21.15.7:0/2515440889 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e41e48f000 0x55e41e4913f0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65dc208b964a7b8d server_cookie=226a92a57bd9baf1 in_seq=0 out_seq=0 2024-06-10T01:14:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.303+0000 7f425effd640 1 -- 172.21.15.7:0/2515440889 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e41e3eff10 con 0x55e41e48f000 2024-06-10T01:14:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.303+0000 7f425effd640 10 client.26957 handle_command_reply: tid=0 2024-06-10T01:14:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.303+0000 7f4288c67740 1 client.26957 shutdown 2024-06-10T01:14:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.303+0000 7f4288c67740 1 -- 172.21.15.7:0/2515440889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41e47eb70 msgr2=0x55e41e48a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.303+0000 7f4288c67740 1 --2- 172.21.15.7:0/2515440889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41e47eb70 0x55e41e48a7d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4278005c40 tx=0x7f4278009f60 comp rx=0 tx=0).stop 2024-06-10T01:14:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.303+0000 7f4288c67740 1 -- 172.21.15.7:0/2515440889 shutdown_connections 2024-06-10T01:14:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.303+0000 7f4288c67740 1 --2- 172.21.15.7:0/2515440889 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e41e48f000 0x55e41e4913f0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.303+0000 7f4288c67740 1 --2- 172.21.15.7:0/2515440889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41e47eb70 0x55e41e48a7d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.303+0000 7f4288c67740 1 --2- 172.21.15.7:0/2515440889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e41e479a00 0x55e41e483790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.303+0000 7f425effd640 0 client.26957 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.303+0000 7f4288c67740 1 --2- 172.21.15.7:0/2515440889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e41e479010 0x55e41e483250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.303+0000 7f4288c67740 1 -- 172.21.15.7:0/2515440889 >> 172.21.15.7:0/2515440889 conn(0x55e41e474680 msgr2=0x55e41e4737e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.303+0000 7f4288c67740 1 -- 172.21.15.7:0/2515440889 shutdown_connections 2024-06-10T01:14:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.303+0000 7f4288c67740 1 -- 172.21.15.7:0/2515440889 wait complete. 2024-06-10T01:14:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.304+0000 7f4288c67740 20 client.26957 trim_cache size 0 max 16384 2024-06-10T01:14:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.320+0000 7f4288c67740 1 Processor -- start 2024-06-10T01:14:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.320+0000 7f4288c67740 1 -- start start 2024-06-10T01:14:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.320+0000 7f4288c67740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41e479010 0x55e41e47abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.320+0000 7f4288c67740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e41e479a00 0x55e41e48a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.320+0000 7f4288c67740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e41e47eb70 0x55e41e47b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.320+0000 7f4288c67740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f42801af110 con 0x55e41e47eb70 2024-06-10T01:14:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.320+0000 7f4288c67740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f42801ac980 con 0x55e41e479010 2024-06-10T01:14:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.320+0000 7f4288c67740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f42801aa7d0 con 0x55e41e479a00 2024-06-10T01:14:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.320+0000 7f427ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e41e47eb70 0x55e41e47b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.320+0000 7f427ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e41e47eb70 0x55e41e47b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60682/0 (socket says 172.21.15.7:60682) 2024-06-10T01:14:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.320+0000 7f427ffff640 1 -- 172.21.15.7:0/675668030 learned_addr learned my addr 172.21.15.7:0/675668030 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.320+0000 7f4276ffd640 1 --2- 172.21.15.7:0/675668030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e41e479a00 0x55e41e48a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.320+0000 7f427f7fe640 1 --2- 172.21.15.7:0/675668030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41e479010 0x55e41e47abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.320+0000 7f427ffff640 1 -- 172.21.15.7:0/675668030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e41e479a00 msgr2=0x55e41e48a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.320+0000 7f427ffff640 1 --2- 172.21.15.7:0/675668030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e41e479a00 0x55e41e48a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.320+0000 7f427ffff640 1 -- 172.21.15.7:0/675668030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41e479010 msgr2=0x55e41e47abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.320+0000 7f427ffff640 1 --2- 172.21.15.7:0/675668030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41e479010 0x55e41e47abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.320+0000 7f427ffff640 1 -- 172.21.15.7:0/675668030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e41e4744b0 con 0x55e41e47eb70 2024-06-10T01:14:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.320+0000 7f427ffff640 1 --2- 172.21.15.7:0/675668030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e41e47eb70 0x55e41e47b120 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f427800ff00 tx=0x7f42780021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7354aae7aa24f2e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.320+0000 7f42757fa640 1 -- 172.21.15.7:0/675668030 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4278017040 con 0x55e41e47eb70 2024-06-10T01:14:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.320+0000 7f42757fa640 1 -- 172.21.15.7:0/675668030 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4278026a20 con 0x55e41e47eb70 2024-06-10T01:14:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.321+0000 7f42757fa640 1 -- 172.21.15.7:0/675668030 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4278026620 con 0x55e41e47eb70 2024-06-10T01:14:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.321+0000 7f4288c67740 1 -- 172.21.15.7:0/675668030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e41e47eb70 msgr2=0x55e41e47b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.321+0000 7f4288c67740 1 --2- 172.21.15.7:0/675668030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e41e47eb70 0x55e41e47b120 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f427800ff00 tx=0x7f42780021f0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.321+0000 7f4288c67740 1 -- 172.21.15.7:0/675668030 shutdown_connections 2024-06-10T01:14:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.321+0000 7f4288c67740 1 --2- 172.21.15.7:0/675668030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e41e47eb70 0x55e41e47b120 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.321+0000 7f4288c67740 1 --2- 172.21.15.7:0/675668030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e41e479a00 0x55e41e48a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.321+0000 7f4288c67740 1 --2- 172.21.15.7:0/675668030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41e479010 0x55e41e47abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.321+0000 7f4288c67740 1 -- 172.21.15.7:0/675668030 >> 172.21.15.7:0/675668030 conn(0x55e41e474680 msgr2=0x55e41e483250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.321+0000 7f4288c67740 1 -- 172.21.15.7:0/675668030 shutdown_connections 2024-06-10T01:14:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.321+0000 7f4288c67740 1 -- 172.21.15.7:0/675668030 wait complete. 2024-06-10T01:14:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.321+0000 7f4288c67740 1 -- start start 2024-06-10T01:14:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.322+0000 7f4288c67740 1 Processor -- start 2024-06-10T01:14:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.322+0000 7f4288c67740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41e479010 0x55e41e48a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.322+0000 7f4288c67740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e41e479a00 0x55e41e486ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.322+0000 7f4288c67740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e41e47eb70 0x55e41e487410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.322+0000 7f4288c67740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f42801aae40 con 0x55e41e479a00 2024-06-10T01:14:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.322+0000 7f4288c67740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f42801ac2b0 con 0x55e41e479010 2024-06-10T01:14:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.322+0000 7f4288c67740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f42801ab510 con 0x55e41e47eb70 2024-06-10T01:14:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.322+0000 7f4288c67740 10 client.0 _ll_register_callbacks cb 0x7f42700d2780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:14:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.322+0000 7f4276ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e41e479a00 0x55e41e486ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.322+0000 7f427f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41e479010 0x55e41e48a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.322+0000 7f4276ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e41e479a00 0x55e41e486ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60684/0 (socket says 172.21.15.7:60684) 2024-06-10T01:14:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.322+0000 7f4276ffd640 1 -- 172.21.15.7:0/3708718794 learned_addr learned my addr 172.21.15.7:0/3708718794 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.322+0000 7f427ffff640 1 --2- 172.21.15.7:0/3708718794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e41e47eb70 0x55e41e487410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.322+0000 7f4276ffd640 1 -- 172.21.15.7:0/3708718794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e41e47eb70 msgr2=0x55e41e487410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.322+0000 7f4276ffd640 1 --2- 172.21.15.7:0/3708718794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e41e47eb70 0x55e41e487410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.322+0000 7f4276ffd640 1 -- 172.21.15.7:0/3708718794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41e479010 msgr2=0x55e41e48a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.322+0000 7f4276ffd640 1 --2- 172.21.15.7:0/3708718794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41e479010 0x55e41e48a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.322+0000 7f4276ffd640 1 -- 172.21.15.7:0/3708718794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e41e4879d0 con 0x55e41e479a00 2024-06-10T01:14:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.322+0000 7f427ffff640 1 --2- 172.21.15.7:0/3708718794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e41e47eb70 0x55e41e487410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.322+0000 7f427f7fe640 1 --2- 172.21.15.7:0/3708718794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41e479010 0x55e41e48a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.323+0000 7f4276ffd640 1 --2- 172.21.15.7:0/3708718794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e41e479a00 0x55e41e486ed0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f426802a930 tx=0x7f42680053b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=695bf4f402ef5ed9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.323+0000 7f425effd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:14:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.323+0000 7f425effd640 1 -- 172.21.15.7:0/3708718794 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426804b060 con 0x55e41e479a00 2024-06-10T01:14:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.323+0000 7f4288c67740 1 -- 172.21.15.7:0/3708718794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4278026a20 con 0x55e41e479a00 2024-06-10T01:14:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.323+0000 7f425effd640 1 -- 172.21.15.7:0/3708718794 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f4268004950 con 0x55e41e479a00 2024-06-10T01:14:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.323+0000 7f425effd640 1 -- 172.21.15.7:0/3708718794 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4268035040 con 0x55e41e479a00 2024-06-10T01:14:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.323+0000 7f4288c67740 10 client.17565 fetch_fsmap learned FSMap version 35 2024-06-10T01:14:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.323+0000 7f4288c67740 1 -- 172.21.15.7:0/3708718794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e41e4f8d40 con 0x55e41e479a00 2024-06-10T01:14:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.323+0000 7f425effd640 1 -- 172.21.15.7:0/3708718794 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4278026a20 con 0x55e41e479a00 2024-06-10T01:14:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.323+0000 7f425effd640 1 -- 172.21.15.7:0/3708718794 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f42680035f0 con 0x55e41e479a00 2024-06-10T01:14:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.323+0000 7f4288c67740 10 client.17565 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:14:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.323+0000 7f4288c67740 10 client.17565 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:14:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.323+0000 7f4288c67740 20 client.17565 populate_metadata read hostname 'smithi007' 2024-06-10T01:14:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.323+0000 7f4288c67740 1 --2- 172.21.15.7:0/3708718794 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e41e4f9780 0x55e41e4fbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.323+0000 7f4288c67740 4 client.17565 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:14:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.323+0000 7f4288c67740 1 -- 172.21.15.7:0/3708718794 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e41e3e9c20 con 0x55e41e4f9780 2024-06-10T01:14:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.324+0000 7f427f7fe640 1 --2- 172.21.15.7:0/3708718794 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e41e4f9780 0x55e41e4fbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.324+0000 7f425effd640 10 client.17565 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.324+0000 7f427f7fe640 1 --2- 172.21.15.7:0/3708718794 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e41e4f9780 0x55e41e4fbba0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f4be40841a15a3f server_cookie=65e4fdf3a226b3a6 in_seq=0 out_seq=0 2024-06-10T01:14:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.418+0000 7f425effd640 1 -- 172.21.15.7:0/3708718794 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e41e3e9c20 con 0x55e41e4f9780 2024-06-10T01:14:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.418+0000 7f425effd640 10 client.17565 handle_command_reply: tid=0 2024-06-10T01:14:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.418+0000 7f4288c67740 1 client.17565 shutdown 2024-06-10T01:14:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.418+0000 7f4288c67740 1 -- 172.21.15.7:0/3708718794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e41e479a00 msgr2=0x55e41e486ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.418+0000 7f4288c67740 1 --2- 172.21.15.7:0/3708718794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e41e479a00 0x55e41e486ed0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f426802a930 tx=0x7f42680053b0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.418+0000 7f4288c67740 1 -- 172.21.15.7:0/3708718794 shutdown_connections 2024-06-10T01:14:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.418+0000 7f4288c67740 1 --2- 172.21.15.7:0/3708718794 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e41e4f9780 0x55e41e4fbba0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.418+0000 7f4288c67740 1 --2- 172.21.15.7:0/3708718794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e41e47eb70 0x55e41e487410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.418+0000 7f4288c67740 1 --2- 172.21.15.7:0/3708718794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e41e479a00 0x55e41e486ed0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.418+0000 7f425effd640 0 client.17565 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.418+0000 7f4288c67740 1 --2- 172.21.15.7:0/3708718794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41e479010 0x55e41e48a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.418+0000 7f4288c67740 1 -- 172.21.15.7:0/3708718794 >> 172.21.15.7:0/3708718794 conn(0x55e41e474680 msgr2=0x55e41e483250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.418+0000 7f4288c67740 1 -- 172.21.15.7:0/3708718794 shutdown_connections 2024-06-10T01:14:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.418+0000 7f4288c67740 1 -- 172.21.15.7:0/3708718794 wait complete. 2024-06-10T01:14:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.418+0000 7f4288c67740 20 client.17565 trim_cache size 0 max 16384 2024-06-10T01:14:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:14:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:14:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:14:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.419+0000 7f4285f5a640 1 -- 172.21.15.7:0/788308535 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f426c0a2b80 msgr2=0x7f426c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.419+0000 7f4285f5a640 1 --2- 172.21.15.7:0/788308535 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f426c0a2b80 0x7f426c0a5040 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4278022000 tx=0x7f4278023000 comp rx=0 tx=0).stop 2024-06-10T01:14:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.419+0000 7f4285f5a640 1 -- 172.21.15.7:0/788308535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4280102490 msgr2=0x7f4280101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.419+0000 7f4285f5a640 1 --2- 172.21.15.7:0/788308535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4280102490 0x7f4280101e00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f427000d6d0 tx=0x7f427000db90 comp rx=0 tx=0).stop 2024-06-10T01:14:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.419+0000 7f4285f5a640 1 -- 172.21.15.7:0/788308535 shutdown_connections 2024-06-10T01:14:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.419+0000 7f4285f5a640 1 --2- 172.21.15.7:0/788308535 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f426c0a2b80 0x7f426c0a5040 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.419+0000 7f4285f5a640 1 --2- 172.21.15.7:0/788308535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42801076f0 0x7f4280100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.419+0000 7f4285f5a640 1 --2- 172.21.15.7:0/788308535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4280104dc0 0x7f4280100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.419+0000 7f4285f5a640 1 --2- 172.21.15.7:0/788308535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4280102490 0x7f4280101e00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.419+0000 7f4285f5a640 1 -- 172.21.15.7:0/788308535 >> 172.21.15.7:0/788308535 conn(0x7f42800fbe80 msgr2=0x7f4280102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.419+0000 7f4285f5a640 1 -- 172.21.15.7:0/788308535 shutdown_connections 2024-06-10T01:14:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:26.419+0000 7f4285f5a640 1 -- 172.21.15.7:0/788308535 wait complete. 2024-06-10T01:14:26.741 INFO:tasks.workunit.client.0.smithi007.stdout: 1 610277 7.71 MB/sec execute 307 sec latency 44.534 ms 2024-06-10T01:14:27.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:27 smithi175 ceph-mon[36636]: pgmap v685: 129 pgs: 129 active+clean; 99 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-06-10T01:14:27.741 INFO:tasks.workunit.client.0.smithi007.stdout: 1 611487 7.71 MB/sec execute 308 sec latency 61.768 ms 2024-06-10T01:14:27.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:27 smithi007 ceph-mon[32398]: pgmap v685: 129 pgs: 129 active+clean; 99 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-06-10T01:14:27.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:27 smithi078 ceph-mon[36683]: pgmap v685: 129 pgs: 129 active+clean; 99 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-06-10T01:14:27.954 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:14:28.235 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:14:28.744 INFO:tasks.workunit.client.0.smithi007.stdout: 1 613771 7.72 MB/sec execute 309 sec latency 23.780 ms 2024-06-10T01:14:29.699 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:29 smithi007 ceph-mon[32398]: pgmap v686: 129 pgs: 129 active+clean; 101 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-06-10T01:14:29.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:29 smithi175 ceph-mon[36636]: pgmap v686: 129 pgs: 129 active+clean; 101 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-06-10T01:14:29.741 INFO:tasks.workunit.client.0.smithi007.stdout: 1 615351 7.72 MB/sec execute 310 sec latency 15.634 ms 2024-06-10T01:14:29.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:29 smithi078 ceph-mon[36683]: pgmap v686: 129 pgs: 129 active+clean; 101 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-06-10T01:14:29.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.965+0000 7f9aeeac5700 1 -- 172.21.15.7:0/4024184668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ae8108650 msgr2=0x7f9ae8108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:29.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.965+0000 7f9aeeac5700 1 --2- 172.21.15.7:0/4024184668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ae8108650 0x7f9ae8108a30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9adc009990 tx=0x7f9adc009ca0 comp rx=0 tx=0).stop 2024-06-10T01:14:29.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.965+0000 7f9aeeac5700 1 -- 172.21.15.7:0/4024184668 shutdown_connections 2024-06-10T01:14:29.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.965+0000 7f9aeeac5700 1 --2- 172.21.15.7:0/4024184668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ae806b580 0x7f9ae810f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:29.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.965+0000 7f9aeeac5700 1 --2- 172.21.15.7:0/4024184668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ae8108650 0x7f9ae8108a30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:29.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.965+0000 7f9aeeac5700 1 --2- 172.21.15.7:0/4024184668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ae806abc0 0x7f9ae806b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:29.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.965+0000 7f9aeeac5700 1 -- 172.21.15.7:0/4024184668 >> 172.21.15.7:0/4024184668 conn(0x7f9ae8078160 msgr2=0x7f9ae8078570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:29.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.965+0000 7f9aeeac5700 1 -- 172.21.15.7:0/4024184668 shutdown_connections 2024-06-10T01:14:29.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.965+0000 7f9aeeac5700 1 -- 172.21.15.7:0/4024184668 wait complete. 2024-06-10T01:14:29.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.966+0000 7f9aeeac5700 1 Processor -- start 2024-06-10T01:14:29.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.966+0000 7f9aeeac5700 1 -- start start 2024-06-10T01:14:29.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.966+0000 7f9aeeac5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ae806abc0 0x7f9ae819e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:29.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.966+0000 7f9aeeac5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ae806b580 0x7f9ae819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:29.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.967+0000 7f9aeeac5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ae8108650 0x7f9ae8198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:29.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.967+0000 7f9aeeac5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae81127e0 con 0x7f9ae806b580 2024-06-10T01:14:29.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.967+0000 7f9aeeac5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae8112660 con 0x7f9ae806abc0 2024-06-10T01:14:29.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.967+0000 7f9aeeac5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae8112960 con 0x7f9ae8108650 2024-06-10T01:14:29.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.967+0000 7f9aec861700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ae806abc0 0x7f9ae819e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:29.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.967+0000 7f9aec861700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ae806abc0 0x7f9ae819e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:33276/0 (socket says 172.21.15.7:33276) 2024-06-10T01:14:29.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.967+0000 7f9aec861700 1 -- 172.21.15.7:0/697490396 learned_addr learned my addr 172.21.15.7:0/697490396 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:29.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.967+0000 7f9ae7fff700 1 --2- 172.21.15.7:0/697490396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ae806b580 0x7f9ae819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:29.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.967+0000 7f9aed062700 1 --2- 172.21.15.7:0/697490396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ae8108650 0x7f9ae8198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:29.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.967+0000 7f9aec861700 1 -- 172.21.15.7:0/697490396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ae8108650 msgr2=0x7f9ae8198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:29.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.967+0000 7f9aec861700 1 --2- 172.21.15.7:0/697490396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ae8108650 0x7f9ae8198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:29.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.967+0000 7f9aec861700 1 -- 172.21.15.7:0/697490396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ae806b580 msgr2=0x7f9ae819ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:29.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.967+0000 7f9aec861700 1 --2- 172.21.15.7:0/697490396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ae806b580 0x7f9ae819ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:29.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.967+0000 7f9aec861700 1 -- 172.21.15.7:0/697490396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae81991c0 con 0x7f9ae806abc0 2024-06-10T01:14:29.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.967+0000 7f9aec861700 1 --2- 172.21.15.7:0/697490396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ae806abc0 0x7f9ae819e850 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9adc005910 tx=0x7f9adc00bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:29.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.968+0000 7f9ae5ffb700 1 -- 172.21.15.7:0/697490396 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc027070 con 0x7f9ae806abc0 2024-06-10T01:14:29.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.968+0000 7f9aeeac5700 1 -- 172.21.15.7:0/697490396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ae8199450 con 0x7f9ae806abc0 2024-06-10T01:14:29.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.968+0000 7f9ae5ffb700 1 -- 172.21.15.7:0/697490396 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9adc003d70 con 0x7f9ae806abc0 2024-06-10T01:14:29.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.968+0000 7f9ae5ffb700 1 -- 172.21.15.7:0/697490396 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc00f460 con 0x7f9ae806abc0 2024-06-10T01:14:29.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.968+0000 7f9aeeac5700 1 -- 172.21.15.7:0/697490396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ae81016b0 con 0x7f9ae806abc0 2024-06-10T01:14:29.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.970+0000 7f9ae5ffb700 1 -- 172.21.15.7:0/697490396 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9adc02a070 con 0x7f9ae806abc0 2024-06-10T01:14:29.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.971+0000 7f9ae5ffb700 1 --2- 172.21.15.7:0/697490396 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9ad80a2ca0 0x7f9ad80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:29.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.972+0000 7f9ae5ffb700 1 -- 172.21.15.7:0/697490396 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f9adc0d0120 con 0x7f9ae806abc0 2024-06-10T01:14:29.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.972+0000 7f9ae7fff700 1 --2- 172.21.15.7:0/697490396 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9ad80a2ca0 0x7f9ad80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:29.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.972+0000 7f9ae7fff700 1 --2- 172.21.15.7:0/697490396 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9ad80a2ca0 0x7f9ad80a5160 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80786d0 tx=0x7f9ad4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:29.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.972+0000 7f9af9348b80 1 Processor -- start 2024-06-10T01:14:29.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.972+0000 7f9af9348b80 1 -- start start 2024-06-10T01:14:29.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.973+0000 7f9af9348b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4645aef60 0x55a4645af340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:29.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.973+0000 7f9af9348b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a4645af880 0x55a4645a9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:29.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.973+0000 7f9af9348b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4645a97a0 0x55a4645a9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:29.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.973+0000 7f9af9348b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a46449bf60 con 0x55a4645af880 2024-06-10T01:14:29.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.973+0000 7f9af9348b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a4644fa850 con 0x55a4645a97a0 2024-06-10T01:14:29.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.973+0000 7f9af9348b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a464462460 con 0x55a4645aef60 2024-06-10T01:14:29.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.973+0000 7f9aec861700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4645aef60 0x55a4645af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:29.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.973+0000 7f9ae7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a4645af880 0x55a4645a9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:29.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.973+0000 7f9aec861700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4645aef60 0x55a4645af340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:60730/0 (socket says 172.21.15.7:60730) 2024-06-10T01:14:29.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.973+0000 7f9aec861700 1 -- 172.21.15.7:0/3975234551 learned_addr learned my addr 172.21.15.7:0/3975234551 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.973+0000 7f9aed062700 1 --2- 172.21.15.7:0/3975234551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4645a97a0 0x55a4645a9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.973+0000 7f9aec861700 1 -- 172.21.15.7:0/3975234551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4645a97a0 msgr2=0x55a4645a9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.973+0000 7f9aec861700 1 --2- 172.21.15.7:0/3975234551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4645a97a0 0x55a4645a9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.973+0000 7f9aec861700 1 -- 172.21.15.7:0/3975234551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a4645af880 msgr2=0x55a4645a9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.973+0000 7f9aec861700 1 --2- 172.21.15.7:0/3975234551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a4645af880 0x55a4645a9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.973+0000 7f9aec861700 1 -- 172.21.15.7:0/3975234551 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4644498d0 con 0x55a4645aef60 2024-06-10T01:14:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.973+0000 7f9aec861700 1 --2- 172.21.15.7:0/3975234551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4645aef60 0x55a4645af340 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9adc099000 tx=0x7f9adc009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa6f6fc04a684983 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.973+0000 7f9ad2ffd700 1 -- 172.21.15.7:0/3975234551 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc09bf90 con 0x55a4645aef60 2024-06-10T01:14:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.974+0000 7f9ad2ffd700 1 -- 172.21.15.7:0/3975234551 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9adc0043d0 con 0x55a4645aef60 2024-06-10T01:14:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.974+0000 7f9ad2ffd700 1 -- 172.21.15.7:0/3975234551 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc09a930 con 0x55a4645aef60 2024-06-10T01:14:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.974+0000 7f9af9348b80 1 -- 172.21.15.7:0/3975234551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4645aef60 msgr2=0x55a4645af340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.974+0000 7f9af9348b80 1 --2- 172.21.15.7:0/3975234551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4645aef60 0x55a4645af340 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9adc099000 tx=0x7f9adc009ca0 comp rx=0 tx=0).stop 2024-06-10T01:14:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.975+0000 7f9af9348b80 1 -- 172.21.15.7:0/3975234551 shutdown_connections 2024-06-10T01:14:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.975+0000 7f9af9348b80 1 --2- 172.21.15.7:0/3975234551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4645a97a0 0x55a4645a9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:29.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.975+0000 7f9af9348b80 1 --2- 172.21.15.7:0/3975234551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4645aef60 0x55a4645af340 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:29.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.975+0000 7f9af9348b80 1 --2- 172.21.15.7:0/3975234551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a4645af880 0x55a4645a9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:29.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.975+0000 7f9af9348b80 1 -- 172.21.15.7:0/3975234551 >> 172.21.15.7:0/3975234551 conn(0x55a4644d6080 msgr2=0x55a4645a4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:29.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.975+0000 7f9af9348b80 1 -- 172.21.15.7:0/3975234551 shutdown_connections 2024-06-10T01:14:29.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.975+0000 7f9af9348b80 1 -- 172.21.15.7:0/3975234551 wait complete. 2024-06-10T01:14:29.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.975+0000 7f9af9348b80 1 -- start start 2024-06-10T01:14:29.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.975+0000 7f9af9348b80 1 Processor -- start 2024-06-10T01:14:29.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.976+0000 7f9af9348b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4645a97a0 0x55a4645b5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:29.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.976+0000 7f9af9348b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4645aef60 0x55a4645b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:29.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.976+0000 7f9af9348b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a4645af880 0x55a4645b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:29.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.976+0000 7f9af9348b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a46449afc0 con 0x55a4645af880 2024-06-10T01:14:29.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.976+0000 7f9af9348b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a4644f6810 con 0x55a4645a97a0 2024-06-10T01:14:29.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.976+0000 7f9af9348b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a4645b6010 con 0x55a4645aef60 2024-06-10T01:14:29.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.976+0000 7f9af9348b80 10 client.0 _ll_register_callbacks cb 0x55a4644905b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:14:29.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.976+0000 7f9aed062700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a4645af880 0x55a4645b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:29.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.976+0000 7f9ae7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4645aef60 0x55a4645b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:29.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.976+0000 7f9aec861700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4645a97a0 0x55a4645b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:29.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.976+0000 7f9ae7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4645aef60 0x55a4645b29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:60738/0 (socket says 172.21.15.7:60738) 2024-06-10T01:14:29.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.976+0000 7f9aec861700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4645a97a0 0x55a4645b5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:33292/0 (socket says 172.21.15.7:33292) 2024-06-10T01:14:29.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.976+0000 7f9ae7fff700 1 -- 172.21.15.7:0/3477845668 learned_addr learned my addr 172.21.15.7:0/3477845668 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:29.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.976+0000 7f9aed062700 1 -- 172.21.15.7:0/3477845668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4645aef60 msgr2=0x55a4645b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:29.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.976+0000 7f9aed062700 1 --2- 172.21.15.7:0/3477845668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4645aef60 0x55a4645b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:29.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.976+0000 7f9aed062700 1 -- 172.21.15.7:0/3477845668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4645a97a0 msgr2=0x55a4645b5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:29.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.976+0000 7f9aed062700 1 --2- 172.21.15.7:0/3477845668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4645a97a0 0x55a4645b5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:29.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.976+0000 7f9aed062700 1 -- 172.21.15.7:0/3477845668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4644c0750 con 0x55a4645af880 2024-06-10T01:14:29.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.976+0000 7f9ae7fff700 1 --2- 172.21.15.7:0/3477845668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4645aef60 0x55a4645b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:29.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.976+0000 7f9aec861700 1 --2- 172.21.15.7:0/3477845668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4645a97a0 0x55a4645b5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:29.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.976+0000 7f9aed062700 1 --2- 172.21.15.7:0/3477845668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a4645af880 0x55a4645b2f20 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0007b10 tx=0x7f9ae0007b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa37e2dfcd4b255 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:29.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.976+0000 7f9ac7fff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:14:29.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.976+0000 7f9ac7fff700 1 -- 172.21.15.7:0/3477845668 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae001a030 con 0x55a4645af880 2024-06-10T01:14:29.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.976+0000 7f9af9348b80 1 -- 172.21.15.7:0/3477845668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9adc0043d0 con 0x55a4645af880 2024-06-10T01:14:29.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.976+0000 7f9ac7fff700 1 -- 172.21.15.7:0/3477845668 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f9ae0016700 con 0x55a4645af880 2024-06-10T01:14:29.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.977+0000 7f9ac7fff700 1 -- 172.21.15.7:0/3477845668 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9ae0027050 con 0x55a4645af880 2024-06-10T01:14:29.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.977+0000 7f9af9348b80 10 client.17577 fetch_fsmap learned FSMap version 35 2024-06-10T01:14:29.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.977+0000 7f9af9348b80 1 -- 172.21.15.7:0/3477845668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a4645c08a0 con 0x55a4645af880 2024-06-10T01:14:29.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.977+0000 7f9ac7fff700 1 -- 172.21.15.7:0/3477845668 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9adc0043d0 con 0x55a4645af880 2024-06-10T01:14:29.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.977+0000 7f9ac7fff700 1 -- 172.21.15.7:0/3477845668 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f9ae001fb10 con 0x55a4645af880 2024-06-10T01:14:29.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.977+0000 7f9af9348b80 10 client.17577 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:14:29.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.977+0000 7f9af9348b80 10 client.17577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:14:29.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.977+0000 7f9af9348b80 20 client.17577 populate_metadata read hostname 'smithi007' 2024-06-10T01:14:29.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.977+0000 7f9af9348b80 1 --2- 172.21.15.7:0/3477845668 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a4645c2e80 0x55a4645c5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:29.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.977+0000 7f9af9348b80 4 client.17577 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:14:29.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.977+0000 7f9af9348b80 1 -- 172.21.15.7:0/3477845668 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a464507d50 con 0x55a4645c2e80 2024-06-10T01:14:29.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.977+0000 7f9aec861700 1 --2- 172.21.15.7:0/3477845668 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a4645c2e80 0x55a4645c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:29.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.978+0000 7f9aec861700 1 --2- 172.21.15.7:0/3477845668 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a4645c2e80 0x55a4645c5270 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb86a03638e952dd server_cookie=119482fa13f41df3 in_seq=0 out_seq=0 2024-06-10T01:14:29.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.978+0000 7f9ac7fff700 10 client.17577 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:29.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.978+0000 7f9ac7fff700 1 -- 172.21.15.7:0/3477845668 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a464507d50 con 0x55a4645c2e80 2024-06-10T01:14:29.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.978+0000 7f9ac7fff700 10 client.17577 handle_command_reply: tid=0 2024-06-10T01:14:29.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.978+0000 7f9af9348b80 1 client.17577 shutdown 2024-06-10T01:14:29.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.979+0000 7f9af9348b80 1 -- 172.21.15.7:0/3477845668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a4645af880 msgr2=0x55a4645b2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:29.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.979+0000 7f9af9348b80 1 --2- 172.21.15.7:0/3477845668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a4645af880 0x55a4645b2f20 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0007b10 tx=0x7f9ae0007b40 comp rx=0 tx=0).stop 2024-06-10T01:14:29.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.979+0000 7f9af9348b80 1 -- 172.21.15.7:0/3477845668 shutdown_connections 2024-06-10T01:14:29.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.979+0000 7f9af9348b80 1 --2- 172.21.15.7:0/3477845668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a4645af880 0x55a4645b2f20 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:29.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.979+0000 7f9af9348b80 1 --2- 172.21.15.7:0/3477845668 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a4645c2e80 0x55a4645c5270 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:29.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.979+0000 7f9ac7fff700 0 client.17577 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:29.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.979+0000 7f9af9348b80 1 --2- 172.21.15.7:0/3477845668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4645a97a0 0x55a4645b5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:29.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.979+0000 7f9af9348b80 1 --2- 172.21.15.7:0/3477845668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4645aef60 0x55a4645b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:29.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.979+0000 7f9af9348b80 1 -- 172.21.15.7:0/3477845668 >> 172.21.15.7:0/3477845668 conn(0x55a4644d6080 msgr2=0x55a4645a3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:29.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.979+0000 7f9af9348b80 1 -- 172.21.15.7:0/3477845668 shutdown_connections 2024-06-10T01:14:29.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.979+0000 7f9af9348b80 1 -- 172.21.15.7:0/3477845668 wait complete. 2024-06-10T01:14:29.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:29.979+0000 7f9af9348b80 20 client.17577 trim_cache size 0 max 16384 2024-06-10T01:14:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.002+0000 7f9af9348b80 1 Processor -- start 2024-06-10T01:14:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.002+0000 7f9af9348b80 1 -- start start 2024-06-10T01:14:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.002+0000 7f9af9348b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4645a97a0 0x55a4645ae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.002+0000 7f9af9348b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4645aef60 0x55a4645b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.002+0000 7f9af9348b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a4645af880 0x55a4645a3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.002+0000 7f9af9348b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae81b2c50 con 0x55a4645af880 2024-06-10T01:14:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.002+0000 7f9af9348b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae81b0480 con 0x55a4645a97a0 2024-06-10T01:14:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.002+0000 7f9af9348b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae81ae2d0 con 0x55a4645aef60 2024-06-10T01:14:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.003+0000 7f9ae7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4645aef60 0x55a4645b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.003+0000 7f9ae7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4645aef60 0x55a4645b51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:60740/0 (socket says 172.21.15.7:60740) 2024-06-10T01:14:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.003+0000 7f9aed062700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a4645af880 0x55a4645a3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.003+0000 7f9ae7fff700 1 -- 172.21.15.7:0/2677750177 learned_addr learned my addr 172.21.15.7:0/2677750177 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.003+0000 7f9aec861700 1 --2- 172.21.15.7:0/2677750177 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4645a97a0 0x55a4645ae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.003+0000 7f9ae7fff700 1 -- 172.21.15.7:0/2677750177 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4645a97a0 msgr2=0x55a4645ae1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.003+0000 7f9ae7fff700 1 --2- 172.21.15.7:0/2677750177 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4645a97a0 0x55a4645ae1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.003+0000 7f9ae7fff700 1 -- 172.21.15.7:0/2677750177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a4645af880 msgr2=0x55a4645a3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.003+0000 7f9ae7fff700 1 --2- 172.21.15.7:0/2677750177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a4645af880 0x55a4645a3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.003+0000 7f9ae7fff700 1 -- 172.21.15.7:0/2677750177 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4645a0810 con 0x55a4645aef60 2024-06-10T01:14:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.003+0000 7f9aec861700 1 --2- 172.21.15.7:0/2677750177 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4645a97a0 0x55a4645ae1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:14:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.003+0000 7f9ae7fff700 1 --2- 172.21.15.7:0/2677750177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4645aef60 0x55a4645b51d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4004200 tx=0x7f9ad4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9eb233678fd38264 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.003+0000 7f9ad27fc700 1 -- 172.21.15.7:0/2677750177 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4003a10 con 0x55a4645aef60 2024-06-10T01:14:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.004+0000 7f9ad27fc700 1 -- 172.21.15.7:0/2677750177 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ad4004d10 con 0x55a4645aef60 2024-06-10T01:14:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.004+0000 7f9ad27fc700 1 -- 172.21.15.7:0/2677750177 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad401e420 con 0x55a4645aef60 2024-06-10T01:14:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.004+0000 7f9af9348b80 1 -- 172.21.15.7:0/2677750177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4645aef60 msgr2=0x55a4645b51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.004+0000 7f9af9348b80 1 --2- 172.21.15.7:0/2677750177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4645aef60 0x55a4645b51d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4004200 tx=0x7f9ad4014000 comp rx=0 tx=0).stop 2024-06-10T01:14:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.004+0000 7f9af9348b80 1 -- 172.21.15.7:0/2677750177 shutdown_connections 2024-06-10T01:14:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.004+0000 7f9af9348b80 1 --2- 172.21.15.7:0/2677750177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a4645af880 0x55a4645a3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.004+0000 7f9af9348b80 1 --2- 172.21.15.7:0/2677750177 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4645a97a0 0x55a4645ae1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.004+0000 7f9af9348b80 1 --2- 172.21.15.7:0/2677750177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4645aef60 0x55a4645b51d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.004+0000 7f9af9348b80 1 -- 172.21.15.7:0/2677750177 >> 172.21.15.7:0/2677750177 conn(0x55a4644d6080 msgr2=0x55a4645adaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.004+0000 7f9af9348b80 1 -- 172.21.15.7:0/2677750177 shutdown_connections 2024-06-10T01:14:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.004+0000 7f9af9348b80 1 -- 172.21.15.7:0/2677750177 wait complete. 2024-06-10T01:14:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.005+0000 7f9af9348b80 1 -- start start 2024-06-10T01:14:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.005+0000 7f9af9348b80 1 Processor -- start 2024-06-10T01:14:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.005+0000 7f9af9348b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a4645a97a0 0x55a4645bb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.005+0000 7f9af9348b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4645aef60 0x55a4645bb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.005+0000 7f9af9348b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4645af880 0x55a4645b8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.006+0000 7f9af9348b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae81ae940 con 0x55a4645a97a0 2024-06-10T01:14:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.006+0000 7f9aec861700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a4645a97a0 0x55a4645bb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.006+0000 7f9af9348b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae81afdb0 con 0x55a4645aef60 2024-06-10T01:14:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.006+0000 7f9ae7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4645aef60 0x55a4645bb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.006+0000 7f9af9348b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae81af010 con 0x55a4645af880 2024-06-10T01:14:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.006+0000 7f9ae7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4645aef60 0x55a4645bb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:33314/0 (socket says 172.21.15.7:33314) 2024-06-10T01:14:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.006+0000 7f9ae7fff700 1 -- 172.21.15.7:0/2111140255 learned_addr learned my addr 172.21.15.7:0/2111140255 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.006+0000 7f9aec861700 1 --2- 172.21.15.7:0/2111140255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a4645a97a0 0x55a4645bb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60726/0 (socket says 172.21.15.7:60726) 2024-06-10T01:14:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.006+0000 7f9aed062700 1 --2- 172.21.15.7:0/2111140255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4645af880 0x55a4645b8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.006+0000 7f9af9348b80 10 client.0 _ll_register_callbacks cb 0x55a4644905b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:14:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.006+0000 7f9ae7fff700 1 -- 172.21.15.7:0/2111140255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4645af880 msgr2=0x55a4645b8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.006+0000 7f9ae7fff700 1 --2- 172.21.15.7:0/2111140255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4645af880 0x55a4645b8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.006+0000 7f9ae7fff700 1 -- 172.21.15.7:0/2111140255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a4645a97a0 msgr2=0x55a4645bb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.006+0000 7f9ae7fff700 1 --2- 172.21.15.7:0/2111140255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a4645a97a0 0x55a4645bb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.006+0000 7f9ae7fff700 1 -- 172.21.15.7:0/2111140255 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4645b87f0 con 0x55a4645aef60 2024-06-10T01:14:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.006+0000 7f9aed062700 1 --2- 172.21.15.7:0/2111140255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4645af880 0x55a4645b8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.006+0000 7f9ae7fff700 1 --2- 172.21.15.7:0/2111140255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4645aef60 0x55a4645bb7f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40117d0 tx=0x7f9ad4003f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82fde92dcbe287 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.006+0000 7f9ac7fff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:14:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.006+0000 7f9ac7fff700 1 -- 172.21.15.7:0/2111140255 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4004ec0 con 0x55a4645aef60 2024-06-10T01:14:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.007+0000 7f9ac7fff700 1 -- 172.21.15.7:0/2111140255 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f9ad4025070 con 0x55a4645aef60 2024-06-10T01:14:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.007+0000 7f9af9348b80 1 -- 172.21.15.7:0/2111140255 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ad4004d10 con 0x55a4645aef60 2024-06-10T01:14:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.007+0000 7f9ac7fff700 1 -- 172.21.15.7:0/2111140255 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9ad4003c20 con 0x55a4645aef60 2024-06-10T01:14:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.007+0000 7f9af9348b80 10 client.26978 fetch_fsmap learned FSMap version 35 2024-06-10T01:14:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.007+0000 7f9af9348b80 1 -- 172.21.15.7:0/2111140255 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a46462e2c0 con 0x55a4645aef60 2024-06-10T01:14:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.007+0000 7f9ac7fff700 1 -- 172.21.15.7:0/2111140255 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ad4004d10 con 0x55a4645aef60 2024-06-10T01:14:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.007+0000 7f9ac7fff700 1 -- 172.21.15.7:0/2111140255 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f9ad40299c0 con 0x55a4645aef60 2024-06-10T01:14:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.007+0000 7f9af9348b80 10 client.26978 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:14:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.007+0000 7f9af9348b80 10 client.26978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:14:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.007+0000 7f9af9348b80 20 client.26978 populate_metadata read hostname 'smithi007' 2024-06-10T01:14:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.007+0000 7f9af9348b80 1 --2- 172.21.15.7:0/2111140255 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a46462e8b0 0x55a464630cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.007+0000 7f9af9348b80 4 client.26978 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:14:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.007+0000 7f9af9348b80 1 -- 172.21.15.7:0/2111140255 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a4645022d0 con 0x55a46462e8b0 2024-06-10T01:14:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.008+0000 7f9aec861700 1 --2- 172.21.15.7:0/2111140255 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a46462e8b0 0x55a464630cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:30.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.008+0000 7f9aec861700 1 --2- 172.21.15.7:0/2111140255 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a46462e8b0 0x55a464630cd0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8791c54fc81d0e15 server_cookie=8d3d709ec46605bc in_seq=0 out_seq=0 2024-06-10T01:14:30.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.008+0000 7f9ac7fff700 10 client.26978 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:30.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.008+0000 7f9ac7fff700 1 -- 172.21.15.7:0/2111140255 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a4645022d0 con 0x55a46462e8b0 2024-06-10T01:14:30.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.008+0000 7f9ac7fff700 10 client.26978 handle_command_reply: tid=0 2024-06-10T01:14:30.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.008+0000 7f9af9348b80 1 client.26978 shutdown 2024-06-10T01:14:30.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.008+0000 7f9af9348b80 1 -- 172.21.15.7:0/2111140255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4645aef60 msgr2=0x55a4645bb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:30.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.008+0000 7f9af9348b80 1 --2- 172.21.15.7:0/2111140255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4645aef60 0x55a4645bb7f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40117d0 tx=0x7f9ad4003f50 comp rx=0 tx=0).stop 2024-06-10T01:14:30.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.008+0000 7f9af9348b80 1 -- 172.21.15.7:0/2111140255 shutdown_connections 2024-06-10T01:14:30.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.008+0000 7f9af9348b80 1 --2- 172.21.15.7:0/2111140255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a4645af880 0x55a4645b8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:30.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.008+0000 7f9af9348b80 1 --2- 172.21.15.7:0/2111140255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a4645a97a0 0x55a4645bb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:30.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.008+0000 7f9af9348b80 1 --2- 172.21.15.7:0/2111140255 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a46462e8b0 0x55a464630cd0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:30.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.008+0000 7f9af9348b80 1 --2- 172.21.15.7:0/2111140255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4645aef60 0x55a4645bb7f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:30.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.008+0000 7f9ac7fff700 0 client.26978 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:30.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.008+0000 7f9af9348b80 1 -- 172.21.15.7:0/2111140255 >> 172.21.15.7:0/2111140255 conn(0x55a4644d6080 msgr2=0x55a4645adaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:30.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.009+0000 7f9af9348b80 1 -- 172.21.15.7:0/2111140255 shutdown_connections 2024-06-10T01:14:30.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.009+0000 7f9af9348b80 1 -- 172.21.15.7:0/2111140255 wait complete. 2024-06-10T01:14:30.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.009+0000 7f9af9348b80 20 client.26978 trim_cache size 0 max 16384 2024-06-10T01:14:30.009 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:14:30.009 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-10T01:14:30.009 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-10T01:14:30.009 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:14:30.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.009+0000 7f9aeeac5700 1 -- 172.21.15.7:0/697490396 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9ad80a2ca0 msgr2=0x7f9ad80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:30.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.009+0000 7f9aeeac5700 1 --2- 172.21.15.7:0/697490396 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9ad80a2ca0 0x7f9ad80a5160 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80786d0 tx=0x7f9ad4009260 comp rx=0 tx=0).stop 2024-06-10T01:14:30.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.009+0000 7f9aeeac5700 1 -- 172.21.15.7:0/697490396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ae806abc0 msgr2=0x7f9ae819e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:30.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.009+0000 7f9aeeac5700 1 --2- 172.21.15.7:0/697490396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ae806abc0 0x7f9ae819e850 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9adc005910 tx=0x7f9adc00bab0 comp rx=0 tx=0).stop 2024-06-10T01:14:30.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.009+0000 7f9aeeac5700 1 -- 172.21.15.7:0/697490396 shutdown_connections 2024-06-10T01:14:30.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.009+0000 7f9aeeac5700 1 --2- 172.21.15.7:0/697490396 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9ad80a2ca0 0x7f9ad80a5160 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:30.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.009+0000 7f9aeeac5700 1 --2- 172.21.15.7:0/697490396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ae8108650 0x7f9ae8198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:30.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.009+0000 7f9aeeac5700 1 --2- 172.21.15.7:0/697490396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ae806abc0 0x7f9ae819e850 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:30.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.009+0000 7f9aeeac5700 1 --2- 172.21.15.7:0/697490396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ae806b580 0x7f9ae819ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:30.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.009+0000 7f9aeeac5700 1 -- 172.21.15.7:0/697490396 >> 172.21.15.7:0/697490396 conn(0x7f9ae8078160 msgr2=0x7f9ae8100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:30.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.010+0000 7f9aeeac5700 1 -- 172.21.15.7:0/697490396 shutdown_connections 2024-06-10T01:14:30.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:30.010+0000 7f9aeeac5700 1 -- 172.21.15.7:0/697490396 wait complete. 2024-06-10T01:14:30.697 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:14:30.698 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:14:30.741 INFO:tasks.workunit.client.0.smithi007.stdout: 1 617949 7.75 MB/sec execute 311 sec latency 46.710 ms 2024-06-10T01:14:30.966 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:14:31.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:31 smithi175 ceph-mon[36636]: pgmap v687: 129 pgs: 129 active+clean; 102 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 2.6 MiB/s wr, 266 op/s 2024-06-10T01:14:31.741 INFO:tasks.workunit.client.0.smithi007.stdout: 1 619408 7.74 MB/sec execute 312 sec latency 21.591 ms 2024-06-10T01:14:31.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:31 smithi007 ceph-mon[32398]: pgmap v687: 129 pgs: 129 active+clean; 102 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 2.6 MiB/s wr, 266 op/s 2024-06-10T01:14:31.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:31 smithi078 ceph-mon[36683]: pgmap v687: 129 pgs: 129 active+clean; 102 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 2.6 MiB/s wr, 266 op/s 2024-06-10T01:14:32.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.720+0000 7f02963b2700 1 -- 172.21.15.7:0/1121307519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f029006a1a0 msgr2=0x7f029006a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:32.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.720+0000 7f02963b2700 1 --2- 172.21.15.7:0/1121307519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f029006a1a0 0x7f029006a580 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f0280009990 tx=0x7f0280009ca0 comp rx=0 tx=0).stop 2024-06-10T01:14:32.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.720+0000 7f02963b2700 1 -- 172.21.15.7:0/1121307519 shutdown_connections 2024-06-10T01:14:32.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.720+0000 7f02963b2700 1 --2- 172.21.15.7:0/1121307519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f029010ef40 0x7f0290111330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:32.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.720+0000 7f02963b2700 1 --2- 172.21.15.7:0/1121307519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f029006a1a0 0x7f029006a580 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:32.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.720+0000 7f02963b2700 1 --2- 172.21.15.7:0/1121307519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f029006aac0 0x7f029010ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:32.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.720+0000 7f02963b2700 1 -- 172.21.15.7:0/1121307519 >> 172.21.15.7:0/1121307519 conn(0x7f0290078340 msgr2=0x7f0290078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:32.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.720+0000 7f02963b2700 1 -- 172.21.15.7:0/1121307519 shutdown_connections 2024-06-10T01:14:32.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.720+0000 7f02963b2700 1 -- 172.21.15.7:0/1121307519 wait complete. 2024-06-10T01:14:32.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.721+0000 7f02963b2700 1 Processor -- start 2024-06-10T01:14:32.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.721+0000 7f02963b2700 1 -- start start 2024-06-10T01:14:32.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.721+0000 7f02963b2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f029006a1a0 0x7f029019aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:32.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.721+0000 7f02963b2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f029006aac0 0x7f029019b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:32.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.721+0000 7f02963b2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f029010ef40 0x7f029019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:32.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.721+0000 7f02963b2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f02901149b0 con 0x7f029010ef40 2024-06-10T01:14:32.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.721+0000 7f02963b2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0290114830 con 0x7f029006aac0 2024-06-10T01:14:32.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.721+0000 7f02963b2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0290114b30 con 0x7f029006a1a0 2024-06-10T01:14:32.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.721+0000 7f029494f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f029010ef40 0x7f029019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:32.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.721+0000 7f029494f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f029010ef40 0x7f029019f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60840/0 (socket says 172.21.15.7:60840) 2024-06-10T01:14:32.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.721+0000 7f029494f700 1 -- 172.21.15.7:0/1899387197 learned_addr learned my addr 172.21.15.7:0/1899387197 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:32.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.721+0000 7f028f7fe700 1 --2- 172.21.15.7:0/1899387197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f029006aac0 0x7f029019b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:32.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.721+0000 7f028ffff700 1 --2- 172.21.15.7:0/1899387197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f029006a1a0 0x7f029019aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:32.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.722+0000 7f029494f700 1 -- 172.21.15.7:0/1899387197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f029006a1a0 msgr2=0x7f029019aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:32.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.722+0000 7f029494f700 1 --2- 172.21.15.7:0/1899387197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f029006a1a0 0x7f029019aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:32.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.722+0000 7f029494f700 1 -- 172.21.15.7:0/1899387197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f029006aac0 msgr2=0x7f029019b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:32.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.722+0000 7f029494f700 1 --2- 172.21.15.7:0/1899387197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f029006aac0 0x7f029019b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:32.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.722+0000 7f029494f700 1 -- 172.21.15.7:0/1899387197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f029019fe80 con 0x7f029010ef40 2024-06-10T01:14:32.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.722+0000 7f028f7fe700 1 --2- 172.21.15.7:0/1899387197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f029006aac0 0x7f029019b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:32.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.722+0000 7f029494f700 1 --2- 172.21.15.7:0/1899387197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f029010ef40 0x7f029019f740 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f028800ef90 tx=0x7f028800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:32.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.722+0000 7f028d7fa700 1 -- 172.21.15.7:0/1899387197 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02880092e0 con 0x7f029010ef40 2024-06-10T01:14:32.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.722+0000 7f02963b2700 1 -- 172.21.15.7:0/1899387197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02901a0110 con 0x7f029010ef40 2024-06-10T01:14:32.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.722+0000 7f028d7fa700 1 -- 172.21.15.7:0/1899387197 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0288004bb0 con 0x7f029010ef40 2024-06-10T01:14:32.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.722+0000 7f028d7fa700 1 -- 172.21.15.7:0/1899387197 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0288004280 con 0x7f029010ef40 2024-06-10T01:14:32.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.722+0000 7f02963b2700 1 -- 172.21.15.7:0/1899387197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02901a79f0 con 0x7f029010ef40 2024-06-10T01:14:32.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.723+0000 7f02963b2700 1 -- 172.21.15.7:0/1899387197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0290069180 con 0x7f029010ef40 2024-06-10T01:14:32.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.724+0000 7f028d7fa700 1 -- 172.21.15.7:0/1899387197 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f028801b030 con 0x7f029010ef40 2024-06-10T01:14:32.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.725+0000 7f028d7fa700 1 --2- 172.21.15.7:0/1899387197 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f02780a2c20 0x7f02780a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:32.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.725+0000 7f028ffff700 1 --2- 172.21.15.7:0/1899387197 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f02780a2c20 0x7f02780a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:32.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.725+0000 7f028d7fa700 1 -- 172.21.15.7:0/1899387197 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f02880ceee0 con 0x7f029010ef40 2024-06-10T01:14:32.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.726+0000 7f028ffff700 1 --2- 172.21.15.7:0/1899387197 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f02780a2c20 0x7f02780a50e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f028000b2e0 tx=0x7f0280005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:32.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:32.730+0000 7f028d7fa700 1 -- 172.21.15.7:0/1899387197 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0288022020 con 0x7f029010ef40 2024-06-10T01:14:32.741 INFO:tasks.workunit.client.0.smithi007.stdout: 1 621347 7.75 MB/sec execute 313 sec latency 90.405 ms 2024-06-10T01:14:33.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:33.006+0000 7f02963b2700 1 -- 172.21.15.7:0/1899387197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f029002dca0 con 0x7f029010ef40 2024-06-10T01:14:33.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:33.007+0000 7f028d7fa700 1 -- 172.21.15.7:0/1899387197 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11859 (secure 0 0 0) 0x7f0288010040 con 0x7f029010ef40 2024-06-10T01:14:33.001 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:14:33.002 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:13:22.220710+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:14:33.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:33.009+0000 7f02963b2700 1 -- 172.21.15.7:0/1899387197 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f02780a2c20 msgr2=0x7f02780a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:33.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:33.009+0000 7f02963b2700 1 --2- 172.21.15.7:0/1899387197 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f02780a2c20 0x7f02780a50e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f028000b2e0 tx=0x7f0280005960 comp rx=0 tx=0).stop 2024-06-10T01:14:33.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:33.009+0000 7f02963b2700 1 -- 172.21.15.7:0/1899387197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f029010ef40 msgr2=0x7f029019f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:33.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:33.009+0000 7f02963b2700 1 --2- 172.21.15.7:0/1899387197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f029010ef40 0x7f029019f740 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f028800ef90 tx=0x7f028800c3a0 comp rx=0 tx=0).stop 2024-06-10T01:14:33.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:33.009+0000 7f02963b2700 1 -- 172.21.15.7:0/1899387197 shutdown_connections 2024-06-10T01:14:33.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:33.009+0000 7f02963b2700 1 --2- 172.21.15.7:0/1899387197 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f02780a2c20 0x7f02780a50e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:33.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:33.009+0000 7f02963b2700 1 --2- 172.21.15.7:0/1899387197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f029010ef40 0x7f029019f740 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:33.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:33.009+0000 7f02963b2700 1 --2- 172.21.15.7:0/1899387197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f029006a1a0 0x7f029019aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:33.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:33.010+0000 7f02963b2700 1 --2- 172.21.15.7:0/1899387197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f029006aac0 0x7f029019b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:33.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:33.010+0000 7f02963b2700 1 -- 172.21.15.7:0/1899387197 >> 172.21.15.7:0/1899387197 conn(0x7f0290078340 msgr2=0x7f0290110390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:33.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:33.010+0000 7f02963b2700 1 -- 172.21.15.7:0/1899387197 shutdown_connections 2024-06-10T01:14:33.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:33.010+0000 7f02963b2700 1 -- 172.21.15.7:0/1899387197 wait complete. 2024-06-10T01:14:33.005 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 35 2024-06-10T01:14:33.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:33 smithi175 ceph-mon[36636]: pgmap v688: 129 pgs: 129 active+clean; 104 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 2.5 MiB/s wr, 294 op/s 2024-06-10T01:14:33.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:33 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1899387197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:14:33.741 INFO:tasks.workunit.client.0.smithi007.stdout: 1 622507 7.74 MB/sec execute 314 sec latency 98.357 ms 2024-06-10T01:14:33.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:33 smithi007 ceph-mon[32398]: pgmap v688: 129 pgs: 129 active+clean; 104 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 2.5 MiB/s wr, 294 op/s 2024-06-10T01:14:33.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:33 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1899387197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:14:33.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:33 smithi078 ceph-mon[36683]: pgmap v688: 129 pgs: 129 active+clean; 104 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 2.5 MiB/s wr, 294 op/s 2024-06-10T01:14:33.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:33 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1899387197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:14:34.376 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:14:34.693 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:14:34.741 INFO:tasks.workunit.client.0.smithi007.stdout: 1 625178 7.77 MB/sec execute 315 sec latency 16.052 ms 2024-06-10T01:14:35.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:35 smithi175 ceph-mon[36636]: pgmap v689: 129 pgs: 129 active+clean; 104 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 2.8 MiB/s wr, 303 op/s 2024-06-10T01:14:35.741 INFO:tasks.workunit.client.0.smithi007.stdout: 1 626768 7.77 MB/sec execute 316 sec latency 15.437 ms 2024-06-10T01:14:35.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:35 smithi007 ceph-mon[32398]: pgmap v689: 129 pgs: 129 active+clean; 104 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 2.8 MiB/s wr, 303 op/s 2024-06-10T01:14:35.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:35 smithi078 ceph-mon[36683]: pgmap v689: 129 pgs: 129 active+clean; 104 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 2.8 MiB/s wr, 303 op/s 2024-06-10T01:14:36.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.400+0000 7f75bddb9700 1 -- 172.21.15.7:0/1299810622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75b8102770 msgr2=0x7f75b8102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:36.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.400+0000 7f75bddb9700 1 --2- 172.21.15.7:0/1299810622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75b8102770 0x7f75b8102b50 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f75a80099c0 tx=0x7f75a8009cd0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.400+0000 7f75bddb9700 1 -- 172.21.15.7:0/1299810622 shutdown_connections 2024-06-10T01:14:36.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.400+0000 7f75bddb9700 1 --2- 172.21.15.7:0/1299810622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75b8107860 0x7f75b810f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.400+0000 7f75bddb9700 1 --2- 172.21.15.7:0/1299810622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75b8102770 0x7f75b8102b50 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.400+0000 7f75bddb9700 1 --2- 172.21.15.7:0/1299810622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75b8103120 0x7f75b8107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.400+0000 7f75bddb9700 1 -- 172.21.15.7:0/1299810622 >> 172.21.15.7:0/1299810622 conn(0x7f75b8079600 msgr2=0x7f75b8079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:36.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.400+0000 7f75bddb9700 1 -- 172.21.15.7:0/1299810622 shutdown_connections 2024-06-10T01:14:36.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.400+0000 7f75bddb9700 1 -- 172.21.15.7:0/1299810622 wait complete. 2024-06-10T01:14:36.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.401+0000 7f75bddb9700 1 Processor -- start 2024-06-10T01:14:36.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.401+0000 7f75bddb9700 1 -- start start 2024-06-10T01:14:36.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.401+0000 7f75bddb9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75b8102770 0x7f75b8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:36.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.401+0000 7f75bddb9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75b8103120 0x7f75b8074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:36.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.401+0000 7f75bddb9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75b8107860 0x7f75b8075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:36.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.401+0000 7f75bddb9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f75b8112780 con 0x7f75b8107860 2024-06-10T01:14:36.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.401+0000 7f75bddb9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f75b8112600 con 0x7f75b8102770 2024-06-10T01:14:36.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.401+0000 7f75bddb9700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f75b8112900 con 0x7f75b8103120 2024-06-10T01:14:36.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.401+0000 7f75b77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75b8102770 0x7f75b8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:36.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.401+0000 7f75b7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75b8107860 0x7f75b8075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:36.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.402+0000 7f75b6ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75b8103120 0x7f75b8074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:36.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.402+0000 7f75b77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75b8102770 0x7f75b8074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:58502/0 (socket says 172.21.15.7:58502) 2024-06-10T01:14:36.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.402+0000 7f75b77fe700 1 -- 172.21.15.7:0/2733281382 learned_addr learned my addr 172.21.15.7:0/2733281382 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:36.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.402+0000 7f75b77fe700 1 -- 172.21.15.7:0/2733281382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75b8103120 msgr2=0x7f75b8074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:36.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.402+0000 7f75b77fe700 1 --2- 172.21.15.7:0/2733281382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75b8103120 0x7f75b8074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.402+0000 7f75b77fe700 1 -- 172.21.15.7:0/2733281382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75b8107860 msgr2=0x7f75b8075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:36.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.402+0000 7f75b77fe700 1 --2- 172.21.15.7:0/2733281382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75b8107860 0x7f75b8075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.402+0000 7f75b77fe700 1 -- 172.21.15.7:0/2733281382 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b81a70e0 con 0x7f75b8102770 2024-06-10T01:14:36.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.402+0000 7f75b77fe700 1 --2- 172.21.15.7:0/2733281382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75b8102770 0x7f75b8074790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f75a8006010 tx=0x7f75a800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:36.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.402+0000 7f75b4ff9700 1 -- 172.21.15.7:0/2733281382 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8003680 con 0x7f75b8102770 2024-06-10T01:14:36.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.402+0000 7f75b4ff9700 1 -- 172.21.15.7:0/2733281382 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75a80050d0 con 0x7f75b8102770 2024-06-10T01:14:36.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.402+0000 7f75bddb9700 1 -- 172.21.15.7:0/2733281382 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b81a7370 con 0x7f75b8102770 2024-06-10T01:14:36.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.402+0000 7f75b4ff9700 1 -- 172.21.15.7:0/2733281382 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8003db0 con 0x7f75b8102770 2024-06-10T01:14:36.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.402+0000 7f75bddb9700 1 -- 172.21.15.7:0/2733281382 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b81a77c0 con 0x7f75b8102770 2024-06-10T01:14:36.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.405+0000 7f75b4ff9700 1 -- 172.21.15.7:0/2733281382 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f75a8005280 con 0x7f75b8102770 2024-06-10T01:14:36.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.406+0000 7f75b4ff9700 1 --2- 172.21.15.7:0/2733281382 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f75a40a2dc0 0x7f75a40a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:36.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.406+0000 7f75b6ffd700 1 --2- 172.21.15.7:0/2733281382 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f75a40a2dc0 0x7f75a40a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:36.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.406+0000 7f75b4ff9700 1 -- 172.21.15.7:0/2733281382 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f75a80d0050 con 0x7f75b8102770 2024-06-10T01:14:36.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.406+0000 7f75b6ffd700 1 --2- 172.21.15.7:0/2733281382 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f75a40a2dc0 0x7f75a40a5280 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f75b8107290 tx=0x7f75a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:36.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.406+0000 7f75c863cb80 1 Processor -- start 2024-06-10T01:14:36.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.406+0000 7f75c863cb80 1 -- start start 2024-06-10T01:14:36.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.407+0000 7f75c863cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c267c46640 0x55c267c46a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:36.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.407+0000 7f75c863cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c267c46f60 0x55c267c409e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:36.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.407+0000 7f75c863cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c267c40f20 0x55c267c41340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:36.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.407+0000 7f75c863cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c267b92810 con 0x55c267c40f20 2024-06-10T01:14:36.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.407+0000 7f75c863cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c267b3a720 con 0x55c267c46640 2024-06-10T01:14:36.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.407+0000 7f75c863cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c267b8e7d0 con 0x55c267c46f60 2024-06-10T01:14:36.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.407+0000 7f75b77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c267c46640 0x55c267c46a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:36.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.407+0000 7f75b7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c267c40f20 0x55c267c41340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:36.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.407+0000 7f75b77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c267c46640 0x55c267c46a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:58518/0 (socket says 172.21.15.7:58518) 2024-06-10T01:14:36.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.407+0000 7f75b77fe700 1 -- 172.21.15.7:0/3873659357 learned_addr learned my addr 172.21.15.7:0/3873659357 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:36.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.407+0000 7f75b6ffd700 1 --2- 172.21.15.7:0/3873659357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c267c46f60 0x55c267c409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:36.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.407+0000 7f75b77fe700 1 -- 172.21.15.7:0/3873659357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c267c46f60 msgr2=0x55c267c409e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:36.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.407+0000 7f75b77fe700 1 --2- 172.21.15.7:0/3873659357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c267c46f60 0x55c267c409e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.407+0000 7f75b77fe700 1 -- 172.21.15.7:0/3873659357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c267c40f20 msgr2=0x55c267c41340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:36.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.407+0000 7f75b77fe700 1 --2- 172.21.15.7:0/3873659357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c267c40f20 0x55c267c41340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.407+0000 7f75b77fe700 1 -- 172.21.15.7:0/3873659357 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c267b30f50 con 0x55c267c46640 2024-06-10T01:14:36.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.407+0000 7f75b77fe700 1 --2- 172.21.15.7:0/3873659357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c267c46640 0x55c267c46a20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f75a80a4c70 tx=0x7f75a809c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1d7318d6693338c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:36.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.407+0000 7f75adffb700 1 -- 172.21.15.7:0/3873659357 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a80bc040 con 0x55c267c46640 2024-06-10T01:14:36.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.408+0000 7f75adffb700 1 -- 172.21.15.7:0/3873659357 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75a809ccc0 con 0x55c267c46640 2024-06-10T01:14:36.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.408+0000 7f75adffb700 1 -- 172.21.15.7:0/3873659357 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a80163a0 con 0x55c267c46640 2024-06-10T01:14:36.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.408+0000 7f75c863cb80 1 -- 172.21.15.7:0/3873659357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c267c46640 msgr2=0x55c267c46a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:36.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.408+0000 7f75c863cb80 1 --2- 172.21.15.7:0/3873659357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c267c46640 0x55c267c46a20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f75a80a4c70 tx=0x7f75a809c390 comp rx=0 tx=0).stop 2024-06-10T01:14:36.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.409+0000 7f75c863cb80 1 -- 172.21.15.7:0/3873659357 shutdown_connections 2024-06-10T01:14:36.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.409+0000 7f75c863cb80 1 --2- 172.21.15.7:0/3873659357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c267c40f20 0x55c267c41340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.409+0000 7f75c863cb80 1 --2- 172.21.15.7:0/3873659357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c267c46640 0x55c267c46a20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.409+0000 7f75c863cb80 1 --2- 172.21.15.7:0/3873659357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c267c46f60 0x55c267c409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.409+0000 7f75c863cb80 1 -- 172.21.15.7:0/3873659357 >> 172.21.15.7:0/3873659357 conn(0x55c267b6bcc0 msgr2=0x55c267c3b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:36.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.409+0000 7f75c863cb80 1 -- 172.21.15.7:0/3873659357 shutdown_connections 2024-06-10T01:14:36.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.409+0000 7f75c863cb80 1 -- 172.21.15.7:0/3873659357 wait complete. 2024-06-10T01:14:36.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.409+0000 7f75c863cb80 1 -- start start 2024-06-10T01:14:36.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.409+0000 7f75c863cb80 1 Processor -- start 2024-06-10T01:14:36.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.410+0000 7f75c863cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c267c40f20 0x55c267c4cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:36.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.410+0000 7f75c863cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c267c46640 0x55c267c4a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:36.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.410+0000 7f75c863cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c267c46f60 0x55c267c4a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:36.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.410+0000 7f75c863cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c267c4d6f0 con 0x55c267c40f20 2024-06-10T01:14:36.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.410+0000 7f75c863cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c267b32bf0 con 0x55c267c46f60 2024-06-10T01:14:36.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.410+0000 7f75c863cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c267c4d870 con 0x55c267c46640 2024-06-10T01:14:36.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.410+0000 7f75c863cb80 10 client.0 _ll_register_callbacks cb 0x55c267b2c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:14:36.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.410+0000 7f75b77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c267c40f20 0x55c267c4cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:36.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.410+0000 7f75b77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c267c40f20 0x55c267c4cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60872/0 (socket says 172.21.15.7:60872) 2024-06-10T01:14:36.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.410+0000 7f75b77fe700 1 -- 172.21.15.7:0/2690741767 learned_addr learned my addr 172.21.15.7:0/2690741767 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:36.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.410+0000 7f75b6ffd700 1 --2- 172.21.15.7:0/2690741767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c267c46640 0x55c267c4a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:36.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.411+0000 7f75b77fe700 1 -- 172.21.15.7:0/2690741767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c267c46640 msgr2=0x55c267c4a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:36.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.411+0000 7f75b77fe700 1 --2- 172.21.15.7:0/2690741767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c267c46640 0x55c267c4a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.411+0000 7f75b77fe700 1 -- 172.21.15.7:0/2690741767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c267c46f60 msgr2=0x55c267c4a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:36.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.411+0000 7f75b7fff700 1 --2- 172.21.15.7:0/2690741767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c267c46f60 0x55c267c4a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:36.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.411+0000 7f75b77fe700 1 --2- 172.21.15.7:0/2690741767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c267c46f60 0x55c267c4a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.411+0000 7f75b77fe700 1 -- 172.21.15.7:0/2690741767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c267ae18d0 con 0x55c267c40f20 2024-06-10T01:14:36.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.411+0000 7f75b7fff700 1 --2- 172.21.15.7:0/2690741767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c267c46f60 0x55c267c4a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:14:36.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.412+0000 7f75b77fe700 1 --2- 172.21.15.7:0/2690741767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c267c40f20 0x55c267c4cd20 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f75a80cdb40 tx=0x7f75a80cd780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9137a661743c5d2f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:36.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.412+0000 7f7596ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:14:36.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.412+0000 7f7596ffd700 1 -- 172.21.15.7:0/2690741767 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a80b5060 con 0x55c267c40f20 2024-06-10T01:14:36.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.412+0000 7f7596ffd700 1 -- 172.21.15.7:0/2690741767 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f75a809c820 con 0x55c267c40f20 2024-06-10T01:14:36.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.412+0000 7f75c863cb80 1 -- 172.21.15.7:0/2690741767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75a809ccc0 con 0x55c267c40f20 2024-06-10T01:14:36.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.412+0000 7f7596ffd700 1 -- 172.21.15.7:0/2690741767 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f75a80bc040 con 0x55c267c40f20 2024-06-10T01:14:36.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.412+0000 7f75c863cb80 10 client.17601 fetch_fsmap learned FSMap version 35 2024-06-10T01:14:36.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.412+0000 7f75c863cb80 1 -- 172.21.15.7:0/2690741767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c267c4d6f0 con 0x55c267c40f20 2024-06-10T01:14:36.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.412+0000 7f7596ffd700 1 -- 172.21.15.7:0/2690741767 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75a809ccc0 con 0x55c267c40f20 2024-06-10T01:14:36.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.412+0000 7f7596ffd700 1 -- 172.21.15.7:0/2690741767 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f75a8085be0 con 0x55c267c40f20 2024-06-10T01:14:36.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.413+0000 7f75c863cb80 10 client.17601 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:14:36.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.413+0000 7f75c863cb80 10 client.17601 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:14:36.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.413+0000 7f75c863cb80 20 client.17601 populate_metadata read hostname 'smithi007' 2024-06-10T01:14:36.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.413+0000 7f75c863cb80 1 --2- 172.21.15.7:0/2690741767 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55c267c5a320 0x55c267c5c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:36.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.413+0000 7f75c863cb80 4 client.17601 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:14:36.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.413+0000 7f75c863cb80 1 -- 172.21.15.7:0/2690741767 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c267b9fd10 con 0x55c267c5a320 2024-06-10T01:14:36.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.413+0000 7f75b6ffd700 1 --2- 172.21.15.7:0/2690741767 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55c267c5a320 0x55c267c5c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:36.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.413+0000 7f7596ffd700 10 client.17601 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:14:36.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.413+0000 7f75b6ffd700 1 --2- 172.21.15.7:0/2690741767 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55c267c5a320 0x55c267c5c710 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eff94e7ef13f7600 server_cookie=f568537d52e528a4 in_seq=0 out_seq=0 2024-06-10T01:14:36.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.414+0000 7f7596ffd700 1 -- 172.21.15.7:0/2690741767 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c267b9fd10 con 0x55c267c5a320 2024-06-10T01:14:36.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.414+0000 7f7596ffd700 10 client.17601 handle_command_reply: tid=0 2024-06-10T01:14:36.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.414+0000 7f75c863cb80 1 client.17601 shutdown 2024-06-10T01:14:36.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.414+0000 7f75c863cb80 1 -- 172.21.15.7:0/2690741767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c267c40f20 msgr2=0x55c267c4cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:36.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.414+0000 7f75c863cb80 1 --2- 172.21.15.7:0/2690741767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c267c40f20 0x55c267c4cd20 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f75a80cdb40 tx=0x7f75a80cd780 comp rx=0 tx=0).stop 2024-06-10T01:14:36.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.414+0000 7f75c863cb80 1 -- 172.21.15.7:0/2690741767 shutdown_connections 2024-06-10T01:14:36.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.414+0000 7f75c863cb80 1 --2- 172.21.15.7:0/2690741767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c267c46f60 0x55c267c4a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.414+0000 7f75c863cb80 1 --2- 172.21.15.7:0/2690741767 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55c267c5a320 0x55c267c5c710 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.414+0000 7f75c863cb80 1 --2- 172.21.15.7:0/2690741767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c267c40f20 0x55c267c4cd20 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.414+0000 7f75c863cb80 1 --2- 172.21.15.7:0/2690741767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c267c46640 0x55c267c4a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.414+0000 7f7596ffd700 0 client.17601 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:14:36.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.414+0000 7f75c863cb80 1 -- 172.21.15.7:0/2690741767 >> 172.21.15.7:0/2690741767 conn(0x55c267b6bcc0 msgr2=0x55c267c3b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:36.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.415+0000 7f75c863cb80 1 -- 172.21.15.7:0/2690741767 shutdown_connections 2024-06-10T01:14:36.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.415+0000 7f75c863cb80 1 -- 172.21.15.7:0/2690741767 wait complete. 2024-06-10T01:14:36.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.415+0000 7f75c863cb80 20 client.17601 trim_cache size 0 max 16384 2024-06-10T01:14:36.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.432+0000 7f75c863cb80 1 Processor -- start 2024-06-10T01:14:36.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.432+0000 7f75c863cb80 1 -- start start 2024-06-10T01:14:36.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.432+0000 7f75c863cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c267c40f20 0x55c267c409e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:36.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.432+0000 7f75c863cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c267c46640 0x55c267c47a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:36.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.432+0000 7f75c863cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c267c46f60 0x55c267c61f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:36.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.432+0000 7f75c863cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81b2e20 con 0x55c267c40f20 2024-06-10T01:14:36.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.432+0000 7f75c863cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81b0650 con 0x55c267c46640 2024-06-10T01:14:36.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.432+0000 7f75c863cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81ae4a0 con 0x55c267c46f60 2024-06-10T01:14:36.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.432+0000 7f75b77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c267c40f20 0x55c267c409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:36.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.432+0000 7f75b7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c267c46f60 0x55c267c61f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:36.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.432+0000 7f75b77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c267c40f20 0x55c267c409e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60886/0 (socket says 172.21.15.7:60886) 2024-06-10T01:14:36.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.432+0000 7f75b77fe700 1 -- 172.21.15.7:0/567013193 learned_addr learned my addr 172.21.15.7:0/567013193 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:36.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.432+0000 7f75b7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c267c46f60 0x55c267c61f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:50130/0 (socket says 172.21.15.7:50130) 2024-06-10T01:14:36.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.432+0000 7f75b6ffd700 1 --2- 172.21.15.7:0/567013193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c267c46640 0x55c267c47a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:36.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.432+0000 7f75b77fe700 1 -- 172.21.15.7:0/567013193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c267c46f60 msgr2=0x55c267c61f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:36.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.432+0000 7f75b77fe700 1 --2- 172.21.15.7:0/567013193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c267c46f60 0x55c267c61f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.432+0000 7f75b77fe700 1 -- 172.21.15.7:0/567013193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c267c46640 msgr2=0x55c267c47a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:36.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.432+0000 7f75b77fe700 1 --2- 172.21.15.7:0/567013193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c267c46640 0x55c267c47a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.432+0000 7f75b77fe700 1 -- 172.21.15.7:0/567013193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c267c4c960 con 0x55c267c40f20 2024-06-10T01:14:36.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.433+0000 7f75b77fe700 1 --2- 172.21.15.7:0/567013193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c267c40f20 0x55c267c409e0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f75a809c550 tx=0x7f75a8003860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2955f41e6ec4da04 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:36.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.433+0000 7f75ad7fa700 1 -- 172.21.15.7:0/567013193 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a80b5040 con 0x55c267c40f20 2024-06-10T01:14:36.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.433+0000 7f75ad7fa700 1 -- 172.21.15.7:0/567013193 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75a809ccc0 con 0x55c267c40f20 2024-06-10T01:14:36.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.433+0000 7f75ad7fa700 1 -- 172.21.15.7:0/567013193 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a80a4f70 con 0x55c267c40f20 2024-06-10T01:14:36.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.434+0000 7f75c863cb80 1 -- 172.21.15.7:0/567013193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c267c40f20 msgr2=0x55c267c409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:36.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.434+0000 7f75c863cb80 1 --2- 172.21.15.7:0/567013193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c267c40f20 0x55c267c409e0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f75a809c550 tx=0x7f75a8003860 comp rx=0 tx=0).stop 2024-06-10T01:14:36.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.434+0000 7f75c863cb80 1 -- 172.21.15.7:0/567013193 shutdown_connections 2024-06-10T01:14:36.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.434+0000 7f75c863cb80 1 --2- 172.21.15.7:0/567013193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c267c46f60 0x55c267c61f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.434+0000 7f75c863cb80 1 --2- 172.21.15.7:0/567013193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c267c40f20 0x55c267c409e0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.434+0000 7f75c863cb80 1 --2- 172.21.15.7:0/567013193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c267c46640 0x55c267c47a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.434+0000 7f75c863cb80 1 -- 172.21.15.7:0/567013193 >> 172.21.15.7:0/567013193 conn(0x55c267b6bcc0 msgr2=0x55c267c47340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:36.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.434+0000 7f75c863cb80 1 -- 172.21.15.7:0/567013193 shutdown_connections 2024-06-10T01:14:36.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.434+0000 7f75c863cb80 1 -- 172.21.15.7:0/567013193 wait complete. 2024-06-10T01:14:36.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.434+0000 7f75c863cb80 1 -- start start 2024-06-10T01:14:36.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.434+0000 7f75c863cb80 1 Processor -- start 2024-06-10T01:14:36.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.435+0000 7f75c863cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c267c40f20 0x55c267c52ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:36.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.435+0000 7f75c863cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c267c46640 0x55c267c53220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:36.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.435+0000 7f75c863cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c267c46f60 0x55c267c61540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:36.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.435+0000 7f75c863cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81aeb10 con 0x55c267c46f60 2024-06-10T01:14:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.435+0000 7f75c863cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81aff80 con 0x55c267c40f20 2024-06-10T01:14:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.435+0000 7f75c863cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81af1e0 con 0x55c267c46640 2024-06-10T01:14:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.435+0000 7f75c863cb80 10 client.0 _ll_register_callbacks cb 0x55c267b2c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:14:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.435+0000 7f75b77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c267c40f20 0x55c267c52ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.435+0000 7f75b77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c267c40f20 0x55c267c52ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:58546/0 (socket says 172.21.15.7:58546) 2024-06-10T01:14:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.435+0000 7f75b77fe700 1 -- 172.21.15.7:0/3452236628 learned_addr learned my addr 172.21.15.7:0/3452236628 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.435+0000 7f75b7fff700 1 --2- 172.21.15.7:0/3452236628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c267c46f60 0x55c267c61540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.435+0000 7f75b6ffd700 1 --2- 172.21.15.7:0/3452236628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c267c46640 0x55c267c53220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.435+0000 7f75b77fe700 1 -- 172.21.15.7:0/3452236628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c267c46640 msgr2=0x55c267c53220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.435+0000 7f75b77fe700 1 --2- 172.21.15.7:0/3452236628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c267c46640 0x55c267c53220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.435+0000 7f75b77fe700 1 -- 172.21.15.7:0/3452236628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c267c46f60 msgr2=0x55c267c61540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.435+0000 7f75b77fe700 1 --2- 172.21.15.7:0/3452236628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c267c46f60 0x55c267c61540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.435+0000 7f75b77fe700 1 -- 172.21.15.7:0/3452236628 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c267c61d10 con 0x55c267c40f20 2024-06-10T01:14:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.435+0000 7f75b77fe700 1 --2- 172.21.15.7:0/3452236628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c267c40f20 0x55c267c52ce0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f75a8098f90 tx=0x7f75a8009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0d38fabdfaf4e4a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.435+0000 7f7596ffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:14:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.436+0000 7f7596ffd700 1 -- 172.21.15.7:0/3452236628 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a80a51a0 con 0x55c267c40f20 2024-06-10T01:14:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.436+0000 7f75c863cb80 1 -- 172.21.15.7:0/3452236628 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75a809ccc0 con 0x55c267c40f20 2024-06-10T01:14:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.436+0000 7f7596ffd700 1 -- 172.21.15.7:0/3452236628 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f75a80a4f20 con 0x55c267c40f20 2024-06-10T01:14:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.436+0000 7f7596ffd700 1 -- 172.21.15.7:0/3452236628 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f75a809c5c0 con 0x55c267c40f20 2024-06-10T01:14:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.436+0000 7f75c863cb80 10 client.26999 fetch_fsmap learned FSMap version 35 2024-06-10T01:14:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.436+0000 7f75c863cb80 1 -- 172.21.15.7:0/3452236628 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c267c61d10 con 0x55c267c40f20 2024-06-10T01:14:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.436+0000 7f7596ffd700 1 -- 172.21.15.7:0/3452236628 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75a809ccc0 con 0x55c267c40f20 2024-06-10T01:14:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.436+0000 7f7596ffd700 1 -- 172.21.15.7:0/3452236628 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f75a8098310 con 0x55c267c40f20 2024-06-10T01:14:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.436+0000 7f75c863cb80 10 client.26999 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:14:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.436+0000 7f75c863cb80 10 client.26999 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:14:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.436+0000 7f75c863cb80 20 client.26999 populate_metadata read hostname 'smithi007' 2024-06-10T01:14:36.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.436+0000 7f75c863cb80 1 --2- 172.21.15.7:0/3452236628 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55c267cc4c80 0x55c267cc70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:36.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.436+0000 7f75c863cb80 4 client.26999 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:14:36.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.436+0000 7f75c863cb80 1 -- 172.21.15.7:0/3452236628 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c267b9a290 con 0x55c267cc4c80 2024-06-10T01:14:36.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.437+0000 7f75b6ffd700 1 --2- 172.21.15.7:0/3452236628 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55c267cc4c80 0x55c267cc70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:36.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.437+0000 7f7596ffd700 10 client.26999 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:14:36.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.437+0000 7f75b6ffd700 1 --2- 172.21.15.7:0/3452236628 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55c267cc4c80 0x55c267cc70a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7a9017a629c2cf98 server_cookie=299e3338fbf1a777 in_seq=0 out_seq=0 2024-06-10T01:14:36.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.438+0000 7f7596ffd700 1 -- 172.21.15.7:0/3452236628 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c267b9a290 con 0x55c267cc4c80 2024-06-10T01:14:36.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.438+0000 7f7596ffd700 10 client.26999 handle_command_reply: tid=0 2024-06-10T01:14:36.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.438+0000 7f75c863cb80 1 client.26999 shutdown 2024-06-10T01:14:36.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.438+0000 7f75c863cb80 1 -- 172.21.15.7:0/3452236628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c267c40f20 msgr2=0x55c267c52ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:36.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.438+0000 7f75c863cb80 1 --2- 172.21.15.7:0/3452236628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c267c40f20 0x55c267c52ce0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f75a8098f90 tx=0x7f75a8009ed0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.439+0000 7f75c863cb80 1 -- 172.21.15.7:0/3452236628 shutdown_connections 2024-06-10T01:14:36.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.439+0000 7f75c863cb80 1 --2- 172.21.15.7:0/3452236628 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55c267cc4c80 0x55c267cc70a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.439+0000 7f75c863cb80 1 --2- 172.21.15.7:0/3452236628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c267c46f60 0x55c267c61540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.439+0000 7f75c863cb80 1 --2- 172.21.15.7:0/3452236628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c267c40f20 0x55c267c52ce0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.439+0000 7f75c863cb80 1 --2- 172.21.15.7:0/3452236628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c267c46640 0x55c267c53220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.439+0000 7f7596ffd700 0 client.26999 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:14:36.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.439+0000 7f75c863cb80 1 -- 172.21.15.7:0/3452236628 >> 172.21.15.7:0/3452236628 conn(0x55c267b6bcc0 msgr2=0x55c267c47340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:36.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.439+0000 7f75c863cb80 1 -- 172.21.15.7:0/3452236628 shutdown_connections 2024-06-10T01:14:36.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.439+0000 7f75c863cb80 1 -- 172.21.15.7:0/3452236628 wait complete. 2024-06-10T01:14:36.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.439+0000 7f75c863cb80 20 client.26999 trim_cache size 0 max 16384 2024-06-10T01:14:36.437 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:14:36.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.439+0000 7f75bddb9700 1 -- 172.21.15.7:0/2733281382 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f75a40a2dc0 msgr2=0x7f75a40a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:36.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.439+0000 7f75bddb9700 1 --2- 172.21.15.7:0/2733281382 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f75a40a2dc0 0x7f75a40a5280 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f75b8107290 tx=0x7f75a0009260 comp rx=0 tx=0).stop 2024-06-10T01:14:36.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.439+0000 7f75bddb9700 1 -- 172.21.15.7:0/2733281382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75b8102770 msgr2=0x7f75b8074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:36.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.439+0000 7f75bddb9700 1 --2- 172.21.15.7:0/2733281382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75b8102770 0x7f75b8074790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f75a8006010 tx=0x7f75a800f690 comp rx=0 tx=0).stop 2024-06-10T01:14:36.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.439+0000 7f75bddb9700 1 -- 172.21.15.7:0/2733281382 shutdown_connections 2024-06-10T01:14:36.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.439+0000 7f75bddb9700 1 --2- 172.21.15.7:0/2733281382 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f75a40a2dc0 0x7f75a40a5280 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.439+0000 7f75bddb9700 1 --2- 172.21.15.7:0/2733281382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75b8107860 0x7f75b8075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.439+0000 7f75bddb9700 1 --2- 172.21.15.7:0/2733281382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75b8102770 0x7f75b8074790 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.439+0000 7f75bddb9700 1 --2- 172.21.15.7:0/2733281382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75b8103120 0x7f75b8074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:36.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.439+0000 7f75bddb9700 1 -- 172.21.15.7:0/2733281382 >> 172.21.15.7:0/2733281382 conn(0x7f75b8079600 msgr2=0x7f75b8101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:36.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.440+0000 7f75bddb9700 1 -- 172.21.15.7:0/2733281382 shutdown_connections 2024-06-10T01:14:36.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:36.440+0000 7f75bddb9700 1 -- 172.21.15.7:0/2733281382 wait complete. 2024-06-10T01:14:36.741 INFO:tasks.workunit.client.0.smithi007.stdout: 1 629012 7.79 MB/sec execute 317 sec latency 41.699 ms 2024-06-10T01:14:37.602 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:37 smithi007 ceph-mon[32398]: pgmap v690: 129 pgs: 129 active+clean; 107 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 2.7 MiB/s wr, 340 op/s 2024-06-10T01:14:37.630 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:14:37.741 INFO:tasks.workunit.client.0.smithi007.stdout: 1 630964 7.79 MB/sec execute 318 sec latency 49.474 ms 2024-06-10T01:14:37.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:37 smithi078 ceph-mon[36683]: pgmap v690: 129 pgs: 129 active+clean; 107 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 2.7 MiB/s wr, 340 op/s 2024-06-10T01:14:37.901 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:14:37.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:37 smithi175 ceph-mon[36636]: pgmap v690: 129 pgs: 129 active+clean; 107 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 2.7 MiB/s wr, 340 op/s 2024-06-10T01:14:38.741 INFO:tasks.workunit.client.0.smithi007.stdout: 1 632706 7.80 MB/sec execute 319 sec latency 16.485 ms 2024-06-10T01:14:38.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:38 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:14:38.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:38 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:14:38.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:38 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:14:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.518+0000 7f4f03f9d640 1 -- 172.21.15.7:0/2938450662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4efc1035c0 msgr2=0x7f4efc109e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.518+0000 7f4f03f9d640 1 --2- 172.21.15.7:0/2938450662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4efc1035c0 0x7f4efc109e50 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4009ca0 tx=0x7f4ef402a8c0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.518+0000 7f4f03f9d640 1 -- 172.21.15.7:0/2938450662 shutdown_connections 2024-06-10T01:14:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.518+0000 7f4f03f9d640 1 --2- 172.21.15.7:0/2938450662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4efc1035c0 0x7f4efc109e50 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.518+0000 7f4f03f9d640 1 --2- 172.21.15.7:0/2938450662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4efc102c00 0x7f4efc103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.518+0000 7f4f03f9d640 1 --2- 172.21.15.7:0/2938450662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4efc101a10 0x7f4efc101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.518+0000 7f4f03f9d640 1 -- 172.21.15.7:0/2938450662 >> 172.21.15.7:0/2938450662 conn(0x7f4efc0fcf80 msgr2=0x7f4efc0ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.518+0000 7f4f03f9d640 1 -- 172.21.15.7:0/2938450662 shutdown_connections 2024-06-10T01:14:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.518+0000 7f4f03f9d640 1 -- 172.21.15.7:0/2938450662 wait complete. 2024-06-10T01:14:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.519+0000 7f4f03f9d640 1 Processor -- start 2024-06-10T01:14:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.519+0000 7f4f03f9d640 1 -- start start 2024-06-10T01:14:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.519+0000 7f4f03f9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4efc101a10 0x7f4efc197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.519+0000 7f4f03f9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4efc102c00 0x7f4efc198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.519+0000 7f4f03f9d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4efc1035c0 0x7f4efc19f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.519+0000 7f4f03f9d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4efc10d670 con 0x7f4efc102c00 2024-06-10T01:14:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.519+0000 7f4f03f9d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4efc10d4f0 con 0x7f4efc101a10 2024-06-10T01:14:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.519+0000 7f4f03f9d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4efc10d7f0 con 0x7f4efc1035c0 2024-06-10T01:14:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.519+0000 7f4f01538640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4efc102c00 0x7f4efc198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.519+0000 7f4f01d39640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4efc101a10 0x7f4efc197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.519+0000 7f4f01d39640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4efc101a10 0x7f4efc197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:58558/0 (socket says 172.21.15.7:58558) 2024-06-10T01:14:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.519+0000 7f4f0253a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4efc1035c0 0x7f4efc19f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.519+0000 7f4f01538640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4efc102c00 0x7f4efc198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60906/0 (socket says 172.21.15.7:60906) 2024-06-10T01:14:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.519+0000 7f4f01d39640 1 -- 172.21.15.7:0/4134939875 learned_addr learned my addr 172.21.15.7:0/4134939875 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.519+0000 7f4f01538640 1 -- 172.21.15.7:0/4134939875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4efc1035c0 msgr2=0x7f4efc19f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.519+0000 7f4f01538640 1 --2- 172.21.15.7:0/4134939875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4efc1035c0 0x7f4efc19f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.519+0000 7f4f01538640 1 -- 172.21.15.7:0/4134939875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4efc101a10 msgr2=0x7f4efc197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.519+0000 7f4f01538640 1 --2- 172.21.15.7:0/4134939875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4efc101a10 0x7f4efc197d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.519+0000 7f4f01538640 1 -- 172.21.15.7:0/4134939875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4efc19f910 con 0x7f4efc102c00 2024-06-10T01:14:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.519+0000 7f4f01d39640 1 --2- 172.21.15.7:0/4134939875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4efc101a10 0x7f4efc197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.519+0000 7f4f01538640 1 --2- 172.21.15.7:0/4134939875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4efc102c00 0x7f4efc198240 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f4eec00e9f0 tx=0x7f4eec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.520+0000 7f4ef2ffd640 1 -- 172.21.15.7:0/4134939875 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec00cd80 con 0x7f4efc102c00 2024-06-10T01:14:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.520+0000 7f4ef2ffd640 1 -- 172.21.15.7:0/4134939875 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4eec0035f0 con 0x7f4efc102c00 2024-06-10T01:14:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.520+0000 7f4f03f9d640 1 -- 172.21.15.7:0/4134939875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4efc19fc00 con 0x7f4efc102c00 2024-06-10T01:14:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.520+0000 7f4ef2ffd640 1 -- 172.21.15.7:0/4134939875 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec010690 con 0x7f4efc102c00 2024-06-10T01:14:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.520+0000 7f4f03f9d640 1 -- 172.21.15.7:0/4134939875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4efc1a0150 con 0x7f4efc102c00 2024-06-10T01:14:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.522+0000 7f4ef2ffd640 1 -- 172.21.15.7:0/4134939875 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f4eec003b30 con 0x7f4efc102c00 2024-06-10T01:14:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.522+0000 7f4f06d82740 1 Processor -- start 2024-06-10T01:14:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.523+0000 7f4f06d82740 1 -- start start 2024-06-10T01:14:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.523+0000 7f4f06d82740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582f658f420 0x5582f658f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.523+0000 7f4f06d82740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582f658fe10 0x5582f6594910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.523+0000 7f4f06d82740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f6594f80 0x5582f6597410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.523+0000 7f4f06d82740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5582f64ee4f0 con 0x5582f658fe10 2024-06-10T01:14:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.523+0000 7f4f06d82740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5582f648c870 con 0x5582f6594f80 2024-06-10T01:14:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.523+0000 7f4f06d82740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5582f64ea4b0 con 0x5582f658f420 2024-06-10T01:14:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.523+0000 7f4f0253a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582f658f420 0x5582f658f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.523+0000 7f4f01d39640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582f658fe10 0x5582f6594910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.523+0000 7f4f0253a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582f658f420 0x5582f658f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:50164/0 (socket says 172.21.15.7:50164) 2024-06-10T01:14:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.523+0000 7f4f0253a640 1 -- 172.21.15.7:0/1713095878 learned_addr learned my addr 172.21.15.7:0/1713095878 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.523+0000 7f4ef2ffd640 1 --2- 172.21.15.7:0/4134939875 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4ee80a2b80 0x7f4ee80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.523+0000 7f4f01538640 1 --2- 172.21.15.7:0/1713095878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f6594f80 0x5582f6597410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.524+0000 7f4f0253a640 1 --2- 172.21.15.7:0/4134939875 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4ee80a2b80 0x7f4ee80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.524+0000 7f4ef2ffd640 1 -- 172.21.15.7:0/4134939875 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f4eec0cf790 con 0x7f4efc102c00 2024-06-10T01:14:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.524+0000 7f4f0253a640 1 -- 172.21.15.7:0/1713095878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f6594f80 msgr2=0x5582f6597410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.524+0000 7f4f0253a640 1 --2- 172.21.15.7:0/1713095878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f6594f80 0x5582f6597410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.524+0000 7f4f0253a640 1 -- 172.21.15.7:0/1713095878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582f658fe10 msgr2=0x5582f6594910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.524+0000 7f4f0253a640 1 --2- 172.21.15.7:0/1713095878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582f658fe10 0x5582f6594910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.524+0000 7f4f0253a640 1 -- 172.21.15.7:0/1713095878 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582f64a89b0 con 0x5582f658f420 2024-06-10T01:14:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.524+0000 7f4f0253a640 1 --2- 172.21.15.7:0/1713095878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582f658f420 0x5582f658f840 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4035000 tx=0x7f4ef4031d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b52622046070e9ce server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.524+0000 7f4f0253a640 1 --2- 172.21.15.7:0/4134939875 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4ee80a2b80 0x7f4ee80a5040 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4048000 tx=0x7f4ef404a000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.524+0000 7f4ee7fff640 1 -- 172.21.15.7:0/1713095878 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef404f070 con 0x5582f658f420 2024-06-10T01:14:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.525+0000 7f4ee7fff640 1 -- 172.21.15.7:0/1713095878 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ef4036040 con 0x5582f658f420 2024-06-10T01:14:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.525+0000 7f4ee7fff640 1 -- 172.21.15.7:0/1713095878 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef402f610 con 0x5582f658f420 2024-06-10T01:14:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.525+0000 7f4f06d82740 1 -- 172.21.15.7:0/1713095878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582f658f420 msgr2=0x5582f658f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.525+0000 7f4f06d82740 1 --2- 172.21.15.7:0/1713095878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582f658f420 0x5582f658f840 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4035000 tx=0x7f4ef4031d30 comp rx=0 tx=0).stop 2024-06-10T01:14:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.526+0000 7f4f06d82740 1 -- 172.21.15.7:0/1713095878 shutdown_connections 2024-06-10T01:14:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.526+0000 7f4f06d82740 1 --2- 172.21.15.7:0/1713095878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f6594f80 0x5582f6597410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.526+0000 7f4f06d82740 1 --2- 172.21.15.7:0/1713095878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582f658fe10 0x5582f6594910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.526+0000 7f4f06d82740 1 --2- 172.21.15.7:0/1713095878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582f658f420 0x5582f658f840 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.526+0000 7f4f06d82740 1 -- 172.21.15.7:0/1713095878 >> 172.21.15.7:0/1713095878 conn(0x5582f6586680 msgr2=0x5582f6586a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.526+0000 7f4f06d82740 1 -- 172.21.15.7:0/1713095878 shutdown_connections 2024-06-10T01:14:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.526+0000 7f4f06d82740 1 -- 172.21.15.7:0/1713095878 wait complete. 2024-06-10T01:14:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.526+0000 7f4f06d82740 1 -- start start 2024-06-10T01:14:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.526+0000 7f4f06d82740 1 Processor -- start 2024-06-10T01:14:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.527+0000 7f4f06d82740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582f658f420 0x5582f65994e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.527+0000 7f4f06d82740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582f658fe10 0x5582f6599a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.527+0000 7f4f06d82740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f6594f80 0x5582f65a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.527+0000 7f4f06d82740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5582f6597d50 con 0x5582f658fe10 2024-06-10T01:14:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.527+0000 7f4f06d82740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5582f6477ae0 con 0x5582f6594f80 2024-06-10T01:14:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.527+0000 7f4f06d82740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5582f6597ed0 con 0x5582f658f420 2024-06-10T01:14:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.527+0000 7f4f06d82740 10 client.0 _ll_register_callbacks cb 0x5582f64b09b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:14:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.527+0000 7f4f01d39640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582f658f420 0x5582f65994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.527+0000 7f4f01d39640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582f658f420 0x5582f65994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:50170/0 (socket says 172.21.15.7:50170) 2024-06-10T01:14:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.527+0000 7f4f01d39640 1 -- 172.21.15.7:0/625175693 learned_addr learned my addr 172.21.15.7:0/625175693 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.527+0000 7f4f0253a640 1 --2- 172.21.15.7:0/625175693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f6594f80 0x5582f65a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.527+0000 7f4f01538640 1 --2- 172.21.15.7:0/625175693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582f658fe10 0x5582f6599a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.527+0000 7f4f01d39640 1 -- 172.21.15.7:0/625175693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f6594f80 msgr2=0x5582f65a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.527+0000 7f4f01d39640 1 --2- 172.21.15.7:0/625175693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f6594f80 0x5582f65a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.527+0000 7f4f01d39640 1 -- 172.21.15.7:0/625175693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582f658fe10 msgr2=0x5582f6599a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.527+0000 7f4f01d39640 1 --2- 172.21.15.7:0/625175693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582f658fe10 0x5582f6599a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.527+0000 7f4f01d39640 1 -- 172.21.15.7:0/625175693 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582f6202210 con 0x5582f658f420 2024-06-10T01:14:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.527+0000 7f4f01538640 1 --2- 172.21.15.7:0/625175693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582f658fe10 0x5582f6599a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:14:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.528+0000 7f4f01d39640 1 --2- 172.21.15.7:0/625175693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582f658f420 0x5582f65994e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8012cf0 tx=0x7f4ef800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbf7f618b89fc2d2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.528+0000 7f4ee57fa640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:14:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.528+0000 7f4ee57fa640 1 -- 172.21.15.7:0/625175693 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef80120d0 con 0x5582f658f420 2024-06-10T01:14:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.528+0000 7f4ee57fa640 1 -- 172.21.15.7:0/625175693 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f4ef8005020 con 0x5582f658f420 2024-06-10T01:14:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.528+0000 7f4f06d82740 1 -- 172.21.15.7:0/625175693 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ef4036040 con 0x5582f658f420 2024-06-10T01:14:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.528+0000 7f4ee57fa640 1 -- 172.21.15.7:0/625175693 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4ef8025050 con 0x5582f658f420 2024-06-10T01:14:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.528+0000 7f4f06d82740 10 client.27040 fetch_fsmap learned FSMap version 35 2024-06-10T01:14:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.528+0000 7f4f06d82740 1 -- 172.21.15.7:0/625175693 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5582f62ce620 con 0x5582f658f420 2024-06-10T01:14:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.528+0000 7f4ee57fa640 1 -- 172.21.15.7:0/625175693 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ef4036040 con 0x5582f658f420 2024-06-10T01:14:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.528+0000 7f4ee57fa640 1 -- 172.21.15.7:0/625175693 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f4ef8004c10 con 0x5582f658f420 2024-06-10T01:14:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.528+0000 7f4f06d82740 10 client.27040 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:14:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.528+0000 7f4f06d82740 10 client.27040 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:14:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.529+0000 7f4f06d82740 20 client.27040 populate_metadata read hostname 'smithi007' 2024-06-10T01:14:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.529+0000 7f4f06d82740 1 --2- 172.21.15.7:0/625175693 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5582f65a53c0 0x5582f65a7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.529+0000 7f4f06d82740 4 client.27040 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:14:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.529+0000 7f4f06d82740 1 -- 172.21.15.7:0/625175693 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582f6501f10 con 0x5582f65a53c0 2024-06-10T01:14:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.529+0000 7f4f01538640 1 --2- 172.21.15.7:0/625175693 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5582f65a53c0 0x5582f65a7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.529+0000 7f4ee57fa640 10 client.27040 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.529+0000 7f4f01538640 1 --2- 172.21.15.7:0/625175693 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5582f65a53c0 0x5582f65a7860 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59f5de7651ea54a1 server_cookie=86f88018ad63e692 in_seq=0 out_seq=0 2024-06-10T01:14:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.530+0000 7f4ee57fa640 1 -- 172.21.15.7:0/625175693 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5582f6501f10 con 0x5582f65a53c0 2024-06-10T01:14:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.530+0000 7f4ee57fa640 10 client.27040 handle_command_reply: tid=0 2024-06-10T01:14:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.530+0000 7f4f06d82740 1 client.27040 shutdown 2024-06-10T01:14:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.530+0000 7f4f06d82740 1 -- 172.21.15.7:0/625175693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582f658f420 msgr2=0x5582f65994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.530+0000 7f4f06d82740 1 --2- 172.21.15.7:0/625175693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582f658f420 0x5582f65994e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8012cf0 tx=0x7f4ef800f790 comp rx=0 tx=0).stop 2024-06-10T01:14:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.530+0000 7f4f06d82740 1 -- 172.21.15.7:0/625175693 shutdown_connections 2024-06-10T01:14:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.530+0000 7f4f06d82740 1 --2- 172.21.15.7:0/625175693 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5582f65a53c0 0x5582f65a7860 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.530+0000 7f4f06d82740 1 --2- 172.21.15.7:0/625175693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f6594f80 0x5582f65a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.530+0000 7f4f06d82740 1 --2- 172.21.15.7:0/625175693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582f658fe10 0x5582f6599a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.530+0000 7f4f06d82740 1 --2- 172.21.15.7:0/625175693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582f658f420 0x5582f65994e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.530+0000 7f4ee57fa640 0 client.27040 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.530+0000 7f4f06d82740 1 -- 172.21.15.7:0/625175693 >> 172.21.15.7:0/625175693 conn(0x5582f6586680 msgr2=0x5582f65857e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.530+0000 7f4f06d82740 1 -- 172.21.15.7:0/625175693 shutdown_connections 2024-06-10T01:14:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.530+0000 7f4f06d82740 1 -- 172.21.15.7:0/625175693 wait complete. 2024-06-10T01:14:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.530+0000 7f4f06d82740 20 client.27040 trim_cache size 0 max 16384 2024-06-10T01:14:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.547+0000 7f4f06d82740 1 Processor -- start 2024-06-10T01:14:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.547+0000 7f4f06d82740 1 -- start start 2024-06-10T01:14:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.548+0000 7f4f06d82740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582f658f420 0x5582f6598b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.548+0000 7f4f06d82740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f658fe10 0x5582f65a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.548+0000 7f4f06d82740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582f6594f80 0x5582f6599040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.548+0000 7f4f06d82740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4efc1aadc0 con 0x5582f658f420 2024-06-10T01:14:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.548+0000 7f4f06d82740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4efc1a8630 con 0x5582f658fe10 2024-06-10T01:14:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.548+0000 7f4f06d82740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4efc1a6480 con 0x5582f6594f80 2024-06-10T01:14:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.548+0000 7f4f01d39640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582f658f420 0x5582f6598b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.548+0000 7f4f01d39640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582f658f420 0x5582f6598b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60940/0 (socket says 172.21.15.7:60940) 2024-06-10T01:14:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.548+0000 7f4f01d39640 1 -- 172.21.15.7:0/854928490 learned_addr learned my addr 172.21.15.7:0/854928490 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.548+0000 7f4f01538640 1 --2- 172.21.15.7:0/854928490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f658fe10 0x5582f65a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.548+0000 7f4f0253a640 1 --2- 172.21.15.7:0/854928490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582f6594f80 0x5582f6599040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.548+0000 7f4f01d39640 1 -- 172.21.15.7:0/854928490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582f6594f80 msgr2=0x5582f6599040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.548+0000 7f4f01d39640 1 --2- 172.21.15.7:0/854928490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582f6594f80 0x5582f6599040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.548+0000 7f4f01d39640 1 -- 172.21.15.7:0/854928490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f658fe10 msgr2=0x5582f65a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.548+0000 7f4f01d39640 1 --2- 172.21.15.7:0/854928490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f658fe10 0x5582f65a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.548+0000 7f4f01d39640 1 -- 172.21.15.7:0/854928490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582f6581fe0 con 0x5582f658f420 2024-06-10T01:14:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.548+0000 7f4f0253a640 1 --2- 172.21.15.7:0/854928490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582f6594f80 0x5582f6599040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:14:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.548+0000 7f4f01538640 1 --2- 172.21.15.7:0/854928490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f658fe10 0x5582f65a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.548+0000 7f4f01d39640 1 --2- 172.21.15.7:0/854928490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582f658f420 0x5582f6598b00 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8012260 tx=0x7f4ef8012290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d330fcf01d9b567e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.548+0000 7f4ee77fe640 1 -- 172.21.15.7:0/854928490 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef802e070 con 0x5582f658f420 2024-06-10T01:14:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.548+0000 7f4ee77fe640 1 -- 172.21.15.7:0/854928490 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ef8004b60 con 0x5582f658f420 2024-06-10T01:14:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.549+0000 7f4ee77fe640 1 -- 172.21.15.7:0/854928490 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8022040 con 0x5582f658f420 2024-06-10T01:14:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.549+0000 7f4f06d82740 1 -- 172.21.15.7:0/854928490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582f658f420 msgr2=0x5582f6598b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.549+0000 7f4f06d82740 1 --2- 172.21.15.7:0/854928490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582f658f420 0x5582f6598b00 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8012260 tx=0x7f4ef8012290 comp rx=0 tx=0).stop 2024-06-10T01:14:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.549+0000 7f4f06d82740 1 -- 172.21.15.7:0/854928490 shutdown_connections 2024-06-10T01:14:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.549+0000 7f4f06d82740 1 --2- 172.21.15.7:0/854928490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582f6594f80 0x5582f6599040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.549+0000 7f4f06d82740 1 --2- 172.21.15.7:0/854928490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f658fe10 0x5582f65a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.549+0000 7f4f06d82740 1 --2- 172.21.15.7:0/854928490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582f658f420 0x5582f6598b00 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.549+0000 7f4f06d82740 1 -- 172.21.15.7:0/854928490 >> 172.21.15.7:0/854928490 conn(0x5582f6586680 msgr2=0x5582f6589df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.549+0000 7f4f06d82740 1 -- 172.21.15.7:0/854928490 shutdown_connections 2024-06-10T01:14:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.550+0000 7f4f06d82740 1 -- 172.21.15.7:0/854928490 wait complete. 2024-06-10T01:14:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.550+0000 7f4f06d82740 1 -- start start 2024-06-10T01:14:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.550+0000 7f4f06d82740 1 Processor -- start 2024-06-10T01:14:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.550+0000 7f4f06d82740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582f658f420 0x5582f659def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.550+0000 7f4f06d82740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f658fe10 0x5582f659e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.550+0000 7f4f06d82740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582f6594f80 0x5582f659f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.551+0000 7f4f06d82740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4efc1a6af0 con 0x5582f6594f80 2024-06-10T01:14:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.551+0000 7f4f06d82740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4efc1a7f60 con 0x5582f658fe10 2024-06-10T01:14:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.551+0000 7f4f06d82740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4efc1a71c0 con 0x5582f658f420 2024-06-10T01:14:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.551+0000 7f4f06d82740 10 client.0 _ll_register_callbacks cb 0x7f4ef8009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:14:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.551+0000 7f4f0253a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582f6594f80 0x5582f659f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.551+0000 7f4f01d39640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582f658f420 0x5582f659def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.551+0000 7f4f0253a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582f6594f80 0x5582f659f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60948/0 (socket says 172.21.15.7:60948) 2024-06-10T01:14:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.551+0000 7f4f01538640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f658fe10 0x5582f659e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.551+0000 7f4f0253a640 1 -- 172.21.15.7:0/681383798 learned_addr learned my addr 172.21.15.7:0/681383798 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.551+0000 7f4f01538640 1 -- 172.21.15.7:0/681383798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582f658f420 msgr2=0x5582f659def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.551+0000 7f4f01538640 1 --2- 172.21.15.7:0/681383798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582f658f420 0x5582f659def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.551+0000 7f4f01538640 1 -- 172.21.15.7:0/681383798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582f6594f80 msgr2=0x5582f659f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.551+0000 7f4f01538640 1 --2- 172.21.15.7:0/681383798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582f6594f80 0x5582f659f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.551+0000 7f4f01538640 1 -- 172.21.15.7:0/681383798 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582f659f9e0 con 0x5582f658fe10 2024-06-10T01:14:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.551+0000 7f4f01d39640 1 --2- 172.21.15.7:0/681383798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582f658f420 0x5582f659def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.551+0000 7f4f0253a640 1 --2- 172.21.15.7:0/681383798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582f6594f80 0x5582f659f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.551+0000 7f4f01538640 1 --2- 172.21.15.7:0/681383798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f658fe10 0x5582f659e430 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4eec00e9c0 tx=0x7f4eec01d640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3da1a8cd0a1cb66c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.551+0000 7f4ee57fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:14:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.551+0000 7f4ee57fa640 1 -- 172.21.15.7:0/681383798 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec09bc10 con 0x5582f658fe10 2024-06-10T01:14:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.551+0000 7f4ee57fa640 1 -- 172.21.15.7:0/681383798 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f4eec0037a0 con 0x5582f658fe10 2024-06-10T01:14:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.551+0000 7f4f06d82740 1 -- 172.21.15.7:0/681383798 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ef8004b60 con 0x5582f658fe10 2024-06-10T01:14:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.552+0000 7f4ee57fa640 1 -- 172.21.15.7:0/681383798 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4eec09ffc0 con 0x5582f658fe10 2024-06-10T01:14:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.552+0000 7f4f06d82740 10 client.27017 fetch_fsmap learned FSMap version 35 2024-06-10T01:14:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.552+0000 7f4f06d82740 1 -- 172.21.15.7:0/681383798 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5582f65a0d60 con 0x5582f658fe10 2024-06-10T01:14:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.552+0000 7f4ee57fa640 1 -- 172.21.15.7:0/681383798 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ef8004b60 con 0x5582f658fe10 2024-06-10T01:14:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.552+0000 7f4ee57fa640 1 -- 172.21.15.7:0/681383798 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f4eec0cf3f0 con 0x5582f658fe10 2024-06-10T01:14:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.552+0000 7f4f06d82740 10 client.27017 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:14:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.552+0000 7f4f06d82740 10 client.27017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:14:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.552+0000 7f4f06d82740 20 client.27017 populate_metadata read hostname 'smithi007' 2024-06-10T01:14:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.552+0000 7f4f06d82740 1 --2- 172.21.15.7:0/681383798 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5582f6610920 0x5582f6612d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.552+0000 7f4f06d82740 4 client.27017 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:14:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.552+0000 7f4f06d82740 1 -- 172.21.15.7:0/681383798 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5582f64fbc20 con 0x5582f6610920 2024-06-10T01:14:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.552+0000 7f4f01d39640 1 --2- 172.21.15.7:0/681383798 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5582f6610920 0x5582f6612d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.553+0000 7f4f01d39640 1 --2- 172.21.15.7:0/681383798 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5582f6610920 0x5582f6612d40 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddaa47fc3612785a server_cookie=18b7755b0615d84f in_seq=0 out_seq=0 2024-06-10T01:14:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.553+0000 7f4ee57fa640 10 client.27017 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:39.635 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:39 smithi007 ceph-mon[32398]: pgmap v691: 129 pgs: 129 active+clean; 107 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 2.8 MiB/s wr, 320 op/s 2024-06-10T01:14:39.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.641+0000 7f4ee57fa640 1 -- 172.21.15.7:0/681383798 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5582f64fbc20 con 0x5582f6610920 2024-06-10T01:14:39.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.641+0000 7f4ee57fa640 10 client.27017 handle_command_reply: tid=0 2024-06-10T01:14:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.641+0000 7f4f06d82740 1 client.27017 shutdown 2024-06-10T01:14:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.641+0000 7f4f06d82740 1 -- 172.21.15.7:0/681383798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f658fe10 msgr2=0x5582f659e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.641+0000 7f4f06d82740 1 --2- 172.21.15.7:0/681383798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f658fe10 0x5582f659e430 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4eec00e9c0 tx=0x7f4eec01d640 comp rx=0 tx=0).stop 2024-06-10T01:14:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.641+0000 7f4f06d82740 1 -- 172.21.15.7:0/681383798 shutdown_connections 2024-06-10T01:14:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.641+0000 7f4f06d82740 1 --2- 172.21.15.7:0/681383798 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5582f6610920 0x5582f6612d40 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.641+0000 7f4ee57fa640 0 client.27017 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.641+0000 7f4f06d82740 1 --2- 172.21.15.7:0/681383798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582f6594f80 0x5582f659f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.641+0000 7f4f06d82740 1 --2- 172.21.15.7:0/681383798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f658fe10 0x5582f659e430 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.641+0000 7f4f06d82740 1 --2- 172.21.15.7:0/681383798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582f658f420 0x5582f659def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.641+0000 7f4f06d82740 1 -- 172.21.15.7:0/681383798 >> 172.21.15.7:0/681383798 conn(0x5582f6586680 msgr2=0x5582f6584f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.642+0000 7f4f06d82740 1 -- 172.21.15.7:0/681383798 shutdown_connections 2024-06-10T01:14:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.642+0000 7f4f06d82740 1 -- 172.21.15.7:0/681383798 wait complete. 2024-06-10T01:14:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.642+0000 7f4f06d82740 20 client.27017 trim_cache size 0 max 16384 2024-06-10T01:14:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:14:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:14:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:14:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.642+0000 7f4f03f9d640 1 -- 172.21.15.7:0/4134939875 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4ee80a2b80 msgr2=0x7f4ee80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.642+0000 7f4f03f9d640 1 --2- 172.21.15.7:0/4134939875 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4ee80a2b80 0x7f4ee80a5040 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4048000 tx=0x7f4ef404a000 comp rx=0 tx=0).stop 2024-06-10T01:14:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.642+0000 7f4f03f9d640 1 -- 172.21.15.7:0/4134939875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4efc102c00 msgr2=0x7f4efc198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.642+0000 7f4f03f9d640 1 --2- 172.21.15.7:0/4134939875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4efc102c00 0x7f4efc198240 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f4eec00e9f0 tx=0x7f4eec00eeb0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.642+0000 7f4f03f9d640 1 -- 172.21.15.7:0/4134939875 shutdown_connections 2024-06-10T01:14:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.642+0000 7f4f03f9d640 1 --2- 172.21.15.7:0/4134939875 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4ee80a2b80 0x7f4ee80a5040 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.642+0000 7f4f03f9d640 1 --2- 172.21.15.7:0/4134939875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4efc1035c0 0x7f4efc19f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.642+0000 7f4f03f9d640 1 --2- 172.21.15.7:0/4134939875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4efc102c00 0x7f4efc198240 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.642+0000 7f4f03f9d640 1 --2- 172.21.15.7:0/4134939875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4efc101a10 0x7f4efc197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.642+0000 7f4f03f9d640 1 -- 172.21.15.7:0/4134939875 >> 172.21.15.7:0/4134939875 conn(0x7f4efc0fcf80 msgr2=0x7f4efc105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.643+0000 7f4f03f9d640 1 -- 172.21.15.7:0/4134939875 shutdown_connections 2024-06-10T01:14:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:39.643+0000 7f4f03f9d640 1 -- 172.21.15.7:0/4134939875 wait complete. 2024-06-10T01:14:39.741 INFO:tasks.workunit.client.0.smithi007.stdout: 1 634307 7.79 MB/sec execute 320 sec latency 84.000 ms 2024-06-10T01:14:39.838 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:39 smithi078 ceph-mon[36683]: pgmap v691: 129 pgs: 129 active+clean; 107 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 2.8 MiB/s wr, 320 op/s 2024-06-10T01:14:39.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:39 smithi175 ceph-mon[36636]: pgmap v691: 129 pgs: 129 active+clean; 107 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 2.8 MiB/s wr, 320 op/s 2024-06-10T01:14:40.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.224+0000 7f08bed31700 1 -- 172.21.15.7:0/4170218010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08b8107400 msgr2=0x7f08b81077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:40.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.224+0000 7f08bed31700 1 --2- 172.21.15.7:0/4170218010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08b8107400 0x7f08b81077e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f08ac00b0a0 tx=0x7f08ac00b3b0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.224+0000 7f08bed31700 1 -- 172.21.15.7:0/4170218010 shutdown_connections 2024-06-10T01:14:40.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.224+0000 7f08bed31700 1 --2- 172.21.15.7:0/4170218010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08b8101d50 0x7f08b810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.224+0000 7f08bed31700 1 --2- 172.21.15.7:0/4170218010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08b8107400 0x7f08b81077e0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.224+0000 7f08bed31700 1 --2- 172.21.15.7:0/4170218010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08b8101390 0x7f08b8101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.224+0000 7f08bed31700 1 -- 172.21.15.7:0/4170218010 >> 172.21.15.7:0/4170218010 conn(0x7f08b8078340 msgr2=0x7f08b8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:40.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.224+0000 7f08bed31700 1 -- 172.21.15.7:0/4170218010 shutdown_connections 2024-06-10T01:14:40.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.224+0000 7f08bed31700 1 -- 172.21.15.7:0/4170218010 wait complete. 2024-06-10T01:14:40.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.224+0000 7f08bed31700 1 Processor -- start 2024-06-10T01:14:40.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.225+0000 7f08bed31700 1 -- start start 2024-06-10T01:14:40.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.225+0000 7f08bed31700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08b8101390 0x7f08b8197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:40.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.225+0000 7f08bed31700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08b8101d50 0x7f08b8197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:40.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.225+0000 7f08bed31700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08b8107400 0x7f08b81982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:40.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.225+0000 7f08bed31700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f08b8111670 con 0x7f08b8107400 2024-06-10T01:14:40.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.225+0000 7f08bed31700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f08b81114f0 con 0x7f08b8101390 2024-06-10T01:14:40.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.225+0000 7f08bed31700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f08b81117f0 con 0x7f08b8101d50 2024-06-10T01:14:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.225+0000 7f08b7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08b8101d50 0x7f08b8197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.225+0000 7f08bd2ce700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08b8107400 0x7f08b81982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.225+0000 7f08b7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08b8101d50 0x7f08b8197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:50216/0 (socket says 172.21.15.7:50216) 2024-06-10T01:14:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.225+0000 7f08b7fff700 1 -- 172.21.15.7:0/2248756859 learned_addr learned my addr 172.21.15.7:0/2248756859 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.225+0000 7f08bcacd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08b8101390 0x7f08b8197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.225+0000 7f08b7fff700 1 -- 172.21.15.7:0/2248756859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08b8101390 msgr2=0x7f08b8197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.225+0000 7f08b7fff700 1 --2- 172.21.15.7:0/2248756859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08b8101390 0x7f08b8197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.225+0000 7f08b7fff700 1 -- 172.21.15.7:0/2248756859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08b8107400 msgr2=0x7f08b81982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.225+0000 7f08b7fff700 1 --2- 172.21.15.7:0/2248756859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08b8107400 0x7f08b81982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.225+0000 7f08b7fff700 1 -- 172.21.15.7:0/2248756859 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b81a70d0 con 0x7f08b8101d50 2024-06-10T01:14:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.225+0000 7f08bd2ce700 1 --2- 172.21.15.7:0/2248756859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08b8107400 0x7f08b81982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.226+0000 7f08b7fff700 1 --2- 172.21.15.7:0/2248756859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08b8101d50 0x7f08b8197da0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f08a400e9a0 tx=0x7f08a400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.226+0000 7f08b5ffb700 1 -- 172.21.15.7:0/2248756859 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a400ca10 con 0x7f08b8101d50 2024-06-10T01:14:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.226+0000 7f08bed31700 1 -- 172.21.15.7:0/2248756859 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b81a73c0 con 0x7f08b8101d50 2024-06-10T01:14:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.226+0000 7f08b5ffb700 1 -- 172.21.15.7:0/2248756859 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08a400cbb0 con 0x7f08b8101d50 2024-06-10T01:14:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.226+0000 7f08b5ffb700 1 -- 172.21.15.7:0/2248756859 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a4018680 con 0x7f08b8101d50 2024-06-10T01:14:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.226+0000 7f08bed31700 1 -- 172.21.15.7:0/2248756859 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b81a7950 con 0x7f08b8101d50 2024-06-10T01:14:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.228+0000 7f08b5ffb700 1 -- 172.21.15.7:0/2248756859 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f08a4010430 con 0x7f08b8101d50 2024-06-10T01:14:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.229+0000 7f08b5ffb700 1 --2- 172.21.15.7:0/2248756859 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f08a80a2ca0 0x7f08a80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.229+0000 7f08bcacd700 1 --2- 172.21.15.7:0/2248756859 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f08a80a2ca0 0x7f08a80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.229+0000 7f08b5ffb700 1 -- 172.21.15.7:0/2248756859 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f08a40cf5e0 con 0x7f08b8101d50 2024-06-10T01:14:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.230+0000 7f08c95b4b80 1 Processor -- start 2024-06-10T01:14:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.230+0000 7f08bcacd700 1 --2- 172.21.15.7:0/2248756859 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f08a80a2ca0 0x7f08a80a5160 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f08ac009e60 tx=0x7f08ac00b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.230+0000 7f08c95b4b80 1 -- start start 2024-06-10T01:14:40.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.230+0000 7f08c95b4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e32b99640 0x560e32b99a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:40.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.230+0000 7f08c95b4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e32b99f60 0x560e32b939e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:40.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.230+0000 7f08c95b4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e32b93f20 0x560e32b94340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:40.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.230+0000 7f08c95b4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560e32ae5810 con 0x560e32b99640 2024-06-10T01:14:40.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.230+0000 7f08c95b4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560e32a8d720 con 0x560e32b93f20 2024-06-10T01:14:40.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.230+0000 7f08c95b4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560e32ae17d0 con 0x560e32b99f60 2024-06-10T01:14:40.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.231+0000 7f08bcacd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e32b99640 0x560e32b99a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:40.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.231+0000 7f08bcacd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e32b99640 0x560e32b99a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60974/0 (socket says 172.21.15.7:60974) 2024-06-10T01:14:40.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.231+0000 7f08bcacd700 1 -- 172.21.15.7:0/3966969570 learned_addr learned my addr 172.21.15.7:0/3966969570 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:40.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.231+0000 7f08b7fff700 1 --2- 172.21.15.7:0/3966969570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e32b99f60 0x560e32b939e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:40.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.231+0000 7f08bd2ce700 1 --2- 172.21.15.7:0/3966969570 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e32b93f20 0x560e32b94340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:40.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.231+0000 7f08bd2ce700 1 -- 172.21.15.7:0/3966969570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e32b99f60 msgr2=0x560e32b939e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:40.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.231+0000 7f08bd2ce700 1 --2- 172.21.15.7:0/3966969570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e32b99f60 0x560e32b939e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.231+0000 7f08bd2ce700 1 -- 172.21.15.7:0/3966969570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e32b99640 msgr2=0x560e32b99a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:40.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.231+0000 7f08bd2ce700 1 --2- 172.21.15.7:0/3966969570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e32b99640 0x560e32b99a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.231+0000 7f08bd2ce700 1 -- 172.21.15.7:0/3966969570 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e32a83f50 con 0x560e32b93f20 2024-06-10T01:14:40.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.231+0000 7f08bd2ce700 1 --2- 172.21.15.7:0/3966969570 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e32b93f20 0x560e32b94340 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f08b0009fc0 tx=0x7f08b0011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ef75fd6298defed server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:40.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.231+0000 7f08a2ffd700 1 -- 172.21.15.7:0/3966969570 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b0004310 con 0x560e32b93f20 2024-06-10T01:14:40.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.232+0000 7f08a2ffd700 1 -- 172.21.15.7:0/3966969570 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b00044b0 con 0x560e32b93f20 2024-06-10T01:14:40.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.232+0000 7f08a2ffd700 1 -- 172.21.15.7:0/3966969570 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b0003680 con 0x560e32b93f20 2024-06-10T01:14:40.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.232+0000 7f08c95b4b80 1 -- 172.21.15.7:0/3966969570 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e32b93f20 msgr2=0x560e32b94340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:40.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.232+0000 7f08c95b4b80 1 --2- 172.21.15.7:0/3966969570 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e32b93f20 0x560e32b94340 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f08b0009fc0 tx=0x7f08b0011720 comp rx=0 tx=0).stop 2024-06-10T01:14:40.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.233+0000 7f08c95b4b80 1 -- 172.21.15.7:0/3966969570 shutdown_connections 2024-06-10T01:14:40.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.233+0000 7f08c95b4b80 1 --2- 172.21.15.7:0/3966969570 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e32b93f20 0x560e32b94340 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.233+0000 7f08c95b4b80 1 --2- 172.21.15.7:0/3966969570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e32b99640 0x560e32b99a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.233+0000 7f08c95b4b80 1 --2- 172.21.15.7:0/3966969570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e32b99f60 0x560e32b939e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.233+0000 7f08c95b4b80 1 -- 172.21.15.7:0/3966969570 >> 172.21.15.7:0/3966969570 conn(0x560e32abecc0 msgr2=0x560e32b8e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:40.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.233+0000 7f08c95b4b80 1 -- 172.21.15.7:0/3966969570 shutdown_connections 2024-06-10T01:14:40.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.233+0000 7f08c95b4b80 1 -- 172.21.15.7:0/3966969570 wait complete. 2024-06-10T01:14:40.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.234+0000 7f08c95b4b80 1 -- start start 2024-06-10T01:14:40.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.234+0000 7f08c95b4b80 1 Processor -- start 2024-06-10T01:14:40.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.234+0000 7f08c95b4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e32b93f20 0x560e32b9fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:40.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.234+0000 7f08c95b4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e32b99640 0x560e32b9d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:40.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.234+0000 7f08c95b4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e32b99f60 0x560e32b9d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:40.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.234+0000 7f08c95b4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560e32ba09d0 con 0x560e32b99640 2024-06-10T01:14:40.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.234+0000 7f08c95b4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560e32a85bf0 con 0x560e32b93f20 2024-06-10T01:14:40.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.234+0000 7f08c95b4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560e32ba0b50 con 0x560e32b99f60 2024-06-10T01:14:40.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.234+0000 7f08c95b4b80 10 client.0 _ll_register_callbacks cb 0x560e32a7f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:14:40.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.234+0000 7f08b7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e32b99640 0x560e32b9d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:40.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.234+0000 7f08bcacd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e32b93f20 0x560e32b9fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:40.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.234+0000 7f08bcacd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e32b93f20 0x560e32b9fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:58624/0 (socket says 172.21.15.7:58624) 2024-06-10T01:14:40.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.234+0000 7f08bcacd700 1 -- 172.21.15.7:0/118801225 learned_addr learned my addr 172.21.15.7:0/118801225 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:40.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.235+0000 7f08bd2ce700 1 --2- 172.21.15.7:0/118801225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e32b99f60 0x560e32b9d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:40.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.235+0000 7f08bcacd700 1 -- 172.21.15.7:0/118801225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e32b99f60 msgr2=0x560e32b9d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:40.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.235+0000 7f08bcacd700 1 --2- 172.21.15.7:0/118801225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e32b99f60 0x560e32b9d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.235+0000 7f08bcacd700 1 -- 172.21.15.7:0/118801225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e32b99640 msgr2=0x560e32b9d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:40.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.235+0000 7f08bcacd700 1 --2- 172.21.15.7:0/118801225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e32b99640 0x560e32b9d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.235+0000 7f08bcacd700 1 -- 172.21.15.7:0/118801225 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e32a348d0 con 0x560e32b93f20 2024-06-10T01:14:40.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.235+0000 7f08bcacd700 1 --2- 172.21.15.7:0/118801225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e32b93f20 0x560e32b9fd80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f08ac003a80 tx=0x7f08ac0099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6e98409be13bfa5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:40.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.235+0000 7f0897fff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:14:40.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.235+0000 7f0897fff700 1 -- 172.21.15.7:0/118801225 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac0078f0 con 0x560e32b93f20 2024-06-10T01:14:40.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.235+0000 7f08c95b4b80 1 -- 172.21.15.7:0/118801225 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08b00044b0 con 0x560e32b93f20 2024-06-10T01:14:40.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.235+0000 7f0897fff700 1 -- 172.21.15.7:0/118801225 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f08ac007a90 con 0x560e32b93f20 2024-06-10T01:14:40.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.235+0000 7f0897fff700 1 -- 172.21.15.7:0/118801225 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f08ac03b050 con 0x560e32b93f20 2024-06-10T01:14:40.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.236+0000 7f08c95b4b80 10 client.27029 fetch_fsmap learned FSMap version 35 2024-06-10T01:14:40.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.236+0000 7f08c95b4b80 1 -- 172.21.15.7:0/118801225 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560e32bab1f0 con 0x560e32b93f20 2024-06-10T01:14:40.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.236+0000 7f0897fff700 1 -- 172.21.15.7:0/118801225 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b00044b0 con 0x560e32b93f20 2024-06-10T01:14:40.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.236+0000 7f0897fff700 1 -- 172.21.15.7:0/118801225 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f08ac011070 con 0x560e32b93f20 2024-06-10T01:14:40.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.236+0000 7f08c95b4b80 10 client.27029 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:14:40.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.236+0000 7f08c95b4b80 10 client.27029 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:14:40.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.236+0000 7f08c95b4b80 20 client.27029 populate_metadata read hostname 'smithi007' 2024-06-10T01:14:40.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.236+0000 7f08c95b4b80 1 --2- 172.21.15.7:0/118801225 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x560e32bad780 0x560e32bafb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:40.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.236+0000 7f08c95b4b80 4 client.27029 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:14:40.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.236+0000 7f08c95b4b80 1 -- 172.21.15.7:0/118801225 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e32af2d10 con 0x560e32bad780 2024-06-10T01:14:40.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.236+0000 7f08b7fff700 1 --2- 172.21.15.7:0/118801225 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x560e32bad780 0x560e32bafb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:40.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.237+0000 7f0897fff700 10 client.27029 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:14:40.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.237+0000 7f08b7fff700 1 --2- 172.21.15.7:0/118801225 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x560e32bad780 0x560e32bafb70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=540c8951b5e60484 server_cookie=35b1778955b4e9 in_seq=0 out_seq=0 2024-06-10T01:14:40.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.237+0000 7f0897fff700 1 -- 172.21.15.7:0/118801225 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560e32af2d10 con 0x560e32bad780 2024-06-10T01:14:40.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.237+0000 7f0897fff700 10 client.27029 handle_command_reply: tid=0 2024-06-10T01:14:40.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.238+0000 7f08c95b4b80 1 client.27029 shutdown 2024-06-10T01:14:40.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.238+0000 7f08c95b4b80 1 -- 172.21.15.7:0/118801225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e32b93f20 msgr2=0x560e32b9fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:40.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.238+0000 7f08c95b4b80 1 --2- 172.21.15.7:0/118801225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e32b93f20 0x560e32b9fd80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f08ac003a80 tx=0x7f08ac0099e0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.238+0000 7f08c95b4b80 1 -- 172.21.15.7:0/118801225 shutdown_connections 2024-06-10T01:14:40.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.238+0000 7f08c95b4b80 1 --2- 172.21.15.7:0/118801225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e32b99f60 0x560e32b9d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.238+0000 7f08c95b4b80 1 --2- 172.21.15.7:0/118801225 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x560e32bad780 0x560e32bafb70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.238+0000 7f08c95b4b80 1 --2- 172.21.15.7:0/118801225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e32b93f20 0x560e32b9fd80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.238+0000 7f0897fff700 0 client.27029 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:14:40.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.238+0000 7f08c95b4b80 1 --2- 172.21.15.7:0/118801225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e32b99640 0x560e32b9d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.238+0000 7f08c95b4b80 1 -- 172.21.15.7:0/118801225 >> 172.21.15.7:0/118801225 conn(0x560e32abecc0 msgr2=0x560e32b929d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:40.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.238+0000 7f08c95b4b80 1 -- 172.21.15.7:0/118801225 shutdown_connections 2024-06-10T01:14:40.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.238+0000 7f08c95b4b80 1 -- 172.21.15.7:0/118801225 wait complete. 2024-06-10T01:14:40.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.238+0000 7f08c95b4b80 20 client.27029 trim_cache size 0 max 16384 2024-06-10T01:14:40.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.255+0000 7f08c95b4b80 1 Processor -- start 2024-06-10T01:14:40.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.255+0000 7f08c95b4b80 1 -- start start 2024-06-10T01:14:40.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.255+0000 7f08c95b4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e32b93f20 0x560e32ba9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:40.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.255+0000 7f08c95b4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e32b99640 0x560e32b939e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:40.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.255+0000 7f08c95b4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e32b99f60 0x560e32b9abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:40.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.255+0000 7f08c95b4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f08b81b2e00 con 0x560e32b99f60 2024-06-10T01:14:40.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.255+0000 7f08c95b4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f08b81b0630 con 0x560e32b99640 2024-06-10T01:14:40.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.255+0000 7f08c95b4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f08b81ae480 con 0x560e32b93f20 2024-06-10T01:14:40.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.255+0000 7f08bd2ce700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e32b99f60 0x560e32b9abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:40.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.255+0000 7f08bd2ce700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e32b99f60 0x560e32b9abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60986/0 (socket says 172.21.15.7:60986) 2024-06-10T01:14:40.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.255+0000 7f08bd2ce700 1 -- 172.21.15.7:0/2919842690 learned_addr learned my addr 172.21.15.7:0/2919842690 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:40.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.255+0000 7f08bcacd700 1 --2- 172.21.15.7:0/2919842690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e32b93f20 0x560e32ba9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:40.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.256+0000 7f08b7fff700 1 --2- 172.21.15.7:0/2919842690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e32b99640 0x560e32b939e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:40.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.256+0000 7f08bd2ce700 1 -- 172.21.15.7:0/2919842690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e32b93f20 msgr2=0x560e32ba9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:40.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.256+0000 7f08bd2ce700 1 --2- 172.21.15.7:0/2919842690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e32b93f20 0x560e32ba9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.256+0000 7f08bd2ce700 1 -- 172.21.15.7:0/2919842690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e32b99640 msgr2=0x560e32b939e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:40.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.256+0000 7f08bd2ce700 1 --2- 172.21.15.7:0/2919842690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e32b99640 0x560e32b939e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.256+0000 7f08bd2ce700 1 -- 172.21.15.7:0/2919842690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e32bb01d0 con 0x560e32b99f60 2024-06-10T01:14:40.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.256+0000 7f08bd2ce700 1 --2- 172.21.15.7:0/2919842690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e32b99f60 0x560e32b9abd0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f08b0019000 tx=0x7f08b0005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db72be04266aac64 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:40.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.256+0000 7f08a27fc700 1 -- 172.21.15.7:0/2919842690 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b0027070 con 0x560e32b99f60 2024-06-10T01:14:40.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.256+0000 7f08a27fc700 1 -- 172.21.15.7:0/2919842690 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b00050c0 con 0x560e32b99f60 2024-06-10T01:14:40.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.256+0000 7f08a27fc700 1 -- 172.21.15.7:0/2919842690 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b0003e60 con 0x560e32b99f60 2024-06-10T01:14:40.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.257+0000 7f08c95b4b80 1 -- 172.21.15.7:0/2919842690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e32b99f60 msgr2=0x560e32b9abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:40.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.257+0000 7f08c95b4b80 1 --2- 172.21.15.7:0/2919842690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e32b99f60 0x560e32b9abd0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f08b0019000 tx=0x7f08b0005ce0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.257+0000 7f08c95b4b80 1 -- 172.21.15.7:0/2919842690 shutdown_connections 2024-06-10T01:14:40.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.257+0000 7f08c95b4b80 1 --2- 172.21.15.7:0/2919842690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e32b99f60 0x560e32b9abd0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.257+0000 7f08c95b4b80 1 --2- 172.21.15.7:0/2919842690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e32b93f20 0x560e32ba9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.257+0000 7f08c95b4b80 1 --2- 172.21.15.7:0/2919842690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e32b99640 0x560e32b939e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.257+0000 7f08c95b4b80 1 -- 172.21.15.7:0/2919842690 >> 172.21.15.7:0/2919842690 conn(0x560e32abecc0 msgr2=0x560e32b9b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:40.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.257+0000 7f08c95b4b80 1 -- 172.21.15.7:0/2919842690 shutdown_connections 2024-06-10T01:14:40.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.257+0000 7f08c95b4b80 1 -- 172.21.15.7:0/2919842690 wait complete. 2024-06-10T01:14:40.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.257+0000 7f08c95b4b80 1 -- start start 2024-06-10T01:14:40.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.258+0000 7f08c95b4b80 1 Processor -- start 2024-06-10T01:14:40.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.258+0000 7f08c95b4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e32b93f20 0x560e32ba6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:40.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.258+0000 7f08c95b4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e32b99640 0x560e32ba1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:40.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.258+0000 7f08c95b4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e32b99f60 0x560e32ba1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:40.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.258+0000 7f08c95b4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f08b81aeaf0 con 0x560e32b99640 2024-06-10T01:14:40.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.258+0000 7f08c95b4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f08b81aff60 con 0x560e32b93f20 2024-06-10T01:14:40.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.258+0000 7f08c95b4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f08b81af1c0 con 0x560e32b99f60 2024-06-10T01:14:40.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.258+0000 7f08c95b4b80 10 client.0 _ll_register_callbacks cb 0x560e32a7f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:14:40.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.258+0000 7f08b7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e32b99640 0x560e32ba1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:40.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.258+0000 7f08b7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e32b99640 0x560e32ba1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60992/0 (socket says 172.21.15.7:60992) 2024-06-10T01:14:40.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.258+0000 7f08b7fff700 1 -- 172.21.15.7:0/1932834403 learned_addr learned my addr 172.21.15.7:0/1932834403 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:40.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.258+0000 7f08bcacd700 1 --2- 172.21.15.7:0/1932834403 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e32b93f20 0x560e32ba6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:40.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.258+0000 7f08bd2ce700 1 --2- 172.21.15.7:0/1932834403 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e32b99f60 0x560e32ba1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:40.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.259+0000 7f08b7fff700 1 -- 172.21.15.7:0/1932834403 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e32b99f60 msgr2=0x560e32ba1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:40.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.259+0000 7f08b7fff700 1 --2- 172.21.15.7:0/1932834403 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e32b99f60 0x560e32ba1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.259+0000 7f08b7fff700 1 -- 172.21.15.7:0/1932834403 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e32b93f20 msgr2=0x560e32ba6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:40.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.259+0000 7f08b7fff700 1 --2- 172.21.15.7:0/1932834403 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e32b93f20 0x560e32ba6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.259+0000 7f08b7fff700 1 -- 172.21.15.7:0/1932834403 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e32ba2650 con 0x560e32b99640 2024-06-10T01:14:40.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.259+0000 7f08bd2ce700 1 --2- 172.21.15.7:0/1932834403 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e32b99f60 0x560e32ba1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:40.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.259+0000 7f08bcacd700 1 --2- 172.21.15.7:0/1932834403 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e32b93f20 0x560e32ba6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:40.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.259+0000 7f08b7fff700 1 --2- 172.21.15.7:0/1932834403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e32b99640 0x560e32ba1900 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f08a409b760 tx=0x7f08a401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985278113b4a1159 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:40.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.259+0000 7f0897fff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:14:40.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.259+0000 7f0897fff700 1 -- 172.21.15.7:0/1932834403 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a40a46d0 con 0x560e32b99640 2024-06-10T01:14:40.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.259+0000 7f0897fff700 1 -- 172.21.15.7:0/1932834403 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f08a400cd60 con 0x560e32b99640 2024-06-10T01:14:40.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.259+0000 7f08c95b4b80 1 -- 172.21.15.7:0/1932834403 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08b00050c0 con 0x560e32b99640 2024-06-10T01:14:40.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.260+0000 7f0897fff700 1 -- 172.21.15.7:0/1932834403 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f08a40ba300 con 0x560e32b99640 2024-06-10T01:14:40.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.260+0000 7f08c95b4b80 10 client.17643 fetch_fsmap learned FSMap version 35 2024-06-10T01:14:40.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.260+0000 7f08c95b4b80 1 -- 172.21.15.7:0/1932834403 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560e32c18430 con 0x560e32b99640 2024-06-10T01:14:40.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.260+0000 7f0897fff700 1 -- 172.21.15.7:0/1932834403 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b00050c0 con 0x560e32b99640 2024-06-10T01:14:40.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.260+0000 7f0897fff700 1 -- 172.21.15.7:0/1932834403 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f08a4010720 con 0x560e32b99640 2024-06-10T01:14:40.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.260+0000 7f08c95b4b80 10 client.17643 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:14:40.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.260+0000 7f08c95b4b80 10 client.17643 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:14:40.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.260+0000 7f08c95b4b80 20 client.17643 populate_metadata read hostname 'smithi007' 2024-06-10T01:14:40.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.260+0000 7f08c95b4b80 1 --2- 172.21.15.7:0/1932834403 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x560e32c18ad0 0x560e32c1aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:40.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.260+0000 7f08c95b4b80 4 client.17643 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:14:40.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.260+0000 7f08c95b4b80 1 -- 172.21.15.7:0/1932834403 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560e32aed290 con 0x560e32c18ad0 2024-06-10T01:14:40.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.260+0000 7f08bcacd700 1 --2- 172.21.15.7:0/1932834403 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x560e32c18ad0 0x560e32c1aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:40.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.261+0000 7f08bcacd700 1 --2- 172.21.15.7:0/1932834403 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x560e32c18ad0 0x560e32c1aef0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e18e8bad24b1a082 server_cookie=fd18d3dedd9c3136 in_seq=0 out_seq=0 2024-06-10T01:14:40.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.261+0000 7f0897fff700 10 client.17643 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:14:40.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.261+0000 7f0897fff700 1 -- 172.21.15.7:0/1932834403 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560e32aed290 con 0x560e32c18ad0 2024-06-10T01:14:40.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.261+0000 7f0897fff700 10 client.17643 handle_command_reply: tid=0 2024-06-10T01:14:40.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.261+0000 7f08c95b4b80 1 client.17643 shutdown 2024-06-10T01:14:40.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.261+0000 7f08c95b4b80 1 -- 172.21.15.7:0/1932834403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e32b99640 msgr2=0x560e32ba1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:40.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.261+0000 7f08c95b4b80 1 --2- 172.21.15.7:0/1932834403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e32b99640 0x560e32ba1900 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f08a409b760 tx=0x7f08a401a040 comp rx=0 tx=0).stop 2024-06-10T01:14:40.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.261+0000 7f08c95b4b80 1 -- 172.21.15.7:0/1932834403 shutdown_connections 2024-06-10T01:14:40.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.261+0000 7f08c95b4b80 1 --2- 172.21.15.7:0/1932834403 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e32b99f60 0x560e32ba1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.261+0000 7f08c95b4b80 1 --2- 172.21.15.7:0/1932834403 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x560e32c18ad0 0x560e32c1aef0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.261+0000 7f08c95b4b80 1 --2- 172.21.15.7:0/1932834403 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e32b93f20 0x560e32ba6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.261+0000 7f08c95b4b80 1 --2- 172.21.15.7:0/1932834403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e32b99640 0x560e32ba1900 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.261+0000 7f08c95b4b80 1 -- 172.21.15.7:0/1932834403 >> 172.21.15.7:0/1932834403 conn(0x560e32abecc0 msgr2=0x560e32b98a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:40.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.261+0000 7f0897fff700 0 client.17643 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:14:40.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.261+0000 7f08c95b4b80 1 -- 172.21.15.7:0/1932834403 shutdown_connections 2024-06-10T01:14:40.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.261+0000 7f08c95b4b80 1 -- 172.21.15.7:0/1932834403 wait complete. 2024-06-10T01:14:40.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.262+0000 7f08c95b4b80 20 client.17643 trim_cache size 0 max 16384 2024-06-10T01:14:40.261 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:14:40.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.262+0000 7f08bed31700 1 -- 172.21.15.7:0/2248756859 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f08a80a2ca0 msgr2=0x7f08a80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:40.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.262+0000 7f08bed31700 1 --2- 172.21.15.7:0/2248756859 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f08a80a2ca0 0x7f08a80a5160 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f08ac009e60 tx=0x7f08ac00b700 comp rx=0 tx=0).stop 2024-06-10T01:14:40.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.262+0000 7f08bed31700 1 -- 172.21.15.7:0/2248756859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08b8101d50 msgr2=0x7f08b8197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:40.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.262+0000 7f08bed31700 1 --2- 172.21.15.7:0/2248756859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08b8101d50 0x7f08b8197da0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f08a400e9a0 tx=0x7f08a400ecb0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.262+0000 7f08bed31700 1 -- 172.21.15.7:0/2248756859 shutdown_connections 2024-06-10T01:14:40.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.262+0000 7f08bed31700 1 --2- 172.21.15.7:0/2248756859 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f08a80a2ca0 0x7f08a80a5160 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.262+0000 7f08bed31700 1 --2- 172.21.15.7:0/2248756859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08b8107400 0x7f08b81982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.262+0000 7f08bed31700 1 --2- 172.21.15.7:0/2248756859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08b8101390 0x7f08b8197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.262+0000 7f08bed31700 1 --2- 172.21.15.7:0/2248756859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08b8101d50 0x7f08b8197da0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:40.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.262+0000 7f08bed31700 1 -- 172.21.15.7:0/2248756859 >> 172.21.15.7:0/2248756859 conn(0x7f08b8078340 msgr2=0x7f08b8100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:40.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.262+0000 7f08bed31700 1 -- 172.21.15.7:0/2248756859 shutdown_connections 2024-06-10T01:14:40.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:40.262+0000 7f08bed31700 1 -- 172.21.15.7:0/2248756859 wait complete. 2024-06-10T01:14:40.742 INFO:tasks.workunit.client.0.smithi007.stdout: 1 636126 7.81 MB/sec execute 321 sec latency 60.569 ms 2024-06-10T01:14:41.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:41 smithi007 ceph-mon[32398]: pgmap v692: 129 pgs: 129 active+clean; 110 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 2.8 MiB/s wr, 349 op/s 2024-06-10T01:14:41.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:41 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:14:41.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:41 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:14:41.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:41 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:14:41.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:41 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:14:41.540 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:14:41.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:41 smithi175 ceph-mon[36636]: pgmap v692: 129 pgs: 129 active+clean; 110 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 2.8 MiB/s wr, 349 op/s 2024-06-10T01:14:41.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:41 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:14:41.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:41 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:14:41.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:41 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:14:41.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:41 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:14:41.741 INFO:tasks.workunit.client.0.smithi007.stdout: 1 638268 7.81 MB/sec execute 322 sec latency 14.861 ms 2024-06-10T01:14:41.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:41 smithi078 ceph-mon[36683]: pgmap v692: 129 pgs: 129 active+clean; 110 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 2.8 MiB/s wr, 349 op/s 2024-06-10T01:14:41.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:41 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:14:41.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:41 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:14:41.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:41 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:14:41.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:41 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:14:41.841 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:14:42.742 INFO:tasks.workunit.client.0.smithi007.stdout: 1 639989 7.82 MB/sec execute 323 sec latency 10.826 ms 2024-06-10T01:14:43.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.468+0000 7fb649ebe700 1 -- 172.21.15.7:0/3165858276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb644106720 msgr2=0x7fb64410e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:43.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.468+0000 7fb649ebe700 1 --2- 172.21.15.7:0/3165858276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb644106720 0x7fb64410e2c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00b0d0 tx=0x7fb63c00b3e0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.468+0000 7fb649ebe700 1 -- 172.21.15.7:0/3165858276 shutdown_connections 2024-06-10T01:14:43.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.468+0000 7fb649ebe700 1 --2- 172.21.15.7:0/3165858276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb644106720 0x7fb64410e2c0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.468+0000 7fb649ebe700 1 --2- 172.21.15.7:0/3165858276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6441019d0 0x7fb644101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.468+0000 7fb649ebe700 1 --2- 172.21.15.7:0/3165858276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb644102380 0x7fb644105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.468+0000 7fb649ebe700 1 -- 172.21.15.7:0/3165858276 >> 172.21.15.7:0/3165858276 conn(0x7fb644078340 msgr2=0x7fb644078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:43.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.468+0000 7fb649ebe700 1 -- 172.21.15.7:0/3165858276 shutdown_connections 2024-06-10T01:14:43.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.468+0000 7fb649ebe700 1 -- 172.21.15.7:0/3165858276 wait complete. 2024-06-10T01:14:43.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.469+0000 7fb649ebe700 1 Processor -- start 2024-06-10T01:14:43.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.469+0000 7fb649ebe700 1 -- start start 2024-06-10T01:14:43.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.470+0000 7fb649ebe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6441019d0 0x7fb64419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.470+0000 7fb649ebe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb644102380 0x7fb64419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.470+0000 7fb649ebe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb644106720 0x7fb644198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.470+0000 7fb649ebe700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb644111700 con 0x7fb6441019d0 2024-06-10T01:14:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.470+0000 7fb649ebe700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb644111580 con 0x7fb644102380 2024-06-10T01:14:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.470+0000 7fb649ebe700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb644111880 con 0x7fb644106720 2024-06-10T01:14:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.470+0000 7fb6437fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6441019d0 0x7fb64419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.470+0000 7fb6437fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6441019d0 0x7fb64419e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45408/0 (socket says 172.21.15.7:45408) 2024-06-10T01:14:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.470+0000 7fb6437fe700 1 -- 172.21.15.7:0/701609958 learned_addr learned my addr 172.21.15.7:0/701609958 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.470+0000 7fb642ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb644102380 0x7fb64419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.470+0000 7fb643fff700 1 --2- 172.21.15.7:0/701609958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb644106720 0x7fb644198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.470+0000 7fb6437fe700 1 -- 172.21.15.7:0/701609958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb644106720 msgr2=0x7fb644198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.470+0000 7fb6437fe700 1 --2- 172.21.15.7:0/701609958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb644106720 0x7fb644198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.470+0000 7fb6437fe700 1 -- 172.21.15.7:0/701609958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb644102380 msgr2=0x7fb64419ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:43.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.470+0000 7fb6437fe700 1 --2- 172.21.15.7:0/701609958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb644102380 0x7fb64419ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.470+0000 7fb6437fe700 1 -- 172.21.15.7:0/701609958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb644199170 con 0x7fb6441019d0 2024-06-10T01:14:43.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.470+0000 7fb642ffd700 1 --2- 172.21.15.7:0/701609958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb644102380 0x7fb64419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:43.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.470+0000 7fb6437fe700 1 --2- 172.21.15.7:0/701609958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6441019d0 0x7fb64419e850 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fb63400e9c0 tx=0x7fb63400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:43.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.471+0000 7fb640ff9700 1 -- 172.21.15.7:0/701609958 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63400cbd0 con 0x7fb6441019d0 2024-06-10T01:14:43.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.471+0000 7fb649ebe700 1 -- 172.21.15.7:0/701609958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb644199460 con 0x7fb6441019d0 2024-06-10T01:14:43.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.471+0000 7fb640ff9700 1 -- 172.21.15.7:0/701609958 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb63400ee60 con 0x7fb6441019d0 2024-06-10T01:14:43.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.471+0000 7fb640ff9700 1 -- 172.21.15.7:0/701609958 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb634005630 con 0x7fb6441019d0 2024-06-10T01:14:43.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.471+0000 7fb649ebe700 1 -- 172.21.15.7:0/701609958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb644102fc0 con 0x7fb6441019d0 2024-06-10T01:14:43.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.473+0000 7fb640ff9700 1 -- 172.21.15.7:0/701609958 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb634004750 con 0x7fb6441019d0 2024-06-10T01:14:43.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.474+0000 7fb640ff9700 1 --2- 172.21.15.7:0/701609958 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb62c0a2c20 0x7fb62c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:43.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.474+0000 7fb640ff9700 1 -- 172.21.15.7:0/701609958 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb6340cf6b0 con 0x7fb6441019d0 2024-06-10T01:14:43.469 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:43 smithi007 ceph-mon[32398]: pgmap v693: 129 pgs: 129 active+clean; 106 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 2.2 MiB/s wr, 277 op/s 2024-06-10T01:14:43.469 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:43 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:14:43.469 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:43 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:14:43.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.474+0000 7fb642ffd700 1 --2- 172.21.15.7:0/701609958 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb62c0a2c20 0x7fb62c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:43.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.475+0000 7fb642ffd700 1 --2- 172.21.15.7:0/701609958 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb62c0a2c20 0x7fb62c0a50e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb644107c20 tx=0x7fb638009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:43.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.475+0000 7fb654741b80 1 Processor -- start 2024-06-10T01:14:43.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.475+0000 7fb654741b80 1 -- start start 2024-06-10T01:14:43.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.475+0000 7fb654741b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6985dd640 0x55b6985dda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:43.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.475+0000 7fb654741b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b6985ddf60 0x55b6985d79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:43.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.475+0000 7fb654741b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6985d7f20 0x55b6985d8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:43.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.475+0000 7fb654741b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b698529810 con 0x55b6985ddf60 2024-06-10T01:14:43.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.475+0000 7fb654741b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b6984d1720 con 0x55b6985d7f20 2024-06-10T01:14:43.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.475+0000 7fb654741b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b6985257d0 con 0x55b6985dd640 2024-06-10T01:14:43.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.476+0000 7fb6437fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6985dd640 0x55b6985dda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:43.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.476+0000 7fb642ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b6985ddf60 0x55b6985d79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:43.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.476+0000 7fb6437fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6985dd640 0x55b6985dda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52646/0 (socket says 172.21.15.7:52646) 2024-06-10T01:14:43.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.476+0000 7fb6437fe700 1 -- 172.21.15.7:0/1572579759 learned_addr learned my addr 172.21.15.7:0/1572579759 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:43.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.476+0000 7fb643fff700 1 --2- 172.21.15.7:0/1572579759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6985d7f20 0x55b6985d8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:43.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.476+0000 7fb6437fe700 1 -- 172.21.15.7:0/1572579759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6985d7f20 msgr2=0x55b6985d8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:43.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.476+0000 7fb6437fe700 1 --2- 172.21.15.7:0/1572579759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6985d7f20 0x55b6985d8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.476+0000 7fb6437fe700 1 -- 172.21.15.7:0/1572579759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b6985ddf60 msgr2=0x55b6985d79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:43.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.476+0000 7fb6437fe700 1 --2- 172.21.15.7:0/1572579759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b6985ddf60 0x55b6985d79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.476+0000 7fb6437fe700 1 -- 172.21.15.7:0/1572579759 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6984c7f50 con 0x55b6985dd640 2024-06-10T01:14:43.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.476+0000 7fb6437fe700 1 --2- 172.21.15.7:0/1572579759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6985dd640 0x55b6985dda20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb63409bac0 tx=0x7fb63409b780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ec213c6143a6736 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:43.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.476+0000 7fb629ffb700 1 -- 172.21.15.7:0/1572579759 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63409ce60 con 0x55b6985dd640 2024-06-10T01:14:43.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.476+0000 7fb629ffb700 1 -- 172.21.15.7:0/1572579759 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb63409fee0 con 0x55b6985dd640 2024-06-10T01:14:43.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.476+0000 7fb629ffb700 1 -- 172.21.15.7:0/1572579759 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63409bd90 con 0x55b6985dd640 2024-06-10T01:14:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.477+0000 7fb654741b80 1 -- 172.21.15.7:0/1572579759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6985dd640 msgr2=0x55b6985dda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.477+0000 7fb654741b80 1 --2- 172.21.15.7:0/1572579759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6985dd640 0x55b6985dda20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb63409bac0 tx=0x7fb63409b780 comp rx=0 tx=0).stop 2024-06-10T01:14:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.477+0000 7fb654741b80 1 -- 172.21.15.7:0/1572579759 shutdown_connections 2024-06-10T01:14:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.477+0000 7fb654741b80 1 --2- 172.21.15.7:0/1572579759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6985d7f20 0x55b6985d8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.477+0000 7fb654741b80 1 --2- 172.21.15.7:0/1572579759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6985dd640 0x55b6985dda20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.477+0000 7fb654741b80 1 --2- 172.21.15.7:0/1572579759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b6985ddf60 0x55b6985d79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.477+0000 7fb654741b80 1 -- 172.21.15.7:0/1572579759 >> 172.21.15.7:0/1572579759 conn(0x55b698502cc0 msgr2=0x55b6985d2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.478+0000 7fb654741b80 1 -- 172.21.15.7:0/1572579759 shutdown_connections 2024-06-10T01:14:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.478+0000 7fb654741b80 1 -- 172.21.15.7:0/1572579759 wait complete. 2024-06-10T01:14:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.478+0000 7fb654741b80 1 -- start start 2024-06-10T01:14:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.478+0000 7fb654741b80 1 Processor -- start 2024-06-10T01:14:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.479+0000 7fb654741b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b6985d7f20 0x55b6985e3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.479+0000 7fb654741b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6985dd640 0x55b6985e1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.479+0000 7fb654741b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6985ddf60 0x55b6985e15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.479+0000 7fb654741b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b6985e49d0 con 0x55b6985d7f20 2024-06-10T01:14:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.479+0000 7fb654741b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b6984c9bf0 con 0x55b6985dd640 2024-06-10T01:14:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.479+0000 7fb654741b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b6985e4b50 con 0x55b6985ddf60 2024-06-10T01:14:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.479+0000 7fb654741b80 10 client.0 _ll_register_callbacks cb 0x55b6984c3920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:14:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.479+0000 7fb643fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6985ddf60 0x55b6985e15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:43.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.479+0000 7fb643fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6985ddf60 0x55b6985e15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52660/0 (socket says 172.21.15.7:52660) 2024-06-10T01:14:43.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.479+0000 7fb643fff700 1 -- 172.21.15.7:0/306022598 learned_addr learned my addr 172.21.15.7:0/306022598 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:43.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.479+0000 7fb642ffd700 1 --2- 172.21.15.7:0/306022598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6985dd640 0x55b6985e1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:43.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.479+0000 7fb6437fe700 1 --2- 172.21.15.7:0/306022598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b6985d7f20 0x55b6985e3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:43.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.480+0000 7fb643fff700 1 -- 172.21.15.7:0/306022598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6985dd640 msgr2=0x55b6985e1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:43.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.480+0000 7fb643fff700 1 --2- 172.21.15.7:0/306022598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6985dd640 0x55b6985e1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.480+0000 7fb643fff700 1 -- 172.21.15.7:0/306022598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b6985d7f20 msgr2=0x55b6985e3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:43.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.480+0000 7fb643fff700 1 --2- 172.21.15.7:0/306022598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b6985d7f20 0x55b6985e3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.480+0000 7fb643fff700 1 -- 172.21.15.7:0/306022598 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6984788d0 con 0x55b6985ddf60 2024-06-10T01:14:43.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.480+0000 7fb6437fe700 1 --2- 172.21.15.7:0/306022598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b6985d7f20 0x55b6985e3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:14:43.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.480+0000 7fb643fff700 1 --2- 172.21.15.7:0/306022598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6985ddf60 0x55b6985e15c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00bcd0 tx=0x7fb63c00bd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16ed3cf086c30830 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:43.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.480+0000 7fb622ffd700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:14:43.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.480+0000 7fb622ffd700 1 -- 172.21.15.7:0/306022598 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c003e10 con 0x55b6985ddf60 2024-06-10T01:14:43.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.480+0000 7fb654741b80 1 -- 172.21.15.7:0/306022598 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb63409fee0 con 0x55b6985ddf60 2024-06-10T01:14:43.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.480+0000 7fb622ffd700 1 -- 172.21.15.7:0/306022598 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb63c005070 con 0x55b6985ddf60 2024-06-10T01:14:43.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.480+0000 7fb622ffd700 1 -- 172.21.15.7:0/306022598 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb63c02f050 con 0x55b6985ddf60 2024-06-10T01:14:43.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.480+0000 7fb654741b80 10 client.27076 fetch_fsmap learned FSMap version 35 2024-06-10T01:14:43.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.480+0000 7fb654741b80 1 -- 172.21.15.7:0/306022598 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b6985ef3b0 con 0x55b6985ddf60 2024-06-10T01:14:43.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.481+0000 7fb622ffd700 1 -- 172.21.15.7:0/306022598 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb63409fee0 con 0x55b6985ddf60 2024-06-10T01:14:43.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.481+0000 7fb622ffd700 1 -- 172.21.15.7:0/306022598 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7fb63c024020 con 0x55b6985ddf60 2024-06-10T01:14:43.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.481+0000 7fb654741b80 10 client.27076 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:14:43.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.481+0000 7fb654741b80 10 client.27076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:14:43.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.481+0000 7fb654741b80 20 client.27076 populate_metadata read hostname 'smithi007' 2024-06-10T01:14:43.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.481+0000 7fb654741b80 1 --2- 172.21.15.7:0/306022598 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b6985f1990 0x55b6985f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:43.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.481+0000 7fb654741b80 4 client.27076 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:14:43.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.481+0000 7fb654741b80 1 -- 172.21.15.7:0/306022598 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b698536d10 con 0x55b6985f1990 2024-06-10T01:14:43.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.481+0000 7fb6437fe700 1 --2- 172.21.15.7:0/306022598 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b6985f1990 0x55b6985f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:43.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.482+0000 7fb622ffd700 10 client.27076 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:43.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.482+0000 7fb6437fe700 1 --2- 172.21.15.7:0/306022598 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b6985f1990 0x55b6985f3de0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48b710e2328fd883 server_cookie=fd718436d97c3301 in_seq=0 out_seq=0 2024-06-10T01:14:43.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.483+0000 7fb622ffd700 1 -- 172.21.15.7:0/306022598 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b698536d10 con 0x55b6985f1990 2024-06-10T01:14:43.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.483+0000 7fb622ffd700 10 client.27076 handle_command_reply: tid=0 2024-06-10T01:14:43.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.483+0000 7fb654741b80 1 client.27076 shutdown 2024-06-10T01:14:43.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.483+0000 7fb654741b80 1 -- 172.21.15.7:0/306022598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6985ddf60 msgr2=0x55b6985e15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:43.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.483+0000 7fb654741b80 1 --2- 172.21.15.7:0/306022598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6985ddf60 0x55b6985e15c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00bcd0 tx=0x7fb63c00bd00 comp rx=0 tx=0).stop 2024-06-10T01:14:43.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.483+0000 7fb654741b80 1 -- 172.21.15.7:0/306022598 shutdown_connections 2024-06-10T01:14:43.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.483+0000 7fb654741b80 1 --2- 172.21.15.7:0/306022598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6985ddf60 0x55b6985e15c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.483+0000 7fb654741b80 1 --2- 172.21.15.7:0/306022598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b6985d7f20 0x55b6985e3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.483+0000 7fb654741b80 1 --2- 172.21.15.7:0/306022598 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b6985f1990 0x55b6985f3de0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.483+0000 7fb622ffd700 0 client.27076 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:43.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.483+0000 7fb654741b80 1 --2- 172.21.15.7:0/306022598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6985dd640 0x55b6985e1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.483+0000 7fb654741b80 1 -- 172.21.15.7:0/306022598 >> 172.21.15.7:0/306022598 conn(0x55b698502cc0 msgr2=0x55b6985d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:43.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.484+0000 7fb654741b80 1 -- 172.21.15.7:0/306022598 shutdown_connections 2024-06-10T01:14:43.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.484+0000 7fb654741b80 1 -- 172.21.15.7:0/306022598 wait complete. 2024-06-10T01:14:43.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.484+0000 7fb654741b80 20 client.27076 trim_cache size 0 max 16384 2024-06-10T01:14:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.501+0000 7fb654741b80 1 Processor -- start 2024-06-10T01:14:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.501+0000 7fb654741b80 1 -- start start 2024-06-10T01:14:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.501+0000 7fb654741b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6985d7f20 0x55b6985edc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.501+0000 7fb654741b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6985dd640 0x55b6985d79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.501+0000 7fb654741b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b6985ddf60 0x55b6985db5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.501+0000 7fb6437fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6985d7f20 0x55b6985edc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.501+0000 7fb654741b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb6441b2cf0 con 0x55b6985ddf60 2024-06-10T01:14:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.501+0000 7fb6437fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6985d7f20 0x55b6985edc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52668/0 (socket says 172.21.15.7:52668) 2024-06-10T01:14:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.501+0000 7fb654741b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb6441b0520 con 0x55b6985dd640 2024-06-10T01:14:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.501+0000 7fb654741b80 1 -- 172.21.15.7:0/2538919710 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb6441ae370 con 0x55b6985d7f20 2024-06-10T01:14:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.501+0000 7fb6437fe700 1 -- 172.21.15.7:0/2538919710 learned_addr learned my addr 172.21.15.7:0/2538919710 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:43.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.501+0000 7fb642ffd700 1 --2- 172.21.15.7:0/2538919710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6985dd640 0x55b6985d79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:43.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.502+0000 7fb643fff700 1 --2- 172.21.15.7:0/2538919710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b6985ddf60 0x55b6985db5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:43.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.502+0000 7fb6437fe700 1 -- 172.21.15.7:0/2538919710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6985dd640 msgr2=0x55b6985d79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:43.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.502+0000 7fb6437fe700 1 --2- 172.21.15.7:0/2538919710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6985dd640 0x55b6985d79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.502+0000 7fb6437fe700 1 -- 172.21.15.7:0/2538919710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b6985ddf60 msgr2=0x55b6985db5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:43.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.502+0000 7fb6437fe700 1 --2- 172.21.15.7:0/2538919710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b6985ddf60 0x55b6985db5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.502+0000 7fb6437fe700 1 -- 172.21.15.7:0/2538919710 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6985dbb00 con 0x55b6985d7f20 2024-06-10T01:14:43.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.502+0000 7fb6437fe700 1 --2- 172.21.15.7:0/2538919710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6985d7f20 0x55b6985edc70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb634098500 tx=0x7fb63409ba30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a8be2bf81794fcd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:43.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.502+0000 7fb6297fa700 1 -- 172.21.15.7:0/2538919710 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63409ce60 con 0x55b6985d7f20 2024-06-10T01:14:43.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.502+0000 7fb6297fa700 1 -- 172.21.15.7:0/2538919710 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb634064070 con 0x55b6985d7f20 2024-06-10T01:14:43.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.502+0000 7fb6297fa700 1 -- 172.21.15.7:0/2538919710 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb634016330 con 0x55b6985d7f20 2024-06-10T01:14:43.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.503+0000 7fb654741b80 1 -- 172.21.15.7:0/2538919710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6985d7f20 msgr2=0x55b6985edc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:43.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.503+0000 7fb654741b80 1 --2- 172.21.15.7:0/2538919710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6985d7f20 0x55b6985edc70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb634098500 tx=0x7fb63409ba30 comp rx=0 tx=0).stop 2024-06-10T01:14:43.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.503+0000 7fb654741b80 1 -- 172.21.15.7:0/2538919710 shutdown_connections 2024-06-10T01:14:43.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.503+0000 7fb654741b80 1 --2- 172.21.15.7:0/2538919710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b6985ddf60 0x55b6985db5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.503+0000 7fb654741b80 1 --2- 172.21.15.7:0/2538919710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6985d7f20 0x55b6985edc70 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.503+0000 7fb654741b80 1 --2- 172.21.15.7:0/2538919710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6985dd640 0x55b6985d79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.503+0000 7fb654741b80 1 -- 172.21.15.7:0/2538919710 >> 172.21.15.7:0/2538919710 conn(0x55b698502cc0 msgr2=0x55b6985daeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:43.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.503+0000 7fb654741b80 1 -- 172.21.15.7:0/2538919710 shutdown_connections 2024-06-10T01:14:43.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.503+0000 7fb654741b80 1 -- 172.21.15.7:0/2538919710 wait complete. 2024-06-10T01:14:43.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.504+0000 7fb654741b80 1 -- start start 2024-06-10T01:14:43.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.504+0000 7fb654741b80 1 Processor -- start 2024-06-10T01:14:43.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.504+0000 7fb654741b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6985d7f20 0x55b6985ea230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:43.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.504+0000 7fb654741b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6985dd640 0x55b6985f8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:43.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.504+0000 7fb654741b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b6985ddf60 0x55b6985f91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:43.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.504+0000 7fb654741b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb6441ae9e0 con 0x55b6985ddf60 2024-06-10T01:14:43.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.504+0000 7fb654741b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb6441afe50 con 0x55b6985d7f20 2024-06-10T01:14:43.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.504+0000 7fb654741b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb6441af0b0 con 0x55b6985dd640 2024-06-10T01:14:43.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.504+0000 7fb654741b80 10 client.0 _ll_register_callbacks cb 0x55b6984c3920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:14:43.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.505+0000 7fb6437fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6985d7f20 0x55b6985ea230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:43.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.505+0000 7fb643fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b6985ddf60 0x55b6985f91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:43.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.505+0000 7fb6437fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6985d7f20 0x55b6985ea230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:44310/0 (socket says 172.21.15.7:44310) 2024-06-10T01:14:43.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.505+0000 7fb6437fe700 1 -- 172.21.15.7:0/4274909114 learned_addr learned my addr 172.21.15.7:0/4274909114 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:43.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.505+0000 7fb642ffd700 1 --2- 172.21.15.7:0/4274909114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6985dd640 0x55b6985f8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:43.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.505+0000 7fb6437fe700 1 -- 172.21.15.7:0/4274909114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6985dd640 msgr2=0x55b6985f8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:43.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.505+0000 7fb6437fe700 1 --2- 172.21.15.7:0/4274909114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6985dd640 0x55b6985f8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.505+0000 7fb6437fe700 1 -- 172.21.15.7:0/4274909114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b6985ddf60 msgr2=0x55b6985f91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:43.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.505+0000 7fb6437fe700 1 --2- 172.21.15.7:0/4274909114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b6985ddf60 0x55b6985f91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.505+0000 7fb6437fe700 1 -- 172.21.15.7:0/4274909114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6985f96e0 con 0x55b6985d7f20 2024-06-10T01:14:43.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.505+0000 7fb6437fe700 1 --2- 172.21.15.7:0/4274909114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6985d7f20 0x55b6985ea230 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb63400f010 tx=0x7fb634099ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffc385277bab8c67 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:43.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.505+0000 7fb622ffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:14:43.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.505+0000 7fb622ffd700 1 -- 172.21.15.7:0/4274909114 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63409ce60 con 0x55b6985d7f20 2024-06-10T01:14:43.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.505+0000 7fb622ffd700 1 -- 172.21.15.7:0/4274909114 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb63400cd70 con 0x55b6985d7f20 2024-06-10T01:14:43.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.505+0000 7fb654741b80 1 -- 172.21.15.7:0/4274909114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb634064070 con 0x55b6985d7f20 2024-06-10T01:14:43.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.506+0000 7fb622ffd700 1 -- 172.21.15.7:0/4274909114 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb6340cf3a0 con 0x55b6985d7f20 2024-06-10T01:14:43.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.506+0000 7fb654741b80 10 client.27053 fetch_fsmap learned FSMap version 35 2024-06-10T01:14:43.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.506+0000 7fb654741b80 1 -- 172.21.15.7:0/4274909114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b698664ba0 con 0x55b6985d7f20 2024-06-10T01:14:43.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.506+0000 7fb622ffd700 1 -- 172.21.15.7:0/4274909114 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb634064070 con 0x55b6985d7f20 2024-06-10T01:14:43.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.506+0000 7fb622ffd700 1 -- 172.21.15.7:0/4274909114 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7fb63409bd90 con 0x55b6985d7f20 2024-06-10T01:14:43.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.506+0000 7fb654741b80 10 client.27053 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:14:43.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.506+0000 7fb654741b80 10 client.27053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:14:43.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.506+0000 7fb654741b80 20 client.27053 populate_metadata read hostname 'smithi007' 2024-06-10T01:14:43.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.506+0000 7fb654741b80 1 --2- 172.21.15.7:0/4274909114 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b6985f1990 0x55b698666e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:43.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.506+0000 7fb654741b80 4 client.27053 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:14:43.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.506+0000 7fb654741b80 1 -- 172.21.15.7:0/4274909114 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b698531290 con 0x55b6985f1990 2024-06-10T01:14:43.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.506+0000 7fb642ffd700 1 --2- 172.21.15.7:0/4274909114 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b6985f1990 0x55b698666e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:43.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.507+0000 7fb622ffd700 10 client.27053 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:43.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.507+0000 7fb642ffd700 1 --2- 172.21.15.7:0/4274909114 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b6985f1990 0x55b698666e30 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb9f9c3169134874 server_cookie=10d20effda65a8e8 in_seq=0 out_seq=0 2024-06-10T01:14:43.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.507+0000 7fb622ffd700 1 -- 172.21.15.7:0/4274909114 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b698531290 con 0x55b6985f1990 2024-06-10T01:14:43.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.507+0000 7fb622ffd700 10 client.27053 handle_command_reply: tid=0 2024-06-10T01:14:43.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.507+0000 7fb654741b80 1 client.27053 shutdown 2024-06-10T01:14:43.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.508+0000 7fb654741b80 1 -- 172.21.15.7:0/4274909114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6985d7f20 msgr2=0x55b6985ea230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:43.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.508+0000 7fb654741b80 1 --2- 172.21.15.7:0/4274909114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6985d7f20 0x55b6985ea230 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb63400f010 tx=0x7fb634099ed0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.508+0000 7fb654741b80 1 -- 172.21.15.7:0/4274909114 shutdown_connections 2024-06-10T01:14:43.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.508+0000 7fb654741b80 1 --2- 172.21.15.7:0/4274909114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b6985ddf60 0x55b6985f91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.508+0000 7fb654741b80 1 --2- 172.21.15.7:0/4274909114 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b6985f1990 0x55b698666e30 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.508+0000 7fb654741b80 1 --2- 172.21.15.7:0/4274909114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6985d7f20 0x55b6985ea230 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.508+0000 7fb654741b80 1 --2- 172.21.15.7:0/4274909114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6985dd640 0x55b6985f8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.508+0000 7fb654741b80 1 -- 172.21.15.7:0/4274909114 >> 172.21.15.7:0/4274909114 conn(0x55b698502cc0 msgr2=0x55b6985cbe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:43.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.508+0000 7fb622ffd700 0 client.27053 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:43.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.508+0000 7fb654741b80 1 -- 172.21.15.7:0/4274909114 shutdown_connections 2024-06-10T01:14:43.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.508+0000 7fb654741b80 1 -- 172.21.15.7:0/4274909114 wait complete. 2024-06-10T01:14:43.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.508+0000 7fb654741b80 20 client.27053 trim_cache size 0 max 16384 2024-06-10T01:14:43.507 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:14:43.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.509+0000 7fb649ebe700 1 -- 172.21.15.7:0/701609958 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb62c0a2c20 msgr2=0x7fb62c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:43.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.509+0000 7fb649ebe700 1 --2- 172.21.15.7:0/701609958 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb62c0a2c20 0x7fb62c0a50e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb644107c20 tx=0x7fb638009260 comp rx=0 tx=0).stop 2024-06-10T01:14:43.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.509+0000 7fb649ebe700 1 -- 172.21.15.7:0/701609958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6441019d0 msgr2=0x7fb64419e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:43.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.509+0000 7fb649ebe700 1 --2- 172.21.15.7:0/701609958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6441019d0 0x7fb64419e850 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fb63400e9c0 tx=0x7fb63400ecd0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.509+0000 7fb649ebe700 1 -- 172.21.15.7:0/701609958 shutdown_connections 2024-06-10T01:14:43.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.509+0000 7fb649ebe700 1 --2- 172.21.15.7:0/701609958 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb62c0a2c20 0x7fb62c0a50e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.509+0000 7fb649ebe700 1 --2- 172.21.15.7:0/701609958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb644106720 0x7fb644198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.509+0000 7fb649ebe700 1 --2- 172.21.15.7:0/701609958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6441019d0 0x7fb64419e850 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.509+0000 7fb649ebe700 1 --2- 172.21.15.7:0/701609958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb644102380 0x7fb64419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:43.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.509+0000 7fb649ebe700 1 -- 172.21.15.7:0/701609958 >> 172.21.15.7:0/701609958 conn(0x7fb644078340 msgr2=0x7fb64406ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:43.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.509+0000 7fb649ebe700 1 -- 172.21.15.7:0/701609958 shutdown_connections 2024-06-10T01:14:43.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:43.509+0000 7fb649ebe700 1 -- 172.21.15.7:0/701609958 wait complete. 2024-06-10T01:14:43.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:43 smithi175 ceph-mon[36636]: pgmap v693: 129 pgs: 129 active+clean; 106 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 2.2 MiB/s wr, 277 op/s 2024-06-10T01:14:43.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:43 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:14:43.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:43 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:14:43.741 INFO:tasks.workunit.client.0.smithi007.stdout: 1 642595 7.84 MB/sec execute 324 sec latency 16.347 ms 2024-06-10T01:14:43.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:43 smithi078 ceph-mon[36683]: pgmap v693: 129 pgs: 129 active+clean; 106 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 2.2 MiB/s wr, 277 op/s 2024-06-10T01:14:43.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:43 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:14:43.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:43 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:14:44.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:44 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:14:44.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:44 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:14:44.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:44 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:14:44.742 INFO:tasks.workunit.client.0.smithi007.stdout: 1 643955 7.84 MB/sec execute 325 sec latency 17.054 ms 2024-06-10T01:14:44.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:44 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:14:44.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:44 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:14:44.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:44 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:14:44.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:44 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:14:44.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:44 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:14:44.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:44 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:14:45.172 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:14:45.172 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:14:45.436 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:14:45.512 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:45 smithi007 ceph-mon[32398]: pgmap v694: 129 pgs: 129 active+clean; 108 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 2.2 MiB/s wr, 294 op/s 2024-06-10T01:14:45.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:45 smithi175 ceph-mon[36636]: pgmap v694: 129 pgs: 129 active+clean; 108 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 2.2 MiB/s wr, 294 op/s 2024-06-10T01:14:45.742 INFO:tasks.workunit.client.0.smithi007.stdout: 1 646557 7.87 MB/sec execute 326 sec latency 33.447 ms 2024-06-10T01:14:45.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:45 smithi078 ceph-mon[36683]: pgmap v694: 129 pgs: 129 active+clean; 108 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 2.2 MiB/s wr, 294 op/s 2024-06-10T01:14:46.742 INFO:tasks.workunit.client.0.smithi007.stdout: 1 648240 7.86 MB/sec execute 327 sec latency 19.232 ms 2024-06-10T01:14:47.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.082+0000 7f4b8a0e7700 1 -- 172.21.15.7:0/19070161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b84102510 msgr2=0x7f4b84102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:47.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.082+0000 7f4b8a0e7700 1 --2- 172.21.15.7:0/19070161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b84102510 0x7f4b84102990 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c009990 tx=0x7f4b6c009ca0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.082+0000 7f4b8a0e7700 1 -- 172.21.15.7:0/19070161 shutdown_connections 2024-06-10T01:14:47.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.082+0000 7f4b8a0e7700 1 --2- 172.21.15.7:0/19070161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b84102ed0 0x7f4b8410f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.082+0000 7f4b8a0e7700 1 --2- 172.21.15.7:0/19070161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b84108520 0x7f4b84108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.082+0000 7f4b8a0e7700 1 --2- 172.21.15.7:0/19070161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b84102510 0x7f4b84102990 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.082+0000 7f4b8a0e7700 1 -- 172.21.15.7:0/19070161 >> 172.21.15.7:0/19070161 conn(0x7f4b84079560 msgr2=0x7f4b84079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:47.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.082+0000 7f4b8a0e7700 1 -- 172.21.15.7:0/19070161 shutdown_connections 2024-06-10T01:14:47.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.082+0000 7f4b8a0e7700 1 -- 172.21.15.7:0/19070161 wait complete. 2024-06-10T01:14:47.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.083+0000 7f4b8a0e7700 1 Processor -- start 2024-06-10T01:14:47.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.083+0000 7f4b8a0e7700 1 -- start start 2024-06-10T01:14:47.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.084+0000 7f4b8a0e7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b84102510 0x7f4b8419e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:47.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.084+0000 7f4b8a0e7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b84102ed0 0x7f4b8419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:47.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.084+0000 7f4b8a0e7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b84108520 0x7f4b841989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:47.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.084+0000 7f4b8a0e7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4b841127a0 con 0x7f4b84108520 2024-06-10T01:14:47.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.084+0000 7f4b8a0e7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4b84112620 con 0x7f4b84102ed0 2024-06-10T01:14:47.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.084+0000 7f4b8a0e7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4b84112920 con 0x7f4b84102510 2024-06-10T01:14:47.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.084+0000 7f4b837fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b84102510 0x7f4b8419e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:47.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.084+0000 7f4b837fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b84102510 0x7f4b8419e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52708/0 (socket says 172.21.15.7:52708) 2024-06-10T01:14:47.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.084+0000 7f4b83fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b84108520 0x7f4b841989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:47.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.084+0000 7f4b837fe700 1 -- 172.21.15.7:0/2987387773 learned_addr learned my addr 172.21.15.7:0/2987387773 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:47.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.084+0000 7f4b7affd700 1 --2- 172.21.15.7:0/2987387773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b84102ed0 0x7f4b8419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:47.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.084+0000 7f4b837fe700 1 -- 172.21.15.7:0/2987387773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b84102ed0 msgr2=0x7f4b8419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:47.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.084+0000 7f4b837fe700 1 --2- 172.21.15.7:0/2987387773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b84102ed0 0x7f4b8419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.084+0000 7f4b837fe700 1 -- 172.21.15.7:0/2987387773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b84108520 msgr2=0x7f4b841989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:47.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.084+0000 7f4b837fe700 1 --2- 172.21.15.7:0/2987387773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b84108520 0x7f4b841989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.084+0000 7f4b837fe700 1 -- 172.21.15.7:0/2987387773 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b84199100 con 0x7f4b84102510 2024-06-10T01:14:47.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.084+0000 7f4b83fff700 1 --2- 172.21.15.7:0/2987387773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b84108520 0x7f4b841989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:47.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.084+0000 7f4b837fe700 1 --2- 172.21.15.7:0/2987387773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b84102510 0x7f4b8419e7e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4b7400eb30 tx=0x7f4b7400ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:47.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.085+0000 7f4b817fa700 1 -- 172.21.15.7:0/2987387773 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7400cbd0 con 0x7f4b84102510 2024-06-10T01:14:47.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.085+0000 7f4b8a0e7700 1 -- 172.21.15.7:0/2987387773 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b841993f0 con 0x7f4b84102510 2024-06-10T01:14:47.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.085+0000 7f4b8a0e7700 1 -- 172.21.15.7:0/2987387773 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b840746c0 con 0x7f4b84102510 2024-06-10T01:14:47.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.085+0000 7f4b817fa700 1 -- 172.21.15.7:0/2987387773 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b74004500 con 0x7f4b84102510 2024-06-10T01:14:47.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.085+0000 7f4b817fa700 1 -- 172.21.15.7:0/2987387773 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b74010430 con 0x7f4b84102510 2024-06-10T01:14:47.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.087+0000 7f4b817fa700 1 -- 172.21.15.7:0/2987387773 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f4b74003680 con 0x7f4b84102510 2024-06-10T01:14:47.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.088+0000 7f4b817fa700 1 --2- 172.21.15.7:0/2987387773 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4b700a2ca0 0x7f4b700a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:47.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.088+0000 7f4b817fa700 1 -- 172.21.15.7:0/2987387773 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f4b740cf5e0 con 0x7f4b84102510 2024-06-10T01:14:47.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.088+0000 7f4b7affd700 1 --2- 172.21.15.7:0/2987387773 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4b700a2ca0 0x7f4b700a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:47.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.088+0000 7f4b7affd700 1 --2- 172.21.15.7:0/2987387773 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4b700a2ca0 0x7f4b700a5160 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00b2e0 tx=0x7f4b6c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:47.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.089+0000 7f4b9496ab80 1 Processor -- start 2024-06-10T01:14:47.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.089+0000 7f4b9496ab80 1 -- start start 2024-06-10T01:14:47.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.089+0000 7f4b9496ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d678e306c0 0x55d678e30aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:47.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.089+0000 7f4b9496ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d678e30fe0 0x55d678e2aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:47.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.089+0000 7f4b9496ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d678e2afa0 0x55d678e2b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:47.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.089+0000 7f4b9496ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d678d7c890 con 0x55d678e306c0 2024-06-10T01:14:47.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.089+0000 7f4b9496ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d678d247a0 con 0x55d678e30fe0 2024-06-10T01:14:47.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.089+0000 7f4b9496ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d678d78850 con 0x55d678e2afa0 2024-06-10T01:14:47.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.089+0000 7f4b837fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d678e306c0 0x55d678e30aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:47.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.089+0000 7f4b7affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d678e30fe0 0x55d678e2aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:47.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.089+0000 7f4b7affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d678e30fe0 0x55d678e2aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:44326/0 (socket says 172.21.15.7:44326) 2024-06-10T01:14:47.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.089+0000 7f4b7affd700 1 -- 172.21.15.7:0/38990937 learned_addr learned my addr 172.21.15.7:0/38990937 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:47.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.089+0000 7f4b837fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d678e306c0 0x55d678e30aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45466/0 (socket says 172.21.15.7:45466) 2024-06-10T01:14:47.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.089+0000 7f4b83fff700 1 --2- 172.21.15.7:0/38990937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d678e2afa0 0x55d678e2b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:47.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.090+0000 7f4b837fe700 1 -- 172.21.15.7:0/38990937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d678e2afa0 msgr2=0x55d678e2b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:47.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.090+0000 7f4b837fe700 1 --2- 172.21.15.7:0/38990937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d678e2afa0 0x55d678e2b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.090+0000 7f4b837fe700 1 -- 172.21.15.7:0/38990937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d678e30fe0 msgr2=0x55d678e2aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:47.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.090+0000 7f4b837fe700 1 --2- 172.21.15.7:0/38990937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d678e30fe0 0x55d678e2aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.090+0000 7f4b837fe700 1 -- 172.21.15.7:0/38990937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d678d1afd0 con 0x55d678e306c0 2024-06-10T01:14:47.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.090+0000 7f4b837fe700 1 --2- 172.21.15.7:0/38990937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d678e306c0 0x55d678e30aa0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f4b74000c00 tx=0x7f4b74097e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f72b12aa7d9410 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:47.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.090+0000 7f4b79ffb700 1 -- 172.21.15.7:0/38990937 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7409b420 con 0x55d678e306c0 2024-06-10T01:14:47.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.090+0000 7f4b79ffb700 1 -- 172.21.15.7:0/38990937 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b740b4070 con 0x55d678e306c0 2024-06-10T01:14:47.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.090+0000 7f4b79ffb700 1 -- 172.21.15.7:0/38990937 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b74099dc0 con 0x55d678e306c0 2024-06-10T01:14:47.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.091+0000 7f4b9496ab80 1 -- 172.21.15.7:0/38990937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d678e306c0 msgr2=0x55d678e30aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:47.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.091+0000 7f4b9496ab80 1 --2- 172.21.15.7:0/38990937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d678e306c0 0x55d678e30aa0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f4b74000c00 tx=0x7f4b74097e80 comp rx=0 tx=0).stop 2024-06-10T01:14:47.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.091+0000 7f4b9496ab80 1 -- 172.21.15.7:0/38990937 shutdown_connections 2024-06-10T01:14:47.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.091+0000 7f4b9496ab80 1 --2- 172.21.15.7:0/38990937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d678e2afa0 0x55d678e2b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.091+0000 7f4b9496ab80 1 --2- 172.21.15.7:0/38990937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d678e306c0 0x55d678e30aa0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.091+0000 7f4b9496ab80 1 --2- 172.21.15.7:0/38990937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d678e30fe0 0x55d678e2aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.091+0000 7f4b9496ab80 1 -- 172.21.15.7:0/38990937 >> 172.21.15.7:0/38990937 conn(0x55d678d55d40 msgr2=0x55d678e258d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:47.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.091+0000 7f4b9496ab80 1 -- 172.21.15.7:0/38990937 shutdown_connections 2024-06-10T01:14:47.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.091+0000 7f4b9496ab80 1 -- 172.21.15.7:0/38990937 wait complete. 2024-06-10T01:14:47.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.092+0000 7f4b9496ab80 1 -- start start 2024-06-10T01:14:47.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.092+0000 7f4b9496ab80 1 Processor -- start 2024-06-10T01:14:47.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.092+0000 7f4b9496ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d678e2afa0 0x55d678e36d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:47.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.092+0000 7f4b9496ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d678e306c0 0x55d678e34050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:47.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.092+0000 7f4b9496ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d678e30fe0 0x55d678e34590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:47.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.092+0000 7f4b9496ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d678e379f0 con 0x55d678e2afa0 2024-06-10T01:14:47.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.092+0000 7f4b9496ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d678d1cc70 con 0x55d678e306c0 2024-06-10T01:14:47.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.092+0000 7f4b9496ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d678e37b70 con 0x55d678e30fe0 2024-06-10T01:14:47.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.092+0000 7f4b9496ab80 10 client.0 _ll_register_callbacks cb 0x55d678d169a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:14:47.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.093+0000 7f4b83fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d678e30fe0 0x55d678e34590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:47.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.093+0000 7f4b83fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d678e30fe0 0x55d678e34590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52726/0 (socket says 172.21.15.7:52726) 2024-06-10T01:14:47.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.093+0000 7f4b83fff700 1 -- 172.21.15.7:0/4254034784 learned_addr learned my addr 172.21.15.7:0/4254034784 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:47.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.093+0000 7f4b837fe700 1 --2- 172.21.15.7:0/4254034784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d678e2afa0 0x55d678e36d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:47.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.093+0000 7f4b7affd700 1 --2- 172.21.15.7:0/4254034784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d678e306c0 0x55d678e34050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:47.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.093+0000 7f4b83fff700 1 -- 172.21.15.7:0/4254034784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d678e306c0 msgr2=0x55d678e34050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:47.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.093+0000 7f4b83fff700 1 --2- 172.21.15.7:0/4254034784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d678e306c0 0x55d678e34050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.093+0000 7f4b83fff700 1 -- 172.21.15.7:0/4254034784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d678e2afa0 msgr2=0x55d678e36d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:47.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.093+0000 7f4b83fff700 1 --2- 172.21.15.7:0/4254034784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d678e2afa0 0x55d678e36d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.093+0000 7f4b83fff700 1 -- 172.21.15.7:0/4254034784 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d678ccb950 con 0x55d678e30fe0 2024-06-10T01:14:47.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.093+0000 7f4b837fe700 1 --2- 172.21.15.7:0/4254034784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d678e2afa0 0x55d678e36d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:47.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.093+0000 7f4b83fff700 1 --2- 172.21.15.7:0/4254034784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d678e30fe0 0x55d678e34590 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c0046e0 tx=0x7f4b7c0049f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a66bba8b3ba1696 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:47.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.093+0000 7f4b62ffd700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:14:47.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.093+0000 7f4b62ffd700 1 -- 172.21.15.7:0/4254034784 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c00ea40 con 0x55d678e30fe0 2024-06-10T01:14:47.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.093+0000 7f4b62ffd700 1 -- 172.21.15.7:0/4254034784 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f4b7c00ebe0 con 0x55d678e30fe0 2024-06-10T01:14:47.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.093+0000 7f4b9496ab80 1 -- 172.21.15.7:0/4254034784 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b740b4070 con 0x55d678e30fe0 2024-06-10T01:14:47.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.094+0000 7f4b62ffd700 1 -- 172.21.15.7:0/4254034784 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4b7c027050 con 0x55d678e30fe0 2024-06-10T01:14:47.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.094+0000 7f4b9496ab80 10 client.27094 fetch_fsmap learned FSMap version 35 2024-06-10T01:14:47.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.094+0000 7f4b9496ab80 1 -- 172.21.15.7:0/4254034784 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d678e42350 con 0x55d678e30fe0 2024-06-10T01:14:47.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.094+0000 7f4b62ffd700 1 -- 172.21.15.7:0/4254034784 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b740b4070 con 0x55d678e30fe0 2024-06-10T01:14:47.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.094+0000 7f4b62ffd700 1 -- 172.21.15.7:0/4254034784 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f4b7c01e520 con 0x55d678e30fe0 2024-06-10T01:14:47.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.094+0000 7f4b9496ab80 10 client.27094 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:14:47.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.094+0000 7f4b9496ab80 10 client.27094 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:14:47.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.094+0000 7f4b9496ab80 20 client.27094 populate_metadata read hostname 'smithi007' 2024-06-10T01:14:47.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.094+0000 7f4b9496ab80 1 --2- 172.21.15.7:0/4254034784 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d678e44d20 0x55d678e47110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:47.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.094+0000 7f4b9496ab80 4 client.27094 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:14:47.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.094+0000 7f4b9496ab80 1 -- 172.21.15.7:0/4254034784 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d678d89d90 con 0x55d678e44d20 2024-06-10T01:14:47.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.094+0000 7f4b837fe700 1 --2- 172.21.15.7:0/4254034784 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d678e44d20 0x55d678e47110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:47.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.095+0000 7f4b837fe700 1 --2- 172.21.15.7:0/4254034784 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d678e44d20 0x55d678e47110 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c4a22dd7d46c161 server_cookie=1a343bb44da6fc35 in_seq=0 out_seq=0 2024-06-10T01:14:47.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.095+0000 7f4b62ffd700 10 client.27094 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:47.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.095+0000 7f4b62ffd700 1 -- 172.21.15.7:0/4254034784 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d678d89d90 con 0x55d678e44d20 2024-06-10T01:14:47.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.095+0000 7f4b62ffd700 10 client.27094 handle_command_reply: tid=0 2024-06-10T01:14:47.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.096+0000 7f4b9496ab80 1 client.27094 shutdown 2024-06-10T01:14:47.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.096+0000 7f4b9496ab80 1 -- 172.21.15.7:0/4254034784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d678e30fe0 msgr2=0x55d678e34590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:47.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.096+0000 7f4b9496ab80 1 --2- 172.21.15.7:0/4254034784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d678e30fe0 0x55d678e34590 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c0046e0 tx=0x7f4b7c0049f0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.096+0000 7f4b9496ab80 1 -- 172.21.15.7:0/4254034784 shutdown_connections 2024-06-10T01:14:47.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.096+0000 7f4b9496ab80 1 --2- 172.21.15.7:0/4254034784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d678e30fe0 0x55d678e34590 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.096+0000 7f4b9496ab80 1 --2- 172.21.15.7:0/4254034784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d678e2afa0 0x55d678e36d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.096+0000 7f4b9496ab80 1 --2- 172.21.15.7:0/4254034784 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d678e44d20 0x55d678e47110 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.096+0000 7f4b9496ab80 1 --2- 172.21.15.7:0/4254034784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d678e306c0 0x55d678e34050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.096+0000 7f4b9496ab80 1 -- 172.21.15.7:0/4254034784 >> 172.21.15.7:0/4254034784 conn(0x55d678d55d40 msgr2=0x55d678e25670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:47.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.096+0000 7f4b62ffd700 0 client.27094 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:47.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.096+0000 7f4b9496ab80 1 -- 172.21.15.7:0/4254034784 shutdown_connections 2024-06-10T01:14:47.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.096+0000 7f4b9496ab80 1 -- 172.21.15.7:0/4254034784 wait complete. 2024-06-10T01:14:47.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.096+0000 7f4b9496ab80 20 client.27094 trim_cache size 0 max 16384 2024-06-10T01:14:47.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.119+0000 7f4b9496ab80 1 Processor -- start 2024-06-10T01:14:47.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.120+0000 7f4b9496ab80 1 -- start start 2024-06-10T01:14:47.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.120+0000 7f4b9496ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d678e2afa0 0x55d678e2aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:47.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.120+0000 7f4b9496ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d678e306c0 0x55d678e34680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:47.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.120+0000 7f4b9496ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d678e30fe0 0x55d678e313c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:47.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.120+0000 7f4b9496ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4b64011360 con 0x55d678e30fe0 2024-06-10T01:14:47.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.120+0000 7f4b9496ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6400eb90 con 0x55d678e2afa0 2024-06-10T01:14:47.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.120+0000 7f4b9496ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6400c9e0 con 0x55d678e306c0 2024-06-10T01:14:47.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.120+0000 7f4b837fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d678e2afa0 0x55d678e2aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:47.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.120+0000 7f4b7affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d678e306c0 0x55d678e34680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:47.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.120+0000 7f4b83fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d678e30fe0 0x55d678e313c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:47.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.120+0000 7f4b7affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d678e306c0 0x55d678e34680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52728/0 (socket says 172.21.15.7:52728) 2024-06-10T01:14:47.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.120+0000 7f4b837fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d678e2afa0 0x55d678e2aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:44346/0 (socket says 172.21.15.7:44346) 2024-06-10T01:14:47.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.120+0000 7f4b7affd700 1 -- 172.21.15.7:0/3276919859 learned_addr learned my addr 172.21.15.7:0/3276919859 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:47.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.120+0000 7f4b7affd700 1 -- 172.21.15.7:0/3276919859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d678e2afa0 msgr2=0x55d678e2aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:47.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.120+0000 7f4b7affd700 1 --2- 172.21.15.7:0/3276919859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d678e2afa0 0x55d678e2aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.120+0000 7f4b7affd700 1 -- 172.21.15.7:0/3276919859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d678e30fe0 msgr2=0x55d678e313c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:47.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.120+0000 7f4b7affd700 1 --2- 172.21.15.7:0/3276919859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d678e30fe0 0x55d678e313c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.120+0000 7f4b7affd700 1 -- 172.21.15.7:0/3276919859 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d678e325c0 con 0x55d678e306c0 2024-06-10T01:14:47.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.120+0000 7f4b837fe700 1 --2- 172.21.15.7:0/3276919859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d678e2afa0 0x55d678e2aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:47.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.120+0000 7f4b83fff700 1 --2- 172.21.15.7:0/3276919859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d678e30fe0 0x55d678e313c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:47.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.120+0000 7f4b7affd700 1 --2- 172.21.15.7:0/3276919859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d678e306c0 0x55d678e34680 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c009990 tx=0x7f4b6c025000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d299710465336150 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:47.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.121+0000 7f4b797fa700 1 -- 172.21.15.7:0/3276919859 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c005270 con 0x55d678e306c0 2024-06-10T01:14:47.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.121+0000 7f4b797fa700 1 -- 172.21.15.7:0/3276919859 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b6c005410 con 0x55d678e306c0 2024-06-10T01:14:47.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.121+0000 7f4b797fa700 1 -- 172.21.15.7:0/3276919859 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c00f6c0 con 0x55d678e306c0 2024-06-10T01:14:47.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.121+0000 7f4b9496ab80 1 -- 172.21.15.7:0/3276919859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d678e306c0 msgr2=0x55d678e34680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:47.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.121+0000 7f4b9496ab80 1 --2- 172.21.15.7:0/3276919859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d678e306c0 0x55d678e34680 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c009990 tx=0x7f4b6c025000 comp rx=0 tx=0).stop 2024-06-10T01:14:47.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.121+0000 7f4b9496ab80 1 -- 172.21.15.7:0/3276919859 shutdown_connections 2024-06-10T01:14:47.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.121+0000 7f4b9496ab80 1 --2- 172.21.15.7:0/3276919859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d678e30fe0 0x55d678e313c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.121+0000 7f4b9496ab80 1 --2- 172.21.15.7:0/3276919859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d678e2afa0 0x55d678e2aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.121+0000 7f4b9496ab80 1 --2- 172.21.15.7:0/3276919859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d678e306c0 0x55d678e34680 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.121+0000 7f4b9496ab80 1 -- 172.21.15.7:0/3276919859 >> 172.21.15.7:0/3276919859 conn(0x55d678d55d40 msgr2=0x55d678e32940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:47.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.122+0000 7f4b9496ab80 1 -- 172.21.15.7:0/3276919859 shutdown_connections 2024-06-10T01:14:47.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.122+0000 7f4b9496ab80 1 -- 172.21.15.7:0/3276919859 wait complete. 2024-06-10T01:14:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.122+0000 7f4b9496ab80 1 -- start start 2024-06-10T01:14:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.122+0000 7f4b9496ab80 1 Processor -- start 2024-06-10T01:14:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.122+0000 7f4b9496ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d678e2afa0 0x55d678e3d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.122+0000 7f4b9496ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d678e306c0 0x55d678e4c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.122+0000 7f4b9496ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d678e30fe0 0x55d678e4c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.122+0000 7f4b9496ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6400d050 con 0x55d678e306c0 2024-06-10T01:14:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.122+0000 7f4b9496ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6400e4c0 con 0x55d678e2afa0 2024-06-10T01:14:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.122+0000 7f4b9496ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6400d720 con 0x55d678e30fe0 2024-06-10T01:14:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.122+0000 7f4b9496ab80 10 client.0 _ll_register_callbacks cb 0x55d678d169a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:14:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.123+0000 7f4b7affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d678e306c0 0x55d678e4c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.123+0000 7f4b837fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d678e2afa0 0x55d678e3d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.123+0000 7f4b7affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d678e306c0 0x55d678e4c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45494/0 (socket says 172.21.15.7:45494) 2024-06-10T01:14:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.123+0000 7f4b837fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d678e2afa0 0x55d678e3d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:44350/0 (socket says 172.21.15.7:44350) 2024-06-10T01:14:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.123+0000 7f4b7affd700 1 -- 172.21.15.7:0/619599170 learned_addr learned my addr 172.21.15.7:0/619599170 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.123+0000 7f4b83fff700 1 --2- 172.21.15.7:0/619599170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d678e30fe0 0x55d678e4c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.123+0000 7f4b7affd700 1 -- 172.21.15.7:0/619599170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d678e30fe0 msgr2=0x55d678e4c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.123+0000 7f4b7affd700 1 --2- 172.21.15.7:0/619599170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d678e30fe0 0x55d678e4c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.123+0000 7f4b7affd700 1 -- 172.21.15.7:0/619599170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d678e2afa0 msgr2=0x55d678e3d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.123+0000 7f4b7affd700 1 --2- 172.21.15.7:0/619599170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d678e2afa0 0x55d678e3d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.123+0000 7f4b7affd700 1 -- 172.21.15.7:0/619599170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d678e4d030 con 0x55d678e306c0 2024-06-10T01:14:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.123+0000 7f4b837fe700 1 --2- 172.21.15.7:0/619599170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d678e2afa0 0x55d678e3d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.123+0000 7f4b83fff700 1 --2- 172.21.15.7:0/619599170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d678e30fe0 0x55d678e4c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.123+0000 7f4b7affd700 1 --2- 172.21.15.7:0/619599170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d678e306c0 0x55d678e4c2c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c003ea0 tx=0x7f4b6c003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36aeebd3d13e62ac server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.123+0000 7f4b62ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:14:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.124+0000 7f4b62ffd700 1 -- 172.21.15.7:0/619599170 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c003b90 con 0x55d678e306c0 2024-06-10T01:14:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.124+0000 7f4b62ffd700 1 -- 172.21.15.7:0/619599170 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f4b6c01b050 con 0x55d678e306c0 2024-06-10T01:14:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.124+0000 7f4b9496ab80 1 -- 172.21.15.7:0/619599170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b6c005410 con 0x55d678e306c0 2024-06-10T01:14:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.124+0000 7f4b62ffd700 1 -- 172.21.15.7:0/619599170 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4b6c01b250 con 0x55d678e306c0 2024-06-10T01:14:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.124+0000 7f4b9496ab80 10 client.17676 fetch_fsmap learned FSMap version 35 2024-06-10T01:14:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.124+0000 7f4b9496ab80 1 -- 172.21.15.7:0/619599170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d678eb8180 con 0x55d678e306c0 2024-06-10T01:14:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.124+0000 7f4b62ffd700 1 -- 172.21.15.7:0/619599170 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b6c01b3f0 con 0x55d678e306c0 2024-06-10T01:14:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.124+0000 7f4b62ffd700 1 -- 172.21.15.7:0/619599170 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f4b6c026410 con 0x55d678e306c0 2024-06-10T01:14:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.124+0000 7f4b9496ab80 10 client.17676 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:14:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.124+0000 7f4b9496ab80 10 client.17676 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:14:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.124+0000 7f4b9496ab80 20 client.17676 populate_metadata read hostname 'smithi007' 2024-06-10T01:14:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.124+0000 7f4b9496ab80 1 --2- 172.21.15.7:0/619599170 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d678eb8770 0x55d678ebab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.125+0000 7f4b9496ab80 4 client.17676 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:14:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.125+0000 7f4b9496ab80 1 -- 172.21.15.7:0/619599170 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f4b6c005410 con 0x55d678eb8770 2024-06-10T01:14:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.125+0000 7f4b837fe700 1 --2- 172.21.15.7:0/619599170 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d678eb8770 0x55d678ebab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.125+0000 7f4b837fe700 1 --2- 172.21.15.7:0/619599170 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d678eb8770 0x55d678ebab90 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36c76954e08bad0f server_cookie=34cfa2310404aea2 in_seq=0 out_seq=0 2024-06-10T01:14:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.125+0000 7f4b62ffd700 10 client.17676 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.126+0000 7f4b62ffd700 1 -- 172.21.15.7:0/619599170 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f4b6c005410 con 0x55d678eb8770 2024-06-10T01:14:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.126+0000 7f4b62ffd700 10 client.17676 handle_command_reply: tid=0 2024-06-10T01:14:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.126+0000 7f4b9496ab80 1 client.17676 shutdown 2024-06-10T01:14:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.127+0000 7f4b9496ab80 1 -- 172.21.15.7:0/619599170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d678e306c0 msgr2=0x55d678e4c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.127+0000 7f4b9496ab80 1 --2- 172.21.15.7:0/619599170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d678e306c0 0x55d678e4c2c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c003ea0 tx=0x7f4b6c003f70 comp rx=0 tx=0).stop 2024-06-10T01:14:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.127+0000 7f4b9496ab80 1 -- 172.21.15.7:0/619599170 shutdown_connections 2024-06-10T01:14:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.127+0000 7f4b9496ab80 1 --2- 172.21.15.7:0/619599170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d678e30fe0 0x55d678e4c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.127+0000 7f4b9496ab80 1 --2- 172.21.15.7:0/619599170 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d678eb8770 0x55d678ebab90 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.127+0000 7f4b9496ab80 1 --2- 172.21.15.7:0/619599170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d678e2afa0 0x55d678e3d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.127+0000 7f4b9496ab80 1 --2- 172.21.15.7:0/619599170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d678e306c0 0x55d678e4c2c0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.127+0000 7f4b9496ab80 1 -- 172.21.15.7:0/619599170 >> 172.21.15.7:0/619599170 conn(0x55d678d55d40 msgr2=0x55d678e1ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.127+0000 7f4b62ffd700 0 client.17676 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.127+0000 7f4b9496ab80 1 -- 172.21.15.7:0/619599170 shutdown_connections 2024-06-10T01:14:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.127+0000 7f4b9496ab80 1 -- 172.21.15.7:0/619599170 wait complete. 2024-06-10T01:14:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.127+0000 7f4b9496ab80 20 client.17676 trim_cache size 0 max 16384 2024-06-10T01:14:47.126 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:14:47.126 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-10T01:14:47.126 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "5c292e44-8484-4839-a4b0-54e84fece3e0", 2024-06-10T01:14:47.127 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-10T01:14:47.127 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:14:47.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.128+0000 7f4b8a0e7700 1 -- 172.21.15.7:0/2987387773 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4b700a2ca0 msgr2=0x7f4b700a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:47.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.128+0000 7f4b8a0e7700 1 --2- 172.21.15.7:0/2987387773 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4b700a2ca0 0x7f4b700a5160 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00b2e0 tx=0x7f4b6c005960 comp rx=0 tx=0).stop 2024-06-10T01:14:47.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.128+0000 7f4b8a0e7700 1 -- 172.21.15.7:0/2987387773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b84102510 msgr2=0x7f4b8419e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:47.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.128+0000 7f4b8a0e7700 1 --2- 172.21.15.7:0/2987387773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b84102510 0x7f4b8419e7e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4b7400eb30 tx=0x7f4b7400ee40 comp rx=0 tx=0).stop 2024-06-10T01:14:47.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.128+0000 7f4b8a0e7700 1 -- 172.21.15.7:0/2987387773 shutdown_connections 2024-06-10T01:14:47.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.128+0000 7f4b8a0e7700 1 --2- 172.21.15.7:0/2987387773 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4b700a2ca0 0x7f4b700a5160 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.128+0000 7f4b8a0e7700 1 --2- 172.21.15.7:0/2987387773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b84108520 0x7f4b841989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.128+0000 7f4b8a0e7700 1 --2- 172.21.15.7:0/2987387773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b84102510 0x7f4b8419e7e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.128+0000 7f4b8a0e7700 1 --2- 172.21.15.7:0/2987387773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b84102ed0 0x7f4b8419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:47.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.128+0000 7f4b8a0e7700 1 -- 172.21.15.7:0/2987387773 >> 172.21.15.7:0/2987387773 conn(0x7f4b84079560 msgr2=0x7f4b84101120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:47.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.128+0000 7f4b8a0e7700 1 -- 172.21.15.7:0/2987387773 shutdown_connections 2024-06-10T01:14:47.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:47.128+0000 7f4b8a0e7700 1 -- 172.21.15.7:0/2987387773 wait complete. 2024-06-10T01:14:47.691 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:47 smithi007 ceph-mon[32398]: pgmap v695: 129 pgs: 129 active+clean; 110 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 1.9 MiB/s wr, 312 op/s 2024-06-10T01:14:47.691 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:47 smithi007 ceph-mon[32398]: scrub queued for path: / 2024-06-10T01:14:47.691 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:47 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:14:47.691 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:47 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:14:47.691 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:47 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:14:47.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:47 smithi175 ceph-mon[36636]: pgmap v695: 129 pgs: 129 active+clean; 110 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 1.9 MiB/s wr, 312 op/s 2024-06-10T01:14:47.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:47 smithi175 ceph-mon[36636]: scrub queued for path: / 2024-06-10T01:14:47.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:47 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:14:47.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:47 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:14:47.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:47 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:14:47.742 INFO:tasks.workunit.client.0.smithi007.stdout: 1 650363 7.88 MB/sec execute 328 sec latency 33.755 ms 2024-06-10T01:14:47.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:47 smithi078 ceph-mon[36683]: pgmap v695: 129 pgs: 129 active+clean; 110 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 1.9 MiB/s wr, 312 op/s 2024-06-10T01:14:47.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:47 smithi078 ceph-mon[36683]: scrub queued for path: / 2024-06-10T01:14:47.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:47 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:14:47.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:47 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:14:47.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:47 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:14:48.006 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:14:48.270 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:14:48.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:48 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:14:48.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:48 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:14:48.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:48 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:14:48.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:48 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:14:48.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:48 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:14:48.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:48 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:14:48.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:48 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:14:48.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:48 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:14:48.712 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:48 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:14:48.712 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:48 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:14:48.712 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:48 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:14:48.712 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:48 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:14:48.713 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:48 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:14:48.713 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:48 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:14:48.713 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:48 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:14:48.713 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:48 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:14:48.742 INFO:tasks.workunit.client.0.smithi007.stdout: 1 652665 7.89 MB/sec execute 329 sec latency 15.696 ms 2024-06-10T01:14:48.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:48 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:14:48.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:48 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:14:48.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:48 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:14:48.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:48 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:14:48.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:48 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:14:48.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:48 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:14:48.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:48 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:14:48.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:48 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:14:49.657 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:49 smithi007 ceph-mon[32398]: pgmap v696: 129 pgs: 129 active+clean; 110 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 1.5 MiB/s wr, 252 op/s 2024-06-10T01:14:49.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:49 smithi175 ceph-mon[36636]: pgmap v696: 129 pgs: 129 active+clean; 110 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 1.5 MiB/s wr, 252 op/s 2024-06-10T01:14:49.742 INFO:tasks.workunit.client.0.smithi007.stdout: 1 654246 7.89 MB/sec execute 330 sec latency 35.167 ms 2024-06-10T01:14:49.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:49 smithi078 ceph-mon[36683]: pgmap v696: 129 pgs: 129 active+clean; 110 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 1.5 MiB/s wr, 252 op/s 2024-06-10T01:14:49.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.920+0000 7fc957e2b700 1 -- 172.21.15.7:0/2651225720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc95006b350 msgr2=0x7fc95010e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:49.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.920+0000 7fc957e2b700 1 --2- 172.21.15.7:0/2651225720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc95006b350 0x7fc95010e9f0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fc940009990 tx=0x7fc940009ca0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.920+0000 7fc957e2b700 1 -- 172.21.15.7:0/2651225720 shutdown_connections 2024-06-10T01:14:49.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.920+0000 7fc957e2b700 1 --2- 172.21.15.7:0/2651225720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc95010ef30 0x7fc950111320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.920+0000 7fc957e2b700 1 --2- 172.21.15.7:0/2651225720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc95006aa30 0x7fc95006ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.920+0000 7fc957e2b700 1 --2- 172.21.15.7:0/2651225720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc95006b350 0x7fc95010e9f0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.920+0000 7fc957e2b700 1 -- 172.21.15.7:0/2651225720 >> 172.21.15.7:0/2651225720 conn(0x7fc950078300 msgr2=0x7fc950078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:49.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.920+0000 7fc957e2b700 1 -- 172.21.15.7:0/2651225720 shutdown_connections 2024-06-10T01:14:49.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.920+0000 7fc957e2b700 1 -- 172.21.15.7:0/2651225720 wait complete. 2024-06-10T01:14:49.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.921+0000 7fc957e2b700 1 Processor -- start 2024-06-10T01:14:49.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.921+0000 7fc957e2b700 1 -- start start 2024-06-10T01:14:49.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.921+0000 7fc957e2b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc95006aa30 0x7fc95019af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:49.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.921+0000 7fc957e2b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc95006b350 0x7fc95019b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:49.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.921+0000 7fc957e2b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc95010ef30 0x7fc95019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:49.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.921+0000 7fc957e2b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc9501149b0 con 0x7fc95006b350 2024-06-10T01:14:49.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.921+0000 7fc957e2b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc950114830 con 0x7fc95006aa30 2024-06-10T01:14:49.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.921+0000 7fc957e2b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc950114b30 con 0x7fc95010ef30 2024-06-10T01:14:49.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.921+0000 7fc955bc7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc95006aa30 0x7fc95019af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:49.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.921+0000 7fc955bc7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc95006aa30 0x7fc95019af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:44360/0 (socket says 172.21.15.7:44360) 2024-06-10T01:14:49.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.921+0000 7fc955bc7700 1 -- 172.21.15.7:0/1589244568 learned_addr learned my addr 172.21.15.7:0/1589244568 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:49.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.921+0000 7fc9563c8700 1 --2- 172.21.15.7:0/1589244568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc95010ef30 0x7fc95019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:49.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.922+0000 7fc9553c6700 1 --2- 172.21.15.7:0/1589244568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc95006b350 0x7fc95019b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:49.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.922+0000 7fc9553c6700 1 -- 172.21.15.7:0/1589244568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc95010ef30 msgr2=0x7fc95019f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:49.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.922+0000 7fc9553c6700 1 --2- 172.21.15.7:0/1589244568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc95010ef30 0x7fc95019f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.922+0000 7fc9553c6700 1 -- 172.21.15.7:0/1589244568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc95006aa30 msgr2=0x7fc95019af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:49.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.922+0000 7fc9553c6700 1 --2- 172.21.15.7:0/1589244568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc95006aa30 0x7fc95019af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.922+0000 7fc9553c6700 1 -- 172.21.15.7:0/1589244568 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc95019ff00 con 0x7fc95006b350 2024-06-10T01:14:49.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.922+0000 7fc9563c8700 1 --2- 172.21.15.7:0/1589244568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc95010ef30 0x7fc95019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:49.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.922+0000 7fc9553c6700 1 --2- 172.21.15.7:0/1589244568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc95006b350 0x7fc95019b480 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fc9400059d0 tx=0x7fc94000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:49.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.923+0000 7fc946ffd700 1 -- 172.21.15.7:0/1589244568 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc940027070 con 0x7fc95006b350 2024-06-10T01:14:49.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.923+0000 7fc957e2b700 1 -- 172.21.15.7:0/1589244568 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9501a0190 con 0x7fc95006b350 2024-06-10T01:14:49.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.923+0000 7fc957e2b700 1 -- 172.21.15.7:0/1589244568 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9501a7a70 con 0x7fc95006b350 2024-06-10T01:14:49.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.923+0000 7fc946ffd700 1 -- 172.21.15.7:0/1589244568 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc940003d70 con 0x7fc95006b350 2024-06-10T01:14:49.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.923+0000 7fc946ffd700 1 -- 172.21.15.7:0/1589244568 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc940022e30 con 0x7fc95006b350 2024-06-10T01:14:49.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.925+0000 7fc946ffd700 1 -- 172.21.15.7:0/1589244568 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fc94002a070 con 0x7fc95006b350 2024-06-10T01:14:49.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.926+0000 7fc946ffd700 1 --2- 172.21.15.7:0/1589244568 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc93c0a2ca0 0x7fc93c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:49.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.926+0000 7fc955bc7700 1 --2- 172.21.15.7:0/1589244568 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc93c0a2ca0 0x7fc93c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:49.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.927+0000 7fc946ffd700 1 -- 172.21.15.7:0/1589244568 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc9400cff90 con 0x7fc95006b350 2024-06-10T01:14:49.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.927+0000 7fc955bc7700 1 --2- 172.21.15.7:0/1589244568 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc93c0a2ca0 0x7fc93c0a5160 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc94c009d60 tx=0x7fc94c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:49.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.928+0000 7fc9626aeb80 1 Processor -- start 2024-06-10T01:14:49.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.928+0000 7fc9626aeb80 1 -- start start 2024-06-10T01:14:49.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.928+0000 7fc9626aeb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e23d23af60 0x55e23d23b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:49.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.928+0000 7fc9626aeb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e23d23b880 0x55e23d235260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:49.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.928+0000 7fc9626aeb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e23d2357a0 0x55e23d235bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:49.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.928+0000 7fc9626aeb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e23d127f60 con 0x55e23d23af60 2024-06-10T01:14:49.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.928+0000 7fc9626aeb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e23d186850 con 0x55e23d23b880 2024-06-10T01:14:49.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.928+0000 7fc9626aeb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e23d0ee460 con 0x55e23d2357a0 2024-06-10T01:14:49.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.928+0000 7fc955bc7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e23d23af60 0x55e23d23b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:49.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.928+0000 7fc9553c6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e23d23b880 0x55e23d235260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:49.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.928+0000 7fc9553c6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e23d23b880 0x55e23d235260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:44376/0 (socket says 172.21.15.7:44376) 2024-06-10T01:14:49.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.928+0000 7fc955bc7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e23d23af60 0x55e23d23b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45508/0 (socket says 172.21.15.7:45508) 2024-06-10T01:14:49.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.928+0000 7fc9553c6700 1 -- 172.21.15.7:0/132580045 learned_addr learned my addr 172.21.15.7:0/132580045 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:49.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.928+0000 7fc9563c8700 1 --2- 172.21.15.7:0/132580045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e23d2357a0 0x55e23d235bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:49.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.929+0000 7fc9553c6700 1 -- 172.21.15.7:0/132580045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e23d2357a0 msgr2=0x55e23d235bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:49.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.929+0000 7fc9553c6700 1 --2- 172.21.15.7:0/132580045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e23d2357a0 0x55e23d235bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.929+0000 7fc9553c6700 1 -- 172.21.15.7:0/132580045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e23d23af60 msgr2=0x55e23d23b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:49.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.929+0000 7fc9553c6700 1 --2- 172.21.15.7:0/132580045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e23d23af60 0x55e23d23b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.929+0000 7fc9553c6700 1 -- 172.21.15.7:0/132580045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e23d0d58d0 con 0x55e23d23b880 2024-06-10T01:14:49.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.929+0000 7fc9553c6700 1 --2- 172.21.15.7:0/132580045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e23d23b880 0x55e23d235260 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc9400a4b50 tx=0x7fc94009c050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=938a041545fe9199 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:49.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.929+0000 7fc93bfff700 1 -- 172.21.15.7:0/132580045 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9400095d0 con 0x55e23d23b880 2024-06-10T01:14:49.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.929+0000 7fc93bfff700 1 -- 172.21.15.7:0/132580045 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc94009cba0 con 0x55e23d23b880 2024-06-10T01:14:49.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.929+0000 7fc93bfff700 1 -- 172.21.15.7:0/132580045 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9400a4e50 con 0x55e23d23b880 2024-06-10T01:14:49.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.930+0000 7fc9626aeb80 1 -- 172.21.15.7:0/132580045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e23d23b880 msgr2=0x55e23d235260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:49.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.930+0000 7fc9626aeb80 1 --2- 172.21.15.7:0/132580045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e23d23b880 0x55e23d235260 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc9400a4b50 tx=0x7fc94009c050 comp rx=0 tx=0).stop 2024-06-10T01:14:49.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.930+0000 7fc9626aeb80 1 -- 172.21.15.7:0/132580045 shutdown_connections 2024-06-10T01:14:49.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.930+0000 7fc9626aeb80 1 --2- 172.21.15.7:0/132580045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e23d2357a0 0x55e23d235bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.930+0000 7fc9626aeb80 1 --2- 172.21.15.7:0/132580045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e23d23af60 0x55e23d23b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.930+0000 7fc9626aeb80 1 --2- 172.21.15.7:0/132580045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e23d23b880 0x55e23d235260 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.930+0000 7fc9626aeb80 1 -- 172.21.15.7:0/132580045 >> 172.21.15.7:0/132580045 conn(0x55e23d162080 msgr2=0x55e23d230220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:49.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.930+0000 7fc9626aeb80 1 -- 172.21.15.7:0/132580045 shutdown_connections 2024-06-10T01:14:49.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.930+0000 7fc9626aeb80 1 -- 172.21.15.7:0/132580045 wait complete. 2024-06-10T01:14:49.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.930+0000 7fc9626aeb80 1 -- start start 2024-06-10T01:14:49.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.930+0000 7fc9626aeb80 1 Processor -- start 2024-06-10T01:14:49.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.931+0000 7fc9626aeb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e23d2357a0 0x55e23d2415f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:49.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.931+0000 7fc9626aeb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e23d23af60 0x55e23d23e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:49.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.931+0000 7fc9626aeb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e23d23b880 0x55e23d23ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:49.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.931+0000 7fc9626aeb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e23d126fc0 con 0x55e23d23af60 2024-06-10T01:14:49.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.931+0000 7fc9626aeb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e23d182810 con 0x55e23d2357a0 2024-06-10T01:14:49.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.931+0000 7fc9626aeb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e23d242010 con 0x55e23d23b880 2024-06-10T01:14:49.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.931+0000 7fc9626aeb80 10 client.0 _ll_register_callbacks cb 0x55e23d11c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:14:49.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.931+0000 7fc9553c6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e23d23af60 0x55e23d23e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:49.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.931+0000 7fc955bc7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e23d2357a0 0x55e23d2415f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:49.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.931+0000 7fc9563c8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e23d23b880 0x55e23d23ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:49.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.931+0000 7fc955bc7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e23d2357a0 0x55e23d2415f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:44386/0 (socket says 172.21.15.7:44386) 2024-06-10T01:14:49.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.931+0000 7fc955bc7700 1 -- 172.21.15.7:0/2330503095 learned_addr learned my addr 172.21.15.7:0/2330503095 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:49.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.931+0000 7fc9563c8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e23d23b880 0x55e23d23ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52774/0 (socket says 172.21.15.7:52774) 2024-06-10T01:14:49.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.931+0000 7fc955bc7700 1 -- 172.21.15.7:0/2330503095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e23d23b880 msgr2=0x55e23d23ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:49.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.931+0000 7fc955bc7700 1 --2- 172.21.15.7:0/2330503095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e23d23b880 0x55e23d23ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.931+0000 7fc955bc7700 1 -- 172.21.15.7:0/2330503095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e23d23af60 msgr2=0x55e23d23e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:49.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.931+0000 7fc955bc7700 1 --2- 172.21.15.7:0/2330503095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e23d23af60 0x55e23d23e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.932+0000 7fc955bc7700 1 -- 172.21.15.7:0/2330503095 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e23d14c750 con 0x55e23d2357a0 2024-06-10T01:14:49.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.932+0000 7fc955bc7700 1 --2- 172.21.15.7:0/2330503095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e23d2357a0 0x55e23d2415f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc94c009870 tx=0x7fc94c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ba85ebd2727bfe5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:49.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.932+0000 7fc9397fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:14:49.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.932+0000 7fc9397fa700 1 -- 172.21.15.7:0/2330503095 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c02b060 con 0x55e23d2357a0 2024-06-10T01:14:49.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.932+0000 7fc9397fa700 1 -- 172.21.15.7:0/2330503095 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc94c009040 con 0x55e23d2357a0 2024-06-10T01:14:49.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.932+0000 7fc9626aeb80 1 -- 172.21.15.7:0/2330503095 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc94009cba0 con 0x55e23d2357a0 2024-06-10T01:14:49.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.932+0000 7fc9397fa700 1 -- 172.21.15.7:0/2330503095 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc94c032050 con 0x55e23d2357a0 2024-06-10T01:14:49.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.932+0000 7fc9626aeb80 10 client.27083 fetch_fsmap learned FSMap version 35 2024-06-10T01:14:49.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.932+0000 7fc9626aeb80 1 -- 172.21.15.7:0/2330503095 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e23d24c770 con 0x55e23d2357a0 2024-06-10T01:14:49.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.932+0000 7fc9397fa700 1 -- 172.21.15.7:0/2330503095 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc94009cba0 con 0x55e23d2357a0 2024-06-10T01:14:49.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.933+0000 7fc9397fa700 1 -- 172.21.15.7:0/2330503095 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7fc94c026bd0 con 0x55e23d2357a0 2024-06-10T01:14:49.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.933+0000 7fc9626aeb80 10 client.27083 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:14:49.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.933+0000 7fc9626aeb80 10 client.27083 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:14:49.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.933+0000 7fc9626aeb80 20 client.27083 populate_metadata read hostname 'smithi007' 2024-06-10T01:14:49.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.933+0000 7fc9626aeb80 1 --2- 172.21.15.7:0/2330503095 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e23d24ee50 0x55e23d251290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:49.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.933+0000 7fc9626aeb80 4 client.27083 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:14:49.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.933+0000 7fc9626aeb80 1 -- 172.21.15.7:0/2330503095 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e23d193d50 con 0x55e23d24ee50 2024-06-10T01:14:49.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.933+0000 7fc9553c6700 1 --2- 172.21.15.7:0/2330503095 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e23d24ee50 0x55e23d251290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:49.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.933+0000 7fc9397fa700 10 client.27083 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:49.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.933+0000 7fc9553c6700 1 --2- 172.21.15.7:0/2330503095 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e23d24ee50 0x55e23d251290 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=917efd1b0a3c5b97 server_cookie=949e9cd1a01aed0c in_seq=0 out_seq=0 2024-06-10T01:14:49.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.934+0000 7fc9397fa700 1 -- 172.21.15.7:0/2330503095 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e23d193d50 con 0x55e23d24ee50 2024-06-10T01:14:49.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.934+0000 7fc9397fa700 10 client.27083 handle_command_reply: tid=0 2024-06-10T01:14:49.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.934+0000 7fc9626aeb80 1 client.27083 shutdown 2024-06-10T01:14:49.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.934+0000 7fc9626aeb80 1 -- 172.21.15.7:0/2330503095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e23d2357a0 msgr2=0x55e23d2415f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:49.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.934+0000 7fc9626aeb80 1 --2- 172.21.15.7:0/2330503095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e23d2357a0 0x55e23d2415f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc94c009870 tx=0x7fc94c0047c0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.934+0000 7fc9626aeb80 1 -- 172.21.15.7:0/2330503095 shutdown_connections 2024-06-10T01:14:49.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.934+0000 7fc9626aeb80 1 --2- 172.21.15.7:0/2330503095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e23d23b880 0x55e23d23ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.934+0000 7fc9626aeb80 1 --2- 172.21.15.7:0/2330503095 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e23d24ee50 0x55e23d251290 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.934+0000 7fc9626aeb80 1 --2- 172.21.15.7:0/2330503095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e23d2357a0 0x55e23d2415f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.934+0000 7fc9397fa700 0 client.27083 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:49.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.934+0000 7fc9626aeb80 1 --2- 172.21.15.7:0/2330503095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e23d23af60 0x55e23d23e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.934+0000 7fc9626aeb80 1 -- 172.21.15.7:0/2330503095 >> 172.21.15.7:0/2330503095 conn(0x55e23d162080 msgr2=0x55e23d22f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:49.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.934+0000 7fc9626aeb80 1 -- 172.21.15.7:0/2330503095 shutdown_connections 2024-06-10T01:14:49.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.934+0000 7fc9626aeb80 1 -- 172.21.15.7:0/2330503095 wait complete. 2024-06-10T01:14:49.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.934+0000 7fc9626aeb80 20 client.27083 trim_cache size 0 max 16384 2024-06-10T01:14:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.956+0000 7fc9626aeb80 1 Processor -- start 2024-06-10T01:14:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.956+0000 7fc9626aeb80 1 -- start start 2024-06-10T01:14:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.956+0000 7fc9626aeb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e23d2357a0 0x55e23d235260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.956+0000 7fc9626aeb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e23d23af60 0x55e23d23ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.956+0000 7fc9626aeb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e23d23b880 0x55e23d23c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.956+0000 7fc9626aeb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc934011360 con 0x55e23d23af60 2024-06-10T01:14:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.956+0000 7fc9626aeb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc93400eb90 con 0x55e23d2357a0 2024-06-10T01:14:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.956+0000 7fc9626aeb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc93400c9e0 con 0x55e23d23b880 2024-06-10T01:14:49.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.957+0000 7fc955bc7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e23d2357a0 0x55e23d235260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:49.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.957+0000 7fc9553c6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e23d23af60 0x55e23d23ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:49.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.957+0000 7fc955bc7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e23d2357a0 0x55e23d235260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:44398/0 (socket says 172.21.15.7:44398) 2024-06-10T01:14:49.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.957+0000 7fc955bc7700 1 -- 172.21.15.7:0/993826423 learned_addr learned my addr 172.21.15.7:0/993826423 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:49.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.957+0000 7fc9553c6700 1 --2- 172.21.15.7:0/993826423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e23d23af60 0x55e23d23ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45512/0 (socket says 172.21.15.7:45512) 2024-06-10T01:14:49.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.957+0000 7fc9563c8700 1 --2- 172.21.15.7:0/993826423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e23d23b880 0x55e23d23c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:49.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.957+0000 7fc955bc7700 1 -- 172.21.15.7:0/993826423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e23d23b880 msgr2=0x55e23d23c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:49.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.957+0000 7fc955bc7700 1 --2- 172.21.15.7:0/993826423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e23d23b880 0x55e23d23c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.957+0000 7fc955bc7700 1 -- 172.21.15.7:0/993826423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e23d23af60 msgr2=0x55e23d23ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:49.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.957+0000 7fc955bc7700 1 --2- 172.21.15.7:0/993826423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e23d23af60 0x55e23d23ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.957+0000 7fc955bc7700 1 -- 172.21.15.7:0/993826423 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e23d23b680 con 0x55e23d2357a0 2024-06-10T01:14:49.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.957+0000 7fc9563c8700 1 --2- 172.21.15.7:0/993826423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e23d23b880 0x55e23d23c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:49.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.957+0000 7fc9553c6700 1 --2- 172.21.15.7:0/993826423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e23d23af60 0x55e23d23ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:49.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.957+0000 7fc955bc7700 1 --2- 172.21.15.7:0/993826423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e23d2357a0 0x55e23d235260 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc94c016040 tx=0x7fc94c01d7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ac7d5e0c037bc3a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:49.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.957+0000 7fc93b7fe700 1 -- 172.21.15.7:0/993826423 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c03b070 con 0x55e23d2357a0 2024-06-10T01:14:49.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.957+0000 7fc93b7fe700 1 -- 172.21.15.7:0/993826423 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc94c009a90 con 0x55e23d2357a0 2024-06-10T01:14:49.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.957+0000 7fc93b7fe700 1 -- 172.21.15.7:0/993826423 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c0263e0 con 0x55e23d2357a0 2024-06-10T01:14:49.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.958+0000 7fc9626aeb80 1 -- 172.21.15.7:0/993826423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e23d2357a0 msgr2=0x55e23d235260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:49.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.958+0000 7fc9626aeb80 1 --2- 172.21.15.7:0/993826423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e23d2357a0 0x55e23d235260 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc94c016040 tx=0x7fc94c01d7b0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.958+0000 7fc9626aeb80 1 -- 172.21.15.7:0/993826423 shutdown_connections 2024-06-10T01:14:49.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.958+0000 7fc9626aeb80 1 --2- 172.21.15.7:0/993826423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e23d23b880 0x55e23d23c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.958+0000 7fc9626aeb80 1 --2- 172.21.15.7:0/993826423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e23d2357a0 0x55e23d235260 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.958+0000 7fc9626aeb80 1 --2- 172.21.15.7:0/993826423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e23d23af60 0x55e23d23ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.958+0000 7fc9626aeb80 1 -- 172.21.15.7:0/993826423 >> 172.21.15.7:0/993826423 conn(0x55e23d162080 msgr2=0x55e23d240120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:49.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.958+0000 7fc9626aeb80 1 -- 172.21.15.7:0/993826423 shutdown_connections 2024-06-10T01:14:49.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.958+0000 7fc9626aeb80 1 -- 172.21.15.7:0/993826423 wait complete. 2024-06-10T01:14:49.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.959+0000 7fc9626aeb80 1 -- start start 2024-06-10T01:14:49.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.959+0000 7fc9626aeb80 1 Processor -- start 2024-06-10T01:14:49.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.959+0000 7fc9626aeb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e23d2357a0 0x55e23d247b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:49.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.959+0000 7fc9626aeb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e23d23af60 0x55e23d256660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:49.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.959+0000 7fc9626aeb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e23d23b880 0x55e23d256d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:49.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.959+0000 7fc9626aeb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc93400d050 con 0x55e23d23b880 2024-06-10T01:14:49.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.959+0000 7fc9626aeb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc93400e4c0 con 0x55e23d2357a0 2024-06-10T01:14:49.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.959+0000 7fc9626aeb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc93400d720 con 0x55e23d23af60 2024-06-10T01:14:49.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.959+0000 7fc955bc7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e23d2357a0 0x55e23d247b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:49.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.959+0000 7fc9563c8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e23d23b880 0x55e23d256d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:49.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.959+0000 7fc9563c8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e23d23b880 0x55e23d256d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45516/0 (socket says 172.21.15.7:45516) 2024-06-10T01:14:49.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.959+0000 7fc9553c6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e23d23af60 0x55e23d256660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:49.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.959+0000 7fc9563c8700 1 -- 172.21.15.7:0/4248534127 learned_addr learned my addr 172.21.15.7:0/4248534127 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:49.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.959+0000 7fc9626aeb80 10 client.0 _ll_register_callbacks cb 0x55e23d11c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:14:49.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.959+0000 7fc955bc7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e23d2357a0 0x55e23d247b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:44402/0 (socket says 172.21.15.7:44402) 2024-06-10T01:14:49.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.960+0000 7fc9563c8700 1 -- 172.21.15.7:0/4248534127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e23d23af60 msgr2=0x55e23d256660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:49.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.960+0000 7fc9563c8700 1 --2- 172.21.15.7:0/4248534127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e23d23af60 0x55e23d256660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.960+0000 7fc9563c8700 1 -- 172.21.15.7:0/4248534127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e23d2357a0 msgr2=0x55e23d247b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:49.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.960+0000 7fc9563c8700 1 --2- 172.21.15.7:0/4248534127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e23d2357a0 0x55e23d247b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.960+0000 7fc9563c8700 1 -- 172.21.15.7:0/4248534127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e23d2b8b30 con 0x55e23d23b880 2024-06-10T01:14:49.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.960+0000 7fc9553c6700 1 --2- 172.21.15.7:0/4248534127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e23d23af60 0x55e23d256660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:49.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.960+0000 7fc9563c8700 1 --2- 172.21.15.7:0/4248534127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e23d23b880 0x55e23d256d50 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fc94800ee50 tx=0x7fc9480077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a08844702284688 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:49.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.960+0000 7fc9397fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:14:49.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.960+0000 7fc9397fa700 1 -- 172.21.15.7:0/4248534127 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc948007e50 con 0x55e23d23b880 2024-06-10T01:14:49.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.960+0000 7fc9626aeb80 1 -- 172.21.15.7:0/4248534127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e23d18e2d0 con 0x55e23d23b880 2024-06-10T01:14:49.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.960+0000 7fc9397fa700 1 -- 172.21.15.7:0/4248534127 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc948004bb0 con 0x55e23d23b880 2024-06-10T01:14:49.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.960+0000 7fc9397fa700 1 -- 172.21.15.7:0/4248534127 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc94802b050 con 0x55e23d23b880 2024-06-10T01:14:49.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.960+0000 7fc9626aeb80 10 client.17694 fetch_fsmap learned FSMap version 35 2024-06-10T01:14:49.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.960+0000 7fc9626aeb80 1 -- 172.21.15.7:0/4248534127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e23d2b9d50 con 0x55e23d23b880 2024-06-10T01:14:49.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.961+0000 7fc9397fa700 1 -- 172.21.15.7:0/4248534127 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e23d18e2d0 con 0x55e23d23b880 2024-06-10T01:14:49.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.961+0000 7fc9397fa700 1 -- 172.21.15.7:0/4248534127 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7fc9480166a0 con 0x55e23d23b880 2024-06-10T01:14:49.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.961+0000 7fc9626aeb80 10 client.17694 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:14:49.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.961+0000 7fc9626aeb80 10 client.17694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:14:49.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.961+0000 7fc9626aeb80 20 client.17694 populate_metadata read hostname 'smithi007' 2024-06-10T01:14:49.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.961+0000 7fc9626aeb80 1 --2- 172.21.15.7:0/4248534127 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e23d2ba520 0x55e23d2bc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:49.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.961+0000 7fc9626aeb80 4 client.17694 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:14:49.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.961+0000 7fc9626aeb80 1 -- 172.21.15.7:0/4248534127 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e23d141f50 con 0x55e23d2ba520 2024-06-10T01:14:49.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.961+0000 7fc955bc7700 1 --2- 172.21.15.7:0/4248534127 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e23d2ba520 0x55e23d2bc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:49.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.961+0000 7fc9397fa700 10 client.17694 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:49.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.962+0000 7fc955bc7700 1 --2- 172.21.15.7:0/4248534127 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e23d2ba520 0x55e23d2bc940 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=587a1ebd91093ca1 server_cookie=7861459f92c08254 in_seq=0 out_seq=0 2024-06-10T01:14:49.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.962+0000 7fc9397fa700 1 -- 172.21.15.7:0/4248534127 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e23d141f50 con 0x55e23d2ba520 2024-06-10T01:14:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.962+0000 7fc9397fa700 10 client.17694 handle_command_reply: tid=0 2024-06-10T01:14:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.962+0000 7fc9626aeb80 1 client.17694 shutdown 2024-06-10T01:14:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.962+0000 7fc9626aeb80 1 -- 172.21.15.7:0/4248534127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e23d23b880 msgr2=0x55e23d256d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.962+0000 7fc9626aeb80 1 --2- 172.21.15.7:0/4248534127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e23d23b880 0x55e23d256d50 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fc94800ee50 tx=0x7fc9480077e0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.962+0000 7fc9626aeb80 1 -- 172.21.15.7:0/4248534127 shutdown_connections 2024-06-10T01:14:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.962+0000 7fc9626aeb80 1 --2- 172.21.15.7:0/4248534127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e23d23b880 0x55e23d256d50 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.962+0000 7fc9626aeb80 1 --2- 172.21.15.7:0/4248534127 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e23d2ba520 0x55e23d2bc940 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.962+0000 7fc9626aeb80 1 --2- 172.21.15.7:0/4248534127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e23d2357a0 0x55e23d247b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.962+0000 7fc9626aeb80 1 --2- 172.21.15.7:0/4248534127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e23d23af60 0x55e23d256660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.962+0000 7fc9397fa700 0 client.17694 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:49.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.962+0000 7fc9626aeb80 1 -- 172.21.15.7:0/4248534127 >> 172.21.15.7:0/4248534127 conn(0x55e23d162080 msgr2=0x55e23d23a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:49.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.962+0000 7fc9626aeb80 1 -- 172.21.15.7:0/4248534127 shutdown_connections 2024-06-10T01:14:49.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.962+0000 7fc9626aeb80 1 -- 172.21.15.7:0/4248534127 wait complete. 2024-06-10T01:14:49.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.963+0000 7fc9626aeb80 20 client.17694 trim_cache size 0 max 16384 2024-06-10T01:14:49.964 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:14:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T01:14:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-10T01:14:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "5c292e44-8484-4839-a4b0-54e84fece3e0": { 2024-06-10T01:14:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-10T01:14:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "5c292e44-8484-4839-a4b0-54e84fece3e0", 2024-06-10T01:14:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-10T01:14:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:14:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:14:49.965 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:14:49.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.963+0000 7fc957e2b700 1 -- 172.21.15.7:0/1589244568 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc93c0a2ca0 msgr2=0x7fc93c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:49.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.963+0000 7fc957e2b700 1 --2- 172.21.15.7:0/1589244568 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc93c0a2ca0 0x7fc93c0a5160 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc94c009d60 tx=0x7fc94c009260 comp rx=0 tx=0).stop 2024-06-10T01:14:49.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.963+0000 7fc957e2b700 1 -- 172.21.15.7:0/1589244568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc95006b350 msgr2=0x7fc95019b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:49.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.963+0000 7fc957e2b700 1 --2- 172.21.15.7:0/1589244568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc95006b350 0x7fc95019b480 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fc9400059d0 tx=0x7fc94000bab0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.963+0000 7fc957e2b700 1 -- 172.21.15.7:0/1589244568 shutdown_connections 2024-06-10T01:14:49.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.963+0000 7fc957e2b700 1 --2- 172.21.15.7:0/1589244568 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc93c0a2ca0 0x7fc93c0a5160 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.963+0000 7fc957e2b700 1 --2- 172.21.15.7:0/1589244568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc95010ef30 0x7fc95019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.963+0000 7fc957e2b700 1 --2- 172.21.15.7:0/1589244568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc95006aa30 0x7fc95019af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.963+0000 7fc957e2b700 1 --2- 172.21.15.7:0/1589244568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc95006b350 0x7fc95019b480 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:49.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.963+0000 7fc957e2b700 1 -- 172.21.15.7:0/1589244568 >> 172.21.15.7:0/1589244568 conn(0x7fc950078300 msgr2=0x7fc950110420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:49.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.964+0000 7fc957e2b700 1 -- 172.21.15.7:0/1589244568 shutdown_connections 2024-06-10T01:14:49.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:49.964+0000 7fc957e2b700 1 -- 172.21.15.7:0/1589244568 wait complete. 2024-06-10T01:14:50.198 INFO:tasks.quiescer.fs.[cephfs]:Total duration (35.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-06-10T01:14:50.198 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.8 2024-06-10T01:14:50.198 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:14:50.576 INFO:tasks.cephfs.filesystem:scrub status for tag:5c292e44-8484-4839-a4b0-54e84fece3e0 - {'path': '/', 'tag': '5c292e44-8484-4839-a4b0-54e84fece3e0', 'options': 'recursive,force'} 2024-06-10T01:14:50.620 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:14:50.742 INFO:tasks.workunit.client.0.smithi007.stdout: 1 656648 7.90 MB/sec execute 331 sec latency 16.061 ms 2024-06-10T01:14:51.693 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:51 smithi007 ceph-mon[32398]: pgmap v697: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 1.6 MiB/s wr, 323 op/s 2024-06-10T01:14:51.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:51 smithi175 ceph-mon[36636]: pgmap v697: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 1.6 MiB/s wr, 323 op/s 2024-06-10T01:14:51.742 INFO:tasks.workunit.client.0.smithi007.stdout: 1 658054 7.90 MB/sec execute 332 sec latency 23.823 ms 2024-06-10T01:14:51.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:51 smithi078 ceph-mon[36683]: pgmap v697: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 1.6 MiB/s wr, 323 op/s 2024-06-10T01:14:52.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.215+0000 7fbbee05d700 1 -- 172.21.15.7:0/1684649249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe8107450 msgr2=0x7fbbe8107830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:52.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.215+0000 7fbbee05d700 1 --2- 172.21.15.7:0/1684649249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe8107450 0x7fbbe8107830 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80098f0 tx=0x7fbbd8009c00 comp rx=0 tx=0).stop 2024-06-10T01:14:52.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.215+0000 7fbbee05d700 1 -- 172.21.15.7:0/1684649249 shutdown_connections 2024-06-10T01:14:52.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.215+0000 7fbbee05d700 1 --2- 172.21.15.7:0/1684649249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbe8102ee0 0x7fbbe810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.215+0000 7fbbee05d700 1 --2- 172.21.15.7:0/1684649249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe8107450 0x7fbbe8107830 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.215+0000 7fbbee05d700 1 --2- 172.21.15.7:0/1684649249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbe8102540 0x7fbbe81029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.215+0000 7fbbee05d700 1 -- 172.21.15.7:0/1684649249 >> 172.21.15.7:0/1684649249 conn(0x7fbbe80795f0 msgr2=0x7fbbe8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:52.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.215+0000 7fbbee05d700 1 -- 172.21.15.7:0/1684649249 shutdown_connections 2024-06-10T01:14:52.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.215+0000 7fbbee05d700 1 -- 172.21.15.7:0/1684649249 wait complete. 2024-06-10T01:14:52.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.215+0000 7fbbee05d700 1 Processor -- start 2024-06-10T01:14:52.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.216+0000 7fbbee05d700 1 -- start start 2024-06-10T01:14:52.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.216+0000 7fbbee05d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbe8102540 0x7fbbe819e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:52.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.216+0000 7fbbee05d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe8102ee0 0x7fbbe819ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:52.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.216+0000 7fbbee05d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbe8107450 0x7fbbe8198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:52.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.216+0000 7fbbee05d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe81127c0 con 0x7fbbe8102540 2024-06-10T01:14:52.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.216+0000 7fbbee05d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe8112640 con 0x7fbbe8102ee0 2024-06-10T01:14:52.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.216+0000 7fbbee05d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe8112940 con 0x7fbbe8107450 2024-06-10T01:14:52.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.216+0000 7fbbe77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbe8102540 0x7fbbe819e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:52.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.216+0000 7fbbe6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe8102ee0 0x7fbbe819ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:52.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.216+0000 7fbbe77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbe8102540 0x7fbbe819e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58204/0 (socket says 172.21.15.7:58204) 2024-06-10T01:14:52.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.216+0000 7fbbe77fe700 1 -- 172.21.15.7:0/664538379 learned_addr learned my addr 172.21.15.7:0/664538379 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:52.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.216+0000 7fbbe7fff700 1 --2- 172.21.15.7:0/664538379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbe8107450 0x7fbbe8198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:52.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.216+0000 7fbbe77fe700 1 -- 172.21.15.7:0/664538379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbe8107450 msgr2=0x7fbbe8198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:52.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.216+0000 7fbbe77fe700 1 --2- 172.21.15.7:0/664538379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbe8107450 0x7fbbe8198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.216+0000 7fbbe77fe700 1 -- 172.21.15.7:0/664538379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe8102ee0 msgr2=0x7fbbe819ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:52.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.216+0000 7fbbe77fe700 1 --2- 172.21.15.7:0/664538379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe8102ee0 0x7fbbe819ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.216+0000 7fbbe77fe700 1 -- 172.21.15.7:0/664538379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe8199260 con 0x7fbbe8102540 2024-06-10T01:14:52.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.217+0000 7fbbe77fe700 1 --2- 172.21.15.7:0/664538379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbe8102540 0x7fbbe819e8f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009fd0 tx=0x7fbbd800fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:52.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.217+0000 7fbbe4ff9700 1 -- 172.21.15.7:0/664538379 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8005180 con 0x7fbbe8102540 2024-06-10T01:14:52.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.217+0000 7fbbee05d700 1 -- 172.21.15.7:0/664538379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe81994f0 con 0x7fbbe8102540 2024-06-10T01:14:52.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.217+0000 7fbbee05d700 1 -- 172.21.15.7:0/664538379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe81a57c0 con 0x7fbbe8102540 2024-06-10T01:14:52.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.217+0000 7fbbe4ff9700 1 -- 172.21.15.7:0/664538379 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbd800b560 con 0x7fbbe8102540 2024-06-10T01:14:52.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.217+0000 7fbbe4ff9700 1 -- 172.21.15.7:0/664538379 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8003a20 con 0x7fbbe8102540 2024-06-10T01:14:52.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.218+0000 7fbbee05d700 1 -- 172.21.15.7:0/664538379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbe810aa80 con 0x7fbbe8102540 2024-06-10T01:14:52.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.219+0000 7fbbe4ff9700 1 -- 172.21.15.7:0/664538379 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fbbd8003ca0 con 0x7fbbe8102540 2024-06-10T01:14:52.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.220+0000 7fbbe4ff9700 1 --2- 172.21.15.7:0/664538379 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fbbd00a2c20 0x7fbbd00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:52.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.220+0000 7fbbe6ffd700 1 --2- 172.21.15.7:0/664538379 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fbbd00a2c20 0x7fbbd00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:52.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.220+0000 7fbbe4ff9700 1 -- 172.21.15.7:0/664538379 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fbbd80d07c0 con 0x7fbbe8102540 2024-06-10T01:14:52.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.220+0000 7fbbe6ffd700 1 --2- 172.21.15.7:0/664538379 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fbbd00a2c20 0x7fbbd00a50e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8079bd0 tx=0x7fbbdc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:52.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.224+0000 7fbbe4ff9700 1 -- 172.21.15.7:0/664538379 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbbd805e850 con 0x7fbbe8102540 2024-06-10T01:14:52.512 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:52 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:14:52.512 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:52 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:14:52.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.520+0000 7fbbee05d700 1 -- 172.21.15.7:0/664538379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbbe8199fe0 con 0x7fbbe8102540 2024-06-10T01:14:52.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.521+0000 7fbbe4ff9700 1 -- 172.21.15.7:0/664538379 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11860 (secure 0 0 0) 0x7fbbd8013030 con 0x7fbbe8102540 2024-06-10T01:14:52.516 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:14:52.516 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:14:51.406569+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24434,24436,24451]},"id":1}]} 2024-06-10T01:14:52.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.523+0000 7fbbee05d700 1 -- 172.21.15.7:0/664538379 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fbbd00a2c20 msgr2=0x7fbbd00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:52.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.523+0000 7fbbee05d700 1 --2- 172.21.15.7:0/664538379 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fbbd00a2c20 0x7fbbd00a50e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8079bd0 tx=0x7fbbdc009260 comp rx=0 tx=0).stop 2024-06-10T01:14:52.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.524+0000 7fbbee05d700 1 -- 172.21.15.7:0/664538379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbe8102540 msgr2=0x7fbbe819e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:52.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.524+0000 7fbbee05d700 1 --2- 172.21.15.7:0/664538379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbe8102540 0x7fbbe819e8f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009fd0 tx=0x7fbbd800fa70 comp rx=0 tx=0).stop 2024-06-10T01:14:52.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.524+0000 7fbbee05d700 1 -- 172.21.15.7:0/664538379 shutdown_connections 2024-06-10T01:14:52.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.524+0000 7fbbee05d700 1 --2- 172.21.15.7:0/664538379 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fbbd00a2c20 0x7fbbd00a50e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.524+0000 7fbbee05d700 1 --2- 172.21.15.7:0/664538379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbe8107450 0x7fbbe8198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.524+0000 7fbbee05d700 1 --2- 172.21.15.7:0/664538379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbe8102540 0x7fbbe819e8f0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.524+0000 7fbbee05d700 1 --2- 172.21.15.7:0/664538379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe8102ee0 0x7fbbe819ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.524+0000 7fbbee05d700 1 -- 172.21.15.7:0/664538379 >> 172.21.15.7:0/664538379 conn(0x7fbbe80795f0 msgr2=0x7fbbe8100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:52.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.524+0000 7fbbee05d700 1 -- 172.21.15.7:0/664538379 shutdown_connections 2024-06-10T01:14:52.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:52.524+0000 7fbbee05d700 1 -- 172.21.15.7:0/664538379 wait complete. 2024-06-10T01:14:52.521 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 36 2024-06-10T01:14:52.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:52 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:14:52.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:52 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:14:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.741+0000 7f63b031d640 1 -- 172.21.15.7:0/3444780242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63a8101a30 msgr2=0x7f63a8101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.741+0000 7f63b031d640 1 --2- 172.21.15.7:0/3444780242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63a8101a30 0x7f63a8101e50 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009900 tx=0x7f63a402a410 comp rx=0 tx=0).stop 2024-06-10T01:14:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.741+0000 7f63b031d640 1 -- 172.21.15.7:0/3444780242 shutdown_connections 2024-06-10T01:14:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.741+0000 7f63b031d640 1 --2- 172.21.15.7:0/3444780242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63a81035e0 0x7f63a8109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.741+0000 7f63b031d640 1 --2- 172.21.15.7:0/3444780242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63a8102c20 0x7f63a81030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.741+0000 7f63b031d640 1 --2- 172.21.15.7:0/3444780242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63a8101a30 0x7f63a8101e50 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.741+0000 7f63b031d640 1 -- 172.21.15.7:0/3444780242 >> 172.21.15.7:0/3444780242 conn(0x7f63a80fcfc0 msgr2=0x7f63a80ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.741+0000 7f63b031d640 1 -- 172.21.15.7:0/3444780242 shutdown_connections 2024-06-10T01:14:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.741+0000 7f63b031d640 1 -- 172.21.15.7:0/3444780242 wait complete. 2024-06-10T01:14:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.742+0000 7f63b031d640 1 Processor -- start 2024-06-10T01:14:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.742+0000 7f63b031d640 1 -- start start 2024-06-10T01:14:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.742+0000 7f63b031d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63a8101a30 0x7f63a8199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.742+0000 7f63b031d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63a8102c20 0x7f63a819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.742+0000 7f63b031d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63a81035e0 0x7f63a81a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.742+0000 7f63b031d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f63a810d6a0 con 0x7f63a8101a30 2024-06-10T01:14:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.742+0000 7f63b031d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f63a810d520 con 0x7f63a8102c20 2024-06-10T01:14:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.742+0000 7f63b031d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f63a810d820 con 0x7f63a81035e0 2024-06-10T01:14:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.742+0000 7f63ae0b9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63a8101a30 0x7f63a8199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.742+0000 7f63ae0b9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63a8101a30 0x7f63a8199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58214/0 (socket says 172.21.15.7:58214) 2024-06-10T01:14:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.742+0000 7f63ae0b9640 1 -- 172.21.15.7:0/2776681504 learned_addr learned my addr 172.21.15.7:0/2776681504 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.742+0000 7f63ad8b8640 1 --2- 172.21.15.7:0/2776681504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63a8102c20 0x7f63a819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.742+0000 7f63ae8ba640 1 --2- 172.21.15.7:0/2776681504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63a81035e0 0x7f63a81a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.742+0000 7f63ad8b8640 1 -- 172.21.15.7:0/2776681504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63a81035e0 msgr2=0x7f63a81a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.742+0000 7f63ad8b8640 1 --2- 172.21.15.7:0/2776681504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63a81035e0 0x7f63a81a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.742+0000 7f63ad8b8640 1 -- 172.21.15.7:0/2776681504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63a8101a30 msgr2=0x7f63a8199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.742+0000 7f63ad8b8640 1 --2- 172.21.15.7:0/2776681504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63a8101a30 0x7f63a8199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.742+0000 7f63ad8b8640 1 -- 172.21.15.7:0/2776681504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a81a1b80 con 0x7f63a8102c20 2024-06-10T01:14:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.742+0000 7f63ae0b9640 1 --2- 172.21.15.7:0/2776681504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63a8101a30 0x7f63a8199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.742+0000 7f63ae8ba640 1 --2- 172.21.15.7:0/2776681504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63a81035e0 0x7f63a81a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.743+0000 7f63ad8b8640 1 --2- 172.21.15.7:0/2776681504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63a8102c20 0x7f63a819a4f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f639800e9f0 tx=0x7f639800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:52.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.743+0000 7f639f7fe640 1 -- 172.21.15.7:0/2776681504 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639800ccc0 con 0x7f63a8102c20 2024-06-10T01:14:52.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.743+0000 7f639f7fe640 1 -- 172.21.15.7:0/2776681504 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6398005020 con 0x7f63a8102c20 2024-06-10T01:14:52.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.743+0000 7f639f7fe640 1 -- 172.21.15.7:0/2776681504 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63980049e0 con 0x7f63a8102c20 2024-06-10T01:14:52.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.743+0000 7f63b031d640 1 -- 172.21.15.7:0/2776681504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a81a1e70 con 0x7f63a8102c20 2024-06-10T01:14:52.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.743+0000 7f63b031d640 1 -- 172.21.15.7:0/2776681504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a81a2330 con 0x7f63a8102c20 2024-06-10T01:14:52.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.746+0000 7f639f7fe640 1 -- 172.21.15.7:0/2776681504 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f6398005560 con 0x7f63a8102c20 2024-06-10T01:14:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.746+0000 7f639f7fe640 1 --2- 172.21.15.7:0/2776681504 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f63940a2c00 0x7f63940a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.746+0000 7f63b3021740 1 Processor -- start 2024-06-10T01:14:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.747+0000 7f639f7fe640 1 -- 172.21.15.7:0/2776681504 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f63980cf2d0 con 0x7f63a8102c20 2024-06-10T01:14:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.747+0000 7f63ae0b9640 1 --2- 172.21.15.7:0/2776681504 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f63940a2c00 0x7f63940a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.747+0000 7f63b3021740 1 -- start start 2024-06-10T01:14:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.747+0000 7f63b3021740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de72204010 0x55de72204430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.747+0000 7f63b3021740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de72204a00 0x55de72209500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.747+0000 7f63b3021740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de72209b70 0x55de7220c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.747+0000 7f63b3021740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55de721674f0 con 0x55de72204a00 2024-06-10T01:14:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.747+0000 7f63b3021740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55de72105870 con 0x55de72209b70 2024-06-10T01:14:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.747+0000 7f63b3021740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55de721634b0 con 0x55de72204010 2024-06-10T01:14:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.747+0000 7f63ad8b8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de72204a00 0x55de72209500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.747+0000 7f63ad8b8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de72204a00 0x55de72209500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58218/0 (socket says 172.21.15.7:58218) 2024-06-10T01:14:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.747+0000 7f63ad8b8640 1 -- 172.21.15.7:0/3675859677 learned_addr learned my addr 172.21.15.7:0/3675859677 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.747+0000 7f63ae0b9640 1 --2- 172.21.15.7:0/3675859677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de72204010 0x55de72204430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.747+0000 7f63ae0b9640 1 --2- 172.21.15.7:0/2776681504 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f63940a2c00 0x7f63940a50c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f63a4000c00 tx=0x7f63a40316b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.747+0000 7f63ae0b9640 1 -- 172.21.15.7:0/3675859677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de72209b70 msgr2=0x55de7220c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.747+0000 7f63ae8ba640 1 --2- 172.21.15.7:0/3675859677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de72209b70 0x55de7220c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.748+0000 7f63ae0b9640 1 --2- 172.21.15.7:0/3675859677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de72209b70 0x55de7220c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.748+0000 7f63ae0b9640 1 -- 172.21.15.7:0/3675859677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de72204a00 msgr2=0x55de72209500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.748+0000 7f63ae0b9640 1 --2- 172.21.15.7:0/3675859677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de72204a00 0x55de72209500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.748+0000 7f63ae0b9640 1 -- 172.21.15.7:0/3675859677 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de721219b0 con 0x55de72204010 2024-06-10T01:14:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.748+0000 7f63ad8b8640 1 --2- 172.21.15.7:0/3675859677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de72204a00 0x55de72209500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.748+0000 7f63ae8ba640 1 --2- 172.21.15.7:0/3675859677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de72209b70 0x55de7220c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:14:52.744 INFO:tasks.workunit.client.0.smithi007.stdout: 1 660535 7.93 MB/sec execute 333 sec latency 57.270 ms 2024-06-10T01:14:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.748+0000 7f63ae0b9640 1 --2- 172.21.15.7:0/3675859677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de72204010 0x55de72204430 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f63a4005ab0 tx=0x7f63a4045000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48a9cd55e529adfe server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.748+0000 7f639cbfe640 1 -- 172.21.15.7:0/3675859677 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4043070 con 0x55de72204010 2024-06-10T01:14:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.748+0000 7f639cbfe640 1 -- 172.21.15.7:0/3675859677 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63a40044c0 con 0x55de72204010 2024-06-10T01:14:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.748+0000 7f639cbfe640 1 -- 172.21.15.7:0/3675859677 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4035040 con 0x55de72204010 2024-06-10T01:14:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.749+0000 7f63b3021740 1 -- 172.21.15.7:0/3675859677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de72204010 msgr2=0x55de72204430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.749+0000 7f63b3021740 1 --2- 172.21.15.7:0/3675859677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de72204010 0x55de72204430 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f63a4005ab0 tx=0x7f63a4045000 comp rx=0 tx=0).stop 2024-06-10T01:14:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.749+0000 7f63b3021740 1 -- 172.21.15.7:0/3675859677 shutdown_connections 2024-06-10T01:14:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.749+0000 7f63b3021740 1 --2- 172.21.15.7:0/3675859677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de72209b70 0x55de7220c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.749+0000 7f63b3021740 1 --2- 172.21.15.7:0/3675859677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de72204a00 0x55de72209500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.749+0000 7f63b3021740 1 --2- 172.21.15.7:0/3675859677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de72204010 0x55de72204430 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.749+0000 7f63b3021740 1 -- 172.21.15.7:0/3675859677 >> 172.21.15.7:0/3675859677 conn(0x55de721ff680 msgr2=0x55de721ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.749+0000 7f63b3021740 1 -- 172.21.15.7:0/3675859677 shutdown_connections 2024-06-10T01:14:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.749+0000 7f63b3021740 1 -- 172.21.15.7:0/3675859677 wait complete. 2024-06-10T01:14:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.750+0000 7f63b3021740 1 -- start start 2024-06-10T01:14:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.750+0000 7f63b3021740 1 Processor -- start 2024-06-10T01:14:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.750+0000 7f63b3021740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de72204010 0x55de7220e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.750+0000 7f63b3021740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de72204a00 0x55de7220e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.750+0000 7f63b3021740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de72209b70 0x55de72215710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.750+0000 7f63b3021740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55de7220cac0 con 0x55de72204010 2024-06-10T01:14:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.750+0000 7f63b3021740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55de720f0ae0 con 0x55de72209b70 2024-06-10T01:14:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.750+0000 7f63b3021740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55de7220cc40 con 0x55de72204a00 2024-06-10T01:14:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.750+0000 7f63b3021740 10 client.0 _ll_register_callbacks cb 0x55de721299b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:14:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.750+0000 7f63ae0b9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de72204010 0x55de7220e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.750+0000 7f63ae0b9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de72204010 0x55de7220e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58220/0 (socket says 172.21.15.7:58220) 2024-06-10T01:14:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.750+0000 7f63ae0b9640 1 -- 172.21.15.7:0/583379455 learned_addr learned my addr 172.21.15.7:0/583379455 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.750+0000 7f63ae8ba640 1 --2- 172.21.15.7:0/583379455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de72209b70 0x55de72215710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.750+0000 7f63ad8b8640 1 --2- 172.21.15.7:0/583379455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de72204a00 0x55de7220e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.750+0000 7f63ae0b9640 1 -- 172.21.15.7:0/583379455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de72204a00 msgr2=0x55de7220e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.750+0000 7f63ae0b9640 1 --2- 172.21.15.7:0/583379455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de72204a00 0x55de7220e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.750+0000 7f63ae0b9640 1 -- 172.21.15.7:0/583379455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de72209b70 msgr2=0x55de72215710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.750+0000 7f63ae0b9640 1 --2- 172.21.15.7:0/583379455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de72209b70 0x55de72215710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.750+0000 7f63ae0b9640 1 -- 172.21.15.7:0/583379455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de71e7b210 con 0x55de72204010 2024-06-10T01:14:52.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.751+0000 7f63ae0b9640 1 --2- 172.21.15.7:0/583379455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de72204010 0x55de7220e190 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009850 tx=0x7f63a4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7e65ceb4a4c7ee server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:52.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.751+0000 7f6391ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:14:52.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.751+0000 7f6391ffb640 1 -- 172.21.15.7:0/583379455 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a405c060 con 0x55de72204010 2024-06-10T01:14:52.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.751+0000 7f6391ffb640 1 -- 172.21.15.7:0/583379455 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f63a403cc40 con 0x55de72204010 2024-06-10T01:14:52.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.751+0000 7f63b3021740 1 -- 172.21.15.7:0/583379455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55de7217af10 con 0x55de72204010 2024-06-10T01:14:52.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.751+0000 7f6391ffb640 1 -- 172.21.15.7:0/583379455 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f63a4043040 con 0x55de72204010 2024-06-10T01:14:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.751+0000 7f63b3021740 10 client.17712 fetch_fsmap learned FSMap version 36 2024-06-10T01:14:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.751+0000 7f63b3021740 1 -- 172.21.15.7:0/583379455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55de71f47620 con 0x55de72204010 2024-06-10T01:14:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.751+0000 7f6391ffb640 1 -- 172.21.15.7:0/583379455 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55de7217af10 con 0x55de72204010 2024-06-10T01:14:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.752+0000 7f6391ffb640 1 -- 172.21.15.7:0/583379455 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f63a4047690 con 0x55de72204010 2024-06-10T01:14:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.752+0000 7f63b3021740 10 client.17712 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T01:14:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.752+0000 7f63b3021740 10 client.17712 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:14:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.752+0000 7f63b3021740 20 client.17712 populate_metadata read hostname 'smithi007' 2024-06-10T01:14:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.752+0000 7f63b3021740 1 --2- 172.21.15.7:0/583379455 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55de72219b70 0x55de7221bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.752+0000 7f63b3021740 4 client.17712 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:14:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.752+0000 7f63b3021740 1 -- 172.21.15.7:0/583379455 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de72174c20 con 0x55de72219b70 2024-06-10T01:14:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.752+0000 7f63ad8b8640 1 --2- 172.21.15.7:0/583379455 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55de72219b70 0x55de7221bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.752+0000 7f63ad8b8640 1 --2- 172.21.15.7:0/583379455 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55de72219b70 0x55de7221bf60 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b2b06b19eddc572 server_cookie=700902c4ff0798d in_seq=0 out_seq=0 2024-06-10T01:14:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.752+0000 7f6391ffb640 10 client.17712 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.753+0000 7f6391ffb640 1 -- 172.21.15.7:0/583379455 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55de72174c20 con 0x55de72219b70 2024-06-10T01:14:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.753+0000 7f6391ffb640 10 client.17712 handle_command_reply: tid=0 2024-06-10T01:14:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.753+0000 7f63b3021740 1 client.17712 shutdown 2024-06-10T01:14:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.753+0000 7f63b3021740 1 -- 172.21.15.7:0/583379455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de72204010 msgr2=0x55de7220e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.753+0000 7f63b3021740 1 --2- 172.21.15.7:0/583379455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de72204010 0x55de7220e190 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009850 tx=0x7f63a4005d50 comp rx=0 tx=0).stop 2024-06-10T01:14:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.753+0000 7f63b3021740 1 -- 172.21.15.7:0/583379455 shutdown_connections 2024-06-10T01:14:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.753+0000 7f63b3021740 1 --2- 172.21.15.7:0/583379455 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55de72219b70 0x55de7221bf60 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.753+0000 7f6391ffb640 0 client.17712 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.753+0000 7f63b3021740 1 --2- 172.21.15.7:0/583379455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de72209b70 0x55de72215710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.753+0000 7f63b3021740 1 --2- 172.21.15.7:0/583379455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de72204a00 0x55de7220e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.753+0000 7f63b3021740 1 --2- 172.21.15.7:0/583379455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de72204010 0x55de7220e190 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.753+0000 7f63b3021740 1 -- 172.21.15.7:0/583379455 >> 172.21.15.7:0/583379455 conn(0x55de721ff680 msgr2=0x55de721fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.753+0000 7f63b3021740 1 -- 172.21.15.7:0/583379455 shutdown_connections 2024-06-10T01:14:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.753+0000 7f63b3021740 1 -- 172.21.15.7:0/583379455 wait complete. 2024-06-10T01:14:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.753+0000 7f63b3021740 20 client.17712 trim_cache size 0 max 16384 2024-06-10T01:14:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.770+0000 7f63b3021740 1 Processor -- start 2024-06-10T01:14:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.771+0000 7f63b3021740 1 -- start start 2024-06-10T01:14:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.771+0000 7f63b3021740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de72204010 0x55de72202590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.771+0000 7f63b3021740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de72204a00 0x55de722156d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.771+0000 7f63b3021740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de72209b70 0x55de72202ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.771+0000 7f63b3021740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f63a81af1f0 con 0x55de72209b70 2024-06-10T01:14:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.771+0000 7f63b3021740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f63a81aca30 con 0x55de72204a00 2024-06-10T01:14:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.771+0000 7f63b3021740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f63a81aa880 con 0x55de72204010 2024-06-10T01:14:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.771+0000 7f63ae8ba640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de72209b70 0x55de72202ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.771+0000 7f63ad8b8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de72204a00 0x55de722156d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.771+0000 7f63ae8ba640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de72209b70 0x55de72202ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58234/0 (socket says 172.21.15.7:58234) 2024-06-10T01:14:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.771+0000 7f63ae8ba640 1 -- 172.21.15.7:0/3271918882 learned_addr learned my addr 172.21.15.7:0/3271918882 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.771+0000 7f63ae0b9640 1 --2- 172.21.15.7:0/3271918882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de72204010 0x55de72202590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.771+0000 7f63ae8ba640 1 -- 172.21.15.7:0/3271918882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de72204010 msgr2=0x55de72202590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.771+0000 7f63ae8ba640 1 --2- 172.21.15.7:0/3271918882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de72204010 0x55de72202590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.771+0000 7f63ae8ba640 1 -- 172.21.15.7:0/3271918882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de72204a00 msgr2=0x55de722156d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.771+0000 7f63ae8ba640 1 --2- 172.21.15.7:0/3271918882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de72204a00 0x55de722156d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.771+0000 7f63ae8ba640 1 -- 172.21.15.7:0/3271918882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de7221c5c0 con 0x55de72209b70 2024-06-10T01:14:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.771+0000 7f63ae0b9640 1 --2- 172.21.15.7:0/3271918882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de72204010 0x55de72202590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.771+0000 7f63ad8b8640 1 --2- 172.21.15.7:0/3271918882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de72204a00 0x55de722156d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.771+0000 7f63ae8ba640 1 --2- 172.21.15.7:0/3271918882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de72209b70 0x55de72202ad0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f63a0014920 tx=0x7f63a0014de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=face25d3fcf63eae server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.772+0000 7f6393fff640 1 -- 172.21.15.7:0/3271918882 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a00141d0 con 0x55de72209b70 2024-06-10T01:14:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.772+0000 7f6393fff640 1 -- 172.21.15.7:0/3271918882 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63a0003970 con 0x55de72209b70 2024-06-10T01:14:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.772+0000 7f6393fff640 1 -- 172.21.15.7:0/3271918882 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0004530 con 0x55de72209b70 2024-06-10T01:14:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.772+0000 7f63b3021740 1 -- 172.21.15.7:0/3271918882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de72209b70 msgr2=0x55de72202ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.772+0000 7f63b3021740 1 --2- 172.21.15.7:0/3271918882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de72209b70 0x55de72202ad0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f63a0014920 tx=0x7f63a0014de0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.772+0000 7f63b3021740 1 -- 172.21.15.7:0/3271918882 shutdown_connections 2024-06-10T01:14:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.772+0000 7f63b3021740 1 --2- 172.21.15.7:0/3271918882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de72209b70 0x55de72202ad0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.772+0000 7f63b3021740 1 --2- 172.21.15.7:0/3271918882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de72204a00 0x55de722156d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.772+0000 7f63b3021740 1 --2- 172.21.15.7:0/3271918882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de72204010 0x55de72202590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.772+0000 7f63b3021740 1 -- 172.21.15.7:0/3271918882 >> 172.21.15.7:0/3271918882 conn(0x55de721ff680 msgr2=0x55de7220dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.772+0000 7f63b3021740 1 -- 172.21.15.7:0/3271918882 shutdown_connections 2024-06-10T01:14:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.772+0000 7f63b3021740 1 -- 172.21.15.7:0/3271918882 wait complete. 2024-06-10T01:14:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.773+0000 7f63b3021740 1 -- start start 2024-06-10T01:14:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.773+0000 7f63b3021740 1 Processor -- start 2024-06-10T01:14:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.773+0000 7f63b3021740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de72204010 0x55de722129d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.773+0000 7f63b3021740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de72204a00 0x55de72212f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.773+0000 7f63b3021740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de72209b70 0x55de72213f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.773+0000 7f63ae0b9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de72204010 0x55de722129d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.773+0000 7f63b3021740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f63a81aaef0 con 0x55de72204010 2024-06-10T01:14:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.773+0000 7f63ae0b9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de72204010 0x55de722129d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58236/0 (socket says 172.21.15.7:58236) 2024-06-10T01:14:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.773+0000 7f63ae0b9640 1 -- 172.21.15.7:0/3164466795 learned_addr learned my addr 172.21.15.7:0/3164466795 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.773+0000 7f63ad8b8640 1 --2- 172.21.15.7:0/3164466795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de72204a00 0x55de72212f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.773+0000 7f63b3021740 1 -- 172.21.15.7:0/3164466795 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f63a81ac360 con 0x55de72209b70 2024-06-10T01:14:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.773+0000 7f63ae8ba640 1 --2- 172.21.15.7:0/3164466795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de72209b70 0x55de72213f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.774+0000 7f63b3021740 1 -- 172.21.15.7:0/3164466795 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f63a81ab5c0 con 0x55de72204a00 2024-06-10T01:14:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.774+0000 7f63b3021740 10 client.0 _ll_register_callbacks cb 0x7f63a4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:14:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.774+0000 7f63ae0b9640 1 -- 172.21.15.7:0/3164466795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de72204a00 msgr2=0x55de72212f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.774+0000 7f63ae0b9640 1 --2- 172.21.15.7:0/3164466795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de72204a00 0x55de72212f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.774+0000 7f63ae0b9640 1 -- 172.21.15.7:0/3164466795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de72209b70 msgr2=0x55de72213f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.774+0000 7f63ae0b9640 1 --2- 172.21.15.7:0/3164466795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de72209b70 0x55de72213f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.774+0000 7f63ae0b9640 1 -- 172.21.15.7:0/3164466795 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de72214510 con 0x55de72204010 2024-06-10T01:14:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.774+0000 7f63ae8ba640 1 --2- 172.21.15.7:0/3164466795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de72209b70 0x55de72213f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.774+0000 7f63b3021740 1 -- 172.21.15.7:0/3164466795 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63a0003970 con 0x55de72204010 2024-06-10T01:14:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.774+0000 7f63ae0b9640 1 --2- 172.21.15.7:0/3164466795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de72204010 0x55de722129d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f63a4005ab0 tx=0x7f63a403c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f591f0162a0eb809 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.774+0000 7f6391ffb640 10 client.17718 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:14:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.774+0000 7f6391ffb640 1 -- 172.21.15.7:0/3164466795 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a406c060 con 0x55de72204010 2024-06-10T01:14:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.774+0000 7f6391ffb640 1 -- 172.21.15.7:0/3164466795 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f63a40042c0 con 0x55de72204010 2024-06-10T01:14:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.774+0000 7f6391ffb640 1 -- 172.21.15.7:0/3164466795 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f63a4063030 con 0x55de72204010 2024-06-10T01:14:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.774+0000 7f63b3021740 10 client.17718 fetch_fsmap learned FSMap version 36 2024-06-10T01:14:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.775+0000 7f63b3021740 1 -- 172.21.15.7:0/3164466795 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=36}) -- 0x55de72215790 con 0x55de72204010 2024-06-10T01:14:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.775+0000 7f6391ffb640 1 -- 172.21.15.7:0/3164466795 <== mon.0 v2:172.21.15.7:3300/0 4 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f63a4047630 con 0x55de72204010 2024-06-10T01:14:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.775+0000 7f63b3021740 10 client.17718 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T01:14:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.775+0000 7f63b3021740 10 client.17718 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:14:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.775+0000 7f63b3021740 20 client.17718 populate_metadata read hostname 'smithi007' 2024-06-10T01:14:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.775+0000 7f63b3021740 1 --2- 172.21.15.7:0/3164466795 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55de72284b50 0x55de72286f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.775+0000 7f63b3021740 4 client.17718 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:14:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.775+0000 7f63b3021740 1 -- 172.21.15.7:0/3164466795 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55de7216ef70 con 0x55de72284b50 2024-06-10T01:14:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.775+0000 7f63ad8b8640 1 --2- 172.21.15.7:0/3164466795 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55de72284b50 0x55de72286f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.775+0000 7f6391ffb640 10 client.17718 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.775+0000 7f63ad8b8640 1 --2- 172.21.15.7:0/3164466795 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55de72284b50 0x55de72286f70 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=897efe0bbab82dc server_cookie=1572141c13447bbe in_seq=0 out_seq=0 2024-06-10T01:14:52.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:52 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:14:52.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:52 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:14:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.871+0000 7f6391ffb640 1 -- 172.21.15.7:0/3164466795 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55de7216ef70 con 0x55de72284b50 2024-06-10T01:14:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.871+0000 7f6391ffb640 10 client.17718 handle_command_reply: tid=0 2024-06-10T01:14:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.871+0000 7f63b3021740 1 client.17718 shutdown 2024-06-10T01:14:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.872+0000 7f63b3021740 1 -- 172.21.15.7:0/3164466795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de72204010 msgr2=0x55de722129d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.872+0000 7f63b3021740 1 --2- 172.21.15.7:0/3164466795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de72204010 0x55de722129d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f63a4005ab0 tx=0x7f63a403c430 comp rx=0 tx=0).stop 2024-06-10T01:14:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.872+0000 7f63b3021740 1 -- 172.21.15.7:0/3164466795 shutdown_connections 2024-06-10T01:14:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.872+0000 7f63b3021740 1 --2- 172.21.15.7:0/3164466795 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55de72284b50 0x55de72286f70 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.872+0000 7f63b3021740 1 --2- 172.21.15.7:0/3164466795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de72209b70 0x55de72213f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.872+0000 7f63b3021740 1 --2- 172.21.15.7:0/3164466795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de72204a00 0x55de72212f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.872+0000 7f63b3021740 1 --2- 172.21.15.7:0/3164466795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de72204010 0x55de722129d0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.872+0000 7f6391ffb640 0 client.17718 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.872+0000 7f63b3021740 1 -- 172.21.15.7:0/3164466795 >> 172.21.15.7:0/3164466795 conn(0x55de721ff680 msgr2=0x55de72202590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:14:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:14:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:14:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.872+0000 7f63b3021740 1 -- 172.21.15.7:0/3164466795 shutdown_connections 2024-06-10T01:14:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.872+0000 7f63b3021740 1 -- 172.21.15.7:0/3164466795 wait complete. 2024-06-10T01:14:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.872+0000 7f63b3021740 20 client.17718 trim_cache size 0 max 16384 2024-06-10T01:14:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.872+0000 7f63b031d640 1 -- 172.21.15.7:0/2776681504 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f63940a2c00 msgr2=0x7f63940a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.872+0000 7f63b031d640 1 --2- 172.21.15.7:0/2776681504 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f63940a2c00 0x7f63940a50c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f63a4000c00 tx=0x7f63a40316b0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.873+0000 7f63b031d640 1 -- 172.21.15.7:0/2776681504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63a8102c20 msgr2=0x7f63a819a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.873+0000 7f63b031d640 1 --2- 172.21.15.7:0/2776681504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63a8102c20 0x7f63a819a4f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f639800e9f0 tx=0x7f639800eeb0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.873+0000 7f63b031d640 1 -- 172.21.15.7:0/2776681504 shutdown_connections 2024-06-10T01:14:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.873+0000 7f63b031d640 1 --2- 172.21.15.7:0/2776681504 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f63940a2c00 0x7f63940a50c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.873+0000 7f63b031d640 1 --2- 172.21.15.7:0/2776681504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63a81035e0 0x7f63a81a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.873+0000 7f63b031d640 1 --2- 172.21.15.7:0/2776681504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63a8102c20 0x7f63a819a4f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.873+0000 7f63b031d640 1 --2- 172.21.15.7:0/2776681504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63a8101a30 0x7f63a8199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.873+0000 7f63b031d640 1 -- 172.21.15.7:0/2776681504 >> 172.21.15.7:0/2776681504 conn(0x7f63a80fcfc0 msgr2=0x7f63a81083f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.873+0000 7f63b031d640 1 -- 172.21.15.7:0/2776681504 shutdown_connections 2024-06-10T01:14:52.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:14:52.873+0000 7f63b031d640 1 -- 172.21.15.7:0/2776681504 wait complete. 2024-06-10T01:14:53.247 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db / --timeout 90 --expiration 137.8 --await-for 45.0' 2024-06-10T01:14:53.248 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db / --timeout 90 --expiration 137.8 --await-for 45.0 2024-06-10T01:14:53.512 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:53 smithi007 ceph-mon[32398]: pgmap v698: 129 pgs: 129 active+clean; 116 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 1.2 MiB/s wr, 242 op/s 2024-06-10T01:14:53.512 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:53 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:14:53.512 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:53 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:14:53.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:53 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/664538379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:14:53.533 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:14:53.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:53 smithi175 ceph-mon[36636]: pgmap v698: 129 pgs: 129 active+clean; 116 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 1.2 MiB/s wr, 242 op/s 2024-06-10T01:14:53.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:53 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:14:53.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:53 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:14:53.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:53 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/664538379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:14:53.729 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:53 smithi078 ceph-mon[36683]: pgmap v698: 129 pgs: 129 active+clean; 116 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 1.2 MiB/s wr, 242 op/s 2024-06-10T01:14:53.729 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:53 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:14:53.730 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:53 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:14:53.730 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:53 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/664538379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:14:53.742 INFO:tasks.workunit.client.0.smithi007.stdout: 1 661602 7.91 MB/sec execute 334 sec latency 40.928 ms 2024-06-10T01:14:54.742 INFO:tasks.workunit.client.0.smithi007.stdout: 1 664153 7.94 MB/sec execute 335 sec latency 11.040 ms 2024-06-10T01:14:55.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.151+0000 7f9343d0f700 1 -- 172.21.15.7:0/3878410632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f933c107850 msgr2=0x7f933c10f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:55.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.151+0000 7f9343d0f700 1 --2- 172.21.15.7:0/3878410632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f933c107850 0x7f933c10f3d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f933400b0d0 tx=0x7f933400b3e0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.151+0000 7f9343d0f700 1 -- 172.21.15.7:0/3878410632 shutdown_connections 2024-06-10T01:14:55.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.151+0000 7f9343d0f700 1 --2- 172.21.15.7:0/3878410632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f933c107850 0x7f933c10f3d0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.151+0000 7f9343d0f700 1 --2- 172.21.15.7:0/3878410632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f933c06a3d0 0x7f933c06a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.151+0000 7f9343d0f700 1 --2- 172.21.15.7:0/3878410632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f933c06ad80 0x7f933c107120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.151+0000 7f9343d0f700 1 -- 172.21.15.7:0/3878410632 >> 172.21.15.7:0/3878410632 conn(0x7f933c078330 msgr2=0x7f933c078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:55.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.151+0000 7f9343d0f700 1 -- 172.21.15.7:0/3878410632 shutdown_connections 2024-06-10T01:14:55.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.151+0000 7f9343d0f700 1 -- 172.21.15.7:0/3878410632 wait complete. 2024-06-10T01:14:55.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.151+0000 7f9343d0f700 1 Processor -- start 2024-06-10T01:14:55.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.152+0000 7f9343d0f700 1 -- start start 2024-06-10T01:14:55.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.152+0000 7f9343d0f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f933c06a3d0 0x7f933c19e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:55.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.152+0000 7f9343d0f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f933c06ad80 0x7f933c19ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:55.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.152+0000 7f9343d0f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f933c107850 0x7f933c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:55.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.152+0000 7f9343d0f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f933c112790 con 0x7f933c06ad80 2024-06-10T01:14:55.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.152+0000 7f9343d0f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f933c112610 con 0x7f933c06a3d0 2024-06-10T01:14:55.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.152+0000 7f9343d0f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f933c112910 con 0x7f933c107850 2024-06-10T01:14:55.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.152+0000 7f9341aab700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f933c06a3d0 0x7f933c19e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:55.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.152+0000 7f9341aab700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f933c06a3d0 0x7f933c19e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:49574/0 (socket says 172.21.15.7:49574) 2024-06-10T01:14:55.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.152+0000 7f9341aab700 1 -- 172.21.15.7:0/4096894557 learned_addr learned my addr 172.21.15.7:0/4096894557 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:55.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.152+0000 7f93422ac700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f933c107850 0x7f933c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:55.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.152+0000 7f93412aa700 1 --2- 172.21.15.7:0/4096894557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f933c06ad80 0x7f933c19ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:55.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.152+0000 7f93422ac700 1 -- 172.21.15.7:0/4096894557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f933c06a3d0 msgr2=0x7f933c19e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:55.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.152+0000 7f93422ac700 1 --2- 172.21.15.7:0/4096894557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f933c06a3d0 0x7f933c19e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.152+0000 7f93422ac700 1 -- 172.21.15.7:0/4096894557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f933c06ad80 msgr2=0x7f933c19ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:55.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.152+0000 7f93422ac700 1 --2- 172.21.15.7:0/4096894557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f933c06ad80 0x7f933c19ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.152+0000 7f93422ac700 1 -- 172.21.15.7:0/4096894557 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f933c199050 con 0x7f933c107850 2024-06-10T01:14:55.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.152+0000 7f93412aa700 1 --2- 172.21.15.7:0/4096894557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f933c06ad80 0x7f933c19ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:55.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.152+0000 7f9341aab700 1 --2- 172.21.15.7:0/4096894557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f933c06a3d0 0x7f933c19e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:55.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.153+0000 7f93422ac700 1 --2- 172.21.15.7:0/4096894557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f933c107850 0x7f933c1988f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f933400b770 tx=0x7f9334007a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:55.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.153+0000 7f9332ffd700 1 -- 172.21.15.7:0/4096894557 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9334021020 con 0x7f933c107850 2024-06-10T01:14:55.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.153+0000 7f9343d0f700 1 -- 172.21.15.7:0/4096894557 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f933c1992e0 con 0x7f933c107850 2024-06-10T01:14:55.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.153+0000 7f9332ffd700 1 -- 172.21.15.7:0/4096894557 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9334004c50 con 0x7f933c107850 2024-06-10T01:14:55.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.153+0000 7f9332ffd700 1 -- 172.21.15.7:0/4096894557 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93340052b0 con 0x7f933c107850 2024-06-10T01:14:55.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.153+0000 7f9343d0f700 1 -- 172.21.15.7:0/4096894557 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f933c1a55b0 con 0x7f933c107850 2024-06-10T01:14:55.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.155+0000 7f9332ffd700 1 -- 172.21.15.7:0/4096894557 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9334024020 con 0x7f933c107850 2024-06-10T01:14:55.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.156+0000 7f9332ffd700 1 --2- 172.21.15.7:0/4096894557 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f93280a2cc0 0x7f93280a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:55.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.157+0000 7f9341aab700 1 --2- 172.21.15.7:0/4096894557 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f93280a2cc0 0x7f93280a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:55.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.157+0000 7f9332ffd700 1 -- 172.21.15.7:0/4096894557 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f933400f020 con 0x7f933c107850 2024-06-10T01:14:55.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.157+0000 7f934e592b80 1 Processor -- start 2024-06-10T01:14:55.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.157+0000 7f9341aab700 1 --2- 172.21.15.7:0/4096894557 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f93280a2cc0 0x7f93280a5180 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f933c1072d0 tx=0x7f9338009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:55.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.157+0000 7f934e592b80 1 -- start start 2024-06-10T01:14:55.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.157+0000 7f934e592b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597db2199a0 0x5597db219d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:55.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.157+0000 7f934e592b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597db21a2c0 0x5597db213bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:55.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.157+0000 7f934e592b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597db2140f0 0x5597db214510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:55.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.157+0000 7f934e592b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5597db1020f0 con 0x5597db2199a0 2024-06-10T01:14:55.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.157+0000 7f934e592b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5597db1609e0 con 0x5597db21a2c0 2024-06-10T01:14:55.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.157+0000 7f934e592b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5597db0c85f0 con 0x5597db2140f0 2024-06-10T01:14:55.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.158+0000 7f9341aab700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597db2199a0 0x5597db219d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:55.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.158+0000 7f93422ac700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597db2140f0 0x5597db214510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:55.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.158+0000 7f9341aab700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597db2199a0 0x5597db219d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58268/0 (socket says 172.21.15.7:58268) 2024-06-10T01:14:55.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.158+0000 7f9341aab700 1 -- 172.21.15.7:0/2378238302 learned_addr learned my addr 172.21.15.7:0/2378238302 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:55.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.158+0000 7f93412aa700 1 --2- 172.21.15.7:0/2378238302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597db21a2c0 0x5597db213bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:55.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.158+0000 7f93422ac700 1 -- 172.21.15.7:0/2378238302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597db21a2c0 msgr2=0x5597db213bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:55.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.158+0000 7f93422ac700 1 --2- 172.21.15.7:0/2378238302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597db21a2c0 0x5597db213bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.158+0000 7f93422ac700 1 -- 172.21.15.7:0/2378238302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597db2199a0 msgr2=0x5597db219d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:55.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.158+0000 7f93422ac700 1 --2- 172.21.15.7:0/2378238302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597db2199a0 0x5597db219d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.158+0000 7f93422ac700 1 -- 172.21.15.7:0/2378238302 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597db0afa60 con 0x5597db2140f0 2024-06-10T01:14:55.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.158+0000 7f9341aab700 1 --2- 172.21.15.7:0/2378238302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597db2199a0 0x5597db219d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:55.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.158+0000 7f93412aa700 1 --2- 172.21.15.7:0/2378238302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597db21a2c0 0x5597db213bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:55.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.158+0000 7f93422ac700 1 --2- 172.21.15.7:0/2378238302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597db2140f0 0x5597db214510 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f933409c4e0 tx=0x7f9334009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8a0499083fd938d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:55.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.158+0000 7f9327fff700 1 -- 172.21.15.7:0/2378238302 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93340a0b90 con 0x5597db2140f0 2024-06-10T01:14:55.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.158+0000 7f9327fff700 1 -- 172.21.15.7:0/2378238302 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93340a53c0 con 0x5597db2140f0 2024-06-10T01:14:55.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.158+0000 7f9327fff700 1 -- 172.21.15.7:0/2378238302 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9334009ca0 con 0x5597db2140f0 2024-06-10T01:14:55.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.159+0000 7f934e592b80 1 -- 172.21.15.7:0/2378238302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597db2140f0 msgr2=0x5597db214510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:55.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.159+0000 7f934e592b80 1 --2- 172.21.15.7:0/2378238302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597db2140f0 0x5597db214510 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f933409c4e0 tx=0x7f9334009b80 comp rx=0 tx=0).stop 2024-06-10T01:14:55.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.159+0000 7f934e592b80 1 -- 172.21.15.7:0/2378238302 shutdown_connections 2024-06-10T01:14:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.159+0000 7f934e592b80 1 --2- 172.21.15.7:0/2378238302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597db2140f0 0x5597db214510 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.159+0000 7f934e592b80 1 --2- 172.21.15.7:0/2378238302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597db2199a0 0x5597db219d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.159+0000 7f934e592b80 1 --2- 172.21.15.7:0/2378238302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597db21a2c0 0x5597db213bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.159+0000 7f934e592b80 1 -- 172.21.15.7:0/2378238302 >> 172.21.15.7:0/2378238302 conn(0x5597db13c210 msgr2=0x5597db20aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.159+0000 7f934e592b80 1 -- 172.21.15.7:0/2378238302 shutdown_connections 2024-06-10T01:14:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.159+0000 7f934e592b80 1 -- 172.21.15.7:0/2378238302 wait complete. 2024-06-10T01:14:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.160+0000 7f934e592b80 1 -- start start 2024-06-10T01:14:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.160+0000 7f934e592b80 1 Processor -- start 2024-06-10T01:14:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.160+0000 7f934e592b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597db2140f0 0x5597db220080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.160+0000 7f934e592b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597db2199a0 0x5597db21d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.160+0000 7f934e592b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597db21a2c0 0x5597db21d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.160+0000 7f934e592b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5597db101150 con 0x5597db2199a0 2024-06-10T01:14:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.160+0000 7f934e592b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5597db15c9a0 con 0x5597db2140f0 2024-06-10T01:14:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.160+0000 7f934e592b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5597db0c85f0 con 0x5597db21a2c0 2024-06-10T01:14:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.160+0000 7f934e592b80 10 client.0 _ll_register_callbacks cb 0x5597db0f6740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:14:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.160+0000 7f93412aa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597db2199a0 0x5597db21d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.161+0000 7f93412aa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597db2199a0 0x5597db21d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58284/0 (socket says 172.21.15.7:58284) 2024-06-10T01:14:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.161+0000 7f93422ac700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597db21a2c0 0x5597db21d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.161+0000 7f93412aa700 1 -- 172.21.15.7:0/274062283 learned_addr learned my addr 172.21.15.7:0/274062283 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.161+0000 7f9341aab700 1 --2- 172.21.15.7:0/274062283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597db2140f0 0x5597db220080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.161+0000 7f93422ac700 1 -- 172.21.15.7:0/274062283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597db2140f0 msgr2=0x5597db220080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.161+0000 7f93422ac700 1 --2- 172.21.15.7:0/274062283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597db2140f0 0x5597db220080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.161+0000 7f93422ac700 1 -- 172.21.15.7:0/274062283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597db2199a0 msgr2=0x5597db21d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.161+0000 7f93422ac700 1 --2- 172.21.15.7:0/274062283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597db2199a0 0x5597db21d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.161+0000 7f93422ac700 1 -- 172.21.15.7:0/274062283 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597db1268e0 con 0x5597db21a2c0 2024-06-10T01:14:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.161+0000 7f93412aa700 1 --2- 172.21.15.7:0/274062283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597db2199a0 0x5597db21d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.161+0000 7f93422ac700 1 --2- 172.21.15.7:0/274062283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597db21a2c0 0x5597db21d8c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x5597db220960 tx=0x7f9334098bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb519a4b678f1040 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.161+0000 7f93257fa700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:14:55.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.161+0000 7f93257fa700 1 -- 172.21.15.7:0/274062283 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93340b5020 con 0x5597db21a2c0 2024-06-10T01:14:55.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.161+0000 7f93257fa700 1 -- 172.21.15.7:0/274062283 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f9334009c10 con 0x5597db21a2c0 2024-06-10T01:14:55.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.161+0000 7f934e592b80 1 -- 172.21.15.7:0/274062283 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93340a53c0 con 0x5597db21a2c0 2024-06-10T01:14:55.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.161+0000 7f93257fa700 1 -- 172.21.15.7:0/274062283 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f93340074e0 con 0x5597db21a2c0 2024-06-10T01:14:55.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.161+0000 7f934e592b80 10 client.27154 fetch_fsmap learned FSMap version 36 2024-06-10T01:14:55.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.162+0000 7f934e592b80 1 -- 172.21.15.7:0/274062283 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5597db22b000 con 0x5597db21a2c0 2024-06-10T01:14:55.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.162+0000 7f93257fa700 1 -- 172.21.15.7:0/274062283 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93340a53c0 con 0x5597db21a2c0 2024-06-10T01:14:55.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.162+0000 7f93257fa700 1 -- 172.21.15.7:0/274062283 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f933402e060 con 0x5597db21a2c0 2024-06-10T01:14:55.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.162+0000 7f934e592b80 10 client.27154 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T01:14:55.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.162+0000 7f934e592b80 10 client.27154 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:14:55.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.162+0000 7f934e592b80 20 client.27154 populate_metadata read hostname 'smithi007' 2024-06-10T01:14:55.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.162+0000 7f934e592b80 1 --2- 172.21.15.7:0/274062283 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5597db22d3f0 0x5597db22f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:55.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.162+0000 7f934e592b80 4 client.27154 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:14:55.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.162+0000 7f934e592b80 1 -- 172.21.15.7:0/274062283 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597db16dee0 con 0x5597db22d3f0 2024-06-10T01:14:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.162+0000 7f9341aab700 1 --2- 172.21.15.7:0/274062283 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5597db22d3f0 0x5597db22f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.163+0000 7f9341aab700 1 --2- 172.21.15.7:0/274062283 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5597db22d3f0 0x5597db22f7e0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=689f96a7e83e0a5 server_cookie=b3ca0746da223522 in_seq=0 out_seq=0 2024-06-10T01:14:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.163+0000 7f93257fa700 10 client.27154 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.163+0000 7f93257fa700 1 -- 172.21.15.7:0/274062283 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5597db16dee0 con 0x5597db22d3f0 2024-06-10T01:14:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.163+0000 7f93257fa700 10 client.27154 handle_command_reply: tid=0 2024-06-10T01:14:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.163+0000 7f934e592b80 1 client.27154 shutdown 2024-06-10T01:14:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.163+0000 7f934e592b80 1 -- 172.21.15.7:0/274062283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597db21a2c0 msgr2=0x5597db21d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.163+0000 7f934e592b80 1 --2- 172.21.15.7:0/274062283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597db21a2c0 0x5597db21d8c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x5597db220960 tx=0x7f9334098bb0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.164+0000 7f934e592b80 1 -- 172.21.15.7:0/274062283 shutdown_connections 2024-06-10T01:14:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.164+0000 7f934e592b80 1 --2- 172.21.15.7:0/274062283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597db21a2c0 0x5597db21d8c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.164+0000 7f934e592b80 1 --2- 172.21.15.7:0/274062283 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5597db22d3f0 0x5597db22f7e0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.164+0000 7f934e592b80 1 --2- 172.21.15.7:0/274062283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597db2140f0 0x5597db220080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.164+0000 7f934e592b80 1 --2- 172.21.15.7:0/274062283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597db2199a0 0x5597db21d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.164+0000 7f93257fa700 0 client.27154 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.164+0000 7f934e592b80 1 -- 172.21.15.7:0/274062283 >> 172.21.15.7:0/274062283 conn(0x5597db13c210 msgr2=0x5597db209c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.164+0000 7f934e592b80 1 -- 172.21.15.7:0/274062283 shutdown_connections 2024-06-10T01:14:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.164+0000 7f934e592b80 1 -- 172.21.15.7:0/274062283 wait complete. 2024-06-10T01:14:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.164+0000 7f934e592b80 20 client.27154 trim_cache size 0 max 16384 2024-06-10T01:14:55.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.184+0000 7f934e592b80 1 Processor -- start 2024-06-10T01:14:55.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.184+0000 7f934e592b80 1 -- start start 2024-06-10T01:14:55.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.185+0000 7f934e592b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597db2140f0 0x5597db21c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:55.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.185+0000 7f934e592b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597db2199a0 0x5597db21f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:55.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.185+0000 7f934e592b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597db21a2c0 0x5597db20a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:55.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.185+0000 7f934e592b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1ae7d0 con 0x5597db21a2c0 2024-06-10T01:14:55.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.185+0000 7f934e592b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1abfd0 con 0x5597db2199a0 2024-06-10T01:14:55.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.185+0000 7f934e592b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1a9e20 con 0x5597db2140f0 2024-06-10T01:14:55.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.185+0000 7f93422ac700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597db21a2c0 0x5597db20a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:55.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.185+0000 7f9341aab700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597db2140f0 0x5597db21c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:55.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.185+0000 7f93422ac700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597db21a2c0 0x5597db20a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58290/0 (socket says 172.21.15.7:58290) 2024-06-10T01:14:55.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.185+0000 7f9341aab700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597db2140f0 0x5597db21c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:46320/0 (socket says 172.21.15.7:46320) 2024-06-10T01:14:55.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.185+0000 7f93422ac700 1 -- 172.21.15.7:0/196855874 learned_addr learned my addr 172.21.15.7:0/196855874 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:55.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.185+0000 7f93412aa700 1 --2- 172.21.15.7:0/196855874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597db2199a0 0x5597db21f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:55.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.185+0000 7f93422ac700 1 -- 172.21.15.7:0/196855874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597db2140f0 msgr2=0x5597db21c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:55.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.185+0000 7f93422ac700 1 --2- 172.21.15.7:0/196855874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597db2140f0 0x5597db21c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.185+0000 7f93422ac700 1 -- 172.21.15.7:0/196855874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597db2199a0 msgr2=0x5597db21f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:55.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.185+0000 7f93422ac700 1 --2- 172.21.15.7:0/196855874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597db2199a0 0x5597db21f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.185+0000 7f93422ac700 1 -- 172.21.15.7:0/196855874 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597db202530 con 0x5597db21a2c0 2024-06-10T01:14:55.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.185+0000 7f9341aab700 1 --2- 172.21.15.7:0/196855874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597db2140f0 0x5597db21c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:55.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.185+0000 7f93422ac700 1 --2- 172.21.15.7:0/196855874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597db21a2c0 0x5597db20a3c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f93340a4ff0 tx=0x7f93340a4f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616d86c735a5cc3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:55.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.186+0000 7f93277fe700 1 -- 172.21.15.7:0/196855874 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93340b5070 con 0x5597db21a2c0 2024-06-10T01:14:55.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.186+0000 7f93277fe700 1 -- 172.21.15.7:0/196855874 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93340a53c0 con 0x5597db21a2c0 2024-06-10T01:14:55.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.186+0000 7f93277fe700 1 -- 172.21.15.7:0/196855874 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9334066020 con 0x5597db21a2c0 2024-06-10T01:14:55.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.186+0000 7f934e592b80 1 -- 172.21.15.7:0/196855874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597db21a2c0 msgr2=0x5597db20a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:55.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.186+0000 7f934e592b80 1 --2- 172.21.15.7:0/196855874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597db21a2c0 0x5597db20a3c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f93340a4ff0 tx=0x7f93340a4f30 comp rx=0 tx=0).stop 2024-06-10T01:14:55.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.186+0000 7f934e592b80 1 -- 172.21.15.7:0/196855874 shutdown_connections 2024-06-10T01:14:55.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.186+0000 7f934e592b80 1 --2- 172.21.15.7:0/196855874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597db21a2c0 0x5597db20a3c0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.186+0000 7f934e592b80 1 --2- 172.21.15.7:0/196855874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597db2140f0 0x5597db21c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.186+0000 7f934e592b80 1 --2- 172.21.15.7:0/196855874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597db2199a0 0x5597db21f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.186+0000 7f934e592b80 1 -- 172.21.15.7:0/196855874 >> 172.21.15.7:0/196855874 conn(0x5597db13c210 msgr2=0x5597db21ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:55.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.186+0000 7f934e592b80 1 -- 172.21.15.7:0/196855874 shutdown_connections 2024-06-10T01:14:55.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.186+0000 7f934e592b80 1 -- 172.21.15.7:0/196855874 wait complete. 2024-06-10T01:14:55.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.187+0000 7f934e592b80 1 -- start start 2024-06-10T01:14:55.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.187+0000 7f934e592b80 1 Processor -- start 2024-06-10T01:14:55.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.187+0000 7f934e592b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597db2140f0 0x5597db225d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:55.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.187+0000 7f934e592b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597db2199a0 0x5597db226240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:55.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.187+0000 7f934e592b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597db21a2c0 0x5597db222af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:55.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.187+0000 7f934e592b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1aa490 con 0x5597db2140f0 2024-06-10T01:14:55.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.187+0000 7f934e592b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1ab900 con 0x5597db2199a0 2024-06-10T01:14:55.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.187+0000 7f934e592b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1aab60 con 0x5597db21a2c0 2024-06-10T01:14:55.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.187+0000 7f9341aab700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597db2140f0 0x5597db225d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:55.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.187+0000 7f934e592b80 10 client.0 _ll_register_callbacks cb 0x5597db0f6740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:14:55.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.187+0000 7f9341aab700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597db2140f0 0x5597db225d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58298/0 (socket says 172.21.15.7:58298) 2024-06-10T01:14:55.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.187+0000 7f9341aab700 1 -- 172.21.15.7:0/2790240474 learned_addr learned my addr 172.21.15.7:0/2790240474 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:14:55.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.187+0000 7f93422ac700 1 --2- 172.21.15.7:0/2790240474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597db21a2c0 0x5597db222af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:55.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.188+0000 7f93412aa700 1 --2- 172.21.15.7:0/2790240474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597db2199a0 0x5597db226240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:55.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.188+0000 7f93422ac700 1 -- 172.21.15.7:0/2790240474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597db2199a0 msgr2=0x5597db226240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:55.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.188+0000 7f93422ac700 1 --2- 172.21.15.7:0/2790240474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597db2199a0 0x5597db226240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.188+0000 7f93422ac700 1 -- 172.21.15.7:0/2790240474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597db2140f0 msgr2=0x5597db225d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:14:55.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.188+0000 7f93422ac700 1 --2- 172.21.15.7:0/2790240474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597db2140f0 0x5597db225d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.188+0000 7f93422ac700 1 -- 172.21.15.7:0/2790240474 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597db223130 con 0x5597db21a2c0 2024-06-10T01:14:55.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.188+0000 7f9341aab700 1 --2- 172.21.15.7:0/2790240474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597db2140f0 0x5597db225d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:14:55.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.188+0000 7f93422ac700 1 --2- 172.21.15.7:0/2790240474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597db21a2c0 0x5597db222af0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f933400b0d0 tx=0x7f933409acf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=760939f6d755e7b8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:14:55.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.188+0000 7f93257fa700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:14:55.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.188+0000 7f93257fa700 1 -- 172.21.15.7:0/2790240474 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9334066070 con 0x5597db21a2c0 2024-06-10T01:14:55.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.188+0000 7f934e592b80 1 -- 172.21.15.7:0/2790240474 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93340a53c0 con 0x5597db21a2c0 2024-06-10T01:14:55.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.188+0000 7f93257fa700 1 -- 172.21.15.7:0/2790240474 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f933409db10 con 0x5597db21a2c0 2024-06-10T01:14:55.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.188+0000 7f93257fa700 1 -- 172.21.15.7:0/2790240474 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f93340aa050 con 0x5597db21a2c0 2024-06-10T01:14:55.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.188+0000 7f934e592b80 10 client.27160 fetch_fsmap learned FSMap version 36 2024-06-10T01:14:55.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.188+0000 7f934e592b80 1 -- 172.21.15.7:0/2790240474 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5597db223130 con 0x5597db21a2c0 2024-06-10T01:14:55.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.188+0000 7f93257fa700 1 -- 172.21.15.7:0/2790240474 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93340a53c0 con 0x5597db21a2c0 2024-06-10T01:14:55.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.189+0000 7f93257fa700 1 -- 172.21.15.7:0/2790240474 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f9334011070 con 0x5597db21a2c0 2024-06-10T01:14:55.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.189+0000 7f934e592b80 10 client.27160 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T01:14:55.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.189+0000 7f934e592b80 10 client.27160 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:14:55.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.189+0000 7f934e592b80 20 client.27160 populate_metadata read hostname 'smithi007' 2024-06-10T01:14:55.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.189+0000 7f934e592b80 1 --2- 172.21.15.7:0/2790240474 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5597db298cf0 0x5597db29b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:14:55.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.189+0000 7f934e592b80 4 client.27160 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.8, "await_for": 45.0}] 2024-06-10T01:14:55.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.189+0000 7f934e592b80 1 -- 172.21.15.7:0/2790240474 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.8, "await_for": 45.0}) -- 0x5597db168460 con 0x5597db298cf0 2024-06-10T01:14:55.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.189+0000 7f9341aab700 1 --2- 172.21.15.7:0/2790240474 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5597db298cf0 0x5597db29b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:14:55.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.189+0000 7f93257fa700 10 client.27160 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:55.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.189+0000 7f9341aab700 1 --2- 172.21.15.7:0/2790240474 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5597db298cf0 0x5597db29b110 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1e90be84e788074 server_cookie=c19d15a4d97ec623 in_seq=0 out_seq=0 2024-06-10T01:14:55.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.261+0000 7f93257fa700 1 -- 172.21.15.7:0/2790240474 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5597db168460 con 0x5597db298cf0 2024-06-10T01:14:55.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.261+0000 7f93257fa700 10 client.27160 handle_command_reply: tid=0 2024-06-10T01:14:55.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.261+0000 7f934e592b80 1 client.27160 shutdown 2024-06-10T01:14:55.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.261+0000 7f934e592b80 1 -- 172.21.15.7:0/2790240474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597db21a2c0 msgr2=0x5597db222af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:55.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.261+0000 7f934e592b80 1 --2- 172.21.15.7:0/2790240474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597db21a2c0 0x5597db222af0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f933400b0d0 tx=0x7f933409acf0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.261+0000 7f934e592b80 1 -- 172.21.15.7:0/2790240474 shutdown_connections 2024-06-10T01:14:55.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.261+0000 7f934e592b80 1 --2- 172.21.15.7:0/2790240474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5597db21a2c0 0x5597db222af0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.261+0000 7f934e592b80 1 --2- 172.21.15.7:0/2790240474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597db2140f0 0x5597db225d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.261+0000 7f934e592b80 1 --2- 172.21.15.7:0/2790240474 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5597db298cf0 0x5597db29b110 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.261+0000 7f934e592b80 1 --2- 172.21.15.7:0/2790240474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597db2199a0 0x5597db226240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.261+0000 7f93257fa700 0 client.27160 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:14:55.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.261+0000 7f934e592b80 1 -- 172.21.15.7:0/2790240474 >> 172.21.15.7:0/2790240474 conn(0x5597db13c210 msgr2=0x5597db21f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:55.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.261+0000 7f934e592b80 1 -- 172.21.15.7:0/2790240474 shutdown_connections 2024-06-10T01:14:55.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.261+0000 7f934e592b80 1 -- 172.21.15.7:0/2790240474 wait complete. 2024-06-10T01:14:55.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.261+0000 7f934e592b80 20 client.27160 trim_cache size 0 max 16384 2024-06-10T01:14:55.260 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:14:55.260 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 36, 2024-06-10T01:14:55.260 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:14:55.260 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 26, 2024-06-10T01:14:55.261 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:14:55.261 INFO:teuthology.orchestra.run.smithi007.stdout: "76318bf4": { 2024-06-10T01:14:55.261 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 26, 2024-06-10T01:14:55.261 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:14:55.261 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:14:55.261 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:14:55.261 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:14:55.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:14:55.261 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:14:55.262 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 137.8, 2024-06-10T01:14:55.262 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:14:55.262 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:14:55.262 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:14:55.262 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:14:55.262 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:14:55.262 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:14:55.262 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:14:55.262 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:14:55.263 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:14:55.263 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:14:55.263 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:14:55.263 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:14:55.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.262+0000 7f9343d0f700 1 -- 172.21.15.7:0/4096894557 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f93280a2cc0 msgr2=0x7f93280a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:55.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.262+0000 7f9343d0f700 1 --2- 172.21.15.7:0/4096894557 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f93280a2cc0 0x7f93280a5180 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f933c1072d0 tx=0x7f9338009260 comp rx=0 tx=0).stop 2024-06-10T01:14:55.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.262+0000 7f9343d0f700 1 -- 172.21.15.7:0/4096894557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f933c107850 msgr2=0x7f933c1988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:14:55.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.262+0000 7f9343d0f700 1 --2- 172.21.15.7:0/4096894557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f933c107850 0x7f933c1988f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f933400b770 tx=0x7f9334007a20 comp rx=0 tx=0).stop 2024-06-10T01:14:55.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.262+0000 7f9343d0f700 1 -- 172.21.15.7:0/4096894557 shutdown_connections 2024-06-10T01:14:55.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.262+0000 7f9343d0f700 1 --2- 172.21.15.7:0/4096894557 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f93280a2cc0 0x7f93280a5180 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.262+0000 7f9343d0f700 1 --2- 172.21.15.7:0/4096894557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f933c107850 0x7f933c1988f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.262+0000 7f9343d0f700 1 --2- 172.21.15.7:0/4096894557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f933c06a3d0 0x7f933c19e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.262+0000 7f9343d0f700 1 --2- 172.21.15.7:0/4096894557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f933c06ad80 0x7f933c19ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:14:55.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.262+0000 7f9343d0f700 1 -- 172.21.15.7:0/4096894557 >> 172.21.15.7:0/4096894557 conn(0x7f933c078330 msgr2=0x7f933c0ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:14:55.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.262+0000 7f9343d0f700 1 -- 172.21.15.7:0/4096894557 shutdown_connections 2024-06-10T01:14:55.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:14:55.262+0000 7f9343d0f700 1 -- 172.21.15.7:0/4096894557 wait complete. 2024-06-10T01:14:55.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:55 smithi175 ceph-mon[36636]: pgmap v699: 129 pgs: 129 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.5 MiB/s wr, 294 op/s 2024-06-10T01:14:55.742 INFO:tasks.workunit.client.0.smithi007.stdout: 1 664628 7.92 MB/sec execute 336 sec latency 497.279 ms 2024-06-10T01:14:55.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:55 smithi007 ceph-mon[32398]: pgmap v699: 129 pgs: 129 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.5 MiB/s wr, 294 op/s 2024-06-10T01:14:55.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:55 smithi078 ceph-mon[36683]: pgmap v699: 129 pgs: 129 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.5 MiB/s wr, 294 op/s 2024-06-10T01:14:55.961 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '76318bf4', quiesce took 5.8 seconds. Will release after: 12.0 2024-06-10T01:14:56.742 INFO:tasks.workunit.client.0.smithi007.stdout: 1 664628 7.89 MB/sec execute 337 sec latency 1497.349 ms 2024-06-10T01:14:57.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:57 smithi175 ceph-mon[36636]: pgmap v700: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 1.5 MiB/s wr, 271 op/s 2024-06-10T01:14:57.742 INFO:tasks.workunit.client.0.smithi007.stdout: 1 664628 7.87 MB/sec execute 338 sec latency 2497.458 ms 2024-06-10T01:14:57.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:57 smithi007 ceph-mon[32398]: pgmap v700: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 1.5 MiB/s wr, 271 op/s 2024-06-10T01:14:57.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:57 smithi078 ceph-mon[36683]: pgmap v700: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 1.5 MiB/s wr, 271 op/s 2024-06-10T01:14:58.742 INFO:tasks.workunit.client.0.smithi007.stdout: 1 664628 7.85 MB/sec execute 339 sec latency 3497.558 ms 2024-06-10T01:14:59.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:14:59 smithi175 ceph-mon[36636]: pgmap v701: 129 pgs: 129 active+clean; 115 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 1.5 MiB/s wr, 258 op/s 2024-06-10T01:14:59.743 INFO:tasks.workunit.client.0.smithi007.stdout: 1 664628 7.82 MB/sec execute 340 sec latency 4497.681 ms 2024-06-10T01:14:59.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:14:59 smithi007 ceph-mon[32398]: pgmap v701: 129 pgs: 129 active+clean; 115 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 1.5 MiB/s wr, 258 op/s 2024-06-10T01:14:59.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:14:59 smithi078 ceph-mon[36683]: pgmap v701: 129 pgs: 129 active+clean; 115 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 1.5 MiB/s wr, 258 op/s 2024-06-10T01:15:00.742 INFO:tasks.workunit.client.0.smithi007.stdout: 1 664628 7.80 MB/sec execute 341 sec latency 5497.778 ms 2024-06-10T01:15:01.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:01 smithi175 ceph-mon[36636]: pgmap v702: 129 pgs: 129 active+clean; 122 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 2.0 MiB/s wr, 282 op/s 2024-06-10T01:15:01.743 INFO:tasks.workunit.client.0.smithi007.stdout: 1 664628 7.78 MB/sec execute 342 sec latency 6497.834 ms 2024-06-10T01:15:01.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:01 smithi007 ceph-mon[32398]: pgmap v702: 129 pgs: 129 active+clean; 122 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 2.0 MiB/s wr, 282 op/s 2024-06-10T01:15:01.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:01 smithi078 ceph-mon[36683]: pgmap v702: 129 pgs: 129 active+clean; 122 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 2.0 MiB/s wr, 282 op/s 2024-06-10T01:15:02.743 INFO:tasks.workunit.client.0.smithi007.stdout: 1 664628 7.75 MB/sec execute 343 sec latency 7497.938 ms 2024-06-10T01:15:03.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:03 smithi175 ceph-mon[36636]: pgmap v703: 129 pgs: 129 active+clean; 122 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 1.8 MiB/s wr, 203 op/s 2024-06-10T01:15:03.743 INFO:tasks.workunit.client.0.smithi007.stdout: 1 664628 7.73 MB/sec execute 344 sec latency 8498.029 ms 2024-06-10T01:15:03.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:03 smithi007 ceph-mon[32398]: pgmap v703: 129 pgs: 129 active+clean; 122 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 1.8 MiB/s wr, 203 op/s 2024-06-10T01:15:03.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:03 smithi078 ceph-mon[36683]: pgmap v703: 129 pgs: 129 active+clean; 122 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 1.8 MiB/s wr, 203 op/s 2024-06-10T01:15:04.743 INFO:tasks.workunit.client.0.smithi007.stdout: 1 664628 7.71 MB/sec execute 345 sec latency 9498.102 ms 2024-06-10T01:15:05.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:05 smithi175 ceph-mon[36636]: pgmap v704: 129 pgs: 129 active+clean; 120 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 1.9 MiB/s wr, 206 op/s 2024-06-10T01:15:05.743 INFO:tasks.workunit.client.0.smithi007.stdout: 1 664628 7.69 MB/sec execute 346 sec latency 10498.215 ms 2024-06-10T01:15:05.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:05 smithi007 ceph-mon[32398]: pgmap v704: 129 pgs: 129 active+clean; 120 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 1.9 MiB/s wr, 206 op/s 2024-06-10T01:15:05.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:05 smithi078 ceph-mon[36683]: pgmap v704: 129 pgs: 129 active+clean; 120 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 1.9 MiB/s wr, 206 op/s 2024-06-10T01:15:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.973+0000 7fea1229e640 1 -- 172.21.15.7:0/3696312434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea0c102570 msgr2=0x7fea0c104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.973+0000 7fea1229e640 1 --2- 172.21.15.7:0/3696312434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea0c102570 0x7fea0c104960 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc009900 tx=0x7fe9fc02a410 comp rx=0 tx=0).stop 2024-06-10T01:15:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.974+0000 7fea1229e640 1 -- 172.21.15.7:0/3696312434 shutdown_connections 2024-06-10T01:15:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.974+0000 7fea1229e640 1 --2- 172.21.15.7:0/3696312434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea0c1077d0 0x7fea0c109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.974+0000 7fea1229e640 1 --2- 172.21.15.7:0/3696312434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea0c104ea0 0x7fea0c107290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.974+0000 7fea1229e640 1 --2- 172.21.15.7:0/3696312434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea0c102570 0x7fea0c104960 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.974+0000 7fea1229e640 1 -- 172.21.15.7:0/3696312434 >> 172.21.15.7:0/3696312434 conn(0x7fea0c0fbec0 msgr2=0x7fea0c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.974+0000 7fea1229e640 1 -- 172.21.15.7:0/3696312434 shutdown_connections 2024-06-10T01:15:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.974+0000 7fea1229e640 1 -- 172.21.15.7:0/3696312434 wait complete. 2024-06-10T01:15:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.974+0000 7fea1229e640 1 Processor -- start 2024-06-10T01:15:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.974+0000 7fea1229e640 1 -- start start 2024-06-10T01:15:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.975+0000 7fea1229e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea0c102570 0x7fea0c197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.975+0000 7fea1229e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea0c104ea0 0x7fea0c1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.975+0000 7fea1229e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea0c1077d0 0x7fea0c19f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.975+0000 7fea1229e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c10d6e0 con 0x7fea0c102570 2024-06-10T01:15:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.975+0000 7fea1229e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c10d560 con 0x7fea0c1077d0 2024-06-10T01:15:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.975+0000 7fea1229e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c10d860 con 0x7fea0c104ea0 2024-06-10T01:15:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.975+0000 7fea0bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea0c102570 0x7fea0c197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.975+0000 7fea0bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea0c102570 0x7fea0c197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58430/0 (socket says 172.21.15.7:58430) 2024-06-10T01:15:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.975+0000 7fea0bfff640 1 -- 172.21.15.7:0/178074743 learned_addr learned my addr 172.21.15.7:0/178074743 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.975+0000 7fea0b7fe640 1 --2- 172.21.15.7:0/178074743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea0c104ea0 0x7fea0c1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.975+0000 7fea1083b640 1 --2- 172.21.15.7:0/178074743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea0c1077d0 0x7fea0c19f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.975+0000 7fea0bfff640 1 -- 172.21.15.7:0/178074743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea0c104ea0 msgr2=0x7fea0c1981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.975+0000 7fea0bfff640 1 --2- 172.21.15.7:0/178074743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea0c104ea0 0x7fea0c1981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.975+0000 7fea0bfff640 1 -- 172.21.15.7:0/178074743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea0c1077d0 msgr2=0x7fea0c19f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.975+0000 7fea0bfff640 1 --2- 172.21.15.7:0/178074743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea0c1077d0 0x7fea0c19f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.975+0000 7fea0bfff640 1 -- 172.21.15.7:0/178074743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea0c19f890 con 0x7fea0c102570 2024-06-10T01:15:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.975+0000 7fea1083b640 1 --2- 172.21.15.7:0/178074743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea0c1077d0 0x7fea0c19f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:15:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.975+0000 7fea0b7fe640 1 --2- 172.21.15.7:0/178074743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea0c104ea0 0x7fea0c1981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:15:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.975+0000 7fea0bfff640 1 --2- 172.21.15.7:0/178074743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea0c102570 0x7fea0c197c80 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc009230 tx=0x7fe9fc031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.976+0000 7fea097fa640 1 -- 172.21.15.7:0/178074743 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc03f030 con 0x7fea0c102570 2024-06-10T01:15:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.976+0000 7fea097fa640 1 -- 172.21.15.7:0/178074743 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9fc003ad0 con 0x7fea0c102570 2024-06-10T01:15:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.976+0000 7fea1229e640 1 -- 172.21.15.7:0/178074743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea0c19fb20 con 0x7fea0c102570 2024-06-10T01:15:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.976+0000 7fea097fa640 1 -- 172.21.15.7:0/178074743 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc035070 con 0x7fea0c102570 2024-06-10T01:15:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.976+0000 7fea1229e640 1 -- 172.21.15.7:0/178074743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea0c1a0010 con 0x7fea0c102570 2024-06-10T01:15:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.978+0000 7fea097fa640 1 -- 172.21.15.7:0/178074743 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fe9fc042030 con 0x7fea0c102570 2024-06-10T01:15:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.978+0000 7fea14fad740 1 Processor -- start 2024-06-10T01:15:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.978+0000 7fea14fad740 1 -- start start 2024-06-10T01:15:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.979+0000 7fea14fad740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d13367420 0x564d13367840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.979+0000 7fea14fad740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d13367e10 0x564d1336c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.979+0000 7fea14fad740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d1336cf80 0x564d1336f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.979+0000 7fea14fad740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564d132c64f0 con 0x564d1336cf80 2024-06-10T01:15:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.979+0000 7fea14fad740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564d13264870 con 0x564d13367e10 2024-06-10T01:15:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.979+0000 7fea14fad740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564d132c24b0 con 0x564d13367420 2024-06-10T01:15:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.979+0000 7fea097fa640 1 --2- 172.21.15.7:0/178074743 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe9f40a2b80 0x7fe9f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.979+0000 7fea0b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d1336cf80 0x564d1336f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.979+0000 7fea0b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d1336cf80 0x564d1336f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58432/0 (socket says 172.21.15.7:58432) 2024-06-10T01:15:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.979+0000 7fea0b7fe640 1 -- 172.21.15.7:0/795961985 learned_addr learned my addr 172.21.15.7:0/795961985 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.979+0000 7fea1083b640 1 --2- 172.21.15.7:0/795961985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d13367420 0x564d13367840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.979+0000 7fea097fa640 1 -- 172.21.15.7:0/178074743 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fe9fc0efa90 con 0x7fea0c102570 2024-06-10T01:15:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.979+0000 7fea0bfff640 1 --2- 172.21.15.7:0/795961985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d13367e10 0x564d1336c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.979+0000 7fea1083b640 1 --2- 172.21.15.7:0/178074743 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe9f40a2b80 0x7fe9f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.979+0000 7fea0b7fe640 1 -- 172.21.15.7:0/795961985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d13367420 msgr2=0x564d13367840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.979+0000 7fea0b7fe640 1 --2- 172.21.15.7:0/795961985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d13367420 0x564d13367840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.979+0000 7fea0b7fe640 1 -- 172.21.15.7:0/795961985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d13367e10 msgr2=0x564d1336c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.979+0000 7fea0b7fe640 1 --2- 172.21.15.7:0/795961985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d13367e10 0x564d1336c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.979+0000 7fea0b7fe640 1 -- 172.21.15.7:0/795961985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d132809b0 con 0x564d1336cf80 2024-06-10T01:15:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.979+0000 7fea0bfff640 1 --2- 172.21.15.7:0/795961985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d13367e10 0x564d1336c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:15:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.979+0000 7fea0b7fe640 1 --2- 172.21.15.7:0/795961985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d1336cf80 0x564d1336f410 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fea00008c10 tx=0x7fea0000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a237b1a46402f8fd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.980+0000 7fea1083b640 1 --2- 172.21.15.7:0/178074743 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe9f40a2b80 0x7fe9f40a5040 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fea04005c40 tx=0x7fea04016040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.980+0000 7fe9f27fc640 1 -- 172.21.15.7:0/795961985 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00014070 con 0x564d1336cf80 2024-06-10T01:15:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.980+0000 7fe9f27fc640 1 -- 172.21.15.7:0/795961985 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea000047a0 con 0x564d1336cf80 2024-06-10T01:15:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.980+0000 7fe9f27fc640 1 -- 172.21.15.7:0/795961985 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00010760 con 0x564d1336cf80 2024-06-10T01:15:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.980+0000 7fea14fad740 1 -- 172.21.15.7:0/795961985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d1336cf80 msgr2=0x564d1336f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.980+0000 7fea14fad740 1 --2- 172.21.15.7:0/795961985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d1336cf80 0x564d1336f410 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fea00008c10 tx=0x7fea0000e720 comp rx=0 tx=0).stop 2024-06-10T01:15:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.980+0000 7fea14fad740 1 -- 172.21.15.7:0/795961985 shutdown_connections 2024-06-10T01:15:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.980+0000 7fea14fad740 1 --2- 172.21.15.7:0/795961985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d1336cf80 0x564d1336f410 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.980+0000 7fea14fad740 1 --2- 172.21.15.7:0/795961985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d13367e10 0x564d1336c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.980+0000 7fea14fad740 1 --2- 172.21.15.7:0/795961985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d13367420 0x564d13367840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.980+0000 7fea14fad740 1 -- 172.21.15.7:0/795961985 >> 172.21.15.7:0/795961985 conn(0x564d1335e680 msgr2=0x564d1335ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.981+0000 7fea14fad740 1 -- 172.21.15.7:0/795961985 shutdown_connections 2024-06-10T01:15:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.981+0000 7fea14fad740 1 -- 172.21.15.7:0/795961985 wait complete. 2024-06-10T01:15:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.981+0000 7fea14fad740 1 -- start start 2024-06-10T01:15:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.981+0000 7fea14fad740 1 Processor -- start 2024-06-10T01:15:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.981+0000 7fea14fad740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d13367420 0x564d13371630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.981+0000 7fea14fad740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d13367e10 0x564d13371b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.981+0000 7fea14fad740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d1336cf80 0x564d13378bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.981+0000 7fea14fad740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564d1336ff30 con 0x564d1336cf80 2024-06-10T01:15:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.981+0000 7fea14fad740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564d1324fae0 con 0x564d13367420 2024-06-10T01:15:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.981+0000 7fea14fad740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564d133700b0 con 0x564d13367e10 2024-06-10T01:15:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.981+0000 7fea14fad740 10 client.0 _ll_register_callbacks cb 0x564d132889b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:15:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.982+0000 7fea1083b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d1336cf80 0x564d13378bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.982+0000 7fea1083b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d1336cf80 0x564d13378bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58442/0 (socket says 172.21.15.7:58442) 2024-06-10T01:15:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.982+0000 7fea1083b640 1 -- 172.21.15.7:0/1086377837 learned_addr learned my addr 172.21.15.7:0/1086377837 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.982+0000 7fea0bfff640 1 --2- 172.21.15.7:0/1086377837 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d13367420 0x564d13371630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.982+0000 7fea0b7fe640 1 --2- 172.21.15.7:0/1086377837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d13367e10 0x564d13371b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.982+0000 7fea1083b640 1 -- 172.21.15.7:0/1086377837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d13367e10 msgr2=0x564d13371b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.982+0000 7fea1083b640 1 --2- 172.21.15.7:0/1086377837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d13367e10 0x564d13371b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.982+0000 7fea1083b640 1 -- 172.21.15.7:0/1086377837 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d13367420 msgr2=0x564d13371630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.982+0000 7fea1083b640 1 --2- 172.21.15.7:0/1086377837 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d13367420 0x564d13371630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.982+0000 7fea1083b640 1 -- 172.21.15.7:0/1086377837 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d12fda210 con 0x564d1336cf80 2024-06-10T01:15:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.982+0000 7fea1083b640 1 --2- 172.21.15.7:0/1086377837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d1336cf80 0x564d13378bb0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fea04011f80 tx=0x7fea04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a727780003d368a6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.982+0000 7fe9eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:15:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.982+0000 7fe9eb7fe640 1 -- 172.21.15.7:0/1086377837 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0400fe60 con 0x564d1336cf80 2024-06-10T01:15:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.982+0000 7fea14fad740 1 -- 172.21.15.7:0/1086377837 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea000047a0 con 0x564d1336cf80 2024-06-10T01:15:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.982+0000 7fe9eb7fe640 1 -- 172.21.15.7:0/1086377837 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fea0401d810 con 0x564d1336cf80 2024-06-10T01:15:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.983+0000 7fe9eb7fe640 1 -- 172.21.15.7:0/1086377837 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fea04032050 con 0x564d1336cf80 2024-06-10T01:15:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.983+0000 7fea14fad740 10 client.17748 fetch_fsmap learned FSMap version 36 2024-06-10T01:15:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.983+0000 7fea14fad740 1 -- 172.21.15.7:0/1086377837 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564d130a6620 con 0x564d1336cf80 2024-06-10T01:15:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.983+0000 7fe9eb7fe640 1 -- 172.21.15.7:0/1086377837 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea000047a0 con 0x564d1336cf80 2024-06-10T01:15:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.983+0000 7fe9eb7fe640 1 -- 172.21.15.7:0/1086377837 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fea0401eb70 con 0x564d1336cf80 2024-06-10T01:15:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.983+0000 7fea14fad740 10 client.17748 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T01:15:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.983+0000 7fea14fad740 10 client.17748 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:15:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.983+0000 7fea14fad740 20 client.17748 populate_metadata read hostname 'smithi007' 2024-06-10T01:15:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.983+0000 7fea14fad740 1 --2- 172.21.15.7:0/1086377837 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564d1337d4f0 0x564d1337f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.983+0000 7fea14fad740 4 client.17748 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:15:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.983+0000 7fea14fad740 1 -- 172.21.15.7:0/1086377837 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d132d9f10 con 0x564d1337d4f0 2024-06-10T01:15:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.983+0000 7fea0bfff640 1 --2- 172.21.15.7:0/1086377837 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564d1337d4f0 0x564d1337f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.984+0000 7fea0bfff640 1 --2- 172.21.15.7:0/1086377837 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564d1337d4f0 0x564d1337f9b0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfc54cd9055ea756 server_cookie=c72b62efc62e8484 in_seq=0 out_seq=0 2024-06-10T01:15:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.984+0000 7fe9eb7fe640 10 client.17748 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.985+0000 7fe9eb7fe640 1 -- 172.21.15.7:0/1086377837 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564d132d9f10 con 0x564d1337d4f0 2024-06-10T01:15:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.985+0000 7fe9eb7fe640 10 client.17748 handle_command_reply: tid=0 2024-06-10T01:15:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.985+0000 7fea14fad740 1 client.17748 shutdown 2024-06-10T01:15:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.985+0000 7fea14fad740 1 -- 172.21.15.7:0/1086377837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d1336cf80 msgr2=0x564d13378bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.985+0000 7fea14fad740 1 --2- 172.21.15.7:0/1086377837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d1336cf80 0x564d13378bb0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fea04011f80 tx=0x7fea04004a40 comp rx=0 tx=0).stop 2024-06-10T01:15:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.985+0000 7fea14fad740 1 -- 172.21.15.7:0/1086377837 shutdown_connections 2024-06-10T01:15:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.985+0000 7fea14fad740 1 --2- 172.21.15.7:0/1086377837 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564d1337d4f0 0x564d1337f9b0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.985+0000 7fea14fad740 1 --2- 172.21.15.7:0/1086377837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d1336cf80 0x564d13378bb0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.985+0000 7fea14fad740 1 --2- 172.21.15.7:0/1086377837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d13367e10 0x564d13371b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.985+0000 7fe9eb7fe640 0 client.17748 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.985+0000 7fea14fad740 1 --2- 172.21.15.7:0/1086377837 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d13367420 0x564d13371630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.985+0000 7fea14fad740 1 -- 172.21.15.7:0/1086377837 >> 172.21.15.7:0/1086377837 conn(0x564d1335e680 msgr2=0x564d1335d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.985+0000 7fea14fad740 1 -- 172.21.15.7:0/1086377837 shutdown_connections 2024-06-10T01:15:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.985+0000 7fea14fad740 1 -- 172.21.15.7:0/1086377837 wait complete. 2024-06-10T01:15:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:05.985+0000 7fea14fad740 20 client.17748 trim_cache size 0 max 16384 2024-06-10T01:15:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.001+0000 7fea14fad740 1 Processor -- start 2024-06-10T01:15:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.001+0000 7fea14fad740 1 -- start start 2024-06-10T01:15:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.001+0000 7fea14fad740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d13367420 0x564d1336c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.001+0000 7fea14fad740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d13367e10 0x564d13378b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.001+0000 7fea14fad740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d1336cf80 0x564d13368fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.001+0000 7fea14fad740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c1aaba0 con 0x564d1336cf80 2024-06-10T01:15:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.001+0000 7fea14fad740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c1a83e0 con 0x564d13367420 2024-06-10T01:15:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.001+0000 7fea14fad740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c1a6230 con 0x564d13367e10 2024-06-10T01:15:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.001+0000 7fea1083b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d1336cf80 0x564d13368fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.001+0000 7fea1083b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d1336cf80 0x564d13368fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58446/0 (socket says 172.21.15.7:58446) 2024-06-10T01:15:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.001+0000 7fea1083b640 1 -- 172.21.15.7:0/1012604340 learned_addr learned my addr 172.21.15.7:0/1012604340 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.001+0000 7fea0bfff640 1 --2- 172.21.15.7:0/1012604340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d13367420 0x564d1336c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.001+0000 7fea0b7fe640 1 --2- 172.21.15.7:0/1012604340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d13367e10 0x564d13378b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.001+0000 7fea1083b640 1 -- 172.21.15.7:0/1012604340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d13367e10 msgr2=0x564d13378b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.001+0000 7fea1083b640 1 --2- 172.21.15.7:0/1012604340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d13367e10 0x564d13378b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.001+0000 7fea1083b640 1 -- 172.21.15.7:0/1012604340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d13367420 msgr2=0x564d1336c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.001+0000 7fea1083b640 1 --2- 172.21.15.7:0/1012604340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d13367420 0x564d1336c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.001+0000 7fea1083b640 1 -- 172.21.15.7:0/1012604340 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d1335e4b0 con 0x564d1336cf80 2024-06-10T01:15:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.001+0000 7fea1083b640 1 --2- 172.21.15.7:0/1012604340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d1336cf80 0x564d13368fc0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fea04009230 tx=0x7fea04011fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16ed63fe1af1fa2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.002+0000 7fe9f1ffb640 1 -- 172.21.15.7:0/1012604340 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0403b070 con 0x564d1336cf80 2024-06-10T01:15:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.002+0000 7fe9f1ffb640 1 -- 172.21.15.7:0/1012604340 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea0400fbf0 con 0x564d1336cf80 2024-06-10T01:15:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.002+0000 7fe9f1ffb640 1 -- 172.21.15.7:0/1012604340 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea040044a0 con 0x564d1336cf80 2024-06-10T01:15:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.002+0000 7fea14fad740 1 -- 172.21.15.7:0/1012604340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d1336cf80 msgr2=0x564d13368fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.002+0000 7fea14fad740 1 --2- 172.21.15.7:0/1012604340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d1336cf80 0x564d13368fc0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fea04009230 tx=0x7fea04011fd0 comp rx=0 tx=0).stop 2024-06-10T01:15:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.002+0000 7fea14fad740 1 -- 172.21.15.7:0/1012604340 shutdown_connections 2024-06-10T01:15:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.002+0000 7fea14fad740 1 --2- 172.21.15.7:0/1012604340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d1336cf80 0x564d13368fc0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.002+0000 7fea14fad740 1 --2- 172.21.15.7:0/1012604340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d13367e10 0x564d13378b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.002+0000 7fea14fad740 1 --2- 172.21.15.7:0/1012604340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d13367420 0x564d1336c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.002+0000 7fea14fad740 1 -- 172.21.15.7:0/1012604340 >> 172.21.15.7:0/1012604340 conn(0x564d1335e680 msgr2=0x564d1335ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.002+0000 7fea14fad740 1 -- 172.21.15.7:0/1012604340 shutdown_connections 2024-06-10T01:15:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.002+0000 7fea14fad740 1 -- 172.21.15.7:0/1012604340 wait complete. 2024-06-10T01:15:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.003+0000 7fea14fad740 1 -- start start 2024-06-10T01:15:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.003+0000 7fea14fad740 1 Processor -- start 2024-06-10T01:15:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.003+0000 7fea14fad740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d13367420 0x564d13375cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.003+0000 7fea14fad740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d13367e10 0x564d13376210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.003+0000 7fea14fad740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d1336cf80 0x564d13377220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.003+0000 7fea14fad740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c1a68a0 con 0x564d13367420 2024-06-10T01:15:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.003+0000 7fea14fad740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c1a7d10 con 0x564d13367e10 2024-06-10T01:15:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.003+0000 7fea14fad740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c1a6f70 con 0x564d1336cf80 2024-06-10T01:15:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.003+0000 7fea14fad740 10 client.0 _ll_register_callbacks cb 0x7fea04014f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:15:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.003+0000 7fea0bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d13367420 0x564d13375cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.003+0000 7fea0bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d13367420 0x564d13375cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58460/0 (socket says 172.21.15.7:58460) 2024-06-10T01:15:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.003+0000 7fea0bfff640 1 -- 172.21.15.7:0/2250900116 learned_addr learned my addr 172.21.15.7:0/2250900116 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.003+0000 7fea0b7fe640 1 --2- 172.21.15.7:0/2250900116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d13367e10 0x564d13376210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.003+0000 7fea1083b640 1 --2- 172.21.15.7:0/2250900116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d1336cf80 0x564d13377220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.003+0000 7fea0bfff640 1 -- 172.21.15.7:0/2250900116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d1336cf80 msgr2=0x564d13377220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.003+0000 7fea0bfff640 1 --2- 172.21.15.7:0/2250900116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d1336cf80 0x564d13377220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.003+0000 7fea0bfff640 1 -- 172.21.15.7:0/2250900116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d13367e10 msgr2=0x564d13376210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.003+0000 7fea0bfff640 1 --2- 172.21.15.7:0/2250900116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d13367e10 0x564d13376210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.003+0000 7fea0bfff640 1 -- 172.21.15.7:0/2250900116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d13377810 con 0x564d13367420 2024-06-10T01:15:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.004+0000 7fea0bfff640 1 --2- 172.21.15.7:0/2250900116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d13367420 0x564d13375cd0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc031440 tx=0x7fe9fc0dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8ba8d68dc54178 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.004+0000 7fe9eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:15:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.004+0000 7fe9eb7fe640 1 -- 172.21.15.7:0/2250900116 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc0d6070 con 0x564d13367420 2024-06-10T01:15:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.004+0000 7fe9eb7fe640 1 -- 172.21.15.7:0/2250900116 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fe9fc0b8550 con 0x564d13367420 2024-06-10T01:15:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.004+0000 7fea14fad740 1 -- 172.21.15.7:0/2250900116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea0400fbf0 con 0x564d13367420 2024-06-10T01:15:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.004+0000 7fe9eb7fe640 1 -- 172.21.15.7:0/2250900116 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe9fc086050 con 0x564d13367420 2024-06-10T01:15:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.004+0000 7fea14fad740 10 client.17754 fetch_fsmap learned FSMap version 36 2024-06-10T01:15:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.004+0000 7fea14fad740 1 -- 172.21.15.7:0/2250900116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564d13378bf0 con 0x564d13367420 2024-06-10T01:15:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.004+0000 7fe9eb7fe640 1 -- 172.21.15.7:0/2250900116 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea0400fbf0 con 0x564d13367420 2024-06-10T01:15:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.004+0000 7fe9eb7fe640 1 -- 172.21.15.7:0/2250900116 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fe9fc0d9020 con 0x564d13367420 2024-06-10T01:15:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.004+0000 7fea14fad740 10 client.17754 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T01:15:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.004+0000 7fea14fad740 10 client.17754 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:15:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.004+0000 7fea14fad740 20 client.17754 populate_metadata read hostname 'smithi007' 2024-06-10T01:15:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.004+0000 7fea14fad740 1 --2- 172.21.15.7:0/2250900116 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564d133e8840 0x564d133eac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.004+0000 7fea14fad740 4 client.17754 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:15:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.004+0000 7fea14fad740 1 -- 172.21.15.7:0/2250900116 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564d132d3c20 con 0x564d133e8840 2024-06-10T01:15:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.005+0000 7fea0b7fe640 1 --2- 172.21.15.7:0/2250900116 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564d133e8840 0x564d133eac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.005+0000 7fea0b7fe640 1 --2- 172.21.15.7:0/2250900116 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564d133e8840 0x564d133eac60 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eedaf3ef1621973a server_cookie=fd88fe6ce3cf28b5 in_seq=0 out_seq=0 2024-06-10T01:15:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.005+0000 7fe9eb7fe640 10 client.17754 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.052+0000 7fe9eb7fe640 1 -- 172.21.15.7:0/2250900116 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564d132d3c20 con 0x564d133e8840 2024-06-10T01:15:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.052+0000 7fe9eb7fe640 10 client.17754 handle_command_reply: tid=0 2024-06-10T01:15:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.052+0000 7fea14fad740 1 client.17754 shutdown 2024-06-10T01:15:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.053+0000 7fea14fad740 1 -- 172.21.15.7:0/2250900116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d13367420 msgr2=0x564d13375cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.053+0000 7fea14fad740 1 --2- 172.21.15.7:0/2250900116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d13367420 0x564d13375cd0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc031440 tx=0x7fe9fc0dc040 comp rx=0 tx=0).stop 2024-06-10T01:15:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.053+0000 7fea14fad740 1 -- 172.21.15.7:0/2250900116 shutdown_connections 2024-06-10T01:15:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.053+0000 7fea14fad740 1 --2- 172.21.15.7:0/2250900116 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564d133e8840 0x564d133eac60 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.053+0000 7fea14fad740 1 --2- 172.21.15.7:0/2250900116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d1336cf80 0x564d13377220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.053+0000 7fea14fad740 1 --2- 172.21.15.7:0/2250900116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d13367e10 0x564d13376210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.053+0000 7fe9eb7fe640 0 client.17754 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.053+0000 7fea14fad740 1 --2- 172.21.15.7:0/2250900116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d13367420 0x564d13375cd0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.053+0000 7fea14fad740 1 -- 172.21.15.7:0/2250900116 >> 172.21.15.7:0/2250900116 conn(0x564d1335e680 msgr2=0x564d13368fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.053+0000 7fea14fad740 1 -- 172.21.15.7:0/2250900116 shutdown_connections 2024-06-10T01:15:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.053+0000 7fea14fad740 1 -- 172.21.15.7:0/2250900116 wait complete. 2024-06-10T01:15:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.053+0000 7fea14fad740 20 client.17754 trim_cache size 0 max 16384 2024-06-10T01:15:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:15:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:15:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:15:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.053+0000 7fea1229e640 1 -- 172.21.15.7:0/178074743 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe9f40a2b80 msgr2=0x7fe9f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.053+0000 7fea1229e640 1 --2- 172.21.15.7:0/178074743 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe9f40a2b80 0x7fe9f40a5040 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fea04005c40 tx=0x7fea04016040 comp rx=0 tx=0).stop 2024-06-10T01:15:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.053+0000 7fea1229e640 1 -- 172.21.15.7:0/178074743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea0c102570 msgr2=0x7fea0c197c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.053+0000 7fea1229e640 1 --2- 172.21.15.7:0/178074743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea0c102570 0x7fea0c197c80 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc009230 tx=0x7fe9fc031b60 comp rx=0 tx=0).stop 2024-06-10T01:15:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.054+0000 7fea1229e640 1 -- 172.21.15.7:0/178074743 shutdown_connections 2024-06-10T01:15:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.054+0000 7fea1229e640 1 --2- 172.21.15.7:0/178074743 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe9f40a2b80 0x7fe9f40a5040 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.054+0000 7fea1229e640 1 --2- 172.21.15.7:0/178074743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea0c1077d0 0x7fea0c19f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.054+0000 7fea1229e640 1 --2- 172.21.15.7:0/178074743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea0c104ea0 0x7fea0c1981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.054+0000 7fea1229e640 1 --2- 172.21.15.7:0/178074743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea0c102570 0x7fea0c197c80 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.054+0000 7fea1229e640 1 -- 172.21.15.7:0/178074743 >> 172.21.15.7:0/178074743 conn(0x7fea0c0fbec0 msgr2=0x7fea0c102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:06.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.054+0000 7fea1229e640 1 -- 172.21.15.7:0/178074743 shutdown_connections 2024-06-10T01:15:06.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:06.054+0000 7fea1229e640 1 -- 172.21.15.7:0/178074743 wait complete. 2024-06-10T01:15:06.743 INFO:tasks.workunit.client.0.smithi007.stdout: 1 664628 7.67 MB/sec execute 347 sec latency 11498.275 ms 2024-06-10T01:15:07.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:07 smithi175 ceph-mon[36636]: pgmap v705: 129 pgs: 129 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.8 MiB/s wr, 143 op/s 2024-06-10T01:15:07.743 INFO:tasks.workunit.client.0.smithi007.stdout: 1 664628 7.64 MB/sec execute 348 sec latency 12498.333 ms 2024-06-10T01:15:07.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:07 smithi007 ceph-mon[32398]: pgmap v705: 129 pgs: 129 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.8 MiB/s wr, 143 op/s 2024-06-10T01:15:07.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:07 smithi078 ceph-mon[36683]: pgmap v705: 129 pgs: 129 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.8 MiB/s wr, 143 op/s 2024-06-10T01:15:07.963 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:15:08.222 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:15:08.743 INFO:tasks.workunit.client.0.smithi007.stdout: 1 664628 7.62 MB/sec execute 349 sec latency 13498.379 ms 2024-06-10T01:15:09.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:09 smithi175 ceph-mon[36636]: pgmap v706: 129 pgs: 129 active+clean; 117 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 1.7 MiB/s wr, 114 op/s 2024-06-10T01:15:09.743 INFO:tasks.workunit.client.0.smithi007.stdout: 1 664628 7.60 MB/sec execute 350 sec latency 14498.414 ms 2024-06-10T01:15:09.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:09 smithi007 ceph-mon[32398]: pgmap v706: 129 pgs: 129 active+clean; 117 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 1.7 MiB/s wr, 114 op/s 2024-06-10T01:15:09.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:09 smithi078 ceph-mon[36683]: pgmap v706: 129 pgs: 129 active+clean; 117 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 1.7 MiB/s wr, 114 op/s 2024-06-10T01:15:10.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.232+0000 7fb37cbf5700 1 -- 172.21.15.7:0/255111081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb378101390 msgr2=0x7fb378101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:10.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.232+0000 7fb37cbf5700 1 --2- 172.21.15.7:0/255111081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb378101390 0x7fb378101810 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fb360009990 tx=0x7fb360009ca0 comp rx=0 tx=0).stop 2024-06-10T01:15:10.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.232+0000 7fb37cbf5700 1 -- 172.21.15.7:0/255111081 shutdown_connections 2024-06-10T01:15:10.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.232+0000 7fb37cbf5700 1 --2- 172.21.15.7:0/255111081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb378101d50 0x7fb37810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:10.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.232+0000 7fb37cbf5700 1 --2- 172.21.15.7:0/255111081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb378107400 0x7fb3781077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:10.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.232+0000 7fb37cbf5700 1 --2- 172.21.15.7:0/255111081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb378101390 0x7fb378101810 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:10.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.232+0000 7fb37cbf5700 1 -- 172.21.15.7:0/255111081 >> 172.21.15.7:0/255111081 conn(0x7fb378078340 msgr2=0x7fb378078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:10.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.232+0000 7fb37cbf5700 1 -- 172.21.15.7:0/255111081 shutdown_connections 2024-06-10T01:15:10.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.232+0000 7fb37cbf5700 1 -- 172.21.15.7:0/255111081 wait complete. 2024-06-10T01:15:10.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.233+0000 7fb37cbf5700 1 Processor -- start 2024-06-10T01:15:10.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.233+0000 7fb37cbf5700 1 -- start start 2024-06-10T01:15:10.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.233+0000 7fb37cbf5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb378101390 0x7fb37819e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:10.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.233+0000 7fb37cbf5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb378101d50 0x7fb37819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:10.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.233+0000 7fb37cbf5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb378107400 0x7fb3781989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:10.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.233+0000 7fb37cbf5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb378111670 con 0x7fb378107400 2024-06-10T01:15:10.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.233+0000 7fb37cbf5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb3781114f0 con 0x7fb378101390 2024-06-10T01:15:10.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.233+0000 7fb37cbf5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb3781117f0 con 0x7fb378101d50 2024-06-10T01:15:10.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.233+0000 7fb376d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb378107400 0x7fb3781989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:10.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.234+0000 7fb376d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb378107400 0x7fb3781989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58466/0 (socket says 172.21.15.7:58466) 2024-06-10T01:15:10.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.234+0000 7fb376d9d700 1 -- 172.21.15.7:0/1419483931 learned_addr learned my addr 172.21.15.7:0/1419483931 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:10.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.234+0000 7fb36dd9b700 1 --2- 172.21.15.7:0/1419483931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb378101d50 0x7fb37819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:10.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.234+0000 7fb37659c700 1 --2- 172.21.15.7:0/1419483931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb378101390 0x7fb37819e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:10.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.234+0000 7fb376d9d700 1 -- 172.21.15.7:0/1419483931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb378101d50 msgr2=0x7fb37819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:10.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.234+0000 7fb376d9d700 1 --2- 172.21.15.7:0/1419483931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb378101d50 0x7fb37819ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:10.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.234+0000 7fb376d9d700 1 -- 172.21.15.7:0/1419483931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb378101390 msgr2=0x7fb37819e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:10.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.234+0000 7fb376d9d700 1 --2- 172.21.15.7:0/1419483931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb378101390 0x7fb37819e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:10.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.234+0000 7fb376d9d700 1 -- 172.21.15.7:0/1419483931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb378199120 con 0x7fb378107400 2024-06-10T01:15:10.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.234+0000 7fb376d9d700 1 --2- 172.21.15.7:0/1419483931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb378107400 0x7fb3781989c0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fb370007ee0 tx=0x7fb37000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:10.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.234+0000 7fb36ffff700 1 -- 172.21.15.7:0/1419483931 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370004bb0 con 0x7fb378107400 2024-06-10T01:15:10.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.234+0000 7fb37cbf5700 1 -- 172.21.15.7:0/1419483931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb378199410 con 0x7fb378107400 2024-06-10T01:15:10.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.234+0000 7fb36ffff700 1 -- 172.21.15.7:0/1419483931 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb370004d50 con 0x7fb378107400 2024-06-10T01:15:10.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.234+0000 7fb36ffff700 1 -- 172.21.15.7:0/1419483931 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370005960 con 0x7fb378107400 2024-06-10T01:15:10.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.234+0000 7fb37cbf5700 1 -- 172.21.15.7:0/1419483931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3781a56d0 con 0x7fb378107400 2024-06-10T01:15:10.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.235+0000 7fb37cbf5700 1 -- 172.21.15.7:0/1419483931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb378069180 con 0x7fb378107400 2024-06-10T01:15:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.236+0000 7fb36ffff700 1 -- 172.21.15.7:0/1419483931 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb37001a020 con 0x7fb378107400 2024-06-10T01:15:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.237+0000 7fb36ffff700 1 --2- 172.21.15.7:0/1419483931 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb3640a2c20 0x7fb3640a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.237+0000 7fb36ffff700 1 -- 172.21.15.7:0/1419483931 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb3700cf1d0 con 0x7fb378107400 2024-06-10T01:15:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.237+0000 7fb37659c700 1 --2- 172.21.15.7:0/1419483931 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb3640a2c20 0x7fb3640a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.238+0000 7fb37659c700 1 --2- 172.21.15.7:0/1419483931 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb3640a2c20 0x7fb3640a50e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb378199b00 tx=0x7fb368009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:10.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.241+0000 7fb36ffff700 1 -- 172.21.15.7:0/1419483931 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb370010070 con 0x7fb378107400 2024-06-10T01:15:10.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.496+0000 7fb37cbf5700 1 -- 172.21.15.7:0/1419483931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3781a58a0 con 0x7fb378107400 2024-06-10T01:15:10.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.497+0000 7fb36ffff700 1 -- 172.21.15.7:0/1419483931 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11860 (secure 0 0 0) 0x7fb37009bdb0 con 0x7fb378107400 2024-06-10T01:15:10.491 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:15:10.491 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:14:51.406569+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24434,24436,24451]},"id":1}]} 2024-06-10T01:15:10.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.500+0000 7fb37cbf5700 1 -- 172.21.15.7:0/1419483931 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb3640a2c20 msgr2=0x7fb3640a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:10.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.500+0000 7fb37cbf5700 1 --2- 172.21.15.7:0/1419483931 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb3640a2c20 0x7fb3640a50e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb378199b00 tx=0x7fb368009260 comp rx=0 tx=0).stop 2024-06-10T01:15:10.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.500+0000 7fb37cbf5700 1 -- 172.21.15.7:0/1419483931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb378107400 msgr2=0x7fb3781989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:10.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.500+0000 7fb37cbf5700 1 --2- 172.21.15.7:0/1419483931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb378107400 0x7fb3781989c0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fb370007ee0 tx=0x7fb37000d3b0 comp rx=0 tx=0).stop 2024-06-10T01:15:10.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.500+0000 7fb37cbf5700 1 -- 172.21.15.7:0/1419483931 shutdown_connections 2024-06-10T01:15:10.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.500+0000 7fb37cbf5700 1 --2- 172.21.15.7:0/1419483931 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb3640a2c20 0x7fb3640a50e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:10.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.500+0000 7fb37cbf5700 1 --2- 172.21.15.7:0/1419483931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb378107400 0x7fb3781989c0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:10.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.500+0000 7fb37cbf5700 1 --2- 172.21.15.7:0/1419483931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb378101390 0x7fb37819e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:10.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.500+0000 7fb37cbf5700 1 --2- 172.21.15.7:0/1419483931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb378101d50 0x7fb37819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:10.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.500+0000 7fb37cbf5700 1 -- 172.21.15.7:0/1419483931 >> 172.21.15.7:0/1419483931 conn(0x7fb378078340 msgr2=0x7fb3781005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:10.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.500+0000 7fb37cbf5700 1 -- 172.21.15.7:0/1419483931 shutdown_connections 2024-06-10T01:15:10.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:10.500+0000 7fb37cbf5700 1 -- 172.21.15.7:0/1419483931 wait complete. 2024-06-10T01:15:10.495 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 36 2024-06-10T01:15:10.743 INFO:tasks.workunit.client.0.smithi007.stdout: 1 664628 7.58 MB/sec execute 351 sec latency 15498.543 ms 2024-06-10T01:15:11.283 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db --set-id 76318bf4 --release --await' 2024-06-10T01:15:11.284 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db --set-id 76318bf4 --release --await 2024-06-10T01:15:11.512 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:11 smithi007 ceph-mon[32398]: pgmap v707: 129 pgs: 129 active+clean; 114 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 1.9 MiB/s wr, 104 op/s 2024-06-10T01:15:11.512 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:11 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1419483931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:15:11.538 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:15:11.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:11 smithi175 ceph-mon[36636]: pgmap v707: 129 pgs: 129 active+clean; 114 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 1.9 MiB/s wr, 104 op/s 2024-06-10T01:15:11.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:11 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1419483931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:15:11.743 INFO:tasks.workunit.client.0.smithi007.stdout: 1 664628 7.56 MB/sec execute 352 sec latency 16498.662 ms 2024-06-10T01:15:11.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:11 smithi078 ceph-mon[36683]: pgmap v707: 129 pgs: 129 active+clean; 114 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 1.9 MiB/s wr, 104 op/s 2024-06-10T01:15:11.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:11 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1419483931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:15:12.744 INFO:tasks.workunit.client.0.smithi007.stdout: 1 664628 7.54 MB/sec execute 353 sec latency 17498.722 ms 2024-06-10T01:15:13.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.267+0000 7f1fa6adb700 1 -- 172.21.15.7:0/3949854009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fa006b770 msgr2=0x7f1fa0107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.267+0000 7f1fa6adb700 1 --2- 172.21.15.7:0/3949854009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fa006b770 0x7f1fa0107130 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f1f940099c0 tx=0x7f1f94009cd0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.267+0000 7f1fa6adb700 1 -- 172.21.15.7:0/3949854009 shutdown_connections 2024-06-10T01:15:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.267+0000 7f1fa6adb700 1 --2- 172.21.15.7:0/3949854009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fa0107860 0x7f1fa010f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.267+0000 7f1fa6adb700 1 --2- 172.21.15.7:0/3949854009 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fa006adc0 0x7f1fa006b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.267+0000 7f1fa6adb700 1 --2- 172.21.15.7:0/3949854009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fa006b770 0x7f1fa0107130 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.267+0000 7f1fa6adb700 1 -- 172.21.15.7:0/3949854009 >> 172.21.15.7:0/3949854009 conn(0x7f1fa0078340 msgr2=0x7f1fa0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.267+0000 7f1fa6adb700 1 -- 172.21.15.7:0/3949854009 shutdown_connections 2024-06-10T01:15:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.267+0000 7f1fa6adb700 1 -- 172.21.15.7:0/3949854009 wait complete. 2024-06-10T01:15:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.268+0000 7f1fa6adb700 1 Processor -- start 2024-06-10T01:15:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.268+0000 7f1fa6adb700 1 -- start start 2024-06-10T01:15:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.268+0000 7f1fa6adb700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fa006adc0 0x7f1fa019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.268+0000 7f1fa6adb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fa006b770 0x7f1fa019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.268+0000 7f1fa6adb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fa0107860 0x7f1fa0198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.268+0000 7f1fa6adb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa0112770 con 0x7f1fa0107860 2024-06-10T01:15:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.268+0000 7f1fa6adb700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa01125f0 con 0x7f1fa006b770 2024-06-10T01:15:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.268+0000 7f1fa6adb700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa01128f0 con 0x7f1fa006adc0 2024-06-10T01:15:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.268+0000 7f1fa5078700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fa0107860 0x7f1fa0198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.268+0000 7f1fa5078700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fa0107860 0x7f1fa0198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50954/0 (socket says 172.21.15.7:50954) 2024-06-10T01:15:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.268+0000 7f1fa5078700 1 -- 172.21.15.7:0/2204445237 learned_addr learned my addr 172.21.15.7:0/2204445237 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.268+0000 7f1f9ffff700 1 --2- 172.21.15.7:0/2204445237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fa006b770 0x7f1fa019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.268+0000 7f1fa4877700 1 --2- 172.21.15.7:0/2204445237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fa006adc0 0x7f1fa019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.269+0000 7f1fa5078700 1 -- 172.21.15.7:0/2204445237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fa006adc0 msgr2=0x7f1fa019e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.269+0000 7f1fa5078700 1 --2- 172.21.15.7:0/2204445237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fa006adc0 0x7f1fa019e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.269+0000 7f1fa5078700 1 -- 172.21.15.7:0/2204445237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fa006b770 msgr2=0x7f1fa019ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.269+0000 7f1fa5078700 1 --2- 172.21.15.7:0/2204445237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fa006b770 0x7f1fa019ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.269+0000 7f1fa5078700 1 -- 172.21.15.7:0/2204445237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa0199210 con 0x7f1fa0107860 2024-06-10T01:15:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.269+0000 7f1f9ffff700 1 --2- 172.21.15.7:0/2204445237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fa006b770 0x7f1fa019ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:13.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.269+0000 7f1fa5078700 1 --2- 172.21.15.7:0/2204445237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fa0107860 0x7f1fa0198ab0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800ada0 tx=0x7f1f9800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:13.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.269+0000 7f1f9dffb700 1 -- 172.21.15.7:0/2204445237 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f98017020 con 0x7f1fa0107860 2024-06-10T01:15:13.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.269+0000 7f1fa6adb700 1 -- 172.21.15.7:0/2204445237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fa0199500 con 0x7f1fa0107860 2024-06-10T01:15:13.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.269+0000 7f1f9dffb700 1 -- 172.21.15.7:0/2204445237 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f9800cdb0 con 0x7f1fa0107860 2024-06-10T01:15:13.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.269+0000 7f1f9dffb700 1 -- 172.21.15.7:0/2204445237 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f980053e0 con 0x7f1fa0107860 2024-06-10T01:15:13.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.269+0000 7f1fa6adb700 1 -- 172.21.15.7:0/2204445237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fa01a5770 con 0x7f1fa0107860 2024-06-10T01:15:13.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.271+0000 7f1f9dffb700 1 -- 172.21.15.7:0/2204445237 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f1f9800c8d0 con 0x7f1fa0107860 2024-06-10T01:15:13.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.272+0000 7f1f9dffb700 1 --2- 172.21.15.7:0/2204445237 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1f880a2c20 0x7f1f880a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:13.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.273+0000 7f1fa4877700 1 --2- 172.21.15.7:0/2204445237 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1f880a2c20 0x7f1f880a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:13.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.273+0000 7f1f9dffb700 1 -- 172.21.15.7:0/2204445237 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f1f980cead0 con 0x7f1fa0107860 2024-06-10T01:15:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.273+0000 7f1fb135eb80 1 Processor -- start 2024-06-10T01:15:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.273+0000 7f1fa4877700 1 --2- 172.21.15.7:0/2204445237 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1f880a2c20 0x7f1f880a50e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0107310 tx=0x7f1f90009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.273+0000 7f1fb135eb80 1 -- start start 2024-06-10T01:15:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.273+0000 7f1fb135eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625c3ee9500 0x5625c3ee98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.273+0000 7f1fb135eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625c3ee9e20 0x5625c3ee3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.273+0000 7f1fb135eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625c3ee3c50 0x5625c3ee4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.273+0000 7f1fb135eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5625c3dd6060 con 0x5625c3ee9500 2024-06-10T01:15:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.273+0000 7f1fb135eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5625c3e34950 con 0x5625c3ee9e20 2024-06-10T01:15:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.273+0000 7f1fb135eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5625c3d9c560 con 0x5625c3ee3c50 2024-06-10T01:15:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.274+0000 7f1fa4877700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625c3ee9500 0x5625c3ee98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.274+0000 7f1fa4877700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625c3ee9500 0x5625c3ee98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50962/0 (socket says 172.21.15.7:50962) 2024-06-10T01:15:13.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.274+0000 7f1fa4877700 1 -- 172.21.15.7:0/1658457024 learned_addr learned my addr 172.21.15.7:0/1658457024 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:13.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.274+0000 7f1f9ffff700 1 --2- 172.21.15.7:0/1658457024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625c3ee9e20 0x5625c3ee3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:13.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.274+0000 7f1fa5078700 1 --2- 172.21.15.7:0/1658457024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625c3ee3c50 0x5625c3ee4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:13.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.274+0000 7f1f9ffff700 1 -- 172.21.15.7:0/1658457024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625c3ee3c50 msgr2=0x5625c3ee4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:13.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.274+0000 7f1f9ffff700 1 --2- 172.21.15.7:0/1658457024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625c3ee3c50 0x5625c3ee4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.274+0000 7f1f9ffff700 1 -- 172.21.15.7:0/1658457024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625c3ee9500 msgr2=0x5625c3ee98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:13.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.274+0000 7f1f9ffff700 1 --2- 172.21.15.7:0/1658457024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625c3ee9500 0x5625c3ee98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.274+0000 7f1f9ffff700 1 -- 172.21.15.7:0/1658457024 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625c3d839d0 con 0x5625c3ee9e20 2024-06-10T01:15:13.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.274+0000 7f1fa4877700 1 --2- 172.21.15.7:0/1658457024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625c3ee9500 0x5625c3ee98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:13.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.274+0000 7f1f9ffff700 1 --2- 172.21.15.7:0/1658457024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625c3ee9e20 0x5625c3ee3710 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1f94009990 tx=0x7f1f94003730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6362adac398ea57 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:13.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.274+0000 7f1f86ffd700 1 -- 172.21.15.7:0/1658457024 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94028070 con 0x5625c3ee9e20 2024-06-10T01:15:13.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.274+0000 7f1f86ffd700 1 -- 172.21.15.7:0/1658457024 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f940055b0 con 0x5625c3ee9e20 2024-06-10T01:15:13.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.274+0000 7f1f86ffd700 1 -- 172.21.15.7:0/1658457024 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94023460 con 0x5625c3ee9e20 2024-06-10T01:15:13.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.275+0000 7f1fb135eb80 1 -- 172.21.15.7:0/1658457024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625c3ee9e20 msgr2=0x5625c3ee3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:13.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.275+0000 7f1fb135eb80 1 --2- 172.21.15.7:0/1658457024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625c3ee9e20 0x5625c3ee3710 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1f94009990 tx=0x7f1f94003730 comp rx=0 tx=0).stop 2024-06-10T01:15:13.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.275+0000 7f1fb135eb80 1 -- 172.21.15.7:0/1658457024 shutdown_connections 2024-06-10T01:15:13.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.275+0000 7f1fb135eb80 1 --2- 172.21.15.7:0/1658457024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625c3ee3c50 0x5625c3ee4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.275+0000 7f1fb135eb80 1 --2- 172.21.15.7:0/1658457024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625c3ee9500 0x5625c3ee98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.275+0000 7f1fb135eb80 1 --2- 172.21.15.7:0/1658457024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625c3ee9e20 0x5625c3ee3710 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.275+0000 7f1fb135eb80 1 -- 172.21.15.7:0/1658457024 >> 172.21.15.7:0/1658457024 conn(0x5625c3e10180 msgr2=0x5625c3edea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:13.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.275+0000 7f1fb135eb80 1 -- 172.21.15.7:0/1658457024 shutdown_connections 2024-06-10T01:15:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.275+0000 7f1fb135eb80 1 -- 172.21.15.7:0/1658457024 wait complete. 2024-06-10T01:15:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.276+0000 7f1fb135eb80 1 -- start start 2024-06-10T01:15:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.276+0000 7f1fb135eb80 1 Processor -- start 2024-06-10T01:15:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.276+0000 7f1fb135eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625c3ee3c50 0x5625c3eefbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.276+0000 7f1fb135eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625c3ee9500 0x5625c3eecf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.276+0000 7f1fb135eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625c3ee9e20 0x5625c3eed450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.276+0000 7f1fb135eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5625c3dd50c0 con 0x5625c3ee3c50 2024-06-10T01:15:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.276+0000 7f1fb135eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5625c3e30910 con 0x5625c3ee9500 2024-06-10T01:15:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.276+0000 7f1fb135eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5625c3ef07a0 con 0x5625c3ee9e20 2024-06-10T01:15:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.276+0000 7f1fb135eb80 10 client.0 _ll_register_callbacks cb 0x5625c3dca6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:15:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.276+0000 7f1fa4877700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625c3ee3c50 0x5625c3eefbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.276+0000 7f1fa4877700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625c3ee3c50 0x5625c3eefbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50972/0 (socket says 172.21.15.7:50972) 2024-06-10T01:15:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.276+0000 7f1fa4877700 1 -- 172.21.15.7:0/1814022389 learned_addr learned my addr 172.21.15.7:0/1814022389 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.276+0000 7f1fa5078700 1 --2- 172.21.15.7:0/1814022389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625c3ee9e20 0x5625c3eed450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.277+0000 7f1f9ffff700 1 --2- 172.21.15.7:0/1814022389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625c3ee9500 0x5625c3eecf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.277+0000 7f1fa5078700 1 -- 172.21.15.7:0/1814022389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625c3ee9500 msgr2=0x5625c3eecf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.277+0000 7f1fa5078700 1 --2- 172.21.15.7:0/1814022389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625c3ee9500 0x5625c3eecf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.277+0000 7f1fa5078700 1 -- 172.21.15.7:0/1814022389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625c3ee3c50 msgr2=0x5625c3eefbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.277+0000 7f1fa5078700 1 --2- 172.21.15.7:0/1814022389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625c3ee3c50 0x5625c3eefbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.277+0000 7f1fa5078700 1 -- 172.21.15.7:0/1814022389 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625c3dfa850 con 0x5625c3ee9e20 2024-06-10T01:15:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.277+0000 7f1fa4877700 1 --2- 172.21.15.7:0/1814022389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625c3ee3c50 0x5625c3eefbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.277+0000 7f1fa5078700 1 --2- 172.21.15.7:0/1814022389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625c3ee9e20 0x5625c3eed450 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800c680 tx=0x7f1f9809ac70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7c9551826c0ec4b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.277+0000 7f1f7ffff700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:15:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.277+0000 7f1f7ffff700 1 -- 172.21.15.7:0/1814022389 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f980a3990 con 0x5625c3ee9e20 2024-06-10T01:15:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.277+0000 7f1fb135eb80 1 -- 172.21.15.7:0/1814022389 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f940055b0 con 0x5625c3ee9e20 2024-06-10T01:15:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.277+0000 7f1f7ffff700 1 -- 172.21.15.7:0/1814022389 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f1f9800a1f0 con 0x5625c3ee9e20 2024-06-10T01:15:13.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.277+0000 7f1f7ffff700 1 -- 172.21.15.7:0/1814022389 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1f98004ad0 con 0x5625c3ee9e20 2024-06-10T01:15:13.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.278+0000 7f1fb135eb80 10 client.27193 fetch_fsmap learned FSMap version 36 2024-06-10T01:15:13.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.278+0000 7f1fb135eb80 1 -- 172.21.15.7:0/1814022389 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5625c3efafd0 con 0x5625c3ee9e20 2024-06-10T01:15:13.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.278+0000 7f1f7ffff700 1 -- 172.21.15.7:0/1814022389 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f940055b0 con 0x5625c3ee9e20 2024-06-10T01:15:13.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.278+0000 7f1f7ffff700 1 -- 172.21.15.7:0/1814022389 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f1f98012070 con 0x5625c3ee9e20 2024-06-10T01:15:13.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.278+0000 7f1fb135eb80 10 client.27193 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T01:15:13.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.278+0000 7f1fb135eb80 10 client.27193 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:15:13.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.278+0000 7f1fb135eb80 20 client.27193 populate_metadata read hostname 'smithi007' 2024-06-10T01:15:13.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.278+0000 7f1fb135eb80 1 --2- 172.21.15.7:0/1814022389 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5625c3efd4a0 0x5625c3eff900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:13.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.278+0000 7f1fb135eb80 4 client.27193 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:15:13.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.278+0000 7f1fb135eb80 1 -- 172.21.15.7:0/1814022389 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625c3e41e50 con 0x5625c3efd4a0 2024-06-10T01:15:13.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.278+0000 7f1fa4877700 1 --2- 172.21.15.7:0/1814022389 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5625c3efd4a0 0x5625c3eff900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:13.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.279+0000 7f1fa4877700 1 --2- 172.21.15.7:0/1814022389 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5625c3efd4a0 0x5625c3eff900 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc346493369f5b61 server_cookie=5272d0bd160268cd in_seq=0 out_seq=0 2024-06-10T01:15:13.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.279+0000 7f1f7ffff700 10 client.27193 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.279+0000 7f1f7ffff700 1 -- 172.21.15.7:0/1814022389 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5625c3e41e50 con 0x5625c3efd4a0 2024-06-10T01:15:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.279+0000 7f1f7ffff700 10 client.27193 handle_command_reply: tid=0 2024-06-10T01:15:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.279+0000 7f1fb135eb80 1 client.27193 shutdown 2024-06-10T01:15:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.279+0000 7f1fb135eb80 1 -- 172.21.15.7:0/1814022389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625c3ee9e20 msgr2=0x5625c3eed450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.279+0000 7f1fb135eb80 1 --2- 172.21.15.7:0/1814022389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625c3ee9e20 0x5625c3eed450 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800c680 tx=0x7f1f9809ac70 comp rx=0 tx=0).stop 2024-06-10T01:15:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.279+0000 7f1fb135eb80 1 -- 172.21.15.7:0/1814022389 shutdown_connections 2024-06-10T01:15:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.279+0000 7f1fb135eb80 1 --2- 172.21.15.7:0/1814022389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625c3ee9e20 0x5625c3eed450 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.279+0000 7f1fb135eb80 1 --2- 172.21.15.7:0/1814022389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625c3ee3c50 0x5625c3eefbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.279+0000 7f1fb135eb80 1 --2- 172.21.15.7:0/1814022389 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5625c3efd4a0 0x5625c3eff900 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.280+0000 7f1fb135eb80 1 --2- 172.21.15.7:0/1814022389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625c3ee9500 0x5625c3eecf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.280+0000 7f1f7ffff700 0 client.27193 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.280+0000 7f1fb135eb80 1 -- 172.21.15.7:0/1814022389 >> 172.21.15.7:0/1814022389 conn(0x5625c3e10180 msgr2=0x5625c3eddbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.280+0000 7f1fb135eb80 1 -- 172.21.15.7:0/1814022389 shutdown_connections 2024-06-10T01:15:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.280+0000 7f1fb135eb80 1 -- 172.21.15.7:0/1814022389 wait complete. 2024-06-10T01:15:13.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.280+0000 7f1fb135eb80 20 client.27193 trim_cache size 0 max 16384 2024-06-10T01:15:13.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.300+0000 7f1fb135eb80 1 Processor -- start 2024-06-10T01:15:13.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.300+0000 7f1fb135eb80 1 -- start start 2024-06-10T01:15:13.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.300+0000 7f1fb135eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625c3ee3c50 0x5625c3ede330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:13.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.300+0000 7f1fb135eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625c3ee9500 0x5625c3ede870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:13.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.300+0000 7f1fb135eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625c3ee9e20 0x5625c3f05220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:13.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.300+0000 7f1fb135eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa01ae890 con 0x5625c3ee9e20 2024-06-10T01:15:13.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.300+0000 7f1fb135eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa01ac090 con 0x5625c3ee3c50 2024-06-10T01:15:13.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.300+0000 7f1fb135eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa01a9ee0 con 0x5625c3ee9500 2024-06-10T01:15:13.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.300+0000 7f1fa5078700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625c3ee9e20 0x5625c3f05220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:13.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.300+0000 7f1fa5078700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625c3ee9e20 0x5625c3f05220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50980/0 (socket says 172.21.15.7:50980) 2024-06-10T01:15:13.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.300+0000 7f1fa5078700 1 -- 172.21.15.7:0/950332241 learned_addr learned my addr 172.21.15.7:0/950332241 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:13.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.301+0000 7f1fa4877700 1 --2- 172.21.15.7:0/950332241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625c3ee3c50 0x5625c3ede330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:13.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.301+0000 7f1f9ffff700 1 --2- 172.21.15.7:0/950332241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625c3ee9500 0x5625c3ede870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:13.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.301+0000 7f1fa5078700 1 -- 172.21.15.7:0/950332241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625c3ee9500 msgr2=0x5625c3ede870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:13.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.301+0000 7f1fa5078700 1 --2- 172.21.15.7:0/950332241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625c3ee9500 0x5625c3ede870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.301+0000 7f1fa5078700 1 -- 172.21.15.7:0/950332241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625c3ee3c50 msgr2=0x5625c3ede330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.301+0000 7f1fa5078700 1 --2- 172.21.15.7:0/950332241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625c3ee3c50 0x5625c3ede330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.301+0000 7f1fa5078700 1 -- 172.21.15.7:0/950332241 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625c3efade0 con 0x5625c3ee9e20 2024-06-10T01:15:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.301+0000 7f1f9ffff700 1 --2- 172.21.15.7:0/950332241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625c3ee9500 0x5625c3ede870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:15:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.301+0000 7f1fa5078700 1 --2- 172.21.15.7:0/950332241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625c3ee9e20 0x5625c3f05220 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f1f9809afd0 tx=0x7f1f980cc2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a003147d54629474 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.301+0000 7f1f867fc700 1 -- 172.21.15.7:0/950332241 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9809b040 con 0x5625c3ee9e20 2024-06-10T01:15:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.301+0000 7f1f867fc700 1 -- 172.21.15.7:0/950332241 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f9809f3f0 con 0x5625c3ee9e20 2024-06-10T01:15:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.301+0000 7f1f867fc700 1 -- 172.21.15.7:0/950332241 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f98099f00 con 0x5625c3ee9e20 2024-06-10T01:15:13.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.302+0000 7f1fb135eb80 1 -- 172.21.15.7:0/950332241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625c3ee9e20 msgr2=0x5625c3f05220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:13.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.302+0000 7f1fb135eb80 1 --2- 172.21.15.7:0/950332241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625c3ee9e20 0x5625c3f05220 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f1f9809afd0 tx=0x7f1f980cc2c0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.302+0000 7f1fb135eb80 1 -- 172.21.15.7:0/950332241 shutdown_connections 2024-06-10T01:15:13.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.302+0000 7f1fb135eb80 1 --2- 172.21.15.7:0/950332241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625c3ee9e20 0x5625c3f05220 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.302+0000 7f1fb135eb80 1 --2- 172.21.15.7:0/950332241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625c3ee3c50 0x5625c3ede330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.302+0000 7f1fb135eb80 1 --2- 172.21.15.7:0/950332241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625c3ee9500 0x5625c3ede870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.302+0000 7f1fb135eb80 1 -- 172.21.15.7:0/950332241 >> 172.21.15.7:0/950332241 conn(0x5625c3e10180 msgr2=0x5625c3eebf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:13.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.302+0000 7f1fb135eb80 1 -- 172.21.15.7:0/950332241 shutdown_connections 2024-06-10T01:15:13.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.302+0000 7f1fb135eb80 1 -- 172.21.15.7:0/950332241 wait complete. 2024-06-10T01:15:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.302+0000 7f1fb135eb80 1 -- start start 2024-06-10T01:15:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.302+0000 7f1fb135eb80 1 Processor -- start 2024-06-10T01:15:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.303+0000 7f1fb135eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625c3ee3c50 0x5625c3ef2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.303+0000 7f1fb135eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625c3ee9500 0x5625c3ef3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.303+0000 7f1fb135eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625c3ee9e20 0x5625c3f68200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.303+0000 7f1fb135eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa01aa550 con 0x5625c3ee9500 2024-06-10T01:15:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.303+0000 7f1fb135eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa01ab9c0 con 0x5625c3ee3c50 2024-06-10T01:15:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.303+0000 7f1fb135eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa01aac20 con 0x5625c3ee9e20 2024-06-10T01:15:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.303+0000 7f1fb135eb80 10 client.0 _ll_register_callbacks cb 0x5625c3dca6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:15:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.303+0000 7f1f9ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625c3ee9500 0x5625c3ef3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.303+0000 7f1f9ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625c3ee9500 0x5625c3ef3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50984/0 (socket says 172.21.15.7:50984) 2024-06-10T01:15:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.303+0000 7f1f9ffff700 1 -- 172.21.15.7:0/1330368347 learned_addr learned my addr 172.21.15.7:0/1330368347 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.303+0000 7f1fa4877700 1 --2- 172.21.15.7:0/1330368347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625c3ee3c50 0x5625c3ef2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.303+0000 7f1fa5078700 1 --2- 172.21.15.7:0/1330368347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625c3ee9e20 0x5625c3f68200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:13.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.303+0000 7f1f9ffff700 1 -- 172.21.15.7:0/1330368347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625c3ee9e20 msgr2=0x5625c3f68200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:13.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.303+0000 7f1f9ffff700 1 --2- 172.21.15.7:0/1330368347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625c3ee9e20 0x5625c3f68200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.303+0000 7f1f9ffff700 1 -- 172.21.15.7:0/1330368347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625c3ee3c50 msgr2=0x5625c3ef2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:13.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.303+0000 7f1f9ffff700 1 --2- 172.21.15.7:0/1330368347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625c3ee3c50 0x5625c3ef2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.303+0000 7f1f9ffff700 1 -- 172.21.15.7:0/1330368347 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625c3f68740 con 0x5625c3ee9500 2024-06-10T01:15:13.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.303+0000 7f1fa4877700 1 --2- 172.21.15.7:0/1330368347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625c3ee3c50 0x5625c3ef2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:13.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.303+0000 7f1f9ffff700 1 --2- 172.21.15.7:0/1330368347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625c3ee9500 0x5625c3ef3520 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400b910 tx=0x7f1f9400b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9327beb96018709c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:13.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.303+0000 7f1f7ffff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:15:13.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.304+0000 7f1f7ffff700 1 -- 172.21.15.7:0/1330368347 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94028040 con 0x5625c3ee9500 2024-06-10T01:15:13.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.304+0000 7f1fb135eb80 1 -- 172.21.15.7:0/1330368347 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f9809f3f0 con 0x5625c3ee9500 2024-06-10T01:15:13.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.304+0000 7f1f7ffff700 1 -- 172.21.15.7:0/1330368347 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f1f94003af0 con 0x5625c3ee9500 2024-06-10T01:15:13.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.304+0000 7f1f7ffff700 1 -- 172.21.15.7:0/1330368347 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1f9403a050 con 0x5625c3ee9500 2024-06-10T01:15:13.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.304+0000 7f1fb135eb80 10 client.17778 fetch_fsmap learned FSMap version 36 2024-06-10T01:15:13.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.304+0000 7f1fb135eb80 1 -- 172.21.15.7:0/1330368347 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5625c3f699d0 con 0x5625c3ee9500 2024-06-10T01:15:13.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.304+0000 7f1f7ffff700 1 -- 172.21.15.7:0/1330368347 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f9809f3f0 con 0x5625c3ee9500 2024-06-10T01:15:13.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.304+0000 7f1f7ffff700 1 -- 172.21.15.7:0/1330368347 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f1f9402b050 con 0x5625c3ee9500 2024-06-10T01:15:13.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.304+0000 7f1fb135eb80 10 client.17778 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T01:15:13.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.304+0000 7f1fb135eb80 10 client.17778 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:15:13.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.304+0000 7f1fb135eb80 20 client.17778 populate_metadata read hostname 'smithi007' 2024-06-10T01:15:13.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.304+0000 7f1fb135eb80 1 --2- 172.21.15.7:0/1330368347 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5625c3f69cb0 0x5625c3f6c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:13.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.304+0000 7f1fb135eb80 4 client.17778 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "76318bf4", "release": true, "await": true}] 2024-06-10T01:15:13.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.304+0000 7f1fb135eb80 1 -- 172.21.15.7:0/1330368347 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "set_id": "76318bf4", "release": true, "await": true}) -- 0x5625c3e3c3d0 con 0x5625c3f69cb0 2024-06-10T01:15:13.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.304+0000 7f1fa4877700 1 --2- 172.21.15.7:0/1330368347 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5625c3f69cb0 0x5625c3f6c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:13.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.305+0000 7f1fa4877700 1 --2- 172.21.15.7:0/1330368347 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5625c3f69cb0 0x5625c3f6c0a0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3ce99cd32706a20 server_cookie=9e68b1ee5d52e879 in_seq=0 out_seq=0 2024-06-10T01:15:13.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.305+0000 7f1f7ffff700 10 client.17778 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:13.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.314+0000 7f1f7ffff700 1 -- 172.21.15.7:0/1330368347 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5625c3e3c3d0 con 0x5625c3f69cb0 2024-06-10T01:15:13.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.314+0000 7f1f7ffff700 10 client.17778 handle_command_reply: tid=0 2024-06-10T01:15:13.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.314+0000 7f1fb135eb80 1 client.17778 shutdown 2024-06-10T01:15:13.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.314+0000 7f1fb135eb80 1 -- 172.21.15.7:0/1330368347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625c3ee9500 msgr2=0x5625c3ef3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:13.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.314+0000 7f1fb135eb80 1 --2- 172.21.15.7:0/1330368347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625c3ee9500 0x5625c3ef3520 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400b910 tx=0x7f1f9400b620 comp rx=0 tx=0).stop 2024-06-10T01:15:13.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.314+0000 7f1fb135eb80 1 -- 172.21.15.7:0/1330368347 shutdown_connections 2024-06-10T01:15:13.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.314+0000 7f1fb135eb80 1 --2- 172.21.15.7:0/1330368347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625c3ee9e20 0x5625c3f68200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.314+0000 7f1fb135eb80 1 --2- 172.21.15.7:0/1330368347 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5625c3f69cb0 0x5625c3f6c0a0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.314+0000 7f1fb135eb80 1 --2- 172.21.15.7:0/1330368347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625c3ee3c50 0x5625c3ef2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.314+0000 7f1fb135eb80 1 --2- 172.21.15.7:0/1330368347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625c3ee9500 0x5625c3ef3520 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.314+0000 7f1fb135eb80 1 -- 172.21.15.7:0/1330368347 >> 172.21.15.7:0/1330368347 conn(0x5625c3e10180 msgr2=0x5625c3eeb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:13.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.314+0000 7f1f7ffff700 0 client.17778 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:13.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.314+0000 7f1fb135eb80 1 -- 172.21.15.7:0/1330368347 shutdown_connections 2024-06-10T01:15:13.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.314+0000 7f1fb135eb80 1 -- 172.21.15.7:0/1330368347 wait complete. 2024-06-10T01:15:13.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.314+0000 7f1fb135eb80 20 client.17778 trim_cache size 0 max 16384 2024-06-10T01:15:13.310 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:15:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 36, 2024-06-10T01:15:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:15:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 28, 2024-06-10T01:15:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:15:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "76318bf4": { 2024-06-10T01:15:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 28, 2024-06-10T01:15:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:15:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:15:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:15:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:15:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:15:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:15:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 137.8, 2024-06-10T01:15:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:15:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:15:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:15:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:15:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:15:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:15:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:15:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:15:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:15:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:15:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:15:13.312 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:15:13.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.315+0000 7f1fa6adb700 1 -- 172.21.15.7:0/2204445237 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1f880a2c20 msgr2=0x7f1f880a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:13.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.315+0000 7f1fa6adb700 1 --2- 172.21.15.7:0/2204445237 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1f880a2c20 0x7f1f880a50e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0107310 tx=0x7f1f90009260 comp rx=0 tx=0).stop 2024-06-10T01:15:13.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.315+0000 7f1fa6adb700 1 -- 172.21.15.7:0/2204445237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fa0107860 msgr2=0x7f1fa0198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:13.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.315+0000 7f1fa6adb700 1 --2- 172.21.15.7:0/2204445237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fa0107860 0x7f1fa0198ab0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800ada0 tx=0x7f1f9800c330 comp rx=0 tx=0).stop 2024-06-10T01:15:13.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.315+0000 7f1fa6adb700 1 -- 172.21.15.7:0/2204445237 shutdown_connections 2024-06-10T01:15:13.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.315+0000 7f1fa6adb700 1 --2- 172.21.15.7:0/2204445237 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1f880a2c20 0x7f1f880a50e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.315+0000 7f1fa6adb700 1 --2- 172.21.15.7:0/2204445237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fa0107860 0x7f1fa0198ab0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.315+0000 7f1fa6adb700 1 --2- 172.21.15.7:0/2204445237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fa006adc0 0x7f1fa019e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.315+0000 7f1fa6adb700 1 --2- 172.21.15.7:0/2204445237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fa006b770 0x7f1fa019ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:13.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.315+0000 7f1fa6adb700 1 -- 172.21.15.7:0/2204445237 >> 172.21.15.7:0/2204445237 conn(0x7f1fa0078340 msgr2=0x7f1fa00ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:13.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.315+0000 7f1fa6adb700 1 -- 172.21.15.7:0/2204445237 shutdown_connections 2024-06-10T01:15:13.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:13.315+0000 7f1fa6adb700 1 -- 172.21.15.7:0/2204445237 wait complete. 2024-06-10T01:15:13.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:13 smithi175 ceph-mon[36636]: pgmap v708: 129 pgs: 129 active+clean; 114 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1.4 MiB/s wr, 82 op/s 2024-06-10T01:15:13.744 INFO:tasks.workunit.client.0.smithi007.stdout: 1 665237 7.52 MB/sec execute 354 sec latency 18065.011 ms 2024-06-10T01:15:13.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:13 smithi007 ceph-mon[32398]: pgmap v708: 129 pgs: 129 active+clean; 114 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1.4 MiB/s wr, 82 op/s 2024-06-10T01:15:13.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:13 smithi078 ceph-mon[36683]: pgmap v708: 129 pgs: 129 active+clean; 114 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1.4 MiB/s wr, 82 op/s 2024-06-10T01:15:14.323 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '76318bf4', total seconds elapsed: 24.1 2024-06-10T01:15:14.323 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.8 2024-06-10T01:15:14.324 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:15:14.609 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:15:14.744 INFO:tasks.workunit.client.0.smithi007.stdout: 1 667630 7.54 MB/sec execute 355 sec latency 20.129 ms 2024-06-10T01:15:15.744 INFO:tasks.workunit.client.0.smithi007.stdout: 1 669047 7.54 MB/sec execute 356 sec latency 16.748 ms 2024-06-10T01:15:15.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:15 smithi007 ceph-mon[32398]: pgmap v709: 129 pgs: 129 active+clean; 116 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1.7 MiB/s wr, 87 op/s 2024-06-10T01:15:15.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:15 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:15:15.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:15 smithi078 ceph-mon[36683]: pgmap v709: 129 pgs: 129 active+clean; 116 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1.7 MiB/s wr, 87 op/s 2024-06-10T01:15:15.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:15 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:15:15.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:15 smithi175 ceph-mon[36636]: pgmap v709: 129 pgs: 129 active+clean; 116 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1.7 MiB/s wr, 87 op/s 2024-06-10T01:15:15.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:15 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:15:16.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.517+0000 7f99e6b73700 1 -- 172.21.15.7:0/1315164979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99e0108500 msgr2=0x7f99e01088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:16.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.517+0000 7f99e6b73700 1 --2- 172.21.15.7:0/1315164979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99e0108500 0x7f99e01088e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f99d4009990 tx=0x7f99d4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:15:16.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.517+0000 7f99e6b73700 1 -- 172.21.15.7:0/1315164979 shutdown_connections 2024-06-10T01:15:16.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.517+0000 7f99e6b73700 1 --2- 172.21.15.7:0/1315164979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99e006ab30 0x7f99e010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:16.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.517+0000 7f99e6b73700 1 --2- 172.21.15.7:0/1315164979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99e0108500 0x7f99e01088e0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:16.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.517+0000 7f99e6b73700 1 --2- 172.21.15.7:0/1315164979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99e006a170 0x7f99e006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:16.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.517+0000 7f99e6b73700 1 -- 172.21.15.7:0/1315164979 >> 172.21.15.7:0/1315164979 conn(0x7f99e0078340 msgr2=0x7f99e0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:16.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.517+0000 7f99e6b73700 1 -- 172.21.15.7:0/1315164979 shutdown_connections 2024-06-10T01:15:16.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.517+0000 7f99e6b73700 1 -- 172.21.15.7:0/1315164979 wait complete. 2024-06-10T01:15:16.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.518+0000 7f99e6b73700 1 Processor -- start 2024-06-10T01:15:16.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.518+0000 7f99e6b73700 1 -- start start 2024-06-10T01:15:16.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.518+0000 7f99e6b73700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99e006a170 0x7f99e0101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:16.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.518+0000 7f99e6b73700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99e006ab30 0x7f99e01019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:16.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.518+0000 7f99e6b73700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99e0108500 0x7f99e0101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:16.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.518+0000 7f99e6b73700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f99e0112780 con 0x7f99e006a170 2024-06-10T01:15:16.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.518+0000 7f99e6b73700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f99e0112600 con 0x7f99e0108500 2024-06-10T01:15:16.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.518+0000 7f99e6b73700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f99e0112900 con 0x7f99e006ab30 2024-06-10T01:15:16.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.518+0000 7f99dffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99e006ab30 0x7f99e01019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:16.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.518+0000 7f99dffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99e006ab30 0x7f99e01019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:59718/0 (socket says 172.21.15.7:59718) 2024-06-10T01:15:16.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.518+0000 7f99dffff700 1 -- 172.21.15.7:0/3022973765 learned_addr learned my addr 172.21.15.7:0/3022973765 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:16.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.518+0000 7f99e490f700 1 --2- 172.21.15.7:0/3022973765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99e006a170 0x7f99e0101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:16.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.519+0000 7f99e5110700 1 --2- 172.21.15.7:0/3022973765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99e0108500 0x7f99e0101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:16.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.519+0000 7f99dffff700 1 -- 172.21.15.7:0/3022973765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99e0108500 msgr2=0x7f99e0101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:16.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.519+0000 7f99dffff700 1 --2- 172.21.15.7:0/3022973765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99e0108500 0x7f99e0101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:16.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.519+0000 7f99dffff700 1 -- 172.21.15.7:0/3022973765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99e006a170 msgr2=0x7f99e0101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:16.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.519+0000 7f99dffff700 1 --2- 172.21.15.7:0/3022973765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99e006a170 0x7f99e0101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:16.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.519+0000 7f99dffff700 1 -- 172.21.15.7:0/3022973765 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e01a71e0 con 0x7f99e006ab30 2024-06-10T01:15:16.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.519+0000 7f99dffff700 1 --2- 172.21.15.7:0/3022973765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99e006ab30 0x7f99e01019a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00e9c0 tx=0x7f99cc00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:16.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.519+0000 7f99ddffb700 1 -- 172.21.15.7:0/3022973765 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc00cbd0 con 0x7f99e006ab30 2024-06-10T01:15:16.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.519+0000 7f99ddffb700 1 -- 172.21.15.7:0/3022973765 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99cc00ee60 con 0x7f99e006ab30 2024-06-10T01:15:16.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.519+0000 7f99ddffb700 1 -- 172.21.15.7:0/3022973765 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc005630 con 0x7f99e006ab30 2024-06-10T01:15:16.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.520+0000 7f99e6b73700 1 -- 172.21.15.7:0/3022973765 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99e01a74d0 con 0x7f99e006ab30 2024-06-10T01:15:16.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.520+0000 7f99e6b73700 1 -- 172.21.15.7:0/3022973765 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99e01a79c0 con 0x7f99e006ab30 2024-06-10T01:15:16.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.521+0000 7f99e6b73700 1 -- 172.21.15.7:0/3022973765 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99e010a9e0 con 0x7f99e006ab30 2024-06-10T01:15:16.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.522+0000 7f99ddffb700 1 -- 172.21.15.7:0/3022973765 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f99cc004750 con 0x7f99e006ab30 2024-06-10T01:15:16.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.523+0000 7f99ddffb700 1 --2- 172.21.15.7:0/3022973765 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f99d00a2ca0 0x7f99d00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:16.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.523+0000 7f99e490f700 1 --2- 172.21.15.7:0/3022973765 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f99d00a2ca0 0x7f99d00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:16.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.523+0000 7f99ddffb700 1 -- 172.21.15.7:0/3022973765 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f99cc0cf810 con 0x7f99e006ab30 2024-06-10T01:15:16.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.523+0000 7f99e490f700 1 --2- 172.21.15.7:0/3022973765 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f99d00a2ca0 0x7f99d00a5160 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f99d400b2e0 tx=0x7f99d4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:16.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.527+0000 7f99ddffb700 1 -- 172.21.15.7:0/3022973765 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f99cc053850 con 0x7f99e006ab30 2024-06-10T01:15:16.744 INFO:tasks.workunit.client.0.smithi007.stdout: 1 671304 7.55 MB/sec execute 357 sec latency 23.467 ms 2024-06-10T01:15:16.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.795+0000 7f99e6b73700 1 -- 172.21.15.7:0/3022973765 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99e0069180 con 0x7f99e006ab30 2024-06-10T01:15:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.796+0000 7f99ddffb700 1 -- 172.21.15.7:0/3022973765 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11859 (secure 0 0 0) 0x7f99cc0b3e70 con 0x7f99e006ab30 2024-06-10T01:15:16.790 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:15:16.791 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:15:13.790052+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24451,24434,24436]},"id":1}]} 2024-06-10T01:15:16.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.799+0000 7f99e6b73700 1 -- 172.21.15.7:0/3022973765 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f99d00a2ca0 msgr2=0x7f99d00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:16.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.799+0000 7f99e6b73700 1 --2- 172.21.15.7:0/3022973765 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f99d00a2ca0 0x7f99d00a5160 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f99d400b2e0 tx=0x7f99d4005960 comp rx=0 tx=0).stop 2024-06-10T01:15:16.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.799+0000 7f99e6b73700 1 -- 172.21.15.7:0/3022973765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99e006ab30 msgr2=0x7f99e01019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:16.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.799+0000 7f99e6b73700 1 --2- 172.21.15.7:0/3022973765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99e006ab30 0x7f99e01019a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00e9c0 tx=0x7f99cc00ecd0 comp rx=0 tx=0).stop 2024-06-10T01:15:16.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.799+0000 7f99e6b73700 1 -- 172.21.15.7:0/3022973765 shutdown_connections 2024-06-10T01:15:16.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.799+0000 7f99e6b73700 1 --2- 172.21.15.7:0/3022973765 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f99d00a2ca0 0x7f99d00a5160 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:16.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.799+0000 7f99e6b73700 1 --2- 172.21.15.7:0/3022973765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99e0108500 0x7f99e0101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:16.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.799+0000 7f99e6b73700 1 --2- 172.21.15.7:0/3022973765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99e006a170 0x7f99e0101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:16.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.799+0000 7f99e6b73700 1 --2- 172.21.15.7:0/3022973765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99e006ab30 0x7f99e01019a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:16.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.799+0000 7f99e6b73700 1 -- 172.21.15.7:0/3022973765 >> 172.21.15.7:0/3022973765 conn(0x7f99e0078340 msgr2=0x7f99e01000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:16.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.799+0000 7f99e6b73700 1 -- 172.21.15.7:0/3022973765 shutdown_connections 2024-06-10T01:15:16.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:16.799+0000 7f99e6b73700 1 -- 172.21.15.7:0/3022973765 wait complete. 2024-06-10T01:15:16.794 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 37 2024-06-10T01:15:17.555 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db / --timeout 90 --expiration 137.8 --await-for 45.0' 2024-06-10T01:15:17.555 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db / --timeout 90 --expiration 137.8 --await-for 45.0 2024-06-10T01:15:17.744 INFO:tasks.workunit.client.0.smithi007.stdout: 1 672617 7.55 MB/sec execute 358 sec latency 17.417 ms 2024-06-10T01:15:17.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:17 smithi007 ceph-mon[32398]: pgmap v710: 129 pgs: 129 active+clean; 115 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 1.8 MiB/s wr, 91 op/s 2024-06-10T01:15:17.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:17 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3022973765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:15:17.817 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:15:17.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:17 smithi078 ceph-mon[36683]: pgmap v710: 129 pgs: 129 active+clean; 115 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 1.8 MiB/s wr, 91 op/s 2024-06-10T01:15:17.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:17 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3022973765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:15:17.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:17 smithi175 ceph-mon[36636]: pgmap v710: 129 pgs: 129 active+clean; 115 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 1.8 MiB/s wr, 91 op/s 2024-06-10T01:15:17.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:17 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3022973765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:15:18.744 INFO:tasks.workunit.client.0.smithi007.stdout: 1 674769 7.56 MB/sec execute 359 sec latency 65.190 ms 2024-06-10T01:15:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.147+0000 7fb19784c640 1 -- 172.21.15.7:0/3417842656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb190101a10 msgr2=0x7fb190101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.147+0000 7fb19784c640 1 --2- 172.21.15.7:0/3417842656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb190101a10 0x7fb190101e30 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00fb10 tx=0x7fb18c02a160 comp rx=0 tx=0).stop 2024-06-10T01:15:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.148+0000 7fb19784c640 1 -- 172.21.15.7:0/3417842656 shutdown_connections 2024-06-10T01:15:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.148+0000 7fb19784c640 1 --2- 172.21.15.7:0/3417842656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1901035c0 0x7fb190109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.148+0000 7fb19784c640 1 --2- 172.21.15.7:0/3417842656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb190102c00 0x7fb190103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.148+0000 7fb19784c640 1 --2- 172.21.15.7:0/3417842656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb190101a10 0x7fb190101e30 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.148+0000 7fb19784c640 1 -- 172.21.15.7:0/3417842656 >> 172.21.15.7:0/3417842656 conn(0x7fb1900fcf80 msgr2=0x7fb1900ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.148+0000 7fb19784c640 1 -- 172.21.15.7:0/3417842656 shutdown_connections 2024-06-10T01:15:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.148+0000 7fb19784c640 1 -- 172.21.15.7:0/3417842656 wait complete. 2024-06-10T01:15:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.148+0000 7fb19784c640 1 Processor -- start 2024-06-10T01:15:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.148+0000 7fb19784c640 1 -- start start 2024-06-10T01:15:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.148+0000 7fb19784c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb190101a10 0x7fb190195b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.148+0000 7fb19784c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb190102c00 0x7fb190196060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.148+0000 7fb19784c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1901035c0 0x7fb19019d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.148+0000 7fb19784c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb19010b4b0 con 0x7fb190101a10 2024-06-10T01:15:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.149+0000 7fb19784c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb19010b330 con 0x7fb1901035c0 2024-06-10T01:15:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.149+0000 7fb19784c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb19010b630 con 0x7fb190102c00 2024-06-10T01:15:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.149+0000 7fb1955e8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb190101a10 0x7fb190195b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.149+0000 7fb1955e8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb190101a10 0x7fb190195b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51020/0 (socket says 172.21.15.7:51020) 2024-06-10T01:15:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.149+0000 7fb1955e8640 1 -- 172.21.15.7:0/3742537646 learned_addr learned my addr 172.21.15.7:0/3742537646 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.149+0000 7fb194de7640 1 --2- 172.21.15.7:0/3742537646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb190102c00 0x7fb190196060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.149+0000 7fb195de9640 1 --2- 172.21.15.7:0/3742537646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1901035c0 0x7fb19019d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.149+0000 7fb1955e8640 1 -- 172.21.15.7:0/3742537646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb190102c00 msgr2=0x7fb190196060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.149+0000 7fb1955e8640 1 --2- 172.21.15.7:0/3742537646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb190102c00 0x7fb190196060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.149+0000 7fb1955e8640 1 -- 172.21.15.7:0/3742537646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1901035c0 msgr2=0x7fb19019d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.149+0000 7fb1955e8640 1 --2- 172.21.15.7:0/3742537646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1901035c0 0x7fb19019d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.149+0000 7fb1955e8640 1 -- 172.21.15.7:0/3742537646 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb19019d740 con 0x7fb190101a10 2024-06-10T01:15:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.149+0000 7fb1955e8640 1 --2- 172.21.15.7:0/3742537646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb190101a10 0x7fb190195b20 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fb18c000c00 tx=0x7fb18c004960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.149+0000 7fb1867fc640 1 -- 172.21.15.7:0/3742537646 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c03f030 con 0x7fb190101a10 2024-06-10T01:15:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.149+0000 7fb19784c640 1 -- 172.21.15.7:0/3742537646 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb19019d9d0 con 0x7fb190101a10 2024-06-10T01:15:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.149+0000 7fb1867fc640 1 -- 172.21.15.7:0/3742537646 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb18c02ad70 con 0x7fb190101a10 2024-06-10T01:15:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.149+0000 7fb1867fc640 1 -- 172.21.15.7:0/3742537646 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c0035f0 con 0x7fb190101a10 2024-06-10T01:15:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.149+0000 7fb19784c640 1 -- 172.21.15.7:0/3742537646 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb19019dec0 con 0x7fb190101a10 2024-06-10T01:15:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.152+0000 7fb1867fc640 1 -- 172.21.15.7:0/3742537646 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb18c042070 con 0x7fb190101a10 2024-06-10T01:15:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.152+0000 7fb19a630740 1 Processor -- start 2024-06-10T01:15:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.152+0000 7fb19a630740 1 -- start start 2024-06-10T01:15:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.152+0000 7fb19a630740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c80cff010 0x558c80cff430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.152+0000 7fb19a630740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c80cffa00 0x558c80d04500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.152+0000 7fb19a630740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c80d04b70 0x558c80d07000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.152+0000 7fb19a630740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558c80c624f0 con 0x558c80d04b70 2024-06-10T01:15:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.152+0000 7fb19a630740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558c80c00870 con 0x558c80cffa00 2024-06-10T01:15:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.152+0000 7fb19a630740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558c80c5e4b0 con 0x558c80cff010 2024-06-10T01:15:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.152+0000 7fb194de7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c80d04b70 0x558c80d07000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.152+0000 7fb194de7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c80d04b70 0x558c80d07000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51036/0 (socket says 172.21.15.7:51036) 2024-06-10T01:15:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.152+0000 7fb194de7640 1 -- 172.21.15.7:0/840065418 learned_addr learned my addr 172.21.15.7:0/840065418 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.153+0000 7fb1955e8640 1 --2- 172.21.15.7:0/840065418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c80cffa00 0x558c80d04500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.153+0000 7fb195de9640 1 --2- 172.21.15.7:0/840065418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c80cff010 0x558c80cff430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.153+0000 7fb1867fc640 1 --2- 172.21.15.7:0/3742537646 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb17c0a2b80 0x7fb17c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.153+0000 7fb195de9640 1 --2- 172.21.15.7:0/3742537646 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb17c0a2b80 0x7fb17c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.153+0000 7fb194de7640 1 -- 172.21.15.7:0/840065418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c80cff010 msgr2=0x558c80cff430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.153+0000 7fb194de7640 1 --2- 172.21.15.7:0/840065418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c80cff010 0x558c80cff430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.153+0000 7fb194de7640 1 -- 172.21.15.7:0/840065418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c80cffa00 msgr2=0x558c80d04500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.153+0000 7fb194de7640 1 --2- 172.21.15.7:0/840065418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c80cffa00 0x558c80d04500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.153+0000 7fb194de7640 1 -- 172.21.15.7:0/840065418 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c80c1c9b0 con 0x558c80d04b70 2024-06-10T01:15:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.153+0000 7fb1867fc640 1 -- 172.21.15.7:0/3742537646 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb18c0ee470 con 0x7fb190101a10 2024-06-10T01:15:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.153+0000 7fb195de9640 1 --2- 172.21.15.7:0/840065418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c80cff010 0x558c80cff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.153+0000 7fb194de7640 1 --2- 172.21.15.7:0/840065418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c80d04b70 0x558c80d07000 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fb1800043f0 tx=0x7fb1800039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840c488074ab16e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.153+0000 7fb195de9640 1 --2- 172.21.15.7:0/3742537646 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb17c0a2b80 0x7fb17c0a5040 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x558c80d00260 tx=0x7fb188004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.153+0000 7fb17b7fe640 1 -- 172.21.15.7:0/840065418 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1800096f0 con 0x558c80d04b70 2024-06-10T01:15:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.153+0000 7fb17b7fe640 1 -- 172.21.15.7:0/840065418 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb180024070 con 0x558c80d04b70 2024-06-10T01:15:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.153+0000 7fb17b7fe640 1 -- 172.21.15.7:0/840065418 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18001fd50 con 0x558c80d04b70 2024-06-10T01:15:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.154+0000 7fb19a630740 1 -- 172.21.15.7:0/840065418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c80d04b70 msgr2=0x558c80d07000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.154+0000 7fb19a630740 1 --2- 172.21.15.7:0/840065418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c80d04b70 0x558c80d07000 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fb1800043f0 tx=0x7fb1800039c0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.154+0000 7fb19a630740 1 -- 172.21.15.7:0/840065418 shutdown_connections 2024-06-10T01:15:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.154+0000 7fb19a630740 1 --2- 172.21.15.7:0/840065418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c80d04b70 0x558c80d07000 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.154+0000 7fb19a630740 1 --2- 172.21.15.7:0/840065418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c80cffa00 0x558c80d04500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.154+0000 7fb19a630740 1 --2- 172.21.15.7:0/840065418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c80cff010 0x558c80cff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.154+0000 7fb19a630740 1 -- 172.21.15.7:0/840065418 >> 172.21.15.7:0/840065418 conn(0x558c80cfa680 msgr2=0x558c80cfaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.154+0000 7fb19a630740 1 -- 172.21.15.7:0/840065418 shutdown_connections 2024-06-10T01:15:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.154+0000 7fb19a630740 1 -- 172.21.15.7:0/840065418 wait complete. 2024-06-10T01:15:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.154+0000 7fb19a630740 1 -- start start 2024-06-10T01:15:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.155+0000 7fb19a630740 1 Processor -- start 2024-06-10T01:15:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.155+0000 7fb19a630740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c80cff010 0x558c80d091f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.155+0000 7fb19a630740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c80cffa00 0x558c80d09730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.155+0000 7fb19a630740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c80d04b70 0x558c80d10770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.155+0000 7fb19a630740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558c80d07b20 con 0x558c80d04b70 2024-06-10T01:15:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.155+0000 7fb19a630740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558c80bebae0 con 0x558c80cffa00 2024-06-10T01:15:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.155+0000 7fb19a630740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558c80d07ca0 con 0x558c80cff010 2024-06-10T01:15:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.155+0000 7fb19a630740 10 client.0 _ll_register_callbacks cb 0x558c80c249b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:15:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.155+0000 7fb195de9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c80d04b70 0x558c80d10770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.155+0000 7fb1955e8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c80cff010 0x558c80d091f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.155+0000 7fb195de9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c80d04b70 0x558c80d10770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51048/0 (socket says 172.21.15.7:51048) 2024-06-10T01:15:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.155+0000 7fb195de9640 1 -- 172.21.15.7:0/1119132592 learned_addr learned my addr 172.21.15.7:0/1119132592 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.155+0000 7fb194de7640 1 --2- 172.21.15.7:0/1119132592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c80cffa00 0x558c80d09730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.155+0000 7fb195de9640 1 -- 172.21.15.7:0/1119132592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c80cff010 msgr2=0x558c80d091f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.155+0000 7fb195de9640 1 --2- 172.21.15.7:0/1119132592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c80cff010 0x558c80d091f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.155+0000 7fb195de9640 1 -- 172.21.15.7:0/1119132592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c80cffa00 msgr2=0x558c80d09730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.155+0000 7fb195de9640 1 --2- 172.21.15.7:0/1119132592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c80cffa00 0x558c80d09730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.155+0000 7fb195de9640 1 -- 172.21.15.7:0/1119132592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c80976210 con 0x558c80d04b70 2024-06-10T01:15:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.155+0000 7fb195de9640 1 --2- 172.21.15.7:0/1119132592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c80d04b70 0x558c80d10770 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fb18800fcb0 tx=0x7fb188004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8109e0bb5bbe60 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.155+0000 7fb178ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:15:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.156+0000 7fb178ff9640 1 -- 172.21.15.7:0/1119132592 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb188025070 con 0x558c80d04b70 2024-06-10T01:15:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.156+0000 7fb178ff9640 1 -- 172.21.15.7:0/1119132592 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb1880047d0 con 0x558c80d04b70 2024-06-10T01:15:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.156+0000 7fb19a630740 1 -- 172.21.15.7:0/1119132592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb180024070 con 0x558c80d04b70 2024-06-10T01:15:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.156+0000 7fb178ff9640 1 -- 172.21.15.7:0/1119132592 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb188031050 con 0x558c80d04b70 2024-06-10T01:15:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.156+0000 7fb19a630740 10 client.17796 fetch_fsmap learned FSMap version 37 2024-06-10T01:15:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.156+0000 7fb19a630740 1 -- 172.21.15.7:0/1119132592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558c80a42620 con 0x558c80d04b70 2024-06-10T01:15:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.156+0000 7fb178ff9640 1 -- 172.21.15.7:0/1119132592 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb180024070 con 0x558c80d04b70 2024-06-10T01:15:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.156+0000 7fb178ff9640 1 -- 172.21.15.7:0/1119132592 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7fb18801f040 con 0x558c80d04b70 2024-06-10T01:15:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.156+0000 7fb19a630740 10 client.17796 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T01:15:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.156+0000 7fb19a630740 10 client.17796 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:15:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.156+0000 7fb19a630740 20 client.17796 populate_metadata read hostname 'smithi007' 2024-06-10T01:15:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.156+0000 7fb19a630740 1 --2- 172.21.15.7:0/1119132592 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c80d151e0 0x558c80d176a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.156+0000 7fb19a630740 4 client.17796 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:15:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.156+0000 7fb19a630740 1 -- 172.21.15.7:0/1119132592 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c80c75f10 con 0x558c80d151e0 2024-06-10T01:15:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.157+0000 7fb1955e8640 1 --2- 172.21.15.7:0/1119132592 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c80d151e0 0x558c80d176a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.157+0000 7fb1955e8640 1 --2- 172.21.15.7:0/1119132592 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c80d151e0 0x558c80d176a0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fd217a774d75846 server_cookie=49fd432d7c970625 in_seq=0 out_seq=0 2024-06-10T01:15:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.157+0000 7fb178ff9640 10 client.17796 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.158+0000 7fb178ff9640 1 -- 172.21.15.7:0/1119132592 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558c80c75f10 con 0x558c80d151e0 2024-06-10T01:15:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.158+0000 7fb178ff9640 10 client.17796 handle_command_reply: tid=0 2024-06-10T01:15:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.158+0000 7fb19a630740 1 client.17796 shutdown 2024-06-10T01:15:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.158+0000 7fb19a630740 1 -- 172.21.15.7:0/1119132592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c80d04b70 msgr2=0x558c80d10770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.158+0000 7fb19a630740 1 --2- 172.21.15.7:0/1119132592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c80d04b70 0x558c80d10770 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fb18800fcb0 tx=0x7fb188004a40 comp rx=0 tx=0).stop 2024-06-10T01:15:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.158+0000 7fb19a630740 1 -- 172.21.15.7:0/1119132592 shutdown_connections 2024-06-10T01:15:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.158+0000 7fb19a630740 1 --2- 172.21.15.7:0/1119132592 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c80d151e0 0x558c80d176a0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.158+0000 7fb19a630740 1 --2- 172.21.15.7:0/1119132592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c80d04b70 0x558c80d10770 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.158+0000 7fb19a630740 1 --2- 172.21.15.7:0/1119132592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c80cffa00 0x558c80d09730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.158+0000 7fb178ff9640 0 client.17796 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.158+0000 7fb19a630740 1 --2- 172.21.15.7:0/1119132592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c80cff010 0x558c80d091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.158+0000 7fb19a630740 1 -- 172.21.15.7:0/1119132592 >> 172.21.15.7:0/1119132592 conn(0x558c80cfa680 msgr2=0x558c80cf97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.159+0000 7fb19a630740 1 -- 172.21.15.7:0/1119132592 shutdown_connections 2024-06-10T01:15:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.159+0000 7fb19a630740 1 -- 172.21.15.7:0/1119132592 wait complete. 2024-06-10T01:15:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.159+0000 7fb19a630740 20 client.17796 trim_cache size 0 max 16384 2024-06-10T01:15:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.174+0000 7fb19a630740 1 Processor -- start 2024-06-10T01:15:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.174+0000 7fb19a630740 1 -- start start 2024-06-10T01:15:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.174+0000 7fb19a630740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c80cff010 0x558c80d04220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.174+0000 7fb19a630740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c80cffa00 0x558c80d10730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.174+0000 7fb19a630740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c80d04b70 0x558c80d00be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.174+0000 7fb19a630740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb1901aacf0 con 0x558c80d04b70 2024-06-10T01:15:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.174+0000 7fb19a630740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb1901a8530 con 0x558c80cff010 2024-06-10T01:15:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.174+0000 7fb19a630740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb1901a6380 con 0x558c80cffa00 2024-06-10T01:15:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.174+0000 7fb195de9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c80d04b70 0x558c80d00be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.174+0000 7fb195de9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c80d04b70 0x558c80d00be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51060/0 (socket says 172.21.15.7:51060) 2024-06-10T01:15:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.174+0000 7fb195de9640 1 -- 172.21.15.7:0/4250076423 learned_addr learned my addr 172.21.15.7:0/4250076423 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.174+0000 7fb1955e8640 1 --2- 172.21.15.7:0/4250076423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c80cff010 0x558c80d04220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.174+0000 7fb194de7640 1 --2- 172.21.15.7:0/4250076423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c80cffa00 0x558c80d10730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.174+0000 7fb195de9640 1 -- 172.21.15.7:0/4250076423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c80cffa00 msgr2=0x558c80d10730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.174+0000 7fb195de9640 1 --2- 172.21.15.7:0/4250076423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c80cffa00 0x558c80d10730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.174+0000 7fb195de9640 1 -- 172.21.15.7:0/4250076423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c80cff010 msgr2=0x558c80d04220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.174+0000 7fb195de9640 1 --2- 172.21.15.7:0/4250076423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c80cff010 0x558c80d04220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.175+0000 7fb195de9640 1 -- 172.21.15.7:0/4250076423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c80cfa4b0 con 0x558c80d04b70 2024-06-10T01:15:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.175+0000 7fb195de9640 1 --2- 172.21.15.7:0/4250076423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c80d04b70 0x558c80d00be0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fb188005c40 tx=0x7fb188013430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8d967992ee4f49 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.175+0000 7fb17affd640 1 -- 172.21.15.7:0/4250076423 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb188025070 con 0x558c80d04b70 2024-06-10T01:15:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.175+0000 7fb17affd640 1 -- 172.21.15.7:0/4250076423 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb188004620 con 0x558c80d04b70 2024-06-10T01:15:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.175+0000 7fb17affd640 1 -- 172.21.15.7:0/4250076423 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1880217d0 con 0x558c80d04b70 2024-06-10T01:15:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.175+0000 7fb19a630740 1 -- 172.21.15.7:0/4250076423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c80d04b70 msgr2=0x558c80d00be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.175+0000 7fb19a630740 1 --2- 172.21.15.7:0/4250076423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c80d04b70 0x558c80d00be0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fb188005c40 tx=0x7fb188013430 comp rx=0 tx=0).stop 2024-06-10T01:15:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.175+0000 7fb19a630740 1 -- 172.21.15.7:0/4250076423 shutdown_connections 2024-06-10T01:15:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.175+0000 7fb19a630740 1 --2- 172.21.15.7:0/4250076423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c80d04b70 0x558c80d00be0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.175+0000 7fb19a630740 1 --2- 172.21.15.7:0/4250076423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c80cffa00 0x558c80d10730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.175+0000 7fb19a630740 1 --2- 172.21.15.7:0/4250076423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c80cff010 0x558c80d04220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.175+0000 7fb19a630740 1 -- 172.21.15.7:0/4250076423 >> 172.21.15.7:0/4250076423 conn(0x558c80cfa680 msgr2=0x558c80cfaa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.176+0000 7fb19a630740 1 -- 172.21.15.7:0/4250076423 shutdown_connections 2024-06-10T01:15:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.176+0000 7fb19a630740 1 -- 172.21.15.7:0/4250076423 wait complete. 2024-06-10T01:15:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.176+0000 7fb19a630740 1 -- start start 2024-06-10T01:15:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.176+0000 7fb19a630740 1 Processor -- start 2024-06-10T01:15:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.176+0000 7fb19a630740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c80cff010 0x558c80d0d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.176+0000 7fb19a630740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c80cffa00 0x558c80d0dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.176+0000 7fb19a630740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c80d04b70 0x558c80d7df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.176+0000 7fb19a630740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb1901a69f0 con 0x558c80cff010 2024-06-10T01:15:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.176+0000 7fb19a630740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb1901a7e60 con 0x558c80cffa00 2024-06-10T01:15:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.176+0000 7fb19a630740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb1901a70c0 con 0x558c80d04b70 2024-06-10T01:15:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.176+0000 7fb19a630740 10 client.0 _ll_register_callbacks cb 0x7fb1880136d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:15:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.176+0000 7fb1955e8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c80cff010 0x558c80d0d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.176+0000 7fb1955e8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c80cff010 0x558c80d0d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51068/0 (socket says 172.21.15.7:51068) 2024-06-10T01:15:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.176+0000 7fb1955e8640 1 -- 172.21.15.7:0/1743465378 learned_addr learned my addr 172.21.15.7:0/1743465378 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.176+0000 7fb194de7640 1 --2- 172.21.15.7:0/1743465378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c80cffa00 0x558c80d0dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.176+0000 7fb195de9640 1 --2- 172.21.15.7:0/1743465378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c80d04b70 0x558c80d7df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.177+0000 7fb1955e8640 1 -- 172.21.15.7:0/1743465378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c80d04b70 msgr2=0x558c80d7df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.177+0000 7fb1955e8640 1 --2- 172.21.15.7:0/1743465378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c80d04b70 0x558c80d7df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.177+0000 7fb1955e8640 1 -- 172.21.15.7:0/1743465378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c80cffa00 msgr2=0x558c80d0dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.177+0000 7fb1955e8640 1 --2- 172.21.15.7:0/1743465378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c80cffa00 0x558c80d0dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.177+0000 7fb1955e8640 1 -- 172.21.15.7:0/1743465378 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c80d006d0 con 0x558c80cff010 2024-06-10T01:15:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.177+0000 7fb194de7640 1 --2- 172.21.15.7:0/1743465378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c80cffa00 0x558c80d0dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.177+0000 7fb1955e8640 1 --2- 172.21.15.7:0/1743465378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c80cff010 0x558c80d0d660 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fb18c0ba8e0 tx=0x7fb18c005130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f86676c83c82581 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.177+0000 7fb178ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:15:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.177+0000 7fb178ff9640 1 -- 172.21.15.7:0/1743465378 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c0ba720 con 0x558c80cff010 2024-06-10T01:15:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.177+0000 7fb178ff9640 1 -- 172.21.15.7:0/1743465378 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb18c0ee190 con 0x558c80cff010 2024-06-10T01:15:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.177+0000 7fb19a630740 1 -- 172.21.15.7:0/1743465378 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb188004620 con 0x558c80cff010 2024-06-10T01:15:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.177+0000 7fb178ff9640 1 -- 172.21.15.7:0/1743465378 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb18c0ba2b0 con 0x558c80cff010 2024-06-10T01:15:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.177+0000 7fb19a630740 10 client.17802 fetch_fsmap learned FSMap version 37 2024-06-10T01:15:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.177+0000 7fb19a630740 1 -- 172.21.15.7:0/1743465378 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558c80d7f610 con 0x558c80cff010 2024-06-10T01:15:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.177+0000 7fb178ff9640 1 -- 172.21.15.7:0/1743465378 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb188004620 con 0x558c80cff010 2024-06-10T01:15:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.177+0000 7fb178ff9640 1 -- 172.21.15.7:0/1743465378 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7fb18c004d90 con 0x558c80cff010 2024-06-10T01:15:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.178+0000 7fb19a630740 10 client.17802 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T01:15:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.178+0000 7fb19a630740 10 client.17802 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:15:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.178+0000 7fb19a630740 20 client.17802 populate_metadata read hostname 'smithi007' 2024-06-10T01:15:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.178+0000 7fb19a630740 1 --2- 172.21.15.7:0/1743465378 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c80d7ff70 0x558c80d82390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.178+0000 7fb19a630740 4 client.17802 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:15:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.178+0000 7fb19a630740 1 -- 172.21.15.7:0/1743465378 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558c80c6fc20 con 0x558c80d7ff70 2024-06-10T01:15:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.178+0000 7fb194de7640 1 --2- 172.21.15.7:0/1743465378 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c80d7ff70 0x558c80d82390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.178+0000 7fb178ff9640 10 client.17802 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.178+0000 7fb194de7640 1 --2- 172.21.15.7:0/1743465378 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c80d7ff70 0x558c80d82390 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92f8a746397bd02e server_cookie=ff25db4fa02f2cfb in_seq=0 out_seq=0 2024-06-10T01:15:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.252+0000 7fb178ff9640 1 -- 172.21.15.7:0/1743465378 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558c80c6fc20 con 0x558c80d7ff70 2024-06-10T01:15:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.252+0000 7fb178ff9640 10 client.17802 handle_command_reply: tid=0 2024-06-10T01:15:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.252+0000 7fb19a630740 1 client.17802 shutdown 2024-06-10T01:15:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.253+0000 7fb19a630740 1 -- 172.21.15.7:0/1743465378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c80cff010 msgr2=0x558c80d0d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.253+0000 7fb19a630740 1 --2- 172.21.15.7:0/1743465378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c80cff010 0x558c80d0d660 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fb18c0ba8e0 tx=0x7fb18c005130 comp rx=0 tx=0).stop 2024-06-10T01:15:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.253+0000 7fb19a630740 1 -- 172.21.15.7:0/1743465378 shutdown_connections 2024-06-10T01:15:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.253+0000 7fb19a630740 1 --2- 172.21.15.7:0/1743465378 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c80d7ff70 0x558c80d82390 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.253+0000 7fb19a630740 1 --2- 172.21.15.7:0/1743465378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c80d04b70 0x558c80d7df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.253+0000 7fb19a630740 1 --2- 172.21.15.7:0/1743465378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c80cffa00 0x558c80d0dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.253+0000 7fb19a630740 1 --2- 172.21.15.7:0/1743465378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c80cff010 0x558c80d0d660 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.253+0000 7fb178ff9640 0 client.17802 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.253+0000 7fb19a630740 1 -- 172.21.15.7:0/1743465378 >> 172.21.15.7:0/1743465378 conn(0x558c80cfa680 msgr2=0x558c80d03df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.253+0000 7fb19a630740 1 -- 172.21.15.7:0/1743465378 shutdown_connections 2024-06-10T01:15:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.253+0000 7fb19a630740 1 -- 172.21.15.7:0/1743465378 wait complete. 2024-06-10T01:15:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.253+0000 7fb19a630740 20 client.17802 trim_cache size 0 max 16384 2024-06-10T01:15:19.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:15:19.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:15:19.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:15:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.253+0000 7fb19784c640 1 -- 172.21.15.7:0/3742537646 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb17c0a2b80 msgr2=0x7fb17c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.253+0000 7fb19784c640 1 --2- 172.21.15.7:0/3742537646 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb17c0a2b80 0x7fb17c0a5040 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x558c80d00260 tx=0x7fb188004ab0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.253+0000 7fb19784c640 1 -- 172.21.15.7:0/3742537646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb190101a10 msgr2=0x7fb190195b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.253+0000 7fb19784c640 1 --2- 172.21.15.7:0/3742537646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb190101a10 0x7fb190195b20 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fb18c000c00 tx=0x7fb18c004960 comp rx=0 tx=0).stop 2024-06-10T01:15:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.254+0000 7fb19784c640 1 -- 172.21.15.7:0/3742537646 shutdown_connections 2024-06-10T01:15:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.254+0000 7fb19784c640 1 --2- 172.21.15.7:0/3742537646 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb17c0a2b80 0x7fb17c0a5040 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.254+0000 7fb19784c640 1 --2- 172.21.15.7:0/3742537646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1901035c0 0x7fb19019d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.254+0000 7fb19784c640 1 --2- 172.21.15.7:0/3742537646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb190102c00 0x7fb190196060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.254+0000 7fb19784c640 1 --2- 172.21.15.7:0/3742537646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb190101a10 0x7fb190195b20 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.254+0000 7fb19784c640 1 -- 172.21.15.7:0/3742537646 >> 172.21.15.7:0/3742537646 conn(0x7fb1900fcf80 msgr2=0x7fb190105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.254+0000 7fb19784c640 1 -- 172.21.15.7:0/3742537646 shutdown_connections 2024-06-10T01:15:19.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:19.254+0000 7fb19784c640 1 -- 172.21.15.7:0/3742537646 wait complete. 2024-06-10T01:15:19.512 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:19 smithi007 ceph-mon[32398]: pgmap v711: 129 pgs: 129 active+clean; 114 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.7 MiB/s wr, 104 op/s 2024-06-10T01:15:19.744 INFO:tasks.workunit.client.0.smithi007.stdout: 1 675825 7.55 MB/sec execute 360 sec latency 62.428 ms 2024-06-10T01:15:19.754 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:19 smithi175 ceph-mon[36636]: pgmap v711: 129 pgs: 129 active+clean; 114 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.7 MiB/s wr, 104 op/s 2024-06-10T01:15:19.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:19 smithi078 ceph-mon[36683]: pgmap v711: 129 pgs: 129 active+clean; 114 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.7 MiB/s wr, 104 op/s 2024-06-10T01:15:19.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.853+0000 7f6fb48bd700 1 -- 172.21.15.7:0/2577197319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb0107430 msgr2=0x7f6fb0107810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:19.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.853+0000 7f6fb48bd700 1 --2- 172.21.15.7:0/2577197319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb0107430 0x7f6fb0107810 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0009990 tx=0x7f6fa0009ca0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.854+0000 7f6fb48bd700 1 -- 172.21.15.7:0/2577197319 shutdown_connections 2024-06-10T01:15:19.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.854+0000 7f6fb48bd700 1 --2- 172.21.15.7:0/2577197319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fb0102f80 0x7f6fb010f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.854+0000 7f6fb48bd700 1 --2- 172.21.15.7:0/2577197319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb0107430 0x7f6fb0107810 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.854+0000 7f6fb48bd700 1 --2- 172.21.15.7:0/2577197319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fb01025e0 0x7f6fb0102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.854+0000 7f6fb48bd700 1 -- 172.21.15.7:0/2577197319 >> 172.21.15.7:0/2577197319 conn(0x7f6fb0079680 msgr2=0x7f6fb0079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:19.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.854+0000 7f6fb48bd700 1 -- 172.21.15.7:0/2577197319 shutdown_connections 2024-06-10T01:15:19.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.854+0000 7f6fb48bd700 1 -- 172.21.15.7:0/2577197319 wait complete. 2024-06-10T01:15:19.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.854+0000 7f6fb48bd700 1 Processor -- start 2024-06-10T01:15:19.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.854+0000 7f6fb48bd700 1 -- start start 2024-06-10T01:15:19.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.855+0000 7f6fb48bd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fb01025e0 0x7f6fb019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.855+0000 7f6fb48bd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb0102f80 0x7f6fb019ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.855+0000 7f6fb48bd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fb0107430 0x7f6fb0198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.855+0000 7f6fb48bd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb01127a0 con 0x7f6fb0102f80 2024-06-10T01:15:19.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.855+0000 7f6fb48bd700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb0112620 con 0x7f6fb0107430 2024-06-10T01:15:19.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.855+0000 7f6fb48bd700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb0112920 con 0x7f6fb01025e0 2024-06-10T01:15:19.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.855+0000 7f6fadd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb0102f80 0x7f6fb019ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.855+0000 7f6fae59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fb01025e0 0x7f6fb019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.855+0000 7f6fadd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb0102f80 0x7f6fb019ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51080/0 (socket says 172.21.15.7:51080) 2024-06-10T01:15:19.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.855+0000 7f6fae59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fb01025e0 0x7f6fb019e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:59792/0 (socket says 172.21.15.7:59792) 2024-06-10T01:15:19.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.855+0000 7f6fadd9b700 1 -- 172.21.15.7:0/387209237 learned_addr learned my addr 172.21.15.7:0/387209237 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:19.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.855+0000 7f6faed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fb0107430 0x7f6fb0198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.855+0000 7f6fadd9b700 1 -- 172.21.15.7:0/387209237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fb01025e0 msgr2=0x7f6fb019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:19.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.855+0000 7f6fadd9b700 1 --2- 172.21.15.7:0/387209237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fb01025e0 0x7f6fb019e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.855+0000 7f6fadd9b700 1 -- 172.21.15.7:0/387209237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fb0107430 msgr2=0x7f6fb0198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:19.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.855+0000 7f6fadd9b700 1 --2- 172.21.15.7:0/387209237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fb0107430 0x7f6fb0198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.855+0000 7f6fadd9b700 1 -- 172.21.15.7:0/387209237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb0199180 con 0x7f6fb0102f80 2024-06-10T01:15:19.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.855+0000 7f6fae59c700 1 --2- 172.21.15.7:0/387209237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fb01025e0 0x7f6fb019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:19.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.855+0000 7f6faed9d700 1 --2- 172.21.15.7:0/387209237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fb0107430 0x7f6fb0198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:19.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.856+0000 7f6fadd9b700 1 --2- 172.21.15.7:0/387209237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb0102f80 0x7f6fb019ed00 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400e9a0 tx=0x7f6fa400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:19.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.856+0000 7f6f9f7fe700 1 -- 172.21.15.7:0/387209237 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa400ca10 con 0x7f6fb0102f80 2024-06-10T01:15:19.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.856+0000 7f6f9f7fe700 1 -- 172.21.15.7:0/387209237 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fa400cbb0 con 0x7f6fb0102f80 2024-06-10T01:15:19.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.856+0000 7f6fb48bd700 1 -- 172.21.15.7:0/387209237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fb0199470 con 0x7f6fb0102f80 2024-06-10T01:15:19.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.856+0000 7f6f9f7fe700 1 -- 172.21.15.7:0/387209237 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4018680 con 0x7f6fb0102f80 2024-06-10T01:15:19.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.856+0000 7f6fb48bd700 1 -- 172.21.15.7:0/387209237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fb01a5690 con 0x7f6fb0102f80 2024-06-10T01:15:19.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.858+0000 7f6f9f7fe700 1 -- 172.21.15.7:0/387209237 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f6fa4010430 con 0x7f6fb0102f80 2024-06-10T01:15:19.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.859+0000 7f6f9f7fe700 1 --2- 172.21.15.7:0/387209237 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6f980a2cc0 0x7f6f980a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.859+0000 7f6f9f7fe700 1 -- 172.21.15.7:0/387209237 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f6fa40cf5e0 con 0x7f6fb0102f80 2024-06-10T01:15:19.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.859+0000 7f6fae59c700 1 --2- 172.21.15.7:0/387209237 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6f980a2cc0 0x7f6f980a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.859+0000 7f6fae59c700 1 --2- 172.21.15.7:0/387209237 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6f980a2cc0 0x7f6f980a5180 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0006010 tx=0x7f6fa0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:19.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.860+0000 7f6fbf140b80 1 Processor -- start 2024-06-10T01:15:19.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.860+0000 7f6fbf140b80 1 -- start start 2024-06-10T01:15:19.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.860+0000 7f6fbf140b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5cc0dcd0 0x564d5cc0e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.860+0000 7f6fbf140b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d5cc0e5f0 0x564d5cc07f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.860+0000 7f6fbf140b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d5cc08460 0x564d5cc088a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.860+0000 7f6fbf140b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564d5cb599a0 con 0x564d5cc0e5f0 2024-06-10T01:15:19.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.860+0000 7f6fbf140b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564d5cb018b0 con 0x564d5cc0dcd0 2024-06-10T01:15:19.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.860+0000 7f6fbf140b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564d5cb55960 con 0x564d5cc08460 2024-06-10T01:15:19.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.860+0000 7f6fadd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d5cc0e5f0 0x564d5cc07f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.860+0000 7f6fadd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d5cc0e5f0 0x564d5cc07f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51094/0 (socket says 172.21.15.7:51094) 2024-06-10T01:15:19.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.860+0000 7f6fadd9b700 1 -- 172.21.15.7:0/2901456452 learned_addr learned my addr 172.21.15.7:0/2901456452 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:19.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.860+0000 7f6fae59c700 1 --2- 172.21.15.7:0/2901456452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5cc0dcd0 0x564d5cc0e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.860+0000 7f6faed9d700 1 --2- 172.21.15.7:0/2901456452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d5cc08460 0x564d5cc088a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.861+0000 7f6fadd9b700 1 -- 172.21.15.7:0/2901456452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d5cc08460 msgr2=0x564d5cc088a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:19.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.861+0000 7f6fadd9b700 1 --2- 172.21.15.7:0/2901456452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d5cc08460 0x564d5cc088a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.861+0000 7f6fadd9b700 1 -- 172.21.15.7:0/2901456452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5cc0dcd0 msgr2=0x564d5cc0e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:19.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.861+0000 7f6fadd9b700 1 --2- 172.21.15.7:0/2901456452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5cc0dcd0 0x564d5cc0e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.861+0000 7f6fadd9b700 1 -- 172.21.15.7:0/2901456452 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d5caf80e0 con 0x564d5cc0e5f0 2024-06-10T01:15:19.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.861+0000 7f6fae59c700 1 --2- 172.21.15.7:0/2901456452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5cc0dcd0 0x564d5cc0e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:19.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.861+0000 7f6fadd9b700 1 --2- 172.21.15.7:0/2901456452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d5cc0e5f0 0x564d5cc07f20 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4097f20 tx=0x7f6fa409b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7deef7412f9bc6db server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:19.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.861+0000 7f6f9cff9700 1 -- 172.21.15.7:0/2901456452 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa40a4730 con 0x564d5cc0e5f0 2024-06-10T01:15:19.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.861+0000 7f6f9cff9700 1 -- 172.21.15.7:0/2901456452 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fa400cd60 con 0x564d5cc0e5f0 2024-06-10T01:15:19.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.861+0000 7f6f9cff9700 1 -- 172.21.15.7:0/2901456452 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4010720 con 0x564d5cc0e5f0 2024-06-10T01:15:19.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.862+0000 7f6fbf140b80 1 -- 172.21.15.7:0/2901456452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d5cc0e5f0 msgr2=0x564d5cc07f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:19.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.862+0000 7f6fbf140b80 1 --2- 172.21.15.7:0/2901456452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d5cc0e5f0 0x564d5cc07f20 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4097f20 tx=0x7f6fa409b710 comp rx=0 tx=0).stop 2024-06-10T01:15:19.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.862+0000 7f6fbf140b80 1 -- 172.21.15.7:0/2901456452 shutdown_connections 2024-06-10T01:15:19.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.862+0000 7f6fbf140b80 1 --2- 172.21.15.7:0/2901456452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d5cc08460 0x564d5cc088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.862+0000 7f6fbf140b80 1 --2- 172.21.15.7:0/2901456452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5cc0dcd0 0x564d5cc0e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.862+0000 7f6fbf140b80 1 --2- 172.21.15.7:0/2901456452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d5cc0e5f0 0x564d5cc07f20 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.862+0000 7f6fbf140b80 1 -- 172.21.15.7:0/2901456452 >> 172.21.15.7:0/2901456452 conn(0x564d5cb32e50 msgr2=0x564d5cc031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:19.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.862+0000 7f6fbf140b80 1 -- 172.21.15.7:0/2901456452 shutdown_connections 2024-06-10T01:15:19.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.862+0000 7f6fbf140b80 1 -- 172.21.15.7:0/2901456452 wait complete. 2024-06-10T01:15:19.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.863+0000 7f6fbf140b80 1 -- start start 2024-06-10T01:15:19.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.863+0000 7f6fbf140b80 1 Processor -- start 2024-06-10T01:15:19.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.863+0000 7f6fbf140b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d5cc08460 0x564d5cc14360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.863+0000 7f6fbf140b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5cc0dcd0 0x564d5cc11660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.863+0000 7f6fbf140b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d5cc0e5f0 0x564d5cc11ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.863+0000 7f6fbf140b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564d5cc15000 con 0x564d5cc08460 2024-06-10T01:15:19.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.863+0000 7f6fbf140b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564d5caf9d80 con 0x564d5cc0dcd0 2024-06-10T01:15:19.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.863+0000 7f6fbf140b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564d5cc15180 con 0x564d5cc0e5f0 2024-06-10T01:15:19.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.863+0000 7f6fbf140b80 10 client.0 _ll_register_callbacks cb 0x564d5caf3ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:15:19.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.863+0000 7f6fae59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d5cc08460 0x564d5cc14360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.863+0000 7f6fae59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d5cc08460 0x564d5cc14360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51106/0 (socket says 172.21.15.7:51106) 2024-06-10T01:15:19.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.863+0000 7f6fae59c700 1 -- 172.21.15.7:0/444852848 learned_addr learned my addr 172.21.15.7:0/444852848 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:19.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.863+0000 7f6faed9d700 1 --2- 172.21.15.7:0/444852848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d5cc0e5f0 0x564d5cc11ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.863+0000 7f6fadd9b700 1 --2- 172.21.15.7:0/444852848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5cc0dcd0 0x564d5cc11660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.864+0000 7f6faed9d700 1 -- 172.21.15.7:0/444852848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5cc0dcd0 msgr2=0x564d5cc11660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:19.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.864+0000 7f6faed9d700 1 --2- 172.21.15.7:0/444852848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5cc0dcd0 0x564d5cc11660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.864+0000 7f6faed9d700 1 -- 172.21.15.7:0/444852848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d5cc08460 msgr2=0x564d5cc14360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:19.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.864+0000 7f6faed9d700 1 --2- 172.21.15.7:0/444852848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d5cc08460 0x564d5cc14360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.864+0000 7f6faed9d700 1 -- 172.21.15.7:0/444852848 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d5caa8a60 con 0x564d5cc0e5f0 2024-06-10T01:15:19.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.864+0000 7f6fae59c700 1 --2- 172.21.15.7:0/444852848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d5cc08460 0x564d5cc14360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:19.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.864+0000 7f6fadd9b700 1 --2- 172.21.15.7:0/444852848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5cc0dcd0 0x564d5cc11660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:19.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.864+0000 7f6faed9d700 1 --2- 172.21.15.7:0/444852848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d5cc0e5f0 0x564d5cc11ba0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8004660 tx=0x7f6fa8004970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd5cb9582fa9b31b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:19.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.864+0000 7f6f95ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:15:19.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.864+0000 7f6f95ffb700 1 -- 172.21.15.7:0/444852848 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa800ea20 con 0x564d5cc0e5f0 2024-06-10T01:15:19.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.864+0000 7f6fbf140b80 1 -- 172.21.15.7:0/444852848 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fa400cd60 con 0x564d5cc0e5f0 2024-06-10T01:15:19.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.864+0000 7f6f95ffb700 1 -- 172.21.15.7:0/444852848 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f6fa800ebc0 con 0x564d5cc0e5f0 2024-06-10T01:15:19.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.864+0000 7f6f95ffb700 1 -- 172.21.15.7:0/444852848 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6fa800edc0 con 0x564d5cc0e5f0 2024-06-10T01:15:19.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.864+0000 7f6fbf140b80 10 client.27232 fetch_fsmap learned FSMap version 37 2024-06-10T01:15:19.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.864+0000 7f6fbf140b80 1 -- 172.21.15.7:0/444852848 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564d5cc1f9b0 con 0x564d5cc0e5f0 2024-06-10T01:15:19.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.864+0000 7f6f95ffb700 1 -- 172.21.15.7:0/444852848 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fa400cd60 con 0x564d5cc0e5f0 2024-06-10T01:15:19.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.865+0000 7f6f95ffb700 1 -- 172.21.15.7:0/444852848 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f6fa801e490 con 0x564d5cc0e5f0 2024-06-10T01:15:19.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.865+0000 7f6fbf140b80 10 client.27232 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T01:15:19.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.865+0000 7f6fbf140b80 10 client.27232 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:15:19.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.865+0000 7f6fbf140b80 20 client.27232 populate_metadata read hostname 'smithi007' 2024-06-10T01:15:19.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.865+0000 7f6fbf140b80 1 --2- 172.21.15.7:0/444852848 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564d5cc21f40 0x564d5cc24370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.865+0000 7f6fbf140b80 4 client.27232 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:15:19.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.865+0000 7f6fbf140b80 1 -- 172.21.15.7:0/444852848 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d5cb66ea0 con 0x564d5cc21f40 2024-06-10T01:15:19.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.865+0000 7f6fae59c700 1 --2- 172.21.15.7:0/444852848 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564d5cc21f40 0x564d5cc24370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.865+0000 7f6fae59c700 1 --2- 172.21.15.7:0/444852848 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564d5cc21f40 0x564d5cc24370 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d5836c5bf889dfe server_cookie=515d62a34e7987d2 in_seq=0 out_seq=0 2024-06-10T01:15:19.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.865+0000 7f6f95ffb700 10 client.27232 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:19.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.866+0000 7f6f95ffb700 1 -- 172.21.15.7:0/444852848 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564d5cb66ea0 con 0x564d5cc21f40 2024-06-10T01:15:19.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.866+0000 7f6f95ffb700 10 client.27232 handle_command_reply: tid=0 2024-06-10T01:15:19.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.866+0000 7f6fbf140b80 1 client.27232 shutdown 2024-06-10T01:15:19.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.866+0000 7f6fbf140b80 1 -- 172.21.15.7:0/444852848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d5cc0e5f0 msgr2=0x564d5cc11ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:19.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.866+0000 7f6fbf140b80 1 --2- 172.21.15.7:0/444852848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d5cc0e5f0 0x564d5cc11ba0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8004660 tx=0x7f6fa8004970 comp rx=0 tx=0).stop 2024-06-10T01:15:19.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.866+0000 7f6fbf140b80 1 -- 172.21.15.7:0/444852848 shutdown_connections 2024-06-10T01:15:19.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.866+0000 7f6fbf140b80 1 --2- 172.21.15.7:0/444852848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d5cc0e5f0 0x564d5cc11ba0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.866+0000 7f6fbf140b80 1 --2- 172.21.15.7:0/444852848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d5cc08460 0x564d5cc14360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.866+0000 7f6fbf140b80 1 --2- 172.21.15.7:0/444852848 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564d5cc21f40 0x564d5cc24370 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.866+0000 7f6fbf140b80 1 --2- 172.21.15.7:0/444852848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5cc0dcd0 0x564d5cc11660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.866+0000 7f6f95ffb700 0 client.27232 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:19.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.866+0000 7f6fbf140b80 1 -- 172.21.15.7:0/444852848 >> 172.21.15.7:0/444852848 conn(0x564d5cb32e50 msgr2=0x564d5cc06f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:19.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.867+0000 7f6fbf140b80 1 -- 172.21.15.7:0/444852848 shutdown_connections 2024-06-10T01:15:19.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.867+0000 7f6fbf140b80 1 -- 172.21.15.7:0/444852848 wait complete. 2024-06-10T01:15:19.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.867+0000 7f6fbf140b80 20 client.27232 trim_cache size 0 max 16384 2024-06-10T01:15:19.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.890+0000 7f6fbf140b80 1 Processor -- start 2024-06-10T01:15:19.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.890+0000 7f6fbf140b80 1 -- start start 2024-06-10T01:15:19.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.890+0000 7f6fbf140b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5cc08460 0x564d5cc0b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.890+0000 7f6fbf140b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d5cc0dcd0 0x564d5cc0bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.890+0000 7f6fbf140b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d5cc0e5f0 0x564d5cc0ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.890+0000 7f6fbf140b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb01b0c00 con 0x564d5cc0dcd0 2024-06-10T01:15:19.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.890+0000 7f6fbf140b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb01ae400 con 0x564d5cc08460 2024-06-10T01:15:19.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.890+0000 7f6fbf140b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb01ac250 con 0x564d5cc0e5f0 2024-06-10T01:15:19.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.890+0000 7f6fadd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d5cc0dcd0 0x564d5cc0bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.890+0000 7f6fadd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d5cc0dcd0 0x564d5cc0bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51116/0 (socket says 172.21.15.7:51116) 2024-06-10T01:15:19.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.890+0000 7f6fae59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5cc08460 0x564d5cc0b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.890+0000 7f6fadd9b700 1 -- 172.21.15.7:0/1920926541 learned_addr learned my addr 172.21.15.7:0/1920926541 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:19.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.890+0000 7f6faed9d700 1 --2- 172.21.15.7:0/1920926541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d5cc0e5f0 0x564d5cc0ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.891+0000 7f6fadd9b700 1 -- 172.21.15.7:0/1920926541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d5cc0e5f0 msgr2=0x564d5cc0ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:19.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.891+0000 7f6fadd9b700 1 --2- 172.21.15.7:0/1920926541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d5cc0e5f0 0x564d5cc0ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.891+0000 7f6fadd9b700 1 -- 172.21.15.7:0/1920926541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5cc08460 msgr2=0x564d5cc0b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:19.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.891+0000 7f6fadd9b700 1 --2- 172.21.15.7:0/1920926541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5cc08460 0x564d5cc0b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.891+0000 7f6fadd9b700 1 -- 172.21.15.7:0/1920926541 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d5cc249f0 con 0x564d5cc0dcd0 2024-06-10T01:15:19.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.891+0000 7f6fae59c700 1 --2- 172.21.15.7:0/1920926541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5cc08460 0x564d5cc0b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:19.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.891+0000 7f6fadd9b700 1 --2- 172.21.15.7:0/1920926541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d5cc0dcd0 0x564d5cc0bc90 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400cfd0 tx=0x7f6fa400cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd6e6394def5ef3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:19.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.891+0000 7f6f97fff700 1 -- 172.21.15.7:0/1920926541 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa40a4730 con 0x564d5cc0dcd0 2024-06-10T01:15:19.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.891+0000 7f6f97fff700 1 -- 172.21.15.7:0/1920926541 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fa4099dc0 con 0x564d5cc0dcd0 2024-06-10T01:15:19.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.891+0000 7f6f97fff700 1 -- 172.21.15.7:0/1920926541 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4010980 con 0x564d5cc0dcd0 2024-06-10T01:15:19.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.892+0000 7f6fbf140b80 1 -- 172.21.15.7:0/1920926541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d5cc0dcd0 msgr2=0x564d5cc0bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:19.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.892+0000 7f6fbf140b80 1 --2- 172.21.15.7:0/1920926541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d5cc0dcd0 0x564d5cc0bc90 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400cfd0 tx=0x7f6fa400cf10 comp rx=0 tx=0).stop 2024-06-10T01:15:19.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.892+0000 7f6fbf140b80 1 -- 172.21.15.7:0/1920926541 shutdown_connections 2024-06-10T01:15:19.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.892+0000 7f6fbf140b80 1 --2- 172.21.15.7:0/1920926541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d5cc0e5f0 0x564d5cc0ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.892+0000 7f6fbf140b80 1 --2- 172.21.15.7:0/1920926541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5cc08460 0x564d5cc0b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.892+0000 7f6fbf140b80 1 --2- 172.21.15.7:0/1920926541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d5cc0dcd0 0x564d5cc0bc90 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.892+0000 7f6fbf140b80 1 -- 172.21.15.7:0/1920926541 >> 172.21.15.7:0/1920926541 conn(0x564d5cb32e50 msgr2=0x564d5cc0ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:19.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.892+0000 7f6fbf140b80 1 -- 172.21.15.7:0/1920926541 shutdown_connections 2024-06-10T01:15:19.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.892+0000 7f6fbf140b80 1 -- 172.21.15.7:0/1920926541 wait complete. 2024-06-10T01:15:19.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.892+0000 7f6fbf140b80 1 -- start start 2024-06-10T01:15:19.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.892+0000 7f6fbf140b80 1 Processor -- start 2024-06-10T01:15:19.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.893+0000 7f6fbf140b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d5cc08460 0x564d5cc1a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.893+0000 7f6fbf140b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d5cc0dcd0 0x564d5cc1a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.893+0000 7f6fbf140b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5cc0e5f0 0x564d5cc173e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.893+0000 7f6fbf140b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb01ac8c0 con 0x564d5cc08460 2024-06-10T01:15:19.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.893+0000 7f6fbf140b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb01add30 con 0x564d5cc0e5f0 2024-06-10T01:15:19.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.893+0000 7f6fbf140b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb01acf90 con 0x564d5cc0dcd0 2024-06-10T01:15:19.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.893+0000 7f6fbf140b80 10 client.0 _ll_register_callbacks cb 0x564d5caf3ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:15:19.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.893+0000 7f6fae59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d5cc08460 0x564d5cc1a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.893+0000 7f6faed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5cc0e5f0 0x564d5cc173e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.893+0000 7f6fae59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d5cc08460 0x564d5cc1a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51130/0 (socket says 172.21.15.7:51130) 2024-06-10T01:15:19.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.893+0000 7f6fadd9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d5cc0dcd0 0x564d5cc1a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.893+0000 7f6fae59c700 1 -- 172.21.15.7:0/1317775778 learned_addr learned my addr 172.21.15.7:0/1317775778 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:19.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.893+0000 7f6fae59c700 1 -- 172.21.15.7:0/1317775778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d5cc0dcd0 msgr2=0x564d5cc1a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:19.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.893+0000 7f6fae59c700 1 --2- 172.21.15.7:0/1317775778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d5cc0dcd0 0x564d5cc1a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.893+0000 7f6fae59c700 1 -- 172.21.15.7:0/1317775778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5cc0e5f0 msgr2=0x564d5cc173e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:19.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.893+0000 7f6fae59c700 1 --2- 172.21.15.7:0/1317775778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5cc0e5f0 0x564d5cc173e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.893+0000 7f6fae59c700 1 -- 172.21.15.7:0/1317775778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d5cc179d0 con 0x564d5cc08460 2024-06-10T01:15:19.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.893+0000 7f6faed9d700 1 --2- 172.21.15.7:0/1317775778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5cc0e5f0 0x564d5cc173e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:19.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.893+0000 7f6fadd9b700 1 --2- 172.21.15.7:0/1317775778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d5cc0dcd0 0x564d5cc1a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:19.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.893+0000 7f6fae59c700 1 --2- 172.21.15.7:0/1317775778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d5cc08460 0x564d5cc1a410 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f6fa00059d0 tx=0x7f6fa0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4410fc4c5f2f9e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:19.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.893+0000 7f6f95ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:15:19.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.894+0000 7f6f95ffb700 1 -- 172.21.15.7:0/1317775778 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0004a30 con 0x564d5cc08460 2024-06-10T01:15:19.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.894+0000 7f6fbf140b80 1 -- 172.21.15.7:0/1317775778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fa4099dc0 con 0x564d5cc08460 2024-06-10T01:15:19.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.894+0000 7f6f95ffb700 1 -- 172.21.15.7:0/1317775778 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f6fa0004bd0 con 0x564d5cc08460 2024-06-10T01:15:19.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.894+0000 7f6f95ffb700 1 -- 172.21.15.7:0/1317775778 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6fa0045050 con 0x564d5cc08460 2024-06-10T01:15:19.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.894+0000 7f6fbf140b80 10 client.17820 fetch_fsmap learned FSMap version 37 2024-06-10T01:15:19.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.894+0000 7f6fbf140b80 1 -- 172.21.15.7:0/1317775778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564d5cc8d5b0 con 0x564d5cc08460 2024-06-10T01:15:19.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.894+0000 7f6f95ffb700 1 -- 172.21.15.7:0/1317775778 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fa4099dc0 con 0x564d5cc08460 2024-06-10T01:15:19.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.894+0000 7f6f95ffb700 1 -- 172.21.15.7:0/1317775778 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f6fa003c680 con 0x564d5cc08460 2024-06-10T01:15:19.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.894+0000 7f6fbf140b80 10 client.17820 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T01:15:19.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.894+0000 7f6fbf140b80 10 client.17820 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:15:19.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.895+0000 7f6fbf140b80 20 client.17820 populate_metadata read hostname 'smithi007' 2024-06-10T01:15:19.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.895+0000 7f6fbf140b80 1 --2- 172.21.15.7:0/1317775778 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564d5cc8dd40 0x564d5cc90160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:19.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.895+0000 7f6fbf140b80 4 client.17820 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.8, "await_for": 45.0}] 2024-06-10T01:15:19.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.895+0000 7f6fbf140b80 1 -- 172.21.15.7:0/1317775778 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.8, "await_for": 45.0}) -- 0x564d5cb61420 con 0x564d5cc8dd40 2024-06-10T01:15:19.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.895+0000 7f6fadd9b700 1 --2- 172.21.15.7:0/1317775778 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564d5cc8dd40 0x564d5cc90160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:19.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.895+0000 7f6f95ffb700 10 client.17820 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:19.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.895+0000 7f6fadd9b700 1 --2- 172.21.15.7:0/1317775778 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564d5cc8dd40 0x564d5cc90160 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43ab8498d46fa997 server_cookie=88391ba419a940ae in_seq=0 out_seq=0 2024-06-10T01:15:19.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.948+0000 7f6f95ffb700 1 -- 172.21.15.7:0/1317775778 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564d5cb61420 con 0x564d5cc8dd40 2024-06-10T01:15:19.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.948+0000 7f6f95ffb700 10 client.17820 handle_command_reply: tid=0 2024-06-10T01:15:19.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.948+0000 7f6fbf140b80 1 client.17820 shutdown 2024-06-10T01:15:19.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.948+0000 7f6fbf140b80 1 -- 172.21.15.7:0/1317775778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d5cc08460 msgr2=0x564d5cc1a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:19.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.948+0000 7f6fbf140b80 1 --2- 172.21.15.7:0/1317775778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d5cc08460 0x564d5cc1a410 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f6fa00059d0 tx=0x7f6fa0009ca0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.948+0000 7f6fbf140b80 1 -- 172.21.15.7:0/1317775778 shutdown_connections 2024-06-10T01:15:19.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.948+0000 7f6fbf140b80 1 --2- 172.21.15.7:0/1317775778 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564d5cc8dd40 0x564d5cc90160 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.948+0000 7f6fbf140b80 1 --2- 172.21.15.7:0/1317775778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5cc0e5f0 0x564d5cc173e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.948+0000 7f6fbf140b80 1 --2- 172.21.15.7:0/1317775778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d5cc08460 0x564d5cc1a410 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.948+0000 7f6f95ffb700 0 client.17820 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:19.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.948+0000 7f6fbf140b80 1 --2- 172.21.15.7:0/1317775778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564d5cc0dcd0 0x564d5cc1a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.948+0000 7f6fbf140b80 1 -- 172.21.15.7:0/1317775778 >> 172.21.15.7:0/1317775778 conn(0x564d5cb32e50 msgr2=0x564d5cc0ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:19.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.949+0000 7f6fbf140b80 1 -- 172.21.15.7:0/1317775778 shutdown_connections 2024-06-10T01:15:19.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.949+0000 7f6fbf140b80 1 -- 172.21.15.7:0/1317775778 wait complete. 2024-06-10T01:15:19.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.949+0000 7f6fbf140b80 20 client.17820 trim_cache size 0 max 16384 2024-06-10T01:15:19.944 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:15:19.945 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 37, 2024-06-10T01:15:19.945 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:15:19.945 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 30, 2024-06-10T01:15:19.945 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:15:19.945 INFO:teuthology.orchestra.run.smithi007.stdout: "844e91e0": { 2024-06-10T01:15:19.945 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 30, 2024-06-10T01:15:19.945 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:15:19.945 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:15:19.945 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:15:19.945 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:15:19.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:15:19.945 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:15:19.945 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 137.8, 2024-06-10T01:15:19.946 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:15:19.946 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:15:19.946 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:15:19.946 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:15:19.946 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:15:19.946 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:15:19.946 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:15:19.946 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:15:19.946 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:15:19.946 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:15:19.946 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:15:19.946 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:15:19.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.949+0000 7f6fb48bd700 1 -- 172.21.15.7:0/387209237 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6f980a2cc0 msgr2=0x7f6f980a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:19.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.949+0000 7f6fb48bd700 1 --2- 172.21.15.7:0/387209237 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6f980a2cc0 0x7f6f980a5180 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0006010 tx=0x7f6fa0005960 comp rx=0 tx=0).stop 2024-06-10T01:15:19.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.949+0000 7f6fb48bd700 1 -- 172.21.15.7:0/387209237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb0102f80 msgr2=0x7f6fb019ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:19.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.949+0000 7f6fb48bd700 1 --2- 172.21.15.7:0/387209237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb0102f80 0x7f6fb019ed00 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400e9a0 tx=0x7f6fa400ecb0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.949+0000 7f6fb48bd700 1 -- 172.21.15.7:0/387209237 shutdown_connections 2024-06-10T01:15:19.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.949+0000 7f6fb48bd700 1 --2- 172.21.15.7:0/387209237 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6f980a2cc0 0x7f6f980a5180 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.949+0000 7f6fb48bd700 1 --2- 172.21.15.7:0/387209237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fb0107430 0x7f6fb0198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.949+0000 7f6fb48bd700 1 --2- 172.21.15.7:0/387209237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fb01025e0 0x7f6fb019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.949+0000 7f6fb48bd700 1 --2- 172.21.15.7:0/387209237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb0102f80 0x7f6fb019ed00 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:19.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.949+0000 7f6fb48bd700 1 -- 172.21.15.7:0/387209237 >> 172.21.15.7:0/387209237 conn(0x7f6fb0079680 msgr2=0x7f6fb0100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:19.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.950+0000 7f6fb48bd700 1 -- 172.21.15.7:0/387209237 shutdown_connections 2024-06-10T01:15:19.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:19.950+0000 7f6fb48bd700 1 -- 172.21.15.7:0/387209237 wait complete. 2024-06-10T01:15:20.577 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:15:20.744 INFO:tasks.workunit.client.0.smithi007.stdout: 1 676011 7.53 MB/sec execute 361 sec latency 848.142 ms 2024-06-10T01:15:20.762 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '844e91e0', quiesce took 6.4 seconds. Will release after: 11.4 2024-06-10T01:15:20.832 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:15:21.744 INFO:tasks.workunit.client.0.smithi007.stdout: 1 676011 7.51 MB/sec execute 362 sec latency 1848.223 ms 2024-06-10T01:15:21.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:21 smithi007 ceph-mon[32398]: pgmap v712: 129 pgs: 129 active+clean; 111 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 2.0 MiB/s wr, 178 op/s 2024-06-10T01:15:21.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:21 smithi078 ceph-mon[36683]: pgmap v712: 129 pgs: 129 active+clean; 111 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 2.0 MiB/s wr, 178 op/s 2024-06-10T01:15:21.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:21 smithi175 ceph-mon[36636]: pgmap v712: 129 pgs: 129 active+clean; 111 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 2.0 MiB/s wr, 178 op/s 2024-06-10T01:15:22.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.628+0000 7fa68e4e6700 1 -- 172.21.15.7:0/4037628804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa688102740 msgr2=0x7fa688102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:22.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.628+0000 7fa68e4e6700 1 --2- 172.21.15.7:0/4037628804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa688102740 0x7fa688102b20 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fa6780099c0 tx=0x7fa678009cd0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.628+0000 7fa68e4e6700 1 -- 172.21.15.7:0/4037628804 shutdown_connections 2024-06-10T01:15:22.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.628+0000 7fa68e4e6700 1 --2- 172.21.15.7:0/4037628804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa688107840 0x7fa68810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.628+0000 7fa68e4e6700 1 --2- 172.21.15.7:0/4037628804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa688102740 0x7fa688102b20 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.628+0000 7fa68e4e6700 1 --2- 172.21.15.7:0/4037628804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6881030f0 0x7fa688107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.628+0000 7fa68e4e6700 1 -- 172.21.15.7:0/4037628804 >> 172.21.15.7:0/4037628804 conn(0x7fa6880795f0 msgr2=0x7fa688079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:22.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.628+0000 7fa68e4e6700 1 -- 172.21.15.7:0/4037628804 shutdown_connections 2024-06-10T01:15:22.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.628+0000 7fa68e4e6700 1 -- 172.21.15.7:0/4037628804 wait complete. 2024-06-10T01:15:22.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.628+0000 7fa68e4e6700 1 Processor -- start 2024-06-10T01:15:22.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.629+0000 7fa68e4e6700 1 -- start start 2024-06-10T01:15:22.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.629+0000 7fa68e4e6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa688102740 0x7fa68819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:22.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.629+0000 7fa68e4e6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6881030f0 0x7fa68819ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:22.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.629+0000 7fa68e4e6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa688107840 0x7fa688198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:22.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.629+0000 7fa68e4e6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa688112850 con 0x7fa688102740 2024-06-10T01:15:22.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.629+0000 7fa68e4e6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa6881126d0 con 0x7fa6881030f0 2024-06-10T01:15:22.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.629+0000 7fa68e4e6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa6881129d0 con 0x7fa688107840 2024-06-10T01:15:22.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.629+0000 7fa687fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa688102740 0x7fa68819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:22.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.629+0000 7fa687fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa688102740 0x7fa68819e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50048/0 (socket says 172.21.15.7:50048) 2024-06-10T01:15:22.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.629+0000 7fa687fff700 1 -- 172.21.15.7:0/83125064 learned_addr learned my addr 172.21.15.7:0/83125064 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:22.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.629+0000 7fa67ffff700 1 --2- 172.21.15.7:0/83125064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6881030f0 0x7fa68819ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:22.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.629+0000 7fa68ca83700 1 --2- 172.21.15.7:0/83125064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa688107840 0x7fa688198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:22.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.629+0000 7fa687fff700 1 -- 172.21.15.7:0/83125064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa688107840 msgr2=0x7fa688198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:22.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.629+0000 7fa687fff700 1 --2- 172.21.15.7:0/83125064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa688107840 0x7fa688198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.629+0000 7fa687fff700 1 -- 172.21.15.7:0/83125064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6881030f0 msgr2=0x7fa68819ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:22.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.629+0000 7fa687fff700 1 --2- 172.21.15.7:0/83125064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6881030f0 0x7fa68819ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.629+0000 7fa687fff700 1 -- 172.21.15.7:0/83125064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa688199290 con 0x7fa688102740 2024-06-10T01:15:22.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.629+0000 7fa67ffff700 1 --2- 172.21.15.7:0/83125064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6881030f0 0x7fa68819ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:15:22.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.629+0000 7fa687fff700 1 --2- 172.21.15.7:0/83125064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa688102740 0x7fa68819e920 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa678006010 tx=0x7fa678005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:22.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.630+0000 7fa685ffb700 1 -- 172.21.15.7:0/83125064 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa678027070 con 0x7fa688102740 2024-06-10T01:15:22.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.630+0000 7fa685ffb700 1 -- 172.21.15.7:0/83125064 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6780040d0 con 0x7fa688102740 2024-06-10T01:15:22.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.630+0000 7fa68e4e6700 1 -- 172.21.15.7:0/83125064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa688199520 con 0x7fa688102740 2024-06-10T01:15:22.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.630+0000 7fa685ffb700 1 -- 172.21.15.7:0/83125064 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67800f460 con 0x7fa688102740 2024-06-10T01:15:22.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.630+0000 7fa68e4e6700 1 -- 172.21.15.7:0/83125064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6881a57f0 con 0x7fa688102740 2024-06-10T01:15:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.632+0000 7fa685ffb700 1 -- 172.21.15.7:0/83125064 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa67802a070 con 0x7fa688102740 2024-06-10T01:15:22.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.633+0000 7fa685ffb700 1 --2- 172.21.15.7:0/83125064 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa6740a2c20 0x7fa6740a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:22.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.633+0000 7fa67ffff700 1 --2- 172.21.15.7:0/83125064 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa6740a2c20 0x7fa6740a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:22.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.633+0000 7fa685ffb700 1 -- 172.21.15.7:0/83125064 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa6780d02b0 con 0x7fa688102740 2024-06-10T01:15:22.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.634+0000 7fa67ffff700 1 --2- 172.21.15.7:0/83125064 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa6740a2c20 0x7fa6740a50e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa6881071e0 tx=0x7fa670008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:22.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.634+0000 7fa698d69b80 1 Processor -- start 2024-06-10T01:15:22.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.634+0000 7fa698d69b80 1 -- start start 2024-06-10T01:15:22.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.634+0000 7fa698d69b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af8ff95f60 0x55af8ff96340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:22.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.634+0000 7fa698d69b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af8ff96880 0x55af8ff90260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:22.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.634+0000 7fa698d69b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af8ff907a0 0x55af8ff90bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:22.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.634+0000 7fa698d69b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55af8fe82f60 con 0x55af8ff907a0 2024-06-10T01:15:22.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.634+0000 7fa698d69b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55af8fee1850 con 0x55af8ff95f60 2024-06-10T01:15:22.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.634+0000 7fa698d69b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55af8fe49460 con 0x55af8ff96880 2024-06-10T01:15:22.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.634+0000 7fa68ca83700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af8ff907a0 0x55af8ff90bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:22.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.634+0000 7fa68ca83700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af8ff907a0 0x55af8ff90bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50058/0 (socket says 172.21.15.7:50058) 2024-06-10T01:15:22.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.634+0000 7fa68ca83700 1 -- 172.21.15.7:0/2665998041 learned_addr learned my addr 172.21.15.7:0/2665998041 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:22.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.635+0000 7fa67ffff700 1 --2- 172.21.15.7:0/2665998041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af8ff96880 0x55af8ff90260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:22.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.635+0000 7fa687fff700 1 --2- 172.21.15.7:0/2665998041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af8ff95f60 0x55af8ff96340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.635+0000 7fa68ca83700 1 -- 172.21.15.7:0/2665998041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af8ff96880 msgr2=0x55af8ff90260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.635+0000 7fa68ca83700 1 --2- 172.21.15.7:0/2665998041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af8ff96880 0x55af8ff90260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.635+0000 7fa68ca83700 1 -- 172.21.15.7:0/2665998041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af8ff95f60 msgr2=0x55af8ff96340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.635+0000 7fa68ca83700 1 --2- 172.21.15.7:0/2665998041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af8ff95f60 0x55af8ff96340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.635+0000 7fa68ca83700 1 -- 172.21.15.7:0/2665998041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af8fe308d0 con 0x55af8ff907a0 2024-06-10T01:15:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.635+0000 7fa67ffff700 1 --2- 172.21.15.7:0/2665998041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af8ff96880 0x55af8ff90260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:15:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.635+0000 7fa68ca83700 1 --2- 172.21.15.7:0/2665998041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af8ff907a0 0x55af8ff90bc0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa680009f20 tx=0x7fa680009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a359e3e837a161a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.635+0000 7fa67e7fc700 1 -- 172.21.15.7:0/2665998041 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68001a070 con 0x55af8ff907a0 2024-06-10T01:15:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.635+0000 7fa67e7fc700 1 -- 172.21.15.7:0/2665998041 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa680015410 con 0x55af8ff907a0 2024-06-10T01:15:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.635+0000 7fa67e7fc700 1 -- 172.21.15.7:0/2665998041 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68000a660 con 0x55af8ff907a0 2024-06-10T01:15:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.636+0000 7fa698d69b80 1 -- 172.21.15.7:0/2665998041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af8ff907a0 msgr2=0x55af8ff90bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.636+0000 7fa698d69b80 1 --2- 172.21.15.7:0/2665998041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af8ff907a0 0x55af8ff90bc0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa680009f20 tx=0x7fa680009f50 comp rx=0 tx=0).stop 2024-06-10T01:15:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.636+0000 7fa698d69b80 1 -- 172.21.15.7:0/2665998041 shutdown_connections 2024-06-10T01:15:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.636+0000 7fa698d69b80 1 --2- 172.21.15.7:0/2665998041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af8ff907a0 0x55af8ff90bc0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.636+0000 7fa698d69b80 1 --2- 172.21.15.7:0/2665998041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af8ff95f60 0x55af8ff96340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.636+0000 7fa698d69b80 1 --2- 172.21.15.7:0/2665998041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af8ff96880 0x55af8ff90260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.636+0000 7fa698d69b80 1 -- 172.21.15.7:0/2665998041 >> 172.21.15.7:0/2665998041 conn(0x55af8febd080 msgr2=0x55af8ff8b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.636+0000 7fa698d69b80 1 -- 172.21.15.7:0/2665998041 shutdown_connections 2024-06-10T01:15:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.636+0000 7fa698d69b80 1 -- 172.21.15.7:0/2665998041 wait complete. 2024-06-10T01:15:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.636+0000 7fa698d69b80 1 -- start start 2024-06-10T01:15:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.637+0000 7fa698d69b80 1 Processor -- start 2024-06-10T01:15:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.637+0000 7fa698d69b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af8ff907a0 0x55af8ff9c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.637+0000 7fa698d69b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af8ff95f60 0x55af8ff998f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.637+0000 7fa698d69b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af8ff96880 0x55af8ff99e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.637+0000 7fa698d69b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55af8fe81fc0 con 0x55af8ff95f60 2024-06-10T01:15:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.637+0000 7fa698d69b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55af8fedd810 con 0x55af8ff907a0 2024-06-10T01:15:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.637+0000 7fa698d69b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55af8ff9d260 con 0x55af8ff96880 2024-06-10T01:15:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.637+0000 7fa698d69b80 10 client.0 _ll_register_callbacks cb 0x55af8fe775b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:15:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.637+0000 7fa67ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af8ff95f60 0x55af8ff998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.637+0000 7fa67ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af8ff95f60 0x55af8ff998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50060/0 (socket says 172.21.15.7:50060) 2024-06-10T01:15:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.637+0000 7fa67ffff700 1 -- 172.21.15.7:0/3546455968 learned_addr learned my addr 172.21.15.7:0/3546455968 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.637+0000 7fa687fff700 1 --2- 172.21.15.7:0/3546455968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af8ff907a0 0x55af8ff9c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.637+0000 7fa68ca83700 1 --2- 172.21.15.7:0/3546455968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af8ff96880 0x55af8ff99e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.637+0000 7fa67ffff700 1 -- 172.21.15.7:0/3546455968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af8ff96880 msgr2=0x55af8ff99e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.637+0000 7fa67ffff700 1 --2- 172.21.15.7:0/3546455968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af8ff96880 0x55af8ff99e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.637+0000 7fa67ffff700 1 -- 172.21.15.7:0/3546455968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af8ff907a0 msgr2=0x55af8ff9c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.637+0000 7fa67ffff700 1 --2- 172.21.15.7:0/3546455968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af8ff907a0 0x55af8ff9c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.637+0000 7fa67ffff700 1 -- 172.21.15.7:0/3546455968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af8fea7750 con 0x55af8ff95f60 2024-06-10T01:15:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.637+0000 7fa68ca83700 1 --2- 172.21.15.7:0/3546455968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af8ff96880 0x55af8ff99e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:15:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.638+0000 7fa67ffff700 1 --2- 172.21.15.7:0/3546455968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af8ff95f60 0x55af8ff998f0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fa67000def0 tx=0x7fa670004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef76d1030587de1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.638+0000 7fa6677fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:15:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.638+0000 7fa6677fe700 1 -- 172.21.15.7:0/3546455968 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670014e60 con 0x55af8ff95f60 2024-06-10T01:15:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.638+0000 7fa6677fe700 1 -- 172.21.15.7:0/3546455968 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa67000f040 con 0x55af8ff95f60 2024-06-10T01:15:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.638+0000 7fa698d69b80 1 -- 172.21.15.7:0/3546455968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa680015410 con 0x55af8ff95f60 2024-06-10T01:15:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.638+0000 7fa6677fe700 1 -- 172.21.15.7:0/3546455968 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa67002e050 con 0x55af8ff95f60 2024-06-10T01:15:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.638+0000 7fa698d69b80 10 client.17832 fetch_fsmap learned FSMap version 37 2024-06-10T01:15:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.638+0000 7fa698d69b80 1 -- 172.21.15.7:0/3546455968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55af8ffa79e0 con 0x55af8ff95f60 2024-06-10T01:15:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.638+0000 7fa6677fe700 1 -- 172.21.15.7:0/3546455968 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa680015410 con 0x55af8ff95f60 2024-06-10T01:15:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.638+0000 7fa6677fe700 1 -- 172.21.15.7:0/3546455968 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7fa670013070 con 0x55af8ff95f60 2024-06-10T01:15:22.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.638+0000 7fa698d69b80 10 client.17832 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T01:15:22.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.638+0000 7fa698d69b80 10 client.17832 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:15:22.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.638+0000 7fa698d69b80 20 client.17832 populate_metadata read hostname 'smithi007' 2024-06-10T01:15:22.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.638+0000 7fa698d69b80 1 --2- 172.21.15.7:0/3546455968 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55af8ffaa070 0x55af8ffac490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:22.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.638+0000 7fa698d69b80 4 client.17832 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:15:22.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.638+0000 7fa698d69b80 1 -- 172.21.15.7:0/3546455968 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af8feeed50 con 0x55af8ffaa070 2024-06-10T01:15:22.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.639+0000 7fa687fff700 1 --2- 172.21.15.7:0/3546455968 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55af8ffaa070 0x55af8ffac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:22.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.639+0000 7fa687fff700 1 --2- 172.21.15.7:0/3546455968 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55af8ffaa070 0x55af8ffac490 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fb00d36bfa9663e server_cookie=52a7c9504085be27 in_seq=0 out_seq=0 2024-06-10T01:15:22.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.639+0000 7fa6677fe700 10 client.17832 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:22.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.640+0000 7fa6677fe700 1 -- 172.21.15.7:0/3546455968 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55af8feeed50 con 0x55af8ffaa070 2024-06-10T01:15:22.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.640+0000 7fa6677fe700 10 client.17832 handle_command_reply: tid=0 2024-06-10T01:15:22.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.640+0000 7fa698d69b80 1 client.17832 shutdown 2024-06-10T01:15:22.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.641+0000 7fa698d69b80 1 -- 172.21.15.7:0/3546455968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af8ff95f60 msgr2=0x55af8ff998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:22.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.641+0000 7fa698d69b80 1 --2- 172.21.15.7:0/3546455968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af8ff95f60 0x55af8ff998f0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fa67000def0 tx=0x7fa670004750 comp rx=0 tx=0).stop 2024-06-10T01:15:22.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.641+0000 7fa698d69b80 1 -- 172.21.15.7:0/3546455968 shutdown_connections 2024-06-10T01:15:22.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.641+0000 7fa698d69b80 1 --2- 172.21.15.7:0/3546455968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af8ff96880 0x55af8ff99e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.641+0000 7fa698d69b80 1 --2- 172.21.15.7:0/3546455968 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55af8ffaa070 0x55af8ffac490 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.641+0000 7fa6677fe700 0 client.17832 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:22.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.641+0000 7fa698d69b80 1 --2- 172.21.15.7:0/3546455968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af8ff907a0 0x55af8ff9c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.641+0000 7fa698d69b80 1 --2- 172.21.15.7:0/3546455968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af8ff95f60 0x55af8ff998f0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.641+0000 7fa698d69b80 1 -- 172.21.15.7:0/3546455968 >> 172.21.15.7:0/3546455968 conn(0x55af8febd080 msgr2=0x55af8ff8a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:22.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.641+0000 7fa698d69b80 1 -- 172.21.15.7:0/3546455968 shutdown_connections 2024-06-10T01:15:22.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.641+0000 7fa698d69b80 1 -- 172.21.15.7:0/3546455968 wait complete. 2024-06-10T01:15:22.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.641+0000 7fa698d69b80 20 client.17832 trim_cache size 0 max 16384 2024-06-10T01:15:22.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.661+0000 7fa698d69b80 1 Processor -- start 2024-06-10T01:15:22.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.661+0000 7fa698d69b80 1 -- start start 2024-06-10T01:15:22.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.662+0000 7fa698d69b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af8ff907a0 0x55af8ff90260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:22.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.662+0000 7fa698d69b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af8ff95f60 0x55af8ff974f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:22.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.662+0000 7fa698d69b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af8ff96880 0x55af8ff9b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:22.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.662+0000 7fa698d69b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa6881b0bc0 con 0x55af8ff96880 2024-06-10T01:15:22.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.662+0000 7fa698d69b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa6881ae3c0 con 0x55af8ff907a0 2024-06-10T01:15:22.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.662+0000 7fa698d69b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa6881ac210 con 0x55af8ff95f60 2024-06-10T01:15:22.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.662+0000 7fa68ca83700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af8ff96880 0x55af8ff9b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:22.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.662+0000 7fa68ca83700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af8ff96880 0x55af8ff9b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50068/0 (socket says 172.21.15.7:50068) 2024-06-10T01:15:22.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.662+0000 7fa67ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af8ff95f60 0x55af8ff974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:22.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.662+0000 7fa68ca83700 1 -- 172.21.15.7:0/887173838 learned_addr learned my addr 172.21.15.7:0/887173838 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:22.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.662+0000 7fa687fff700 1 --2- 172.21.15.7:0/887173838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af8ff907a0 0x55af8ff90260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:22.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.662+0000 7fa68ca83700 1 -- 172.21.15.7:0/887173838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af8ff95f60 msgr2=0x55af8ff974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:22.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.662+0000 7fa68ca83700 1 --2- 172.21.15.7:0/887173838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af8ff95f60 0x55af8ff974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.662+0000 7fa68ca83700 1 -- 172.21.15.7:0/887173838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af8ff907a0 msgr2=0x55af8ff90260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:22.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.662+0000 7fa68ca83700 1 --2- 172.21.15.7:0/887173838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af8ff907a0 0x55af8ff90260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.662+0000 7fa68ca83700 1 -- 172.21.15.7:0/887173838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af8ff97e60 con 0x55af8ff96880 2024-06-10T01:15:22.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.662+0000 7fa68ca83700 1 --2- 172.21.15.7:0/887173838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af8ff96880 0x55af8ff9b830 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fa680009ec0 tx=0x7fa68000c2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede925c613bd0368 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:22.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.663+0000 7fa67dffb700 1 -- 172.21.15.7:0/887173838 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68001a040 con 0x55af8ff96880 2024-06-10T01:15:22.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.663+0000 7fa67dffb700 1 -- 172.21.15.7:0/887173838 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa680009390 con 0x55af8ff96880 2024-06-10T01:15:22.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.663+0000 7fa67dffb700 1 -- 172.21.15.7:0/887173838 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68001f6c0 con 0x55af8ff96880 2024-06-10T01:15:22.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.663+0000 7fa698d69b80 1 -- 172.21.15.7:0/887173838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af8ff96880 msgr2=0x55af8ff9b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:22.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.663+0000 7fa698d69b80 1 --2- 172.21.15.7:0/887173838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af8ff96880 0x55af8ff9b830 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fa680009ec0 tx=0x7fa68000c2e0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.663+0000 7fa698d69b80 1 -- 172.21.15.7:0/887173838 shutdown_connections 2024-06-10T01:15:22.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.663+0000 7fa698d69b80 1 --2- 172.21.15.7:0/887173838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af8ff96880 0x55af8ff9b830 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.663+0000 7fa698d69b80 1 --2- 172.21.15.7:0/887173838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af8ff907a0 0x55af8ff90260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.663+0000 7fa698d69b80 1 --2- 172.21.15.7:0/887173838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af8ff95f60 0x55af8ff974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.663+0000 7fa698d69b80 1 -- 172.21.15.7:0/887173838 >> 172.21.15.7:0/887173838 conn(0x55af8febd080 msgr2=0x55af8ff9b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:22.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.663+0000 7fa698d69b80 1 -- 172.21.15.7:0/887173838 shutdown_connections 2024-06-10T01:15:22.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.663+0000 7fa698d69b80 1 -- 172.21.15.7:0/887173838 wait complete. 2024-06-10T01:15:22.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.664+0000 7fa698d69b80 1 -- start start 2024-06-10T01:15:22.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.664+0000 7fa698d69b80 1 Processor -- start 2024-06-10T01:15:22.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.664+0000 7fa698d69b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af8ff907a0 0x55af8ffa2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:22.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.664+0000 7fa698d69b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af8ff95f60 0x55af8ffa2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:22.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.664+0000 7fa698d69b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af8ff96880 0x55af8ff9f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:22.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.664+0000 7fa698d69b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa6881ac880 con 0x55af8ff96880 2024-06-10T01:15:22.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.664+0000 7fa698d69b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa6881adcf0 con 0x55af8ff95f60 2024-06-10T01:15:22.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.664+0000 7fa698d69b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa6881acf50 con 0x55af8ff907a0 2024-06-10T01:15:22.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.664+0000 7fa698d69b80 10 client.0 _ll_register_callbacks cb 0x55af8fe775b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:15:22.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.665+0000 7fa68ca83700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af8ff96880 0x55af8ff9f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:22.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.665+0000 7fa68ca83700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af8ff96880 0x55af8ff9f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50082/0 (socket says 172.21.15.7:50082) 2024-06-10T01:15:22.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.665+0000 7fa68ca83700 1 -- 172.21.15.7:0/1243410064 learned_addr learned my addr 172.21.15.7:0/1243410064 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:22.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.665+0000 7fa67ffff700 1 --2- 172.21.15.7:0/1243410064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af8ff95f60 0x55af8ffa2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:22.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.665+0000 7fa687fff700 1 --2- 172.21.15.7:0/1243410064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af8ff907a0 0x55af8ffa2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:22.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.665+0000 7fa68ca83700 1 -- 172.21.15.7:0/1243410064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af8ff907a0 msgr2=0x55af8ffa2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:22.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.665+0000 7fa68ca83700 1 --2- 172.21.15.7:0/1243410064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af8ff907a0 0x55af8ffa2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.665+0000 7fa68ca83700 1 -- 172.21.15.7:0/1243410064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af8ff95f60 msgr2=0x55af8ffa2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:22.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.665+0000 7fa68ca83700 1 --2- 172.21.15.7:0/1243410064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af8ff95f60 0x55af8ffa2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.665+0000 7fa68ca83700 1 -- 172.21.15.7:0/1243410064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af8ff9fa30 con 0x55af8ff96880 2024-06-10T01:15:22.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.665+0000 7fa68ca83700 1 --2- 172.21.15.7:0/1243410064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af8ff96880 0x55af8ff9f4f0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fa680015f40 tx=0x7fa680009fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0f5280a7a5e69c6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:22.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.665+0000 7fa6677fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:15:22.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.665+0000 7fa6677fe700 1 -- 172.21.15.7:0/1243410064 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68002e070 con 0x55af8ff96880 2024-06-10T01:15:22.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.665+0000 7fa6677fe700 1 -- 172.21.15.7:0/1243410064 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa68000cd70 con 0x55af8ff96880 2024-06-10T01:15:22.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.665+0000 7fa698d69b80 1 -- 172.21.15.7:0/1243410064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa680009390 con 0x55af8ff96880 2024-06-10T01:15:22.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.665+0000 7fa6677fe700 1 -- 172.21.15.7:0/1243410064 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa680039050 con 0x55af8ff96880 2024-06-10T01:15:22.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.665+0000 7fa698d69b80 10 client.17838 fetch_fsmap learned FSMap version 37 2024-06-10T01:15:22.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.666+0000 7fa698d69b80 1 -- 172.21.15.7:0/1243410064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55af900197a0 con 0x55af8ff96880 2024-06-10T01:15:22.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.666+0000 7fa6677fe700 1 -- 172.21.15.7:0/1243410064 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa680009390 con 0x55af8ff96880 2024-06-10T01:15:22.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.666+0000 7fa6677fe700 1 -- 172.21.15.7:0/1243410064 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7fa68002ace0 con 0x55af8ff96880 2024-06-10T01:15:22.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.666+0000 7fa698d69b80 10 client.17838 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T01:15:22.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.666+0000 7fa698d69b80 10 client.17838 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:15:22.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.666+0000 7fa698d69b80 20 client.17838 populate_metadata read hostname 'smithi007' 2024-06-10T01:15:22.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.666+0000 7fa698d69b80 1 --2- 172.21.15.7:0/1243410064 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55af8ffaa070 0x55af9001bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:22.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.666+0000 7fa698d69b80 4 client.17838 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:15:22.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.666+0000 7fa698d69b80 1 -- 172.21.15.7:0/1243410064 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55af8fee92d0 con 0x55af8ffaa070 2024-06-10T01:15:22.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.666+0000 7fa687fff700 1 --2- 172.21.15.7:0/1243410064 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55af8ffaa070 0x55af9001bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:22.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.667+0000 7fa687fff700 1 --2- 172.21.15.7:0/1243410064 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55af8ffaa070 0x55af9001bb70 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=452436c88481bcfa server_cookie=63eb076926bc8e16 in_seq=0 out_seq=0 2024-06-10T01:15:22.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.667+0000 7fa6677fe700 10 client.17838 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:22.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.667+0000 7fa6677fe700 1 -- 172.21.15.7:0/1243410064 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55af8fee92d0 con 0x55af8ffaa070 2024-06-10T01:15:22.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.667+0000 7fa6677fe700 10 client.17838 handle_command_reply: tid=0 2024-06-10T01:15:22.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.667+0000 7fa698d69b80 1 client.17838 shutdown 2024-06-10T01:15:22.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.667+0000 7fa698d69b80 1 -- 172.21.15.7:0/1243410064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af8ff96880 msgr2=0x55af8ff9f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:22.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.667+0000 7fa698d69b80 1 --2- 172.21.15.7:0/1243410064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af8ff96880 0x55af8ff9f4f0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fa680015f40 tx=0x7fa680009fb0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.667+0000 7fa698d69b80 1 -- 172.21.15.7:0/1243410064 shutdown_connections 2024-06-10T01:15:22.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.667+0000 7fa698d69b80 1 --2- 172.21.15.7:0/1243410064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af8ff96880 0x55af8ff9f4f0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.667+0000 7fa698d69b80 1 --2- 172.21.15.7:0/1243410064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af8ff907a0 0x55af8ffa2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.667+0000 7fa698d69b80 1 --2- 172.21.15.7:0/1243410064 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55af8ffaa070 0x55af9001bb70 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.667+0000 7fa6677fe700 0 client.17838 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:22.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.667+0000 7fa698d69b80 1 --2- 172.21.15.7:0/1243410064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af8ff95f60 0x55af8ffa2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.667+0000 7fa698d69b80 1 -- 172.21.15.7:0/1243410064 >> 172.21.15.7:0/1243410064 conn(0x55af8febd080 msgr2=0x55af8ff9b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:22.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.667+0000 7fa698d69b80 1 -- 172.21.15.7:0/1243410064 shutdown_connections 2024-06-10T01:15:22.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.667+0000 7fa698d69b80 1 -- 172.21.15.7:0/1243410064 wait complete. 2024-06-10T01:15:22.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.667+0000 7fa698d69b80 20 client.17838 trim_cache size 0 max 16384 2024-06-10T01:15:22.674 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:15:22.674 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-10T01:15:22.674 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-10T01:15:22.674 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:15:22.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.668+0000 7fa68e4e6700 1 -- 172.21.15.7:0/83125064 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa6740a2c20 msgr2=0x7fa6740a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:22.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.668+0000 7fa68e4e6700 1 --2- 172.21.15.7:0/83125064 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa6740a2c20 0x7fa6740a50e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa6881071e0 tx=0x7fa670008040 comp rx=0 tx=0).stop 2024-06-10T01:15:22.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.668+0000 7fa68e4e6700 1 -- 172.21.15.7:0/83125064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa688102740 msgr2=0x7fa68819e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:22.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.668+0000 7fa68e4e6700 1 --2- 172.21.15.7:0/83125064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa688102740 0x7fa68819e920 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa678006010 tx=0x7fa678005640 comp rx=0 tx=0).stop 2024-06-10T01:15:22.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.668+0000 7fa68e4e6700 1 -- 172.21.15.7:0/83125064 shutdown_connections 2024-06-10T01:15:22.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.668+0000 7fa68e4e6700 1 --2- 172.21.15.7:0/83125064 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa6740a2c20 0x7fa6740a50e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.668+0000 7fa68e4e6700 1 --2- 172.21.15.7:0/83125064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa688107840 0x7fa688198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.668+0000 7fa68e4e6700 1 --2- 172.21.15.7:0/83125064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa688102740 0x7fa68819e920 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.668+0000 7fa68e4e6700 1 --2- 172.21.15.7:0/83125064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6881030f0 0x7fa68819ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:22.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.668+0000 7fa68e4e6700 1 -- 172.21.15.7:0/83125064 >> 172.21.15.7:0/83125064 conn(0x7fa6880795f0 msgr2=0x7fa688101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:22.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.668+0000 7fa68e4e6700 1 -- 172.21.15.7:0/83125064 shutdown_connections 2024-06-10T01:15:22.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:22.668+0000 7fa68e4e6700 1 -- 172.21.15.7:0/83125064 wait complete. 2024-06-10T01:15:22.744 INFO:tasks.workunit.client.0.smithi007.stdout: 1 676011 7.49 MB/sec execute 363 sec latency 2848.308 ms 2024-06-10T01:15:23.435 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:15:23.435 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:15:23.683 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:15:23.744 INFO:tasks.workunit.client.0.smithi007.stdout: 1 676011 7.47 MB/sec execute 364 sec latency 3848.377 ms 2024-06-10T01:15:23.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:23 smithi007 ceph-mon[32398]: pgmap v713: 129 pgs: 129 active+clean; 111 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-06-10T01:15:23.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:23 smithi078 ceph-mon[36683]: pgmap v713: 129 pgs: 129 active+clean; 111 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-06-10T01:15:23.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:23 smithi175 ceph-mon[36636]: pgmap v713: 129 pgs: 129 active+clean; 111 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-06-10T01:15:24.744 INFO:tasks.workunit.client.0.smithi007.stdout: 1 676011 7.45 MB/sec execute 365 sec latency 4848.461 ms 2024-06-10T01:15:25.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.559+0000 7f0d27222700 1 -- 172.21.15.7:0/2876653110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d20108670 msgr2=0x7f0d20108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:25.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.559+0000 7f0d27222700 1 --2- 172.21.15.7:0/2876653110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d20108670 0x7f0d20108a50 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f0d14009990 tx=0x7f0d14009ca0 comp rx=0 tx=0).stop 2024-06-10T01:15:25.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.559+0000 7f0d27222700 1 -- 172.21.15.7:0/2876653110 shutdown_connections 2024-06-10T01:15:25.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.559+0000 7f0d27222700 1 --2- 172.21.15.7:0/2876653110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d20101880 0x7f0d2010f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:25.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.559+0000 7f0d27222700 1 --2- 172.21.15.7:0/2876653110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d20108670 0x7f0d20108a50 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:25.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.559+0000 7f0d27222700 1 --2- 172.21.15.7:0/2876653110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d20100ec0 0x7f0d20101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:25.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.559+0000 7f0d27222700 1 -- 172.21.15.7:0/2876653110 >> 172.21.15.7:0/2876653110 conn(0x7f0d20076600 msgr2=0x7f0d20076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:25.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.559+0000 7f0d27222700 1 -- 172.21.15.7:0/2876653110 shutdown_connections 2024-06-10T01:15:25.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.559+0000 7f0d27222700 1 -- 172.21.15.7:0/2876653110 wait complete. 2024-06-10T01:15:25.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.560+0000 7f0d27222700 1 Processor -- start 2024-06-10T01:15:25.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.560+0000 7f0d27222700 1 -- start start 2024-06-10T01:15:25.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.560+0000 7f0d27222700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d20100ec0 0x7f0d201020a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:25.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.560+0000 7f0d27222700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d20101880 0x7f0d201025e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:25.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.560+0000 7f0d27222700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d20108670 0x7f0d20102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:25.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.560+0000 7f0d27222700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0d20112790 con 0x7f0d20101880 2024-06-10T01:15:25.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.560+0000 7f0d27222700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0d20112610 con 0x7f0d20100ec0 2024-06-10T01:15:25.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.560+0000 7f0d27222700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0d20112910 con 0x7f0d20108670 2024-06-10T01:15:25.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.560+0000 7f0d1ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d20101880 0x7f0d201025e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:25.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.560+0000 7f0d1ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d20101880 0x7f0d201025e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50098/0 (socket says 172.21.15.7:50098) 2024-06-10T01:15:25.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.560+0000 7f0d1ffff700 1 -- 172.21.15.7:0/2200455991 learned_addr learned my addr 172.21.15.7:0/2200455991 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:25.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.560+0000 7f0d24fbe700 1 --2- 172.21.15.7:0/2200455991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d20100ec0 0x7f0d201020a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:25.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.561+0000 7f0d257bf700 1 --2- 172.21.15.7:0/2200455991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d20108670 0x7f0d20102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:25.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.561+0000 7f0d1ffff700 1 -- 172.21.15.7:0/2200455991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d20108670 msgr2=0x7f0d20102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.561+0000 7f0d1ffff700 1 --2- 172.21.15.7:0/2200455991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d20108670 0x7f0d20102b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.561+0000 7f0d1ffff700 1 -- 172.21.15.7:0/2200455991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d20100ec0 msgr2=0x7f0d201020a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.561+0000 7f0d1ffff700 1 --2- 172.21.15.7:0/2200455991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d20100ec0 0x7f0d201020a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.561+0000 7f0d1ffff700 1 -- 172.21.15.7:0/2200455991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d201a7050 con 0x7f0d20101880 2024-06-10T01:15:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.561+0000 7f0d1ffff700 1 --2- 172.21.15.7:0/2200455991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d20101880 0x7f0d201025e0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00c9b0 tx=0x7f0d0c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.561+0000 7f0d1dffb700 1 -- 172.21.15.7:0/2200455991 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c007b00 con 0x7f0d20101880 2024-06-10T01:15:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.561+0000 7f0d1dffb700 1 -- 172.21.15.7:0/2200455991 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d0c007ca0 con 0x7f0d20101880 2024-06-10T01:15:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.561+0000 7f0d27222700 1 -- 172.21.15.7:0/2200455991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d201a7340 con 0x7f0d20101880 2024-06-10T01:15:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.561+0000 7f0d1dffb700 1 -- 172.21.15.7:0/2200455991 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c018730 con 0x7f0d20101880 2024-06-10T01:15:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.561+0000 7f0d27222700 1 -- 172.21.15.7:0/2200455991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d201a7880 con 0x7f0d20101880 2024-06-10T01:15:25.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.562+0000 7f0d27222700 1 -- 172.21.15.7:0/2200455991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d20069180 con 0x7f0d20101880 2024-06-10T01:15:25.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.563+0000 7f0d1dffb700 1 -- 172.21.15.7:0/2200455991 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0d0c0188d0 con 0x7f0d20101880 2024-06-10T01:15:25.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.564+0000 7f0d1dffb700 1 --2- 172.21.15.7:0/2200455991 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0d100a2c20 0x7f0d100a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:25.570 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:25 smithi007 ceph-mon[32398]: pgmap v714: 129 pgs: 129 active+clean; 114 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 2.4 MiB/s wr, 207 op/s 2024-06-10T01:15:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.564+0000 7f0d1dffb700 1 -- 172.21.15.7:0/2200455991 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f0d0c0cf390 con 0x7f0d20101880 2024-06-10T01:15:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.564+0000 7f0d24fbe700 1 --2- 172.21.15.7:0/2200455991 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0d100a2c20 0x7f0d100a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.565+0000 7f0d24fbe700 1 --2- 172.21.15.7:0/2200455991 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0d100a2c20 0x7f0d100a50e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0d1400b5c0 tx=0x7f0d14005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.569+0000 7f0d1dffb700 1 -- 172.21.15.7:0/2200455991 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0d0c014030 con 0x7f0d20101880 2024-06-10T01:15:25.745 INFO:tasks.workunit.client.0.smithi007.stdout: 1 676011 7.43 MB/sec execute 366 sec latency 5848.508 ms 2024-06-10T01:15:25.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.822+0000 7f0d27222700 1 -- 172.21.15.7:0/2200455991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d201035d0 con 0x7f0d20101880 2024-06-10T01:15:25.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.823+0000 7f0d1dffb700 1 -- 172.21.15.7:0/2200455991 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11859 (secure 0 0 0) 0x7f0d201035d0 con 0x7f0d20101880 2024-06-10T01:15:25.817 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:15:25.817 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:15:13.790052+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24451,24434,24436]},"id":1}]} 2024-06-10T01:15:25.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.825+0000 7f0d27222700 1 -- 172.21.15.7:0/2200455991 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0d100a2c20 msgr2=0x7f0d100a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:25.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.825+0000 7f0d27222700 1 --2- 172.21.15.7:0/2200455991 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0d100a2c20 0x7f0d100a50e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0d1400b5c0 tx=0x7f0d14005960 comp rx=0 tx=0).stop 2024-06-10T01:15:25.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.825+0000 7f0d27222700 1 -- 172.21.15.7:0/2200455991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d20101880 msgr2=0x7f0d201025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:25.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.825+0000 7f0d27222700 1 --2- 172.21.15.7:0/2200455991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d20101880 0x7f0d201025e0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00c9b0 tx=0x7f0d0c00ccc0 comp rx=0 tx=0).stop 2024-06-10T01:15:25.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.825+0000 7f0d27222700 1 -- 172.21.15.7:0/2200455991 shutdown_connections 2024-06-10T01:15:25.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.825+0000 7f0d27222700 1 --2- 172.21.15.7:0/2200455991 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0d100a2c20 0x7f0d100a50e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:25.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.825+0000 7f0d27222700 1 --2- 172.21.15.7:0/2200455991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d20108670 0x7f0d20102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:25.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.825+0000 7f0d27222700 1 --2- 172.21.15.7:0/2200455991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d20100ec0 0x7f0d201020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:25.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.825+0000 7f0d27222700 1 --2- 172.21.15.7:0/2200455991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d20101880 0x7f0d201025e0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:25.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.825+0000 7f0d27222700 1 -- 172.21.15.7:0/2200455991 >> 172.21.15.7:0/2200455991 conn(0x7f0d20076600 msgr2=0x7f0d200ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:25.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.825+0000 7f0d27222700 1 -- 172.21.15.7:0/2200455991 shutdown_connections 2024-06-10T01:15:25.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:25.825+0000 7f0d27222700 1 -- 172.21.15.7:0/2200455991 wait complete. 2024-06-10T01:15:25.820 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 37 2024-06-10T01:15:25.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:25 smithi078 ceph-mon[36683]: pgmap v714: 129 pgs: 129 active+clean; 114 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 2.4 MiB/s wr, 207 op/s 2024-06-10T01:15:25.953 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:25 smithi175 ceph-mon[36636]: pgmap v714: 129 pgs: 129 active+clean; 114 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 2.4 MiB/s wr, 207 op/s 2024-06-10T01:15:26.632 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:15:26.744 INFO:tasks.workunit.client.0.smithi007.stdout: 1 676011 7.41 MB/sec execute 367 sec latency 6848.548 ms 2024-06-10T01:15:26.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:26 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2200455991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:15:26.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:26 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2200455991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:15:26.892 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:15:26.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:26 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2200455991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:15:27.604 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:27 smithi007 ceph-mon[32398]: pgmap v715: 129 pgs: 129 active+clean; 115 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 2.2 MiB/s wr, 210 op/s 2024-06-10T01:15:27.744 INFO:tasks.workunit.client.0.smithi007.stdout: 1 676011 7.39 MB/sec execute 368 sec latency 7848.623 ms 2024-06-10T01:15:27.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:27 smithi078 ceph-mon[36683]: pgmap v715: 129 pgs: 129 active+clean; 115 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 2.2 MiB/s wr, 210 op/s 2024-06-10T01:15:27.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:27 smithi175 ceph-mon[36636]: pgmap v715: 129 pgs: 129 active+clean; 115 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 2.2 MiB/s wr, 210 op/s 2024-06-10T01:15:28.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.429+0000 7f849fc20700 1 -- 172.21.15.7:0/3362930131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8498107840 msgr2=0x7f849810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:28.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.429+0000 7f849fc20700 1 --2- 172.21.15.7:0/3362930131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8498107840 0x7f849810f3c0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f849000b0d0 tx=0x7f849000b3e0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.429+0000 7f849fc20700 1 -- 172.21.15.7:0/3362930131 shutdown_connections 2024-06-10T01:15:28.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.429+0000 7f849fc20700 1 --2- 172.21.15.7:0/3362930131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8498107840 0x7f849810f3c0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.429+0000 7f849fc20700 1 --2- 172.21.15.7:0/3362930131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8498102740 0x7f8498102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.429+0000 7f849fc20700 1 --2- 172.21.15.7:0/3362930131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84981030f0 0x7f8498107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.429+0000 7f849fc20700 1 -- 172.21.15.7:0/3362930131 >> 172.21.15.7:0/3362930131 conn(0x7f84980795f0 msgr2=0x7f8498079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:28.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.429+0000 7f849fc20700 1 -- 172.21.15.7:0/3362930131 shutdown_connections 2024-06-10T01:15:28.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.429+0000 7f849fc20700 1 -- 172.21.15.7:0/3362930131 wait complete. 2024-06-10T01:15:28.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.429+0000 7f849fc20700 1 Processor -- start 2024-06-10T01:15:28.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.430+0000 7f849fc20700 1 -- start start 2024-06-10T01:15:28.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.430+0000 7f849fc20700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8498102740 0x7f849819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:28.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.430+0000 7f849fc20700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84981030f0 0x7f849819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:28.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.430+0000 7f849fc20700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8498107840 0x7f8498198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:28.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.430+0000 7f849fc20700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8498112780 con 0x7f84981030f0 2024-06-10T01:15:28.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.430+0000 7f849fc20700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8498112600 con 0x7f8498102740 2024-06-10T01:15:28.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.430+0000 7f849fc20700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8498112900 con 0x7f8498107840 2024-06-10T01:15:28.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.430+0000 7f849d1bb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84981030f0 0x7f849819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:28.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.430+0000 7f849d1bb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84981030f0 0x7f849819ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50108/0 (socket says 172.21.15.7:50108) 2024-06-10T01:15:28.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.430+0000 7f849d1bb700 1 -- 172.21.15.7:0/3382642778 learned_addr learned my addr 172.21.15.7:0/3382642778 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:28.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.430+0000 7f849d9bc700 1 --2- 172.21.15.7:0/3382642778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8498102740 0x7f849819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:28.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.430+0000 7f849d1bb700 1 -- 172.21.15.7:0/3382642778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8498107840 msgr2=0x7f8498198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.430+0000 7f849d1bb700 1 --2- 172.21.15.7:0/3382642778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8498107840 0x7f8498198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.430+0000 7f849d1bb700 1 -- 172.21.15.7:0/3382642778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8498102740 msgr2=0x7f849819e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.430+0000 7f849d1bb700 1 --2- 172.21.15.7:0/3382642778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8498102740 0x7f849819e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.430+0000 7f849d1bb700 1 -- 172.21.15.7:0/3382642778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84981990a0 con 0x7f84981030f0 2024-06-10T01:15:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.430+0000 7f849d1bb700 1 --2- 172.21.15.7:0/3382642778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84981030f0 0x7f849819ecc0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f848800d980 tx=0x7f848800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.431+0000 7f848effd700 1 -- 172.21.15.7:0/3382642778 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8488017070 con 0x7f84981030f0 2024-06-10T01:15:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.431+0000 7f848effd700 1 -- 172.21.15.7:0/3382642778 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8488005020 con 0x7f84981030f0 2024-06-10T01:15:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.431+0000 7f849fc20700 1 -- 172.21.15.7:0/3382642778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8498199390 con 0x7f84981030f0 2024-06-10T01:15:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.431+0000 7f848effd700 1 -- 172.21.15.7:0/3382642778 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8488009bc0 con 0x7f84981030f0 2024-06-10T01:15:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.431+0000 7f849fc20700 1 -- 172.21.15.7:0/3382642778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84981a5650 con 0x7f84981030f0 2024-06-10T01:15:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.433+0000 7f848effd700 1 -- 172.21.15.7:0/3382642778 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f84880051d0 con 0x7f84981030f0 2024-06-10T01:15:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.434+0000 7f848effd700 1 --2- 172.21.15.7:0/3382642778 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f84840a2c20 0x7f84840a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.434+0000 7f849d9bc700 1 --2- 172.21.15.7:0/3382642778 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f84840a2c20 0x7f84840a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.434+0000 7f848effd700 1 -- 172.21.15.7:0/3382642778 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f84880cf2f0 con 0x7f84981030f0 2024-06-10T01:15:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.434+0000 7f849d9bc700 1 --2- 172.21.15.7:0/3382642778 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f84840a2c20 0x7f84840a50e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8494005950 tx=0x7f849400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.434+0000 7f84aa4a3b80 1 Processor -- start 2024-06-10T01:15:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.435+0000 7f84aa4a3b80 1 -- start start 2024-06-10T01:15:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.435+0000 7f84aa4a3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa28a47f60 0x55fa28a48340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.435+0000 7f84aa4a3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa28a48880 0x55fa28a42260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.435+0000 7f84aa4a3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa28a427a0 0x55fa28a42bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.435+0000 7f84aa4a3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fa28934f60 con 0x55fa28a48880 2024-06-10T01:15:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.435+0000 7f84aa4a3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fa28993850 con 0x55fa28a47f60 2024-06-10T01:15:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.435+0000 7f84aa4a3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fa288fb460 con 0x55fa28a427a0 2024-06-10T01:15:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.435+0000 7f849d1bb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa28a48880 0x55fa28a42260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.435+0000 7f849d1bb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa28a48880 0x55fa28a42260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50112/0 (socket says 172.21.15.7:50112) 2024-06-10T01:15:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.435+0000 7f849d1bb700 1 -- 172.21.15.7:0/3324522738 learned_addr learned my addr 172.21.15.7:0/3324522738 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.435+0000 7f849d9bc700 1 --2- 172.21.15.7:0/3324522738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa28a47f60 0x55fa28a48340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.435+0000 7f849e1bd700 1 --2- 172.21.15.7:0/3324522738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa28a427a0 0x55fa28a42bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.435+0000 7f849d1bb700 1 -- 172.21.15.7:0/3324522738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa28a427a0 msgr2=0x55fa28a42bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.435+0000 7f849d1bb700 1 --2- 172.21.15.7:0/3324522738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa28a427a0 0x55fa28a42bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.435+0000 7f849d1bb700 1 -- 172.21.15.7:0/3324522738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa28a47f60 msgr2=0x55fa28a48340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.435+0000 7f849d1bb700 1 --2- 172.21.15.7:0/3324522738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa28a47f60 0x55fa28a48340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.436+0000 7f849d1bb700 1 -- 172.21.15.7:0/3324522738 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa288e28d0 con 0x55fa28a48880 2024-06-10T01:15:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.436+0000 7f849e1bd700 1 --2- 172.21.15.7:0/3324522738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa28a427a0 0x55fa28a42bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:15:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.436+0000 7f849d1bb700 1 --2- 172.21.15.7:0/3324522738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa28a48880 0x55fa28a42260 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f8488097c30 tx=0x7f848809b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7a451b36b789f0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.436+0000 7f8483fff700 1 -- 172.21.15.7:0/3324522738 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848809fb80 con 0x55fa28a48880 2024-06-10T01:15:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.436+0000 7f8483fff700 1 -- 172.21.15.7:0/3324522738 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84880a4440 con 0x55fa28a48880 2024-06-10T01:15:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.436+0000 7f8483fff700 1 -- 172.21.15.7:0/3324522738 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84880a3fb0 con 0x55fa28a48880 2024-06-10T01:15:28.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.437+0000 7f84aa4a3b80 1 -- 172.21.15.7:0/3324522738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa28a48880 msgr2=0x55fa28a42260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:28.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.437+0000 7f84aa4a3b80 1 --2- 172.21.15.7:0/3324522738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa28a48880 0x55fa28a42260 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f8488097c30 tx=0x7f848809b7d0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.437+0000 7f84aa4a3b80 1 -- 172.21.15.7:0/3324522738 shutdown_connections 2024-06-10T01:15:28.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.437+0000 7f84aa4a3b80 1 --2- 172.21.15.7:0/3324522738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa28a427a0 0x55fa28a42bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.437+0000 7f84aa4a3b80 1 --2- 172.21.15.7:0/3324522738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa28a47f60 0x55fa28a48340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.437+0000 7f84aa4a3b80 1 --2- 172.21.15.7:0/3324522738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa28a48880 0x55fa28a42260 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.437+0000 7f84aa4a3b80 1 -- 172.21.15.7:0/3324522738 >> 172.21.15.7:0/3324522738 conn(0x55fa2896f080 msgr2=0x55fa28a3d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:28.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.437+0000 7f84aa4a3b80 1 -- 172.21.15.7:0/3324522738 shutdown_connections 2024-06-10T01:15:28.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.437+0000 7f84aa4a3b80 1 -- 172.21.15.7:0/3324522738 wait complete. 2024-06-10T01:15:28.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.437+0000 7f84aa4a3b80 1 -- start start 2024-06-10T01:15:28.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.437+0000 7f84aa4a3b80 1 Processor -- start 2024-06-10T01:15:28.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.438+0000 7f84aa4a3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa28a427a0 0x55fa28a4e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:28.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.438+0000 7f84aa4a3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa28a47f60 0x55fa28a4b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:28.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.438+0000 7f84aa4a3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa28a48880 0x55fa28a4be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:28.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.438+0000 7f84aa4a3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fa28933fc0 con 0x55fa28a48880 2024-06-10T01:15:28.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.438+0000 7f84aa4a3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fa2898f810 con 0x55fa28a47f60 2024-06-10T01:15:28.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.438+0000 7f84aa4a3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fa28a4f260 con 0x55fa28a427a0 2024-06-10T01:15:28.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.438+0000 7f84aa4a3b80 10 client.0 _ll_register_callbacks cb 0x55fa289295b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:15:28.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.438+0000 7f849e1bd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa28a48880 0x55fa28a4be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:28.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.438+0000 7f849e1bd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa28a48880 0x55fa28a4be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50120/0 (socket says 172.21.15.7:50120) 2024-06-10T01:15:28.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.438+0000 7f849e1bd700 1 -- 172.21.15.7:0/1221451646 learned_addr learned my addr 172.21.15.7:0/1221451646 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:28.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.438+0000 7f849d1bb700 1 --2- 172.21.15.7:0/1221451646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa28a47f60 0x55fa28a4b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:28.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.438+0000 7f849d9bc700 1 --2- 172.21.15.7:0/1221451646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa28a427a0 0x55fa28a4e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:28.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.438+0000 7f849e1bd700 1 -- 172.21.15.7:0/1221451646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa28a427a0 msgr2=0x55fa28a4e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:28.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.438+0000 7f849e1bd700 1 --2- 172.21.15.7:0/1221451646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa28a427a0 0x55fa28a4e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.438+0000 7f849e1bd700 1 -- 172.21.15.7:0/1221451646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa28a47f60 msgr2=0x55fa28a4b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:28.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.438+0000 7f849e1bd700 1 --2- 172.21.15.7:0/1221451646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa28a47f60 0x55fa28a4b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.438+0000 7f849e1bd700 1 -- 172.21.15.7:0/1221451646 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa28959750 con 0x55fa28a48880 2024-06-10T01:15:28.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.438+0000 7f849d9bc700 1 --2- 172.21.15.7:0/1221451646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa28a427a0 0x55fa28a4e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:15:28.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.438+0000 7f849d1bb700 1 --2- 172.21.15.7:0/1221451646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa28a47f60 0x55fa28a4b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:28.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.439+0000 7f849e1bd700 1 --2- 172.21.15.7:0/1221451646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa28a48880 0x55fa28a4be70 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f8490004c60 tx=0x7f8490004c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222b81b5a3f40550 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:28.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.439+0000 7f84817fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:15:28.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.439+0000 7f84817fa700 1 -- 172.21.15.7:0/1221451646 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490022070 con 0x55fa28a48880 2024-06-10T01:15:28.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.439+0000 7f84817fa700 1 -- 172.21.15.7:0/1221451646 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f84900076e0 con 0x55fa28a48880 2024-06-10T01:15:28.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.439+0000 7f84aa4a3b80 1 -- 172.21.15.7:0/1221451646 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84880a4440 con 0x55fa28a48880 2024-06-10T01:15:28.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.439+0000 7f84817fa700 1 -- 172.21.15.7:0/1221451646 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f849002e050 con 0x55fa28a48880 2024-06-10T01:15:28.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.439+0000 7f84aa4a3b80 10 client.17856 fetch_fsmap learned FSMap version 37 2024-06-10T01:15:28.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.439+0000 7f84aa4a3b80 1 -- 172.21.15.7:0/1221451646 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55fa28a59a70 con 0x55fa28a48880 2024-06-10T01:15:28.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.439+0000 7f84817fa700 1 -- 172.21.15.7:0/1221451646 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84880a4440 con 0x55fa28a48880 2024-06-10T01:15:28.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.439+0000 7f84817fa700 1 -- 172.21.15.7:0/1221451646 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f8490011070 con 0x55fa28a48880 2024-06-10T01:15:28.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.440+0000 7f84aa4a3b80 10 client.17856 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T01:15:28.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.440+0000 7f84aa4a3b80 10 client.17856 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:15:28.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.440+0000 7f84aa4a3b80 20 client.17856 populate_metadata read hostname 'smithi007' 2024-06-10T01:15:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.440+0000 7f84aa4a3b80 1 --2- 172.21.15.7:0/1221451646 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55fa28a5c050 0x55fa28a5e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.440+0000 7f84aa4a3b80 4 client.17856 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:15:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.440+0000 7f84aa4a3b80 1 -- 172.21.15.7:0/1221451646 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa289a0d50 con 0x55fa28a5c050 2024-06-10T01:15:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.440+0000 7f849d9bc700 1 --2- 172.21.15.7:0/1221451646 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55fa28a5c050 0x55fa28a5e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.440+0000 7f849d9bc700 1 --2- 172.21.15.7:0/1221451646 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55fa28a5c050 0x55fa28a5e440 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3af5788327a0e04a server_cookie=c53aaede8a75c29d in_seq=0 out_seq=0 2024-06-10T01:15:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.440+0000 7f84817fa700 10 client.17856 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:15:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.441+0000 7f84817fa700 1 -- 172.21.15.7:0/1221451646 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fa289a0d50 con 0x55fa28a5c050 2024-06-10T01:15:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.441+0000 7f84817fa700 10 client.17856 handle_command_reply: tid=0 2024-06-10T01:15:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.441+0000 7f84aa4a3b80 1 client.17856 shutdown 2024-06-10T01:15:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.441+0000 7f84aa4a3b80 1 -- 172.21.15.7:0/1221451646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa28a48880 msgr2=0x55fa28a4be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.441+0000 7f84aa4a3b80 1 --2- 172.21.15.7:0/1221451646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa28a48880 0x55fa28a4be70 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f8490004c60 tx=0x7f8490004c90 comp rx=0 tx=0).stop 2024-06-10T01:15:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.441+0000 7f84aa4a3b80 1 -- 172.21.15.7:0/1221451646 shutdown_connections 2024-06-10T01:15:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.441+0000 7f84aa4a3b80 1 --2- 172.21.15.7:0/1221451646 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55fa28a5c050 0x55fa28a5e440 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.441+0000 7f84aa4a3b80 1 --2- 172.21.15.7:0/1221451646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa28a48880 0x55fa28a4be70 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.441+0000 7f84aa4a3b80 1 --2- 172.21.15.7:0/1221451646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa28a427a0 0x55fa28a4e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.441+0000 7f84817fa700 0 client.17856 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:15:28.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.441+0000 7f84aa4a3b80 1 --2- 172.21.15.7:0/1221451646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa28a47f60 0x55fa28a4b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.441+0000 7f84aa4a3b80 1 -- 172.21.15.7:0/1221451646 >> 172.21.15.7:0/1221451646 conn(0x55fa2896f080 msgr2=0x55fa28a3c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:28.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.441+0000 7f84aa4a3b80 1 -- 172.21.15.7:0/1221451646 shutdown_connections 2024-06-10T01:15:28.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.441+0000 7f84aa4a3b80 1 -- 172.21.15.7:0/1221451646 wait complete. 2024-06-10T01:15:28.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.441+0000 7f84aa4a3b80 20 client.17856 trim_cache size 0 max 16384 2024-06-10T01:15:28.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.457+0000 7f84aa4a3b80 1 Processor -- start 2024-06-10T01:15:28.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.457+0000 7f84aa4a3b80 1 -- start start 2024-06-10T01:15:28.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.457+0000 7f84aa4a3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa28a427a0 0x55fa28a49390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:28.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.457+0000 7f84aa4a3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa28a47f60 0x55fa28a3ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:28.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.457+0000 7f84aa4a3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa28a48880 0x55fa28a3d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:28.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.457+0000 7f84aa4a3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f84981b0a10 con 0x55fa28a427a0 2024-06-10T01:15:28.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.457+0000 7f84aa4a3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f84981ae210 con 0x55fa28a48880 2024-06-10T01:15:28.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.457+0000 7f84aa4a3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f84981ac060 con 0x55fa28a47f60 2024-06-10T01:15:28.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.457+0000 7f849d9bc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa28a427a0 0x55fa28a49390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:28.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.457+0000 7f849d9bc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa28a427a0 0x55fa28a49390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50122/0 (socket says 172.21.15.7:50122) 2024-06-10T01:15:28.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.457+0000 7f849d9bc700 1 -- 172.21.15.7:0/1644711225 learned_addr learned my addr 172.21.15.7:0/1644711225 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:28.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.457+0000 7f849d1bb700 1 --2- 172.21.15.7:0/1644711225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa28a47f60 0x55fa28a3ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:28.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.457+0000 7f849d9bc700 1 -- 172.21.15.7:0/1644711225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa28a47f60 msgr2=0x55fa28a3ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:28.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.457+0000 7f849d9bc700 1 --2- 172.21.15.7:0/1644711225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa28a47f60 0x55fa28a3ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.457+0000 7f849d9bc700 1 -- 172.21.15.7:0/1644711225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa28a48880 msgr2=0x55fa28a3d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:28.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.457+0000 7f849d9bc700 1 --2- 172.21.15.7:0/1644711225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa28a48880 0x55fa28a3d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.457+0000 7f849d9bc700 1 -- 172.21.15.7:0/1644711225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa28a4c870 con 0x55fa28a427a0 2024-06-10T01:15:28.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.457+0000 7f849d1bb700 1 --2- 172.21.15.7:0/1644711225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa28a47f60 0x55fa28a3ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:15:28.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.458+0000 7f849d9bc700 1 --2- 172.21.15.7:0/1644711225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa28a427a0 0x55fa28a49390 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f849400a9d0 tx=0x7f8494012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=366ad72766fc2309 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:28.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.458+0000 7f84837fe700 1 -- 172.21.15.7:0/1644711225 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8494016070 con 0x55fa28a427a0 2024-06-10T01:15:28.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.458+0000 7f84837fe700 1 -- 172.21.15.7:0/1644711225 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8494010bb0 con 0x55fa28a427a0 2024-06-10T01:15:28.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.458+0000 7f84837fe700 1 -- 172.21.15.7:0/1644711225 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849401c810 con 0x55fa28a427a0 2024-06-10T01:15:28.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.458+0000 7f84aa4a3b80 1 -- 172.21.15.7:0/1644711225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa28a427a0 msgr2=0x55fa28a49390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:28.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.459+0000 7f84aa4a3b80 1 --2- 172.21.15.7:0/1644711225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa28a427a0 0x55fa28a49390 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f849400a9d0 tx=0x7f8494012000 comp rx=0 tx=0).stop 2024-06-10T01:15:28.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.459+0000 7f84aa4a3b80 1 -- 172.21.15.7:0/1644711225 shutdown_connections 2024-06-10T01:15:28.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.459+0000 7f84aa4a3b80 1 --2- 172.21.15.7:0/1644711225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa28a48880 0x55fa28a3d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.459+0000 7f84aa4a3b80 1 --2- 172.21.15.7:0/1644711225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa28a427a0 0x55fa28a49390 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.459+0000 7f84aa4a3b80 1 --2- 172.21.15.7:0/1644711225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa28a47f60 0x55fa28a3ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.459+0000 7f84aa4a3b80 1 -- 172.21.15.7:0/1644711225 >> 172.21.15.7:0/1644711225 conn(0x55fa2896f080 msgr2=0x55fa28a4d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:28.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.459+0000 7f84aa4a3b80 1 -- 172.21.15.7:0/1644711225 shutdown_connections 2024-06-10T01:15:28.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.459+0000 7f84aa4a3b80 1 -- 172.21.15.7:0/1644711225 wait complete. 2024-06-10T01:15:28.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.459+0000 7f84aa4a3b80 1 -- start start 2024-06-10T01:15:28.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.459+0000 7f84aa4a3b80 1 Processor -- start 2024-06-10T01:15:28.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.459+0000 7f84aa4a3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa28a427a0 0x55fa28a51cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:28.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.459+0000 7f84aa4a3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa28a47f60 0x55fa28a521f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:28.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.459+0000 7f84aa4a3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa28a48880 0x55fa28ac4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:28.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.459+0000 7f84aa4a3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f84981ac6d0 con 0x55fa28a47f60 2024-06-10T01:15:28.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.459+0000 7f84aa4a3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f84981adb40 con 0x55fa28a48880 2024-06-10T01:15:28.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.459+0000 7f84aa4a3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f84981acda0 con 0x55fa28a427a0 2024-06-10T01:15:28.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.460+0000 7f84aa4a3b80 10 client.0 _ll_register_callbacks cb 0x55fa289295b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:15:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.460+0000 7f849d1bb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa28a47f60 0x55fa28a521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.460+0000 7f849d1bb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa28a47f60 0x55fa28a521f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50126/0 (socket says 172.21.15.7:50126) 2024-06-10T01:15:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.460+0000 7f849d1bb700 1 -- 172.21.15.7:0/705034916 learned_addr learned my addr 172.21.15.7:0/705034916 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.460+0000 7f849d9bc700 1 --2- 172.21.15.7:0/705034916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa28a427a0 0x55fa28a51cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.460+0000 7f849e1bd700 1 --2- 172.21.15.7:0/705034916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa28a48880 0x55fa28ac4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.460+0000 7f849d1bb700 1 -- 172.21.15.7:0/705034916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa28a427a0 msgr2=0x55fa28a51cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.460+0000 7f849d1bb700 1 --2- 172.21.15.7:0/705034916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa28a427a0 0x55fa28a51cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.460+0000 7f849d1bb700 1 -- 172.21.15.7:0/705034916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa28a48880 msgr2=0x55fa28ac4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.460+0000 7f849d1bb700 1 --2- 172.21.15.7:0/705034916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa28a48880 0x55fa28ac4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.460+0000 7f849d1bb700 1 -- 172.21.15.7:0/705034916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa28ac5310 con 0x55fa28a47f60 2024-06-10T01:15:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.460+0000 7f849d1bb700 1 --2- 172.21.15.7:0/705034916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa28a47f60 0x55fa28a521f0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f848809b710 tx=0x7f84880cf170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f584b6fbe46743 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.460+0000 7f84817fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:15:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.460+0000 7f84817fa700 1 -- 172.21.15.7:0/705034916 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84880b8070 con 0x55fa28a47f60 2024-06-10T01:15:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.460+0000 7f84aa4a3b80 1 -- 172.21.15.7:0/705034916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8494010bb0 con 0x55fa28a47f60 2024-06-10T01:15:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.460+0000 7f84817fa700 1 -- 172.21.15.7:0/705034916 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f8488005560 con 0x55fa28a47f60 2024-06-10T01:15:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.460+0000 7f84817fa700 1 -- 172.21.15.7:0/705034916 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f848809fb80 con 0x55fa28a47f60 2024-06-10T01:15:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.460+0000 7f84aa4a3b80 10 client.17862 fetch_fsmap learned FSMap version 37 2024-06-10T01:15:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.460+0000 7f84aa4a3b80 1 -- 172.21.15.7:0/705034916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55fa28ac6580 con 0x55fa28a47f60 2024-06-10T01:15:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.461+0000 7f84817fa700 1 -- 172.21.15.7:0/705034916 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8494010bb0 con 0x55fa28a47f60 2024-06-10T01:15:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.461+0000 7f84817fa700 1 -- 172.21.15.7:0/705034916 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f848800ebc0 con 0x55fa28a47f60 2024-06-10T01:15:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.461+0000 7f84aa4a3b80 10 client.17862 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T01:15:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.461+0000 7f84aa4a3b80 10 client.17862 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:15:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.461+0000 7f84aa4a3b80 20 client.17862 populate_metadata read hostname 'smithi007' 2024-06-10T01:15:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.461+0000 7f84aa4a3b80 1 --2- 172.21.15.7:0/705034916 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55fa28ac68b0 0x55fa28ac8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.461+0000 7f84aa4a3b80 4 client.17862 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:15:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.461+0000 7f84aa4a3b80 1 -- 172.21.15.7:0/705034916 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa2899b2d0 con 0x55fa28ac68b0 2024-06-10T01:15:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.461+0000 7f849d9bc700 1 --2- 172.21.15.7:0/705034916 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55fa28ac68b0 0x55fa28ac8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.461+0000 7f849d9bc700 1 --2- 172.21.15.7:0/705034916 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55fa28ac68b0 0x55fa28ac8cd0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=438dddb79f78496f server_cookie=eb70b822bcd2a51c in_seq=0 out_seq=0 2024-06-10T01:15:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.461+0000 7f84817fa700 10 client.17862 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:15:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.462+0000 7f84817fa700 1 -- 172.21.15.7:0/705034916 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa2899b2d0 con 0x55fa28ac68b0 2024-06-10T01:15:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.462+0000 7f84817fa700 10 client.17862 handle_command_reply: tid=0 2024-06-10T01:15:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.462+0000 7f84aa4a3b80 1 client.17862 shutdown 2024-06-10T01:15:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.462+0000 7f84aa4a3b80 1 -- 172.21.15.7:0/705034916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa28a47f60 msgr2=0x55fa28a521f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.462+0000 7f84aa4a3b80 1 --2- 172.21.15.7:0/705034916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa28a47f60 0x55fa28a521f0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f848809b710 tx=0x7f84880cf170 comp rx=0 tx=0).stop 2024-06-10T01:15:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.462+0000 7f84aa4a3b80 1 -- 172.21.15.7:0/705034916 shutdown_connections 2024-06-10T01:15:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.462+0000 7f84aa4a3b80 1 --2- 172.21.15.7:0/705034916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa28a48880 0x55fa28ac4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.462+0000 7f84aa4a3b80 1 --2- 172.21.15.7:0/705034916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa28a427a0 0x55fa28a51cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.462+0000 7f84aa4a3b80 1 --2- 172.21.15.7:0/705034916 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55fa28ac68b0 0x55fa28ac8cd0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.462+0000 7f84aa4a3b80 1 --2- 172.21.15.7:0/705034916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa28a47f60 0x55fa28a521f0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.462+0000 7f84817fa700 0 client.17862 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:15:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.462+0000 7f84aa4a3b80 1 -- 172.21.15.7:0/705034916 >> 172.21.15.7:0/705034916 conn(0x55fa2896f080 msgr2=0x55fa28a47320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.462+0000 7f84aa4a3b80 1 -- 172.21.15.7:0/705034916 shutdown_connections 2024-06-10T01:15:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.462+0000 7f84aa4a3b80 1 -- 172.21.15.7:0/705034916 wait complete. 2024-06-10T01:15:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.462+0000 7f84aa4a3b80 20 client.17862 trim_cache size 0 max 16384 2024-06-10T01:15:28.462 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:15:28.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.463+0000 7f849fc20700 1 -- 172.21.15.7:0/3382642778 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f84840a2c20 msgr2=0x7f84840a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:28.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.463+0000 7f849fc20700 1 --2- 172.21.15.7:0/3382642778 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f84840a2c20 0x7f84840a50e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8494005950 tx=0x7f849400a300 comp rx=0 tx=0).stop 2024-06-10T01:15:28.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.463+0000 7f849fc20700 1 -- 172.21.15.7:0/3382642778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84981030f0 msgr2=0x7f849819ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:28.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.463+0000 7f849fc20700 1 --2- 172.21.15.7:0/3382642778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84981030f0 0x7f849819ecc0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f848800d980 tx=0x7f848800dc90 comp rx=0 tx=0).stop 2024-06-10T01:15:28.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.463+0000 7f849fc20700 1 -- 172.21.15.7:0/3382642778 shutdown_connections 2024-06-10T01:15:28.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.463+0000 7f849fc20700 1 --2- 172.21.15.7:0/3382642778 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f84840a2c20 0x7f84840a50e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.463+0000 7f849fc20700 1 --2- 172.21.15.7:0/3382642778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8498107840 0x7f8498198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.463+0000 7f849fc20700 1 --2- 172.21.15.7:0/3382642778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8498102740 0x7f849819e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.463+0000 7f849fc20700 1 --2- 172.21.15.7:0/3382642778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84981030f0 0x7f849819ecc0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:28.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.463+0000 7f849fc20700 1 -- 172.21.15.7:0/3382642778 >> 172.21.15.7:0/3382642778 conn(0x7f84980795f0 msgr2=0x7f8498100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:28.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.463+0000 7f849fc20700 1 -- 172.21.15.7:0/3382642778 shutdown_connections 2024-06-10T01:15:28.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:28.463+0000 7f849fc20700 1 -- 172.21.15.7:0/3382642778 wait complete. 2024-06-10T01:15:28.744 INFO:tasks.workunit.client.0.smithi007.stdout: 1 676011 7.37 MB/sec execute 369 sec latency 8848.702 ms 2024-06-10T01:15:29.119 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:15:29.377 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:15:29.727 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:29 smithi007 ceph-mon[32398]: pgmap v716: 129 pgs: 129 active+clean; 115 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-06-10T01:15:29.744 INFO:tasks.workunit.client.0.smithi007.stdout: 1 676011 7.35 MB/sec execute 370 sec latency 9848.753 ms 2024-06-10T01:15:29.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:29 smithi078 ceph-mon[36683]: pgmap v716: 129 pgs: 129 active+clean; 115 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-06-10T01:15:29.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:29 smithi175 ceph-mon[36636]: pgmap v716: 129 pgs: 129 active+clean; 115 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-06-10T01:15:30.744 INFO:tasks.workunit.client.0.smithi007.stdout: 1 676011 7.33 MB/sec execute 371 sec latency 10848.840 ms 2024-06-10T01:15:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.136+0000 7f48bb6c0700 1 -- 172.21.15.7:0/2793888561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48b41085a0 msgr2=0x7f48b4108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.136+0000 7f48bb6c0700 1 --2- 172.21.15.7:0/2793888561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48b41085a0 0x7f48b4108980 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f48b0009990 tx=0x7f48b0009ca0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.136+0000 7f48bb6c0700 1 -- 172.21.15.7:0/2793888561 shutdown_connections 2024-06-10T01:15:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.136+0000 7f48bb6c0700 1 --2- 172.21.15.7:0/2793888561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48b4102f00 0x7f48b410f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.136+0000 7f48bb6c0700 1 --2- 172.21.15.7:0/2793888561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48b41085a0 0x7f48b4108980 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.136+0000 7f48bb6c0700 1 --2- 172.21.15.7:0/2793888561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48b4102540 0x7f48b41029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.136+0000 7f48bb6c0700 1 -- 172.21.15.7:0/2793888561 >> 172.21.15.7:0/2793888561 conn(0x7f48b4079560 msgr2=0x7f48b4079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:31.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.136+0000 7f48bb6c0700 1 -- 172.21.15.7:0/2793888561 shutdown_connections 2024-06-10T01:15:31.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.136+0000 7f48bb6c0700 1 -- 172.21.15.7:0/2793888561 wait complete. 2024-06-10T01:15:31.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.137+0000 7f48bb6c0700 1 Processor -- start 2024-06-10T01:15:31.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.137+0000 7f48bb6c0700 1 -- start start 2024-06-10T01:15:31.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.137+0000 7f48bb6c0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48b4102540 0x7f48b419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:31.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.137+0000 7f48bb6c0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48b4102f00 0x7f48b419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:31.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.137+0000 7f48bb6c0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48b41085a0 0x7f48b4198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:31.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.137+0000 7f48bb6c0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f48b4112850 con 0x7f48b4102f00 2024-06-10T01:15:31.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.137+0000 7f48bb6c0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f48b41126d0 con 0x7f48b4102540 2024-06-10T01:15:31.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.137+0000 7f48bb6c0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f48b41129d0 con 0x7f48b41085a0 2024-06-10T01:15:31.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.137+0000 7f48b8c5b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48b4102f00 0x7f48b419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:31.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.137+0000 7f48b8c5b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48b4102f00 0x7f48b419edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50152/0 (socket says 172.21.15.7:50152) 2024-06-10T01:15:31.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.137+0000 7f48b8c5b700 1 -- 172.21.15.7:0/2719960202 learned_addr learned my addr 172.21.15.7:0/2719960202 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:31.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.137+0000 7f48b945c700 1 --2- 172.21.15.7:0/2719960202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48b4102540 0x7f48b419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:31.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.138+0000 7f48b8c5b700 1 -- 172.21.15.7:0/2719960202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48b41085a0 msgr2=0x7f48b4198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:31.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.138+0000 7f48b9c5d700 1 --2- 172.21.15.7:0/2719960202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48b41085a0 0x7f48b4198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:31.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.138+0000 7f48b8c5b700 1 --2- 172.21.15.7:0/2719960202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48b41085a0 0x7f48b4198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.138+0000 7f48b8c5b700 1 -- 172.21.15.7:0/2719960202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48b4102540 msgr2=0x7f48b419e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:31.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.138+0000 7f48b8c5b700 1 --2- 172.21.15.7:0/2719960202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48b4102540 0x7f48b419e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.138+0000 7f48b8c5b700 1 -- 172.21.15.7:0/2719960202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b41991d0 con 0x7f48b4102f00 2024-06-10T01:15:31.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.138+0000 7f48b8c5b700 1 --2- 172.21.15.7:0/2719960202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48b4102f00 0x7f48b419edf0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f48a400c920 tx=0x7f48a400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:31.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.138+0000 7f48aa7fc700 1 -- 172.21.15.7:0/2719960202 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a4007b00 con 0x7f48b4102f00 2024-06-10T01:15:31.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.138+0000 7f48aa7fc700 1 -- 172.21.15.7:0/2719960202 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48a4004d10 con 0x7f48b4102f00 2024-06-10T01:15:31.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.138+0000 7f48bb6c0700 1 -- 172.21.15.7:0/2719960202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48b41994c0 con 0x7f48b4102f00 2024-06-10T01:15:31.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.138+0000 7f48aa7fc700 1 -- 172.21.15.7:0/2719960202 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a4005680 con 0x7f48b4102f00 2024-06-10T01:15:31.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.138+0000 7f48bb6c0700 1 -- 172.21.15.7:0/2719960202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48b41a5780 con 0x7f48b4102f00 2024-06-10T01:15:31.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.140+0000 7f48aa7fc700 1 -- 172.21.15.7:0/2719960202 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f48a4004750 con 0x7f48b4102f00 2024-06-10T01:15:31.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.141+0000 7f48aa7fc700 1 --2- 172.21.15.7:0/2719960202 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f48a00a2c20 0x7f48a00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:31.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.141+0000 7f48aa7fc700 1 -- 172.21.15.7:0/2719960202 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f48a40cef10 con 0x7f48b4102f00 2024-06-10T01:15:31.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.141+0000 7f48b945c700 1 --2- 172.21.15.7:0/2719960202 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f48a00a2c20 0x7f48a00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:31.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.142+0000 7f48b945c700 1 --2- 172.21.15.7:0/2719960202 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f48a00a2c20 0x7f48a00a50e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f48b4199bb0 tx=0x7f48b0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:31.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.142+0000 7f48c5f43b80 1 Processor -- start 2024-06-10T01:15:31.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.142+0000 7f48c5f43b80 1 -- start start 2024-06-10T01:15:31.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.143+0000 7f48c5f43b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a758faf470 0x55a758faf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:31.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.143+0000 7f48c5f43b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a758fa9670 0x55a758fa9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:31.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.143+0000 7f48c5f43b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a758faa030 0x55a758fb6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:31.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.143+0000 7f48c5f43b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a758e97f60 con 0x55a758fa9670 2024-06-10T01:15:31.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.143+0000 7f48c5f43b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a758ef6850 con 0x55a758faf470 2024-06-10T01:15:31.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.143+0000 7f48c5f43b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a758e5e460 con 0x55a758faa030 2024-06-10T01:15:31.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.143+0000 7f48b8c5b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a758fa9670 0x55a758fa9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:31.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.143+0000 7f48b8c5b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a758fa9670 0x55a758fa9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50156/0 (socket says 172.21.15.7:50156) 2024-06-10T01:15:31.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.143+0000 7f48b8c5b700 1 -- 172.21.15.7:0/2767607460 learned_addr learned my addr 172.21.15.7:0/2767607460 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:31.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.143+0000 7f48b945c700 1 --2- 172.21.15.7:0/2767607460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a758faf470 0x55a758faf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:31.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.143+0000 7f48b9c5d700 1 --2- 172.21.15.7:0/2767607460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a758faa030 0x55a758fb6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:31.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.143+0000 7f48b8c5b700 1 -- 172.21.15.7:0/2767607460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a758faa030 msgr2=0x55a758fb6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:31.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.143+0000 7f48b8c5b700 1 --2- 172.21.15.7:0/2767607460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a758faa030 0x55a758fb6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.143+0000 7f48b8c5b700 1 -- 172.21.15.7:0/2767607460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a758faf470 msgr2=0x55a758faf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:31.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.143+0000 7f48b8c5b700 1 --2- 172.21.15.7:0/2767607460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a758faf470 0x55a758faf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.143+0000 7f48b8c5b700 1 -- 172.21.15.7:0/2767607460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a758e458d0 con 0x55a758fa9670 2024-06-10T01:15:31.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.143+0000 7f48b945c700 1 --2- 172.21.15.7:0/2767607460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a758faf470 0x55a758faf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:15:31.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.143+0000 7f48b8c5b700 1 --2- 172.21.15.7:0/2767607460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a758fa9670 0x55a758fa9af0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f48a4097850 tx=0x7f48a409b2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deadd8cdd4358a73 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:31.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.143+0000 7f489f7fe700 1 -- 172.21.15.7:0/2767607460 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a401d040 con 0x55a758fa9670 2024-06-10T01:15:31.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.143+0000 7f489f7fe700 1 -- 172.21.15.7:0/2767607460 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48a409f7a0 con 0x55a758fa9670 2024-06-10T01:15:31.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.144+0000 7f489f7fe700 1 -- 172.21.15.7:0/2767607460 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a409b650 con 0x55a758fa9670 2024-06-10T01:15:31.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.144+0000 7f48c5f43b80 1 -- 172.21.15.7:0/2767607460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a758fa9670 msgr2=0x55a758fa9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:31.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.144+0000 7f48c5f43b80 1 --2- 172.21.15.7:0/2767607460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a758fa9670 0x55a758fa9af0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f48a4097850 tx=0x7f48a409b2f0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.144+0000 7f48c5f43b80 1 -- 172.21.15.7:0/2767607460 shutdown_connections 2024-06-10T01:15:31.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.144+0000 7f48c5f43b80 1 --2- 172.21.15.7:0/2767607460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a758faa030 0x55a758fb6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.144+0000 7f48c5f43b80 1 --2- 172.21.15.7:0/2767607460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a758faf470 0x55a758faf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.144+0000 7f48c5f43b80 1 --2- 172.21.15.7:0/2767607460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a758fa9670 0x55a758fa9af0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.144+0000 7f48c5f43b80 1 -- 172.21.15.7:0/2767607460 >> 172.21.15.7:0/2767607460 conn(0x55a758ed2080 msgr2=0x55a758fa0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:31.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.144+0000 7f48c5f43b80 1 -- 172.21.15.7:0/2767607460 shutdown_connections 2024-06-10T01:15:31.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.144+0000 7f48c5f43b80 1 -- 172.21.15.7:0/2767607460 wait complete. 2024-06-10T01:15:31.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.145+0000 7f48c5f43b80 1 -- start start 2024-06-10T01:15:31.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.145+0000 7f48c5f43b80 1 Processor -- start 2024-06-10T01:15:31.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.145+0000 7f48c5f43b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a758fa9670 0x55a758fb5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:31.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.145+0000 7f48c5f43b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a758faa030 0x55a758fb3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:31.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.145+0000 7f48c5f43b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a758faf470 0x55a758fb36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:31.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.145+0000 7f48c5f43b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a758e96fc0 con 0x55a758faa030 2024-06-10T01:15:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.145+0000 7f48c5f43b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a758ef2810 con 0x55a758fa9670 2024-06-10T01:15:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.145+0000 7f48c5f43b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a758fb6750 con 0x55a758faf470 2024-06-10T01:15:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.145+0000 7f48c5f43b80 10 client.0 _ll_register_callbacks cb 0x55a758e8c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:15:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.145+0000 7f48b8c5b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a758faa030 0x55a758fb3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.145+0000 7f48b8c5b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a758faa030 0x55a758fb3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50168/0 (socket says 172.21.15.7:50168) 2024-06-10T01:15:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.145+0000 7f48b8c5b700 1 -- 172.21.15.7:0/1533307769 learned_addr learned my addr 172.21.15.7:0/1533307769 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.146+0000 7f48b945c700 1 --2- 172.21.15.7:0/1533307769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a758fa9670 0x55a758fb5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.146+0000 7f48b9c5d700 1 --2- 172.21.15.7:0/1533307769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a758faf470 0x55a758fb36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.146+0000 7f48b8c5b700 1 -- 172.21.15.7:0/1533307769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a758faf470 msgr2=0x55a758fb36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.146+0000 7f48b8c5b700 1 --2- 172.21.15.7:0/1533307769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a758faf470 0x55a758fb36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.146+0000 7f48b8c5b700 1 -- 172.21.15.7:0/1533307769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a758fa9670 msgr2=0x55a758fb5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:31.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.146+0000 7f48b8c5b700 1 --2- 172.21.15.7:0/1533307769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a758fa9670 0x55a758fb5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.146+0000 7f48b8c5b700 1 -- 172.21.15.7:0/1533307769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a758ebc750 con 0x55a758faa030 2024-06-10T01:15:31.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.146+0000 7f48b9c5d700 1 --2- 172.21.15.7:0/1533307769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a758faf470 0x55a758fb36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:15:31.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.146+0000 7f48b945c700 1 --2- 172.21.15.7:0/1533307769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a758fa9670 0x55a758fb5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:15:31.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.146+0000 7f48b8c5b700 1 --2- 172.21.15.7:0/1533307769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a758faa030 0x55a758fb3180 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f48a40a3b30 tx=0x7f48a40a4110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c290703a2757ee2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:31.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.146+0000 7f489cff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:15:31.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.146+0000 7f489cff9700 1 -- 172.21.15.7:0/1533307769 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a40b8070 con 0x55a758faa030 2024-06-10T01:15:31.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.146+0000 7f489cff9700 1 -- 172.21.15.7:0/1533307769 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f48a40b4280 con 0x55a758faa030 2024-06-10T01:15:31.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.146+0000 7f48c5f43b80 1 -- 172.21.15.7:0/1533307769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48a409f7a0 con 0x55a758faa030 2024-06-10T01:15:31.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.146+0000 7f489cff9700 1 -- 172.21.15.7:0/1533307769 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f48a401d040 con 0x55a758faa030 2024-06-10T01:15:31.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.146+0000 7f48c5f43b80 10 client.17874 fetch_fsmap learned FSMap version 37 2024-06-10T01:15:31.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.146+0000 7f48c5f43b80 1 -- 172.21.15.7:0/1533307769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a758fc0da0 con 0x55a758faa030 2024-06-10T01:15:31.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.146+0000 7f489cff9700 1 -- 172.21.15.7:0/1533307769 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48a409f7a0 con 0x55a758faa030 2024-06-10T01:15:31.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.147+0000 7f489cff9700 1 -- 172.21.15.7:0/1533307769 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f48a40ccb50 con 0x55a758faa030 2024-06-10T01:15:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.147+0000 7f48c5f43b80 10 client.17874 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T01:15:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.147+0000 7f48c5f43b80 10 client.17874 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:15:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.147+0000 7f48c5f43b80 20 client.17874 populate_metadata read hostname 'smithi007' 2024-06-10T01:15:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.147+0000 7f48c5f43b80 1 --2- 172.21.15.7:0/1533307769 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a758fc3270 0x55a758fc5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.147+0000 7f48c5f43b80 4 client.17874 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:15:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.147+0000 7f48c5f43b80 1 -- 172.21.15.7:0/1533307769 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a758f03d50 con 0x55a758fc3270 2024-06-10T01:15:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.147+0000 7f48b945c700 1 --2- 172.21.15.7:0/1533307769 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a758fc3270 0x55a758fc5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.148+0000 7f48b945c700 1 --2- 172.21.15.7:0/1533307769 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a758fc3270 0x55a758fc5660 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=21f49efefe1318d7 server_cookie=dffe70dbf350a9d8 in_seq=0 out_seq=0 2024-06-10T01:15:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.148+0000 7f489cff9700 10 client.17874 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:15:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.148+0000 7f489cff9700 1 -- 172.21.15.7:0/1533307769 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a758f03d50 con 0x55a758fc3270 2024-06-10T01:15:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.148+0000 7f489cff9700 10 client.17874 handle_command_reply: tid=0 2024-06-10T01:15:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.148+0000 7f48c5f43b80 1 client.17874 shutdown 2024-06-10T01:15:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.149+0000 7f48c5f43b80 1 -- 172.21.15.7:0/1533307769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a758faa030 msgr2=0x55a758fb3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.149+0000 7f48c5f43b80 1 --2- 172.21.15.7:0/1533307769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a758faa030 0x55a758fb3180 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f48a40a3b30 tx=0x7f48a40a4110 comp rx=0 tx=0).stop 2024-06-10T01:15:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.149+0000 7f48c5f43b80 1 -- 172.21.15.7:0/1533307769 shutdown_connections 2024-06-10T01:15:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.149+0000 7f48c5f43b80 1 --2- 172.21.15.7:0/1533307769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a758faf470 0x55a758fb36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.149+0000 7f48c5f43b80 1 --2- 172.21.15.7:0/1533307769 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a758fc3270 0x55a758fc5660 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.149+0000 7f48c5f43b80 1 --2- 172.21.15.7:0/1533307769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a758fa9670 0x55a758fb5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.149+0000 7f489cff9700 0 client.17874 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:15:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.149+0000 7f48c5f43b80 1 --2- 172.21.15.7:0/1533307769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a758faa030 0x55a758fb3180 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.149+0000 7f48c5f43b80 1 -- 172.21.15.7:0/1533307769 >> 172.21.15.7:0/1533307769 conn(0x55a758ed2080 msgr2=0x55a758f9f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.149+0000 7f48c5f43b80 1 -- 172.21.15.7:0/1533307769 shutdown_connections 2024-06-10T01:15:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.149+0000 7f48c5f43b80 1 -- 172.21.15.7:0/1533307769 wait complete. 2024-06-10T01:15:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.149+0000 7f48c5f43b80 20 client.17874 trim_cache size 0 max 16384 2024-06-10T01:15:31.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.165+0000 7f48c5f43b80 1 Processor -- start 2024-06-10T01:15:31.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.165+0000 7f48c5f43b80 1 -- start start 2024-06-10T01:15:31.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.165+0000 7f48c5f43b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a758fa9670 0x55a758fb20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:31.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.165+0000 7f48c5f43b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a758faa030 0x55a758fb59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:31.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.165+0000 7f48c5f43b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a758faf470 0x55a758f9fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:31.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.165+0000 7f48c5f43b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f48b41aea60 con 0x55a758fa9670 2024-06-10T01:15:31.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.165+0000 7f48c5f43b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f48b41ac260 con 0x55a758faa030 2024-06-10T01:15:31.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.165+0000 7f48c5f43b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f48b41aa0b0 con 0x55a758faf470 2024-06-10T01:15:31.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.165+0000 7f48b945c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a758fa9670 0x55a758fb20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:31.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.165+0000 7f48b945c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a758fa9670 0x55a758fb20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50170/0 (socket says 172.21.15.7:50170) 2024-06-10T01:15:31.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.165+0000 7f48b945c700 1 -- 172.21.15.7:0/3424044163 learned_addr learned my addr 172.21.15.7:0/3424044163 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:31.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.166+0000 7f48b945c700 1 -- 172.21.15.7:0/3424044163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a758faf470 msgr2=0x55a758f9fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:31.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.166+0000 7f48b945c700 1 --2- 172.21.15.7:0/3424044163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a758faf470 0x55a758f9fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.166+0000 7f48b945c700 1 -- 172.21.15.7:0/3424044163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a758faa030 msgr2=0x55a758fb59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:31.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.166+0000 7f48b945c700 1 --2- 172.21.15.7:0/3424044163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a758faa030 0x55a758fb59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.166+0000 7f48b945c700 1 -- 172.21.15.7:0/3424044163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a758fb3a40 con 0x55a758fa9670 2024-06-10T01:15:31.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.166+0000 7f48b945c700 1 --2- 172.21.15.7:0/3424044163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a758fa9670 0x55a758fb20e0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f48b0000c00 tx=0x7f48b0023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f18c44869c00b3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:31.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.166+0000 7f489effd700 1 -- 172.21.15.7:0/3424044163 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b000fb70 con 0x55a758fa9670 2024-06-10T01:15:31.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.166+0000 7f489effd700 1 -- 172.21.15.7:0/3424044163 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48b000fd10 con 0x55a758fa9670 2024-06-10T01:15:31.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.166+0000 7f489effd700 1 -- 172.21.15.7:0/3424044163 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b0024610 con 0x55a758fa9670 2024-06-10T01:15:31.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.167+0000 7f48c5f43b80 1 -- 172.21.15.7:0/3424044163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a758fa9670 msgr2=0x55a758fb20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:31.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.167+0000 7f48c5f43b80 1 --2- 172.21.15.7:0/3424044163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a758fa9670 0x55a758fb20e0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f48b0000c00 tx=0x7f48b0023000 comp rx=0 tx=0).stop 2024-06-10T01:15:31.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.167+0000 7f48c5f43b80 1 -- 172.21.15.7:0/3424044163 shutdown_connections 2024-06-10T01:15:31.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.167+0000 7f48c5f43b80 1 --2- 172.21.15.7:0/3424044163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a758faf470 0x55a758f9fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.167+0000 7f48c5f43b80 1 --2- 172.21.15.7:0/3424044163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a758fa9670 0x55a758fb20e0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.167+0000 7f48c5f43b80 1 --2- 172.21.15.7:0/3424044163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a758faa030 0x55a758fb59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.167+0000 7f48c5f43b80 1 -- 172.21.15.7:0/3424044163 >> 172.21.15.7:0/3424044163 conn(0x55a758ed2080 msgr2=0x55a758fad6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:31.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.167+0000 7f48c5f43b80 1 -- 172.21.15.7:0/3424044163 shutdown_connections 2024-06-10T01:15:31.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.167+0000 7f48c5f43b80 1 -- 172.21.15.7:0/3424044163 wait complete. 2024-06-10T01:15:31.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.167+0000 7f48c5f43b80 1 -- start start 2024-06-10T01:15:31.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.167+0000 7f48c5f43b80 1 Processor -- start 2024-06-10T01:15:31.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.168+0000 7f48c5f43b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a758fa9670 0x55a758fc5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:31.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.168+0000 7f48c5f43b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a758faa030 0x55a758fc6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:31.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.168+0000 7f48c5f43b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a758faf470 0x55a75902bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:31.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.168+0000 7f48c5f43b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f48b41aa720 con 0x55a758faf470 2024-06-10T01:15:31.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.168+0000 7f48c5f43b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f48b41abb90 con 0x55a758faa030 2024-06-10T01:15:31.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.168+0000 7f48c5f43b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f48b41aadf0 con 0x55a758fa9670 2024-06-10T01:15:31.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.168+0000 7f48c5f43b80 10 client.0 _ll_register_callbacks cb 0x55a758e8c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:15:31.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.168+0000 7f48b9c5d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a758faf470 0x55a75902bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:31.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.168+0000 7f48b9c5d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a758faf470 0x55a75902bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50186/0 (socket says 172.21.15.7:50186) 2024-06-10T01:15:31.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.168+0000 7f48b9c5d700 1 -- 172.21.15.7:0/3087189537 learned_addr learned my addr 172.21.15.7:0/3087189537 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:31.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.168+0000 7f48b945c700 1 --2- 172.21.15.7:0/3087189537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a758fa9670 0x55a758fc5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:31.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.168+0000 7f48b8c5b700 1 --2- 172.21.15.7:0/3087189537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a758faa030 0x55a758fc6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:31.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.168+0000 7f48b9c5d700 1 -- 172.21.15.7:0/3087189537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a758fa9670 msgr2=0x55a758fc5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:31.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.168+0000 7f48b9c5d700 1 --2- 172.21.15.7:0/3087189537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a758fa9670 0x55a758fc5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.168+0000 7f48b9c5d700 1 -- 172.21.15.7:0/3087189537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a758faa030 msgr2=0x55a758fc6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:31.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.168+0000 7f48b9c5d700 1 --2- 172.21.15.7:0/3087189537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a758faa030 0x55a758fc6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.168+0000 7f48b9c5d700 1 -- 172.21.15.7:0/3087189537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a75902c1b0 con 0x55a758faf470 2024-06-10T01:15:31.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.168+0000 7f48b9c5d700 1 --2- 172.21.15.7:0/3087189537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a758faf470 0x55a75902bba0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f48ac010930 tx=0x7f48ac010c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b251df0370dcb0d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:31.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.168+0000 7f489cff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:15:31.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.169+0000 7f489cff9700 1 -- 172.21.15.7:0/3087189537 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ac019070 con 0x55a758faf470 2024-06-10T01:15:31.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.169+0000 7f48c5f43b80 1 -- 172.21.15.7:0/3087189537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48b000fd10 con 0x55a758faf470 2024-06-10T01:15:31.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.169+0000 7f489cff9700 1 -- 172.21.15.7:0/3087189537 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f48ac012840 con 0x55a758faf470 2024-06-10T01:15:31.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.169+0000 7f489cff9700 1 -- 172.21.15.7:0/3087189537 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f48ac025050 con 0x55a758faf470 2024-06-10T01:15:31.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.169+0000 7f48c5f43b80 10 client.17880 fetch_fsmap learned FSMap version 37 2024-06-10T01:15:31.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.169+0000 7f48c5f43b80 1 -- 172.21.15.7:0/3087189537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a75902d500 con 0x55a758faf470 2024-06-10T01:15:31.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.169+0000 7f489cff9700 1 -- 172.21.15.7:0/3087189537 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48b000fd10 con 0x55a758faf470 2024-06-10T01:15:31.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.169+0000 7f489cff9700 1 -- 172.21.15.7:0/3087189537 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f48ac013c20 con 0x55a758faf470 2024-06-10T01:15:31.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.169+0000 7f48c5f43b80 10 client.17880 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T01:15:31.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.169+0000 7f48c5f43b80 10 client.17880 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:15:31.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.169+0000 7f48c5f43b80 20 client.17880 populate_metadata read hostname 'smithi007' 2024-06-10T01:15:31.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.169+0000 7f48c5f43b80 1 --2- 172.21.15.7:0/3087189537 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a758fc3270 0x55a75902fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:31.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.169+0000 7f48c5f43b80 4 client.17880 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:15:31.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.169+0000 7f48c5f43b80 1 -- 172.21.15.7:0/3087189537 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a758efe2d0 con 0x55a758fc3270 2024-06-10T01:15:31.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.169+0000 7f48b945c700 1 --2- 172.21.15.7:0/3087189537 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a758fc3270 0x55a75902fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:31.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.170+0000 7f48b945c700 1 --2- 172.21.15.7:0/3087189537 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a758fc3270 0x55a75902fc90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b11fe1f203d8e8d4 server_cookie=4c9c1b9f8743efc9 in_seq=0 out_seq=0 2024-06-10T01:15:31.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.170+0000 7f489cff9700 10 client.17880 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:15:31.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.170+0000 7f489cff9700 1 -- 172.21.15.7:0/3087189537 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a758efe2d0 con 0x55a758fc3270 2024-06-10T01:15:31.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.170+0000 7f489cff9700 10 client.17880 handle_command_reply: tid=0 2024-06-10T01:15:31.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.170+0000 7f48c5f43b80 1 client.17880 shutdown 2024-06-10T01:15:31.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.170+0000 7f48c5f43b80 1 -- 172.21.15.7:0/3087189537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a758faf470 msgr2=0x55a75902bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:31.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.170+0000 7f48c5f43b80 1 --2- 172.21.15.7:0/3087189537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a758faf470 0x55a75902bba0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f48ac010930 tx=0x7f48ac010c40 comp rx=0 tx=0).stop 2024-06-10T01:15:31.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.170+0000 7f48c5f43b80 1 -- 172.21.15.7:0/3087189537 shutdown_connections 2024-06-10T01:15:31.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.170+0000 7f48c5f43b80 1 --2- 172.21.15.7:0/3087189537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a758faf470 0x55a75902bba0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.170+0000 7f48c5f43b80 1 --2- 172.21.15.7:0/3087189537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a758fa9670 0x55a758fc5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.170+0000 7f48c5f43b80 1 --2- 172.21.15.7:0/3087189537 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a758fc3270 0x55a75902fc90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.170+0000 7f48c5f43b80 1 --2- 172.21.15.7:0/3087189537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a758faa030 0x55a758fc6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.170+0000 7f48c5f43b80 1 -- 172.21.15.7:0/3087189537 >> 172.21.15.7:0/3087189537 conn(0x55a758ed2080 msgr2=0x55a758fa4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:31.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.170+0000 7f489cff9700 0 client.17880 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:15:31.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.171+0000 7f48c5f43b80 1 -- 172.21.15.7:0/3087189537 shutdown_connections 2024-06-10T01:15:31.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.171+0000 7f48c5f43b80 1 -- 172.21.15.7:0/3087189537 wait complete. 2024-06-10T01:15:31.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.171+0000 7f48c5f43b80 20 client.17880 trim_cache size 0 max 16384 2024-06-10T01:15:31.173 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:15:31.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.171+0000 7f48bb6c0700 1 -- 172.21.15.7:0/2719960202 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f48a00a2c20 msgr2=0x7f48a00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:31.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.171+0000 7f48bb6c0700 1 --2- 172.21.15.7:0/2719960202 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f48a00a2c20 0x7f48a00a50e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f48b4199bb0 tx=0x7f48b0005960 comp rx=0 tx=0).stop 2024-06-10T01:15:31.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.171+0000 7f48bb6c0700 1 -- 172.21.15.7:0/2719960202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48b4102f00 msgr2=0x7f48b419edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:31.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.171+0000 7f48bb6c0700 1 --2- 172.21.15.7:0/2719960202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48b4102f00 0x7f48b419edf0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f48a400c920 tx=0x7f48a400cc30 comp rx=0 tx=0).stop 2024-06-10T01:15:31.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.171+0000 7f48bb6c0700 1 -- 172.21.15.7:0/2719960202 shutdown_connections 2024-06-10T01:15:31.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.171+0000 7f48bb6c0700 1 --2- 172.21.15.7:0/2719960202 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f48a00a2c20 0x7f48a00a50e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.171+0000 7f48bb6c0700 1 --2- 172.21.15.7:0/2719960202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48b41085a0 0x7f48b4198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.171+0000 7f48bb6c0700 1 --2- 172.21.15.7:0/2719960202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48b4102540 0x7f48b419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.171+0000 7f48bb6c0700 1 --2- 172.21.15.7:0/2719960202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48b4102f00 0x7f48b419edf0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:31.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.171+0000 7f48bb6c0700 1 -- 172.21.15.7:0/2719960202 >> 172.21.15.7:0/2719960202 conn(0x7f48b4079560 msgr2=0x7f48b4101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:31.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.171+0000 7f48bb6c0700 1 -- 172.21.15.7:0/2719960202 shutdown_connections 2024-06-10T01:15:31.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:31.171+0000 7f48bb6c0700 1 -- 172.21.15.7:0/2719960202 wait complete. 2024-06-10T01:15:31.745 INFO:tasks.workunit.client.0.smithi007.stdout: 1 676011 7.31 MB/sec execute 372 sec latency 11848.929 ms 2024-06-10T01:15:31.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:31 smithi007 ceph-mon[32398]: pgmap v717: 129 pgs: 129 active+clean; 111 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 2.0 MiB/s wr, 196 op/s 2024-06-10T01:15:31.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:31 smithi078 ceph-mon[36683]: pgmap v717: 129 pgs: 129 active+clean; 111 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 2.0 MiB/s wr, 196 op/s 2024-06-10T01:15:31.954 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:31 smithi175 ceph-mon[36636]: pgmap v717: 129 pgs: 129 active+clean; 111 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 2.0 MiB/s wr, 196 op/s 2024-06-10T01:15:32.164 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:15:32.174 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:15:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.346+0000 7f8539f6a640 1 -- 172.21.15.7:0/2166074147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8534102c00 msgr2=0x7f8534103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.346+0000 7f8539f6a640 1 --2- 172.21.15.7:0/2166074147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8534102c00 0x7f8534103080 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f8528009900 tx=0x7f852802a410 comp rx=0 tx=0).stop 2024-06-10T01:15:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.347+0000 7f8539f6a640 1 -- 172.21.15.7:0/2166074147 shutdown_connections 2024-06-10T01:15:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.347+0000 7f8539f6a640 1 --2- 172.21.15.7:0/2166074147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85341035c0 0x7f8534109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.347+0000 7f8539f6a640 1 --2- 172.21.15.7:0/2166074147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8534102c00 0x7f8534103080 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.347+0000 7f8539f6a640 1 --2- 172.21.15.7:0/2166074147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8534101a10 0x7f8534101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.347+0000 7f8539f6a640 1 -- 172.21.15.7:0/2166074147 >> 172.21.15.7:0/2166074147 conn(0x7f85340fcfc0 msgr2=0x7f85340ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.347+0000 7f8539f6a640 1 -- 172.21.15.7:0/2166074147 shutdown_connections 2024-06-10T01:15:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.347+0000 7f8539f6a640 1 -- 172.21.15.7:0/2166074147 wait complete. 2024-06-10T01:15:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.347+0000 7f8539f6a640 1 Processor -- start 2024-06-10T01:15:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.347+0000 7f8539f6a640 1 -- start start 2024-06-10T01:15:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.348+0000 7f8539f6a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8534101a10 0x7f8534075750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.348+0000 7f8539f6a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8534102c00 0x7f8534073d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.348+0000 7f8539f6a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85341035c0 0x7f85340742d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.348+0000 7f8539f6a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f853410d690 con 0x7f8534101a10 2024-06-10T01:15:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.348+0000 7f8539f6a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f853410d510 con 0x7f8534102c00 2024-06-10T01:15:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.348+0000 7f8539f6a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f853410d810 con 0x7f85341035c0 2024-06-10T01:15:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.348+0000 7f85337fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8534101a10 0x7f8534075750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.348+0000 7f85337fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8534101a10 0x7f8534075750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51762/0 (socket says 172.21.15.7:51762) 2024-06-10T01:15:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.348+0000 7f85337fe640 1 -- 172.21.15.7:0/1021022828 learned_addr learned my addr 172.21.15.7:0/1021022828 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.348+0000 7f8533fff640 1 --2- 172.21.15.7:0/1021022828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85341035c0 0x7f85340742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.348+0000 7f8532ffd640 1 --2- 172.21.15.7:0/1021022828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8534102c00 0x7f8534073d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.348+0000 7f85337fe640 1 -- 172.21.15.7:0/1021022828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85341035c0 msgr2=0x7f85340742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.348+0000 7f85337fe640 1 --2- 172.21.15.7:0/1021022828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85341035c0 0x7f85340742d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.348+0000 7f85337fe640 1 -- 172.21.15.7:0/1021022828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8534102c00 msgr2=0x7f8534073d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.348+0000 7f85337fe640 1 --2- 172.21.15.7:0/1021022828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8534102c00 0x7f8534073d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.348+0000 7f85337fe640 1 -- 172.21.15.7:0/1021022828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8534074b60 con 0x7f8534101a10 2024-06-10T01:15:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.348+0000 7f85337fe640 1 --2- 172.21.15.7:0/1021022828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8534101a10 0x7f8534075750 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f852400e9c0 tx=0x7f852400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.349+0000 7f8530ff9640 1 -- 172.21.15.7:0/1021022828 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852400cd80 con 0x7f8534101a10 2024-06-10T01:15:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.349+0000 7f8530ff9640 1 -- 172.21.15.7:0/1021022828 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85240035f0 con 0x7f8534101a10 2024-06-10T01:15:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.349+0000 7f8539f6a640 1 -- 172.21.15.7:0/1021022828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85341a1e50 con 0x7f8534101a10 2024-06-10T01:15:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.349+0000 7f8530ff9640 1 -- 172.21.15.7:0/1021022828 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8524010690 con 0x7f8534101a10 2024-06-10T01:15:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.349+0000 7f8539f6a640 1 -- 172.21.15.7:0/1021022828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85341a2200 con 0x7f8534101a10 2024-06-10T01:15:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.352+0000 7f853cc71740 1 Processor -- start 2024-06-10T01:15:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.352+0000 7f853cc71740 1 -- start start 2024-06-10T01:15:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.352+0000 7f8530ff9640 1 -- 172.21.15.7:0/1021022828 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f8524010830 con 0x7f8534101a10 2024-06-10T01:15:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.352+0000 7f853cc71740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b761ea010 0x561b761ea430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.352+0000 7f853cc71740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561b761eaa00 0x561b761ef500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.352+0000 7f853cc71740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b761efb70 0x561b761f2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.352+0000 7f853cc71740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561b7614d4f0 con 0x561b761efb70 2024-06-10T01:15:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.352+0000 7f853cc71740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561b760eb870 con 0x561b761ea010 2024-06-10T01:15:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.352+0000 7f853cc71740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561b761494b0 con 0x561b761eaa00 2024-06-10T01:15:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.352+0000 7f8532ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b761efb70 0x561b761f2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.352+0000 7f8532ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b761efb70 0x561b761f2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51778/0 (socket says 172.21.15.7:51778) 2024-06-10T01:15:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.352+0000 7f8532ffd640 1 -- 172.21.15.7:0/1890716341 learned_addr learned my addr 172.21.15.7:0/1890716341 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.353+0000 7f8533fff640 1 --2- 172.21.15.7:0/1890716341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b761ea010 0x561b761ea430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.353+0000 7f85337fe640 1 --2- 172.21.15.7:0/1890716341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561b761eaa00 0x561b761ef500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.353+0000 7f8532ffd640 1 -- 172.21.15.7:0/1890716341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561b761eaa00 msgr2=0x561b761ef500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.353+0000 7f8532ffd640 1 --2- 172.21.15.7:0/1890716341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561b761eaa00 0x561b761ef500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.353+0000 7f8532ffd640 1 -- 172.21.15.7:0/1890716341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b761ea010 msgr2=0x561b761ea430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.353+0000 7f8532ffd640 1 --2- 172.21.15.7:0/1890716341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b761ea010 0x561b761ea430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.353+0000 7f8532ffd640 1 -- 172.21.15.7:0/1890716341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b761079b0 con 0x561b761efb70 2024-06-10T01:15:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.353+0000 7f8530ff9640 1 --2- 172.21.15.7:0/1021022828 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f851c0a2b80 0x7f851c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.353+0000 7f8532ffd640 1 --2- 172.21.15.7:0/1890716341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b761efb70 0x561b761f2000 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f852802a930 tx=0x7f8528032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b4d10e0d3f526f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.353+0000 7f8533fff640 1 --2- 172.21.15.7:0/1021022828 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f851c0a2b80 0x7f851c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.353+0000 7f8530ff9640 1 -- 172.21.15.7:0/1021022828 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f85240cf8a0 con 0x7f8534101a10 2024-06-10T01:15:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.353+0000 7f8519ffb640 1 -- 172.21.15.7:0/1890716341 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852803f070 con 0x561b761efb70 2024-06-10T01:15:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.354+0000 7f8519ffb640 1 -- 172.21.15.7:0/1890716341 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85280318b0 con 0x561b761efb70 2024-06-10T01:15:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.354+0000 7f8519ffb640 1 -- 172.21.15.7:0/1890716341 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8528035070 con 0x561b761efb70 2024-06-10T01:15:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.354+0000 7f853cc71740 1 -- 172.21.15.7:0/1890716341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b761efb70 msgr2=0x561b761f2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.354+0000 7f8533fff640 1 --2- 172.21.15.7:0/1021022828 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f851c0a2b80 0x7f851c0a5040 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x561b761eb260 tx=0x7f852c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.354+0000 7f853cc71740 1 --2- 172.21.15.7:0/1890716341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b761efb70 0x561b761f2000 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f852802a930 tx=0x7f8528032cb0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.354+0000 7f853cc71740 1 -- 172.21.15.7:0/1890716341 shutdown_connections 2024-06-10T01:15:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.354+0000 7f853cc71740 1 --2- 172.21.15.7:0/1890716341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b761efb70 0x561b761f2000 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.354+0000 7f853cc71740 1 --2- 172.21.15.7:0/1890716341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561b761eaa00 0x561b761ef500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.354+0000 7f853cc71740 1 --2- 172.21.15.7:0/1890716341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b761ea010 0x561b761ea430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.354+0000 7f853cc71740 1 -- 172.21.15.7:0/1890716341 >> 172.21.15.7:0/1890716341 conn(0x561b761e5680 msgr2=0x561b761e5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.354+0000 7f853cc71740 1 -- 172.21.15.7:0/1890716341 shutdown_connections 2024-06-10T01:15:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.354+0000 7f853cc71740 1 -- 172.21.15.7:0/1890716341 wait complete. 2024-06-10T01:15:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.355+0000 7f853cc71740 1 -- start start 2024-06-10T01:15:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.355+0000 7f853cc71740 1 Processor -- start 2024-06-10T01:15:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.355+0000 7f853cc71740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561b761ea010 0x561b761f4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.355+0000 7f853cc71740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b761eaa00 0x561b761f4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.356+0000 7f853cc71740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b761f9980 0x561b761fbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.356+0000 7f853cc71740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561b761f2ac0 con 0x561b761f9980 2024-06-10T01:15:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.356+0000 7f853cc71740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561b760d6ae0 con 0x561b761eaa00 2024-06-10T01:15:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.356+0000 7f853cc71740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561b761f2c40 con 0x561b761ea010 2024-06-10T01:15:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.356+0000 7f8532ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b761eaa00 0x561b761f4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.356+0000 7f85337fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561b761ea010 0x561b761f4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.356+0000 7f853cc71740 10 client.0 _ll_register_callbacks cb 0x561b7610f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:15:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.356+0000 7f8532ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b761eaa00 0x561b761f4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:45500/0 (socket says 172.21.15.7:45500) 2024-06-10T01:15:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.356+0000 7f8532ffd640 1 -- 172.21.15.7:0/1618995115 learned_addr learned my addr 172.21.15.7:0/1618995115 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.356+0000 7f8533fff640 1 --2- 172.21.15.7:0/1618995115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b761f9980 0x561b761fbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.356+0000 7f8532ffd640 1 -- 172.21.15.7:0/1618995115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561b761ea010 msgr2=0x561b761f4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.356+0000 7f8532ffd640 1 --2- 172.21.15.7:0/1618995115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561b761ea010 0x561b761f4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.356+0000 7f8532ffd640 1 -- 172.21.15.7:0/1618995115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b761f9980 msgr2=0x561b761fbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.356+0000 7f8532ffd640 1 --2- 172.21.15.7:0/1618995115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b761f9980 0x561b761fbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.356+0000 7f8532ffd640 1 -- 172.21.15.7:0/1618995115 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b75e61210 con 0x561b761eaa00 2024-06-10T01:15:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.356+0000 7f8532ffd640 1 --2- 172.21.15.7:0/1618995115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b761eaa00 0x561b761f4950 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8528031440 tx=0x7f852802aa90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=856687b48716ec0e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.357+0000 7f8512ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:15:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.357+0000 7f8512ffd640 1 -- 172.21.15.7:0/1618995115 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852804a020 con 0x561b761eaa00 2024-06-10T01:15:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.357+0000 7f853cc71740 1 -- 172.21.15.7:0/1618995115 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85280318b0 con 0x561b761eaa00 2024-06-10T01:15:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.357+0000 7f8512ffd640 1 -- 172.21.15.7:0/1618995115 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f8528003800 con 0x561b761eaa00 2024-06-10T01:15:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.357+0000 7f8512ffd640 1 -- 172.21.15.7:0/1618995115 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8528055050 con 0x561b761eaa00 2024-06-10T01:15:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.357+0000 7f853cc71740 10 client.27260 fetch_fsmap learned FSMap version 37 2024-06-10T01:15:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.357+0000 7f853cc71740 1 -- 172.21.15.7:0/1618995115 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561b75f2d620 con 0x561b761eaa00 2024-06-10T01:15:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.357+0000 7f8512ffd640 1 -- 172.21.15.7:0/1618995115 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85280318b0 con 0x561b761eaa00 2024-06-10T01:15:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.358+0000 7f8512ffd640 1 -- 172.21.15.7:0/1618995115 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f852803ba70 con 0x561b761eaa00 2024-06-10T01:15:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.358+0000 7f853cc71740 10 client.27260 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T01:15:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.358+0000 7f853cc71740 10 client.27260 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:15:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.358+0000 7f853cc71740 20 client.27260 populate_metadata read hostname 'smithi007' 2024-06-10T01:15:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.358+0000 7f853cc71740 1 --2- 172.21.15.7:0/1618995115 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x561b76200260 0x561b76202650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.358+0000 7f853cc71740 4 client.27260 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:15:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.358+0000 7f853cc71740 1 -- 172.21.15.7:0/1618995115 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b76160f10 con 0x561b76200260 2024-06-10T01:15:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.358+0000 7f85337fe640 1 --2- 172.21.15.7:0/1618995115 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x561b76200260 0x561b76202650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.359+0000 7f85337fe640 1 --2- 172.21.15.7:0/1618995115 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x561b76200260 0x561b76202650 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e11e87d6b259c08e server_cookie=48daaa24f447f26 in_seq=0 out_seq=0 2024-06-10T01:15:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.359+0000 7f8512ffd640 10 client.27260 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.359+0000 7f8512ffd640 1 -- 172.21.15.7:0/1618995115 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561b76160f10 con 0x561b76200260 2024-06-10T01:15:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.359+0000 7f8512ffd640 10 client.27260 handle_command_reply: tid=0 2024-06-10T01:15:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.359+0000 7f853cc71740 1 client.27260 shutdown 2024-06-10T01:15:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.360+0000 7f853cc71740 1 -- 172.21.15.7:0/1618995115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b761eaa00 msgr2=0x561b761f4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.360+0000 7f853cc71740 1 --2- 172.21.15.7:0/1618995115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b761eaa00 0x561b761f4950 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8528031440 tx=0x7f852802aa90 comp rx=0 tx=0).stop 2024-06-10T01:15:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.360+0000 7f853cc71740 1 -- 172.21.15.7:0/1618995115 shutdown_connections 2024-06-10T01:15:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.360+0000 7f853cc71740 1 --2- 172.21.15.7:0/1618995115 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x561b76200260 0x561b76202650 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.360+0000 7f853cc71740 1 --2- 172.21.15.7:0/1618995115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b761f9980 0x561b761fbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.360+0000 7f853cc71740 1 --2- 172.21.15.7:0/1618995115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b761eaa00 0x561b761f4950 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.360+0000 7f8512ffd640 0 client.27260 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.360+0000 7f853cc71740 1 --2- 172.21.15.7:0/1618995115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561b761ea010 0x561b761f4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.360+0000 7f853cc71740 1 -- 172.21.15.7:0/1618995115 >> 172.21.15.7:0/1618995115 conn(0x561b761e5680 msgr2=0x561b761e47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.360+0000 7f853cc71740 1 -- 172.21.15.7:0/1618995115 shutdown_connections 2024-06-10T01:15:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.360+0000 7f853cc71740 1 -- 172.21.15.7:0/1618995115 wait complete. 2024-06-10T01:15:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.360+0000 7f853cc71740 20 client.27260 trim_cache size 0 max 16384 2024-06-10T01:15:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.376+0000 7f853cc71740 1 Processor -- start 2024-06-10T01:15:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.376+0000 7f853cc71740 1 -- start start 2024-06-10T01:15:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.376+0000 7f853cc71740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b761ea010 0x561b761ebe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.376+0000 7f853cc71740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b761eaa00 0x561b761ec3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.376+0000 7f853cc71740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561b761f9980 0x561b761e8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.376+0000 7f853cc71740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f85341af0c0 con 0x561b761ea010 2024-06-10T01:15:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.376+0000 7f853cc71740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f85341ac900 con 0x561b761eaa00 2024-06-10T01:15:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.376+0000 7f853cc71740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f85341aa750 con 0x561b761f9980 2024-06-10T01:15:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.377+0000 7f85337fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b761ea010 0x561b761ebe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.377+0000 7f85337fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b761ea010 0x561b761ebe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51792/0 (socket says 172.21.15.7:51792) 2024-06-10T01:15:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.377+0000 7f85337fe640 1 -- 172.21.15.7:0/1019356449 learned_addr learned my addr 172.21.15.7:0/1019356449 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.377+0000 7f8532ffd640 1 --2- 172.21.15.7:0/1019356449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b761eaa00 0x561b761ec3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.377+0000 7f8533fff640 1 --2- 172.21.15.7:0/1019356449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561b761f9980 0x561b761e8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.377+0000 7f85337fe640 1 -- 172.21.15.7:0/1019356449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561b761f9980 msgr2=0x561b761e8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.377+0000 7f85337fe640 1 --2- 172.21.15.7:0/1019356449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561b761f9980 0x561b761e8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.377+0000 7f85337fe640 1 -- 172.21.15.7:0/1019356449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b761eaa00 msgr2=0x561b761ec3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.377+0000 7f85337fe640 1 --2- 172.21.15.7:0/1019356449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b761eaa00 0x561b761ec3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.377+0000 7f85337fe640 1 -- 172.21.15.7:0/1019356449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b762029c0 con 0x561b761ea010 2024-06-10T01:15:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.377+0000 7f85337fe640 1 --2- 172.21.15.7:0/1019356449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b761ea010 0x561b761ebe60 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f8524098750 tx=0x7f85240ccef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ffe37cc360add9e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.377+0000 7f85197fa640 1 -- 172.21.15.7:0/1019356449 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852409b8c0 con 0x561b761ea010 2024-06-10T01:15:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.377+0000 7f85197fa640 1 -- 172.21.15.7:0/1019356449 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f852409c480 con 0x561b761ea010 2024-06-10T01:15:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.377+0000 7f85197fa640 1 -- 172.21.15.7:0/1019356449 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852409b6e0 con 0x561b761ea010 2024-06-10T01:15:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.378+0000 7f853cc71740 1 -- 172.21.15.7:0/1019356449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b761ea010 msgr2=0x561b761ebe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.378+0000 7f853cc71740 1 --2- 172.21.15.7:0/1019356449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b761ea010 0x561b761ebe60 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f8524098750 tx=0x7f85240ccef0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.378+0000 7f853cc71740 1 -- 172.21.15.7:0/1019356449 shutdown_connections 2024-06-10T01:15:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.378+0000 7f853cc71740 1 --2- 172.21.15.7:0/1019356449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561b761f9980 0x561b761e8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.378+0000 7f853cc71740 1 --2- 172.21.15.7:0/1019356449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b761eaa00 0x561b761ec3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.378+0000 7f853cc71740 1 --2- 172.21.15.7:0/1019356449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b761ea010 0x561b761ebe60 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.378+0000 7f853cc71740 1 -- 172.21.15.7:0/1019356449 >> 172.21.15.7:0/1019356449 conn(0x561b761e5680 msgr2=0x561b761e5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.378+0000 7f853cc71740 1 -- 172.21.15.7:0/1019356449 shutdown_connections 2024-06-10T01:15:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.378+0000 7f853cc71740 1 -- 172.21.15.7:0/1019356449 wait complete. 2024-06-10T01:15:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.378+0000 7f853cc71740 1 -- start start 2024-06-10T01:15:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.379+0000 7f853cc71740 1 Processor -- start 2024-06-10T01:15:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.379+0000 7f853cc71740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561b761ea010 0x561b76211a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.379+0000 7f853cc71740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b761eaa00 0x561b76211fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.379+0000 7f853cc71740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b761f9980 0x561b76212fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.379+0000 7f853cc71740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f85341aadc0 con 0x561b761f9980 2024-06-10T01:15:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.379+0000 7f853cc71740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f85341ac230 con 0x561b761eaa00 2024-06-10T01:15:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.379+0000 7f853cc71740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f85341ab490 con 0x561b761ea010 2024-06-10T01:15:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.379+0000 7f853cc71740 10 client.0 _ll_register_callbacks cb 0x7f852c0099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:15:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.379+0000 7f8533fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b761f9980 0x561b76212fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.379+0000 7f8533fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b761f9980 0x561b76212fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51794/0 (socket says 172.21.15.7:51794) 2024-06-10T01:15:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.379+0000 7f8533fff640 1 -- 172.21.15.7:0/3365148362 learned_addr learned my addr 172.21.15.7:0/3365148362 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.379+0000 7f8532ffd640 1 --2- 172.21.15.7:0/3365148362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b761eaa00 0x561b76211fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.379+0000 7f85337fe640 1 --2- 172.21.15.7:0/3365148362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561b761ea010 0x561b76211a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.379+0000 7f8533fff640 1 -- 172.21.15.7:0/3365148362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561b761ea010 msgr2=0x561b76211a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.379+0000 7f8533fff640 1 --2- 172.21.15.7:0/3365148362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561b761ea010 0x561b76211a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.379+0000 7f8533fff640 1 -- 172.21.15.7:0/3365148362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b761eaa00 msgr2=0x561b76211fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.379+0000 7f8533fff640 1 --2- 172.21.15.7:0/3365148362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b761eaa00 0x561b76211fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.379+0000 7f8533fff640 1 -- 172.21.15.7:0/3365148362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b762135c0 con 0x561b761f9980 2024-06-10T01:15:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.379+0000 7f8532ffd640 1 --2- 172.21.15.7:0/3365148362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b761eaa00 0x561b76211fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.380+0000 7f8533fff640 1 --2- 172.21.15.7:0/3365148362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b761f9980 0x561b76212fd0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f852c01ea00 tx=0x7f852c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f57e77b8addbfa6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.380+0000 7f8512ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:15:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.380+0000 7f8512ffd640 1 -- 172.21.15.7:0/3365148362 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c02d060 con 0x561b761f9980 2024-06-10T01:15:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.380+0000 7f8512ffd640 1 -- 172.21.15.7:0/3365148362 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f852c0119c0 con 0x561b761f9980 2024-06-10T01:15:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.380+0000 7f853cc71740 1 -- 172.21.15.7:0/3365148362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f852409c480 con 0x561b761f9980 2024-06-10T01:15:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.380+0000 7f8512ffd640 1 -- 172.21.15.7:0/3365148362 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f852c034050 con 0x561b761f9980 2024-06-10T01:15:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.380+0000 7f853cc71740 10 client.17898 fetch_fsmap learned FSMap version 37 2024-06-10T01:15:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.380+0000 7f853cc71740 1 -- 172.21.15.7:0/3365148362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561b7626ada0 con 0x561b761f9980 2024-06-10T01:15:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.380+0000 7f8512ffd640 1 -- 172.21.15.7:0/3365148362 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f852409c480 con 0x561b761f9980 2024-06-10T01:15:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.380+0000 7f8512ffd640 1 -- 172.21.15.7:0/3365148362 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f852c01c3f0 con 0x561b761f9980 2024-06-10T01:15:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.381+0000 7f853cc71740 10 client.17898 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T01:15:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.381+0000 7f853cc71740 10 client.17898 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:15:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.381+0000 7f853cc71740 20 client.17898 populate_metadata read hostname 'smithi007' 2024-06-10T01:15:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.381+0000 7f853cc71740 1 --2- 172.21.15.7:0/3365148362 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x561b7626b710 0x561b7626db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.381+0000 7f853cc71740 4 client.17898 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:15:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.381+0000 7f853cc71740 1 -- 172.21.15.7:0/3365148362 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561b7615ac20 con 0x561b7626b710 2024-06-10T01:15:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.381+0000 7f85337fe640 1 --2- 172.21.15.7:0/3365148362 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x561b7626b710 0x561b7626db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.381+0000 7f8512ffd640 10 client.17898 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.381+0000 7f85337fe640 1 --2- 172.21.15.7:0/3365148362 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x561b7626b710 0x561b7626db30 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a08b83e9ccef293 server_cookie=f8551a60d51d13e7 in_seq=0 out_seq=0 2024-06-10T01:15:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.422+0000 7f8512ffd640 1 -- 172.21.15.7:0/3365148362 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561b7615ac20 con 0x561b7626b710 2024-06-10T01:15:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.422+0000 7f8512ffd640 10 client.17898 handle_command_reply: tid=0 2024-06-10T01:15:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.422+0000 7f853cc71740 1 client.17898 shutdown 2024-06-10T01:15:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.422+0000 7f853cc71740 1 -- 172.21.15.7:0/3365148362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b761f9980 msgr2=0x561b76212fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.422+0000 7f853cc71740 1 --2- 172.21.15.7:0/3365148362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b761f9980 0x561b76212fd0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f852c01ea00 tx=0x7f852c016000 comp rx=0 tx=0).stop 2024-06-10T01:15:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.422+0000 7f853cc71740 1 -- 172.21.15.7:0/3365148362 shutdown_connections 2024-06-10T01:15:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.422+0000 7f853cc71740 1 --2- 172.21.15.7:0/3365148362 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x561b7626b710 0x561b7626db30 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.422+0000 7f853cc71740 1 --2- 172.21.15.7:0/3365148362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b761f9980 0x561b76212fd0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.422+0000 7f853cc71740 1 --2- 172.21.15.7:0/3365148362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b761eaa00 0x561b76211fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.422+0000 7f853cc71740 1 --2- 172.21.15.7:0/3365148362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561b761ea010 0x561b76211a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.422+0000 7f8512ffd640 0 client.17898 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.422+0000 7f853cc71740 1 -- 172.21.15.7:0/3365148362 >> 172.21.15.7:0/3365148362 conn(0x561b761e5680 msgr2=0x561b761f43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.422+0000 7f853cc71740 1 -- 172.21.15.7:0/3365148362 shutdown_connections 2024-06-10T01:15:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.422+0000 7f853cc71740 1 -- 172.21.15.7:0/3365148362 wait complete. 2024-06-10T01:15:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.422+0000 7f853cc71740 20 client.17898 trim_cache size 0 max 16384 2024-06-10T01:15:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:15:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:15:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:15:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.423+0000 7f8539f6a640 1 -- 172.21.15.7:0/1021022828 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f851c0a2b80 msgr2=0x7f851c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.423+0000 7f8539f6a640 1 --2- 172.21.15.7:0/1021022828 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f851c0a2b80 0x7f851c0a5040 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x561b761eb260 tx=0x7f852c009500 comp rx=0 tx=0).stop 2024-06-10T01:15:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.423+0000 7f8539f6a640 1 -- 172.21.15.7:0/1021022828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8534101a10 msgr2=0x7f8534075750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.423+0000 7f8539f6a640 1 --2- 172.21.15.7:0/1021022828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8534101a10 0x7f8534075750 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f852400e9c0 tx=0x7f852400ee80 comp rx=0 tx=0).stop 2024-06-10T01:15:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.423+0000 7f8539f6a640 1 -- 172.21.15.7:0/1021022828 shutdown_connections 2024-06-10T01:15:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.423+0000 7f8539f6a640 1 --2- 172.21.15.7:0/1021022828 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f851c0a2b80 0x7f851c0a5040 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.423+0000 7f8539f6a640 1 --2- 172.21.15.7:0/1021022828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85341035c0 0x7f85340742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.423+0000 7f8539f6a640 1 --2- 172.21.15.7:0/1021022828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8534102c00 0x7f8534073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.423+0000 7f8539f6a640 1 --2- 172.21.15.7:0/1021022828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8534101a10 0x7f8534075750 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.423+0000 7f8539f6a640 1 -- 172.21.15.7:0/1021022828 >> 172.21.15.7:0/1021022828 conn(0x7f85340fcfc0 msgr2=0x7f8534105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.423+0000 7f8539f6a640 1 -- 172.21.15.7:0/1021022828 shutdown_connections 2024-06-10T01:15:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:32.423+0000 7f8539f6a640 1 -- 172.21.15.7:0/1021022828 wait complete. 2024-06-10T01:15:32.442 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:15:32.745 INFO:tasks.workunit.client.0.smithi007.stdout: 1 676011 7.29 MB/sec execute 373 sec latency 12849.016 ms 2024-06-10T01:15:33.148 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:15:33.550 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:33 smithi007 ceph-mon[32398]: pgmap v718: 129 pgs: 129 active+clean; 111 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-06-10T01:15:33.745 INFO:tasks.workunit.client.0.smithi007.stdout: 1 676011 7.27 MB/sec execute 374 sec latency 13849.424 ms 2024-06-10T01:15:33.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:33 smithi078 ceph-mon[36683]: pgmap v718: 129 pgs: 129 active+clean; 111 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-06-10T01:15:33.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:33 smithi175 ceph-mon[36636]: pgmap v718: 129 pgs: 129 active+clean; 111 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-06-10T01:15:34.745 INFO:tasks.workunit.client.0.smithi007.stdout: 1 676011 7.25 MB/sec execute 375 sec latency 14849.483 ms 2024-06-10T01:15:35.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.333+0000 7fdbcf551700 1 -- 172.21.15.7:0/467300791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbc810eec0 msgr2=0x7fdbc81112b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:35.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.333+0000 7fdbcf551700 1 --2- 172.21.15.7:0/467300791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbc810eec0 0x7fdbc81112b0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000b0a0 tx=0x7fdbc000b3b0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.333+0000 7fdbcf551700 1 -- 172.21.15.7:0/467300791 shutdown_connections 2024-06-10T01:15:35.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.333+0000 7fdbcf551700 1 --2- 172.21.15.7:0/467300791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbc810eec0 0x7fdbc81112b0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.333+0000 7fdbcf551700 1 --2- 172.21.15.7:0/467300791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbc80746c0 0x7fdbc8074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.333+0000 7fdbcf551700 1 --2- 172.21.15.7:0/467300791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbc8074fe0 0x7fdbc810e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.333+0000 7fdbcf551700 1 -- 172.21.15.7:0/467300791 >> 172.21.15.7:0/467300791 conn(0x7fdbc8079600 msgr2=0x7fdbc8079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:35.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.333+0000 7fdbcf551700 1 -- 172.21.15.7:0/467300791 shutdown_connections 2024-06-10T01:15:35.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.333+0000 7fdbcf551700 1 -- 172.21.15.7:0/467300791 wait complete. 2024-06-10T01:15:35.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.334+0000 7fdbcf551700 1 Processor -- start 2024-06-10T01:15:35.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.334+0000 7fdbcf551700 1 -- start start 2024-06-10T01:15:35.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.334+0000 7fdbcf551700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbc80746c0 0x7fdbc819ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:35.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.334+0000 7fdbcf551700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbc8074fe0 0x7fdbc819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:35.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.334+0000 7fdbcf551700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbc810eec0 0x7fdbc819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:35.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.334+0000 7fdbcf551700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc8114a20 con 0x7fdbc8074fe0 2024-06-10T01:15:35.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.334+0000 7fdbcf551700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc81148a0 con 0x7fdbc80746c0 2024-06-10T01:15:35.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.334+0000 7fdbcf551700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc8114ba0 con 0x7fdbc810eec0 2024-06-10T01:15:35.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.334+0000 7fdbccaec700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbc8074fe0 0x7fdbc819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:35.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.334+0000 7fdbccaec700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbc8074fe0 0x7fdbc819b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51818/0 (socket says 172.21.15.7:51818) 2024-06-10T01:15:35.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.334+0000 7fdbccaec700 1 -- 172.21.15.7:0/1150229578 learned_addr learned my addr 172.21.15.7:0/1150229578 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:35.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.334+0000 7fdbcdaee700 1 --2- 172.21.15.7:0/1150229578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbc810eec0 0x7fdbc819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:35.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.334+0000 7fdbcd2ed700 1 --2- 172.21.15.7:0/1150229578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbc80746c0 0x7fdbc819ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:35.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.334+0000 7fdbccaec700 1 -- 172.21.15.7:0/1150229578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbc810eec0 msgr2=0x7fdbc819f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:35.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.334+0000 7fdbccaec700 1 --2- 172.21.15.7:0/1150229578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbc810eec0 0x7fdbc819f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.334+0000 7fdbccaec700 1 -- 172.21.15.7:0/1150229578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbc80746c0 msgr2=0x7fdbc819ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:35.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.334+0000 7fdbccaec700 1 --2- 172.21.15.7:0/1150229578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbc80746c0 0x7fdbc819ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.334+0000 7fdbccaec700 1 -- 172.21.15.7:0/1150229578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc819fe10 con 0x7fdbc8074fe0 2024-06-10T01:15:35.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.334+0000 7fdbcdaee700 1 --2- 172.21.15.7:0/1150229578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbc810eec0 0x7fdbc819f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:15:35.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.335+0000 7fdbccaec700 1 --2- 172.21.15.7:0/1150229578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbc8074fe0 0x7fdbc819b390 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800c9b0 tx=0x7fdbb800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:35.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.335+0000 7fdbbe7fc700 1 -- 172.21.15.7:0/1150229578 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8007b00 con 0x7fdbc8074fe0 2024-06-10T01:15:35.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.335+0000 7fdbbe7fc700 1 -- 172.21.15.7:0/1150229578 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbb8007ca0 con 0x7fdbc8074fe0 2024-06-10T01:15:35.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.335+0000 7fdbcf551700 1 -- 172.21.15.7:0/1150229578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbc81a0100 con 0x7fdbc8074fe0 2024-06-10T01:15:35.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.335+0000 7fdbbe7fc700 1 -- 172.21.15.7:0/1150229578 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8018780 con 0x7fdbc8074fe0 2024-06-10T01:15:35.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.335+0000 7fdbcf551700 1 -- 172.21.15.7:0/1150229578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbc81a7980 con 0x7fdbc8074fe0 2024-06-10T01:15:35.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.336+0000 7fdbcf551700 1 -- 172.21.15.7:0/1150229578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbc8069180 con 0x7fdbc8074fe0 2024-06-10T01:15:35.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.337+0000 7fdbbe7fc700 1 -- 172.21.15.7:0/1150229578 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fdbb800f450 con 0x7fdbc8074fe0 2024-06-10T01:15:35.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.338+0000 7fdbbe7fc700 1 --2- 172.21.15.7:0/1150229578 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdbb40a2ca0 0x7fdbb40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:35.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.338+0000 7fdbbe7fc700 1 -- 172.21.15.7:0/1150229578 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fdbb80cf340 con 0x7fdbc8074fe0 2024-06-10T01:15:35.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.338+0000 7fdbcd2ed700 1 --2- 172.21.15.7:0/1150229578 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdbb40a2ca0 0x7fdbb40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:35.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.338+0000 7fdbcd2ed700 1 --2- 172.21.15.7:0/1150229578 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdbb40a2ca0 0x7fdbb40a5160 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4006fd0 tx=0x7fdbc4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:35.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.347+0000 7fdbbe7fc700 1 -- 172.21.15.7:0/1150229578 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdbb8014030 con 0x7fdbc8074fe0 2024-06-10T01:15:35.584 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:35 smithi007 ceph-mon[32398]: pgmap v719: 129 pgs: 129 active+clean; 110 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 1.2 MiB/s wr, 88 op/s 2024-06-10T01:15:35.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.606+0000 7fdbcf551700 1 -- 172.21.15.7:0/1150229578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbc81a7bb0 con 0x7fdbc8074fe0 2024-06-10T01:15:35.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.607+0000 7fdbbe7fc700 1 -- 172.21.15.7:0/1150229578 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11859 (secure 0 0 0) 0x7fdbb809bfc0 con 0x7fdbc8074fe0 2024-06-10T01:15:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:15:35.603 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:15:13.790052+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24451,24434,24436]},"id":1}]} 2024-06-10T01:15:35.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.609+0000 7fdbcf551700 1 -- 172.21.15.7:0/1150229578 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdbb40a2ca0 msgr2=0x7fdbb40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:35.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.609+0000 7fdbcf551700 1 --2- 172.21.15.7:0/1150229578 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdbb40a2ca0 0x7fdbb40a5160 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4006fd0 tx=0x7fdbc4008040 comp rx=0 tx=0).stop 2024-06-10T01:15:35.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.609+0000 7fdbcf551700 1 -- 172.21.15.7:0/1150229578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbc8074fe0 msgr2=0x7fdbc819b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:35.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.609+0000 7fdbcf551700 1 --2- 172.21.15.7:0/1150229578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbc8074fe0 0x7fdbc819b390 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800c9b0 tx=0x7fdbb800cd70 comp rx=0 tx=0).stop 2024-06-10T01:15:35.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.609+0000 7fdbcf551700 1 -- 172.21.15.7:0/1150229578 shutdown_connections 2024-06-10T01:15:35.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.609+0000 7fdbcf551700 1 --2- 172.21.15.7:0/1150229578 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdbb40a2ca0 0x7fdbb40a5160 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.609+0000 7fdbcf551700 1 --2- 172.21.15.7:0/1150229578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbc810eec0 0x7fdbc819f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.609+0000 7fdbcf551700 1 --2- 172.21.15.7:0/1150229578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbc80746c0 0x7fdbc819ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.609+0000 7fdbcf551700 1 --2- 172.21.15.7:0/1150229578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbc8074fe0 0x7fdbc819b390 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.609+0000 7fdbcf551700 1 -- 172.21.15.7:0/1150229578 >> 172.21.15.7:0/1150229578 conn(0x7fdbc8079600 msgr2=0x7fdbc80ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:35.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.609+0000 7fdbcf551700 1 -- 172.21.15.7:0/1150229578 shutdown_connections 2024-06-10T01:15:35.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.609+0000 7fdbcf551700 1 -- 172.21.15.7:0/1150229578 wait complete. 2024-06-10T01:15:35.607 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 37 2024-06-10T01:15:35.745 INFO:tasks.workunit.client.0.smithi007.stdout: 1 676011 7.23 MB/sec execute 376 sec latency 15849.535 ms 2024-06-10T01:15:35.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.802+0000 7f04df7e4700 1 -- 172.21.15.7:0/38248516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04d8102030 msgr2=0x7f04d8105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:35.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.802+0000 7f04df7e4700 1 --2- 172.21.15.7:0/38248516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04d8102030 0x7f04d8105ff0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f04c80099c0 tx=0x7f04c8009cd0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.802+0000 7f04df7e4700 1 -- 172.21.15.7:0/38248516 shutdown_connections 2024-06-10T01:15:35.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.802+0000 7f04df7e4700 1 --2- 172.21.15.7:0/38248516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d81066d0 0x7f04d810e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.802+0000 7f04df7e4700 1 --2- 172.21.15.7:0/38248516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04d8101680 0x7f04d8101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.802+0000 7f04df7e4700 1 --2- 172.21.15.7:0/38248516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04d8102030 0x7f04d8105ff0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.802+0000 7f04df7e4700 1 -- 172.21.15.7:0/38248516 >> 172.21.15.7:0/38248516 conn(0x7f04d80773e0 msgr2=0x7f04d80777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:35.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.802+0000 7f04df7e4700 1 -- 172.21.15.7:0/38248516 shutdown_connections 2024-06-10T01:15:35.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.802+0000 7f04df7e4700 1 -- 172.21.15.7:0/38248516 wait complete. 2024-06-10T01:15:35.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.803+0000 7f04df7e4700 1 Processor -- start 2024-06-10T01:15:35.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.803+0000 7f04df7e4700 1 -- start start 2024-06-10T01:15:35.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.803+0000 7f04df7e4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04d8101680 0x7f04d8103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:35.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.803+0000 7f04df7e4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04d8102030 0x7f04d81035d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:35.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.803+0000 7f04df7e4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d81066d0 0x7f04d8103b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:35.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.803+0000 7f04df7e4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f04d8111710 con 0x7f04d8101680 2024-06-10T01:15:35.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.803+0000 7f04df7e4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f04d8111590 con 0x7f04d81066d0 2024-06-10T01:15:35.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.803+0000 7f04df7e4700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f04d8111890 con 0x7f04d8102030 2024-06-10T01:15:35.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.803+0000 7f04dd580700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04d8101680 0x7f04d8103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:35.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.803+0000 7f04dd580700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04d8101680 0x7f04d8103090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51836/0 (socket says 172.21.15.7:51836) 2024-06-10T01:15:35.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.803+0000 7f04dd580700 1 -- 172.21.15.7:0/613208532 learned_addr learned my addr 172.21.15.7:0/613208532 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:35.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.804+0000 7f04dcd7f700 1 --2- 172.21.15.7:0/613208532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04d8102030 0x7f04d81035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:35.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.804+0000 7f04ddd81700 1 --2- 172.21.15.7:0/613208532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d81066d0 0x7f04d8103b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:35.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.804+0000 7f04dd580700 1 -- 172.21.15.7:0/613208532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04d8102030 msgr2=0x7f04d81035d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:35.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.804+0000 7f04dd580700 1 --2- 172.21.15.7:0/613208532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04d8102030 0x7f04d81035d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.804+0000 7f04dd580700 1 -- 172.21.15.7:0/613208532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d81066d0 msgr2=0x7f04d8103b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:35.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.804+0000 7f04dd580700 1 --2- 172.21.15.7:0/613208532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d81066d0 0x7f04d8103b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.804+0000 7f04dd580700 1 -- 172.21.15.7:0/613208532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d81a7210 con 0x7f04d8101680 2024-06-10T01:15:35.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.804+0000 7f04dd580700 1 --2- 172.21.15.7:0/613208532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04d8101680 0x7f04d8103090 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f04d400b530 tx=0x7f04d400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:35.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.804+0000 7f04ce7fc700 1 -- 172.21.15.7:0/613208532 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4015020 con 0x7f04d8101680 2024-06-10T01:15:35.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.804+0000 7f04ce7fc700 1 -- 172.21.15.7:0/613208532 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04d4011d10 con 0x7f04d8101680 2024-06-10T01:15:35.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.804+0000 7f04df7e4700 1 -- 172.21.15.7:0/613208532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d81a7500 con 0x7f04d8101680 2024-06-10T01:15:35.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.804+0000 7f04ce7fc700 1 -- 172.21.15.7:0/613208532 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4010e50 con 0x7f04d8101680 2024-06-10T01:15:35.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.804+0000 7f04df7e4700 1 -- 172.21.15.7:0/613208532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d81a7a10 con 0x7f04d8101680 2024-06-10T01:15:35.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.807+0000 7f04ce7fc700 1 -- 172.21.15.7:0/613208532 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f04d4018020 con 0x7f04d8101680 2024-06-10T01:15:35.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.807+0000 7f04ce7fc700 1 --2- 172.21.15.7:0/613208532 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f04c40a2dc0 0x7f04c40a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:35.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.808+0000 7f04ce7fc700 1 -- 172.21.15.7:0/613208532 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f04d40cec50 con 0x7f04d8101680 2024-06-10T01:15:35.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.808+0000 7f04dcd7f700 1 --2- 172.21.15.7:0/613208532 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f04c40a2dc0 0x7f04c40a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:35.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.808+0000 7f04dcd7f700 1 --2- 172.21.15.7:0/613208532 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f04c40a2dc0 0x7f04c40a5280 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f04c8000c00 tx=0x7f04c8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:35.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.808+0000 7f04ea067b80 1 Processor -- start 2024-06-10T01:15:35.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.808+0000 7f04ea067b80 1 -- start start 2024-06-10T01:15:35.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.809+0000 7f04ea067b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8d4e36f60 0x55b8d4e37340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:35.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.809+0000 7f04ea067b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8d4e37880 0x55b8d4e31260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:35.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.809+0000 7f04ea067b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8d4e317a0 0x55b8d4e31bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:35.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.809+0000 7f04ea067b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b8d4d23f60 con 0x55b8d4e317a0 2024-06-10T01:15:35.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.809+0000 7f04ea067b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b8d4d82850 con 0x55b8d4e37880 2024-06-10T01:15:35.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.809+0000 7f04ea067b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b8d4cea460 con 0x55b8d4e36f60 2024-06-10T01:15:35.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.809+0000 7f04ddd81700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8d4e317a0 0x55b8d4e31bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:35.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.809+0000 7f04ddd81700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8d4e317a0 0x55b8d4e31bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51842/0 (socket says 172.21.15.7:51842) 2024-06-10T01:15:35.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.809+0000 7f04ddd81700 1 -- 172.21.15.7:0/4280086533 learned_addr learned my addr 172.21.15.7:0/4280086533 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:35.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.809+0000 7f04dd580700 1 --2- 172.21.15.7:0/4280086533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8d4e36f60 0x55b8d4e37340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:35.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.809+0000 7f04dcd7f700 1 --2- 172.21.15.7:0/4280086533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8d4e37880 0x55b8d4e31260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:35.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.809+0000 7f04ddd81700 1 -- 172.21.15.7:0/4280086533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8d4e36f60 msgr2=0x55b8d4e37340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:35.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.809+0000 7f04ddd81700 1 --2- 172.21.15.7:0/4280086533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8d4e36f60 0x55b8d4e37340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.809+0000 7f04ddd81700 1 -- 172.21.15.7:0/4280086533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8d4e37880 msgr2=0x55b8d4e31260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:35.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.809+0000 7f04ddd81700 1 --2- 172.21.15.7:0/4280086533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8d4e37880 0x55b8d4e31260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.809+0000 7f04ddd81700 1 -- 172.21.15.7:0/4280086533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8d4cd18d0 con 0x55b8d4e317a0 2024-06-10T01:15:35.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.809+0000 7f04dd580700 1 --2- 172.21.15.7:0/4280086533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8d4e36f60 0x55b8d4e37340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:35.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.809+0000 7f04ddd81700 1 --2- 172.21.15.7:0/4280086533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8d4e317a0 0x55b8d4e31bc0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f04d0008550 tx=0x7f04d0008860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8872b97c264d088 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:35.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.810+0000 7f04c37fe700 1 -- 172.21.15.7:0/4280086533 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0003c00 con 0x55b8d4e317a0 2024-06-10T01:15:35.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.810+0000 7f04c37fe700 1 -- 172.21.15.7:0/4280086533 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04d0003da0 con 0x55b8d4e317a0 2024-06-10T01:15:35.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.810+0000 7f04c37fe700 1 -- 172.21.15.7:0/4280086533 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d00129a0 con 0x55b8d4e317a0 2024-06-10T01:15:35.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.810+0000 7f04ea067b80 1 -- 172.21.15.7:0/4280086533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8d4e317a0 msgr2=0x55b8d4e31bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:35.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.810+0000 7f04ea067b80 1 --2- 172.21.15.7:0/4280086533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8d4e317a0 0x55b8d4e31bc0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f04d0008550 tx=0x7f04d0008860 comp rx=0 tx=0).stop 2024-06-10T01:15:35.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.810+0000 7f04ea067b80 1 -- 172.21.15.7:0/4280086533 shutdown_connections 2024-06-10T01:15:35.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.810+0000 7f04ea067b80 1 --2- 172.21.15.7:0/4280086533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8d4e317a0 0x55b8d4e31bc0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.810+0000 7f04ea067b80 1 --2- 172.21.15.7:0/4280086533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8d4e36f60 0x55b8d4e37340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.810+0000 7f04ea067b80 1 --2- 172.21.15.7:0/4280086533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8d4e37880 0x55b8d4e31260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.810+0000 7f04ea067b80 1 -- 172.21.15.7:0/4280086533 >> 172.21.15.7:0/4280086533 conn(0x55b8d4d5e080 msgr2=0x55b8d4e2c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:35.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.811+0000 7f04ea067b80 1 -- 172.21.15.7:0/4280086533 shutdown_connections 2024-06-10T01:15:35.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.811+0000 7f04ea067b80 1 -- 172.21.15.7:0/4280086533 wait complete. 2024-06-10T01:15:35.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.811+0000 7f04ea067b80 1 -- start start 2024-06-10T01:15:35.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.811+0000 7f04ea067b80 1 Processor -- start 2024-06-10T01:15:35.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.811+0000 7f04ea067b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8d4e317a0 0x55b8d4e3d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:35.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.811+0000 7f04ea067b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8d4e36f60 0x55b8d4e3a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:35.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.811+0000 7f04ea067b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8d4e37880 0x55b8d4e3ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:35.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.811+0000 7f04ea067b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b8d4d22fc0 con 0x55b8d4e37880 2024-06-10T01:15:35.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.811+0000 7f04ea067b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b8d4d7e810 con 0x55b8d4e317a0 2024-06-10T01:15:35.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.811+0000 7f04ea067b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b8d4e3e260 con 0x55b8d4e36f60 2024-06-10T01:15:35.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.811+0000 7f04ea067b80 10 client.0 _ll_register_callbacks cb 0x55b8d4d185b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:15:35.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.811+0000 7f04ddd81700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8d4e37880 0x55b8d4e3ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:35.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.811+0000 7f04ddd81700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8d4e37880 0x55b8d4e3ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51850/0 (socket says 172.21.15.7:51850) 2024-06-10T01:15:35.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.812+0000 7f04ddd81700 1 -- 172.21.15.7:0/3796034759 learned_addr learned my addr 172.21.15.7:0/3796034759 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:35.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.812+0000 7f04dd580700 1 --2- 172.21.15.7:0/3796034759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8d4e317a0 0x55b8d4e3d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:35.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.812+0000 7f04dcd7f700 1 --2- 172.21.15.7:0/3796034759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8d4e36f60 0x55b8d4e3a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:35.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.812+0000 7f04ddd81700 1 -- 172.21.15.7:0/3796034759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8d4e36f60 msgr2=0x55b8d4e3a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:35.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.812+0000 7f04ddd81700 1 --2- 172.21.15.7:0/3796034759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8d4e36f60 0x55b8d4e3a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.812+0000 7f04ddd81700 1 -- 172.21.15.7:0/3796034759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8d4e317a0 msgr2=0x55b8d4e3d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:35.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.812+0000 7f04ddd81700 1 --2- 172.21.15.7:0/3796034759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8d4e317a0 0x55b8d4e3d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.812+0000 7f04ddd81700 1 -- 172.21.15.7:0/3796034759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8d4d48750 con 0x55b8d4e37880 2024-06-10T01:15:35.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.812+0000 7f04dcd7f700 1 --2- 172.21.15.7:0/3796034759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8d4e36f60 0x55b8d4e3a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:15:35.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.812+0000 7f04ddd81700 1 --2- 172.21.15.7:0/3796034759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8d4e37880 0x55b8d4e3ae90 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f04d00125e0 tx=0x7f04d0004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d4ee9e65a9845e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:35.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.812+0000 7f04c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:15:35.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.812+0000 7f04c0ff9700 1 -- 172.21.15.7:0/3796034759 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0021030 con 0x55b8d4e37880 2024-06-10T01:15:35.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.812+0000 7f04ea067b80 1 -- 172.21.15.7:0/3796034759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04d0003da0 con 0x55b8d4e37880 2024-06-10T01:15:35.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.812+0000 7f04c0ff9700 1 -- 172.21.15.7:0/3796034759 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f04d001a040 con 0x55b8d4e37880 2024-06-10T01:15:35.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.812+0000 7f04c0ff9700 1 -- 172.21.15.7:0/3796034759 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f04d002b050 con 0x55b8d4e37880 2024-06-10T01:15:35.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.812+0000 7f04ea067b80 10 client.17916 fetch_fsmap learned FSMap version 37 2024-06-10T01:15:35.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.812+0000 7f04ea067b80 1 -- 172.21.15.7:0/3796034759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b8d4e48a20 con 0x55b8d4e37880 2024-06-10T01:15:35.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.813+0000 7f04c0ff9700 1 -- 172.21.15.7:0/3796034759 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04d0003da0 con 0x55b8d4e37880 2024-06-10T01:15:35.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.813+0000 7f04c0ff9700 1 -- 172.21.15.7:0/3796034759 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f04d0013070 con 0x55b8d4e37880 2024-06-10T01:15:35.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.813+0000 7f04ea067b80 10 client.17916 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T01:15:35.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.813+0000 7f04ea067b80 10 client.17916 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:15:35.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.813+0000 7f04ea067b80 20 client.17916 populate_metadata read hostname 'smithi007' 2024-06-10T01:15:35.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.813+0000 7f04ea067b80 1 --2- 172.21.15.7:0/3796034759 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b8d4e4b000 0x55b8d4e4d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:35.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.813+0000 7f04ea067b80 4 client.17916 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:15:35.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.813+0000 7f04ea067b80 1 -- 172.21.15.7:0/3796034759 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8d4d8fd50 con 0x55b8d4e4b000 2024-06-10T01:15:35.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.813+0000 7f04dd580700 1 --2- 172.21.15.7:0/3796034759 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b8d4e4b000 0x55b8d4e4d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:35.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.814+0000 7f04dd580700 1 --2- 172.21.15.7:0/3796034759 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b8d4e4b000 0x55b8d4e4d3f0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84f691081b7f583e server_cookie=f8c9ebac3dfb36b2 in_seq=0 out_seq=0 2024-06-10T01:15:35.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.814+0000 7f04c0ff9700 10 client.17916 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:35.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.814+0000 7f04c0ff9700 1 -- 172.21.15.7:0/3796034759 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b8d4d8fd50 con 0x55b8d4e4b000 2024-06-10T01:15:35.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.814+0000 7f04c0ff9700 10 client.17916 handle_command_reply: tid=0 2024-06-10T01:15:35.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.814+0000 7f04ea067b80 1 client.17916 shutdown 2024-06-10T01:15:35.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.814+0000 7f04ea067b80 1 -- 172.21.15.7:0/3796034759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8d4e37880 msgr2=0x55b8d4e3ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:35.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.814+0000 7f04ea067b80 1 --2- 172.21.15.7:0/3796034759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8d4e37880 0x55b8d4e3ae90 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f04d00125e0 tx=0x7f04d0004200 comp rx=0 tx=0).stop 2024-06-10T01:15:35.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.815+0000 7f04ea067b80 1 -- 172.21.15.7:0/3796034759 shutdown_connections 2024-06-10T01:15:35.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.815+0000 7f04ea067b80 1 --2- 172.21.15.7:0/3796034759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8d4e37880 0x55b8d4e3ae90 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.815+0000 7f04ea067b80 1 --2- 172.21.15.7:0/3796034759 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b8d4e4b000 0x55b8d4e4d3f0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.815+0000 7f04ea067b80 1 --2- 172.21.15.7:0/3796034759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8d4e317a0 0x55b8d4e3d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.815+0000 7f04ea067b80 1 --2- 172.21.15.7:0/3796034759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8d4e36f60 0x55b8d4e3a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.815+0000 7f04c0ff9700 0 client.17916 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:35.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.815+0000 7f04ea067b80 1 -- 172.21.15.7:0/3796034759 >> 172.21.15.7:0/3796034759 conn(0x55b8d4d5e080 msgr2=0x55b8d4e2b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:35.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.815+0000 7f04ea067b80 1 -- 172.21.15.7:0/3796034759 shutdown_connections 2024-06-10T01:15:35.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.815+0000 7f04ea067b80 1 -- 172.21.15.7:0/3796034759 wait complete. 2024-06-10T01:15:35.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.815+0000 7f04ea067b80 20 client.17916 trim_cache size 0 max 16384 2024-06-10T01:15:35.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.830+0000 7f04ea067b80 1 Processor -- start 2024-06-10T01:15:35.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.830+0000 7f04ea067b80 1 -- start start 2024-06-10T01:15:35.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.830+0000 7f04ea067b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8d4e317a0 0x55b8d4e31260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:35.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.830+0000 7f04ea067b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8d4e36f60 0x55b8d4e382d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:35.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.830+0000 7f04ea067b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8d4e37880 0x55b8d4e39ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:35.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.830+0000 7f04ea067b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81b2ee0 con 0x55b8d4e37880 2024-06-10T01:15:35.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.830+0000 7f04ea067b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81b06e0 con 0x55b8d4e36f60 2024-06-10T01:15:35.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.830+0000 7f04ea067b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81ae530 con 0x55b8d4e317a0 2024-06-10T01:15:35.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.831+0000 7f04ddd81700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8d4e37880 0x55b8d4e39ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:35.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.831+0000 7f04ddd81700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8d4e37880 0x55b8d4e39ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51858/0 (socket says 172.21.15.7:51858) 2024-06-10T01:15:35.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.831+0000 7f04ddd81700 1 -- 172.21.15.7:0/1725847729 learned_addr learned my addr 172.21.15.7:0/1725847729 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:35.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.831+0000 7f04dd580700 1 --2- 172.21.15.7:0/1725847729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8d4e317a0 0x55b8d4e31260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:35.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.831+0000 7f04ddd81700 1 -- 172.21.15.7:0/1725847729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8d4e317a0 msgr2=0x55b8d4e31260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:35.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.831+0000 7f04ddd81700 1 --2- 172.21.15.7:0/1725847729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8d4e317a0 0x55b8d4e31260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.831+0000 7f04ddd81700 1 -- 172.21.15.7:0/1725847729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8d4e36f60 msgr2=0x55b8d4e382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:35.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.831+0000 7f04ddd81700 1 --2- 172.21.15.7:0/1725847729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8d4e36f60 0x55b8d4e382d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.831+0000 7f04ddd81700 1 -- 172.21.15.7:0/1725847729 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8d4e37680 con 0x55b8d4e37880 2024-06-10T01:15:35.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.831+0000 7f04dd580700 1 --2- 172.21.15.7:0/1725847729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8d4e317a0 0x55b8d4e31260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:15:35.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.831+0000 7f04ddd81700 1 --2- 172.21.15.7:0/1725847729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8d4e37880 0x55b8d4e39ad0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f04d00088a0 tx=0x7f04d0008ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9e22997b6e00af server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:35.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.831+0000 7f04c2ffd700 1 -- 172.21.15.7:0/1725847729 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0034070 con 0x55b8d4e37880 2024-06-10T01:15:35.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.831+0000 7f04c2ffd700 1 -- 172.21.15.7:0/1725847729 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04d0004320 con 0x55b8d4e37880 2024-06-10T01:15:35.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.831+0000 7f04c2ffd700 1 -- 172.21.15.7:0/1725847729 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0021040 con 0x55b8d4e37880 2024-06-10T01:15:35.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.832+0000 7f04ea067b80 1 -- 172.21.15.7:0/1725847729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8d4e37880 msgr2=0x55b8d4e39ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:35.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.832+0000 7f04ea067b80 1 --2- 172.21.15.7:0/1725847729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8d4e37880 0x55b8d4e39ad0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f04d00088a0 tx=0x7f04d0008ce0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.832+0000 7f04ea067b80 1 -- 172.21.15.7:0/1725847729 shutdown_connections 2024-06-10T01:15:35.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.832+0000 7f04ea067b80 1 --2- 172.21.15.7:0/1725847729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8d4e37880 0x55b8d4e39ad0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.832+0000 7f04ea067b80 1 --2- 172.21.15.7:0/1725847729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8d4e317a0 0x55b8d4e31260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.832+0000 7f04ea067b80 1 --2- 172.21.15.7:0/1725847729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8d4e36f60 0x55b8d4e382d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.832+0000 7f04ea067b80 1 -- 172.21.15.7:0/1725847729 >> 172.21.15.7:0/1725847729 conn(0x55b8d4d5e080 msgr2=0x55b8d4e3c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:35.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.832+0000 7f04ea067b80 1 -- 172.21.15.7:0/1725847729 shutdown_connections 2024-06-10T01:15:35.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.832+0000 7f04ea067b80 1 -- 172.21.15.7:0/1725847729 wait complete. 2024-06-10T01:15:35.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.832+0000 7f04ea067b80 1 -- start start 2024-06-10T01:15:35.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.832+0000 7f04ea067b80 1 Processor -- start 2024-06-10T01:15:35.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.833+0000 7f04ea067b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8d4e317a0 0x55b8d4e43520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:35.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.833+0000 7f04ea067b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8d4e36f60 0x55b8d4e43a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:35.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.833+0000 7f04ea067b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8d4e37880 0x55b8d4e3eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:35.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.833+0000 7f04ea067b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81aeba0 con 0x55b8d4e36f60 2024-06-10T01:15:35.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.833+0000 7f04ea067b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81b0010 con 0x55b8d4e317a0 2024-06-10T01:15:35.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.833+0000 7f04ea067b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81af270 con 0x55b8d4e37880 2024-06-10T01:15:35.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.833+0000 7f04ea067b80 10 client.0 _ll_register_callbacks cb 0x55b8d4d185b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:15:35.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.833+0000 7f04dcd7f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8d4e36f60 0x55b8d4e43a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:35.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.833+0000 7f04dcd7f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8d4e36f60 0x55b8d4e43a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51872/0 (socket says 172.21.15.7:51872) 2024-06-10T01:15:35.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.833+0000 7f04dcd7f700 1 -- 172.21.15.7:0/3667604894 learned_addr learned my addr 172.21.15.7:0/3667604894 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:35.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.833+0000 7f04dd580700 1 --2- 172.21.15.7:0/3667604894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8d4e317a0 0x55b8d4e43520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:35.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.833+0000 7f04ddd81700 1 --2- 172.21.15.7:0/3667604894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8d4e37880 0x55b8d4e3eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:35.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.833+0000 7f04dcd7f700 1 -- 172.21.15.7:0/3667604894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8d4e37880 msgr2=0x55b8d4e3eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:35.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.833+0000 7f04dcd7f700 1 --2- 172.21.15.7:0/3667604894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8d4e37880 0x55b8d4e3eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.833+0000 7f04dcd7f700 1 -- 172.21.15.7:0/3667604894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8d4e317a0 msgr2=0x55b8d4e43520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:35.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.833+0000 7f04dcd7f700 1 --2- 172.21.15.7:0/3667604894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8d4e317a0 0x55b8d4e43520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.833+0000 7f04dcd7f700 1 -- 172.21.15.7:0/3667604894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8d4e3f300 con 0x55b8d4e36f60 2024-06-10T01:15:35.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.833+0000 7f04dd580700 1 --2- 172.21.15.7:0/3667604894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8d4e317a0 0x55b8d4e43520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:15:35.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.833+0000 7f04ddd81700 1 --2- 172.21.15.7:0/3667604894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8d4e37880 0x55b8d4e3eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:15:35.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.833+0000 7f04dcd7f700 1 --2- 172.21.15.7:0/3667604894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8d4e36f60 0x55b8d4e43a60 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f04d806aa20 tx=0x7f04c8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9293ea0e0f8933f1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:35.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.833+0000 7f04c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:15:35.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.833+0000 7f04c0ff9700 1 -- 172.21.15.7:0/3667604894 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8026ac0 con 0x55b8d4e36f60 2024-06-10T01:15:35.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.834+0000 7f04c0ff9700 1 -- 172.21.15.7:0/3667604894 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f04c8026c60 con 0x55b8d4e36f60 2024-06-10T01:15:35.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.834+0000 7f04ea067b80 1 -- 172.21.15.7:0/3667604894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04d0004320 con 0x55b8d4e36f60 2024-06-10T01:15:35.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.834+0000 7f04c0ff9700 1 -- 172.21.15.7:0/3667604894 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f04c803e050 con 0x55b8d4e36f60 2024-06-10T01:15:35.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.834+0000 7f04ea067b80 10 client.17922 fetch_fsmap learned FSMap version 37 2024-06-10T01:15:35.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.834+0000 7f04ea067b80 1 -- 172.21.15.7:0/3667604894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b8d4eb53d0 con 0x55b8d4e36f60 2024-06-10T01:15:35.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.834+0000 7f04c0ff9700 1 -- 172.21.15.7:0/3667604894 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04d0004320 con 0x55b8d4e36f60 2024-06-10T01:15:35.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.834+0000 7f04c0ff9700 1 -- 172.21.15.7:0/3667604894 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f04c8039030 con 0x55b8d4e36f60 2024-06-10T01:15:35.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.834+0000 7f04ea067b80 10 client.17922 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T01:15:35.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.834+0000 7f04ea067b80 10 client.17922 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:15:35.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.834+0000 7f04ea067b80 20 client.17922 populate_metadata read hostname 'smithi007' 2024-06-10T01:15:35.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.834+0000 7f04ea067b80 1 --2- 172.21.15.7:0/3667604894 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b8d4eb5bc0 0x55b8d4eb7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:35.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.834+0000 7f04ea067b80 4 client.17922 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:15:35.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.834+0000 7f04ea067b80 1 -- 172.21.15.7:0/3667604894 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b8d4d8a2d0 con 0x55b8d4eb5bc0 2024-06-10T01:15:35.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.835+0000 7f04dd580700 1 --2- 172.21.15.7:0/3667604894 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b8d4eb5bc0 0x55b8d4eb7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:35.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.835+0000 7f04dd580700 1 --2- 172.21.15.7:0/3667604894 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b8d4eb5bc0 0x55b8d4eb7fe0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4224e5968027ab73 server_cookie=d7a3d58e2ba71460 in_seq=0 out_seq=0 2024-06-10T01:15:35.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.835+0000 7f04c0ff9700 10 client.17922 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:35.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.835+0000 7f04c0ff9700 1 -- 172.21.15.7:0/3667604894 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b8d4d8a2d0 con 0x55b8d4eb5bc0 2024-06-10T01:15:35.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.835+0000 7f04c0ff9700 10 client.17922 handle_command_reply: tid=0 2024-06-10T01:15:35.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.835+0000 7f04ea067b80 1 client.17922 shutdown 2024-06-10T01:15:35.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.835+0000 7f04ea067b80 1 -- 172.21.15.7:0/3667604894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8d4e36f60 msgr2=0x55b8d4e43a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:35.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.835+0000 7f04ea067b80 1 --2- 172.21.15.7:0/3667604894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8d4e36f60 0x55b8d4e43a60 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f04d806aa20 tx=0x7f04c8009310 comp rx=0 tx=0).stop 2024-06-10T01:15:35.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.835+0000 7f04ea067b80 1 -- 172.21.15.7:0/3667604894 shutdown_connections 2024-06-10T01:15:35.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.835+0000 7f04ea067b80 1 --2- 172.21.15.7:0/3667604894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8d4e37880 0x55b8d4e3eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.835+0000 7f04ea067b80 1 --2- 172.21.15.7:0/3667604894 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b8d4eb5bc0 0x55b8d4eb7fe0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.835+0000 7f04ea067b80 1 --2- 172.21.15.7:0/3667604894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8d4e317a0 0x55b8d4e43520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.835+0000 7f04ea067b80 1 --2- 172.21.15.7:0/3667604894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8d4e36f60 0x55b8d4e43a60 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.835+0000 7f04c0ff9700 0 client.17922 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:35.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.835+0000 7f04ea067b80 1 -- 172.21.15.7:0/3667604894 >> 172.21.15.7:0/3667604894 conn(0x55b8d4d5e080 msgr2=0x55b8d4e35a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:35.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.836+0000 7f04ea067b80 1 -- 172.21.15.7:0/3667604894 shutdown_connections 2024-06-10T01:15:35.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.836+0000 7f04ea067b80 1 -- 172.21.15.7:0/3667604894 wait complete. 2024-06-10T01:15:35.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.836+0000 7f04ea067b80 20 client.17922 trim_cache size 0 max 16384 2024-06-10T01:15:35.841 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:15:35.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.836+0000 7f04df7e4700 1 -- 172.21.15.7:0/613208532 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f04c40a2dc0 msgr2=0x7f04c40a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:35.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.836+0000 7f04df7e4700 1 --2- 172.21.15.7:0/613208532 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f04c40a2dc0 0x7f04c40a5280 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f04c8000c00 tx=0x7f04c8005fd0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.836+0000 7f04df7e4700 1 -- 172.21.15.7:0/613208532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04d8101680 msgr2=0x7f04d8103090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:35.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.836+0000 7f04df7e4700 1 --2- 172.21.15.7:0/613208532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04d8101680 0x7f04d8103090 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f04d400b530 tx=0x7f04d400b8f0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.836+0000 7f04df7e4700 1 -- 172.21.15.7:0/613208532 shutdown_connections 2024-06-10T01:15:35.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.836+0000 7f04df7e4700 1 --2- 172.21.15.7:0/613208532 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f04c40a2dc0 0x7f04c40a5280 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.836+0000 7f04df7e4700 1 --2- 172.21.15.7:0/613208532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d81066d0 0x7f04d8103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.836+0000 7f04df7e4700 1 --2- 172.21.15.7:0/613208532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04d8101680 0x7f04d8103090 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.836+0000 7f04df7e4700 1 --2- 172.21.15.7:0/613208532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04d8102030 0x7f04d81035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:35.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.836+0000 7f04df7e4700 1 -- 172.21.15.7:0/613208532 >> 172.21.15.7:0/613208532 conn(0x7f04d80773e0 msgr2=0x7f04d806aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:35.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.836+0000 7f04df7e4700 1 -- 172.21.15.7:0/613208532 shutdown_connections 2024-06-10T01:15:35.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:35.836+0000 7f04df7e4700 1 -- 172.21.15.7:0/613208532 wait complete. 2024-06-10T01:15:35.844 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:35 smithi078 ceph-mon[36683]: pgmap v719: 129 pgs: 129 active+clean; 110 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 1.2 MiB/s wr, 88 op/s 2024-06-10T01:15:35.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:35 smithi175 ceph-mon[36636]: pgmap v719: 129 pgs: 129 active+clean; 110 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 1.2 MiB/s wr, 88 op/s 2024-06-10T01:15:36.599 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:36 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1150229578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:15:36.745 INFO:tasks.workunit.client.0.smithi007.stdout: 1 676011 7.21 MB/sec execute 377 sec latency 16849.646 ms 2024-06-10T01:15:36.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:36 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1150229578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:15:36.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:36 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1150229578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:15:37.378 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db --set-id 844e91e0 --release --await' 2024-06-10T01:15:37.379 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db --set-id 844e91e0 --release --await 2024-06-10T01:15:37.637 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:15:37.745 INFO:tasks.workunit.client.0.smithi007.stdout: 1 676011 7.19 MB/sec execute 378 sec latency 17849.697 ms 2024-06-10T01:15:37.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:37 smithi007 ceph-mon[32398]: pgmap v720: 129 pgs: 129 active+clean; 109 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 711 KiB/s wr, 63 op/s 2024-06-10T01:15:37.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:37 smithi078 ceph-mon[36683]: pgmap v720: 129 pgs: 129 active+clean; 109 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 711 KiB/s wr, 63 op/s 2024-06-10T01:15:37.907 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:15:37.908 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:15:37.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:37 smithi175 ceph-mon[36636]: pgmap v720: 129 pgs: 129 active+clean; 109 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 711 KiB/s wr, 63 op/s 2024-06-10T01:15:38.394 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:15:38.745 INFO:tasks.workunit.client.0.smithi007.stdout: 1 676011 7.18 MB/sec execute 379 sec latency 18849.779 ms 2024-06-10T01:15:39.746 INFO:tasks.workunit.client.0.smithi007.stdout: 1 676011 7.16 MB/sec execute 380 sec latency 19849.830 ms 2024-06-10T01:15:39.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:39 smithi007 ceph-mon[32398]: pgmap v721: 129 pgs: 129 active+clean; 109 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 424 KiB/s wr, 51 op/s 2024-06-10T01:15:39.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:39 smithi078 ceph-mon[36683]: pgmap v721: 129 pgs: 129 active+clean; 109 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 424 KiB/s wr, 51 op/s 2024-06-10T01:15:39.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:39 smithi175 ceph-mon[36636]: pgmap v721: 129 pgs: 129 active+clean; 109 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 424 KiB/s wr, 51 op/s 2024-06-10T01:15:40.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.553+0000 7f82a9c5f700 1 -- 172.21.15.7:0/3743129944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82a4102ee0 msgr2=0x7f82a410f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:40.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.553+0000 7f82a9c5f700 1 --2- 172.21.15.7:0/3743129944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82a4102ee0 0x7f82a410f3c0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f829c00b0a0 tx=0x7f829c00b3b0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.554+0000 7f82a9c5f700 1 -- 172.21.15.7:0/3743129944 shutdown_connections 2024-06-10T01:15:40.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.554+0000 7f82a9c5f700 1 --2- 172.21.15.7:0/3743129944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82a4102ee0 0x7f82a410f3c0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.554+0000 7f82a9c5f700 1 --2- 172.21.15.7:0/3743129944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82a4107450 0x7f82a4107830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.554+0000 7f82a9c5f700 1 --2- 172.21.15.7:0/3743129944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82a4102540 0x7f82a41029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.554+0000 7f82a9c5f700 1 -- 172.21.15.7:0/3743129944 >> 172.21.15.7:0/3743129944 conn(0x7f82a40795f0 msgr2=0x7f82a4079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:40.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.554+0000 7f82a9c5f700 1 -- 172.21.15.7:0/3743129944 shutdown_connections 2024-06-10T01:15:40.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.554+0000 7f82a9c5f700 1 -- 172.21.15.7:0/3743129944 wait complete. 2024-06-10T01:15:40.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.554+0000 7f82a9c5f700 1 Processor -- start 2024-06-10T01:15:40.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.554+0000 7f82a9c5f700 1 -- start start 2024-06-10T01:15:40.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.555+0000 7f82a9c5f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82a4102540 0x7f82a4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:40.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.555+0000 7f82a9c5f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82a4102ee0 0x7f82a4074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:40.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.555+0000 7f82a9c5f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82a4107450 0x7f82a4075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:40.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.555+0000 7f82a9c5f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f82a41127e0 con 0x7f82a4102540 2024-06-10T01:15:40.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.555+0000 7f82a9c5f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f82a4112660 con 0x7f82a4102ee0 2024-06-10T01:15:40.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.555+0000 7f82a9c5f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f82a4112960 con 0x7f82a4107450 2024-06-10T01:15:40.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.555+0000 7f82a37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82a4102540 0x7f82a4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:40.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.555+0000 7f82a37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82a4102540 0x7f82a4074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51882/0 (socket says 172.21.15.7:51882) 2024-06-10T01:15:40.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.555+0000 7f82a37fe700 1 -- 172.21.15.7:0/90920470 learned_addr learned my addr 172.21.15.7:0/90920470 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:40.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.555+0000 7f82a3fff700 1 --2- 172.21.15.7:0/90920470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82a4107450 0x7f82a4075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:40.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.555+0000 7f82a2ffd700 1 --2- 172.21.15.7:0/90920470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82a4102ee0 0x7f82a4074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:40.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.555+0000 7f82a37fe700 1 -- 172.21.15.7:0/90920470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82a4107450 msgr2=0x7f82a4075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:40.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.555+0000 7f82a37fe700 1 --2- 172.21.15.7:0/90920470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82a4107450 0x7f82a4075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.555+0000 7f82a37fe700 1 -- 172.21.15.7:0/90920470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82a4102ee0 msgr2=0x7f82a4074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:40.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.555+0000 7f82a37fe700 1 --2- 172.21.15.7:0/90920470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82a4102ee0 0x7f82a4074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.555+0000 7f82a37fe700 1 -- 172.21.15.7:0/90920470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a41a70e0 con 0x7f82a4102540 2024-06-10T01:15:40.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.555+0000 7f82a2ffd700 1 --2- 172.21.15.7:0/90920470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82a4102ee0 0x7f82a4074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:15:40.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.555+0000 7f82a37fe700 1 --2- 172.21.15.7:0/90920470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82a4102540 0x7f82a4074790 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f829400e970 tx=0x7f829400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:40.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.556+0000 7f82a0ff9700 1 -- 172.21.15.7:0/90920470 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829400cbd0 con 0x7f82a4102540 2024-06-10T01:15:40.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.556+0000 7f82a0ff9700 1 -- 172.21.15.7:0/90920470 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8294004500 con 0x7f82a4102540 2024-06-10T01:15:40.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.556+0000 7f82a9c5f700 1 -- 172.21.15.7:0/90920470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82a41a73d0 con 0x7f82a4102540 2024-06-10T01:15:40.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.556+0000 7f82a0ff9700 1 -- 172.21.15.7:0/90920470 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294003e30 con 0x7f82a4102540 2024-06-10T01:15:40.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.556+0000 7f82a9c5f700 1 -- 172.21.15.7:0/90920470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82a41a7960 con 0x7f82a4102540 2024-06-10T01:15:40.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.558+0000 7f82a0ff9700 1 -- 172.21.15.7:0/90920470 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f8294004000 con 0x7f82a4102540 2024-06-10T01:15:40.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.559+0000 7f82a0ff9700 1 --2- 172.21.15.7:0/90920470 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f828c0a2ca0 0x7f828c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:40.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.559+0000 7f82a0ff9700 1 -- 172.21.15.7:0/90920470 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f82940cf580 con 0x7f82a4102540 2024-06-10T01:15:40.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.559+0000 7f82a2ffd700 1 --2- 172.21.15.7:0/90920470 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f828c0a2ca0 0x7f828c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:40.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.559+0000 7f82a2ffd700 1 --2- 172.21.15.7:0/90920470 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f828c0a2ca0 0x7f828c0a5160 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8298005fd0 tx=0x7f8298005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:40.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.560+0000 7f82b44e2b80 1 Processor -- start 2024-06-10T01:15:40.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.560+0000 7f82b44e2b80 1 -- start start 2024-06-10T01:15:40.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.560+0000 7f82b44e2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ea74ea500 0x564ea74ea8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:40.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.560+0000 7f82b44e2b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ea74eae20 0x564ea74e4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:40.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.560+0000 7f82b44e2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ea74e4c50 0x564ea74e5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:40.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.561+0000 7f82b44e2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564ea73d7060 con 0x564ea74e4c50 2024-06-10T01:15:40.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.561+0000 7f82b44e2b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564ea7435950 con 0x564ea74eae20 2024-06-10T01:15:40.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.561+0000 7f82b44e2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564ea739d560 con 0x564ea74ea500 2024-06-10T01:15:40.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.561+0000 7f82a3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ea74e4c50 0x564ea74e5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:40.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.561+0000 7f82a3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ea74e4c50 0x564ea74e5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51886/0 (socket says 172.21.15.7:51886) 2024-06-10T01:15:40.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.561+0000 7f82a3fff700 1 -- 172.21.15.7:0/839525412 learned_addr learned my addr 172.21.15.7:0/839525412 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:40.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.561+0000 7f82a37fe700 1 --2- 172.21.15.7:0/839525412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ea74ea500 0x564ea74ea8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:40.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.561+0000 7f82a2ffd700 1 --2- 172.21.15.7:0/839525412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ea74eae20 0x564ea74e4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:40.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.561+0000 7f82a3fff700 1 -- 172.21.15.7:0/839525412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ea74ea500 msgr2=0x564ea74ea8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:40.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.561+0000 7f82a3fff700 1 --2- 172.21.15.7:0/839525412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ea74ea500 0x564ea74ea8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.561+0000 7f82a3fff700 1 -- 172.21.15.7:0/839525412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ea74eae20 msgr2=0x564ea74e4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:40.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.561+0000 7f82a3fff700 1 --2- 172.21.15.7:0/839525412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ea74eae20 0x564ea74e4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.561+0000 7f82a3fff700 1 -- 172.21.15.7:0/839525412 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ea73849d0 con 0x564ea74e4c50 2024-06-10T01:15:40.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.561+0000 7f82a2ffd700 1 --2- 172.21.15.7:0/839525412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ea74eae20 0x564ea74e4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:15:40.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.561+0000 7f82a3fff700 1 --2- 172.21.15.7:0/839525412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ea74e4c50 0x564ea74e5070 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f829c009520 tx=0x7f829c009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ad1c5f6d8f6c0b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:40.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.561+0000 7f8289ffb700 1 -- 172.21.15.7:0/839525412 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c020070 con 0x564ea74e4c50 2024-06-10T01:15:40.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.561+0000 7f8289ffb700 1 -- 172.21.15.7:0/839525412 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f829c00bd30 con 0x564ea74e4c50 2024-06-10T01:15:40.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.561+0000 7f8289ffb700 1 -- 172.21.15.7:0/839525412 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c005040 con 0x564ea74e4c50 2024-06-10T01:15:40.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.562+0000 7f82b44e2b80 1 -- 172.21.15.7:0/839525412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ea74e4c50 msgr2=0x564ea74e5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:40.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.562+0000 7f82b44e2b80 1 --2- 172.21.15.7:0/839525412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ea74e4c50 0x564ea74e5070 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f829c009520 tx=0x7f829c009550 comp rx=0 tx=0).stop 2024-06-10T01:15:40.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.562+0000 7f82b44e2b80 1 -- 172.21.15.7:0/839525412 shutdown_connections 2024-06-10T01:15:40.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.562+0000 7f82b44e2b80 1 --2- 172.21.15.7:0/839525412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ea74e4c50 0x564ea74e5070 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.562+0000 7f82b44e2b80 1 --2- 172.21.15.7:0/839525412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ea74ea500 0x564ea74ea8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.562+0000 7f82b44e2b80 1 --2- 172.21.15.7:0/839525412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ea74eae20 0x564ea74e4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.562+0000 7f82b44e2b80 1 -- 172.21.15.7:0/839525412 >> 172.21.15.7:0/839525412 conn(0x564ea7411180 msgr2=0x564ea74dfa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:40.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.562+0000 7f82b44e2b80 1 -- 172.21.15.7:0/839525412 shutdown_connections 2024-06-10T01:15:40.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.562+0000 7f82b44e2b80 1 -- 172.21.15.7:0/839525412 wait complete. 2024-06-10T01:15:40.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.562+0000 7f82b44e2b80 1 -- start start 2024-06-10T01:15:40.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.563+0000 7f82b44e2b80 1 Processor -- start 2024-06-10T01:15:40.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.563+0000 7f82b44e2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ea74e4c50 0x564ea74f0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:40.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.563+0000 7f82b44e2b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ea74ea500 0x564ea74ede60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:40.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.563+0000 7f82b44e2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ea74eae20 0x564ea74ee3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:40.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.563+0000 7f82b44e2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564ea73d60c0 con 0x564ea74eae20 2024-06-10T01:15:40.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.563+0000 7f82b44e2b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564ea7431910 con 0x564ea74ea500 2024-06-10T01:15:40.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.563+0000 7f82b44e2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564ea74f17a0 con 0x564ea74e4c50 2024-06-10T01:15:40.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.563+0000 7f82b44e2b80 10 client.0 _ll_register_callbacks cb 0x564ea73cb6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:15:40.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.563+0000 7f82a3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ea74eae20 0x564ea74ee3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:40.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.563+0000 7f82a3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ea74eae20 0x564ea74ee3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51892/0 (socket says 172.21.15.7:51892) 2024-06-10T01:15:40.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.563+0000 7f82a3fff700 1 -- 172.21.15.7:0/3637081485 learned_addr learned my addr 172.21.15.7:0/3637081485 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:40.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.563+0000 7f82a37fe700 1 --2- 172.21.15.7:0/3637081485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ea74e4c50 0x564ea74f0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:40.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.563+0000 7f82a2ffd700 1 --2- 172.21.15.7:0/3637081485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ea74ea500 0x564ea74ede60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:40.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.564+0000 7f82a3fff700 1 -- 172.21.15.7:0/3637081485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ea74e4c50 msgr2=0x564ea74f0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:40.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.564+0000 7f82a3fff700 1 --2- 172.21.15.7:0/3637081485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ea74e4c50 0x564ea74f0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.564+0000 7f82a3fff700 1 -- 172.21.15.7:0/3637081485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ea74ea500 msgr2=0x564ea74ede60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:40.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.564+0000 7f82a3fff700 1 --2- 172.21.15.7:0/3637081485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ea74ea500 0x564ea74ede60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.564+0000 7f82a3fff700 1 -- 172.21.15.7:0/3637081485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ea73fb850 con 0x564ea74eae20 2024-06-10T01:15:40.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.564+0000 7f82a37fe700 1 --2- 172.21.15.7:0/3637081485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ea74e4c50 0x564ea74f0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:40.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.564+0000 7f82a3fff700 1 --2- 172.21.15.7:0/3637081485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ea74eae20 0x564ea74ee3a0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f829c0094f0 tx=0x7f829c007cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b433dc5ca1dfb0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:40.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.564+0000 7f8282ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:15:40.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.564+0000 7f8282ffd700 1 -- 172.21.15.7:0/3637081485 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c020040 con 0x564ea74eae20 2024-06-10T01:15:40.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.564+0000 7f8282ffd700 1 -- 172.21.15.7:0/3637081485 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f829c005040 con 0x564ea74eae20 2024-06-10T01:15:40.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.564+0000 7f82b44e2b80 1 -- 172.21.15.7:0/3637081485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f829c00bd30 con 0x564ea74eae20 2024-06-10T01:15:40.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.564+0000 7f8282ffd700 1 -- 172.21.15.7:0/3637081485 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f829c005240 con 0x564ea74eae20 2024-06-10T01:15:40.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.564+0000 7f82b44e2b80 10 client.17934 fetch_fsmap learned FSMap version 37 2024-06-10T01:15:40.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.564+0000 7f82b44e2b80 1 -- 172.21.15.7:0/3637081485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564ea74fbf30 con 0x564ea74eae20 2024-06-10T01:15:40.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.564+0000 7f8282ffd700 1 -- 172.21.15.7:0/3637081485 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f829c00bd30 con 0x564ea74eae20 2024-06-10T01:15:40.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.564+0000 7f8282ffd700 1 -- 172.21.15.7:0/3637081485 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f829c011070 con 0x564ea74eae20 2024-06-10T01:15:40.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.565+0000 7f82b44e2b80 10 client.17934 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T01:15:40.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.565+0000 7f82b44e2b80 10 client.17934 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:15:40.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.565+0000 7f82b44e2b80 20 client.17934 populate_metadata read hostname 'smithi007' 2024-06-10T01:15:40.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.565+0000 7f82b44e2b80 1 --2- 172.21.15.7:0/3637081485 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564ea74fe400 0x564ea75007f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:40.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.565+0000 7f82b44e2b80 4 client.17934 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:15:40.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.565+0000 7f82b44e2b80 1 -- 172.21.15.7:0/3637081485 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ea7442e50 con 0x564ea74fe400 2024-06-10T01:15:40.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.565+0000 7f82a37fe700 1 --2- 172.21.15.7:0/3637081485 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564ea74fe400 0x564ea75007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:40.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.565+0000 7f8282ffd700 10 client.17934 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:40.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.565+0000 7f82a37fe700 1 --2- 172.21.15.7:0/3637081485 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564ea74fe400 0x564ea75007f0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27e55f48681050ae server_cookie=a0fd30b81920ef83 in_seq=0 out_seq=0 2024-06-10T01:15:40.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.566+0000 7f8282ffd700 1 -- 172.21.15.7:0/3637081485 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564ea7442e50 con 0x564ea74fe400 2024-06-10T01:15:40.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.566+0000 7f8282ffd700 10 client.17934 handle_command_reply: tid=0 2024-06-10T01:15:40.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.566+0000 7f82b44e2b80 1 client.17934 shutdown 2024-06-10T01:15:40.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.566+0000 7f82b44e2b80 1 -- 172.21.15.7:0/3637081485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ea74eae20 msgr2=0x564ea74ee3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:40.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.566+0000 7f82b44e2b80 1 --2- 172.21.15.7:0/3637081485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ea74eae20 0x564ea74ee3a0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f829c0094f0 tx=0x7f829c007cf0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.566+0000 7f82b44e2b80 1 -- 172.21.15.7:0/3637081485 shutdown_connections 2024-06-10T01:15:40.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.566+0000 7f82b44e2b80 1 --2- 172.21.15.7:0/3637081485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ea74eae20 0x564ea74ee3a0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.566+0000 7f82b44e2b80 1 --2- 172.21.15.7:0/3637081485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ea74e4c50 0x564ea74f0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.566+0000 7f82b44e2b80 1 --2- 172.21.15.7:0/3637081485 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564ea74fe400 0x564ea75007f0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.566+0000 7f8282ffd700 0 client.17934 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:40.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.566+0000 7f82b44e2b80 1 --2- 172.21.15.7:0/3637081485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ea74ea500 0x564ea74ede60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.567+0000 7f82b44e2b80 1 -- 172.21.15.7:0/3637081485 >> 172.21.15.7:0/3637081485 conn(0x564ea7411180 msgr2=0x564ea74debb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:40.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.567+0000 7f82b44e2b80 1 -- 172.21.15.7:0/3637081485 shutdown_connections 2024-06-10T01:15:40.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.567+0000 7f82b44e2b80 1 -- 172.21.15.7:0/3637081485 wait complete. 2024-06-10T01:15:40.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.567+0000 7f82b44e2b80 20 client.17934 trim_cache size 0 max 16384 2024-06-10T01:15:40.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.587+0000 7f82b44e2b80 1 Processor -- start 2024-06-10T01:15:40.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.587+0000 7f82b44e2b80 1 -- start start 2024-06-10T01:15:40.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.587+0000 7f82b44e2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ea74e4c50 0x564ea74e4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:40.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.588+0000 7f82b44e2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ea74ea500 0x564ea74df330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:40.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.588+0000 7f82b44e2b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ea74eae20 0x564ea74df870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:40.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.588+0000 7f82b44e2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f82a41b2cd0 con 0x564ea74ea500 2024-06-10T01:15:40.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.588+0000 7f82b44e2b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f82a41b04d0 con 0x564ea74eae20 2024-06-10T01:15:40.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.588+0000 7f82b44e2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f82a41ae320 con 0x564ea74e4c50 2024-06-10T01:15:40.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.588+0000 7f82a2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ea74ea500 0x564ea74df330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:40.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.588+0000 7f82a2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ea74ea500 0x564ea74df330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51898/0 (socket says 172.21.15.7:51898) 2024-06-10T01:15:40.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.588+0000 7f82a2ffd700 1 -- 172.21.15.7:0/24387611 learned_addr learned my addr 172.21.15.7:0/24387611 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:40.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.588+0000 7f82a3fff700 1 --2- 172.21.15.7:0/24387611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ea74eae20 0x564ea74df870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:40.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.588+0000 7f82a2ffd700 1 -- 172.21.15.7:0/24387611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ea74e4c50 msgr2=0x564ea74e4710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:15:40.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.588+0000 7f82a2ffd700 1 --2- 172.21.15.7:0/24387611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ea74e4c50 0x564ea74e4710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.588+0000 7f82a2ffd700 1 -- 172.21.15.7:0/24387611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ea74eae20 msgr2=0x564ea74df870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:40.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.588+0000 7f82a2ffd700 1 --2- 172.21.15.7:0/24387611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ea74eae20 0x564ea74df870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.588+0000 7f82a2ffd700 1 -- 172.21.15.7:0/24387611 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ea74edc30 con 0x564ea74ea500 2024-06-10T01:15:40.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.588+0000 7f82a2ffd700 1 --2- 172.21.15.7:0/24387611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ea74ea500 0x564ea74df330 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f82a4079b60 tx=0x7f8298011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae7e6f8923eb71 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:40.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.588+0000 7f82897fa700 1 -- 172.21.15.7:0/24387611 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829800ee70 con 0x564ea74ea500 2024-06-10T01:15:40.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.588+0000 7f82897fa700 1 -- 172.21.15.7:0/24387611 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8298023070 con 0x564ea74ea500 2024-06-10T01:15:40.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.588+0000 7f82897fa700 1 -- 172.21.15.7:0/24387611 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829801f840 con 0x564ea74ea500 2024-06-10T01:15:40.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.589+0000 7f82b44e2b80 1 -- 172.21.15.7:0/24387611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ea74ea500 msgr2=0x564ea74df330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:40.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.589+0000 7f82b44e2b80 1 --2- 172.21.15.7:0/24387611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ea74ea500 0x564ea74df330 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f82a4079b60 tx=0x7f8298011000 comp rx=0 tx=0).stop 2024-06-10T01:15:40.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.589+0000 7f82b44e2b80 1 -- 172.21.15.7:0/24387611 shutdown_connections 2024-06-10T01:15:40.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.589+0000 7f82b44e2b80 1 --2- 172.21.15.7:0/24387611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ea74eae20 0x564ea74df870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.589+0000 7f82b44e2b80 1 --2- 172.21.15.7:0/24387611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ea74e4c50 0x564ea74e4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.589+0000 7f82b44e2b80 1 --2- 172.21.15.7:0/24387611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ea74ea500 0x564ea74df330 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.589+0000 7f82b44e2b80 1 -- 172.21.15.7:0/24387611 >> 172.21.15.7:0/24387611 conn(0x564ea7411180 msgr2=0x564ea74eff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:40.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.589+0000 7f82b44e2b80 1 -- 172.21.15.7:0/24387611 shutdown_connections 2024-06-10T01:15:40.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.589+0000 7f82b44e2b80 1 -- 172.21.15.7:0/24387611 wait complete. 2024-06-10T01:15:40.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.589+0000 7f82b44e2b80 1 -- start start 2024-06-10T01:15:40.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.590+0000 7f82b44e2b80 1 Processor -- start 2024-06-10T01:15:40.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.590+0000 7f82b44e2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ea74e4c50 0x564ea7501900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:40.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.590+0000 7f82b44e2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ea74ea500 0x564ea7501e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:40.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.590+0000 7f82b44e2b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ea74eae20 0x564ea7567d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:40.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.590+0000 7f82b44e2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f82a41ae990 con 0x564ea74ea500 2024-06-10T01:15:40.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.590+0000 7f82b44e2b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f82a41afe00 con 0x564ea74eae20 2024-06-10T01:15:40.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.590+0000 7f82b44e2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f82a41af060 con 0x564ea74e4c50 2024-06-10T01:15:40.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.590+0000 7f82b44e2b80 10 client.0 _ll_register_callbacks cb 0x564ea74e8aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:15:40.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.590+0000 7f82a2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ea74ea500 0x564ea7501e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:40.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.590+0000 7f82a2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ea74ea500 0x564ea7501e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51906/0 (socket says 172.21.15.7:51906) 2024-06-10T01:15:40.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.590+0000 7f82a2ffd700 1 -- 172.21.15.7:0/2482515785 learned_addr learned my addr 172.21.15.7:0/2482515785 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:40.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.590+0000 7f82a37fe700 1 --2- 172.21.15.7:0/2482515785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ea74e4c50 0x564ea7501900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:40.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.590+0000 7f82a3fff700 1 --2- 172.21.15.7:0/2482515785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ea74eae20 0x564ea7567d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:40.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.590+0000 7f82a2ffd700 1 -- 172.21.15.7:0/2482515785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ea74e4c50 msgr2=0x564ea7501900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:40.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.590+0000 7f82a2ffd700 1 --2- 172.21.15.7:0/2482515785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ea74e4c50 0x564ea7501900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.590+0000 7f82a2ffd700 1 -- 172.21.15.7:0/2482515785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ea74eae20 msgr2=0x564ea7567d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:40.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.590+0000 7f82a2ffd700 1 --2- 172.21.15.7:0/2482515785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ea74eae20 0x564ea7567d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.590+0000 7f82a2ffd700 1 -- 172.21.15.7:0/2482515785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ea75026e0 con 0x564ea74ea500 2024-06-10T01:15:40.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.590+0000 7f82a3fff700 1 --2- 172.21.15.7:0/2482515785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ea74eae20 0x564ea7567d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:15:40.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.590+0000 7f82a37fe700 1 --2- 172.21.15.7:0/2482515785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ea74e4c50 0x564ea7501900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:15:40.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.591+0000 7f82a2ffd700 1 --2- 172.21.15.7:0/2482515785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ea74ea500 0x564ea7501e40 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f8298028fd0 tx=0x7f8298028cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d51c67e526cd95 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:40.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.591+0000 7f8282ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:15:40.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.591+0000 7f8282ffd700 1 -- 172.21.15.7:0/2482515785 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829801f440 con 0x564ea74ea500 2024-06-10T01:15:40.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.591+0000 7f8282ffd700 1 -- 172.21.15.7:0/2482515785 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f829801f5e0 con 0x564ea74ea500 2024-06-10T01:15:40.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.591+0000 7f82b44e2b80 1 -- 172.21.15.7:0/2482515785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8298023070 con 0x564ea74ea500 2024-06-10T01:15:40.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.591+0000 7f8282ffd700 1 -- 172.21.15.7:0/2482515785 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8298004850 con 0x564ea74ea500 2024-06-10T01:15:40.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.591+0000 7f82b44e2b80 10 client.17940 fetch_fsmap learned FSMap version 37 2024-06-10T01:15:40.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.591+0000 7f82b44e2b80 1 -- 172.21.15.7:0/2482515785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564ea75693c0 con 0x564ea74ea500 2024-06-10T01:15:40.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.591+0000 7f8282ffd700 1 -- 172.21.15.7:0/2482515785 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8298023070 con 0x564ea74ea500 2024-06-10T01:15:40.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.591+0000 7f8282ffd700 1 -- 172.21.15.7:0/2482515785 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f8298005020 con 0x564ea74ea500 2024-06-10T01:15:40.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.592+0000 7f82b44e2b80 10 client.17940 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T01:15:40.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.592+0000 7f82b44e2b80 10 client.17940 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:15:40.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.592+0000 7f82b44e2b80 20 client.17940 populate_metadata read hostname 'smithi007' 2024-06-10T01:15:40.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.592+0000 7f82b44e2b80 1 --2- 172.21.15.7:0/2482515785 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564ea7569b00 0x564ea756bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:40.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.592+0000 7f82b44e2b80 4 client.17940 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "844e91e0", "release": true, "await": true}] 2024-06-10T01:15:40.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.592+0000 7f82b44e2b80 1 -- 172.21.15.7:0/2482515785 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "set_id": "844e91e0", "release": true, "await": true}) -- 0x564ea74ee820 con 0x564ea7569b00 2024-06-10T01:15:40.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.592+0000 7f82a37fe700 1 --2- 172.21.15.7:0/2482515785 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564ea7569b00 0x564ea756bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:40.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.592+0000 7f8282ffd700 10 client.17940 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:40.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.592+0000 7f82a37fe700 1 --2- 172.21.15.7:0/2482515785 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564ea7569b00 0x564ea756bf20 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3922ee91642e2877 server_cookie=982a2d121143a4d4 in_seq=0 out_seq=0 2024-06-10T01:15:40.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.602+0000 7f8282ffd700 1 -- 172.21.15.7:0/2482515785 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564ea74ee820 con 0x564ea7569b00 2024-06-10T01:15:40.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.602+0000 7f8282ffd700 10 client.17940 handle_command_reply: tid=0 2024-06-10T01:15:40.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.602+0000 7f82b44e2b80 1 client.17940 shutdown 2024-06-10T01:15:40.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.602+0000 7f82b44e2b80 1 -- 172.21.15.7:0/2482515785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ea74ea500 msgr2=0x564ea7501e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:40.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.602+0000 7f82b44e2b80 1 --2- 172.21.15.7:0/2482515785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ea74ea500 0x564ea7501e40 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f8298028fd0 tx=0x7f8298028cd0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.602+0000 7f82b44e2b80 1 -- 172.21.15.7:0/2482515785 shutdown_connections 2024-06-10T01:15:40.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.602+0000 7f82b44e2b80 1 --2- 172.21.15.7:0/2482515785 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564ea7569b00 0x564ea756bf20 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.602+0000 7f82b44e2b80 1 --2- 172.21.15.7:0/2482515785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ea74eae20 0x564ea7567d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.602+0000 7f82b44e2b80 1 --2- 172.21.15.7:0/2482515785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ea74e4c50 0x564ea7501900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.602+0000 7f82b44e2b80 1 --2- 172.21.15.7:0/2482515785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ea74ea500 0x564ea7501e40 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.602+0000 7f82b44e2b80 1 -- 172.21.15.7:0/2482515785 >> 172.21.15.7:0/2482515785 conn(0x564ea7411180 msgr2=0x564ea74ef690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:40.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.602+0000 7f8282ffd700 0 client.17940 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:40.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.602+0000 7f82b44e2b80 1 -- 172.21.15.7:0/2482515785 shutdown_connections 2024-06-10T01:15:40.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.602+0000 7f82b44e2b80 1 -- 172.21.15.7:0/2482515785 wait complete. 2024-06-10T01:15:40.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.602+0000 7f82b44e2b80 20 client.17940 trim_cache size 0 max 16384 2024-06-10T01:15:40.599 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:15:40.599 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 37, 2024-06-10T01:15:40.599 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:15:40.599 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 32, 2024-06-10T01:15:40.599 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:15:40.599 INFO:teuthology.orchestra.run.smithi007.stdout: "844e91e0": { 2024-06-10T01:15:40.599 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 32, 2024-06-10T01:15:40.599 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:15:40.599 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:15:40.599 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:15:40.599 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:15:40.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:15:40.600 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:15:40.600 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 137.8, 2024-06-10T01:15:40.600 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:15:40.600 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:15:40.600 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:15:40.600 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:15:40.600 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:15:40.600 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:15:40.600 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:15:40.600 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:15:40.600 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:15:40.600 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:15:40.600 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:15:40.600 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:15:40.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.603+0000 7f82a9c5f700 1 -- 172.21.15.7:0/90920470 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f828c0a2ca0 msgr2=0x7f828c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:40.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.603+0000 7f82a9c5f700 1 --2- 172.21.15.7:0/90920470 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f828c0a2ca0 0x7f828c0a5160 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8298005fd0 tx=0x7f8298005e30 comp rx=0 tx=0).stop 2024-06-10T01:15:40.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.603+0000 7f82a9c5f700 1 -- 172.21.15.7:0/90920470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82a4102540 msgr2=0x7f82a4074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:40.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.603+0000 7f82a9c5f700 1 --2- 172.21.15.7:0/90920470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82a4102540 0x7f82a4074790 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f829400e970 tx=0x7f829400ec80 comp rx=0 tx=0).stop 2024-06-10T01:15:40.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.603+0000 7f82a9c5f700 1 -- 172.21.15.7:0/90920470 shutdown_connections 2024-06-10T01:15:40.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.603+0000 7f82a9c5f700 1 --2- 172.21.15.7:0/90920470 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f828c0a2ca0 0x7f828c0a5160 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.603+0000 7f82a9c5f700 1 --2- 172.21.15.7:0/90920470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82a4107450 0x7f82a4075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.603+0000 7f82a9c5f700 1 --2- 172.21.15.7:0/90920470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82a4102540 0x7f82a4074790 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.603+0000 7f82a9c5f700 1 --2- 172.21.15.7:0/90920470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82a4102ee0 0x7f82a4074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:40.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.603+0000 7f82a9c5f700 1 -- 172.21.15.7:0/90920470 >> 172.21.15.7:0/90920470 conn(0x7f82a40795f0 msgr2=0x7f82a4100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:40.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.603+0000 7f82a9c5f700 1 -- 172.21.15.7:0/90920470 shutdown_connections 2024-06-10T01:15:40.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:40.603+0000 7f82a9c5f700 1 -- 172.21.15.7:0/90920470 wait complete. 2024-06-10T01:15:40.745 INFO:tasks.workunit.client.0.smithi007.stdout: 1 676375 7.15 MB/sec execute 381 sec latency 20696.208 ms 2024-06-10T01:15:41.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.259+0000 7fa951f4b700 1 -- 172.21.15.7:0/3015753734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa94c1085d0 msgr2=0x7fa94c1089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:41.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.259+0000 7fa951f4b700 1 --2- 172.21.15.7:0/3015753734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa94c1085d0 0x7fa94c1089b0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0099c0 tx=0x7fa93c009cd0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.259+0000 7fa951f4b700 1 -- 172.21.15.7:0/3015753734 shutdown_connections 2024-06-10T01:15:41.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.259+0000 7fa951f4b700 1 --2- 172.21.15.7:0/3015753734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa94c102ef0 0x7fa94c10f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.259+0000 7fa951f4b700 1 --2- 172.21.15.7:0/3015753734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa94c1085d0 0x7fa94c1089b0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.259+0000 7fa951f4b700 1 --2- 172.21.15.7:0/3015753734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa94c102530 0x7fa94c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.259+0000 7fa951f4b700 1 -- 172.21.15.7:0/3015753734 >> 172.21.15.7:0/3015753734 conn(0x7fa94c079640 msgr2=0x7fa94c079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:41.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.259+0000 7fa951f4b700 1 -- 172.21.15.7:0/3015753734 shutdown_connections 2024-06-10T01:15:41.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.259+0000 7fa951f4b700 1 -- 172.21.15.7:0/3015753734 wait complete. 2024-06-10T01:15:41.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.260+0000 7fa951f4b700 1 Processor -- start 2024-06-10T01:15:41.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.260+0000 7fa951f4b700 1 -- start start 2024-06-10T01:15:41.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.260+0000 7fa951f4b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa94c102530 0x7fa94c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:41.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.260+0000 7fa951f4b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa94c102ef0 0x7fa94c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:41.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.260+0000 7fa951f4b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa94c1085d0 0x7fa94c075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:41.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.261+0000 7fa951f4b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c112820 con 0x7fa94c1085d0 2024-06-10T01:15:41.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.261+0000 7fa951f4b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c1126a0 con 0x7fa94c102530 2024-06-10T01:15:41.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.261+0000 7fa951f4b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c1129a0 con 0x7fa94c102ef0 2024-06-10T01:15:41.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.261+0000 7fa94affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa94c102ef0 0x7fa94c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:41.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.261+0000 7fa94bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa94c1085d0 0x7fa94c075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:41.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.261+0000 7fa94affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa94c102ef0 0x7fa94c074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:45976/0 (socket says 172.21.15.7:45976) 2024-06-10T01:15:41.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.261+0000 7fa94bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa94c1085d0 0x7fa94c075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51920/0 (socket says 172.21.15.7:51920) 2024-06-10T01:15:41.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.261+0000 7fa94affd700 1 -- 172.21.15.7:0/2404907884 learned_addr learned my addr 172.21.15.7:0/2404907884 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:41.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.261+0000 7fa94b7fe700 1 --2- 172.21.15.7:0/2404907884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa94c102530 0x7fa94c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:41.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.261+0000 7fa94affd700 1 -- 172.21.15.7:0/2404907884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa94c102530 msgr2=0x7fa94c074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:41.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.261+0000 7fa94affd700 1 --2- 172.21.15.7:0/2404907884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa94c102530 0x7fa94c074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.261+0000 7fa94affd700 1 -- 172.21.15.7:0/2404907884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa94c1085d0 msgr2=0x7fa94c075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:41.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.261+0000 7fa94affd700 1 --2- 172.21.15.7:0/2404907884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa94c1085d0 0x7fa94c075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.261+0000 7fa94affd700 1 -- 172.21.15.7:0/2404907884 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94c1a7140 con 0x7fa94c102ef0 2024-06-10T01:15:41.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.261+0000 7fa94b7fe700 1 --2- 172.21.15.7:0/2404907884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa94c102530 0x7fa94c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:41.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.261+0000 7fa94bfff700 1 --2- 172.21.15.7:0/2404907884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa94c1085d0 0x7fa94c075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:41.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.261+0000 7fa94affd700 1 --2- 172.21.15.7:0/2404907884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa94c102ef0 0x7fa94c074cd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa94000eb30 tx=0x7fa94000ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:41.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.262+0000 7fa948ff9700 1 -- 172.21.15.7:0/2404907884 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94000cbd0 con 0x7fa94c102ef0 2024-06-10T01:15:41.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.262+0000 7fa948ff9700 1 -- 172.21.15.7:0/2404907884 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa940004500 con 0x7fa94c102ef0 2024-06-10T01:15:41.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.262+0000 7fa951f4b700 1 -- 172.21.15.7:0/2404907884 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa94c1a7430 con 0x7fa94c102ef0 2024-06-10T01:15:41.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.262+0000 7fa948ff9700 1 -- 172.21.15.7:0/2404907884 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940010430 con 0x7fa94c102ef0 2024-06-10T01:15:41.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.262+0000 7fa951f4b700 1 -- 172.21.15.7:0/2404907884 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa94c1a79c0 con 0x7fa94c102ef0 2024-06-10T01:15:41.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.264+0000 7fa948ff9700 1 -- 172.21.15.7:0/2404907884 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa940003f00 con 0x7fa94c102ef0 2024-06-10T01:15:41.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.265+0000 7fa948ff9700 1 --2- 172.21.15.7:0/2404907884 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa9340a2ca0 0x7fa9340a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:41.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.265+0000 7fa94b7fe700 1 --2- 172.21.15.7:0/2404907884 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa9340a2ca0 0x7fa9340a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:41.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.265+0000 7fa948ff9700 1 -- 172.21.15.7:0/2404907884 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa9400cf700 con 0x7fa94c102ef0 2024-06-10T01:15:41.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.265+0000 7fa95c7ceb80 1 Processor -- start 2024-06-10T01:15:41.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.265+0000 7fa95c7ceb80 1 -- start start 2024-06-10T01:15:41.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.266+0000 7fa94b7fe700 1 --2- 172.21.15.7:0/2404907884 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa9340a2ca0 0x7fa9340a5160 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa93c000c00 tx=0x7fa93c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:41.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.266+0000 7fa95c7ceb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df12386c0 0x558df1238aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:41.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.266+0000 7fa95c7ceb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df1238fe0 0x558df1232a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:41.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.266+0000 7fa95c7ceb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558df1232fa0 0x558df12333c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:41.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.266+0000 7fa95c7ceb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558df1184890 con 0x558df12386c0 2024-06-10T01:15:41.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.266+0000 7fa95c7ceb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558df112c7a0 con 0x558df1238fe0 2024-06-10T01:15:41.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.266+0000 7fa95c7ceb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558df1180850 con 0x558df1232fa0 2024-06-10T01:15:41.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.266+0000 7fa94b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df12386c0 0x558df1238aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:41.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.266+0000 7fa94b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df12386c0 0x558df1238aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51934/0 (socket says 172.21.15.7:51934) 2024-06-10T01:15:41.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.266+0000 7fa94b7fe700 1 -- 172.21.15.7:0/2379014844 learned_addr learned my addr 172.21.15.7:0/2379014844 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:41.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.266+0000 7fa94affd700 1 --2- 172.21.15.7:0/2379014844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df1238fe0 0x558df1232a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:41.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.266+0000 7fa94bfff700 1 --2- 172.21.15.7:0/2379014844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558df1232fa0 0x558df12333c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:41.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.266+0000 7fa94affd700 1 -- 172.21.15.7:0/2379014844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558df1232fa0 msgr2=0x558df12333c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:41.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.266+0000 7fa94affd700 1 --2- 172.21.15.7:0/2379014844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558df1232fa0 0x558df12333c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.266+0000 7fa94affd700 1 -- 172.21.15.7:0/2379014844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df12386c0 msgr2=0x558df1238aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:41.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.266+0000 7fa94affd700 1 --2- 172.21.15.7:0/2379014844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df12386c0 0x558df1238aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.266+0000 7fa94affd700 1 -- 172.21.15.7:0/2379014844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558df1122fd0 con 0x558df1238fe0 2024-06-10T01:15:41.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.266+0000 7fa94b7fe700 1 --2- 172.21.15.7:0/2379014844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df12386c0 0x558df1238aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:41.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.266+0000 7fa94affd700 1 --2- 172.21.15.7:0/2379014844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df1238fe0 0x558df1232a60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa94009b880 tx=0x7fa940097ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6f932a7e8f76570 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:41.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.267+0000 7fa931ffb700 1 -- 172.21.15.7:0/2379014844 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94009c280 con 0x558df1238fe0 2024-06-10T01:15:41.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.267+0000 7fa931ffb700 1 -- 172.21.15.7:0/2379014844 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa94009bb80 con 0x558df1238fe0 2024-06-10T01:15:41.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.267+0000 7fa931ffb700 1 -- 172.21.15.7:0/2379014844 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9400cf3f0 con 0x558df1238fe0 2024-06-10T01:15:41.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.267+0000 7fa95c7ceb80 1 -- 172.21.15.7:0/2379014844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df1238fe0 msgr2=0x558df1232a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:41.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.267+0000 7fa95c7ceb80 1 --2- 172.21.15.7:0/2379014844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df1238fe0 0x558df1232a60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa94009b880 tx=0x7fa940097ef0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.267+0000 7fa95c7ceb80 1 -- 172.21.15.7:0/2379014844 shutdown_connections 2024-06-10T01:15:41.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.267+0000 7fa95c7ceb80 1 --2- 172.21.15.7:0/2379014844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558df1232fa0 0x558df12333c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.267+0000 7fa95c7ceb80 1 --2- 172.21.15.7:0/2379014844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df12386c0 0x558df1238aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.267+0000 7fa95c7ceb80 1 --2- 172.21.15.7:0/2379014844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df1238fe0 0x558df1232a60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.267+0000 7fa95c7ceb80 1 -- 172.21.15.7:0/2379014844 >> 172.21.15.7:0/2379014844 conn(0x558df115dd40 msgr2=0x558df122d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:41.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.267+0000 7fa95c7ceb80 1 -- 172.21.15.7:0/2379014844 shutdown_connections 2024-06-10T01:15:41.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.267+0000 7fa95c7ceb80 1 -- 172.21.15.7:0/2379014844 wait complete. 2024-06-10T01:15:41.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.268+0000 7fa95c7ceb80 1 -- start start 2024-06-10T01:15:41.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.268+0000 7fa95c7ceb80 1 Processor -- start 2024-06-10T01:15:41.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.268+0000 7fa95c7ceb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df1232fa0 0x558df123edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:41.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.268+0000 7fa95c7ceb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558df12386c0 0x558df123c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:41.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.268+0000 7fa95c7ceb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df1238fe0 0x558df123c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:41.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.268+0000 7fa95c7ceb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558df123fa50 con 0x558df1232fa0 2024-06-10T01:15:41.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.268+0000 7fa95c7ceb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558df1124c70 con 0x558df1238fe0 2024-06-10T01:15:41.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.268+0000 7fa95c7ceb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558df123fbd0 con 0x558df12386c0 2024-06-10T01:15:41.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.268+0000 7fa95c7ceb80 10 client.0 _ll_register_callbacks cb 0x558df111e9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:15:41.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.268+0000 7fa94b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df1232fa0 0x558df123edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:41.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.268+0000 7fa94affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558df12386c0 0x558df123c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:41.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.268+0000 7fa94b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df1232fa0 0x558df123edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51948/0 (socket says 172.21.15.7:51948) 2024-06-10T01:15:41.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.269+0000 7fa94b7fe700 1 -- 172.21.15.7:0/2949154065 learned_addr learned my addr 172.21.15.7:0/2949154065 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:41.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.269+0000 7fa94bfff700 1 --2- 172.21.15.7:0/2949154065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df1238fe0 0x558df123c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:41.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.269+0000 7fa94b7fe700 1 -- 172.21.15.7:0/2949154065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558df12386c0 msgr2=0x558df123c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:41.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.269+0000 7fa94b7fe700 1 --2- 172.21.15.7:0/2949154065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558df12386c0 0x558df123c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.269+0000 7fa94b7fe700 1 -- 172.21.15.7:0/2949154065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df1238fe0 msgr2=0x558df123c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:41.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.269+0000 7fa94b7fe700 1 --2- 172.21.15.7:0/2949154065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df1238fe0 0x558df123c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.269+0000 7fa94b7fe700 1 -- 172.21.15.7:0/2949154065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558df10d3950 con 0x558df1232fa0 2024-06-10T01:15:41.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.269+0000 7fa94bfff700 1 --2- 172.21.15.7:0/2949154065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df1238fe0 0x558df123c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:15:41.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.269+0000 7fa94affd700 1 --2- 172.21.15.7:0/2949154065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558df12386c0 0x558df123c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:41.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.269+0000 7fa94b7fe700 1 --2- 172.21.15.7:0/2949154065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df1232fa0 0x558df123edf0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0098e0 tx=0x7fa93c01a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a750ac6ad127 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:41.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.269+0000 7fa92affd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:15:41.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.270+0000 7fa92affd700 1 -- 172.21.15.7:0/2949154065 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c035060 con 0x558df1232fa0 2024-06-10T01:15:41.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.270+0000 7fa95c7ceb80 1 -- 172.21.15.7:0/2949154065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa94009bb80 con 0x558df1232fa0 2024-06-10T01:15:41.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.270+0000 7fa92affd700 1 -- 172.21.15.7:0/2949154065 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa93c00f870 con 0x558df1232fa0 2024-06-10T01:15:41.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.270+0000 7fa92affd700 1 -- 172.21.15.7:0/2949154065 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa93c03c050 con 0x558df1232fa0 2024-06-10T01:15:41.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.270+0000 7fa95c7ceb80 10 client.17952 fetch_fsmap learned FSMap version 37 2024-06-10T01:15:41.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.270+0000 7fa95c7ceb80 1 -- 172.21.15.7:0/2949154065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558df124a3d0 con 0x558df1232fa0 2024-06-10T01:15:41.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.270+0000 7fa92affd700 1 -- 172.21.15.7:0/2949154065 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa94009bb80 con 0x558df1232fa0 2024-06-10T01:15:41.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.270+0000 7fa92affd700 1 -- 172.21.15.7:0/2949154065 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7fa93c030ac0 con 0x558df1232fa0 2024-06-10T01:15:41.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.270+0000 7fa95c7ceb80 10 client.17952 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T01:15:41.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.270+0000 7fa95c7ceb80 10 client.17952 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:15:41.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.271+0000 7fa95c7ceb80 20 client.17952 populate_metadata read hostname 'smithi007' 2024-06-10T01:15:41.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.271+0000 7fa95c7ceb80 1 --2- 172.21.15.7:0/2949154065 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558df124c960 0x558df124ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:41.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.271+0000 7fa95c7ceb80 4 client.17952 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:15:41.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.271+0000 7fa95c7ceb80 1 -- 172.21.15.7:0/2949154065 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558df1191d90 con 0x558df124c960 2024-06-10T01:15:41.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.271+0000 7fa94affd700 1 --2- 172.21.15.7:0/2949154065 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558df124c960 0x558df124ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:41.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.271+0000 7fa92affd700 10 client.17952 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:41.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.271+0000 7fa94affd700 1 --2- 172.21.15.7:0/2949154065 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558df124c960 0x558df124ed90 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fee5de014f21b56 server_cookie=cf6324cb2e7fe8e5 in_seq=0 out_seq=0 2024-06-10T01:15:41.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.272+0000 7fa92affd700 1 -- 172.21.15.7:0/2949154065 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558df1191d90 con 0x558df124c960 2024-06-10T01:15:41.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.272+0000 7fa92affd700 10 client.17952 handle_command_reply: tid=0 2024-06-10T01:15:41.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.272+0000 7fa95c7ceb80 1 client.17952 shutdown 2024-06-10T01:15:41.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.272+0000 7fa95c7ceb80 1 -- 172.21.15.7:0/2949154065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df1232fa0 msgr2=0x558df123edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:41.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.272+0000 7fa95c7ceb80 1 --2- 172.21.15.7:0/2949154065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df1232fa0 0x558df123edf0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0098e0 tx=0x7fa93c01a050 comp rx=0 tx=0).stop 2024-06-10T01:15:41.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.272+0000 7fa95c7ceb80 1 -- 172.21.15.7:0/2949154065 shutdown_connections 2024-06-10T01:15:41.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.272+0000 7fa95c7ceb80 1 --2- 172.21.15.7:0/2949154065 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558df124c960 0x558df124ed90 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.272+0000 7fa95c7ceb80 1 --2- 172.21.15.7:0/2949154065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df1238fe0 0x558df123c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.272+0000 7fa92affd700 0 client.17952 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:41.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.272+0000 7fa95c7ceb80 1 --2- 172.21.15.7:0/2949154065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df1232fa0 0x558df123edf0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.272+0000 7fa95c7ceb80 1 --2- 172.21.15.7:0/2949154065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558df12386c0 0x558df123c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.273+0000 7fa95c7ceb80 1 -- 172.21.15.7:0/2949154065 >> 172.21.15.7:0/2949154065 conn(0x558df115dd40 msgr2=0x558df122d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:41.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.273+0000 7fa95c7ceb80 1 -- 172.21.15.7:0/2949154065 shutdown_connections 2024-06-10T01:15:41.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.273+0000 7fa95c7ceb80 1 -- 172.21.15.7:0/2949154065 wait complete. 2024-06-10T01:15:41.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.273+0000 7fa95c7ceb80 20 client.17952 trim_cache size 0 max 16384 2024-06-10T01:15:41.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.297+0000 7fa95c7ceb80 1 Processor -- start 2024-06-10T01:15:41.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.297+0000 7fa95c7ceb80 1 -- start start 2024-06-10T01:15:41.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.297+0000 7fa95c7ceb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df1232fa0 0x558df1232a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:41.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.297+0000 7fa95c7ceb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558df12386c0 0x558df123b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:41.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.297+0000 7fa95c7ceb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df1238fe0 0x558df1239c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:41.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.297+0000 7fa95c7ceb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c1b2e20 con 0x558df1238fe0 2024-06-10T01:15:41.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.297+0000 7fa95c7ceb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c1b0620 con 0x558df1232fa0 2024-06-10T01:15:41.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.297+0000 7fa95c7ceb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c1ae470 con 0x558df12386c0 2024-06-10T01:15:41.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.297+0000 7fa94bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df1238fe0 0x558df1239c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:41.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.297+0000 7fa94bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df1238fe0 0x558df1239c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51954/0 (socket says 172.21.15.7:51954) 2024-06-10T01:15:41.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.297+0000 7fa94b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df1232fa0 0x558df1232a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:41.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.297+0000 7fa94bfff700 1 -- 172.21.15.7:0/2156466190 learned_addr learned my addr 172.21.15.7:0/2156466190 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:41.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.297+0000 7fa94affd700 1 --2- 172.21.15.7:0/2156466190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558df12386c0 0x558df123b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:41.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.297+0000 7fa94b7fe700 1 -- 172.21.15.7:0/2156466190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558df12386c0 msgr2=0x558df123b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:41.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.298+0000 7fa94b7fe700 1 --2- 172.21.15.7:0/2156466190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558df12386c0 0x558df123b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.298+0000 7fa94b7fe700 1 -- 172.21.15.7:0/2156466190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df1238fe0 msgr2=0x558df1239c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:41.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.298+0000 7fa94b7fe700 1 --2- 172.21.15.7:0/2156466190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df1238fe0 0x558df1239c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.298+0000 7fa94b7fe700 1 -- 172.21.15.7:0/2156466190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558df123a6b0 con 0x558df1232fa0 2024-06-10T01:15:41.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.298+0000 7fa94bfff700 1 --2- 172.21.15.7:0/2156466190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df1238fe0 0x558df1239c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:41.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.298+0000 7fa94b7fe700 1 --2- 172.21.15.7:0/2156466190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df1232fa0 0x558df1232a60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00b600 tx=0x7fa93c005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=430cda9b75960ed3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:41.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.298+0000 7fa9317fa700 1 -- 172.21.15.7:0/2156466190 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c045070 con 0x558df1232fa0 2024-06-10T01:15:41.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.298+0000 7fa9317fa700 1 -- 172.21.15.7:0/2156466190 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa93c024e40 con 0x558df1232fa0 2024-06-10T01:15:41.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.298+0000 7fa9317fa700 1 -- 172.21.15.7:0/2156466190 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c035040 con 0x558df1232fa0 2024-06-10T01:15:41.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.299+0000 7fa95c7ceb80 1 -- 172.21.15.7:0/2156466190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df1232fa0 msgr2=0x558df1232a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:41.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.299+0000 7fa95c7ceb80 1 --2- 172.21.15.7:0/2156466190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df1232fa0 0x558df1232a60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00b600 tx=0x7fa93c005ea0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.299+0000 7fa95c7ceb80 1 -- 172.21.15.7:0/2156466190 shutdown_connections 2024-06-10T01:15:41.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.299+0000 7fa95c7ceb80 1 --2- 172.21.15.7:0/2156466190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df1238fe0 0x558df1239c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.299+0000 7fa95c7ceb80 1 --2- 172.21.15.7:0/2156466190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df1232fa0 0x558df1232a60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.299+0000 7fa95c7ceb80 1 --2- 172.21.15.7:0/2156466190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558df12386c0 0x558df123b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.299+0000 7fa95c7ceb80 1 -- 172.21.15.7:0/2156466190 >> 172.21.15.7:0/2156466190 conn(0x558df115dd40 msgr2=0x558df1226e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:41.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.299+0000 7fa95c7ceb80 1 -- 172.21.15.7:0/2156466190 shutdown_connections 2024-06-10T01:15:41.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.299+0000 7fa95c7ceb80 1 -- 172.21.15.7:0/2156466190 wait complete. 2024-06-10T01:15:41.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.299+0000 7fa95c7ceb80 1 -- start start 2024-06-10T01:15:41.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.300+0000 7fa95c7ceb80 1 Processor -- start 2024-06-10T01:15:41.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.300+0000 7fa95c7ceb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df1232fa0 0x558df1245040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:41.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.300+0000 7fa95c7ceb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df12386c0 0x558df1245580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:41.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.300+0000 7fa95c7ceb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558df1238fe0 0x558df1241e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:41.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.300+0000 7fa95c7ceb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c1aeae0 con 0x558df1232fa0 2024-06-10T01:15:41.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.300+0000 7fa95c7ceb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c1aff50 con 0x558df12386c0 2024-06-10T01:15:41.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.300+0000 7fa95c7ceb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c1af1b0 con 0x558df1238fe0 2024-06-10T01:15:41.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.300+0000 7fa95c7ceb80 10 client.0 _ll_register_callbacks cb 0x558df111e9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:15:41.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.300+0000 7fa94bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558df1238fe0 0x558df1241e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:41.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.300+0000 7fa94b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df1232fa0 0x558df1245040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:41.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.300+0000 7fa94b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df1232fa0 0x558df1245040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51966/0 (socket says 172.21.15.7:51966) 2024-06-10T01:15:41.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.300+0000 7fa94b7fe700 1 -- 172.21.15.7:0/1688856522 learned_addr learned my addr 172.21.15.7:0/1688856522 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:41.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.300+0000 7fa94affd700 1 --2- 172.21.15.7:0/1688856522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df12386c0 0x558df1245580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:41.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.300+0000 7fa94b7fe700 1 -- 172.21.15.7:0/1688856522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558df1238fe0 msgr2=0x558df1241e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:41.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.300+0000 7fa94b7fe700 1 --2- 172.21.15.7:0/1688856522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558df1238fe0 0x558df1241e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.301+0000 7fa94b7fe700 1 -- 172.21.15.7:0/1688856522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df12386c0 msgr2=0x558df1245580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:41.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.301+0000 7fa94b7fe700 1 --2- 172.21.15.7:0/1688856522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df12386c0 0x558df1245580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.301+0000 7fa94b7fe700 1 -- 172.21.15.7:0/1688856522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558df1242500 con 0x558df1232fa0 2024-06-10T01:15:41.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.301+0000 7fa94affd700 1 --2- 172.21.15.7:0/1688856522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df12386c0 0x558df1245580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:15:41.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.301+0000 7fa94bfff700 1 --2- 172.21.15.7:0/1688856522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558df1238fe0 0x558df1241e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:41.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.301+0000 7fa94b7fe700 1 --2- 172.21.15.7:0/1688856522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df1232fa0 0x558df1245040 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009350 tx=0x7fa93c024b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c94510fe85c66a8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:41.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.301+0000 7fa92affd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:15:41.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.301+0000 7fa92affd700 1 -- 172.21.15.7:0/1688856522 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c045030 con 0x558df1232fa0 2024-06-10T01:15:41.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.301+0000 7fa92affd700 1 -- 172.21.15.7:0/1688856522 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa93c024660 con 0x558df1232fa0 2024-06-10T01:15:41.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.301+0000 7fa95c7ceb80 1 -- 172.21.15.7:0/1688856522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558df118c310 con 0x558df1232fa0 2024-06-10T01:15:41.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.301+0000 7fa92affd700 1 -- 172.21.15.7:0/1688856522 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa93c035070 con 0x558df1232fa0 2024-06-10T01:15:41.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.301+0000 7fa95c7ceb80 10 client.17958 fetch_fsmap learned FSMap version 37 2024-06-10T01:15:41.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.301+0000 7fa95c7ceb80 1 -- 172.21.15.7:0/1688856522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558df12b80d0 con 0x558df1232fa0 2024-06-10T01:15:41.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.302+0000 7fa92affd700 1 -- 172.21.15.7:0/1688856522 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558df118c310 con 0x558df1232fa0 2024-06-10T01:15:41.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.302+0000 7fa92affd700 1 -- 172.21.15.7:0/1688856522 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7fa93c030590 con 0x558df1232fa0 2024-06-10T01:15:41.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.302+0000 7fa95c7ceb80 10 client.17958 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T01:15:41.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.302+0000 7fa95c7ceb80 10 client.17958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:15:41.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.302+0000 7fa95c7ceb80 20 client.17958 populate_metadata read hostname 'smithi007' 2024-06-10T01:15:41.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.302+0000 7fa95c7ceb80 1 --2- 172.21.15.7:0/1688856522 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558df124c960 0x558df12ba450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:41.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.302+0000 7fa95c7ceb80 4 client.17958 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:15:41.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.302+0000 7fa95c7ceb80 1 -- 172.21.15.7:0/1688856522 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558df10062d0 con 0x558df124c960 2024-06-10T01:15:41.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.302+0000 7fa94affd700 1 --2- 172.21.15.7:0/1688856522 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558df124c960 0x558df12ba450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:41.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.302+0000 7fa94affd700 1 --2- 172.21.15.7:0/1688856522 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558df124c960 0x558df12ba450 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaf838dd70dd8baf server_cookie=5529e7b60fb07a90 in_seq=0 out_seq=0 2024-06-10T01:15:41.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.302+0000 7fa92affd700 10 client.17958 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:41.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.303+0000 7fa92affd700 1 -- 172.21.15.7:0/1688856522 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558df10062d0 con 0x558df124c960 2024-06-10T01:15:41.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.303+0000 7fa92affd700 10 client.17958 handle_command_reply: tid=0 2024-06-10T01:15:41.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.303+0000 7fa95c7ceb80 1 client.17958 shutdown 2024-06-10T01:15:41.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.304+0000 7fa95c7ceb80 1 -- 172.21.15.7:0/1688856522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df1232fa0 msgr2=0x558df1245040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:41.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.304+0000 7fa95c7ceb80 1 --2- 172.21.15.7:0/1688856522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df1232fa0 0x558df1245040 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009350 tx=0x7fa93c024b90 comp rx=0 tx=0).stop 2024-06-10T01:15:41.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.304+0000 7fa95c7ceb80 1 -- 172.21.15.7:0/1688856522 shutdown_connections 2024-06-10T01:15:41.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.304+0000 7fa95c7ceb80 1 --2- 172.21.15.7:0/1688856522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558df1238fe0 0x558df1241e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.304+0000 7fa95c7ceb80 1 --2- 172.21.15.7:0/1688856522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df1232fa0 0x558df1245040 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.304+0000 7fa95c7ceb80 1 --2- 172.21.15.7:0/1688856522 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558df124c960 0x558df12ba450 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.304+0000 7fa95c7ceb80 1 --2- 172.21.15.7:0/1688856522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df12386c0 0x558df1245580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.304+0000 7fa95c7ceb80 1 -- 172.21.15.7:0/1688856522 >> 172.21.15.7:0/1688856522 conn(0x558df115dd40 msgr2=0x558df1237200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:41.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.304+0000 7fa92affd700 0 client.17958 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:41.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.304+0000 7fa95c7ceb80 1 -- 172.21.15.7:0/1688856522 shutdown_connections 2024-06-10T01:15:41.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.304+0000 7fa95c7ceb80 1 -- 172.21.15.7:0/1688856522 wait complete. 2024-06-10T01:15:41.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.304+0000 7fa95c7ceb80 20 client.17958 trim_cache size 0 max 16384 2024-06-10T01:15:41.304 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:15:41.304 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-10T01:15:41.304 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "8173abd2-d4b6-4232-8489-8b7e45a1a4f6", 2024-06-10T01:15:41.304 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-10T01:15:41.304 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:15:41.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.305+0000 7fa951f4b700 1 -- 172.21.15.7:0/2404907884 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa9340a2ca0 msgr2=0x7fa9340a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:41.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.305+0000 7fa951f4b700 1 --2- 172.21.15.7:0/2404907884 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa9340a2ca0 0x7fa9340a5160 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa93c000c00 tx=0x7fa93c005fb0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.305+0000 7fa951f4b700 1 -- 172.21.15.7:0/2404907884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa94c102ef0 msgr2=0x7fa94c074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:41.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.305+0000 7fa951f4b700 1 --2- 172.21.15.7:0/2404907884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa94c102ef0 0x7fa94c074cd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa94000eb30 tx=0x7fa94000ee40 comp rx=0 tx=0).stop 2024-06-10T01:15:41.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.305+0000 7fa951f4b700 1 -- 172.21.15.7:0/2404907884 shutdown_connections 2024-06-10T01:15:41.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.305+0000 7fa951f4b700 1 --2- 172.21.15.7:0/2404907884 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa9340a2ca0 0x7fa9340a5160 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.305+0000 7fa951f4b700 1 --2- 172.21.15.7:0/2404907884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa94c1085d0 0x7fa94c075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.305+0000 7fa951f4b700 1 --2- 172.21.15.7:0/2404907884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa94c102530 0x7fa94c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.305+0000 7fa951f4b700 1 --2- 172.21.15.7:0/2404907884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa94c102ef0 0x7fa94c074cd0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:41.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.305+0000 7fa951f4b700 1 -- 172.21.15.7:0/2404907884 >> 172.21.15.7:0/2404907884 conn(0x7fa94c079640 msgr2=0x7fa94c100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:41.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.305+0000 7fa951f4b700 1 -- 172.21.15.7:0/2404907884 shutdown_connections 2024-06-10T01:15:41.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:41.306+0000 7fa951f4b700 1 -- 172.21.15.7:0/2404907884 wait complete. 2024-06-10T01:15:41.643 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:41 smithi007 ceph-mon[32398]: pgmap v722: 129 pgs: 129 active+clean; 109 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 312 KiB/s wr, 40 op/s 2024-06-10T01:15:41.643 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:41 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:15:41.643 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:41 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:15:41.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:41 smithi175 ceph-mon[36636]: pgmap v722: 129 pgs: 129 active+clean; 109 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 312 KiB/s wr, 40 op/s 2024-06-10T01:15:41.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:41 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:15:41.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:41 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:15:41.703 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jun 10 01:15:41 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-i[66802]: 2024-06-10T01:15:41.314+0000 7fe512307700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000215 [...2,head] /volumes/qa/sv_1/56b71f3b-2893-4c82-ac3a-2d6fa4194c93/client.0/tmp/clients/client0/~dmtmp/COREL/ auth{2=1} v108618 ap=2 f(v2700 m2024-06-10T01:15:19.300213+0000 11=11+0) n(v44 rc2024-06-10T01:15:19.300213+0000 b2861056 12=11+1) (inest sync->mix g=2) caps={24973=pAsLsXsFs/pAsLsXsFsxcral@7502} mcw={2=AsLsXsFs} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x55e19b93b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000215:[<0x10000000009/COREL v1556>,<0x10000000008/~dmtmp v34>,<0x10000000007/client0 v18>,<0x10000000006/clients v8>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/56b71f3b-2893-4c82-ac3a-2d6fa4194c93 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v37>,<0x1/volumes v37>]//[]","memoryvalue":"(2)0x10000000215:[<0x10000000009/COREL v108618>,<0x10000000008/~dmtmp v32039>,<0x10000000007/client0 v962>,<0x10000000006/clients v885>,<0x10000000005/tmp v871>,<0x100000001fb/client.0 v870>,<0x100000001fa/56b71f3b-2893-4c82-ac3a-2d6fa4194c93 v871>,<0x10000000001/sv_1 v878>,<0x10000000000/qa v858>,<0x1/volumes v821>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-10T01:15:18.871219+0000 15=15+0)","ondisk_value.rstat":"n(v0 rc2024-06-10T01:15:19.300213+0000 b2861056 12=11+1)","memory_value.dirstat":"f(v2700 m2024-06-10T01:15:19.300213+0000 11=11+0)","memory_value.rstat":"n(v44 rc2024-06-10T01:15:19.300213+0000 b2861056 12=11+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-10T01:15:41.703 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jun 10 01:15:41 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-i[66802]: 2024-06-10T01:15:41.320+0000 7fe512307700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023e [...2,head] /volumes/qa/sv_1/56b71f3b-2893-4c82-ac3a-2d6fa4194c93/client.0/tmp/clients/client0/~dmtmp/ACCESS/ auth{2=1} v108628 ap=2 f(v369 m2024-06-10T01:15:19.374212+0000 1=1+0) n(v13 rc2024-06-10T01:15:19.374212+0000 b260096 2=1+1) (inest sync->mix g=2) caps={24973=pAsLsXsFs/pAsLsXsFsxcral@6465} mcw={2=AsLsXsFs} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55e19b952100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000023e:[<0x10000000009/ACCESS v1608>,<0x10000000008/~dmtmp v34>,<0x10000000007/client0 v18>,<0x10000000006/clients v8>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/56b71f3b-2893-4c82-ac3a-2d6fa4194c93 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v37>,<0x1/volumes v37>]//[]","memoryvalue":"(2)0x1000000023e:[<0x10000000009/ACCESS v108628>,<0x10000000008/~dmtmp v32039>,<0x10000000007/client0 v962>,<0x10000000006/clients v885>,<0x10000000005/tmp v871>,<0x100000001fb/client.0 v870>,<0x100000001fa/56b71f3b-2893-4c82-ac3a-2d6fa4194c93 v871>,<0x10000000001/sv_1 v878>,<0x10000000000/qa v858>,<0x1/volumes v821>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-10T01:15:18.701221+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-06-10T01:15:19.374212+0000 b260096 2=1+1)","memory_value.dirstat":"f(v369 m2024-06-10T01:15:19.374212+0000 1=1+0)","memory_value.rstat":"n(v13 rc2024-06-10T01:15:19.374212+0000 b260096 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-10T01:15:41.745 INFO:tasks.workunit.client.0.smithi007.stdout: 1 678604 7.16 MB/sec execute 382 sec latency 13.224 ms 2024-06-10T01:15:41.780 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '844e91e0', total seconds elapsed: 27.5 2024-06-10T01:15:41.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:41 smithi078 ceph-mon[36683]: pgmap v722: 129 pgs: 129 active+clean; 109 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 312 KiB/s wr, 40 op/s 2024-06-10T01:15:41.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:41 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:15:41.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:41 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:15:42.466 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:15:42.726 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:15:42.746 INFO:tasks.workunit.client.0.smithi007.stdout: 1 680170 7.16 MB/sec execute 383 sec latency 17.292 ms 2024-06-10T01:15:42.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:42 smithi007 ceph-mon[32398]: scrub queued for path: / 2024-06-10T01:15:42.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:42 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:15:42.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:42 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:15:42.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:42 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:15:42.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:42 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:15:42.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:42 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:15:42.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:42 smithi007 ceph-mon[32398]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_1/56b71f3b-2893-4c82-ac3a-2d6fa4194c93/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.i log and `damage ls` output for details 2024-06-10T01:15:42.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:42 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:15:42.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:42 smithi007 ceph-mon[32398]: Scrub error on inode 0x1000000023e (/volumes/qa/sv_1/56b71f3b-2893-4c82-ac3a-2d6fa4194c93/client.0/tmp/clients/client0/~dmtmp/ACCESS) see mds.i log and `damage ls` output for details 2024-06-10T01:15:42.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:42 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:15:42.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:42 smithi078 ceph-mon[36683]: scrub queued for path: / 2024-06-10T01:15:42.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:42 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:15:42.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:42 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:15:42.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:42 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:15:42.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:42 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:15:42.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:42 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:15:42.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:42 smithi078 ceph-mon[36683]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_1/56b71f3b-2893-4c82-ac3a-2d6fa4194c93/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.i log and `damage ls` output for details 2024-06-10T01:15:42.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:42 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:15:42.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:42 smithi078 ceph-mon[36683]: Scrub error on inode 0x1000000023e (/volumes/qa/sv_1/56b71f3b-2893-4c82-ac3a-2d6fa4194c93/client.0/tmp/clients/client0/~dmtmp/ACCESS) see mds.i log and `damage ls` output for details 2024-06-10T01:15:42.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:42 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:15:42.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:42 smithi175 ceph-mon[36636]: scrub queued for path: / 2024-06-10T01:15:42.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:42 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:15:42.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:42 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:15:42.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:42 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:15:42.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:42 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:15:42.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:42 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:15:42.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:42 smithi175 ceph-mon[36636]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_1/56b71f3b-2893-4c82-ac3a-2d6fa4194c93/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.i log and `damage ls` output for details 2024-06-10T01:15:42.953 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:42 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:15:42.953 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:42 smithi175 ceph-mon[36636]: Scrub error on inode 0x1000000023e (/volumes/qa/sv_1/56b71f3b-2893-4c82-ac3a-2d6fa4194c93/client.0/tmp/clients/client0/~dmtmp/ACCESS) see mds.i log and `damage ls` output for details 2024-06-10T01:15:42.953 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:42 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:15:43.746 INFO:tasks.workunit.client.0.smithi007.stdout: 1 682273 7.18 MB/sec execute 384 sec latency 13.099 ms 2024-06-10T01:15:43.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:43 smithi007 ceph-mon[32398]: pgmap v723: 129 pgs: 129 active+clean; 109 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 160 KiB/s wr, 28 op/s 2024-06-10T01:15:43.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:43 smithi078 ceph-mon[36683]: pgmap v723: 129 pgs: 129 active+clean; 109 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 160 KiB/s wr, 28 op/s 2024-06-10T01:15:43.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:43 smithi175 ceph-mon[36636]: pgmap v723: 129 pgs: 129 active+clean; 109 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 160 KiB/s wr, 28 op/s 2024-06-10T01:15:44.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.394+0000 7f21db59d700 1 -- 172.21.15.7:0/397558065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21d4101d40 msgr2=0x7f21d410e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:44.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.394+0000 7f21db59d700 1 --2- 172.21.15.7:0/397558065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21d4101d40 0x7f21d410e200 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00b0a0 tx=0x7f21cc00b3b0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.394+0000 7f21db59d700 1 -- 172.21.15.7:0/397558065 shutdown_connections 2024-06-10T01:15:44.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.394+0000 7f21db59d700 1 --2- 172.21.15.7:0/397558065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21d4101d40 0x7f21d410e200 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.394+0000 7f21db59d700 1 --2- 172.21.15.7:0/397558065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21d41073c0 0x7f21d41077a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.394+0000 7f21db59d700 1 --2- 172.21.15.7:0/397558065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21d4101380 0x7f21d4101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.394+0000 7f21db59d700 1 -- 172.21.15.7:0/397558065 >> 172.21.15.7:0/397558065 conn(0x7f21d4078330 msgr2=0x7f21d4078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:44.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.395+0000 7f21db59d700 1 -- 172.21.15.7:0/397558065 shutdown_connections 2024-06-10T01:15:44.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.395+0000 7f21db59d700 1 -- 172.21.15.7:0/397558065 wait complete. 2024-06-10T01:15:44.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.395+0000 7f21db59d700 1 Processor -- start 2024-06-10T01:15:44.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.395+0000 7f21db59d700 1 -- start start 2024-06-10T01:15:44.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.396+0000 7f21db59d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21d4101380 0x7f21d4199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:44.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.396+0000 7f21db59d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21d4101d40 0x7f21d4199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:44.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.396+0000 7f21d9339700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21d4101380 0x7f21d4199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:44.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.396+0000 7f21db59d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21d41073c0 0x7f21d41a4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:44.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.396+0000 7f21db59d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f21d4111630 con 0x7f21d4101d40 2024-06-10T01:15:44.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.396+0000 7f21d9339700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21d4101380 0x7f21d4199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:40582/0 (socket says 172.21.15.7:40582) 2024-06-10T01:15:44.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.396+0000 7f21d9339700 1 -- 172.21.15.7:0/85799791 learned_addr learned my addr 172.21.15.7:0/85799791 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:44.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.396+0000 7f21d8b38700 1 --2- 172.21.15.7:0/85799791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21d4101d40 0x7f21d4199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:44.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.396+0000 7f21db59d700 1 -- 172.21.15.7:0/85799791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f21d41114b0 con 0x7f21d4101380 2024-06-10T01:15:44.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.396+0000 7f21db59d700 1 -- 172.21.15.7:0/85799791 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f21d41117b0 con 0x7f21d41073c0 2024-06-10T01:15:44.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.396+0000 7f21d9b3a700 1 --2- 172.21.15.7:0/85799791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21d41073c0 0x7f21d41a4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:44.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.397+0000 7f21d8b38700 1 -- 172.21.15.7:0/85799791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21d41073c0 msgr2=0x7f21d41a4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:44.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.397+0000 7f21d8b38700 1 --2- 172.21.15.7:0/85799791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21d41073c0 0x7f21d41a4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.397+0000 7f21d8b38700 1 -- 172.21.15.7:0/85799791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21d4101380 msgr2=0x7f21d4199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:44.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.397+0000 7f21d8b38700 1 --2- 172.21.15.7:0/85799791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21d4101380 0x7f21d4199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.397+0000 7f21d8b38700 1 -- 172.21.15.7:0/85799791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d41a4f50 con 0x7f21d4101d40 2024-06-10T01:15:44.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.397+0000 7f21d9339700 1 --2- 172.21.15.7:0/85799791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21d4101380 0x7f21d4199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:15:44.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.397+0000 7f21d8b38700 1 --2- 172.21.15.7:0/85799791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21d4101d40 0x7f21d4199ed0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f21c400eb50 tx=0x7f21c400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:44.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.397+0000 7f21ca7fc700 1 -- 172.21.15.7:0/85799791 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c400ca60 con 0x7f21d4101d40 2024-06-10T01:15:44.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.397+0000 7f21ca7fc700 1 -- 172.21.15.7:0/85799791 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21c400cc00 con 0x7f21d4101d40 2024-06-10T01:15:44.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.397+0000 7f21db59d700 1 -- 172.21.15.7:0/85799791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21d41a5180 con 0x7f21d4101d40 2024-06-10T01:15:44.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.397+0000 7f21ca7fc700 1 -- 172.21.15.7:0/85799791 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c4018710 con 0x7f21d4101d40 2024-06-10T01:15:44.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.397+0000 7f21db59d700 1 -- 172.21.15.7:0/85799791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21d41a5620 con 0x7f21d4101d40 2024-06-10T01:15:44.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.400+0000 7f21ca7fc700 1 -- 172.21.15.7:0/85799791 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f21c4010430 con 0x7f21d4101d40 2024-06-10T01:15:44.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.401+0000 7f21ca7fc700 1 --2- 172.21.15.7:0/85799791 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21c00a2d20 0x7f21c00a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:44.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.401+0000 7f21d9339700 1 --2- 172.21.15.7:0/85799791 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21c00a2d20 0x7f21c00a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:44.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.401+0000 7f21ca7fc700 1 -- 172.21.15.7:0/85799791 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f21c40cf5a0 con 0x7f21d4101d40 2024-06-10T01:15:44.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.402+0000 7f21d9339700 1 --2- 172.21.15.7:0/85799791 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21c00a2d20 0x7f21c00a51e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f21d0009d80 tx=0x7f21d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:44.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.403+0000 7f21e5e20b80 1 Processor -- start 2024-06-10T01:15:44.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.403+0000 7f21e5e20b80 1 -- start start 2024-06-10T01:15:44.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.403+0000 7f21e5e20b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcab120640 0x55bcab120a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:44.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.403+0000 7f21e5e20b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bcab120f60 0x55bcab11a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:44.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.403+0000 7f21e5e20b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcab11af20 0x55bcab11b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:44.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.403+0000 7f21e5e20b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bcab06c810 con 0x55bcab120640 2024-06-10T01:15:44.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.403+0000 7f21e5e20b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bcab014720 con 0x55bcab11af20 2024-06-10T01:15:44.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.403+0000 7f21e5e20b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bcab0687d0 con 0x55bcab120f60 2024-06-10T01:15:44.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.403+0000 7f21d9339700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcab120640 0x55bcab120a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:44.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.403+0000 7f21d9b3a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcab11af20 0x55bcab11b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:44.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.403+0000 7f21d9b3a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcab11af20 0x55bcab11b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:40584/0 (socket says 172.21.15.7:40584) 2024-06-10T01:15:44.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.403+0000 7f21d9b3a700 1 -- 172.21.15.7:0/2488242623 learned_addr learned my addr 172.21.15.7:0/2488242623 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:44.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.403+0000 7f21d9339700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcab120640 0x55bcab120a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53928/0 (socket says 172.21.15.7:53928) 2024-06-10T01:15:44.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.404+0000 7f21d9b3a700 1 -- 172.21.15.7:0/2488242623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bcab120f60 msgr2=0x55bcab11a9e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T01:15:44.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.404+0000 7f21d9b3a700 1 --2- 172.21.15.7:0/2488242623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bcab120f60 0x55bcab11a9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.404+0000 7f21d9b3a700 1 -- 172.21.15.7:0/2488242623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcab120640 msgr2=0x55bcab120a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:44.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.404+0000 7f21d9b3a700 1 --2- 172.21.15.7:0/2488242623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcab120640 0x55bcab120a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.404+0000 7f21d9b3a700 1 -- 172.21.15.7:0/2488242623 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcab00af50 con 0x55bcab11af20 2024-06-10T01:15:44.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.404+0000 7f21d9b3a700 1 --2- 172.21.15.7:0/2488242623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcab11af20 0x55bcab11b340 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f21cc009e60 tx=0x7f21cc0092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0f02139c66daf93 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:44.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.404+0000 7f21bf7fe700 1 -- 172.21.15.7:0/2488242623 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc020070 con 0x55bcab11af20 2024-06-10T01:15:44.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.404+0000 7f21bf7fe700 1 -- 172.21.15.7:0/2488242623 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21cc006020 con 0x55bcab11af20 2024-06-10T01:15:44.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.404+0000 7f21bf7fe700 1 -- 172.21.15.7:0/2488242623 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc007c50 con 0x55bcab11af20 2024-06-10T01:15:44.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.405+0000 7f21e5e20b80 1 -- 172.21.15.7:0/2488242623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcab11af20 msgr2=0x55bcab11b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:44.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.405+0000 7f21e5e20b80 1 --2- 172.21.15.7:0/2488242623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcab11af20 0x55bcab11b340 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f21cc009e60 tx=0x7f21cc0092e0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.405+0000 7f21e5e20b80 1 -- 172.21.15.7:0/2488242623 shutdown_connections 2024-06-10T01:15:44.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.405+0000 7f21e5e20b80 1 --2- 172.21.15.7:0/2488242623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcab11af20 0x55bcab11b340 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.405+0000 7f21e5e20b80 1 --2- 172.21.15.7:0/2488242623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcab120640 0x55bcab120a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.405+0000 7f21e5e20b80 1 --2- 172.21.15.7:0/2488242623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bcab120f60 0x55bcab11a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.405+0000 7f21e5e20b80 1 -- 172.21.15.7:0/2488242623 >> 172.21.15.7:0/2488242623 conn(0x55bcab045cc0 msgr2=0x55bcab115850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:44.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.405+0000 7f21e5e20b80 1 -- 172.21.15.7:0/2488242623 shutdown_connections 2024-06-10T01:15:44.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.405+0000 7f21e5e20b80 1 -- 172.21.15.7:0/2488242623 wait complete. 2024-06-10T01:15:44.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.405+0000 7f21e5e20b80 1 -- start start 2024-06-10T01:15:44.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.406+0000 7f21e5e20b80 1 Processor -- start 2024-06-10T01:15:44.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.406+0000 7f21e5e20b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcab11af20 0x55bcab126e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:44.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.406+0000 7f21e5e20b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcab120640 0x55bcab124100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:44.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.406+0000 7f21e5e20b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bcab120f60 0x55bcab124640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:44.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.406+0000 7f21e5e20b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bcab127970 con 0x55bcab11af20 2024-06-10T01:15:44.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.406+0000 7f21e5e20b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bcab00cbf0 con 0x55bcab120640 2024-06-10T01:15:44.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.406+0000 7f21e5e20b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bcab127af0 con 0x55bcab120f60 2024-06-10T01:15:44.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.406+0000 7f21e5e20b80 10 client.0 _ll_register_callbacks cb 0x55bcab006920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:15:44.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.406+0000 7f21d8b38700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcab120640 0x55bcab124100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:44.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.406+0000 7f21d8b38700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcab120640 0x55bcab124100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:40586/0 (socket says 172.21.15.7:40586) 2024-06-10T01:15:44.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.406+0000 7f21d8b38700 1 -- 172.21.15.7:0/634101184 learned_addr learned my addr 172.21.15.7:0/634101184 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:44.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.407+0000 7f21d9339700 1 --2- 172.21.15.7:0/634101184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcab11af20 0x55bcab126e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:44.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.407+0000 7f21d8b38700 1 -- 172.21.15.7:0/634101184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bcab120f60 msgr2=0x55bcab124640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T01:15:44.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.407+0000 7f21d8b38700 1 --2- 172.21.15.7:0/634101184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bcab120f60 0x55bcab124640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.407+0000 7f21d8b38700 1 -- 172.21.15.7:0/634101184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcab11af20 msgr2=0x55bcab126e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:44.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.407+0000 7f21d8b38700 1 --2- 172.21.15.7:0/634101184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcab11af20 0x55bcab126e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.407+0000 7f21d8b38700 1 -- 172.21.15.7:0/634101184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcaafbb8d0 con 0x55bcab120640 2024-06-10T01:15:44.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.407+0000 7f21d9339700 1 --2- 172.21.15.7:0/634101184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcab11af20 0x55bcab126e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:15:44.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.407+0000 7f21d8b38700 1 --2- 172.21.15.7:0/634101184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcab120640 0x55bcab124100 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f21c40a4160 tx=0x7f21c409b4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6443e897bbd27fc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:44.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.407+0000 7f21bcff9700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:15:44.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.407+0000 7f21bcff9700 1 -- 172.21.15.7:0/634101184 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c409ba20 con 0x55bcab120640 2024-06-10T01:15:44.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.407+0000 7f21bcff9700 1 -- 172.21.15.7:0/634101184 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f21c4099d20 con 0x55bcab120640 2024-06-10T01:15:44.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.407+0000 7f21e5e20b80 1 -- 172.21.15.7:0/634101184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21cc006020 con 0x55bcab120640 2024-06-10T01:15:44.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.408+0000 7f21bcff9700 1 -- 172.21.15.7:0/634101184 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f21c409fdd0 con 0x55bcab120640 2024-06-10T01:15:44.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.408+0000 7f21e5e20b80 10 client.27323 fetch_fsmap learned FSMap version 37 2024-06-10T01:15:44.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.408+0000 7f21e5e20b80 1 -- 172.21.15.7:0/634101184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bcab143470 con 0x55bcab120640 2024-06-10T01:15:44.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.408+0000 7f21bcff9700 1 -- 172.21.15.7:0/634101184 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21cc006020 con 0x55bcab120640 2024-06-10T01:15:44.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.408+0000 7f21bcff9700 1 -- 172.21.15.7:0/634101184 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f21c409ae70 con 0x55bcab120640 2024-06-10T01:15:44.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.408+0000 7f21e5e20b80 10 client.27323 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T01:15:44.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.408+0000 7f21e5e20b80 10 client.27323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:15:44.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.408+0000 7f21e5e20b80 20 client.27323 populate_metadata read hostname 'smithi007' 2024-06-10T01:15:44.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.408+0000 7f21e5e20b80 1 --2- 172.21.15.7:0/634101184 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55bcab145f80 0x55bcab1483c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:44.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.408+0000 7f21e5e20b80 4 client.27323 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:15:44.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.408+0000 7f21e5e20b80 1 -- 172.21.15.7:0/634101184 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcab079d10 con 0x55bcab145f80 2024-06-10T01:15:44.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.408+0000 7f21d9339700 1 --2- 172.21.15.7:0/634101184 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55bcab145f80 0x55bcab1483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:44.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.409+0000 7f21bcff9700 10 client.27323 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:44.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.410+0000 7f21d9339700 1 --2- 172.21.15.7:0/634101184 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55bcab145f80 0x55bcab1483c0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a228fe4b59fc9634 server_cookie=fd16ada4c36d7670 in_seq=0 out_seq=0 2024-06-10T01:15:44.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.411+0000 7f21bcff9700 1 -- 172.21.15.7:0/634101184 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bcab079d10 con 0x55bcab145f80 2024-06-10T01:15:44.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.411+0000 7f21bcff9700 10 client.27323 handle_command_reply: tid=0 2024-06-10T01:15:44.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.411+0000 7f21e5e20b80 1 client.27323 shutdown 2024-06-10T01:15:44.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.411+0000 7f21e5e20b80 1 -- 172.21.15.7:0/634101184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcab120640 msgr2=0x55bcab124100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:44.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.411+0000 7f21e5e20b80 1 --2- 172.21.15.7:0/634101184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcab120640 0x55bcab124100 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f21c40a4160 tx=0x7f21c409b4b0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.411+0000 7f21e5e20b80 1 -- 172.21.15.7:0/634101184 shutdown_connections 2024-06-10T01:15:44.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.411+0000 7f21e5e20b80 1 --2- 172.21.15.7:0/634101184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bcab120f60 0x55bcab124640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.411+0000 7f21e5e20b80 1 --2- 172.21.15.7:0/634101184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcab11af20 0x55bcab126e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.411+0000 7f21e5e20b80 1 --2- 172.21.15.7:0/634101184 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55bcab145f80 0x55bcab1483c0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.411+0000 7f21e5e20b80 1 --2- 172.21.15.7:0/634101184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcab120640 0x55bcab124100 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.411+0000 7f21e5e20b80 1 -- 172.21.15.7:0/634101184 >> 172.21.15.7:0/634101184 conn(0x55bcab045cc0 msgr2=0x55bcab1199d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:44.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.411+0000 7f21bcff9700 0 client.27323 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:44.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.411+0000 7f21e5e20b80 1 -- 172.21.15.7:0/634101184 shutdown_connections 2024-06-10T01:15:44.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.411+0000 7f21e5e20b80 1 -- 172.21.15.7:0/634101184 wait complete. 2024-06-10T01:15:44.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.411+0000 7f21e5e20b80 20 client.27323 trim_cache size 0 max 16384 2024-06-10T01:15:44.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.439+0000 7f21e5e20b80 1 Processor -- start 2024-06-10T01:15:44.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.439+0000 7f21e5e20b80 1 -- start start 2024-06-10T01:15:44.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.439+0000 7f21e5e20b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcab11af20 0x55bcab11a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:44.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.439+0000 7f21e5e20b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcab120640 0x55bcab11e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:44.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.439+0000 7f21e5e20b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bcab120f60 0x55bcab14dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:44.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.439+0000 7f21e5e20b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f21d41aa550 con 0x55bcab11af20 2024-06-10T01:15:44.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.439+0000 7f21e5e20b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f21d40726c0 con 0x55bcab120640 2024-06-10T01:15:44.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.439+0000 7f21e5e20b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f21d4070510 con 0x55bcab120f60 2024-06-10T01:15:44.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.440+0000 7f21d9339700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcab11af20 0x55bcab11a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:44.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.440+0000 7f21d8b38700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcab120640 0x55bcab11e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:44.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.440+0000 7f21d8b38700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcab120640 0x55bcab11e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:40590/0 (socket says 172.21.15.7:40590) 2024-06-10T01:15:44.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.440+0000 7f21d9339700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcab11af20 0x55bcab11a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53948/0 (socket says 172.21.15.7:53948) 2024-06-10T01:15:44.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.440+0000 7f21d8b38700 1 -- 172.21.15.7:0/4143272067 learned_addr learned my addr 172.21.15.7:0/4143272067 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:44.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.440+0000 7f21d9b3a700 1 --2- 172.21.15.7:0/4143272067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bcab120f60 0x55bcab14dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:44.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.440+0000 7f21d8b38700 1 -- 172.21.15.7:0/4143272067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bcab120f60 msgr2=0x55bcab14dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:44.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.440+0000 7f21d8b38700 1 --2- 172.21.15.7:0/4143272067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bcab120f60 0x55bcab14dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.440+0000 7f21d8b38700 1 -- 172.21.15.7:0/4143272067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcab11af20 msgr2=0x55bcab11a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:44.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.440+0000 7f21d8b38700 1 --2- 172.21.15.7:0/4143272067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcab11af20 0x55bcab11a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.440+0000 7f21d8b38700 1 -- 172.21.15.7:0/4143272067 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcab126a40 con 0x55bcab120640 2024-06-10T01:15:44.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.440+0000 7f21d9339700 1 --2- 172.21.15.7:0/4143272067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcab11af20 0x55bcab11a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:44.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.440+0000 7f21d8b38700 1 --2- 172.21.15.7:0/4143272067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcab120640 0x55bcab11e7d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f21c409b240 tx=0x7f21c40092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=880f85ba45ed2cdf server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:44.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.440+0000 7f21beffd700 1 -- 172.21.15.7:0/4143272067 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c40a4690 con 0x55bcab120640 2024-06-10T01:15:44.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.441+0000 7f21beffd700 1 -- 172.21.15.7:0/4143272067 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21c40ce030 con 0x55bcab120640 2024-06-10T01:15:44.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.441+0000 7f21beffd700 1 -- 172.21.15.7:0/4143272067 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c4005550 con 0x55bcab120640 2024-06-10T01:15:44.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.441+0000 7f21e5e20b80 1 -- 172.21.15.7:0/4143272067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcab120640 msgr2=0x55bcab11e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:44.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.441+0000 7f21e5e20b80 1 --2- 172.21.15.7:0/4143272067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcab120640 0x55bcab11e7d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f21c409b240 tx=0x7f21c40092d0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.441+0000 7f21e5e20b80 1 -- 172.21.15.7:0/4143272067 shutdown_connections 2024-06-10T01:15:44.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.441+0000 7f21e5e20b80 1 --2- 172.21.15.7:0/4143272067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bcab120f60 0x55bcab14dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.441+0000 7f21e5e20b80 1 --2- 172.21.15.7:0/4143272067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcab11af20 0x55bcab11a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.441+0000 7f21e5e20b80 1 --2- 172.21.15.7:0/4143272067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcab120640 0x55bcab11e7d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.441+0000 7f21e5e20b80 1 -- 172.21.15.7:0/4143272067 >> 172.21.15.7:0/4143272067 conn(0x55bcab045cc0 msgr2=0x55bcab11e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:44.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.441+0000 7f21e5e20b80 1 -- 172.21.15.7:0/4143272067 shutdown_connections 2024-06-10T01:15:44.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.441+0000 7f21e5e20b80 1 -- 172.21.15.7:0/4143272067 wait complete. 2024-06-10T01:15:44.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.442+0000 7f21e5e20b80 1 -- start start 2024-06-10T01:15:44.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.442+0000 7f21e5e20b80 1 Processor -- start 2024-06-10T01:15:44.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.442+0000 7f21e5e20b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcab11af20 0x55bcab12d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:44.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.442+0000 7f21e5e20b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcab120640 0x55bcab14d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:44.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.442+0000 7f21e5e20b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bcab120f60 0x55bcab14d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:44.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.442+0000 7f21e5e20b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f21d4070b80 con 0x55bcab120640 2024-06-10T01:15:44.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.442+0000 7f21e5e20b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f21d4071ff0 con 0x55bcab11af20 2024-06-10T01:15:44.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.443+0000 7f21d9339700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcab11af20 0x55bcab12d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:44.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.443+0000 7f21d9339700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcab11af20 0x55bcab12d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:40602/0 (socket says 172.21.15.7:40602) 2024-06-10T01:15:44.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.443+0000 7f21d9339700 1 -- 172.21.15.7:0/1847695645 learned_addr learned my addr 172.21.15.7:0/1847695645 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:44.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.443+0000 7f21e5e20b80 1 -- 172.21.15.7:0/1847695645 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f21d4071250 con 0x55bcab120f60 2024-06-10T01:15:44.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.443+0000 7f21e5e20b80 10 client.0 _ll_register_callbacks cb 0x55bcab006920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:15:44.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.443+0000 7f21d8b38700 1 --2- 172.21.15.7:0/1847695645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcab120640 0x55bcab14d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:44.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.443+0000 7f21d9b3a700 1 --2- 172.21.15.7:0/1847695645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bcab120f60 0x55bcab14d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:44.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.443+0000 7f21d9b3a700 1 -- 172.21.15.7:0/1847695645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcab11af20 msgr2=0x55bcab12d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:44.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.443+0000 7f21d9b3a700 1 --2- 172.21.15.7:0/1847695645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcab11af20 0x55bcab12d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.443+0000 7f21d9b3a700 1 -- 172.21.15.7:0/1847695645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcab120640 msgr2=0x55bcab14d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:44.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.443+0000 7f21d9b3a700 1 --2- 172.21.15.7:0/1847695645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcab120640 0x55bcab14d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.443+0000 7f21d9b3a700 1 -- 172.21.15.7:0/1847695645 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcab14dd80 con 0x55bcab120f60 2024-06-10T01:15:44.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.443+0000 7f21d8b38700 1 --2- 172.21.15.7:0/1847695645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcab120640 0x55bcab14d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:44.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.443+0000 7f21d9b3a700 1 --2- 172.21.15.7:0/1847695645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bcab120f60 0x55bcab14d840 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f21cc000f80 tx=0x7f21cc003c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcdbf476e5b1026f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:44.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.444+0000 7f21bcff9700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:15:44.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.444+0000 7f21bcff9700 1 -- 172.21.15.7:0/1847695645 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc020040 con 0x55bcab120f60 2024-06-10T01:15:44.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.444+0000 7f21e5e20b80 1 -- 172.21.15.7:0/1847695645 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21c40ce030 con 0x55bcab120f60 2024-06-10T01:15:44.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.444+0000 7f21bcff9700 1 -- 172.21.15.7:0/1847695645 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f21cc0074a0 con 0x55bcab120f60 2024-06-10T01:15:44.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.444+0000 7f21bcff9700 1 -- 172.21.15.7:0/1847695645 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f21cc035050 con 0x55bcab120f60 2024-06-10T01:15:44.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.444+0000 7f21e5e20b80 10 client.27349 fetch_fsmap learned FSMap version 37 2024-06-10T01:15:44.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.444+0000 7f21e5e20b80 1 -- 172.21.15.7:0/1847695645 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bcab1b0960 con 0x55bcab120f60 2024-06-10T01:15:44.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.444+0000 7f21bcff9700 1 -- 172.21.15.7:0/1847695645 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21c40ce030 con 0x55bcab120f60 2024-06-10T01:15:44.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.445+0000 7f21bcff9700 1 -- 172.21.15.7:0/1847695645 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f21cc011070 con 0x55bcab120f60 2024-06-10T01:15:44.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.445+0000 7f21e5e20b80 10 client.27349 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T01:15:44.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.445+0000 7f21e5e20b80 10 client.27349 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:15:44.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.445+0000 7f21e5e20b80 20 client.27349 populate_metadata read hostname 'smithi007' 2024-06-10T01:15:44.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.445+0000 7f21e5e20b80 1 --2- 172.21.15.7:0/1847695645 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55bcab1b0bd0 0x55bcab1b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:44.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.445+0000 7f21e5e20b80 4 client.27349 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:15:44.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.445+0000 7f21e5e20b80 1 -- 172.21.15.7:0/1847695645 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bcab074290 con 0x55bcab1b0bd0 2024-06-10T01:15:44.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.445+0000 7f21d9339700 1 --2- 172.21.15.7:0/1847695645 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55bcab1b0bd0 0x55bcab1b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:44.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.445+0000 7f21d9339700 1 --2- 172.21.15.7:0/1847695645 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55bcab1b0bd0 0x55bcab1b2fc0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=148fb03cf8b87f60 server_cookie=990571f304e4b1bf in_seq=0 out_seq=0 2024-06-10T01:15:44.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.445+0000 7f21bcff9700 10 client.27349 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:44.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.446+0000 7f21bcff9700 1 -- 172.21.15.7:0/1847695645 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bcab074290 con 0x55bcab1b0bd0 2024-06-10T01:15:44.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.446+0000 7f21bcff9700 10 client.27349 handle_command_reply: tid=0 2024-06-10T01:15:44.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.446+0000 7f21e5e20b80 1 client.27349 shutdown 2024-06-10T01:15:44.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.446+0000 7f21e5e20b80 1 -- 172.21.15.7:0/1847695645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bcab120f60 msgr2=0x55bcab14d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:44.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.446+0000 7f21e5e20b80 1 --2- 172.21.15.7:0/1847695645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bcab120f60 0x55bcab14d840 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f21cc000f80 tx=0x7f21cc003c30 comp rx=0 tx=0).stop 2024-06-10T01:15:44.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.446+0000 7f21e5e20b80 1 -- 172.21.15.7:0/1847695645 shutdown_connections 2024-06-10T01:15:44.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.446+0000 7f21e5e20b80 1 --2- 172.21.15.7:0/1847695645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bcab120f60 0x55bcab14d840 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.446+0000 7f21e5e20b80 1 --2- 172.21.15.7:0/1847695645 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55bcab1b0bd0 0x55bcab1b2fc0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.446+0000 7f21e5e20b80 1 --2- 172.21.15.7:0/1847695645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcab11af20 0x55bcab12d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.446+0000 7f21e5e20b80 1 --2- 172.21.15.7:0/1847695645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcab120640 0x55bcab14d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.446+0000 7f21bcff9700 0 client.27349 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:44.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.446+0000 7f21e5e20b80 1 -- 172.21.15.7:0/1847695645 >> 172.21.15.7:0/1847695645 conn(0x55bcab045cc0 msgr2=0x55bcab10ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:44.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.446+0000 7f21e5e20b80 1 -- 172.21.15.7:0/1847695645 shutdown_connections 2024-06-10T01:15:44.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.446+0000 7f21e5e20b80 1 -- 172.21.15.7:0/1847695645 wait complete. 2024-06-10T01:15:44.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.446+0000 7f21e5e20b80 20 client.27349 trim_cache size 0 max 16384 2024-06-10T01:15:44.446 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:15:44.446 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T01:15:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-10T01:15:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: "8173abd2-d4b6-4232-8489-8b7e45a1a4f6": { 2024-06-10T01:15:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-10T01:15:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "8173abd2-d4b6-4232-8489-8b7e45a1a4f6", 2024-06-10T01:15:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-10T01:15:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:15:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:15:44.447 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:15:44.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.447+0000 7f21db59d700 1 -- 172.21.15.7:0/85799791 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21c00a2d20 msgr2=0x7f21c00a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:44.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.447+0000 7f21db59d700 1 --2- 172.21.15.7:0/85799791 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21c00a2d20 0x7f21c00a51e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f21d0009d80 tx=0x7f21d0009260 comp rx=0 tx=0).stop 2024-06-10T01:15:44.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.447+0000 7f21db59d700 1 -- 172.21.15.7:0/85799791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21d4101d40 msgr2=0x7f21d4199ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:44.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.447+0000 7f21db59d700 1 --2- 172.21.15.7:0/85799791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21d4101d40 0x7f21d4199ed0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f21c400eb50 tx=0x7f21c400ee60 comp rx=0 tx=0).stop 2024-06-10T01:15:44.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.447+0000 7f21db59d700 1 -- 172.21.15.7:0/85799791 shutdown_connections 2024-06-10T01:15:44.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.447+0000 7f21db59d700 1 --2- 172.21.15.7:0/85799791 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21c00a2d20 0x7f21c00a51e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.447+0000 7f21db59d700 1 --2- 172.21.15.7:0/85799791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21d41073c0 0x7f21d41a4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.447+0000 7f21db59d700 1 --2- 172.21.15.7:0/85799791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21d4101380 0x7f21d4199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.448+0000 7f21db59d700 1 --2- 172.21.15.7:0/85799791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21d4101d40 0x7f21d4199ed0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:44.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.448+0000 7f21db59d700 1 -- 172.21.15.7:0/85799791 >> 172.21.15.7:0/85799791 conn(0x7f21d4078330 msgr2=0x7f21d40fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:44.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.448+0000 7f21db59d700 1 -- 172.21.15.7:0/85799791 shutdown_connections 2024-06-10T01:15:44.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:15:44.448+0000 7f21db59d700 1 -- 172.21.15.7:0/85799791 wait complete. 2024-06-10T01:15:44.746 INFO:tasks.workunit.client.0.smithi007.stdout: 1 683836 7.17 MB/sec execute 385 sec latency 30.860 ms 2024-06-10T01:15:44.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:44 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:15:44.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:44 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:15:44.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:44 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:15:44.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:44 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:15:44.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:44 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:15:44.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:44 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:15:44.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:44 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:15:44.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:44 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:15:44.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:44 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:15:44.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:44 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:15:44.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:44 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:15:44.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:44 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:15:44.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:44 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:15:44.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:44 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:15:44.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:44 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:15:45.132 INFO:tasks.cephfs.filesystem:scrub status for tag:8173abd2-d4b6-4232-8489-8b7e45a1a4f6 - {'path': '/', 'tag': '8173abd2-d4b6-4232-8489-8b7e45a1a4f6', 'options': 'recursive,force'} 2024-06-10T01:15:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.521+0000 7f78e6bce640 1 -- 172.21.15.7:0/955352132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78e0107780 msgr2=0x7f78e0109b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.521+0000 7f78e6bce640 1 --2- 172.21.15.7:0/955352132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78e0107780 0x7f78e0109b70 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f78d8009ca0 tx=0x7f78d802a8c0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.521+0000 7f78e6bce640 1 -- 172.21.15.7:0/955352132 shutdown_connections 2024-06-10T01:15:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.521+0000 7f78e6bce640 1 --2- 172.21.15.7:0/955352132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78e0107780 0x7f78e0109b70 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.521+0000 7f78e6bce640 1 --2- 172.21.15.7:0/955352132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78e0104e50 0x7f78e0107240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.521+0000 7f78e6bce640 1 --2- 172.21.15.7:0/955352132 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78e0102520 0x7f78e0104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.521+0000 7f78e6bce640 1 -- 172.21.15.7:0/955352132 >> 172.21.15.7:0/955352132 conn(0x7f78e00fbeb0 msgr2=0x7f78e00fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.521+0000 7f78e6bce640 1 -- 172.21.15.7:0/955352132 shutdown_connections 2024-06-10T01:15:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.521+0000 7f78e6bce640 1 -- 172.21.15.7:0/955352132 wait complete. 2024-06-10T01:15:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.522+0000 7f78e6bce640 1 Processor -- start 2024-06-10T01:15:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.522+0000 7f78e6bce640 1 -- start start 2024-06-10T01:15:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.522+0000 7f78e6bce640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78e0102520 0x7f78e0197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.522+0000 7f78e6bce640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78e0104e50 0x7f78e0198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.522+0000 7f78e6bce640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78e0107780 0x7f78e019f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.522+0000 7f78e6bce640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f78e010d6c0 con 0x7f78e0107780 2024-06-10T01:15:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.522+0000 7f78e6bce640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f78e010d540 con 0x7f78e0102520 2024-06-10T01:15:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.522+0000 7f78e6bce640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f78e010d840 con 0x7f78e0104e50 2024-06-10T01:15:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.522+0000 7f78e516b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78e0107780 0x7f78e019f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.522+0000 7f78e516b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78e0107780 0x7f78e019f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53974/0 (socket says 172.21.15.7:53974) 2024-06-10T01:15:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.522+0000 7f78e516b640 1 -- 172.21.15.7:0/4147000100 learned_addr learned my addr 172.21.15.7:0/4147000100 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.522+0000 7f78d7fff640 1 --2- 172.21.15.7:0/4147000100 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78e0104e50 0x7f78e0198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.522+0000 7f78e496a640 1 --2- 172.21.15.7:0/4147000100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78e0102520 0x7f78e0197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.522+0000 7f78e516b640 1 -- 172.21.15.7:0/4147000100 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78e0104e50 msgr2=0x7f78e0198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.522+0000 7f78e516b640 1 --2- 172.21.15.7:0/4147000100 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78e0104e50 0x7f78e0198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.522+0000 7f78e516b640 1 -- 172.21.15.7:0/4147000100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78e0102520 msgr2=0x7f78e0197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.522+0000 7f78e516b640 1 --2- 172.21.15.7:0/4147000100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78e0102520 0x7f78e0197d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.522+0000 7f78e516b640 1 -- 172.21.15.7:0/4147000100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e019f910 con 0x7f78e0107780 2024-06-10T01:15:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.523+0000 7f78e516b640 1 --2- 172.21.15.7:0/4147000100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78e0107780 0x7f78e019f270 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f78d8031ae0 tx=0x7f78d8031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.523+0000 7f78d5ffb640 1 -- 172.21.15.7:0/4147000100 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d8003c30 con 0x7f78e0107780 2024-06-10T01:15:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.523+0000 7f78d5ffb640 1 -- 172.21.15.7:0/4147000100 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78d8003dd0 con 0x7f78e0107780 2024-06-10T01:15:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.523+0000 7f78e6bce640 1 -- 172.21.15.7:0/4147000100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78e019fba0 con 0x7f78e0107780 2024-06-10T01:15:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.523+0000 7f78d5ffb640 1 -- 172.21.15.7:0/4147000100 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d802fcb0 con 0x7f78e0107780 2024-06-10T01:15:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.523+0000 7f78e6bce640 1 -- 172.21.15.7:0/4147000100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78e01a00f0 con 0x7f78e0107780 2024-06-10T01:15:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.526+0000 7f78e99bb740 1 Processor -- start 2024-06-10T01:15:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.526+0000 7f78d5ffb640 1 -- 172.21.15.7:0/4147000100 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f78d8004050 con 0x7f78e0107780 2024-06-10T01:15:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.526+0000 7f78e99bb740 1 -- start start 2024-06-10T01:15:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.526+0000 7f78e99bb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c313fd2f90 0x55c313fd33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.526+0000 7f78e99bb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c313fd3980 0x55c313fd6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.526+0000 7f78e99bb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c313fd6ae0 0x55c313fe3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.526+0000 7f78e99bb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c313f364f0 con 0x55c313fd3980 2024-06-10T01:15:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.526+0000 7f78e99bb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c313ed4870 con 0x55c313fd2f90 2024-06-10T01:15:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.526+0000 7f78e99bb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c313f324b0 con 0x55c313fd6ae0 2024-06-10T01:15:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.527+0000 7f78e496a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c313fd3980 0x55c313fd6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.527+0000 7f78e496a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c313fd3980 0x55c313fd6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53984/0 (socket says 172.21.15.7:53984) 2024-06-10T01:15:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.527+0000 7f78e496a640 1 -- 172.21.15.7:0/2662080537 learned_addr learned my addr 172.21.15.7:0/2662080537 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.527+0000 7f78d7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c313fd2f90 0x55c313fd33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.527+0000 7f78d7fff640 1 --2- 172.21.15.7:0/2662080537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c313fd6ae0 0x55c313fe3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.527+0000 7f78e496a640 1 -- 172.21.15.7:0/2662080537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c313fd6ae0 msgr2=0x55c313fe3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.527+0000 7f78e496a640 1 --2- 172.21.15.7:0/2662080537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c313fd6ae0 0x55c313fe3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.527+0000 7f78e496a640 1 -- 172.21.15.7:0/2662080537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c313fd2f90 msgr2=0x55c313fd33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.527+0000 7f78e496a640 1 --2- 172.21.15.7:0/2662080537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c313fd2f90 0x55c313fd33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.527+0000 7f78e496a640 1 -- 172.21.15.7:0/2662080537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c313ef09b0 con 0x55c313fd3980 2024-06-10T01:15:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.527+0000 7f78d7fff640 1 --2- 172.21.15.7:0/2662080537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c313fd2f90 0x55c313fd33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.527+0000 7f78d5ffb640 1 --2- 172.21.15.7:0/4147000100 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f78c80a2b80 0x7f78c80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.527+0000 7f78d7fff640 1 --2- 172.21.15.7:0/4147000100 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f78c80a2b80 0x7f78c80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.527+0000 7f78e496a640 1 --2- 172.21.15.7:0/2662080537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c313fd3980 0x55c313fd6470 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00bcb0 tx=0x7f78dc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d023d3807f994aa2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.527+0000 7f78d5ffb640 1 -- 172.21.15.7:0/4147000100 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f78d80f06a0 con 0x7f78e0107780 2024-06-10T01:15:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.527+0000 7f78c6ffd640 1 -- 172.21.15.7:0/2662080537 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc01c480 con 0x55c313fd3980 2024-06-10T01:15:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.527+0000 7f78c6ffd640 1 -- 172.21.15.7:0/2662080537 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78dc01cb00 con 0x55c313fd3980 2024-06-10T01:15:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.527+0000 7f78c6ffd640 1 -- 172.21.15.7:0/2662080537 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc011cd0 con 0x55c313fd3980 2024-06-10T01:15:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.528+0000 7f78e99bb740 1 -- 172.21.15.7:0/2662080537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c313fd3980 msgr2=0x55c313fd6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.528+0000 7f78d7fff640 1 --2- 172.21.15.7:0/4147000100 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f78c80a2b80 0x7f78c80a5040 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55c313fd41e0 tx=0x7f78d0004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.528+0000 7f78e99bb740 1 --2- 172.21.15.7:0/2662080537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c313fd3980 0x55c313fd6470 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00bcb0 tx=0x7f78dc0107a0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.528+0000 7f78e99bb740 1 -- 172.21.15.7:0/2662080537 shutdown_connections 2024-06-10T01:15:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.528+0000 7f78e99bb740 1 --2- 172.21.15.7:0/2662080537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c313fd6ae0 0x55c313fe3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.528+0000 7f78e99bb740 1 --2- 172.21.15.7:0/2662080537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c313fd3980 0x55c313fd6470 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.528+0000 7f78e99bb740 1 --2- 172.21.15.7:0/2662080537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c313fd2f90 0x55c313fd33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.528+0000 7f78e99bb740 1 -- 172.21.15.7:0/2662080537 >> 172.21.15.7:0/2662080537 conn(0x55c313fce680 msgr2=0x55c313fcea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.528+0000 7f78e99bb740 1 -- 172.21.15.7:0/2662080537 shutdown_connections 2024-06-10T01:15:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.528+0000 7f78e99bb740 1 -- 172.21.15.7:0/2662080537 wait complete. 2024-06-10T01:15:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.529+0000 7f78e99bb740 1 -- start start 2024-06-10T01:15:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.529+0000 7f78e99bb740 1 Processor -- start 2024-06-10T01:15:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.529+0000 7f78e99bb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c313fd2f90 0x55c313fe5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.529+0000 7f78e99bb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c313fd3980 0x55c313fe5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.529+0000 7f78e99bb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c313fd6ae0 0x55c313fed010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.529+0000 7f78e99bb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c313fe4350 con 0x55c313fd2f90 2024-06-10T01:15:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.529+0000 7f78e99bb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c313ebfae0 con 0x55c313fd6ae0 2024-06-10T01:15:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.529+0000 7f78e99bb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c313fe44d0 con 0x55c313fd3980 2024-06-10T01:15:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.529+0000 7f78e99bb740 10 client.0 _ll_register_callbacks cb 0x55c313ef89b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:15:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.529+0000 7f78e496a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c313fd2f90 0x55c313fe5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.529+0000 7f78e496a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c313fd2f90 0x55c313fe5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53990/0 (socket says 172.21.15.7:53990) 2024-06-10T01:15:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.529+0000 7f78e496a640 1 -- 172.21.15.7:0/3965523525 learned_addr learned my addr 172.21.15.7:0/3965523525 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.529+0000 7f78e516b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c313fd6ae0 0x55c313fed010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.529+0000 7f78d7fff640 1 --2- 172.21.15.7:0/3965523525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c313fd3980 0x55c313fe5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.529+0000 7f78e496a640 1 -- 172.21.15.7:0/3965523525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c313fd3980 msgr2=0x55c313fe5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.529+0000 7f78e496a640 1 --2- 172.21.15.7:0/3965523525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c313fd3980 0x55c313fe5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.529+0000 7f78e496a640 1 -- 172.21.15.7:0/3965523525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c313fd6ae0 msgr2=0x55c313fed010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.529+0000 7f78e496a640 1 --2- 172.21.15.7:0/3965523525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c313fd6ae0 0x55c313fed010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.530+0000 7f78e496a640 1 -- 172.21.15.7:0/3965523525 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c313c4a210 con 0x55c313fd2f90 2024-06-10T01:15:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.530+0000 7f78e516b640 1 --2- 172.21.15.7:0/3965523525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c313fd6ae0 0x55c313fed010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.530+0000 7f78e496a640 1 --2- 172.21.15.7:0/3965523525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c313fd2f90 0x55c313fe5a90 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f78dc000c00 tx=0x7f78dc00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad0eafaa8a5ce7f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.530+0000 7f78bffff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:15:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.530+0000 7f78bffff640 1 -- 172.21.15.7:0/3965523525 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc011de0 con 0x55c313fd2f90 2024-06-10T01:15:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.530+0000 7f78bffff640 1 -- 172.21.15.7:0/3965523525 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f78dc01c480 con 0x55c313fd2f90 2024-06-10T01:15:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.530+0000 7f78e99bb740 1 -- 172.21.15.7:0/3965523525 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78dc01cb00 con 0x55c313fd2f90 2024-06-10T01:15:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.530+0000 7f78bffff640 1 -- 172.21.15.7:0/3965523525 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f78dc033050 con 0x55c313fd2f90 2024-06-10T01:15:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.530+0000 7f78e99bb740 10 client.17985 fetch_fsmap learned FSMap version 37 2024-06-10T01:15:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.530+0000 7f78e99bb740 1 -- 172.21.15.7:0/3965523525 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c313d16620 con 0x55c313fd2f90 2024-06-10T01:15:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.530+0000 7f78bffff640 1 -- 172.21.15.7:0/3965523525 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78dc01cb00 con 0x55c313fd2f90 2024-06-10T01:15:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.530+0000 7f78bffff640 1 -- 172.21.15.7:0/3965523525 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f78dc0046c0 con 0x55c313fd2f90 2024-06-10T01:15:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.531+0000 7f78e99bb740 10 client.17985 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T01:15:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.531+0000 7f78e99bb740 10 client.17985 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:15:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.531+0000 7f78e99bb740 20 client.17985 populate_metadata read hostname 'smithi007' 2024-06-10T01:15:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.531+0000 7f78e99bb740 1 --2- 172.21.15.7:0/3965523525 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c313ff1600 0x55c313ff3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.531+0000 7f78e99bb740 4 client.17985 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:15:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.531+0000 7f78e99bb740 1 -- 172.21.15.7:0/3965523525 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c313f49f10 con 0x55c313ff1600 2024-06-10T01:15:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.531+0000 7f78d7fff640 1 --2- 172.21.15.7:0/3965523525 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c313ff1600 0x55c313ff3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.531+0000 7f78d7fff640 1 --2- 172.21.15.7:0/3965523525 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c313ff1600 0x55c313ff3aa0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=717dccbeae246739 server_cookie=390461d63955c94a in_seq=0 out_seq=0 2024-06-10T01:15:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.531+0000 7f78bffff640 10 client.17985 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.532+0000 7f78bffff640 1 -- 172.21.15.7:0/3965523525 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c313f49f10 con 0x55c313ff1600 2024-06-10T01:15:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.532+0000 7f78bffff640 10 client.17985 handle_command_reply: tid=0 2024-06-10T01:15:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.532+0000 7f78e99bb740 1 client.17985 shutdown 2024-06-10T01:15:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.533+0000 7f78e99bb740 1 -- 172.21.15.7:0/3965523525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c313fd2f90 msgr2=0x55c313fe5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.533+0000 7f78e99bb740 1 --2- 172.21.15.7:0/3965523525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c313fd2f90 0x55c313fe5a90 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f78dc000c00 tx=0x7f78dc00bfd0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.533+0000 7f78e99bb740 1 -- 172.21.15.7:0/3965523525 shutdown_connections 2024-06-10T01:15:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.533+0000 7f78e99bb740 1 --2- 172.21.15.7:0/3965523525 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c313ff1600 0x55c313ff3aa0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.533+0000 7f78e99bb740 1 --2- 172.21.15.7:0/3965523525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c313fd6ae0 0x55c313fed010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.533+0000 7f78e99bb740 1 --2- 172.21.15.7:0/3965523525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c313fd3980 0x55c313fe5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.533+0000 7f78e99bb740 1 --2- 172.21.15.7:0/3965523525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c313fd2f90 0x55c313fe5a90 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.533+0000 7f78bffff640 0 client.17985 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.533+0000 7f78e99bb740 1 -- 172.21.15.7:0/3965523525 >> 172.21.15.7:0/3965523525 conn(0x55c313fce680 msgr2=0x55c313fcd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.533+0000 7f78e99bb740 1 -- 172.21.15.7:0/3965523525 shutdown_connections 2024-06-10T01:15:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.533+0000 7f78e99bb740 1 -- 172.21.15.7:0/3965523525 wait complete. 2024-06-10T01:15:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.533+0000 7f78e99bb740 20 client.17985 trim_cache size 0 max 16384 2024-06-10T01:15:45.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.548+0000 7f78e99bb740 1 Processor -- start 2024-06-10T01:15:45.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.548+0000 7f78e99bb740 1 -- start start 2024-06-10T01:15:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.549+0000 7f78e99bb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c313fd2f90 0x55c313fd6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.549+0000 7f78e99bb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c313fd3980 0x55c313fecfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.549+0000 7f78e99bb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c313fd6ae0 0x55c313ff3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.549+0000 7f78e99bb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f78e01aac70 con 0x55c313fd2f90 2024-06-10T01:15:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.549+0000 7f78e99bb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f78e01a84b0 con 0x55c313fd3980 2024-06-10T01:15:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.549+0000 7f78e99bb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f78e01a6300 con 0x55c313fd6ae0 2024-06-10T01:15:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.549+0000 7f78e496a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c313fd2f90 0x55c313fd6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.549+0000 7f78d7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c313fd3980 0x55c313fecfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.549+0000 7f78e496a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c313fd2f90 0x55c313fd6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53996/0 (socket says 172.21.15.7:53996) 2024-06-10T01:15:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.549+0000 7f78e496a640 1 -- 172.21.15.7:0/956766397 learned_addr learned my addr 172.21.15.7:0/956766397 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.549+0000 7f78e516b640 1 --2- 172.21.15.7:0/956766397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c313fd6ae0 0x55c313ff3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.549+0000 7f78e496a640 1 -- 172.21.15.7:0/956766397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c313fd6ae0 msgr2=0x55c313ff3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.549+0000 7f78e496a640 1 --2- 172.21.15.7:0/956766397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c313fd6ae0 0x55c313ff3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.549+0000 7f78e496a640 1 -- 172.21.15.7:0/956766397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c313fd3980 msgr2=0x55c313fecfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.549+0000 7f78e496a640 1 --2- 172.21.15.7:0/956766397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c313fd3980 0x55c313fecfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.549+0000 7f78e496a640 1 -- 172.21.15.7:0/956766397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c313fdfb60 con 0x55c313fd2f90 2024-06-10T01:15:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.549+0000 7f78d7fff640 1 --2- 172.21.15.7:0/956766397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c313fd3980 0x55c313fecfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.549+0000 7f78e516b640 1 --2- 172.21.15.7:0/956766397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c313fd6ae0 0x55c313ff3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:15:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.549+0000 7f78e496a640 1 --2- 172.21.15.7:0/956766397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c313fd2f90 0x55c313fd6220 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f78dc005940 tx=0x7f78dc00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3f033092db19f2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.549+0000 7f78c67fc640 1 -- 172.21.15.7:0/956766397 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc03b020 con 0x55c313fd2f90 2024-06-10T01:15:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.549+0000 7f78c67fc640 1 -- 172.21.15.7:0/956766397 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78dc02a070 con 0x55c313fd2f90 2024-06-10T01:15:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.549+0000 7f78c67fc640 1 -- 172.21.15.7:0/956766397 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc0039f0 con 0x55c313fd2f90 2024-06-10T01:15:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.550+0000 7f78e99bb740 1 -- 172.21.15.7:0/956766397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c313fd2f90 msgr2=0x55c313fd6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.550+0000 7f78e99bb740 1 --2- 172.21.15.7:0/956766397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c313fd2f90 0x55c313fd6220 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f78dc005940 tx=0x7f78dc00b390 comp rx=0 tx=0).stop 2024-06-10T01:15:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.550+0000 7f78e99bb740 1 -- 172.21.15.7:0/956766397 shutdown_connections 2024-06-10T01:15:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.550+0000 7f78e99bb740 1 --2- 172.21.15.7:0/956766397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c313fd6ae0 0x55c313ff3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.550+0000 7f78e99bb740 1 --2- 172.21.15.7:0/956766397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c313fd3980 0x55c313fecfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.550+0000 7f78e99bb740 1 --2- 172.21.15.7:0/956766397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c313fd2f90 0x55c313fd6220 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.550+0000 7f78e99bb740 1 -- 172.21.15.7:0/956766397 >> 172.21.15.7:0/956766397 conn(0x55c313fce680 msgr2=0x55c313fe3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.550+0000 7f78e99bb740 1 -- 172.21.15.7:0/956766397 shutdown_connections 2024-06-10T01:15:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.550+0000 7f78e99bb740 1 -- 172.21.15.7:0/956766397 wait complete. 2024-06-10T01:15:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.550+0000 7f78e99bb740 1 -- start start 2024-06-10T01:15:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.551+0000 7f78e99bb740 1 Processor -- start 2024-06-10T01:15:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.551+0000 7f78e99bb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c313fd2f90 0x55c313fe9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.551+0000 7f78e99bb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c313fd3980 0x55c313fe9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.551+0000 7f78e99bb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c313fd6ae0 0x55c313fec7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.551+0000 7f78e99bb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f78e01a6970 con 0x55c313fd3980 2024-06-10T01:15:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.551+0000 7f78e99bb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f78e01a7de0 con 0x55c313fd6ae0 2024-06-10T01:15:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.551+0000 7f78e99bb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f78e01a7040 con 0x55c313fd2f90 2024-06-10T01:15:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.551+0000 7f78e99bb740 10 client.0 _ll_register_callbacks cb 0x7f78d0009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:15:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.551+0000 7f78d7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c313fd3980 0x55c313fe9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.551+0000 7f78d7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c313fd3980 0x55c313fe9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54002/0 (socket says 172.21.15.7:54002) 2024-06-10T01:15:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.551+0000 7f78d7fff640 1 -- 172.21.15.7:0/2393398130 learned_addr learned my addr 172.21.15.7:0/2393398130 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.551+0000 7f78e516b640 1 --2- 172.21.15.7:0/2393398130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c313fd6ae0 0x55c313fec7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.551+0000 7f78e496a640 1 --2- 172.21.15.7:0/2393398130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c313fd2f90 0x55c313fe9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.551+0000 7f78d7fff640 1 -- 172.21.15.7:0/2393398130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c313fd2f90 msgr2=0x55c313fe9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.551+0000 7f78d7fff640 1 --2- 172.21.15.7:0/2393398130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c313fd2f90 0x55c313fe9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.551+0000 7f78d7fff640 1 -- 172.21.15.7:0/2393398130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c313fd6ae0 msgr2=0x55c313fec7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.551+0000 7f78d7fff640 1 --2- 172.21.15.7:0/2393398130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c313fd6ae0 0x55c313fec7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.551+0000 7f78d7fff640 1 -- 172.21.15.7:0/2393398130 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c313fecda0 con 0x55c313fd3980 2024-06-10T01:15:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.551+0000 7f78e516b640 1 --2- 172.21.15.7:0/2393398130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c313fd6ae0 0x55c313fec7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.552+0000 7f78d7fff640 1 --2- 172.21.15.7:0/2393398130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c313fd3980 0x55c313fe9790 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009fd0 tx=0x7f78d0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4acf29cd22474f97 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.552+0000 7f78bffff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:15:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.552+0000 7f78bffff640 1 -- 172.21.15.7:0/2393398130 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d0030020 con 0x55c313fd3980 2024-06-10T01:15:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.552+0000 7f78bffff640 1 -- 172.21.15.7:0/2393398130 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f78d001a070 con 0x55c313fd3980 2024-06-10T01:15:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.552+0000 7f78e99bb740 1 -- 172.21.15.7:0/2393398130 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78dc02a070 con 0x55c313fd3980 2024-06-10T01:15:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.552+0000 7f78bffff640 1 -- 172.21.15.7:0/2393398130 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f78d003b050 con 0x55c313fd3980 2024-06-10T01:15:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.552+0000 7f78e99bb740 10 client.17991 fetch_fsmap learned FSMap version 37 2024-06-10T01:15:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.552+0000 7f78e99bb740 1 -- 172.21.15.7:0/2393398130 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c31405afb0 con 0x55c313fd3980 2024-06-10T01:15:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.552+0000 7f78bffff640 1 -- 172.21.15.7:0/2393398130 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78dc02a070 con 0x55c313fd3980 2024-06-10T01:15:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.552+0000 7f78bffff640 1 -- 172.21.15.7:0/2393398130 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f78d001d070 con 0x55c313fd3980 2024-06-10T01:15:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.552+0000 7f78e99bb740 10 client.17991 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T01:15:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.552+0000 7f78e99bb740 10 client.17991 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:15:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.552+0000 7f78e99bb740 20 client.17991 populate_metadata read hostname 'smithi007' 2024-06-10T01:15:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.552+0000 7f78e99bb740 1 --2- 172.21.15.7:0/2393398130 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c31405ba90 0x55c31405deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.552+0000 7f78e99bb740 4 client.17991 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:15:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.552+0000 7f78e99bb740 1 -- 172.21.15.7:0/2393398130 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c313f43c20 con 0x55c31405ba90 2024-06-10T01:15:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.553+0000 7f78e496a640 1 --2- 172.21.15.7:0/2393398130 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c31405ba90 0x55c31405deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.553+0000 7f78bffff640 10 client.17991 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.553+0000 7f78e496a640 1 --2- 172.21.15.7:0/2393398130 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c31405ba90 0x55c31405deb0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d109157b92a1cf1b server_cookie=eccab54e2d631792 in_seq=0 out_seq=0 2024-06-10T01:15:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.643+0000 7f78bffff640 1 -- 172.21.15.7:0/2393398130 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c313f43c20 con 0x55c31405ba90 2024-06-10T01:15:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.643+0000 7f78bffff640 10 client.17991 handle_command_reply: tid=0 2024-06-10T01:15:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.643+0000 7f78e99bb740 1 client.17991 shutdown 2024-06-10T01:15:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.643+0000 7f78e99bb740 1 -- 172.21.15.7:0/2393398130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c313fd3980 msgr2=0x55c313fe9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.643+0000 7f78e99bb740 1 --2- 172.21.15.7:0/2393398130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c313fd3980 0x55c313fe9790 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009fd0 tx=0x7f78d0005f90 comp rx=0 tx=0).stop 2024-06-10T01:15:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.643+0000 7f78e99bb740 1 -- 172.21.15.7:0/2393398130 shutdown_connections 2024-06-10T01:15:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.643+0000 7f78e99bb740 1 --2- 172.21.15.7:0/2393398130 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c31405ba90 0x55c31405deb0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.643+0000 7f78e99bb740 1 --2- 172.21.15.7:0/2393398130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c313fd6ae0 0x55c313fec7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.643+0000 7f78e99bb740 1 --2- 172.21.15.7:0/2393398130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c313fd3980 0x55c313fe9790 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.643+0000 7f78bffff640 0 client.17991 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.643+0000 7f78e99bb740 1 --2- 172.21.15.7:0/2393398130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c313fd2f90 0x55c313fe9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.643+0000 7f78e99bb740 1 -- 172.21.15.7:0/2393398130 >> 172.21.15.7:0/2393398130 conn(0x55c313fce680 msgr2=0x55c313fccf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.643+0000 7f78e99bb740 1 -- 172.21.15.7:0/2393398130 shutdown_connections 2024-06-10T01:15:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.643+0000 7f78e99bb740 1 -- 172.21.15.7:0/2393398130 wait complete. 2024-06-10T01:15:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.643+0000 7f78e99bb740 20 client.17991 trim_cache size 0 max 16384 2024-06-10T01:15:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:15:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:15:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:15:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.644+0000 7f78e6bce640 1 -- 172.21.15.7:0/4147000100 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f78c80a2b80 msgr2=0x7f78c80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.644+0000 7f78e6bce640 1 --2- 172.21.15.7:0/4147000100 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f78c80a2b80 0x7f78c80a5040 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55c313fd41e0 tx=0x7f78d0004730 comp rx=0 tx=0).stop 2024-06-10T01:15:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.644+0000 7f78e6bce640 1 -- 172.21.15.7:0/4147000100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78e0107780 msgr2=0x7f78e019f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.644+0000 7f78e6bce640 1 --2- 172.21.15.7:0/4147000100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78e0107780 0x7f78e019f270 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f78d8031ae0 tx=0x7f78d8031ee0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.644+0000 7f78e6bce640 1 -- 172.21.15.7:0/4147000100 shutdown_connections 2024-06-10T01:15:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.644+0000 7f78e6bce640 1 --2- 172.21.15.7:0/4147000100 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f78c80a2b80 0x7f78c80a5040 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.644+0000 7f78e6bce640 1 --2- 172.21.15.7:0/4147000100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78e0107780 0x7f78e019f270 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.644+0000 7f78e6bce640 1 --2- 172.21.15.7:0/4147000100 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78e0104e50 0x7f78e0198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.644+0000 7f78e6bce640 1 --2- 172.21.15.7:0/4147000100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78e0102520 0x7f78e0197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.644+0000 7f78e6bce640 1 -- 172.21.15.7:0/4147000100 >> 172.21.15.7:0/4147000100 conn(0x7f78e00fbeb0 msgr2=0x7f78e0102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.644+0000 7f78e6bce640 1 -- 172.21.15.7:0/4147000100 shutdown_connections 2024-06-10T01:15:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:45.644+0000 7f78e6bce640 1 -- 172.21.15.7:0/4147000100 wait complete. 2024-06-10T01:15:45.746 INFO:tasks.workunit.client.0.smithi007.stdout: 1 685740 7.19 MB/sec execute 386 sec latency 50.295 ms 2024-06-10T01:15:45.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:45 smithi007 ceph-mon[32398]: pgmap v724: 129 pgs: 129 active+clean; 111 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 210 KiB/s wr, 48 op/s 2024-06-10T01:15:45.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:45 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:15:45.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:45 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:15:45.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:45 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:15:45.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:45 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:15:45.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:45 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:15:45.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:45 smithi078 ceph-mon[36683]: pgmap v724: 129 pgs: 129 active+clean; 111 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 210 KiB/s wr, 48 op/s 2024-06-10T01:15:45.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:45 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:15:45.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:45 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:15:45.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:45 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:15:45.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:45 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:15:45.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:45 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:15:45.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:45 smithi175 ceph-mon[36636]: pgmap v724: 129 pgs: 129 active+clean; 111 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 210 KiB/s wr, 48 op/s 2024-06-10T01:15:45.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:45 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:15:45.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:45 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:15:45.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:45 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:15:45.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:45 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:15:45.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:45 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:15:46.746 INFO:tasks.workunit.client.0.smithi007.stdout: 1 687649 7.18 MB/sec execute 387 sec latency 35.087 ms 2024-06-10T01:15:46.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:46 smithi078 ceph-mon[36683]: pgmap v725: 129 pgs: 129 active+clean; 112 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 283 KiB/s wr, 72 op/s 2024-06-10T01:15:46.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:46 smithi175 ceph-mon[36636]: pgmap v725: 129 pgs: 129 active+clean; 112 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 283 KiB/s wr, 72 op/s 2024-06-10T01:15:47.012 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:46 smithi007 ceph-mon[32398]: pgmap v725: 129 pgs: 129 active+clean; 112 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 283 KiB/s wr, 72 op/s 2024-06-10T01:15:47.746 INFO:tasks.workunit.client.0.smithi007.stdout: 1 689515 7.20 MB/sec execute 388 sec latency 9.475 ms 2024-06-10T01:15:47.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:47 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:15:47.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:47 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:15:47.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:47 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:15:47.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:47 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:15:48.012 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:47 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:15:48.012 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:47 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:15:48.746 INFO:tasks.workunit.client.0.smithi007.stdout: 1 691717 7.21 MB/sec execute 389 sec latency 34.842 ms 2024-06-10T01:15:48.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:48 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:15:48.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:48 smithi078 ceph-mon[36683]: pgmap v726: 129 pgs: 129 active+clean; 113 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 392 KiB/s wr, 102 op/s 2024-06-10T01:15:48.953 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:48 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:15:48.953 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:48 smithi175 ceph-mon[36636]: pgmap v726: 129 pgs: 129 active+clean; 113 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 392 KiB/s wr, 102 op/s 2024-06-10T01:15:49.012 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:48 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:15:49.012 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:48 smithi007 ceph-mon[32398]: pgmap v726: 129 pgs: 129 active+clean; 113 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 392 KiB/s wr, 102 op/s 2024-06-10T01:15:49.746 INFO:tasks.workunit.client.0.smithi007.stdout: 1 693180 7.21 MB/sec execute 390 sec latency 24.241 ms 2024-06-10T01:15:50.746 INFO:tasks.workunit.client.0.smithi007.stdout: 1 695420 7.23 MB/sec execute 391 sec latency 33.026 ms 2024-06-10T01:15:51.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:50 smithi078 ceph-mon[36683]: pgmap v727: 129 pgs: 129 active+clean; 112 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 932 KiB/s wr, 159 op/s 2024-06-10T01:15:51.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:50 smithi175 ceph-mon[36636]: pgmap v727: 129 pgs: 129 active+clean; 112 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 932 KiB/s wr, 159 op/s 2024-06-10T01:15:51.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:50 smithi007 ceph-mon[32398]: pgmap v727: 129 pgs: 129 active+clean; 112 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 932 KiB/s wr, 159 op/s 2024-06-10T01:15:51.746 INFO:tasks.workunit.client.0.smithi007.stdout: 1 696953 7.23 MB/sec execute 392 sec latency 9.995 ms 2024-06-10T01:15:52.746 INFO:tasks.workunit.client.0.smithi007.stdout: 1 699414 7.24 MB/sec execute 393 sec latency 21.967 ms 2024-06-10T01:15:53.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:52 smithi078 ceph-mon[36683]: pgmap v728: 129 pgs: 129 active+clean; 115 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 1.3 MiB/s wr, 235 op/s 2024-06-10T01:15:53.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:52 smithi175 ceph-mon[36636]: pgmap v728: 129 pgs: 129 active+clean; 115 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 1.3 MiB/s wr, 235 op/s 2024-06-10T01:15:53.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:52 smithi007 ceph-mon[32398]: pgmap v728: 129 pgs: 129 active+clean; 115 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 1.3 MiB/s wr, 235 op/s 2024-06-10T01:15:53.746 INFO:tasks.workunit.client.0.smithi007.stdout: 1 700762 7.24 MB/sec execute 394 sec latency 16.479 ms 2024-06-10T01:15:54.746 INFO:tasks.workunit.client.0.smithi007.stdout: 1 703216 7.26 MB/sec execute 395 sec latency 23.499 ms 2024-06-10T01:15:55.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:54 smithi078 ceph-mon[36683]: pgmap v729: 129 pgs: 129 active+clean; 116 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 1.7 MiB/s wr, 249 op/s 2024-06-10T01:15:55.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:54 smithi175 ceph-mon[36636]: pgmap v729: 129 pgs: 129 active+clean; 116 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 1.7 MiB/s wr, 249 op/s 2024-06-10T01:15:55.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:54 smithi007 ceph-mon[32398]: pgmap v729: 129 pgs: 129 active+clean; 116 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 1.7 MiB/s wr, 249 op/s 2024-06-10T01:15:55.746 INFO:tasks.workunit.client.0.smithi007.stdout: 1 704603 7.26 MB/sec execute 396 sec latency 15.703 ms 2024-06-10T01:15:56.746 INFO:tasks.workunit.client.0.smithi007.stdout: 1 706995 7.28 MB/sec execute 397 sec latency 10.989 ms 2024-06-10T01:15:57.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:56 smithi175 ceph-mon[36636]: pgmap v730: 129 pgs: 129 active+clean; 116 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.9 MiB/s wr, 290 op/s 2024-06-10T01:15:57.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:56 smithi007 ceph-mon[32398]: pgmap v730: 129 pgs: 129 active+clean; 116 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.9 MiB/s wr, 290 op/s 2024-06-10T01:15:57.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:56 smithi078 ceph-mon[36683]: pgmap v730: 129 pgs: 129 active+clean; 116 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.9 MiB/s wr, 290 op/s 2024-06-10T01:15:57.746 INFO:tasks.workunit.client.0.smithi007.stdout: 1 709070 7.28 MB/sec execute 398 sec latency 28.458 ms 2024-06-10T01:15:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.736+0000 7f622c3a2640 1 -- 172.21.15.7:0/2460880036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62241012a0 msgr2=0x7f6224101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.736+0000 7f622c3a2640 1 --2- 172.21.15.7:0/2460880036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62241012a0 0x7f6224101720 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f6214009900 tx=0x7f621402a410 comp rx=0 tx=0).stop 2024-06-10T01:15:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.736+0000 7f622c3a2640 1 -- 172.21.15.7:0/2460880036 shutdown_connections 2024-06-10T01:15:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.736+0000 7f622c3a2640 1 --2- 172.21.15.7:0/2460880036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6224101c60 0x7f62241084f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.736+0000 7f622c3a2640 1 --2- 172.21.15.7:0/2460880036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62241012a0 0x7f6224101720 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.736+0000 7f622c3a2640 1 --2- 172.21.15.7:0/2460880036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62241000b0 0x7f62241004d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.736+0000 7f622c3a2640 1 -- 172.21.15.7:0/2460880036 >> 172.21.15.7:0/2460880036 conn(0x7f62240fb600 msgr2=0x7f62240fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.736+0000 7f622c3a2640 1 -- 172.21.15.7:0/2460880036 shutdown_connections 2024-06-10T01:15:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.736+0000 7f622c3a2640 1 -- 172.21.15.7:0/2460880036 wait complete. 2024-06-10T01:15:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.737+0000 7f622c3a2640 1 Processor -- start 2024-06-10T01:15:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.737+0000 7f622c3a2640 1 -- start start 2024-06-10T01:15:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.737+0000 7f622c3a2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62241000b0 0x7f622410b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.737+0000 7f622c3a2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62241012a0 0x7f6224109d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.737+0000 7f622c3a2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6224101c60 0x7f622410a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.737+0000 7f622c3a2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f622410d6c0 con 0x7f6224101c60 2024-06-10T01:15:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.737+0000 7f622c3a2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f622410d540 con 0x7f62241012a0 2024-06-10T01:15:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.737+0000 7f622c3a2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f622410d840 con 0x7f62241000b0 2024-06-10T01:15:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.737+0000 7f622a93f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6224101c60 0x7f622410a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.737+0000 7f622a13e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62241000b0 0x7f622410b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.738+0000 7f622993d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62241012a0 0x7f6224109d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.738+0000 7f622a93f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6224101c60 0x7f622410a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40290/0 (socket says 172.21.15.7:40290) 2024-06-10T01:15:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.738+0000 7f622a93f640 1 -- 172.21.15.7:0/2546649577 learned_addr learned my addr 172.21.15.7:0/2546649577 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.738+0000 7f622a13e640 1 -- 172.21.15.7:0/2546649577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62241012a0 msgr2=0x7f6224109d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.738+0000 7f622a13e640 1 --2- 172.21.15.7:0/2546649577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62241012a0 0x7f6224109d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.738+0000 7f622a13e640 1 -- 172.21.15.7:0/2546649577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6224101c60 msgr2=0x7f622410a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.738+0000 7f622a13e640 1 --2- 172.21.15.7:0/2546649577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6224101c60 0x7f622410a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.738+0000 7f622a13e640 1 -- 172.21.15.7:0/2546649577 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f622410aad0 con 0x7f62241000b0 2024-06-10T01:15:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.738+0000 7f622a93f640 1 --2- 172.21.15.7:0/2546649577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6224101c60 0x7f622410a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.738+0000 7f622a13e640 1 --2- 172.21.15.7:0/2546649577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62241000b0 0x7f622410b6c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f622000e9c0 tx=0x7f622000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.738+0000 7f621b7fe640 1 -- 172.21.15.7:0/2546649577 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622000ccc0 con 0x7f62241000b0 2024-06-10T01:15:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.738+0000 7f622c3a2640 1 -- 172.21.15.7:0/2546649577 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62241a1e60 con 0x7f62241000b0 2024-06-10T01:15:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.738+0000 7f621b7fe640 1 -- 172.21.15.7:0/2546649577 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62200035f0 con 0x7f62241000b0 2024-06-10T01:15:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.738+0000 7f621b7fe640 1 -- 172.21.15.7:0/2546649577 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622000ce60 con 0x7f62241000b0 2024-06-10T01:15:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.738+0000 7f622c3a2640 1 -- 172.21.15.7:0/2546649577 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62241a2260 con 0x7f62241000b0 2024-06-10T01:15:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.741+0000 7f622f194740 1 Processor -- start 2024-06-10T01:15:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.741+0000 7f621b7fe640 1 -- 172.21.15.7:0/2546649577 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f62200037a0 con 0x7f62241000b0 2024-06-10T01:15:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.741+0000 7f622f194740 1 -- start start 2024-06-10T01:15:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.741+0000 7f622f194740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c6f4e010 0x55c7c6f4e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.741+0000 7f622f194740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7c6f4ea00 0x55c7c6f53500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.741+0000 7f622f194740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7c6f53b70 0x55c7c6f56000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.741+0000 7f622f194740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c7c6eb14f0 con 0x55c7c6f4e010 2024-06-10T01:15:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.741+0000 7f622f194740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c7c6e4f870 con 0x55c7c6f4ea00 2024-06-10T01:15:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.741+0000 7f622f194740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c7c6ead4b0 con 0x55c7c6f53b70 2024-06-10T01:15:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.742+0000 7f622a93f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c6f4e010 0x55c7c6f4e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.742+0000 7f622a93f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c6f4e010 0x55c7c6f4e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40302/0 (socket says 172.21.15.7:40302) 2024-06-10T01:15:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.742+0000 7f622a93f640 1 -- 172.21.15.7:0/2080758588 learned_addr learned my addr 172.21.15.7:0/2080758588 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.742+0000 7f622993d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7c6f53b70 0x55c7c6f56000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.742+0000 7f622a13e640 1 --2- 172.21.15.7:0/2080758588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7c6f4ea00 0x55c7c6f53500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.742+0000 7f622a93f640 1 -- 172.21.15.7:0/2080758588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7c6f53b70 msgr2=0x55c7c6f56000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.742+0000 7f622a93f640 1 --2- 172.21.15.7:0/2080758588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7c6f53b70 0x55c7c6f56000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.742+0000 7f622a93f640 1 -- 172.21.15.7:0/2080758588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7c6f4ea00 msgr2=0x55c7c6f53500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.742+0000 7f622a93f640 1 --2- 172.21.15.7:0/2080758588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7c6f4ea00 0x55c7c6f53500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.742+0000 7f622a93f640 1 -- 172.21.15.7:0/2080758588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7c6e6b9b0 con 0x55c7c6f4e010 2024-06-10T01:15:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.742+0000 7f621b7fe640 1 --2- 172.21.15.7:0/2546649577 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f62100a2b80 0x7f62100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.742+0000 7f622a13e640 1 --2- 172.21.15.7:0/2546649577 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f62100a2b80 0x7f62100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.742+0000 7f622a93f640 1 --2- 172.21.15.7:0/2080758588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c6f4e010 0x55c7c6f4e430 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f621c010890 tx=0x7f621c010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388a5fd9270cbeab server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.742+0000 7f621b7fe640 1 -- 172.21.15.7:0/2546649577 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f62200cf900 con 0x7f62241000b0 2024-06-10T01:15:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.742+0000 7f6218bfe640 1 -- 172.21.15.7:0/2080758588 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c00ebf0 con 0x55c7c6f4e010 2024-06-10T01:15:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.743+0000 7f6218bfe640 1 -- 172.21.15.7:0/2080758588 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f621c015070 con 0x55c7c6f4e010 2024-06-10T01:15:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.743+0000 7f6218bfe640 1 -- 172.21.15.7:0/2080758588 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c00f620 con 0x55c7c6f4e010 2024-06-10T01:15:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.743+0000 7f622f194740 1 -- 172.21.15.7:0/2080758588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c6f4e010 msgr2=0x55c7c6f4e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.743+0000 7f622f194740 1 --2- 172.21.15.7:0/2080758588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c6f4e010 0x55c7c6f4e430 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f621c010890 tx=0x7f621c010d50 comp rx=0 tx=0).stop 2024-06-10T01:15:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.743+0000 7f622a13e640 1 --2- 172.21.15.7:0/2546649577 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f62100a2b80 0x7f62100a5040 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6220098750 tx=0x7f62200d5530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.743+0000 7f622f194740 1 -- 172.21.15.7:0/2080758588 shutdown_connections 2024-06-10T01:15:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.743+0000 7f622f194740 1 --2- 172.21.15.7:0/2080758588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7c6f53b70 0x55c7c6f56000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.743+0000 7f622f194740 1 --2- 172.21.15.7:0/2080758588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7c6f4ea00 0x55c7c6f53500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.743+0000 7f622f194740 1 --2- 172.21.15.7:0/2080758588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c6f4e010 0x55c7c6f4e430 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.743+0000 7f622f194740 1 -- 172.21.15.7:0/2080758588 >> 172.21.15.7:0/2080758588 conn(0x55c7c6f49680 msgr2=0x55c7c6f49a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.743+0000 7f622f194740 1 -- 172.21.15.7:0/2080758588 shutdown_connections 2024-06-10T01:15:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.743+0000 7f622f194740 1 -- 172.21.15.7:0/2080758588 wait complete. 2024-06-10T01:15:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.744+0000 7f622f194740 1 -- start start 2024-06-10T01:15:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.744+0000 7f622f194740 1 Processor -- start 2024-06-10T01:15:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.744+0000 7f622f194740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7c6f4e010 0x55c7c6f58250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.744+0000 7f622f194740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c6f4ea00 0x55c7c6f58790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.744+0000 7f622f194740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7c6f53b70 0x55c7c6f5f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.744+0000 7f622f194740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c7c6f56b20 con 0x55c7c6f4ea00 2024-06-10T01:15:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.744+0000 7f622f194740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c7c6e3aae0 con 0x55c7c6f53b70 2024-06-10T01:15:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.744+0000 7f622f194740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c7c6f56ca0 con 0x55c7c6f4e010 2024-06-10T01:15:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.744+0000 7f622f194740 10 client.0 _ll_register_callbacks cb 0x55c7c6e739b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:15:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.744+0000 7f622993d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c6f4ea00 0x55c7c6f58790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.744+0000 7f622993d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c6f4ea00 0x55c7c6f58790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40318/0 (socket says 172.21.15.7:40318) 2024-06-10T01:15:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.744+0000 7f622993d640 1 -- 172.21.15.7:0/4260959438 learned_addr learned my addr 172.21.15.7:0/4260959438 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.744+0000 7f622a93f640 1 --2- 172.21.15.7:0/4260959438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7c6f4e010 0x55c7c6f58250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.744+0000 7f622a93f640 1 --2- 172.21.15.7:0/4260959438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7c6f53b70 0x55c7c6f5f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.745+0000 7f622993d640 1 -- 172.21.15.7:0/4260959438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7c6f4e010 msgr2=0x55c7c6f58250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.745+0000 7f622993d640 1 --2- 172.21.15.7:0/4260959438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7c6f4e010 0x55c7c6f58250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.745+0000 7f622993d640 1 -- 172.21.15.7:0/4260959438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7c6f53b70 msgr2=0x55c7c6f5f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.745+0000 7f622993d640 1 --2- 172.21.15.7:0/4260959438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7c6f53b70 0x55c7c6f5f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.745+0000 7f622993d640 1 -- 172.21.15.7:0/4260959438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7c6bc5210 con 0x55c7c6f4ea00 2024-06-10T01:15:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.745+0000 7f622a93f640 1 --2- 172.21.15.7:0/4260959438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7c6f53b70 0x55c7c6f5f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:15:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.745+0000 7f622993d640 1 --2- 172.21.15.7:0/4260959438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c6f4ea00 0x55c7c6f58790 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6214009900 tx=0x7f62140318e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4028b7ad61c3e76a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.745+0000 7f620dffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:15:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.745+0000 7f620dffb640 1 -- 172.21.15.7:0/4260959438 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6214035040 con 0x55c7c6f4ea00 2024-06-10T01:15:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.745+0000 7f620dffb640 1 -- 172.21.15.7:0/4260959438 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f6214004040 con 0x55c7c6f4ea00 2024-06-10T01:15:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.745+0000 7f622f194740 1 -- 172.21.15.7:0/4260959438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f621c015070 con 0x55c7c6f4ea00 2024-06-10T01:15:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.745+0000 7f620dffb640 1 -- 172.21.15.7:0/4260959438 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f621404f050 con 0x55c7c6f4ea00 2024-06-10T01:15:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.745+0000 7f622f194740 10 client.18003 fetch_fsmap learned FSMap version 37 2024-06-10T01:15:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.745+0000 7f622f194740 1 -- 172.21.15.7:0/4260959438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c7c6c91620 con 0x55c7c6f4ea00 2024-06-10T01:15:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.745+0000 7f620dffb640 1 -- 172.21.15.7:0/4260959438 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f621c015070 con 0x55c7c6f4ea00 2024-06-10T01:15:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.745+0000 7f620dffb640 1 -- 172.21.15.7:0/4260959438 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f6214004860 con 0x55c7c6f4ea00 2024-06-10T01:15:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.745+0000 7f622f194740 10 client.18003 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T01:15:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.745+0000 7f622f194740 10 client.18003 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:15:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.746+0000 7f622f194740 20 client.18003 populate_metadata read hostname 'smithi007' 2024-06-10T01:15:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.746+0000 7f622f194740 1 --2- 172.21.15.7:0/4260959438 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c7c6f642e0 0x55c7c6f666d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.746+0000 7f622f194740 4 client.18003 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:15:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.746+0000 7f622f194740 1 -- 172.21.15.7:0/4260959438 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7c6ec4f10 con 0x55c7c6f642e0 2024-06-10T01:15:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.746+0000 7f622a93f640 1 --2- 172.21.15.7:0/4260959438 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c7c6f642e0 0x55c7c6f666d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.746+0000 7f622a93f640 1 --2- 172.21.15.7:0/4260959438 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c7c6f642e0 0x55c7c6f666d0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b33b3fc720c6098f server_cookie=b441f581dbb64a5 in_seq=0 out_seq=0 2024-06-10T01:15:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.746+0000 7f620dffb640 10 client.18003 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.747+0000 7f620dffb640 1 -- 172.21.15.7:0/4260959438 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c7c6ec4f10 con 0x55c7c6f642e0 2024-06-10T01:15:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.747+0000 7f620dffb640 10 client.18003 handle_command_reply: tid=0 2024-06-10T01:15:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.747+0000 7f622f194740 1 client.18003 shutdown 2024-06-10T01:15:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.747+0000 7f622f194740 1 -- 172.21.15.7:0/4260959438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c6f4ea00 msgr2=0x55c7c6f58790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.747+0000 7f622f194740 1 --2- 172.21.15.7:0/4260959438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c6f4ea00 0x55c7c6f58790 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6214009900 tx=0x7f62140318e0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.747+0000 7f622f194740 1 -- 172.21.15.7:0/4260959438 shutdown_connections 2024-06-10T01:15:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.747+0000 7f622f194740 1 --2- 172.21.15.7:0/4260959438 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c7c6f642e0 0x55c7c6f666d0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.747+0000 7f622f194740 1 --2- 172.21.15.7:0/4260959438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7c6f53b70 0x55c7c6f5f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.747+0000 7f622f194740 1 --2- 172.21.15.7:0/4260959438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c6f4ea00 0x55c7c6f58790 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.747+0000 7f620dffb640 0 client.18003 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.747+0000 7f622f194740 1 --2- 172.21.15.7:0/4260959438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7c6f4e010 0x55c7c6f58250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.747+0000 7f622f194740 1 -- 172.21.15.7:0/4260959438 >> 172.21.15.7:0/4260959438 conn(0x55c7c6f49680 msgr2=0x55c7c6f487e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.747+0000 7f622f194740 1 -- 172.21.15.7:0/4260959438 shutdown_connections 2024-06-10T01:15:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.747+0000 7f622f194740 1 -- 172.21.15.7:0/4260959438 wait complete. 2024-06-10T01:15:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.747+0000 7f622f194740 20 client.18003 trim_cache size 0 max 16384 2024-06-10T01:15:58.757 INFO:tasks.workunit.client.0.smithi007.stdout: 1 710900 7.29 MB/sec execute 399 sec latency 10.798 ms 2024-06-10T01:15:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.767+0000 7f622f194740 1 Processor -- start 2024-06-10T01:15:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.767+0000 7f622f194740 1 -- start start 2024-06-10T01:15:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.767+0000 7f622f194740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7c6f4e010 0x55c7c6f4fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.767+0000 7f622f194740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7c6f4ea00 0x55c7c6f50180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.767+0000 7f622f194740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c6f53b70 0x55c7c6f5f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.767+0000 7f622f194740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f62241af0b0 con 0x55c7c6f53b70 2024-06-10T01:15:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.767+0000 7f622f194740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f62241ac8f0 con 0x55c7c6f4e010 2024-06-10T01:15:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.767+0000 7f622f194740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f62241aa740 con 0x55c7c6f4ea00 2024-06-10T01:15:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.768+0000 7f622a93f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7c6f4e010 0x55c7c6f4fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.768+0000 7f622993d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7c6f4ea00 0x55c7c6f50180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.768+0000 7f622a93f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7c6f4e010 0x55c7c6f4fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:45696/0 (socket says 172.21.15.7:45696) 2024-06-10T01:15:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.768+0000 7f622a93f640 1 -- 172.21.15.7:0/4220052610 learned_addr learned my addr 172.21.15.7:0/4220052610 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.768+0000 7f622a93f640 1 --2- 172.21.15.7:0/4220052610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c6f53b70 0x55c7c6f5f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.768+0000 7f622a93f640 1 -- 172.21.15.7:0/4220052610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7c6f4ea00 msgr2=0x55c7c6f50180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.768+0000 7f622a93f640 1 --2- 172.21.15.7:0/4220052610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7c6f4ea00 0x55c7c6f50180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.768+0000 7f622a93f640 1 -- 172.21.15.7:0/4220052610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c6f53b70 msgr2=0x55c7c6f5f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.768+0000 7f622a93f640 1 --2- 172.21.15.7:0/4220052610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c6f53b70 0x55c7c6f5f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.768+0000 7f622a93f640 1 -- 172.21.15.7:0/4220052610 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7c6f494b0 con 0x55c7c6f4e010 2024-06-10T01:15:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.768+0000 7f622993d640 1 --2- 172.21.15.7:0/4220052610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7c6f4ea00 0x55c7c6f50180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.768+0000 7f622a93f640 1 --2- 172.21.15.7:0/4220052610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7c6f4e010 0x55c7c6f4fc40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f621c002230 tx=0x7f621c000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82a94aa2e6230c0a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.768+0000 7f620ffff640 1 -- 172.21.15.7:0/4220052610 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c023040 con 0x55c7c6f4e010 2024-06-10T01:15:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.769+0000 7f620ffff640 1 -- 172.21.15.7:0/4220052610 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f621c015030 con 0x55c7c6f4e010 2024-06-10T01:15:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.769+0000 7f620ffff640 1 -- 172.21.15.7:0/4220052610 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c02bb50 con 0x55c7c6f4e010 2024-06-10T01:15:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.769+0000 7f622f194740 1 -- 172.21.15.7:0/4220052610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7c6f4e010 msgr2=0x55c7c6f4fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.769+0000 7f622f194740 1 --2- 172.21.15.7:0/4220052610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7c6f4e010 0x55c7c6f4fc40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f621c002230 tx=0x7f621c000f40 comp rx=0 tx=0).stop 2024-06-10T01:15:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.769+0000 7f622f194740 1 -- 172.21.15.7:0/4220052610 shutdown_connections 2024-06-10T01:15:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.769+0000 7f622f194740 1 --2- 172.21.15.7:0/4220052610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c6f53b70 0x55c7c6f5f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.769+0000 7f622f194740 1 --2- 172.21.15.7:0/4220052610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7c6f4ea00 0x55c7c6f50180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.769+0000 7f622f194740 1 --2- 172.21.15.7:0/4220052610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7c6f4e010 0x55c7c6f4fc40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.769+0000 7f622f194740 1 -- 172.21.15.7:0/4220052610 >> 172.21.15.7:0/4220052610 conn(0x55c7c6f49680 msgr2=0x55c7c6f58250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.769+0000 7f622f194740 1 -- 172.21.15.7:0/4220052610 shutdown_connections 2024-06-10T01:15:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.769+0000 7f622f194740 1 -- 172.21.15.7:0/4220052610 wait complete. 2024-06-10T01:15:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.770+0000 7f622f194740 1 -- start start 2024-06-10T01:15:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.770+0000 7f622f194740 1 Processor -- start 2024-06-10T01:15:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.770+0000 7f622f194740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7c6f4e010 0x55c7c6f5c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.770+0000 7f622f194740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7c6f4ea00 0x55c7c6f5ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.770+0000 7f622f194740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c6f53b70 0x55c7c6fcce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.770+0000 7f622f194740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f62241aadb0 con 0x55c7c6f53b70 2024-06-10T01:15:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.770+0000 7f622f194740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f62241ac220 con 0x55c7c6f4ea00 2024-06-10T01:15:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.770+0000 7f622f194740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f62241ab480 con 0x55c7c6f4e010 2024-06-10T01:15:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.770+0000 7f622f194740 10 client.0 _ll_register_callbacks cb 0x7f6214009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:15:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.770+0000 7f622a93f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c6f53b70 0x55c7c6fcce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.770+0000 7f622993d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7c6f4ea00 0x55c7c6f5ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.770+0000 7f622a93f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c6f53b70 0x55c7c6fcce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40336/0 (socket says 172.21.15.7:40336) 2024-06-10T01:15:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.770+0000 7f622993d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7c6f4ea00 0x55c7c6f5ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:45712/0 (socket says 172.21.15.7:45712) 2024-06-10T01:15:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.770+0000 7f622a93f640 1 -- 172.21.15.7:0/1653140739 learned_addr learned my addr 172.21.15.7:0/1653140739 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:15:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.770+0000 7f622a93f640 1 --2- 172.21.15.7:0/1653140739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7c6f4e010 0x55c7c6f5c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.770+0000 7f622993d640 1 -- 172.21.15.7:0/1653140739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7c6f4e010 msgr2=0x55c7c6f5c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.770+0000 7f622993d640 1 --2- 172.21.15.7:0/1653140739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7c6f4e010 0x55c7c6f5c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.770+0000 7f622993d640 1 -- 172.21.15.7:0/1653140739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c6f53b70 msgr2=0x55c7c6fcce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:15:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.770+0000 7f622993d640 1 --2- 172.21.15.7:0/1653140739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c6f53b70 0x55c7c6fcce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.770+0000 7f622993d640 1 -- 172.21.15.7:0/1653140739 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7c6f5d070 con 0x55c7c6f4ea00 2024-06-10T01:15:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.770+0000 7f622a93f640 1 --2- 172.21.15.7:0/1653140739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c6f53b70 0x55c7c6fcce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:15:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.771+0000 7f622993d640 1 --2- 172.21.15.7:0/1653140739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7c6f4ea00 0x55c7c6f5ca00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6214004e20 tx=0x7f6214031950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5e1e1ae09cb043c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:15:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.771+0000 7f620dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:15:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.771+0000 7f620dffb640 1 -- 172.21.15.7:0/1653140739 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6214057020 con 0x55c7c6f4ea00 2024-06-10T01:15:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.771+0000 7f620dffb640 1 -- 172.21.15.7:0/1653140739 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f6214031b10 con 0x55c7c6f4ea00 2024-06-10T01:15:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.771+0000 7f622f194740 1 -- 172.21.15.7:0/1653140739 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f621c015030 con 0x55c7c6f4ea00 2024-06-10T01:15:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.771+0000 7f620dffb640 1 -- 172.21.15.7:0/1653140739 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6214062050 con 0x55c7c6f4ea00 2024-06-10T01:15:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.771+0000 7f622f194740 10 client.27365 fetch_fsmap learned FSMap version 37 2024-06-10T01:15:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.771+0000 7f622f194740 1 -- 172.21.15.7:0/1653140739 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c7c6fce450 con 0x55c7c6f4ea00 2024-06-10T01:15:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.772+0000 7f620dffb640 1 -- 172.21.15.7:0/1653140739 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f621c015030 con 0x55c7c6f4ea00 2024-06-10T01:15:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.772+0000 7f620dffb640 1 -- 172.21.15.7:0/1653140739 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f6214046470 con 0x55c7c6f4ea00 2024-06-10T01:15:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.772+0000 7f622f194740 10 client.27365 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T01:15:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.772+0000 7f622f194740 10 client.27365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:15:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.772+0000 7f622f194740 20 client.27365 populate_metadata read hostname 'smithi007' 2024-06-10T01:15:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.772+0000 7f622f194740 1 --2- 172.21.15.7:0/1653140739 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c7c6fceee0 0x55c7c6fd1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:15:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.772+0000 7f622f194740 4 client.27365 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:15:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.772+0000 7f622f194740 1 -- 172.21.15.7:0/1653140739 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c7c6ebec20 con 0x55c7c6fceee0 2024-06-10T01:15:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.772+0000 7f622a93f640 1 --2- 172.21.15.7:0/1653140739 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c7c6fceee0 0x55c7c6fd1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:15:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.772+0000 7f622a93f640 1 --2- 172.21.15.7:0/1653140739 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c7c6fceee0 0x55c7c6fd1300 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8af623969f5ea716 server_cookie=3294ca20c27207b9 in_seq=0 out_seq=0 2024-06-10T01:15:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.772+0000 7f620dffb640 10 client.27365 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:58.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.862+0000 7f620dffb640 1 -- 172.21.15.7:0/1653140739 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c7c6ebec20 con 0x55c7c6fceee0 2024-06-10T01:15:58.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.862+0000 7f620dffb640 10 client.27365 handle_command_reply: tid=0 2024-06-10T01:15:58.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.862+0000 7f622f194740 1 client.27365 shutdown 2024-06-10T01:15:58.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.862+0000 7f622f194740 1 -- 172.21.15.7:0/1653140739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7c6f4ea00 msgr2=0x55c7c6f5ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:58.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.862+0000 7f622f194740 1 --2- 172.21.15.7:0/1653140739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7c6f4ea00 0x55c7c6f5ca00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6214004e20 tx=0x7f6214031950 comp rx=0 tx=0).stop 2024-06-10T01:15:58.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.862+0000 7f622f194740 1 -- 172.21.15.7:0/1653140739 shutdown_connections 2024-06-10T01:15:58.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.862+0000 7f622f194740 1 --2- 172.21.15.7:0/1653140739 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c7c6fceee0 0x55c7c6fd1300 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.862+0000 7f622f194740 1 --2- 172.21.15.7:0/1653140739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c6f53b70 0x55c7c6fcce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.862+0000 7f622f194740 1 --2- 172.21.15.7:0/1653140739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7c6f4ea00 0x55c7c6f5ca00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.862+0000 7f622f194740 1 --2- 172.21.15.7:0/1653140739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7c6f4e010 0x55c7c6f5c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.862+0000 7f620dffb640 0 client.27365 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:15:58.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.862+0000 7f622f194740 1 -- 172.21.15.7:0/1653140739 >> 172.21.15.7:0/1653140739 conn(0x55c7c6f49680 msgr2=0x55c7c6f58120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:58.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.862+0000 7f622f194740 1 -- 172.21.15.7:0/1653140739 shutdown_connections 2024-06-10T01:15:58.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.862+0000 7f622f194740 1 -- 172.21.15.7:0/1653140739 wait complete. 2024-06-10T01:15:58.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.862+0000 7f622f194740 20 client.27365 trim_cache size 0 max 16384 2024-06-10T01:15:58.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:15:58.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:15:58.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:15:58.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.863+0000 7f622c3a2640 1 -- 172.21.15.7:0/2546649577 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f62100a2b80 msgr2=0x7f62100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:58.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.863+0000 7f622c3a2640 1 --2- 172.21.15.7:0/2546649577 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f62100a2b80 0x7f62100a5040 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6220098750 tx=0x7f62200d5530 comp rx=0 tx=0).stop 2024-06-10T01:15:58.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.863+0000 7f622c3a2640 1 -- 172.21.15.7:0/2546649577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62241000b0 msgr2=0x7f622410b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:15:58.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.863+0000 7f622c3a2640 1 --2- 172.21.15.7:0/2546649577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62241000b0 0x7f622410b6c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f622000e9c0 tx=0x7f622000ee80 comp rx=0 tx=0).stop 2024-06-10T01:15:58.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.863+0000 7f622c3a2640 1 -- 172.21.15.7:0/2546649577 shutdown_connections 2024-06-10T01:15:58.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.863+0000 7f622c3a2640 1 --2- 172.21.15.7:0/2546649577 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f62100a2b80 0x7f62100a5040 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.863+0000 7f622c3a2640 1 --2- 172.21.15.7:0/2546649577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6224101c60 0x7f622410a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.863+0000 7f622c3a2640 1 --2- 172.21.15.7:0/2546649577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62241012a0 0x7f6224109d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.863+0000 7f622c3a2640 1 --2- 172.21.15.7:0/2546649577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62241000b0 0x7f622410b6c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:15:58.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.863+0000 7f622c3a2640 1 -- 172.21.15.7:0/2546649577 >> 172.21.15.7:0/2546649577 conn(0x7f62240fb600 msgr2=0x7f62241045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:15:58.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.863+0000 7f622c3a2640 1 -- 172.21.15.7:0/2546649577 shutdown_connections 2024-06-10T01:15:58.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:15:58.863+0000 7f622c3a2640 1 -- 172.21.15.7:0/2546649577 wait complete. 2024-06-10T01:15:59.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:15:58 smithi175 ceph-mon[36636]: pgmap v731: 129 pgs: 129 active+clean; 117 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 1.9 MiB/s wr, 283 op/s 2024-06-10T01:15:59.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:15:58 smithi007 ceph-mon[32398]: pgmap v731: 129 pgs: 129 active+clean; 117 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 1.9 MiB/s wr, 283 op/s 2024-06-10T01:15:59.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:15:58 smithi078 ceph-mon[36683]: pgmap v731: 129 pgs: 129 active+clean; 117 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 1.9 MiB/s wr, 283 op/s 2024-06-10T01:15:59.746 INFO:tasks.workunit.client.0.smithi007.stdout: 1 713244 7.30 MB/sec execute 400 sec latency 89.528 ms 2024-06-10T01:16:00.746 INFO:tasks.workunit.client.0.smithi007.stdout: 1 714919 7.31 MB/sec execute 401 sec latency 15.454 ms 2024-06-10T01:16:01.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:00 smithi175 ceph-mon[36636]: pgmap v732: 129 pgs: 129 active+clean; 119 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 2.0 MiB/s wr, 296 op/s 2024-06-10T01:16:01.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:00 smithi007 ceph-mon[32398]: pgmap v732: 129 pgs: 129 active+clean; 119 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 2.0 MiB/s wr, 296 op/s 2024-06-10T01:16:01.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:00 smithi078 ceph-mon[36683]: pgmap v732: 129 pgs: 129 active+clean; 119 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 2.0 MiB/s wr, 296 op/s 2024-06-10T01:16:01.746 INFO:tasks.workunit.client.0.smithi007.stdout: 1 717539 7.33 MB/sec execute 402 sec latency 12.424 ms 2024-06-10T01:16:02.746 INFO:tasks.workunit.client.0.smithi007.stdout: 1 718962 7.33 MB/sec execute 403 sec latency 28.007 ms 2024-06-10T01:16:03.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:02 smithi175 ceph-mon[36636]: pgmap v733: 129 pgs: 129 active+clean; 116 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 1.6 MiB/s wr, 281 op/s 2024-06-10T01:16:03.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:02 smithi007 ceph-mon[32398]: pgmap v733: 129 pgs: 129 active+clean; 116 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 1.6 MiB/s wr, 281 op/s 2024-06-10T01:16:03.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:02 smithi078 ceph-mon[36683]: pgmap v733: 129 pgs: 129 active+clean; 116 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 1.6 MiB/s wr, 281 op/s 2024-06-10T01:16:03.747 INFO:tasks.workunit.client.0.smithi007.stdout: 1 721258 7.34 MB/sec execute 404 sec latency 55.716 ms 2024-06-10T01:16:04.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:03 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:16:04.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:03 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:16:04.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:03 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:16:04.746 INFO:tasks.workunit.client.0.smithi007.stdout: 1 723464 7.34 MB/sec execute 405 sec latency 10.313 ms 2024-06-10T01:16:05.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:04 smithi175 ceph-mon[36636]: pgmap v734: 129 pgs: 129 active+clean; 118 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 1.5 MiB/s wr, 270 op/s 2024-06-10T01:16:05.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:04 smithi007 ceph-mon[32398]: pgmap v734: 129 pgs: 129 active+clean; 118 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 1.5 MiB/s wr, 270 op/s 2024-06-10T01:16:05.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:04 smithi078 ceph-mon[36683]: pgmap v734: 129 pgs: 129 active+clean; 118 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 1.5 MiB/s wr, 270 op/s 2024-06-10T01:16:05.746 INFO:tasks.workunit.client.0.smithi007.stdout: 1 725144 7.35 MB/sec execute 406 sec latency 10.936 ms 2024-06-10T01:16:06.747 INFO:tasks.workunit.client.0.smithi007.stdout: 1 727706 7.37 MB/sec execute 407 sec latency 22.447 ms 2024-06-10T01:16:07.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:06 smithi175 ceph-mon[36636]: pgmap v735: 129 pgs: 129 active+clean; 120 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.3 MiB/s wr, 294 op/s 2024-06-10T01:16:07.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:06 smithi007 ceph-mon[32398]: pgmap v735: 129 pgs: 129 active+clean; 120 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.3 MiB/s wr, 294 op/s 2024-06-10T01:16:07.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:06 smithi078 ceph-mon[36683]: pgmap v735: 129 pgs: 129 active+clean; 120 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.3 MiB/s wr, 294 op/s 2024-06-10T01:16:07.746 INFO:tasks.workunit.client.0.smithi007.stdout: 1 729005 7.37 MB/sec execute 408 sec latency 19.101 ms 2024-06-10T01:16:08.746 INFO:tasks.workunit.client.0.smithi007.stdout: 1 731695 7.39 MB/sec execute 409 sec latency 20.645 ms 2024-06-10T01:16:09.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:08 smithi175 ceph-mon[36636]: pgmap v736: 129 pgs: 129 active+clean; 121 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.1 MiB/s wr, 236 op/s 2024-06-10T01:16:09.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:08 smithi007 ceph-mon[32398]: pgmap v736: 129 pgs: 129 active+clean; 121 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.1 MiB/s wr, 236 op/s 2024-06-10T01:16:09.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:08 smithi078 ceph-mon[36683]: pgmap v736: 129 pgs: 129 active+clean; 121 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.1 MiB/s wr, 236 op/s 2024-06-10T01:16:09.746 INFO:tasks.workunit.client.0.smithi007.stdout: 1 733452 7.39 MB/sec execute 410 sec latency 10.835 ms 2024-06-10T01:16:10.747 INFO:tasks.workunit.client.0.smithi007.stdout: 1 735648 7.40 MB/sec execute 411 sec latency 10.073 ms 2024-06-10T01:16:11.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:10 smithi175 ceph-mon[36636]: pgmap v737: 129 pgs: 129 active+clean; 124 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 1.2 MiB/s wr, 275 op/s 2024-06-10T01:16:11.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:10 smithi007 ceph-mon[32398]: pgmap v737: 129 pgs: 129 active+clean; 124 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 1.2 MiB/s wr, 275 op/s 2024-06-10T01:16:11.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:10 smithi078 ceph-mon[36683]: pgmap v737: 129 pgs: 129 active+clean; 124 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 1.2 MiB/s wr, 275 op/s 2024-06-10T01:16:11.747 INFO:tasks.workunit.client.0.smithi007.stdout: 1 738206 7.42 MB/sec execute 412 sec latency 15.408 ms 2024-06-10T01:16:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.960+0000 7f8a62e4d640 1 -- 172.21.15.7:0/2596054419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a5c1024c0 msgr2=0x7f8a5c1048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.960+0000 7f8a62e4d640 1 --2- 172.21.15.7:0/2596054419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a5c1024c0 0x7f8a5c1048b0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f8a50009900 tx=0x7f8a5002a410 comp rx=0 tx=0).stop 2024-06-10T01:16:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.960+0000 7f8a62e4d640 1 -- 172.21.15.7:0/2596054419 shutdown_connections 2024-06-10T01:16:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.960+0000 7f8a62e4d640 1 --2- 172.21.15.7:0/2596054419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a5c1077b0 0x7f8a5c109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.960+0000 7f8a62e4d640 1 --2- 172.21.15.7:0/2596054419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a5c104df0 0x7f8a5c1071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.960+0000 7f8a62e4d640 1 --2- 172.21.15.7:0/2596054419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a5c1024c0 0x7f8a5c1048b0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.960+0000 7f8a62e4d640 1 -- 172.21.15.7:0/2596054419 >> 172.21.15.7:0/2596054419 conn(0x7f8a5c0fbeb0 msgr2=0x7f8a5c0fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.960+0000 7f8a62e4d640 1 -- 172.21.15.7:0/2596054419 shutdown_connections 2024-06-10T01:16:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.960+0000 7f8a62e4d640 1 -- 172.21.15.7:0/2596054419 wait complete. 2024-06-10T01:16:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.960+0000 7f8a62e4d640 1 Processor -- start 2024-06-10T01:16:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.961+0000 7f8a62e4d640 1 -- start start 2024-06-10T01:16:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.961+0000 7f8a62e4d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a5c1024c0 0x7f8a5c197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.961+0000 7f8a62e4d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a5c104df0 0x7f8a5c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.961+0000 7f8a62e4d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a5c1077b0 0x7f8a5c19f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.961+0000 7f8a62e4d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c10d7b0 con 0x7f8a5c1024c0 2024-06-10T01:16:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.961+0000 7f8a62e4d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c10d630 con 0x7f8a5c1077b0 2024-06-10T01:16:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.961+0000 7f8a62e4d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c10d930 con 0x7f8a5c104df0 2024-06-10T01:16:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.961+0000 7f8a60be9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a5c1024c0 0x7f8a5c197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.961+0000 7f8a60be9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a5c1024c0 0x7f8a5c197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46950/0 (socket says 172.21.15.7:46950) 2024-06-10T01:16:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.961+0000 7f8a60be9640 1 -- 172.21.15.7:0/795116473 learned_addr learned my addr 172.21.15.7:0/795116473 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.961+0000 7f8a613ea640 1 --2- 172.21.15.7:0/795116473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a5c1077b0 0x7f8a5c19f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.961+0000 7f8a5bfff640 1 --2- 172.21.15.7:0/795116473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a5c104df0 0x7f8a5c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.961+0000 7f8a60be9640 1 -- 172.21.15.7:0/795116473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a5c104df0 msgr2=0x7f8a5c198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.961+0000 7f8a60be9640 1 --2- 172.21.15.7:0/795116473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a5c104df0 0x7f8a5c198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.961+0000 7f8a60be9640 1 -- 172.21.15.7:0/795116473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a5c1077b0 msgr2=0x7f8a5c19f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.961+0000 7f8a60be9640 1 --2- 172.21.15.7:0/795116473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a5c1077b0 0x7f8a5c19f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.961+0000 7f8a60be9640 1 -- 172.21.15.7:0/795116473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a5c19fa70 con 0x7f8a5c1024c0 2024-06-10T01:16:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.961+0000 7f8a60be9640 1 --2- 172.21.15.7:0/795116473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a5c1024c0 0x7f8a5c197e50 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f8a50009230 tx=0x7f8a500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.961+0000 7f8a59ffb640 1 -- 172.21.15.7:0/795116473 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5003f030 con 0x7f8a5c1024c0 2024-06-10T01:16:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.962+0000 7f8a59ffb640 1 -- 172.21.15.7:0/795116473 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a50003ce0 con 0x7f8a5c1024c0 2024-06-10T01:16:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.962+0000 7f8a62e4d640 1 -- 172.21.15.7:0/795116473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a5c19fd00 con 0x7f8a5c1024c0 2024-06-10T01:16:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.962+0000 7f8a59ffb640 1 -- 172.21.15.7:0/795116473 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a50035070 con 0x7f8a5c1024c0 2024-06-10T01:16:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.962+0000 7f8a62e4d640 1 -- 172.21.15.7:0/795116473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a5c1a0190 con 0x7f8a5c1024c0 2024-06-10T01:16:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.964+0000 7f8a59ffb640 1 -- 172.21.15.7:0/795116473 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f8a50042030 con 0x7f8a5c1024c0 2024-06-10T01:16:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.964+0000 7f8a65c38740 1 Processor -- start 2024-06-10T01:16:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.964+0000 7f8a65c38740 1 -- start start 2024-06-10T01:16:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.965+0000 7f8a65c38740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630502e4420 0x5630502e4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.965+0000 7f8a65c38740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630502e4e10 0x5630502e9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.965+0000 7f8a65c38740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630502e9f80 0x5630502ec410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.965+0000 7f8a65c38740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5630502434f0 con 0x5630502e9f80 2024-06-10T01:16:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.965+0000 7f8a65c38740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5630501e1870 con 0x5630502e4420 2024-06-10T01:16:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.965+0000 7f8a65c38740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56305023f4b0 con 0x5630502e4e10 2024-06-10T01:16:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.965+0000 7f8a613ea640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630502e4420 0x5630502e4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.965+0000 7f8a5bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630502e9f80 0x5630502ec410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.965+0000 7f8a613ea640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630502e4420 0x5630502e4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:36742/0 (socket says 172.21.15.7:36742) 2024-06-10T01:16:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.965+0000 7f8a613ea640 1 -- 172.21.15.7:0/3080921344 learned_addr learned my addr 172.21.15.7:0/3080921344 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.965+0000 7f8a60be9640 1 --2- 172.21.15.7:0/3080921344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630502e4e10 0x5630502e9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.965+0000 7f8a59ffb640 1 --2- 172.21.15.7:0/795116473 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8a4c0a2c00 0x7f8a4c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.965+0000 7f8a5bfff640 1 -- 172.21.15.7:0/3080921344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630502e4e10 msgr2=0x5630502e9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.965+0000 7f8a5bfff640 1 --2- 172.21.15.7:0/3080921344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630502e4e10 0x5630502e9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.965+0000 7f8a5bfff640 1 -- 172.21.15.7:0/3080921344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630502e4420 msgr2=0x5630502e4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.965+0000 7f8a5bfff640 1 --2- 172.21.15.7:0/3080921344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630502e4420 0x5630502e4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.965+0000 7f8a5bfff640 1 -- 172.21.15.7:0/3080921344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630501fd9b0 con 0x5630502e9f80 2024-06-10T01:16:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.965+0000 7f8a613ea640 1 --2- 172.21.15.7:0/3080921344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630502e4420 0x5630502e4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.965+0000 7f8a60be9640 1 --2- 172.21.15.7:0/3080921344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630502e4e10 0x5630502e9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:16:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.965+0000 7f8a613ea640 1 --2- 172.21.15.7:0/795116473 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8a4c0a2c00 0x7f8a4c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.965+0000 7f8a59ffb640 1 -- 172.21.15.7:0/795116473 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f8a500ef7e0 con 0x7f8a5c1024c0 2024-06-10T01:16:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.965+0000 7f8a5bfff640 1 --2- 172.21.15.7:0/3080921344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630502e9f80 0x5630502ec410 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c06a7f0 tx=0x7f8a48011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d427791f868b30e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.966+0000 7f8a46ffd640 1 -- 172.21.15.7:0/3080921344 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4801e070 con 0x5630502e9f80 2024-06-10T01:16:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.966+0000 7f8a46ffd640 1 -- 172.21.15.7:0/3080921344 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a48009d70 con 0x5630502e9f80 2024-06-10T01:16:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.966+0000 7f8a46ffd640 1 -- 172.21.15.7:0/3080921344 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a480187a0 con 0x5630502e9f80 2024-06-10T01:16:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.967+0000 7f8a65c38740 1 -- 172.21.15.7:0/3080921344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630502e9f80 msgr2=0x5630502ec410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.967+0000 7f8a65c38740 1 --2- 172.21.15.7:0/3080921344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630502e9f80 0x5630502ec410 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c06a7f0 tx=0x7f8a48011810 comp rx=0 tx=0).stop 2024-06-10T01:16:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.967+0000 7f8a613ea640 1 --2- 172.21.15.7:0/795116473 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8a4c0a2c00 0x7f8a4c0a50c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x5630502e5670 tx=0x7f8a54004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.967+0000 7f8a65c38740 1 -- 172.21.15.7:0/3080921344 shutdown_connections 2024-06-10T01:16:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.967+0000 7f8a65c38740 1 --2- 172.21.15.7:0/3080921344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630502e9f80 0x5630502ec410 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.967+0000 7f8a65c38740 1 --2- 172.21.15.7:0/3080921344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630502e4e10 0x5630502e9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.967+0000 7f8a65c38740 1 --2- 172.21.15.7:0/3080921344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630502e4420 0x5630502e4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.967+0000 7f8a65c38740 1 -- 172.21.15.7:0/3080921344 >> 172.21.15.7:0/3080921344 conn(0x5630502db680 msgr2=0x5630502dba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.967+0000 7f8a65c38740 1 -- 172.21.15.7:0/3080921344 shutdown_connections 2024-06-10T01:16:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.967+0000 7f8a65c38740 1 -- 172.21.15.7:0/3080921344 wait complete. 2024-06-10T01:16:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.967+0000 7f8a65c38740 1 -- start start 2024-06-10T01:16:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.967+0000 7f8a65c38740 1 Processor -- start 2024-06-10T01:16:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.967+0000 7f8a65c38740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630502e4420 0x5630502ee6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.967+0000 7f8a65c38740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630502e4e10 0x5630502eec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.967+0000 7f8a65c38740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630502e9f80 0x5630502f5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.967+0000 7f8a65c38740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5630502ecf30 con 0x5630502e9f80 2024-06-10T01:16:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.968+0000 7f8a65c38740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5630501ccae0 con 0x5630502e4e10 2024-06-10T01:16:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.968+0000 7f8a65c38740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5630502ed0b0 con 0x5630502e4420 2024-06-10T01:16:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.968+0000 7f8a65c38740 10 client.0 _ll_register_callbacks cb 0x5630502059b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:16:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.968+0000 7f8a613ea640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630502e9f80 0x5630502f5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.968+0000 7f8a60be9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630502e4420 0x5630502ee6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.968+0000 7f8a5bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630502e4e10 0x5630502eec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.968+0000 7f8a613ea640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630502e9f80 0x5630502f5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46978/0 (socket says 172.21.15.7:46978) 2024-06-10T01:16:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.968+0000 7f8a60be9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630502e4420 0x5630502ee6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47874/0 (socket says 172.21.15.7:47874) 2024-06-10T01:16:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.968+0000 7f8a613ea640 1 -- 172.21.15.7:0/2747628363 learned_addr learned my addr 172.21.15.7:0/2747628363 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.968+0000 7f8a60be9640 1 -- 172.21.15.7:0/2747628363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630502e4e10 msgr2=0x5630502eec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.968+0000 7f8a60be9640 1 --2- 172.21.15.7:0/2747628363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630502e4e10 0x5630502eec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.968+0000 7f8a60be9640 1 -- 172.21.15.7:0/2747628363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630502e9f80 msgr2=0x5630502f5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.968+0000 7f8a60be9640 1 --2- 172.21.15.7:0/2747628363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630502e9f80 0x5630502f5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.968+0000 7f8a60be9640 1 -- 172.21.15.7:0/2747628363 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56304ff57210 con 0x5630502e4420 2024-06-10T01:16:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.968+0000 7f8a613ea640 1 --2- 172.21.15.7:0/2747628363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630502e9f80 0x5630502f5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.968+0000 7f8a60be9640 1 --2- 172.21.15.7:0/2747628363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630502e4420 0x5630502ee6c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8a500ecfb0 tx=0x7f8a500bbb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e8053053b959a18 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.968+0000 7f8a3bfff640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:16:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.968+0000 7f8a3bfff640 1 -- 172.21.15.7:0/2747628363 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a50049070 con 0x5630502e4420 2024-06-10T01:16:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.968+0000 7f8a3bfff640 1 -- 172.21.15.7:0/2747628363 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f8a50003800 con 0x5630502e4420 2024-06-10T01:16:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.968+0000 7f8a65c38740 1 -- 172.21.15.7:0/2747628363 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a48009d70 con 0x5630502e4420 2024-06-10T01:16:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.969+0000 7f8a3bfff640 1 -- 172.21.15.7:0/2747628363 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f8a500bbd70 con 0x5630502e4420 2024-06-10T01:16:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.969+0000 7f8a65c38740 10 client.27397 fetch_fsmap learned FSMap version 38 2024-06-10T01:16:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.969+0000 7f8a65c38740 1 -- 172.21.15.7:0/2747628363 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563050023620 con 0x5630502e4420 2024-06-10T01:16:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.969+0000 7f8a3bfff640 1 -- 172.21.15.7:0/2747628363 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a48009d70 con 0x5630502e4420 2024-06-10T01:16:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.969+0000 7f8a3bfff640 1 -- 172.21.15.7:0/2747628363 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f8a5003bc10 con 0x5630502e4420 2024-06-10T01:16:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.969+0000 7f8a65c38740 10 client.27397 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T01:16:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.969+0000 7f8a65c38740 10 client.27397 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:16:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.969+0000 7f8a65c38740 20 client.27397 populate_metadata read hostname 'smithi007' 2024-06-10T01:16:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.969+0000 7f8a65c38740 1 --2- 172.21.15.7:0/2747628363 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5630502fa590 0x5630502fca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.969+0000 7f8a65c38740 4 client.27397 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:16:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.969+0000 7f8a65c38740 1 -- 172.21.15.7:0/2747628363 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563050256f10 con 0x5630502fa590 2024-06-10T01:16:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.969+0000 7f8a5bfff640 1 --2- 172.21.15.7:0/2747628363 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5630502fa590 0x5630502fca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.970+0000 7f8a5bfff640 1 --2- 172.21.15.7:0/2747628363 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5630502fa590 0x5630502fca00 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faa475571b998507 server_cookie=fc745b6b046e0305 in_seq=0 out_seq=0 2024-06-10T01:16:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.970+0000 7f8a3bfff640 10 client.27397 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.970+0000 7f8a3bfff640 1 -- 172.21.15.7:0/2747628363 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563050256f10 con 0x5630502fa590 2024-06-10T01:16:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.970+0000 7f8a3bfff640 10 client.27397 handle_command_reply: tid=0 2024-06-10T01:16:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.970+0000 7f8a65c38740 1 client.27397 shutdown 2024-06-10T01:16:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.970+0000 7f8a65c38740 1 -- 172.21.15.7:0/2747628363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630502e4420 msgr2=0x5630502ee6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.970+0000 7f8a65c38740 1 --2- 172.21.15.7:0/2747628363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630502e4420 0x5630502ee6c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8a500ecfb0 tx=0x7f8a500bbb40 comp rx=0 tx=0).stop 2024-06-10T01:16:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.970+0000 7f8a65c38740 1 -- 172.21.15.7:0/2747628363 shutdown_connections 2024-06-10T01:16:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.970+0000 7f8a65c38740 1 --2- 172.21.15.7:0/2747628363 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5630502fa590 0x5630502fca00 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.970+0000 7f8a65c38740 1 --2- 172.21.15.7:0/2747628363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630502e9f80 0x5630502f5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.970+0000 7f8a65c38740 1 --2- 172.21.15.7:0/2747628363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630502e4e10 0x5630502eec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.970+0000 7f8a65c38740 1 --2- 172.21.15.7:0/2747628363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630502e4420 0x5630502ee6c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.970+0000 7f8a3bfff640 0 client.27397 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.970+0000 7f8a65c38740 1 -- 172.21.15.7:0/2747628363 >> 172.21.15.7:0/2747628363 conn(0x5630502db680 msgr2=0x5630502da7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.971+0000 7f8a65c38740 1 -- 172.21.15.7:0/2747628363 shutdown_connections 2024-06-10T01:16:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.971+0000 7f8a65c38740 1 -- 172.21.15.7:0/2747628363 wait complete. 2024-06-10T01:16:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.971+0000 7f8a65c38740 20 client.27397 trim_cache size 0 max 16384 2024-06-10T01:16:11.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.987+0000 7f8a65c38740 1 Processor -- start 2024-06-10T01:16:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.987+0000 7f8a65c38740 1 -- start start 2024-06-10T01:16:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.987+0000 7f8a65c38740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630502db680 0x5630502deca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.987+0000 7f8a65c38740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630502e4420 0x5630502f5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.987+0000 7f8a65c38740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630502e4e10 0x5630502ee2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.987+0000 7f8a65c38740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c1aac70 con 0x5630502e4e10 2024-06-10T01:16:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.987+0000 7f8a65c38740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c1a84e0 con 0x5630502e4420 2024-06-10T01:16:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.987+0000 7f8a65c38740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c1a6330 con 0x5630502db680 2024-06-10T01:16:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.987+0000 7f8a613ea640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630502e4e10 0x5630502ee2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.987+0000 7f8a613ea640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630502e4e10 0x5630502ee2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46992/0 (socket says 172.21.15.7:46992) 2024-06-10T01:16:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.987+0000 7f8a613ea640 1 -- 172.21.15.7:0/3286490220 learned_addr learned my addr 172.21.15.7:0/3286490220 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.987+0000 7f8a5bfff640 1 --2- 172.21.15.7:0/3286490220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630502e4420 0x5630502f5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.987+0000 7f8a60be9640 1 --2- 172.21.15.7:0/3286490220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630502db680 0x5630502deca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.987+0000 7f8a613ea640 1 -- 172.21.15.7:0/3286490220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630502db680 msgr2=0x5630502deca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.987+0000 7f8a613ea640 1 --2- 172.21.15.7:0/3286490220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630502db680 0x5630502deca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.987+0000 7f8a613ea640 1 -- 172.21.15.7:0/3286490220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630502e4420 msgr2=0x5630502f5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.987+0000 7f8a613ea640 1 --2- 172.21.15.7:0/3286490220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630502e4420 0x5630502f5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.987+0000 7f8a613ea640 1 -- 172.21.15.7:0/3286490220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630502db4b0 con 0x5630502e4e10 2024-06-10T01:16:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.987+0000 7f8a5bfff640 1 --2- 172.21.15.7:0/3286490220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630502e4420 0x5630502f5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.988+0000 7f8a613ea640 1 --2- 172.21.15.7:0/3286490220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630502e4e10 0x5630502ee2d0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f8a54011ba0 tx=0x7f8a54019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d1fd3457466f5b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.988+0000 7f8a467fc640 1 -- 172.21.15.7:0/3286490220 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a54011c90 con 0x5630502e4e10 2024-06-10T01:16:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.988+0000 7f8a467fc640 1 -- 172.21.15.7:0/3286490220 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a54011e30 con 0x5630502e4e10 2024-06-10T01:16:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.988+0000 7f8a467fc640 1 -- 172.21.15.7:0/3286490220 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a54005850 con 0x5630502e4e10 2024-06-10T01:16:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.988+0000 7f8a65c38740 1 -- 172.21.15.7:0/3286490220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630502e4e10 msgr2=0x5630502ee2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.988+0000 7f8a65c38740 1 --2- 172.21.15.7:0/3286490220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630502e4e10 0x5630502ee2d0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f8a54011ba0 tx=0x7f8a54019000 comp rx=0 tx=0).stop 2024-06-10T01:16:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.988+0000 7f8a65c38740 1 -- 172.21.15.7:0/3286490220 shutdown_connections 2024-06-10T01:16:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.988+0000 7f8a65c38740 1 --2- 172.21.15.7:0/3286490220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630502e4e10 0x5630502ee2d0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.989+0000 7f8a65c38740 1 --2- 172.21.15.7:0/3286490220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630502e4420 0x5630502f5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.989+0000 7f8a65c38740 1 --2- 172.21.15.7:0/3286490220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630502db680 0x5630502deca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.989+0000 7f8a65c38740 1 -- 172.21.15.7:0/3286490220 >> 172.21.15.7:0/3286490220 conn(0x5630502e9a50 msgr2=0x5630502de590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.989+0000 7f8a65c38740 1 -- 172.21.15.7:0/3286490220 shutdown_connections 2024-06-10T01:16:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.989+0000 7f8a65c38740 1 -- 172.21.15.7:0/3286490220 wait complete. 2024-06-10T01:16:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.989+0000 7f8a65c38740 1 -- start start 2024-06-10T01:16:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.989+0000 7f8a65c38740 1 Processor -- start 2024-06-10T01:16:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.989+0000 7f8a65c38740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630502db680 0x5630502f2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.989+0000 7f8a65c38740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630502e4420 0x5630502f31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.989+0000 7f8a65c38740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630502e4e10 0x5630502f41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.989+0000 7f8a65c38740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c1a69a0 con 0x5630502e4e10 2024-06-10T01:16:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.989+0000 7f8a65c38740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c1a7e10 con 0x5630502e4420 2024-06-10T01:16:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.989+0000 7f8a65c38740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c1a7070 con 0x5630502db680 2024-06-10T01:16:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.989+0000 7f8a65c38740 10 client.0 _ll_register_callbacks cb 0x7f8a50005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:16:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.989+0000 7f8a60be9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630502db680 0x5630502f2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.989+0000 7f8a613ea640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630502e4e10 0x5630502f41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.989+0000 7f8a60be9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630502db680 0x5630502f2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47900/0 (socket says 172.21.15.7:47900) 2024-06-10T01:16:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.989+0000 7f8a60be9640 1 -- 172.21.15.7:0/2662548585 learned_addr learned my addr 172.21.15.7:0/2662548585 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.990+0000 7f8a5bfff640 1 --2- 172.21.15.7:0/2662548585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630502e4420 0x5630502f31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.990+0000 7f8a60be9640 1 -- 172.21.15.7:0/2662548585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630502e4420 msgr2=0x5630502f31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.990+0000 7f8a60be9640 1 --2- 172.21.15.7:0/2662548585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630502e4420 0x5630502f31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.990+0000 7f8a60be9640 1 -- 172.21.15.7:0/2662548585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630502e4e10 msgr2=0x5630502f41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.990+0000 7f8a60be9640 1 --2- 172.21.15.7:0/2662548585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630502e4e10 0x5630502f41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.990+0000 7f8a60be9640 1 -- 172.21.15.7:0/2662548585 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630502f47c0 con 0x5630502db680 2024-06-10T01:16:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.990+0000 7f8a613ea640 1 --2- 172.21.15.7:0/2662548585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630502e4e10 0x5630502f41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.990+0000 7f8a5bfff640 1 --2- 172.21.15.7:0/2662548585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630502e4420 0x5630502f31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.990+0000 7f8a60be9640 1 --2- 172.21.15.7:0/2662548585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630502db680 0x5630502f2c80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8a500314e0 tx=0x7f8a50003e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d886a5db7de710 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.990+0000 7f8a3bfff640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:16:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.990+0000 7f8a3bfff640 1 -- 172.21.15.7:0/2662548585 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a50049050 con 0x5630502db680 2024-06-10T01:16:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.990+0000 7f8a65c38740 1 -- 172.21.15.7:0/2662548585 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a54011e30 con 0x5630502db680 2024-06-10T01:16:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.990+0000 7f8a3bfff640 1 -- 172.21.15.7:0/2662548585 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f8a50003800 con 0x5630502db680 2024-06-10T01:16:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.990+0000 7f8a3bfff640 1 -- 172.21.15.7:0/2662548585 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f8a500c0070 con 0x5630502db680 2024-06-10T01:16:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.990+0000 7f8a65c38740 10 client.27403 fetch_fsmap learned FSMap version 38 2024-06-10T01:16:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.990+0000 7f8a65c38740 1 -- 172.21.15.7:0/2662548585 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5630502f5ad0 con 0x5630502db680 2024-06-10T01:16:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.991+0000 7f8a3bfff640 1 -- 172.21.15.7:0/2662548585 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a54011e30 con 0x5630502db680 2024-06-10T01:16:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.991+0000 7f8a3bfff640 1 -- 172.21.15.7:0/2662548585 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f8a5009abe0 con 0x5630502db680 2024-06-10T01:16:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.991+0000 7f8a65c38740 10 client.27403 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T01:16:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.991+0000 7f8a65c38740 10 client.27403 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:16:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.991+0000 7f8a65c38740 20 client.27403 populate_metadata read hostname 'smithi007' 2024-06-10T01:16:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.991+0000 7f8a65c38740 1 --2- 172.21.15.7:0/2662548585 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5630502e9f80 0x563050367570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.991+0000 7f8a65c38740 4 client.27403 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:16:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.991+0000 7f8a65c38740 1 -- 172.21.15.7:0/2662548585 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563050250c20 con 0x5630502e9f80 2024-06-10T01:16:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.991+0000 7f8a5bfff640 1 --2- 172.21.15.7:0/2662548585 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5630502e9f80 0x563050367570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.991+0000 7f8a3bfff640 10 client.27403 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:11.991+0000 7f8a5bfff640 1 --2- 172.21.15.7:0/2662548585 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5630502e9f80 0x563050367570 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a2079f02bcbbaa9 server_cookie=b83e20300c70a80d in_seq=0 out_seq=0 2024-06-10T01:16:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:12.115+0000 7f8a3bfff640 1 -- 172.21.15.7:0/2662548585 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563050250c20 con 0x5630502e9f80 2024-06-10T01:16:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:12.115+0000 7f8a3bfff640 10 client.27403 handle_command_reply: tid=0 2024-06-10T01:16:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:12.115+0000 7f8a65c38740 1 client.27403 shutdown 2024-06-10T01:16:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:12.116+0000 7f8a65c38740 1 -- 172.21.15.7:0/2662548585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630502db680 msgr2=0x5630502f2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:12.116+0000 7f8a65c38740 1 --2- 172.21.15.7:0/2662548585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630502db680 0x5630502f2c80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8a500314e0 tx=0x7f8a50003e90 comp rx=0 tx=0).stop 2024-06-10T01:16:12.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:12.116+0000 7f8a65c38740 1 -- 172.21.15.7:0/2662548585 shutdown_connections 2024-06-10T01:16:12.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:12.116+0000 7f8a65c38740 1 --2- 172.21.15.7:0/2662548585 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5630502e9f80 0x563050367570 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:12.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:12.116+0000 7f8a65c38740 1 --2- 172.21.15.7:0/2662548585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630502e4e10 0x5630502f41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:12.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:12.116+0000 7f8a65c38740 1 --2- 172.21.15.7:0/2662548585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630502e4420 0x5630502f31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:12.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:12.116+0000 7f8a3bfff640 0 client.27403 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:12.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:12.116+0000 7f8a65c38740 1 --2- 172.21.15.7:0/2662548585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630502db680 0x5630502f2c80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:12.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:12.116+0000 7f8a65c38740 1 -- 172.21.15.7:0/2662548585 >> 172.21.15.7:0/2662548585 conn(0x5630502e9a50 msgr2=0x5630502ee1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:12.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:12.116+0000 7f8a65c38740 1 -- 172.21.15.7:0/2662548585 shutdown_connections 2024-06-10T01:16:12.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:12.116+0000 7f8a65c38740 1 -- 172.21.15.7:0/2662548585 wait complete. 2024-06-10T01:16:12.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:12.116+0000 7f8a65c38740 20 client.27403 trim_cache size 0 max 16384 2024-06-10T01:16:12.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:16:12.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:16:12.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:16:12.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:12.116+0000 7f8a62e4d640 1 -- 172.21.15.7:0/795116473 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8a4c0a2c00 msgr2=0x7f8a4c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:12.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:12.116+0000 7f8a62e4d640 1 --2- 172.21.15.7:0/795116473 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8a4c0a2c00 0x7f8a4c0a50c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x5630502e5670 tx=0x7f8a54004a40 comp rx=0 tx=0).stop 2024-06-10T01:16:12.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:12.116+0000 7f8a62e4d640 1 -- 172.21.15.7:0/795116473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a5c1024c0 msgr2=0x7f8a5c197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:12.116+0000 7f8a62e4d640 1 --2- 172.21.15.7:0/795116473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a5c1024c0 0x7f8a5c197e50 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f8a50009230 tx=0x7f8a500042f0 comp rx=0 tx=0).stop 2024-06-10T01:16:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:12.117+0000 7f8a62e4d640 1 -- 172.21.15.7:0/795116473 shutdown_connections 2024-06-10T01:16:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:12.117+0000 7f8a62e4d640 1 --2- 172.21.15.7:0/795116473 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8a4c0a2c00 0x7f8a4c0a50c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:12.117+0000 7f8a62e4d640 1 --2- 172.21.15.7:0/795116473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a5c1077b0 0x7f8a5c19f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:12.117+0000 7f8a62e4d640 1 --2- 172.21.15.7:0/795116473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a5c104df0 0x7f8a5c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:12.117+0000 7f8a62e4d640 1 --2- 172.21.15.7:0/795116473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a5c1024c0 0x7f8a5c197e50 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:12.117+0000 7f8a62e4d640 1 -- 172.21.15.7:0/795116473 >> 172.21.15.7:0/795116473 conn(0x7f8a5c0fbeb0 msgr2=0x7f8a5c102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:12.117+0000 7f8a62e4d640 1 -- 172.21.15.7:0/795116473 shutdown_connections 2024-06-10T01:16:12.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:12.117+0000 7f8a62e4d640 1 -- 172.21.15.7:0/795116473 wait complete. 2024-06-10T01:16:12.747 INFO:tasks.workunit.client.0.smithi007.stdout: 1 739473 7.41 MB/sec execute 413 sec latency 119.702 ms 2024-06-10T01:16:13.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:12 smithi175 ceph-mon[36636]: pgmap v738: 129 pgs: 129 active+clean; 126 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 1.2 MiB/s wr, 265 op/s 2024-06-10T01:16:13.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:12 smithi007 ceph-mon[32398]: pgmap v738: 129 pgs: 129 active+clean; 126 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 1.2 MiB/s wr, 265 op/s 2024-06-10T01:16:13.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:12 smithi078 ceph-mon[36683]: pgmap v738: 129 pgs: 129 active+clean; 126 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 1.2 MiB/s wr, 265 op/s 2024-06-10T01:16:13.747 INFO:tasks.workunit.client.0.smithi007.stdout: 1 742103 7.44 MB/sec execute 414 sec latency 21.703 ms 2024-06-10T01:16:14.747 INFO:tasks.workunit.client.0.smithi007.stdout: 1 743200 7.43 MB/sec execute 415 sec latency 26.754 ms 2024-06-10T01:16:15.133 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:16:15.173 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:14 smithi007 ceph-mon[32398]: pgmap v739: 129 pgs: 129 active+clean; 123 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 1.1 MiB/s wr, 249 op/s 2024-06-10T01:16:15.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:14 smithi175 ceph-mon[36636]: pgmap v739: 129 pgs: 129 active+clean; 123 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 1.1 MiB/s wr, 249 op/s 2024-06-10T01:16:15.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:14 smithi078 ceph-mon[36683]: pgmap v739: 129 pgs: 129 active+clean; 123 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 1.1 MiB/s wr, 249 op/s 2024-06-10T01:16:15.435 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:16:15.747 INFO:tasks.workunit.client.0.smithi007.stdout: 1 745762 7.45 MB/sec execute 416 sec latency 37.416 ms 2024-06-10T01:16:16.747 INFO:tasks.workunit.client.0.smithi007.stdout: 1 747371 7.45 MB/sec execute 417 sec latency 10.271 ms 2024-06-10T01:16:17.111 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:16 smithi007 ceph-mon[32398]: pgmap v740: 129 pgs: 129 active+clean; 127 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.3 MiB/s wr, 290 op/s 2024-06-10T01:16:17.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:16 smithi175 ceph-mon[36636]: pgmap v740: 129 pgs: 129 active+clean; 127 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.3 MiB/s wr, 290 op/s 2024-06-10T01:16:17.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.239+0000 7f596f1ba700 1 -- 172.21.15.7:0/664454776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5968108560 msgr2=0x7f5968108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:17.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.239+0000 7f596f1ba700 1 --2- 172.21.15.7:0/664454776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5968108560 0x7f5968108940 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f595c0099c0 tx=0x7f595c009cd0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.239+0000 7f596f1ba700 1 -- 172.21.15.7:0/664454776 shutdown_connections 2024-06-10T01:16:17.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.239+0000 7f596f1ba700 1 --2- 172.21.15.7:0/664454776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f596806b560 0x7f596810f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.239+0000 7f596f1ba700 1 --2- 172.21.15.7:0/664454776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5968108560 0x7f5968108940 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.239+0000 7f596f1ba700 1 --2- 172.21.15.7:0/664454776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f596806aba0 0x7f596806b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.239+0000 7f596f1ba700 1 -- 172.21.15.7:0/664454776 >> 172.21.15.7:0/664454776 conn(0x7f59680780f0 msgr2=0x7f5968078500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:17.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.239+0000 7f596f1ba700 1 -- 172.21.15.7:0/664454776 shutdown_connections 2024-06-10T01:16:17.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.239+0000 7f596f1ba700 1 -- 172.21.15.7:0/664454776 wait complete. 2024-06-10T01:16:17.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.239+0000 7f596f1ba700 1 Processor -- start 2024-06-10T01:16:17.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.239+0000 7f596f1ba700 1 -- start start 2024-06-10T01:16:17.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.240+0000 7f596f1ba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f596806aba0 0x7f596819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:17.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.240+0000 7f596f1ba700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f596806b560 0x7f596819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:17.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.240+0000 7f596f1ba700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5968108560 0x7f59681989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:17.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.240+0000 7f596f1ba700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f59681127d0 con 0x7f596806aba0 2024-06-10T01:16:17.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.240+0000 7f596f1ba700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5968112650 con 0x7f596806b560 2024-06-10T01:16:17.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.240+0000 7f596f1ba700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5968112950 con 0x7f5968108560 2024-06-10T01:16:17.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.240+0000 7f596cf56700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f596806aba0 0x7f596819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:17.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.240+0000 7f596d757700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5968108560 0x7f59681989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:17.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.240+0000 7f5967fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f596806b560 0x7f596819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:17.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.240+0000 7f596cf56700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f596806aba0 0x7f596819e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47020/0 (socket says 172.21.15.7:47020) 2024-06-10T01:16:17.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.240+0000 7f596d757700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5968108560 0x7f59681989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47920/0 (socket says 172.21.15.7:47920) 2024-06-10T01:16:17.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.240+0000 7f596cf56700 1 -- 172.21.15.7:0/1223800891 learned_addr learned my addr 172.21.15.7:0/1223800891 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:17.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.240+0000 7f5967fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f596806b560 0x7f596819ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:36806/0 (socket says 172.21.15.7:36806) 2024-06-10T01:16:17.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.240+0000 7f5967fff700 1 -- 172.21.15.7:0/1223800891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5968108560 msgr2=0x7f59681989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:17.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.240+0000 7f5967fff700 1 --2- 172.21.15.7:0/1223800891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5968108560 0x7f59681989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.240+0000 7f5967fff700 1 -- 172.21.15.7:0/1223800891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f596806aba0 msgr2=0x7f596819e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:17.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.240+0000 7f5967fff700 1 --2- 172.21.15.7:0/1223800891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f596806aba0 0x7f596819e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.240+0000 7f5967fff700 1 -- 172.21.15.7:0/1223800891 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5968199150 con 0x7f596806b560 2024-06-10T01:16:17.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.240+0000 7f596d757700 1 --2- 172.21.15.7:0/1223800891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5968108560 0x7f59681989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:17.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.240+0000 7f596cf56700 1 --2- 172.21.15.7:0/1223800891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f596806aba0 0x7f596819e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:17.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.241+0000 7f5967fff700 1 --2- 172.21.15.7:0/1223800891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f596806b560 0x7f596819ed70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f595400e9f0 tx=0x7f595400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:17.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.241+0000 7f5965ffb700 1 -- 172.21.15.7:0/1223800891 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595400c9c0 con 0x7f596806b560 2024-06-10T01:16:17.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.241+0000 7f5965ffb700 1 -- 172.21.15.7:0/1223800891 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f595400cb60 con 0x7f596806b560 2024-06-10T01:16:17.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.241+0000 7f5965ffb700 1 -- 172.21.15.7:0/1223800891 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5954018680 con 0x7f596806b560 2024-06-10T01:16:17.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.241+0000 7f596f1ba700 1 -- 172.21.15.7:0/1223800891 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5968199440 con 0x7f596806b560 2024-06-10T01:16:17.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.241+0000 7f596f1ba700 1 -- 172.21.15.7:0/1223800891 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59681a5700 con 0x7f596806b560 2024-06-10T01:16:17.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.243+0000 7f5965ffb700 1 -- 172.21.15.7:0/1223800891 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f5954010430 con 0x7f596806b560 2024-06-10T01:16:17.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.244+0000 7f5965ffb700 1 --2- 172.21.15.7:0/1223800891 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f59580a2ca0 0x7f59580a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:17.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.244+0000 7f596cf56700 1 --2- 172.21.15.7:0/1223800891 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f59580a2ca0 0x7f59580a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:17.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.244+0000 7f5965ffb700 1 -- 172.21.15.7:0/1223800891 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f59540cf440 con 0x7f596806b560 2024-06-10T01:16:17.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.245+0000 7f596cf56700 1 --2- 172.21.15.7:0/1223800891 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f59580a2ca0 0x7f59580a5160 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f595c000c00 tx=0x7f595c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:17.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.245+0000 7f5979a3db80 1 Processor -- start 2024-06-10T01:16:17.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.245+0000 7f5979a3db80 1 -- start start 2024-06-10T01:16:17.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.245+0000 7f5979a3db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fe294af60 0x563fe294b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:17.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.245+0000 7f5979a3db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563fe294b880 0x563fe2945260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:17.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.245+0000 7f5979a3db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fe29457a0 0x563fe2945bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:17.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.245+0000 7f5979a3db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563fe2837f60 con 0x563fe29457a0 2024-06-10T01:16:17.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.245+0000 7f5979a3db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563fe2896850 con 0x563fe294af60 2024-06-10T01:16:17.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.245+0000 7f5979a3db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563fe27fe460 con 0x563fe294b880 2024-06-10T01:16:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.246+0000 7f596d757700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fe29457a0 0x563fe2945bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.246+0000 7f596d757700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fe29457a0 0x563fe2945bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47034/0 (socket says 172.21.15.7:47034) 2024-06-10T01:16:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.246+0000 7f5967fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563fe294b880 0x563fe2945260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.246+0000 7f596d757700 1 -- 172.21.15.7:0/798822272 learned_addr learned my addr 172.21.15.7:0/798822272 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.246+0000 7f596cf56700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fe294af60 0x563fe294b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.246+0000 7f5967fff700 1 -- 172.21.15.7:0/798822272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fe294af60 msgr2=0x563fe294b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.246+0000 7f5967fff700 1 --2- 172.21.15.7:0/798822272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fe294af60 0x563fe294b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.246+0000 7f5967fff700 1 -- 172.21.15.7:0/798822272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fe29457a0 msgr2=0x563fe2945bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.246+0000 7f5967fff700 1 --2- 172.21.15.7:0/798822272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fe29457a0 0x563fe2945bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.246+0000 7f5967fff700 1 -- 172.21.15.7:0/798822272 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fe27e58d0 con 0x563fe294b880 2024-06-10T01:16:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.246+0000 7f596d757700 1 --2- 172.21.15.7:0/798822272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fe29457a0 0x563fe2945bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.246+0000 7f5967fff700 1 --2- 172.21.15.7:0/798822272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563fe294b880 0x563fe2945260 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f595400e9c0 tx=0x7f595409b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58832bb33548ddc7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.246+0000 7f5952ffd700 1 -- 172.21.15.7:0/798822272 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595409b920 con 0x563fe294b880 2024-06-10T01:16:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.247+0000 7f5952ffd700 1 -- 172.21.15.7:0/798822272 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f595409fcd0 con 0x563fe294b880 2024-06-10T01:16:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.247+0000 7f5952ffd700 1 -- 172.21.15.7:0/798822272 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5954010720 con 0x563fe294b880 2024-06-10T01:16:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.247+0000 7f5979a3db80 1 -- 172.21.15.7:0/798822272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563fe294b880 msgr2=0x563fe2945260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:17.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.247+0000 7f5979a3db80 1 --2- 172.21.15.7:0/798822272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563fe294b880 0x563fe2945260 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f595400e9c0 tx=0x7f595409b820 comp rx=0 tx=0).stop 2024-06-10T01:16:17.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.247+0000 7f5979a3db80 1 -- 172.21.15.7:0/798822272 shutdown_connections 2024-06-10T01:16:17.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.247+0000 7f5979a3db80 1 --2- 172.21.15.7:0/798822272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fe29457a0 0x563fe2945bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.247+0000 7f5979a3db80 1 --2- 172.21.15.7:0/798822272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fe294af60 0x563fe294b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.247+0000 7f5979a3db80 1 --2- 172.21.15.7:0/798822272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563fe294b880 0x563fe2945260 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.247+0000 7f5979a3db80 1 -- 172.21.15.7:0/798822272 >> 172.21.15.7:0/798822272 conn(0x563fe2872080 msgr2=0x563fe2940220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:17.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.247+0000 7f5979a3db80 1 -- 172.21.15.7:0/798822272 shutdown_connections 2024-06-10T01:16:17.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.247+0000 7f5979a3db80 1 -- 172.21.15.7:0/798822272 wait complete. 2024-06-10T01:16:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.248+0000 7f5979a3db80 1 -- start start 2024-06-10T01:16:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.248+0000 7f5979a3db80 1 Processor -- start 2024-06-10T01:16:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.248+0000 7f5979a3db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fe29457a0 0x563fe2951610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.248+0000 7f5979a3db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fe294af60 0x563fe294e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.248+0000 7f5979a3db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563fe294b880 0x563fe294ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.248+0000 7f5979a3db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563fe2836fc0 con 0x563fe294af60 2024-06-10T01:16:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.248+0000 7f5979a3db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563fe2892810 con 0x563fe29457a0 2024-06-10T01:16:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.248+0000 7f5979a3db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563fe2952200 con 0x563fe294b880 2024-06-10T01:16:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.248+0000 7f5979a3db80 10 client.0 _ll_register_callbacks cb 0x563fe282c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:16:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.248+0000 7f596cf56700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fe29457a0 0x563fe2951610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.248+0000 7f5967fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fe294af60 0x563fe294e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.248+0000 7f596d757700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563fe294b880 0x563fe294ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.248+0000 7f596cf56700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fe29457a0 0x563fe2951610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:36822/0 (socket says 172.21.15.7:36822) 2024-06-10T01:16:17.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.248+0000 7f596cf56700 1 -- 172.21.15.7:0/3554247035 learned_addr learned my addr 172.21.15.7:0/3554247035 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:17.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.249+0000 7f5967fff700 1 -- 172.21.15.7:0/3554247035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563fe294b880 msgr2=0x563fe294ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:17.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.249+0000 7f5967fff700 1 --2- 172.21.15.7:0/3554247035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563fe294b880 0x563fe294ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.249+0000 7f5967fff700 1 -- 172.21.15.7:0/3554247035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fe29457a0 msgr2=0x563fe2951610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:17.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.249+0000 7f5967fff700 1 --2- 172.21.15.7:0/3554247035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fe29457a0 0x563fe2951610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.249+0000 7f5967fff700 1 -- 172.21.15.7:0/3554247035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fe285c750 con 0x563fe294af60 2024-06-10T01:16:17.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.249+0000 7f5967fff700 1 --2- 172.21.15.7:0/3554247035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fe294af60 0x563fe294e910 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f595400e9c0 tx=0x7f5954009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d20c284ef04120a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:17.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.249+0000 7f5947fff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:16:17.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.249+0000 7f5947fff700 1 -- 172.21.15.7:0/3554247035 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59540b3070 con 0x563fe294af60 2024-06-10T01:16:17.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.249+0000 7f5947fff700 1 -- 172.21.15.7:0/3554247035 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f5954010720 con 0x563fe294af60 2024-06-10T01:16:17.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.249+0000 7f5979a3db80 1 -- 172.21.15.7:0/3554247035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f595409fcd0 con 0x563fe294af60 2024-06-10T01:16:17.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.249+0000 7f5947fff700 1 -- 172.21.15.7:0/3554247035 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f59540a4590 con 0x563fe294af60 2024-06-10T01:16:17.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.249+0000 7f5979a3db80 10 client.18039 fetch_fsmap learned FSMap version 38 2024-06-10T01:16:17.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.249+0000 7f5979a3db80 1 -- 172.21.15.7:0/3554247035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563fe295c980 con 0x563fe294af60 2024-06-10T01:16:17.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.250+0000 7f5947fff700 1 -- 172.21.15.7:0/3554247035 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f595409fcd0 con 0x563fe294af60 2024-06-10T01:16:17.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.250+0000 7f5947fff700 1 -- 172.21.15.7:0/3554247035 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f595409ad70 con 0x563fe294af60 2024-06-10T01:16:17.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.250+0000 7f5979a3db80 10 client.18039 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T01:16:17.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.250+0000 7f5979a3db80 10 client.18039 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:16:17.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.250+0000 7f5979a3db80 20 client.18039 populate_metadata read hostname 'smithi007' 2024-06-10T01:16:17.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.250+0000 7f5979a3db80 1 --2- 172.21.15.7:0/3554247035 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563fe295ee50 0x563fe2961240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:17.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.250+0000 7f5979a3db80 4 client.18039 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:16:17.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.250+0000 7f5979a3db80 1 -- 172.21.15.7:0/3554247035 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fe28a3d50 con 0x563fe295ee50 2024-06-10T01:16:17.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.250+0000 7f596cf56700 1 --2- 172.21.15.7:0/3554247035 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563fe295ee50 0x563fe2961240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:17.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.250+0000 7f596cf56700 1 --2- 172.21.15.7:0/3554247035 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563fe295ee50 0x563fe2961240 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=135031549e6cd563 server_cookie=3aaf35d81f341994 in_seq=0 out_seq=0 2024-06-10T01:16:17.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.250+0000 7f5947fff700 10 client.18039 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:17.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.251+0000 7f5947fff700 1 -- 172.21.15.7:0/3554247035 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563fe28a3d50 con 0x563fe295ee50 2024-06-10T01:16:17.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.251+0000 7f5947fff700 10 client.18039 handle_command_reply: tid=0 2024-06-10T01:16:17.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.251+0000 7f5979a3db80 1 client.18039 shutdown 2024-06-10T01:16:17.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.251+0000 7f5979a3db80 1 -- 172.21.15.7:0/3554247035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fe294af60 msgr2=0x563fe294e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:17.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.251+0000 7f5979a3db80 1 --2- 172.21.15.7:0/3554247035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fe294af60 0x563fe294e910 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f595400e9c0 tx=0x7f5954009ef0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.251+0000 7f5979a3db80 1 -- 172.21.15.7:0/3554247035 shutdown_connections 2024-06-10T01:16:17.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.251+0000 7f5979a3db80 1 --2- 172.21.15.7:0/3554247035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563fe294b880 0x563fe294ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.251+0000 7f5979a3db80 1 --2- 172.21.15.7:0/3554247035 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563fe295ee50 0x563fe2961240 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.251+0000 7f5979a3db80 1 --2- 172.21.15.7:0/3554247035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fe29457a0 0x563fe2951610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.251+0000 7f5979a3db80 1 --2- 172.21.15.7:0/3554247035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fe294af60 0x563fe294e910 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.251+0000 7f5947fff700 0 client.18039 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:17.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.251+0000 7f5979a3db80 1 -- 172.21.15.7:0/3554247035 >> 172.21.15.7:0/3554247035 conn(0x563fe2872080 msgr2=0x563fe293f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:17.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.251+0000 7f5979a3db80 1 -- 172.21.15.7:0/3554247035 shutdown_connections 2024-06-10T01:16:17.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.251+0000 7f5979a3db80 1 -- 172.21.15.7:0/3554247035 wait complete. 2024-06-10T01:16:17.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.251+0000 7f5979a3db80 20 client.18039 trim_cache size 0 max 16384 2024-06-10T01:16:17.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.275+0000 7f5979a3db80 1 Processor -- start 2024-06-10T01:16:17.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.275+0000 7f5979a3db80 1 -- start start 2024-06-10T01:16:17.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.275+0000 7f5979a3db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fe29457a0 0x563fe294d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:17.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.275+0000 7f5979a3db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fe294af60 0x563fe2949060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:17.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.275+0000 7f5979a3db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563fe294b880 0x563fe2951150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:17.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.275+0000 7f5979a3db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f59681b0cc0 con 0x563fe29457a0 2024-06-10T01:16:17.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.275+0000 7f5979a3db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f59681ae4f0 con 0x563fe294af60 2024-06-10T01:16:17.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.275+0000 7f5979a3db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f59681ac340 con 0x563fe294b880 2024-06-10T01:16:17.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.275+0000 7f596cf56700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fe29457a0 0x563fe294d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:17.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.275+0000 7f596d757700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563fe294b880 0x563fe2951150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:17.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.275+0000 7f5967fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fe294af60 0x563fe2949060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:17.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.275+0000 7f596cf56700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fe29457a0 0x563fe294d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47052/0 (socket says 172.21.15.7:47052) 2024-06-10T01:16:17.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.275+0000 7f596cf56700 1 -- 172.21.15.7:0/1091532893 learned_addr learned my addr 172.21.15.7:0/1091532893 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:17.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.276+0000 7f596cf56700 1 -- 172.21.15.7:0/1091532893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563fe294b880 msgr2=0x563fe2951150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:17.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.276+0000 7f596cf56700 1 --2- 172.21.15.7:0/1091532893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563fe294b880 0x563fe2951150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.276+0000 7f596cf56700 1 -- 172.21.15.7:0/1091532893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fe294af60 msgr2=0x563fe2949060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:17.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.276+0000 7f596cf56700 1 --2- 172.21.15.7:0/1091532893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fe294af60 0x563fe2949060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.276+0000 7f596cf56700 1 -- 172.21.15.7:0/1091532893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fe294ce80 con 0x563fe29457a0 2024-06-10T01:16:17.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.276+0000 7f596d757700 1 --2- 172.21.15.7:0/1091532893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563fe294b880 0x563fe2951150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:17.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.276+0000 7f5967fff700 1 --2- 172.21.15.7:0/1091532893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fe294af60 0x563fe2949060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:17.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.276+0000 7f596cf56700 1 --2- 172.21.15.7:0/1091532893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fe29457a0 0x563fe294d8d0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f595c009990 tx=0x7f595c023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e17ff115aabd9e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:17.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.276+0000 7f59527fc700 1 -- 172.21.15.7:0/1091532893 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c005370 con 0x563fe29457a0 2024-06-10T01:16:17.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.276+0000 7f59527fc700 1 -- 172.21.15.7:0/1091532893 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f595c005510 con 0x563fe29457a0 2024-06-10T01:16:17.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.276+0000 7f59527fc700 1 -- 172.21.15.7:0/1091532893 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c024670 con 0x563fe29457a0 2024-06-10T01:16:17.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.277+0000 7f5979a3db80 1 -- 172.21.15.7:0/1091532893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fe29457a0 msgr2=0x563fe294d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:17.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.277+0000 7f5979a3db80 1 --2- 172.21.15.7:0/1091532893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fe29457a0 0x563fe294d8d0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f595c009990 tx=0x7f595c023000 comp rx=0 tx=0).stop 2024-06-10T01:16:17.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.277+0000 7f5979a3db80 1 -- 172.21.15.7:0/1091532893 shutdown_connections 2024-06-10T01:16:17.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.277+0000 7f5979a3db80 1 --2- 172.21.15.7:0/1091532893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563fe294b880 0x563fe2951150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.277+0000 7f5979a3db80 1 --2- 172.21.15.7:0/1091532893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fe29457a0 0x563fe294d8d0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.277+0000 7f5979a3db80 1 --2- 172.21.15.7:0/1091532893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fe294af60 0x563fe2949060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.277+0000 7f5979a3db80 1 -- 172.21.15.7:0/1091532893 >> 172.21.15.7:0/1091532893 conn(0x563fe2872080 msgr2=0x563fe2949a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:17.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.277+0000 7f5979a3db80 1 -- 172.21.15.7:0/1091532893 shutdown_connections 2024-06-10T01:16:17.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.277+0000 7f5979a3db80 1 -- 172.21.15.7:0/1091532893 wait complete. 2024-06-10T01:16:17.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.277+0000 7f5979a3db80 1 -- start start 2024-06-10T01:16:17.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.277+0000 7f5979a3db80 1 Processor -- start 2024-06-10T01:16:17.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.278+0000 7f5979a3db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fe29457a0 0x563fe29621d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:17.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.278+0000 7f5979a3db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fe294af60 0x563fe2962710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:17.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.278+0000 7f5979a3db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563fe294b880 0x563fe2962c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:17.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.278+0000 7f5979a3db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f59681ac9b0 con 0x563fe294af60 2024-06-10T01:16:17.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.278+0000 7f5979a3db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f59681ade20 con 0x563fe29457a0 2024-06-10T01:16:17.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.278+0000 7f5979a3db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f59681ad080 con 0x563fe294b880 2024-06-10T01:16:17.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.278+0000 7f5979a3db80 10 client.0 _ll_register_callbacks cb 0x563fe282c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:16:17.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.278+0000 7f596cf56700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fe29457a0 0x563fe29621d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:17.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.278+0000 7f596cf56700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fe29457a0 0x563fe29621d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:36836/0 (socket says 172.21.15.7:36836) 2024-06-10T01:16:17.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.278+0000 7f596cf56700 1 -- 172.21.15.7:0/1028370960 learned_addr learned my addr 172.21.15.7:0/1028370960 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:17.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.278+0000 7f5967fff700 1 --2- 172.21.15.7:0/1028370960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fe294af60 0x563fe2962710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:17.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.278+0000 7f596d757700 1 --2- 172.21.15.7:0/1028370960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563fe294b880 0x563fe2962c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:17.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.278+0000 7f596cf56700 1 -- 172.21.15.7:0/1028370960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563fe294b880 msgr2=0x563fe2962c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:17.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.278+0000 7f596cf56700 1 --2- 172.21.15.7:0/1028370960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563fe294b880 0x563fe2962c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.278+0000 7f596cf56700 1 -- 172.21.15.7:0/1028370960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fe294af60 msgr2=0x563fe2962710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:17.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.278+0000 7f596cf56700 1 --2- 172.21.15.7:0/1028370960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fe294af60 0x563fe2962710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.278+0000 7f596cf56700 1 -- 172.21.15.7:0/1028370960 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fe29c8950 con 0x563fe29457a0 2024-06-10T01:16:17.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.279+0000 7f596cf56700 1 --2- 172.21.15.7:0/1028370960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fe29457a0 0x563fe29621d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f595c009730 tx=0x7f595c00b560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6022e228a3d1c410 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:17.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.279+0000 7f5947fff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:16:17.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.279+0000 7f5947fff700 1 -- 172.21.15.7:0/1028370960 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c044060 con 0x563fe29457a0 2024-06-10T01:16:17.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.279+0000 7f5947fff700 1 -- 172.21.15.7:0/1028370960 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f595c00f460 con 0x563fe29457a0 2024-06-10T01:16:17.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.279+0000 7f5979a3db80 1 -- 172.21.15.7:0/1028370960 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f595c005510 con 0x563fe29457a0 2024-06-10T01:16:17.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.279+0000 7f5947fff700 1 -- 172.21.15.7:0/1028370960 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f595c04b050 con 0x563fe29457a0 2024-06-10T01:16:17.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.279+0000 7f5979a3db80 10 client.27401 fetch_fsmap learned FSMap version 38 2024-06-10T01:16:17.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.279+0000 7f5979a3db80 1 -- 172.21.15.7:0/1028370960 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563fe29c9c40 con 0x563fe29457a0 2024-06-10T01:16:17.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.279+0000 7f5947fff700 1 -- 172.21.15.7:0/1028370960 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f595c005510 con 0x563fe29457a0 2024-06-10T01:16:17.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.279+0000 7f5947fff700 1 -- 172.21.15.7:0/1028370960 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f595c0248d0 con 0x563fe29457a0 2024-06-10T01:16:17.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.279+0000 7f5979a3db80 10 client.27401 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T01:16:17.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.279+0000 7f5979a3db80 10 client.27401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:16:17.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.279+0000 7f5979a3db80 20 client.27401 populate_metadata read hostname 'smithi007' 2024-06-10T01:16:17.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.279+0000 7f5979a3db80 1 --2- 172.21.15.7:0/1028370960 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563fe29ca230 0x563fe29cc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:17.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.279+0000 7f5979a3db80 4 client.27401 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:16:17.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.279+0000 7f5979a3db80 1 -- 172.21.15.7:0/1028370960 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563fe289e2d0 con 0x563fe29ca230 2024-06-10T01:16:17.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.280+0000 7f5967fff700 1 --2- 172.21.15.7:0/1028370960 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563fe29ca230 0x563fe29cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:17.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.280+0000 7f5947fff700 10 client.27401 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:17.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.280+0000 7f5967fff700 1 --2- 172.21.15.7:0/1028370960 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563fe29ca230 0x563fe29cc620 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3e80708dcaebb65 server_cookie=7184e57b51da44e3 in_seq=0 out_seq=0 2024-06-10T01:16:17.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.280+0000 7f5947fff700 1 -- 172.21.15.7:0/1028370960 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563fe289e2d0 con 0x563fe29ca230 2024-06-10T01:16:17.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.280+0000 7f5947fff700 10 client.27401 handle_command_reply: tid=0 2024-06-10T01:16:17.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.280+0000 7f5979a3db80 1 client.27401 shutdown 2024-06-10T01:16:17.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.280+0000 7f5979a3db80 1 -- 172.21.15.7:0/1028370960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fe29457a0 msgr2=0x563fe29621d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:17.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.280+0000 7f5979a3db80 1 --2- 172.21.15.7:0/1028370960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fe29457a0 0x563fe29621d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f595c009730 tx=0x7f595c00b560 comp rx=0 tx=0).stop 2024-06-10T01:16:17.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.280+0000 7f5979a3db80 1 -- 172.21.15.7:0/1028370960 shutdown_connections 2024-06-10T01:16:17.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.280+0000 7f5979a3db80 1 --2- 172.21.15.7:0/1028370960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563fe294b880 0x563fe2962c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.280+0000 7f5979a3db80 1 --2- 172.21.15.7:0/1028370960 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563fe29ca230 0x563fe29cc620 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.280+0000 7f5979a3db80 1 --2- 172.21.15.7:0/1028370960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fe29457a0 0x563fe29621d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.280+0000 7f5947fff700 0 client.27401 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:17.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.280+0000 7f5979a3db80 1 --2- 172.21.15.7:0/1028370960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fe294af60 0x563fe2962710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.280+0000 7f5979a3db80 1 -- 172.21.15.7:0/1028370960 >> 172.21.15.7:0/1028370960 conn(0x563fe2872080 msgr2=0x563fe294a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:17.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.281+0000 7f5979a3db80 1 -- 172.21.15.7:0/1028370960 shutdown_connections 2024-06-10T01:16:17.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.281+0000 7f5979a3db80 1 -- 172.21.15.7:0/1028370960 wait complete. 2024-06-10T01:16:17.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.281+0000 7f5979a3db80 20 client.27401 trim_cache size 0 max 16384 2024-06-10T01:16:17.281 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:16:17.281 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-10T01:16:17.281 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-10T01:16:17.282 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:16:17.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.281+0000 7f596f1ba700 1 -- 172.21.15.7:0/1223800891 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f59580a2ca0 msgr2=0x7f59580a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:17.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.281+0000 7f596f1ba700 1 --2- 172.21.15.7:0/1223800891 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f59580a2ca0 0x7f59580a5160 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f595c000c00 tx=0x7f595c005fb0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.281+0000 7f596f1ba700 1 -- 172.21.15.7:0/1223800891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f596806b560 msgr2=0x7f596819ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:17.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.281+0000 7f596f1ba700 1 --2- 172.21.15.7:0/1223800891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f596806b560 0x7f596819ed70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f595400e9f0 tx=0x7f595400ed00 comp rx=0 tx=0).stop 2024-06-10T01:16:17.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.281+0000 7f596f1ba700 1 -- 172.21.15.7:0/1223800891 shutdown_connections 2024-06-10T01:16:17.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.281+0000 7f596f1ba700 1 --2- 172.21.15.7:0/1223800891 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f59580a2ca0 0x7f59580a5160 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.281+0000 7f596f1ba700 1 --2- 172.21.15.7:0/1223800891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5968108560 0x7f59681989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.281+0000 7f596f1ba700 1 --2- 172.21.15.7:0/1223800891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f596806aba0 0x7f596819e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.281+0000 7f596f1ba700 1 --2- 172.21.15.7:0/1223800891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f596806b560 0x7f596819ed70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:17.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.281+0000 7f596f1ba700 1 -- 172.21.15.7:0/1223800891 >> 172.21.15.7:0/1223800891 conn(0x7f59680780f0 msgr2=0x7f5968100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:17.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.282+0000 7f596f1ba700 1 -- 172.21.15.7:0/1223800891 shutdown_connections 2024-06-10T01:16:17.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:17.282+0000 7f596f1ba700 1 -- 172.21.15.7:0/1223800891 wait complete. 2024-06-10T01:16:17.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:16 smithi078 ceph-mon[36683]: pgmap v740: 129 pgs: 129 active+clean; 127 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.3 MiB/s wr, 290 op/s 2024-06-10T01:16:17.747 INFO:tasks.workunit.client.0.smithi007.stdout: 1 749582 7.46 MB/sec execute 418 sec latency 60.968 ms 2024-06-10T01:16:18.677 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:16:18.678 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:16:18.747 INFO:tasks.workunit.client.0.smithi007.stdout: 1 751614 7.46 MB/sec execute 419 sec latency 20.727 ms 2024-06-10T01:16:19.001 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:16:19.012 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:18 smithi007 ceph-mon[32398]: pgmap v741: 129 pgs: 129 active+clean; 127 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 1.4 MiB/s wr, 280 op/s 2024-06-10T01:16:19.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:18 smithi175 ceph-mon[36636]: pgmap v741: 129 pgs: 129 active+clean; 127 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 1.4 MiB/s wr, 280 op/s 2024-06-10T01:16:19.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:18 smithi078 ceph-mon[36683]: pgmap v741: 129 pgs: 129 active+clean; 127 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 1.4 MiB/s wr, 280 op/s 2024-06-10T01:16:19.747 INFO:tasks.workunit.client.0.smithi007.stdout: 1 753190 7.47 MB/sec execute 420 sec latency 81.418 ms 2024-06-10T01:16:20.748 INFO:tasks.workunit.client.0.smithi007.stdout: 1 754973 7.47 MB/sec execute 421 sec latency 59.045 ms 2024-06-10T01:16:20.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.890+0000 7f63a1398700 1 -- 172.21.15.7:0/2203929290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f639c102530 msgr2=0x7f639c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:20.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.890+0000 7f63a1398700 1 --2- 172.21.15.7:0/2203929290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f639c102530 0x7f639c1029b0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f63900099c0 tx=0x7f6390009cd0 comp rx=0 tx=0).stop 2024-06-10T01:16:20.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.890+0000 7f63a1398700 1 -- 172.21.15.7:0/2203929290 shutdown_connections 2024-06-10T01:16:20.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.890+0000 7f63a1398700 1 --2- 172.21.15.7:0/2203929290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f639c102ef0 0x7f639c10f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:20.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.890+0000 7f63a1398700 1 --2- 172.21.15.7:0/2203929290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f639c1085d0 0x7f639c1089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:20.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.890+0000 7f63a1398700 1 --2- 172.21.15.7:0/2203929290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f639c102530 0x7f639c1029b0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:20.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.890+0000 7f63a1398700 1 -- 172.21.15.7:0/2203929290 >> 172.21.15.7:0/2203929290 conn(0x7f639c079640 msgr2=0x7f639c079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:20.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.890+0000 7f63a1398700 1 -- 172.21.15.7:0/2203929290 shutdown_connections 2024-06-10T01:16:20.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.890+0000 7f63a1398700 1 -- 172.21.15.7:0/2203929290 wait complete. 2024-06-10T01:16:20.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.890+0000 7f63a1398700 1 Processor -- start 2024-06-10T01:16:20.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.891+0000 7f63a1398700 1 -- start start 2024-06-10T01:16:20.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.891+0000 7f63a1398700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f639c102530 0x7f639c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:20.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.891+0000 7f63a1398700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f639c102ef0 0x7f639c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:20.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.891+0000 7f63a1398700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f639c1085d0 0x7f639c075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:20.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.891+0000 7f63a1398700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f639c112810 con 0x7f639c1085d0 2024-06-10T01:16:20.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.891+0000 7f63a1398700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f639c112690 con 0x7f639c102ef0 2024-06-10T01:16:20.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.891+0000 7f63a1398700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f639c112990 con 0x7f639c102530 2024-06-10T01:16:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.891+0000 7f639affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f639c102530 0x7f639c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.891+0000 7f639b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f639c1085d0 0x7f639c075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.891+0000 7f639affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f639c102530 0x7f639c074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47974/0 (socket says 172.21.15.7:47974) 2024-06-10T01:16:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.891+0000 7f639affd700 1 -- 172.21.15.7:0/2337450326 learned_addr learned my addr 172.21.15.7:0/2337450326 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.891+0000 7f639b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f639c1085d0 0x7f639c075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47094/0 (socket says 172.21.15.7:47094) 2024-06-10T01:16:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.891+0000 7f639a7fc700 1 --2- 172.21.15.7:0/2337450326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f639c102ef0 0x7f639c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.891+0000 7f639affd700 1 -- 172.21.15.7:0/2337450326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f639c102ef0 msgr2=0x7f639c074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.891+0000 7f639affd700 1 --2- 172.21.15.7:0/2337450326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f639c102ef0 0x7f639c074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.891+0000 7f639affd700 1 -- 172.21.15.7:0/2337450326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f639c1085d0 msgr2=0x7f639c075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.891+0000 7f639affd700 1 --2- 172.21.15.7:0/2337450326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f639c1085d0 0x7f639c075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.891+0000 7f639affd700 1 -- 172.21.15.7:0/2337450326 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f639c1a7150 con 0x7f639c102530 2024-06-10T01:16:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.891+0000 7f639b7fe700 1 --2- 172.21.15.7:0/2337450326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f639c1085d0 0x7f639c075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:20.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.892+0000 7f639affd700 1 --2- 172.21.15.7:0/2337450326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f639c102530 0x7f639c074790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f638c00e9a0 tx=0x7f638c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:20.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.892+0000 7f6383fff700 1 -- 172.21.15.7:0/2337450326 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c00ca10 con 0x7f639c102530 2024-06-10T01:16:20.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.892+0000 7f63a1398700 1 -- 172.21.15.7:0/2337450326 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f639c1a7440 con 0x7f639c102530 2024-06-10T01:16:20.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.892+0000 7f6383fff700 1 -- 172.21.15.7:0/2337450326 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f638c00cbb0 con 0x7f639c102530 2024-06-10T01:16:20.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.892+0000 7f6383fff700 1 -- 172.21.15.7:0/2337450326 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c018680 con 0x7f639c102530 2024-06-10T01:16:20.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.892+0000 7f63a1398700 1 -- 172.21.15.7:0/2337450326 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f639c1a7950 con 0x7f639c102530 2024-06-10T01:16:20.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.893+0000 7f63a1398700 1 -- 172.21.15.7:0/2337450326 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f639c103c40 con 0x7f639c102530 2024-06-10T01:16:20.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.894+0000 7f6383fff700 1 -- 172.21.15.7:0/2337450326 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f638c010430 con 0x7f639c102530 2024-06-10T01:16:20.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.895+0000 7f6383fff700 1 --2- 172.21.15.7:0/2337450326 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f63840a2c20 0x7f63840a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:20.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.895+0000 7f639a7fc700 1 --2- 172.21.15.7:0/2337450326 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f63840a2c20 0x7f63840a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:20.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.895+0000 7f6383fff700 1 -- 172.21.15.7:0/2337450326 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f638c0cf950 con 0x7f639c102530 2024-06-10T01:16:20.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.895+0000 7f639a7fc700 1 --2- 172.21.15.7:0/2337450326 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f63840a2c20 0x7f63840a50e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f639c076220 tx=0x7f6390005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:20.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:20.898+0000 7f6383fff700 1 -- 172.21.15.7:0/2337450326 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f638c09c4d0 con 0x7f639c102530 2024-06-10T01:16:20.892 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:20 smithi007 ceph-mon[32398]: pgmap v742: 129 pgs: 129 active+clean; 127 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 1.7 MiB/s wr, 311 op/s 2024-06-10T01:16:21.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:21.194+0000 7f63a1398700 1 -- 172.21.15.7:0/2337450326 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f639c02dca0 con 0x7f639c102530 2024-06-10T01:16:21.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:21.195+0000 7f6383fff700 1 -- 172.21.15.7:0/2337450326 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11857 (secure 0 0 0) 0x7f638c09bdd0 con 0x7f639c102530 2024-06-10T01:16:21.190 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:16:21.190 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:16:02.225658+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:16:21.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:21.198+0000 7f63a1398700 1 -- 172.21.15.7:0/2337450326 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f63840a2c20 msgr2=0x7f63840a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:21.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:21.198+0000 7f63a1398700 1 --2- 172.21.15.7:0/2337450326 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f63840a2c20 0x7f63840a50e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f639c076220 tx=0x7f6390005fb0 comp rx=0 tx=0).stop 2024-06-10T01:16:21.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:21.198+0000 7f63a1398700 1 -- 172.21.15.7:0/2337450326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f639c102530 msgr2=0x7f639c074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:21.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:21.198+0000 7f63a1398700 1 --2- 172.21.15.7:0/2337450326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f639c102530 0x7f639c074790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f638c00e9a0 tx=0x7f638c00ecb0 comp rx=0 tx=0).stop 2024-06-10T01:16:21.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:21.198+0000 7f63a1398700 1 -- 172.21.15.7:0/2337450326 shutdown_connections 2024-06-10T01:16:21.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:21.198+0000 7f63a1398700 1 --2- 172.21.15.7:0/2337450326 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f63840a2c20 0x7f63840a50e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:21.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:21.198+0000 7f63a1398700 1 --2- 172.21.15.7:0/2337450326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f639c1085d0 0x7f639c075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:21.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:21.198+0000 7f63a1398700 1 --2- 172.21.15.7:0/2337450326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f639c102530 0x7f639c074790 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:21.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:21.198+0000 7f63a1398700 1 --2- 172.21.15.7:0/2337450326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f639c102ef0 0x7f639c074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:21.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:21.198+0000 7f63a1398700 1 -- 172.21.15.7:0/2337450326 >> 172.21.15.7:0/2337450326 conn(0x7f639c079640 msgr2=0x7f639c101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:21.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:21.198+0000 7f63a1398700 1 -- 172.21.15.7:0/2337450326 shutdown_connections 2024-06-10T01:16:21.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:21.198+0000 7f63a1398700 1 -- 172.21.15.7:0/2337450326 wait complete. 2024-06-10T01:16:21.193 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 38 2024-06-10T01:16:21.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:20 smithi175 ceph-mon[36636]: pgmap v742: 129 pgs: 129 active+clean; 127 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 1.7 MiB/s wr, 311 op/s 2024-06-10T01:16:21.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:20 smithi078 ceph-mon[36683]: pgmap v742: 129 pgs: 129 active+clean; 127 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 1.7 MiB/s wr, 311 op/s 2024-06-10T01:16:21.748 INFO:tasks.workunit.client.0.smithi007.stdout: 1 756870 7.48 MB/sec execute 422 sec latency 10.373 ms 2024-06-10T01:16:21.855 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:16:22.176 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:16:22.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:21 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2337450326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:16:22.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:21 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2337450326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:16:22.335 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:21 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2337450326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:16:22.748 INFO:tasks.workunit.client.0.smithi007.stdout: 1 759098 7.49 MB/sec execute 423 sec latency 16.756 ms 2024-06-10T01:16:23.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:22 smithi175 ceph-mon[36636]: pgmap v743: 129 pgs: 129 active+clean; 131 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 1.9 MiB/s wr, 317 op/s 2024-06-10T01:16:23.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:22 smithi007 ceph-mon[32398]: pgmap v743: 129 pgs: 129 active+clean; 131 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 1.9 MiB/s wr, 317 op/s 2024-06-10T01:16:23.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:22 smithi078 ceph-mon[36683]: pgmap v743: 129 pgs: 129 active+clean; 131 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 1.9 MiB/s wr, 317 op/s 2024-06-10T01:16:23.748 INFO:tasks.workunit.client.0.smithi007.stdout: 1 760552 7.49 MB/sec execute 424 sec latency 76.747 ms 2024-06-10T01:16:24.748 INFO:tasks.workunit.client.0.smithi007.stdout: 1 762424 7.49 MB/sec execute 425 sec latency 77.549 ms 2024-06-10T01:16:25.130 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:24 smithi078 ceph-mon[36683]: pgmap v744: 129 pgs: 129 active+clean; 131 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 2.1 MiB/s wr, 298 op/s 2024-06-10T01:16:25.136 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:24 smithi007 ceph-mon[32398]: pgmap v744: 129 pgs: 129 active+clean; 131 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 2.1 MiB/s wr, 298 op/s 2024-06-10T01:16:25.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:24 smithi175 ceph-mon[36636]: pgmap v744: 129 pgs: 129 active+clean; 131 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 2.1 MiB/s wr, 298 op/s 2024-06-10T01:16:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.256+0000 7fa9bc528640 1 -- 172.21.15.7:0/3546874787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9b40759e0 msgr2=0x7fa9b4073e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.256+0000 7fa9bc528640 1 --2- 172.21.15.7:0/3546874787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9b40759e0 0x7fa9b4073e30 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac009c10 tx=0x7fa9ac02a710 comp rx=0 tx=0).stop 2024-06-10T01:16:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.256+0000 7fa9bc528640 1 -- 172.21.15.7:0/3546874787 shutdown_connections 2024-06-10T01:16:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.256+0000 7fa9bc528640 1 --2- 172.21.15.7:0/3546874787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9b410dff0 0x7fa9b41103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.256+0000 7fa9bc528640 1 --2- 172.21.15.7:0/3546874787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9b4074400 0x7fa9b4074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.256+0000 7fa9bc528640 1 --2- 172.21.15.7:0/3546874787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9b40759e0 0x7fa9b4073e30 secure :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac009c10 tx=0x7fa9ac02a710 comp rx=0 tx=0).stop 2024-06-10T01:16:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.256+0000 7fa9bc528640 1 -- 172.21.15.7:0/3546874787 >> 172.21.15.7:0/3546874787 conn(0x7fa9b40fcf90 msgr2=0x7fa9b40ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.257+0000 7fa9bc528640 1 -- 172.21.15.7:0/3546874787 shutdown_connections 2024-06-10T01:16:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.257+0000 7fa9bc528640 1 -- 172.21.15.7:0/3546874787 wait complete. 2024-06-10T01:16:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.258+0000 7fa9bc528640 1 Processor -- start 2024-06-10T01:16:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.258+0000 7fa9bc528640 1 -- start start 2024-06-10T01:16:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.258+0000 7fa9bc528640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9b4074400 0x7fa9b40705c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.258+0000 7fa9bc528640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9b410dff0 0x7fa9b4070b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.258+0000 7fa9bc528640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9b41ac3c0 0x7fa9b41ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.258+0000 7fa9bc528640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b4113cc0 con 0x7fa9b4074400 2024-06-10T01:16:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.258+0000 7fa9bc528640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b4113b40 con 0x7fa9b410dff0 2024-06-10T01:16:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.258+0000 7fa9bc528640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b4113e40 con 0x7fa9b41ac3c0 2024-06-10T01:16:25.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.259+0000 7fa9baac5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9b41ac3c0 0x7fa9b41ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.259+0000 7fa9baac5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9b41ac3c0 0x7fa9b41ae7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:44694/0 (socket says 172.21.15.7:44694) 2024-06-10T01:16:25.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.259+0000 7fa9baac5640 1 -- 172.21.15.7:0/3705177188 learned_addr learned my addr 172.21.15.7:0/3705177188 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:25.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.259+0000 7fa9b9ac3640 1 --2- 172.21.15.7:0/3705177188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9b410dff0 0x7fa9b4070b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.259+0000 7fa9baac5640 1 -- 172.21.15.7:0/3705177188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9b410dff0 msgr2=0x7fa9b4070b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:25.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.259+0000 7fa9baac5640 1 --2- 172.21.15.7:0/3705177188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9b410dff0 0x7fa9b4070b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.259+0000 7fa9baac5640 1 -- 172.21.15.7:0/3705177188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9b4074400 msgr2=0x7fa9b40705c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T01:16:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.259+0000 7fa9baac5640 1 --2- 172.21.15.7:0/3705177188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9b4074400 0x7fa9b40705c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.259+0000 7fa9baac5640 1 -- 172.21.15.7:0/3705177188 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b4105890 con 0x7fa9b41ac3c0 2024-06-10T01:16:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.260+0000 7fa9baac5640 1 --2- 172.21.15.7:0/3705177188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9b41ac3c0 0x7fa9b41ae7b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa9a800d750 tx=0x7fa9a800dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.260+0000 7fa9a77fe640 1 -- 172.21.15.7:0/3705177188 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8017070 con 0x7fa9b41ac3c0 2024-06-10T01:16:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.260+0000 7fa9bc528640 1 -- 172.21.15.7:0/3705177188 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b4105b80 con 0x7fa9b41ac3c0 2024-06-10T01:16:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.260+0000 7fa9bc528640 1 -- 172.21.15.7:0/3705177188 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b41060d0 con 0x7fa9b41ac3c0 2024-06-10T01:16:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.260+0000 7fa9a77fe640 1 -- 172.21.15.7:0/3705177188 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9a8005500 con 0x7fa9b41ac3c0 2024-06-10T01:16:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.260+0000 7fa9a77fe640 1 -- 172.21.15.7:0/3705177188 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8004e10 con 0x7fa9b41ac3c0 2024-06-10T01:16:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.263+0000 7fa9a77fe640 1 -- 172.21.15.7:0/3705177188 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa9a8005020 con 0x7fa9b41ac3c0 2024-06-10T01:16:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.263+0000 7fa9bf230740 1 Processor -- start 2024-06-10T01:16:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.263+0000 7fa9bf230740 1 -- start start 2024-06-10T01:16:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.264+0000 7fa9bf230740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55becf755010 0x55becf755430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.264+0000 7fa9bf230740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becf755a00 0x55becf75a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.264+0000 7fa9bf230740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55becf75ab70 0x55becf75d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.264+0000 7fa9bf230740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55becf6b84f0 con 0x55becf75ab70 2024-06-10T01:16:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.264+0000 7fa9bf230740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55becf656870 con 0x55becf755a00 2024-06-10T01:16:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.264+0000 7fa9bf230740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55becf6b44b0 con 0x55becf755010 2024-06-10T01:16:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.264+0000 7fa9a77fe640 1 --2- 172.21.15.7:0/3705177188 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa9a00a2cc0 0x7fa9a00a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.264+0000 7fa9ba2c4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becf755a00 0x55becf75a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.264+0000 7fa9b9ac3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55becf755010 0x55becf755430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.264+0000 7fa9ba2c4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becf755a00 0x55becf75a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:48286/0 (socket says 172.21.15.7:48286) 2024-06-10T01:16:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.264+0000 7fa9ba2c4640 1 -- 172.21.15.7:0/2633515731 learned_addr learned my addr 172.21.15.7:0/2633515731 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.264+0000 7fa9b9ac3640 1 --2- 172.21.15.7:0/2633515731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55becf75ab70 0x55becf75d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.264+0000 7fa9a77fe640 1 -- 172.21.15.7:0/3705177188 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa9a80ced10 con 0x7fa9b41ac3c0 2024-06-10T01:16:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.264+0000 7fa9ba2c4640 1 -- 172.21.15.7:0/2633515731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55becf755010 msgr2=0x55becf755430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.264+0000 7fa9ba2c4640 1 --2- 172.21.15.7:0/2633515731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55becf755010 0x55becf755430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.264+0000 7fa9baac5640 1 --2- 172.21.15.7:0/3705177188 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa9a00a2cc0 0x7fa9a00a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.264+0000 7fa9ba2c4640 1 -- 172.21.15.7:0/2633515731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55becf75ab70 msgr2=0x55becf75d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.264+0000 7fa9ba2c4640 1 --2- 172.21.15.7:0/2633515731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55becf75ab70 0x55becf75d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.264+0000 7fa9ba2c4640 1 -- 172.21.15.7:0/2633515731 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55becf6729b0 con 0x55becf755a00 2024-06-10T01:16:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.264+0000 7fa9ba2c4640 1 --2- 172.21.15.7:0/2633515731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becf755a00 0x55becf75a500 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac02ac80 tx=0x7fa9ac031440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53147666fc824a97 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.265+0000 7fa9baac5640 1 --2- 172.21.15.7:0/3705177188 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa9a00a2cc0 0x7fa9a00a5180 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa9a809b120 tx=0x7fa9a8005f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.265+0000 7fa9a4bfe640 1 -- 172.21.15.7:0/2633515731 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac03f030 con 0x55becf755a00 2024-06-10T01:16:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.265+0000 7fa9a4bfe640 1 -- 172.21.15.7:0/2633515731 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9ac0303a0 con 0x55becf755a00 2024-06-10T01:16:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.265+0000 7fa9a4bfe640 1 -- 172.21.15.7:0/2633515731 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac036040 con 0x55becf755a00 2024-06-10T01:16:25.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.265+0000 7fa9bf230740 1 -- 172.21.15.7:0/2633515731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becf755a00 msgr2=0x55becf75a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.265+0000 7fa9bf230740 1 --2- 172.21.15.7:0/2633515731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becf755a00 0x55becf75a500 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac02ac80 tx=0x7fa9ac031440 comp rx=0 tx=0).stop 2024-06-10T01:16:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.265+0000 7fa9bf230740 1 -- 172.21.15.7:0/2633515731 shutdown_connections 2024-06-10T01:16:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.265+0000 7fa9bf230740 1 --2- 172.21.15.7:0/2633515731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55becf75ab70 0x55becf75d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.265+0000 7fa9bf230740 1 --2- 172.21.15.7:0/2633515731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becf755a00 0x55becf75a500 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.265+0000 7fa9bf230740 1 --2- 172.21.15.7:0/2633515731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55becf755010 0x55becf755430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.265+0000 7fa9bf230740 1 -- 172.21.15.7:0/2633515731 >> 172.21.15.7:0/2633515731 conn(0x55becf750680 msgr2=0x55becf750a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.265+0000 7fa9bf230740 1 -- 172.21.15.7:0/2633515731 shutdown_connections 2024-06-10T01:16:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.266+0000 7fa9bf230740 1 -- 172.21.15.7:0/2633515731 wait complete. 2024-06-10T01:16:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.266+0000 7fa9bf230740 1 -- start start 2024-06-10T01:16:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.266+0000 7fa9bf230740 1 Processor -- start 2024-06-10T01:16:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.266+0000 7fa9bf230740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becf755010 0x55becf75f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.266+0000 7fa9bf230740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55becf755a00 0x55becf75f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.266+0000 7fa9bf230740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55becf75ab70 0x55becf766710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.266+0000 7fa9bf230740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55becf75dac0 con 0x55becf75ab70 2024-06-10T01:16:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.266+0000 7fa9bf230740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55becf641ae0 con 0x55becf755010 2024-06-10T01:16:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.266+0000 7fa9bf230740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55becf75dc40 con 0x55becf755a00 2024-06-10T01:16:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.266+0000 7fa9bf230740 10 client.0 _ll_register_callbacks cb 0x55becf67a9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:16:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.266+0000 7fa9b9ac3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becf755010 0x55becf75f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.266+0000 7fa9b9ac3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55becf75ab70 0x55becf766710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.266+0000 7fa9b9ac3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55becf75ab70 0x55becf766710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40034/0 (socket says 172.21.15.7:40034) 2024-06-10T01:16:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.266+0000 7fa9b9ac3640 1 -- 172.21.15.7:0/3058492337 learned_addr learned my addr 172.21.15.7:0/3058492337 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.266+0000 7fa9ba2c4640 1 --2- 172.21.15.7:0/3058492337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55becf755a00 0x55becf75f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.267+0000 7fa9b9ac3640 1 -- 172.21.15.7:0/3058492337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55becf755a00 msgr2=0x55becf75f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.267+0000 7fa9b9ac3640 1 --2- 172.21.15.7:0/3058492337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55becf755a00 0x55becf75f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.267+0000 7fa9b9ac3640 1 -- 172.21.15.7:0/3058492337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becf755010 msgr2=0x55becf75f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.267+0000 7fa9b9ac3640 1 --2- 172.21.15.7:0/3058492337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becf755010 0x55becf75f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.267+0000 7fa9b9ac3640 1 -- 172.21.15.7:0/3058492337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55becf3cc210 con 0x55becf75ab70 2024-06-10T01:16:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.267+0000 7fa9b9ac3640 1 --2- 172.21.15.7:0/3058492337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55becf75ab70 0x55becf766710 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0009690 tx=0x7fa9b0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ffbc2a3d361faee server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.267+0000 7fa99dffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:16:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.267+0000 7fa99dffb640 1 -- 172.21.15.7:0/3058492337 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0026060 con 0x55becf75ab70 2024-06-10T01:16:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.267+0000 7fa9bf230740 1 -- 172.21.15.7:0/3058492337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9ac0303a0 con 0x55becf75ab70 2024-06-10T01:16:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.267+0000 7fa99dffb640 1 -- 172.21.15.7:0/3058492337 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa9b0004260 con 0x55becf75ab70 2024-06-10T01:16:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.267+0000 7fa99dffb640 1 -- 172.21.15.7:0/3058492337 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fa9b002d050 con 0x55becf75ab70 2024-06-10T01:16:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.267+0000 7fa9bf230740 10 client.18060 fetch_fsmap learned FSMap version 38 2024-06-10T01:16:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.267+0000 7fa9bf230740 1 -- 172.21.15.7:0/3058492337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55becf498620 con 0x55becf75ab70 2024-06-10T01:16:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.267+0000 7fa99dffb640 1 -- 172.21.15.7:0/3058492337 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9ac0303a0 con 0x55becf75ab70 2024-06-10T01:16:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.267+0000 7fa99dffb640 1 -- 172.21.15.7:0/3058492337 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7fa9b0020040 con 0x55becf75ab70 2024-06-10T01:16:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.268+0000 7fa9bf230740 10 client.18060 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T01:16:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.268+0000 7fa9bf230740 10 client.18060 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:16:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.268+0000 7fa9bf230740 20 client.18060 populate_metadata read hostname 'smithi007' 2024-06-10T01:16:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.268+0000 7fa9bf230740 1 --2- 172.21.15.7:0/3058492337 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55becf76b060 0x55becf76d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.268+0000 7fa9bf230740 4 client.18060 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:16:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.268+0000 7fa9bf230740 1 -- 172.21.15.7:0/3058492337 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55becf6cbf10 con 0x55becf76b060 2024-06-10T01:16:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.268+0000 7fa9ba2c4640 1 --2- 172.21.15.7:0/3058492337 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55becf76b060 0x55becf76d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.268+0000 7fa9ba2c4640 1 --2- 172.21.15.7:0/3058492337 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55becf76b060 0x55becf76d4c0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2978d8e453ec0bbe server_cookie=26afc1371eada9e1 in_seq=0 out_seq=0 2024-06-10T01:16:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.268+0000 7fa99dffb640 10 client.18060 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.269+0000 7fa99dffb640 1 -- 172.21.15.7:0/3058492337 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55becf6cbf10 con 0x55becf76b060 2024-06-10T01:16:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.269+0000 7fa99dffb640 10 client.18060 handle_command_reply: tid=0 2024-06-10T01:16:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.269+0000 7fa9bf230740 1 client.18060 shutdown 2024-06-10T01:16:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.269+0000 7fa9bf230740 1 -- 172.21.15.7:0/3058492337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55becf75ab70 msgr2=0x55becf766710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.269+0000 7fa9bf230740 1 --2- 172.21.15.7:0/3058492337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55becf75ab70 0x55becf766710 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0009690 tx=0x7fa9b0005f90 comp rx=0 tx=0).stop 2024-06-10T01:16:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.269+0000 7fa9bf230740 1 -- 172.21.15.7:0/3058492337 shutdown_connections 2024-06-10T01:16:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.269+0000 7fa9bf230740 1 --2- 172.21.15.7:0/3058492337 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55becf76b060 0x55becf76d4c0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.269+0000 7fa9bf230740 1 --2- 172.21.15.7:0/3058492337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55becf75ab70 0x55becf766710 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.269+0000 7fa9bf230740 1 --2- 172.21.15.7:0/3058492337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55becf755a00 0x55becf75f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.269+0000 7fa9bf230740 1 --2- 172.21.15.7:0/3058492337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becf755010 0x55becf75f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.269+0000 7fa99dffb640 0 client.18060 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.269+0000 7fa9bf230740 1 -- 172.21.15.7:0/3058492337 >> 172.21.15.7:0/3058492337 conn(0x55becf750680 msgr2=0x55becf74f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.269+0000 7fa9bf230740 1 -- 172.21.15.7:0/3058492337 shutdown_connections 2024-06-10T01:16:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.269+0000 7fa9bf230740 1 -- 172.21.15.7:0/3058492337 wait complete. 2024-06-10T01:16:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.270+0000 7fa9bf230740 20 client.18060 trim_cache size 0 max 16384 2024-06-10T01:16:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.288+0000 7fa9bf230740 1 Processor -- start 2024-06-10T01:16:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.288+0000 7fa9bf230740 1 -- start start 2024-06-10T01:16:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.289+0000 7fa9bf230740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55becf755010 0x55becf75f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.289+0000 7fa9bf230740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55becf755a00 0x55becf75f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.289+0000 7fa9bf230740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becf75ab70 0x55becf7666a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.289+0000 7fa9bf230740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa998012470 con 0x55becf755a00 2024-06-10T01:16:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.289+0000 7fa9bf230740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa99800fce0 con 0x55becf75ab70 2024-06-10T01:16:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.289+0000 7fa9bf230740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa99800db30 con 0x55becf755010 2024-06-10T01:16:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.289+0000 7fa9ba2c4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55becf755a00 0x55becf75f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.289+0000 7fa9b9ac3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55becf755010 0x55becf75f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.289+0000 7fa9ba2c4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55becf755a00 0x55becf75f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40038/0 (socket says 172.21.15.7:40038) 2024-06-10T01:16:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.289+0000 7fa9ba2c4640 1 -- 172.21.15.7:0/3570537917 learned_addr learned my addr 172.21.15.7:0/3570537917 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.289+0000 7fa9b9ac3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55becf755010 0x55becf75f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:44734/0 (socket says 172.21.15.7:44734) 2024-06-10T01:16:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.289+0000 7fa9b9ac3640 1 --2- 172.21.15.7:0/3570537917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becf75ab70 0x55becf7666a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.289+0000 7fa9ba2c4640 1 -- 172.21.15.7:0/3570537917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55becf755010 msgr2=0x55becf75f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.289+0000 7fa9ba2c4640 1 --2- 172.21.15.7:0/3570537917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55becf755010 0x55becf75f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.289+0000 7fa9ba2c4640 1 -- 172.21.15.7:0/3570537917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becf75ab70 msgr2=0x55becf7666a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.289+0000 7fa9ba2c4640 1 --2- 172.21.15.7:0/3570537917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becf75ab70 0x55becf7666a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.289+0000 7fa9ba2c4640 1 -- 172.21.15.7:0/3570537917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55becf760190 con 0x55becf755a00 2024-06-10T01:16:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.289+0000 7fa9b9ac3640 1 --2- 172.21.15.7:0/3570537917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55becf755010 0x55becf75f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.290+0000 7fa9ba2c4640 1 --2- 172.21.15.7:0/3570537917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55becf755a00 0x55becf75f6d0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac03e040 tx=0x7fa9ac02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98f1ce15f288936 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.290+0000 7fa99ffff640 1 -- 172.21.15.7:0/3570537917 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac052070 con 0x55becf755a00 2024-06-10T01:16:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.290+0000 7fa99ffff640 1 -- 172.21.15.7:0/3570537917 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9ac040040 con 0x55becf755a00 2024-06-10T01:16:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.290+0000 7fa99ffff640 1 -- 172.21.15.7:0/3570537917 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac04d070 con 0x55becf755a00 2024-06-10T01:16:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.291+0000 7fa9bf230740 1 -- 172.21.15.7:0/3570537917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55becf755a00 msgr2=0x55becf75f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.291+0000 7fa9bf230740 1 --2- 172.21.15.7:0/3570537917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55becf755a00 0x55becf75f6d0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac03e040 tx=0x7fa9ac02a710 comp rx=0 tx=0).stop 2024-06-10T01:16:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.291+0000 7fa9bf230740 1 -- 172.21.15.7:0/3570537917 shutdown_connections 2024-06-10T01:16:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.291+0000 7fa9bf230740 1 --2- 172.21.15.7:0/3570537917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becf75ab70 0x55becf7666a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.291+0000 7fa9bf230740 1 --2- 172.21.15.7:0/3570537917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55becf755a00 0x55becf75f6d0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.291+0000 7fa9bf230740 1 --2- 172.21.15.7:0/3570537917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55becf755010 0x55becf75f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.291+0000 7fa9bf230740 1 -- 172.21.15.7:0/3570537917 >> 172.21.15.7:0/3570537917 conn(0x55becf750680 msgr2=0x55becf750a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.291+0000 7fa9bf230740 1 -- 172.21.15.7:0/3570537917 shutdown_connections 2024-06-10T01:16:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.291+0000 7fa9bf230740 1 -- 172.21.15.7:0/3570537917 wait complete. 2024-06-10T01:16:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.291+0000 7fa9bf230740 1 -- start start 2024-06-10T01:16:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.291+0000 7fa9bf230740 1 Processor -- start 2024-06-10T01:16:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.292+0000 7fa9bf230740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becf755010 0x55becf7632b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.292+0000 7fa9bf230740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55becf755a00 0x55becf7637f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.292+0000 7fa9bf230740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55becf75ab70 0x55becf7d3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.292+0000 7fa9bf230740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa99800e1a0 con 0x55becf75ab70 2024-06-10T01:16:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.292+0000 7fa9bf230740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa99800f610 con 0x55becf755010 2024-06-10T01:16:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.292+0000 7fa9bf230740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa99800e870 con 0x55becf755a00 2024-06-10T01:16:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.292+0000 7fa9bf230740 10 client.0 _ll_register_callbacks cb 0x7fa9b0009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:16:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.292+0000 7fa9b9ac3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becf755010 0x55becf7632b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.292+0000 7fa9b9ac3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55becf75ab70 0x55becf7d3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.292+0000 7fa9b9ac3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55becf75ab70 0x55becf7d3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40048/0 (socket says 172.21.15.7:40048) 2024-06-10T01:16:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.292+0000 7fa9b9ac3640 1 -- 172.21.15.7:0/4021612524 learned_addr learned my addr 172.21.15.7:0/4021612524 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.292+0000 7fa9ba2c4640 1 --2- 172.21.15.7:0/4021612524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55becf755a00 0x55becf7637f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.292+0000 7fa9b9ac3640 1 -- 172.21.15.7:0/4021612524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55becf755a00 msgr2=0x55becf7637f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.292+0000 7fa9b9ac3640 1 --2- 172.21.15.7:0/4021612524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55becf755a00 0x55becf7637f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.292+0000 7fa9b9ac3640 1 -- 172.21.15.7:0/4021612524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becf755010 msgr2=0x55becf7632b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.292+0000 7fa9b9ac3640 1 --2- 172.21.15.7:0/4021612524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becf755010 0x55becf7632b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.292+0000 7fa9b9ac3640 1 -- 172.21.15.7:0/4021612524 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55becf7668c0 con 0x55becf75ab70 2024-06-10T01:16:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.292+0000 7fa9b9ac3640 1 --2- 172.21.15.7:0/4021612524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55becf75ab70 0x55becf7d3d90 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0009660 tx=0x7fa9b0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c935d2e2b4a2718 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.292+0000 7fa99dffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:16:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.292+0000 7fa99dffb640 1 -- 172.21.15.7:0/4021612524 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0026030 con 0x55becf75ab70 2024-06-10T01:16:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.292+0000 7fa99dffb640 1 -- 172.21.15.7:0/4021612524 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa9b0004470 con 0x55becf75ab70 2024-06-10T01:16:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.293+0000 7fa9bf230740 1 -- 172.21.15.7:0/4021612524 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9ac040040 con 0x55becf75ab70 2024-06-10T01:16:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.293+0000 7fa99dffb640 1 -- 172.21.15.7:0/4021612524 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fa9b0040050 con 0x55becf75ab70 2024-06-10T01:16:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.293+0000 7fa9bf230740 10 client.18066 fetch_fsmap learned FSMap version 38 2024-06-10T01:16:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.293+0000 7fa9bf230740 1 -- 172.21.15.7:0/4021612524 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55becf7d5210 con 0x55becf75ab70 2024-06-10T01:16:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.293+0000 7fa99dffb640 1 -- 172.21.15.7:0/4021612524 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9ac040040 con 0x55becf75ab70 2024-06-10T01:16:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.293+0000 7fa99dffb640 1 -- 172.21.15.7:0/4021612524 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7fa9b0030040 con 0x55becf75ab70 2024-06-10T01:16:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.293+0000 7fa9bf230740 10 client.18066 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T01:16:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.293+0000 7fa9bf230740 10 client.18066 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:16:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.293+0000 7fa9bf230740 20 client.18066 populate_metadata read hostname 'smithi007' 2024-06-10T01:16:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.293+0000 7fa9bf230740 1 --2- 172.21.15.7:0/4021612524 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55becf76b060 0x55becf7d7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.293+0000 7fa9bf230740 4 client.18066 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:16:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.293+0000 7fa9bf230740 1 -- 172.21.15.7:0/4021612524 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55becf6c5c20 con 0x55becf76b060 2024-06-10T01:16:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.293+0000 7fa9ba2c4640 1 --2- 172.21.15.7:0/4021612524 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55becf76b060 0x55becf7d7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.294+0000 7fa99dffb640 10 client.18066 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.294+0000 7fa9ba2c4640 1 --2- 172.21.15.7:0/4021612524 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55becf76b060 0x55becf7d7c00 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4466d1f663613d6 server_cookie=1d0fbce574366a4d in_seq=0 out_seq=0 2024-06-10T01:16:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.376+0000 7fa99dffb640 1 -- 172.21.15.7:0/4021612524 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55becf6c5c20 con 0x55becf76b060 2024-06-10T01:16:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.376+0000 7fa99dffb640 10 client.18066 handle_command_reply: tid=0 2024-06-10T01:16:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.376+0000 7fa9bf230740 1 client.18066 shutdown 2024-06-10T01:16:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.376+0000 7fa9bf230740 1 -- 172.21.15.7:0/4021612524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55becf75ab70 msgr2=0x55becf7d3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.376+0000 7fa9bf230740 1 --2- 172.21.15.7:0/4021612524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55becf75ab70 0x55becf7d3d90 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0009660 tx=0x7fa9b0000bc0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.376+0000 7fa9bf230740 1 -- 172.21.15.7:0/4021612524 shutdown_connections 2024-06-10T01:16:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.376+0000 7fa9bf230740 1 --2- 172.21.15.7:0/4021612524 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55becf76b060 0x55becf7d7c00 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.376+0000 7fa99dffb640 0 client.18066 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.376+0000 7fa9bf230740 1 --2- 172.21.15.7:0/4021612524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55becf75ab70 0x55becf7d3d90 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.376+0000 7fa9bf230740 1 --2- 172.21.15.7:0/4021612524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55becf755a00 0x55becf7637f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.376+0000 7fa9bf230740 1 --2- 172.21.15.7:0/4021612524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becf755010 0x55becf7632b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.376+0000 7fa9bf230740 1 -- 172.21.15.7:0/4021612524 >> 172.21.15.7:0/4021612524 conn(0x55becf750680 msgr2=0x55becf75f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:25.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.376+0000 7fa9bf230740 1 -- 172.21.15.7:0/4021612524 shutdown_connections 2024-06-10T01:16:25.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.376+0000 7fa9bf230740 1 -- 172.21.15.7:0/4021612524 wait complete. 2024-06-10T01:16:25.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.377+0000 7fa9bf230740 20 client.18066 trim_cache size 0 max 16384 2024-06-10T01:16:25.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:16:25.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:16:25.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:16:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.377+0000 7fa9bc528640 1 -- 172.21.15.7:0/3705177188 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa9a00a2cc0 msgr2=0x7fa9a00a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.377+0000 7fa9bc528640 1 --2- 172.21.15.7:0/3705177188 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa9a00a2cc0 0x7fa9a00a5180 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa9a809b120 tx=0x7fa9a8005f10 comp rx=0 tx=0).stop 2024-06-10T01:16:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.377+0000 7fa9bc528640 1 -- 172.21.15.7:0/3705177188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9b41ac3c0 msgr2=0x7fa9b41ae7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.377+0000 7fa9bc528640 1 --2- 172.21.15.7:0/3705177188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9b41ac3c0 0x7fa9b41ae7b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa9a800d750 tx=0x7fa9a800dc10 comp rx=0 tx=0).stop 2024-06-10T01:16:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.377+0000 7fa9bc528640 1 -- 172.21.15.7:0/3705177188 shutdown_connections 2024-06-10T01:16:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.377+0000 7fa9bc528640 1 --2- 172.21.15.7:0/3705177188 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa9a00a2cc0 0x7fa9a00a5180 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.377+0000 7fa9bc528640 1 --2- 172.21.15.7:0/3705177188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9b41ac3c0 0x7fa9b41ae7b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.377+0000 7fa9bc528640 1 --2- 172.21.15.7:0/3705177188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9b410dff0 0x7fa9b4070b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.377+0000 7fa9bc528640 1 --2- 172.21.15.7:0/3705177188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9b4074400 0x7fa9b40705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.377+0000 7fa9bc528640 1 -- 172.21.15.7:0/3705177188 >> 172.21.15.7:0/3705177188 conn(0x7fa9b40fcf90 msgr2=0x7fa9b4101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.378+0000 7fa9bc528640 1 -- 172.21.15.7:0/3705177188 shutdown_connections 2024-06-10T01:16:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:25.378+0000 7fa9bc528640 1 -- 172.21.15.7:0/3705177188 wait complete. 2024-06-10T01:16:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.432+0000 7f952ee6d700 1 -- 172.21.15.7:0/3170677273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f952806a0e0 msgr2=0x7f9528109e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.432+0000 7f952ee6d700 1 --2- 172.21.15.7:0/3170677273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f952806a0e0 0x7f9528109e50 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f951c0098f0 tx=0x7f951c009c00 comp rx=0 tx=0).stop 2024-06-10T01:16:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.432+0000 7f952ee6d700 1 -- 172.21.15.7:0/3170677273 shutdown_connections 2024-06-10T01:16:25.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.432+0000 7f952ee6d700 1 --2- 172.21.15.7:0/3170677273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f952806a0e0 0x7f9528109e50 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.432+0000 7f952ee6d700 1 --2- 172.21.15.7:0/3170677273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f952810dee0 0x7f952810e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.432+0000 7f952ee6d700 1 --2- 172.21.15.7:0/3170677273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9528109490 0x7f9528109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.432+0000 7f952ee6d700 1 -- 172.21.15.7:0/3170677273 >> 172.21.15.7:0/3170677273 conn(0x7f952806d9e0 msgr2=0x7f952806ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:25.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.433+0000 7f952ee6d700 1 -- 172.21.15.7:0/3170677273 shutdown_connections 2024-06-10T01:16:25.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.433+0000 7f952ee6d700 1 -- 172.21.15.7:0/3170677273 wait complete. 2024-06-10T01:16:25.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.433+0000 7f952ee6d700 1 Processor -- start 2024-06-10T01:16:25.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.434+0000 7f952ee6d700 1 -- start start 2024-06-10T01:16:25.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.434+0000 7f952ee6d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f952806a0e0 0x7f95281177f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.434+0000 7f952ee6d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9528109490 0x7f9528117d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.434+0000 7f952ee6d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f952810dee0 0x7f9528118270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.434+0000 7f952ee6d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f952811aff0 con 0x7f9528109490 2024-06-10T01:16:25.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.434+0000 7f952ee6d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f952811ae70 con 0x7f952810dee0 2024-06-10T01:16:25.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.434+0000 7f952ee6d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f952811b170 con 0x7f952806a0e0 2024-06-10T01:16:25.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.434+0000 7f952d40a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f952810dee0 0x7f9528118270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.434+0000 7f952d40a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f952810dee0 0x7f9528118270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:48322/0 (socket says 172.21.15.7:48322) 2024-06-10T01:16:25.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.434+0000 7f952d40a700 1 -- 172.21.15.7:0/4001105248 learned_addr learned my addr 172.21.15.7:0/4001105248 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:25.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.434+0000 7f9527fff700 1 --2- 172.21.15.7:0/4001105248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9528109490 0x7f9528117d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.435+0000 7f952cc09700 1 --2- 172.21.15.7:0/4001105248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f952806a0e0 0x7f95281177f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.435+0000 7f952d40a700 1 -- 172.21.15.7:0/4001105248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f952806a0e0 msgr2=0x7f95281177f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:25.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.435+0000 7f952d40a700 1 --2- 172.21.15.7:0/4001105248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f952806a0e0 0x7f95281177f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.435+0000 7f952d40a700 1 -- 172.21.15.7:0/4001105248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9528109490 msgr2=0x7f9528117d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:25.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.435+0000 7f952d40a700 1 --2- 172.21.15.7:0/4001105248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9528109490 0x7f9528117d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.435+0000 7f952d40a700 1 -- 172.21.15.7:0/4001105248 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95281b3d70 con 0x7f952810dee0 2024-06-10T01:16:25.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.435+0000 7f952d40a700 1 --2- 172.21.15.7:0/4001105248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f952810dee0 0x7f9528118270 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f951c00c010 tx=0x7f951c005370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:25.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.436+0000 7f9525ffb700 1 -- 172.21.15.7:0/4001105248 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c00b760 con 0x7f952810dee0 2024-06-10T01:16:25.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.436+0000 7f9525ffb700 1 -- 172.21.15.7:0/4001105248 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f951c003680 con 0x7f952810dee0 2024-06-10T01:16:25.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.437+0000 7f9525ffb700 1 -- 172.21.15.7:0/4001105248 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c0218b0 con 0x7f952810dee0 2024-06-10T01:16:25.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.437+0000 7f952ee6d700 1 -- 172.21.15.7:0/4001105248 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95281b3fa0 con 0x7f952810dee0 2024-06-10T01:16:25.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.437+0000 7f952ee6d700 1 -- 172.21.15.7:0/4001105248 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95281b4340 con 0x7f952810dee0 2024-06-10T01:16:25.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.442+0000 7f9525ffb700 1 -- 172.21.15.7:0/4001105248 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f951c003830 con 0x7f952810dee0 2024-06-10T01:16:25.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.443+0000 7f9525ffb700 1 --2- 172.21.15.7:0/4001105248 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f95100a2bd0 0x7f95100a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.443+0000 7f9525ffb700 1 -- 172.21.15.7:0/4001105248 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f951c0d0200 con 0x7f952810dee0 2024-06-10T01:16:25.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.444+0000 7f952cc09700 1 --2- 172.21.15.7:0/4001105248 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f95100a2bd0 0x7f95100a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.444+0000 7f952cc09700 1 --2- 172.21.15.7:0/4001105248 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f95100a2bd0 0x7f95100a5090 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9518005fd0 tx=0x7f9518009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:25.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.445+0000 7f95396f0b80 1 Processor -- start 2024-06-10T01:16:25.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.445+0000 7f95396f0b80 1 -- start start 2024-06-10T01:16:25.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.445+0000 7f95396f0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a19e18640 0x562a19e18a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.445+0000 7f95396f0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a19e18f60 0x562a19e129e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.445+0000 7f95396f0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a19e12f20 0x562a19e13340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.446+0000 7f95396f0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562a19d64810 con 0x562a19e18640 2024-06-10T01:16:25.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.446+0000 7f95396f0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562a19d0c720 con 0x562a19e12f20 2024-06-10T01:16:25.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.446+0000 7f95396f0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562a19d607d0 con 0x562a19e18f60 2024-06-10T01:16:25.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.446+0000 7f952d40a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a19e12f20 0x562a19e13340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.446+0000 7f952d40a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a19e12f20 0x562a19e13340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:48336/0 (socket says 172.21.15.7:48336) 2024-06-10T01:16:25.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.446+0000 7f952d40a700 1 -- 172.21.15.7:0/4192161323 learned_addr learned my addr 172.21.15.7:0/4192161323 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:25.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.446+0000 7f952cc09700 1 --2- 172.21.15.7:0/4192161323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a19e18640 0x562a19e18a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.446+0000 7f9527fff700 1 --2- 172.21.15.7:0/4192161323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a19e18f60 0x562a19e129e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.447+0000 7f952d40a700 1 -- 172.21.15.7:0/4192161323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a19e18f60 msgr2=0x562a19e129e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:25.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.447+0000 7f952d40a700 1 --2- 172.21.15.7:0/4192161323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a19e18f60 0x562a19e129e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.447+0000 7f952d40a700 1 -- 172.21.15.7:0/4192161323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a19e18640 msgr2=0x562a19e18a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:25.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.447+0000 7f952d40a700 1 --2- 172.21.15.7:0/4192161323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a19e18640 0x562a19e18a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.447+0000 7f952d40a700 1 -- 172.21.15.7:0/4192161323 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a19d02f50 con 0x562a19e12f20 2024-06-10T01:16:25.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.448+0000 7f952d40a700 1 --2- 172.21.15.7:0/4192161323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a19e12f20 0x562a19e13340 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f951c0990e0 tx=0x7f951c09c5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99b6ccda79ad6fe0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:25.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.450+0000 7f950effd700 1 -- 172.21.15.7:0/4192161323 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c09c710 con 0x562a19e12f20 2024-06-10T01:16:25.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.450+0000 7f950effd700 1 -- 172.21.15.7:0/4192161323 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f951c0a0ac0 con 0x562a19e12f20 2024-06-10T01:16:25.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.450+0000 7f950effd700 1 -- 172.21.15.7:0/4192161323 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c0a4e60 con 0x562a19e12f20 2024-06-10T01:16:25.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.451+0000 7f95396f0b80 1 -- 172.21.15.7:0/4192161323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a19e12f20 msgr2=0x562a19e13340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:25.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.451+0000 7f95396f0b80 1 --2- 172.21.15.7:0/4192161323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a19e12f20 0x562a19e13340 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f951c0990e0 tx=0x7f951c09c5b0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.451+0000 7f95396f0b80 1 -- 172.21.15.7:0/4192161323 shutdown_connections 2024-06-10T01:16:25.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.451+0000 7f95396f0b80 1 --2- 172.21.15.7:0/4192161323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a19e12f20 0x562a19e13340 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.451+0000 7f95396f0b80 1 --2- 172.21.15.7:0/4192161323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a19e18640 0x562a19e18a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.451+0000 7f95396f0b80 1 --2- 172.21.15.7:0/4192161323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a19e18f60 0x562a19e129e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.451+0000 7f95396f0b80 1 -- 172.21.15.7:0/4192161323 >> 172.21.15.7:0/4192161323 conn(0x562a19d3dcc0 msgr2=0x562a19e0d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:25.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.453+0000 7f95396f0b80 1 -- 172.21.15.7:0/4192161323 shutdown_connections 2024-06-10T01:16:25.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.453+0000 7f95396f0b80 1 -- 172.21.15.7:0/4192161323 wait complete. 2024-06-10T01:16:25.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.453+0000 7f95396f0b80 1 -- start start 2024-06-10T01:16:25.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.453+0000 7f95396f0b80 1 Processor -- start 2024-06-10T01:16:25.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.454+0000 7f95396f0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a19e12f20 0x562a19e1ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.454+0000 7f95396f0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a19e18640 0x562a19e1c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.454+0000 7f95396f0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a19e18f60 0x562a19e1c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.454+0000 7f95396f0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562a19d0c720 con 0x562a19e18640 2024-06-10T01:16:25.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.454+0000 7f95396f0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562a19d04bf0 con 0x562a19e18f60 2024-06-10T01:16:25.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.454+0000 7f95396f0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562a19e1f6f0 con 0x562a19e12f20 2024-06-10T01:16:25.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.454+0000 7f95396f0b80 10 client.0 _ll_register_callbacks cb 0x562a19cfe920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:16:25.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.454+0000 7f952d40a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a19e18f60 0x562a19e1c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.454+0000 7f952d40a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a19e18f60 0x562a19e1c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:48340/0 (socket says 172.21.15.7:48340) 2024-06-10T01:16:25.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.454+0000 7f952d40a700 1 -- 172.21.15.7:0/4015224275 learned_addr learned my addr 172.21.15.7:0/4015224275 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:25.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.455+0000 7f952d40a700 1 -- 172.21.15.7:0/4015224275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a19e12f20 msgr2=0x562a19e1ed30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:16:25.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.455+0000 7f952d40a700 1 --2- 172.21.15.7:0/4015224275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a19e12f20 0x562a19e1ed30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.455+0000 7f952d40a700 1 -- 172.21.15.7:0/4015224275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a19e18640 msgr2=0x562a19e1c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:25.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.455+0000 7f952d40a700 1 --2- 172.21.15.7:0/4015224275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a19e18640 0x562a19e1c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.455+0000 7f952d40a700 1 -- 172.21.15.7:0/4015224275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a19cb38d0 con 0x562a19e18f60 2024-06-10T01:16:25.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.455+0000 7f952d40a700 1 --2- 172.21.15.7:0/4015224275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a19e18f60 0x562a19e1c5c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f951c098b70 tx=0x7f951c098390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f82f0f13d26b34a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:25.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.455+0000 7f9507fff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:16:25.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.455+0000 7f9507fff700 1 -- 172.21.15.7:0/4015224275 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c0a4e60 con 0x562a19e18f60 2024-06-10T01:16:25.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.456+0000 7f9507fff700 1 -- 172.21.15.7:0/4015224275 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f951c0a5000 con 0x562a19e18f60 2024-06-10T01:16:25.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.456+0000 7f95396f0b80 1 -- 172.21.15.7:0/4015224275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f951c0a0ac0 con 0x562a19e18f60 2024-06-10T01:16:25.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.456+0000 7f9507fff700 1 -- 172.21.15.7:0/4015224275 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f951c0b0ba0 con 0x562a19e18f60 2024-06-10T01:16:25.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.456+0000 7f95396f0b80 10 client.27437 fetch_fsmap learned FSMap version 38 2024-06-10T01:16:25.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.456+0000 7f95396f0b80 1 -- 172.21.15.7:0/4015224275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x562a19e29f00 con 0x562a19e18f60 2024-06-10T01:16:25.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.456+0000 7f9507fff700 1 -- 172.21.15.7:0/4015224275 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f951c0a0ac0 con 0x562a19e18f60 2024-06-10T01:16:25.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.456+0000 7f9507fff700 1 -- 172.21.15.7:0/4015224275 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f951c029060 con 0x562a19e18f60 2024-06-10T01:16:25.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.456+0000 7f95396f0b80 10 client.27437 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T01:16:25.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.456+0000 7f95396f0b80 10 client.27437 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:16:25.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.456+0000 7f95396f0b80 20 client.27437 populate_metadata read hostname 'smithi007' 2024-06-10T01:16:25.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.456+0000 7f95396f0b80 1 --2- 172.21.15.7:0/4015224275 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x562a19e2c250 0x562a19e2e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.456+0000 7f95396f0b80 4 client.27437 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:16:25.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.456+0000 7f95396f0b80 1 -- 172.21.15.7:0/4015224275 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a19d71d10 con 0x562a19e2c250 2024-06-10T01:16:25.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.457+0000 7f952cc09700 1 --2- 172.21.15.7:0/4015224275 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x562a19e2c250 0x562a19e2e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.457+0000 7f952cc09700 1 --2- 172.21.15.7:0/4015224275 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x562a19e2c250 0x562a19e2e660 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cb208ad8de788e93 server_cookie=7ff970d322d8a327 in_seq=0 out_seq=0 2024-06-10T01:16:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.457+0000 7f9507fff700 10 client.27437 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:16:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.458+0000 7f9507fff700 1 -- 172.21.15.7:0/4015224275 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562a19d71d10 con 0x562a19e2c250 2024-06-10T01:16:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.458+0000 7f9507fff700 10 client.27437 handle_command_reply: tid=0 2024-06-10T01:16:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.458+0000 7f95396f0b80 1 client.27437 shutdown 2024-06-10T01:16:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.458+0000 7f95396f0b80 1 -- 172.21.15.7:0/4015224275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a19e18f60 msgr2=0x562a19e1c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.458+0000 7f95396f0b80 1 --2- 172.21.15.7:0/4015224275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a19e18f60 0x562a19e1c5c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f951c098b70 tx=0x7f951c098390 comp rx=0 tx=0).stop 2024-06-10T01:16:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.458+0000 7f95396f0b80 1 -- 172.21.15.7:0/4015224275 shutdown_connections 2024-06-10T01:16:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.458+0000 7f95396f0b80 1 --2- 172.21.15.7:0/4015224275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a19e18f60 0x562a19e1c5c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.458+0000 7f95396f0b80 1 --2- 172.21.15.7:0/4015224275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a19e12f20 0x562a19e1ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.458+0000 7f95396f0b80 1 --2- 172.21.15.7:0/4015224275 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x562a19e2c250 0x562a19e2e660 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.458+0000 7f95396f0b80 1 --2- 172.21.15.7:0/4015224275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a19e18640 0x562a19e1c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.458+0000 7f95396f0b80 1 -- 172.21.15.7:0/4015224275 >> 172.21.15.7:0/4015224275 conn(0x562a19d3dcc0 msgr2=0x562a19e119d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:25.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.458+0000 7f9507fff700 0 client.27437 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:16:25.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.458+0000 7f95396f0b80 1 -- 172.21.15.7:0/4015224275 shutdown_connections 2024-06-10T01:16:25.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.458+0000 7f95396f0b80 1 -- 172.21.15.7:0/4015224275 wait complete. 2024-06-10T01:16:25.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.458+0000 7f95396f0b80 20 client.27437 trim_cache size 0 max 16384 2024-06-10T01:16:25.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.475+0000 7f95396f0b80 1 Processor -- start 2024-06-10T01:16:25.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.475+0000 7f95396f0b80 1 -- start start 2024-06-10T01:16:25.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.475+0000 7f95396f0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a19e12f20 0x562a19e129e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.475+0000 7f95396f0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a19e18640 0x562a19e167d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.475+0000 7f95396f0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a19e18f60 0x562a19e1c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.475+0000 7f95396f0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f95281bf670 con 0x562a19e12f20 2024-06-10T01:16:25.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.475+0000 7f95396f0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f95281bcea0 con 0x562a19e18640 2024-06-10T01:16:25.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.475+0000 7f95396f0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f95281bacf0 con 0x562a19e18f60 2024-06-10T01:16:25.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.475+0000 7f9527fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a19e18640 0x562a19e167d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.475+0000 7f952cc09700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a19e12f20 0x562a19e129e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.475+0000 7f9527fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a19e18640 0x562a19e167d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:48344/0 (socket says 172.21.15.7:48344) 2024-06-10T01:16:25.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.475+0000 7f952cc09700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a19e12f20 0x562a19e129e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40082/0 (socket says 172.21.15.7:40082) 2024-06-10T01:16:25.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.475+0000 7f9527fff700 1 -- 172.21.15.7:0/1519251149 learned_addr learned my addr 172.21.15.7:0/1519251149 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:25.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.476+0000 7f952d40a700 1 --2- 172.21.15.7:0/1519251149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a19e18f60 0x562a19e1c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.476+0000 7f9527fff700 1 -- 172.21.15.7:0/1519251149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a19e18f60 msgr2=0x562a19e1c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:25.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.476+0000 7f9527fff700 1 --2- 172.21.15.7:0/1519251149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a19e18f60 0x562a19e1c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.476+0000 7f9527fff700 1 -- 172.21.15.7:0/1519251149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a19e12f20 msgr2=0x562a19e129e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:25.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.476+0000 7f9527fff700 1 --2- 172.21.15.7:0/1519251149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a19e12f20 0x562a19e129e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.476+0000 7f9527fff700 1 -- 172.21.15.7:0/1519251149 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a19e0e520 con 0x562a19e18640 2024-06-10T01:16:25.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.476+0000 7f952cc09700 1 --2- 172.21.15.7:0/1519251149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a19e12f20 0x562a19e129e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:25.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.476+0000 7f9527fff700 1 --2- 172.21.15.7:0/1519251149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a19e18640 0x562a19e167d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9520009430 tx=0x7f9520009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12dd94b957283f20 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:25.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.476+0000 7f950e7fc700 1 -- 172.21.15.7:0/1519251149 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9520003c00 con 0x562a19e18640 2024-06-10T01:16:25.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.476+0000 7f950e7fc700 1 -- 172.21.15.7:0/1519251149 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9520007e50 con 0x562a19e18640 2024-06-10T01:16:25.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.476+0000 7f950e7fc700 1 -- 172.21.15.7:0/1519251149 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9520005030 con 0x562a19e18640 2024-06-10T01:16:25.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.477+0000 7f95396f0b80 1 -- 172.21.15.7:0/1519251149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a19e18640 msgr2=0x562a19e167d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:25.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.477+0000 7f95396f0b80 1 --2- 172.21.15.7:0/1519251149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a19e18640 0x562a19e167d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9520009430 tx=0x7f9520009920 comp rx=0 tx=0).stop 2024-06-10T01:16:25.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.477+0000 7f95396f0b80 1 -- 172.21.15.7:0/1519251149 shutdown_connections 2024-06-10T01:16:25.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.477+0000 7f95396f0b80 1 --2- 172.21.15.7:0/1519251149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a19e18f60 0x562a19e1c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.477+0000 7f95396f0b80 1 --2- 172.21.15.7:0/1519251149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a19e12f20 0x562a19e129e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.477+0000 7f95396f0b80 1 --2- 172.21.15.7:0/1519251149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a19e18640 0x562a19e167d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.477+0000 7f95396f0b80 1 -- 172.21.15.7:0/1519251149 >> 172.21.15.7:0/1519251149 conn(0x562a19d3dcc0 msgr2=0x562a19e160c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:25.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.477+0000 7f95396f0b80 1 -- 172.21.15.7:0/1519251149 shutdown_connections 2024-06-10T01:16:25.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.477+0000 7f95396f0b80 1 -- 172.21.15.7:0/1519251149 wait complete. 2024-06-10T01:16:25.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.477+0000 7f95396f0b80 1 -- start start 2024-06-10T01:16:25.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.478+0000 7f95396f0b80 1 Processor -- start 2024-06-10T01:16:25.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.478+0000 7f95396f0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a19e12f20 0x562a19e24d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.478+0000 7f95396f0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a19e18640 0x562a19e252b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.478+0000 7f95396f0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a19e18f60 0x562a19e33720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.478+0000 7f95396f0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f95281bb360 con 0x562a19e18640 2024-06-10T01:16:25.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.478+0000 7f95396f0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f95281bc7d0 con 0x562a19e12f20 2024-06-10T01:16:25.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.478+0000 7f9527fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a19e18640 0x562a19e252b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.478+0000 7f9527fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a19e18640 0x562a19e252b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40096/0 (socket says 172.21.15.7:40096) 2024-06-10T01:16:25.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.478+0000 7f9527fff700 1 -- 172.21.15.7:0/695287588 learned_addr learned my addr 172.21.15.7:0/695287588 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:25.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.478+0000 7f95396f0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f95281bba30 con 0x562a19e18f60 2024-06-10T01:16:25.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.478+0000 7f952d40a700 1 --2- 172.21.15.7:0/695287588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a19e18f60 0x562a19e33720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.478+0000 7f952cc09700 1 --2- 172.21.15.7:0/695287588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a19e12f20 0x562a19e24d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.478+0000 7f95396f0b80 10 client.0 _ll_register_callbacks cb 0x562a19cfe920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:16:25.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.479+0000 7f952d40a700 1 -- 172.21.15.7:0/695287588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a19e12f20 msgr2=0x562a19e24d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:25.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.479+0000 7f952d40a700 1 --2- 172.21.15.7:0/695287588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a19e12f20 0x562a19e24d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.479+0000 7f952d40a700 1 -- 172.21.15.7:0/695287588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a19e18640 msgr2=0x562a19e252b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:25.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.479+0000 7f952d40a700 1 --2- 172.21.15.7:0/695287588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a19e18640 0x562a19e252b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.479+0000 7f952d40a700 1 -- 172.21.15.7:0/695287588 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a19e33f60 con 0x562a19e18f60 2024-06-10T01:16:25.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.479+0000 7f9527fff700 1 --2- 172.21.15.7:0/695287588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a19e18640 0x562a19e252b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:25.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.479+0000 7f952d40a700 1 --2- 172.21.15.7:0/695287588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a19e18f60 0x562a19e33720 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f951c003fd0 tx=0x7f951c0092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=896419dbf02d6611 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:25.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.479+0000 7f9507fff700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:16:25.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.479+0000 7f9507fff700 1 -- 172.21.15.7:0/695287588 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c09c4e0 con 0x562a19e18f60 2024-06-10T01:16:25.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.479+0000 7f95396f0b80 1 -- 172.21.15.7:0/695287588 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9520007e50 con 0x562a19e18f60 2024-06-10T01:16:25.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.479+0000 7f9507fff700 1 -- 172.21.15.7:0/695287588 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f951c0a4e60 con 0x562a19e18f60 2024-06-10T01:16:25.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.479+0000 7f9507fff700 1 -- 172.21.15.7:0/695287588 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f951c0a0ac0 con 0x562a19e18f60 2024-06-10T01:16:25.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.479+0000 7f95396f0b80 10 client.27460 fetch_fsmap learned FSMap version 38 2024-06-10T01:16:25.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.479+0000 7f95396f0b80 1 -- 172.21.15.7:0/695287588 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x562a19e33f60 con 0x562a19e18f60 2024-06-10T01:16:25.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.479+0000 7f9507fff700 1 -- 172.21.15.7:0/695287588 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9520007e50 con 0x562a19e18f60 2024-06-10T01:16:25.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.479+0000 7f9507fff700 1 -- 172.21.15.7:0/695287588 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f951c01f020 con 0x562a19e18f60 2024-06-10T01:16:25.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.480+0000 7f95396f0b80 10 client.27460 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T01:16:25.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.480+0000 7f95396f0b80 10 client.27460 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:16:25.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.480+0000 7f95396f0b80 20 client.27460 populate_metadata read hostname 'smithi007' 2024-06-10T01:16:25.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.480+0000 7f95396f0b80 1 --2- 172.21.15.7:0/695287588 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x562a19e96d50 0x562a19e99170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:25.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.480+0000 7f95396f0b80 4 client.27460 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:16:25.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.480+0000 7f95396f0b80 1 -- 172.21.15.7:0/695287588 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562a19d6c290 con 0x562a19e96d50 2024-06-10T01:16:25.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.480+0000 7f952cc09700 1 --2- 172.21.15.7:0/695287588 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x562a19e96d50 0x562a19e99170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:25.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.480+0000 7f952cc09700 1 --2- 172.21.15.7:0/695287588 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x562a19e96d50 0x562a19e99170 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=11b8e22f3b300410 server_cookie=1bed2575ecb89468 in_seq=0 out_seq=0 2024-06-10T01:16:25.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.480+0000 7f9507fff700 10 client.27460 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:16:25.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.480+0000 7f9507fff700 1 -- 172.21.15.7:0/695287588 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562a19d6c290 con 0x562a19e96d50 2024-06-10T01:16:25.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.480+0000 7f9507fff700 10 client.27460 handle_command_reply: tid=0 2024-06-10T01:16:25.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.480+0000 7f95396f0b80 1 client.27460 shutdown 2024-06-10T01:16:25.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.480+0000 7f95396f0b80 1 -- 172.21.15.7:0/695287588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a19e18f60 msgr2=0x562a19e33720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:25.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.480+0000 7f95396f0b80 1 --2- 172.21.15.7:0/695287588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a19e18f60 0x562a19e33720 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f951c003fd0 tx=0x7f951c0092d0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.481+0000 7f95396f0b80 1 -- 172.21.15.7:0/695287588 shutdown_connections 2024-06-10T01:16:25.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.481+0000 7f95396f0b80 1 --2- 172.21.15.7:0/695287588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a19e18f60 0x562a19e33720 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.481+0000 7f95396f0b80 1 --2- 172.21.15.7:0/695287588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a19e12f20 0x562a19e24d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.481+0000 7f95396f0b80 1 --2- 172.21.15.7:0/695287588 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x562a19e96d50 0x562a19e99170 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.481+0000 7f95396f0b80 1 --2- 172.21.15.7:0/695287588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a19e18640 0x562a19e252b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.481+0000 7f95396f0b80 1 -- 172.21.15.7:0/695287588 >> 172.21.15.7:0/695287588 conn(0x562a19d3dcc0 msgr2=0x562a19e1c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:25.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.481+0000 7f9507fff700 0 client.27460 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:16:25.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.481+0000 7f95396f0b80 1 -- 172.21.15.7:0/695287588 shutdown_connections 2024-06-10T01:16:25.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.481+0000 7f95396f0b80 1 -- 172.21.15.7:0/695287588 wait complete. 2024-06-10T01:16:25.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.481+0000 7f95396f0b80 20 client.27460 trim_cache size 0 max 16384 2024-06-10T01:16:25.481 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:16:25.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.481+0000 7f952ee6d700 1 -- 172.21.15.7:0/4001105248 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f95100a2bd0 msgr2=0x7f95100a5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:25.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.481+0000 7f952ee6d700 1 --2- 172.21.15.7:0/4001105248 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f95100a2bd0 0x7f95100a5090 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9518005fd0 tx=0x7f9518009310 comp rx=0 tx=0).stop 2024-06-10T01:16:25.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.481+0000 7f952ee6d700 1 -- 172.21.15.7:0/4001105248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f952810dee0 msgr2=0x7f9528118270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:25.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.481+0000 7f952ee6d700 1 --2- 172.21.15.7:0/4001105248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f952810dee0 0x7f9528118270 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f951c00c010 tx=0x7f951c005370 comp rx=0 tx=0).stop 2024-06-10T01:16:25.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.482+0000 7f952ee6d700 1 -- 172.21.15.7:0/4001105248 shutdown_connections 2024-06-10T01:16:25.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.482+0000 7f952ee6d700 1 --2- 172.21.15.7:0/4001105248 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f95100a2bd0 0x7f95100a5090 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.482+0000 7f952ee6d700 1 --2- 172.21.15.7:0/4001105248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f952810dee0 0x7f9528118270 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.482+0000 7f952ee6d700 1 --2- 172.21.15.7:0/4001105248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f952806a0e0 0x7f95281177f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.482+0000 7f952ee6d700 1 --2- 172.21.15.7:0/4001105248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9528109490 0x7f9528117d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:25.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.482+0000 7f952ee6d700 1 -- 172.21.15.7:0/4001105248 >> 172.21.15.7:0/4001105248 conn(0x7f952806d9e0 msgr2=0x7f952810cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:25.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.482+0000 7f952ee6d700 1 -- 172.21.15.7:0/4001105248 shutdown_connections 2024-06-10T01:16:25.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:25.482+0000 7f952ee6d700 1 -- 172.21.15.7:0/4001105248 wait complete. 2024-06-10T01:16:25.748 INFO:tasks.workunit.client.0.smithi007.stdout: 1 763831 7.50 MB/sec execute 426 sec latency 51.987 ms 2024-06-10T01:16:26.492 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:16:26.748 INFO:tasks.workunit.client.0.smithi007.stdout: 1 765858 7.50 MB/sec execute 427 sec latency 16.239 ms 2024-06-10T01:16:26.764 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:16:27.167 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:26 smithi007 ceph-mon[32398]: pgmap v745: 129 pgs: 129 active+clean; 132 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 334 op/s 2024-06-10T01:16:27.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:26 smithi175 ceph-mon[36636]: pgmap v745: 129 pgs: 129 active+clean; 132 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 334 op/s 2024-06-10T01:16:27.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:26 smithi078 ceph-mon[36683]: pgmap v745: 129 pgs: 129 active+clean; 132 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 334 op/s 2024-06-10T01:16:27.748 INFO:tasks.workunit.client.0.smithi007.stdout: 1 767623 7.51 MB/sec execute 428 sec latency 12.417 ms 2024-06-10T01:16:28.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.510+0000 7f8478f5e700 1 -- 172.21.15.7:0/4219322692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84741084f0 msgr2=0x7f84741088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:28.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.510+0000 7f8478f5e700 1 --2- 172.21.15.7:0/4219322692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84741084f0 0x7f84741088d0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f8464009990 tx=0x7f8464009ca0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.510+0000 7f8478f5e700 1 -- 172.21.15.7:0/4219322692 shutdown_connections 2024-06-10T01:16:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.510+0000 7f8478f5e700 1 --2- 172.21.15.7:0/4219322692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f847406b520 0x7f847410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.510+0000 7f8478f5e700 1 --2- 172.21.15.7:0/4219322692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84741084f0 0x7f84741088d0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.510+0000 7f8478f5e700 1 --2- 172.21.15.7:0/4219322692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f847406ab60 0x7f847406afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.510+0000 7f8478f5e700 1 -- 172.21.15.7:0/4219322692 >> 172.21.15.7:0/4219322692 conn(0x7f8474078340 msgr2=0x7f8474078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.510+0000 7f8478f5e700 1 -- 172.21.15.7:0/4219322692 shutdown_connections 2024-06-10T01:16:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.510+0000 7f8478f5e700 1 -- 172.21.15.7:0/4219322692 wait complete. 2024-06-10T01:16:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.511+0000 7f8478f5e700 1 Processor -- start 2024-06-10T01:16:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.511+0000 7f8478f5e700 1 -- start start 2024-06-10T01:16:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.511+0000 7f8478f5e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f847406ab60 0x7f847419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.511+0000 7f8478f5e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847406b520 0x7f847419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.511+0000 7f8478f5e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84741084f0 0x7f8474198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.511+0000 7f8478f5e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8474112750 con 0x7f847406b520 2024-06-10T01:16:28.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.511+0000 7f8478f5e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f84741125d0 con 0x7f847406ab60 2024-06-10T01:16:28.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.511+0000 7f8478f5e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f84741128d0 con 0x7f84741084f0 2024-06-10T01:16:28.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.511+0000 7f847259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f847406ab60 0x7f847419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:28.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.511+0000 7f8471d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847406b520 0x7f847419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:28.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.511+0000 7f8471d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847406b520 0x7f847419ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40112/0 (socket says 172.21.15.7:40112) 2024-06-10T01:16:28.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.511+0000 7f847259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f847406ab60 0x7f847419e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:48352/0 (socket says 172.21.15.7:48352) 2024-06-10T01:16:28.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.511+0000 7f8471d9b700 1 -- 172.21.15.7:0/3912320941 learned_addr learned my addr 172.21.15.7:0/3912320941 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:28.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.511+0000 7f8472d9d700 1 --2- 172.21.15.7:0/3912320941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84741084f0 0x7f8474198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:28.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.512+0000 7f8471d9b700 1 -- 172.21.15.7:0/3912320941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84741084f0 msgr2=0x7f8474198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:28.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.512+0000 7f8471d9b700 1 --2- 172.21.15.7:0/3912320941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84741084f0 0x7f8474198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.512+0000 7f8471d9b700 1 -- 172.21.15.7:0/3912320941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f847406ab60 msgr2=0x7f847419e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:28.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.512+0000 7f8471d9b700 1 --2- 172.21.15.7:0/3912320941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f847406ab60 0x7f847419e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.512+0000 7f8471d9b700 1 -- 172.21.15.7:0/3912320941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8474199190 con 0x7f847406b520 2024-06-10T01:16:28.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.512+0000 7f8471d9b700 1 --2- 172.21.15.7:0/3912320941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847406b520 0x7f847419ed60 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f846800c9b0 tx=0x7f846800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:28.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.512+0000 7f84637fe700 1 -- 172.21.15.7:0/3912320941 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8468022070 con 0x7f847406b520 2024-06-10T01:16:28.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.512+0000 7f8478f5e700 1 -- 172.21.15.7:0/3912320941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8474199480 con 0x7f847406b520 2024-06-10T01:16:28.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.512+0000 7f8478f5e700 1 -- 172.21.15.7:0/3912320941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8474101770 con 0x7f847406b520 2024-06-10T01:16:28.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.512+0000 7f84637fe700 1 -- 172.21.15.7:0/3912320941 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f846800ce50 con 0x7f847406b520 2024-06-10T01:16:28.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.513+0000 7f84637fe700 1 -- 172.21.15.7:0/3912320941 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846801dbb0 con 0x7f847406b520 2024-06-10T01:16:28.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.515+0000 7f84637fe700 1 -- 172.21.15.7:0/3912320941 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f8468004830 con 0x7f847406b520 2024-06-10T01:16:28.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.516+0000 7f84637fe700 1 --2- 172.21.15.7:0/3912320941 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f845c0a2c20 0x7f845c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:28.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.516+0000 7f84637fe700 1 -- 172.21.15.7:0/3912320941 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f84680d7060 con 0x7f847406b520 2024-06-10T01:16:28.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.516+0000 7f847259c700 1 --2- 172.21.15.7:0/3912320941 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f845c0a2c20 0x7f845c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:28.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.516+0000 7f84837e1b80 1 Processor -- start 2024-06-10T01:16:28.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.517+0000 7f847259c700 1 --2- 172.21.15.7:0/3912320941 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f845c0a2c20 0x7f845c0a50e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f846400b2e0 tx=0x7f846401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:28.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.517+0000 7f84837e1b80 1 -- start start 2024-06-10T01:16:28.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.517+0000 7f84837e1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561de2b5a640 0x561de2b5aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:28.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.517+0000 7f84837e1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561de2b5af60 0x561de2b549e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:28.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.517+0000 7f84837e1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561de2b54f20 0x561de2b55340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:28.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.517+0000 7f84837e1b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561de2aa6810 con 0x561de2b54f20 2024-06-10T01:16:28.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.517+0000 7f84837e1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561de2a4e720 con 0x561de2b5a640 2024-06-10T01:16:28.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.517+0000 7f84837e1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561de2aa27d0 con 0x561de2b5af60 2024-06-10T01:16:28.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.517+0000 7f8471d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561de2b5af60 0x561de2b549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:28.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.517+0000 7f8472d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561de2b54f20 0x561de2b55340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:28.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.517+0000 7f8471d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561de2b5af60 0x561de2b549e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:44828/0 (socket says 172.21.15.7:44828) 2024-06-10T01:16:28.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.517+0000 7f8471d9b700 1 -- 172.21.15.7:0/855916956 learned_addr learned my addr 172.21.15.7:0/855916956 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:28.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.517+0000 7f847259c700 1 --2- 172.21.15.7:0/855916956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561de2b5a640 0x561de2b5aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:28.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.518+0000 7f8472d9d700 1 -- 172.21.15.7:0/855916956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561de2b5af60 msgr2=0x561de2b549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:28.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.518+0000 7f8472d9d700 1 --2- 172.21.15.7:0/855916956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561de2b5af60 0x561de2b549e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.518+0000 7f8472d9d700 1 -- 172.21.15.7:0/855916956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561de2b5a640 msgr2=0x561de2b5aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:28.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.518+0000 7f8472d9d700 1 --2- 172.21.15.7:0/855916956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561de2b5a640 0x561de2b5aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.518+0000 7f8472d9d700 1 -- 172.21.15.7:0/855916956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561de2a44f50 con 0x561de2b54f20 2024-06-10T01:16:28.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.518+0000 7f8471d9b700 1 --2- 172.21.15.7:0/855916956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561de2b5af60 0x561de2b549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:28.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.518+0000 7f8472d9d700 1 --2- 172.21.15.7:0/855916956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561de2b54f20 0x561de2b55340 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f846c0093c0 tx=0x7f846c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee0381fa98b07c3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:28.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.518+0000 7f8460ff9700 1 -- 172.21.15.7:0/855916956 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846c004bb0 con 0x561de2b54f20 2024-06-10T01:16:28.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.518+0000 7f8460ff9700 1 -- 172.21.15.7:0/855916956 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f846c004d50 con 0x561de2b54f20 2024-06-10T01:16:28.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.518+0000 7f8460ff9700 1 -- 172.21.15.7:0/855916956 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846c004830 con 0x561de2b54f20 2024-06-10T01:16:28.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.519+0000 7f84837e1b80 1 -- 172.21.15.7:0/855916956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561de2b54f20 msgr2=0x561de2b55340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:28.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.519+0000 7f84837e1b80 1 --2- 172.21.15.7:0/855916956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561de2b54f20 0x561de2b55340 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f846c0093c0 tx=0x7f846c009920 comp rx=0 tx=0).stop 2024-06-10T01:16:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.519+0000 7f84837e1b80 1 -- 172.21.15.7:0/855916956 shutdown_connections 2024-06-10T01:16:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.519+0000 7f84837e1b80 1 --2- 172.21.15.7:0/855916956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561de2b54f20 0x561de2b55340 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.519+0000 7f84837e1b80 1 --2- 172.21.15.7:0/855916956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561de2b5a640 0x561de2b5aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.519+0000 7f84837e1b80 1 --2- 172.21.15.7:0/855916956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561de2b5af60 0x561de2b549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.519+0000 7f84837e1b80 1 -- 172.21.15.7:0/855916956 >> 172.21.15.7:0/855916956 conn(0x561de2a7fcc0 msgr2=0x561de2b4f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.519+0000 7f84837e1b80 1 -- 172.21.15.7:0/855916956 shutdown_connections 2024-06-10T01:16:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.519+0000 7f84837e1b80 1 -- 172.21.15.7:0/855916956 wait complete. 2024-06-10T01:16:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.520+0000 7f84837e1b80 1 -- start start 2024-06-10T01:16:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.520+0000 7f84837e1b80 1 Processor -- start 2024-06-10T01:16:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.520+0000 7f84837e1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561de2b54f20 0x561de2b60c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.520+0000 7f84837e1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561de2b5a640 0x561de2b5dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.520+0000 7f84837e1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561de2b5af60 0x561de2b5e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:28.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.520+0000 7f84837e1b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561de2b619d0 con 0x561de2b5a640 2024-06-10T01:16:28.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.520+0000 7f84837e1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561de2a46bf0 con 0x561de2b5af60 2024-06-10T01:16:28.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.520+0000 7f84837e1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561de2b61b50 con 0x561de2b54f20 2024-06-10T01:16:28.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.520+0000 7f84837e1b80 10 client.0 _ll_register_callbacks cb 0x561de2a40920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:16:28.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.520+0000 7f847259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561de2b54f20 0x561de2b60c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:28.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.520+0000 7f847259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561de2b54f20 0x561de2b60c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:44840/0 (socket says 172.21.15.7:44840) 2024-06-10T01:16:28.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.520+0000 7f847259c700 1 -- 172.21.15.7:0/1572167561 learned_addr learned my addr 172.21.15.7:0/1572167561 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:28.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.520+0000 7f8471d9b700 1 --2- 172.21.15.7:0/1572167561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561de2b5a640 0x561de2b5dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:28.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.521+0000 7f8472d9d700 1 --2- 172.21.15.7:0/1572167561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561de2b5af60 0x561de2b5e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:28.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.521+0000 7f847259c700 1 -- 172.21.15.7:0/1572167561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561de2b5af60 msgr2=0x561de2b5e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:28.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.521+0000 7f847259c700 1 --2- 172.21.15.7:0/1572167561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561de2b5af60 0x561de2b5e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.521+0000 7f847259c700 1 -- 172.21.15.7:0/1572167561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561de2b5a640 msgr2=0x561de2b5dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:28.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.521+0000 7f847259c700 1 --2- 172.21.15.7:0/1572167561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561de2b5a640 0x561de2b5dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.521+0000 7f847259c700 1 -- 172.21.15.7:0/1572167561 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561de29f58d0 con 0x561de2b54f20 2024-06-10T01:16:28.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.521+0000 7f8472d9d700 1 --2- 172.21.15.7:0/1572167561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561de2b5af60 0x561de2b5e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:16:28.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.521+0000 7f847259c700 1 --2- 172.21.15.7:0/1572167561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561de2b54f20 0x561de2b60c90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8464009990 tx=0x7f8464009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abaa009d9d77834d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:28.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.521+0000 7f8459ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:16:28.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.521+0000 7f8459ffb700 1 -- 172.21.15.7:0/1572167561 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464003c70 con 0x561de2b54f20 2024-06-10T01:16:28.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.521+0000 7f84837e1b80 1 -- 172.21.15.7:0/1572167561 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f846c004d50 con 0x561de2b54f20 2024-06-10T01:16:28.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.521+0000 7f8459ffb700 1 -- 172.21.15.7:0/1572167561 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f8464031460 con 0x561de2b54f20 2024-06-10T01:16:28.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.522+0000 7f8459ffb700 1 -- 172.21.15.7:0/1572167561 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8464003e10 con 0x561de2b54f20 2024-06-10T01:16:28.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.522+0000 7f84837e1b80 10 client.27472 fetch_fsmap learned FSMap version 39 2024-06-10T01:16:28.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.522+0000 7f84837e1b80 1 -- 172.21.15.7:0/1572167561 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561de2b6c2f0 con 0x561de2b54f20 2024-06-10T01:16:28.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.522+0000 7f8459ffb700 1 -- 172.21.15.7:0/1572167561 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f846c004d50 con 0x561de2b54f20 2024-06-10T01:16:28.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.522+0000 7f8459ffb700 1 -- 172.21.15.7:0/1572167561 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7f846400f780 con 0x561de2b54f20 2024-06-10T01:16:28.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.522+0000 7f84837e1b80 10 client.27472 fetch_fsmap finished waiting for FSMap version 39 2024-06-10T01:16:28.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.522+0000 7f84837e1b80 10 client.27472 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:16:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.522+0000 7f84837e1b80 20 client.27472 populate_metadata read hostname 'smithi007' 2024-06-10T01:16:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.522+0000 7f84837e1b80 1 --2- 172.21.15.7:0/1572167561 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x561de2b6e930 0x561de2b70d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.522+0000 7f84837e1b80 4 client.27472 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:16:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.522+0000 7f84837e1b80 1 -- 172.21.15.7:0/1572167561 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561de2ab3d10 con 0x561de2b6e930 2024-06-10T01:16:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.522+0000 7f8471d9b700 1 --2- 172.21.15.7:0/1572167561 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x561de2b6e930 0x561de2b70d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.523+0000 7f8471d9b700 1 --2- 172.21.15.7:0/1572167561 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x561de2b6e930 0x561de2b70d90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=270257c679989e31 server_cookie=97c0aac3f5160fdc in_seq=0 out_seq=0 2024-06-10T01:16:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.523+0000 7f8459ffb700 10 client.27472 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:16:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.524+0000 7f8459ffb700 1 -- 172.21.15.7:0/1572167561 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561de2ab3d10 con 0x561de2b6e930 2024-06-10T01:16:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.524+0000 7f8459ffb700 10 client.27472 handle_command_reply: tid=0 2024-06-10T01:16:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.524+0000 7f84837e1b80 1 client.27472 shutdown 2024-06-10T01:16:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.524+0000 7f84837e1b80 1 -- 172.21.15.7:0/1572167561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561de2b54f20 msgr2=0x561de2b60c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.524+0000 7f84837e1b80 1 --2- 172.21.15.7:0/1572167561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561de2b54f20 0x561de2b60c90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8464009990 tx=0x7f8464009310 comp rx=0 tx=0).stop 2024-06-10T01:16:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.524+0000 7f84837e1b80 1 -- 172.21.15.7:0/1572167561 shutdown_connections 2024-06-10T01:16:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.524+0000 7f84837e1b80 1 --2- 172.21.15.7:0/1572167561 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x561de2b6e930 0x561de2b70d90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.524+0000 7f84837e1b80 1 --2- 172.21.15.7:0/1572167561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561de2b5af60 0x561de2b5e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.524+0000 7f8459ffb700 0 client.27472 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:16:28.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.524+0000 7f84837e1b80 1 --2- 172.21.15.7:0/1572167561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561de2b54f20 0x561de2b60c90 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.524+0000 7f84837e1b80 1 --2- 172.21.15.7:0/1572167561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561de2b5a640 0x561de2b5dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.525+0000 7f84837e1b80 1 -- 172.21.15.7:0/1572167561 >> 172.21.15.7:0/1572167561 conn(0x561de2a7fcc0 msgr2=0x561de2b539d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:28.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.525+0000 7f84837e1b80 1 -- 172.21.15.7:0/1572167561 shutdown_connections 2024-06-10T01:16:28.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.525+0000 7f84837e1b80 1 -- 172.21.15.7:0/1572167561 wait complete. 2024-06-10T01:16:28.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.525+0000 7f84837e1b80 20 client.27472 trim_cache size 0 max 16384 2024-06-10T01:16:28.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.543+0000 7f84837e1b80 1 Processor -- start 2024-06-10T01:16:28.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.543+0000 7f84837e1b80 1 -- start start 2024-06-10T01:16:28.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.543+0000 7f84837e1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561de2b54f20 0x561de2b549e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:28.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.543+0000 7f84837e1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561de2b5a640 0x561de2b5e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:28.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.543+0000 7f84837e1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561de2b5af60 0x561de2b5ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:28.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.543+0000 7f84837e1b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f84540112f0 con 0x561de2b54f20 2024-06-10T01:16:28.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.543+0000 7f84837e1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f845400eb50 con 0x561de2b5a640 2024-06-10T01:16:28.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.543+0000 7f84837e1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f845400c9a0 con 0x561de2b5af60 2024-06-10T01:16:28.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.543+0000 7f847259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561de2b54f20 0x561de2b549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:28.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.543+0000 7f847259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561de2b54f20 0x561de2b549e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40130/0 (socket says 172.21.15.7:40130) 2024-06-10T01:16:28.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.543+0000 7f847259c700 1 -- 172.21.15.7:0/2544225459 learned_addr learned my addr 172.21.15.7:0/2544225459 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:28.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.543+0000 7f8472d9d700 1 --2- 172.21.15.7:0/2544225459 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561de2b5af60 0x561de2b5ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:28.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.543+0000 7f8471d9b700 1 --2- 172.21.15.7:0/2544225459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561de2b5a640 0x561de2b5e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:28.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.544+0000 7f847259c700 1 -- 172.21.15.7:0/2544225459 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561de2b5af60 msgr2=0x561de2b5ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:28.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.544+0000 7f847259c700 1 --2- 172.21.15.7:0/2544225459 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561de2b5af60 0x561de2b5ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.544+0000 7f847259c700 1 -- 172.21.15.7:0/2544225459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561de2b5a640 msgr2=0x561de2b5e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:28.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.544+0000 7f847259c700 1 --2- 172.21.15.7:0/2544225459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561de2b5a640 0x561de2b5e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.544+0000 7f847259c700 1 -- 172.21.15.7:0/2544225459 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561de2b55690 con 0x561de2b54f20 2024-06-10T01:16:28.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.544+0000 7f847259c700 1 --2- 172.21.15.7:0/2544225459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561de2b54f20 0x561de2b549e0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f8464005be0 tx=0x7f8464005ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b966ae1d247845 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:28.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.544+0000 7f845bfff700 1 -- 172.21.15.7:0/2544225459 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464003c70 con 0x561de2b54f20 2024-06-10T01:16:28.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.544+0000 7f845bfff700 1 -- 172.21.15.7:0/2544225459 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84640058e0 con 0x561de2b54f20 2024-06-10T01:16:28.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.544+0000 7f845bfff700 1 -- 172.21.15.7:0/2544225459 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464031ce0 con 0x561de2b54f20 2024-06-10T01:16:28.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.545+0000 7f84837e1b80 1 -- 172.21.15.7:0/2544225459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561de2b54f20 msgr2=0x561de2b549e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:28.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.545+0000 7f84837e1b80 1 --2- 172.21.15.7:0/2544225459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561de2b54f20 0x561de2b549e0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f8464005be0 tx=0x7f8464005ec0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.545+0000 7f84837e1b80 1 -- 172.21.15.7:0/2544225459 shutdown_connections 2024-06-10T01:16:28.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.545+0000 7f84837e1b80 1 --2- 172.21.15.7:0/2544225459 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561de2b5af60 0x561de2b5ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.545+0000 7f84837e1b80 1 --2- 172.21.15.7:0/2544225459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561de2b54f20 0x561de2b549e0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.545+0000 7f84837e1b80 1 --2- 172.21.15.7:0/2544225459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561de2b5a640 0x561de2b5e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.545+0000 7f84837e1b80 1 -- 172.21.15.7:0/2544225459 >> 172.21.15.7:0/2544225459 conn(0x561de2a7fcc0 msgr2=0x561de2b5b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:28.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.545+0000 7f84837e1b80 1 -- 172.21.15.7:0/2544225459 shutdown_connections 2024-06-10T01:16:28.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.545+0000 7f84837e1b80 1 -- 172.21.15.7:0/2544225459 wait complete. 2024-06-10T01:16:28.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.545+0000 7f84837e1b80 1 -- start start 2024-06-10T01:16:28.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.545+0000 7f84837e1b80 1 Processor -- start 2024-06-10T01:16:28.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.546+0000 7f84837e1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561de2b54f20 0x561de2b67440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:28.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.546+0000 7f84837e1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561de2b5a640 0x561de2b75df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:28.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.546+0000 7f84837e1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561de2b5af60 0x561de2b76330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:28.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.546+0000 7f84837e1b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f845400d010 con 0x561de2b5a640 2024-06-10T01:16:28.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.546+0000 7f84837e1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f845400e480 con 0x561de2b54f20 2024-06-10T01:16:28.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.546+0000 7f84837e1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f845400d6e0 con 0x561de2b5af60 2024-06-10T01:16:28.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.546+0000 7f84837e1b80 10 client.0 _ll_register_callbacks cb 0x561de2a40920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:16:28.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.546+0000 7f847259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561de2b54f20 0x561de2b67440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:28.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.546+0000 7f8471d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561de2b5a640 0x561de2b75df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:28.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.546+0000 7f8471d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561de2b5a640 0x561de2b75df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40140/0 (socket says 172.21.15.7:40140) 2024-06-10T01:16:28.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.546+0000 7f847259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561de2b54f20 0x561de2b67440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:48398/0 (socket says 172.21.15.7:48398) 2024-06-10T01:16:28.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.546+0000 7f8471d9b700 1 -- 172.21.15.7:0/848823074 learned_addr learned my addr 172.21.15.7:0/848823074 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:28.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.546+0000 7f8472d9d700 1 --2- 172.21.15.7:0/848823074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561de2b5af60 0x561de2b76330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:28.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.546+0000 7f8471d9b700 1 -- 172.21.15.7:0/848823074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561de2b5af60 msgr2=0x561de2b76330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:28.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.546+0000 7f8471d9b700 1 --2- 172.21.15.7:0/848823074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561de2b5af60 0x561de2b76330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.546+0000 7f8471d9b700 1 -- 172.21.15.7:0/848823074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561de2b54f20 msgr2=0x561de2b67440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:28.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.546+0000 7f8471d9b700 1 --2- 172.21.15.7:0/848823074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561de2b54f20 0x561de2b67440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.546+0000 7f8471d9b700 1 -- 172.21.15.7:0/848823074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561de2b769e0 con 0x561de2b5a640 2024-06-10T01:16:28.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.546+0000 7f847259c700 1 --2- 172.21.15.7:0/848823074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561de2b54f20 0x561de2b67440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:28.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.547+0000 7f8471d9b700 1 --2- 172.21.15.7:0/848823074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561de2b5a640 0x561de2b75df0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f846801d760 tx=0x7f8468017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d598c983ed4c37be server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:28.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.547+0000 7f8459ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:16:28.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.547+0000 7f8459ffb700 1 -- 172.21.15.7:0/848823074 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84680a34e0 con 0x561de2b5a640 2024-06-10T01:16:28.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.547+0000 7f8459ffb700 1 -- 172.21.15.7:0/848823074 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f846801d440 con 0x561de2b5a640 2024-06-10T01:16:28.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.547+0000 7f84837e1b80 1 -- 172.21.15.7:0/848823074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84640058e0 con 0x561de2b5a640 2024-06-10T01:16:28.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.547+0000 7f8459ffb700 1 -- 172.21.15.7:0/848823074 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f84680bc260 con 0x561de2b5a640 2024-06-10T01:16:28.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.547+0000 7f84837e1b80 10 client.18099 fetch_fsmap learned FSMap version 39 2024-06-10T01:16:28.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.547+0000 7f84837e1b80 1 -- 172.21.15.7:0/848823074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561de2bd95a0 con 0x561de2b5a640 2024-06-10T01:16:28.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.547+0000 7f8459ffb700 1 -- 172.21.15.7:0/848823074 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84640058e0 con 0x561de2b5a640 2024-06-10T01:16:28.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.548+0000 7f8459ffb700 1 -- 172.21.15.7:0/848823074 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7f84680a29c0 con 0x561de2b5a640 2024-06-10T01:16:28.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.548+0000 7f84837e1b80 10 client.18099 fetch_fsmap finished waiting for FSMap version 39 2024-06-10T01:16:28.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.548+0000 7f84837e1b80 10 client.18099 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:16:28.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.548+0000 7f84837e1b80 20 client.18099 populate_metadata read hostname 'smithi007' 2024-06-10T01:16:28.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.548+0000 7f84837e1b80 1 --2- 172.21.15.7:0/848823074 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x561de2b6e930 0x561de2bdbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:28.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.548+0000 7f84837e1b80 4 client.18099 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:16:28.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.548+0000 7f84837e1b80 1 -- 172.21.15.7:0/848823074 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561de2aae290 con 0x561de2b6e930 2024-06-10T01:16:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.548+0000 7f847259c700 1 --2- 172.21.15.7:0/848823074 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x561de2b6e930 0x561de2bdbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.548+0000 7f847259c700 1 --2- 172.21.15.7:0/848823074 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x561de2b6e930 0x561de2bdbcd0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=61fc558af2b84649 server_cookie=6c558f8f193363b4 in_seq=0 out_seq=0 2024-06-10T01:16:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.548+0000 7f8459ffb700 10 client.18099 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:16:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.548+0000 7f8459ffb700 1 -- 172.21.15.7:0/848823074 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561de2aae290 con 0x561de2b6e930 2024-06-10T01:16:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.548+0000 7f8459ffb700 10 client.18099 handle_command_reply: tid=0 2024-06-10T01:16:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.548+0000 7f84837e1b80 1 client.18099 shutdown 2024-06-10T01:16:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.549+0000 7f84837e1b80 1 -- 172.21.15.7:0/848823074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561de2b5a640 msgr2=0x561de2b75df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.549+0000 7f84837e1b80 1 --2- 172.21.15.7:0/848823074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561de2b5a640 0x561de2b75df0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f846801d760 tx=0x7f8468017040 comp rx=0 tx=0).stop 2024-06-10T01:16:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.549+0000 7f84837e1b80 1 -- 172.21.15.7:0/848823074 shutdown_connections 2024-06-10T01:16:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.549+0000 7f84837e1b80 1 --2- 172.21.15.7:0/848823074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561de2b5af60 0x561de2b76330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.549+0000 7f84837e1b80 1 --2- 172.21.15.7:0/848823074 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x561de2b6e930 0x561de2bdbcd0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.549+0000 7f84837e1b80 1 --2- 172.21.15.7:0/848823074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561de2b54f20 0x561de2b67440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.549+0000 7f84837e1b80 1 --2- 172.21.15.7:0/848823074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561de2b5a640 0x561de2b75df0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.549+0000 7f8459ffb700 0 client.18099 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:16:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.549+0000 7f84837e1b80 1 -- 172.21.15.7:0/848823074 >> 172.21.15.7:0/848823074 conn(0x561de2a7fcc0 msgr2=0x561de2b48e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.549+0000 7f84837e1b80 1 -- 172.21.15.7:0/848823074 shutdown_connections 2024-06-10T01:16:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.549+0000 7f84837e1b80 1 -- 172.21.15.7:0/848823074 wait complete. 2024-06-10T01:16:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.549+0000 7f84837e1b80 20 client.18099 trim_cache size 0 max 16384 2024-06-10T01:16:28.551 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:16:28.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.549+0000 7f8478f5e700 1 -- 172.21.15.7:0/3912320941 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f845c0a2c20 msgr2=0x7f845c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:28.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.549+0000 7f8478f5e700 1 --2- 172.21.15.7:0/3912320941 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f845c0a2c20 0x7f845c0a50e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f846400b2e0 tx=0x7f846401a040 comp rx=0 tx=0).stop 2024-06-10T01:16:28.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.549+0000 7f8478f5e700 1 -- 172.21.15.7:0/3912320941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847406b520 msgr2=0x7f847419ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:28.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.549+0000 7f8478f5e700 1 --2- 172.21.15.7:0/3912320941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847406b520 0x7f847419ed60 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f846800c9b0 tx=0x7f846800ccc0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.549+0000 7f8478f5e700 1 -- 172.21.15.7:0/3912320941 shutdown_connections 2024-06-10T01:16:28.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.549+0000 7f8478f5e700 1 --2- 172.21.15.7:0/3912320941 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f845c0a2c20 0x7f845c0a50e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.549+0000 7f8478f5e700 1 --2- 172.21.15.7:0/3912320941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84741084f0 0x7f8474198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.549+0000 7f8478f5e700 1 --2- 172.21.15.7:0/3912320941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f847406ab60 0x7f847419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.549+0000 7f8478f5e700 1 --2- 172.21.15.7:0/3912320941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847406b520 0x7f847419ed60 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:28.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.549+0000 7f8478f5e700 1 -- 172.21.15.7:0/3912320941 >> 172.21.15.7:0/3912320941 conn(0x7f8474078340 msgr2=0x7f8474100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:28.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.550+0000 7f8478f5e700 1 -- 172.21.15.7:0/3912320941 shutdown_connections 2024-06-10T01:16:28.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:28.550+0000 7f8478f5e700 1 -- 172.21.15.7:0/3912320941 wait complete. 2024-06-10T01:16:28.748 INFO:tasks.workunit.client.0.smithi007.stdout: 1 769831 7.52 MB/sec execute 429 sec latency 34.887 ms 2024-06-10T01:16:29.150 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:28 smithi007 ceph-mon[32398]: pgmap v746: 129 pgs: 129 active+clean; 133 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 2.3 MiB/s wr, 264 op/s 2024-06-10T01:16:29.151 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:28 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:16:29.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:28 smithi175 ceph-mon[36636]: pgmap v746: 129 pgs: 129 active+clean; 133 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 2.3 MiB/s wr, 264 op/s 2024-06-10T01:16:29.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:28 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:16:29.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:28 smithi078 ceph-mon[36683]: pgmap v746: 129 pgs: 129 active+clean; 133 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 2.3 MiB/s wr, 264 op/s 2024-06-10T01:16:29.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:28 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:16:29.568 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:16:29.748 INFO:tasks.workunit.client.0.smithi007.stdout: 1 771057 7.52 MB/sec execute 430 sec latency 78.846 ms 2024-06-10T01:16:29.864 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:16:30.748 INFO:tasks.workunit.client.0.smithi007.stdout: 1 772929 7.52 MB/sec execute 431 sec latency 41.860 ms 2024-06-10T01:16:31.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:30 smithi175 ceph-mon[36636]: pgmap v747: 129 pgs: 129 active+clean; 135 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 2.9 MiB/s wr, 327 op/s 2024-06-10T01:16:31.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:30 smithi007 ceph-mon[32398]: pgmap v747: 129 pgs: 129 active+clean; 135 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 2.9 MiB/s wr, 327 op/s 2024-06-10T01:16:31.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:30 smithi078 ceph-mon[36683]: pgmap v747: 129 pgs: 129 active+clean; 135 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 2.9 MiB/s wr, 327 op/s 2024-06-10T01:16:31.748 INFO:tasks.workunit.client.0.smithi007.stdout: 1 774618 7.53 MB/sec execute 432 sec latency 13.364 ms 2024-06-10T01:16:31.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.963+0000 7f8c3ae93700 1 -- 172.21.15.7:0/880754700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c34102550 msgr2=0x7f8c341029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:31.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.963+0000 7f8c3ae93700 1 --2- 172.21.15.7:0/880754700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c34102550 0x7f8c341029d0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f8c200099c0 tx=0x7f8c20009cd0 comp rx=0 tx=0).stop 2024-06-10T01:16:31.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.963+0000 7f8c3ae93700 1 -- 172.21.15.7:0/880754700 shutdown_connections 2024-06-10T01:16:31.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.963+0000 7f8c3ae93700 1 --2- 172.21.15.7:0/880754700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c34102f10 0x7f8c3410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:31.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.963+0000 7f8c3ae93700 1 --2- 172.21.15.7:0/880754700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c341073a0 0x7f8c34107780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:31.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.963+0000 7f8c3ae93700 1 --2- 172.21.15.7:0/880754700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c34102550 0x7f8c341029d0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:31.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.963+0000 7f8c3ae93700 1 -- 172.21.15.7:0/880754700 >> 172.21.15.7:0/880754700 conn(0x7f8c34079650 msgr2=0x7f8c34079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:31.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.963+0000 7f8c3ae93700 1 -- 172.21.15.7:0/880754700 shutdown_connections 2024-06-10T01:16:31.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.963+0000 7f8c3ae93700 1 -- 172.21.15.7:0/880754700 wait complete. 2024-06-10T01:16:31.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.964+0000 7f8c3ae93700 1 Processor -- start 2024-06-10T01:16:31.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.964+0000 7f8c3ae93700 1 -- start start 2024-06-10T01:16:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.964+0000 7f8c3ae93700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c34102550 0x7f8c3419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.964+0000 7f8c3ae93700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c34102f10 0x7f8c3419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.964+0000 7f8c3ae93700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c341073a0 0x7f8c34198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.964+0000 7f8c3ae93700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8c34112760 con 0x7f8c341073a0 2024-06-10T01:16:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.964+0000 7f8c3ae93700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8c341125e0 con 0x7f8c34102f10 2024-06-10T01:16:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.964+0000 7f8c3ae93700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8c341128e0 con 0x7f8c34102550 2024-06-10T01:16:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.965+0000 7f8c38c2f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c34102550 0x7f8c3419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.965+0000 7f8c39430700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c341073a0 0x7f8c34198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.965+0000 7f8c38c2f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c34102550 0x7f8c3419e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:59782/0 (socket says 172.21.15.7:59782) 2024-06-10T01:16:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.965+0000 7f8c38c2f700 1 -- 172.21.15.7:0/2769857319 learned_addr learned my addr 172.21.15.7:0/2769857319 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.965+0000 7f8c39430700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c341073a0 0x7f8c34198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38794/0 (socket says 172.21.15.7:38794) 2024-06-10T01:16:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.965+0000 7f8c33fff700 1 --2- 172.21.15.7:0/2769857319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c34102f10 0x7f8c3419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.965+0000 7f8c38c2f700 1 -- 172.21.15.7:0/2769857319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c34102f10 msgr2=0x7f8c3419ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.965+0000 7f8c38c2f700 1 --2- 172.21.15.7:0/2769857319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c34102f10 0x7f8c3419ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.965+0000 7f8c38c2f700 1 -- 172.21.15.7:0/2769857319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c341073a0 msgr2=0x7f8c34198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.965+0000 7f8c38c2f700 1 --2- 172.21.15.7:0/2769857319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c341073a0 0x7f8c34198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.965+0000 7f8c38c2f700 1 -- 172.21.15.7:0/2769857319 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c34199180 con 0x7f8c34102550 2024-06-10T01:16:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.965+0000 7f8c39430700 1 --2- 172.21.15.7:0/2769857319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c341073a0 0x7f8c34198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.965+0000 7f8c38c2f700 1 --2- 172.21.15.7:0/2769857319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c34102550 0x7f8c3419e810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800e9a0 tx=0x7f8c2800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.965+0000 7f8c31ffb700 1 -- 172.21.15.7:0/2769857319 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2800ca10 con 0x7f8c34102550 2024-06-10T01:16:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.965+0000 7f8c31ffb700 1 -- 172.21.15.7:0/2769857319 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c2800cbb0 con 0x7f8c34102550 2024-06-10T01:16:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.965+0000 7f8c3ae93700 1 -- 172.21.15.7:0/2769857319 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c34199470 con 0x7f8c34102550 2024-06-10T01:16:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.966+0000 7f8c31ffb700 1 -- 172.21.15.7:0/2769857319 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28018680 con 0x7f8c34102550 2024-06-10T01:16:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.966+0000 7f8c3ae93700 1 -- 172.21.15.7:0/2769857319 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c341a56e0 con 0x7f8c34102550 2024-06-10T01:16:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.968+0000 7f8c31ffb700 1 -- 172.21.15.7:0/2769857319 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f8c28010430 con 0x7f8c34102550 2024-06-10T01:16:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.969+0000 7f8c31ffb700 1 --2- 172.21.15.7:0/2769857319 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8c240a2c20 0x7f8c240a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.969+0000 7f8c33fff700 1 --2- 172.21.15.7:0/2769857319 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8c240a2c20 0x7f8c240a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.969+0000 7f8c31ffb700 1 -- 172.21.15.7:0/2769857319 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f8c280cf570 con 0x7f8c34102550 2024-06-10T01:16:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.969+0000 7f8c45716b80 1 Processor -- start 2024-06-10T01:16:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.969+0000 7f8c33fff700 1 --2- 172.21.15.7:0/2769857319 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8c240a2c20 0x7f8c240a50e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8c34079b50 tx=0x7f8c20005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.969+0000 7f8c45716b80 1 -- start start 2024-06-10T01:16:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.970+0000 7f8c45716b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c1fe67850 0x560c1fe67c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.970+0000 7f8c45716b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c1fe68170 0x560c1fe6c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.970+0000 7f8c45716b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c1fe6c550 0x560c1fe6c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.970+0000 7f8c45716b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560c1fd55f60 con 0x560c1fe6c550 2024-06-10T01:16:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.970+0000 7f8c45716b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560c1fdb4850 con 0x560c1fe67850 2024-06-10T01:16:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.970+0000 7f8c45716b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560c1fd1c460 con 0x560c1fe68170 2024-06-10T01:16:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.970+0000 7f8c38c2f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c1fe67850 0x560c1fe67c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:31.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.970+0000 7f8c38c2f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c1fe67850 0x560c1fe67c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:36296/0 (socket says 172.21.15.7:36296) 2024-06-10T01:16:31.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.970+0000 7f8c38c2f700 1 -- 172.21.15.7:0/1872577816 learned_addr learned my addr 172.21.15.7:0/1872577816 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:31.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.970+0000 7f8c33fff700 1 --2- 172.21.15.7:0/1872577816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c1fe68170 0x560c1fe6c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:31.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.970+0000 7f8c39430700 1 --2- 172.21.15.7:0/1872577816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c1fe6c550 0x560c1fe6c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:31.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.970+0000 7f8c38c2f700 1 -- 172.21.15.7:0/1872577816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c1fe68170 msgr2=0x560c1fe6c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:31.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.970+0000 7f8c38c2f700 1 --2- 172.21.15.7:0/1872577816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c1fe68170 0x560c1fe6c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:31.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.970+0000 7f8c38c2f700 1 -- 172.21.15.7:0/1872577816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c1fe6c550 msgr2=0x560c1fe6c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:31.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.970+0000 7f8c38c2f700 1 --2- 172.21.15.7:0/1872577816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c1fe6c550 0x560c1fe6c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:31.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.970+0000 7f8c38c2f700 1 -- 172.21.15.7:0/1872577816 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c1fd038d0 con 0x560c1fe67850 2024-06-10T01:16:31.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.970+0000 7f8c38c2f700 1 --2- 172.21.15.7:0/1872577816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c1fe67850 0x560c1fe67c30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8c280a4190 tx=0x7f8c2809c150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7dc03c31b899f55 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:31.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.971+0000 7f8c1effd700 1 -- 172.21.15.7:0/1872577816 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2809fe00 con 0x560c1fe67850 2024-06-10T01:16:31.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.971+0000 7f8c1effd700 1 -- 172.21.15.7:0/1872577816 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c280a46c0 con 0x560c1fe67850 2024-06-10T01:16:31.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.971+0000 7f8c1effd700 1 -- 172.21.15.7:0/1872577816 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28010720 con 0x560c1fe67850 2024-06-10T01:16:31.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.971+0000 7f8c45716b80 1 -- 172.21.15.7:0/1872577816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c1fe67850 msgr2=0x560c1fe67c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:31.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.971+0000 7f8c45716b80 1 --2- 172.21.15.7:0/1872577816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c1fe67850 0x560c1fe67c30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8c280a4190 tx=0x7f8c2809c150 comp rx=0 tx=0).stop 2024-06-10T01:16:31.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.971+0000 7f8c45716b80 1 -- 172.21.15.7:0/1872577816 shutdown_connections 2024-06-10T01:16:31.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.971+0000 7f8c45716b80 1 --2- 172.21.15.7:0/1872577816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c1fe6c550 0x560c1fe6c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:31.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.971+0000 7f8c45716b80 1 --2- 172.21.15.7:0/1872577816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c1fe67850 0x560c1fe67c30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:31.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.971+0000 7f8c45716b80 1 --2- 172.21.15.7:0/1872577816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c1fe68170 0x560c1fe6c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:31.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.971+0000 7f8c45716b80 1 -- 172.21.15.7:0/1872577816 >> 172.21.15.7:0/1872577816 conn(0x560c1fd90080 msgr2=0x560c1fe5e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:31.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.972+0000 7f8c45716b80 1 -- 172.21.15.7:0/1872577816 shutdown_connections 2024-06-10T01:16:31.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.972+0000 7f8c45716b80 1 -- 172.21.15.7:0/1872577816 wait complete. 2024-06-10T01:16:31.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.972+0000 7f8c45716b80 1 -- start start 2024-06-10T01:16:31.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.972+0000 7f8c45716b80 1 Processor -- start 2024-06-10T01:16:31.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.973+0000 7f8c45716b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c1fe67850 0x560c1fe73b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:31.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.973+0000 7f8c45716b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c1fe68170 0x560c1fe70e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:31.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.973+0000 7f8c45716b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c1fe6c550 0x560c1fe713a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:31.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.973+0000 7f8c45716b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560c1fd54fc0 con 0x560c1fe6c550 2024-06-10T01:16:31.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.973+0000 7f8c45716b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560c1fdb0810 con 0x560c1fe68170 2024-06-10T01:16:31.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.973+0000 7f8c45716b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560c1fe74680 con 0x560c1fe67850 2024-06-10T01:16:31.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.973+0000 7f8c38c2f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c1fe67850 0x560c1fe73b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:31.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.973+0000 7f8c39430700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c1fe6c550 0x560c1fe713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:31.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.973+0000 7f8c45716b80 10 client.0 _ll_register_callbacks cb 0x560c1fd4a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:16:31.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.973+0000 7f8c33fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c1fe68170 0x560c1fe70e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:31.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.973+0000 7f8c38c2f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c1fe67850 0x560c1fe73b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:59812/0 (socket says 172.21.15.7:59812) 2024-06-10T01:16:31.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.973+0000 7f8c38c2f700 1 -- 172.21.15.7:0/1465213290 learned_addr learned my addr 172.21.15.7:0/1465213290 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:31.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.973+0000 7f8c38c2f700 1 -- 172.21.15.7:0/1465213290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c1fe68170 msgr2=0x560c1fe70e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:31.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.973+0000 7f8c38c2f700 1 --2- 172.21.15.7:0/1465213290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c1fe68170 0x560c1fe70e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:31.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.973+0000 7f8c38c2f700 1 -- 172.21.15.7:0/1465213290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c1fe6c550 msgr2=0x560c1fe713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:31.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.973+0000 7f8c38c2f700 1 --2- 172.21.15.7:0/1465213290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c1fe6c550 0x560c1fe713a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:31.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.973+0000 7f8c38c2f700 1 -- 172.21.15.7:0/1465213290 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c1fd7a750 con 0x560c1fe67850 2024-06-10T01:16:31.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.973+0000 7f8c33fff700 1 --2- 172.21.15.7:0/1465213290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c1fe68170 0x560c1fe70e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:31.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.973+0000 7f8c39430700 1 --2- 172.21.15.7:0/1465213290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c1fe6c550 0x560c1fe713a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:31.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.973+0000 7f8c38c2f700 1 --2- 172.21.15.7:0/1465213290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c1fe67850 0x560c1fe73b60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800cfd0 tx=0x7f8c2800c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5e5ea7e9b773b6d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:31.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.973+0000 7f8c13fff700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:16:31.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.974+0000 7f8c13fff700 1 -- 172.21.15.7:0/1465213290 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c280b8070 con 0x560c1fe67850 2024-06-10T01:16:31.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.974+0000 7f8c13fff700 1 -- 172.21.15.7:0/1465213290 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f8c280cf290 con 0x560c1fe67850 2024-06-10T01:16:31.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.974+0000 7f8c45716b80 1 -- 172.21.15.7:0/1465213290 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c280a46c0 con 0x560c1fe67850 2024-06-10T01:16:31.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.974+0000 7f8c13fff700 1 -- 172.21.15.7:0/1465213290 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8c280b3070 con 0x560c1fe67850 2024-06-10T01:16:31.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.974+0000 7f8c45716b80 10 client.27490 fetch_fsmap learned FSMap version 39 2024-06-10T01:16:31.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.974+0000 7f8c45716b80 1 -- 172.21.15.7:0/1465213290 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560c1fe7eee0 con 0x560c1fe67850 2024-06-10T01:16:31.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.974+0000 7f8c13fff700 1 -- 172.21.15.7:0/1465213290 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c280a46c0 con 0x560c1fe67850 2024-06-10T01:16:31.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.974+0000 7f8c13fff700 1 -- 172.21.15.7:0/1465213290 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7f8c2809aea0 con 0x560c1fe67850 2024-06-10T01:16:31.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.974+0000 7f8c45716b80 10 client.27490 fetch_fsmap finished waiting for FSMap version 39 2024-06-10T01:16:31.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.974+0000 7f8c45716b80 10 client.27490 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:16:31.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.974+0000 7f8c45716b80 20 client.27490 populate_metadata read hostname 'smithi007' 2024-06-10T01:16:31.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.974+0000 7f8c45716b80 1 --2- 172.21.15.7:0/1465213290 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560c1fe813b0 0x560c1fe837a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:31.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.974+0000 7f8c45716b80 4 client.27490 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:16:31.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.974+0000 7f8c45716b80 1 -- 172.21.15.7:0/1465213290 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c1fdc1d50 con 0x560c1fe813b0 2024-06-10T01:16:31.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.975+0000 7f8c33fff700 1 --2- 172.21.15.7:0/1465213290 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560c1fe813b0 0x560c1fe837a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:31.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.975+0000 7f8c33fff700 1 --2- 172.21.15.7:0/1465213290 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560c1fe813b0 0x560c1fe837a0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64db00cdb8b91399 server_cookie=2669e038c8586488 in_seq=0 out_seq=0 2024-06-10T01:16:31.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.975+0000 7f8c13fff700 10 client.27490 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:31.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.976+0000 7f8c13fff700 1 -- 172.21.15.7:0/1465213290 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560c1fdc1d50 con 0x560c1fe813b0 2024-06-10T01:16:31.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.976+0000 7f8c13fff700 10 client.27490 handle_command_reply: tid=0 2024-06-10T01:16:31.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.976+0000 7f8c45716b80 1 client.27490 shutdown 2024-06-10T01:16:31.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.976+0000 7f8c45716b80 1 -- 172.21.15.7:0/1465213290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c1fe67850 msgr2=0x560c1fe73b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:31.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.976+0000 7f8c45716b80 1 --2- 172.21.15.7:0/1465213290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c1fe67850 0x560c1fe73b60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800cfd0 tx=0x7f8c2800c480 comp rx=0 tx=0).stop 2024-06-10T01:16:31.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.976+0000 7f8c45716b80 1 -- 172.21.15.7:0/1465213290 shutdown_connections 2024-06-10T01:16:31.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.976+0000 7f8c45716b80 1 --2- 172.21.15.7:0/1465213290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c1fe6c550 0x560c1fe713a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:31.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.976+0000 7f8c45716b80 1 --2- 172.21.15.7:0/1465213290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c1fe67850 0x560c1fe73b60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:31.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.976+0000 7f8c45716b80 1 --2- 172.21.15.7:0/1465213290 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560c1fe813b0 0x560c1fe837a0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:31.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.976+0000 7f8c45716b80 1 --2- 172.21.15.7:0/1465213290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c1fe68170 0x560c1fe70e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:31.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.976+0000 7f8c45716b80 1 -- 172.21.15.7:0/1465213290 >> 172.21.15.7:0/1465213290 conn(0x560c1fd90080 msgr2=0x560c1fe6da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:31.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.976+0000 7f8c13fff700 0 client.27490 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:31.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.976+0000 7f8c45716b80 1 -- 172.21.15.7:0/1465213290 shutdown_connections 2024-06-10T01:16:31.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.976+0000 7f8c45716b80 1 -- 172.21.15.7:0/1465213290 wait complete. 2024-06-10T01:16:31.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.976+0000 7f8c45716b80 20 client.27490 trim_cache size 0 max 16384 2024-06-10T01:16:31.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.992+0000 7f8c45716b80 1 Processor -- start 2024-06-10T01:16:31.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.992+0000 7f8c45716b80 1 -- start start 2024-06-10T01:16:31.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.993+0000 7f8c45716b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c1fe67850 0x560c1fe6ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:31.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.993+0000 7f8c45716b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c1fe68170 0x560c1fe5d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:31.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.993+0000 7f8c45716b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c1fe6c550 0x560c1fe6fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:31.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.993+0000 7f8c45716b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8c341ae950 con 0x560c1fe67850 2024-06-10T01:16:31.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.993+0000 7f8c45716b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8c341ac1b0 con 0x560c1fe6c550 2024-06-10T01:16:31.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.993+0000 7f8c45716b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8c341aa000 con 0x560c1fe68170 2024-06-10T01:16:31.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.993+0000 7f8c38c2f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c1fe67850 0x560c1fe6ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:31.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.993+0000 7f8c33fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c1fe68170 0x560c1fe5d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:31.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.993+0000 7f8c33fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c1fe68170 0x560c1fe5d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:59822/0 (socket says 172.21.15.7:59822) 2024-06-10T01:16:31.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.993+0000 7f8c33fff700 1 -- 172.21.15.7:0/1114203502 learned_addr learned my addr 172.21.15.7:0/1114203502 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:31.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.993+0000 7f8c39430700 1 --2- 172.21.15.7:0/1114203502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c1fe6c550 0x560c1fe6fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:31.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.993+0000 7f8c33fff700 1 -- 172.21.15.7:0/1114203502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c1fe6c550 msgr2=0x560c1fe6fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:31.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.993+0000 7f8c33fff700 1 --2- 172.21.15.7:0/1114203502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c1fe6c550 0x560c1fe6fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:31.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.993+0000 7f8c33fff700 1 -- 172.21.15.7:0/1114203502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c1fe67850 msgr2=0x560c1fe6ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:31.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.993+0000 7f8c33fff700 1 --2- 172.21.15.7:0/1114203502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c1fe67850 0x560c1fe6ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:31.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.993+0000 7f8c33fff700 1 -- 172.21.15.7:0/1114203502 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c1fe6f3d0 con 0x560c1fe68170 2024-06-10T01:16:31.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.993+0000 7f8c38c2f700 1 --2- 172.21.15.7:0/1114203502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c1fe67850 0x560c1fe6ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:31.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.994+0000 7f8c33fff700 1 --2- 172.21.15.7:0/1114203502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c1fe68170 0x560c1fe5d1e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8c20009750 tx=0x7f8c20023000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4bf114f65192403 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:31.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.994+0000 7f8c1e7fc700 1 -- 172.21.15.7:0/1114203502 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c20024860 con 0x560c1fe68170 2024-06-10T01:16:31.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.994+0000 7f8c1e7fc700 1 -- 172.21.15.7:0/1114203502 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c2000f460 con 0x560c1fe68170 2024-06-10T01:16:31.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.994+0000 7f8c1e7fc700 1 -- 172.21.15.7:0/1114203502 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2003b680 con 0x560c1fe68170 2024-06-10T01:16:31.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.995+0000 7f8c45716b80 1 -- 172.21.15.7:0/1114203502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c1fe68170 msgr2=0x560c1fe5d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:31.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.995+0000 7f8c45716b80 1 --2- 172.21.15.7:0/1114203502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c1fe68170 0x560c1fe5d1e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8c20009750 tx=0x7f8c20023000 comp rx=0 tx=0).stop 2024-06-10T01:16:31.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.995+0000 7f8c45716b80 1 -- 172.21.15.7:0/1114203502 shutdown_connections 2024-06-10T01:16:31.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.995+0000 7f8c45716b80 1 --2- 172.21.15.7:0/1114203502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c1fe6c550 0x560c1fe6fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:31.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.995+0000 7f8c45716b80 1 --2- 172.21.15.7:0/1114203502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c1fe67850 0x560c1fe6ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:31.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.995+0000 7f8c45716b80 1 --2- 172.21.15.7:0/1114203502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c1fe68170 0x560c1fe5d1e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:31.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.995+0000 7f8c45716b80 1 -- 172.21.15.7:0/1114203502 >> 172.21.15.7:0/1114203502 conn(0x560c1fd90080 msgr2=0x560c1fe6f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:31.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.995+0000 7f8c45716b80 1 -- 172.21.15.7:0/1114203502 shutdown_connections 2024-06-10T01:16:31.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.995+0000 7f8c45716b80 1 -- 172.21.15.7:0/1114203502 wait complete. 2024-06-10T01:16:31.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.995+0000 7f8c45716b80 1 -- start start 2024-06-10T01:16:31.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.995+0000 7f8c45716b80 1 Processor -- start 2024-06-10T01:16:31.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.996+0000 7f8c45716b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c1fe67850 0x560c1fe832d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:31.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.996+0000 7f8c45716b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c1fe68170 0x560c1fe83810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:31.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.996+0000 7f8c45716b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c1fe6c550 0x560c1fe83d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:31.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.996+0000 7f8c45716b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8c341aa670 con 0x560c1fe68170 2024-06-10T01:16:31.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.996+0000 7f8c45716b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8c341abae0 con 0x560c1fe67850 2024-06-10T01:16:31.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.996+0000 7f8c45716b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8c341aad40 con 0x560c1fe6c550 2024-06-10T01:16:31.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.996+0000 7f8c45716b80 10 client.0 _ll_register_callbacks cb 0x560c1fd4a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:16:31.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.996+0000 7f8c33fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c1fe68170 0x560c1fe83810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:31.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.996+0000 7f8c33fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c1fe68170 0x560c1fe83810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38834/0 (socket says 172.21.15.7:38834) 2024-06-10T01:16:31.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.996+0000 7f8c33fff700 1 -- 172.21.15.7:0/1258237345 learned_addr learned my addr 172.21.15.7:0/1258237345 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:31.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.996+0000 7f8c38c2f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c1fe67850 0x560c1fe832d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:31.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.996+0000 7f8c39430700 1 --2- 172.21.15.7:0/1258237345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c1fe6c550 0x560c1fe83d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:31.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.996+0000 7f8c38c2f700 1 -- 172.21.15.7:0/1258237345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c1fe6c550 msgr2=0x560c1fe83d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:31.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.996+0000 7f8c38c2f700 1 --2- 172.21.15.7:0/1258237345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c1fe6c550 0x560c1fe83d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:31.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.996+0000 7f8c38c2f700 1 -- 172.21.15.7:0/1258237345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c1fe68170 msgr2=0x560c1fe83810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:31.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.996+0000 7f8c38c2f700 1 --2- 172.21.15.7:0/1258237345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c1fe68170 0x560c1fe83810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:31.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.996+0000 7f8c38c2f700 1 -- 172.21.15.7:0/1258237345 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c1fee9e20 con 0x560c1fe67850 2024-06-10T01:16:31.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.996+0000 7f8c38c2f700 1 --2- 172.21.15.7:0/1258237345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c1fe67850 0x560c1fe832d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800c2d0 tx=0x7f8c28009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=287cf3a824ce1364 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:31.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.996+0000 7f8c13fff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:16:31.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.996+0000 7f8c13fff700 1 -- 172.21.15.7:0/1258237345 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2809c150 con 0x560c1fe67850 2024-06-10T01:16:31.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.996+0000 7f8c45716b80 1 -- 172.21.15.7:0/1258237345 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c2000f460 con 0x560c1fe67850 2024-06-10T01:16:31.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.996+0000 7f8c13fff700 1 -- 172.21.15.7:0/1258237345 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f8c28010720 con 0x560c1fe67850 2024-06-10T01:16:31.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.997+0000 7f8c13fff700 1 -- 172.21.15.7:0/1258237345 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8c280b8030 con 0x560c1fe67850 2024-06-10T01:16:31.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.997+0000 7f8c45716b80 10 client.27473 fetch_fsmap learned FSMap version 39 2024-06-10T01:16:31.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.997+0000 7f8c45716b80 1 -- 172.21.15.7:0/1258237345 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560c1fee9e20 con 0x560c1fe67850 2024-06-10T01:16:31.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.997+0000 7f8c13fff700 1 -- 172.21.15.7:0/1258237345 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c2000f460 con 0x560c1fe67850 2024-06-10T01:16:31.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.997+0000 7f8c13fff700 1 -- 172.21.15.7:0/1258237345 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7f8c280a0f90 con 0x560c1fe67850 2024-06-10T01:16:31.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.997+0000 7f8c45716b80 10 client.27473 fetch_fsmap finished waiting for FSMap version 39 2024-06-10T01:16:31.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.997+0000 7f8c45716b80 10 client.27473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:16:31.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.997+0000 7f8c45716b80 20 client.27473 populate_metadata read hostname 'smithi007' 2024-06-10T01:16:31.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.997+0000 7f8c45716b80 1 --2- 172.21.15.7:0/1258237345 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560c1feeb340 0x560c1feed730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:31.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.997+0000 7f8c45716b80 4 client.27473 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:16:31.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.997+0000 7f8c45716b80 1 -- 172.21.15.7:0/1258237345 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560c1fdbc2d0 con 0x560c1feeb340 2024-06-10T01:16:31.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.997+0000 7f8c33fff700 1 --2- 172.21.15.7:0/1258237345 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560c1feeb340 0x560c1feed730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:31.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.998+0000 7f8c13fff700 10 client.27473 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:31.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.998+0000 7f8c33fff700 1 --2- 172.21.15.7:0/1258237345 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560c1feeb340 0x560c1feed730 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca0ad527723833a7 server_cookie=b685a266a4be2d8a in_seq=0 out_seq=0 2024-06-10T01:16:31.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.998+0000 7f8c13fff700 1 -- 172.21.15.7:0/1258237345 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560c1fdbc2d0 con 0x560c1feeb340 2024-06-10T01:16:31.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.998+0000 7f8c13fff700 10 client.27473 handle_command_reply: tid=0 2024-06-10T01:16:31.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.998+0000 7f8c45716b80 1 client.27473 shutdown 2024-06-10T01:16:31.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.998+0000 7f8c45716b80 1 -- 172.21.15.7:0/1258237345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c1fe67850 msgr2=0x560c1fe832d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:31.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.998+0000 7f8c45716b80 1 --2- 172.21.15.7:0/1258237345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c1fe67850 0x560c1fe832d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800c2d0 tx=0x7f8c28009790 comp rx=0 tx=0).stop 2024-06-10T01:16:31.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.998+0000 7f8c45716b80 1 -- 172.21.15.7:0/1258237345 shutdown_connections 2024-06-10T01:16:31.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.998+0000 7f8c45716b80 1 --2- 172.21.15.7:0/1258237345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560c1fe6c550 0x560c1fe83d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:31.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.998+0000 7f8c45716b80 1 --2- 172.21.15.7:0/1258237345 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560c1feeb340 0x560c1feed730 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:31.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.998+0000 7f8c45716b80 1 --2- 172.21.15.7:0/1258237345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c1fe67850 0x560c1fe832d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:31.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.998+0000 7f8c45716b80 1 --2- 172.21.15.7:0/1258237345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c1fe68170 0x560c1fe83810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:31.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.998+0000 7f8c45716b80 1 -- 172.21.15.7:0/1258237345 >> 172.21.15.7:0/1258237345 conn(0x560c1fd90080 msgr2=0x560c1fe6b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:31.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.998+0000 7f8c13fff700 0 client.27473 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:31.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.998+0000 7f8c45716b80 1 -- 172.21.15.7:0/1258237345 shutdown_connections 2024-06-10T01:16:31.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.998+0000 7f8c45716b80 1 -- 172.21.15.7:0/1258237345 wait complete. 2024-06-10T01:16:31.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.999+0000 7f8c45716b80 20 client.27473 trim_cache size 0 max 16384 2024-06-10T01:16:31.998 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:16:31.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.999+0000 7f8c3ae93700 1 -- 172.21.15.7:0/2769857319 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8c240a2c20 msgr2=0x7f8c240a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:31.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.999+0000 7f8c3ae93700 1 --2- 172.21.15.7:0/2769857319 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8c240a2c20 0x7f8c240a50e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8c34079b50 tx=0x7f8c20005fb0 comp rx=0 tx=0).stop 2024-06-10T01:16:31.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.999+0000 7f8c3ae93700 1 -- 172.21.15.7:0/2769857319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c34102550 msgr2=0x7f8c3419e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:31.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.999+0000 7f8c3ae93700 1 --2- 172.21.15.7:0/2769857319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c34102550 0x7f8c3419e810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800e9a0 tx=0x7f8c2800ecb0 comp rx=0 tx=0).stop 2024-06-10T01:16:31.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.999+0000 7f8c3ae93700 1 -- 172.21.15.7:0/2769857319 shutdown_connections 2024-06-10T01:16:32.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.999+0000 7f8c3ae93700 1 --2- 172.21.15.7:0/2769857319 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8c240a2c20 0x7f8c240a50e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:32.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.999+0000 7f8c3ae93700 1 --2- 172.21.15.7:0/2769857319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c341073a0 0x7f8c34198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:32.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.999+0000 7f8c3ae93700 1 --2- 172.21.15.7:0/2769857319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c34102550 0x7f8c3419e810 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:32.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.999+0000 7f8c3ae93700 1 --2- 172.21.15.7:0/2769857319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c34102f10 0x7f8c3419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:32.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:31.999+0000 7f8c3ae93700 1 -- 172.21.15.7:0/2769857319 >> 172.21.15.7:0/2769857319 conn(0x7f8c34079650 msgr2=0x7f8c341005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:32.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:32.000+0000 7f8c3ae93700 1 -- 172.21.15.7:0/2769857319 shutdown_connections 2024-06-10T01:16:32.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:32.000+0000 7f8c3ae93700 1 -- 172.21.15.7:0/2769857319 wait complete. 2024-06-10T01:16:32.748 INFO:tasks.workunit.client.0.smithi007.stdout: 1 776811 7.54 MB/sec execute 433 sec latency 17.028 ms 2024-06-10T01:16:33.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:32 smithi175 ceph-mon[36636]: pgmap v748: 129 pgs: 129 active+clean; 132 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 318 op/s 2024-06-10T01:16:33.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:32 smithi007 ceph-mon[32398]: pgmap v748: 129 pgs: 129 active+clean; 132 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 318 op/s 2024-06-10T01:16:33.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:32 smithi078 ceph-mon[36683]: pgmap v748: 129 pgs: 129 active+clean; 132 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 318 op/s 2024-06-10T01:16:33.748 INFO:tasks.workunit.client.0.smithi007.stdout: 1 778371 7.54 MB/sec execute 434 sec latency 14.879 ms 2024-06-10T01:16:33.814 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:16:33.815 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:16:34.079 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:16:34.748 INFO:tasks.workunit.client.0.smithi007.stdout: 1 780778 7.55 MB/sec execute 435 sec latency 11.424 ms 2024-06-10T01:16:35.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:34 smithi175 ceph-mon[36636]: pgmap v749: 129 pgs: 129 active+clean; 132 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 2.6 MiB/s wr, 277 op/s 2024-06-10T01:16:35.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:34 smithi007 ceph-mon[32398]: pgmap v749: 129 pgs: 129 active+clean; 132 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 2.6 MiB/s wr, 277 op/s 2024-06-10T01:16:35.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:34 smithi078 ceph-mon[36683]: pgmap v749: 129 pgs: 129 active+clean; 132 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 2.6 MiB/s wr, 277 op/s 2024-06-10T01:16:35.748 INFO:tasks.workunit.client.0.smithi007.stdout: 1 781815 7.55 MB/sec execute 436 sec latency 74.925 ms 2024-06-10T01:16:35.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.877+0000 7f4756708700 1 -- 172.21.15.7:0/628367576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4750101680 msgr2=0x7f4750101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:35.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.877+0000 7f4756708700 1 --2- 172.21.15.7:0/628367576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4750101680 0x7f4750101a60 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f47400099c0 tx=0x7f4740009cd0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.877+0000 7f4756708700 1 -- 172.21.15.7:0/628367576 shutdown_connections 2024-06-10T01:16:35.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.877+0000 7f4756708700 1 --2- 172.21.15.7:0/628367576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47501066d0 0x7f475010e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.877+0000 7f4756708700 1 --2- 172.21.15.7:0/628367576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4750101680 0x7f4750101a60 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.877+0000 7f4756708700 1 --2- 172.21.15.7:0/628367576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4750102030 0x7f4750105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.877+0000 7f4756708700 1 -- 172.21.15.7:0/628367576 >> 172.21.15.7:0/628367576 conn(0x7f47500773e0 msgr2=0x7f47500777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:35.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.877+0000 7f4756708700 1 -- 172.21.15.7:0/628367576 shutdown_connections 2024-06-10T01:16:35.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.877+0000 7f4756708700 1 -- 172.21.15.7:0/628367576 wait complete. 2024-06-10T01:16:35.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.878+0000 7f4756708700 1 Processor -- start 2024-06-10T01:16:35.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.878+0000 7f4756708700 1 -- start start 2024-06-10T01:16:35.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.878+0000 7f4756708700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4750101680 0x7f475019e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:35.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.878+0000 7f4756708700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4750102030 0x7f475019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:35.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.878+0000 7f4756708700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47501066d0 0x7f4750198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:35.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.879+0000 7f4756708700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f47501116b0 con 0x7f4750101680 2024-06-10T01:16:35.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.879+0000 7f4756708700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4750111530 con 0x7f47501066d0 2024-06-10T01:16:35.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.879+0000 7f4756708700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4750111830 con 0x7f4750102030 2024-06-10T01:16:35.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.879+0000 7f474ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4750101680 0x7f475019e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:35.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.879+0000 7f474ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4750101680 0x7f475019e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38848/0 (socket says 172.21.15.7:38848) 2024-06-10T01:16:35.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.879+0000 7f474f7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4750102030 0x7f475019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:35.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.879+0000 7f474ffff700 1 -- 172.21.15.7:0/3025232127 learned_addr learned my addr 172.21.15.7:0/3025232127 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:35.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.879+0000 7f4754ca5700 1 --2- 172.21.15.7:0/3025232127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47501066d0 0x7f4750198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:35.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.879+0000 7f474ffff700 1 -- 172.21.15.7:0/3025232127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4750102030 msgr2=0x7f475019ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:35.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.879+0000 7f474ffff700 1 --2- 172.21.15.7:0/3025232127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4750102030 0x7f475019ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.879+0000 7f474ffff700 1 -- 172.21.15.7:0/3025232127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47501066d0 msgr2=0x7f4750198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:35.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.879+0000 7f474ffff700 1 --2- 172.21.15.7:0/3025232127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47501066d0 0x7f4750198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.879+0000 7f474ffff700 1 -- 172.21.15.7:0/3025232127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4750199210 con 0x7f4750101680 2024-06-10T01:16:35.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.879+0000 7f474f7fe700 1 --2- 172.21.15.7:0/3025232127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4750102030 0x7f475019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:35.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.879+0000 7f4754ca5700 1 --2- 172.21.15.7:0/3025232127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47501066d0 0x7f4750198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:35.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.879+0000 7f474ffff700 1 --2- 172.21.15.7:0/3025232127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4750101680 0x7f475019e8f0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f4740006010 tx=0x7f474000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:35.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.879+0000 7f474d7fa700 1 -- 172.21.15.7:0/3025232127 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4740003680 con 0x7f4750101680 2024-06-10T01:16:35.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.879+0000 7f4756708700 1 -- 172.21.15.7:0/3025232127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47501994a0 con 0x7f4750101680 2024-06-10T01:16:35.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.879+0000 7f474d7fa700 1 -- 172.21.15.7:0/3025232127 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47400050d0 con 0x7f4750101680 2024-06-10T01:16:35.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.879+0000 7f474d7fa700 1 -- 172.21.15.7:0/3025232127 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4740003c80 con 0x7f4750101680 2024-06-10T01:16:35.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.879+0000 7f4756708700 1 -- 172.21.15.7:0/3025232127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4750102fc0 con 0x7f4750101680 2024-06-10T01:16:35.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.882+0000 7f474d7fa700 1 -- 172.21.15.7:0/3025232127 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f474002b440 con 0x7f4750101680 2024-06-10T01:16:35.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.882+0000 7f474d7fa700 1 --2- 172.21.15.7:0/3025232127 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f47380a2dc0 0x7f47380a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:35.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.883+0000 7f474d7fa700 1 -- 172.21.15.7:0/3025232127 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f47400d0010 con 0x7f4750101680 2024-06-10T01:16:35.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.883+0000 7f474f7fe700 1 --2- 172.21.15.7:0/3025232127 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f47380a2dc0 0x7f47380a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:35.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.883+0000 7f4760f8bb80 1 Processor -- start 2024-06-10T01:16:35.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.883+0000 7f4760f8bb80 1 -- start start 2024-06-10T01:16:35.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.883+0000 7f474f7fe700 1 --2- 172.21.15.7:0/3025232127 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f47380a2dc0 0x7f47380a5280 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4750107b30 tx=0x7f4744009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:35.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.883+0000 7f4760f8bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562340140fe0 0x5623401413c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:35.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.883+0000 7f4760f8bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562340141900 0x56234013b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:35.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.883+0000 7f4760f8bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56234013b820 0x56234013bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:35.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.883+0000 7f4760f8bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56234002dfe0 con 0x56234013b820 2024-06-10T01:16:35.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.883+0000 7f4760f8bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56234008c8d0 con 0x562340141900 2024-06-10T01:16:35.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.883+0000 7f4760f8bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56233fff44e0 con 0x562340140fe0 2024-06-10T01:16:35.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.883+0000 7f474ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562340140fe0 0x5623401413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:35.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.883+0000 7f4754ca5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56234013b820 0x56234013bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:35.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.883+0000 7f474f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562340141900 0x56234013b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:35.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.883+0000 7f474ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562340140fe0 0x5623401413c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:59864/0 (socket says 172.21.15.7:59864) 2024-06-10T01:16:35.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.883+0000 7f474ffff700 1 -- 172.21.15.7:0/1382924497 learned_addr learned my addr 172.21.15.7:0/1382924497 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:35.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.884+0000 7f4754ca5700 1 -- 172.21.15.7:0/1382924497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562340140fe0 msgr2=0x5623401413c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:35.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.884+0000 7f4754ca5700 1 --2- 172.21.15.7:0/1382924497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562340140fe0 0x5623401413c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.884+0000 7f4754ca5700 1 -- 172.21.15.7:0/1382924497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562340141900 msgr2=0x56234013b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:35.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.884+0000 7f4754ca5700 1 --2- 172.21.15.7:0/1382924497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562340141900 0x56234013b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.884+0000 7f4754ca5700 1 -- 172.21.15.7:0/1382924497 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56233ffdb950 con 0x56234013b820 2024-06-10T01:16:35.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.884+0000 7f474ffff700 1 --2- 172.21.15.7:0/1382924497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562340140fe0 0x5623401413c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:35.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.884+0000 7f4754ca5700 1 --2- 172.21.15.7:0/1382924497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56234013b820 0x56234013bc40 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f4748010bd0 tx=0x7f474800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ece72288c1962b3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:35.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.884+0000 7f47367fc700 1 -- 172.21.15.7:0/1382924497 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4748007700 con 0x56234013b820 2024-06-10T01:16:35.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.884+0000 7f47367fc700 1 -- 172.21.15.7:0/1382924497 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4748020070 con 0x56234013b820 2024-06-10T01:16:35.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.884+0000 7f47367fc700 1 -- 172.21.15.7:0/1382924497 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4748015bf0 con 0x56234013b820 2024-06-10T01:16:35.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.885+0000 7f4760f8bb80 1 -- 172.21.15.7:0/1382924497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56234013b820 msgr2=0x56234013bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:35.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.885+0000 7f4760f8bb80 1 --2- 172.21.15.7:0/1382924497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56234013b820 0x56234013bc40 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f4748010bd0 tx=0x7f474800b040 comp rx=0 tx=0).stop 2024-06-10T01:16:35.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.885+0000 7f4760f8bb80 1 -- 172.21.15.7:0/1382924497 shutdown_connections 2024-06-10T01:16:35.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.885+0000 7f4760f8bb80 1 --2- 172.21.15.7:0/1382924497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56234013b820 0x56234013bc40 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.885+0000 7f4760f8bb80 1 --2- 172.21.15.7:0/1382924497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562340140fe0 0x5623401413c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.885+0000 7f4760f8bb80 1 --2- 172.21.15.7:0/1382924497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562340141900 0x56234013b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.885+0000 7f4760f8bb80 1 -- 172.21.15.7:0/1382924497 >> 172.21.15.7:0/1382924497 conn(0x562340068100 msgr2=0x5623401362a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:35.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.885+0000 7f4760f8bb80 1 -- 172.21.15.7:0/1382924497 shutdown_connections 2024-06-10T01:16:35.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.885+0000 7f4760f8bb80 1 -- 172.21.15.7:0/1382924497 wait complete. 2024-06-10T01:16:35.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.885+0000 7f4760f8bb80 1 -- start start 2024-06-10T01:16:35.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.886+0000 7f4760f8bb80 1 Processor -- start 2024-06-10T01:16:35.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.886+0000 7f4760f8bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56234013b820 0x5623401475f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:35.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.886+0000 7f4760f8bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562340140fe0 0x5623401448f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:35.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.886+0000 7f4760f8bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562340141900 0x562340144e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:35.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.886+0000 7f4760f8bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56234002d040 con 0x562340140fe0 2024-06-10T01:16:35.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.886+0000 7f4760f8bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562340088890 con 0x56234013b820 2024-06-10T01:16:35.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.886+0000 7f4760f8bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562340148280 con 0x562340141900 2024-06-10T01:16:35.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.886+0000 7f4760f8bb80 10 client.0 _ll_register_callbacks cb 0x562340022630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:16:35.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.886+0000 7f474f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562340140fe0 0x5623401448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:35.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.886+0000 7f474f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562340140fe0 0x5623401448f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38862/0 (socket says 172.21.15.7:38862) 2024-06-10T01:16:35.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.886+0000 7f474f7fe700 1 -- 172.21.15.7:0/1274439921 learned_addr learned my addr 172.21.15.7:0/1274439921 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:35.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.886+0000 7f474ffff700 1 --2- 172.21.15.7:0/1274439921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56234013b820 0x5623401475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:35.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.886+0000 7f4754ca5700 1 --2- 172.21.15.7:0/1274439921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562340141900 0x562340144e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:35.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.886+0000 7f474ffff700 1 -- 172.21.15.7:0/1274439921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562340141900 msgr2=0x562340144e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:35.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.886+0000 7f474ffff700 1 --2- 172.21.15.7:0/1274439921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562340141900 0x562340144e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.886+0000 7f474ffff700 1 -- 172.21.15.7:0/1274439921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562340140fe0 msgr2=0x5623401448f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:35.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.887+0000 7f474ffff700 1 --2- 172.21.15.7:0/1274439921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562340140fe0 0x5623401448f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.887+0000 7f474ffff700 1 -- 172.21.15.7:0/1274439921 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623400527d0 con 0x56234013b820 2024-06-10T01:16:35.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.887+0000 7f474f7fe700 1 --2- 172.21.15.7:0/1274439921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562340140fe0 0x5623401448f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:35.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.887+0000 7f474ffff700 1 --2- 172.21.15.7:0/1274439921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56234013b820 0x5623401475f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4740098f50 tx=0x7f474009c140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c481c956b78ed08f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:35.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.887+0000 7f472f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:16:35.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.887+0000 7f472f7fe700 1 -- 172.21.15.7:0/1274439921 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474009c580 con 0x56234013b820 2024-06-10T01:16:35.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.887+0000 7f472f7fe700 1 -- 172.21.15.7:0/1274439921 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f474009c7e0 con 0x56234013b820 2024-06-10T01:16:35.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.887+0000 7f4760f8bb80 1 -- 172.21.15.7:0/1274439921 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4748020070 con 0x56234013b820 2024-06-10T01:16:35.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.887+0000 7f472f7fe700 1 -- 172.21.15.7:0/1274439921 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f47400a0930 con 0x56234013b820 2024-06-10T01:16:35.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.887+0000 7f4760f8bb80 10 client.27485 fetch_fsmap learned FSMap version 39 2024-06-10T01:16:35.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.887+0000 7f4760f8bb80 1 -- 172.21.15.7:0/1274439921 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562340152990 con 0x56234013b820 2024-06-10T01:16:35.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.887+0000 7f472f7fe700 1 -- 172.21.15.7:0/1274439921 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4748020070 con 0x56234013b820 2024-06-10T01:16:35.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.887+0000 7f472f7fe700 1 -- 172.21.15.7:0/1274439921 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7f474009ba60 con 0x56234013b820 2024-06-10T01:16:35.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.888+0000 7f4760f8bb80 10 client.27485 fetch_fsmap finished waiting for FSMap version 39 2024-06-10T01:16:35.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.888+0000 7f4760f8bb80 10 client.27485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:16:35.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.888+0000 7f4760f8bb80 20 client.27485 populate_metadata read hostname 'smithi007' 2024-06-10T01:16:35.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.888+0000 7f4760f8bb80 1 --2- 172.21.15.7:0/1274439921 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562340154f50 0x562340157390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:35.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.888+0000 7f4760f8bb80 4 client.27485 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:16:35.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.888+0000 7f4760f8bb80 1 -- 172.21.15.7:0/1274439921 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562340099dd0 con 0x562340154f50 2024-06-10T01:16:35.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.888+0000 7f474f7fe700 1 --2- 172.21.15.7:0/1274439921 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562340154f50 0x562340157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:35.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.888+0000 7f474f7fe700 1 --2- 172.21.15.7:0/1274439921 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562340154f50 0x562340157390 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5daaffc314505fb1 server_cookie=9471dd9a2d524053 in_seq=0 out_seq=0 2024-06-10T01:16:35.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.888+0000 7f472f7fe700 10 client.27485 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:35.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.889+0000 7f472f7fe700 1 -- 172.21.15.7:0/1274439921 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562340099dd0 con 0x562340154f50 2024-06-10T01:16:35.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.889+0000 7f472f7fe700 10 client.27485 handle_command_reply: tid=0 2024-06-10T01:16:35.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.889+0000 7f4760f8bb80 1 client.27485 shutdown 2024-06-10T01:16:35.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.889+0000 7f4760f8bb80 1 -- 172.21.15.7:0/1274439921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56234013b820 msgr2=0x5623401475f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:35.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.889+0000 7f4760f8bb80 1 --2- 172.21.15.7:0/1274439921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56234013b820 0x5623401475f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4740098f50 tx=0x7f474009c140 comp rx=0 tx=0).stop 2024-06-10T01:16:35.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.889+0000 7f4760f8bb80 1 -- 172.21.15.7:0/1274439921 shutdown_connections 2024-06-10T01:16:35.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.889+0000 7f4760f8bb80 1 --2- 172.21.15.7:0/1274439921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562340141900 0x562340144e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.889+0000 7f4760f8bb80 1 --2- 172.21.15.7:0/1274439921 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562340154f50 0x562340157390 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.889+0000 7f472f7fe700 0 client.27485 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:35.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.889+0000 7f4760f8bb80 1 --2- 172.21.15.7:0/1274439921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56234013b820 0x5623401475f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.889+0000 7f4760f8bb80 1 --2- 172.21.15.7:0/1274439921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562340140fe0 0x5623401448f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.889+0000 7f4760f8bb80 1 -- 172.21.15.7:0/1274439921 >> 172.21.15.7:0/1274439921 conn(0x562340068100 msgr2=0x562340135780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:35.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.890+0000 7f4760f8bb80 1 -- 172.21.15.7:0/1274439921 shutdown_connections 2024-06-10T01:16:35.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.890+0000 7f4760f8bb80 1 -- 172.21.15.7:0/1274439921 wait complete. 2024-06-10T01:16:35.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.890+0000 7f4760f8bb80 20 client.27485 trim_cache size 0 max 16384 2024-06-10T01:16:35.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.913+0000 7f4760f8bb80 1 Processor -- start 2024-06-10T01:16:35.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.913+0000 7f4760f8bb80 1 -- start start 2024-06-10T01:16:35.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.914+0000 7f4760f8bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56234013b820 0x56234013b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:35.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.914+0000 7f4760f8bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562340140fe0 0x562340135f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:35.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.914+0000 7f4760f8bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562340141900 0x562340136440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:35.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.914+0000 7f4760f8bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f47501ae8c0 con 0x562340140fe0 2024-06-10T01:16:35.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.914+0000 7f4760f8bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f47501ac120 con 0x562340141900 2024-06-10T01:16:35.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.914+0000 7f4760f8bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f47501a9f90 con 0x56234013b820 2024-06-10T01:16:35.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.914+0000 7f474f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562340140fe0 0x562340135f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:35.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.914+0000 7f474f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562340140fe0 0x562340135f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38870/0 (socket says 172.21.15.7:38870) 2024-06-10T01:16:35.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.914+0000 7f4754ca5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562340141900 0x562340136440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:35.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.914+0000 7f474ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56234013b820 0x56234013b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:35.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.914+0000 7f4754ca5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562340141900 0x562340136440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:36364/0 (socket says 172.21.15.7:36364) 2024-06-10T01:16:35.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.914+0000 7f4754ca5700 1 -- 172.21.15.7:0/2797201623 learned_addr learned my addr 172.21.15.7:0/2797201623 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:35.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.914+0000 7f4754ca5700 1 -- 172.21.15.7:0/2797201623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56234013b820 msgr2=0x56234013b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:35.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.914+0000 7f4754ca5700 1 --2- 172.21.15.7:0/2797201623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56234013b820 0x56234013b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.914+0000 7f4754ca5700 1 -- 172.21.15.7:0/2797201623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562340140fe0 msgr2=0x562340135f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:35.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.914+0000 7f4754ca5700 1 --2- 172.21.15.7:0/2797201623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562340140fe0 0x562340135f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.914+0000 7f4754ca5700 1 -- 172.21.15.7:0/2797201623 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562340157970 con 0x562340141900 2024-06-10T01:16:35.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.914+0000 7f474ffff700 1 --2- 172.21.15.7:0/2797201623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56234013b820 0x56234013b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:35.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.915+0000 7f4754ca5700 1 --2- 172.21.15.7:0/2797201623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562340141900 0x562340136440 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f47480157e0 tx=0x7f4748015810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=883599e04d3e7202 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:35.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.915+0000 7f4735ffb700 1 -- 172.21.15.7:0/2797201623 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4748003c00 con 0x562340141900 2024-06-10T01:16:35.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.915+0000 7f4735ffb700 1 -- 172.21.15.7:0/2797201623 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4748020040 con 0x562340141900 2024-06-10T01:16:35.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.915+0000 7f4735ffb700 1 -- 172.21.15.7:0/2797201623 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4748007900 con 0x562340141900 2024-06-10T01:16:35.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.916+0000 7f4760f8bb80 1 -- 172.21.15.7:0/2797201623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562340141900 msgr2=0x562340136440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:35.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.916+0000 7f4760f8bb80 1 --2- 172.21.15.7:0/2797201623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562340141900 0x562340136440 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f47480157e0 tx=0x7f4748015810 comp rx=0 tx=0).stop 2024-06-10T01:16:35.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.916+0000 7f4760f8bb80 1 -- 172.21.15.7:0/2797201623 shutdown_connections 2024-06-10T01:16:35.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.916+0000 7f4760f8bb80 1 --2- 172.21.15.7:0/2797201623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562340141900 0x562340136440 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.916+0000 7f4760f8bb80 1 --2- 172.21.15.7:0/2797201623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56234013b820 0x56234013b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.916+0000 7f4760f8bb80 1 --2- 172.21.15.7:0/2797201623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562340140fe0 0x562340135f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.916+0000 7f4760f8bb80 1 -- 172.21.15.7:0/2797201623 >> 172.21.15.7:0/2797201623 conn(0x562340068100 msgr2=0x5623401441a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:35.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.916+0000 7f4760f8bb80 1 -- 172.21.15.7:0/2797201623 shutdown_connections 2024-06-10T01:16:35.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.916+0000 7f4760f8bb80 1 -- 172.21.15.7:0/2797201623 wait complete. 2024-06-10T01:16:35.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.916+0000 7f4760f8bb80 1 -- start start 2024-06-10T01:16:35.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.916+0000 7f4760f8bb80 1 Processor -- start 2024-06-10T01:16:35.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.916+0000 7f4760f8bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56234013b820 0x56234014a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:35.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.917+0000 7f4760f8bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562340140fe0 0x56234014aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:35.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.917+0000 7f4760f8bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562340141900 0x56234014af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:35.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.917+0000 7f4760f8bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f47501aa5e0 con 0x562340141900 2024-06-10T01:16:35.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.917+0000 7f4760f8bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f47501aba50 con 0x56234013b820 2024-06-10T01:16:35.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.917+0000 7f4760f8bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f47501aacb0 con 0x562340140fe0 2024-06-10T01:16:35.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.917+0000 7f4760f8bb80 10 client.0 _ll_register_callbacks cb 0x562340022630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:16:35.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.917+0000 7f4754ca5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562340141900 0x56234014af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:35.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.917+0000 7f474f7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562340140fe0 0x56234014aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:35.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.917+0000 7f474ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56234013b820 0x56234014a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:35.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.917+0000 7f4754ca5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562340141900 0x56234014af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38874/0 (socket says 172.21.15.7:38874) 2024-06-10T01:16:35.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.917+0000 7f474f7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562340140fe0 0x56234014aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:59898/0 (socket says 172.21.15.7:59898) 2024-06-10T01:16:35.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.917+0000 7f4754ca5700 1 -- 172.21.15.7:0/2748191358 learned_addr learned my addr 172.21.15.7:0/2748191358 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:35.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.917+0000 7f474ffff700 1 -- 172.21.15.7:0/2748191358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562340140fe0 msgr2=0x56234014aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:35.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.917+0000 7f474ffff700 1 --2- 172.21.15.7:0/2748191358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562340140fe0 0x56234014aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.917+0000 7f474ffff700 1 -- 172.21.15.7:0/2748191358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562340141900 msgr2=0x56234014af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:35.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.917+0000 7f474ffff700 1 --2- 172.21.15.7:0/2748191358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562340141900 0x56234014af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.917+0000 7f474ffff700 1 -- 172.21.15.7:0/2748191358 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623401bfe00 con 0x56234013b820 2024-06-10T01:16:35.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.917+0000 7f4754ca5700 1 --2- 172.21.15.7:0/2748191358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562340141900 0x56234014af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:35.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.917+0000 7f474ffff700 1 --2- 172.21.15.7:0/2748191358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56234013b820 0x56234014a510 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4740000c00 tx=0x7f474009aa90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e64689a8a8d8c28c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:35.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.917+0000 7f472f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:16:35.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.917+0000 7f472f7fe700 1 -- 172.21.15.7:0/2748191358 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474009bee0 con 0x56234013b820 2024-06-10T01:16:35.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.917+0000 7f472f7fe700 1 -- 172.21.15.7:0/2748191358 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f47400a4ed0 con 0x56234013b820 2024-06-10T01:16:35.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.917+0000 7f4760f8bb80 1 -- 172.21.15.7:0/2748191358 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4748020040 con 0x56234013b820 2024-06-10T01:16:35.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.918+0000 7f472f7fe700 1 -- 172.21.15.7:0/2748191358 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f474009c580 con 0x56234013b820 2024-06-10T01:16:35.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.918+0000 7f4760f8bb80 10 client.27491 fetch_fsmap learned FSMap version 39 2024-06-10T01:16:35.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.918+0000 7f4760f8bb80 1 -- 172.21.15.7:0/2748191358 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5623401c1120 con 0x56234013b820 2024-06-10T01:16:35.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.918+0000 7f472f7fe700 1 -- 172.21.15.7:0/2748191358 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4748020040 con 0x56234013b820 2024-06-10T01:16:35.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.918+0000 7f472f7fe700 1 -- 172.21.15.7:0/2748191358 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7f47400982d0 con 0x56234013b820 2024-06-10T01:16:35.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.918+0000 7f4760f8bb80 10 client.27491 fetch_fsmap finished waiting for FSMap version 39 2024-06-10T01:16:35.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.918+0000 7f4760f8bb80 10 client.27491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:16:35.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.918+0000 7f4760f8bb80 20 client.27491 populate_metadata read hostname 'smithi007' 2024-06-10T01:16:35.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.918+0000 7f4760f8bb80 1 --2- 172.21.15.7:0/2748191358 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562340154f50 0x5623401c3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:35.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.918+0000 7f4760f8bb80 4 client.27491 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:16:35.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.918+0000 7f4760f8bb80 1 -- 172.21.15.7:0/2748191358 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562340094350 con 0x562340154f50 2024-06-10T01:16:35.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.918+0000 7f474f7fe700 1 --2- 172.21.15.7:0/2748191358 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562340154f50 0x5623401c3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:35.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.919+0000 7f474f7fe700 1 --2- 172.21.15.7:0/2748191358 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562340154f50 0x5623401c3680 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49cf8c3e8ac3d7e0 server_cookie=5779c69d19d720b5 in_seq=0 out_seq=0 2024-06-10T01:16:35.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.919+0000 7f472f7fe700 10 client.27491 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:35.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.919+0000 7f472f7fe700 1 -- 172.21.15.7:0/2748191358 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562340094350 con 0x562340154f50 2024-06-10T01:16:35.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.919+0000 7f472f7fe700 10 client.27491 handle_command_reply: tid=0 2024-06-10T01:16:35.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.919+0000 7f4760f8bb80 1 client.27491 shutdown 2024-06-10T01:16:35.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.920+0000 7f4760f8bb80 1 -- 172.21.15.7:0/2748191358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56234013b820 msgr2=0x56234014a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:35.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.920+0000 7f4760f8bb80 1 --2- 172.21.15.7:0/2748191358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56234013b820 0x56234014a510 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4740000c00 tx=0x7f474009aa90 comp rx=0 tx=0).stop 2024-06-10T01:16:35.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.920+0000 7f4760f8bb80 1 -- 172.21.15.7:0/2748191358 shutdown_connections 2024-06-10T01:16:35.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.920+0000 7f4760f8bb80 1 --2- 172.21.15.7:0/2748191358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562340141900 0x56234014af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.920+0000 7f4760f8bb80 1 --2- 172.21.15.7:0/2748191358 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562340154f50 0x5623401c3680 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.920+0000 7f4760f8bb80 1 --2- 172.21.15.7:0/2748191358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56234013b820 0x56234014a510 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.920+0000 7f4760f8bb80 1 --2- 172.21.15.7:0/2748191358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562340140fe0 0x56234014aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.920+0000 7f472f7fe700 0 client.27491 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:35.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.920+0000 7f4760f8bb80 1 -- 172.21.15.7:0/2748191358 >> 172.21.15.7:0/2748191358 conn(0x562340068100 msgr2=0x562340144a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:35.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.920+0000 7f4760f8bb80 1 -- 172.21.15.7:0/2748191358 shutdown_connections 2024-06-10T01:16:35.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.920+0000 7f4760f8bb80 1 -- 172.21.15.7:0/2748191358 wait complete. 2024-06-10T01:16:35.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.920+0000 7f4760f8bb80 20 client.27491 trim_cache size 0 max 16384 2024-06-10T01:16:35.922 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:16:35.922 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-10T01:16:35.922 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "fdcbb000-3493-460b-b23a-adb0a4a8881e", 2024-06-10T01:16:35.922 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-10T01:16:35.922 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:16:35.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.921+0000 7f4756708700 1 -- 172.21.15.7:0/3025232127 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f47380a2dc0 msgr2=0x7f47380a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:35.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.921+0000 7f4756708700 1 --2- 172.21.15.7:0/3025232127 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f47380a2dc0 0x7f47380a5280 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4750107b30 tx=0x7f4744009260 comp rx=0 tx=0).stop 2024-06-10T01:16:35.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.921+0000 7f4756708700 1 -- 172.21.15.7:0/3025232127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4750101680 msgr2=0x7f475019e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:35.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.921+0000 7f4756708700 1 --2- 172.21.15.7:0/3025232127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4750101680 0x7f475019e8f0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f4740006010 tx=0x7f474000f690 comp rx=0 tx=0).stop 2024-06-10T01:16:35.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.921+0000 7f4756708700 1 -- 172.21.15.7:0/3025232127 shutdown_connections 2024-06-10T01:16:35.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.921+0000 7f4756708700 1 --2- 172.21.15.7:0/3025232127 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f47380a2dc0 0x7f47380a5280 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.921+0000 7f4756708700 1 --2- 172.21.15.7:0/3025232127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47501066d0 0x7f4750198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.921+0000 7f4756708700 1 --2- 172.21.15.7:0/3025232127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4750101680 0x7f475019e8f0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.921+0000 7f4756708700 1 --2- 172.21.15.7:0/3025232127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4750102030 0x7f475019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:35.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.921+0000 7f4756708700 1 -- 172.21.15.7:0/3025232127 >> 172.21.15.7:0/3025232127 conn(0x7f47500773e0 msgr2=0x7f4750100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:35.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.921+0000 7f4756708700 1 -- 172.21.15.7:0/3025232127 shutdown_connections 2024-06-10T01:16:35.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:35.921+0000 7f4756708700 1 -- 172.21.15.7:0/3025232127 wait complete. 2024-06-10T01:16:36.731 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:16:36.748 INFO:tasks.workunit.client.0.smithi007.stdout: 1 783913 7.56 MB/sec execute 437 sec latency 16.723 ms 2024-06-10T01:16:36.997 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:16:37.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:36 smithi175 ceph-mon[36636]: pgmap v750: 129 pgs: 129 active+clean; 133 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 2.6 MiB/s wr, 303 op/s 2024-06-10T01:16:37.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:36 smithi175 ceph-mon[36636]: scrub queued for path: / 2024-06-10T01:16:37.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:36 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:16:37.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:36 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:16:37.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:36 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:16:37.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:36 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:16:37.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:36 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:16:37.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:36 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:16:37.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:36 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:16:37.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:36 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:16:37.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:36 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:16:37.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:36 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:16:37.203 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:36 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:16:37.242 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:36 smithi007 ceph-mon[32398]: pgmap v750: 129 pgs: 129 active+clean; 133 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 2.6 MiB/s wr, 303 op/s 2024-06-10T01:16:37.242 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:36 smithi007 ceph-mon[32398]: scrub queued for path: / 2024-06-10T01:16:37.242 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:36 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:16:37.242 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:36 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:16:37.243 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:36 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:16:37.243 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:36 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:16:37.243 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:36 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:16:37.243 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:36 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:16:37.243 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:36 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:16:37.243 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:36 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:16:37.243 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:36 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:16:37.243 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:36 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:16:37.243 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:36 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:16:37.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:36 smithi078 ceph-mon[36683]: pgmap v750: 129 pgs: 129 active+clean; 133 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 2.6 MiB/s wr, 303 op/s 2024-06-10T01:16:37.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:36 smithi078 ceph-mon[36683]: scrub queued for path: / 2024-06-10T01:16:37.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:36 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:16:37.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:36 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:16:37.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:36 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:16:37.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:36 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:16:37.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:36 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:16:37.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:36 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:16:37.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:36 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:16:37.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:36 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:16:37.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:36 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:16:37.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:36 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:16:37.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:36 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:16:37.749 INFO:tasks.workunit.client.0.smithi007.stdout: 1 785501 7.56 MB/sec execute 438 sec latency 11.489 ms 2024-06-10T01:16:38.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.478+0000 7f4f34476640 1 -- 172.21.15.7:0/4001684854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f2c102480 msgr2=0x7f4f2c104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:38.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.478+0000 7f4f34476640 1 --2- 172.21.15.7:0/4001684854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f2c102480 0x7f4f2c104870 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009900 tx=0x7f4f2802a410 comp rx=0 tx=0).stop 2024-06-10T01:16:38.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.478+0000 7f4f34476640 1 -- 172.21.15.7:0/4001684854 shutdown_connections 2024-06-10T01:16:38.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.478+0000 7f4f34476640 1 --2- 172.21.15.7:0/4001684854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f2c1076e0 0x7f4f2c109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.478+0000 7f4f34476640 1 --2- 172.21.15.7:0/4001684854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f2c104db0 0x7f4f2c1071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.478+0000 7f4f34476640 1 --2- 172.21.15.7:0/4001684854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f2c102480 0x7f4f2c104870 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.478+0000 7f4f34476640 1 -- 172.21.15.7:0/4001684854 >> 172.21.15.7:0/4001684854 conn(0x7f4f2c0fbe70 msgr2=0x7f4f2c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.478+0000 7f4f34476640 1 -- 172.21.15.7:0/4001684854 shutdown_connections 2024-06-10T01:16:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.478+0000 7f4f34476640 1 -- 172.21.15.7:0/4001684854 wait complete. 2024-06-10T01:16:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.479+0000 7f4f34476640 1 Processor -- start 2024-06-10T01:16:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.479+0000 7f4f34476640 1 -- start start 2024-06-10T01:16:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.479+0000 7f4f34476640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f2c102480 0x7f4f2c197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.479+0000 7f4f34476640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f2c104db0 0x7f4f2c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.479+0000 7f4f34476640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f2c1076e0 0x7f4f2c19f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.479+0000 7f4f34476640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c10d6d0 con 0x7f4f2c102480 2024-06-10T01:16:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.479+0000 7f4f34476640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c10d550 con 0x7f4f2c104db0 2024-06-10T01:16:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.479+0000 7f4f34476640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c10d850 con 0x7f4f2c1076e0 2024-06-10T01:16:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.479+0000 7f4f32212640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f2c102480 0x7f4f2c197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.479+0000 7f4f31a11640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f2c104db0 0x7f4f2c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.479+0000 7f4f32212640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f2c102480 0x7f4f2c197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38888/0 (socket says 172.21.15.7:38888) 2024-06-10T01:16:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.479+0000 7f4f32212640 1 -- 172.21.15.7:0/778227959 learned_addr learned my addr 172.21.15.7:0/778227959 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.479+0000 7f4f32a13640 1 --2- 172.21.15.7:0/778227959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f2c1076e0 0x7f4f2c19f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.479+0000 7f4f31a11640 1 -- 172.21.15.7:0/778227959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f2c1076e0 msgr2=0x7f4f2c19f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.479+0000 7f4f31a11640 1 --2- 172.21.15.7:0/778227959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f2c1076e0 0x7f4f2c19f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.479+0000 7f4f31a11640 1 -- 172.21.15.7:0/778227959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f2c102480 msgr2=0x7f4f2c197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.479+0000 7f4f31a11640 1 --2- 172.21.15.7:0/778227959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f2c102480 0x7f4f2c197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.480+0000 7f4f31a11640 1 -- 172.21.15.7:0/778227959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f2c19f960 con 0x7f4f2c104db0 2024-06-10T01:16:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.480+0000 7f4f31a11640 1 --2- 172.21.15.7:0/778227959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f2c104db0 0x7f4f2c198280 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c00e9c0 tx=0x7f4f1c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.480+0000 7f4f237fe640 1 -- 172.21.15.7:0/778227959 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c00cd80 con 0x7f4f2c104db0 2024-06-10T01:16:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.480+0000 7f4f237fe640 1 -- 172.21.15.7:0/778227959 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f1c0035f0 con 0x7f4f2c104db0 2024-06-10T01:16:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.480+0000 7f4f34476640 1 -- 172.21.15.7:0/778227959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f2c19fc50 con 0x7f4f2c104db0 2024-06-10T01:16:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.480+0000 7f4f237fe640 1 -- 172.21.15.7:0/778227959 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c010690 con 0x7f4f2c104db0 2024-06-10T01:16:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.480+0000 7f4f34476640 1 -- 172.21.15.7:0/778227959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f2c1a01a0 con 0x7f4f2c104db0 2024-06-10T01:16:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.482+0000 7f4f237fe640 1 -- 172.21.15.7:0/778227959 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f4f1c0037a0 con 0x7f4f2c104db0 2024-06-10T01:16:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.483+0000 7f4f237fe640 1 --2- 172.21.15.7:0/778227959 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4f180a2b80 0x7f4f180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.483+0000 7f4f237fe640 1 -- 172.21.15.7:0/778227959 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f4f1c0cf6a0 con 0x7f4f2c104db0 2024-06-10T01:16:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.483+0000 7f4f32212640 1 --2- 172.21.15.7:0/778227959 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4f180a2b80 0x7f4f180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.484+0000 7f4f32212640 1 --2- 172.21.15.7:0/778227959 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4f180a2b80 0x7f4f180a5040 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4f280096c0 tx=0x7f4f280315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.484+0000 7f4f37188740 1 Processor -- start 2024-06-10T01:16:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.484+0000 7f4f37188740 1 -- start start 2024-06-10T01:16:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.484+0000 7f4f37188740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583a475a010 0x5583a475a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.484+0000 7f4f37188740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583a475aa00 0x5583a475f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.484+0000 7f4f37188740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583a475fb70 0x5583a4762000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.484+0000 7f4f37188740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5583a46bd4f0 con 0x5583a475fb70 2024-06-10T01:16:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.484+0000 7f4f37188740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5583a465b870 con 0x5583a475aa00 2024-06-10T01:16:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.484+0000 7f4f37188740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5583a46b94b0 con 0x5583a475a010 2024-06-10T01:16:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.484+0000 7f4f32a13640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583a475fb70 0x5583a4762000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.484+0000 7f4f32a13640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583a475fb70 0x5583a4762000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38892/0 (socket says 172.21.15.7:38892) 2024-06-10T01:16:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.484+0000 7f4f32a13640 1 -- 172.21.15.7:0/3176443732 learned_addr learned my addr 172.21.15.7:0/3176443732 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.485+0000 7f4f32212640 1 --2- 172.21.15.7:0/3176443732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583a475a010 0x5583a475a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.485+0000 7f4f32a13640 1 -- 172.21.15.7:0/3176443732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583a475a010 msgr2=0x5583a475a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.485+0000 7f4f32a13640 1 --2- 172.21.15.7:0/3176443732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583a475a010 0x5583a475a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.485+0000 7f4f32a13640 1 -- 172.21.15.7:0/3176443732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583a475aa00 msgr2=0x5583a475f500 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:16:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.485+0000 7f4f32a13640 1 --2- 172.21.15.7:0/3176443732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583a475aa00 0x5583a475f500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.485+0000 7f4f32a13640 1 -- 172.21.15.7:0/3176443732 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583a46779b0 con 0x5583a475fb70 2024-06-10T01:16:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.485+0000 7f4f32a13640 1 --2- 172.21.15.7:0/3176443732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583a475fb70 0x5583a4762000 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f4f24009fd0 tx=0x7f4f240118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44bdd5d90382f5e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.485+0000 7f4f20bfe640 1 -- 172.21.15.7:0/3176443732 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f24004d20 con 0x5583a475fb70 2024-06-10T01:16:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.485+0000 7f4f20bfe640 1 -- 172.21.15.7:0/3176443732 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f24004050 con 0x5583a475fb70 2024-06-10T01:16:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.485+0000 7f4f20bfe640 1 -- 172.21.15.7:0/3176443732 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f24005220 con 0x5583a475fb70 2024-06-10T01:16:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.486+0000 7f4f37188740 1 -- 172.21.15.7:0/3176443732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583a475fb70 msgr2=0x5583a4762000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.486+0000 7f4f37188740 1 --2- 172.21.15.7:0/3176443732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583a475fb70 0x5583a4762000 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f4f24009fd0 tx=0x7f4f240118d0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.486+0000 7f4f37188740 1 -- 172.21.15.7:0/3176443732 shutdown_connections 2024-06-10T01:16:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.486+0000 7f4f37188740 1 --2- 172.21.15.7:0/3176443732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583a475fb70 0x5583a4762000 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.486+0000 7f4f37188740 1 --2- 172.21.15.7:0/3176443732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583a475aa00 0x5583a475f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.486+0000 7f4f37188740 1 --2- 172.21.15.7:0/3176443732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583a475a010 0x5583a475a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.486+0000 7f4f37188740 1 -- 172.21.15.7:0/3176443732 >> 172.21.15.7:0/3176443732 conn(0x5583a4755680 msgr2=0x5583a4755a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.486+0000 7f4f37188740 1 -- 172.21.15.7:0/3176443732 shutdown_connections 2024-06-10T01:16:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.486+0000 7f4f37188740 1 -- 172.21.15.7:0/3176443732 wait complete. 2024-06-10T01:16:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.486+0000 7f4f37188740 1 -- start start 2024-06-10T01:16:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.486+0000 7f4f37188740 1 Processor -- start 2024-06-10T01:16:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.487+0000 7f4f37188740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583a475a010 0x5583a4764290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.487+0000 7f4f37188740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583a475aa00 0x5583a47647d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.487+0000 7f4f37188740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583a475fb70 0x5583a476b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.487+0000 7f4f37188740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5583a4762b20 con 0x5583a475a010 2024-06-10T01:16:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.487+0000 7f4f37188740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5583a4646ae0 con 0x5583a475aa00 2024-06-10T01:16:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.487+0000 7f4f37188740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5583a4762ca0 con 0x5583a475fb70 2024-06-10T01:16:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.487+0000 7f4f37188740 10 client.0 _ll_register_callbacks cb 0x5583a467f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:16:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.487+0000 7f4f32212640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583a475a010 0x5583a4764290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.487+0000 7f4f32212640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583a475a010 0x5583a4764290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38906/0 (socket says 172.21.15.7:38906) 2024-06-10T01:16:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.487+0000 7f4f32212640 1 -- 172.21.15.7:0/1935596295 learned_addr learned my addr 172.21.15.7:0/1935596295 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.487+0000 7f4f32a13640 1 --2- 172.21.15.7:0/1935596295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583a475fb70 0x5583a476b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.487+0000 7f4f32212640 1 -- 172.21.15.7:0/1935596295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583a475fb70 msgr2=0x5583a476b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.487+0000 7f4f32212640 1 --2- 172.21.15.7:0/1935596295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583a475fb70 0x5583a476b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.487+0000 7f4f32212640 1 -- 172.21.15.7:0/1935596295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583a475aa00 msgr2=0x5583a47647d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:16:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.487+0000 7f4f32212640 1 --2- 172.21.15.7:0/1935596295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583a475aa00 0x5583a47647d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.487+0000 7f4f32212640 1 -- 172.21.15.7:0/1935596295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583a43d1210 con 0x5583a475a010 2024-06-10T01:16:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.487+0000 7f4f32212640 1 --2- 172.21.15.7:0/1935596295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583a475a010 0x5583a4764290 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f4f2802aa00 tx=0x7f4f28009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a07eaa4f38f137 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.487+0000 7f4f15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:16:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.487+0000 7f4f15ffb640 1 -- 172.21.15.7:0/1935596295 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f28035040 con 0x5583a475a010 2024-06-10T01:16:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.487+0000 7f4f15ffb640 1 -- 172.21.15.7:0/1935596295 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f4f28031cc0 con 0x5583a475a010 2024-06-10T01:16:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.487+0000 7f4f37188740 1 -- 172.21.15.7:0/1935596295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f24004050 con 0x5583a475a010 2024-06-10T01:16:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.488+0000 7f4f15ffb640 1 -- 172.21.15.7:0/1935596295 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4f280035f0 con 0x5583a475a010 2024-06-10T01:16:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.488+0000 7f4f37188740 10 client.18147 fetch_fsmap learned FSMap version 39 2024-06-10T01:16:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.488+0000 7f4f37188740 1 -- 172.21.15.7:0/1935596295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5583a449d620 con 0x5583a475a010 2024-06-10T01:16:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.488+0000 7f4f15ffb640 1 -- 172.21.15.7:0/1935596295 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f24004050 con 0x5583a475a010 2024-06-10T01:16:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.488+0000 7f4f15ffb640 1 -- 172.21.15.7:0/1935596295 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7f4f2803c830 con 0x5583a475a010 2024-06-10T01:16:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.488+0000 7f4f37188740 10 client.18147 fetch_fsmap finished waiting for FSMap version 39 2024-06-10T01:16:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.488+0000 7f4f37188740 10 client.18147 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:16:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.488+0000 7f4f37188740 20 client.18147 populate_metadata read hostname 'smithi007' 2024-06-10T01:16:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.488+0000 7f4f37188740 1 --2- 172.21.15.7:0/1935596295 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5583a4770160 0x5583a4772620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.488+0000 7f4f37188740 4 client.18147 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:16:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.488+0000 7f4f37188740 1 -- 172.21.15.7:0/1935596295 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583a46d0f10 con 0x5583a4770160 2024-06-10T01:16:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.488+0000 7f4f31a11640 1 --2- 172.21.15.7:0/1935596295 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5583a4770160 0x5583a4772620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.489+0000 7f4f31a11640 1 --2- 172.21.15.7:0/1935596295 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5583a4770160 0x5583a4772620 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c44d0ee4ace89565 server_cookie=e7dc3142b2df04b0 in_seq=0 out_seq=0 2024-06-10T01:16:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.489+0000 7f4f15ffb640 10 client.18147 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.489+0000 7f4f15ffb640 1 -- 172.21.15.7:0/1935596295 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5583a46d0f10 con 0x5583a4770160 2024-06-10T01:16:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.489+0000 7f4f15ffb640 10 client.18147 handle_command_reply: tid=0 2024-06-10T01:16:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.489+0000 7f4f37188740 1 client.18147 shutdown 2024-06-10T01:16:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.489+0000 7f4f37188740 1 -- 172.21.15.7:0/1935596295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583a475a010 msgr2=0x5583a4764290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.489+0000 7f4f37188740 1 --2- 172.21.15.7:0/1935596295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583a475a010 0x5583a4764290 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f4f2802aa00 tx=0x7f4f28009260 comp rx=0 tx=0).stop 2024-06-10T01:16:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.490+0000 7f4f37188740 1 -- 172.21.15.7:0/1935596295 shutdown_connections 2024-06-10T01:16:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.490+0000 7f4f37188740 1 --2- 172.21.15.7:0/1935596295 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5583a4770160 0x5583a4772620 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.490+0000 7f4f15ffb640 0 client.18147 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.490+0000 7f4f37188740 1 --2- 172.21.15.7:0/1935596295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583a475fb70 0x5583a476b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.490+0000 7f4f37188740 1 --2- 172.21.15.7:0/1935596295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583a475aa00 0x5583a47647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.490+0000 7f4f37188740 1 --2- 172.21.15.7:0/1935596295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583a475a010 0x5583a4764290 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.490+0000 7f4f37188740 1 -- 172.21.15.7:0/1935596295 >> 172.21.15.7:0/1935596295 conn(0x5583a4755680 msgr2=0x5583a47547e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.490+0000 7f4f37188740 1 -- 172.21.15.7:0/1935596295 shutdown_connections 2024-06-10T01:16:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.490+0000 7f4f37188740 1 -- 172.21.15.7:0/1935596295 wait complete. 2024-06-10T01:16:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.490+0000 7f4f37188740 20 client.18147 trim_cache size 0 max 16384 2024-06-10T01:16:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.506+0000 7f4f37188740 1 Processor -- start 2024-06-10T01:16:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.506+0000 7f4f37188740 1 -- start start 2024-06-10T01:16:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.506+0000 7f4f37188740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583a475a010 0x5583a4758ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.506+0000 7f4f37188740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583a475aa00 0x5583a476b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.506+0000 7f4f37188740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583a475fb70 0x5583a4763ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.506+0000 7f4f37188740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c1acf60 con 0x5583a475aa00 2024-06-10T01:16:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.506+0000 7f4f37188740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c1aa800 con 0x5583a475a010 2024-06-10T01:16:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.506+0000 7f4f37188740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c1a8650 con 0x5583a475fb70 2024-06-10T01:16:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.506+0000 7f4f31a11640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583a475aa00 0x5583a476b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.506+0000 7f4f32a13640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583a475fb70 0x5583a4763ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.506+0000 7f4f32212640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583a475a010 0x5583a4758ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.506+0000 7f4f31a11640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583a475aa00 0x5583a476b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38922/0 (socket says 172.21.15.7:38922) 2024-06-10T01:16:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.506+0000 7f4f31a11640 1 -- 172.21.15.7:0/1365320091 learned_addr learned my addr 172.21.15.7:0/1365320091 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.506+0000 7f4f32212640 1 -- 172.21.15.7:0/1365320091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583a475fb70 msgr2=0x5583a4763ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.506+0000 7f4f32212640 1 --2- 172.21.15.7:0/1365320091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583a475fb70 0x5583a4763ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.506+0000 7f4f32212640 1 -- 172.21.15.7:0/1365320091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583a475aa00 msgr2=0x5583a476b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.506+0000 7f4f32212640 1 --2- 172.21.15.7:0/1365320091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583a475aa00 0x5583a476b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.506+0000 7f4f32212640 1 -- 172.21.15.7:0/1365320091 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583a47554b0 con 0x5583a475a010 2024-06-10T01:16:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.506+0000 7f4f31a11640 1 --2- 172.21.15.7:0/1365320091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583a475aa00 0x5583a476b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.507+0000 7f4f32212640 1 --2- 172.21.15.7:0/1365320091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583a475a010 0x5583a4758ca0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4f28004770 tx=0x7f4f2803cf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef8b6b2b962804ec server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.507+0000 7f4f17fff640 1 -- 172.21.15.7:0/1365320091 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f28063070 con 0x5583a475a010 2024-06-10T01:16:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.507+0000 7f4f17fff640 1 -- 172.21.15.7:0/1365320091 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f2803c440 con 0x5583a475a010 2024-06-10T01:16:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.507+0000 7f4f17fff640 1 -- 172.21.15.7:0/1365320091 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f28035070 con 0x5583a475a010 2024-06-10T01:16:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.508+0000 7f4f37188740 1 -- 172.21.15.7:0/1365320091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583a475a010 msgr2=0x5583a4758ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.508+0000 7f4f37188740 1 --2- 172.21.15.7:0/1365320091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583a475a010 0x5583a4758ca0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4f28004770 tx=0x7f4f2803cf10 comp rx=0 tx=0).stop 2024-06-10T01:16:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.508+0000 7f4f37188740 1 -- 172.21.15.7:0/1365320091 shutdown_connections 2024-06-10T01:16:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.508+0000 7f4f37188740 1 --2- 172.21.15.7:0/1365320091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583a475fb70 0x5583a4763ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.508+0000 7f4f37188740 1 --2- 172.21.15.7:0/1365320091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583a475aa00 0x5583a476b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.508+0000 7f4f37188740 1 --2- 172.21.15.7:0/1365320091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583a475a010 0x5583a4758ca0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.508+0000 7f4f37188740 1 -- 172.21.15.7:0/1365320091 >> 172.21.15.7:0/1365320091 conn(0x5583a4755680 msgr2=0x5583a4758590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.508+0000 7f4f37188740 1 -- 172.21.15.7:0/1365320091 shutdown_connections 2024-06-10T01:16:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.508+0000 7f4f37188740 1 -- 172.21.15.7:0/1365320091 wait complete. 2024-06-10T01:16:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.508+0000 7f4f37188740 1 -- start start 2024-06-10T01:16:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.508+0000 7f4f37188740 1 Processor -- start 2024-06-10T01:16:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.508+0000 7f4f37188740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583a475a010 0x5583a4768a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.508+0000 7f4f37188740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583a475aa00 0x5583a4768f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.508+0000 7f4f37188740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583a475fb70 0x5583a4769eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.508+0000 7f4f37188740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c1a8cc0 con 0x5583a475fb70 2024-06-10T01:16:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.508+0000 7f4f37188740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c1aa130 con 0x5583a475a010 2024-06-10T01:16:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.508+0000 7f4f37188740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c1a9390 con 0x5583a475aa00 2024-06-10T01:16:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.508+0000 7f4f37188740 10 client.0 _ll_register_callbacks cb 0x7f4f1c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:16:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.508+0000 7f4f31a11640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583a475aa00 0x5583a4768f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.508+0000 7f4f32a13640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583a475fb70 0x5583a4769eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.508+0000 7f4f32212640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583a475a010 0x5583a4768a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.508+0000 7f4f31a11640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583a475aa00 0x5583a4768f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:59934/0 (socket says 172.21.15.7:59934) 2024-06-10T01:16:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.508+0000 7f4f32212640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583a475a010 0x5583a4768a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:36438/0 (socket says 172.21.15.7:36438) 2024-06-10T01:16:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.508+0000 7f4f31a11640 1 -- 172.21.15.7:0/1584261215 learned_addr learned my addr 172.21.15.7:0/1584261215 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.509+0000 7f4f31a11640 1 -- 172.21.15.7:0/1584261215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583a475a010 msgr2=0x5583a4768a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.509+0000 7f4f31a11640 1 --2- 172.21.15.7:0/1584261215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583a475a010 0x5583a4768a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.509+0000 7f4f31a11640 1 -- 172.21.15.7:0/1584261215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583a475fb70 msgr2=0x5583a4769eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.509+0000 7f4f31a11640 1 --2- 172.21.15.7:0/1584261215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583a475fb70 0x5583a4769eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.509+0000 7f4f31a11640 1 -- 172.21.15.7:0/1584261215 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583a476a4a0 con 0x5583a475aa00 2024-06-10T01:16:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.509+0000 7f4f32a13640 1 --2- 172.21.15.7:0/1584261215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583a475fb70 0x5583a4769eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.509+0000 7f4f31a11640 1 --2- 172.21.15.7:0/1584261215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583a475aa00 0x5583a4768f40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c09b750 tx=0x7f4f1c09ffe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23e4fe668e60a53f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.509+0000 7f4f15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:16:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.509+0000 7f4f15ffb640 1 -- 172.21.15.7:0/1584261215 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c003b30 con 0x5583a475aa00 2024-06-10T01:16:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.509+0000 7f4f15ffb640 1 -- 172.21.15.7:0/1584261215 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f4f1c0cf360 con 0x5583a475aa00 2024-06-10T01:16:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.509+0000 7f4f37188740 1 -- 172.21.15.7:0/1584261215 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f2803c440 con 0x5583a475aa00 2024-06-10T01:16:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.509+0000 7f4f15ffb640 1 -- 172.21.15.7:0/1584261215 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4f1c098190 con 0x5583a475aa00 2024-06-10T01:16:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.509+0000 7f4f37188740 10 client.27532 fetch_fsmap learned FSMap version 39 2024-06-10T01:16:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.509+0000 7f4f37188740 1 -- 172.21.15.7:0/1584261215 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5583a476b880 con 0x5583a475aa00 2024-06-10T01:16:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.510+0000 7f4f15ffb640 1 -- 172.21.15.7:0/1584261215 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f2803c440 con 0x5583a475aa00 2024-06-10T01:16:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.510+0000 7f4f15ffb640 1 -- 172.21.15.7:0/1584261215 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7f4f1c0cf040 con 0x5583a475aa00 2024-06-10T01:16:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.510+0000 7f4f37188740 10 client.27532 fetch_fsmap finished waiting for FSMap version 39 2024-06-10T01:16:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.510+0000 7f4f37188740 10 client.27532 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:16:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.510+0000 7f4f37188740 20 client.27532 populate_metadata read hostname 'smithi007' 2024-06-10T01:16:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.510+0000 7f4f37188740 1 --2- 172.21.15.7:0/1584261215 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5583a4770160 0x5583a47dd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.510+0000 7f4f37188740 4 client.27532 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:16:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.510+0000 7f4f37188740 1 -- 172.21.15.7:0/1584261215 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5583a46cac20 con 0x5583a4770160 2024-06-10T01:16:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.510+0000 7f4f32212640 1 --2- 172.21.15.7:0/1584261215 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5583a4770160 0x5583a47dd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.510+0000 7f4f32212640 1 --2- 172.21.15.7:0/1584261215 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5583a4770160 0x5583a47dd370 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71cccbb0b16fef7e server_cookie=ef5a876b037aad95 in_seq=0 out_seq=0 2024-06-10T01:16:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.510+0000 7f4f15ffb640 10 client.27532 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.629+0000 7f4f15ffb640 1 -- 172.21.15.7:0/1584261215 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5583a46cac20 con 0x5583a4770160 2024-06-10T01:16:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.629+0000 7f4f15ffb640 10 client.27532 handle_command_reply: tid=0 2024-06-10T01:16:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.629+0000 7f4f37188740 1 client.27532 shutdown 2024-06-10T01:16:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.630+0000 7f4f37188740 1 -- 172.21.15.7:0/1584261215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583a475aa00 msgr2=0x5583a4768f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.630+0000 7f4f37188740 1 --2- 172.21.15.7:0/1584261215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583a475aa00 0x5583a4768f40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c09b750 tx=0x7f4f1c09ffe0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.630+0000 7f4f37188740 1 -- 172.21.15.7:0/1584261215 shutdown_connections 2024-06-10T01:16:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.630+0000 7f4f37188740 1 --2- 172.21.15.7:0/1584261215 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5583a4770160 0x5583a47dd370 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.630+0000 7f4f37188740 1 --2- 172.21.15.7:0/1584261215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583a475fb70 0x5583a4769eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.630+0000 7f4f37188740 1 --2- 172.21.15.7:0/1584261215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583a475aa00 0x5583a4768f40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.630+0000 7f4f15ffb640 0 client.27532 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.630+0000 7f4f37188740 1 --2- 172.21.15.7:0/1584261215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583a475a010 0x5583a4768a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.630+0000 7f4f37188740 1 -- 172.21.15.7:0/1584261215 >> 172.21.15.7:0/1584261215 conn(0x5583a4755680 msgr2=0x5583a4763e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.630+0000 7f4f37188740 1 -- 172.21.15.7:0/1584261215 shutdown_connections 2024-06-10T01:16:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.630+0000 7f4f37188740 1 -- 172.21.15.7:0/1584261215 wait complete. 2024-06-10T01:16:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.630+0000 7f4f37188740 20 client.27532 trim_cache size 0 max 16384 2024-06-10T01:16:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:16:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:16:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:16:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.631+0000 7f4f34476640 1 -- 172.21.15.7:0/778227959 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4f180a2b80 msgr2=0x7f4f180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.631+0000 7f4f34476640 1 --2- 172.21.15.7:0/778227959 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4f180a2b80 0x7f4f180a5040 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4f280096c0 tx=0x7f4f280315b0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.631+0000 7f4f34476640 1 -- 172.21.15.7:0/778227959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f2c104db0 msgr2=0x7f4f2c198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.631+0000 7f4f34476640 1 --2- 172.21.15.7:0/778227959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f2c104db0 0x7f4f2c198280 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c00e9c0 tx=0x7f4f1c00ee80 comp rx=0 tx=0).stop 2024-06-10T01:16:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.631+0000 7f4f34476640 1 -- 172.21.15.7:0/778227959 shutdown_connections 2024-06-10T01:16:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.631+0000 7f4f34476640 1 --2- 172.21.15.7:0/778227959 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4f180a2b80 0x7f4f180a5040 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.631+0000 7f4f34476640 1 --2- 172.21.15.7:0/778227959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f2c1076e0 0x7f4f2c19f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.631+0000 7f4f34476640 1 --2- 172.21.15.7:0/778227959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f2c104db0 0x7f4f2c198280 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.631+0000 7f4f34476640 1 --2- 172.21.15.7:0/778227959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f2c102480 0x7f4f2c197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.631+0000 7f4f34476640 1 -- 172.21.15.7:0/778227959 >> 172.21.15.7:0/778227959 conn(0x7f4f2c0fbe70 msgr2=0x7f4f2c102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.631+0000 7f4f34476640 1 -- 172.21.15.7:0/778227959 shutdown_connections 2024-06-10T01:16:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:38.631+0000 7f4f34476640 1 -- 172.21.15.7:0/778227959 wait complete. 2024-06-10T01:16:38.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.708+0000 7f91997aa700 1 -- 172.21.15.7:0/807330102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91941073c0 msgr2=0x7f91941077a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:38.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.708+0000 7f91997aa700 1 --2- 172.21.15.7:0/807330102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91941073c0 0x7f91941077a0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f9184009990 tx=0x7f9184009ca0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.708+0000 7f91997aa700 1 -- 172.21.15.7:0/807330102 shutdown_connections 2024-06-10T01:16:38.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.708+0000 7f91997aa700 1 --2- 172.21.15.7:0/807330102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9194101d40 0x7f919410e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.708+0000 7f91997aa700 1 --2- 172.21.15.7:0/807330102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91941073c0 0x7f91941077a0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.708+0000 7f91997aa700 1 --2- 172.21.15.7:0/807330102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9194101380 0x7f9194101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.708+0000 7f91997aa700 1 -- 172.21.15.7:0/807330102 >> 172.21.15.7:0/807330102 conn(0x7f9194078330 msgr2=0x7f9194078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:38.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.708+0000 7f91997aa700 1 -- 172.21.15.7:0/807330102 shutdown_connections 2024-06-10T01:16:38.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.708+0000 7f91997aa700 1 -- 172.21.15.7:0/807330102 wait complete. 2024-06-10T01:16:38.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.708+0000 7f91997aa700 1 Processor -- start 2024-06-10T01:16:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.709+0000 7f91997aa700 1 -- start start 2024-06-10T01:16:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.709+0000 7f91997aa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9194101380 0x7f9194103030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.709+0000 7f91997aa700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9194101d40 0x7f9194103570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.709+0000 7f91997aa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91941073c0 0x7f9194103ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.709+0000 7f91997aa700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9194111660 con 0x7f91941073c0 2024-06-10T01:16:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.709+0000 7f91997aa700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f91941114e0 con 0x7f9194101380 2024-06-10T01:16:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.709+0000 7f91997aa700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f91941117e0 con 0x7f9194101d40 2024-06-10T01:16:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.709+0000 7f9192ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9194101380 0x7f9194103030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.709+0000 7f91937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91941073c0 0x7f9194103ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.709+0000 7f9192ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9194101380 0x7f9194103030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:36464/0 (socket says 172.21.15.7:36464) 2024-06-10T01:16:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.709+0000 7f91927fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9194101d40 0x7f9194103570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.709+0000 7f9192ffd700 1 -- 172.21.15.7:0/2183110090 learned_addr learned my addr 172.21.15.7:0/2183110090 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:38.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.709+0000 7f9192ffd700 1 -- 172.21.15.7:0/2183110090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9194101d40 msgr2=0x7f9194103570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:38.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.709+0000 7f9192ffd700 1 --2- 172.21.15.7:0/2183110090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9194101d40 0x7f9194103570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.709+0000 7f9192ffd700 1 -- 172.21.15.7:0/2183110090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91941073c0 msgr2=0x7f9194103ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:38.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.709+0000 7f9192ffd700 1 --2- 172.21.15.7:0/2183110090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91941073c0 0x7f9194103ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.709+0000 7f9192ffd700 1 -- 172.21.15.7:0/2183110090 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91941a7130 con 0x7f9194101380 2024-06-10T01:16:38.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.709+0000 7f91937fe700 1 --2- 172.21.15.7:0/2183110090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91941073c0 0x7f9194103ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:38.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.710+0000 7f9192ffd700 1 --2- 172.21.15.7:0/2183110090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9194101380 0x7f9194103030 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9184000c00 tx=0x7f91840047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:38.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.710+0000 7f917bfff700 1 -- 172.21.15.7:0/2183110090 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9184027070 con 0x7f9194101380 2024-06-10T01:16:38.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.710+0000 7f917bfff700 1 -- 172.21.15.7:0/2183110090 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9184003e40 con 0x7f9194101380 2024-06-10T01:16:38.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.710+0000 7f91997aa700 1 -- 172.21.15.7:0/2183110090 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91941a73c0 con 0x7f9194101380 2024-06-10T01:16:38.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.710+0000 7f917bfff700 1 -- 172.21.15.7:0/2183110090 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9184022d10 con 0x7f9194101380 2024-06-10T01:16:38.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.710+0000 7f91997aa700 1 -- 172.21.15.7:0/2183110090 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91941a7810 con 0x7f9194101380 2024-06-10T01:16:38.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.712+0000 7f917bfff700 1 -- 172.21.15.7:0/2183110090 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f918402a070 con 0x7f9194101380 2024-06-10T01:16:38.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.713+0000 7f917bfff700 1 --2- 172.21.15.7:0/2183110090 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f917c0a2c20 0x7f917c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.713+0000 7f91927fc700 1 --2- 172.21.15.7:0/2183110090 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f917c0a2c20 0x7f917c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.713+0000 7f917bfff700 1 -- 172.21.15.7:0/2183110090 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f91840d0240 con 0x7f9194101380 2024-06-10T01:16:38.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.714+0000 7f91a402db80 1 Processor -- start 2024-06-10T01:16:38.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.714+0000 7f91927fc700 1 --2- 172.21.15.7:0/2183110090 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f917c0a2c20 0x7f917c0a50e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9194078930 tx=0x7f91880073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:38.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.714+0000 7f91a402db80 1 -- start start 2024-06-10T01:16:38.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.714+0000 7f91a402db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dce6685f60 0x55dce6686340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.714+0000 7f91a402db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dce6686880 0x55dce6680260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.714+0000 7f91a402db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dce66807a0 0x55dce6680bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.714+0000 7f91a402db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55dce6572f60 con 0x55dce66807a0 2024-06-10T01:16:38.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.714+0000 7f91a402db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55dce65d1850 con 0x55dce6686880 2024-06-10T01:16:38.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.714+0000 7f91a402db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dce6539460 con 0x55dce6685f60 2024-06-10T01:16:38.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.714+0000 7f91937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dce66807a0 0x55dce6680bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.714+0000 7f91937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dce66807a0 0x55dce6680bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38964/0 (socket says 172.21.15.7:38964) 2024-06-10T01:16:38.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.714+0000 7f91937fe700 1 -- 172.21.15.7:0/393097469 learned_addr learned my addr 172.21.15.7:0/393097469 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:38.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.714+0000 7f9192ffd700 1 --2- 172.21.15.7:0/393097469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dce6685f60 0x55dce6686340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.714+0000 7f91927fc700 1 --2- 172.21.15.7:0/393097469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dce6686880 0x55dce6680260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.714+0000 7f91937fe700 1 -- 172.21.15.7:0/393097469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dce6685f60 msgr2=0x55dce6686340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:38.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.714+0000 7f91937fe700 1 --2- 172.21.15.7:0/393097469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dce6685f60 0x55dce6686340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.714+0000 7f91937fe700 1 -- 172.21.15.7:0/393097469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dce6686880 msgr2=0x55dce6680260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:38.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.714+0000 7f91937fe700 1 --2- 172.21.15.7:0/393097469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dce6686880 0x55dce6680260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.714+0000 7f91937fe700 1 -- 172.21.15.7:0/393097469 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dce65208d0 con 0x55dce66807a0 2024-06-10T01:16:38.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.715+0000 7f91937fe700 1 --2- 172.21.15.7:0/393097469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dce66807a0 0x55dce6680bc0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f918c007740 tx=0x7f918c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c79a4ad1a6e6324a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:38.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.715+0000 7f91797fa700 1 -- 172.21.15.7:0/393097469 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c019020 con 0x55dce66807a0 2024-06-10T01:16:38.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.715+0000 7f91797fa700 1 -- 172.21.15.7:0/393097469 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f918c015d30 con 0x55dce66807a0 2024-06-10T01:16:38.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.715+0000 7f91797fa700 1 -- 172.21.15.7:0/393097469 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c012360 con 0x55dce66807a0 2024-06-10T01:16:38.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.716+0000 7f91a402db80 1 -- 172.21.15.7:0/393097469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dce66807a0 msgr2=0x55dce6680bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:38.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.716+0000 7f91a402db80 1 --2- 172.21.15.7:0/393097469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dce66807a0 0x55dce6680bc0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f918c007740 tx=0x7f918c009f90 comp rx=0 tx=0).stop 2024-06-10T01:16:38.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.716+0000 7f91a402db80 1 -- 172.21.15.7:0/393097469 shutdown_connections 2024-06-10T01:16:38.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.716+0000 7f91a402db80 1 --2- 172.21.15.7:0/393097469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dce66807a0 0x55dce6680bc0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.716+0000 7f91a402db80 1 --2- 172.21.15.7:0/393097469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dce6685f60 0x55dce6686340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.716+0000 7f91a402db80 1 --2- 172.21.15.7:0/393097469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dce6686880 0x55dce6680260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.716+0000 7f91a402db80 1 -- 172.21.15.7:0/393097469 >> 172.21.15.7:0/393097469 conn(0x55dce65ad080 msgr2=0x55dce667b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:38.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.716+0000 7f91a402db80 1 -- 172.21.15.7:0/393097469 shutdown_connections 2024-06-10T01:16:38.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.716+0000 7f91a402db80 1 -- 172.21.15.7:0/393097469 wait complete. 2024-06-10T01:16:38.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.716+0000 7f91a402db80 1 -- start start 2024-06-10T01:16:38.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.716+0000 7f91a402db80 1 Processor -- start 2024-06-10T01:16:38.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.716+0000 7f91a402db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dce66807a0 0x55dce668c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.716+0000 7f91a402db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dce6685f60 0x55dce6689920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.716+0000 7f91a402db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dce6686880 0x55dce6689e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.716+0000 7f91a402db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55dce6571fc0 con 0x55dce6686880 2024-06-10T01:16:38.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.716+0000 7f91a402db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55dce65cd810 con 0x55dce6685f60 2024-06-10T01:16:38.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.716+0000 7f91a402db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dce668d260 con 0x55dce66807a0 2024-06-10T01:16:38.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.717+0000 7f91a402db80 10 client.0 _ll_register_callbacks cb 0x55dce65675b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:16:38.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.717+0000 7f91937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dce6686880 0x55dce6689e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.717+0000 7f91927fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dce6685f60 0x55dce6689920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.717+0000 7f9192ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dce66807a0 0x55dce668c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.717+0000 7f91927fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dce6685f60 0x55dce6689920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:36480/0 (socket says 172.21.15.7:36480) 2024-06-10T01:16:38.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.717+0000 7f91937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dce6686880 0x55dce6689e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38968/0 (socket says 172.21.15.7:38968) 2024-06-10T01:16:38.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.717+0000 7f91927fc700 1 -- 172.21.15.7:0/569455675 learned_addr learned my addr 172.21.15.7:0/569455675 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:38.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.717+0000 7f91927fc700 1 -- 172.21.15.7:0/569455675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dce66807a0 msgr2=0x55dce668c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:38.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.717+0000 7f91927fc700 1 --2- 172.21.15.7:0/569455675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dce66807a0 0x55dce668c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.717+0000 7f91927fc700 1 -- 172.21.15.7:0/569455675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dce6686880 msgr2=0x55dce6689e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:38.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.717+0000 7f91927fc700 1 --2- 172.21.15.7:0/569455675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dce6686880 0x55dce6689e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.717+0000 7f91927fc700 1 -- 172.21.15.7:0/569455675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dce6597750 con 0x55dce6685f60 2024-06-10T01:16:38.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.717+0000 7f9192ffd700 1 --2- 172.21.15.7:0/569455675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dce66807a0 0x55dce668c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:38.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.717+0000 7f91937fe700 1 --2- 172.21.15.7:0/569455675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dce6686880 0x55dce6689e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:38.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.717+0000 7f91927fc700 1 --2- 172.21.15.7:0/569455675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dce6685f60 0x55dce6689920 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9188010bf0 tx=0x7f91880047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd1cc23261b2c945 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:38.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.717+0000 7f9176ff5700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:16:38.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.717+0000 7f9176ff5700 1 -- 172.21.15.7:0/569455675 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188024070 con 0x55dce6685f60 2024-06-10T01:16:38.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.717+0000 7f9176ff5700 1 -- 172.21.15.7:0/569455675 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f9188003a90 con 0x55dce6685f60 2024-06-10T01:16:38.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.717+0000 7f91a402db80 1 -- 172.21.15.7:0/569455675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f918c015d30 con 0x55dce6685f60 2024-06-10T01:16:38.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.718+0000 7f9176ff5700 1 -- 172.21.15.7:0/569455675 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9188030050 con 0x55dce6685f60 2024-06-10T01:16:38.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.718+0000 7f91a402db80 10 client.27515 fetch_fsmap learned FSMap version 39 2024-06-10T01:16:38.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.718+0000 7f91a402db80 1 -- 172.21.15.7:0/569455675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55dce6697aa0 con 0x55dce6685f60 2024-06-10T01:16:38.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.718+0000 7f9176ff5700 1 -- 172.21.15.7:0/569455675 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f918c015d30 con 0x55dce6685f60 2024-06-10T01:16:38.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.718+0000 7f9176ff5700 1 -- 172.21.15.7:0/569455675 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7f9188020410 con 0x55dce6685f60 2024-06-10T01:16:38.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.718+0000 7f91a402db80 10 client.27515 fetch_fsmap finished waiting for FSMap version 39 2024-06-10T01:16:38.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.718+0000 7f91a402db80 10 client.27515 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:16:38.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.718+0000 7f91a402db80 20 client.27515 populate_metadata read hostname 'smithi007' 2024-06-10T01:16:38.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.718+0000 7f91a402db80 1 --2- 172.21.15.7:0/569455675 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55dce669a080 0x55dce669c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.718+0000 7f91a402db80 4 client.27515 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:16:38.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.718+0000 7f91a402db80 1 -- 172.21.15.7:0/569455675 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dce65ded50 con 0x55dce669a080 2024-06-10T01:16:38.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.718+0000 7f9192ffd700 1 --2- 172.21.15.7:0/569455675 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55dce669a080 0x55dce669c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.719+0000 7f9192ffd700 1 --2- 172.21.15.7:0/569455675 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55dce669a080 0x55dce669c4a0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b4c49bdadca4b0c server_cookie=d0d7c25d46d76e3d in_seq=0 out_seq=0 2024-06-10T01:16:38.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.719+0000 7f9176ff5700 10 client.27515 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:38.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.719+0000 7f9176ff5700 1 -- 172.21.15.7:0/569455675 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55dce65ded50 con 0x55dce669a080 2024-06-10T01:16:38.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.719+0000 7f9176ff5700 10 client.27515 handle_command_reply: tid=0 2024-06-10T01:16:38.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.719+0000 7f91a402db80 1 client.27515 shutdown 2024-06-10T01:16:38.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.720+0000 7f91a402db80 1 -- 172.21.15.7:0/569455675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dce6685f60 msgr2=0x55dce6689920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:38.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.720+0000 7f91a402db80 1 --2- 172.21.15.7:0/569455675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dce6685f60 0x55dce6689920 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9188010bf0 tx=0x7f91880047c0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.720+0000 7f91a402db80 1 -- 172.21.15.7:0/569455675 shutdown_connections 2024-06-10T01:16:38.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.720+0000 7f91a402db80 1 --2- 172.21.15.7:0/569455675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dce6686880 0x55dce6689e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.720+0000 7f91a402db80 1 --2- 172.21.15.7:0/569455675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dce66807a0 0x55dce668c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.720+0000 7f91a402db80 1 --2- 172.21.15.7:0/569455675 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55dce669a080 0x55dce669c4a0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.720+0000 7f91a402db80 1 --2- 172.21.15.7:0/569455675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dce6685f60 0x55dce6689920 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.720+0000 7f91a402db80 1 -- 172.21.15.7:0/569455675 >> 172.21.15.7:0/569455675 conn(0x55dce65ad080 msgr2=0x55dce667a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:38.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.720+0000 7f9176ff5700 0 client.27515 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:38.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.720+0000 7f91a402db80 1 -- 172.21.15.7:0/569455675 shutdown_connections 2024-06-10T01:16:38.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.720+0000 7f91a402db80 1 -- 172.21.15.7:0/569455675 wait complete. 2024-06-10T01:16:38.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.720+0000 7f91a402db80 20 client.27515 trim_cache size 0 max 16384 2024-06-10T01:16:38.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.742+0000 7f91a402db80 1 Processor -- start 2024-06-10T01:16:38.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.742+0000 7f91a402db80 1 -- start start 2024-06-10T01:16:38.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.742+0000 7f91a402db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dce66807a0 0x55dce6680260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.742+0000 7f91a402db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dce6685f60 0x55dce66874f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.742+0000 7f91a402db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dce6686880 0x55dce6689e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.742+0000 7f91a402db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f91941b2c00 con 0x55dce6685f60 2024-06-10T01:16:38.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.742+0000 7f91a402db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f91941b0460 con 0x55dce6686880 2024-06-10T01:16:38.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.742+0000 7f91a402db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f91941ae2b0 con 0x55dce66807a0 2024-06-10T01:16:38.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.742+0000 7f91927fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dce6685f60 0x55dce66874f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.742+0000 7f9192ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dce66807a0 0x55dce6680260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.742+0000 7f91927fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dce6685f60 0x55dce66874f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38982/0 (socket says 172.21.15.7:38982) 2024-06-10T01:16:38.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.742+0000 7f91927fc700 1 -- 172.21.15.7:0/2589200891 learned_addr learned my addr 172.21.15.7:0/2589200891 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:38.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.742+0000 7f91937fe700 1 --2- 172.21.15.7:0/2589200891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dce6686880 0x55dce6689e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.742+0000 7f9192ffd700 1 -- 172.21.15.7:0/2589200891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dce6686880 msgr2=0x55dce6689e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:38.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.742+0000 7f9192ffd700 1 --2- 172.21.15.7:0/2589200891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dce6686880 0x55dce6689e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.742+0000 7f9192ffd700 1 -- 172.21.15.7:0/2589200891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dce6685f60 msgr2=0x55dce66874f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:38.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.742+0000 7f9192ffd700 1 --2- 172.21.15.7:0/2589200891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dce6685f60 0x55dce66874f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.742+0000 7f9192ffd700 1 -- 172.21.15.7:0/2589200891 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dce668a850 con 0x55dce66807a0 2024-06-10T01:16:38.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.743+0000 7f91927fc700 1 --2- 172.21.15.7:0/2589200891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dce6685f60 0x55dce66874f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:38.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.743+0000 7f9192ffd700 1 --2- 172.21.15.7:0/2589200891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dce66807a0 0x55dce6680260 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f91840cd960 tx=0x7f918401a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e653f2af89793d9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:38.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.743+0000 7f9178ff9700 1 -- 172.21.15.7:0/2589200891 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91840b4070 con 0x55dce66807a0 2024-06-10T01:16:38.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.743+0000 7f9178ff9700 1 -- 172.21.15.7:0/2589200891 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91840a0b00 con 0x55dce66807a0 2024-06-10T01:16:38.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.743+0000 7f9178ff9700 1 -- 172.21.15.7:0/2589200891 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9184003ff0 con 0x55dce66807a0 2024-06-10T01:16:38.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.744+0000 7f91a402db80 1 -- 172.21.15.7:0/2589200891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dce66807a0 msgr2=0x55dce6680260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:38.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.744+0000 7f91a402db80 1 --2- 172.21.15.7:0/2589200891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dce66807a0 0x55dce6680260 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f91840cd960 tx=0x7f918401a040 comp rx=0 tx=0).stop 2024-06-10T01:16:38.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.744+0000 7f91a402db80 1 -- 172.21.15.7:0/2589200891 shutdown_connections 2024-06-10T01:16:38.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.744+0000 7f91a402db80 1 --2- 172.21.15.7:0/2589200891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dce6686880 0x55dce6689e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.744+0000 7f91a402db80 1 --2- 172.21.15.7:0/2589200891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dce66807a0 0x55dce6680260 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.744+0000 7f91a402db80 1 --2- 172.21.15.7:0/2589200891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dce6685f60 0x55dce66874f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.744+0000 7f91a402db80 1 -- 172.21.15.7:0/2589200891 >> 172.21.15.7:0/2589200891 conn(0x55dce65ad080 msgr2=0x55dce6683e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:38.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.744+0000 7f91a402db80 1 -- 172.21.15.7:0/2589200891 shutdown_connections 2024-06-10T01:16:38.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.744+0000 7f91a402db80 1 -- 172.21.15.7:0/2589200891 wait complete. 2024-06-10T01:16:38.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.744+0000 7f91a402db80 1 -- start start 2024-06-10T01:16:38.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.744+0000 7f91a402db80 1 Processor -- start 2024-06-10T01:16:38.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.744+0000 7f91a402db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dce66807a0 0x55dce66927e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.744+0000 7f91a402db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dce6685f60 0x55dce6692d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.744+0000 7f91a402db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dce6686880 0x55dce66a1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.744+0000 7f91a402db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f91941ae920 con 0x55dce6686880 2024-06-10T01:16:38.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.744+0000 7f91a402db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f91941afd90 con 0x55dce66807a0 2024-06-10T01:16:38.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.744+0000 7f91a402db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f91941aeff0 con 0x55dce6685f60 2024-06-10T01:16:38.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.745+0000 7f91a402db80 10 client.0 _ll_register_callbacks cb 0x55dce65675b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:16:38.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.745+0000 7f9192ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dce66807a0 0x55dce66927e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.745+0000 7f91927fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dce6685f60 0x55dce6692d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.745+0000 7f9192ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dce66807a0 0x55dce66927e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:36492/0 (socket says 172.21.15.7:36492) 2024-06-10T01:16:38.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.745+0000 7f9192ffd700 1 -- 172.21.15.7:0/2306024684 learned_addr learned my addr 172.21.15.7:0/2306024684 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:38.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.745+0000 7f91937fe700 1 --2- 172.21.15.7:0/2306024684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dce6686880 0x55dce66a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.745+0000 7f9192ffd700 1 -- 172.21.15.7:0/2306024684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dce6685f60 msgr2=0x55dce6692d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:38.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.745+0000 7f9192ffd700 1 --2- 172.21.15.7:0/2306024684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dce6685f60 0x55dce6692d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.745+0000 7f9192ffd700 1 -- 172.21.15.7:0/2306024684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dce6686880 msgr2=0x55dce66a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:38.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.745+0000 7f9192ffd700 1 --2- 172.21.15.7:0/2306024684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dce6686880 0x55dce66a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.745+0000 7f9192ffd700 1 -- 172.21.15.7:0/2306024684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dce66a1b00 con 0x55dce66807a0 2024-06-10T01:16:38.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.745+0000 7f9192ffd700 1 --2- 172.21.15.7:0/2306024684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dce66807a0 0x55dce66927e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9184099120 tx=0x7f9184009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7105021c3bb58cc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:38.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.745+0000 7f9176ff5700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:16:38.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.745+0000 7f9176ff5700 1 -- 172.21.15.7:0/2306024684 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91840b9020 con 0x55dce66807a0 2024-06-10T01:16:38.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.745+0000 7f9176ff5700 1 -- 172.21.15.7:0/2306024684 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f918409aa50 con 0x55dce66807a0 2024-06-10T01:16:38.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.745+0000 7f91a402db80 1 -- 172.21.15.7:0/2306024684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91840a0b00 con 0x55dce66807a0 2024-06-10T01:16:38.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.746+0000 7f9176ff5700 1 -- 172.21.15.7:0/2306024684 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f91840b4070 con 0x55dce66807a0 2024-06-10T01:16:38.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.746+0000 7f91a402db80 10 client.27521 fetch_fsmap learned FSMap version 39 2024-06-10T01:16:38.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.746+0000 7f91a402db80 1 -- 172.21.15.7:0/2306024684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55dce6704810 con 0x55dce66807a0 2024-06-10T01:16:38.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.746+0000 7f9176ff5700 1 -- 172.21.15.7:0/2306024684 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91840a0b00 con 0x55dce66807a0 2024-06-10T01:16:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.746+0000 7f9176ff5700 1 -- 172.21.15.7:0/2306024684 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7f9184085ce0 con 0x55dce66807a0 2024-06-10T01:16:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.746+0000 7f91a402db80 10 client.27521 fetch_fsmap finished waiting for FSMap version 39 2024-06-10T01:16:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.746+0000 7f91a402db80 10 client.27521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:16:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.746+0000 7f91a402db80 20 client.27521 populate_metadata read hostname 'smithi007' 2024-06-10T01:16:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.746+0000 7f91a402db80 1 --2- 172.21.15.7:0/2306024684 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55dce6704ee0 0x55dce6707300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.746+0000 7f91a402db80 4 client.27521 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:16:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.746+0000 7f91a402db80 1 -- 172.21.15.7:0/2306024684 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dce65d92d0 con 0x55dce6704ee0 2024-06-10T01:16:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.746+0000 7f91927fc700 1 --2- 172.21.15.7:0/2306024684 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55dce6704ee0 0x55dce6707300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.747+0000 7f91927fc700 1 --2- 172.21.15.7:0/2306024684 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55dce6704ee0 0x55dce6707300 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef60f2a5ce34ddaa server_cookie=17a35cca6db4cbef in_seq=0 out_seq=0 2024-06-10T01:16:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.747+0000 7f9176ff5700 10 client.27521 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.747+0000 7f9176ff5700 1 -- 172.21.15.7:0/2306024684 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55dce65d92d0 con 0x55dce6704ee0 2024-06-10T01:16:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.747+0000 7f9176ff5700 10 client.27521 handle_command_reply: tid=0 2024-06-10T01:16:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.747+0000 7f91a402db80 1 client.27521 shutdown 2024-06-10T01:16:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.747+0000 7f91a402db80 1 -- 172.21.15.7:0/2306024684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dce66807a0 msgr2=0x55dce66927e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.747+0000 7f91a402db80 1 --2- 172.21.15.7:0/2306024684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dce66807a0 0x55dce66927e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9184099120 tx=0x7f9184009ca0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.747+0000 7f91a402db80 1 -- 172.21.15.7:0/2306024684 shutdown_connections 2024-06-10T01:16:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.747+0000 7f91a402db80 1 --2- 172.21.15.7:0/2306024684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dce6686880 0x55dce66a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.747+0000 7f91a402db80 1 --2- 172.21.15.7:0/2306024684 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55dce6704ee0 0x55dce6707300 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.747+0000 7f91a402db80 1 --2- 172.21.15.7:0/2306024684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dce66807a0 0x55dce66927e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.747+0000 7f91a402db80 1 --2- 172.21.15.7:0/2306024684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dce6685f60 0x55dce6692d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.747+0000 7f91a402db80 1 -- 172.21.15.7:0/2306024684 >> 172.21.15.7:0/2306024684 conn(0x55dce65ad080 msgr2=0x55dce668b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.747+0000 7f9176ff5700 0 client.27521 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.747+0000 7f91a402db80 1 -- 172.21.15.7:0/2306024684 shutdown_connections 2024-06-10T01:16:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.747+0000 7f91a402db80 1 -- 172.21.15.7:0/2306024684 wait complete. 2024-06-10T01:16:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.747+0000 7f91a402db80 20 client.27521 trim_cache size 0 max 16384 2024-06-10T01:16:38.748 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:16:38.748 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T01:16:38.748 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-10T01:16:38.748 INFO:teuthology.orchestra.run.smithi007.stdout: "fdcbb000-3493-460b-b23a-adb0a4a8881e": { 2024-06-10T01:16:38.748 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-10T01:16:38.748 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "fdcbb000-3493-460b-b23a-adb0a4a8881e", 2024-06-10T01:16:38.748 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-10T01:16:38.748 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:16:38.748 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:16:38.748 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:16:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.748+0000 7f91997aa700 1 -- 172.21.15.7:0/2183110090 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f917c0a2c20 msgr2=0x7f917c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.748+0000 7f91997aa700 1 --2- 172.21.15.7:0/2183110090 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f917c0a2c20 0x7f917c0a50e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9194078930 tx=0x7f91880073d0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.748+0000 7f91997aa700 1 -- 172.21.15.7:0/2183110090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9194101380 msgr2=0x7f9194103030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.748+0000 7f91997aa700 1 --2- 172.21.15.7:0/2183110090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9194101380 0x7f9194103030 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9184000c00 tx=0x7f91840047b0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.748+0000 7f91997aa700 1 -- 172.21.15.7:0/2183110090 shutdown_connections 2024-06-10T01:16:38.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.748+0000 7f91997aa700 1 --2- 172.21.15.7:0/2183110090 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f917c0a2c20 0x7f917c0a50e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.748+0000 7f91997aa700 1 --2- 172.21.15.7:0/2183110090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91941073c0 0x7f9194103ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.748+0000 7f91997aa700 1 --2- 172.21.15.7:0/2183110090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9194101380 0x7f9194103030 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.748+0000 7f91997aa700 1 --2- 172.21.15.7:0/2183110090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9194101d40 0x7f9194103570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:38.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.748+0000 7f91997aa700 1 -- 172.21.15.7:0/2183110090 >> 172.21.15.7:0/2183110090 conn(0x7f9194078330 msgr2=0x7f91941006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:38.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.748+0000 7f91997aa700 1 -- 172.21.15.7:0/2183110090 shutdown_connections 2024-06-10T01:16:38.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:38.748+0000 7f91997aa700 1 -- 172.21.15.7:0/2183110090 wait complete. 2024-06-10T01:16:38.750 INFO:tasks.workunit.client.0.smithi007.stdout: 1 787686 7.57 MB/sec execute 439 sec latency 73.434 ms 2024-06-10T01:16:39.012 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:38 smithi007 ceph-mon[32398]: pgmap v751: 129 pgs: 129 active+clean; 135 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 2.5 MiB/s wr, 300 op/s 2024-06-10T01:16:39.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:38 smithi175 ceph-mon[36636]: pgmap v751: 129 pgs: 129 active+clean; 135 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 2.5 MiB/s wr, 300 op/s 2024-06-10T01:16:39.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:38 smithi078 ceph-mon[36683]: pgmap v751: 129 pgs: 129 active+clean; 135 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 2.5 MiB/s wr, 300 op/s 2024-06-10T01:16:39.434 INFO:tasks.cephfs.filesystem:scrub status for tag:fdcbb000-3493-460b-b23a-adb0a4a8881e - {'path': '/', 'tag': 'fdcbb000-3493-460b-b23a-adb0a4a8881e', 'options': 'recursive,force'} 2024-06-10T01:16:39.748 INFO:tasks.workunit.client.0.smithi007.stdout: 1 789104 7.57 MB/sec execute 440 sec latency 12.110 ms 2024-06-10T01:16:40.748 INFO:tasks.workunit.client.0.smithi007.stdout: 1 790715 7.57 MB/sec execute 441 sec latency 63.341 ms 2024-06-10T01:16:41.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:40 smithi175 ceph-mon[36636]: pgmap v752: 129 pgs: 129 active+clean; 132 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 3.0 MiB/s wr, 338 op/s 2024-06-10T01:16:41.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:40 smithi007 ceph-mon[32398]: pgmap v752: 129 pgs: 129 active+clean; 132 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 3.0 MiB/s wr, 338 op/s 2024-06-10T01:16:41.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:40 smithi078 ceph-mon[36683]: pgmap v752: 129 pgs: 129 active+clean; 132 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 3.0 MiB/s wr, 338 op/s 2024-06-10T01:16:41.749 INFO:tasks.workunit.client.0.smithi007.stdout: 1 792375 7.58 MB/sec execute 442 sec latency 20.025 ms 2024-06-10T01:16:42.748 INFO:tasks.workunit.client.0.smithi007.stdout: 1 794649 7.58 MB/sec execute 443 sec latency 16.515 ms 2024-06-10T01:16:43.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:42 smithi175 ceph-mon[36636]: pgmap v753: 129 pgs: 129 active+clean; 135 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 2.5 MiB/s wr, 288 op/s 2024-06-10T01:16:43.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:42 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:16:43.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:42 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:16:43.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:42 smithi007 ceph-mon[32398]: pgmap v753: 129 pgs: 129 active+clean; 135 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 2.5 MiB/s wr, 288 op/s 2024-06-10T01:16:43.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:42 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:16:43.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:42 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:16:43.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:42 smithi078 ceph-mon[36683]: pgmap v753: 129 pgs: 129 active+clean; 135 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 2.5 MiB/s wr, 288 op/s 2024-06-10T01:16:43.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:42 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:16:43.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:42 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:16:43.749 INFO:tasks.workunit.client.0.smithi007.stdout: 1 796101 7.59 MB/sec execute 444 sec latency 12.068 ms 2024-06-10T01:16:44.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:43 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:16:44.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:43 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:16:44.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:43 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:16:44.749 INFO:tasks.workunit.client.0.smithi007.stdout: 1 798342 7.60 MB/sec execute 445 sec latency 12.616 ms 2024-06-10T01:16:45.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:44 smithi175 ceph-mon[36636]: pgmap v754: 129 pgs: 129 active+clean; 136 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 2.8 MiB/s wr, 325 op/s 2024-06-10T01:16:45.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:44 smithi007 ceph-mon[32398]: pgmap v754: 129 pgs: 129 active+clean; 136 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 2.8 MiB/s wr, 325 op/s 2024-06-10T01:16:45.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:44 smithi078 ceph-mon[36683]: pgmap v754: 129 pgs: 129 active+clean; 136 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 2.8 MiB/s wr, 325 op/s 2024-06-10T01:16:45.749 INFO:tasks.workunit.client.0.smithi007.stdout: 1 799463 7.59 MB/sec execute 446 sec latency 62.313 ms 2024-06-10T01:16:46.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:45 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:16:46.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:45 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:16:46.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:45 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:16:46.749 INFO:tasks.workunit.client.0.smithi007.stdout: 1 801425 7.59 MB/sec execute 447 sec latency 20.728 ms 2024-06-10T01:16:47.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:46 smithi175 ceph-mon[36636]: pgmap v755: 129 pgs: 129 active+clean; 135 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 2.7 MiB/s wr, 323 op/s 2024-06-10T01:16:47.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:46 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:16:47.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:46 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:16:47.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:46 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:16:47.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:46 smithi007 ceph-mon[32398]: pgmap v755: 129 pgs: 129 active+clean; 135 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 2.7 MiB/s wr, 323 op/s 2024-06-10T01:16:47.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:46 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:16:47.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:46 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:16:47.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:46 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:16:47.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:46 smithi078 ceph-mon[36683]: pgmap v755: 129 pgs: 129 active+clean; 135 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 2.7 MiB/s wr, 323 op/s 2024-06-10T01:16:47.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:46 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:16:47.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:46 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:16:47.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:46 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:16:47.749 INFO:tasks.workunit.client.0.smithi007.stdout: 1 803078 7.60 MB/sec execute 448 sec latency 10.966 ms 2024-06-10T01:16:47.896 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 15.8 2024-06-10T01:16:47.896 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:16:48.170 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:16:48.749 INFO:tasks.workunit.client.0.smithi007.stdout: 1 805392 7.61 MB/sec execute 449 sec latency 16.047 ms 2024-06-10T01:16:49.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:48 smithi175 ceph-mon[36636]: pgmap v756: 129 pgs: 129 active+clean; 137 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 2.6 MiB/s wr, 327 op/s 2024-06-10T01:16:49.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:48 smithi007 ceph-mon[32398]: pgmap v756: 129 pgs: 129 active+clean; 137 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 2.6 MiB/s wr, 327 op/s 2024-06-10T01:16:49.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:48 smithi078 ceph-mon[36683]: pgmap v756: 129 pgs: 129 active+clean; 137 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 2.6 MiB/s wr, 327 op/s 2024-06-10T01:16:49.749 INFO:tasks.workunit.client.0.smithi007.stdout: 1 806934 7.61 MB/sec execute 450 sec latency 12.612 ms 2024-06-10T01:16:50.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.140+0000 7f68e4681700 1 -- 172.21.15.7:0/2288777809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68dc101d50 msgr2=0x7f68dc10e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:50.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.140+0000 7f68e4681700 1 --2- 172.21.15.7:0/2288777809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68dc101d50 0x7f68dc10e240 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f68d400b0a0 tx=0x7f68d400b3b0 comp rx=0 tx=0).stop 2024-06-10T01:16:50.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.140+0000 7f68e4681700 1 -- 172.21.15.7:0/2288777809 shutdown_connections 2024-06-10T01:16:50.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.140+0000 7f68e4681700 1 --2- 172.21.15.7:0/2288777809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68dc101d50 0x7f68dc10e240 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:50.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.140+0000 7f68e4681700 1 --2- 172.21.15.7:0/2288777809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68dc107400 0x7f68dc1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:50.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.140+0000 7f68e4681700 1 --2- 172.21.15.7:0/2288777809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68dc101390 0x7f68dc101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:50.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.140+0000 7f68e4681700 1 -- 172.21.15.7:0/2288777809 >> 172.21.15.7:0/2288777809 conn(0x7f68dc078340 msgr2=0x7f68dc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:50.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.140+0000 7f68e4681700 1 -- 172.21.15.7:0/2288777809 shutdown_connections 2024-06-10T01:16:50.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.141+0000 7f68e4681700 1 -- 172.21.15.7:0/2288777809 wait complete. 2024-06-10T01:16:50.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.142+0000 7f68e4681700 1 Processor -- start 2024-06-10T01:16:50.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.142+0000 7f68e4681700 1 -- start start 2024-06-10T01:16:50.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.143+0000 7f68e4681700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68dc101390 0x7f68dc1082c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:50.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.143+0000 7f68e4681700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68dc101d50 0x7f68dc108800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:50.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.143+0000 7f68e4681700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68dc107400 0x7f68dc108d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:50.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.143+0000 7f68e4681700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc111760 con 0x7f68dc101390 2024-06-10T01:16:50.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.143+0000 7f68e4681700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc1115e0 con 0x7f68dc101d50 2024-06-10T01:16:50.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.143+0000 7f68e4681700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc1118e0 con 0x7f68dc107400 2024-06-10T01:16:50.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.144+0000 7f68e241d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68dc101390 0x7f68dc1082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:50.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.144+0000 7f68e241d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68dc101390 0x7f68dc1082c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40134/0 (socket says 172.21.15.7:40134) 2024-06-10T01:16:50.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.144+0000 7f68e241d700 1 -- 172.21.15.7:0/2628117784 learned_addr learned my addr 172.21.15.7:0/2628117784 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:50.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.144+0000 7f68e2c1e700 1 --2- 172.21.15.7:0/2628117784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68dc107400 0x7f68dc108d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:50.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.145+0000 7f68e1c1c700 1 --2- 172.21.15.7:0/2628117784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68dc101d50 0x7f68dc108800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:50.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.145+0000 7f68e2c1e700 1 -- 172.21.15.7:0/2628117784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68dc101d50 msgr2=0x7f68dc108800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:50.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.145+0000 7f68e2c1e700 1 --2- 172.21.15.7:0/2628117784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68dc101d50 0x7f68dc108800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:50.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.145+0000 7f68e2c1e700 1 -- 172.21.15.7:0/2628117784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68dc101390 msgr2=0x7f68dc1082c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:50.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.145+0000 7f68e2c1e700 1 --2- 172.21.15.7:0/2628117784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68dc101390 0x7f68dc1082c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:50.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.146+0000 7f68e2c1e700 1 -- 172.21.15.7:0/2628117784 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68dc1ab5e0 con 0x7f68dc107400 2024-06-10T01:16:50.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.146+0000 7f68e1c1c700 1 --2- 172.21.15.7:0/2628117784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68dc101d50 0x7f68dc108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:16:50.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.146+0000 7f68e241d700 1 --2- 172.21.15.7:0/2628117784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68dc101390 0x7f68dc1082c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:50.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.146+0000 7f68e2c1e700 1 --2- 172.21.15.7:0/2628117784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68dc107400 0x7f68dc108d40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f68d4015000 tx=0x7f68d4004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:50.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.146+0000 7f68d37fe700 1 -- 172.21.15.7:0/2628117784 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d4021070 con 0x7f68dc107400 2024-06-10T01:16:50.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.146+0000 7f68e4681700 1 -- 172.21.15.7:0/2628117784 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68dc1ab870 con 0x7f68dc107400 2024-06-10T01:16:50.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.146+0000 7f68d37fe700 1 -- 172.21.15.7:0/2628117784 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68d4007b80 con 0x7f68dc107400 2024-06-10T01:16:50.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.146+0000 7f68e4681700 1 -- 172.21.15.7:0/2628117784 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68dc1abd70 con 0x7f68dc107400 2024-06-10T01:16:50.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.147+0000 7f68d37fe700 1 -- 172.21.15.7:0/2628117784 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d40052a0 con 0x7f68dc107400 2024-06-10T01:16:50.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.148+0000 7f68e4681700 1 -- 172.21.15.7:0/2628117784 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68dc102ab0 con 0x7f68dc107400 2024-06-10T01:16:50.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.154+0000 7f68d37fe700 1 -- 172.21.15.7:0/2628117784 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f68d4024070 con 0x7f68dc107400 2024-06-10T01:16:50.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.156+0000 7f68d37fe700 1 --2- 172.21.15.7:0/2628117784 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f68c80a2d70 0x7f68c80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:50.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.156+0000 7f68d37fe700 1 -- 172.21.15.7:0/2628117784 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f68d400f020 con 0x7f68dc107400 2024-06-10T01:16:50.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.159+0000 7f68e241d700 1 --2- 172.21.15.7:0/2628117784 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f68c80a2d70 0x7f68c80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:50.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.160+0000 7f68d37fe700 1 -- 172.21.15.7:0/2628117784 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f68d402b020 con 0x7f68dc107400 2024-06-10T01:16:50.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.166+0000 7f68e241d700 1 --2- 172.21.15.7:0/2628117784 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f68c80a2d70 0x7f68c80a5230 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009ea0 tx=0x7f68d8009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:50.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.509+0000 7f68e4681700 1 -- 172.21.15.7:0/2628117784 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68dc1abf40 con 0x7f68dc107400 2024-06-10T01:16:50.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.510+0000 7f68d37fe700 1 -- 172.21.15.7:0/2628117784 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11856 (secure 0 0 0) 0x7f68d4011e50 con 0x7f68dc107400 2024-06-10T01:16:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:16:50.504 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:16:27.227033+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24434,24436,24451]},"id":1}]} 2024-06-10T01:16:50.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.512+0000 7f68e4681700 1 -- 172.21.15.7:0/2628117784 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f68c80a2d70 msgr2=0x7f68c80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:50.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.512+0000 7f68e4681700 1 --2- 172.21.15.7:0/2628117784 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f68c80a2d70 0x7f68c80a5230 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009ea0 tx=0x7f68d8009210 comp rx=0 tx=0).stop 2024-06-10T01:16:50.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.512+0000 7f68e4681700 1 -- 172.21.15.7:0/2628117784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68dc107400 msgr2=0x7f68dc108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:50.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.512+0000 7f68e4681700 1 --2- 172.21.15.7:0/2628117784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68dc107400 0x7f68dc108d40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f68d4015000 tx=0x7f68d4004c80 comp rx=0 tx=0).stop 2024-06-10T01:16:50.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.512+0000 7f68e4681700 1 -- 172.21.15.7:0/2628117784 shutdown_connections 2024-06-10T01:16:50.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.512+0000 7f68e4681700 1 --2- 172.21.15.7:0/2628117784 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f68c80a2d70 0x7f68c80a5230 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:50.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.512+0000 7f68e4681700 1 --2- 172.21.15.7:0/2628117784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68dc107400 0x7f68dc108d40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:50.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.512+0000 7f68e4681700 1 --2- 172.21.15.7:0/2628117784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68dc101390 0x7f68dc1082c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:50.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.512+0000 7f68e4681700 1 --2- 172.21.15.7:0/2628117784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68dc101d50 0x7f68dc108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:50.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.512+0000 7f68e4681700 1 -- 172.21.15.7:0/2628117784 >> 172.21.15.7:0/2628117784 conn(0x7f68dc078340 msgr2=0x7f68dc0ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:50.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.512+0000 7f68e4681700 1 -- 172.21.15.7:0/2628117784 shutdown_connections 2024-06-10T01:16:50.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:50.512+0000 7f68e4681700 1 -- 172.21.15.7:0/2628117784 wait complete. 2024-06-10T01:16:50.507 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 39 2024-06-10T01:16:50.749 INFO:tasks.workunit.client.0.smithi007.stdout: 1 809214 7.63 MB/sec execute 451 sec latency 63.056 ms 2024-06-10T01:16:51.026 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:50 smithi007 ceph-mon[32398]: pgmap v757: 129 pgs: 129 active+clean; 138 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 3.0 MiB/s wr, 342 op/s 2024-06-10T01:16:51.027 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:50 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2628117784' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:16:51.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:50 smithi175 ceph-mon[36636]: pgmap v757: 129 pgs: 129 active+clean; 138 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 3.0 MiB/s wr, 342 op/s 2024-06-10T01:16:51.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:50 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2628117784' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:16:51.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:50 smithi078 ceph-mon[36683]: pgmap v757: 129 pgs: 129 active+clean; 138 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 3.0 MiB/s wr, 342 op/s 2024-06-10T01:16:51.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:50 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2628117784' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:16:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.730+0000 7f010013b640 1 -- 172.21.15.7:0/2379884244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00f8100900 msgr2=0x7f00f8100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.730+0000 7f010013b640 1 --2- 172.21.15.7:0/2379884244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00f8100900 0x7f00f8100d20 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f00f4009900 tx=0x7f00f402a410 comp rx=0 tx=0).stop 2024-06-10T01:16:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.730+0000 7f010013b640 1 -- 172.21.15.7:0/2379884244 shutdown_connections 2024-06-10T01:16:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.730+0000 7f010013b640 1 --2- 172.21.15.7:0/2379884244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00f81024b0 0x7f00f8108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.730+0000 7f010013b640 1 --2- 172.21.15.7:0/2379884244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00f8101af0 0x7f00f8101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.730+0000 7f010013b640 1 --2- 172.21.15.7:0/2379884244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00f8100900 0x7f00f8100d20 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.730+0000 7f010013b640 1 -- 172.21.15.7:0/2379884244 >> 172.21.15.7:0/2379884244 conn(0x7f00f80fbe70 msgr2=0x7f00f80fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.730+0000 7f010013b640 1 -- 172.21.15.7:0/2379884244 shutdown_connections 2024-06-10T01:16:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.730+0000 7f010013b640 1 -- 172.21.15.7:0/2379884244 wait complete. 2024-06-10T01:16:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.731+0000 7f010013b640 1 Processor -- start 2024-06-10T01:16:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.731+0000 7f010013b640 1 -- start start 2024-06-10T01:16:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.731+0000 7f010013b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00f8100900 0x7f00f8195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.731+0000 7f010013b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00f8101af0 0x7f00f8195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.731+0000 7f010013b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00f81024b0 0x7f00f819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.731+0000 7f010013b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f00f810b480 con 0x7f00f81024b0 2024-06-10T01:16:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.731+0000 7f010013b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f00f810b300 con 0x7f00f8100900 2024-06-10T01:16:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.731+0000 7f010013b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f00f810b600 con 0x7f00f8101af0 2024-06-10T01:16:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.731+0000 7f00fe6d8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00f81024b0 0x7f00f819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.731+0000 7f00fded7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00f8100900 0x7f00f8195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.731+0000 7f00fded7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00f8100900 0x7f00f8195900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:52414/0 (socket says 172.21.15.7:52414) 2024-06-10T01:16:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.731+0000 7f00fded7640 1 -- 172.21.15.7:0/1375298439 learned_addr learned my addr 172.21.15.7:0/1375298439 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.731+0000 7f00fd6d6640 1 --2- 172.21.15.7:0/1375298439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00f8101af0 0x7f00f8195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.731+0000 7f00fe6d8640 1 -- 172.21.15.7:0/1375298439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00f8101af0 msgr2=0x7f00f8195e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.731+0000 7f00fe6d8640 1 --2- 172.21.15.7:0/1375298439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00f8101af0 0x7f00f8195e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.731+0000 7f00fe6d8640 1 -- 172.21.15.7:0/1375298439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00f8100900 msgr2=0x7f00f8195900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.731+0000 7f00fe6d8640 1 --2- 172.21.15.7:0/1375298439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00f8100900 0x7f00f8195900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.731+0000 7f00fe6d8640 1 -- 172.21.15.7:0/1375298439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f819d520 con 0x7f00f81024b0 2024-06-10T01:16:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.731+0000 7f00fded7640 1 --2- 172.21.15.7:0/1375298439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00f8100900 0x7f00f8195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.732+0000 7f00fe6d8640 1 --2- 172.21.15.7:0/1375298439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00f81024b0 0x7f00f819ceb0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f00f0009fd0 tx=0x7f00f000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.732+0000 7f00eeffd640 1 -- 172.21.15.7:0/1375298439 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f000ce20 con 0x7f00f81024b0 2024-06-10T01:16:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.732+0000 7f00eeffd640 1 -- 172.21.15.7:0/1375298439 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00f0005020 con 0x7f00f81024b0 2024-06-10T01:16:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.732+0000 7f010013b640 1 -- 172.21.15.7:0/1375298439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00f819d810 con 0x7f00f81024b0 2024-06-10T01:16:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.732+0000 7f00eeffd640 1 -- 172.21.15.7:0/1375298439 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f0010720 con 0x7f00f81024b0 2024-06-10T01:16:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.732+0000 7f010013b640 1 -- 172.21.15.7:0/1375298439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00f819dd60 con 0x7f00f81024b0 2024-06-10T01:16:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.735+0000 7f00eeffd640 1 -- 172.21.15.7:0/1375298439 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f00f0005540 con 0x7f00f81024b0 2024-06-10T01:16:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.735+0000 7f0102e46740 1 Processor -- start 2024-06-10T01:16:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.736+0000 7f0102e46740 1 -- start start 2024-06-10T01:16:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.736+0000 7f00eeffd640 1 --2- 172.21.15.7:0/1375298439 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f00e40a2c00 0x7f00e40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.736+0000 7f0102e46740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55761faad820 0x55761faadc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.736+0000 7f0102e46740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55761faae210 0x55761fab2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.736+0000 7f0102e46740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55761fab3380 0x55761fab5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.736+0000 7f0102e46740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55761fa084f0 con 0x55761fab3380 2024-06-10T01:16:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.736+0000 7f0102e46740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55761f9a6870 con 0x55761faad820 2024-06-10T01:16:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.736+0000 7f0102e46740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55761fa044b0 con 0x55761faae210 2024-06-10T01:16:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.736+0000 7f00fd6d6640 1 --2- 172.21.15.7:0/1375298439 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f00e40a2c00 0x7f00e40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.736+0000 7f00fded7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55761faad820 0x55761faadc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.736+0000 7f00fd6d6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55761faae210 0x55761fab2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.736+0000 7f00fded7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55761faad820 0x55761faadc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:52430/0 (socket says 172.21.15.7:52430) 2024-06-10T01:16:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.736+0000 7f00fe6d8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55761fab3380 0x55761fab5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.736+0000 7f00fded7640 1 -- 172.21.15.7:0/221191781 learned_addr learned my addr 172.21.15.7:0/221191781 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.736+0000 7f00eeffd640 1 -- 172.21.15.7:0/1375298439 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f00f00cf980 con 0x7f00f81024b0 2024-06-10T01:16:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.737+0000 7f00fded7640 1 -- 172.21.15.7:0/221191781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55761faae210 msgr2=0x55761fab2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.737+0000 7f00fded7640 1 --2- 172.21.15.7:0/221191781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55761faae210 0x55761fab2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.737+0000 7f00fded7640 1 -- 172.21.15.7:0/221191781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55761fab3380 msgr2=0x55761fab5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.737+0000 7f00fded7640 1 --2- 172.21.15.7:0/221191781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55761fab3380 0x55761fab5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.737+0000 7f00fded7640 1 -- 172.21.15.7:0/221191781 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55761f9c29b0 con 0x55761faad820 2024-06-10T01:16:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.737+0000 7f00fe6d8640 1 --2- 172.21.15.7:0/221191781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55761fab3380 0x55761fab5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.737+0000 7f00fd6d6640 1 --2- 172.21.15.7:0/221191781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55761faae210 0x55761fab2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.737+0000 7f00fd6d6640 1 --2- 172.21.15.7:0/1375298439 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f00e40a2c00 0x7f00e40a50c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f00f8196e10 tx=0x7f00e800bba0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.737+0000 7f00fded7640 1 --2- 172.21.15.7:0/221191781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55761faad820 0x55761faadc40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f00f4005eb0 tx=0x7f00f4031620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fd3446a4f16b5c5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.737+0000 7f00e3fff640 1 -- 172.21.15.7:0/221191781 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f4035040 con 0x55761faad820 2024-06-10T01:16:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.737+0000 7f00e3fff640 1 -- 172.21.15.7:0/221191781 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00f4004230 con 0x55761faad820 2024-06-10T01:16:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.737+0000 7f00e3fff640 1 -- 172.21.15.7:0/221191781 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f403c640 con 0x55761faad820 2024-06-10T01:16:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.738+0000 7f0102e46740 1 -- 172.21.15.7:0/221191781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55761faad820 msgr2=0x55761faadc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.738+0000 7f0102e46740 1 --2- 172.21.15.7:0/221191781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55761faad820 0x55761faadc40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f00f4005eb0 tx=0x7f00f4031620 comp rx=0 tx=0).stop 2024-06-10T01:16:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.738+0000 7f0102e46740 1 -- 172.21.15.7:0/221191781 shutdown_connections 2024-06-10T01:16:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.738+0000 7f0102e46740 1 --2- 172.21.15.7:0/221191781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55761fab3380 0x55761fab5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.738+0000 7f0102e46740 1 --2- 172.21.15.7:0/221191781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55761faae210 0x55761fab2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.738+0000 7f0102e46740 1 --2- 172.21.15.7:0/221191781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55761faad820 0x55761faadc40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.738+0000 7f0102e46740 1 -- 172.21.15.7:0/221191781 >> 172.21.15.7:0/221191781 conn(0x55761faa0680 msgr2=0x55761faa0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.738+0000 7f0102e46740 1 -- 172.21.15.7:0/221191781 shutdown_connections 2024-06-10T01:16:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.738+0000 7f0102e46740 1 -- 172.21.15.7:0/221191781 wait complete. 2024-06-10T01:16:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.739+0000 7f0102e46740 1 -- start start 2024-06-10T01:16:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.739+0000 7f0102e46740 1 Processor -- start 2024-06-10T01:16:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.740+0000 7f0102e46740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55761faad820 0x55761fab7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.740+0000 7f0102e46740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55761faae210 0x55761fab7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.740+0000 7f0102e46740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55761fab3380 0x55761fabeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.740+0000 7f0102e46740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55761fab62d0 con 0x55761fab3380 2024-06-10T01:16:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.740+0000 7f0102e46740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55761f991ae0 con 0x55761faae210 2024-06-10T01:16:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.740+0000 7f0102e46740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55761fab6450 con 0x55761faad820 2024-06-10T01:16:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.740+0000 7f00fd6d6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55761faae210 0x55761fab7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.740+0000 7f00fded7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55761faad820 0x55761fab7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.740+0000 7f00fded7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55761faad820 0x55761fab7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:38066/0 (socket says 172.21.15.7:38066) 2024-06-10T01:16:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.740+0000 7f00fd6d6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55761faae210 0x55761fab7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:52442/0 (socket says 172.21.15.7:52442) 2024-06-10T01:16:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.740+0000 7f00fded7640 1 -- 172.21.15.7:0/3040334983 learned_addr learned my addr 172.21.15.7:0/3040334983 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.740+0000 7f0102e46740 10 client.0 _ll_register_callbacks cb 0x55761f9ca9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:16:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.740+0000 7f00fe6d8640 1 --2- 172.21.15.7:0/3040334983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55761fab3380 0x55761fabeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.740+0000 7f00fded7640 1 -- 172.21.15.7:0/3040334983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55761faae210 msgr2=0x55761fab7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.740+0000 7f00fded7640 1 --2- 172.21.15.7:0/3040334983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55761faae210 0x55761fab7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.740+0000 7f00fded7640 1 -- 172.21.15.7:0/3040334983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55761fab3380 msgr2=0x55761fabeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.740+0000 7f00fded7640 1 --2- 172.21.15.7:0/3040334983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55761fab3380 0x55761fabeff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.740+0000 7f00fded7640 1 -- 172.21.15.7:0/3040334983 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55761f71c210 con 0x55761faad820 2024-06-10T01:16:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.740+0000 7f00fe6d8640 1 --2- 172.21.15.7:0/3040334983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55761fab3380 0x55761fabeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:16:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.741+0000 7f00fded7640 1 --2- 172.21.15.7:0/3040334983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55761faad820 0x55761fab7a70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f00f4005e80 tx=0x7f00f40092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed7882bc4fc005b6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.741+0000 7f00e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:16:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.741+0000 7f00e17fa640 1 -- 172.21.15.7:0/3040334983 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f404e070 con 0x55761faad820 2024-06-10T01:16:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.741+0000 7f00e17fa640 1 -- 172.21.15.7:0/3040334983 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f00f4031d00 con 0x55761faad820 2024-06-10T01:16:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.741+0000 7f0102e46740 1 -- 172.21.15.7:0/3040334983 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00f4004230 con 0x55761faad820 2024-06-10T01:16:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.741+0000 7f00e17fa640 1 -- 172.21.15.7:0/3040334983 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f00f405a050 con 0x55761faad820 2024-06-10T01:16:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.741+0000 7f0102e46740 10 client.27565 fetch_fsmap learned FSMap version 39 2024-06-10T01:16:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.741+0000 7f0102e46740 1 -- 172.21.15.7:0/3040334983 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55761f7e8620 con 0x55761faad820 2024-06-10T01:16:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.741+0000 7f00e17fa640 1 -- 172.21.15.7:0/3040334983 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00f4004230 con 0x55761faad820 2024-06-10T01:16:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.741+0000 7f00e17fa640 1 -- 172.21.15.7:0/3040334983 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7f00f403cac0 con 0x55761faad820 2024-06-10T01:16:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.742+0000 7f0102e46740 10 client.27565 fetch_fsmap finished waiting for FSMap version 39 2024-06-10T01:16:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.742+0000 7f0102e46740 10 client.27565 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:16:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.742+0000 7f0102e46740 20 client.27565 populate_metadata read hostname 'smithi007' 2024-06-10T01:16:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.742+0000 7f0102e46740 1 --2- 172.21.15.7:0/3040334983 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55761fac34c0 0x55761fac58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.742+0000 7f0102e46740 4 client.27565 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:16:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.742+0000 7f0102e46740 1 -- 172.21.15.7:0/3040334983 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55761fa1bf10 con 0x55761fac34c0 2024-06-10T01:16:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.742+0000 7f00fd6d6640 1 --2- 172.21.15.7:0/3040334983 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55761fac34c0 0x55761fac58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.742+0000 7f00e17fa640 10 client.27565 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.742+0000 7f00fd6d6640 1 --2- 172.21.15.7:0/3040334983 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55761fac34c0 0x55761fac58b0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=add6d745b0458a18 server_cookie=12cb7d250fdc70a0 in_seq=0 out_seq=0 2024-06-10T01:16:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.743+0000 7f00e17fa640 1 -- 172.21.15.7:0/3040334983 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55761fa1bf10 con 0x55761fac34c0 2024-06-10T01:16:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.743+0000 7f00e17fa640 10 client.27565 handle_command_reply: tid=0 2024-06-10T01:16:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.743+0000 7f0102e46740 1 client.27565 shutdown 2024-06-10T01:16:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.743+0000 7f0102e46740 1 -- 172.21.15.7:0/3040334983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55761faad820 msgr2=0x55761fab7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.743+0000 7f0102e46740 1 --2- 172.21.15.7:0/3040334983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55761faad820 0x55761fab7a70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f00f4005e80 tx=0x7f00f40092d0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.743+0000 7f0102e46740 1 -- 172.21.15.7:0/3040334983 shutdown_connections 2024-06-10T01:16:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.743+0000 7f0102e46740 1 --2- 172.21.15.7:0/3040334983 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55761fac34c0 0x55761fac58b0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.743+0000 7f0102e46740 1 --2- 172.21.15.7:0/3040334983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55761fab3380 0x55761fabeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.743+0000 7f0102e46740 1 --2- 172.21.15.7:0/3040334983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55761faae210 0x55761fab7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.743+0000 7f0102e46740 1 --2- 172.21.15.7:0/3040334983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55761faad820 0x55761fab7a70 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.743+0000 7f0102e46740 1 -- 172.21.15.7:0/3040334983 >> 172.21.15.7:0/3040334983 conn(0x55761faa0680 msgr2=0x55761fa9f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.743+0000 7f00e17fa640 0 client.27565 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.743+0000 7f0102e46740 1 -- 172.21.15.7:0/3040334983 shutdown_connections 2024-06-10T01:16:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.743+0000 7f0102e46740 1 -- 172.21.15.7:0/3040334983 wait complete. 2024-06-10T01:16:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.743+0000 7f0102e46740 20 client.27565 trim_cache size 0 max 16384 2024-06-10T01:16:51.749 INFO:tasks.workunit.client.0.smithi007.stdout: 1 810560 7.62 MB/sec execute 452 sec latency 18.336 ms 2024-06-10T01:16:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.761+0000 7f0102e46740 1 Processor -- start 2024-06-10T01:16:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.761+0000 7f0102e46740 1 -- start start 2024-06-10T01:16:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.761+0000 7f0102e46740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55761faad820 0x55761fa9b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.761+0000 7f0102e46740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55761faae210 0x55761fabefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.761+0000 7f0102e46740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55761fab3380 0x55761fab7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.761+0000 7f0102e46740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f00f81aa9b0 con 0x55761faad820 2024-06-10T01:16:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.761+0000 7f0102e46740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f00f81a8250 con 0x55761fab3380 2024-06-10T01:16:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.761+0000 7f0102e46740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f00f81a60a0 con 0x55761faae210 2024-06-10T01:16:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.761+0000 7f00fded7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55761faad820 0x55761fa9b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.761+0000 7f00fded7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55761faad820 0x55761fa9b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40162/0 (socket says 172.21.15.7:40162) 2024-06-10T01:16:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.761+0000 7f00fded7640 1 -- 172.21.15.7:0/164209564 learned_addr learned my addr 172.21.15.7:0/164209564 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.761+0000 7f00fe6d8640 1 --2- 172.21.15.7:0/164209564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55761fab3380 0x55761fab7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.761+0000 7f00fd6d6640 1 --2- 172.21.15.7:0/164209564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55761faae210 0x55761fabefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.761+0000 7f00fe6d8640 1 -- 172.21.15.7:0/164209564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55761faae210 msgr2=0x55761fabefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.761+0000 7f00fe6d8640 1 --2- 172.21.15.7:0/164209564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55761faae210 0x55761fabefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.761+0000 7f00fe6d8640 1 -- 172.21.15.7:0/164209564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55761faad820 msgr2=0x55761fa9b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.761+0000 7f00fe6d8640 1 --2- 172.21.15.7:0/164209564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55761faad820 0x55761fa9b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.761+0000 7f00fe6d8640 1 -- 172.21.15.7:0/164209564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55761faaea70 con 0x55761fab3380 2024-06-10T01:16:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.761+0000 7f00fd6d6640 1 --2- 172.21.15.7:0/164209564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55761faae210 0x55761fabefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:16:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.761+0000 7f00fded7640 1 --2- 172.21.15.7:0/164209564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55761faad820 0x55761fa9b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.762+0000 7f00fe6d8640 1 --2- 172.21.15.7:0/164209564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55761fab3380 0x55761fab7090 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f00f00cd0d0 tx=0x7f00f001c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=490eb8fda6ec8350 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.762+0000 7f00e37fe640 1 -- 172.21.15.7:0/164209564 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f00a0210 con 0x55761fab3380 2024-06-10T01:16:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.762+0000 7f00e37fe640 1 -- 172.21.15.7:0/164209564 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00f00a4ad0 con 0x55761fab3380 2024-06-10T01:16:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.762+0000 7f00e37fe640 1 -- 172.21.15.7:0/164209564 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f0018cd0 con 0x55761fab3380 2024-06-10T01:16:51.760 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db / --timeout 90 --expiration 135.8 --await-for 45.0' 2024-06-10T01:16:51.760 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db / --timeout 90 --expiration 135.8 --await-for 45.0 2024-06-10T01:16:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.763+0000 7f0102e46740 1 -- 172.21.15.7:0/164209564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55761fab3380 msgr2=0x55761fab7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.763+0000 7f0102e46740 1 --2- 172.21.15.7:0/164209564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55761fab3380 0x55761fab7090 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f00f00cd0d0 tx=0x7f00f001c000 comp rx=0 tx=0).stop 2024-06-10T01:16:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.763+0000 7f0102e46740 1 -- 172.21.15.7:0/164209564 shutdown_connections 2024-06-10T01:16:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.763+0000 7f0102e46740 1 --2- 172.21.15.7:0/164209564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55761fab3380 0x55761fab7090 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.763+0000 7f0102e46740 1 --2- 172.21.15.7:0/164209564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55761faae210 0x55761fabefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.763+0000 7f0102e46740 1 --2- 172.21.15.7:0/164209564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55761faad820 0x55761fa9b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.763+0000 7f0102e46740 1 -- 172.21.15.7:0/164209564 >> 172.21.15.7:0/164209564 conn(0x55761faa0680 msgr2=0x55761faa3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.763+0000 7f0102e46740 1 -- 172.21.15.7:0/164209564 shutdown_connections 2024-06-10T01:16:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.763+0000 7f0102e46740 1 -- 172.21.15.7:0/164209564 wait complete. 2024-06-10T01:16:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.763+0000 7f0102e46740 1 -- start start 2024-06-10T01:16:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.763+0000 7f0102e46740 1 Processor -- start 2024-06-10T01:16:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.764+0000 7f0102e46740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55761faad820 0x55761fabbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.764+0000 7f0102e46740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55761faae210 0x55761fabc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.764+0000 7f0102e46740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55761fab3380 0x55761fb2c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.764+0000 7f0102e46740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f00f81a6710 con 0x55761fab3380 2024-06-10T01:16:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.764+0000 7f0102e46740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f00f81a7b80 con 0x55761faad820 2024-06-10T01:16:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.764+0000 7f0102e46740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f00f81a6de0 con 0x55761faae210 2024-06-10T01:16:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.764+0000 7f0102e46740 10 client.0 _ll_register_callbacks cb 0x7f00f4040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:16:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.764+0000 7f00fe6d8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55761fab3380 0x55761fb2c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.764+0000 7f00fded7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55761faad820 0x55761fabbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.764+0000 7f00fe6d8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55761fab3380 0x55761fb2c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40170/0 (socket says 172.21.15.7:40170) 2024-06-10T01:16:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.764+0000 7f00fe6d8640 1 -- 172.21.15.7:0/3775535268 learned_addr learned my addr 172.21.15.7:0/3775535268 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.764+0000 7f00fd6d6640 1 --2- 172.21.15.7:0/3775535268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55761faae210 0x55761fabc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.764+0000 7f00fded7640 1 -- 172.21.15.7:0/3775535268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55761faae210 msgr2=0x55761fabc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.764+0000 7f00fded7640 1 --2- 172.21.15.7:0/3775535268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55761faae210 0x55761fabc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.764+0000 7f00fded7640 1 -- 172.21.15.7:0/3775535268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55761fab3380 msgr2=0x55761fb2c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.764+0000 7f00fded7640 1 --2- 172.21.15.7:0/3775535268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55761fab3380 0x55761fb2c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.764+0000 7f00fded7640 1 -- 172.21.15.7:0/3775535268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55761fabf230 con 0x55761faad820 2024-06-10T01:16:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.764+0000 7f00fe6d8640 1 --2- 172.21.15.7:0/3775535268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55761fab3380 0x55761fb2c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.765+0000 7f00fded7640 1 --2- 172.21.15.7:0/3775535268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55761faad820 0x55761fabbc20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f00f403f040 tx=0x7f00f4009590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b201f912f7097c46 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.765+0000 7f00e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:16:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.765+0000 7f00e17fa640 1 -- 172.21.15.7:0/3775535268 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f404e040 con 0x55761faad820 2024-06-10T01:16:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.765+0000 7f00e17fa640 1 -- 172.21.15.7:0/3775535268 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f00f4031a50 con 0x55761faad820 2024-06-10T01:16:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.765+0000 7f0102e46740 1 -- 172.21.15.7:0/3775535268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00f00a4ad0 con 0x55761faad820 2024-06-10T01:16:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.765+0000 7f00e17fa640 1 -- 172.21.15.7:0/3775535268 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f00f4035040 con 0x55761faad820 2024-06-10T01:16:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.765+0000 7f0102e46740 10 client.27542 fetch_fsmap learned FSMap version 39 2024-06-10T01:16:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.765+0000 7f0102e46740 1 -- 172.21.15.7:0/3775535268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55761fb2d6a0 con 0x55761faad820 2024-06-10T01:16:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.765+0000 7f00e17fa640 1 -- 172.21.15.7:0/3775535268 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00f00a4ad0 con 0x55761faad820 2024-06-10T01:16:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.766+0000 7f00e17fa640 1 -- 172.21.15.7:0/3775535268 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7f00f4003710 con 0x55761faad820 2024-06-10T01:16:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.766+0000 7f0102e46740 10 client.27542 fetch_fsmap finished waiting for FSMap version 39 2024-06-10T01:16:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.766+0000 7f0102e46740 10 client.27542 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:16:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.766+0000 7f0102e46740 20 client.27542 populate_metadata read hostname 'smithi007' 2024-06-10T01:16:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.766+0000 7f0102e46740 1 --2- 172.21.15.7:0/3775535268 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55761fb2de00 0x55761fb30220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.766+0000 7f0102e46740 4 client.27542 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:16:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.766+0000 7f0102e46740 1 -- 172.21.15.7:0/3775535268 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55761fa15c20 con 0x55761fb2de00 2024-06-10T01:16:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.766+0000 7f00fd6d6640 1 --2- 172.21.15.7:0/3775535268 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55761fb2de00 0x55761fb30220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.766+0000 7f00fd6d6640 1 --2- 172.21.15.7:0/3775535268 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55761fb2de00 0x55761fb30220 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=447e1900533f86da server_cookie=3a945cf25a73b4fc in_seq=0 out_seq=0 2024-06-10T01:16:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.766+0000 7f00e17fa640 10 client.27542 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.848+0000 7f00e17fa640 1 -- 172.21.15.7:0/3775535268 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55761fa15c20 con 0x55761fb2de00 2024-06-10T01:16:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.848+0000 7f00e17fa640 10 client.27542 handle_command_reply: tid=0 2024-06-10T01:16:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.848+0000 7f0102e46740 1 client.27542 shutdown 2024-06-10T01:16:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.848+0000 7f0102e46740 1 -- 172.21.15.7:0/3775535268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55761faad820 msgr2=0x55761fabbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.848+0000 7f0102e46740 1 --2- 172.21.15.7:0/3775535268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55761faad820 0x55761fabbc20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f00f403f040 tx=0x7f00f4009590 comp rx=0 tx=0).stop 2024-06-10T01:16:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.848+0000 7f0102e46740 1 -- 172.21.15.7:0/3775535268 shutdown_connections 2024-06-10T01:16:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.848+0000 7f0102e46740 1 --2- 172.21.15.7:0/3775535268 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55761fb2de00 0x55761fb30220 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.848+0000 7f0102e46740 1 --2- 172.21.15.7:0/3775535268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55761fab3380 0x55761fb2c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.848+0000 7f0102e46740 1 --2- 172.21.15.7:0/3775535268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55761faae210 0x55761fabc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.848+0000 7f0102e46740 1 --2- 172.21.15.7:0/3775535268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55761faad820 0x55761fabbc20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.848+0000 7f0102e46740 1 -- 172.21.15.7:0/3775535268 >> 172.21.15.7:0/3775535268 conn(0x55761faa0680 msgr2=0x55761fa9ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.848+0000 7f00e17fa640 0 client.27542 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.848+0000 7f0102e46740 1 -- 172.21.15.7:0/3775535268 shutdown_connections 2024-06-10T01:16:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.849+0000 7f0102e46740 1 -- 172.21.15.7:0/3775535268 wait complete. 2024-06-10T01:16:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.849+0000 7f0102e46740 20 client.27542 trim_cache size 0 max 16384 2024-06-10T01:16:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:16:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:16:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:16:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.849+0000 7f010013b640 1 -- 172.21.15.7:0/1375298439 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f00e40a2c00 msgr2=0x7f00e40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.849+0000 7f010013b640 1 --2- 172.21.15.7:0/1375298439 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f00e40a2c00 0x7f00e40a50c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f00f8196e10 tx=0x7f00e800bba0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.849+0000 7f010013b640 1 -- 172.21.15.7:0/1375298439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00f81024b0 msgr2=0x7f00f819ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.849+0000 7f010013b640 1 --2- 172.21.15.7:0/1375298439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00f81024b0 0x7f00f819ceb0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f00f0009fd0 tx=0x7f00f000ef10 comp rx=0 tx=0).stop 2024-06-10T01:16:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.850+0000 7f010013b640 1 -- 172.21.15.7:0/1375298439 shutdown_connections 2024-06-10T01:16:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.850+0000 7f010013b640 1 --2- 172.21.15.7:0/1375298439 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f00e40a2c00 0x7f00e40a50c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.850+0000 7f010013b640 1 --2- 172.21.15.7:0/1375298439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00f81024b0 0x7f00f819ceb0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.850+0000 7f010013b640 1 --2- 172.21.15.7:0/1375298439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00f8101af0 0x7f00f8195e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.850+0000 7f010013b640 1 --2- 172.21.15.7:0/1375298439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00f8100900 0x7f00f8195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.850+0000 7f010013b640 1 -- 172.21.15.7:0/1375298439 >> 172.21.15.7:0/1375298439 conn(0x7f00f80fbe70 msgr2=0x7f00f8104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.850+0000 7f010013b640 1 -- 172.21.15.7:0/1375298439 shutdown_connections 2024-06-10T01:16:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:16:51.850+0000 7f010013b640 1 -- 172.21.15.7:0/1375298439 wait complete. 2024-06-10T01:16:52.086 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:16:52.749 INFO:tasks.workunit.client.0.smithi007.stdout: 1 813053 7.64 MB/sec execute 453 sec latency 66.525 ms 2024-06-10T01:16:53.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:52 smithi175 ceph-mon[36636]: pgmap v758: 129 pgs: 129 active+clean; 139 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 2.5 MiB/s wr, 281 op/s 2024-06-10T01:16:53.251 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:52 smithi007 ceph-mon[32398]: pgmap v758: 129 pgs: 129 active+clean; 139 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 2.5 MiB/s wr, 281 op/s 2024-06-10T01:16:53.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:52 smithi078 ceph-mon[36683]: pgmap v758: 129 pgs: 129 active+clean; 139 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 2.5 MiB/s wr, 281 op/s 2024-06-10T01:16:53.749 INFO:tasks.workunit.client.0.smithi007.stdout: 1 814454 7.64 MB/sec execute 454 sec latency 56.364 ms 2024-06-10T01:16:53.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.910+0000 7f0d970cd700 1 -- 172.21.15.7:0/906471551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d90103150 msgr2=0x7f0d90107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:53.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.910+0000 7f0d970cd700 1 --2- 172.21.15.7:0/906471551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d90103150 0x7f0d90107140 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f0d780099c0 tx=0x7f0d78009cd0 comp rx=0 tx=0).stop 2024-06-10T01:16:53.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.910+0000 7f0d970cd700 1 -- 172.21.15.7:0/906471551 shutdown_connections 2024-06-10T01:16:53.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.910+0000 7f0d970cd700 1 --2- 172.21.15.7:0/906471551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d90107870 0x7f0d9010f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:53.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.910+0000 7f0d970cd700 1 --2- 172.21.15.7:0/906471551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d901027a0 0x7f0d90102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:53.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.910+0000 7f0d970cd700 1 --2- 172.21.15.7:0/906471551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d90103150 0x7f0d90107140 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:53.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.910+0000 7f0d970cd700 1 -- 172.21.15.7:0/906471551 >> 172.21.15.7:0/906471551 conn(0x7f0d90079570 msgr2=0x7f0d90079980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:53.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.910+0000 7f0d970cd700 1 -- 172.21.15.7:0/906471551 shutdown_connections 2024-06-10T01:16:53.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.910+0000 7f0d970cd700 1 -- 172.21.15.7:0/906471551 wait complete. 2024-06-10T01:16:53.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.911+0000 7f0d970cd700 1 Processor -- start 2024-06-10T01:16:53.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.911+0000 7f0d970cd700 1 -- start start 2024-06-10T01:16:53.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.912+0000 7f0d970cd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d901027a0 0x7f0d9019e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:53.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.912+0000 7f0d970cd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d90103150 0x7f0d9019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:53.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.912+0000 7f0d970cd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d90107870 0x7f0d901988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:53.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.912+0000 7f0d970cd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0d901127a0 con 0x7f0d90103150 2024-06-10T01:16:53.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.912+0000 7f0d970cd700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0d90112620 con 0x7f0d90107870 2024-06-10T01:16:53.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.912+0000 7f0d970cd700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0d90112920 con 0x7f0d901027a0 2024-06-10T01:16:53.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.912+0000 7f0d8ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d90103150 0x7f0d9019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:53.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.912+0000 7f0d8ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d90103150 0x7f0d9019ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33168/0 (socket says 172.21.15.7:33168) 2024-06-10T01:16:53.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.912+0000 7f0d8ffff700 1 -- 172.21.15.7:0/1236864436 learned_addr learned my addr 172.21.15.7:0/1236864436 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:53.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.912+0000 7f0d9566a700 1 --2- 172.21.15.7:0/1236864436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d90107870 0x7f0d901988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:53.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.912+0000 7f0d94e69700 1 --2- 172.21.15.7:0/1236864436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d901027a0 0x7f0d9019e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:53.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.912+0000 7f0d8ffff700 1 -- 172.21.15.7:0/1236864436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d901027a0 msgr2=0x7f0d9019e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:53.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.912+0000 7f0d8ffff700 1 --2- 172.21.15.7:0/1236864436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d901027a0 0x7f0d9019e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:53.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.912+0000 7f0d8ffff700 1 -- 172.21.15.7:0/1236864436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d90107870 msgr2=0x7f0d901988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:53.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.912+0000 7f0d8ffff700 1 --2- 172.21.15.7:0/1236864436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d90107870 0x7f0d901988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:53.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.912+0000 7f0d8ffff700 1 -- 172.21.15.7:0/1236864436 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d90199000 con 0x7f0d90103150 2024-06-10T01:16:53.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.912+0000 7f0d9566a700 1 --2- 172.21.15.7:0/1236864436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d90107870 0x7f0d901988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:53.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.912+0000 7f0d8ffff700 1 --2- 172.21.15.7:0/1236864436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d90103150 0x7f0d9019ec20 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f0d78006010 tx=0x7f0d7800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:53.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.913+0000 7f0d8dffb700 1 -- 172.21.15.7:0/1236864436 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d78003680 con 0x7f0d90103150 2024-06-10T01:16:53.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.913+0000 7f0d970cd700 1 -- 172.21.15.7:0/1236864436 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d90199290 con 0x7f0d90103150 2024-06-10T01:16:53.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.913+0000 7f0d8dffb700 1 -- 172.21.15.7:0/1236864436 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d780050d0 con 0x7f0d90103150 2024-06-10T01:16:53.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.913+0000 7f0d8dffb700 1 -- 172.21.15.7:0/1236864436 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d78003db0 con 0x7f0d90103150 2024-06-10T01:16:53.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.913+0000 7f0d970cd700 1 -- 172.21.15.7:0/1236864436 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d901a55b0 con 0x7f0d90103150 2024-06-10T01:16:53.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.915+0000 7f0d8dffb700 1 -- 172.21.15.7:0/1236864436 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0d78005280 con 0x7f0d90103150 2024-06-10T01:16:53.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.916+0000 7f0d8dffb700 1 --2- 172.21.15.7:0/1236864436 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0d7c0a2e60 0x7f0d7c0a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:53.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.916+0000 7f0d8dffb700 1 -- 172.21.15.7:0/1236864436 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f0d780d0050 con 0x7f0d90103150 2024-06-10T01:16:53.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.916+0000 7f0d94e69700 1 --2- 172.21.15.7:0/1236864436 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0d7c0a2e60 0x7f0d7c0a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:53.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.916+0000 7f0d94e69700 1 --2- 172.21.15.7:0/1236864436 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0d7c0a2e60 0x7f0d7c0a5320 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0d901999e0 tx=0x7f0d8000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:53.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.917+0000 7f0da1950b80 1 Processor -- start 2024-06-10T01:16:53.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.917+0000 7f0da1950b80 1 -- start start 2024-06-10T01:16:53.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.917+0000 7f0da1950b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a292e00cd0 0x55a292e010b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:53.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.917+0000 7f0da1950b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a292e015f0 0x55a292dfaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:53.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.917+0000 7f0da1950b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a292dfb460 0x55a292dfb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:53.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.917+0000 7f0da1950b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a292d4c9a0 con 0x55a292e015f0 2024-06-10T01:16:53.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.917+0000 7f0da1950b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a292cf48b0 con 0x55a292dfb460 2024-06-10T01:16:53.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.917+0000 7f0da1950b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a292d48960 con 0x55a292e00cd0 2024-06-10T01:16:53.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.917+0000 7f0d8ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a292e015f0 0x55a292dfaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:53.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.917+0000 7f0d94e69700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a292e00cd0 0x55a292e010b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:53.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.917+0000 7f0d94e69700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a292e00cd0 0x55a292e010b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:46696/0 (socket says 172.21.15.7:46696) 2024-06-10T01:16:53.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.917+0000 7f0d94e69700 1 -- 172.21.15.7:0/2796300897 learned_addr learned my addr 172.21.15.7:0/2796300897 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:53.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.918+0000 7f0d9566a700 1 --2- 172.21.15.7:0/2796300897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a292dfb460 0x55a292dfb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:53.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.918+0000 7f0d9566a700 1 -- 172.21.15.7:0/2796300897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a292e00cd0 msgr2=0x55a292e010b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:53.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.918+0000 7f0d9566a700 1 --2- 172.21.15.7:0/2796300897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a292e00cd0 0x55a292e010b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:53.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.918+0000 7f0d9566a700 1 -- 172.21.15.7:0/2796300897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a292e015f0 msgr2=0x55a292dfaf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:53.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.918+0000 7f0d9566a700 1 --2- 172.21.15.7:0/2796300897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a292e015f0 0x55a292dfaf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:53.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.918+0000 7f0d9566a700 1 -- 172.21.15.7:0/2796300897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a292ceb0e0 con 0x55a292dfb460 2024-06-10T01:16:53.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.918+0000 7f0d9566a700 1 --2- 172.21.15.7:0/2796300897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a292dfb460 0x55a292dfb8a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0d88011440 tx=0x7f0d88011750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=810ca99748739a90 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:53.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.918+0000 7f0d86ffd700 1 -- 172.21.15.7:0/2796300897 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d88004310 con 0x55a292dfb460 2024-06-10T01:16:53.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.918+0000 7f0d86ffd700 1 -- 172.21.15.7:0/2796300897 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d880044b0 con 0x55a292dfb460 2024-06-10T01:16:53.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.918+0000 7f0d86ffd700 1 -- 172.21.15.7:0/2796300897 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d88003680 con 0x55a292dfb460 2024-06-10T01:16:53.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.919+0000 7f0da1950b80 1 -- 172.21.15.7:0/2796300897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a292dfb460 msgr2=0x55a292dfb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:53.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.919+0000 7f0da1950b80 1 --2- 172.21.15.7:0/2796300897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a292dfb460 0x55a292dfb8a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0d88011440 tx=0x7f0d88011750 comp rx=0 tx=0).stop 2024-06-10T01:16:53.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.919+0000 7f0da1950b80 1 -- 172.21.15.7:0/2796300897 shutdown_connections 2024-06-10T01:16:53.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.919+0000 7f0da1950b80 1 --2- 172.21.15.7:0/2796300897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a292dfb460 0x55a292dfb8a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:53.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.919+0000 7f0da1950b80 1 --2- 172.21.15.7:0/2796300897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a292e00cd0 0x55a292e010b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:53.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.919+0000 7f0da1950b80 1 --2- 172.21.15.7:0/2796300897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a292e015f0 0x55a292dfaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:53.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.919+0000 7f0da1950b80 1 -- 172.21.15.7:0/2796300897 >> 172.21.15.7:0/2796300897 conn(0x55a292d25e50 msgr2=0x55a292df61b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:53.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.919+0000 7f0da1950b80 1 -- 172.21.15.7:0/2796300897 shutdown_connections 2024-06-10T01:16:53.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.919+0000 7f0da1950b80 1 -- 172.21.15.7:0/2796300897 wait complete. 2024-06-10T01:16:53.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.920+0000 7f0da1950b80 1 -- start start 2024-06-10T01:16:53.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.920+0000 7f0da1950b80 1 Processor -- start 2024-06-10T01:16:53.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.920+0000 7f0da1950b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a292dfb460 0x55a292e07400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:53.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.920+0000 7f0da1950b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a292e00cd0 0x55a292e04750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:53.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.920+0000 7f0da1950b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a292e015f0 0x55a292e04c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:53.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.920+0000 7f0da1950b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a292e08060 con 0x55a292dfb460 2024-06-10T01:16:53.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.920+0000 7f0da1950b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a292cecd80 con 0x55a292e015f0 2024-06-10T01:16:53.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.920+0000 7f0da1950b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a292e081e0 con 0x55a292e00cd0 2024-06-10T01:16:53.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.920+0000 7f0da1950b80 10 client.0 _ll_register_callbacks cb 0x55a292ce6ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:16:53.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.920+0000 7f0d94e69700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a292dfb460 0x55a292e07400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:53.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.920+0000 7f0d9566a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a292e015f0 0x55a292e04c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:53.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.920+0000 7f0d94e69700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a292dfb460 0x55a292e07400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33188/0 (socket says 172.21.15.7:33188) 2024-06-10T01:16:53.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.920+0000 7f0d94e69700 1 -- 172.21.15.7:0/2726357275 learned_addr learned my addr 172.21.15.7:0/2726357275 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:53.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.920+0000 7f0d8ffff700 1 --2- 172.21.15.7:0/2726357275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a292e00cd0 0x55a292e04750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:53.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.921+0000 7f0d94e69700 1 -- 172.21.15.7:0/2726357275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a292e00cd0 msgr2=0x55a292e04750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:53.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.921+0000 7f0d94e69700 1 --2- 172.21.15.7:0/2726357275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a292e00cd0 0x55a292e04750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:53.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.921+0000 7f0d94e69700 1 -- 172.21.15.7:0/2726357275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a292e015f0 msgr2=0x55a292e04c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:53.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.921+0000 7f0d94e69700 1 --2- 172.21.15.7:0/2726357275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a292e015f0 0x55a292e04c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:53.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.921+0000 7f0d94e69700 1 -- 172.21.15.7:0/2726357275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a292c9ba60 con 0x55a292dfb460 2024-06-10T01:16:53.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.921+0000 7f0d94e69700 1 --2- 172.21.15.7:0/2726357275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a292dfb460 0x55a292e07400 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000fc60 tx=0x7f0d80004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b2e23bbb7a6f34 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:53.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.921+0000 7f0d6ffff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:16:53.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.921+0000 7f0d6ffff700 1 -- 172.21.15.7:0/2726357275 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d80027070 con 0x55a292dfb460 2024-06-10T01:16:53.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.921+0000 7f0da1950b80 1 -- 172.21.15.7:0/2726357275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d880044b0 con 0x55a292dfb460 2024-06-10T01:16:53.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.921+0000 7f0d6ffff700 1 -- 172.21.15.7:0/2726357275 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f0d80004db0 con 0x55a292dfb460 2024-06-10T01:16:53.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.922+0000 7f0d6ffff700 1 -- 172.21.15.7:0/2726357275 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0d80032050 con 0x55a292dfb460 2024-06-10T01:16:53.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.922+0000 7f0da1950b80 10 client.18207 fetch_fsmap learned FSMap version 39 2024-06-10T01:16:53.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.922+0000 7f0da1950b80 1 -- 172.21.15.7:0/2726357275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a292e12a40 con 0x55a292dfb460 2024-06-10T01:16:53.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.922+0000 7f0d6ffff700 1 -- 172.21.15.7:0/2726357275 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d880044b0 con 0x55a292dfb460 2024-06-10T01:16:53.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.922+0000 7f0d6ffff700 1 -- 172.21.15.7:0/2726357275 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7f0d80020050 con 0x55a292dfb460 2024-06-10T01:16:53.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.922+0000 7f0da1950b80 10 client.18207 fetch_fsmap finished waiting for FSMap version 39 2024-06-10T01:16:53.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.922+0000 7f0da1950b80 10 client.18207 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:16:53.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.922+0000 7f0da1950b80 20 client.18207 populate_metadata read hostname 'smithi007' 2024-06-10T01:16:53.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.922+0000 7f0da1950b80 1 --2- 172.21.15.7:0/2726357275 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a292e14fd0 0x55a292e173e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:53.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.922+0000 7f0da1950b80 4 client.18207 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:16:53.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.922+0000 7f0da1950b80 1 -- 172.21.15.7:0/2726357275 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a292d59ea0 con 0x55a292e14fd0 2024-06-10T01:16:53.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.922+0000 7f0d8ffff700 1 --2- 172.21.15.7:0/2726357275 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a292e14fd0 0x55a292e173e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:53.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.923+0000 7f0d8ffff700 1 --2- 172.21.15.7:0/2726357275 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a292e14fd0 0x55a292e173e0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=400dcf3e150bea39 server_cookie=659a564cd8c7c813 in_seq=0 out_seq=0 2024-06-10T01:16:53.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.923+0000 7f0d6ffff700 10 client.18207 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:53.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.923+0000 7f0d6ffff700 1 -- 172.21.15.7:0/2726357275 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a292d59ea0 con 0x55a292e14fd0 2024-06-10T01:16:53.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.923+0000 7f0d6ffff700 10 client.18207 handle_command_reply: tid=0 2024-06-10T01:16:53.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.924+0000 7f0da1950b80 1 client.18207 shutdown 2024-06-10T01:16:53.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.924+0000 7f0da1950b80 1 -- 172.21.15.7:0/2726357275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a292dfb460 msgr2=0x55a292e07400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:53.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.924+0000 7f0da1950b80 1 --2- 172.21.15.7:0/2726357275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a292dfb460 0x55a292e07400 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000fc60 tx=0x7f0d80004ad0 comp rx=0 tx=0).stop 2024-06-10T01:16:53.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.924+0000 7f0da1950b80 1 -- 172.21.15.7:0/2726357275 shutdown_connections 2024-06-10T01:16:53.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.924+0000 7f0da1950b80 1 --2- 172.21.15.7:0/2726357275 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a292e14fd0 0x55a292e173e0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:53.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.924+0000 7f0d6ffff700 0 client.18207 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:53.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.924+0000 7f0da1950b80 1 --2- 172.21.15.7:0/2726357275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a292e015f0 0x55a292e04c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:53.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.924+0000 7f0da1950b80 1 --2- 172.21.15.7:0/2726357275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a292dfb460 0x55a292e07400 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:53.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.924+0000 7f0da1950b80 1 --2- 172.21.15.7:0/2726357275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a292e00cd0 0x55a292e04750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:53.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.924+0000 7f0da1950b80 1 -- 172.21.15.7:0/2726357275 >> 172.21.15.7:0/2726357275 conn(0x55a292d25e50 msgr2=0x55a292df5da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:53.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.924+0000 7f0da1950b80 1 -- 172.21.15.7:0/2726357275 shutdown_connections 2024-06-10T01:16:53.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.924+0000 7f0da1950b80 1 -- 172.21.15.7:0/2726357275 wait complete. 2024-06-10T01:16:53.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.924+0000 7f0da1950b80 20 client.18207 trim_cache size 0 max 16384 2024-06-10T01:16:53.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.947+0000 7f0da1950b80 1 Processor -- start 2024-06-10T01:16:53.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.947+0000 7f0da1950b80 1 -- start start 2024-06-10T01:16:53.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.948+0000 7f0da1950b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a292dfb460 0x55a292dfaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:53.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.948+0000 7f0da1950b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a292e00cd0 0x55a292e03750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:53.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.948+0000 7f0da1950b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a292e015f0 0x55a292e1ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:53.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.948+0000 7f0da1950b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0d901b0860 con 0x55a292dfb460 2024-06-10T01:16:53.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.948+0000 7f0da1950b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0d901ae0c0 con 0x55a292e015f0 2024-06-10T01:16:53.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.948+0000 7f0da1950b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0d901abf10 con 0x55a292e00cd0 2024-06-10T01:16:53.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.948+0000 7f0d94e69700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a292dfb460 0x55a292dfaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:53.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.948+0000 7f0d94e69700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a292dfb460 0x55a292dfaf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33194/0 (socket says 172.21.15.7:33194) 2024-06-10T01:16:53.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.948+0000 7f0d94e69700 1 -- 172.21.15.7:0/3616227529 learned_addr learned my addr 172.21.15.7:0/3616227529 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:53.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.948+0000 7f0d9566a700 1 --2- 172.21.15.7:0/3616227529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a292e015f0 0x55a292e1ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:53.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.948+0000 7f0d8ffff700 1 --2- 172.21.15.7:0/3616227529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a292e00cd0 0x55a292e03750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:53.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.948+0000 7f0d94e69700 1 -- 172.21.15.7:0/3616227529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a292e00cd0 msgr2=0x55a292e03750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:53.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.948+0000 7f0d94e69700 1 --2- 172.21.15.7:0/3616227529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a292e00cd0 0x55a292e03750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:53.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.948+0000 7f0d94e69700 1 -- 172.21.15.7:0/3616227529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a292e015f0 msgr2=0x55a292e1ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:53.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.948+0000 7f0d94e69700 1 --2- 172.21.15.7:0/3616227529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a292e015f0 0x55a292e1ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:53.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.948+0000 7f0d94e69700 1 -- 172.21.15.7:0/3616227529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a292e02cc0 con 0x55a292dfb460 2024-06-10T01:16:53.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.948+0000 7f0d94e69700 1 --2- 172.21.15.7:0/3616227529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a292dfb460 0x55a292dfaf20 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f0d80023e20 tx=0x7f0d80009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ee88136698bc11 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:53.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.949+0000 7f0d867fc700 1 -- 172.21.15.7:0/3616227529 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8003b070 con 0x55a292dfb460 2024-06-10T01:16:53.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.949+0000 7f0d867fc700 1 -- 172.21.15.7:0/3616227529 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d8001b030 con 0x55a292dfb460 2024-06-10T01:16:53.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.949+0000 7f0d867fc700 1 -- 172.21.15.7:0/3616227529 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d80027070 con 0x55a292dfb460 2024-06-10T01:16:53.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.949+0000 7f0da1950b80 1 -- 172.21.15.7:0/3616227529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a292dfb460 msgr2=0x55a292dfaf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:53.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.949+0000 7f0da1950b80 1 --2- 172.21.15.7:0/3616227529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a292dfb460 0x55a292dfaf20 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f0d80023e20 tx=0x7f0d80009920 comp rx=0 tx=0).stop 2024-06-10T01:16:53.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.949+0000 7f0da1950b80 1 -- 172.21.15.7:0/3616227529 shutdown_connections 2024-06-10T01:16:53.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.949+0000 7f0da1950b80 1 --2- 172.21.15.7:0/3616227529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a292e015f0 0x55a292e1ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:53.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.949+0000 7f0da1950b80 1 --2- 172.21.15.7:0/3616227529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a292dfb460 0x55a292dfaf20 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:53.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.949+0000 7f0da1950b80 1 --2- 172.21.15.7:0/3616227529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a292e00cd0 0x55a292e03750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:53.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.949+0000 7f0da1950b80 1 -- 172.21.15.7:0/3616227529 >> 172.21.15.7:0/3616227529 conn(0x55a292d25e50 msgr2=0x55a292e03040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:53.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.950+0000 7f0da1950b80 1 -- 172.21.15.7:0/3616227529 shutdown_connections 2024-06-10T01:16:53.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.950+0000 7f0da1950b80 1 -- 172.21.15.7:0/3616227529 wait complete. 2024-06-10T01:16:53.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.950+0000 7f0da1950b80 1 -- start start 2024-06-10T01:16:53.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.950+0000 7f0da1950b80 1 Processor -- start 2024-06-10T01:16:53.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.950+0000 7f0da1950b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a292dfb460 0x55a292e0a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:53.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.950+0000 7f0da1950b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a292e00cd0 0x55a292e0a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:53.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.950+0000 7f0da1950b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a292e015f0 0x55a292e0ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:53.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.950+0000 7f0da1950b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0d901ac580 con 0x55a292dfb460 2024-06-10T01:16:53.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.950+0000 7f0da1950b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0d901ad9f0 con 0x55a292e00cd0 2024-06-10T01:16:53.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.950+0000 7f0da1950b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0d901acc50 con 0x55a292e015f0 2024-06-10T01:16:53.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.950+0000 7f0da1950b80 10 client.0 _ll_register_callbacks cb 0x55a292ce6ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:16:53.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.951+0000 7f0d94e69700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a292dfb460 0x55a292e0a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:53.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.951+0000 7f0d8ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a292e00cd0 0x55a292e0a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:53.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.951+0000 7f0d94e69700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a292dfb460 0x55a292e0a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33206/0 (socket says 172.21.15.7:33206) 2024-06-10T01:16:53.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.951+0000 7f0d94e69700 1 -- 172.21.15.7:0/291523675 learned_addr learned my addr 172.21.15.7:0/291523675 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:16:53.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.951+0000 7f0d9566a700 1 --2- 172.21.15.7:0/291523675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a292e015f0 0x55a292e0ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:53.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.951+0000 7f0d94e69700 1 -- 172.21.15.7:0/291523675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a292e015f0 msgr2=0x55a292e0ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:53.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.951+0000 7f0d94e69700 1 --2- 172.21.15.7:0/291523675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a292e015f0 0x55a292e0ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:53.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.951+0000 7f0d94e69700 1 -- 172.21.15.7:0/291523675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a292e00cd0 msgr2=0x55a292e0a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:16:53.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.951+0000 7f0d94e69700 1 --2- 172.21.15.7:0/291523675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a292e00cd0 0x55a292e0a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:53.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.951+0000 7f0d94e69700 1 -- 172.21.15.7:0/291523675 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a292e7fc90 con 0x55a292dfb460 2024-06-10T01:16:53.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.951+0000 7f0d8ffff700 1 --2- 172.21.15.7:0/291523675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a292e00cd0 0x55a292e0a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:16:53.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.951+0000 7f0d94e69700 1 --2- 172.21.15.7:0/291523675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a292dfb460 0x55a292e0a1d0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000f3e0 tx=0x7f0d80023c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa9d2b9edef7d44 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:16:53.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.951+0000 7f0d6ffff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:16:53.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.951+0000 7f0d6ffff700 1 -- 172.21.15.7:0/291523675 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8003b070 con 0x55a292dfb460 2024-06-10T01:16:53.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.951+0000 7f0d6ffff700 1 -- 172.21.15.7:0/291523675 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f0d80017ab0 con 0x55a292dfb460 2024-06-10T01:16:53.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.951+0000 7f0da1950b80 1 -- 172.21.15.7:0/291523675 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d8001b030 con 0x55a292dfb460 2024-06-10T01:16:53.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.952+0000 7f0d6ffff700 1 -- 172.21.15.7:0/291523675 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0d8004b050 con 0x55a292dfb460 2024-06-10T01:16:53.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.952+0000 7f0da1950b80 10 client.18213 fetch_fsmap learned FSMap version 39 2024-06-10T01:16:53.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.952+0000 7f0da1950b80 1 -- 172.21.15.7:0/291523675 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a292e80ef0 con 0x55a292dfb460 2024-06-10T01:16:53.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.952+0000 7f0d6ffff700 1 -- 172.21.15.7:0/291523675 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d8001b030 con 0x55a292dfb460 2024-06-10T01:16:53.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.952+0000 7f0d6ffff700 1 -- 172.21.15.7:0/291523675 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7f0d80020040 con 0x55a292dfb460 2024-06-10T01:16:53.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.952+0000 7f0da1950b80 10 client.18213 fetch_fsmap finished waiting for FSMap version 39 2024-06-10T01:16:53.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.952+0000 7f0da1950b80 10 client.18213 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:16:53.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.952+0000 7f0da1950b80 20 client.18213 populate_metadata read hostname 'smithi007' 2024-06-10T01:16:53.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.952+0000 7f0da1950b80 1 --2- 172.21.15.7:0/291523675 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a292e14fd0 0x55a292e83590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:16:53.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.952+0000 7f0da1950b80 4 client.18213 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 135.8, "await_for": 45.0}] 2024-06-10T01:16:53.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.952+0000 7f0da1950b80 1 -- 172.21.15.7:0/291523675 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 135.8, "await_for": 45.0}) -- 0x55a292d54420 con 0x55a292e14fd0 2024-06-10T01:16:53.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.952+0000 7f0d8ffff700 1 --2- 172.21.15.7:0/291523675 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a292e14fd0 0x55a292e83590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:16:53.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.953+0000 7f0d8ffff700 1 --2- 172.21.15.7:0/291523675 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a292e14fd0 0x55a292e83590 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f99d9f01cffd58f server_cookie=310ec98c3f69ddbc in_seq=0 out_seq=0 2024-06-10T01:16:53.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:53.953+0000 7f0d6ffff700 10 client.18213 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:54.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:54.018+0000 7f0d6ffff700 1 -- 172.21.15.7:0/291523675 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a292d54420 con 0x55a292e14fd0 2024-06-10T01:16:54.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:54.018+0000 7f0d6ffff700 10 client.18213 handle_command_reply: tid=0 2024-06-10T01:16:54.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:54.018+0000 7f0da1950b80 1 client.18213 shutdown 2024-06-10T01:16:54.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:54.019+0000 7f0da1950b80 1 -- 172.21.15.7:0/291523675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a292dfb460 msgr2=0x55a292e0a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:54.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:54.019+0000 7f0da1950b80 1 --2- 172.21.15.7:0/291523675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a292dfb460 0x55a292e0a1d0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000f3e0 tx=0x7f0d80023c80 comp rx=0 tx=0).stop 2024-06-10T01:16:54.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:54.019+0000 7f0da1950b80 1 -- 172.21.15.7:0/291523675 shutdown_connections 2024-06-10T01:16:54.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:54.019+0000 7f0da1950b80 1 --2- 172.21.15.7:0/291523675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a292e015f0 0x55a292e0ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:54.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:54.019+0000 7f0da1950b80 1 --2- 172.21.15.7:0/291523675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a292dfb460 0x55a292e0a1d0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:54.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:54.019+0000 7f0da1950b80 1 --2- 172.21.15.7:0/291523675 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a292e14fd0 0x55a292e83590 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:54.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:54.019+0000 7f0da1950b80 1 --2- 172.21.15.7:0/291523675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a292e00cd0 0x55a292e0a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:54.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:54.019+0000 7f0d6ffff700 0 client.18213 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:16:54.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:54.019+0000 7f0da1950b80 1 -- 172.21.15.7:0/291523675 >> 172.21.15.7:0/291523675 conn(0x55a292d25e50 msgr2=0x55a292e067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:54.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:54.019+0000 7f0da1950b80 1 -- 172.21.15.7:0/291523675 shutdown_connections 2024-06-10T01:16:54.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:54.019+0000 7f0da1950b80 1 -- 172.21.15.7:0/291523675 wait complete. 2024-06-10T01:16:54.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:54.019+0000 7f0da1950b80 20 client.18213 trim_cache size 0 max 16384 2024-06-10T01:16:54.015 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:16:54.016 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 39, 2024-06-10T01:16:54.016 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:16:54.016 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 34, 2024-06-10T01:16:54.016 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:16:54.016 INFO:teuthology.orchestra.run.smithi007.stdout: "af0b3e22": { 2024-06-10T01:16:54.016 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 34, 2024-06-10T01:16:54.016 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:16:54.016 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:16:54.016 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:16:54.016 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:16:54.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:16:54.016 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:16:54.016 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 135.8, 2024-06-10T01:16:54.016 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:16:54.017 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:16:54.017 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:16:54.017 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:16:54.017 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:16:54.017 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:16:54.017 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:16:54.017 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:16:54.017 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:16:54.017 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:16:54.017 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:16:54.017 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:16:54.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:54.020+0000 7f0d970cd700 1 -- 172.21.15.7:0/1236864436 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0d7c0a2e60 msgr2=0x7f0d7c0a5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:54.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:54.020+0000 7f0d970cd700 1 --2- 172.21.15.7:0/1236864436 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0d7c0a2e60 0x7f0d7c0a5320 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0d901999e0 tx=0x7f0d8000b040 comp rx=0 tx=0).stop 2024-06-10T01:16:54.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:54.020+0000 7f0d970cd700 1 -- 172.21.15.7:0/1236864436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d90103150 msgr2=0x7f0d9019ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:16:54.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:54.020+0000 7f0d970cd700 1 --2- 172.21.15.7:0/1236864436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d90103150 0x7f0d9019ec20 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f0d78006010 tx=0x7f0d7800f690 comp rx=0 tx=0).stop 2024-06-10T01:16:54.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:54.020+0000 7f0d970cd700 1 -- 172.21.15.7:0/1236864436 shutdown_connections 2024-06-10T01:16:54.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:54.020+0000 7f0d970cd700 1 --2- 172.21.15.7:0/1236864436 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0d7c0a2e60 0x7f0d7c0a5320 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:54.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:54.020+0000 7f0d970cd700 1 --2- 172.21.15.7:0/1236864436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d90107870 0x7f0d901988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:54.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:54.020+0000 7f0d970cd700 1 --2- 172.21.15.7:0/1236864436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d901027a0 0x7f0d9019e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:54.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:54.020+0000 7f0d970cd700 1 --2- 172.21.15.7:0/1236864436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d90103150 0x7f0d9019ec20 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:16:54.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:54.020+0000 7f0d970cd700 1 -- 172.21.15.7:0/1236864436 >> 172.21.15.7:0/1236864436 conn(0x7f0d90079570 msgr2=0x7f0d901004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:16:54.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:54.020+0000 7f0d970cd700 1 -- 172.21.15.7:0/1236864436 shutdown_connections 2024-06-10T01:16:54.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:16:54.020+0000 7f0d970cd700 1 -- 172.21.15.7:0/1236864436 wait complete. 2024-06-10T01:16:54.750 INFO:tasks.workunit.client.0.smithi007.stdout: 1 815000 7.62 MB/sec execute 455 sec latency 736.361 ms 2024-06-10T01:16:54.765 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'af0b3e22', quiesce took 6.9 seconds. Will release after: 8.9 2024-06-10T01:16:55.012 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:54 smithi007 ceph-mon[32398]: pgmap v759: 129 pgs: 129 active+clean; 138 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 2.8 MiB/s wr, 319 op/s 2024-06-10T01:16:55.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:54 smithi175 ceph-mon[36636]: pgmap v759: 129 pgs: 129 active+clean; 138 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 2.8 MiB/s wr, 319 op/s 2024-06-10T01:16:55.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:54 smithi078 ceph-mon[36683]: pgmap v759: 129 pgs: 129 active+clean; 138 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 2.8 MiB/s wr, 319 op/s 2024-06-10T01:16:55.750 INFO:tasks.workunit.client.0.smithi007.stdout: 1 815000 7.61 MB/sec execute 456 sec latency 1736.749 ms 2024-06-10T01:16:56.750 INFO:tasks.workunit.client.0.smithi007.stdout: 1 815000 7.59 MB/sec execute 457 sec latency 2736.810 ms 2024-06-10T01:16:57.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:56 smithi175 ceph-mon[36636]: pgmap v760: 129 pgs: 129 active+clean; 138 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 2.7 MiB/s wr, 289 op/s 2024-06-10T01:16:57.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:56 smithi007 ceph-mon[32398]: pgmap v760: 129 pgs: 129 active+clean; 138 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 2.7 MiB/s wr, 289 op/s 2024-06-10T01:16:57.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:56 smithi078 ceph-mon[36683]: pgmap v760: 129 pgs: 129 active+clean; 138 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 2.7 MiB/s wr, 289 op/s 2024-06-10T01:16:57.750 INFO:tasks.workunit.client.0.smithi007.stdout: 1 815000 7.57 MB/sec execute 458 sec latency 3736.911 ms 2024-06-10T01:16:58.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:57 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:16:58.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:57 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:16:58.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:57 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:16:58.750 INFO:tasks.workunit.client.0.smithi007.stdout: 1 815000 7.56 MB/sec execute 459 sec latency 4736.963 ms 2024-06-10T01:16:59.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:16:58 smithi175 ceph-mon[36636]: pgmap v761: 129 pgs: 129 active+clean; 140 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 2.7 MiB/s wr, 278 op/s 2024-06-10T01:16:59.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:16:58 smithi007 ceph-mon[32398]: pgmap v761: 129 pgs: 129 active+clean; 140 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 2.7 MiB/s wr, 278 op/s 2024-06-10T01:16:59.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:16:58 smithi078 ceph-mon[36683]: pgmap v761: 129 pgs: 129 active+clean; 140 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 2.7 MiB/s wr, 278 op/s 2024-06-10T01:16:59.750 INFO:tasks.workunit.client.0.smithi007.stdout: 1 815000 7.54 MB/sec execute 460 sec latency 5737.070 ms 2024-06-10T01:17:00.750 INFO:tasks.workunit.client.0.smithi007.stdout: 1 815000 7.52 MB/sec execute 461 sec latency 6737.193 ms 2024-06-10T01:17:01.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:00 smithi175 ceph-mon[36636]: pgmap v762: 129 pgs: 129 active+clean; 143 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 2.8 MiB/s wr, 256 op/s 2024-06-10T01:17:01.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:00 smithi007 ceph-mon[32398]: pgmap v762: 129 pgs: 129 active+clean; 143 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 2.8 MiB/s wr, 256 op/s 2024-06-10T01:17:01.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:00 smithi078 ceph-mon[36683]: pgmap v762: 129 pgs: 129 active+clean; 143 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 2.8 MiB/s wr, 256 op/s 2024-06-10T01:17:01.750 INFO:tasks.workunit.client.0.smithi007.stdout: 1 815000 7.51 MB/sec execute 462 sec latency 7737.292 ms 2024-06-10T01:17:02.750 INFO:tasks.workunit.client.0.smithi007.stdout: 1 815000 7.49 MB/sec execute 463 sec latency 8737.418 ms 2024-06-10T01:17:03.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:02 smithi175 ceph-mon[36636]: pgmap v763: 129 pgs: 129 active+clean; 141 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 2.1 MiB/s wr, 189 op/s 2024-06-10T01:17:03.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:02 smithi007 ceph-mon[32398]: pgmap v763: 129 pgs: 129 active+clean; 141 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 2.1 MiB/s wr, 189 op/s 2024-06-10T01:17:03.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:02 smithi078 ceph-mon[36683]: pgmap v763: 129 pgs: 129 active+clean; 141 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 2.1 MiB/s wr, 189 op/s 2024-06-10T01:17:03.666 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:17:03.751 INFO:tasks.workunit.client.0.smithi007.stdout: 1 815000 7.48 MB/sec execute 464 sec latency 9737.472 ms 2024-06-10T01:17:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:17:04.751 INFO:tasks.workunit.client.0.smithi007.stdout: 1 815000 7.46 MB/sec execute 465 sec latency 10737.516 ms 2024-06-10T01:17:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.947+0000 7f317a44d640 1 -- 172.21.15.7:0/2854828819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3174100b90 msgr2=0x7f3174101010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.947+0000 7f317a44d640 1 --2- 172.21.15.7:0/2854828819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3174100b90 0x7f3174101010 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f3168009900 tx=0x7f316802a410 comp rx=0 tx=0).stop 2024-06-10T01:17:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.947+0000 7f317a44d640 1 -- 172.21.15.7:0/2854828819 shutdown_connections 2024-06-10T01:17:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.947+0000 7f317a44d640 1 --2- 172.21.15.7:0/2854828819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3174101550 0x7f3174108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.947+0000 7f317a44d640 1 --2- 172.21.15.7:0/2854828819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3174100b90 0x7f3174101010 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.947+0000 7f317a44d640 1 --2- 172.21.15.7:0/2854828819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3174101f40 0x7f3174102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.947+0000 7f317a44d640 1 -- 172.21.15.7:0/2854828819 >> 172.21.15.7:0/2854828819 conn(0x7f31740fb600 msgr2=0x7f31740fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.948+0000 7f317a44d640 1 -- 172.21.15.7:0/2854828819 shutdown_connections 2024-06-10T01:17:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.948+0000 7f317a44d640 1 -- 172.21.15.7:0/2854828819 wait complete. 2024-06-10T01:17:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.948+0000 7f317a44d640 1 Processor -- start 2024-06-10T01:17:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.948+0000 7f317a44d640 1 -- start start 2024-06-10T01:17:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.948+0000 7f317a44d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3174100b90 0x7f31741005f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.948+0000 7f317a44d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3174101550 0x7f317419ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.948+0000 7f317a44d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3174101f40 0x7f317419f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.948+0000 7f317a44d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f317410c5a0 con 0x7f3174100b90 2024-06-10T01:17:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.948+0000 7f317a44d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f317410c420 con 0x7f3174101f40 2024-06-10T01:17:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.948+0000 7f317a44d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f317410c720 con 0x7f3174101550 2024-06-10T01:17:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.949+0000 7f3173fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3174100b90 0x7f31741005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.949+0000 7f3173fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3174100b90 0x7f31741005f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38574/0 (socket says 172.21.15.7:38574) 2024-06-10T01:17:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.949+0000 7f3173fff640 1 -- 172.21.15.7:0/411816745 learned_addr learned my addr 172.21.15.7:0/411816745 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.949+0000 7f31789ea640 1 --2- 172.21.15.7:0/411816745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3174101f40 0x7f317419f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.949+0000 7f31737fe640 1 --2- 172.21.15.7:0/411816745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3174101550 0x7f317419ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.949+0000 7f3173fff640 1 -- 172.21.15.7:0/411816745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3174101550 msgr2=0x7f317419ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.949+0000 7f3173fff640 1 --2- 172.21.15.7:0/411816745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3174101550 0x7f317419ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.949+0000 7f3173fff640 1 -- 172.21.15.7:0/411816745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3174101f40 msgr2=0x7f317419f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.949+0000 7f3173fff640 1 --2- 172.21.15.7:0/411816745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3174101f40 0x7f317419f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.949+0000 7f3173fff640 1 -- 172.21.15.7:0/411816745 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f317419f9c0 con 0x7f3174100b90 2024-06-10T01:17:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.949+0000 7f3173fff640 1 --2- 172.21.15.7:0/411816745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3174100b90 0x7f31741005f0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f316400d770 tx=0x7f316400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.949+0000 7f31717fa640 1 -- 172.21.15.7:0/411816745 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3164017070 con 0x7f3174100b90 2024-06-10T01:17:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.949+0000 7f31717fa640 1 -- 172.21.15.7:0/411816745 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3164009c30 con 0x7f3174100b90 2024-06-10T01:17:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.949+0000 7f317a44d640 1 -- 172.21.15.7:0/411816745 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f317419fc50 con 0x7f3174100b90 2024-06-10T01:17:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.949+0000 7f31717fa640 1 -- 172.21.15.7:0/411816745 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3164003dd0 con 0x7f3174100b90 2024-06-10T01:17:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.950+0000 7f317a44d640 1 -- 172.21.15.7:0/411816745 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31741a0170 con 0x7f3174100b90 2024-06-10T01:17:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.953+0000 7f31717fa640 1 -- 172.21.15.7:0/411816745 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f3164003f70 con 0x7f3174100b90 2024-06-10T01:17:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.953+0000 7f317d237740 1 Processor -- start 2024-06-10T01:17:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.953+0000 7f317d237740 1 -- start start 2024-06-10T01:17:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.953+0000 7f317d237740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc5ceed010 0x55fc5ceed430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.953+0000 7f317d237740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc5ceeda00 0x55fc5cef2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.953+0000 7f317d237740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc5cef2b70 0x55fc5cef5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.953+0000 7f317d237740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fc5ce504f0 con 0x55fc5ceed010 2024-06-10T01:17:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.953+0000 7f317d237740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fc5cdee870 con 0x55fc5cef2b70 2024-06-10T01:17:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.953+0000 7f317d237740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fc5ce4c4b0 con 0x55fc5ceeda00 2024-06-10T01:17:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.953+0000 7f31789ea640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc5ceed010 0x55fc5ceed430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.953+0000 7f31789ea640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc5ceed010 0x55fc5ceed430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38588/0 (socket says 172.21.15.7:38588) 2024-06-10T01:17:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.953+0000 7f31789ea640 1 -- 172.21.15.7:0/2313290668 learned_addr learned my addr 172.21.15.7:0/2313290668 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.953+0000 7f3173fff640 1 --2- 172.21.15.7:0/2313290668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc5ceeda00 0x55fc5cef2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.953+0000 7f31737fe640 1 --2- 172.21.15.7:0/2313290668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc5cef2b70 0x55fc5cef5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.954+0000 7f31789ea640 1 -- 172.21.15.7:0/2313290668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc5ceeda00 msgr2=0x55fc5cef2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.954+0000 7f31789ea640 1 --2- 172.21.15.7:0/2313290668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc5ceeda00 0x55fc5cef2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.954+0000 7f31789ea640 1 -- 172.21.15.7:0/2313290668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc5cef2b70 msgr2=0x55fc5cef5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.954+0000 7f31789ea640 1 --2- 172.21.15.7:0/2313290668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc5cef2b70 0x55fc5cef5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.954+0000 7f31789ea640 1 -- 172.21.15.7:0/2313290668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc5ce0a9b0 con 0x55fc5ceed010 2024-06-10T01:17:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.954+0000 7f31717fa640 1 --2- 172.21.15.7:0/411816745 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f315c0a2b80 0x7f315c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.954+0000 7f3173fff640 1 --2- 172.21.15.7:0/411816745 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f315c0a2b80 0x7f315c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.954+0000 7f31789ea640 1 --2- 172.21.15.7:0/2313290668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc5ceed010 0x55fc5ceed430 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f316c010370 tx=0x7f316c010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bfdf1f02a366fcc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.954+0000 7f31717fa640 1 -- 172.21.15.7:0/411816745 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f31640cecc0 con 0x7f3174100b90 2024-06-10T01:17:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.954+0000 7f315a7fc640 1 -- 172.21.15.7:0/2313290668 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c016070 con 0x55fc5ceed010 2024-06-10T01:17:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.954+0000 7f315a7fc640 1 -- 172.21.15.7:0/2313290668 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f316c004d20 con 0x55fc5ceed010 2024-06-10T01:17:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.954+0000 7f315a7fc640 1 -- 172.21.15.7:0/2313290668 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c005480 con 0x55fc5ceed010 2024-06-10T01:17:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.955+0000 7f317d237740 1 -- 172.21.15.7:0/2313290668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc5ceed010 msgr2=0x55fc5ceed430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.955+0000 7f3173fff640 1 --2- 172.21.15.7:0/411816745 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f315c0a2b80 0x7f315c0a5040 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f31640cc7b0 tx=0x7f3164004590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.955+0000 7f317d237740 1 --2- 172.21.15.7:0/2313290668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc5ceed010 0x55fc5ceed430 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f316c010370 tx=0x7f316c010830 comp rx=0 tx=0).stop 2024-06-10T01:17:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.955+0000 7f317d237740 1 -- 172.21.15.7:0/2313290668 shutdown_connections 2024-06-10T01:17:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.955+0000 7f317d237740 1 --2- 172.21.15.7:0/2313290668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc5cef2b70 0x55fc5cef5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.955+0000 7f317d237740 1 --2- 172.21.15.7:0/2313290668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc5ceeda00 0x55fc5cef2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.955+0000 7f317d237740 1 --2- 172.21.15.7:0/2313290668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc5ceed010 0x55fc5ceed430 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.955+0000 7f317d237740 1 -- 172.21.15.7:0/2313290668 >> 172.21.15.7:0/2313290668 conn(0x55fc5cee8680 msgr2=0x55fc5cee8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.955+0000 7f317d237740 1 -- 172.21.15.7:0/2313290668 shutdown_connections 2024-06-10T01:17:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.955+0000 7f317d237740 1 -- 172.21.15.7:0/2313290668 wait complete. 2024-06-10T01:17:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.955+0000 7f317d237740 1 -- start start 2024-06-10T01:17:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.955+0000 7f317d237740 1 Processor -- start 2024-06-10T01:17:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.956+0000 7f317d237740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc5ceed010 0x55fc5cef7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.956+0000 7f317d237740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc5ceeda00 0x55fc5cef77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.956+0000 7f317d237740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc5cef2b70 0x55fc5cefe800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.956+0000 7f317d237740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fc5cef5b20 con 0x55fc5cef2b70 2024-06-10T01:17:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.956+0000 7f317d237740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fc5cdd9ae0 con 0x55fc5ceed010 2024-06-10T01:17:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.956+0000 7f317d237740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fc5cef5ca0 con 0x55fc5ceeda00 2024-06-10T01:17:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.956+0000 7f317d237740 10 client.0 _ll_register_callbacks cb 0x55fc5ce129b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:17:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.956+0000 7f31789ea640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc5cef2b70 0x55fc5cefe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.956+0000 7f31789ea640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc5cef2b70 0x55fc5cefe800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38590/0 (socket says 172.21.15.7:38590) 2024-06-10T01:17:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.956+0000 7f31789ea640 1 -- 172.21.15.7:0/554239065 learned_addr learned my addr 172.21.15.7:0/554239065 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.956+0000 7f31789ea640 1 --2- 172.21.15.7:0/554239065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc5ceed010 0x55fc5cef7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.956+0000 7f31737fe640 1 --2- 172.21.15.7:0/554239065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc5ceeda00 0x55fc5cef77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.956+0000 7f31789ea640 1 -- 172.21.15.7:0/554239065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc5ceeda00 msgr2=0x55fc5cef77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.956+0000 7f31789ea640 1 --2- 172.21.15.7:0/554239065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc5ceeda00 0x55fc5cef77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.956+0000 7f31789ea640 1 -- 172.21.15.7:0/554239065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc5ceed010 msgr2=0x55fc5cef7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.956+0000 7f31789ea640 1 --2- 172.21.15.7:0/554239065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc5ceed010 0x55fc5cef7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.956+0000 7f31789ea640 1 -- 172.21.15.7:0/554239065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc5cb64210 con 0x55fc5cef2b70 2024-06-10T01:17:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.956+0000 7f31789ea640 1 --2- 172.21.15.7:0/554239065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc5cef2b70 0x55fc5cefe800 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f316c005fd0 tx=0x7f316c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e86c3f5c93ceeab server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.956+0000 7f31537fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:17:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.956+0000 7f31537fe640 1 -- 172.21.15.7:0/554239065 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c016030 con 0x55fc5cef2b70 2024-06-10T01:17:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.957+0000 7f31537fe640 1 -- 172.21.15.7:0/554239065 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f316c003970 con 0x55fc5cef2b70 2024-06-10T01:17:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.957+0000 7f317d237740 1 -- 172.21.15.7:0/554239065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f316c004d20 con 0x55fc5cef2b70 2024-06-10T01:17:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.957+0000 7f31537fe640 1 -- 172.21.15.7:0/554239065 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f316c032050 con 0x55fc5cef2b70 2024-06-10T01:17:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.957+0000 7f317d237740 10 client.18225 fetch_fsmap learned FSMap version 40 2024-06-10T01:17:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.957+0000 7f317d237740 1 -- 172.21.15.7:0/554239065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55fc5cc30620 con 0x55fc5cef2b70 2024-06-10T01:17:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.957+0000 7f31537fe640 1 -- 172.21.15.7:0/554239065 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f316c004d20 con 0x55fc5cef2b70 2024-06-10T01:17:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.957+0000 7f31537fe640 1 -- 172.21.15.7:0/554239065 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f316c00f040 con 0x55fc5cef2b70 2024-06-10T01:17:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.957+0000 7f317d237740 10 client.18225 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:17:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.957+0000 7f317d237740 10 client.18225 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:17:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.957+0000 7f317d237740 20 client.18225 populate_metadata read hostname 'smithi007' 2024-06-10T01:17:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.957+0000 7f317d237740 1 --2- 172.21.15.7:0/554239065 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fc5cf02df0 0x55fc5cf051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.957+0000 7f317d237740 4 client.18225 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:17:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.957+0000 7f317d237740 1 -- 172.21.15.7:0/554239065 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc5ce63f10 con 0x55fc5cf02df0 2024-06-10T01:17:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.957+0000 7f31737fe640 1 --2- 172.21.15.7:0/554239065 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fc5cf02df0 0x55fc5cf051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.958+0000 7f31737fe640 1 --2- 172.21.15.7:0/554239065 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fc5cf02df0 0x55fc5cf051e0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7dea48ff495019a server_cookie=8e2be77a9026f649 in_seq=0 out_seq=0 2024-06-10T01:17:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.958+0000 7f31537fe640 10 client.18225 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.958+0000 7f31537fe640 1 -- 172.21.15.7:0/554239065 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fc5ce63f10 con 0x55fc5cf02df0 2024-06-10T01:17:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.958+0000 7f31537fe640 10 client.18225 handle_command_reply: tid=0 2024-06-10T01:17:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.958+0000 7f317d237740 1 client.18225 shutdown 2024-06-10T01:17:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.959+0000 7f317d237740 1 -- 172.21.15.7:0/554239065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc5cef2b70 msgr2=0x55fc5cefe800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.959+0000 7f317d237740 1 --2- 172.21.15.7:0/554239065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc5cef2b70 0x55fc5cefe800 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f316c005fd0 tx=0x7f316c009bb0 comp rx=0 tx=0).stop 2024-06-10T01:17:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.959+0000 7f317d237740 1 -- 172.21.15.7:0/554239065 shutdown_connections 2024-06-10T01:17:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.959+0000 7f317d237740 1 --2- 172.21.15.7:0/554239065 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fc5cf02df0 0x55fc5cf051e0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.959+0000 7f317d237740 1 --2- 172.21.15.7:0/554239065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc5cef2b70 0x55fc5cefe800 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.959+0000 7f317d237740 1 --2- 172.21.15.7:0/554239065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc5ceeda00 0x55fc5cef77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.959+0000 7f31537fe640 0 client.18225 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.959+0000 7f317d237740 1 --2- 172.21.15.7:0/554239065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc5ceed010 0x55fc5cef7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.959+0000 7f317d237740 1 -- 172.21.15.7:0/554239065 >> 172.21.15.7:0/554239065 conn(0x55fc5cee8680 msgr2=0x55fc5cee77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.959+0000 7f317d237740 1 -- 172.21.15.7:0/554239065 shutdown_connections 2024-06-10T01:17:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.959+0000 7f317d237740 1 -- 172.21.15.7:0/554239065 wait complete. 2024-06-10T01:17:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.959+0000 7f317d237740 20 client.18225 trim_cache size 0 max 16384 2024-06-10T01:17:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.974+0000 7f317d237740 1 Processor -- start 2024-06-10T01:17:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.974+0000 7f317d237740 1 -- start start 2024-06-10T01:17:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.974+0000 7f317d237740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc5ceed010 0x55fc5ceeb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.974+0000 7f317d237740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc5ceeda00 0x55fc5cefe790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.974+0000 7f317d237740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc5cef2b70 0x55fc5ceeb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.974+0000 7f317d237740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f31741acef0 con 0x55fc5ceeda00 2024-06-10T01:17:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.974+0000 7f317d237740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f31741aa7c0 con 0x55fc5cef2b70 2024-06-10T01:17:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.974+0000 7f317d237740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f31741a8610 con 0x55fc5ceed010 2024-06-10T01:17:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.974+0000 7f31737fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc5ceeda00 0x55fc5cefe790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.974+0000 7f31737fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc5ceeda00 0x55fc5cefe790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38606/0 (socket says 172.21.15.7:38606) 2024-06-10T01:17:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.974+0000 7f31737fe640 1 -- 172.21.15.7:0/1607964964 learned_addr learned my addr 172.21.15.7:0/1607964964 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.975+0000 7f31789ea640 1 --2- 172.21.15.7:0/1607964964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc5ceed010 0x55fc5ceeb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.975+0000 7f31737fe640 1 -- 172.21.15.7:0/1607964964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc5ceed010 msgr2=0x55fc5ceeb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.975+0000 7f31737fe640 1 --2- 172.21.15.7:0/1607964964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc5ceed010 0x55fc5ceeb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.975+0000 7f31789ea640 1 --2- 172.21.15.7:0/1607964964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc5cef2b70 0x55fc5ceeb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.975+0000 7f31737fe640 1 -- 172.21.15.7:0/1607964964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc5cef2b70 msgr2=0x55fc5ceeb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.975+0000 7f31737fe640 1 --2- 172.21.15.7:0/1607964964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc5cef2b70 0x55fc5ceeb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.975+0000 7f31737fe640 1 -- 172.21.15.7:0/1607964964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc5cee84b0 con 0x55fc5ceeda00 2024-06-10T01:17:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.975+0000 7f31789ea640 1 --2- 172.21.15.7:0/1607964964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc5cef2b70 0x55fc5ceeb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:17:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.975+0000 7f31737fe640 1 --2- 172.21.15.7:0/1607964964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc5ceeda00 0x55fc5cefe790 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f31680314e0 tx=0x7f3168005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665e028b327b8e6d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.975+0000 7f3159ffb640 1 -- 172.21.15.7:0/1607964964 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316804a070 con 0x55fc5ceeda00 2024-06-10T01:17:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.975+0000 7f3159ffb640 1 -- 172.21.15.7:0/1607964964 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3168005520 con 0x55fc5ceeda00 2024-06-10T01:17:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.975+0000 7f3159ffb640 1 -- 172.21.15.7:0/1607964964 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3168035040 con 0x55fc5ceeda00 2024-06-10T01:17:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.976+0000 7f317d237740 1 -- 172.21.15.7:0/1607964964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc5ceeda00 msgr2=0x55fc5cefe790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.976+0000 7f317d237740 1 --2- 172.21.15.7:0/1607964964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc5ceeda00 0x55fc5cefe790 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f31680314e0 tx=0x7f3168005fd0 comp rx=0 tx=0).stop 2024-06-10T01:17:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.976+0000 7f317d237740 1 -- 172.21.15.7:0/1607964964 shutdown_connections 2024-06-10T01:17:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.976+0000 7f317d237740 1 --2- 172.21.15.7:0/1607964964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc5cef2b70 0x55fc5ceeb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.976+0000 7f317d237740 1 --2- 172.21.15.7:0/1607964964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc5ceeda00 0x55fc5cefe790 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.976+0000 7f317d237740 1 --2- 172.21.15.7:0/1607964964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc5ceed010 0x55fc5ceeb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.976+0000 7f317d237740 1 -- 172.21.15.7:0/1607964964 >> 172.21.15.7:0/1607964964 conn(0x55fc5cee8680 msgr2=0x55fc5ceeec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.976+0000 7f317d237740 1 -- 172.21.15.7:0/1607964964 shutdown_connections 2024-06-10T01:17:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.976+0000 7f317d237740 1 -- 172.21.15.7:0/1607964964 wait complete. 2024-06-10T01:17:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.976+0000 7f317d237740 1 -- start start 2024-06-10T01:17:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.976+0000 7f317d237740 1 Processor -- start 2024-06-10T01:17:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.976+0000 7f317d237740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc5ceed010 0x55fc5cf687a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.977+0000 7f317d237740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc5ceeda00 0x55fc5cf68ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.977+0000 7f317d237740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc5cef2b70 0x55fc5cf6fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.977+0000 7f317d237740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f31741a8c80 con 0x55fc5cef2b70 2024-06-10T01:17:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.977+0000 7f317d237740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f31741aa0f0 con 0x55fc5ceed010 2024-06-10T01:17:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.977+0000 7f317d237740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f31741a9350 con 0x55fc5ceeda00 2024-06-10T01:17:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.977+0000 7f317d237740 10 client.0 _ll_register_callbacks cb 0x7f316c009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:17:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.977+0000 7f31789ea640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc5cef2b70 0x55fc5cf6fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.977+0000 7f31789ea640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc5cef2b70 0x55fc5cf6fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38620/0 (socket says 172.21.15.7:38620) 2024-06-10T01:17:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.977+0000 7f31789ea640 1 -- 172.21.15.7:0/2757663259 learned_addr learned my addr 172.21.15.7:0/2757663259 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.977+0000 7f31737fe640 1 --2- 172.21.15.7:0/2757663259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc5ceeda00 0x55fc5cf68ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.977+0000 7f31789ea640 1 --2- 172.21.15.7:0/2757663259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc5ceed010 0x55fc5cf687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.977+0000 7f31789ea640 1 -- 172.21.15.7:0/2757663259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc5ceeda00 msgr2=0x55fc5cf68ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.977+0000 7f31789ea640 1 --2- 172.21.15.7:0/2757663259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc5ceeda00 0x55fc5cf68ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.977+0000 7f31789ea640 1 -- 172.21.15.7:0/2757663259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc5ceed010 msgr2=0x55fc5cf687a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.977+0000 7f31789ea640 1 --2- 172.21.15.7:0/2757663259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc5ceed010 0x55fc5cf687a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.977+0000 7f31789ea640 1 -- 172.21.15.7:0/2757663259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc5cf70310 con 0x55fc5cef2b70 2024-06-10T01:17:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.977+0000 7f31789ea640 1 --2- 172.21.15.7:0/2757663259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc5cef2b70 0x55fc5cf6fd20 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f316c009ac0 tx=0x7f316c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fddf44daa2ea604 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.977+0000 7f31537fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:17:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.977+0000 7f31537fe640 1 -- 172.21.15.7:0/2757663259 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c016040 con 0x55fc5cef2b70 2024-06-10T01:17:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.977+0000 7f31537fe640 1 -- 172.21.15.7:0/2757663259 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f316c005220 con 0x55fc5cef2b70 2024-06-10T01:17:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.977+0000 7f317d237740 1 -- 172.21.15.7:0/2757663259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3168005520 con 0x55fc5cef2b70 2024-06-10T01:17:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.977+0000 7f31537fe640 1 -- 172.21.15.7:0/2757663259 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f316c023040 con 0x55fc5cef2b70 2024-06-10T01:17:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.978+0000 7f317d237740 10 client.18231 fetch_fsmap learned FSMap version 40 2024-06-10T01:17:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.978+0000 7f317d237740 1 -- 172.21.15.7:0/2757663259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55fc5cf71590 con 0x55fc5cef2b70 2024-06-10T01:17:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.978+0000 7f31537fe640 1 -- 172.21.15.7:0/2757663259 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3168005520 con 0x55fc5cef2b70 2024-06-10T01:17:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.978+0000 7f31537fe640 1 -- 172.21.15.7:0/2757663259 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f316c037440 con 0x55fc5cef2b70 2024-06-10T01:17:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.978+0000 7f317d237740 10 client.18231 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:17:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.978+0000 7f317d237740 10 client.18231 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:17:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.978+0000 7f317d237740 20 client.18231 populate_metadata read hostname 'smithi007' 2024-06-10T01:17:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.978+0000 7f317d237740 1 --2- 172.21.15.7:0/2757663259 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fc5cf71f60 0x55fc5cf74380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.978+0000 7f317d237740 4 client.18231 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:17:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.978+0000 7f317d237740 1 -- 172.21.15.7:0/2757663259 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fc5ce5dc20 con 0x55fc5cf71f60 2024-06-10T01:17:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.978+0000 7f31737fe640 1 --2- 172.21.15.7:0/2757663259 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fc5cf71f60 0x55fc5cf74380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.979+0000 7f31737fe640 1 --2- 172.21.15.7:0/2757663259 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fc5cf71f60 0x55fc5cf74380 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f66f57c3361c37ee server_cookie=a7a32b93f47a0ded in_seq=0 out_seq=0 2024-06-10T01:17:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:04.979+0000 7f31537fe640 10 client.18231 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:05.013+0000 7f31537fe640 1 -- 172.21.15.7:0/2757663259 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fc5ce5dc20 con 0x55fc5cf71f60 2024-06-10T01:17:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:05.013+0000 7f31537fe640 10 client.18231 handle_command_reply: tid=0 2024-06-10T01:17:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:05.013+0000 7f317d237740 1 client.18231 shutdown 2024-06-10T01:17:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:05.014+0000 7f317d237740 1 -- 172.21.15.7:0/2757663259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc5cef2b70 msgr2=0x55fc5cf6fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:05.014+0000 7f317d237740 1 --2- 172.21.15.7:0/2757663259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc5cef2b70 0x55fc5cf6fd20 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f316c009ac0 tx=0x7f316c009bb0 comp rx=0 tx=0).stop 2024-06-10T01:17:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:05.014+0000 7f317d237740 1 -- 172.21.15.7:0/2757663259 shutdown_connections 2024-06-10T01:17:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:05.014+0000 7f317d237740 1 --2- 172.21.15.7:0/2757663259 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fc5cf71f60 0x55fc5cf74380 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:05.014+0000 7f317d237740 1 --2- 172.21.15.7:0/2757663259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc5cef2b70 0x55fc5cf6fd20 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:05.014+0000 7f31537fe640 0 client.18231 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:05.014+0000 7f317d237740 1 --2- 172.21.15.7:0/2757663259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fc5ceeda00 0x55fc5cf68ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:05.014+0000 7f317d237740 1 --2- 172.21.15.7:0/2757663259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc5ceed010 0x55fc5cf687a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:05.014+0000 7f317d237740 1 -- 172.21.15.7:0/2757663259 >> 172.21.15.7:0/2757663259 conn(0x55fc5cee8680 msgr2=0x55fc5ceeb360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:05.014+0000 7f317d237740 1 -- 172.21.15.7:0/2757663259 shutdown_connections 2024-06-10T01:17:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:05.014+0000 7f317d237740 1 -- 172.21.15.7:0/2757663259 wait complete. 2024-06-10T01:17:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:05.014+0000 7f317d237740 20 client.18231 trim_cache size 0 max 16384 2024-06-10T01:17:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:17:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:17:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:17:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:05.014+0000 7f317a44d640 1 -- 172.21.15.7:0/411816745 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f315c0a2b80 msgr2=0x7f315c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:05.014+0000 7f317a44d640 1 --2- 172.21.15.7:0/411816745 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f315c0a2b80 0x7f315c0a5040 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f31640cc7b0 tx=0x7f3164004590 comp rx=0 tx=0).stop 2024-06-10T01:17:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:05.014+0000 7f317a44d640 1 -- 172.21.15.7:0/411816745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3174100b90 msgr2=0x7f31741005f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:05.015+0000 7f317a44d640 1 --2- 172.21.15.7:0/411816745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3174100b90 0x7f31741005f0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f316400d770 tx=0x7f316400dc30 comp rx=0 tx=0).stop 2024-06-10T01:17:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:05.015+0000 7f317a44d640 1 -- 172.21.15.7:0/411816745 shutdown_connections 2024-06-10T01:17:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:05.015+0000 7f317a44d640 1 --2- 172.21.15.7:0/411816745 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f315c0a2b80 0x7f315c0a5040 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:05.015+0000 7f317a44d640 1 --2- 172.21.15.7:0/411816745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3174101f40 0x7f317419f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:05.015+0000 7f317a44d640 1 --2- 172.21.15.7:0/411816745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3174101550 0x7f317419ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:05.015+0000 7f317a44d640 1 --2- 172.21.15.7:0/411816745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3174100b90 0x7f31741005f0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:05.015+0000 7f317a44d640 1 -- 172.21.15.7:0/411816745 >> 172.21.15.7:0/411816745 conn(0x7f31740fb600 msgr2=0x7f3174104f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:05.015+0000 7f317a44d640 1 -- 172.21.15.7:0/411816745 shutdown_connections 2024-06-10T01:17:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:05.015+0000 7f317a44d640 1 -- 172.21.15.7:0/411816745 wait complete. 2024-06-10T01:17:05.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:04 smithi175 ceph-mon[36636]: pgmap v764: 129 pgs: 129 active+clean; 139 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 2.2 MiB/s wr, 189 op/s 2024-06-10T01:17:05.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:04 smithi007 ceph-mon[32398]: pgmap v764: 129 pgs: 129 active+clean; 139 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 2.2 MiB/s wr, 189 op/s 2024-06-10T01:17:05.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:04 smithi078 ceph-mon[36683]: pgmap v764: 129 pgs: 129 active+clean; 139 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 2.2 MiB/s wr, 189 op/s 2024-06-10T01:17:05.750 INFO:tasks.workunit.client.0.smithi007.stdout: 1 815000 7.44 MB/sec execute 466 sec latency 11737.578 ms 2024-06-10T01:17:05.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.789+0000 7fedf041d700 1 -- 172.21.15.7:0/3052245790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede8101090 msgr2=0x7fede8101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:05.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.789+0000 7fedf041d700 1 --2- 172.21.15.7:0/3052245790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede8101090 0x7fede8101470 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fede4009990 tx=0x7fede4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:17:05.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.789+0000 7fedf041d700 1 -- 172.21.15.7:0/3052245790 shutdown_connections 2024-06-10T01:17:05.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.789+0000 7fedf041d700 1 --2- 172.21.15.7:0/3052245790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fede8105f30 0x7fede810da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:05.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.789+0000 7fedf041d700 1 --2- 172.21.15.7:0/3052245790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede8101090 0x7fede8101470 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:05.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.789+0000 7fedf041d700 1 --2- 172.21.15.7:0/3052245790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fede8101a40 0x7fede8105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:05.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.789+0000 7fedf041d700 1 -- 172.21.15.7:0/3052245790 >> 172.21.15.7:0/3052245790 conn(0x7fede80766b0 msgr2=0x7fede8076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:05.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.789+0000 7fedf041d700 1 -- 172.21.15.7:0/3052245790 shutdown_connections 2024-06-10T01:17:05.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.789+0000 7fedf041d700 1 -- 172.21.15.7:0/3052245790 wait complete. 2024-06-10T01:17:05.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.789+0000 7fedf041d700 1 Processor -- start 2024-06-10T01:17:05.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.789+0000 7fedf041d700 1 -- start start 2024-06-10T01:17:05.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.790+0000 7fedf041d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede8101090 0x7fede810f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:05.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.790+0000 7fedf041d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fede8101a40 0x7fede810f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:05.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.790+0000 7fedf041d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fede8105f30 0x7fede810fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:05.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.790+0000 7fedf041d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fede8112850 con 0x7fede8101090 2024-06-10T01:17:05.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.790+0000 7fedf041d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fede81126d0 con 0x7fede8101a40 2024-06-10T01:17:05.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.790+0000 7fedf041d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fede81129d0 con 0x7fede8105f30 2024-06-10T01:17:05.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.790+0000 7fedee1b9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede8101090 0x7fede810f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:05.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.790+0000 7fedee1b9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede8101090 0x7fede810f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38636/0 (socket says 172.21.15.7:38636) 2024-06-10T01:17:05.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.790+0000 7fedee1b9700 1 -- 172.21.15.7:0/3230072817 learned_addr learned my addr 172.21.15.7:0/3230072817 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:05.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.790+0000 7feded9b8700 1 --2- 172.21.15.7:0/3230072817 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fede8101a40 0x7fede810f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:05.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.790+0000 7fedee9ba700 1 --2- 172.21.15.7:0/3230072817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fede8105f30 0x7fede810fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:05.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.790+0000 7fedee1b9700 1 -- 172.21.15.7:0/3230072817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fede8105f30 msgr2=0x7fede810fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:05.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.790+0000 7fedee1b9700 1 --2- 172.21.15.7:0/3230072817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fede8105f30 0x7fede810fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:05.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.790+0000 7fedee1b9700 1 -- 172.21.15.7:0/3230072817 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fede8101a40 msgr2=0x7fede810f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:05.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.790+0000 7fedee1b9700 1 --2- 172.21.15.7:0/3230072817 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fede8101a40 0x7fede810f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:05.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.790+0000 7fedee1b9700 1 -- 172.21.15.7:0/3230072817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede81a7190 con 0x7fede8101090 2024-06-10T01:17:05.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.790+0000 7fedee1b9700 1 --2- 172.21.15.7:0/3230072817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede8101090 0x7fede810f190 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fede400bae0 tx=0x7fede400bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:05.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.791+0000 7feddf7fe700 1 -- 172.21.15.7:0/3230072817 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede4026070 con 0x7fede8101090 2024-06-10T01:17:05.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.791+0000 7fedf041d700 1 -- 172.21.15.7:0/3230072817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede81a7420 con 0x7fede8101090 2024-06-10T01:17:05.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.791+0000 7feddf7fe700 1 -- 172.21.15.7:0/3230072817 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fede4003d70 con 0x7fede8101090 2024-06-10T01:17:05.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.791+0000 7feddf7fe700 1 -- 172.21.15.7:0/3230072817 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede4021410 con 0x7fede8101090 2024-06-10T01:17:05.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.791+0000 7fedf041d700 1 -- 172.21.15.7:0/3230072817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede81a7900 con 0x7fede8101090 2024-06-10T01:17:05.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.791+0000 7fedf041d700 1 -- 172.21.15.7:0/3230072817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fede81090c0 con 0x7fede8101090 2024-06-10T01:17:05.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.793+0000 7feddf7fe700 1 -- 172.21.15.7:0/3230072817 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fede4029070 con 0x7fede8101090 2024-06-10T01:17:05.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.794+0000 7feddf7fe700 1 --2- 172.21.15.7:0/3230072817 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fedd40a2c20 0x7fedd40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:05.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.794+0000 7feded9b8700 1 --2- 172.21.15.7:0/3230072817 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fedd40a2c20 0x7fedd40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:05.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.794+0000 7feddf7fe700 1 -- 172.21.15.7:0/3230072817 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fede40d07a0 con 0x7fede8101090 2024-06-10T01:17:05.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.794+0000 7feded9b8700 1 --2- 172.21.15.7:0/3230072817 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fedd40a2c20 0x7fedd40a50e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fede8105980 tx=0x7fedd800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:05.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:05.797+0000 7feddf7fe700 1 -- 172.21.15.7:0/3230072817 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fede4030080 con 0x7fede8101090 2024-06-10T01:17:06.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:06.053+0000 7fedf041d700 1 -- 172.21.15.7:0/3230072817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fede81106c0 con 0x7fede8101090 2024-06-10T01:17:06.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:06.054+0000 7feddf7fe700 1 -- 172.21.15.7:0/3230072817 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11855 (secure 0 0 0) 0x7fede4013070 con 0x7fede8101090 2024-06-10T01:17:06.048 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:17:06.048 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:16:56.409130+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24451,24434,24436]},"id":1}]} 2024-06-10T01:17:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:06.056+0000 7fedf041d700 1 -- 172.21.15.7:0/3230072817 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fedd40a2c20 msgr2=0x7fedd40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:06.056+0000 7fedf041d700 1 --2- 172.21.15.7:0/3230072817 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fedd40a2c20 0x7fedd40a50e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fede8105980 tx=0x7fedd800a300 comp rx=0 tx=0).stop 2024-06-10T01:17:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:06.056+0000 7fedf041d700 1 -- 172.21.15.7:0/3230072817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede8101090 msgr2=0x7fede810f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:06.056+0000 7fedf041d700 1 --2- 172.21.15.7:0/3230072817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede8101090 0x7fede810f190 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fede400bae0 tx=0x7fede400bb10 comp rx=0 tx=0).stop 2024-06-10T01:17:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:06.056+0000 7fedf041d700 1 -- 172.21.15.7:0/3230072817 shutdown_connections 2024-06-10T01:17:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:06.056+0000 7fedf041d700 1 --2- 172.21.15.7:0/3230072817 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fedd40a2c20 0x7fedd40a50e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:06.056+0000 7fedf041d700 1 --2- 172.21.15.7:0/3230072817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fede8105f30 0x7fede810fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:06.056+0000 7fedf041d700 1 --2- 172.21.15.7:0/3230072817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede8101090 0x7fede810f190 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:06.056+0000 7fedf041d700 1 --2- 172.21.15.7:0/3230072817 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fede8101a40 0x7fede810f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:06.056+0000 7fedf041d700 1 -- 172.21.15.7:0/3230072817 >> 172.21.15.7:0/3230072817 conn(0x7fede80766b0 msgr2=0x7fede8107040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:06.057+0000 7fedf041d700 1 -- 172.21.15.7:0/3230072817 shutdown_connections 2024-06-10T01:17:06.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:06.057+0000 7fedf041d700 1 -- 172.21.15.7:0/3230072817 wait complete. 2024-06-10T01:17:06.052 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 40 2024-06-10T01:17:06.751 INFO:tasks.workunit.client.0.smithi007.stdout: 1 815000 7.43 MB/sec execute 467 sec latency 12737.659 ms 2024-06-10T01:17:06.848 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db --set-id af0b3e22 --release --await' 2024-06-10T01:17:06.848 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db --set-id af0b3e22 --release --await 2024-06-10T01:17:07.106 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:17:07.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:06 smithi175 ceph-mon[36636]: pgmap v765: 129 pgs: 129 active+clean; 138 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-06-10T01:17:07.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:06 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3230072817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:17:07.229 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:06 smithi007 ceph-mon[32398]: pgmap v765: 129 pgs: 129 active+clean; 138 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-06-10T01:17:07.229 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:06 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3230072817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:17:07.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:06 smithi078 ceph-mon[36683]: pgmap v765: 129 pgs: 129 active+clean; 138 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-06-10T01:17:07.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:06 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3230072817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:17:07.751 INFO:tasks.workunit.client.0.smithi007.stdout: 1 815000 7.41 MB/sec execute 468 sec latency 13737.723 ms 2024-06-10T01:17:08.751 INFO:tasks.workunit.client.0.smithi007.stdout: 1 815000 7.40 MB/sec execute 469 sec latency 14737.784 ms 2024-06-10T01:17:08.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.899+0000 7f5167bdd700 1 -- 172.21.15.7:0/813715563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5160103150 msgr2=0x7f5160107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:08.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.899+0000 7f5167bdd700 1 --2- 172.21.15.7:0/813715563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5160103150 0x7f5160107140 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f51540099c0 tx=0x7f5154009cd0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.899+0000 7f5167bdd700 1 -- 172.21.15.7:0/813715563 shutdown_connections 2024-06-10T01:17:08.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.899+0000 7f5167bdd700 1 --2- 172.21.15.7:0/813715563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5160107870 0x7f516010f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.899+0000 7f5167bdd700 1 --2- 172.21.15.7:0/813715563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51601027a0 0x7f5160102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.899+0000 7f5167bdd700 1 --2- 172.21.15.7:0/813715563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5160103150 0x7f5160107140 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.899+0000 7f5167bdd700 1 -- 172.21.15.7:0/813715563 >> 172.21.15.7:0/813715563 conn(0x7f5160079520 msgr2=0x7f5160079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:08.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.899+0000 7f5167bdd700 1 -- 172.21.15.7:0/813715563 shutdown_connections 2024-06-10T01:17:08.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.899+0000 7f5167bdd700 1 -- 172.21.15.7:0/813715563 wait complete. 2024-06-10T01:17:08.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.899+0000 7f5167bdd700 1 Processor -- start 2024-06-10T01:17:08.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.900+0000 7f5167bdd700 1 -- start start 2024-06-10T01:17:08.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.900+0000 7f5167bdd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51601027a0 0x7f516019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:08.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.900+0000 7f5167bdd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5160103150 0x7f516019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:08.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.900+0000 7f5167bdd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5160107870 0x7f51601989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:08.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.900+0000 7f5167bdd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5160112800 con 0x7f51601027a0 2024-06-10T01:17:08.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.900+0000 7f5167bdd700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5160112680 con 0x7f5160107870 2024-06-10T01:17:08.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.900+0000 7f5167bdd700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5160112980 con 0x7f5160103150 2024-06-10T01:17:08.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.900+0000 7f5165979700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51601027a0 0x7f516019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:08.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.900+0000 7f5165979700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51601027a0 0x7f516019e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38658/0 (socket says 172.21.15.7:38658) 2024-06-10T01:17:08.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.900+0000 7f5165979700 1 -- 172.21.15.7:0/1456763428 learned_addr learned my addr 172.21.15.7:0/1456763428 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:08.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.900+0000 7f516617a700 1 --2- 172.21.15.7:0/1456763428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5160107870 0x7f51601989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:08.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.900+0000 7f5165979700 1 -- 172.21.15.7:0/1456763428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5160103150 msgr2=0x7f516019ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:08.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.900+0000 7f5165979700 1 --2- 172.21.15.7:0/1456763428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5160103150 0x7f516019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.900+0000 7f5165979700 1 -- 172.21.15.7:0/1456763428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5160107870 msgr2=0x7f51601989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:08.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.900+0000 7f5165979700 1 --2- 172.21.15.7:0/1456763428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5160107870 0x7f51601989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.900+0000 7f5165979700 1 -- 172.21.15.7:0/1456763428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5160199140 con 0x7f51601027a0 2024-06-10T01:17:08.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.901+0000 7f5165979700 1 --2- 172.21.15.7:0/1456763428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51601027a0 0x7f516019e820 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f515c00b530 tx=0x7f515c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:08.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.901+0000 7f5152ffd700 1 -- 172.21.15.7:0/1456763428 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515c015020 con 0x7f51601027a0 2024-06-10T01:17:08.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.901+0000 7f5167bdd700 1 -- 172.21.15.7:0/1456763428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5160199430 con 0x7f51601027a0 2024-06-10T01:17:08.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.901+0000 7f5152ffd700 1 -- 172.21.15.7:0/1456763428 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f515c011d10 con 0x7f51601027a0 2024-06-10T01:17:08.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.901+0000 7f5152ffd700 1 -- 172.21.15.7:0/1456763428 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515c00f2e0 con 0x7f51601027a0 2024-06-10T01:17:08.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.901+0000 7f5167bdd700 1 -- 172.21.15.7:0/1456763428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51601a56f0 con 0x7f51601027a0 2024-06-10T01:17:08.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.903+0000 7f5152ffd700 1 -- 172.21.15.7:0/1456763428 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f515c018020 con 0x7f51601027a0 2024-06-10T01:17:08.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.904+0000 7f5152ffd700 1 --2- 172.21.15.7:0/1456763428 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f514c0a2ca0 0x7f514c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:08.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.904+0000 7f5165178700 1 --2- 172.21.15.7:0/1456763428 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f514c0a2ca0 0x7f514c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:08.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.904+0000 7f5152ffd700 1 -- 172.21.15.7:0/1456763428 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f515c0cec30 con 0x7f51601027a0 2024-06-10T01:17:08.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.905+0000 7f5165178700 1 --2- 172.21.15.7:0/1456763428 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f514c0a2ca0 0x7f514c0a5160 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f51540098e0 tx=0x7f5154000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:08.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.905+0000 7f5172460b80 1 Processor -- start 2024-06-10T01:17:08.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.905+0000 7f5172460b80 1 -- start start 2024-06-10T01:17:08.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.905+0000 7f5172460b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a265788910 0x55a265788cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:08.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.905+0000 7f5172460b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a265789230 0x55a265782b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:08.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.905+0000 7f5172460b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a265783060 0x55a265783480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:08.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.905+0000 7f5172460b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a265671060 con 0x55a265788910 2024-06-10T01:17:08.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.905+0000 7f5172460b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a2656cf950 con 0x55a265789230 2024-06-10T01:17:08.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.905+0000 7f5172460b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a265637560 con 0x55a265783060 2024-06-10T01:17:08.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.906+0000 7f5165979700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a265788910 0x55a265788cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:08.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.906+0000 7f5165979700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a265788910 0x55a265788cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38662/0 (socket says 172.21.15.7:38662) 2024-06-10T01:17:08.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.906+0000 7f5165979700 1 -- 172.21.15.7:0/3106759274 learned_addr learned my addr 172.21.15.7:0/3106759274 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:08.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.906+0000 7f5165178700 1 --2- 172.21.15.7:0/3106759274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a265789230 0x55a265782b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:08.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.906+0000 7f516617a700 1 --2- 172.21.15.7:0/3106759274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a265783060 0x55a265783480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:08.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.906+0000 7f5165979700 1 -- 172.21.15.7:0/3106759274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a265783060 msgr2=0x55a265783480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:08.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.906+0000 7f5165979700 1 --2- 172.21.15.7:0/3106759274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a265783060 0x55a265783480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.906+0000 7f5165979700 1 -- 172.21.15.7:0/3106759274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a265789230 msgr2=0x55a265782b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:08.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.906+0000 7f5165979700 1 --2- 172.21.15.7:0/3106759274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a265789230 0x55a265782b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.906+0000 7f5165979700 1 -- 172.21.15.7:0/3106759274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a26561e9d0 con 0x55a265788910 2024-06-10T01:17:08.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.906+0000 7f5165979700 1 --2- 172.21.15.7:0/3106759274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a265788910 0x55a265788cf0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f515c097570 tx=0x7f515c09b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cbd94f2d206ccf8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:08.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.906+0000 7f514bfff700 1 -- 172.21.15.7:0/3106759274 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515c09f4c0 con 0x55a265788910 2024-06-10T01:17:08.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.906+0000 7f514bfff700 1 -- 172.21.15.7:0/3106759274 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f515c0a3d80 con 0x55a265788910 2024-06-10T01:17:08.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.906+0000 7f514bfff700 1 -- 172.21.15.7:0/3106759274 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515c0bb0b0 con 0x55a265788910 2024-06-10T01:17:08.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.907+0000 7f5172460b80 1 -- 172.21.15.7:0/3106759274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a265788910 msgr2=0x55a265788cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:08.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.907+0000 7f5172460b80 1 --2- 172.21.15.7:0/3106759274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a265788910 0x55a265788cf0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f515c097570 tx=0x7f515c09b8e0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.907+0000 7f5172460b80 1 -- 172.21.15.7:0/3106759274 shutdown_connections 2024-06-10T01:17:08.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.907+0000 7f5172460b80 1 --2- 172.21.15.7:0/3106759274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a265783060 0x55a265783480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.907+0000 7f5172460b80 1 --2- 172.21.15.7:0/3106759274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a265788910 0x55a265788cf0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.907+0000 7f5172460b80 1 --2- 172.21.15.7:0/3106759274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a265789230 0x55a265782b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.907+0000 7f5172460b80 1 -- 172.21.15.7:0/3106759274 >> 172.21.15.7:0/3106759274 conn(0x55a2656ab180 msgr2=0x55a265779a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:08.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.907+0000 7f5172460b80 1 -- 172.21.15.7:0/3106759274 shutdown_connections 2024-06-10T01:17:08.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.907+0000 7f5172460b80 1 -- 172.21.15.7:0/3106759274 wait complete. 2024-06-10T01:17:08.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.908+0000 7f5172460b80 1 -- start start 2024-06-10T01:17:08.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.908+0000 7f5172460b80 1 Processor -- start 2024-06-10T01:17:08.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.909+0000 7f5172460b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a265783060 0x55a26578f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:08.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.909+0000 7f5172460b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a265788910 0x55a26578c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:08.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.909+0000 7f5172460b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a265789230 0x55a26578c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:08.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.909+0000 7f5172460b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a2656700c0 con 0x55a265789230 2024-06-10T01:17:08.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.909+0000 7f5172460b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a2656cb910 con 0x55a265783060 2024-06-10T01:17:08.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.909+0000 7f5172460b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a26578fc10 con 0x55a265788910 2024-06-10T01:17:08.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.909+0000 7f5172460b80 10 client.0 _ll_register_callbacks cb 0x55a2656656b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:17:08.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.909+0000 7f5165178700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a265788910 0x55a26578c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:08.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.909+0000 7f5165178700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a265788910 0x55a26578c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:35112/0 (socket says 172.21.15.7:35112) 2024-06-10T01:17:08.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.909+0000 7f5165979700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a265783060 0x55a26578f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:08.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.909+0000 7f5165178700 1 -- 172.21.15.7:0/1935997259 learned_addr learned my addr 172.21.15.7:0/1935997259 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:08.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.909+0000 7f516617a700 1 --2- 172.21.15.7:0/1935997259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a265789230 0x55a26578c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:08.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.909+0000 7f5165178700 1 -- 172.21.15.7:0/1935997259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a265783060 msgr2=0x55a26578f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:08.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.909+0000 7f5165178700 1 --2- 172.21.15.7:0/1935997259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a265783060 0x55a26578f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.909+0000 7f5165178700 1 -- 172.21.15.7:0/1935997259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a265789230 msgr2=0x55a26578c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:08.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.909+0000 7f5165178700 1 --2- 172.21.15.7:0/1935997259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a265789230 0x55a26578c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.909+0000 7f5165178700 1 -- 172.21.15.7:0/1935997259 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a265695850 con 0x55a265788910 2024-06-10T01:17:08.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.910+0000 7f5165979700 1 --2- 172.21.15.7:0/1935997259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a265783060 0x55a26578f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:17:08.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.910+0000 7f5165178700 1 --2- 172.21.15.7:0/1935997259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a265788910 0x55a26578c310 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5160107210 tx=0x7f515400b540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17c68a08f806dba4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:08.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.910+0000 7f51497fa700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:17:08.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.910+0000 7f51497fa700 1 -- 172.21.15.7:0/1935997259 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51540056b0 con 0x55a265788910 2024-06-10T01:17:08.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.910+0000 7f51497fa700 1 -- 172.21.15.7:0/1935997259 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f515400be00 con 0x55a265788910 2024-06-10T01:17:08.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.910+0000 7f5172460b80 1 -- 172.21.15.7:0/1935997259 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f515c0a3d80 con 0x55a265788910 2024-06-10T01:17:08.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.910+0000 7f51497fa700 1 -- 172.21.15.7:0/1935997259 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5154003e10 con 0x55a265788910 2024-06-10T01:17:08.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.910+0000 7f5172460b80 10 client.27610 fetch_fsmap learned FSMap version 40 2024-06-10T01:17:08.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.910+0000 7f5172460b80 1 -- 172.21.15.7:0/1935997259 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a26579a420 con 0x55a265788910 2024-06-10T01:17:08.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.910+0000 7f51497fa700 1 -- 172.21.15.7:0/1935997259 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f515c0a3d80 con 0x55a265788910 2024-06-10T01:17:08.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.911+0000 7f51497fa700 1 -- 172.21.15.7:0/1935997259 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f515402f5a0 con 0x55a265788910 2024-06-10T01:17:08.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.911+0000 7f5172460b80 10 client.27610 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:17:08.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.911+0000 7f5172460b80 10 client.27610 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:17:08.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.911+0000 7f5172460b80 20 client.27610 populate_metadata read hostname 'smithi007' 2024-06-10T01:17:08.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.911+0000 7f5172460b80 1 --2- 172.21.15.7:0/1935997259 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a26579cb00 0x55a26579eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:08.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.911+0000 7f5172460b80 4 client.27610 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:17:08.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.911+0000 7f5172460b80 1 -- 172.21.15.7:0/1935997259 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2656dce50 con 0x55a26579cb00 2024-06-10T01:17:08.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.911+0000 7f5165979700 1 --2- 172.21.15.7:0/1935997259 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a26579cb00 0x55a26579eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:08.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.911+0000 7f51497fa700 10 client.27610 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:08.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.911+0000 7f5165979700 1 --2- 172.21.15.7:0/1935997259 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a26579cb00 0x55a26579eef0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=830171601b7137ac server_cookie=62ed022fe0a29df2 in_seq=0 out_seq=0 2024-06-10T01:17:08.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.912+0000 7f51497fa700 1 -- 172.21.15.7:0/1935997259 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a2656dce50 con 0x55a26579cb00 2024-06-10T01:17:08.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.912+0000 7f51497fa700 10 client.27610 handle_command_reply: tid=0 2024-06-10T01:17:08.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.912+0000 7f5172460b80 1 client.27610 shutdown 2024-06-10T01:17:08.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.912+0000 7f5172460b80 1 -- 172.21.15.7:0/1935997259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a265788910 msgr2=0x55a26578c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:08.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.912+0000 7f5172460b80 1 --2- 172.21.15.7:0/1935997259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a265788910 0x55a26578c310 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5160107210 tx=0x7f515400b540 comp rx=0 tx=0).stop 2024-06-10T01:17:08.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.913+0000 7f5172460b80 1 -- 172.21.15.7:0/1935997259 shutdown_connections 2024-06-10T01:17:08.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.913+0000 7f5172460b80 1 --2- 172.21.15.7:0/1935997259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a265789230 0x55a26578c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.913+0000 7f5172460b80 1 --2- 172.21.15.7:0/1935997259 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a26579cb00 0x55a26579eef0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.913+0000 7f5172460b80 1 --2- 172.21.15.7:0/1935997259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a265783060 0x55a26578f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.913+0000 7f5172460b80 1 --2- 172.21.15.7:0/1935997259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a265788910 0x55a26578c310 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.913+0000 7f51497fa700 0 client.27610 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:08.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.913+0000 7f5172460b80 1 -- 172.21.15.7:0/1935997259 >> 172.21.15.7:0/1935997259 conn(0x55a2656ab180 msgr2=0x55a26577d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:08.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.913+0000 7f5172460b80 1 -- 172.21.15.7:0/1935997259 shutdown_connections 2024-06-10T01:17:08.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.913+0000 7f5172460b80 1 -- 172.21.15.7:0/1935997259 wait complete. 2024-06-10T01:17:08.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.913+0000 7f5172460b80 20 client.27610 trim_cache size 0 max 16384 2024-06-10T01:17:08.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.933+0000 7f5172460b80 1 Processor -- start 2024-06-10T01:17:08.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.933+0000 7f5172460b80 1 -- start start 2024-06-10T01:17:08.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.933+0000 7f5172460b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a265783060 0x55a265789ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:08.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.933+0000 7f5172460b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a265788910 0x55a265779a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:08.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.934+0000 7f5172460b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a265789230 0x55a265778a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:08.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.934+0000 7f5172460b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f51601ae9b0 con 0x55a265788910 2024-06-10T01:17:08.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.934+0000 7f5172460b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f51601ac240 con 0x55a265789230 2024-06-10T01:17:08.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.934+0000 7f5172460b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f51601aa0b0 con 0x55a265783060 2024-06-10T01:17:08.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.934+0000 7f5165178700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a265788910 0x55a265779a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:08.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.934+0000 7f5165178700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a265788910 0x55a265779a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38674/0 (socket says 172.21.15.7:38674) 2024-06-10T01:17:08.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.934+0000 7f5165178700 1 -- 172.21.15.7:0/738170982 learned_addr learned my addr 172.21.15.7:0/738170982 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:08.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.934+0000 7f5165979700 1 --2- 172.21.15.7:0/738170982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a265783060 0x55a265789ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:08.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.934+0000 7f516617a700 1 --2- 172.21.15.7:0/738170982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a265789230 0x55a265778a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:08.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.934+0000 7f5165178700 1 -- 172.21.15.7:0/738170982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a265783060 msgr2=0x55a265789ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:08.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.934+0000 7f5165178700 1 --2- 172.21.15.7:0/738170982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a265783060 0x55a265789ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.934+0000 7f5165178700 1 -- 172.21.15.7:0/738170982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a265789230 msgr2=0x55a265778a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:08.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.934+0000 7f5165178700 1 --2- 172.21.15.7:0/738170982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a265789230 0x55a265778a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.934+0000 7f5165178700 1 -- 172.21.15.7:0/738170982 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a26579f550 con 0x55a265788910 2024-06-10T01:17:08.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.934+0000 7f516617a700 1 --2- 172.21.15.7:0/738170982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a265789230 0x55a265778a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:17:08.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.934+0000 7f5165178700 1 --2- 172.21.15.7:0/738170982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a265788910 0x55a265779a80 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f5154003fd0 tx=0x7f5154003850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d7abab53306f3fc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:08.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.935+0000 7f514b7fe700 1 -- 172.21.15.7:0/738170982 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515400be00 con 0x55a265788910 2024-06-10T01:17:08.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.935+0000 7f514b7fe700 1 -- 172.21.15.7:0/738170982 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f515400f460 con 0x55a265788910 2024-06-10T01:17:08.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.935+0000 7f514b7fe700 1 -- 172.21.15.7:0/738170982 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5154024680 con 0x55a265788910 2024-06-10T01:17:08.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.935+0000 7f5172460b80 1 -- 172.21.15.7:0/738170982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a265788910 msgr2=0x55a265779a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:08.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.935+0000 7f5172460b80 1 --2- 172.21.15.7:0/738170982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a265788910 0x55a265779a80 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f5154003fd0 tx=0x7f5154003850 comp rx=0 tx=0).stop 2024-06-10T01:17:08.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.935+0000 7f5172460b80 1 -- 172.21.15.7:0/738170982 shutdown_connections 2024-06-10T01:17:08.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.935+0000 7f5172460b80 1 --2- 172.21.15.7:0/738170982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a265789230 0x55a265778a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.935+0000 7f5172460b80 1 --2- 172.21.15.7:0/738170982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a265783060 0x55a265789ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.935+0000 7f5172460b80 1 --2- 172.21.15.7:0/738170982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a265788910 0x55a265779a80 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.935+0000 7f5172460b80 1 -- 172.21.15.7:0/738170982 >> 172.21.15.7:0/738170982 conn(0x55a2656ab180 msgr2=0x55a265778320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:08.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.936+0000 7f5172460b80 1 -- 172.21.15.7:0/738170982 shutdown_connections 2024-06-10T01:17:08.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.936+0000 7f5172460b80 1 -- 172.21.15.7:0/738170982 wait complete. 2024-06-10T01:17:08.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.936+0000 7f5172460b80 1 -- start start 2024-06-10T01:17:08.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.936+0000 7f5172460b80 1 Processor -- start 2024-06-10T01:17:08.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.936+0000 7f5172460b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a265783060 0x55a2657802e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:08.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.936+0000 7f5172460b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a265788910 0x55a265780820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:08.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.936+0000 7f5172460b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a265789230 0x55a265792f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:08.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.936+0000 7f5172460b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f51601aa700 con 0x55a265788910 2024-06-10T01:17:08.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.936+0000 7f5172460b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f51601abb70 con 0x55a265789230 2024-06-10T01:17:08.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.936+0000 7f5172460b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f51601aadd0 con 0x55a265783060 2024-06-10T01:17:08.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.937+0000 7f5172460b80 10 client.0 _ll_register_callbacks cb 0x55a2656656b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:17:08.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.937+0000 7f5165178700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a265788910 0x55a265780820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:08.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.937+0000 7f5165979700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a265783060 0x55a2657802e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:08.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.937+0000 7f5165178700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a265788910 0x55a265780820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38690/0 (socket says 172.21.15.7:38690) 2024-06-10T01:17:08.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.937+0000 7f5165979700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a265783060 0x55a2657802e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:35120/0 (socket says 172.21.15.7:35120) 2024-06-10T01:17:08.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.937+0000 7f5165178700 1 -- 172.21.15.7:0/3481854126 learned_addr learned my addr 172.21.15.7:0/3481854126 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:08.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.937+0000 7f516617a700 1 --2- 172.21.15.7:0/3481854126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a265789230 0x55a265792f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:08.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.937+0000 7f5165979700 1 -- 172.21.15.7:0/3481854126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a265789230 msgr2=0x55a265792f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:08.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.937+0000 7f5165979700 1 --2- 172.21.15.7:0/3481854126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a265789230 0x55a265792f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.937+0000 7f5165979700 1 -- 172.21.15.7:0/3481854126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a265788910 msgr2=0x55a265780820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:08.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.937+0000 7f5165979700 1 --2- 172.21.15.7:0/3481854126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a265788910 0x55a265780820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.937+0000 7f5165979700 1 -- 172.21.15.7:0/3481854126 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2657714a0 con 0x55a265783060 2024-06-10T01:17:08.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.937+0000 7f5165979700 1 --2- 172.21.15.7:0/3481854126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a265783060 0x55a2657802e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f515c0cc720 tx=0x7f515c0cee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=207914e858b367ab server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:08.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.937+0000 7f51497fa700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:17:08.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.938+0000 7f51497fa700 1 -- 172.21.15.7:0/3481854126 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515c010730 con 0x55a265783060 2024-06-10T01:17:08.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.938+0000 7f51497fa700 1 -- 172.21.15.7:0/3481854126 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f515c010380 con 0x55a265783060 2024-06-10T01:17:08.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.938+0000 7f5172460b80 1 -- 172.21.15.7:0/3481854126 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f515400f460 con 0x55a265783060 2024-06-10T01:17:08.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.938+0000 7f51497fa700 1 -- 172.21.15.7:0/3481854126 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f515c099410 con 0x55a265783060 2024-06-10T01:17:08.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.938+0000 7f5172460b80 10 client.27616 fetch_fsmap learned FSMap version 40 2024-06-10T01:17:08.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.938+0000 7f5172460b80 1 -- 172.21.15.7:0/3481854126 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a265794670 con 0x55a265783060 2024-06-10T01:17:08.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.938+0000 7f51497fa700 1 -- 172.21.15.7:0/3481854126 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f515400f460 con 0x55a265783060 2024-06-10T01:17:08.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.938+0000 7f51497fa700 1 -- 172.21.15.7:0/3481854126 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f515c09b370 con 0x55a265783060 2024-06-10T01:17:08.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.938+0000 7f5172460b80 10 client.27616 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:17:08.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.938+0000 7f5172460b80 10 client.27616 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:17:08.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.938+0000 7f5172460b80 20 client.27616 populate_metadata read hostname 'smithi007' 2024-06-10T01:17:08.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.938+0000 7f5172460b80 1 --2- 172.21.15.7:0/3481854126 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a265794900 0x55a265794ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:08.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.938+0000 7f5172460b80 4 client.27616 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "af0b3e22", "release": true, "await": true}] 2024-06-10T01:17:08.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.938+0000 7f5172460b80 1 -- 172.21.15.7:0/3481854126 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "set_id": "af0b3e22", "release": true, "await": true}) -- 0x55a2656d73d0 con 0x55a265794900 2024-06-10T01:17:08.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.939+0000 7f5165178700 1 --2- 172.21.15.7:0/3481854126 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a265794900 0x55a265794ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:08.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.939+0000 7f5165178700 1 --2- 172.21.15.7:0/3481854126 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a265794900 0x55a265794ce0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b830698a0402ea03 server_cookie=74de2a73f7f361a7 in_seq=0 out_seq=0 2024-06-10T01:17:08.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.939+0000 7f51497fa700 10 client.27616 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:08.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.947+0000 7f51497fa700 1 -- 172.21.15.7:0/3481854126 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a2656d73d0 con 0x55a265794900 2024-06-10T01:17:08.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.947+0000 7f51497fa700 10 client.27616 handle_command_reply: tid=0 2024-06-10T01:17:08.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.947+0000 7f5172460b80 1 client.27616 shutdown 2024-06-10T01:17:08.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.947+0000 7f5172460b80 1 -- 172.21.15.7:0/3481854126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a265783060 msgr2=0x55a2657802e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:08.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.947+0000 7f5172460b80 1 --2- 172.21.15.7:0/3481854126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a265783060 0x55a2657802e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f515c0cc720 tx=0x7f515c0cee30 comp rx=0 tx=0).stop 2024-06-10T01:17:08.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.947+0000 7f5172460b80 1 -- 172.21.15.7:0/3481854126 shutdown_connections 2024-06-10T01:17:08.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.947+0000 7f5172460b80 1 --2- 172.21.15.7:0/3481854126 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a265794900 0x55a265794ce0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.947+0000 7f5172460b80 1 --2- 172.21.15.7:0/3481854126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a265789230 0x55a265792f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.947+0000 7f5172460b80 1 --2- 172.21.15.7:0/3481854126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a265783060 0x55a2657802e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.947+0000 7f51497fa700 0 client.27616 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:08.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.947+0000 7f5172460b80 1 --2- 172.21.15.7:0/3481854126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a265788910 0x55a265780820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.947+0000 7f5172460b80 1 -- 172.21.15.7:0/3481854126 >> 172.21.15.7:0/3481854126 conn(0x55a2656ab180 msgr2=0x55a265778320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:08.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.947+0000 7f5172460b80 1 -- 172.21.15.7:0/3481854126 shutdown_connections 2024-06-10T01:17:08.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.947+0000 7f5172460b80 1 -- 172.21.15.7:0/3481854126 wait complete. 2024-06-10T01:17:08.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.947+0000 7f5172460b80 20 client.27616 trim_cache size 0 max 16384 2024-06-10T01:17:08.943 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:17:08.943 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 40, 2024-06-10T01:17:08.943 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:17:08.943 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 36, 2024-06-10T01:17:08.943 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:17:08.943 INFO:teuthology.orchestra.run.smithi007.stdout: "af0b3e22": { 2024-06-10T01:17:08.943 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 36, 2024-06-10T01:17:08.943 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:17:08.943 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:17:08.943 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:17:08.944 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:17:08.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:17:08.944 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:17:08.944 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 135.8, 2024-06-10T01:17:08.944 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:17:08.944 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:17:08.944 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:17:08.944 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:17:08.944 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:17:08.944 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:17:08.944 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:17:08.944 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:17:08.944 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:17:08.944 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:17:08.945 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:17:08.945 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:17:08.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.948+0000 7f5167bdd700 1 -- 172.21.15.7:0/1456763428 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f514c0a2ca0 msgr2=0x7f514c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:08.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.948+0000 7f5167bdd700 1 --2- 172.21.15.7:0/1456763428 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f514c0a2ca0 0x7f514c0a5160 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f51540098e0 tx=0x7f5154000bc0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.948+0000 7f5167bdd700 1 -- 172.21.15.7:0/1456763428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51601027a0 msgr2=0x7f516019e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:08.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.948+0000 7f5167bdd700 1 --2- 172.21.15.7:0/1456763428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51601027a0 0x7f516019e820 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f515c00b530 tx=0x7f515c00b840 comp rx=0 tx=0).stop 2024-06-10T01:17:08.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.948+0000 7f5167bdd700 1 -- 172.21.15.7:0/1456763428 shutdown_connections 2024-06-10T01:17:08.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.948+0000 7f5167bdd700 1 --2- 172.21.15.7:0/1456763428 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f514c0a2ca0 0x7f514c0a5160 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.948+0000 7f5167bdd700 1 --2- 172.21.15.7:0/1456763428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5160107870 0x7f51601989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.948+0000 7f5167bdd700 1 --2- 172.21.15.7:0/1456763428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51601027a0 0x7f516019e820 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.948+0000 7f5167bdd700 1 --2- 172.21.15.7:0/1456763428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5160103150 0x7f516019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:08.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.948+0000 7f5167bdd700 1 -- 172.21.15.7:0/1456763428 >> 172.21.15.7:0/1456763428 conn(0x7f5160079520 msgr2=0x7f5160101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:08.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.949+0000 7f5167bdd700 1 -- 172.21.15.7:0/1456763428 shutdown_connections 2024-06-10T01:17:08.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:08.949+0000 7f5167bdd700 1 -- 172.21.15.7:0/1456763428 wait complete. 2024-06-10T01:17:09.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:08 smithi175 ceph-mon[36636]: pgmap v766: 129 pgs: 129 active+clean; 139 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 1.5 MiB/s wr, 86 op/s 2024-06-10T01:17:09.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:08 smithi007 ceph-mon[32398]: pgmap v766: 129 pgs: 129 active+clean; 139 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 1.5 MiB/s wr, 86 op/s 2024-06-10T01:17:09.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:08 smithi078 ceph-mon[36683]: pgmap v766: 129 pgs: 129 active+clean; 139 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 1.5 MiB/s wr, 86 op/s 2024-06-10T01:17:09.435 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:17:09.751 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'af0b3e22', total seconds elapsed: 21.9 2024-06-10T01:17:09.751 INFO:tasks.workunit.client.0.smithi007.stdout: 1 816844 7.41 MB/sec execute 470 sec latency 14931.563 ms 2024-06-10T01:17:09.791 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:17:10.751 INFO:tasks.workunit.client.0.smithi007.stdout: 1 818288 7.40 MB/sec execute 471 sec latency 49.400 ms 2024-06-10T01:17:11.012 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:10 smithi007 ceph-mon[32398]: pgmap v767: 129 pgs: 129 active+clean; 140 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 1.7 MiB/s wr, 96 op/s 2024-06-10T01:17:11.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:10 smithi175 ceph-mon[36636]: pgmap v767: 129 pgs: 129 active+clean; 140 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 1.7 MiB/s wr, 96 op/s 2024-06-10T01:17:11.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:10 smithi078 ceph-mon[36683]: pgmap v767: 129 pgs: 129 active+clean; 140 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 1.7 MiB/s wr, 96 op/s 2024-06-10T01:17:11.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.756+0000 7fc091a0c700 1 -- 172.21.15.7:0/205843139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc08c1084f0 msgr2=0x7fc08c1088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:11.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.756+0000 7fc091a0c700 1 --2- 172.21.15.7:0/205843139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc08c1084f0 0x7fc08c1088d0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009990 tx=0x7fc07c009ca0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.757+0000 7fc091a0c700 1 -- 172.21.15.7:0/205843139 shutdown_connections 2024-06-10T01:17:11.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.757+0000 7fc091a0c700 1 --2- 172.21.15.7:0/205843139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc08c06b3c0 0x7fc08c10f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.757+0000 7fc091a0c700 1 --2- 172.21.15.7:0/205843139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc08c1084f0 0x7fc08c1088d0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.757+0000 7fc091a0c700 1 --2- 172.21.15.7:0/205843139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc08c06aa00 0x7fc08c06ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.757+0000 7fc091a0c700 1 -- 172.21.15.7:0/205843139 >> 172.21.15.7:0/205843139 conn(0x7fc08c078300 msgr2=0x7fc08c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:11.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.757+0000 7fc091a0c700 1 -- 172.21.15.7:0/205843139 shutdown_connections 2024-06-10T01:17:11.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.757+0000 7fc091a0c700 1 -- 172.21.15.7:0/205843139 wait complete. 2024-06-10T01:17:11.752 INFO:tasks.workunit.client.0.smithi007.stdout: 1 820427 7.42 MB/sec execute 472 sec latency 63.283 ms 2024-06-10T01:17:11.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.757+0000 7fc091a0c700 1 Processor -- start 2024-06-10T01:17:11.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.757+0000 7fc091a0c700 1 -- start start 2024-06-10T01:17:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.758+0000 7fc091a0c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc08c06aa00 0x7fc08c19e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.758+0000 7fc091a0c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc08c06b3c0 0x7fc08c19ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.758+0000 7fc091a0c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc08c1084f0 0x7fc08c198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.758+0000 7fc091a0c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c112750 con 0x7fc08c1084f0 2024-06-10T01:17:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.758+0000 7fc091a0c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c1125d0 con 0x7fc08c06aa00 2024-06-10T01:17:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.758+0000 7fc091a0c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c1128d0 con 0x7fc08c06b3c0 2024-06-10T01:17:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.758+0000 7fc08a7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc08c06b3c0 0x7fc08c19ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.758+0000 7fc08b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc08c1084f0 0x7fc08c198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.758+0000 7fc08a7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc08c06b3c0 0x7fc08c19ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:35136/0 (socket says 172.21.15.7:35136) 2024-06-10T01:17:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.758+0000 7fc08b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc08c1084f0 0x7fc08c198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38714/0 (socket says 172.21.15.7:38714) 2024-06-10T01:17:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.758+0000 7fc08a7fc700 1 -- 172.21.15.7:0/3838106260 learned_addr learned my addr 172.21.15.7:0/3838106260 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.758+0000 7fc08affd700 1 --2- 172.21.15.7:0/3838106260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc08c06aa00 0x7fc08c19e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.758+0000 7fc08a7fc700 1 -- 172.21.15.7:0/3838106260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc08c06aa00 msgr2=0x7fc08c19e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.758+0000 7fc08a7fc700 1 --2- 172.21.15.7:0/3838106260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc08c06aa00 0x7fc08c19e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.758+0000 7fc08a7fc700 1 -- 172.21.15.7:0/3838106260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc08c1084f0 msgr2=0x7fc08c198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.758+0000 7fc08a7fc700 1 --2- 172.21.15.7:0/3838106260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc08c1084f0 0x7fc08c198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.758+0000 7fc08a7fc700 1 -- 172.21.15.7:0/3838106260 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc08c199210 con 0x7fc08c06b3c0 2024-06-10T01:17:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.758+0000 7fc08b7fe700 1 --2- 172.21.15.7:0/3838106260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc08c1084f0 0x7fc08c198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:17:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.758+0000 7fc08affd700 1 --2- 172.21.15.7:0/3838106260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc08c06aa00 0x7fc08c19e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:17:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.758+0000 7fc08a7fc700 1 --2- 172.21.15.7:0/3838106260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc08c06b3c0 0x7fc08c19ee30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc08000e9a0 tx=0x7fc08000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.759+0000 7fc090a0a700 1 -- 172.21.15.7:0/3838106260 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08000cbd0 con 0x7fc08c06b3c0 2024-06-10T01:17:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.759+0000 7fc091a0c700 1 -- 172.21.15.7:0/3838106260 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc08c199500 con 0x7fc08c06b3c0 2024-06-10T01:17:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.759+0000 7fc090a0a700 1 -- 172.21.15.7:0/3838106260 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc08000cd70 con 0x7fc08c06b3c0 2024-06-10T01:17:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.759+0000 7fc090a0a700 1 -- 172.21.15.7:0/3838106260 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc080018750 con 0x7fc08c06b3c0 2024-06-10T01:17:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.759+0000 7fc091a0c700 1 -- 172.21.15.7:0/3838106260 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc08c1a57c0 con 0x7fc08c06b3c0 2024-06-10T01:17:11.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.761+0000 7fc090a0a700 1 -- 172.21.15.7:0/3838106260 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fc080010430 con 0x7fc08c06b3c0 2024-06-10T01:17:11.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.762+0000 7fc090a0a700 1 --2- 172.21.15.7:0/3838106260 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc0740a2c20 0x7fc0740a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:11.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.762+0000 7fc08affd700 1 --2- 172.21.15.7:0/3838106260 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc0740a2c20 0x7fc0740a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:11.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.762+0000 7fc090a0a700 1 -- 172.21.15.7:0/3838106260 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc0800cf700 con 0x7fc08c06b3c0 2024-06-10T01:17:11.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.762+0000 7fc08affd700 1 --2- 172.21.15.7:0/3838106260 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc0740a2c20 0x7fc0740a50e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc07c006010 tx=0x7fc07c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:11.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.762+0000 7fc09c28fb80 1 Processor -- start 2024-06-10T01:17:11.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.762+0000 7fc09c28fb80 1 -- start start 2024-06-10T01:17:11.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.763+0000 7fc09c28fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55844f6ffa70 0x55844f6ffe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:11.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.763+0000 7fc09c28fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55844f700390 0x55844f6f9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:11.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.763+0000 7fc09c28fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55844f6fa330 0x55844f6fa750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:11.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.763+0000 7fc09c28fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55844f647810 con 0x55844f700390 2024-06-10T01:17:11.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.763+0000 7fc09c28fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55844f5ef720 con 0x55844f6ffa70 2024-06-10T01:17:11.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.763+0000 7fc09c28fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55844f6437d0 con 0x55844f6fa330 2024-06-10T01:17:11.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.763+0000 7fc08a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55844f700390 0x55844f6f9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:11.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.763+0000 7fc08a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55844f700390 0x55844f6f9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38718/0 (socket says 172.21.15.7:38718) 2024-06-10T01:17:11.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.763+0000 7fc08a7fc700 1 -- 172.21.15.7:0/3814025089 learned_addr learned my addr 172.21.15.7:0/3814025089 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:11.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.763+0000 7fc08b7fe700 1 --2- 172.21.15.7:0/3814025089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55844f6fa330 0x55844f6fa750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:11.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.763+0000 7fc08affd700 1 --2- 172.21.15.7:0/3814025089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55844f6ffa70 0x55844f6ffe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:11.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.763+0000 7fc08a7fc700 1 -- 172.21.15.7:0/3814025089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55844f6fa330 msgr2=0x55844f6fa750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:11.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.763+0000 7fc08a7fc700 1 --2- 172.21.15.7:0/3814025089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55844f6fa330 0x55844f6fa750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.763+0000 7fc08a7fc700 1 -- 172.21.15.7:0/3814025089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55844f6ffa70 msgr2=0x55844f6ffe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:11.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.763+0000 7fc08a7fc700 1 --2- 172.21.15.7:0/3814025089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55844f6ffa70 0x55844f6ffe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.763+0000 7fc08a7fc700 1 -- 172.21.15.7:0/3814025089 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55844f5e5f50 con 0x55844f700390 2024-06-10T01:17:11.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.763+0000 7fc08a7fc700 1 --2- 172.21.15.7:0/3814025089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55844f700390 0x55844f6f9df0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fc0800a42c0 tx=0x7fc08009b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1535804682481e2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:11.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.764+0000 7fc071ffb700 1 -- 172.21.15.7:0/3814025089 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08009c280 con 0x55844f700390 2024-06-10T01:17:11.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.764+0000 7fc071ffb700 1 -- 172.21.15.7:0/3814025089 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc08009bb80 con 0x55844f700390 2024-06-10T01:17:11.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.764+0000 7fc071ffb700 1 -- 172.21.15.7:0/3814025089 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0800cf3c0 con 0x55844f700390 2024-06-10T01:17:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.764+0000 7fc09c28fb80 1 -- 172.21.15.7:0/3814025089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55844f700390 msgr2=0x55844f6f9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.764+0000 7fc09c28fb80 1 --2- 172.21.15.7:0/3814025089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55844f700390 0x55844f6f9df0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fc0800a42c0 tx=0x7fc08009b740 comp rx=0 tx=0).stop 2024-06-10T01:17:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.765+0000 7fc09c28fb80 1 -- 172.21.15.7:0/3814025089 shutdown_connections 2024-06-10T01:17:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.765+0000 7fc09c28fb80 1 --2- 172.21.15.7:0/3814025089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55844f6fa330 0x55844f6fa750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.765+0000 7fc09c28fb80 1 --2- 172.21.15.7:0/3814025089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55844f6ffa70 0x55844f6ffe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.765+0000 7fc09c28fb80 1 --2- 172.21.15.7:0/3814025089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55844f700390 0x55844f6f9df0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.765+0000 7fc09c28fb80 1 -- 172.21.15.7:0/3814025089 >> 172.21.15.7:0/3814025089 conn(0x55844f620cc0 msgr2=0x55844f6f0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.765+0000 7fc09c28fb80 1 -- 172.21.15.7:0/3814025089 shutdown_connections 2024-06-10T01:17:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.765+0000 7fc09c28fb80 1 -- 172.21.15.7:0/3814025089 wait complete. 2024-06-10T01:17:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.765+0000 7fc09c28fb80 1 -- start start 2024-06-10T01:17:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.765+0000 7fc09c28fb80 1 Processor -- start 2024-06-10T01:17:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.765+0000 7fc09c28fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55844f6fa330 0x55844f706100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.765+0000 7fc09c28fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55844f6ffa70 0x55844f703400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.765+0000 7fc09c28fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55844f700390 0x55844f703940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.766+0000 7fc09c28fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55844f706dd0 con 0x55844f6fa330 2024-06-10T01:17:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.766+0000 7fc09c28fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55844f5e7bf0 con 0x55844f6ffa70 2024-06-10T01:17:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.766+0000 7fc09c28fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55844f706f50 con 0x55844f700390 2024-06-10T01:17:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.766+0000 7fc09c28fb80 10 client.0 _ll_register_callbacks cb 0x55844f5e1920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:17:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.766+0000 7fc08affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55844f6fa330 0x55844f706100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:11.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.766+0000 7fc08affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55844f6fa330 0x55844f706100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38724/0 (socket says 172.21.15.7:38724) 2024-06-10T01:17:11.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.766+0000 7fc08affd700 1 -- 172.21.15.7:0/4175885099 learned_addr learned my addr 172.21.15.7:0/4175885099 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:11.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.766+0000 7fc08b7fe700 1 --2- 172.21.15.7:0/4175885099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55844f700390 0x55844f703940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:11.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.766+0000 7fc08a7fc700 1 --2- 172.21.15.7:0/4175885099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55844f6ffa70 0x55844f703400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:11.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.766+0000 7fc08b7fe700 1 -- 172.21.15.7:0/4175885099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55844f6ffa70 msgr2=0x55844f703400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:11.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.766+0000 7fc08b7fe700 1 --2- 172.21.15.7:0/4175885099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55844f6ffa70 0x55844f703400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.766+0000 7fc08b7fe700 1 -- 172.21.15.7:0/4175885099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55844f6fa330 msgr2=0x55844f706100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:11.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.766+0000 7fc08b7fe700 1 --2- 172.21.15.7:0/4175885099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55844f6fa330 0x55844f706100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.766+0000 7fc08b7fe700 1 -- 172.21.15.7:0/4175885099 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55844f5968d0 con 0x55844f700390 2024-06-10T01:17:11.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.766+0000 7fc08b7fe700 1 --2- 172.21.15.7:0/4175885099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55844f700390 0x55844f703940 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc084004660 tx=0x7fc084004970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c30021b26f4b2bf6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:11.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.766+0000 7fc06affd700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:17:11.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.766+0000 7fc06affd700 1 -- 172.21.15.7:0/4175885099 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08400ec20 con 0x55844f700390 2024-06-10T01:17:11.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.766+0000 7fc06affd700 1 -- 172.21.15.7:0/4175885099 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc08400edc0 con 0x55844f700390 2024-06-10T01:17:11.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.766+0000 7fc09c28fb80 1 -- 172.21.15.7:0/4175885099 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc08009bb80 con 0x55844f700390 2024-06-10T01:17:11.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.767+0000 7fc06affd700 1 -- 172.21.15.7:0/4175885099 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc084027050 con 0x55844f700390 2024-06-10T01:17:11.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.767+0000 7fc09c28fb80 10 client.27628 fetch_fsmap learned FSMap version 40 2024-06-10T01:17:11.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.767+0000 7fc09c28fb80 1 -- 172.21.15.7:0/4175885099 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55844f711730 con 0x55844f700390 2024-06-10T01:17:11.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.767+0000 7fc06affd700 1 -- 172.21.15.7:0/4175885099 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc08009bb80 con 0x55844f700390 2024-06-10T01:17:11.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.767+0000 7fc06affd700 1 -- 172.21.15.7:0/4175885099 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7fc08401e400 con 0x55844f700390 2024-06-10T01:17:11.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.767+0000 7fc09c28fb80 10 client.27628 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:17:11.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.767+0000 7fc09c28fb80 10 client.27628 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:17:11.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.767+0000 7fc09c28fb80 20 client.27628 populate_metadata read hostname 'smithi007' 2024-06-10T01:17:11.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.767+0000 7fc09c28fb80 1 --2- 172.21.15.7:0/4175885099 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55844f713d70 0x55844f716160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:11.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.767+0000 7fc09c28fb80 4 client.27628 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:17:11.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.767+0000 7fc09c28fb80 1 -- 172.21.15.7:0/4175885099 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55844f654d10 con 0x55844f713d70 2024-06-10T01:17:11.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.767+0000 7fc08affd700 1 --2- 172.21.15.7:0/4175885099 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55844f713d70 0x55844f716160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:11.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.768+0000 7fc08affd700 1 --2- 172.21.15.7:0/4175885099 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55844f713d70 0x55844f716160 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60af0170005465cb server_cookie=77dd6fb9f6303cf6 in_seq=0 out_seq=0 2024-06-10T01:17:11.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.768+0000 7fc06affd700 10 client.27628 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:11.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.769+0000 7fc06affd700 1 -- 172.21.15.7:0/4175885099 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55844f654d10 con 0x55844f713d70 2024-06-10T01:17:11.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.769+0000 7fc06affd700 10 client.27628 handle_command_reply: tid=0 2024-06-10T01:17:11.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.769+0000 7fc09c28fb80 1 client.27628 shutdown 2024-06-10T01:17:11.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.769+0000 7fc09c28fb80 1 -- 172.21.15.7:0/4175885099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55844f700390 msgr2=0x55844f703940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:11.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.769+0000 7fc09c28fb80 1 --2- 172.21.15.7:0/4175885099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55844f700390 0x55844f703940 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc084004660 tx=0x7fc084004970 comp rx=0 tx=0).stop 2024-06-10T01:17:11.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.769+0000 7fc09c28fb80 1 -- 172.21.15.7:0/4175885099 shutdown_connections 2024-06-10T01:17:11.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.769+0000 7fc09c28fb80 1 --2- 172.21.15.7:0/4175885099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55844f700390 0x55844f703940 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.769+0000 7fc09c28fb80 1 --2- 172.21.15.7:0/4175885099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55844f6fa330 0x55844f706100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.769+0000 7fc09c28fb80 1 --2- 172.21.15.7:0/4175885099 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55844f713d70 0x55844f716160 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.769+0000 7fc09c28fb80 1 --2- 172.21.15.7:0/4175885099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55844f6ffa70 0x55844f703400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.769+0000 7fc06affd700 0 client.27628 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:11.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.769+0000 7fc09c28fb80 1 -- 172.21.15.7:0/4175885099 >> 172.21.15.7:0/4175885099 conn(0x55844f620cc0 msgr2=0x55844f6f49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:11.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.769+0000 7fc09c28fb80 1 -- 172.21.15.7:0/4175885099 shutdown_connections 2024-06-10T01:17:11.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.769+0000 7fc09c28fb80 1 -- 172.21.15.7:0/4175885099 wait complete. 2024-06-10T01:17:11.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.769+0000 7fc09c28fb80 20 client.27628 trim_cache size 0 max 16384 2024-06-10T01:17:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.790+0000 7fc09c28fb80 1 Processor -- start 2024-06-10T01:17:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.790+0000 7fc09c28fb80 1 -- start start 2024-06-10T01:17:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.791+0000 7fc09c28fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55844f6fa330 0x55844f6f06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.791+0000 7fc09c28fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55844f6ffa70 0x55844f700f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.791+0000 7fc09c28fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55844f700390 0x55844f71ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.791+0000 7fc09c28fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c1aea30 con 0x55844f700390 2024-06-10T01:17:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.791+0000 7fc09c28fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c1ac2c0 con 0x55844f6ffa70 2024-06-10T01:17:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.791+0000 7fc09c28fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c1aa110 con 0x55844f6fa330 2024-06-10T01:17:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.791+0000 7fc08b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55844f700390 0x55844f71ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.791+0000 7fc08affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55844f6fa330 0x55844f6f06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.791+0000 7fc08b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55844f700390 0x55844f71ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37472/0 (socket says 172.21.15.7:37472) 2024-06-10T01:17:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.791+0000 7fc08b7fe700 1 -- 172.21.15.7:0/3050708779 learned_addr learned my addr 172.21.15.7:0/3050708779 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.791+0000 7fc08a7fc700 1 --2- 172.21.15.7:0/3050708779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55844f6ffa70 0x55844f700f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.791+0000 7fc08b7fe700 1 -- 172.21.15.7:0/3050708779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55844f6fa330 msgr2=0x55844f6f06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.791+0000 7fc08b7fe700 1 --2- 172.21.15.7:0/3050708779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55844f6fa330 0x55844f6f06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.791+0000 7fc08b7fe700 1 -- 172.21.15.7:0/3050708779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55844f6ffa70 msgr2=0x55844f700f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:11.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.791+0000 7fc08b7fe700 1 --2- 172.21.15.7:0/3050708779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55844f6ffa70 0x55844f700f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.791+0000 7fc08b7fe700 1 -- 172.21.15.7:0/3050708779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55844f7049a0 con 0x55844f700390 2024-06-10T01:17:11.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.791+0000 7fc08a7fc700 1 --2- 172.21.15.7:0/3050708779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55844f6ffa70 0x55844f700f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:17:11.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.791+0000 7fc08b7fe700 1 --2- 172.21.15.7:0/3050708779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55844f700390 0x55844f71ba30 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fc084000f80 tx=0x7fc084003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dcca5a375ec44a2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:11.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.792+0000 7fc0717fa700 1 -- 172.21.15.7:0/3050708779 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08402f030 con 0x55844f700390 2024-06-10T01:17:11.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.792+0000 7fc0717fa700 1 -- 172.21.15.7:0/3050708779 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc08401e400 con 0x55844f700390 2024-06-10T01:17:11.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.792+0000 7fc0717fa700 1 -- 172.21.15.7:0/3050708779 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08400eb10 con 0x55844f700390 2024-06-10T01:17:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.792+0000 7fc09c28fb80 1 -- 172.21.15.7:0/3050708779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55844f700390 msgr2=0x55844f71ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.792+0000 7fc09c28fb80 1 --2- 172.21.15.7:0/3050708779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55844f700390 0x55844f71ba30 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fc084000f80 tx=0x7fc084003f60 comp rx=0 tx=0).stop 2024-06-10T01:17:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.792+0000 7fc09c28fb80 1 -- 172.21.15.7:0/3050708779 shutdown_connections 2024-06-10T01:17:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.792+0000 7fc09c28fb80 1 --2- 172.21.15.7:0/3050708779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55844f700390 0x55844f71ba30 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.792+0000 7fc09c28fb80 1 --2- 172.21.15.7:0/3050708779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55844f6fa330 0x55844f6f06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.792+0000 7fc09c28fb80 1 --2- 172.21.15.7:0/3050708779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55844f6ffa70 0x55844f700f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.792+0000 7fc09c28fb80 1 -- 172.21.15.7:0/3050708779 >> 172.21.15.7:0/3050708779 conn(0x55844f620cc0 msgr2=0x55844f700830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.792+0000 7fc09c28fb80 1 -- 172.21.15.7:0/3050708779 shutdown_connections 2024-06-10T01:17:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.792+0000 7fc09c28fb80 1 -- 172.21.15.7:0/3050708779 wait complete. 2024-06-10T01:17:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.793+0000 7fc09c28fb80 1 -- start start 2024-06-10T01:17:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.793+0000 7fc09c28fb80 1 Processor -- start 2024-06-10T01:17:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.793+0000 7fc09c28fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55844f6fa330 0x55844f70c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.793+0000 7fc09c28fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55844f6ffa70 0x55844f70c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.793+0000 7fc09c28fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55844f700390 0x55844f7091b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.793+0000 7fc09c28fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c1aa780 con 0x55844f6fa330 2024-06-10T01:17:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.793+0000 7fc09c28fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c1abbf0 con 0x55844f700390 2024-06-10T01:17:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.793+0000 7fc09c28fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c1aae50 con 0x55844f6ffa70 2024-06-10T01:17:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.793+0000 7fc09c28fb80 10 client.0 _ll_register_callbacks cb 0x55844f5e1920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:17:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.793+0000 7fc08affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55844f6fa330 0x55844f70c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.793+0000 7fc08affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55844f6fa330 0x55844f70c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37476/0 (socket says 172.21.15.7:37476) 2024-06-10T01:17:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.793+0000 7fc08affd700 1 -- 172.21.15.7:0/1171305420 learned_addr learned my addr 172.21.15.7:0/1171305420 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.793+0000 7fc08a7fc700 1 --2- 172.21.15.7:0/1171305420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55844f6ffa70 0x55844f70c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.794+0000 7fc08b7fe700 1 --2- 172.21.15.7:0/1171305420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55844f700390 0x55844f7091b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.794+0000 7fc08affd700 1 -- 172.21.15.7:0/1171305420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55844f6ffa70 msgr2=0x55844f70c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.794+0000 7fc08affd700 1 --2- 172.21.15.7:0/1171305420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55844f6ffa70 0x55844f70c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.794+0000 7fc08affd700 1 -- 172.21.15.7:0/1171305420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55844f700390 msgr2=0x55844f7091b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.794+0000 7fc08affd700 1 --2- 172.21.15.7:0/1171305420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55844f700390 0x55844f7091b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.794+0000 7fc08affd700 1 -- 172.21.15.7:0/1171305420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55844f70ce40 con 0x55844f6fa330 2024-06-10T01:17:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.794+0000 7fc08b7fe700 1 --2- 172.21.15.7:0/1171305420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55844f700390 0x55844f7091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:17:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.794+0000 7fc08affd700 1 --2- 172.21.15.7:0/1171305420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55844f6fa330 0x55844f70c1e0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fc07c005c50 tx=0x7fc07c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476e2b5e060af30a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.794+0000 7fc06affd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:17:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.794+0000 7fc06affd700 1 -- 172.21.15.7:0/1171305420 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c004a30 con 0x55844f6fa330 2024-06-10T01:17:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.794+0000 7fc06affd700 1 -- 172.21.15.7:0/1171305420 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc07c004bd0 con 0x55844f6fa330 2024-06-10T01:17:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.794+0000 7fc09c28fb80 1 -- 172.21.15.7:0/1171305420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc08401e400 con 0x55844f6fa330 2024-06-10T01:17:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.794+0000 7fc06affd700 1 -- 172.21.15.7:0/1171305420 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc07c004dd0 con 0x55844f6fa330 2024-06-10T01:17:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.794+0000 7fc09c28fb80 10 client.18273 fetch_fsmap learned FSMap version 40 2024-06-10T01:17:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.794+0000 7fc09c28fb80 1 -- 172.21.15.7:0/1171305420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55844f77f050 con 0x55844f6fa330 2024-06-10T01:17:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.795+0000 7fc06affd700 1 -- 172.21.15.7:0/1171305420 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc08401e400 con 0x55844f6fa330 2024-06-10T01:17:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.795+0000 7fc06affd700 1 -- 172.21.15.7:0/1171305420 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7fc07c022070 con 0x55844f6fa330 2024-06-10T01:17:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.795+0000 7fc09c28fb80 10 client.18273 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:17:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.795+0000 7fc09c28fb80 10 client.18273 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:17:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.795+0000 7fc09c28fb80 20 client.18273 populate_metadata read hostname 'smithi007' 2024-06-10T01:17:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.795+0000 7fc09c28fb80 1 --2- 172.21.15.7:0/1171305420 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55844f77f730 0x55844f781b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.795+0000 7fc09c28fb80 4 client.18273 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:17:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.795+0000 7fc09c28fb80 1 -- 172.21.15.7:0/1171305420 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55844f64f290 con 0x55844f77f730 2024-06-10T01:17:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.795+0000 7fc08a7fc700 1 --2- 172.21.15.7:0/1171305420 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55844f77f730 0x55844f781b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.796+0000 7fc08a7fc700 1 --2- 172.21.15.7:0/1171305420 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55844f77f730 0x55844f781b50 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e39a66aa4d2e66bd server_cookie=c17f108bf435403 in_seq=0 out_seq=0 2024-06-10T01:17:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.796+0000 7fc06affd700 10 client.18273 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.797+0000 7fc06affd700 1 -- 172.21.15.7:0/1171305420 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55844f64f290 con 0x55844f77f730 2024-06-10T01:17:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.797+0000 7fc06affd700 10 client.18273 handle_command_reply: tid=0 2024-06-10T01:17:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.797+0000 7fc09c28fb80 1 client.18273 shutdown 2024-06-10T01:17:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.797+0000 7fc09c28fb80 1 -- 172.21.15.7:0/1171305420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55844f6fa330 msgr2=0x55844f70c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.797+0000 7fc09c28fb80 1 --2- 172.21.15.7:0/1171305420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55844f6fa330 0x55844f70c1e0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fc07c005c50 tx=0x7fc07c009310 comp rx=0 tx=0).stop 2024-06-10T01:17:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.797+0000 7fc09c28fb80 1 -- 172.21.15.7:0/1171305420 shutdown_connections 2024-06-10T01:17:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.797+0000 7fc09c28fb80 1 --2- 172.21.15.7:0/1171305420 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55844f77f730 0x55844f781b50 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.797+0000 7fc09c28fb80 1 --2- 172.21.15.7:0/1171305420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55844f700390 0x55844f7091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.797+0000 7fc09c28fb80 1 --2- 172.21.15.7:0/1171305420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55844f6fa330 0x55844f70c1e0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.797+0000 7fc06affd700 0 client.18273 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.797+0000 7fc09c28fb80 1 --2- 172.21.15.7:0/1171305420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55844f6ffa70 0x55844f70c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.797+0000 7fc09c28fb80 1 -- 172.21.15.7:0/1171305420 >> 172.21.15.7:0/1171305420 conn(0x55844f620cc0 msgr2=0x55844f6e9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.797+0000 7fc09c28fb80 1 -- 172.21.15.7:0/1171305420 shutdown_connections 2024-06-10T01:17:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.797+0000 7fc09c28fb80 1 -- 172.21.15.7:0/1171305420 wait complete. 2024-06-10T01:17:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.797+0000 7fc09c28fb80 20 client.18273 trim_cache size 0 max 16384 2024-06-10T01:17:11.796 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:17:11.796 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-10T01:17:11.796 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-10T01:17:11.796 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:17:11.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.798+0000 7fc091a0c700 1 -- 172.21.15.7:0/3838106260 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc0740a2c20 msgr2=0x7fc0740a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:11.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.798+0000 7fc091a0c700 1 --2- 172.21.15.7:0/3838106260 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc0740a2c20 0x7fc0740a50e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc07c006010 tx=0x7fc07c01a040 comp rx=0 tx=0).stop 2024-06-10T01:17:11.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.798+0000 7fc091a0c700 1 -- 172.21.15.7:0/3838106260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc08c06b3c0 msgr2=0x7fc08c19ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:11.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.798+0000 7fc091a0c700 1 --2- 172.21.15.7:0/3838106260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc08c06b3c0 0x7fc08c19ee30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc08000e9a0 tx=0x7fc08000ecb0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.798+0000 7fc091a0c700 1 -- 172.21.15.7:0/3838106260 shutdown_connections 2024-06-10T01:17:11.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.798+0000 7fc091a0c700 1 --2- 172.21.15.7:0/3838106260 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc0740a2c20 0x7fc0740a50e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.798+0000 7fc091a0c700 1 --2- 172.21.15.7:0/3838106260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc08c1084f0 0x7fc08c198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.798+0000 7fc091a0c700 1 --2- 172.21.15.7:0/3838106260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc08c06aa00 0x7fc08c19e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.798+0000 7fc091a0c700 1 --2- 172.21.15.7:0/3838106260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc08c06b3c0 0x7fc08c19ee30 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:11.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.798+0000 7fc091a0c700 1 -- 172.21.15.7:0/3838106260 >> 172.21.15.7:0/3838106260 conn(0x7fc08c078300 msgr2=0x7fc08c100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:11.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.798+0000 7fc091a0c700 1 -- 172.21.15.7:0/3838106260 shutdown_connections 2024-06-10T01:17:11.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:11.798+0000 7fc091a0c700 1 -- 172.21.15.7:0/3838106260 wait complete. 2024-06-10T01:17:12.623 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:17:12.623 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:17:12.751 INFO:tasks.workunit.client.0.smithi007.stdout: 1 822531 7.42 MB/sec execute 473 sec latency 16.111 ms 2024-06-10T01:17:12.925 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:17:13.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:12 smithi175 ceph-mon[36636]: pgmap v768: 129 pgs: 129 active+clean; 139 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 1.3 MiB/s wr, 78 op/s 2024-06-10T01:17:13.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:12 smithi007 ceph-mon[32398]: pgmap v768: 129 pgs: 129 active+clean; 139 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 1.3 MiB/s wr, 78 op/s 2024-06-10T01:17:13.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:12 smithi078 ceph-mon[36683]: pgmap v768: 129 pgs: 129 active+clean; 139 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 1.3 MiB/s wr, 78 op/s 2024-06-10T01:17:13.751 INFO:tasks.workunit.client.0.smithi007.stdout: 1 824241 7.43 MB/sec execute 474 sec latency 15.431 ms 2024-06-10T01:17:14.751 INFO:tasks.workunit.client.0.smithi007.stdout: 1 826656 7.44 MB/sec execute 475 sec latency 24.877 ms 2024-06-10T01:17:14.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.778+0000 7f9d18d2c700 1 -- 172.21.15.7:0/4110169702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d14102550 msgr2=0x7f9d141029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:14.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.778+0000 7f9d18d2c700 1 --2- 172.21.15.7:0/4110169702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d14102550 0x7f9d141029d0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f9d08009990 tx=0x7f9d08009ca0 comp rx=0 tx=0).stop 2024-06-10T01:17:14.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.778+0000 7f9d18d2c700 1 -- 172.21.15.7:0/4110169702 shutdown_connections 2024-06-10T01:17:14.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.778+0000 7f9d18d2c700 1 --2- 172.21.15.7:0/4110169702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d14102f10 0x7f9d1410f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:14.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.778+0000 7f9d18d2c700 1 --2- 172.21.15.7:0/4110169702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d141084f0 0x7f9d141088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:14.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.778+0000 7f9d18d2c700 1 --2- 172.21.15.7:0/4110169702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d14102550 0x7f9d141029d0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:14.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.778+0000 7f9d18d2c700 1 -- 172.21.15.7:0/4110169702 >> 172.21.15.7:0/4110169702 conn(0x7f9d14079680 msgr2=0x7f9d14079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:14.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.779+0000 7f9d18d2c700 1 -- 172.21.15.7:0/4110169702 shutdown_connections 2024-06-10T01:17:14.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.779+0000 7f9d18d2c700 1 -- 172.21.15.7:0/4110169702 wait complete. 2024-06-10T01:17:14.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.779+0000 7f9d18d2c700 1 Processor -- start 2024-06-10T01:17:14.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.779+0000 7f9d18d2c700 1 -- start start 2024-06-10T01:17:14.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.780+0000 7f9d18d2c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d14102550 0x7f9d14074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:14.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.780+0000 7f9d18d2c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d14102f10 0x7f9d14074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:14.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.780+0000 7f9d18d2c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d141084f0 0x7f9d14075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:14.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.780+0000 7f9d18d2c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9d14112770 con 0x7f9d141084f0 2024-06-10T01:17:14.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.780+0000 7f9d18d2c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9d141125f0 con 0x7f9d14102f10 2024-06-10T01:17:14.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.780+0000 7f9d18d2c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9d141128f0 con 0x7f9d14102550 2024-06-10T01:17:14.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.780+0000 7f9d1259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d14102550 0x7f9d14074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:14.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.780+0000 7f9d12d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d141084f0 0x7f9d14075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:14.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.780+0000 7f9d11d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d14102f10 0x7f9d14074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:14.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.780+0000 7f9d12d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d141084f0 0x7f9d14075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37490/0 (socket says 172.21.15.7:37490) 2024-06-10T01:17:14.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.780+0000 7f9d11d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d14102f10 0x7f9d14074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:40582/0 (socket says 172.21.15.7:40582) 2024-06-10T01:17:14.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.780+0000 7f9d1259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d14102550 0x7f9d14074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:58234/0 (socket says 172.21.15.7:58234) 2024-06-10T01:17:14.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.780+0000 7f9d12d9d700 1 -- 172.21.15.7:0/3482990572 learned_addr learned my addr 172.21.15.7:0/3482990572 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:14.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.780+0000 7f9d11d9b700 1 -- 172.21.15.7:0/3482990572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d14102550 msgr2=0x7f9d14074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:14.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.780+0000 7f9d11d9b700 1 --2- 172.21.15.7:0/3482990572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d14102550 0x7f9d14074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:14.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.780+0000 7f9d11d9b700 1 -- 172.21.15.7:0/3482990572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d141084f0 msgr2=0x7f9d14075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:14.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.780+0000 7f9d11d9b700 1 --2- 172.21.15.7:0/3482990572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d141084f0 0x7f9d14075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:14.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.780+0000 7f9d11d9b700 1 -- 172.21.15.7:0/3482990572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d141a7130 con 0x7f9d14102f10 2024-06-10T01:17:14.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.780+0000 7f9d11d9b700 1 --2- 172.21.15.7:0/3482990572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d14102f10 0x7f9d14074cd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9d080059d0 tx=0x7f9d0800bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:14.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.781+0000 7f9d037fe700 1 -- 172.21.15.7:0/3482990572 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d08027070 con 0x7f9d14102f10 2024-06-10T01:17:14.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.781+0000 7f9d18d2c700 1 -- 172.21.15.7:0/3482990572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d141a73c0 con 0x7f9d14102f10 2024-06-10T01:17:14.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.781+0000 7f9d037fe700 1 -- 172.21.15.7:0/3482990572 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d08003d70 con 0x7f9d14102f10 2024-06-10T01:17:14.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.781+0000 7f9d037fe700 1 -- 172.21.15.7:0/3482990572 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d08022e30 con 0x7f9d14102f10 2024-06-10T01:17:14.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.781+0000 7f9d18d2c700 1 -- 172.21.15.7:0/3482990572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d141a7870 con 0x7f9d14102f10 2024-06-10T01:17:14.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.782+0000 7f9d18d2c700 1 -- 172.21.15.7:0/3482990572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d1410a9d0 con 0x7f9d14102f10 2024-06-10T01:17:14.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.783+0000 7f9d037fe700 1 -- 172.21.15.7:0/3482990572 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9d0802a070 con 0x7f9d14102f10 2024-06-10T01:17:14.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.784+0000 7f9d037fe700 1 --2- 172.21.15.7:0/3482990572 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9cfc0a2c20 0x7f9cfc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:14.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.784+0000 7f9d037fe700 1 -- 172.21.15.7:0/3482990572 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f9d080d0550 con 0x7f9d14102f10 2024-06-10T01:17:14.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.784+0000 7f9d1259c700 1 --2- 172.21.15.7:0/3482990572 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9cfc0a2c20 0x7f9cfc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:14.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.785+0000 7f9d1259c700 1 --2- 172.21.15.7:0/3482990572 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9cfc0a2c20 0x7f9cfc0a50e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9d04009be0 tx=0x7f9d04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:14.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:14.787+0000 7f9d037fe700 1 -- 172.21.15.7:0/3482990572 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9d08013070 con 0x7f9d14102f10 2024-06-10T01:17:15.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:15.063+0000 7f9d18d2c700 1 -- 172.21.15.7:0/3482990572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d14075cc0 con 0x7f9d14102f10 2024-06-10T01:17:15.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:15.064+0000 7f9d037fe700 1 -- 172.21.15.7:0/3482990572 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11855 (secure 0 0 0) 0x7f9d0809d160 con 0x7f9d14102f10 2024-06-10T01:17:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:17:15.058 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:16:56.409130+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24451,24434,24436]},"id":1}]} 2024-06-10T01:17:15.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:15.067+0000 7f9d18d2c700 1 -- 172.21.15.7:0/3482990572 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9cfc0a2c20 msgr2=0x7f9cfc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:15.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:15.067+0000 7f9d18d2c700 1 --2- 172.21.15.7:0/3482990572 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9cfc0a2c20 0x7f9cfc0a50e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9d04009be0 tx=0x7f9d04009260 comp rx=0 tx=0).stop 2024-06-10T01:17:15.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:15.067+0000 7f9d18d2c700 1 -- 172.21.15.7:0/3482990572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d14102f10 msgr2=0x7f9d14074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:15.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:15.067+0000 7f9d18d2c700 1 --2- 172.21.15.7:0/3482990572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d14102f10 0x7f9d14074cd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9d080059d0 tx=0x7f9d0800bab0 comp rx=0 tx=0).stop 2024-06-10T01:17:15.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:15.067+0000 7f9d18d2c700 1 -- 172.21.15.7:0/3482990572 shutdown_connections 2024-06-10T01:17:15.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:15.067+0000 7f9d18d2c700 1 --2- 172.21.15.7:0/3482990572 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9cfc0a2c20 0x7f9cfc0a50e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:15.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:15.067+0000 7f9d18d2c700 1 --2- 172.21.15.7:0/3482990572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d141084f0 0x7f9d14075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:15.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:15.067+0000 7f9d18d2c700 1 --2- 172.21.15.7:0/3482990572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d14102550 0x7f9d14074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:15.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:15.067+0000 7f9d18d2c700 1 --2- 172.21.15.7:0/3482990572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d14102f10 0x7f9d14074cd0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:15.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:15.067+0000 7f9d18d2c700 1 -- 172.21.15.7:0/3482990572 >> 172.21.15.7:0/3482990572 conn(0x7f9d14079680 msgr2=0x7f9d14101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:15.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:15.067+0000 7f9d18d2c700 1 -- 172.21.15.7:0/3482990572 shutdown_connections 2024-06-10T01:17:15.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:15.067+0000 7f9d18d2c700 1 -- 172.21.15.7:0/3482990572 wait complete. 2024-06-10T01:17:15.062 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 40 2024-06-10T01:17:15.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:14 smithi175 ceph-mon[36636]: pgmap v769: 129 pgs: 129 active+clean; 137 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.8 MiB/s wr, 126 op/s 2024-06-10T01:17:15.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:14 smithi007 ceph-mon[32398]: pgmap v769: 129 pgs: 129 active+clean; 137 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.8 MiB/s wr, 126 op/s 2024-06-10T01:17:15.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:14 smithi078 ceph-mon[36683]: pgmap v769: 129 pgs: 129 active+clean; 137 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.8 MiB/s wr, 126 op/s 2024-06-10T01:17:15.751 INFO:tasks.workunit.client.0.smithi007.stdout: 1 828054 7.44 MB/sec execute 476 sec latency 45.629 ms 2024-06-10T01:17:15.846 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:17:16.118 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:17:16.242 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:15 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3482990572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:17:16.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:15 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3482990572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:17:16.452 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:15 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3482990572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:17:16.751 INFO:tasks.workunit.client.0.smithi007.stdout: 1 830198 7.45 MB/sec execute 477 sec latency 60.294 ms 2024-06-10T01:17:17.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:16 smithi007 ceph-mon[32398]: pgmap v770: 129 pgs: 129 active+clean; 138 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 1.8 MiB/s wr, 168 op/s 2024-06-10T01:17:17.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:16 smithi078 ceph-mon[36683]: pgmap v770: 129 pgs: 129 active+clean; 138 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 1.8 MiB/s wr, 168 op/s 2024-06-10T01:17:17.452 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:16 smithi175 ceph-mon[36636]: pgmap v770: 129 pgs: 129 active+clean; 138 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 1.8 MiB/s wr, 168 op/s 2024-06-10T01:17:17.751 INFO:tasks.workunit.client.0.smithi007.stdout: 1 831678 7.45 MB/sec execute 478 sec latency 11.267 ms 2024-06-10T01:17:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.801+0000 7f9ada176700 1 -- 172.21.15.7:0/3716698405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ad406b3a0 msgr2=0x7f9ad410f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.801+0000 7f9ada176700 1 --2- 172.21.15.7:0/3716698405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ad406b3a0 0x7f9ad410f310 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f9acc009f90 tx=0x7f9acc00b330 comp rx=0 tx=0).stop 2024-06-10T01:17:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.801+0000 7f9ada176700 1 -- 172.21.15.7:0/3716698405 shutdown_connections 2024-06-10T01:17:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.801+0000 7f9ada176700 1 --2- 172.21.15.7:0/3716698405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ad406b3a0 0x7f9ad410f310 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.801+0000 7f9ada176700 1 --2- 172.21.15.7:0/3716698405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ad41084d0 0x7f9ad41088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.801+0000 7f9ada176700 1 --2- 172.21.15.7:0/3716698405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ad406aa00 0x7f9ad406ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.801+0000 7f9ada176700 1 -- 172.21.15.7:0/3716698405 >> 172.21.15.7:0/3716698405 conn(0x7f9ad4078330 msgr2=0x7f9ad4078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.801+0000 7f9ada176700 1 -- 172.21.15.7:0/3716698405 shutdown_connections 2024-06-10T01:17:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.801+0000 7f9ada176700 1 -- 172.21.15.7:0/3716698405 wait complete. 2024-06-10T01:17:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.802+0000 7f9ada176700 1 Processor -- start 2024-06-10T01:17:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.802+0000 7f9ada176700 1 -- start start 2024-06-10T01:17:17.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.802+0000 7f9ada176700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ad406aa00 0x7f9ad419e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:17.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.802+0000 7f9ada176700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ad406b3a0 0x7f9ad419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:17.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.802+0000 7f9ada176700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ad41084d0 0x7f9ad41989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:17.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.802+0000 7f9ada176700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad4112720 con 0x7f9ad41084d0 2024-06-10T01:17:17.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.802+0000 7f9ada176700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad41125a0 con 0x7f9ad406aa00 2024-06-10T01:17:17.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.802+0000 7f9ada176700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad41128a0 con 0x7f9ad406b3a0 2024-06-10T01:17:17.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.802+0000 7f9ad37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ad406aa00 0x7f9ad419e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.802+0000 7f9ad37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ad406aa00 0x7f9ad419e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:40600/0 (socket says 172.21.15.7:40600) 2024-06-10T01:17:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.802+0000 7f9ad3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ad41084d0 0x7f9ad41989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.802+0000 7f9ad37fe700 1 -- 172.21.15.7:0/1025877136 learned_addr learned my addr 172.21.15.7:0/1025877136 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.802+0000 7f9acaffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ad406b3a0 0x7f9ad419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.803+0000 7f9ad37fe700 1 -- 172.21.15.7:0/1025877136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ad406b3a0 msgr2=0x7f9ad419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.803+0000 7f9ad37fe700 1 --2- 172.21.15.7:0/1025877136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ad406b3a0 0x7f9ad419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.803+0000 7f9ad37fe700 1 -- 172.21.15.7:0/1025877136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ad41084d0 msgr2=0x7f9ad41989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.803+0000 7f9ad37fe700 1 --2- 172.21.15.7:0/1025877136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ad41084d0 0x7f9ad41989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.803+0000 7f9ad37fe700 1 -- 172.21.15.7:0/1025877136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad4199100 con 0x7f9ad406aa00 2024-06-10T01:17:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.803+0000 7f9acaffd700 1 --2- 172.21.15.7:0/1025877136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ad406b3a0 0x7f9ad419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:17:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.803+0000 7f9ad37fe700 1 --2- 172.21.15.7:0/1025877136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ad406aa00 0x7f9ad419e7e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400ec10 tx=0x7f9ac400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.803+0000 7f9ad17fa700 1 -- 172.21.15.7:0/1025877136 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac400cc90 con 0x7f9ad406aa00 2024-06-10T01:17:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.803+0000 7f9ad17fa700 1 -- 172.21.15.7:0/1025877136 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ac400ce30 con 0x7f9ad406aa00 2024-06-10T01:17:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.803+0000 7f9ada176700 1 -- 172.21.15.7:0/1025877136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ad41993f0 con 0x7f9ad406aa00 2024-06-10T01:17:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.803+0000 7f9ad17fa700 1 -- 172.21.15.7:0/1025877136 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac4010690 con 0x7f9ad406aa00 2024-06-10T01:17:17.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.803+0000 7f9ada176700 1 -- 172.21.15.7:0/1025877136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ad4101380 con 0x7f9ad406aa00 2024-06-10T01:17:17.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.805+0000 7f9ad17fa700 1 -- 172.21.15.7:0/1025877136 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9ac4003680 con 0x7f9ad406aa00 2024-06-10T01:17:17.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.806+0000 7f9ad17fa700 1 --2- 172.21.15.7:0/1025877136 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9ac00a2ca0 0x7f9ac00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:17.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.806+0000 7f9ad17fa700 1 -- 172.21.15.7:0/1025877136 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f9ac40cf680 con 0x7f9ad406aa00 2024-06-10T01:17:17.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.806+0000 7f9acaffd700 1 --2- 172.21.15.7:0/1025877136 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9ac00a2ca0 0x7f9ac00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:17.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.807+0000 7f9acaffd700 1 --2- 172.21.15.7:0/1025877136 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9ac00a2ca0 0x7f9ac00a5160 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40788a0 tx=0x7f9abc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:17.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.807+0000 7f9ae49f9b80 1 Processor -- start 2024-06-10T01:17:17.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.807+0000 7f9ae49f9b80 1 -- start start 2024-06-10T01:17:17.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.807+0000 7f9ae49f9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f396f22f60 0x55f396f23340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:17.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.807+0000 7f9ae49f9b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f396f23880 0x55f396f1d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:17.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.807+0000 7f9ae49f9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f396f1d7a0 0x55f396f1dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:17.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.807+0000 7f9ae49f9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f396e0ff60 con 0x55f396f22f60 2024-06-10T01:17:17.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.807+0000 7f9ae49f9b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f396e6e850 con 0x55f396f23880 2024-06-10T01:17:17.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.807+0000 7f9ae49f9b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f396dd6460 con 0x55f396f1d7a0 2024-06-10T01:17:17.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.808+0000 7f9ad37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f396f22f60 0x55f396f23340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:17.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.808+0000 7f9ad37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f396f22f60 0x55f396f23340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37522/0 (socket says 172.21.15.7:37522) 2024-06-10T01:17:17.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.808+0000 7f9ad37fe700 1 -- 172.21.15.7:0/735868510 learned_addr learned my addr 172.21.15.7:0/735868510 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.808+0000 7f9ad3fff700 1 --2- 172.21.15.7:0/735868510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f396f1d7a0 0x55f396f1dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.808+0000 7f9acaffd700 1 --2- 172.21.15.7:0/735868510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f396f23880 0x55f396f1d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.808+0000 7f9ad3fff700 1 -- 172.21.15.7:0/735868510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f396f23880 msgr2=0x55f396f1d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.808+0000 7f9ad3fff700 1 --2- 172.21.15.7:0/735868510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f396f23880 0x55f396f1d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.808+0000 7f9ad3fff700 1 -- 172.21.15.7:0/735868510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f396f22f60 msgr2=0x55f396f23340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.808+0000 7f9ad3fff700 1 --2- 172.21.15.7:0/735868510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f396f22f60 0x55f396f23340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.808+0000 7f9ad3fff700 1 -- 172.21.15.7:0/735868510 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f396dbd8d0 con 0x55f396f1d7a0 2024-06-10T01:17:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.808+0000 7f9ad37fe700 1 --2- 172.21.15.7:0/735868510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f396f22f60 0x55f396f23340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:17:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.808+0000 7f9ad3fff700 1 --2- 172.21.15.7:0/735868510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f396f1d7a0 0x55f396f1dbc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9acc009f90 tx=0x7f9acc009300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa2f584d8be301aa server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.808+0000 7f9ac9ffb700 1 -- 172.21.15.7:0/735868510 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc020070 con 0x55f396f1d7a0 2024-06-10T01:17:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.808+0000 7f9ac9ffb700 1 -- 172.21.15.7:0/735868510 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9acc003c80 con 0x55f396f1d7a0 2024-06-10T01:17:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.808+0000 7f9ac9ffb700 1 -- 172.21.15.7:0/735868510 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc005870 con 0x55f396f1d7a0 2024-06-10T01:17:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.809+0000 7f9ae49f9b80 1 -- 172.21.15.7:0/735868510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f396f1d7a0 msgr2=0x55f396f1dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.809+0000 7f9ae49f9b80 1 --2- 172.21.15.7:0/735868510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f396f1d7a0 0x55f396f1dbc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9acc009f90 tx=0x7f9acc009300 comp rx=0 tx=0).stop 2024-06-10T01:17:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.809+0000 7f9ae49f9b80 1 -- 172.21.15.7:0/735868510 shutdown_connections 2024-06-10T01:17:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.809+0000 7f9ae49f9b80 1 --2- 172.21.15.7:0/735868510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f396f1d7a0 0x55f396f1dbc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.809+0000 7f9ae49f9b80 1 --2- 172.21.15.7:0/735868510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f396f22f60 0x55f396f23340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.809+0000 7f9ae49f9b80 1 --2- 172.21.15.7:0/735868510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f396f23880 0x55f396f1d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.809+0000 7f9ae49f9b80 1 -- 172.21.15.7:0/735868510 >> 172.21.15.7:0/735868510 conn(0x55f396e4a080 msgr2=0x55f396f18220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.809+0000 7f9ae49f9b80 1 -- 172.21.15.7:0/735868510 shutdown_connections 2024-06-10T01:17:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.809+0000 7f9ae49f9b80 1 -- 172.21.15.7:0/735868510 wait complete. 2024-06-10T01:17:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.810+0000 7f9ae49f9b80 1 -- start start 2024-06-10T01:17:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.810+0000 7f9ae49f9b80 1 Processor -- start 2024-06-10T01:17:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.810+0000 7f9ae49f9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f396f1d7a0 0x55f396f295d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.810+0000 7f9ae49f9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f396f22f60 0x55f396f268d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.810+0000 7f9ae49f9b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f396f23880 0x55f396f26e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.810+0000 7f9ae49f9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f396e0efc0 con 0x55f396f22f60 2024-06-10T01:17:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.810+0000 7f9ae49f9b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f396e6a810 con 0x55f396f23880 2024-06-10T01:17:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.810+0000 7f9ae49f9b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f396f2a200 con 0x55f396f1d7a0 2024-06-10T01:17:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.810+0000 7f9ae49f9b80 10 client.0 _ll_register_callbacks cb 0x55f396e045b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:17:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.810+0000 7f9acaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f396f22f60 0x55f396f268d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.810+0000 7f9ad37fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f396f1d7a0 0x55f396f295d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.810+0000 7f9ad3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f396f23880 0x55f396f26e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.810+0000 7f9ad37fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f396f1d7a0 0x55f396f295d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:58268/0 (socket says 172.21.15.7:58268) 2024-06-10T01:17:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.810+0000 7f9ad37fe700 1 -- 172.21.15.7:0/2272246220 learned_addr learned my addr 172.21.15.7:0/2272246220 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.811+0000 7f9ad37fe700 1 -- 172.21.15.7:0/2272246220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f396f23880 msgr2=0x55f396f26e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.811+0000 7f9ad37fe700 1 --2- 172.21.15.7:0/2272246220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f396f23880 0x55f396f26e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.811+0000 7f9ad37fe700 1 -- 172.21.15.7:0/2272246220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f396f22f60 msgr2=0x55f396f268d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.811+0000 7f9ad37fe700 1 --2- 172.21.15.7:0/2272246220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f396f22f60 0x55f396f268d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.811+0000 7f9ad37fe700 1 -- 172.21.15.7:0/2272246220 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f396e34750 con 0x55f396f1d7a0 2024-06-10T01:17:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.811+0000 7f9ad3fff700 1 --2- 172.21.15.7:0/2272246220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f396f23880 0x55f396f26e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:17:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.811+0000 7f9acaffd700 1 --2- 172.21.15.7:0/2272246220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f396f22f60 0x55f396f268d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:17:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.811+0000 7f9ad37fe700 1 --2- 172.21.15.7:0/2272246220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f396f1d7a0 0x55f396f295d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4005cb0 tx=0x7f9ac409b4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5007ddbfa8c0708b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.811+0000 7f9ab2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:17:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.811+0000 7f9ab2ffd700 1 -- 172.21.15.7:0/2272246220 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac40b9060 con 0x55f396f1d7a0 2024-06-10T01:17:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.811+0000 7f9ae49f9b80 1 -- 172.21.15.7:0/2272246220 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9acc003c80 con 0x55f396f1d7a0 2024-06-10T01:17:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.811+0000 7f9ab2ffd700 1 -- 172.21.15.7:0/2272246220 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f9ac409bd60 con 0x55f396f1d7a0 2024-06-10T01:17:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.811+0000 7f9ab2ffd700 1 -- 172.21.15.7:0/2272246220 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9ac409bb00 con 0x55f396f1d7a0 2024-06-10T01:17:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.811+0000 7f9ae49f9b80 10 client.27652 fetch_fsmap learned FSMap version 40 2024-06-10T01:17:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.811+0000 7f9ae49f9b80 1 -- 172.21.15.7:0/2272246220 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f396f349f0 con 0x55f396f1d7a0 2024-06-10T01:17:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.812+0000 7f9ab2ffd700 1 -- 172.21.15.7:0/2272246220 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9acc003c80 con 0x55f396f1d7a0 2024-06-10T01:17:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.812+0000 7f9ab2ffd700 1 -- 172.21.15.7:0/2272246220 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f9ac40a4370 con 0x55f396f1d7a0 2024-06-10T01:17:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.812+0000 7f9ae49f9b80 10 client.27652 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:17:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.812+0000 7f9ae49f9b80 10 client.27652 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:17:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.812+0000 7f9ae49f9b80 20 client.27652 populate_metadata read hostname 'smithi007' 2024-06-10T01:17:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.812+0000 7f9ae49f9b80 1 --2- 172.21.15.7:0/2272246220 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55f396f37030 0x55f396f39420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.812+0000 7f9ae49f9b80 4 client.27652 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:17:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.812+0000 7f9ae49f9b80 1 -- 172.21.15.7:0/2272246220 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f396e7bd50 con 0x55f396f37030 2024-06-10T01:17:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.812+0000 7f9acaffd700 1 --2- 172.21.15.7:0/2272246220 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55f396f37030 0x55f396f39420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.812+0000 7f9acaffd700 1 --2- 172.21.15.7:0/2272246220 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55f396f37030 0x55f396f39420 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5c16dc495156aece server_cookie=dffce4c728317a0 in_seq=0 out_seq=0 2024-06-10T01:17:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.812+0000 7f9ab2ffd700 10 client.27652 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:17:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.813+0000 7f9ab2ffd700 1 -- 172.21.15.7:0/2272246220 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f396e7bd50 con 0x55f396f37030 2024-06-10T01:17:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.813+0000 7f9ab2ffd700 10 client.27652 handle_command_reply: tid=0 2024-06-10T01:17:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.813+0000 7f9ae49f9b80 1 client.27652 shutdown 2024-06-10T01:17:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.813+0000 7f9ae49f9b80 1 -- 172.21.15.7:0/2272246220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f396f1d7a0 msgr2=0x55f396f295d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.813+0000 7f9ae49f9b80 1 --2- 172.21.15.7:0/2272246220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f396f1d7a0 0x55f396f295d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4005cb0 tx=0x7f9ac409b4e0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.813+0000 7f9ae49f9b80 1 -- 172.21.15.7:0/2272246220 shutdown_connections 2024-06-10T01:17:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.813+0000 7f9ae49f9b80 1 --2- 172.21.15.7:0/2272246220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f396f23880 0x55f396f26e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.814+0000 7f9ae49f9b80 1 --2- 172.21.15.7:0/2272246220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f396f1d7a0 0x55f396f295d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.814+0000 7f9ae49f9b80 1 --2- 172.21.15.7:0/2272246220 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55f396f37030 0x55f396f39420 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.814+0000 7f9ae49f9b80 1 --2- 172.21.15.7:0/2272246220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f396f22f60 0x55f396f268d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.814+0000 7f9ae49f9b80 1 -- 172.21.15.7:0/2272246220 >> 172.21.15.7:0/2272246220 conn(0x55f396e4a080 msgr2=0x55f396f17700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.814+0000 7f9ab2ffd700 0 client.27652 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:17:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.814+0000 7f9ae49f9b80 1 -- 172.21.15.7:0/2272246220 shutdown_connections 2024-06-10T01:17:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.814+0000 7f9ae49f9b80 1 -- 172.21.15.7:0/2272246220 wait complete. 2024-06-10T01:17:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.814+0000 7f9ae49f9b80 20 client.27652 trim_cache size 0 max 16384 2024-06-10T01:17:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.830+0000 7f9ae49f9b80 1 Processor -- start 2024-06-10T01:17:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.830+0000 7f9ae49f9b80 1 -- start start 2024-06-10T01:17:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.831+0000 7f9ae49f9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f396f1d7a0 0x55f396f1d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.831+0000 7f9ae49f9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f396f22f60 0x55f396f24370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.831+0000 7f9ae49f9b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f396f23880 0x55f396f17e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.831+0000 7f9ae49f9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad41b2bd0 con 0x55f396f22f60 2024-06-10T01:17:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.831+0000 7f9ae49f9b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad41b0460 con 0x55f396f23880 2024-06-10T01:17:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.831+0000 7f9ae49f9b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad41ae2b0 con 0x55f396f1d7a0 2024-06-10T01:17:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.831+0000 7f9acaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f396f22f60 0x55f396f24370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.831+0000 7f9acaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f396f22f60 0x55f396f24370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37548/0 (socket says 172.21.15.7:37548) 2024-06-10T01:17:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.831+0000 7f9acaffd700 1 -- 172.21.15.7:0/1890449398 learned_addr learned my addr 172.21.15.7:0/1890449398 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.831+0000 7f9ad3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f396f23880 0x55f396f17e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.831+0000 7f9ad37fe700 1 --2- 172.21.15.7:0/1890449398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f396f1d7a0 0x55f396f1d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.831+0000 7f9ad3fff700 1 -- 172.21.15.7:0/1890449398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f396f1d7a0 msgr2=0x55f396f1d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.831+0000 7f9ad3fff700 1 --2- 172.21.15.7:0/1890449398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f396f1d7a0 0x55f396f1d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.831+0000 7f9ad3fff700 1 -- 172.21.15.7:0/1890449398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f396f22f60 msgr2=0x55f396f24370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.831+0000 7f9ad3fff700 1 --2- 172.21.15.7:0/1890449398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f396f22f60 0x55f396f24370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.831+0000 7f9ad3fff700 1 -- 172.21.15.7:0/1890449398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f396f23680 con 0x55f396f23880 2024-06-10T01:17:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.831+0000 7f9acaffd700 1 --2- 172.21.15.7:0/1890449398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f396f22f60 0x55f396f24370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:17:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.831+0000 7f9ad3fff700 1 --2- 172.21.15.7:0/1890449398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f396f23880 0x55f396f17e80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9acc00b370 tx=0x7f9acc005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce577c4d69e8fb0c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:17.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.832+0000 7f9ac97fa700 1 -- 172.21.15.7:0/1890449398 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc02d070 con 0x55f396f23880 2024-06-10T01:17:17.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.832+0000 7f9ac97fa700 1 -- 172.21.15.7:0/1890449398 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9acc005920 con 0x55f396f23880 2024-06-10T01:17:17.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.832+0000 7f9ac97fa700 1 -- 172.21.15.7:0/1890449398 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc020040 con 0x55f396f23880 2024-06-10T01:17:17.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.833+0000 7f9ae49f9b80 1 -- 172.21.15.7:0/1890449398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f396f23880 msgr2=0x55f396f17e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:17.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.833+0000 7f9ae49f9b80 1 --2- 172.21.15.7:0/1890449398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f396f23880 0x55f396f17e80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9acc00b370 tx=0x7f9acc005040 comp rx=0 tx=0).stop 2024-06-10T01:17:17.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.833+0000 7f9ae49f9b80 1 -- 172.21.15.7:0/1890449398 shutdown_connections 2024-06-10T01:17:17.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.833+0000 7f9ae49f9b80 1 --2- 172.21.15.7:0/1890449398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f396f23880 0x55f396f17e80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.833+0000 7f9ae49f9b80 1 --2- 172.21.15.7:0/1890449398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f396f1d7a0 0x55f396f1d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.833+0000 7f9ae49f9b80 1 --2- 172.21.15.7:0/1890449398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f396f22f60 0x55f396f24370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.833+0000 7f9ae49f9b80 1 -- 172.21.15.7:0/1890449398 >> 172.21.15.7:0/1890449398 conn(0x55f396e4a080 msgr2=0x55f396f23c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:17.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.833+0000 7f9ae49f9b80 1 -- 172.21.15.7:0/1890449398 shutdown_connections 2024-06-10T01:17:17.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.833+0000 7f9ae49f9b80 1 -- 172.21.15.7:0/1890449398 wait complete. 2024-06-10T01:17:17.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.833+0000 7f9ae49f9b80 1 -- start start 2024-06-10T01:17:17.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.833+0000 7f9ae49f9b80 1 Processor -- start 2024-06-10T01:17:17.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.834+0000 7f9ae49f9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f396f1d7a0 0x55f396f2f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:17.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.834+0000 7f9ae49f9b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f396f22f60 0x55f396f2fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:17.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.834+0000 7f9ae49f9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f396f23880 0x55f396f3db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:17.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.834+0000 7f9ae49f9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad41ae920 con 0x55f396f1d7a0 2024-06-10T01:17:17.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.834+0000 7f9ae49f9b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad41afd90 con 0x55f396f22f60 2024-06-10T01:17:17.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.834+0000 7f9ae49f9b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad41aeff0 con 0x55f396f23880 2024-06-10T01:17:17.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.834+0000 7f9ae49f9b80 10 client.0 _ll_register_callbacks cb 0x55f396e045b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:17:17.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.834+0000 7f9acaffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f396f22f60 0x55f396f2fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:17.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.834+0000 7f9ad37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f396f1d7a0 0x55f396f2f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:17.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.834+0000 7f9acaffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f396f22f60 0x55f396f2fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:40638/0 (socket says 172.21.15.7:40638) 2024-06-10T01:17:17.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.834+0000 7f9acaffd700 1 -- 172.21.15.7:0/2009109832 learned_addr learned my addr 172.21.15.7:0/2009109832 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:17.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.834+0000 7f9ad37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f396f1d7a0 0x55f396f2f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37560/0 (socket says 172.21.15.7:37560) 2024-06-10T01:17:17.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.834+0000 7f9ad3fff700 1 --2- 172.21.15.7:0/2009109832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f396f23880 0x55f396f3db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:17.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.834+0000 7f9acaffd700 1 -- 172.21.15.7:0/2009109832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f396f23880 msgr2=0x55f396f3db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.834+0000 7f9acaffd700 1 --2- 172.21.15.7:0/2009109832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f396f23880 0x55f396f3db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.834+0000 7f9acaffd700 1 -- 172.21.15.7:0/2009109832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f396f1d7a0 msgr2=0x55f396f2f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.834+0000 7f9acaffd700 1 --2- 172.21.15.7:0/2009109832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f396f1d7a0 0x55f396f2f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.834+0000 7f9acaffd700 1 -- 172.21.15.7:0/2009109832 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f396f30180 con 0x55f396f22f60 2024-06-10T01:17:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.834+0000 7f9ad37fe700 1 --2- 172.21.15.7:0/2009109832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f396f1d7a0 0x55f396f2f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:17:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.834+0000 7f9acaffd700 1 --2- 172.21.15.7:0/2009109832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f396f22f60 0x55f396f2fa30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9abc005950 tx=0x7f9abc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43a655aa2bec0d21 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.834+0000 7f9ab2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:17:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.835+0000 7f9ab2ffd700 1 -- 172.21.15.7:0/2009109832 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc02a610 con 0x55f396f22f60 2024-06-10T01:17:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.835+0000 7f9ae49f9b80 1 -- 172.21.15.7:0/2009109832 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9acc005920 con 0x55f396f22f60 2024-06-10T01:17:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.835+0000 7f9ab2ffd700 1 -- 172.21.15.7:0/2009109832 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f9abc02a7b0 con 0x55f396f22f60 2024-06-10T01:17:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.835+0000 7f9ab2ffd700 1 -- 172.21.15.7:0/2009109832 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9abc03b050 con 0x55f396f22f60 2024-06-10T01:17:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.835+0000 7f9ae49f9b80 10 client.27635 fetch_fsmap learned FSMap version 40 2024-06-10T01:17:17.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.835+0000 7f9ae49f9b80 1 -- 172.21.15.7:0/2009109832 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f396fa1250 con 0x55f396f22f60 2024-06-10T01:17:17.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.835+0000 7f9ab2ffd700 1 -- 172.21.15.7:0/2009109832 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9acc005920 con 0x55f396f22f60 2024-06-10T01:17:17.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.836+0000 7f9ab2ffd700 1 -- 172.21.15.7:0/2009109832 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f9abc031030 con 0x55f396f22f60 2024-06-10T01:17:17.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.836+0000 7f9ae49f9b80 10 client.27635 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:17:17.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.836+0000 7f9ae49f9b80 10 client.27635 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:17:17.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.836+0000 7f9ae49f9b80 20 client.27635 populate_metadata read hostname 'smithi007' 2024-06-10T01:17:17.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.836+0000 7f9ae49f9b80 1 --2- 172.21.15.7:0/2009109832 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55f396fa1970 0x55f396fa3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:17.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.836+0000 7f9ae49f9b80 4 client.27635 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:17:17.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.836+0000 7f9ae49f9b80 1 -- 172.21.15.7:0/2009109832 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f396e762d0 con 0x55f396fa1970 2024-06-10T01:17:17.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.836+0000 7f9ad37fe700 1 --2- 172.21.15.7:0/2009109832 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55f396fa1970 0x55f396fa3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:17.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.836+0000 7f9ab2ffd700 10 client.27635 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:17:17.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.836+0000 7f9ad37fe700 1 --2- 172.21.15.7:0/2009109832 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55f396fa1970 0x55f396fa3d90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=19cf659e16f395cc server_cookie=e0108c983cbe23e8 in_seq=0 out_seq=0 2024-06-10T01:17:17.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.837+0000 7f9ab2ffd700 1 -- 172.21.15.7:0/2009109832 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f396e762d0 con 0x55f396fa1970 2024-06-10T01:17:17.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.837+0000 7f9ab2ffd700 10 client.27635 handle_command_reply: tid=0 2024-06-10T01:17:17.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.837+0000 7f9ae49f9b80 1 client.27635 shutdown 2024-06-10T01:17:17.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.837+0000 7f9ae49f9b80 1 -- 172.21.15.7:0/2009109832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f396f22f60 msgr2=0x55f396f2fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:17.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.837+0000 7f9ae49f9b80 1 --2- 172.21.15.7:0/2009109832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f396f22f60 0x55f396f2fa30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9abc005950 tx=0x7f9abc0047c0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.837+0000 7f9ae49f9b80 1 -- 172.21.15.7:0/2009109832 shutdown_connections 2024-06-10T01:17:17.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.837+0000 7f9ae49f9b80 1 --2- 172.21.15.7:0/2009109832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f396f23880 0x55f396f3db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.837+0000 7f9ae49f9b80 1 --2- 172.21.15.7:0/2009109832 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55f396fa1970 0x55f396fa3d90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.837+0000 7f9ae49f9b80 1 --2- 172.21.15.7:0/2009109832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f396f1d7a0 0x55f396f2f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.837+0000 7f9ae49f9b80 1 --2- 172.21.15.7:0/2009109832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f396f22f60 0x55f396f2fa30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.837+0000 7f9ab2ffd700 0 client.27635 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:17:17.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.837+0000 7f9ae49f9b80 1 -- 172.21.15.7:0/2009109832 >> 172.21.15.7:0/2009109832 conn(0x55f396e4a080 msgr2=0x55f396f28100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:17.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.837+0000 7f9ae49f9b80 1 -- 172.21.15.7:0/2009109832 shutdown_connections 2024-06-10T01:17:17.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.837+0000 7f9ae49f9b80 1 -- 172.21.15.7:0/2009109832 wait complete. 2024-06-10T01:17:17.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.837+0000 7f9ae49f9b80 20 client.27635 trim_cache size 0 max 16384 2024-06-10T01:17:17.836 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:17:17.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.838+0000 7f9ada176700 1 -- 172.21.15.7:0/1025877136 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9ac00a2ca0 msgr2=0x7f9ac00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:17.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.838+0000 7f9ada176700 1 --2- 172.21.15.7:0/1025877136 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9ac00a2ca0 0x7f9ac00a5160 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40788a0 tx=0x7f9abc00b410 comp rx=0 tx=0).stop 2024-06-10T01:17:17.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.838+0000 7f9ada176700 1 -- 172.21.15.7:0/1025877136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ad406aa00 msgr2=0x7f9ad419e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:17.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.838+0000 7f9ada176700 1 --2- 172.21.15.7:0/1025877136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ad406aa00 0x7f9ad419e7e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400ec10 tx=0x7f9ac400ef20 comp rx=0 tx=0).stop 2024-06-10T01:17:17.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.838+0000 7f9ada176700 1 -- 172.21.15.7:0/1025877136 shutdown_connections 2024-06-10T01:17:17.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.838+0000 7f9ada176700 1 --2- 172.21.15.7:0/1025877136 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9ac00a2ca0 0x7f9ac00a5160 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.838+0000 7f9ada176700 1 --2- 172.21.15.7:0/1025877136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ad41084d0 0x7f9ad41989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.838+0000 7f9ada176700 1 --2- 172.21.15.7:0/1025877136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ad406aa00 0x7f9ad419e7e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.838+0000 7f9ada176700 1 --2- 172.21.15.7:0/1025877136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ad406b3a0 0x7f9ad419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:17.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.838+0000 7f9ada176700 1 -- 172.21.15.7:0/1025877136 >> 172.21.15.7:0/1025877136 conn(0x7f9ad4078330 msgr2=0x7f9ad40ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:17.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.838+0000 7f9ada176700 1 -- 172.21.15.7:0/1025877136 shutdown_connections 2024-06-10T01:17:17.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:17.838+0000 7f9ada176700 1 -- 172.21.15.7:0/1025877136 wait complete. 2024-06-10T01:17:18.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.111+0000 7f3b45b10640 1 -- 172.21.15.7:0/90650374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b40104db0 msgr2=0x7f3b401071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:18.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.111+0000 7f3b45b10640 1 --2- 172.21.15.7:0/90650374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b40104db0 0x7f3b401071a0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f3b34009900 tx=0x7f3b3402a410 comp rx=0 tx=0).stop 2024-06-10T01:17:18.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.111+0000 7f3b45b10640 1 -- 172.21.15.7:0/90650374 shutdown_connections 2024-06-10T01:17:18.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.111+0000 7f3b45b10640 1 --2- 172.21.15.7:0/90650374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b401076e0 0x7f3b40109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.111+0000 7f3b45b10640 1 --2- 172.21.15.7:0/90650374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b40104db0 0x7f3b401071a0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.111+0000 7f3b45b10640 1 --2- 172.21.15.7:0/90650374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b40102480 0x7f3b40104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.111+0000 7f3b45b10640 1 -- 172.21.15.7:0/90650374 >> 172.21.15.7:0/90650374 conn(0x7f3b400fbe70 msgr2=0x7f3b400fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.111+0000 7f3b45b10640 1 -- 172.21.15.7:0/90650374 shutdown_connections 2024-06-10T01:17:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.111+0000 7f3b45b10640 1 -- 172.21.15.7:0/90650374 wait complete. 2024-06-10T01:17:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.112+0000 7f3b45b10640 1 Processor -- start 2024-06-10T01:17:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.112+0000 7f3b45b10640 1 -- start start 2024-06-10T01:17:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.112+0000 7f3b45b10640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b40102480 0x7f3b40101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.112+0000 7f3b45b10640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b40104db0 0x7f3b40100430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.112+0000 7f3b45b10640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b401076e0 0x7f3b40100970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.112+0000 7f3b45b10640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4010d670 con 0x7f3b401076e0 2024-06-10T01:17:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.112+0000 7f3b45b10640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4010d4f0 con 0x7f3b40104db0 2024-06-10T01:17:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.112+0000 7f3b45b10640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4010d7f0 con 0x7f3b40102480 2024-06-10T01:17:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.112+0000 7f3b3ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b401076e0 0x7f3b40100970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.112+0000 7f3b3f7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b40102480 0x7f3b40101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.112+0000 7f3b3ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b401076e0 0x7f3b40100970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37578/0 (socket says 172.21.15.7:37578) 2024-06-10T01:17:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.112+0000 7f3b3ffff640 1 -- 172.21.15.7:0/3263613218 learned_addr learned my addr 172.21.15.7:0/3263613218 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.112+0000 7f3b3effd640 1 --2- 172.21.15.7:0/3263613218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b40104db0 0x7f3b40100430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.113+0000 7f3b3ffff640 1 -- 172.21.15.7:0/3263613218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b40102480 msgr2=0x7f3b40101df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.113+0000 7f3b3ffff640 1 --2- 172.21.15.7:0/3263613218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b40102480 0x7f3b40101df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.113+0000 7f3b3ffff640 1 -- 172.21.15.7:0/3263613218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b40104db0 msgr2=0x7f3b40100430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.113+0000 7f3b3ffff640 1 --2- 172.21.15.7:0/3263613218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b40104db0 0x7f3b40100430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.113+0000 7f3b3ffff640 1 -- 172.21.15.7:0/3263613218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b40101200 con 0x7f3b401076e0 2024-06-10T01:17:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.113+0000 7f3b3f7fe640 1 --2- 172.21.15.7:0/3263613218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b40102480 0x7f3b40101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:17:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.113+0000 7f3b3effd640 1 --2- 172.21.15.7:0/3263613218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b40104db0 0x7f3b40100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:17:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.113+0000 7f3b3ffff640 1 --2- 172.21.15.7:0/3263613218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b401076e0 0x7f3b40100970 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800efd0 tx=0x7f3b3800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.113+0000 7f3b3cff9640 1 -- 172.21.15.7:0/3263613218 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38003b70 con 0x7f3b401076e0 2024-06-10T01:17:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.113+0000 7f3b45b10640 1 -- 172.21.15.7:0/3263613218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b401a1e40 con 0x7f3b401076e0 2024-06-10T01:17:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.113+0000 7f3b3cff9640 1 -- 172.21.15.7:0/3263613218 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b38003d10 con 0x7f3b401076e0 2024-06-10T01:17:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.113+0000 7f3b3cff9640 1 -- 172.21.15.7:0/3263613218 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b380109f0 con 0x7f3b401076e0 2024-06-10T01:17:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.113+0000 7f3b45b10640 1 -- 172.21.15.7:0/3263613218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b401a22a0 con 0x7f3b401076e0 2024-06-10T01:17:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.115+0000 7f3b3cff9640 1 -- 172.21.15.7:0/3263613218 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f3b38004020 con 0x7f3b401076e0 2024-06-10T01:17:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.116+0000 7f3b48818740 1 Processor -- start 2024-06-10T01:17:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.116+0000 7f3b48818740 1 -- start start 2024-06-10T01:17:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.116+0000 7f3b48818740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563022981010 0x563022981430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.116+0000 7f3b48818740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563022981a00 0x563022986500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.116+0000 7f3b48818740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563022986b70 0x563022989000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.116+0000 7f3b48818740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5630228e44f0 con 0x563022986b70 2024-06-10T01:17:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.116+0000 7f3b48818740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563022882870 con 0x563022981010 2024-06-10T01:17:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.116+0000 7f3b48818740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5630228e04b0 con 0x563022981a00 2024-06-10T01:17:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.116+0000 7f3b3effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563022986b70 0x563022989000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.116+0000 7f3b3effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563022986b70 0x563022989000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37582/0 (socket says 172.21.15.7:37582) 2024-06-10T01:17:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.116+0000 7f3b3f7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563022981a00 0x563022986500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.116+0000 7f3b3effd640 1 -- 172.21.15.7:0/991285315 learned_addr learned my addr 172.21.15.7:0/991285315 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.116+0000 7f3b3effd640 1 --2- 172.21.15.7:0/991285315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563022981010 0x563022981430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.116+0000 7f3b3cff9640 1 --2- 172.21.15.7:0/3263613218 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3b280a2c00 0x7f3b280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.117+0000 7f3b3ffff640 1 --2- 172.21.15.7:0/3263613218 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3b280a2c00 0x7f3b280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.117+0000 7f3b3f7fe640 1 -- 172.21.15.7:0/991285315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563022981010 msgr2=0x563022981430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.117+0000 7f3b3f7fe640 1 --2- 172.21.15.7:0/991285315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563022981010 0x563022981430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.117+0000 7f3b3f7fe640 1 -- 172.21.15.7:0/991285315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563022986b70 msgr2=0x563022989000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.117+0000 7f3b3f7fe640 1 --2- 172.21.15.7:0/991285315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563022986b70 0x563022989000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.117+0000 7f3b3f7fe640 1 -- 172.21.15.7:0/991285315 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56302289e9b0 con 0x563022981a00 2024-06-10T01:17:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.117+0000 7f3b3cff9640 1 -- 172.21.15.7:0/3263613218 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f3b380cfbf0 con 0x7f3b401076e0 2024-06-10T01:17:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.117+0000 7f3b3f7fe640 1 --2- 172.21.15.7:0/991285315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563022981a00 0x563022986500 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009fd0 tx=0x7f3b300118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f3e194e6ee8368b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.117+0000 7f3b3ffff640 1 --2- 172.21.15.7:0/3263613218 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3b280a2c00 0x7f3b280a50c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3b380984d0 tx=0x7f3b38004920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.117+0000 7f3b25ffb640 1 -- 172.21.15.7:0/991285315 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30004030 con 0x563022981a00 2024-06-10T01:17:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.117+0000 7f3b25ffb640 1 -- 172.21.15.7:0/991285315 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b30005500 con 0x563022981a00 2024-06-10T01:17:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.117+0000 7f3b25ffb640 1 -- 172.21.15.7:0/991285315 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30004d80 con 0x563022981a00 2024-06-10T01:17:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.118+0000 7f3b48818740 1 -- 172.21.15.7:0/991285315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563022981a00 msgr2=0x563022986500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.118+0000 7f3b48818740 1 --2- 172.21.15.7:0/991285315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563022981a00 0x563022986500 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009fd0 tx=0x7f3b300118d0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.118+0000 7f3b48818740 1 -- 172.21.15.7:0/991285315 shutdown_connections 2024-06-10T01:17:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.118+0000 7f3b48818740 1 --2- 172.21.15.7:0/991285315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563022986b70 0x563022989000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.118+0000 7f3b48818740 1 --2- 172.21.15.7:0/991285315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563022981a00 0x563022986500 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.118+0000 7f3b48818740 1 --2- 172.21.15.7:0/991285315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563022981010 0x563022981430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.118+0000 7f3b48818740 1 -- 172.21.15.7:0/991285315 >> 172.21.15.7:0/991285315 conn(0x56302297c680 msgr2=0x56302297ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.118+0000 7f3b48818740 1 -- 172.21.15.7:0/991285315 shutdown_connections 2024-06-10T01:17:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.118+0000 7f3b48818740 1 -- 172.21.15.7:0/991285315 wait complete. 2024-06-10T01:17:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.118+0000 7f3b48818740 1 -- start start 2024-06-10T01:17:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.119+0000 7f3b48818740 1 Processor -- start 2024-06-10T01:17:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.119+0000 7f3b48818740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563022981010 0x56302298b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.119+0000 7f3b48818740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563022981a00 0x56302298ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.119+0000 7f3b48818740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563022986b70 0x563022992a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.119+0000 7f3b48818740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563022989b20 con 0x563022981010 2024-06-10T01:17:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.119+0000 7f3b48818740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56302286dae0 con 0x563022981a00 2024-06-10T01:17:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.119+0000 7f3b48818740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563022989ca0 con 0x563022986b70 2024-06-10T01:17:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.119+0000 7f3b48818740 10 client.0 _ll_register_callbacks cb 0x5630228a69b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:17:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.119+0000 7f3b3f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563022981a00 0x56302298ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.119+0000 7f3b3f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563022981a00 0x56302298ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:40682/0 (socket says 172.21.15.7:40682) 2024-06-10T01:17:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.119+0000 7f3b3f7fe640 1 -- 172.21.15.7:0/3822154630 learned_addr learned my addr 172.21.15.7:0/3822154630 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.119+0000 7f3b3effd640 1 --2- 172.21.15.7:0/3822154630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563022981010 0x56302298b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.119+0000 7f3b3f7fe640 1 -- 172.21.15.7:0/3822154630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563022986b70 msgr2=0x563022992a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.119+0000 7f3b3f7fe640 1 --2- 172.21.15.7:0/3822154630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563022986b70 0x563022992a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.119+0000 7f3b3f7fe640 1 -- 172.21.15.7:0/3822154630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563022981010 msgr2=0x56302298b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.119+0000 7f3b3f7fe640 1 --2- 172.21.15.7:0/3822154630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563022981010 0x56302298b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.119+0000 7f3b3f7fe640 1 -- 172.21.15.7:0/3822154630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630225f8210 con 0x563022981a00 2024-06-10T01:17:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.119+0000 7f3b3f7fe640 1 --2- 172.21.15.7:0/3822154630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563022981a00 0x56302298ba00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3b3001b040 tx=0x7f3b300038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74f12830b548715d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.120+0000 7f3b1effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:17:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.120+0000 7f3b1effd640 1 -- 172.21.15.7:0/3822154630 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30028020 con 0x563022981a00 2024-06-10T01:17:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.120+0000 7f3b1effd640 1 -- 172.21.15.7:0/3822154630 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f3b30016040 con 0x563022981a00 2024-06-10T01:17:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.120+0000 7f3b48818740 1 -- 172.21.15.7:0/3822154630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b30005500 con 0x563022981a00 2024-06-10T01:17:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.120+0000 7f3b1effd640 1 -- 172.21.15.7:0/3822154630 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f3b30004470 con 0x563022981a00 2024-06-10T01:17:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.120+0000 7f3b48818740 10 client.27647 fetch_fsmap learned FSMap version 40 2024-06-10T01:17:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.120+0000 7f3b48818740 1 -- 172.21.15.7:0/3822154630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5630226c4620 con 0x563022981a00 2024-06-10T01:17:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.120+0000 7f3b1effd640 1 -- 172.21.15.7:0/3822154630 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b30005500 con 0x563022981a00 2024-06-10T01:17:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.120+0000 7f3b1effd640 1 -- 172.21.15.7:0/3822154630 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f3b300051a0 con 0x563022981a00 2024-06-10T01:17:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.120+0000 7f3b48818740 10 client.27647 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:17:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.120+0000 7f3b48818740 10 client.27647 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:17:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.120+0000 7f3b48818740 20 client.27647 populate_metadata read hostname 'smithi007' 2024-06-10T01:17:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.120+0000 7f3b48818740 1 --2- 172.21.15.7:0/3822154630 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5630229971e0 0x5630229995f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.120+0000 7f3b48818740 4 client.27647 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:17:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.120+0000 7f3b48818740 1 -- 172.21.15.7:0/3822154630 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630228f7f10 con 0x5630229971e0 2024-06-10T01:17:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.121+0000 7f3b3effd640 1 --2- 172.21.15.7:0/3822154630 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5630229971e0 0x5630229995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.121+0000 7f3b3effd640 1 --2- 172.21.15.7:0/3822154630 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5630229971e0 0x5630229995f0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=347164862f389b79 server_cookie=8ccfabf279072794 in_seq=0 out_seq=0 2024-06-10T01:17:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.121+0000 7f3b1effd640 10 client.27647 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.122+0000 7f3b1effd640 1 -- 172.21.15.7:0/3822154630 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5630228f7f10 con 0x5630229971e0 2024-06-10T01:17:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.122+0000 7f3b1effd640 10 client.27647 handle_command_reply: tid=0 2024-06-10T01:17:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.122+0000 7f3b48818740 1 client.27647 shutdown 2024-06-10T01:17:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.122+0000 7f3b48818740 1 -- 172.21.15.7:0/3822154630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563022981a00 msgr2=0x56302298ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.122+0000 7f3b48818740 1 --2- 172.21.15.7:0/3822154630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563022981a00 0x56302298ba00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3b3001b040 tx=0x7f3b300038a0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.122+0000 7f3b48818740 1 -- 172.21.15.7:0/3822154630 shutdown_connections 2024-06-10T01:17:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.122+0000 7f3b48818740 1 --2- 172.21.15.7:0/3822154630 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5630229971e0 0x5630229995f0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.122+0000 7f3b48818740 1 --2- 172.21.15.7:0/3822154630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563022986b70 0x563022992a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.122+0000 7f3b48818740 1 --2- 172.21.15.7:0/3822154630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563022981a00 0x56302298ba00 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.122+0000 7f3b48818740 1 --2- 172.21.15.7:0/3822154630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563022981010 0x56302298b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.122+0000 7f3b48818740 1 -- 172.21.15.7:0/3822154630 >> 172.21.15.7:0/3822154630 conn(0x56302297c680 msgr2=0x56302297b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.122+0000 7f3b1effd640 0 client.27647 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.122+0000 7f3b48818740 1 -- 172.21.15.7:0/3822154630 shutdown_connections 2024-06-10T01:17:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.122+0000 7f3b48818740 1 -- 172.21.15.7:0/3822154630 wait complete. 2024-06-10T01:17:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.123+0000 7f3b48818740 20 client.27647 trim_cache size 0 max 16384 2024-06-10T01:17:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.139+0000 7f3b48818740 1 Processor -- start 2024-06-10T01:17:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.139+0000 7f3b48818740 1 -- start start 2024-06-10T01:17:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.139+0000 7f3b48818740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563022981010 0x56302297af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.139+0000 7f3b48818740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563022981a00 0x563022992a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.139+0000 7f3b48818740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563022986b70 0x56302297b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.139+0000 7f3b48818740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401af0b0 con 0x563022981a00 2024-06-10T01:17:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.139+0000 7f3b48818740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401ac980 con 0x563022986b70 2024-06-10T01:17:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.139+0000 7f3b48818740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401aa7d0 con 0x563022981010 2024-06-10T01:17:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.139+0000 7f3b3f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563022981a00 0x563022992a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.139+0000 7f3b3f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563022981a00 0x563022992a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37590/0 (socket says 172.21.15.7:37590) 2024-06-10T01:17:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.139+0000 7f3b3f7fe640 1 -- 172.21.15.7:0/2939324166 learned_addr learned my addr 172.21.15.7:0/2939324166 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.139+0000 7f3b3effd640 1 --2- 172.21.15.7:0/2939324166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563022986b70 0x56302297b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.140+0000 7f3b3effd640 1 --2- 172.21.15.7:0/2939324166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563022981010 0x56302297af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.140+0000 7f3b3f7fe640 1 -- 172.21.15.7:0/2939324166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563022981010 msgr2=0x56302297af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.140+0000 7f3b3f7fe640 1 --2- 172.21.15.7:0/2939324166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563022981010 0x56302297af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.140+0000 7f3b3f7fe640 1 -- 172.21.15.7:0/2939324166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563022986b70 msgr2=0x56302297b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.140+0000 7f3b3f7fe640 1 --2- 172.21.15.7:0/2939324166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563022986b70 0x56302297b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.140+0000 7f3b3f7fe640 1 -- 172.21.15.7:0/2939324166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563022981770 con 0x563022981a00 2024-06-10T01:17:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.140+0000 7f3b3f7fe640 1 --2- 172.21.15.7:0/2939324166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563022981a00 0x563022992a00 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000f800 tx=0x7f3b30005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5734731d0c39fb5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.140+0000 7f3b257fa640 1 -- 172.21.15.7:0/2939324166 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30028040 con 0x563022981a00 2024-06-10T01:17:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.140+0000 7f3b257fa640 1 -- 172.21.15.7:0/2939324166 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b30005520 con 0x563022981a00 2024-06-10T01:17:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.140+0000 7f3b257fa640 1 -- 172.21.15.7:0/2939324166 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30004610 con 0x563022981a00 2024-06-10T01:17:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.141+0000 7f3b48818740 1 -- 172.21.15.7:0/2939324166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563022981a00 msgr2=0x563022992a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.141+0000 7f3b48818740 1 --2- 172.21.15.7:0/2939324166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563022981a00 0x563022992a00 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000f800 tx=0x7f3b30005df0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.141+0000 7f3b48818740 1 -- 172.21.15.7:0/2939324166 shutdown_connections 2024-06-10T01:17:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.141+0000 7f3b48818740 1 --2- 172.21.15.7:0/2939324166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563022986b70 0x56302297b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.141+0000 7f3b48818740 1 --2- 172.21.15.7:0/2939324166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563022981a00 0x563022992a00 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.141+0000 7f3b48818740 1 --2- 172.21.15.7:0/2939324166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563022981010 0x56302297af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.141+0000 7f3b48818740 1 -- 172.21.15.7:0/2939324166 >> 172.21.15.7:0/2939324166 conn(0x56302297c680 msgr2=0x563022993490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.141+0000 7f3b48818740 1 -- 172.21.15.7:0/2939324166 shutdown_connections 2024-06-10T01:17:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.141+0000 7f3b48818740 1 -- 172.21.15.7:0/2939324166 wait complete. 2024-06-10T01:17:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.141+0000 7f3b48818740 1 -- start start 2024-06-10T01:17:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.141+0000 7f3b48818740 1 Processor -- start 2024-06-10T01:17:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.142+0000 7f3b48818740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563022981010 0x56302298f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.142+0000 7f3b48818740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563022981a00 0x56302298fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.142+0000 7f3b48818740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563022986b70 0x5630229ffec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.142+0000 7f3b48818740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401aae40 con 0x563022981010 2024-06-10T01:17:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.142+0000 7f3b48818740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401ac2b0 con 0x563022986b70 2024-06-10T01:17:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.142+0000 7f3b48818740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401ab510 con 0x563022981a00 2024-06-10T01:17:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.142+0000 7f3b48818740 10 client.0 _ll_register_callbacks cb 0x7f3b380093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:17:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.142+0000 7f3b3effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563022981010 0x56302298f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.142+0000 7f3b3effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563022981010 0x56302298f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37602/0 (socket says 172.21.15.7:37602) 2024-06-10T01:17:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.142+0000 7f3b3effd640 1 -- 172.21.15.7:0/546260785 learned_addr learned my addr 172.21.15.7:0/546260785 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.142+0000 7f3b3f7fe640 1 --2- 172.21.15.7:0/546260785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563022981a00 0x56302298fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.142+0000 7f3b3effd640 1 --2- 172.21.15.7:0/546260785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563022986b70 0x5630229ffec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.142+0000 7f3b3effd640 1 -- 172.21.15.7:0/546260785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563022981a00 msgr2=0x56302298fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.142+0000 7f3b3effd640 1 --2- 172.21.15.7:0/546260785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563022981a00 0x56302298fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.142+0000 7f3b3effd640 1 -- 172.21.15.7:0/546260785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563022986b70 msgr2=0x5630229ffec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.142+0000 7f3b3effd640 1 --2- 172.21.15.7:0/546260785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563022986b70 0x5630229ffec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.142+0000 7f3b3effd640 1 -- 172.21.15.7:0/546260785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563022992bd0 con 0x563022981010 2024-06-10T01:17:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.142+0000 7f3b3effd640 1 --2- 172.21.15.7:0/546260785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563022981010 0x56302298f610 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f3b340314e0 tx=0x7f3b30004350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b915815f816e0f25 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.142+0000 7f3b1effd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:17:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.143+0000 7f3b1effd640 1 -- 172.21.15.7:0/546260785 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34035040 con 0x563022981010 2024-06-10T01:17:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.143+0000 7f3b48818740 1 -- 172.21.15.7:0/546260785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b30005520 con 0x563022981010 2024-06-10T01:17:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.143+0000 7f3b1effd640 1 -- 172.21.15.7:0/546260785 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f3b340035f0 con 0x563022981010 2024-06-10T01:17:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.143+0000 7f3b1effd640 1 -- 172.21.15.7:0/546260785 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f3b34003810 con 0x563022981010 2024-06-10T01:17:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.143+0000 7f3b48818740 10 client.18315 fetch_fsmap learned FSMap version 40 2024-06-10T01:17:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.143+0000 7f3b48818740 1 -- 172.21.15.7:0/546260785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563022a01460 con 0x563022981010 2024-06-10T01:17:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.143+0000 7f3b1effd640 1 -- 172.21.15.7:0/546260785 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b30005520 con 0x563022981010 2024-06-10T01:17:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.143+0000 7f3b1effd640 1 -- 172.21.15.7:0/546260785 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f3b3403a040 con 0x563022981010 2024-06-10T01:17:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.143+0000 7f3b48818740 10 client.18315 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:17:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.143+0000 7f3b48818740 10 client.18315 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:17:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.143+0000 7f3b48818740 20 client.18315 populate_metadata read hostname 'smithi007' 2024-06-10T01:17:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.143+0000 7f3b48818740 1 --2- 172.21.15.7:0/546260785 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563022a01d90 0x563022a041b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.143+0000 7f3b48818740 4 client.18315 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:17:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.143+0000 7f3b48818740 1 -- 172.21.15.7:0/546260785 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5630228f1c20 con 0x563022a01d90 2024-06-10T01:17:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.143+0000 7f3b3f7fe640 1 --2- 172.21.15.7:0/546260785 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563022a01d90 0x563022a041b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.144+0000 7f3b1effd640 10 client.18315 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.144+0000 7f3b3f7fe640 1 --2- 172.21.15.7:0/546260785 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563022a01d90 0x563022a041b0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=962e448d96769efe server_cookie=7ad1db8964029879 in_seq=0 out_seq=0 2024-06-10T01:17:18.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.234+0000 7f3b1effd640 1 -- 172.21.15.7:0/546260785 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5630228f1c20 con 0x563022a01d90 2024-06-10T01:17:18.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.234+0000 7f3b1effd640 10 client.18315 handle_command_reply: tid=0 2024-06-10T01:17:18.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.234+0000 7f3b48818740 1 client.18315 shutdown 2024-06-10T01:17:18.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.234+0000 7f3b48818740 1 -- 172.21.15.7:0/546260785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563022981010 msgr2=0x56302298f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:18.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.234+0000 7f3b48818740 1 --2- 172.21.15.7:0/546260785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563022981010 0x56302298f610 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f3b340314e0 tx=0x7f3b30004350 comp rx=0 tx=0).stop 2024-06-10T01:17:18.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.234+0000 7f3b48818740 1 -- 172.21.15.7:0/546260785 shutdown_connections 2024-06-10T01:17:18.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.234+0000 7f3b48818740 1 --2- 172.21.15.7:0/546260785 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563022a01d90 0x563022a041b0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.234+0000 7f3b48818740 1 --2- 172.21.15.7:0/546260785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563022986b70 0x5630229ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.234+0000 7f3b48818740 1 --2- 172.21.15.7:0/546260785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563022981a00 0x56302298fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.234+0000 7f3b48818740 1 --2- 172.21.15.7:0/546260785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563022981010 0x56302298f610 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.234+0000 7f3b48818740 1 -- 172.21.15.7:0/546260785 >> 172.21.15.7:0/546260785 conn(0x56302297c680 msgr2=0x56302297f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:18.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.234+0000 7f3b1effd640 0 client.18315 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:18.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.235+0000 7f3b48818740 1 -- 172.21.15.7:0/546260785 shutdown_connections 2024-06-10T01:17:18.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.235+0000 7f3b48818740 1 -- 172.21.15.7:0/546260785 wait complete. 2024-06-10T01:17:18.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.235+0000 7f3b48818740 20 client.18315 trim_cache size 0 max 16384 2024-06-10T01:17:18.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:17:18.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:17:18.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:17:18.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.235+0000 7f3b45b10640 1 -- 172.21.15.7:0/3263613218 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3b280a2c00 msgr2=0x7f3b280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:18.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.235+0000 7f3b45b10640 1 --2- 172.21.15.7:0/3263613218 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3b280a2c00 0x7f3b280a50c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3b380984d0 tx=0x7f3b38004920 comp rx=0 tx=0).stop 2024-06-10T01:17:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.235+0000 7f3b45b10640 1 -- 172.21.15.7:0/3263613218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b401076e0 msgr2=0x7f3b40100970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.235+0000 7f3b45b10640 1 --2- 172.21.15.7:0/3263613218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b401076e0 0x7f3b40100970 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800efd0 tx=0x7f3b3800c6a0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.235+0000 7f3b45b10640 1 -- 172.21.15.7:0/3263613218 shutdown_connections 2024-06-10T01:17:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.235+0000 7f3b45b10640 1 --2- 172.21.15.7:0/3263613218 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3b280a2c00 0x7f3b280a50c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.235+0000 7f3b45b10640 1 --2- 172.21.15.7:0/3263613218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b401076e0 0x7f3b40100970 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.235+0000 7f3b45b10640 1 --2- 172.21.15.7:0/3263613218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b40104db0 0x7f3b40100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.235+0000 7f3b45b10640 1 --2- 172.21.15.7:0/3263613218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b40102480 0x7f3b40101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.235+0000 7f3b45b10640 1 -- 172.21.15.7:0/3263613218 >> 172.21.15.7:0/3263613218 conn(0x7f3b400fbe70 msgr2=0x7f3b40102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.236+0000 7f3b45b10640 1 -- 172.21.15.7:0/3263613218 shutdown_connections 2024-06-10T01:17:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:18.236+0000 7f3b45b10640 1 -- 172.21.15.7:0/3263613218 wait complete. 2024-06-10T01:17:18.505 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:17:18.751 INFO:tasks.workunit.client.0.smithi007.stdout: 1 833982 7.46 MB/sec execute 479 sec latency 70.328 ms 2024-06-10T01:17:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:17:19.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:18 smithi007 ceph-mon[32398]: pgmap v771: 129 pgs: 129 active+clean; 138 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 1.9 MiB/s wr, 171 op/s 2024-06-10T01:17:19.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:18 smithi078 ceph-mon[36683]: pgmap v771: 129 pgs: 129 active+clean; 138 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 1.9 MiB/s wr, 171 op/s 2024-06-10T01:17:19.452 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:18 smithi175 ceph-mon[36636]: pgmap v771: 129 pgs: 129 active+clean; 138 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 1.9 MiB/s wr, 171 op/s 2024-06-10T01:17:19.751 INFO:tasks.workunit.client.0.smithi007.stdout: 1 835377 7.46 MB/sec execute 480 sec latency 18.136 ms 2024-06-10T01:17:20.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.470+0000 7f3031a0b700 1 -- 172.21.15.7:0/2581987923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f302c06ab60 msgr2=0x7f302c06afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:20.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.470+0000 7f3031a0b700 1 --2- 172.21.15.7:0/2581987923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f302c06ab60 0x7f302c06afe0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f3014009990 tx=0x7f3014009ca0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.470+0000 7f3031a0b700 1 -- 172.21.15.7:0/2581987923 shutdown_connections 2024-06-10T01:17:20.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.470+0000 7f3031a0b700 1 --2- 172.21.15.7:0/2581987923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f302c06b520 0x7f302c10f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.470+0000 7f3031a0b700 1 --2- 172.21.15.7:0/2581987923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f302c108520 0x7f302c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.470+0000 7f3031a0b700 1 --2- 172.21.15.7:0/2581987923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f302c06ab60 0x7f302c06afe0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.470+0000 7f3031a0b700 1 -- 172.21.15.7:0/2581987923 >> 172.21.15.7:0/2581987923 conn(0x7f302c078340 msgr2=0x7f302c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:20.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.470+0000 7f3031a0b700 1 -- 172.21.15.7:0/2581987923 shutdown_connections 2024-06-10T01:17:20.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.470+0000 7f3031a0b700 1 -- 172.21.15.7:0/2581987923 wait complete. 2024-06-10T01:17:20.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.471+0000 7f3031a0b700 1 Processor -- start 2024-06-10T01:17:20.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.471+0000 7f3031a0b700 1 -- start start 2024-06-10T01:17:20.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.472+0000 7f3031a0b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f302c06ab60 0x7f302c101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:20.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.472+0000 7f3031a0b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f302c06b520 0x7f302c101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.472+0000 7f3031a0b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f302c108520 0x7f302c1022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.472+0000 7f3031a0b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f302c112780 con 0x7f302c108520 2024-06-10T01:17:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.472+0000 7f3031a0b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f302c112600 con 0x7f302c06b520 2024-06-10T01:17:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.472+0000 7f3031a0b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f302c112900 con 0x7f302c06ab60 2024-06-10T01:17:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.472+0000 7f30227fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f302c06b520 0x7f302c101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.472+0000 7f302b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f302c108520 0x7f302c1022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.472+0000 7f30227fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f302c06b520 0x7f302c101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:40734/0 (socket says 172.21.15.7:40734) 2024-06-10T01:17:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.472+0000 7f30227fc700 1 -- 172.21.15.7:0/4045054175 learned_addr learned my addr 172.21.15.7:0/4045054175 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.472+0000 7f302affd700 1 --2- 172.21.15.7:0/4045054175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f302c06ab60 0x7f302c101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.473+0000 7f302b7fe700 1 -- 172.21.15.7:0/4045054175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f302c06ab60 msgr2=0x7f302c101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.473+0000 7f302b7fe700 1 --2- 172.21.15.7:0/4045054175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f302c06ab60 0x7f302c101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.473+0000 7f302b7fe700 1 -- 172.21.15.7:0/4045054175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f302c06b520 msgr2=0x7f302c101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.473+0000 7f302b7fe700 1 --2- 172.21.15.7:0/4045054175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f302c06b520 0x7f302c101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.473+0000 7f302b7fe700 1 -- 172.21.15.7:0/4045054175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f302c1ab5e0 con 0x7f302c108520 2024-06-10T01:17:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.473+0000 7f30227fc700 1 --2- 172.21.15.7:0/4045054175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f302c06b520 0x7f302c101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:17:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.473+0000 7f302b7fe700 1 --2- 172.21.15.7:0/4045054175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f302c108520 0x7f302c1022c0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f302400efb0 tx=0x7f302400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.473+0000 7f3030a09700 1 -- 172.21.15.7:0/4045054175 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3024003a00 con 0x7f302c108520 2024-06-10T01:17:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.473+0000 7f3031a0b700 1 -- 172.21.15.7:0/4045054175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f302c1ab870 con 0x7f302c108520 2024-06-10T01:17:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.473+0000 7f3030a09700 1 -- 172.21.15.7:0/4045054175 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3024003ba0 con 0x7f302c108520 2024-06-10T01:17:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.473+0000 7f3030a09700 1 -- 172.21.15.7:0/4045054175 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3024004850 con 0x7f302c108520 2024-06-10T01:17:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.473+0000 7f3031a0b700 1 -- 172.21.15.7:0/4045054175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f302c1abe00 con 0x7f302c108520 2024-06-10T01:17:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.476+0000 7f3030a09700 1 -- 172.21.15.7:0/4045054175 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f3024012050 con 0x7f302c108520 2024-06-10T01:17:20.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.476+0000 7f3030a09700 1 --2- 172.21.15.7:0/4045054175 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f30180a2c20 0x7f30180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:20.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.477+0000 7f3030a09700 1 -- 172.21.15.7:0/4045054175 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f30240cedc0 con 0x7f302c108520 2024-06-10T01:17:20.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.477+0000 7f302affd700 1 --2- 172.21.15.7:0/4045054175 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f30180a2c20 0x7f30180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:20.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.477+0000 7f303c28eb80 1 Processor -- start 2024-06-10T01:17:20.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.477+0000 7f302affd700 1 --2- 172.21.15.7:0/4045054175 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f30180a2c20 0x7f30180a50e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f301c005950 tx=0x7f301c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:20.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.477+0000 7f303c28eb80 1 -- start start 2024-06-10T01:17:20.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.477+0000 7f303c28eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560ee9832640 0x560ee9832a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:20.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.477+0000 7f303c28eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ee9832f60 0x560ee982c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.477+0000 7f303c28eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ee982cf20 0x560ee982d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.477+0000 7f303c28eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560ee977e810 con 0x560ee9832640 2024-06-10T01:17:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.477+0000 7f303c28eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560ee9726720 con 0x560ee9832f60 2024-06-10T01:17:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.477+0000 7f303c28eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560ee977a7d0 con 0x560ee982cf20 2024-06-10T01:17:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.477+0000 7f302affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560ee9832640 0x560ee9832a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.477+0000 7f302affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560ee9832640 0x560ee9832a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37614/0 (socket says 172.21.15.7:37614) 2024-06-10T01:17:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.477+0000 7f302affd700 1 -- 172.21.15.7:0/414682572 learned_addr learned my addr 172.21.15.7:0/414682572 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.478+0000 7f30227fc700 1 --2- 172.21.15.7:0/414682572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ee9832f60 0x560ee982c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.478+0000 7f302b7fe700 1 --2- 172.21.15.7:0/414682572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ee982cf20 0x560ee982d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.478+0000 7f302affd700 1 -- 172.21.15.7:0/414682572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ee982cf20 msgr2=0x560ee982d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.478+0000 7f302affd700 1 --2- 172.21.15.7:0/414682572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ee982cf20 0x560ee982d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.478+0000 7f302affd700 1 -- 172.21.15.7:0/414682572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ee9832f60 msgr2=0x560ee982c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.478+0000 7f302affd700 1 --2- 172.21.15.7:0/414682572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ee9832f60 0x560ee982c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.478+0000 7f302affd700 1 -- 172.21.15.7:0/414682572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ee971cf50 con 0x560ee9832640 2024-06-10T01:17:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.478+0000 7f30227fc700 1 --2- 172.21.15.7:0/414682572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ee9832f60 0x560ee982c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:17:20.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.478+0000 7f302affd700 1 --2- 172.21.15.7:0/414682572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560ee9832640 0x560ee9832a20 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f301c010e60 tx=0x7f301c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b4fdce803215333 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:20.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.478+0000 7f3021ffb700 1 -- 172.21.15.7:0/414682572 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c011db0 con 0x560ee9832640 2024-06-10T01:17:20.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.478+0000 7f3021ffb700 1 -- 172.21.15.7:0/414682572 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f301c010e90 con 0x560ee9832640 2024-06-10T01:17:20.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.478+0000 7f3021ffb700 1 -- 172.21.15.7:0/414682572 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c01c790 con 0x560ee9832640 2024-06-10T01:17:20.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.479+0000 7f303c28eb80 1 -- 172.21.15.7:0/414682572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560ee9832640 msgr2=0x560ee9832a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:20.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.479+0000 7f303c28eb80 1 --2- 172.21.15.7:0/414682572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560ee9832640 0x560ee9832a20 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f301c010e60 tx=0x7f301c013000 comp rx=0 tx=0).stop 2024-06-10T01:17:20.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.479+0000 7f303c28eb80 1 -- 172.21.15.7:0/414682572 shutdown_connections 2024-06-10T01:17:20.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.479+0000 7f303c28eb80 1 --2- 172.21.15.7:0/414682572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ee982cf20 0x560ee982d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.479+0000 7f303c28eb80 1 --2- 172.21.15.7:0/414682572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560ee9832640 0x560ee9832a20 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.479+0000 7f303c28eb80 1 --2- 172.21.15.7:0/414682572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ee9832f60 0x560ee982c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.479+0000 7f303c28eb80 1 -- 172.21.15.7:0/414682572 >> 172.21.15.7:0/414682572 conn(0x560ee9757cc0 msgr2=0x560ee9827850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:20.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.479+0000 7f303c28eb80 1 -- 172.21.15.7:0/414682572 shutdown_connections 2024-06-10T01:17:20.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.479+0000 7f303c28eb80 1 -- 172.21.15.7:0/414682572 wait complete. 2024-06-10T01:17:20.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.479+0000 7f303c28eb80 1 -- start start 2024-06-10T01:17:20.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.480+0000 7f303c28eb80 1 Processor -- start 2024-06-10T01:17:20.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.480+0000 7f303c28eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ee982cf20 0x560ee9838cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:20.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.480+0000 7f303c28eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ee9832640 0x560ee9835fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:20.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.480+0000 7f303c28eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560ee9832f60 0x560ee9836510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:20.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.480+0000 7f303c28eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560ee98399d0 con 0x560ee9832f60 2024-06-10T01:17:20.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.480+0000 7f303c28eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560ee971ebf0 con 0x560ee982cf20 2024-06-10T01:17:20.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.480+0000 7f303c28eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560ee9839b50 con 0x560ee9832640 2024-06-10T01:17:20.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.480+0000 7f303c28eb80 10 client.0 _ll_register_callbacks cb 0x560ee9718920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:17:20.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.480+0000 7f302b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560ee9832f60 0x560ee9836510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:20.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.480+0000 7f302affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ee982cf20 0x560ee9838cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:20.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.480+0000 7f30227fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ee9832640 0x560ee9835fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:20.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.480+0000 7f302affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ee982cf20 0x560ee9838cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:40748/0 (socket says 172.21.15.7:40748) 2024-06-10T01:17:20.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.480+0000 7f302affd700 1 -- 172.21.15.7:0/4109240940 learned_addr learned my addr 172.21.15.7:0/4109240940 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:20.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.481+0000 7f302b7fe700 1 -- 172.21.15.7:0/4109240940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ee9832640 msgr2=0x560ee9835fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:20.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.481+0000 7f302b7fe700 1 --2- 172.21.15.7:0/4109240940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ee9832640 0x560ee9835fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.481+0000 7f302b7fe700 1 -- 172.21.15.7:0/4109240940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ee982cf20 msgr2=0x560ee9838cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:20.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.481+0000 7f302b7fe700 1 --2- 172.21.15.7:0/4109240940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ee982cf20 0x560ee9838cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.481+0000 7f302b7fe700 1 -- 172.21.15.7:0/4109240940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ee96cd8d0 con 0x560ee9832f60 2024-06-10T01:17:20.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.481+0000 7f30227fc700 1 --2- 172.21.15.7:0/4109240940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ee9832640 0x560ee9835fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:17:20.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.481+0000 7f302b7fe700 1 --2- 172.21.15.7:0/4109240940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560ee9832f60 0x560ee9836510 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f302400c6f0 tx=0x7f302409b230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbfc160cd62adb5e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:20.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.481+0000 7f300affd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:17:20.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.481+0000 7f300affd700 1 -- 172.21.15.7:0/4109240940 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3024004dd0 con 0x560ee9832f60 2024-06-10T01:17:20.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.481+0000 7f300affd700 1 -- 172.21.15.7:0/4109240940 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f30240a3a80 con 0x560ee9832f60 2024-06-10T01:17:20.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.481+0000 7f303c28eb80 1 -- 172.21.15.7:0/4109240940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f301c010e90 con 0x560ee9832f60 2024-06-10T01:17:20.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.481+0000 7f300affd700 1 -- 172.21.15.7:0/4109240940 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f3024003d50 con 0x560ee9832f60 2024-06-10T01:17:20.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.481+0000 7f303c28eb80 10 client.18327 fetch_fsmap learned FSMap version 40 2024-06-10T01:17:20.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.481+0000 7f303c28eb80 1 -- 172.21.15.7:0/4109240940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560ee9844360 con 0x560ee9832f60 2024-06-10T01:17:20.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.482+0000 7f300affd700 1 -- 172.21.15.7:0/4109240940 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f301c010e90 con 0x560ee9832f60 2024-06-10T01:17:20.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.482+0000 7f300affd700 1 -- 172.21.15.7:0/4109240940 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f30240bb0c0 con 0x560ee9832f60 2024-06-10T01:17:20.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.482+0000 7f303c28eb80 10 client.18327 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:17:20.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.482+0000 7f303c28eb80 10 client.18327 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:17:20.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.482+0000 7f303c28eb80 20 client.18327 populate_metadata read hostname 'smithi007' 2024-06-10T01:17:20.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.482+0000 7f303c28eb80 1 --2- 172.21.15.7:0/4109240940 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x560ee98469a0 0x560ee9848d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:20.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.482+0000 7f303c28eb80 4 client.18327 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:17:20.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.482+0000 7f303c28eb80 1 -- 172.21.15.7:0/4109240940 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ee978bd10 con 0x560ee98469a0 2024-06-10T01:17:20.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.482+0000 7f302affd700 1 --2- 172.21.15.7:0/4109240940 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x560ee98469a0 0x560ee9848d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:20.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.483+0000 7f300affd700 10 client.18327 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:17:20.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.483+0000 7f302affd700 1 --2- 172.21.15.7:0/4109240940 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x560ee98469a0 0x560ee9848d90 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef5a57986af529e server_cookie=13da94a0973d6de9 in_seq=0 out_seq=0 2024-06-10T01:17:20.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.484+0000 7f300affd700 1 -- 172.21.15.7:0/4109240940 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560ee978bd10 con 0x560ee98469a0 2024-06-10T01:17:20.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.484+0000 7f300affd700 10 client.18327 handle_command_reply: tid=0 2024-06-10T01:17:20.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.484+0000 7f303c28eb80 1 client.18327 shutdown 2024-06-10T01:17:20.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.484+0000 7f303c28eb80 1 -- 172.21.15.7:0/4109240940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560ee9832f60 msgr2=0x560ee9836510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:20.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.484+0000 7f303c28eb80 1 --2- 172.21.15.7:0/4109240940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560ee9832f60 0x560ee9836510 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f302400c6f0 tx=0x7f302409b230 comp rx=0 tx=0).stop 2024-06-10T01:17:20.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.484+0000 7f303c28eb80 1 -- 172.21.15.7:0/4109240940 shutdown_connections 2024-06-10T01:17:20.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.484+0000 7f303c28eb80 1 --2- 172.21.15.7:0/4109240940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560ee9832f60 0x560ee9836510 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.484+0000 7f303c28eb80 1 --2- 172.21.15.7:0/4109240940 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x560ee98469a0 0x560ee9848d90 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.484+0000 7f300affd700 0 client.18327 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:17:20.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.484+0000 7f303c28eb80 1 --2- 172.21.15.7:0/4109240940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ee982cf20 0x560ee9838cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.484+0000 7f303c28eb80 1 --2- 172.21.15.7:0/4109240940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ee9832640 0x560ee9835fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.484+0000 7f303c28eb80 1 -- 172.21.15.7:0/4109240940 >> 172.21.15.7:0/4109240940 conn(0x560ee9757cc0 msgr2=0x560ee982b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:20.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.484+0000 7f303c28eb80 1 -- 172.21.15.7:0/4109240940 shutdown_connections 2024-06-10T01:17:20.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.484+0000 7f303c28eb80 1 -- 172.21.15.7:0/4109240940 wait complete. 2024-06-10T01:17:20.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.485+0000 7f303c28eb80 20 client.18327 trim_cache size 0 max 16384 2024-06-10T01:17:20.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.503+0000 7f303c28eb80 1 Processor -- start 2024-06-10T01:17:20.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.503+0000 7f303c28eb80 1 -- start start 2024-06-10T01:17:20.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.503+0000 7f303c28eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560ee982cf20 0x560ee982c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:20.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.503+0000 7f303c28eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ee9832640 0x560ee9833bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:20.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.503+0000 7f303c28eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ee9832f60 0x560ee9834f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:20.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.503+0000 7f303c28eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f302c1b72e0 con 0x560ee982cf20 2024-06-10T01:17:20.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.503+0000 7f303c28eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f302c1b4b70 con 0x560ee9832f60 2024-06-10T01:17:20.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.503+0000 7f303c28eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f302c1b29c0 con 0x560ee9832640 2024-06-10T01:17:20.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.503+0000 7f302affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560ee982cf20 0x560ee982c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:20.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.503+0000 7f302affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560ee982cf20 0x560ee982c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37640/0 (socket says 172.21.15.7:37640) 2024-06-10T01:17:20.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.503+0000 7f302affd700 1 -- 172.21.15.7:0/239748028 learned_addr learned my addr 172.21.15.7:0/239748028 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:20.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.503+0000 7f30227fc700 1 --2- 172.21.15.7:0/239748028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ee9832640 0x560ee9833bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:20.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.503+0000 7f302b7fe700 1 --2- 172.21.15.7:0/239748028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ee9832f60 0x560ee9834f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:20.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.504+0000 7f302affd700 1 -- 172.21.15.7:0/239748028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ee9832640 msgr2=0x560ee9833bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:20.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.504+0000 7f302affd700 1 --2- 172.21.15.7:0/239748028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ee9832640 0x560ee9833bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.504+0000 7f302affd700 1 -- 172.21.15.7:0/239748028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ee9832f60 msgr2=0x560ee9834f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:20.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.504+0000 7f302affd700 1 --2- 172.21.15.7:0/239748028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ee9832f60 0x560ee9834f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.504+0000 7f302affd700 1 -- 172.21.15.7:0/239748028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ee9828720 con 0x560ee982cf20 2024-06-10T01:17:20.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.504+0000 7f302affd700 1 --2- 172.21.15.7:0/239748028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560ee982cf20 0x560ee982c9e0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f301c011a00 tx=0x7f301c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5af5b4958670179d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:20.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.504+0000 7f30217fa700 1 -- 172.21.15.7:0/239748028 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c037020 con 0x560ee982cf20 2024-06-10T01:17:20.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.504+0000 7f30217fa700 1 -- 172.21.15.7:0/239748028 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f301c011e50 con 0x560ee982cf20 2024-06-10T01:17:20.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.504+0000 7f30217fa700 1 -- 172.21.15.7:0/239748028 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c01e420 con 0x560ee982cf20 2024-06-10T01:17:20.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.505+0000 7f303c28eb80 1 -- 172.21.15.7:0/239748028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560ee982cf20 msgr2=0x560ee982c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:20.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.505+0000 7f303c28eb80 1 --2- 172.21.15.7:0/239748028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560ee982cf20 0x560ee982c9e0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f301c011a00 tx=0x7f301c016000 comp rx=0 tx=0).stop 2024-06-10T01:17:20.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.505+0000 7f303c28eb80 1 -- 172.21.15.7:0/239748028 shutdown_connections 2024-06-10T01:17:20.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.505+0000 7f303c28eb80 1 --2- 172.21.15.7:0/239748028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ee9832f60 0x560ee9834f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.505+0000 7f303c28eb80 1 --2- 172.21.15.7:0/239748028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560ee982cf20 0x560ee982c9e0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.505+0000 7f303c28eb80 1 --2- 172.21.15.7:0/239748028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ee9832640 0x560ee9833bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.505+0000 7f303c28eb80 1 -- 172.21.15.7:0/239748028 >> 172.21.15.7:0/239748028 conn(0x560ee9757cc0 msgr2=0x560ee9820e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:20.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.505+0000 7f303c28eb80 1 -- 172.21.15.7:0/239748028 shutdown_connections 2024-06-10T01:17:20.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.505+0000 7f303c28eb80 1 -- 172.21.15.7:0/239748028 wait complete. 2024-06-10T01:17:20.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.505+0000 7f303c28eb80 1 -- start start 2024-06-10T01:17:20.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.505+0000 7f303c28eb80 1 Processor -- start 2024-06-10T01:17:20.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.506+0000 7f303c28eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560ee982cf20 0x560ee983f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:20.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.506+0000 7f303c28eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ee9832640 0x560ee983f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:20.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.506+0000 7f303c28eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ee9832f60 0x560ee984dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:20.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.506+0000 7f303c28eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f302c1b3030 con 0x560ee982cf20 2024-06-10T01:17:20.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.506+0000 7f303c28eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f302c1b44a0 con 0x560ee9832640 2024-06-10T01:17:20.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.506+0000 7f303c28eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f302c1b3700 con 0x560ee9832f60 2024-06-10T01:17:20.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.506+0000 7f303c28eb80 10 client.0 _ll_register_callbacks cb 0x560ee9718920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:17:20.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.506+0000 7f302affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560ee982cf20 0x560ee983f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:20.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.506+0000 7f302affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560ee982cf20 0x560ee983f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37656/0 (socket says 172.21.15.7:37656) 2024-06-10T01:17:20.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.506+0000 7f302affd700 1 -- 172.21.15.7:0/2929359547 learned_addr learned my addr 172.21.15.7:0/2929359547 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:20.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.506+0000 7f30227fc700 1 --2- 172.21.15.7:0/2929359547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ee9832640 0x560ee983f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:20.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.506+0000 7f302b7fe700 1 --2- 172.21.15.7:0/2929359547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ee9832f60 0x560ee984dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:20.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.506+0000 7f302affd700 1 -- 172.21.15.7:0/2929359547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ee9832f60 msgr2=0x560ee984dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:20.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.506+0000 7f302affd700 1 --2- 172.21.15.7:0/2929359547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ee9832f60 0x560ee984dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.506+0000 7f302affd700 1 -- 172.21.15.7:0/2929359547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ee9832640 msgr2=0x560ee983f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:20.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.506+0000 7f302affd700 1 --2- 172.21.15.7:0/2929359547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ee9832640 0x560ee983f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.506+0000 7f302affd700 1 -- 172.21.15.7:0/2929359547 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ee984e610 con 0x560ee982cf20 2024-06-10T01:17:20.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.506+0000 7f30227fc700 1 --2- 172.21.15.7:0/2929359547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ee9832640 0x560ee983f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:17:20.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.506+0000 7f302affd700 1 --2- 172.21.15.7:0/2929359547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560ee982cf20 0x560ee983f220 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f301c004b10 tx=0x7f301c011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d35b7d70e9be16a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:20.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.506+0000 7f300affd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:17:20.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.507+0000 7f300affd700 1 -- 172.21.15.7:0/2929359547 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c037070 con 0x560ee982cf20 2024-06-10T01:17:20.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.507+0000 7f300affd700 1 -- 172.21.15.7:0/2929359547 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f301c01e620 con 0x560ee982cf20 2024-06-10T01:17:20.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.507+0000 7f303c28eb80 1 -- 172.21.15.7:0/2929359547 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f301c011e50 con 0x560ee982cf20 2024-06-10T01:17:20.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.507+0000 7f300affd700 1 -- 172.21.15.7:0/2929359547 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f301c04d050 con 0x560ee982cf20 2024-06-10T01:17:20.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.507+0000 7f303c28eb80 10 client.18333 fetch_fsmap learned FSMap version 40 2024-06-10T01:17:20.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.507+0000 7f303c28eb80 1 -- 172.21.15.7:0/2929359547 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560ee98b1240 con 0x560ee982cf20 2024-06-10T01:17:20.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.507+0000 7f300affd700 1 -- 172.21.15.7:0/2929359547 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f301c011e50 con 0x560ee982cf20 2024-06-10T01:17:20.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.508+0000 7f300affd700 1 -- 172.21.15.7:0/2929359547 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f301c03ca20 con 0x560ee982cf20 2024-06-10T01:17:20.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.508+0000 7f303c28eb80 10 client.18333 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:17:20.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.508+0000 7f303c28eb80 10 client.18333 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:17:20.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.508+0000 7f303c28eb80 20 client.18333 populate_metadata read hostname 'smithi007' 2024-06-10T01:17:20.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.508+0000 7f303c28eb80 1 --2- 172.21.15.7:0/2929359547 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x560ee98469a0 0x560ee98b3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:20.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.508+0000 7f303c28eb80 4 client.18333 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:17:20.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.508+0000 7f303c28eb80 1 -- 172.21.15.7:0/2929359547 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560ee9786290 con 0x560ee98469a0 2024-06-10T01:17:20.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.508+0000 7f30227fc700 1 --2- 172.21.15.7:0/2929359547 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x560ee98469a0 0x560ee98b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:20.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.509+0000 7f30227fc700 1 --2- 172.21.15.7:0/2929359547 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x560ee98469a0 0x560ee98b3760 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7fed3870de75ef21 server_cookie=1e3b9540ba99c874 in_seq=0 out_seq=0 2024-06-10T01:17:20.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.509+0000 7f300affd700 10 client.18333 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:17:20.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.509+0000 7f300affd700 1 -- 172.21.15.7:0/2929359547 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560ee9786290 con 0x560ee98469a0 2024-06-10T01:17:20.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.509+0000 7f300affd700 10 client.18333 handle_command_reply: tid=0 2024-06-10T01:17:20.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.509+0000 7f303c28eb80 1 client.18333 shutdown 2024-06-10T01:17:20.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.510+0000 7f303c28eb80 1 -- 172.21.15.7:0/2929359547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560ee982cf20 msgr2=0x560ee983f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:20.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.510+0000 7f303c28eb80 1 --2- 172.21.15.7:0/2929359547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560ee982cf20 0x560ee983f220 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f301c004b10 tx=0x7f301c011c50 comp rx=0 tx=0).stop 2024-06-10T01:17:20.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.510+0000 7f303c28eb80 1 -- 172.21.15.7:0/2929359547 shutdown_connections 2024-06-10T01:17:20.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.510+0000 7f303c28eb80 1 --2- 172.21.15.7:0/2929359547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ee9832f60 0x560ee984dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.510+0000 7f303c28eb80 1 --2- 172.21.15.7:0/2929359547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560ee982cf20 0x560ee983f220 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.510+0000 7f303c28eb80 1 --2- 172.21.15.7:0/2929359547 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x560ee98469a0 0x560ee98b3760 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.510+0000 7f303c28eb80 1 --2- 172.21.15.7:0/2929359547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ee9832640 0x560ee983f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.510+0000 7f303c28eb80 1 -- 172.21.15.7:0/2929359547 >> 172.21.15.7:0/2929359547 conn(0x560ee9757cc0 msgr2=0x560ee9833a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:20.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.510+0000 7f300affd700 0 client.18333 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:17:20.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.510+0000 7f303c28eb80 1 -- 172.21.15.7:0/2929359547 shutdown_connections 2024-06-10T01:17:20.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.510+0000 7f303c28eb80 1 -- 172.21.15.7:0/2929359547 wait complete. 2024-06-10T01:17:20.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.510+0000 7f303c28eb80 20 client.18333 trim_cache size 0 max 16384 2024-06-10T01:17:20.507 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:17:20.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.510+0000 7f3031a0b700 1 -- 172.21.15.7:0/4045054175 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f30180a2c20 msgr2=0x7f30180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:20.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.510+0000 7f3031a0b700 1 --2- 172.21.15.7:0/4045054175 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f30180a2c20 0x7f30180a50e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f301c005950 tx=0x7f301c00b410 comp rx=0 tx=0).stop 2024-06-10T01:17:20.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.510+0000 7f3031a0b700 1 -- 172.21.15.7:0/4045054175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f302c108520 msgr2=0x7f302c1022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:20.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.510+0000 7f3031a0b700 1 --2- 172.21.15.7:0/4045054175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f302c108520 0x7f302c1022c0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f302400efb0 tx=0x7f302400c3a0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.510+0000 7f3031a0b700 1 -- 172.21.15.7:0/4045054175 shutdown_connections 2024-06-10T01:17:20.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.510+0000 7f3031a0b700 1 --2- 172.21.15.7:0/4045054175 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f30180a2c20 0x7f30180a50e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.510+0000 7f3031a0b700 1 --2- 172.21.15.7:0/4045054175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f302c108520 0x7f302c1022c0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.510+0000 7f3031a0b700 1 --2- 172.21.15.7:0/4045054175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f302c06ab60 0x7f302c101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.511+0000 7f3031a0b700 1 --2- 172.21.15.7:0/4045054175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f302c06b520 0x7f302c101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:20.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.511+0000 7f3031a0b700 1 -- 172.21.15.7:0/4045054175 >> 172.21.15.7:0/4045054175 conn(0x7f302c078340 msgr2=0x7f302c100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:20.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.511+0000 7f3031a0b700 1 -- 172.21.15.7:0/4045054175 shutdown_connections 2024-06-10T01:17:20.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:20.511+0000 7f3031a0b700 1 -- 172.21.15.7:0/4045054175 wait complete. 2024-06-10T01:17:20.751 INFO:tasks.workunit.client.0.smithi007.stdout: 1 837867 7.47 MB/sec execute 481 sec latency 56.002 ms 2024-06-10T01:17:21.240 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:20 smithi007 ceph-mon[32398]: pgmap v772: 129 pgs: 129 active+clean; 137 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 2.5 MiB/s wr, 272 op/s 2024-06-10T01:17:21.253 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:20 smithi175 ceph-mon[36636]: pgmap v772: 129 pgs: 129 active+clean; 137 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 2.5 MiB/s wr, 272 op/s 2024-06-10T01:17:21.276 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:17:21.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:20 smithi078 ceph-mon[36683]: pgmap v772: 129 pgs: 129 active+clean; 137 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 2.5 MiB/s wr, 272 op/s 2024-06-10T01:17:21.588 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:17:21.751 INFO:tasks.workunit.client.0.smithi007.stdout: 1 838944 7.47 MB/sec execute 482 sec latency 88.230 ms 2024-06-10T01:17:22.751 INFO:tasks.workunit.client.0.smithi007.stdout: 1 841567 7.48 MB/sec execute 483 sec latency 19.917 ms 2024-06-10T01:17:23.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:22 smithi007 ceph-mon[32398]: pgmap v773: 129 pgs: 129 active+clean; 137 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 2.3 MiB/s wr, 267 op/s 2024-06-10T01:17:23.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:22 smithi078 ceph-mon[36683]: pgmap v773: 129 pgs: 129 active+clean; 137 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 2.3 MiB/s wr, 267 op/s 2024-06-10T01:17:23.452 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:22 smithi175 ceph-mon[36636]: pgmap v773: 129 pgs: 129 active+clean; 137 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 2.3 MiB/s wr, 267 op/s 2024-06-10T01:17:23.751 INFO:tasks.workunit.client.0.smithi007.stdout: 1 842686 7.48 MB/sec execute 484 sec latency 52.675 ms 2024-06-10T01:17:23.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.940+0000 7f329beaa700 1 -- 172.21.15.7:0/3529713071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f329406aa00 msgr2=0x7f329406ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:23.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.940+0000 7f329beaa700 1 --2- 172.21.15.7:0/3529713071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f329406aa00 0x7f329406ae60 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f3284009990 tx=0x7f3284009ca0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.941+0000 7f329beaa700 1 -- 172.21.15.7:0/3529713071 shutdown_connections 2024-06-10T01:17:23.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.941+0000 7f329beaa700 1 --2- 172.21.15.7:0/3529713071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f329406b3a0 0x7f329410f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.941+0000 7f329beaa700 1 --2- 172.21.15.7:0/3529713071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32941084d0 0x7f32941088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.941+0000 7f329beaa700 1 --2- 172.21.15.7:0/3529713071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f329406aa00 0x7f329406ae60 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.941+0000 7f329beaa700 1 -- 172.21.15.7:0/3529713071 >> 172.21.15.7:0/3529713071 conn(0x7f3294078330 msgr2=0x7f3294078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:23.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.941+0000 7f329beaa700 1 -- 172.21.15.7:0/3529713071 shutdown_connections 2024-06-10T01:17:23.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.941+0000 7f329beaa700 1 -- 172.21.15.7:0/3529713071 wait complete. 2024-06-10T01:17:23.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.941+0000 7f329beaa700 1 Processor -- start 2024-06-10T01:17:23.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.942+0000 7f329beaa700 1 -- start start 2024-06-10T01:17:23.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.942+0000 7f329beaa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f329406aa00 0x7f329419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:23.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.942+0000 7f329beaa700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f329406b3a0 0x7f329419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:23.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.942+0000 7f329beaa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32941084d0 0x7f3294198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:23.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.942+0000 7f329beaa700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3294112730 con 0x7f329406aa00 2024-06-10T01:17:23.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.942+0000 7f329beaa700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f32941125b0 con 0x7f32941084d0 2024-06-10T01:17:23.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.942+0000 7f329beaa700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f32941128b0 con 0x7f329406b3a0 2024-06-10T01:17:23.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.942+0000 7f3299c46700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f329406aa00 0x7f329419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:23.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.942+0000 7f3299c46700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f329406aa00 0x7f329419e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42200/0 (socket says 172.21.15.7:42200) 2024-06-10T01:17:23.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.942+0000 7f3299c46700 1 -- 172.21.15.7:0/1209516407 learned_addr learned my addr 172.21.15.7:0/1209516407 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:23.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.942+0000 7f3299445700 1 --2- 172.21.15.7:0/1209516407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f329406b3a0 0x7f329419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:23.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.942+0000 7f329a447700 1 --2- 172.21.15.7:0/1209516407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32941084d0 0x7f3294198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:23.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.942+0000 7f3299c46700 1 -- 172.21.15.7:0/1209516407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f329406b3a0 msgr2=0x7f329419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:23.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.942+0000 7f3299c46700 1 --2- 172.21.15.7:0/1209516407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f329406b3a0 0x7f329419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.942+0000 7f3299c46700 1 -- 172.21.15.7:0/1209516407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32941084d0 msgr2=0x7f3294198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:23.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.942+0000 7f3299c46700 1 --2- 172.21.15.7:0/1209516407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32941084d0 0x7f3294198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.942+0000 7f3299c46700 1 -- 172.21.15.7:0/1209516407 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32941991f0 con 0x7f329406aa00 2024-06-10T01:17:23.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.942+0000 7f3299c46700 1 --2- 172.21.15.7:0/1209516407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f329406aa00 0x7f329419e880 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f329000e9a0 tx=0x7f329000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:23.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.943+0000 7f328affd700 1 -- 172.21.15.7:0/1209516407 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329000ca10 con 0x7f329406aa00 2024-06-10T01:17:23.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.943+0000 7f329beaa700 1 -- 172.21.15.7:0/1209516407 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32941994e0 con 0x7f329406aa00 2024-06-10T01:17:23.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.943+0000 7f328affd700 1 -- 172.21.15.7:0/1209516407 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f329000cbb0 con 0x7f329406aa00 2024-06-10T01:17:23.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.943+0000 7f328affd700 1 -- 172.21.15.7:0/1209516407 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3290018680 con 0x7f329406aa00 2024-06-10T01:17:23.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.943+0000 7f329beaa700 1 -- 172.21.15.7:0/1209516407 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32941a5750 con 0x7f329406aa00 2024-06-10T01:17:23.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.945+0000 7f328affd700 1 -- 172.21.15.7:0/1209516407 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f3290010430 con 0x7f329406aa00 2024-06-10T01:17:23.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.946+0000 7f328affd700 1 --2- 172.21.15.7:0/1209516407 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f32800a2c20 0x7f32800a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:23.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.946+0000 7f3299445700 1 --2- 172.21.15.7:0/1209516407 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f32800a2c20 0x7f32800a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:23.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.946+0000 7f328affd700 1 -- 172.21.15.7:0/1209516407 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f32900cf570 con 0x7f329406aa00 2024-06-10T01:17:23.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.946+0000 7f3299445700 1 --2- 172.21.15.7:0/1209516407 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f32800a2c20 0x7f32800a50e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f328400b2e0 tx=0x7f3284005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:23.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.947+0000 7f32a672db80 1 Processor -- start 2024-06-10T01:17:23.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.947+0000 7f32a672db80 1 -- start start 2024-06-10T01:17:23.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.947+0000 7f32a672db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1752ff60 0x560e17530340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:23.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.947+0000 7f32a672db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e17530880 0x560e1752a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:23.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.947+0000 7f32a672db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e1752a7a0 0x560e1752abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:23.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.947+0000 7f32a672db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560e1741cf60 con 0x560e1752a7a0 2024-06-10T01:17:23.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.947+0000 7f32a672db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560e1747b850 con 0x560e1752ff60 2024-06-10T01:17:23.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.947+0000 7f32a672db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560e173e3460 con 0x560e17530880 2024-06-10T01:17:23.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.947+0000 7f329a447700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e1752a7a0 0x560e1752abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:23.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.947+0000 7f329a447700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e1752a7a0 0x560e1752abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42206/0 (socket says 172.21.15.7:42206) 2024-06-10T01:17:23.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.947+0000 7f3299445700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e17530880 0x560e1752a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:23.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.947+0000 7f329a447700 1 -- 172.21.15.7:0/3218308265 learned_addr learned my addr 172.21.15.7:0/3218308265 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:23.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.947+0000 7f3299c46700 1 --2- 172.21.15.7:0/3218308265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1752ff60 0x560e17530340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:23.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.948+0000 7f329a447700 1 -- 172.21.15.7:0/3218308265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e17530880 msgr2=0x560e1752a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:23.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.948+0000 7f329a447700 1 --2- 172.21.15.7:0/3218308265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e17530880 0x560e1752a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.948+0000 7f329a447700 1 -- 172.21.15.7:0/3218308265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1752ff60 msgr2=0x560e17530340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:23.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.948+0000 7f329a447700 1 --2- 172.21.15.7:0/3218308265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1752ff60 0x560e17530340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.948+0000 7f329a447700 1 -- 172.21.15.7:0/3218308265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e173ca8d0 con 0x560e1752a7a0 2024-06-10T01:17:23.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.948+0000 7f329a447700 1 --2- 172.21.15.7:0/3218308265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e1752a7a0 0x560e1752abc0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f328c011680 tx=0x7f328c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=848a076af1b4396d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:23.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.948+0000 7f327ffff700 1 -- 172.21.15.7:0/3218308265 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c019030 con 0x560e1752a7a0 2024-06-10T01:17:23.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.948+0000 7f327ffff700 1 -- 172.21.15.7:0/3218308265 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f328c0092e0 con 0x560e1752a7a0 2024-06-10T01:17:23.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.948+0000 7f327ffff700 1 -- 172.21.15.7:0/3218308265 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c007500 con 0x560e1752a7a0 2024-06-10T01:17:23.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.949+0000 7f32a672db80 1 -- 172.21.15.7:0/3218308265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e1752a7a0 msgr2=0x560e1752abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:23.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.949+0000 7f32a672db80 1 --2- 172.21.15.7:0/3218308265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e1752a7a0 0x560e1752abc0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f328c011680 tx=0x7f328c009920 comp rx=0 tx=0).stop 2024-06-10T01:17:23.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.949+0000 7f32a672db80 1 -- 172.21.15.7:0/3218308265 shutdown_connections 2024-06-10T01:17:23.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.949+0000 7f32a672db80 1 --2- 172.21.15.7:0/3218308265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e1752a7a0 0x560e1752abc0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.949+0000 7f32a672db80 1 --2- 172.21.15.7:0/3218308265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1752ff60 0x560e17530340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.949+0000 7f32a672db80 1 --2- 172.21.15.7:0/3218308265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e17530880 0x560e1752a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.949+0000 7f32a672db80 1 -- 172.21.15.7:0/3218308265 >> 172.21.15.7:0/3218308265 conn(0x560e17457080 msgr2=0x560e17525220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:23.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.949+0000 7f32a672db80 1 -- 172.21.15.7:0/3218308265 shutdown_connections 2024-06-10T01:17:23.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.949+0000 7f32a672db80 1 -- 172.21.15.7:0/3218308265 wait complete. 2024-06-10T01:17:23.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.949+0000 7f32a672db80 1 -- start start 2024-06-10T01:17:23.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.949+0000 7f32a672db80 1 Processor -- start 2024-06-10T01:17:23.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.950+0000 7f32a672db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1752a7a0 0x560e17536580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:23.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.950+0000 7f32a672db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e1752ff60 0x560e17533880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:23.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.950+0000 7f32a672db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e17530880 0x560e17533dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:23.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.950+0000 7f32a672db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560e1741bfc0 con 0x560e1752ff60 2024-06-10T01:17:23.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.950+0000 7f32a672db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560e17477810 con 0x560e1752a7a0 2024-06-10T01:17:23.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.950+0000 7f32a672db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560e17537260 con 0x560e17530880 2024-06-10T01:17:23.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.950+0000 7f32a672db80 10 client.0 _ll_register_callbacks cb 0x560e174115b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:17:23.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.950+0000 7f3299445700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e1752ff60 0x560e17533880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:23.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.950+0000 7f3299445700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e1752ff60 0x560e17533880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42222/0 (socket says 172.21.15.7:42222) 2024-06-10T01:17:23.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.950+0000 7f3299445700 1 -- 172.21.15.7:0/1340806771 learned_addr learned my addr 172.21.15.7:0/1340806771 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:23.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.950+0000 7f329a447700 1 --2- 172.21.15.7:0/1340806771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e17530880 0x560e17533dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:23.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.950+0000 7f3299c46700 1 --2- 172.21.15.7:0/1340806771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1752a7a0 0x560e17536580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:23.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.950+0000 7f3299445700 1 -- 172.21.15.7:0/1340806771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e17530880 msgr2=0x560e17533dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:23.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.950+0000 7f3299445700 1 --2- 172.21.15.7:0/1340806771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e17530880 0x560e17533dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.950+0000 7f3299445700 1 -- 172.21.15.7:0/1340806771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1752a7a0 msgr2=0x560e17536580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:23.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.950+0000 7f3299445700 1 --2- 172.21.15.7:0/1340806771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1752a7a0 0x560e17536580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.950+0000 7f3299445700 1 -- 172.21.15.7:0/1340806771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e17441750 con 0x560e1752ff60 2024-06-10T01:17:23.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.950+0000 7f3299c46700 1 --2- 172.21.15.7:0/1340806771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1752a7a0 0x560e17536580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:17:23.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.950+0000 7f3299445700 1 --2- 172.21.15.7:0/1340806771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e1752ff60 0x560e17533880 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f3284009990 tx=0x7f3284009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe7ed49c779d48c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:23.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.950+0000 7f327d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:17:23.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.951+0000 7f327d7fa700 1 -- 172.21.15.7:0/1340806771 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328400f460 con 0x560e1752ff60 2024-06-10T01:17:23.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.951+0000 7f327d7fa700 1 -- 172.21.15.7:0/1340806771 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f328400f600 con 0x560e1752ff60 2024-06-10T01:17:23.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.951+0000 7f32a672db80 1 -- 172.21.15.7:0/1340806771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f328c0092e0 con 0x560e1752ff60 2024-06-10T01:17:23.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.951+0000 7f327d7fa700 1 -- 172.21.15.7:0/1340806771 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f328403c050 con 0x560e1752ff60 2024-06-10T01:17:23.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.951+0000 7f32a672db80 10 client.18345 fetch_fsmap learned FSMap version 40 2024-06-10T01:17:23.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.951+0000 7f32a672db80 1 -- 172.21.15.7:0/1340806771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560e175419c0 con 0x560e1752ff60 2024-06-10T01:17:23.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.951+0000 7f327d7fa700 1 -- 172.21.15.7:0/1340806771 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f328c0092e0 con 0x560e1752ff60 2024-06-10T01:17:23.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.951+0000 7f327d7fa700 1 -- 172.21.15.7:0/1340806771 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f3284022880 con 0x560e1752ff60 2024-06-10T01:17:23.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.951+0000 7f32a672db80 10 client.18345 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:17:23.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.951+0000 7f32a672db80 10 client.18345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:17:23.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.951+0000 7f32a672db80 20 client.18345 populate_metadata read hostname 'smithi007' 2024-06-10T01:17:23.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.951+0000 7f32a672db80 1 --2- 172.21.15.7:0/1340806771 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560e17544000 0x560e17546450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:23.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.951+0000 7f32a672db80 4 client.18345 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:17:23.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.951+0000 7f32a672db80 1 -- 172.21.15.7:0/1340806771 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e17488d50 con 0x560e17544000 2024-06-10T01:17:23.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.952+0000 7f3299c46700 1 --2- 172.21.15.7:0/1340806771 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560e17544000 0x560e17546450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:23.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.952+0000 7f3299c46700 1 --2- 172.21.15.7:0/1340806771 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560e17544000 0x560e17546450 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7edcf62dbe06579 server_cookie=2946cc36a8f798e4 in_seq=0 out_seq=0 2024-06-10T01:17:23.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.952+0000 7f327d7fa700 10 client.18345 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:23.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.953+0000 7f327d7fa700 1 -- 172.21.15.7:0/1340806771 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560e17488d50 con 0x560e17544000 2024-06-10T01:17:23.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.953+0000 7f327d7fa700 10 client.18345 handle_command_reply: tid=0 2024-06-10T01:17:23.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.953+0000 7f32a672db80 1 client.18345 shutdown 2024-06-10T01:17:23.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.953+0000 7f32a672db80 1 -- 172.21.15.7:0/1340806771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e1752ff60 msgr2=0x560e17533880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:23.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.953+0000 7f32a672db80 1 --2- 172.21.15.7:0/1340806771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e1752ff60 0x560e17533880 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f3284009990 tx=0x7f3284009ca0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.953+0000 7f32a672db80 1 -- 172.21.15.7:0/1340806771 shutdown_connections 2024-06-10T01:17:23.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.953+0000 7f32a672db80 1 --2- 172.21.15.7:0/1340806771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e17530880 0x560e17533dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.953+0000 7f32a672db80 1 --2- 172.21.15.7:0/1340806771 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560e17544000 0x560e17546450 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.953+0000 7f32a672db80 1 --2- 172.21.15.7:0/1340806771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1752a7a0 0x560e17536580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.953+0000 7f32a672db80 1 --2- 172.21.15.7:0/1340806771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e1752ff60 0x560e17533880 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.953+0000 7f327d7fa700 0 client.18345 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:23.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.953+0000 7f32a672db80 1 -- 172.21.15.7:0/1340806771 >> 172.21.15.7:0/1340806771 conn(0x560e17457080 msgr2=0x560e17524700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:23.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.953+0000 7f32a672db80 1 -- 172.21.15.7:0/1340806771 shutdown_connections 2024-06-10T01:17:23.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.953+0000 7f32a672db80 1 -- 172.21.15.7:0/1340806771 wait complete. 2024-06-10T01:17:23.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.953+0000 7f32a672db80 20 client.18345 trim_cache size 0 max 16384 2024-06-10T01:17:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.969+0000 7f32a672db80 1 Processor -- start 2024-06-10T01:17:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.969+0000 7f32a672db80 1 -- start start 2024-06-10T01:17:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.969+0000 7f32a672db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e1752a7a0 0x560e1752a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.969+0000 7f32a672db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1752ff60 0x560e17524e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.969+0000 7f32a672db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e17530880 0x560e175253c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.969+0000 7f32a672db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f32941b0b50 con 0x560e1752a7a0 2024-06-10T01:17:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.969+0000 7f32a672db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f32941ae3e0 con 0x560e1752ff60 2024-06-10T01:17:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.969+0000 7f32a672db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f32941ac230 con 0x560e17530880 2024-06-10T01:17:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.969+0000 7f3299c46700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e1752a7a0 0x560e1752a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.969+0000 7f3299c46700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e1752a7a0 0x560e1752a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42230/0 (socket says 172.21.15.7:42230) 2024-06-10T01:17:23.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.970+0000 7f3299c46700 1 -- 172.21.15.7:0/3536226776 learned_addr learned my addr 172.21.15.7:0/3536226776 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:23.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.970+0000 7f329a447700 1 --2- 172.21.15.7:0/3536226776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e17530880 0x560e175253c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:23.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.970+0000 7f3299445700 1 --2- 172.21.15.7:0/3536226776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1752ff60 0x560e17524e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:23.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.970+0000 7f3299c46700 1 -- 172.21.15.7:0/3536226776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e17530880 msgr2=0x560e175253c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:23.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.970+0000 7f3299c46700 1 --2- 172.21.15.7:0/3536226776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e17530880 0x560e175253c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.970+0000 7f3299c46700 1 -- 172.21.15.7:0/3536226776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1752ff60 msgr2=0x560e17524e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:23.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.970+0000 7f3299c46700 1 --2- 172.21.15.7:0/3536226776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1752ff60 0x560e17524e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.970+0000 7f3299c46700 1 -- 172.21.15.7:0/3536226776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e17531df0 con 0x560e1752a7a0 2024-06-10T01:17:23.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.970+0000 7f3299c46700 1 --2- 172.21.15.7:0/3536226776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e1752a7a0 0x560e1752a260 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f3290000c00 tx=0x7f3290086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d179d593e5a10ee server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:23.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.970+0000 7f327f7fe700 1 -- 172.21.15.7:0/3536226776 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3290099af0 con 0x560e1752a7a0 2024-06-10T01:17:23.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.970+0000 7f327f7fe700 1 -- 172.21.15.7:0/3536226776 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f329000cd60 con 0x560e1752a7a0 2024-06-10T01:17:23.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.970+0000 7f327f7fe700 1 -- 172.21.15.7:0/3536226776 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32900a4490 con 0x560e1752a7a0 2024-06-10T01:17:23.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.971+0000 7f32a672db80 1 -- 172.21.15.7:0/3536226776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e1752a7a0 msgr2=0x560e1752a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:23.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.971+0000 7f32a672db80 1 --2- 172.21.15.7:0/3536226776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e1752a7a0 0x560e1752a260 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f3290000c00 tx=0x7f3290086040 comp rx=0 tx=0).stop 2024-06-10T01:17:23.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.971+0000 7f32a672db80 1 -- 172.21.15.7:0/3536226776 shutdown_connections 2024-06-10T01:17:23.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.971+0000 7f32a672db80 1 --2- 172.21.15.7:0/3536226776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e17530880 0x560e175253c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.971+0000 7f32a672db80 1 --2- 172.21.15.7:0/3536226776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e1752a7a0 0x560e1752a260 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.971+0000 7f32a672db80 1 --2- 172.21.15.7:0/3536226776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1752ff60 0x560e17524e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.971+0000 7f32a672db80 1 -- 172.21.15.7:0/3536226776 >> 172.21.15.7:0/3536226776 conn(0x560e17457080 msgr2=0x560e17532170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:23.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.971+0000 7f32a672db80 1 -- 172.21.15.7:0/3536226776 shutdown_connections 2024-06-10T01:17:23.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.971+0000 7f32a672db80 1 -- 172.21.15.7:0/3536226776 wait complete. 2024-06-10T01:17:23.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.971+0000 7f32a672db80 1 -- start start 2024-06-10T01:17:23.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.971+0000 7f32a672db80 1 Processor -- start 2024-06-10T01:17:23.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.972+0000 7f32a672db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e1752a7a0 0x560e1753c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:23.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.972+0000 7f32a672db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1752ff60 0x560e1753cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:23.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.972+0000 7f32a672db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e17530880 0x560e1754b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:23.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.972+0000 7f32a672db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f32941ac8a0 con 0x560e17530880 2024-06-10T01:17:23.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.972+0000 7f32a672db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f32941add10 con 0x560e1752ff60 2024-06-10T01:17:23.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.972+0000 7f32a672db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f32941acf70 con 0x560e1752a7a0 2024-06-10T01:17:23.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.972+0000 7f32a672db80 10 client.0 _ll_register_callbacks cb 0x560e174115b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:17:23.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.972+0000 7f329a447700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e17530880 0x560e1754b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:23.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.972+0000 7f329a447700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e17530880 0x560e1754b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42244/0 (socket says 172.21.15.7:42244) 2024-06-10T01:17:23.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.972+0000 7f3299c46700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e1752a7a0 0x560e1753c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:23.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.972+0000 7f3299445700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1752ff60 0x560e1753cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:23.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.972+0000 7f329a447700 1 -- 172.21.15.7:0/2537021717 learned_addr learned my addr 172.21.15.7:0/2537021717 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:23.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.972+0000 7f3299c46700 1 -- 172.21.15.7:0/2537021717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1752ff60 msgr2=0x560e1753cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:23.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.972+0000 7f3299c46700 1 --2- 172.21.15.7:0/2537021717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1752ff60 0x560e1753cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.972+0000 7f3299c46700 1 -- 172.21.15.7:0/2537021717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e17530880 msgr2=0x560e1754b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:23.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.972+0000 7f3299c46700 1 --2- 172.21.15.7:0/2537021717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e17530880 0x560e1754b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.972+0000 7f3299c46700 1 -- 172.21.15.7:0/2537021717 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e1754bb70 con 0x560e1752a7a0 2024-06-10T01:17:23.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.973+0000 7f3299c46700 1 --2- 172.21.15.7:0/2537021717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e1752a7a0 0x560e1753c7e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x560e17526800 tx=0x7f329009b480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c66c0367a0fa4e70 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:23.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.973+0000 7f327d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:17:23.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.973+0000 7f327d7fa700 1 -- 172.21.15.7:0/2537021717 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32900b9070 con 0x560e1752a7a0 2024-06-10T01:17:23.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.973+0000 7f327d7fa700 1 -- 172.21.15.7:0/2537021717 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f329009bcb0 con 0x560e1752a7a0 2024-06-10T01:17:23.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.973+0000 7f32a672db80 1 -- 172.21.15.7:0/2537021717 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f329000cd60 con 0x560e1752a7a0 2024-06-10T01:17:23.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.973+0000 7f327d7fa700 1 -- 172.21.15.7:0/2537021717 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f32900ba460 con 0x560e1752a7a0 2024-06-10T01:17:23.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.973+0000 7f32a672db80 10 client.27709 fetch_fsmap learned FSMap version 40 2024-06-10T01:17:23.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.973+0000 7f32a672db80 1 -- 172.21.15.7:0/2537021717 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560e175ae870 con 0x560e1752a7a0 2024-06-10T01:17:23.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.973+0000 7f327d7fa700 1 -- 172.21.15.7:0/2537021717 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f329000cd60 con 0x560e1752a7a0 2024-06-10T01:17:23.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.973+0000 7f327d7fa700 1 -- 172.21.15.7:0/2537021717 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f3290010720 con 0x560e1752a7a0 2024-06-10T01:17:23.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.974+0000 7f32a672db80 10 client.27709 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:17:23.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.974+0000 7f32a672db80 10 client.27709 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:17:23.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.974+0000 7f32a672db80 20 client.27709 populate_metadata read hostname 'smithi007' 2024-06-10T01:17:23.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.974+0000 7f32a672db80 1 --2- 172.21.15.7:0/2537021717 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560e175aeee0 0x560e175b1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:23.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.974+0000 7f32a672db80 4 client.27709 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:17:23.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.974+0000 7f32a672db80 1 -- 172.21.15.7:0/2537021717 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560e174832d0 con 0x560e175aeee0 2024-06-10T01:17:23.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.974+0000 7f3299445700 1 --2- 172.21.15.7:0/2537021717 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560e175aeee0 0x560e175b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:23.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.974+0000 7f3299445700 1 --2- 172.21.15.7:0/2537021717 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560e175aeee0 0x560e175b1300 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a99610a34fe6f342 server_cookie=b9153924dc633ebc in_seq=0 out_seq=0 2024-06-10T01:17:23.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.974+0000 7f327d7fa700 10 client.27709 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:23.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.974+0000 7f327d7fa700 1 -- 172.21.15.7:0/2537021717 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560e174832d0 con 0x560e175aeee0 2024-06-10T01:17:23.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.974+0000 7f327d7fa700 10 client.27709 handle_command_reply: tid=0 2024-06-10T01:17:23.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.974+0000 7f32a672db80 1 client.27709 shutdown 2024-06-10T01:17:23.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.974+0000 7f32a672db80 1 -- 172.21.15.7:0/2537021717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e1752a7a0 msgr2=0x560e1753c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:23.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.974+0000 7f32a672db80 1 --2- 172.21.15.7:0/2537021717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e1752a7a0 0x560e1753c7e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x560e17526800 tx=0x7f329009b480 comp rx=0 tx=0).stop 2024-06-10T01:17:23.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.975+0000 7f32a672db80 1 -- 172.21.15.7:0/2537021717 shutdown_connections 2024-06-10T01:17:23.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.975+0000 7f32a672db80 1 --2- 172.21.15.7:0/2537021717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e17530880 0x560e1754b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.975+0000 7f32a672db80 1 --2- 172.21.15.7:0/2537021717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560e1752a7a0 0x560e1753c7e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.975+0000 7f32a672db80 1 --2- 172.21.15.7:0/2537021717 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560e175aeee0 0x560e175b1300 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.975+0000 7f32a672db80 1 --2- 172.21.15.7:0/2537021717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1752ff60 0x560e1753cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.975+0000 7f327d7fa700 0 client.27709 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:23.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.975+0000 7f32a672db80 1 -- 172.21.15.7:0/2537021717 >> 172.21.15.7:0/2537021717 conn(0x560e17457080 msgr2=0x560e17530c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:23.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.975+0000 7f32a672db80 1 -- 172.21.15.7:0/2537021717 shutdown_connections 2024-06-10T01:17:23.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.975+0000 7f32a672db80 1 -- 172.21.15.7:0/2537021717 wait complete. 2024-06-10T01:17:23.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.975+0000 7f32a672db80 20 client.27709 trim_cache size 0 max 16384 2024-06-10T01:17:23.974 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:17:23.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.975+0000 7f329beaa700 1 -- 172.21.15.7:0/1209516407 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f32800a2c20 msgr2=0x7f32800a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:23.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.975+0000 7f329beaa700 1 --2- 172.21.15.7:0/1209516407 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f32800a2c20 0x7f32800a50e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f328400b2e0 tx=0x7f3284005960 comp rx=0 tx=0).stop 2024-06-10T01:17:23.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.975+0000 7f329beaa700 1 -- 172.21.15.7:0/1209516407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f329406aa00 msgr2=0x7f329419e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:23.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.975+0000 7f329beaa700 1 --2- 172.21.15.7:0/1209516407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f329406aa00 0x7f329419e880 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f329000e9a0 tx=0x7f329000ecb0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.975+0000 7f329beaa700 1 -- 172.21.15.7:0/1209516407 shutdown_connections 2024-06-10T01:17:23.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.975+0000 7f329beaa700 1 --2- 172.21.15.7:0/1209516407 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f32800a2c20 0x7f32800a50e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.975+0000 7f329beaa700 1 --2- 172.21.15.7:0/1209516407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32941084d0 0x7f3294198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.975+0000 7f329beaa700 1 --2- 172.21.15.7:0/1209516407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f329406aa00 0x7f329419e880 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.975+0000 7f329beaa700 1 --2- 172.21.15.7:0/1209516407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f329406b3a0 0x7f329419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:23.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.976+0000 7f329beaa700 1 -- 172.21.15.7:0/1209516407 >> 172.21.15.7:0/1209516407 conn(0x7f3294078330 msgr2=0x7f32940ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:23.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.976+0000 7f329beaa700 1 -- 172.21.15.7:0/1209516407 shutdown_connections 2024-06-10T01:17:23.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:23.976+0000 7f329beaa700 1 -- 172.21.15.7:0/1209516407 wait complete. 2024-06-10T01:17:24.751 INFO:tasks.workunit.client.0.smithi007.stdout: 1 845241 7.49 MB/sec execute 485 sec latency 10.887 ms 2024-06-10T01:17:25.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:24 smithi007 ceph-mon[32398]: pgmap v774: 129 pgs: 129 active+clean; 138 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 2.7 MiB/s wr, 286 op/s 2024-06-10T01:17:25.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:24 smithi078 ceph-mon[36683]: pgmap v774: 129 pgs: 129 active+clean; 138 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 2.7 MiB/s wr, 286 op/s 2024-06-10T01:17:25.452 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:24 smithi175 ceph-mon[36636]: pgmap v774: 129 pgs: 129 active+clean; 138 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 2.7 MiB/s wr, 286 op/s 2024-06-10T01:17:25.689 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:17:25.689 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:17:25.752 INFO:tasks.workunit.client.0.smithi007.stdout: 1 846579 7.49 MB/sec execute 486 sec latency 36.939 ms 2024-06-10T01:17:25.972 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:17:26.751 INFO:tasks.workunit.client.0.smithi007.stdout: 1 848849 7.50 MB/sec execute 487 sec latency 50.155 ms 2024-06-10T01:17:27.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:26 smithi007 ceph-mon[32398]: pgmap v775: 129 pgs: 129 active+clean; 140 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 2.7 MiB/s wr, 309 op/s 2024-06-10T01:17:27.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:26 smithi078 ceph-mon[36683]: pgmap v775: 129 pgs: 129 active+clean; 140 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 2.7 MiB/s wr, 309 op/s 2024-06-10T01:17:27.452 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:26 smithi175 ceph-mon[36636]: pgmap v775: 129 pgs: 129 active+clean; 140 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 2.7 MiB/s wr, 309 op/s 2024-06-10T01:17:27.751 INFO:tasks.workunit.client.0.smithi007.stdout: 1 850258 7.50 MB/sec execute 488 sec latency 57.559 ms 2024-06-10T01:17:28.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.312+0000 7f79b2aca700 1 -- 172.21.15.7:0/2015947470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79ac06ab60 msgr2=0x7f79ac06afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:28.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.312+0000 7f79b2aca700 1 --2- 172.21.15.7:0/2015947470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79ac06ab60 0x7f79ac06afe0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f7998009990 tx=0x7f7998009ca0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.313+0000 7f79b2aca700 1 -- 172.21.15.7:0/2015947470 shutdown_connections 2024-06-10T01:17:28.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.313+0000 7f79b2aca700 1 --2- 172.21.15.7:0/2015947470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79ac06b520 0x7f79ac10f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.313+0000 7f79b2aca700 1 --2- 172.21.15.7:0/2015947470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79ac108530 0x7f79ac108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.313+0000 7f79b2aca700 1 --2- 172.21.15.7:0/2015947470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79ac06ab60 0x7f79ac06afe0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.313+0000 7f79b2aca700 1 -- 172.21.15.7:0/2015947470 >> 172.21.15.7:0/2015947470 conn(0x7f79ac0783d0 msgr2=0x7f79ac0787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:28.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.313+0000 7f79b2aca700 1 -- 172.21.15.7:0/2015947470 shutdown_connections 2024-06-10T01:17:28.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.313+0000 7f79b2aca700 1 -- 172.21.15.7:0/2015947470 wait complete. 2024-06-10T01:17:28.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.313+0000 7f79b2aca700 1 Processor -- start 2024-06-10T01:17:28.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.313+0000 7f79b2aca700 1 -- start start 2024-06-10T01:17:28.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.314+0000 7f79b2aca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79ac06ab60 0x7f79ac101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:28.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.314+0000 7f79b2aca700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79ac06b520 0x7f79ac1019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:28.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.314+0000 7f79b2aca700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79ac108530 0x7f79ac101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:28.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.314+0000 7f79b2aca700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac1127b0 con 0x7f79ac06ab60 2024-06-10T01:17:28.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.314+0000 7f79b2aca700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac112630 con 0x7f79ac06b520 2024-06-10T01:17:28.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.314+0000 7f79b2aca700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac112930 con 0x7f79ac108530 2024-06-10T01:17:28.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.314+0000 7f79b0866700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79ac06ab60 0x7f79ac101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:28.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.314+0000 7f79b0866700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79ac06ab60 0x7f79ac101490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42264/0 (socket says 172.21.15.7:42264) 2024-06-10T01:17:28.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.314+0000 7f79b0866700 1 -- 172.21.15.7:0/41097633 learned_addr learned my addr 172.21.15.7:0/41097633 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:28.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.314+0000 7f79b1067700 1 --2- 172.21.15.7:0/41097633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79ac108530 0x7f79ac101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:28.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.314+0000 7f79abfff700 1 --2- 172.21.15.7:0/41097633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79ac06b520 0x7f79ac1019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:28.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.314+0000 7f79b0866700 1 -- 172.21.15.7:0/41097633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79ac108530 msgr2=0x7f79ac101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:28.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.314+0000 7f79b0866700 1 --2- 172.21.15.7:0/41097633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79ac108530 0x7f79ac101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.314+0000 7f79b0866700 1 -- 172.21.15.7:0/41097633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79ac06b520 msgr2=0x7f79ac1019d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:28.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.314+0000 7f79b0866700 1 --2- 172.21.15.7:0/41097633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79ac06b520 0x7f79ac1019d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.314+0000 7f79b0866700 1 -- 172.21.15.7:0/41097633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79ac1a71d0 con 0x7f79ac06ab60 2024-06-10T01:17:28.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.314+0000 7f79abfff700 1 --2- 172.21.15.7:0/41097633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79ac06b520 0x7f79ac1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:17:28.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.314+0000 7f79b0866700 1 --2- 172.21.15.7:0/41097633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79ac06ab60 0x7f79ac101490 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f79a000e9a0 tx=0x7f79a000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:28.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.314+0000 7f79a9ffb700 1 -- 172.21.15.7:0/41097633 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a000ca10 con 0x7f79ac06ab60 2024-06-10T01:17:28.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.314+0000 7f79a9ffb700 1 -- 172.21.15.7:0/41097633 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79a000cbb0 con 0x7f79ac06ab60 2024-06-10T01:17:28.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.315+0000 7f79b2aca700 1 -- 172.21.15.7:0/41097633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79ac1a74c0 con 0x7f79ac06ab60 2024-06-10T01:17:28.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.315+0000 7f79a9ffb700 1 -- 172.21.15.7:0/41097633 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a0018680 con 0x7f79ac06ab60 2024-06-10T01:17:28.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.315+0000 7f79b2aca700 1 -- 172.21.15.7:0/41097633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79ac1a79d0 con 0x7f79ac06ab60 2024-06-10T01:17:28.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.318+0000 7f79a9ffb700 1 -- 172.21.15.7:0/41097633 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f79a0010430 con 0x7f79ac06ab60 2024-06-10T01:17:28.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.318+0000 7f79a9ffb700 1 --2- 172.21.15.7:0/41097633 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f799c0a2c20 0x7f799c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:28.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.319+0000 7f79a9ffb700 1 -- 172.21.15.7:0/41097633 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f79a00cf570 con 0x7f79ac06ab60 2024-06-10T01:17:28.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.319+0000 7f79abfff700 1 --2- 172.21.15.7:0/41097633 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f799c0a2c20 0x7f799c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:28.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.319+0000 7f79abfff700 1 --2- 172.21.15.7:0/41097633 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f799c0a2c20 0x7f799c0a50e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f799800b2e0 tx=0x7f7998005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:28.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.319+0000 7f79bd34db80 1 Processor -- start 2024-06-10T01:17:28.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.319+0000 7f79bd34db80 1 -- start start 2024-06-10T01:17:28.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.319+0000 7f79bd34db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a37a87fe0 0x559a37a883c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:28.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.319+0000 7f79bd34db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a37a88900 0x559a37a822e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:28.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.319+0000 7f79bd34db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a37a82820 0x559a37a82c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:28.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.319+0000 7f79bd34db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559a37974fe0 con 0x559a37a87fe0 2024-06-10T01:17:28.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.319+0000 7f79bd34db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559a379d38d0 con 0x559a37a88900 2024-06-10T01:17:28.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.320+0000 7f79bd34db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559a3793b4e0 con 0x559a37a82820 2024-06-10T01:17:28.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.320+0000 7f79b0866700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a37a87fe0 0x559a37a883c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:28.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.320+0000 7f79b0866700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a37a87fe0 0x559a37a883c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42278/0 (socket says 172.21.15.7:42278) 2024-06-10T01:17:28.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.320+0000 7f79b0866700 1 -- 172.21.15.7:0/2602829945 learned_addr learned my addr 172.21.15.7:0/2602829945 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:28.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.320+0000 7f79abfff700 1 --2- 172.21.15.7:0/2602829945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a37a88900 0x559a37a822e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:28.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.320+0000 7f79b1067700 1 --2- 172.21.15.7:0/2602829945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a37a82820 0x559a37a82c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:28.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.320+0000 7f79b0866700 1 -- 172.21.15.7:0/2602829945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a37a82820 msgr2=0x559a37a82c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:28.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.320+0000 7f79b0866700 1 --2- 172.21.15.7:0/2602829945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a37a82820 0x559a37a82c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.320+0000 7f79b0866700 1 -- 172.21.15.7:0/2602829945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a37a88900 msgr2=0x559a37a822e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:28.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.320+0000 7f79b0866700 1 --2- 172.21.15.7:0/2602829945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a37a88900 0x559a37a822e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.320+0000 7f79b0866700 1 -- 172.21.15.7:0/2602829945 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a37922950 con 0x559a37a87fe0 2024-06-10T01:17:28.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.320+0000 7f79b1067700 1 --2- 172.21.15.7:0/2602829945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a37a82820 0x559a37a82c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:17:28.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.320+0000 7f79abfff700 1 --2- 172.21.15.7:0/2602829945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a37a88900 0x559a37a822e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:17:28.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.320+0000 7f79b0866700 1 --2- 172.21.15.7:0/2602829945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a37a87fe0 0x559a37a883c0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f79a009b9e0 tx=0x7f79a009c2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a065514835e40887 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:28.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.320+0000 7f7996ffd700 1 -- 172.21.15.7:0/2602829945 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a00a46c0 con 0x559a37a87fe0 2024-06-10T01:17:28.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.320+0000 7f7996ffd700 1 -- 172.21.15.7:0/2602829945 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79a000cd60 con 0x559a37a87fe0 2024-06-10T01:17:28.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.320+0000 7f7996ffd700 1 -- 172.21.15.7:0/2602829945 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a0010980 con 0x559a37a87fe0 2024-06-10T01:17:28.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.321+0000 7f79bd34db80 1 -- 172.21.15.7:0/2602829945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a37a87fe0 msgr2=0x559a37a883c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:28.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.321+0000 7f79bd34db80 1 --2- 172.21.15.7:0/2602829945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a37a87fe0 0x559a37a883c0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f79a009b9e0 tx=0x7f79a009c2a0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.321+0000 7f79bd34db80 1 -- 172.21.15.7:0/2602829945 shutdown_connections 2024-06-10T01:17:28.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.321+0000 7f79bd34db80 1 --2- 172.21.15.7:0/2602829945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a37a82820 0x559a37a82c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.321+0000 7f79bd34db80 1 --2- 172.21.15.7:0/2602829945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a37a87fe0 0x559a37a883c0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.321+0000 7f79bd34db80 1 --2- 172.21.15.7:0/2602829945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a37a88900 0x559a37a822e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.321+0000 7f79bd34db80 1 -- 172.21.15.7:0/2602829945 >> 172.21.15.7:0/2602829945 conn(0x559a379af100 msgr2=0x559a37a7d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.321+0000 7f79bd34db80 1 -- 172.21.15.7:0/2602829945 shutdown_connections 2024-06-10T01:17:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.321+0000 7f79bd34db80 1 -- 172.21.15.7:0/2602829945 wait complete. 2024-06-10T01:17:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.321+0000 7f79bd34db80 1 -- start start 2024-06-10T01:17:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.322+0000 7f79bd34db80 1 Processor -- start 2024-06-10T01:17:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.322+0000 7f79bd34db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a37a82820 0x559a37a8e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.322+0000 7f79bd34db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a37a87fe0 0x559a37a8b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.322+0000 7f79bd34db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a37a88900 0x559a37a8beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.322+0000 7f79bd34db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559a37974040 con 0x559a37a82820 2024-06-10T01:17:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.322+0000 7f79bd34db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559a379cf890 con 0x559a37a88900 2024-06-10T01:17:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.322+0000 7f79bd34db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559a37a8f280 con 0x559a37a87fe0 2024-06-10T01:17:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.322+0000 7f79bd34db80 10 client.0 _ll_register_callbacks cb 0x559a37969630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:17:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.322+0000 7f79b0866700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a37a82820 0x559a37a8e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.322+0000 7f79b1067700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a37a88900 0x559a37a8beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.322+0000 7f79b0866700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a37a82820 0x559a37a8e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42294/0 (socket says 172.21.15.7:42294) 2024-06-10T01:17:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.322+0000 7f79b0866700 1 -- 172.21.15.7:0/1357491214 learned_addr learned my addr 172.21.15.7:0/1357491214 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.322+0000 7f79abfff700 1 --2- 172.21.15.7:0/1357491214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a37a87fe0 0x559a37a8b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.322+0000 7f79b0866700 1 -- 172.21.15.7:0/1357491214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a37a87fe0 msgr2=0x559a37a8b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.323+0000 7f79b0866700 1 --2- 172.21.15.7:0/1357491214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a37a87fe0 0x559a37a8b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.323+0000 7f79b0866700 1 -- 172.21.15.7:0/1357491214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a37a88900 msgr2=0x559a37a8beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.323+0000 7f79b0866700 1 --2- 172.21.15.7:0/1357491214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a37a88900 0x559a37a8beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.323+0000 7f79b0866700 1 -- 172.21.15.7:0/1357491214 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a379997d0 con 0x559a37a82820 2024-06-10T01:17:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.323+0000 7f79b0866700 1 --2- 172.21.15.7:0/1357491214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a37a82820 0x559a37a8e670 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f79a00ccd40 tx=0x7f79a0009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e24169c9e00e6af6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.323+0000 7f798bfff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:17:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.323+0000 7f798bfff700 1 -- 172.21.15.7:0/1357491214 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a00b9060 con 0x559a37a82820 2024-06-10T01:17:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.323+0000 7f798bfff700 1 -- 172.21.15.7:0/1357491214 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f79a0010720 con 0x559a37a82820 2024-06-10T01:17:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.323+0000 7f79bd34db80 1 -- 172.21.15.7:0/1357491214 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79a000cd60 con 0x559a37a82820 2024-06-10T01:17:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.323+0000 7f798bfff700 1 -- 172.21.15.7:0/1357491214 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f79a00b3030 con 0x559a37a82820 2024-06-10T01:17:28.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.323+0000 7f79bd34db80 10 client.18363 fetch_fsmap learned FSMap version 40 2024-06-10T01:17:28.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.323+0000 7f79bd34db80 1 -- 172.21.15.7:0/1357491214 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559a37a99a30 con 0x559a37a82820 2024-06-10T01:17:28.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.323+0000 7f798bfff700 1 -- 172.21.15.7:0/1357491214 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79a000cd60 con 0x559a37a82820 2024-06-10T01:17:28.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.323+0000 7f798bfff700 1 -- 172.21.15.7:0/1357491214 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f79a009aea0 con 0x559a37a82820 2024-06-10T01:17:28.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.323+0000 7f79bd34db80 10 client.18363 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:17:28.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.324+0000 7f79bd34db80 10 client.18363 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:17:28.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.324+0000 7f79bd34db80 20 client.18363 populate_metadata read hostname 'smithi007' 2024-06-10T01:17:28.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.324+0000 7f79bd34db80 1 --2- 172.21.15.7:0/1357491214 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559a37a9bf00 0x559a37a9e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:28.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.324+0000 7f79bd34db80 4 client.18363 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:17:28.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.324+0000 7f79bd34db80 1 -- 172.21.15.7:0/1357491214 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a379e0dd0 con 0x559a37a9bf00 2024-06-10T01:17:28.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.324+0000 7f79abfff700 1 --2- 172.21.15.7:0/1357491214 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559a37a9bf00 0x559a37a9e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:28.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.324+0000 7f79abfff700 1 --2- 172.21.15.7:0/1357491214 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559a37a9bf00 0x559a37a9e2f0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a98c7f13e7a6b2e3 server_cookie=a0177fa3cd1e412a in_seq=0 out_seq=0 2024-06-10T01:17:28.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.324+0000 7f798bfff700 10 client.18363 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:28.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.325+0000 7f798bfff700 1 -- 172.21.15.7:0/1357491214 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559a379e0dd0 con 0x559a37a9bf00 2024-06-10T01:17:28.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.325+0000 7f798bfff700 10 client.18363 handle_command_reply: tid=0 2024-06-10T01:17:28.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.325+0000 7f79bd34db80 1 client.18363 shutdown 2024-06-10T01:17:28.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.325+0000 7f79bd34db80 1 -- 172.21.15.7:0/1357491214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a37a82820 msgr2=0x559a37a8e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:28.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.325+0000 7f79bd34db80 1 --2- 172.21.15.7:0/1357491214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a37a82820 0x559a37a8e670 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f79a00ccd40 tx=0x7f79a0009790 comp rx=0 tx=0).stop 2024-06-10T01:17:28.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.325+0000 7f79bd34db80 1 -- 172.21.15.7:0/1357491214 shutdown_connections 2024-06-10T01:17:28.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.325+0000 7f79bd34db80 1 --2- 172.21.15.7:0/1357491214 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559a37a9bf00 0x559a37a9e2f0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.325+0000 7f79bd34db80 1 --2- 172.21.15.7:0/1357491214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a37a88900 0x559a37a8beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.325+0000 7f79bd34db80 1 --2- 172.21.15.7:0/1357491214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a37a82820 0x559a37a8e670 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.325+0000 7f79bd34db80 1 --2- 172.21.15.7:0/1357491214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a37a87fe0 0x559a37a8b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.325+0000 7f79bd34db80 1 -- 172.21.15.7:0/1357491214 >> 172.21.15.7:0/1357491214 conn(0x559a379af100 msgr2=0x559a37a7c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:28.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.325+0000 7f798bfff700 0 client.18363 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:28.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.326+0000 7f79bd34db80 1 -- 172.21.15.7:0/1357491214 shutdown_connections 2024-06-10T01:17:28.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.326+0000 7f79bd34db80 1 -- 172.21.15.7:0/1357491214 wait complete. 2024-06-10T01:17:28.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.326+0000 7f79bd34db80 20 client.18363 trim_cache size 0 max 16384 2024-06-10T01:17:28.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.347+0000 7f79bd34db80 1 Processor -- start 2024-06-10T01:17:28.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.348+0000 7f79bd34db80 1 -- start start 2024-06-10T01:17:28.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.348+0000 7f79bd34db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a37a82820 0x559a37a7cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:28.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.348+0000 7f79bd34db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a37a87fe0 0x559a37a7d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:28.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.348+0000 7f79bd34db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a37a88900 0x559a37a87190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:28.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.348+0000 7f79bd34db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac1b2d10 con 0x559a37a82820 2024-06-10T01:17:28.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.348+0000 7f79bd34db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac1b05a0 con 0x559a37a88900 2024-06-10T01:17:28.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.348+0000 7f79bd34db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac1ae3f0 con 0x559a37a87fe0 2024-06-10T01:17:28.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.348+0000 7f79b0866700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a37a82820 0x559a37a7cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:28.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.348+0000 7f79b0866700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a37a82820 0x559a37a7cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42304/0 (socket says 172.21.15.7:42304) 2024-06-10T01:17:28.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.348+0000 7f79b0866700 1 -- 172.21.15.7:0/353693044 learned_addr learned my addr 172.21.15.7:0/353693044 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:28.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.348+0000 7f79abfff700 1 --2- 172.21.15.7:0/353693044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a37a87fe0 0x559a37a7d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:28.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.348+0000 7f79b1067700 1 --2- 172.21.15.7:0/353693044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a37a88900 0x559a37a87190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:28.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.348+0000 7f79b0866700 1 -- 172.21.15.7:0/353693044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a37a87fe0 msgr2=0x559a37a7d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:28.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.348+0000 7f79b0866700 1 --2- 172.21.15.7:0/353693044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a37a87fe0 0x559a37a7d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.348+0000 7f79b0866700 1 -- 172.21.15.7:0/353693044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a37a88900 msgr2=0x559a37a87190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:28.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.348+0000 7f79b0866700 1 --2- 172.21.15.7:0/353693044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a37a88900 0x559a37a87190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.348+0000 7f79b0866700 1 -- 172.21.15.7:0/353693044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a37a89ee0 con 0x559a37a82820 2024-06-10T01:17:28.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.348+0000 7f79b0866700 1 --2- 172.21.15.7:0/353693044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a37a82820 0x559a37a7cf00 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f79a000cf80 tx=0x7f79a009c2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87995ca32cbe86c0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:28.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.349+0000 7f79967fc700 1 -- 172.21.15.7:0/353693044 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a00b9070 con 0x559a37a82820 2024-06-10T01:17:28.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.349+0000 7f79967fc700 1 -- 172.21.15.7:0/353693044 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79a000cd60 con 0x559a37a82820 2024-06-10T01:17:28.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.349+0000 7f79967fc700 1 -- 172.21.15.7:0/353693044 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a00ba250 con 0x559a37a82820 2024-06-10T01:17:28.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.349+0000 7f79bd34db80 1 -- 172.21.15.7:0/353693044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a37a82820 msgr2=0x559a37a7cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:28.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.349+0000 7f79bd34db80 1 --2- 172.21.15.7:0/353693044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a37a82820 0x559a37a7cf00 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f79a000cf80 tx=0x7f79a009c2a0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.349+0000 7f79bd34db80 1 -- 172.21.15.7:0/353693044 shutdown_connections 2024-06-10T01:17:28.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.349+0000 7f79bd34db80 1 --2- 172.21.15.7:0/353693044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a37a88900 0x559a37a87190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.349+0000 7f79bd34db80 1 --2- 172.21.15.7:0/353693044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a37a82820 0x559a37a7cf00 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.349+0000 7f79bd34db80 1 --2- 172.21.15.7:0/353693044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a37a87fe0 0x559a37a7d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.349+0000 7f79bd34db80 1 -- 172.21.15.7:0/353693044 >> 172.21.15.7:0/353693044 conn(0x559a379af100 msgr2=0x559a37a86a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:28.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.350+0000 7f79bd34db80 1 -- 172.21.15.7:0/353693044 shutdown_connections 2024-06-10T01:17:28.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.350+0000 7f79bd34db80 1 -- 172.21.15.7:0/353693044 wait complete. 2024-06-10T01:17:28.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.350+0000 7f79bd34db80 1 -- start start 2024-06-10T01:17:28.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.350+0000 7f79bd34db80 1 Processor -- start 2024-06-10T01:17:28.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.350+0000 7f79bd34db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a37a82820 0x559a37aa0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:28.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.351+0000 7f79bd34db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a37a87fe0 0x559a37aa0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:28.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.351+0000 7f79bd34db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a37a88900 0x559a37b06850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:28.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.351+0000 7f79bd34db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac1aea60 con 0x559a37a82820 2024-06-10T01:17:28.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.351+0000 7f79bd34db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac1afed0 con 0x559a37a87fe0 2024-06-10T01:17:28.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.351+0000 7f79bd34db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac1af130 con 0x559a37a88900 2024-06-10T01:17:28.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.351+0000 7f79bd34db80 10 client.0 _ll_register_callbacks cb 0x559a37969630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:17:28.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.351+0000 7f79b0866700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a37a82820 0x559a37aa0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:28.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.351+0000 7f79b1067700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a37a88900 0x559a37b06850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:28.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.351+0000 7f79b1067700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a37a88900 0x559a37b06850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:45312/0 (socket says 172.21.15.7:45312) 2024-06-10T01:17:28.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.351+0000 7f79b1067700 1 -- 172.21.15.7:0/4061509099 learned_addr learned my addr 172.21.15.7:0/4061509099 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:28.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.351+0000 7f79abfff700 1 --2- 172.21.15.7:0/4061509099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a37a87fe0 0x559a37aa0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:28.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.351+0000 7f79b1067700 1 -- 172.21.15.7:0/4061509099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a37a87fe0 msgr2=0x559a37aa0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:28.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.351+0000 7f79b1067700 1 --2- 172.21.15.7:0/4061509099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a37a87fe0 0x559a37aa0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.351+0000 7f79b1067700 1 -- 172.21.15.7:0/4061509099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a37a82820 msgr2=0x559a37aa0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:28.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.351+0000 7f79b1067700 1 --2- 172.21.15.7:0/4061509099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a37a82820 0x559a37aa0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.351+0000 7f79b1067700 1 -- 172.21.15.7:0/4061509099 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a37aa0e40 con 0x559a37a88900 2024-06-10T01:17:28.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.351+0000 7f79b0866700 1 --2- 172.21.15.7:0/4061509099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a37a82820 0x559a37aa0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:17:28.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.351+0000 7f79abfff700 1 --2- 172.21.15.7:0/4061509099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a37a87fe0 0x559a37aa0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:17:28.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.351+0000 7f79b1067700 1 --2- 172.21.15.7:0/4061509099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a37a88900 0x559a37b06850 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f79a4007e70 tx=0x7f79a4007ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffeeb32020ad418e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:28.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.351+0000 7f798bfff700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:17:28.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.351+0000 7f798bfff700 1 -- 172.21.15.7:0/4061509099 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a4018ca0 con 0x559a37a88900 2024-06-10T01:17:28.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.352+0000 7f79bd34db80 1 -- 172.21.15.7:0/4061509099 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79a000cd60 con 0x559a37a88900 2024-06-10T01:17:28.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.352+0000 7f798bfff700 1 -- 172.21.15.7:0/4061509099 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f79a4003a00 con 0x559a37a88900 2024-06-10T01:17:28.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.352+0000 7f798bfff700 1 -- 172.21.15.7:0/4061509099 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f79a402a050 con 0x559a37a88900 2024-06-10T01:17:28.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.352+0000 7f79bd34db80 10 client.27727 fetch_fsmap learned FSMap version 40 2024-06-10T01:17:28.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.352+0000 7f79bd34db80 1 -- 172.21.15.7:0/4061509099 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559a37b07fb0 con 0x559a37a88900 2024-06-10T01:17:28.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.352+0000 7f798bfff700 1 -- 172.21.15.7:0/4061509099 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79a000cd60 con 0x559a37a88900 2024-06-10T01:17:28.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.352+0000 7f798bfff700 1 -- 172.21.15.7:0/4061509099 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f79a4021480 con 0x559a37a88900 2024-06-10T01:17:28.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.352+0000 7f79bd34db80 10 client.27727 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:17:28.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.352+0000 7f79bd34db80 10 client.27727 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:17:28.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.352+0000 7f79bd34db80 20 client.27727 populate_metadata read hostname 'smithi007' 2024-06-10T01:17:28.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.352+0000 7f79bd34db80 1 --2- 172.21.15.7:0/4061509099 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559a37b086a0 0x559a37b0aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:28.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.352+0000 7f79bd34db80 4 client.27727 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:17:28.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.352+0000 7f79bd34db80 1 -- 172.21.15.7:0/4061509099 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559a379db350 con 0x559a37b086a0 2024-06-10T01:17:28.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.352+0000 7f79b0866700 1 --2- 172.21.15.7:0/4061509099 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559a37b086a0 0x559a37b0aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:28.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.353+0000 7f798bfff700 10 client.27727 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:28.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.353+0000 7f79b0866700 1 --2- 172.21.15.7:0/4061509099 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559a37b086a0 0x559a37b0aac0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad7bcd3fffd3d3a4 server_cookie=763988119d46185 in_seq=0 out_seq=0 2024-06-10T01:17:28.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.354+0000 7f798bfff700 1 -- 172.21.15.7:0/4061509099 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559a379db350 con 0x559a37b086a0 2024-06-10T01:17:28.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.354+0000 7f798bfff700 10 client.27727 handle_command_reply: tid=0 2024-06-10T01:17:28.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.354+0000 7f79bd34db80 1 client.27727 shutdown 2024-06-10T01:17:28.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.355+0000 7f79bd34db80 1 -- 172.21.15.7:0/4061509099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a37a88900 msgr2=0x559a37b06850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:28.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.355+0000 7f79bd34db80 1 --2- 172.21.15.7:0/4061509099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a37a88900 0x559a37b06850 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f79a4007e70 tx=0x7f79a4007ea0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.355+0000 7f79bd34db80 1 -- 172.21.15.7:0/4061509099 shutdown_connections 2024-06-10T01:17:28.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.355+0000 7f79bd34db80 1 --2- 172.21.15.7:0/4061509099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a37a88900 0x559a37b06850 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.355+0000 7f79bd34db80 1 --2- 172.21.15.7:0/4061509099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a37a82820 0x559a37aa0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.355+0000 7f79bd34db80 1 --2- 172.21.15.7:0/4061509099 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559a37b086a0 0x559a37b0aac0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.355+0000 7f79bd34db80 1 --2- 172.21.15.7:0/4061509099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a37a87fe0 0x559a37aa0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.355+0000 7f798bfff700 0 client.27727 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:28.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.355+0000 7f79bd34db80 1 -- 172.21.15.7:0/4061509099 >> 172.21.15.7:0/4061509099 conn(0x559a379af100 msgr2=0x559a37a86a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:28.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.355+0000 7f79bd34db80 1 -- 172.21.15.7:0/4061509099 shutdown_connections 2024-06-10T01:17:28.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.355+0000 7f79bd34db80 1 -- 172.21.15.7:0/4061509099 wait complete. 2024-06-10T01:17:28.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.355+0000 7f79bd34db80 20 client.27727 trim_cache size 0 max 16384 2024-06-10T01:17:28.353 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:17:28.354 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-10T01:17:28.354 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "2fca286c-35aa-4d1f-abfd-650b1d9ca6a7", 2024-06-10T01:17:28.354 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-10T01:17:28.354 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:17:28.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.356+0000 7f79b2aca700 1 -- 172.21.15.7:0/41097633 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f799c0a2c20 msgr2=0x7f799c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:28.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.356+0000 7f79b2aca700 1 --2- 172.21.15.7:0/41097633 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f799c0a2c20 0x7f799c0a50e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f799800b2e0 tx=0x7f7998005960 comp rx=0 tx=0).stop 2024-06-10T01:17:28.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.356+0000 7f79b2aca700 1 -- 172.21.15.7:0/41097633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79ac06ab60 msgr2=0x7f79ac101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:28.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.356+0000 7f79b2aca700 1 --2- 172.21.15.7:0/41097633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79ac06ab60 0x7f79ac101490 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f79a000e9a0 tx=0x7f79a000ecb0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.356+0000 7f79b2aca700 1 -- 172.21.15.7:0/41097633 shutdown_connections 2024-06-10T01:17:28.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.356+0000 7f79b2aca700 1 --2- 172.21.15.7:0/41097633 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f799c0a2c20 0x7f799c0a50e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.356+0000 7f79b2aca700 1 --2- 172.21.15.7:0/41097633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79ac108530 0x7f79ac101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.356+0000 7f79b2aca700 1 --2- 172.21.15.7:0/41097633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79ac06ab60 0x7f79ac101490 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.356+0000 7f79b2aca700 1 --2- 172.21.15.7:0/41097633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79ac06b520 0x7f79ac1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:28.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.356+0000 7f79b2aca700 1 -- 172.21.15.7:0/41097633 >> 172.21.15.7:0/41097633 conn(0x7f79ac0783d0 msgr2=0x7f79ac100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:28.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.356+0000 7f79b2aca700 1 -- 172.21.15.7:0/41097633 shutdown_connections 2024-06-10T01:17:28.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:28.356+0000 7f79b2aca700 1 -- 172.21.15.7:0/41097633 wait complete. 2024-06-10T01:17:28.752 INFO:tasks.workunit.client.0.smithi007.stdout: 1 852499 7.51 MB/sec execute 489 sec latency 17.582 ms 2024-06-10T01:17:29.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:28 smithi007 ceph-mon[32398]: pgmap v776: 129 pgs: 129 active+clean; 141 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 2.5 MiB/s wr, 285 op/s 2024-06-10T01:17:29.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:28 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:17:29.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:28 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:17:29.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:28 smithi078 ceph-mon[36683]: pgmap v776: 129 pgs: 129 active+clean; 141 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 2.5 MiB/s wr, 285 op/s 2024-06-10T01:17:29.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:28 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:17:29.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:28 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:17:29.335 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:17:29.452 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:28 smithi175 ceph-mon[36636]: pgmap v776: 129 pgs: 129 active+clean; 141 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 2.5 MiB/s wr, 285 op/s 2024-06-10T01:17:29.452 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:28 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:17:29.452 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:28 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:17:29.601 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:17:29.752 INFO:tasks.workunit.client.0.smithi007.stdout: 1 854602 7.51 MB/sec execute 490 sec latency 9.873 ms 2024-06-10T01:17:30.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:29 smithi007 ceph-mon[32398]: scrub queued for path: / 2024-06-10T01:17:30.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:29 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:17:30.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:29 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:17:30.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:29 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:17:30.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:29 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:17:30.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:29 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:17:30.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:29 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:17:30.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:29 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:17:30.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:29 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:17:30.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:29 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:17:30.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:29 smithi078 ceph-mon[36683]: scrub queued for path: / 2024-06-10T01:17:30.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:29 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:17:30.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:29 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:17:30.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:29 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:17:30.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:29 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:17:30.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:29 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:17:30.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:29 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:17:30.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:29 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:17:30.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:29 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:17:30.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:29 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:17:30.452 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:29 smithi175 ceph-mon[36636]: scrub queued for path: / 2024-06-10T01:17:30.452 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:29 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:17:30.452 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:29 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:17:30.452 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:29 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:17:30.452 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:29 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:17:30.452 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:29 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:17:30.452 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:29 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:17:30.452 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:29 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:17:30.452 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:29 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:17:30.452 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:29 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:17:30.752 INFO:tasks.workunit.client.0.smithi007.stdout: 1 856221 7.52 MB/sec execute 491 sec latency 29.212 ms 2024-06-10T01:17:31.252 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:30 smithi007 ceph-mon[32398]: pgmap v777: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 2.7 MiB/s wr, 317 op/s 2024-06-10T01:17:31.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.257+0000 7eff77255700 1 -- 172.21.15.7:0/2039308647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff70107400 msgr2=0x7eff701077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:31.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.257+0000 7eff77255700 1 --2- 172.21.15.7:0/2039308647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff70107400 0x7eff701077e0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7eff64009990 tx=0x7eff64009ca0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.257+0000 7eff77255700 1 -- 172.21.15.7:0/2039308647 shutdown_connections 2024-06-10T01:17:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.257+0000 7eff77255700 1 --2- 172.21.15.7:0/2039308647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff70101d50 0x7eff7010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.257+0000 7eff77255700 1 --2- 172.21.15.7:0/2039308647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff70107400 0x7eff701077e0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.257+0000 7eff77255700 1 --2- 172.21.15.7:0/2039308647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff70101390 0x7eff70101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.257+0000 7eff77255700 1 -- 172.21.15.7:0/2039308647 >> 172.21.15.7:0/2039308647 conn(0x7eff70078340 msgr2=0x7eff70078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.258+0000 7eff77255700 1 -- 172.21.15.7:0/2039308647 shutdown_connections 2024-06-10T01:17:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.258+0000 7eff77255700 1 -- 172.21.15.7:0/2039308647 wait complete. 2024-06-10T01:17:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.258+0000 7eff77255700 1 Processor -- start 2024-06-10T01:17:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.258+0000 7eff77255700 1 -- start start 2024-06-10T01:17:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.259+0000 7eff77255700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff70101390 0x7eff70199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.259+0000 7eff77255700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff70101d50 0x7eff7019a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.259+0000 7eff74ff1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff70101390 0x7eff70199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.259+0000 7eff74ff1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff70101390 0x7eff70199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:36612/0 (socket says 172.21.15.7:36612) 2024-06-10T01:17:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.259+0000 7eff77255700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff70107400 0x7eff701a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.259+0000 7eff74ff1700 1 -- 172.21.15.7:0/10171163 learned_addr learned my addr 172.21.15.7:0/10171163 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.259+0000 7eff77255700 1 -- 172.21.15.7:0/10171163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7eff701116a0 con 0x7eff70101d50 2024-06-10T01:17:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.259+0000 7eff77255700 1 -- 172.21.15.7:0/10171163 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7eff70111520 con 0x7eff70101390 2024-06-10T01:17:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.259+0000 7eff757f2700 1 --2- 172.21.15.7:0/10171163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff70107400 0x7eff701a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.259+0000 7eff77255700 1 -- 172.21.15.7:0/10171163 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7eff70111820 con 0x7eff70107400 2024-06-10T01:17:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.260+0000 7eff6ffff700 1 --2- 172.21.15.7:0/10171163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff70101d50 0x7eff7019a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.260+0000 7eff74ff1700 1 -- 172.21.15.7:0/10171163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff70107400 msgr2=0x7eff701a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.260+0000 7eff74ff1700 1 --2- 172.21.15.7:0/10171163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff70107400 0x7eff701a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.260+0000 7eff74ff1700 1 -- 172.21.15.7:0/10171163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff70101d50 msgr2=0x7eff7019a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.260+0000 7eff74ff1700 1 --2- 172.21.15.7:0/10171163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff70101d50 0x7eff7019a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.260+0000 7eff74ff1700 1 -- 172.21.15.7:0/10171163 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff701a5100 con 0x7eff70101390 2024-06-10T01:17:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.260+0000 7eff74ff1700 1 --2- 172.21.15.7:0/10171163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff70101390 0x7eff70199b40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7eff6400b2e0 tx=0x7eff6400b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.260+0000 7eff6dffb700 1 -- 172.21.15.7:0/10171163 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff64026070 con 0x7eff70101390 2024-06-10T01:17:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.260+0000 7eff6dffb700 1 -- 172.21.15.7:0/10171163 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff64021410 con 0x7eff70101390 2024-06-10T01:17:31.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.260+0000 7eff6dffb700 1 -- 172.21.15.7:0/10171163 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff64017650 con 0x7eff70101390 2024-06-10T01:17:31.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.260+0000 7eff77255700 1 -- 172.21.15.7:0/10171163 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff701a52d0 con 0x7eff70101390 2024-06-10T01:17:31.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.261+0000 7eff77255700 1 -- 172.21.15.7:0/10171163 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff701a5670 con 0x7eff70101390 2024-06-10T01:17:31.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.263+0000 7eff6dffb700 1 -- 172.21.15.7:0/10171163 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7eff64029070 con 0x7eff70101390 2024-06-10T01:17:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.264+0000 7eff6dffb700 1 --2- 172.21.15.7:0/10171163 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7eff600a2ca0 0x7eff600a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.264+0000 7eff6ffff700 1 --2- 172.21.15.7:0/10171163 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7eff600a2ca0 0x7eff600a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.264+0000 7eff6dffb700 1 -- 172.21.15.7:0/10171163 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7eff640cffe0 con 0x7eff70101390 2024-06-10T01:17:31.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.265+0000 7eff81ad8b80 1 Processor -- start 2024-06-10T01:17:31.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.265+0000 7eff81ad8b80 1 -- start start 2024-06-10T01:17:31.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.265+0000 7eff6ffff700 1 --2- 172.21.15.7:0/10171163 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7eff600a2ca0 0x7eff600a5160 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009680 tx=0x7eff5c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:31.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.265+0000 7eff81ad8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b500f6f60 0x560b500f7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.265+0000 7eff81ad8b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b500f7880 0x560b500f1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.265+0000 7eff81ad8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b500f17a0 0x560b500f1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.265+0000 7eff81ad8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560b4ffe3f60 con 0x560b500f17a0 2024-06-10T01:17:31.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.265+0000 7eff81ad8b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560b50042850 con 0x560b500f7880 2024-06-10T01:17:31.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.265+0000 7eff81ad8b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560b4ffaa460 con 0x560b500f6f60 2024-06-10T01:17:31.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.265+0000 7eff6ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b500f7880 0x560b500f1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.265+0000 7eff74ff1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b500f6f60 0x560b500f7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.265+0000 7eff6ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b500f7880 0x560b500f1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:36628/0 (socket says 172.21.15.7:36628) 2024-06-10T01:17:31.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.265+0000 7eff6ffff700 1 -- 172.21.15.7:0/2616115710 learned_addr learned my addr 172.21.15.7:0/2616115710 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:31.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.265+0000 7eff757f2700 1 --2- 172.21.15.7:0/2616115710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b500f17a0 0x560b500f1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.265+0000 7eff6ffff700 1 -- 172.21.15.7:0/2616115710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b500f6f60 msgr2=0x560b500f7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:31.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.266+0000 7eff6ffff700 1 --2- 172.21.15.7:0/2616115710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b500f6f60 0x560b500f7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.266+0000 7eff6ffff700 1 -- 172.21.15.7:0/2616115710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b500f17a0 msgr2=0x560b500f1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:31.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.266+0000 7eff6ffff700 1 --2- 172.21.15.7:0/2616115710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b500f17a0 0x560b500f1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.266+0000 7eff6ffff700 1 -- 172.21.15.7:0/2616115710 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b4ff918d0 con 0x560b500f7880 2024-06-10T01:17:31.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.266+0000 7eff757f2700 1 --2- 172.21.15.7:0/2616115710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b500f17a0 0x560b500f1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:17:31.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.266+0000 7eff6ffff700 1 --2- 172.21.15.7:0/2616115710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b500f7880 0x560b500f1260 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7eff5c011e20 tx=0x7eff5c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be63a24184d73bea server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:31.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.266+0000 7eff74ff1700 1 --2- 172.21.15.7:0/2616115710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b500f6f60 0x560b500f7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:17:31.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.266+0000 7eff5affd700 1 -- 172.21.15.7:0/2616115710 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c01fd60 con 0x560b500f7880 2024-06-10T01:17:31.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.266+0000 7eff5affd700 1 -- 172.21.15.7:0/2616115710 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff5c023070 con 0x560b500f7880 2024-06-10T01:17:31.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.266+0000 7eff5affd700 1 -- 172.21.15.7:0/2616115710 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c005020 con 0x560b500f7880 2024-06-10T01:17:31.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.267+0000 7eff81ad8b80 1 -- 172.21.15.7:0/2616115710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b500f7880 msgr2=0x560b500f1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:31.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.267+0000 7eff81ad8b80 1 --2- 172.21.15.7:0/2616115710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b500f7880 0x560b500f1260 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7eff5c011e20 tx=0x7eff5c014000 comp rx=0 tx=0).stop 2024-06-10T01:17:31.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.267+0000 7eff81ad8b80 1 -- 172.21.15.7:0/2616115710 shutdown_connections 2024-06-10T01:17:31.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.267+0000 7eff81ad8b80 1 --2- 172.21.15.7:0/2616115710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b500f17a0 0x560b500f1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.267+0000 7eff81ad8b80 1 --2- 172.21.15.7:0/2616115710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b500f6f60 0x560b500f7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.267+0000 7eff81ad8b80 1 --2- 172.21.15.7:0/2616115710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b500f7880 0x560b500f1260 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.267+0000 7eff81ad8b80 1 -- 172.21.15.7:0/2616115710 >> 172.21.15.7:0/2616115710 conn(0x560b5001e080 msgr2=0x560b500ec220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:31.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.267+0000 7eff81ad8b80 1 -- 172.21.15.7:0/2616115710 shutdown_connections 2024-06-10T01:17:31.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.268+0000 7eff81ad8b80 1 -- 172.21.15.7:0/2616115710 wait complete. 2024-06-10T01:17:31.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.268+0000 7eff81ad8b80 1 -- start start 2024-06-10T01:17:31.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.268+0000 7eff81ad8b80 1 Processor -- start 2024-06-10T01:17:31.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.269+0000 7eff81ad8b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b500f17a0 0x560b500fd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.269+0000 7eff81ad8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b500f6f60 0x560b500fa890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.269+0000 7eff81ad8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b500f7880 0x560b500fadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.269+0000 7eff81ad8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560b4ffe2fc0 con 0x560b500f7880 2024-06-10T01:17:31.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.269+0000 7eff74ff1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b500f17a0 0x560b500fd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.269+0000 7eff74ff1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b500f17a0 0x560b500fd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:36630/0 (socket says 172.21.15.7:36630) 2024-06-10T01:17:31.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.269+0000 7eff74ff1700 1 -- 172.21.15.7:0/3526983271 learned_addr learned my addr 172.21.15.7:0/3526983271 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:31.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.269+0000 7eff6ffff700 1 --2- 172.21.15.7:0/3526983271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b500f6f60 0x560b500fa890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.269+0000 7eff757f2700 1 --2- 172.21.15.7:0/3526983271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b500f7880 0x560b500fadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.269+0000 7eff81ad8b80 1 -- 172.21.15.7:0/3526983271 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560b5003e810 con 0x560b500f17a0 2024-06-10T01:17:31.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.270+0000 7eff81ad8b80 1 -- 172.21.15.7:0/3526983271 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560b500fe260 con 0x560b500f6f60 2024-06-10T01:17:31.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.270+0000 7eff81ad8b80 10 client.0 _ll_register_callbacks cb 0x560b4ffd85b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:17:31.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.270+0000 7eff74ff1700 1 -- 172.21.15.7:0/3526983271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b500f6f60 msgr2=0x560b500fa890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:31.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.270+0000 7eff74ff1700 1 --2- 172.21.15.7:0/3526983271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b500f6f60 0x560b500fa890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.270+0000 7eff74ff1700 1 -- 172.21.15.7:0/3526983271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b500f7880 msgr2=0x560b500fadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:31.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.270+0000 7eff74ff1700 1 --2- 172.21.15.7:0/3526983271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b500f7880 0x560b500fadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.270+0000 7eff74ff1700 1 -- 172.21.15.7:0/3526983271 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b50008750 con 0x560b500f17a0 2024-06-10T01:17:31.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.270+0000 7eff74ff1700 1 --2- 172.21.15.7:0/3526983271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b500f17a0 0x560b500fd590 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7eff6409c5a0 tx=0x7eff6409c1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ac5bbb82c386c6b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:31.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.270+0000 7eff4ffff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:17:31.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.271+0000 7eff4ffff700 1 -- 172.21.15.7:0/3526983271 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff640b4020 con 0x560b500f17a0 2024-06-10T01:17:31.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.271+0000 7eff4ffff700 1 -- 172.21.15.7:0/3526983271 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7eff6409ca70 con 0x560b500f17a0 2024-06-10T01:17:31.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.271+0000 7eff81ad8b80 1 -- 172.21.15.7:0/3526983271 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff5c023070 con 0x560b500f17a0 2024-06-10T01:17:31.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.271+0000 7eff4ffff700 1 -- 172.21.15.7:0/3526983271 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7eff640095b0 con 0x560b500f17a0 2024-06-10T01:17:31.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.271+0000 7eff81ad8b80 10 client.27719 fetch_fsmap learned FSMap version 40 2024-06-10T01:17:31.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.271+0000 7eff81ad8b80 1 -- 172.21.15.7:0/3526983271 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560b501089a0 con 0x560b500f17a0 2024-06-10T01:17:31.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.271+0000 7eff4ffff700 1 -- 172.21.15.7:0/3526983271 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff64098020 con 0x560b500f17a0 2024-06-10T01:17:31.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.272+0000 7eff4ffff700 1 -- 172.21.15.7:0/3526983271 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7eff640cdbc0 con 0x560b500f17a0 2024-06-10T01:17:31.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.272+0000 7eff81ad8b80 10 client.27719 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:17:31.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.272+0000 7eff81ad8b80 10 client.27719 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:17:31.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.272+0000 7eff81ad8b80 20 client.27719 populate_metadata read hostname 'smithi007' 2024-06-10T01:17:31.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.272+0000 7eff81ad8b80 1 --2- 172.21.15.7:0/3526983271 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560b5010b0f0 0x560b5010d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.272+0000 7eff81ad8b80 4 client.27719 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:17:31.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.272+0000 7eff81ad8b80 1 -- 172.21.15.7:0/3526983271 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7eff5c023070 con 0x560b5010b0f0 2024-06-10T01:17:31.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.272+0000 7eff6ffff700 1 --2- 172.21.15.7:0/3526983271 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560b5010b0f0 0x560b5010d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.273+0000 7eff6ffff700 1 --2- 172.21.15.7:0/3526983271 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560b5010b0f0 0x560b5010d540 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6756f2df5d6c03b4 server_cookie=880b14807eb405bc in_seq=0 out_seq=0 2024-06-10T01:17:31.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.273+0000 7eff4ffff700 10 client.27719 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:31.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.275+0000 7eff4ffff700 1 -- 172.21.15.7:0/3526983271 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7eff5c023070 con 0x560b5010b0f0 2024-06-10T01:17:31.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.275+0000 7eff4ffff700 10 client.27719 handle_command_reply: tid=0 2024-06-10T01:17:31.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.275+0000 7eff81ad8b80 1 client.27719 shutdown 2024-06-10T01:17:31.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.275+0000 7eff81ad8b80 1 -- 172.21.15.7:0/3526983271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b500f17a0 msgr2=0x560b500fd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:31.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.275+0000 7eff81ad8b80 1 --2- 172.21.15.7:0/3526983271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b500f17a0 0x560b500fd590 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7eff6409c5a0 tx=0x7eff6409c1d0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.275+0000 7eff81ad8b80 1 -- 172.21.15.7:0/3526983271 shutdown_connections 2024-06-10T01:17:31.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.275+0000 7eff81ad8b80 1 --2- 172.21.15.7:0/3526983271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b500f7880 0x560b500fadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.275+0000 7eff81ad8b80 1 --2- 172.21.15.7:0/3526983271 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560b5010b0f0 0x560b5010d540 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.275+0000 7eff81ad8b80 1 --2- 172.21.15.7:0/3526983271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b500f17a0 0x560b500fd590 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.275+0000 7eff81ad8b80 1 --2- 172.21.15.7:0/3526983271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b500f6f60 0x560b500fa890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.275+0000 7eff4ffff700 0 client.27719 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:31.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.275+0000 7eff81ad8b80 1 -- 172.21.15.7:0/3526983271 >> 172.21.15.7:0/3526983271 conn(0x560b5001e080 msgr2=0x560b500eb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:31.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.275+0000 7eff81ad8b80 1 -- 172.21.15.7:0/3526983271 shutdown_connections 2024-06-10T01:17:31.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.275+0000 7eff81ad8b80 1 -- 172.21.15.7:0/3526983271 wait complete. 2024-06-10T01:17:31.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.275+0000 7eff81ad8b80 20 client.27719 trim_cache size 0 max 16384 2024-06-10T01:17:31.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.298+0000 7eff81ad8b80 1 Processor -- start 2024-06-10T01:17:31.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.298+0000 7eff81ad8b80 1 -- start start 2024-06-10T01:17:31.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.298+0000 7eff81ad8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b500f17a0 0x560b500f8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.298+0000 7eff81ad8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b500f6f60 0x560b500f9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.298+0000 7eff81ad8b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b500f7880 0x560b500fc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.298+0000 7eff81ad8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7eff701ae840 con 0x560b500f17a0 2024-06-10T01:17:31.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.298+0000 7eff81ad8b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7eff701ac0d0 con 0x560b500f7880 2024-06-10T01:17:31.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.298+0000 7eff81ad8b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7eff701a9f20 con 0x560b500f6f60 2024-06-10T01:17:31.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.298+0000 7eff74ff1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b500f17a0 0x560b500f8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.298+0000 7eff74ff1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b500f17a0 0x560b500f8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42354/0 (socket says 172.21.15.7:42354) 2024-06-10T01:17:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.298+0000 7eff74ff1700 1 -- 172.21.15.7:0/3850831879 learned_addr learned my addr 172.21.15.7:0/3850831879 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.298+0000 7eff757f2700 1 --2- 172.21.15.7:0/3850831879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b500f7880 0x560b500fc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.299+0000 7eff6ffff700 1 --2- 172.21.15.7:0/3850831879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b500f6f60 0x560b500f9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.299+0000 7eff74ff1700 1 -- 172.21.15.7:0/3850831879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b500f6f60 msgr2=0x560b500f9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.299+0000 7eff74ff1700 1 --2- 172.21.15.7:0/3850831879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b500f6f60 0x560b500f9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.299+0000 7eff74ff1700 1 -- 172.21.15.7:0/3850831879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b500f7880 msgr2=0x560b500fc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.299+0000 7eff74ff1700 1 --2- 172.21.15.7:0/3850831879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b500f7880 0x560b500fc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.299+0000 7eff74ff1700 1 -- 172.21.15.7:0/3850831879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b500f8e60 con 0x560b500f17a0 2024-06-10T01:17:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.299+0000 7eff6ffff700 1 --2- 172.21.15.7:0/3850831879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b500f6f60 0x560b500f9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:17:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.299+0000 7eff757f2700 1 --2- 172.21.15.7:0/3850831879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b500f7880 0x560b500fc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:17:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.299+0000 7eff74ff1700 1 --2- 172.21.15.7:0/3850831879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b500f17a0 0x560b500f8530 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7eff6409c310 tx=0x7eff640a3050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e06b5707d1c6b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.299+0000 7eff5a7fc700 1 -- 172.21.15.7:0/3850831879 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff640b4070 con 0x560b500f17a0 2024-06-10T01:17:31.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.299+0000 7eff5a7fc700 1 -- 172.21.15.7:0/3850831879 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff6409cd10 con 0x560b500f17a0 2024-06-10T01:17:31.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.299+0000 7eff5a7fc700 1 -- 172.21.15.7:0/3850831879 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6409cad0 con 0x560b500f17a0 2024-06-10T01:17:31.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.300+0000 7eff81ad8b80 1 -- 172.21.15.7:0/3850831879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b500f17a0 msgr2=0x560b500f8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:31.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.300+0000 7eff81ad8b80 1 --2- 172.21.15.7:0/3850831879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b500f17a0 0x560b500f8530 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7eff6409c310 tx=0x7eff640a3050 comp rx=0 tx=0).stop 2024-06-10T01:17:31.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.300+0000 7eff81ad8b80 1 -- 172.21.15.7:0/3850831879 shutdown_connections 2024-06-10T01:17:31.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.300+0000 7eff81ad8b80 1 --2- 172.21.15.7:0/3850831879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b500f7880 0x560b500fc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.300+0000 7eff81ad8b80 1 --2- 172.21.15.7:0/3850831879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b500f17a0 0x560b500f8530 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.300+0000 7eff81ad8b80 1 --2- 172.21.15.7:0/3850831879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b500f6f60 0x560b500f9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.300+0000 7eff81ad8b80 1 -- 172.21.15.7:0/3850831879 >> 172.21.15.7:0/3850831879 conn(0x560b5001e080 msgr2=0x560b500fc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:31.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.300+0000 7eff81ad8b80 1 -- 172.21.15.7:0/3850831879 shutdown_connections 2024-06-10T01:17:31.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.300+0000 7eff81ad8b80 1 -- 172.21.15.7:0/3850831879 wait complete. 2024-06-10T01:17:31.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.301+0000 7eff81ad8b80 1 -- start start 2024-06-10T01:17:31.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.301+0000 7eff81ad8b80 1 Processor -- start 2024-06-10T01:17:31.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.301+0000 7eff81ad8b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b500f17a0 0x560b50113440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.301+0000 7eff81ad8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b500f6f60 0x560b501107e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.301+0000 7eff81ad8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b500f7880 0x560b50110d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.301+0000 7eff81ad8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7eff701aa590 con 0x560b500f7880 2024-06-10T01:17:31.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.301+0000 7eff81ad8b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7eff701aba00 con 0x560b500f17a0 2024-06-10T01:17:31.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.301+0000 7eff81ad8b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7eff701aac60 con 0x560b500f6f60 2024-06-10T01:17:31.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.301+0000 7eff81ad8b80 10 client.0 _ll_register_callbacks cb 0x560b4ffd85b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:17:31.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.301+0000 7eff757f2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b500f7880 0x560b50110d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.301+0000 7eff6ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b500f6f60 0x560b501107e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.301+0000 7eff757f2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b500f7880 0x560b50110d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42356/0 (socket says 172.21.15.7:42356) 2024-06-10T01:17:31.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.301+0000 7eff757f2700 1 -- 172.21.15.7:0/3400403650 learned_addr learned my addr 172.21.15.7:0/3400403650 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:31.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.302+0000 7eff74ff1700 1 --2- 172.21.15.7:0/3400403650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b500f17a0 0x560b50113440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.302+0000 7eff757f2700 1 -- 172.21.15.7:0/3400403650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b500f6f60 msgr2=0x560b501107e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:31.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.302+0000 7eff757f2700 1 --2- 172.21.15.7:0/3400403650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b500f6f60 0x560b501107e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.302+0000 7eff757f2700 1 -- 172.21.15.7:0/3400403650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b500f17a0 msgr2=0x560b50113440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:31.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.302+0000 7eff757f2700 1 --2- 172.21.15.7:0/3400403650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b500f17a0 0x560b50113440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.302+0000 7eff757f2700 1 -- 172.21.15.7:0/3400403650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b50111560 con 0x560b500f7880 2024-06-10T01:17:31.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.302+0000 7eff6ffff700 1 --2- 172.21.15.7:0/3400403650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b500f6f60 0x560b501107e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:17:31.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.302+0000 7eff757f2700 1 --2- 172.21.15.7:0/3400403650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b500f7880 0x560b50110d20 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7eff68008b30 tx=0x7eff68008e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc8445754dca5517 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:31.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.302+0000 7eff4ffff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:17:31.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.302+0000 7eff4ffff700 1 -- 172.21.15.7:0/3400403650 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff68028070 con 0x560b500f7880 2024-06-10T01:17:31.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.302+0000 7eff4ffff700 1 -- 172.21.15.7:0/3400403650 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7eff68003c00 con 0x560b500f7880 2024-06-10T01:17:31.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.302+0000 7eff81ad8b80 1 -- 172.21.15.7:0/3400403650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff6409cd10 con 0x560b500f7880 2024-06-10T01:17:31.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.302+0000 7eff4ffff700 1 -- 172.21.15.7:0/3400403650 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7eff68034050 con 0x560b500f7880 2024-06-10T01:17:31.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.302+0000 7eff81ad8b80 10 client.18387 fetch_fsmap learned FSMap version 40 2024-06-10T01:17:31.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.302+0000 7eff81ad8b80 1 -- 172.21.15.7:0/3400403650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560b50175d90 con 0x560b500f7880 2024-06-10T01:17:31.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.303+0000 7eff4ffff700 1 -- 172.21.15.7:0/3400403650 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff6409cd10 con 0x560b500f7880 2024-06-10T01:17:31.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.303+0000 7eff4ffff700 1 -- 172.21.15.7:0/3400403650 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7eff680042c0 con 0x560b500f7880 2024-06-10T01:17:31.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.303+0000 7eff81ad8b80 10 client.18387 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:17:31.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.303+0000 7eff81ad8b80 10 client.18387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:17:31.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.303+0000 7eff81ad8b80 20 client.18387 populate_metadata read hostname 'smithi007' 2024-06-10T01:17:31.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.303+0000 7eff81ad8b80 1 --2- 172.21.15.7:0/3400403650 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560b501764a0 0x560b501788c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.303+0000 7eff81ad8b80 4 client.18387 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:17:31.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.303+0000 7eff81ad8b80 1 -- 172.21.15.7:0/3400403650 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560b5004fd50 con 0x560b501764a0 2024-06-10T01:17:31.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.303+0000 7eff74ff1700 1 --2- 172.21.15.7:0/3400403650 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560b501764a0 0x560b501788c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.304+0000 7eff74ff1700 1 --2- 172.21.15.7:0/3400403650 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560b501764a0 0x560b501788c0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b07cea38f00259c server_cookie=54549ad22baf93cd in_seq=0 out_seq=0 2024-06-10T01:17:31.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.304+0000 7eff4ffff700 10 client.18387 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:31.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.304+0000 7eff4ffff700 1 -- 172.21.15.7:0/3400403650 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560b5004fd50 con 0x560b501764a0 2024-06-10T01:17:31.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.304+0000 7eff4ffff700 10 client.18387 handle_command_reply: tid=0 2024-06-10T01:17:31.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.304+0000 7eff81ad8b80 1 client.18387 shutdown 2024-06-10T01:17:31.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.304+0000 7eff81ad8b80 1 -- 172.21.15.7:0/3400403650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b500f7880 msgr2=0x560b50110d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:31.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.304+0000 7eff81ad8b80 1 --2- 172.21.15.7:0/3400403650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b500f7880 0x560b50110d20 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7eff68008b30 tx=0x7eff68008e40 comp rx=0 tx=0).stop 2024-06-10T01:17:31.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.304+0000 7eff81ad8b80 1 -- 172.21.15.7:0/3400403650 shutdown_connections 2024-06-10T01:17:31.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.304+0000 7eff81ad8b80 1 --2- 172.21.15.7:0/3400403650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b500f7880 0x560b50110d20 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.304+0000 7eff81ad8b80 1 --2- 172.21.15.7:0/3400403650 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560b501764a0 0x560b501788c0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.304+0000 7eff81ad8b80 1 --2- 172.21.15.7:0/3400403650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b500f17a0 0x560b50113440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.304+0000 7eff4ffff700 0 client.18387 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:31.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.304+0000 7eff81ad8b80 1 --2- 172.21.15.7:0/3400403650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b500f6f60 0x560b501107e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.304+0000 7eff81ad8b80 1 -- 172.21.15.7:0/3400403650 >> 172.21.15.7:0/3400403650 conn(0x560b5001e080 msgr2=0x560b500fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:31.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.304+0000 7eff81ad8b80 1 -- 172.21.15.7:0/3400403650 shutdown_connections 2024-06-10T01:17:31.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.304+0000 7eff81ad8b80 1 -- 172.21.15.7:0/3400403650 wait complete. 2024-06-10T01:17:31.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.304+0000 7eff81ad8b80 20 client.18387 trim_cache size 0 max 16384 2024-06-10T01:17:31.303 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:17:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T01:17:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-10T01:17:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "2fca286c-35aa-4d1f-abfd-650b1d9ca6a7": { 2024-06-10T01:17:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-10T01:17:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "2fca286c-35aa-4d1f-abfd-650b1d9ca6a7", 2024-06-10T01:17:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-10T01:17:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:17:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:17:31.304 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:17:31.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.305+0000 7eff77255700 1 -- 172.21.15.7:0/10171163 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7eff600a2ca0 msgr2=0x7eff600a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:31.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.305+0000 7eff77255700 1 --2- 172.21.15.7:0/10171163 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7eff600a2ca0 0x7eff600a5160 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009680 tx=0x7eff5c009260 comp rx=0 tx=0).stop 2024-06-10T01:17:31.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.305+0000 7eff77255700 1 -- 172.21.15.7:0/10171163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff70101390 msgr2=0x7eff70199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:31.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.305+0000 7eff77255700 1 --2- 172.21.15.7:0/10171163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff70101390 0x7eff70199b40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7eff6400b2e0 tx=0x7eff6400b610 comp rx=0 tx=0).stop 2024-06-10T01:17:31.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.305+0000 7eff77255700 1 -- 172.21.15.7:0/10171163 shutdown_connections 2024-06-10T01:17:31.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.305+0000 7eff77255700 1 --2- 172.21.15.7:0/10171163 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7eff600a2ca0 0x7eff600a5160 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.305+0000 7eff77255700 1 --2- 172.21.15.7:0/10171163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff70107400 0x7eff701a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.305+0000 7eff77255700 1 --2- 172.21.15.7:0/10171163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff70101390 0x7eff70199b40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.305+0000 7eff77255700 1 --2- 172.21.15.7:0/10171163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff70101d50 0x7eff7019a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.305+0000 7eff77255700 1 -- 172.21.15.7:0/10171163 >> 172.21.15.7:0/10171163 conn(0x7eff70078340 msgr2=0x7eff700fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:31.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.305+0000 7eff77255700 1 -- 172.21.15.7:0/10171163 shutdown_connections 2024-06-10T01:17:31.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:31.305+0000 7eff77255700 1 -- 172.21.15.7:0/10171163 wait complete. 2024-06-10T01:17:31.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:30 smithi078 ceph-mon[36683]: pgmap v777: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 2.7 MiB/s wr, 317 op/s 2024-06-10T01:17:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.345+0000 7f1ed2af7640 1 -- 172.21.15.7:0/2510601772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ecc1024b0 msgr2=0x7f1ecc108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.345+0000 7f1ed2af7640 1 --2- 172.21.15.7:0/2510601772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ecc1024b0 0x7f1ecc108d40 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4009ca0 tx=0x7f1ec402a8c0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.345+0000 7f1ed2af7640 1 -- 172.21.15.7:0/2510601772 shutdown_connections 2024-06-10T01:17:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.345+0000 7f1ed2af7640 1 --2- 172.21.15.7:0/2510601772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ecc1024b0 0x7f1ecc108d40 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.345+0000 7f1ed2af7640 1 --2- 172.21.15.7:0/2510601772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ecc101af0 0x7f1ecc101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.345+0000 7f1ed2af7640 1 --2- 172.21.15.7:0/2510601772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ecc100900 0x7f1ecc100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.345+0000 7f1ed2af7640 1 -- 172.21.15.7:0/2510601772 >> 172.21.15.7:0/2510601772 conn(0x7f1ecc0fbe70 msgr2=0x7f1ecc0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.345+0000 7f1ed2af7640 1 -- 172.21.15.7:0/2510601772 shutdown_connections 2024-06-10T01:17:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.345+0000 7f1ed2af7640 1 -- 172.21.15.7:0/2510601772 wait complete. 2024-06-10T01:17:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.346+0000 7f1ed2af7640 1 Processor -- start 2024-06-10T01:17:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.346+0000 7f1ed2af7640 1 -- start start 2024-06-10T01:17:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.346+0000 7f1ed2af7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ecc100900 0x7f1ecc197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.346+0000 7f1ed2af7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ecc101af0 0x7f1ecc198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.346+0000 7f1ed2af7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ecc1024b0 0x7f1ecc19f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.346+0000 7f1ed2af7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc10c580 con 0x7f1ecc101af0 2024-06-10T01:17:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.346+0000 7f1ed2af7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc10c400 con 0x7f1ecc100900 2024-06-10T01:17:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.346+0000 7f1ed2af7640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc10c700 con 0x7f1ecc1024b0 2024-06-10T01:17:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.346+0000 7f1ec3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ecc101af0 0x7f1ecc198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.346+0000 7f1ed0893640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ecc100900 0x7f1ecc197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.346+0000 7f1ed0893640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ecc100900 0x7f1ecc197d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:36656/0 (socket says 172.21.15.7:36656) 2024-06-10T01:17:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.346+0000 7f1ed0893640 1 -- 172.21.15.7:0/1173050266 learned_addr learned my addr 172.21.15.7:0/1173050266 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.346+0000 7f1ec3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ecc101af0 0x7f1ecc198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42374/0 (socket says 172.21.15.7:42374) 2024-06-10T01:17:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.346+0000 7f1ed1094640 1 --2- 172.21.15.7:0/1173050266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ecc1024b0 0x7f1ecc19f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.346+0000 7f1ec3fff640 1 -- 172.21.15.7:0/1173050266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ecc1024b0 msgr2=0x7f1ecc19f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.346+0000 7f1ec3fff640 1 --2- 172.21.15.7:0/1173050266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ecc1024b0 0x7f1ecc19f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.346+0000 7f1ec3fff640 1 -- 172.21.15.7:0/1173050266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ecc100900 msgr2=0x7f1ecc197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.346+0000 7f1ec3fff640 1 --2- 172.21.15.7:0/1173050266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ecc100900 0x7f1ecc197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.346+0000 7f1ec3fff640 1 -- 172.21.15.7:0/1173050266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ecc19f940 con 0x7f1ecc101af0 2024-06-10T01:17:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.346+0000 7f1ed0893640 1 --2- 172.21.15.7:0/1173050266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ecc100900 0x7f1ecc197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:17:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.347+0000 7f1ec3fff640 1 --2- 172.21.15.7:0/1173050266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ecc101af0 0x7f1ecc198270 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00e9f0 tx=0x7f1ebc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.347+0000 7f1ec1ffb640 1 -- 172.21.15.7:0/1173050266 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc00ccc0 con 0x7f1ecc101af0 2024-06-10T01:17:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.347+0000 7f1ec1ffb640 1 -- 172.21.15.7:0/1173050266 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ebc005020 con 0x7f1ecc101af0 2024-06-10T01:17:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.347+0000 7f1ed2af7640 1 -- 172.21.15.7:0/1173050266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ecc19fc30 con 0x7f1ecc101af0 2024-06-10T01:17:31.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.347+0000 7f1ed2af7640 1 -- 172.21.15.7:0/1173050266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ecc1a0180 con 0x7f1ecc101af0 2024-06-10T01:17:31.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.347+0000 7f1ec1ffb640 1 -- 172.21.15.7:0/1173050266 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc0049e0 con 0x7f1ecc101af0 2024-06-10T01:17:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.349+0000 7f1ec1ffb640 1 -- 172.21.15.7:0/1173050266 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f1ebc005560 con 0x7f1ecc101af0 2024-06-10T01:17:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.350+0000 7f1ed57fd740 1 Processor -- start 2024-06-10T01:17:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.350+0000 7f1ed57fd740 1 -- start start 2024-06-10T01:17:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.350+0000 7f1ed57fd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3b9d9f420 0x55f3b9d9f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.350+0000 7f1ed57fd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3b9d9fe10 0x55f3b9da4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.350+0000 7f1ed1094640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3b9d9f420 0x55f3b9d9f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.351+0000 7f1ed0893640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3b9d9fe10 0x55f3b9da4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.351+0000 7f1ed1094640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3b9d9f420 0x55f3b9d9f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:45392/0 (socket says 172.21.15.7:45392) 2024-06-10T01:17:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.351+0000 7f1ed1094640 1 -- 172.21.15.7:0/2780344274 learned_addr learned my addr 172.21.15.7:0/2780344274 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.351+0000 7f1ec1ffb640 1 --2- 172.21.15.7:0/1173050266 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1eb40a2c00 0x7f1eb40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.351+0000 7f1ed57fd740 1 --2- 172.21.15.7:0/2780344274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3b9da4f80 0x55f3b9da7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.351+0000 7f1ed57fd740 1 -- 172.21.15.7:0/2780344274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f3b9cfe4f0 con 0x55f3b9d9fe10 2024-06-10T01:17:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.351+0000 7f1ed57fd740 1 -- 172.21.15.7:0/2780344274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f3b9c9c870 con 0x55f3b9da4f80 2024-06-10T01:17:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.351+0000 7f1ed57fd740 1 -- 172.21.15.7:0/2780344274 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f3b9cfa4b0 con 0x55f3b9d9f420 2024-06-10T01:17:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.351+0000 7f1ec3fff640 1 --2- 172.21.15.7:0/1173050266 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1eb40a2c00 0x7f1eb40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.351+0000 7f1ec1ffb640 1 -- 172.21.15.7:0/1173050266 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f1ebc0cf350 con 0x7f1ecc101af0 2024-06-10T01:17:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.351+0000 7f1ed1094640 1 -- 172.21.15.7:0/2780344274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3b9da4f80 msgr2=0x55f3b9da7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.351+0000 7f1ed1094640 1 --2- 172.21.15.7:0/2780344274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3b9da4f80 0x55f3b9da7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.351+0000 7f1ed1094640 1 -- 172.21.15.7:0/2780344274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3b9d9fe10 msgr2=0x55f3b9da4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.351+0000 7f1ed1094640 1 --2- 172.21.15.7:0/2780344274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3b9d9fe10 0x55f3b9da4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.351+0000 7f1ed1094640 1 -- 172.21.15.7:0/2780344274 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3b9cb89b0 con 0x55f3b9d9f420 2024-06-10T01:17:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.351+0000 7f1ed0893640 1 --2- 172.21.15.7:0/2780344274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3b9d9fe10 0x55f3b9da4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:17:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.351+0000 7f1ec3fff640 1 --2- 172.21.15.7:0/1173050266 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1eb40a2c00 0x7f1eb40a50c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc097c00 tx=0x7f1ebc0c63c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.351+0000 7f1ed1094640 1 --2- 172.21.15.7:0/2780344274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3b9d9f420 0x55f3b9d9f840 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4009ca0 tx=0x7f1ec40093b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f437bd761e13f2c0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.351+0000 7f1eb2ffd640 1 -- 172.21.15.7:0/2780344274 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec40098c0 con 0x55f3b9d9f420 2024-06-10T01:17:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.352+0000 7f1eb2ffd640 1 -- 172.21.15.7:0/2780344274 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ec4036040 con 0x55f3b9d9f420 2024-06-10T01:17:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.352+0000 7f1eb2ffd640 1 -- 172.21.15.7:0/2780344274 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec40043b0 con 0x55f3b9d9f420 2024-06-10T01:17:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.352+0000 7f1ed57fd740 1 -- 172.21.15.7:0/2780344274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3b9d9f420 msgr2=0x55f3b9d9f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.352+0000 7f1ed57fd740 1 --2- 172.21.15.7:0/2780344274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3b9d9f420 0x55f3b9d9f840 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4009ca0 tx=0x7f1ec40093b0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.352+0000 7f1ed57fd740 1 -- 172.21.15.7:0/2780344274 shutdown_connections 2024-06-10T01:17:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.352+0000 7f1ed57fd740 1 --2- 172.21.15.7:0/2780344274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3b9da4f80 0x55f3b9da7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.352+0000 7f1ed57fd740 1 --2- 172.21.15.7:0/2780344274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3b9d9fe10 0x55f3b9da4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.352+0000 7f1ed57fd740 1 --2- 172.21.15.7:0/2780344274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3b9d9f420 0x55f3b9d9f840 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.352+0000 7f1ed57fd740 1 -- 172.21.15.7:0/2780344274 >> 172.21.15.7:0/2780344274 conn(0x55f3b9d96680 msgr2=0x55f3b9d96a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.352+0000 7f1ed57fd740 1 -- 172.21.15.7:0/2780344274 shutdown_connections 2024-06-10T01:17:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.352+0000 7f1ed57fd740 1 -- 172.21.15.7:0/2780344274 wait complete. 2024-06-10T01:17:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.353+0000 7f1ed57fd740 1 -- start start 2024-06-10T01:17:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.353+0000 7f1ed57fd740 1 Processor -- start 2024-06-10T01:17:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.353+0000 7f1ed57fd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3b9d9f420 0x55f3b9da9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.353+0000 7f1ed57fd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3b9d9fe10 0x55f3b9da99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.353+0000 7f1ed57fd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3b9da4f80 0x55f3b9db0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.353+0000 7f1ed57fd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f3b9da7cf0 con 0x55f3b9d9f420 2024-06-10T01:17:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.353+0000 7f1ed57fd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f3b9c87ae0 con 0x55f3b9da4f80 2024-06-10T01:17:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.353+0000 7f1ed57fd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f3b9da7e70 con 0x55f3b9d9fe10 2024-06-10T01:17:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.353+0000 7f1ed57fd740 10 client.0 _ll_register_callbacks cb 0x55f3b9cc09b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:17:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.353+0000 7f1ed1094640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3b9d9f420 0x55f3b9da9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.353+0000 7f1ed1094640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3b9d9f420 0x55f3b9da9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42378/0 (socket says 172.21.15.7:42378) 2024-06-10T01:17:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.353+0000 7f1ed1094640 1 -- 172.21.15.7:0/838401746 learned_addr learned my addr 172.21.15.7:0/838401746 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.353+0000 7f1ed0893640 1 --2- 172.21.15.7:0/838401746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3b9d9fe10 0x55f3b9da99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.353+0000 7f1ed1094640 1 --2- 172.21.15.7:0/838401746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3b9da4f80 0x55f3b9db0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.354+0000 7f1ed1094640 1 -- 172.21.15.7:0/838401746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3b9d9fe10 msgr2=0x55f3b9da99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.354+0000 7f1ed1094640 1 --2- 172.21.15.7:0/838401746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3b9d9fe10 0x55f3b9da99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.354+0000 7f1ed1094640 1 -- 172.21.15.7:0/838401746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3b9da4f80 msgr2=0x55f3b9db0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.354+0000 7f1ed1094640 1 --2- 172.21.15.7:0/838401746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3b9da4f80 0x55f3b9db0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.354+0000 7f1ed1094640 1 -- 172.21.15.7:0/838401746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3b9a12210 con 0x55f3b9d9f420 2024-06-10T01:17:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.354+0000 7f1ed1094640 1 --2- 172.21.15.7:0/838401746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3b9d9f420 0x55f3b9da9480 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4003e40 tx=0x7f1ec4003fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e580331971388f6b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.354+0000 7f1eabfff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:17:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.354+0000 7f1eabfff640 1 -- 172.21.15.7:0/838401746 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4031860 con 0x55f3b9d9f420 2024-06-10T01:17:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.354+0000 7f1ed57fd740 1 -- 172.21.15.7:0/838401746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ec4036040 con 0x55f3b9d9f420 2024-06-10T01:17:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.354+0000 7f1eabfff640 1 -- 172.21.15.7:0/838401746 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f1ec4004130 con 0x55f3b9d9f420 2024-06-10T01:17:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.354+0000 7f1eabfff640 1 -- 172.21.15.7:0/838401746 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f1ec4004330 con 0x55f3b9d9f420 2024-06-10T01:17:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.354+0000 7f1ed57fd740 10 client.18399 fetch_fsmap learned FSMap version 40 2024-06-10T01:17:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.354+0000 7f1ed57fd740 1 -- 172.21.15.7:0/838401746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f3b9ade620 con 0x55f3b9d9f420 2024-06-10T01:17:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.355+0000 7f1eabfff640 1 -- 172.21.15.7:0/838401746 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ec4036040 con 0x55f3b9d9f420 2024-06-10T01:17:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.355+0000 7f1eabfff640 1 -- 172.21.15.7:0/838401746 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f1ec404d030 con 0x55f3b9d9f420 2024-06-10T01:17:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.355+0000 7f1ed57fd740 10 client.18399 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:17:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.355+0000 7f1ed57fd740 10 client.18399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:17:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.355+0000 7f1ed57fd740 20 client.18399 populate_metadata read hostname 'smithi007' 2024-06-10T01:17:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.355+0000 7f1ed57fd740 1 --2- 172.21.15.7:0/838401746 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f3b9db5000 0x55f3b9db7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.355+0000 7f1ed57fd740 4 client.18399 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:17:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.355+0000 7f1ed57fd740 1 -- 172.21.15.7:0/838401746 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3b9d11f10 con 0x55f3b9db5000 2024-06-10T01:17:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.355+0000 7f1ed0893640 1 --2- 172.21.15.7:0/838401746 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f3b9db5000 0x55f3b9db7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.355+0000 7f1eabfff640 10 client.18399 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.355+0000 7f1ed0893640 1 --2- 172.21.15.7:0/838401746 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f3b9db5000 0x55f3b9db7420 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25de7c8a4c0b4808 server_cookie=10224a963615689 in_seq=0 out_seq=0 2024-06-10T01:17:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.356+0000 7f1eabfff640 1 -- 172.21.15.7:0/838401746 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f3b9d11f10 con 0x55f3b9db5000 2024-06-10T01:17:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.356+0000 7f1eabfff640 10 client.18399 handle_command_reply: tid=0 2024-06-10T01:17:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.356+0000 7f1ed57fd740 1 client.18399 shutdown 2024-06-10T01:17:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.356+0000 7f1ed57fd740 1 -- 172.21.15.7:0/838401746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3b9d9f420 msgr2=0x55f3b9da9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.356+0000 7f1ed57fd740 1 --2- 172.21.15.7:0/838401746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3b9d9f420 0x55f3b9da9480 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4003e40 tx=0x7f1ec4003fe0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.356+0000 7f1ed57fd740 1 -- 172.21.15.7:0/838401746 shutdown_connections 2024-06-10T01:17:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.356+0000 7f1ed57fd740 1 --2- 172.21.15.7:0/838401746 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f3b9db5000 0x55f3b9db7420 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.356+0000 7f1ed57fd740 1 --2- 172.21.15.7:0/838401746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3b9da4f80 0x55f3b9db0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.356+0000 7f1eabfff640 0 client.18399 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.356+0000 7f1ed57fd740 1 --2- 172.21.15.7:0/838401746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3b9d9fe10 0x55f3b9da99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.356+0000 7f1ed57fd740 1 --2- 172.21.15.7:0/838401746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3b9d9f420 0x55f3b9da9480 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.356+0000 7f1ed57fd740 1 -- 172.21.15.7:0/838401746 >> 172.21.15.7:0/838401746 conn(0x55f3b9d96680 msgr2=0x55f3b9d957e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.357+0000 7f1ed57fd740 1 -- 172.21.15.7:0/838401746 shutdown_connections 2024-06-10T01:17:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.357+0000 7f1ed57fd740 1 -- 172.21.15.7:0/838401746 wait complete. 2024-06-10T01:17:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.357+0000 7f1ed57fd740 20 client.18399 trim_cache size 0 max 16384 2024-06-10T01:17:31.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.373+0000 7f1ed57fd740 1 Processor -- start 2024-06-10T01:17:31.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.373+0000 7f1ed57fd740 1 -- start start 2024-06-10T01:17:31.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.373+0000 7f1ed57fd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3b9d9f420 0x55f3b9da9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.373+0000 7f1ed57fd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3b9d9fe10 0x55f3b9db09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.373+0000 7f1ed57fd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3b9da4f80 0x55f3b9da95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.373+0000 7f1ed57fd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc1aab10 con 0x55f3b9d9fe10 2024-06-10T01:17:31.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.373+0000 7f1ed57fd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc1a83e0 con 0x55f3b9d9f420 2024-06-10T01:17:31.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.373+0000 7f1ed57fd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc1a6230 con 0x55f3b9da4f80 2024-06-10T01:17:31.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.373+0000 7f1ed0893640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3b9d9fe10 0x55f3b9db09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.373+0000 7f1ed0893640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3b9d9fe10 0x55f3b9db09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42382/0 (socket says 172.21.15.7:42382) 2024-06-10T01:17:31.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.373+0000 7f1ed0893640 1 -- 172.21.15.7:0/4040654719 learned_addr learned my addr 172.21.15.7:0/4040654719 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.373+0000 7f1ed1094640 1 --2- 172.21.15.7:0/4040654719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3b9d9f420 0x55f3b9da9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.373+0000 7f1ed1094640 1 --2- 172.21.15.7:0/4040654719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3b9da4f80 0x55f3b9da95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.373+0000 7f1ed0893640 1 -- 172.21.15.7:0/4040654719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3b9da4f80 msgr2=0x55f3b9da95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.373+0000 7f1ed0893640 1 --2- 172.21.15.7:0/4040654719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3b9da4f80 0x55f3b9da95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.373+0000 7f1ed0893640 1 -- 172.21.15.7:0/4040654719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3b9d9f420 msgr2=0x55f3b9da9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.373+0000 7f1ed0893640 1 --2- 172.21.15.7:0/4040654719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3b9d9f420 0x55f3b9da9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.373+0000 7f1ed0893640 1 -- 172.21.15.7:0/4040654719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3b9da0ae0 con 0x55f3b9d9fe10 2024-06-10T01:17:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.374+0000 7f1ed0893640 1 --2- 172.21.15.7:0/4040654719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3b9d9fe10 0x55f3b9db09c0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8009fd0 tx=0x7f1ec8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b36c5cf8d0e91036 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.374+0000 7f1eb27fc640 1 -- 172.21.15.7:0/4040654719 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8010b00 con 0x55f3b9d9fe10 2024-06-10T01:17:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.374+0000 7f1eb27fc640 1 -- 172.21.15.7:0/4040654719 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ec8005500 con 0x55f3b9d9fe10 2024-06-10T01:17:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.374+0000 7f1eb27fc640 1 -- 172.21.15.7:0/4040654719 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8004e10 con 0x55f3b9d9fe10 2024-06-10T01:17:31.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.375+0000 7f1ed57fd740 1 -- 172.21.15.7:0/4040654719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3b9d9fe10 msgr2=0x55f3b9db09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:31.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.375+0000 7f1ed57fd740 1 --2- 172.21.15.7:0/4040654719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3b9d9fe10 0x55f3b9db09c0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8009fd0 tx=0x7f1ec8005f90 comp rx=0 tx=0).stop 2024-06-10T01:17:31.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.375+0000 7f1ed57fd740 1 -- 172.21.15.7:0/4040654719 shutdown_connections 2024-06-10T01:17:31.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.375+0000 7f1ed57fd740 1 --2- 172.21.15.7:0/4040654719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3b9da4f80 0x55f3b9da95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.375+0000 7f1ed57fd740 1 --2- 172.21.15.7:0/4040654719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3b9d9fe10 0x55f3b9db09c0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.375+0000 7f1ed57fd740 1 --2- 172.21.15.7:0/4040654719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3b9d9f420 0x55f3b9da9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.375+0000 7f1ed57fd740 1 -- 172.21.15.7:0/4040654719 >> 172.21.15.7:0/4040654719 conn(0x55f3b9d96680 msgr2=0x55f3b9da4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:31.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.375+0000 7f1ed57fd740 1 -- 172.21.15.7:0/4040654719 shutdown_connections 2024-06-10T01:17:31.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.375+0000 7f1ed57fd740 1 -- 172.21.15.7:0/4040654719 wait complete. 2024-06-10T01:17:31.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.375+0000 7f1ed57fd740 1 -- start start 2024-06-10T01:17:31.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.375+0000 7f1ed57fd740 1 Processor -- start 2024-06-10T01:17:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.376+0000 7f1ed57fd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3b9d9f420 0x55f3b9dadab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.376+0000 7f1ed57fd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3b9d9fe10 0x55f3b9dadff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.376+0000 7f1ed57fd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3b9da4f80 0x55f3b9daf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.376+0000 7f1ed57fd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc1a68a0 con 0x55f3b9da4f80 2024-06-10T01:17:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.376+0000 7f1ed57fd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc1a7d10 con 0x55f3b9d9fe10 2024-06-10T01:17:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.376+0000 7f1ed57fd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc1a6f70 con 0x55f3b9d9f420 2024-06-10T01:17:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.376+0000 7f1ed57fd740 10 client.0 _ll_register_callbacks cb 0x7f1ec8009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:17:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.376+0000 7f1ed1094640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3b9da4f80 0x55f3b9daf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.376+0000 7f1ed0893640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3b9d9fe10 0x55f3b9dadff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.376+0000 7f1ed0893640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3b9d9fe10 0x55f3b9dadff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:36682/0 (socket says 172.21.15.7:36682) 2024-06-10T01:17:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.376+0000 7f1ed1094640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3b9d9f420 0x55f3b9dadab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.376+0000 7f1ed0893640 1 -- 172.21.15.7:0/3879332920 learned_addr learned my addr 172.21.15.7:0/3879332920 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.376+0000 7f1ed0893640 1 -- 172.21.15.7:0/3879332920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3b9d9f420 msgr2=0x55f3b9dadab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.376+0000 7f1ed0893640 1 --2- 172.21.15.7:0/3879332920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3b9d9f420 0x55f3b9dadab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.376+0000 7f1ed0893640 1 -- 172.21.15.7:0/3879332920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3b9da4f80 msgr2=0x55f3b9daf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.376+0000 7f1ed0893640 1 --2- 172.21.15.7:0/3879332920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3b9da4f80 0x55f3b9daf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.376+0000 7f1ed0893640 1 -- 172.21.15.7:0/3879332920 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3b9daf5f0 con 0x55f3b9d9fe10 2024-06-10T01:17:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.376+0000 7f1ed1094640 1 --2- 172.21.15.7:0/3879332920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3b9d9f420 0x55f3b9dadab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:17:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.376+0000 7f1ed0893640 1 --2- 172.21.15.7:0/3879332920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3b9d9fe10 0x55f3b9dadff0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1ec80126b0 tx=0x7f1ec80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ae8045f414ee1d4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.376+0000 7f1eabfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:17:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.376+0000 7f1eabfff640 1 -- 172.21.15.7:0/3879332920 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8004b90 con 0x55f3b9d9fe10 2024-06-10T01:17:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.377+0000 7f1ed57fd740 1 -- 172.21.15.7:0/3879332920 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ec8005500 con 0x55f3b9d9fe10 2024-06-10T01:17:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.377+0000 7f1eabfff640 1 -- 172.21.15.7:0/3879332920 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f1ec8004d30 con 0x55f3b9d9fe10 2024-06-10T01:17:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.377+0000 7f1eabfff640 1 -- 172.21.15.7:0/3879332920 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f1ec803a050 con 0x55f3b9d9fe10 2024-06-10T01:17:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.377+0000 7f1ed57fd740 10 client.27740 fetch_fsmap learned FSMap version 40 2024-06-10T01:17:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.377+0000 7f1ed57fd740 1 -- 172.21.15.7:0/3879332920 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f3b9db0980 con 0x55f3b9d9fe10 2024-06-10T01:17:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.377+0000 7f1eabfff640 1 -- 172.21.15.7:0/3879332920 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ec8005500 con 0x55f3b9d9fe10 2024-06-10T01:17:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.377+0000 7f1eabfff640 1 -- 172.21.15.7:0/3879332920 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f1ec8031520 con 0x55f3b9d9fe10 2024-06-10T01:17:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.377+0000 7f1ed57fd740 10 client.27740 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:17:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.377+0000 7f1ed57fd740 10 client.27740 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:17:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.377+0000 7f1ed57fd740 20 client.27740 populate_metadata read hostname 'smithi007' 2024-06-10T01:17:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.377+0000 7f1ed57fd740 1 --2- 172.21.15.7:0/3879332920 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f3b9db5000 0x55f3b9e22050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.377+0000 7f1ed57fd740 4 client.27740 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:17:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.377+0000 7f1ed57fd740 1 -- 172.21.15.7:0/3879332920 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f3b9d0bc20 con 0x55f3b9db5000 2024-06-10T01:17:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.377+0000 7f1ed1094640 1 --2- 172.21.15.7:0/3879332920 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f3b9db5000 0x55f3b9e22050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.378+0000 7f1eabfff640 10 client.27740 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.378+0000 7f1ed1094640 1 --2- 172.21.15.7:0/3879332920 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f3b9db5000 0x55f3b9e22050 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd5a7aae88f1915c server_cookie=d1608392e93da8c0 in_seq=0 out_seq=0 2024-06-10T01:17:31.452 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:30 smithi175 ceph-mon[36636]: pgmap v777: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 2.7 MiB/s wr, 317 op/s 2024-06-10T01:17:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.466+0000 7f1eabfff640 1 -- 172.21.15.7:0/3879332920 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f3b9d0bc20 con 0x55f3b9db5000 2024-06-10T01:17:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.466+0000 7f1eabfff640 10 client.27740 handle_command_reply: tid=0 2024-06-10T01:17:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.466+0000 7f1ed57fd740 1 client.27740 shutdown 2024-06-10T01:17:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.466+0000 7f1ed57fd740 1 -- 172.21.15.7:0/3879332920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3b9d9fe10 msgr2=0x55f3b9dadff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.466+0000 7f1ed57fd740 1 --2- 172.21.15.7:0/3879332920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3b9d9fe10 0x55f3b9dadff0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1ec80126b0 tx=0x7f1ec80046c0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.466+0000 7f1ed57fd740 1 -- 172.21.15.7:0/3879332920 shutdown_connections 2024-06-10T01:17:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.466+0000 7f1ed57fd740 1 --2- 172.21.15.7:0/3879332920 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f3b9db5000 0x55f3b9e22050 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.466+0000 7f1ed57fd740 1 --2- 172.21.15.7:0/3879332920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3b9da4f80 0x55f3b9daf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.466+0000 7f1ed57fd740 1 --2- 172.21.15.7:0/3879332920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3b9d9fe10 0x55f3b9dadff0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.466+0000 7f1eabfff640 0 client.27740 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.466+0000 7f1ed57fd740 1 --2- 172.21.15.7:0/3879332920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3b9d9f420 0x55f3b9dadab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.466+0000 7f1ed57fd740 1 -- 172.21.15.7:0/3879332920 >> 172.21.15.7:0/3879332920 conn(0x55f3b9d96680 msgr2=0x55f3b9da9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.466+0000 7f1ed57fd740 1 -- 172.21.15.7:0/3879332920 shutdown_connections 2024-06-10T01:17:31.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.466+0000 7f1ed57fd740 1 -- 172.21.15.7:0/3879332920 wait complete. 2024-06-10T01:17:31.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.467+0000 7f1ed57fd740 20 client.27740 trim_cache size 0 max 16384 2024-06-10T01:17:31.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:17:31.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:17:31.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:17:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.467+0000 7f1ed2af7640 1 -- 172.21.15.7:0/1173050266 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1eb40a2c00 msgr2=0x7f1eb40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.467+0000 7f1ed2af7640 1 --2- 172.21.15.7:0/1173050266 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1eb40a2c00 0x7f1eb40a50c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc097c00 tx=0x7f1ebc0c63c0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.467+0000 7f1ed2af7640 1 -- 172.21.15.7:0/1173050266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ecc101af0 msgr2=0x7f1ecc198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.467+0000 7f1ed2af7640 1 --2- 172.21.15.7:0/1173050266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ecc101af0 0x7f1ecc198270 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00e9f0 tx=0x7f1ebc00eeb0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.467+0000 7f1ed2af7640 1 -- 172.21.15.7:0/1173050266 shutdown_connections 2024-06-10T01:17:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.467+0000 7f1ed2af7640 1 --2- 172.21.15.7:0/1173050266 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1eb40a2c00 0x7f1eb40a50c0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.467+0000 7f1ed2af7640 1 --2- 172.21.15.7:0/1173050266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ecc1024b0 0x7f1ecc19f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.467+0000 7f1ed2af7640 1 --2- 172.21.15.7:0/1173050266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ecc101af0 0x7f1ecc198270 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.467+0000 7f1ed2af7640 1 --2- 172.21.15.7:0/1173050266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ecc100900 0x7f1ecc197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.467+0000 7f1ed2af7640 1 -- 172.21.15.7:0/1173050266 >> 172.21.15.7:0/1173050266 conn(0x7f1ecc0fbe70 msgr2=0x7f1ecc0fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.467+0000 7f1ed2af7640 1 -- 172.21.15.7:0/1173050266 shutdown_connections 2024-06-10T01:17:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:31.467+0000 7f1ed2af7640 1 -- 172.21.15.7:0/1173050266 wait complete. 2024-06-10T01:17:31.752 INFO:tasks.workunit.client.0.smithi007.stdout: 1 858132 7.52 MB/sec execute 492 sec latency 81.975 ms 2024-06-10T01:17:32.135 INFO:tasks.cephfs.filesystem:scrub status for tag:2fca286c-35aa-4d1f-abfd-650b1d9ca6a7 - {'path': '/', 'tag': '2fca286c-35aa-4d1f-abfd-650b1d9ca6a7', 'options': 'recursive,force'} 2024-06-10T01:17:32.752 INFO:tasks.workunit.client.0.smithi007.stdout: 1 859869 7.53 MB/sec execute 493 sec latency 11.318 ms 2024-06-10T01:17:33.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:33 smithi007 ceph-mon[32398]: pgmap v778: 129 pgs: 129 active+clean; 143 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 2.1 MiB/s wr, 259 op/s 2024-06-10T01:17:33.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:33 smithi078 ceph-mon[36683]: pgmap v778: 129 pgs: 129 active+clean; 143 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 2.1 MiB/s wr, 259 op/s 2024-06-10T01:17:33.452 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:33 smithi175 ceph-mon[36636]: pgmap v778: 129 pgs: 129 active+clean; 143 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 2.1 MiB/s wr, 259 op/s 2024-06-10T01:17:33.752 INFO:tasks.workunit.client.0.smithi007.stdout: 1 862192 7.54 MB/sec execute 494 sec latency 15.770 ms 2024-06-10T01:17:34.752 INFO:tasks.workunit.client.0.smithi007.stdout: 1 863688 7.54 MB/sec execute 495 sec latency 10.733 ms 2024-06-10T01:17:35.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:35 smithi007 ceph-mon[32398]: pgmap v779: 129 pgs: 129 active+clean; 145 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 2.0 MiB/s wr, 273 op/s 2024-06-10T01:17:35.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:35 smithi078 ceph-mon[36683]: pgmap v779: 129 pgs: 129 active+clean; 145 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 2.0 MiB/s wr, 273 op/s 2024-06-10T01:17:35.452 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:35 smithi175 ceph-mon[36636]: pgmap v779: 129 pgs: 129 active+clean; 145 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 2.0 MiB/s wr, 273 op/s 2024-06-10T01:17:35.752 INFO:tasks.workunit.client.0.smithi007.stdout: 1 866107 7.55 MB/sec execute 496 sec latency 67.427 ms 2024-06-10T01:17:36.752 INFO:tasks.workunit.client.0.smithi007.stdout: 1 867272 7.55 MB/sec execute 497 sec latency 27.598 ms 2024-06-10T01:17:37.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:37 smithi175 ceph-mon[36636]: pgmap v780: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 1.7 MiB/s wr, 278 op/s 2024-06-10T01:17:37.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:37 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:17:37.752 INFO:tasks.workunit.client.0.smithi007.stdout: 1 869800 7.56 MB/sec execute 498 sec latency 22.848 ms 2024-06-10T01:17:37.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:37 smithi007 ceph-mon[32398]: pgmap v780: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 1.7 MiB/s wr, 278 op/s 2024-06-10T01:17:37.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:37 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:17:37.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:37 smithi078 ceph-mon[36683]: pgmap v780: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 1.7 MiB/s wr, 278 op/s 2024-06-10T01:17:37.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:37 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:17:38.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:38 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:17:38.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:38 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:17:38.752 INFO:tasks.workunit.client.0.smithi007.stdout: 1 871086 7.56 MB/sec execute 499 sec latency 16.271 ms 2024-06-10T01:17:38.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:38 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:17:38.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:38 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:17:38.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:38 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:17:38.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:38 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:17:39.094 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.2 2024-06-10T01:17:39.095 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:17:39.385 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:17:39.648 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:39 smithi007 ceph-mon[32398]: pgmap v781: 129 pgs: 129 active+clean; 145 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 1.4 MiB/s wr, 262 op/s 2024-06-10T01:17:39.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:39 smithi175 ceph-mon[36636]: pgmap v781: 129 pgs: 129 active+clean; 145 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 1.4 MiB/s wr, 262 op/s 2024-06-10T01:17:39.752 INFO:tasks.workunit.client.0.smithi007.stdout: 1 873625 7.58 MB/sec execute 500 sec latency 10.416 ms 2024-06-10T01:17:39.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:39 smithi078 ceph-mon[36683]: pgmap v781: 129 pgs: 129 active+clean; 145 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 1.4 MiB/s wr, 262 op/s 2024-06-10T01:17:40.752 INFO:tasks.workunit.client.0.smithi007.stdout: 1 875363 7.57 MB/sec execute 501 sec latency 18.594 ms 2024-06-10T01:17:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.097+0000 7fee8760d700 1 -- 172.21.15.7:0/798513251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee80107400 msgr2=0x7fee801077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.097+0000 7fee8760d700 1 --2- 172.21.15.7:0/798513251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee80107400 0x7fee801077e0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fee7c009990 tx=0x7fee7c009ca0 comp rx=0 tx=0).stop 2024-06-10T01:17:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.097+0000 7fee8760d700 1 -- 172.21.15.7:0/798513251 shutdown_connections 2024-06-10T01:17:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.097+0000 7fee8760d700 1 --2- 172.21.15.7:0/798513251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee80101d50 0x7fee8010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.097+0000 7fee8760d700 1 --2- 172.21.15.7:0/798513251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee80107400 0x7fee801077e0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.097+0000 7fee8760d700 1 --2- 172.21.15.7:0/798513251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee80101390 0x7fee80101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.097+0000 7fee8760d700 1 -- 172.21.15.7:0/798513251 >> 172.21.15.7:0/798513251 conn(0x7fee80078340 msgr2=0x7fee80078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.098+0000 7fee8760d700 1 -- 172.21.15.7:0/798513251 shutdown_connections 2024-06-10T01:17:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.098+0000 7fee8760d700 1 -- 172.21.15.7:0/798513251 wait complete. 2024-06-10T01:17:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.098+0000 7fee8760d700 1 Processor -- start 2024-06-10T01:17:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.098+0000 7fee8760d700 1 -- start start 2024-06-10T01:17:41.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.098+0000 7fee8760d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee80101390 0x7fee80103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:41.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.098+0000 7fee8760d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee80101d50 0x7fee801035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:41.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.098+0000 7fee8760d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee80107400 0x7fee80103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:41.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.099+0000 7fee8760d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fee801116a0 con 0x7fee80101d50 2024-06-10T01:17:41.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.099+0000 7fee84ba8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee80101d50 0x7fee801035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:41.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.099+0000 7fee853a9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee80101390 0x7fee80103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:41.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.099+0000 7fee853a9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee80101390 0x7fee80103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49464/0 (socket says 172.21.15.7:49464) 2024-06-10T01:17:41.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.099+0000 7fee84ba8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee80101d50 0x7fee801035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45346/0 (socket says 172.21.15.7:45346) 2024-06-10T01:17:41.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.099+0000 7fee853a9700 1 -- 172.21.15.7:0/3694496359 learned_addr learned my addr 172.21.15.7:0/3694496359 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:41.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.099+0000 7fee8760d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fee80111520 con 0x7fee80107400 2024-06-10T01:17:41.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.099+0000 7fee85baa700 1 --2- 172.21.15.7:0/3694496359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee80107400 0x7fee80103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:41.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.099+0000 7fee8760d700 1 -- 172.21.15.7:0/3694496359 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fee80111820 con 0x7fee80101390 2024-06-10T01:17:41.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.099+0000 7fee853a9700 1 -- 172.21.15.7:0/3694496359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee80107400 msgr2=0x7fee80103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:41.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.099+0000 7fee853a9700 1 --2- 172.21.15.7:0/3694496359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee80107400 0x7fee80103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:41.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.099+0000 7fee853a9700 1 -- 172.21.15.7:0/3694496359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee80101d50 msgr2=0x7fee801035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:41.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.099+0000 7fee853a9700 1 --2- 172.21.15.7:0/3694496359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee80101d50 0x7fee801035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:41.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.099+0000 7fee853a9700 1 -- 172.21.15.7:0/3694496359 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee801a7100 con 0x7fee80101390 2024-06-10T01:17:41.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.099+0000 7fee84ba8700 1 --2- 172.21.15.7:0/3694496359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee80101d50 0x7fee801035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:17:41.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.099+0000 7fee853a9700 1 --2- 172.21.15.7:0/3694496359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee80101390 0x7fee80103070 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fee7c000c00 tx=0x7fee7c00bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:41.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.100+0000 7fee767fc700 1 -- 172.21.15.7:0/3694496359 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c027070 con 0x7fee80101390 2024-06-10T01:17:41.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.100+0000 7fee8760d700 1 -- 172.21.15.7:0/3694496359 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee801a7390 con 0x7fee80101390 2024-06-10T01:17:41.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.100+0000 7fee767fc700 1 -- 172.21.15.7:0/3694496359 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee7c003d70 con 0x7fee80101390 2024-06-10T01:17:41.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.100+0000 7fee767fc700 1 -- 172.21.15.7:0/3694496359 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c00f460 con 0x7fee80101390 2024-06-10T01:17:41.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.100+0000 7fee8760d700 1 -- 172.21.15.7:0/3694496359 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee801a7840 con 0x7fee80101390 2024-06-10T01:17:41.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.101+0000 7fee8760d700 1 -- 172.21.15.7:0/3694496359 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee80069180 con 0x7fee80101390 2024-06-10T01:17:41.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.102+0000 7fee767fc700 1 -- 172.21.15.7:0/3694496359 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fee7c02a070 con 0x7fee80101390 2024-06-10T01:17:41.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.103+0000 7fee767fc700 1 --2- 172.21.15.7:0/3694496359 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fee6c0a2ca0 0x7fee6c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:41.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.103+0000 7fee84ba8700 1 --2- 172.21.15.7:0/3694496359 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fee6c0a2ca0 0x7fee6c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:41.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.103+0000 7fee767fc700 1 -- 172.21.15.7:0/3694496359 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fee7c0d0140 con 0x7fee80101390 2024-06-10T01:17:41.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.103+0000 7fee84ba8700 1 --2- 172.21.15.7:0/3694496359 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fee6c0a2ca0 0x7fee6c0a5160 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fee800ffb70 tx=0x7fee70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:41.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.108+0000 7fee767fc700 1 -- 172.21.15.7:0/3694496359 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fee7c013070 con 0x7fee80101390 2024-06-10T01:17:41.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.405+0000 7fee8760d700 1 -- 172.21.15.7:0/3694496359 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee801a7ad0 con 0x7fee80101390 2024-06-10T01:17:41.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.406+0000 7fee767fc700 1 -- 172.21.15.7:0/3694496359 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11855 (secure 0 0 0) 0x7fee7c09cd50 con 0x7fee80101390 2024-06-10T01:17:41.400 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:17:41.400 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:16:56.409130+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24451,24434,24436]},"id":1}]} 2024-06-10T01:17:41.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.409+0000 7fee8760d700 1 -- 172.21.15.7:0/3694496359 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fee6c0a2ca0 msgr2=0x7fee6c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:41.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.409+0000 7fee8760d700 1 --2- 172.21.15.7:0/3694496359 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fee6c0a2ca0 0x7fee6c0a5160 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fee800ffb70 tx=0x7fee70009260 comp rx=0 tx=0).stop 2024-06-10T01:17:41.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.409+0000 7fee8760d700 1 -- 172.21.15.7:0/3694496359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee80101390 msgr2=0x7fee80103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:41.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.409+0000 7fee8760d700 1 --2- 172.21.15.7:0/3694496359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee80101390 0x7fee80103070 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fee7c000c00 tx=0x7fee7c00bab0 comp rx=0 tx=0).stop 2024-06-10T01:17:41.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.409+0000 7fee8760d700 1 -- 172.21.15.7:0/3694496359 shutdown_connections 2024-06-10T01:17:41.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.409+0000 7fee8760d700 1 --2- 172.21.15.7:0/3694496359 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fee6c0a2ca0 0x7fee6c0a5160 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:41.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.409+0000 7fee8760d700 1 --2- 172.21.15.7:0/3694496359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee80107400 0x7fee80103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:41.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.409+0000 7fee8760d700 1 --2- 172.21.15.7:0/3694496359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee80101390 0x7fee80103070 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:41.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.409+0000 7fee8760d700 1 --2- 172.21.15.7:0/3694496359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee80101d50 0x7fee801035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:41.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.409+0000 7fee8760d700 1 -- 172.21.15.7:0/3694496359 >> 172.21.15.7:0/3694496359 conn(0x7fee80078340 msgr2=0x7fee801005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:41.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.409+0000 7fee8760d700 1 -- 172.21.15.7:0/3694496359 shutdown_connections 2024-06-10T01:17:41.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:41.409+0000 7fee8760d700 1 -- 172.21.15.7:0/3694496359 wait complete. 2024-06-10T01:17:41.404 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 40 2024-06-10T01:17:41.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:41 smithi175 ceph-mon[36636]: pgmap v782: 129 pgs: 129 active+clean; 146 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 1.3 MiB/s wr, 265 op/s 2024-06-10T01:17:41.752 INFO:tasks.workunit.client.0.smithi007.stdout: 1 877255 7.58 MB/sec execute 502 sec latency 57.809 ms 2024-06-10T01:17:41.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:41 smithi007 ceph-mon[32398]: pgmap v782: 129 pgs: 129 active+clean; 146 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 1.3 MiB/s wr, 265 op/s 2024-06-10T01:17:41.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:41 smithi078 ceph-mon[36683]: pgmap v782: 129 pgs: 129 active+clean; 146 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 1.3 MiB/s wr, 265 op/s 2024-06-10T01:17:42.111 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db / --timeout 90 --expiration 147.2 --await-for 45.0' 2024-06-10T01:17:42.111 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db / --timeout 90 --expiration 147.2 --await-for 45.0 2024-06-10T01:17:42.440 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:17:42.694 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:42 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3694496359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:17:42.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:42 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3694496359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:17:42.752 INFO:tasks.workunit.client.0.smithi007.stdout: 1 878788 7.58 MB/sec execute 503 sec latency 52.109 ms 2024-06-10T01:17:42.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:42 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3694496359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:17:43.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:43 smithi175 ceph-mon[36636]: pgmap v783: 129 pgs: 129 active+clean; 148 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 1.1 MiB/s wr, 247 op/s 2024-06-10T01:17:43.728 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:43 smithi007 ceph-mon[32398]: pgmap v783: 129 pgs: 129 active+clean; 148 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 1.1 MiB/s wr, 247 op/s 2024-06-10T01:17:43.752 INFO:tasks.workunit.client.0.smithi007.stdout: 1 880958 7.59 MB/sec execute 504 sec latency 11.021 ms 2024-06-10T01:17:43.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:43 smithi078 ceph-mon[36683]: pgmap v783: 129 pgs: 129 active+clean; 148 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 1.1 MiB/s wr, 247 op/s 2024-06-10T01:17:44.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.136+0000 7f339d2f0700 1 -- 172.21.15.7:0/178916932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3398105090 msgr2=0x7f33981115a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:44.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.136+0000 7f339d2f0700 1 --2- 172.21.15.7:0/178916932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3398105090 0x7f33981115a0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f339000b0a0 tx=0x7f339000b3b0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.136+0000 7f339d2f0700 1 -- 172.21.15.7:0/178916932 shutdown_connections 2024-06-10T01:17:44.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.136+0000 7f339d2f0700 1 --2- 172.21.15.7:0/178916932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3398105090 0x7f33981115a0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.136+0000 7f339d2f0700 1 --2- 172.21.15.7:0/178916932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f339810a6d0 0x7f339810aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.136+0000 7f339d2f0700 1 --2- 172.21.15.7:0/178916932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33981046d0 0x7f3398104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.136+0000 7f339d2f0700 1 -- 172.21.15.7:0/178916932 >> 172.21.15.7:0/178916932 conn(0x7f33981001c0 msgr2=0x7f33981025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:44.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.136+0000 7f339d2f0700 1 -- 172.21.15.7:0/178916932 shutdown_connections 2024-06-10T01:17:44.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.136+0000 7f339d2f0700 1 -- 172.21.15.7:0/178916932 wait complete. 2024-06-10T01:17:44.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.137+0000 7f339d2f0700 1 Processor -- start 2024-06-10T01:17:44.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.137+0000 7f339d2f0700 1 -- start start 2024-06-10T01:17:44.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.138+0000 7f339d2f0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33981046d0 0x7f339819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.138+0000 7f339d2f0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3398105090 0x7f339819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.138+0000 7f339d2f0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f339810a6d0 0x7f339819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.138+0000 7f339d2f0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3398114a60 con 0x7f3398105090 2024-06-10T01:17:44.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.138+0000 7f339d2f0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f33981148e0 con 0x7f339810a6d0 2024-06-10T01:17:44.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.138+0000 7f339d2f0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3398114be0 con 0x7f33981046d0 2024-06-10T01:17:44.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.138+0000 7f33967fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3398105090 0x7f339819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.138+0000 7f3396ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33981046d0 0x7f339819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.138+0000 7f3396ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33981046d0 0x7f339819adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:51908/0 (socket says 172.21.15.7:51908) 2024-06-10T01:17:44.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.138+0000 7f33967fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3398105090 0x7f339819b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50880/0 (socket says 172.21.15.7:50880) 2024-06-10T01:17:44.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.138+0000 7f3396ffd700 1 -- 172.21.15.7:0/2779706130 learned_addr learned my addr 172.21.15.7:0/2779706130 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:44.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.138+0000 7f33977fe700 1 --2- 172.21.15.7:0/2779706130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f339810a6d0 0x7f339819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.138+0000 7f3396ffd700 1 -- 172.21.15.7:0/2779706130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f339810a6d0 msgr2=0x7f339819f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:44.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.138+0000 7f3396ffd700 1 --2- 172.21.15.7:0/2779706130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f339810a6d0 0x7f339819f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.138+0000 7f3396ffd700 1 -- 172.21.15.7:0/2779706130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3398105090 msgr2=0x7f339819b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:44.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.138+0000 7f3396ffd700 1 --2- 172.21.15.7:0/2779706130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3398105090 0x7f339819b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.138+0000 7f3396ffd700 1 -- 172.21.15.7:0/2779706130 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f339819fdb0 con 0x7f33981046d0 2024-06-10T01:17:44.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.138+0000 7f33967fc700 1 --2- 172.21.15.7:0/2779706130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3398105090 0x7f339819b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:17:44.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.138+0000 7f3396ffd700 1 --2- 172.21.15.7:0/2779706130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33981046d0 0x7f339819adf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f338800e9c0 tx=0x7f338800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:44.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.139+0000 7f337ffff700 1 -- 172.21.15.7:0/2779706130 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338800cbd0 con 0x7f33981046d0 2024-06-10T01:17:44.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.139+0000 7f337ffff700 1 -- 172.21.15.7:0/2779706130 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f338800ee60 con 0x7f33981046d0 2024-06-10T01:17:44.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.139+0000 7f339d2f0700 1 -- 172.21.15.7:0/2779706130 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33981a00a0 con 0x7f33981046d0 2024-06-10T01:17:44.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.139+0000 7f337ffff700 1 -- 172.21.15.7:0/2779706130 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388005630 con 0x7f33981046d0 2024-06-10T01:17:44.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.139+0000 7f339d2f0700 1 -- 172.21.15.7:0/2779706130 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33981a7920 con 0x7f33981046d0 2024-06-10T01:17:44.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.142+0000 7f337ffff700 1 -- 172.21.15.7:0/2779706130 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f3388004750 con 0x7f33981046d0 2024-06-10T01:17:44.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.142+0000 7f337ffff700 1 --2- 172.21.15.7:0/2779706130 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f33800a2cc0 0x7f33800a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.143+0000 7f33967fc700 1 --2- 172.21.15.7:0/2779706130 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f33800a2cc0 0x7f33800a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.143+0000 7f337ffff700 1 -- 172.21.15.7:0/2779706130 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f33880cf5c0 con 0x7f33981046d0 2024-06-10T01:17:44.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.143+0000 7f33a7b73b80 1 Processor -- start 2024-06-10T01:17:44.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.143+0000 7f33967fc700 1 --2- 172.21.15.7:0/2779706130 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f33800a2cc0 0x7f33800a5180 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3398109b60 tx=0x7f338c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:44.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.143+0000 7f33a7b73b80 1 -- start start 2024-06-10T01:17:44.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.143+0000 7f33a7b73b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1f148ccd0 0x55c1f148d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.144+0000 7f33a7b73b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1f148d5f0 0x55c1f1486f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.144+0000 7f33a7b73b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1f1487460 0x55c1f14878a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.144+0000 7f33a7b73b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c1f13d89a0 con 0x55c1f148d5f0 2024-06-10T01:17:44.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.144+0000 7f33a7b73b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c1f13808b0 con 0x55c1f148ccd0 2024-06-10T01:17:44.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.144+0000 7f33a7b73b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c1f13d4960 con 0x55c1f1487460 2024-06-10T01:17:44.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.144+0000 7f3396ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1f148ccd0 0x55c1f148d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.144+0000 7f33967fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1f148d5f0 0x55c1f1486f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.144+0000 7f3396ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1f148ccd0 0x55c1f148d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:54124/0 (socket says 172.21.15.7:54124) 2024-06-10T01:17:44.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.144+0000 7f3396ffd700 1 -- 172.21.15.7:0/708389314 learned_addr learned my addr 172.21.15.7:0/708389314 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:44.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.144+0000 7f33977fe700 1 --2- 172.21.15.7:0/708389314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1f1487460 0x55c1f14878a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.144+0000 7f3396ffd700 1 -- 172.21.15.7:0/708389314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1f1487460 msgr2=0x55c1f14878a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:44.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.144+0000 7f3396ffd700 1 --2- 172.21.15.7:0/708389314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1f1487460 0x55c1f14878a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.144+0000 7f3396ffd700 1 -- 172.21.15.7:0/708389314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1f148d5f0 msgr2=0x55c1f1486f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:44.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.144+0000 7f3396ffd700 1 --2- 172.21.15.7:0/708389314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1f148d5f0 0x55c1f1486f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.144+0000 7f3396ffd700 1 -- 172.21.15.7:0/708389314 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1f13770e0 con 0x55c1f148ccd0 2024-06-10T01:17:44.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.144+0000 7f33967fc700 1 --2- 172.21.15.7:0/708389314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1f148d5f0 0x55c1f1486f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:17:44.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.144+0000 7f3396ffd700 1 --2- 172.21.15.7:0/708389314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1f148ccd0 0x55c1f148d0b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3388097f00 tx=0x7f338809b4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a07f79bba03da74 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:44.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.144+0000 7f337d7fa700 1 -- 172.21.15.7:0/708389314 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338809fe50 con 0x55c1f148ccd0 2024-06-10T01:17:44.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.145+0000 7f337d7fa700 1 -- 172.21.15.7:0/708389314 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33880b4070 con 0x55c1f148ccd0 2024-06-10T01:17:44.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.145+0000 7f337d7fa700 1 -- 172.21.15.7:0/708389314 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338809bd00 con 0x55c1f148ccd0 2024-06-10T01:17:44.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.145+0000 7f33a7b73b80 1 -- 172.21.15.7:0/708389314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1f148ccd0 msgr2=0x55c1f148d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:44.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.145+0000 7f33a7b73b80 1 --2- 172.21.15.7:0/708389314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1f148ccd0 0x55c1f148d0b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3388097f00 tx=0x7f338809b4d0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.145+0000 7f33a7b73b80 1 -- 172.21.15.7:0/708389314 shutdown_connections 2024-06-10T01:17:44.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.145+0000 7f33a7b73b80 1 --2- 172.21.15.7:0/708389314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1f1487460 0x55c1f14878a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.145+0000 7f33a7b73b80 1 --2- 172.21.15.7:0/708389314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1f148ccd0 0x55c1f148d0b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.145+0000 7f33a7b73b80 1 --2- 172.21.15.7:0/708389314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1f148d5f0 0x55c1f1486f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.145+0000 7f33a7b73b80 1 -- 172.21.15.7:0/708389314 >> 172.21.15.7:0/708389314 conn(0x55c1f13b1e50 msgr2=0x55c1f14821b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:44.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.146+0000 7f33a7b73b80 1 -- 172.21.15.7:0/708389314 shutdown_connections 2024-06-10T01:17:44.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.146+0000 7f33a7b73b80 1 -- 172.21.15.7:0/708389314 wait complete. 2024-06-10T01:17:44.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.146+0000 7f33a7b73b80 1 -- start start 2024-06-10T01:17:44.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.146+0000 7f33a7b73b80 1 Processor -- start 2024-06-10T01:17:44.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.146+0000 7f33a7b73b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1f1487460 0x55c1f1493390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.146+0000 7f33a7b73b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1f148ccd0 0x55c1f14906e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.146+0000 7f33a7b73b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1f148d5f0 0x55c1f1490c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.146+0000 7f33a7b73b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c1f1494000 con 0x55c1f148ccd0 2024-06-10T01:17:44.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.146+0000 7f33a7b73b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c1f1378d80 con 0x55c1f1487460 2024-06-10T01:17:44.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.146+0000 7f33a7b73b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c1f1494180 con 0x55c1f148d5f0 2024-06-10T01:17:44.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.146+0000 7f33a7b73b80 10 client.0 _ll_register_callbacks cb 0x55c1f1372ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:17:44.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.147+0000 7f3396ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1f1487460 0x55c1f1493390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.147+0000 7f33967fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1f148ccd0 0x55c1f14906e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.147+0000 7f3396ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1f1487460 0x55c1f1493390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:54130/0 (socket says 172.21.15.7:54130) 2024-06-10T01:17:44.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.147+0000 7f3396ffd700 1 -- 172.21.15.7:0/2193910912 learned_addr learned my addr 172.21.15.7:0/2193910912 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:44.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.147+0000 7f33977fe700 1 --2- 172.21.15.7:0/2193910912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1f148d5f0 0x55c1f1490c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.147+0000 7f3396ffd700 1 -- 172.21.15.7:0/2193910912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1f148d5f0 msgr2=0x55c1f1490c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:44.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.147+0000 7f3396ffd700 1 --2- 172.21.15.7:0/2193910912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1f148d5f0 0x55c1f1490c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.147+0000 7f3396ffd700 1 -- 172.21.15.7:0/2193910912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1f148ccd0 msgr2=0x55c1f14906e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:44.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.147+0000 7f3396ffd700 1 --2- 172.21.15.7:0/2193910912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1f148ccd0 0x55c1f14906e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.147+0000 7f3396ffd700 1 -- 172.21.15.7:0/2193910912 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1f1327a60 con 0x55c1f1487460 2024-06-10T01:17:44.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.147+0000 7f33967fc700 1 --2- 172.21.15.7:0/2193910912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1f148ccd0 0x55c1f14906e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:17:44.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.147+0000 7f3396ffd700 1 --2- 172.21.15.7:0/2193910912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1f1487460 0x55c1f1493390 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3388086040 tx=0x7f3388005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c016b3132456a51 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:44.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.147+0000 7f337aff5700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:17:44.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.147+0000 7f337aff5700 1 -- 172.21.15.7:0/2193910912 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33880ba030 con 0x55c1f1487460 2024-06-10T01:17:44.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.147+0000 7f337aff5700 1 -- 172.21.15.7:0/2193910912 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f338809bd00 con 0x55c1f1487460 2024-06-10T01:17:44.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.147+0000 7f33a7b73b80 1 -- 172.21.15.7:0/2193910912 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33880b4070 con 0x55c1f1487460 2024-06-10T01:17:44.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.147+0000 7f337aff5700 1 -- 172.21.15.7:0/2193910912 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f3388099b40 con 0x55c1f1487460 2024-06-10T01:17:44.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.147+0000 7f33a7b73b80 10 client.27758 fetch_fsmap learned FSMap version 40 2024-06-10T01:17:44.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.147+0000 7f33a7b73b80 1 -- 172.21.15.7:0/2193910912 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55c1f149e900 con 0x55c1f1487460 2024-06-10T01:17:44.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.148+0000 7f337aff5700 1 -- 172.21.15.7:0/2193910912 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33880b4070 con 0x55c1f1487460 2024-06-10T01:17:44.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.148+0000 7f337aff5700 1 -- 172.21.15.7:0/2193910912 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f33880ba1d0 con 0x55c1f1487460 2024-06-10T01:17:44.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.148+0000 7f33a7b73b80 10 client.27758 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:17:44.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.148+0000 7f33a7b73b80 10 client.27758 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:17:44.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.148+0000 7f33a7b73b80 20 client.27758 populate_metadata read hostname 'smithi007' 2024-06-10T01:17:44.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.148+0000 7f33a7b73b80 1 --2- 172.21.15.7:0/2193910912 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c1f14a0dd0 0x55c1f14a31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.148+0000 7f33a7b73b80 4 client.27758 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:17:44.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.148+0000 7f33a7b73b80 1 -- 172.21.15.7:0/2193910912 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1f13e5ea0 con 0x55c1f14a0dd0 2024-06-10T01:17:44.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.148+0000 7f33967fc700 1 --2- 172.21.15.7:0/2193910912 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c1f14a0dd0 0x55c1f14a31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.149+0000 7f33967fc700 1 --2- 172.21.15.7:0/2193910912 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c1f14a0dd0 0x55c1f14a31c0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a560bfe0504cadd server_cookie=43168141e8a72cd8 in_seq=0 out_seq=0 2024-06-10T01:17:44.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.149+0000 7f337aff5700 10 client.27758 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:44.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.149+0000 7f337aff5700 1 -- 172.21.15.7:0/2193910912 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c1f13e5ea0 con 0x55c1f14a0dd0 2024-06-10T01:17:44.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.149+0000 7f337aff5700 10 client.27758 handle_command_reply: tid=0 2024-06-10T01:17:44.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.149+0000 7f33a7b73b80 1 client.27758 shutdown 2024-06-10T01:17:44.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.149+0000 7f33a7b73b80 1 -- 172.21.15.7:0/2193910912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1f1487460 msgr2=0x55c1f1493390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:44.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.149+0000 7f33a7b73b80 1 --2- 172.21.15.7:0/2193910912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1f1487460 0x55c1f1493390 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3388086040 tx=0x7f3388005850 comp rx=0 tx=0).stop 2024-06-10T01:17:44.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.150+0000 7f33a7b73b80 1 -- 172.21.15.7:0/2193910912 shutdown_connections 2024-06-10T01:17:44.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.150+0000 7f33a7b73b80 1 --2- 172.21.15.7:0/2193910912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1f148d5f0 0x55c1f1490c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.150+0000 7f33a7b73b80 1 --2- 172.21.15.7:0/2193910912 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c1f14a0dd0 0x55c1f14a31c0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.150+0000 7f33a7b73b80 1 --2- 172.21.15.7:0/2193910912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1f1487460 0x55c1f1493390 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.150+0000 7f337aff5700 0 client.27758 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:44.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.150+0000 7f33a7b73b80 1 --2- 172.21.15.7:0/2193910912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1f148ccd0 0x55c1f14906e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.150+0000 7f33a7b73b80 1 -- 172.21.15.7:0/2193910912 >> 172.21.15.7:0/2193910912 conn(0x55c1f13b1e50 msgr2=0x55c1f1485f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:44.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.150+0000 7f33a7b73b80 1 -- 172.21.15.7:0/2193910912 shutdown_connections 2024-06-10T01:17:44.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.150+0000 7f33a7b73b80 1 -- 172.21.15.7:0/2193910912 wait complete. 2024-06-10T01:17:44.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.150+0000 7f33a7b73b80 20 client.27758 trim_cache size 0 max 16384 2024-06-10T01:17:44.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.172+0000 7f33a7b73b80 1 Processor -- start 2024-06-10T01:17:44.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.172+0000 7f33a7b73b80 1 -- start start 2024-06-10T01:17:44.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.172+0000 7f33a7b73b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1f1487460 0x55c1f1485f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.172+0000 7f33a7b73b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1f148ccd0 0x55c1f148f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.172+0000 7f33a7b73b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1f148d5f0 0x55c1f1481fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.172+0000 7f33a7b73b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f33981b2d60 con 0x55c1f148ccd0 2024-06-10T01:17:44.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.172+0000 7f33a7b73b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f33981b05f0 con 0x55c1f148d5f0 2024-06-10T01:17:44.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.172+0000 7f33a7b73b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f33981ae440 con 0x55c1f1487460 2024-06-10T01:17:44.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.172+0000 7f33967fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1f148ccd0 0x55c1f148f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.172+0000 7f33967fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1f148ccd0 0x55c1f148f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50906/0 (socket says 172.21.15.7:50906) 2024-06-10T01:17:44.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.172+0000 7f33967fc700 1 -- 172.21.15.7:0/2697049138 learned_addr learned my addr 172.21.15.7:0/2697049138 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:44.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.172+0000 7f3396ffd700 1 --2- 172.21.15.7:0/2697049138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1f1487460 0x55c1f1485f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.172+0000 7f33977fe700 1 --2- 172.21.15.7:0/2697049138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1f148d5f0 0x55c1f1481fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.172+0000 7f33967fc700 1 -- 172.21.15.7:0/2697049138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1f1487460 msgr2=0x55c1f1485f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:44.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.172+0000 7f33967fc700 1 --2- 172.21.15.7:0/2697049138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1f1487460 0x55c1f1485f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.172+0000 7f33967fc700 1 -- 172.21.15.7:0/2697049138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1f148d5f0 msgr2=0x55c1f1481fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:44.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.172+0000 7f33967fc700 1 --2- 172.21.15.7:0/2697049138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1f148d5f0 0x55c1f1481fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.172+0000 7f33967fc700 1 -- 172.21.15.7:0/2697049138 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1f148ec50 con 0x55c1f148ccd0 2024-06-10T01:17:44.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.173+0000 7f33967fc700 1 --2- 172.21.15.7:0/2697049138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1f148ccd0 0x55c1f148f860 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f338c013010 tx=0x7f338c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d4e4821e793059 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:44.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.173+0000 7f337cff9700 1 -- 172.21.15.7:0/2697049138 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338c01f7e0 con 0x55c1f148ccd0 2024-06-10T01:17:44.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.173+0000 7f337cff9700 1 -- 172.21.15.7:0/2697049138 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f338c02a3d0 con 0x55c1f148ccd0 2024-06-10T01:17:44.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.173+0000 7f337cff9700 1 -- 172.21.15.7:0/2697049138 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338c01fe60 con 0x55c1f148ccd0 2024-06-10T01:17:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.174+0000 7f33a7b73b80 1 -- 172.21.15.7:0/2697049138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1f148ccd0 msgr2=0x55c1f148f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.174+0000 7f33a7b73b80 1 --2- 172.21.15.7:0/2697049138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1f148ccd0 0x55c1f148f860 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f338c013010 tx=0x7f338c014000 comp rx=0 tx=0).stop 2024-06-10T01:17:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.174+0000 7f33a7b73b80 1 -- 172.21.15.7:0/2697049138 shutdown_connections 2024-06-10T01:17:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.174+0000 7f33a7b73b80 1 --2- 172.21.15.7:0/2697049138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1f148d5f0 0x55c1f1481fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.174+0000 7f33a7b73b80 1 --2- 172.21.15.7:0/2697049138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1f1487460 0x55c1f1485f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.174+0000 7f33a7b73b80 1 --2- 172.21.15.7:0/2697049138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1f148ccd0 0x55c1f148f860 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.174+0000 7f33a7b73b80 1 -- 172.21.15.7:0/2697049138 >> 172.21.15.7:0/2697049138 conn(0x55c1f13b1e50 msgr2=0x55c1f14818c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.174+0000 7f33a7b73b80 1 -- 172.21.15.7:0/2697049138 shutdown_connections 2024-06-10T01:17:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.174+0000 7f33a7b73b80 1 -- 172.21.15.7:0/2697049138 wait complete. 2024-06-10T01:17:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.174+0000 7f33a7b73b80 1 -- start start 2024-06-10T01:17:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.174+0000 7f33a7b73b80 1 Processor -- start 2024-06-10T01:17:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.175+0000 7f33a7b73b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1f1487460 0x55c1f14a6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.175+0000 7f33a7b73b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1f148ccd0 0x55c1f14a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.175+0000 7f33a7b73b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1f148d5f0 0x55c1f14a5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.175+0000 7f33a7b73b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f33981aeab0 con 0x55c1f1487460 2024-06-10T01:17:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.175+0000 7f33a7b73b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f33981aff20 con 0x55c1f148ccd0 2024-06-10T01:17:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.175+0000 7f33a7b73b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f33981af180 con 0x55c1f148d5f0 2024-06-10T01:17:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.175+0000 7f33a7b73b80 10 client.0 _ll_register_callbacks cb 0x55c1f1372ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:17:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.175+0000 7f3396ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1f1487460 0x55c1f14a6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.175+0000 7f33967fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1f148ccd0 0x55c1f14a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.175+0000 7f33967fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1f148ccd0 0x55c1f14a4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:54144/0 (socket says 172.21.15.7:54144) 2024-06-10T01:17:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.175+0000 7f33967fc700 1 -- 172.21.15.7:0/3446903206 learned_addr learned my addr 172.21.15.7:0/3446903206 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.175+0000 7f3396ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1f1487460 0x55c1f14a6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50922/0 (socket says 172.21.15.7:50922) 2024-06-10T01:17:44.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.175+0000 7f33977fe700 1 --2- 172.21.15.7:0/3446903206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1f148d5f0 0x55c1f14a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.175+0000 7f3396ffd700 1 -- 172.21.15.7:0/3446903206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1f148d5f0 msgr2=0x55c1f14a5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:44.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.175+0000 7f3396ffd700 1 --2- 172.21.15.7:0/3446903206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1f148d5f0 0x55c1f14a5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.175+0000 7f3396ffd700 1 -- 172.21.15.7:0/3446903206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1f148ccd0 msgr2=0x55c1f14a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:44.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.175+0000 7f3396ffd700 1 --2- 172.21.15.7:0/3446903206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1f148ccd0 0x55c1f14a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.175+0000 7f3396ffd700 1 -- 172.21.15.7:0/3446903206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1f14a58b0 con 0x55c1f1487460 2024-06-10T01:17:44.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.175+0000 7f3396ffd700 1 --2- 172.21.15.7:0/3446903206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1f1487460 0x55c1f14a6820 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f33880a4620 tx=0x7f33880ba410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abfe24a3c438ff5f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:44.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.175+0000 7f337aff5700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:17:44.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.175+0000 7f337aff5700 1 -- 172.21.15.7:0/3446903206 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33880ba040 con 0x55c1f1487460 2024-06-10T01:17:44.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.175+0000 7f337aff5700 1 -- 172.21.15.7:0/3446903206 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f338809bf10 con 0x55c1f1487460 2024-06-10T01:17:44.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.175+0000 7f33a7b73b80 1 -- 172.21.15.7:0/3446903206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f338c02a3d0 con 0x55c1f1487460 2024-06-10T01:17:44.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.176+0000 7f337aff5700 1 -- 172.21.15.7:0/3446903206 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f338809b400 con 0x55c1f1487460 2024-06-10T01:17:44.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.176+0000 7f33a7b73b80 10 client.18429 fetch_fsmap learned FSMap version 40 2024-06-10T01:17:44.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.176+0000 7f33a7b73b80 1 -- 172.21.15.7:0/3446903206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55c1f14a58b0 con 0x55c1f1487460 2024-06-10T01:17:44.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.176+0000 7f337aff5700 1 -- 172.21.15.7:0/3446903206 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f338c02a3d0 con 0x55c1f1487460 2024-06-10T01:17:44.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.176+0000 7f337aff5700 1 -- 172.21.15.7:0/3446903206 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f33880977f0 con 0x55c1f1487460 2024-06-10T01:17:44.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.176+0000 7f33a7b73b80 10 client.18429 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:17:44.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.176+0000 7f33a7b73b80 10 client.18429 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:17:44.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.176+0000 7f33a7b73b80 20 client.18429 populate_metadata read hostname 'smithi007' 2024-06-10T01:17:44.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.176+0000 7f33a7b73b80 1 --2- 172.21.15.7:0/3446903206 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c1f14a0dd0 0x55c1f150eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.176+0000 7f33a7b73b80 4 client.18429 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.2, "await_for": 45.0}] 2024-06-10T01:17:44.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.176+0000 7f33a7b73b80 1 -- 172.21.15.7:0/3446903206 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.2, "await_for": 45.0}) -- 0x55c1f13e0420 con 0x55c1f14a0dd0 2024-06-10T01:17:44.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.176+0000 7f33967fc700 1 --2- 172.21.15.7:0/3446903206 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c1f14a0dd0 0x55c1f150eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.177+0000 7f33967fc700 1 --2- 172.21.15.7:0/3446903206 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c1f14a0dd0 0x55c1f150eaa0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a56bfabb92872392 server_cookie=66de91ec2103a3fb in_seq=0 out_seq=0 2024-06-10T01:17:44.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.177+0000 7f337aff5700 10 client.18429 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.235+0000 7f337aff5700 1 -- 172.21.15.7:0/3446903206 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c1f13e0420 con 0x55c1f14a0dd0 2024-06-10T01:17:44.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.235+0000 7f337aff5700 10 client.18429 handle_command_reply: tid=0 2024-06-10T01:17:44.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.235+0000 7f33a7b73b80 1 client.18429 shutdown 2024-06-10T01:17:44.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.235+0000 7f33a7b73b80 1 -- 172.21.15.7:0/3446903206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1f1487460 msgr2=0x55c1f14a6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:44.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.235+0000 7f33a7b73b80 1 --2- 172.21.15.7:0/3446903206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1f1487460 0x55c1f14a6820 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f33880a4620 tx=0x7f33880ba410 comp rx=0 tx=0).stop 2024-06-10T01:17:44.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.235+0000 7f33a7b73b80 1 -- 172.21.15.7:0/3446903206 shutdown_connections 2024-06-10T01:17:44.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.235+0000 7f33a7b73b80 1 --2- 172.21.15.7:0/3446903206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1f148d5f0 0x55c1f14a5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.235+0000 7f33a7b73b80 1 --2- 172.21.15.7:0/3446903206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1f1487460 0x55c1f14a6820 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.235+0000 7f33a7b73b80 1 --2- 172.21.15.7:0/3446903206 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c1f14a0dd0 0x55c1f150eaa0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.235+0000 7f33a7b73b80 1 --2- 172.21.15.7:0/3446903206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1f148ccd0 0x55c1f14a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.235+0000 7f337aff5700 0 client.18429 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:44.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.235+0000 7f33a7b73b80 1 -- 172.21.15.7:0/3446903206 >> 172.21.15.7:0/3446903206 conn(0x55c1f13b1e50 msgr2=0x55c1f148b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:44.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.235+0000 7f33a7b73b80 1 -- 172.21.15.7:0/3446903206 shutdown_connections 2024-06-10T01:17:44.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.235+0000 7f33a7b73b80 1 -- 172.21.15.7:0/3446903206 wait complete. 2024-06-10T01:17:44.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.235+0000 7f33a7b73b80 20 client.18429 trim_cache size 0 max 16384 2024-06-10T01:17:44.231 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:17:44.231 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 40, 2024-06-10T01:17:44.231 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:17:44.231 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 38, 2024-06-10T01:17:44.231 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:17:44.231 INFO:teuthology.orchestra.run.smithi007.stdout: "b6c01176": { 2024-06-10T01:17:44.232 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 38, 2024-06-10T01:17:44.232 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:17:44.232 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:17:44.232 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:17:44.232 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:17:44.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:17:44.232 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:17:44.232 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 147.2, 2024-06-10T01:17:44.232 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:17:44.232 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:17:44.232 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:17:44.232 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:17:44.232 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:17:44.233 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:17:44.233 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:17:44.233 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:17:44.233 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:17:44.233 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:17:44.233 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:17:44.233 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:17:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.236+0000 7f339d2f0700 1 -- 172.21.15.7:0/2779706130 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f33800a2cc0 msgr2=0x7f33800a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.236+0000 7f339d2f0700 1 --2- 172.21.15.7:0/2779706130 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f33800a2cc0 0x7f33800a5180 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3398109b60 tx=0x7f338c009260 comp rx=0 tx=0).stop 2024-06-10T01:17:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.236+0000 7f339d2f0700 1 -- 172.21.15.7:0/2779706130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33981046d0 msgr2=0x7f339819adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.236+0000 7f339d2f0700 1 --2- 172.21.15.7:0/2779706130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33981046d0 0x7f339819adf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f338800e9c0 tx=0x7f338800ecd0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.236+0000 7f339d2f0700 1 -- 172.21.15.7:0/2779706130 shutdown_connections 2024-06-10T01:17:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.236+0000 7f339d2f0700 1 --2- 172.21.15.7:0/2779706130 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f33800a2cc0 0x7f33800a5180 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.236+0000 7f339d2f0700 1 --2- 172.21.15.7:0/2779706130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f339810a6d0 0x7f339819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.236+0000 7f339d2f0700 1 --2- 172.21.15.7:0/2779706130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33981046d0 0x7f339819adf0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.236+0000 7f339d2f0700 1 --2- 172.21.15.7:0/2779706130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3398105090 0x7f339819b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.236+0000 7f339d2f0700 1 -- 172.21.15.7:0/2779706130 >> 172.21.15.7:0/2779706130 conn(0x7f33981001c0 msgr2=0x7f33981006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:44.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.236+0000 7f339d2f0700 1 -- 172.21.15.7:0/2779706130 shutdown_connections 2024-06-10T01:17:44.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:17:44.236+0000 7f339d2f0700 1 -- 172.21.15.7:0/2779706130 wait complete. 2024-06-10T01:17:44.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.567+0000 7f9cb2622640 1 -- 172.21.15.7:0/3759871193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cac104de0 msgr2=0x7f9cac1071d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:44.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.567+0000 7f9cb2622640 1 --2- 172.21.15.7:0/3759871193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cac104de0 0x7f9cac1071d0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0009900 tx=0x7f9ca002a410 comp rx=0 tx=0).stop 2024-06-10T01:17:44.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.567+0000 7f9cb2622640 1 -- 172.21.15.7:0/3759871193 shutdown_connections 2024-06-10T01:17:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.567+0000 7f9cb2622640 1 --2- 172.21.15.7:0/3759871193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cac107710 0x7f9cac109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.567+0000 7f9cb2622640 1 --2- 172.21.15.7:0/3759871193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cac104de0 0x7f9cac1071d0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.567+0000 7f9cb2622640 1 --2- 172.21.15.7:0/3759871193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cac1024b0 0x7f9cac1048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.567+0000 7f9cb2622640 1 -- 172.21.15.7:0/3759871193 >> 172.21.15.7:0/3759871193 conn(0x7f9cac0fbec0 msgr2=0x7f9cac0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.567+0000 7f9cb2622640 1 -- 172.21.15.7:0/3759871193 shutdown_connections 2024-06-10T01:17:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.567+0000 7f9cb2622640 1 -- 172.21.15.7:0/3759871193 wait complete. 2024-06-10T01:17:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.568+0000 7f9cb2622640 1 Processor -- start 2024-06-10T01:17:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.568+0000 7f9cb2622640 1 -- start start 2024-06-10T01:17:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.568+0000 7f9cb2622640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cac1024b0 0x7f9cac197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.568+0000 7f9cb2622640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cac104de0 0x7f9cac198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.568+0000 7f9cb2622640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cac107710 0x7f9cac19f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.568+0000 7f9cb2622640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac10d6a0 con 0x7f9cac107710 2024-06-10T01:17:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.568+0000 7f9cb2622640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac10d520 con 0x7f9cac104de0 2024-06-10T01:17:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.568+0000 7f9cb2622640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac10d820 con 0x7f9cac1024b0 2024-06-10T01:17:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.568+0000 7f9cb0bbf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cac107710 0x7f9cac19f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.568+0000 7f9cb0bbf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cac107710 0x7f9cac19f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50936/0 (socket says 172.21.15.7:50936) 2024-06-10T01:17:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.569+0000 7f9cb0bbf640 1 -- 172.21.15.7:0/330167848 learned_addr learned my addr 172.21.15.7:0/330167848 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.569+0000 7f9cab7fe640 1 --2- 172.21.15.7:0/330167848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cac104de0 0x7f9cac198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.569+0000 7f9cabfff640 1 --2- 172.21.15.7:0/330167848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cac1024b0 0x7f9cac197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.569+0000 7f9cb0bbf640 1 -- 172.21.15.7:0/330167848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cac1024b0 msgr2=0x7f9cac197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.569+0000 7f9cb0bbf640 1 --2- 172.21.15.7:0/330167848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cac1024b0 0x7f9cac197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.569+0000 7f9cb0bbf640 1 -- 172.21.15.7:0/330167848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cac104de0 msgr2=0x7f9cac198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.569+0000 7f9cb0bbf640 1 --2- 172.21.15.7:0/330167848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cac104de0 0x7f9cac198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.569+0000 7f9cb0bbf640 1 -- 172.21.15.7:0/330167848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cac19f960 con 0x7f9cac107710 2024-06-10T01:17:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.569+0000 7f9cb0bbf640 1 --2- 172.21.15.7:0/330167848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cac107710 0x7f9cac19f2f0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f9ca400eb10 tx=0x7f9ca400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.569+0000 7f9ca97fa640 1 -- 172.21.15.7:0/330167848 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4004b20 con 0x7f9cac107710 2024-06-10T01:17:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.569+0000 7f9ca97fa640 1 -- 172.21.15.7:0/330167848 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ca4004cc0 con 0x7f9cac107710 2024-06-10T01:17:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.569+0000 7f9cb2622640 1 -- 172.21.15.7:0/330167848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cac19fc50 con 0x7f9cac107710 2024-06-10T01:17:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.569+0000 7f9ca97fa640 1 -- 172.21.15.7:0/330167848 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4010760 con 0x7f9cac107710 2024-06-10T01:17:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.569+0000 7f9cb2622640 1 -- 172.21.15.7:0/330167848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cac1a0120 con 0x7f9cac107710 2024-06-10T01:17:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.571+0000 7f9ca97fa640 1 -- 172.21.15.7:0/330167848 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9ca4005020 con 0x7f9cac107710 2024-06-10T01:17:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.572+0000 7f9cb5405740 1 Processor -- start 2024-06-10T01:17:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.572+0000 7f9cb5405740 1 -- start start 2024-06-10T01:17:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.572+0000 7f9cb5405740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618d337f010 0x5618d337f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.572+0000 7f9cb5405740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618d337fa00 0x5618d3384500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.572+0000 7f9cb5405740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618d3384b70 0x5618d3387000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.572+0000 7f9cb5405740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5618d32e24f0 con 0x5618d337fa00 2024-06-10T01:17:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.572+0000 7f9cb5405740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5618d3280870 con 0x5618d337f010 2024-06-10T01:17:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.572+0000 7f9cb5405740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5618d32de4b0 con 0x5618d3384b70 2024-06-10T01:17:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.572+0000 7f9cabfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618d337fa00 0x5618d3384500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.572+0000 7f9ca97fa640 1 --2- 172.21.15.7:0/330167848 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9c940a2b80 0x7f9c940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.572+0000 7f9cabfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618d337fa00 0x5618d3384500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50946/0 (socket says 172.21.15.7:50946) 2024-06-10T01:17:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.572+0000 7f9cabfff640 1 -- 172.21.15.7:0/4093037959 learned_addr learned my addr 172.21.15.7:0/4093037959 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.572+0000 7f9cab7fe640 1 --2- 172.21.15.7:0/4093037959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618d337f010 0x5618d337f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.572+0000 7f9cb0bbf640 1 --2- 172.21.15.7:0/330167848 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9c940a2b80 0x7f9c940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.572+0000 7f9cab7fe640 1 --2- 172.21.15.7:0/4093037959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618d3384b70 0x5618d3387000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.572+0000 7f9ca97fa640 1 -- 172.21.15.7:0/330167848 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f9ca40cf9d0 con 0x7f9cac107710 2024-06-10T01:17:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.572+0000 7f9cabfff640 1 -- 172.21.15.7:0/4093037959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618d3384b70 msgr2=0x5618d3387000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.572+0000 7f9cabfff640 1 --2- 172.21.15.7:0/4093037959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618d3384b70 0x5618d3387000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.572+0000 7f9cabfff640 1 -- 172.21.15.7:0/4093037959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618d337f010 msgr2=0x5618d337f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.572+0000 7f9cabfff640 1 --2- 172.21.15.7:0/4093037959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618d337f010 0x5618d337f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.572+0000 7f9cabfff640 1 -- 172.21.15.7:0/4093037959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618d329c9b0 con 0x5618d337fa00 2024-06-10T01:17:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.572+0000 7f9cab7fe640 1 --2- 172.21.15.7:0/4093037959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618d337f010 0x5618d337f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:17:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.573+0000 7f9cabfff640 1 --2- 172.21.15.7:0/4093037959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618d337fa00 0x5618d3384500 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c00bd10 tx=0x7f9c9c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24dbdbe729156ee1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.573+0000 7f9cb0bbf640 1 --2- 172.21.15.7:0/330167848 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9c940a2b80 0x7f9c940a5040 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40049d0 tx=0x7f9ca4076040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.573+0000 7f9c927fc640 1 -- 172.21.15.7:0/4093037959 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c004030 con 0x5618d337fa00 2024-06-10T01:17:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.573+0000 7f9c927fc640 1 -- 172.21.15.7:0/4093037959 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c9c0041d0 con 0x5618d337fa00 2024-06-10T01:17:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.573+0000 7f9c927fc640 1 -- 172.21.15.7:0/4093037959 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c0117e0 con 0x5618d337fa00 2024-06-10T01:17:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.573+0000 7f9cb5405740 1 -- 172.21.15.7:0/4093037959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618d337fa00 msgr2=0x5618d3384500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.573+0000 7f9cb5405740 1 --2- 172.21.15.7:0/4093037959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618d337fa00 0x5618d3384500 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c00bd10 tx=0x7f9c9c0107a0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.573+0000 7f9cb5405740 1 -- 172.21.15.7:0/4093037959 shutdown_connections 2024-06-10T01:17:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.573+0000 7f9cb5405740 1 --2- 172.21.15.7:0/4093037959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618d3384b70 0x5618d3387000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.573+0000 7f9cb5405740 1 --2- 172.21.15.7:0/4093037959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618d337fa00 0x5618d3384500 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.573+0000 7f9cb5405740 1 --2- 172.21.15.7:0/4093037959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618d337f010 0x5618d337f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.574+0000 7f9cb5405740 1 -- 172.21.15.7:0/4093037959 >> 172.21.15.7:0/4093037959 conn(0x5618d337a680 msgr2=0x5618d337aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.574+0000 7f9cb5405740 1 -- 172.21.15.7:0/4093037959 shutdown_connections 2024-06-10T01:17:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.574+0000 7f9cb5405740 1 -- 172.21.15.7:0/4093037959 wait complete. 2024-06-10T01:17:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.574+0000 7f9cb5405740 1 -- start start 2024-06-10T01:17:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.574+0000 7f9cb5405740 1 Processor -- start 2024-06-10T01:17:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.574+0000 7f9cb5405740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618d337f010 0x5618d3389250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.574+0000 7f9cb5405740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618d337fa00 0x5618d3389790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.574+0000 7f9cb5405740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618d3384b70 0x5618d33907d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.574+0000 7f9cb5405740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5618d3387b20 con 0x5618d3384b70 2024-06-10T01:17:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.574+0000 7f9cb5405740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5618d326bae0 con 0x5618d337fa00 2024-06-10T01:17:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.574+0000 7f9cb5405740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5618d3387ca0 con 0x5618d337f010 2024-06-10T01:17:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.574+0000 7f9cb5405740 10 client.0 _ll_register_callbacks cb 0x5618d32a49b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:17:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.574+0000 7f9cab7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618d3384b70 0x5618d33907d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.574+0000 7f9cab7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618d3384b70 0x5618d33907d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50960/0 (socket says 172.21.15.7:50960) 2024-06-10T01:17:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.574+0000 7f9cab7fe640 1 -- 172.21.15.7:0/1073484640 learned_addr learned my addr 172.21.15.7:0/1073484640 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.575+0000 7f9cabfff640 1 --2- 172.21.15.7:0/1073484640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618d337fa00 0x5618d3389790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.575+0000 7f9cab7fe640 1 --2- 172.21.15.7:0/1073484640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618d337f010 0x5618d3389250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.575+0000 7f9cab7fe640 1 -- 172.21.15.7:0/1073484640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618d337f010 msgr2=0x5618d3389250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.575+0000 7f9cab7fe640 1 --2- 172.21.15.7:0/1073484640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618d337f010 0x5618d3389250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.575+0000 7f9cab7fe640 1 -- 172.21.15.7:0/1073484640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618d337fa00 msgr2=0x5618d3389790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.575+0000 7f9cab7fe640 1 --2- 172.21.15.7:0/1073484640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618d337fa00 0x5618d3389790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.575+0000 7f9cab7fe640 1 -- 172.21.15.7:0/1073484640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618d2ff6210 con 0x5618d3384b70 2024-06-10T01:17:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.575+0000 7f9cab7fe640 1 --2- 172.21.15.7:0/1073484640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618d3384b70 0x5618d33907d0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0031610 tx=0x7f9ca002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da9327147b919cc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.575+0000 7f9c8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:17:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.575+0000 7f9c8b7fe640 1 -- 172.21.15.7:0/1073484640 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0035040 con 0x5618d3384b70 2024-06-10T01:17:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.575+0000 7f9c8b7fe640 1 -- 172.21.15.7:0/1073484640 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f9ca0004220 con 0x5618d3384b70 2024-06-10T01:17:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.575+0000 7f9cb5405740 1 -- 172.21.15.7:0/1073484640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c9c0041d0 con 0x5618d3384b70 2024-06-10T01:17:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.575+0000 7f9c8b7fe640 1 -- 172.21.15.7:0/1073484640 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9ca0004420 con 0x5618d3384b70 2024-06-10T01:17:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.575+0000 7f9cb5405740 10 client.18441 fetch_fsmap learned FSMap version 40 2024-06-10T01:17:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.575+0000 7f9cb5405740 1 -- 172.21.15.7:0/1073484640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5618d30c2620 con 0x5618d3384b70 2024-06-10T01:17:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.575+0000 7f9c8b7fe640 1 -- 172.21.15.7:0/1073484640 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c9c0041d0 con 0x5618d3384b70 2024-06-10T01:17:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.576+0000 7f9c8b7fe640 1 -- 172.21.15.7:0/1073484640 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f9ca0004780 con 0x5618d3384b70 2024-06-10T01:17:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.576+0000 7f9cb5405740 10 client.18441 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:17:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.576+0000 7f9cb5405740 10 client.18441 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:17:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.576+0000 7f9cb5405740 20 client.18441 populate_metadata read hostname 'smithi007' 2024-06-10T01:17:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.576+0000 7f9cb5405740 1 --2- 172.21.15.7:0/1073484640 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5618d3395060 0x5618d3397490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.576+0000 7f9cb5405740 4 client.18441 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:17:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.576+0000 7f9cb5405740 1 -- 172.21.15.7:0/1073484640 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618d32f5f10 con 0x5618d3395060 2024-06-10T01:17:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.576+0000 7f9cabfff640 1 --2- 172.21.15.7:0/1073484640 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5618d3395060 0x5618d3397490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.577+0000 7f9cabfff640 1 --2- 172.21.15.7:0/1073484640 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5618d3395060 0x5618d3397490 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4896ea2435a8cf9 server_cookie=734f75797e16e025 in_seq=0 out_seq=0 2024-06-10T01:17:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.577+0000 7f9c8b7fe640 10 client.18441 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.577+0000 7f9c8b7fe640 1 -- 172.21.15.7:0/1073484640 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5618d32f5f10 con 0x5618d3395060 2024-06-10T01:17:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.577+0000 7f9c8b7fe640 10 client.18441 handle_command_reply: tid=0 2024-06-10T01:17:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.577+0000 7f9cb5405740 1 client.18441 shutdown 2024-06-10T01:17:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.577+0000 7f9cb5405740 1 -- 172.21.15.7:0/1073484640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618d3384b70 msgr2=0x5618d33907d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.577+0000 7f9cb5405740 1 --2- 172.21.15.7:0/1073484640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618d3384b70 0x5618d33907d0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0031610 tx=0x7f9ca002a410 comp rx=0 tx=0).stop 2024-06-10T01:17:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.578+0000 7f9cb5405740 1 -- 172.21.15.7:0/1073484640 shutdown_connections 2024-06-10T01:17:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.578+0000 7f9cb5405740 1 --2- 172.21.15.7:0/1073484640 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5618d3395060 0x5618d3397490 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.578+0000 7f9cb5405740 1 --2- 172.21.15.7:0/1073484640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618d3384b70 0x5618d33907d0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.578+0000 7f9cb5405740 1 --2- 172.21.15.7:0/1073484640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618d337fa00 0x5618d3389790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.578+0000 7f9c8b7fe640 0 client.18441 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.578+0000 7f9cb5405740 1 --2- 172.21.15.7:0/1073484640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618d337f010 0x5618d3389250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.578+0000 7f9cb5405740 1 -- 172.21.15.7:0/1073484640 >> 172.21.15.7:0/1073484640 conn(0x5618d337a680 msgr2=0x5618d33797e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.578+0000 7f9cb5405740 1 -- 172.21.15.7:0/1073484640 shutdown_connections 2024-06-10T01:17:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.578+0000 7f9cb5405740 1 -- 172.21.15.7:0/1073484640 wait complete. 2024-06-10T01:17:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.578+0000 7f9cb5405740 20 client.18441 trim_cache size 0 max 16384 2024-06-10T01:17:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.593+0000 7f9cb5405740 1 Processor -- start 2024-06-10T01:17:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.593+0000 7f9cb5405740 1 -- start start 2024-06-10T01:17:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.593+0000 7f9cb5405740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618d337f010 0x5618d3380be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.593+0000 7f9cb5405740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618d337fa00 0x5618d3390790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.593+0000 7f9cb5405740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618d3384b70 0x5618d3381120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.593+0000 7f9cb5405740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac1acf10 con 0x5618d337fa00 2024-06-10T01:17:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.593+0000 7f9cb5405740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac1aa7e0 con 0x5618d337f010 2024-06-10T01:17:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.593+0000 7f9cb5405740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac1a8630 con 0x5618d3384b70 2024-06-10T01:17:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.593+0000 7f9cabfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618d337fa00 0x5618d3390790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.593+0000 7f9cabfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618d337fa00 0x5618d3390790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50972/0 (socket says 172.21.15.7:50972) 2024-06-10T01:17:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.593+0000 7f9cabfff640 1 -- 172.21.15.7:0/3126199470 learned_addr learned my addr 172.21.15.7:0/3126199470 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.593+0000 7f9cab7fe640 1 --2- 172.21.15.7:0/3126199470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618d337f010 0x5618d3380be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.593+0000 7f9cab7fe640 1 --2- 172.21.15.7:0/3126199470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618d3384b70 0x5618d3381120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.593+0000 7f9cabfff640 1 -- 172.21.15.7:0/3126199470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618d3384b70 msgr2=0x5618d3381120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.593+0000 7f9cabfff640 1 --2- 172.21.15.7:0/3126199470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618d3384b70 0x5618d3381120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.593+0000 7f9cabfff640 1 -- 172.21.15.7:0/3126199470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618d337f010 msgr2=0x5618d3380be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.593+0000 7f9cabfff640 1 --2- 172.21.15.7:0/3126199470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618d337f010 0x5618d3380be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.593+0000 7f9cabfff640 1 -- 172.21.15.7:0/3126199470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618d33806d0 con 0x5618d337fa00 2024-06-10T01:17:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.593+0000 7f9cab7fe640 1 --2- 172.21.15.7:0/3126199470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618d3384b70 0x5618d3381120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:17:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.594+0000 7f9cabfff640 1 --2- 172.21.15.7:0/3126199470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618d337fa00 0x5618d3390790 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c00bd10 tx=0x7f9c9c005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c0f87c72ef8c504 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.594+0000 7f9c91ffb640 1 -- 172.21.15.7:0/3126199470 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c005240 con 0x5618d337fa00 2024-06-10T01:17:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.594+0000 7f9c91ffb640 1 -- 172.21.15.7:0/3126199470 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c9c0053e0 con 0x5618d337fa00 2024-06-10T01:17:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.594+0000 7f9c91ffb640 1 -- 172.21.15.7:0/3126199470 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c011730 con 0x5618d337fa00 2024-06-10T01:17:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.594+0000 7f9cb5405740 1 -- 172.21.15.7:0/3126199470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618d337fa00 msgr2=0x5618d3390790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.594+0000 7f9cb5405740 1 --2- 172.21.15.7:0/3126199470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618d337fa00 0x5618d3390790 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c00bd10 tx=0x7f9c9c005c50 comp rx=0 tx=0).stop 2024-06-10T01:17:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.594+0000 7f9cb5405740 1 -- 172.21.15.7:0/3126199470 shutdown_connections 2024-06-10T01:17:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.594+0000 7f9cb5405740 1 --2- 172.21.15.7:0/3126199470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618d3384b70 0x5618d3381120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.594+0000 7f9cb5405740 1 --2- 172.21.15.7:0/3126199470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618d337fa00 0x5618d3390790 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.594+0000 7f9cb5405740 1 --2- 172.21.15.7:0/3126199470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618d337f010 0x5618d3380be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.594+0000 7f9cb5405740 1 -- 172.21.15.7:0/3126199470 >> 172.21.15.7:0/3126199470 conn(0x5618d337a680 msgr2=0x5618d3388040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.594+0000 7f9cb5405740 1 -- 172.21.15.7:0/3126199470 shutdown_connections 2024-06-10T01:17:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.594+0000 7f9cb5405740 1 -- 172.21.15.7:0/3126199470 wait complete. 2024-06-10T01:17:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.595+0000 7f9cb5405740 1 -- start start 2024-06-10T01:17:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.595+0000 7f9cb5405740 1 Processor -- start 2024-06-10T01:17:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.595+0000 7f9cb5405740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618d337f010 0x5618d338dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.595+0000 7f9cb5405740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618d337fa00 0x5618d338dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.595+0000 7f9cb5405740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618d3384b70 0x5618d338f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.595+0000 7f9cb5405740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac1a8ca0 con 0x5618d337fa00 2024-06-10T01:17:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.595+0000 7f9cb5405740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac1aa110 con 0x5618d337f010 2024-06-10T01:17:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.595+0000 7f9cb5405740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac1a9370 con 0x5618d3384b70 2024-06-10T01:17:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.595+0000 7f9cb5405740 10 client.0 _ll_register_callbacks cb 0x7f9ca0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:17:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.595+0000 7f9cabfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618d337fa00 0x5618d338dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.595+0000 7f9cabfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618d337fa00 0x5618d338dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50974/0 (socket says 172.21.15.7:50974) 2024-06-10T01:17:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.595+0000 7f9cabfff640 1 -- 172.21.15.7:0/3143129154 learned_addr learned my addr 172.21.15.7:0/3143129154 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.595+0000 7f9cab7fe640 1 --2- 172.21.15.7:0/3143129154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618d337f010 0x5618d338dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.595+0000 7f9cabfff640 1 -- 172.21.15.7:0/3143129154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618d3384b70 msgr2=0x5618d338f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.595+0000 7f9cabfff640 1 --2- 172.21.15.7:0/3143129154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618d3384b70 0x5618d338f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.595+0000 7f9cabfff640 1 -- 172.21.15.7:0/3143129154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618d337f010 msgr2=0x5618d338dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.595+0000 7f9cabfff640 1 --2- 172.21.15.7:0/3143129154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618d337f010 0x5618d338dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.595+0000 7f9cabfff640 1 -- 172.21.15.7:0/3143129154 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618d338f5f0 con 0x5618d337fa00 2024-06-10T01:17:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.595+0000 7f9cab7fe640 1 --2- 172.21.15.7:0/3143129154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618d337f010 0x5618d338dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:17:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.596+0000 7f9cabfff640 1 --2- 172.21.15.7:0/3143129154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618d337fa00 0x5618d338dff0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0107e0 tx=0x7f9c9c005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1825f5ce6bb1e2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.596+0000 7f9c8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:17:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.596+0000 7f9c8b7fe640 1 -- 172.21.15.7:0/3143129154 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c037070 con 0x5618d337fa00 2024-06-10T01:17:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.596+0000 7f9c8b7fe640 1 -- 172.21.15.7:0/3143129154 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f9c9c0037f0 con 0x5618d337fa00 2024-06-10T01:17:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.596+0000 7f9cb5405740 1 -- 172.21.15.7:0/3143129154 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c9c0053e0 con 0x5618d337fa00 2024-06-10T01:17:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.596+0000 7f9c8b7fe640 1 -- 172.21.15.7:0/3143129154 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9c9c0039f0 con 0x5618d337fa00 2024-06-10T01:17:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.596+0000 7f9cb5405740 10 client.18447 fetch_fsmap learned FSMap version 40 2024-06-10T01:17:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.596+0000 7f9cb5405740 1 -- 172.21.15.7:0/3143129154 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5618d3390870 con 0x5618d337fa00 2024-06-10T01:17:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.596+0000 7f9c8b7fe640 1 -- 172.21.15.7:0/3143129154 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c9c0053e0 con 0x5618d337fa00 2024-06-10T01:17:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.596+0000 7f9c8b7fe640 1 -- 172.21.15.7:0/3143129154 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f9c9c01b420 con 0x5618d337fa00 2024-06-10T01:17:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.596+0000 7f9cb5405740 10 client.18447 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:17:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.596+0000 7f9cb5405740 10 client.18447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:17:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.596+0000 7f9cb5405740 20 client.18447 populate_metadata read hostname 'smithi007' 2024-06-10T01:17:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.596+0000 7f9cb5405740 1 --2- 172.21.15.7:0/3143129154 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5618d34000b0 0x5618d34024a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.596+0000 7f9cb5405740 4 client.18447 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:17:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.596+0000 7f9cb5405740 1 -- 172.21.15.7:0/3143129154 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5618d32efc20 con 0x5618d34000b0 2024-06-10T01:17:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.597+0000 7f9cab7fe640 1 --2- 172.21.15.7:0/3143129154 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5618d34000b0 0x5618d34024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.597+0000 7f9cab7fe640 1 --2- 172.21.15.7:0/3143129154 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5618d34000b0 0x5618d34024a0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=944ec6022e1aa63f server_cookie=740ef58633f88bac in_seq=0 out_seq=0 2024-06-10T01:17:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.597+0000 7f9c8b7fe640 10 client.18447 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.700+0000 7f9c8b7fe640 1 -- 172.21.15.7:0/3143129154 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5618d32efc20 con 0x5618d34000b0 2024-06-10T01:17:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.700+0000 7f9c8b7fe640 10 client.18447 handle_command_reply: tid=0 2024-06-10T01:17:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.700+0000 7f9cb5405740 1 client.18447 shutdown 2024-06-10T01:17:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.700+0000 7f9cb5405740 1 -- 172.21.15.7:0/3143129154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618d337fa00 msgr2=0x5618d338dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.700+0000 7f9cb5405740 1 --2- 172.21.15.7:0/3143129154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618d337fa00 0x5618d338dff0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0107e0 tx=0x7f9c9c005610 comp rx=0 tx=0).stop 2024-06-10T01:17:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.700+0000 7f9cb5405740 1 -- 172.21.15.7:0/3143129154 shutdown_connections 2024-06-10T01:17:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.700+0000 7f9cb5405740 1 --2- 172.21.15.7:0/3143129154 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5618d34000b0 0x5618d34024a0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.700+0000 7f9cb5405740 1 --2- 172.21.15.7:0/3143129154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618d3384b70 0x5618d338f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.700+0000 7f9c8b7fe640 0 client.18447 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.700+0000 7f9cb5405740 1 --2- 172.21.15.7:0/3143129154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618d337fa00 0x5618d338dff0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.700+0000 7f9cb5405740 1 --2- 172.21.15.7:0/3143129154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618d337f010 0x5618d338dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.700+0000 7f9cb5405740 1 -- 172.21.15.7:0/3143129154 >> 172.21.15.7:0/3143129154 conn(0x5618d337a680 msgr2=0x5618d3389210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.700+0000 7f9cb5405740 1 -- 172.21.15.7:0/3143129154 shutdown_connections 2024-06-10T01:17:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.700+0000 7f9cb5405740 1 -- 172.21.15.7:0/3143129154 wait complete. 2024-06-10T01:17:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.700+0000 7f9cb5405740 20 client.18447 trim_cache size 0 max 16384 2024-06-10T01:17:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:17:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:17:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:17:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.701+0000 7f9cb2622640 1 -- 172.21.15.7:0/330167848 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9c940a2b80 msgr2=0x7f9c940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.701+0000 7f9cb2622640 1 --2- 172.21.15.7:0/330167848 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9c940a2b80 0x7f9c940a5040 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40049d0 tx=0x7f9ca4076040 comp rx=0 tx=0).stop 2024-06-10T01:17:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.701+0000 7f9cb2622640 1 -- 172.21.15.7:0/330167848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cac107710 msgr2=0x7f9cac19f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.701+0000 7f9cb2622640 1 --2- 172.21.15.7:0/330167848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cac107710 0x7f9cac19f2f0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f9ca400eb10 tx=0x7f9ca400c2d0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.701+0000 7f9cb2622640 1 -- 172.21.15.7:0/330167848 shutdown_connections 2024-06-10T01:17:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.701+0000 7f9cb2622640 1 --2- 172.21.15.7:0/330167848 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9c940a2b80 0x7f9c940a5040 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.701+0000 7f9cb2622640 1 --2- 172.21.15.7:0/330167848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cac107710 0x7f9cac19f2f0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.701+0000 7f9cb2622640 1 --2- 172.21.15.7:0/330167848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cac104de0 0x7f9cac198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.701+0000 7f9cb2622640 1 --2- 172.21.15.7:0/330167848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cac1024b0 0x7f9cac197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.701+0000 7f9cb2622640 1 -- 172.21.15.7:0/330167848 >> 172.21.15.7:0/330167848 conn(0x7f9cac0fbec0 msgr2=0x7f9cac102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.701+0000 7f9cb2622640 1 -- 172.21.15.7:0/330167848 shutdown_connections 2024-06-10T01:17:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:44.701+0000 7f9cb2622640 1 -- 172.21.15.7:0/330167848 wait complete. 2024-06-10T01:17:44.752 INFO:tasks.workunit.client.0.smithi007.stdout: 1 881536 7.58 MB/sec execute 505 sec latency 555.625 ms 2024-06-10T01:17:45.027 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b6c01176', quiesce took 5.9 seconds. Will release after: 21.299999999999997 2024-06-10T01:17:45.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:45 smithi175 ceph-mon[36636]: pgmap v784: 129 pgs: 129 active+clean; 148 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 1.2 MiB/s wr, 222 op/s 2024-06-10T01:17:45.753 INFO:tasks.workunit.client.0.smithi007.stdout: 1 881536 7.57 MB/sec execute 506 sec latency 1555.744 ms 2024-06-10T01:17:45.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:45 smithi007 ceph-mon[32398]: pgmap v784: 129 pgs: 129 active+clean; 148 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 1.2 MiB/s wr, 222 op/s 2024-06-10T01:17:45.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:45 smithi078 ceph-mon[36683]: pgmap v784: 129 pgs: 129 active+clean; 148 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 1.2 MiB/s wr, 222 op/s 2024-06-10T01:17:46.753 INFO:tasks.workunit.client.0.smithi007.stdout: 1 881536 7.55 MB/sec execute 507 sec latency 2555.776 ms 2024-06-10T01:17:47.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:47 smithi175 ceph-mon[36636]: pgmap v785: 129 pgs: 129 active+clean; 151 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 1.5 MiB/s wr, 252 op/s 2024-06-10T01:17:47.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:47 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:17:47.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:47 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:17:47.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:47 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:17:47.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:47 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:17:47.753 INFO:tasks.workunit.client.0.smithi007.stdout: 1 881536 7.54 MB/sec execute 508 sec latency 3555.878 ms 2024-06-10T01:17:47.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:47 smithi007 ceph-mon[32398]: pgmap v785: 129 pgs: 129 active+clean; 151 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 1.5 MiB/s wr, 252 op/s 2024-06-10T01:17:47.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:47 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:17:47.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:47 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:17:47.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:47 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:17:47.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:47 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:17:47.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:47 smithi078 ceph-mon[36683]: pgmap v785: 129 pgs: 129 active+clean; 151 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 1.5 MiB/s wr, 252 op/s 2024-06-10T01:17:47.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:47 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:17:47.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:47 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:17:47.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:47 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:17:47.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:47 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:17:48.753 INFO:tasks.workunit.client.0.smithi007.stdout: 1 881536 7.52 MB/sec execute 509 sec latency 4555.955 ms 2024-06-10T01:17:49.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:49 smithi175 ceph-mon[36636]: pgmap v786: 129 pgs: 129 active+clean; 148 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 1.5 MiB/s wr, 247 op/s 2024-06-10T01:17:49.753 INFO:tasks.workunit.client.0.smithi007.stdout: 1 881536 7.51 MB/sec execute 510 sec latency 5556.058 ms 2024-06-10T01:17:49.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:49 smithi007 ceph-mon[32398]: pgmap v786: 129 pgs: 129 active+clean; 148 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 1.5 MiB/s wr, 247 op/s 2024-06-10T01:17:49.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:49 smithi078 ceph-mon[36683]: pgmap v786: 129 pgs: 129 active+clean; 148 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 1.5 MiB/s wr, 247 op/s 2024-06-10T01:17:50.753 INFO:tasks.workunit.client.0.smithi007.stdout: 1 881536 7.49 MB/sec execute 511 sec latency 6556.117 ms 2024-06-10T01:17:51.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:51 smithi175 ceph-mon[36636]: pgmap v787: 129 pgs: 129 active+clean; 149 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 1.6 MiB/s wr, 232 op/s 2024-06-10T01:17:51.753 INFO:tasks.workunit.client.0.smithi007.stdout: 1 881536 7.48 MB/sec execute 512 sec latency 7556.218 ms 2024-06-10T01:17:51.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:51 smithi007 ceph-mon[32398]: pgmap v787: 129 pgs: 129 active+clean; 149 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 1.6 MiB/s wr, 232 op/s 2024-06-10T01:17:51.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:51 smithi078 ceph-mon[36683]: pgmap v787: 129 pgs: 129 active+clean; 149 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 1.6 MiB/s wr, 232 op/s 2024-06-10T01:17:52.753 INFO:tasks.workunit.client.0.smithi007.stdout: 1 881536 7.46 MB/sec execute 513 sec latency 8556.309 ms 2024-06-10T01:17:53.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:53 smithi175 ceph-mon[36636]: pgmap v788: 129 pgs: 129 active+clean; 149 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 1.7 MiB/s wr, 208 op/s 2024-06-10T01:17:53.753 INFO:tasks.workunit.client.0.smithi007.stdout: 1 881536 7.45 MB/sec execute 514 sec latency 9556.409 ms 2024-06-10T01:17:53.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:53 smithi007 ceph-mon[32398]: pgmap v788: 129 pgs: 129 active+clean; 149 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 1.7 MiB/s wr, 208 op/s 2024-06-10T01:17:53.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:53 smithi078 ceph-mon[36683]: pgmap v788: 129 pgs: 129 active+clean; 149 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 1.7 MiB/s wr, 208 op/s 2024-06-10T01:17:54.753 INFO:tasks.workunit.client.0.smithi007.stdout: 1 881536 7.43 MB/sec execute 515 sec latency 10556.522 ms 2024-06-10T01:17:55.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:55 smithi175 ceph-mon[36636]: pgmap v789: 129 pgs: 129 active+clean; 150 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 1.9 MiB/s wr, 193 op/s 2024-06-10T01:17:55.753 INFO:tasks.workunit.client.0.smithi007.stdout: 1 881536 7.42 MB/sec execute 516 sec latency 11556.573 ms 2024-06-10T01:17:55.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:55 smithi007 ceph-mon[32398]: pgmap v789: 129 pgs: 129 active+clean; 150 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 1.9 MiB/s wr, 193 op/s 2024-06-10T01:17:55.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:55 smithi078 ceph-mon[36683]: pgmap v789: 129 pgs: 129 active+clean; 150 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 1.9 MiB/s wr, 193 op/s 2024-06-10T01:17:56.753 INFO:tasks.workunit.client.0.smithi007.stdout: 1 881536 7.40 MB/sec execute 517 sec latency 12556.611 ms 2024-06-10T01:17:57.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:57 smithi175 ceph-mon[36636]: pgmap v790: 129 pgs: 129 active+clean; 151 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-06-10T01:17:57.753 INFO:tasks.workunit.client.0.smithi007.stdout: 1 881536 7.39 MB/sec execute 518 sec latency 13556.651 ms 2024-06-10T01:17:57.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:57 smithi007 ceph-mon[32398]: pgmap v790: 129 pgs: 129 active+clean; 151 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-06-10T01:17:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.801+0000 7f6aad0d4640 1 -- 172.21.15.7:0/1130928986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6aa8101a30 msgr2=0x7f6aa8101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.801+0000 7f6aad0d4640 1 --2- 172.21.15.7:0/1130928986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6aa8101a30 0x7f6aa8101e50 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009900 tx=0x7f6a9802a410 comp rx=0 tx=0).stop 2024-06-10T01:17:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.801+0000 7f6aad0d4640 1 -- 172.21.15.7:0/1130928986 shutdown_connections 2024-06-10T01:17:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.801+0000 7f6aad0d4640 1 --2- 172.21.15.7:0/1130928986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6aa81035e0 0x7f6aa8109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.801+0000 7f6aad0d4640 1 --2- 172.21.15.7:0/1130928986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6aa8102c20 0x7f6aa81030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.801+0000 7f6aad0d4640 1 --2- 172.21.15.7:0/1130928986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6aa8101a30 0x7f6aa8101e50 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.801+0000 7f6aad0d4640 1 -- 172.21.15.7:0/1130928986 >> 172.21.15.7:0/1130928986 conn(0x7f6aa80fcfc0 msgr2=0x7f6aa80ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:57.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.801+0000 7f6aad0d4640 1 -- 172.21.15.7:0/1130928986 shutdown_connections 2024-06-10T01:17:57.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.801+0000 7f6aad0d4640 1 -- 172.21.15.7:0/1130928986 wait complete. 2024-06-10T01:17:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.802+0000 7f6aad0d4640 1 Processor -- start 2024-06-10T01:17:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.802+0000 7f6aad0d4640 1 -- start start 2024-06-10T01:17:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.802+0000 7f6aad0d4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6aa8101a30 0x7f6aa8197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.802+0000 7f6aad0d4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6aa8102c20 0x7f6aa8198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.802+0000 7f6aad0d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6aa81035e0 0x7f6aa819f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.802+0000 7f6aad0d4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa810d6b0 con 0x7f6aa8102c20 2024-06-10T01:17:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.803+0000 7f6aad0d4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa810d530 con 0x7f6aa81035e0 2024-06-10T01:17:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.803+0000 7f6aad0d4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa810d830 con 0x7f6aa8101a30 2024-06-10T01:17:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.803+0000 7f6aa659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6aa8102c20 0x7f6aa8198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.803+0000 7f6aa659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6aa8102c20 0x7f6aa8198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41004/0 (socket says 172.21.15.7:41004) 2024-06-10T01:17:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.803+0000 7f6aa659c640 1 -- 172.21.15.7:0/50171737 learned_addr learned my addr 172.21.15.7:0/50171737 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.803+0000 7f6aa6d9d640 1 --2- 172.21.15.7:0/50171737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6aa8101a30 0x7f6aa8197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.803+0000 7f6aa759e640 1 --2- 172.21.15.7:0/50171737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6aa81035e0 0x7f6aa819f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.803+0000 7f6aa659c640 1 -- 172.21.15.7:0/50171737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6aa8101a30 msgr2=0x7f6aa8197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.803+0000 7f6aa659c640 1 --2- 172.21.15.7:0/50171737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6aa8101a30 0x7f6aa8197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.803+0000 7f6aa659c640 1 -- 172.21.15.7:0/50171737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6aa81035e0 msgr2=0x7f6aa819f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.803+0000 7f6aa659c640 1 --2- 172.21.15.7:0/50171737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6aa81035e0 0x7f6aa819f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.803+0000 7f6aa659c640 1 -- 172.21.15.7:0/50171737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa819f950 con 0x7f6aa8102c20 2024-06-10T01:17:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.803+0000 7f6aa659c640 1 --2- 172.21.15.7:0/50171737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6aa8102c20 0x7f6aa8198280 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00e9f0 tx=0x7f6a9c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.803+0000 7f6a8ffff640 1 -- 172.21.15.7:0/50171737 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c00cd80 con 0x7f6aa8102c20 2024-06-10T01:17:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.803+0000 7f6aad0d4640 1 -- 172.21.15.7:0/50171737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aa819fc40 con 0x7f6aa8102c20 2024-06-10T01:17:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.803+0000 7f6a8ffff640 1 -- 172.21.15.7:0/50171737 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a9c010910 con 0x7f6aa8102c20 2024-06-10T01:17:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.803+0000 7f6a8ffff640 1 -- 172.21.15.7:0/50171737 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c018990 con 0x7f6aa8102c20 2024-06-10T01:17:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.804+0000 7f6aad0d4640 1 -- 172.21.15.7:0/50171737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aa81a0160 con 0x7f6aa8102c20 2024-06-10T01:17:57.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.806+0000 7f6a8ffff640 1 -- 172.21.15.7:0/50171737 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f6a9c010ac0 con 0x7f6aa8102c20 2024-06-10T01:17:57.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.806+0000 7f6aafde6740 1 Processor -- start 2024-06-10T01:17:57.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.806+0000 7f6aafde6740 1 -- start start 2024-06-10T01:17:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.806+0000 7f6aafde6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56208236f010 0x56208236f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.806+0000 7f6aafde6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56208236fa00 0x562082374500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.806+0000 7f6aafde6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562082374b70 0x562082377000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.806+0000 7f6aafde6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5620822d24f0 con 0x56208236fa00 2024-06-10T01:17:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.806+0000 7f6aafde6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562082270870 con 0x56208236f010 2024-06-10T01:17:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.806+0000 7f6aafde6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5620822ce4b0 con 0x562082374b70 2024-06-10T01:17:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.807+0000 7f6aa6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56208236fa00 0x562082374500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.807+0000 7f6aa6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56208236fa00 0x562082374500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41012/0 (socket says 172.21.15.7:41012) 2024-06-10T01:17:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.807+0000 7f6aa6d9d640 1 -- 172.21.15.7:0/2487904216 learned_addr learned my addr 172.21.15.7:0/2487904216 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.807+0000 7f6a8ffff640 1 --2- 172.21.15.7:0/50171737 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6a900a2b80 0x7f6a900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.807+0000 7f6aa659c640 1 --2- 172.21.15.7:0/2487904216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562082374b70 0x562082377000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.807+0000 7f6aa759e640 1 --2- 172.21.15.7:0/2487904216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56208236f010 0x56208236f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.807+0000 7f6aa759e640 1 --2- 172.21.15.7:0/50171737 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6a900a2b80 0x7f6a900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.807+0000 7f6aa6d9d640 1 -- 172.21.15.7:0/2487904216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562082374b70 msgr2=0x562082377000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.807+0000 7f6aa6d9d640 1 --2- 172.21.15.7:0/2487904216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562082374b70 0x562082377000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.807+0000 7f6aa6d9d640 1 -- 172.21.15.7:0/2487904216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56208236f010 msgr2=0x56208236f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.807+0000 7f6aa6d9d640 1 --2- 172.21.15.7:0/2487904216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56208236f010 0x56208236f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.807+0000 7f6aa6d9d640 1 -- 172.21.15.7:0/2487904216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56208228c9b0 con 0x56208236fa00 2024-06-10T01:17:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.807+0000 7f6a8ffff640 1 -- 172.21.15.7:0/50171737 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f6a9c0cf760 con 0x7f6aa8102c20 2024-06-10T01:17:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.807+0000 7f6aa6d9d640 1 --2- 172.21.15.7:0/2487904216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56208236fa00 0x562082374500 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f6a980314e0 tx=0x7f6a98005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d2e815f9c0476 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.807+0000 7f6aa759e640 1 --2- 172.21.15.7:0/50171737 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6a900a2b80 0x7f6a900a5040 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0005c40 tx=0x7f6aa0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.807+0000 7f6a8d7fa640 1 -- 172.21.15.7:0/2487904216 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98035040 con 0x56208236fa00 2024-06-10T01:17:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.807+0000 7f6a8d7fa640 1 -- 172.21.15.7:0/2487904216 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a98043be0 con 0x56208236fa00 2024-06-10T01:17:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.807+0000 7f6a8d7fa640 1 -- 172.21.15.7:0/2487904216 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98005330 con 0x56208236fa00 2024-06-10T01:17:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.808+0000 7f6aafde6740 1 -- 172.21.15.7:0/2487904216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56208236fa00 msgr2=0x562082374500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.808+0000 7f6aafde6740 1 --2- 172.21.15.7:0/2487904216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56208236fa00 0x562082374500 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f6a980314e0 tx=0x7f6a98005de0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.808+0000 7f6aafde6740 1 -- 172.21.15.7:0/2487904216 shutdown_connections 2024-06-10T01:17:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.808+0000 7f6aafde6740 1 --2- 172.21.15.7:0/2487904216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562082374b70 0x562082377000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.808+0000 7f6aafde6740 1 --2- 172.21.15.7:0/2487904216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56208236fa00 0x562082374500 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.808+0000 7f6aafde6740 1 --2- 172.21.15.7:0/2487904216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56208236f010 0x56208236f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.808+0000 7f6aafde6740 1 -- 172.21.15.7:0/2487904216 >> 172.21.15.7:0/2487904216 conn(0x56208236a680 msgr2=0x56208236aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.808+0000 7f6aafde6740 1 -- 172.21.15.7:0/2487904216 shutdown_connections 2024-06-10T01:17:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.808+0000 7f6aafde6740 1 -- 172.21.15.7:0/2487904216 wait complete. 2024-06-10T01:17:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.808+0000 7f6aafde6740 1 -- start start 2024-06-10T01:17:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.809+0000 7f6aafde6740 1 Processor -- start 2024-06-10T01:17:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.809+0000 7f6aafde6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56208236f010 0x5620823792c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.809+0000 7f6aafde6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56208236fa00 0x562082379800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.809+0000 7f6aafde6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562082374b70 0x5620823807d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.809+0000 7f6aafde6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562082377ac0 con 0x56208236f010 2024-06-10T01:17:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.809+0000 7f6aafde6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56208225bae0 con 0x56208236fa00 2024-06-10T01:17:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.809+0000 7f6aafde6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562082377c40 con 0x562082374b70 2024-06-10T01:17:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.809+0000 7f6aafde6740 10 client.0 _ll_register_callbacks cb 0x5620822949b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:17:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.809+0000 7f6aa6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56208236f010 0x5620823792c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.809+0000 7f6aa6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56208236f010 0x5620823792c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41018/0 (socket says 172.21.15.7:41018) 2024-06-10T01:17:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.809+0000 7f6aa6d9d640 1 -- 172.21.15.7:0/3074819249 learned_addr learned my addr 172.21.15.7:0/3074819249 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.809+0000 7f6aa759e640 1 --2- 172.21.15.7:0/3074819249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562082374b70 0x5620823807d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.809+0000 7f6aa659c640 1 --2- 172.21.15.7:0/3074819249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56208236fa00 0x562082379800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.809+0000 7f6aa6d9d640 1 -- 172.21.15.7:0/3074819249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562082374b70 msgr2=0x5620823807d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.809+0000 7f6aa6d9d640 1 --2- 172.21.15.7:0/3074819249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562082374b70 0x5620823807d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.809+0000 7f6aa6d9d640 1 -- 172.21.15.7:0/3074819249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56208236fa00 msgr2=0x562082379800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.809+0000 7f6aa6d9d640 1 --2- 172.21.15.7:0/3074819249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56208236fa00 0x562082379800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.809+0000 7f6aa6d9d640 1 -- 172.21.15.7:0/3074819249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562081fe6210 con 0x56208236f010 2024-06-10T01:17:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.809+0000 7f6aa6d9d640 1 --2- 172.21.15.7:0/3074819249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56208236f010 0x5620823792c0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009900 tx=0x7f6a98043d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b297cdd14ea78f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.809+0000 7f6a8aff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:17:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.810+0000 7f6a8aff5640 1 -- 172.21.15.7:0/3074819249 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9805a060 con 0x56208236f010 2024-06-10T01:17:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.810+0000 7f6a8aff5640 1 -- 172.21.15.7:0/3074819249 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f6a980316b0 con 0x56208236f010 2024-06-10T01:17:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.810+0000 7f6aafde6740 1 -- 172.21.15.7:0/3074819249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a98043be0 con 0x56208236f010 2024-06-10T01:17:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.810+0000 7f6a8aff5640 1 -- 172.21.15.7:0/3074819249 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f6a98061050 con 0x56208236f010 2024-06-10T01:17:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.810+0000 7f6aafde6740 10 client.18459 fetch_fsmap learned FSMap version 40 2024-06-10T01:17:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.810+0000 7f6aafde6740 1 -- 172.21.15.7:0/3074819249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5620820b2620 con 0x56208236f010 2024-06-10T01:17:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.810+0000 7f6a8aff5640 1 -- 172.21.15.7:0/3074819249 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a98043be0 con 0x56208236f010 2024-06-10T01:17:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.810+0000 7f6a8aff5640 1 -- 172.21.15.7:0/3074819249 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f6a98004b50 con 0x56208236f010 2024-06-10T01:17:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.810+0000 7f6aafde6740 10 client.18459 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:17:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.810+0000 7f6aafde6740 10 client.18459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:17:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.810+0000 7f6aafde6740 20 client.18459 populate_metadata read hostname 'smithi007' 2024-06-10T01:17:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.810+0000 7f6aafde6740 1 --2- 172.21.15.7:0/3074819249 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562082384cc0 0x5620823870b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.810+0000 7f6aafde6740 4 client.18459 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:17:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.810+0000 7f6aafde6740 1 -- 172.21.15.7:0/3074819249 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620822e5f10 con 0x562082384cc0 2024-06-10T01:17:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.811+0000 7f6aa659c640 1 --2- 172.21.15.7:0/3074819249 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562082384cc0 0x5620823870b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.811+0000 7f6aa659c640 1 --2- 172.21.15.7:0/3074819249 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562082384cc0 0x5620823870b0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae716fea79121969 server_cookie=2a46ef6305018a48 in_seq=0 out_seq=0 2024-06-10T01:17:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.811+0000 7f6a8aff5640 10 client.18459 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.812+0000 7f6a8aff5640 1 -- 172.21.15.7:0/3074819249 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5620822e5f10 con 0x562082384cc0 2024-06-10T01:17:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.812+0000 7f6a8aff5640 10 client.18459 handle_command_reply: tid=0 2024-06-10T01:17:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.812+0000 7f6aafde6740 1 client.18459 shutdown 2024-06-10T01:17:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.812+0000 7f6aafde6740 1 -- 172.21.15.7:0/3074819249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56208236f010 msgr2=0x5620823792c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.812+0000 7f6aafde6740 1 --2- 172.21.15.7:0/3074819249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56208236f010 0x5620823792c0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009900 tx=0x7f6a98043d90 comp rx=0 tx=0).stop 2024-06-10T01:17:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.812+0000 7f6aafde6740 1 -- 172.21.15.7:0/3074819249 shutdown_connections 2024-06-10T01:17:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.812+0000 7f6aafde6740 1 --2- 172.21.15.7:0/3074819249 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562082384cc0 0x5620823870b0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.812+0000 7f6aafde6740 1 --2- 172.21.15.7:0/3074819249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562082374b70 0x5620823807d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.812+0000 7f6aafde6740 1 --2- 172.21.15.7:0/3074819249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56208236fa00 0x562082379800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.812+0000 7f6aafde6740 1 --2- 172.21.15.7:0/3074819249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56208236f010 0x5620823792c0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.812+0000 7f6a8aff5640 0 client.18459 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.812+0000 7f6aafde6740 1 -- 172.21.15.7:0/3074819249 >> 172.21.15.7:0/3074819249 conn(0x56208236a680 msgr2=0x5620823697e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.812+0000 7f6aafde6740 1 -- 172.21.15.7:0/3074819249 shutdown_connections 2024-06-10T01:17:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.812+0000 7f6aafde6740 1 -- 172.21.15.7:0/3074819249 wait complete. 2024-06-10T01:17:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.812+0000 7f6aafde6740 20 client.18459 trim_cache size 0 max 16384 2024-06-10T01:17:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.828+0000 7f6aafde6740 1 Processor -- start 2024-06-10T01:17:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.828+0000 7f6aafde6740 1 -- start start 2024-06-10T01:17:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.829+0000 7f6aafde6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56208236f010 0x56208236dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.829+0000 7f6aafde6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56208236fa00 0x562082380790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.829+0000 7f6aafde6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562082374b70 0x5620823785b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.829+0000 7f6aafde6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa81aac10 con 0x56208236f010 2024-06-10T01:17:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.829+0000 7f6aafde6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa81a84e0 con 0x56208236fa00 2024-06-10T01:17:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.829+0000 7f6aafde6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa81a6330 con 0x562082374b70 2024-06-10T01:17:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.829+0000 7f6aa6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56208236f010 0x56208236dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.829+0000 7f6aa6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56208236f010 0x56208236dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41020/0 (socket says 172.21.15.7:41020) 2024-06-10T01:17:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.829+0000 7f6aa6d9d640 1 -- 172.21.15.7:0/568806656 learned_addr learned my addr 172.21.15.7:0/568806656 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.829+0000 7f6aa759e640 1 --2- 172.21.15.7:0/568806656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562082374b70 0x5620823785b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.829+0000 7f6aa659c640 1 --2- 172.21.15.7:0/568806656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56208236fa00 0x562082380790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.829+0000 7f6aa6d9d640 1 -- 172.21.15.7:0/568806656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562082374b70 msgr2=0x5620823785b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.829+0000 7f6aa6d9d640 1 --2- 172.21.15.7:0/568806656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562082374b70 0x5620823785b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.829+0000 7f6aa6d9d640 1 -- 172.21.15.7:0/568806656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56208236fa00 msgr2=0x562082380790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.829+0000 7f6aa6d9d640 1 --2- 172.21.15.7:0/568806656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56208236fa00 0x562082380790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.829+0000 7f6aa6d9d640 1 -- 172.21.15.7:0/568806656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620823773c0 con 0x56208236f010 2024-06-10T01:17:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.829+0000 7f6aa659c640 1 --2- 172.21.15.7:0/568806656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56208236fa00 0x562082380790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:17:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.829+0000 7f6aa6d9d640 1 --2- 172.21.15.7:0/568806656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56208236f010 0x56208236dc40 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f6a98006010 tx=0x7f6a98005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b35ae48f5c7e91 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.829+0000 7f6a8cff9640 1 -- 172.21.15.7:0/568806656 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9805a070 con 0x56208236f010 2024-06-10T01:17:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.829+0000 7f6a8cff9640 1 -- 172.21.15.7:0/568806656 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a9804c940 con 0x56208236f010 2024-06-10T01:17:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.829+0000 7f6a8cff9640 1 -- 172.21.15.7:0/568806656 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98035050 con 0x56208236f010 2024-06-10T01:17:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.830+0000 7f6aafde6740 1 -- 172.21.15.7:0/568806656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56208236f010 msgr2=0x56208236dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.830+0000 7f6aafde6740 1 --2- 172.21.15.7:0/568806656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56208236f010 0x56208236dc40 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f6a98006010 tx=0x7f6a98005a60 comp rx=0 tx=0).stop 2024-06-10T01:17:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.830+0000 7f6aafde6740 1 -- 172.21.15.7:0/568806656 shutdown_connections 2024-06-10T01:17:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.830+0000 7f6aafde6740 1 --2- 172.21.15.7:0/568806656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562082374b70 0x5620823785b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.830+0000 7f6aafde6740 1 --2- 172.21.15.7:0/568806656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56208236fa00 0x562082380790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.830+0000 7f6aafde6740 1 --2- 172.21.15.7:0/568806656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56208236f010 0x56208236dc40 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.830+0000 7f6aafde6740 1 -- 172.21.15.7:0/568806656 >> 172.21.15.7:0/568806656 conn(0x56208236a680 msgr2=0x56208236d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.830+0000 7f6aafde6740 1 -- 172.21.15.7:0/568806656 shutdown_connections 2024-06-10T01:17:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.830+0000 7f6aafde6740 1 -- 172.21.15.7:0/568806656 wait complete. 2024-06-10T01:17:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.830+0000 7f6aafde6740 1 -- start start 2024-06-10T01:17:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.830+0000 7f6aafde6740 1 Processor -- start 2024-06-10T01:17:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.831+0000 7f6aafde6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56208236f010 0x562082380600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.831+0000 7f6aafde6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56208236fa00 0x56208237d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.831+0000 7f6aafde6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562082374b70 0x56208237d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.831+0000 7f6aafde6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa81a69a0 con 0x562082374b70 2024-06-10T01:17:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.831+0000 7f6aafde6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa81a7e10 con 0x56208236fa00 2024-06-10T01:17:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.831+0000 7f6aafde6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa81a7070 con 0x56208236f010 2024-06-10T01:17:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.831+0000 7f6aafde6740 10 client.0 _ll_register_callbacks cb 0x7f6a98009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:17:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.831+0000 7f6aa759e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562082374b70 0x56208237d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.831+0000 7f6aa759e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562082374b70 0x56208237d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41028/0 (socket says 172.21.15.7:41028) 2024-06-10T01:17:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.831+0000 7f6aa659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56208236fa00 0x56208237d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.831+0000 7f6aa759e640 1 -- 172.21.15.7:0/514278533 learned_addr learned my addr 172.21.15.7:0/514278533 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:17:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.831+0000 7f6aa6d9d640 1 --2- 172.21.15.7:0/514278533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56208236f010 0x562082380600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.831+0000 7f6aa759e640 1 -- 172.21.15.7:0/514278533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56208236f010 msgr2=0x562082380600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.831+0000 7f6aa759e640 1 --2- 172.21.15.7:0/514278533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56208236f010 0x562082380600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.831+0000 7f6aa759e640 1 -- 172.21.15.7:0/514278533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56208236fa00 msgr2=0x56208237d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:17:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.831+0000 7f6aa759e640 1 --2- 172.21.15.7:0/514278533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56208236fa00 0x56208237d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.831+0000 7f6aa759e640 1 -- 172.21.15.7:0/514278533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56208237dc80 con 0x562082374b70 2024-06-10T01:17:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.831+0000 7f6aa6d9d640 1 --2- 172.21.15.7:0/514278533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56208236f010 0x562082380600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:17:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.831+0000 7f6aa759e640 1 --2- 172.21.15.7:0/514278533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562082374b70 0x56208237d5e0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0010940 tx=0x7f6aa0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d4da2735292c1a8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:17:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.831+0000 7f6a8aff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:17:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.831+0000 7f6a8aff5640 1 -- 172.21.15.7:0/514278533 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0003e20 con 0x562082374b70 2024-06-10T01:17:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.831+0000 7f6a8aff5640 1 -- 172.21.15.7:0/514278533 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f6aa00045d0 con 0x562082374b70 2024-06-10T01:17:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.831+0000 7f6aafde6740 1 -- 172.21.15.7:0/514278533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a9804c940 con 0x562082374b70 2024-06-10T01:17:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.832+0000 7f6a8aff5640 1 -- 172.21.15.7:0/514278533 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f6aa0031050 con 0x562082374b70 2024-06-10T01:17:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.832+0000 7f6aafde6740 10 client.18465 fetch_fsmap learned FSMap version 40 2024-06-10T01:17:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.832+0000 7f6aafde6740 1 -- 172.21.15.7:0/514278533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5620823eea40 con 0x562082374b70 2024-06-10T01:17:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.832+0000 7f6a8aff5640 1 -- 172.21.15.7:0/514278533 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a9804c940 con 0x562082374b70 2024-06-10T01:17:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.832+0000 7f6a8aff5640 1 -- 172.21.15.7:0/514278533 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f6aa000f3c0 con 0x562082374b70 2024-06-10T01:17:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.832+0000 7f6aafde6740 10 client.18465 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:17:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.832+0000 7f6aafde6740 10 client.18465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:17:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.832+0000 7f6aafde6740 20 client.18465 populate_metadata read hostname 'smithi007' 2024-06-10T01:17:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.832+0000 7f6aafde6740 1 --2- 172.21.15.7:0/514278533 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5620823ef4a0 0x5620823f18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:17:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.832+0000 7f6aafde6740 4 client.18465 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:17:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.832+0000 7f6aafde6740 1 -- 172.21.15.7:0/514278533 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5620822dfc20 con 0x5620823ef4a0 2024-06-10T01:17:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.832+0000 7f6aa6d9d640 1 --2- 172.21.15.7:0/514278533 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5620823ef4a0 0x5620823f18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:17:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.833+0000 7f6a8aff5640 10 client.18465 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.833+0000 7f6aa6d9d640 1 --2- 172.21.15.7:0/514278533 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5620823ef4a0 0x5620823f18c0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d891b5cf2a448e5 server_cookie=3f65f9cf024fe72a in_seq=0 out_seq=0 2024-06-10T01:17:57.832 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:57 smithi078 ceph-mon[36683]: pgmap v790: 129 pgs: 129 active+clean; 151 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-06-10T01:17:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.863+0000 7f6a8aff5640 1 -- 172.21.15.7:0/514278533 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5620822dfc20 con 0x5620823ef4a0 2024-06-10T01:17:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.863+0000 7f6a8aff5640 10 client.18465 handle_command_reply: tid=0 2024-06-10T01:17:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.863+0000 7f6aafde6740 1 client.18465 shutdown 2024-06-10T01:17:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.863+0000 7f6aafde6740 1 -- 172.21.15.7:0/514278533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562082374b70 msgr2=0x56208237d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.863+0000 7f6aafde6740 1 --2- 172.21.15.7:0/514278533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562082374b70 0x56208237d5e0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0010940 tx=0x7f6aa0004a40 comp rx=0 tx=0).stop 2024-06-10T01:17:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.863+0000 7f6aafde6740 1 -- 172.21.15.7:0/514278533 shutdown_connections 2024-06-10T01:17:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.863+0000 7f6aafde6740 1 --2- 172.21.15.7:0/514278533 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5620823ef4a0 0x5620823f18c0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.863+0000 7f6aafde6740 1 --2- 172.21.15.7:0/514278533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562082374b70 0x56208237d5e0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.863+0000 7f6aafde6740 1 --2- 172.21.15.7:0/514278533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56208236fa00 0x56208237d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.863+0000 7f6a8aff5640 0 client.18465 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:17:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.863+0000 7f6aafde6740 1 --2- 172.21.15.7:0/514278533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56208236f010 0x562082380600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.863+0000 7f6aafde6740 1 -- 172.21.15.7:0/514278533 >> 172.21.15.7:0/514278533 conn(0x56208236a680 msgr2=0x56208236ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.863+0000 7f6aafde6740 1 -- 172.21.15.7:0/514278533 shutdown_connections 2024-06-10T01:17:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.863+0000 7f6aafde6740 1 -- 172.21.15.7:0/514278533 wait complete. 2024-06-10T01:17:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.863+0000 7f6aafde6740 20 client.18465 trim_cache size 0 max 16384 2024-06-10T01:17:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:17:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:17:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:17:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.864+0000 7f6aad0d4640 1 -- 172.21.15.7:0/50171737 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6a900a2b80 msgr2=0x7f6a900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.864+0000 7f6aad0d4640 1 --2- 172.21.15.7:0/50171737 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6a900a2b80 0x7f6a900a5040 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0005c40 tx=0x7f6aa0009500 comp rx=0 tx=0).stop 2024-06-10T01:17:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.864+0000 7f6aad0d4640 1 -- 172.21.15.7:0/50171737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6aa8102c20 msgr2=0x7f6aa8198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:17:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.864+0000 7f6aad0d4640 1 --2- 172.21.15.7:0/50171737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6aa8102c20 0x7f6aa8198280 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00e9f0 tx=0x7f6a9c00eeb0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.864+0000 7f6aad0d4640 1 -- 172.21.15.7:0/50171737 shutdown_connections 2024-06-10T01:17:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.864+0000 7f6aad0d4640 1 --2- 172.21.15.7:0/50171737 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6a900a2b80 0x7f6a900a5040 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.864+0000 7f6aad0d4640 1 --2- 172.21.15.7:0/50171737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6aa81035e0 0x7f6aa819f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.864+0000 7f6aad0d4640 1 --2- 172.21.15.7:0/50171737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6aa8102c20 0x7f6aa8198280 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.864+0000 7f6aad0d4640 1 --2- 172.21.15.7:0/50171737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6aa8101a30 0x7f6aa8197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:17:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.864+0000 7f6aad0d4640 1 -- 172.21.15.7:0/50171737 >> 172.21.15.7:0/50171737 conn(0x7f6aa80fcfc0 msgr2=0x7f6aa8105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:17:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.864+0000 7f6aad0d4640 1 -- 172.21.15.7:0/50171737 shutdown_connections 2024-06-10T01:17:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:17:57.864+0000 7f6aad0d4640 1 -- 172.21.15.7:0/50171737 wait complete. 2024-06-10T01:17:58.754 INFO:tasks.workunit.client.0.smithi007.stdout: 1 881536 7.38 MB/sec execute 519 sec latency 14556.698 ms 2024-06-10T01:17:59.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:17:59 smithi175 ceph-mon[36636]: pgmap v791: 129 pgs: 129 active+clean; 150 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 1.5 MiB/s wr, 120 op/s 2024-06-10T01:17:59.754 INFO:tasks.workunit.client.0.smithi007.stdout: 1 881536 7.36 MB/sec execute 520 sec latency 15556.820 ms 2024-06-10T01:17:59.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:17:59 smithi007 ceph-mon[32398]: pgmap v791: 129 pgs: 129 active+clean; 150 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 1.5 MiB/s wr, 120 op/s 2024-06-10T01:17:59.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:17:59 smithi078 ceph-mon[36683]: pgmap v791: 129 pgs: 129 active+clean; 150 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 1.5 MiB/s wr, 120 op/s 2024-06-10T01:18:00.754 INFO:tasks.workunit.client.0.smithi007.stdout: 1 881536 7.35 MB/sec execute 521 sec latency 16556.892 ms 2024-06-10T01:18:01.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:01 smithi175 ceph-mon[36636]: pgmap v792: 129 pgs: 129 active+clean; 148 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.7 MiB/s wr, 115 op/s 2024-06-10T01:18:01.754 INFO:tasks.workunit.client.0.smithi007.stdout: 1 881536 7.33 MB/sec execute 522 sec latency 17557.006 ms 2024-06-10T01:18:01.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:01 smithi007 ceph-mon[32398]: pgmap v792: 129 pgs: 129 active+clean; 148 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.7 MiB/s wr, 115 op/s 2024-06-10T01:18:01.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:01 smithi078 ceph-mon[36683]: pgmap v792: 129 pgs: 129 active+clean; 148 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.7 MiB/s wr, 115 op/s 2024-06-10T01:18:02.136 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:18:02.436 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:18:02.754 INFO:tasks.workunit.client.0.smithi007.stdout: 1 881536 7.32 MB/sec execute 523 sec latency 18557.084 ms 2024-06-10T01:18:03.512 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:03 smithi007 ceph-mon[32398]: pgmap v793: 129 pgs: 129 active+clean; 148 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 1.6 MiB/s wr, 76 op/s 2024-06-10T01:18:03.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:03 smithi175 ceph-mon[36636]: pgmap v793: 129 pgs: 129 active+clean; 148 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 1.6 MiB/s wr, 76 op/s 2024-06-10T01:18:03.754 INFO:tasks.workunit.client.0.smithi007.stdout: 1 881536 7.31 MB/sec execute 524 sec latency 19557.166 ms 2024-06-10T01:18:03.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:03 smithi078 ceph-mon[36683]: pgmap v793: 129 pgs: 129 active+clean; 148 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 1.6 MiB/s wr, 76 op/s 2024-06-10T01:18:04.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.178+0000 7f5d88842700 1 -- 172.21.15.7:0/4197172182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d841030f0 msgr2=0x7f5d84107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:04.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.178+0000 7f5d88842700 1 --2- 172.21.15.7:0/4197172182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d841030f0 0x7f5d84107110 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f5d780099c0 tx=0x7f5d78009cd0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.178+0000 7f5d88842700 1 -- 172.21.15.7:0/4197172182 shutdown_connections 2024-06-10T01:18:04.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.178+0000 7f5d88842700 1 --2- 172.21.15.7:0/4197172182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d84107840 0x7f5d8410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.178+0000 7f5d88842700 1 --2- 172.21.15.7:0/4197172182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d84102740 0x7f5d84102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.178+0000 7f5d88842700 1 --2- 172.21.15.7:0/4197172182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d841030f0 0x7f5d84107110 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.178+0000 7f5d88842700 1 -- 172.21.15.7:0/4197172182 >> 172.21.15.7:0/4197172182 conn(0x7f5d840795f0 msgr2=0x7f5d84079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:04.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.178+0000 7f5d88842700 1 -- 172.21.15.7:0/4197172182 shutdown_connections 2024-06-10T01:18:04.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.178+0000 7f5d88842700 1 -- 172.21.15.7:0/4197172182 wait complete. 2024-06-10T01:18:04.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.178+0000 7f5d88842700 1 Processor -- start 2024-06-10T01:18:04.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.179+0000 7f5d88842700 1 -- start start 2024-06-10T01:18:04.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.179+0000 7f5d88842700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d84102740 0x7f5d8419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:04.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.179+0000 7f5d88842700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d841030f0 0x7f5d8419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:04.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.179+0000 7f5d88842700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d84107840 0x7f5d841989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:04.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.179+0000 7f5d88842700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5d84112750 con 0x7f5d84107840 2024-06-10T01:18:04.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.179+0000 7f5d88842700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5d841125d0 con 0x7f5d841030f0 2024-06-10T01:18:04.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.179+0000 7f5d88842700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5d841128d0 con 0x7f5d84102740 2024-06-10T01:18:04.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.179+0000 7f5d82d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d84107840 0x7f5d841989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:04.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.179+0000 7f5d82d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d84107840 0x7f5d841989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45414/0 (socket says 172.21.15.7:45414) 2024-06-10T01:18:04.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.179+0000 7f5d82d9d700 1 -- 172.21.15.7:0/4006601799 learned_addr learned my addr 172.21.15.7:0/4006601799 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:04.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.179+0000 7f5d81d9b700 1 --2- 172.21.15.7:0/4006601799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d841030f0 0x7f5d8419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:04.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.179+0000 7f5d8259c700 1 --2- 172.21.15.7:0/4006601799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d84102740 0x7f5d8419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:04.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.179+0000 7f5d82d9d700 1 -- 172.21.15.7:0/4006601799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d84102740 msgr2=0x7f5d8419e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:04.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.179+0000 7f5d82d9d700 1 --2- 172.21.15.7:0/4006601799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d84102740 0x7f5d8419e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.179+0000 7f5d82d9d700 1 -- 172.21.15.7:0/4006601799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d841030f0 msgr2=0x7f5d8419ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:04.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.179+0000 7f5d82d9d700 1 --2- 172.21.15.7:0/4006601799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d841030f0 0x7f5d8419ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.179+0000 7f5d82d9d700 1 -- 172.21.15.7:0/4006601799 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d84199130 con 0x7f5d84107840 2024-06-10T01:18:04.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.179+0000 7f5d81d9b700 1 --2- 172.21.15.7:0/4006601799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d841030f0 0x7f5d8419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:18:04.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.180+0000 7f5d82d9d700 1 --2- 172.21.15.7:0/4006601799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d84107840 0x7f5d841989d0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00ef30 tx=0x7f5d7c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:04.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.180+0000 7f5d737fe700 1 -- 172.21.15.7:0/4006601799 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0092e0 con 0x7f5d84107840 2024-06-10T01:18:04.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.180+0000 7f5d737fe700 1 -- 172.21.15.7:0/4006601799 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d7c004bb0 con 0x7f5d84107840 2024-06-10T01:18:04.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.180+0000 7f5d88842700 1 -- 172.21.15.7:0/4006601799 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d841993c0 con 0x7f5d84107840 2024-06-10T01:18:04.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.180+0000 7f5d737fe700 1 -- 172.21.15.7:0/4006601799 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c004280 con 0x7f5d84107840 2024-06-10T01:18:04.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.180+0000 7f5d88842700 1 -- 172.21.15.7:0/4006601799 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d841a5690 con 0x7f5d84107840 2024-06-10T01:18:04.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.182+0000 7f5d737fe700 1 -- 172.21.15.7:0/4006601799 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f5d7c01b030 con 0x7f5d84107840 2024-06-10T01:18:04.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.183+0000 7f5d737fe700 1 --2- 172.21.15.7:0/4006601799 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5d6c0a2ca0 0x7f5d6c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:04.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.183+0000 7f5d737fe700 1 -- 172.21.15.7:0/4006601799 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f5d7c0cedb0 con 0x7f5d84107840 2024-06-10T01:18:04.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.183+0000 7f5d8259c700 1 --2- 172.21.15.7:0/4006601799 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5d6c0a2ca0 0x7f5d6c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:04.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.183+0000 7f5d8259c700 1 --2- 172.21.15.7:0/4006601799 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5d6c0a2ca0 0x7f5d6c0a5160 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5d841073f0 tx=0x7f5d740073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:04.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.183+0000 7f5d930c5b80 1 Processor -- start 2024-06-10T01:18:04.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.184+0000 7f5d930c5b80 1 -- start start 2024-06-10T01:18:04.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.184+0000 7f5d930c5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8bf99f60 0x558c8bf9a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:04.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.184+0000 7f5d930c5b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8bf9a880 0x558c8bf94260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:04.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.184+0000 7f5d930c5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8bf947a0 0x558c8bf94bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:04.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.184+0000 7f5d930c5b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558c8be86f60 con 0x558c8bf9a880 2024-06-10T01:18:04.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.184+0000 7f5d930c5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558c8bee5850 con 0x558c8bf947a0 2024-06-10T01:18:04.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.184+0000 7f5d930c5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558c8be4d460 con 0x558c8bf99f60 2024-06-10T01:18:04.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.184+0000 7f5d81d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8bf9a880 0x558c8bf94260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:04.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.184+0000 7f5d81d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8bf9a880 0x558c8bf94260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45422/0 (socket says 172.21.15.7:45422) 2024-06-10T01:18:04.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.184+0000 7f5d81d9b700 1 -- 172.21.15.7:0/3655428455 learned_addr learned my addr 172.21.15.7:0/3655428455 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:04.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.184+0000 7f5d82d9d700 1 --2- 172.21.15.7:0/3655428455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8bf947a0 0x558c8bf94bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:04.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.184+0000 7f5d8259c700 1 --2- 172.21.15.7:0/3655428455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8bf99f60 0x558c8bf9a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:04.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.184+0000 7f5d81d9b700 1 -- 172.21.15.7:0/3655428455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8bf99f60 msgr2=0x558c8bf9a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:04.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.184+0000 7f5d81d9b700 1 --2- 172.21.15.7:0/3655428455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8bf99f60 0x558c8bf9a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.184+0000 7f5d81d9b700 1 -- 172.21.15.7:0/3655428455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8bf947a0 msgr2=0x558c8bf94bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:04.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.184+0000 7f5d81d9b700 1 --2- 172.21.15.7:0/3655428455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8bf947a0 0x558c8bf94bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.184+0000 7f5d81d9b700 1 -- 172.21.15.7:0/3655428455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c8be348d0 con 0x558c8bf9a880 2024-06-10T01:18:04.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.185+0000 7f5d81d9b700 1 --2- 172.21.15.7:0/3655428455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8bf9a880 0x558c8bf94260 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f5d780099c0 tx=0x7f5d78003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4388528e47a7d2ea server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:04.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.185+0000 7f5d70ff9700 1 -- 172.21.15.7:0/3655428455 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d78028070 con 0x558c8bf9a880 2024-06-10T01:18:04.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.185+0000 7f5d70ff9700 1 -- 172.21.15.7:0/3655428455 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d78003d40 con 0x558c8bf9a880 2024-06-10T01:18:04.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.185+0000 7f5d70ff9700 1 -- 172.21.15.7:0/3655428455 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d780055d0 con 0x558c8bf9a880 2024-06-10T01:18:04.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.186+0000 7f5d930c5b80 1 -- 172.21.15.7:0/3655428455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8bf9a880 msgr2=0x558c8bf94260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:04.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.186+0000 7f5d930c5b80 1 --2- 172.21.15.7:0/3655428455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8bf9a880 0x558c8bf94260 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f5d780099c0 tx=0x7f5d78003680 comp rx=0 tx=0).stop 2024-06-10T01:18:04.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.186+0000 7f5d930c5b80 1 -- 172.21.15.7:0/3655428455 shutdown_connections 2024-06-10T01:18:04.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.186+0000 7f5d930c5b80 1 --2- 172.21.15.7:0/3655428455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8bf947a0 0x558c8bf94bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.186+0000 7f5d930c5b80 1 --2- 172.21.15.7:0/3655428455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8bf99f60 0x558c8bf9a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.186+0000 7f5d930c5b80 1 --2- 172.21.15.7:0/3655428455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8bf9a880 0x558c8bf94260 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.186+0000 7f5d930c5b80 1 -- 172.21.15.7:0/3655428455 >> 172.21.15.7:0/3655428455 conn(0x558c8bec1080 msgr2=0x558c8bf8f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:04.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.186+0000 7f5d930c5b80 1 -- 172.21.15.7:0/3655428455 shutdown_connections 2024-06-10T01:18:04.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.186+0000 7f5d930c5b80 1 -- 172.21.15.7:0/3655428455 wait complete. 2024-06-10T01:18:04.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.186+0000 7f5d930c5b80 1 -- start start 2024-06-10T01:18:04.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.186+0000 7f5d930c5b80 1 Processor -- start 2024-06-10T01:18:04.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.187+0000 7f5d930c5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8bf947a0 0x558c8bfa05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:04.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.187+0000 7f5d930c5b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8bf99f60 0x558c8bf9d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:04.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.187+0000 7f5d930c5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8bf9a880 0x558c8bf9de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:04.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.187+0000 7f5d930c5b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558c8be85fc0 con 0x558c8bf99f60 2024-06-10T01:18:04.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.187+0000 7f5d930c5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558c8bee1810 con 0x558c8bf9a880 2024-06-10T01:18:04.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.187+0000 7f5d930c5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558c8bfa1200 con 0x558c8bf947a0 2024-06-10T01:18:04.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.187+0000 7f5d930c5b80 10 client.0 _ll_register_callbacks cb 0x558c8be7b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:18:04.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.187+0000 7f5d81d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8bf99f60 0x558c8bf9d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:04.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.187+0000 7f5d81d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8bf99f60 0x558c8bf9d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45432/0 (socket says 172.21.15.7:45432) 2024-06-10T01:18:04.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.187+0000 7f5d81d9b700 1 -- 172.21.15.7:0/4225250505 learned_addr learned my addr 172.21.15.7:0/4225250505 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:04.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.187+0000 7f5d8259c700 1 --2- 172.21.15.7:0/4225250505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8bf947a0 0x558c8bfa05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:04.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.187+0000 7f5d82d9d700 1 --2- 172.21.15.7:0/4225250505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8bf9a880 0x558c8bf9de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:04.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.187+0000 7f5d81d9b700 1 -- 172.21.15.7:0/4225250505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8bf947a0 msgr2=0x558c8bfa05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:04.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.187+0000 7f5d81d9b700 1 --2- 172.21.15.7:0/4225250505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8bf947a0 0x558c8bfa05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.187+0000 7f5d81d9b700 1 -- 172.21.15.7:0/4225250505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8bf9a880 msgr2=0x558c8bf9de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:04.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.187+0000 7f5d81d9b700 1 --2- 172.21.15.7:0/4225250505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8bf9a880 0x558c8bf9de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.187+0000 7f5d81d9b700 1 -- 172.21.15.7:0/4225250505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c8beab750 con 0x558c8bf99f60 2024-06-10T01:18:04.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.187+0000 7f5d81d9b700 1 --2- 172.21.15.7:0/4225250505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8bf99f60 0x558c8bf9d8c0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f5d78005a80 tx=0x7f5d7800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f8905a55bb76a8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:04.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.187+0000 7f5d69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:18:04.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.187+0000 7f5d69ffb700 1 -- 172.21.15.7:0/4225250505 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d78028030 con 0x558c8bf99f60 2024-06-10T01:18:04.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.187+0000 7f5d69ffb700 1 -- 172.21.15.7:0/4225250505 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f5d78003b30 con 0x558c8bf99f60 2024-06-10T01:18:04.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.187+0000 7f5d930c5b80 1 -- 172.21.15.7:0/4225250505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d78003d40 con 0x558c8bf99f60 2024-06-10T01:18:04.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.188+0000 7f5d69ffb700 1 -- 172.21.15.7:0/4225250505 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5d7803a050 con 0x558c8bf99f60 2024-06-10T01:18:04.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.188+0000 7f5d930c5b80 10 client.18477 fetch_fsmap learned FSMap version 40 2024-06-10T01:18:04.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.188+0000 7f5d930c5b80 1 -- 172.21.15.7:0/4225250505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x558c8bfab960 con 0x558c8bf99f60 2024-06-10T01:18:04.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.188+0000 7f5d69ffb700 1 -- 172.21.15.7:0/4225250505 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d78003d40 con 0x558c8bf99f60 2024-06-10T01:18:04.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.188+0000 7f5d69ffb700 1 -- 172.21.15.7:0/4225250505 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f5d7800fc90 con 0x558c8bf99f60 2024-06-10T01:18:04.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.188+0000 7f5d930c5b80 10 client.18477 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:18:04.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.188+0000 7f5d930c5b80 10 client.18477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:18:04.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.188+0000 7f5d930c5b80 20 client.18477 populate_metadata read hostname 'smithi007' 2024-06-10T01:18:04.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.188+0000 7f5d930c5b80 1 --2- 172.21.15.7:0/4225250505 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c8bfade30 0x558c8bfb0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:04.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.188+0000 7f5d930c5b80 4 client.18477 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:18:04.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.188+0000 7f5d930c5b80 1 -- 172.21.15.7:0/4225250505 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c8bef2d50 con 0x558c8bfade30 2024-06-10T01:18:04.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.188+0000 7f5d8259c700 1 --2- 172.21.15.7:0/4225250505 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c8bfade30 0x558c8bfb0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:04.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.189+0000 7f5d8259c700 1 --2- 172.21.15.7:0/4225250505 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c8bfade30 0x558c8bfb0220 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf9adcf141d6d7c0 server_cookie=84dff60f83a962d2 in_seq=0 out_seq=0 2024-06-10T01:18:04.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.189+0000 7f5d69ffb700 10 client.18477 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:04.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.189+0000 7f5d69ffb700 1 -- 172.21.15.7:0/4225250505 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558c8bef2d50 con 0x558c8bfade30 2024-06-10T01:18:04.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.189+0000 7f5d69ffb700 10 client.18477 handle_command_reply: tid=0 2024-06-10T01:18:04.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.190+0000 7f5d930c5b80 1 client.18477 shutdown 2024-06-10T01:18:04.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.190+0000 7f5d930c5b80 1 -- 172.21.15.7:0/4225250505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8bf99f60 msgr2=0x558c8bf9d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:04.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.190+0000 7f5d930c5b80 1 --2- 172.21.15.7:0/4225250505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8bf99f60 0x558c8bf9d8c0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f5d78005a80 tx=0x7f5d7800b8a0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.190+0000 7f5d930c5b80 1 -- 172.21.15.7:0/4225250505 shutdown_connections 2024-06-10T01:18:04.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.190+0000 7f5d930c5b80 1 --2- 172.21.15.7:0/4225250505 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c8bfade30 0x558c8bfb0220 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.190+0000 7f5d930c5b80 1 --2- 172.21.15.7:0/4225250505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8bf9a880 0x558c8bf9de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.190+0000 7f5d69ffb700 0 client.18477 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:04.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.190+0000 7f5d930c5b80 1 --2- 172.21.15.7:0/4225250505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8bf947a0 0x558c8bfa05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.190+0000 7f5d930c5b80 1 --2- 172.21.15.7:0/4225250505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8bf99f60 0x558c8bf9d8c0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.190+0000 7f5d930c5b80 1 -- 172.21.15.7:0/4225250505 >> 172.21.15.7:0/4225250505 conn(0x558c8bec1080 msgr2=0x558c8bf8e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:04.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.190+0000 7f5d930c5b80 1 -- 172.21.15.7:0/4225250505 shutdown_connections 2024-06-10T01:18:04.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.190+0000 7f5d930c5b80 1 -- 172.21.15.7:0/4225250505 wait complete. 2024-06-10T01:18:04.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.190+0000 7f5d930c5b80 20 client.18477 trim_cache size 0 max 16384 2024-06-10T01:18:04.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.211+0000 7f5d930c5b80 1 Processor -- start 2024-06-10T01:18:04.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.211+0000 7f5d930c5b80 1 -- start start 2024-06-10T01:18:04.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.211+0000 7f5d930c5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8bf947a0 0x558c8bfb3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:04.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.211+0000 7f5d930c5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8bf99f60 0x558c8bfb3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:04.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.211+0000 7f5d930c5b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8bf9a880 0x558c8bf8ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:04.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.211+0000 7f5d930c5b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5d841b0ad0 con 0x558c8bf9a880 2024-06-10T01:18:04.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.211+0000 7f5d930c5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5d841ae360 con 0x558c8bf99f60 2024-06-10T01:18:04.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.211+0000 7f5d930c5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5d841ac1b0 con 0x558c8bf947a0 2024-06-10T01:18:04.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.211+0000 7f5d82d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8bf9a880 0x558c8bf8ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:04.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.211+0000 7f5d82d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8bf9a880 0x558c8bf8ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45448/0 (socket says 172.21.15.7:45448) 2024-06-10T01:18:04.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.211+0000 7f5d82d9d700 1 -- 172.21.15.7:0/1876027357 learned_addr learned my addr 172.21.15.7:0/1876027357 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:04.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.211+0000 7f5d81d9b700 1 --2- 172.21.15.7:0/1876027357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8bf99f60 0x558c8bfb3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:04.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.211+0000 7f5d8259c700 1 --2- 172.21.15.7:0/1876027357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8bf947a0 0x558c8bfb3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:04.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.212+0000 7f5d82d9d700 1 -- 172.21.15.7:0/1876027357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8bf947a0 msgr2=0x558c8bfb3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:04.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.212+0000 7f5d82d9d700 1 --2- 172.21.15.7:0/1876027357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8bf947a0 0x558c8bfb3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.212+0000 7f5d82d9d700 1 -- 172.21.15.7:0/1876027357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8bf99f60 msgr2=0x558c8bfb3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:04.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.212+0000 7f5d82d9d700 1 --2- 172.21.15.7:0/1876027357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8bf99f60 0x558c8bfb3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.212+0000 7f5d82d9d700 1 -- 172.21.15.7:0/1876027357 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c8bf90800 con 0x558c8bf9a880 2024-06-10T01:18:04.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.212+0000 7f5d81d9b700 1 --2- 172.21.15.7:0/1876027357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8bf99f60 0x558c8bfb3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:18:04.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.212+0000 7f5d82d9d700 1 --2- 172.21.15.7:0/1876027357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8bf9a880 0x558c8bf8ee80 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0090a0 tx=0x7f5d7c009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf45b3f1953dfd68 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:04.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.212+0000 7f5d6bfff700 1 -- 172.21.15.7:0/1876027357 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c00ce80 con 0x558c8bf9a880 2024-06-10T01:18:04.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.212+0000 7f5d6bfff700 1 -- 172.21.15.7:0/1876027357 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d7c01b320 con 0x558c8bf9a880 2024-06-10T01:18:04.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.212+0000 7f5d6bfff700 1 -- 172.21.15.7:0/1876027357 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0995c0 con 0x558c8bf9a880 2024-06-10T01:18:04.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.213+0000 7f5d930c5b80 1 -- 172.21.15.7:0/1876027357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8bf9a880 msgr2=0x558c8bf8ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:04.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.213+0000 7f5d930c5b80 1 --2- 172.21.15.7:0/1876027357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8bf9a880 0x558c8bf8ee80 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0090a0 tx=0x7f5d7c009ed0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.213+0000 7f5d930c5b80 1 -- 172.21.15.7:0/1876027357 shutdown_connections 2024-06-10T01:18:04.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.213+0000 7f5d930c5b80 1 --2- 172.21.15.7:0/1876027357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8bf9a880 0x558c8bf8ee80 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.213+0000 7f5d930c5b80 1 --2- 172.21.15.7:0/1876027357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8bf947a0 0x558c8bfb3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.213+0000 7f5d930c5b80 1 --2- 172.21.15.7:0/1876027357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8bf99f60 0x558c8bfb3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.213+0000 7f5d930c5b80 1 -- 172.21.15.7:0/1876027357 >> 172.21.15.7:0/1876027357 conn(0x558c8bec1080 msgr2=0x558c8bf9ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:04.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.213+0000 7f5d930c5b80 1 -- 172.21.15.7:0/1876027357 shutdown_connections 2024-06-10T01:18:04.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.213+0000 7f5d930c5b80 1 -- 172.21.15.7:0/1876027357 wait complete. 2024-06-10T01:18:04.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.213+0000 7f5d930c5b80 1 -- start start 2024-06-10T01:18:04.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.213+0000 7f5d930c5b80 1 Processor -- start 2024-06-10T01:18:04.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.214+0000 7f5d930c5b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8bf947a0 0x558c8c010900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:04.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.214+0000 7f5d930c5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8bf99f60 0x558c8c010e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:04.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.214+0000 7f5d930c5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8bf9a880 0x558c8c01b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:04.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.214+0000 7f5d930c5b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5d841ac820 con 0x558c8bf947a0 2024-06-10T01:18:04.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.214+0000 7f5d930c5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5d841adc90 con 0x558c8bf99f60 2024-06-10T01:18:04.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.214+0000 7f5d930c5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5d841acef0 con 0x558c8bf9a880 2024-06-10T01:18:04.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.214+0000 7f5d930c5b80 10 client.0 _ll_register_callbacks cb 0x558c8be7b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:18:04.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.214+0000 7f5d8259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8bf947a0 0x558c8c010900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:04.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.214+0000 7f5d8259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8bf947a0 0x558c8c010900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45456/0 (socket says 172.21.15.7:45456) 2024-06-10T01:18:04.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.214+0000 7f5d8259c700 1 -- 172.21.15.7:0/659352181 learned_addr learned my addr 172.21.15.7:0/659352181 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:04.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.214+0000 7f5d81d9b700 1 --2- 172.21.15.7:0/659352181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8bf99f60 0x558c8c010e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:04.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.214+0000 7f5d82d9d700 1 --2- 172.21.15.7:0/659352181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8bf9a880 0x558c8c01b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:04.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.214+0000 7f5d8259c700 1 -- 172.21.15.7:0/659352181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8bf9a880 msgr2=0x558c8c01b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:04.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.214+0000 7f5d8259c700 1 --2- 172.21.15.7:0/659352181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8bf9a880 0x558c8c01b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.214+0000 7f5d8259c700 1 -- 172.21.15.7:0/659352181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8bf99f60 msgr2=0x558c8c010e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:04.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.214+0000 7f5d8259c700 1 --2- 172.21.15.7:0/659352181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8bf99f60 0x558c8c010e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.214+0000 7f5d8259c700 1 -- 172.21.15.7:0/659352181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c8c01bce0 con 0x558c8bf947a0 2024-06-10T01:18:04.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.214+0000 7f5d8259c700 1 --2- 172.21.15.7:0/659352181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8bf947a0 0x558c8c010900 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f5d74004170 tx=0x7f5d74013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1b0f1a0c90617a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:04.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.214+0000 7f5d69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:18:04.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.215+0000 7f5d69ffb700 1 -- 172.21.15.7:0/659352181 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d74003a90 con 0x558c8bf947a0 2024-06-10T01:18:04.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.215+0000 7f5d69ffb700 1 -- 172.21.15.7:0/659352181 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f5d74003c30 con 0x558c8bf947a0 2024-06-10T01:18:04.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.215+0000 7f5d930c5b80 1 -- 172.21.15.7:0/659352181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d7c01b320 con 0x558c8bf947a0 2024-06-10T01:18:04.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.215+0000 7f5d69ffb700 1 -- 172.21.15.7:0/659352181 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5d74038050 con 0x558c8bf947a0 2024-06-10T01:18:04.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.215+0000 7f5d930c5b80 10 client.18483 fetch_fsmap learned FSMap version 40 2024-06-10T01:18:04.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.215+0000 7f5d930c5b80 1 -- 172.21.15.7:0/659352181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x558c8c01d020 con 0x558c8bf947a0 2024-06-10T01:18:04.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.215+0000 7f5d69ffb700 1 -- 172.21.15.7:0/659352181 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d7c01b320 con 0x558c8bf947a0 2024-06-10T01:18:04.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.215+0000 7f5d69ffb700 1 -- 172.21.15.7:0/659352181 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f5d74021420 con 0x558c8bf947a0 2024-06-10T01:18:04.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.215+0000 7f5d930c5b80 10 client.18483 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:18:04.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.215+0000 7f5d930c5b80 10 client.18483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:18:04.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.215+0000 7f5d930c5b80 20 client.18483 populate_metadata read hostname 'smithi007' 2024-06-10T01:18:04.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.215+0000 7f5d930c5b80 1 --2- 172.21.15.7:0/659352181 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c8bfade30 0x558c8c01f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:04.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.215+0000 7f5d930c5b80 4 client.18483 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:18:04.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.215+0000 7f5d930c5b80 1 -- 172.21.15.7:0/659352181 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558c8beed2d0 con 0x558c8bfade30 2024-06-10T01:18:04.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.216+0000 7f5d81d9b700 1 --2- 172.21.15.7:0/659352181 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c8bfade30 0x558c8c01f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:04.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.216+0000 7f5d69ffb700 10 client.18483 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:04.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.216+0000 7f5d81d9b700 1 --2- 172.21.15.7:0/659352181 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c8bfade30 0x558c8c01f6f0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba695283c7ae328f server_cookie=727fe4988af66bd in_seq=0 out_seq=0 2024-06-10T01:18:04.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.216+0000 7f5d69ffb700 1 -- 172.21.15.7:0/659352181 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558c8beed2d0 con 0x558c8bfade30 2024-06-10T01:18:04.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.216+0000 7f5d69ffb700 10 client.18483 handle_command_reply: tid=0 2024-06-10T01:18:04.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.216+0000 7f5d930c5b80 1 client.18483 shutdown 2024-06-10T01:18:04.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.216+0000 7f5d930c5b80 1 -- 172.21.15.7:0/659352181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8bf947a0 msgr2=0x558c8c010900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:04.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.216+0000 7f5d930c5b80 1 --2- 172.21.15.7:0/659352181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8bf947a0 0x558c8c010900 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f5d74004170 tx=0x7f5d74013070 comp rx=0 tx=0).stop 2024-06-10T01:18:04.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.216+0000 7f5d930c5b80 1 -- 172.21.15.7:0/659352181 shutdown_connections 2024-06-10T01:18:04.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.216+0000 7f5d930c5b80 1 --2- 172.21.15.7:0/659352181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558c8bf9a880 0x558c8c01b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.216+0000 7f5d930c5b80 1 --2- 172.21.15.7:0/659352181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c8bf947a0 0x558c8c010900 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.216+0000 7f5d930c5b80 1 --2- 172.21.15.7:0/659352181 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558c8bfade30 0x558c8c01f6f0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.216+0000 7f5d930c5b80 1 --2- 172.21.15.7:0/659352181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8bf99f60 0x558c8c010e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.216+0000 7f5d930c5b80 1 -- 172.21.15.7:0/659352181 >> 172.21.15.7:0/659352181 conn(0x558c8bec1080 msgr2=0x558c8bf9f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:04.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.216+0000 7f5d69ffb700 0 client.18483 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:04.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.217+0000 7f5d930c5b80 1 -- 172.21.15.7:0/659352181 shutdown_connections 2024-06-10T01:18:04.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.217+0000 7f5d930c5b80 1 -- 172.21.15.7:0/659352181 wait complete. 2024-06-10T01:18:04.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.217+0000 7f5d930c5b80 20 client.18483 trim_cache size 0 max 16384 2024-06-10T01:18:04.215 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:18:04.216 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-10T01:18:04.216 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-10T01:18:04.216 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:18:04.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.217+0000 7f5d88842700 1 -- 172.21.15.7:0/4006601799 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5d6c0a2ca0 msgr2=0x7f5d6c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:04.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.217+0000 7f5d88842700 1 --2- 172.21.15.7:0/4006601799 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5d6c0a2ca0 0x7f5d6c0a5160 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5d841073f0 tx=0x7f5d740073d0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.217+0000 7f5d88842700 1 -- 172.21.15.7:0/4006601799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d84107840 msgr2=0x7f5d841989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:04.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.217+0000 7f5d88842700 1 --2- 172.21.15.7:0/4006601799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d84107840 0x7f5d841989d0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00ef30 tx=0x7f5d7c00c3a0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.217+0000 7f5d88842700 1 -- 172.21.15.7:0/4006601799 shutdown_connections 2024-06-10T01:18:04.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.217+0000 7f5d88842700 1 --2- 172.21.15.7:0/4006601799 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5d6c0a2ca0 0x7f5d6c0a5160 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.217+0000 7f5d88842700 1 --2- 172.21.15.7:0/4006601799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d84107840 0x7f5d841989d0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.217+0000 7f5d88842700 1 --2- 172.21.15.7:0/4006601799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d84102740 0x7f5d8419e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.217+0000 7f5d88842700 1 --2- 172.21.15.7:0/4006601799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d841030f0 0x7f5d8419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:04.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.217+0000 7f5d88842700 1 -- 172.21.15.7:0/4006601799 >> 172.21.15.7:0/4006601799 conn(0x7f5d840795f0 msgr2=0x7f5d84101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:04.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.217+0000 7f5d88842700 1 -- 172.21.15.7:0/4006601799 shutdown_connections 2024-06-10T01:18:04.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:04.218+0000 7f5d88842700 1 -- 172.21.15.7:0/4006601799 wait complete. 2024-06-10T01:18:04.754 INFO:tasks.workunit.client.0.smithi007.stdout: 1 881536 7.29 MB/sec execute 525 sec latency 20557.210 ms 2024-06-10T01:18:04.972 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:18:04.972 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:18:05.224 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:18:05.680 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:05 smithi007 ceph-mon[32398]: pgmap v794: 129 pgs: 129 active+clean; 147 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 1.7 MiB/s wr, 83 op/s 2024-06-10T01:18:05.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:05 smithi175 ceph-mon[36636]: pgmap v794: 129 pgs: 129 active+clean; 147 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 1.7 MiB/s wr, 83 op/s 2024-06-10T01:18:05.754 INFO:tasks.workunit.client.0.smithi007.stdout: 1 881536 7.28 MB/sec execute 526 sec latency 21557.294 ms 2024-06-10T01:18:05.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:05 smithi078 ceph-mon[36683]: pgmap v794: 129 pgs: 129 active+clean; 147 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 1.7 MiB/s wr, 83 op/s 2024-06-10T01:18:06.328 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:18:06.754 INFO:tasks.workunit.client.0.smithi007.stdout: 1 881536 7.26 MB/sec execute 527 sec latency 22557.338 ms 2024-06-10T01:18:07.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.007+0000 7fb1dea55700 1 -- 172.21.15.7:0/305714970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1d8102ea0 msgr2=0x7fb1d810f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:07.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.007+0000 7fb1dea55700 1 --2- 172.21.15.7:0/305714970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1d8102ea0 0x7fb1d810f380 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fb1d000b0a0 tx=0x7fb1d000b3b0 comp rx=0 tx=0).stop 2024-06-10T01:18:07.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.007+0000 7fb1dea55700 1 -- 172.21.15.7:0/305714970 shutdown_connections 2024-06-10T01:18:07.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.007+0000 7fb1dea55700 1 --2- 172.21.15.7:0/305714970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1d8102ea0 0x7fb1d810f380 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:07.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.007+0000 7fb1dea55700 1 --2- 172.21.15.7:0/305714970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1d81085f0 0x7fb1d81089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:07.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.007+0000 7fb1dea55700 1 --2- 172.21.15.7:0/305714970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1d81024e0 0x7fb1d8102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:07.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.007+0000 7fb1dea55700 1 -- 172.21.15.7:0/305714970 >> 172.21.15.7:0/305714970 conn(0x7fb1d8079640 msgr2=0x7fb1d8079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:07.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.007+0000 7fb1dea55700 1 -- 172.21.15.7:0/305714970 shutdown_connections 2024-06-10T01:18:07.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.007+0000 7fb1dea55700 1 -- 172.21.15.7:0/305714970 wait complete. 2024-06-10T01:18:07.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.008+0000 7fb1dea55700 1 Processor -- start 2024-06-10T01:18:07.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.008+0000 7fb1dea55700 1 -- start start 2024-06-10T01:18:07.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.009+0000 7fb1dea55700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1d81024e0 0x7fb1d81100a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:07.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.009+0000 7fb1dea55700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1d8102ea0 0x7fb1d81105e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:07.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.009+0000 7fb1dea55700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1d81085f0 0x7fb1d8110b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.009+0000 7fb1dea55700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d8114a00 con 0x7fb1d81085f0 2024-06-10T01:18:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.009+0000 7fb1dea55700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d8114880 con 0x7fb1d8102ea0 2024-06-10T01:18:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.009+0000 7fb1dea55700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d8114b80 con 0x7fb1d81024e0 2024-06-10T01:18:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.009+0000 7fb1d7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1d81024e0 0x7fb1d81100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.009+0000 7fb1d7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1d81024e0 0x7fb1d81100a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:50068/0 (socket says 172.21.15.7:50068) 2024-06-10T01:18:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.009+0000 7fb1d7fff700 1 -- 172.21.15.7:0/1874888496 learned_addr learned my addr 172.21.15.7:0/1874888496 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.009+0000 7fb1d77fe700 1 --2- 172.21.15.7:0/1874888496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1d8102ea0 0x7fb1d81105e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.009+0000 7fb1dcff2700 1 --2- 172.21.15.7:0/1874888496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1d81085f0 0x7fb1d8110b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.009+0000 7fb1d77fe700 1 -- 172.21.15.7:0/1874888496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1d81024e0 msgr2=0x7fb1d81100a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.009+0000 7fb1d77fe700 1 --2- 172.21.15.7:0/1874888496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1d81024e0 0x7fb1d81100a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.009+0000 7fb1d77fe700 1 -- 172.21.15.7:0/1874888496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1d81085f0 msgr2=0x7fb1d8110b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.009+0000 7fb1d77fe700 1 --2- 172.21.15.7:0/1874888496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1d81085f0 0x7fb1d8110b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.009+0000 7fb1d77fe700 1 -- 172.21.15.7:0/1874888496 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d81ab240 con 0x7fb1d8102ea0 2024-06-10T01:18:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.010+0000 7fb1d77fe700 1 --2- 172.21.15.7:0/1874888496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1d8102ea0 0x7fb1d81105e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc00ea20 tx=0x7fb1cc00ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.010+0000 7fb1d57fa700 1 -- 172.21.15.7:0/1874888496 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc00cbd0 con 0x7fb1d8102ea0 2024-06-10T01:18:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.010+0000 7fb1d57fa700 1 -- 172.21.15.7:0/1874888496 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1cc00ee60 con 0x7fb1d8102ea0 2024-06-10T01:18:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.010+0000 7fb1dea55700 1 -- 172.21.15.7:0/1874888496 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1d81ab530 con 0x7fb1d8102ea0 2024-06-10T01:18:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.010+0000 7fb1d57fa700 1 -- 172.21.15.7:0/1874888496 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc018680 con 0x7fb1d8102ea0 2024-06-10T01:18:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.010+0000 7fb1dea55700 1 -- 172.21.15.7:0/1874888496 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1d81aba00 con 0x7fb1d8102ea0 2024-06-10T01:18:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.011+0000 7fb1dea55700 1 -- 172.21.15.7:0/1874888496 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1d8103c00 con 0x7fb1d8102ea0 2024-06-10T01:18:07.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.012+0000 7fb1d57fa700 1 -- 172.21.15.7:0/1874888496 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb1cc018820 con 0x7fb1d8102ea0 2024-06-10T01:18:07.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.013+0000 7fb1d57fa700 1 --2- 172.21.15.7:0/1874888496 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb1c00a2ca0 0x7fb1c00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:07.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.013+0000 7fb1d57fa700 1 -- 172.21.15.7:0/1874888496 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb1cc0cf7e0 con 0x7fb1d8102ea0 2024-06-10T01:18:07.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.013+0000 7fb1d7fff700 1 --2- 172.21.15.7:0/1874888496 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb1c00a2ca0 0x7fb1c00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:07.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.014+0000 7fb1d7fff700 1 --2- 172.21.15.7:0/1874888496 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb1c00a2ca0 0x7fb1c00a5160 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8005950 tx=0x7fb1c800b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:07.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.016+0000 7fb1d57fa700 1 -- 172.21.15.7:0/1874888496 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb1cc09c360 con 0x7fb1d8102ea0 2024-06-10T01:18:07.021 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:18:07.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.304+0000 7fb1dea55700 1 -- 172.21.15.7:0/1874888496 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1d8069180 con 0x7fb1d8102ea0 2024-06-10T01:18:07.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.307+0000 7fb1d57fa700 1 -- 172.21.15.7:0/1874888496 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11855 (secure 0 0 0) 0x7fb1d8069180 con 0x7fb1d8102ea0 2024-06-10T01:18:07.302 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:18:07.303 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:16:56.409130+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24451,24434,24436]},"id":1}]} 2024-06-10T01:18:07.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.311+0000 7fb1dea55700 1 -- 172.21.15.7:0/1874888496 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb1c00a2ca0 msgr2=0x7fb1c00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:07.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.311+0000 7fb1dea55700 1 --2- 172.21.15.7:0/1874888496 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb1c00a2ca0 0x7fb1c00a5160 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8005950 tx=0x7fb1c800b500 comp rx=0 tx=0).stop 2024-06-10T01:18:07.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.311+0000 7fb1dea55700 1 -- 172.21.15.7:0/1874888496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1d8102ea0 msgr2=0x7fb1d81105e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:07.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.311+0000 7fb1dea55700 1 --2- 172.21.15.7:0/1874888496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1d8102ea0 0x7fb1d81105e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc00ea20 tx=0x7fb1cc00ed30 comp rx=0 tx=0).stop 2024-06-10T01:18:07.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.312+0000 7fb1dea55700 1 -- 172.21.15.7:0/1874888496 shutdown_connections 2024-06-10T01:18:07.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.312+0000 7fb1dea55700 1 --2- 172.21.15.7:0/1874888496 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb1c00a2ca0 0x7fb1c00a5160 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:07.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.312+0000 7fb1dea55700 1 --2- 172.21.15.7:0/1874888496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1d81085f0 0x7fb1d8110b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:07.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.312+0000 7fb1dea55700 1 --2- 172.21.15.7:0/1874888496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1d81024e0 0x7fb1d81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:07.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.312+0000 7fb1dea55700 1 --2- 172.21.15.7:0/1874888496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1d8102ea0 0x7fb1d81105e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:07.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.312+0000 7fb1dea55700 1 -- 172.21.15.7:0/1874888496 >> 172.21.15.7:0/1874888496 conn(0x7fb1d8079640 msgr2=0x7fb1d8109330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:07.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.312+0000 7fb1dea55700 1 -- 172.21.15.7:0/1874888496 shutdown_connections 2024-06-10T01:18:07.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:07.312+0000 7fb1dea55700 1 -- 172.21.15.7:0/1874888496 wait complete. 2024-06-10T01:18:07.311 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 40 2024-06-10T01:18:07.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:07 smithi007 ceph-mon[32398]: pgmap v795: 129 pgs: 129 active+clean; 146 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 1.6 MiB/s wr, 81 op/s 2024-06-10T01:18:07.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:07 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1874888496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:18:07.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:07 smithi175 ceph-mon[36636]: pgmap v795: 129 pgs: 129 active+clean; 146 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 1.6 MiB/s wr, 81 op/s 2024-06-10T01:18:07.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:07 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1874888496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:18:07.754 INFO:tasks.workunit.client.0.smithi007.stdout: 1 881536 7.25 MB/sec execute 528 sec latency 23557.386 ms 2024-06-10T01:18:07.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:07 smithi078 ceph-mon[36683]: pgmap v795: 129 pgs: 129 active+clean; 146 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 1.6 MiB/s wr, 81 op/s 2024-06-10T01:18:07.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:07 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1874888496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:18:08.754 INFO:tasks.workunit.client.0.smithi007.stdout: 1 881536 7.24 MB/sec execute 529 sec latency 24557.418 ms 2024-06-10T01:18:09.094 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:18:09.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.279+0000 7fdf01b48700 1 -- 172.21.15.7:0/4041808980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdefc102740 msgr2=0x7fdefc102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:09.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.279+0000 7fdf01b48700 1 --2- 172.21.15.7:0/4041808980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdefc102740 0x7fdefc102b20 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fdeec0099c0 tx=0x7fdeec009cd0 comp rx=0 tx=0).stop 2024-06-10T01:18:09.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.279+0000 7fdf01b48700 1 -- 172.21.15.7:0/4041808980 shutdown_connections 2024-06-10T01:18:09.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.279+0000 7fdf01b48700 1 --2- 172.21.15.7:0/4041808980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdefc107840 0x7fdefc10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:09.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.279+0000 7fdf01b48700 1 --2- 172.21.15.7:0/4041808980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdefc102740 0x7fdefc102b20 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:09.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.279+0000 7fdf01b48700 1 --2- 172.21.15.7:0/4041808980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdefc1030f0 0x7fdefc107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:09.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.279+0000 7fdf01b48700 1 -- 172.21.15.7:0/4041808980 >> 172.21.15.7:0/4041808980 conn(0x7fdefc0795f0 msgr2=0x7fdefc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:09.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.279+0000 7fdf01b48700 1 -- 172.21.15.7:0/4041808980 shutdown_connections 2024-06-10T01:18:09.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.279+0000 7fdf01b48700 1 -- 172.21.15.7:0/4041808980 wait complete. 2024-06-10T01:18:09.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.280+0000 7fdf01b48700 1 Processor -- start 2024-06-10T01:18:09.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.280+0000 7fdf01b48700 1 -- start start 2024-06-10T01:18:09.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.280+0000 7fdf01b48700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdefc102740 0x7fdefc19ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:09.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.280+0000 7fdf01b48700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdefc1030f0 0x7fdefc19b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:09.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.280+0000 7fdf01b48700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdefc107840 0x7fdefc19f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:09.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.280+0000 7fdf01b48700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc1127d0 con 0x7fdefc1030f0 2024-06-10T01:18:09.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.280+0000 7fdf01b48700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc112650 con 0x7fdefc102740 2024-06-10T01:18:09.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.280+0000 7fdf01b48700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc112950 con 0x7fdefc107840 2024-06-10T01:18:09.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.280+0000 7fdefaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdefc1030f0 0x7fdefc19b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:09.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.281+0000 7fdefaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdefc1030f0 0x7fdefc19b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45498/0 (socket says 172.21.15.7:45498) 2024-06-10T01:18:09.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.281+0000 7fdefaffd700 1 -- 172.21.15.7:0/1928486970 learned_addr learned my addr 172.21.15.7:0/1928486970 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:09.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.281+0000 7fdefbfff700 1 --2- 172.21.15.7:0/1928486970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdefc107840 0x7fdefc19f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:09.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.281+0000 7fdefb7fe700 1 --2- 172.21.15.7:0/1928486970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdefc102740 0x7fdefc19ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:09.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.281+0000 7fdefaffd700 1 -- 172.21.15.7:0/1928486970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdefc107840 msgr2=0x7fdefc19f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:09.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.281+0000 7fdefaffd700 1 --2- 172.21.15.7:0/1928486970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdefc107840 0x7fdefc19f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:09.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.281+0000 7fdefaffd700 1 -- 172.21.15.7:0/1928486970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdefc102740 msgr2=0x7fdefc19ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:09.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.281+0000 7fdefaffd700 1 --2- 172.21.15.7:0/1928486970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdefc102740 0x7fdefc19ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:09.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.281+0000 7fdefaffd700 1 -- 172.21.15.7:0/1928486970 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdefc19fe30 con 0x7fdefc1030f0 2024-06-10T01:18:09.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.281+0000 7fdefb7fe700 1 --2- 172.21.15.7:0/1928486970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdefc102740 0x7fdefc19ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:18:09.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.281+0000 7fdefaffd700 1 --2- 172.21.15.7:0/1928486970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdefc1030f0 0x7fdefc19b360 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fdef000e9a0 tx=0x7fdef000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:09.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.281+0000 7fdef8ff9700 1 -- 172.21.15.7:0/1928486970 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef000cbd0 con 0x7fdefc1030f0 2024-06-10T01:18:09.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.281+0000 7fdf01b48700 1 -- 172.21.15.7:0/1928486970 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdefc1a0120 con 0x7fdefc1030f0 2024-06-10T01:18:09.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.281+0000 7fdef8ff9700 1 -- 172.21.15.7:0/1928486970 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdef000cd70 con 0x7fdefc1030f0 2024-06-10T01:18:09.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.281+0000 7fdef8ff9700 1 -- 172.21.15.7:0/1928486970 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef0018750 con 0x7fdefc1030f0 2024-06-10T01:18:09.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.281+0000 7fdf01b48700 1 -- 172.21.15.7:0/1928486970 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdefc1a79a0 con 0x7fdefc1030f0 2024-06-10T01:18:09.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.283+0000 7fdf01b48700 1 -- 172.21.15.7:0/1928486970 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdefc069180 con 0x7fdefc1030f0 2024-06-10T01:18:09.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.284+0000 7fdef8ff9700 1 -- 172.21.15.7:0/1928486970 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fdef0010430 con 0x7fdefc1030f0 2024-06-10T01:18:09.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.285+0000 7fdef8ff9700 1 --2- 172.21.15.7:0/1928486970 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdee40a2ca0 0x7fdee40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:09.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.285+0000 7fdef8ff9700 1 -- 172.21.15.7:0/1928486970 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fdef00cf700 con 0x7fdefc1030f0 2024-06-10T01:18:09.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.285+0000 7fdefb7fe700 1 --2- 172.21.15.7:0/1928486970 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdee40a2ca0 0x7fdee40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:09.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.285+0000 7fdefb7fe700 1 --2- 172.21.15.7:0/1928486970 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdee40a2ca0 0x7fdee40a5160 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdeec000c00 tx=0x7fdeec005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:09.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.290+0000 7fdef8ff9700 1 -- 172.21.15.7:0/1928486970 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdef009c280 con 0x7fdefc1030f0 2024-06-10T01:18:09.354 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:18:09.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.582+0000 7fdf01b48700 1 -- 172.21.15.7:0/1928486970 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdefc1a7bd0 con 0x7fdefc1030f0 2024-06-10T01:18:09.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.583+0000 7fdef8ff9700 1 -- 172.21.15.7:0/1928486970 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11855 (secure 0 0 0) 0x7fdef009bb80 con 0x7fdefc1030f0 2024-06-10T01:18:09.578 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:18:09.578 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:16:56.409130+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24451,24434,24436]},"id":1}]} 2024-06-10T01:18:09.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.585+0000 7fdf01b48700 1 -- 172.21.15.7:0/1928486970 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdee40a2ca0 msgr2=0x7fdee40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:09.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.585+0000 7fdf01b48700 1 --2- 172.21.15.7:0/1928486970 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdee40a2ca0 0x7fdee40a5160 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdeec000c00 tx=0x7fdeec005fb0 comp rx=0 tx=0).stop 2024-06-10T01:18:09.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.585+0000 7fdf01b48700 1 -- 172.21.15.7:0/1928486970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdefc1030f0 msgr2=0x7fdefc19b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:09.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.585+0000 7fdf01b48700 1 --2- 172.21.15.7:0/1928486970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdefc1030f0 0x7fdefc19b360 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fdef000e9a0 tx=0x7fdef000ecb0 comp rx=0 tx=0).stop 2024-06-10T01:18:09.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.585+0000 7fdf01b48700 1 -- 172.21.15.7:0/1928486970 shutdown_connections 2024-06-10T01:18:09.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.585+0000 7fdf01b48700 1 --2- 172.21.15.7:0/1928486970 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdee40a2ca0 0x7fdee40a5160 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:09.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.585+0000 7fdf01b48700 1 --2- 172.21.15.7:0/1928486970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdefc107840 0x7fdefc19f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:09.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.585+0000 7fdf01b48700 1 --2- 172.21.15.7:0/1928486970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdefc102740 0x7fdefc19ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:09.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.585+0000 7fdf01b48700 1 --2- 172.21.15.7:0/1928486970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdefc1030f0 0x7fdefc19b360 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:09.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.585+0000 7fdf01b48700 1 -- 172.21.15.7:0/1928486970 >> 172.21.15.7:0/1928486970 conn(0x7fdefc0795f0 msgr2=0x7fdefc108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:09.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.585+0000 7fdf01b48700 1 -- 172.21.15.7:0/1928486970 shutdown_connections 2024-06-10T01:18:09.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:09.585+0000 7fdf01b48700 1 -- 172.21.15.7:0/1928486970 wait complete. 2024-06-10T01:18:09.585 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 40 2024-06-10T01:18:09.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:09 smithi175 ceph-mon[36636]: pgmap v796: 129 pgs: 129 active+clean; 146 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.4 MiB/s wr, 78 op/s 2024-06-10T01:18:09.754 INFO:tasks.workunit.client.0.smithi007.stdout: 1 881536 7.22 MB/sec execute 530 sec latency 25557.448 ms 2024-06-10T01:18:09.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:09 smithi007 ceph-mon[32398]: pgmap v796: 129 pgs: 129 active+clean; 146 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.4 MiB/s wr, 78 op/s 2024-06-10T01:18:09.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:09 smithi078 ceph-mon[36683]: pgmap v796: 129 pgs: 129 active+clean; 146 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.4 MiB/s wr, 78 op/s 2024-06-10T01:18:10.626 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:10 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1928486970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:18:10.659 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db --set-id b6c01176 --release --await' 2024-06-10T01:18:10.659 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db --set-id b6c01176 --release --await 2024-06-10T01:18:10.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:10 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1928486970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:18:10.754 INFO:tasks.workunit.client.0.smithi007.stdout: 1 881536 7.21 MB/sec execute 531 sec latency 26557.497 ms 2024-06-10T01:18:10.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:10 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1928486970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:18:10.950 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.953+0000 7f03fc688640 1 -- 172.21.15.7:0/374637379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03f4107780 msgr2=0x7f03f4109b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:10.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.953+0000 7f03fc688640 1 --2- 172.21.15.7:0/374637379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03f4107780 0x7f03f4109b70 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f03ec009900 tx=0x7f03ec02a410 comp rx=0 tx=0).stop 2024-06-10T01:18:10.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.953+0000 7f03fc688640 1 -- 172.21.15.7:0/374637379 shutdown_connections 2024-06-10T01:18:10.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.953+0000 7f03fc688640 1 --2- 172.21.15.7:0/374637379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03f4107780 0x7f03f4109b70 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:10.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.953+0000 7f03fc688640 1 --2- 172.21.15.7:0/374637379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03f4104e50 0x7f03f4107240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:10.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.953+0000 7f03fc688640 1 --2- 172.21.15.7:0/374637379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03f4102520 0x7f03f4104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:10.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.953+0000 7f03fc688640 1 -- 172.21.15.7:0/374637379 >> 172.21.15.7:0/374637379 conn(0x7f03f40fbe70 msgr2=0x7f03f40fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:10.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.953+0000 7f03fc688640 1 -- 172.21.15.7:0/374637379 shutdown_connections 2024-06-10T01:18:10.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.953+0000 7f03fc688640 1 -- 172.21.15.7:0/374637379 wait complete. 2024-06-10T01:18:10.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.954+0000 7f03fc688640 1 Processor -- start 2024-06-10T01:18:10.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.954+0000 7f03fc688640 1 -- start start 2024-06-10T01:18:10.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.954+0000 7f03fc688640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03f4102520 0x7f03f4199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:10.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.954+0000 7f03fc688640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03f4104e50 0x7f03f419a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.954+0000 7f03fc688640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03f4107780 0x7f03f41a1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.954+0000 7f03fc688640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f03f410d6a0 con 0x7f03f4107780 2024-06-10T01:18:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.954+0000 7f03fc688640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f03f410d520 con 0x7f03f4104e50 2024-06-10T01:18:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.954+0000 7f03fc688640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f03f410d820 con 0x7f03f4102520 2024-06-10T01:18:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.954+0000 7f03fac25640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03f4107780 0x7f03f41a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.954+0000 7f03fac25640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03f4107780 0x7f03f41a1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45520/0 (socket says 172.21.15.7:45520) 2024-06-10T01:18:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.954+0000 7f03fac25640 1 -- 172.21.15.7:0/3575882130 learned_addr learned my addr 172.21.15.7:0/3575882130 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.954+0000 7f03fa424640 1 --2- 172.21.15.7:0/3575882130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03f4102520 0x7f03f4199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.954+0000 7f03fac25640 1 -- 172.21.15.7:0/3575882130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03f4102520 msgr2=0x7f03f4199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.954+0000 7f03f9c23640 1 --2- 172.21.15.7:0/3575882130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03f4104e50 0x7f03f419a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.954+0000 7f03fac25640 1 --2- 172.21.15.7:0/3575882130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03f4102520 0x7f03f4199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.954+0000 7f03fac25640 1 -- 172.21.15.7:0/3575882130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03f4104e50 msgr2=0x7f03f419a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.954+0000 7f03fac25640 1 --2- 172.21.15.7:0/3575882130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03f4104e50 0x7f03f419a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:10.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.954+0000 7f03fac25640 1 -- 172.21.15.7:0/3575882130 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f41a1a90 con 0x7f03f4107780 2024-06-10T01:18:10.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.955+0000 7f03fac25640 1 --2- 172.21.15.7:0/3575882130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03f4107780 0x7f03f41a1420 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f03ec0314e0 tx=0x7f03ec004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:10.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.955+0000 7f03e77fe640 1 -- 172.21.15.7:0/3575882130 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec03f030 con 0x7f03f4107780 2024-06-10T01:18:10.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.955+0000 7f03e77fe640 1 -- 172.21.15.7:0/3575882130 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03ec003ce0 con 0x7f03f4107780 2024-06-10T01:18:10.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.955+0000 7f03fc688640 1 -- 172.21.15.7:0/3575882130 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f41a1d20 con 0x7f03f4107780 2024-06-10T01:18:10.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.955+0000 7f03e77fe640 1 -- 172.21.15.7:0/3575882130 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec035070 con 0x7f03f4107780 2024-06-10T01:18:10.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.955+0000 7f03fc688640 1 -- 172.21.15.7:0/3575882130 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f41a21c0 con 0x7f03f4107780 2024-06-10T01:18:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.957+0000 7f03e77fe640 1 -- 172.21.15.7:0/3575882130 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f03ec042030 con 0x7f03f4107780 2024-06-10T01:18:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.957+0000 7f03ff391740 1 Processor -- start 2024-06-10T01:18:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.958+0000 7f03ff391740 1 -- start start 2024-06-10T01:18:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.958+0000 7f03ff391740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aff4ed010 0x561aff4ed430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.958+0000 7f03ff391740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aff4eda00 0x561aff4f2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.958+0000 7f03ff391740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561aff4f2b70 0x561aff4f5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.958+0000 7f03ff391740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561aff4504f0 con 0x561aff4ed010 2024-06-10T01:18:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.958+0000 7f03ff391740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561aff3ee870 con 0x561aff4eda00 2024-06-10T01:18:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.958+0000 7f03ff391740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561aff44c4b0 con 0x561aff4f2b70 2024-06-10T01:18:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.958+0000 7f03f9c23640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aff4ed010 0x561aff4ed430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.958+0000 7f03f9c23640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aff4ed010 0x561aff4ed430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45522/0 (socket says 172.21.15.7:45522) 2024-06-10T01:18:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.958+0000 7f03f9c23640 1 -- 172.21.15.7:0/1586474765 learned_addr learned my addr 172.21.15.7:0/1586474765 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.958+0000 7f03e77fe640 1 --2- 172.21.15.7:0/3575882130 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f03e00a2b80 0x7f03e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.958+0000 7f03fa424640 1 --2- 172.21.15.7:0/1586474765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aff4eda00 0x561aff4f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.958+0000 7f03f9c23640 1 -- 172.21.15.7:0/1586474765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561aff4f2b70 msgr2=0x561aff4f5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.958+0000 7f03f9c23640 1 --2- 172.21.15.7:0/1586474765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561aff4f2b70 0x561aff4f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.958+0000 7f03fac25640 1 --2- 172.21.15.7:0/3575882130 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f03e00a2b80 0x7f03e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.958+0000 7f03f9c23640 1 -- 172.21.15.7:0/1586474765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aff4eda00 msgr2=0x561aff4f2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.958+0000 7f03f9c23640 1 --2- 172.21.15.7:0/1586474765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aff4eda00 0x561aff4f2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.958+0000 7f03f9c23640 1 -- 172.21.15.7:0/1586474765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561aff40a9b0 con 0x561aff4ed010 2024-06-10T01:18:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.958+0000 7f03e77fe640 1 -- 172.21.15.7:0/3575882130 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f03ec0efe20 con 0x7f03f4107780 2024-06-10T01:18:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.958+0000 7f03f9c23640 1 --2- 172.21.15.7:0/1586474765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aff4ed010 0x561aff4ed430 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f03f0008e70 tx=0x7f03f000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5e5debe9584d83 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.959+0000 7f03e4ff9640 1 -- 172.21.15.7:0/1586474765 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0015070 con 0x561aff4ed010 2024-06-10T01:18:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.959+0000 7f03fac25640 1 --2- 172.21.15.7:0/3575882130 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f03e00a2b80 0x7f03e00a5040 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f03ec0ed5f0 tx=0x7f03ec0b17e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.959+0000 7f03e4ff9640 1 -- 172.21.15.7:0/1586474765 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03f0004d20 con 0x561aff4ed010 2024-06-10T01:18:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.959+0000 7f03e4ff9640 1 -- 172.21.15.7:0/1586474765 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f00054c0 con 0x561aff4ed010 2024-06-10T01:18:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.959+0000 7f03ff391740 1 -- 172.21.15.7:0/1586474765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aff4ed010 msgr2=0x561aff4ed430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.959+0000 7f03ff391740 1 --2- 172.21.15.7:0/1586474765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aff4ed010 0x561aff4ed430 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f03f0008e70 tx=0x7f03f000f730 comp rx=0 tx=0).stop 2024-06-10T01:18:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.959+0000 7f03ff391740 1 -- 172.21.15.7:0/1586474765 shutdown_connections 2024-06-10T01:18:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.959+0000 7f03ff391740 1 --2- 172.21.15.7:0/1586474765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561aff4f2b70 0x561aff4f5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.959+0000 7f03ff391740 1 --2- 172.21.15.7:0/1586474765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aff4eda00 0x561aff4f2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.959+0000 7f03ff391740 1 --2- 172.21.15.7:0/1586474765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aff4ed010 0x561aff4ed430 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.959+0000 7f03ff391740 1 -- 172.21.15.7:0/1586474765 >> 172.21.15.7:0/1586474765 conn(0x561aff4e8680 msgr2=0x561aff4e8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.959+0000 7f03ff391740 1 -- 172.21.15.7:0/1586474765 shutdown_connections 2024-06-10T01:18:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.959+0000 7f03ff391740 1 -- 172.21.15.7:0/1586474765 wait complete. 2024-06-10T01:18:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.960+0000 7f03ff391740 1 -- start start 2024-06-10T01:18:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.960+0000 7f03ff391740 1 Processor -- start 2024-06-10T01:18:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.960+0000 7f03ff391740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aff4ed010 0x561aff4f7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.960+0000 7f03ff391740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561aff4eda00 0x561aff4f77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.960+0000 7f03ff391740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aff4f2b70 0x561aff4fe800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.960+0000 7f03ff391740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561aff4f5b20 con 0x561aff4ed010 2024-06-10T01:18:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.960+0000 7f03ff391740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561aff3d9ae0 con 0x561aff4f2b70 2024-06-10T01:18:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.960+0000 7f03ff391740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561aff4f5ca0 con 0x561aff4eda00 2024-06-10T01:18:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.960+0000 7f03ff391740 10 client.0 _ll_register_callbacks cb 0x561aff4129b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:18:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.960+0000 7f03f9c23640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aff4ed010 0x561aff4f7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.960+0000 7f03f9c23640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aff4ed010 0x561aff4f7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45524/0 (socket says 172.21.15.7:45524) 2024-06-10T01:18:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.960+0000 7f03f9c23640 1 -- 172.21.15.7:0/3019493869 learned_addr learned my addr 172.21.15.7:0/3019493869 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.960+0000 7f03f9c23640 1 -- 172.21.15.7:0/3019493869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561aff4eda00 msgr2=0x561aff4f77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.960+0000 7f03f9c23640 1 --2- 172.21.15.7:0/3019493869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561aff4eda00 0x561aff4f77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.960+0000 7f03f9c23640 1 -- 172.21.15.7:0/3019493869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aff4f2b70 msgr2=0x561aff4fe800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.960+0000 7f03f9c23640 1 --2- 172.21.15.7:0/3019493869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aff4f2b70 0x561aff4fe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.960+0000 7f03f9c23640 1 -- 172.21.15.7:0/3019493869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561aff164210 con 0x561aff4ed010 2024-06-10T01:18:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.961+0000 7f03f9c23640 1 --2- 172.21.15.7:0/3019493869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aff4ed010 0x561aff4f7280 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f03f0006010 tx=0x7f03f0011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322dab20dfc6114f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.961+0000 7f03ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:18:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.961+0000 7f03ddffb640 1 -- 172.21.15.7:0/3019493869 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0015040 con 0x561aff4ed010 2024-06-10T01:18:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.961+0000 7f03ddffb640 1 -- 172.21.15.7:0/3019493869 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f03f0005440 con 0x561aff4ed010 2024-06-10T01:18:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.961+0000 7f03ff391740 1 -- 172.21.15.7:0/3019493869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03f0004d20 con 0x561aff4ed010 2024-06-10T01:18:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.961+0000 7f03ddffb640 1 -- 172.21.15.7:0/3019493869 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f03f002f050 con 0x561aff4ed010 2024-06-10T01:18:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.961+0000 7f03ff391740 10 client.18507 fetch_fsmap learned FSMap version 40 2024-06-10T01:18:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.961+0000 7f03ff391740 1 -- 172.21.15.7:0/3019493869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561aff230620 con 0x561aff4ed010 2024-06-10T01:18:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.961+0000 7f03ddffb640 1 -- 172.21.15.7:0/3019493869 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03f0004d20 con 0x561aff4ed010 2024-06-10T01:18:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.961+0000 7f03ddffb640 1 -- 172.21.15.7:0/3019493869 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f03f0019aa0 con 0x561aff4ed010 2024-06-10T01:18:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.961+0000 7f03ff391740 10 client.18507 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:18:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.961+0000 7f03ff391740 10 client.18507 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:18:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.961+0000 7f03ff391740 20 client.18507 populate_metadata read hostname 'smithi007' 2024-06-10T01:18:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.961+0000 7f03ff391740 1 --2- 172.21.15.7:0/3019493869 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x561aff502f30 0x561aff505320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.961+0000 7f03ff391740 4 client.18507 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:18:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.961+0000 7f03ff391740 1 -- 172.21.15.7:0/3019493869 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561aff463f10 con 0x561aff502f30 2024-06-10T01:18:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.962+0000 7f03fa424640 1 --2- 172.21.15.7:0/3019493869 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x561aff502f30 0x561aff505320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.962+0000 7f03fa424640 1 --2- 172.21.15.7:0/3019493869 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x561aff502f30 0x561aff505320 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad5c2429c3266f82 server_cookie=91100e38a5ed662c in_seq=0 out_seq=0 2024-06-10T01:18:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.962+0000 7f03ddffb640 10 client.18507 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.963+0000 7f03ddffb640 1 -- 172.21.15.7:0/3019493869 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561aff463f10 con 0x561aff502f30 2024-06-10T01:18:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.963+0000 7f03ddffb640 10 client.18507 handle_command_reply: tid=0 2024-06-10T01:18:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.963+0000 7f03ff391740 1 client.18507 shutdown 2024-06-10T01:18:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.963+0000 7f03ff391740 1 -- 172.21.15.7:0/3019493869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aff4ed010 msgr2=0x561aff4f7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.963+0000 7f03ff391740 1 --2- 172.21.15.7:0/3019493869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aff4ed010 0x561aff4f7280 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f03f0006010 tx=0x7f03f0011e50 comp rx=0 tx=0).stop 2024-06-10T01:18:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.963+0000 7f03ff391740 1 -- 172.21.15.7:0/3019493869 shutdown_connections 2024-06-10T01:18:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.963+0000 7f03ff391740 1 --2- 172.21.15.7:0/3019493869 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x561aff502f30 0x561aff505320 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.963+0000 7f03ff391740 1 --2- 172.21.15.7:0/3019493869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aff4f2b70 0x561aff4fe800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.963+0000 7f03ff391740 1 --2- 172.21.15.7:0/3019493869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561aff4eda00 0x561aff4f77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.963+0000 7f03ddffb640 0 client.18507 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.963+0000 7f03ff391740 1 --2- 172.21.15.7:0/3019493869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aff4ed010 0x561aff4f7280 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.963+0000 7f03ff391740 1 -- 172.21.15.7:0/3019493869 >> 172.21.15.7:0/3019493869 conn(0x561aff4e8680 msgr2=0x561aff4e77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.963+0000 7f03ff391740 1 -- 172.21.15.7:0/3019493869 shutdown_connections 2024-06-10T01:18:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.963+0000 7f03ff391740 1 -- 172.21.15.7:0/3019493869 wait complete. 2024-06-10T01:18:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.963+0000 7f03ff391740 20 client.18507 trim_cache size 0 max 16384 2024-06-10T01:18:10.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.978+0000 7f03ff391740 1 Processor -- start 2024-06-10T01:18:10.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.979+0000 7f03ff391740 1 -- start start 2024-06-10T01:18:10.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.979+0000 7f03ff391740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561aff4ed010 0x561aff4f7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:10.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.979+0000 7f03ff391740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aff4eda00 0x561aff4fe790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:10.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.979+0000 7f03ff391740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aff4f2b70 0x561aff4e7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:10.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.979+0000 7f03ff391740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f03f41aef90 con 0x561aff4eda00 2024-06-10T01:18:10.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.979+0000 7f03ff391740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f03f41ac860 con 0x561aff4f2b70 2024-06-10T01:18:10.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.979+0000 7f03ff391740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f03f41aa6b0 con 0x561aff4ed010 2024-06-10T01:18:10.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.979+0000 7f03fa424640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aff4eda00 0x561aff4fe790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:10.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.979+0000 7f03fa424640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aff4eda00 0x561aff4fe790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45534/0 (socket says 172.21.15.7:45534) 2024-06-10T01:18:10.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.979+0000 7f03fa424640 1 -- 172.21.15.7:0/1041221378 learned_addr learned my addr 172.21.15.7:0/1041221378 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:10.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.979+0000 7f03f9c23640 1 --2- 172.21.15.7:0/1041221378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aff4f2b70 0x561aff4e7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.979+0000 7f03f9c23640 1 --2- 172.21.15.7:0/1041221378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561aff4ed010 0x561aff4f7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.979+0000 7f03fa424640 1 -- 172.21.15.7:0/1041221378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561aff4ed010 msgr2=0x561aff4f7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.979+0000 7f03fa424640 1 --2- 172.21.15.7:0/1041221378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561aff4ed010 0x561aff4f7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.979+0000 7f03fa424640 1 -- 172.21.15.7:0/1041221378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aff4f2b70 msgr2=0x561aff4e7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.979+0000 7f03fa424640 1 --2- 172.21.15.7:0/1041221378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aff4f2b70 0x561aff4e7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.979+0000 7f03fa424640 1 -- 172.21.15.7:0/1041221378 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561aff505690 con 0x561aff4eda00 2024-06-10T01:18:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.979+0000 7f03f9c23640 1 --2- 172.21.15.7:0/1041221378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561aff4ed010 0x561aff4f7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:18:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.979+0000 7f03fa424640 1 --2- 172.21.15.7:0/1041221378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aff4eda00 0x561aff4fe790 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f03e800bb00 tx=0x7f03e8013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd17316e0706bf0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.979+0000 7f03dffff640 1 -- 172.21.15.7:0/1041221378 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8005220 con 0x561aff4eda00 2024-06-10T01:18:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.979+0000 7f03dffff640 1 -- 172.21.15.7:0/1041221378 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03e8004c80 con 0x561aff4eda00 2024-06-10T01:18:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.979+0000 7f03dffff640 1 -- 172.21.15.7:0/1041221378 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e801b9f0 con 0x561aff4eda00 2024-06-10T01:18:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.980+0000 7f03ff391740 1 -- 172.21.15.7:0/1041221378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aff4eda00 msgr2=0x561aff4fe790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.980+0000 7f03ff391740 1 --2- 172.21.15.7:0/1041221378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aff4eda00 0x561aff4fe790 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f03e800bb00 tx=0x7f03e8013830 comp rx=0 tx=0).stop 2024-06-10T01:18:10.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.980+0000 7f03ff391740 1 -- 172.21.15.7:0/1041221378 shutdown_connections 2024-06-10T01:18:10.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.980+0000 7f03ff391740 1 --2- 172.21.15.7:0/1041221378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aff4f2b70 0x561aff4e7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:10.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.980+0000 7f03ff391740 1 --2- 172.21.15.7:0/1041221378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aff4eda00 0x561aff4fe790 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:10.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.980+0000 7f03ff391740 1 --2- 172.21.15.7:0/1041221378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561aff4ed010 0x561aff4f7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:10.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.980+0000 7f03ff391740 1 -- 172.21.15.7:0/1041221378 >> 172.21.15.7:0/1041221378 conn(0x561aff4e8680 msgr2=0x561aff4eb590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:10.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.980+0000 7f03ff391740 1 -- 172.21.15.7:0/1041221378 shutdown_connections 2024-06-10T01:18:10.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.980+0000 7f03ff391740 1 -- 172.21.15.7:0/1041221378 wait complete. 2024-06-10T01:18:10.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.980+0000 7f03ff391740 1 -- start start 2024-06-10T01:18:10.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.980+0000 7f03ff391740 1 Processor -- start 2024-06-10T01:18:10.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.981+0000 7f03ff391740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aff4ed010 0x561aff4fe520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:10.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.981+0000 7f03ff391740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561aff4eda00 0x561aff4faff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:10.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.981+0000 7f03ff391740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aff4f2b70 0x561aff4fb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:10.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.981+0000 7f03ff391740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f03f41aad20 con 0x561aff4f2b70 2024-06-10T01:18:10.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.981+0000 7f03ff391740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f03f41ac190 con 0x561aff4ed010 2024-06-10T01:18:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.981+0000 7f03ff391740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f03f41ab3f0 con 0x561aff4eda00 2024-06-10T01:18:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.981+0000 7f03ff391740 10 client.0 _ll_register_callbacks cb 0x7f03f0008d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:18:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.981+0000 7f03f9c23640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aff4f2b70 0x561aff4fb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.981+0000 7f03f9c23640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aff4f2b70 0x561aff4fb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45548/0 (socket says 172.21.15.7:45548) 2024-06-10T01:18:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.981+0000 7f03f9c23640 1 -- 172.21.15.7:0/2194770962 learned_addr learned my addr 172.21.15.7:0/2194770962 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.981+0000 7f03f9c23640 1 --2- 172.21.15.7:0/2194770962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aff4ed010 0x561aff4fe520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.981+0000 7f03fa424640 1 --2- 172.21.15.7:0/2194770962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561aff4eda00 0x561aff4faff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.981+0000 7f03f9c23640 1 -- 172.21.15.7:0/2194770962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561aff4eda00 msgr2=0x561aff4faff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.981+0000 7f03f9c23640 1 --2- 172.21.15.7:0/2194770962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561aff4eda00 0x561aff4faff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.981+0000 7f03f9c23640 1 -- 172.21.15.7:0/2194770962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aff4ed010 msgr2=0x561aff4fe520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.981+0000 7f03f9c23640 1 --2- 172.21.15.7:0/2194770962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aff4ed010 0x561aff4fe520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.981+0000 7f03f9c23640 1 -- 172.21.15.7:0/2194770962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561aff56bbb0 con 0x561aff4f2b70 2024-06-10T01:18:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.981+0000 7f03f9c23640 1 --2- 172.21.15.7:0/2194770962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aff4f2b70 0x561aff4fb530 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f03f00088e0 tx=0x7f03f0004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb0c1acb4354a3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:10.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.981+0000 7f03ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:18:10.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.981+0000 7f03ddffb640 1 -- 172.21.15.7:0/2194770962 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0015040 con 0x561aff4f2b70 2024-06-10T01:18:10.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.981+0000 7f03ddffb640 1 -- 172.21.15.7:0/2194770962 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f03f00292b0 con 0x561aff4f2b70 2024-06-10T01:18:10.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.981+0000 7f03ff391740 1 -- 172.21.15.7:0/2194770962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03e8004c80 con 0x561aff4f2b70 2024-06-10T01:18:10.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.982+0000 7f03ddffb640 1 -- 172.21.15.7:0/2194770962 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f03f0040050 con 0x561aff4f2b70 2024-06-10T01:18:10.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.982+0000 7f03ff391740 10 client.18513 fetch_fsmap learned FSMap version 40 2024-06-10T01:18:10.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.982+0000 7f03ff391740 1 -- 172.21.15.7:0/2194770962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561aff56cc90 con 0x561aff4f2b70 2024-06-10T01:18:10.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.982+0000 7f03ddffb640 1 -- 172.21.15.7:0/2194770962 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03e8004c80 con 0x561aff4f2b70 2024-06-10T01:18:10.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.982+0000 7f03ddffb640 1 -- 172.21.15.7:0/2194770962 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f03f00195c0 con 0x561aff4f2b70 2024-06-10T01:18:10.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.982+0000 7f03ff391740 10 client.18513 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:18:10.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.982+0000 7f03ff391740 10 client.18513 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:18:10.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.982+0000 7f03ff391740 20 client.18513 populate_metadata read hostname 'smithi007' 2024-06-10T01:18:10.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.982+0000 7f03ff391740 1 --2- 172.21.15.7:0/2194770962 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x561aff56d5c0 0x561aff56f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:10.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.982+0000 7f03ff391740 4 client.18513 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:18:10.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.982+0000 7f03ff391740 1 -- 172.21.15.7:0/2194770962 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561aff45dc20 con 0x561aff56d5c0 2024-06-10T01:18:10.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.982+0000 7f03fa424640 1 --2- 172.21.15.7:0/2194770962 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x561aff56d5c0 0x561aff56f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:10.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.983+0000 7f03ddffb640 10 client.18513 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:10.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:10.983+0000 7f03fa424640 1 --2- 172.21.15.7:0/2194770962 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x561aff56d5c0 0x561aff56f9e0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fb597a217186bd7 server_cookie=daac945b05d0e419 in_seq=0 out_seq=0 2024-06-10T01:18:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:11.010+0000 7f03ddffb640 1 -- 172.21.15.7:0/2194770962 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561aff45dc20 con 0x561aff56d5c0 2024-06-10T01:18:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:11.010+0000 7f03ddffb640 10 client.18513 handle_command_reply: tid=0 2024-06-10T01:18:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:11.010+0000 7f03ff391740 1 client.18513 shutdown 2024-06-10T01:18:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:11.010+0000 7f03ff391740 1 -- 172.21.15.7:0/2194770962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aff4f2b70 msgr2=0x561aff4fb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:11.010+0000 7f03ff391740 1 --2- 172.21.15.7:0/2194770962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aff4f2b70 0x561aff4fb530 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f03f00088e0 tx=0x7f03f0004ed0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:11.010+0000 7f03ff391740 1 -- 172.21.15.7:0/2194770962 shutdown_connections 2024-06-10T01:18:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:11.010+0000 7f03ff391740 1 --2- 172.21.15.7:0/2194770962 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x561aff56d5c0 0x561aff56f9e0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:11.010+0000 7f03ff391740 1 --2- 172.21.15.7:0/2194770962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aff4f2b70 0x561aff4fb530 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:11.010+0000 7f03ff391740 1 --2- 172.21.15.7:0/2194770962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561aff4eda00 0x561aff4faff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:11.010+0000 7f03ff391740 1 --2- 172.21.15.7:0/2194770962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aff4ed010 0x561aff4fe520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:11.010+0000 7f03ddffb640 0 client.18513 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:11.010+0000 7f03ff391740 1 -- 172.21.15.7:0/2194770962 >> 172.21.15.7:0/2194770962 conn(0x561aff4e8680 msgr2=0x561aff4eb590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:11.010+0000 7f03ff391740 1 -- 172.21.15.7:0/2194770962 shutdown_connections 2024-06-10T01:18:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:11.010+0000 7f03ff391740 1 -- 172.21.15.7:0/2194770962 wait complete. 2024-06-10T01:18:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:11.011+0000 7f03ff391740 20 client.18513 trim_cache size 0 max 16384 2024-06-10T01:18:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:18:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:18:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:18:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:11.011+0000 7f03fc688640 1 -- 172.21.15.7:0/3575882130 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f03e00a2b80 msgr2=0x7f03e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:11.011+0000 7f03fc688640 1 --2- 172.21.15.7:0/3575882130 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f03e00a2b80 0x7f03e00a5040 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f03ec0ed5f0 tx=0x7f03ec0b17e0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:11.011+0000 7f03fc688640 1 -- 172.21.15.7:0/3575882130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03f4107780 msgr2=0x7f03f41a1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:11.011+0000 7f03fc688640 1 --2- 172.21.15.7:0/3575882130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03f4107780 0x7f03f41a1420 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f03ec0314e0 tx=0x7f03ec004360 comp rx=0 tx=0).stop 2024-06-10T01:18:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:11.011+0000 7f03fc688640 1 -- 172.21.15.7:0/3575882130 shutdown_connections 2024-06-10T01:18:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:11.011+0000 7f03fc688640 1 --2- 172.21.15.7:0/3575882130 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f03e00a2b80 0x7f03e00a5040 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:11.011+0000 7f03fc688640 1 --2- 172.21.15.7:0/3575882130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03f4107780 0x7f03f41a1420 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:11.011+0000 7f03fc688640 1 --2- 172.21.15.7:0/3575882130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03f4104e50 0x7f03f419a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:11.011+0000 7f03fc688640 1 --2- 172.21.15.7:0/3575882130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03f4102520 0x7f03f4199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:11.011+0000 7f03fc688640 1 -- 172.21.15.7:0/3575882130 >> 172.21.15.7:0/3575882130 conn(0x7f03f40fbe70 msgr2=0x7f03f4101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:11.011+0000 7f03fc688640 1 -- 172.21.15.7:0/3575882130 shutdown_connections 2024-06-10T01:18:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:11.011+0000 7f03fc688640 1 -- 172.21.15.7:0/3575882130 wait complete. 2024-06-10T01:18:11.615 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:18:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.653+0000 7f1b76b30700 1 -- 172.21.15.7:0/4046320795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b70107840 msgr2=0x7f1b7010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.653+0000 7f1b76b30700 1 --2- 172.21.15.7:0/4046320795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b70107840 0x7f1b7010f3c0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800b0d0 tx=0x7f1b6800b3e0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.653+0000 7f1b76b30700 1 -- 172.21.15.7:0/4046320795 shutdown_connections 2024-06-10T01:18:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.653+0000 7f1b76b30700 1 --2- 172.21.15.7:0/4046320795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b70107840 0x7f1b7010f3c0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.653+0000 7f1b76b30700 1 --2- 172.21.15.7:0/4046320795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b70102740 0x7f1b70102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.653+0000 7f1b76b30700 1 --2- 172.21.15.7:0/4046320795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b701030f0 0x7f1b70107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.653+0000 7f1b76b30700 1 -- 172.21.15.7:0/4046320795 >> 172.21.15.7:0/4046320795 conn(0x7f1b700795f0 msgr2=0x7f1b70079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:11.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.653+0000 7f1b76b30700 1 -- 172.21.15.7:0/4046320795 shutdown_connections 2024-06-10T01:18:11.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.653+0000 7f1b76b30700 1 -- 172.21.15.7:0/4046320795 wait complete. 2024-06-10T01:18:11.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.654+0000 7f1b76b30700 1 Processor -- start 2024-06-10T01:18:11.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.654+0000 7f1b76b30700 1 -- start start 2024-06-10T01:18:11.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.654+0000 7f1b76b30700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b70102740 0x7f1b7019e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:11.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.654+0000 7f1b76b30700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b701030f0 0x7f1b7019ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:11.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.655+0000 7f1b76b30700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b70107840 0x7f1b70198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:11.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.655+0000 7f1b76b30700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1b70112760 con 0x7f1b70102740 2024-06-10T01:18:11.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.655+0000 7f1b76b30700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701125e0 con 0x7f1b701030f0 2024-06-10T01:18:11.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.655+0000 7f1b76b30700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701128e0 con 0x7f1b70107840 2024-06-10T01:18:11.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.655+0000 7f1b748cc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b70102740 0x7f1b7019e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:11.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.655+0000 7f1b748cc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b70102740 0x7f1b7019e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45566/0 (socket says 172.21.15.7:45566) 2024-06-10T01:18:11.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.655+0000 7f1b748cc700 1 -- 172.21.15.7:0/2025451020 learned_addr learned my addr 172.21.15.7:0/2025451020 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:11.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.655+0000 7f1b6ffff700 1 --2- 172.21.15.7:0/2025451020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b701030f0 0x7f1b7019ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:11.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.655+0000 7f1b750cd700 1 --2- 172.21.15.7:0/2025451020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b70107840 0x7f1b70198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:11.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.655+0000 7f1b748cc700 1 -- 172.21.15.7:0/2025451020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b70107840 msgr2=0x7f1b70198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:11.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.655+0000 7f1b748cc700 1 --2- 172.21.15.7:0/2025451020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b70107840 0x7f1b70198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.655+0000 7f1b748cc700 1 -- 172.21.15.7:0/2025451020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b701030f0 msgr2=0x7f1b7019ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:11.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.655+0000 7f1b748cc700 1 --2- 172.21.15.7:0/2025451020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b701030f0 0x7f1b7019ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.655+0000 7f1b748cc700 1 -- 172.21.15.7:0/2025451020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b701990c0 con 0x7f1b70102740 2024-06-10T01:18:11.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.655+0000 7f1b750cd700 1 --2- 172.21.15.7:0/2025451020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b70107840 0x7f1b70198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:18:11.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.655+0000 7f1b748cc700 1 --2- 172.21.15.7:0/2025451020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b70102740 0x7f1b7019e750 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400d980 tx=0x7f1b6400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:11.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.655+0000 7f1b6dffb700 1 -- 172.21.15.7:0/2025451020 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64016070 con 0x7f1b70102740 2024-06-10T01:18:11.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.655+0000 7f1b76b30700 1 -- 172.21.15.7:0/2025451020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b701993b0 con 0x7f1b70102740 2024-06-10T01:18:11.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.656+0000 7f1b6dffb700 1 -- 172.21.15.7:0/2025451020 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b64004d10 con 0x7f1b70102740 2024-06-10T01:18:11.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.656+0000 7f1b6dffb700 1 -- 172.21.15.7:0/2025451020 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b640055f0 con 0x7f1b70102740 2024-06-10T01:18:11.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.656+0000 7f1b76b30700 1 -- 172.21.15.7:0/2025451020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b701a5620 con 0x7f1b70102740 2024-06-10T01:18:11.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.658+0000 7f1b6dffb700 1 -- 172.21.15.7:0/2025451020 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f1b64004750 con 0x7f1b70102740 2024-06-10T01:18:11.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.659+0000 7f1b6dffb700 1 --2- 172.21.15.7:0/2025451020 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1b600a2c20 0x7f1b600a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.659+0000 7f1b6ffff700 1 --2- 172.21.15.7:0/2025451020 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1b600a2c20 0x7f1b600a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.659+0000 7f1b6dffb700 1 -- 172.21.15.7:0/2025451020 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f1b640cf290 con 0x7f1b70102740 2024-06-10T01:18:11.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.659+0000 7f1b6ffff700 1 --2- 172.21.15.7:0/2025451020 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1b600a2c20 0x7f1b600a50e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1b70108bd0 tx=0x7f1b5c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:11.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.660+0000 7f1b813b3b80 1 Processor -- start 2024-06-10T01:18:11.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.660+0000 7f1b813b3b80 1 -- start start 2024-06-10T01:18:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.660+0000 7f1b813b3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589007999a0 0x558900799d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.660+0000 7f1b813b3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55890079a2c0 0x558900794df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.660+0000 7f1b813b3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558900795330 0x558900795770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.660+0000 7f1b813b3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5589006e2810 con 0x558900795330 2024-06-10T01:18:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.660+0000 7f1b813b3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55890068a720 con 0x5589007999a0 2024-06-10T01:18:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.660+0000 7f1b813b3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5589006de7d0 con 0x55890079a2c0 2024-06-10T01:18:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.660+0000 7f1b750cd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558900795330 0x558900795770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.660+0000 7f1b750cd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558900795330 0x558900795770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45568/0 (socket says 172.21.15.7:45568) 2024-06-10T01:18:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.660+0000 7f1b750cd700 1 -- 172.21.15.7:0/757976203 learned_addr learned my addr 172.21.15.7:0/757976203 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.660+0000 7f1b748cc700 1 --2- 172.21.15.7:0/757976203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589007999a0 0x558900799d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.660+0000 7f1b6ffff700 1 --2- 172.21.15.7:0/757976203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55890079a2c0 0x558900794df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:11.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.660+0000 7f1b750cd700 1 -- 172.21.15.7:0/757976203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55890079a2c0 msgr2=0x558900794df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:11.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.660+0000 7f1b750cd700 1 --2- 172.21.15.7:0/757976203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55890079a2c0 0x558900794df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.661+0000 7f1b750cd700 1 -- 172.21.15.7:0/757976203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589007999a0 msgr2=0x558900799d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:11.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.661+0000 7f1b750cd700 1 --2- 172.21.15.7:0/757976203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589007999a0 0x558900799d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.661+0000 7f1b750cd700 1 -- 172.21.15.7:0/757976203 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558900680f50 con 0x558900795330 2024-06-10T01:18:11.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.661+0000 7f1b750cd700 1 --2- 172.21.15.7:0/757976203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558900795330 0x558900795770 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f1b68009310 tx=0x7f1b68009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1658931a48e16dbc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:11.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.661+0000 7f1b5affd700 1 -- 172.21.15.7:0/757976203 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68022070 con 0x558900795330 2024-06-10T01:18:11.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.661+0000 7f1b5affd700 1 -- 172.21.15.7:0/757976203 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b680074e0 con 0x558900795330 2024-06-10T01:18:11.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.661+0000 7f1b5affd700 1 -- 172.21.15.7:0/757976203 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6801c680 con 0x558900795330 2024-06-10T01:18:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.662+0000 7f1b813b3b80 1 -- 172.21.15.7:0/757976203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558900795330 msgr2=0x558900795770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.662+0000 7f1b813b3b80 1 --2- 172.21.15.7:0/757976203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558900795330 0x558900795770 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f1b68009310 tx=0x7f1b68009340 comp rx=0 tx=0).stop 2024-06-10T01:18:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.662+0000 7f1b813b3b80 1 -- 172.21.15.7:0/757976203 shutdown_connections 2024-06-10T01:18:11.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.662+0000 7f1b813b3b80 1 --2- 172.21.15.7:0/757976203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558900795330 0x558900795770 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.662+0000 7f1b813b3b80 1 --2- 172.21.15.7:0/757976203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589007999a0 0x558900799d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.662+0000 7f1b813b3b80 1 --2- 172.21.15.7:0/757976203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55890079a2c0 0x558900794df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.662+0000 7f1b813b3b80 1 -- 172.21.15.7:0/757976203 >> 172.21.15.7:0/757976203 conn(0x5589006bbcc0 msgr2=0x55890078b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:11.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.662+0000 7f1b813b3b80 1 -- 172.21.15.7:0/757976203 shutdown_connections 2024-06-10T01:18:11.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.662+0000 7f1b813b3b80 1 -- 172.21.15.7:0/757976203 wait complete. 2024-06-10T01:18:11.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.662+0000 7f1b813b3b80 1 -- start start 2024-06-10T01:18:11.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.662+0000 7f1b813b3b80 1 Processor -- start 2024-06-10T01:18:11.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.663+0000 7f1b813b3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558900795330 0x558900794680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:11.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.663+0000 7f1b813b3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589007999a0 0x558900794bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:11.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.663+0000 7f1b813b3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55890079a2c0 0x55890078d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:11.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.663+0000 7f1b813b3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5589007a1da0 con 0x55890079a2c0 2024-06-10T01:18:11.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.663+0000 7f1b750cd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55890079a2c0 0x55890078d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.663+0000 7f1b750cd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55890079a2c0 0x55890078d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45570/0 (socket says 172.21.15.7:45570) 2024-06-10T01:18:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.663+0000 7f1b748cc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558900795330 0x558900794680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.663+0000 7f1b750cd700 1 -- 172.21.15.7:0/3854186957 learned_addr learned my addr 172.21.15.7:0/3854186957 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.663+0000 7f1b813b3b80 1 -- 172.21.15.7:0/3854186957 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558900682bf0 con 0x558900795330 2024-06-10T01:18:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.663+0000 7f1b813b3b80 1 -- 172.21.15.7:0/3854186957 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5589007a1f20 con 0x5589007999a0 2024-06-10T01:18:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.663+0000 7f1b813b3b80 10 client.0 _ll_register_callbacks cb 0x55890067c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:18:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.663+0000 7f1b6ffff700 1 --2- 172.21.15.7:0/3854186957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589007999a0 0x558900794bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.664+0000 7f1b750cd700 1 -- 172.21.15.7:0/3854186957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589007999a0 msgr2=0x558900794bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.664+0000 7f1b750cd700 1 --2- 172.21.15.7:0/3854186957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589007999a0 0x558900794bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.664+0000 7f1b750cd700 1 -- 172.21.15.7:0/3854186957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558900795330 msgr2=0x558900794680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.664+0000 7f1b750cd700 1 --2- 172.21.15.7:0/3854186957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558900795330 0x558900794680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.664+0000 7f1b750cd700 1 -- 172.21.15.7:0/3854186957 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589006318d0 con 0x55890079a2c0 2024-06-10T01:18:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.664+0000 7f1b6ffff700 1 --2- 172.21.15.7:0/3854186957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589007999a0 0x558900794bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:18:11.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.664+0000 7f1b750cd700 1 --2- 172.21.15.7:0/3854186957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55890079a2c0 0x55890078d9c0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f1b68007e30 tx=0x7f1b68004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe7d9e45d879ad server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:11.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.664+0000 7f1b4ffff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:18:11.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.664+0000 7f1b4ffff700 1 -- 172.21.15.7:0/3854186957 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6802a030 con 0x55890079a2c0 2024-06-10T01:18:11.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.664+0000 7f1b4ffff700 1 -- 172.21.15.7:0/3854186957 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f1b68009c90 con 0x55890079a2c0 2024-06-10T01:18:11.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.664+0000 7f1b813b3b80 1 -- 172.21.15.7:0/3854186957 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b680074e0 con 0x55890079a2c0 2024-06-10T01:18:11.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.664+0000 7f1b4ffff700 1 -- 172.21.15.7:0/3854186957 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f1b68035050 con 0x55890079a2c0 2024-06-10T01:18:11.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.664+0000 7f1b813b3b80 10 client.18525 fetch_fsmap learned FSMap version 40 2024-06-10T01:18:11.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.664+0000 7f1b813b3b80 1 -- 172.21.15.7:0/3854186957 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5589007b0a60 con 0x55890079a2c0 2024-06-10T01:18:11.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.665+0000 7f1b4ffff700 1 -- 172.21.15.7:0/3854186957 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b680074e0 con 0x55890079a2c0 2024-06-10T01:18:11.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.665+0000 7f1b4ffff700 1 -- 172.21.15.7:0/3854186957 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f1b68011070 con 0x55890079a2c0 2024-06-10T01:18:11.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.665+0000 7f1b813b3b80 10 client.18525 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:18:11.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.665+0000 7f1b813b3b80 10 client.18525 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:18:11.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.665+0000 7f1b813b3b80 20 client.18525 populate_metadata read hostname 'smithi007' 2024-06-10T01:18:11.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.665+0000 7f1b813b3b80 1 --2- 172.21.15.7:0/3854186957 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5589007b2f30 0x5589007b5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:11.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.665+0000 7f1b813b3b80 4 client.18525 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:18:11.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.665+0000 7f1b813b3b80 1 -- 172.21.15.7:0/3854186957 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589006efd10 con 0x5589007b2f30 2024-06-10T01:18:11.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.665+0000 7f1b748cc700 1 --2- 172.21.15.7:0/3854186957 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5589007b2f30 0x5589007b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:11.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.665+0000 7f1b4ffff700 10 client.18525 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:18:11.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.666+0000 7f1b748cc700 1 --2- 172.21.15.7:0/3854186957 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5589007b2f30 0x5589007b5320 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8976c74d04545bce server_cookie=ca65732d6b46f3c0 in_seq=0 out_seq=0 2024-06-10T01:18:11.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.666+0000 7f1b4ffff700 1 -- 172.21.15.7:0/3854186957 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5589006efd10 con 0x5589007b2f30 2024-06-10T01:18:11.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.666+0000 7f1b4ffff700 10 client.18525 handle_command_reply: tid=0 2024-06-10T01:18:11.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:11 smithi007 ceph-mon[32398]: pgmap v797: 129 pgs: 129 active+clean; 146 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 1.7 MiB/s wr, 87 op/s 2024-06-10T01:18:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.666+0000 7f1b813b3b80 1 client.18525 shutdown 2024-06-10T01:18:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.667+0000 7f1b813b3b80 1 -- 172.21.15.7:0/3854186957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55890079a2c0 msgr2=0x55890078d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.667+0000 7f1b813b3b80 1 --2- 172.21.15.7:0/3854186957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55890079a2c0 0x55890078d9c0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f1b68007e30 tx=0x7f1b68004cc0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.667+0000 7f1b813b3b80 1 -- 172.21.15.7:0/3854186957 shutdown_connections 2024-06-10T01:18:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.667+0000 7f1b813b3b80 1 --2- 172.21.15.7:0/3854186957 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5589007b2f30 0x5589007b5320 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.667+0000 7f1b813b3b80 1 --2- 172.21.15.7:0/3854186957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55890079a2c0 0x55890078d9c0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.667+0000 7f1b4ffff700 0 client.18525 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:18:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.667+0000 7f1b813b3b80 1 --2- 172.21.15.7:0/3854186957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558900795330 0x558900794680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.667+0000 7f1b813b3b80 1 --2- 172.21.15.7:0/3854186957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589007999a0 0x558900794bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.667+0000 7f1b813b3b80 1 -- 172.21.15.7:0/3854186957 >> 172.21.15.7:0/3854186957 conn(0x5589006bbcc0 msgr2=0x55890079a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.667+0000 7f1b813b3b80 1 -- 172.21.15.7:0/3854186957 shutdown_connections 2024-06-10T01:18:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.667+0000 7f1b813b3b80 1 -- 172.21.15.7:0/3854186957 wait complete. 2024-06-10T01:18:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.667+0000 7f1b813b3b80 20 client.18525 trim_cache size 0 max 16384 2024-06-10T01:18:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.682+0000 7f1b813b3b80 1 Processor -- start 2024-06-10T01:18:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.682+0000 7f1b813b3b80 1 -- start start 2024-06-10T01:18:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.682+0000 7f1b813b3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558900795330 0x558900785510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.682+0000 7f1b813b3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589007999a0 0x5589007a1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.682+0000 7f1b813b3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55890079a2c0 0x55890079eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.682+0000 7f1b813b3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701ae7b0 con 0x55890079a2c0 2024-06-10T01:18:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.682+0000 7f1b813b3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701ac040 con 0x558900795330 2024-06-10T01:18:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.682+0000 7f1b813b3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701a9e90 con 0x5589007999a0 2024-06-10T01:18:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.683+0000 7f1b750cd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55890079a2c0 0x55890079eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.683+0000 7f1b750cd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55890079a2c0 0x55890079eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45578/0 (socket says 172.21.15.7:45578) 2024-06-10T01:18:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.683+0000 7f1b750cd700 1 -- 172.21.15.7:0/283942725 learned_addr learned my addr 172.21.15.7:0/283942725 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.683+0000 7f1b748cc700 1 --2- 172.21.15.7:0/283942725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558900795330 0x558900785510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.683+0000 7f1b6ffff700 1 --2- 172.21.15.7:0/283942725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589007999a0 0x5589007a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.683+0000 7f1b750cd700 1 -- 172.21.15.7:0/283942725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589007999a0 msgr2=0x5589007a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.683+0000 7f1b750cd700 1 --2- 172.21.15.7:0/283942725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589007999a0 0x5589007a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.683+0000 7f1b750cd700 1 -- 172.21.15.7:0/283942725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558900795330 msgr2=0x558900785510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.683+0000 7f1b750cd700 1 --2- 172.21.15.7:0/283942725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558900795330 0x558900785510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.683+0000 7f1b750cd700 1 -- 172.21.15.7:0/283942725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55890078d470 con 0x55890079a2c0 2024-06-10T01:18:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.683+0000 7f1b6ffff700 1 --2- 172.21.15.7:0/283942725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589007999a0 0x5589007a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:18:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.683+0000 7f1b750cd700 1 --2- 172.21.15.7:0/283942725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55890079a2c0 0x55890079eb10 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f1b68009460 tx=0x7f1b680099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc33bc8860712732 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.683+0000 7f1b5a7fc700 1 -- 172.21.15.7:0/283942725 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6802a030 con 0x55890079a2c0 2024-06-10T01:18:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.683+0000 7f1b5a7fc700 1 -- 172.21.15.7:0/283942725 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b68020040 con 0x55890079a2c0 2024-06-10T01:18:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.683+0000 7f1b5a7fc700 1 -- 172.21.15.7:0/283942725 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68022040 con 0x55890079a2c0 2024-06-10T01:18:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.684+0000 7f1b813b3b80 1 -- 172.21.15.7:0/283942725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55890079a2c0 msgr2=0x55890079eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.684+0000 7f1b813b3b80 1 --2- 172.21.15.7:0/283942725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55890079a2c0 0x55890079eb10 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f1b68009460 tx=0x7f1b680099e0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.684+0000 7f1b813b3b80 1 -- 172.21.15.7:0/283942725 shutdown_connections 2024-06-10T01:18:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.684+0000 7f1b813b3b80 1 --2- 172.21.15.7:0/283942725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55890079a2c0 0x55890079eb10 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.684+0000 7f1b813b3b80 1 --2- 172.21.15.7:0/283942725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558900795330 0x558900785510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.684+0000 7f1b813b3b80 1 --2- 172.21.15.7:0/283942725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589007999a0 0x5589007a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.684+0000 7f1b813b3b80 1 -- 172.21.15.7:0/283942725 >> 172.21.15.7:0/283942725 conn(0x5589006bbcc0 msgr2=0x558900784e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.684+0000 7f1b813b3b80 1 -- 172.21.15.7:0/283942725 shutdown_connections 2024-06-10T01:18:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.684+0000 7f1b813b3b80 1 -- 172.21.15.7:0/283942725 wait complete. 2024-06-10T01:18:11.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.684+0000 7f1b813b3b80 1 -- start start 2024-06-10T01:18:11.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.685+0000 7f1b813b3b80 1 Processor -- start 2024-06-10T01:18:11.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.685+0000 7f1b813b3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558900795330 0x5589007acb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:11.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.685+0000 7f1b813b3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589007999a0 0x5589007ad040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:11.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.685+0000 7f1b813b3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55890079a2c0 0x5589007ad580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:11.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.685+0000 7f1b813b3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701aa500 con 0x55890079a2c0 2024-06-10T01:18:11.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.685+0000 7f1b813b3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701ab970 con 0x558900795330 2024-06-10T01:18:11.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.685+0000 7f1b813b3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701aabd0 con 0x5589007999a0 2024-06-10T01:18:11.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.685+0000 7f1b813b3b80 10 client.0 _ll_register_callbacks cb 0x55890067c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:18:11.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.685+0000 7f1b750cd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55890079a2c0 0x5589007ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:11.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.685+0000 7f1b750cd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55890079a2c0 0x5589007ad580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45588/0 (socket says 172.21.15.7:45588) 2024-06-10T01:18:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.685+0000 7f1b750cd700 1 -- 172.21.15.7:0/3981135984 learned_addr learned my addr 172.21.15.7:0/3981135984 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.685+0000 7f1b748cc700 1 --2- 172.21.15.7:0/3981135984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558900795330 0x5589007acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.685+0000 7f1b6ffff700 1 --2- 172.21.15.7:0/3981135984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589007999a0 0x5589007ad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.685+0000 7f1b750cd700 1 -- 172.21.15.7:0/3981135984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589007999a0 msgr2=0x5589007ad040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.685+0000 7f1b750cd700 1 --2- 172.21.15.7:0/3981135984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589007999a0 0x5589007ad040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.685+0000 7f1b750cd700 1 -- 172.21.15.7:0/3981135984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558900795330 msgr2=0x5589007acb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.685+0000 7f1b750cd700 1 --2- 172.21.15.7:0/3981135984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558900795330 0x5589007acb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.686+0000 7f1b750cd700 1 -- 172.21.15.7:0/3981135984 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55890081b9d0 con 0x55890079a2c0 2024-06-10T01:18:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.686+0000 7f1b748cc700 1 --2- 172.21.15.7:0/3981135984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558900795330 0x5589007acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:18:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.686+0000 7f1b750cd700 1 --2- 172.21.15.7:0/3981135984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55890079a2c0 0x5589007ad580 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f1b68004d00 tx=0x7f1b68005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897b58195bbedc70 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.686+0000 7f1b4ffff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:18:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.686+0000 7f1b4ffff700 1 -- 172.21.15.7:0/3981135984 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68022040 con 0x55890079a2c0 2024-06-10T01:18:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.686+0000 7f1b4ffff700 1 -- 172.21.15.7:0/3981135984 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f1b680056a0 con 0x55890079a2c0 2024-06-10T01:18:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.686+0000 7f1b813b3b80 1 -- 172.21.15.7:0/3981135984 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b68020040 con 0x55890079a2c0 2024-06-10T01:18:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.686+0000 7f1b4ffff700 1 -- 172.21.15.7:0/3981135984 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f1b68047050 con 0x55890079a2c0 2024-06-10T01:18:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.686+0000 7f1b813b3b80 10 client.18531 fetch_fsmap learned FSMap version 40 2024-06-10T01:18:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.686+0000 7f1b813b3b80 1 -- 172.21.15.7:0/3981135984 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55890081cc20 con 0x55890079a2c0 2024-06-10T01:18:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.686+0000 7f1b4ffff700 1 -- 172.21.15.7:0/3981135984 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b68020040 con 0x55890079a2c0 2024-06-10T01:18:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.686+0000 7f1b4ffff700 1 -- 172.21.15.7:0/3981135984 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f1b68042030 con 0x55890079a2c0 2024-06-10T01:18:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.686+0000 7f1b813b3b80 10 client.18531 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:18:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.686+0000 7f1b813b3b80 10 client.18531 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:18:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.687+0000 7f1b813b3b80 20 client.18531 populate_metadata read hostname 'smithi007' 2024-06-10T01:18:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.687+0000 7f1b813b3b80 1 --2- 172.21.15.7:0/3981135984 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5589007b2f30 0x55890081f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.687+0000 7f1b813b3b80 4 client.18531 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:18:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.687+0000 7f1b813b3b80 1 -- 172.21.15.7:0/3981135984 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5589006ea290 con 0x5589007b2f30 2024-06-10T01:18:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.687+0000 7f1b748cc700 1 --2- 172.21.15.7:0/3981135984 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5589007b2f30 0x55890081f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.687+0000 7f1b748cc700 1 --2- 172.21.15.7:0/3981135984 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5589007b2f30 0x55890081f2a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=22829c034f974890 server_cookie=5b6c988cff75a422 in_seq=0 out_seq=0 2024-06-10T01:18:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.687+0000 7f1b4ffff700 10 client.18531 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:18:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.687+0000 7f1b4ffff700 1 -- 172.21.15.7:0/3981135984 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5589006ea290 con 0x5589007b2f30 2024-06-10T01:18:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.687+0000 7f1b4ffff700 10 client.18531 handle_command_reply: tid=0 2024-06-10T01:18:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.687+0000 7f1b813b3b80 1 client.18531 shutdown 2024-06-10T01:18:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.687+0000 7f1b813b3b80 1 -- 172.21.15.7:0/3981135984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55890079a2c0 msgr2=0x5589007ad580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.687+0000 7f1b813b3b80 1 --2- 172.21.15.7:0/3981135984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55890079a2c0 0x5589007ad580 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f1b68004d00 tx=0x7f1b68005f10 comp rx=0 tx=0).stop 2024-06-10T01:18:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.687+0000 7f1b813b3b80 1 -- 172.21.15.7:0/3981135984 shutdown_connections 2024-06-10T01:18:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.687+0000 7f1b813b3b80 1 --2- 172.21.15.7:0/3981135984 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5589007b2f30 0x55890081f2a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.687+0000 7f1b4ffff700 0 client.18531 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:18:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.687+0000 7f1b813b3b80 1 --2- 172.21.15.7:0/3981135984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55890079a2c0 0x5589007ad580 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.687+0000 7f1b813b3b80 1 --2- 172.21.15.7:0/3981135984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558900795330 0x5589007acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.687+0000 7f1b813b3b80 1 --2- 172.21.15.7:0/3981135984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589007999a0 0x5589007ad040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.687+0000 7f1b813b3b80 1 -- 172.21.15.7:0/3981135984 >> 172.21.15.7:0/3981135984 conn(0x5589006bbcc0 msgr2=0x558900793130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:11.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.688+0000 7f1b813b3b80 1 -- 172.21.15.7:0/3981135984 shutdown_connections 2024-06-10T01:18:11.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.688+0000 7f1b813b3b80 1 -- 172.21.15.7:0/3981135984 wait complete. 2024-06-10T01:18:11.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.688+0000 7f1b813b3b80 20 client.18531 trim_cache size 0 max 16384 2024-06-10T01:18:11.689 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:18:11.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.688+0000 7f1b76b30700 1 -- 172.21.15.7:0/2025451020 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1b600a2c20 msgr2=0x7f1b600a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:11.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.688+0000 7f1b76b30700 1 --2- 172.21.15.7:0/2025451020 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1b600a2c20 0x7f1b600a50e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1b70108bd0 tx=0x7f1b5c00a380 comp rx=0 tx=0).stop 2024-06-10T01:18:11.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.688+0000 7f1b76b30700 1 -- 172.21.15.7:0/2025451020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b70102740 msgr2=0x7f1b7019e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:11.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.688+0000 7f1b76b30700 1 --2- 172.21.15.7:0/2025451020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b70102740 0x7f1b7019e750 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400d980 tx=0x7f1b6400dc90 comp rx=0 tx=0).stop 2024-06-10T01:18:11.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.688+0000 7f1b76b30700 1 -- 172.21.15.7:0/2025451020 shutdown_connections 2024-06-10T01:18:11.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.688+0000 7f1b76b30700 1 --2- 172.21.15.7:0/2025451020 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1b600a2c20 0x7f1b600a50e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.688+0000 7f1b76b30700 1 --2- 172.21.15.7:0/2025451020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b70107840 0x7f1b70198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.688+0000 7f1b76b30700 1 --2- 172.21.15.7:0/2025451020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b70102740 0x7f1b7019e750 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.688+0000 7f1b76b30700 1 --2- 172.21.15.7:0/2025451020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b701030f0 0x7f1b7019ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:11.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.688+0000 7f1b76b30700 1 -- 172.21.15.7:0/2025451020 >> 172.21.15.7:0/2025451020 conn(0x7f1b700795f0 msgr2=0x7f1b70100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:11.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.688+0000 7f1b76b30700 1 -- 172.21.15.7:0/2025451020 shutdown_connections 2024-06-10T01:18:11.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:11.689+0000 7f1b76b30700 1 -- 172.21.15.7:0/2025451020 wait complete. 2024-06-10T01:18:11.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:11 smithi175 ceph-mon[36636]: pgmap v797: 129 pgs: 129 active+clean; 146 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 1.7 MiB/s wr, 87 op/s 2024-06-10T01:18:11.754 INFO:tasks.workunit.client.0.smithi007.stdout: 1 881536 7.20 MB/sec execute 532 sec latency 27557.523 ms 2024-06-10T01:18:11.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:11 smithi078 ceph-mon[36683]: pgmap v797: 129 pgs: 129 active+clean; 146 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 1.7 MiB/s wr, 87 op/s 2024-06-10T01:18:12.754 INFO:tasks.workunit.client.0.smithi007.stdout: 1 881536 7.18 MB/sec execute 533 sec latency 28557.548 ms 2024-06-10T01:18:13.146 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:18:13.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:13 smithi175 ceph-mon[36636]: pgmap v798: 129 pgs: 129 active+clean; 146 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 1.5 MiB/s wr, 72 op/s 2024-06-10T01:18:13.755 INFO:tasks.workunit.client.0.smithi007.stdout: 1 881536 7.17 MB/sec execute 534 sec latency 29557.619 ms 2024-06-10T01:18:13.759 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:13 smithi007 ceph-mon[32398]: pgmap v798: 129 pgs: 129 active+clean; 146 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 1.5 MiB/s wr, 72 op/s 2024-06-10T01:18:13.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:13 smithi078 ceph-mon[36683]: pgmap v798: 129 pgs: 129 active+clean; 146 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 1.5 MiB/s wr, 72 op/s 2024-06-10T01:18:14.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.157+0000 7fe9d0238700 1 -- 172.21.15.7:0/2191784568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9c8101fd0 msgr2=0x7fe9c8106000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:14.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.157+0000 7fe9d0238700 1 --2- 172.21.15.7:0/2191784568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9c8101fd0 0x7fe9c8106000 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009990 tx=0x7fe9b8009ca0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.157+0000 7fe9d0238700 1 -- 172.21.15.7:0/2191784568 shutdown_connections 2024-06-10T01:18:14.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.157+0000 7fe9d0238700 1 --2- 172.21.15.7:0/2191784568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9c81066e0 0x7fe9c810e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.157+0000 7fe9d0238700 1 --2- 172.21.15.7:0/2191784568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9c8101620 0x7fe9c8101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.157+0000 7fe9d0238700 1 --2- 172.21.15.7:0/2191784568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9c8101fd0 0x7fe9c8106000 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.157+0000 7fe9d0238700 1 -- 172.21.15.7:0/2191784568 >> 172.21.15.7:0/2191784568 conn(0x7fe9c8078330 msgr2=0x7fe9c8078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:14.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.157+0000 7fe9d0238700 1 -- 172.21.15.7:0/2191784568 shutdown_connections 2024-06-10T01:18:14.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.157+0000 7fe9d0238700 1 -- 172.21.15.7:0/2191784568 wait complete. 2024-06-10T01:18:14.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.157+0000 7fe9d0238700 1 Processor -- start 2024-06-10T01:18:14.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.158+0000 7fe9d0238700 1 -- start start 2024-06-10T01:18:14.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.158+0000 7fe9d0238700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9c8101620 0x7fe9c819e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:14.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.158+0000 7fe9d0238700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9c8101fd0 0x7fe9c819ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:14.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.158+0000 7fe9d0238700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9c81066e0 0x7fe9c8198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:14.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.158+0000 7fe9d0238700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c8111680 con 0x7fe9c81066e0 2024-06-10T01:18:14.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.158+0000 7fe9d0238700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c8111500 con 0x7fe9c8101fd0 2024-06-10T01:18:14.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.158+0000 7fe9d0238700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c8111800 con 0x7fe9c8101620 2024-06-10T01:18:14.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.158+0000 7fe9ce7d5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9c81066e0 0x7fe9c8198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:14.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.158+0000 7fe9ce7d5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9c81066e0 0x7fe9c8198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44996/0 (socket says 172.21.15.7:44996) 2024-06-10T01:18:14.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.158+0000 7fe9ce7d5700 1 -- 172.21.15.7:0/851486846 learned_addr learned my addr 172.21.15.7:0/851486846 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:14.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.158+0000 7fe9cd7d3700 1 --2- 172.21.15.7:0/851486846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9c8101fd0 0x7fe9c819ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:14.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.158+0000 7fe9cdfd4700 1 --2- 172.21.15.7:0/851486846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9c8101620 0x7fe9c819e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:14.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.158+0000 7fe9ce7d5700 1 -- 172.21.15.7:0/851486846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9c8101620 msgr2=0x7fe9c819e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:14.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.158+0000 7fe9ce7d5700 1 --2- 172.21.15.7:0/851486846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9c8101620 0x7fe9c819e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.158+0000 7fe9ce7d5700 1 -- 172.21.15.7:0/851486846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9c8101fd0 msgr2=0x7fe9c819ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:14.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.158+0000 7fe9ce7d5700 1 --2- 172.21.15.7:0/851486846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9c8101fd0 0x7fe9c819ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.158+0000 7fe9ce7d5700 1 -- 172.21.15.7:0/851486846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c8199220 con 0x7fe9c81066e0 2024-06-10T01:18:14.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.159+0000 7fe9ce7d5700 1 --2- 172.21.15.7:0/851486846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9c81066e0 0x7fe9c8198ac0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000ef30 tx=0x7fe9c000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:14.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.159+0000 7fe9beffd700 1 -- 172.21.15.7:0/851486846 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c00092e0 con 0x7fe9c81066e0 2024-06-10T01:18:14.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.159+0000 7fe9beffd700 1 -- 172.21.15.7:0/851486846 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c0004bb0 con 0x7fe9c81066e0 2024-06-10T01:18:14.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.159+0000 7fe9d0238700 1 -- 172.21.15.7:0/851486846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9c81994b0 con 0x7fe9c81066e0 2024-06-10T01:18:14.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.159+0000 7fe9beffd700 1 -- 172.21.15.7:0/851486846 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0004280 con 0x7fe9c81066e0 2024-06-10T01:18:14.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.159+0000 7fe9d0238700 1 -- 172.21.15.7:0/851486846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9c81a57d0 con 0x7fe9c81066e0 2024-06-10T01:18:14.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.161+0000 7fe9beffd700 1 -- 172.21.15.7:0/851486846 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fe9c001b030 con 0x7fe9c81066e0 2024-06-10T01:18:14.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.162+0000 7fe9beffd700 1 --2- 172.21.15.7:0/851486846 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe9b40a2c20 0x7fe9b40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:14.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.162+0000 7fe9cdfd4700 1 --2- 172.21.15.7:0/851486846 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe9b40a2c20 0x7fe9b40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:14.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.162+0000 7fe9beffd700 1 -- 172.21.15.7:0/851486846 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fe9c00cee20 con 0x7fe9c81066e0 2024-06-10T01:18:14.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.163+0000 7fe9daabbb80 1 Processor -- start 2024-06-10T01:18:14.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.163+0000 7fe9cdfd4700 1 --2- 172.21.15.7:0/851486846 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe9b40a2c20 0x7fe9b40a50e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8199c00 tx=0x7fe9c400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:14.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.163+0000 7fe9daabbb80 1 -- start start 2024-06-10T01:18:14.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.163+0000 7fe9daabbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d117d2f050 0x55d117d2f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:14.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.163+0000 7fe9daabbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d117d2f970 0x55d117d292a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:14.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.163+0000 7fe9daabbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d117d297e0 0x55d117d29c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:14.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.163+0000 7fe9daabbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d117c76910 con 0x55d117d297e0 2024-06-10T01:18:14.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.163+0000 7fe9daabbb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d117c1e820 con 0x55d117d2f970 2024-06-10T01:18:14.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.163+0000 7fe9daabbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d117c728d0 con 0x55d117d2f050 2024-06-10T01:18:14.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.163+0000 7fe9ce7d5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d117d297e0 0x55d117d29c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:14.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.163+0000 7fe9ce7d5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d117d297e0 0x55d117d29c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45012/0 (socket says 172.21.15.7:45012) 2024-06-10T01:18:14.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.163+0000 7fe9ce7d5700 1 -- 172.21.15.7:0/2771149155 learned_addr learned my addr 172.21.15.7:0/2771149155 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:14.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.163+0000 7fe9cd7d3700 1 --2- 172.21.15.7:0/2771149155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d117d2f970 0x55d117d292a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:14.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.163+0000 7fe9cdfd4700 1 --2- 172.21.15.7:0/2771149155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d117d2f050 0x55d117d2f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:14.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.164+0000 7fe9ce7d5700 1 -- 172.21.15.7:0/2771149155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d117d2f050 msgr2=0x55d117d2f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:14.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.164+0000 7fe9ce7d5700 1 --2- 172.21.15.7:0/2771149155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d117d2f050 0x55d117d2f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.164+0000 7fe9ce7d5700 1 -- 172.21.15.7:0/2771149155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d117d2f970 msgr2=0x55d117d292a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:14.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.164+0000 7fe9ce7d5700 1 --2- 172.21.15.7:0/2771149155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d117d2f970 0x55d117d292a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.164+0000 7fe9ce7d5700 1 -- 172.21.15.7:0/2771149155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d117c15050 con 0x55d117d297e0 2024-06-10T01:18:14.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.164+0000 7fe9cdfd4700 1 --2- 172.21.15.7:0/2771149155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d117d2f050 0x55d117d2f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:18:14.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.164+0000 7fe9ce7d5700 1 --2- 172.21.15.7:0/2771149155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d117d297e0 0x55d117d29c20 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00090a0 tx=0x7fe9c009acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2296bf6f2389c9dd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:14.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.164+0000 7fe9b3fff700 1 -- 172.21.15.7:0/2771149155 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c009ba90 con 0x55d117d297e0 2024-06-10T01:18:14.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.164+0000 7fe9b3fff700 1 -- 172.21.15.7:0/2771149155 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c009b390 con 0x55d117d297e0 2024-06-10T01:18:14.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.164+0000 7fe9b3fff700 1 -- 172.21.15.7:0/2771149155 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c00a3ae0 con 0x55d117d297e0 2024-06-10T01:18:14.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.165+0000 7fe9daabbb80 1 -- 172.21.15.7:0/2771149155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d117d297e0 msgr2=0x55d117d29c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:14.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.165+0000 7fe9daabbb80 1 --2- 172.21.15.7:0/2771149155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d117d297e0 0x55d117d29c20 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00090a0 tx=0x7fe9c009acf0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.165+0000 7fe9daabbb80 1 -- 172.21.15.7:0/2771149155 shutdown_connections 2024-06-10T01:18:14.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.165+0000 7fe9daabbb80 1 --2- 172.21.15.7:0/2771149155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d117d297e0 0x55d117d29c20 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.165+0000 7fe9daabbb80 1 --2- 172.21.15.7:0/2771149155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d117d2f050 0x55d117d2f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.165+0000 7fe9daabbb80 1 --2- 172.21.15.7:0/2771149155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d117d2f970 0x55d117d292a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.165+0000 7fe9daabbb80 1 -- 172.21.15.7:0/2771149155 >> 172.21.15.7:0/2771149155 conn(0x55d117c4fdc0 msgr2=0x55d117d20120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:14.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.165+0000 7fe9daabbb80 1 -- 172.21.15.7:0/2771149155 shutdown_connections 2024-06-10T01:18:14.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.165+0000 7fe9daabbb80 1 -- 172.21.15.7:0/2771149155 wait complete. 2024-06-10T01:18:14.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.166+0000 7fe9daabbb80 1 -- start start 2024-06-10T01:18:14.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.166+0000 7fe9daabbb80 1 Processor -- start 2024-06-10T01:18:14.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.166+0000 7fe9daabbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d117d297e0 0x55d117d35640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:14.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.166+0000 7fe9daabbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d117d2f050 0x55d117d329b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:14.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.166+0000 7fe9daabbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d117d2f970 0x55d117d32ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:14.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.166+0000 7fe9daabbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d117d363e0 con 0x55d117d2f050 2024-06-10T01:18:14.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.166+0000 7fe9daabbb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d117c16cf0 con 0x55d117d297e0 2024-06-10T01:18:14.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.166+0000 7fe9daabbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d117d36560 con 0x55d117d2f970 2024-06-10T01:18:14.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.167+0000 7fe9daabbb80 10 client.0 _ll_register_callbacks cb 0x55d117c10a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:18:14.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.167+0000 7fe9cd7d3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d117d2f050 0x55d117d329b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:14.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.167+0000 7fe9cd7d3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d117d2f050 0x55d117d329b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45018/0 (socket says 172.21.15.7:45018) 2024-06-10T01:18:14.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.167+0000 7fe9cdfd4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d117d297e0 0x55d117d35640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:14.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.167+0000 7fe9cd7d3700 1 -- 172.21.15.7:0/1819891689 learned_addr learned my addr 172.21.15.7:0/1819891689 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:14.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.167+0000 7fe9ce7d5700 1 --2- 172.21.15.7:0/1819891689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d117d2f970 0x55d117d32ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:14.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.167+0000 7fe9cdfd4700 1 -- 172.21.15.7:0/1819891689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d117d2f970 msgr2=0x55d117d32ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:14.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.167+0000 7fe9cdfd4700 1 --2- 172.21.15.7:0/1819891689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d117d2f970 0x55d117d32ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.167+0000 7fe9cdfd4700 1 -- 172.21.15.7:0/1819891689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d117d2f050 msgr2=0x55d117d329b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:14.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.167+0000 7fe9cdfd4700 1 --2- 172.21.15.7:0/1819891689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d117d2f050 0x55d117d329b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.167+0000 7fe9cdfd4700 1 -- 172.21.15.7:0/1819891689 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d117bc59d0 con 0x55d117d297e0 2024-06-10T01:18:14.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.167+0000 7fe9cd7d3700 1 --2- 172.21.15.7:0/1819891689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d117d2f050 0x55d117d329b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:18:14.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.167+0000 7fe9cdfd4700 1 --2- 172.21.15.7:0/1819891689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d117d297e0 0x55d117d35640 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4005950 tx=0x7fe9c40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ef671ed8f8b20ce server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:14.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.167+0000 7fe9b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:18:14.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.167+0000 7fe9b17fa700 1 -- 172.21.15.7:0/1819891689 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4028670 con 0x55d117d297e0 2024-06-10T01:18:14.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.167+0000 7fe9b17fa700 1 -- 172.21.15.7:0/1819891689 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fe9c4028810 con 0x55d117d297e0 2024-06-10T01:18:14.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.168+0000 7fe9daabbb80 1 -- 172.21.15.7:0/1819891689 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9c009b390 con 0x55d117d297e0 2024-06-10T01:18:14.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.168+0000 7fe9b17fa700 1 -- 172.21.15.7:0/1819891689 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe9c4031050 con 0x55d117d297e0 2024-06-10T01:18:14.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.168+0000 7fe9daabbb80 10 client.27860 fetch_fsmap learned FSMap version 40 2024-06-10T01:18:14.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.168+0000 7fe9daabbb80 1 -- 172.21.15.7:0/1819891689 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d117d40d80 con 0x55d117d297e0 2024-06-10T01:18:14.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.168+0000 7fe9b17fa700 1 -- 172.21.15.7:0/1819891689 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c4023070 con 0x55d117d297e0 2024-06-10T01:18:14.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.168+0000 7fe9b17fa700 1 -- 172.21.15.7:0/1819891689 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7fe9c401fa90 con 0x55d117d297e0 2024-06-10T01:18:14.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.168+0000 7fe9daabbb80 10 client.27860 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:18:14.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.168+0000 7fe9daabbb80 10 client.27860 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:18:14.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.168+0000 7fe9daabbb80 20 client.27860 populate_metadata read hostname 'smithi007' 2024-06-10T01:18:14.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.168+0000 7fe9daabbb80 1 --2- 172.21.15.7:0/1819891689 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d117d433c0 0x55d117d457b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:14.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.168+0000 7fe9daabbb80 4 client.27860 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:18:14.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.168+0000 7fe9daabbb80 1 -- 172.21.15.7:0/1819891689 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe9c009b390 con 0x55d117d433c0 2024-06-10T01:18:14.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.169+0000 7fe9cd7d3700 1 --2- 172.21.15.7:0/1819891689 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d117d433c0 0x55d117d457b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:14.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.169+0000 7fe9cd7d3700 1 --2- 172.21.15.7:0/1819891689 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d117d433c0 0x55d117d457b0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b8bf183130cc75 server_cookie=85d57eab6f8ab4e6 in_seq=0 out_seq=0 2024-06-10T01:18:14.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.169+0000 7fe9b17fa700 10 client.27860 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:14.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.170+0000 7fe9b17fa700 1 -- 172.21.15.7:0/1819891689 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fe9c009b390 con 0x55d117d433c0 2024-06-10T01:18:14.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.170+0000 7fe9b17fa700 10 client.27860 handle_command_reply: tid=0 2024-06-10T01:18:14.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.170+0000 7fe9daabbb80 1 client.27860 shutdown 2024-06-10T01:18:14.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.170+0000 7fe9daabbb80 1 -- 172.21.15.7:0/1819891689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d117d297e0 msgr2=0x55d117d35640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:14.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.170+0000 7fe9daabbb80 1 --2- 172.21.15.7:0/1819891689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d117d297e0 0x55d117d35640 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4005950 tx=0x7fe9c40047c0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.170+0000 7fe9daabbb80 1 -- 172.21.15.7:0/1819891689 shutdown_connections 2024-06-10T01:18:14.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.170+0000 7fe9daabbb80 1 --2- 172.21.15.7:0/1819891689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d117d2f970 0x55d117d32ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.170+0000 7fe9daabbb80 1 --2- 172.21.15.7:0/1819891689 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d117d433c0 0x55d117d457b0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.170+0000 7fe9daabbb80 1 --2- 172.21.15.7:0/1819891689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d117d297e0 0x55d117d35640 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.170+0000 7fe9daabbb80 1 --2- 172.21.15.7:0/1819891689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d117d2f050 0x55d117d329b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.170+0000 7fe9b17fa700 0 client.27860 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:14.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.170+0000 7fe9daabbb80 1 -- 172.21.15.7:0/1819891689 >> 172.21.15.7:0/1819891689 conn(0x55d117c4fdc0 msgr2=0x55d117d1fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:14.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.171+0000 7fe9daabbb80 1 -- 172.21.15.7:0/1819891689 shutdown_connections 2024-06-10T01:18:14.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.171+0000 7fe9daabbb80 1 -- 172.21.15.7:0/1819891689 wait complete. 2024-06-10T01:18:14.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.171+0000 7fe9daabbb80 20 client.27860 trim_cache size 0 max 16384 2024-06-10T01:18:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.192+0000 7fe9daabbb80 1 Processor -- start 2024-06-10T01:18:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.192+0000 7fe9daabbb80 1 -- start start 2024-06-10T01:18:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.192+0000 7fe9daabbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d117d297e0 0x55d117d303c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.192+0000 7fe9daabbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d117d2f050 0x55d117d4b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.192+0000 7fe9daabbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d117d2f970 0x55d117d4b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.192+0000 7fe9daabbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c81aea10 con 0x55d117d2f050 2024-06-10T01:18:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.192+0000 7fe9daabbb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c81ac2a0 con 0x55d117d2f970 2024-06-10T01:18:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.192+0000 7fe9daabbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c81aa110 con 0x55d117d297e0 2024-06-10T01:18:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.192+0000 7fe9cd7d3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d117d2f050 0x55d117d4b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:14.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.192+0000 7fe9cdfd4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d117d297e0 0x55d117d303c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:14.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.192+0000 7fe9cd7d3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d117d2f050 0x55d117d4b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45034/0 (socket says 172.21.15.7:45034) 2024-06-10T01:18:14.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.192+0000 7fe9cd7d3700 1 -- 172.21.15.7:0/3254935742 learned_addr learned my addr 172.21.15.7:0/3254935742 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.193+0000 7fe9cdfd4700 1 -- 172.21.15.7:0/3254935742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d117d2f970 msgr2=0x55d117d4b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.193+0000 7fe9cdfd4700 1 --2- 172.21.15.7:0/3254935742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d117d2f970 0x55d117d4b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.193+0000 7fe9cdfd4700 1 -- 172.21.15.7:0/3254935742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d117d2f050 msgr2=0x55d117d4b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.193+0000 7fe9cdfd4700 1 --2- 172.21.15.7:0/3254935742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d117d2f050 0x55d117d4b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.193+0000 7fe9cdfd4700 1 -- 172.21.15.7:0/3254935742 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d117d20660 con 0x55d117d297e0 2024-06-10T01:18:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.193+0000 7fe9cdfd4700 1 --2- 172.21.15.7:0/3254935742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d117d297e0 0x55d117d303c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4022040 tx=0x7fe9c4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dfa6245aaac55c4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.193+0000 7fe9b37fe700 1 -- 172.21.15.7:0/3254935742 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4039020 con 0x55d117d297e0 2024-06-10T01:18:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.193+0000 7fe9b37fe700 1 -- 172.21.15.7:0/3254935742 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c4023040 con 0x55d117d297e0 2024-06-10T01:18:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.193+0000 7fe9b37fe700 1 -- 172.21.15.7:0/3254935742 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c40266e0 con 0x55d117d297e0 2024-06-10T01:18:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.194+0000 7fe9daabbb80 1 -- 172.21.15.7:0/3254935742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d117d297e0 msgr2=0x55d117d303c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.194+0000 7fe9daabbb80 1 --2- 172.21.15.7:0/3254935742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d117d297e0 0x55d117d303c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4022040 tx=0x7fe9c4004750 comp rx=0 tx=0).stop 2024-06-10T01:18:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.194+0000 7fe9daabbb80 1 -- 172.21.15.7:0/3254935742 shutdown_connections 2024-06-10T01:18:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.194+0000 7fe9daabbb80 1 --2- 172.21.15.7:0/3254935742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d117d2f970 0x55d117d4b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.194+0000 7fe9daabbb80 1 --2- 172.21.15.7:0/3254935742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d117d297e0 0x55d117d303c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.194+0000 7fe9daabbb80 1 --2- 172.21.15.7:0/3254935742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d117d2f050 0x55d117d4b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.194+0000 7fe9daabbb80 1 -- 172.21.15.7:0/3254935742 >> 172.21.15.7:0/3254935742 conn(0x55d117c4fdc0 msgr2=0x55d117d24710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.194+0000 7fe9daabbb80 1 -- 172.21.15.7:0/3254935742 shutdown_connections 2024-06-10T01:18:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.194+0000 7fe9daabbb80 1 -- 172.21.15.7:0/3254935742 wait complete. 2024-06-10T01:18:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.195+0000 7fe9daabbb80 1 -- start start 2024-06-10T01:18:14.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.195+0000 7fe9daabbb80 1 Processor -- start 2024-06-10T01:18:14.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.195+0000 7fe9daabbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d117d297e0 0x55d117d3a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:14.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.195+0000 7fe9daabbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d117d2f050 0x55d117d3ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:14.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.195+0000 7fe9daabbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d117d2f970 0x55d117d3b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:14.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.196+0000 7fe9daabbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c81aa760 con 0x55d117d2f970 2024-06-10T01:18:14.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.196+0000 7fe9daabbb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c81abbd0 con 0x55d117d297e0 2024-06-10T01:18:14.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.196+0000 7fe9daabbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c81aae30 con 0x55d117d2f050 2024-06-10T01:18:14.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.196+0000 7fe9ce7d5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d117d2f970 0x55d117d3b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:14.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.196+0000 7fe9daabbb80 10 client.0 _ll_register_callbacks cb 0x55d117c10a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:18:14.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.196+0000 7fe9ce7d5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d117d2f970 0x55d117d3b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45048/0 (socket says 172.21.15.7:45048) 2024-06-10T01:18:14.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.196+0000 7fe9ce7d5700 1 -- 172.21.15.7:0/4275158225 learned_addr learned my addr 172.21.15.7:0/4275158225 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:14.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.196+0000 7fe9cdfd4700 1 --2- 172.21.15.7:0/4275158225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d117d297e0 0x55d117d3a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:14.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.196+0000 7fe9cd7d3700 1 --2- 172.21.15.7:0/4275158225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d117d2f050 0x55d117d3ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:14.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.196+0000 7fe9ce7d5700 1 -- 172.21.15.7:0/4275158225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d117d2f050 msgr2=0x55d117d3ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:14.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.196+0000 7fe9ce7d5700 1 --2- 172.21.15.7:0/4275158225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d117d2f050 0x55d117d3ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.196+0000 7fe9ce7d5700 1 -- 172.21.15.7:0/4275158225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d117d297e0 msgr2=0x55d117d3a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:14.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.196+0000 7fe9ce7d5700 1 --2- 172.21.15.7:0/4275158225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d117d297e0 0x55d117d3a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.196+0000 7fe9ce7d5700 1 -- 172.21.15.7:0/4275158225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d117d2eb50 con 0x55d117d2f970 2024-06-10T01:18:14.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.196+0000 7fe9cdfd4700 1 --2- 172.21.15.7:0/4275158225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d117d297e0 0x55d117d3a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:18:14.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.196+0000 7fe9ce7d5700 1 --2- 172.21.15.7:0/4275158225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d117d2f970 0x55d117d3b2a0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fe9c009b090 tx=0x7fe9c0097030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6968a9e10730f268 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:14.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.196+0000 7fe9b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:18:14.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.196+0000 7fe9b17fa700 1 -- 172.21.15.7:0/4275158225 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c009ba90 con 0x55d117d2f970 2024-06-10T01:18:14.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.196+0000 7fe9b17fa700 1 -- 172.21.15.7:0/4275158225 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fe9c00b4260 con 0x55d117d2f970 2024-06-10T01:18:14.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.196+0000 7fe9daabbb80 1 -- 172.21.15.7:0/4275158225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9c4023040 con 0x55d117d2f970 2024-06-10T01:18:14.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.197+0000 7fe9b17fa700 1 -- 172.21.15.7:0/4275158225 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe9c00a3f70 con 0x55d117d2f970 2024-06-10T01:18:14.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.197+0000 7fe9daabbb80 10 client.18549 fetch_fsmap learned FSMap version 40 2024-06-10T01:18:14.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.197+0000 7fe9daabbb80 1 -- 172.21.15.7:0/4275158225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d117daf3c0 con 0x55d117d2f970 2024-06-10T01:18:14.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.197+0000 7fe9b17fa700 1 -- 172.21.15.7:0/4275158225 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c4023040 con 0x55d117d2f970 2024-06-10T01:18:14.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.197+0000 7fe9b17fa700 1 -- 172.21.15.7:0/4275158225 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7fe9c0066250 con 0x55d117d2f970 2024-06-10T01:18:14.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.197+0000 7fe9daabbb80 10 client.18549 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:18:14.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.197+0000 7fe9daabbb80 10 client.18549 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:18:14.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.197+0000 7fe9daabbb80 20 client.18549 populate_metadata read hostname 'smithi007' 2024-06-10T01:18:14.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.197+0000 7fe9daabbb80 1 --2- 172.21.15.7:0/4275158225 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d117daf6a0 0x55d117db1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:14.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.197+0000 7fe9daabbb80 4 client.18549 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b6c01176", "release": true, "await": true}] 2024-06-10T01:18:14.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.197+0000 7fe9daabbb80 1 -- 172.21.15.7:0/4275158225 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b6c01176", "release": true, "await": true}) -- 0x55d117d20450 con 0x55d117daf6a0 2024-06-10T01:18:14.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.197+0000 7fe9cdfd4700 1 --2- 172.21.15.7:0/4275158225 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d117daf6a0 0x55d117db1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:14.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.198+0000 7fe9b17fa700 10 client.18549 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:14.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.198+0000 7fe9cdfd4700 1 --2- 172.21.15.7:0/4275158225 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d117daf6a0 0x55d117db1a90 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c1b52ee7b57591b server_cookie=374f214e7aeb8839 in_seq=0 out_seq=0 2024-06-10T01:18:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.205+0000 7fe9b17fa700 1 -- 172.21.15.7:0/4275158225 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d117d20450 con 0x55d117daf6a0 2024-06-10T01:18:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.205+0000 7fe9b17fa700 10 client.18549 handle_command_reply: tid=0 2024-06-10T01:18:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.205+0000 7fe9daabbb80 1 client.18549 shutdown 2024-06-10T01:18:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.205+0000 7fe9daabbb80 1 -- 172.21.15.7:0/4275158225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d117d2f970 msgr2=0x55d117d3b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.205+0000 7fe9daabbb80 1 --2- 172.21.15.7:0/4275158225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d117d2f970 0x55d117d3b2a0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fe9c009b090 tx=0x7fe9c0097030 comp rx=0 tx=0).stop 2024-06-10T01:18:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.205+0000 7fe9daabbb80 1 -- 172.21.15.7:0/4275158225 shutdown_connections 2024-06-10T01:18:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.205+0000 7fe9daabbb80 1 --2- 172.21.15.7:0/4275158225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d117d2f970 0x55d117d3b2a0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.205+0000 7fe9daabbb80 1 --2- 172.21.15.7:0/4275158225 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d117daf6a0 0x55d117db1a90 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.205+0000 7fe9daabbb80 1 --2- 172.21.15.7:0/4275158225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d117d297e0 0x55d117d3a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.205+0000 7fe9daabbb80 1 --2- 172.21.15.7:0/4275158225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d117d2f050 0x55d117d3ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.205+0000 7fe9daabbb80 1 -- 172.21.15.7:0/4275158225 >> 172.21.15.7:0/4275158225 conn(0x55d117c4fdc0 msgr2=0x55d117d24710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.205+0000 7fe9b17fa700 0 client.18549 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.205+0000 7fe9daabbb80 1 -- 172.21.15.7:0/4275158225 shutdown_connections 2024-06-10T01:18:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.205+0000 7fe9daabbb80 1 -- 172.21.15.7:0/4275158225 wait complete. 2024-06-10T01:18:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.206+0000 7fe9daabbb80 20 client.18549 trim_cache size 0 max 16384 2024-06-10T01:18:14.202 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:18:14.202 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 40, 2024-06-10T01:18:14.202 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:18:14.203 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 40, 2024-06-10T01:18:14.203 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:18:14.203 INFO:teuthology.orchestra.run.smithi007.stdout: "b6c01176": { 2024-06-10T01:18:14.203 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 40, 2024-06-10T01:18:14.203 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:18:14.203 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:18:14.203 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:18:14.203 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:18:14.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:18:14.203 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:18:14.203 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 147.2, 2024-06-10T01:18:14.203 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:18:14.203 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:18:14.204 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:18:14.204 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:18:14.204 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:18:14.204 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:18:14.204 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:18:14.204 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:18:14.204 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:18:14.204 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:18:14.204 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:18:14.204 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:18:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.206+0000 7fe9d0238700 1 -- 172.21.15.7:0/851486846 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe9b40a2c20 msgr2=0x7fe9b40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.206+0000 7fe9d0238700 1 --2- 172.21.15.7:0/851486846 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe9b40a2c20 0x7fe9b40a50e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8199c00 tx=0x7fe9c400b410 comp rx=0 tx=0).stop 2024-06-10T01:18:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.206+0000 7fe9d0238700 1 -- 172.21.15.7:0/851486846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9c81066e0 msgr2=0x7fe9c8198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.206+0000 7fe9d0238700 1 --2- 172.21.15.7:0/851486846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9c81066e0 0x7fe9c8198ac0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000ef30 tx=0x7fe9c000c3a0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.206+0000 7fe9d0238700 1 -- 172.21.15.7:0/851486846 shutdown_connections 2024-06-10T01:18:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.206+0000 7fe9d0238700 1 --2- 172.21.15.7:0/851486846 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe9b40a2c20 0x7fe9b40a50e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.206+0000 7fe9d0238700 1 --2- 172.21.15.7:0/851486846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9c81066e0 0x7fe9c8198ac0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.206+0000 7fe9d0238700 1 --2- 172.21.15.7:0/851486846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9c8101620 0x7fe9c819e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.206+0000 7fe9d0238700 1 --2- 172.21.15.7:0/851486846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9c8101fd0 0x7fe9c819ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:14.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.206+0000 7fe9d0238700 1 -- 172.21.15.7:0/851486846 >> 172.21.15.7:0/851486846 conn(0x7fe9c8078330 msgr2=0x7fe9c8100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:14.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.206+0000 7fe9d0238700 1 -- 172.21.15.7:0/851486846 shutdown_connections 2024-06-10T01:18:14.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:14.206+0000 7fe9d0238700 1 -- 172.21.15.7:0/851486846 wait complete. 2024-06-10T01:18:14.206 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:18:14.754 INFO:tasks.workunit.client.0.smithi007.stdout: 1 882371 7.16 MB/sec execute 535 sec latency 30003.030 ms 2024-06-10T01:18:15.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:15 smithi175 ceph-mon[36636]: pgmap v799: 129 pgs: 129 active+clean; 145 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 1.5 MiB/s wr, 75 op/s 2024-06-10T01:18:15.754 INFO:tasks.workunit.client.0.smithi007.stdout: 1 883904 7.17 MB/sec execute 536 sec latency 259.285 ms 2024-06-10T01:18:15.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:15 smithi007 ceph-mon[32398]: pgmap v799: 129 pgs: 129 active+clean; 145 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 1.5 MiB/s wr, 75 op/s 2024-06-10T01:18:15.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:15 smithi078 ceph-mon[36683]: pgmap v799: 129 pgs: 129 active+clean; 145 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 1.5 MiB/s wr, 75 op/s 2024-06-10T01:18:16.442 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b6c01176', total seconds elapsed: 37.3 2024-06-10T01:18:16.754 INFO:tasks.workunit.client.0.smithi007.stdout: 1 884932 7.16 MB/sec execute 537 sec latency 287.349 ms 2024-06-10T01:18:16.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.989+0000 7f47ab80f700 1 -- 172.21.15.7:0/1046981070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a41013c0 msgr2=0x7f47a4101820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:16.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.989+0000 7f47ab80f700 1 --2- 172.21.15.7:0/1046981070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a41013c0 0x7f47a4101820 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f4794009990 tx=0x7f4794009ca0 comp rx=0 tx=0).stop 2024-06-10T01:18:16.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.989+0000 7f47ab80f700 1 -- 172.21.15.7:0/1046981070 shutdown_connections 2024-06-10T01:18:16.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.989+0000 7f47ab80f700 1 --2- 172.21.15.7:0/1046981070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a4101d60 0x7f47a410e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:16.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.989+0000 7f47ab80f700 1 --2- 172.21.15.7:0/1046981070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47a4107400 0x7f47a41077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:16.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.989+0000 7f47ab80f700 1 --2- 172.21.15.7:0/1046981070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a41013c0 0x7f47a4101820 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:16.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.989+0000 7f47ab80f700 1 -- 172.21.15.7:0/1046981070 >> 172.21.15.7:0/1046981070 conn(0x7f47a4078330 msgr2=0x7f47a4078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:16.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.989+0000 7f47ab80f700 1 -- 172.21.15.7:0/1046981070 shutdown_connections 2024-06-10T01:18:16.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.989+0000 7f47ab80f700 1 -- 172.21.15.7:0/1046981070 wait complete. 2024-06-10T01:18:16.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.990+0000 7f47ab80f700 1 Processor -- start 2024-06-10T01:18:16.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.990+0000 7f47ab80f700 1 -- start start 2024-06-10T01:18:16.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.990+0000 7f47ab80f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a41013c0 0x7f47a4103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:16.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.990+0000 7f47ab80f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a4101d60 0x7f47a41035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:16.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.990+0000 7f47ab80f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47a4107400 0x7f47a4103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:16.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.990+0000 7f47ab80f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f47a4111660 con 0x7f47a4101d60 2024-06-10T01:18:16.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.990+0000 7f47ab80f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f47a41114e0 con 0x7f47a41013c0 2024-06-10T01:18:16.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.990+0000 7f47ab80f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f47a41117e0 con 0x7f47a4107400 2024-06-10T01:18:16.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.990+0000 7f47a8daa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a4101d60 0x7f47a41035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:16.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.990+0000 7f47a95ab700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a41013c0 0x7f47a4103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:16.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.990+0000 7f47a95ab700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a41013c0 0x7f47a4103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:41012/0 (socket says 172.21.15.7:41012) 2024-06-10T01:18:16.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.990+0000 7f47a95ab700 1 -- 172.21.15.7:0/2091316902 learned_addr learned my addr 172.21.15.7:0/2091316902 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:16.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.990+0000 7f47a9dac700 1 --2- 172.21.15.7:0/2091316902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47a4107400 0x7f47a4103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:16.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.991+0000 7f47a95ab700 1 -- 172.21.15.7:0/2091316902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47a4107400 msgr2=0x7f47a4103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:16.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.991+0000 7f47a95ab700 1 --2- 172.21.15.7:0/2091316902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47a4107400 0x7f47a4103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:16.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.991+0000 7f47a95ab700 1 -- 172.21.15.7:0/2091316902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a4101d60 msgr2=0x7f47a41035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:16.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.991+0000 7f47a95ab700 1 --2- 172.21.15.7:0/2091316902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a4101d60 0x7f47a41035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:16.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.991+0000 7f47a95ab700 1 -- 172.21.15.7:0/2091316902 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a41a71a0 con 0x7f47a41013c0 2024-06-10T01:18:16.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.991+0000 7f47a95ab700 1 --2- 172.21.15.7:0/2091316902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a41013c0 0x7f47a4103070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f47a000e9c0 tx=0x7f47a000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:16.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.991+0000 7f479a7fc700 1 -- 172.21.15.7:0/2091316902 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a000cbd0 con 0x7f47a41013c0 2024-06-10T01:18:16.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.991+0000 7f479a7fc700 1 -- 172.21.15.7:0/2091316902 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47a000ee60 con 0x7f47a41013c0 2024-06-10T01:18:16.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.991+0000 7f479a7fc700 1 -- 172.21.15.7:0/2091316902 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a00056d0 con 0x7f47a41013c0 2024-06-10T01:18:16.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.991+0000 7f47ab80f700 1 -- 172.21.15.7:0/2091316902 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a41a7490 con 0x7f47a41013c0 2024-06-10T01:18:16.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.991+0000 7f47ab80f700 1 -- 172.21.15.7:0/2091316902 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a41a79a0 con 0x7f47a41013c0 2024-06-10T01:18:16.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.994+0000 7f479a7fc700 1 -- 172.21.15.7:0/2091316902 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f47a0004750 con 0x7f47a41013c0 2024-06-10T01:18:16.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.995+0000 7f479a7fc700 1 --2- 172.21.15.7:0/2091316902 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f47900a2f10 0x7f47900a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:16.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.995+0000 7f47a8daa700 1 --2- 172.21.15.7:0/2091316902 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f47900a2f10 0x7f47900a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:16.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.995+0000 7f479a7fc700 1 -- 172.21.15.7:0/2091316902 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f47a00cf490 con 0x7f47a41013c0 2024-06-10T01:18:16.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.996+0000 7f47a8daa700 1 --2- 172.21.15.7:0/2091316902 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f47900a2f10 0x7f47900a53d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f479400b2e0 tx=0x7f4794005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:16.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.996+0000 7f47b6092b80 1 Processor -- start 2024-06-10T01:18:16.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.996+0000 7f47b6092b80 1 -- start start 2024-06-10T01:18:16.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.996+0000 7f47b6092b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be4569f60 0x564be456a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:16.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.996+0000 7f47b6092b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be456a880 0x564be4564260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:16.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.996+0000 7f47b6092b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564be45647a0 0x564be4564bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:16.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.996+0000 7f47b6092b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564be4456f60 con 0x564be456a880 2024-06-10T01:18:16.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.996+0000 7f47b6092b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564be44b5850 con 0x564be45647a0 2024-06-10T01:18:16.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.996+0000 7f47b6092b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564be441d460 con 0x564be4569f60 2024-06-10T01:18:16.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.997+0000 7f47a9dac700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564be45647a0 0x564be4564bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:16.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.997+0000 7f47a8daa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be456a880 0x564be4564260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:16.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.997+0000 7f47a9dac700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564be45647a0 0x564be4564bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:41014/0 (socket says 172.21.15.7:41014) 2024-06-10T01:18:16.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.997+0000 7f47a9dac700 1 -- 172.21.15.7:0/3620948994 learned_addr learned my addr 172.21.15.7:0/3620948994 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:16.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.997+0000 7f47a95ab700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be4569f60 0x564be456a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:16.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.997+0000 7f47a9dac700 1 -- 172.21.15.7:0/3620948994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be4569f60 msgr2=0x564be456a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:16.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.997+0000 7f47a9dac700 1 --2- 172.21.15.7:0/3620948994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be4569f60 0x564be456a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:16.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.997+0000 7f47a9dac700 1 -- 172.21.15.7:0/3620948994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be456a880 msgr2=0x564be4564260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:16.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.997+0000 7f47a9dac700 1 --2- 172.21.15.7:0/3620948994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be456a880 0x564be4564260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:16.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.997+0000 7f47a9dac700 1 -- 172.21.15.7:0/3620948994 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564be44048d0 con 0x564be45647a0 2024-06-10T01:18:16.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.997+0000 7f47a95ab700 1 --2- 172.21.15.7:0/3620948994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be4569f60 0x564be456a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:18:16.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.997+0000 7f47a9dac700 1 --2- 172.21.15.7:0/3620948994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564be45647a0 0x564be4564bc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f479c011680 tx=0x7f479c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7ec5b0d7cedfa6f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:16.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.997+0000 7f478f7fe700 1 -- 172.21.15.7:0/3620948994 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c019020 con 0x564be45647a0 2024-06-10T01:18:16.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.997+0000 7f478f7fe700 1 -- 172.21.15.7:0/3620948994 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f479c0079e0 con 0x564be45647a0 2024-06-10T01:18:16.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.997+0000 7f478f7fe700 1 -- 172.21.15.7:0/3620948994 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c015b00 con 0x564be45647a0 2024-06-10T01:18:16.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.998+0000 7f47b6092b80 1 -- 172.21.15.7:0/3620948994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564be45647a0 msgr2=0x564be4564bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:16.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.998+0000 7f47b6092b80 1 --2- 172.21.15.7:0/3620948994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564be45647a0 0x564be4564bc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f479c011680 tx=0x7f479c009920 comp rx=0 tx=0).stop 2024-06-10T01:18:16.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.998+0000 7f47b6092b80 1 -- 172.21.15.7:0/3620948994 shutdown_connections 2024-06-10T01:18:16.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.998+0000 7f47b6092b80 1 --2- 172.21.15.7:0/3620948994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564be45647a0 0x564be4564bc0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:16.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.998+0000 7f47b6092b80 1 --2- 172.21.15.7:0/3620948994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be4569f60 0x564be456a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:16.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.998+0000 7f47b6092b80 1 --2- 172.21.15.7:0/3620948994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be456a880 0x564be4564260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:16.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.998+0000 7f47b6092b80 1 -- 172.21.15.7:0/3620948994 >> 172.21.15.7:0/3620948994 conn(0x564be4491080 msgr2=0x564be455f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:16.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.998+0000 7f47b6092b80 1 -- 172.21.15.7:0/3620948994 shutdown_connections 2024-06-10T01:18:16.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.998+0000 7f47b6092b80 1 -- 172.21.15.7:0/3620948994 wait complete. 2024-06-10T01:18:16.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.999+0000 7f47b6092b80 1 -- start start 2024-06-10T01:18:16.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.999+0000 7f47b6092b80 1 Processor -- start 2024-06-10T01:18:16.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.999+0000 7f47b6092b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be45647a0 0x564be4570600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:16.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.999+0000 7f47b6092b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be4569f60 0x564be456d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:16.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.999+0000 7f47b6092b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564be456a880 0x564be456de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:16.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.999+0000 7f47b6092b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564be4455fc0 con 0x564be4569f60 2024-06-10T01:18:16.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.999+0000 7f47b6092b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564be44b1810 con 0x564be456a880 2024-06-10T01:18:16.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.999+0000 7f47b6092b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564be4571260 con 0x564be45647a0 2024-06-10T01:18:16.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.999+0000 7f47b6092b80 10 client.0 _ll_register_callbacks cb 0x564be444b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:18:16.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.999+0000 7f47a8daa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be4569f60 0x564be456d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:16.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.999+0000 7f47a8daa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be4569f60 0x564be456d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45074/0 (socket says 172.21.15.7:45074) 2024-06-10T01:18:16.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.999+0000 7f47a8daa700 1 -- 172.21.15.7:0/4287097778 learned_addr learned my addr 172.21.15.7:0/4287097778 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:16.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.999+0000 7f47a9dac700 1 --2- 172.21.15.7:0/4287097778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564be456a880 0x564be456de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:16.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:16.999+0000 7f47a95ab700 1 --2- 172.21.15.7:0/4287097778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be45647a0 0x564be4570600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:16.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.000+0000 7f47a8daa700 1 -- 172.21.15.7:0/4287097778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be45647a0 msgr2=0x564be4570600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:17.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.000+0000 7f47a8daa700 1 --2- 172.21.15.7:0/4287097778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be45647a0 0x564be4570600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:17.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.000+0000 7f47a8daa700 1 -- 172.21.15.7:0/4287097778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564be456a880 msgr2=0x564be456de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:17.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.000+0000 7f47a8daa700 1 --2- 172.21.15.7:0/4287097778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564be456a880 0x564be456de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:17.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.000+0000 7f47a8daa700 1 -- 172.21.15.7:0/4287097778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564be447b750 con 0x564be4569f60 2024-06-10T01:18:17.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.000+0000 7f47a9dac700 1 --2- 172.21.15.7:0/4287097778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564be456a880 0x564be456de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:18:17.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.000+0000 7f47a95ab700 1 --2- 172.21.15.7:0/4287097778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be45647a0 0x564be4570600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:18:17.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.000+0000 7f47a8daa700 1 --2- 172.21.15.7:0/4287097778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be4569f60 0x564be456d900 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f4794009990 tx=0x7f4794009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248c348a4f6a76ad server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:17.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.000+0000 7f478cff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:18:17.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.000+0000 7f478cff9700 1 -- 172.21.15.7:0/4287097778 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479400f460 con 0x564be4569f60 2024-06-10T01:18:17.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.000+0000 7f478cff9700 1 -- 172.21.15.7:0/4287097778 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f479400f600 con 0x564be4569f60 2024-06-10T01:18:17.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.000+0000 7f47b6092b80 1 -- 172.21.15.7:0/4287097778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f479c0079e0 con 0x564be4569f60 2024-06-10T01:18:17.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.000+0000 7f478cff9700 1 -- 172.21.15.7:0/4287097778 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f479403c050 con 0x564be4569f60 2024-06-10T01:18:17.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.000+0000 7f47b6092b80 10 client.18561 fetch_fsmap learned FSMap version 40 2024-06-10T01:18:17.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.000+0000 7f47b6092b80 1 -- 172.21.15.7:0/4287097778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x564be457b9a0 con 0x564be4569f60 2024-06-10T01:18:17.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.001+0000 7f478cff9700 1 -- 172.21.15.7:0/4287097778 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f479c0079e0 con 0x564be4569f60 2024-06-10T01:18:17.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.001+0000 7f478cff9700 1 -- 172.21.15.7:0/4287097778 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f4794022710 con 0x564be4569f60 2024-06-10T01:18:17.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.001+0000 7f47b6092b80 10 client.18561 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:18:17.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.001+0000 7f47b6092b80 10 client.18561 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:18:17.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.001+0000 7f47b6092b80 20 client.18561 populate_metadata read hostname 'smithi007' 2024-06-10T01:18:17.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.001+0000 7f47b6092b80 1 --2- 172.21.15.7:0/4287097778 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x564be457e030 0x564be4580450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:17.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.001+0000 7f47b6092b80 4 client.18561 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:18:17.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.001+0000 7f47b6092b80 1 -- 172.21.15.7:0/4287097778 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564be44c2d50 con 0x564be457e030 2024-06-10T01:18:17.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.001+0000 7f47a95ab700 1 --2- 172.21.15.7:0/4287097778 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x564be457e030 0x564be4580450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:17.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.001+0000 7f478cff9700 10 client.18561 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:18:17.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.001+0000 7f47a95ab700 1 --2- 172.21.15.7:0/4287097778 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x564be457e030 0x564be4580450 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1c8e581298378d70 server_cookie=4174187754154122 in_seq=0 out_seq=0 2024-06-10T01:18:17.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.002+0000 7f478cff9700 1 -- 172.21.15.7:0/4287097778 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564be44c2d50 con 0x564be457e030 2024-06-10T01:18:17.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.002+0000 7f478cff9700 10 client.18561 handle_command_reply: tid=0 2024-06-10T01:18:17.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.002+0000 7f47b6092b80 1 client.18561 shutdown 2024-06-10T01:18:17.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.002+0000 7f47b6092b80 1 -- 172.21.15.7:0/4287097778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be4569f60 msgr2=0x564be456d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:17.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.002+0000 7f47b6092b80 1 --2- 172.21.15.7:0/4287097778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be4569f60 0x564be456d900 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f4794009990 tx=0x7f4794009ca0 comp rx=0 tx=0).stop 2024-06-10T01:18:17.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.002+0000 7f47b6092b80 1 -- 172.21.15.7:0/4287097778 shutdown_connections 2024-06-10T01:18:17.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.002+0000 7f47b6092b80 1 --2- 172.21.15.7:0/4287097778 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x564be457e030 0x564be4580450 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:17.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.002+0000 7f47b6092b80 1 --2- 172.21.15.7:0/4287097778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564be456a880 0x564be456de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:17.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.002+0000 7f47b6092b80 1 --2- 172.21.15.7:0/4287097778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be45647a0 0x564be4570600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:17.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.002+0000 7f47b6092b80 1 --2- 172.21.15.7:0/4287097778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be4569f60 0x564be456d900 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:17.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.002+0000 7f47b6092b80 1 -- 172.21.15.7:0/4287097778 >> 172.21.15.7:0/4287097778 conn(0x564be4491080 msgr2=0x564be455e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:17.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.002+0000 7f478cff9700 0 client.18561 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:18:17.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.003+0000 7f47b6092b80 1 -- 172.21.15.7:0/4287097778 shutdown_connections 2024-06-10T01:18:17.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.003+0000 7f47b6092b80 1 -- 172.21.15.7:0/4287097778 wait complete. 2024-06-10T01:18:17.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.003+0000 7f47b6092b80 20 client.18561 trim_cache size 0 max 16384 2024-06-10T01:18:17.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.021+0000 7f47b6092b80 1 Processor -- start 2024-06-10T01:18:17.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.021+0000 7f47b6092b80 1 -- start start 2024-06-10T01:18:17.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.021+0000 7f47b6092b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be45647a0 0x564be4564260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:17.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.021+0000 7f47b6092b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be4569f60 0x564be456b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:17.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.021+0000 7f47b6092b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564be456a880 0x564be456c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:17.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.021+0000 7f47b6092b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f47a41b2d50 con 0x564be45647a0 2024-06-10T01:18:17.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.021+0000 7f47b6092b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f47a41b05e0 con 0x564be456a880 2024-06-10T01:18:17.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.021+0000 7f47b6092b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f47a41ae430 con 0x564be4569f60 2024-06-10T01:18:17.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.021+0000 7f47a95ab700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be45647a0 0x564be4564260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:17.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.021+0000 7f47a95ab700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be45647a0 0x564be4564260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45088/0 (socket says 172.21.15.7:45088) 2024-06-10T01:18:17.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.021+0000 7f47a8daa700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be4569f60 0x564be456b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:17.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.021+0000 7f47a95ab700 1 -- 172.21.15.7:0/1728466098 learned_addr learned my addr 172.21.15.7:0/1728466098 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:17.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.021+0000 7f47a9dac700 1 --2- 172.21.15.7:0/1728466098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564be456a880 0x564be456c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:17.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.021+0000 7f47a95ab700 1 -- 172.21.15.7:0/1728466098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be4569f60 msgr2=0x564be456b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:17.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.021+0000 7f47a95ab700 1 --2- 172.21.15.7:0/1728466098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be4569f60 0x564be456b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:17.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.021+0000 7f47a95ab700 1 -- 172.21.15.7:0/1728466098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564be456a880 msgr2=0x564be456c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:17.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.021+0000 7f47a95ab700 1 --2- 172.21.15.7:0/1728466098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564be456a880 0x564be456c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:17.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.021+0000 7f47a95ab700 1 -- 172.21.15.7:0/1728466098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564be456be60 con 0x564be45647a0 2024-06-10T01:18:17.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.021+0000 7f47a8daa700 1 --2- 172.21.15.7:0/1728466098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be4569f60 0x564be456b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:18:17.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.021+0000 7f47a9dac700 1 --2- 172.21.15.7:0/1728466098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564be456a880 0x564be456c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:18:17.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.022+0000 7f47a95ab700 1 --2- 172.21.15.7:0/1728466098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be45647a0 0x564be4564260 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f47a009b650 tx=0x7f47a00ccac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67dfb9130261eb49 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:17.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.022+0000 7f478effd700 1 -- 172.21.15.7:0/1728466098 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a00b9070 con 0x564be45647a0 2024-06-10T01:18:17.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.022+0000 7f478effd700 1 -- 172.21.15.7:0/1728466098 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47a000cd70 con 0x564be45647a0 2024-06-10T01:18:17.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.022+0000 7f478effd700 1 -- 172.21.15.7:0/1728466098 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a009bbb0 con 0x564be45647a0 2024-06-10T01:18:17.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.023+0000 7f47b6092b80 1 -- 172.21.15.7:0/1728466098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be45647a0 msgr2=0x564be4564260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:17.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.023+0000 7f47b6092b80 1 --2- 172.21.15.7:0/1728466098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be45647a0 0x564be4564260 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f47a009b650 tx=0x7f47a00ccac0 comp rx=0 tx=0).stop 2024-06-10T01:18:17.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.023+0000 7f47b6092b80 1 -- 172.21.15.7:0/1728466098 shutdown_connections 2024-06-10T01:18:17.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.023+0000 7f47b6092b80 1 --2- 172.21.15.7:0/1728466098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564be456a880 0x564be456c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:17.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.023+0000 7f47b6092b80 1 --2- 172.21.15.7:0/1728466098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be45647a0 0x564be4564260 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:17.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.023+0000 7f47b6092b80 1 --2- 172.21.15.7:0/1728466098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be4569f60 0x564be456b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.023+0000 7f47b6092b80 1 -- 172.21.15.7:0/1728466098 >> 172.21.15.7:0/1728466098 conn(0x564be4491080 msgr2=0x564be456c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.023+0000 7f47b6092b80 1 -- 172.21.15.7:0/1728466098 shutdown_connections 2024-06-10T01:18:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.023+0000 7f47b6092b80 1 -- 172.21.15.7:0/1728466098 wait complete. 2024-06-10T01:18:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.023+0000 7f47b6092b80 1 -- start start 2024-06-10T01:18:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.023+0000 7f47b6092b80 1 Processor -- start 2024-06-10T01:18:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.024+0000 7f47b6092b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be45647a0 0x564be4576a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.024+0000 7f47b6092b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be4569f60 0x564be4585350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.024+0000 7f47b6092b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564be456a880 0x564be4585890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.024+0000 7f47b6092b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f47a41aeaa0 con 0x564be4569f60 2024-06-10T01:18:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.024+0000 7f47b6092b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f47a41aff10 con 0x564be456a880 2024-06-10T01:18:17.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.024+0000 7f47b6092b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f47a41af170 con 0x564be45647a0 2024-06-10T01:18:17.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.024+0000 7f47b6092b80 10 client.0 _ll_register_callbacks cb 0x564be444b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:18:17.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.024+0000 7f47a8daa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be4569f60 0x564be4585350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:17.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.024+0000 7f47a8daa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be4569f60 0x564be4585350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45104/0 (socket says 172.21.15.7:45104) 2024-06-10T01:18:17.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.024+0000 7f47a8daa700 1 -- 172.21.15.7:0/2800334501 learned_addr learned my addr 172.21.15.7:0/2800334501 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:17.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.024+0000 7f47a95ab700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be45647a0 0x564be4576a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:17.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.024+0000 7f47a9dac700 1 --2- 172.21.15.7:0/2800334501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564be456a880 0x564be4585890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:17.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.024+0000 7f47a8daa700 1 -- 172.21.15.7:0/2800334501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be45647a0 msgr2=0x564be4576a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:17.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.024+0000 7f47a8daa700 1 --2- 172.21.15.7:0/2800334501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be45647a0 0x564be4576a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:17.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.024+0000 7f47a8daa700 1 -- 172.21.15.7:0/2800334501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564be456a880 msgr2=0x564be4585890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:17.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.024+0000 7f47a8daa700 1 --2- 172.21.15.7:0/2800334501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564be456a880 0x564be4585890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:17.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.024+0000 7f47a8daa700 1 -- 172.21.15.7:0/2800334501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564be4585dd0 con 0x564be4569f60 2024-06-10T01:18:17.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.024+0000 7f47a8daa700 1 --2- 172.21.15.7:0/2800334501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be4569f60 0x564be4585350 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f4794000c00 tx=0x7f4794003f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f67da4d38bc2b5d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:17.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.024+0000 7f478cff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:18:17.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.024+0000 7f478cff9700 1 -- 172.21.15.7:0/2800334501 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4794022800 con 0x564be4569f60 2024-06-10T01:18:17.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.024+0000 7f478cff9700 1 -- 172.21.15.7:0/2800334501 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f479400f460 con 0x564be4569f60 2024-06-10T01:18:17.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.025+0000 7f47b6092b80 1 -- 172.21.15.7:0/2800334501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47a000cd70 con 0x564be4569f60 2024-06-10T01:18:17.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.025+0000 7f478cff9700 1 -- 172.21.15.7:0/2800334501 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f479404e050 con 0x564be4569f60 2024-06-10T01:18:17.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.025+0000 7f47b6092b80 10 client.18567 fetch_fsmap learned FSMap version 40 2024-06-10T01:18:17.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.025+0000 7f47b6092b80 1 -- 172.21.15.7:0/2800334501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x564be45e89e0 con 0x564be4569f60 2024-06-10T01:18:17.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.025+0000 7f478cff9700 1 -- 172.21.15.7:0/2800334501 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47a000cd70 con 0x564be4569f60 2024-06-10T01:18:17.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.025+0000 7f478cff9700 1 -- 172.21.15.7:0/2800334501 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f47940405c0 con 0x564be4569f60 2024-06-10T01:18:17.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.025+0000 7f47b6092b80 10 client.18567 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:18:17.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.025+0000 7f47b6092b80 10 client.18567 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:18:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.025+0000 7f47b6092b80 20 client.18567 populate_metadata read hostname 'smithi007' 2024-06-10T01:18:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.025+0000 7f47b6092b80 1 --2- 172.21.15.7:0/2800334501 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x564be45e9050 0x564be45eb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.025+0000 7f47b6092b80 4 client.18567 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:18:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.025+0000 7f47b6092b80 1 -- 172.21.15.7:0/2800334501 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564be44bd2d0 con 0x564be45e9050 2024-06-10T01:18:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.026+0000 7f47a95ab700 1 --2- 172.21.15.7:0/2800334501 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x564be45e9050 0x564be45eb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.026+0000 7f47a95ab700 1 --2- 172.21.15.7:0/2800334501 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x564be45e9050 0x564be45eb470 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=31a4684f7ae78bb3 server_cookie=307b05b8f77e0781 in_seq=0 out_seq=0 2024-06-10T01:18:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.026+0000 7f478cff9700 10 client.18567 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:18:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.026+0000 7f478cff9700 1 -- 172.21.15.7:0/2800334501 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564be44bd2d0 con 0x564be45e9050 2024-06-10T01:18:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.026+0000 7f478cff9700 10 client.18567 handle_command_reply: tid=0 2024-06-10T01:18:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.026+0000 7f47b6092b80 1 client.18567 shutdown 2024-06-10T01:18:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.027+0000 7f47b6092b80 1 -- 172.21.15.7:0/2800334501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be4569f60 msgr2=0x564be4585350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.027+0000 7f47b6092b80 1 --2- 172.21.15.7:0/2800334501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be4569f60 0x564be4585350 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f4794000c00 tx=0x7f4794003f20 comp rx=0 tx=0).stop 2024-06-10T01:18:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.027+0000 7f47b6092b80 1 -- 172.21.15.7:0/2800334501 shutdown_connections 2024-06-10T01:18:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.027+0000 7f47b6092b80 1 --2- 172.21.15.7:0/2800334501 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x564be45e9050 0x564be45eb470 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.027+0000 7f47b6092b80 1 --2- 172.21.15.7:0/2800334501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564be456a880 0x564be4585890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.027+0000 7f47b6092b80 1 --2- 172.21.15.7:0/2800334501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be45647a0 0x564be4576a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.027+0000 7f47b6092b80 1 --2- 172.21.15.7:0/2800334501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be4569f60 0x564be4585350 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.027+0000 7f47b6092b80 1 -- 172.21.15.7:0/2800334501 >> 172.21.15.7:0/2800334501 conn(0x564be4491080 msgr2=0x564be456d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.027+0000 7f478cff9700 0 client.18567 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:18:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.027+0000 7f47b6092b80 1 -- 172.21.15.7:0/2800334501 shutdown_connections 2024-06-10T01:18:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.027+0000 7f47b6092b80 1 -- 172.21.15.7:0/2800334501 wait complete. 2024-06-10T01:18:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.027+0000 7f47b6092b80 20 client.18567 trim_cache size 0 max 16384 2024-06-10T01:18:17.027 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:18:17.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.027+0000 7f47ab80f700 1 -- 172.21.15.7:0/2091316902 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f47900a2f10 msgr2=0x7f47900a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:17.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.027+0000 7f47ab80f700 1 --2- 172.21.15.7:0/2091316902 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f47900a2f10 0x7f47900a53d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f479400b2e0 tx=0x7f4794005960 comp rx=0 tx=0).stop 2024-06-10T01:18:17.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.027+0000 7f47ab80f700 1 -- 172.21.15.7:0/2091316902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a41013c0 msgr2=0x7f47a4103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:17.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.027+0000 7f47ab80f700 1 --2- 172.21.15.7:0/2091316902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a41013c0 0x7f47a4103070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f47a000e9c0 tx=0x7f47a000ecd0 comp rx=0 tx=0).stop 2024-06-10T01:18:17.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.028+0000 7f47ab80f700 1 -- 172.21.15.7:0/2091316902 shutdown_connections 2024-06-10T01:18:17.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.028+0000 7f47ab80f700 1 --2- 172.21.15.7:0/2091316902 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f47900a2f10 0x7f47900a53d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:17.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.028+0000 7f47ab80f700 1 --2- 172.21.15.7:0/2091316902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47a4107400 0x7f47a4103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:17.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.028+0000 7f47ab80f700 1 --2- 172.21.15.7:0/2091316902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a41013c0 0x7f47a4103070 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:17.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.028+0000 7f47ab80f700 1 --2- 172.21.15.7:0/2091316902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a4101d60 0x7f47a41035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:17.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.028+0000 7f47ab80f700 1 -- 172.21.15.7:0/2091316902 >> 172.21.15.7:0/2091316902 conn(0x7f47a4078330 msgr2=0x7f47a4100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:17.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.028+0000 7f47ab80f700 1 -- 172.21.15.7:0/2091316902 shutdown_connections 2024-06-10T01:18:17.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:17.028+0000 7f47ab80f700 1 -- 172.21.15.7:0/2091316902 wait complete. 2024-06-10T01:18:17.623 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:17 smithi007 ceph-mon[32398]: pgmap v800: 129 pgs: 129 active+clean; 145 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 83 op/s 2024-06-10T01:18:17.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:17 smithi175 ceph-mon[36636]: pgmap v800: 129 pgs: 129 active+clean; 145 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 83 op/s 2024-06-10T01:18:17.719 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:18:17.755 INFO:tasks.workunit.client.0.smithi007.stdout: 1 886674 7.16 MB/sec execute 538 sec latency 45.070 ms 2024-06-10T01:18:17.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:17 smithi078 ceph-mon[36683]: pgmap v800: 129 pgs: 129 active+clean; 145 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 83 op/s 2024-06-10T01:18:17.986 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:18:18.755 INFO:tasks.workunit.client.0.smithi007.stdout: 1 888029 7.17 MB/sec execute 539 sec latency 36.246 ms 2024-06-10T01:18:19.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:19 smithi007 ceph-mon[32398]: pgmap v801: 129 pgs: 129 active+clean; 144 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 87 op/s 2024-06-10T01:18:19.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:19 smithi175 ceph-mon[36636]: pgmap v801: 129 pgs: 129 active+clean; 144 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 87 op/s 2024-06-10T01:18:19.754 INFO:tasks.workunit.client.0.smithi007.stdout: 1 889019 7.17 MB/sec execute 540 sec latency 208.515 ms 2024-06-10T01:18:19.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.742+0000 7fd4035c4700 1 -- 172.21.15.7:0/3840724735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3fc107550 msgr2=0x7fd3fc107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:19.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.742+0000 7fd4035c4700 1 --2- 172.21.15.7:0/3840724735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3fc107550 0x7fd3fc107930 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8009990 tx=0x7fd3f8009ca0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.742+0000 7fd4035c4700 1 -- 172.21.15.7:0/3840724735 shutdown_connections 2024-06-10T01:18:19.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.742+0000 7fd4035c4700 1 --2- 172.21.15.7:0/3840724735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3fc1017f0 0x7fd3fc10f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.742+0000 7fd4035c4700 1 --2- 172.21.15.7:0/3840724735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3fc107550 0x7fd3fc107930 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.742+0000 7fd4035c4700 1 --2- 172.21.15.7:0/3840724735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3fc100e30 0x7fd3fc1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.742+0000 7fd4035c4700 1 -- 172.21.15.7:0/3840724735 >> 172.21.15.7:0/3840724735 conn(0x7fd3fc0766b0 msgr2=0x7fd3fc076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:19.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.742+0000 7fd4035c4700 1 -- 172.21.15.7:0/3840724735 shutdown_connections 2024-06-10T01:18:19.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.742+0000 7fd4035c4700 1 -- 172.21.15.7:0/3840724735 wait complete. 2024-06-10T01:18:19.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.743+0000 7fd4035c4700 1 Processor -- start 2024-06-10T01:18:19.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.743+0000 7fd4035c4700 1 -- start start 2024-06-10T01:18:19.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.743+0000 7fd4035c4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3fc100e30 0x7fd3fc19b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:19.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.743+0000 7fd4035c4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3fc1017f0 0x7fd3fc19b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:19.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.743+0000 7fd4035c4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3fc19f9c0 0x7fd3fc19fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:19.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.743+0000 7fd4035c4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc112790 con 0x7fd3fc100e30 2024-06-10T01:18:19.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.743+0000 7fd4035c4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc112610 con 0x7fd3fc19f9c0 2024-06-10T01:18:19.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.743+0000 7fd4035c4700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc112910 con 0x7fd3fc1017f0 2024-06-10T01:18:19.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.743+0000 7fd401360700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3fc100e30 0x7fd3fc19b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:19.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.743+0000 7fd400b5f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3fc1017f0 0x7fd3fc19b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:19.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.744+0000 7fd400b5f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3fc1017f0 0x7fd3fc19b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:53784/0 (socket says 172.21.15.7:53784) 2024-06-10T01:18:19.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.744+0000 7fd401360700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3fc100e30 0x7fd3fc19b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45124/0 (socket says 172.21.15.7:45124) 2024-06-10T01:18:19.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.744+0000 7fd400b5f700 1 -- 172.21.15.7:0/3308091335 learned_addr learned my addr 172.21.15.7:0/3308091335 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:19.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.744+0000 7fd401b61700 1 --2- 172.21.15.7:0/3308091335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3fc19f9c0 0x7fd3fc19fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:19.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.744+0000 7fd400b5f700 1 -- 172.21.15.7:0/3308091335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3fc19f9c0 msgr2=0x7fd3fc19fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:19.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.744+0000 7fd400b5f700 1 --2- 172.21.15.7:0/3308091335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3fc19f9c0 0x7fd3fc19fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.744+0000 7fd400b5f700 1 -- 172.21.15.7:0/3308091335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3fc100e30 msgr2=0x7fd3fc19b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:19.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.744+0000 7fd400b5f700 1 --2- 172.21.15.7:0/3308091335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3fc100e30 0x7fd3fc19b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.744+0000 7fd400b5f700 1 -- 172.21.15.7:0/3308091335 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3fc1a04e0 con 0x7fd3fc1017f0 2024-06-10T01:18:19.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.744+0000 7fd401b61700 1 --2- 172.21.15.7:0/3308091335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3fc19f9c0 0x7fd3fc19fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:18:19.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.744+0000 7fd401360700 1 --2- 172.21.15.7:0/3308091335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3fc100e30 0x7fd3fc19b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:18:19.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.744+0000 7fd400b5f700 1 --2- 172.21.15.7:0/3308091335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3fc1017f0 0x7fd3fc19b630 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec00e9a0 tx=0x7fd3ec00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:19.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.744+0000 7fd3f27fc700 1 -- 172.21.15.7:0/3308091335 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec00cbd0 con 0x7fd3fc1017f0 2024-06-10T01:18:19.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.744+0000 7fd4035c4700 1 -- 172.21.15.7:0/3308091335 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3fc1a07d0 con 0x7fd3fc1017f0 2024-06-10T01:18:19.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.744+0000 7fd3f27fc700 1 -- 172.21.15.7:0/3308091335 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3ec00cd70 con 0x7fd3fc1017f0 2024-06-10T01:18:19.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.744+0000 7fd3f27fc700 1 -- 172.21.15.7:0/3308091335 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec018750 con 0x7fd3fc1017f0 2024-06-10T01:18:19.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.744+0000 7fd4035c4700 1 -- 172.21.15.7:0/3308091335 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3fc1a8000 con 0x7fd3fc1017f0 2024-06-10T01:18:19.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.747+0000 7fd3f27fc700 1 -- 172.21.15.7:0/3308091335 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fd3ec010430 con 0x7fd3fc1017f0 2024-06-10T01:18:19.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.747+0000 7fd3f27fc700 1 --2- 172.21.15.7:0/3308091335 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd3e80a2c20 0x7fd3e80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:19.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.748+0000 7fd401360700 1 --2- 172.21.15.7:0/3308091335 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd3e80a2c20 0x7fd3e80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:19.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.748+0000 7fd3f27fc700 1 -- 172.21.15.7:0/3308091335 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fd3ec0cf710 con 0x7fd3fc1017f0 2024-06-10T01:18:19.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.748+0000 7fd40de47b80 1 Processor -- start 2024-06-10T01:18:19.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.748+0000 7fd40de47b80 1 -- start start 2024-06-10T01:18:19.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.748+0000 7fd401360700 1 --2- 172.21.15.7:0/3308091335 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd3e80a2c20 0x7fd3e80a50e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8000c00 tx=0x7fd3f8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:19.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.748+0000 7fd40de47b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bf4285f60 0x556bf4286340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:19.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.748+0000 7fd40de47b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bf4286880 0x556bf4280260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:19.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.748+0000 7fd40de47b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bf42807a0 0x556bf4280bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:19.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.748+0000 7fd40de47b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556bf4172f60 con 0x556bf42807a0 2024-06-10T01:18:19.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.748+0000 7fd40de47b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556bf41d1850 con 0x556bf4286880 2024-06-10T01:18:19.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.748+0000 7fd40de47b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556bf4139460 con 0x556bf4285f60 2024-06-10T01:18:19.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.748+0000 7fd401b61700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bf42807a0 0x556bf4280bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:19.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.748+0000 7fd401b61700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bf42807a0 0x556bf4280bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45130/0 (socket says 172.21.15.7:45130) 2024-06-10T01:18:19.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.748+0000 7fd401b61700 1 -- 172.21.15.7:0/2466877183 learned_addr learned my addr 172.21.15.7:0/2466877183 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:19.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.748+0000 7fd401360700 1 --2- 172.21.15.7:0/2466877183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bf4285f60 0x556bf4286340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:19.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.748+0000 7fd400b5f700 1 --2- 172.21.15.7:0/2466877183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bf4286880 0x556bf4280260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:19.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.749+0000 7fd401b61700 1 -- 172.21.15.7:0/2466877183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bf4285f60 msgr2=0x556bf4286340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:19.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.749+0000 7fd401b61700 1 --2- 172.21.15.7:0/2466877183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bf4285f60 0x556bf4286340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.749+0000 7fd401b61700 1 -- 172.21.15.7:0/2466877183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bf4286880 msgr2=0x556bf4280260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:19.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.749+0000 7fd401b61700 1 --2- 172.21.15.7:0/2466877183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bf4286880 0x556bf4280260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.749+0000 7fd401b61700 1 -- 172.21.15.7:0/2466877183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bf41208d0 con 0x556bf42807a0 2024-06-10T01:18:19.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.749+0000 7fd401b61700 1 --2- 172.21.15.7:0/2466877183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bf42807a0 0x556bf4280bc0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4009440 tx=0x7fd3f4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2586dbecdbc40ef server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:19.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.749+0000 7fd3e77fe700 1 -- 172.21.15.7:0/2466877183 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f4013750 con 0x556bf42807a0 2024-06-10T01:18:19.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.749+0000 7fd3e77fe700 1 -- 172.21.15.7:0/2466877183 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3f4013dd0 con 0x556bf42807a0 2024-06-10T01:18:19.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.749+0000 7fd3e77fe700 1 -- 172.21.15.7:0/2466877183 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f40143e0 con 0x556bf42807a0 2024-06-10T01:18:19.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.750+0000 7fd40de47b80 1 -- 172.21.15.7:0/2466877183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bf42807a0 msgr2=0x556bf4280bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:19.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.750+0000 7fd40de47b80 1 --2- 172.21.15.7:0/2466877183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bf42807a0 0x556bf4280bc0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4009440 tx=0x7fd3f4009920 comp rx=0 tx=0).stop 2024-06-10T01:18:19.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.750+0000 7fd40de47b80 1 -- 172.21.15.7:0/2466877183 shutdown_connections 2024-06-10T01:18:19.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.750+0000 7fd40de47b80 1 --2- 172.21.15.7:0/2466877183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bf42807a0 0x556bf4280bc0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.750+0000 7fd40de47b80 1 --2- 172.21.15.7:0/2466877183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bf4285f60 0x556bf4286340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.750+0000 7fd40de47b80 1 --2- 172.21.15.7:0/2466877183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bf4286880 0x556bf4280260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.750+0000 7fd40de47b80 1 -- 172.21.15.7:0/2466877183 >> 172.21.15.7:0/2466877183 conn(0x556bf41ad080 msgr2=0x556bf427b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:19.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.750+0000 7fd40de47b80 1 -- 172.21.15.7:0/2466877183 shutdown_connections 2024-06-10T01:18:19.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.750+0000 7fd40de47b80 1 -- 172.21.15.7:0/2466877183 wait complete. 2024-06-10T01:18:19.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.750+0000 7fd40de47b80 1 -- start start 2024-06-10T01:18:19.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.750+0000 7fd40de47b80 1 Processor -- start 2024-06-10T01:18:19.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.751+0000 7fd40de47b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bf42807a0 0x556bf428c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:19.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.751+0000 7fd40de47b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bf4285f60 0x556bf4289890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:19.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.751+0000 7fd40de47b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bf4286880 0x556bf4289dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:19.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.751+0000 7fd40de47b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556bf4171fc0 con 0x556bf42807a0 2024-06-10T01:18:19.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.751+0000 7fd40de47b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556bf41cd810 con 0x556bf4286880 2024-06-10T01:18:19.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.751+0000 7fd40de47b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556bf428d260 con 0x556bf4285f60 2024-06-10T01:18:19.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.751+0000 7fd40de47b80 10 client.0 _ll_register_callbacks cb 0x556bf41675b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:18:19.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.751+0000 7fd401360700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bf42807a0 0x556bf428c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:19.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.751+0000 7fd401360700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bf42807a0 0x556bf428c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45132/0 (socket says 172.21.15.7:45132) 2024-06-10T01:18:19.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.751+0000 7fd400b5f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bf4285f60 0x556bf4289890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:19.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.751+0000 7fd401360700 1 -- 172.21.15.7:0/480440429 learned_addr learned my addr 172.21.15.7:0/480440429 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:19.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.751+0000 7fd401b61700 1 --2- 172.21.15.7:0/480440429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bf4286880 0x556bf4289dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:19.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.751+0000 7fd401360700 1 -- 172.21.15.7:0/480440429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bf4285f60 msgr2=0x556bf4289890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:19.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.751+0000 7fd401360700 1 --2- 172.21.15.7:0/480440429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bf4285f60 0x556bf4289890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.751+0000 7fd401360700 1 -- 172.21.15.7:0/480440429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bf4286880 msgr2=0x556bf4289dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:19.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.751+0000 7fd401360700 1 --2- 172.21.15.7:0/480440429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bf4286880 0x556bf4289dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.751+0000 7fd401360700 1 -- 172.21.15.7:0/480440429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bf4197750 con 0x556bf42807a0 2024-06-10T01:18:19.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.751+0000 7fd401360700 1 --2- 172.21.15.7:0/480440429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bf42807a0 0x556bf428c590 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fd3f800b2e0 tx=0x7fd3f8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=479b2be89a53fac1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:19.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.751+0000 7fd3e4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:18:19.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.751+0000 7fd3e4ff9700 1 -- 172.21.15.7:0/480440429 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f800f460 con 0x556bf42807a0 2024-06-10T01:18:19.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.751+0000 7fd3e4ff9700 1 -- 172.21.15.7:0/480440429 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fd3f800f600 con 0x556bf42807a0 2024-06-10T01:18:19.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.751+0000 7fd40de47b80 1 -- 172.21.15.7:0/480440429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3f4013dd0 con 0x556bf42807a0 2024-06-10T01:18:19.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.752+0000 7fd3e4ff9700 1 -- 172.21.15.7:0/480440429 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd3f800f800 con 0x556bf42807a0 2024-06-10T01:18:19.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.752+0000 7fd40de47b80 10 client.18579 fetch_fsmap learned FSMap version 40 2024-06-10T01:18:19.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.752+0000 7fd40de47b80 1 -- 172.21.15.7:0/480440429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x556bf4297980 con 0x556bf42807a0 2024-06-10T01:18:19.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.752+0000 7fd3e4ff9700 1 -- 172.21.15.7:0/480440429 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3f4013dd0 con 0x556bf42807a0 2024-06-10T01:18:19.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.752+0000 7fd3e4ff9700 1 -- 172.21.15.7:0/480440429 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7fd3f8032b50 con 0x556bf42807a0 2024-06-10T01:18:19.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.752+0000 7fd40de47b80 10 client.18579 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:18:19.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.752+0000 7fd40de47b80 10 client.18579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:18:19.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.752+0000 7fd40de47b80 20 client.18579 populate_metadata read hostname 'smithi007' 2024-06-10T01:18:19.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.752+0000 7fd40de47b80 1 --2- 172.21.15.7:0/480440429 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556bf4299fc0 0x556bf429c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:19.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.752+0000 7fd40de47b80 4 client.18579 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:18:19.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.752+0000 7fd40de47b80 1 -- 172.21.15.7:0/480440429 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bf41ded50 con 0x556bf4299fc0 2024-06-10T01:18:19.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.752+0000 7fd400b5f700 1 --2- 172.21.15.7:0/480440429 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556bf4299fc0 0x556bf429c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:19.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.753+0000 7fd400b5f700 1 --2- 172.21.15.7:0/480440429 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556bf4299fc0 0x556bf429c3f0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8748b1594df92d2 server_cookie=5613dd2e8bee19d5 in_seq=0 out_seq=0 2024-06-10T01:18:19.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.753+0000 7fd3e4ff9700 10 client.18579 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:19.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.753+0000 7fd3e4ff9700 1 -- 172.21.15.7:0/480440429 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556bf41ded50 con 0x556bf4299fc0 2024-06-10T01:18:19.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.753+0000 7fd3e4ff9700 10 client.18579 handle_command_reply: tid=0 2024-06-10T01:18:19.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.753+0000 7fd40de47b80 1 client.18579 shutdown 2024-06-10T01:18:19.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.753+0000 7fd40de47b80 1 -- 172.21.15.7:0/480440429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bf42807a0 msgr2=0x556bf428c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:19.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.753+0000 7fd40de47b80 1 --2- 172.21.15.7:0/480440429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bf42807a0 0x556bf428c590 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fd3f800b2e0 tx=0x7fd3f8020000 comp rx=0 tx=0).stop 2024-06-10T01:18:19.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.753+0000 7fd40de47b80 1 -- 172.21.15.7:0/480440429 shutdown_connections 2024-06-10T01:18:19.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.753+0000 7fd40de47b80 1 --2- 172.21.15.7:0/480440429 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556bf4299fc0 0x556bf429c3f0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.753+0000 7fd40de47b80 1 --2- 172.21.15.7:0/480440429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bf4286880 0x556bf4289dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.753+0000 7fd40de47b80 1 --2- 172.21.15.7:0/480440429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bf42807a0 0x556bf428c590 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.753+0000 7fd3e4ff9700 0 client.18579 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:19.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.753+0000 7fd40de47b80 1 --2- 172.21.15.7:0/480440429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bf4285f60 0x556bf4289890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.753+0000 7fd40de47b80 1 -- 172.21.15.7:0/480440429 >> 172.21.15.7:0/480440429 conn(0x556bf41ad080 msgr2=0x556bf427a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:19.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.754+0000 7fd40de47b80 1 -- 172.21.15.7:0/480440429 shutdown_connections 2024-06-10T01:18:19.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.754+0000 7fd40de47b80 1 -- 172.21.15.7:0/480440429 wait complete. 2024-06-10T01:18:19.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.754+0000 7fd40de47b80 20 client.18579 trim_cache size 0 max 16384 2024-06-10T01:18:19.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.769+0000 7fd40de47b80 1 Processor -- start 2024-06-10T01:18:19.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.769+0000 7fd40de47b80 1 -- start start 2024-06-10T01:18:19.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.770+0000 7fd40de47b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bf42807a0 0x556bf4280260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:19.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.770+0000 7fd40de47b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bf4285f60 0x556bf4287370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:19.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.770+0000 7fd40de47b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bf4286880 0x556bf427ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:19.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.770+0000 7fd40de47b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc1b3510 con 0x556bf42807a0 2024-06-10T01:18:19.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.770+0000 7fd40de47b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc1b0da0 con 0x556bf4285f60 2024-06-10T01:18:19.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.770+0000 7fd40de47b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc1aebf0 con 0x556bf4286880 2024-06-10T01:18:19.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.770+0000 7fd401360700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bf42807a0 0x556bf4280260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:19.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.770+0000 7fd401360700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bf42807a0 0x556bf4280260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45148/0 (socket says 172.21.15.7:45148) 2024-06-10T01:18:19.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.770+0000 7fd401360700 1 -- 172.21.15.7:0/2898280629 learned_addr learned my addr 172.21.15.7:0/2898280629 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:19.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.770+0000 7fd400b5f700 1 --2- 172.21.15.7:0/2898280629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bf4285f60 0x556bf4287370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:19.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.770+0000 7fd401b61700 1 --2- 172.21.15.7:0/2898280629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bf4286880 0x556bf427ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:19.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.770+0000 7fd401360700 1 -- 172.21.15.7:0/2898280629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bf4286880 msgr2=0x556bf427ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:19.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.770+0000 7fd401360700 1 --2- 172.21.15.7:0/2898280629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bf4286880 0x556bf427ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.770+0000 7fd401360700 1 -- 172.21.15.7:0/2898280629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bf4285f60 msgr2=0x556bf4287370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:19.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.770+0000 7fd401360700 1 --2- 172.21.15.7:0/2898280629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bf4285f60 0x556bf4287370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.770+0000 7fd401360700 1 -- 172.21.15.7:0/2898280629 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bf429ca70 con 0x556bf42807a0 2024-06-10T01:18:19.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.770+0000 7fd401360700 1 --2- 172.21.15.7:0/2898280629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bf42807a0 0x556bf4280260 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8032ec0 tx=0x7fd3f8024380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e1b3dc4049e156 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:19.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.771+0000 7fd3e6ffd700 1 -- 172.21.15.7:0/2898280629 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8044020 con 0x556bf42807a0 2024-06-10T01:18:19.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.771+0000 7fd3e6ffd700 1 -- 172.21.15.7:0/2898280629 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3f8005040 con 0x556bf42807a0 2024-06-10T01:18:19.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.771+0000 7fd3e6ffd700 1 -- 172.21.15.7:0/2898280629 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f800f660 con 0x556bf42807a0 2024-06-10T01:18:19.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.771+0000 7fd40de47b80 1 -- 172.21.15.7:0/2898280629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bf42807a0 msgr2=0x556bf4280260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:19.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.771+0000 7fd40de47b80 1 --2- 172.21.15.7:0/2898280629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bf42807a0 0x556bf4280260 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8032ec0 tx=0x7fd3f8024380 comp rx=0 tx=0).stop 2024-06-10T01:18:19.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.771+0000 7fd40de47b80 1 -- 172.21.15.7:0/2898280629 shutdown_connections 2024-06-10T01:18:19.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.771+0000 7fd40de47b80 1 --2- 172.21.15.7:0/2898280629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bf4286880 0x556bf427ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.771+0000 7fd40de47b80 1 --2- 172.21.15.7:0/2898280629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bf42807a0 0x556bf4280260 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.771+0000 7fd40de47b80 1 --2- 172.21.15.7:0/2898280629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bf4285f60 0x556bf4287370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.771+0000 7fd40de47b80 1 -- 172.21.15.7:0/2898280629 >> 172.21.15.7:0/2898280629 conn(0x556bf41ad080 msgr2=0x556bf4286c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:19.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.771+0000 7fd40de47b80 1 -- 172.21.15.7:0/2898280629 shutdown_connections 2024-06-10T01:18:19.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.771+0000 7fd40de47b80 1 -- 172.21.15.7:0/2898280629 wait complete. 2024-06-10T01:18:19.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.772+0000 7fd40de47b80 1 -- start start 2024-06-10T01:18:19.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.772+0000 7fd40de47b80 1 Processor -- start 2024-06-10T01:18:19.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.772+0000 7fd40de47b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bf42807a0 0x556bf4292b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:19.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.772+0000 7fd40de47b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bf4285f60 0x556bf42a1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:19.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.772+0000 7fd40de47b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bf4286880 0x556bf42a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:19.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.772+0000 7fd40de47b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc1af260 con 0x556bf4285f60 2024-06-10T01:18:19.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.772+0000 7fd40de47b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc1b06d0 con 0x556bf4286880 2024-06-10T01:18:19.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.772+0000 7fd40de47b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc1af930 con 0x556bf42807a0 2024-06-10T01:18:19.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.772+0000 7fd40de47b80 10 client.0 _ll_register_callbacks cb 0x556bf41675b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:18:19.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.772+0000 7fd400b5f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bf4285f60 0x556bf42a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:19.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.772+0000 7fd401360700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bf42807a0 0x556bf4292b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:19.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.772+0000 7fd400b5f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bf4285f60 0x556bf42a1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45162/0 (socket says 172.21.15.7:45162) 2024-06-10T01:18:19.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.772+0000 7fd400b5f700 1 -- 172.21.15.7:0/3097371469 learned_addr learned my addr 172.21.15.7:0/3097371469 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:19.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.772+0000 7fd401b61700 1 --2- 172.21.15.7:0/3097371469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bf4286880 0x556bf42a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:19.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.773+0000 7fd401360700 1 -- 172.21.15.7:0/3097371469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bf4286880 msgr2=0x556bf42a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:19.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.773+0000 7fd401360700 1 --2- 172.21.15.7:0/3097371469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bf4286880 0x556bf42a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.773+0000 7fd401360700 1 -- 172.21.15.7:0/3097371469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bf4285f60 msgr2=0x556bf42a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:19.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.773+0000 7fd401360700 1 --2- 172.21.15.7:0/3097371469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bf4285f60 0x556bf42a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.773+0000 7fd401360700 1 -- 172.21.15.7:0/3097371469 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bf42a2140 con 0x556bf42807a0 2024-06-10T01:18:19.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.773+0000 7fd400b5f700 1 --2- 172.21.15.7:0/3097371469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bf4285f60 0x556bf42a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:18:19.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.773+0000 7fd401360700 1 --2- 172.21.15.7:0/3097371469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bf42807a0 0x556bf4292b40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd3f800c010 tx=0x7fd3f800f710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8f61fce496e560c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:19.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.773+0000 7fd3e4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:18:19.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.773+0000 7fd3e4ff9700 1 -- 172.21.15.7:0/3097371469 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8044070 con 0x556bf42807a0 2024-06-10T01:18:19.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.773+0000 7fd40de47b80 1 -- 172.21.15.7:0/3097371469 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3f8005040 con 0x556bf42807a0 2024-06-10T01:18:19.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.773+0000 7fd3e4ff9700 1 -- 172.21.15.7:0/3097371469 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fd3f80051f0 con 0x556bf42807a0 2024-06-10T01:18:19.817 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:18:19.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.773+0000 7fd3e4ff9700 1 -- 172.21.15.7:0/3097371469 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd3f8056050 con 0x556bf42807a0 2024-06-10T01:18:19.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.773+0000 7fd40de47b80 10 client.27919 fetch_fsmap learned FSMap version 40 2024-06-10T01:18:19.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.773+0000 7fd40de47b80 1 -- 172.21.15.7:0/3097371469 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x556bf42a2140 con 0x556bf42807a0 2024-06-10T01:18:19.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.774+0000 7fd3e4ff9700 1 -- 172.21.15.7:0/3097371469 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3f8005040 con 0x556bf42807a0 2024-06-10T01:18:19.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.774+0000 7fd3e4ff9700 1 -- 172.21.15.7:0/3097371469 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7fd3f8051030 con 0x556bf42807a0 2024-06-10T01:18:19.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.774+0000 7fd40de47b80 10 client.27919 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:18:19.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.774+0000 7fd40de47b80 10 client.27919 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:18:19.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.774+0000 7fd40de47b80 20 client.27919 populate_metadata read hostname 'smithi007' 2024-06-10T01:18:19.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.774+0000 7fd40de47b80 1 --2- 172.21.15.7:0/3097371469 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556bf4304eb0 0x556bf43072d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:19.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.774+0000 7fd40de47b80 4 client.27919 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:18:19.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.774+0000 7fd40de47b80 1 -- 172.21.15.7:0/3097371469 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556bf41d92d0 con 0x556bf4304eb0 2024-06-10T01:18:19.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.774+0000 7fd400b5f700 1 --2- 172.21.15.7:0/3097371469 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556bf4304eb0 0x556bf43072d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:19.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.775+0000 7fd3e4ff9700 10 client.27919 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:19.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.775+0000 7fd400b5f700 1 --2- 172.21.15.7:0/3097371469 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556bf4304eb0 0x556bf43072d0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=955cea2538af4418 server_cookie=1e28fea33227a65a in_seq=0 out_seq=0 2024-06-10T01:18:19.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.775+0000 7fd3e4ff9700 1 -- 172.21.15.7:0/3097371469 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556bf41d92d0 con 0x556bf4304eb0 2024-06-10T01:18:19.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.775+0000 7fd3e4ff9700 10 client.27919 handle_command_reply: tid=0 2024-06-10T01:18:19.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.775+0000 7fd40de47b80 1 client.27919 shutdown 2024-06-10T01:18:19.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.775+0000 7fd40de47b80 1 -- 172.21.15.7:0/3097371469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bf42807a0 msgr2=0x556bf4292b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:19.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.775+0000 7fd40de47b80 1 --2- 172.21.15.7:0/3097371469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bf42807a0 0x556bf4292b40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd3f800c010 tx=0x7fd3f800f710 comp rx=0 tx=0).stop 2024-06-10T01:18:19.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.775+0000 7fd40de47b80 1 -- 172.21.15.7:0/3097371469 shutdown_connections 2024-06-10T01:18:19.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.775+0000 7fd40de47b80 1 --2- 172.21.15.7:0/3097371469 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556bf4304eb0 0x556bf43072d0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.775+0000 7fd40de47b80 1 --2- 172.21.15.7:0/3097371469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bf4286880 0x556bf42a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.775+0000 7fd40de47b80 1 --2- 172.21.15.7:0/3097371469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bf42807a0 0x556bf4292b40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.775+0000 7fd3e4ff9700 0 client.27919 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:19.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.775+0000 7fd40de47b80 1 --2- 172.21.15.7:0/3097371469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bf4285f60 0x556bf42a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.775+0000 7fd40de47b80 1 -- 172.21.15.7:0/3097371469 >> 172.21.15.7:0/3097371469 conn(0x556bf41ad080 msgr2=0x556bf428b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:19.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.775+0000 7fd40de47b80 1 -- 172.21.15.7:0/3097371469 shutdown_connections 2024-06-10T01:18:19.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.775+0000 7fd40de47b80 1 -- 172.21.15.7:0/3097371469 wait complete. 2024-06-10T01:18:19.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.775+0000 7fd40de47b80 20 client.27919 trim_cache size 0 max 16384 2024-06-10T01:18:19.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.776+0000 7fd4035c4700 1 -- 172.21.15.7:0/3308091335 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd3e80a2c20 msgr2=0x7fd3e80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:19.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.776+0000 7fd4035c4700 1 --2- 172.21.15.7:0/3308091335 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd3e80a2c20 0x7fd3e80a50e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8000c00 tx=0x7fd3f8005fd0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.776+0000 7fd4035c4700 1 -- 172.21.15.7:0/3308091335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3fc1017f0 msgr2=0x7fd3fc19b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:19.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.776+0000 7fd4035c4700 1 --2- 172.21.15.7:0/3308091335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3fc1017f0 0x7fd3fc19b630 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec00e9a0 tx=0x7fd3ec00ecb0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.776+0000 7fd4035c4700 1 -- 172.21.15.7:0/3308091335 shutdown_connections 2024-06-10T01:18:19.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.776+0000 7fd4035c4700 1 --2- 172.21.15.7:0/3308091335 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd3e80a2c20 0x7fd3e80a50e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.776+0000 7fd4035c4700 1 --2- 172.21.15.7:0/3308091335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3fc19f9c0 0x7fd3fc19fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.776+0000 7fd4035c4700 1 --2- 172.21.15.7:0/3308091335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3fc100e30 0x7fd3fc19b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.776+0000 7fd4035c4700 1 --2- 172.21.15.7:0/3308091335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3fc1017f0 0x7fd3fc19b630 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:19.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.776+0000 7fd4035c4700 1 -- 172.21.15.7:0/3308091335 >> 172.21.15.7:0/3308091335 conn(0x7fd3fc0766b0 msgr2=0x7fd3fc0ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:19.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.776+0000 7fd4035c4700 1 -- 172.21.15.7:0/3308091335 shutdown_connections 2024-06-10T01:18:19.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:19.776+0000 7fd4035c4700 1 -- 172.21.15.7:0/3308091335 wait complete. 2024-06-10T01:18:19.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:19 smithi078 ceph-mon[36683]: pgmap v801: 129 pgs: 129 active+clean; 144 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 87 op/s 2024-06-10T01:18:20.755 INFO:tasks.workunit.client.0.smithi007.stdout: 1 890941 7.17 MB/sec execute 541 sec latency 35.118 ms 2024-06-10T01:18:21.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:21 smithi175 ceph-mon[36636]: pgmap v802: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.1 MiB/s wr, 193 op/s 2024-06-10T01:18:21.755 INFO:tasks.workunit.client.0.smithi007.stdout: 1 892147 7.17 MB/sec execute 542 sec latency 12.798 ms 2024-06-10T01:18:21.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:21 smithi007 ceph-mon[32398]: pgmap v802: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.1 MiB/s wr, 193 op/s 2024-06-10T01:18:21.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:21 smithi078 ceph-mon[36683]: pgmap v802: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.1 MiB/s wr, 193 op/s 2024-06-10T01:18:21.989 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:18:21.990 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:18:22.252 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:18:22.755 INFO:tasks.workunit.client.0.smithi007.stdout: 1 893814 7.17 MB/sec execute 543 sec latency 248.969 ms 2024-06-10T01:18:23.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:23 smithi175 ceph-mon[36636]: pgmap v803: 129 pgs: 129 active+clean; 143 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.8 MiB/s wr, 196 op/s 2024-06-10T01:18:23.711 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:23 smithi007 ceph-mon[32398]: pgmap v803: 129 pgs: 129 active+clean; 143 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.8 MiB/s wr, 196 op/s 2024-06-10T01:18:23.755 INFO:tasks.workunit.client.0.smithi007.stdout: 1 895128 7.17 MB/sec execute 544 sec latency 21.552 ms 2024-06-10T01:18:23.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:23 smithi078 ceph-mon[36683]: pgmap v803: 129 pgs: 129 active+clean; 143 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.8 MiB/s wr, 196 op/s 2024-06-10T01:18:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.112+0000 7f5b49b1d640 1 -- 172.21.15.7:0/3282295996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b44107700 msgr2=0x7f5b44109b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.112+0000 7f5b49b1d640 1 --2- 172.21.15.7:0/3282295996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b44107700 0x7f5b44109b50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c009ca0 tx=0x7f5b3c02a8c0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.112+0000 7f5b49b1d640 1 -- 172.21.15.7:0/3282295996 shutdown_connections 2024-06-10T01:18:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.112+0000 7f5b49b1d640 1 --2- 172.21.15.7:0/3282295996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b44107700 0x7f5b44109b50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.112+0000 7f5b49b1d640 1 --2- 172.21.15.7:0/3282295996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b44104dd0 0x7f5b441071c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.112+0000 7f5b49b1d640 1 --2- 172.21.15.7:0/3282295996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b441024a0 0x7f5b44104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.112+0000 7f5b49b1d640 1 -- 172.21.15.7:0/3282295996 >> 172.21.15.7:0/3282295996 conn(0x7f5b440fbeb0 msgr2=0x7f5b440fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.112+0000 7f5b49b1d640 1 -- 172.21.15.7:0/3282295996 shutdown_connections 2024-06-10T01:18:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.113+0000 7f5b49b1d640 1 -- 172.21.15.7:0/3282295996 wait complete. 2024-06-10T01:18:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.113+0000 7f5b49b1d640 1 Processor -- start 2024-06-10T01:18:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.114+0000 7f5b49b1d640 1 -- start start 2024-06-10T01:18:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.114+0000 7f5b49b1d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b441024a0 0x7f5b44197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.114+0000 7f5b49b1d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b44104dd0 0x7f5b44198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.114+0000 7f5b49b1d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b44107700 0x7f5b4419f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.114+0000 7f5b49b1d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4410d710 con 0x7f5b44107700 2024-06-10T01:18:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.114+0000 7f5b49b1d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4410d590 con 0x7f5b441024a0 2024-06-10T01:18:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.114+0000 7f5b49b1d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4410d890 con 0x7f5b44104dd0 2024-06-10T01:18:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.114+0000 7f5b437fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b441024a0 0x7f5b44197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.114+0000 7f5b437fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b441024a0 0x7f5b44197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:52920/0 (socket says 172.21.15.7:52920) 2024-06-10T01:18:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.114+0000 7f5b437fe640 1 -- 172.21.15.7:0/661017461 learned_addr learned my addr 172.21.15.7:0/661017461 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.114+0000 7f5b42ffd640 1 --2- 172.21.15.7:0/661017461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b44104dd0 0x7f5b44198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.114+0000 7f5b43fff640 1 --2- 172.21.15.7:0/661017461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b44107700 0x7f5b4419f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.114+0000 7f5b437fe640 1 -- 172.21.15.7:0/661017461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b44104dd0 msgr2=0x7f5b44198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.114+0000 7f5b437fe640 1 --2- 172.21.15.7:0/661017461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b44104dd0 0x7f5b44198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.114+0000 7f5b437fe640 1 -- 172.21.15.7:0/661017461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b44107700 msgr2=0x7f5b4419f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.114+0000 7f5b437fe640 1 --2- 172.21.15.7:0/661017461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b44107700 0x7f5b4419f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.114+0000 7f5b437fe640 1 -- 172.21.15.7:0/661017461 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4419f930 con 0x7f5b441024a0 2024-06-10T01:18:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.115+0000 7f5b437fe640 1 --2- 172.21.15.7:0/661017461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b441024a0 0x7f5b44197d20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400e9f0 tx=0x7f5b3400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.115+0000 7f5b40ff9640 1 -- 172.21.15.7:0/661017461 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3400cd80 con 0x7f5b441024a0 2024-06-10T01:18:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.115+0000 7f5b40ff9640 1 -- 172.21.15.7:0/661017461 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b34005500 con 0x7f5b441024a0 2024-06-10T01:18:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.115+0000 7f5b40ff9640 1 -- 172.21.15.7:0/661017461 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b34010690 con 0x7f5b441024a0 2024-06-10T01:18:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.115+0000 7f5b49b1d640 1 -- 172.21.15.7:0/661017461 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b4419fc20 con 0x7f5b441024a0 2024-06-10T01:18:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.115+0000 7f5b49b1d640 1 -- 172.21.15.7:0/661017461 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b441a0120 con 0x7f5b441024a0 2024-06-10T01:18:24.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.115+0000 7f8c2eaa8700 1 -- 172.21.15.7:0/1464742827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c28101380 msgr2=0x7f8c28101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:24.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.115+0000 7f8c2eaa8700 1 --2- 172.21.15.7:0/1464742827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c28101380 0x7f8c28101800 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f8c140098f0 tx=0x7f8c14009c00 comp rx=0 tx=0).stop 2024-06-10T01:18:24.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.115+0000 7f8c2eaa8700 1 -- 172.21.15.7:0/1464742827 shutdown_connections 2024-06-10T01:18:24.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.115+0000 7f8c2eaa8700 1 --2- 172.21.15.7:0/1464742827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c28101d40 0x7f8c2810e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.115+0000 7f8c2eaa8700 1 --2- 172.21.15.7:0/1464742827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c281073f0 0x7f8c281077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.115+0000 7f8c2eaa8700 1 --2- 172.21.15.7:0/1464742827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c28101380 0x7f8c28101800 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.115+0000 7f8c2eaa8700 1 -- 172.21.15.7:0/1464742827 >> 172.21.15.7:0/1464742827 conn(0x7f8c28078300 msgr2=0x7f8c28078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:24.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.116+0000 7f8c2eaa8700 1 -- 172.21.15.7:0/1464742827 shutdown_connections 2024-06-10T01:18:24.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.116+0000 7f8c2eaa8700 1 -- 172.21.15.7:0/1464742827 wait complete. 2024-06-10T01:18:24.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.116+0000 7f8c2eaa8700 1 Processor -- start 2024-06-10T01:18:24.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.116+0000 7f8c2eaa8700 1 -- start start 2024-06-10T01:18:24.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.117+0000 7f8c2eaa8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c28101380 0x7f8c28103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.117+0000 7f8c2eaa8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c28101d40 0x7f8c281035a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.117+0000 7f8c2eaa8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c281073f0 0x7f8c28103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.117+0000 7f8c2eaa8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8c28111660 con 0x7f8c28101380 2024-06-10T01:18:24.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.117+0000 7f8c2eaa8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8c281114e0 con 0x7f8c28101d40 2024-06-10T01:18:24.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.117+0000 7f8c2eaa8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8c281117e0 con 0x7f8c281073f0 2024-06-10T01:18:24.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.117+0000 7f8c2c844700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c28101380 0x7f8c28103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.117+0000 7f8c2c844700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c28101380 0x7f8c28103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54288/0 (socket says 172.21.15.7:54288) 2024-06-10T01:18:24.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.117+0000 7f8c2d045700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c281073f0 0x7f8c28103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.117+0000 7f8c2c844700 1 -- 172.21.15.7:0/465978552 learned_addr learned my addr 172.21.15.7:0/465978552 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:24.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.117+0000 7f8c27fff700 1 --2- 172.21.15.7:0/465978552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c28101d40 0x7f8c281035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.117+0000 7f8c2c844700 1 -- 172.21.15.7:0/465978552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c281073f0 msgr2=0x7f8c28103ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:24.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.117+0000 7f8c2c844700 1 --2- 172.21.15.7:0/465978552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c281073f0 0x7f8c28103ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.117+0000 7f8c2c844700 1 -- 172.21.15.7:0/465978552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c28101d40 msgr2=0x7f8c281035a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:24.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.117+0000 7f8c2c844700 1 --2- 172.21.15.7:0/465978552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c28101d40 0x7f8c281035a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.117+0000 7f8c2c844700 1 -- 172.21.15.7:0/465978552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c281a71a0 con 0x7f8c28101380 2024-06-10T01:18:24.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.118+0000 7f8c2c844700 1 --2- 172.21.15.7:0/465978552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c28101380 0x7f8c28103060 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00ec10 tx=0x7f8c1c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:24.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.118+0000 7f8c25ffb700 1 -- 172.21.15.7:0/465978552 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c00cc80 con 0x7f8c28101380 2024-06-10T01:18:24.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.118+0000 7f8c25ffb700 1 -- 172.21.15.7:0/465978552 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c1c004500 con 0x7f8c28101380 2024-06-10T01:18:24.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.118+0000 7f8c25ffb700 1 -- 172.21.15.7:0/465978552 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c010690 con 0x7f8c28101380 2024-06-10T01:18:24.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.118+0000 7f8c2eaa8700 1 -- 172.21.15.7:0/465978552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c281a7490 con 0x7f8c28101380 2024-06-10T01:18:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.118+0000 7f5b40ff9640 1 -- 172.21.15.7:0/661017461 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f5b340046c0 con 0x7f5b441024a0 2024-06-10T01:18:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.118+0000 7f5b4c822740 1 Processor -- start 2024-06-10T01:18:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.119+0000 7f5b4c822740 1 -- start start 2024-06-10T01:18:24.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.119+0000 7f8c2eaa8700 1 -- 172.21.15.7:0/465978552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c281a79a0 con 0x7f8c28101380 2024-06-10T01:18:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.119+0000 7f5b4c822740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558328974010 0x558328974430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.119+0000 7f5b4c822740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558328974a00 0x558328979500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.119+0000 7f5b4c822740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558328979b70 0x55832897c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.119+0000 7f5b4c822740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5583288d74f0 con 0x558328974010 2024-06-10T01:18:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.119+0000 7f5b4c822740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558328875870 con 0x558328974a00 2024-06-10T01:18:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.119+0000 7f5b4c822740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5583288d34b0 con 0x558328979b70 2024-06-10T01:18:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.119+0000 7f5b42ffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558328979b70 0x55832897c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.119+0000 7f5b42ffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558328979b70 0x55832897c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:39920/0 (socket says 172.21.15.7:39920) 2024-06-10T01:18:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.119+0000 7f5b42ffd640 1 -- 172.21.15.7:0/3347660673 learned_addr learned my addr 172.21.15.7:0/3347660673 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.119+0000 7f5b43fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558328974010 0x558328974430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.119+0000 7f5b437fe640 1 --2- 172.21.15.7:0/3347660673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558328974a00 0x558328979500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.120+0000 7f5b42ffd640 1 -- 172.21.15.7:0/3347660673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558328974a00 msgr2=0x558328979500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.120+0000 7f5b42ffd640 1 --2- 172.21.15.7:0/3347660673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558328974a00 0x558328979500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.120+0000 7f5b42ffd640 1 -- 172.21.15.7:0/3347660673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558328974010 msgr2=0x558328974430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.120+0000 7f5b42ffd640 1 --2- 172.21.15.7:0/3347660673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558328974010 0x558328974430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.120+0000 7f5b42ffd640 1 -- 172.21.15.7:0/3347660673 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583288919b0 con 0x558328979b70 2024-06-10T01:18:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.120+0000 7f5b40ff9640 1 --2- 172.21.15.7:0/661017461 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5b2c0a2b80 0x7f5b2c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.120+0000 7f5b42ffd640 1 --2- 172.21.15.7:0/3347660673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558328979b70 0x55832897c000 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5b38009c90 tx=0x7f5b38011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20a396d5a5797efe server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.120+0000 7f5b43fff640 1 --2- 172.21.15.7:0/661017461 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5b2c0a2b80 0x7f5b2c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.120+0000 7f5b29ffb640 1 -- 172.21.15.7:0/3347660673 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b38023070 con 0x558328979b70 2024-06-10T01:18:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.120+0000 7f5b29ffb640 1 -- 172.21.15.7:0/3347660673 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b38009e40 con 0x558328979b70 2024-06-10T01:18:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.120+0000 7f5b40ff9640 1 -- 172.21.15.7:0/661017461 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f5b340cf890 con 0x7f5b441024a0 2024-06-10T01:18:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.121+0000 7f5b29ffb640 1 -- 172.21.15.7:0/3347660673 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3801eaf0 con 0x558328979b70 2024-06-10T01:18:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.121+0000 7f5b43fff640 1 --2- 172.21.15.7:0/661017461 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5b2c0a2b80 0x7f5b2c0a5040 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c006010 tx=0x7f5b3c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.121+0000 7f5b4c822740 1 -- 172.21.15.7:0/3347660673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558328979b70 msgr2=0x55832897c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.121+0000 7f5b4c822740 1 --2- 172.21.15.7:0/3347660673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558328979b70 0x55832897c000 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5b38009c90 tx=0x7f5b38011810 comp rx=0 tx=0).stop 2024-06-10T01:18:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.121+0000 7f5b4c822740 1 -- 172.21.15.7:0/3347660673 shutdown_connections 2024-06-10T01:18:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.121+0000 7f5b4c822740 1 --2- 172.21.15.7:0/3347660673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558328979b70 0x55832897c000 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.121+0000 7f5b4c822740 1 --2- 172.21.15.7:0/3347660673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558328974a00 0x558328979500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.121+0000 7f5b4c822740 1 --2- 172.21.15.7:0/3347660673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558328974010 0x558328974430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.121+0000 7f5b4c822740 1 -- 172.21.15.7:0/3347660673 >> 172.21.15.7:0/3347660673 conn(0x55832896f680 msgr2=0x55832896fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.121+0000 7f5b4c822740 1 -- 172.21.15.7:0/3347660673 shutdown_connections 2024-06-10T01:18:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.121+0000 7f5b4c822740 1 -- 172.21.15.7:0/3347660673 wait complete. 2024-06-10T01:18:24.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.121+0000 7f8c25ffb700 1 -- 172.21.15.7:0/465978552 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f8c1c003e90 con 0x7f8c28101380 2024-06-10T01:18:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.122+0000 7f5b4c822740 1 -- start start 2024-06-10T01:18:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.122+0000 7f5b4c822740 1 Processor -- start 2024-06-10T01:18:24.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.122+0000 7f8c25ffb700 1 --2- 172.21.15.7:0/465978552 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8c180a2d70 0x7f8c180a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.122+0000 7f8c25ffb700 1 -- 172.21.15.7:0/465978552 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f8c1c0cf570 con 0x7f8c28101380 2024-06-10T01:18:24.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.122+0000 7f8c27fff700 1 --2- 172.21.15.7:0/465978552 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8c180a2d70 0x7f8c180a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.123+0000 7f8c27fff700 1 --2- 172.21.15.7:0/465978552 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8c180a2d70 0x7f8c180a5230 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8c140098c0 tx=0x7f8c14019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.123+0000 7f5b4c822740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558328974010 0x55832897e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.123+0000 7f5b4c822740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558328974a00 0x55832897e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.123+0000 7f5b437fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558328974010 0x55832897e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.123+0000 7f5b437fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558328974010 0x55832897e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:52926/0 (socket says 172.21.15.7:52926) 2024-06-10T01:18:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.123+0000 7f5b437fe640 1 -- 172.21.15.7:0/2619717063 learned_addr learned my addr 172.21.15.7:0/2619717063 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:24.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.124+0000 7f8c3932bb80 1 Processor -- start 2024-06-10T01:18:24.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.124+0000 7f5b42ffd640 1 --2- 172.21.15.7:0/2619717063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558328974a00 0x55832897e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.124+0000 7f5b4c822740 1 --2- 172.21.15.7:0/2619717063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558328979b70 0x558328985840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.124+0000 7f8c3932bb80 1 -- start start 2024-06-10T01:18:24.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.124+0000 7f5b4c822740 1 -- 172.21.15.7:0/2619717063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55832897cbc0 con 0x558328974a00 2024-06-10T01:18:24.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.124+0000 7f8c3932bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ae6fcb6c0 0x564ae6fcbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.124+0000 7f8c3932bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ae6fcbfe0 0x564ae6fc5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.124+0000 7f8c3932bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ae6fc5fa0 0x564ae6fc63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.124+0000 7f8c3932bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564ae6f17890 con 0x564ae6fcb6c0 2024-06-10T01:18:24.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.124+0000 7f8c3932bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564ae6ebf7a0 con 0x564ae6fcbfe0 2024-06-10T01:18:24.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.124+0000 7f8c3932bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564ae6f13850 con 0x564ae6fc5fa0 2024-06-10T01:18:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.124+0000 7f5b43fff640 1 --2- 172.21.15.7:0/2619717063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558328979b70 0x558328985840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.124+0000 7f5b4c822740 1 -- 172.21.15.7:0/2619717063 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558328860ae0 con 0x558328974010 2024-06-10T01:18:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.124+0000 7f5b4c822740 1 -- 172.21.15.7:0/2619717063 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55832897cd40 con 0x558328979b70 2024-06-10T01:18:24.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.124+0000 7f8c27fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ae6fcbfe0 0x564ae6fc5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.124+0000 7f8c27fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ae6fcbfe0 0x564ae6fc5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:52940/0 (socket says 172.21.15.7:52940) 2024-06-10T01:18:24.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.124+0000 7f8c27fff700 1 -- 172.21.15.7:0/2156603409 learned_addr learned my addr 172.21.15.7:0/2156603409 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:24.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.124+0000 7f8c2c844700 1 --2- 172.21.15.7:0/2156603409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ae6fcb6c0 0x564ae6fcbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.124+0000 7f8c2d045700 1 --2- 172.21.15.7:0/2156603409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ae6fc5fa0 0x564ae6fc63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.124+0000 7f5b4c822740 10 client.0 _ll_register_callbacks cb 0x5583288999b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:18:24.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.124+0000 7f8c2d045700 1 -- 172.21.15.7:0/2156603409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ae6fcbfe0 msgr2=0x564ae6fc5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:24.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.125+0000 7f8c2d045700 1 --2- 172.21.15.7:0/2156603409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ae6fcbfe0 0x564ae6fc5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.125+0000 7f5b437fe640 1 -- 172.21.15.7:0/2619717063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558328979b70 msgr2=0x558328985840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.125+0000 7f5b437fe640 1 --2- 172.21.15.7:0/2619717063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558328979b70 0x558328985840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.125+0000 7f5b437fe640 1 -- 172.21.15.7:0/2619717063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558328974a00 msgr2=0x55832897e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:24.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.125+0000 7f8c2d045700 1 -- 172.21.15.7:0/2156603409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ae6fcb6c0 msgr2=0x564ae6fcbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:24.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.125+0000 7f8c2d045700 1 --2- 172.21.15.7:0/2156603409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ae6fcb6c0 0x564ae6fcbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.125+0000 7f8c2d045700 1 -- 172.21.15.7:0/2156603409 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ae6eb5fd0 con 0x564ae6fc5fa0 2024-06-10T01:18:24.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.125+0000 7f8c2c844700 1 --2- 172.21.15.7:0/2156603409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ae6fcb6c0 0x564ae6fcbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:18:24.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.125+0000 7f8c27fff700 1 --2- 172.21.15.7:0/2156603409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ae6fcbfe0 0x564ae6fc5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:18:24.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.125+0000 7f8c2d045700 1 --2- 172.21.15.7:0/2156603409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ae6fc5fa0 0x564ae6fc63c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8c20011550 tx=0x7f8c20009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f05c1ae19c64f25e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.125+0000 7f5b437fe640 1 --2- 172.21.15.7:0/2619717063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558328974a00 0x55832897e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.125+0000 7f5b437fe640 1 -- 172.21.15.7:0/2619717063 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583285eb210 con 0x558328974010 2024-06-10T01:18:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.125+0000 7f5b4c822740 1 -- 172.21.15.7:0/2619717063 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5583288eaf10 con 0x558328974010 2024-06-10T01:18:24.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.125+0000 7f8c12ffd700 1 -- 172.21.15.7:0/2156603409 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c20019020 con 0x564ae6fc5fa0 2024-06-10T01:18:24.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.125+0000 7f8c12ffd700 1 -- 172.21.15.7:0/2156603409 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c20009310 con 0x564ae6fc5fa0 2024-06-10T01:18:24.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.125+0000 7f8c12ffd700 1 -- 172.21.15.7:0/2156603409 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c20007760 con 0x564ae6fc5fa0 2024-06-10T01:18:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.125+0000 7f5b437fe640 1 --2- 172.21.15.7:0/2619717063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558328974010 0x55832897e2c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5b3409ba10 tx=0x7f5b3409b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eceebad1269eb895 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.125+0000 7f5b22ffd640 10 client.27920 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:18:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.126+0000 7f5b22ffd640 1 -- 172.21.15.7:0/2619717063 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3409bd10 con 0x558328974010 2024-06-10T01:18:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.126+0000 7f5b22ffd640 1 -- 172.21.15.7:0/2619717063 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f5b340a4700 con 0x558328974010 2024-06-10T01:18:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.126+0000 7f5b22ffd640 1 -- 172.21.15.7:0/2619717063 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5b340a00c0 con 0x558328974010 2024-06-10T01:18:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.126+0000 7f5b4c822740 10 client.27920 fetch_fsmap learned FSMap version 40 2024-06-10T01:18:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.126+0000 7f5b4c822740 1 -- 172.21.15.7:0/2619717063 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=40}) -- 0x5583286b7620 con 0x558328974010 2024-06-10T01:18:24.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.126+0000 7f8c3932bb80 1 -- 172.21.15.7:0/2156603409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ae6fc5fa0 msgr2=0x564ae6fc63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:24.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.126+0000 7f8c3932bb80 1 --2- 172.21.15.7:0/2156603409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ae6fc5fa0 0x564ae6fc63c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8c20011550 tx=0x7f8c20009920 comp rx=0 tx=0).stop 2024-06-10T01:18:24.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.126+0000 7f8c3932bb80 1 -- 172.21.15.7:0/2156603409 shutdown_connections 2024-06-10T01:18:24.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.126+0000 7f8c3932bb80 1 --2- 172.21.15.7:0/2156603409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ae6fc5fa0 0x564ae6fc63c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.126+0000 7f8c3932bb80 1 --2- 172.21.15.7:0/2156603409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ae6fcb6c0 0x564ae6fcbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.126+0000 7f8c3932bb80 1 --2- 172.21.15.7:0/2156603409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ae6fcbfe0 0x564ae6fc5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.126+0000 7f8c3932bb80 1 -- 172.21.15.7:0/2156603409 >> 172.21.15.7:0/2156603409 conn(0x564ae6ef0d40 msgr2=0x564ae6fc08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:24.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.126+0000 7f8c3932bb80 1 -- 172.21.15.7:0/2156603409 shutdown_connections 2024-06-10T01:18:24.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.126+0000 7f8c3932bb80 1 -- 172.21.15.7:0/2156603409 wait complete. 2024-06-10T01:18:24.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.126+0000 7f5b22ffd640 1 -- 172.21.15.7:0/2619717063 <== mon.2 v2:172.21.15.78:3300/0 4 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f5b340a4bd0 con 0x558328974010 2024-06-10T01:18:24.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.126+0000 7f5b4c822740 10 client.27920 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:18:24.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.126+0000 7f5b4c822740 10 client.27920 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:18:24.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.126+0000 7f5b4c822740 20 client.27920 populate_metadata read hostname 'smithi007' 2024-06-10T01:18:24.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.126+0000 7f5b4c822740 1 --2- 172.21.15.7:0/2619717063 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55832898a480 0x55832898c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.126+0000 7f5b4c822740 4 client.27920 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:18:24.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.126+0000 7f5b4c822740 1 -- 172.21.15.7:0/2619717063 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583288e4c20 con 0x55832898a480 2024-06-10T01:18:24.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.127+0000 7f5b42ffd640 1 --2- 172.21.15.7:0/2619717063 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55832898a480 0x55832898c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.127+0000 7f8c3932bb80 1 -- start start 2024-06-10T01:18:24.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.127+0000 7f8c3932bb80 1 Processor -- start 2024-06-10T01:18:24.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.127+0000 7f5b42ffd640 1 --2- 172.21.15.7:0/2619717063 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55832898a480 0x55832898c940 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d46d0900b32387ad server_cookie=79dc0f51d57528f2 in_seq=0 out_seq=0 2024-06-10T01:18:24.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.127+0000 7f5b22ffd640 10 client.27920 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:24.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.127+0000 7f8c3932bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ae6fc5fa0 0x564ae6fd1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.127+0000 7f8c3932bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ae6fcb6c0 0x564ae6fcf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.127+0000 7f8c3932bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ae6fcbfe0 0x564ae6fcf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.127+0000 7f8c3932bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564ae6fd2a50 con 0x564ae6fcb6c0 2024-06-10T01:18:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.127+0000 7f8c3932bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564ae6eb7c70 con 0x564ae6fcbfe0 2024-06-10T01:18:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.127+0000 7f8c3932bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564ae6fd2bd0 con 0x564ae6fc5fa0 2024-06-10T01:18:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.128+0000 7f8c3932bb80 10 client.0 _ll_register_callbacks cb 0x564ae6eb19a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:18:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.128+0000 7f8c2c844700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ae6fc5fa0 0x564ae6fd1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.128+0000 7f8c27fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ae6fcb6c0 0x564ae6fcf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.128+0000 7f5b22ffd640 1 -- 172.21.15.7:0/2619717063 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5583288e4c20 con 0x55832898a480 2024-06-10T01:18:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.128+0000 7f5b22ffd640 10 client.27920 handle_command_reply: tid=0 2024-06-10T01:18:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.128+0000 7f8c2d045700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ae6fcbfe0 0x564ae6fcf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.128+0000 7f8c2c844700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ae6fc5fa0 0x564ae6fd1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:39948/0 (socket says 172.21.15.7:39948) 2024-06-10T01:18:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.128+0000 7f8c2c844700 1 -- 172.21.15.7:0/2990786473 learned_addr learned my addr 172.21.15.7:0/2990786473 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.128+0000 7f5b4c822740 1 client.27920 shutdown 2024-06-10T01:18:24.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.128+0000 7f8c2d045700 1 -- 172.21.15.7:0/2990786473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ae6fc5fa0 msgr2=0x564ae6fd1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:24.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.128+0000 7f8c2d045700 1 --2- 172.21.15.7:0/2990786473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ae6fc5fa0 0x564ae6fd1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.128+0000 7f8c2d045700 1 -- 172.21.15.7:0/2990786473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ae6fcb6c0 msgr2=0x564ae6fcf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:24.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.128+0000 7f8c2d045700 1 --2- 172.21.15.7:0/2990786473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ae6fcb6c0 0x564ae6fcf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.128+0000 7f8c2d045700 1 -- 172.21.15.7:0/2990786473 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ae6e66950 con 0x564ae6fcbfe0 2024-06-10T01:18:24.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.128+0000 7f8c27fff700 1 --2- 172.21.15.7:0/2990786473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ae6fcb6c0 0x564ae6fcf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:18:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.128+0000 7f5b4c822740 1 -- 172.21.15.7:0/2619717063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558328974010 msgr2=0x55832897e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.128+0000 7f5b4c822740 1 --2- 172.21.15.7:0/2619717063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558328974010 0x55832897e2c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5b3409ba10 tx=0x7f5b3409b910 comp rx=0 tx=0).stop 2024-06-10T01:18:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.128+0000 7f5b4c822740 1 -- 172.21.15.7:0/2619717063 shutdown_connections 2024-06-10T01:18:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.128+0000 7f5b4c822740 1 --2- 172.21.15.7:0/2619717063 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55832898a480 0x55832898c940 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.128+0000 7f5b4c822740 1 --2- 172.21.15.7:0/2619717063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558328979b70 0x558328985840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.128+0000 7f8c2d045700 1 --2- 172.21.15.7:0/2990786473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ae6fcbfe0 0x564ae6fcf590 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8c200046a0 tx=0x7f8c20004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0e8bf79fd462e0b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.128+0000 7f5b4c822740 1 --2- 172.21.15.7:0/2619717063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558328974a00 0x55832897e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.128+0000 7f5b22ffd640 0 client.27920 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.128+0000 7f5b4c822740 1 --2- 172.21.15.7:0/2619717063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558328974010 0x55832897e2c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.128+0000 7f5b4c822740 1 -- 172.21.15.7:0/2619717063 >> 172.21.15.7:0/2619717063 conn(0x55832896f680 msgr2=0x55832896e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:24.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.128+0000 7f8c07fff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:18:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.128+0000 7f5b4c822740 1 -- 172.21.15.7:0/2619717063 shutdown_connections 2024-06-10T01:18:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.129+0000 7f5b4c822740 1 -- 172.21.15.7:0/2619717063 wait complete. 2024-06-10T01:18:24.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.129+0000 7f8c07fff700 1 -- 172.21.15.7:0/2990786473 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c20019040 con 0x564ae6fcbfe0 2024-06-10T01:18:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.129+0000 7f5b4c822740 20 client.27920 trim_cache size 0 max 16384 2024-06-10T01:18:24.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.129+0000 7f8c3932bb80 1 -- 172.21.15.7:0/2990786473 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c20009310 con 0x564ae6fcbfe0 2024-06-10T01:18:24.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.129+0000 7f8c07fff700 1 -- 172.21.15.7:0/2990786473 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f8c20003c10 con 0x564ae6fcbfe0 2024-06-10T01:18:24.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.129+0000 7f8c07fff700 1 -- 172.21.15.7:0/2990786473 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8c20031050 con 0x564ae6fcbfe0 2024-06-10T01:18:24.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.129+0000 7f8c3932bb80 10 client.27923 fetch_fsmap learned FSMap version 40 2024-06-10T01:18:24.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.129+0000 7f8c3932bb80 1 -- 172.21.15.7:0/2990786473 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x564ae6fdd3b0 con 0x564ae6fcbfe0 2024-06-10T01:18:24.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.129+0000 7f8c07fff700 1 -- 172.21.15.7:0/2990786473 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c20009310 con 0x564ae6fcbfe0 2024-06-10T01:18:24.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.129+0000 7f8c07fff700 1 -- 172.21.15.7:0/2990786473 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f8c20007c40 con 0x564ae6fcbfe0 2024-06-10T01:18:24.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.129+0000 7f8c3932bb80 10 client.27923 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:18:24.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.129+0000 7f8c3932bb80 10 client.27923 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:18:24.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.129+0000 7f8c3932bb80 20 client.27923 populate_metadata read hostname 'smithi007' 2024-06-10T01:18:24.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.129+0000 7f8c3932bb80 1 --2- 172.21.15.7:0/2990786473 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564ae6fdf8e0 0x564ae6fe1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.130+0000 7f8c3932bb80 4 client.27923 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:18:24.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.130+0000 7f8c3932bb80 1 -- 172.21.15.7:0/2990786473 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ae6f24d90 con 0x564ae6fdf8e0 2024-06-10T01:18:24.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.130+0000 7f8c2c844700 1 --2- 172.21.15.7:0/2990786473 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564ae6fdf8e0 0x564ae6fe1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.130+0000 7f8c07fff700 10 client.27923 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:24.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.130+0000 7f8c2c844700 1 --2- 172.21.15.7:0/2990786473 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564ae6fdf8e0 0x564ae6fe1cd0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a6f8b3cf87f35df server_cookie=12cded7fd00f7bf1 in_seq=0 out_seq=0 2024-06-10T01:18:24.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.131+0000 7f8c07fff700 1 -- 172.21.15.7:0/2990786473 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564ae6f24d90 con 0x564ae6fdf8e0 2024-06-10T01:18:24.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.131+0000 7f8c07fff700 10 client.27923 handle_command_reply: tid=0 2024-06-10T01:18:24.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.131+0000 7f8c3932bb80 1 client.27923 shutdown 2024-06-10T01:18:24.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.131+0000 7f8c3932bb80 1 -- 172.21.15.7:0/2990786473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ae6fcbfe0 msgr2=0x564ae6fcf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:24.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.131+0000 7f8c3932bb80 1 --2- 172.21.15.7:0/2990786473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ae6fcbfe0 0x564ae6fcf590 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8c200046a0 tx=0x7f8c20004d00 comp rx=0 tx=0).stop 2024-06-10T01:18:24.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.131+0000 7f8c3932bb80 1 -- 172.21.15.7:0/2990786473 shutdown_connections 2024-06-10T01:18:24.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.131+0000 7f8c3932bb80 1 --2- 172.21.15.7:0/2990786473 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564ae6fdf8e0 0x564ae6fe1cd0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.131+0000 7f8c07fff700 0 client.27923 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:24.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.131+0000 7f8c3932bb80 1 --2- 172.21.15.7:0/2990786473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ae6fcbfe0 0x564ae6fcf590 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.131+0000 7f8c3932bb80 1 --2- 172.21.15.7:0/2990786473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ae6fc5fa0 0x564ae6fd1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.131+0000 7f8c3932bb80 1 --2- 172.21.15.7:0/2990786473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ae6fcb6c0 0x564ae6fcf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.131+0000 7f8c3932bb80 1 -- 172.21.15.7:0/2990786473 >> 172.21.15.7:0/2990786473 conn(0x564ae6ef0d40 msgr2=0x564ae6fc4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:24.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.131+0000 7f8c3932bb80 1 -- 172.21.15.7:0/2990786473 shutdown_connections 2024-06-10T01:18:24.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.131+0000 7f8c3932bb80 1 -- 172.21.15.7:0/2990786473 wait complete. 2024-06-10T01:18:24.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.131+0000 7f8c3932bb80 20 client.27923 trim_cache size 0 max 16384 2024-06-10T01:18:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.145+0000 7f5b4c822740 1 Processor -- start 2024-06-10T01:18:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.145+0000 7f5b4c822740 1 -- start start 2024-06-10T01:18:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.145+0000 7f5b4c822740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558328974010 0x55832896fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.145+0000 7f5b4c822740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558328974a00 0x558328985800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.145+0000 7f5b4c822740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558328979b70 0x558328979210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.145+0000 7f5b4c822740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5b441aabb0 con 0x558328974010 2024-06-10T01:18:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.145+0000 7f5b437fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558328974010 0x55832896fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.145+0000 7f5b4c822740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b441a8480 con 0x558328979b70 2024-06-10T01:18:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.145+0000 7f5b4c822740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5b441a62d0 con 0x558328974a00 2024-06-10T01:18:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.145+0000 7f5b437fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558328974010 0x55832896fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54336/0 (socket says 172.21.15.7:54336) 2024-06-10T01:18:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.145+0000 7f5b437fe640 1 -- 172.21.15.7:0/2488297692 learned_addr learned my addr 172.21.15.7:0/2488297692 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.145+0000 7f5b42ffd640 1 --2- 172.21.15.7:0/2488297692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558328974a00 0x558328985800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.145+0000 7f5b43fff640 1 --2- 172.21.15.7:0/2488297692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558328979b70 0x558328979210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.146+0000 7f5b437fe640 1 -- 172.21.15.7:0/2488297692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558328974a00 msgr2=0x558328985800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.146+0000 7f5b437fe640 1 --2- 172.21.15.7:0/2488297692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558328974a00 0x558328985800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.146+0000 7f5b437fe640 1 -- 172.21.15.7:0/2488297692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558328979b70 msgr2=0x558328979210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.146+0000 7f5b437fe640 1 --2- 172.21.15.7:0/2488297692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558328979b70 0x558328979210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.146+0000 7f5b437fe640 1 -- 172.21.15.7:0/2488297692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55832896f4b0 con 0x558328974010 2024-06-10T01:18:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.146+0000 7f5b437fe640 1 --2- 172.21.15.7:0/2488297692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558328974010 0x55832896fa60 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f5b340049d0 tx=0x7f5b340d1000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b27c673140d89ee server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.146+0000 7f5b297fa640 1 -- 172.21.15.7:0/2488297692 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3409bd10 con 0x558328974010 2024-06-10T01:18:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.146+0000 7f5b297fa640 1 -- 172.21.15.7:0/2488297692 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5583288eaf10 con 0x558328974010 2024-06-10T01:18:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.146+0000 7f5b297fa640 1 -- 172.21.15.7:0/2488297692 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b34099e50 con 0x558328974010 2024-06-10T01:18:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.147+0000 7f5b4c822740 1 -- 172.21.15.7:0/2488297692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558328974010 msgr2=0x55832896fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.147+0000 7f5b4c822740 1 --2- 172.21.15.7:0/2488297692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558328974010 0x55832896fa60 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f5b340049d0 tx=0x7f5b340d1000 comp rx=0 tx=0).stop 2024-06-10T01:18:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.147+0000 7f5b4c822740 1 -- 172.21.15.7:0/2488297692 shutdown_connections 2024-06-10T01:18:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.147+0000 7f5b4c822740 1 --2- 172.21.15.7:0/2488297692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558328979b70 0x558328979210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.147+0000 7f5b4c822740 1 --2- 172.21.15.7:0/2488297692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558328974a00 0x558328985800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.147+0000 7f5b4c822740 1 --2- 172.21.15.7:0/2488297692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558328974010 0x55832896fa60 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.147+0000 7f5b4c822740 1 -- 172.21.15.7:0/2488297692 >> 172.21.15.7:0/2488297692 conn(0x55832896f680 msgr2=0x55832897d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.147+0000 7f5b4c822740 1 -- 172.21.15.7:0/2488297692 shutdown_connections 2024-06-10T01:18:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.147+0000 7f5b4c822740 1 -- 172.21.15.7:0/2488297692 wait complete. 2024-06-10T01:18:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.147+0000 7f5b4c822740 1 -- start start 2024-06-10T01:18:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.147+0000 7f5b4c822740 1 Processor -- start 2024-06-10T01:18:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.148+0000 7f5b4c822740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558328974010 0x5583289853b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.148+0000 7f5b4c822740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558328974a00 0x558328981e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.148+0000 7f5b4c822740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558328979b70 0x558328982390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.148+0000 7f5b437fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558328974010 0x5583289853b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.148+0000 7f5b437fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558328974010 0x5583289853b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:39968/0 (socket says 172.21.15.7:39968) 2024-06-10T01:18:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.148+0000 7f5b437fe640 1 -- 172.21.15.7:0/4276886125 learned_addr learned my addr 172.21.15.7:0/4276886125 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.148+0000 7f5b4c822740 1 -- 172.21.15.7:0/4276886125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5b441a6940 con 0x558328974a00 2024-06-10T01:18:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.148+0000 7f5b42ffd640 1 --2- 172.21.15.7:0/4276886125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558328974a00 0x558328981e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.148+0000 7f5b4c822740 1 -- 172.21.15.7:0/4276886125 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b441a7db0 con 0x558328979b70 2024-06-10T01:18:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.148+0000 7f5b4c822740 1 -- 172.21.15.7:0/4276886125 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5b441a7010 con 0x558328974010 2024-06-10T01:18:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.148+0000 7f5b43fff640 1 --2- 172.21.15.7:0/4276886125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558328979b70 0x558328982390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.148+0000 7f5b4c822740 10 client.0 _ll_register_callbacks cb 0x7f5b34009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:18:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.148+0000 7f5b42ffd640 1 -- 172.21.15.7:0/4276886125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558328974010 msgr2=0x5583289853b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.148+0000 7f5b42ffd640 1 --2- 172.21.15.7:0/4276886125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558328974010 0x5583289853b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.149+0000 7f5b42ffd640 1 -- 172.21.15.7:0/4276886125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558328979b70 msgr2=0x558328982390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.149+0000 7f5b42ffd640 1 --2- 172.21.15.7:0/4276886125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558328979b70 0x558328982390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.149+0000 7f5b42ffd640 1 -- 172.21.15.7:0/4276886125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583289859e0 con 0x558328974a00 2024-06-10T01:18:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.149+0000 7f5b437fe640 1 --2- 172.21.15.7:0/4276886125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558328974010 0x5583289853b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:18:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.149+0000 7f5b42ffd640 1 --2- 172.21.15.7:0/4276886125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558328974a00 0x558328981e50 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f5b38011d10 tx=0x7f5b38009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1428f96ecef6db server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.149+0000 7f5b22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:18:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.149+0000 7f5b22ffd640 1 -- 172.21.15.7:0/4276886125 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b38023040 con 0x558328974a00 2024-06-10T01:18:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.149+0000 7f5b4c822740 1 -- 172.21.15.7:0/4276886125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5583288eaf10 con 0x558328974a00 2024-06-10T01:18:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.149+0000 7f5b22ffd640 1 -- 172.21.15.7:0/4276886125 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f5b380035f0 con 0x558328974a00 2024-06-10T01:18:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.149+0000 7f5b22ffd640 1 -- 172.21.15.7:0/4276886125 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5b380037f0 con 0x558328974a00 2024-06-10T01:18:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.149+0000 7f5b4c822740 10 client.18615 fetch_fsmap learned FSMap version 40 2024-06-10T01:18:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.149+0000 7f5b4c822740 1 -- 172.21.15.7:0/4276886125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5583289f41b0 con 0x558328974a00 2024-06-10T01:18:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.149+0000 7f5b22ffd640 1 -- 172.21.15.7:0/4276886125 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5583288eaf10 con 0x558328974a00 2024-06-10T01:18:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.150+0000 7f5b22ffd640 1 -- 172.21.15.7:0/4276886125 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f5b38016b50 con 0x558328974a00 2024-06-10T01:18:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.150+0000 7f5b4c822740 10 client.18615 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:18:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.150+0000 7f5b4c822740 10 client.18615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:18:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.150+0000 7f5b4c822740 20 client.18615 populate_metadata read hostname 'smithi007' 2024-06-10T01:18:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.150+0000 7f5b4c822740 1 --2- 172.21.15.7:0/4276886125 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5583289f4960 0x5583289f6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.150+0000 7f5b4c822740 4 client.18615 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:18:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.150+0000 7f5b4c822740 1 -- 172.21.15.7:0/4276886125 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5583288def70 con 0x5583289f4960 2024-06-10T01:18:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.150+0000 7f5b437fe640 1 --2- 172.21.15.7:0/4276886125 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5583289f4960 0x5583289f6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.150+0000 7f5b22ffd640 10 client.18615 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:24.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.150+0000 7f5b437fe640 1 --2- 172.21.15.7:0/4276886125 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5583289f4960 0x5583289f6d50 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5945672151d7565a server_cookie=d13195d4580473ff in_seq=0 out_seq=0 2024-06-10T01:18:24.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.156+0000 7f8c3932bb80 1 Processor -- start 2024-06-10T01:18:24.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.156+0000 7f8c3932bb80 1 -- start start 2024-06-10T01:18:24.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.156+0000 7f8c3932bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ae6fc5fa0 0x564ae6fc5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.156+0000 7f8c3932bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ae6fcb6c0 0x564ae6fccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.157+0000 7f8c3932bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ae6fcbfe0 0x564ae6fba590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.157+0000 7f8c3932bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8c281b2de0 con 0x564ae6fc5fa0 2024-06-10T01:18:24.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.157+0000 7f8c3932bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8c281b0670 con 0x564ae6fcbfe0 2024-06-10T01:18:24.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.157+0000 7f8c3932bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8c281ae4c0 con 0x564ae6fcb6c0 2024-06-10T01:18:24.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.157+0000 7f8c2c844700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ae6fc5fa0 0x564ae6fc5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.157+0000 7f8c2d045700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ae6fcbfe0 0x564ae6fba590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.157+0000 7f8c2d045700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ae6fcbfe0 0x564ae6fba590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:52970/0 (socket says 172.21.15.7:52970) 2024-06-10T01:18:24.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.157+0000 7f8c2c844700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ae6fc5fa0 0x564ae6fc5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54364/0 (socket says 172.21.15.7:54364) 2024-06-10T01:18:24.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.157+0000 7f8c27fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ae6fcb6c0 0x564ae6fccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.157+0000 7f8c2d045700 1 -- 172.21.15.7:0/555070383 learned_addr learned my addr 172.21.15.7:0/555070383 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:24.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.157+0000 7f8c2d045700 1 -- 172.21.15.7:0/555070383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ae6fcb6c0 msgr2=0x564ae6fccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:24.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.157+0000 7f8c2d045700 1 --2- 172.21.15.7:0/555070383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ae6fcb6c0 0x564ae6fccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.157+0000 7f8c2d045700 1 -- 172.21.15.7:0/555070383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ae6fc5fa0 msgr2=0x564ae6fc5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:24.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.157+0000 7f8c2d045700 1 --2- 172.21.15.7:0/555070383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ae6fc5fa0 0x564ae6fc5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.157+0000 7f8c2d045700 1 -- 172.21.15.7:0/555070383 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ae6fcbe10 con 0x564ae6fcbfe0 2024-06-10T01:18:24.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.157+0000 7f8c2c844700 1 --2- 172.21.15.7:0/555070383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ae6fc5fa0 0x564ae6fc5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:18:24.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.157+0000 7f8c2d045700 1 --2- 172.21.15.7:0/555070383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ae6fcbfe0 0x564ae6fba590 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8c20000f80 tx=0x7f8c200021d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=379fdb166152945c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:24.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.158+0000 7f8c127fc700 1 -- 172.21.15.7:0/555070383 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c20019070 con 0x564ae6fcbfe0 2024-06-10T01:18:24.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.158+0000 7f8c127fc700 1 -- 172.21.15.7:0/555070383 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c20023070 con 0x564ae6fcbfe0 2024-06-10T01:18:24.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.158+0000 7f8c127fc700 1 -- 172.21.15.7:0/555070383 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c20003c10 con 0x564ae6fcbfe0 2024-06-10T01:18:24.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.158+0000 7f8c3932bb80 1 -- 172.21.15.7:0/555070383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ae6fcbfe0 msgr2=0x564ae6fba590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:24.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.158+0000 7f8c3932bb80 1 --2- 172.21.15.7:0/555070383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ae6fcbfe0 0x564ae6fba590 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8c20000f80 tx=0x7f8c200021d0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.159+0000 7f8c3932bb80 1 -- 172.21.15.7:0/555070383 shutdown_connections 2024-06-10T01:18:24.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.159+0000 7f8c3932bb80 1 --2- 172.21.15.7:0/555070383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ae6fcbfe0 0x564ae6fba590 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.159+0000 7f8c3932bb80 1 --2- 172.21.15.7:0/555070383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ae6fc5fa0 0x564ae6fc5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.159+0000 7f8c3932bb80 1 --2- 172.21.15.7:0/555070383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ae6fcb6c0 0x564ae6fccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.159+0000 7f8c3932bb80 1 -- 172.21.15.7:0/555070383 >> 172.21.15.7:0/555070383 conn(0x564ae6ef0d40 msgr2=0x564ae6fb9e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:24.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.159+0000 7f8c3932bb80 1 -- 172.21.15.7:0/555070383 shutdown_connections 2024-06-10T01:18:24.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.159+0000 7f8c3932bb80 1 -- 172.21.15.7:0/555070383 wait complete. 2024-06-10T01:18:24.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.159+0000 7f8c3932bb80 1 -- start start 2024-06-10T01:18:24.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.159+0000 7f8c3932bb80 1 Processor -- start 2024-06-10T01:18:24.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.160+0000 7f8c3932bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ae6fc5fa0 0x564ae6fe3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.160+0000 7f8c3932bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ae6fcb6c0 0x564ae6fe36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.160+0000 7f8c3932bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ae6fcbfe0 0x564ae7051a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.160+0000 7f8c3932bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8c281aeb30 con 0x564ae6fcb6c0 2024-06-10T01:18:24.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.160+0000 7f8c3932bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8c281affa0 con 0x564ae6fc5fa0 2024-06-10T01:18:24.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.160+0000 7f8c3932bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8c281af200 con 0x564ae6fcbfe0 2024-06-10T01:18:24.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.160+0000 7f8c2c844700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ae6fc5fa0 0x564ae6fe3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.160+0000 7f8c2c844700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ae6fc5fa0 0x564ae6fe3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:52978/0 (socket says 172.21.15.7:52978) 2024-06-10T01:18:24.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.160+0000 7f8c2c844700 1 -- 172.21.15.7:0/2246640687 learned_addr learned my addr 172.21.15.7:0/2246640687 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:24.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.160+0000 7f8c27fff700 1 --2- 172.21.15.7:0/2246640687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ae6fcb6c0 0x564ae6fe36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.160+0000 7f8c3932bb80 10 client.0 _ll_register_callbacks cb 0x564ae6eb19a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:18:24.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.160+0000 7f8c2c844700 1 -- 172.21.15.7:0/2246640687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ae6fcbfe0 msgr2=0x564ae7051a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:24.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.160+0000 7f8c2c844700 1 --2- 172.21.15.7:0/2246640687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ae6fcbfe0 0x564ae7051a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.160+0000 7f8c2c844700 1 -- 172.21.15.7:0/2246640687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ae6fcb6c0 msgr2=0x564ae6fe36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:24.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.160+0000 7f8c2c844700 1 --2- 172.21.15.7:0/2246640687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ae6fcb6c0 0x564ae6fe36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.160+0000 7f8c2c844700 1 -- 172.21.15.7:0/2246640687 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ae7051f70 con 0x564ae6fc5fa0 2024-06-10T01:18:24.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.160+0000 7f8c27fff700 1 --2- 172.21.15.7:0/2246640687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ae6fcb6c0 0x564ae6fe36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:18:24.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.160+0000 7f8c2c844700 1 --2- 172.21.15.7:0/2246640687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ae6fc5fa0 0x564ae6fe3190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c098420 tx=0x7f8c1c003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed4dca1666e9b38e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:24.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.160+0000 7f8c07fff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:18:24.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.161+0000 7f8c07fff700 1 -- 172.21.15.7:0/2246640687 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c0a8070 con 0x564ae6fc5fa0 2024-06-10T01:18:24.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.161+0000 7f8c3932bb80 1 -- 172.21.15.7:0/2246640687 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c20023070 con 0x564ae6fc5fa0 2024-06-10T01:18:24.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.161+0000 7f8c07fff700 1 -- 172.21.15.7:0/2246640687 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f8c1c099d50 con 0x564ae6fc5fa0 2024-06-10T01:18:24.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.161+0000 7f8c07fff700 1 -- 172.21.15.7:0/2246640687 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8c1c0ba070 con 0x564ae6fc5fa0 2024-06-10T01:18:24.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.161+0000 7f8c3932bb80 10 client.27935 fetch_fsmap learned FSMap version 40 2024-06-10T01:18:24.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.161+0000 7f8c3932bb80 1 -- 172.21.15.7:0/2246640687 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x564ae7053310 con 0x564ae6fc5fa0 2024-06-10T01:18:24.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.161+0000 7f8c07fff700 1 -- 172.21.15.7:0/2246640687 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c20023070 con 0x564ae6fc5fa0 2024-06-10T01:18:24.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.161+0000 7f8c07fff700 1 -- 172.21.15.7:0/2246640687 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f8c1c0ab030 con 0x564ae6fc5fa0 2024-06-10T01:18:24.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.161+0000 7f8c3932bb80 10 client.27935 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:18:24.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.161+0000 7f8c3932bb80 10 client.27935 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:18:24.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.161+0000 7f8c3932bb80 20 client.27935 populate_metadata read hostname 'smithi007' 2024-06-10T01:18:24.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.161+0000 7f8c3932bb80 1 --2- 172.21.15.7:0/2246640687 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564ae6fdf8e0 0x564ae7055a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:24.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.161+0000 7f8c3932bb80 4 client.27935 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:18:24.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.161+0000 7f8c3932bb80 1 -- 172.21.15.7:0/2246640687 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564ae6f1f310 con 0x564ae6fdf8e0 2024-06-10T01:18:24.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.162+0000 7f8c27fff700 1 --2- 172.21.15.7:0/2246640687 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564ae6fdf8e0 0x564ae7055a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:24.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.162+0000 7f8c27fff700 1 --2- 172.21.15.7:0/2246640687 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564ae6fdf8e0 0x564ae7055a60 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c1b18705d57f6af server_cookie=7e6544c15b76ae54 in_seq=0 out_seq=0 2024-06-10T01:18:24.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.162+0000 7f8c07fff700 10 client.27935 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:24.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.186+0000 7f8c07fff700 1 -- 172.21.15.7:0/2246640687 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564ae6f1f310 con 0x564ae6fdf8e0 2024-06-10T01:18:24.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.186+0000 7f8c07fff700 10 client.27935 handle_command_reply: tid=0 2024-06-10T01:18:24.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.187+0000 7f8c3932bb80 1 client.27935 shutdown 2024-06-10T01:18:24.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.187+0000 7f8c3932bb80 1 -- 172.21.15.7:0/2246640687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ae6fc5fa0 msgr2=0x564ae6fe3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:24.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.187+0000 7f8c3932bb80 1 --2- 172.21.15.7:0/2246640687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ae6fc5fa0 0x564ae6fe3190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c098420 tx=0x7f8c1c003680 comp rx=0 tx=0).stop 2024-06-10T01:18:24.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.187+0000 7f8c3932bb80 1 -- 172.21.15.7:0/2246640687 shutdown_connections 2024-06-10T01:18:24.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.187+0000 7f8c3932bb80 1 --2- 172.21.15.7:0/2246640687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ae6fcbfe0 0x564ae7051a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.187+0000 7f8c3932bb80 1 --2- 172.21.15.7:0/2246640687 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564ae6fdf8e0 0x564ae7055a60 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.187+0000 7f8c3932bb80 1 --2- 172.21.15.7:0/2246640687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ae6fc5fa0 0x564ae6fe3190 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.187+0000 7f8c07fff700 0 client.27935 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:24.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.187+0000 7f8c3932bb80 1 --2- 172.21.15.7:0/2246640687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ae6fcb6c0 0x564ae6fe36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.187+0000 7f8c3932bb80 1 -- 172.21.15.7:0/2246640687 >> 172.21.15.7:0/2246640687 conn(0x564ae6ef0d40 msgr2=0x564ae6fca1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:24.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.191+0000 7f8c3932bb80 1 -- 172.21.15.7:0/2246640687 shutdown_connections 2024-06-10T01:18:24.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.191+0000 7f8c3932bb80 1 -- 172.21.15.7:0/2246640687 wait complete. 2024-06-10T01:18:24.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.191+0000 7f8c3932bb80 20 client.27935 trim_cache size 0 max 16384 2024-06-10T01:18:24.187 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:18:24.187 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-10T01:18:24.187 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "7c8ede2f-9fdc-46a8-af60-d019640adb5c", 2024-06-10T01:18:24.187 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-10T01:18:24.187 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:18:24.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.193+0000 7f8c2eaa8700 1 -- 172.21.15.7:0/465978552 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8c180a2d70 msgr2=0x7f8c180a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:24.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.193+0000 7f8c2eaa8700 1 --2- 172.21.15.7:0/465978552 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8c180a2d70 0x7f8c180a5230 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8c140098c0 tx=0x7f8c14019040 comp rx=0 tx=0).stop 2024-06-10T01:18:24.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.193+0000 7f8c2eaa8700 1 -- 172.21.15.7:0/465978552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c28101380 msgr2=0x7f8c28103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:24.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.193+0000 7f8c2eaa8700 1 --2- 172.21.15.7:0/465978552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c28101380 0x7f8c28103060 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00ec10 tx=0x7f8c1c00ef20 comp rx=0 tx=0).stop 2024-06-10T01:18:24.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.193+0000 7f8c2eaa8700 1 -- 172.21.15.7:0/465978552 shutdown_connections 2024-06-10T01:18:24.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.193+0000 7f8c2eaa8700 1 --2- 172.21.15.7:0/465978552 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8c180a2d70 0x7f8c180a5230 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.193+0000 7f8c2eaa8700 1 --2- 172.21.15.7:0/465978552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c281073f0 0x7f8c28103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.193+0000 7f8c2eaa8700 1 --2- 172.21.15.7:0/465978552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c28101380 0x7f8c28103060 secure :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00ec10 tx=0x7f8c1c00ef20 comp rx=0 tx=0).stop 2024-06-10T01:18:24.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.193+0000 7f8c2eaa8700 1 --2- 172.21.15.7:0/465978552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c28101d40 0x7f8c281035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.193+0000 7f8c2eaa8700 1 -- 172.21.15.7:0/465978552 >> 172.21.15.7:0/465978552 conn(0x7f8c28078300 msgr2=0x7f8c281006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:24.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.194+0000 7f8c2eaa8700 1 -- 172.21.15.7:0/465978552 shutdown_connections 2024-06-10T01:18:24.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:24.194+0000 7f8c2eaa8700 1 -- 172.21.15.7:0/465978552 wait complete. 2024-06-10T01:18:24.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.255+0000 7f5b22ffd640 1 -- 172.21.15.7:0/4276886125 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5583288def70 con 0x5583289f4960 2024-06-10T01:18:24.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.255+0000 7f5b22ffd640 10 client.18615 handle_command_reply: tid=0 2024-06-10T01:18:24.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.255+0000 7f5b4c822740 1 client.18615 shutdown 2024-06-10T01:18:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.255+0000 7f5b4c822740 1 -- 172.21.15.7:0/4276886125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558328974a00 msgr2=0x558328981e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.255+0000 7f5b4c822740 1 --2- 172.21.15.7:0/4276886125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558328974a00 0x558328981e50 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f5b38011d10 tx=0x7f5b38009350 comp rx=0 tx=0).stop 2024-06-10T01:18:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.255+0000 7f5b4c822740 1 -- 172.21.15.7:0/4276886125 shutdown_connections 2024-06-10T01:18:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.255+0000 7f5b4c822740 1 --2- 172.21.15.7:0/4276886125 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5583289f4960 0x5583289f6d50 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.255+0000 7f5b22ffd640 0 client.18615 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.255+0000 7f5b4c822740 1 --2- 172.21.15.7:0/4276886125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558328979b70 0x558328982390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.255+0000 7f5b4c822740 1 --2- 172.21.15.7:0/4276886125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558328974a00 0x558328981e50 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.255+0000 7f5b4c822740 1 --2- 172.21.15.7:0/4276886125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558328974010 0x5583289853b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.255+0000 7f5b4c822740 1 -- 172.21.15.7:0/4276886125 >> 172.21.15.7:0/4276886125 conn(0x55832896f680 msgr2=0x55832897d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.256+0000 7f5b4c822740 1 -- 172.21.15.7:0/4276886125 shutdown_connections 2024-06-10T01:18:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.256+0000 7f5b4c822740 1 -- 172.21.15.7:0/4276886125 wait complete. 2024-06-10T01:18:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.256+0000 7f5b4c822740 20 client.18615 trim_cache size 0 max 16384 2024-06-10T01:18:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:18:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:18:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:18:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.256+0000 7f5b49b1d640 1 -- 172.21.15.7:0/661017461 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5b2c0a2b80 msgr2=0x7f5b2c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.256+0000 7f5b49b1d640 1 --2- 172.21.15.7:0/661017461 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5b2c0a2b80 0x7f5b2c0a5040 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c006010 tx=0x7f5b3c004a40 comp rx=0 tx=0).stop 2024-06-10T01:18:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.256+0000 7f5b49b1d640 1 -- 172.21.15.7:0/661017461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b441024a0 msgr2=0x7f5b44197d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.256+0000 7f5b49b1d640 1 --2- 172.21.15.7:0/661017461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b441024a0 0x7f5b44197d20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400e9f0 tx=0x7f5b3400edf0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.257+0000 7f5b49b1d640 1 -- 172.21.15.7:0/661017461 shutdown_connections 2024-06-10T01:18:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.257+0000 7f5b49b1d640 1 --2- 172.21.15.7:0/661017461 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5b2c0a2b80 0x7f5b2c0a5040 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.257+0000 7f5b49b1d640 1 --2- 172.21.15.7:0/661017461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b44107700 0x7f5b4419f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.257+0000 7f5b49b1d640 1 --2- 172.21.15.7:0/661017461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b44104dd0 0x7f5b44198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.257+0000 7f5b49b1d640 1 --2- 172.21.15.7:0/661017461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b441024a0 0x7f5b44197d20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.257+0000 7f5b49b1d640 1 -- 172.21.15.7:0/661017461 >> 172.21.15.7:0/661017461 conn(0x7f5b440fbeb0 msgr2=0x7f5b44102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.257+0000 7f5b49b1d640 1 -- 172.21.15.7:0/661017461 shutdown_connections 2024-06-10T01:18:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:24.257+0000 7f5b49b1d640 1 -- 172.21.15.7:0/661017461 wait complete. 2024-06-10T01:18:24.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:24 smithi175 ceph-mon[36636]: scrub queued for path: / 2024-06-10T01:18:24.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:24 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:18:24.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:24 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:18:24.755 INFO:tasks.workunit.client.0.smithi007.stdout: 1 896547 7.17 MB/sec execute 545 sec latency 70.342 ms 2024-06-10T01:18:24.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:24 smithi007 ceph-mon[32398]: scrub queued for path: / 2024-06-10T01:18:24.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:24 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:18:24.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:24 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:18:24.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:24 smithi078 ceph-mon[36683]: scrub queued for path: / 2024-06-10T01:18:24.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:24 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:18:24.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:24 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:18:25.012 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:18:25.283 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:18:25.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:25 smithi175 ceph-mon[36636]: pgmap v804: 129 pgs: 129 active+clean; 144 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.8 MiB/s wr, 208 op/s 2024-06-10T01:18:25.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:25 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:18:25.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:25 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:18:25.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:25 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:18:25.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:25 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:18:25.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:25 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:18:25.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:25 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:18:25.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:25 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:18:25.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:25 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:18:25.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:25 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:18:25.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:25 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:18:25.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:25 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:18:25.704 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:25 smithi007 ceph-mon[32398]: pgmap v804: 129 pgs: 129 active+clean; 144 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.8 MiB/s wr, 208 op/s 2024-06-10T01:18:25.704 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:25 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:18:25.704 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:25 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:18:25.704 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:25 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:18:25.704 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:25 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:18:25.704 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:25 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:18:25.704 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:25 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:18:25.704 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:25 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:18:25.704 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:25 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:18:25.704 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:25 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:18:25.705 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:25 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:18:25.705 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:25 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:18:25.729 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:25 smithi078 ceph-mon[36683]: pgmap v804: 129 pgs: 129 active+clean; 144 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.8 MiB/s wr, 208 op/s 2024-06-10T01:18:25.729 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:25 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:18:25.729 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:25 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:18:25.729 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:25 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:18:25.729 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:25 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:18:25.729 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:25 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:18:25.729 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:25 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:18:25.729 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:25 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:18:25.729 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:25 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:18:25.730 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:25 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:18:25.730 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:25 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:18:25.730 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:25 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:18:25.755 INFO:tasks.workunit.client.0.smithi007.stdout: 1 898086 7.18 MB/sec execute 546 sec latency 175.784 ms 2024-06-10T01:18:26.755 INFO:tasks.workunit.client.0.smithi007.stdout: 1 898566 7.17 MB/sec execute 547 sec latency 686.612 ms 2024-06-10T01:18:27.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.365+0000 7f14d9c17700 1 -- 172.21.15.7:0/2481377836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d410a660 msgr2=0x7f14d410aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:27.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.365+0000 7f14d9c17700 1 --2- 172.21.15.7:0/2481377836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d410a660 0x7f14d410aa40 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009990 tx=0x7f14c4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.365+0000 7f14d9c17700 1 -- 172.21.15.7:0/2481377836 shutdown_connections 2024-06-10T01:18:27.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.365+0000 7f14d9c17700 1 --2- 172.21.15.7:0/2481377836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14d4105020 0x7f14d4111530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.365+0000 7f14d9c17700 1 --2- 172.21.15.7:0/2481377836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d410a660 0x7f14d410aa40 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.365+0000 7f14d9c17700 1 --2- 172.21.15.7:0/2481377836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14d4104660 0x7f14d4104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.365+0000 7f14d9c17700 1 -- 172.21.15.7:0/2481377836 >> 172.21.15.7:0/2481377836 conn(0x7f14d4100190 msgr2=0x7f14d41025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:27.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.365+0000 7f14d9c17700 1 -- 172.21.15.7:0/2481377836 shutdown_connections 2024-06-10T01:18:27.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.365+0000 7f14d9c17700 1 -- 172.21.15.7:0/2481377836 wait complete. 2024-06-10T01:18:27.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.366+0000 7f14d9c17700 1 Processor -- start 2024-06-10T01:18:27.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.366+0000 7f14d9c17700 1 -- start start 2024-06-10T01:18:27.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.366+0000 7f14d9c17700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d4104660 0x7f14d419ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:27.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.366+0000 7f14d9c17700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14d4105020 0x7f14d419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:27.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.366+0000 7f14d9c17700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14d410a660 0x7f14d419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:27.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.366+0000 7f14d9c17700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f14d4114990 con 0x7f14d4104660 2024-06-10T01:18:27.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.366+0000 7f14d9c17700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f14d4114810 con 0x7f14d4105020 2024-06-10T01:18:27.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.366+0000 7f14d9c17700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f14d4114b10 con 0x7f14d410a660 2024-06-10T01:18:27.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.366+0000 7f14d37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d4104660 0x7f14d419ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:27.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.367+0000 7f14d37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d4104660 0x7f14d419ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54400/0 (socket says 172.21.15.7:54400) 2024-06-10T01:18:27.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.367+0000 7f14d3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14d410a660 0x7f14d419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:27.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.367+0000 7f14d2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14d4105020 0x7f14d419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:27.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.367+0000 7f14d37fe700 1 -- 172.21.15.7:0/3752048385 learned_addr learned my addr 172.21.15.7:0/3752048385 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:27.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.367+0000 7f14d37fe700 1 -- 172.21.15.7:0/3752048385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14d410a660 msgr2=0x7f14d419f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:27.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.367+0000 7f14d37fe700 1 --2- 172.21.15.7:0/3752048385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14d410a660 0x7f14d419f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.367+0000 7f14d37fe700 1 -- 172.21.15.7:0/3752048385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14d4105020 msgr2=0x7f14d419b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:27.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.367+0000 7f14d37fe700 1 --2- 172.21.15.7:0/3752048385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14d4105020 0x7f14d419b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.367+0000 7f14d37fe700 1 -- 172.21.15.7:0/3752048385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d419fe30 con 0x7f14d4104660 2024-06-10T01:18:27.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.367+0000 7f14d3fff700 1 --2- 172.21.15.7:0/3752048385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14d410a660 0x7f14d419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:18:27.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.367+0000 7f14d2ffd700 1 --2- 172.21.15.7:0/3752048385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14d4105020 0x7f14d419b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:18:27.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.367+0000 7f14d37fe700 1 --2- 172.21.15.7:0/3752048385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d4104660 0x7f14d419ae20 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f14c4000c00 tx=0x7f14c400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:27.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.367+0000 7f14d0ff9700 1 -- 172.21.15.7:0/3752048385 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c4027070 con 0x7f14d4104660 2024-06-10T01:18:27.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.367+0000 7f14d9c17700 1 -- 172.21.15.7:0/3752048385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d41a00c0 con 0x7f14d4104660 2024-06-10T01:18:27.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.367+0000 7f14d0ff9700 1 -- 172.21.15.7:0/3752048385 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c4003d70 con 0x7f14d4104660 2024-06-10T01:18:27.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.367+0000 7f14d0ff9700 1 -- 172.21.15.7:0/3752048385 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c400f460 con 0x7f14d4104660 2024-06-10T01:18:27.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.367+0000 7f14d9c17700 1 -- 172.21.15.7:0/3752048385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d41a79a0 con 0x7f14d4104660 2024-06-10T01:18:27.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.370+0000 7f14d0ff9700 1 -- 172.21.15.7:0/3752048385 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f14c402a070 con 0x7f14d4104660 2024-06-10T01:18:27.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.371+0000 7f14d0ff9700 1 --2- 172.21.15.7:0/3752048385 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f14bc0a2c20 0x7f14bc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:27.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.371+0000 7f14d0ff9700 1 -- 172.21.15.7:0/3752048385 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f14c40d0150 con 0x7f14d4104660 2024-06-10T01:18:27.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.371+0000 7f14d2ffd700 1 --2- 172.21.15.7:0/3752048385 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f14bc0a2c20 0x7f14bc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:27.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.371+0000 7f14d2ffd700 1 --2- 172.21.15.7:0/3752048385 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f14bc0a2c20 0x7f14bc0a50e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f14d41027d0 tx=0x7f14c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:27.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.371+0000 7f14e449ab80 1 Processor -- start 2024-06-10T01:18:27.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.371+0000 7f14e449ab80 1 -- start start 2024-06-10T01:18:27.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.372+0000 7f14e449ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ba937cf60 0x564ba937d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:27.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.372+0000 7f14e449ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ba937d880 0x564ba9377260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:27.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.372+0000 7f14e449ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ba93777a0 0x564ba9377bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:27.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.372+0000 7f14e449ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564ba9269f60 con 0x564ba93777a0 2024-06-10T01:18:27.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.372+0000 7f14e449ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564ba92c8850 con 0x564ba937d880 2024-06-10T01:18:27.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.372+0000 7f14e449ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564ba9230460 con 0x564ba937cf60 2024-06-10T01:18:27.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.372+0000 7f14d37fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ba937cf60 0x564ba937d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:27.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.372+0000 7f14d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ba93777a0 0x564ba9377bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:27.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.372+0000 7f14d2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ba937d880 0x564ba9377260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:27.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.372+0000 7f14d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ba93777a0 0x564ba9377bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54404/0 (socket says 172.21.15.7:54404) 2024-06-10T01:18:27.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.372+0000 7f14d3fff700 1 -- 172.21.15.7:0/3772570578 learned_addr learned my addr 172.21.15.7:0/3772570578 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:27.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.372+0000 7f14d2ffd700 1 -- 172.21.15.7:0/3772570578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ba937cf60 msgr2=0x564ba937d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:27.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.372+0000 7f14d2ffd700 1 --2- 172.21.15.7:0/3772570578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ba937cf60 0x564ba937d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.372+0000 7f14d2ffd700 1 -- 172.21.15.7:0/3772570578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ba93777a0 msgr2=0x564ba9377bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:27.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.372+0000 7f14d2ffd700 1 --2- 172.21.15.7:0/3772570578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ba93777a0 0x564ba9377bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.372+0000 7f14d2ffd700 1 -- 172.21.15.7:0/3772570578 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ba92178d0 con 0x564ba937d880 2024-06-10T01:18:27.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.372+0000 7f14d37fe700 1 --2- 172.21.15.7:0/3772570578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ba937cf60 0x564ba937d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:18:27.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.372+0000 7f14d3fff700 1 --2- 172.21.15.7:0/3772570578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ba93777a0 0x564ba9377bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:18:27.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.373+0000 7f14d2ffd700 1 --2- 172.21.15.7:0/3772570578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ba937d880 0x564ba9377260 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f14c80042f0 tx=0x7f14c8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=859e27ba0e0dfe5e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:27.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.373+0000 7f14b9ffb700 1 -- 172.21.15.7:0/3772570578 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8003d10 con 0x564ba937d880 2024-06-10T01:18:27.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.373+0000 7f14b9ffb700 1 -- 172.21.15.7:0/3772570578 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c80268e0 con 0x564ba937d880 2024-06-10T01:18:27.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.373+0000 7f14b9ffb700 1 -- 172.21.15.7:0/3772570578 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c801e860 con 0x564ba937d880 2024-06-10T01:18:27.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.374+0000 7f14e449ab80 1 -- 172.21.15.7:0/3772570578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ba937d880 msgr2=0x564ba9377260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:27.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.374+0000 7f14e449ab80 1 --2- 172.21.15.7:0/3772570578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ba937d880 0x564ba9377260 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f14c80042f0 tx=0x7f14c8014000 comp rx=0 tx=0).stop 2024-06-10T01:18:27.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.374+0000 7f14e449ab80 1 -- 172.21.15.7:0/3772570578 shutdown_connections 2024-06-10T01:18:27.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.374+0000 7f14e449ab80 1 --2- 172.21.15.7:0/3772570578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ba93777a0 0x564ba9377bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.374+0000 7f14e449ab80 1 --2- 172.21.15.7:0/3772570578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ba937cf60 0x564ba937d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.374+0000 7f14e449ab80 1 --2- 172.21.15.7:0/3772570578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ba937d880 0x564ba9377260 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.374+0000 7f14e449ab80 1 -- 172.21.15.7:0/3772570578 >> 172.21.15.7:0/3772570578 conn(0x564ba92a4080 msgr2=0x564ba9372220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:27.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.374+0000 7f14e449ab80 1 -- 172.21.15.7:0/3772570578 shutdown_connections 2024-06-10T01:18:27.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.374+0000 7f14e449ab80 1 -- 172.21.15.7:0/3772570578 wait complete. 2024-06-10T01:18:27.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.374+0000 7f14e449ab80 1 -- start start 2024-06-10T01:18:27.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.375+0000 7f14e449ab80 1 Processor -- start 2024-06-10T01:18:27.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.375+0000 7f14e449ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ba93777a0 0x564ba9383670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:27.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.375+0000 7f14e449ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ba937cf60 0x564ba9380970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:27.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.375+0000 7f14e449ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ba937d880 0x564ba9380eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:27.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.375+0000 7f14e449ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564ba9268fc0 con 0x564ba937d880 2024-06-10T01:18:27.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.375+0000 7f14d2ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ba937cf60 0x564ba9380970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:27.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.375+0000 7f14e449ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564ba92c4810 con 0x564ba93777a0 2024-06-10T01:18:27.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.375+0000 7f14e449ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564ba9384260 con 0x564ba937cf60 2024-06-10T01:18:27.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.375+0000 7f14d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ba937d880 0x564ba9380eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:27.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.375+0000 7f14d2ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ba937cf60 0x564ba9380970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:40024/0 (socket says 172.21.15.7:40024) 2024-06-10T01:18:27.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.375+0000 7f14d37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ba93777a0 0x564ba9383670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:27.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.375+0000 7f14d2ffd700 1 -- 172.21.15.7:0/1774630593 learned_addr learned my addr 172.21.15.7:0/1774630593 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:27.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.375+0000 7f14e449ab80 10 client.0 _ll_register_callbacks cb 0x564ba925e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:18:27.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.375+0000 7f14d2ffd700 1 -- 172.21.15.7:0/1774630593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ba93777a0 msgr2=0x564ba9383670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:27.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.375+0000 7f14d2ffd700 1 --2- 172.21.15.7:0/1774630593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ba93777a0 0x564ba9383670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.375+0000 7f14d2ffd700 1 -- 172.21.15.7:0/1774630593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ba937d880 msgr2=0x564ba9380eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:27.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.375+0000 7f14d2ffd700 1 --2- 172.21.15.7:0/1774630593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ba937d880 0x564ba9380eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.375+0000 7f14d2ffd700 1 -- 172.21.15.7:0/1774630593 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ba928e750 con 0x564ba937cf60 2024-06-10T01:18:27.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.375+0000 7f14d37fe700 1 --2- 172.21.15.7:0/1774630593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ba93777a0 0x564ba9383670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:18:27.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.376+0000 7f14d2ffd700 1 --2- 172.21.15.7:0/1774630593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ba937cf60 0x564ba9380970 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009790 tx=0x7f14c8003dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97192b850e8e09fa server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:27.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.376+0000 7f14b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:18:27.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.376+0000 7f14b2ffd700 1 -- 172.21.15.7:0/1774630593 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8026c90 con 0x564ba937cf60 2024-06-10T01:18:27.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.376+0000 7f14b2ffd700 1 -- 172.21.15.7:0/1774630593 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f14c8005070 con 0x564ba937cf60 2024-06-10T01:18:27.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.376+0000 7f14e449ab80 1 -- 172.21.15.7:0/1774630593 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14c80268e0 con 0x564ba937cf60 2024-06-10T01:18:27.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.376+0000 7f14b2ffd700 1 -- 172.21.15.7:0/1774630593 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f14c803a050 con 0x564ba937cf60 2024-06-10T01:18:27.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.376+0000 7f14e449ab80 10 client.27964 fetch_fsmap learned FSMap version 40 2024-06-10T01:18:27.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.376+0000 7f14e449ab80 1 -- 172.21.15.7:0/1774630593 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x564ba938ea00 con 0x564ba937cf60 2024-06-10T01:18:27.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.376+0000 7f14b2ffd700 1 -- 172.21.15.7:0/1774630593 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c80268e0 con 0x564ba937cf60 2024-06-10T01:18:27.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.377+0000 7f14b2ffd700 1 -- 172.21.15.7:0/1774630593 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f14c802f430 con 0x564ba937cf60 2024-06-10T01:18:27.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.377+0000 7f14e449ab80 10 client.27964 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:18:27.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.377+0000 7f14e449ab80 10 client.27964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:18:27.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.377+0000 7f14e449ab80 20 client.27964 populate_metadata read hostname 'smithi007' 2024-06-10T01:18:27.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.377+0000 7f14e449ab80 1 --2- 172.21.15.7:0/1774630593 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564ba9390f80 0x564ba9393370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:27.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.377+0000 7f14e449ab80 4 client.27964 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:18:27.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.377+0000 7f14e449ab80 1 -- 172.21.15.7:0/1774630593 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ba92d5d50 con 0x564ba9390f80 2024-06-10T01:18:27.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.377+0000 7f14d37fe700 1 --2- 172.21.15.7:0/1774630593 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564ba9390f80 0x564ba9393370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:27.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.377+0000 7f14d37fe700 1 --2- 172.21.15.7:0/1774630593 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564ba9390f80 0x564ba9393370 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=872466e7b1308b95 server_cookie=b042071e3cbd2aa8 in_seq=0 out_seq=0 2024-06-10T01:18:27.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.377+0000 7f14b2ffd700 10 client.27964 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:27.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.378+0000 7f14b2ffd700 1 -- 172.21.15.7:0/1774630593 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564ba92d5d50 con 0x564ba9390f80 2024-06-10T01:18:27.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.378+0000 7f14b2ffd700 10 client.27964 handle_command_reply: tid=0 2024-06-10T01:18:27.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.378+0000 7f14e449ab80 1 client.27964 shutdown 2024-06-10T01:18:27.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.378+0000 7f14e449ab80 1 -- 172.21.15.7:0/1774630593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ba937cf60 msgr2=0x564ba9380970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:27.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.378+0000 7f14e449ab80 1 --2- 172.21.15.7:0/1774630593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ba937cf60 0x564ba9380970 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009790 tx=0x7f14c8003dc0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.378+0000 7f14e449ab80 1 -- 172.21.15.7:0/1774630593 shutdown_connections 2024-06-10T01:18:27.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.378+0000 7f14e449ab80 1 --2- 172.21.15.7:0/1774630593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ba937d880 0x564ba9380eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.378+0000 7f14e449ab80 1 --2- 172.21.15.7:0/1774630593 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564ba9390f80 0x564ba9393370 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.378+0000 7f14e449ab80 1 --2- 172.21.15.7:0/1774630593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ba93777a0 0x564ba9383670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.378+0000 7f14b2ffd700 0 client.27964 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:27.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.378+0000 7f14e449ab80 1 --2- 172.21.15.7:0/1774630593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ba937cf60 0x564ba9380970 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.378+0000 7f14e449ab80 1 -- 172.21.15.7:0/1774630593 >> 172.21.15.7:0/1774630593 conn(0x564ba92a4080 msgr2=0x564ba9371700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:27.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.378+0000 7f14e449ab80 1 -- 172.21.15.7:0/1774630593 shutdown_connections 2024-06-10T01:18:27.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.378+0000 7f14e449ab80 1 -- 172.21.15.7:0/1774630593 wait complete. 2024-06-10T01:18:27.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.379+0000 7f14e449ab80 20 client.27964 trim_cache size 0 max 16384 2024-06-10T01:18:27.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.399+0000 7f14e449ab80 1 Processor -- start 2024-06-10T01:18:27.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.399+0000 7f14e449ab80 1 -- start start 2024-06-10T01:18:27.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.400+0000 7f14e449ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ba93777a0 0x564ba937e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:27.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.400+0000 7f14e449ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ba937cf60 0x564ba9380fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:27.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.400+0000 7f14e449ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ba937d880 0x564ba937f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:27.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.400+0000 7f14e449ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f14d41b2cf0 con 0x564ba937d880 2024-06-10T01:18:27.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.400+0000 7f14e449ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f14d41b0580 con 0x564ba93777a0 2024-06-10T01:18:27.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.400+0000 7f14e449ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f14d41ae3d0 con 0x564ba937cf60 2024-06-10T01:18:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.400+0000 7f14d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ba937d880 0x564ba937f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.400+0000 7f14d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ba937d880 0x564ba937f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54424/0 (socket says 172.21.15.7:54424) 2024-06-10T01:18:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.400+0000 7f14d3fff700 1 -- 172.21.15.7:0/1003510409 learned_addr learned my addr 172.21.15.7:0/1003510409 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.400+0000 7f14d37fe700 1 --2- 172.21.15.7:0/1003510409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ba93777a0 0x564ba937e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.400+0000 7f14d2ffd700 1 --2- 172.21.15.7:0/1003510409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ba937cf60 0x564ba9380fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.400+0000 7f14d3fff700 1 -- 172.21.15.7:0/1003510409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ba937cf60 msgr2=0x564ba9380fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.400+0000 7f14d3fff700 1 --2- 172.21.15.7:0/1003510409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ba937cf60 0x564ba9380fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.400+0000 7f14d3fff700 1 -- 172.21.15.7:0/1003510409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ba93777a0 msgr2=0x564ba937e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.400+0000 7f14d3fff700 1 --2- 172.21.15.7:0/1003510409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ba93777a0 0x564ba937e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.400+0000 7f14d3fff700 1 -- 172.21.15.7:0/1003510409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ba9377260 con 0x564ba937d880 2024-06-10T01:18:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.400+0000 7f14d3fff700 1 --2- 172.21.15.7:0/1003510409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ba937d880 0x564ba937f970 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00e990 tx=0x7f14cc009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae03bd1836fe977 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.400+0000 7f14b97fa700 1 -- 172.21.15.7:0/1003510409 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc017810 con 0x564ba937d880 2024-06-10T01:18:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.401+0000 7f14b97fa700 1 -- 172.21.15.7:0/1003510409 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14cc0093a0 con 0x564ba937d880 2024-06-10T01:18:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.401+0000 7f14b97fa700 1 -- 172.21.15.7:0/1003510409 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc01f3e0 con 0x564ba937d880 2024-06-10T01:18:27.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.401+0000 7f14e449ab80 1 -- 172.21.15.7:0/1003510409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ba937d880 msgr2=0x564ba937f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:27.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.401+0000 7f14e449ab80 1 --2- 172.21.15.7:0/1003510409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ba937d880 0x564ba937f970 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00e990 tx=0x7f14cc009920 comp rx=0 tx=0).stop 2024-06-10T01:18:27.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.401+0000 7f14e449ab80 1 -- 172.21.15.7:0/1003510409 shutdown_connections 2024-06-10T01:18:27.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.401+0000 7f14e449ab80 1 --2- 172.21.15.7:0/1003510409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ba937d880 0x564ba937f970 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.401+0000 7f14e449ab80 1 --2- 172.21.15.7:0/1003510409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ba93777a0 0x564ba937e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.401+0000 7f14e449ab80 1 --2- 172.21.15.7:0/1003510409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ba937cf60 0x564ba9380fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.401+0000 7f14e449ab80 1 -- 172.21.15.7:0/1003510409 >> 172.21.15.7:0/1003510409 conn(0x564ba92a4080 msgr2=0x564ba937f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:27.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.401+0000 7f14e449ab80 1 -- 172.21.15.7:0/1003510409 shutdown_connections 2024-06-10T01:18:27.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.401+0000 7f14e449ab80 1 -- 172.21.15.7:0/1003510409 wait complete. 2024-06-10T01:18:27.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.402+0000 7f14e449ab80 1 -- start start 2024-06-10T01:18:27.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.402+0000 7f14e449ab80 1 Processor -- start 2024-06-10T01:18:27.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.402+0000 7f14e449ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ba93777a0 0x564ba9389dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:27.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.402+0000 7f14e449ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ba937cf60 0x564ba9398540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:27.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.402+0000 7f14e449ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ba937d880 0x564ba9398c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:27.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.402+0000 7f14e449ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f14d41aea40 con 0x564ba937cf60 2024-06-10T01:18:27.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.402+0000 7f14e449ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f14d41afeb0 con 0x564ba937d880 2024-06-10T01:18:27.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.402+0000 7f14e449ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f14d41af110 con 0x564ba93777a0 2024-06-10T01:18:27.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.402+0000 7f14e449ab80 10 client.0 _ll_register_callbacks cb 0x564ba925e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:18:27.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.402+0000 7f14d2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ba937cf60 0x564ba9398540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:27.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.403+0000 7f14d37fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ba93777a0 0x564ba9389dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:27.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.403+0000 7f14d37fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ba93777a0 0x564ba9389dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:40038/0 (socket says 172.21.15.7:40038) 2024-06-10T01:18:27.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.403+0000 7f14d37fe700 1 -- 172.21.15.7:0/2125179163 learned_addr learned my addr 172.21.15.7:0/2125179163 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:27.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.403+0000 7f14d3fff700 1 --2- 172.21.15.7:0/2125179163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ba937d880 0x564ba9398c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:27.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.403+0000 7f14d2ffd700 1 -- 172.21.15.7:0/2125179163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ba93777a0 msgr2=0x564ba9389dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:27.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.403+0000 7f14d2ffd700 1 --2- 172.21.15.7:0/2125179163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ba93777a0 0x564ba9389dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.403+0000 7f14d2ffd700 1 -- 172.21.15.7:0/2125179163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ba937d880 msgr2=0x564ba9398c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:27.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.403+0000 7f14d2ffd700 1 --2- 172.21.15.7:0/2125179163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ba937d880 0x564ba9398c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.403+0000 7f14d2ffd700 1 -- 172.21.15.7:0/2125179163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ba93faa60 con 0x564ba937cf60 2024-06-10T01:18:27.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.403+0000 7f14d37fe700 1 --2- 172.21.15.7:0/2125179163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ba93777a0 0x564ba9389dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:18:27.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.403+0000 7f14d2ffd700 1 --2- 172.21.15.7:0/2125179163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ba937cf60 0x564ba9398540 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f14c802f7d0 tx=0x7f14c802f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec9ef0b7bb2bf87 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:27.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.403+0000 7f14b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:18:27.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.403+0000 7f14b2ffd700 1 -- 172.21.15.7:0/2125179163 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8003960 con 0x564ba937cf60 2024-06-10T01:18:27.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.403+0000 7f14b2ffd700 1 -- 172.21.15.7:0/2125179163 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f14c80268e0 con 0x564ba937cf60 2024-06-10T01:18:27.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.403+0000 7f14e449ab80 1 -- 172.21.15.7:0/2125179163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14cc0093a0 con 0x564ba937cf60 2024-06-10T01:18:27.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.404+0000 7f14b2ffd700 1 -- 172.21.15.7:0/2125179163 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f14c804d050 con 0x564ba937cf60 2024-06-10T01:18:27.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.404+0000 7f14e449ab80 10 client.18639 fetch_fsmap learned FSMap version 40 2024-06-10T01:18:27.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.404+0000 7f14e449ab80 1 -- 172.21.15.7:0/2125179163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x564ba93fbc00 con 0x564ba937cf60 2024-06-10T01:18:27.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.404+0000 7f14b2ffd700 1 -- 172.21.15.7:0/2125179163 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14cc0093a0 con 0x564ba937cf60 2024-06-10T01:18:27.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.404+0000 7f14b2ffd700 1 -- 172.21.15.7:0/2125179163 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f14c803e7d0 con 0x564ba937cf60 2024-06-10T01:18:27.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.404+0000 7f14e449ab80 10 client.18639 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:18:27.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.404+0000 7f14e449ab80 10 client.18639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:18:27.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.404+0000 7f14e449ab80 20 client.18639 populate_metadata read hostname 'smithi007' 2024-06-10T01:18:27.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.404+0000 7f14e449ab80 1 --2- 172.21.15.7:0/2125179163 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564ba93fc2a0 0x564ba93fe6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:27.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.404+0000 7f14e449ab80 4 client.18639 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:18:27.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.404+0000 7f14e449ab80 1 -- 172.21.15.7:0/2125179163 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564ba92d02d0 con 0x564ba93fc2a0 2024-06-10T01:18:27.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.404+0000 7f14d37fe700 1 --2- 172.21.15.7:0/2125179163 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564ba93fc2a0 0x564ba93fe6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:27.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.405+0000 7f14d37fe700 1 --2- 172.21.15.7:0/2125179163 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564ba93fc2a0 0x564ba93fe6c0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1be6e46b37ca6fa server_cookie=633f5593191aa38a in_seq=0 out_seq=0 2024-06-10T01:18:27.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.405+0000 7f14b2ffd700 10 client.18639 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:27.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.405+0000 7f14b2ffd700 1 -- 172.21.15.7:0/2125179163 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564ba92d02d0 con 0x564ba93fc2a0 2024-06-10T01:18:27.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.405+0000 7f14b2ffd700 10 client.18639 handle_command_reply: tid=0 2024-06-10T01:18:27.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.405+0000 7f14e449ab80 1 client.18639 shutdown 2024-06-10T01:18:27.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.405+0000 7f14e449ab80 1 -- 172.21.15.7:0/2125179163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ba937cf60 msgr2=0x564ba9398540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:27.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.405+0000 7f14e449ab80 1 --2- 172.21.15.7:0/2125179163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ba937cf60 0x564ba9398540 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f14c802f7d0 tx=0x7f14c802f6c0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.405+0000 7f14e449ab80 1 -- 172.21.15.7:0/2125179163 shutdown_connections 2024-06-10T01:18:27.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.405+0000 7f14e449ab80 1 --2- 172.21.15.7:0/2125179163 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564ba93fc2a0 0x564ba93fe6c0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.405+0000 7f14e449ab80 1 --2- 172.21.15.7:0/2125179163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ba937d880 0x564ba9398c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.405+0000 7f14b2ffd700 0 client.18639 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:27.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.405+0000 7f14e449ab80 1 --2- 172.21.15.7:0/2125179163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564ba93777a0 0x564ba9389dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.405+0000 7f14e449ab80 1 --2- 172.21.15.7:0/2125179163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564ba937cf60 0x564ba9398540 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.405+0000 7f14e449ab80 1 -- 172.21.15.7:0/2125179163 >> 172.21.15.7:0/2125179163 conn(0x564ba92a4080 msgr2=0x564ba937e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:27.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.405+0000 7f14e449ab80 1 -- 172.21.15.7:0/2125179163 shutdown_connections 2024-06-10T01:18:27.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.405+0000 7f14e449ab80 1 -- 172.21.15.7:0/2125179163 wait complete. 2024-06-10T01:18:27.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.406+0000 7f14e449ab80 20 client.18639 trim_cache size 0 max 16384 2024-06-10T01:18:27.404 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:18:27.404 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T01:18:27.404 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-10T01:18:27.405 INFO:teuthology.orchestra.run.smithi007.stdout: "7c8ede2f-9fdc-46a8-af60-d019640adb5c": { 2024-06-10T01:18:27.405 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-10T01:18:27.405 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "7c8ede2f-9fdc-46a8-af60-d019640adb5c", 2024-06-10T01:18:27.405 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-10T01:18:27.405 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:18:27.405 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:18:27.405 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:18:27.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.406+0000 7f14d9c17700 1 -- 172.21.15.7:0/3752048385 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f14bc0a2c20 msgr2=0x7f14bc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:27.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.406+0000 7f14d9c17700 1 --2- 172.21.15.7:0/3752048385 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f14bc0a2c20 0x7f14bc0a50e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f14d41027d0 tx=0x7f14c8009260 comp rx=0 tx=0).stop 2024-06-10T01:18:27.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.406+0000 7f14d9c17700 1 -- 172.21.15.7:0/3752048385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d4104660 msgr2=0x7f14d419ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:27.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.406+0000 7f14d9c17700 1 --2- 172.21.15.7:0/3752048385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d4104660 0x7f14d419ae20 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f14c4000c00 tx=0x7f14c400bab0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.406+0000 7f14d9c17700 1 -- 172.21.15.7:0/3752048385 shutdown_connections 2024-06-10T01:18:27.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.406+0000 7f14d9c17700 1 --2- 172.21.15.7:0/3752048385 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f14bc0a2c20 0x7f14bc0a50e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.406+0000 7f14d9c17700 1 --2- 172.21.15.7:0/3752048385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14d410a660 0x7f14d419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.406+0000 7f14d9c17700 1 --2- 172.21.15.7:0/3752048385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d4104660 0x7f14d419ae20 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.406+0000 7f14d9c17700 1 --2- 172.21.15.7:0/3752048385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14d4105020 0x7f14d419b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:27.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.406+0000 7f14d9c17700 1 -- 172.21.15.7:0/3752048385 >> 172.21.15.7:0/3752048385 conn(0x7f14d4100190 msgr2=0x7f14d41008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:27.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.406+0000 7f14d9c17700 1 -- 172.21.15.7:0/3752048385 shutdown_connections 2024-06-10T01:18:27.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:18:27.406+0000 7f14d9c17700 1 -- 172.21.15.7:0/3752048385 wait complete. 2024-06-10T01:18:27.512 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:27 smithi007 ceph-mon[32398]: pgmap v805: 129 pgs: 129 active+clean; 145 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 2.0 MiB/s wr, 295 op/s 2024-06-10T01:18:27.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:27 smithi175 ceph-mon[36636]: pgmap v805: 129 pgs: 129 active+clean; 145 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 2.0 MiB/s wr, 295 op/s 2024-06-10T01:18:27.755 INFO:tasks.workunit.client.0.smithi007.stdout: 1 899525 7.16 MB/sec execute 548 sec latency 58.954 ms 2024-06-10T01:18:27.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:27 smithi078 ceph-mon[36683]: pgmap v805: 129 pgs: 129 active+clean; 145 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 2.0 MiB/s wr, 295 op/s 2024-06-10T01:18:28.175 INFO:tasks.cephfs.filesystem:scrub status for tag:7c8ede2f-9fdc-46a8-af60-d019640adb5c - {'path': '/', 'tag': '7c8ede2f-9fdc-46a8-af60-d019640adb5c', 'options': 'recursive,force'} 2024-06-10T01:18:28.755 INFO:tasks.workunit.client.0.smithi007.stdout: 1 901172 7.17 MB/sec execute 549 sec latency 278.572 ms 2024-06-10T01:18:29.758 INFO:tasks.workunit.client.0.smithi007.stdout: 1 902032 7.17 MB/sec execute 550 sec latency 395.311 ms 2024-06-10T01:18:29.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:29 smithi007 ceph-mon[32398]: pgmap v806: 129 pgs: 129 active+clean; 147 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 1.8 MiB/s wr, 321 op/s 2024-06-10T01:18:29.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:29 smithi078 ceph-mon[36683]: pgmap v806: 129 pgs: 129 active+clean; 147 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 1.8 MiB/s wr, 321 op/s 2024-06-10T01:18:29.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:29 smithi175 ceph-mon[36636]: pgmap v806: 129 pgs: 129 active+clean; 147 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 1.8 MiB/s wr, 321 op/s 2024-06-10T01:18:30.755 INFO:tasks.workunit.client.0.smithi007.stdout: 1 903074 7.16 MB/sec execute 551 sec latency 157.595 ms 2024-06-10T01:18:31.755 INFO:tasks.workunit.client.0.smithi007.stdout: 1 904955 7.16 MB/sec execute 552 sec latency 206.063 ms 2024-06-10T01:18:31.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:31 smithi007 ceph-mon[32398]: pgmap v807: 129 pgs: 129 active+clean; 144 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 67 MiB/s rd, 2.0 MiB/s wr, 417 op/s 2024-06-10T01:18:31.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:31 smithi078 ceph-mon[36683]: pgmap v807: 129 pgs: 129 active+clean; 144 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 67 MiB/s rd, 2.0 MiB/s wr, 417 op/s 2024-06-10T01:18:31.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:31 smithi175 ceph-mon[36636]: pgmap v807: 129 pgs: 129 active+clean; 144 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 67 MiB/s rd, 2.0 MiB/s wr, 417 op/s 2024-06-10T01:18:32.755 INFO:tasks.workunit.client.0.smithi007.stdout: 1 906080 7.16 MB/sec execute 553 sec latency 10.913 ms 2024-06-10T01:18:32.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:32 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:18:32.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:32 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:18:32.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:32 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:18:33.755 INFO:tasks.workunit.client.0.smithi007.stdout: 1 908016 7.17 MB/sec execute 554 sec latency 16.473 ms 2024-06-10T01:18:33.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:33 smithi007 ceph-mon[32398]: pgmap v808: 129 pgs: 129 active+clean; 146 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s rd, 1.4 MiB/s wr, 340 op/s 2024-06-10T01:18:33.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:33 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:18:33.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:33 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:18:33.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:33 smithi078 ceph-mon[36683]: pgmap v808: 129 pgs: 129 active+clean; 146 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s rd, 1.4 MiB/s wr, 340 op/s 2024-06-10T01:18:33.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:33 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:18:33.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:33 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:18:33.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:33 smithi175 ceph-mon[36636]: pgmap v808: 129 pgs: 129 active+clean; 146 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s rd, 1.4 MiB/s wr, 340 op/s 2024-06-10T01:18:33.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:33 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:18:33.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:33 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:18:34.755 INFO:tasks.workunit.client.0.smithi007.stdout: 1 909289 7.17 MB/sec execute 555 sec latency 11.196 ms 2024-06-10T01:18:35.755 INFO:tasks.workunit.client.0.smithi007.stdout: 1 910818 7.17 MB/sec execute 556 sec latency 19.590 ms 2024-06-10T01:18:35.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:35 smithi007 ceph-mon[32398]: pgmap v809: 129 pgs: 129 active+clean; 146 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s rd, 1.4 MiB/s wr, 337 op/s 2024-06-10T01:18:35.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:35 smithi078 ceph-mon[36683]: pgmap v809: 129 pgs: 129 active+clean; 146 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s rd, 1.4 MiB/s wr, 337 op/s 2024-06-10T01:18:35.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:35 smithi175 ceph-mon[36636]: pgmap v809: 129 pgs: 129 active+clean; 146 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s rd, 1.4 MiB/s wr, 337 op/s 2024-06-10T01:18:36.755 INFO:tasks.workunit.client.0.smithi007.stdout: 1 912267 7.17 MB/sec execute 557 sec latency 116.467 ms 2024-06-10T01:18:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.363+0000 7fb6f5aae640 1 -- 172.21.15.7:0/2312937091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6f0101af0 msgr2=0x7fb6f0101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.363+0000 7fb6f5aae640 1 --2- 172.21.15.7:0/2312937091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6f0101af0 0x7fb6f0101f70 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009900 tx=0x7fb6e002a410 comp rx=0 tx=0).stop 2024-06-10T01:18:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.363+0000 7fb6f5aae640 1 -- 172.21.15.7:0/2312937091 shutdown_connections 2024-06-10T01:18:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.363+0000 7fb6f5aae640 1 --2- 172.21.15.7:0/2312937091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6f01024b0 0x7fb6f0108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.363+0000 7fb6f5aae640 1 --2- 172.21.15.7:0/2312937091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6f0101af0 0x7fb6f0101f70 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.363+0000 7fb6f5aae640 1 --2- 172.21.15.7:0/2312937091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6f0100900 0x7fb6f0100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.363+0000 7fb6f5aae640 1 -- 172.21.15.7:0/2312937091 >> 172.21.15.7:0/2312937091 conn(0x7fb6f00fbe70 msgr2=0x7fb6f00fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.363+0000 7fb6f5aae640 1 -- 172.21.15.7:0/2312937091 shutdown_connections 2024-06-10T01:18:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.363+0000 7fb6f5aae640 1 -- 172.21.15.7:0/2312937091 wait complete. 2024-06-10T01:18:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.363+0000 7fb6f5aae640 1 Processor -- start 2024-06-10T01:18:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.364+0000 7fb6f5aae640 1 -- start start 2024-06-10T01:18:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.364+0000 7fb6f5aae640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6f0100900 0x7fb6f0197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.364+0000 7fb6f5aae640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6f0101af0 0x7fb6f01982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.364+0000 7fb6f5aae640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6f01024b0 0x7fb6f019f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.364+0000 7fb6f5aae640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f010c560 con 0x7fb6f01024b0 2024-06-10T01:18:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.364+0000 7fb6f5aae640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f010c3e0 con 0x7fb6f0100900 2024-06-10T01:18:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.364+0000 7fb6f5aae640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f010c6e0 con 0x7fb6f0101af0 2024-06-10T01:18:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.364+0000 7fb6effff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6f01024b0 0x7fb6f019f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.364+0000 7fb6ef7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6f0100900 0x7fb6f0197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.364+0000 7fb6ef7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6f0100900 0x7fb6f0197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:46420/0 (socket says 172.21.15.7:46420) 2024-06-10T01:18:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.364+0000 7fb6eeffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6f0101af0 0x7fb6f01982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.364+0000 7fb6ef7fe640 1 -- 172.21.15.7:0/2608493783 learned_addr learned my addr 172.21.15.7:0/2608493783 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.364+0000 7fb6effff640 1 -- 172.21.15.7:0/2608493783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6f0101af0 msgr2=0x7fb6f01982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.364+0000 7fb6effff640 1 --2- 172.21.15.7:0/2608493783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6f0101af0 0x7fb6f01982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.364+0000 7fb6effff640 1 -- 172.21.15.7:0/2608493783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6f0100900 msgr2=0x7fb6f0197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.364+0000 7fb6effff640 1 --2- 172.21.15.7:0/2608493783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6f0100900 0x7fb6f0197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.364+0000 7fb6effff640 1 -- 172.21.15.7:0/2608493783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f019f9b0 con 0x7fb6f01024b0 2024-06-10T01:18:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.364+0000 7fb6effff640 1 --2- 172.21.15.7:0/2608493783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6f01024b0 0x7fb6f019f310 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800ec50 tx=0x7fb6e800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:37.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.365+0000 7fb6ecff9640 1 -- 172.21.15.7:0/2608493783 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8003b70 con 0x7fb6f01024b0 2024-06-10T01:18:37.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.365+0000 7fb6ecff9640 1 -- 172.21.15.7:0/2608493783 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6e8003d10 con 0x7fb6f01024b0 2024-06-10T01:18:37.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.365+0000 7fb6f5aae640 1 -- 172.21.15.7:0/2608493783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f019fca0 con 0x7fb6f01024b0 2024-06-10T01:18:37.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.365+0000 7fb6ecff9640 1 -- 172.21.15.7:0/2608493783 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8010890 con 0x7fb6f01024b0 2024-06-10T01:18:37.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.365+0000 7fb6f5aae640 1 -- 172.21.15.7:0/2608493783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f01a01a0 con 0x7fb6f01024b0 2024-06-10T01:18:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.367+0000 7fb6ecff9640 1 -- 172.21.15.7:0/2608493783 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb6e8003fc0 con 0x7fb6f01024b0 2024-06-10T01:18:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.368+0000 7fb6f87b7740 1 Processor -- start 2024-06-10T01:18:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.368+0000 7fb6f87b7740 1 -- start start 2024-06-10T01:18:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.368+0000 7fb6f87b7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcdd52010 0x560bcdd52430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.368+0000 7fb6f87b7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcdd52a00 0x560bcdd57500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.368+0000 7fb6f87b7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcdd57b70 0x560bcdd5a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.368+0000 7fb6f87b7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560bcdcb54f0 con 0x560bcdd52a00 2024-06-10T01:18:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.368+0000 7fb6f87b7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560bcdc53870 con 0x560bcdd52010 2024-06-10T01:18:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.368+0000 7fb6f87b7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560bcdcb14b0 con 0x560bcdd57b70 2024-06-10T01:18:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.368+0000 7fb6ef7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcdd52a00 0x560bcdd57500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.368+0000 7fb6eeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcdd52010 0x560bcdd52430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.368+0000 7fb6ef7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcdd52a00 0x560bcdd57500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46314/0 (socket says 172.21.15.7:46314) 2024-06-10T01:18:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.368+0000 7fb6ef7fe640 1 -- 172.21.15.7:0/4031106659 learned_addr learned my addr 172.21.15.7:0/4031106659 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.368+0000 7fb6eeffd640 1 --2- 172.21.15.7:0/4031106659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcdd57b70 0x560bcdd5a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.368+0000 7fb6ef7fe640 1 -- 172.21.15.7:0/4031106659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcdd57b70 msgr2=0x560bcdd5a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.368+0000 7fb6ef7fe640 1 --2- 172.21.15.7:0/4031106659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcdd57b70 0x560bcdd5a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.368+0000 7fb6ef7fe640 1 -- 172.21.15.7:0/4031106659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcdd52010 msgr2=0x560bcdd52430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.368+0000 7fb6ef7fe640 1 --2- 172.21.15.7:0/4031106659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcdd52010 0x560bcdd52430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.368+0000 7fb6ef7fe640 1 -- 172.21.15.7:0/4031106659 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bcdc6f9b0 con 0x560bcdd52a00 2024-06-10T01:18:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.368+0000 7fb6eeffd640 1 --2- 172.21.15.7:0/4031106659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcdd57b70 0x560bcdd5a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:18:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.369+0000 7fb6ecff9640 1 --2- 172.21.15.7:0/2608493783 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb6dc0a2c00 0x7fb6dc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.369+0000 7fb6ef7fe640 1 --2- 172.21.15.7:0/4031106659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcdd52a00 0x560bcdd57500 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800be20 tx=0x7fb6d80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9574321baad6be server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.369+0000 7fb6eeffd640 1 --2- 172.21.15.7:0/2608493783 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb6dc0a2c00 0x7fb6dc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.369+0000 7fb6e5ffb640 1 -- 172.21.15.7:0/4031106659 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8011640 con 0x560bcdd52a00 2024-06-10T01:18:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.369+0000 7fb6ecff9640 1 -- 172.21.15.7:0/2608493783 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb6e80cf910 con 0x7fb6f01024b0 2024-06-10T01:18:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.369+0000 7fb6e5ffb640 1 -- 172.21.15.7:0/4031106659 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6d8011cc0 con 0x560bcdd52a00 2024-06-10T01:18:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.369+0000 7fb6e5ffb640 1 -- 172.21.15.7:0/4031106659 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8019d70 con 0x560bcdd52a00 2024-06-10T01:18:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.370+0000 7fb6f87b7740 1 -- 172.21.15.7:0/4031106659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcdd52a00 msgr2=0x560bcdd57500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.370+0000 7fb6eeffd640 1 --2- 172.21.15.7:0/2608493783 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb6dc0a2c00 0x7fb6dc0a50c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00314e0 tx=0x7fb6e0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.370+0000 7fb6f87b7740 1 --2- 172.21.15.7:0/4031106659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcdd52a00 0x560bcdd57500 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800be20 tx=0x7fb6d80107a0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.370+0000 7fb6f87b7740 1 -- 172.21.15.7:0/4031106659 shutdown_connections 2024-06-10T01:18:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.370+0000 7fb6f87b7740 1 --2- 172.21.15.7:0/4031106659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcdd57b70 0x560bcdd5a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.370+0000 7fb6f87b7740 1 --2- 172.21.15.7:0/4031106659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcdd52a00 0x560bcdd57500 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.370+0000 7fb6f87b7740 1 --2- 172.21.15.7:0/4031106659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcdd52010 0x560bcdd52430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.370+0000 7fb6f87b7740 1 -- 172.21.15.7:0/4031106659 >> 172.21.15.7:0/4031106659 conn(0x560bcdd4d680 msgr2=0x560bcdd4da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.370+0000 7fb6f87b7740 1 -- 172.21.15.7:0/4031106659 shutdown_connections 2024-06-10T01:18:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.370+0000 7fb6f87b7740 1 -- 172.21.15.7:0/4031106659 wait complete. 2024-06-10T01:18:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.370+0000 7fb6f87b7740 1 -- start start 2024-06-10T01:18:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.370+0000 7fb6f87b7740 1 Processor -- start 2024-06-10T01:18:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.370+0000 7fb6f87b7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcdd52010 0x560bcdd5c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.370+0000 7fb6f87b7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcdd52a00 0x560bcdd5c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.370+0000 7fb6f87b7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcdd57b70 0x560bcdd63780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.371+0000 7fb6f87b7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560bcdd5ab20 con 0x560bcdd52a00 2024-06-10T01:18:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.371+0000 7fb6f87b7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560bcdc3eae0 con 0x560bcdd52010 2024-06-10T01:18:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.371+0000 7fb6f87b7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560bcdd5aca0 con 0x560bcdd57b70 2024-06-10T01:18:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.371+0000 7fb6eeffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcdd52a00 0x560bcdd5c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.371+0000 7fb6f87b7740 10 client.0 _ll_register_callbacks cb 0x560bcdc779b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:18:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.371+0000 7fb6eeffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcdd52a00 0x560bcdd5c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46330/0 (socket says 172.21.15.7:46330) 2024-06-10T01:18:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.371+0000 7fb6eeffd640 1 -- 172.21.15.7:0/640294419 learned_addr learned my addr 172.21.15.7:0/640294419 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.371+0000 7fb6ef7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcdd52010 0x560bcdd5c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.371+0000 7fb6effff640 1 --2- 172.21.15.7:0/640294419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcdd57b70 0x560bcdd63780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.371+0000 7fb6ef7fe640 1 -- 172.21.15.7:0/640294419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcdd57b70 msgr2=0x560bcdd63780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.371+0000 7fb6ef7fe640 1 --2- 172.21.15.7:0/640294419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcdd57b70 0x560bcdd63780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.371+0000 7fb6ef7fe640 1 -- 172.21.15.7:0/640294419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcdd52a00 msgr2=0x560bcdd5c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.371+0000 7fb6ef7fe640 1 --2- 172.21.15.7:0/640294419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcdd52a00 0x560bcdd5c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.371+0000 7fb6ef7fe640 1 -- 172.21.15.7:0/640294419 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bcd9c9210 con 0x560bcdd52010 2024-06-10T01:18:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.371+0000 7fb6eeffd640 1 --2- 172.21.15.7:0/640294419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcdd52a00 0x560bcdd5c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:18:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.371+0000 7fb6ef7fe640 1 --2- 172.21.15.7:0/640294419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcdd52010 0x560bcdd5c200 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x560bcdd4e550 tx=0x7fb6d8011f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3de04dcf2aa86f40 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.371+0000 7fb6ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:18:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.371+0000 7fb6ceffd640 1 -- 172.21.15.7:0/640294419 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8027020 con 0x560bcdd52010 2024-06-10T01:18:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.371+0000 7fb6f87b7740 1 -- 172.21.15.7:0/640294419 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6d8011cc0 con 0x560bcdd52010 2024-06-10T01:18:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.371+0000 7fb6ceffd640 1 -- 172.21.15.7:0/640294419 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb6d800b970 con 0x560bcdd52010 2024-06-10T01:18:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.372+0000 7fb6ceffd640 1 -- 172.21.15.7:0/640294419 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb6d8032050 con 0x560bcdd52010 2024-06-10T01:18:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.372+0000 7fb6f87b7740 10 client.27965 fetch_fsmap learned FSMap version 40 2024-06-10T01:18:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.372+0000 7fb6f87b7740 1 -- 172.21.15.7:0/640294419 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560bcda95620 con 0x560bcdd52010 2024-06-10T01:18:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.372+0000 7fb6ceffd640 1 -- 172.21.15.7:0/640294419 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6d8011cc0 con 0x560bcdd52010 2024-06-10T01:18:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.372+0000 7fb6ceffd640 1 -- 172.21.15.7:0/640294419 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7fb6d8011440 con 0x560bcdd52010 2024-06-10T01:18:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.372+0000 7fb6f87b7740 10 client.27965 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:18:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.372+0000 7fb6f87b7740 10 client.27965 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:18:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.372+0000 7fb6f87b7740 20 client.27965 populate_metadata read hostname 'smithi007' 2024-06-10T01:18:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.372+0000 7fb6f87b7740 1 --2- 172.21.15.7:0/640294419 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560bcdd6c130 0x560bcdd6e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.372+0000 7fb6f87b7740 4 client.27965 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:18:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.372+0000 7fb6f87b7740 1 -- 172.21.15.7:0/640294419 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bcdcc8f10 con 0x560bcdd6c130 2024-06-10T01:18:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.372+0000 7fb6eeffd640 1 --2- 172.21.15.7:0/640294419 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560bcdd6c130 0x560bcdd6e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.373+0000 7fb6eeffd640 1 --2- 172.21.15.7:0/640294419 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560bcdd6c130 0x560bcdd6e520 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=564a40c8f09ef436 server_cookie=78d506676fdc949b in_seq=0 out_seq=0 2024-06-10T01:18:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.373+0000 7fb6ceffd640 10 client.27965 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.373+0000 7fb6ceffd640 1 -- 172.21.15.7:0/640294419 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560bcdcc8f10 con 0x560bcdd6c130 2024-06-10T01:18:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.373+0000 7fb6ceffd640 10 client.27965 handle_command_reply: tid=0 2024-06-10T01:18:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.374+0000 7fb6f87b7740 1 client.27965 shutdown 2024-06-10T01:18:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.374+0000 7fb6f87b7740 1 -- 172.21.15.7:0/640294419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcdd52010 msgr2=0x560bcdd5c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.374+0000 7fb6f87b7740 1 --2- 172.21.15.7:0/640294419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcdd52010 0x560bcdd5c200 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x560bcdd4e550 tx=0x7fb6d8011f00 comp rx=0 tx=0).stop 2024-06-10T01:18:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.374+0000 7fb6f87b7740 1 -- 172.21.15.7:0/640294419 shutdown_connections 2024-06-10T01:18:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.374+0000 7fb6f87b7740 1 --2- 172.21.15.7:0/640294419 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560bcdd6c130 0x560bcdd6e520 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.374+0000 7fb6f87b7740 1 --2- 172.21.15.7:0/640294419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcdd57b70 0x560bcdd63780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.374+0000 7fb6ceffd640 0 client.27965 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.374+0000 7fb6f87b7740 1 --2- 172.21.15.7:0/640294419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcdd52a00 0x560bcdd5c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.374+0000 7fb6f87b7740 1 --2- 172.21.15.7:0/640294419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcdd52010 0x560bcdd5c200 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.374+0000 7fb6f87b7740 1 -- 172.21.15.7:0/640294419 >> 172.21.15.7:0/640294419 conn(0x560bcdd4d680 msgr2=0x560bcdd4c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.374+0000 7fb6f87b7740 1 -- 172.21.15.7:0/640294419 shutdown_connections 2024-06-10T01:18:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.374+0000 7fb6f87b7740 1 -- 172.21.15.7:0/640294419 wait complete. 2024-06-10T01:18:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.374+0000 7fb6f87b7740 20 client.27965 trim_cache size 0 max 16384 2024-06-10T01:18:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.390+0000 7fb6f87b7740 1 Processor -- start 2024-06-10T01:18:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.390+0000 7fb6f87b7740 1 -- start start 2024-06-10T01:18:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.390+0000 7fb6f87b7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcdd52010 0x560bcdd50ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.390+0000 7fb6f87b7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcdd52a00 0x560bcdd636a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.390+0000 7fb6f87b7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcdd57b70 0x560bcdd5b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.390+0000 7fb6f87b7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f01aacb0 con 0x560bcdd57b70 2024-06-10T01:18:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.390+0000 7fb6f87b7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f01a8580 con 0x560bcdd52a00 2024-06-10T01:18:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.390+0000 7fb6f87b7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f01a63d0 con 0x560bcdd52010 2024-06-10T01:18:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.390+0000 7fb6effff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcdd57b70 0x560bcdd5b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.390+0000 7fb6ef7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcdd52010 0x560bcdd50ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.390+0000 7fb6effff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcdd57b70 0x560bcdd5b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46342/0 (socket says 172.21.15.7:46342) 2024-06-10T01:18:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.390+0000 7fb6eeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcdd52a00 0x560bcdd636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.390+0000 7fb6effff640 1 -- 172.21.15.7:0/2174480277 learned_addr learned my addr 172.21.15.7:0/2174480277 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.390+0000 7fb6effff640 1 -- 172.21.15.7:0/2174480277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcdd52010 msgr2=0x560bcdd50ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.390+0000 7fb6effff640 1 --2- 172.21.15.7:0/2174480277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcdd52010 0x560bcdd50ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.390+0000 7fb6effff640 1 -- 172.21.15.7:0/2174480277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcdd52a00 msgr2=0x560bcdd636a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.390+0000 7fb6effff640 1 --2- 172.21.15.7:0/2174480277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcdd52a00 0x560bcdd636a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.390+0000 7fb6effff640 1 -- 172.21.15.7:0/2174480277 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bcdd6eb50 con 0x560bcdd57b70 2024-06-10T01:18:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.390+0000 7fb6eeffd640 1 --2- 172.21.15.7:0/2174480277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcdd52a00 0x560bcdd636a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:18:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.391+0000 7fb6effff640 1 --2- 172.21.15.7:0/2174480277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcdd57b70 0x560bcdd5b820 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fb6e80cd0e0 tx=0x7fb6e800ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44fff3786b94160f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.391+0000 7fb6e57fa640 1 -- 172.21.15.7:0/2174480277 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8099f30 con 0x560bcdd57b70 2024-06-10T01:18:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.391+0000 7fb6e57fa640 1 -- 172.21.15.7:0/2174480277 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6e8009820 con 0x560bcdd57b70 2024-06-10T01:18:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.391+0000 7fb6e57fa640 1 -- 172.21.15.7:0/2174480277 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e80a45d0 con 0x560bcdd57b70 2024-06-10T01:18:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.392+0000 7fb6f87b7740 1 -- 172.21.15.7:0/2174480277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcdd57b70 msgr2=0x560bcdd5b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.392+0000 7fb6f87b7740 1 --2- 172.21.15.7:0/2174480277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcdd57b70 0x560bcdd5b820 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fb6e80cd0e0 tx=0x7fb6e800ce70 comp rx=0 tx=0).stop 2024-06-10T01:18:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.392+0000 7fb6f87b7740 1 -- 172.21.15.7:0/2174480277 shutdown_connections 2024-06-10T01:18:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.392+0000 7fb6f87b7740 1 --2- 172.21.15.7:0/2174480277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcdd57b70 0x560bcdd5b820 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.392+0000 7fb6f87b7740 1 --2- 172.21.15.7:0/2174480277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcdd52a00 0x560bcdd636a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.392+0000 7fb6f87b7740 1 --2- 172.21.15.7:0/2174480277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcdd52010 0x560bcdd50ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.392+0000 7fb6f87b7740 1 -- 172.21.15.7:0/2174480277 >> 172.21.15.7:0/2174480277 conn(0x560bcdd4d680 msgr2=0x560bcdd50590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.392+0000 7fb6f87b7740 1 -- 172.21.15.7:0/2174480277 shutdown_connections 2024-06-10T01:18:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.392+0000 7fb6f87b7740 1 -- 172.21.15.7:0/2174480277 wait complete. 2024-06-10T01:18:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.392+0000 7fb6f87b7740 1 -- start start 2024-06-10T01:18:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.392+0000 7fb6f87b7740 1 Processor -- start 2024-06-10T01:18:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.393+0000 7fb6f87b7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcdd52010 0x560bcdd608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.393+0000 7fb6f87b7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcdd52a00 0x560bcdd60df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.393+0000 7fb6f87b7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcdd57b70 0x560bcdd61e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.393+0000 7fb6f87b7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f01a6a40 con 0x560bcdd57b70 2024-06-10T01:18:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.393+0000 7fb6f87b7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f01a7eb0 con 0x560bcdd52a00 2024-06-10T01:18:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.393+0000 7fb6f87b7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f01a7110 con 0x560bcdd52010 2024-06-10T01:18:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.393+0000 7fb6f87b7740 10 client.0 _ll_register_callbacks cb 0x7fb6e0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:18:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.393+0000 7fb6effff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcdd57b70 0x560bcdd61e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.393+0000 7fb6eeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcdd52a00 0x560bcdd60df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.393+0000 7fb6effff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcdd57b70 0x560bcdd61e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46358/0 (socket says 172.21.15.7:46358) 2024-06-10T01:18:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.393+0000 7fb6effff640 1 -- 172.21.15.7:0/1750557355 learned_addr learned my addr 172.21.15.7:0/1750557355 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.393+0000 7fb6ef7fe640 1 --2- 172.21.15.7:0/1750557355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcdd52010 0x560bcdd608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.393+0000 7fb6effff640 1 -- 172.21.15.7:0/1750557355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcdd52010 msgr2=0x560bcdd608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.393+0000 7fb6effff640 1 --2- 172.21.15.7:0/1750557355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcdd52010 0x560bcdd608b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.393+0000 7fb6effff640 1 -- 172.21.15.7:0/1750557355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcdd52a00 msgr2=0x560bcdd60df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.393+0000 7fb6effff640 1 --2- 172.21.15.7:0/1750557355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcdd52a00 0x560bcdd60df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.393+0000 7fb6effff640 1 -- 172.21.15.7:0/1750557355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bcdd4e350 con 0x560bcdd57b70 2024-06-10T01:18:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.393+0000 7fb6eeffd640 1 --2- 172.21.15.7:0/1750557355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcdd52a00 0x560bcdd60df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:18:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.393+0000 7fb6effff640 1 --2- 172.21.15.7:0/1750557355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcdd57b70 0x560bcdd61e00 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fb6e80cd400 tx=0x7fb6e800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f943ca7a3e3908bb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.393+0000 7fb6ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:18:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.393+0000 7fb6ceffd640 1 -- 172.21.15.7:0/1750557355 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e807e020 con 0x560bcdd57b70 2024-06-10T01:18:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.393+0000 7fb6ceffd640 1 -- 172.21.15.7:0/1750557355 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb6e807add0 con 0x560bcdd57b70 2024-06-10T01:18:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.393+0000 7fb6f87b7740 1 -- 172.21.15.7:0/1750557355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6e8009820 con 0x560bcdd57b70 2024-06-10T01:18:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.393+0000 7fb6ceffd640 1 -- 172.21.15.7:0/1750557355 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb6e809c4f0 con 0x560bcdd57b70 2024-06-10T01:18:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.394+0000 7fb6f87b7740 10 client.18657 fetch_fsmap learned FSMap version 40 2024-06-10T01:18:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.394+0000 7fb6f87b7740 1 -- 172.21.15.7:0/1750557355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560bcdd4d4b0 con 0x560bcdd57b70 2024-06-10T01:18:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.394+0000 7fb6ceffd640 1 -- 172.21.15.7:0/1750557355 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6e8009820 con 0x560bcdd57b70 2024-06-10T01:18:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.394+0000 7fb6ceffd640 1 -- 172.21.15.7:0/1750557355 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7fb6e8086040 con 0x560bcdd57b70 2024-06-10T01:18:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.394+0000 7fb6f87b7740 10 client.18657 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:18:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.394+0000 7fb6f87b7740 10 client.18657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:18:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.394+0000 7fb6f87b7740 20 client.18657 populate_metadata read hostname 'smithi007' 2024-06-10T01:18:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.394+0000 7fb6f87b7740 1 --2- 172.21.15.7:0/1750557355 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560bcddd6ee0 0x560bcddd92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.394+0000 7fb6f87b7740 4 client.18657 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:18:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.394+0000 7fb6f87b7740 1 -- 172.21.15.7:0/1750557355 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560bcdcc2c20 con 0x560bcddd6ee0 2024-06-10T01:18:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.394+0000 7fb6ef7fe640 1 --2- 172.21.15.7:0/1750557355 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560bcddd6ee0 0x560bcddd92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.394+0000 7fb6ef7fe640 1 --2- 172.21.15.7:0/1750557355 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560bcddd6ee0 0x560bcddd92d0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71ef6802f16f56fb server_cookie=7df4d060af67a87c in_seq=0 out_seq=0 2024-06-10T01:18:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.394+0000 7fb6ceffd640 10 client.18657 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.473+0000 7fb6ceffd640 1 -- 172.21.15.7:0/1750557355 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560bcdcc2c20 con 0x560bcddd6ee0 2024-06-10T01:18:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.473+0000 7fb6ceffd640 10 client.18657 handle_command_reply: tid=0 2024-06-10T01:18:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.473+0000 7fb6f87b7740 1 client.18657 shutdown 2024-06-10T01:18:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.473+0000 7fb6f87b7740 1 -- 172.21.15.7:0/1750557355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcdd57b70 msgr2=0x560bcdd61e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.473+0000 7fb6f87b7740 1 --2- 172.21.15.7:0/1750557355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcdd57b70 0x560bcdd61e00 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fb6e80cd400 tx=0x7fb6e800cf70 comp rx=0 tx=0).stop 2024-06-10T01:18:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.473+0000 7fb6f87b7740 1 -- 172.21.15.7:0/1750557355 shutdown_connections 2024-06-10T01:18:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.473+0000 7fb6f87b7740 1 --2- 172.21.15.7:0/1750557355 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560bcddd6ee0 0x560bcddd92d0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.473+0000 7fb6f87b7740 1 --2- 172.21.15.7:0/1750557355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560bcdd57b70 0x560bcdd61e00 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.473+0000 7fb6f87b7740 1 --2- 172.21.15.7:0/1750557355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bcdd52a00 0x560bcdd60df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.473+0000 7fb6ceffd640 0 client.18657 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.473+0000 7fb6f87b7740 1 --2- 172.21.15.7:0/1750557355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bcdd52010 0x560bcdd608b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.473+0000 7fb6f87b7740 1 -- 172.21.15.7:0/1750557355 >> 172.21.15.7:0/1750557355 conn(0x560bcdd4d680 msgr2=0x560bcdd4bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.474+0000 7fb6f87b7740 1 -- 172.21.15.7:0/1750557355 shutdown_connections 2024-06-10T01:18:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.474+0000 7fb6f87b7740 1 -- 172.21.15.7:0/1750557355 wait complete. 2024-06-10T01:18:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.474+0000 7fb6f87b7740 20 client.18657 trim_cache size 0 max 16384 2024-06-10T01:18:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:18:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:18:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:18:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.474+0000 7fb6f5aae640 1 -- 172.21.15.7:0/2608493783 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb6dc0a2c00 msgr2=0x7fb6dc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.474+0000 7fb6f5aae640 1 --2- 172.21.15.7:0/2608493783 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb6dc0a2c00 0x7fb6dc0a50c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00314e0 tx=0x7fb6e0005fd0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.474+0000 7fb6f5aae640 1 -- 172.21.15.7:0/2608493783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6f01024b0 msgr2=0x7fb6f019f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.474+0000 7fb6f5aae640 1 --2- 172.21.15.7:0/2608493783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6f01024b0 0x7fb6f019f310 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800ec50 tx=0x7fb6e800c6a0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.474+0000 7fb6f5aae640 1 -- 172.21.15.7:0/2608493783 shutdown_connections 2024-06-10T01:18:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.474+0000 7fb6f5aae640 1 --2- 172.21.15.7:0/2608493783 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb6dc0a2c00 0x7fb6dc0a50c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.474+0000 7fb6f5aae640 1 --2- 172.21.15.7:0/2608493783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6f01024b0 0x7fb6f019f310 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.474+0000 7fb6f5aae640 1 --2- 172.21.15.7:0/2608493783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6f0101af0 0x7fb6f01982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.474+0000 7fb6f5aae640 1 --2- 172.21.15.7:0/2608493783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6f0100900 0x7fb6f0197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.474+0000 7fb6f5aae640 1 -- 172.21.15.7:0/2608493783 >> 172.21.15.7:0/2608493783 conn(0x7fb6f00fbe70 msgr2=0x7fb6f00fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.474+0000 7fb6f5aae640 1 -- 172.21.15.7:0/2608493783 shutdown_connections 2024-06-10T01:18:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:37.474+0000 7fb6f5aae640 1 -- 172.21.15.7:0/2608493783 wait complete. 2024-06-10T01:18:37.755 INFO:tasks.workunit.client.0.smithi007.stdout: 1 913194 7.17 MB/sec execute 558 sec latency 152.082 ms 2024-06-10T01:18:37.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:37 smithi007 ceph-mon[32398]: pgmap v810: 129 pgs: 129 active+clean; 147 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 62 MiB/s rd, 1.7 MiB/s wr, 394 op/s 2024-06-10T01:18:37.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:37 smithi078 ceph-mon[36683]: pgmap v810: 129 pgs: 129 active+clean; 147 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 62 MiB/s rd, 1.7 MiB/s wr, 394 op/s 2024-06-10T01:18:37.953 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:37 smithi175 ceph-mon[36636]: pgmap v810: 129 pgs: 129 active+clean; 147 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 62 MiB/s rd, 1.7 MiB/s wr, 394 op/s 2024-06-10T01:18:38.756 INFO:tasks.workunit.client.0.smithi007.stdout: 1 914208 7.16 MB/sec execute 559 sec latency 401.392 ms 2024-06-10T01:18:39.756 INFO:tasks.workunit.client.0.smithi007.stdout: 1 916109 7.17 MB/sec execute 560 sec latency 42.524 ms 2024-06-10T01:18:39.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:39 smithi007 ceph-mon[32398]: pgmap v811: 129 pgs: 129 active+clean; 149 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 MiB/s rd, 1.4 MiB/s wr, 333 op/s 2024-06-10T01:18:39.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:39 smithi078 ceph-mon[36683]: pgmap v811: 129 pgs: 129 active+clean; 149 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 MiB/s rd, 1.4 MiB/s wr, 333 op/s 2024-06-10T01:18:39.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:39 smithi175 ceph-mon[36636]: pgmap v811: 129 pgs: 129 active+clean; 149 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 MiB/s rd, 1.4 MiB/s wr, 333 op/s 2024-06-10T01:18:40.756 INFO:tasks.workunit.client.0.smithi007.stdout: 1 916993 7.18 MB/sec execute 561 sec latency 35.353 ms 2024-06-10T01:18:41.756 INFO:tasks.workunit.client.0.smithi007.stdout: 1 919124 7.20 MB/sec execute 562 sec latency 25.054 ms 2024-06-10T01:18:41.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:41 smithi007 ceph-mon[32398]: pgmap v812: 129 pgs: 129 active+clean; 150 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 74 MiB/s rd, 1.7 MiB/s wr, 379 op/s 2024-06-10T01:18:41.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:41 smithi078 ceph-mon[36683]: pgmap v812: 129 pgs: 129 active+clean; 150 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 74 MiB/s rd, 1.7 MiB/s wr, 379 op/s 2024-06-10T01:18:41.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:41 smithi175 ceph-mon[36636]: pgmap v812: 129 pgs: 129 active+clean; 150 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 74 MiB/s rd, 1.7 MiB/s wr, 379 op/s 2024-06-10T01:18:42.756 INFO:tasks.workunit.client.0.smithi007.stdout: 1 920794 7.21 MB/sec execute 563 sec latency 49.713 ms 2024-06-10T01:18:43.756 INFO:tasks.workunit.client.0.smithi007.stdout: 1 923160 7.22 MB/sec execute 564 sec latency 16.070 ms 2024-06-10T01:18:43.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:43 smithi007 ceph-mon[32398]: pgmap v813: 129 pgs: 129 active+clean; 151 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 MiB/s rd, 1.5 MiB/s wr, 307 op/s 2024-06-10T01:18:43.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:43 smithi078 ceph-mon[36683]: pgmap v813: 129 pgs: 129 active+clean; 151 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 MiB/s rd, 1.5 MiB/s wr, 307 op/s 2024-06-10T01:18:43.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:43 smithi175 ceph-mon[36636]: pgmap v813: 129 pgs: 129 active+clean; 151 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 MiB/s rd, 1.5 MiB/s wr, 307 op/s 2024-06-10T01:18:44.756 INFO:tasks.workunit.client.0.smithi007.stdout: 1 924703 7.22 MB/sec execute 565 sec latency 9.965 ms 2024-06-10T01:18:45.756 INFO:tasks.workunit.client.0.smithi007.stdout: 1 927397 7.23 MB/sec execute 566 sec latency 15.425 ms 2024-06-10T01:18:45.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:45 smithi007 ceph-mon[32398]: pgmap v814: 129 pgs: 129 active+clean; 147 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 1.6 MiB/s wr, 330 op/s 2024-06-10T01:18:45.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:45 smithi078 ceph-mon[36683]: pgmap v814: 129 pgs: 129 active+clean; 147 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 1.6 MiB/s wr, 330 op/s 2024-06-10T01:18:45.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:45 smithi175 ceph-mon[36636]: pgmap v814: 129 pgs: 129 active+clean; 147 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 1.6 MiB/s wr, 330 op/s 2024-06-10T01:18:46.756 INFO:tasks.workunit.client.0.smithi007.stdout: 1 928722 7.23 MB/sec execute 567 sec latency 18.532 ms 2024-06-10T01:18:47.756 INFO:tasks.workunit.client.0.smithi007.stdout: 1 931147 7.24 MB/sec execute 568 sec latency 11.108 ms 2024-06-10T01:18:47.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:47 smithi007 ceph-mon[32398]: pgmap v815: 129 pgs: 129 active+clean; 142 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 MiB/s rd, 1.7 MiB/s wr, 342 op/s 2024-06-10T01:18:47.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:47 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:18:47.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:47 smithi078 ceph-mon[36683]: pgmap v815: 129 pgs: 129 active+clean; 142 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 MiB/s rd, 1.7 MiB/s wr, 342 op/s 2024-06-10T01:18:47.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:47 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:18:47.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:47 smithi175 ceph-mon[36636]: pgmap v815: 129 pgs: 129 active+clean; 142 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 MiB/s rd, 1.7 MiB/s wr, 342 op/s 2024-06-10T01:18:47.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:47 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:18:48.756 INFO:tasks.workunit.client.0.smithi007.stdout: 1 932115 7.24 MB/sec execute 569 sec latency 67.691 ms 2024-06-10T01:18:48.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:48 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:18:48.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:48 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:18:48.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:48 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:18:48.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:48 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:18:48.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:48 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:18:48.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:48 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:18:48.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:48 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:18:48.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:48 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:18:48.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:48 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:18:49.756 INFO:tasks.workunit.client.0.smithi007.stdout: 1 934695 7.25 MB/sec execute 570 sec latency 10.059 ms 2024-06-10T01:18:49.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:49 smithi007 ceph-mon[32398]: pgmap v816: 129 pgs: 129 active+clean; 141 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 1.2 MiB/s wr, 273 op/s 2024-06-10T01:18:49.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:49 smithi078 ceph-mon[36683]: pgmap v816: 129 pgs: 129 active+clean; 141 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 1.2 MiB/s wr, 273 op/s 2024-06-10T01:18:49.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:49 smithi175 ceph-mon[36636]: pgmap v816: 129 pgs: 129 active+clean; 141 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 1.2 MiB/s wr, 273 op/s 2024-06-10T01:18:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.568+0000 7f49d274f640 1 -- 172.21.15.7:0/83231307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49cc102480 msgr2=0x7f49cc104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.568+0000 7f49d274f640 1 --2- 172.21.15.7:0/83231307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49cc102480 0x7f49cc104870 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f49bc009900 tx=0x7f49bc02a410 comp rx=0 tx=0).stop 2024-06-10T01:18:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.568+0000 7f49d274f640 1 -- 172.21.15.7:0/83231307 shutdown_connections 2024-06-10T01:18:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.568+0000 7f49d274f640 1 --2- 172.21.15.7:0/83231307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49cc1076e0 0x7f49cc109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.568+0000 7f49d274f640 1 --2- 172.21.15.7:0/83231307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49cc104db0 0x7f49cc1071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.568+0000 7f49d274f640 1 --2- 172.21.15.7:0/83231307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49cc102480 0x7f49cc104870 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.568+0000 7f49d274f640 1 -- 172.21.15.7:0/83231307 >> 172.21.15.7:0/83231307 conn(0x7f49cc0fbe70 msgr2=0x7f49cc0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.568+0000 7f49d274f640 1 -- 172.21.15.7:0/83231307 shutdown_connections 2024-06-10T01:18:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.568+0000 7f49d274f640 1 -- 172.21.15.7:0/83231307 wait complete. 2024-06-10T01:18:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.568+0000 7f49d274f640 1 Processor -- start 2024-06-10T01:18:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.568+0000 7f49d274f640 1 -- start start 2024-06-10T01:18:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.569+0000 7f49d274f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49cc102480 0x7f49cc197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.569+0000 7f49d274f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49cc104db0 0x7f49cc1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.569+0000 7f49d274f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49cc1076e0 0x7f49cc19f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.569+0000 7f49d274f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc10d6b0 con 0x7f49cc102480 2024-06-10T01:18:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.569+0000 7f49d274f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc10d530 con 0x7f49cc104db0 2024-06-10T01:18:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.569+0000 7f49d274f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc10d830 con 0x7f49cc1076e0 2024-06-10T01:18:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.569+0000 7f49cbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49cc102480 0x7f49cc197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.569+0000 7f49cbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49cc102480 0x7f49cc197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58758/0 (socket says 172.21.15.7:58758) 2024-06-10T01:18:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.569+0000 7f49cbfff640 1 -- 172.21.15.7:0/2070118867 learned_addr learned my addr 172.21.15.7:0/2070118867 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.569+0000 7f49cb7fe640 1 --2- 172.21.15.7:0/2070118867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49cc104db0 0x7f49cc1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.569+0000 7f49d0cec640 1 --2- 172.21.15.7:0/2070118867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49cc1076e0 0x7f49cc19f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.569+0000 7f49cbfff640 1 -- 172.21.15.7:0/2070118867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49cc1076e0 msgr2=0x7f49cc19f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.569+0000 7f49cbfff640 1 --2- 172.21.15.7:0/2070118867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49cc1076e0 0x7f49cc19f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.569+0000 7f49cbfff640 1 -- 172.21.15.7:0/2070118867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49cc104db0 msgr2=0x7f49cc1982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.569+0000 7f49cbfff640 1 --2- 172.21.15.7:0/2070118867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49cc104db0 0x7f49cc1982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.569+0000 7f49cbfff640 1 -- 172.21.15.7:0/2070118867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49cc19f9d0 con 0x7f49cc102480 2024-06-10T01:18:50.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.569+0000 7f49cb7fe640 1 --2- 172.21.15.7:0/2070118867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49cc104db0 0x7f49cc1982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:18:50.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.570+0000 7f49cbfff640 1 --2- 172.21.15.7:0/2070118867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49cc102480 0x7f49cc197db0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f49bc009230 tx=0x7f49bc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:50.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.570+0000 7f49c97fa640 1 -- 172.21.15.7:0/2070118867 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc03f030 con 0x7f49cc102480 2024-06-10T01:18:50.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.570+0000 7f49c97fa640 1 -- 172.21.15.7:0/2070118867 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49bc003ce0 con 0x7f49cc102480 2024-06-10T01:18:50.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.570+0000 7f49d274f640 1 -- 172.21.15.7:0/2070118867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49cc19fc60 con 0x7f49cc102480 2024-06-10T01:18:50.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.570+0000 7f49c97fa640 1 -- 172.21.15.7:0/2070118867 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc035070 con 0x7f49cc102480 2024-06-10T01:18:50.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.570+0000 7f49d274f640 1 -- 172.21.15.7:0/2070118867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49cc1a0150 con 0x7f49cc102480 2024-06-10T01:18:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.572+0000 7f49c97fa640 1 -- 172.21.15.7:0/2070118867 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f49bc042030 con 0x7f49cc102480 2024-06-10T01:18:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.572+0000 7f49d545a740 1 Processor -- start 2024-06-10T01:18:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.572+0000 7f49d545a740 1 -- start start 2024-06-10T01:18:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.572+0000 7f49d545a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558be1d7f010 0x558be1d7f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.572+0000 7f49d545a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be1d7fa00 0x558be1d84500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.572+0000 7f49d545a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558be1d84b70 0x558be1d87000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.572+0000 7f49d545a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558be1ce24f0 con 0x558be1d84b70 2024-06-10T01:18:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.572+0000 7f49d545a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558be1c80870 con 0x558be1d7fa00 2024-06-10T01:18:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.573+0000 7f49d545a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558be1cde4b0 con 0x558be1d7f010 2024-06-10T01:18:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.573+0000 7f49d0cec640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558be1d7f010 0x558be1d7f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.573+0000 7f49cb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558be1d84b70 0x558be1d87000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.573+0000 7f49d0cec640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558be1d7f010 0x558be1d7f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:40654/0 (socket says 172.21.15.7:40654) 2024-06-10T01:18:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.573+0000 7f49d0cec640 1 -- 172.21.15.7:0/9144323 learned_addr learned my addr 172.21.15.7:0/9144323 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.573+0000 7f49cbfff640 1 --2- 172.21.15.7:0/9144323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be1d7fa00 0x558be1d84500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.573+0000 7f49c97fa640 1 --2- 172.21.15.7:0/2070118867 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f49b40a2b80 0x7f49b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.573+0000 7f49cb7fe640 1 -- 172.21.15.7:0/9144323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558be1d7f010 msgr2=0x558be1d7f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.573+0000 7f49cb7fe640 1 --2- 172.21.15.7:0/9144323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558be1d7f010 0x558be1d7f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.573+0000 7f49cb7fe640 1 -- 172.21.15.7:0/9144323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be1d7fa00 msgr2=0x558be1d84500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.573+0000 7f49cb7fe640 1 --2- 172.21.15.7:0/9144323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be1d7fa00 0x558be1d84500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.573+0000 7f49cb7fe640 1 -- 172.21.15.7:0/9144323 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558be1c9c9b0 con 0x558be1d84b70 2024-06-10T01:18:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.573+0000 7f49d0cec640 1 --2- 172.21.15.7:0/9144323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558be1d7f010 0x558be1d7f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:18:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.573+0000 7f49d0cec640 1 --2- 172.21.15.7:0/2070118867 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f49b40a2b80 0x7f49b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.573+0000 7f49c97fa640 1 -- 172.21.15.7:0/2070118867 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f49bc0ef810 con 0x7f49cc102480 2024-06-10T01:18:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.573+0000 7f49cb7fe640 1 --2- 172.21.15.7:0/9144323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558be1d84b70 0x558be1d87000 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f49cc069d20 tx=0x7f49c00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f07c357d135d71a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.573+0000 7f49b27fc640 1 -- 172.21.15.7:0/9144323 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c0004310 con 0x558be1d84b70 2024-06-10T01:18:50.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.573+0000 7f49b27fc640 1 -- 172.21.15.7:0/9144323 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49c00044b0 con 0x558be1d84b70 2024-06-10T01:18:50.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.573+0000 7f49b27fc640 1 -- 172.21.15.7:0/9144323 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c00035f0 con 0x558be1d84b70 2024-06-10T01:18:50.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.574+0000 7f49d545a740 1 -- 172.21.15.7:0/9144323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558be1d84b70 msgr2=0x558be1d87000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:50.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.574+0000 7f49d0cec640 1 --2- 172.21.15.7:0/2070118867 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f49b40a2b80 0x7f49b40a5040 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x558be1d80260 tx=0x7f49c4004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:50.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.574+0000 7f49d545a740 1 --2- 172.21.15.7:0/9144323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558be1d84b70 0x558be1d87000 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f49cc069d20 tx=0x7f49c00118d0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.574+0000 7f49d545a740 1 -- 172.21.15.7:0/9144323 shutdown_connections 2024-06-10T01:18:50.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.574+0000 7f49d545a740 1 --2- 172.21.15.7:0/9144323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558be1d84b70 0x558be1d87000 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.574+0000 7f49d545a740 1 --2- 172.21.15.7:0/9144323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be1d7fa00 0x558be1d84500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.574+0000 7f49d545a740 1 --2- 172.21.15.7:0/9144323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558be1d7f010 0x558be1d7f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.574+0000 7f49d545a740 1 -- 172.21.15.7:0/9144323 >> 172.21.15.7:0/9144323 conn(0x558be1d7a680 msgr2=0x558be1d7aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:50.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.574+0000 7f49d545a740 1 -- 172.21.15.7:0/9144323 shutdown_connections 2024-06-10T01:18:50.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.574+0000 7f49d545a740 1 -- 172.21.15.7:0/9144323 wait complete. 2024-06-10T01:18:50.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.574+0000 7f49d545a740 1 -- start start 2024-06-10T01:18:50.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.575+0000 7f49d545a740 1 Processor -- start 2024-06-10T01:18:50.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.575+0000 7f49d545a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558be1d7f010 0x558be1d89220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:50.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.575+0000 7f49d545a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be1d7fa00 0x558be1d89760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:50.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.575+0000 7f49d545a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558be1d84b70 0x558be1d907a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:50.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.575+0000 7f49d545a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558be1d87b20 con 0x558be1d7f010 2024-06-10T01:18:50.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.575+0000 7f49d545a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558be1c6bae0 con 0x558be1d7fa00 2024-06-10T01:18:50.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.575+0000 7f49d545a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558be1d87ca0 con 0x558be1d84b70 2024-06-10T01:18:50.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.575+0000 7f49d545a740 10 client.0 _ll_register_callbacks cb 0x558be1ca49b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:18:50.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.575+0000 7f49cbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558be1d7f010 0x558be1d89220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:50.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.575+0000 7f49d0cec640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558be1d84b70 0x558be1d907a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:50.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.575+0000 7f49cbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558be1d7f010 0x558be1d89220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58774/0 (socket says 172.21.15.7:58774) 2024-06-10T01:18:50.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.575+0000 7f49cb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be1d7fa00 0x558be1d89760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:50.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.575+0000 7f49cbfff640 1 -- 172.21.15.7:0/3469173525 learned_addr learned my addr 172.21.15.7:0/3469173525 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:50.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.575+0000 7f49cbfff640 1 -- 172.21.15.7:0/3469173525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558be1d84b70 msgr2=0x558be1d907a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:50.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.575+0000 7f49cbfff640 1 --2- 172.21.15.7:0/3469173525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558be1d84b70 0x558be1d907a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.575+0000 7f49cbfff640 1 -- 172.21.15.7:0/3469173525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be1d7fa00 msgr2=0x558be1d89760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:50.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.575+0000 7f49cbfff640 1 --2- 172.21.15.7:0/3469173525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be1d7fa00 0x558be1d89760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.575+0000 7f49cbfff640 1 -- 172.21.15.7:0/3469173525 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558be19f6210 con 0x558be1d7f010 2024-06-10T01:18:50.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.576+0000 7f49cbfff640 1 --2- 172.21.15.7:0/3469173525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558be1d7f010 0x558be1d89220 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f49bc009a20 tx=0x7f49bc0b7930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee4f60698161af6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:50.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.576+0000 7f49ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:18:50.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.576+0000 7f49ab7fe640 1 -- 172.21.15.7:0/3469173525 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc049070 con 0x558be1d7f010 2024-06-10T01:18:50.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.576+0000 7f49ab7fe640 1 -- 172.21.15.7:0/3469173525 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f49bc0b9ff0 con 0x558be1d7f010 2024-06-10T01:18:50.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.576+0000 7f49d545a740 1 -- 172.21.15.7:0/3469173525 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49c00044b0 con 0x558be1d7f010 2024-06-10T01:18:50.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.576+0000 7f49ab7fe640 1 -- 172.21.15.7:0/3469173525 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f49bc0bb650 con 0x558be1d7f010 2024-06-10T01:18:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.576+0000 7f49d545a740 10 client.18669 fetch_fsmap learned FSMap version 40 2024-06-10T01:18:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.576+0000 7f49d545a740 1 -- 172.21.15.7:0/3469173525 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x558be1ac2620 con 0x558be1d7f010 2024-06-10T01:18:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.576+0000 7f49ab7fe640 1 -- 172.21.15.7:0/3469173525 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49c00044b0 con 0x558be1d7f010 2024-06-10T01:18:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.576+0000 7f49ab7fe640 1 -- 172.21.15.7:0/3469173525 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f49bc003800 con 0x558be1d7f010 2024-06-10T01:18:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.576+0000 7f49d545a740 10 client.18669 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:18:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.576+0000 7f49d545a740 10 client.18669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:18:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.576+0000 7f49d545a740 20 client.18669 populate_metadata read hostname 'smithi007' 2024-06-10T01:18:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.576+0000 7f49d545a740 1 --2- 172.21.15.7:0/3469173525 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558be1d95020 0x558be1d97490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.576+0000 7f49d545a740 4 client.18669 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:18:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.576+0000 7f49d545a740 1 -- 172.21.15.7:0/3469173525 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558be1cf5f10 con 0x558be1d95020 2024-06-10T01:18:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.577+0000 7f49cb7fe640 1 --2- 172.21.15.7:0/3469173525 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558be1d95020 0x558be1d97490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.577+0000 7f49cb7fe640 1 --2- 172.21.15.7:0/3469173525 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558be1d95020 0x558be1d97490 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97d53078fb705543 server_cookie=26d458ba62c97ee4 in_seq=0 out_seq=0 2024-06-10T01:18:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.577+0000 7f49ab7fe640 10 client.18669 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.578+0000 7f49ab7fe640 1 -- 172.21.15.7:0/3469173525 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558be1cf5f10 con 0x558be1d95020 2024-06-10T01:18:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.578+0000 7f49ab7fe640 10 client.18669 handle_command_reply: tid=0 2024-06-10T01:18:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.578+0000 7f49d545a740 1 client.18669 shutdown 2024-06-10T01:18:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.578+0000 7f49d545a740 1 -- 172.21.15.7:0/3469173525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558be1d7f010 msgr2=0x558be1d89220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.578+0000 7f49d545a740 1 --2- 172.21.15.7:0/3469173525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558be1d7f010 0x558be1d89220 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f49bc009a20 tx=0x7f49bc0b7930 comp rx=0 tx=0).stop 2024-06-10T01:18:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.578+0000 7f49d545a740 1 -- 172.21.15.7:0/3469173525 shutdown_connections 2024-06-10T01:18:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.578+0000 7f49d545a740 1 --2- 172.21.15.7:0/3469173525 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558be1d95020 0x558be1d97490 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.578+0000 7f49d545a740 1 --2- 172.21.15.7:0/3469173525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558be1d84b70 0x558be1d907a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.578+0000 7f49d545a740 1 --2- 172.21.15.7:0/3469173525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be1d7fa00 0x558be1d89760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.578+0000 7f49d545a740 1 --2- 172.21.15.7:0/3469173525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558be1d7f010 0x558be1d89220 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.578+0000 7f49ab7fe640 0 client.18669 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.578+0000 7f49d545a740 1 -- 172.21.15.7:0/3469173525 >> 172.21.15.7:0/3469173525 conn(0x558be1d7a680 msgr2=0x558be1d797e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.578+0000 7f49d545a740 1 -- 172.21.15.7:0/3469173525 shutdown_connections 2024-06-10T01:18:50.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.578+0000 7f49d545a740 1 -- 172.21.15.7:0/3469173525 wait complete. 2024-06-10T01:18:50.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.578+0000 7f49d545a740 20 client.18669 trim_cache size 0 max 16384 2024-06-10T01:18:50.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.595+0000 7f49d545a740 1 Processor -- start 2024-06-10T01:18:50.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.595+0000 7f49d545a740 1 -- start start 2024-06-10T01:18:50.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.595+0000 7f49d545a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558be1d7f010 0x558be1d84210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:50.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.595+0000 7f49d545a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be1d7fa00 0x558be1d90760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:50.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.595+0000 7f49d545a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558be1d84b70 0x558be1d7d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:50.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.595+0000 7f49d545a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc1acdb0 con 0x558be1d84b70 2024-06-10T01:18:50.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.595+0000 7f49d545a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc1aa680 con 0x558be1d7fa00 2024-06-10T01:18:50.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.595+0000 7f49d545a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc1a84d0 con 0x558be1d7f010 2024-06-10T01:18:50.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.595+0000 7f49d0cec640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558be1d84b70 0x558be1d7d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:50.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.595+0000 7f49d0cec640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558be1d84b70 0x558be1d7d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58786/0 (socket says 172.21.15.7:58786) 2024-06-10T01:18:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.595+0000 7f49cb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be1d7fa00 0x558be1d90760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.595+0000 7f49d0cec640 1 -- 172.21.15.7:0/4030617495 learned_addr learned my addr 172.21.15.7:0/4030617495 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.595+0000 7f49cbfff640 1 --2- 172.21.15.7:0/4030617495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558be1d7f010 0x558be1d84210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.596+0000 7f49cb7fe640 1 -- 172.21.15.7:0/4030617495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558be1d7f010 msgr2=0x558be1d84210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.596+0000 7f49cb7fe640 1 --2- 172.21.15.7:0/4030617495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558be1d7f010 0x558be1d84210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.596+0000 7f49cb7fe640 1 -- 172.21.15.7:0/4030617495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558be1d84b70 msgr2=0x558be1d7d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.596+0000 7f49cb7fe640 1 --2- 172.21.15.7:0/4030617495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558be1d84b70 0x558be1d7d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.596+0000 7f49cb7fe640 1 -- 172.21.15.7:0/4030617495 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558be1d7a4b0 con 0x558be1d7fa00 2024-06-10T01:18:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.596+0000 7f49d0cec640 1 --2- 172.21.15.7:0/4030617495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558be1d84b70 0x558be1d7d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:18:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.596+0000 7f49cb7fe640 1 --2- 172.21.15.7:0/4030617495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be1d7fa00 0x558be1d90760 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f49c0011dd0 tx=0x7f49c0005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc01aa5b52ab72b3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.596+0000 7f49b1ffb640 1 -- 172.21.15.7:0/4030617495 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c0033070 con 0x558be1d7fa00 2024-06-10T01:18:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.596+0000 7f49b1ffb640 1 -- 172.21.15.7:0/4030617495 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49c0009b30 con 0x558be1d7fa00 2024-06-10T01:18:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.596+0000 7f49b1ffb640 1 -- 172.21.15.7:0/4030617495 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c002e3e0 con 0x558be1d7fa00 2024-06-10T01:18:50.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.597+0000 7f49d545a740 1 -- 172.21.15.7:0/4030617495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be1d7fa00 msgr2=0x558be1d90760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:50.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.597+0000 7f49d545a740 1 --2- 172.21.15.7:0/4030617495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be1d7fa00 0x558be1d90760 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f49c0011dd0 tx=0x7f49c0005d30 comp rx=0 tx=0).stop 2024-06-10T01:18:50.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.597+0000 7f49d545a740 1 -- 172.21.15.7:0/4030617495 shutdown_connections 2024-06-10T01:18:50.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.597+0000 7f49d545a740 1 --2- 172.21.15.7:0/4030617495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558be1d84b70 0x558be1d7d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.597+0000 7f49d545a740 1 --2- 172.21.15.7:0/4030617495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be1d7fa00 0x558be1d90760 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.597+0000 7f49d545a740 1 --2- 172.21.15.7:0/4030617495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558be1d7f010 0x558be1d84210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.597+0000 7f49d545a740 1 -- 172.21.15.7:0/4030617495 >> 172.21.15.7:0/4030617495 conn(0x558be1d7a680 msgr2=0x558be1d7aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:50.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.597+0000 7f49d545a740 1 -- 172.21.15.7:0/4030617495 shutdown_connections 2024-06-10T01:18:50.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.597+0000 7f49d545a740 1 -- 172.21.15.7:0/4030617495 wait complete. 2024-06-10T01:18:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.597+0000 7f49d545a740 1 -- start start 2024-06-10T01:18:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.597+0000 7f49d545a740 1 Processor -- start 2024-06-10T01:18:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.598+0000 7f49d545a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be1d7f010 0x558be1d8d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.598+0000 7f49d545a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558be1d7fa00 0x558be1d8d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.598+0000 7f49d545a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558be1d84b70 0x558be1dfdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.598+0000 7f49d545a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc1a8b40 con 0x558be1d84b70 2024-06-10T01:18:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.598+0000 7f49d545a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc1a9fb0 con 0x558be1d7f010 2024-06-10T01:18:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.598+0000 7f49d545a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc1a9210 con 0x558be1d7fa00 2024-06-10T01:18:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.598+0000 7f49d545a740 10 client.0 _ll_register_callbacks cb 0x7f49bc003e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:18:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.598+0000 7f49d0cec640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558be1d84b70 0x558be1dfdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.598+0000 7f49d0cec640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558be1d84b70 0x558be1dfdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58790/0 (socket says 172.21.15.7:58790) 2024-06-10T01:18:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.598+0000 7f49cb7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558be1d7fa00 0x558be1d8d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.598+0000 7f49d0cec640 1 -- 172.21.15.7:0/1856988335 learned_addr learned my addr 172.21.15.7:0/1856988335 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:18:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.598+0000 7f49cbfff640 1 --2- 172.21.15.7:0/1856988335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be1d7f010 0x558be1d8d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.598+0000 7f49d0cec640 1 -- 172.21.15.7:0/1856988335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558be1d7fa00 msgr2=0x558be1d8d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.598+0000 7f49d0cec640 1 --2- 172.21.15.7:0/1856988335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558be1d7fa00 0x558be1d8d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.598+0000 7f49d0cec640 1 -- 172.21.15.7:0/1856988335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be1d7f010 msgr2=0x558be1d8d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:18:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.598+0000 7f49d0cec640 1 --2- 172.21.15.7:0/1856988335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be1d7f010 0x558be1d8d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.598+0000 7f49d0cec640 1 -- 172.21.15.7:0/1856988335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558be1d909d0 con 0x558be1d84b70 2024-06-10T01:18:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.598+0000 7f49cb7fe640 1 --2- 172.21.15.7:0/1856988335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558be1d7fa00 0x558be1d8d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:18:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.598+0000 7f49d0cec640 1 --2- 172.21.15.7:0/1856988335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558be1d84b70 0x558be1dfdd60 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f49c401dc60 tx=0x7f49c4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad87b7dc9c53ba6d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:18:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.598+0000 7f49ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:18:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.598+0000 7f49ab7fe640 1 -- 172.21.15.7:0/1856988335 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c402b440 con 0x558be1d84b70 2024-06-10T01:18:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.599+0000 7f49ab7fe640 1 -- 172.21.15.7:0/1856988335 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f49c402b5e0 con 0x558be1d84b70 2024-06-10T01:18:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.599+0000 7f49d545a740 1 -- 172.21.15.7:0/1856988335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49c0009b30 con 0x558be1d84b70 2024-06-10T01:18:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.599+0000 7f49ab7fe640 1 -- 172.21.15.7:0/1856988335 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f49c4039050 con 0x558be1d84b70 2024-06-10T01:18:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.599+0000 7f49d545a740 10 client.18675 fetch_fsmap learned FSMap version 40 2024-06-10T01:18:50.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.599+0000 7f49d545a740 1 -- 172.21.15.7:0/1856988335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x558be1dff350 con 0x558be1d84b70 2024-06-10T01:18:50.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.599+0000 7f49ab7fe640 1 -- 172.21.15.7:0/1856988335 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49c0009b30 con 0x558be1d84b70 2024-06-10T01:18:50.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.599+0000 7f49ab7fe640 1 -- 172.21.15.7:0/1856988335 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f49c401c900 con 0x558be1d84b70 2024-06-10T01:18:50.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.599+0000 7f49d545a740 10 client.18675 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:18:50.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.599+0000 7f49d545a740 10 client.18675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:18:50.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.599+0000 7f49d545a740 20 client.18675 populate_metadata read hostname 'smithi007' 2024-06-10T01:18:50.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.599+0000 7f49d545a740 1 --2- 172.21.15.7:0/1856988335 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558be1dffe00 0x558be1e02220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:18:50.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.599+0000 7f49d545a740 4 client.18675 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:18:50.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.599+0000 7f49d545a740 1 -- 172.21.15.7:0/1856988335 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558be1cefc20 con 0x558be1dffe00 2024-06-10T01:18:50.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.599+0000 7f49cbfff640 1 --2- 172.21.15.7:0/1856988335 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558be1dffe00 0x558be1e02220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:18:50.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.600+0000 7f49ab7fe640 10 client.18675 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:50.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.600+0000 7f49cbfff640 1 --2- 172.21.15.7:0/1856988335 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558be1dffe00 0x558be1e02220 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8a2cac18cd25a9a server_cookie=a0d5157b87995d in_seq=0 out_seq=0 2024-06-10T01:18:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.712+0000 7f49ab7fe640 1 -- 172.21.15.7:0/1856988335 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558be1cefc20 con 0x558be1dffe00 2024-06-10T01:18:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.712+0000 7f49ab7fe640 10 client.18675 handle_command_reply: tid=0 2024-06-10T01:18:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.712+0000 7f49d545a740 1 client.18675 shutdown 2024-06-10T01:18:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.712+0000 7f49d545a740 1 -- 172.21.15.7:0/1856988335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558be1d84b70 msgr2=0x558be1dfdd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.712+0000 7f49d545a740 1 --2- 172.21.15.7:0/1856988335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558be1d84b70 0x558be1dfdd60 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f49c401dc60 tx=0x7f49c4004ab0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.712+0000 7f49d545a740 1 -- 172.21.15.7:0/1856988335 shutdown_connections 2024-06-10T01:18:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.712+0000 7f49d545a740 1 --2- 172.21.15.7:0/1856988335 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558be1dffe00 0x558be1e02220 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.712+0000 7f49d545a740 1 --2- 172.21.15.7:0/1856988335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558be1d84b70 0x558be1dfdd60 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.712+0000 7f49d545a740 1 --2- 172.21.15.7:0/1856988335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558be1d7fa00 0x558be1d8d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.712+0000 7f49d545a740 1 --2- 172.21.15.7:0/1856988335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be1d7f010 0x558be1d8d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.712+0000 7f49ab7fe640 0 client.18675 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:18:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.712+0000 7f49d545a740 1 -- 172.21.15.7:0/1856988335 >> 172.21.15.7:0/1856988335 conn(0x558be1d7a680 msgr2=0x558be1d83de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.712+0000 7f49d545a740 1 -- 172.21.15.7:0/1856988335 shutdown_connections 2024-06-10T01:18:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.712+0000 7f49d545a740 1 -- 172.21.15.7:0/1856988335 wait complete. 2024-06-10T01:18:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.712+0000 7f49d545a740 20 client.18675 trim_cache size 0 max 16384 2024-06-10T01:18:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:18:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:18:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:18:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.713+0000 7f49d274f640 1 -- 172.21.15.7:0/2070118867 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f49b40a2b80 msgr2=0x7f49b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.713+0000 7f49d274f640 1 --2- 172.21.15.7:0/2070118867 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f49b40a2b80 0x7f49b40a5040 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x558be1d80260 tx=0x7f49c4004a40 comp rx=0 tx=0).stop 2024-06-10T01:18:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.713+0000 7f49d274f640 1 -- 172.21.15.7:0/2070118867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49cc102480 msgr2=0x7f49cc197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:18:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.713+0000 7f49d274f640 1 --2- 172.21.15.7:0/2070118867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49cc102480 0x7f49cc197db0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f49bc009230 tx=0x7f49bc0042f0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.713+0000 7f49d274f640 1 -- 172.21.15.7:0/2070118867 shutdown_connections 2024-06-10T01:18:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.713+0000 7f49d274f640 1 --2- 172.21.15.7:0/2070118867 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f49b40a2b80 0x7f49b40a5040 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.713+0000 7f49d274f640 1 --2- 172.21.15.7:0/2070118867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49cc1076e0 0x7f49cc19f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.713+0000 7f49d274f640 1 --2- 172.21.15.7:0/2070118867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49cc104db0 0x7f49cc1982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.713+0000 7f49d274f640 1 --2- 172.21.15.7:0/2070118867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49cc102480 0x7f49cc197db0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:18:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.713+0000 7f49d274f640 1 -- 172.21.15.7:0/2070118867 >> 172.21.15.7:0/2070118867 conn(0x7f49cc0fbe70 msgr2=0x7f49cc102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:18:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.713+0000 7f49d274f640 1 -- 172.21.15.7:0/2070118867 shutdown_connections 2024-06-10T01:18:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:18:50.713+0000 7f49d274f640 1 -- 172.21.15.7:0/2070118867 wait complete. 2024-06-10T01:18:50.757 INFO:tasks.workunit.client.0.smithi007.stdout: 1 936091 7.25 MB/sec execute 571 sec latency 55.688 ms 2024-06-10T01:18:51.753 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:51 smithi175 ceph-mon[36636]: pgmap v817: 129 pgs: 129 active+clean; 141 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 1.7 MiB/s wr, 308 op/s 2024-06-10T01:18:51.756 INFO:tasks.workunit.client.0.smithi007.stdout: 1 938447 7.26 MB/sec execute 572 sec latency 11.520 ms 2024-06-10T01:18:51.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:51 smithi007 ceph-mon[32398]: pgmap v817: 129 pgs: 129 active+clean; 141 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 1.7 MiB/s wr, 308 op/s 2024-06-10T01:18:51.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:51 smithi078 ceph-mon[36683]: pgmap v817: 129 pgs: 129 active+clean; 141 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 1.7 MiB/s wr, 308 op/s 2024-06-10T01:18:52.756 INFO:tasks.workunit.client.0.smithi007.stdout: 1 940497 7.26 MB/sec execute 573 sec latency 19.655 ms 2024-06-10T01:18:53.756 INFO:tasks.workunit.client.0.smithi007.stdout: 1 941986 7.27 MB/sec execute 574 sec latency 52.162 ms 2024-06-10T01:18:53.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:53 smithi007 ceph-mon[32398]: pgmap v818: 129 pgs: 129 active+clean; 137 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 245 op/s 2024-06-10T01:18:53.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:53 smithi078 ceph-mon[36683]: pgmap v818: 129 pgs: 129 active+clean; 137 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 245 op/s 2024-06-10T01:18:53.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:53 smithi175 ceph-mon[36636]: pgmap v818: 129 pgs: 129 active+clean; 137 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 245 op/s 2024-06-10T01:18:54.756 INFO:tasks.workunit.client.0.smithi007.stdout: 1 943992 7.27 MB/sec execute 575 sec latency 17.306 ms 2024-06-10T01:18:55.758 INFO:tasks.workunit.client.0.smithi007.stdout: 1 945648 7.28 MB/sec execute 576 sec latency 10.853 ms 2024-06-10T01:18:55.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:55 smithi007 ceph-mon[32398]: pgmap v819: 129 pgs: 129 active+clean; 138 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 288 op/s 2024-06-10T01:18:55.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:55 smithi078 ceph-mon[36683]: pgmap v819: 129 pgs: 129 active+clean; 138 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 288 op/s 2024-06-10T01:18:55.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:55 smithi175 ceph-mon[36636]: pgmap v819: 129 pgs: 129 active+clean; 138 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 288 op/s 2024-06-10T01:18:56.756 INFO:tasks.workunit.client.0.smithi007.stdout: 1 947707 7.28 MB/sec execute 577 sec latency 16.836 ms 2024-06-10T01:18:57.757 INFO:tasks.workunit.client.0.smithi007.stdout: 1 949245 7.29 MB/sec execute 578 sec latency 64.911 ms 2024-06-10T01:18:57.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:57 smithi007 ceph-mon[32398]: pgmap v820: 129 pgs: 129 active+clean; 139 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 284 op/s 2024-06-10T01:18:57.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:57 smithi078 ceph-mon[36683]: pgmap v820: 129 pgs: 129 active+clean; 139 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 284 op/s 2024-06-10T01:18:57.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:57 smithi175 ceph-mon[36636]: pgmap v820: 129 pgs: 129 active+clean; 139 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 284 op/s 2024-06-10T01:18:58.176 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:18:58.499 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:18:58.757 INFO:tasks.workunit.client.0.smithi007.stdout: 1 951070 7.29 MB/sec execute 579 sec latency 33.533 ms 2024-06-10T01:18:59.757 INFO:tasks.workunit.client.0.smithi007.stdout: 1 952753 7.29 MB/sec execute 580 sec latency 10.772 ms 2024-06-10T01:18:59.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:18:59 smithi007 ceph-mon[32398]: pgmap v821: 129 pgs: 129 active+clean; 139 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 1.9 MiB/s wr, 266 op/s 2024-06-10T01:18:59.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:18:59 smithi078 ceph-mon[36683]: pgmap v821: 129 pgs: 129 active+clean; 139 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 1.9 MiB/s wr, 266 op/s 2024-06-10T01:18:59.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:18:59 smithi175 ceph-mon[36636]: pgmap v821: 129 pgs: 129 active+clean; 139 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 1.9 MiB/s wr, 266 op/s 2024-06-10T01:19:00.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.404+0000 7f510aadb700 1 -- 172.21.15.7:0/3631385558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51041084f0 msgr2=0x7f51041088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:00.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.404+0000 7f510aadb700 1 --2- 172.21.15.7:0/3631385558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51041084f0 0x7f51041088d0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009990 tx=0x7f50f8009ca0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.404+0000 7f510aadb700 1 -- 172.21.15.7:0/3631385558 shutdown_connections 2024-06-10T01:19:00.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.404+0000 7f510aadb700 1 --2- 172.21.15.7:0/3631385558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f510406b3c0 0x7f510410f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.404+0000 7f510aadb700 1 --2- 172.21.15.7:0/3631385558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51041084f0 0x7f51041088d0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.404+0000 7f510aadb700 1 --2- 172.21.15.7:0/3631385558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f510406aa00 0x7f510406ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.404+0000 7f510aadb700 1 -- 172.21.15.7:0/3631385558 >> 172.21.15.7:0/3631385558 conn(0x7f5104078300 msgr2=0x7f5104078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.405+0000 7f510aadb700 1 -- 172.21.15.7:0/3631385558 shutdown_connections 2024-06-10T01:19:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.405+0000 7f510aadb700 1 -- 172.21.15.7:0/3631385558 wait complete. 2024-06-10T01:19:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.405+0000 7f510aadb700 1 Processor -- start 2024-06-10T01:19:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.405+0000 7f510aadb700 1 -- start start 2024-06-10T01:19:00.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.406+0000 7f510aadb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f510406aa00 0x7f510419f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:00.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.406+0000 7f510aadb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f510406b3c0 0x7f510419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:00.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.406+0000 7f510aadb700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51041084f0 0x7f51041a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:00.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.406+0000 7f510aadb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f51041105c0 con 0x7f510406aa00 2024-06-10T01:19:00.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.406+0000 7f510aadb700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5104110440 con 0x7f510406b3c0 2024-06-10T01:19:00.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.406+0000 7f510aadb700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5104110740 con 0x7f51041084f0 2024-06-10T01:19:00.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.406+0000 7f5108877700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f510406aa00 0x7f510419f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:00.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.406+0000 7f5108877700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f510406aa00 0x7f510419f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37822/0 (socket says 172.21.15.7:37822) 2024-06-10T01:19:00.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.406+0000 7f5109078700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51041084f0 0x7f51041a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:00.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.406+0000 7f5108877700 1 -- 172.21.15.7:0/500150432 learned_addr learned my addr 172.21.15.7:0/500150432 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:00.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.406+0000 7f5103fff700 1 --2- 172.21.15.7:0/500150432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f510406b3c0 0x7f510419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:00.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.406+0000 7f5108877700 1 -- 172.21.15.7:0/500150432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51041084f0 msgr2=0x7f51041a3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:00.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.406+0000 7f5108877700 1 --2- 172.21.15.7:0/500150432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51041084f0 0x7f51041a3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.406+0000 7f5108877700 1 -- 172.21.15.7:0/500150432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f510406b3c0 msgr2=0x7f510419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:00.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.406+0000 7f5108877700 1 --2- 172.21.15.7:0/500150432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f510406b3c0 0x7f510419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.406+0000 7f5108877700 1 -- 172.21.15.7:0/500150432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51041a4250 con 0x7f510406aa00 2024-06-10T01:19:00.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.406+0000 7f5109078700 1 --2- 172.21.15.7:0/500150432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51041084f0 0x7f51041a3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:00.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.407+0000 7f5108877700 1 --2- 172.21.15.7:0/500150432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f510406aa00 0x7f510419f240 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f50f8010b20 tx=0x7f50f8010b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:00.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.407+0000 7f5101ffb700 1 -- 172.21.15.7:0/500150432 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f80052c0 con 0x7f510406aa00 2024-06-10T01:19:00.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.407+0000 7f510aadb700 1 -- 172.21.15.7:0/500150432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51041a44e0 con 0x7f510406aa00 2024-06-10T01:19:00.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.407+0000 7f5101ffb700 1 -- 172.21.15.7:0/500150432 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50f800bcd0 con 0x7f510406aa00 2024-06-10T01:19:00.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.407+0000 7f5101ffb700 1 -- 172.21.15.7:0/500150432 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f8018890 con 0x7f510406aa00 2024-06-10T01:19:00.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.407+0000 7f510aadb700 1 -- 172.21.15.7:0/500150432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51041abdc0 con 0x7f510406aa00 2024-06-10T01:19:00.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.409+0000 7f5101ffb700 1 -- 172.21.15.7:0/500150432 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f50f8028060 con 0x7f510406aa00 2024-06-10T01:19:00.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.410+0000 7f5101ffb700 1 --2- 172.21.15.7:0/500150432 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f50f40a2c20 0x7f50f40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:00.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.410+0000 7f5103fff700 1 --2- 172.21.15.7:0/500150432 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f50f40a2c20 0x7f50f40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:00.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.410+0000 7f5101ffb700 1 -- 172.21.15.7:0/500150432 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f50f80d06f0 con 0x7f510406aa00 2024-06-10T01:19:00.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.411+0000 7f5103fff700 1 --2- 172.21.15.7:0/500150432 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f50f40a2c20 0x7f50f40a50e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5104078970 tx=0x7f50f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:00.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.411+0000 7f511535eb80 1 Processor -- start 2024-06-10T01:19:00.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.411+0000 7f511535eb80 1 -- start start 2024-06-10T01:19:00.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.411+0000 7f511535eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361695bf60 0x56361695c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:00.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.411+0000 7f511535eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56361695c880 0x563616956260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:00.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.411+0000 7f511535eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636169567a0 0x563616956bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:00.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.411+0000 7f511535eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563616848f60 con 0x56361695bf60 2024-06-10T01:19:00.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.411+0000 7f511535eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5636168a7850 con 0x56361695c880 2024-06-10T01:19:00.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.411+0000 7f511535eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56361680f460 con 0x5636169567a0 2024-06-10T01:19:00.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.411+0000 7f5108877700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361695bf60 0x56361695c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:00.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.411+0000 7f5108877700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361695bf60 0x56361695c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37824/0 (socket says 172.21.15.7:37824) 2024-06-10T01:19:00.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.411+0000 7f5108877700 1 -- 172.21.15.7:0/2345580673 learned_addr learned my addr 172.21.15.7:0/2345580673 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:00.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.411+0000 7f5103fff700 1 --2- 172.21.15.7:0/2345580673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56361695c880 0x563616956260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:00.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.411+0000 7f5109078700 1 --2- 172.21.15.7:0/2345580673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636169567a0 0x563616956bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:00.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.412+0000 7f5103fff700 1 -- 172.21.15.7:0/2345580673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636169567a0 msgr2=0x563616956bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:00.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.412+0000 7f5103fff700 1 --2- 172.21.15.7:0/2345580673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636169567a0 0x563616956bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.412+0000 7f5103fff700 1 -- 172.21.15.7:0/2345580673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361695bf60 msgr2=0x56361695c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:00.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.412+0000 7f5103fff700 1 --2- 172.21.15.7:0/2345580673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361695bf60 0x56361695c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.412+0000 7f5103fff700 1 -- 172.21.15.7:0/2345580673 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636167f68d0 con 0x56361695c880 2024-06-10T01:19:00.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.412+0000 7f5108877700 1 --2- 172.21.15.7:0/2345580673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361695bf60 0x56361695c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:00.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.412+0000 7f5103fff700 1 --2- 172.21.15.7:0/2345580673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56361695c880 0x563616956260 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f50f0011c90 tx=0x7f50f0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21a3037523e4849f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:00.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.412+0000 7f50eeffd700 1 -- 172.21.15.7:0/2345580673 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0004230 con 0x56361695c880 2024-06-10T01:19:00.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.412+0000 7f50eeffd700 1 -- 172.21.15.7:0/2345580673 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50f000ed20 con 0x56361695c880 2024-06-10T01:19:00.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.412+0000 7f50eeffd700 1 -- 172.21.15.7:0/2345580673 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0026470 con 0x56361695c880 2024-06-10T01:19:00.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.413+0000 7f511535eb80 1 -- 172.21.15.7:0/2345580673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56361695c880 msgr2=0x563616956260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:00.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.413+0000 7f511535eb80 1 --2- 172.21.15.7:0/2345580673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56361695c880 0x563616956260 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f50f0011c90 tx=0x7f50f0014000 comp rx=0 tx=0).stop 2024-06-10T01:19:00.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.413+0000 7f511535eb80 1 -- 172.21.15.7:0/2345580673 shutdown_connections 2024-06-10T01:19:00.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.413+0000 7f511535eb80 1 --2- 172.21.15.7:0/2345580673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636169567a0 0x563616956bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.413+0000 7f511535eb80 1 --2- 172.21.15.7:0/2345580673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361695bf60 0x56361695c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.413+0000 7f511535eb80 1 --2- 172.21.15.7:0/2345580673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56361695c880 0x563616956260 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.413+0000 7f511535eb80 1 -- 172.21.15.7:0/2345580673 >> 172.21.15.7:0/2345580673 conn(0x563616883080 msgr2=0x563616951220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:00.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.413+0000 7f511535eb80 1 -- 172.21.15.7:0/2345580673 shutdown_connections 2024-06-10T01:19:00.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.413+0000 7f511535eb80 1 -- 172.21.15.7:0/2345580673 wait complete. 2024-06-10T01:19:00.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.414+0000 7f511535eb80 1 -- start start 2024-06-10T01:19:00.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.414+0000 7f511535eb80 1 Processor -- start 2024-06-10T01:19:00.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.414+0000 7f511535eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636169567a0 0x563616962650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:00.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.414+0000 7f511535eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56361695bf60 0x56361695f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:00.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.414+0000 7f511535eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361695c880 0x56361695fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:00.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.414+0000 7f511535eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563616847fc0 con 0x56361695c880 2024-06-10T01:19:00.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.414+0000 7f511535eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5636168a3810 con 0x5636169567a0 2024-06-10T01:19:00.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.414+0000 7f511535eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563616963260 con 0x56361695bf60 2024-06-10T01:19:00.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.414+0000 7f511535eb80 10 client.0 _ll_register_callbacks cb 0x56361683d5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:19:00.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.414+0000 7f5109078700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361695c880 0x56361695fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:00.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.414+0000 7f5108877700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636169567a0 0x563616962650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:00.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.414+0000 7f5109078700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361695c880 0x56361695fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37838/0 (socket says 172.21.15.7:37838) 2024-06-10T01:19:00.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.414+0000 7f5109078700 1 -- 172.21.15.7:0/3961998524 learned_addr learned my addr 172.21.15.7:0/3961998524 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:00.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.414+0000 7f5103fff700 1 --2- 172.21.15.7:0/3961998524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56361695bf60 0x56361695f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:00.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.415+0000 7f5108877700 1 -- 172.21.15.7:0/3961998524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56361695bf60 msgr2=0x56361695f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:00.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.415+0000 7f5108877700 1 --2- 172.21.15.7:0/3961998524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56361695bf60 0x56361695f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.415+0000 7f5108877700 1 -- 172.21.15.7:0/3961998524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361695c880 msgr2=0x56361695fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:00.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.415+0000 7f5108877700 1 --2- 172.21.15.7:0/3961998524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361695c880 0x56361695fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.415+0000 7f5108877700 1 -- 172.21.15.7:0/3961998524 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56361686d750 con 0x5636169567a0 2024-06-10T01:19:00.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.415+0000 7f5108877700 1 --2- 172.21.15.7:0/3961998524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636169567a0 0x563616962650 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f50f8099630 tx=0x7f50f809c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=384645b2b9fb7bc4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:00.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.415+0000 7f50e3fff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:19:00.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.415+0000 7f50e3fff700 1 -- 172.21.15.7:0/3961998524 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f800be80 con 0x5636169567a0 2024-06-10T01:19:00.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.415+0000 7f50e3fff700 1 -- 172.21.15.7:0/3961998524 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f50f80d0410 con 0x5636169567a0 2024-06-10T01:19:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.415+0000 7f511535eb80 1 -- 172.21.15.7:0/3961998524 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50f000ed20 con 0x5636169567a0 2024-06-10T01:19:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.416+0000 7f50e3fff700 1 -- 172.21.15.7:0/3961998524 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f50f809d360 con 0x5636169567a0 2024-06-10T01:19:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.416+0000 7f511535eb80 10 client.28001 fetch_fsmap learned FSMap version 40 2024-06-10T01:19:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.416+0000 7f511535eb80 1 -- 172.21.15.7:0/3961998524 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56361696da70 con 0x5636169567a0 2024-06-10T01:19:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.416+0000 7f50e3fff700 1 -- 172.21.15.7:0/3961998524 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50f000ed20 con 0x5636169567a0 2024-06-10T01:19:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.416+0000 7f50e3fff700 1 -- 172.21.15.7:0/3961998524 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f50f809cec0 con 0x5636169567a0 2024-06-10T01:19:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.416+0000 7f511535eb80 10 client.28001 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:19:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.416+0000 7f511535eb80 10 client.28001 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:19:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.416+0000 7f511535eb80 20 client.28001 populate_metadata read hostname 'smithi007' 2024-06-10T01:19:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.416+0000 7f511535eb80 1 --2- 172.21.15.7:0/3961998524 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563616970080 0x5636169724b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.416+0000 7f511535eb80 4 client.28001 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:19:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.416+0000 7f511535eb80 1 -- 172.21.15.7:0/3961998524 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636168b4d50 con 0x563616970080 2024-06-10T01:19:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.416+0000 7f5103fff700 1 --2- 172.21.15.7:0/3961998524 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563616970080 0x5636169724b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.417+0000 7f5103fff700 1 --2- 172.21.15.7:0/3961998524 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563616970080 0x5636169724b0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc6ea60d56f85b00 server_cookie=c971c30ff1b6f1bf in_seq=0 out_seq=0 2024-06-10T01:19:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.417+0000 7f50e3fff700 10 client.28001 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.417+0000 7f50e3fff700 1 -- 172.21.15.7:0/3961998524 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5636168b4d50 con 0x563616970080 2024-06-10T01:19:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.417+0000 7f50e3fff700 10 client.28001 handle_command_reply: tid=0 2024-06-10T01:19:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.417+0000 7f511535eb80 1 client.28001 shutdown 2024-06-10T01:19:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.418+0000 7f511535eb80 1 -- 172.21.15.7:0/3961998524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636169567a0 msgr2=0x563616962650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.418+0000 7f511535eb80 1 --2- 172.21.15.7:0/3961998524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636169567a0 0x563616962650 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f50f8099630 tx=0x7f50f809c820 comp rx=0 tx=0).stop 2024-06-10T01:19:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.418+0000 7f511535eb80 1 -- 172.21.15.7:0/3961998524 shutdown_connections 2024-06-10T01:19:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.418+0000 7f511535eb80 1 --2- 172.21.15.7:0/3961998524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361695c880 0x56361695fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.418+0000 7f511535eb80 1 --2- 172.21.15.7:0/3961998524 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563616970080 0x5636169724b0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.418+0000 7f511535eb80 1 --2- 172.21.15.7:0/3961998524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636169567a0 0x563616962650 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.418+0000 7f511535eb80 1 --2- 172.21.15.7:0/3961998524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56361695bf60 0x56361695f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.418+0000 7f511535eb80 1 -- 172.21.15.7:0/3961998524 >> 172.21.15.7:0/3961998524 conn(0x563616883080 msgr2=0x563616950700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.418+0000 7f50e3fff700 0 client.28001 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.418+0000 7f511535eb80 1 -- 172.21.15.7:0/3961998524 shutdown_connections 2024-06-10T01:19:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.418+0000 7f511535eb80 1 -- 172.21.15.7:0/3961998524 wait complete. 2024-06-10T01:19:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.418+0000 7f511535eb80 20 client.28001 trim_cache size 0 max 16384 2024-06-10T01:19:00.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.440+0000 7f511535eb80 1 Processor -- start 2024-06-10T01:19:00.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.440+0000 7f511535eb80 1 -- start start 2024-06-10T01:19:00.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.440+0000 7f511535eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636169567a0 0x563616956260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:00.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.440+0000 7f511535eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361695bf60 0x563616961890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:00.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.440+0000 7f511535eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56361695c880 0x563616950e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:00.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.440+0000 7f511535eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f51041b7100 con 0x56361695bf60 2024-06-10T01:19:00.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.440+0000 7f511535eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f51041b4990 con 0x56361695c880 2024-06-10T01:19:00.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.440+0000 7f511535eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f51041b27e0 con 0x5636169567a0 2024-06-10T01:19:00.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.440+0000 7f5103fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361695bf60 0x563616961890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:00.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.440+0000 7f5103fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361695bf60 0x563616961890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37852/0 (socket says 172.21.15.7:37852) 2024-06-10T01:19:00.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.440+0000 7f5103fff700 1 -- 172.21.15.7:0/2616544563 learned_addr learned my addr 172.21.15.7:0/2616544563 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:00.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.440+0000 7f5108877700 1 --2- 172.21.15.7:0/2616544563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636169567a0 0x563616956260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:00.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.440+0000 7f5109078700 1 --2- 172.21.15.7:0/2616544563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56361695c880 0x563616950e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:00.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.440+0000 7f5103fff700 1 -- 172.21.15.7:0/2616544563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636169567a0 msgr2=0x563616956260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:00.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.440+0000 7f5103fff700 1 --2- 172.21.15.7:0/2616544563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636169567a0 0x563616956260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.440+0000 7f5103fff700 1 -- 172.21.15.7:0/2616544563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56361695c880 msgr2=0x563616950e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:00.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.440+0000 7f5103fff700 1 --2- 172.21.15.7:0/2616544563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56361695c880 0x563616950e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.440+0000 7f5103fff700 1 -- 172.21.15.7:0/2616544563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56361694d810 con 0x56361695bf60 2024-06-10T01:19:00.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.441+0000 7f5103fff700 1 --2- 172.21.15.7:0/2616544563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361695bf60 0x563616961890 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f50f0003a50 tx=0x7f50f001f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9081b680294286 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:00.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.441+0000 7f50ee7fc700 1 -- 172.21.15.7:0/2616544563 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0003ba0 con 0x56361695bf60 2024-06-10T01:19:00.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.441+0000 7f50ee7fc700 1 -- 172.21.15.7:0/2616544563 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50f0003d40 con 0x56361695bf60 2024-06-10T01:19:00.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.441+0000 7f50ee7fc700 1 -- 172.21.15.7:0/2616544563 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f000ee60 con 0x56361695bf60 2024-06-10T01:19:00.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.441+0000 7f511535eb80 1 -- 172.21.15.7:0/2616544563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361695bf60 msgr2=0x563616961890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:00.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.442+0000 7f511535eb80 1 --2- 172.21.15.7:0/2616544563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361695bf60 0x563616961890 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f50f0003a50 tx=0x7f50f001f040 comp rx=0 tx=0).stop 2024-06-10T01:19:00.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.442+0000 7f511535eb80 1 -- 172.21.15.7:0/2616544563 shutdown_connections 2024-06-10T01:19:00.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.442+0000 7f511535eb80 1 --2- 172.21.15.7:0/2616544563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56361695c880 0x563616950e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.442+0000 7f511535eb80 1 --2- 172.21.15.7:0/2616544563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636169567a0 0x563616956260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.442+0000 7f511535eb80 1 --2- 172.21.15.7:0/2616544563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361695bf60 0x563616961890 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.442+0000 7f511535eb80 1 -- 172.21.15.7:0/2616544563 >> 172.21.15.7:0/2616544563 conn(0x563616883080 msgr2=0x563616961180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:00.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.442+0000 7f511535eb80 1 -- 172.21.15.7:0/2616544563 shutdown_connections 2024-06-10T01:19:00.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.442+0000 7f511535eb80 1 -- 172.21.15.7:0/2616544563 wait complete. 2024-06-10T01:19:00.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.442+0000 7f511535eb80 1 -- start start 2024-06-10T01:19:00.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.442+0000 7f511535eb80 1 Processor -- start 2024-06-10T01:19:00.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.443+0000 7f511535eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636169567a0 0x563616977b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:00.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.443+0000 7f511535eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56361695bf60 0x5636169780d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:00.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.443+0000 7f511535eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361695c880 0x563616978610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:00.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.443+0000 7f511535eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f51041b2e50 con 0x56361695c880 2024-06-10T01:19:00.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.443+0000 7f511535eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f51041b42c0 con 0x56361695bf60 2024-06-10T01:19:00.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.443+0000 7f511535eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f51041b3520 con 0x5636169567a0 2024-06-10T01:19:00.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.443+0000 7f511535eb80 10 client.0 _ll_register_callbacks cb 0x56361683d5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:19:00.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.443+0000 7f5103fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56361695bf60 0x5636169780d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:00.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.443+0000 7f5108877700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636169567a0 0x563616977b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:00.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.443+0000 7f5109078700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361695c880 0x563616978610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:00.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.443+0000 7f5108877700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636169567a0 0x563616977b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36480/0 (socket says 172.21.15.7:36480) 2024-06-10T01:19:00.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.443+0000 7f5108877700 1 -- 172.21.15.7:0/636646121 learned_addr learned my addr 172.21.15.7:0/636646121 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:00.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.443+0000 7f5108877700 1 -- 172.21.15.7:0/636646121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56361695bf60 msgr2=0x5636169780d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:00.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.443+0000 7f5108877700 1 --2- 172.21.15.7:0/636646121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56361695bf60 0x5636169780d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.443+0000 7f5108877700 1 -- 172.21.15.7:0/636646121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361695c880 msgr2=0x563616978610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:00.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.443+0000 7f5108877700 1 --2- 172.21.15.7:0/636646121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361695c880 0x563616978610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.443+0000 7f5108877700 1 -- 172.21.15.7:0/636646121 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636169d9f40 con 0x5636169567a0 2024-06-10T01:19:00.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.443+0000 7f5103fff700 1 --2- 172.21.15.7:0/636646121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56361695bf60 0x5636169780d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:00.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.443+0000 7f5108877700 1 --2- 172.21.15.7:0/636646121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636169567a0 0x563616977b90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f50f80990c0 tx=0x7f50f8011db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29466b1f772cc33e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:00.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.443+0000 7f50e3fff700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:19:00.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.444+0000 7f50e3fff700 1 -- 172.21.15.7:0/636646121 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f809c5c0 con 0x5636169567a0 2024-06-10T01:19:00.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.444+0000 7f50e3fff700 1 -- 172.21.15.7:0/636646121 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f50f80d0410 con 0x5636169567a0 2024-06-10T01:19:00.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.444+0000 7f511535eb80 1 -- 172.21.15.7:0/636646121 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50f0003d40 con 0x5636169567a0 2024-06-10T01:19:00.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.444+0000 7f50e3fff700 1 -- 172.21.15.7:0/636646121 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f50f800be80 con 0x5636169567a0 2024-06-10T01:19:00.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.444+0000 7f511535eb80 10 client.28024 fetch_fsmap learned FSMap version 40 2024-06-10T01:19:00.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.444+0000 7f511535eb80 1 -- 172.21.15.7:0/636646121 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5636169d9f40 con 0x5636169567a0 2024-06-10T01:19:00.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.444+0000 7f50e3fff700 1 -- 172.21.15.7:0/636646121 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50f0003d40 con 0x5636169567a0 2024-06-10T01:19:00.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.444+0000 7f50e3fff700 1 -- 172.21.15.7:0/636646121 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f50f80bb370 con 0x5636169567a0 2024-06-10T01:19:00.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.444+0000 7f511535eb80 10 client.28024 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:19:00.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.444+0000 7f511535eb80 10 client.28024 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:19:00.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.444+0000 7f511535eb80 20 client.28024 populate_metadata read hostname 'smithi007' 2024-06-10T01:19:00.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.444+0000 7f511535eb80 1 --2- 172.21.15.7:0/636646121 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5636169db540 0x5636169dd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:00.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.444+0000 7f511535eb80 4 client.28024 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:19:00.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.444+0000 7f511535eb80 1 -- 172.21.15.7:0/636646121 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5636168af2d0 con 0x5636169db540 2024-06-10T01:19:00.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.444+0000 7f5103fff700 1 --2- 172.21.15.7:0/636646121 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5636169db540 0x5636169dd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:00.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.445+0000 7f50e3fff700 10 client.28024 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:00.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.445+0000 7f5103fff700 1 --2- 172.21.15.7:0/636646121 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5636169db540 0x5636169dd930 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=232d5d0b61ba2d8f server_cookie=dcb9dbc26931809f in_seq=0 out_seq=0 2024-06-10T01:19:00.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.445+0000 7f50e3fff700 1 -- 172.21.15.7:0/636646121 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5636168af2d0 con 0x5636169db540 2024-06-10T01:19:00.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.445+0000 7f50e3fff700 10 client.28024 handle_command_reply: tid=0 2024-06-10T01:19:00.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.445+0000 7f511535eb80 1 client.28024 shutdown 2024-06-10T01:19:00.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.445+0000 7f511535eb80 1 -- 172.21.15.7:0/636646121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636169567a0 msgr2=0x563616977b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:00.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.445+0000 7f511535eb80 1 --2- 172.21.15.7:0/636646121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636169567a0 0x563616977b90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f50f80990c0 tx=0x7f50f8011db0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.445+0000 7f511535eb80 1 -- 172.21.15.7:0/636646121 shutdown_connections 2024-06-10T01:19:00.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.445+0000 7f511535eb80 1 --2- 172.21.15.7:0/636646121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361695c880 0x563616978610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.445+0000 7f511535eb80 1 --2- 172.21.15.7:0/636646121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636169567a0 0x563616977b90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.445+0000 7f511535eb80 1 --2- 172.21.15.7:0/636646121 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5636169db540 0x5636169dd930 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.445+0000 7f511535eb80 1 --2- 172.21.15.7:0/636646121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56361695bf60 0x5636169780d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.445+0000 7f50e3fff700 0 client.28024 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:00.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.445+0000 7f511535eb80 1 -- 172.21.15.7:0/636646121 >> 172.21.15.7:0/636646121 conn(0x563616883080 msgr2=0x56361695aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:00.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.445+0000 7f511535eb80 1 -- 172.21.15.7:0/636646121 shutdown_connections 2024-06-10T01:19:00.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.445+0000 7f511535eb80 1 -- 172.21.15.7:0/636646121 wait complete. 2024-06-10T01:19:00.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.446+0000 7f511535eb80 20 client.28024 trim_cache size 0 max 16384 2024-06-10T01:19:00.444 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:19:00.444 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-10T01:19:00.444 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-10T01:19:00.445 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:19:00.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.446+0000 7f510aadb700 1 -- 172.21.15.7:0/500150432 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f50f40a2c20 msgr2=0x7f50f40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:00.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.446+0000 7f510aadb700 1 --2- 172.21.15.7:0/500150432 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f50f40a2c20 0x7f50f40a50e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5104078970 tx=0x7f50f0009260 comp rx=0 tx=0).stop 2024-06-10T01:19:00.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.446+0000 7f510aadb700 1 -- 172.21.15.7:0/500150432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f510406aa00 msgr2=0x7f510419f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:00.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.446+0000 7f510aadb700 1 --2- 172.21.15.7:0/500150432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f510406aa00 0x7f510419f240 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f50f8010b20 tx=0x7f50f8010b50 comp rx=0 tx=0).stop 2024-06-10T01:19:00.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.446+0000 7f510aadb700 1 -- 172.21.15.7:0/500150432 shutdown_connections 2024-06-10T01:19:00.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.446+0000 7f510aadb700 1 --2- 172.21.15.7:0/500150432 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f50f40a2c20 0x7f50f40a50e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.446+0000 7f510aadb700 1 --2- 172.21.15.7:0/500150432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51041084f0 0x7f51041a3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.446+0000 7f510aadb700 1 --2- 172.21.15.7:0/500150432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f510406aa00 0x7f510419f240 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.446+0000 7f510aadb700 1 --2- 172.21.15.7:0/500150432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f510406b3c0 0x7f510419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:00.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.446+0000 7f510aadb700 1 -- 172.21.15.7:0/500150432 >> 172.21.15.7:0/500150432 conn(0x7f5104078300 msgr2=0x7f51040fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:00.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.446+0000 7f510aadb700 1 -- 172.21.15.7:0/500150432 shutdown_connections 2024-06-10T01:19:00.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:00.446+0000 7f510aadb700 1 -- 172.21.15.7:0/500150432 wait complete. 2024-06-10T01:19:00.757 INFO:tasks.workunit.client.0.smithi007.stdout: 1 954968 7.30 MB/sec execute 581 sec latency 23.797 ms 2024-06-10T01:19:01.236 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:19:01.236 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:19:01.497 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:19:01.744 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:01 smithi007 ceph-mon[32398]: pgmap v822: 129 pgs: 129 active+clean; 142 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 2.5 MiB/s wr, 346 op/s 2024-06-10T01:19:01.756 INFO:tasks.workunit.client.0.smithi007.stdout: 1 956533 7.30 MB/sec execute 582 sec latency 10.793 ms 2024-06-10T01:19:01.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:01 smithi078 ceph-mon[36683]: pgmap v822: 129 pgs: 129 active+clean; 142 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 2.5 MiB/s wr, 346 op/s 2024-06-10T01:19:01.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:01 smithi175 ceph-mon[36636]: pgmap v822: 129 pgs: 129 active+clean; 142 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 2.5 MiB/s wr, 346 op/s 2024-06-10T01:19:02.757 INFO:tasks.workunit.client.0.smithi007.stdout: 1 958939 7.31 MB/sec execute 583 sec latency 16.126 ms 2024-06-10T01:19:03.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.297+0000 7fa5c3c29700 1 -- 172.21.15.7:0/903349421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5bc10eac0 msgr2=0x7fa5bc10eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:03.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.297+0000 7fa5c3c29700 1 --2- 172.21.15.7:0/903349421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5bc10eac0 0x7fa5bc10eea0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac009990 tx=0x7fa5ac009ca0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.298+0000 7fa5c3c29700 1 -- 172.21.15.7:0/903349421 shutdown_connections 2024-06-10T01:19:03.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.298+0000 7fa5c3c29700 1 --2- 172.21.15.7:0/903349421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5bc109480 0x7fa5bc115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.298+0000 7fa5c3c29700 1 --2- 172.21.15.7:0/903349421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5bc10eac0 0x7fa5bc10eea0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.298+0000 7fa5c3c29700 1 --2- 172.21.15.7:0/903349421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5bc108ac0 0x7fa5bc108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.298+0000 7fa5c3c29700 1 -- 172.21.15.7:0/903349421 >> 172.21.15.7:0/903349421 conn(0x7fa5bc075c60 msgr2=0x7fa5bc1072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:03.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.298+0000 7fa5c3c29700 1 -- 172.21.15.7:0/903349421 shutdown_connections 2024-06-10T01:19:03.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.298+0000 7fa5c3c29700 1 -- 172.21.15.7:0/903349421 wait complete. 2024-06-10T01:19:03.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.299+0000 7fa5c3c29700 1 Processor -- start 2024-06-10T01:19:03.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.299+0000 7fa5c3c29700 1 -- start start 2024-06-10T01:19:03.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.300+0000 7fa5c3c29700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5bc108ac0 0x7fa5bc073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:03.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.300+0000 7fa5c3c29700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5bc109480 0x7fa5bc06cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:03.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.300+0000 7fa5c3c29700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5bc10eac0 0x7fa5bc06d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:03.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.300+0000 7fa5c3c29700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc118f40 con 0x7fa5bc108ac0 2024-06-10T01:19:03.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.300+0000 7fa5c3c29700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc118dc0 con 0x7fa5bc109480 2024-06-10T01:19:03.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.300+0000 7fa5c3c29700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc1190c0 con 0x7fa5bc10eac0 2024-06-10T01:19:03.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.300+0000 7fa5c19c5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5bc108ac0 0x7fa5bc073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:03.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.300+0000 7fa5c19c5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5bc108ac0 0x7fa5bc073c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55400/0 (socket says 172.21.15.7:55400) 2024-06-10T01:19:03.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.300+0000 7fa5c19c5700 1 -- 172.21.15.7:0/1714102281 learned_addr learned my addr 172.21.15.7:0/1714102281 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:03.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.300+0000 7fa5c21c6700 1 --2- 172.21.15.7:0/1714102281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5bc10eac0 0x7fa5bc06d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:03.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.300+0000 7fa5c19c5700 1 -- 172.21.15.7:0/1714102281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5bc10eac0 msgr2=0x7fa5bc06d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:03.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.300+0000 7fa5c19c5700 1 --2- 172.21.15.7:0/1714102281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5bc10eac0 0x7fa5bc06d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.300+0000 7fa5c19c5700 1 -- 172.21.15.7:0/1714102281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5bc109480 msgr2=0x7fa5bc06cd40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-10T01:19:03.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.300+0000 7fa5c19c5700 1 --2- 172.21.15.7:0/1714102281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5bc109480 0x7fa5bc06cd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.300+0000 7fa5c19c5700 1 -- 172.21.15.7:0/1714102281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5bc06d870 con 0x7fa5bc108ac0 2024-06-10T01:19:03.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.301+0000 7fa5c19c5700 1 --2- 172.21.15.7:0/1714102281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5bc108ac0 0x7fa5bc073c60 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac0057c0 tx=0x7fa5ac005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:03.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.301+0000 7fa5b2ffd700 1 -- 172.21.15.7:0/1714102281 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac027070 con 0x7fa5bc108ac0 2024-06-10T01:19:03.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.301+0000 7fa5c3c29700 1 -- 172.21.15.7:0/1714102281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5bc06db00 con 0x7fa5bc108ac0 2024-06-10T01:19:03.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.301+0000 7fa5b2ffd700 1 -- 172.21.15.7:0/1714102281 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5ac0040f0 con 0x7fa5bc108ac0 2024-06-10T01:19:03.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.301+0000 7fa5b2ffd700 1 -- 172.21.15.7:0/1714102281 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac00f460 con 0x7fa5bc108ac0 2024-06-10T01:19:03.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.301+0000 7fa5c3c29700 1 -- 172.21.15.7:0/1714102281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5bc0719b0 con 0x7fa5bc108ac0 2024-06-10T01:19:03.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.303+0000 7fa5c3c29700 1 -- 172.21.15.7:0/1714102281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5bc10a200 con 0x7fa5bc108ac0 2024-06-10T01:19:03.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.305+0000 7fa5b2ffd700 1 -- 172.21.15.7:0/1714102281 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa5ac02a070 con 0x7fa5bc108ac0 2024-06-10T01:19:03.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.305+0000 7fa5b2ffd700 1 --2- 172.21.15.7:0/1714102281 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa5a80a2ca0 0x7fa5a80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:03.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.306+0000 7fa5b2ffd700 1 -- 172.21.15.7:0/1714102281 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa5ac0d0b40 con 0x7fa5bc108ac0 2024-06-10T01:19:03.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.306+0000 7fa5c11c4700 1 --2- 172.21.15.7:0/1714102281 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa5a80a2ca0 0x7fa5a80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:03.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.306+0000 7fa5c11c4700 1 --2- 172.21.15.7:0/1714102281 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa5a80a2ca0 0x7fa5a80a5160 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc06e4b0 tx=0x7fa5b8006c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:03.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.309+0000 7fa5b2ffd700 1 -- 172.21.15.7:0/1714102281 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa5ac031020 con 0x7fa5bc108ac0 2024-06-10T01:19:03.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.669+0000 7fa5c3c29700 1 -- 172.21.15.7:0/1714102281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5bc069180 con 0x7fa5bc108ac0 2024-06-10T01:19:03.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:03 smithi007 ceph-mon[32398]: pgmap v823: 129 pgs: 129 active+clean; 142 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 1.9 MiB/s wr, 280 op/s 2024-06-10T01:19:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.670+0000 7fa5b2ffd700 1 -- 172.21.15.7:0/1714102281 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11855 (secure 0 0 0) 0x7fa5ac013070 con 0x7fa5bc108ac0 2024-06-10T01:19:03.664 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:19:03.665 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:16:56.409130+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24451,24434,24436]},"id":1}]} 2024-06-10T01:19:03.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.673+0000 7fa5c3c29700 1 -- 172.21.15.7:0/1714102281 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa5a80a2ca0 msgr2=0x7fa5a80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:03.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.673+0000 7fa5c3c29700 1 --2- 172.21.15.7:0/1714102281 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa5a80a2ca0 0x7fa5a80a5160 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc06e4b0 tx=0x7fa5b8006c60 comp rx=0 tx=0).stop 2024-06-10T01:19:03.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.673+0000 7fa5c3c29700 1 -- 172.21.15.7:0/1714102281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5bc108ac0 msgr2=0x7fa5bc073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:03.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.673+0000 7fa5c3c29700 1 --2- 172.21.15.7:0/1714102281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5bc108ac0 0x7fa5bc073c60 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac0057c0 tx=0x7fa5ac005f10 comp rx=0 tx=0).stop 2024-06-10T01:19:03.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.673+0000 7fa5c3c29700 1 -- 172.21.15.7:0/1714102281 shutdown_connections 2024-06-10T01:19:03.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.673+0000 7fa5c3c29700 1 --2- 172.21.15.7:0/1714102281 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa5a80a2ca0 0x7fa5a80a5160 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.673+0000 7fa5c3c29700 1 --2- 172.21.15.7:0/1714102281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5bc10eac0 0x7fa5bc06d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.673+0000 7fa5c3c29700 1 --2- 172.21.15.7:0/1714102281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5bc108ac0 0x7fa5bc073c60 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.673+0000 7fa5c3c29700 1 --2- 172.21.15.7:0/1714102281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5bc109480 0x7fa5bc06cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.673+0000 7fa5c3c29700 1 -- 172.21.15.7:0/1714102281 >> 172.21.15.7:0/1714102281 conn(0x7fa5bc075c60 msgr2=0x7fa5bc0762c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:03.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.673+0000 7fa5c3c29700 1 -- 172.21.15.7:0/1714102281 shutdown_connections 2024-06-10T01:19:03.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:03.673+0000 7fa5c3c29700 1 -- 172.21.15.7:0/1714102281 wait complete. 2024-06-10T01:19:03.668 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 40 2024-06-10T01:19:03.757 INFO:tasks.workunit.client.0.smithi007.stdout: 1 960066 7.31 MB/sec execute 584 sec latency 26.403 ms 2024-06-10T01:19:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.813+0000 7f870bce1640 1 -- 172.21.15.7:0/3938756691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8704104db0 msgr2=0x7f87041071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.813+0000 7f870bce1640 1 --2- 172.21.15.7:0/3938756691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8704104db0 0x7f87041071a0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f86f4009900 tx=0x7f86f402a410 comp rx=0 tx=0).stop 2024-06-10T01:19:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.813+0000 7f870bce1640 1 -- 172.21.15.7:0/3938756691 shutdown_connections 2024-06-10T01:19:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.813+0000 7f870bce1640 1 --2- 172.21.15.7:0/3938756691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87041076e0 0x7f8704109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.813+0000 7f870bce1640 1 --2- 172.21.15.7:0/3938756691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8704104db0 0x7f87041071a0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.813+0000 7f870bce1640 1 --2- 172.21.15.7:0/3938756691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8704102480 0x7f8704104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.813+0000 7f870bce1640 1 -- 172.21.15.7:0/3938756691 >> 172.21.15.7:0/3938756691 conn(0x7f87040fbe70 msgr2=0x7f87040fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.813+0000 7f870bce1640 1 -- 172.21.15.7:0/3938756691 shutdown_connections 2024-06-10T01:19:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.813+0000 7f870bce1640 1 -- 172.21.15.7:0/3938756691 wait complete. 2024-06-10T01:19:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.814+0000 7f870bce1640 1 Processor -- start 2024-06-10T01:19:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.814+0000 7f870bce1640 1 -- start start 2024-06-10T01:19:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.814+0000 7f870bce1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8704102480 0x7f8704197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.814+0000 7f870bce1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8704104db0 0x7f8704198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.814+0000 7f870bce1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87041076e0 0x7f870419f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.814+0000 7f870bce1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f870410d690 con 0x7f8704104db0 2024-06-10T01:19:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.814+0000 7f870bce1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f870410d510 con 0x7f8704102480 2024-06-10T01:19:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.814+0000 7f870bce1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f870410d810 con 0x7f87041076e0 2024-06-10T01:19:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.814+0000 7f8709a7d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8704102480 0x7f8704197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.814+0000 7f870927c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8704104db0 0x7f8704198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.814+0000 7f8709a7d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8704102480 0x7f8704197d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:40086/0 (socket says 172.21.15.7:40086) 2024-06-10T01:19:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.814+0000 7f8709a7d640 1 -- 172.21.15.7:0/295537693 learned_addr learned my addr 172.21.15.7:0/295537693 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.814+0000 7f870a27e640 1 --2- 172.21.15.7:0/295537693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87041076e0 0x7f870419f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.814+0000 7f870927c640 1 -- 172.21.15.7:0/295537693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87041076e0 msgr2=0x7f870419f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.814+0000 7f870927c640 1 --2- 172.21.15.7:0/295537693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87041076e0 0x7f870419f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.814+0000 7f870927c640 1 -- 172.21.15.7:0/295537693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8704102480 msgr2=0x7f8704197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.814+0000 7f870927c640 1 --2- 172.21.15.7:0/295537693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8704102480 0x7f8704197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.814+0000 7f870927c640 1 -- 172.21.15.7:0/295537693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870419f940 con 0x7f8704104db0 2024-06-10T01:19:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.815+0000 7f870927c640 1 --2- 172.21.15.7:0/295537693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8704104db0 0x7f8704198270 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f86f4009230 tx=0x7f86f4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.815+0000 7f86faffd640 1 -- 172.21.15.7:0/295537693 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f403f070 con 0x7f8704104db0 2024-06-10T01:19:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.815+0000 7f86faffd640 1 -- 172.21.15.7:0/295537693 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86f4003cd0 con 0x7f8704104db0 2024-06-10T01:19:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.815+0000 7f870bce1640 1 -- 172.21.15.7:0/295537693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f870419fbd0 con 0x7f8704104db0 2024-06-10T01:19:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.815+0000 7f86faffd640 1 -- 172.21.15.7:0/295537693 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f4035070 con 0x7f8704104db0 2024-06-10T01:19:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.815+0000 7f870bce1640 1 -- 172.21.15.7:0/295537693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87041a00e0 con 0x7f8704104db0 2024-06-10T01:19:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.817+0000 7f86faffd640 1 -- 172.21.15.7:0/295537693 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f86f403b460 con 0x7f8704104db0 2024-06-10T01:19:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.818+0000 7f870e9f3740 1 Processor -- start 2024-06-10T01:19:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.818+0000 7f870e9f3740 1 -- start start 2024-06-10T01:19:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.818+0000 7f870e9f3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b622009420 0x55b622009840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.818+0000 7f870e9f3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b622009e10 0x55b62200e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.818+0000 7f870e9f3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b62200ef80 0x55b622011410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.818+0000 7f870e9f3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b621f684f0 con 0x55b622009420 2024-06-10T01:19:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.818+0000 7f870e9f3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b621f06870 con 0x55b62200ef80 2024-06-10T01:19:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.818+0000 7f870e9f3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b621f644b0 con 0x55b622009e10 2024-06-10T01:19:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.818+0000 7f870a27e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b622009420 0x55b622009840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.818+0000 7f8709a7d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b622009e10 0x55b62200e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:03.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.818+0000 7f870a27e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b622009420 0x55b622009840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55428/0 (socket says 172.21.15.7:55428) 2024-06-10T01:19:03.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.818+0000 7f870a27e640 1 -- 172.21.15.7:0/2123153747 learned_addr learned my addr 172.21.15.7:0/2123153747 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:03.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.818+0000 7f870927c640 1 --2- 172.21.15.7:0/2123153747 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b62200ef80 0x55b622011410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:03.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.819+0000 7f8709a7d640 1 -- 172.21.15.7:0/2123153747 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b62200ef80 msgr2=0x55b622011410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:03.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.819+0000 7f8709a7d640 1 --2- 172.21.15.7:0/2123153747 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b62200ef80 0x55b622011410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.819+0000 7f8709a7d640 1 -- 172.21.15.7:0/2123153747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b622009420 msgr2=0x55b622009840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:03.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.819+0000 7f8709a7d640 1 --2- 172.21.15.7:0/2123153747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b622009420 0x55b622009840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.819+0000 7f8709a7d640 1 -- 172.21.15.7:0/2123153747 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b621f229b0 con 0x55b622009e10 2024-06-10T01:19:03.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.819+0000 7f86faffd640 1 --2- 172.21.15.7:0/295537693 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f86f00a2c00 0x7f86f00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:03.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.819+0000 7f870a27e640 1 --2- 172.21.15.7:0/2123153747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b622009420 0x55b622009840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:03.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.819+0000 7f870927c640 1 --2- 172.21.15.7:0/295537693 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f86f00a2c00 0x7f86f00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:03.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.819+0000 7f8709a7d640 1 --2- 172.21.15.7:0/2123153747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b622009e10 0x55b62200e910 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8700009cc0 tx=0x7f8700011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18e63e9984970010 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.819+0000 7f86faffd640 1 -- 172.21.15.7:0/295537693 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f86f40ef780 con 0x7f8704104db0 2024-06-10T01:19:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.819+0000 7f86effff640 1 -- 172.21.15.7:0/2123153747 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8700005220 con 0x55b622009e10 2024-06-10T01:19:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.819+0000 7f870927c640 1 --2- 172.21.15.7:0/295537693 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f86f00a2c00 0x7f86f00a50c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f86f40b8650 tx=0x7f86f40f5550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.819+0000 7f86effff640 1 -- 172.21.15.7:0/2123153747 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8700004c80 con 0x55b622009e10 2024-06-10T01:19:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.819+0000 7f86effff640 1 -- 172.21.15.7:0/2123153747 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8700019890 con 0x55b622009e10 2024-06-10T01:19:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.820+0000 7f870e9f3740 1 -- 172.21.15.7:0/2123153747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b622009e10 msgr2=0x55b62200e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.820+0000 7f870e9f3740 1 --2- 172.21.15.7:0/2123153747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b622009e10 0x55b62200e910 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8700009cc0 tx=0x7f8700011810 comp rx=0 tx=0).stop 2024-06-10T01:19:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.820+0000 7f870e9f3740 1 -- 172.21.15.7:0/2123153747 shutdown_connections 2024-06-10T01:19:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.820+0000 7f870e9f3740 1 --2- 172.21.15.7:0/2123153747 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b62200ef80 0x55b622011410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.820+0000 7f870e9f3740 1 --2- 172.21.15.7:0/2123153747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b622009e10 0x55b62200e910 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.820+0000 7f870e9f3740 1 --2- 172.21.15.7:0/2123153747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b622009420 0x55b622009840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.820+0000 7f870e9f3740 1 -- 172.21.15.7:0/2123153747 >> 172.21.15.7:0/2123153747 conn(0x55b622000680 msgr2=0x55b622000a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.820+0000 7f870e9f3740 1 -- 172.21.15.7:0/2123153747 shutdown_connections 2024-06-10T01:19:03.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.820+0000 7f870e9f3740 1 -- 172.21.15.7:0/2123153747 wait complete. 2024-06-10T01:19:03.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.820+0000 7f870e9f3740 1 -- start start 2024-06-10T01:19:03.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.821+0000 7f870e9f3740 1 Processor -- start 2024-06-10T01:19:03.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.821+0000 7f870e9f3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b622009420 0x55b622013600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:03.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.821+0000 7f870e9f3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b622009e10 0x55b622013b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:03.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.821+0000 7f870e9f3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b62200ef80 0x55b62201ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:03.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.821+0000 7f870e9f3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b622011f30 con 0x55b622009420 2024-06-10T01:19:03.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.821+0000 7f870e9f3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b621ef1ae0 con 0x55b622009e10 2024-06-10T01:19:03.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.821+0000 7f870e9f3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b6220120b0 con 0x55b62200ef80 2024-06-10T01:19:03.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.821+0000 7f870e9f3740 10 client.0 _ll_register_callbacks cb 0x55b621f2a9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:19:03.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.821+0000 7f870a27e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b622009420 0x55b622013600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:03.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.821+0000 7f870a27e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b622009420 0x55b622013600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55434/0 (socket says 172.21.15.7:55434) 2024-06-10T01:19:03.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.821+0000 7f870a27e640 1 -- 172.21.15.7:0/1106971643 learned_addr learned my addr 172.21.15.7:0/1106971643 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:03.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.821+0000 7f8709a7d640 1 --2- 172.21.15.7:0/1106971643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b622009e10 0x55b622013b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.821+0000 7f870a27e640 1 --2- 172.21.15.7:0/1106971643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b62200ef80 0x55b62201ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.821+0000 7f870a27e640 1 -- 172.21.15.7:0/1106971643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b62200ef80 msgr2=0x55b62201ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.821+0000 7f870a27e640 1 --2- 172.21.15.7:0/1106971643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b62200ef80 0x55b62201ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.821+0000 7f870a27e640 1 -- 172.21.15.7:0/1106971643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b622009e10 msgr2=0x55b622013b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.821+0000 7f870a27e640 1 --2- 172.21.15.7:0/1106971643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b622009e10 0x55b622013b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.821+0000 7f870a27e640 1 -- 172.21.15.7:0/1106971643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b621c7c210 con 0x55b622009420 2024-06-10T01:19:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.822+0000 7f870a27e640 1 --2- 172.21.15.7:0/1106971643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b622009420 0x55b622013600 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00f460 tx=0x7f86fc009d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f4e724091297a8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.822+0000 7f86ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:19:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.822+0000 7f86ed7fa640 1 -- 172.21.15.7:0/1106971643 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc003d80 con 0x55b622009420 2024-06-10T01:19:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.822+0000 7f86ed7fa640 1 -- 172.21.15.7:0/1106971643 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f86fc004050 con 0x55b622009420 2024-06-10T01:19:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.822+0000 7f870e9f3740 1 -- 172.21.15.7:0/1106971643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8700004c80 con 0x55b622009420 2024-06-10T01:19:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.822+0000 7f86ed7fa640 1 -- 172.21.15.7:0/1106971643 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f86fc004250 con 0x55b622009420 2024-06-10T01:19:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.822+0000 7f870e9f3740 10 client.18708 fetch_fsmap learned FSMap version 40 2024-06-10T01:19:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.822+0000 7f870e9f3740 1 -- 172.21.15.7:0/1106971643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b621d48620 con 0x55b622009420 2024-06-10T01:19:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.822+0000 7f86ed7fa640 1 -- 172.21.15.7:0/1106971643 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8700004c80 con 0x55b622009420 2024-06-10T01:19:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.823+0000 7f86ed7fa640 1 -- 172.21.15.7:0/1106971643 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f86fc01e020 con 0x55b622009420 2024-06-10T01:19:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.823+0000 7f870e9f3740 10 client.18708 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:19:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.823+0000 7f870e9f3740 10 client.18708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:19:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.823+0000 7f870e9f3740 20 client.18708 populate_metadata read hostname 'smithi007' 2024-06-10T01:19:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.823+0000 7f870e9f3740 1 --2- 172.21.15.7:0/1106971643 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b62201f460 0x55b6220218f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.823+0000 7f870e9f3740 4 client.18708 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:19:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.823+0000 7f870e9f3740 1 -- 172.21.15.7:0/1106971643 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b621f7bf10 con 0x55b62201f460 2024-06-10T01:19:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.823+0000 7f8709a7d640 1 --2- 172.21.15.7:0/1106971643 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b62201f460 0x55b6220218f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.823+0000 7f8709a7d640 1 --2- 172.21.15.7:0/1106971643 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b62201f460 0x55b6220218f0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da3fbf54b5593dee server_cookie=f745302261a5613f in_seq=0 out_seq=0 2024-06-10T01:19:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.823+0000 7f86ed7fa640 10 client.18708 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.824+0000 7f86ed7fa640 1 -- 172.21.15.7:0/1106971643 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b621f7bf10 con 0x55b62201f460 2024-06-10T01:19:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.824+0000 7f86ed7fa640 10 client.18708 handle_command_reply: tid=0 2024-06-10T01:19:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.825+0000 7f870e9f3740 1 client.18708 shutdown 2024-06-10T01:19:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.825+0000 7f870e9f3740 1 -- 172.21.15.7:0/1106971643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b622009420 msgr2=0x55b622013600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.825+0000 7f870e9f3740 1 --2- 172.21.15.7:0/1106971643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b622009420 0x55b622013600 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00f460 tx=0x7f86fc009d70 comp rx=0 tx=0).stop 2024-06-10T01:19:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.825+0000 7f870e9f3740 1 -- 172.21.15.7:0/1106971643 shutdown_connections 2024-06-10T01:19:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.825+0000 7f870e9f3740 1 --2- 172.21.15.7:0/1106971643 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b62201f460 0x55b6220218f0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.825+0000 7f870e9f3740 1 --2- 172.21.15.7:0/1106971643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b62200ef80 0x55b62201ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.825+0000 7f870e9f3740 1 --2- 172.21.15.7:0/1106971643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b622009e10 0x55b622013b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.825+0000 7f870e9f3740 1 --2- 172.21.15.7:0/1106971643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b622009420 0x55b622013600 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.825+0000 7f86ed7fa640 0 client.18708 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.825+0000 7f870e9f3740 1 -- 172.21.15.7:0/1106971643 >> 172.21.15.7:0/1106971643 conn(0x55b622000680 msgr2=0x55b621fff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.825+0000 7f870e9f3740 1 -- 172.21.15.7:0/1106971643 shutdown_connections 2024-06-10T01:19:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.825+0000 7f870e9f3740 1 -- 172.21.15.7:0/1106971643 wait complete. 2024-06-10T01:19:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.825+0000 7f870e9f3740 20 client.18708 trim_cache size 0 max 16384 2024-06-10T01:19:03.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:03 smithi078 ceph-mon[36683]: pgmap v823: 129 pgs: 129 active+clean; 142 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 1.9 MiB/s wr, 280 op/s 2024-06-10T01:19:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.841+0000 7f870e9f3740 1 Processor -- start 2024-06-10T01:19:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.841+0000 7f870e9f3740 1 -- start start 2024-06-10T01:19:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.842+0000 7f870e9f3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b622009420 0x55b622003ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.842+0000 7f870e9f3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b622009e10 0x55b62201ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.842+0000 7f870e9f3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b62200ef80 0x55b622013210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.842+0000 7f870e9f3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f87041aabc0 con 0x55b622009420 2024-06-10T01:19:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.842+0000 7f870e9f3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f87041a8490 con 0x55b622009e10 2024-06-10T01:19:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.842+0000 7f870e9f3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f87041a62e0 con 0x55b62200ef80 2024-06-10T01:19:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.842+0000 7f870a27e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b622009420 0x55b622003ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.842+0000 7f870a27e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b622009420 0x55b622003ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55442/0 (socket says 172.21.15.7:55442) 2024-06-10T01:19:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.842+0000 7f870a27e640 1 -- 172.21.15.7:0/1509604416 learned_addr learned my addr 172.21.15.7:0/1509604416 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.842+0000 7f8709a7d640 1 --2- 172.21.15.7:0/1509604416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b622009e10 0x55b62201ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.842+0000 7f870a27e640 1 --2- 172.21.15.7:0/1509604416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b62200ef80 0x55b622013210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.842+0000 7f8709a7d640 1 -- 172.21.15.7:0/1509604416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b62200ef80 msgr2=0x55b622013210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.842+0000 7f8709a7d640 1 --2- 172.21.15.7:0/1509604416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b62200ef80 0x55b622013210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.842+0000 7f8709a7d640 1 -- 172.21.15.7:0/1509604416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b622009420 msgr2=0x55b622003ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.842+0000 7f8709a7d640 1 --2- 172.21.15.7:0/1509604416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b622009420 0x55b622003ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.842+0000 7f8709a7d640 1 -- 172.21.15.7:0/1509604416 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6220004b0 con 0x55b622009e10 2024-06-10T01:19:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.842+0000 7f870a27e640 1 --2- 172.21.15.7:0/1509604416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b622009420 0x55b622003ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.842+0000 7f8709a7d640 1 --2- 172.21.15.7:0/1509604416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b622009e10 0x55b62201ab40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x55b622001550 tx=0x7f8700009880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ea5c42e0f509d29 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.843+0000 7f86ef7fe640 1 -- 172.21.15.7:0/1509604416 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8700004a90 con 0x55b622009e10 2024-06-10T01:19:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.843+0000 7f86ef7fe640 1 -- 172.21.15.7:0/1509604416 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8700004c30 con 0x55b622009e10 2024-06-10T01:19:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.843+0000 7f86ef7fe640 1 -- 172.21.15.7:0/1509604416 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8700019830 con 0x55b622009e10 2024-06-10T01:19:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.843+0000 7f870e9f3740 1 -- 172.21.15.7:0/1509604416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b622009e10 msgr2=0x55b62201ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.843+0000 7f870e9f3740 1 --2- 172.21.15.7:0/1509604416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b622009e10 0x55b62201ab40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x55b622001550 tx=0x7f8700009880 comp rx=0 tx=0).stop 2024-06-10T01:19:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.843+0000 7f870e9f3740 1 -- 172.21.15.7:0/1509604416 shutdown_connections 2024-06-10T01:19:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.843+0000 7f870e9f3740 1 --2- 172.21.15.7:0/1509604416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b62200ef80 0x55b622013210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.843+0000 7f870e9f3740 1 --2- 172.21.15.7:0/1509604416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b622009e10 0x55b62201ab40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.843+0000 7f870e9f3740 1 --2- 172.21.15.7:0/1509604416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b622009420 0x55b622003ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.843+0000 7f870e9f3740 1 -- 172.21.15.7:0/1509604416 >> 172.21.15.7:0/1509604416 conn(0x55b622000680 msgr2=0x55b622003590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.843+0000 7f870e9f3740 1 -- 172.21.15.7:0/1509604416 shutdown_connections 2024-06-10T01:19:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.844+0000 7f870e9f3740 1 -- 172.21.15.7:0/1509604416 wait complete. 2024-06-10T01:19:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.844+0000 7f870e9f3740 1 -- start start 2024-06-10T01:19:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.844+0000 7f870e9f3740 1 Processor -- start 2024-06-10T01:19:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.844+0000 7f870e9f3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b622009420 0x55b622017d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.844+0000 7f870e9f3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b622009e10 0x55b6220182a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.844+0000 7f870e9f3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b62200ef80 0x55b6220192b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.844+0000 7f870e9f3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f87041a6950 con 0x55b622009e10 2024-06-10T01:19:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.844+0000 7f870e9f3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f87041a7dc0 con 0x55b62200ef80 2024-06-10T01:19:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.844+0000 7f870e9f3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f87041a7020 con 0x55b622009420 2024-06-10T01:19:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.844+0000 7f870e9f3740 10 client.0 _ll_register_callbacks cb 0x7f86fc009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:19:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.844+0000 7f8709a7d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b622009e10 0x55b6220182a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.844+0000 7f870a27e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b622009420 0x55b622017d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.844+0000 7f8709a7d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b622009e10 0x55b6220182a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55444/0 (socket says 172.21.15.7:55444) 2024-06-10T01:19:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.844+0000 7f8709a7d640 1 -- 172.21.15.7:0/4230239342 learned_addr learned my addr 172.21.15.7:0/4230239342 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.844+0000 7f870a27e640 1 --2- 172.21.15.7:0/4230239342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b62200ef80 0x55b6220192b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.845+0000 7f8709a7d640 1 -- 172.21.15.7:0/4230239342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b622009420 msgr2=0x55b622017d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.845+0000 7f8709a7d640 1 --2- 172.21.15.7:0/4230239342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b622009420 0x55b622017d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.845+0000 7f8709a7d640 1 -- 172.21.15.7:0/4230239342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b62200ef80 msgr2=0x55b6220192b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.845+0000 7f8709a7d640 1 --2- 172.21.15.7:0/4230239342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b62200ef80 0x55b6220192b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.845+0000 7f8709a7d640 1 -- 172.21.15.7:0/4230239342 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6220198a0 con 0x55b622009e10 2024-06-10T01:19:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.845+0000 7f870a27e640 1 --2- 172.21.15.7:0/4230239342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b622009420 0x55b622017d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.845+0000 7f8709a7d640 1 --2- 172.21.15.7:0/4230239342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b622009e10 0x55b6220182a0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f8700009970 tx=0x7f8700005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf85425b2db9a9b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.845+0000 7f86ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:19:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.845+0000 7f86ed7fa640 1 -- 172.21.15.7:0/4230239342 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870003a020 con 0x55b622009e10 2024-06-10T01:19:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.845+0000 7f86ed7fa640 1 -- 172.21.15.7:0/4230239342 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f8700005200 con 0x55b622009e10 2024-06-10T01:19:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.845+0000 7f870e9f3740 1 -- 172.21.15.7:0/4230239342 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8700004c30 con 0x55b622009e10 2024-06-10T01:19:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.845+0000 7f86ed7fa640 1 -- 172.21.15.7:0/4230239342 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8700045050 con 0x55b622009e10 2024-06-10T01:19:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.845+0000 7f870e9f3740 10 client.18714 fetch_fsmap learned FSMap version 40 2024-06-10T01:19:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.845+0000 7f870e9f3740 1 -- 172.21.15.7:0/4230239342 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b62201abb0 con 0x55b622009e10 2024-06-10T01:19:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.846+0000 7f86ed7fa640 1 -- 172.21.15.7:0/4230239342 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8700004c30 con 0x55b622009e10 2024-06-10T01:19:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.846+0000 7f86ed7fa640 1 -- 172.21.15.7:0/4230239342 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f8700019cc0 con 0x55b622009e10 2024-06-10T01:19:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.846+0000 7f870e9f3740 10 client.18714 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:19:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.846+0000 7f870e9f3740 10 client.18714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:19:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.846+0000 7f870e9f3740 20 client.18714 populate_metadata read hostname 'smithi007' 2024-06-10T01:19:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.846+0000 7f870e9f3740 1 --2- 172.21.15.7:0/4230239342 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b62208a4a0 0x55b62208c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.846+0000 7f870e9f3740 4 client.18714 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:19:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.846+0000 7f870e9f3740 1 -- 172.21.15.7:0/4230239342 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b621f75c20 con 0x55b62208a4a0 2024-06-10T01:19:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.846+0000 7f870a27e640 1 --2- 172.21.15.7:0/4230239342 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b62208a4a0 0x55b62208c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.846+0000 7f870a27e640 1 --2- 172.21.15.7:0/4230239342 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b62208a4a0 0x55b62208c890 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f6245afbc1351d9 server_cookie=7d6f68d6c4dc1c0f in_seq=0 out_seq=0 2024-06-10T01:19:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.846+0000 7f86ed7fa640 10 client.18714 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.941+0000 7f86ed7fa640 1 -- 172.21.15.7:0/4230239342 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b621f75c20 con 0x55b62208a4a0 2024-06-10T01:19:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.941+0000 7f86ed7fa640 10 client.18714 handle_command_reply: tid=0 2024-06-10T01:19:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.941+0000 7f870e9f3740 1 client.18714 shutdown 2024-06-10T01:19:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.941+0000 7f870e9f3740 1 -- 172.21.15.7:0/4230239342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b622009e10 msgr2=0x55b6220182a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.941+0000 7f870e9f3740 1 --2- 172.21.15.7:0/4230239342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b622009e10 0x55b6220182a0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f8700009970 tx=0x7f8700005d30 comp rx=0 tx=0).stop 2024-06-10T01:19:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.942+0000 7f870e9f3740 1 -- 172.21.15.7:0/4230239342 shutdown_connections 2024-06-10T01:19:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.942+0000 7f870e9f3740 1 --2- 172.21.15.7:0/4230239342 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b62208a4a0 0x55b62208c890 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.942+0000 7f870e9f3740 1 --2- 172.21.15.7:0/4230239342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b62200ef80 0x55b6220192b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.942+0000 7f870e9f3740 1 --2- 172.21.15.7:0/4230239342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b622009e10 0x55b6220182a0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.942+0000 7f870e9f3740 1 --2- 172.21.15.7:0/4230239342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b622009420 0x55b622017d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.942+0000 7f86ed7fa640 0 client.18714 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.942+0000 7f870e9f3740 1 -- 172.21.15.7:0/4230239342 >> 172.21.15.7:0/4230239342 conn(0x55b622000680 msgr2=0x55b622013210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.942+0000 7f870e9f3740 1 -- 172.21.15.7:0/4230239342 shutdown_connections 2024-06-10T01:19:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.942+0000 7f870e9f3740 1 -- 172.21.15.7:0/4230239342 wait complete. 2024-06-10T01:19:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.942+0000 7f870e9f3740 20 client.18714 trim_cache size 0 max 16384 2024-06-10T01:19:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:19:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:19:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:19:03.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.942+0000 7f870bce1640 1 -- 172.21.15.7:0/295537693 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f86f00a2c00 msgr2=0x7f86f00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:03.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.942+0000 7f870bce1640 1 --2- 172.21.15.7:0/295537693 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f86f00a2c00 0x7f86f00a50c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f86f40b8650 tx=0x7f86f40f5550 comp rx=0 tx=0).stop 2024-06-10T01:19:03.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.942+0000 7f870bce1640 1 -- 172.21.15.7:0/295537693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8704104db0 msgr2=0x7f8704198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:03.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.942+0000 7f870bce1640 1 --2- 172.21.15.7:0/295537693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8704104db0 0x7f8704198270 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f86f4009230 tx=0x7f86f4004270 comp rx=0 tx=0).stop 2024-06-10T01:19:03.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.942+0000 7f870bce1640 1 -- 172.21.15.7:0/295537693 shutdown_connections 2024-06-10T01:19:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.942+0000 7f870bce1640 1 --2- 172.21.15.7:0/295537693 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f86f00a2c00 0x7f86f00a50c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.942+0000 7f870bce1640 1 --2- 172.21.15.7:0/295537693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87041076e0 0x7f870419f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.942+0000 7f870bce1640 1 --2- 172.21.15.7:0/295537693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8704104db0 0x7f8704198270 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.942+0000 7f870bce1640 1 --2- 172.21.15.7:0/295537693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8704102480 0x7f8704197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.942+0000 7f870bce1640 1 -- 172.21.15.7:0/295537693 >> 172.21.15.7:0/295537693 conn(0x7f87040fbe70 msgr2=0x7f8704102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.942+0000 7f870bce1640 1 -- 172.21.15.7:0/295537693 shutdown_connections 2024-06-10T01:19:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:03.942+0000 7f870bce1640 1 -- 172.21.15.7:0/295537693 wait complete. 2024-06-10T01:19:03.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:03 smithi175 ceph-mon[36636]: pgmap v823: 129 pgs: 129 active+clean; 142 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 1.9 MiB/s wr, 280 op/s 2024-06-10T01:19:04.527 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:19:04.757 INFO:tasks.workunit.client.0.smithi007.stdout: 1 962200 7.32 MB/sec execute 585 sec latency 92.412 ms 2024-06-10T01:19:04.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:04 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1714102281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:19:04.793 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:19:04.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:04 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1714102281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:19:04.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:04 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1714102281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:19:05.757 INFO:tasks.workunit.client.0.smithi007.stdout: 1 963762 7.32 MB/sec execute 586 sec latency 11.158 ms 2024-06-10T01:19:05.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:05 smithi007 ceph-mon[32398]: pgmap v824: 129 pgs: 129 active+clean; 144 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 2.2 MiB/s wr, 289 op/s 2024-06-10T01:19:05.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:05 smithi078 ceph-mon[36683]: pgmap v824: 129 pgs: 129 active+clean; 144 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 2.2 MiB/s wr, 289 op/s 2024-06-10T01:19:05.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:05 smithi175 ceph-mon[36636]: pgmap v824: 129 pgs: 129 active+clean; 144 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 2.2 MiB/s wr, 289 op/s 2024-06-10T01:19:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.466+0000 7f9319011700 1 -- 172.21.15.7:0/674234302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93141073d0 msgr2=0x7f93141077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.466+0000 7f9319011700 1 --2- 172.21.15.7:0/674234302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93141073d0 0x7f93141077b0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f9304009990 tx=0x7f9304009ca0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.466+0000 7f9319011700 1 -- 172.21.15.7:0/674234302 shutdown_connections 2024-06-10T01:19:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.466+0000 7f9319011700 1 --2- 172.21.15.7:0/674234302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9314101d30 0x7f931410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.466+0000 7f9319011700 1 --2- 172.21.15.7:0/674234302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93141073d0 0x7f93141077b0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.466+0000 7f9319011700 1 --2- 172.21.15.7:0/674234302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9314101390 0x7f93141017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.466+0000 7f9319011700 1 -- 172.21.15.7:0/674234302 >> 172.21.15.7:0/674234302 conn(0x7f93140782c0 msgr2=0x7f93140786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.466+0000 7f9319011700 1 -- 172.21.15.7:0/674234302 shutdown_connections 2024-06-10T01:19:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.466+0000 7f9319011700 1 -- 172.21.15.7:0/674234302 wait complete. 2024-06-10T01:19:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.466+0000 7f9319011700 1 Processor -- start 2024-06-10T01:19:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.467+0000 7f9319011700 1 -- start start 2024-06-10T01:19:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.467+0000 7f9319011700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9314101390 0x7f931419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.467+0000 7f9319011700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9314101d30 0x7f931419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.467+0000 7f9319011700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93141073d0 0x7f9314198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.467+0000 7f9319011700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9314111660 con 0x7f9314101390 2024-06-10T01:19:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.467+0000 7f9319011700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f93141114e0 con 0x7f9314101d30 2024-06-10T01:19:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.467+0000 7f9319011700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f93141117e0 con 0x7f93141073d0 2024-06-10T01:19:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.467+0000 7f9312d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9314101390 0x7f931419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.467+0000 7f931259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9314101d30 0x7f931419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.467+0000 7f9312d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9314101390 0x7f931419e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55470/0 (socket says 172.21.15.7:55470) 2024-06-10T01:19:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.467+0000 7f9312d9d700 1 -- 172.21.15.7:0/3095812686 learned_addr learned my addr 172.21.15.7:0/3095812686 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.467+0000 7f931359e700 1 --2- 172.21.15.7:0/3095812686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93141073d0 0x7f9314198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.467+0000 7f9312d9d700 1 -- 172.21.15.7:0/3095812686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93141073d0 msgr2=0x7f9314198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.467+0000 7f9312d9d700 1 --2- 172.21.15.7:0/3095812686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93141073d0 0x7f9314198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.467+0000 7f9312d9d700 1 -- 172.21.15.7:0/3095812686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9314101d30 msgr2=0x7f931419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.467+0000 7f9312d9d700 1 --2- 172.21.15.7:0/3095812686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9314101d30 0x7f931419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.467+0000 7f9312d9d700 1 -- 172.21.15.7:0/3095812686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93141991a0 con 0x7f9314101390 2024-06-10T01:19:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.467+0000 7f931259c700 1 --2- 172.21.15.7:0/3095812686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9314101d30 0x7f931419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.468+0000 7f9312d9d700 1 --2- 172.21.15.7:0/3095812686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9314101390 0x7f931419e880 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f9304005f50 tx=0x7f93040047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.468+0000 7f92fbfff700 1 -- 172.21.15.7:0/3095812686 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304027070 con 0x7f9314101390 2024-06-10T01:19:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.468+0000 7f9319011700 1 -- 172.21.15.7:0/3095812686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9314199430 con 0x7f9314101390 2024-06-10T01:19:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.468+0000 7f92fbfff700 1 -- 172.21.15.7:0/3095812686 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f930400fde0 con 0x7f9314101390 2024-06-10T01:19:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.468+0000 7f92fbfff700 1 -- 172.21.15.7:0/3095812686 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93040173f0 con 0x7f9314101390 2024-06-10T01:19:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.468+0000 7f9319011700 1 -- 172.21.15.7:0/3095812686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93141a5750 con 0x7f9314101390 2024-06-10T01:19:06.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.471+0000 7f92fbfff700 1 -- 172.21.15.7:0/3095812686 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f930402a070 con 0x7f9314101390 2024-06-10T01:19:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.472+0000 7f92fbfff700 1 --2- 172.21.15.7:0/3095812686 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f92fc0a2ca0 0x7f92fc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.472+0000 7f92fbfff700 1 -- 172.21.15.7:0/3095812686 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f93040d0090 con 0x7f9314101390 2024-06-10T01:19:06.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.472+0000 7f931259c700 1 --2- 172.21.15.7:0/3095812686 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f92fc0a2ca0 0x7f92fc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:06.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.472+0000 7f9323894b80 1 Processor -- start 2024-06-10T01:19:06.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.472+0000 7f9323894b80 1 -- start start 2024-06-10T01:19:06.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.472+0000 7f931259c700 1 --2- 172.21.15.7:0/3095812686 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f92fc0a2ca0 0x7f92fc0a5160 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f93140ffb40 tx=0x7f9308009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:06.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.473+0000 7f9323894b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2e2341640 0x55c2e2341a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:06.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.473+0000 7f9323894b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2e2341f60 0x55c2e233b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:06.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.473+0000 7f9323894b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2e233bf20 0x55c2e233c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:06.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.473+0000 7f9323894b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c2e228d810 con 0x55c2e2341640 2024-06-10T01:19:06.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.473+0000 7f9323894b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c2e2235720 con 0x55c2e233bf20 2024-06-10T01:19:06.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.473+0000 7f9323894b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c2e22897d0 con 0x55c2e2341f60 2024-06-10T01:19:06.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.473+0000 7f931259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2e2341f60 0x55c2e233b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:06.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.473+0000 7f931359e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2e233bf20 0x55c2e233c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:06.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.473+0000 7f931259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2e2341f60 0x55c2e233b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:48104/0 (socket says 172.21.15.7:48104) 2024-06-10T01:19:06.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.473+0000 7f931259c700 1 -- 172.21.15.7:0/2057489925 learned_addr learned my addr 172.21.15.7:0/2057489925 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:06.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.473+0000 7f9312d9d700 1 --2- 172.21.15.7:0/2057489925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2e2341640 0x55c2e2341a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:06.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.473+0000 7f931359e700 1 -- 172.21.15.7:0/2057489925 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2e2341f60 msgr2=0x55c2e233b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:06.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.473+0000 7f931359e700 1 --2- 172.21.15.7:0/2057489925 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2e2341f60 0x55c2e233b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.473+0000 7f931359e700 1 -- 172.21.15.7:0/2057489925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2e2341640 msgr2=0x55c2e2341a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:06.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.473+0000 7f931359e700 1 --2- 172.21.15.7:0/2057489925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2e2341640 0x55c2e2341a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.473+0000 7f931359e700 1 -- 172.21.15.7:0/2057489925 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2e222bf50 con 0x55c2e233bf20 2024-06-10T01:19:06.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.473+0000 7f931259c700 1 --2- 172.21.15.7:0/2057489925 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2e2341f60 0x55c2e233b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:06.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.473+0000 7f931359e700 1 --2- 172.21.15.7:0/2057489925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2e233bf20 0x55c2e233c340 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f930c0060a0 tx=0x7f930c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa5b2c0a05e7abd4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:06.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.474+0000 7f92f97fa700 1 -- 172.21.15.7:0/2057489925 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c004bb0 con 0x55c2e233bf20 2024-06-10T01:19:06.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.474+0000 7f92f97fa700 1 -- 172.21.15.7:0/2057489925 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f930c004d50 con 0x55c2e233bf20 2024-06-10T01:19:06.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.474+0000 7f92f97fa700 1 -- 172.21.15.7:0/2057489925 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c004030 con 0x55c2e233bf20 2024-06-10T01:19:06.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.474+0000 7f9323894b80 1 -- 172.21.15.7:0/2057489925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2e233bf20 msgr2=0x55c2e233c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:06.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.474+0000 7f9323894b80 1 --2- 172.21.15.7:0/2057489925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2e233bf20 0x55c2e233c340 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f930c0060a0 tx=0x7f930c009920 comp rx=0 tx=0).stop 2024-06-10T01:19:06.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.475+0000 7f9323894b80 1 -- 172.21.15.7:0/2057489925 shutdown_connections 2024-06-10T01:19:06.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.475+0000 7f9323894b80 1 --2- 172.21.15.7:0/2057489925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2e233bf20 0x55c2e233c340 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.475+0000 7f9323894b80 1 --2- 172.21.15.7:0/2057489925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2e2341640 0x55c2e2341a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.475+0000 7f9323894b80 1 --2- 172.21.15.7:0/2057489925 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2e2341f60 0x55c2e233b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.475+0000 7f9323894b80 1 -- 172.21.15.7:0/2057489925 >> 172.21.15.7:0/2057489925 conn(0x55c2e2266cc0 msgr2=0x55c2e2336850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:06.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.475+0000 7f9323894b80 1 -- 172.21.15.7:0/2057489925 shutdown_connections 2024-06-10T01:19:06.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.475+0000 7f9323894b80 1 -- 172.21.15.7:0/2057489925 wait complete. 2024-06-10T01:19:06.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.475+0000 7f9323894b80 1 -- start start 2024-06-10T01:19:06.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.475+0000 7f9323894b80 1 Processor -- start 2024-06-10T01:19:06.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.475+0000 7f9323894b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2e233bf20 0x55c2e2347cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:06.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.475+0000 7f9323894b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2e2341640 0x55c2e2344fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:06.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.475+0000 7f9323894b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2e2341f60 0x55c2e23454f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:06.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.475+0000 7f9323894b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c2e23489d0 con 0x55c2e2341640 2024-06-10T01:19:06.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.476+0000 7f9323894b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c2e222dbf0 con 0x55c2e2341f60 2024-06-10T01:19:06.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.476+0000 7f9323894b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c2e2348b50 con 0x55c2e233bf20 2024-06-10T01:19:06.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.476+0000 7f9323894b80 10 client.0 _ll_register_callbacks cb 0x55c2e2227920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:19:06.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.476+0000 7f931359e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2e2341f60 0x55c2e23454f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:06.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.476+0000 7f9312d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2e233bf20 0x55c2e2347cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:06.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.476+0000 7f931359e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2e2341f60 0x55c2e23454f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:40172/0 (socket says 172.21.15.7:40172) 2024-06-10T01:19:06.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.476+0000 7f931359e700 1 -- 172.21.15.7:0/293360295 learned_addr learned my addr 172.21.15.7:0/293360295 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:06.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.476+0000 7f931259c700 1 --2- 172.21.15.7:0/293360295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2e2341640 0x55c2e2344fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:06.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.476+0000 7f9312d9d700 1 -- 172.21.15.7:0/293360295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2e2341f60 msgr2=0x55c2e23454f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:06.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.476+0000 7f9312d9d700 1 --2- 172.21.15.7:0/293360295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2e2341f60 0x55c2e23454f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.476+0000 7f9312d9d700 1 -- 172.21.15.7:0/293360295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2e2341640 msgr2=0x55c2e2344fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:06.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.476+0000 7f9312d9d700 1 --2- 172.21.15.7:0/293360295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2e2341640 0x55c2e2344fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.476+0000 7f9312d9d700 1 -- 172.21.15.7:0/293360295 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2e21dc8d0 con 0x55c2e233bf20 2024-06-10T01:19:06.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.476+0000 7f931359e700 1 --2- 172.21.15.7:0/293360295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2e2341f60 0x55c2e23454f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:06.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.476+0000 7f931259c700 1 --2- 172.21.15.7:0/293360295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2e2341640 0x55c2e2344fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:06.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.476+0000 7f9312d9d700 1 --2- 172.21.15.7:0/293360295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2e233bf20 0x55c2e2347cb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9304006010 tx=0x7f9304022a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc50052f01cbeea6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:06.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.476+0000 7f92f6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:19:06.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.476+0000 7f92f6ff5700 1 -- 172.21.15.7:0/293360295 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93040b5060 con 0x55c2e233bf20 2024-06-10T01:19:06.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.477+0000 7f9323894b80 1 -- 172.21.15.7:0/293360295 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f930c004d50 con 0x55c2e233bf20 2024-06-10T01:19:06.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.477+0000 7f92f6ff5700 1 -- 172.21.15.7:0/293360295 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f93040a1bd0 con 0x55c2e233bf20 2024-06-10T01:19:06.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.477+0000 7f92f6ff5700 1 -- 172.21.15.7:0/293360295 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f93040870b0 con 0x55c2e233bf20 2024-06-10T01:19:06.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.477+0000 7f9323894b80 10 client.28057 fetch_fsmap learned FSMap version 40 2024-06-10T01:19:06.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.477+0000 7f9323894b80 1 -- 172.21.15.7:0/293360295 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55c2e2348a80 con 0x55c2e233bf20 2024-06-10T01:19:06.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.477+0000 7f92f6ff5700 1 -- 172.21.15.7:0/293360295 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f930c004d50 con 0x55c2e233bf20 2024-06-10T01:19:06.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.477+0000 7f92f6ff5700 1 -- 172.21.15.7:0/293360295 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f93040a4de0 con 0x55c2e233bf20 2024-06-10T01:19:06.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.477+0000 7f9323894b80 10 client.28057 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:19:06.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.477+0000 7f9323894b80 10 client.28057 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:19:06.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.477+0000 7f9323894b80 20 client.28057 populate_metadata read hostname 'smithi007' 2024-06-10T01:19:06.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.477+0000 7f9323894b80 1 --2- 172.21.15.7:0/293360295 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55c2e23557f0 0x55c2e2357c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:06.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.477+0000 7f9323894b80 4 client.28057 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:19:06.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.477+0000 7f9323894b80 1 -- 172.21.15.7:0/293360295 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2e229ad10 con 0x55c2e23557f0 2024-06-10T01:19:06.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.477+0000 7f931259c700 1 --2- 172.21.15.7:0/293360295 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55c2e23557f0 0x55c2e2357c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:06.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.478+0000 7f92f6ff5700 10 client.28057 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:19:06.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.478+0000 7f931259c700 1 --2- 172.21.15.7:0/293360295 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55c2e23557f0 0x55c2e2357c20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3c08d6c5e076281d server_cookie=e8db085264966056 in_seq=0 out_seq=0 2024-06-10T01:19:06.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.478+0000 7f92f6ff5700 1 -- 172.21.15.7:0/293360295 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c2e229ad10 con 0x55c2e23557f0 2024-06-10T01:19:06.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.478+0000 7f92f6ff5700 10 client.28057 handle_command_reply: tid=0 2024-06-10T01:19:06.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.479+0000 7f9323894b80 1 client.28057 shutdown 2024-06-10T01:19:06.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.479+0000 7f9323894b80 1 -- 172.21.15.7:0/293360295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2e233bf20 msgr2=0x55c2e2347cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:06.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.479+0000 7f9323894b80 1 --2- 172.21.15.7:0/293360295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2e233bf20 0x55c2e2347cb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9304006010 tx=0x7f9304022a30 comp rx=0 tx=0).stop 2024-06-10T01:19:06.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.479+0000 7f9323894b80 1 -- 172.21.15.7:0/293360295 shutdown_connections 2024-06-10T01:19:06.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.479+0000 7f9323894b80 1 --2- 172.21.15.7:0/293360295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2e2341f60 0x55c2e23454f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.479+0000 7f9323894b80 1 --2- 172.21.15.7:0/293360295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2e233bf20 0x55c2e2347cb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.479+0000 7f9323894b80 1 --2- 172.21.15.7:0/293360295 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55c2e23557f0 0x55c2e2357c20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.479+0000 7f9323894b80 1 --2- 172.21.15.7:0/293360295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2e2341640 0x55c2e2344fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.479+0000 7f9323894b80 1 -- 172.21.15.7:0/293360295 >> 172.21.15.7:0/293360295 conn(0x55c2e2266cc0 msgr2=0x55c2e233a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:06.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.479+0000 7f92f6ff5700 0 client.28057 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:19:06.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.479+0000 7f9323894b80 1 -- 172.21.15.7:0/293360295 shutdown_connections 2024-06-10T01:19:06.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.479+0000 7f9323894b80 1 -- 172.21.15.7:0/293360295 wait complete. 2024-06-10T01:19:06.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.479+0000 7f9323894b80 20 client.28057 trim_cache size 0 max 16384 2024-06-10T01:19:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.496+0000 7f9323894b80 1 Processor -- start 2024-06-10T01:19:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.496+0000 7f9323894b80 1 -- start start 2024-06-10T01:19:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.496+0000 7f9323894b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2e233bf20 0x55c2e233b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.496+0000 7f9323894b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2e2341640 0x55c2e23369f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.496+0000 7f9323894b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2e2341f60 0x55c2e2342a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.496+0000 7f9323894b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f93141ae7e0 con 0x55c2e2341640 2024-06-10T01:19:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.496+0000 7f9323894b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f93141ac070 con 0x55c2e233bf20 2024-06-10T01:19:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.496+0000 7f9323894b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f93141a9ec0 con 0x55c2e2341f60 2024-06-10T01:19:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.496+0000 7f931259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2e2341640 0x55c2e23369f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.496+0000 7f931259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2e2341640 0x55c2e23369f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55512/0 (socket says 172.21.15.7:55512) 2024-06-10T01:19:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.496+0000 7f931359e700 1 --2- 172.21.15.7:0/3579107440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2e2341f60 0x55c2e2342a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.496+0000 7f931259c700 1 -- 172.21.15.7:0/3579107440 learned_addr learned my addr 172.21.15.7:0/3579107440 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.496+0000 7f9312d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2e233bf20 0x55c2e233b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:06.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.497+0000 7f931259c700 1 -- 172.21.15.7:0/3579107440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2e2341f60 msgr2=0x55c2e2342a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:06.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.497+0000 7f931259c700 1 --2- 172.21.15.7:0/3579107440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2e2341f60 0x55c2e2342a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.497+0000 7f931259c700 1 -- 172.21.15.7:0/3579107440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2e233bf20 msgr2=0x55c2e233b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:06.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.497+0000 7f931259c700 1 --2- 172.21.15.7:0/3579107440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2e233bf20 0x55c2e233b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.497+0000 7f931259c700 1 -- 172.21.15.7:0/3579107440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2e2338080 con 0x55c2e2341640 2024-06-10T01:19:06.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.497+0000 7f931359e700 1 --2- 172.21.15.7:0/3579107440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2e2341f60 0x55c2e2342a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:06.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.497+0000 7f9312d9d700 1 --2- 172.21.15.7:0/3579107440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2e233bf20 0x55c2e233b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:06.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.497+0000 7f931259c700 1 --2- 172.21.15.7:0/3579107440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2e2341640 0x55c2e23369f0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f93080041d0 tx=0x7f9308014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=879c67763c29a0c1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:06.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.497+0000 7f92f8ff9700 1 -- 172.21.15.7:0/3579107440 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930801f960 con 0x55c2e2341640 2024-06-10T01:19:06.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.497+0000 7f92f8ff9700 1 -- 172.21.15.7:0/3579107440 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9308029470 con 0x55c2e2341640 2024-06-10T01:19:06.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.497+0000 7f92f8ff9700 1 -- 172.21.15.7:0/3579107440 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930800e610 con 0x55c2e2341640 2024-06-10T01:19:06.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.498+0000 7f9323894b80 1 -- 172.21.15.7:0/3579107440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2e2341640 msgr2=0x55c2e23369f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:06.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.498+0000 7f9323894b80 1 --2- 172.21.15.7:0/3579107440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2e2341640 0x55c2e23369f0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f93080041d0 tx=0x7f9308014000 comp rx=0 tx=0).stop 2024-06-10T01:19:06.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.498+0000 7f9323894b80 1 -- 172.21.15.7:0/3579107440 shutdown_connections 2024-06-10T01:19:06.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.498+0000 7f9323894b80 1 --2- 172.21.15.7:0/3579107440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2e2341f60 0x55c2e2342a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.498+0000 7f9323894b80 1 --2- 172.21.15.7:0/3579107440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2e233bf20 0x55c2e233b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.498+0000 7f9323894b80 1 --2- 172.21.15.7:0/3579107440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2e2341640 0x55c2e23369f0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.498+0000 7f9323894b80 1 -- 172.21.15.7:0/3579107440 >> 172.21.15.7:0/3579107440 conn(0x55c2e2266cc0 msgr2=0x55c2e2342340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:06.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.498+0000 7f9323894b80 1 -- 172.21.15.7:0/3579107440 shutdown_connections 2024-06-10T01:19:06.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.498+0000 7f9323894b80 1 -- 172.21.15.7:0/3579107440 wait complete. 2024-06-10T01:19:06.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.498+0000 7f9323894b80 1 -- start start 2024-06-10T01:19:06.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.498+0000 7f9323894b80 1 Processor -- start 2024-06-10T01:19:06.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.499+0000 7f9323894b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2e233bf20 0x55c2e234de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:06.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.499+0000 7f9323894b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2e2341640 0x55c2e234e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:06.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.499+0000 7f9323894b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2e2341f60 0x55c2e235c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:06.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.499+0000 7f9323894b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f93141aa530 con 0x55c2e2341640 2024-06-10T01:19:06.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.499+0000 7f9323894b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f93141ab9a0 con 0x55c2e233bf20 2024-06-10T01:19:06.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.499+0000 7f9323894b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f93141aac00 con 0x55c2e2341f60 2024-06-10T01:19:06.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.499+0000 7f9323894b80 10 client.0 _ll_register_callbacks cb 0x55c2e2227920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:19:06.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.499+0000 7f931259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2e2341640 0x55c2e234e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:06.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.499+0000 7f9312d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2e233bf20 0x55c2e234de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:06.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.499+0000 7f931259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2e2341640 0x55c2e234e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55528/0 (socket says 172.21.15.7:55528) 2024-06-10T01:19:06.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.499+0000 7f931259c700 1 -- 172.21.15.7:0/654073472 learned_addr learned my addr 172.21.15.7:0/654073472 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:06.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.499+0000 7f931359e700 1 --2- 172.21.15.7:0/654073472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2e2341f60 0x55c2e235c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:06.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.499+0000 7f931259c700 1 -- 172.21.15.7:0/654073472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2e2341f60 msgr2=0x55c2e235c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:06.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.499+0000 7f931259c700 1 --2- 172.21.15.7:0/654073472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2e2341f60 0x55c2e235c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.499+0000 7f931259c700 1 -- 172.21.15.7:0/654073472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2e233bf20 msgr2=0x55c2e234de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:06.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.499+0000 7f931259c700 1 --2- 172.21.15.7:0/654073472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2e233bf20 0x55c2e234de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.499+0000 7f931259c700 1 -- 172.21.15.7:0/654073472 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2e2342f90 con 0x55c2e2341640 2024-06-10T01:19:06.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.499+0000 7f931359e700 1 --2- 172.21.15.7:0/654073472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2e2341f60 0x55c2e235c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:19:06.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.499+0000 7f9312d9d700 1 --2- 172.21.15.7:0/654073472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2e233bf20 0x55c2e234de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:06.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.500+0000 7f931259c700 1 --2- 172.21.15.7:0/654073472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2e2341640 0x55c2e234e350 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f93080057e0 tx=0x7f9308029bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d8c7ac48d7b8ec server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:06.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.500+0000 7f92f6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:19:06.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.500+0000 7f92f6ff5700 1 -- 172.21.15.7:0/654073472 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9308029830 con 0x55c2e2341640 2024-06-10T01:19:06.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.500+0000 7f9323894b80 1 -- 172.21.15.7:0/654073472 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9308029470 con 0x55c2e2341640 2024-06-10T01:19:06.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.500+0000 7f92f6ff5700 1 -- 172.21.15.7:0/654073472 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f930801f760 con 0x55c2e2341640 2024-06-10T01:19:06.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.500+0000 7f92f6ff5700 1 -- 172.21.15.7:0/654073472 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9308045050 con 0x55c2e2341640 2024-06-10T01:19:06.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.500+0000 7f9323894b80 10 client.18732 fetch_fsmap learned FSMap version 40 2024-06-10T01:19:06.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.500+0000 7f9323894b80 1 -- 172.21.15.7:0/654073472 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55c2e2348020 con 0x55c2e2341640 2024-06-10T01:19:06.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.501+0000 7f92f6ff5700 1 -- 172.21.15.7:0/654073472 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9308029470 con 0x55c2e2341640 2024-06-10T01:19:06.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.501+0000 7f92f6ff5700 1 -- 172.21.15.7:0/654073472 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f930803c820 con 0x55c2e2341640 2024-06-10T01:19:06.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.501+0000 7f9323894b80 10 client.18732 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:19:06.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.501+0000 7f9323894b80 10 client.18732 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:19:06.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.501+0000 7f9323894b80 20 client.18732 populate_metadata read hostname 'smithi007' 2024-06-10T01:19:06.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.501+0000 7f9323894b80 1 --2- 172.21.15.7:0/654073472 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55c2e23bfe40 0x55c2e23c2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:06.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.501+0000 7f9323894b80 4 client.18732 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:19:06.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.501+0000 7f9323894b80 1 -- 172.21.15.7:0/654073472 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c2e2295290 con 0x55c2e23bfe40 2024-06-10T01:19:06.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.501+0000 7f9312d9d700 1 --2- 172.21.15.7:0/654073472 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55c2e23bfe40 0x55c2e23c2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:06.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.501+0000 7f92f6ff5700 10 client.18732 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:19:06.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.501+0000 7f9312d9d700 1 --2- 172.21.15.7:0/654073472 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55c2e23bfe40 0x55c2e23c2260 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dd0bd76f74adc7f5 server_cookie=f21f9c6b2a31c26c in_seq=0 out_seq=0 2024-06-10T01:19:06.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.502+0000 7f92f6ff5700 1 -- 172.21.15.7:0/654073472 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c2e2295290 con 0x55c2e23bfe40 2024-06-10T01:19:06.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.502+0000 7f92f6ff5700 10 client.18732 handle_command_reply: tid=0 2024-06-10T01:19:06.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.502+0000 7f9323894b80 1 client.18732 shutdown 2024-06-10T01:19:06.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.502+0000 7f9323894b80 1 -- 172.21.15.7:0/654073472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2e2341640 msgr2=0x55c2e234e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:06.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.502+0000 7f9323894b80 1 --2- 172.21.15.7:0/654073472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2e2341640 0x55c2e234e350 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f93080057e0 tx=0x7f9308029bf0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.502+0000 7f9323894b80 1 -- 172.21.15.7:0/654073472 shutdown_connections 2024-06-10T01:19:06.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.502+0000 7f9323894b80 1 --2- 172.21.15.7:0/654073472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2e2341f60 0x55c2e235c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.502+0000 7f9323894b80 1 --2- 172.21.15.7:0/654073472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2e233bf20 0x55c2e234de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.502+0000 7f9323894b80 1 --2- 172.21.15.7:0/654073472 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55c2e23bfe40 0x55c2e23c2260 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.502+0000 7f9323894b80 1 --2- 172.21.15.7:0/654073472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2e2341640 0x55c2e234e350 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.502+0000 7f9323894b80 1 -- 172.21.15.7:0/654073472 >> 172.21.15.7:0/654073472 conn(0x55c2e2266cc0 msgr2=0x55c2e2343490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:06.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.502+0000 7f92f6ff5700 0 client.18732 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:19:06.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.502+0000 7f9323894b80 1 -- 172.21.15.7:0/654073472 shutdown_connections 2024-06-10T01:19:06.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.502+0000 7f9323894b80 1 -- 172.21.15.7:0/654073472 wait complete. 2024-06-10T01:19:06.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.502+0000 7f9323894b80 20 client.18732 trim_cache size 0 max 16384 2024-06-10T01:19:06.501 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:19:06.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.503+0000 7f9319011700 1 -- 172.21.15.7:0/3095812686 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f92fc0a2ca0 msgr2=0x7f92fc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:06.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.503+0000 7f9319011700 1 --2- 172.21.15.7:0/3095812686 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f92fc0a2ca0 0x7f92fc0a5160 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f93140ffb40 tx=0x7f9308009260 comp rx=0 tx=0).stop 2024-06-10T01:19:06.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.503+0000 7f9319011700 1 -- 172.21.15.7:0/3095812686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9314101390 msgr2=0x7f931419e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:06.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.503+0000 7f9319011700 1 --2- 172.21.15.7:0/3095812686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9314101390 0x7f931419e880 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f9304005f50 tx=0x7f93040047b0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.503+0000 7f9319011700 1 -- 172.21.15.7:0/3095812686 shutdown_connections 2024-06-10T01:19:06.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.503+0000 7f9319011700 1 --2- 172.21.15.7:0/3095812686 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f92fc0a2ca0 0x7f92fc0a5160 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.503+0000 7f9319011700 1 --2- 172.21.15.7:0/3095812686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93141073d0 0x7f9314198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.503+0000 7f9319011700 1 --2- 172.21.15.7:0/3095812686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9314101390 0x7f931419e880 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.503+0000 7f9319011700 1 --2- 172.21.15.7:0/3095812686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9314101d30 0x7f931419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:06.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.503+0000 7f9319011700 1 -- 172.21.15.7:0/3095812686 >> 172.21.15.7:0/3095812686 conn(0x7f93140782c0 msgr2=0x7f9314100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:06.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.503+0000 7f9319011700 1 -- 172.21.15.7:0/3095812686 shutdown_connections 2024-06-10T01:19:06.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:06.503+0000 7f9319011700 1 -- 172.21.15.7:0/3095812686 wait complete. 2024-06-10T01:19:06.757 INFO:tasks.workunit.client.0.smithi007.stdout: 1 966384 7.33 MB/sec execute 587 sec latency 16.425 ms 2024-06-10T01:19:06.956 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-06-10T01:19:06.957 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:19:07.155 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:19:07.222 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:19:07.757 INFO:tasks.workunit.client.0.smithi007.stdout: 1 967937 7.33 MB/sec execute 588 sec latency 15.230 ms 2024-06-10T01:19:07.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:07 smithi007 ceph-mon[32398]: pgmap v825: 129 pgs: 129 active+clean; 146 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 2.0 MiB/s wr, 269 op/s 2024-06-10T01:19:07.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:07 smithi078 ceph-mon[36683]: pgmap v825: 129 pgs: 129 active+clean; 146 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 2.0 MiB/s wr, 269 op/s 2024-06-10T01:19:07.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:07 smithi175 ceph-mon[36636]: pgmap v825: 129 pgs: 129 active+clean; 146 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 2.0 MiB/s wr, 269 op/s 2024-06-10T01:19:08.757 INFO:tasks.workunit.client.0.smithi007.stdout: 1 970214 7.34 MB/sec execute 589 sec latency 56.257 ms 2024-06-10T01:19:09.089 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:19:09.646 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:09 smithi007 ceph-mon[32398]: pgmap v826: 129 pgs: 129 active+clean; 147 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 1.9 MiB/s wr, 253 op/s 2024-06-10T01:19:09.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.762+0000 7f043f644700 1 -- 172.21.15.7:0/3218871050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04381015f0 msgr2=0x7f0438107150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:09.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.762+0000 7f043f644700 1 --2- 172.21.15.7:0/3218871050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04381015f0 0x7f0438107150 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f04280099c0 tx=0x7f0428009cd0 comp rx=0 tx=0).stop 2024-06-10T01:19:09.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.762+0000 7f043f644700 1 -- 172.21.15.7:0/3218871050 shutdown_connections 2024-06-10T01:19:09.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.762+0000 7f043f644700 1 --2- 172.21.15.7:0/3218871050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0438107880 0x7f043810f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:09.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.762+0000 7f043f644700 1 --2- 172.21.15.7:0/3218871050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0438100c40 0x7f0438101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:09.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.762+0000 7f043f644700 1 --2- 172.21.15.7:0/3218871050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04381015f0 0x7f0438107150 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:09.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.762+0000 7f043f644700 1 -- 172.21.15.7:0/3218871050 >> 172.21.15.7:0/3218871050 conn(0x7f0438077990 msgr2=0x7f04380fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:09.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.762+0000 7f043f644700 1 -- 172.21.15.7:0/3218871050 shutdown_connections 2024-06-10T01:19:09.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.762+0000 7f043f644700 1 -- 172.21.15.7:0/3218871050 wait complete. 2024-06-10T01:19:09.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.763+0000 7f043f644700 1 Processor -- start 2024-06-10T01:19:09.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.763+0000 7f043f644700 1 -- start start 2024-06-10T01:19:09.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.763+0000 7f043f644700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0438100c40 0x7f043819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:09.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.763+0000 7f043f644700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04381015f0 0x7f043819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:09.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.763+0000 7f043f644700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0438107880 0x7f0438198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:09.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.763+0000 7f043f644700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0438112790 con 0x7f0438100c40 2024-06-10T01:19:09.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.763+0000 7f043f644700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0438112610 con 0x7f04381015f0 2024-06-10T01:19:09.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.763+0000 7f043f644700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0438112910 con 0x7f0438107880 2024-06-10T01:19:09.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.763+0000 7f043dbe1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0438107880 0x7f0438198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:09.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.763+0000 7f043cbdf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04381015f0 0x7f043819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:09.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.763+0000 7f043dbe1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0438107880 0x7f0438198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:48136/0 (socket says 172.21.15.7:48136) 2024-06-10T01:19:09.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.763+0000 7f043dbe1700 1 -- 172.21.15.7:0/1939177445 learned_addr learned my addr 172.21.15.7:0/1939177445 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:09.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.763+0000 7f043d3e0700 1 --2- 172.21.15.7:0/1939177445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0438100c40 0x7f043819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:09.759 INFO:tasks.workunit.client.0.smithi007.stdout: 1 971447 7.34 MB/sec execute 590 sec latency 59.609 ms 2024-06-10T01:19:09.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.764+0000 7f043dbe1700 1 -- 172.21.15.7:0/1939177445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04381015f0 msgr2=0x7f043819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:09.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.764+0000 7f043dbe1700 1 --2- 172.21.15.7:0/1939177445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04381015f0 0x7f043819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:09.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.764+0000 7f043dbe1700 1 -- 172.21.15.7:0/1939177445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0438100c40 msgr2=0x7f043819e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:09.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.764+0000 7f043dbe1700 1 --2- 172.21.15.7:0/1939177445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0438100c40 0x7f043819e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:09.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.764+0000 7f043dbe1700 1 -- 172.21.15.7:0/1939177445 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0438199190 con 0x7f0438107880 2024-06-10T01:19:09.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.764+0000 7f043cbdf700 1 --2- 172.21.15.7:0/1939177445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04381015f0 0x7f043819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:09.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.764+0000 7f043dbe1700 1 --2- 172.21.15.7:0/1939177445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0438107880 0x7f0438198a30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f043000efc0 tx=0x7f043000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:09.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.764+0000 7f042e7fc700 1 -- 172.21.15.7:0/1939177445 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04300092e0 con 0x7f0438107880 2024-06-10T01:19:09.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.764+0000 7f043f644700 1 -- 172.21.15.7:0/1939177445 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0438199420 con 0x7f0438107880 2024-06-10T01:19:09.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.764+0000 7f042e7fc700 1 -- 172.21.15.7:0/1939177445 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0430004bb0 con 0x7f0438107880 2024-06-10T01:19:09.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.764+0000 7f042e7fc700 1 -- 172.21.15.7:0/1939177445 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0430004020 con 0x7f0438107880 2024-06-10T01:19:09.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.764+0000 7f043f644700 1 -- 172.21.15.7:0/1939177445 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04381a56f0 con 0x7f0438107880 2024-06-10T01:19:09.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.765+0000 7f043f644700 1 -- 172.21.15.7:0/1939177445 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0438069180 con 0x7f0438107880 2024-06-10T01:19:09.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.766+0000 7f042e7fc700 1 -- 172.21.15.7:0/1939177445 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f043001b030 con 0x7f0438107880 2024-06-10T01:19:09.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.767+0000 7f042e7fc700 1 --2- 172.21.15.7:0/1939177445 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f04240a2c20 0x7f04240a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:09.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.767+0000 7f043d3e0700 1 --2- 172.21.15.7:0/1939177445 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f04240a2c20 0x7f04240a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:09.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.767+0000 7f042e7fc700 1 -- 172.21.15.7:0/1939177445 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f04300cf680 con 0x7f0438107880 2024-06-10T01:19:09.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.768+0000 7f043d3e0700 1 --2- 172.21.15.7:0/1939177445 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f04240a2c20 0x7f04240a50e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0438107570 tx=0x7f0434009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:09.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:09.771+0000 7f042e7fc700 1 -- 172.21.15.7:0/1939177445 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0430022020 con 0x7f0438107880 2024-06-10T01:19:09.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:09 smithi078 ceph-mon[36683]: pgmap v826: 129 pgs: 129 active+clean; 147 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 1.9 MiB/s wr, 253 op/s 2024-06-10T01:19:09.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:09 smithi175 ceph-mon[36636]: pgmap v826: 129 pgs: 129 active+clean; 147 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 1.9 MiB/s wr, 253 op/s 2024-06-10T01:19:10.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:10.057+0000 7f043f644700 1 -- 172.21.15.7:0/1939177445 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f043802dca0 con 0x7f0438107880 2024-06-10T01:19:10.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:10.058+0000 7f042e7fc700 1 -- 172.21.15.7:0/1939177445 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11855 (secure 0 0 0) 0x7f0430010040 con 0x7f0438107880 2024-06-10T01:19:10.052 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:19:10.052 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:16:56.409130+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24451,24434,24436]},"id":1}]} 2024-06-10T01:19:10.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:10.060+0000 7f043f644700 1 -- 172.21.15.7:0/1939177445 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f04240a2c20 msgr2=0x7f04240a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:10.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:10.060+0000 7f043f644700 1 --2- 172.21.15.7:0/1939177445 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f04240a2c20 0x7f04240a50e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0438107570 tx=0x7f0434009260 comp rx=0 tx=0).stop 2024-06-10T01:19:10.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:10.060+0000 7f043f644700 1 -- 172.21.15.7:0/1939177445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0438107880 msgr2=0x7f0438198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:10.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:10.060+0000 7f043f644700 1 --2- 172.21.15.7:0/1939177445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0438107880 0x7f0438198a30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f043000efc0 tx=0x7f043000c3a0 comp rx=0 tx=0).stop 2024-06-10T01:19:10.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:10.060+0000 7f043f644700 1 -- 172.21.15.7:0/1939177445 shutdown_connections 2024-06-10T01:19:10.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:10.060+0000 7f043f644700 1 --2- 172.21.15.7:0/1939177445 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f04240a2c20 0x7f04240a50e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:10.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:10.060+0000 7f043f644700 1 --2- 172.21.15.7:0/1939177445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0438107880 0x7f0438198a30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:10.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:10.060+0000 7f043f644700 1 --2- 172.21.15.7:0/1939177445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0438100c40 0x7f043819e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:10.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:10.060+0000 7f043f644700 1 --2- 172.21.15.7:0/1939177445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04381015f0 0x7f043819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:10.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:10.060+0000 7f043f644700 1 -- 172.21.15.7:0/1939177445 >> 172.21.15.7:0/1939177445 conn(0x7f0438077990 msgr2=0x7f0438105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:10.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:10.060+0000 7f043f644700 1 -- 172.21.15.7:0/1939177445 shutdown_connections 2024-06-10T01:19:10.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:10.060+0000 7f043f644700 1 -- 172.21.15.7:0/1939177445 wait complete. 2024-06-10T01:19:10.056 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 40 2024-06-10T01:19:10.757 INFO:tasks.workunit.client.0.smithi007.stdout: 1 973827 7.35 MB/sec execute 591 sec latency 21.719 ms 2024-06-10T01:19:10.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:10 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1939177445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:19:10.817 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-06-10T01:19:10.818 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-06-10T01:19:10.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:10 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1939177445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:19:10.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:10 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1939177445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:19:11.758 INFO:tasks.workunit.client.0.smithi007.stdout: 1 975845 7.35 MB/sec execute 592 sec latency 15.590 ms 2024-06-10T01:19:11.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:11 smithi007 ceph-mon[32398]: pgmap v827: 129 pgs: 129 active+clean; 146 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 2.2 MiB/s wr, 308 op/s 2024-06-10T01:19:11.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.822+0000 7f87f33bf700 1 -- 172.21.15.7:0/1618392385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87ec101d50 msgr2=0x7f87ec10e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:11.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.822+0000 7f87f33bf700 1 --2- 172.21.15.7:0/1618392385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87ec101d50 0x7f87ec10e240 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f87e400b0a0 tx=0x7f87e400b3b0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.822+0000 7f87f33bf700 1 -- 172.21.15.7:0/1618392385 shutdown_connections 2024-06-10T01:19:11.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.822+0000 7f87f33bf700 1 --2- 172.21.15.7:0/1618392385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87ec101d50 0x7f87ec10e240 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.822+0000 7f87f33bf700 1 --2- 172.21.15.7:0/1618392385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87ec107400 0x7f87ec1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.822+0000 7f87f33bf700 1 --2- 172.21.15.7:0/1618392385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87ec101390 0x7f87ec101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.822+0000 7f87f33bf700 1 -- 172.21.15.7:0/1618392385 >> 172.21.15.7:0/1618392385 conn(0x7f87ec078340 msgr2=0x7f87ec078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:11.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.822+0000 7f87f33bf700 1 -- 172.21.15.7:0/1618392385 shutdown_connections 2024-06-10T01:19:11.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.822+0000 7f87f33bf700 1 -- 172.21.15.7:0/1618392385 wait complete. 2024-06-10T01:19:11.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.823+0000 7f87f33bf700 1 Processor -- start 2024-06-10T01:19:11.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.823+0000 7f87f33bf700 1 -- start start 2024-06-10T01:19:11.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.823+0000 7f87f33bf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87ec101390 0x7f87ec19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:11.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.823+0000 7f87f33bf700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87ec101d50 0x7f87ec19f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:11.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.823+0000 7f87f33bf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87ec107400 0x7f87ec1a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:11.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.823+0000 7f87f33bf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec1105d0 con 0x7f87ec107400 2024-06-10T01:19:11.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.823+0000 7f87f33bf700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec110450 con 0x7f87ec101390 2024-06-10T01:19:11.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.823+0000 7f87f33bf700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec110750 con 0x7f87ec101d50 2024-06-10T01:19:11.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.824+0000 7f87f115b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87ec101390 0x7f87ec19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:11.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.824+0000 7f87f195c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87ec107400 0x7f87ec1a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:11.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.824+0000 7f87f115b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87ec101390 0x7f87ec19f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:48834/0 (socket says 172.21.15.7:48834) 2024-06-10T01:19:11.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.824+0000 7f87f115b700 1 -- 172.21.15.7:0/1455283961 learned_addr learned my addr 172.21.15.7:0/1455283961 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:11.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.824+0000 7f87f095a700 1 --2- 172.21.15.7:0/1455283961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87ec101d50 0x7f87ec19f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:11.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.824+0000 7f87f115b700 1 -- 172.21.15.7:0/1455283961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87ec101d50 msgr2=0x7f87ec19f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:11.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.824+0000 7f87f115b700 1 --2- 172.21.15.7:0/1455283961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87ec101d50 0x7f87ec19f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.824+0000 7f87f115b700 1 -- 172.21.15.7:0/1455283961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87ec107400 msgr2=0x7f87ec1a3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:11.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.824+0000 7f87f115b700 1 --2- 172.21.15.7:0/1455283961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87ec107400 0x7f87ec1a3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.824+0000 7f87f115b700 1 -- 172.21.15.7:0/1455283961 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87ec1a4060 con 0x7f87ec101390 2024-06-10T01:19:11.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.824+0000 7f87f115b700 1 --2- 172.21.15.7:0/1455283961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87ec101390 0x7f87ec19f050 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f87e8007a00 tx=0x7f87e8007d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:11.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.824+0000 7f87e27fc700 1 -- 172.21.15.7:0/1455283961 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e8007e40 con 0x7f87ec101390 2024-06-10T01:19:11.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.825+0000 7f87e27fc700 1 -- 172.21.15.7:0/1455283961 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87e8004500 con 0x7f87ec101390 2024-06-10T01:19:11.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.825+0000 7f87f33bf700 1 -- 172.21.15.7:0/1455283961 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87ec1a4350 con 0x7f87ec101390 2024-06-10T01:19:11.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.825+0000 7f87e27fc700 1 -- 172.21.15.7:0/1455283961 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e8003b40 con 0x7f87ec101390 2024-06-10T01:19:11.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.825+0000 7f87f33bf700 1 -- 172.21.15.7:0/1455283961 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87ec1abbd0 con 0x7f87ec101390 2024-06-10T01:19:11.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.828+0000 7f87e27fc700 1 -- 172.21.15.7:0/1455283961 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f87e801b020 con 0x7f87ec101390 2024-06-10T01:19:11.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.828+0000 7f87e27fc700 1 --2- 172.21.15.7:0/1455283961 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f87d80a2c20 0x7f87d80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:11.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.828+0000 7f87f095a700 1 --2- 172.21.15.7:0/1455283961 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f87d80a2c20 0x7f87d80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:11.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.828+0000 7f87e27fc700 1 -- 172.21.15.7:0/1455283961 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f87e80ce490 con 0x7f87ec101390 2024-06-10T01:19:11.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.829+0000 7f87f095a700 1 --2- 172.21.15.7:0/1455283961 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f87d80a2c20 0x7f87d80a50e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f87ec078c80 tx=0x7f87dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:11.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.830+0000 7f87fdc42b80 1 Processor -- start 2024-06-10T01:19:11.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.830+0000 7f87fdc42b80 1 -- start start 2024-06-10T01:19:11.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.830+0000 7f87fdc42b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561595b49f60 0x561595b4a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:11.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.830+0000 7f87fdc42b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561595b4a880 0x561595b44260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:11.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.830+0000 7f87fdc42b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561595b447a0 0x561595b44bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:11.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.830+0000 7f87fdc42b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561595a36f60 con 0x561595b447a0 2024-06-10T01:19:11.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.830+0000 7f87fdc42b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561595a95850 con 0x561595b4a880 2024-06-10T01:19:11.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.830+0000 7f87fdc42b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5615959fd460 con 0x561595b49f60 2024-06-10T01:19:11.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.831+0000 7f87f115b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561595b49f60 0x561595b4a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:11.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.831+0000 7f87f115b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561595b49f60 0x561595b4a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47574/0 (socket says 172.21.15.7:47574) 2024-06-10T01:19:11.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.831+0000 7f87f115b700 1 -- 172.21.15.7:0/2505584252 learned_addr learned my addr 172.21.15.7:0/2505584252 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:11.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.831+0000 7f87f195c700 1 --2- 172.21.15.7:0/2505584252 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561595b447a0 0x561595b44bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:11.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.831+0000 7f87f115b700 1 -- 172.21.15.7:0/2505584252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561595b4a880 msgr2=0x561595b44260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:19:11.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.831+0000 7f87f115b700 1 --2- 172.21.15.7:0/2505584252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561595b4a880 0x561595b44260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.831+0000 7f87f115b700 1 -- 172.21.15.7:0/2505584252 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561595b447a0 msgr2=0x561595b44bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:11.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.831+0000 7f87f115b700 1 --2- 172.21.15.7:0/2505584252 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561595b447a0 0x561595b44bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.831+0000 7f87f115b700 1 -- 172.21.15.7:0/2505584252 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615959e48d0 con 0x561595b49f60 2024-06-10T01:19:11.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.831+0000 7f87f115b700 1 --2- 172.21.15.7:0/2505584252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561595b49f60 0x561595b4a340 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f87e8097340 tx=0x7f87e809a850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=578199029cc3e294 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:11.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.831+0000 7f87d77fe700 1 -- 172.21.15.7:0/2505584252 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e809ed20 con 0x561595b49f60 2024-06-10T01:19:11.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.831+0000 7f87d77fe700 1 -- 172.21.15.7:0/2505584252 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87e80a35e0 con 0x561595b49f60 2024-06-10T01:19:11.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.832+0000 7f87d77fe700 1 -- 172.21.15.7:0/2505584252 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e80a3150 con 0x561595b49f60 2024-06-10T01:19:11.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.832+0000 7f87fdc42b80 1 -- 172.21.15.7:0/2505584252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561595b49f60 msgr2=0x561595b4a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:11.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.832+0000 7f87fdc42b80 1 --2- 172.21.15.7:0/2505584252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561595b49f60 0x561595b4a340 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f87e8097340 tx=0x7f87e809a850 comp rx=0 tx=0).stop 2024-06-10T01:19:11.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.832+0000 7f87fdc42b80 1 -- 172.21.15.7:0/2505584252 shutdown_connections 2024-06-10T01:19:11.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.832+0000 7f87fdc42b80 1 --2- 172.21.15.7:0/2505584252 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561595b447a0 0x561595b44bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.832+0000 7f87fdc42b80 1 --2- 172.21.15.7:0/2505584252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561595b49f60 0x561595b4a340 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.832+0000 7f87fdc42b80 1 --2- 172.21.15.7:0/2505584252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561595b4a880 0x561595b44260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.832+0000 7f87fdc42b80 1 -- 172.21.15.7:0/2505584252 >> 172.21.15.7:0/2505584252 conn(0x561595a71080 msgr2=0x561595b3f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:11.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.832+0000 7f87fdc42b80 1 -- 172.21.15.7:0/2505584252 shutdown_connections 2024-06-10T01:19:11.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.833+0000 7f87fdc42b80 1 -- 172.21.15.7:0/2505584252 wait complete. 2024-06-10T01:19:11.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.833+0000 7f87fdc42b80 1 -- start start 2024-06-10T01:19:11.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.833+0000 7f87fdc42b80 1 Processor -- start 2024-06-10T01:19:11.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.833+0000 7f87fdc42b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561595b447a0 0x561595b50650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:11.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.834+0000 7f87fdc42b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561595b49f60 0x561595b4d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:11.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.834+0000 7f87fdc42b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561595b4a880 0x561595b4de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:11.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.834+0000 7f87fdc42b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561595a35fc0 con 0x561595b447a0 2024-06-10T01:19:11.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.834+0000 7f87fdc42b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561595a91810 con 0x561595b49f60 2024-06-10T01:19:11.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.834+0000 7f87fdc42b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561595b51260 con 0x561595b4a880 2024-06-10T01:19:11.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.834+0000 7f87fdc42b80 10 client.0 _ll_register_callbacks cb 0x561595a2b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:19:11.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.834+0000 7f87f095a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561595b49f60 0x561595b4d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:11.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.834+0000 7f87f195c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561595b4a880 0x561595b4de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:11.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.834+0000 7f87f195c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561595b4a880 0x561595b4de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47576/0 (socket says 172.21.15.7:47576) 2024-06-10T01:19:11.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.834+0000 7f87f195c700 1 -- 172.21.15.7:0/1615864120 learned_addr learned my addr 172.21.15.7:0/1615864120 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:11.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.834+0000 7f87f095a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561595b49f60 0x561595b4d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:48854/0 (socket says 172.21.15.7:48854) 2024-06-10T01:19:11.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.834+0000 7f87f115b700 1 --2- 172.21.15.7:0/1615864120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561595b447a0 0x561595b50650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:11.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.834+0000 7f87f195c700 1 -- 172.21.15.7:0/1615864120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561595b49f60 msgr2=0x561595b4d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:11.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.834+0000 7f87f195c700 1 --2- 172.21.15.7:0/1615864120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561595b49f60 0x561595b4d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.834+0000 7f87f195c700 1 -- 172.21.15.7:0/1615864120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561595b447a0 msgr2=0x561595b50650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:11.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.834+0000 7f87f195c700 1 --2- 172.21.15.7:0/1615864120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561595b447a0 0x561595b50650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.834+0000 7f87f195c700 1 -- 172.21.15.7:0/1615864120 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561595a5b750 con 0x561595b4a880 2024-06-10T01:19:11.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.834+0000 7f87f195c700 1 --2- 172.21.15.7:0/1615864120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561595b4a880 0x561595b4de90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f87e4015000 tx=0x7f87e4004c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d78982d1abb4b448 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:11.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.834+0000 7f87d4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:19:11.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.835+0000 7f87d4ff9700 1 -- 172.21.15.7:0/1615864120 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e40076e0 con 0x561595b4a880 2024-06-10T01:19:11.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.835+0000 7f87fdc42b80 1 -- 172.21.15.7:0/1615864120 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87e80a35e0 con 0x561595b4a880 2024-06-10T01:19:11.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.835+0000 7f87d4ff9700 1 -- 172.21.15.7:0/1615864120 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f87e4007880 con 0x561595b4a880 2024-06-10T01:19:11.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.835+0000 7f87d4ff9700 1 -- 172.21.15.7:0/1615864120 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f87e4031050 con 0x561595b4a880 2024-06-10T01:19:11.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.835+0000 7f87fdc42b80 10 client.28081 fetch_fsmap learned FSMap version 40 2024-06-10T01:19:11.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.835+0000 7f87fdc42b80 1 -- 172.21.15.7:0/1615864120 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561595b5ba60 con 0x561595b4a880 2024-06-10T01:19:11.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.836+0000 7f87d4ff9700 1 -- 172.21.15.7:0/1615864120 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87e4028480 con 0x561595b4a880 2024-06-10T01:19:11.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.836+0000 7f87d4ff9700 1 -- 172.21.15.7:0/1615864120 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f87e4011070 con 0x561595b4a880 2024-06-10T01:19:11.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.836+0000 7f87fdc42b80 10 client.28081 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:19:11.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.836+0000 7f87fdc42b80 10 client.28081 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:19:11.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.836+0000 7f87fdc42b80 20 client.28081 populate_metadata read hostname 'smithi007' 2024-06-10T01:19:11.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.836+0000 7f87fdc42b80 1 --2- 172.21.15.7:0/1615864120 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x561595b5e0a0 0x561595b60490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:11.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.836+0000 7f87fdc42b80 4 client.28081 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:19:11.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.836+0000 7f87fdc42b80 1 -- 172.21.15.7:0/1615864120 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f87e80a35e0 con 0x561595b5e0a0 2024-06-10T01:19:11.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.836+0000 7f87f115b700 1 --2- 172.21.15.7:0/1615864120 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x561595b5e0a0 0x561595b60490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:11.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.836+0000 7f87d4ff9700 10 client.28081 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:19:11.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.836+0000 7f87f115b700 1 --2- 172.21.15.7:0/1615864120 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x561595b5e0a0 0x561595b60490 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=40652274dddedbd3 server_cookie=fcb8a8480b81522d in_seq=0 out_seq=0 2024-06-10T01:19:11.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.837+0000 7f87d4ff9700 1 -- 172.21.15.7:0/1615864120 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f87e80a35e0 con 0x561595b5e0a0 2024-06-10T01:19:11.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.837+0000 7f87d4ff9700 10 client.28081 handle_command_reply: tid=0 2024-06-10T01:19:11.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.837+0000 7f87fdc42b80 1 client.28081 shutdown 2024-06-10T01:19:11.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.838+0000 7f87fdc42b80 1 -- 172.21.15.7:0/1615864120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561595b4a880 msgr2=0x561595b4de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:11.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.838+0000 7f87fdc42b80 1 --2- 172.21.15.7:0/1615864120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561595b4a880 0x561595b4de90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f87e4015000 tx=0x7f87e4004c40 comp rx=0 tx=0).stop 2024-06-10T01:19:11.836 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:11 smithi078 ceph-mon[36683]: pgmap v827: 129 pgs: 129 active+clean; 146 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 2.2 MiB/s wr, 308 op/s 2024-06-10T01:19:11.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.838+0000 7f87fdc42b80 1 -- 172.21.15.7:0/1615864120 shutdown_connections 2024-06-10T01:19:11.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.838+0000 7f87fdc42b80 1 --2- 172.21.15.7:0/1615864120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561595b4a880 0x561595b4de90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.838+0000 7f87fdc42b80 1 --2- 172.21.15.7:0/1615864120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561595b447a0 0x561595b50650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.838+0000 7f87fdc42b80 1 --2- 172.21.15.7:0/1615864120 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x561595b5e0a0 0x561595b60490 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.838+0000 7f87fdc42b80 1 --2- 172.21.15.7:0/1615864120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561595b49f60 0x561595b4d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.838+0000 7f87fdc42b80 1 -- 172.21.15.7:0/1615864120 >> 172.21.15.7:0/1615864120 conn(0x561595a71080 msgr2=0x561595b3e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:11.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.838+0000 7f87d4ff9700 0 client.28081 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:19:11.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.838+0000 7f87fdc42b80 1 -- 172.21.15.7:0/1615864120 shutdown_connections 2024-06-10T01:19:11.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.838+0000 7f87fdc42b80 1 -- 172.21.15.7:0/1615864120 wait complete. 2024-06-10T01:19:11.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.838+0000 7f87fdc42b80 20 client.28081 trim_cache size 0 max 16384 2024-06-10T01:19:11.837 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:19:11.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.857+0000 7f87fdc42b80 1 Processor -- start 2024-06-10T01:19:11.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.857+0000 7f87fdc42b80 1 -- start start 2024-06-10T01:19:11.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.857+0000 7f87fdc42b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561595b447a0 0x561595b44260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:11.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.857+0000 7f87fdc42b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561595b49f60 0x561595b4b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:11.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.857+0000 7f87fdc42b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561595b4a880 0x561595b3ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:11.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.857+0000 7f87fdc42b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec1ae8d0 con 0x561595b4a880 2024-06-10T01:19:11.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.857+0000 7f87fdc42b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec1ac160 con 0x561595b49f60 2024-06-10T01:19:11.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.857+0000 7f87fdc42b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec0728d0 con 0x561595b447a0 2024-06-10T01:19:11.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.857+0000 7f87f195c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561595b4a880 0x561595b3ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:11.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.857+0000 7f87f115b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561595b447a0 0x561595b44260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:11.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.857+0000 7f87f095a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561595b49f60 0x561595b4b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:11.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.857+0000 7f87f195c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561595b4a880 0x561595b3ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55234/0 (socket says 172.21.15.7:55234) 2024-06-10T01:19:11.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.857+0000 7f87f195c700 1 -- 172.21.15.7:0/3087049250 learned_addr learned my addr 172.21.15.7:0/3087049250 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:11.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.857+0000 7f87f115b700 1 --2- 172.21.15.7:0/3087049250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561595b447a0 0x561595b44260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47584/0 (socket says 172.21.15.7:47584) 2024-06-10T01:19:11.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.858+0000 7f87f095a700 1 -- 172.21.15.7:0/3087049250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561595b447a0 msgr2=0x561595b44260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:11.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.858+0000 7f87f095a700 1 --2- 172.21.15.7:0/3087049250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561595b447a0 0x561595b44260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.858+0000 7f87f095a700 1 -- 172.21.15.7:0/3087049250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561595b4a880 msgr2=0x561595b3ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:11.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.858+0000 7f87f095a700 1 --2- 172.21.15.7:0/3087049250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561595b4a880 0x561595b3ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.858+0000 7f87f095a700 1 -- 172.21.15.7:0/3087049250 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561595b3b810 con 0x561595b49f60 2024-06-10T01:19:11.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.858+0000 7f87f195c700 1 --2- 172.21.15.7:0/3087049250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561595b4a880 0x561595b3ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:11.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.858+0000 7f87f095a700 1 --2- 172.21.15.7:0/3087049250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561595b49f60 0x561595b4b370 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x561595b4ef20 tx=0x7f87dc014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8042f84358a0848 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:11.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.858+0000 7f87d6ffd700 1 -- 172.21.15.7:0/3087049250 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc004460 con 0x561595b49f60 2024-06-10T01:19:11.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.858+0000 7f87d6ffd700 1 -- 172.21.15.7:0/3087049250 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87dc003b60 con 0x561595b49f60 2024-06-10T01:19:11.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.858+0000 7f87d6ffd700 1 -- 172.21.15.7:0/3087049250 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc01dc40 con 0x561595b49f60 2024-06-10T01:19:11.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.859+0000 7f87fdc42b80 1 -- 172.21.15.7:0/3087049250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561595b49f60 msgr2=0x561595b4b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:11.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.859+0000 7f87fdc42b80 1 --2- 172.21.15.7:0/3087049250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561595b49f60 0x561595b4b370 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x561595b4ef20 tx=0x7f87dc014000 comp rx=0 tx=0).stop 2024-06-10T01:19:11.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.859+0000 7f87fdc42b80 1 -- 172.21.15.7:0/3087049250 shutdown_connections 2024-06-10T01:19:11.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.859+0000 7f87fdc42b80 1 --2- 172.21.15.7:0/3087049250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561595b4a880 0x561595b3ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.859+0000 7f87fdc42b80 1 --2- 172.21.15.7:0/3087049250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561595b447a0 0x561595b44260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.859+0000 7f87fdc42b80 1 --2- 172.21.15.7:0/3087049250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561595b49f60 0x561595b4b370 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.859+0000 7f87fdc42b80 1 -- 172.21.15.7:0/3087049250 >> 172.21.15.7:0/3087049250 conn(0x561595a71080 msgr2=0x561595b4ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:11.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.859+0000 7f87fdc42b80 1 -- 172.21.15.7:0/3087049250 shutdown_connections 2024-06-10T01:19:11.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.859+0000 7f87fdc42b80 1 -- 172.21.15.7:0/3087049250 wait complete. 2024-06-10T01:19:11.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.859+0000 7f87fdc42b80 1 -- start start 2024-06-10T01:19:11.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.859+0000 7f87fdc42b80 1 Processor -- start 2024-06-10T01:19:11.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.860+0000 7f87fdc42b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561595b447a0 0x561595b568b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:11.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.860+0000 7f87fdc42b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561595b49f60 0x561595b56df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:11.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.860+0000 7f87fdc42b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561595b4a880 0x561595b654d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:11.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.860+0000 7f87fdc42b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec072f40 con 0x561595b4a880 2024-06-10T01:19:11.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.860+0000 7f87fdc42b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec0743b0 con 0x561595b447a0 2024-06-10T01:19:11.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.860+0000 7f87fdc42b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec073610 con 0x561595b49f60 2024-06-10T01:19:11.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.860+0000 7f87fdc42b80 10 client.0 _ll_register_callbacks cb 0x561595a2b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:19:11.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.860+0000 7f87f095a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561595b49f60 0x561595b56df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:11.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.860+0000 7f87f095a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561595b49f60 0x561595b56df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47590/0 (socket says 172.21.15.7:47590) 2024-06-10T01:19:11.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.860+0000 7f87f095a700 1 -- 172.21.15.7:0/4008336511 learned_addr learned my addr 172.21.15.7:0/4008336511 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:11.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.860+0000 7f87f115b700 1 --2- 172.21.15.7:0/4008336511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561595b447a0 0x561595b568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:11.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.860+0000 7f87f195c700 1 --2- 172.21.15.7:0/4008336511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561595b4a880 0x561595b654d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:11.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.860+0000 7f87f095a700 1 -- 172.21.15.7:0/4008336511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561595b447a0 msgr2=0x561595b568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:11.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.860+0000 7f87f095a700 1 --2- 172.21.15.7:0/4008336511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561595b447a0 0x561595b568b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.860+0000 7f87f095a700 1 -- 172.21.15.7:0/4008336511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561595b4a880 msgr2=0x561595b654d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:11.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.860+0000 7f87f095a700 1 --2- 172.21.15.7:0/4008336511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561595b4a880 0x561595b654d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.860+0000 7f87f095a700 1 -- 172.21.15.7:0/4008336511 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561595b65cb0 con 0x561595b49f60 2024-06-10T01:19:11.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.860+0000 7f87f115b700 1 --2- 172.21.15.7:0/4008336511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561595b447a0 0x561595b568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:11.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.861+0000 7f87f095a700 1 --2- 172.21.15.7:0/4008336511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561595b49f60 0x561595b56df0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f87dc0045f0 tx=0x7f87dc004620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edc6918ed7e6065f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:11.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.861+0000 7f87d4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:19:11.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.861+0000 7f87d4ff9700 1 -- 172.21.15.7:0/4008336511 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc01d5d0 con 0x561595b49f60 2024-06-10T01:19:11.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.861+0000 7f87d4ff9700 1 -- 172.21.15.7:0/4008336511 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f87dc023040 con 0x561595b49f60 2024-06-10T01:19:11.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.861+0000 7f87fdc42b80 1 -- 172.21.15.7:0/4008336511 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87dc003b60 con 0x561595b49f60 2024-06-10T01:19:11.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.861+0000 7f87d4ff9700 1 -- 172.21.15.7:0/4008336511 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f87dc039050 con 0x561595b49f60 2024-06-10T01:19:11.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.861+0000 7f87fdc42b80 10 client.28087 fetch_fsmap learned FSMap version 40 2024-06-10T01:19:11.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.861+0000 7f87fdc42b80 1 -- 172.21.15.7:0/4008336511 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561595bc8930 con 0x561595b49f60 2024-06-10T01:19:11.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.861+0000 7f87d4ff9700 1 -- 172.21.15.7:0/4008336511 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87dc003b60 con 0x561595b49f60 2024-06-10T01:19:11.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.861+0000 7f87d4ff9700 1 -- 172.21.15.7:0/4008336511 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f87dc004750 con 0x561595b49f60 2024-06-10T01:19:11.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.861+0000 7f87fdc42b80 10 client.28087 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:19:11.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.861+0000 7f87fdc42b80 10 client.28087 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:19:11.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.862+0000 7f87fdc42b80 20 client.28087 populate_metadata read hostname 'smithi007' 2024-06-10T01:19:11.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.862+0000 7f87fdc42b80 1 --2- 172.21.15.7:0/4008336511 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x561595bc8fb0 0x561595bcb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:11.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.862+0000 7f87fdc42b80 4 client.28087 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:19:11.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.862+0000 7f87fdc42b80 1 -- 172.21.15.7:0/4008336511 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561595aa2d50 con 0x561595bc8fb0 2024-06-10T01:19:11.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.862+0000 7f87f115b700 1 --2- 172.21.15.7:0/4008336511 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x561595bc8fb0 0x561595bcb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:11.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.862+0000 7f87f115b700 1 --2- 172.21.15.7:0/4008336511 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x561595bc8fb0 0x561595bcb3d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=45e3e7b1d3721c18 server_cookie=e7f65f2b1fee4380 in_seq=0 out_seq=0 2024-06-10T01:19:11.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.862+0000 7f87d4ff9700 10 client.28087 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:19:11.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.863+0000 7f87d4ff9700 1 -- 172.21.15.7:0/4008336511 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561595aa2d50 con 0x561595bc8fb0 2024-06-10T01:19:11.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.863+0000 7f87d4ff9700 10 client.28087 handle_command_reply: tid=0 2024-06-10T01:19:11.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.863+0000 7f87fdc42b80 1 client.28087 shutdown 2024-06-10T01:19:11.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.863+0000 7f87fdc42b80 1 -- 172.21.15.7:0/4008336511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561595b49f60 msgr2=0x561595b56df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:11.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.863+0000 7f87fdc42b80 1 --2- 172.21.15.7:0/4008336511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561595b49f60 0x561595b56df0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f87dc0045f0 tx=0x7f87dc004620 comp rx=0 tx=0).stop 2024-06-10T01:19:11.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.863+0000 7f87fdc42b80 1 -- 172.21.15.7:0/4008336511 shutdown_connections 2024-06-10T01:19:11.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.863+0000 7f87fdc42b80 1 --2- 172.21.15.7:0/4008336511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561595b4a880 0x561595b654d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.863+0000 7f87fdc42b80 1 --2- 172.21.15.7:0/4008336511 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x561595bc8fb0 0x561595bcb3d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.863+0000 7f87fdc42b80 1 --2- 172.21.15.7:0/4008336511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561595b447a0 0x561595b568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.863+0000 7f87fdc42b80 1 --2- 172.21.15.7:0/4008336511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561595b49f60 0x561595b56df0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.863+0000 7f87fdc42b80 1 -- 172.21.15.7:0/4008336511 >> 172.21.15.7:0/4008336511 conn(0x561595a71080 msgr2=0x561595b4c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:11.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.863+0000 7f87d4ff9700 0 client.28087 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:19:11.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.863+0000 7f87fdc42b80 1 -- 172.21.15.7:0/4008336511 shutdown_connections 2024-06-10T01:19:11.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.863+0000 7f87fdc42b80 1 -- 172.21.15.7:0/4008336511 wait complete. 2024-06-10T01:19:11.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.863+0000 7f87fdc42b80 20 client.28087 trim_cache size 0 max 16384 2024-06-10T01:19:11.862 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:19:11.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.864+0000 7f87f33bf700 1 -- 172.21.15.7:0/1455283961 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f87d80a2c20 msgr2=0x7f87d80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:11.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.864+0000 7f87f33bf700 1 --2- 172.21.15.7:0/1455283961 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f87d80a2c20 0x7f87d80a50e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f87ec078c80 tx=0x7f87dc009260 comp rx=0 tx=0).stop 2024-06-10T01:19:11.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.864+0000 7f87f33bf700 1 -- 172.21.15.7:0/1455283961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87ec101390 msgr2=0x7f87ec19f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:11.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.864+0000 7f87f33bf700 1 --2- 172.21.15.7:0/1455283961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87ec101390 0x7f87ec19f050 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f87e8007a00 tx=0x7f87e8007d10 comp rx=0 tx=0).stop 2024-06-10T01:19:11.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.864+0000 7f87f33bf700 1 -- 172.21.15.7:0/1455283961 shutdown_connections 2024-06-10T01:19:11.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.864+0000 7f87f33bf700 1 --2- 172.21.15.7:0/1455283961 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f87d80a2c20 0x7f87d80a50e0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.864+0000 7f87f33bf700 1 --2- 172.21.15.7:0/1455283961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87ec107400 0x7f87ec1a3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.864+0000 7f87f33bf700 1 --2- 172.21.15.7:0/1455283961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87ec101390 0x7f87ec19f050 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.864+0000 7f87f33bf700 1 --2- 172.21.15.7:0/1455283961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87ec101d50 0x7f87ec19f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:11.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.864+0000 7f87f33bf700 1 -- 172.21.15.7:0/1455283961 >> 172.21.15.7:0/1455283961 conn(0x7f87ec078340 msgr2=0x7f87ec108280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:11.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.864+0000 7f87f33bf700 1 -- 172.21.15.7:0/1455283961 shutdown_connections 2024-06-10T01:19:11.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:11.864+0000 7f87f33bf700 1 -- 172.21.15.7:0/1455283961 wait complete. 2024-06-10T01:19:11.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:11 smithi175 ceph-mon[36636]: pgmap v827: 129 pgs: 129 active+clean; 146 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 2.2 MiB/s wr, 308 op/s 2024-06-10T01:19:12.757 INFO:tasks.workunit.client.0.smithi007.stdout: 1 977599 7.36 MB/sec execute 593 sec latency 16.009 ms 2024-06-10T01:19:13.757 INFO:tasks.workunit.client.0.smithi007.stdout: 1 979607 7.36 MB/sec execute 594 sec latency 64.055 ms 2024-06-10T01:19:13.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:13 smithi007 ceph-mon[32398]: pgmap v828: 129 pgs: 129 active+clean; 146 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 1.8 MiB/s wr, 233 op/s 2024-06-10T01:19:13.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:13 smithi078 ceph-mon[36683]: pgmap v828: 129 pgs: 129 active+clean; 146 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 1.8 MiB/s wr, 233 op/s 2024-06-10T01:19:13.915 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:19:13.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:13 smithi175 ceph-mon[36636]: pgmap v828: 129 pgs: 129 active+clean; 146 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 1.8 MiB/s wr, 233 op/s 2024-06-10T01:19:14.758 INFO:tasks.workunit.client.0.smithi007.stdout: 1 981157 7.37 MB/sec execute 595 sec latency 22.630 ms 2024-06-10T01:19:15.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.058+0000 7f6667ff3700 1 -- 172.21.15.7:0/3372731005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66601013c0 msgr2=0x7f6660101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:15.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.058+0000 7f6667ff3700 1 --2- 172.21.15.7:0/3372731005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66601013c0 0x7f6660101840 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f6650009990 tx=0x7f6650009ca0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.058+0000 7f6667ff3700 1 -- 172.21.15.7:0/3372731005 shutdown_connections 2024-06-10T01:19:15.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.058+0000 7f6667ff3700 1 --2- 172.21.15.7:0/3372731005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6660101d80 0x7f666010e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.058+0000 7f6667ff3700 1 --2- 172.21.15.7:0/3372731005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6660107470 0x7f6660107850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.058+0000 7f6667ff3700 1 --2- 172.21.15.7:0/3372731005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66601013c0 0x7f6660101840 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.058+0000 7f6667ff3700 1 -- 172.21.15.7:0/3372731005 >> 172.21.15.7:0/3372731005 conn(0x7f66600783d0 msgr2=0x7f66600787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:15.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.058+0000 7f6667ff3700 1 -- 172.21.15.7:0/3372731005 shutdown_connections 2024-06-10T01:19:15.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.058+0000 7f6667ff3700 1 -- 172.21.15.7:0/3372731005 wait complete. 2024-06-10T01:19:15.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.059+0000 7f6667ff3700 1 Processor -- start 2024-06-10T01:19:15.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.060+0000 7f6667ff3700 1 -- start start 2024-06-10T01:19:15.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.060+0000 7f6667ff3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66601013c0 0x7f666019d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:15.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.060+0000 7f6667ff3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6660101d80 0x7f666019dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:15.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.060+0000 7f6667ff3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6660107470 0x7f6660197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.060+0000 7f6665d8f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66601013c0 0x7f666019d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.060+0000 7f6667ff3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6660111680 con 0x7f66601013c0 2024-06-10T01:19:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.060+0000 7f6667ff3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6660111500 con 0x7f6660101d80 2024-06-10T01:19:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.060+0000 7f6667ff3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6660111800 con 0x7f6660107470 2024-06-10T01:19:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.060+0000 7f6665d8f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66601013c0 0x7f666019d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55256/0 (socket says 172.21.15.7:55256) 2024-06-10T01:19:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.060+0000 7f6665d8f700 1 -- 172.21.15.7:0/3003351836 learned_addr learned my addr 172.21.15.7:0/3003351836 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.060+0000 7f6666590700 1 --2- 172.21.15.7:0/3003351836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6660107470 0x7f6660197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.060+0000 7f666558e700 1 --2- 172.21.15.7:0/3003351836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6660101d80 0x7f666019dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.061+0000 7f6665d8f700 1 -- 172.21.15.7:0/3003351836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6660107470 msgr2=0x7f6660197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.061+0000 7f6665d8f700 1 --2- 172.21.15.7:0/3003351836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6660107470 0x7f6660197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.061+0000 7f6665d8f700 1 -- 172.21.15.7:0/3003351836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6660101d80 msgr2=0x7f666019dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.061+0000 7f6665d8f700 1 --2- 172.21.15.7:0/3003351836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6660101d80 0x7f666019dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.061+0000 7f6665d8f700 1 -- 172.21.15.7:0/3003351836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6660197fe0 con 0x7f66601013c0 2024-06-10T01:19:15.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.061+0000 7f666558e700 1 --2- 172.21.15.7:0/3003351836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6660101d80 0x7f666019dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:19:15.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.061+0000 7f6665d8f700 1 --2- 172.21.15.7:0/3003351836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66601013c0 0x7f666019d6c0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f665c00eab0 tx=0x7f665c00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:15.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.061+0000 7f6656ffd700 1 -- 172.21.15.7:0/3003351836 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c01c070 con 0x7f66601013c0 2024-06-10T01:19:15.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.061+0000 7f6656ffd700 1 -- 172.21.15.7:0/3003351836 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f665c017410 con 0x7f66601013c0 2024-06-10T01:19:15.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.061+0000 7f6656ffd700 1 -- 172.21.15.7:0/3003351836 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c016610 con 0x7f66601013c0 2024-06-10T01:19:15.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.062+0000 7f6667ff3700 1 -- 172.21.15.7:0/3003351836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66601982d0 con 0x7f66601013c0 2024-06-10T01:19:15.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.062+0000 7f6667ff3700 1 -- 172.21.15.7:0/3003351836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f666006aa00 con 0x7f66601013c0 2024-06-10T01:19:15.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.064+0000 7f6656ffd700 1 -- 172.21.15.7:0/3003351836 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f665c0175c0 con 0x7f66601013c0 2024-06-10T01:19:15.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.065+0000 7f6656ffd700 1 --2- 172.21.15.7:0/3003351836 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f664c0a7240 0x7f664c0a9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:15.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.065+0000 7f6656ffd700 1 -- 172.21.15.7:0/3003351836 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f665c0cef20 con 0x7f66601013c0 2024-06-10T01:19:15.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.065+0000 7f666558e700 1 --2- 172.21.15.7:0/3003351836 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f664c0a7240 0x7f664c0a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:15.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.066+0000 7f666558e700 1 --2- 172.21.15.7:0/3003351836 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f664c0a7240 0x7f664c0a9700 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6650005b40 tx=0x7f665001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:15.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.066+0000 7f6672876b80 1 Processor -- start 2024-06-10T01:19:15.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.066+0000 7f6672876b80 1 -- start start 2024-06-10T01:19:15.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.066+0000 7f6672876b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564005fdc590 0x564005fdc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:15.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.066+0000 7f6672876b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564005fdceb0 0x564005fd67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:15.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.066+0000 7f6672876b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564005fd6ce0 0x564005fd7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:15.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.067+0000 7f6672876b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564005ec90f0 con 0x564005fd6ce0 2024-06-10T01:19:15.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.067+0000 7f6665d8f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564005fdc590 0x564005fdc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:15.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.067+0000 7f6666590700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564005fd6ce0 0x564005fd7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:15.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.067+0000 7f6672876b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564005f279e0 con 0x564005fdc590 2024-06-10T01:19:15.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.067+0000 7f6666590700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564005fd6ce0 0x564005fd7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55262/0 (socket says 172.21.15.7:55262) 2024-06-10T01:19:15.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.067+0000 7f6665d8f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564005fdc590 0x564005fdc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:48906/0 (socket says 172.21.15.7:48906) 2024-06-10T01:19:15.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.067+0000 7f6665d8f700 1 -- 172.21.15.7:0/3173686774 learned_addr learned my addr 172.21.15.7:0/3173686774 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:15.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.067+0000 7f6672876b80 1 -- 172.21.15.7:0/3173686774 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564005e8f5f0 con 0x564005fdceb0 2024-06-10T01:19:15.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.067+0000 7f666558e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564005fdceb0 0x564005fd67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:15.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.067+0000 7f6665d8f700 1 -- 172.21.15.7:0/3173686774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564005fdceb0 msgr2=0x564005fd67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:15.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.067+0000 7f6665d8f700 1 --2- 172.21.15.7:0/3173686774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564005fdceb0 0x564005fd67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.067+0000 7f6665d8f700 1 -- 172.21.15.7:0/3173686774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564005fd6ce0 msgr2=0x564005fd7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.067+0000 7f6665d8f700 1 --2- 172.21.15.7:0/3173686774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564005fd6ce0 0x564005fd7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.067+0000 7f6665d8f700 1 -- 172.21.15.7:0/3173686774 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564005e76a60 con 0x564005fdc590 2024-06-10T01:19:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.067+0000 7f6665d8f700 1 --2- 172.21.15.7:0/3173686774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564005fdc590 0x564005fdc970 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f665c009230 tx=0x7f665c097050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ca1d014344112dc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.067+0000 7f664bfff700 1 -- 172.21.15.7:0/3173686774 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c0b3020 con 0x564005fdc590 2024-06-10T01:19:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.067+0000 7f664bfff700 1 -- 172.21.15.7:0/3173686774 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f665c09b400 con 0x564005fdc590 2024-06-10T01:19:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.067+0000 7f664bfff700 1 -- 172.21.15.7:0/3173686774 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c09b660 con 0x564005fdc590 2024-06-10T01:19:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.068+0000 7f6672876b80 1 -- 172.21.15.7:0/3173686774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564005fdc590 msgr2=0x564005fdc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.068+0000 7f6672876b80 1 --2- 172.21.15.7:0/3173686774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564005fdc590 0x564005fdc970 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f665c009230 tx=0x7f665c097050 comp rx=0 tx=0).stop 2024-06-10T01:19:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.068+0000 7f6672876b80 1 -- 172.21.15.7:0/3173686774 shutdown_connections 2024-06-10T01:19:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.068+0000 7f6672876b80 1 --2- 172.21.15.7:0/3173686774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564005fd6ce0 0x564005fd7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.068+0000 7f6672876b80 1 --2- 172.21.15.7:0/3173686774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564005fdc590 0x564005fdc970 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.068+0000 7f6672876b80 1 --2- 172.21.15.7:0/3173686774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564005fdceb0 0x564005fd67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.068+0000 7f6672876b80 1 -- 172.21.15.7:0/3173686774 >> 172.21.15.7:0/3173686774 conn(0x564005f03210 msgr2=0x564005fd1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.069+0000 7f6672876b80 1 -- 172.21.15.7:0/3173686774 shutdown_connections 2024-06-10T01:19:15.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.069+0000 7f6672876b80 1 -- 172.21.15.7:0/3173686774 wait complete. 2024-06-10T01:19:15.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.069+0000 7f6672876b80 1 -- start start 2024-06-10T01:19:15.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.069+0000 7f6672876b80 1 Processor -- start 2024-06-10T01:19:15.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.069+0000 7f6672876b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564005fd6ce0 0x564005fe2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:15.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.070+0000 7f6672876b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564005fdc590 0x564005fdff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:15.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.070+0000 7f6672876b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564005fdceb0 0x564005fe0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:15.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.070+0000 7f6672876b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564005ec8150 con 0x564005fd6ce0 2024-06-10T01:19:15.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.070+0000 7f6672876b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564005f239a0 con 0x564005fdc590 2024-06-10T01:19:15.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.070+0000 7f6672876b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564005fe3890 con 0x564005fdceb0 2024-06-10T01:19:15.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.070+0000 7f6665d8f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564005fd6ce0 0x564005fe2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:15.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.070+0000 7f666558e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564005fdc590 0x564005fdff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:15.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.070+0000 7f6665d8f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564005fd6ce0 0x564005fe2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55264/0 (socket says 172.21.15.7:55264) 2024-06-10T01:19:15.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.070+0000 7f6672876b80 10 client.0 _ll_register_callbacks cb 0x564005ebd740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:19:15.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.070+0000 7f6665d8f700 1 -- 172.21.15.7:0/220604685 learned_addr learned my addr 172.21.15.7:0/220604685 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:15.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.070+0000 7f6666590700 1 --2- 172.21.15.7:0/220604685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564005fdceb0 0x564005fe0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:15.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.070+0000 7f6665d8f700 1 -- 172.21.15.7:0/220604685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564005fdceb0 msgr2=0x564005fe0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:15.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.070+0000 7f6665d8f700 1 --2- 172.21.15.7:0/220604685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564005fdceb0 0x564005fe0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.070+0000 7f6665d8f700 1 -- 172.21.15.7:0/220604685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564005fdc590 msgr2=0x564005fdff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:15.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.070+0000 7f6665d8f700 1 --2- 172.21.15.7:0/220604685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564005fdc590 0x564005fdff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.070+0000 7f6665d8f700 1 -- 172.21.15.7:0/220604685 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564005eed8e0 con 0x564005fd6ce0 2024-06-10T01:19:15.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.070+0000 7f666558e700 1 --2- 172.21.15.7:0/220604685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564005fdc590 0x564005fdff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:15.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.070+0000 7f6665d8f700 1 --2- 172.21.15.7:0/220604685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564005fd6ce0 0x564005fe2c40 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f665c000c00 tx=0x7f665c017c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e178a93deb983f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:15.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.070+0000 7f66497fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:19:15.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.071+0000 7f66497fa700 1 -- 172.21.15.7:0/220604685 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c0b8030 con 0x564005fd6ce0 2024-06-10T01:19:15.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.071+0000 7f66497fa700 1 -- 172.21.15.7:0/220604685 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f665c099700 con 0x564005fd6ce0 2024-06-10T01:19:15.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.071+0000 7f6672876b80 1 -- 172.21.15.7:0/220604685 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f665c09b400 con 0x564005fd6ce0 2024-06-10T01:19:15.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.071+0000 7f66497fa700 1 -- 172.21.15.7:0/220604685 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f665c0b3070 con 0x564005fd6ce0 2024-06-10T01:19:15.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.071+0000 7f6672876b80 10 client.18768 fetch_fsmap learned FSMap version 40 2024-06-10T01:19:15.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.071+0000 7f6672876b80 1 -- 172.21.15.7:0/220604685 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x564005fee010 con 0x564005fd6ce0 2024-06-10T01:19:15.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.071+0000 7f66497fa700 1 -- 172.21.15.7:0/220604685 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f665c09b400 con 0x564005fd6ce0 2024-06-10T01:19:15.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.071+0000 7f66497fa700 1 -- 172.21.15.7:0/220604685 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f665c07b020 con 0x564005fd6ce0 2024-06-10T01:19:15.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.071+0000 7f6672876b80 10 client.18768 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:19:15.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.071+0000 7f6672876b80 10 client.18768 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:19:15.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.071+0000 7f6672876b80 20 client.18768 populate_metadata read hostname 'smithi007' 2024-06-10T01:19:15.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.071+0000 7f6672876b80 1 --2- 172.21.15.7:0/220604685 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564005ff04e0 0x564005ff28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:15.068 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:19:15.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.071+0000 7f6672876b80 4 client.18768 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:19:15.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.071+0000 7f6672876b80 1 -- 172.21.15.7:0/220604685 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564005f34ee0 con 0x564005ff04e0 2024-06-10T01:19:15.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.072+0000 7f666558e700 1 --2- 172.21.15.7:0/220604685 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564005ff04e0 0x564005ff28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:15.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.072+0000 7f666558e700 1 --2- 172.21.15.7:0/220604685 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564005ff04e0 0x564005ff28d0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76f927eee3769a5e server_cookie=90240fa71e715aab in_seq=0 out_seq=0 2024-06-10T01:19:15.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.072+0000 7f66497fa700 10 client.18768 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:15.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.073+0000 7f66497fa700 1 -- 172.21.15.7:0/220604685 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564005f34ee0 con 0x564005ff04e0 2024-06-10T01:19:15.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.073+0000 7f66497fa700 10 client.18768 handle_command_reply: tid=0 2024-06-10T01:19:15.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.073+0000 7f6672876b80 1 client.18768 shutdown 2024-06-10T01:19:15.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.073+0000 7f6672876b80 1 -- 172.21.15.7:0/220604685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564005fd6ce0 msgr2=0x564005fe2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:15.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.073+0000 7f6672876b80 1 --2- 172.21.15.7:0/220604685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564005fd6ce0 0x564005fe2c40 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f665c000c00 tx=0x7f665c017c50 comp rx=0 tx=0).stop 2024-06-10T01:19:15.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.073+0000 7f6672876b80 1 -- 172.21.15.7:0/220604685 shutdown_connections 2024-06-10T01:19:15.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.073+0000 7f6672876b80 1 --2- 172.21.15.7:0/220604685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564005fdceb0 0x564005fe0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.073+0000 7f6672876b80 1 --2- 172.21.15.7:0/220604685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564005fd6ce0 0x564005fe2c40 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.073+0000 7f6672876b80 1 --2- 172.21.15.7:0/220604685 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564005ff04e0 0x564005ff28d0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.073+0000 7f6672876b80 1 --2- 172.21.15.7:0/220604685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564005fdc590 0x564005fdff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.073+0000 7f6672876b80 1 -- 172.21.15.7:0/220604685 >> 172.21.15.7:0/220604685 conn(0x564005f03210 msgr2=0x564005fd03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:15.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.073+0000 7f66497fa700 0 client.18768 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:15.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.073+0000 7f6672876b80 1 -- 172.21.15.7:0/220604685 shutdown_connections 2024-06-10T01:19:15.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.073+0000 7f6672876b80 1 -- 172.21.15.7:0/220604685 wait complete. 2024-06-10T01:19:15.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.073+0000 7f6672876b80 20 client.18768 trim_cache size 0 max 16384 2024-06-10T01:19:15.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.095+0000 7f6672876b80 1 Processor -- start 2024-06-10T01:19:15.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.095+0000 7f6672876b80 1 -- start start 2024-06-10T01:19:15.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.095+0000 7f6672876b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564005fd6ce0 0x564005fd67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:15.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.095+0000 7f6672876b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564005fdc590 0x564005fe2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:15.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.096+0000 7f6672876b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564005fdceb0 0x564005fdb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:15.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.096+0000 7f6672876b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f66601ae970 con 0x564005fd6ce0 2024-06-10T01:19:15.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.096+0000 7f6672876b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f66601ac200 con 0x564005fdceb0 2024-06-10T01:19:15.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.096+0000 7f6672876b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f66601aa050 con 0x564005fdc590 2024-06-10T01:19:15.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.096+0000 7f666558e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564005fdc590 0x564005fe2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:15.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.096+0000 7f666558e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564005fdc590 0x564005fe2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47644/0 (socket says 172.21.15.7:47644) 2024-06-10T01:19:15.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.096+0000 7f666558e700 1 -- 172.21.15.7:0/2763190656 learned_addr learned my addr 172.21.15.7:0/2763190656 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:15.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.096+0000 7f6666590700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564005fdceb0 0x564005fdb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:15.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.096+0000 7f6665d8f700 1 --2- 172.21.15.7:0/2763190656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564005fd6ce0 0x564005fd67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:15.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.096+0000 7f666558e700 1 -- 172.21.15.7:0/2763190656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564005fdceb0 msgr2=0x564005fdb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:15.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.096+0000 7f666558e700 1 --2- 172.21.15.7:0/2763190656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564005fdceb0 0x564005fdb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.096+0000 7f666558e700 1 -- 172.21.15.7:0/2763190656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564005fd6ce0 msgr2=0x564005fd67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:15.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.096+0000 7f666558e700 1 --2- 172.21.15.7:0/2763190656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564005fd6ce0 0x564005fd67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.096+0000 7f666558e700 1 -- 172.21.15.7:0/2763190656 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564005fe0e70 con 0x564005fdc590 2024-06-10T01:19:15.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.096+0000 7f6665d8f700 1 --2- 172.21.15.7:0/2763190656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564005fd6ce0 0x564005fd67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:19:15.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.096+0000 7f666558e700 1 --2- 172.21.15.7:0/2763190656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564005fdc590 0x564005fe2780 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6650000c00 tx=0x7f665001e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a68e8fd918dd0fd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:15.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.096+0000 7f664b7fe700 1 -- 172.21.15.7:0/2763190656 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650010e40 con 0x564005fdc590 2024-06-10T01:19:15.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.097+0000 7f664b7fe700 1 -- 172.21.15.7:0/2763190656 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6650005530 con 0x564005fdc590 2024-06-10T01:19:15.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.097+0000 7f664b7fe700 1 -- 172.21.15.7:0/2763190656 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66500336b0 con 0x564005fdc590 2024-06-10T01:19:15.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.097+0000 7f6672876b80 1 -- 172.21.15.7:0/2763190656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564005fdc590 msgr2=0x564005fe2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:15.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.097+0000 7f6672876b80 1 --2- 172.21.15.7:0/2763190656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564005fdc590 0x564005fe2780 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6650000c00 tx=0x7f665001e000 comp rx=0 tx=0).stop 2024-06-10T01:19:15.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.097+0000 7f6672876b80 1 -- 172.21.15.7:0/2763190656 shutdown_connections 2024-06-10T01:19:15.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.097+0000 7f6672876b80 1 --2- 172.21.15.7:0/2763190656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564005fdceb0 0x564005fdb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.097+0000 7f6672876b80 1 --2- 172.21.15.7:0/2763190656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564005fd6ce0 0x564005fd67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.097+0000 7f6672876b80 1 --2- 172.21.15.7:0/2763190656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564005fdc590 0x564005fe2780 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.097+0000 7f6672876b80 1 -- 172.21.15.7:0/2763190656 >> 172.21.15.7:0/2763190656 conn(0x564005f03210 msgr2=0x564005fdd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.097+0000 7f6672876b80 1 -- 172.21.15.7:0/2763190656 shutdown_connections 2024-06-10T01:19:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.097+0000 7f6672876b80 1 -- 172.21.15.7:0/2763190656 wait complete. 2024-06-10T01:19:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.098+0000 7f6672876b80 1 -- start start 2024-06-10T01:19:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.098+0000 7f6672876b80 1 Processor -- start 2024-06-10T01:19:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.098+0000 7f6672876b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564005fd6ce0 0x564005ff39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.098+0000 7f6672876b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564005fdc590 0x564005ff3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.099+0000 7f6672876b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564005fdceb0 0x564005ff4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.099+0000 7f6672876b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f66601aa6c0 con 0x564005fdc590 2024-06-10T01:19:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.099+0000 7f6672876b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f66601abb30 con 0x564005fdceb0 2024-06-10T01:19:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.099+0000 7f6672876b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f66601aad90 con 0x564005fd6ce0 2024-06-10T01:19:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.099+0000 7f6672876b80 10 client.0 _ll_register_callbacks cb 0x564005ebd740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:19:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.099+0000 7f6665d8f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564005fd6ce0 0x564005ff39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.099+0000 7f6665d8f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564005fd6ce0 0x564005ff39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47658/0 (socket says 172.21.15.7:47658) 2024-06-10T01:19:15.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.099+0000 7f6665d8f700 1 -- 172.21.15.7:0/766935138 learned_addr learned my addr 172.21.15.7:0/766935138 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:15.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.099+0000 7f666558e700 1 --2- 172.21.15.7:0/766935138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564005fdc590 0x564005ff3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:15.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.099+0000 7f6666590700 1 --2- 172.21.15.7:0/766935138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564005fdceb0 0x564005ff4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:15.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.099+0000 7f6665d8f700 1 -- 172.21.15.7:0/766935138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564005fdceb0 msgr2=0x564005ff4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:15.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.099+0000 7f6665d8f700 1 --2- 172.21.15.7:0/766935138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564005fdceb0 0x564005ff4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.099+0000 7f6665d8f700 1 -- 172.21.15.7:0/766935138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564005fdc590 msgr2=0x564005ff3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:15.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.099+0000 7f6665d8f700 1 --2- 172.21.15.7:0/766935138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564005fdc590 0x564005ff3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.099+0000 7f6665d8f700 1 -- 172.21.15.7:0/766935138 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56400605a120 con 0x564005fd6ce0 2024-06-10T01:19:15.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.099+0000 7f6665d8f700 1 --2- 172.21.15.7:0/766935138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564005fd6ce0 0x564005ff39c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f665c0ccb60 tx=0x7f665c009880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=200ffdf8f3963784 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:15.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.099+0000 7f66497fa700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:19:15.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.099+0000 7f66497fa700 1 -- 172.21.15.7:0/766935138 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c0b8070 con 0x564005fd6ce0 2024-06-10T01:19:15.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.100+0000 7f66497fa700 1 -- 172.21.15.7:0/766935138 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f665c09b660 con 0x564005fd6ce0 2024-06-10T01:19:15.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.100+0000 7f6672876b80 1 -- 172.21.15.7:0/766935138 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6650005530 con 0x564005fd6ce0 2024-06-10T01:19:15.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.100+0000 7f66497fa700 1 -- 172.21.15.7:0/766935138 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f665c09a850 con 0x564005fd6ce0 2024-06-10T01:19:15.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.100+0000 7f6672876b80 10 client.28105 fetch_fsmap learned FSMap version 40 2024-06-10T01:19:15.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.100+0000 7f6672876b80 1 -- 172.21.15.7:0/766935138 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56400605b400 con 0x564005fd6ce0 2024-06-10T01:19:15.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.100+0000 7f66497fa700 1 -- 172.21.15.7:0/766935138 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6650005530 con 0x564005fd6ce0 2024-06-10T01:19:15.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.100+0000 7f66497fa700 1 -- 172.21.15.7:0/766935138 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f665c07b050 con 0x564005fd6ce0 2024-06-10T01:19:15.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.100+0000 7f6672876b80 10 client.28105 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:19:15.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.100+0000 7f6672876b80 10 client.28105 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:19:15.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.100+0000 7f6672876b80 20 client.28105 populate_metadata read hostname 'smithi007' 2024-06-10T01:19:15.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.100+0000 7f6672876b80 1 --2- 172.21.15.7:0/766935138 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56400605b8e0 0x56400605dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:15.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.100+0000 7f6672876b80 4 client.28105 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-06-10T01:19:15.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.100+0000 7f6672876b80 1 -- 172.21.15.7:0/766935138 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x564005f2f460 con 0x56400605b8e0 2024-06-10T01:19:15.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.100+0000 7f666558e700 1 --2- 172.21.15.7:0/766935138 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56400605b8e0 0x56400605dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:15.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.101+0000 7f666558e700 1 --2- 172.21.15.7:0/766935138 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56400605b8e0 0x56400605dcd0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc546f84a57c19d1 server_cookie=f8e33772b711b1ad in_seq=0 out_seq=0 2024-06-10T01:19:15.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.101+0000 7f66497fa700 10 client.28105 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:15.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.151+0000 7f66497fa700 1 -- 172.21.15.7:0/766935138 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564005f2f460 con 0x56400605b8e0 2024-06-10T01:19:15.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.151+0000 7f66497fa700 10 client.28105 handle_command_reply: tid=0 2024-06-10T01:19:15.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.151+0000 7f6672876b80 1 client.28105 shutdown 2024-06-10T01:19:15.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.152+0000 7f6672876b80 1 -- 172.21.15.7:0/766935138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564005fd6ce0 msgr2=0x564005ff39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:15.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.152+0000 7f6672876b80 1 --2- 172.21.15.7:0/766935138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564005fd6ce0 0x564005ff39c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f665c0ccb60 tx=0x7f665c009880 comp rx=0 tx=0).stop 2024-06-10T01:19:15.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.152+0000 7f6672876b80 1 -- 172.21.15.7:0/766935138 shutdown_connections 2024-06-10T01:19:15.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.152+0000 7f6672876b80 1 --2- 172.21.15.7:0/766935138 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56400605b8e0 0x56400605dcd0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.152+0000 7f6672876b80 1 --2- 172.21.15.7:0/766935138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564005fdceb0 0x564005ff4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.152+0000 7f6672876b80 1 --2- 172.21.15.7:0/766935138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564005fd6ce0 0x564005ff39c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.152+0000 7f6672876b80 1 --2- 172.21.15.7:0/766935138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564005fdc590 0x564005ff3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.152+0000 7f66497fa700 0 client.28105 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:15.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.152+0000 7f6672876b80 1 -- 172.21.15.7:0/766935138 >> 172.21.15.7:0/766935138 conn(0x564005f03210 msgr2=0x564005fd0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:15.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.152+0000 7f6672876b80 1 -- 172.21.15.7:0/766935138 shutdown_connections 2024-06-10T01:19:15.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.152+0000 7f6672876b80 1 -- 172.21.15.7:0/766935138 wait complete. 2024-06-10T01:19:15.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.152+0000 7f6672876b80 20 client.28105 trim_cache size 0 max 16384 2024-06-10T01:19:15.147 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:19:15.147 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 40, 2024-06-10T01:19:15.147 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:19:15.147 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 42, 2024-06-10T01:19:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:19:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "b3de2d3a": { 2024-06-10T01:19:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 42, 2024-06-10T01:19:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:19:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:19:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:19:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:19:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:19:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:19:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 130.0, 2024-06-10T01:19:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:19:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:19:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:19:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:19:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:19:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:19:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:19:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:19:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:19:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:19:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:19:15.149 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:19:15.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.153+0000 7f6667ff3700 1 -- 172.21.15.7:0/3003351836 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f664c0a7240 msgr2=0x7f664c0a9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:15.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.153+0000 7f6667ff3700 1 --2- 172.21.15.7:0/3003351836 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f664c0a7240 0x7f664c0a9700 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6650005b40 tx=0x7f665001a040 comp rx=0 tx=0).stop 2024-06-10T01:19:15.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.153+0000 7f6667ff3700 1 -- 172.21.15.7:0/3003351836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66601013c0 msgr2=0x7f666019d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:15.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.153+0000 7f6667ff3700 1 --2- 172.21.15.7:0/3003351836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66601013c0 0x7f666019d6c0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f665c00eab0 tx=0x7f665c00edc0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.153+0000 7f6667ff3700 1 -- 172.21.15.7:0/3003351836 shutdown_connections 2024-06-10T01:19:15.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.153+0000 7f6667ff3700 1 --2- 172.21.15.7:0/3003351836 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f664c0a7240 0x7f664c0a9700 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.153+0000 7f6667ff3700 1 --2- 172.21.15.7:0/3003351836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6660107470 0x7f6660197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.153+0000 7f6667ff3700 1 --2- 172.21.15.7:0/3003351836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66601013c0 0x7f666019d6c0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.153+0000 7f6667ff3700 1 --2- 172.21.15.7:0/3003351836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6660101d80 0x7f666019dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:15.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.153+0000 7f6667ff3700 1 -- 172.21.15.7:0/3003351836 >> 172.21.15.7:0/3003351836 conn(0x7f66600783d0 msgr2=0x7f6660100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:15.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.153+0000 7f6667ff3700 1 -- 172.21.15.7:0/3003351836 shutdown_connections 2024-06-10T01:19:15.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:15.153+0000 7f6667ff3700 1 -- 172.21.15.7:0/3003351836 wait complete. 2024-06-10T01:19:15.623 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:15 smithi007 ceph-mon[32398]: pgmap v829: 129 pgs: 129 active+clean; 162 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 3.4 MiB/s wr, 311 op/s 2024-06-10T01:19:15.758 INFO:tasks.workunit.client.0.smithi007.stdout: 1 981607 7.36 MB/sec execute 596 sec latency 662.602 ms 2024-06-10T01:19:15.830 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:15 smithi078 ceph-mon[36683]: pgmap v829: 129 pgs: 129 active+clean; 162 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 3.4 MiB/s wr, 311 op/s 2024-06-10T01:19:15.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:15 smithi175 ceph-mon[36636]: pgmap v829: 129 pgs: 129 active+clean; 162 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 3.4 MiB/s wr, 311 op/s 2024-06-10T01:19:16.596 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b3de2d3a', quiesce took 9.6 seconds. Will release after: 0.40000000000000036 2024-06-10T01:19:16.758 INFO:tasks.workunit.client.0.smithi007.stdout: 1 981607 7.34 MB/sec execute 597 sec latency 1662.674 ms 2024-06-10T01:19:16.998 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:19:17.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.046+0000 7f58058da640 1 -- 172.21.15.7:0/3938225572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5800109c00 msgr2=0x7f5800110490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:17.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.046+0000 7f58058da640 1 --2- 172.21.15.7:0/3938225572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5800109c00 0x7f5800110490 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f57f8009ca0 tx=0x7f57f802a8c0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.046+0000 7f58058da640 1 -- 172.21.15.7:0/3938225572 shutdown_connections 2024-06-10T01:19:17.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.046+0000 7f58058da640 1 --2- 172.21.15.7:0/3938225572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5800109c00 0x7f5800110490 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.046+0000 7f58058da640 1 --2- 172.21.15.7:0/3938225572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5800109240 0x7f58001096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.046+0000 7f58058da640 1 --2- 172.21.15.7:0/3938225572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58001080a0 0x7f58001084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.046+0000 7f58058da640 1 -- 172.21.15.7:0/3938225572 >> 172.21.15.7:0/3938225572 conn(0x7f5800076ff0 msgr2=0x7f5800079430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:17.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.046+0000 7f58058da640 1 -- 172.21.15.7:0/3938225572 shutdown_connections 2024-06-10T01:19:17.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.046+0000 7f58058da640 1 -- 172.21.15.7:0/3938225572 wait complete. 2024-06-10T01:19:17.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.047+0000 7f58058da640 1 Processor -- start 2024-06-10T01:19:17.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.047+0000 7f58058da640 1 -- start start 2024-06-10T01:19:17.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.047+0000 7f58058da640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58001080a0 0x7f580019e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.047+0000 7f58058da640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5800109240 0x7f580019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.047+0000 7f58058da640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5800109c00 0x7f58001a5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.047+0000 7f58058da640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5800113ce0 con 0x7f5800109c00 2024-06-10T01:19:17.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.047+0000 7f58058da640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5800113b60 con 0x7f5800109240 2024-06-10T01:19:17.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.047+0000 7f58058da640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5800113e60 con 0x7f58001080a0 2024-06-10T01:19:17.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.047+0000 7f57ff7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5800109c00 0x7f58001a5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.047+0000 7f57ff7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5800109c00 0x7f58001a5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55298/0 (socket says 172.21.15.7:55298) 2024-06-10T01:19:17.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.047+0000 7f57ff7fe640 1 -- 172.21.15.7:0/398052713 learned_addr learned my addr 172.21.15.7:0/398052713 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:17.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.047+0000 7f57f7fff640 1 --2- 172.21.15.7:0/398052713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5800109240 0x7f580019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.047+0000 7f57feffd640 1 --2- 172.21.15.7:0/398052713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58001080a0 0x7f580019e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.048+0000 7f57ff7fe640 1 -- 172.21.15.7:0/398052713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58001080a0 msgr2=0x7f580019e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:17.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.048+0000 7f57ff7fe640 1 --2- 172.21.15.7:0/398052713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58001080a0 0x7f580019e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.048+0000 7f57ff7fe640 1 -- 172.21.15.7:0/398052713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5800109240 msgr2=0x7f580019e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:17.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.048+0000 7f57ff7fe640 1 --2- 172.21.15.7:0/398052713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5800109240 0x7f580019e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.048+0000 7f57ff7fe640 1 -- 172.21.15.7:0/398052713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58001a5ed0 con 0x7f5800109c00 2024-06-10T01:19:17.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.048+0000 7f57f7fff640 1 --2- 172.21.15.7:0/398052713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5800109240 0x7f580019e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:17.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.048+0000 7f57ff7fe640 1 --2- 172.21.15.7:0/398052713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5800109c00 0x7f58001a5860 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f57f8031ae0 tx=0x7f57f8031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:17.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.048+0000 7f58048d8640 1 -- 172.21.15.7:0/398052713 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f8003c30 con 0x7f5800109c00 2024-06-10T01:19:17.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.048+0000 7f58048d8640 1 -- 172.21.15.7:0/398052713 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57f8003dd0 con 0x7f5800109c00 2024-06-10T01:19:17.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.048+0000 7f58058da640 1 -- 172.21.15.7:0/398052713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58001a6160 con 0x7f5800109c00 2024-06-10T01:19:17.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.048+0000 7f58048d8640 1 -- 172.21.15.7:0/398052713 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f802fcb0 con 0x7f5800109c00 2024-06-10T01:19:17.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.048+0000 7f58058da640 1 -- 172.21.15.7:0/398052713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58001a66b0 con 0x7f5800109c00 2024-06-10T01:19:17.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.051+0000 7f58048d8640 1 -- 172.21.15.7:0/398052713 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f57f8004050 con 0x7f5800109c00 2024-06-10T01:19:17.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.052+0000 7f58048d8640 1 --2- 172.21.15.7:0/398052713 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f57ec0a2b80 0x7f57ec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.052+0000 7f58085ef740 1 Processor -- start 2024-06-10T01:19:17.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.052+0000 7f58085ef740 1 -- start start 2024-06-10T01:19:17.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.052+0000 7f57feffd640 1 --2- 172.21.15.7:0/398052713 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f57ec0a2b80 0x7f57ec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.052+0000 7f58048d8640 1 -- 172.21.15.7:0/398052713 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f57f80f07f0 con 0x7f5800109c00 2024-06-10T01:19:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.052+0000 7f58085ef740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e03321010 0x556e03321430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.052+0000 7f58085ef740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e03321a00 0x556e03326500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.052+0000 7f58085ef740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e03326b70 0x556e03329000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.052+0000 7f58085ef740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556e032844f0 con 0x556e03326b70 2024-06-10T01:19:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.052+0000 7f58085ef740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556e03222870 con 0x556e03321a00 2024-06-10T01:19:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.052+0000 7f58085ef740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556e032804b0 con 0x556e03321010 2024-06-10T01:19:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.052+0000 7f57ff7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e03326b70 0x556e03329000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.052+0000 7f57ff7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e03326b70 0x556e03329000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55306/0 (socket says 172.21.15.7:55306) 2024-06-10T01:19:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.052+0000 7f57ff7fe640 1 -- 172.21.15.7:0/3452645610 learned_addr learned my addr 172.21.15.7:0/3452645610 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.053+0000 7f57feffd640 1 --2- 172.21.15.7:0/3452645610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e03321010 0x556e03321430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.053+0000 7f57f7fff640 1 --2- 172.21.15.7:0/3452645610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e03321a00 0x556e03326500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.053+0000 7f57feffd640 1 --2- 172.21.15.7:0/398052713 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f57ec0a2b80 0x7f57ec0a5040 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f58001090a0 tx=0x7f57f000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.053+0000 7f57ff7fe640 1 -- 172.21.15.7:0/3452645610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e03321010 msgr2=0x556e03321430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.053+0000 7f57ff7fe640 1 --2- 172.21.15.7:0/3452645610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e03321010 0x556e03321430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.053+0000 7f57ff7fe640 1 -- 172.21.15.7:0/3452645610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e03321a00 msgr2=0x556e03326500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.053+0000 7f57ff7fe640 1 --2- 172.21.15.7:0/3452645610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e03321a00 0x556e03326500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.053+0000 7f57ff7fe640 1 -- 172.21.15.7:0/3452645610 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e0323e9b0 con 0x556e03326b70 2024-06-10T01:19:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.053+0000 7f57f7fff640 1 --2- 172.21.15.7:0/3452645610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e03321a00 0x556e03326500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:19:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.053+0000 7f57ff7fe640 1 --2- 172.21.15.7:0/3452645610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e03326b70 0x556e03329000 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f57f80c5410 tx=0x7f57f8005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c142c1f106a5511 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.053+0000 7f57f5ffb640 1 -- 172.21.15.7:0/3452645610 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f80bd3d0 con 0x556e03326b70 2024-06-10T01:19:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.053+0000 7f57f5ffb640 1 -- 172.21.15.7:0/3452645610 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57f8036060 con 0x556e03326b70 2024-06-10T01:19:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.053+0000 7f57f5ffb640 1 -- 172.21.15.7:0/3452645610 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f80bafd0 con 0x556e03326b70 2024-06-10T01:19:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.054+0000 7f58085ef740 1 -- 172.21.15.7:0/3452645610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e03326b70 msgr2=0x556e03329000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.054+0000 7f58085ef740 1 --2- 172.21.15.7:0/3452645610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e03326b70 0x556e03329000 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f57f80c5410 tx=0x7f57f8005ef0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.054+0000 7f58085ef740 1 -- 172.21.15.7:0/3452645610 shutdown_connections 2024-06-10T01:19:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.054+0000 7f58085ef740 1 --2- 172.21.15.7:0/3452645610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e03326b70 0x556e03329000 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.054+0000 7f58085ef740 1 --2- 172.21.15.7:0/3452645610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e03321a00 0x556e03326500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.054+0000 7f58085ef740 1 --2- 172.21.15.7:0/3452645610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e03321010 0x556e03321430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.054+0000 7f58085ef740 1 -- 172.21.15.7:0/3452645610 >> 172.21.15.7:0/3452645610 conn(0x556e0331c680 msgr2=0x556e0331ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.054+0000 7f58085ef740 1 -- 172.21.15.7:0/3452645610 shutdown_connections 2024-06-10T01:19:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.054+0000 7f58085ef740 1 -- 172.21.15.7:0/3452645610 wait complete. 2024-06-10T01:19:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.055+0000 7f58085ef740 1 -- start start 2024-06-10T01:19:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.055+0000 7f58085ef740 1 Processor -- start 2024-06-10T01:19:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.055+0000 7f58085ef740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e03321010 0x556e0332ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.055+0000 7f58085ef740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e03321a00 0x556e0332b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.055+0000 7f58085ef740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e03326b70 0x556e033323a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.055+0000 7f58085ef740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556e032844f0 con 0x556e03321010 2024-06-10T01:19:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.055+0000 7f58085ef740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556e0320dae0 con 0x556e03321a00 2024-06-10T01:19:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.055+0000 7f58085ef740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556e033298e0 con 0x556e03326b70 2024-06-10T01:19:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.056+0000 7f58085ef740 10 client.0 _ll_register_callbacks cb 0x556e032469b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:19:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.056+0000 7f57feffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e03321010 0x556e0332ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.056+0000 7f57feffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e03321010 0x556e0332ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55310/0 (socket says 172.21.15.7:55310) 2024-06-10T01:19:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.056+0000 7f57feffd640 1 -- 172.21.15.7:0/2700801059 learned_addr learned my addr 172.21.15.7:0/2700801059 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.056+0000 7f57f7fff640 1 --2- 172.21.15.7:0/2700801059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e03321a00 0x556e0332b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.056+0000 7f57ff7fe640 1 --2- 172.21.15.7:0/2700801059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e03326b70 0x556e033323a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.056+0000 7f57feffd640 1 -- 172.21.15.7:0/2700801059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e03326b70 msgr2=0x556e033323a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.056+0000 7f57feffd640 1 --2- 172.21.15.7:0/2700801059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e03326b70 0x556e033323a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.056+0000 7f57feffd640 1 -- 172.21.15.7:0/2700801059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e03321a00 msgr2=0x556e0332b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:17.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.056+0000 7f57feffd640 1 --2- 172.21.15.7:0/2700801059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e03321a00 0x556e0332b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.056+0000 7f57feffd640 1 -- 172.21.15.7:0/2700801059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e02f98210 con 0x556e03321010 2024-06-10T01:19:17.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.056+0000 7f57feffd640 1 --2- 172.21.15.7:0/2700801059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e03321010 0x556e0332ae20 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f57f00116a0 tx=0x7f57f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=313eca93ddd10098 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:17.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.056+0000 7f57deffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:19:17.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.056+0000 7f57deffd640 1 -- 172.21.15.7:0/2700801059 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0004220 con 0x556e03321010 2024-06-10T01:19:17.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.057+0000 7f58085ef740 1 -- 172.21.15.7:0/2700801059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57f8036060 con 0x556e03321010 2024-06-10T01:19:17.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.057+0000 7f57deffd640 1 -- 172.21.15.7:0/2700801059 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f57f00043c0 con 0x556e03321010 2024-06-10T01:19:17.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.057+0000 7f57deffd640 1 -- 172.21.15.7:0/2700801059 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f57f00035f0 con 0x556e03321010 2024-06-10T01:19:17.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.057+0000 7f58085ef740 10 client.18786 fetch_fsmap learned FSMap version 40 2024-06-10T01:19:17.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.057+0000 7f58085ef740 1 -- 172.21.15.7:0/2700801059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x556e03064620 con 0x556e03321010 2024-06-10T01:19:17.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.057+0000 7f57deffd640 1 -- 172.21.15.7:0/2700801059 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57f8036060 con 0x556e03321010 2024-06-10T01:19:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.057+0000 7f57deffd640 1 -- 172.21.15.7:0/2700801059 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f57f001ea90 con 0x556e03321010 2024-06-10T01:19:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.057+0000 7f58085ef740 10 client.18786 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:19:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.057+0000 7f58085ef740 10 client.18786 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:19:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.057+0000 7f58085ef740 20 client.18786 populate_metadata read hostname 'smithi007' 2024-06-10T01:19:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.057+0000 7f58085ef740 1 --2- 172.21.15.7:0/2700801059 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556e03336d50 0x556e033391a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.057+0000 7f58085ef740 4 client.18786 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:19:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.057+0000 7f58085ef740 1 -- 172.21.15.7:0/2700801059 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e03297f10 con 0x556e03336d50 2024-06-10T01:19:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.058+0000 7f57f7fff640 1 --2- 172.21.15.7:0/2700801059 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556e03336d50 0x556e033391a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.058+0000 7f57f7fff640 1 --2- 172.21.15.7:0/2700801059 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556e03336d50 0x556e033391a0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d534229ececcca25 server_cookie=751209ad922c54fb in_seq=0 out_seq=0 2024-06-10T01:19:17.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.058+0000 7f57deffd640 10 client.18786 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:17.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.059+0000 7f57deffd640 1 -- 172.21.15.7:0/2700801059 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556e03297f10 con 0x556e03336d50 2024-06-10T01:19:17.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.059+0000 7f57deffd640 10 client.18786 handle_command_reply: tid=0 2024-06-10T01:19:17.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.059+0000 7f58085ef740 1 client.18786 shutdown 2024-06-10T01:19:17.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.059+0000 7f58085ef740 1 -- 172.21.15.7:0/2700801059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e03321010 msgr2=0x556e0332ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:17.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.059+0000 7f58085ef740 1 --2- 172.21.15.7:0/2700801059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e03321010 0x556e0332ae20 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f57f00116a0 tx=0x7f57f0004730 comp rx=0 tx=0).stop 2024-06-10T01:19:17.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.059+0000 7f58085ef740 1 -- 172.21.15.7:0/2700801059 shutdown_connections 2024-06-10T01:19:17.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.059+0000 7f58085ef740 1 --2- 172.21.15.7:0/2700801059 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556e03336d50 0x556e033391a0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.059+0000 7f58085ef740 1 --2- 172.21.15.7:0/2700801059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e03326b70 0x556e033323a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.059+0000 7f57deffd640 0 client.18786 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:17.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.059+0000 7f58085ef740 1 --2- 172.21.15.7:0/2700801059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e03321a00 0x556e0332b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.059+0000 7f58085ef740 1 --2- 172.21.15.7:0/2700801059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e03321010 0x556e0332ae20 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.059+0000 7f58085ef740 1 -- 172.21.15.7:0/2700801059 >> 172.21.15.7:0/2700801059 conn(0x556e0331c680 msgr2=0x556e0331b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:17.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.059+0000 7f58085ef740 1 -- 172.21.15.7:0/2700801059 shutdown_connections 2024-06-10T01:19:17.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.060+0000 7f58085ef740 1 -- 172.21.15.7:0/2700801059 wait complete. 2024-06-10T01:19:17.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.060+0000 7f58085ef740 20 client.18786 trim_cache size 0 max 16384 2024-06-10T01:19:17.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.085+0000 7f58085ef740 1 Processor -- start 2024-06-10T01:19:17.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.085+0000 7f58085ef740 1 -- start start 2024-06-10T01:19:17.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.086+0000 7f58085ef740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e03321010 0x556e03322c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.086+0000 7f58085ef740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e03321a00 0x556e03323160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.086+0000 7f58085ef740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e03326b70 0x556e03332330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.086+0000 7f58085ef740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f58001b3480 con 0x556e03321a00 2024-06-10T01:19:17.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.086+0000 7f58085ef740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f58001b0d50 con 0x556e03321010 2024-06-10T01:19:17.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.086+0000 7f58085ef740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f58001aeba0 con 0x556e03326b70 2024-06-10T01:19:17.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.086+0000 7f57feffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e03321010 0x556e03322c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.086+0000 7f57f7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e03321a00 0x556e03323160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.086+0000 7f57feffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e03321010 0x556e03322c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:48968/0 (socket says 172.21.15.7:48968) 2024-06-10T01:19:17.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.086+0000 7f57feffd640 1 -- 172.21.15.7:0/77958201 learned_addr learned my addr 172.21.15.7:0/77958201 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:17.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.086+0000 7f57ff7fe640 1 --2- 172.21.15.7:0/77958201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e03326b70 0x556e03332330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.086+0000 7f57feffd640 1 -- 172.21.15.7:0/77958201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e03326b70 msgr2=0x556e03332330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:17.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.086+0000 7f57feffd640 1 --2- 172.21.15.7:0/77958201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e03326b70 0x556e03332330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.086+0000 7f57feffd640 1 -- 172.21.15.7:0/77958201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e03321a00 msgr2=0x556e03323160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:17.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.086+0000 7f57feffd640 1 --2- 172.21.15.7:0/77958201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e03321a00 0x556e03323160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.086+0000 7f57feffd640 1 -- 172.21.15.7:0/77958201 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e0331c4b0 con 0x556e03321010 2024-06-10T01:19:17.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.087+0000 7f57feffd640 1 --2- 172.21.15.7:0/77958201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e03321010 0x556e03322c20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f57f001ef00 tx=0x7f57f000b400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47d785d38dc939ed server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.087+0000 7f57f57fa640 1 -- 172.21.15.7:0/77958201 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f003a070 con 0x556e03321010 2024-06-10T01:19:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.087+0000 7f57f57fa640 1 -- 172.21.15.7:0/77958201 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57f001e500 con 0x556e03321010 2024-06-10T01:19:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.087+0000 7f57f57fa640 1 -- 172.21.15.7:0/77958201 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0036670 con 0x556e03321010 2024-06-10T01:19:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.088+0000 7f58085ef740 1 -- 172.21.15.7:0/77958201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e03321010 msgr2=0x556e03322c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.088+0000 7f58085ef740 1 --2- 172.21.15.7:0/77958201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e03321010 0x556e03322c20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f57f001ef00 tx=0x7f57f000b400 comp rx=0 tx=0).stop 2024-06-10T01:19:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.088+0000 7f58085ef740 1 -- 172.21.15.7:0/77958201 shutdown_connections 2024-06-10T01:19:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.088+0000 7f58085ef740 1 --2- 172.21.15.7:0/77958201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e03326b70 0x556e03332330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.088+0000 7f58085ef740 1 --2- 172.21.15.7:0/77958201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e03321a00 0x556e03323160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.088+0000 7f58085ef740 1 --2- 172.21.15.7:0/77958201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e03321010 0x556e03322c20 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.088+0000 7f58085ef740 1 -- 172.21.15.7:0/77958201 >> 172.21.15.7:0/77958201 conn(0x556e0331c680 msgr2=0x556e03326280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.088+0000 7f58085ef740 1 -- 172.21.15.7:0/77958201 shutdown_connections 2024-06-10T01:19:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.088+0000 7f58085ef740 1 -- 172.21.15.7:0/77958201 wait complete. 2024-06-10T01:19:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.088+0000 7f58085ef740 1 -- start start 2024-06-10T01:19:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.088+0000 7f58085ef740 1 Processor -- start 2024-06-10T01:19:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.089+0000 7f58085ef740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e03321010 0x556e03332230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.089+0000 7f58085ef740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e03321a00 0x556e0332ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.089+0000 7f58085ef740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e03326b70 0x556e0332f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.089+0000 7f58085ef740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f58001af210 con 0x556e03321a00 2024-06-10T01:19:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.089+0000 7f58085ef740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f58001b0680 con 0x556e03321010 2024-06-10T01:19:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.089+0000 7f58085ef740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f58001af8e0 con 0x556e03326b70 2024-06-10T01:19:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.089+0000 7f58085ef740 10 client.0 _ll_register_callbacks cb 0x7f57f0010c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:19:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.089+0000 7f57f7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e03321a00 0x556e0332ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.089+0000 7f57feffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e03321010 0x556e03332230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.089+0000 7f57f7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e03321a00 0x556e0332ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55336/0 (socket says 172.21.15.7:55336) 2024-06-10T01:19:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.089+0000 7f57f7fff640 1 -- 172.21.15.7:0/2208250873 learned_addr learned my addr 172.21.15.7:0/2208250873 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.089+0000 7f57ff7fe640 1 --2- 172.21.15.7:0/2208250873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e03326b70 0x556e0332f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.089+0000 7f57feffd640 1 -- 172.21.15.7:0/2208250873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e03326b70 msgr2=0x556e0332f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.089+0000 7f57feffd640 1 --2- 172.21.15.7:0/2208250873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e03326b70 0x556e0332f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.089+0000 7f57feffd640 1 -- 172.21.15.7:0/2208250873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e03321a00 msgr2=0x556e0332ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.089+0000 7f57feffd640 1 --2- 172.21.15.7:0/2208250873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e03321a00 0x556e0332ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.089+0000 7f57feffd640 1 -- 172.21.15.7:0/2208250873 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e0339fa70 con 0x556e03321010 2024-06-10T01:19:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.089+0000 7f57f7fff640 1 --2- 172.21.15.7:0/2208250873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e03321a00 0x556e0332ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.090+0000 7f57feffd640 1 --2- 172.21.15.7:0/2208250873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e03321010 0x556e03332230 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f57f0010860 tx=0x7f57f001ea20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d4c26702faf7032 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.090+0000 7f57deffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:19:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.090+0000 7f57deffd640 1 -- 172.21.15.7:0/2208250873 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f003a030 con 0x556e03321010 2024-06-10T01:19:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.090+0000 7f58085ef740 1 -- 172.21.15.7:0/2208250873 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57f001e500 con 0x556e03321010 2024-06-10T01:19:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.090+0000 7f57deffd640 1 -- 172.21.15.7:0/2208250873 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f57f0028bc0 con 0x556e03321010 2024-06-10T01:19:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.090+0000 7f57deffd640 1 -- 172.21.15.7:0/2208250873 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f57f0028dc0 con 0x556e03321010 2024-06-10T01:19:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.090+0000 7f58085ef740 10 client.28103 fetch_fsmap learned FSMap version 40 2024-06-10T01:19:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.090+0000 7f58085ef740 1 -- 172.21.15.7:0/2208250873 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x556e033a0c00 con 0x556e03321010 2024-06-10T01:19:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.090+0000 7f57deffd640 1 -- 172.21.15.7:0/2208250873 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57f001e500 con 0x556e03321010 2024-06-10T01:19:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.091+0000 7f57deffd640 1 -- 172.21.15.7:0/2208250873 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f57f003f520 con 0x556e03321010 2024-06-10T01:19:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.091+0000 7f58085ef740 10 client.28103 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:19:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.091+0000 7f58085ef740 10 client.28103 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:19:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.091+0000 7f58085ef740 20 client.28103 populate_metadata read hostname 'smithi007' 2024-06-10T01:19:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.091+0000 7f58085ef740 1 --2- 172.21.15.7:0/2208250873 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556e03336d50 0x556e033a3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.091+0000 7f58085ef740 4 client.28103 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:19:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.091+0000 7f58085ef740 1 -- 172.21.15.7:0/2208250873 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556e03291c20 con 0x556e03336d50 2024-06-10T01:19:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.091+0000 7f57f7fff640 1 --2- 172.21.15.7:0/2208250873 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556e03336d50 0x556e033a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.091+0000 7f57deffd640 10 client.28103 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.091+0000 7f57f7fff640 1 --2- 172.21.15.7:0/2208250873 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556e03336d50 0x556e033a3570 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=298143e4a51c3b6c server_cookie=9e89ba494e25e3e9 in_seq=0 out_seq=0 2024-06-10T01:19:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.172+0000 7f57deffd640 1 -- 172.21.15.7:0/2208250873 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556e03291c20 con 0x556e03336d50 2024-06-10T01:19:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.172+0000 7f57deffd640 10 client.28103 handle_command_reply: tid=0 2024-06-10T01:19:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.172+0000 7f58085ef740 1 client.28103 shutdown 2024-06-10T01:19:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.172+0000 7f58085ef740 1 -- 172.21.15.7:0/2208250873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e03321010 msgr2=0x556e03332230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.172+0000 7f58085ef740 1 --2- 172.21.15.7:0/2208250873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e03321010 0x556e03332230 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f57f0010860 tx=0x7f57f001ea20 comp rx=0 tx=0).stop 2024-06-10T01:19:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.172+0000 7f58085ef740 1 -- 172.21.15.7:0/2208250873 shutdown_connections 2024-06-10T01:19:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.172+0000 7f58085ef740 1 --2- 172.21.15.7:0/2208250873 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556e03336d50 0x556e033a3570 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.172+0000 7f58085ef740 1 --2- 172.21.15.7:0/2208250873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e03326b70 0x556e0332f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.172+0000 7f58085ef740 1 --2- 172.21.15.7:0/2208250873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e03321a00 0x556e0332ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.172+0000 7f58085ef740 1 --2- 172.21.15.7:0/2208250873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e03321010 0x556e03332230 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.172+0000 7f58085ef740 1 -- 172.21.15.7:0/2208250873 >> 172.21.15.7:0/2208250873 conn(0x556e0331c680 msgr2=0x556e03325e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.172+0000 7f57deffd640 0 client.28103 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.172+0000 7f58085ef740 1 -- 172.21.15.7:0/2208250873 shutdown_connections 2024-06-10T01:19:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.172+0000 7f58085ef740 1 -- 172.21.15.7:0/2208250873 wait complete. 2024-06-10T01:19:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.172+0000 7f58085ef740 20 client.28103 trim_cache size 0 max 16384 2024-06-10T01:19:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:19:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:19:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:19:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.173+0000 7f58058da640 1 -- 172.21.15.7:0/398052713 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f57ec0a2b80 msgr2=0x7f57ec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.173+0000 7f58058da640 1 --2- 172.21.15.7:0/398052713 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f57ec0a2b80 0x7f57ec0a5040 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f58001090a0 tx=0x7f57f000b390 comp rx=0 tx=0).stop 2024-06-10T01:19:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.173+0000 7f58058da640 1 -- 172.21.15.7:0/398052713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5800109c00 msgr2=0x7f58001a5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.173+0000 7f58058da640 1 --2- 172.21.15.7:0/398052713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5800109c00 0x7f58001a5860 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f57f8031ae0 tx=0x7f57f8031ee0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.173+0000 7f58058da640 1 -- 172.21.15.7:0/398052713 shutdown_connections 2024-06-10T01:19:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.173+0000 7f58058da640 1 --2- 172.21.15.7:0/398052713 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f57ec0a2b80 0x7f57ec0a5040 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.173+0000 7f58058da640 1 --2- 172.21.15.7:0/398052713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5800109c00 0x7f58001a5860 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.173+0000 7f58058da640 1 --2- 172.21.15.7:0/398052713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5800109240 0x7f580019e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.173+0000 7f58058da640 1 --2- 172.21.15.7:0/398052713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58001080a0 0x7f580019e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.173+0000 7f58058da640 1 -- 172.21.15.7:0/398052713 >> 172.21.15.7:0/398052713 conn(0x7f5800076ff0 msgr2=0x7f580010c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.173+0000 7f58058da640 1 -- 172.21.15.7:0/398052713 shutdown_connections 2024-06-10T01:19:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:17.173+0000 7f58058da640 1 -- 172.21.15.7:0/398052713 wait complete. 2024-06-10T01:19:17.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.561+0000 7f4f767a7700 1 -- 172.21.15.7:0/221467315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f70106b90 msgr2=0x7f4f70106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.561+0000 7f4f767a7700 1 --2- 172.21.15.7:0/221467315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f70106b90 0x7f4f70106f70 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f4f60009990 tx=0x7f4f60009ca0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.561+0000 7f4f767a7700 1 -- 172.21.15.7:0/221467315 shutdown_connections 2024-06-10T01:19:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.561+0000 7f4f767a7700 1 --2- 172.21.15.7:0/221467315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f70101510 0x7f4f7010d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.561+0000 7f4f767a7700 1 --2- 172.21.15.7:0/221467315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f70106b90 0x7f4f70106f70 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.561+0000 7f4f767a7700 1 --2- 172.21.15.7:0/221467315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f70100b50 0x7f4f70100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.561+0000 7f4f767a7700 1 -- 172.21.15.7:0/221467315 >> 172.21.15.7:0/221467315 conn(0x7f4f70077a00 msgr2=0x7f4f700fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.561+0000 7f4f767a7700 1 -- 172.21.15.7:0/221467315 shutdown_connections 2024-06-10T01:19:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.561+0000 7f4f767a7700 1 -- 172.21.15.7:0/221467315 wait complete. 2024-06-10T01:19:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.562+0000 7f4f767a7700 1 Processor -- start 2024-06-10T01:19:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.562+0000 7f4f767a7700 1 -- start start 2024-06-10T01:19:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.562+0000 7f4f767a7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f70100b50 0x7f4f7019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.562+0000 7f4f767a7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f70101510 0x7f4f7019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.562+0000 7f4f767a7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f70106b90 0x7f4f701989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.562+0000 7f4f767a7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4f701105c0 con 0x7f4f70100b50 2024-06-10T01:19:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.562+0000 7f4f767a7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4f70110440 con 0x7f4f70101510 2024-06-10T01:19:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.562+0000 7f4f767a7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4f70110740 con 0x7f4f70106b90 2024-06-10T01:19:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.563+0000 7f4f6ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f70100b50 0x7f4f7019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.563+0000 7f4f67fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f70101510 0x7f4f7019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.563+0000 7f4f67fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f70101510 0x7f4f7019ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:49002/0 (socket says 172.21.15.7:49002) 2024-06-10T01:19:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.563+0000 7f4f67fff700 1 -- 172.21.15.7:0/1425213136 learned_addr learned my addr 172.21.15.7:0/1425213136 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.563+0000 7f4f74d44700 1 --2- 172.21.15.7:0/1425213136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f70106b90 0x7f4f701989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.563+0000 7f4f67fff700 1 -- 172.21.15.7:0/1425213136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f70106b90 msgr2=0x7f4f701989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.563+0000 7f4f67fff700 1 --2- 172.21.15.7:0/1425213136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f70106b90 0x7f4f701989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.563+0000 7f4f67fff700 1 -- 172.21.15.7:0/1425213136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f70100b50 msgr2=0x7f4f7019e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:17.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.563+0000 7f4f67fff700 1 --2- 172.21.15.7:0/1425213136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f70100b50 0x7f4f7019e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.563+0000 7f4f67fff700 1 -- 172.21.15.7:0/1425213136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f70199140 con 0x7f4f70101510 2024-06-10T01:19:17.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.563+0000 7f4f6ffff700 1 --2- 172.21.15.7:0/1425213136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f70100b50 0x7f4f7019e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:17.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.563+0000 7f4f67fff700 1 --2- 172.21.15.7:0/1425213136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f70101510 0x7f4f7019ed60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4f5800eab0 tx=0x7f4f5800edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:17.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.563+0000 7f4f6dffb700 1 -- 172.21.15.7:0/1425213136 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5800ccd0 con 0x7f4f70101510 2024-06-10T01:19:17.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.564+0000 7f4f767a7700 1 -- 172.21.15.7:0/1425213136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f70199430 con 0x7f4f70101510 2024-06-10T01:19:17.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.564+0000 7f4f6dffb700 1 -- 172.21.15.7:0/1425213136 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f580168c0 con 0x7f4f70101510 2024-06-10T01:19:17.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.564+0000 7f4f6dffb700 1 -- 172.21.15.7:0/1425213136 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f58015850 con 0x7f4f70101510 2024-06-10T01:19:17.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.564+0000 7f4f767a7700 1 -- 172.21.15.7:0/1425213136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f701a56f0 con 0x7f4f70101510 2024-06-10T01:19:17.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.567+0000 7f4f6dffb700 1 -- 172.21.15.7:0/1425213136 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f4f580163e0 con 0x7f4f70101510 2024-06-10T01:19:17.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:17 smithi007 ceph-mon[32398]: pgmap v830: 129 pgs: 129 active+clean; 170 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 3.7 MiB/s wr, 286 op/s 2024-06-10T01:19:17.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.567+0000 7f4f6dffb700 1 --2- 172.21.15.7:0/1425213136 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4f5c0a2ca0 0x7f4f5c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.568+0000 7f4f6dffb700 1 -- 172.21.15.7:0/1425213136 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f4f580cf6b0 con 0x7f4f70101510 2024-06-10T01:19:17.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.568+0000 7f4f6ffff700 1 --2- 172.21.15.7:0/1425213136 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4f5c0a2ca0 0x7f4f5c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.568+0000 7f4f8102ab80 1 Processor -- start 2024-06-10T01:19:17.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.568+0000 7f4f8102ab80 1 -- start start 2024-06-10T01:19:17.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.568+0000 7f4f6ffff700 1 --2- 172.21.15.7:0/1425213136 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4f5c0a2ca0 0x7f4f5c0a5160 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4f6000b2e0 tx=0x7f4f60005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:17.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.568+0000 7f4f8102ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b96849440 0x559b96849820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.568+0000 7f4f8102ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b96849d60 0x559b9684dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.568+0000 7f4f8102ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b9684e1e0 0x559b9684e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.568+0000 7f4f8102ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559b9673bf60 con 0x559b96849440 2024-06-10T01:19:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.568+0000 7f4f8102ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559b9679a850 con 0x559b96849d60 2024-06-10T01:19:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.568+0000 7f4f8102ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559b96702460 con 0x559b9684e1e0 2024-06-10T01:19:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.568+0000 7f4f6ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b96849440 0x559b96849820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.568+0000 7f4f6ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b96849440 0x559b96849820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55358/0 (socket says 172.21.15.7:55358) 2024-06-10T01:19:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.568+0000 7f4f6ffff700 1 -- 172.21.15.7:0/49534377 learned_addr learned my addr 172.21.15.7:0/49534377 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.568+0000 7f4f67fff700 1 --2- 172.21.15.7:0/49534377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b96849d60 0x559b9684dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.569+0000 7f4f74d44700 1 --2- 172.21.15.7:0/49534377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b9684e1e0 0x559b9684e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.569+0000 7f4f67fff700 1 -- 172.21.15.7:0/49534377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b9684e1e0 msgr2=0x559b9684e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.569+0000 7f4f67fff700 1 --2- 172.21.15.7:0/49534377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b9684e1e0 0x559b9684e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.569+0000 7f4f67fff700 1 -- 172.21.15.7:0/49534377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b96849440 msgr2=0x559b96849820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.569+0000 7f4f67fff700 1 --2- 172.21.15.7:0/49534377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b96849440 0x559b96849820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.569+0000 7f4f67fff700 1 -- 172.21.15.7:0/49534377 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b966e98d0 con 0x559b96849d60 2024-06-10T01:19:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.569+0000 7f4f6ffff700 1 --2- 172.21.15.7:0/49534377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b96849440 0x559b96849820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.569+0000 7f4f67fff700 1 --2- 172.21.15.7:0/49534377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b96849d60 0x559b9684dca0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4f5800c6b0 tx=0x7f4f5809b590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e2d5057c937983a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.569+0000 7f4f667fc700 1 -- 172.21.15.7:0/49534377 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f580b4020 con 0x559b96849d60 2024-06-10T01:19:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.569+0000 7f4f667fc700 1 -- 172.21.15.7:0/49534377 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f5809bb30 con 0x559b96849d60 2024-06-10T01:19:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.569+0000 7f4f667fc700 1 -- 172.21.15.7:0/49534377 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5809ce60 con 0x559b96849d60 2024-06-10T01:19:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.570+0000 7f4f8102ab80 1 -- 172.21.15.7:0/49534377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b96849d60 msgr2=0x559b9684dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.570+0000 7f4f8102ab80 1 --2- 172.21.15.7:0/49534377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b96849d60 0x559b9684dca0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4f5800c6b0 tx=0x7f4f5809b590 comp rx=0 tx=0).stop 2024-06-10T01:19:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.570+0000 7f4f8102ab80 1 -- 172.21.15.7:0/49534377 shutdown_connections 2024-06-10T01:19:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.570+0000 7f4f8102ab80 1 --2- 172.21.15.7:0/49534377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b9684e1e0 0x559b9684e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.570+0000 7f4f8102ab80 1 --2- 172.21.15.7:0/49534377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b96849440 0x559b96849820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.570+0000 7f4f8102ab80 1 --2- 172.21.15.7:0/49534377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b96849d60 0x559b9684dca0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.570+0000 7f4f8102ab80 1 -- 172.21.15.7:0/49534377 >> 172.21.15.7:0/49534377 conn(0x559b96776080 msgr2=0x559b96844470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.570+0000 7f4f8102ab80 1 -- 172.21.15.7:0/49534377 shutdown_connections 2024-06-10T01:19:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.570+0000 7f4f8102ab80 1 -- 172.21.15.7:0/49534377 wait complete. 2024-06-10T01:19:17.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.571+0000 7f4f8102ab80 1 -- start start 2024-06-10T01:19:17.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.571+0000 7f4f8102ab80 1 Processor -- start 2024-06-10T01:19:17.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.571+0000 7f4f8102ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b96849440 0x559b96869cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.571+0000 7f4f8102ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b96849d60 0x559b9686a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.571+0000 7f4f8102ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9684e1e0 0x559b96864070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.572+0000 7f4f8102ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559b9673afc0 con 0x559b96849d60 2024-06-10T01:19:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.572+0000 7f4f8102ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559b96796810 con 0x559b9684e1e0 2024-06-10T01:19:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.572+0000 7f4f8102ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559b9684afb0 con 0x559b96849440 2024-06-10T01:19:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.572+0000 7f4f6ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b96849440 0x559b96869cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.572+0000 7f4f6ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b96849440 0x559b96869cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47762/0 (socket says 172.21.15.7:47762) 2024-06-10T01:19:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.572+0000 7f4f6ffff700 1 -- 172.21.15.7:0/771340287 learned_addr learned my addr 172.21.15.7:0/771340287 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.572+0000 7f4f67fff700 1 --2- 172.21.15.7:0/771340287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b96849d60 0x559b9686a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.572+0000 7f4f8102ab80 10 client.0 _ll_register_callbacks cb 0x559b967305b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:19:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.572+0000 7f4f74d44700 1 --2- 172.21.15.7:0/771340287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9684e1e0 0x559b96864070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.572+0000 7f4f67fff700 1 -- 172.21.15.7:0/771340287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b96849440 msgr2=0x559b96869cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.572+0000 7f4f67fff700 1 --2- 172.21.15.7:0/771340287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b96849440 0x559b96869cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.572+0000 7f4f67fff700 1 -- 172.21.15.7:0/771340287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9684e1e0 msgr2=0x559b96864070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.572+0000 7f4f67fff700 1 --2- 172.21.15.7:0/771340287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9684e1e0 0x559b96864070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.572+0000 7f4f67fff700 1 -- 172.21.15.7:0/771340287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b96760750 con 0x559b96849d60 2024-06-10T01:19:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.572+0000 7f4f67fff700 1 --2- 172.21.15.7:0/771340287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b96849d60 0x559b9686a230 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x559b9684a330 tx=0x7f4f5809ff50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350d83a140446ae7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.572+0000 7f4f4f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:19:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.573+0000 7f4f4f7fe700 1 -- 172.21.15.7:0/771340287 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f580b4040 con 0x559b96849d60 2024-06-10T01:19:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.573+0000 7f4f4f7fe700 1 -- 172.21.15.7:0/771340287 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f4f580ba300 con 0x559b96849d60 2024-06-10T01:19:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.573+0000 7f4f8102ab80 1 -- 172.21.15.7:0/771340287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f5809bb30 con 0x559b96849d60 2024-06-10T01:19:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.573+0000 7f4f4f7fe700 1 -- 172.21.15.7:0/771340287 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f4f5809ce60 con 0x559b96849d60 2024-06-10T01:19:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.573+0000 7f4f8102ab80 10 client.18804 fetch_fsmap learned FSMap version 40 2024-06-10T01:19:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.573+0000 7f4f8102ab80 1 -- 172.21.15.7:0/771340287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559b96871a50 con 0x559b96849d60 2024-06-10T01:19:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.573+0000 7f4f4f7fe700 1 -- 172.21.15.7:0/771340287 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f5807f070 con 0x559b96849d60 2024-06-10T01:19:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.573+0000 7f4f4f7fe700 1 -- 172.21.15.7:0/771340287 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f4f58015070 con 0x559b96849d60 2024-06-10T01:19:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.573+0000 7f4f8102ab80 10 client.18804 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:19:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.573+0000 7f4f8102ab80 10 client.18804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:19:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.574+0000 7f4f8102ab80 20 client.18804 populate_metadata read hostname 'smithi007' 2024-06-10T01:19:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.574+0000 7f4f8102ab80 1 --2- 172.21.15.7:0/771340287 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559b96873f20 0x559b96876310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.574+0000 7f4f8102ab80 4 client.18804 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:19:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.574+0000 7f4f8102ab80 1 -- 172.21.15.7:0/771340287 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4f5809bb30 con 0x559b96873f20 2024-06-10T01:19:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.574+0000 7f4f6ffff700 1 --2- 172.21.15.7:0/771340287 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559b96873f20 0x559b96876310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.575+0000 7f4f4f7fe700 10 client.18804 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.575+0000 7f4f6ffff700 1 --2- 172.21.15.7:0/771340287 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559b96873f20 0x559b96876310 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c04a33df8e25580 server_cookie=70c071fa91f5ddcf in_seq=0 out_seq=0 2024-06-10T01:19:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.575+0000 7f4f4f7fe700 1 -- 172.21.15.7:0/771340287 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f4f5809bb30 con 0x559b96873f20 2024-06-10T01:19:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.575+0000 7f4f4f7fe700 10 client.18804 handle_command_reply: tid=0 2024-06-10T01:19:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.575+0000 7f4f8102ab80 1 client.18804 shutdown 2024-06-10T01:19:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.576+0000 7f4f8102ab80 1 -- 172.21.15.7:0/771340287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b96849d60 msgr2=0x559b9686a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.576+0000 7f4f8102ab80 1 --2- 172.21.15.7:0/771340287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b96849d60 0x559b9686a230 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x559b9684a330 tx=0x7f4f5809ff50 comp rx=0 tx=0).stop 2024-06-10T01:19:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.576+0000 7f4f8102ab80 1 -- 172.21.15.7:0/771340287 shutdown_connections 2024-06-10T01:19:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.576+0000 7f4f8102ab80 1 --2- 172.21.15.7:0/771340287 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559b96873f20 0x559b96876310 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.576+0000 7f4f4f7fe700 0 client.18804 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.576+0000 7f4f8102ab80 1 --2- 172.21.15.7:0/771340287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9684e1e0 0x559b96864070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.576+0000 7f4f8102ab80 1 --2- 172.21.15.7:0/771340287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b96849440 0x559b96869cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.576+0000 7f4f8102ab80 1 --2- 172.21.15.7:0/771340287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b96849d60 0x559b9686a230 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.576+0000 7f4f8102ab80 1 -- 172.21.15.7:0/771340287 >> 172.21.15.7:0/771340287 conn(0x559b96776080 msgr2=0x559b96842e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:17.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.576+0000 7f4f8102ab80 1 -- 172.21.15.7:0/771340287 shutdown_connections 2024-06-10T01:19:17.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.576+0000 7f4f8102ab80 1 -- 172.21.15.7:0/771340287 wait complete. 2024-06-10T01:19:17.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.576+0000 7f4f8102ab80 20 client.18804 trim_cache size 0 max 16384 2024-06-10T01:19:17.580 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:19:17.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.592+0000 7f4f8102ab80 1 Processor -- start 2024-06-10T01:19:17.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.592+0000 7f4f8102ab80 1 -- start start 2024-06-10T01:19:17.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.593+0000 7f4f8102ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b96849440 0x559b96844600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.593+0000 7f4f8102ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b96849d60 0x559b968699c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.593+0000 7f4f8102ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b9684e1e0 0x559b96869f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.593+0000 7f4f8102ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4f701ae970 con 0x559b9684e1e0 2024-06-10T01:19:17.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.593+0000 7f4f8102ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4f701ac200 con 0x559b96849d60 2024-06-10T01:19:17.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.593+0000 7f4f8102ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4f701aa070 con 0x559b96849440 2024-06-10T01:19:17.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.593+0000 7f4f74d44700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b9684e1e0 0x559b96869f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.593+0000 7f4f74d44700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b9684e1e0 0x559b96869f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55374/0 (socket says 172.21.15.7:55374) 2024-06-10T01:19:17.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.593+0000 7f4f74d44700 1 -- 172.21.15.7:0/3370333332 learned_addr learned my addr 172.21.15.7:0/3370333332 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:17.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.593+0000 7f4f67fff700 1 --2- 172.21.15.7:0/3370333332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b96849d60 0x559b968699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.593+0000 7f4f6ffff700 1 --2- 172.21.15.7:0/3370333332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b96849440 0x559b96844600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.593+0000 7f4f74d44700 1 -- 172.21.15.7:0/3370333332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b96849440 msgr2=0x559b96844600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:17.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.593+0000 7f4f74d44700 1 --2- 172.21.15.7:0/3370333332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b96849440 0x559b96844600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.593+0000 7f4f74d44700 1 -- 172.21.15.7:0/3370333332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b96849d60 msgr2=0x559b968699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:17.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.593+0000 7f4f74d44700 1 --2- 172.21.15.7:0/3370333332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b96849d60 0x559b968699c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.593+0000 7f4f74d44700 1 -- 172.21.15.7:0/3370333332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b96870b40 con 0x559b9684e1e0 2024-06-10T01:19:17.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.593+0000 7f4f74d44700 1 --2- 172.21.15.7:0/3370333332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b9684e1e0 0x559b96869f00 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f4f68011a00 tx=0x7f4f68009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc170612a6ed019 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:17.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.594+0000 7f4f65ffb700 1 -- 172.21.15.7:0/3370333332 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f68017690 con 0x559b9684e1e0 2024-06-10T01:19:17.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.594+0000 7f4f65ffb700 1 -- 172.21.15.7:0/3370333332 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f68017d10 con 0x559b9684e1e0 2024-06-10T01:19:17.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.594+0000 7f4f65ffb700 1 -- 172.21.15.7:0/3370333332 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6801f3e0 con 0x559b9684e1e0 2024-06-10T01:19:17.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.594+0000 7f4f8102ab80 1 -- 172.21.15.7:0/3370333332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b9684e1e0 msgr2=0x559b96869f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:17.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.594+0000 7f4f8102ab80 1 --2- 172.21.15.7:0/3370333332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b9684e1e0 0x559b96869f00 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f4f68011a00 tx=0x7f4f68009920 comp rx=0 tx=0).stop 2024-06-10T01:19:17.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.594+0000 7f4f8102ab80 1 -- 172.21.15.7:0/3370333332 shutdown_connections 2024-06-10T01:19:17.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.594+0000 7f4f8102ab80 1 --2- 172.21.15.7:0/3370333332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b9684e1e0 0x559b96869f00 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.594+0000 7f4f8102ab80 1 --2- 172.21.15.7:0/3370333332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b96849440 0x559b96844600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.594+0000 7f4f8102ab80 1 --2- 172.21.15.7:0/3370333332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b96849d60 0x559b968699c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.594+0000 7f4f8102ab80 1 -- 172.21.15.7:0/3370333332 >> 172.21.15.7:0/3370333332 conn(0x559b96776080 msgr2=0x559b96843ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:17.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.595+0000 7f4f8102ab80 1 -- 172.21.15.7:0/3370333332 shutdown_connections 2024-06-10T01:19:17.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.595+0000 7f4f8102ab80 1 -- 172.21.15.7:0/3370333332 wait complete. 2024-06-10T01:19:17.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.595+0000 7f4f8102ab80 1 -- start start 2024-06-10T01:19:17.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.595+0000 7f4f8102ab80 1 Processor -- start 2024-06-10T01:19:17.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.595+0000 7f4f8102ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b96849440 0x559b968d6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.595+0000 7f4f8102ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b96849d60 0x559b968d7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.595+0000 7f4f8102ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b9684e1e0 0x559b968d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.595+0000 7f4f8102ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4f701aa6c0 con 0x559b96849d60 2024-06-10T01:19:17.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.595+0000 7f4f8102ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4f701abb30 con 0x559b96849440 2024-06-10T01:19:17.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.595+0000 7f4f8102ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4f701aad90 con 0x559b9684e1e0 2024-06-10T01:19:17.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.595+0000 7f4f8102ab80 10 client.0 _ll_register_callbacks cb 0x559b967305b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:19:17.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.596+0000 7f4f67fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b96849d60 0x559b968d7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.596+0000 7f4f6ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b96849440 0x559b968d6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.596+0000 7f4f67fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b96849d60 0x559b968d7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55380/0 (socket says 172.21.15.7:55380) 2024-06-10T01:19:17.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.596+0000 7f4f74d44700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b9684e1e0 0x559b968d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.596+0000 7f4f67fff700 1 -- 172.21.15.7:0/398739008 learned_addr learned my addr 172.21.15.7:0/398739008 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:17.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.596+0000 7f4f67fff700 1 -- 172.21.15.7:0/398739008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b9684e1e0 msgr2=0x559b968d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:17.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.596+0000 7f4f67fff700 1 --2- 172.21.15.7:0/398739008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b9684e1e0 0x559b968d0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.596+0000 7f4f67fff700 1 -- 172.21.15.7:0/398739008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b96849440 msgr2=0x559b968d6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:17.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.596+0000 7f4f67fff700 1 --2- 172.21.15.7:0/398739008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b96849440 0x559b968d6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.596+0000 7f4f67fff700 1 -- 172.21.15.7:0/398739008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b968d1630 con 0x559b96849d60 2024-06-10T01:19:17.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.596+0000 7f4f6ffff700 1 --2- 172.21.15.7:0/398739008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b96849440 0x559b968d6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:17.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.596+0000 7f4f74d44700 1 --2- 172.21.15.7:0/398739008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b9684e1e0 0x559b968d0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:17.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.596+0000 7f4f67fff700 1 --2- 172.21.15.7:0/398739008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b96849d60 0x559b968d7050 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f4f58015360 tx=0x7f4f5809b3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc3ae60fccccdb1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:17.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.596+0000 7f4f4f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:19:17.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.596+0000 7f4f4f7fe700 1 -- 172.21.15.7:0/398739008 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f580b4070 con 0x559b96849d60 2024-06-10T01:19:17.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.596+0000 7f4f4f7fe700 1 -- 172.21.15.7:0/398739008 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f4f5809bf90 con 0x559b96849d60 2024-06-10T01:19:17.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.597+0000 7f4f8102ab80 1 -- 172.21.15.7:0/398739008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f68017d10 con 0x559b96849d60 2024-06-10T01:19:17.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.597+0000 7f4f4f7fe700 1 -- 172.21.15.7:0/398739008 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f4f58097dd0 con 0x559b96849d60 2024-06-10T01:19:17.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.597+0000 7f4f8102ab80 10 client.18810 fetch_fsmap learned FSMap version 40 2024-06-10T01:19:17.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.597+0000 7f4f8102ab80 1 -- 172.21.15.7:0/398739008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559b968d1630 con 0x559b96849d60 2024-06-10T01:19:17.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.597+0000 7f4f4f7fe700 1 -- 172.21.15.7:0/398739008 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f68017d10 con 0x559b96849d60 2024-06-10T01:19:17.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.597+0000 7f4f4f7fe700 1 -- 172.21.15.7:0/398739008 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f4f58015040 con 0x559b96849d60 2024-06-10T01:19:17.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.597+0000 7f4f8102ab80 10 client.18810 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:19:17.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.597+0000 7f4f8102ab80 10 client.18810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:19:17.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.597+0000 7f4f8102ab80 20 client.18810 populate_metadata read hostname 'smithi007' 2024-06-10T01:19:17.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.597+0000 7f4f8102ab80 1 --2- 172.21.15.7:0/398739008 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559b968de5d0 0x559b968e09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:17.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.597+0000 7f4f8102ab80 4 client.18810 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:19:17.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.597+0000 7f4f8102ab80 1 -- 172.21.15.7:0/398739008 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559b967a7d50 con 0x559b968de5d0 2024-06-10T01:19:17.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.597+0000 7f4f6ffff700 1 --2- 172.21.15.7:0/398739008 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559b968de5d0 0x559b968e09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:17.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.598+0000 7f4f6ffff700 1 --2- 172.21.15.7:0/398739008 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559b968de5d0 0x559b968e09c0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3657b0c494ce6157 server_cookie=d0c3bc5181f58ab0 in_seq=0 out_seq=0 2024-06-10T01:19:17.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.598+0000 7f4f4f7fe700 10 client.18810 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:17.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.598+0000 7f4f4f7fe700 1 -- 172.21.15.7:0/398739008 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559b967a7d50 con 0x559b968de5d0 2024-06-10T01:19:17.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.598+0000 7f4f4f7fe700 10 client.18810 handle_command_reply: tid=0 2024-06-10T01:19:17.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.598+0000 7f4f8102ab80 1 client.18810 shutdown 2024-06-10T01:19:17.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.598+0000 7f4f8102ab80 1 -- 172.21.15.7:0/398739008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b96849d60 msgr2=0x559b968d7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:17.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.598+0000 7f4f8102ab80 1 --2- 172.21.15.7:0/398739008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b96849d60 0x559b968d7050 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f4f58015360 tx=0x7f4f5809b3d0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.598+0000 7f4f8102ab80 1 -- 172.21.15.7:0/398739008 shutdown_connections 2024-06-10T01:19:17.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.598+0000 7f4f8102ab80 1 --2- 172.21.15.7:0/398739008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b9684e1e0 0x559b968d0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.598+0000 7f4f8102ab80 1 --2- 172.21.15.7:0/398739008 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559b968de5d0 0x559b968e09c0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.598+0000 7f4f4f7fe700 0 client.18810 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:17.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.598+0000 7f4f8102ab80 1 --2- 172.21.15.7:0/398739008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b96849440 0x559b968d6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.598+0000 7f4f8102ab80 1 --2- 172.21.15.7:0/398739008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b96849d60 0x559b968d7050 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.598+0000 7f4f8102ab80 1 -- 172.21.15.7:0/398739008 >> 172.21.15.7:0/398739008 conn(0x559b96776080 msgr2=0x559b968698c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:17.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.598+0000 7f4f8102ab80 1 -- 172.21.15.7:0/398739008 shutdown_connections 2024-06-10T01:19:17.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.598+0000 7f4f8102ab80 1 -- 172.21.15.7:0/398739008 wait complete. 2024-06-10T01:19:17.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.598+0000 7f4f8102ab80 20 client.18810 trim_cache size 0 max 16384 2024-06-10T01:19:17.598 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:19:17.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.599+0000 7f4f767a7700 1 -- 172.21.15.7:0/1425213136 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4f5c0a2ca0 msgr2=0x7f4f5c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:17.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.599+0000 7f4f767a7700 1 --2- 172.21.15.7:0/1425213136 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4f5c0a2ca0 0x7f4f5c0a5160 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4f6000b2e0 tx=0x7f4f60005960 comp rx=0 tx=0).stop 2024-06-10T01:19:17.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.599+0000 7f4f767a7700 1 -- 172.21.15.7:0/1425213136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f70101510 msgr2=0x7f4f7019ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:17.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.599+0000 7f4f767a7700 1 --2- 172.21.15.7:0/1425213136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f70101510 0x7f4f7019ed60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4f5800eab0 tx=0x7f4f5800edc0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.599+0000 7f4f767a7700 1 -- 172.21.15.7:0/1425213136 shutdown_connections 2024-06-10T01:19:17.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.599+0000 7f4f767a7700 1 --2- 172.21.15.7:0/1425213136 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4f5c0a2ca0 0x7f4f5c0a5160 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.599+0000 7f4f767a7700 1 --2- 172.21.15.7:0/1425213136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f70106b90 0x7f4f701989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.599+0000 7f4f767a7700 1 --2- 172.21.15.7:0/1425213136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f70100b50 0x7f4f7019e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.599+0000 7f4f767a7700 1 --2- 172.21.15.7:0/1425213136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f70101510 0x7f4f7019ed60 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:17.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.599+0000 7f4f767a7700 1 -- 172.21.15.7:0/1425213136 >> 172.21.15.7:0/1425213136 conn(0x7f4f70077a00 msgr2=0x7f4f70103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:17.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.599+0000 7f4f767a7700 1 -- 172.21.15.7:0/1425213136 shutdown_connections 2024-06-10T01:19:17.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:17.599+0000 7f4f767a7700 1 -- 172.21.15.7:0/1425213136 wait complete. 2024-06-10T01:19:17.758 INFO:tasks.workunit.client.0.smithi007.stdout: 1 981607 7.33 MB/sec execute 598 sec latency 2662.736 ms 2024-06-10T01:19:17.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:17 smithi078 ceph-mon[36683]: pgmap v830: 129 pgs: 129 active+clean; 170 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 3.7 MiB/s wr, 286 op/s 2024-06-10T01:19:17.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:17 smithi175 ceph-mon[36636]: pgmap v830: 129 pgs: 129 active+clean; 170 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 3.7 MiB/s wr, 286 op/s 2024-06-10T01:19:18.758 INFO:tasks.workunit.client.0.smithi007.stdout: 1 981607 7.32 MB/sec execute 599 sec latency 3662.784 ms 2024-06-10T01:19:19.758 INFO:tasks.workunit.client.0.smithi007.stdout: 1 cleanup 600 sec 2024-06-10T01:19:19.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:19 smithi007 ceph-mon[32398]: pgmap v831: 129 pgs: 129 active+clean; 173 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 3.7 MiB/s wr, 256 op/s 2024-06-10T01:19:19.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:19 smithi078 ceph-mon[36683]: pgmap v831: 129 pgs: 129 active+clean; 173 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 3.7 MiB/s wr, 256 op/s 2024-06-10T01:19:19.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:19 smithi175 ceph-mon[36636]: pgmap v831: 129 pgs: 129 active+clean; 173 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 3.7 MiB/s wr, 256 op/s 2024-06-10T01:19:19.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.983+0000 7f826278a700 1 -- 172.21.15.7:0/3430921473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f825c101440 msgr2=0x7f825c10f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:19.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.983+0000 7f826278a700 1 --2- 172.21.15.7:0/3430921473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f825c101440 0x7f825c10f330 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f825400b0a0 tx=0x7f825400b3b0 comp rx=0 tx=0).stop 2024-06-10T01:19:19.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.983+0000 7f826278a700 1 -- 172.21.15.7:0/3430921473 shutdown_connections 2024-06-10T01:19:19.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.983+0000 7f826278a700 1 --2- 172.21.15.7:0/3430921473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f825c101440 0x7f825c10f330 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:19.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.983+0000 7f826278a700 1 --2- 172.21.15.7:0/3430921473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f825c108510 0x7f825c1088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:19.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.983+0000 7f826278a700 1 --2- 172.21.15.7:0/3430921473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f825c100a80 0x7f825c100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:19.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.983+0000 7f826278a700 1 -- 172.21.15.7:0/3430921473 >> 172.21.15.7:0/3430921473 conn(0x7f825c077a00 msgr2=0x7f825c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:19.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.983+0000 7f826278a700 1 -- 172.21.15.7:0/3430921473 shutdown_connections 2024-06-10T01:19:19.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.983+0000 7f826278a700 1 -- 172.21.15.7:0/3430921473 wait complete. 2024-06-10T01:19:19.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.984+0000 7f826278a700 1 Processor -- start 2024-06-10T01:19:19.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.984+0000 7f826278a700 1 -- start start 2024-06-10T01:19:19.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.984+0000 7f826278a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f825c100a80 0x7f825c101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:19.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.984+0000 7f826278a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f825c101440 0x7f825c1021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:19.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.984+0000 7f826278a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f825c108510 0x7f825c1026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:19.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.984+0000 7f826278a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f825c112770 con 0x7f825c108510 2024-06-10T01:19:19.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.984+0000 7f826278a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f825c1125f0 con 0x7f825c101440 2024-06-10T01:19:19.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.984+0000 7f826278a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f825c1128f0 con 0x7f825c100a80 2024-06-10T01:19:19.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.984+0000 7f8260d27700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f825c108510 0x7f825c1026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:19.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.984+0000 7f8260d27700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f825c108510 0x7f825c1026e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55392/0 (socket says 172.21.15.7:55392) 2024-06-10T01:19:19.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.984+0000 7f8260d27700 1 -- 172.21.15.7:0/20619509 learned_addr learned my addr 172.21.15.7:0/20619509 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:19.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.984+0000 7f825bfff700 1 --2- 172.21.15.7:0/20619509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f825c100a80 0x7f825c101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:19.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.985+0000 7f8253fff700 1 --2- 172.21.15.7:0/20619509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f825c101440 0x7f825c1021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:19.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.985+0000 7f8260d27700 1 -- 172.21.15.7:0/20619509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f825c100a80 msgr2=0x7f825c101c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:19.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.985+0000 7f8260d27700 1 --2- 172.21.15.7:0/20619509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f825c100a80 0x7f825c101c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:19.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.985+0000 7f8260d27700 1 -- 172.21.15.7:0/20619509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f825c101440 msgr2=0x7f825c1021a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:19.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.985+0000 7f8260d27700 1 --2- 172.21.15.7:0/20619509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f825c101440 0x7f825c1021a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:19.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.985+0000 7f8260d27700 1 -- 172.21.15.7:0/20619509 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825c1a7030 con 0x7f825c108510 2024-06-10T01:19:19.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.985+0000 7f8260d27700 1 --2- 172.21.15.7:0/20619509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f825c108510 0x7f825c1026e0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f82540062a0 tx=0x7f8254004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:19.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.985+0000 7f8259ffb700 1 -- 172.21.15.7:0/20619509 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8254021070 con 0x7f825c108510 2024-06-10T01:19:19.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.985+0000 7f826278a700 1 -- 172.21.15.7:0/20619509 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f825c1a72c0 con 0x7f825c108510 2024-06-10T01:19:19.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.985+0000 7f8259ffb700 1 -- 172.21.15.7:0/20619509 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8254007c00 con 0x7f825c108510 2024-06-10T01:19:19.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.985+0000 7f8259ffb700 1 -- 172.21.15.7:0/20619509 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825401d450 con 0x7f825c108510 2024-06-10T01:19:19.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.985+0000 7f826278a700 1 -- 172.21.15.7:0/20619509 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f825c1a7770 con 0x7f825c108510 2024-06-10T01:19:19.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.986+0000 7f826278a700 1 -- 172.21.15.7:0/20619509 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f825c069180 con 0x7f825c108510 2024-06-10T01:19:19.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.988+0000 7f8259ffb700 1 -- 172.21.15.7:0/20619509 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f8254004000 con 0x7f825c108510 2024-06-10T01:19:19.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.988+0000 7f8259ffb700 1 --2- 172.21.15.7:0/20619509 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f82480a2c20 0x7f82480a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:19.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.988+0000 7f825bfff700 1 --2- 172.21.15.7:0/20619509 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f82480a2c20 0x7f82480a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:19.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.988+0000 7f8259ffb700 1 -- 172.21.15.7:0/20619509 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f825400f020 con 0x7f825c108510 2024-06-10T01:19:19.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.989+0000 7f825bfff700 1 --2- 172.21.15.7:0/20619509 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f82480a2c20 0x7f82480a50e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f824c005950 tx=0x7f824c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:19.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:19.992+0000 7f8259ffb700 1 -- 172.21.15.7:0/20619509 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f825402a020 con 0x7f825c108510 2024-06-10T01:19:20.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:20.253+0000 7f826278a700 1 -- 172.21.15.7:0/20619509 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f825c02de60 con 0x7f825c108510 2024-06-10T01:19:20.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:20.254+0000 7f8259ffb700 1 -- 172.21.15.7:0/20619509 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11855 (secure 0 0 0) 0x7f825409d560 con 0x7f825c108510 2024-06-10T01:19:20.247 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:19:20.248 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:16:56.409130+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24451,24434,24436]},"id":1}]} 2024-06-10T01:19:20.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:20.256+0000 7f826278a700 1 -- 172.21.15.7:0/20619509 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f82480a2c20 msgr2=0x7f82480a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:20.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:20.256+0000 7f826278a700 1 --2- 172.21.15.7:0/20619509 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f82480a2c20 0x7f82480a50e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f824c005950 tx=0x7f824c00b410 comp rx=0 tx=0).stop 2024-06-10T01:19:20.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:20.256+0000 7f826278a700 1 -- 172.21.15.7:0/20619509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f825c108510 msgr2=0x7f825c1026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:20.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:20.256+0000 7f826278a700 1 --2- 172.21.15.7:0/20619509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f825c108510 0x7f825c1026e0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f82540062a0 tx=0x7f8254004d70 comp rx=0 tx=0).stop 2024-06-10T01:19:20.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:20.256+0000 7f826278a700 1 -- 172.21.15.7:0/20619509 shutdown_connections 2024-06-10T01:19:20.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:20.256+0000 7f826278a700 1 --2- 172.21.15.7:0/20619509 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f82480a2c20 0x7f82480a50e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:20.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:20.256+0000 7f826278a700 1 --2- 172.21.15.7:0/20619509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f825c108510 0x7f825c1026e0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:20.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:20.256+0000 7f826278a700 1 --2- 172.21.15.7:0/20619509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f825c100a80 0x7f825c101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:20.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:20.256+0000 7f826278a700 1 --2- 172.21.15.7:0/20619509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f825c101440 0x7f825c1021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:20.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:20.256+0000 7f826278a700 1 -- 172.21.15.7:0/20619509 >> 172.21.15.7:0/20619509 conn(0x7f825c077a00 msgr2=0x7f825c0fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:20.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:20.256+0000 7f826278a700 1 -- 172.21.15.7:0/20619509 shutdown_connections 2024-06-10T01:19:20.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:20.256+0000 7f826278a700 1 -- 172.21.15.7:0/20619509 wait complete. 2024-06-10T01:19:20.251 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 40 2024-06-10T01:19:20.755 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:20 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/20619509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:19:20.758 INFO:tasks.workunit.client.0.smithi007.stdout: 1 cleanup 601 sec 2024-06-10T01:19:20.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:20 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/20619509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:19:20.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:20 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/20619509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:19:21.043 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db --set-id b3de2d3a --release --await' 2024-06-10T01:19:21.043 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db --set-id b3de2d3a --release --await 2024-06-10T01:19:21.295 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:19:21.419 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:19:21.419 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:19:21.676 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:21 smithi007 ceph-mon[32398]: pgmap v832: 129 pgs: 129 active+clean; 170 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 4.0 MiB/s wr, 254 op/s 2024-06-10T01:19:21.758 INFO:tasks.workunit.client.0.smithi007.stdout: 1 cleanup 602 sec 2024-06-10T01:19:21.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:21 smithi078 ceph-mon[36683]: pgmap v832: 129 pgs: 129 active+clean; 170 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 4.0 MiB/s wr, 254 op/s 2024-06-10T01:19:21.852 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:21 smithi175 ceph-mon[36636]: pgmap v832: 129 pgs: 129 active+clean; 170 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 4.0 MiB/s wr, 254 op/s 2024-06-10T01:19:22.454 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:19:22.758 INFO:tasks.workunit.client.0.smithi007.stdout: 1 cleanup 603 sec 2024-06-10T01:19:23.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.155+0000 7fa40e928700 1 -- 172.21.15.7:0/1711436627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa408100a80 msgr2=0x7fa408100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:23.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.155+0000 7fa40e928700 1 --2- 172.21.15.7:0/1711436627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa408100a80 0x7fa408100f00 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0009990 tx=0x7fa3f0009ca0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.155+0000 7fa40e928700 1 -- 172.21.15.7:0/1711436627 shutdown_connections 2024-06-10T01:19:23.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.155+0000 7fa40e928700 1 --2- 172.21.15.7:0/1711436627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa408101440 0x7fa40810f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.155+0000 7fa40e928700 1 --2- 172.21.15.7:0/1711436627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa408108510 0x7fa4081088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.155+0000 7fa40e928700 1 --2- 172.21.15.7:0/1711436627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa408100a80 0x7fa408100f00 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.155+0000 7fa40e928700 1 -- 172.21.15.7:0/1711436627 >> 172.21.15.7:0/1711436627 conn(0x7fa408077a00 msgr2=0x7fa4080fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:23.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.155+0000 7fa40e928700 1 -- 172.21.15.7:0/1711436627 shutdown_connections 2024-06-10T01:19:23.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.155+0000 7fa40e928700 1 -- 172.21.15.7:0/1711436627 wait complete. 2024-06-10T01:19:23.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.156+0000 7fa40e928700 1 Processor -- start 2024-06-10T01:19:23.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.156+0000 7fa40e928700 1 -- start start 2024-06-10T01:19:23.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.156+0000 7fa40e928700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa408100a80 0x7fa40819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:23.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.156+0000 7fa40e928700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa408101440 0x7fa40819ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:23.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.156+0000 7fa40e928700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa408108510 0x7fa408198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:23.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.156+0000 7fa40e928700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa4081127f0 con 0x7fa408100a80 2024-06-10T01:19:23.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.156+0000 7fa40e928700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa408112670 con 0x7fa408108510 2024-06-10T01:19:23.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.156+0000 7fa40e928700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa408112970 con 0x7fa408101440 2024-06-10T01:19:23.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.157+0000 7fa407fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa408100a80 0x7fa40819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:23.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.157+0000 7fa407fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa408100a80 0x7fa40819e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39120/0 (socket says 172.21.15.7:39120) 2024-06-10T01:19:23.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.157+0000 7fa407fff700 1 -- 172.21.15.7:0/869006498 learned_addr learned my addr 172.21.15.7:0/869006498 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:23.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.157+0000 7fa3fffff700 1 --2- 172.21.15.7:0/869006498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa408101440 0x7fa40819ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:23.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.157+0000 7fa40cec5700 1 --2- 172.21.15.7:0/869006498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa408108510 0x7fa408198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:23.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.157+0000 7fa407fff700 1 -- 172.21.15.7:0/869006498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa408101440 msgr2=0x7fa40819ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:23.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.157+0000 7fa407fff700 1 --2- 172.21.15.7:0/869006498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa408101440 0x7fa40819ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.157+0000 7fa407fff700 1 -- 172.21.15.7:0/869006498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa408108510 msgr2=0x7fa408198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:23.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.157+0000 7fa407fff700 1 --2- 172.21.15.7:0/869006498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa408108510 0x7fa408198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.157+0000 7fa407fff700 1 -- 172.21.15.7:0/869006498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa408199250 con 0x7fa408100a80 2024-06-10T01:19:23.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.157+0000 7fa40cec5700 1 --2- 172.21.15.7:0/869006498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa408108510 0x7fa408198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:19:23.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.157+0000 7fa407fff700 1 --2- 172.21.15.7:0/869006498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa408100a80 0x7fa40819e930 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800b530 tx=0x7fa3f800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:23.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.157+0000 7fa405ffb700 1 -- 172.21.15.7:0/869006498 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8015020 con 0x7fa408100a80 2024-06-10T01:19:23.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.157+0000 7fa40e928700 1 -- 172.21.15.7:0/869006498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa408199540 con 0x7fa408100a80 2024-06-10T01:19:23.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.157+0000 7fa40e928700 1 -- 172.21.15.7:0/869006498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4081a5800 con 0x7fa408100a80 2024-06-10T01:19:23.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.158+0000 7fa405ffb700 1 -- 172.21.15.7:0/869006498 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3f8011d10 con 0x7fa408100a80 2024-06-10T01:19:23.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.158+0000 7fa405ffb700 1 -- 172.21.15.7:0/869006498 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f800f2e0 con 0x7fa408100a80 2024-06-10T01:19:23.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.160+0000 7fa405ffb700 1 -- 172.21.15.7:0/869006498 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa3f8018020 con 0x7fa408100a80 2024-06-10T01:19:23.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.161+0000 7fa405ffb700 1 --2- 172.21.15.7:0/869006498 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa3f40a2ca0 0x7fa3f40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:23.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.161+0000 7fa3fffff700 1 --2- 172.21.15.7:0/869006498 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa3f40a2ca0 0x7fa3f40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:23.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.161+0000 7fa405ffb700 1 -- 172.21.15.7:0/869006498 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa3f80cec60 con 0x7fa408100a80 2024-06-10T01:19:23.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.161+0000 7fa3fffff700 1 --2- 172.21.15.7:0/869006498 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa3f40a2ca0 0x7fa3f40a5160 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000b5c0 tx=0x7fa3f0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:23.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.161+0000 7fa4191abb80 1 Processor -- start 2024-06-10T01:19:23.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.161+0000 7fa4191abb80 1 -- start start 2024-06-10T01:19:23.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.162+0000 7fa4191abb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55736106e500 0x55736106e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:23.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.162+0000 7fa4191abb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55736106ee20 0x557361068710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:23.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.162+0000 7fa4191abb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557361068c50 0x557361069070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:23.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.162+0000 7fa4191abb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557360f5b060 con 0x55736106ee20 2024-06-10T01:19:23.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.162+0000 7fa4191abb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557360fb9950 con 0x557361068c50 2024-06-10T01:19:23.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.162+0000 7fa4191abb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557360f21560 con 0x55736106e500 2024-06-10T01:19:23.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.162+0000 7fa3fffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55736106ee20 0x557361068710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:23.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.162+0000 7fa3fffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55736106ee20 0x557361068710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39122/0 (socket says 172.21.15.7:39122) 2024-06-10T01:19:23.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.162+0000 7fa3fffff700 1 -- 172.21.15.7:0/2831906587 learned_addr learned my addr 172.21.15.7:0/2831906587 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:23.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.162+0000 7fa407fff700 1 --2- 172.21.15.7:0/2831906587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55736106e500 0x55736106e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:23.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.162+0000 7fa40cec5700 1 --2- 172.21.15.7:0/2831906587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557361068c50 0x557361069070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:23.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.162+0000 7fa3fffff700 1 -- 172.21.15.7:0/2831906587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55736106e500 msgr2=0x55736106e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:23.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.162+0000 7fa3fffff700 1 --2- 172.21.15.7:0/2831906587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55736106e500 0x55736106e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.162+0000 7fa3fffff700 1 -- 172.21.15.7:0/2831906587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557361068c50 msgr2=0x557361069070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:23.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.162+0000 7fa3fffff700 1 --2- 172.21.15.7:0/2831906587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557361068c50 0x557361069070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.162+0000 7fa3fffff700 1 -- 172.21.15.7:0/2831906587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557360f089d0 con 0x55736106ee20 2024-06-10T01:19:23.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.162+0000 7fa40cec5700 1 --2- 172.21.15.7:0/2831906587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557361068c50 0x557361069070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:19:23.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.162+0000 7fa3fffff700 1 --2- 172.21.15.7:0/2831906587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55736106ee20 0x557361068710 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0015000 tx=0x7fa3f0020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e63032bd34c568 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:23.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.163+0000 7fa3fe7fc700 1 -- 172.21.15.7:0/2831906587 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f000bda0 con 0x55736106ee20 2024-06-10T01:19:23.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.163+0000 7fa3fe7fc700 1 -- 172.21.15.7:0/2831906587 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3f000fd50 con 0x55736106ee20 2024-06-10T01:19:23.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.163+0000 7fa3fe7fc700 1 -- 172.21.15.7:0/2831906587 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f002f400 con 0x55736106ee20 2024-06-10T01:19:23.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.163+0000 7fa4191abb80 1 -- 172.21.15.7:0/2831906587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55736106ee20 msgr2=0x557361068710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:23.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.163+0000 7fa4191abb80 1 --2- 172.21.15.7:0/2831906587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55736106ee20 0x557361068710 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0015000 tx=0x7fa3f0020000 comp rx=0 tx=0).stop 2024-06-10T01:19:23.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.164+0000 7fa4191abb80 1 -- 172.21.15.7:0/2831906587 shutdown_connections 2024-06-10T01:19:23.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.164+0000 7fa4191abb80 1 --2- 172.21.15.7:0/2831906587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557361068c50 0x557361069070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.164+0000 7fa4191abb80 1 --2- 172.21.15.7:0/2831906587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55736106e500 0x55736106e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.164+0000 7fa4191abb80 1 --2- 172.21.15.7:0/2831906587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55736106ee20 0x557361068710 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.164+0000 7fa4191abb80 1 -- 172.21.15.7:0/2831906587 >> 172.21.15.7:0/2831906587 conn(0x557360f95180 msgr2=0x557361063a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:23.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.164+0000 7fa4191abb80 1 -- 172.21.15.7:0/2831906587 shutdown_connections 2024-06-10T01:19:23.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.164+0000 7fa4191abb80 1 -- 172.21.15.7:0/2831906587 wait complete. 2024-06-10T01:19:23.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.164+0000 7fa4191abb80 1 -- start start 2024-06-10T01:19:23.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.164+0000 7fa4191abb80 1 Processor -- start 2024-06-10T01:19:23.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.165+0000 7fa4191abb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557361068c50 0x557361074b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:23.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.165+0000 7fa4191abb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55736106e500 0x557361071e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:23.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.165+0000 7fa4191abb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55736106ee20 0x5573610723d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:23.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.165+0000 7fa4191abb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557360f5a0c0 con 0x557361068c50 2024-06-10T01:19:23.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.165+0000 7fa4191abb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557360fb5910 con 0x55736106ee20 2024-06-10T01:19:23.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.165+0000 7fa4191abb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557361075740 con 0x55736106e500 2024-06-10T01:19:23.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.165+0000 7fa4191abb80 10 client.0 _ll_register_callbacks cb 0x557360f4f6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:19:23.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.165+0000 7fa407fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557361068c50 0x557361074b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:23.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.165+0000 7fa407fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557361068c50 0x557361074b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39134/0 (socket says 172.21.15.7:39134) 2024-06-10T01:19:23.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.165+0000 7fa407fff700 1 -- 172.21.15.7:0/1781457805 learned_addr learned my addr 172.21.15.7:0/1781457805 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:23.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.165+0000 7fa3fffff700 1 --2- 172.21.15.7:0/1781457805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55736106e500 0x557361071e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:23.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.165+0000 7fa40cec5700 1 --2- 172.21.15.7:0/1781457805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55736106ee20 0x5573610723d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:23.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.165+0000 7fa407fff700 1 -- 172.21.15.7:0/1781457805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55736106e500 msgr2=0x557361071e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:23.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.165+0000 7fa407fff700 1 --2- 172.21.15.7:0/1781457805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55736106e500 0x557361071e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.165+0000 7fa407fff700 1 -- 172.21.15.7:0/1781457805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55736106ee20 msgr2=0x5573610723d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:23.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.165+0000 7fa407fff700 1 --2- 172.21.15.7:0/1781457805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55736106ee20 0x5573610723d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.165+0000 7fa407fff700 1 -- 172.21.15.7:0/1781457805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557360f7f850 con 0x557361068c50 2024-06-10T01:19:23.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.165+0000 7fa407fff700 1 --2- 172.21.15.7:0/1781457805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557361068c50 0x557361074b90 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fa3f809ade0 tx=0x7fa3f809b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2073e2062e290c52 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:23.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.165+0000 7fa3e77fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:19:23.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.165+0000 7fa3e77fe700 1 -- 172.21.15.7:0/1781457805 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f80a3d50 con 0x557361068c50 2024-06-10T01:19:23.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.166+0000 7fa3e77fe700 1 -- 172.21.15.7:0/1781457805 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa3f809f170 con 0x557361068c50 2024-06-10T01:19:23.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.166+0000 7fa4191abb80 1 -- 172.21.15.7:0/1781457805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3f000fd50 con 0x557361068c50 2024-06-10T01:19:23.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.166+0000 7fa3e77fe700 1 -- 172.21.15.7:0/1781457805 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa3f8010610 con 0x557361068c50 2024-06-10T01:19:23.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.166+0000 7fa4191abb80 10 client.18828 fetch_fsmap learned FSMap version 40 2024-06-10T01:19:23.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.166+0000 7fa4191abb80 1 -- 172.21.15.7:0/1781457805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55736107ff10 con 0x557361068c50 2024-06-10T01:19:23.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.166+0000 7fa3e77fe700 1 -- 172.21.15.7:0/1781457805 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3f000fd50 con 0x557361068c50 2024-06-10T01:19:23.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.166+0000 7fa3e77fe700 1 -- 172.21.15.7:0/1781457805 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7fa3f809b340 con 0x557361068c50 2024-06-10T01:19:23.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.166+0000 7fa4191abb80 10 client.18828 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:19:23.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.166+0000 7fa4191abb80 10 client.18828 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:19:23.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.166+0000 7fa4191abb80 20 client.18828 populate_metadata read hostname 'smithi007' 2024-06-10T01:19:23.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.166+0000 7fa4191abb80 1 --2- 172.21.15.7:0/1781457805 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5573610824a0 0x557361084890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:23.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.166+0000 7fa4191abb80 4 client.18828 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:19:23.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.166+0000 7fa4191abb80 1 -- 172.21.15.7:0/1781457805 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557360fc6e50 con 0x5573610824a0 2024-06-10T01:19:23.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.167+0000 7fa3fffff700 1 --2- 172.21.15.7:0/1781457805 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5573610824a0 0x557361084890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:23.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.167+0000 7fa3fffff700 1 --2- 172.21.15.7:0/1781457805 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5573610824a0 0x557361084890 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=968f5731df10306f server_cookie=6808a2517d905304 in_seq=0 out_seq=0 2024-06-10T01:19:23.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.167+0000 7fa3e77fe700 10 client.18828 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:23.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.168+0000 7fa3e77fe700 1 -- 172.21.15.7:0/1781457805 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557360fc6e50 con 0x5573610824a0 2024-06-10T01:19:23.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.168+0000 7fa3e77fe700 10 client.18828 handle_command_reply: tid=0 2024-06-10T01:19:23.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.168+0000 7fa4191abb80 1 client.18828 shutdown 2024-06-10T01:19:23.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.168+0000 7fa4191abb80 1 -- 172.21.15.7:0/1781457805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557361068c50 msgr2=0x557361074b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:23.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.168+0000 7fa4191abb80 1 --2- 172.21.15.7:0/1781457805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557361068c50 0x557361074b90 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fa3f809ade0 tx=0x7fa3f809b8b0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.168+0000 7fa4191abb80 1 -- 172.21.15.7:0/1781457805 shutdown_connections 2024-06-10T01:19:23.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.168+0000 7fa4191abb80 1 --2- 172.21.15.7:0/1781457805 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5573610824a0 0x557361084890 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.168+0000 7fa4191abb80 1 --2- 172.21.15.7:0/1781457805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55736106ee20 0x5573610723d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.168+0000 7fa4191abb80 1 --2- 172.21.15.7:0/1781457805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557361068c50 0x557361074b90 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.168+0000 7fa4191abb80 1 --2- 172.21.15.7:0/1781457805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55736106e500 0x557361071e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.168+0000 7fa4191abb80 1 -- 172.21.15.7:0/1781457805 >> 172.21.15.7:0/1781457805 conn(0x557360f95180 msgr2=0x557361062bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:23.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.168+0000 7fa3e77fe700 0 client.18828 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:23.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.168+0000 7fa4191abb80 1 -- 172.21.15.7:0/1781457805 shutdown_connections 2024-06-10T01:19:23.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.168+0000 7fa4191abb80 1 -- 172.21.15.7:0/1781457805 wait complete. 2024-06-10T01:19:23.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.168+0000 7fa4191abb80 20 client.18828 trim_cache size 0 max 16384 2024-06-10T01:19:23.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.189+0000 7fa4191abb80 1 Processor -- start 2024-06-10T01:19:23.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.189+0000 7fa4191abb80 1 -- start start 2024-06-10T01:19:23.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.189+0000 7fa4191abb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557361068c50 0x5573610746d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:23.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.189+0000 7fa4191abb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55736106e500 0x557361071fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:23.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.189+0000 7fa4191abb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55736106ee20 0x557361063330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:23.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.189+0000 7fa4191abb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa4081b0ac0 con 0x557361068c50 2024-06-10T01:19:23.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.189+0000 7fa4191abb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa4081ae350 con 0x55736106ee20 2024-06-10T01:19:23.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.189+0000 7fa4191abb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa4081ac1a0 con 0x55736106e500 2024-06-10T01:19:23.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.189+0000 7fa407fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557361068c50 0x5573610746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:23.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.189+0000 7fa407fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557361068c50 0x5573610746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39140/0 (socket says 172.21.15.7:39140) 2024-06-10T01:19:23.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.189+0000 7fa407fff700 1 -- 172.21.15.7:0/2984576734 learned_addr learned my addr 172.21.15.7:0/2984576734 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:23.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.189+0000 7fa3fffff700 1 --2- 172.21.15.7:0/2984576734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55736106e500 0x557361071fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:23.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.189+0000 7fa40cec5700 1 --2- 172.21.15.7:0/2984576734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55736106ee20 0x557361063330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:23.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.189+0000 7fa407fff700 1 -- 172.21.15.7:0/2984576734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55736106e500 msgr2=0x557361071fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:23.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.189+0000 7fa407fff700 1 --2- 172.21.15.7:0/2984576734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55736106e500 0x557361071fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.189+0000 7fa407fff700 1 -- 172.21.15.7:0/2984576734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55736106ee20 msgr2=0x557361063330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:23.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.189+0000 7fa407fff700 1 --2- 172.21.15.7:0/2984576734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55736106ee20 0x557361063330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.189+0000 7fa407fff700 1 -- 172.21.15.7:0/2984576734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55736105b4a0 con 0x557361068c50 2024-06-10T01:19:23.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.190+0000 7fa407fff700 1 --2- 172.21.15.7:0/2984576734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557361068c50 0x5573610746d0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8017660 tx=0x7fa3f8011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3469cccd4b3f5d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:23.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.190+0000 7fa3fdffb700 1 -- 172.21.15.7:0/2984576734 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f80a3d50 con 0x557361068c50 2024-06-10T01:19:23.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.190+0000 7fa3fdffb700 1 -- 172.21.15.7:0/2984576734 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3f80b4070 con 0x557361068c50 2024-06-10T01:19:23.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.190+0000 7fa3fdffb700 1 -- 172.21.15.7:0/2984576734 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f80172c0 con 0x557361068c50 2024-06-10T01:19:23.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.190+0000 7fa4191abb80 1 -- 172.21.15.7:0/2984576734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557361068c50 msgr2=0x5573610746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:23.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.190+0000 7fa4191abb80 1 --2- 172.21.15.7:0/2984576734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557361068c50 0x5573610746d0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8017660 tx=0x7fa3f8011f70 comp rx=0 tx=0).stop 2024-06-10T01:19:23.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.190+0000 7fa4191abb80 1 -- 172.21.15.7:0/2984576734 shutdown_connections 2024-06-10T01:19:23.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.190+0000 7fa4191abb80 1 --2- 172.21.15.7:0/2984576734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55736106ee20 0x557361063330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.190+0000 7fa4191abb80 1 --2- 172.21.15.7:0/2984576734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557361068c50 0x5573610746d0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.190+0000 7fa4191abb80 1 --2- 172.21.15.7:0/2984576734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55736106e500 0x557361071fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.190+0000 7fa4191abb80 1 -- 172.21.15.7:0/2984576734 >> 172.21.15.7:0/2984576734 conn(0x557360f95180 msgr2=0x55736106d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:23.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.190+0000 7fa4191abb80 1 -- 172.21.15.7:0/2984576734 shutdown_connections 2024-06-10T01:19:23.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.190+0000 7fa4191abb80 1 -- 172.21.15.7:0/2984576734 wait complete. 2024-06-10T01:19:23.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.191+0000 7fa4191abb80 1 -- start start 2024-06-10T01:19:23.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.191+0000 7fa4191abb80 1 Processor -- start 2024-06-10T01:19:23.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.191+0000 7fa4191abb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557361068c50 0x55736107ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:23.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.191+0000 7fa4191abb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55736106e500 0x55736107b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.191+0000 7fa4191abb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55736106ee20 0x557361077ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.191+0000 7fa4191abb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa4081ac810 con 0x55736106ee20 2024-06-10T01:19:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.191+0000 7fa4191abb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa4081adc80 con 0x55736106e500 2024-06-10T01:19:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.191+0000 7fa4191abb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa4081acee0 con 0x557361068c50 2024-06-10T01:19:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.191+0000 7fa4191abb80 10 client.0 _ll_register_callbacks cb 0x557360f4f6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:19:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.191+0000 7fa40cec5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55736106ee20 0x557361077ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.191+0000 7fa40cec5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55736106ee20 0x557361077ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39154/0 (socket says 172.21.15.7:39154) 2024-06-10T01:19:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.191+0000 7fa40cec5700 1 -- 172.21.15.7:0/378468334 learned_addr learned my addr 172.21.15.7:0/378468334 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.192+0000 7fa407fff700 1 --2- 172.21.15.7:0/378468334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557361068c50 0x55736107ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.192+0000 7fa3fffff700 1 --2- 172.21.15.7:0/378468334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55736106e500 0x55736107b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.192+0000 7fa40cec5700 1 -- 172.21.15.7:0/378468334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557361068c50 msgr2=0x55736107ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.192+0000 7fa40cec5700 1 --2- 172.21.15.7:0/378468334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557361068c50 0x55736107ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.192+0000 7fa40cec5700 1 -- 172.21.15.7:0/378468334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55736106e500 msgr2=0x55736107b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.192+0000 7fa40cec5700 1 --2- 172.21.15.7:0/378468334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55736106e500 0x55736107b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.192+0000 7fa40cec5700 1 -- 172.21.15.7:0/378468334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557361078290 con 0x55736106ee20 2024-06-10T01:19:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.192+0000 7fa3fffff700 1 --2- 172.21.15.7:0/378468334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55736106e500 0x55736107b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:19:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.192+0000 7fa40cec5700 1 --2- 172.21.15.7:0/378468334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55736106ee20 0x557361077ac0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fa400008f50 tx=0x7fa400014530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b45b42a6d582100 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.192+0000 7fa3e77fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:19:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.192+0000 7fa3e77fe700 1 -- 172.21.15.7:0/378468334 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40001c070 con 0x55736106ee20 2024-06-10T01:19:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.192+0000 7fa3e77fe700 1 -- 172.21.15.7:0/378468334 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa400014cd0 con 0x55736106ee20 2024-06-10T01:19:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.192+0000 7fa4191abb80 1 -- 172.21.15.7:0/378468334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3f80b4070 con 0x55736106ee20 2024-06-10T01:19:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.192+0000 7fa3e77fe700 1 -- 172.21.15.7:0/378468334 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa400028050 con 0x55736106ee20 2024-06-10T01:19:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.192+0000 7fa4191abb80 10 client.18834 fetch_fsmap learned FSMap version 40 2024-06-10T01:19:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.192+0000 7fa4191abb80 1 -- 172.21.15.7:0/378468334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5573610eddf0 con 0x55736106ee20 2024-06-10T01:19:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.193+0000 7fa3e77fe700 1 -- 172.21.15.7:0/378468334 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3f80b4070 con 0x55736106ee20 2024-06-10T01:19:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.193+0000 7fa3e77fe700 1 -- 172.21.15.7:0/378468334 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7fa400017560 con 0x55736106ee20 2024-06-10T01:19:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.193+0000 7fa4191abb80 10 client.18834 fetch_fsmap finished waiting for FSMap version 40 2024-06-10T01:19:23.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.193+0000 7fa4191abb80 10 client.18834 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:19:23.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.193+0000 7fa4191abb80 20 client.18834 populate_metadata read hostname 'smithi007' 2024-06-10T01:19:23.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.193+0000 7fa4191abb80 1 --2- 172.21.15.7:0/378468334 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5573610ee530 0x5573610f09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:23.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.193+0000 7fa4191abb80 4 client.18834 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b3de2d3a", "release": true, "await": true}] 2024-06-10T01:19:23.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.193+0000 7fa4191abb80 1 -- 172.21.15.7:0/378468334 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b3de2d3a", "release": true, "await": true}) -- 0x557361071560 con 0x5573610ee530 2024-06-10T01:19:23.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.193+0000 7fa407fff700 1 --2- 172.21.15.7:0/378468334 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5573610ee530 0x5573610f09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:23.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.193+0000 7fa407fff700 1 --2- 172.21.15.7:0/378468334 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5573610ee530 0x5573610f09d0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c6a9e7765ed7eb server_cookie=9509cbc3181c13c9 in_seq=0 out_seq=0 2024-06-10T01:19:23.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.193+0000 7fa3e77fe700 10 client.18834 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:23.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.202+0000 7fa3e77fe700 1 -- 172.21.15.7:0/378468334 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557361071560 con 0x5573610ee530 2024-06-10T01:19:23.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.202+0000 7fa3e77fe700 10 client.18834 handle_command_reply: tid=0 2024-06-10T01:19:23.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.202+0000 7fa4191abb80 1 client.18834 shutdown 2024-06-10T01:19:23.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.202+0000 7fa4191abb80 1 -- 172.21.15.7:0/378468334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55736106ee20 msgr2=0x557361077ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:23.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.202+0000 7fa4191abb80 1 --2- 172.21.15.7:0/378468334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55736106ee20 0x557361077ac0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fa400008f50 tx=0x7fa400014530 comp rx=0 tx=0).stop 2024-06-10T01:19:23.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.202+0000 7fa4191abb80 1 -- 172.21.15.7:0/378468334 shutdown_connections 2024-06-10T01:19:23.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.202+0000 7fa4191abb80 1 --2- 172.21.15.7:0/378468334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55736106ee20 0x557361077ac0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.202+0000 7fa4191abb80 1 --2- 172.21.15.7:0/378468334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557361068c50 0x55736107ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.202+0000 7fa4191abb80 1 --2- 172.21.15.7:0/378468334 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5573610ee530 0x5573610f09d0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.202+0000 7fa4191abb80 1 --2- 172.21.15.7:0/378468334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55736106e500 0x55736107b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.202+0000 7fa4191abb80 1 -- 172.21.15.7:0/378468334 >> 172.21.15.7:0/378468334 conn(0x557360f95180 msgr2=0x557361071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:23.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.202+0000 7fa3e77fe700 0 client.18834 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:23.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.203+0000 7fa4191abb80 1 -- 172.21.15.7:0/378468334 shutdown_connections 2024-06-10T01:19:23.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.203+0000 7fa4191abb80 1 -- 172.21.15.7:0/378468334 wait complete. 2024-06-10T01:19:23.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.203+0000 7fa4191abb80 20 client.18834 trim_cache size 0 max 16384 2024-06-10T01:19:23.199 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:19:23.199 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 40, 2024-06-10T01:19:23.199 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:19:23.199 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 44, 2024-06-10T01:19:23.199 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:19:23.199 INFO:teuthology.orchestra.run.smithi007.stdout: "b3de2d3a": { 2024-06-10T01:19:23.199 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 44, 2024-06-10T01:19:23.199 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:19:23.199 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:19:23.199 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:19:23.199 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:19:23.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:19:23.199 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:19:23.200 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 130.0, 2024-06-10T01:19:23.200 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:19:23.200 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:19:23.200 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:19:23.200 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:19:23.200 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:19:23.200 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:19:23.200 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:19:23.200 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:19:23.200 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:19:23.200 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:19:23.200 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:19:23.200 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:19:23.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.203+0000 7fa40e928700 1 -- 172.21.15.7:0/869006498 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa3f40a2ca0 msgr2=0x7fa3f40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:23.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.203+0000 7fa40e928700 1 --2- 172.21.15.7:0/869006498 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa3f40a2ca0 0x7fa3f40a5160 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000b5c0 tx=0x7fa3f0005960 comp rx=0 tx=0).stop 2024-06-10T01:19:23.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.203+0000 7fa40e928700 1 -- 172.21.15.7:0/869006498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa408100a80 msgr2=0x7fa40819e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:23.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.203+0000 7fa40e928700 1 --2- 172.21.15.7:0/869006498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa408100a80 0x7fa40819e930 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800b530 tx=0x7fa3f800b840 comp rx=0 tx=0).stop 2024-06-10T01:19:23.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.203+0000 7fa40e928700 1 -- 172.21.15.7:0/869006498 shutdown_connections 2024-06-10T01:19:23.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.203+0000 7fa40e928700 1 --2- 172.21.15.7:0/869006498 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa3f40a2ca0 0x7fa3f40a5160 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.203+0000 7fa40e928700 1 --2- 172.21.15.7:0/869006498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa408108510 0x7fa408198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.203+0000 7fa40e928700 1 --2- 172.21.15.7:0/869006498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa408100a80 0x7fa40819e930 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.203+0000 7fa40e928700 1 --2- 172.21.15.7:0/869006498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa408101440 0x7fa40819ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:23.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.203+0000 7fa40e928700 1 -- 172.21.15.7:0/869006498 >> 172.21.15.7:0/869006498 conn(0x7fa408077a00 msgr2=0x7fa4080ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:23.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.203+0000 7fa40e928700 1 -- 172.21.15.7:0/869006498 shutdown_connections 2024-06-10T01:19:23.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:23.203+0000 7fa40e928700 1 -- 172.21.15.7:0/869006498 wait complete. 2024-06-10T01:19:23.364 INFO:tasks.workunit.client.0.smithi007.stdout: 0 cleanup 604 sec 2024-06-10T01:19:23.364 INFO:tasks.workunit.client.0.smithi007.stdout: 2024-06-10T01:19:23.364 INFO:tasks.workunit.client.0.smithi007.stdout: Operation Count AvgLat MaxLat 2024-06-10T01:19:23.364 INFO:tasks.workunit.client.0.smithi007.stdout: ---------------------------------------- 2024-06-10T01:19:23.365 INFO:tasks.workunit.client.0.smithi007.stdout: NTCreateX 139564 1.770 34556.410 2024-06-10T01:19:23.365 INFO:tasks.workunit.client.0.smithi007.stdout: Close 102514 0.002 0.147 2024-06-10T01:19:23.365 INFO:tasks.workunit.client.0.smithi007.stdout: Rename 5908 9.547 27739.160 2024-06-10T01:19:23.365 INFO:tasks.workunit.client.0.smithi007.stdout: Unlink 28201 3.510 18065.003 2024-06-10T01:19:23.365 INFO:tasks.workunit.client.0.smithi007.stdout: Deltree 4 91.733 157.976 2024-06-10T01:19:23.365 INFO:tasks.workunit.client.0.smithi007.stdout: Mkdir 2 0.008 0.010 2024-06-10T01:19:23.365 INFO:tasks.workunit.client.0.smithi007.stdout: Qpathinfo 126467 0.898 125.478 2024-06-10T01:19:23.365 INFO:tasks.workunit.client.0.smithi007.stdout: Qfileinfo 22172 0.001 0.050 2024-06-10T01:19:23.365 INFO:tasks.workunit.client.0.smithi007.stdout: Qfsinfo 23199 0.200 55.989 2024-06-10T01:19:23.365 INFO:tasks.workunit.client.0.smithi007.stdout: Sfileinfo 11376 0.011 22.515 2024-06-10T01:19:23.365 INFO:tasks.workunit.client.0.smithi007.stdout: Find 48908 0.177 98.346 2024-06-10T01:19:23.365 INFO:tasks.workunit.client.0.smithi007.stdout: WriteX 69669 0.324 20696.203 2024-06-10T01:19:23.365 INFO:tasks.workunit.client.0.smithi007.stdout: ReadX 218666 0.041 686.606 2024-06-10T01:19:23.365 INFO:tasks.workunit.client.0.smithi007.stdout: LockX 454 0.727 18.114 2024-06-10T01:19:23.366 INFO:tasks.workunit.client.0.smithi007.stdout: UnlockX 454 0.644 16.228 2024-06-10T01:19:23.366 INFO:tasks.workunit.client.0.smithi007.stdout: Flush 9797 4.026 126.695 2024-06-10T01:19:23.366 INFO:tasks.workunit.client.0.smithi007.stdout: 2024-06-10T01:19:23.366 INFO:tasks.workunit.client.0.smithi007.stdout:Throughput 7.32034 MB/sec 1 clients 1 procs max_latency=34556.416 ms 2024-06-10T01:19:23.366 INFO:tasks.workunit.client.0.smithi007.stderr:+ dbench 10 2024-06-10T01:19:23.367 INFO:tasks.workunit.client.0.smithi007.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-06-10T01:19:23.367 INFO:tasks.workunit.client.0.smithi007.stdout: 2024-06-10T01:19:23.367 INFO:tasks.workunit.client.0.smithi007.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-06-10T01:19:23.367 INFO:tasks.workunit.client.0.smithi007.stdout:8 of 10 processes prepared for launch 0 sec 2024-06-10T01:19:23.466 INFO:tasks.workunit.client.0.smithi007.stdout:10 of 10 processes prepared for launch 0 sec 2024-06-10T01:19:23.466 INFO:tasks.workunit.client.0.smithi007.stdout:releasing clients 2024-06-10T01:19:23.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:23 smithi078 ceph-mon[36683]: pgmap v833: 129 pgs: 129 active+clean; 170 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 3.8 MiB/s wr, 201 op/s 2024-06-10T01:19:23.828 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:23 smithi007 ceph-mon[32398]: pgmap v833: 129 pgs: 129 active+clean; 170 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 3.8 MiB/s wr, 201 op/s 2024-06-10T01:19:23.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:23 smithi175 ceph-mon[36636]: pgmap v833: 129 pgs: 129 active+clean; 170 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 3.8 MiB/s wr, 201 op/s 2024-06-10T01:19:24.466 INFO:tasks.workunit.client.0.smithi007.stdout: 10 343 123.69 MB/sec warmup 1 sec latency 104.431 ms 2024-06-10T01:19:24.871 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b3de2d3a', total seconds elapsed: 17.9 2024-06-10T01:19:25.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.381+0000 7f8b353c0700 1 -- 172.21.15.7:0/237685759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b30102530 msgr2=0x7f8b301029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:25.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.381+0000 7f8b353c0700 1 --2- 172.21.15.7:0/237685759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b30102530 0x7f8b301029b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8b24009920 tx=0x7f8b24009c30 comp rx=0 tx=0).stop 2024-06-10T01:19:25.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.381+0000 7f8b353c0700 1 -- 172.21.15.7:0/237685759 shutdown_connections 2024-06-10T01:19:25.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.381+0000 7f8b353c0700 1 --2- 172.21.15.7:0/237685759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b30102ef0 0x7f8b3010f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:25.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.381+0000 7f8b353c0700 1 --2- 172.21.15.7:0/237685759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b301085d0 0x7f8b301089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:25.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.381+0000 7f8b353c0700 1 --2- 172.21.15.7:0/237685759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b30102530 0x7f8b301029b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:25.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.381+0000 7f8b353c0700 1 -- 172.21.15.7:0/237685759 >> 172.21.15.7:0/237685759 conn(0x7f8b30079640 msgr2=0x7f8b30079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:25.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.381+0000 7f8b353c0700 1 -- 172.21.15.7:0/237685759 shutdown_connections 2024-06-10T01:19:25.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.381+0000 7f8b353c0700 1 -- 172.21.15.7:0/237685759 wait complete. 2024-06-10T01:19:25.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.382+0000 7f8b353c0700 1 Processor -- start 2024-06-10T01:19:25.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.382+0000 7f8b353c0700 1 -- start start 2024-06-10T01:19:25.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.383+0000 7f8b353c0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b30102530 0x7f8b3019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:25.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.383+0000 7f8b353c0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b30102ef0 0x7f8b3019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:25.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.383+0000 7f8b353c0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b301085d0 0x7f8b30198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:25.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.383+0000 7f8b353c0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8b30112870 con 0x7f8b30102530 2024-06-10T01:19:25.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.383+0000 7f8b353c0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8b301126f0 con 0x7f8b30102ef0 2024-06-10T01:19:25.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.383+0000 7f8b353c0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8b301129f0 con 0x7f8b301085d0 2024-06-10T01:19:25.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.383+0000 7f8b2effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b30102530 0x7f8b3019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:25.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.383+0000 7f8b2e7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b30102ef0 0x7f8b3019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:25.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.383+0000 7f8b2effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b30102530 0x7f8b3019e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39170/0 (socket says 172.21.15.7:39170) 2024-06-10T01:19:25.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.383+0000 7f8b2effd700 1 -- 172.21.15.7:0/2145669833 learned_addr learned my addr 172.21.15.7:0/2145669833 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:25.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.383+0000 7f8b2f7fe700 1 --2- 172.21.15.7:0/2145669833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b301085d0 0x7f8b30198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:25.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.383+0000 7f8b2effd700 1 -- 172.21.15.7:0/2145669833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b301085d0 msgr2=0x7f8b30198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:25.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.383+0000 7f8b2effd700 1 --2- 172.21.15.7:0/2145669833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b301085d0 0x7f8b30198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:25.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.383+0000 7f8b2effd700 1 -- 172.21.15.7:0/2145669833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b30102ef0 msgr2=0x7f8b3019ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:25.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.383+0000 7f8b2effd700 1 --2- 172.21.15.7:0/2145669833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b30102ef0 0x7f8b3019ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:25.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.383+0000 7f8b2effd700 1 -- 172.21.15.7:0/2145669833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b30199250 con 0x7f8b30102530 2024-06-10T01:19:25.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.384+0000 7f8b2effd700 1 --2- 172.21.15.7:0/2145669833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b30102530 0x7f8b3019e930 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f8b2000ec10 tx=0x7f8b2000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:25.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.384+0000 7f8b17fff700 1 -- 172.21.15.7:0/2145669833 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2000cc90 con 0x7f8b30102530 2024-06-10T01:19:25.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.384+0000 7f8b353c0700 1 -- 172.21.15.7:0/2145669833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b30199540 con 0x7f8b30102530 2024-06-10T01:19:25.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.384+0000 7f8b17fff700 1 -- 172.21.15.7:0/2145669833 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b2000ce30 con 0x7f8b30102530 2024-06-10T01:19:25.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.384+0000 7f8b17fff700 1 -- 172.21.15.7:0/2145669833 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b20010690 con 0x7f8b30102530 2024-06-10T01:19:25.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.384+0000 7f8b353c0700 1 -- 172.21.15.7:0/2145669833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b301a5800 con 0x7f8b30102530 2024-06-10T01:19:25.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.386+0000 7f8b17fff700 1 -- 172.21.15.7:0/2145669833 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f8b20003680 con 0x7f8b30102530 2024-06-10T01:19:25.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.387+0000 7f8b17fff700 1 --2- 172.21.15.7:0/2145669833 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8b180a2c20 0x7f8b180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:25.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.387+0000 7f8b2e7fc700 1 --2- 172.21.15.7:0/2145669833 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8b180a2c20 0x7f8b180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:25.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.387+0000 7f8b17fff700 1 -- 172.21.15.7:0/2145669833 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f8b200cf720 con 0x7f8b30102530 2024-06-10T01:19:25.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.388+0000 7f8b2e7fc700 1 --2- 172.21.15.7:0/2145669833 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8b180a2c20 0x7f8b180a50e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8b240098f0 tx=0x7f8b24019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:25.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.388+0000 7f8b3fc43b80 1 Processor -- start 2024-06-10T01:19:25.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.388+0000 7f8b3fc43b80 1 -- start start 2024-06-10T01:19:25.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.388+0000 7f8b3fc43b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a972285050 0x55a972285430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:25.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.388+0000 7f8b3fc43b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a972285970 0x55a972289740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:25.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.388+0000 7f8b3fc43b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a972289c80 0x55a97228a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:25.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.388+0000 7f8b3fc43b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a9721d2890 con 0x55a972289c80 2024-06-10T01:19:25.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.388+0000 7f8b3fc43b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a97217a7a0 con 0x55a972285970 2024-06-10T01:19:25.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.388+0000 7f8b3fc43b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a9721ce850 con 0x55a972285050 2024-06-10T01:19:25.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.389+0000 7f8b2f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a972289c80 0x55a97228a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:25.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.389+0000 7f8b2e7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a972285970 0x55a972289740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:25.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.389+0000 7f8b2effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a972285050 0x55a972285430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:25.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.389+0000 7f8b2f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a972289c80 0x55a97228a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39180/0 (socket says 172.21.15.7:39180) 2024-06-10T01:19:25.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.389+0000 7f8b2f7fe700 1 -- 172.21.15.7:0/1850656698 learned_addr learned my addr 172.21.15.7:0/1850656698 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:25.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.389+0000 7f8b2f7fe700 1 -- 172.21.15.7:0/1850656698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a972285050 msgr2=0x55a972285430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:25.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.389+0000 7f8b2f7fe700 1 --2- 172.21.15.7:0/1850656698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a972285050 0x55a972285430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:25.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.389+0000 7f8b2f7fe700 1 -- 172.21.15.7:0/1850656698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a972285970 msgr2=0x55a972289740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:25.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.389+0000 7f8b2f7fe700 1 --2- 172.21.15.7:0/1850656698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a972285970 0x55a972289740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:25.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.389+0000 7f8b2f7fe700 1 -- 172.21.15.7:0/1850656698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a972170fd0 con 0x55a972289c80 2024-06-10T01:19:25.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.389+0000 7f8b2f7fe700 1 --2- 172.21.15.7:0/1850656698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a972289c80 0x55a97228a0e0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f8b28011680 tx=0x7f8b28009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a35820b5505917d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:25.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.389+0000 7f8b157fa700 1 -- 172.21.15.7:0/1850656698 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b28019030 con 0x55a972289c80 2024-06-10T01:19:25.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.389+0000 7f8b157fa700 1 -- 172.21.15.7:0/1850656698 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b280092e0 con 0x55a972289c80 2024-06-10T01:19:25.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.389+0000 7f8b157fa700 1 -- 172.21.15.7:0/1850656698 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b28007500 con 0x55a972289c80 2024-06-10T01:19:25.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.390+0000 7f8b3fc43b80 1 -- 172.21.15.7:0/1850656698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a972289c80 msgr2=0x55a97228a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:25.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.390+0000 7f8b3fc43b80 1 --2- 172.21.15.7:0/1850656698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a972289c80 0x55a97228a0e0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f8b28011680 tx=0x7f8b28009920 comp rx=0 tx=0).stop 2024-06-10T01:19:25.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.390+0000 7f8b3fc43b80 1 -- 172.21.15.7:0/1850656698 shutdown_connections 2024-06-10T01:19:25.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.390+0000 7f8b3fc43b80 1 --2- 172.21.15.7:0/1850656698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a972289c80 0x55a97228a0e0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:25.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.390+0000 7f8b3fc43b80 1 --2- 172.21.15.7:0/1850656698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a972285050 0x55a972285430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:25.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.390+0000 7f8b3fc43b80 1 --2- 172.21.15.7:0/1850656698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a972285970 0x55a972289740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:25.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.390+0000 7f8b3fc43b80 1 -- 172.21.15.7:0/1850656698 >> 172.21.15.7:0/1850656698 conn(0x55a9721abd40 msgr2=0x55a97227b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:25.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.390+0000 7f8b3fc43b80 1 -- 172.21.15.7:0/1850656698 shutdown_connections 2024-06-10T01:19:25.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.390+0000 7f8b3fc43b80 1 -- 172.21.15.7:0/1850656698 wait complete. 2024-06-10T01:19:25.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.391+0000 7f8b3fc43b80 1 -- start start 2024-06-10T01:19:25.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.391+0000 7f8b3fc43b80 1 Processor -- start 2024-06-10T01:19:25.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.391+0000 7f8b3fc43b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a972285050 0x55a972291120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:25.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.391+0000 7f8b3fc43b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a972285970 0x55a97228e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:25.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.391+0000 7f8b3fc43b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a972289c80 0x55a97228e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:25.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.391+0000 7f8b3fc43b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a972291e90 con 0x55a972285050 2024-06-10T01:19:25.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.391+0000 7f8b3fc43b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a972172c70 con 0x55a972289c80 2024-06-10T01:19:25.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.391+0000 7f8b3fc43b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a972292010 con 0x55a972285970 2024-06-10T01:19:25.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.391+0000 7f8b3fc43b80 10 client.0 _ll_register_callbacks cb 0x55a97216c9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:19:25.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.391+0000 7f8b2effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a972285050 0x55a972291120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:25.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.391+0000 7f8b2effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a972285050 0x55a972291120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39186/0 (socket says 172.21.15.7:39186) 2024-06-10T01:19:25.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.391+0000 7f8b2f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a972289c80 0x55a97228e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:25.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.391+0000 7f8b2effd700 1 -- 172.21.15.7:0/3570012239 learned_addr learned my addr 172.21.15.7:0/3570012239 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:25.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.391+0000 7f8b2e7fc700 1 --2- 172.21.15.7:0/3570012239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a972285970 0x55a97228e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:25.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.392+0000 7f8b2effd700 1 -- 172.21.15.7:0/3570012239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a972285970 msgr2=0x55a97228e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:25.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.392+0000 7f8b2effd700 1 --2- 172.21.15.7:0/3570012239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a972285970 0x55a97228e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:25.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.392+0000 7f8b2effd700 1 -- 172.21.15.7:0/3570012239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a972289c80 msgr2=0x55a97228e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:25.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.392+0000 7f8b2effd700 1 --2- 172.21.15.7:0/3570012239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a972289c80 0x55a97228e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:25.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.392+0000 7f8b2effd700 1 -- 172.21.15.7:0/3570012239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a972121950 con 0x55a972285050 2024-06-10T01:19:25.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.392+0000 7f8b2f7fe700 1 --2- 172.21.15.7:0/3570012239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a972289c80 0x55a97228e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:25.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.392+0000 7f8b2effd700 1 --2- 172.21.15.7:0/3570012239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a972285050 0x55a972291120 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f8b200cd210 tx=0x7f8b2009b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7981f6b2a0d0ada0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:25.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.392+0000 7f8b12ff5700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:19:25.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.392+0000 7f8b12ff5700 1 -- 172.21.15.7:0/3570012239 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2009ffb0 con 0x55a972285050 2024-06-10T01:19:25.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.392+0000 7f8b3fc43b80 1 -- 172.21.15.7:0/3570012239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b280092e0 con 0x55a972285050 2024-06-10T01:19:25.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.392+0000 7f8b12ff5700 1 -- 172.21.15.7:0/3570012239 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f8b200b4e00 con 0x55a972285050 2024-06-10T01:19:25.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.392+0000 7f8b12ff5700 1 -- 172.21.15.7:0/3570012239 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8b200a4870 con 0x55a972285050 2024-06-10T01:19:25.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.392+0000 7f8b3fc43b80 10 client.18846 fetch_fsmap learned FSMap version 41 2024-06-10T01:19:25.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.392+0000 7f8b3fc43b80 1 -- 172.21.15.7:0/3570012239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55a97229c7c0 con 0x55a972285050 2024-06-10T01:19:25.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.393+0000 7f8b12ff5700 1 -- 172.21.15.7:0/3570012239 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b280092e0 con 0x55a972285050 2024-06-10T01:19:25.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.393+0000 7f8b12ff5700 1 -- 172.21.15.7:0/3570012239 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 41) ==== 8350+0+0 (secure 0 0 0) 0x7f8b2009b050 con 0x55a972285050 2024-06-10T01:19:25.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.393+0000 7f8b3fc43b80 10 client.18846 fetch_fsmap finished waiting for FSMap version 41 2024-06-10T01:19:25.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.393+0000 7f8b3fc43b80 10 client.18846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:19:25.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.393+0000 7f8b3fc43b80 20 client.18846 populate_metadata read hostname 'smithi007' 2024-06-10T01:19:25.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.393+0000 7f8b3fc43b80 1 --2- 172.21.15.7:0/3570012239 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a97229f1f0 0x55a9722a1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:25.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.393+0000 7f8b3fc43b80 4 client.18846 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:19:25.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.393+0000 7f8b3fc43b80 1 -- 172.21.15.7:0/3570012239 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9721dfd90 con 0x55a97229f1f0 2024-06-10T01:19:25.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.393+0000 7f8b2e7fc700 1 --2- 172.21.15.7:0/3570012239 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a97229f1f0 0x55a9722a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:25.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.394+0000 7f8b12ff5700 10 client.18846 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:25.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.394+0000 7f8b2e7fc700 1 --2- 172.21.15.7:0/3570012239 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a97229f1f0 0x55a9722a1630 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=833293b8ce3a7c3 server_cookie=b5b9cf561cd1b874 in_seq=0 out_seq=0 2024-06-10T01:19:25.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.397+0000 7f8b12ff5700 1 -- 172.21.15.7:0/3570012239 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a9721dfd90 con 0x55a97229f1f0 2024-06-10T01:19:25.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.397+0000 7f8b12ff5700 10 client.18846 handle_command_reply: tid=0 2024-06-10T01:19:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.397+0000 7f8b3fc43b80 1 client.18846 shutdown 2024-06-10T01:19:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.397+0000 7f8b3fc43b80 1 -- 172.21.15.7:0/3570012239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a972285050 msgr2=0x55a972291120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.397+0000 7f8b3fc43b80 1 --2- 172.21.15.7:0/3570012239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a972285050 0x55a972291120 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f8b200cd210 tx=0x7f8b2009b650 comp rx=0 tx=0).stop 2024-06-10T01:19:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.397+0000 7f8b3fc43b80 1 -- 172.21.15.7:0/3570012239 shutdown_connections 2024-06-10T01:19:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.397+0000 7f8b3fc43b80 1 --2- 172.21.15.7:0/3570012239 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a97229f1f0 0x55a9722a1630 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.397+0000 7f8b12ff5700 0 client.18846 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.397+0000 7f8b3fc43b80 1 --2- 172.21.15.7:0/3570012239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a972289c80 0x55a97228e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.397+0000 7f8b3fc43b80 1 --2- 172.21.15.7:0/3570012239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a972285050 0x55a972291120 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.397+0000 7f8b3fc43b80 1 --2- 172.21.15.7:0/3570012239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a972285970 0x55a97228e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.397+0000 7f8b3fc43b80 1 -- 172.21.15.7:0/3570012239 >> 172.21.15.7:0/3570012239 conn(0x55a9721abd40 msgr2=0x55a97227fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.398+0000 7f8b3fc43b80 1 -- 172.21.15.7:0/3570012239 shutdown_connections 2024-06-10T01:19:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.398+0000 7f8b3fc43b80 1 -- 172.21.15.7:0/3570012239 wait complete. 2024-06-10T01:19:25.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.398+0000 7f8b3fc43b80 20 client.18846 trim_cache size 0 max 16384 2024-06-10T01:19:25.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.424+0000 7f8b3fc43b80 1 Processor -- start 2024-06-10T01:19:25.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.424+0000 7f8b3fc43b80 1 -- start start 2024-06-10T01:19:25.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.425+0000 7f8b3fc43b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a972285050 0x55a972275590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:25.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.425+0000 7f8b3fc43b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a972285970 0x55a97228b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:25.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.425+0000 7f8b3fc43b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a972289c80 0x55a97228bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:25.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.425+0000 7f8b3fc43b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8b301ae950 con 0x55a972285050 2024-06-10T01:19:25.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.425+0000 7f8b3fc43b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8b301ac1e0 con 0x55a972285970 2024-06-10T01:19:25.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.425+0000 7f8b3fc43b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8b301aa030 con 0x55a972289c80 2024-06-10T01:19:25.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.425+0000 7f8b2effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a972285050 0x55a972275590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:25.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.425+0000 7f8b2f7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a972289c80 0x55a97228bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:25.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.425+0000 7f8b2effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a972285050 0x55a972275590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39196/0 (socket says 172.21.15.7:39196) 2024-06-10T01:19:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.425+0000 7f8b2effd700 1 -- 172.21.15.7:0/1194376801 learned_addr learned my addr 172.21.15.7:0/1194376801 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.425+0000 7f8b2e7fc700 1 --2- 172.21.15.7:0/1194376801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a972285970 0x55a97228b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.425+0000 7f8b2effd700 1 -- 172.21.15.7:0/1194376801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a972289c80 msgr2=0x55a97228bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.425+0000 7f8b2effd700 1 --2- 172.21.15.7:0/1194376801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a972289c80 0x55a97228bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.425+0000 7f8b2effd700 1 -- 172.21.15.7:0/1194376801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a972285970 msgr2=0x55a97228b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.425+0000 7f8b2effd700 1 --2- 172.21.15.7:0/1194376801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a972285970 0x55a97228b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.425+0000 7f8b2effd700 1 -- 172.21.15.7:0/1194376801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9722a19a0 con 0x55a972285050 2024-06-10T01:19:25.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.426+0000 7f8b2effd700 1 --2- 172.21.15.7:0/1194376801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a972285050 0x55a972275590 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f8b20099eb0 tx=0x7f8b2009a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04a90891d86a49f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:25.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.426+0000 7f8b14ff9700 1 -- 172.21.15.7:0/1194376801 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b200a43e0 con 0x55a972285050 2024-06-10T01:19:25.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.426+0000 7f8b14ff9700 1 -- 172.21.15.7:0/1194376801 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b2009ffb0 con 0x55a972285050 2024-06-10T01:19:25.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.426+0000 7f8b14ff9700 1 -- 172.21.15.7:0/1194376801 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2009be60 con 0x55a972285050 2024-06-10T01:19:25.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.427+0000 7f8b3fc43b80 1 -- 172.21.15.7:0/1194376801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a972285050 msgr2=0x55a972275590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:25.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.427+0000 7f8b3fc43b80 1 --2- 172.21.15.7:0/1194376801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a972285050 0x55a972275590 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f8b20099eb0 tx=0x7f8b2009a110 comp rx=0 tx=0).stop 2024-06-10T01:19:25.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.427+0000 7f8b3fc43b80 1 -- 172.21.15.7:0/1194376801 shutdown_connections 2024-06-10T01:19:25.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.427+0000 7f8b3fc43b80 1 --2- 172.21.15.7:0/1194376801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a972289c80 0x55a97228bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:25.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.427+0000 7f8b3fc43b80 1 --2- 172.21.15.7:0/1194376801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a972285050 0x55a972275590 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:25.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.427+0000 7f8b3fc43b80 1 --2- 172.21.15.7:0/1194376801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a972285970 0x55a97228b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:25.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.427+0000 7f8b3fc43b80 1 -- 172.21.15.7:0/1194376801 >> 172.21.15.7:0/1194376801 conn(0x55a9721abd40 msgr2=0x55a972274e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:25.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.427+0000 7f8b3fc43b80 1 -- 172.21.15.7:0/1194376801 shutdown_connections 2024-06-10T01:19:25.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.427+0000 7f8b3fc43b80 1 -- 172.21.15.7:0/1194376801 wait complete. 2024-06-10T01:19:25.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.427+0000 7f8b3fc43b80 1 -- start start 2024-06-10T01:19:25.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.428+0000 7f8b3fc43b80 1 Processor -- start 2024-06-10T01:19:25.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.428+0000 7f8b3fc43b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a972285050 0x55a972297590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:25.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.428+0000 7f8b3fc43b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a972285970 0x55a972297ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:25.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.428+0000 7f8b3fc43b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a972289c80 0x55a9722923a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:25.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.428+0000 7f8b3fc43b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8b301aa6a0 con 0x55a972289c80 2024-06-10T01:19:25.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.428+0000 7f8b3fc43b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8b301abb10 con 0x55a972285970 2024-06-10T01:19:25.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.428+0000 7f8b2f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a972289c80 0x55a9722923a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:25.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.428+0000 7f8b2e7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a972285970 0x55a972297ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:25.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.428+0000 7f8b3fc43b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8b301aad70 con 0x55a972285050 2024-06-10T01:19:25.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.428+0000 7f8b2f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a972289c80 0x55a9722923a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39202/0 (socket says 172.21.15.7:39202) 2024-06-10T01:19:25.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.428+0000 7f8b2e7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a972285970 0x55a972297ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:52352/0 (socket says 172.21.15.7:52352) 2024-06-10T01:19:25.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.428+0000 7f8b2f7fe700 1 -- 172.21.15.7:0/1544059846 learned_addr learned my addr 172.21.15.7:0/1544059846 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:25.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.428+0000 7f8b3fc43b80 10 client.0 _ll_register_callbacks cb 0x55a97216c9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:19:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.428+0000 7f8b2effd700 1 --2- 172.21.15.7:0/1544059846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a972285050 0x55a972297590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.429+0000 7f8b2f7fe700 1 -- 172.21.15.7:0/1544059846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a972285050 msgr2=0x55a972297590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.429+0000 7f8b2f7fe700 1 --2- 172.21.15.7:0/1544059846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a972285050 0x55a972297590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.429+0000 7f8b2f7fe700 1 -- 172.21.15.7:0/1544059846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a972285970 msgr2=0x55a972297ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.429+0000 7f8b2f7fe700 1 --2- 172.21.15.7:0/1544059846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a972285970 0x55a972297ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.429+0000 7f8b2f7fe700 1 -- 172.21.15.7:0/1544059846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a972292b70 con 0x55a972289c80 2024-06-10T01:19:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.429+0000 7f8b2e7fc700 1 --2- 172.21.15.7:0/1544059846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a972285970 0x55a972297ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:19:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.429+0000 7f8b2f7fe700 1 --2- 172.21.15.7:0/1544059846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a972289c80 0x55a9722923a0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800afe0 tx=0x7f8b28006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a46fb05eb45ac server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.429+0000 7f8b12ff5700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:19:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.429+0000 7f8b12ff5700 1 -- 172.21.15.7:0/1544059846 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b28003e10 con 0x55a972289c80 2024-06-10T01:19:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.429+0000 7f8b12ff5700 1 -- 172.21.15.7:0/1544059846 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f8b28003fb0 con 0x55a972289c80 2024-06-10T01:19:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.429+0000 7f8b3fc43b80 1 -- 172.21.15.7:0/1544059846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b2009ffb0 con 0x55a972289c80 2024-06-10T01:19:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.429+0000 7f8b12ff5700 1 -- 172.21.15.7:0/1544059846 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8b280093a0 con 0x55a972289c80 2024-06-10T01:19:25.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.429+0000 7f8b3fc43b80 10 client.18852 fetch_fsmap learned FSMap version 41 2024-06-10T01:19:25.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.429+0000 7f8b3fc43b80 1 -- 172.21.15.7:0/1544059846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55a97230aaf0 con 0x55a972289c80 2024-06-10T01:19:25.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.430+0000 7f8b12ff5700 1 -- 172.21.15.7:0/1544059846 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b2009ffb0 con 0x55a972289c80 2024-06-10T01:19:25.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.430+0000 7f8b12ff5700 1 -- 172.21.15.7:0/1544059846 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 41) ==== 8350+0+0 (secure 0 0 0) 0x7f8b28027070 con 0x55a972289c80 2024-06-10T01:19:25.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.430+0000 7f8b3fc43b80 10 client.18852 fetch_fsmap finished waiting for FSMap version 41 2024-06-10T01:19:25.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.430+0000 7f8b3fc43b80 10 client.18852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:19:25.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.430+0000 7f8b3fc43b80 20 client.18852 populate_metadata read hostname 'smithi007' 2024-06-10T01:19:25.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.430+0000 7f8b3fc43b80 1 --2- 172.21.15.7:0/1544059846 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a97229f1f0 0x55a97230d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:25.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.430+0000 7f8b3fc43b80 4 client.18852 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:19:25.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.430+0000 7f8b3fc43b80 1 -- 172.21.15.7:0/1544059846 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a9721da310 con 0x55a97229f1f0 2024-06-10T01:19:25.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.430+0000 7f8b2effd700 1 --2- 172.21.15.7:0/1544059846 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a97229f1f0 0x55a97230d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:25.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.430+0000 7f8b2effd700 1 --2- 172.21.15.7:0/1544059846 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a97229f1f0 0x55a97230d040 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f40c094cb45a46f server_cookie=549fed834cc64fea in_seq=0 out_seq=0 2024-06-10T01:19:25.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.430+0000 7f8b12ff5700 10 client.18852 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:25.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.437+0000 7f8b12ff5700 1 -- 172.21.15.7:0/1544059846 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a9721da310 con 0x55a97229f1f0 2024-06-10T01:19:25.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.437+0000 7f8b12ff5700 10 client.18852 handle_command_reply: tid=0 2024-06-10T01:19:25.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.437+0000 7f8b3fc43b80 1 client.18852 shutdown 2024-06-10T01:19:25.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.438+0000 7f8b3fc43b80 1 -- 172.21.15.7:0/1544059846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a972289c80 msgr2=0x55a9722923a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:25.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.438+0000 7f8b3fc43b80 1 --2- 172.21.15.7:0/1544059846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a972289c80 0x55a9722923a0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800afe0 tx=0x7f8b28006000 comp rx=0 tx=0).stop 2024-06-10T01:19:25.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.438+0000 7f8b3fc43b80 1 -- 172.21.15.7:0/1544059846 shutdown_connections 2024-06-10T01:19:25.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.438+0000 7f8b3fc43b80 1 --2- 172.21.15.7:0/1544059846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a972289c80 0x55a9722923a0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:25.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.438+0000 7f8b3fc43b80 1 --2- 172.21.15.7:0/1544059846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a972285050 0x55a972297590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:25.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.438+0000 7f8b3fc43b80 1 --2- 172.21.15.7:0/1544059846 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a97229f1f0 0x55a97230d040 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:25.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.438+0000 7f8b3fc43b80 1 --2- 172.21.15.7:0/1544059846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a972285970 0x55a972297ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:25.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.438+0000 7f8b12ff5700 0 client.18852 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:25.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.438+0000 7f8b3fc43b80 1 -- 172.21.15.7:0/1544059846 >> 172.21.15.7:0/1544059846 conn(0x55a9721abd40 msgr2=0x55a97228fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:25.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.438+0000 7f8b3fc43b80 1 -- 172.21.15.7:0/1544059846 shutdown_connections 2024-06-10T01:19:25.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.438+0000 7f8b3fc43b80 1 -- 172.21.15.7:0/1544059846 wait complete. 2024-06-10T01:19:25.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.438+0000 7f8b3fc43b80 20 client.18852 trim_cache size 0 max 16384 2024-06-10T01:19:25.434 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:19:25.434 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-10T01:19:25.434 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "318eb243-8a51-4cfd-aeb3-9fb634995914", 2024-06-10T01:19:25.434 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-10T01:19:25.434 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:19:25.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.439+0000 7f8b353c0700 1 -- 172.21.15.7:0/2145669833 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8b180a2c20 msgr2=0x7f8b180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:25.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.439+0000 7f8b353c0700 1 --2- 172.21.15.7:0/2145669833 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8b180a2c20 0x7f8b180a50e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8b240098f0 tx=0x7f8b24019040 comp rx=0 tx=0).stop 2024-06-10T01:19:25.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.439+0000 7f8b353c0700 1 -- 172.21.15.7:0/2145669833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b30102530 msgr2=0x7f8b3019e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:25.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.439+0000 7f8b353c0700 1 --2- 172.21.15.7:0/2145669833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b30102530 0x7f8b3019e930 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f8b2000ec10 tx=0x7f8b2000ef20 comp rx=0 tx=0).stop 2024-06-10T01:19:25.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.439+0000 7f8b353c0700 1 -- 172.21.15.7:0/2145669833 shutdown_connections 2024-06-10T01:19:25.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.439+0000 7f8b353c0700 1 --2- 172.21.15.7:0/2145669833 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8b180a2c20 0x7f8b180a50e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:25.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.439+0000 7f8b353c0700 1 --2- 172.21.15.7:0/2145669833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b301085d0 0x7f8b30198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:25.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.439+0000 7f8b353c0700 1 --2- 172.21.15.7:0/2145669833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b30102530 0x7f8b3019e930 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:25.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.439+0000 7f8b353c0700 1 --2- 172.21.15.7:0/2145669833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b30102ef0 0x7f8b3019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:25.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.439+0000 7f8b353c0700 1 -- 172.21.15.7:0/2145669833 >> 172.21.15.7:0/2145669833 conn(0x7f8b30079640 msgr2=0x7f8b30101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:25.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.439+0000 7f8b353c0700 1 -- 172.21.15.7:0/2145669833 shutdown_connections 2024-06-10T01:19:25.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:25.439+0000 7f8b353c0700 1 -- 172.21.15.7:0/2145669833 wait complete. 2024-06-10T01:19:25.466 INFO:tasks.workunit.client.0.smithi007.stdout: 10 724 104.10 MB/sec warmup 2 sec latency 138.728 ms 2024-06-10T01:19:25.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:25 smithi007 ceph-mon[32398]: pgmap v834: 129 pgs: 129 active+clean; 168 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 3.7 MiB/s wr, 201 op/s 2024-06-10T01:19:25.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:25 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:19:25.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:25 smithi078 ceph-mon[36683]: pgmap v834: 129 pgs: 129 active+clean; 168 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 3.7 MiB/s wr, 201 op/s 2024-06-10T01:19:25.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:25 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:19:25.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:25 smithi175 ceph-mon[36636]: pgmap v834: 129 pgs: 129 active+clean; 168 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 3.7 MiB/s wr, 201 op/s 2024-06-10T01:19:25.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:25 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:19:26.273 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:19:26.466 INFO:tasks.workunit.client.0.smithi007.stdout: 10 1132 76.21 MB/sec warmup 3 sec latency 77.457 ms 2024-06-10T01:19:26.563 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:19:26.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:26 smithi078 ceph-mon[36683]: scrub queued for path: / 2024-06-10T01:19:26.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:26 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:19:26.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:26 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:19:26.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:26 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:19:26.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:26 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:19:26.833 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:26 smithi007 ceph-mon[32398]: scrub queued for path: / 2024-06-10T01:19:26.834 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:26 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:19:26.834 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:26 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:19:26.834 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:26 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:19:26.834 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:26 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:19:26.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:26 smithi175 ceph-mon[36636]: scrub queued for path: / 2024-06-10T01:19:26.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:26 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:19:26.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:26 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:19:26.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:26 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:19:26.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:26 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:19:27.466 INFO:tasks.workunit.client.0.smithi007.stdout: 10 1809 65.14 MB/sec warmup 4 sec latency 69.932 ms 2024-06-10T01:19:27.811 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:27 smithi007 ceph-mon[32398]: pgmap v835: 129 pgs: 129 active+clean; 163 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 2.3 MiB/s wr, 133 op/s 2024-06-10T01:19:27.811 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:27 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:19:27.812 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:27 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:19:27.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:27 smithi078 ceph-mon[36683]: pgmap v835: 129 pgs: 129 active+clean; 163 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 2.3 MiB/s wr, 133 op/s 2024-06-10T01:19:27.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:27 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:19:27.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:27 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:19:27.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:27 smithi175 ceph-mon[36636]: pgmap v835: 129 pgs: 129 active+clean; 163 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 2.3 MiB/s wr, 133 op/s 2024-06-10T01:19:27.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:27 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:19:27.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:27 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:19:28.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.179+0000 7fecd9e33700 1 -- 172.21.15.7:0/2884820710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd4101390 msgr2=0x7fecd41017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:28.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.179+0000 7fecd9e33700 1 --2- 172.21.15.7:0/2884820710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd4101390 0x7fecd41017f0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fecc8009990 tx=0x7fecc8009ca0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.179+0000 7fecd9e33700 1 -- 172.21.15.7:0/2884820710 shutdown_connections 2024-06-10T01:19:28.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.179+0000 7fecd9e33700 1 --2- 172.21.15.7:0/2884820710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecd4101d30 0x7fecd410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.179+0000 7fecd9e33700 1 --2- 172.21.15.7:0/2884820710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fecd41073d0 0x7fecd41077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.179+0000 7fecd9e33700 1 --2- 172.21.15.7:0/2884820710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd4101390 0x7fecd41017f0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.179+0000 7fecd9e33700 1 -- 172.21.15.7:0/2884820710 >> 172.21.15.7:0/2884820710 conn(0x7fecd40782c0 msgr2=0x7fecd40786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:28.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.179+0000 7fecd9e33700 1 -- 172.21.15.7:0/2884820710 shutdown_connections 2024-06-10T01:19:28.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.179+0000 7fecd9e33700 1 -- 172.21.15.7:0/2884820710 wait complete. 2024-06-10T01:19:28.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.180+0000 7fecd9e33700 1 Processor -- start 2024-06-10T01:19:28.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.180+0000 7fecd9e33700 1 -- start start 2024-06-10T01:19:28.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.180+0000 7fecd9e33700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecd4101390 0x7fecd4103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:28.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.180+0000 7fecd9e33700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd4101d30 0x7fecd4103580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:28.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.180+0000 7fecd9e33700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fecd41073d0 0x7fecd4103ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:28.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.180+0000 7fecd9e33700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fecd4111630 con 0x7fecd4101d30 2024-06-10T01:19:28.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.180+0000 7fecd9e33700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fecd41114b0 con 0x7fecd41073d0 2024-06-10T01:19:28.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.180+0000 7fecd9e33700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fecd41117b0 con 0x7fecd4101390 2024-06-10T01:19:28.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.180+0000 7fecd2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd4101d30 0x7fecd4103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:28.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.180+0000 7fecd37fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecd4101390 0x7fecd4103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:28.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.180+0000 7fecd2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd4101d30 0x7fecd4103580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39230/0 (socket says 172.21.15.7:39230) 2024-06-10T01:19:28.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.180+0000 7fecd37fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecd4101390 0x7fecd4103040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36974/0 (socket says 172.21.15.7:36974) 2024-06-10T01:19:28.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.180+0000 7fecd2ffd700 1 -- 172.21.15.7:0/3314090974 learned_addr learned my addr 172.21.15.7:0/3314090974 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:28.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.181+0000 7fecd3fff700 1 --2- 172.21.15.7:0/3314090974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fecd41073d0 0x7fecd4103ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:28.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.181+0000 7fecd37fe700 1 -- 172.21.15.7:0/3314090974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fecd41073d0 msgr2=0x7fecd4103ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:28.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.181+0000 7fecd37fe700 1 --2- 172.21.15.7:0/3314090974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fecd41073d0 0x7fecd4103ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.181+0000 7fecd37fe700 1 -- 172.21.15.7:0/3314090974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd4101d30 msgr2=0x7fecd4103580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:28.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.181+0000 7fecd37fe700 1 --2- 172.21.15.7:0/3314090974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd4101d30 0x7fecd4103580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.181+0000 7fecd37fe700 1 -- 172.21.15.7:0/3314090974 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd41a7180 con 0x7fecd4101390 2024-06-10T01:19:28.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.181+0000 7fecd2ffd700 1 --2- 172.21.15.7:0/3314090974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd4101d30 0x7fecd4103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:28.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.181+0000 7fecd3fff700 1 --2- 172.21.15.7:0/3314090974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fecd41073d0 0x7fecd4103ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:19:28.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.181+0000 7fecd37fe700 1 --2- 172.21.15.7:0/3314090974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecd4101390 0x7fecd4103040 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fecc400d950 tx=0x7fecc400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:28.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.181+0000 7fecd0ff9700 1 -- 172.21.15.7:0/3314090974 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc4017070 con 0x7fecd4101390 2024-06-10T01:19:28.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.181+0000 7fecd9e33700 1 -- 172.21.15.7:0/3314090974 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecd41a7470 con 0x7fecd4101390 2024-06-10T01:19:28.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.181+0000 7fecd0ff9700 1 -- 172.21.15.7:0/3314090974 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecc4009e10 con 0x7fecd4101390 2024-06-10T01:19:28.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.181+0000 7fecd9e33700 1 -- 172.21.15.7:0/3314090974 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecd41a7a00 con 0x7fecd4101390 2024-06-10T01:19:28.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.181+0000 7fecd0ff9700 1 -- 172.21.15.7:0/3314090974 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc400f3c0 con 0x7fecd4101390 2024-06-10T01:19:28.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.184+0000 7fecd0ff9700 1 -- 172.21.15.7:0/3314090974 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fecc401a070 con 0x7fecd4101390 2024-06-10T01:19:28.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.184+0000 7fecd0ff9700 1 --2- 172.21.15.7:0/3314090974 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fecbc0a2ca0 0x7fecbc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:28.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.184+0000 7fecd2ffd700 1 --2- 172.21.15.7:0/3314090974 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fecbc0a2ca0 0x7fecbc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:28.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.185+0000 7fecd0ff9700 1 -- 172.21.15.7:0/3314090974 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fecc40cecb0 con 0x7fecd4101390 2024-06-10T01:19:28.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.185+0000 7fecd2ffd700 1 --2- 172.21.15.7:0/3314090974 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fecbc0a2ca0 0x7fecbc0a5160 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fecd4078850 tx=0x7fecc8005c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:28.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.185+0000 7fece46b6b80 1 Processor -- start 2024-06-10T01:19:28.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.185+0000 7fece46b6b80 1 -- start start 2024-06-10T01:19:28.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.186+0000 7fece46b6b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a77ce0640 0x556a77ce0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:28.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.186+0000 7fece46b6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77ce0f60 0x556a77cda9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:28.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.186+0000 7fece46b6b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a77cdaf20 0x556a77cdb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:28.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.186+0000 7fece46b6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556a77c2c810 con 0x556a77ce0f60 2024-06-10T01:19:28.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.186+0000 7fece46b6b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556a77bd4720 con 0x556a77cdaf20 2024-06-10T01:19:28.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.186+0000 7fece46b6b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556a77c287d0 con 0x556a77ce0640 2024-06-10T01:19:28.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.186+0000 7fecd2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77ce0f60 0x556a77cda9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:28.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.186+0000 7fecd2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77ce0f60 0x556a77cda9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39240/0 (socket says 172.21.15.7:39240) 2024-06-10T01:19:28.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.186+0000 7fecd3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a77cdaf20 0x556a77cdb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:28.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.186+0000 7fecd2ffd700 1 -- 172.21.15.7:0/2236002807 learned_addr learned my addr 172.21.15.7:0/2236002807 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:28.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.186+0000 7fecd37fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a77ce0640 0x556a77ce0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:28.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.186+0000 7fecd2ffd700 1 -- 172.21.15.7:0/2236002807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a77ce0640 msgr2=0x556a77ce0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:28.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.186+0000 7fecd2ffd700 1 --2- 172.21.15.7:0/2236002807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a77ce0640 0x556a77ce0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.186+0000 7fecd2ffd700 1 -- 172.21.15.7:0/2236002807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a77cdaf20 msgr2=0x556a77cdb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:28.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.186+0000 7fecd2ffd700 1 --2- 172.21.15.7:0/2236002807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a77cdaf20 0x556a77cdb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.186+0000 7fecd2ffd700 1 -- 172.21.15.7:0/2236002807 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a77bcaf50 con 0x556a77ce0f60 2024-06-10T01:19:28.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.186+0000 7fecd2ffd700 1 --2- 172.21.15.7:0/2236002807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77ce0f60 0x556a77cda9e0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fecc800be10 tx=0x7fecc8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5eca205cd530199 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:28.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.186+0000 7fecb9ffb700 1 -- 172.21.15.7:0/2236002807 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc800be40 con 0x556a77ce0f60 2024-06-10T01:19:28.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.187+0000 7fecb9ffb700 1 -- 172.21.15.7:0/2236002807 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecc800fb50 con 0x556a77ce0f60 2024-06-10T01:19:28.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.187+0000 7fecb9ffb700 1 -- 172.21.15.7:0/2236002807 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc802f600 con 0x556a77ce0f60 2024-06-10T01:19:28.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.187+0000 7fece46b6b80 1 -- 172.21.15.7:0/2236002807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77ce0f60 msgr2=0x556a77cda9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:28.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.187+0000 7fece46b6b80 1 --2- 172.21.15.7:0/2236002807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77ce0f60 0x556a77cda9e0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fecc800be10 tx=0x7fecc8020000 comp rx=0 tx=0).stop 2024-06-10T01:19:28.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.187+0000 7fece46b6b80 1 -- 172.21.15.7:0/2236002807 shutdown_connections 2024-06-10T01:19:28.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.187+0000 7fece46b6b80 1 --2- 172.21.15.7:0/2236002807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a77cdaf20 0x556a77cdb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.187+0000 7fece46b6b80 1 --2- 172.21.15.7:0/2236002807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a77ce0640 0x556a77ce0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.187+0000 7fece46b6b80 1 --2- 172.21.15.7:0/2236002807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77ce0f60 0x556a77cda9e0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.187+0000 7fece46b6b80 1 -- 172.21.15.7:0/2236002807 >> 172.21.15.7:0/2236002807 conn(0x556a77c05cc0 msgr2=0x556a77cd5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:28.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.187+0000 7fece46b6b80 1 -- 172.21.15.7:0/2236002807 shutdown_connections 2024-06-10T01:19:28.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.187+0000 7fece46b6b80 1 -- 172.21.15.7:0/2236002807 wait complete. 2024-06-10T01:19:28.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.188+0000 7fece46b6b80 1 -- start start 2024-06-10T01:19:28.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.188+0000 7fece46b6b80 1 Processor -- start 2024-06-10T01:19:28.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.188+0000 7fece46b6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77cdaf20 0x556a77ce6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:28.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.188+0000 7fece46b6b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a77ce0640 0x556a77ce3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:28.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.188+0000 7fece46b6b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a77ce0f60 0x556a77ce4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:28.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.188+0000 7fece46b6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556a77ce7910 con 0x556a77cdaf20 2024-06-10T01:19:28.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.188+0000 7fece46b6b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556a77bccbf0 con 0x556a77ce0f60 2024-06-10T01:19:28.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.188+0000 7fece46b6b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556a77ce7a90 con 0x556a77ce0640 2024-06-10T01:19:28.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.188+0000 7fece46b6b80 10 client.0 _ll_register_callbacks cb 0x556a77bc6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:19:28.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.188+0000 7fecd37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77cdaf20 0x556a77ce6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:28.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.188+0000 7fecd2ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a77ce0640 0x556a77ce3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:28.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.188+0000 7fecd37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77cdaf20 0x556a77ce6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39242/0 (socket says 172.21.15.7:39242) 2024-06-10T01:19:28.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.188+0000 7fecd37fe700 1 -- 172.21.15.7:0/2363467146 learned_addr learned my addr 172.21.15.7:0/2363467146 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:28.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.188+0000 7fecd3fff700 1 --2- 172.21.15.7:0/2363467146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a77ce0f60 0x556a77ce4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:28.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.189+0000 7fecd37fe700 1 -- 172.21.15.7:0/2363467146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a77ce0640 msgr2=0x556a77ce3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:28.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.189+0000 7fecd37fe700 1 --2- 172.21.15.7:0/2363467146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a77ce0640 0x556a77ce3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.189+0000 7fecd37fe700 1 -- 172.21.15.7:0/2363467146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a77ce0f60 msgr2=0x556a77ce4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:28.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.189+0000 7fecd37fe700 1 --2- 172.21.15.7:0/2363467146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a77ce0f60 0x556a77ce4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.189+0000 7fecd37fe700 1 -- 172.21.15.7:0/2363467146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a77b7b8d0 con 0x556a77cdaf20 2024-06-10T01:19:28.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.189+0000 7fecd3fff700 1 --2- 172.21.15.7:0/2363467146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a77ce0f60 0x556a77ce4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:28.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.189+0000 7fecd37fe700 1 --2- 172.21.15.7:0/2363467146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77cdaf20 0x556a77ce6cd0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fecc40cc7a0 tx=0x7fecc409aaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d0a7bb7c262d4d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:28.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.189+0000 7fecb2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:19:28.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.189+0000 7fecb2ffd700 1 -- 172.21.15.7:0/2363467146 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc40b4060 con 0x556a77cdaf20 2024-06-10T01:19:28.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.189+0000 7fecb2ffd700 1 -- 172.21.15.7:0/2363467146 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fecc409f170 con 0x556a77cdaf20 2024-06-10T01:19:28.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.189+0000 7fece46b6b80 1 -- 172.21.15.7:0/2363467146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecc800fb50 con 0x556a77cdaf20 2024-06-10T01:19:28.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.189+0000 7fecb2ffd700 1 -- 172.21.15.7:0/2363467146 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fecc4021040 con 0x556a77cdaf20 2024-06-10T01:19:28.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.189+0000 7fece46b6b80 10 client.18864 fetch_fsmap learned FSMap version 42 2024-06-10T01:19:28.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.189+0000 7fece46b6b80 1 -- 172.21.15.7:0/2363467146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556a77cf2260 con 0x556a77cdaf20 2024-06-10T01:19:28.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.190+0000 7fecb2ffd700 1 -- 172.21.15.7:0/2363467146 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecc800fb50 con 0x556a77cdaf20 2024-06-10T01:19:28.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.190+0000 7fecb2ffd700 1 -- 172.21.15.7:0/2363467146 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 42) ==== 8354+0+0 (secure 0 0 0) 0x7fecc40bb0b0 con 0x556a77cdaf20 2024-06-10T01:19:28.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.190+0000 7fece46b6b80 10 client.18864 fetch_fsmap finished waiting for FSMap version 42 2024-06-10T01:19:28.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.190+0000 7fece46b6b80 10 client.18864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:19:28.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.190+0000 7fece46b6b80 20 client.18864 populate_metadata read hostname 'smithi007' 2024-06-10T01:19:28.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.190+0000 7fece46b6b80 1 --2- 172.21.15.7:0/2363467146 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556a77cf48a0 0x556a77cf6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:28.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.190+0000 7fece46b6b80 4 client.18864 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:19:28.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.190+0000 7fece46b6b80 1 -- 172.21.15.7:0/2363467146 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a77c39d10 con 0x556a77cf48a0 2024-06-10T01:19:28.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.190+0000 7fecd2ffd700 1 --2- 172.21.15.7:0/2363467146 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556a77cf48a0 0x556a77cf6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:28.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.191+0000 7fecd2ffd700 1 --2- 172.21.15.7:0/2363467146 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556a77cf48a0 0x556a77cf6c90 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8070e6066659df06 server_cookie=4985e937c70e228e in_seq=0 out_seq=0 2024-06-10T01:19:28.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.191+0000 7fecb2ffd700 10 client.18864 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:28.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.195+0000 7fecb2ffd700 1 -- 172.21.15.7:0/2363467146 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556a77c39d10 con 0x556a77cf48a0 2024-06-10T01:19:28.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.195+0000 7fecb2ffd700 10 client.18864 handle_command_reply: tid=0 2024-06-10T01:19:28.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.195+0000 7fece46b6b80 1 client.18864 shutdown 2024-06-10T01:19:28.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.195+0000 7fece46b6b80 1 -- 172.21.15.7:0/2363467146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77cdaf20 msgr2=0x556a77ce6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:28.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.195+0000 7fece46b6b80 1 --2- 172.21.15.7:0/2363467146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77cdaf20 0x556a77ce6cd0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fecc40cc7a0 tx=0x7fecc409aaf0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.195+0000 7fece46b6b80 1 -- 172.21.15.7:0/2363467146 shutdown_connections 2024-06-10T01:19:28.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.195+0000 7fece46b6b80 1 --2- 172.21.15.7:0/2363467146 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556a77cf48a0 0x556a77cf6c90 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.195+0000 7fece46b6b80 1 --2- 172.21.15.7:0/2363467146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a77ce0f60 0x556a77ce4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.195+0000 7fece46b6b80 1 --2- 172.21.15.7:0/2363467146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77cdaf20 0x556a77ce6cd0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.195+0000 7fece46b6b80 1 --2- 172.21.15.7:0/2363467146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a77ce0640 0x556a77ce3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.195+0000 7fece46b6b80 1 -- 172.21.15.7:0/2363467146 >> 172.21.15.7:0/2363467146 conn(0x556a77c05cc0 msgr2=0x556a77cd55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:28.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.195+0000 7fecb2ffd700 0 client.18864 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:28.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.195+0000 7fece46b6b80 1 -- 172.21.15.7:0/2363467146 shutdown_connections 2024-06-10T01:19:28.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.195+0000 7fece46b6b80 1 -- 172.21.15.7:0/2363467146 wait complete. 2024-06-10T01:19:28.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.195+0000 7fece46b6b80 20 client.18864 trim_cache size 0 max 16384 2024-06-10T01:19:28.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.216+0000 7fece46b6b80 1 Processor -- start 2024-06-10T01:19:28.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.216+0000 7fece46b6b80 1 -- start start 2024-06-10T01:19:28.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.216+0000 7fece46b6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77cdaf20 0x556a77cda9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:28.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.216+0000 7fece46b6b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a77ce0640 0x556a77ccf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:28.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.216+0000 7fece46b6b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a77ce0f60 0x556a77cfc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:28.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.216+0000 7fece46b6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fecd41b2c90 con 0x556a77cdaf20 2024-06-10T01:19:28.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.216+0000 7fece46b6b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fecd41b04f0 con 0x556a77ce0640 2024-06-10T01:19:28.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.216+0000 7fece46b6b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fecd41ae340 con 0x556a77ce0f60 2024-06-10T01:19:28.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.216+0000 7fecd37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77cdaf20 0x556a77cda9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:28.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.216+0000 7fecd37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77cdaf20 0x556a77cda9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39248/0 (socket says 172.21.15.7:39248) 2024-06-10T01:19:28.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.216+0000 7fecd37fe700 1 -- 172.21.15.7:0/3950285419 learned_addr learned my addr 172.21.15.7:0/3950285419 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:28.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.216+0000 7fecd3fff700 1 --2- 172.21.15.7:0/3950285419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a77ce0f60 0x556a77cfc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:28.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.216+0000 7fecd2ffd700 1 --2- 172.21.15.7:0/3950285419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a77ce0640 0x556a77ccf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:28.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.217+0000 7fecd37fe700 1 -- 172.21.15.7:0/3950285419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a77ce0f60 msgr2=0x556a77cfc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:28.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.217+0000 7fecd37fe700 1 --2- 172.21.15.7:0/3950285419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a77ce0f60 0x556a77cfc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.217+0000 7fecd37fe700 1 -- 172.21.15.7:0/3950285419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a77ce0640 msgr2=0x556a77ccf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:28.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.217+0000 7fecd37fe700 1 --2- 172.21.15.7:0/3950285419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a77ce0640 0x556a77ccf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.217+0000 7fecd37fe700 1 -- 172.21.15.7:0/3950285419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a77ce0140 con 0x556a77cdaf20 2024-06-10T01:19:28.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.217+0000 7fecd2ffd700 1 --2- 172.21.15.7:0/3950285419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a77ce0640 0x556a77ccf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:19:28.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.217+0000 7fecd37fe700 1 --2- 172.21.15.7:0/3950285419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77cdaf20 0x556a77cda9e0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fecc4005950 tx=0x7fecc40996a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ad76e6d5ff5631 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:28.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.217+0000 7fecb97fa700 1 -- 172.21.15.7:0/3950285419 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc4014020 con 0x556a77cdaf20 2024-06-10T01:19:28.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.217+0000 7fecb97fa700 1 -- 172.21.15.7:0/3950285419 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecc40a3e00 con 0x556a77cdaf20 2024-06-10T01:19:28.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.217+0000 7fecb97fa700 1 -- 172.21.15.7:0/3950285419 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc40b4040 con 0x556a77cdaf20 2024-06-10T01:19:28.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.218+0000 7fece46b6b80 1 -- 172.21.15.7:0/3950285419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77cdaf20 msgr2=0x556a77cda9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:28.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.218+0000 7fece46b6b80 1 --2- 172.21.15.7:0/3950285419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77cdaf20 0x556a77cda9e0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fecc4005950 tx=0x7fecc40996a0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.218+0000 7fece46b6b80 1 -- 172.21.15.7:0/3950285419 shutdown_connections 2024-06-10T01:19:28.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.218+0000 7fece46b6b80 1 --2- 172.21.15.7:0/3950285419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a77ce0f60 0x556a77cfc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.218+0000 7fece46b6b80 1 --2- 172.21.15.7:0/3950285419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77cdaf20 0x556a77cda9e0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.218+0000 7fece46b6b80 1 --2- 172.21.15.7:0/3950285419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a77ce0640 0x556a77ccf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.218+0000 7fece46b6b80 1 -- 172.21.15.7:0/3950285419 >> 172.21.15.7:0/3950285419 conn(0x556a77c05cc0 msgr2=0x556a77ce19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:28.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.218+0000 7fece46b6b80 1 -- 172.21.15.7:0/3950285419 shutdown_connections 2024-06-10T01:19:28.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.218+0000 7fece46b6b80 1 -- 172.21.15.7:0/3950285419 wait complete. 2024-06-10T01:19:28.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.218+0000 7fece46b6b80 1 -- start start 2024-06-10T01:19:28.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.218+0000 7fece46b6b80 1 Processor -- start 2024-06-10T01:19:28.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.219+0000 7fece46b6b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a77cdaf20 0x556a77ce84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:28.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.219+0000 7fece46b6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77ce0640 0x556a77ce8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:28.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.219+0000 7fece46b6b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a77ce0f60 0x556a77d5d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:28.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.219+0000 7fece46b6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fecd41ae9b0 con 0x556a77ce0640 2024-06-10T01:19:28.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.219+0000 7fece46b6b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fecd41afe20 con 0x556a77cdaf20 2024-06-10T01:19:28.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.219+0000 7fece46b6b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fecd41af080 con 0x556a77ce0f60 2024-06-10T01:19:28.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.219+0000 7fece46b6b80 10 client.0 _ll_register_callbacks cb 0x556a77bc6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:19:28.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.219+0000 7fecd2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77ce0640 0x556a77ce8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:28.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.219+0000 7fecd2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77ce0640 0x556a77ce8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39252/0 (socket says 172.21.15.7:39252) 2024-06-10T01:19:28.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.219+0000 7fecd2ffd700 1 -- 172.21.15.7:0/1639422405 learned_addr learned my addr 172.21.15.7:0/1639422405 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:28.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.219+0000 7fecd3fff700 1 --2- 172.21.15.7:0/1639422405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a77ce0f60 0x556a77d5d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:28.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.219+0000 7fecd37fe700 1 --2- 172.21.15.7:0/1639422405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a77cdaf20 0x556a77ce84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:28.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.219+0000 7fecd2ffd700 1 -- 172.21.15.7:0/1639422405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a77ce0f60 msgr2=0x556a77d5d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:28.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.219+0000 7fecd2ffd700 1 --2- 172.21.15.7:0/1639422405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a77ce0f60 0x556a77d5d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.219+0000 7fecd2ffd700 1 -- 172.21.15.7:0/1639422405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a77cdaf20 msgr2=0x556a77ce84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:28.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.219+0000 7fecd2ffd700 1 --2- 172.21.15.7:0/1639422405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a77cdaf20 0x556a77ce84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.219+0000 7fecd2ffd700 1 -- 172.21.15.7:0/1639422405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a77d5daf0 con 0x556a77ce0640 2024-06-10T01:19:28.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.219+0000 7fecd2ffd700 1 --2- 172.21.15.7:0/1639422405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77ce0640 0x556a77ce8a10 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fecc800bde0 tx=0x7fecc8005980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db77baa3b53c7fc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:28.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.219+0000 7fecb2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:19:28.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.220+0000 7fecb2ffd700 1 -- 172.21.15.7:0/1639422405 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc800fdf0 con 0x556a77ce0640 2024-06-10T01:19:28.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.220+0000 7fece46b6b80 1 -- 172.21.15.7:0/1639422405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecc40a3e00 con 0x556a77ce0640 2024-06-10T01:19:28.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.220+0000 7fecb2ffd700 1 -- 172.21.15.7:0/1639422405 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fecc8035040 con 0x556a77ce0640 2024-06-10T01:19:28.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.220+0000 7fecb2ffd700 1 -- 172.21.15.7:0/1639422405 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fecc8024410 con 0x556a77ce0640 2024-06-10T01:19:28.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.220+0000 7fece46b6b80 10 client.18870 fetch_fsmap learned FSMap version 42 2024-06-10T01:19:28.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.220+0000 7fece46b6b80 1 -- 172.21.15.7:0/1639422405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556a77d5ede0 con 0x556a77ce0640 2024-06-10T01:19:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.220+0000 7fecb2ffd700 1 -- 172.21.15.7:0/1639422405 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecc40a3e00 con 0x556a77ce0640 2024-06-10T01:19:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.220+0000 7fecb2ffd700 1 -- 172.21.15.7:0/1639422405 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 42) ==== 8354+0+0 (secure 0 0 0) 0x7fecc8039a10 con 0x556a77ce0640 2024-06-10T01:19:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.220+0000 7fece46b6b80 10 client.18870 fetch_fsmap finished waiting for FSMap version 42 2024-06-10T01:19:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.220+0000 7fece46b6b80 10 client.18870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:19:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.220+0000 7fece46b6b80 20 client.18870 populate_metadata read hostname 'smithi007' 2024-06-10T01:19:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.220+0000 7fece46b6b80 1 --2- 172.21.15.7:0/1639422405 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556a77d5f070 0x556a77d61460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.220+0000 7fece46b6b80 4 client.18870 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:19:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.220+0000 7fece46b6b80 1 -- 172.21.15.7:0/1639422405 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556a77c34290 con 0x556a77d5f070 2024-06-10T01:19:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.220+0000 7fecd37fe700 1 --2- 172.21.15.7:0/1639422405 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556a77d5f070 0x556a77d61460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.221+0000 7fecb2ffd700 10 client.18870 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.221+0000 7fecd37fe700 1 --2- 172.21.15.7:0/1639422405 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556a77d5f070 0x556a77d61460 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22fa6db7c28944b8 server_cookie=8addc3609f173561 in_seq=0 out_seq=0 2024-06-10T01:19:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.222+0000 7fecb2ffd700 1 -- 172.21.15.7:0/1639422405 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556a77c34290 con 0x556a77d5f070 2024-06-10T01:19:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.222+0000 7fecb2ffd700 10 client.18870 handle_command_reply: tid=0 2024-06-10T01:19:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.222+0000 7fece46b6b80 1 client.18870 shutdown 2024-06-10T01:19:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.222+0000 7fece46b6b80 1 -- 172.21.15.7:0/1639422405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77ce0640 msgr2=0x556a77ce8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.222+0000 7fece46b6b80 1 --2- 172.21.15.7:0/1639422405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77ce0640 0x556a77ce8a10 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fecc800bde0 tx=0x7fecc8005980 comp rx=0 tx=0).stop 2024-06-10T01:19:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.222+0000 7fece46b6b80 1 -- 172.21.15.7:0/1639422405 shutdown_connections 2024-06-10T01:19:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.222+0000 7fece46b6b80 1 --2- 172.21.15.7:0/1639422405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a77ce0f60 0x556a77d5d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.222+0000 7fece46b6b80 1 --2- 172.21.15.7:0/1639422405 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556a77d5f070 0x556a77d61460 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.222+0000 7fece46b6b80 1 --2- 172.21.15.7:0/1639422405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a77cdaf20 0x556a77ce84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.222+0000 7fece46b6b80 1 --2- 172.21.15.7:0/1639422405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77ce0640 0x556a77ce8a10 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.222+0000 7fecb2ffd700 0 client.18870 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.222+0000 7fece46b6b80 1 -- 172.21.15.7:0/1639422405 >> 172.21.15.7:0/1639422405 conn(0x556a77c05cc0 msgr2=0x556a77cdf130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.222+0000 7fece46b6b80 1 -- 172.21.15.7:0/1639422405 shutdown_connections 2024-06-10T01:19:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.222+0000 7fece46b6b80 1 -- 172.21.15.7:0/1639422405 wait complete. 2024-06-10T01:19:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.222+0000 7fece46b6b80 20 client.18870 trim_cache size 0 max 16384 2024-06-10T01:19:28.222 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:19:28.222 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T01:19:28.222 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-10T01:19:28.222 INFO:teuthology.orchestra.run.smithi007.stdout: "318eb243-8a51-4cfd-aeb3-9fb634995914": { 2024-06-10T01:19:28.222 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-10T01:19:28.222 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "318eb243-8a51-4cfd-aeb3-9fb634995914", 2024-06-10T01:19:28.222 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-10T01:19:28.222 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:19:28.222 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:19:28.222 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:19:28.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.223+0000 7fecd9e33700 1 -- 172.21.15.7:0/3314090974 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fecbc0a2ca0 msgr2=0x7fecbc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:28.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.223+0000 7fecd9e33700 1 --2- 172.21.15.7:0/3314090974 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fecbc0a2ca0 0x7fecbc0a5160 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fecd4078850 tx=0x7fecc8005c00 comp rx=0 tx=0).stop 2024-06-10T01:19:28.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.223+0000 7fecd9e33700 1 -- 172.21.15.7:0/3314090974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecd4101390 msgr2=0x7fecd4103040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:28.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.223+0000 7fecd9e33700 1 --2- 172.21.15.7:0/3314090974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecd4101390 0x7fecd4103040 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fecc400d950 tx=0x7fecc400dc60 comp rx=0 tx=0).stop 2024-06-10T01:19:28.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.223+0000 7fecd9e33700 1 -- 172.21.15.7:0/3314090974 shutdown_connections 2024-06-10T01:19:28.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.223+0000 7fecd9e33700 1 --2- 172.21.15.7:0/3314090974 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fecbc0a2ca0 0x7fecbc0a5160 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.223+0000 7fecd9e33700 1 --2- 172.21.15.7:0/3314090974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fecd41073d0 0x7fecd4103ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.223+0000 7fecd9e33700 1 --2- 172.21.15.7:0/3314090974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecd4101390 0x7fecd4103040 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.223+0000 7fecd9e33700 1 --2- 172.21.15.7:0/3314090974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd4101d30 0x7fecd4103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:28.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.223+0000 7fecd9e33700 1 -- 172.21.15.7:0/3314090974 >> 172.21.15.7:0/3314090974 conn(0x7fecd40782c0 msgr2=0x7fecd406a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:28.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.223+0000 7fecd9e33700 1 -- 172.21.15.7:0/3314090974 shutdown_connections 2024-06-10T01:19:28.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:28.223+0000 7fecd9e33700 1 -- 172.21.15.7:0/3314090974 wait complete. 2024-06-10T01:19:28.466 INFO:tasks.workunit.client.0.smithi007.stdout: 10 2489 57.91 MB/sec warmup 5 sec latency 44.253 ms 2024-06-10T01:19:28.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:28 smithi078 ceph-mon[36683]: pgmap v836: 129 pgs: 129 active+clean; 162 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-06-10T01:19:28.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:28 smithi175 ceph-mon[36636]: pgmap v836: 129 pgs: 129 active+clean; 162 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-06-10T01:19:29.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:28 smithi007 ceph-mon[32398]: pgmap v836: 129 pgs: 129 active+clean; 162 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-06-10T01:19:29.065 INFO:tasks.cephfs.filesystem:scrub status for tag:318eb243-8a51-4cfd-aeb3-9fb634995914 - {'path': '/', 'tag': '318eb243-8a51-4cfd-aeb3-9fb634995914', 'options': 'recursive,force'} 2024-06-10T01:19:29.466 INFO:tasks.workunit.client.0.smithi007.stdout: 10 3130 56.21 MB/sec warmup 6 sec latency 84.719 ms 2024-06-10T01:19:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.265+0000 7fd9baba0640 1 -- 172.21.15.7:0/1685105262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9b41022e0 msgr2=0x7fd9b4108b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.265+0000 7fd9baba0640 1 --2- 172.21.15.7:0/1685105262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9b41022e0 0x7fd9b4108b70 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac009ca0 tx=0x7fd9ac02a8c0 comp rx=0 tx=0).stop 2024-06-10T01:19:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.265+0000 7fd9baba0640 1 -- 172.21.15.7:0/1685105262 shutdown_connections 2024-06-10T01:19:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.265+0000 7fd9baba0640 1 --2- 172.21.15.7:0/1685105262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9b41022e0 0x7fd9b4108b70 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.265+0000 7fd9baba0640 1 --2- 172.21.15.7:0/1685105262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9b4101920 0x7fd9b4101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.265+0000 7fd9baba0640 1 --2- 172.21.15.7:0/1685105262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9b4100730 0x7fd9b4100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.265+0000 7fd9baba0640 1 -- 172.21.15.7:0/1685105262 >> 172.21.15.7:0/1685105262 conn(0x7fd9b40fbcc0 msgr2=0x7fd9b40fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.265+0000 7fd9baba0640 1 -- 172.21.15.7:0/1685105262 shutdown_connections 2024-06-10T01:19:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.265+0000 7fd9baba0640 1 -- 172.21.15.7:0/1685105262 wait complete. 2024-06-10T01:19:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.265+0000 7fd9baba0640 1 Processor -- start 2024-06-10T01:19:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.266+0000 7fd9baba0640 1 -- start start 2024-06-10T01:19:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.266+0000 7fd9baba0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9b4100730 0x7fd9b4197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.266+0000 7fd9baba0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9b4101920 0x7fd9b4198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.266+0000 7fd9baba0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9b41022e0 0x7fd9b419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.266+0000 7fd9baba0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b410c3b0 con 0x7fd9b4101920 2024-06-10T01:19:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.266+0000 7fd9baba0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b410c230 con 0x7fd9b41022e0 2024-06-10T01:19:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.266+0000 7fd9baba0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b410c530 con 0x7fd9b4100730 2024-06-10T01:19:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.266+0000 7fd9b939d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9b4101920 0x7fd9b4198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.266+0000 7fd9b939d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9b4101920 0x7fd9b4198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39268/0 (socket says 172.21.15.7:39268) 2024-06-10T01:19:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.266+0000 7fd9b9b9e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9b4100730 0x7fd9b4197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.266+0000 7fd9b939d640 1 -- 172.21.15.7:0/3967650783 learned_addr learned my addr 172.21.15.7:0/3967650783 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.266+0000 7fd9ba39f640 1 --2- 172.21.15.7:0/3967650783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9b41022e0 0x7fd9b419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.266+0000 7fd9b939d640 1 -- 172.21.15.7:0/3967650783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9b4100730 msgr2=0x7fd9b4197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.266+0000 7fd9b939d640 1 --2- 172.21.15.7:0/3967650783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9b4100730 0x7fd9b4197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.266+0000 7fd9b939d640 1 -- 172.21.15.7:0/3967650783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9b41022e0 msgr2=0x7fd9b419f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.266+0000 7fd9b939d640 1 --2- 172.21.15.7:0/3967650783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9b41022e0 0x7fd9b419f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.266+0000 7fd9b939d640 1 -- 172.21.15.7:0/3967650783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b419f710 con 0x7fd9b4101920 2024-06-10T01:19:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.266+0000 7fd9b9b9e640 1 --2- 172.21.15.7:0/3967650783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9b4100730 0x7fd9b4197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.266+0000 7fd9b939d640 1 --2- 172.21.15.7:0/3967650783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9b4101920 0x7fd9b4198030 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fd9a400e9f0 tx=0x7fd9a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.267+0000 7fd9aaffd640 1 -- 172.21.15.7:0/3967650783 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a400cd80 con 0x7fd9b4101920 2024-06-10T01:19:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.267+0000 7fd9baba0640 1 -- 172.21.15.7:0/3967650783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9b419fa00 con 0x7fd9b4101920 2024-06-10T01:19:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.267+0000 7fd9aaffd640 1 -- 172.21.15.7:0/3967650783 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9a4010910 con 0x7fd9b4101920 2024-06-10T01:19:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.267+0000 7fd9aaffd640 1 -- 172.21.15.7:0/3967650783 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4018990 con 0x7fd9b4101920 2024-06-10T01:19:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.267+0000 7fd9baba0640 1 -- 172.21.15.7:0/3967650783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9b419ff50 con 0x7fd9b4101920 2024-06-10T01:19:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.269+0000 7fd9aaffd640 1 -- 172.21.15.7:0/3967650783 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fd9a4010ac0 con 0x7fd9b4101920 2024-06-10T01:19:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.269+0000 7fd9bebea740 1 Processor -- start 2024-06-10T01:19:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.270+0000 7fd9bebea740 1 -- start start 2024-06-10T01:19:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.270+0000 7fd9bebea740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559cec051200 0x559cec051620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.270+0000 7fd9bebea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cec051bf0 0x559cec0566f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.270+0000 7fd9bebea740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cec056d60 0x559cec0591f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.270+0000 7fd9bebea740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559cebfb44f0 con 0x559cec056d60 2024-06-10T01:19:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.270+0000 7fd9bebea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559cebf52870 con 0x559cec051bf0 2024-06-10T01:19:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.270+0000 7fd9bebea740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559cebfb04b0 con 0x559cec051200 2024-06-10T01:19:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.270+0000 7fd9b939d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cec056d60 0x559cec0591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.270+0000 7fd9ba39f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559cec051200 0x559cec051620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.270+0000 7fd9ba39f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559cec051200 0x559cec051620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:37036/0 (socket says 172.21.15.7:37036) 2024-06-10T01:19:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.270+0000 7fd9ba39f640 1 -- 172.21.15.7:0/4171711355 learned_addr learned my addr 172.21.15.7:0/4171711355 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.270+0000 7fd9b9b9e640 1 --2- 172.21.15.7:0/4171711355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cec051bf0 0x559cec0566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.270+0000 7fd9aaffd640 1 --2- 172.21.15.7:0/3967650783 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd9a00a2b70 0x7fd9a00a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.270+0000 7fd9ba39f640 1 -- 172.21.15.7:0/4171711355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cec051bf0 msgr2=0x559cec0566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.270+0000 7fd9ba39f640 1 --2- 172.21.15.7:0/4171711355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cec051bf0 0x559cec0566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.270+0000 7fd9ba39f640 1 -- 172.21.15.7:0/4171711355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cec056d60 msgr2=0x559cec0591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.270+0000 7fd9ba39f640 1 --2- 172.21.15.7:0/4171711355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cec056d60 0x559cec0591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.270+0000 7fd9ba39f640 1 -- 172.21.15.7:0/4171711355 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cebf6e9b0 con 0x559cec051200 2024-06-10T01:19:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.270+0000 7fd9b9b9e640 1 --2- 172.21.15.7:0/4171711355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cec051bf0 0x559cec0566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.270+0000 7fd9aaffd640 1 -- 172.21.15.7:0/3967650783 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fd9a40cf730 con 0x7fd9b4101920 2024-06-10T01:19:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.271+0000 7fd9ba39f640 1 --2- 172.21.15.7:0/3967650783 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd9a00a2b70 0x7fd9a00a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.271+0000 7fd9ba39f640 1 --2- 172.21.15.7:0/4171711355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559cec051200 0x559cec051620 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac009ca0 tx=0x7fd9ac0093b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebd219e0adfd206c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.271+0000 7fd99ffff640 1 -- 172.21.15.7:0/4171711355 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac042040 con 0x559cec051200 2024-06-10T01:19:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.271+0000 7fd99ffff640 1 -- 172.21.15.7:0/4171711355 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9ac044070 con 0x559cec051200 2024-06-10T01:19:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.271+0000 7fd99ffff640 1 -- 172.21.15.7:0/4171711355 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac051430 con 0x559cec051200 2024-06-10T01:19:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.272+0000 7fd9bebea740 1 -- 172.21.15.7:0/4171711355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559cec051200 msgr2=0x559cec051620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.272+0000 7fd9ba39f640 1 --2- 172.21.15.7:0/3967650783 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd9a00a2b70 0x7fd9a00a5030 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac04b000 tx=0x7fd9ac04c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.272+0000 7fd9bebea740 1 --2- 172.21.15.7:0/4171711355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559cec051200 0x559cec051620 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac009ca0 tx=0x7fd9ac0093b0 comp rx=0 tx=0).stop 2024-06-10T01:19:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.272+0000 7fd9bebea740 1 -- 172.21.15.7:0/4171711355 shutdown_connections 2024-06-10T01:19:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.272+0000 7fd9bebea740 1 --2- 172.21.15.7:0/4171711355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cec056d60 0x559cec0591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.272+0000 7fd9bebea740 1 --2- 172.21.15.7:0/4171711355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cec051bf0 0x559cec0566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.272+0000 7fd9bebea740 1 --2- 172.21.15.7:0/4171711355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559cec051200 0x559cec051620 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.272+0000 7fd9bebea740 1 -- 172.21.15.7:0/4171711355 >> 172.21.15.7:0/4171711355 conn(0x559cec04c870 msgr2=0x559cec04cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.272+0000 7fd9bebea740 1 -- 172.21.15.7:0/4171711355 shutdown_connections 2024-06-10T01:19:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.272+0000 7fd9bebea740 1 -- 172.21.15.7:0/4171711355 wait complete. 2024-06-10T01:19:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.272+0000 7fd9bebea740 1 -- start start 2024-06-10T01:19:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.272+0000 7fd9bebea740 1 Processor -- start 2024-06-10T01:19:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.273+0000 7fd9bebea740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cec051200 0x559cec05b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.273+0000 7fd9bebea740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559cec051bf0 0x559cec05b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.273+0000 7fd9bebea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cec056d60 0x559cec062a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.273+0000 7fd9bebea740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559cec059d10 con 0x559cec051200 2024-06-10T01:19:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.273+0000 7fd9bebea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559cebf3dae0 con 0x559cec056d60 2024-06-10T01:19:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.273+0000 7fd9bebea740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559cec059e90 con 0x559cec051bf0 2024-06-10T01:19:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.273+0000 7fd9bebea740 10 client.0 _ll_register_callbacks cb 0x559cebf769b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:19:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.273+0000 7fd9b9b9e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cec051200 0x559cec05b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.273+0000 7fd9b939d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559cec051bf0 0x559cec05b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.273+0000 7fd9ba39f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cec056d60 0x559cec062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.273+0000 7fd9b939d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559cec051bf0 0x559cec05b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:37052/0 (socket says 172.21.15.7:37052) 2024-06-10T01:19:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.273+0000 7fd9b939d640 1 -- 172.21.15.7:0/1917313340 learned_addr learned my addr 172.21.15.7:0/1917313340 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.273+0000 7fd9b9b9e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cec051200 0x559cec05b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39286/0 (socket says 172.21.15.7:39286) 2024-06-10T01:19:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.273+0000 7fd9b939d640 1 -- 172.21.15.7:0/1917313340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cec056d60 msgr2=0x559cec062a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.273+0000 7fd9b939d640 1 --2- 172.21.15.7:0/1917313340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cec056d60 0x559cec062a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.273+0000 7fd9b939d640 1 -- 172.21.15.7:0/1917313340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cec051200 msgr2=0x559cec05b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.273+0000 7fd9b939d640 1 --2- 172.21.15.7:0/1917313340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cec051200 0x559cec05b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.273+0000 7fd9b939d640 1 -- 172.21.15.7:0/1917313340 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cebcc8210 con 0x559cec051bf0 2024-06-10T01:19:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.273+0000 7fd9b939d640 1 --2- 172.21.15.7:0/1917313340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559cec051bf0 0x559cec05b9f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4098630 tx=0x7fd9a409b890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69abe5c3824529a6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.273+0000 7fd99d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:19:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.274+0000 7fd99d7fa640 1 -- 172.21.15.7:0/1917313340 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a409bc60 con 0x559cec051bf0 2024-06-10T01:19:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.274+0000 7fd99d7fa640 1 -- 172.21.15.7:0/1917313340 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fd9a4018b30 con 0x559cec051bf0 2024-06-10T01:19:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.274+0000 7fd9bebea740 1 -- 172.21.15.7:0/1917313340 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9ac044070 con 0x559cec051bf0 2024-06-10T01:19:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.274+0000 7fd99d7fa640 1 -- 172.21.15.7:0/1917313340 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd9a4018d30 con 0x559cec051bf0 2024-06-10T01:19:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.274+0000 7fd9bebea740 10 client.28210 fetch_fsmap learned FSMap version 42 2024-06-10T01:19:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.274+0000 7fd9bebea740 1 -- 172.21.15.7:0/1917313340 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x559cebd94620 con 0x559cec051bf0 2024-06-10T01:19:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.274+0000 7fd99d7fa640 1 -- 172.21.15.7:0/1917313340 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9ac044070 con 0x559cec051bf0 2024-06-10T01:19:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.274+0000 7fd99d7fa640 1 -- 172.21.15.7:0/1917313340 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 42) ==== 8354+0+0 (secure 0 0 0) 0x7fd9a40105c0 con 0x559cec051bf0 2024-06-10T01:19:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.274+0000 7fd9bebea740 10 client.28210 fetch_fsmap finished waiting for FSMap version 42 2024-06-10T01:19:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.274+0000 7fd9bebea740 10 client.28210 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:19:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.274+0000 7fd9bebea740 20 client.28210 populate_metadata read hostname 'smithi007' 2024-06-10T01:19:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.274+0000 7fd9bebea740 1 --2- 172.21.15.7:0/1917313340 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559cec0673c0 0x559cec069830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.274+0000 7fd9bebea740 4 client.28210 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:19:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.274+0000 7fd9bebea740 1 -- 172.21.15.7:0/1917313340 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cebfc7f10 con 0x559cec0673c0 2024-06-10T01:19:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.274+0000 7fd9b9b9e640 1 --2- 172.21.15.7:0/1917313340 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559cec0673c0 0x559cec069830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.275+0000 7fd99d7fa640 10 client.28210 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.275+0000 7fd9b9b9e640 1 --2- 172.21.15.7:0/1917313340 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559cec0673c0 0x559cec069830 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44ab7b6519b7a73b server_cookie=8aab66de4b31aff4 in_seq=0 out_seq=0 2024-06-10T01:19:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.278+0000 7fd99d7fa640 1 -- 172.21.15.7:0/1917313340 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559cebfc7f10 con 0x559cec0673c0 2024-06-10T01:19:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.278+0000 7fd99d7fa640 10 client.28210 handle_command_reply: tid=0 2024-06-10T01:19:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.278+0000 7fd9bebea740 1 client.28210 shutdown 2024-06-10T01:19:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.278+0000 7fd9bebea740 1 -- 172.21.15.7:0/1917313340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559cec051bf0 msgr2=0x559cec05b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.278+0000 7fd9bebea740 1 --2- 172.21.15.7:0/1917313340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559cec051bf0 0x559cec05b9f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4098630 tx=0x7fd9a409b890 comp rx=0 tx=0).stop 2024-06-10T01:19:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.278+0000 7fd9bebea740 1 -- 172.21.15.7:0/1917313340 shutdown_connections 2024-06-10T01:19:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.278+0000 7fd9bebea740 1 --2- 172.21.15.7:0/1917313340 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559cec0673c0 0x559cec069830 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.278+0000 7fd9bebea740 1 --2- 172.21.15.7:0/1917313340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cec056d60 0x559cec062a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.278+0000 7fd9bebea740 1 --2- 172.21.15.7:0/1917313340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559cec051bf0 0x559cec05b9f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.278+0000 7fd9bebea740 1 --2- 172.21.15.7:0/1917313340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cec051200 0x559cec05b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.278+0000 7fd99d7fa640 0 client.28210 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.278+0000 7fd9bebea740 1 -- 172.21.15.7:0/1917313340 >> 172.21.15.7:0/1917313340 conn(0x559cec04c870 msgr2=0x559cec04b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.278+0000 7fd9bebea740 1 -- 172.21.15.7:0/1917313340 shutdown_connections 2024-06-10T01:19:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.278+0000 7fd9bebea740 1 -- 172.21.15.7:0/1917313340 wait complete. 2024-06-10T01:19:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.278+0000 7fd9bebea740 20 client.28210 trim_cache size 0 max 16384 2024-06-10T01:19:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.294+0000 7fd9bebea740 1 Processor -- start 2024-06-10T01:19:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.295+0000 7fd9bebea740 1 -- start start 2024-06-10T01:19:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.295+0000 7fd9bebea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cec051200 0x559cec05b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.295+0000 7fd9bebea740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559cec051bf0 0x559cec0629f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.295+0000 7fd9bebea740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cec056d60 0x559cec05b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.295+0000 7fd9bebea740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41acc90 con 0x559cec056d60 2024-06-10T01:19:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.295+0000 7fd9bebea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41aa530 con 0x559cec051200 2024-06-10T01:19:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.295+0000 7fd9bebea740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41a8380 con 0x559cec051bf0 2024-06-10T01:19:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.295+0000 7fd9ba39f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cec056d60 0x559cec05b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.295+0000 7fd9b939d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559cec051bf0 0x559cec0629f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.295+0000 7fd9b9b9e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cec051200 0x559cec05b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.295+0000 7fd9ba39f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cec056d60 0x559cec05b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39298/0 (socket says 172.21.15.7:39298) 2024-06-10T01:19:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.295+0000 7fd9ba39f640 1 -- 172.21.15.7:0/1472004392 learned_addr learned my addr 172.21.15.7:0/1472004392 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.295+0000 7fd9ba39f640 1 -- 172.21.15.7:0/1472004392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559cec051bf0 msgr2=0x559cec0629f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.295+0000 7fd9ba39f640 1 --2- 172.21.15.7:0/1472004392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559cec051bf0 0x559cec0629f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.295+0000 7fd9ba39f640 1 -- 172.21.15.7:0/1472004392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cec051200 msgr2=0x559cec05b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.295+0000 7fd9ba39f640 1 --2- 172.21.15.7:0/1472004392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cec051200 0x559cec05b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.295+0000 7fd9ba39f640 1 -- 172.21.15.7:0/1472004392 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cec069eb0 con 0x559cec056d60 2024-06-10T01:19:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.295+0000 7fd9ba39f640 1 --2- 172.21.15.7:0/1472004392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cec056d60 0x559cec05b9f0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac009ca0 tx=0x7fd9ac03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53bcc095cb44c47 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.296+0000 7fd99f7fe640 1 -- 172.21.15.7:0/1472004392 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac05f020 con 0x559cec056d60 2024-06-10T01:19:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.296+0000 7fd99f7fe640 1 -- 172.21.15.7:0/1472004392 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9ac036040 con 0x559cec056d60 2024-06-10T01:19:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.296+0000 7fd99f7fe640 1 -- 172.21.15.7:0/1472004392 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac042030 con 0x559cec056d60 2024-06-10T01:19:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.296+0000 7fd9bebea740 1 -- 172.21.15.7:0/1472004392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cec056d60 msgr2=0x559cec05b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.296+0000 7fd9bebea740 1 --2- 172.21.15.7:0/1472004392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cec056d60 0x559cec05b9f0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac009ca0 tx=0x7fd9ac03f000 comp rx=0 tx=0).stop 2024-06-10T01:19:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.296+0000 7fd9bebea740 1 -- 172.21.15.7:0/1472004392 shutdown_connections 2024-06-10T01:19:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.296+0000 7fd9bebea740 1 --2- 172.21.15.7:0/1472004392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cec056d60 0x559cec05b9f0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.296+0000 7fd9bebea740 1 --2- 172.21.15.7:0/1472004392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559cec051bf0 0x559cec0629f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.296+0000 7fd9bebea740 1 --2- 172.21.15.7:0/1472004392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cec051200 0x559cec05b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.296+0000 7fd9bebea740 1 -- 172.21.15.7:0/1472004392 >> 172.21.15.7:0/1472004392 conn(0x559cec04c870 msgr2=0x559cec0564b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.296+0000 7fd9bebea740 1 -- 172.21.15.7:0/1472004392 shutdown_connections 2024-06-10T01:19:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.296+0000 7fd9bebea740 1 -- 172.21.15.7:0/1472004392 wait complete. 2024-06-10T01:19:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.297+0000 7fd9bebea740 1 -- start start 2024-06-10T01:19:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.297+0000 7fd9bebea740 1 Processor -- start 2024-06-10T01:19:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.297+0000 7fd9bebea740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cec051200 0x559cec0cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.297+0000 7fd9bebea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cec051bf0 0x559cec0cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.297+0000 7fd9bebea740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559cec056d60 0x559cec0d4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.297+0000 7fd9bebea740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41a89f0 con 0x559cec051200 2024-06-10T01:19:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.297+0000 7fd9bebea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41a9e60 con 0x559cec051bf0 2024-06-10T01:19:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.297+0000 7fd9bebea740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41a90c0 con 0x559cec056d60 2024-06-10T01:19:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.297+0000 7fd9bebea740 10 client.0 _ll_register_callbacks cb 0x7fd9a40d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:19:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.297+0000 7fd9b9b9e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cec051200 0x559cec0cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.297+0000 7fd9b9b9e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cec051200 0x559cec0cd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39312/0 (socket says 172.21.15.7:39312) 2024-06-10T01:19:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.297+0000 7fd9b9b9e640 1 -- 172.21.15.7:0/3388740895 learned_addr learned my addr 172.21.15.7:0/3388740895 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.297+0000 7fd9b939d640 1 --2- 172.21.15.7:0/3388740895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cec051bf0 0x559cec0cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.297+0000 7fd9ba39f640 1 --2- 172.21.15.7:0/3388740895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559cec056d60 0x559cec0d4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.297+0000 7fd9b9b9e640 1 -- 172.21.15.7:0/3388740895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559cec056d60 msgr2=0x559cec0d4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.297+0000 7fd9b9b9e640 1 --2- 172.21.15.7:0/3388740895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559cec056d60 0x559cec0d4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.297+0000 7fd9b9b9e640 1 -- 172.21.15.7:0/3388740895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cec051bf0 msgr2=0x559cec0cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.297+0000 7fd9b9b9e640 1 --2- 172.21.15.7:0/3388740895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cec051bf0 0x559cec0cd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.297+0000 7fd9b9b9e640 1 -- 172.21.15.7:0/3388740895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cec046ce0 con 0x559cec051200 2024-06-10T01:19:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.298+0000 7fd9b9b9e640 1 --2- 172.21.15.7:0/3388740895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cec051200 0x559cec0cd090 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0004700 tx=0x7fd9b0012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff3e2c2898abe64 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.298+0000 7fd99d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:19:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.298+0000 7fd99d7fa640 1 -- 172.21.15.7:0/3388740895 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b001f6c0 con 0x559cec051200 2024-06-10T01:19:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.298+0000 7fd99d7fa640 1 -- 172.21.15.7:0/3388740895 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fd9b001f860 con 0x559cec051200 2024-06-10T01:19:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.298+0000 7fd9bebea740 1 -- 172.21.15.7:0/3388740895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9ac036040 con 0x559cec051200 2024-06-10T01:19:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.298+0000 7fd99d7fa640 1 -- 172.21.15.7:0/3388740895 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd9b001fa60 con 0x559cec051200 2024-06-10T01:19:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.298+0000 7fd9bebea740 10 client.18888 fetch_fsmap learned FSMap version 42 2024-06-10T01:19:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.298+0000 7fd9bebea740 1 -- 172.21.15.7:0/3388740895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x559cec0d5e10 con 0x559cec051200 2024-06-10T01:19:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.298+0000 7fd99d7fa640 1 -- 172.21.15.7:0/3388740895 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9ac036040 con 0x559cec051200 2024-06-10T01:19:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.298+0000 7fd99d7fa640 1 -- 172.21.15.7:0/3388740895 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 42) ==== 8354+0+0 (secure 0 0 0) 0x7fd9b0026410 con 0x559cec051200 2024-06-10T01:19:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.298+0000 7fd9bebea740 10 client.18888 fetch_fsmap finished waiting for FSMap version 42 2024-06-10T01:19:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.298+0000 7fd9bebea740 10 client.18888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:19:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.298+0000 7fd9bebea740 20 client.18888 populate_metadata read hostname 'smithi007' 2024-06-10T01:19:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.299+0000 7fd9bebea740 1 --2- 172.21.15.7:0/3388740895 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559cec0d6840 0x559cec0d8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.299+0000 7fd9bebea740 4 client.18888 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:19:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.299+0000 7fd9bebea740 1 -- 172.21.15.7:0/3388740895 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559cebfc1c20 con 0x559cec0d6840 2024-06-10T01:19:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.299+0000 7fd9b939d640 1 --2- 172.21.15.7:0/3388740895 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559cec0d6840 0x559cec0d8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.299+0000 7fd99d7fa640 10 client.18888 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:30.299+0000 7fd9b939d640 1 --2- 172.21.15.7:0/3388740895 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559cec0d6840 0x559cec0d8c60 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c0873086d4ffef5 server_cookie=3863c09c58667fe1 in_seq=0 out_seq=0 2024-06-10T01:19:30.466 INFO:tasks.workunit.client.0.smithi007.stdout: 10 3638 54.98 MB/sec warmup 7 sec latency 148.011 ms 2024-06-10T01:19:31.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:30 smithi175 ceph-mon[36636]: pgmap v837: 129 pgs: 129 active+clean; 160 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 2.7 MiB/s wr, 214 op/s 2024-06-10T01:19:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:31.227+0000 7fd99d7fa640 1 -- 172.21.15.7:0/3388740895 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559cebfc1c20 con 0x559cec0d6840 2024-06-10T01:19:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:31.227+0000 7fd99d7fa640 10 client.18888 handle_command_reply: tid=0 2024-06-10T01:19:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:31.227+0000 7fd9bebea740 1 client.18888 shutdown 2024-06-10T01:19:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:31.227+0000 7fd9bebea740 1 -- 172.21.15.7:0/3388740895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cec051200 msgr2=0x559cec0cd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:31.227+0000 7fd9bebea740 1 --2- 172.21.15.7:0/3388740895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cec051200 0x559cec0cd090 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0004700 tx=0x7fd9b0012b00 comp rx=0 tx=0).stop 2024-06-10T01:19:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:31.227+0000 7fd9bebea740 1 -- 172.21.15.7:0/3388740895 shutdown_connections 2024-06-10T01:19:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:31.227+0000 7fd9bebea740 1 --2- 172.21.15.7:0/3388740895 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559cec0d6840 0x559cec0d8c60 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:31.227+0000 7fd9bebea740 1 --2- 172.21.15.7:0/3388740895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559cec056d60 0x559cec0d4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:31.227+0000 7fd9bebea740 1 --2- 172.21.15.7:0/3388740895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cec051bf0 0x559cec0cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:31.227+0000 7fd99d7fa640 0 client.18888 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:31.227+0000 7fd9bebea740 1 --2- 172.21.15.7:0/3388740895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cec051200 0x559cec0cd090 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:31.227+0000 7fd9bebea740 1 -- 172.21.15.7:0/3388740895 >> 172.21.15.7:0/3388740895 conn(0x559cec04c870 msgr2=0x559cec056070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:31.227+0000 7fd9bebea740 1 -- 172.21.15.7:0/3388740895 shutdown_connections 2024-06-10T01:19:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:31.227+0000 7fd9bebea740 1 -- 172.21.15.7:0/3388740895 wait complete. 2024-06-10T01:19:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:31.227+0000 7fd9bebea740 20 client.18888 trim_cache size 0 max 16384 2024-06-10T01:19:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:19:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:19:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:19:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:31.228+0000 7fd9baba0640 1 -- 172.21.15.7:0/3967650783 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd9a00a2b70 msgr2=0x7fd9a00a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:31.228+0000 7fd9baba0640 1 --2- 172.21.15.7:0/3967650783 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd9a00a2b70 0x7fd9a00a5030 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac04b000 tx=0x7fd9ac04c000 comp rx=0 tx=0).stop 2024-06-10T01:19:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:31.228+0000 7fd9baba0640 1 -- 172.21.15.7:0/3967650783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9b4101920 msgr2=0x7fd9b4198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:31.228+0000 7fd9baba0640 1 --2- 172.21.15.7:0/3967650783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9b4101920 0x7fd9b4198030 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fd9a400e9f0 tx=0x7fd9a400eeb0 comp rx=0 tx=0).stop 2024-06-10T01:19:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:31.228+0000 7fd9baba0640 1 -- 172.21.15.7:0/3967650783 shutdown_connections 2024-06-10T01:19:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:31.228+0000 7fd9baba0640 1 --2- 172.21.15.7:0/3967650783 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd9a00a2b70 0x7fd9a00a5030 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:31.228+0000 7fd9baba0640 1 --2- 172.21.15.7:0/3967650783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9b41022e0 0x7fd9b419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:31.228+0000 7fd9baba0640 1 --2- 172.21.15.7:0/3967650783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9b4101920 0x7fd9b4198030 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:31.228+0000 7fd9baba0640 1 --2- 172.21.15.7:0/3967650783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9b4100730 0x7fd9b4197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:31.228+0000 7fd9baba0640 1 -- 172.21.15.7:0/3967650783 >> 172.21.15.7:0/3967650783 conn(0x7fd9b40fbcc0 msgr2=0x7fd9b40fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:31.228+0000 7fd9baba0640 1 -- 172.21.15.7:0/3967650783 shutdown_connections 2024-06-10T01:19:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:31.228+0000 7fd9baba0640 1 -- 172.21.15.7:0/3967650783 wait complete. 2024-06-10T01:19:31.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:30 smithi007 ceph-mon[32398]: pgmap v837: 129 pgs: 129 active+clean; 160 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 2.7 MiB/s wr, 214 op/s 2024-06-10T01:19:31.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:30 smithi078 ceph-mon[36683]: pgmap v837: 129 pgs: 129 active+clean; 160 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 2.7 MiB/s wr, 214 op/s 2024-06-10T01:19:31.466 INFO:tasks.workunit.client.0.smithi007.stdout: 10 3732 48.92 MB/sec warmup 8 sec latency 409.600 ms 2024-06-10T01:19:32.466 INFO:tasks.workunit.client.0.smithi007.stdout: 10 3924 44.00 MB/sec warmup 9 sec latency 82.782 ms 2024-06-10T01:19:33.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:32 smithi175 ceph-mon[36636]: pgmap v838: 129 pgs: 129 active+clean; 162 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 2.5 MiB/s wr, 259 op/s 2024-06-10T01:19:33.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:32 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:19:33.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:32 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:19:33.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:32 smithi007 ceph-mon[32398]: pgmap v838: 129 pgs: 129 active+clean; 162 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 2.5 MiB/s wr, 259 op/s 2024-06-10T01:19:33.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:32 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:19:33.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:32 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:19:33.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:32 smithi078 ceph-mon[36683]: pgmap v838: 129 pgs: 129 active+clean; 162 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 2.5 MiB/s wr, 259 op/s 2024-06-10T01:19:33.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:32 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:19:33.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:32 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:19:33.466 INFO:tasks.workunit.client.0.smithi007.stdout: 10 4128 39.97 MB/sec warmup 10 sec latency 278.863 ms 2024-06-10T01:19:34.467 INFO:tasks.workunit.client.0.smithi007.stdout: 10 4392 36.99 MB/sec warmup 11 sec latency 296.587 ms 2024-06-10T01:19:35.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:34 smithi175 ceph-mon[36636]: pgmap v839: 129 pgs: 129 active+clean; 161 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 3.1 MiB/s wr, 451 op/s 2024-06-10T01:19:35.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:34 smithi007 ceph-mon[32398]: pgmap v839: 129 pgs: 129 active+clean; 161 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 3.1 MiB/s wr, 451 op/s 2024-06-10T01:19:35.331 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:34 smithi078 ceph-mon[36683]: pgmap v839: 129 pgs: 129 active+clean; 161 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 3.1 MiB/s wr, 451 op/s 2024-06-10T01:19:35.466 INFO:tasks.workunit.client.0.smithi007.stdout: 10 4878 37.19 MB/sec warmup 12 sec latency 64.329 ms 2024-06-10T01:19:36.467 INFO:tasks.workunit.client.0.smithi007.stdout: 10 5645 35.23 MB/sec warmup 13 sec latency 56.620 ms 2024-06-10T01:19:37.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:36 smithi175 ceph-mon[36636]: pgmap v840: 129 pgs: 129 active+clean; 165 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 3.5 MiB/s wr, 617 op/s 2024-06-10T01:19:37.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:36 smithi007 ceph-mon[32398]: pgmap v840: 129 pgs: 129 active+clean; 165 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 3.5 MiB/s wr, 617 op/s 2024-06-10T01:19:37.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:36 smithi078 ceph-mon[36683]: pgmap v840: 129 pgs: 129 active+clean; 165 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 3.5 MiB/s wr, 617 op/s 2024-06-10T01:19:37.466 INFO:tasks.workunit.client.0.smithi007.stdout: 10 6011 34.44 MB/sec warmup 14 sec latency 111.433 ms 2024-06-10T01:19:38.467 INFO:tasks.workunit.client.0.smithi007.stdout: 10 6671 35.50 MB/sec warmup 15 sec latency 98.235 ms 2024-06-10T01:19:39.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:38 smithi175 ceph-mon[36636]: pgmap v841: 129 pgs: 129 active+clean; 170 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 644 op/s 2024-06-10T01:19:39.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:38 smithi007 ceph-mon[32398]: pgmap v841: 129 pgs: 129 active+clean; 170 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 644 op/s 2024-06-10T01:19:39.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:38 smithi078 ceph-mon[36683]: pgmap v841: 129 pgs: 129 active+clean; 170 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 644 op/s 2024-06-10T01:19:39.467 INFO:tasks.workunit.client.0.smithi007.stdout: 10 7006 35.37 MB/sec warmup 16 sec latency 256.303 ms 2024-06-10T01:19:40.467 INFO:tasks.workunit.client.0.smithi007.stdout: 10 7154 33.84 MB/sec warmup 17 sec latency 635.813 ms 2024-06-10T01:19:41.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:40 smithi175 ceph-mon[36636]: pgmap v842: 129 pgs: 129 active+clean; 198 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.4 MiB/s wr, 687 op/s 2024-06-10T01:19:41.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:40 smithi007 ceph-mon[32398]: pgmap v842: 129 pgs: 129 active+clean; 198 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.4 MiB/s wr, 687 op/s 2024-06-10T01:19:41.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:40 smithi078 ceph-mon[36683]: pgmap v842: 129 pgs: 129 active+clean; 198 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.4 MiB/s wr, 687 op/s 2024-06-10T01:19:41.467 INFO:tasks.workunit.client.0.smithi007.stdout: 10 7372 32.20 MB/sec warmup 18 sec latency 70.186 ms 2024-06-10T01:19:42.467 INFO:tasks.workunit.client.0.smithi007.stdout: 10 7522 30.56 MB/sec warmup 19 sec latency 121.384 ms 2024-06-10T01:19:43.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:42 smithi175 ceph-mon[36636]: pgmap v843: 129 pgs: 129 active+clean; 232 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.4 MiB/s wr, 625 op/s 2024-06-10T01:19:43.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:42 smithi007 ceph-mon[32398]: pgmap v843: 129 pgs: 129 active+clean; 232 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.4 MiB/s wr, 625 op/s 2024-06-10T01:19:43.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:42 smithi078 ceph-mon[36683]: pgmap v843: 129 pgs: 129 active+clean; 232 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.4 MiB/s wr, 625 op/s 2024-06-10T01:19:43.444 INFO:tasks.quiescer.fs.[cephfs]:Total duration (55.3) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-06-10T01:19:43.444 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.65 2024-06-10T01:19:43.445 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:19:43.467 INFO:tasks.workunit.client.0.smithi007.stdout: 10 7798 29.53 MB/sec warmup 20 sec latency 230.570 ms 2024-06-10T01:19:43.767 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:19:44.168 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:43 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:19:44.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:43 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:19:44.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:43 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:19:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.397+0000 7fdb09ca5640 1 -- 172.21.15.7:0/4100941176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb041080a0 msgr2=0x7fdb041084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.397+0000 7fdb09ca5640 1 --2- 172.21.15.7:0/4100941176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb041080a0 0x7fdb041084c0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009900 tx=0x7fdaec02a420 comp rx=0 tx=0).stop 2024-06-10T01:19:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.400+0000 7fdb09ca5640 1 -- 172.21.15.7:0/4100941176 shutdown_connections 2024-06-10T01:19:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.400+0000 7fdb09ca5640 1 --2- 172.21.15.7:0/4100941176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb04109c00 0x7fdb04110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.400+0000 7fdb09ca5640 1 --2- 172.21.15.7:0/4100941176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb04109240 0x7fdb041096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.400+0000 7fdb09ca5640 1 --2- 172.21.15.7:0/4100941176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb041080a0 0x7fdb041084c0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.400+0000 7fdb09ca5640 1 -- 172.21.15.7:0/4100941176 >> 172.21.15.7:0/4100941176 conn(0x7fdb04076ff0 msgr2=0x7fdb04079430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.405+0000 7fdb09ca5640 1 -- 172.21.15.7:0/4100941176 shutdown_connections 2024-06-10T01:19:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.405+0000 7fdb09ca5640 1 -- 172.21.15.7:0/4100941176 wait complete. 2024-06-10T01:19:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.407+0000 7fdb09ca5640 1 Processor -- start 2024-06-10T01:19:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.407+0000 7fdb09ca5640 1 -- start start 2024-06-10T01:19:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.407+0000 7fdb09ca5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb041080a0 0x7fdb0407f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.407+0000 7fdb09ca5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb04109240 0x7fdb0407f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.407+0000 7fdb09ca5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb04109c00 0x7fdb0406f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.407+0000 7fdb09ca5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdb04113cf0 con 0x7fdb04109240 2024-06-10T01:19:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.407+0000 7fdb09ca5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb04113b70 con 0x7fdb041080a0 2024-06-10T01:19:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.407+0000 7fdb09ca5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdb04113e70 con 0x7fdb04109c00 2024-06-10T01:19:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.408+0000 7fdb03fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb04109c00 0x7fdb0406f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.408+0000 7fdb03fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb04109c00 0x7fdb0406f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47974/0 (socket says 172.21.15.7:47974) 2024-06-10T01:19:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.408+0000 7fdb03fff640 1 -- 172.21.15.7:0/703392166 learned_addr learned my addr 172.21.15.7:0/703392166 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.408+0000 7fdb02ffd640 1 --2- 172.21.15.7:0/703392166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb04109240 0x7fdb0407f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.408+0000 7fdb037fe640 1 --2- 172.21.15.7:0/703392166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb041080a0 0x7fdb0407f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.408+0000 7fdb03fff640 1 -- 172.21.15.7:0/703392166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb041080a0 msgr2=0x7fdb0407f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.408+0000 7fdb03fff640 1 --2- 172.21.15.7:0/703392166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb041080a0 0x7fdb0407f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.408+0000 7fdb03fff640 1 -- 172.21.15.7:0/703392166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb04109240 msgr2=0x7fdb0407f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.408+0000 7fdb03fff640 1 --2- 172.21.15.7:0/703392166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb04109240 0x7fdb0407f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.408+0000 7fdb03fff640 1 -- 172.21.15.7:0/703392166 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb0406f920 con 0x7fdb04109c00 2024-06-10T01:19:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.409+0000 7fdb03fff640 1 --2- 172.21.15.7:0/703392166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb04109c00 0x7fdb0406f3e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdafc00eaa0 tx=0x7fdafc00ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.413+0000 7fdb00ff9640 1 -- 172.21.15.7:0/703392166 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc00cdb0 con 0x7fdb04109c00 2024-06-10T01:19:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.413+0000 7fdb09ca5640 1 -- 172.21.15.7:0/703392166 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb0406fc10 con 0x7fdb04109c00 2024-06-10T01:19:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.413+0000 7fdb09ca5640 1 -- 172.21.15.7:0/703392166 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb04070160 con 0x7fdb04109c00 2024-06-10T01:19:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.413+0000 7fdb00ff9640 1 -- 172.21.15.7:0/703392166 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdafc005020 con 0x7fdb04109c00 2024-06-10T01:19:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.413+0000 7fdb00ff9640 1 -- 172.21.15.7:0/703392166 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0116b0 con 0x7fdb04109c00 2024-06-10T01:19:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.419+0000 7fdb0c9bc740 1 Processor -- start 2024-06-10T01:19:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.419+0000 7fdb0c9bc740 1 -- start start 2024-06-10T01:19:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.420+0000 7fdb0c9bc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557acefdc010 0x557acefdc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.420+0000 7fdb0c9bc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557acefdca00 0x557acefe1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.420+0000 7fdb00ff9640 1 -- 172.21.15.7:0/703392166 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fdafc004a40 con 0x7fdb04109c00 2024-06-10T01:19:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.420+0000 7fdb0c9bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557acefe1b70 0x557acefe4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.420+0000 7fdb0c9bc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557acef3f4f0 con 0x557acefdca00 2024-06-10T01:19:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.420+0000 7fdb0c9bc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557aceedd870 con 0x557acefe1b70 2024-06-10T01:19:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.420+0000 7fdb0c9bc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557acef3b4b0 con 0x557acefdc010 2024-06-10T01:19:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.420+0000 7fdb00ff9640 1 --2- 172.21.15.7:0/703392166 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdaf00a2d90 0x7fdaf00a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.421+0000 7fdb03fff640 1 --2- 172.21.15.7:0/703392166 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdaf00a2d90 0x7fdaf00a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.421+0000 7fdb00ff9640 1 -- 172.21.15.7:0/703392166 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fdafc0cf500 con 0x7fdb04109c00 2024-06-10T01:19:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.421+0000 7fdb03fff640 1 --2- 172.21.15.7:0/703392166 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdaf00a2d90 0x7fdaf00a5250 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fdafc09b680 tx=0x7fdafc097580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.424+0000 7fdb02ffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557acefdc010 0x557acefdc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.424+0000 7fdb02ffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557acefdc010 0x557acefdc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47988/0 (socket says 172.21.15.7:47988) 2024-06-10T01:19:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.424+0000 7fdb02ffd640 1 -- 172.21.15.7:0/2096825343 learned_addr learned my addr 172.21.15.7:0/2096825343 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.424+0000 7fdb02ffd640 1 --2- 172.21.15.7:0/2096825343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557acefe1b70 0x557acefe4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.426+0000 7fdb037fe640 1 --2- 172.21.15.7:0/2096825343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557acefdca00 0x557acefe1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.428+0000 7fdb02ffd640 1 -- 172.21.15.7:0/2096825343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557acefe1b70 msgr2=0x557acefe4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.428+0000 7fdb02ffd640 1 --2- 172.21.15.7:0/2096825343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557acefe1b70 0x557acefe4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.428+0000 7fdb02ffd640 1 -- 172.21.15.7:0/2096825343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557acefdca00 msgr2=0x557acefe1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.428+0000 7fdb02ffd640 1 --2- 172.21.15.7:0/2096825343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557acefdca00 0x557acefe1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.428+0000 7fdb02ffd640 1 -- 172.21.15.7:0/2096825343 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557aceef99b0 con 0x557acefdc010 2024-06-10T01:19:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.428+0000 7fdb02ffd640 1 --2- 172.21.15.7:0/2096825343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557acefdc010 0x557acefdc430 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdaf40149c0 tx=0x7fdaf400c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58f4bc1272fd36b1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.432+0000 7fdaf9ffb640 1 -- 172.21.15.7:0/2096825343 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf40142d0 con 0x557acefdc010 2024-06-10T01:19:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.432+0000 7fdaf9ffb640 1 -- 172.21.15.7:0/2096825343 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaf400fc40 con 0x557acefdc010 2024-06-10T01:19:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.433+0000 7fdb0c9bc740 1 -- 172.21.15.7:0/2096825343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557acefdc010 msgr2=0x557acefdc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.433+0000 7fdb0c9bc740 1 --2- 172.21.15.7:0/2096825343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557acefdc010 0x557acefdc430 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdaf40149c0 tx=0x7fdaf400c4d0 comp rx=0 tx=0).stop 2024-06-10T01:19:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.436+0000 7fdb0c9bc740 1 -- 172.21.15.7:0/2096825343 shutdown_connections 2024-06-10T01:19:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.436+0000 7fdb0c9bc740 1 --2- 172.21.15.7:0/2096825343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557acefe1b70 0x557acefe4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.436+0000 7fdb0c9bc740 1 --2- 172.21.15.7:0/2096825343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557acefdca00 0x557acefe1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.436+0000 7fdb0c9bc740 1 --2- 172.21.15.7:0/2096825343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557acefdc010 0x557acefdc430 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.436+0000 7fdb0c9bc740 1 -- 172.21.15.7:0/2096825343 >> 172.21.15.7:0/2096825343 conn(0x557acefd7680 msgr2=0x557acefd7a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.438+0000 7fdb0c9bc740 1 -- 172.21.15.7:0/2096825343 shutdown_connections 2024-06-10T01:19:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.438+0000 7fdb0c9bc740 1 -- 172.21.15.7:0/2096825343 wait complete. 2024-06-10T01:19:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.439+0000 7fdb0c9bc740 1 -- start start 2024-06-10T01:19:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.439+0000 7fdb0c9bc740 1 Processor -- start 2024-06-10T01:19:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.439+0000 7fdb0c9bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557acefdc010 0x557acefe6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.440+0000 7fdb0c9bc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557acefdca00 0x557acefe6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.440+0000 7fdb0c9bc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557acefe1b70 0x557acefed7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.440+0000 7fdb0c9bc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557acefe4b70 con 0x557acefdca00 2024-06-10T01:19:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.440+0000 7fdb0c9bc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557aceec8ae0 con 0x557acefdc010 2024-06-10T01:19:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.440+0000 7fdb0c9bc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557acefe4cf0 con 0x557acefe1b70 2024-06-10T01:19:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.440+0000 7fdb0c9bc740 10 client.0 _ll_register_callbacks cb 0x557acef019b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:19:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.440+0000 7fdb02ffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557acefe1b70 0x557acefed7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.440+0000 7fdb02ffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557acefe1b70 0x557acefed7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47994/0 (socket says 172.21.15.7:47994) 2024-06-10T01:19:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.440+0000 7fdb02ffd640 1 -- 172.21.15.7:0/559204076 learned_addr learned my addr 172.21.15.7:0/559204076 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.440+0000 7fdb02ffd640 1 --2- 172.21.15.7:0/559204076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557acefdc010 0x557acefe6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.441+0000 7fdb02ffd640 1 -- 172.21.15.7:0/559204076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557acefdc010 msgr2=0x557acefe6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.441+0000 7fdb02ffd640 1 --2- 172.21.15.7:0/559204076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557acefdc010 0x557acefe6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.441+0000 7fdb02ffd640 1 -- 172.21.15.7:0/559204076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557acefdca00 msgr2=0x557acefe6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.441+0000 7fdb02ffd640 1 --2- 172.21.15.7:0/559204076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557acefdca00 0x557acefe6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.441+0000 7fdb02ffd640 1 -- 172.21.15.7:0/559204076 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557acec53210 con 0x557acefe1b70 2024-06-10T01:19:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.443+0000 7fdb02ffd640 1 --2- 172.21.15.7:0/559204076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557acefe1b70 0x557acefed7b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdaf40058c0 tx=0x7fdaf4005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53d97409833dc3da server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.444+0000 7fdae2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:19:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.444+0000 7fdae2ffd640 1 -- 172.21.15.7:0/559204076 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf40140d0 con 0x557acefe1b70 2024-06-10T01:19:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.444+0000 7fdae2ffd640 1 -- 172.21.15.7:0/559204076 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fdaf4014270 con 0x557acefe1b70 2024-06-10T01:19:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.446+0000 7fdb0c9bc740 1 -- 172.21.15.7:0/559204076 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaf400fc40 con 0x557acefe1b70 2024-06-10T01:19:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.450+0000 7fdae2ffd640 1 -- 172.21.15.7:0/559204076 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fdaf400f730 con 0x557acefe1b70 2024-06-10T01:19:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.456+0000 7fdb0c9bc740 10 client.28228 fetch_fsmap learned FSMap version 43 2024-06-10T01:19:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.456+0000 7fdb0c9bc740 1 -- 172.21.15.7:0/559204076 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x557aced1f620 con 0x557acefe1b70 2024-06-10T01:19:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.465+0000 7fdae2ffd640 1 -- 172.21.15.7:0/559204076 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaf400fc40 con 0x557acefe1b70 2024-06-10T01:19:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.465+0000 7fdae2ffd640 1 -- 172.21.15.7:0/559204076 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 43) ==== 8358+0+0 (secure 0 0 0) 0x7fdaf4036020 con 0x557acefe1b70 2024-06-10T01:19:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.466+0000 7fdb0c9bc740 10 client.28228 fetch_fsmap finished waiting for FSMap version 43 2024-06-10T01:19:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.466+0000 7fdb0c9bc740 10 client.28228 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:19:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.466+0000 7fdb0c9bc740 20 client.28228 populate_metadata read hostname 'smithi007' 2024-06-10T01:19:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.466+0000 7fdb0c9bc740 1 --2- 172.21.15.7:0/559204076 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557aceff1e90 0x557aceff4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.466+0000 7fdb0c9bc740 4 client.28228 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:19:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.466+0000 7fdb0c9bc740 1 -- 172.21.15.7:0/559204076 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557acef52f10 con 0x557aceff1e90 2024-06-10T01:19:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.469+0000 7fdb037fe640 1 --2- 172.21.15.7:0/559204076 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557aceff1e90 0x557aceff4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:44.467 INFO:tasks.workunit.client.0.smithi007.stdout: 10 8396 29.38 MB/sec warmup 21 sec latency 270.557 ms 2024-06-10T01:19:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.474+0000 7fdb037fe640 1 --2- 172.21.15.7:0/559204076 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557aceff1e90 0x557aceff4330 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41ccc613ce8e863a server_cookie=7d2a52b75691982d in_seq=0 out_seq=0 2024-06-10T01:19:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.478+0000 7fdae2ffd640 10 client.28228 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.557+0000 7fdae2ffd640 1 -- 172.21.15.7:0/559204076 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557acef52f10 con 0x557aceff1e90 2024-06-10T01:19:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.558+0000 7fdae2ffd640 10 client.28228 handle_command_reply: tid=0 2024-06-10T01:19:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.558+0000 7fdb0c9bc740 1 client.28228 shutdown 2024-06-10T01:19:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.558+0000 7fdb0c9bc740 1 -- 172.21.15.7:0/559204076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557acefe1b70 msgr2=0x557acefed7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.558+0000 7fdb0c9bc740 1 --2- 172.21.15.7:0/559204076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557acefe1b70 0x557acefed7b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdaf40058c0 tx=0x7fdaf4005850 comp rx=0 tx=0).stop 2024-06-10T01:19:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.558+0000 7fdb0c9bc740 1 -- 172.21.15.7:0/559204076 shutdown_connections 2024-06-10T01:19:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.558+0000 7fdb0c9bc740 1 --2- 172.21.15.7:0/559204076 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557aceff1e90 0x557aceff4330 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.558+0000 7fdb0c9bc740 1 --2- 172.21.15.7:0/559204076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557acefe1b70 0x557acefed7b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.558+0000 7fdb0c9bc740 1 --2- 172.21.15.7:0/559204076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557acefdca00 0x557acefe6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.558+0000 7fdb0c9bc740 1 --2- 172.21.15.7:0/559204076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557acefdc010 0x557acefe6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.558+0000 7fdb0c9bc740 1 -- 172.21.15.7:0/559204076 >> 172.21.15.7:0/559204076 conn(0x557acefd7680 msgr2=0x557acefd67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.558+0000 7fdae2ffd640 0 client.28228 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.558+0000 7fdb0c9bc740 1 -- 172.21.15.7:0/559204076 shutdown_connections 2024-06-10T01:19:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.558+0000 7fdb0c9bc740 1 -- 172.21.15.7:0/559204076 wait complete. 2024-06-10T01:19:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.559+0000 7fdb0c9bc740 20 client.28228 trim_cache size 0 max 16384 2024-06-10T01:19:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.593+0000 7fdb0c9bc740 1 Processor -- start 2024-06-10T01:19:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.593+0000 7fdb0c9bc740 1 -- start start 2024-06-10T01:19:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.594+0000 7fdb0c9bc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557acefdc010 0x557acefe6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.594+0000 7fdb0c9bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557acefdca00 0x557acefe6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.594+0000 7fdb0c9bc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557acefe1b70 0x557acefed740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.594+0000 7fdb0c9bc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdae40125d0 con 0x557acefe1b70 2024-06-10T01:19:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.594+0000 7fdb0c9bc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdae400fe40 con 0x557acefdca00 2024-06-10T01:19:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.594+0000 7fdb0c9bc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdae400dc90 con 0x557acefdc010 2024-06-10T01:19:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.594+0000 7fdb037fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557acefdca00 0x557acefe6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.594+0000 7fdb037fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557acefdca00 0x557acefe6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:59078/0 (socket says 172.21.15.7:59078) 2024-06-10T01:19:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.594+0000 7fdb037fe640 1 -- 172.21.15.7:0/3092980621 learned_addr learned my addr 172.21.15.7:0/3092980621 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.596+0000 7fdb037fe640 1 -- 172.21.15.7:0/3092980621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557acefdc010 msgr2=0x557acefe6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.596+0000 7fdb037fe640 1 --2- 172.21.15.7:0/3092980621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557acefdc010 0x557acefe6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.596+0000 7fdb037fe640 1 -- 172.21.15.7:0/3092980621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557acefe1b70 msgr2=0x557acefed740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.596+0000 7fdb037fe640 1 --2- 172.21.15.7:0/3092980621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557acefe1b70 0x557acefed740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.596+0000 7fdb037fe640 1 -- 172.21.15.7:0/3092980621 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557acefe0e10 con 0x557acefdca00 2024-06-10T01:19:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.599+0000 7fdb037fe640 1 --2- 172.21.15.7:0/3092980621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557acefdca00 0x557acefe6770 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009230 tx=0x7fdaec009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5592612f240c82f5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.600+0000 7fdaf97fa640 1 -- 172.21.15.7:0/3092980621 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec035040 con 0x557acefdca00 2024-06-10T01:19:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.600+0000 7fdaf97fa640 1 -- 172.21.15.7:0/3092980621 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaec005500 con 0x557acefdca00 2024-06-10T01:19:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.600+0000 7fdaf97fa640 1 -- 172.21.15.7:0/3092980621 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec0316b0 con 0x557acefdca00 2024-06-10T01:19:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.601+0000 7fdb0c9bc740 1 -- 172.21.15.7:0/3092980621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557acefdca00 msgr2=0x557acefe6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.601+0000 7fdb0c9bc740 1 --2- 172.21.15.7:0/3092980621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557acefdca00 0x557acefe6770 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009230 tx=0x7fdaec009260 comp rx=0 tx=0).stop 2024-06-10T01:19:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.601+0000 7fdb0c9bc740 1 -- 172.21.15.7:0/3092980621 shutdown_connections 2024-06-10T01:19:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.601+0000 7fdb0c9bc740 1 --2- 172.21.15.7:0/3092980621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557acefe1b70 0x557acefed740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.601+0000 7fdb0c9bc740 1 --2- 172.21.15.7:0/3092980621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557acefdca00 0x557acefe6770 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.601+0000 7fdb0c9bc740 1 --2- 172.21.15.7:0/3092980621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557acefdc010 0x557acefe6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.601+0000 7fdb0c9bc740 1 -- 172.21.15.7:0/3092980621 >> 172.21.15.7:0/3092980621 conn(0x557acefd7680 msgr2=0x557acefd7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.604+0000 7fdb0c9bc740 1 -- 172.21.15.7:0/3092980621 shutdown_connections 2024-06-10T01:19:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.604+0000 7fdb0c9bc740 1 -- 172.21.15.7:0/3092980621 wait complete. 2024-06-10T01:19:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.605+0000 7fdb0c9bc740 1 -- start start 2024-06-10T01:19:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.605+0000 7fdb0c9bc740 1 Processor -- start 2024-06-10T01:19:44.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.606+0000 7fdb0c9bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557acefdc010 0x557acefed460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.606+0000 7fdb0c9bc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557acefdca00 0x557acefea060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.606+0000 7fdb02ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557acefdc010 0x557acefed460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.606+0000 7fdb02ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557acefdc010 0x557acefed460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:59092/0 (socket says 172.21.15.7:59092) 2024-06-10T01:19:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.606+0000 7fdb02ffd640 1 -- 172.21.15.7:0/873774572 learned_addr learned my addr 172.21.15.7:0/873774572 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.606+0000 7fdb0c9bc740 1 --2- 172.21.15.7:0/873774572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557acefe1b70 0x557acefea5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.607+0000 7fdb0c9bc740 1 -- 172.21.15.7:0/873774572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdae400e300 con 0x557acefe1b70 2024-06-10T01:19:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.607+0000 7fdb0c9bc740 1 -- 172.21.15.7:0/873774572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdae400f770 con 0x557acefdc010 2024-06-10T01:19:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.607+0000 7fdb0c9bc740 1 -- 172.21.15.7:0/873774572 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdae400e9d0 con 0x557acefdca00 2024-06-10T01:19:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.607+0000 7fdb0c9bc740 10 client.0 _ll_register_callbacks cb 0x7fdaf400cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:19:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.608+0000 7fdb02ffd640 1 --2- 172.21.15.7:0/873774572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557acefe1b70 0x557acefea5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.608+0000 7fdb02ffd640 1 -- 172.21.15.7:0/873774572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557acefdca00 msgr2=0x557acefea060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.608+0000 7fdb02ffd640 1 --2- 172.21.15.7:0/873774572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557acefdca00 0x557acefea060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.608+0000 7fdb02ffd640 1 -- 172.21.15.7:0/873774572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557acefe1b70 msgr2=0x557acefea5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.608+0000 7fdb02ffd640 1 --2- 172.21.15.7:0/873774572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557acefe1b70 0x557acefea5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.608+0000 7fdb02ffd640 1 -- 172.21.15.7:0/873774572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557acefeaae0 con 0x557acefdc010 2024-06-10T01:19:44.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.610+0000 7fdb02ffd640 1 --2- 172.21.15.7:0/873774572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557acefdc010 0x557acefed460 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400cb00 tx=0x7fdaf4022040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb0b2e6eca020cbb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:44.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.610+0000 7fdae2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:19:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.612+0000 7fdae2ffd640 1 -- 172.21.15.7:0/873774572 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf40035f0 con 0x557acefdc010 2024-06-10T01:19:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.612+0000 7fdae2ffd640 1 -- 172.21.15.7:0/873774572 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fdaf4029070 con 0x557acefdc010 2024-06-10T01:19:44.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.613+0000 7fdb0c9bc740 1 -- 172.21.15.7:0/873774572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaec005500 con 0x557acefdc010 2024-06-10T01:19:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.614+0000 7fdae2ffd640 1 -- 172.21.15.7:0/873774572 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fdaf4012450 con 0x557acefdc010 2024-06-10T01:19:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.615+0000 7fdb0c9bc740 10 client.28205 fetch_fsmap learned FSMap version 43 2024-06-10T01:19:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.615+0000 7fdb0c9bc740 1 -- 172.21.15.7:0/873774572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x557acf05bbd0 con 0x557acefdc010 2024-06-10T01:19:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.617+0000 7fdae2ffd640 1 -- 172.21.15.7:0/873774572 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaf40125f0 con 0x557acefdc010 2024-06-10T01:19:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.617+0000 7fdae2ffd640 1 -- 172.21.15.7:0/873774572 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) ==== 8358+0+0 (secure 0 0 0) 0x7fdaf4036040 con 0x557acefdc010 2024-06-10T01:19:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.617+0000 7fdb0c9bc740 10 client.28205 fetch_fsmap finished waiting for FSMap version 43 2024-06-10T01:19:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.617+0000 7fdb0c9bc740 10 client.28205 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:19:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.617+0000 7fdb0c9bc740 20 client.28205 populate_metadata read hostname 'smithi007' 2024-06-10T01:19:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.617+0000 7fdb0c9bc740 1 --2- 172.21.15.7:0/873774572 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557acf05c5e0 0x557acf05ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:44.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.617+0000 7fdb0c9bc740 4 client.28205 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:19:44.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.617+0000 7fdb0c9bc740 1 -- 172.21.15.7:0/873774572 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fdaec005500 con 0x557acf05c5e0 2024-06-10T01:19:44.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.618+0000 7fdb037fe640 1 --2- 172.21.15.7:0/873774572 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557acf05c5e0 0x557acf05ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:44.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.625+0000 7fdb037fe640 1 --2- 172.21.15.7:0/873774572 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557acf05c5e0 0x557acf05ea00 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d10e23f4aa9ad63 server_cookie=9fe713d177f3e89 in_seq=0 out_seq=0 2024-06-10T01:19:44.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:44.625+0000 7fdae2ffd640 10 client.28205 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:45.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:44 smithi175 ceph-mon[36636]: pgmap v844: 129 pgs: 129 active+clean; 312 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 16 MiB/s wr, 615 op/s 2024-06-10T01:19:45.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:44 smithi007 ceph-mon[32398]: pgmap v844: 129 pgs: 129 active+clean; 312 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 16 MiB/s wr, 615 op/s 2024-06-10T01:19:45.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:44 smithi078 ceph-mon[36683]: pgmap v844: 129 pgs: 129 active+clean; 312 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 16 MiB/s wr, 615 op/s 2024-06-10T01:19:45.467 INFO:tasks.workunit.client.0.smithi007.stdout: 10 8496 28.20 MB/sec warmup 22 sec latency 719.483 ms 2024-06-10T01:19:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:46.366+0000 7fdae2ffd640 1 -- 172.21.15.7:0/873774572 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fdaec005500 con 0x557acf05c5e0 2024-06-10T01:19:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:46.367+0000 7fdae2ffd640 10 client.28205 handle_command_reply: tid=0 2024-06-10T01:19:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:46.367+0000 7fdb0c9bc740 1 client.28205 shutdown 2024-06-10T01:19:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:46.367+0000 7fdb0c9bc740 1 -- 172.21.15.7:0/873774572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557acefdc010 msgr2=0x557acefed460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:46.367+0000 7fdb0c9bc740 1 --2- 172.21.15.7:0/873774572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557acefdc010 0x557acefed460 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400cb00 tx=0x7fdaf4022040 comp rx=0 tx=0).stop 2024-06-10T01:19:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:46.367+0000 7fdb0c9bc740 1 -- 172.21.15.7:0/873774572 shutdown_connections 2024-06-10T01:19:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:46.367+0000 7fdb0c9bc740 1 --2- 172.21.15.7:0/873774572 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557acf05c5e0 0x557acf05ea00 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:46.367+0000 7fdb0c9bc740 1 --2- 172.21.15.7:0/873774572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557acefe1b70 0x557acefea5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:46.368+0000 7fdb0c9bc740 1 --2- 172.21.15.7:0/873774572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557acefdca00 0x557acefea060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:46.368+0000 7fdb0c9bc740 1 --2- 172.21.15.7:0/873774572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557acefdc010 0x557acefed460 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:46.368+0000 7fdae2ffd640 0 client.28205 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:46.368+0000 7fdb0c9bc740 1 -- 172.21.15.7:0/873774572 >> 172.21.15.7:0/873774572 conn(0x557acefd7680 msgr2=0x557acefe6210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:46.368+0000 7fdb0c9bc740 1 -- 172.21.15.7:0/873774572 shutdown_connections 2024-06-10T01:19:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:46.368+0000 7fdb0c9bc740 1 -- 172.21.15.7:0/873774572 wait complete. 2024-06-10T01:19:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:46.368+0000 7fdb0c9bc740 20 client.28205 trim_cache size 0 max 16384 2024-06-10T01:19:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:19:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:19:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:19:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:46.369+0000 7fdb09ca5640 1 -- 172.21.15.7:0/703392166 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdaf00a2d90 msgr2=0x7fdaf00a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:46.369+0000 7fdb09ca5640 1 --2- 172.21.15.7:0/703392166 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdaf00a2d90 0x7fdaf00a5250 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fdafc09b680 tx=0x7fdafc097580 comp rx=0 tx=0).stop 2024-06-10T01:19:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:46.369+0000 7fdb09ca5640 1 -- 172.21.15.7:0/703392166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb04109c00 msgr2=0x7fdb0406f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:46.369+0000 7fdb09ca5640 1 --2- 172.21.15.7:0/703392166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb04109c00 0x7fdb0406f3e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdafc00eaa0 tx=0x7fdafc00ef60 comp rx=0 tx=0).stop 2024-06-10T01:19:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:46.369+0000 7fdb09ca5640 1 -- 172.21.15.7:0/703392166 shutdown_connections 2024-06-10T01:19:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:46.369+0000 7fdb09ca5640 1 --2- 172.21.15.7:0/703392166 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdaf00a2d90 0x7fdaf00a5250 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:46.369+0000 7fdb09ca5640 1 --2- 172.21.15.7:0/703392166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb04109c00 0x7fdb0406f3e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:46.369+0000 7fdb09ca5640 1 --2- 172.21.15.7:0/703392166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb04109240 0x7fdb0407f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:46.369+0000 7fdb09ca5640 1 --2- 172.21.15.7:0/703392166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb041080a0 0x7fdb0407f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:46.369+0000 7fdb09ca5640 1 -- 172.21.15.7:0/703392166 >> 172.21.15.7:0/703392166 conn(0x7fdb04076ff0 msgr2=0x7fdb0410c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:46.369+0000 7fdb09ca5640 1 -- 172.21.15.7:0/703392166 shutdown_connections 2024-06-10T01:19:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:46.370+0000 7fdb09ca5640 1 -- 172.21.15.7:0/703392166 wait complete. 2024-06-10T01:19:46.468 INFO:tasks.workunit.client.0.smithi007.stdout: 10 8729 28.06 MB/sec warmup 23 sec latency 664.943 ms 2024-06-10T01:19:47.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:46 smithi175 ceph-mon[36636]: pgmap v845: 129 pgs: 129 active+clean; 339 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 18 MiB/s wr, 474 op/s 2024-06-10T01:19:47.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:46 smithi007 ceph-mon[32398]: pgmap v845: 129 pgs: 129 active+clean; 339 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 18 MiB/s wr, 474 op/s 2024-06-10T01:19:47.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:46 smithi078 ceph-mon[36683]: pgmap v845: 129 pgs: 129 active+clean; 339 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 18 MiB/s wr, 474 op/s 2024-06-10T01:19:47.467 INFO:tasks.workunit.client.0.smithi007.stdout: 10 9321 28.25 MB/sec warmup 24 sec latency 103.334 ms 2024-06-10T01:19:48.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.031+0000 7f7416640700 1 -- 172.21.15.7:0/528270200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f740809ac60 msgr2=0x7f7408006950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:48.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.031+0000 7f7416640700 1 --2- 172.21.15.7:0/528270200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f740809ac60 0x7f7408006950 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f740c009990 tx=0x7f740c009ca0 comp rx=0 tx=0).stop 2024-06-10T01:19:48.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.031+0000 7f7416640700 1 -- 172.21.15.7:0/528270200 shutdown_connections 2024-06-10T01:19:48.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.031+0000 7f7416640700 1 --2- 172.21.15.7:0/528270200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7408098770 0x7f7408098c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:48.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.031+0000 7f7416640700 1 --2- 172.21.15.7:0/528270200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f740809ac60 0x7f7408006950 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:48.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.031+0000 7f7416640700 1 --2- 172.21.15.7:0/528270200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f740809b0d0 0x7f7408006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:48.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.031+0000 7f7416640700 1 -- 172.21.15.7:0/528270200 >> 172.21.15.7:0/528270200 conn(0x7f740800a7a0 msgr2=0x7f740800abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:48.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.031+0000 7f7416640700 1 -- 172.21.15.7:0/528270200 shutdown_connections 2024-06-10T01:19:48.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.032+0000 7f7416640700 1 -- 172.21.15.7:0/528270200 wait complete. 2024-06-10T01:19:48.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.033+0000 7f7416640700 1 Processor -- start 2024-06-10T01:19:48.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.033+0000 7f7416640700 1 -- start start 2024-06-10T01:19:48.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.033+0000 7f7416640700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7408098770 0x7f74081386c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:48.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.033+0000 7f7416640700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f740809ac60 0x7f7408138c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:48.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.033+0000 7f7416640700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f740809b0d0 0x7f7408132880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:48.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.033+0000 7f7416640700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f74080ac600 con 0x7f740809b0d0 2024-06-10T01:19:48.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.033+0000 7f7416640700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f74080ac480 con 0x7f740809ac60 2024-06-10T01:19:48.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.033+0000 7f7416640700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f74080ac780 con 0x7f7408098770 2024-06-10T01:19:48.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.034+0000 7f7415e3f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f740809b0d0 0x7f7408132880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:48.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.034+0000 7f7414e3d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f740809ac60 0x7f7408138c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:48.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.034+0000 7f7415e3f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f740809b0d0 0x7f7408132880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43346/0 (socket says 172.21.15.7:43346) 2024-06-10T01:19:48.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.034+0000 7f7415e3f700 1 -- 172.21.15.7:0/1930605047 learned_addr learned my addr 172.21.15.7:0/1930605047 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:48.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.034+0000 7f7415e3f700 1 -- 172.21.15.7:0/1930605047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7408098770 msgr2=0x7f74081386c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:48.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.034+0000 7f7415e3f700 1 --2- 172.21.15.7:0/1930605047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7408098770 0x7f74081386c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:48.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.034+0000 7f7415e3f700 1 -- 172.21.15.7:0/1930605047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f740809ac60 msgr2=0x7f7408138c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:48.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.034+0000 7f7415e3f700 1 --2- 172.21.15.7:0/1930605047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f740809ac60 0x7f7408138c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:48.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.034+0000 7f7415e3f700 1 -- 172.21.15.7:0/1930605047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7408132fe0 con 0x7f740809b0d0 2024-06-10T01:19:48.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.034+0000 7f7415e3f700 1 --2- 172.21.15.7:0/1930605047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f740809b0d0 0x7f7408132880 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f7410065600 tx=0x7f7410074bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:48.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.035+0000 7f74067fc700 1 -- 172.21.15.7:0/1930605047 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7410083070 con 0x7f740809b0d0 2024-06-10T01:19:48.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.035+0000 7f7416640700 1 -- 172.21.15.7:0/1930605047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74081332d0 con 0x7f740809b0d0 2024-06-10T01:19:48.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.035+0000 7f7416640700 1 -- 172.21.15.7:0/1930605047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f740813f590 con 0x7f740809b0d0 2024-06-10T01:19:48.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.035+0000 7f74067fc700 1 -- 172.21.15.7:0/1930605047 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f741006ac70 con 0x7f740809b0d0 2024-06-10T01:19:48.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.035+0000 7f74067fc700 1 -- 172.21.15.7:0/1930605047 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741008d060 con 0x7f740809b0d0 2024-06-10T01:19:48.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.037+0000 7f73fbfff700 1 -- 172.21.15.7:0/1930605047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7408005560 con 0x7f740809b0d0 2024-06-10T01:19:48.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.039+0000 7f74067fc700 1 -- 172.21.15.7:0/1930605047 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f7410086070 con 0x7f740809b0d0 2024-06-10T01:19:48.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.040+0000 7f74067fc700 1 --2- 172.21.15.7:0/1930605047 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f73fc0a2cf0 0x7f73fc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:48.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.040+0000 7f74067fc700 1 -- 172.21.15.7:0/1930605047 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f741013be20 con 0x7f740809b0d0 2024-06-10T01:19:48.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.040+0000 7f741563e700 1 --2- 172.21.15.7:0/1930605047 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f73fc0a2cf0 0x7f73fc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:48.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.041+0000 7f741563e700 1 --2- 172.21.15.7:0/1930605047 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f73fc0a2cf0 0x7f73fc0a51b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f740c00b820 tx=0x7f740c00b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:48.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.050+0000 7f74067fc700 1 -- 172.21.15.7:0/1930605047 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f741006ae20 con 0x7f740809b0d0 2024-06-10T01:19:48.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.420+0000 7f73fbfff700 1 -- 172.21.15.7:0/1930605047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7408003150 con 0x7f740809b0d0 2024-06-10T01:19:48.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.421+0000 7f74067fc700 1 -- 172.21.15.7:0/1930605047 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11860 (secure 0 0 0) 0x7f74101089a0 con 0x7f740809b0d0 2024-06-10T01:19:48.415 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:19:48.415 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:19:47.239285+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:19:48.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.424+0000 7f73fbfff700 1 -- 172.21.15.7:0/1930605047 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f73fc0a2cf0 msgr2=0x7f73fc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:48.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.424+0000 7f73fbfff700 1 --2- 172.21.15.7:0/1930605047 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f73fc0a2cf0 0x7f73fc0a51b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f740c00b820 tx=0x7f740c00b610 comp rx=0 tx=0).stop 2024-06-10T01:19:48.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.424+0000 7f73fbfff700 1 -- 172.21.15.7:0/1930605047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f740809b0d0 msgr2=0x7f7408132880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:48.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.424+0000 7f73fbfff700 1 --2- 172.21.15.7:0/1930605047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f740809b0d0 0x7f7408132880 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f7410065600 tx=0x7f7410074bd0 comp rx=0 tx=0).stop 2024-06-10T01:19:48.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.424+0000 7f73fbfff700 1 -- 172.21.15.7:0/1930605047 shutdown_connections 2024-06-10T01:19:48.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.424+0000 7f73fbfff700 1 --2- 172.21.15.7:0/1930605047 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f73fc0a2cf0 0x7f73fc0a51b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:48.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.424+0000 7f73fbfff700 1 --2- 172.21.15.7:0/1930605047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f740809b0d0 0x7f7408132880 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:48.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.424+0000 7f73fbfff700 1 --2- 172.21.15.7:0/1930605047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7408098770 0x7f74081386c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:48.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.424+0000 7f73fbfff700 1 --2- 172.21.15.7:0/1930605047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f740809ac60 0x7f7408138c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:48.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.424+0000 7f73fbfff700 1 -- 172.21.15.7:0/1930605047 >> 172.21.15.7:0/1930605047 conn(0x7f740800a7a0 msgr2=0x7f7408097750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:48.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.425+0000 7f73fbfff700 1 -- 172.21.15.7:0/1930605047 shutdown_connections 2024-06-10T01:19:48.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:48.425+0000 7f73fbfff700 1 -- 172.21.15.7:0/1930605047 wait complete. 2024-06-10T01:19:48.419 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 44 2024-06-10T01:19:48.467 INFO:tasks.workunit.client.0.smithi007.stdout: 10 9862 27.83 MB/sec warmup 25 sec latency 163.271 ms 2024-06-10T01:19:49.069 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:48 smithi007 ceph-mon[32398]: pgmap v846: 129 pgs: 129 active+clean; 359 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 20 MiB/s wr, 456 op/s 2024-06-10T01:19:49.069 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:48 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:19:49.069 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:48 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:19:49.069 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:48 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1930605047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:19:49.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:48 smithi175 ceph-mon[36636]: pgmap v846: 129 pgs: 129 active+clean; 359 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 20 MiB/s wr, 456 op/s 2024-06-10T01:19:49.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:48 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:19:49.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:48 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:19:49.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:48 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1930605047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:19:49.311 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db / --timeout 90 --expiration 147.65 --await-for 45.0' 2024-06-10T01:19:49.311 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db / --timeout 90 --expiration 147.65 --await-for 45.0 2024-06-10T01:19:49.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:48 smithi078 ceph-mon[36683]: pgmap v846: 129 pgs: 129 active+clean; 359 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 20 MiB/s wr, 456 op/s 2024-06-10T01:19:49.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:48 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:19:49.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:48 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:19:49.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:48 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1930605047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:19:49.467 INFO:tasks.workunit.client.0.smithi007.stdout: 10 10679 29.10 MB/sec warmup 26 sec latency 81.948 ms 2024-06-10T01:19:49.708 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:19:50.468 INFO:tasks.workunit.client.0.smithi007.stdout: 10 11349 29.05 MB/sec warmup 27 sec latency 49.764 ms 2024-06-10T01:19:51.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:50 smithi175 ceph-mon[36636]: pgmap v847: 129 pgs: 129 active+clean; 366 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 25 MiB/s wr, 719 op/s 2024-06-10T01:19:51.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:50 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:19:51.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:50 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:19:51.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:50 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:19:51.249 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:50 smithi007 ceph-mon[32398]: pgmap v847: 129 pgs: 129 active+clean; 366 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 25 MiB/s wr, 719 op/s 2024-06-10T01:19:51.249 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:50 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:19:51.249 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:50 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:19:51.249 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:50 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:19:51.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:50 smithi078 ceph-mon[36683]: pgmap v847: 129 pgs: 129 active+clean; 366 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 25 MiB/s wr, 719 op/s 2024-06-10T01:19:51.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:50 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:19:51.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:50 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:19:51.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:50 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:19:51.467 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12045 29.38 MB/sec warmup 28 sec latency 81.398 ms 2024-06-10T01:19:51.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.889+0000 7f486d62b700 1 -- 172.21.15.7:0/2188201428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4868101770 msgr2=0x7f4868101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:51.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.889+0000 7f486d62b700 1 --2- 172.21.15.7:0/2188201428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4868101770 0x7f4868101bf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f485c009990 tx=0x7f485c009ca0 comp rx=0 tx=0).stop 2024-06-10T01:19:51.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.889+0000 7f486d62b700 1 -- 172.21.15.7:0/2188201428 shutdown_connections 2024-06-10T01:19:51.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.889+0000 7f486d62b700 1 --2- 172.21.15.7:0/2188201428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4868102130 0x7f486810e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:51.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.889+0000 7f486d62b700 1 --2- 172.21.15.7:0/2188201428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48681074b0 0x7f4868107890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:51.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.889+0000 7f486d62b700 1 --2- 172.21.15.7:0/2188201428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4868101770 0x7f4868101bf0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:51.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.889+0000 7f486d62b700 1 -- 172.21.15.7:0/2188201428 >> 172.21.15.7:0/2188201428 conn(0x7f4868078340 msgr2=0x7f4868078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:51.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.889+0000 7f486d62b700 1 -- 172.21.15.7:0/2188201428 shutdown_connections 2024-06-10T01:19:51.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.889+0000 7f486d62b700 1 -- 172.21.15.7:0/2188201428 wait complete. 2024-06-10T01:19:51.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.891+0000 7f486d62b700 1 Processor -- start 2024-06-10T01:19:51.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.891+0000 7f486d62b700 1 -- start start 2024-06-10T01:19:51.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.892+0000 7f486d62b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4868101770 0x7f4868073c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:51.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.892+0000 7f486d62b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4868102130 0x7f4868074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:51.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.892+0000 7f486d62b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48681074b0 0x7f486806bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:51.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.892+0000 7f486d62b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4868111750 con 0x7f4868102130 2024-06-10T01:19:51.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.892+0000 7f486d62b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f48681115d0 con 0x7f4868101770 2024-06-10T01:19:51.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.892+0000 7f486d62b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f48681118d0 con 0x7f48681074b0 2024-06-10T01:19:51.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.892+0000 7f48667fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4868102130 0x7f4868074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:51.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.892+0000 7f48667fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4868102130 0x7f4868074170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37946/0 (socket says 172.21.15.7:37946) 2024-06-10T01:19:51.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.892+0000 7f4866ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4868101770 0x7f4868073c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:51.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.892+0000 7f48667fc700 1 -- 172.21.15.7:0/3264950730 learned_addr learned my addr 172.21.15.7:0/3264950730 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:51.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.892+0000 7f48677fe700 1 --2- 172.21.15.7:0/3264950730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48681074b0 0x7f486806bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:51.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.893+0000 7f48667fc700 1 -- 172.21.15.7:0/3264950730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48681074b0 msgr2=0x7f486806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:51.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.893+0000 7f48667fc700 1 --2- 172.21.15.7:0/3264950730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48681074b0 0x7f486806bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:51.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.893+0000 7f48667fc700 1 -- 172.21.15.7:0/3264950730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4868101770 msgr2=0x7f4868073c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:51.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.893+0000 7f48667fc700 1 --2- 172.21.15.7:0/3264950730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4868101770 0x7f4868073c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:51.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.893+0000 7f48667fc700 1 -- 172.21.15.7:0/3264950730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486806c850 con 0x7f4868102130 2024-06-10T01:19:51.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.893+0000 7f48667fc700 1 --2- 172.21.15.7:0/3264950730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4868102130 0x7f4868074170 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f485c000c00 tx=0x7f485c00bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:51.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.894+0000 7f484ffff700 1 -- 172.21.15.7:0/3264950730 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c027070 con 0x7f4868102130 2024-06-10T01:19:51.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.894+0000 7f484ffff700 1 -- 172.21.15.7:0/3264950730 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f485c003d70 con 0x7f4868102130 2024-06-10T01:19:51.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.894+0000 7f484ffff700 1 -- 172.21.15.7:0/3264950730 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c00f460 con 0x7f4868102130 2024-06-10T01:19:51.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.894+0000 7f486d62b700 1 -- 172.21.15.7:0/3264950730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f486806cae0 con 0x7f4868102130 2024-06-10T01:19:51.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.894+0000 7f486d62b700 1 -- 172.21.15.7:0/3264950730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48681120c0 con 0x7f4868102130 2024-06-10T01:19:51.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.897+0000 7f484ffff700 1 -- 172.21.15.7:0/3264950730 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f485c009d10 con 0x7f4868102130 2024-06-10T01:19:51.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.898+0000 7f484ffff700 1 --2- 172.21.15.7:0/3264950730 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f48500a7240 0x7f48500a9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:51.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.898+0000 7f4866ffd700 1 --2- 172.21.15.7:0/3264950730 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f48500a7240 0x7f48500a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:51.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.898+0000 7f4877eaeb80 1 Processor -- start 2024-06-10T01:19:51.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.898+0000 7f484ffff700 1 -- 172.21.15.7:0/3264950730 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f485c0d0240 con 0x7f4868102130 2024-06-10T01:19:51.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.898+0000 7f4877eaeb80 1 -- start start 2024-06-10T01:19:51.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.899+0000 7f4877eaeb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620526e5ce0 0x5620526e60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:51.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.899+0000 7f4877eaeb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620526e6600 0x5620526dff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:51.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.899+0000 7f4877eaeb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620526e0470 0x5620526e08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:51.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.899+0000 7f4877eaeb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5620526319b0 con 0x5620526e6600 2024-06-10T01:19:51.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.899+0000 7f4877eaeb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5620525d98c0 con 0x5620526e0470 2024-06-10T01:19:51.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.899+0000 7f4877eaeb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56205262d970 con 0x5620526e5ce0 2024-06-10T01:19:51.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.899+0000 7f48667fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620526e6600 0x5620526dff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:51.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.899+0000 7f48667fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620526e6600 0x5620526dff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37962/0 (socket says 172.21.15.7:37962) 2024-06-10T01:19:51.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.899+0000 7f48667fc700 1 -- 172.21.15.7:0/3306899587 learned_addr learned my addr 172.21.15.7:0/3306899587 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:51.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.899+0000 7f48677fe700 1 --2- 172.21.15.7:0/3306899587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620526e0470 0x5620526e08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:51.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.899+0000 7f4866ffd700 1 --2- 172.21.15.7:0/3306899587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620526e5ce0 0x5620526e60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:51.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.899+0000 7f4866ffd700 1 --2- 172.21.15.7:0/3264950730 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f48500a7240 0x7f48500a9700 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4858009bd0 tx=0x7f4858009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:51.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.899+0000 7f48667fc700 1 -- 172.21.15.7:0/3306899587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620526e5ce0 msgr2=0x5620526e60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:51.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.899+0000 7f48667fc700 1 --2- 172.21.15.7:0/3306899587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620526e5ce0 0x5620526e60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:51.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.899+0000 7f48667fc700 1 -- 172.21.15.7:0/3306899587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620526e0470 msgr2=0x5620526e08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:51.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.899+0000 7f48667fc700 1 --2- 172.21.15.7:0/3306899587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620526e0470 0x5620526e08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:51.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.899+0000 7f48667fc700 1 -- 172.21.15.7:0/3306899587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620525d00f0 con 0x5620526e6600 2024-06-10T01:19:51.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.900+0000 7f48667fc700 1 --2- 172.21.15.7:0/3306899587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620526e6600 0x5620526dff30 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f485c09c4b0 tx=0x7f485c022a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de25d5bc9352f3fe server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:51.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.900+0000 7f484d7fa700 1 -- 172.21.15.7:0/3306899587 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c09c110 con 0x5620526e6600 2024-06-10T01:19:51.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.900+0000 7f484d7fa700 1 -- 172.21.15.7:0/3306899587 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f485c09ceb0 con 0x5620526e6600 2024-06-10T01:19:51.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.900+0000 7f484d7fa700 1 -- 172.21.15.7:0/3306899587 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c0a30c0 con 0x5620526e6600 2024-06-10T01:19:51.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.901+0000 7f4877eaeb80 1 -- 172.21.15.7:0/3306899587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620526e6600 msgr2=0x5620526dff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:51.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.901+0000 7f4877eaeb80 1 --2- 172.21.15.7:0/3306899587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620526e6600 0x5620526dff30 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f485c09c4b0 tx=0x7f485c022a30 comp rx=0 tx=0).stop 2024-06-10T01:19:51.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.901+0000 7f4877eaeb80 1 -- 172.21.15.7:0/3306899587 shutdown_connections 2024-06-10T01:19:51.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.901+0000 7f4877eaeb80 1 --2- 172.21.15.7:0/3306899587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620526e0470 0x5620526e08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:51.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.901+0000 7f4877eaeb80 1 --2- 172.21.15.7:0/3306899587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620526e5ce0 0x5620526e60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:51.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.901+0000 7f4877eaeb80 1 --2- 172.21.15.7:0/3306899587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620526e6600 0x5620526dff30 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:51.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.901+0000 7f4877eaeb80 1 -- 172.21.15.7:0/3306899587 >> 172.21.15.7:0/3306899587 conn(0x56205260ae60 msgr2=0x5620526db1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:51.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.901+0000 7f4877eaeb80 1 -- 172.21.15.7:0/3306899587 shutdown_connections 2024-06-10T01:19:51.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.901+0000 7f4877eaeb80 1 -- 172.21.15.7:0/3306899587 wait complete. 2024-06-10T01:19:51.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.901+0000 7f4877eaeb80 1 -- start start 2024-06-10T01:19:51.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.901+0000 7f4877eaeb80 1 Processor -- start 2024-06-10T01:19:51.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.902+0000 7f4877eaeb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620526e0470 0x5620526ec360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:51.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.902+0000 7f4877eaeb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620526e5ce0 0x5620526e9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:51.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.902+0000 7f4877eaeb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620526e6600 0x5620526e9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:51.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.902+0000 7f4877eaeb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5620526ecd90 con 0x5620526e6600 2024-06-10T01:19:51.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.902+0000 7f4877eaeb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5620525d1d90 con 0x5620526e5ce0 2024-06-10T01:19:51.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.902+0000 7f4877eaeb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5620526ecf10 con 0x5620526e0470 2024-06-10T01:19:51.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.902+0000 7f4877eaeb80 10 client.0 _ll_register_callbacks cb 0x5620525cbac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:19:51.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.902+0000 7f48677fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620526e6600 0x5620526e9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:51.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.902+0000 7f48677fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620526e6600 0x5620526e9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37974/0 (socket says 172.21.15.7:37974) 2024-06-10T01:19:51.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.902+0000 7f48677fe700 1 -- 172.21.15.7:0/2841728192 learned_addr learned my addr 172.21.15.7:0/2841728192 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:51.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.902+0000 7f48667fc700 1 --2- 172.21.15.7:0/2841728192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620526e5ce0 0x5620526e9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:51.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.903+0000 7f4866ffd700 1 --2- 172.21.15.7:0/2841728192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620526e0470 0x5620526ec360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:51.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.903+0000 7f48667fc700 1 -- 172.21.15.7:0/2841728192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620526e0470 msgr2=0x5620526ec360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:51.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.903+0000 7f48667fc700 1 --2- 172.21.15.7:0/2841728192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620526e0470 0x5620526ec360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:51.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.903+0000 7f48667fc700 1 -- 172.21.15.7:0/2841728192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620526e6600 msgr2=0x5620526e9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:51.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.903+0000 7f48667fc700 1 --2- 172.21.15.7:0/2841728192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620526e6600 0x5620526e9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:51.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.903+0000 7f48667fc700 1 -- 172.21.15.7:0/2841728192 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562052580a70 con 0x5620526e5ce0 2024-06-10T01:19:51.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.903+0000 7f48677fe700 1 --2- 172.21.15.7:0/2841728192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620526e6600 0x5620526e9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:51.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.903+0000 7f4866ffd700 1 --2- 172.21.15.7:0/2841728192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620526e0470 0x5620526ec360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:19:51.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.903+0000 7f48667fc700 1 --2- 172.21.15.7:0/2841728192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620526e5ce0 0x5620526e9660 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f485c09c4b0 tx=0x7f485c005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30c65fcb1c5d4702 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:51.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.903+0000 7f484aff5700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:19:51.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.904+0000 7f484aff5700 1 -- 172.21.15.7:0/2841728192 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c09c110 con 0x5620526e5ce0 2024-06-10T01:19:51.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.904+0000 7f484aff5700 1 -- 172.21.15.7:0/2841728192 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f485c022470 con 0x5620526e5ce0 2024-06-10T01:19:51.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.904+0000 7f4877eaeb80 1 -- 172.21.15.7:0/2841728192 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f485c09ceb0 con 0x5620526e5ce0 2024-06-10T01:19:51.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.904+0000 7f484aff5700 1 -- 172.21.15.7:0/2841728192 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f485c09c7b0 con 0x5620526e5ce0 2024-06-10T01:19:51.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.904+0000 7f4877eaeb80 10 client.28223 fetch_fsmap learned FSMap version 44 2024-06-10T01:19:51.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.904+0000 7f4877eaeb80 1 -- 172.21.15.7:0/2841728192 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5620526ecd90 con 0x5620526e5ce0 2024-06-10T01:19:51.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.904+0000 7f484aff5700 1 -- 172.21.15.7:0/2841728192 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f485c09aab0 con 0x5620526e5ce0 2024-06-10T01:19:51.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.905+0000 7f484aff5700 1 -- 172.21.15.7:0/2841728192 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f485c09bc00 con 0x5620526e5ce0 2024-06-10T01:19:51.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.905+0000 7f4877eaeb80 10 client.28223 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:19:51.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.905+0000 7f4877eaeb80 10 client.28223 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:19:51.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.905+0000 7f4877eaeb80 20 client.28223 populate_metadata read hostname 'smithi007' 2024-06-10T01:19:51.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.905+0000 7f4877eaeb80 1 --2- 172.21.15.7:0/2841728192 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5620526f99b0 0x5620526fbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:51.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.905+0000 7f4877eaeb80 4 client.28223 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:19:51.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.905+0000 7f4877eaeb80 1 -- 172.21.15.7:0/2841728192 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f485c09ceb0 con 0x5620526f99b0 2024-06-10T01:19:51.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.905+0000 7f4866ffd700 1 --2- 172.21.15.7:0/2841728192 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5620526f99b0 0x5620526fbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:51.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.906+0000 7f4866ffd700 1 --2- 172.21.15.7:0/2841728192 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5620526f99b0 0x5620526fbdd0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e91c1cc344dbda9d server_cookie=7746fb50cd20db02 in_seq=0 out_seq=0 2024-06-10T01:19:51.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.906+0000 7f484aff5700 10 client.28223 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:51.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.908+0000 7f484aff5700 1 -- 172.21.15.7:0/2841728192 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f485c09ceb0 con 0x5620526f99b0 2024-06-10T01:19:51.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.908+0000 7f484aff5700 10 client.28223 handle_command_reply: tid=0 2024-06-10T01:19:51.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.908+0000 7f4877eaeb80 1 client.28223 shutdown 2024-06-10T01:19:51.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.908+0000 7f4877eaeb80 1 -- 172.21.15.7:0/2841728192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620526e5ce0 msgr2=0x5620526e9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.908+0000 7f4877eaeb80 1 --2- 172.21.15.7:0/2841728192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620526e5ce0 0x5620526e9660 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f485c09c4b0 tx=0x7f485c005c90 comp rx=0 tx=0).stop 2024-06-10T01:19:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.908+0000 7f4877eaeb80 1 -- 172.21.15.7:0/2841728192 shutdown_connections 2024-06-10T01:19:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.908+0000 7f4877eaeb80 1 --2- 172.21.15.7:0/2841728192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620526e6600 0x5620526e9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.908+0000 7f4877eaeb80 1 --2- 172.21.15.7:0/2841728192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620526e0470 0x5620526ec360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.908+0000 7f4877eaeb80 1 --2- 172.21.15.7:0/2841728192 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5620526f99b0 0x5620526fbdd0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.908+0000 7f4877eaeb80 1 --2- 172.21.15.7:0/2841728192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620526e5ce0 0x5620526e9660 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.908+0000 7f484aff5700 0 client.28223 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.908+0000 7f4877eaeb80 1 -- 172.21.15.7:0/2841728192 >> 172.21.15.7:0/2841728192 conn(0x56205260ae60 msgr2=0x5620526def20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.908+0000 7f4877eaeb80 1 -- 172.21.15.7:0/2841728192 shutdown_connections 2024-06-10T01:19:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.908+0000 7f4877eaeb80 1 -- 172.21.15.7:0/2841728192 wait complete. 2024-06-10T01:19:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.909+0000 7f4877eaeb80 20 client.28223 trim_cache size 0 max 16384 2024-06-10T01:19:51.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.933+0000 7f4877eaeb80 1 Processor -- start 2024-06-10T01:19:51.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.933+0000 7f4877eaeb80 1 -- start start 2024-06-10T01:19:51.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.933+0000 7f4877eaeb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620526e0470 0x5620526df7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:51.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.933+0000 7f4877eaeb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620526e6600 0x5620526dfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:51.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.933+0000 7f4877eaeb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620526f6030 0x5620526e6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:51.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.933+0000 7f4877eaeb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f48681b7080 con 0x5620526f6030 2024-06-10T01:19:51.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.933+0000 7f4877eaeb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f48681b4880 con 0x5620526e0470 2024-06-10T01:19:51.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.933+0000 7f4877eaeb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f48681b26d0 con 0x5620526e6600 2024-06-10T01:19:51.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.933+0000 7f48677fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620526f6030 0x5620526e6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:51.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.933+0000 7f4866ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620526e0470 0x5620526df7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:51.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.933+0000 7f48667fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620526e6600 0x5620526dfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:51.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.933+0000 7f4866ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620526e0470 0x5620526df7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:44278/0 (socket says 172.21.15.7:44278) 2024-06-10T01:19:51.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.933+0000 7f48677fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620526f6030 0x5620526e6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37978/0 (socket says 172.21.15.7:37978) 2024-06-10T01:19:51.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.933+0000 7f4866ffd700 1 -- 172.21.15.7:0/2870347770 learned_addr learned my addr 172.21.15.7:0/2870347770 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:51.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.933+0000 7f48667fc700 1 -- 172.21.15.7:0/2870347770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620526e0470 msgr2=0x5620526df7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:51.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.933+0000 7f48667fc700 1 --2- 172.21.15.7:0/2870347770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620526e0470 0x5620526df7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:51.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.933+0000 7f48667fc700 1 -- 172.21.15.7:0/2870347770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620526f6030 msgr2=0x5620526e6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:51.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.934+0000 7f48667fc700 1 --2- 172.21.15.7:0/2870347770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620526f6030 0x5620526e6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:51.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.934+0000 7f48667fc700 1 -- 172.21.15.7:0/2870347770 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620526d58c0 con 0x5620526e6600 2024-06-10T01:19:51.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.934+0000 7f48677fe700 1 --2- 172.21.15.7:0/2870347770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620526f6030 0x5620526e6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:51.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.934+0000 7f48667fc700 1 --2- 172.21.15.7:0/2870347770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620526e6600 0x5620526dfd10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x5620526ea6f0 tx=0x7f485c005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8c9c775ab67d4d9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:51.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.934+0000 7f484cff9700 1 -- 172.21.15.7:0/2870347770 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c09de50 con 0x5620526e6600 2024-06-10T01:19:51.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.934+0000 7f484cff9700 1 -- 172.21.15.7:0/2870347770 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f485c09ceb0 con 0x5620526e6600 2024-06-10T01:19:51.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.934+0000 7f484cff9700 1 -- 172.21.15.7:0/2870347770 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c0a4f00 con 0x5620526e6600 2024-06-10T01:19:51.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.935+0000 7f4877eaeb80 1 -- 172.21.15.7:0/2870347770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620526e6600 msgr2=0x5620526dfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:51.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.935+0000 7f4877eaeb80 1 --2- 172.21.15.7:0/2870347770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620526e6600 0x5620526dfd10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x5620526ea6f0 tx=0x7f485c005fb0 comp rx=0 tx=0).stop 2024-06-10T01:19:51.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.935+0000 7f4877eaeb80 1 -- 172.21.15.7:0/2870347770 shutdown_connections 2024-06-10T01:19:51.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.935+0000 7f4877eaeb80 1 --2- 172.21.15.7:0/2870347770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620526f6030 0x5620526e6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:51.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.935+0000 7f4877eaeb80 1 --2- 172.21.15.7:0/2870347770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620526e0470 0x5620526df7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:51.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.935+0000 7f4877eaeb80 1 --2- 172.21.15.7:0/2870347770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620526e6600 0x5620526dfd10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:51.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.935+0000 7f4877eaeb80 1 -- 172.21.15.7:0/2870347770 >> 172.21.15.7:0/2870347770 conn(0x56205260ae60 msgr2=0x5620526e7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:51.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.935+0000 7f4877eaeb80 1 -- 172.21.15.7:0/2870347770 shutdown_connections 2024-06-10T01:19:51.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.935+0000 7f4877eaeb80 1 -- 172.21.15.7:0/2870347770 wait complete. 2024-06-10T01:19:51.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.935+0000 7f4877eaeb80 1 -- start start 2024-06-10T01:19:51.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.935+0000 7f4877eaeb80 1 Processor -- start 2024-06-10T01:19:51.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.936+0000 7f4877eaeb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620526e0470 0x5620526f2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:51.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.936+0000 7f4877eaeb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620526e6600 0x5620526f2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:51.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.936+0000 7f4877eaeb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620526f6030 0x5620526ef320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:51.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.936+0000 7f4877eaeb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f48681b2d40 con 0x5620526e0470 2024-06-10T01:19:51.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.936+0000 7f4877eaeb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f48681b41b0 con 0x5620526e6600 2024-06-10T01:19:51.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.936+0000 7f4877eaeb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f48681b3410 con 0x5620526f6030 2024-06-10T01:19:51.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.936+0000 7f4866ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620526e0470 0x5620526f2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:51.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.936+0000 7f4866ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620526e0470 0x5620526f2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37992/0 (socket says 172.21.15.7:37992) 2024-06-10T01:19:51.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.936+0000 7f4866ffd700 1 -- 172.21.15.7:0/635288298 learned_addr learned my addr 172.21.15.7:0/635288298 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:51.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.936+0000 7f48677fe700 1 --2- 172.21.15.7:0/635288298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620526f6030 0x5620526ef320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:51.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.936+0000 7f4877eaeb80 10 client.0 _ll_register_callbacks cb 0x5620525cbac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:19:51.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.936+0000 7f48667fc700 1 --2- 172.21.15.7:0/635288298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620526e6600 0x5620526f2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:51.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.936+0000 7f4866ffd700 1 -- 172.21.15.7:0/635288298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620526f6030 msgr2=0x5620526ef320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:51.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.936+0000 7f4866ffd700 1 --2- 172.21.15.7:0/635288298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620526f6030 0x5620526ef320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:51.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.936+0000 7f4866ffd700 1 -- 172.21.15.7:0/635288298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620526e6600 msgr2=0x5620526f2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:51.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.936+0000 7f4866ffd700 1 --2- 172.21.15.7:0/635288298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620526e6600 0x5620526f2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:51.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.936+0000 7f4866ffd700 1 -- 172.21.15.7:0/635288298 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620526efb60 con 0x5620526e0470 2024-06-10T01:19:51.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.936+0000 7f48667fc700 1 --2- 172.21.15.7:0/635288298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620526e6600 0x5620526f2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:19:51.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.936+0000 7f48677fe700 1 --2- 172.21.15.7:0/635288298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620526f6030 0x5620526ef320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:51.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.937+0000 7f4866ffd700 1 --2- 172.21.15.7:0/635288298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620526e0470 0x5620526f2540 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f4858029fd0 tx=0x7f48580047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce90ccbfcea65dd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:51.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.937+0000 7f484aff5700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:19:51.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.937+0000 7f484aff5700 1 -- 172.21.15.7:0/635288298 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4858029500 con 0x5620526e0470 2024-06-10T01:19:51.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.937+0000 7f484aff5700 1 -- 172.21.15.7:0/635288298 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f48580296a0 con 0x5620526e0470 2024-06-10T01:19:51.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.937+0000 7f4877eaeb80 1 -- 172.21.15.7:0/635288298 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f485c09ceb0 con 0x5620526e0470 2024-06-10T01:19:51.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.937+0000 7f484aff5700 1 -- 172.21.15.7:0/635288298 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f485803b050 con 0x5620526e0470 2024-06-10T01:19:51.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.937+0000 7f4877eaeb80 10 client.18924 fetch_fsmap learned FSMap version 44 2024-06-10T01:19:51.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.937+0000 7f4877eaeb80 1 -- 172.21.15.7:0/635288298 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56205276daf0 con 0x5620526e0470 2024-06-10T01:19:51.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.940+0000 7f484aff5700 1 -- 172.21.15.7:0/635288298 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f485c09ceb0 con 0x5620526e0470 2024-06-10T01:19:51.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.941+0000 7f484aff5700 1 -- 172.21.15.7:0/635288298 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f4858020b70 con 0x5620526e0470 2024-06-10T01:19:51.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.941+0000 7f4877eaeb80 10 client.18924 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:19:51.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.941+0000 7f4877eaeb80 10 client.18924 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:19:51.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.941+0000 7f4877eaeb80 20 client.18924 populate_metadata read hostname 'smithi007' 2024-06-10T01:19:51.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.941+0000 7f4877eaeb80 1 --2- 172.21.15.7:0/635288298 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56205276e300 0x562052770720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:51.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.941+0000 7f4877eaeb80 4 client.18924 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.65, "await_for": 45.0}] 2024-06-10T01:19:51.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.941+0000 7f4877eaeb80 1 -- 172.21.15.7:0/635288298 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.65, "await_for": 45.0}) -- 0x56205263eeb0 con 0x56205276e300 2024-06-10T01:19:51.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.941+0000 7f48667fc700 1 --2- 172.21.15.7:0/635288298 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56205276e300 0x562052770720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:51.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.941+0000 7f48667fc700 1 --2- 172.21.15.7:0/635288298 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56205276e300 0x562052770720 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98e68e052d544679 server_cookie=268a611c73b25292 in_seq=0 out_seq=0 2024-06-10T01:19:51.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:51.941+0000 7f484aff5700 10 client.18924 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:52.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:52.356+0000 7f484aff5700 1 -- 172.21.15.7:0/635288298 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56205263eeb0 con 0x56205276e300 2024-06-10T01:19:52.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:52.356+0000 7f484aff5700 10 client.18924 handle_command_reply: tid=0 2024-06-10T01:19:52.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:52.356+0000 7f4877eaeb80 1 client.18924 shutdown 2024-06-10T01:19:52.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:52.356+0000 7f4877eaeb80 1 -- 172.21.15.7:0/635288298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620526e0470 msgr2=0x5620526f2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:52.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:52.356+0000 7f4877eaeb80 1 --2- 172.21.15.7:0/635288298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620526e0470 0x5620526f2540 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f4858029fd0 tx=0x7f48580047c0 comp rx=0 tx=0).stop 2024-06-10T01:19:52.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:52.356+0000 7f4877eaeb80 1 -- 172.21.15.7:0/635288298 shutdown_connections 2024-06-10T01:19:52.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:52.356+0000 7f4877eaeb80 1 --2- 172.21.15.7:0/635288298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620526f6030 0x5620526ef320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:52.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:52.356+0000 7f4877eaeb80 1 --2- 172.21.15.7:0/635288298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620526e0470 0x5620526f2540 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:52.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:52.356+0000 7f4877eaeb80 1 --2- 172.21.15.7:0/635288298 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56205276e300 0x562052770720 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:52.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:52.356+0000 7f4877eaeb80 1 --2- 172.21.15.7:0/635288298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620526e6600 0x5620526f2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:52.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:52.356+0000 7f484aff5700 0 client.18924 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:52.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:52.356+0000 7f4877eaeb80 1 -- 172.21.15.7:0/635288298 >> 172.21.15.7:0/635288298 conn(0x56205260ae60 msgr2=0x5620526e7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:52.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:52.356+0000 7f4877eaeb80 1 -- 172.21.15.7:0/635288298 shutdown_connections 2024-06-10T01:19:52.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:52.356+0000 7f4877eaeb80 1 -- 172.21.15.7:0/635288298 wait complete. 2024-06-10T01:19:52.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:52.356+0000 7f4877eaeb80 20 client.18924 trim_cache size 0 max 16384 2024-06-10T01:19:52.352 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:19:52.352 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 44, 2024-06-10T01:19:52.352 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:19:52.352 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 46, 2024-06-10T01:19:52.352 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:19:52.352 INFO:teuthology.orchestra.run.smithi007.stdout: "1d08866a": { 2024-06-10T01:19:52.352 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 46, 2024-06-10T01:19:52.352 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:19:52.352 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:19:52.352 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:19:52.352 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:19:52.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:19:52.353 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:19:52.353 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 147.7, 2024-06-10T01:19:52.353 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:19:52.353 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:19:52.353 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:19:52.353 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:19:52.353 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:19:52.353 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:19:52.353 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:19:52.353 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:19:52.353 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:19:52.353 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:19:52.353 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:19:52.354 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:19:52.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:52.357+0000 7f486d62b700 1 -- 172.21.15.7:0/3264950730 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f48500a7240 msgr2=0x7f48500a9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:52.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:52.357+0000 7f486d62b700 1 --2- 172.21.15.7:0/3264950730 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f48500a7240 0x7f48500a9700 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4858009bd0 tx=0x7f4858009260 comp rx=0 tx=0).stop 2024-06-10T01:19:52.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:52.357+0000 7f486d62b700 1 -- 172.21.15.7:0/3264950730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4868102130 msgr2=0x7f4868074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:52.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:52.357+0000 7f486d62b700 1 --2- 172.21.15.7:0/3264950730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4868102130 0x7f4868074170 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f485c000c00 tx=0x7f485c00bb60 comp rx=0 tx=0).stop 2024-06-10T01:19:52.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:52.357+0000 7f486d62b700 1 -- 172.21.15.7:0/3264950730 shutdown_connections 2024-06-10T01:19:52.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:52.357+0000 7f486d62b700 1 --2- 172.21.15.7:0/3264950730 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f48500a7240 0x7f48500a9700 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:52.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:52.357+0000 7f486d62b700 1 --2- 172.21.15.7:0/3264950730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48681074b0 0x7f486806bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:52.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:52.357+0000 7f486d62b700 1 --2- 172.21.15.7:0/3264950730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4868101770 0x7f4868073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:52.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:52.357+0000 7f486d62b700 1 --2- 172.21.15.7:0/3264950730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4868102130 0x7f4868074170 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:52.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:52.357+0000 7f486d62b700 1 -- 172.21.15.7:0/3264950730 >> 172.21.15.7:0/3264950730 conn(0x7f4868078340 msgr2=0x7f48680ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:52.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:52.357+0000 7f486d62b700 1 -- 172.21.15.7:0/3264950730 shutdown_connections 2024-06-10T01:19:52.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:19:52.357+0000 7f486d62b700 1 -- 172.21.15.7:0/3264950730 wait complete. 2024-06-10T01:19:52.467 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12545 29.32 MB/sec warmup 29 sec latency 544.560 ms 2024-06-10T01:19:53.097 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1d08866a', quiesce took 9.7 seconds. Will release after: 17.95 2024-06-10T01:19:53.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:52 smithi175 ceph-mon[36636]: pgmap v848: 129 pgs: 129 active+clean; 358 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 23 MiB/s wr, 806 op/s 2024-06-10T01:19:53.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:52 smithi007 ceph-mon[32398]: pgmap v848: 129 pgs: 129 active+clean; 358 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 23 MiB/s wr, 806 op/s 2024-06-10T01:19:53.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:52 smithi078 ceph-mon[36683]: pgmap v848: 129 pgs: 129 active+clean; 358 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 23 MiB/s wr, 806 op/s 2024-06-10T01:19:53.468 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12545 28.34 MB/sec warmup 30 sec latency 1544.663 ms 2024-06-10T01:19:54.468 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12545 27.42 MB/sec warmup 31 sec latency 2544.798 ms 2024-06-10T01:19:55.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:54 smithi175 ceph-mon[36636]: pgmap v849: 129 pgs: 129 active+clean; 357 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 20 MiB/s wr, 785 op/s 2024-06-10T01:19:55.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:54 smithi007 ceph-mon[32398]: pgmap v849: 129 pgs: 129 active+clean; 357 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 20 MiB/s wr, 785 op/s 2024-06-10T01:19:55.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:54 smithi078 ceph-mon[36683]: pgmap v849: 129 pgs: 129 active+clean; 357 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 20 MiB/s wr, 785 op/s 2024-06-10T01:19:55.468 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12545 26.57 MB/sec warmup 32 sec latency 3544.923 ms 2024-06-10T01:19:56.468 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12545 25.76 MB/sec warmup 33 sec latency 4545.052 ms 2024-06-10T01:19:57.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:56 smithi175 ceph-mon[36636]: pgmap v850: 129 pgs: 129 active+clean; 361 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 13 MiB/s wr, 760 op/s 2024-06-10T01:19:57.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:56 smithi007 ceph-mon[32398]: pgmap v850: 129 pgs: 129 active+clean; 361 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 13 MiB/s wr, 760 op/s 2024-06-10T01:19:57.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:56 smithi078 ceph-mon[36683]: pgmap v850: 129 pgs: 129 active+clean; 361 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 13 MiB/s wr, 760 op/s 2024-06-10T01:19:57.468 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12545 25.00 MB/sec warmup 34 sec latency 5545.173 ms 2024-06-10T01:19:58.468 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12545 24.29 MB/sec warmup 35 sec latency 6545.258 ms 2024-06-10T01:19:59.065 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:19:59.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:19:58 smithi175 ceph-mon[36636]: pgmap v851: 129 pgs: 129 active+clean; 363 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 9.8 MiB/s wr, 738 op/s 2024-06-10T01:19:59.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:19:58 smithi007 ceph-mon[32398]: pgmap v851: 129 pgs: 129 active+clean; 363 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 9.8 MiB/s wr, 738 op/s 2024-06-10T01:19:59.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:19:58 smithi078 ceph-mon[36683]: pgmap v851: 129 pgs: 129 active+clean; 363 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 9.8 MiB/s wr, 738 op/s 2024-06-10T01:19:59.333 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:19:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.467+0000 7fcbc6970640 1 -- 172.21.15.7:0/50178742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbc01035c0 msgr2=0x7fcbc0109e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.467+0000 7fcbc6970640 1 --2- 172.21.15.7:0/50178742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbc01035c0 0x7fcbc0109e50 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8009ca0 tx=0x7fcbb802a8c0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.467+0000 7fcbc6970640 1 -- 172.21.15.7:0/50178742 shutdown_connections 2024-06-10T01:19:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.467+0000 7fcbc6970640 1 --2- 172.21.15.7:0/50178742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbc01035c0 0x7fcbc0109e50 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.467+0000 7fcbc6970640 1 --2- 172.21.15.7:0/50178742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbc0102c00 0x7fcbc0103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.467+0000 7fcbc6970640 1 --2- 172.21.15.7:0/50178742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcbc0101a10 0x7fcbc0101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.467+0000 7fcbc6970640 1 -- 172.21.15.7:0/50178742 >> 172.21.15.7:0/50178742 conn(0x7fcbc00fcf80 msgr2=0x7fcbc00ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.467+0000 7fcbc6970640 1 -- 172.21.15.7:0/50178742 shutdown_connections 2024-06-10T01:19:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.467+0000 7fcbc6970640 1 -- 172.21.15.7:0/50178742 wait complete. 2024-06-10T01:19:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.468+0000 7fcbc6970640 1 Processor -- start 2024-06-10T01:19:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.468+0000 7fcbc6970640 1 -- start start 2024-06-10T01:19:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.468+0000 7fcbc6970640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbc0101a10 0x7fcbc0197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.468+0000 7fcbc6970640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcbc0102c00 0x7fcbc0198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.468+0000 7fcbc6970640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbc01035c0 0x7fcbc019f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.468+0000 7fcbc6970640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc010d690 con 0x7fcbc0101a10 2024-06-10T01:19:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.468+0000 7fcbc6970640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc010d510 con 0x7fcbc0102c00 2024-06-10T01:19:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.468+0000 7fcbc6970640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc010d810 con 0x7fcbc01035c0 2024-06-10T01:19:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.468+0000 7fcbbffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbc0101a10 0x7fcbc0197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.468+0000 7fcbbffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbc0101a10 0x7fcbc0197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37998/0 (socket says 172.21.15.7:37998) 2024-06-10T01:19:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.468+0000 7fcbbffff640 1 -- 172.21.15.7:0/10825044 learned_addr learned my addr 172.21.15.7:0/10825044 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.468+0000 7fcbbf7fe640 1 --2- 172.21.15.7:0/10825044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcbc0102c00 0x7fcbc0198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.468+0000 7fcbc4f0d640 1 --2- 172.21.15.7:0/10825044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbc01035c0 0x7fcbc019f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.468+0000 7fcbbffff640 1 -- 172.21.15.7:0/10825044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbc01035c0 msgr2=0x7fcbc019f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.468+0000 7fcbbffff640 1 --2- 172.21.15.7:0/10825044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbc01035c0 0x7fcbc019f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.468+0000 7fcbbffff640 1 -- 172.21.15.7:0/10825044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcbc0102c00 msgr2=0x7fcbc0198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.468+0000 7fcbbffff640 1 --2- 172.21.15.7:0/10825044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcbc0102c00 0x7fcbc0198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.469+0000 7fcbbffff640 1 -- 172.21.15.7:0/10825044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc019f900 con 0x7fcbc0101a10 2024-06-10T01:19:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.469+0000 7fcbbffff640 1 --2- 172.21.15.7:0/10825044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbc0101a10 0x7fcbc0197cf0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fcbb000e9c0 tx=0x7fcbb000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.469+0000 7fcbbd7fa640 1 -- 172.21.15.7:0/10825044 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb000cd80 con 0x7fcbc0101a10 2024-06-10T01:19:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.469+0000 7fcbbd7fa640 1 -- 172.21.15.7:0/10825044 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbb00035f0 con 0x7fcbc0101a10 2024-06-10T01:19:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.469+0000 7fcbc6970640 1 -- 172.21.15.7:0/10825044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc019fbf0 con 0x7fcbc0101a10 2024-06-10T01:19:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.469+0000 7fcbbd7fa640 1 -- 172.21.15.7:0/10825044 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0010690 con 0x7fcbc0101a10 2024-06-10T01:19:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.469+0000 7fcbc6970640 1 -- 172.21.15.7:0/10825044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc01a0190 con 0x7fcbc0101a10 2024-06-10T01:19:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.471+0000 7fcbbd7fa640 1 -- 172.21.15.7:0/10825044 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fcbb00037a0 con 0x7fcbc0101a10 2024-06-10T01:19:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.471+0000 7fcbc9684740 1 Processor -- start 2024-06-10T01:19:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.471+0000 7fcbc9684740 1 -- start start 2024-06-10T01:19:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.472+0000 7fcbc9684740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563735383010 0x563735383430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.472+0000 7fcbc9684740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563735383a00 0x563735388500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.472+0000 7fcbc9684740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563735388b70 0x56373538b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.472+0000 7fcbc9684740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5637352e64f0 con 0x563735388b70 2024-06-10T01:19:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.472+0000 7fcbc9684740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563735284870 con 0x563735383a00 2024-06-10T01:19:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.472+0000 7fcbc9684740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5637352e24b0 con 0x563735383010 2024-06-10T01:19:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.472+0000 7fcbbf7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563735388b70 0x56373538b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.472+0000 7fcbc4f0d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563735383010 0x563735383430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.472+0000 7fcbbf7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563735388b70 0x56373538b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38014/0 (socket says 172.21.15.7:38014) 2024-06-10T01:19:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.472+0000 7fcbbf7fe640 1 -- 172.21.15.7:0/2597739988 learned_addr learned my addr 172.21.15.7:0/2597739988 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.472+0000 7fcbbffff640 1 --2- 172.21.15.7:0/2597739988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563735383a00 0x563735388500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.472+0000 7fcbbf7fe640 1 -- 172.21.15.7:0/2597739988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563735383010 msgr2=0x563735383430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.472+0000 7fcbbf7fe640 1 --2- 172.21.15.7:0/2597739988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563735383010 0x563735383430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.472+0000 7fcbbf7fe640 1 -- 172.21.15.7:0/2597739988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563735383a00 msgr2=0x563735388500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.472+0000 7fcbbf7fe640 1 --2- 172.21.15.7:0/2597739988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563735383a00 0x563735388500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.472+0000 7fcbbf7fe640 1 -- 172.21.15.7:0/2597739988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637352a09b0 con 0x563735388b70 2024-06-10T01:19:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.472+0000 7fcbbd7fa640 1 --2- 172.21.15.7:0/10825044 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcba80a2c00 0x7fcba80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.472+0000 7fcbc4f0d640 1 --2- 172.21.15.7:0/10825044 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcba80a2c00 0x7fcba80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.472+0000 7fcbbd7fa640 1 -- 172.21.15.7:0/10825044 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fcbb00cf810 con 0x7fcbc0101a10 2024-06-10T01:19:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.472+0000 7fcbbf7fe640 1 --2- 172.21.15.7:0/2597739988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563735388b70 0x56373538b000 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4009d20 tx=0x7fcbb4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff8a497ba1a20db server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.473+0000 7fcba67fc640 1 -- 172.21.15.7:0/2597739988 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb401e070 con 0x563735388b70 2024-06-10T01:19:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.473+0000 7fcba67fc640 1 -- 172.21.15.7:0/2597739988 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbb4019b10 con 0x563735388b70 2024-06-10T01:19:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.473+0000 7fcba67fc640 1 -- 172.21.15.7:0/2597739988 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb400ebf0 con 0x563735388b70 2024-06-10T01:19:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.473+0000 7fcbc9684740 1 -- 172.21.15.7:0/2597739988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563735388b70 msgr2=0x56373538b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.473+0000 7fcbc4f0d640 1 --2- 172.21.15.7:0/10825044 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcba80a2c00 0x7fcba80a50c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8005e30 tx=0x7fcbb803d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.473+0000 7fcbc9684740 1 --2- 172.21.15.7:0/2597739988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563735388b70 0x56373538b000 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4009d20 tx=0x7fcbb4011810 comp rx=0 tx=0).stop 2024-06-10T01:19:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.473+0000 7fcbc9684740 1 -- 172.21.15.7:0/2597739988 shutdown_connections 2024-06-10T01:19:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.473+0000 7fcbc9684740 1 --2- 172.21.15.7:0/2597739988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563735388b70 0x56373538b000 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.473+0000 7fcbc9684740 1 --2- 172.21.15.7:0/2597739988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563735383a00 0x563735388500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.473+0000 7fcbc9684740 1 --2- 172.21.15.7:0/2597739988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563735383010 0x563735383430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.473+0000 7fcbc9684740 1 -- 172.21.15.7:0/2597739988 >> 172.21.15.7:0/2597739988 conn(0x56373537e680 msgr2=0x56373537ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.473+0000 7fcbc9684740 1 -- 172.21.15.7:0/2597739988 shutdown_connections 2024-06-10T01:19:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.473+0000 7fcbc9684740 1 -- 172.21.15.7:0/2597739988 wait complete. 2024-06-10T01:19:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.474+0000 7fcbc9684740 1 -- start start 2024-06-10T01:19:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.474+0000 7fcbc9684740 1 Processor -- start 2024-06-10T01:19:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.474+0000 7fcbc9684740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563735383010 0x56373538d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.474+0000 7fcbc9684740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563735383a00 0x56373538d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.474+0000 7fcbc9684740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563735388b70 0x5637353947a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.474+0000 7fcbc9684740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56373538bb20 con 0x563735383010 2024-06-10T01:19:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.474+0000 7fcbc9684740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56373526fae0 con 0x563735383a00 2024-06-10T01:19:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.474+0000 7fcbc9684740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56373538bca0 con 0x563735388b70 2024-06-10T01:19:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.474+0000 7fcbc9684740 10 client.0 _ll_register_callbacks cb 0x5637352a89b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:19:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.474+0000 7fcbbffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563735383010 0x56373538d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.474+0000 7fcbbffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563735383010 0x56373538d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38022/0 (socket says 172.21.15.7:38022) 2024-06-10T01:19:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.474+0000 7fcbbffff640 1 -- 172.21.15.7:0/2839884990 learned_addr learned my addr 172.21.15.7:0/2839884990 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.474+0000 7fcbc4f0d640 1 --2- 172.21.15.7:0/2839884990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563735388b70 0x5637353947a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.474+0000 7fcbbf7fe640 1 --2- 172.21.15.7:0/2839884990 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563735383a00 0x56373538d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.474+0000 7fcbbffff640 1 -- 172.21.15.7:0/2839884990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563735388b70 msgr2=0x5637353947a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.474+0000 7fcbbffff640 1 --2- 172.21.15.7:0/2839884990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563735388b70 0x5637353947a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.474+0000 7fcbbffff640 1 -- 172.21.15.7:0/2839884990 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563735383a00 msgr2=0x56373538d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.474+0000 7fcbbffff640 1 --2- 172.21.15.7:0/2839884990 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563735383a00 0x56373538d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.474+0000 7fcbbffff640 1 -- 172.21.15.7:0/2839884990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563734ffa210 con 0x563735383010 2024-06-10T01:19:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.475+0000 7fcbbf7fe640 1 --2- 172.21.15.7:0/2839884990 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563735383a00 0x56373538d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:19:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.475+0000 7fcbbffff640 1 --2- 172.21.15.7:0/2839884990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563735383010 0x56373538d220 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fcbb00058c0 tx=0x7fcbb009bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25dbe4527766e17b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.475+0000 7fcb9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:19:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.475+0000 7fcb9f7fe640 1 -- 172.21.15.7:0/2839884990 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb009b2e0 con 0x563735383010 2024-06-10T01:19:59.474 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12545 23.62 MB/sec warmup 36 sec latency 7545.303 ms 2024-06-10T01:19:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.475+0000 7fcb9f7fe640 1 -- 172.21.15.7:0/2839884990 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fcbb0097950 con 0x563735383010 2024-06-10T01:19:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.475+0000 7fcbc9684740 1 -- 172.21.15.7:0/2839884990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbb4019b10 con 0x563735383010 2024-06-10T01:19:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.475+0000 7fcb9f7fe640 1 -- 172.21.15.7:0/2839884990 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fcbb00b9050 con 0x563735383010 2024-06-10T01:19:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.475+0000 7fcbc9684740 10 client.18936 fetch_fsmap learned FSMap version 44 2024-06-10T01:19:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.475+0000 7fcbc9684740 1 -- 172.21.15.7:0/2839884990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5637350c6620 con 0x563735383010 2024-06-10T01:19:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.475+0000 7fcb9f7fe640 1 -- 172.21.15.7:0/2839884990 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbb4019b10 con 0x563735383010 2024-06-10T01:19:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.475+0000 7fcb9f7fe640 1 -- 172.21.15.7:0/2839884990 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fcbb00b4070 con 0x563735383010 2024-06-10T01:19:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.475+0000 7fcbc9684740 10 client.18936 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:19:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.475+0000 7fcbc9684740 10 client.18936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:19:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.475+0000 7fcbc9684740 20 client.18936 populate_metadata read hostname 'smithi007' 2024-06-10T01:19:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.475+0000 7fcbc9684740 1 --2- 172.21.15.7:0/2839884990 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563735399150 0x56373539b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.475+0000 7fcbc9684740 4 client.18936 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:19:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.475+0000 7fcbc9684740 1 -- 172.21.15.7:0/2839884990 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637352f9f10 con 0x563735399150 2024-06-10T01:19:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.476+0000 7fcbbf7fe640 1 --2- 172.21.15.7:0/2839884990 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563735399150 0x56373539b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.476+0000 7fcbbf7fe640 1 --2- 172.21.15.7:0/2839884990 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563735399150 0x56373539b5e0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=414ae506b2d9d0f3 server_cookie=fdc1bab27c8d2586 in_seq=0 out_seq=0 2024-06-10T01:19:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.476+0000 7fcb9f7fe640 10 client.18936 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.477+0000 7fcb9f7fe640 1 -- 172.21.15.7:0/2839884990 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5637352f9f10 con 0x563735399150 2024-06-10T01:19:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.477+0000 7fcb9f7fe640 10 client.18936 handle_command_reply: tid=0 2024-06-10T01:19:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.477+0000 7fcbc9684740 1 client.18936 shutdown 2024-06-10T01:19:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.477+0000 7fcbc9684740 1 -- 172.21.15.7:0/2839884990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563735383010 msgr2=0x56373538d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.477+0000 7fcbc9684740 1 --2- 172.21.15.7:0/2839884990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563735383010 0x56373538d220 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fcbb00058c0 tx=0x7fcbb009bb60 comp rx=0 tx=0).stop 2024-06-10T01:19:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.477+0000 7fcbc9684740 1 -- 172.21.15.7:0/2839884990 shutdown_connections 2024-06-10T01:19:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.477+0000 7fcbc9684740 1 --2- 172.21.15.7:0/2839884990 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563735399150 0x56373539b5e0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.477+0000 7fcbc9684740 1 --2- 172.21.15.7:0/2839884990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563735388b70 0x5637353947a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.477+0000 7fcbc9684740 1 --2- 172.21.15.7:0/2839884990 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563735383a00 0x56373538d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.477+0000 7fcb9f7fe640 0 client.18936 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.477+0000 7fcbc9684740 1 --2- 172.21.15.7:0/2839884990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563735383010 0x56373538d220 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.477+0000 7fcbc9684740 1 -- 172.21.15.7:0/2839884990 >> 172.21.15.7:0/2839884990 conn(0x56373537e680 msgr2=0x56373537d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.477+0000 7fcbc9684740 1 -- 172.21.15.7:0/2839884990 shutdown_connections 2024-06-10T01:19:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.477+0000 7fcbc9684740 1 -- 172.21.15.7:0/2839884990 wait complete. 2024-06-10T01:19:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.477+0000 7fcbc9684740 20 client.18936 trim_cache size 0 max 16384 2024-06-10T01:19:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.493+0000 7fcbc9684740 1 Processor -- start 2024-06-10T01:19:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.493+0000 7fcbc9684740 1 -- start start 2024-06-10T01:19:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.493+0000 7fcbc9684740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563735383010 0x563735388210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.493+0000 7fcbc9684740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563735383a00 0x563735394760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.493+0000 7fcbc9684740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563735388b70 0x563735381590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.493+0000 7fcbc9684740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc01ace70 con 0x563735388b70 2024-06-10T01:19:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.493+0000 7fcbc9684740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc01aa6b0 con 0x563735383a00 2024-06-10T01:19:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.493+0000 7fcbc9684740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc01a8500 con 0x563735383010 2024-06-10T01:19:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.493+0000 7fcbc4f0d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563735388b70 0x563735381590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.493+0000 7fcbc4f0d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563735388b70 0x563735381590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38024/0 (socket says 172.21.15.7:38024) 2024-06-10T01:19:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.494+0000 7fcbc4f0d640 1 -- 172.21.15.7:0/1141836138 learned_addr learned my addr 172.21.15.7:0/1141836138 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.494+0000 7fcbbffff640 1 --2- 172.21.15.7:0/1141836138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563735383010 0x563735388210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.494+0000 7fcbbf7fe640 1 --2- 172.21.15.7:0/1141836138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563735383a00 0x563735394760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.494+0000 7fcbc4f0d640 1 -- 172.21.15.7:0/1141836138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563735383010 msgr2=0x563735388210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.494+0000 7fcbc4f0d640 1 --2- 172.21.15.7:0/1141836138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563735383010 0x563735388210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.494+0000 7fcbc4f0d640 1 -- 172.21.15.7:0/1141836138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563735383a00 msgr2=0x563735394760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.494+0000 7fcbc4f0d640 1 --2- 172.21.15.7:0/1141836138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563735383a00 0x563735394760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.494+0000 7fcbc4f0d640 1 -- 172.21.15.7:0/1141836138 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563735384690 con 0x563735388b70 2024-06-10T01:19:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.494+0000 7fcbbffff640 1 --2- 172.21.15.7:0/1141836138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563735383010 0x563735388210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:19:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.494+0000 7fcbc4f0d640 1 --2- 172.21.15.7:0/1141836138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563735388b70 0x563735381590 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8035000 tx=0x7fcbb8041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90756181d30ec1a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.494+0000 7fcba5ffb640 1 -- 172.21.15.7:0/1141836138 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb802fce0 con 0x563735388b70 2024-06-10T01:19:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.494+0000 7fcba5ffb640 1 -- 172.21.15.7:0/1141836138 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbb8056070 con 0x563735388b70 2024-06-10T01:19:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.494+0000 7fcba5ffb640 1 -- 172.21.15.7:0/1141836138 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb80467d0 con 0x563735388b70 2024-06-10T01:19:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.495+0000 7fcbc9684740 1 -- 172.21.15.7:0/1141836138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563735388b70 msgr2=0x563735381590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.495+0000 7fcbc9684740 1 --2- 172.21.15.7:0/1141836138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563735388b70 0x563735381590 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8035000 tx=0x7fcbb8041000 comp rx=0 tx=0).stop 2024-06-10T01:19:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.495+0000 7fcbc9684740 1 -- 172.21.15.7:0/1141836138 shutdown_connections 2024-06-10T01:19:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.495+0000 7fcbc9684740 1 --2- 172.21.15.7:0/1141836138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563735388b70 0x563735381590 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.495+0000 7fcbc9684740 1 --2- 172.21.15.7:0/1141836138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563735383a00 0x563735394760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.495+0000 7fcbc9684740 1 --2- 172.21.15.7:0/1141836138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563735383010 0x563735388210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.495+0000 7fcbc9684740 1 -- 172.21.15.7:0/1141836138 >> 172.21.15.7:0/1141836138 conn(0x56373537e680 msgr2=0x56373537ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.495+0000 7fcbc9684740 1 -- 172.21.15.7:0/1141836138 shutdown_connections 2024-06-10T01:19:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.495+0000 7fcbc9684740 1 -- 172.21.15.7:0/1141836138 wait complete. 2024-06-10T01:19:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.495+0000 7fcbc9684740 1 -- start start 2024-06-10T01:19:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.496+0000 7fcbc9684740 1 Processor -- start 2024-06-10T01:19:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.496+0000 7fcbc9684740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563735383010 0x563735391750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.496+0000 7fcbc9684740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563735383a00 0x563735391c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.496+0000 7fcbc9684740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563735388b70 0x563735401eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.496+0000 7fcbc9684740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc01a8b70 con 0x563735388b70 2024-06-10T01:19:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.496+0000 7fcbc9684740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc01a9fe0 con 0x563735383010 2024-06-10T01:19:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.496+0000 7fcbc9684740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc01a9240 con 0x563735383a00 2024-06-10T01:19:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.496+0000 7fcbc9684740 10 client.0 _ll_register_callbacks cb 0x7fcbb0003e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:19:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.496+0000 7fcbc4f0d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563735388b70 0x563735401eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.496+0000 7fcbc4f0d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563735388b70 0x563735401eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38028/0 (socket says 172.21.15.7:38028) 2024-06-10T01:19:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.496+0000 7fcbbffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563735383010 0x563735391750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.496+0000 7fcbc4f0d640 1 -- 172.21.15.7:0/3203976981 learned_addr learned my addr 172.21.15.7:0/3203976981 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:19:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.496+0000 7fcbbf7fe640 1 --2- 172.21.15.7:0/3203976981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563735383a00 0x563735391c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.496+0000 7fcbc4f0d640 1 -- 172.21.15.7:0/3203976981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563735383a00 msgr2=0x563735391c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.496+0000 7fcbc4f0d640 1 --2- 172.21.15.7:0/3203976981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563735383a00 0x563735391c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.496+0000 7fcbc4f0d640 1 -- 172.21.15.7:0/3203976981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563735383010 msgr2=0x563735391750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:19:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.496+0000 7fcbc4f0d640 1 --2- 172.21.15.7:0/3203976981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563735383010 0x563735391750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.496+0000 7fcbc4f0d640 1 -- 172.21.15.7:0/3203976981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563735402470 con 0x563735388b70 2024-06-10T01:19:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.496+0000 7fcbbffff640 1 --2- 172.21.15.7:0/3203976981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563735383010 0x563735391750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:19:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.496+0000 7fcbc4f0d640 1 --2- 172.21.15.7:0/3203976981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563735388b70 0x563735401eb0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8009ca0 tx=0x7fcbb802fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e3e1ee9a7eb059 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:19:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.496+0000 7fcb9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:19:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.497+0000 7fcb9f7fe640 1 -- 172.21.15.7:0/3203976981 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb80309d0 con 0x563735388b70 2024-06-10T01:19:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.497+0000 7fcb9f7fe640 1 -- 172.21.15.7:0/3203976981 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fcbb8046510 con 0x563735388b70 2024-06-10T01:19:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.497+0000 7fcbc9684740 1 -- 172.21.15.7:0/3203976981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbb8056070 con 0x563735388b70 2024-06-10T01:19:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.497+0000 7fcb9f7fe640 1 -- 172.21.15.7:0/3203976981 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fcbb8068050 con 0x563735388b70 2024-06-10T01:19:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.497+0000 7fcbc9684740 10 client.18942 fetch_fsmap learned FSMap version 44 2024-06-10T01:19:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.497+0000 7fcbc9684740 1 -- 172.21.15.7:0/3203976981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x563735403780 con 0x563735388b70 2024-06-10T01:19:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.497+0000 7fcb9f7fe640 1 -- 172.21.15.7:0/3203976981 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbb8056070 con 0x563735388b70 2024-06-10T01:19:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.497+0000 7fcb9f7fe640 1 -- 172.21.15.7:0/3203976981 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fcbb805a450 con 0x563735388b70 2024-06-10T01:19:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.497+0000 7fcbc9684740 10 client.18942 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:19:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.497+0000 7fcbc9684740 10 client.18942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:19:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.497+0000 7fcbc9684740 20 client.18942 populate_metadata read hostname 'smithi007' 2024-06-10T01:19:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.497+0000 7fcbc9684740 1 --2- 172.21.15.7:0/3203976981 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5637354041d0 0x5637354065f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:19:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.497+0000 7fcbc9684740 4 client.18942 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:19:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.497+0000 7fcbc9684740 1 -- 172.21.15.7:0/3203976981 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5637352f3c20 con 0x5637354041d0 2024-06-10T01:19:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.498+0000 7fcbbffff640 1 --2- 172.21.15.7:0/3203976981 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5637354041d0 0x5637354065f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:19:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.498+0000 7fcbbffff640 1 --2- 172.21.15.7:0/3203976981 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5637354041d0 0x5637354065f0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98789d2dcb4e613d server_cookie=bd43d4936b65bc77 in_seq=0 out_seq=0 2024-06-10T01:19:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.498+0000 7fcb9f7fe640 10 client.18942 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.700+0000 7fcb9f7fe640 1 -- 172.21.15.7:0/3203976981 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5637352f3c20 con 0x5637354041d0 2024-06-10T01:19:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.700+0000 7fcb9f7fe640 10 client.18942 handle_command_reply: tid=0 2024-06-10T01:19:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.700+0000 7fcbc9684740 1 client.18942 shutdown 2024-06-10T01:19:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.700+0000 7fcbc9684740 1 -- 172.21.15.7:0/3203976981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563735388b70 msgr2=0x563735401eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.700+0000 7fcbc9684740 1 --2- 172.21.15.7:0/3203976981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563735388b70 0x563735401eb0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8009ca0 tx=0x7fcbb802fd40 comp rx=0 tx=0).stop 2024-06-10T01:19:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.700+0000 7fcbc9684740 1 -- 172.21.15.7:0/3203976981 shutdown_connections 2024-06-10T01:19:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.700+0000 7fcbc9684740 1 --2- 172.21.15.7:0/3203976981 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5637354041d0 0x5637354065f0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.700+0000 7fcbc9684740 1 --2- 172.21.15.7:0/3203976981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563735388b70 0x563735401eb0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.700+0000 7fcb9f7fe640 0 client.18942 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:19:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.700+0000 7fcbc9684740 1 --2- 172.21.15.7:0/3203976981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563735383a00 0x563735391c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.700+0000 7fcbc9684740 1 --2- 172.21.15.7:0/3203976981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563735383010 0x563735391750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.700+0000 7fcbc9684740 1 -- 172.21.15.7:0/3203976981 >> 172.21.15.7:0/3203976981 conn(0x56373537e680 msgr2=0x563735384ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.700+0000 7fcbc9684740 1 -- 172.21.15.7:0/3203976981 shutdown_connections 2024-06-10T01:19:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.701+0000 7fcbc9684740 1 -- 172.21.15.7:0/3203976981 wait complete. 2024-06-10T01:19:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.701+0000 7fcbc9684740 20 client.18942 trim_cache size 0 max 16384 2024-06-10T01:19:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:19:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:19:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:19:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.701+0000 7fcbc6970640 1 -- 172.21.15.7:0/10825044 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcba80a2c00 msgr2=0x7fcba80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.701+0000 7fcbc6970640 1 --2- 172.21.15.7:0/10825044 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcba80a2c00 0x7fcba80a50c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8005e30 tx=0x7fcbb803d040 comp rx=0 tx=0).stop 2024-06-10T01:19:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.701+0000 7fcbc6970640 1 -- 172.21.15.7:0/10825044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbc0101a10 msgr2=0x7fcbc0197cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:19:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.701+0000 7fcbc6970640 1 --2- 172.21.15.7:0/10825044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbc0101a10 0x7fcbc0197cf0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fcbb000e9c0 tx=0x7fcbb000ee80 comp rx=0 tx=0).stop 2024-06-10T01:19:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.701+0000 7fcbc6970640 1 -- 172.21.15.7:0/10825044 shutdown_connections 2024-06-10T01:19:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.701+0000 7fcbc6970640 1 --2- 172.21.15.7:0/10825044 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcba80a2c00 0x7fcba80a50c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.701+0000 7fcbc6970640 1 --2- 172.21.15.7:0/10825044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbc01035c0 0x7fcbc019f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.701+0000 7fcbc6970640 1 --2- 172.21.15.7:0/10825044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcbc0102c00 0x7fcbc0198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.701+0000 7fcbc6970640 1 --2- 172.21.15.7:0/10825044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbc0101a10 0x7fcbc0197cf0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:19:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.701+0000 7fcbc6970640 1 -- 172.21.15.7:0/10825044 >> 172.21.15.7:0/10825044 conn(0x7fcbc00fcf80 msgr2=0x7fcbc0105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:19:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.701+0000 7fcbc6970640 1 -- 172.21.15.7:0/10825044 shutdown_connections 2024-06-10T01:19:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:19:59.701+0000 7fcbc6970640 1 -- 172.21.15.7:0/10825044 wait complete. 2024-06-10T01:20:00.468 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12545 22.98 MB/sec warmup 37 sec latency 8545.402 ms 2024-06-10T01:20:01.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.083+0000 7f9ffbcd8700 1 -- 172.21.15.7:0/1446274845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ff40746c0 msgr2=0x7f9ff4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:01.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.083+0000 7f9ffbcd8700 1 --2- 172.21.15.7:0/1446274845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ff40746c0 0x7f9ff4074aa0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400b0a0 tx=0x7f9fe400b3b0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.084+0000 7f9ffbcd8700 1 -- 172.21.15.7:0/1446274845 shutdown_connections 2024-06-10T01:20:01.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.084+0000 7f9ffbcd8700 1 --2- 172.21.15.7:0/1446274845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ff410ee90 0x7f9ff4111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.084+0000 7f9ffbcd8700 1 --2- 172.21.15.7:0/1446274845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ff40746c0 0x7f9ff4074aa0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.084+0000 7f9ffbcd8700 1 --2- 172.21.15.7:0/1446274845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ff4074fe0 0x7f9ff410e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.084+0000 7f9ffbcd8700 1 -- 172.21.15.7:0/1446274845 >> 172.21.15.7:0/1446274845 conn(0x7f9ff40795b0 msgr2=0x7f9ff40799c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:01.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.084+0000 7f9ffbcd8700 1 -- 172.21.15.7:0/1446274845 shutdown_connections 2024-06-10T01:20:01.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.084+0000 7f9ffbcd8700 1 -- 172.21.15.7:0/1446274845 wait complete. 2024-06-10T01:20:01.079 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:00 smithi007 ceph-mon[32398]: pgmap v852: 129 pgs: 129 active+clean; 363 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.7 MiB/s wr, 580 op/s 2024-06-10T01:20:01.079 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:00 smithi007 ceph-mon[32398]: overall HEALTH_OK 2024-06-10T01:20:01.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.084+0000 7f9ffbcd8700 1 Processor -- start 2024-06-10T01:20:01.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.085+0000 7f9ffbcd8700 1 -- start start 2024-06-10T01:20:01.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.085+0000 7f9ffbcd8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ff40746c0 0x7f9ff419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:01.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.085+0000 7f9ffbcd8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ff4074fe0 0x7f9ff419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:01.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.085+0000 7f9ffbcd8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ff410ee90 0x7f9ff419f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:01.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.085+0000 7f9ffbcd8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff4114970 con 0x7f9ff40746c0 2024-06-10T01:20:01.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.085+0000 7f9ffbcd8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff41147f0 con 0x7f9ff410ee90 2024-06-10T01:20:01.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.085+0000 7f9ffbcd8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff4114af0 con 0x7f9ff4074fe0 2024-06-10T01:20:01.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.085+0000 7f9ff9a74700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ff40746c0 0x7f9ff419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:01.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.085+0000 7f9ff9a74700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ff40746c0 0x7f9ff419adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38040/0 (socket says 172.21.15.7:38040) 2024-06-10T01:20:01.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.085+0000 7f9ff9a74700 1 -- 172.21.15.7:0/3565514853 learned_addr learned my addr 172.21.15.7:0/3565514853 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:01.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.085+0000 7f9ff9273700 1 --2- 172.21.15.7:0/3565514853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ff4074fe0 0x7f9ff419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:01.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.085+0000 7f9ff9a74700 1 -- 172.21.15.7:0/3565514853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ff4074fe0 msgr2=0x7f9ff419b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:01.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.085+0000 7f9ff9a74700 1 --2- 172.21.15.7:0/3565514853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ff4074fe0 0x7f9ff419b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.085+0000 7f9ff9a74700 1 -- 172.21.15.7:0/3565514853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ff410ee90 msgr2=0x7f9ff419f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:01.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.085+0000 7f9ffa275700 1 --2- 172.21.15.7:0/3565514853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ff410ee90 0x7f9ff419f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:01.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.085+0000 7f9ff9a74700 1 --2- 172.21.15.7:0/3565514853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ff410ee90 0x7f9ff419f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.085+0000 7f9ff9a74700 1 -- 172.21.15.7:0/3565514853 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff419fdd0 con 0x7f9ff40746c0 2024-06-10T01:20:01.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.085+0000 7f9ffa275700 1 --2- 172.21.15.7:0/3565514853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ff410ee90 0x7f9ff419f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:20:01.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.086+0000 7f9ff9a74700 1 --2- 172.21.15.7:0/3565514853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ff40746c0 0x7f9ff419adc0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400b070 tx=0x7f9fe4004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:01.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.086+0000 7f9ff2ffd700 1 -- 172.21.15.7:0/3565514853 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4021070 con 0x7f9ff40746c0 2024-06-10T01:20:01.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.086+0000 7f9ffbcd8700 1 -- 172.21.15.7:0/3565514853 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff41a0060 con 0x7f9ff40746c0 2024-06-10T01:20:01.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.086+0000 7f9ff2ffd700 1 -- 172.21.15.7:0/3565514853 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fe401d930 con 0x7f9ff40746c0 2024-06-10T01:20:01.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.086+0000 7f9ff2ffd700 1 -- 172.21.15.7:0/3565514853 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4005440 con 0x7f9ff40746c0 2024-06-10T01:20:01.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.086+0000 7f9ffbcd8700 1 -- 172.21.15.7:0/3565514853 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff41a7940 con 0x7f9ff40746c0 2024-06-10T01:20:01.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.088+0000 7f9ff2ffd700 1 -- 172.21.15.7:0/3565514853 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9fe4024070 con 0x7f9ff40746c0 2024-06-10T01:20:01.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.089+0000 7f9ff2ffd700 1 --2- 172.21.15.7:0/3565514853 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9fdc0a2c20 0x7f9fdc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:01.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.089+0000 7f9ff9273700 1 --2- 172.21.15.7:0/3565514853 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9fdc0a2c20 0x7f9fdc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:01.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.089+0000 7f9ff2ffd700 1 -- 172.21.15.7:0/3565514853 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f9fe400f020 con 0x7f9ff40746c0 2024-06-10T01:20:01.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.089+0000 7f9ff9273700 1 --2- 172.21.15.7:0/3565514853 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9fdc0a2c20 0x7f9fdc0a50e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8005ea0 tx=0x7f9fe8005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:01.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.090+0000 7fa00655bb80 1 Processor -- start 2024-06-10T01:20:01.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.090+0000 7fa00655bb80 1 -- start start 2024-06-10T01:20:01.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.090+0000 7fa00655bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a86e467f60 0x55a86e468340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:01.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.090+0000 7fa00655bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86e468880 0x55a86e462260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:01.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.090+0000 7fa00655bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a86e4627a0 0x55a86e462bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:01.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.090+0000 7fa00655bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a86e354f60 con 0x55a86e4627a0 2024-06-10T01:20:01.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.090+0000 7fa00655bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a86e3b3850 con 0x55a86e467f60 2024-06-10T01:20:01.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.090+0000 7fa00655bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a86e31b460 con 0x55a86e468880 2024-06-10T01:20:01.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.090+0000 7f9ffa275700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a86e4627a0 0x55a86e462bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:01.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.090+0000 7f9ffa275700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a86e4627a0 0x55a86e462bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38054/0 (socket says 172.21.15.7:38054) 2024-06-10T01:20:01.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.090+0000 7f9ffa275700 1 -- 172.21.15.7:0/2769877966 learned_addr learned my addr 172.21.15.7:0/2769877966 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:01.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.090+0000 7f9ff9a74700 1 --2- 172.21.15.7:0/2769877966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a86e467f60 0x55a86e468340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:01.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.090+0000 7f9ff9273700 1 --2- 172.21.15.7:0/2769877966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86e468880 0x55a86e462260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:01.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.091+0000 7f9ffa275700 1 -- 172.21.15.7:0/2769877966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86e468880 msgr2=0x55a86e462260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:01.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.091+0000 7f9ffa275700 1 --2- 172.21.15.7:0/2769877966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86e468880 0x55a86e462260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.091+0000 7f9ffa275700 1 -- 172.21.15.7:0/2769877966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a86e467f60 msgr2=0x55a86e468340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:01.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.091+0000 7f9ffa275700 1 --2- 172.21.15.7:0/2769877966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a86e467f60 0x55a86e468340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.091+0000 7f9ffa275700 1 -- 172.21.15.7:0/2769877966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a86e3028d0 con 0x55a86e4627a0 2024-06-10T01:20:01.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.091+0000 7f9ff9273700 1 --2- 172.21.15.7:0/2769877966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86e468880 0x55a86e462260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:20:01.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.091+0000 7f9ffa275700 1 --2- 172.21.15.7:0/2769877966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a86e4627a0 0x55a86e462bc0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f9fec009ba0 tx=0x7f9fec009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49de818eb69eb10f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:01.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.091+0000 7f9fdbfff700 1 -- 172.21.15.7:0/2769877966 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec016070 con 0x55a86e4627a0 2024-06-10T01:20:01.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.091+0000 7f9fdbfff700 1 -- 172.21.15.7:0/2769877966 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fec011470 con 0x55a86e4627a0 2024-06-10T01:20:01.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.091+0000 7f9fdbfff700 1 -- 172.21.15.7:0/2769877966 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec010630 con 0x55a86e4627a0 2024-06-10T01:20:01.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.092+0000 7fa00655bb80 1 -- 172.21.15.7:0/2769877966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a86e4627a0 msgr2=0x55a86e462bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:01.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.092+0000 7fa00655bb80 1 --2- 172.21.15.7:0/2769877966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a86e4627a0 0x55a86e462bc0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f9fec009ba0 tx=0x7f9fec009eb0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.092+0000 7fa00655bb80 1 -- 172.21.15.7:0/2769877966 shutdown_connections 2024-06-10T01:20:01.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.092+0000 7fa00655bb80 1 --2- 172.21.15.7:0/2769877966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a86e4627a0 0x55a86e462bc0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.092+0000 7fa00655bb80 1 --2- 172.21.15.7:0/2769877966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a86e467f60 0x55a86e468340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.092+0000 7fa00655bb80 1 --2- 172.21.15.7:0/2769877966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86e468880 0x55a86e462260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.092+0000 7fa00655bb80 1 -- 172.21.15.7:0/2769877966 >> 172.21.15.7:0/2769877966 conn(0x55a86e38f080 msgr2=0x55a86e45d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:01.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.092+0000 7fa00655bb80 1 -- 172.21.15.7:0/2769877966 shutdown_connections 2024-06-10T01:20:01.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.092+0000 7fa00655bb80 1 -- 172.21.15.7:0/2769877966 wait complete. 2024-06-10T01:20:01.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.092+0000 7fa00655bb80 1 -- start start 2024-06-10T01:20:01.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.092+0000 7fa00655bb80 1 Processor -- start 2024-06-10T01:20:01.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.093+0000 7fa00655bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86e4627a0 0x55a86e46e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:01.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.093+0000 7fa00655bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a86e467f60 0x55a86e46b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:01.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.093+0000 7fa00655bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a86e468880 0x55a86e46be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:01.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.093+0000 7fa00655bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a86e353fc0 con 0x55a86e467f60 2024-06-10T01:20:01.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.093+0000 7fa00655bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a86e3af810 con 0x55a86e468880 2024-06-10T01:20:01.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.093+0000 7fa00655bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a86e46f260 con 0x55a86e4627a0 2024-06-10T01:20:01.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.093+0000 7fa00655bb80 10 client.0 _ll_register_callbacks cb 0x55a86e3495b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:20:01.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.093+0000 7f9ff9273700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a86e467f60 0x55a86e46b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:01.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.093+0000 7f9ff9273700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a86e467f60 0x55a86e46b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38064/0 (socket says 172.21.15.7:38064) 2024-06-10T01:20:01.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.093+0000 7f9ff9273700 1 -- 172.21.15.7:0/3497595338 learned_addr learned my addr 172.21.15.7:0/3497595338 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:01.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.093+0000 7f9ffa275700 1 --2- 172.21.15.7:0/3497595338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a86e468880 0x55a86e46be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:01.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.093+0000 7f9ff9a74700 1 --2- 172.21.15.7:0/3497595338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86e4627a0 0x55a86e46e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:01.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.093+0000 7f9ff9273700 1 -- 172.21.15.7:0/3497595338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86e4627a0 msgr2=0x55a86e46e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:01.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.093+0000 7f9ff9273700 1 --2- 172.21.15.7:0/3497595338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86e4627a0 0x55a86e46e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.093+0000 7f9ff9273700 1 -- 172.21.15.7:0/3497595338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a86e468880 msgr2=0x55a86e46be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:01.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.093+0000 7f9ff9273700 1 --2- 172.21.15.7:0/3497595338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a86e468880 0x55a86e46be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.093+0000 7f9ff9273700 1 -- 172.21.15.7:0/3497595338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a86e379750 con 0x55a86e467f60 2024-06-10T01:20:01.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.093+0000 7f9ff9273700 1 --2- 172.21.15.7:0/3497595338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a86e467f60 0x55a86e46b8c0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f9fe800a9e0 tx=0x7f9fe80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af72b6276a619f13 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:01.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.093+0000 7f9fd97fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:20:01.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.094+0000 7f9fd97fa700 1 -- 172.21.15.7:0/3497595338 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe800fa50 con 0x55a86e467f60 2024-06-10T01:20:01.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.094+0000 7f9fd97fa700 1 -- 172.21.15.7:0/3497595338 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f9fe800fbf0 con 0x55a86e467f60 2024-06-10T01:20:01.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.094+0000 7fa00655bb80 1 -- 172.21.15.7:0/3497595338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fec011470 con 0x55a86e467f60 2024-06-10T01:20:01.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.094+0000 7f9fd97fa700 1 -- 172.21.15.7:0/3497595338 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f9fe802f050 con 0x55a86e467f60 2024-06-10T01:20:01.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.094+0000 7fa00655bb80 10 client.18954 fetch_fsmap learned FSMap version 44 2024-06-10T01:20:01.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.094+0000 7fa00655bb80 1 -- 172.21.15.7:0/3497595338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55a86e479a50 con 0x55a86e467f60 2024-06-10T01:20:01.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.094+0000 7f9fd97fa700 1 -- 172.21.15.7:0/3497595338 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fec011470 con 0x55a86e467f60 2024-06-10T01:20:01.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.094+0000 7f9fd97fa700 1 -- 172.21.15.7:0/3497595338 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f9fe8026400 con 0x55a86e467f60 2024-06-10T01:20:01.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.094+0000 7fa00655bb80 10 client.18954 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:20:01.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.094+0000 7fa00655bb80 10 client.18954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:20:01.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.094+0000 7fa00655bb80 20 client.18954 populate_metadata read hostname 'smithi007' 2024-06-10T01:20:01.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.094+0000 7fa00655bb80 1 --2- 172.21.15.7:0/3497595338 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a86e47c0e0 0x55a86e47e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:01.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.094+0000 7fa00655bb80 4 client.18954 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:20:01.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.094+0000 7fa00655bb80 1 -- 172.21.15.7:0/3497595338 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a86e3c0d50 con 0x55a86e47c0e0 2024-06-10T01:20:01.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.095+0000 7f9ff9a74700 1 --2- 172.21.15.7:0/3497595338 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a86e47c0e0 0x55a86e47e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:01.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.095+0000 7f9ff9a74700 1 --2- 172.21.15.7:0/3497595338 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a86e47c0e0 0x55a86e47e4f0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32a7a1606c09bea7 server_cookie=58a3f46985780484 in_seq=0 out_seq=0 2024-06-10T01:20:01.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.095+0000 7f9fd97fa700 10 client.18954 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:01.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.096+0000 7f9fd97fa700 1 -- 172.21.15.7:0/3497595338 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a86e3c0d50 con 0x55a86e47c0e0 2024-06-10T01:20:01.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.096+0000 7f9fd97fa700 10 client.18954 handle_command_reply: tid=0 2024-06-10T01:20:01.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.096+0000 7fa00655bb80 1 client.18954 shutdown 2024-06-10T01:20:01.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.096+0000 7fa00655bb80 1 -- 172.21.15.7:0/3497595338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a86e467f60 msgr2=0x55a86e46b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:01.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.096+0000 7fa00655bb80 1 --2- 172.21.15.7:0/3497595338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a86e467f60 0x55a86e46b8c0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f9fe800a9e0 tx=0x7f9fe80047c0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.096+0000 7fa00655bb80 1 -- 172.21.15.7:0/3497595338 shutdown_connections 2024-06-10T01:20:01.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.096+0000 7fa00655bb80 1 --2- 172.21.15.7:0/3497595338 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a86e47c0e0 0x55a86e47e4f0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.096+0000 7fa00655bb80 1 --2- 172.21.15.7:0/3497595338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a86e468880 0x55a86e46be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.096+0000 7fa00655bb80 1 --2- 172.21.15.7:0/3497595338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86e4627a0 0x55a86e46e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.096+0000 7f9fd97fa700 0 client.18954 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:01.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.096+0000 7fa00655bb80 1 --2- 172.21.15.7:0/3497595338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a86e467f60 0x55a86e46b8c0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.096+0000 7fa00655bb80 1 -- 172.21.15.7:0/3497595338 >> 172.21.15.7:0/3497595338 conn(0x55a86e38f080 msgr2=0x55a86e45c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:01.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.096+0000 7fa00655bb80 1 -- 172.21.15.7:0/3497595338 shutdown_connections 2024-06-10T01:20:01.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.096+0000 7fa00655bb80 1 -- 172.21.15.7:0/3497595338 wait complete. 2024-06-10T01:20:01.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.096+0000 7fa00655bb80 20 client.18954 trim_cache size 0 max 16384 2024-06-10T01:20:01.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.117+0000 7fa00655bb80 1 Processor -- start 2024-06-10T01:20:01.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.117+0000 7fa00655bb80 1 -- start start 2024-06-10T01:20:01.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.117+0000 7fa00655bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86e4627a0 0x55a86e462260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:01.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.117+0000 7fa00655bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a86e467f60 0x55a86e46ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:01.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.117+0000 7fa00655bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a86e468880 0x55a86e469090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:01.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.117+0000 7fa00655bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff41b2cf0 con 0x55a86e468880 2024-06-10T01:20:01.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.117+0000 7fa00655bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff41b04f0 con 0x55a86e467f60 2024-06-10T01:20:01.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.117+0000 7fa00655bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff41ae340 con 0x55a86e4627a0 2024-06-10T01:20:01.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.118+0000 7f9ffa275700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a86e468880 0x55a86e469090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:01.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.118+0000 7f9ffa275700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a86e468880 0x55a86e469090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38076/0 (socket says 172.21.15.7:38076) 2024-06-10T01:20:01.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.118+0000 7f9ffa275700 1 -- 172.21.15.7:0/3609647942 learned_addr learned my addr 172.21.15.7:0/3609647942 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:01.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.118+0000 7f9ff9273700 1 --2- 172.21.15.7:0/3609647942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a86e467f60 0x55a86e46ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:01.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.118+0000 7f9ff9a74700 1 --2- 172.21.15.7:0/3609647942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86e4627a0 0x55a86e462260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:01.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.118+0000 7f9ffa275700 1 -- 172.21.15.7:0/3609647942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86e4627a0 msgr2=0x55a86e462260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:01.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.118+0000 7f9ffa275700 1 --2- 172.21.15.7:0/3609647942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86e4627a0 0x55a86e462260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.118+0000 7f9ffa275700 1 -- 172.21.15.7:0/3609647942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a86e467f60 msgr2=0x55a86e46ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:01.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.118+0000 7f9ffa275700 1 --2- 172.21.15.7:0/3609647942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a86e467f60 0x55a86e46ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.118+0000 7f9ffa275700 1 -- 172.21.15.7:0/3609647942 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a86e469e30 con 0x55a86e468880 2024-06-10T01:20:01.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.118+0000 7f9ffa275700 1 --2- 172.21.15.7:0/3609647942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a86e468880 0x55a86e469090 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f9fec011e90 tx=0x7f9fec00eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285cc70e24ae60bf server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:01.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.118+0000 7f9fdb7fe700 1 -- 172.21.15.7:0/3609647942 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec016040 con 0x55a86e468880 2024-06-10T01:20:01.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.118+0000 7f9fdb7fe700 1 -- 172.21.15.7:0/3609647942 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fec029070 con 0x55a86e468880 2024-06-10T01:20:01.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.118+0000 7f9fdb7fe700 1 -- 172.21.15.7:0/3609647942 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec004ad0 con 0x55a86e468880 2024-06-10T01:20:01.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.119+0000 7fa00655bb80 1 -- 172.21.15.7:0/3609647942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a86e468880 msgr2=0x55a86e469090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:01.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.119+0000 7fa00655bb80 1 --2- 172.21.15.7:0/3609647942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a86e468880 0x55a86e469090 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f9fec011e90 tx=0x7f9fec00eae0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.119+0000 7fa00655bb80 1 -- 172.21.15.7:0/3609647942 shutdown_connections 2024-06-10T01:20:01.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.119+0000 7fa00655bb80 1 --2- 172.21.15.7:0/3609647942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a86e468880 0x55a86e469090 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.119+0000 7fa00655bb80 1 --2- 172.21.15.7:0/3609647942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86e4627a0 0x55a86e462260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.119+0000 7fa00655bb80 1 --2- 172.21.15.7:0/3609647942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a86e467f60 0x55a86e46ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.119+0000 7fa00655bb80 1 -- 172.21.15.7:0/3609647942 >> 172.21.15.7:0/3609647942 conn(0x55a86e38f080 msgr2=0x55a86e46d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:01.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.119+0000 7fa00655bb80 1 -- 172.21.15.7:0/3609647942 shutdown_connections 2024-06-10T01:20:01.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.119+0000 7fa00655bb80 1 -- 172.21.15.7:0/3609647942 wait complete. 2024-06-10T01:20:01.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.119+0000 7fa00655bb80 1 -- start start 2024-06-10T01:20:01.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.119+0000 7fa00655bb80 1 Processor -- start 2024-06-10T01:20:01.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.120+0000 7fa00655bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a86e4627a0 0x55a86e474550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:01.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.120+0000 7fa00655bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a86e467f60 0x55a86e474a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:01.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.120+0000 7fa00655bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86e468880 0x55a86e471520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:01.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.120+0000 7fa00655bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff41ae9b0 con 0x55a86e4627a0 2024-06-10T01:20:01.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.120+0000 7fa00655bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff41afe20 con 0x55a86e467f60 2024-06-10T01:20:01.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.120+0000 7fa00655bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff41af080 con 0x55a86e468880 2024-06-10T01:20:01.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.120+0000 7fa00655bb80 10 client.0 _ll_register_callbacks cb 0x55a86e3495b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:20:01.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.120+0000 7f9ff9a74700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a86e4627a0 0x55a86e474550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:01.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.120+0000 7f9ff9a74700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a86e4627a0 0x55a86e474550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38080/0 (socket says 172.21.15.7:38080) 2024-06-10T01:20:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.120+0000 7f9ff9a74700 1 -- 172.21.15.7:0/291764556 learned_addr learned my addr 172.21.15.7:0/291764556 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.120+0000 7f9ff9273700 1 --2- 172.21.15.7:0/291764556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a86e467f60 0x55a86e474a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.120+0000 7f9ffa275700 1 --2- 172.21.15.7:0/291764556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86e468880 0x55a86e471520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.120+0000 7f9ff9a74700 1 -- 172.21.15.7:0/291764556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86e468880 msgr2=0x55a86e471520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.120+0000 7f9ff9a74700 1 --2- 172.21.15.7:0/291764556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86e468880 0x55a86e471520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.120+0000 7f9ff9a74700 1 -- 172.21.15.7:0/291764556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a86e467f60 msgr2=0x55a86e474a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.120+0000 7f9ff9a74700 1 --2- 172.21.15.7:0/291764556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a86e467f60 0x55a86e474a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.120+0000 7f9ff9a74700 1 -- 172.21.15.7:0/291764556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a86e471a60 con 0x55a86e4627a0 2024-06-10T01:20:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.120+0000 7f9ffa275700 1 --2- 172.21.15.7:0/291764556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86e468880 0x55a86e471520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:20:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.120+0000 7f9ff9a74700 1 --2- 172.21.15.7:0/291764556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a86e4627a0 0x55a86e474550 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4000f80 tx=0x7f9fe4009c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc37ce3176da838 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.121+0000 7f9fd97fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:20:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.121+0000 7f9fd97fa700 1 -- 172.21.15.7:0/291764556 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4086030 con 0x55a86e4627a0 2024-06-10T01:20:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.121+0000 7f9fd97fa700 1 -- 172.21.15.7:0/291764556 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f9fe409b520 con 0x55a86e4627a0 2024-06-10T01:20:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.121+0000 7fa00655bb80 1 -- 172.21.15.7:0/291764556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fec029070 con 0x55a86e4627a0 2024-06-10T01:20:01.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.121+0000 7f9fd97fa700 1 -- 172.21.15.7:0/291764556 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f9fe40074e0 con 0x55a86e4627a0 2024-06-10T01:20:01.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.121+0000 7fa00655bb80 10 client.18960 fetch_fsmap learned FSMap version 44 2024-06-10T01:20:01.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.121+0000 7fa00655bb80 1 -- 172.21.15.7:0/291764556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55a86e4e75b0 con 0x55a86e4627a0 2024-06-10T01:20:01.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.121+0000 7f9fd97fa700 1 -- 172.21.15.7:0/291764556 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fec029070 con 0x55a86e4627a0 2024-06-10T01:20:01.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.121+0000 7f9fd97fa700 1 -- 172.21.15.7:0/291764556 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f9fe4011070 con 0x55a86e4627a0 2024-06-10T01:20:01.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.121+0000 7fa00655bb80 10 client.18960 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:20:01.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.121+0000 7fa00655bb80 10 client.18960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:20:01.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.121+0000 7fa00655bb80 20 client.18960 populate_metadata read hostname 'smithi007' 2024-06-10T01:20:01.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.121+0000 7fa00655bb80 1 --2- 172.21.15.7:0/291764556 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a86e4e7c60 0x55a86e4ea050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:01.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.121+0000 7fa00655bb80 4 client.18960 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:20:01.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.121+0000 7fa00655bb80 1 -- 172.21.15.7:0/291764556 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a86e3bb2d0 con 0x55a86e4e7c60 2024-06-10T01:20:01.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.122+0000 7f9ff9273700 1 --2- 172.21.15.7:0/291764556 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a86e4e7c60 0x55a86e4ea050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:01.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.122+0000 7f9ff9273700 1 --2- 172.21.15.7:0/291764556 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a86e4e7c60 0x55a86e4ea050 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86cbe06ccac8baac server_cookie=d7051283bff45334 in_seq=0 out_seq=0 2024-06-10T01:20:01.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.122+0000 7f9fd97fa700 10 client.18960 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:01.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.122+0000 7f9fd97fa700 1 -- 172.21.15.7:0/291764556 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a86e3bb2d0 con 0x55a86e4e7c60 2024-06-10T01:20:01.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.122+0000 7f9fd97fa700 10 client.18960 handle_command_reply: tid=0 2024-06-10T01:20:01.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.122+0000 7fa00655bb80 1 client.18960 shutdown 2024-06-10T01:20:01.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.122+0000 7fa00655bb80 1 -- 172.21.15.7:0/291764556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a86e4627a0 msgr2=0x55a86e474550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:01.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.122+0000 7fa00655bb80 1 --2- 172.21.15.7:0/291764556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a86e4627a0 0x55a86e474550 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4000f80 tx=0x7f9fe4009c70 comp rx=0 tx=0).stop 2024-06-10T01:20:01.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.123+0000 7fa00655bb80 1 -- 172.21.15.7:0/291764556 shutdown_connections 2024-06-10T01:20:01.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.123+0000 7fa00655bb80 1 --2- 172.21.15.7:0/291764556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a86e468880 0x55a86e471520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.123+0000 7fa00655bb80 1 --2- 172.21.15.7:0/291764556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a86e4627a0 0x55a86e474550 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.123+0000 7fa00655bb80 1 --2- 172.21.15.7:0/291764556 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a86e4e7c60 0x55a86e4ea050 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.123+0000 7fa00655bb80 1 --2- 172.21.15.7:0/291764556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a86e467f60 0x55a86e474a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.123+0000 7f9fd97fa700 0 client.18960 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:01.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.123+0000 7fa00655bb80 1 -- 172.21.15.7:0/291764556 >> 172.21.15.7:0/291764556 conn(0x55a86e38f080 msgr2=0x55a86e466a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:01.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.123+0000 7fa00655bb80 1 -- 172.21.15.7:0/291764556 shutdown_connections 2024-06-10T01:20:01.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.123+0000 7fa00655bb80 1 -- 172.21.15.7:0/291764556 wait complete. 2024-06-10T01:20:01.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.123+0000 7fa00655bb80 20 client.18960 trim_cache size 0 max 16384 2024-06-10T01:20:01.124 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:20:01.124 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-10T01:20:01.124 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-10T01:20:01.124 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:20:01.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.123+0000 7f9ffbcd8700 1 -- 172.21.15.7:0/3565514853 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9fdc0a2c20 msgr2=0x7f9fdc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:01.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.123+0000 7f9ffbcd8700 1 --2- 172.21.15.7:0/3565514853 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9fdc0a2c20 0x7f9fdc0a50e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8005ea0 tx=0x7f9fe8005e30 comp rx=0 tx=0).stop 2024-06-10T01:20:01.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.123+0000 7f9ffbcd8700 1 -- 172.21.15.7:0/3565514853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ff40746c0 msgr2=0x7f9ff419adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:01.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.123+0000 7f9ffbcd8700 1 --2- 172.21.15.7:0/3565514853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ff40746c0 0x7f9ff419adc0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400b070 tx=0x7f9fe4004ef0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.123+0000 7f9ffbcd8700 1 -- 172.21.15.7:0/3565514853 shutdown_connections 2024-06-10T01:20:01.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.123+0000 7f9ffbcd8700 1 --2- 172.21.15.7:0/3565514853 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9fdc0a2c20 0x7f9fdc0a50e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.123+0000 7f9ffbcd8700 1 --2- 172.21.15.7:0/3565514853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ff410ee90 0x7f9ff419f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.123+0000 7f9ffbcd8700 1 --2- 172.21.15.7:0/3565514853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ff40746c0 0x7f9ff419adc0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.123+0000 7f9ffbcd8700 1 --2- 172.21.15.7:0/3565514853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ff4074fe0 0x7f9ff419b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:01.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.123+0000 7f9ffbcd8700 1 -- 172.21.15.7:0/3565514853 >> 172.21.15.7:0/3565514853 conn(0x7f9ff40795b0 msgr2=0x7f9ff4100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:01.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.123+0000 7f9ffbcd8700 1 -- 172.21.15.7:0/3565514853 shutdown_connections 2024-06-10T01:20:01.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:01.123+0000 7f9ffbcd8700 1 -- 172.21.15.7:0/3565514853 wait complete. 2024-06-10T01:20:01.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:00 smithi175 ceph-mon[36636]: pgmap v852: 129 pgs: 129 active+clean; 363 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.7 MiB/s wr, 580 op/s 2024-06-10T01:20:01.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:00 smithi175 ceph-mon[36636]: overall HEALTH_OK 2024-06-10T01:20:01.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:00 smithi078 ceph-mon[36683]: pgmap v852: 129 pgs: 129 active+clean; 363 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.7 MiB/s wr, 580 op/s 2024-06-10T01:20:01.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:00 smithi078 ceph-mon[36683]: overall HEALTH_OK 2024-06-10T01:20:01.469 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12545 22.37 MB/sec warmup 38 sec latency 9545.493 ms 2024-06-10T01:20:01.833 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:20:01.833 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:20:02.086 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:20:02.469 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12545 21.80 MB/sec warmup 39 sec latency 10545.567 ms 2024-06-10T01:20:03.195 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:02 smithi007 ceph-mon[32398]: pgmap v853: 129 pgs: 129 active+clean; 359 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 297 op/s 2024-06-10T01:20:03.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:02 smithi175 ceph-mon[36636]: pgmap v853: 129 pgs: 129 active+clean; 359 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 297 op/s 2024-06-10T01:20:03.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:02 smithi078 ceph-mon[36683]: pgmap v853: 129 pgs: 129 active+clean; 359 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 297 op/s 2024-06-10T01:20:03.469 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12545 21.25 MB/sec warmup 40 sec latency 11545.678 ms 2024-06-10T01:20:03.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.778+0000 7f607301a700 1 -- 172.21.15.7:0/4101747488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f606c101fc0 msgr2=0x7f606c106010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:03.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.778+0000 7f607301a700 1 --2- 172.21.15.7:0/4101747488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f606c101fc0 0x7f606c106010 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f60540099c0 tx=0x7f6054009cd0 comp rx=0 tx=0).stop 2024-06-10T01:20:03.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.778+0000 7f607301a700 1 -- 172.21.15.7:0/4101747488 shutdown_connections 2024-06-10T01:20:03.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.778+0000 7f607301a700 1 --2- 172.21.15.7:0/4101747488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f606c1066f0 0x7f606c10e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:03.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.778+0000 7f607301a700 1 --2- 172.21.15.7:0/4101747488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f606c101610 0x7f606c1019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:03.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.778+0000 7f607301a700 1 --2- 172.21.15.7:0/4101747488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f606c101fc0 0x7f606c106010 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:03.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.778+0000 7f607301a700 1 -- 172.21.15.7:0/4101747488 >> 172.21.15.7:0/4101747488 conn(0x7f606c078430 msgr2=0x7f606c078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:03.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.778+0000 7f607301a700 1 -- 172.21.15.7:0/4101747488 shutdown_connections 2024-06-10T01:20:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.778+0000 7f607301a700 1 -- 172.21.15.7:0/4101747488 wait complete. 2024-06-10T01:20:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.778+0000 7f607301a700 1 Processor -- start 2024-06-10T01:20:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.778+0000 7f607301a700 1 -- start start 2024-06-10T01:20:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.779+0000 7f607301a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f606c101610 0x7f606c1030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.779+0000 7f607301a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f606c101fc0 0x7f606c1035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.779+0000 7f607301a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f606c1066f0 0x7f606c103b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.779+0000 7f607301a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f606c1117b0 con 0x7f606c101fc0 2024-06-10T01:20:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.779+0000 7f607301a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f606c111630 con 0x7f606c101610 2024-06-10T01:20:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.779+0000 7f607301a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f606c111930 con 0x7f606c1066f0 2024-06-10T01:20:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.779+0000 7f606bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f606c101fc0 0x7f606c1035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.779+0000 7f606bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f606c101fc0 0x7f606c1035f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55534/0 (socket says 172.21.15.7:55534) 2024-06-10T01:20:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.779+0000 7f606bfff700 1 -- 172.21.15.7:0/2074667256 learned_addr learned my addr 172.21.15.7:0/2074667256 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.779+0000 7f6070db6700 1 --2- 172.21.15.7:0/2074667256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f606c101610 0x7f606c1030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.779+0000 7f60715b7700 1 --2- 172.21.15.7:0/2074667256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f606c1066f0 0x7f606c103b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.779+0000 7f606bfff700 1 -- 172.21.15.7:0/2074667256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f606c1066f0 msgr2=0x7f606c103b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.779+0000 7f606bfff700 1 --2- 172.21.15.7:0/2074667256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f606c1066f0 0x7f606c103b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.779+0000 7f606bfff700 1 -- 172.21.15.7:0/2074667256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f606c101610 msgr2=0x7f606c1030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.779+0000 7f606bfff700 1 --2- 172.21.15.7:0/2074667256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f606c101610 0x7f606c1030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.779+0000 7f606bfff700 1 -- 172.21.15.7:0/2074667256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606c1a7320 con 0x7f606c101fc0 2024-06-10T01:20:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.780+0000 7f606bfff700 1 --2- 172.21.15.7:0/2074667256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f606c101fc0 0x7f606c1035f0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f6054009fd0 tx=0x7f6054005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.780+0000 7f6069ffb700 1 -- 172.21.15.7:0/2074667256 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6054026070 con 0x7f606c101fc0 2024-06-10T01:20:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.780+0000 7f607301a700 1 -- 172.21.15.7:0/2074667256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f606c1a75b0 con 0x7f606c101fc0 2024-06-10T01:20:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.780+0000 7f6069ffb700 1 -- 172.21.15.7:0/2074667256 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6054004cd0 con 0x7f606c101fc0 2024-06-10T01:20:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.780+0000 7f6069ffb700 1 -- 172.21.15.7:0/2074667256 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6054003dd0 con 0x7f606c101fc0 2024-06-10T01:20:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.780+0000 7f607301a700 1 -- 172.21.15.7:0/2074667256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f606c1a7a10 con 0x7f606c101fc0 2024-06-10T01:20:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.781+0000 7f607301a700 1 -- 172.21.15.7:0/2074667256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f606c069180 con 0x7f606c101fc0 2024-06-10T01:20:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.782+0000 7f6069ffb700 1 -- 172.21.15.7:0/2074667256 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f6054029070 con 0x7f606c101fc0 2024-06-10T01:20:03.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.783+0000 7f6069ffb700 1 --2- 172.21.15.7:0/2074667256 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f60580a2c20 0x7f60580a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:03.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.784+0000 7f6070db6700 1 --2- 172.21.15.7:0/2074667256 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f60580a2c20 0x7f60580a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:03.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.784+0000 7f6069ffb700 1 -- 172.21.15.7:0/2074667256 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f60540d0170 con 0x7f606c101fc0 2024-06-10T01:20:03.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.784+0000 7f6070db6700 1 --2- 172.21.15.7:0/2074667256 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f60580a2c20 0x7f60580a50e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f605c009a10 tx=0x7f605c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:03.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:03.787+0000 7f6069ffb700 1 -- 172.21.15.7:0/2074667256 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6054030020 con 0x7f606c101fc0 2024-06-10T01:20:04.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:04.041+0000 7f607301a700 1 -- 172.21.15.7:0/2074667256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f606c02dca0 con 0x7f606c101fc0 2024-06-10T01:20:04.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:04.042+0000 7f6069ffb700 1 -- 172.21.15.7:0/2074667256 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11860 (secure 0 0 0) 0x7f6054013070 con 0x7f606c101fc0 2024-06-10T01:20:04.035 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:20:04.036 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:19:47.239285+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:20:04.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:04.044+0000 7f607301a700 1 -- 172.21.15.7:0/2074667256 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f60580a2c20 msgr2=0x7f60580a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:04.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:04.044+0000 7f607301a700 1 --2- 172.21.15.7:0/2074667256 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f60580a2c20 0x7f60580a50e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f605c009a10 tx=0x7f605c009190 comp rx=0 tx=0).stop 2024-06-10T01:20:04.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:04.044+0000 7f607301a700 1 -- 172.21.15.7:0/2074667256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f606c101fc0 msgr2=0x7f606c1035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:04.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:04.044+0000 7f607301a700 1 --2- 172.21.15.7:0/2074667256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f606c101fc0 0x7f606c1035f0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f6054009fd0 tx=0x7f6054005f10 comp rx=0 tx=0).stop 2024-06-10T01:20:04.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:04.044+0000 7f607301a700 1 -- 172.21.15.7:0/2074667256 shutdown_connections 2024-06-10T01:20:04.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:04.044+0000 7f607301a700 1 --2- 172.21.15.7:0/2074667256 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f60580a2c20 0x7f60580a50e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:04.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:04.044+0000 7f607301a700 1 --2- 172.21.15.7:0/2074667256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f606c1066f0 0x7f606c103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:04.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:04.044+0000 7f607301a700 1 --2- 172.21.15.7:0/2074667256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f606c101610 0x7f606c1030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:04.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:04.044+0000 7f607301a700 1 --2- 172.21.15.7:0/2074667256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f606c101fc0 0x7f606c1035f0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:04.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:04.044+0000 7f607301a700 1 -- 172.21.15.7:0/2074667256 >> 172.21.15.7:0/2074667256 conn(0x7f606c078430 msgr2=0x7f606c06af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:04.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:04.044+0000 7f607301a700 1 -- 172.21.15.7:0/2074667256 shutdown_connections 2024-06-10T01:20:04.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:04.044+0000 7f607301a700 1 -- 172.21.15.7:0/2074667256 wait complete. 2024-06-10T01:20:04.039 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 44 2024-06-10T01:20:04.469 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12545 20.74 MB/sec warmup 41 sec latency 12545.779 ms 2024-06-10T01:20:04.671 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:20:04.925 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:20:05.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:04 smithi007 ceph-mon[32398]: pgmap v854: 129 pgs: 129 active+clean; 355 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 935 KiB/s wr, 176 op/s 2024-06-10T01:20:05.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:04 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2074667256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:20:05.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:04 smithi078 ceph-mon[36683]: pgmap v854: 129 pgs: 129 active+clean; 355 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 935 KiB/s wr, 176 op/s 2024-06-10T01:20:05.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:04 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2074667256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:20:05.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:04 smithi175 ceph-mon[36636]: pgmap v854: 129 pgs: 129 active+clean; 355 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 935 KiB/s wr, 176 op/s 2024-06-10T01:20:05.452 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:04 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2074667256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:20:05.469 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12545 20.24 MB/sec warmup 42 sec latency 13545.816 ms 2024-06-10T01:20:06.469 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12545 19.77 MB/sec warmup 43 sec latency 14545.880 ms 2024-06-10T01:20:06.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.513+0000 7efc00e8d700 1 -- 172.21.15.7:0/3617523491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efbfc107400 msgr2=0x7efbfc1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:06.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.513+0000 7efc00e8d700 1 --2- 172.21.15.7:0/3617523491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efbfc107400 0x7efbfc1077e0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7efbec00b0a0 tx=0x7efbec00b3b0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.513+0000 7efc00e8d700 1 -- 172.21.15.7:0/3617523491 shutdown_connections 2024-06-10T01:20:06.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.513+0000 7efc00e8d700 1 --2- 172.21.15.7:0/3617523491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efbfc101d50 0x7efbfc10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.513+0000 7efc00e8d700 1 --2- 172.21.15.7:0/3617523491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efbfc107400 0x7efbfc1077e0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.513+0000 7efc00e8d700 1 --2- 172.21.15.7:0/3617523491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efbfc101390 0x7efbfc101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.513+0000 7efc00e8d700 1 -- 172.21.15.7:0/3617523491 >> 172.21.15.7:0/3617523491 conn(0x7efbfc078340 msgr2=0x7efbfc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:06.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.513+0000 7efc00e8d700 1 -- 172.21.15.7:0/3617523491 shutdown_connections 2024-06-10T01:20:06.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.513+0000 7efc00e8d700 1 -- 172.21.15.7:0/3617523491 wait complete. 2024-06-10T01:20:06.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.514+0000 7efc00e8d700 1 Processor -- start 2024-06-10T01:20:06.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.514+0000 7efc00e8d700 1 -- start start 2024-06-10T01:20:06.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.514+0000 7efc00e8d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efbfc101390 0x7efbfc19e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:06.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.514+0000 7efc00e8d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efbfc101d50 0x7efbfc19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:06.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.514+0000 7efc00e8d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efbfc107400 0x7efbfc198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:06.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.514+0000 7efc00e8d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc111670 con 0x7efbfc101390 2024-06-10T01:20:06.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.514+0000 7efc00e8d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc1114f0 con 0x7efbfc107400 2024-06-10T01:20:06.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.514+0000 7efc00e8d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc1117f0 con 0x7efbfc101d50 2024-06-10T01:20:06.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.514+0000 7efbfa59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efbfc101390 0x7efbfc19e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:06.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.514+0000 7efbfa59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efbfc101390 0x7efbfc19e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55554/0 (socket says 172.21.15.7:55554) 2024-06-10T01:20:06.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.514+0000 7efbfa59c700 1 -- 172.21.15.7:0/1976358862 learned_addr learned my addr 172.21.15.7:0/1976358862 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:06.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.515+0000 7efbfad9d700 1 --2- 172.21.15.7:0/1976358862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efbfc107400 0x7efbfc198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:06.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.515+0000 7efbf9d9b700 1 --2- 172.21.15.7:0/1976358862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efbfc101d50 0x7efbfc19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:06.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.515+0000 7efbfa59c700 1 -- 172.21.15.7:0/1976358862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efbfc101d50 msgr2=0x7efbfc19ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:06.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.515+0000 7efbfa59c700 1 --2- 172.21.15.7:0/1976358862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efbfc101d50 0x7efbfc19ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.515+0000 7efbfa59c700 1 -- 172.21.15.7:0/1976358862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efbfc107400 msgr2=0x7efbfc198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:06.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.515+0000 7efbfa59c700 1 --2- 172.21.15.7:0/1976358862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efbfc107400 0x7efbfc198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.515+0000 7efbfa59c700 1 -- 172.21.15.7:0/1976358862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbfc1990f0 con 0x7efbfc101390 2024-06-10T01:20:06.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.515+0000 7efbf9d9b700 1 --2- 172.21.15.7:0/1976358862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efbfc101d50 0x7efbfc19ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:20:06.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.515+0000 7efbfa59c700 1 --2- 172.21.15.7:0/1976358862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efbfc101390 0x7efbfc19e7d0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7efbec00b070 tx=0x7efbec004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:06.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.515+0000 7efbf37fe700 1 -- 172.21.15.7:0/1976358862 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbec021070 con 0x7efbfc101390 2024-06-10T01:20:06.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.515+0000 7efc00e8d700 1 -- 172.21.15.7:0/1976358862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efbfc199380 con 0x7efbfc101390 2024-06-10T01:20:06.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.515+0000 7efbf37fe700 1 -- 172.21.15.7:0/1976358862 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbec007c80 con 0x7efbfc101390 2024-06-10T01:20:06.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.515+0000 7efbf37fe700 1 -- 172.21.15.7:0/1976358862 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbec005390 con 0x7efbfc101390 2024-06-10T01:20:06.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.515+0000 7efc00e8d700 1 -- 172.21.15.7:0/1976358862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efbfc1a56a0 con 0x7efbfc101390 2024-06-10T01:20:06.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.517+0000 7efbf37fe700 1 -- 172.21.15.7:0/1976358862 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7efbec024070 con 0x7efbfc101390 2024-06-10T01:20:06.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.518+0000 7efbf37fe700 1 --2- 172.21.15.7:0/1976358862 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efbe80a2c20 0x7efbe80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:06.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.518+0000 7efbf9d9b700 1 --2- 172.21.15.7:0/1976358862 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efbe80a2c20 0x7efbe80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:06.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.518+0000 7efbf37fe700 1 -- 172.21.15.7:0/1976358862 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7efbec00f020 con 0x7efbfc101390 2024-06-10T01:20:06.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.519+0000 7efbf9d9b700 1 --2- 172.21.15.7:0/1976358862 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efbe80a2c20 0x7efbe80a50e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7efbe4005fd0 tx=0x7efbe4005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:06.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.519+0000 7efc0b710b80 1 Processor -- start 2024-06-10T01:20:06.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.519+0000 7efc0b710b80 1 -- start start 2024-06-10T01:20:06.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.519+0000 7efc0b710b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea36ff60 0x5629ea370340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:06.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.519+0000 7efc0b710b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea370880 0x5629ea36a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:06.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.519+0000 7efc0b710b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea36a7a0 0x5629ea36abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:06.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.519+0000 7efc0b710b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5629ea25cf60 con 0x5629ea36ff60 2024-06-10T01:20:06.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.519+0000 7efc0b710b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5629ea2bb850 con 0x5629ea36a7a0 2024-06-10T01:20:06.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.519+0000 7efc0b710b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5629ea223460 con 0x5629ea370880 2024-06-10T01:20:06.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.519+0000 7efbfa59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea36ff60 0x5629ea370340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:06.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.519+0000 7efbfa59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea36ff60 0x5629ea370340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55560/0 (socket says 172.21.15.7:55560) 2024-06-10T01:20:06.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.519+0000 7efbfa59c700 1 -- 172.21.15.7:0/774398822 learned_addr learned my addr 172.21.15.7:0/774398822 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:06.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.520+0000 7efbf9d9b700 1 --2- 172.21.15.7:0/774398822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea370880 0x5629ea36a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:06.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.520+0000 7efbfa59c700 1 -- 172.21.15.7:0/774398822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea370880 msgr2=0x5629ea36a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:06.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.520+0000 7efbfad9d700 1 --2- 172.21.15.7:0/774398822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea36a7a0 0x5629ea36abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:06.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.520+0000 7efbfa59c700 1 --2- 172.21.15.7:0/774398822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea370880 0x5629ea36a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.520+0000 7efbfa59c700 1 -- 172.21.15.7:0/774398822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea36a7a0 msgr2=0x5629ea36abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:06.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.520+0000 7efbfa59c700 1 --2- 172.21.15.7:0/774398822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea36a7a0 0x5629ea36abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.520+0000 7efbfa59c700 1 -- 172.21.15.7:0/774398822 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629ea20a8d0 con 0x5629ea36ff60 2024-06-10T01:20:06.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.520+0000 7efbfad9d700 1 --2- 172.21.15.7:0/774398822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea36a7a0 0x5629ea36abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:20:06.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.520+0000 7efbfa59c700 1 --2- 172.21.15.7:0/774398822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea36ff60 0x5629ea370340 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7efbec0a5760 tx=0x7efbec0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=236bb37959f9739b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:06.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.520+0000 7efbf0ff9700 1 -- 172.21.15.7:0/774398822 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbec09d7b0 con 0x5629ea36ff60 2024-06-10T01:20:06.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.520+0000 7efbf0ff9700 1 -- 172.21.15.7:0/774398822 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbec0bd0e0 con 0x5629ea36ff60 2024-06-10T01:20:06.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.520+0000 7efbf0ff9700 1 -- 172.21.15.7:0/774398822 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbec02edb0 con 0x5629ea36ff60 2024-06-10T01:20:06.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.521+0000 7efc0b710b80 1 -- 172.21.15.7:0/774398822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea36ff60 msgr2=0x5629ea370340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:06.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.521+0000 7efc0b710b80 1 --2- 172.21.15.7:0/774398822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea36ff60 0x5629ea370340 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7efbec0a5760 tx=0x7efbec0099e0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.521+0000 7efc0b710b80 1 -- 172.21.15.7:0/774398822 shutdown_connections 2024-06-10T01:20:06.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.521+0000 7efc0b710b80 1 --2- 172.21.15.7:0/774398822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea36a7a0 0x5629ea36abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.521+0000 7efc0b710b80 1 --2- 172.21.15.7:0/774398822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea36ff60 0x5629ea370340 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.521+0000 7efc0b710b80 1 --2- 172.21.15.7:0/774398822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea370880 0x5629ea36a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.521+0000 7efc0b710b80 1 -- 172.21.15.7:0/774398822 >> 172.21.15.7:0/774398822 conn(0x5629ea297080 msgr2=0x5629ea365220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:06.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.521+0000 7efc0b710b80 1 -- 172.21.15.7:0/774398822 shutdown_connections 2024-06-10T01:20:06.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.521+0000 7efc0b710b80 1 -- 172.21.15.7:0/774398822 wait complete. 2024-06-10T01:20:06.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.521+0000 7efc0b710b80 1 -- start start 2024-06-10T01:20:06.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.522+0000 7efc0b710b80 1 Processor -- start 2024-06-10T01:20:06.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.522+0000 7efc0b710b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea36a7a0 0x5629ea3765a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:06.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.522+0000 7efc0b710b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea36ff60 0x5629ea3738a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:06.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.522+0000 7efc0b710b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea370880 0x5629ea373de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:06.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.522+0000 7efc0b710b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5629ea25bfc0 con 0x5629ea370880 2024-06-10T01:20:06.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.522+0000 7efc0b710b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5629ea2b7810 con 0x5629ea36a7a0 2024-06-10T01:20:06.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.522+0000 7efc0b710b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5629ea377010 con 0x5629ea36ff60 2024-06-10T01:20:06.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.522+0000 7efc0b710b80 10 client.0 _ll_register_callbacks cb 0x5629ea2515b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:20:06.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.522+0000 7efbfad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea370880 0x5629ea373de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:06.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.522+0000 7efbfad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea370880 0x5629ea373de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55576/0 (socket says 172.21.15.7:55576) 2024-06-10T01:20:06.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.522+0000 7efbfad9d700 1 -- 172.21.15.7:0/1870742276 learned_addr learned my addr 172.21.15.7:0/1870742276 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:06.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.522+0000 7efbf9d9b700 1 --2- 172.21.15.7:0/1870742276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea36ff60 0x5629ea3738a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:06.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.522+0000 7efbfa59c700 1 --2- 172.21.15.7:0/1870742276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea36a7a0 0x5629ea3765a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:06.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.522+0000 7efbfad9d700 1 -- 172.21.15.7:0/1870742276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea36ff60 msgr2=0x5629ea3738a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:06.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.522+0000 7efbfad9d700 1 --2- 172.21.15.7:0/1870742276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea36ff60 0x5629ea3738a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.522+0000 7efbfad9d700 1 -- 172.21.15.7:0/1870742276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea36a7a0 msgr2=0x5629ea3765a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:06.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.522+0000 7efbfad9d700 1 --2- 172.21.15.7:0/1870742276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea36a7a0 0x5629ea3765a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.522+0000 7efbfad9d700 1 -- 172.21.15.7:0/1870742276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629ea281750 con 0x5629ea370880 2024-06-10T01:20:06.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.522+0000 7efbfa59c700 1 --2- 172.21.15.7:0/1870742276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea36a7a0 0x5629ea3765a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:20:06.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.523+0000 7efbfad9d700 1 --2- 172.21.15.7:0/1870742276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea370880 0x5629ea373de0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7efbf400b910 tx=0x7efbf400bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836c889a8748de83 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:06.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.523+0000 7efbe1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:20:06.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.523+0000 7efbe1ffb700 1 -- 172.21.15.7:0/1870742276 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf4003d70 con 0x5629ea370880 2024-06-10T01:20:06.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.523+0000 7efbe1ffb700 1 -- 172.21.15.7:0/1870742276 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7efbf401b470 con 0x5629ea370880 2024-06-10T01:20:06.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.523+0000 7efc0b710b80 1 -- 172.21.15.7:0/1870742276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbec0bd0e0 con 0x5629ea370880 2024-06-10T01:20:06.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.523+0000 7efbe1ffb700 1 -- 172.21.15.7:0/1870742276 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7efbf4004830 con 0x5629ea370880 2024-06-10T01:20:06.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.523+0000 7efc0b710b80 10 client.18978 fetch_fsmap learned FSMap version 44 2024-06-10T01:20:06.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.523+0000 7efc0b710b80 1 -- 172.21.15.7:0/1870742276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5629ea3816e0 con 0x5629ea370880 2024-06-10T01:20:06.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.523+0000 7efbe1ffb700 1 -- 172.21.15.7:0/1870742276 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbec0bd0e0 con 0x5629ea370880 2024-06-10T01:20:06.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.524+0000 7efbe1ffb700 1 -- 172.21.15.7:0/1870742276 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7efbf401b670 con 0x5629ea370880 2024-06-10T01:20:06.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.524+0000 7efc0b710b80 10 client.18978 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:20:06.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.524+0000 7efc0b710b80 10 client.18978 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:20:06.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.524+0000 7efc0b710b80 20 client.18978 populate_metadata read hostname 'smithi007' 2024-06-10T01:20:06.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.524+0000 7efc0b710b80 1 --2- 172.21.15.7:0/1870742276 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5629ea384180 0x5629ea3865e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:06.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.524+0000 7efc0b710b80 4 client.18978 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:20:06.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.524+0000 7efc0b710b80 1 -- 172.21.15.7:0/1870742276 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629ea2c8d50 con 0x5629ea384180 2024-06-10T01:20:06.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.524+0000 7efbfa59c700 1 --2- 172.21.15.7:0/1870742276 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5629ea384180 0x5629ea3865e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:06.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.525+0000 7efbe1ffb700 10 client.18978 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:20:06.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.525+0000 7efbfa59c700 1 --2- 172.21.15.7:0/1870742276 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5629ea384180 0x5629ea3865e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c9dab4cec3c14ebb server_cookie=3545d6d7a3211abc in_seq=0 out_seq=0 2024-06-10T01:20:06.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.525+0000 7efbe1ffb700 1 -- 172.21.15.7:0/1870742276 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5629ea2c8d50 con 0x5629ea384180 2024-06-10T01:20:06.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.525+0000 7efbe1ffb700 10 client.18978 handle_command_reply: tid=0 2024-06-10T01:20:06.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.525+0000 7efc0b710b80 1 client.18978 shutdown 2024-06-10T01:20:06.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.525+0000 7efc0b710b80 1 -- 172.21.15.7:0/1870742276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea370880 msgr2=0x5629ea373de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:06.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.525+0000 7efc0b710b80 1 --2- 172.21.15.7:0/1870742276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea370880 0x5629ea373de0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7efbf400b910 tx=0x7efbf400bc20 comp rx=0 tx=0).stop 2024-06-10T01:20:06.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.526+0000 7efc0b710b80 1 -- 172.21.15.7:0/1870742276 shutdown_connections 2024-06-10T01:20:06.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.526+0000 7efc0b710b80 1 --2- 172.21.15.7:0/1870742276 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5629ea384180 0x5629ea3865e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.526+0000 7efc0b710b80 1 --2- 172.21.15.7:0/1870742276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea370880 0x5629ea373de0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.526+0000 7efc0b710b80 1 --2- 172.21.15.7:0/1870742276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea36a7a0 0x5629ea3765a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.526+0000 7efc0b710b80 1 --2- 172.21.15.7:0/1870742276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea36ff60 0x5629ea3738a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.526+0000 7efc0b710b80 1 -- 172.21.15.7:0/1870742276 >> 172.21.15.7:0/1870742276 conn(0x5629ea297080 msgr2=0x5629ea364700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:06.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.526+0000 7efbe1ffb700 0 client.18978 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:20:06.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.526+0000 7efc0b710b80 1 -- 172.21.15.7:0/1870742276 shutdown_connections 2024-06-10T01:20:06.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.526+0000 7efc0b710b80 1 -- 172.21.15.7:0/1870742276 wait complete. 2024-06-10T01:20:06.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.526+0000 7efc0b710b80 20 client.18978 trim_cache size 0 max 16384 2024-06-10T01:20:06.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.541+0000 7efc0b710b80 1 Processor -- start 2024-06-10T01:20:06.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.541+0000 7efc0b710b80 1 -- start start 2024-06-10T01:20:06.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.541+0000 7efc0b710b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea36a7a0 0x5629ea37c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:06.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.541+0000 7efc0b710b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea36ff60 0x5629ea37cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:06.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.541+0000 7efc0b710b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea370880 0x5629ea364e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:06.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.541+0000 7efc0b710b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc1b0a50 con 0x5629ea36a7a0 2024-06-10T01:20:06.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.541+0000 7efc0b710b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc1ae250 con 0x5629ea370880 2024-06-10T01:20:06.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.541+0000 7efc0b710b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc1ac0a0 con 0x5629ea36ff60 2024-06-10T01:20:06.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.541+0000 7efbfa59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea36a7a0 0x5629ea37c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:06.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.541+0000 7efbfa59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea36a7a0 0x5629ea37c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55590/0 (socket says 172.21.15.7:55590) 2024-06-10T01:20:06.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.541+0000 7efbfa59c700 1 -- 172.21.15.7:0/2076619751 learned_addr learned my addr 172.21.15.7:0/2076619751 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:06.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.542+0000 7efbf9d9b700 1 --2- 172.21.15.7:0/2076619751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea36ff60 0x5629ea37cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:06.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.542+0000 7efbfa59c700 1 -- 172.21.15.7:0/2076619751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea36ff60 msgr2=0x5629ea37cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:06.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.542+0000 7efbfa59c700 1 --2- 172.21.15.7:0/2076619751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea36ff60 0x5629ea37cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.542+0000 7efbfa59c700 1 -- 172.21.15.7:0/2076619751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea370880 msgr2=0x5629ea364e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:06.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.542+0000 7efbfad9d700 1 --2- 172.21.15.7:0/2076619751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea370880 0x5629ea364e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:06.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.542+0000 7efbfa59c700 1 --2- 172.21.15.7:0/2076619751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea370880 0x5629ea364e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.542+0000 7efbfa59c700 1 -- 172.21.15.7:0/2076619751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629ea374780 con 0x5629ea36a7a0 2024-06-10T01:20:06.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.542+0000 7efbfad9d700 1 --2- 172.21.15.7:0/2076619751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea370880 0x5629ea364e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:20:06.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.542+0000 7efbfa59c700 1 --2- 172.21.15.7:0/2076619751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea36a7a0 0x5629ea37c640 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7efbec0ce310 tx=0x7efbec01dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c01070b5ee8b344 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:06.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.542+0000 7efbe3fff700 1 -- 172.21.15.7:0/2076619751 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbec09b150 con 0x5629ea36a7a0 2024-06-10T01:20:06.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.542+0000 7efbe3fff700 1 -- 172.21.15.7:0/2076619751 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbec09b3b0 con 0x5629ea36a7a0 2024-06-10T01:20:06.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.542+0000 7efbe3fff700 1 -- 172.21.15.7:0/2076619751 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbec09d310 con 0x5629ea36a7a0 2024-06-10T01:20:06.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.543+0000 7efc0b710b80 1 -- 172.21.15.7:0/2076619751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea36a7a0 msgr2=0x5629ea37c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:06.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.543+0000 7efc0b710b80 1 --2- 172.21.15.7:0/2076619751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea36a7a0 0x5629ea37c640 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7efbec0ce310 tx=0x7efbec01dda0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.543+0000 7efc0b710b80 1 -- 172.21.15.7:0/2076619751 shutdown_connections 2024-06-10T01:20:06.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.543+0000 7efc0b710b80 1 --2- 172.21.15.7:0/2076619751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea370880 0x5629ea364e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.543+0000 7efc0b710b80 1 --2- 172.21.15.7:0/2076619751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea36a7a0 0x5629ea37c640 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.543+0000 7efc0b710b80 1 --2- 172.21.15.7:0/2076619751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea36ff60 0x5629ea37cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.543+0000 7efc0b710b80 1 -- 172.21.15.7:0/2076619751 >> 172.21.15.7:0/2076619751 conn(0x5629ea297080 msgr2=0x5629ea373860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:06.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.543+0000 7efc0b710b80 1 -- 172.21.15.7:0/2076619751 shutdown_connections 2024-06-10T01:20:06.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.543+0000 7efc0b710b80 1 -- 172.21.15.7:0/2076619751 wait complete. 2024-06-10T01:20:06.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.543+0000 7efc0b710b80 1 -- start start 2024-06-10T01:20:06.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.543+0000 7efc0b710b80 1 Processor -- start 2024-06-10T01:20:06.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.544+0000 7efc0b710b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea36a7a0 0x5629ea38c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:06.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.544+0000 7efc0b710b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea36ff60 0x5629ea38c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:06.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.544+0000 7efc0b710b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea370880 0x5629ea363ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:06.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.544+0000 7efc0b710b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc1ac710 con 0x5629ea370880 2024-06-10T01:20:06.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.544+0000 7efc0b710b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc1adb80 con 0x5629ea36ff60 2024-06-10T01:20:06.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.544+0000 7efc0b710b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc1acde0 con 0x5629ea36a7a0 2024-06-10T01:20:06.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.544+0000 7efc0b710b80 10 client.0 _ll_register_callbacks cb 0x5629ea2515b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:20:06.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.544+0000 7efbfad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea370880 0x5629ea363ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:06.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.544+0000 7efbf9d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea36ff60 0x5629ea38c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:06.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.544+0000 7efbfad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea370880 0x5629ea363ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55594/0 (socket says 172.21.15.7:55594) 2024-06-10T01:20:06.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.544+0000 7efbfad9d700 1 -- 172.21.15.7:0/3740245264 learned_addr learned my addr 172.21.15.7:0/3740245264 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:06.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.544+0000 7efbfa59c700 1 --2- 172.21.15.7:0/3740245264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea36a7a0 0x5629ea38c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:06.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.544+0000 7efbfad9d700 1 -- 172.21.15.7:0/3740245264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea36a7a0 msgr2=0x5629ea38c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:06.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.544+0000 7efbfad9d700 1 --2- 172.21.15.7:0/3740245264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea36a7a0 0x5629ea38c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.544+0000 7efbfad9d700 1 -- 172.21.15.7:0/3740245264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea36ff60 msgr2=0x5629ea38c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:06.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.544+0000 7efbfad9d700 1 --2- 172.21.15.7:0/3740245264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea36ff60 0x5629ea38c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.544+0000 7efbfad9d700 1 -- 172.21.15.7:0/3740245264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629ea3646c0 con 0x5629ea370880 2024-06-10T01:20:06.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.544+0000 7efbfa59c700 1 --2- 172.21.15.7:0/3740245264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea36a7a0 0x5629ea38c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:20:06.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.544+0000 7efbfad9d700 1 --2- 172.21.15.7:0/3740245264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea370880 0x5629ea363ed0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7efbf401ba60 tx=0x7efbf4003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd6ddfc1fb7e2e4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:06.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.544+0000 7efbe1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:20:06.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.544+0000 7efbe1ffb700 1 -- 172.21.15.7:0/3740245264 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf4003960 con 0x5629ea370880 2024-06-10T01:20:06.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.544+0000 7efbe1ffb700 1 -- 172.21.15.7:0/3740245264 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7efbf4032470 con 0x5629ea370880 2024-06-10T01:20:06.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.544+0000 7efc0b710b80 1 -- 172.21.15.7:0/3740245264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbec09b3b0 con 0x5629ea370880 2024-06-10T01:20:06.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.545+0000 7efbe1ffb700 1 -- 172.21.15.7:0/3740245264 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7efbf4032670 con 0x5629ea370880 2024-06-10T01:20:06.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.545+0000 7efc0b710b80 10 client.18984 fetch_fsmap learned FSMap version 44 2024-06-10T01:20:06.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.545+0000 7efc0b710b80 1 -- 172.21.15.7:0/3740245264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5629ea3f2900 con 0x5629ea370880 2024-06-10T01:20:06.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.545+0000 7efbe1ffb700 1 -- 172.21.15.7:0/3740245264 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbec09b3b0 con 0x5629ea370880 2024-06-10T01:20:06.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.545+0000 7efbe1ffb700 1 -- 172.21.15.7:0/3740245264 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7efbf4012820 con 0x5629ea370880 2024-06-10T01:20:06.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.545+0000 7efc0b710b80 10 client.18984 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:20:06.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.545+0000 7efc0b710b80 10 client.18984 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:20:06.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.545+0000 7efc0b710b80 20 client.18984 populate_metadata read hostname 'smithi007' 2024-06-10T01:20:06.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.545+0000 7efc0b710b80 1 --2- 172.21.15.7:0/3740245264 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5629ea3f3070 0x5629ea3f5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:06.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.545+0000 7efc0b710b80 4 client.18984 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:20:06.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.545+0000 7efc0b710b80 1 -- 172.21.15.7:0/3740245264 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5629ea2c32d0 con 0x5629ea3f3070 2024-06-10T01:20:06.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.545+0000 7efbfa59c700 1 --2- 172.21.15.7:0/3740245264 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5629ea3f3070 0x5629ea3f5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:06.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.546+0000 7efbe1ffb700 10 client.18984 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:20:06.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.546+0000 7efbfa59c700 1 --2- 172.21.15.7:0/3740245264 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5629ea3f3070 0x5629ea3f5490 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=765119113346f52b server_cookie=8190e66c19ce2faa in_seq=0 out_seq=0 2024-06-10T01:20:06.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.546+0000 7efbe1ffb700 1 -- 172.21.15.7:0/3740245264 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5629ea2c32d0 con 0x5629ea3f3070 2024-06-10T01:20:06.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.546+0000 7efbe1ffb700 10 client.18984 handle_command_reply: tid=0 2024-06-10T01:20:06.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.546+0000 7efc0b710b80 1 client.18984 shutdown 2024-06-10T01:20:06.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.546+0000 7efc0b710b80 1 -- 172.21.15.7:0/3740245264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea370880 msgr2=0x5629ea363ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:06.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.546+0000 7efc0b710b80 1 --2- 172.21.15.7:0/3740245264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea370880 0x5629ea363ed0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7efbf401ba60 tx=0x7efbf4003eb0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.546+0000 7efc0b710b80 1 -- 172.21.15.7:0/3740245264 shutdown_connections 2024-06-10T01:20:06.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.546+0000 7efc0b710b80 1 --2- 172.21.15.7:0/3740245264 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5629ea3f3070 0x5629ea3f5490 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.546+0000 7efc0b710b80 1 --2- 172.21.15.7:0/3740245264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629ea370880 0x5629ea363ed0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.546+0000 7efc0b710b80 1 --2- 172.21.15.7:0/3740245264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629ea36a7a0 0x5629ea38c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.546+0000 7efc0b710b80 1 --2- 172.21.15.7:0/3740245264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629ea36ff60 0x5629ea38c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.546+0000 7efc0b710b80 1 -- 172.21.15.7:0/3740245264 >> 172.21.15.7:0/3740245264 conn(0x5629ea297080 msgr2=0x5629ea370c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:06.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.546+0000 7efbe1ffb700 0 client.18984 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:20:06.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.546+0000 7efc0b710b80 1 -- 172.21.15.7:0/3740245264 shutdown_connections 2024-06-10T01:20:06.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.546+0000 7efc0b710b80 1 -- 172.21.15.7:0/3740245264 wait complete. 2024-06-10T01:20:06.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.546+0000 7efc0b710b80 20 client.18984 trim_cache size 0 max 16384 2024-06-10T01:20:06.546 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:20:06.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.547+0000 7efc00e8d700 1 -- 172.21.15.7:0/1976358862 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efbe80a2c20 msgr2=0x7efbe80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:06.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.547+0000 7efc00e8d700 1 --2- 172.21.15.7:0/1976358862 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efbe80a2c20 0x7efbe80a50e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7efbe4005fd0 tx=0x7efbe4005e20 comp rx=0 tx=0).stop 2024-06-10T01:20:06.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.547+0000 7efc00e8d700 1 -- 172.21.15.7:0/1976358862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efbfc101390 msgr2=0x7efbfc19e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:06.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.547+0000 7efc00e8d700 1 --2- 172.21.15.7:0/1976358862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efbfc101390 0x7efbfc19e7d0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7efbec00b070 tx=0x7efbec004e40 comp rx=0 tx=0).stop 2024-06-10T01:20:06.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.547+0000 7efc00e8d700 1 -- 172.21.15.7:0/1976358862 shutdown_connections 2024-06-10T01:20:06.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.547+0000 7efc00e8d700 1 --2- 172.21.15.7:0/1976358862 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efbe80a2c20 0x7efbe80a50e0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.547+0000 7efc00e8d700 1 --2- 172.21.15.7:0/1976358862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efbfc107400 0x7efbfc198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.547+0000 7efc00e8d700 1 --2- 172.21.15.7:0/1976358862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efbfc101390 0x7efbfc19e7d0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.547+0000 7efc00e8d700 1 --2- 172.21.15.7:0/1976358862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efbfc101d50 0x7efbfc19ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:06.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.547+0000 7efc00e8d700 1 -- 172.21.15.7:0/1976358862 >> 172.21.15.7:0/1976358862 conn(0x7efbfc078340 msgr2=0x7efbfc1005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:06.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.547+0000 7efc00e8d700 1 -- 172.21.15.7:0/1976358862 shutdown_connections 2024-06-10T01:20:06.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:06.547+0000 7efc00e8d700 1 -- 172.21.15.7:0/1976358862 wait complete. 2024-06-10T01:20:07.236 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:06 smithi007 ceph-mon[32398]: pgmap v855: 129 pgs: 129 active+clean; 355 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 659 KiB/s wr, 179 op/s 2024-06-10T01:20:07.268 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:20:07.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:06 smithi078 ceph-mon[36683]: pgmap v855: 129 pgs: 129 active+clean; 355 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 659 KiB/s wr, 179 op/s 2024-06-10T01:20:07.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:06 smithi175 ceph-mon[36636]: pgmap v855: 129 pgs: 129 active+clean; 355 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 659 KiB/s wr, 179 op/s 2024-06-10T01:20:07.469 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12545 19.32 MB/sec warmup 44 sec latency 15546.160 ms 2024-06-10T01:20:07.522 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:20:08.469 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12545 18.89 MB/sec warmup 45 sec latency 16546.265 ms 2024-06-10T01:20:09.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:08 smithi007 ceph-mon[32398]: pgmap v856: 129 pgs: 129 active+clean; 355 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 227 KiB/s wr, 132 op/s 2024-06-10T01:20:09.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.301+0000 7f57d3ff0700 1 -- 172.21.15.7:0/434497804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57cc06ab30 msgr2=0x7f57cc10f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:09.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.301+0000 7f57d3ff0700 1 --2- 172.21.15.7:0/434497804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57cc06ab30 0x7f57cc10f320 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f57c4009990 tx=0x7f57c4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.302+0000 7f57d3ff0700 1 -- 172.21.15.7:0/434497804 shutdown_connections 2024-06-10T01:20:09.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.302+0000 7f57d3ff0700 1 --2- 172.21.15.7:0/434497804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57cc06ab30 0x7f57cc10f320 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.302+0000 7f57d3ff0700 1 --2- 172.21.15.7:0/434497804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57cc108500 0x7f57cc1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.302+0000 7f57d3ff0700 1 --2- 172.21.15.7:0/434497804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57cc06a170 0x7f57cc06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.302+0000 7f57d3ff0700 1 -- 172.21.15.7:0/434497804 >> 172.21.15.7:0/434497804 conn(0x7f57cc078340 msgr2=0x7f57cc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:09.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.302+0000 7f57d3ff0700 1 -- 172.21.15.7:0/434497804 shutdown_connections 2024-06-10T01:20:09.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.302+0000 7f57d3ff0700 1 -- 172.21.15.7:0/434497804 wait complete. 2024-06-10T01:20:09.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.302+0000 7f57d3ff0700 1 Processor -- start 2024-06-10T01:20:09.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.303+0000 7f57d3ff0700 1 -- start start 2024-06-10T01:20:09.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.303+0000 7f57d3ff0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57cc06a170 0x7f57cc101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:09.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.303+0000 7f57d3ff0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57cc06ab30 0x7f57cc1019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:09.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.303+0000 7f57d3ff0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57cc108500 0x7f57cc101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:09.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.303+0000 7f57d3ff0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc1127b0 con 0x7f57cc108500 2024-06-10T01:20:09.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.303+0000 7f57d3ff0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc112630 con 0x7f57cc06a170 2024-06-10T01:20:09.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.303+0000 7f57d3ff0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc112930 con 0x7f57cc06ab30 2024-06-10T01:20:09.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.303+0000 7f57d258d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57cc108500 0x7f57cc101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:09.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.303+0000 7f57d258d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57cc108500 0x7f57cc101ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55608/0 (socket says 172.21.15.7:55608) 2024-06-10T01:20:09.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.303+0000 7f57d258d700 1 -- 172.21.15.7:0/1970613538 learned_addr learned my addr 172.21.15.7:0/1970613538 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:09.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.303+0000 7f57d1d8c700 1 --2- 172.21.15.7:0/1970613538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57cc06a170 0x7f57cc101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:09.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.303+0000 7f57d158b700 1 --2- 172.21.15.7:0/1970613538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57cc06ab30 0x7f57cc1019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:09.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.303+0000 7f57d1d8c700 1 -- 172.21.15.7:0/1970613538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57cc06ab30 msgr2=0x7f57cc1019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:09.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.303+0000 7f57d1d8c700 1 --2- 172.21.15.7:0/1970613538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57cc06ab30 0x7f57cc1019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.303+0000 7f57d1d8c700 1 -- 172.21.15.7:0/1970613538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57cc108500 msgr2=0x7f57cc101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:09.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.303+0000 7f57d1d8c700 1 --2- 172.21.15.7:0/1970613538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57cc108500 0x7f57cc101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.303+0000 7f57d1d8c700 1 -- 172.21.15.7:0/1970613538 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57cc1a7120 con 0x7f57cc06a170 2024-06-10T01:20:09.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.303+0000 7f57d258d700 1 --2- 172.21.15.7:0/1970613538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57cc108500 0x7f57cc101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:20:09.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.304+0000 7f57d1d8c700 1 --2- 172.21.15.7:0/1970613538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57cc06a170 0x7f57cc101460 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f57bc007cc0 tx=0x7f57bc00d0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:09.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.304+0000 7f57caffd700 1 -- 172.21.15.7:0/1970613538 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc004db0 con 0x7f57cc06a170 2024-06-10T01:20:09.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.304+0000 7f57d3ff0700 1 -- 172.21.15.7:0/1970613538 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57cc1a7410 con 0x7f57cc06a170 2024-06-10T01:20:09.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.304+0000 7f57caffd700 1 -- 172.21.15.7:0/1970613538 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57bc005020 con 0x7f57cc06a170 2024-06-10T01:20:09.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.304+0000 7f57d3ff0700 1 -- 172.21.15.7:0/1970613538 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57cc1a7950 con 0x7f57cc06a170 2024-06-10T01:20:09.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.304+0000 7f57caffd700 1 -- 172.21.15.7:0/1970613538 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc015640 con 0x7f57cc06a170 2024-06-10T01:20:09.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.306+0000 7f57caffd700 1 -- 172.21.15.7:0/1970613538 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f57bc005560 con 0x7f57cc06a170 2024-06-10T01:20:09.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.307+0000 7f57caffd700 1 --2- 172.21.15.7:0/1970613538 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f57b40a2c20 0x7f57b40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:09.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.307+0000 7f57de873b80 1 Processor -- start 2024-06-10T01:20:09.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.307+0000 7f57d158b700 1 --2- 172.21.15.7:0/1970613538 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f57b40a2c20 0x7f57b40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:09.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.307+0000 7f57caffd700 1 -- 172.21.15.7:0/1970613538 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f57bc0ceda0 con 0x7f57cc06a170 2024-06-10T01:20:09.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.308+0000 7f57de873b80 1 -- start start 2024-06-10T01:20:09.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.308+0000 7f57de873b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a451c6af60 0x55a451c6b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:09.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.308+0000 7f57de873b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a451c6b880 0x55a451c65260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:09.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.308+0000 7f57de873b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a451c657a0 0x55a451c65bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:09.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.308+0000 7f57de873b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a451b57f60 con 0x55a451c6b880 2024-06-10T01:20:09.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.308+0000 7f57de873b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a451bb6850 con 0x55a451c6af60 2024-06-10T01:20:09.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.308+0000 7f57de873b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a451b1e460 con 0x55a451c657a0 2024-06-10T01:20:09.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.308+0000 7f57d158b700 1 --2- 172.21.15.7:0/1970613538 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f57b40a2c20 0x7f57b40a50e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f57cc078940 tx=0x7f57c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:09.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.308+0000 7f57d1d8c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a451c6af60 0x55a451c6b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:09.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.308+0000 7f57d158b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a451c6b880 0x55a451c65260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:09.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.308+0000 7f57d1d8c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a451c6af60 0x55a451c6b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:49122/0 (socket says 172.21.15.7:49122) 2024-06-10T01:20:09.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.308+0000 7f57d1d8c700 1 -- 172.21.15.7:0/4033902693 learned_addr learned my addr 172.21.15.7:0/4033902693 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:09.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.308+0000 7f57d258d700 1 --2- 172.21.15.7:0/4033902693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a451c657a0 0x55a451c65bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:09.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.309+0000 7f57d1d8c700 1 -- 172.21.15.7:0/4033902693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a451c657a0 msgr2=0x55a451c65bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:09.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.309+0000 7f57d1d8c700 1 --2- 172.21.15.7:0/4033902693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a451c657a0 0x55a451c65bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.309+0000 7f57d1d8c700 1 -- 172.21.15.7:0/4033902693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a451c6b880 msgr2=0x55a451c65260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:09.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.309+0000 7f57d1d8c700 1 --2- 172.21.15.7:0/4033902693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a451c6b880 0x55a451c65260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.309+0000 7f57d1d8c700 1 -- 172.21.15.7:0/4033902693 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a451b058d0 con 0x55a451c6af60 2024-06-10T01:20:09.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.309+0000 7f57d158b700 1 --2- 172.21.15.7:0/4033902693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a451c6b880 0x55a451c65260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:20:09.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.309+0000 7f57d1d8c700 1 --2- 172.21.15.7:0/4033902693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a451c6af60 0x55a451c6b340 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f57bc09b1b0 tx=0x7f57bc09b920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fca420db045aae7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:09.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.309+0000 7f57b3fff700 1 -- 172.21.15.7:0/4033902693 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc0b9e60 con 0x55a451c6af60 2024-06-10T01:20:09.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.309+0000 7f57b3fff700 1 -- 172.21.15.7:0/4033902693 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57bc0a3e90 con 0x55a451c6af60 2024-06-10T01:20:09.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.309+0000 7f57b3fff700 1 -- 172.21.15.7:0/4033902693 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc09b6e0 con 0x55a451c6af60 2024-06-10T01:20:09.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.310+0000 7f57de873b80 1 -- 172.21.15.7:0/4033902693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a451c6af60 msgr2=0x55a451c6b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:09.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.310+0000 7f57de873b80 1 --2- 172.21.15.7:0/4033902693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a451c6af60 0x55a451c6b340 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f57bc09b1b0 tx=0x7f57bc09b920 comp rx=0 tx=0).stop 2024-06-10T01:20:09.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.310+0000 7f57de873b80 1 -- 172.21.15.7:0/4033902693 shutdown_connections 2024-06-10T01:20:09.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.310+0000 7f57de873b80 1 --2- 172.21.15.7:0/4033902693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a451c657a0 0x55a451c65bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.310+0000 7f57de873b80 1 --2- 172.21.15.7:0/4033902693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a451c6af60 0x55a451c6b340 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.310+0000 7f57de873b80 1 --2- 172.21.15.7:0/4033902693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a451c6b880 0x55a451c65260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.310+0000 7f57de873b80 1 -- 172.21.15.7:0/4033902693 >> 172.21.15.7:0/4033902693 conn(0x55a451b92080 msgr2=0x55a451c60220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:09.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.310+0000 7f57de873b80 1 -- 172.21.15.7:0/4033902693 shutdown_connections 2024-06-10T01:20:09.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.310+0000 7f57de873b80 1 -- 172.21.15.7:0/4033902693 wait complete. 2024-06-10T01:20:09.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.310+0000 7f57de873b80 1 -- start start 2024-06-10T01:20:09.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.311+0000 7f57de873b80 1 Processor -- start 2024-06-10T01:20:09.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.311+0000 7f57de873b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a451c657a0 0x55a451c716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:09.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.311+0000 7f57de873b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a451c6af60 0x55a451c6e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:09.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.311+0000 7f57de873b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a451c6b880 0x55a451c6ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:09.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.311+0000 7f57de873b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a451b56fc0 con 0x55a451c657a0 2024-06-10T01:20:09.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.311+0000 7f57d1d8c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a451c657a0 0x55a451c716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:09.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.311+0000 7f57d1d8c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a451c657a0 0x55a451c716e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55626/0 (socket says 172.21.15.7:55626) 2024-06-10T01:20:09.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.311+0000 7f57d1d8c700 1 -- 172.21.15.7:0/3752966508 learned_addr learned my addr 172.21.15.7:0/3752966508 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:09.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.311+0000 7f57d258d700 1 --2- 172.21.15.7:0/3752966508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a451c6b880 0x55a451c6ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:09.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.311+0000 7f57de873b80 1 -- 172.21.15.7:0/3752966508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a451bb2810 con 0x55a451c6b880 2024-06-10T01:20:09.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.311+0000 7f57de873b80 1 -- 172.21.15.7:0/3752966508 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a451c72260 con 0x55a451c6af60 2024-06-10T01:20:09.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.311+0000 7f57d158b700 1 --2- 172.21.15.7:0/3752966508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a451c6af60 0x55a451c6e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:09.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.312+0000 7f57de873b80 10 client.0 _ll_register_callbacks cb 0x55a451b4c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:20:09.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.312+0000 7f57d1d8c700 1 -- 172.21.15.7:0/3752966508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a451c6af60 msgr2=0x55a451c6e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:09.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.312+0000 7f57d1d8c700 1 --2- 172.21.15.7:0/3752966508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a451c6af60 0x55a451c6e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.312+0000 7f57d1d8c700 1 -- 172.21.15.7:0/3752966508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a451c6b880 msgr2=0x55a451c6ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:09.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.312+0000 7f57d1d8c700 1 --2- 172.21.15.7:0/3752966508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a451c6b880 0x55a451c6ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.312+0000 7f57d1d8c700 1 -- 172.21.15.7:0/3752966508 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a451b7c750 con 0x55a451c657a0 2024-06-10T01:20:09.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.312+0000 7f57d258d700 1 --2- 172.21.15.7:0/3752966508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a451c6b880 0x55a451c6ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:20:09.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.312+0000 7f57d1d8c700 1 --2- 172.21.15.7:0/3752966508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a451c657a0 0x55a451c716e0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f57bc097630 tx=0x7f57bc00d650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739e9c2595c2122e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:09.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.312+0000 7f57b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:20:09.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.312+0000 7f57b17fa700 1 -- 172.21.15.7:0/3752966508 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc0b4020 con 0x55a451c657a0 2024-06-10T01:20:09.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.312+0000 7f57b17fa700 1 -- 172.21.15.7:0/3752966508 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f57bc09b480 con 0x55a451c657a0 2024-06-10T01:20:09.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.312+0000 7f57de873b80 1 -- 172.21.15.7:0/3752966508 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57bc0a3e90 con 0x55a451c657a0 2024-06-10T01:20:09.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.313+0000 7f57b17fa700 1 -- 172.21.15.7:0/3752966508 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f57bc09f380 con 0x55a451c657a0 2024-06-10T01:20:09.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.313+0000 7f57de873b80 10 client.18996 fetch_fsmap learned FSMap version 44 2024-06-10T01:20:09.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.313+0000 7f57de873b80 1 -- 172.21.15.7:0/3752966508 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55a451c7ca90 con 0x55a451c657a0 2024-06-10T01:20:09.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.313+0000 7f57b17fa700 1 -- 172.21.15.7:0/3752966508 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57bc09b680 con 0x55a451c657a0 2024-06-10T01:20:09.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.314+0000 7f57b17fa700 1 -- 172.21.15.7:0/3752966508 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f57bc0661c0 con 0x55a451c657a0 2024-06-10T01:20:09.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.314+0000 7f57de873b80 10 client.18996 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:20:09.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.314+0000 7f57de873b80 10 client.18996 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:20:09.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.314+0000 7f57de873b80 20 client.18996 populate_metadata read hostname 'smithi007' 2024-06-10T01:20:09.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.314+0000 7f57de873b80 1 --2- 172.21.15.7:0/3752966508 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a451c7f120 0x55a451c81540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:09.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.314+0000 7f57de873b80 4 client.18996 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:20:09.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.314+0000 7f57de873b80 1 -- 172.21.15.7:0/3752966508 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f57bc0a3e90 con 0x55a451c7f120 2024-06-10T01:20:09.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.314+0000 7f57d158b700 1 --2- 172.21.15.7:0/3752966508 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a451c7f120 0x55a451c81540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:09.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.314+0000 7f57d158b700 1 --2- 172.21.15.7:0/3752966508 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a451c7f120 0x55a451c81540 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fb84910c8a8f273e server_cookie=d2bc68f3d42e9b2f in_seq=0 out_seq=0 2024-06-10T01:20:09.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.314+0000 7f57b17fa700 10 client.18996 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:20:09.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.315+0000 7f57b17fa700 1 -- 172.21.15.7:0/3752966508 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f57bc0a3e90 con 0x55a451c7f120 2024-06-10T01:20:09.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.315+0000 7f57b17fa700 10 client.18996 handle_command_reply: tid=0 2024-06-10T01:20:09.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.315+0000 7f57de873b80 1 client.18996 shutdown 2024-06-10T01:20:09.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.315+0000 7f57de873b80 1 -- 172.21.15.7:0/3752966508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a451c657a0 msgr2=0x55a451c716e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:09.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.315+0000 7f57de873b80 1 --2- 172.21.15.7:0/3752966508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a451c657a0 0x55a451c716e0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f57bc097630 tx=0x7f57bc00d650 comp rx=0 tx=0).stop 2024-06-10T01:20:09.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.316+0000 7f57de873b80 1 -- 172.21.15.7:0/3752966508 shutdown_connections 2024-06-10T01:20:09.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.316+0000 7f57de873b80 1 --2- 172.21.15.7:0/3752966508 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a451c7f120 0x55a451c81540 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.316+0000 7f57de873b80 1 --2- 172.21.15.7:0/3752966508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a451c6b880 0x55a451c6ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.316+0000 7f57b17fa700 0 client.18996 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:20:09.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.316+0000 7f57de873b80 1 --2- 172.21.15.7:0/3752966508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a451c657a0 0x55a451c716e0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.316+0000 7f57de873b80 1 --2- 172.21.15.7:0/3752966508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a451c6af60 0x55a451c6e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.316+0000 7f57de873b80 1 -- 172.21.15.7:0/3752966508 >> 172.21.15.7:0/3752966508 conn(0x55a451b92080 msgr2=0x55a451c5f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:09.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.316+0000 7f57de873b80 1 -- 172.21.15.7:0/3752966508 shutdown_connections 2024-06-10T01:20:09.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.316+0000 7f57de873b80 1 -- 172.21.15.7:0/3752966508 wait complete. 2024-06-10T01:20:09.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.316+0000 7f57de873b80 20 client.18996 trim_cache size 0 max 16384 2024-06-10T01:20:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.331+0000 7f57de873b80 1 Processor -- start 2024-06-10T01:20:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.332+0000 7f57de873b80 1 -- start start 2024-06-10T01:20:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.332+0000 7f57de873b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a451c657a0 0x55a451c65260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.332+0000 7f57de873b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a451c6af60 0x55a451c7b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.332+0000 7f57de873b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a451c6b880 0x55a451c71220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.332+0000 7f57de873b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc1b2ee0 con 0x55a451c657a0 2024-06-10T01:20:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.332+0000 7f57de873b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc1b06e0 con 0x55a451c6af60 2024-06-10T01:20:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.332+0000 7f57de873b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc1ae530 con 0x55a451c6b880 2024-06-10T01:20:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.332+0000 7f57d1d8c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a451c657a0 0x55a451c65260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.332+0000 7f57d1d8c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a451c657a0 0x55a451c65260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55630/0 (socket says 172.21.15.7:55630) 2024-06-10T01:20:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.332+0000 7f57d1d8c700 1 -- 172.21.15.7:0/804927545 learned_addr learned my addr 172.21.15.7:0/804927545 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.332+0000 7f57d158b700 1 --2- 172.21.15.7:0/804927545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a451c6af60 0x55a451c7b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.332+0000 7f57d258d700 1 --2- 172.21.15.7:0/804927545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a451c6b880 0x55a451c71220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:09.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.332+0000 7f57d158b700 1 -- 172.21.15.7:0/804927545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a451c6b880 msgr2=0x55a451c71220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:09.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.332+0000 7f57d158b700 1 --2- 172.21.15.7:0/804927545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a451c6b880 0x55a451c71220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.332+0000 7f57d158b700 1 -- 172.21.15.7:0/804927545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a451c657a0 msgr2=0x55a451c65260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:09.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.332+0000 7f57d158b700 1 --2- 172.21.15.7:0/804927545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a451c657a0 0x55a451c65260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.332+0000 7f57d158b700 1 -- 172.21.15.7:0/804927545 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a451c6cf50 con 0x55a451c6af60 2024-06-10T01:20:09.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.332+0000 7f57d1d8c700 1 --2- 172.21.15.7:0/804927545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a451c657a0 0x55a451c65260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:20:09.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.333+0000 7f57d158b700 1 --2- 172.21.15.7:0/804927545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a451c6af60 0x55a451c7b380 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x55a451c6ffb0 tx=0x7f57c0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=789921e6389e8414 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:09.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.333+0000 7f57b37fe700 1 -- 172.21.15.7:0/804927545 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c0004020 con 0x55a451c6af60 2024-06-10T01:20:09.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.333+0000 7f57b37fe700 1 -- 172.21.15.7:0/804927545 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57c0003680 con 0x55a451c6af60 2024-06-10T01:20:09.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.333+0000 7f57b37fe700 1 -- 172.21.15.7:0/804927545 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c002a690 con 0x55a451c6af60 2024-06-10T01:20:09.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.333+0000 7f57de873b80 1 -- 172.21.15.7:0/804927545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a451c6af60 msgr2=0x55a451c7b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:09.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.334+0000 7f57de873b80 1 --2- 172.21.15.7:0/804927545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a451c6af60 0x55a451c7b380 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x55a451c6ffb0 tx=0x7f57c0014000 comp rx=0 tx=0).stop 2024-06-10T01:20:09.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.334+0000 7f57de873b80 1 -- 172.21.15.7:0/804927545 shutdown_connections 2024-06-10T01:20:09.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.334+0000 7f57de873b80 1 --2- 172.21.15.7:0/804927545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a451c6b880 0x55a451c71220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.334+0000 7f57de873b80 1 --2- 172.21.15.7:0/804927545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a451c657a0 0x55a451c65260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.334+0000 7f57de873b80 1 --2- 172.21.15.7:0/804927545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a451c6af60 0x55a451c7b380 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.334+0000 7f57de873b80 1 -- 172.21.15.7:0/804927545 >> 172.21.15.7:0/804927545 conn(0x55a451b92080 msgr2=0x55a451c6c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:09.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.334+0000 7f57de873b80 1 -- 172.21.15.7:0/804927545 shutdown_connections 2024-06-10T01:20:09.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.334+0000 7f57de873b80 1 -- 172.21.15.7:0/804927545 wait complete. 2024-06-10T01:20:09.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.334+0000 7f57de873b80 1 -- start start 2024-06-10T01:20:09.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.334+0000 7f57de873b80 1 Processor -- start 2024-06-10T01:20:09.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:08 smithi078 ceph-mon[36683]: pgmap v856: 129 pgs: 129 active+clean; 355 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 227 KiB/s wr, 132 op/s 2024-06-10T01:20:09.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.335+0000 7f57de873b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a451c657a0 0x55a451c774f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:09.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.335+0000 7f57de873b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a451c6af60 0x55a451c77a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:09.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.335+0000 7f57de873b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a451c6b880 0x55a451c85ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:09.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.335+0000 7f57de873b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc1aeba0 con 0x55a451c657a0 2024-06-10T01:20:09.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.335+0000 7f57de873b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc1b0010 con 0x55a451c6af60 2024-06-10T01:20:09.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.335+0000 7f57de873b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc1af270 con 0x55a451c6b880 2024-06-10T01:20:09.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.335+0000 7f57d1d8c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a451c657a0 0x55a451c774f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:09.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.335+0000 7f57de873b80 10 client.0 _ll_register_callbacks cb 0x55a451b4c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:20:09.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.335+0000 7f57d158b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a451c6af60 0x55a451c77a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:09.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.335+0000 7f57d158b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a451c6af60 0x55a451c77a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:49162/0 (socket says 172.21.15.7:49162) 2024-06-10T01:20:09.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.335+0000 7f57d158b700 1 -- 172.21.15.7:0/869532450 learned_addr learned my addr 172.21.15.7:0/869532450 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:09.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.335+0000 7f57d258d700 1 --2- 172.21.15.7:0/869532450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a451c6b880 0x55a451c85ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:09.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.335+0000 7f57d1d8c700 1 --2- 172.21.15.7:0/869532450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a451c657a0 0x55a451c774f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55632/0 (socket says 172.21.15.7:55632) 2024-06-10T01:20:09.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.336+0000 7f57d158b700 1 -- 172.21.15.7:0/869532450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a451c6b880 msgr2=0x55a451c85ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:09.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.336+0000 7f57d158b700 1 --2- 172.21.15.7:0/869532450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a451c6b880 0x55a451c85ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.336+0000 7f57d158b700 1 -- 172.21.15.7:0/869532450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a451c657a0 msgr2=0x55a451c774f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:09.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.336+0000 7f57d158b700 1 --2- 172.21.15.7:0/869532450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a451c657a0 0x55a451c774f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.336+0000 7f57d158b700 1 -- 172.21.15.7:0/869532450 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a451c86420 con 0x55a451c6af60 2024-06-10T01:20:09.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.336+0000 7f57d158b700 1 --2- 172.21.15.7:0/869532450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a451c6af60 0x55a451c77a30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f57c0009990 tx=0x7f57c0011e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4060dcbfacf7167d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:09.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.336+0000 7f57b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:20:09.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.336+0000 7f57b17fa700 1 -- 172.21.15.7:0/869532450 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c0012e30 con 0x55a451c6af60 2024-06-10T01:20:09.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.336+0000 7f57b17fa700 1 -- 172.21.15.7:0/869532450 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f57c0005260 con 0x55a451c6af60 2024-06-10T01:20:09.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.336+0000 7f57de873b80 1 -- 172.21.15.7:0/869532450 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57c0003680 con 0x55a451c6af60 2024-06-10T01:20:09.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.336+0000 7f57b17fa700 1 -- 172.21.15.7:0/869532450 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f57c0005480 con 0x55a451c6af60 2024-06-10T01:20:09.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.337+0000 7f57de873b80 10 client.28286 fetch_fsmap learned FSMap version 44 2024-06-10T01:20:09.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.337+0000 7f57de873b80 1 -- 172.21.15.7:0/869532450 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55a451ce9630 con 0x55a451c6af60 2024-06-10T01:20:09.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.337+0000 7f57b17fa700 1 -- 172.21.15.7:0/869532450 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57c0005620 con 0x55a451c6af60 2024-06-10T01:20:09.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.337+0000 7f57b17fa700 1 -- 172.21.15.7:0/869532450 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f57c002aa40 con 0x55a451c6af60 2024-06-10T01:20:09.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.337+0000 7f57de873b80 10 client.28286 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:20:09.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.337+0000 7f57de873b80 10 client.28286 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:20:09.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.337+0000 7f57de873b80 20 client.28286 populate_metadata read hostname 'smithi007' 2024-06-10T01:20:09.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.337+0000 7f57de873b80 1 --2- 172.21.15.7:0/869532450 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a451c7f120 0x55a451cebdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:09.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.337+0000 7f57de873b80 4 client.28286 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:20:09.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.337+0000 7f57de873b80 1 -- 172.21.15.7:0/869532450 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f57c0003680 con 0x55a451c7f120 2024-06-10T01:20:09.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.337+0000 7f57d1d8c700 1 --2- 172.21.15.7:0/869532450 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a451c7f120 0x55a451cebdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:09.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.338+0000 7f57d1d8c700 1 --2- 172.21.15.7:0/869532450 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a451c7f120 0x55a451cebdd0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=52ab989db4b4116c server_cookie=dea63f6fee881b3f in_seq=0 out_seq=0 2024-06-10T01:20:09.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.338+0000 7f57b17fa700 10 client.28286 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:20:09.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.338+0000 7f57b17fa700 1 -- 172.21.15.7:0/869532450 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f57c0003680 con 0x55a451c7f120 2024-06-10T01:20:09.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.338+0000 7f57b17fa700 10 client.28286 handle_command_reply: tid=0 2024-06-10T01:20:09.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.338+0000 7f57de873b80 1 client.28286 shutdown 2024-06-10T01:20:09.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.338+0000 7f57de873b80 1 -- 172.21.15.7:0/869532450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a451c6af60 msgr2=0x55a451c77a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:09.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.338+0000 7f57de873b80 1 --2- 172.21.15.7:0/869532450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a451c6af60 0x55a451c77a30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f57c0009990 tx=0x7f57c0011e90 comp rx=0 tx=0).stop 2024-06-10T01:20:09.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.338+0000 7f57de873b80 1 -- 172.21.15.7:0/869532450 shutdown_connections 2024-06-10T01:20:09.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.338+0000 7f57de873b80 1 --2- 172.21.15.7:0/869532450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a451c6b880 0x55a451c85ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.338+0000 7f57de873b80 1 --2- 172.21.15.7:0/869532450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a451c657a0 0x55a451c774f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.338+0000 7f57de873b80 1 --2- 172.21.15.7:0/869532450 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55a451c7f120 0x55a451cebdd0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.338+0000 7f57de873b80 1 --2- 172.21.15.7:0/869532450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a451c6af60 0x55a451c77a30 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.338+0000 7f57b17fa700 0 client.28286 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:20:09.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.338+0000 7f57de873b80 1 -- 172.21.15.7:0/869532450 >> 172.21.15.7:0/869532450 conn(0x55a451b92080 msgr2=0x55a451c69ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:09.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.338+0000 7f57de873b80 1 -- 172.21.15.7:0/869532450 shutdown_connections 2024-06-10T01:20:09.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.338+0000 7f57de873b80 1 -- 172.21.15.7:0/869532450 wait complete. 2024-06-10T01:20:09.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.339+0000 7f57de873b80 20 client.28286 trim_cache size 0 max 16384 2024-06-10T01:20:09.337 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:20:09.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.339+0000 7f57d3ff0700 1 -- 172.21.15.7:0/1970613538 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f57b40a2c20 msgr2=0x7f57b40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:09.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.339+0000 7f57d3ff0700 1 --2- 172.21.15.7:0/1970613538 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f57b40a2c20 0x7f57b40a50e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f57cc078940 tx=0x7f57c0009260 comp rx=0 tx=0).stop 2024-06-10T01:20:09.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.339+0000 7f57d3ff0700 1 -- 172.21.15.7:0/1970613538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57cc06a170 msgr2=0x7f57cc101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:09.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.339+0000 7f57d3ff0700 1 --2- 172.21.15.7:0/1970613538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57cc06a170 0x7f57cc101460 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f57bc007cc0 tx=0x7f57bc00d0d0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.339+0000 7f57d3ff0700 1 -- 172.21.15.7:0/1970613538 shutdown_connections 2024-06-10T01:20:09.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.339+0000 7f57d3ff0700 1 --2- 172.21.15.7:0/1970613538 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f57b40a2c20 0x7f57b40a50e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.339+0000 7f57d3ff0700 1 --2- 172.21.15.7:0/1970613538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57cc108500 0x7f57cc101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.339+0000 7f57d3ff0700 1 --2- 172.21.15.7:0/1970613538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57cc06a170 0x7f57cc101460 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.339+0000 7f57d3ff0700 1 --2- 172.21.15.7:0/1970613538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57cc06ab30 0x7f57cc1019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:09.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.339+0000 7f57d3ff0700 1 -- 172.21.15.7:0/1970613538 >> 172.21.15.7:0/1970613538 conn(0x7f57cc078340 msgr2=0x7f57cc100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:09.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.339+0000 7f57d3ff0700 1 -- 172.21.15.7:0/1970613538 shutdown_connections 2024-06-10T01:20:09.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:09.339+0000 7f57d3ff0700 1 -- 172.21.15.7:0/1970613538 wait complete. 2024-06-10T01:20:09.452 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:08 smithi175 ceph-mon[36636]: pgmap v856: 129 pgs: 129 active+clean; 355 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 227 KiB/s wr, 132 op/s 2024-06-10T01:20:09.469 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12545 18.48 MB/sec warmup 46 sec latency 17546.294 ms 2024-06-10T01:20:10.065 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:20:10.319 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:20:10.470 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12545 18.09 MB/sec warmup 47 sec latency 18546.635 ms 2024-06-10T01:20:11.048 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:20:11.249 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:10 smithi175 ceph-mon[36636]: pgmap v857: 129 pgs: 129 active+clean; 355 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 40 KiB/s wr, 91 op/s 2024-06-10T01:20:11.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:10 smithi007 ceph-mon[32398]: pgmap v857: 129 pgs: 129 active+clean; 355 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 40 KiB/s wr, 91 op/s 2024-06-10T01:20:11.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:10 smithi078 ceph-mon[36683]: pgmap v857: 129 pgs: 129 active+clean; 355 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 40 KiB/s wr, 91 op/s 2024-06-10T01:20:11.470 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12545 17.71 MB/sec warmup 48 sec latency 19546.745 ms 2024-06-10T01:20:11.842 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:20:12.470 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12545 17.35 MB/sec warmup 49 sec latency 20546.790 ms 2024-06-10T01:20:12.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.420+0000 7ffb3a279700 1 -- 172.21.15.7:0/1894337094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb341084f0 msgr2=0x7ffb341088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:12.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.420+0000 7ffb3a279700 1 --2- 172.21.15.7:0/1894337094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb341084f0 0x7ffb341088d0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7ffb24009990 tx=0x7ffb24009ca0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.420+0000 7ffb3a279700 1 -- 172.21.15.7:0/1894337094 shutdown_connections 2024-06-10T01:20:12.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.420+0000 7ffb3a279700 1 --2- 172.21.15.7:0/1894337094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb3406b520 0x7ffb3410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.420+0000 7ffb3a279700 1 --2- 172.21.15.7:0/1894337094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb341084f0 0x7ffb341088d0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.420+0000 7ffb3a279700 1 --2- 172.21.15.7:0/1894337094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb3406ab60 0x7ffb3406afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.420+0000 7ffb3a279700 1 -- 172.21.15.7:0/1894337094 >> 172.21.15.7:0/1894337094 conn(0x7ffb34078340 msgr2=0x7ffb34078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:12.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.420+0000 7ffb3a279700 1 -- 172.21.15.7:0/1894337094 shutdown_connections 2024-06-10T01:20:12.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.420+0000 7ffb3a279700 1 -- 172.21.15.7:0/1894337094 wait complete. 2024-06-10T01:20:12.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.421+0000 7ffb3a279700 1 Processor -- start 2024-06-10T01:20:12.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.421+0000 7ffb3a279700 1 -- start start 2024-06-10T01:20:12.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.421+0000 7ffb3a279700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb3406ab60 0x7ffb34101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.421+0000 7ffb3a279700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb3406b520 0x7ffb34101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.421+0000 7ffb3a279700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb341084f0 0x7ffb341022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.421+0000 7ffb3a279700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffb34112770 con 0x7ffb3406b520 2024-06-10T01:20:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.421+0000 7ffb3a279700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffb341125f0 con 0x7ffb341084f0 2024-06-10T01:20:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.421+0000 7ffb3a279700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffb341128f0 con 0x7ffb3406ab60 2024-06-10T01:20:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.421+0000 7ffb337fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb3406b520 0x7ffb34101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.421+0000 7ffb337fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb3406b520 0x7ffb34101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48728/0 (socket says 172.21.15.7:48728) 2024-06-10T01:20:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.421+0000 7ffb337fe700 1 -- 172.21.15.7:0/1603445091 learned_addr learned my addr 172.21.15.7:0/1603445091 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.422+0000 7ffb38816700 1 --2- 172.21.15.7:0/1603445091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb341084f0 0x7ffb341022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.422+0000 7ffb33fff700 1 --2- 172.21.15.7:0/1603445091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb3406ab60 0x7ffb34101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.422+0000 7ffb337fe700 1 -- 172.21.15.7:0/1603445091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb3406ab60 msgr2=0x7ffb34101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.422+0000 7ffb337fe700 1 --2- 172.21.15.7:0/1603445091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb3406ab60 0x7ffb34101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.422+0000 7ffb337fe700 1 -- 172.21.15.7:0/1603445091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb341084f0 msgr2=0x7ffb341022c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.422+0000 7ffb337fe700 1 --2- 172.21.15.7:0/1603445091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb341084f0 0x7ffb341022c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.422+0000 7ffb337fe700 1 -- 172.21.15.7:0/1603445091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb341a7160 con 0x7ffb3406b520 2024-06-10T01:20:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.422+0000 7ffb33fff700 1 --2- 172.21.15.7:0/1603445091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb3406ab60 0x7ffb34101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:20:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.422+0000 7ffb337fe700 1 --2- 172.21.15.7:0/1603445091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb3406b520 0x7ffb34101d80 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7ffb2800d980 tx=0x7ffb2800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.422+0000 7ffb317fa700 1 -- 172.21.15.7:0/1603445091 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb28017070 con 0x7ffb3406b520 2024-06-10T01:20:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.422+0000 7ffb317fa700 1 -- 172.21.15.7:0/1603445091 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb28005020 con 0x7ffb3406b520 2024-06-10T01:20:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.422+0000 7ffb3a279700 1 -- 172.21.15.7:0/1603445091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb341a7450 con 0x7ffb3406b520 2024-06-10T01:20:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.422+0000 7ffb317fa700 1 -- 172.21.15.7:0/1603445091 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb28009bc0 con 0x7ffb3406b520 2024-06-10T01:20:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.422+0000 7ffb3a279700 1 -- 172.21.15.7:0/1603445091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb341a7940 con 0x7ffb3406b520 2024-06-10T01:20:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.424+0000 7ffb317fa700 1 -- 172.21.15.7:0/1603445091 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7ffb280051d0 con 0x7ffb3406b520 2024-06-10T01:20:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.425+0000 7ffb317fa700 1 --2- 172.21.15.7:0/1603445091 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ffb1c0a2c20 0x7ffb1c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.425+0000 7ffb317fa700 1 -- 172.21.15.7:0/1603445091 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7ffb280cf2f0 con 0x7ffb3406b520 2024-06-10T01:20:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.425+0000 7ffb33fff700 1 --2- 172.21.15.7:0/1603445091 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ffb1c0a2c20 0x7ffb1c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.426+0000 7ffb33fff700 1 --2- 172.21.15.7:0/1603445091 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ffb1c0a2c20 0x7ffb1c0a50e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ffb24006010 tx=0x7ffb2400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.426+0000 7ffb44afcb80 1 Processor -- start 2024-06-10T01:20:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.426+0000 7ffb44afcb80 1 -- start start 2024-06-10T01:20:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.426+0000 7ffb44afcb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56143ae4b640 0x56143ae4ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.426+0000 7ffb44afcb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56143ae4bf60 0x56143ae459e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.426+0000 7ffb44afcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56143ae45f20 0x56143ae46340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.426+0000 7ffb44afcb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56143ad97810 con 0x56143ae4b640 2024-06-10T01:20:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.426+0000 7ffb44afcb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56143ad3f720 con 0x56143ae45f20 2024-06-10T01:20:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.426+0000 7ffb44afcb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56143ad937d0 con 0x56143ae4bf60 2024-06-10T01:20:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.426+0000 7ffb33fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56143ae4b640 0x56143ae4ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.426+0000 7ffb33fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56143ae4b640 0x56143ae4ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48740/0 (socket says 172.21.15.7:48740) 2024-06-10T01:20:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.426+0000 7ffb33fff700 1 -- 172.21.15.7:0/3658368562 learned_addr learned my addr 172.21.15.7:0/3658368562 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.426+0000 7ffb337fe700 1 --2- 172.21.15.7:0/3658368562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56143ae4bf60 0x56143ae459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.426+0000 7ffb33fff700 1 -- 172.21.15.7:0/3658368562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56143ae4bf60 msgr2=0x56143ae459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.426+0000 7ffb33fff700 1 --2- 172.21.15.7:0/3658368562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56143ae4bf60 0x56143ae459e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.426+0000 7ffb33fff700 1 -- 172.21.15.7:0/3658368562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56143ae45f20 msgr2=0x56143ae46340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:12.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.426+0000 7ffb33fff700 1 --2- 172.21.15.7:0/3658368562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56143ae45f20 0x56143ae46340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.426+0000 7ffb33fff700 1 -- 172.21.15.7:0/3658368562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56143ad35f50 con 0x56143ae4b640 2024-06-10T01:20:12.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.427+0000 7ffb33fff700 1 --2- 172.21.15.7:0/3658368562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56143ae4b640 0x56143ae4ba20 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7ffb2400b900 tx=0x7ffb2401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb80094357c3993e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:12.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.427+0000 7ffb1a7fc700 1 -- 172.21.15.7:0/3658368562 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2400bdf0 con 0x56143ae4b640 2024-06-10T01:20:12.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.427+0000 7ffb1a7fc700 1 -- 172.21.15.7:0/3658368562 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb24003e40 con 0x56143ae4b640 2024-06-10T01:20:12.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.427+0000 7ffb1a7fc700 1 -- 172.21.15.7:0/3658368562 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2400f660 con 0x56143ae4b640 2024-06-10T01:20:12.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.427+0000 7ffb44afcb80 1 -- 172.21.15.7:0/3658368562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56143ae4b640 msgr2=0x56143ae4ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:12.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.427+0000 7ffb44afcb80 1 --2- 172.21.15.7:0/3658368562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56143ae4b640 0x56143ae4ba20 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7ffb2400b900 tx=0x7ffb2401f000 comp rx=0 tx=0).stop 2024-06-10T01:20:12.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.428+0000 7ffb44afcb80 1 -- 172.21.15.7:0/3658368562 shutdown_connections 2024-06-10T01:20:12.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.428+0000 7ffb44afcb80 1 --2- 172.21.15.7:0/3658368562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56143ae45f20 0x56143ae46340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.428+0000 7ffb44afcb80 1 --2- 172.21.15.7:0/3658368562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56143ae4b640 0x56143ae4ba20 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.428+0000 7ffb44afcb80 1 --2- 172.21.15.7:0/3658368562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56143ae4bf60 0x56143ae459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.428+0000 7ffb44afcb80 1 -- 172.21.15.7:0/3658368562 >> 172.21.15.7:0/3658368562 conn(0x56143ad70cc0 msgr2=0x56143ae40850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:12.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.428+0000 7ffb44afcb80 1 -- 172.21.15.7:0/3658368562 shutdown_connections 2024-06-10T01:20:12.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.428+0000 7ffb44afcb80 1 -- 172.21.15.7:0/3658368562 wait complete. 2024-06-10T01:20:12.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.428+0000 7ffb44afcb80 1 -- start start 2024-06-10T01:20:12.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.428+0000 7ffb44afcb80 1 Processor -- start 2024-06-10T01:20:12.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.429+0000 7ffb44afcb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56143ae45f20 0x56143ae51cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.429+0000 7ffb44afcb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56143ae4b640 0x56143ae4efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.429+0000 7ffb44afcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56143ae4bf60 0x56143ae4f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.429+0000 7ffb44afcb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56143ae52970 con 0x56143ae4b640 2024-06-10T01:20:12.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.429+0000 7ffb44afcb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56143ad37bf0 con 0x56143ae4bf60 2024-06-10T01:20:12.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.429+0000 7ffb44afcb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56143ae52af0 con 0x56143ae45f20 2024-06-10T01:20:12.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.429+0000 7ffb44afcb80 10 client.0 _ll_register_callbacks cb 0x56143ad31920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:20:12.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.429+0000 7ffb337fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56143ae4b640 0x56143ae4efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.429+0000 7ffb337fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56143ae4b640 0x56143ae4efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48744/0 (socket says 172.21.15.7:48744) 2024-06-10T01:20:12.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.429+0000 7ffb337fe700 1 -- 172.21.15.7:0/3096965993 learned_addr learned my addr 172.21.15.7:0/3096965993 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:12.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.429+0000 7ffb33fff700 1 --2- 172.21.15.7:0/3096965993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56143ae45f20 0x56143ae51cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.429+0000 7ffb38816700 1 --2- 172.21.15.7:0/3096965993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56143ae4bf60 0x56143ae4f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.429+0000 7ffb337fe700 1 -- 172.21.15.7:0/3096965993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56143ae45f20 msgr2=0x56143ae51cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:12.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.429+0000 7ffb337fe700 1 --2- 172.21.15.7:0/3096965993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56143ae45f20 0x56143ae51cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.429+0000 7ffb337fe700 1 -- 172.21.15.7:0/3096965993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56143ae4bf60 msgr2=0x56143ae4f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:12.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.429+0000 7ffb337fe700 1 --2- 172.21.15.7:0/3096965993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56143ae4bf60 0x56143ae4f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.429+0000 7ffb337fe700 1 -- 172.21.15.7:0/3096965993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56143ace68d0 con 0x56143ae4b640 2024-06-10T01:20:12.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.429+0000 7ffb337fe700 1 --2- 172.21.15.7:0/3096965993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56143ae4b640 0x56143ae4efd0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7ffb2809b4d0 tx=0x7ffb2809c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d47325b455b7bc3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:12.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.429+0000 7ffb137fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:20:12.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.429+0000 7ffb137fe700 1 -- 172.21.15.7:0/3096965993 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb280b4060 con 0x56143ae4b640 2024-06-10T01:20:12.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.429+0000 7ffb137fe700 1 -- 172.21.15.7:0/3096965993 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7ffb28005560 con 0x56143ae4b640 2024-06-10T01:20:12.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.429+0000 7ffb44afcb80 1 -- 172.21.15.7:0/3096965993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56143ada4d10 con 0x56143ae4b640 2024-06-10T01:20:12.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.430+0000 7ffb137fe700 1 -- 172.21.15.7:0/3096965993 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ffb2809fb80 con 0x56143ae4b640 2024-06-10T01:20:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.430+0000 7ffb44afcb80 10 client.19014 fetch_fsmap learned FSMap version 44 2024-06-10T01:20:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.430+0000 7ffb44afcb80 1 -- 172.21.15.7:0/3096965993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56143ae5d2d0 con 0x56143ae4b640 2024-06-10T01:20:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.430+0000 7ffb137fe700 1 -- 172.21.15.7:0/3096965993 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56143ada4d10 con 0x56143ae4b640 2024-06-10T01:20:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.430+0000 7ffb137fe700 1 -- 172.21.15.7:0/3096965993 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7ffb280a3fb0 con 0x56143ae4b640 2024-06-10T01:20:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.430+0000 7ffb44afcb80 10 client.19014 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:20:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.430+0000 7ffb44afcb80 10 client.19014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:20:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.430+0000 7ffb44afcb80 20 client.19014 populate_metadata read hostname 'smithi007' 2024-06-10T01:20:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.430+0000 7ffb44afcb80 1 --2- 172.21.15.7:0/3096965993 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56143ae5f860 0x56143ae61c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.430+0000 7ffb44afcb80 4 client.19014 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:20:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.430+0000 7ffb44afcb80 1 -- 172.21.15.7:0/3096965993 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56143ad9f290 con 0x56143ae5f860 2024-06-10T01:20:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.430+0000 7ffb33fff700 1 --2- 172.21.15.7:0/3096965993 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56143ae5f860 0x56143ae61c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.431+0000 7ffb137fe700 10 client.19014 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.431+0000 7ffb33fff700 1 --2- 172.21.15.7:0/3096965993 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56143ae5f860 0x56143ae61c50 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d819bcd7a32ea79 server_cookie=339a8e9f01a94f31 in_seq=0 out_seq=0 2024-06-10T01:20:12.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.432+0000 7ffb137fe700 1 -- 172.21.15.7:0/3096965993 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56143ad9f290 con 0x56143ae5f860 2024-06-10T01:20:12.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.432+0000 7ffb137fe700 10 client.19014 handle_command_reply: tid=0 2024-06-10T01:20:12.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.432+0000 7ffb44afcb80 1 client.19014 shutdown 2024-06-10T01:20:12.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.432+0000 7ffb44afcb80 1 -- 172.21.15.7:0/3096965993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56143ae4b640 msgr2=0x56143ae4efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:12.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.432+0000 7ffb44afcb80 1 --2- 172.21.15.7:0/3096965993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56143ae4b640 0x56143ae4efd0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7ffb2809b4d0 tx=0x7ffb2809c010 comp rx=0 tx=0).stop 2024-06-10T01:20:12.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.432+0000 7ffb44afcb80 1 -- 172.21.15.7:0/3096965993 shutdown_connections 2024-06-10T01:20:12.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.432+0000 7ffb44afcb80 1 --2- 172.21.15.7:0/3096965993 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56143ae5f860 0x56143ae61c50 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.432+0000 7ffb44afcb80 1 --2- 172.21.15.7:0/3096965993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56143ae4bf60 0x56143ae4f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.432+0000 7ffb44afcb80 1 --2- 172.21.15.7:0/3096965993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56143ae45f20 0x56143ae51cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.432+0000 7ffb44afcb80 1 --2- 172.21.15.7:0/3096965993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56143ae4b640 0x56143ae4efd0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.432+0000 7ffb137fe700 0 client.19014 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:12.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.432+0000 7ffb44afcb80 1 -- 172.21.15.7:0/3096965993 >> 172.21.15.7:0/3096965993 conn(0x56143ad70cc0 msgr2=0x56143ae449d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:12.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.432+0000 7ffb44afcb80 1 -- 172.21.15.7:0/3096965993 shutdown_connections 2024-06-10T01:20:12.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.432+0000 7ffb44afcb80 1 -- 172.21.15.7:0/3096965993 wait complete. 2024-06-10T01:20:12.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.432+0000 7ffb44afcb80 20 client.19014 trim_cache size 0 max 16384 2024-06-10T01:20:12.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.447+0000 7ffb44afcb80 1 Processor -- start 2024-06-10T01:20:12.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.448+0000 7ffb44afcb80 1 -- start start 2024-06-10T01:20:12.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.448+0000 7ffb44afcb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56143ae45f20 0x56143ae459e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.448+0000 7ffb44afcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56143ae4b640 0x56143ae3a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.448+0000 7ffb44afcb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56143ae4bf60 0x56143ae674e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.448+0000 7ffb44afcb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffb341b2e40 con 0x56143ae45f20 2024-06-10T01:20:12.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.448+0000 7ffb44afcb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffb341b0640 con 0x56143ae4b640 2024-06-10T01:20:12.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.448+0000 7ffb44afcb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffb341ae490 con 0x56143ae4bf60 2024-06-10T01:20:12.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.448+0000 7ffb33fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56143ae45f20 0x56143ae459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.448+0000 7ffb33fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56143ae45f20 0x56143ae459e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48758/0 (socket says 172.21.15.7:48758) 2024-06-10T01:20:12.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.448+0000 7ffb33fff700 1 -- 172.21.15.7:0/2691746161 learned_addr learned my addr 172.21.15.7:0/2691746161 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:12.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.448+0000 7ffb337fe700 1 --2- 172.21.15.7:0/2691746161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56143ae4b640 0x56143ae3a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.448+0000 7ffb33fff700 1 -- 172.21.15.7:0/2691746161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56143ae4bf60 msgr2=0x56143ae674e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.448+0000 7ffb33fff700 1 --2- 172.21.15.7:0/2691746161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56143ae4bf60 0x56143ae674e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.448+0000 7ffb33fff700 1 -- 172.21.15.7:0/2691746161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56143ae4b640 msgr2=0x56143ae3a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.448+0000 7ffb33fff700 1 --2- 172.21.15.7:0/2691746161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56143ae4b640 0x56143ae3a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.448+0000 7ffb33fff700 1 -- 172.21.15.7:0/2691746161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56143ae622b0 con 0x56143ae45f20 2024-06-10T01:20:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.448+0000 7ffb337fe700 1 --2- 172.21.15.7:0/2691746161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56143ae4b640 0x56143ae3a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:20:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.448+0000 7ffb33fff700 1 --2- 172.21.15.7:0/2691746161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56143ae45f20 0x56143ae459e0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7ffb2400b8a0 tx=0x7ffb24019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6acb11d1c8d310d8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.448+0000 7ffb19ffb700 1 -- 172.21.15.7:0/2691746161 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb24003ae0 con 0x56143ae45f20 2024-06-10T01:20:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.448+0000 7ffb19ffb700 1 -- 172.21.15.7:0/2691746161 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb24003c80 con 0x56143ae45f20 2024-06-10T01:20:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.448+0000 7ffb19ffb700 1 -- 172.21.15.7:0/2691746161 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2400f460 con 0x56143ae45f20 2024-06-10T01:20:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.449+0000 7ffb44afcb80 1 -- 172.21.15.7:0/2691746161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56143ae45f20 msgr2=0x56143ae459e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.449+0000 7ffb44afcb80 1 --2- 172.21.15.7:0/2691746161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56143ae45f20 0x56143ae459e0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7ffb2400b8a0 tx=0x7ffb24019040 comp rx=0 tx=0).stop 2024-06-10T01:20:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.449+0000 7ffb44afcb80 1 -- 172.21.15.7:0/2691746161 shutdown_connections 2024-06-10T01:20:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.449+0000 7ffb44afcb80 1 --2- 172.21.15.7:0/2691746161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56143ae4bf60 0x56143ae674e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.449+0000 7ffb44afcb80 1 --2- 172.21.15.7:0/2691746161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56143ae45f20 0x56143ae459e0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.449+0000 7ffb44afcb80 1 --2- 172.21.15.7:0/2691746161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56143ae4b640 0x56143ae3a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.449+0000 7ffb44afcb80 1 -- 172.21.15.7:0/2691746161 >> 172.21.15.7:0/2691746161 conn(0x56143ad70cc0 msgr2=0x56143ae39e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.449+0000 7ffb44afcb80 1 -- 172.21.15.7:0/2691746161 shutdown_connections 2024-06-10T01:20:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.449+0000 7ffb44afcb80 1 -- 172.21.15.7:0/2691746161 wait complete. 2024-06-10T01:20:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.450+0000 7ffb44afcb80 1 -- start start 2024-06-10T01:20:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.450+0000 7ffb44afcb80 1 Processor -- start 2024-06-10T01:20:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.450+0000 7ffb44afcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56143ae45f20 0x56143ae53550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.450+0000 7ffb44afcb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56143ae4b640 0x56143ae53a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.450+0000 7ffb44afcb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56143ae4bf60 0x56143aec24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.450+0000 7ffb44afcb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffb341aeb00 con 0x56143ae4bf60 2024-06-10T01:20:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.450+0000 7ffb44afcb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffb341aff70 con 0x56143ae45f20 2024-06-10T01:20:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.450+0000 7ffb44afcb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffb341af1d0 con 0x56143ae4b640 2024-06-10T01:20:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.450+0000 7ffb44afcb80 10 client.0 _ll_register_callbacks cb 0x56143ad31920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:20:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.450+0000 7ffb38816700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56143ae4bf60 0x56143aec24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.450+0000 7ffb38816700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56143ae4bf60 0x56143aec24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48774/0 (socket says 172.21.15.7:48774) 2024-06-10T01:20:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.450+0000 7ffb38816700 1 -- 172.21.15.7:0/4113653005 learned_addr learned my addr 172.21.15.7:0/4113653005 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.450+0000 7ffb33fff700 1 --2- 172.21.15.7:0/4113653005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56143ae45f20 0x56143ae53550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.450+0000 7ffb337fe700 1 --2- 172.21.15.7:0/4113653005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56143ae4b640 0x56143ae53a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.451+0000 7ffb38816700 1 -- 172.21.15.7:0/4113653005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56143ae4b640 msgr2=0x56143ae53a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.451+0000 7ffb38816700 1 --2- 172.21.15.7:0/4113653005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56143ae4b640 0x56143ae53a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.451+0000 7ffb38816700 1 -- 172.21.15.7:0/4113653005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56143ae45f20 msgr2=0x56143ae53550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.451+0000 7ffb38816700 1 --2- 172.21.15.7:0/4113653005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56143ae45f20 0x56143ae53550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.451+0000 7ffb38816700 1 -- 172.21.15.7:0/4113653005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56143aec2b50 con 0x56143ae4bf60 2024-06-10T01:20:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.451+0000 7ffb38816700 1 --2- 172.21.15.7:0/4113653005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56143ae4bf60 0x56143aec24e0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c003fd0 tx=0x7ffb2c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d259f91cdbc968 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.451+0000 7ffb137fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:20:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.451+0000 7ffb137fe700 1 -- 172.21.15.7:0/4113653005 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c01a070 con 0x56143ae4bf60 2024-06-10T01:20:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.451+0000 7ffb137fe700 1 -- 172.21.15.7:0/4113653005 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7ffb2c004db0 con 0x56143ae4bf60 2024-06-10T01:20:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.451+0000 7ffb44afcb80 1 -- 172.21.15.7:0/4113653005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56143ac19250 con 0x56143ae4bf60 2024-06-10T01:20:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.451+0000 7ffb137fe700 1 -- 172.21.15.7:0/4113653005 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ffb2c025050 con 0x56143ae4bf60 2024-06-10T01:20:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.451+0000 7ffb44afcb80 10 client.19020 fetch_fsmap learned FSMap version 44 2024-06-10T01:20:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.451+0000 7ffb44afcb80 1 -- 172.21.15.7:0/4113653005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56143aec9f50 con 0x56143ae4bf60 2024-06-10T01:20:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.451+0000 7ffb137fe700 1 -- 172.21.15.7:0/4113653005 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56143ac19250 con 0x56143ae4bf60 2024-06-10T01:20:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.451+0000 7ffb137fe700 1 -- 172.21.15.7:0/4113653005 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7ffb2c016730 con 0x56143ae4bf60 2024-06-10T01:20:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.451+0000 7ffb44afcb80 10 client.19020 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:20:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.451+0000 7ffb44afcb80 10 client.19020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:20:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.451+0000 7ffb44afcb80 20 client.19020 populate_metadata read hostname 'smithi007' 2024-06-10T01:20:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.451+0000 7ffb44afcb80 1 --2- 172.21.15.7:0/4113653005 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56143aeca720 0x56143aeccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.451+0000 7ffb44afcb80 4 client.19020 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:20:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.451+0000 7ffb44afcb80 1 -- 172.21.15.7:0/4113653005 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56143ad68b50 con 0x56143aeca720 2024-06-10T01:20:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.452+0000 7ffb33fff700 1 --2- 172.21.15.7:0/4113653005 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56143aeca720 0x56143aeccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.452+0000 7ffb137fe700 10 client.19020 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:12.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.452+0000 7ffb33fff700 1 --2- 172.21.15.7:0/4113653005 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56143aeca720 0x56143aeccb40 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e2e9991720f2f5a server_cookie=71345d3c04da63b5 in_seq=0 out_seq=0 2024-06-10T01:20:12.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.452+0000 7ffb137fe700 1 -- 172.21.15.7:0/4113653005 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56143ad68b50 con 0x56143aeca720 2024-06-10T01:20:12.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.452+0000 7ffb137fe700 10 client.19020 handle_command_reply: tid=0 2024-06-10T01:20:12.666 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:20:12.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.452+0000 7ffb44afcb80 1 client.19020 shutdown 2024-06-10T01:20:12.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.452+0000 7ffb44afcb80 1 -- 172.21.15.7:0/4113653005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56143ae4bf60 msgr2=0x56143aec24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:12.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.452+0000 7ffb44afcb80 1 --2- 172.21.15.7:0/4113653005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56143ae4bf60 0x56143aec24e0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c003fd0 tx=0x7ffb2c004040 comp rx=0 tx=0).stop 2024-06-10T01:20:12.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.453+0000 7ffb44afcb80 1 -- 172.21.15.7:0/4113653005 shutdown_connections 2024-06-10T01:20:12.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.453+0000 7ffb44afcb80 1 --2- 172.21.15.7:0/4113653005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56143ae4bf60 0x56143aec24e0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.453+0000 7ffb44afcb80 1 --2- 172.21.15.7:0/4113653005 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56143aeca720 0x56143aeccb40 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.453+0000 7ffb44afcb80 1 --2- 172.21.15.7:0/4113653005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56143ae45f20 0x56143ae53550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.453+0000 7ffb44afcb80 1 --2- 172.21.15.7:0/4113653005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56143ae4b640 0x56143ae53a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.453+0000 7ffb137fe700 0 client.19020 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:12.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.453+0000 7ffb44afcb80 1 -- 172.21.15.7:0/4113653005 >> 172.21.15.7:0/4113653005 conn(0x56143ad70cc0 msgr2=0x56143ae4cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:12.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.453+0000 7ffb44afcb80 1 -- 172.21.15.7:0/4113653005 shutdown_connections 2024-06-10T01:20:12.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.453+0000 7ffb44afcb80 1 -- 172.21.15.7:0/4113653005 wait complete. 2024-06-10T01:20:12.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.453+0000 7ffb44afcb80 20 client.19020 trim_cache size 0 max 16384 2024-06-10T01:20:12.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.453+0000 7ffb3a279700 1 -- 172.21.15.7:0/1603445091 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ffb1c0a2c20 msgr2=0x7ffb1c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:12.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.453+0000 7ffb3a279700 1 --2- 172.21.15.7:0/1603445091 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ffb1c0a2c20 0x7ffb1c0a50e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ffb24006010 tx=0x7ffb2400b540 comp rx=0 tx=0).stop 2024-06-10T01:20:12.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.453+0000 7ffb3a279700 1 -- 172.21.15.7:0/1603445091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb3406b520 msgr2=0x7ffb34101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:12.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.453+0000 7ffb3a279700 1 --2- 172.21.15.7:0/1603445091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb3406b520 0x7ffb34101d80 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7ffb2800d980 tx=0x7ffb2800dc90 comp rx=0 tx=0).stop 2024-06-10T01:20:12.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.453+0000 7ffb3a279700 1 -- 172.21.15.7:0/1603445091 shutdown_connections 2024-06-10T01:20:12.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.453+0000 7ffb3a279700 1 --2- 172.21.15.7:0/1603445091 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ffb1c0a2c20 0x7ffb1c0a50e0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.453+0000 7ffb3a279700 1 --2- 172.21.15.7:0/1603445091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb341084f0 0x7ffb341022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.453+0000 7ffb3a279700 1 --2- 172.21.15.7:0/1603445091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb3406ab60 0x7ffb34101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.453+0000 7ffb3a279700 1 --2- 172.21.15.7:0/1603445091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb3406b520 0x7ffb34101d80 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.453+0000 7ffb3a279700 1 -- 172.21.15.7:0/1603445091 >> 172.21.15.7:0/1603445091 conn(0x7ffb34078340 msgr2=0x7ffb341004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:12.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.454+0000 7ffb3a279700 1 -- 172.21.15.7:0/1603445091 shutdown_connections 2024-06-10T01:20:12.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:12.454+0000 7ffb3a279700 1 -- 172.21.15.7:0/1603445091 wait complete. 2024-06-10T01:20:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.793+0000 7fc111095640 1 -- 172.21.15.7:0/2627219655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc10c100900 msgr2=0x7fc10c100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.793+0000 7fc111095640 1 --2- 172.21.15.7:0/2627219655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc10c100900 0x7fc10c100d20 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc009900 tx=0x7fc0fc02a410 comp rx=0 tx=0).stop 2024-06-10T01:20:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.793+0000 7fc111095640 1 -- 172.21.15.7:0/2627219655 shutdown_connections 2024-06-10T01:20:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.793+0000 7fc111095640 1 --2- 172.21.15.7:0/2627219655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc10c1024b0 0x7fc10c108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.793+0000 7fc111095640 1 --2- 172.21.15.7:0/2627219655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc10c101af0 0x7fc10c101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.793+0000 7fc111095640 1 --2- 172.21.15.7:0/2627219655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc10c100900 0x7fc10c100d20 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.793+0000 7fc111095640 1 -- 172.21.15.7:0/2627219655 >> 172.21.15.7:0/2627219655 conn(0x7fc10c0fbe70 msgr2=0x7fc10c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.793+0000 7fc111095640 1 -- 172.21.15.7:0/2627219655 shutdown_connections 2024-06-10T01:20:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.793+0000 7fc111095640 1 -- 172.21.15.7:0/2627219655 wait complete. 2024-06-10T01:20:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.794+0000 7fc111095640 1 Processor -- start 2024-06-10T01:20:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.794+0000 7fc111095640 1 -- start start 2024-06-10T01:20:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.794+0000 7fc111095640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc10c100900 0x7fc10c10b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.794+0000 7fc111095640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc10c101af0 0x7fc10c1098c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.794+0000 7fc111095640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc10c1024b0 0x7fc10c109e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.794+0000 7fc111095640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c10d6a0 con 0x7fc10c1024b0 2024-06-10T01:20:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.794+0000 7fc111095640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c10d520 con 0x7fc10c100900 2024-06-10T01:20:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.794+0000 7fc111095640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c10d820 con 0x7fc10c101af0 2024-06-10T01:20:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.794+0000 7fc10b59e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc10c1024b0 0x7fc10c109e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.794+0000 7fc10b59e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc10c1024b0 0x7fc10c109e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48796/0 (socket says 172.21.15.7:48796) 2024-06-10T01:20:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.794+0000 7fc10b59e640 1 -- 172.21.15.7:0/1822261137 learned_addr learned my addr 172.21.15.7:0/1822261137 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.794+0000 7fc10ad9d640 1 --2- 172.21.15.7:0/1822261137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc10c100900 0x7fc10c10b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.794+0000 7fc103fff640 1 --2- 172.21.15.7:0/1822261137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc10c101af0 0x7fc10c1098c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.794+0000 7fc10b59e640 1 -- 172.21.15.7:0/1822261137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc10c101af0 msgr2=0x7fc10c1098c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.794+0000 7fc10b59e640 1 --2- 172.21.15.7:0/1822261137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc10c101af0 0x7fc10c1098c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.794+0000 7fc10b59e640 1 -- 172.21.15.7:0/1822261137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc10c100900 msgr2=0x7fc10c10b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.794+0000 7fc10b59e640 1 --2- 172.21.15.7:0/1822261137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc10c100900 0x7fc10c10b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.794+0000 7fc10b59e640 1 -- 172.21.15.7:0/1822261137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10c10a690 con 0x7fc10c1024b0 2024-06-10T01:20:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.794+0000 7fc10ad9d640 1 --2- 172.21.15.7:0/1822261137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc10c100900 0x7fc10c10b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:20:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.794+0000 7fc103fff640 1 --2- 172.21.15.7:0/1822261137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc10c101af0 0x7fc10c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:20:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.795+0000 7fc10b59e640 1 --2- 172.21.15.7:0/1822261137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc10c1024b0 0x7fc10c109e00 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fc10400eaa0 tx=0x7fc10400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.795+0000 7fc108d99640 1 -- 172.21.15.7:0/1822261137 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10400cdb0 con 0x7fc10c1024b0 2024-06-10T01:20:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.795+0000 7fc108d99640 1 -- 172.21.15.7:0/1822261137 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc104005020 con 0x7fc10c1024b0 2024-06-10T01:20:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.795+0000 7fc111095640 1 -- 172.21.15.7:0/1822261137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc10c1a1ea0 con 0x7fc10c1024b0 2024-06-10T01:20:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.795+0000 7fc108d99640 1 -- 172.21.15.7:0/1822261137 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1040116d0 con 0x7fc10c1024b0 2024-06-10T01:20:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.795+0000 7fc111095640 1 -- 172.21.15.7:0/1822261137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc10c1a2300 con 0x7fc10c1024b0 2024-06-10T01:20:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.797+0000 7fc108d99640 1 -- 172.21.15.7:0/1822261137 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fc1040054d0 con 0x7fc10c1024b0 2024-06-10T01:20:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.797+0000 7fc113d9c740 1 Processor -- start 2024-06-10T01:20:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.797+0000 7fc113d9c740 1 -- start start 2024-06-10T01:20:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.798+0000 7fc113d9c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56245c6e1010 0x56245c6e1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.798+0000 7fc113d9c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245c6e1a00 0x56245c6e6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.798+0000 7fc113d9c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56245c6e6b70 0x56245c6e9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.798+0000 7fc113d9c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56245c6444f0 con 0x56245c6e1a00 2024-06-10T01:20:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.798+0000 7fc113d9c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56245c5e2870 con 0x56245c6e6b70 2024-06-10T01:20:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.798+0000 7fc113d9c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56245c6404b0 con 0x56245c6e1010 2024-06-10T01:20:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.798+0000 7fc10ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245c6e1a00 0x56245c6e6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.798+0000 7fc108d99640 1 --2- 172.21.15.7:0/1822261137 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc0f80a2b80 0x7fc0f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.798+0000 7fc10ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245c6e1a00 0x56245c6e6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48804/0 (socket says 172.21.15.7:48804) 2024-06-10T01:20:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.798+0000 7fc10ad9d640 1 -- 172.21.15.7:0/2511753030 learned_addr learned my addr 172.21.15.7:0/2511753030 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.798+0000 7fc103fff640 1 --2- 172.21.15.7:0/2511753030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56245c6e1010 0x56245c6e1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.798+0000 7fc10b59e640 1 --2- 172.21.15.7:0/1822261137 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc0f80a2b80 0x7fc0f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.798+0000 7fc103fff640 1 --2- 172.21.15.7:0/2511753030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56245c6e6b70 0x56245c6e9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.798+0000 7fc10ad9d640 1 -- 172.21.15.7:0/2511753030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56245c6e1010 msgr2=0x56245c6e1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.798+0000 7fc10ad9d640 1 --2- 172.21.15.7:0/2511753030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56245c6e1010 0x56245c6e1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.798+0000 7fc10ad9d640 1 -- 172.21.15.7:0/2511753030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56245c6e6b70 msgr2=0x56245c6e9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.798+0000 7fc10ad9d640 1 --2- 172.21.15.7:0/2511753030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56245c6e6b70 0x56245c6e9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.798+0000 7fc10ad9d640 1 -- 172.21.15.7:0/2511753030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56245c5fe9b0 con 0x56245c6e1a00 2024-06-10T01:20:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.798+0000 7fc103fff640 1 --2- 172.21.15.7:0/2511753030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56245c6e6b70 0x56245c6e9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:20:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.798+0000 7fc108d99640 1 -- 172.21.15.7:0/1822261137 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc1040cf570 con 0x7fc10c1024b0 2024-06-10T01:20:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.799+0000 7fc10ad9d640 1 --2- 172.21.15.7:0/2511753030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245c6e1a00 0x56245c6e6500 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fc10c101950 tx=0x7fc0fc005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6380575970020b0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.799+0000 7fc10b59e640 1 --2- 172.21.15.7:0/1822261137 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc0f80a2b80 0x7fc0f80a5040 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc10409b750 tx=0x7fc10404b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.799+0000 7fc1017fa640 1 -- 172.21.15.7:0/2511753030 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc040070 con 0x56245c6e1a00 2024-06-10T01:20:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.799+0000 7fc1017fa640 1 -- 172.21.15.7:0/2511753030 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0fc005040 con 0x56245c6e1a00 2024-06-10T01:20:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.799+0000 7fc1017fa640 1 -- 172.21.15.7:0/2511753030 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc035040 con 0x56245c6e1a00 2024-06-10T01:20:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.799+0000 7fc113d9c740 1 -- 172.21.15.7:0/2511753030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245c6e1a00 msgr2=0x56245c6e6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.799+0000 7fc113d9c740 1 --2- 172.21.15.7:0/2511753030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245c6e1a00 0x56245c6e6500 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fc10c101950 tx=0x7fc0fc005c00 comp rx=0 tx=0).stop 2024-06-10T01:20:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.800+0000 7fc113d9c740 1 -- 172.21.15.7:0/2511753030 shutdown_connections 2024-06-10T01:20:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.800+0000 7fc113d9c740 1 --2- 172.21.15.7:0/2511753030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56245c6e6b70 0x56245c6e9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.800+0000 7fc113d9c740 1 --2- 172.21.15.7:0/2511753030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245c6e1a00 0x56245c6e6500 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.800+0000 7fc113d9c740 1 --2- 172.21.15.7:0/2511753030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56245c6e1010 0x56245c6e1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.800+0000 7fc113d9c740 1 -- 172.21.15.7:0/2511753030 >> 172.21.15.7:0/2511753030 conn(0x56245c6dc680 msgr2=0x56245c6dca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.800+0000 7fc113d9c740 1 -- 172.21.15.7:0/2511753030 shutdown_connections 2024-06-10T01:20:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.800+0000 7fc113d9c740 1 -- 172.21.15.7:0/2511753030 wait complete. 2024-06-10T01:20:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.800+0000 7fc113d9c740 1 -- start start 2024-06-10T01:20:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.800+0000 7fc113d9c740 1 Processor -- start 2024-06-10T01:20:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.800+0000 7fc113d9c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56245c6e1010 0x56245c6eb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.800+0000 7fc113d9c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245c6e1a00 0x56245c6eb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.800+0000 7fc113d9c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56245c6e6b70 0x56245c6f27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.800+0000 7fc113d9c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56245c6e9ac0 con 0x56245c6e1a00 2024-06-10T01:20:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.800+0000 7fc113d9c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56245c5cdae0 con 0x56245c6e1010 2024-06-10T01:20:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.800+0000 7fc113d9c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56245c6e9c40 con 0x56245c6e6b70 2024-06-10T01:20:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.800+0000 7fc113d9c740 10 client.0 _ll_register_callbacks cb 0x56245c6069b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:20:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.800+0000 7fc10ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245c6e1a00 0x56245c6eb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.800+0000 7fc10ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245c6e1a00 0x56245c6eb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48820/0 (socket says 172.21.15.7:48820) 2024-06-10T01:20:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.800+0000 7fc10ad9d640 1 -- 172.21.15.7:0/420568394 learned_addr learned my addr 172.21.15.7:0/420568394 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.801+0000 7fc103fff640 1 --2- 172.21.15.7:0/420568394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56245c6e6b70 0x56245c6f27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.801+0000 7fc103fff640 1 --2- 172.21.15.7:0/420568394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56245c6e1010 0x56245c6eb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.801+0000 7fc10ad9d640 1 -- 172.21.15.7:0/420568394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56245c6e6b70 msgr2=0x56245c6f27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.801+0000 7fc10ad9d640 1 --2- 172.21.15.7:0/420568394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56245c6e6b70 0x56245c6f27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.801+0000 7fc10ad9d640 1 -- 172.21.15.7:0/420568394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56245c6e1010 msgr2=0x56245c6eb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.801+0000 7fc10ad9d640 1 --2- 172.21.15.7:0/420568394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56245c6e1010 0x56245c6eb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.801+0000 7fc10ad9d640 1 -- 172.21.15.7:0/420568394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56245c358210 con 0x56245c6e1a00 2024-06-10T01:20:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.801+0000 7fc103fff640 1 --2- 172.21.15.7:0/420568394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56245c6e6b70 0x56245c6f27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:20:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.801+0000 7fc10ad9d640 1 --2- 172.21.15.7:0/420568394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245c6e1a00 0x56245c6eb770 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc000c00 tx=0x7fc0fc005420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c832a9ffcd50b6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.801+0000 7fc0f27fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:20:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.801+0000 7fc0f27fc640 1 -- 172.21.15.7:0/420568394 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc035070 con 0x56245c6e1a00 2024-06-10T01:20:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.801+0000 7fc0f27fc640 1 -- 172.21.15.7:0/420568394 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc0fc004ce0 con 0x56245c6e1a00 2024-06-10T01:20:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.801+0000 7fc113d9c740 1 -- 172.21.15.7:0/420568394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0fc005040 con 0x56245c6e1a00 2024-06-10T01:20:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.801+0000 7fc0f27fc640 1 -- 172.21.15.7:0/420568394 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc0fc040040 con 0x56245c6e1a00 2024-06-10T01:20:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.801+0000 7fc113d9c740 10 client.19032 fetch_fsmap learned FSMap version 44 2024-06-10T01:20:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.801+0000 7fc113d9c740 1 -- 172.21.15.7:0/420568394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56245c424620 con 0x56245c6e1a00 2024-06-10T01:20:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.801+0000 7fc0f27fc640 1 -- 172.21.15.7:0/420568394 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0fc005040 con 0x56245c6e1a00 2024-06-10T01:20:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.802+0000 7fc0f27fc640 1 -- 172.21.15.7:0/420568394 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fc0fc04d4c0 con 0x56245c6e1a00 2024-06-10T01:20:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.802+0000 7fc113d9c740 10 client.19032 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:20:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.802+0000 7fc113d9c740 10 client.19032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:20:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.802+0000 7fc113d9c740 20 client.19032 populate_metadata read hostname 'smithi007' 2024-06-10T01:20:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.802+0000 7fc113d9c740 1 --2- 172.21.15.7:0/420568394 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56245c6f6c10 0x56245c6f9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.802+0000 7fc113d9c740 4 client.19032 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:20:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.802+0000 7fc113d9c740 1 -- 172.21.15.7:0/420568394 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56245c657f10 con 0x56245c6f6c10 2024-06-10T01:20:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.802+0000 7fc103fff640 1 --2- 172.21.15.7:0/420568394 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56245c6f6c10 0x56245c6f9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.803+0000 7fc0f27fc640 10 client.19032 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.803+0000 7fc103fff640 1 --2- 172.21.15.7:0/420568394 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56245c6f6c10 0x56245c6f9000 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e50330025480a39f server_cookie=51403fa2dd24280f in_seq=0 out_seq=0 2024-06-10T01:20:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.803+0000 7fc0f27fc640 1 -- 172.21.15.7:0/420568394 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56245c657f10 con 0x56245c6f6c10 2024-06-10T01:20:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.803+0000 7fc0f27fc640 10 client.19032 handle_command_reply: tid=0 2024-06-10T01:20:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.803+0000 7fc113d9c740 1 client.19032 shutdown 2024-06-10T01:20:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.803+0000 7fc113d9c740 1 -- 172.21.15.7:0/420568394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245c6e1a00 msgr2=0x56245c6eb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.803+0000 7fc113d9c740 1 --2- 172.21.15.7:0/420568394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245c6e1a00 0x56245c6eb770 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc000c00 tx=0x7fc0fc005420 comp rx=0 tx=0).stop 2024-06-10T01:20:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.803+0000 7fc113d9c740 1 -- 172.21.15.7:0/420568394 shutdown_connections 2024-06-10T01:20:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.803+0000 7fc113d9c740 1 --2- 172.21.15.7:0/420568394 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56245c6f6c10 0x56245c6f9000 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.803+0000 7fc113d9c740 1 --2- 172.21.15.7:0/420568394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56245c6e6b70 0x56245c6f27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.803+0000 7fc113d9c740 1 --2- 172.21.15.7:0/420568394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245c6e1a00 0x56245c6eb770 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.803+0000 7fc0f27fc640 0 client.19032 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.803+0000 7fc113d9c740 1 --2- 172.21.15.7:0/420568394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56245c6e1010 0x56245c6eb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.803+0000 7fc113d9c740 1 -- 172.21.15.7:0/420568394 >> 172.21.15.7:0/420568394 conn(0x56245c6dc680 msgr2=0x56245c6db7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.804+0000 7fc113d9c740 1 -- 172.21.15.7:0/420568394 shutdown_connections 2024-06-10T01:20:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.804+0000 7fc113d9c740 1 -- 172.21.15.7:0/420568394 wait complete. 2024-06-10T01:20:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.804+0000 7fc113d9c740 20 client.19032 trim_cache size 0 max 16384 2024-06-10T01:20:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.819+0000 7fc113d9c740 1 Processor -- start 2024-06-10T01:20:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.819+0000 7fc113d9c740 1 -- start start 2024-06-10T01:20:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.819+0000 7fc113d9c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56245c6e1010 0x56245c6f3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.819+0000 7fc113d9c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245c6e1a00 0x56245c6f2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.819+0000 7fc113d9c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56245c6e6b70 0x56245c6d7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.819+0000 7fc113d9c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c1af230 con 0x56245c6e1a00 2024-06-10T01:20:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.819+0000 7fc113d9c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c1aca70 con 0x56245c6e1010 2024-06-10T01:20:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.819+0000 7fc113d9c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c1aa8c0 con 0x56245c6e6b70 2024-06-10T01:20:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.819+0000 7fc10ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245c6e1a00 0x56245c6f2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.819+0000 7fc10ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245c6e1a00 0x56245c6f2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48828/0 (socket says 172.21.15.7:48828) 2024-06-10T01:20:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.819+0000 7fc10ad9d640 1 -- 172.21.15.7:0/638585323 learned_addr learned my addr 172.21.15.7:0/638585323 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.819+0000 7fc103fff640 1 --2- 172.21.15.7:0/638585323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56245c6e1010 0x56245c6f3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.819+0000 7fc10ad9d640 1 -- 172.21.15.7:0/638585323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56245c6e6b70 msgr2=0x56245c6d7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.819+0000 7fc103fff640 1 --2- 172.21.15.7:0/638585323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56245c6e6b70 0x56245c6d7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.819+0000 7fc10ad9d640 1 --2- 172.21.15.7:0/638585323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56245c6e6b70 0x56245c6d7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.819+0000 7fc10ad9d640 1 -- 172.21.15.7:0/638585323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56245c6e1010 msgr2=0x56245c6f3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.819+0000 7fc10ad9d640 1 --2- 172.21.15.7:0/638585323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56245c6e1010 0x56245c6f3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.819+0000 7fc10ad9d640 1 -- 172.21.15.7:0/638585323 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56245c6f95c0 con 0x56245c6e1a00 2024-06-10T01:20:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.820+0000 7fc10ad9d640 1 --2- 172.21.15.7:0/638585323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245c6e1a00 0x56245c6f2770 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc04d890 tx=0x7fc0fc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c81b8d7bf4cc34 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.820+0000 7fc100ff9640 1 -- 172.21.15.7:0/638585323 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc05b030 con 0x56245c6e1a00 2024-06-10T01:20:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.820+0000 7fc100ff9640 1 -- 172.21.15.7:0/638585323 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0fc005490 con 0x56245c6e1a00 2024-06-10T01:20:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.820+0000 7fc100ff9640 1 -- 172.21.15.7:0/638585323 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc035070 con 0x56245c6e1a00 2024-06-10T01:20:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.820+0000 7fc113d9c740 1 -- 172.21.15.7:0/638585323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245c6e1a00 msgr2=0x56245c6f2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.820+0000 7fc113d9c740 1 --2- 172.21.15.7:0/638585323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245c6e1a00 0x56245c6f2770 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc04d890 tx=0x7fc0fc005fd0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.821+0000 7fc113d9c740 1 -- 172.21.15.7:0/638585323 shutdown_connections 2024-06-10T01:20:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.821+0000 7fc113d9c740 1 --2- 172.21.15.7:0/638585323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56245c6e6b70 0x56245c6d7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.821+0000 7fc113d9c740 1 --2- 172.21.15.7:0/638585323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245c6e1a00 0x56245c6f2770 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.821+0000 7fc113d9c740 1 --2- 172.21.15.7:0/638585323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56245c6e1010 0x56245c6f3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.821+0000 7fc113d9c740 1 -- 172.21.15.7:0/638585323 >> 172.21.15.7:0/638585323 conn(0x56245c6dc680 msgr2=0x56245c6f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.821+0000 7fc113d9c740 1 -- 172.21.15.7:0/638585323 shutdown_connections 2024-06-10T01:20:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.821+0000 7fc113d9c740 1 -- 172.21.15.7:0/638585323 wait complete. 2024-06-10T01:20:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.821+0000 7fc113d9c740 1 -- start start 2024-06-10T01:20:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.821+0000 7fc113d9c740 1 Processor -- start 2024-06-10T01:20:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.821+0000 7fc113d9c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56245c6e1010 0x56245c6efe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.821+0000 7fc113d9c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245c6e1a00 0x56245c6f0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.821+0000 7fc113d9c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56245c6e6b70 0x56245c6f1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.821+0000 7fc113d9c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c1aaf30 con 0x56245c6e1a00 2024-06-10T01:20:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.821+0000 7fc113d9c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c1ac3a0 con 0x56245c6e6b70 2024-06-10T01:20:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.821+0000 7fc113d9c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c1ab600 con 0x56245c6e1010 2024-06-10T01:20:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.821+0000 7fc113d9c740 10 client.0 _ll_register_callbacks cb 0x7fc0fc0316b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:20:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.821+0000 7fc10ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245c6e1a00 0x56245c6f0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.821+0000 7fc10ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245c6e1a00 0x56245c6f0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48830/0 (socket says 172.21.15.7:48830) 2024-06-10T01:20:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.821+0000 7fc10ad9d640 1 -- 172.21.15.7:0/334921240 learned_addr learned my addr 172.21.15.7:0/334921240 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.821+0000 7fc103fff640 1 --2- 172.21.15.7:0/334921240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56245c6e1010 0x56245c6efe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.821+0000 7fc103fff640 1 --2- 172.21.15.7:0/334921240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56245c6e6b70 0x56245c6f1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.821+0000 7fc10ad9d640 1 -- 172.21.15.7:0/334921240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56245c6e1010 msgr2=0x56245c6efe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.821+0000 7fc10ad9d640 1 --2- 172.21.15.7:0/334921240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56245c6e1010 0x56245c6efe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.821+0000 7fc10ad9d640 1 -- 172.21.15.7:0/334921240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56245c6e6b70 msgr2=0x56245c6f1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.821+0000 7fc10ad9d640 1 --2- 172.21.15.7:0/334921240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56245c6e6b70 0x56245c6f1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.822+0000 7fc10ad9d640 1 -- 172.21.15.7:0/334921240 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56245c6f1910 con 0x56245c6e1a00 2024-06-10T01:20:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.822+0000 7fc103fff640 1 --2- 172.21.15.7:0/334921240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56245c6e1010 0x56245c6efe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:20:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.822+0000 7fc10ad9d640 1 --2- 172.21.15.7:0/334921240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245c6e1a00 0x56245c6f0360 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc0314e0 tx=0x7fc0fc031a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4691f9bd192eea server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.822+0000 7fc0f27fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:20:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.822+0000 7fc0f27fc640 1 -- 172.21.15.7:0/334921240 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc063070 con 0x56245c6e1a00 2024-06-10T01:20:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.822+0000 7fc113d9c740 1 -- 172.21.15.7:0/334921240 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0fc005490 con 0x56245c6e1a00 2024-06-10T01:20:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.822+0000 7fc0f27fc640 1 -- 172.21.15.7:0/334921240 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc0fc004b50 con 0x56245c6e1a00 2024-06-10T01:20:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.822+0000 7fc0f27fc640 1 -- 172.21.15.7:0/334921240 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc0fc06f050 con 0x56245c6e1a00 2024-06-10T01:20:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.822+0000 7fc113d9c740 10 client.19038 fetch_fsmap learned FSMap version 44 2024-06-10T01:20:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.822+0000 7fc113d9c740 1 -- 172.21.15.7:0/334921240 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56245c761820 con 0x56245c6e1a00 2024-06-10T01:20:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.822+0000 7fc0f27fc640 1 -- 172.21.15.7:0/334921240 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0fc005490 con 0x56245c6e1a00 2024-06-10T01:20:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.822+0000 7fc0f27fc640 1 -- 172.21.15.7:0/334921240 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fc0fc03ca20 con 0x56245c6e1a00 2024-06-10T01:20:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.822+0000 7fc113d9c740 10 client.19038 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:20:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.822+0000 7fc113d9c740 10 client.19038 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:20:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.822+0000 7fc113d9c740 20 client.19038 populate_metadata read hostname 'smithi007' 2024-06-10T01:20:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.822+0000 7fc113d9c740 1 --2- 172.21.15.7:0/334921240 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56245c761f10 0x56245c764330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.822+0000 7fc113d9c740 4 client.19038 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:20:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.822+0000 7fc113d9c740 1 -- 172.21.15.7:0/334921240 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56245c651c20 con 0x56245c761f10 2024-06-10T01:20:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.823+0000 7fc103fff640 1 --2- 172.21.15.7:0/334921240 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56245c761f10 0x56245c764330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.823+0000 7fc103fff640 1 --2- 172.21.15.7:0/334921240 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56245c761f10 0x56245c764330 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5660442ebab921c9 server_cookie=a690962722a41451 in_seq=0 out_seq=0 2024-06-10T01:20:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.823+0000 7fc0f27fc640 10 client.19038 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.853+0000 7fc0f27fc640 1 -- 172.21.15.7:0/334921240 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56245c651c20 con 0x56245c761f10 2024-06-10T01:20:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.853+0000 7fc0f27fc640 10 client.19038 handle_command_reply: tid=0 2024-06-10T01:20:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.853+0000 7fc113d9c740 1 client.19038 shutdown 2024-06-10T01:20:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.853+0000 7fc113d9c740 1 -- 172.21.15.7:0/334921240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245c6e1a00 msgr2=0x56245c6f0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.853+0000 7fc113d9c740 1 --2- 172.21.15.7:0/334921240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245c6e1a00 0x56245c6f0360 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc0314e0 tx=0x7fc0fc031a40 comp rx=0 tx=0).stop 2024-06-10T01:20:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.853+0000 7fc113d9c740 1 -- 172.21.15.7:0/334921240 shutdown_connections 2024-06-10T01:20:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.853+0000 7fc113d9c740 1 --2- 172.21.15.7:0/334921240 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56245c761f10 0x56245c764330 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.853+0000 7fc113d9c740 1 --2- 172.21.15.7:0/334921240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56245c6e6b70 0x56245c6f1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.853+0000 7fc0f27fc640 0 client.19038 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.853+0000 7fc113d9c740 1 --2- 172.21.15.7:0/334921240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245c6e1a00 0x56245c6f0360 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.853+0000 7fc113d9c740 1 --2- 172.21.15.7:0/334921240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56245c6e1010 0x56245c6efe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.853+0000 7fc113d9c740 1 -- 172.21.15.7:0/334921240 >> 172.21.15.7:0/334921240 conn(0x56245c6dc680 msgr2=0x56245c6df420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.853+0000 7fc113d9c740 1 -- 172.21.15.7:0/334921240 shutdown_connections 2024-06-10T01:20:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.853+0000 7fc113d9c740 1 -- 172.21.15.7:0/334921240 wait complete. 2024-06-10T01:20:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.853+0000 7fc113d9c740 20 client.19038 trim_cache size 0 max 16384 2024-06-10T01:20:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:20:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:20:12.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:20:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.854+0000 7fc111095640 1 -- 172.21.15.7:0/1822261137 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc0f80a2b80 msgr2=0x7fc0f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.854+0000 7fc111095640 1 --2- 172.21.15.7:0/1822261137 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc0f80a2b80 0x7fc0f80a5040 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc10409b750 tx=0x7fc10404b040 comp rx=0 tx=0).stop 2024-06-10T01:20:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.854+0000 7fc111095640 1 -- 172.21.15.7:0/1822261137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc10c1024b0 msgr2=0x7fc10c109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.854+0000 7fc111095640 1 --2- 172.21.15.7:0/1822261137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc10c1024b0 0x7fc10c109e00 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fc10400eaa0 tx=0x7fc10400ef60 comp rx=0 tx=0).stop 2024-06-10T01:20:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.854+0000 7fc111095640 1 -- 172.21.15.7:0/1822261137 shutdown_connections 2024-06-10T01:20:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.854+0000 7fc111095640 1 --2- 172.21.15.7:0/1822261137 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc0f80a2b80 0x7fc0f80a5040 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.854+0000 7fc111095640 1 --2- 172.21.15.7:0/1822261137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc10c1024b0 0x7fc10c109e00 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.854+0000 7fc111095640 1 --2- 172.21.15.7:0/1822261137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc10c101af0 0x7fc10c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.854+0000 7fc111095640 1 --2- 172.21.15.7:0/1822261137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc10c100900 0x7fc10c10b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.854+0000 7fc111095640 1 -- 172.21.15.7:0/1822261137 >> 172.21.15.7:0/1822261137 conn(0x7fc10c0fbe70 msgr2=0x7fc10c104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.854+0000 7fc111095640 1 -- 172.21.15.7:0/1822261137 shutdown_connections 2024-06-10T01:20:12.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:12.854+0000 7fc111095640 1 -- 172.21.15.7:0/1822261137 wait complete. 2024-06-10T01:20:13.470 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12545 17.00 MB/sec warmup 50 sec latency 21547.283 ms 2024-06-10T01:20:13.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:13 smithi175 ceph-mon[36636]: pgmap v858: 129 pgs: 129 active+clean; 355 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 46 KiB/s wr, 92 op/s 2024-06-10T01:20:13.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:13 smithi007 ceph-mon[32398]: pgmap v858: 129 pgs: 129 active+clean; 355 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 46 KiB/s wr, 92 op/s 2024-06-10T01:20:13.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:13 smithi078 ceph-mon[36683]: pgmap v858: 129 pgs: 129 active+clean; 355 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 46 KiB/s wr, 92 op/s 2024-06-10T01:20:14.470 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12545 16.67 MB/sec warmup 51 sec latency 22547.372 ms 2024-06-10T01:20:15.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.125+0000 7f2861b75700 1 -- 172.21.15.7:0/1465722723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f285c110c70 msgr2=0x7f285c109490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:15.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.125+0000 7f2861b75700 1 --2- 172.21.15.7:0/1465722723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f285c110c70 0x7f285c109490 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f28540098f0 tx=0x7f2854009c00 comp rx=0 tx=0).stop 2024-06-10T01:20:15.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.128+0000 7f2861b75700 1 -- 172.21.15.7:0/1465722723 shutdown_connections 2024-06-10T01:20:15.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.128+0000 7f2861b75700 1 --2- 172.21.15.7:0/1465722723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f285c11bc70 0x7f285c11e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:15.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.128+0000 7f2861b75700 1 --2- 172.21.15.7:0/1465722723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f285c110c70 0x7f285c109490 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:15.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.128+0000 7f2861b75700 1 --2- 172.21.15.7:0/1465722723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f285c1099d0 0x7f285c109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:15.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.128+0000 7f2861b75700 1 -- 172.21.15.7:0/1465722723 >> 172.21.15.7:0/1465722723 conn(0x7f285c06d600 msgr2=0x7f285c06da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:15.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.132+0000 7f2861b75700 1 -- 172.21.15.7:0/1465722723 shutdown_connections 2024-06-10T01:20:15.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.132+0000 7f2861b75700 1 -- 172.21.15.7:0/1465722723 wait complete. 2024-06-10T01:20:15.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.133+0000 7f2861b75700 1 Processor -- start 2024-06-10T01:20:15.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.134+0000 7f2861b75700 1 -- start start 2024-06-10T01:20:15.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.134+0000 7f2861b75700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f285c1099d0 0x7f285c11b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:15.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.135+0000 7f2861b75700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f285c110c70 0x7f285c114290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:15.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.135+0000 7f2861b75700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f285c11bc70 0x7f285c1147d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:15.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.135+0000 7f285affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f285c110c70 0x7f285c114290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:15.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.135+0000 7f285affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f285c110c70 0x7f285c114290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:45856/0 (socket says 172.21.15.7:45856) 2024-06-10T01:20:15.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.135+0000 7f285affd700 1 -- 172.21.15.7:0/2253435975 learned_addr learned my addr 172.21.15.7:0/2253435975 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:15.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.135+0000 7f2861b75700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f285c1216f0 con 0x7f285c1099d0 2024-06-10T01:20:15.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.135+0000 7f2861b75700 1 -- 172.21.15.7:0/2253435975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f285c121570 con 0x7f285c11bc70 2024-06-10T01:20:15.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.135+0000 7f2861b75700 1 -- 172.21.15.7:0/2253435975 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f285c121870 con 0x7f285c110c70 2024-06-10T01:20:15.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.136+0000 7f285affd700 1 -- 172.21.15.7:0/2253435975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f285c11bc70 msgr2=0x7f285c1147d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:15.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.136+0000 7f285affd700 1 --2- 172.21.15.7:0/2253435975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f285c11bc70 0x7f285c1147d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:15.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.136+0000 7f285affd700 1 -- 172.21.15.7:0/2253435975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f285c1099d0 msgr2=0x7f285c11b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:15.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.136+0000 7f285affd700 1 --2- 172.21.15.7:0/2253435975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f285c1099d0 0x7f285c11b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:15.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.136+0000 7f285affd700 1 -- 172.21.15.7:0/2253435975 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f285c114dc0 con 0x7f285c110c70 2024-06-10T01:20:15.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.137+0000 7f285affd700 1 --2- 172.21.15.7:0/2253435975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f285c110c70 0x7f285c114290 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f284400d980 tx=0x7f284400dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:15.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.141+0000 7f2858ff9700 1 -- 172.21.15.7:0/2253435975 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2844017070 con 0x7f285c110c70 2024-06-10T01:20:15.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.141+0000 7f2858ff9700 1 -- 172.21.15.7:0/2253435975 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2844004d10 con 0x7f285c110c70 2024-06-10T01:20:15.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.141+0000 7f2858ff9700 1 -- 172.21.15.7:0/2253435975 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2844005550 con 0x7f285c110c70 2024-06-10T01:20:15.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.141+0000 7f2861b75700 1 -- 172.21.15.7:0/2253435975 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f285c1150b0 con 0x7f285c110c70 2024-06-10T01:20:15.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.141+0000 7f2861b75700 1 -- 172.21.15.7:0/2253435975 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f285c119050 con 0x7f285c110c70 2024-06-10T01:20:15.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.146+0000 7f2858ff9700 1 -- 172.21.15.7:0/2253435975 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f2844009b40 con 0x7f285c110c70 2024-06-10T01:20:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.147+0000 7f2858ff9700 1 --2- 172.21.15.7:0/2253435975 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f28480a2e20 0x7f28480a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.148+0000 7f2858ff9700 1 -- 172.21.15.7:0/2253435975 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f28440cf1c0 con 0x7f285c110c70 2024-06-10T01:20:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.148+0000 7f2861b75700 1 -- 172.21.15.7:0/2253435975 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f283c005330 con 0x7f285c110c70 2024-06-10T01:20:15.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.149+0000 7f285b7fe700 1 --2- 172.21.15.7:0/2253435975 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f28480a2e20 0x7f28480a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:15.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.156+0000 7f285b7fe700 1 --2- 172.21.15.7:0/2253435975 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f28480a2e20 0x7f28480a52e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f285400b2e0 tx=0x7f28540058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.158+0000 7f2858ff9700 1 -- 172.21.15.7:0/2253435975 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2844097030 con 0x7f285c110c70 2024-06-10T01:20:15.470 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12545 16.35 MB/sec warmup 52 sec latency 23547.424 ms 2024-06-10T01:20:15.544 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:20:15.545 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:20:15.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.580+0000 7f2861b75700 1 -- 172.21.15.7:0/2253435975 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f283c005620 con 0x7f285c110c70 2024-06-10T01:20:15.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.581+0000 7f2858ff9700 1 -- 172.21.15.7:0/2253435975 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11860 (secure 0 0 0) 0x7f2844020020 con 0x7f285c110c70 2024-06-10T01:20:15.574 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:20:15.575 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:19:47.239285+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:20:15.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.583+0000 7f2861b75700 1 -- 172.21.15.7:0/2253435975 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f28480a2e20 msgr2=0x7f28480a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:15.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.583+0000 7f2861b75700 1 --2- 172.21.15.7:0/2253435975 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f28480a2e20 0x7f28480a52e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f285400b2e0 tx=0x7f28540058e0 comp rx=0 tx=0).stop 2024-06-10T01:20:15.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.583+0000 7f2861b75700 1 -- 172.21.15.7:0/2253435975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f285c110c70 msgr2=0x7f285c114290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:15.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.583+0000 7f2861b75700 1 --2- 172.21.15.7:0/2253435975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f285c110c70 0x7f285c114290 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f284400d980 tx=0x7f284400dd40 comp rx=0 tx=0).stop 2024-06-10T01:20:15.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.583+0000 7f2861b75700 1 -- 172.21.15.7:0/2253435975 shutdown_connections 2024-06-10T01:20:15.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.583+0000 7f2861b75700 1 --2- 172.21.15.7:0/2253435975 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f28480a2e20 0x7f28480a52e0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:15.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.583+0000 7f2861b75700 1 --2- 172.21.15.7:0/2253435975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f285c11bc70 0x7f285c1147d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:15.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.583+0000 7f2861b75700 1 --2- 172.21.15.7:0/2253435975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f285c1099d0 0x7f285c11b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:15.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.583+0000 7f2861b75700 1 --2- 172.21.15.7:0/2253435975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f285c110c70 0x7f285c114290 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:15.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.583+0000 7f2861b75700 1 -- 172.21.15.7:0/2253435975 >> 172.21.15.7:0/2253435975 conn(0x7f285c06d600 msgr2=0x7f285c10a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:15.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.584+0000 7f2861b75700 1 -- 172.21.15.7:0/2253435975 shutdown_connections 2024-06-10T01:20:15.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:15.584+0000 7f2861b75700 1 -- 172.21.15.7:0/2253435975 wait complete. 2024-06-10T01:20:15.578 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 44 2024-06-10T01:20:15.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:15 smithi175 ceph-mon[36636]: pgmap v859: 129 pgs: 129 active+clean; 355 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 40 KiB/s wr, 69 op/s 2024-06-10T01:20:15.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:15 smithi007 ceph-mon[32398]: pgmap v859: 129 pgs: 129 active+clean; 355 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 40 KiB/s wr, 69 op/s 2024-06-10T01:20:15.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:15 smithi078 ceph-mon[36683]: pgmap v859: 129 pgs: 129 active+clean; 355 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 40 KiB/s wr, 69 op/s 2024-06-10T01:20:16.381 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db --set-id 1d08866a --release --await' 2024-06-10T01:20:16.381 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db --set-id 1d08866a --release --await 2024-06-10T01:20:16.439 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:20:16.471 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12545 16.04 MB/sec warmup 53 sec latency 24547.463 ms 2024-06-10T01:20:16.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:16 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2253435975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:20:16.706 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:16 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2253435975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:20:16.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:16 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2253435975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:20:17.470 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12545 15.74 MB/sec warmup 54 sec latency 25547.509 ms 2024-06-10T01:20:17.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:17 smithi175 ceph-mon[36636]: pgmap v860: 129 pgs: 129 active+clean; 358 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 370 KiB/s wr, 47 op/s 2024-06-10T01:20:17.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:17 smithi007 ceph-mon[32398]: pgmap v860: 129 pgs: 129 active+clean; 358 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 370 KiB/s wr, 47 op/s 2024-06-10T01:20:17.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:17 smithi078 ceph-mon[36683]: pgmap v860: 129 pgs: 129 active+clean; 358 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 370 KiB/s wr, 47 op/s 2024-06-10T01:20:18.470 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12545 15.46 MB/sec warmup 55 sec latency 26547.580 ms 2024-06-10T01:20:18.617 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:20:19.471 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12545 15.18 MB/sec warmup 56 sec latency 27547.673 ms 2024-06-10T01:20:19.680 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:19 smithi007 ceph-mon[32398]: pgmap v861: 129 pgs: 129 active+clean; 368 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.4 MiB/s wr, 48 op/s 2024-06-10T01:20:19.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:19 smithi175 ceph-mon[36636]: pgmap v861: 129 pgs: 129 active+clean; 368 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.4 MiB/s wr, 48 op/s 2024-06-10T01:20:19.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:19 smithi078 ceph-mon[36683]: pgmap v861: 129 pgs: 129 active+clean; 368 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.4 MiB/s wr, 48 op/s 2024-06-10T01:20:20.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.422+0000 7fd414ebd700 1 -- 172.21.15.7:0/308184188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd410107400 msgr2=0x7fd4101077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:20.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.422+0000 7fd414ebd700 1 --2- 172.21.15.7:0/308184188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd410107400 0x7fd4101077e0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fd400009990 tx=0x7fd400009ca0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.422+0000 7fd414ebd700 1 -- 172.21.15.7:0/308184188 shutdown_connections 2024-06-10T01:20:20.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.422+0000 7fd414ebd700 1 --2- 172.21.15.7:0/308184188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd410101d50 0x7fd41010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.422+0000 7fd414ebd700 1 --2- 172.21.15.7:0/308184188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd410107400 0x7fd4101077e0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.422+0000 7fd414ebd700 1 --2- 172.21.15.7:0/308184188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd410101390 0x7fd410101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.422+0000 7fd414ebd700 1 -- 172.21.15.7:0/308184188 >> 172.21.15.7:0/308184188 conn(0x7fd410078340 msgr2=0x7fd410078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:20.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.422+0000 7fd414ebd700 1 -- 172.21.15.7:0/308184188 shutdown_connections 2024-06-10T01:20:20.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.422+0000 7fd414ebd700 1 -- 172.21.15.7:0/308184188 wait complete. 2024-06-10T01:20:20.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.423+0000 7fd414ebd700 1 Processor -- start 2024-06-10T01:20:20.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.423+0000 7fd414ebd700 1 -- start start 2024-06-10T01:20:20.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.423+0000 7fd414ebd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd410101390 0x7fd410103300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:20.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.423+0000 7fd414ebd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd410101d50 0x7fd410103840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:20.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.423+0000 7fd414ebd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd410107400 0x7fd410103d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:20.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.423+0000 7fd414ebd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd4101116a0 con 0x7fd410101d50 2024-06-10T01:20:20.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.423+0000 7fd414ebd700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd410111520 con 0x7fd410101390 2024-06-10T01:20:20.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.423+0000 7fd414ebd700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd410111820 con 0x7fd410107400 2024-06-10T01:20:20.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.423+0000 7fd40dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd410101d50 0x7fd410103840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:20.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.423+0000 7fd40dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd410101d50 0x7fd410103840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48876/0 (socket says 172.21.15.7:48876) 2024-06-10T01:20:20.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.423+0000 7fd40dd9b700 1 -- 172.21.15.7:0/3283284544 learned_addr learned my addr 172.21.15.7:0/3283284544 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:20.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.423+0000 7fd40e59c700 1 --2- 172.21.15.7:0/3283284544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd410101390 0x7fd410103300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:20.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.424+0000 7fd40ed9d700 1 --2- 172.21.15.7:0/3283284544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd410107400 0x7fd410103d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:20.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.424+0000 7fd40dd9b700 1 -- 172.21.15.7:0/3283284544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd410107400 msgr2=0x7fd410103d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:20.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.424+0000 7fd40dd9b700 1 --2- 172.21.15.7:0/3283284544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd410107400 0x7fd410103d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.424+0000 7fd40dd9b700 1 -- 172.21.15.7:0/3283284544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd410101390 msgr2=0x7fd410103300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:20.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.424+0000 7fd40dd9b700 1 --2- 172.21.15.7:0/3283284544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd410101390 0x7fd410103300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.424+0000 7fd40dd9b700 1 -- 172.21.15.7:0/3283284544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4101a7150 con 0x7fd410101d50 2024-06-10T01:20:20.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.424+0000 7fd40e59c700 1 --2- 172.21.15.7:0/3283284544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd410101390 0x7fd410103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:20:20.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.424+0000 7fd40dd9b700 1 --2- 172.21.15.7:0/3283284544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd410101d50 0x7fd410103840 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fd40400c970 tx=0x7fd40400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:20.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.424+0000 7fd3ff7fe700 1 -- 172.21.15.7:0/3283284544 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4040078d0 con 0x7fd410101d50 2024-06-10T01:20:20.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.424+0000 7fd414ebd700 1 -- 172.21.15.7:0/3283284544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4101a7440 con 0x7fd410101d50 2024-06-10T01:20:20.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.424+0000 7fd3ff7fe700 1 -- 172.21.15.7:0/3283284544 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd404018480 con 0x7fd410101d50 2024-06-10T01:20:20.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.424+0000 7fd3ff7fe700 1 -- 172.21.15.7:0/3283284544 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40400f6b0 con 0x7fd410101d50 2024-06-10T01:20:20.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.424+0000 7fd414ebd700 1 -- 172.21.15.7:0/3283284544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4101a7980 con 0x7fd410101d50 2024-06-10T01:20:20.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.428+0000 7fd3ff7fe700 1 -- 172.21.15.7:0/3283284544 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fd404018630 con 0x7fd410101d50 2024-06-10T01:20:20.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.428+0000 7fd3ff7fe700 1 --2- 172.21.15.7:0/3283284544 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd3f80a2c20 0x7fd3f80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:20.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.428+0000 7fd3ff7fe700 1 -- 172.21.15.7:0/3283284544 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fd4040cf4e0 con 0x7fd410101d50 2024-06-10T01:20:20.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.428+0000 7fd40e59c700 1 --2- 172.21.15.7:0/3283284544 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd3f80a2c20 0x7fd3f80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:20.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.429+0000 7fd41f740b80 1 Processor -- start 2024-06-10T01:20:20.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.429+0000 7fd40e59c700 1 --2- 172.21.15.7:0/3283284544 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd3f80a2c20 0x7fd3f80a50e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd40000b5c0 tx=0x7fd400005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:20.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.429+0000 7fd41f740b80 1 -- start start 2024-06-10T01:20:20.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.429+0000 7fd41f740b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567dd0c26c0 0x5567dd0c2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:20.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.429+0000 7fd41f740b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567dd0c2fe0 0x5567dd0bca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:20.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.429+0000 7fd41f740b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567dd0bcfa0 0x5567dd0bd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:20.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.429+0000 7fd41f740b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5567dd00e890 con 0x5567dd0bcfa0 2024-06-10T01:20:20.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.429+0000 7fd41f740b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5567dcfb67a0 con 0x5567dd0c26c0 2024-06-10T01:20:20.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.429+0000 7fd41f740b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5567dd00a850 con 0x5567dd0c2fe0 2024-06-10T01:20:20.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.430+0000 7fd40e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567dd0c26c0 0x5567dd0c2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:20.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.430+0000 7fd40e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567dd0c26c0 0x5567dd0c2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:43970/0 (socket says 172.21.15.7:43970) 2024-06-10T01:20:20.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.430+0000 7fd40e59c700 1 -- 172.21.15.7:0/2920936108 learned_addr learned my addr 172.21.15.7:0/2920936108 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:20.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.430+0000 7fd40dd9b700 1 --2- 172.21.15.7:0/2920936108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567dd0c2fe0 0x5567dd0bca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:20.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.430+0000 7fd40ed9d700 1 --2- 172.21.15.7:0/2920936108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567dd0bcfa0 0x5567dd0bd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:20.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.430+0000 7fd40dd9b700 1 -- 172.21.15.7:0/2920936108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567dd0c26c0 msgr2=0x5567dd0c2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:20.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.430+0000 7fd40dd9b700 1 --2- 172.21.15.7:0/2920936108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567dd0c26c0 0x5567dd0c2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.430+0000 7fd40dd9b700 1 -- 172.21.15.7:0/2920936108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567dd0bcfa0 msgr2=0x5567dd0bd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:20.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.430+0000 7fd40dd9b700 1 --2- 172.21.15.7:0/2920936108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567dd0bcfa0 0x5567dd0bd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.430+0000 7fd40dd9b700 1 -- 172.21.15.7:0/2920936108 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567dcfacfd0 con 0x5567dd0c2fe0 2024-06-10T01:20:20.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.430+0000 7fd40dd9b700 1 --2- 172.21.15.7:0/2920936108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567dd0c2fe0 0x5567dd0bca60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd40400c940 tx=0x7fd404097e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e13ee7874cacbb34 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:20.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.430+0000 7fd3fcff9700 1 -- 172.21.15.7:0/2920936108 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40409fe00 con 0x5567dd0c2fe0 2024-06-10T01:20:20.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.430+0000 7fd3fcff9700 1 -- 172.21.15.7:0/2920936108 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4040cf070 con 0x5567dd0c2fe0 2024-06-10T01:20:20.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.430+0000 7fd3fcff9700 1 -- 172.21.15.7:0/2920936108 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4040a43c0 con 0x5567dd0c2fe0 2024-06-10T01:20:20.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.431+0000 7fd41f740b80 1 -- 172.21.15.7:0/2920936108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567dd0c2fe0 msgr2=0x5567dd0bca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:20.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.431+0000 7fd41f740b80 1 --2- 172.21.15.7:0/2920936108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567dd0c2fe0 0x5567dd0bca60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd40400c940 tx=0x7fd404097e20 comp rx=0 tx=0).stop 2024-06-10T01:20:20.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.431+0000 7fd41f740b80 1 -- 172.21.15.7:0/2920936108 shutdown_connections 2024-06-10T01:20:20.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.431+0000 7fd41f740b80 1 --2- 172.21.15.7:0/2920936108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567dd0bcfa0 0x5567dd0bd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.431+0000 7fd41f740b80 1 --2- 172.21.15.7:0/2920936108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567dd0c26c0 0x5567dd0c2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.431+0000 7fd41f740b80 1 --2- 172.21.15.7:0/2920936108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567dd0c2fe0 0x5567dd0bca60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.431+0000 7fd41f740b80 1 -- 172.21.15.7:0/2920936108 >> 172.21.15.7:0/2920936108 conn(0x5567dcfe7d40 msgr2=0x5567dd0b78d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:20.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.431+0000 7fd41f740b80 1 -- 172.21.15.7:0/2920936108 shutdown_connections 2024-06-10T01:20:20.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.431+0000 7fd41f740b80 1 -- 172.21.15.7:0/2920936108 wait complete. 2024-06-10T01:20:20.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.432+0000 7fd41f740b80 1 -- start start 2024-06-10T01:20:20.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.432+0000 7fd41f740b80 1 Processor -- start 2024-06-10T01:20:20.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.432+0000 7fd41f740b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567dd0bcfa0 0x5567dd0c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:20.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.432+0000 7fd41f740b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567dd0c26c0 0x5567dd0c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:20.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.432+0000 7fd41f740b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567dd0c2fe0 0x5567dd0c6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:20.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.432+0000 7fd41f740b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5567dd0c9a50 con 0x5567dd0c26c0 2024-06-10T01:20:20.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.432+0000 7fd41f740b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5567dcfaec70 con 0x5567dd0c2fe0 2024-06-10T01:20:20.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.432+0000 7fd41f740b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5567dd0c9bd0 con 0x5567dd0bcfa0 2024-06-10T01:20:20.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.432+0000 7fd41f740b80 10 client.0 _ll_register_callbacks cb 0x5567dcfa89a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:20:20.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.432+0000 7fd40dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567dd0c26c0 0x5567dd0c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:20.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.432+0000 7fd40dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567dd0c26c0 0x5567dd0c60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48904/0 (socket says 172.21.15.7:48904) 2024-06-10T01:20:20.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.432+0000 7fd40dd9b700 1 -- 172.21.15.7:0/683569490 learned_addr learned my addr 172.21.15.7:0/683569490 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:20.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.432+0000 7fd40e59c700 1 --2- 172.21.15.7:0/683569490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567dd0bcfa0 0x5567dd0c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:20.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.432+0000 7fd40ed9d700 1 --2- 172.21.15.7:0/683569490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567dd0c2fe0 0x5567dd0c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:20.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.433+0000 7fd40dd9b700 1 -- 172.21.15.7:0/683569490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567dd0bcfa0 msgr2=0x5567dd0c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:20.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.433+0000 7fd40dd9b700 1 --2- 172.21.15.7:0/683569490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567dd0bcfa0 0x5567dd0c8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.433+0000 7fd40dd9b700 1 -- 172.21.15.7:0/683569490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567dd0c2fe0 msgr2=0x5567dd0c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:20.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.433+0000 7fd40dd9b700 1 --2- 172.21.15.7:0/683569490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567dd0c2fe0 0x5567dd0c6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.433+0000 7fd40dd9b700 1 -- 172.21.15.7:0/683569490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567dcf5d950 con 0x5567dd0c26c0 2024-06-10T01:20:20.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.433+0000 7fd40dd9b700 1 --2- 172.21.15.7:0/683569490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567dd0c26c0 0x5567dd0c60d0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fd40409c0b0 tx=0x7fd40409bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f257d97301c3c7d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:20.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.433+0000 7fd3f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:20:20.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.433+0000 7fd3f5ffb700 1 -- 172.21.15.7:0/683569490 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4040b3070 con 0x5567dd0c26c0 2024-06-10T01:20:20.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.433+0000 7fd3f5ffb700 1 -- 172.21.15.7:0/683569490 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fd4040189a0 con 0x5567dd0c26c0 2024-06-10T01:20:20.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.433+0000 7fd41f740b80 1 -- 172.21.15.7:0/683569490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4040cf070 con 0x5567dd0c26c0 2024-06-10T01:20:20.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.433+0000 7fd3f5ffb700 1 -- 172.21.15.7:0/683569490 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd4040a4630 con 0x5567dd0c26c0 2024-06-10T01:20:20.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.433+0000 7fd41f740b80 10 client.19053 fetch_fsmap learned FSMap version 44 2024-06-10T01:20:20.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.433+0000 7fd41f740b80 1 -- 172.21.15.7:0/683569490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5567dd0d4380 con 0x5567dd0c26c0 2024-06-10T01:20:20.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.434+0000 7fd3f5ffb700 1 -- 172.21.15.7:0/683569490 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4040cf070 con 0x5567dd0c26c0 2024-06-10T01:20:20.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.434+0000 7fd3f5ffb700 1 -- 172.21.15.7:0/683569490 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fd40409aea0 con 0x5567dd0c26c0 2024-06-10T01:20:20.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.434+0000 7fd41f740b80 10 client.19053 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:20:20.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.434+0000 7fd41f740b80 10 client.19053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:20:20.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.434+0000 7fd41f740b80 20 client.19053 populate_metadata read hostname 'smithi007' 2024-06-10T01:20:20.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.435+0000 7fd41f740b80 1 --2- 172.21.15.7:0/683569490 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5567dd0d6850 0x5567dd0d8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:20.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.435+0000 7fd41f740b80 4 client.19053 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:20:20.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.435+0000 7fd41f740b80 1 -- 172.21.15.7:0/683569490 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567dd01bd90 con 0x5567dd0d6850 2024-06-10T01:20:20.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.435+0000 7fd40e59c700 1 --2- 172.21.15.7:0/683569490 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5567dd0d6850 0x5567dd0d8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:20.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.435+0000 7fd3f5ffb700 10 client.19053 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:20.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.435+0000 7fd40e59c700 1 --2- 172.21.15.7:0/683569490 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5567dd0d6850 0x5567dd0d8c40 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a0bdf751aca2206 server_cookie=8c0ca2b2f2eafbf1 in_seq=0 out_seq=0 2024-06-10T01:20:20.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.436+0000 7fd3f5ffb700 1 -- 172.21.15.7:0/683569490 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5567dd01bd90 con 0x5567dd0d6850 2024-06-10T01:20:20.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.436+0000 7fd3f5ffb700 10 client.19053 handle_command_reply: tid=0 2024-06-10T01:20:20.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.436+0000 7fd41f740b80 1 client.19053 shutdown 2024-06-10T01:20:20.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.436+0000 7fd41f740b80 1 -- 172.21.15.7:0/683569490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567dd0c26c0 msgr2=0x5567dd0c60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:20.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.436+0000 7fd41f740b80 1 --2- 172.21.15.7:0/683569490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567dd0c26c0 0x5567dd0c60d0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fd40409c0b0 tx=0x7fd40409bb80 comp rx=0 tx=0).stop 2024-06-10T01:20:20.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.436+0000 7fd41f740b80 1 -- 172.21.15.7:0/683569490 shutdown_connections 2024-06-10T01:20:20.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.436+0000 7fd41f740b80 1 --2- 172.21.15.7:0/683569490 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5567dd0d6850 0x5567dd0d8c40 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.436+0000 7fd3f5ffb700 0 client.19053 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:20.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.436+0000 7fd41f740b80 1 --2- 172.21.15.7:0/683569490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567dd0c2fe0 0x5567dd0c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.436+0000 7fd41f740b80 1 --2- 172.21.15.7:0/683569490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567dd0bcfa0 0x5567dd0c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.436+0000 7fd41f740b80 1 --2- 172.21.15.7:0/683569490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567dd0c26c0 0x5567dd0c60d0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.436+0000 7fd41f740b80 1 -- 172.21.15.7:0/683569490 >> 172.21.15.7:0/683569490 conn(0x5567dcfe7d40 msgr2=0x5567dd0b7670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:20.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.437+0000 7fd41f740b80 1 -- 172.21.15.7:0/683569490 shutdown_connections 2024-06-10T01:20:20.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.437+0000 7fd41f740b80 1 -- 172.21.15.7:0/683569490 wait complete. 2024-06-10T01:20:20.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.437+0000 7fd41f740b80 20 client.19053 trim_cache size 0 max 16384 2024-06-10T01:20:20.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.464+0000 7fd41f740b80 1 Processor -- start 2024-06-10T01:20:20.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.464+0000 7fd41f740b80 1 -- start start 2024-06-10T01:20:20.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.465+0000 7fd41f740b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567dd0bcfa0 0x5567dd0bca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:20.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.465+0000 7fd41f740b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567dd0c26c0 0x5567dd0c5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:20.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.465+0000 7fd41f740b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567dd0c2fe0 0x5567dd0c3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:20.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.465+0000 7fd41f740b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd4101b2d80 con 0x5567dd0c26c0 2024-06-10T01:20:20.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.465+0000 7fd41f740b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd4101b0580 con 0x5567dd0c2fe0 2024-06-10T01:20:20.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.465+0000 7fd41f740b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd4101ae3d0 con 0x5567dd0bcfa0 2024-06-10T01:20:20.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.465+0000 7fd40dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567dd0c26c0 0x5567dd0c5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:20.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.465+0000 7fd40dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567dd0c26c0 0x5567dd0c5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48916/0 (socket says 172.21.15.7:48916) 2024-06-10T01:20:20.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.465+0000 7fd40dd9b700 1 -- 172.21.15.7:0/4104432585 learned_addr learned my addr 172.21.15.7:0/4104432585 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:20.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.465+0000 7fd40e59c700 1 --2- 172.21.15.7:0/4104432585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567dd0bcfa0 0x5567dd0bca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:20.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.465+0000 7fd40ed9d700 1 --2- 172.21.15.7:0/4104432585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567dd0c2fe0 0x5567dd0c3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:20.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.465+0000 7fd40dd9b700 1 -- 172.21.15.7:0/4104432585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567dd0bcfa0 msgr2=0x5567dd0bca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:20.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.465+0000 7fd40dd9b700 1 --2- 172.21.15.7:0/4104432585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567dd0bcfa0 0x5567dd0bca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.465+0000 7fd40dd9b700 1 -- 172.21.15.7:0/4104432585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567dd0c2fe0 msgr2=0x5567dd0c3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:20.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.465+0000 7fd40dd9b700 1 --2- 172.21.15.7:0/4104432585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567dd0c2fe0 0x5567dd0c3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.465+0000 7fd40dd9b700 1 -- 172.21.15.7:0/4104432585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567dd0c2de0 con 0x5567dd0c26c0 2024-06-10T01:20:20.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.465+0000 7fd40ed9d700 1 --2- 172.21.15.7:0/4104432585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567dd0c2fe0 0x5567dd0c3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:20:20.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.465+0000 7fd40dd9b700 1 --2- 172.21.15.7:0/4104432585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567dd0c26c0 0x5567dd0c5030 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fd40400a5c0 tx=0x7fd40409bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a483f4ab945b3204 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:20.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.466+0000 7fd3f7fff700 1 -- 172.21.15.7:0/4104432585 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4040cf1f0 con 0x5567dd0c26c0 2024-06-10T01:20:20.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.466+0000 7fd3f7fff700 1 -- 172.21.15.7:0/4104432585 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4040cf040 con 0x5567dd0c26c0 2024-06-10T01:20:20.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.466+0000 7fd3f7fff700 1 -- 172.21.15.7:0/4104432585 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4040977a0 con 0x5567dd0c26c0 2024-06-10T01:20:20.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.466+0000 7fd41f740b80 1 -- 172.21.15.7:0/4104432585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567dd0c26c0 msgr2=0x5567dd0c5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:20.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.466+0000 7fd41f740b80 1 --2- 172.21.15.7:0/4104432585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567dd0c26c0 0x5567dd0c5030 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fd40400a5c0 tx=0x7fd40409bb10 comp rx=0 tx=0).stop 2024-06-10T01:20:20.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.466+0000 7fd41f740b80 1 -- 172.21.15.7:0/4104432585 shutdown_connections 2024-06-10T01:20:20.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.466+0000 7fd41f740b80 1 --2- 172.21.15.7:0/4104432585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567dd0c2fe0 0x5567dd0c3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.466+0000 7fd41f740b80 1 --2- 172.21.15.7:0/4104432585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567dd0bcfa0 0x5567dd0bca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.466+0000 7fd41f740b80 1 --2- 172.21.15.7:0/4104432585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567dd0c26c0 0x5567dd0c5030 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.466+0000 7fd41f740b80 1 -- 172.21.15.7:0/4104432585 >> 172.21.15.7:0/4104432585 conn(0x5567dcfe7d40 msgr2=0x5567dd0b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:20.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.466+0000 7fd41f740b80 1 -- 172.21.15.7:0/4104432585 shutdown_connections 2024-06-10T01:20:20.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.466+0000 7fd41f740b80 1 -- 172.21.15.7:0/4104432585 wait complete. 2024-06-10T01:20:20.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.467+0000 7fd41f740b80 1 -- start start 2024-06-10T01:20:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.467+0000 7fd41f740b80 1 Processor -- start 2024-06-10T01:20:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.467+0000 7fd41f740b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567dd0bcfa0 0x5567dd0dbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.467+0000 7fd41f740b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567dd0c26c0 0x5567dd0dc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.467+0000 7fd41f740b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567dd0c2fe0 0x5567dd0da220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.467+0000 7fd41f740b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd4101aea40 con 0x5567dd0c2fe0 2024-06-10T01:20:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.467+0000 7fd41f740b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd4101afeb0 con 0x5567dd0bcfa0 2024-06-10T01:20:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.467+0000 7fd41f740b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd4101af110 con 0x5567dd0c26c0 2024-06-10T01:20:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.467+0000 7fd41f740b80 10 client.0 _ll_register_callbacks cb 0x5567dcfa89a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:20:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.467+0000 7fd40ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567dd0c2fe0 0x5567dd0da220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.467+0000 7fd40ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567dd0c2fe0 0x5567dd0da220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48922/0 (socket says 172.21.15.7:48922) 2024-06-10T01:20:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.467+0000 7fd40ed9d700 1 -- 172.21.15.7:0/995208066 learned_addr learned my addr 172.21.15.7:0/995208066 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.467+0000 7fd40dd9b700 1 --2- 172.21.15.7:0/995208066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567dd0c26c0 0x5567dd0dc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.468+0000 7fd40e59c700 1 --2- 172.21.15.7:0/995208066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567dd0bcfa0 0x5567dd0dbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.468+0000 7fd40dd9b700 1 -- 172.21.15.7:0/995208066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567dd0bcfa0 msgr2=0x5567dd0dbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.468+0000 7fd40dd9b700 1 --2- 172.21.15.7:0/995208066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567dd0bcfa0 0x5567dd0dbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.468+0000 7fd40dd9b700 1 -- 172.21.15.7:0/995208066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567dd0c2fe0 msgr2=0x5567dd0da220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.468+0000 7fd40dd9b700 1 --2- 172.21.15.7:0/995208066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567dd0c2fe0 0x5567dd0da220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.468+0000 7fd40dd9b700 1 -- 172.21.15.7:0/995208066 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567dd0da8f0 con 0x5567dd0c26c0 2024-06-10T01:20:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.468+0000 7fd40ed9d700 1 --2- 172.21.15.7:0/995208066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567dd0c2fe0 0x5567dd0da220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:20:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.468+0000 7fd40dd9b700 1 --2- 172.21.15.7:0/995208066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567dd0c26c0 0x5567dd0dc220 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x5567dd00a930 tx=0x7fd40409b460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f79748fc7c181e97 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.468+0000 7fd3f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:20:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.468+0000 7fd3f5ffb700 1 -- 172.21.15.7:0/995208066 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4040b3030 con 0x5567dd0c26c0 2024-06-10T01:20:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.468+0000 7fd41f740b80 1 -- 172.21.15.7:0/995208066 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4040cf040 con 0x5567dd0c26c0 2024-06-10T01:20:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.468+0000 7fd3f5ffb700 1 -- 172.21.15.7:0/995208066 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fd40409aea0 con 0x5567dd0c26c0 2024-06-10T01:20:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.468+0000 7fd3f5ffb700 1 -- 172.21.15.7:0/995208066 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd40409b0a0 con 0x5567dd0c26c0 2024-06-10T01:20:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.468+0000 7fd41f740b80 10 client.28360 fetch_fsmap learned FSMap version 44 2024-06-10T01:20:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.468+0000 7fd41f740b80 1 -- 172.21.15.7:0/995208066 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5567dd0da8f0 con 0x5567dd0c26c0 2024-06-10T01:20:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.469+0000 7fd3f5ffb700 1 -- 172.21.15.7:0/995208066 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4040cf040 con 0x5567dd0c26c0 2024-06-10T01:20:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.469+0000 7fd3f5ffb700 1 -- 172.21.15.7:0/995208066 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fd40409a910 con 0x5567dd0c26c0 2024-06-10T01:20:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.469+0000 7fd41f740b80 10 client.28360 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:20:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.469+0000 7fd41f740b80 10 client.28360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:20:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.469+0000 7fd41f740b80 20 client.28360 populate_metadata read hostname 'smithi007' 2024-06-10T01:20:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.469+0000 7fd41f740b80 1 --2- 172.21.15.7:0/995208066 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5567dd0d6850 0x5567dd143ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.469+0000 7fd41f740b80 4 client.28360 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:20:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.469+0000 7fd41f740b80 1 -- 172.21.15.7:0/995208066 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5567dd016310 con 0x5567dd0d6850 2024-06-10T01:20:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.469+0000 7fd40e59c700 1 --2- 172.21.15.7:0/995208066 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5567dd0d6850 0x5567dd143ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:20.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.469+0000 7fd40e59c700 1 --2- 172.21.15.7:0/995208066 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5567dd0d6850 0x5567dd143ee0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d024793f5499f0d9 server_cookie=6811b16222e41f52 in_seq=0 out_seq=0 2024-06-10T01:20:20.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.469+0000 7fd3f5ffb700 10 client.28360 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:20.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.470+0000 7fd3f5ffb700 1 -- 172.21.15.7:0/995208066 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5567dd016310 con 0x5567dd0d6850 2024-06-10T01:20:20.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.470+0000 7fd3f5ffb700 10 client.28360 handle_command_reply: tid=0 2024-06-10T01:20:20.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.470+0000 7fd41f740b80 1 client.28360 shutdown 2024-06-10T01:20:20.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.471+0000 7fd41f740b80 1 -- 172.21.15.7:0/995208066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567dd0c26c0 msgr2=0x5567dd0dc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:20.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.471+0000 7fd41f740b80 1 --2- 172.21.15.7:0/995208066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567dd0c26c0 0x5567dd0dc220 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x5567dd00a930 tx=0x7fd40409b460 comp rx=0 tx=0).stop 2024-06-10T01:20:20.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.471+0000 7fd41f740b80 1 -- 172.21.15.7:0/995208066 shutdown_connections 2024-06-10T01:20:20.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.471+0000 7fd41f740b80 1 --2- 172.21.15.7:0/995208066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5567dd0c2fe0 0x5567dd0da220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.471+0000 7fd41f740b80 1 --2- 172.21.15.7:0/995208066 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5567dd0d6850 0x5567dd143ee0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.471+0000 7fd41f740b80 1 --2- 172.21.15.7:0/995208066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567dd0bcfa0 0x5567dd0dbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.471+0000 7fd41f740b80 1 --2- 172.21.15.7:0/995208066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567dd0c26c0 0x5567dd0dc220 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.471+0000 7fd3f5ffb700 0 client.28360 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:20.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.471+0000 7fd41f740b80 1 -- 172.21.15.7:0/995208066 >> 172.21.15.7:0/995208066 conn(0x5567dcfe7d40 msgr2=0x5567dd0b1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:20.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.471+0000 7fd41f740b80 1 -- 172.21.15.7:0/995208066 shutdown_connections 2024-06-10T01:20:20.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.471+0000 7fd41f740b80 1 -- 172.21.15.7:0/995208066 wait complete. 2024-06-10T01:20:20.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.471+0000 7fd41f740b80 20 client.28360 trim_cache size 0 max 16384 2024-06-10T01:20:20.471 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:20:20.471 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-10T01:20:20.472 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "8217e9d5-cd73-478c-ad98-1bbf6eada10c", 2024-06-10T01:20:20.472 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-10T01:20:20.472 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:20:20.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.472+0000 7fd414ebd700 1 -- 172.21.15.7:0/3283284544 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd3f80a2c20 msgr2=0x7fd3f80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:20.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.472+0000 7fd414ebd700 1 --2- 172.21.15.7:0/3283284544 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd3f80a2c20 0x7fd3f80a50e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd40000b5c0 tx=0x7fd400005960 comp rx=0 tx=0).stop 2024-06-10T01:20:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.472+0000 7fd414ebd700 1 -- 172.21.15.7:0/3283284544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd410101d50 msgr2=0x7fd410103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.472+0000 7fd414ebd700 1 --2- 172.21.15.7:0/3283284544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd410101d50 0x7fd410103840 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fd40400c970 tx=0x7fd40400cc80 comp rx=0 tx=0).stop 2024-06-10T01:20:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.472+0000 7fd414ebd700 1 -- 172.21.15.7:0/3283284544 shutdown_connections 2024-06-10T01:20:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.472+0000 7fd414ebd700 1 --2- 172.21.15.7:0/3283284544 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd3f80a2c20 0x7fd3f80a50e0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.472+0000 7fd414ebd700 1 --2- 172.21.15.7:0/3283284544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd410107400 0x7fd410103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.472+0000 7fd414ebd700 1 --2- 172.21.15.7:0/3283284544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd410101390 0x7fd410103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.472+0000 7fd414ebd700 1 --2- 172.21.15.7:0/3283284544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd410101d50 0x7fd410103840 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.472+0000 7fd414ebd700 1 -- 172.21.15.7:0/3283284544 >> 172.21.15.7:0/3283284544 conn(0x7fd410078340 msgr2=0x7fd410105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.472+0000 7fd414ebd700 1 -- 172.21.15.7:0/3283284544 shutdown_connections 2024-06-10T01:20:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:20.472+0000 7fd414ebd700 1 -- 172.21.15.7:0/3283284544 wait complete. 2024-06-10T01:20:20.473 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12545 14.91 MB/sec warmup 57 sec latency 28547.718 ms 2024-06-10T01:20:21.466 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:20:21.470 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12545 14.66 MB/sec warmup 58 sec latency 29547.764 ms 2024-06-10T01:20:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.675+0000 7f680d945700 1 -- 172.21.15.7:0/2631000008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f680806ab30 msgr2=0x7f680810f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.675+0000 7f680d945700 1 --2- 172.21.15.7:0/2631000008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f680806ab30 0x7f680810f320 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f680000b0a0 tx=0x7f680000b3b0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.675+0000 7f680d945700 1 -- 172.21.15.7:0/2631000008 shutdown_connections 2024-06-10T01:20:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.675+0000 7f680d945700 1 --2- 172.21.15.7:0/2631000008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f680806ab30 0x7f680810f320 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.675+0000 7f680d945700 1 --2- 172.21.15.7:0/2631000008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6808108500 0x7f68081088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.675+0000 7f680d945700 1 --2- 172.21.15.7:0/2631000008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f680806a170 0x7f680806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.675+0000 7f680d945700 1 -- 172.21.15.7:0/2631000008 >> 172.21.15.7:0/2631000008 conn(0x7f6808078340 msgr2=0x7f6808078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.675+0000 7f680d945700 1 -- 172.21.15.7:0/2631000008 shutdown_connections 2024-06-10T01:20:21.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.675+0000 7f680d945700 1 -- 172.21.15.7:0/2631000008 wait complete. 2024-06-10T01:20:21.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.676+0000 7f680d945700 1 Processor -- start 2024-06-10T01:20:21.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.676+0000 7f680d945700 1 -- start start 2024-06-10T01:20:21.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.676+0000 7f680d945700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f680806a170 0x7f6808198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:21.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.676+0000 7f680d945700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f680806ab30 0x7f6808199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:21.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.676+0000 7f680d945700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6808108500 0x7f680819d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:21.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.676+0000 7f680d945700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6808112760 con 0x7f6808108500 2024-06-10T01:20:21.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.676+0000 7f680d945700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f68081125e0 con 0x7f680806a170 2024-06-10T01:20:21.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.676+0000 7f680d945700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f68081128e0 con 0x7f680806ab30 2024-06-10T01:20:21.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.677+0000 7f68067fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f680806ab30 0x7f6808199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:21.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.677+0000 7f6806ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f680806a170 0x7f6808198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:21.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.677+0000 7f68067fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f680806ab30 0x7f6808199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:45906/0 (socket says 172.21.15.7:45906) 2024-06-10T01:20:21.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.677+0000 7f68067fc700 1 -- 172.21.15.7:0/491267719 learned_addr learned my addr 172.21.15.7:0/491267719 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:21.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.677+0000 7f68077fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6808108500 0x7f680819d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:21.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.677+0000 7f68077fe700 1 -- 172.21.15.7:0/491267719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f680806ab30 msgr2=0x7f6808199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:21.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.677+0000 7f68077fe700 1 --2- 172.21.15.7:0/491267719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f680806ab30 0x7f6808199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.677+0000 7f68077fe700 1 -- 172.21.15.7:0/491267719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f680806a170 msgr2=0x7f6808198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:21.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.677+0000 7f68077fe700 1 --2- 172.21.15.7:0/491267719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f680806a170 0x7f6808198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.677+0000 7f68077fe700 1 -- 172.21.15.7:0/491267719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680819dc80 con 0x7f6808108500 2024-06-10T01:20:21.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.677+0000 7f68067fc700 1 --2- 172.21.15.7:0/491267719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f680806ab30 0x7f6808199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:20:21.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.677+0000 7f6806ffd700 1 --2- 172.21.15.7:0/491267719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f680806a170 0x7f6808198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:20:21.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.677+0000 7f68077fe700 1 --2- 172.21.15.7:0/491267719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6808108500 0x7f680819d580 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f680000d010 tx=0x7f6800004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:21.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.677+0000 7f680c943700 1 -- 172.21.15.7:0/491267719 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6800021070 con 0x7f6808108500 2024-06-10T01:20:21.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.678+0000 7f680d945700 1 -- 172.21.15.7:0/491267719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f680819df10 con 0x7f6808108500 2024-06-10T01:20:21.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.678+0000 7f680d945700 1 -- 172.21.15.7:0/491267719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6808101390 con 0x7f6808108500 2024-06-10T01:20:21.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.678+0000 7f680c943700 1 -- 172.21.15.7:0/491267719 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6800007b80 con 0x7f6808108500 2024-06-10T01:20:21.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.678+0000 7f680c943700 1 -- 172.21.15.7:0/491267719 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68000052a0 con 0x7f6808108500 2024-06-10T01:20:21.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.681+0000 7f680c943700 1 -- 172.21.15.7:0/491267719 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f6800024070 con 0x7f6808108500 2024-06-10T01:20:21.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.681+0000 7f68181c8b80 1 Processor -- start 2024-06-10T01:20:21.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.682+0000 7f68181c8b80 1 -- start start 2024-06-10T01:20:21.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.682+0000 7f68181c8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d5a76500 0x5595d5a768e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:21.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.682+0000 7f680c943700 1 --2- 172.21.15.7:0/491267719 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f67f00a2ca0 0x7f67f00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:21.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.682+0000 7f68181c8b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595d5a76e20 0x5595d5a70710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:21.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.682+0000 7f68181c8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595d5a70c50 0x5595d5a71070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:21.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.682+0000 7f68181c8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5595d5963060 con 0x5595d5a76500 2024-06-10T01:20:21.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.682+0000 7f68181c8b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5595d59c1950 con 0x5595d5a76e20 2024-06-10T01:20:21.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.682+0000 7f68181c8b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5595d5929560 con 0x5595d5a70c50 2024-06-10T01:20:21.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.682+0000 7f68067fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d5a76500 0x5595d5a768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:21.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.682+0000 7f680c943700 1 -- 172.21.15.7:0/491267719 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f680000f020 con 0x7f6808108500 2024-06-10T01:20:21.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.682+0000 7f68067fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d5a76500 0x5595d5a768e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48952/0 (socket says 172.21.15.7:48952) 2024-06-10T01:20:21.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.682+0000 7f68067fc700 1 -- 172.21.15.7:0/333772140 learned_addr learned my addr 172.21.15.7:0/333772140 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:21.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.682+0000 7f6806ffd700 1 --2- 172.21.15.7:0/491267719 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f67f00a2ca0 0x7f67f00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:21.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.682+0000 7f68077fe700 1 --2- 172.21.15.7:0/333772140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595d5a70c50 0x5595d5a71070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:21.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.682+0000 7f68067fc700 1 --2- 172.21.15.7:0/333772140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595d5a76e20 0x5595d5a70710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:21.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.683+0000 7f68067fc700 1 -- 172.21.15.7:0/333772140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595d5a70c50 msgr2=0x5595d5a71070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:21.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.683+0000 7f68067fc700 1 --2- 172.21.15.7:0/333772140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595d5a70c50 0x5595d5a71070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.683+0000 7f68067fc700 1 -- 172.21.15.7:0/333772140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595d5a76e20 msgr2=0x5595d5a70710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:21.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.683+0000 7f68067fc700 1 --2- 172.21.15.7:0/333772140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595d5a76e20 0x5595d5a70710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.683+0000 7f68067fc700 1 -- 172.21.15.7:0/333772140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595d59109d0 con 0x5595d5a76500 2024-06-10T01:20:21.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.683+0000 7f68077fe700 1 --2- 172.21.15.7:0/333772140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595d5a70c50 0x5595d5a71070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:20:21.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.683+0000 7f6806ffd700 1 --2- 172.21.15.7:0/491267719 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f67f00a2ca0 0x7f67f00a5160 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f67f8005950 tx=0x7f67f800f820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:21.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.683+0000 7f68067fc700 1 --2- 172.21.15.7:0/333772140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d5a76500 0x5595d5a768e0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f67fc005fd0 tx=0x7f67fc007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978466d48ce6ddf server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:21.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.683+0000 7f67edffb700 1 -- 172.21.15.7:0/333772140 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc012080 con 0x5595d5a76500 2024-06-10T01:20:21.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.683+0000 7f67edffb700 1 -- 172.21.15.7:0/333772140 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67fc012220 con 0x5595d5a76500 2024-06-10T01:20:21.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.684+0000 7f67edffb700 1 -- 172.21.15.7:0/333772140 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc010850 con 0x5595d5a76500 2024-06-10T01:20:21.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.684+0000 7f68181c8b80 1 -- 172.21.15.7:0/333772140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d5a76500 msgr2=0x5595d5a768e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:21.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.684+0000 7f68181c8b80 1 --2- 172.21.15.7:0/333772140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d5a76500 0x5595d5a768e0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f67fc005fd0 tx=0x7f67fc007c90 comp rx=0 tx=0).stop 2024-06-10T01:20:21.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.684+0000 7f68181c8b80 1 -- 172.21.15.7:0/333772140 shutdown_connections 2024-06-10T01:20:21.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.684+0000 7f68181c8b80 1 --2- 172.21.15.7:0/333772140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595d5a70c50 0x5595d5a71070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.684+0000 7f68181c8b80 1 --2- 172.21.15.7:0/333772140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d5a76500 0x5595d5a768e0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.684+0000 7f68181c8b80 1 --2- 172.21.15.7:0/333772140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595d5a76e20 0x5595d5a70710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.684+0000 7f68181c8b80 1 -- 172.21.15.7:0/333772140 >> 172.21.15.7:0/333772140 conn(0x5595d599d180 msgr2=0x5595d5a6ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:21.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.684+0000 7f68181c8b80 1 -- 172.21.15.7:0/333772140 shutdown_connections 2024-06-10T01:20:21.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.684+0000 7f68181c8b80 1 -- 172.21.15.7:0/333772140 wait complete. 2024-06-10T01:20:21.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.684+0000 7f68181c8b80 1 -- start start 2024-06-10T01:20:21.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.685+0000 7f68181c8b80 1 Processor -- start 2024-06-10T01:20:21.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.685+0000 7f68181c8b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595d5a70c50 0x5595d5a7ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:21.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.685+0000 7f68181c8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d5a76500 0x5595d5a79d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:21.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.685+0000 7f68181c8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595d5a76e20 0x5595d5a7a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:21.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.685+0000 7f68181c8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5595d59620c0 con 0x5595d5a76500 2024-06-10T01:20:21.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.685+0000 7f68181c8b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5595d59bd910 con 0x5595d5a70c50 2024-06-10T01:20:21.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.685+0000 7f68181c8b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5595d5a7d7f0 con 0x5595d5a76e20 2024-06-10T01:20:21.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.685+0000 7f68181c8b80 10 client.0 _ll_register_callbacks cb 0x5595d59576b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:20:21.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.685+0000 7f68067fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d5a76500 0x5595d5a79d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:21.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.685+0000 7f68067fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d5a76500 0x5595d5a79d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48958/0 (socket says 172.21.15.7:48958) 2024-06-10T01:20:21.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.685+0000 7f68067fc700 1 -- 172.21.15.7:0/1254082871 learned_addr learned my addr 172.21.15.7:0/1254082871 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:21.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.685+0000 7f68077fe700 1 --2- 172.21.15.7:0/1254082871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595d5a76e20 0x5595d5a7a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:21.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.685+0000 7f6806ffd700 1 --2- 172.21.15.7:0/1254082871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595d5a70c50 0x5595d5a7ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:21.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.685+0000 7f68067fc700 1 -- 172.21.15.7:0/1254082871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595d5a76e20 msgr2=0x5595d5a7a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:21.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.685+0000 7f68067fc700 1 --2- 172.21.15.7:0/1254082871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595d5a76e20 0x5595d5a7a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.685+0000 7f68067fc700 1 -- 172.21.15.7:0/1254082871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595d5a70c50 msgr2=0x5595d5a7ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:21.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.685+0000 7f68067fc700 1 --2- 172.21.15.7:0/1254082871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595d5a70c50 0x5595d5a7ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.685+0000 7f68067fc700 1 -- 172.21.15.7:0/1254082871 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595d5987850 con 0x5595d5a76500 2024-06-10T01:20:21.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.686+0000 7f68067fc700 1 --2- 172.21.15.7:0/1254082871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d5a76500 0x5595d5a79d90 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f67fc008010 tx=0x7f67fc00e8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49415c179f314da server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:21.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.686+0000 7f67e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:20:21.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.686+0000 7f67e6ffd700 1 -- 172.21.15.7:0/1254082871 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc012080 con 0x5595d5a76500 2024-06-10T01:20:21.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.686+0000 7f68181c8b80 1 -- 172.21.15.7:0/1254082871 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67fc012220 con 0x5595d5a76500 2024-06-10T01:20:21.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.686+0000 7f67e6ffd700 1 -- 172.21.15.7:0/1254082871 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f67fc005050 con 0x5595d5a76500 2024-06-10T01:20:21.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.686+0000 7f67e6ffd700 1 -- 172.21.15.7:0/1254082871 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f67fc030050 con 0x5595d5a76500 2024-06-10T01:20:21.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.686+0000 7f68181c8b80 10 client.19071 fetch_fsmap learned FSMap version 44 2024-06-10T01:20:21.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.686+0000 7f68181c8b80 1 -- 172.21.15.7:0/1254082871 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5595d5a87e20 con 0x5595d5a76500 2024-06-10T01:20:21.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.686+0000 7f67e6ffd700 1 -- 172.21.15.7:0/1254082871 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67fc012220 con 0x5595d5a76500 2024-06-10T01:20:21.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.686+0000 7f67e6ffd700 1 -- 172.21.15.7:0/1254082871 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f67fc0104f0 con 0x5595d5a76500 2024-06-10T01:20:21.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.687+0000 7f68181c8b80 10 client.19071 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:20:21.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.687+0000 7f68181c8b80 10 client.19071 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:20:21.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.687+0000 7f68181c8b80 20 client.19071 populate_metadata read hostname 'smithi007' 2024-06-10T01:20:21.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.687+0000 7f68181c8b80 1 --2- 172.21.15.7:0/1254082871 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5595d5a8a5b0 0x5595d5a8c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:21.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.687+0000 7f68181c8b80 4 client.19071 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:20:21.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.687+0000 7f68181c8b80 1 -- 172.21.15.7:0/1254082871 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595d59cee50 con 0x5595d5a8a5b0 2024-06-10T01:20:21.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.687+0000 7f6806ffd700 1 --2- 172.21.15.7:0/1254082871 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5595d5a8a5b0 0x5595d5a8c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:21.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.687+0000 7f67e6ffd700 10 client.19071 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:21.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.688+0000 7f6806ffd700 1 --2- 172.21.15.7:0/1254082871 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5595d5a8a5b0 0x5595d5a8c9f0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b97081102c9c856 server_cookie=a7a8ff0af15f1359 in_seq=0 out_seq=0 2024-06-10T01:20:21.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:21 smithi007 ceph-mon[32398]: pgmap v862: 129 pgs: 129 active+clean; 375 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.0 MiB/s wr, 136 op/s 2024-06-10T01:20:21.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:21 smithi007 ceph-mon[32398]: scrub queued for path: / 2024-06-10T01:20:21.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:21 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:20:21.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:21 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:20:21.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:21 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:20:21.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:21 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:20:21.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:21 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:20:21.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:21 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:20:21.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:21 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:20:21.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:21 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:20:21.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:21 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:20:21.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.688+0000 7f67e6ffd700 1 -- 172.21.15.7:0/1254082871 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5595d59cee50 con 0x5595d5a8a5b0 2024-06-10T01:20:21.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.688+0000 7f67e6ffd700 10 client.19071 handle_command_reply: tid=0 2024-06-10T01:20:21.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.688+0000 7f68181c8b80 1 client.19071 shutdown 2024-06-10T01:20:21.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.689+0000 7f68181c8b80 1 -- 172.21.15.7:0/1254082871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d5a76500 msgr2=0x5595d5a79d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:21.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.689+0000 7f68181c8b80 1 --2- 172.21.15.7:0/1254082871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d5a76500 0x5595d5a79d90 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f67fc008010 tx=0x7f67fc00e8e0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.689+0000 7f68181c8b80 1 -- 172.21.15.7:0/1254082871 shutdown_connections 2024-06-10T01:20:21.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.689+0000 7f68181c8b80 1 --2- 172.21.15.7:0/1254082871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595d5a76e20 0x5595d5a7a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.689+0000 7f68181c8b80 1 --2- 172.21.15.7:0/1254082871 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5595d5a8a5b0 0x5595d5a8c9f0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.689+0000 7f68181c8b80 1 --2- 172.21.15.7:0/1254082871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595d5a70c50 0x5595d5a7ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.689+0000 7f68181c8b80 1 --2- 172.21.15.7:0/1254082871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d5a76500 0x5595d5a79d90 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.689+0000 7f68181c8b80 1 -- 172.21.15.7:0/1254082871 >> 172.21.15.7:0/1254082871 conn(0x5595d599d180 msgr2=0x5595d5a6abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:21.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.689+0000 7f67e6ffd700 0 client.19071 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:21.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.689+0000 7f68181c8b80 1 -- 172.21.15.7:0/1254082871 shutdown_connections 2024-06-10T01:20:21.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.689+0000 7f68181c8b80 1 -- 172.21.15.7:0/1254082871 wait complete. 2024-06-10T01:20:21.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.689+0000 7f68181c8b80 20 client.19071 trim_cache size 0 max 16384 2024-06-10T01:20:21.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:21 smithi175 ceph-mon[36636]: pgmap v862: 129 pgs: 129 active+clean; 375 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.0 MiB/s wr, 136 op/s 2024-06-10T01:20:21.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:21 smithi175 ceph-mon[36636]: scrub queued for path: / 2024-06-10T01:20:21.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:21 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:20:21.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:21 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:20:21.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:21 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:20:21.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:21 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:20:21.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:21 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:20:21.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:21 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:20:21.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:21 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:20:21.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:21 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:20:21.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:21 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:20:21.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.711+0000 7f68181c8b80 1 Processor -- start 2024-06-10T01:20:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.711+0000 7f68181c8b80 1 -- start start 2024-06-10T01:20:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.711+0000 7f68181c8b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595d599d180 0x5595d5a70710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.711+0000 7f68181c8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595d5a70c50 0x5595d5a77910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.711+0000 7f68181c8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d5a76500 0x5595d5a6b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.711+0000 7f68181c8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f67f4011390 con 0x5595d5a76500 2024-06-10T01:20:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.711+0000 7f68181c8b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f67f400eb90 con 0x5595d599d180 2024-06-10T01:20:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.711+0000 7f68181c8b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f67f400c9e0 con 0x5595d5a70c50 2024-06-10T01:20:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.712+0000 7f68077fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d5a76500 0x5595d5a6b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.712+0000 7f68077fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d5a76500 0x5595d5a6b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48964/0 (socket says 172.21.15.7:48964) 2024-06-10T01:20:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.712+0000 7f68077fe700 1 -- 172.21.15.7:0/478357960 learned_addr learned my addr 172.21.15.7:0/478357960 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.712+0000 7f68067fc700 1 --2- 172.21.15.7:0/478357960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595d5a70c50 0x5595d5a77910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:21.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.712+0000 7f6806ffd700 1 --2- 172.21.15.7:0/478357960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595d599d180 0x5595d5a70710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:21.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.712+0000 7f68077fe700 1 -- 172.21.15.7:0/478357960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595d5a70c50 msgr2=0x5595d5a77910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:21.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.712+0000 7f68077fe700 1 --2- 172.21.15.7:0/478357960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595d5a70c50 0x5595d5a77910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.712+0000 7f68077fe700 1 -- 172.21.15.7:0/478357960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595d599d180 msgr2=0x5595d5a70710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:21.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.712+0000 7f68077fe700 1 --2- 172.21.15.7:0/478357960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595d599d180 0x5595d5a70710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.712+0000 7f68077fe700 1 -- 172.21.15.7:0/478357960 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595d5a869f0 con 0x5595d5a76500 2024-06-10T01:20:21.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.712+0000 7f68077fe700 1 --2- 172.21.15.7:0/478357960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d5a76500 0x5595d5a6b330 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f6800098a90 tx=0x7f68000099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a62337460df997 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:21.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.712+0000 7f67ed7fa700 1 -- 172.21.15.7:0/478357960 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680007a240 con 0x5595d5a76500 2024-06-10T01:20:21.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.712+0000 7f67ed7fa700 1 -- 172.21.15.7:0/478357960 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68000a0020 con 0x5595d5a76500 2024-06-10T01:20:21.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.712+0000 7f67ed7fa700 1 -- 172.21.15.7:0/478357960 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680009b0b0 con 0x5595d5a76500 2024-06-10T01:20:21.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.713+0000 7f68181c8b80 1 -- 172.21.15.7:0/478357960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d5a76500 msgr2=0x5595d5a6b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:21.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.713+0000 7f68181c8b80 1 --2- 172.21.15.7:0/478357960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d5a76500 0x5595d5a6b330 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f6800098a90 tx=0x7f68000099e0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.713+0000 7f68181c8b80 1 -- 172.21.15.7:0/478357960 shutdown_connections 2024-06-10T01:20:21.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.713+0000 7f68181c8b80 1 --2- 172.21.15.7:0/478357960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d5a76500 0x5595d5a6b330 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.713+0000 7f68181c8b80 1 --2- 172.21.15.7:0/478357960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595d599d180 0x5595d5a70710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.713+0000 7f68181c8b80 1 --2- 172.21.15.7:0/478357960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595d5a70c50 0x5595d5a77910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.713+0000 7f68181c8b80 1 -- 172.21.15.7:0/478357960 >> 172.21.15.7:0/478357960 conn(0x5595d5a79950 msgr2=0x5595d5a77200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:21.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.713+0000 7f68181c8b80 1 -- 172.21.15.7:0/478357960 shutdown_connections 2024-06-10T01:20:21.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.713+0000 7f68181c8b80 1 -- 172.21.15.7:0/478357960 wait complete. 2024-06-10T01:20:21.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.714+0000 7f68181c8b80 1 -- start start 2024-06-10T01:20:21.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.714+0000 7f68181c8b80 1 Processor -- start 2024-06-10T01:20:21.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.714+0000 7f68181c8b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595d599d180 0x5595d5a82d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:21.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.714+0000 7f68181c8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d5a70c50 0x5595d5a83280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:21.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.714+0000 7f68181c8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595d5a76500 0x5595d5a7fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:21.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.714+0000 7f68181c8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f67f400d050 con 0x5595d5a70c50 2024-06-10T01:20:21.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.714+0000 7f68181c8b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f67f400e4c0 con 0x5595d599d180 2024-06-10T01:20:21.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.714+0000 7f68181c8b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f67f400d720 con 0x5595d5a76500 2024-06-10T01:20:21.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.714+0000 7f68067fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d5a70c50 0x5595d5a83280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:21.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.714+0000 7f68181c8b80 10 client.0 _ll_register_callbacks cb 0x5595d59576b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:20:21.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.714+0000 7f68067fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d5a70c50 0x5595d5a83280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48980/0 (socket says 172.21.15.7:48980) 2024-06-10T01:20:21.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.714+0000 7f68067fc700 1 -- 172.21.15.7:0/2054555344 learned_addr learned my addr 172.21.15.7:0/2054555344 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:21.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.714+0000 7f6806ffd700 1 --2- 172.21.15.7:0/2054555344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595d599d180 0x5595d5a82d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:21.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.715+0000 7f68077fe700 1 --2- 172.21.15.7:0/2054555344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595d5a76500 0x5595d5a7fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:21.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.715+0000 7f68067fc700 1 -- 172.21.15.7:0/2054555344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595d5a76500 msgr2=0x5595d5a7fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:21.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.715+0000 7f68067fc700 1 --2- 172.21.15.7:0/2054555344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595d5a76500 0x5595d5a7fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.715+0000 7f68067fc700 1 -- 172.21.15.7:0/2054555344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595d599d180 msgr2=0x5595d5a82d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:21.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.715+0000 7f68067fc700 1 --2- 172.21.15.7:0/2054555344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595d599d180 0x5595d5a82d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.715+0000 7f68067fc700 1 -- 172.21.15.7:0/2054555344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595d5a802d0 con 0x5595d5a70c50 2024-06-10T01:20:21.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.715+0000 7f68077fe700 1 --2- 172.21.15.7:0/2054555344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595d5a76500 0x5595d5a7fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:20:21.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.715+0000 7f6806ffd700 1 --2- 172.21.15.7:0/2054555344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595d599d180 0x5595d5a82d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:20:21.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.715+0000 7f68067fc700 1 --2- 172.21.15.7:0/2054555344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d5a70c50 0x5595d5a83280 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x5595d5a65fe0 tx=0x7f67fc00efc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e40798d1d14675 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:21.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.715+0000 7f67e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:20:21.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.715+0000 7f67e6ffd700 1 -- 172.21.15.7:0/2054555344 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc012720 con 0x5595d5a70c50 2024-06-10T01:20:21.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.715+0000 7f68181c8b80 1 -- 172.21.15.7:0/2054555344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68000a0020 con 0x5595d5a70c50 2024-06-10T01:20:21.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.715+0000 7f67e6ffd700 1 -- 172.21.15.7:0/2054555344 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f67fc010cc0 con 0x5595d5a70c50 2024-06-10T01:20:21.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.715+0000 7f67e6ffd700 1 -- 172.21.15.7:0/2054555344 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f67fc03d050 con 0x5595d5a70c50 2024-06-10T01:20:21.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.716+0000 7f68181c8b80 10 client.19077 fetch_fsmap learned FSMap version 44 2024-06-10T01:20:21.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.716+0000 7f68181c8b80 1 -- 172.21.15.7:0/2054555344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5595d5af5dc0 con 0x5595d5a70c50 2024-06-10T01:20:21.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.716+0000 7f67e6ffd700 1 -- 172.21.15.7:0/2054555344 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68000a0020 con 0x5595d5a70c50 2024-06-10T01:20:21.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.716+0000 7f67e6ffd700 1 -- 172.21.15.7:0/2054555344 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f67fc012070 con 0x5595d5a70c50 2024-06-10T01:20:21.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.716+0000 7f68181c8b80 10 client.19077 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:20:21.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.716+0000 7f68181c8b80 10 client.19077 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:20:21.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.716+0000 7f68181c8b80 20 client.19077 populate_metadata read hostname 'smithi007' 2024-06-10T01:20:21.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.716+0000 7f68181c8b80 1 --2- 172.21.15.7:0/2054555344 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5595d5a76e20 0x5595d5af8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:21.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.716+0000 7f68181c8b80 4 client.19077 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1d08866a", "release": true, "await": true}] 2024-06-10T01:20:21.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.716+0000 7f68181c8b80 1 -- 172.21.15.7:0/2054555344 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1d08866a", "release": true, "await": true}) -- 0x5595d59c93d0 con 0x5595d5a76e20 2024-06-10T01:20:21.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.716+0000 7f6806ffd700 1 --2- 172.21.15.7:0/2054555344 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5595d5a76e20 0x5595d5af8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:21.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.717+0000 7f67e6ffd700 10 client.19077 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:21.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.717+0000 7f6806ffd700 1 --2- 172.21.15.7:0/2054555344 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5595d5a76e20 0x5595d5af8540 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9c87a676a99b7e3 server_cookie=9c069d027c4415cf in_seq=0 out_seq=0 2024-06-10T01:20:21.737 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:20:21.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.770+0000 7f67e6ffd700 1 -- 172.21.15.7:0/2054555344 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5595d59c93d0 con 0x5595d5a76e20 2024-06-10T01:20:21.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.770+0000 7f67e6ffd700 10 client.19077 handle_command_reply: tid=0 2024-06-10T01:20:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.770+0000 7f68181c8b80 1 client.19077 shutdown 2024-06-10T01:20:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.770+0000 7f68181c8b80 1 -- 172.21.15.7:0/2054555344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d5a70c50 msgr2=0x5595d5a83280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.770+0000 7f68181c8b80 1 --2- 172.21.15.7:0/2054555344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d5a70c50 0x5595d5a83280 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x5595d5a65fe0 tx=0x7f67fc00efc0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.770+0000 7f68181c8b80 1 -- 172.21.15.7:0/2054555344 shutdown_connections 2024-06-10T01:20:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.770+0000 7f68181c8b80 1 --2- 172.21.15.7:0/2054555344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595d5a76500 0x5595d5a7fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.770+0000 7f68181c8b80 1 --2- 172.21.15.7:0/2054555344 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5595d5a76e20 0x5595d5af8540 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.770+0000 7f68181c8b80 1 --2- 172.21.15.7:0/2054555344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595d599d180 0x5595d5a82d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.770+0000 7f68181c8b80 1 --2- 172.21.15.7:0/2054555344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d5a70c50 0x5595d5a83280 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.770+0000 7f67e6ffd700 0 client.19077 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.770+0000 7f68181c8b80 1 -- 172.21.15.7:0/2054555344 >> 172.21.15.7:0/2054555344 conn(0x5595d5a79950 msgr2=0x5595d5a859e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.770+0000 7f68181c8b80 1 -- 172.21.15.7:0/2054555344 shutdown_connections 2024-06-10T01:20:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.770+0000 7f68181c8b80 1 -- 172.21.15.7:0/2054555344 wait complete. 2024-06-10T01:20:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.771+0000 7f68181c8b80 20 client.19077 trim_cache size 0 max 16384 2024-06-10T01:20:21.766 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:20:21.766 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 44, 2024-06-10T01:20:21.766 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:20:21.766 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 48, 2024-06-10T01:20:21.766 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:20:21.767 INFO:teuthology.orchestra.run.smithi007.stdout: "1d08866a": { 2024-06-10T01:20:21.767 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 48, 2024-06-10T01:20:21.767 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:20:21.767 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:20:21.767 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:20:21.767 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:20:21.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:20:21.767 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:20:21.767 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 147.7, 2024-06-10T01:20:21.767 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:20:21.767 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:20:21.767 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:20:21.768 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:20:21.768 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:20:21.768 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:20:21.768 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:20:21.768 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:20:21.768 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:20:21.768 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:20:21.768 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:20:21.768 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:20:21.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.771+0000 7f680d945700 1 -- 172.21.15.7:0/491267719 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f67f00a2ca0 msgr2=0x7f67f00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:21.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.771+0000 7f680d945700 1 --2- 172.21.15.7:0/491267719 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f67f00a2ca0 0x7f67f00a5160 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f67f8005950 tx=0x7f67f800f820 comp rx=0 tx=0).stop 2024-06-10T01:20:21.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.771+0000 7f680d945700 1 -- 172.21.15.7:0/491267719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6808108500 msgr2=0x7f680819d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:21.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.771+0000 7f680d945700 1 --2- 172.21.15.7:0/491267719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6808108500 0x7f680819d580 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f680000d010 tx=0x7f6800004d50 comp rx=0 tx=0).stop 2024-06-10T01:20:21.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.771+0000 7f680d945700 1 -- 172.21.15.7:0/491267719 shutdown_connections 2024-06-10T01:20:21.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.771+0000 7f680d945700 1 --2- 172.21.15.7:0/491267719 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f67f00a2ca0 0x7f67f00a5160 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.771+0000 7f680d945700 1 --2- 172.21.15.7:0/491267719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6808108500 0x7f680819d580 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.771+0000 7f680d945700 1 --2- 172.21.15.7:0/491267719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f680806a170 0x7f6808198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.771+0000 7f680d945700 1 --2- 172.21.15.7:0/491267719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f680806ab30 0x7f6808199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:21.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.772+0000 7f680d945700 1 -- 172.21.15.7:0/491267719 >> 172.21.15.7:0/491267719 conn(0x7f6808078340 msgr2=0x7f68080fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:21.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.772+0000 7f680d945700 1 -- 172.21.15.7:0/491267719 shutdown_connections 2024-06-10T01:20:21.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:21.772+0000 7f680d945700 1 -- 172.21.15.7:0/491267719 wait complete. 2024-06-10T01:20:21.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:21 smithi078 ceph-mon[36683]: pgmap v862: 129 pgs: 129 active+clean; 375 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.0 MiB/s wr, 136 op/s 2024-06-10T01:20:21.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:21 smithi078 ceph-mon[36683]: scrub queued for path: / 2024-06-10T01:20:21.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:21 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:20:21.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:21 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:20:21.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:21 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:20:21.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:21 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:20:21.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:21 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:20:21.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:21 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:20:21.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:21 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:20:21.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:21 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:20:21.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:21 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:20:22.471 INFO:tasks.workunit.client.0.smithi007.stdout: 10 13310 14.96 MB/sec warmup 59 sec latency 29809.123 ms 2024-06-10T01:20:22.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:22 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:20:22.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:22 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:20:22.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:22 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:20:22.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:22 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:20:22.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:22 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:20:22.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:22 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:20:23.154 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1d08866a', total seconds elapsed: 39.7 2024-06-10T01:20:23.154 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.65 2024-06-10T01:20:23.154 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:20:23.471 INFO:tasks.workunit.client.0.smithi007.stdout: 10 14251 15.82 MB/sec warmup 60 sec latency 37.970 ms 2024-06-10T01:20:23.590 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:23 smithi007 ceph-mon[32398]: pgmap v863: 129 pgs: 129 active+clean; 367 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 4.1 MiB/s wr, 169 op/s 2024-06-10T01:20:23.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:23 smithi175 ceph-mon[36636]: pgmap v863: 129 pgs: 129 active+clean; 367 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 4.1 MiB/s wr, 169 op/s 2024-06-10T01:20:23.796 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:20:23.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:23 smithi078 ceph-mon[36683]: pgmap v863: 129 pgs: 129 active+clean; 367 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 4.1 MiB/s wr, 169 op/s 2024-06-10T01:20:24.471 INFO:tasks.workunit.client.0.smithi007.stdout: 10 14855 16.00 MB/sec warmup 61 sec latency 92.159 ms 2024-06-10T01:20:24.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.526+0000 7fa214627700 1 -- 172.21.15.7:0/231254945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa20c06aa00 msgr2=0x7fa20c06ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:24.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.526+0000 7fa214627700 1 --2- 172.21.15.7:0/231254945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa20c06aa00 0x7fa20c06ae80 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc009990 tx=0x7fa1fc009ca0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.526+0000 7fa214627700 1 -- 172.21.15.7:0/231254945 shutdown_connections 2024-06-10T01:20:24.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.526+0000 7fa214627700 1 --2- 172.21.15.7:0/231254945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa20c06b3c0 0x7fa20c10f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.526+0000 7fa214627700 1 --2- 172.21.15.7:0/231254945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa20c1084f0 0x7fa20c1088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.526+0000 7fa214627700 1 --2- 172.21.15.7:0/231254945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa20c06aa00 0x7fa20c06ae80 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.526+0000 7fa214627700 1 -- 172.21.15.7:0/231254945 >> 172.21.15.7:0/231254945 conn(0x7fa20c078300 msgr2=0x7fa20c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:24.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.526+0000 7fa214627700 1 -- 172.21.15.7:0/231254945 shutdown_connections 2024-06-10T01:20:24.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.527+0000 7fa214627700 1 -- 172.21.15.7:0/231254945 wait complete. 2024-06-10T01:20:24.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.527+0000 7fa214627700 1 Processor -- start 2024-06-10T01:20:24.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.527+0000 7fa214627700 1 -- start start 2024-06-10T01:20:24.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.528+0000 7fa214627700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa20c06aa00 0x7fa20c19e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:24.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.528+0000 7fa214627700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa20c06b3c0 0x7fa20c19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:24.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.528+0000 7fa214627700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa20c1084f0 0x7fa20c198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:24.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.528+0000 7fa214627700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c112750 con 0x7fa20c06b3c0 2024-06-10T01:20:24.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.528+0000 7fa214627700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c1125d0 con 0x7fa20c1084f0 2024-06-10T01:20:24.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.528+0000 7fa214627700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c1128d0 con 0x7fa20c06aa00 2024-06-10T01:20:24.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.528+0000 7fa211bc2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa20c06b3c0 0x7fa20c19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:24.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.528+0000 7fa2123c3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa20c06aa00 0x7fa20c19e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:24.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.528+0000 7fa211bc2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa20c06b3c0 0x7fa20c19ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53832/0 (socket says 172.21.15.7:53832) 2024-06-10T01:20:24.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.528+0000 7fa211bc2700 1 -- 172.21.15.7:0/3537861588 learned_addr learned my addr 172.21.15.7:0/3537861588 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:24.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.528+0000 7fa2123c3700 1 --2- 172.21.15.7:0/3537861588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa20c06aa00 0x7fa20c19e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:41188/0 (socket says 172.21.15.7:41188) 2024-06-10T01:20:24.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.528+0000 7fa212bc4700 1 --2- 172.21.15.7:0/3537861588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa20c1084f0 0x7fa20c198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:24.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.528+0000 7fa212bc4700 1 -- 172.21.15.7:0/3537861588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa20c06aa00 msgr2=0x7fa20c19e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:24.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.528+0000 7fa212bc4700 1 --2- 172.21.15.7:0/3537861588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa20c06aa00 0x7fa20c19e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.528+0000 7fa212bc4700 1 -- 172.21.15.7:0/3537861588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa20c06b3c0 msgr2=0x7fa20c19ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:24.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.528+0000 7fa212bc4700 1 --2- 172.21.15.7:0/3537861588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa20c06b3c0 0x7fa20c19ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.528+0000 7fa212bc4700 1 -- 172.21.15.7:0/3537861588 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20c199010 con 0x7fa20c1084f0 2024-06-10T01:20:24.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.528+0000 7fa2123c3700 1 --2- 172.21.15.7:0/3537861588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa20c06aa00 0x7fa20c19e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:20:24.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.528+0000 7fa211bc2700 1 --2- 172.21.15.7:0/3537861588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa20c06b3c0 0x7fa20c19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:20:24.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.529+0000 7fa212bc4700 1 --2- 172.21.15.7:0/3537861588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa20c1084f0 0x7fa20c198830 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa20400efc0 tx=0x7fa20400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:24.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.529+0000 7fa2037fe700 1 -- 172.21.15.7:0/3537861588 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2040092e0 con 0x7fa20c1084f0 2024-06-10T01:20:24.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.529+0000 7fa2037fe700 1 -- 172.21.15.7:0/3537861588 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa204004bb0 con 0x7fa20c1084f0 2024-06-10T01:20:24.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.529+0000 7fa214627700 1 -- 172.21.15.7:0/3537861588 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa20c1992a0 con 0x7fa20c1084f0 2024-06-10T01:20:24.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.529+0000 7fa2037fe700 1 -- 172.21.15.7:0/3537861588 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa204004280 con 0x7fa20c1084f0 2024-06-10T01:20:24.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.529+0000 7fa214627700 1 -- 172.21.15.7:0/3537861588 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa20c1a5510 con 0x7fa20c1084f0 2024-06-10T01:20:24.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.531+0000 7fa2037fe700 1 -- 172.21.15.7:0/3537861588 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa20401b030 con 0x7fa20c1084f0 2024-06-10T01:20:24.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.533+0000 7fa2037fe700 1 --2- 172.21.15.7:0/3537861588 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa1f80a2c20 0x7fa1f80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:24.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.533+0000 7fa2123c3700 1 --2- 172.21.15.7:0/3537861588 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa1f80a2c20 0x7fa1f80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:24.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.533+0000 7fa2037fe700 1 -- 172.21.15.7:0/3537861588 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa2040cefa0 con 0x7fa20c1084f0 2024-06-10T01:20:24.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.533+0000 7fa2123c3700 1 --2- 172.21.15.7:0/3537861588 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa1f80a2c20 0x7fa1f80a50e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa208009fd0 tx=0x7fa208009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:24.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.534+0000 7fa21eeaab80 1 Processor -- start 2024-06-10T01:20:24.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.534+0000 7fa21eeaab80 1 -- start start 2024-06-10T01:20:24.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.535+0000 7fa21eeaab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab32820470 0x55ab32820850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:24.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.535+0000 7fa21eeaab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3281a670 0x55ab3281aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:24.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.535+0000 7fa21eeaab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab3281b030 0x55ab32827210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:24.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.535+0000 7fa21eeaab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ab32708f60 con 0x55ab3281b030 2024-06-10T01:20:24.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.535+0000 7fa21eeaab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ab32767850 con 0x55ab32820470 2024-06-10T01:20:24.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.535+0000 7fa21eeaab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ab326cf460 con 0x55ab3281a670 2024-06-10T01:20:24.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.535+0000 7fa2123c3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab32820470 0x55ab32820850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:24.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.535+0000 7fa211bc2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3281a670 0x55ab3281aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:24.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.535+0000 7fa211bc2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3281a670 0x55ab3281aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:41196/0 (socket says 172.21.15.7:41196) 2024-06-10T01:20:24.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.535+0000 7fa2123c3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab32820470 0x55ab32820850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:36404/0 (socket says 172.21.15.7:36404) 2024-06-10T01:20:24.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.535+0000 7fa211bc2700 1 -- 172.21.15.7:0/1017215967 learned_addr learned my addr 172.21.15.7:0/1017215967 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:24.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.535+0000 7fa212bc4700 1 --2- 172.21.15.7:0/1017215967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab3281b030 0x55ab32827210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:24.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.535+0000 7fa211bc2700 1 -- 172.21.15.7:0/1017215967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab32820470 msgr2=0x55ab32820850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:24.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.535+0000 7fa211bc2700 1 --2- 172.21.15.7:0/1017215967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab32820470 0x55ab32820850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.535+0000 7fa211bc2700 1 -- 172.21.15.7:0/1017215967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab3281b030 msgr2=0x55ab32827210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:24.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.535+0000 7fa211bc2700 1 --2- 172.21.15.7:0/1017215967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab3281b030 0x55ab32827210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.535+0000 7fa211bc2700 1 -- 172.21.15.7:0/1017215967 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab326b68d0 con 0x55ab3281a670 2024-06-10T01:20:24.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.535+0000 7fa212bc4700 1 --2- 172.21.15.7:0/1017215967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab3281b030 0x55ab32827210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:20:24.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.536+0000 7fa211bc2700 1 --2- 172.21.15.7:0/1017215967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3281a670 0x55ab3281aaf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc005bf0 tx=0x7fa1fc005c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6a3b1106eef5864 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:24.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.536+0000 7fa200ff9700 1 -- 172.21.15.7:0/1017215967 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc027070 con 0x55ab3281a670 2024-06-10T01:20:24.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.536+0000 7fa200ff9700 1 -- 172.21.15.7:0/1017215967 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1fc00b690 con 0x55ab3281a670 2024-06-10T01:20:24.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.536+0000 7fa200ff9700 1 -- 172.21.15.7:0/1017215967 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc005490 con 0x55ab3281a670 2024-06-10T01:20:24.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.538+0000 7fa21eeaab80 1 -- 172.21.15.7:0/1017215967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3281a670 msgr2=0x55ab3281aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:24.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.538+0000 7fa21eeaab80 1 --2- 172.21.15.7:0/1017215967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3281a670 0x55ab3281aaf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc005bf0 tx=0x7fa1fc005c20 comp rx=0 tx=0).stop 2024-06-10T01:20:24.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.538+0000 7fa21eeaab80 1 -- 172.21.15.7:0/1017215967 shutdown_connections 2024-06-10T01:20:24.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.538+0000 7fa21eeaab80 1 --2- 172.21.15.7:0/1017215967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab3281b030 0x55ab32827210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.538+0000 7fa21eeaab80 1 --2- 172.21.15.7:0/1017215967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab32820470 0x55ab32820850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.538+0000 7fa21eeaab80 1 --2- 172.21.15.7:0/1017215967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3281a670 0x55ab3281aaf0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.538+0000 7fa21eeaab80 1 -- 172.21.15.7:0/1017215967 >> 172.21.15.7:0/1017215967 conn(0x55ab32743080 msgr2=0x55ab32811220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:24.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.538+0000 7fa21eeaab80 1 -- 172.21.15.7:0/1017215967 shutdown_connections 2024-06-10T01:20:24.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.538+0000 7fa21eeaab80 1 -- 172.21.15.7:0/1017215967 wait complete. 2024-06-10T01:20:24.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.539+0000 7fa21eeaab80 1 -- start start 2024-06-10T01:20:24.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.539+0000 7fa21eeaab80 1 Processor -- start 2024-06-10T01:20:24.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.539+0000 7fa21eeaab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab3281a670 0x55ab32826f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:24.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.539+0000 7fa21eeaab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab3281b030 0x55ab32824290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:24.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.539+0000 7fa21eeaab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab32820470 0x55ab328247d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:24.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.539+0000 7fa21eeaab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ab32707fc0 con 0x55ab3281b030 2024-06-10T01:20:24.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.539+0000 7fa21eeaab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ab32763810 con 0x55ab3281a670 2024-06-10T01:20:24.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.539+0000 7fa21eeaab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ab32827750 con 0x55ab32820470 2024-06-10T01:20:24.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.539+0000 7fa21eeaab80 10 client.0 _ll_register_callbacks cb 0x55ab326fd5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:20:24.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.540+0000 7fa211bc2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab3281b030 0x55ab32824290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:24.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.540+0000 7fa212bc4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab32820470 0x55ab328247d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:24.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.540+0000 7fa212bc4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab32820470 0x55ab328247d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:41202/0 (socket says 172.21.15.7:41202) 2024-06-10T01:20:24.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.540+0000 7fa212bc4700 1 -- 172.21.15.7:0/4098457043 learned_addr learned my addr 172.21.15.7:0/4098457043 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:24.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.540+0000 7fa2123c3700 1 --2- 172.21.15.7:0/4098457043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab3281a670 0x55ab32826f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:24.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.540+0000 7fa211bc2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab3281b030 0x55ab32824290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53848/0 (socket says 172.21.15.7:53848) 2024-06-10T01:20:24.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.540+0000 7fa212bc4700 1 -- 172.21.15.7:0/4098457043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab3281a670 msgr2=0x55ab32826f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:24.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.540+0000 7fa212bc4700 1 --2- 172.21.15.7:0/4098457043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab3281a670 0x55ab32826f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.540+0000 7fa212bc4700 1 -- 172.21.15.7:0/4098457043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab3281b030 msgr2=0x55ab32824290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:24.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.540+0000 7fa212bc4700 1 --2- 172.21.15.7:0/4098457043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab3281b030 0x55ab32824290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.540+0000 7fa212bc4700 1 -- 172.21.15.7:0/4098457043 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab3272d750 con 0x55ab32820470 2024-06-10T01:20:24.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.540+0000 7fa211bc2700 1 --2- 172.21.15.7:0/4098457043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab3281b030 0x55ab32824290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:20:24.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.540+0000 7fa212bc4700 1 --2- 172.21.15.7:0/4098457043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab32820470 0x55ab328247d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa204097910 tx=0x7fa20409b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8294f4eff867eedd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:24.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.540+0000 7fa1f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:20:24.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.541+0000 7fa1f5ffb700 1 -- 172.21.15.7:0/4098457043 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2040ccd80 con 0x55ab32820470 2024-06-10T01:20:24.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.541+0000 7fa21eeaab80 1 -- 172.21.15.7:0/4098457043 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ab32774d50 con 0x55ab32820470 2024-06-10T01:20:24.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.541+0000 7fa1f5ffb700 1 -- 172.21.15.7:0/4098457043 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa20409a900 con 0x55ab32820470 2024-06-10T01:20:24.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.541+0000 7fa1f5ffb700 1 -- 172.21.15.7:0/4098457043 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa20409bbb0 con 0x55ab32820470 2024-06-10T01:20:24.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.541+0000 7fa21eeaab80 10 client.28387 fetch_fsmap learned FSMap version 44 2024-06-10T01:20:24.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.541+0000 7fa21eeaab80 1 -- 172.21.15.7:0/4098457043 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ab32831ca0 con 0x55ab32820470 2024-06-10T01:20:24.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.541+0000 7fa1f5ffb700 1 -- 172.21.15.7:0/4098457043 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ab32774d50 con 0x55ab32820470 2024-06-10T01:20:24.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.541+0000 7fa1f5ffb700 1 -- 172.21.15.7:0/4098457043 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fa2040b4200 con 0x55ab32820470 2024-06-10T01:20:24.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.541+0000 7fa21eeaab80 10 client.28387 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:20:24.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.541+0000 7fa21eeaab80 10 client.28387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:20:24.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.541+0000 7fa21eeaab80 20 client.28387 populate_metadata read hostname 'smithi007' 2024-06-10T01:20:24.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.542+0000 7fa21eeaab80 1 --2- 172.21.15.7:0/4098457043 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ab32834650 0x55ab32836a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:24.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.542+0000 7fa21eeaab80 4 client.28387 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:20:24.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.542+0000 7fa21eeaab80 1 -- 172.21.15.7:0/4098457043 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab3276f2d0 con 0x55ab32834650 2024-06-10T01:20:24.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.542+0000 7fa2123c3700 1 --2- 172.21.15.7:0/4098457043 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ab32834650 0x55ab32836a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:24.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.542+0000 7fa2123c3700 1 --2- 172.21.15.7:0/4098457043 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ab32834650 0x55ab32836a70 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e5a50bd3d6d1289 server_cookie=6849fbfe015784a4 in_seq=0 out_seq=0 2024-06-10T01:20:24.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.542+0000 7fa1f5ffb700 10 client.28387 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:24.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.544+0000 7fa1f5ffb700 1 -- 172.21.15.7:0/4098457043 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ab3276f2d0 con 0x55ab32834650 2024-06-10T01:20:24.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.544+0000 7fa1f5ffb700 10 client.28387 handle_command_reply: tid=0 2024-06-10T01:20:24.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.544+0000 7fa21eeaab80 1 client.28387 shutdown 2024-06-10T01:20:24.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.545+0000 7fa21eeaab80 1 -- 172.21.15.7:0/4098457043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab32820470 msgr2=0x55ab328247d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:24.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.545+0000 7fa21eeaab80 1 --2- 172.21.15.7:0/4098457043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab32820470 0x55ab328247d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa204097910 tx=0x7fa20409b9f0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.546+0000 7fa21eeaab80 1 -- 172.21.15.7:0/4098457043 shutdown_connections 2024-06-10T01:20:24.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.546+0000 7fa21eeaab80 1 --2- 172.21.15.7:0/4098457043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab32820470 0x55ab328247d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.546+0000 7fa21eeaab80 1 --2- 172.21.15.7:0/4098457043 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ab32834650 0x55ab32836a70 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.546+0000 7fa21eeaab80 1 --2- 172.21.15.7:0/4098457043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab3281a670 0x55ab32826f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.546+0000 7fa21eeaab80 1 --2- 172.21.15.7:0/4098457043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab3281b030 0x55ab32824290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.546+0000 7fa21eeaab80 1 -- 172.21.15.7:0/4098457043 >> 172.21.15.7:0/4098457043 conn(0x55ab32743080 msgr2=0x55ab32810700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:24.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.546+0000 7fa1f5ffb700 0 client.28387 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:24.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.547+0000 7fa21eeaab80 1 -- 172.21.15.7:0/4098457043 shutdown_connections 2024-06-10T01:20:24.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.547+0000 7fa21eeaab80 1 -- 172.21.15.7:0/4098457043 wait complete. 2024-06-10T01:20:24.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.547+0000 7fa21eeaab80 20 client.28387 trim_cache size 0 max 16384 2024-06-10T01:20:24.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.585+0000 7fa21eeaab80 1 Processor -- start 2024-06-10T01:20:24.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.585+0000 7fa21eeaab80 1 -- start start 2024-06-10T01:20:24.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.586+0000 7fa21eeaab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab3281a670 0x55ab32830460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:24.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.586+0000 7fa21eeaab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3281b030 0x55ab32815960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:24.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.586+0000 7fa21eeaab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab32820470 0x55ab32826ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:24.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.586+0000 7fa21eeaab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c1ae7b0 con 0x55ab3281a670 2024-06-10T01:20:24.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.586+0000 7fa21eeaab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c1abfb0 con 0x55ab32820470 2024-06-10T01:20:24.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.586+0000 7fa21eeaab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c1a9e00 con 0x55ab3281b030 2024-06-10T01:20:24.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.586+0000 7fa2123c3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab3281a670 0x55ab32830460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:24.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.586+0000 7fa212bc4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab32820470 0x55ab32826ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:24.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.586+0000 7fa2123c3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab3281a670 0x55ab32830460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53852/0 (socket says 172.21.15.7:53852) 2024-06-10T01:20:24.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.586+0000 7fa212bc4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab32820470 0x55ab32826ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:36420/0 (socket says 172.21.15.7:36420) 2024-06-10T01:20:24.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.586+0000 7fa2123c3700 1 -- 172.21.15.7:0/2982496147 learned_addr learned my addr 172.21.15.7:0/2982496147 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:24.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.586+0000 7fa211bc2700 1 --2- 172.21.15.7:0/2982496147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3281b030 0x55ab32815960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:24.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.586+0000 7fa212bc4700 1 -- 172.21.15.7:0/2982496147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3281b030 msgr2=0x55ab32815960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:24.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.586+0000 7fa212bc4700 1 --2- 172.21.15.7:0/2982496147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3281b030 0x55ab32815960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.586+0000 7fa212bc4700 1 -- 172.21.15.7:0/2982496147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab3281a670 msgr2=0x55ab32830460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:24.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.586+0000 7fa212bc4700 1 --2- 172.21.15.7:0/2982496147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab3281a670 0x55ab32830460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.586+0000 7fa212bc4700 1 -- 172.21.15.7:0/2982496147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab32824060 con 0x55ab32820470 2024-06-10T01:20:24.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.586+0000 7fa2123c3700 1 --2- 172.21.15.7:0/2982496147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab3281a670 0x55ab32830460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:20:24.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.586+0000 7fa211bc2700 1 --2- 172.21.15.7:0/2982496147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3281b030 0x55ab32815960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:20:24.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.587+0000 7fa212bc4700 1 --2- 172.21.15.7:0/2982496147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab32820470 0x55ab32826ad0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa20400ef90 tx=0x7fa2040999c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f12d4a2e64110677 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:24.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.587+0000 7fa1f7fff700 1 -- 172.21.15.7:0/2982496147 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20409b4b0 con 0x55ab32820470 2024-06-10T01:20:24.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.587+0000 7fa1f7fff700 1 -- 172.21.15.7:0/2982496147 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa20409f860 con 0x55ab32820470 2024-06-10T01:20:24.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.587+0000 7fa1f7fff700 1 -- 172.21.15.7:0/2982496147 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2040ccb80 con 0x55ab32820470 2024-06-10T01:20:24.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.588+0000 7fa21eeaab80 1 -- 172.21.15.7:0/2982496147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab32820470 msgr2=0x55ab32826ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:24.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.588+0000 7fa21eeaab80 1 --2- 172.21.15.7:0/2982496147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab32820470 0x55ab32826ad0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa20400ef90 tx=0x7fa2040999c0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.588+0000 7fa21eeaab80 1 -- 172.21.15.7:0/2982496147 shutdown_connections 2024-06-10T01:20:24.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.588+0000 7fa21eeaab80 1 --2- 172.21.15.7:0/2982496147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab32820470 0x55ab32826ad0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.588+0000 7fa21eeaab80 1 --2- 172.21.15.7:0/2982496147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab3281a670 0x55ab32830460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.588+0000 7fa21eeaab80 1 --2- 172.21.15.7:0/2982496147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3281b030 0x55ab32815960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.588+0000 7fa21eeaab80 1 -- 172.21.15.7:0/2982496147 >> 172.21.15.7:0/2982496147 conn(0x55ab32743080 msgr2=0x55ab32815250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:24.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.589+0000 7fa21eeaab80 1 -- 172.21.15.7:0/2982496147 shutdown_connections 2024-06-10T01:20:24.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.589+0000 7fa21eeaab80 1 -- 172.21.15.7:0/2982496147 wait complete. 2024-06-10T01:20:24.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.589+0000 7fa21eeaab80 1 -- start start 2024-06-10T01:20:24.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.589+0000 7fa21eeaab80 1 Processor -- start 2024-06-10T01:20:24.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.590+0000 7fa21eeaab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab3281a670 0x55ab3283b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:24.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.590+0000 7fa21eeaab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3281b030 0x55ab3283bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:24.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.590+0000 7fa21eeaab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab32820470 0x55ab3282b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:24.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.590+0000 7fa21eeaab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c1aa470 con 0x55ab3281a670 2024-06-10T01:20:24.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.590+0000 7fa21eeaab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c1ab8e0 con 0x55ab32820470 2024-06-10T01:20:24.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.590+0000 7fa21eeaab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c1aab40 con 0x55ab3281b030 2024-06-10T01:20:24.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.590+0000 7fa21eeaab80 10 client.0 _ll_register_callbacks cb 0x55ab326fd5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:20:24.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.590+0000 7fa2123c3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab3281a670 0x55ab3283b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:24.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.590+0000 7fa2123c3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab3281a670 0x55ab3283b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53860/0 (socket says 172.21.15.7:53860) 2024-06-10T01:20:24.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.590+0000 7fa2123c3700 1 -- 172.21.15.7:0/3935401694 learned_addr learned my addr 172.21.15.7:0/3935401694 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:24.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.590+0000 7fa211bc2700 1 --2- 172.21.15.7:0/3935401694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3281b030 0x55ab3283bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:24.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.590+0000 7fa212bc4700 1 --2- 172.21.15.7:0/3935401694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab32820470 0x55ab3282b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:24.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.590+0000 7fa211bc2700 1 -- 172.21.15.7:0/3935401694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab32820470 msgr2=0x55ab3282b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:24.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.590+0000 7fa211bc2700 1 --2- 172.21.15.7:0/3935401694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab32820470 0x55ab3282b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.590+0000 7fa211bc2700 1 -- 172.21.15.7:0/3935401694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab3281a670 msgr2=0x55ab3283b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:24.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.590+0000 7fa211bc2700 1 --2- 172.21.15.7:0/3935401694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab3281a670 0x55ab3283b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.590+0000 7fa211bc2700 1 -- 172.21.15.7:0/3935401694 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab3282baf0 con 0x55ab3281b030 2024-06-10T01:20:24.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.591+0000 7fa211bc2700 1 --2- 172.21.15.7:0/3935401694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3281b030 0x55ab3283bcb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00b2a0 tx=0x7fa1fc005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1756be3c226c0aef server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:24.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.591+0000 7fa1f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:20:24.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.591+0000 7fa1f5ffb700 1 -- 172.21.15.7:0/3935401694 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc032070 con 0x55ab3281b030 2024-06-10T01:20:24.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.591+0000 7fa1f5ffb700 1 -- 172.21.15.7:0/3935401694 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa1fc004bd0 con 0x55ab3281b030 2024-06-10T01:20:24.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.591+0000 7fa21eeaab80 1 -- 172.21.15.7:0/3935401694 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa20409f860 con 0x55ab3281b030 2024-06-10T01:20:24.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.591+0000 7fa1f5ffb700 1 -- 172.21.15.7:0/3935401694 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa1fc027040 con 0x55ab3281b030 2024-06-10T01:20:24.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.591+0000 7fa21eeaab80 10 client.28393 fetch_fsmap learned FSMap version 44 2024-06-10T01:20:24.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.591+0000 7fa21eeaab80 1 -- 172.21.15.7:0/3935401694 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ab3282ce00 con 0x55ab3281b030 2024-06-10T01:20:24.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.592+0000 7fa1f5ffb700 1 -- 172.21.15.7:0/3935401694 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa20409f860 con 0x55ab3281b030 2024-06-10T01:20:24.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.592+0000 7fa1f5ffb700 1 -- 172.21.15.7:0/3935401694 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fa1fc02d040 con 0x55ab3281b030 2024-06-10T01:20:24.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.592+0000 7fa21eeaab80 10 client.28393 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:20:24.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.592+0000 7fa21eeaab80 10 client.28393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:20:24.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.592+0000 7fa21eeaab80 20 client.28393 populate_metadata read hostname 'smithi007' 2024-06-10T01:20:24.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.592+0000 7fa21eeaab80 1 --2- 172.21.15.7:0/3935401694 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ab3289f350 0x55ab328a1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:24.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.592+0000 7fa21eeaab80 4 client.28393 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:20:24.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.592+0000 7fa21eeaab80 1 -- 172.21.15.7:0/3935401694 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ab32722f50 con 0x55ab3289f350 2024-06-10T01:20:24.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.593+0000 7fa2123c3700 1 --2- 172.21.15.7:0/3935401694 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ab3289f350 0x55ab328a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:24.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.593+0000 7fa1f5ffb700 10 client.28393 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:24.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.593+0000 7fa2123c3700 1 --2- 172.21.15.7:0/3935401694 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ab3289f350 0x55ab328a1740 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4a4c00f85eb400f server_cookie=ff2f96001d87758a in_seq=0 out_seq=0 2024-06-10T01:20:24.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.598+0000 7fa1f5ffb700 1 -- 172.21.15.7:0/3935401694 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ab32722f50 con 0x55ab3289f350 2024-06-10T01:20:24.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.598+0000 7fa1f5ffb700 10 client.28393 handle_command_reply: tid=0 2024-06-10T01:20:24.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.598+0000 7fa21eeaab80 1 client.28393 shutdown 2024-06-10T01:20:24.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.599+0000 7fa21eeaab80 1 -- 172.21.15.7:0/3935401694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3281b030 msgr2=0x55ab3283bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:24.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.599+0000 7fa21eeaab80 1 --2- 172.21.15.7:0/3935401694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3281b030 0x55ab3283bcb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00b2a0 tx=0x7fa1fc005770 comp rx=0 tx=0).stop 2024-06-10T01:20:24.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.599+0000 7fa21eeaab80 1 -- 172.21.15.7:0/3935401694 shutdown_connections 2024-06-10T01:20:24.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.599+0000 7fa21eeaab80 1 --2- 172.21.15.7:0/3935401694 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ab3289f350 0x55ab328a1740 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.599+0000 7fa21eeaab80 1 --2- 172.21.15.7:0/3935401694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab32820470 0x55ab3282b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.599+0000 7fa21eeaab80 1 --2- 172.21.15.7:0/3935401694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ab3281a670 0x55ab3283b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.599+0000 7fa21eeaab80 1 --2- 172.21.15.7:0/3935401694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ab3281b030 0x55ab3283bcb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.599+0000 7fa1f5ffb700 0 client.28393 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:24.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.599+0000 7fa21eeaab80 1 -- 172.21.15.7:0/3935401694 >> 172.21.15.7:0/3935401694 conn(0x55ab32743080 msgr2=0x55ab32815ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:24.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.599+0000 7fa21eeaab80 1 -- 172.21.15.7:0/3935401694 shutdown_connections 2024-06-10T01:20:24.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.599+0000 7fa21eeaab80 1 -- 172.21.15.7:0/3935401694 wait complete. 2024-06-10T01:20:24.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.600+0000 7fa21eeaab80 20 client.28393 trim_cache size 0 max 16384 2024-06-10T01:20:24.594 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:20:24.594 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T01:20:24.594 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-10T01:20:24.594 INFO:teuthology.orchestra.run.smithi007.stdout: "8217e9d5-cd73-478c-ad98-1bbf6eada10c": { 2024-06-10T01:20:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-10T01:20:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "8217e9d5-cd73-478c-ad98-1bbf6eada10c", 2024-06-10T01:20:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-10T01:20:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:20:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:20:24.595 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:20:24.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.601+0000 7fa214627700 1 -- 172.21.15.7:0/3537861588 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa1f80a2c20 msgr2=0x7fa1f80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:24.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.601+0000 7fa214627700 1 --2- 172.21.15.7:0/3537861588 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa1f80a2c20 0x7fa1f80a50e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa208009fd0 tx=0x7fa208009260 comp rx=0 tx=0).stop 2024-06-10T01:20:24.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.601+0000 7fa214627700 1 -- 172.21.15.7:0/3537861588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa20c1084f0 msgr2=0x7fa20c198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:24.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.601+0000 7fa214627700 1 --2- 172.21.15.7:0/3537861588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa20c1084f0 0x7fa20c198830 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa20400efc0 tx=0x7fa20400c3a0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.601+0000 7fa214627700 1 -- 172.21.15.7:0/3537861588 shutdown_connections 2024-06-10T01:20:24.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.601+0000 7fa214627700 1 --2- 172.21.15.7:0/3537861588 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa1f80a2c20 0x7fa1f80a50e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.601+0000 7fa214627700 1 --2- 172.21.15.7:0/3537861588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa20c1084f0 0x7fa20c198830 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.601+0000 7fa214627700 1 --2- 172.21.15.7:0/3537861588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa20c06aa00 0x7fa20c19e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.601+0000 7fa214627700 1 --2- 172.21.15.7:0/3537861588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa20c06b3c0 0x7fa20c19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:24.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.601+0000 7fa214627700 1 -- 172.21.15.7:0/3537861588 >> 172.21.15.7:0/3537861588 conn(0x7fa20c078300 msgr2=0x7fa20c0ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:24.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.602+0000 7fa214627700 1 -- 172.21.15.7:0/3537861588 shutdown_connections 2024-06-10T01:20:24.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:24.602+0000 7fa214627700 1 -- 172.21.15.7:0/3537861588 wait complete. 2024-06-10T01:20:25.471 INFO:tasks.workunit.client.0.smithi007.stdout: 10 15591 16.40 MB/sec warmup 62 sec latency 99.442 ms 2024-06-10T01:20:25.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:25 smithi175 ceph-mon[36636]: pgmap v864: 129 pgs: 129 active+clean; 359 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 4.3 MiB/s wr, 217 op/s 2024-06-10T01:20:25.732 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:25 smithi007 ceph-mon[32398]: pgmap v864: 129 pgs: 129 active+clean; 359 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 4.3 MiB/s wr, 217 op/s 2024-06-10T01:20:25.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:25 smithi078 ceph-mon[36683]: pgmap v864: 129 pgs: 129 active+clean; 359 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 4.3 MiB/s wr, 217 op/s 2024-06-10T01:20:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.965+0000 7f3b76638640 1 -- 172.21.15.7:0/1477278321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b701024b0 msgr2=0x7f3b70108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.965+0000 7f3b76638640 1 --2- 172.21.15.7:0/1477278321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b701024b0 0x7f3b70108d40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3b68009be0 tx=0x7f3b6802a6c0 comp rx=0 tx=0).stop 2024-06-10T01:20:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.965+0000 7f3b76638640 1 -- 172.21.15.7:0/1477278321 shutdown_connections 2024-06-10T01:20:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.965+0000 7f3b76638640 1 --2- 172.21.15.7:0/1477278321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b701024b0 0x7f3b70108d40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.965+0000 7f3b76638640 1 --2- 172.21.15.7:0/1477278321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b70101af0 0x7f3b70101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.965+0000 7f3b76638640 1 --2- 172.21.15.7:0/1477278321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b70100900 0x7f3b70100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.965+0000 7f3b76638640 1 -- 172.21.15.7:0/1477278321 >> 172.21.15.7:0/1477278321 conn(0x7f3b700fbe70 msgr2=0x7f3b700fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.965+0000 7f3b76638640 1 -- 172.21.15.7:0/1477278321 shutdown_connections 2024-06-10T01:20:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.965+0000 7f3b76638640 1 -- 172.21.15.7:0/1477278321 wait complete. 2024-06-10T01:20:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.966+0000 7f3b76638640 1 Processor -- start 2024-06-10T01:20:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.966+0000 7f3b76638640 1 -- start start 2024-06-10T01:20:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.966+0000 7f3b76638640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b70100900 0x7f3b7010b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.966+0000 7f3b76638640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b70101af0 0x7f3b701098c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.966+0000 7f3b76638640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b701024b0 0x7f3b70109e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.966+0000 7f3b76638640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7010d640 con 0x7f3b70100900 2024-06-10T01:20:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.966+0000 7f3b76638640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7010d4c0 con 0x7f3b701024b0 2024-06-10T01:20:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.966+0000 7f3b76638640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7010d7c0 con 0x7f3b70101af0 2024-06-10T01:20:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.966+0000 7f3b6ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b70100900 0x7f3b7010b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.966+0000 7f3b74bd5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b701024b0 0x7f3b70109e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.966+0000 7f3b6ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b70100900 0x7f3b7010b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53880/0 (socket says 172.21.15.7:53880) 2024-06-10T01:20:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.966+0000 7f3b6ffff640 1 -- 172.21.15.7:0/2956790549 learned_addr learned my addr 172.21.15.7:0/2956790549 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.966+0000 7f3b67fff640 1 --2- 172.21.15.7:0/2956790549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b70101af0 0x7f3b701098c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.966+0000 7f3b6ffff640 1 -- 172.21.15.7:0/2956790549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b70101af0 msgr2=0x7f3b701098c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.966+0000 7f3b6ffff640 1 --2- 172.21.15.7:0/2956790549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b70101af0 0x7f3b701098c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.966+0000 7f3b6ffff640 1 -- 172.21.15.7:0/2956790549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b701024b0 msgr2=0x7f3b70109e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.966+0000 7f3b6ffff640 1 --2- 172.21.15.7:0/2956790549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b701024b0 0x7f3b70109e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.966+0000 7f3b6ffff640 1 -- 172.21.15.7:0/2956790549 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b7010a690 con 0x7f3b70100900 2024-06-10T01:20:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.967+0000 7f3b6ffff640 1 --2- 172.21.15.7:0/2956790549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b70100900 0x7f3b7010b280 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f3b6000e9c0 tx=0x7f3b6000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.967+0000 7f3b6dffb640 1 -- 172.21.15.7:0/2956790549 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6000cd80 con 0x7f3b70100900 2024-06-10T01:20:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.967+0000 7f3b6dffb640 1 -- 172.21.15.7:0/2956790549 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b600035f0 con 0x7f3b70100900 2024-06-10T01:20:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.967+0000 7f3b76638640 1 -- 172.21.15.7:0/2956790549 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b701a1dc0 con 0x7f3b70100900 2024-06-10T01:20:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.967+0000 7f3b6dffb640 1 -- 172.21.15.7:0/2956790549 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b60010690 con 0x7f3b70100900 2024-06-10T01:20:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.967+0000 7f3b76638640 1 -- 172.21.15.7:0/2956790549 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b701a2220 con 0x7f3b70100900 2024-06-10T01:20:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.970+0000 7f3b7941e740 1 Processor -- start 2024-06-10T01:20:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.970+0000 7f3b6dffb640 1 -- 172.21.15.7:0/2956790549 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f3b600037a0 con 0x7f3b70100900 2024-06-10T01:20:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.970+0000 7f3b7941e740 1 -- start start 2024-06-10T01:20:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.970+0000 7f3b7941e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b1d35010 0x5632b1d35430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.970+0000 7f3b7941e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632b1d35a00 0x5632b1d3a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.970+0000 7f3b7941e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632b1d3ab70 0x5632b1d3d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.970+0000 7f3b7941e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5632b1c984f0 con 0x5632b1d3ab70 2024-06-10T01:20:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.970+0000 7f3b7941e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5632b1c36870 con 0x5632b1d35010 2024-06-10T01:20:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.970+0000 7f3b7941e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5632b1c944b0 con 0x5632b1d35a00 2024-06-10T01:20:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.970+0000 7f3b74bd5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b1d35010 0x5632b1d35430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.970+0000 7f3b74bd5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b1d35010 0x5632b1d35430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:36450/0 (socket says 172.21.15.7:36450) 2024-06-10T01:20:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.970+0000 7f3b74bd5640 1 -- 172.21.15.7:0/1088592683 learned_addr learned my addr 172.21.15.7:0/1088592683 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.970+0000 7f3b67fff640 1 --2- 172.21.15.7:0/1088592683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632b1d3ab70 0x5632b1d3d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.970+0000 7f3b6ffff640 1 --2- 172.21.15.7:0/1088592683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632b1d35a00 0x5632b1d3a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.971+0000 7f3b6dffb640 1 --2- 172.21.15.7:0/2956790549 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3b5c0a2c80 0x7f3b5c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.971+0000 7f3b74bd5640 1 --2- 172.21.15.7:0/2956790549 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3b5c0a2c80 0x7f3b5c0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.971+0000 7f3b6dffb640 1 -- 172.21.15.7:0/2956790549 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f3b600cf7e0 con 0x7f3b70100900 2024-06-10T01:20:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.971+0000 7f3b67fff640 1 -- 172.21.15.7:0/1088592683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632b1d35a00 msgr2=0x5632b1d3a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.971+0000 7f3b67fff640 1 --2- 172.21.15.7:0/1088592683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632b1d35a00 0x5632b1d3a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.971+0000 7f3b67fff640 1 -- 172.21.15.7:0/1088592683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b1d35010 msgr2=0x5632b1d35430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.971+0000 7f3b67fff640 1 --2- 172.21.15.7:0/1088592683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b1d35010 0x5632b1d35430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.971+0000 7f3b67fff640 1 -- 172.21.15.7:0/1088592683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632b1c529b0 con 0x5632b1d3ab70 2024-06-10T01:20:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.971+0000 7f3b74bd5640 1 --2- 172.21.15.7:0/1088592683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b1d35010 0x5632b1d35430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:20:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.971+0000 7f3b67fff640 1 --2- 172.21.15.7:0/1088592683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632b1d3ab70 0x5632b1d3d000 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f3b58005fd0 tx=0x7f3b58009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f69509678e8e8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.971+0000 7f3b74bd5640 1 --2- 172.21.15.7:0/2956790549 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3b5c0a2c80 0x7f3b5c0a5140 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x5632b1d36260 tx=0x7f3b6802a6c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.971+0000 7f3b667fc640 1 -- 172.21.15.7:0/1088592683 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b58011da0 con 0x5632b1d3ab70 2024-06-10T01:20:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.971+0000 7f3b667fc640 1 -- 172.21.15.7:0/1088592683 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b58005500 con 0x5632b1d3ab70 2024-06-10T01:20:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.972+0000 7f3b667fc640 1 -- 172.21.15.7:0/1088592683 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b580126b0 con 0x5632b1d3ab70 2024-06-10T01:20:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.972+0000 7f3b7941e740 1 -- 172.21.15.7:0/1088592683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632b1d3ab70 msgr2=0x5632b1d3d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.972+0000 7f3b7941e740 1 --2- 172.21.15.7:0/1088592683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632b1d3ab70 0x5632b1d3d000 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f3b58005fd0 tx=0x7f3b58009f40 comp rx=0 tx=0).stop 2024-06-10T01:20:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.972+0000 7f3b7941e740 1 -- 172.21.15.7:0/1088592683 shutdown_connections 2024-06-10T01:20:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.972+0000 7f3b7941e740 1 --2- 172.21.15.7:0/1088592683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632b1d3ab70 0x5632b1d3d000 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.972+0000 7f3b7941e740 1 --2- 172.21.15.7:0/1088592683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632b1d35a00 0x5632b1d3a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.972+0000 7f3b7941e740 1 --2- 172.21.15.7:0/1088592683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b1d35010 0x5632b1d35430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.972+0000 7f3b7941e740 1 -- 172.21.15.7:0/1088592683 >> 172.21.15.7:0/1088592683 conn(0x5632b1d30680 msgr2=0x5632b1d30a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.972+0000 7f3b7941e740 1 -- 172.21.15.7:0/1088592683 shutdown_connections 2024-06-10T01:20:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.973+0000 7f3b7941e740 1 -- 172.21.15.7:0/1088592683 wait complete. 2024-06-10T01:20:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.973+0000 7f3b7941e740 1 -- start start 2024-06-10T01:20:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.973+0000 7f3b7941e740 1 Processor -- start 2024-06-10T01:20:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.973+0000 7f3b7941e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632b1d35010 0x5632b1d3f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.973+0000 7f3b7941e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b1d35a00 0x5632b1d3f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.973+0000 7f3b7941e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632b1d3ab70 0x5632b1d467d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.973+0000 7f3b7941e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5632b1d3db20 con 0x5632b1d3ab70 2024-06-10T01:20:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.973+0000 7f3b7941e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5632b1c21ae0 con 0x5632b1d35a00 2024-06-10T01:20:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.973+0000 7f3b7941e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5632b1d3dca0 con 0x5632b1d35010 2024-06-10T01:20:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.973+0000 7f3b7941e740 10 client.0 _ll_register_callbacks cb 0x5632b1c5a9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:20:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.973+0000 7f3b67fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b1d35a00 0x5632b1d3f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.973+0000 7f3b74bd5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632b1d3ab70 0x5632b1d467d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.973+0000 7f3b67fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b1d35a00 0x5632b1d3f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:36464/0 (socket says 172.21.15.7:36464) 2024-06-10T01:20:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.973+0000 7f3b67fff640 1 -- 172.21.15.7:0/3004106637 learned_addr learned my addr 172.21.15.7:0/3004106637 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.974+0000 7f3b74bd5640 1 -- 172.21.15.7:0/3004106637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632b1d35010 msgr2=0x5632b1d3f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.974+0000 7f3b74bd5640 1 --2- 172.21.15.7:0/3004106637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632b1d35010 0x5632b1d3f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.974+0000 7f3b74bd5640 1 -- 172.21.15.7:0/3004106637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b1d35a00 msgr2=0x5632b1d3f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.974+0000 7f3b74bd5640 1 --2- 172.21.15.7:0/3004106637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b1d35a00 0x5632b1d3f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.974+0000 7f3b74bd5640 1 -- 172.21.15.7:0/3004106637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632b19ac210 con 0x5632b1d3ab70 2024-06-10T01:20:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.974+0000 7f3b67fff640 1 --2- 172.21.15.7:0/3004106637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b1d35a00 0x5632b1d3f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:20:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.974+0000 7f3b74bd5640 1 --2- 172.21.15.7:0/3004106637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632b1d3ab70 0x5632b1d467d0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f3b68005cc0 tx=0x7f3b68009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4ff58af9c4700e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.974+0000 7f3b4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:20:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.974+0000 7f3b4f7fe640 1 -- 172.21.15.7:0/3004106637 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b68050020 con 0x5632b1d3ab70 2024-06-10T01:20:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.974+0000 7f3b4f7fe640 1 -- 172.21.15.7:0/3004106637 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f3b68004240 con 0x5632b1d3ab70 2024-06-10T01:20:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.974+0000 7f3b7941e740 1 -- 172.21.15.7:0/3004106637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b58005500 con 0x5632b1d3ab70 2024-06-10T01:20:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.974+0000 7f3b4f7fe640 1 -- 172.21.15.7:0/3004106637 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f3b6805b050 con 0x5632b1d3ab70 2024-06-10T01:20:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.975+0000 7f3b7941e740 10 client.19107 fetch_fsmap learned FSMap version 44 2024-06-10T01:20:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.975+0000 7f3b7941e740 1 -- 172.21.15.7:0/3004106637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5632b1a78620 con 0x5632b1d3ab70 2024-06-10T01:20:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.975+0000 7f3b4f7fe640 1 -- 172.21.15.7:0/3004106637 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b58005500 con 0x5632b1d3ab70 2024-06-10T01:20:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.975+0000 7f3b4f7fe640 1 -- 172.21.15.7:0/3004106637 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f3b6804b030 con 0x5632b1d3ab70 2024-06-10T01:20:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.975+0000 7f3b7941e740 10 client.19107 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:20:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.975+0000 7f3b7941e740 10 client.19107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:20:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.975+0000 7f3b7941e740 20 client.19107 populate_metadata read hostname 'smithi007' 2024-06-10T01:20:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.975+0000 7f3b7941e740 1 --2- 172.21.15.7:0/3004106637 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5632b1d4af80 0x5632b1d4d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.975+0000 7f3b7941e740 4 client.19107 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:20:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.975+0000 7f3b7941e740 1 -- 172.21.15.7:0/3004106637 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632b1cabf10 con 0x5632b1d4af80 2024-06-10T01:20:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.975+0000 7f3b6ffff640 1 --2- 172.21.15.7:0/3004106637 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5632b1d4af80 0x5632b1d4d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.976+0000 7f3b4f7fe640 10 client.19107 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.976+0000 7f3b6ffff640 1 --2- 172.21.15.7:0/3004106637 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5632b1d4af80 0x5632b1d4d3f0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea40a75dcc58fbf2 server_cookie=c73e1236db46841d in_seq=0 out_seq=0 2024-06-10T01:20:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.978+0000 7f3b4f7fe640 1 -- 172.21.15.7:0/3004106637 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5632b1cabf10 con 0x5632b1d4af80 2024-06-10T01:20:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.978+0000 7f3b4f7fe640 10 client.19107 handle_command_reply: tid=0 2024-06-10T01:20:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.978+0000 7f3b7941e740 1 client.19107 shutdown 2024-06-10T01:20:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.978+0000 7f3b7941e740 1 -- 172.21.15.7:0/3004106637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632b1d3ab70 msgr2=0x5632b1d467d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.978+0000 7f3b7941e740 1 --2- 172.21.15.7:0/3004106637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632b1d3ab70 0x5632b1d467d0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f3b68005cc0 tx=0x7f3b68009500 comp rx=0 tx=0).stop 2024-06-10T01:20:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.978+0000 7f3b7941e740 1 -- 172.21.15.7:0/3004106637 shutdown_connections 2024-06-10T01:20:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.978+0000 7f3b7941e740 1 --2- 172.21.15.7:0/3004106637 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5632b1d4af80 0x5632b1d4d3f0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.978+0000 7f3b7941e740 1 --2- 172.21.15.7:0/3004106637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632b1d3ab70 0x5632b1d467d0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.978+0000 7f3b7941e740 1 --2- 172.21.15.7:0/3004106637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b1d35a00 0x5632b1d3f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.978+0000 7f3b7941e740 1 --2- 172.21.15.7:0/3004106637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632b1d35010 0x5632b1d3f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.978+0000 7f3b7941e740 1 -- 172.21.15.7:0/3004106637 >> 172.21.15.7:0/3004106637 conn(0x5632b1d30680 msgr2=0x5632b1d2f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.978+0000 7f3b4f7fe640 0 client.19107 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.979+0000 7f3b7941e740 1 -- 172.21.15.7:0/3004106637 shutdown_connections 2024-06-10T01:20:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.979+0000 7f3b7941e740 1 -- 172.21.15.7:0/3004106637 wait complete. 2024-06-10T01:20:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.979+0000 7f3b7941e740 20 client.19107 trim_cache size 0 max 16384 2024-06-10T01:20:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.996+0000 7f3b7941e740 1 Processor -- start 2024-06-10T01:20:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.996+0000 7f3b7941e740 1 -- start start 2024-06-10T01:20:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.996+0000 7f3b7941e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b1d35010 0x5632b1d36be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:25.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.996+0000 7f3b7941e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632b1d35a00 0x5632b1d46790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:25.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.996+0000 7f3b7941e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632b1d3ab70 0x5632b1d37120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:25.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.996+0000 7f3b7941e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3b701af140 con 0x5632b1d35a00 2024-06-10T01:20:25.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.996+0000 7f3b7941e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b701ac980 con 0x5632b1d35010 2024-06-10T01:20:25.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.996+0000 7f3b7941e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3b701aa7d0 con 0x5632b1d3ab70 2024-06-10T01:20:25.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.997+0000 7f3b67fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632b1d35a00 0x5632b1d46790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:25.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.997+0000 7f3b6ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b1d35010 0x5632b1d36be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:25.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.997+0000 7f3b67fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632b1d35a00 0x5632b1d46790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53916/0 (socket says 172.21.15.7:53916) 2024-06-10T01:20:25.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.997+0000 7f3b67fff640 1 -- 172.21.15.7:0/100854814 learned_addr learned my addr 172.21.15.7:0/100854814 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:25.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.997+0000 7f3b74bd5640 1 --2- 172.21.15.7:0/100854814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632b1d3ab70 0x5632b1d37120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:25.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.997+0000 7f3b67fff640 1 -- 172.21.15.7:0/100854814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632b1d3ab70 msgr2=0x5632b1d37120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:25.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.997+0000 7f3b67fff640 1 --2- 172.21.15.7:0/100854814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632b1d3ab70 0x5632b1d37120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:25.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.997+0000 7f3b67fff640 1 -- 172.21.15.7:0/100854814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b1d35010 msgr2=0x5632b1d36be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:25.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.997+0000 7f3b67fff640 1 --2- 172.21.15.7:0/100854814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b1d35010 0x5632b1d36be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:25.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.997+0000 7f3b67fff640 1 -- 172.21.15.7:0/100854814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632b1d304b0 con 0x5632b1d35a00 2024-06-10T01:20:25.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.997+0000 7f3b67fff640 1 --2- 172.21.15.7:0/100854814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632b1d35a00 0x5632b1d46790 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f3b58009ac0 tx=0x7f3b58004cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a29d8aff9fd1c8d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.998+0000 7f3b65ffb640 1 -- 172.21.15.7:0/100854814 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b58029070 con 0x5632b1d35a00 2024-06-10T01:20:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.998+0000 7f3b65ffb640 1 -- 172.21.15.7:0/100854814 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b58009520 con 0x5632b1d35a00 2024-06-10T01:20:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.998+0000 7f3b65ffb640 1 -- 172.21.15.7:0/100854814 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5801f040 con 0x5632b1d35a00 2024-06-10T01:20:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.999+0000 7f3b7941e740 1 -- 172.21.15.7:0/100854814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632b1d35a00 msgr2=0x5632b1d46790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.999+0000 7f3b7941e740 1 --2- 172.21.15.7:0/100854814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632b1d35a00 0x5632b1d46790 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f3b58009ac0 tx=0x7f3b58004cd0 comp rx=0 tx=0).stop 2024-06-10T01:20:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.999+0000 7f3b7941e740 1 -- 172.21.15.7:0/100854814 shutdown_connections 2024-06-10T01:20:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.999+0000 7f3b7941e740 1 --2- 172.21.15.7:0/100854814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632b1d3ab70 0x5632b1d37120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.999+0000 7f3b7941e740 1 --2- 172.21.15.7:0/100854814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632b1d35a00 0x5632b1d46790 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.999+0000 7f3b7941e740 1 --2- 172.21.15.7:0/100854814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b1d35010 0x5632b1d36be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.999+0000 7f3b7941e740 1 -- 172.21.15.7:0/100854814 >> 172.21.15.7:0/100854814 conn(0x5632b1d30680 msgr2=0x5632b1d3f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.999+0000 7f3b7941e740 1 -- 172.21.15.7:0/100854814 shutdown_connections 2024-06-10T01:20:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.999+0000 7f3b7941e740 1 -- 172.21.15.7:0/100854814 wait complete. 2024-06-10T01:20:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:25.999+0000 7f3b7941e740 1 -- start start 2024-06-10T01:20:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.000+0000 7f3b7941e740 1 Processor -- start 2024-06-10T01:20:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.000+0000 7f3b7941e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632b1d35010 0x5632b1d43360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.000+0000 7f3b7941e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632b1d35a00 0x5632b1d438a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.000+0000 7f3b6ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632b1d35010 0x5632b1d43360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.000+0000 7f3b6ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632b1d35010 0x5632b1d43360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53926/0 (socket says 172.21.15.7:53926) 2024-06-10T01:20:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.000+0000 7f3b67fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632b1d35a00 0x5632b1d438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.000+0000 7f3b67fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632b1d35a00 0x5632b1d438a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:41256/0 (socket says 172.21.15.7:41256) 2024-06-10T01:20:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.000+0000 7f3b6ffff640 1 -- 172.21.15.7:0/1013150204 learned_addr learned my addr 172.21.15.7:0/1013150204 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.001+0000 7f3b7941e740 1 --2- 172.21.15.7:0/1013150204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b1d3ab70 0x5632b1db3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.001+0000 7f3b7941e740 1 -- 172.21.15.7:0/1013150204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3b701aae40 con 0x5632b1d35010 2024-06-10T01:20:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.001+0000 7f3b7941e740 1 -- 172.21.15.7:0/1013150204 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b701ac2b0 con 0x5632b1d3ab70 2024-06-10T01:20:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.001+0000 7f3b7941e740 1 -- 172.21.15.7:0/1013150204 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3b701ab510 con 0x5632b1d35a00 2024-06-10T01:20:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.001+0000 7f3b7941e740 10 client.0 _ll_register_callbacks cb 0x7f3b60005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:20:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.001+0000 7f3b74bd5640 1 --2- 172.21.15.7:0/1013150204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b1d3ab70 0x5632b1db3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.001+0000 7f3b74bd5640 1 -- 172.21.15.7:0/1013150204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632b1d35a00 msgr2=0x5632b1d438a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.001+0000 7f3b74bd5640 1 --2- 172.21.15.7:0/1013150204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632b1d35a00 0x5632b1d438a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.001+0000 7f3b74bd5640 1 -- 172.21.15.7:0/1013150204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632b1d35010 msgr2=0x5632b1d43360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.001+0000 7f3b74bd5640 1 --2- 172.21.15.7:0/1013150204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632b1d35010 0x5632b1d43360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.001+0000 7f3b74bd5640 1 -- 172.21.15.7:0/1013150204 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632b1d366d0 con 0x5632b1d3ab70 2024-06-10T01:20:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.001+0000 7f3b74bd5640 1 --2- 172.21.15.7:0/1013150204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b1d3ab70 0x5632b1db3cc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3b6802abe0 tx=0x7f3b68009a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7da7986c15735e00 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.001+0000 7f3b4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:20:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.002+0000 7f3b4f7fe640 1 -- 172.21.15.7:0/1013150204 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b68064060 con 0x5632b1d3ab70 2024-06-10T01:20:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.002+0000 7f3b4f7fe640 1 -- 172.21.15.7:0/1013150204 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f3b68031640 con 0x5632b1d3ab70 2024-06-10T01:20:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.002+0000 7f3b7941e740 1 -- 172.21.15.7:0/1013150204 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b58009520 con 0x5632b1d3ab70 2024-06-10T01:20:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.002+0000 7f3b4f7fe640 1 -- 172.21.15.7:0/1013150204 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f3b6806b050 con 0x5632b1d3ab70 2024-06-10T01:20:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.002+0000 7f3b7941e740 10 client.28388 fetch_fsmap learned FSMap version 44 2024-06-10T01:20:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.002+0000 7f3b7941e740 1 -- 172.21.15.7:0/1013150204 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5632b1db5030 con 0x5632b1d3ab70 2024-06-10T01:20:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.002+0000 7f3b4f7fe640 1 -- 172.21.15.7:0/1013150204 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b68037040 con 0x5632b1d3ab70 2024-06-10T01:20:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.003+0000 7f3b4f7fe640 1 -- 172.21.15.7:0/1013150204 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f3b6804b070 con 0x5632b1d3ab70 2024-06-10T01:20:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.003+0000 7f3b7941e740 10 client.28388 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:20:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.003+0000 7f3b7941e740 10 client.28388 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:20:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.003+0000 7f3b7941e740 20 client.28388 populate_metadata read hostname 'smithi007' 2024-06-10T01:20:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.003+0000 7f3b7941e740 1 --2- 172.21.15.7:0/1013150204 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5632b1db5a60 0x5632b1db7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.003+0000 7f3b7941e740 4 client.28388 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:20:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.003+0000 7f3b7941e740 1 -- 172.21.15.7:0/1013150204 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f3b58009520 con 0x5632b1db5a60 2024-06-10T01:20:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.003+0000 7f3b6ffff640 1 --2- 172.21.15.7:0/1013150204 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5632b1db5a60 0x5632b1db7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.003+0000 7f3b4f7fe640 10 client.28388 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.003+0000 7f3b6ffff640 1 --2- 172.21.15.7:0/1013150204 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5632b1db5a60 0x5632b1db7e80 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=691e4538e02de18c server_cookie=c6f263e20b30e1e3 in_seq=0 out_seq=0 2024-06-10T01:20:26.232 INFO:tasks.cephfs.filesystem:scrub status for tag:8217e9d5-cd73-478c-ad98-1bbf6eada10c - {'path': '/', 'tag': '8217e9d5-cd73-478c-ad98-1bbf6eada10c', 'options': 'recursive,force'} 2024-06-10T01:20:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.238+0000 7f3b4f7fe640 1 -- 172.21.15.7:0/1013150204 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f3b58009520 con 0x5632b1db5a60 2024-06-10T01:20:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.238+0000 7f3b4f7fe640 10 client.28388 handle_command_reply: tid=0 2024-06-10T01:20:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.238+0000 7f3b7941e740 1 client.28388 shutdown 2024-06-10T01:20:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.238+0000 7f3b7941e740 1 -- 172.21.15.7:0/1013150204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b1d3ab70 msgr2=0x5632b1db3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.238+0000 7f3b7941e740 1 --2- 172.21.15.7:0/1013150204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b1d3ab70 0x5632b1db3cc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3b6802abe0 tx=0x7f3b68009a60 comp rx=0 tx=0).stop 2024-06-10T01:20:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.238+0000 7f3b7941e740 1 -- 172.21.15.7:0/1013150204 shutdown_connections 2024-06-10T01:20:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.238+0000 7f3b7941e740 1 --2- 172.21.15.7:0/1013150204 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5632b1db5a60 0x5632b1db7e80 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.238+0000 7f3b7941e740 1 --2- 172.21.15.7:0/1013150204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b1d3ab70 0x5632b1db3cc0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.238+0000 7f3b7941e740 1 --2- 172.21.15.7:0/1013150204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632b1d35a00 0x5632b1d438a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.238+0000 7f3b4f7fe640 0 client.28388 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.238+0000 7f3b7941e740 1 --2- 172.21.15.7:0/1013150204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632b1d35010 0x5632b1d43360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.238+0000 7f3b7941e740 1 -- 172.21.15.7:0/1013150204 >> 172.21.15.7:0/1013150204 conn(0x5632b1d30680 msgr2=0x5632b1d36be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.239+0000 7f3b7941e740 1 -- 172.21.15.7:0/1013150204 shutdown_connections 2024-06-10T01:20:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.239+0000 7f3b7941e740 1 -- 172.21.15.7:0/1013150204 wait complete. 2024-06-10T01:20:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.239+0000 7f3b7941e740 20 client.28388 trim_cache size 0 max 16384 2024-06-10T01:20:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:20:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:20:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:20:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.239+0000 7f3b76638640 1 -- 172.21.15.7:0/2956790549 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3b5c0a2c80 msgr2=0x7f3b5c0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.239+0000 7f3b76638640 1 --2- 172.21.15.7:0/2956790549 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3b5c0a2c80 0x7f3b5c0a5140 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x5632b1d36260 tx=0x7f3b6802a6c0 comp rx=0 tx=0).stop 2024-06-10T01:20:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.239+0000 7f3b76638640 1 -- 172.21.15.7:0/2956790549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b70100900 msgr2=0x7f3b7010b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.239+0000 7f3b76638640 1 --2- 172.21.15.7:0/2956790549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b70100900 0x7f3b7010b280 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f3b6000e9c0 tx=0x7f3b6000ee80 comp rx=0 tx=0).stop 2024-06-10T01:20:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.239+0000 7f3b76638640 1 -- 172.21.15.7:0/2956790549 shutdown_connections 2024-06-10T01:20:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.239+0000 7f3b76638640 1 --2- 172.21.15.7:0/2956790549 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3b5c0a2c80 0x7f3b5c0a5140 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.239+0000 7f3b76638640 1 --2- 172.21.15.7:0/2956790549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b701024b0 0x7f3b70109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.239+0000 7f3b76638640 1 --2- 172.21.15.7:0/2956790549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b70101af0 0x7f3b701098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.239+0000 7f3b76638640 1 --2- 172.21.15.7:0/2956790549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b70100900 0x7f3b7010b280 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.239+0000 7f3b76638640 1 -- 172.21.15.7:0/2956790549 >> 172.21.15.7:0/2956790549 conn(0x7f3b700fbe70 msgr2=0x7f3b70104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.240+0000 7f3b76638640 1 -- 172.21.15.7:0/2956790549 shutdown_connections 2024-06-10T01:20:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:26.240+0000 7f3b76638640 1 -- 172.21.15.7:0/2956790549 wait complete. 2024-06-10T01:20:26.471 INFO:tasks.workunit.client.0.smithi007.stdout: 10 16407 16.77 MB/sec warmup 63 sec latency 162.673 ms 2024-06-10T01:20:26.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.799+0000 7fa968fa4700 1 -- 172.21.15.7:0/520252406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa964108520 msgr2=0x7fa964108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:26.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.799+0000 7fa968fa4700 1 --2- 172.21.15.7:0/520252406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa964108520 0x7fa964108900 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fa954009990 tx=0x7fa954009ca0 comp rx=0 tx=0).stop 2024-06-10T01:20:26.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.799+0000 7fa968fa4700 1 -- 172.21.15.7:0/520252406 shutdown_connections 2024-06-10T01:20:26.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.799+0000 7fa968fa4700 1 --2- 172.21.15.7:0/520252406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa96406b520 0x7fa96410f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:26.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.799+0000 7fa968fa4700 1 --2- 172.21.15.7:0/520252406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa964108520 0x7fa964108900 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:26.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.799+0000 7fa968fa4700 1 --2- 172.21.15.7:0/520252406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa96406ab60 0x7fa96406afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:26.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.799+0000 7fa968fa4700 1 -- 172.21.15.7:0/520252406 >> 172.21.15.7:0/520252406 conn(0x7fa964078340 msgr2=0x7fa964078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:26.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.799+0000 7fa968fa4700 1 -- 172.21.15.7:0/520252406 shutdown_connections 2024-06-10T01:20:26.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.800+0000 7fa968fa4700 1 -- 172.21.15.7:0/520252406 wait complete. 2024-06-10T01:20:26.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.802+0000 7fa968fa4700 1 Processor -- start 2024-06-10T01:20:26.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.802+0000 7fa968fa4700 1 -- start start 2024-06-10T01:20:26.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.803+0000 7fa968fa4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa96406ab60 0x7fa964109350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:26.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.803+0000 7fa968fa4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa96406b520 0x7fa964109890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:26.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.803+0000 7fa968fa4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa964108520 0x7fa964109dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:26.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.803+0000 7fa961d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa96406b520 0x7fa964109890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:26.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.803+0000 7fa961d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa96406b520 0x7fa964109890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:41270/0 (socket says 172.21.15.7:41270) 2024-06-10T01:20:26.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.803+0000 7fa961d9b700 1 -- 172.21.15.7:0/2320022422 learned_addr learned my addr 172.21.15.7:0/2320022422 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:26.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.803+0000 7fa968fa4700 1 -- 172.21.15.7:0/2320022422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa9641127a0 con 0x7fa964108520 2024-06-10T01:20:26.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.803+0000 7fa968fa4700 1 -- 172.21.15.7:0/2320022422 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa964112620 con 0x7fa96406ab60 2024-06-10T01:20:26.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.803+0000 7fa968fa4700 1 -- 172.21.15.7:0/2320022422 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa964112920 con 0x7fa96406b520 2024-06-10T01:20:26.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.803+0000 7fa962d9d700 1 --2- 172.21.15.7:0/2320022422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa964108520 0x7fa964109dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:26.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.804+0000 7fa96259c700 1 --2- 172.21.15.7:0/2320022422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa96406ab60 0x7fa964109350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:26.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.804+0000 7fa961d9b700 1 -- 172.21.15.7:0/2320022422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa96406ab60 msgr2=0x7fa964109350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:26.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.804+0000 7fa961d9b700 1 --2- 172.21.15.7:0/2320022422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa96406ab60 0x7fa964109350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:26.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.804+0000 7fa961d9b700 1 -- 172.21.15.7:0/2320022422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa964108520 msgr2=0x7fa964109dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:26.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.804+0000 7fa961d9b700 1 --2- 172.21.15.7:0/2320022422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa964108520 0x7fa964109dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:26.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.804+0000 7fa961d9b700 1 -- 172.21.15.7:0/2320022422 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9641ab4d0 con 0x7fa96406b520 2024-06-10T01:20:26.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.804+0000 7fa96259c700 1 --2- 172.21.15.7:0/2320022422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa96406ab60 0x7fa964109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:20:26.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.804+0000 7fa961d9b700 1 --2- 172.21.15.7:0/2320022422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa96406b520 0x7fa964109890 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa95801fa00 tx=0x7fa95801fd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:26.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.804+0000 7fa9537fe700 1 -- 172.21.15.7:0/2320022422 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95800cbd0 con 0x7fa96406b520 2024-06-10T01:20:26.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.805+0000 7fa9537fe700 1 -- 172.21.15.7:0/2320022422 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa95800cd70 con 0x7fa96406b520 2024-06-10T01:20:26.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.805+0000 7fa968fa4700 1 -- 172.21.15.7:0/2320022422 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa96419abc0 con 0x7fa96406b520 2024-06-10T01:20:26.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.805+0000 7fa9537fe700 1 -- 172.21.15.7:0/2320022422 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa958021440 con 0x7fa96406b520 2024-06-10T01:20:26.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.805+0000 7fa968fa4700 1 -- 172.21.15.7:0/2320022422 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa96419b100 con 0x7fa96406b520 2024-06-10T01:20:26.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.806+0000 7fa968fa4700 1 -- 172.21.15.7:0/2320022422 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa96406a410 con 0x7fa96406b520 2024-06-10T01:20:26.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.808+0000 7fa9537fe700 1 -- 172.21.15.7:0/2320022422 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa958055b30 con 0x7fa96406b520 2024-06-10T01:20:26.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.809+0000 7fa9537fe700 1 --2- 172.21.15.7:0/2320022422 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa94c0a2d70 0x7fa94c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:26.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.809+0000 7fa96259c700 1 --2- 172.21.15.7:0/2320022422 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa94c0a2d70 0x7fa94c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:26.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.809+0000 7fa9537fe700 1 -- 172.21.15.7:0/2320022422 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa9580e06b0 con 0x7fa96406b520 2024-06-10T01:20:26.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.810+0000 7fa96259c700 1 --2- 172.21.15.7:0/2320022422 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa94c0a2d70 0x7fa94c0a5230 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa95400b2e0 tx=0x7fa954005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:26.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:26.815+0000 7fa9537fe700 1 -- 172.21.15.7:0/2320022422 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa9580ad310 con 0x7fa96406b520 2024-06-10T01:20:27.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:27.164+0000 7fa968fa4700 1 -- 172.21.15.7:0/2320022422 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa964069180 con 0x7fa96406b520 2024-06-10T01:20:27.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:27.165+0000 7fa9537fe700 1 -- 172.21.15.7:0/2320022422 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11860 (secure 0 0 0) 0x7fa958055e20 con 0x7fa96406b520 2024-06-10T01:20:27.159 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:20:27.159 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:19:47.239285+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:20:27.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:27.168+0000 7fa968fa4700 1 -- 172.21.15.7:0/2320022422 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa94c0a2d70 msgr2=0x7fa94c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:27.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:27.168+0000 7fa968fa4700 1 --2- 172.21.15.7:0/2320022422 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa94c0a2d70 0x7fa94c0a5230 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa95400b2e0 tx=0x7fa954005fd0 comp rx=0 tx=0).stop 2024-06-10T01:20:27.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:27.168+0000 7fa968fa4700 1 -- 172.21.15.7:0/2320022422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa96406b520 msgr2=0x7fa964109890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:27.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:27.168+0000 7fa968fa4700 1 --2- 172.21.15.7:0/2320022422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa96406b520 0x7fa964109890 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa95801fa00 tx=0x7fa95801fd10 comp rx=0 tx=0).stop 2024-06-10T01:20:27.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:27.168+0000 7fa968fa4700 1 -- 172.21.15.7:0/2320022422 shutdown_connections 2024-06-10T01:20:27.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:27.168+0000 7fa968fa4700 1 --2- 172.21.15.7:0/2320022422 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa94c0a2d70 0x7fa94c0a5230 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:27.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:27.168+0000 7fa968fa4700 1 --2- 172.21.15.7:0/2320022422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa964108520 0x7fa964109dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:27.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:27.168+0000 7fa968fa4700 1 --2- 172.21.15.7:0/2320022422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa96406ab60 0x7fa964109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:27.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:27.168+0000 7fa968fa4700 1 --2- 172.21.15.7:0/2320022422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa96406b520 0x7fa964109890 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:27.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:27.168+0000 7fa968fa4700 1 -- 172.21.15.7:0/2320022422 >> 172.21.15.7:0/2320022422 conn(0x7fa964078340 msgr2=0x7fa96410d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:27.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:27.169+0000 7fa968fa4700 1 -- 172.21.15.7:0/2320022422 shutdown_connections 2024-06-10T01:20:27.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:27.169+0000 7fa968fa4700 1 -- 172.21.15.7:0/2320022422 wait complete. 2024-06-10T01:20:27.163 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 44 2024-06-10T01:20:27.471 INFO:tasks.workunit.client.0.smithi007.stdout: 10 17271 17.27 MB/sec warmup 64 sec latency 75.262 ms 2024-06-10T01:20:27.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:27 smithi175 ceph-mon[36636]: pgmap v865: 129 pgs: 129 active+clean; 358 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 311 op/s 2024-06-10T01:20:27.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:27 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2320022422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:20:27.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:27 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:20:27.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:27 smithi007 ceph-mon[32398]: pgmap v865: 129 pgs: 129 active+clean; 358 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 311 op/s 2024-06-10T01:20:27.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:27 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2320022422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:20:27.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:27 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:20:27.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:27 smithi078 ceph-mon[36683]: pgmap v865: 129 pgs: 129 active+clean; 358 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 311 op/s 2024-06-10T01:20:27.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:27 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2320022422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:20:27.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:27 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:20:27.945 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db / --timeout 90 --expiration 147.65 --await-for 45.0' 2024-06-10T01:20:27.945 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db / --timeout 90 --expiration 147.65 --await-for 45.0 2024-06-10T01:20:28.329 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:20:28.471 INFO:tasks.workunit.client.0.smithi007.stdout: 10 18051 17.61 MB/sec warmup 65 sec latency 54.192 ms 2024-06-10T01:20:28.700 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:28 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:20:28.701 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:28 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:20:28.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:28 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:20:28.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:28 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:20:28.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:28 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:20:28.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:28 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:20:29.471 INFO:tasks.workunit.client.0.smithi007.stdout: 10 18912 18.12 MB/sec warmup 66 sec latency 41.588 ms 2024-06-10T01:20:29.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:29 smithi175 ceph-mon[36636]: pgmap v866: 129 pgs: 129 active+clean; 362 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 380 op/s 2024-06-10T01:20:29.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:29 smithi007 ceph-mon[32398]: pgmap v866: 129 pgs: 129 active+clean; 362 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 380 op/s 2024-06-10T01:20:29.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:29 smithi078 ceph-mon[36683]: pgmap v866: 129 pgs: 129 active+clean; 362 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 380 op/s 2024-06-10T01:20:30.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.340+0000 7f224dffb700 1 -- 172.21.15.7:0/879205545 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2240017400 con 0x7f2250109a60 2024-06-10T01:20:30.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.341+0000 7f2255793700 1 -- 172.21.15.7:0/879205545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2250109a60 msgr2=0x7f2250109ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:30.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.341+0000 7f2255793700 1 --2- 172.21.15.7:0/879205545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2250109a60 0x7f2250109ee0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f22400098f0 tx=0x7f2240009c00 comp rx=0 tx=0).stop 2024-06-10T01:20:30.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.341+0000 7f2255793700 1 -- 172.21.15.7:0/879205545 shutdown_connections 2024-06-10T01:20:30.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.341+0000 7f2255793700 1 --2- 172.21.15.7:0/879205545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22501175c0 0x7f22501199b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.341+0000 7f2255793700 1 --2- 172.21.15.7:0/879205545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f225010cc30 0x7f2250109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.341+0000 7f2255793700 1 --2- 172.21.15.7:0/879205545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2250109a60 0x7f2250109ee0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.341+0000 7f2255793700 1 -- 172.21.15.7:0/879205545 >> 172.21.15.7:0/879205545 conn(0x7f225006da80 msgr2=0x7f225006de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:30.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.341+0000 7f2255793700 1 -- 172.21.15.7:0/879205545 shutdown_connections 2024-06-10T01:20:30.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.342+0000 7f2255793700 1 -- 172.21.15.7:0/879205545 wait complete. 2024-06-10T01:20:30.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.342+0000 7f2255793700 1 Processor -- start 2024-06-10T01:20:30.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.343+0000 7f2255793700 1 -- start start 2024-06-10T01:20:30.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.343+0000 7f2255793700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2250109a60 0x7f2250116c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:30.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.343+0000 7f2255793700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f225010cc30 0x7f2250113d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:30.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.343+0000 7f2255793700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22501175c0 0x7f22501142b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:30.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.343+0000 7f2255793700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f225011d2a0 con 0x7f225010cc30 2024-06-10T01:20:30.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.343+0000 7f2255793700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f225011d120 con 0x7f22501175c0 2024-06-10T01:20:30.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.343+0000 7f2255793700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f225011d420 con 0x7f2250109a60 2024-06-10T01:20:30.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.343+0000 7f224f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22501175c0 0x7f22501142b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:30.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.343+0000 7f224effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2250109a60 0x7f2250116c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:30.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.344+0000 7f224effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2250109a60 0x7f2250116c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:41280/0 (socket says 172.21.15.7:41280) 2024-06-10T01:20:30.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.344+0000 7f224effd700 1 -- 172.21.15.7:0/1142902602 learned_addr learned my addr 172.21.15.7:0/1142902602 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:30.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.344+0000 7f224e7fc700 1 --2- 172.21.15.7:0/1142902602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f225010cc30 0x7f2250113d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:30.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.344+0000 7f224effd700 1 -- 172.21.15.7:0/1142902602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22501175c0 msgr2=0x7f22501142b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:30.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.344+0000 7f224effd700 1 --2- 172.21.15.7:0/1142902602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22501175c0 0x7f22501142b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.344+0000 7f224effd700 1 -- 172.21.15.7:0/1142902602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f225010cc30 msgr2=0x7f2250113d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:30.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.344+0000 7f224effd700 1 --2- 172.21.15.7:0/1142902602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f225010cc30 0x7f2250113d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.344+0000 7f224effd700 1 -- 172.21.15.7:0/1142902602 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2250114a80 con 0x7f2250109a60 2024-06-10T01:20:30.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.344+0000 7f224effd700 1 --2- 172.21.15.7:0/1142902602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2250109a60 0x7f2250116c90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f224400c9b0 tx=0x7f224400ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:30.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.345+0000 7f2237fff700 1 -- 172.21.15.7:0/1142902602 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2244007b00 con 0x7f2250109a60 2024-06-10T01:20:30.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.345+0000 7f2237fff700 1 -- 172.21.15.7:0/1142902602 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2244007ca0 con 0x7f2250109a60 2024-06-10T01:20:30.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.346+0000 7f2237fff700 1 -- 172.21.15.7:0/1142902602 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22440186f0 con 0x7f2250109a60 2024-06-10T01:20:30.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.346+0000 7f2255793700 1 -- 172.21.15.7:0/1142902602 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22501b3fe0 con 0x7f2250109a60 2024-06-10T01:20:30.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.346+0000 7f2255793700 1 -- 172.21.15.7:0/1142902602 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22501b4440 con 0x7f2250109a60 2024-06-10T01:20:30.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.351+0000 7f2260016b80 1 Processor -- start 2024-06-10T01:20:30.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.352+0000 7f2260016b80 1 -- start start 2024-06-10T01:20:30.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.352+0000 7f2260016b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c8816d5a0 0x559c8816d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:30.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.352+0000 7f2260016b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c8816dec0 0x559c881677b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:30.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.352+0000 7f224f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c8816d5a0 0x559c8816d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:30.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.352+0000 7f224f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c8816d5a0 0x559c8816d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53984/0 (socket says 172.21.15.7:53984) 2024-06-10T01:20:30.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.352+0000 7f2260016b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c88167cf0 0x559c88168110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:30.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.353+0000 7f224f7fe700 1 -- 172.21.15.7:0/683048803 learned_addr learned my addr 172.21.15.7:0/683048803 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:30.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.353+0000 7f2260016b80 1 -- 172.21.15.7:0/683048803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559c8805a100 con 0x559c8816d5a0 2024-06-10T01:20:30.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.353+0000 7f2260016b80 1 -- 172.21.15.7:0/683048803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559c880b89f0 con 0x559c88167cf0 2024-06-10T01:20:30.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.353+0000 7f2260016b80 1 -- 172.21.15.7:0/683048803 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559c88020600 con 0x559c8816dec0 2024-06-10T01:20:30.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.353+0000 7f224effd700 1 --2- 172.21.15.7:0/683048803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c8816dec0 0x559c881677b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:30.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.353+0000 7f224e7fc700 1 --2- 172.21.15.7:0/683048803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c88167cf0 0x559c88168110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:30.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.353+0000 7f2237fff700 1 -- 172.21.15.7:0/1142902602 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f224400f450 con 0x7f2250109a60 2024-06-10T01:20:30.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.354+0000 7f224effd700 1 -- 172.21.15.7:0/683048803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c88167cf0 msgr2=0x559c88168110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:30.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.354+0000 7f224effd700 1 --2- 172.21.15.7:0/683048803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c88167cf0 0x559c88168110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.354+0000 7f224effd700 1 -- 172.21.15.7:0/683048803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c8816d5a0 msgr2=0x559c8816d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:30.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.354+0000 7f224effd700 1 --2- 172.21.15.7:0/683048803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c8816d5a0 0x559c8816d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.354+0000 7f224effd700 1 -- 172.21.15.7:0/683048803 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c88007a70 con 0x559c8816dec0 2024-06-10T01:20:30.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.354+0000 7f224e7fc700 1 --2- 172.21.15.7:0/683048803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c88167cf0 0x559c88168110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:20:30.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.354+0000 7f224f7fe700 1 --2- 172.21.15.7:0/683048803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c8816d5a0 0x559c8816d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:20:30.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.354+0000 7f224effd700 1 --2- 172.21.15.7:0/683048803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c8816dec0 0x559c881677b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f22440cc870 tx=0x7f22440d7480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27b24f23380a7522 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:30.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.355+0000 7f22357fa700 1 -- 172.21.15.7:0/683048803 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22440e2070 con 0x559c8816dec0 2024-06-10T01:20:30.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.355+0000 7f22357fa700 1 -- 172.21.15.7:0/683048803 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22440cf5e0 con 0x559c8816dec0 2024-06-10T01:20:30.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.355+0000 7f22357fa700 1 -- 172.21.15.7:0/683048803 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22440dd660 con 0x559c8816dec0 2024-06-10T01:20:30.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.355+0000 7f2237fff700 1 --2- 172.21.15.7:0/1142902602 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f22380a2f00 0x7f22380a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:30.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.356+0000 7f2237fff700 1 -- 172.21.15.7:0/1142902602 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f22440cf1b0 con 0x7f2250109a60 2024-06-10T01:20:30.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.356+0000 7f2260016b80 1 -- 172.21.15.7:0/683048803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c8816dec0 msgr2=0x559c881677b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:30.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.356+0000 7f2260016b80 1 --2- 172.21.15.7:0/683048803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c8816dec0 0x559c881677b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f22440cc870 tx=0x7f22440d7480 comp rx=0 tx=0).stop 2024-06-10T01:20:30.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.356+0000 7f224f7fe700 1 --2- 172.21.15.7:0/1142902602 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f22380a2f00 0x7f22380a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:30.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.357+0000 7f224f7fe700 1 --2- 172.21.15.7:0/1142902602 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f22380a2f00 0x7f22380a53c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2248009e60 tx=0x7f2248009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:30.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.357+0000 7f2260016b80 1 -- 172.21.15.7:0/683048803 shutdown_connections 2024-06-10T01:20:30.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.357+0000 7f2260016b80 1 --2- 172.21.15.7:0/683048803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c88167cf0 0x559c88168110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.357+0000 7f2260016b80 1 --2- 172.21.15.7:0/683048803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c8816d5a0 0x559c8816d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.357+0000 7f2260016b80 1 --2- 172.21.15.7:0/683048803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c8816dec0 0x559c881677b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.357+0000 7f2260016b80 1 -- 172.21.15.7:0/683048803 >> 172.21.15.7:0/683048803 conn(0x559c88094220 msgr2=0x559c88162aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:30.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.358+0000 7f2260016b80 1 -- 172.21.15.7:0/683048803 shutdown_connections 2024-06-10T01:20:30.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.358+0000 7f2260016b80 1 -- 172.21.15.7:0/683048803 wait complete. 2024-06-10T01:20:30.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.359+0000 7f2260016b80 1 -- start start 2024-06-10T01:20:30.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.359+0000 7f2260016b80 1 Processor -- start 2024-06-10T01:20:30.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.360+0000 7f2260016b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c88167cf0 0x559c88173b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:30.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.360+0000 7f2260016b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c8816d5a0 0x559c88170e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:30.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.360+0000 7f2260016b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c8816dec0 0x559c881713d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:30.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.360+0000 7f224effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c88167cf0 0x559c88173b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:30.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.360+0000 7f224e7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c8816d5a0 0x559c88170e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:30.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.360+0000 7f2260016b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559c88059160 con 0x559c88167cf0 2024-06-10T01:20:30.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.360+0000 7f224e7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c8816d5a0 0x559c88170e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:41300/0 (socket says 172.21.15.7:41300) 2024-06-10T01:20:30.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.360+0000 7f224effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c88167cf0 0x559c88173b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53986/0 (socket says 172.21.15.7:53986) 2024-06-10T01:20:30.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.360+0000 7f224e7fc700 1 -- 172.21.15.7:0/275227567 learned_addr learned my addr 172.21.15.7:0/275227567 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:30.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.360+0000 7f2260016b80 1 -- 172.21.15.7:0/275227567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559c880b49b0 con 0x559c8816dec0 2024-06-10T01:20:30.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.360+0000 7f224f7fe700 1 --2- 172.21.15.7:0/275227567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c8816dec0 0x559c881713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:30.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.360+0000 7f2260016b80 1 -- 172.21.15.7:0/275227567 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559c881748a0 con 0x559c8816d5a0 2024-06-10T01:20:30.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.360+0000 7f2260016b80 10 client.0 _ll_register_callbacks cb 0x559c8804e750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:20:30.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.361+0000 7f224f7fe700 1 -- 172.21.15.7:0/275227567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c8816d5a0 msgr2=0x559c88170e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:30.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.361+0000 7f224f7fe700 1 --2- 172.21.15.7:0/275227567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c8816d5a0 0x559c88170e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.361+0000 7f224f7fe700 1 -- 172.21.15.7:0/275227567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c88167cf0 msgr2=0x559c88173b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:30.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.361+0000 7f224f7fe700 1 --2- 172.21.15.7:0/275227567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c88167cf0 0x559c88173b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.361+0000 7f224f7fe700 1 -- 172.21.15.7:0/275227567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c8807e8f0 con 0x559c8816dec0 2024-06-10T01:20:30.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.361+0000 7f224f7fe700 1 --2- 172.21.15.7:0/275227567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c8816dec0 0x559c881713d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f224801be20 tx=0x7f224800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b86cf24b0f7d350b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:30.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.361+0000 7f2232ff5700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:20:30.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.361+0000 7f2232ff5700 1 -- 172.21.15.7:0/275227567 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2248028060 con 0x559c8816dec0 2024-06-10T01:20:30.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.361+0000 7f2232ff5700 1 -- 172.21.15.7:0/275227567 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f2248019e00 con 0x559c8816dec0 2024-06-10T01:20:30.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.361+0000 7f2260016b80 1 -- 172.21.15.7:0/275227567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22440cf5e0 con 0x559c8816dec0 2024-06-10T01:20:30.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.361+0000 7f2232ff5700 1 -- 172.21.15.7:0/275227567 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f224801be50 con 0x559c8816dec0 2024-06-10T01:20:30.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.361+0000 7f2260016b80 10 client.28397 fetch_fsmap learned FSMap version 44 2024-06-10T01:20:30.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.362+0000 7f2260016b80 1 -- 172.21.15.7:0/275227567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x559c8817ef70 con 0x559c8816dec0 2024-06-10T01:20:30.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.362+0000 7f2232ff5700 1 -- 172.21.15.7:0/275227567 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22440cf5e0 con 0x559c8816dec0 2024-06-10T01:20:30.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.362+0000 7f2232ff5700 1 -- 172.21.15.7:0/275227567 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f2248017570 con 0x559c8816dec0 2024-06-10T01:20:30.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.362+0000 7f2260016b80 10 client.28397 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:20:30.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.362+0000 7f2260016b80 10 client.28397 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:20:30.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.362+0000 7f2260016b80 20 client.28397 populate_metadata read hostname 'smithi007' 2024-06-10T01:20:30.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.362+0000 7f2260016b80 1 --2- 172.21.15.7:0/275227567 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559c88181600 0x559c88183a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:30.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.362+0000 7f2260016b80 4 client.28397 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:20:30.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.362+0000 7f2260016b80 1 -- 172.21.15.7:0/275227567 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c880c5ef0 con 0x559c88181600 2024-06-10T01:20:30.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.363+0000 7f224effd700 1 --2- 172.21.15.7:0/275227567 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559c88181600 0x559c88183a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:30.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.363+0000 7f224effd700 1 --2- 172.21.15.7:0/275227567 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559c88181600 0x559c88183a20 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee3b9d9ddee7302d server_cookie=1f0a28ee2be241ef in_seq=0 out_seq=0 2024-06-10T01:20:30.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.363+0000 7f2232ff5700 10 client.28397 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:30.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.366+0000 7f2232ff5700 1 -- 172.21.15.7:0/275227567 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559c880c5ef0 con 0x559c88181600 2024-06-10T01:20:30.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.366+0000 7f2232ff5700 10 client.28397 handle_command_reply: tid=0 2024-06-10T01:20:30.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.366+0000 7f2260016b80 1 client.28397 shutdown 2024-06-10T01:20:30.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.366+0000 7f2260016b80 1 -- 172.21.15.7:0/275227567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c8816dec0 msgr2=0x559c881713d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:30.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.366+0000 7f2260016b80 1 --2- 172.21.15.7:0/275227567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c8816dec0 0x559c881713d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f224801be20 tx=0x7f224800b040 comp rx=0 tx=0).stop 2024-06-10T01:20:30.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.366+0000 7f2260016b80 1 -- 172.21.15.7:0/275227567 shutdown_connections 2024-06-10T01:20:30.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.366+0000 7f2260016b80 1 --2- 172.21.15.7:0/275227567 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559c88181600 0x559c88183a20 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.366+0000 7f2232ff5700 0 client.28397 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:30.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.366+0000 7f2260016b80 1 --2- 172.21.15.7:0/275227567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c8816dec0 0x559c881713d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.366+0000 7f2260016b80 1 --2- 172.21.15.7:0/275227567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c88167cf0 0x559c88173b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.366+0000 7f2260016b80 1 --2- 172.21.15.7:0/275227567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c8816d5a0 0x559c88170e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.366+0000 7f2260016b80 1 -- 172.21.15.7:0/275227567 >> 172.21.15.7:0/275227567 conn(0x559c88094220 msgr2=0x559c88161c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:30.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.366+0000 7f2260016b80 1 -- 172.21.15.7:0/275227567 shutdown_connections 2024-06-10T01:20:30.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.366+0000 7f2260016b80 1 -- 172.21.15.7:0/275227567 wait complete. 2024-06-10T01:20:30.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.367+0000 7f2260016b80 20 client.28397 trim_cache size 0 max 16384 2024-06-10T01:20:30.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.395+0000 7f2260016b80 1 Processor -- start 2024-06-10T01:20:30.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.395+0000 7f2260016b80 1 -- start start 2024-06-10T01:20:30.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.395+0000 7f2260016b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c88167cf0 0x559c8817da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.395+0000 7f2260016b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c8816d5a0 0x559c8816c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.395+0000 7f2260016b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c8816dec0 0x559c8816cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.395+0000 7f2260016b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f22501bf9d0 con 0x559c8816d5a0 2024-06-10T01:20:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.395+0000 7f2260016b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f22501bd1d0 con 0x559c88167cf0 2024-06-10T01:20:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.395+0000 7f2260016b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f22501bb020 con 0x559c8816dec0 2024-06-10T01:20:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.395+0000 7f224e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c8816d5a0 0x559c8816c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.395+0000 7f224e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c8816d5a0 0x559c8816c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53994/0 (socket says 172.21.15.7:53994) 2024-06-10T01:20:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.395+0000 7f224e7fc700 1 -- 172.21.15.7:0/3997574279 learned_addr learned my addr 172.21.15.7:0/3997574279 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.395+0000 7f224effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c88167cf0 0x559c8817da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.396+0000 7f224f7fe700 1 --2- 172.21.15.7:0/3997574279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c8816dec0 0x559c8816cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.396+0000 7f224effd700 1 -- 172.21.15.7:0/3997574279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c8816dec0 msgr2=0x559c8816cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.396+0000 7f224effd700 1 --2- 172.21.15.7:0/3997574279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c8816dec0 0x559c8816cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.396+0000 7f224effd700 1 -- 172.21.15.7:0/3997574279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c8816d5a0 msgr2=0x559c8816c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:30.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.396+0000 7f224effd700 1 --2- 172.21.15.7:0/3997574279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c8816d5a0 0x559c8816c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.396+0000 7f224effd700 1 -- 172.21.15.7:0/3997574279 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c8816f400 con 0x559c88167cf0 2024-06-10T01:20:30.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.396+0000 7f224effd700 1 --2- 172.21.15.7:0/3997574279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c88167cf0 0x559c8817da20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2244097b10 tx=0x7f22440d2c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eacf2f2de50bf04a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:30.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.396+0000 7f2234ff9700 1 -- 172.21.15.7:0/3997574279 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224409b480 con 0x559c88167cf0 2024-06-10T01:20:30.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.396+0000 7f2234ff9700 1 -- 172.21.15.7:0/3997574279 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2244007e50 con 0x559c88167cf0 2024-06-10T01:20:30.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.396+0000 7f2234ff9700 1 -- 172.21.15.7:0/3997574279 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224409b010 con 0x559c88167cf0 2024-06-10T01:20:30.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.397+0000 7f2260016b80 1 -- 172.21.15.7:0/3997574279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c88167cf0 msgr2=0x559c8817da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:30.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.397+0000 7f2260016b80 1 --2- 172.21.15.7:0/3997574279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c88167cf0 0x559c8817da20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2244097b10 tx=0x7f22440d2c30 comp rx=0 tx=0).stop 2024-06-10T01:20:30.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.397+0000 7f2260016b80 1 -- 172.21.15.7:0/3997574279 shutdown_connections 2024-06-10T01:20:30.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.397+0000 7f2260016b80 1 --2- 172.21.15.7:0/3997574279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c8816dec0 0x559c8816cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.397+0000 7f2260016b80 1 --2- 172.21.15.7:0/3997574279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c88167cf0 0x559c8817da20 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.397+0000 7f2260016b80 1 --2- 172.21.15.7:0/3997574279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c8816d5a0 0x559c8816c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.397+0000 7f2260016b80 1 -- 172.21.15.7:0/3997574279 >> 172.21.15.7:0/3997574279 conn(0x559c88094220 msgr2=0x559c8816bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:30.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.397+0000 7f2260016b80 1 -- 172.21.15.7:0/3997574279 shutdown_connections 2024-06-10T01:20:30.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.398+0000 7f2260016b80 1 -- 172.21.15.7:0/3997574279 wait complete. 2024-06-10T01:20:30.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.398+0000 7f2260016b80 1 -- start start 2024-06-10T01:20:30.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.398+0000 7f2260016b80 1 Processor -- start 2024-06-10T01:20:30.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.398+0000 7f2260016b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c88167cf0 0x559c8817a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:30.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.399+0000 7f2260016b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c8816d5a0 0x559c88178630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:30.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.399+0000 7f2260016b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c8816dec0 0x559c88178b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:30.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.399+0000 7f2260016b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f22501bb690 con 0x559c8816d5a0 2024-06-10T01:20:30.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.399+0000 7f224effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c88167cf0 0x559c8817a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:30.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.399+0000 7f2260016b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f22501bcb00 con 0x559c88167cf0 2024-06-10T01:20:30.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.399+0000 7f2260016b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f22501bbd60 con 0x559c8816dec0 2024-06-10T01:20:30.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.399+0000 7f224effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c88167cf0 0x559c8817a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:36554/0 (socket says 172.21.15.7:36554) 2024-06-10T01:20:30.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.399+0000 7f2260016b80 10 client.0 _ll_register_callbacks cb 0x559c8804e750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:20:30.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.399+0000 7f224effd700 1 -- 172.21.15.7:0/2276228226 learned_addr learned my addr 172.21.15.7:0/2276228226 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:30.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.399+0000 7f224f7fe700 1 --2- 172.21.15.7:0/2276228226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c8816dec0 0x559c88178b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:30.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.399+0000 7f224e7fc700 1 --2- 172.21.15.7:0/2276228226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c8816d5a0 0x559c88178630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:30.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.399+0000 7f224effd700 1 -- 172.21.15.7:0/2276228226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c8816dec0 msgr2=0x559c88178b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:30.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.399+0000 7f224effd700 1 --2- 172.21.15.7:0/2276228226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c8816dec0 0x559c88178b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.399+0000 7f224effd700 1 -- 172.21.15.7:0/2276228226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c8816d5a0 msgr2=0x559c88178630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:30.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.399+0000 7f224effd700 1 --2- 172.21.15.7:0/2276228226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c8816d5a0 0x559c88178630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.399+0000 7f224effd700 1 -- 172.21.15.7:0/2276228226 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c88179370 con 0x559c88167cf0 2024-06-10T01:20:30.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.399+0000 7f224effd700 1 --2- 172.21.15.7:0/2276228226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c88167cf0 0x559c8817a280 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f22440074a0 tx=0x7f224400a670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10ada09d2b611724 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:30.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.399+0000 7f2232ff5700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:20:30.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.400+0000 7f2232ff5700 1 -- 172.21.15.7:0/2276228226 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224409b1f0 con 0x559c88167cf0 2024-06-10T01:20:30.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.400+0000 7f2232ff5700 1 -- 172.21.15.7:0/2276228226 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f2244016da0 con 0x559c88167cf0 2024-06-10T01:20:30.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.400+0000 7f2260016b80 1 -- 172.21.15.7:0/2276228226 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2244007e50 con 0x559c88167cf0 2024-06-10T01:20:30.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.400+0000 7f2232ff5700 1 -- 172.21.15.7:0/2276228226 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f224409fa60 con 0x559c88167cf0 2024-06-10T01:20:30.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.400+0000 7f2260016b80 10 client.28403 fetch_fsmap learned FSMap version 44 2024-06-10T01:20:30.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.400+0000 7f2260016b80 1 -- 172.21.15.7:0/2276228226 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x559c881ed2b0 con 0x559c88167cf0 2024-06-10T01:20:30.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.400+0000 7f2232ff5700 1 -- 172.21.15.7:0/2276228226 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2244007e50 con 0x559c88167cf0 2024-06-10T01:20:30.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.400+0000 7f2232ff5700 1 -- 172.21.15.7:0/2276228226 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f2244020020 con 0x559c88167cf0 2024-06-10T01:20:30.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.400+0000 7f2260016b80 10 client.28403 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:20:30.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.400+0000 7f2260016b80 10 client.28403 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:20:30.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.400+0000 7f2260016b80 20 client.28403 populate_metadata read hostname 'smithi007' 2024-06-10T01:20:30.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.400+0000 7f2260016b80 1 --2- 172.21.15.7:0/2276228226 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559c88181600 0x559c881ef2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:30.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.400+0000 7f2260016b80 4 client.28403 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.65, "await_for": 45.0}] 2024-06-10T01:20:30.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.400+0000 7f2260016b80 1 -- 172.21.15.7:0/2276228226 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.65, "await_for": 45.0}) -- 0x559c880c0470 con 0x559c88181600 2024-06-10T01:20:30.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.401+0000 7f224e7fc700 1 --2- 172.21.15.7:0/2276228226 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559c88181600 0x559c881ef2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:30.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.401+0000 7f224e7fc700 1 --2- 172.21.15.7:0/2276228226 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559c88181600 0x559c881ef2c0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=828994a222d62e45 server_cookie=dd6a6fb477f11b48 in_seq=0 out_seq=0 2024-06-10T01:20:30.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.401+0000 7f2232ff5700 10 client.28403 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:30.472 INFO:tasks.workunit.client.0.smithi007.stdout: 10 19772 18.54 MB/sec warmup 67 sec latency 73.411 ms 2024-06-10T01:20:30.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.897+0000 7f2232ff5700 1 -- 172.21.15.7:0/2276228226 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559c880c0470 con 0x559c88181600 2024-06-10T01:20:30.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.897+0000 7f2232ff5700 10 client.28403 handle_command_reply: tid=0 2024-06-10T01:20:30.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.897+0000 7f2260016b80 1 client.28403 shutdown 2024-06-10T01:20:30.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.897+0000 7f2260016b80 1 -- 172.21.15.7:0/2276228226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c88167cf0 msgr2=0x559c8817a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:30.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.897+0000 7f2260016b80 1 --2- 172.21.15.7:0/2276228226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c88167cf0 0x559c8817a280 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f22440074a0 tx=0x7f224400a670 comp rx=0 tx=0).stop 2024-06-10T01:20:30.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.897+0000 7f2260016b80 1 -- 172.21.15.7:0/2276228226 shutdown_connections 2024-06-10T01:20:30.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.897+0000 7f2260016b80 1 --2- 172.21.15.7:0/2276228226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c8816dec0 0x559c88178b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.897+0000 7f2260016b80 1 --2- 172.21.15.7:0/2276228226 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559c88181600 0x559c881ef2c0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.897+0000 7f2260016b80 1 --2- 172.21.15.7:0/2276228226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c88167cf0 0x559c8817a280 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.897+0000 7f2260016b80 1 --2- 172.21.15.7:0/2276228226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c8816d5a0 0x559c88178630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.897+0000 7f2232ff5700 0 client.28403 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:30.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.897+0000 7f2260016b80 1 -- 172.21.15.7:0/2276228226 >> 172.21.15.7:0/2276228226 conn(0x559c88094220 msgr2=0x559c8816c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:30.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.897+0000 7f2260016b80 1 -- 172.21.15.7:0/2276228226 shutdown_connections 2024-06-10T01:20:30.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.897+0000 7f2260016b80 1 -- 172.21.15.7:0/2276228226 wait complete. 2024-06-10T01:20:30.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.897+0000 7f2260016b80 20 client.28403 trim_cache size 0 max 16384 2024-06-10T01:20:30.895 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:20:30.895 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 44, 2024-06-10T01:20:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:20:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 50, 2024-06-10T01:20:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:20:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: "8f7fcf77": { 2024-06-10T01:20:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 50, 2024-06-10T01:20:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:20:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:20:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:20:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:20:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:20:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:20:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 147.7, 2024-06-10T01:20:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:20:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:20:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:20:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:20:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:20:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:20:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:20:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:20:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:20:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:20:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:20:30.898 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:20:30.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.898+0000 7f2255793700 1 -- 172.21.15.7:0/1142902602 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f22380a2f00 msgr2=0x7f22380a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:30.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.898+0000 7f2255793700 1 --2- 172.21.15.7:0/1142902602 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f22380a2f00 0x7f22380a53c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2248009e60 tx=0x7f2248009040 comp rx=0 tx=0).stop 2024-06-10T01:20:30.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.898+0000 7f2255793700 1 -- 172.21.15.7:0/1142902602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2250109a60 msgr2=0x7f2250116c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:30.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.898+0000 7f2255793700 1 --2- 172.21.15.7:0/1142902602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2250109a60 0x7f2250116c90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f224400c9b0 tx=0x7f224400ccc0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.898+0000 7f2255793700 1 -- 172.21.15.7:0/1142902602 shutdown_connections 2024-06-10T01:20:30.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.898+0000 7f2255793700 1 --2- 172.21.15.7:0/1142902602 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f22380a2f00 0x7f22380a53c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.898+0000 7f2255793700 1 --2- 172.21.15.7:0/1142902602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22501175c0 0x7f22501142b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.898+0000 7f2255793700 1 --2- 172.21.15.7:0/1142902602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2250109a60 0x7f2250116c90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.898+0000 7f2255793700 1 --2- 172.21.15.7:0/1142902602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f225010cc30 0x7f2250113d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:30.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.898+0000 7f2255793700 1 -- 172.21.15.7:0/1142902602 >> 172.21.15.7:0/1142902602 conn(0x7f225006da80 msgr2=0x7f2250073020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:30.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.898+0000 7f2255793700 1 -- 172.21.15.7:0/1142902602 shutdown_connections 2024-06-10T01:20:30.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:30.898+0000 7f2255793700 1 -- 172.21.15.7:0/1142902602 wait complete. 2024-06-10T01:20:31.472 INFO:tasks.workunit.client.0.smithi007.stdout: 10 19792 18.27 MB/sec warmup 68 sec latency 1067.142 ms 2024-06-10T01:20:31.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:31 smithi007 ceph-mon[32398]: pgmap v867: 129 pgs: 129 active+clean; 372 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.1 MiB/s wr, 524 op/s 2024-06-10T01:20:31.627 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8f7fcf77', quiesce took 8.5 seconds. Will release after: 19.15 2024-06-10T01:20:31.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:31 smithi078 ceph-mon[36683]: pgmap v867: 129 pgs: 129 active+clean; 372 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.1 MiB/s wr, 524 op/s 2024-06-10T01:20:31.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:31 smithi175 ceph-mon[36636]: pgmap v867: 129 pgs: 129 active+clean; 372 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.1 MiB/s wr, 524 op/s 2024-06-10T01:20:32.472 INFO:tasks.workunit.client.0.smithi007.stdout: 10 19792 18.00 MB/sec warmup 69 sec latency 2067.694 ms 2024-06-10T01:20:33.472 INFO:tasks.workunit.client.0.smithi007.stdout: 10 19792 17.75 MB/sec warmup 70 sec latency 3067.764 ms 2024-06-10T01:20:33.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:33 smithi007 ceph-mon[32398]: pgmap v868: 129 pgs: 129 active+clean; 376 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.4 MiB/s wr, 551 op/s 2024-06-10T01:20:33.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:33 smithi078 ceph-mon[36683]: pgmap v868: 129 pgs: 129 active+clean; 376 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.4 MiB/s wr, 551 op/s 2024-06-10T01:20:33.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:33 smithi175 ceph-mon[36636]: pgmap v868: 129 pgs: 129 active+clean; 376 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.4 MiB/s wr, 551 op/s 2024-06-10T01:20:34.472 INFO:tasks.workunit.client.0.smithi007.stdout: 10 19792 17.50 MB/sec warmup 71 sec latency 4067.805 ms 2024-06-10T01:20:35.472 INFO:tasks.workunit.client.0.smithi007.stdout: 10 19792 17.25 MB/sec warmup 72 sec latency 5067.862 ms 2024-06-10T01:20:35.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:35 smithi007 ceph-mon[32398]: pgmap v869: 129 pgs: 129 active+clean; 378 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.5 MiB/s wr, 545 op/s 2024-06-10T01:20:35.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:35 smithi078 ceph-mon[36683]: pgmap v869: 129 pgs: 129 active+clean; 378 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.5 MiB/s wr, 545 op/s 2024-06-10T01:20:35.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:35 smithi175 ceph-mon[36636]: pgmap v869: 129 pgs: 129 active+clean; 378 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.5 MiB/s wr, 545 op/s 2024-06-10T01:20:36.472 INFO:tasks.workunit.client.0.smithi007.stdout: 10 19792 17.02 MB/sec warmup 73 sec latency 6067.993 ms 2024-06-10T01:20:37.472 INFO:tasks.workunit.client.0.smithi007.stdout: 10 19792 16.79 MB/sec warmup 74 sec latency 7068.110 ms 2024-06-10T01:20:37.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:37 smithi007 ceph-mon[32398]: pgmap v870: 129 pgs: 129 active+clean; 379 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.5 MiB/s wr, 528 op/s 2024-06-10T01:20:37.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:37 smithi078 ceph-mon[36683]: pgmap v870: 129 pgs: 129 active+clean; 379 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.5 MiB/s wr, 528 op/s 2024-06-10T01:20:37.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:37 smithi175 ceph-mon[36636]: pgmap v870: 129 pgs: 129 active+clean; 379 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.5 MiB/s wr, 528 op/s 2024-06-10T01:20:38.472 INFO:tasks.workunit.client.0.smithi007.stdout: 10 19792 16.56 MB/sec warmup 75 sec latency 8068.178 ms 2024-06-10T01:20:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.333+0000 7fa5ccfe7640 1 -- 172.21.15.7:0/3357965794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5c81024b0 msgr2=0x7fa5c81048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.333+0000 7fa5ccfe7640 1 --2- 172.21.15.7:0/3357965794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5c81024b0 0x7fa5c81048a0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8009900 tx=0x7fa5b802a410 comp rx=0 tx=0).stop 2024-06-10T01:20:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.333+0000 7fa5ccfe7640 1 -- 172.21.15.7:0/3357965794 shutdown_connections 2024-06-10T01:20:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.333+0000 7fa5ccfe7640 1 --2- 172.21.15.7:0/3357965794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5c8107710 0x7fa5c8109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.333+0000 7fa5ccfe7640 1 --2- 172.21.15.7:0/3357965794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5c8104de0 0x7fa5c81071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.333+0000 7fa5ccfe7640 1 --2- 172.21.15.7:0/3357965794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5c81024b0 0x7fa5c81048a0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.333+0000 7fa5ccfe7640 1 -- 172.21.15.7:0/3357965794 >> 172.21.15.7:0/3357965794 conn(0x7fa5c80fbec0 msgr2=0x7fa5c80fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.333+0000 7fa5ccfe7640 1 -- 172.21.15.7:0/3357965794 shutdown_connections 2024-06-10T01:20:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.333+0000 7fa5ccfe7640 1 -- 172.21.15.7:0/3357965794 wait complete. 2024-06-10T01:20:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.334+0000 7fa5ccfe7640 1 Processor -- start 2024-06-10T01:20:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.334+0000 7fa5ccfe7640 1 -- start start 2024-06-10T01:20:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.334+0000 7fa5ccfe7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5c81024b0 0x7fa5c8197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.334+0000 7fa5ccfe7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5c8104de0 0x7fa5c81982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.334+0000 7fa5ccfe7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5c8107710 0x7fa5c819f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.334+0000 7fa5ccfe7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c810d720 con 0x7fa5c81024b0 2024-06-10T01:20:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.334+0000 7fa5ccfe7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c810d5a0 con 0x7fa5c8104de0 2024-06-10T01:20:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.334+0000 7fa5ccfe7640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c810d8a0 con 0x7fa5c8107710 2024-06-10T01:20:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.335+0000 7fa5c659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5c81024b0 0x7fa5c8197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.335+0000 7fa5c659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5c81024b0 0x7fa5c8197d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46578/0 (socket says 172.21.15.7:46578) 2024-06-10T01:20:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.335+0000 7fa5c659c640 1 -- 172.21.15.7:0/3496117793 learned_addr learned my addr 172.21.15.7:0/3496117793 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.335+0000 7fa5c6d9d640 1 --2- 172.21.15.7:0/3496117793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5c8107710 0x7fa5c819f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.335+0000 7fa5bdd9b640 1 --2- 172.21.15.7:0/3496117793 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5c8104de0 0x7fa5c81982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.335+0000 7fa5c659c640 1 -- 172.21.15.7:0/3496117793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5c8107710 msgr2=0x7fa5c819f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.335+0000 7fa5c659c640 1 --2- 172.21.15.7:0/3496117793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5c8107710 0x7fa5c819f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.335+0000 7fa5c659c640 1 -- 172.21.15.7:0/3496117793 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5c8104de0 msgr2=0x7fa5c81982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.335+0000 7fa5c659c640 1 --2- 172.21.15.7:0/3496117793 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5c8104de0 0x7fa5c81982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.335+0000 7fa5c659c640 1 -- 172.21.15.7:0/3496117793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c819f980 con 0x7fa5c81024b0 2024-06-10T01:20:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.335+0000 7fa5bdd9b640 1 --2- 172.21.15.7:0/3496117793 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5c8104de0 0x7fa5c81982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:20:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.335+0000 7fa5c659c640 1 --2- 172.21.15.7:0/3496117793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5c81024b0 0x7fa5c8197d60 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8009230 tx=0x7fa5b8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.335+0000 7fa5bffff640 1 -- 172.21.15.7:0/3496117793 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b803f030 con 0x7fa5c81024b0 2024-06-10T01:20:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.335+0000 7fa5ccfe7640 1 -- 172.21.15.7:0/3496117793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c819fc10 con 0x7fa5c81024b0 2024-06-10T01:20:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.335+0000 7fa5bffff640 1 -- 172.21.15.7:0/3496117793 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b8003ce0 con 0x7fa5c81024b0 2024-06-10T01:20:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.335+0000 7fa5bffff640 1 -- 172.21.15.7:0/3496117793 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8035070 con 0x7fa5c81024b0 2024-06-10T01:20:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.335+0000 7fa5ccfe7640 1 -- 172.21.15.7:0/3496117793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c81a0100 con 0x7fa5c81024b0 2024-06-10T01:20:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.338+0000 7fa5bffff640 1 -- 172.21.15.7:0/3496117793 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa5b8042030 con 0x7fa5c81024b0 2024-06-10T01:20:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.338+0000 7fa5cfdd2740 1 Processor -- start 2024-06-10T01:20:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.338+0000 7fa5cfdd2740 1 -- start start 2024-06-10T01:20:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.338+0000 7fa5cfdd2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607b7afc010 0x5607b7afc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.338+0000 7fa5cfdd2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607b7afca00 0x5607b7b01500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.338+0000 7fa5cfdd2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607b7b01b70 0x5607b7b04000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.338+0000 7fa5cfdd2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5607b7a5f4f0 con 0x5607b7b01b70 2024-06-10T01:20:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.338+0000 7fa5cfdd2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5607b79fd870 con 0x5607b7afc010 2024-06-10T01:20:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.338+0000 7fa5cfdd2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5607b7a5b4b0 con 0x5607b7afca00 2024-06-10T01:20:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.338+0000 7fa5bdd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607b7b01b70 0x5607b7b04000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:39.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.339+0000 7fa5bdd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607b7b01b70 0x5607b7b04000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46590/0 (socket says 172.21.15.7:46590) 2024-06-10T01:20:39.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.339+0000 7fa5bdd9b640 1 -- 172.21.15.7:0/3806681102 learned_addr learned my addr 172.21.15.7:0/3806681102 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:39.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.339+0000 7fa5c659c640 1 --2- 172.21.15.7:0/3806681102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607b7afca00 0x5607b7b01500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:39.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.339+0000 7fa5bffff640 1 --2- 172.21.15.7:0/3496117793 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa5b40a2b80 0x7fa5b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:39.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.339+0000 7fa5c6d9d640 1 --2- 172.21.15.7:0/3806681102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607b7afc010 0x5607b7afc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:39.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.339+0000 7fa5bdd9b640 1 -- 172.21.15.7:0/3806681102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607b7afca00 msgr2=0x5607b7b01500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:39.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.339+0000 7fa5bdd9b640 1 --2- 172.21.15.7:0/3806681102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607b7afca00 0x5607b7b01500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.339+0000 7fa5bdd9b640 1 -- 172.21.15.7:0/3806681102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607b7afc010 msgr2=0x5607b7afc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:39.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.339+0000 7fa5bdd9b640 1 --2- 172.21.15.7:0/3806681102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607b7afc010 0x5607b7afc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.339+0000 7fa5bdd9b640 1 -- 172.21.15.7:0/3806681102 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607b7a199b0 con 0x5607b7b01b70 2024-06-10T01:20:39.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.339+0000 7fa5c659c640 1 --2- 172.21.15.7:0/3806681102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607b7afca00 0x5607b7b01500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:20:39.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.339+0000 7fa5c6d9d640 1 --2- 172.21.15.7:0/3806681102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607b7afc010 0x5607b7afc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:20:39.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.339+0000 7fa5c6d9d640 1 --2- 172.21.15.7:0/3496117793 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa5b40a2b80 0x7fa5b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.339+0000 7fa5bffff640 1 -- 172.21.15.7:0/3496117793 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa5b80efb60 con 0x7fa5c81024b0 2024-06-10T01:20:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.339+0000 7fa5bdd9b640 1 --2- 172.21.15.7:0/3806681102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607b7b01b70 0x5607b7b04000 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000cc40 tx=0x7fa5b000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c80ab8c54fc4411 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.339+0000 7fa5bcd99640 1 -- 172.21.15.7:0/3806681102 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0014070 con 0x5607b7b01b70 2024-06-10T01:20:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.339+0000 7fa5bcd99640 1 -- 172.21.15.7:0/3806681102 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b00047a0 con 0x5607b7b01b70 2024-06-10T01:20:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.339+0000 7fa5bcd99640 1 -- 172.21.15.7:0/3806681102 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b00106a0 con 0x5607b7b01b70 2024-06-10T01:20:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.340+0000 7fa5cfdd2740 1 -- 172.21.15.7:0/3806681102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607b7b01b70 msgr2=0x5607b7b04000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.340+0000 7fa5c6d9d640 1 --2- 172.21.15.7:0/3496117793 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa5b40a2b80 0x7fa5b40a5040 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x5607b7afd260 tx=0x7fa5c000d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.340+0000 7fa5cfdd2740 1 --2- 172.21.15.7:0/3806681102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607b7b01b70 0x5607b7b04000 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000cc40 tx=0x7fa5b000e720 comp rx=0 tx=0).stop 2024-06-10T01:20:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.340+0000 7fa5cfdd2740 1 -- 172.21.15.7:0/3806681102 shutdown_connections 2024-06-10T01:20:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.340+0000 7fa5cfdd2740 1 --2- 172.21.15.7:0/3806681102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607b7b01b70 0x5607b7b04000 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.340+0000 7fa5cfdd2740 1 --2- 172.21.15.7:0/3806681102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607b7afca00 0x5607b7b01500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.340+0000 7fa5cfdd2740 1 --2- 172.21.15.7:0/3806681102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607b7afc010 0x5607b7afc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.340+0000 7fa5cfdd2740 1 -- 172.21.15.7:0/3806681102 >> 172.21.15.7:0/3806681102 conn(0x5607b7af7680 msgr2=0x5607b7af7a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.340+0000 7fa5cfdd2740 1 -- 172.21.15.7:0/3806681102 shutdown_connections 2024-06-10T01:20:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.340+0000 7fa5cfdd2740 1 -- 172.21.15.7:0/3806681102 wait complete. 2024-06-10T01:20:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.340+0000 7fa5cfdd2740 1 -- start start 2024-06-10T01:20:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.340+0000 7fa5cfdd2740 1 Processor -- start 2024-06-10T01:20:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.341+0000 7fa5cfdd2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607b7afc010 0x5607b7b06220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.341+0000 7fa5cfdd2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607b7afca00 0x5607b7b06760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.341+0000 7fa5cfdd2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607b7b01b70 0x5607b7b0d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.341+0000 7fa5cfdd2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5607b7b04b20 con 0x5607b7afca00 2024-06-10T01:20:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.341+0000 7fa5cfdd2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5607b79e8ae0 con 0x5607b7afc010 2024-06-10T01:20:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.341+0000 7fa5cfdd2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5607b7b04ca0 con 0x5607b7b01b70 2024-06-10T01:20:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.341+0000 7fa5cfdd2740 10 client.0 _ll_register_callbacks cb 0x5607b7a219b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:20:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.341+0000 7fa5bdd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607b7afca00 0x5607b7b06760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.341+0000 7fa5bdd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607b7afca00 0x5607b7b06760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46594/0 (socket says 172.21.15.7:46594) 2024-06-10T01:20:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.341+0000 7fa5bdd9b640 1 -- 172.21.15.7:0/127224453 learned_addr learned my addr 172.21.15.7:0/127224453 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.341+0000 7fa5c659c640 1 --2- 172.21.15.7:0/127224453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607b7afc010 0x5607b7b06220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.341+0000 7fa5c6d9d640 1 --2- 172.21.15.7:0/127224453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607b7b01b70 0x5607b7b0d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.341+0000 7fa5bdd9b640 1 -- 172.21.15.7:0/127224453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607b7b01b70 msgr2=0x5607b7b0d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.341+0000 7fa5bdd9b640 1 --2- 172.21.15.7:0/127224453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607b7b01b70 0x5607b7b0d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.341+0000 7fa5bdd9b640 1 -- 172.21.15.7:0/127224453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607b7afc010 msgr2=0x5607b7b06220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.341+0000 7fa5bdd9b640 1 --2- 172.21.15.7:0/127224453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607b7afc010 0x5607b7b06220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.341+0000 7fa5bdd9b640 1 -- 172.21.15.7:0/127224453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607b7773210 con 0x5607b7afca00 2024-06-10T01:20:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.341+0000 7fa5bdd9b640 1 --2- 172.21.15.7:0/127224453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607b7afca00 0x5607b7b06760 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fa5b00058c0 tx=0x7fa5b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3a9e145dd68d02 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.341+0000 7fa5adffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:20:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.341+0000 7fa5adffb640 1 -- 172.21.15.7:0/127224453 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0027060 con 0x5607b7afca00 2024-06-10T01:20:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.342+0000 7fa5adffb640 1 -- 172.21.15.7:0/127224453 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa5b00104a0 con 0x5607b7afca00 2024-06-10T01:20:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.342+0000 7fa5cfdd2740 1 -- 172.21.15.7:0/127224453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5b00047a0 con 0x5607b7afca00 2024-06-10T01:20:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.342+0000 7fa5adffb640 1 -- 172.21.15.7:0/127224453 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa5b002e050 con 0x5607b7afca00 2024-06-10T01:20:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.342+0000 7fa5cfdd2740 10 client.19146 fetch_fsmap learned FSMap version 44 2024-06-10T01:20:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.342+0000 7fa5cfdd2740 1 -- 172.21.15.7:0/127224453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5607b783f620 con 0x5607b7afca00 2024-06-10T01:20:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.342+0000 7fa5adffb640 1 -- 172.21.15.7:0/127224453 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b00047a0 con 0x5607b7afca00 2024-06-10T01:20:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.342+0000 7fa5adffb640 1 -- 172.21.15.7:0/127224453 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fa5b0022640 con 0x5607b7afca00 2024-06-10T01:20:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.342+0000 7fa5cfdd2740 10 client.19146 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:20:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.342+0000 7fa5cfdd2740 10 client.19146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:20:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.342+0000 7fa5cfdd2740 20 client.19146 populate_metadata read hostname 'smithi007' 2024-06-10T01:20:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.342+0000 7fa5cfdd2740 1 --2- 172.21.15.7:0/127224453 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5607b7b11c60 0x5607b7b14050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.342+0000 7fa5cfdd2740 4 client.19146 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:20:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.342+0000 7fa5cfdd2740 1 -- 172.21.15.7:0/127224453 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607b7a72f10 con 0x5607b7b11c60 2024-06-10T01:20:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.343+0000 7fa5c659c640 1 --2- 172.21.15.7:0/127224453 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5607b7b11c60 0x5607b7b14050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.343+0000 7fa5adffb640 10 client.19146 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.343+0000 7fa5c659c640 1 --2- 172.21.15.7:0/127224453 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5607b7b11c60 0x5607b7b14050 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41616f4b421ed657 server_cookie=c731632dda0a5076 in_seq=0 out_seq=0 2024-06-10T01:20:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.344+0000 7fa5adffb640 1 -- 172.21.15.7:0/127224453 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5607b7a72f10 con 0x5607b7b11c60 2024-06-10T01:20:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.344+0000 7fa5adffb640 10 client.19146 handle_command_reply: tid=0 2024-06-10T01:20:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.344+0000 7fa5cfdd2740 1 client.19146 shutdown 2024-06-10T01:20:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.344+0000 7fa5cfdd2740 1 -- 172.21.15.7:0/127224453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607b7afca00 msgr2=0x5607b7b06760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.344+0000 7fa5cfdd2740 1 --2- 172.21.15.7:0/127224453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607b7afca00 0x5607b7b06760 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fa5b00058c0 tx=0x7fa5b0004730 comp rx=0 tx=0).stop 2024-06-10T01:20:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.344+0000 7fa5cfdd2740 1 -- 172.21.15.7:0/127224453 shutdown_connections 2024-06-10T01:20:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.344+0000 7fa5cfdd2740 1 --2- 172.21.15.7:0/127224453 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5607b7b11c60 0x5607b7b14050 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.344+0000 7fa5cfdd2740 1 --2- 172.21.15.7:0/127224453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607b7b01b70 0x5607b7b0d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.344+0000 7fa5adffb640 0 client.19146 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.344+0000 7fa5cfdd2740 1 --2- 172.21.15.7:0/127224453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607b7afca00 0x5607b7b06760 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.344+0000 7fa5cfdd2740 1 --2- 172.21.15.7:0/127224453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607b7afc010 0x5607b7b06220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.344+0000 7fa5cfdd2740 1 -- 172.21.15.7:0/127224453 >> 172.21.15.7:0/127224453 conn(0x5607b7af7680 msgr2=0x5607b7af67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.344+0000 7fa5cfdd2740 1 -- 172.21.15.7:0/127224453 shutdown_connections 2024-06-10T01:20:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.344+0000 7fa5cfdd2740 1 -- 172.21.15.7:0/127224453 wait complete. 2024-06-10T01:20:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.344+0000 7fa5cfdd2740 20 client.19146 trim_cache size 0 max 16384 2024-06-10T01:20:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.359+0000 7fa5cfdd2740 1 Processor -- start 2024-06-10T01:20:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.359+0000 7fa5cfdd2740 1 -- start start 2024-06-10T01:20:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.359+0000 7fa5cfdd2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607b7afc010 0x5607b7afdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.359+0000 7fa5cfdd2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607b7afca00 0x5607b7b0d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.359+0000 7fa5cfdd2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607b7b01b70 0x5607b7afe0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.359+0000 7fa5cfdd2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c81acf80 con 0x5607b7b01b70 2024-06-10T01:20:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.359+0000 7fa5cfdd2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c81aa7c0 con 0x5607b7afc010 2024-06-10T01:20:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.359+0000 7fa5cfdd2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c81a8610 con 0x5607b7afca00 2024-06-10T01:20:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.359+0000 7fa5c6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607b7b01b70 0x5607b7afe0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.359+0000 7fa5c6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607b7b01b70 0x5607b7afe0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46606/0 (socket says 172.21.15.7:46606) 2024-06-10T01:20:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.359+0000 7fa5c6d9d640 1 -- 172.21.15.7:0/2753536246 learned_addr learned my addr 172.21.15.7:0/2753536246 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.359+0000 7fa5bdd9b640 1 --2- 172.21.15.7:0/2753536246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607b7afca00 0x5607b7b0d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.360+0000 7fa5c659c640 1 --2- 172.21.15.7:0/2753536246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607b7afc010 0x5607b7afdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.360+0000 7fa5c6d9d640 1 -- 172.21.15.7:0/2753536246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607b7afca00 msgr2=0x5607b7b0d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.360+0000 7fa5c6d9d640 1 --2- 172.21.15.7:0/2753536246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607b7afca00 0x5607b7b0d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.360+0000 7fa5c6d9d640 1 -- 172.21.15.7:0/2753536246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607b7afc010 msgr2=0x5607b7afdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.360+0000 7fa5c6d9d640 1 --2- 172.21.15.7:0/2753536246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607b7afc010 0x5607b7afdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.360+0000 7fa5c6d9d640 1 -- 172.21.15.7:0/2753536246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607b7b143c0 con 0x5607b7b01b70 2024-06-10T01:20:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.360+0000 7fa5c659c640 1 --2- 172.21.15.7:0/2753536246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607b7afc010 0x5607b7afdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:20:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.360+0000 7fa5bdd9b640 1 --2- 172.21.15.7:0/2753536246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607b7afca00 0x5607b7b0d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:20:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.360+0000 7fa5c6d9d640 1 --2- 172.21.15.7:0/2753536246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607b7b01b70 0x5607b7afe0f0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fa5c00158c0 tx=0x7fa5c0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73de860a5d29ec9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.360+0000 7fa5affff640 1 -- 172.21.15.7:0/2753536246 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0014d60 con 0x5607b7b01b70 2024-06-10T01:20:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.360+0000 7fa5affff640 1 -- 172.21.15.7:0/2753536246 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5c0026070 con 0x5607b7b01b70 2024-06-10T01:20:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.360+0000 7fa5affff640 1 -- 172.21.15.7:0/2753536246 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c00205e0 con 0x5607b7b01b70 2024-06-10T01:20:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.360+0000 7fa5cfdd2740 1 -- 172.21.15.7:0/2753536246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607b7b01b70 msgr2=0x5607b7afe0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.360+0000 7fa5cfdd2740 1 --2- 172.21.15.7:0/2753536246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607b7b01b70 0x5607b7afe0f0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fa5c00158c0 tx=0x7fa5c0012000 comp rx=0 tx=0).stop 2024-06-10T01:20:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.361+0000 7fa5cfdd2740 1 -- 172.21.15.7:0/2753536246 shutdown_connections 2024-06-10T01:20:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.361+0000 7fa5cfdd2740 1 --2- 172.21.15.7:0/2753536246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607b7b01b70 0x5607b7afe0f0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.361+0000 7fa5cfdd2740 1 --2- 172.21.15.7:0/2753536246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607b7afca00 0x5607b7b0d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.361+0000 7fa5cfdd2740 1 --2- 172.21.15.7:0/2753536246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607b7afc010 0x5607b7afdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.361+0000 7fa5cfdd2740 1 -- 172.21.15.7:0/2753536246 >> 172.21.15.7:0/2753536246 conn(0x5607b7af7680 msgr2=0x5607b7b01240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.361+0000 7fa5cfdd2740 1 -- 172.21.15.7:0/2753536246 shutdown_connections 2024-06-10T01:20:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.361+0000 7fa5cfdd2740 1 -- 172.21.15.7:0/2753536246 wait complete. 2024-06-10T01:20:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.361+0000 7fa5cfdd2740 1 -- start start 2024-06-10T01:20:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.361+0000 7fa5cfdd2740 1 Processor -- start 2024-06-10T01:20:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.361+0000 7fa5cfdd2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607b7afc010 0x5607b7b0a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.361+0000 7fa5cfdd2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607b7afca00 0x5607b7b0aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.361+0000 7fa5cfdd2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607b7b01b70 0x5607b7b7a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.361+0000 7fa5cfdd2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c81a8c80 con 0x5607b7afca00 2024-06-10T01:20:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.361+0000 7fa5cfdd2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c81aa0f0 con 0x5607b7b01b70 2024-06-10T01:20:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.361+0000 7fa5cfdd2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c81a9350 con 0x5607b7afc010 2024-06-10T01:20:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.361+0000 7fa5cfdd2740 10 client.0 _ll_register_callbacks cb 0x7fa5b000ca90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:20:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.361+0000 7fa5bdd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607b7afca00 0x5607b7b0aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.361+0000 7fa5bdd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607b7afca00 0x5607b7b0aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46616/0 (socket says 172.21.15.7:46616) 2024-06-10T01:20:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.361+0000 7fa5bdd9b640 1 -- 172.21.15.7:0/371212189 learned_addr learned my addr 172.21.15.7:0/371212189 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.361+0000 7fa5c6d9d640 1 --2- 172.21.15.7:0/371212189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607b7b01b70 0x5607b7b7a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.361+0000 7fa5c659c640 1 --2- 172.21.15.7:0/371212189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607b7afc010 0x5607b7b0a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.362+0000 7fa5bdd9b640 1 -- 172.21.15.7:0/371212189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607b7afc010 msgr2=0x5607b7b0a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.362+0000 7fa5bdd9b640 1 --2- 172.21.15.7:0/371212189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607b7afc010 0x5607b7b0a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.362+0000 7fa5bdd9b640 1 -- 172.21.15.7:0/371212189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607b7b01b70 msgr2=0x5607b7b7a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.362+0000 7fa5bdd9b640 1 --2- 172.21.15.7:0/371212189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607b7b01b70 0x5607b7b7a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.362+0000 7fa5bdd9b640 1 -- 172.21.15.7:0/371212189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607b7b0b0c0 con 0x5607b7afca00 2024-06-10T01:20:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.362+0000 7fa5c6d9d640 1 --2- 172.21.15.7:0/371212189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607b7b01b70 0x5607b7b7a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:20:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.362+0000 7fa5c659c640 1 --2- 172.21.15.7:0/371212189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607b7afc010 0x5607b7b0a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:20:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.362+0000 7fa5bdd9b640 1 --2- 172.21.15.7:0/371212189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607b7afca00 0x5607b7b0aa80 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000e760 tx=0x7fa5b00057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ddaa40ca35428e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.362+0000 7fa5adffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:20:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.362+0000 7fa5adffb640 1 -- 172.21.15.7:0/371212189 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0027040 con 0x5607b7afca00 2024-06-10T01:20:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.362+0000 7fa5adffb640 1 -- 172.21.15.7:0/371212189 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa5b000ec20 con 0x5607b7afca00 2024-06-10T01:20:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.362+0000 7fa5cfdd2740 1 -- 172.21.15.7:0/371212189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5c0026070 con 0x5607b7afca00 2024-06-10T01:20:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.362+0000 7fa5adffb640 1 -- 172.21.15.7:0/371212189 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa5b0014070 con 0x5607b7afca00 2024-06-10T01:20:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.362+0000 7fa5cfdd2740 10 client.19152 fetch_fsmap learned FSMap version 44 2024-06-10T01:20:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.362+0000 7fa5cfdd2740 1 -- 172.21.15.7:0/371212189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5607b7b7bf50 con 0x5607b7afca00 2024-06-10T01:20:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.362+0000 7fa5adffb640 1 -- 172.21.15.7:0/371212189 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5c0026070 con 0x5607b7afca00 2024-06-10T01:20:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.362+0000 7fa5adffb640 1 -- 172.21.15.7:0/371212189 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fa5b0004220 con 0x5607b7afca00 2024-06-10T01:20:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.362+0000 7fa5cfdd2740 10 client.19152 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:20:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.362+0000 7fa5cfdd2740 10 client.19152 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:20:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.362+0000 7fa5cfdd2740 20 client.19152 populate_metadata read hostname 'smithi007' 2024-06-10T01:20:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.362+0000 7fa5cfdd2740 1 --2- 172.21.15.7:0/371212189 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5607b7b7c9b0 0x5607b7b7edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.362+0000 7fa5cfdd2740 4 client.19152 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:20:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.362+0000 7fa5cfdd2740 1 -- 172.21.15.7:0/371212189 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5607b7a6cc20 con 0x5607b7b7c9b0 2024-06-10T01:20:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.363+0000 7fa5c659c640 1 --2- 172.21.15.7:0/371212189 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5607b7b7c9b0 0x5607b7b7edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.363+0000 7fa5c659c640 1 --2- 172.21.15.7:0/371212189 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5607b7b7c9b0 0x5607b7b7edd0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d0f95589ff18c8 server_cookie=b71f63c7ff24f8d7 in_seq=0 out_seq=0 2024-06-10T01:20:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.363+0000 7fa5adffb640 10 client.19152 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:39.472 INFO:tasks.workunit.client.0.smithi007.stdout: 10 19792 16.35 MB/sec warmup 76 sec latency 9068.214 ms 2024-06-10T01:20:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.532+0000 7fa5adffb640 1 -- 172.21.15.7:0/371212189 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5607b7a6cc20 con 0x5607b7b7c9b0 2024-06-10T01:20:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.532+0000 7fa5adffb640 10 client.19152 handle_command_reply: tid=0 2024-06-10T01:20:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.532+0000 7fa5cfdd2740 1 client.19152 shutdown 2024-06-10T01:20:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.532+0000 7fa5cfdd2740 1 -- 172.21.15.7:0/371212189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607b7afca00 msgr2=0x5607b7b0aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.532+0000 7fa5cfdd2740 1 --2- 172.21.15.7:0/371212189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607b7afca00 0x5607b7b0aa80 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000e760 tx=0x7fa5b00057d0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.532+0000 7fa5cfdd2740 1 -- 172.21.15.7:0/371212189 shutdown_connections 2024-06-10T01:20:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.532+0000 7fa5cfdd2740 1 --2- 172.21.15.7:0/371212189 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5607b7b7c9b0 0x5607b7b7edd0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.532+0000 7fa5cfdd2740 1 --2- 172.21.15.7:0/371212189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607b7b01b70 0x5607b7b7a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.532+0000 7fa5cfdd2740 1 --2- 172.21.15.7:0/371212189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607b7afca00 0x5607b7b0aa80 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.532+0000 7fa5cfdd2740 1 --2- 172.21.15.7:0/371212189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607b7afc010 0x5607b7b0a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.532+0000 7fa5cfdd2740 1 -- 172.21.15.7:0/371212189 >> 172.21.15.7:0/371212189 conn(0x5607b7af7680 msgr2=0x5607b7b00e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.532+0000 7fa5adffb640 0 client.19152 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.533+0000 7fa5cfdd2740 1 -- 172.21.15.7:0/371212189 shutdown_connections 2024-06-10T01:20:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.533+0000 7fa5cfdd2740 1 -- 172.21.15.7:0/371212189 wait complete. 2024-06-10T01:20:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.533+0000 7fa5cfdd2740 20 client.19152 trim_cache size 0 max 16384 2024-06-10T01:20:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:20:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:20:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:20:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.533+0000 7fa5ccfe7640 1 -- 172.21.15.7:0/3496117793 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa5b40a2b80 msgr2=0x7fa5b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.533+0000 7fa5ccfe7640 1 --2- 172.21.15.7:0/3496117793 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa5b40a2b80 0x7fa5b40a5040 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x5607b7afd260 tx=0x7fa5c000d040 comp rx=0 tx=0).stop 2024-06-10T01:20:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.533+0000 7fa5ccfe7640 1 -- 172.21.15.7:0/3496117793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5c81024b0 msgr2=0x7fa5c8197d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.533+0000 7fa5ccfe7640 1 --2- 172.21.15.7:0/3496117793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5c81024b0 0x7fa5c8197d60 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8009230 tx=0x7fa5b8004430 comp rx=0 tx=0).stop 2024-06-10T01:20:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.533+0000 7fa5ccfe7640 1 -- 172.21.15.7:0/3496117793 shutdown_connections 2024-06-10T01:20:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.533+0000 7fa5ccfe7640 1 --2- 172.21.15.7:0/3496117793 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa5b40a2b80 0x7fa5b40a5040 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.533+0000 7fa5ccfe7640 1 --2- 172.21.15.7:0/3496117793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5c8107710 0x7fa5c819f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.533+0000 7fa5ccfe7640 1 --2- 172.21.15.7:0/3496117793 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5c8104de0 0x7fa5c81982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.533+0000 7fa5ccfe7640 1 --2- 172.21.15.7:0/3496117793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5c81024b0 0x7fa5c8197d60 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.533+0000 7fa5ccfe7640 1 -- 172.21.15.7:0/3496117793 >> 172.21.15.7:0/3496117793 conn(0x7fa5c80fbec0 msgr2=0x7fa5c8102d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.533+0000 7fa5ccfe7640 1 -- 172.21.15.7:0/3496117793 shutdown_connections 2024-06-10T01:20:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:39.534+0000 7fa5ccfe7640 1 -- 172.21.15.7:0/3496117793 wait complete. 2024-06-10T01:20:39.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:39 smithi007 ceph-mon[32398]: pgmap v871: 129 pgs: 129 active+clean; 379 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 2.9 MiB/s wr, 434 op/s 2024-06-10T01:20:39.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:39 smithi078 ceph-mon[36683]: pgmap v871: 129 pgs: 129 active+clean; 379 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 2.9 MiB/s wr, 434 op/s 2024-06-10T01:20:39.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:39 smithi175 ceph-mon[36636]: pgmap v871: 129 pgs: 129 active+clean; 379 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 2.9 MiB/s wr, 434 op/s 2024-06-10T01:20:40.472 INFO:tasks.workunit.client.0.smithi007.stdout: 10 19792 16.13 MB/sec warmup 77 sec latency 10068.328 ms 2024-06-10T01:20:41.473 INFO:tasks.workunit.client.0.smithi007.stdout: 10 19792 15.93 MB/sec warmup 78 sec latency 11068.412 ms 2024-06-10T01:20:41.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:41 smithi007 ceph-mon[32398]: pgmap v872: 129 pgs: 129 active+clean; 375 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.3 MiB/s wr, 371 op/s 2024-06-10T01:20:41.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:41 smithi078 ceph-mon[36683]: pgmap v872: 129 pgs: 129 active+clean; 375 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.3 MiB/s wr, 371 op/s 2024-06-10T01:20:41.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:41 smithi175 ceph-mon[36636]: pgmap v872: 129 pgs: 129 active+clean; 375 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.3 MiB/s wr, 371 op/s 2024-06-10T01:20:42.473 INFO:tasks.workunit.client.0.smithi007.stdout: 10 19792 15.73 MB/sec warmup 79 sec latency 12068.487 ms 2024-06-10T01:20:43.473 INFO:tasks.workunit.client.0.smithi007.stdout: 10 19792 15.53 MB/sec warmup 80 sec latency 13068.535 ms 2024-06-10T01:20:43.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:43 smithi007 ceph-mon[32398]: pgmap v873: 129 pgs: 129 active+clean; 375 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.2 MiB/s wr, 221 op/s 2024-06-10T01:20:43.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:43 smithi078 ceph-mon[36683]: pgmap v873: 129 pgs: 129 active+clean; 375 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.2 MiB/s wr, 221 op/s 2024-06-10T01:20:43.953 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:43 smithi175 ceph-mon[36636]: pgmap v873: 129 pgs: 129 active+clean; 375 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.2 MiB/s wr, 221 op/s 2024-06-10T01:20:44.473 INFO:tasks.workunit.client.0.smithi007.stdout: 10 19792 15.34 MB/sec warmup 81 sec latency 14068.591 ms 2024-06-10T01:20:45.473 INFO:tasks.workunit.client.0.smithi007.stdout: 10 19792 15.15 MB/sec warmup 82 sec latency 15068.635 ms 2024-06-10T01:20:45.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:45 smithi007 ceph-mon[32398]: pgmap v874: 129 pgs: 129 active+clean; 400 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 163 op/s 2024-06-10T01:20:45.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:45 smithi078 ceph-mon[36683]: pgmap v874: 129 pgs: 129 active+clean; 400 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 163 op/s 2024-06-10T01:20:45.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:45 smithi175 ceph-mon[36636]: pgmap v874: 129 pgs: 129 active+clean; 400 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 163 op/s 2024-06-10T01:20:46.473 INFO:tasks.workunit.client.0.smithi007.stdout: 10 19792 14.97 MB/sec warmup 83 sec latency 16068.681 ms 2024-06-10T01:20:47.473 INFO:tasks.workunit.client.0.smithi007.stdout: 10 19792 14.79 MB/sec warmup 84 sec latency 17068.729 ms 2024-06-10T01:20:47.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:47 smithi007 ceph-mon[32398]: pgmap v875: 129 pgs: 129 active+clean; 416 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 4.4 MiB/s wr, 179 op/s 2024-06-10T01:20:47.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:47 smithi078 ceph-mon[36683]: pgmap v875: 129 pgs: 129 active+clean; 416 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 4.4 MiB/s wr, 179 op/s 2024-06-10T01:20:47.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:47 smithi175 ceph-mon[36636]: pgmap v875: 129 pgs: 129 active+clean; 416 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 4.4 MiB/s wr, 179 op/s 2024-06-10T01:20:48.473 INFO:tasks.workunit.client.0.smithi007.stdout: 10 19792 14.62 MB/sec warmup 85 sec latency 18068.789 ms 2024-06-10T01:20:49.474 INFO:tasks.workunit.client.0.smithi007.stdout: 10 19792 14.45 MB/sec warmup 86 sec latency 19068.888 ms 2024-06-10T01:20:49.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:49 smithi007 ceph-mon[32398]: pgmap v876: 129 pgs: 129 active+clean; 424 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 5.9 MiB/s wr, 188 op/s 2024-06-10T01:20:49.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:49 smithi078 ceph-mon[36683]: pgmap v876: 129 pgs: 129 active+clean; 424 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 5.9 MiB/s wr, 188 op/s 2024-06-10T01:20:49.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:49 smithi175 ceph-mon[36636]: pgmap v876: 129 pgs: 129 active+clean; 424 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 5.9 MiB/s wr, 188 op/s 2024-06-10T01:20:50.473 INFO:tasks.workunit.client.0.smithi007.stdout: 10 19792 14.28 MB/sec warmup 87 sec latency 20068.990 ms 2024-06-10T01:20:50.779 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:20:51.046 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:20:51.473 INFO:tasks.workunit.client.0.smithi007.stdout: 10 19792 14.12 MB/sec warmup 88 sec latency 21069.046 ms 2024-06-10T01:20:51.752 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:51 smithi175 ceph-mon[36636]: pgmap v877: 129 pgs: 129 active+clean; 388 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 5.9 MiB/s wr, 248 op/s 2024-06-10T01:20:51.752 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:51 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:20:51.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:51 smithi007 ceph-mon[32398]: pgmap v877: 129 pgs: 129 active+clean; 388 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 5.9 MiB/s wr, 248 op/s 2024-06-10T01:20:51.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:51 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:20:51.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:51 smithi078 ceph-mon[36683]: pgmap v877: 129 pgs: 129 active+clean; 388 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 5.9 MiB/s wr, 248 op/s 2024-06-10T01:20:51.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:51 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:20:52.473 INFO:tasks.workunit.client.0.smithi007.stdout: 10 19792 13.96 MB/sec warmup 89 sec latency 22069.091 ms 2024-06-10T01:20:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.628+0000 7f7ba1e91640 1 -- 172.21.15.7:0/3268379410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9c1024c0 msgr2=0x7f7b9c108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.628+0000 7f7ba1e91640 1 --2- 172.21.15.7:0/3268379410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9c1024c0 0x7f7b9c108d50 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009ca0 tx=0x7f7b9402a8c0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.628+0000 7f7ba1e91640 1 -- 172.21.15.7:0/3268379410 shutdown_connections 2024-06-10T01:20:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.628+0000 7f7ba1e91640 1 --2- 172.21.15.7:0/3268379410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9c1024c0 0x7f7b9c108d50 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.628+0000 7f7ba1e91640 1 --2- 172.21.15.7:0/3268379410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b9c101b00 0x7f7b9c101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.628+0000 7f7ba1e91640 1 --2- 172.21.15.7:0/3268379410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b9c100910 0x7f7b9c100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.628+0000 7f7ba1e91640 1 -- 172.21.15.7:0/3268379410 >> 172.21.15.7:0/3268379410 conn(0x7f7b9c0fbe80 msgr2=0x7f7b9c0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.628+0000 7f7ba1e91640 1 -- 172.21.15.7:0/3268379410 shutdown_connections 2024-06-10T01:20:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.628+0000 7f7ba1e91640 1 -- 172.21.15.7:0/3268379410 wait complete. 2024-06-10T01:20:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.629+0000 7f7ba1e91640 1 Processor -- start 2024-06-10T01:20:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.629+0000 7f7ba1e91640 1 -- start start 2024-06-10T01:20:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.629+0000 7f7ba1e91640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9c100910 0x7f7b9c06abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.629+0000 7f7ba1e91640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b9c101b00 0x7f7b9c19c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.629+0000 7f7ba1e91640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b9c1024c0 0x7f7b9c19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.629+0000 7f7ba1e91640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c10c590 con 0x7f7b9c100910 2024-06-10T01:20:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.629+0000 7f7ba1e91640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c10c410 con 0x7f7b9c101b00 2024-06-10T01:20:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.629+0000 7f7ba1e91640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c10c710 con 0x7f7b9c1024c0 2024-06-10T01:20:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.629+0000 7f7b9b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9c100910 0x7f7b9c06abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.629+0000 7f7b9b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9c100910 0x7f7b9c06abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49214/0 (socket says 172.21.15.7:49214) 2024-06-10T01:20:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.629+0000 7f7b9b7fe640 1 -- 172.21.15.7:0/1898186879 learned_addr learned my addr 172.21.15.7:0/1898186879 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.630+0000 7f7b9affd640 1 --2- 172.21.15.7:0/1898186879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b9c101b00 0x7f7b9c19c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.630+0000 7f7b9bfff640 1 --2- 172.21.15.7:0/1898186879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b9c1024c0 0x7f7b9c19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.630+0000 7f7b9b7fe640 1 -- 172.21.15.7:0/1898186879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b9c1024c0 msgr2=0x7f7b9c19f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.630+0000 7f7b9b7fe640 1 --2- 172.21.15.7:0/1898186879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b9c1024c0 0x7f7b9c19f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.630+0000 7f7b9b7fe640 1 -- 172.21.15.7:0/1898186879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b9c101b00 msgr2=0x7f7b9c19c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.630+0000 7f7b9b7fe640 1 --2- 172.21.15.7:0/1898186879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b9c101b00 0x7f7b9c19c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.630+0000 7f7b9b7fe640 1 -- 172.21.15.7:0/1898186879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9c19f8d0 con 0x7f7b9c100910 2024-06-10T01:20:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.630+0000 7f7b9affd640 1 --2- 172.21.15.7:0/1898186879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b9c101b00 0x7f7b9c19c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:20:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.630+0000 7f7b9b7fe640 1 --2- 172.21.15.7:0/1898186879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9c100910 0x7f7b9c06abd0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00d950 tx=0x7f7b8c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.630+0000 7f7b98ff9640 1 -- 172.21.15.7:0/1898186879 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c017070 con 0x7f7b9c100910 2024-06-10T01:20:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.630+0000 7f7b98ff9640 1 -- 172.21.15.7:0/1898186879 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b8c009c30 con 0x7f7b9c100910 2024-06-10T01:20:52.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.630+0000 7f7ba1e91640 1 -- 172.21.15.7:0/1898186879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b9c19fbc0 con 0x7f7b9c100910 2024-06-10T01:20:52.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.630+0000 7f7b98ff9640 1 -- 172.21.15.7:0/1898186879 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c00f600 con 0x7f7b9c100910 2024-06-10T01:20:52.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.630+0000 7f7ba1e91640 1 -- 172.21.15.7:0/1898186879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b9c1a00c0 con 0x7f7b9c100910 2024-06-10T01:20:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.632+0000 7f7b98ff9640 1 -- 172.21.15.7:0/1898186879 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f7b8c01a070 con 0x7f7b9c100910 2024-06-10T01:20:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.633+0000 7f7ba4b95740 1 Processor -- start 2024-06-10T01:20:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.633+0000 7f7ba4b95740 1 -- start start 2024-06-10T01:20:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.633+0000 7f7ba4b95740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555be315e010 0x555be315e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.633+0000 7f7ba4b95740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555be315ea00 0x555be3163500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.633+0000 7f7ba4b95740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555be3163b70 0x555be3166000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.633+0000 7f7ba4b95740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555be30c14f0 con 0x555be315ea00 2024-06-10T01:20:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.633+0000 7f7ba4b95740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555be305f870 con 0x555be3163b70 2024-06-10T01:20:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.633+0000 7f7ba4b95740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555be30bd4b0 con 0x555be315e010 2024-06-10T01:20:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.633+0000 7f7b9b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555be315ea00 0x555be3163500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.633+0000 7f7b9b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555be315ea00 0x555be3163500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49228/0 (socket says 172.21.15.7:49228) 2024-06-10T01:20:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.633+0000 7f7b9b7fe640 1 -- 172.21.15.7:0/3605649920 learned_addr learned my addr 172.21.15.7:0/3605649920 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.633+0000 7f7b9bfff640 1 --2- 172.21.15.7:0/3605649920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555be315e010 0x555be315e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.633+0000 7f7b98ff9640 1 --2- 172.21.15.7:0/1898186879 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7b840a2b80 0x7f7b840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.633+0000 7f7b9affd640 1 --2- 172.21.15.7:0/3605649920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555be3163b70 0x555be3166000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.633+0000 7f7b9b7fe640 1 -- 172.21.15.7:0/3605649920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555be315e010 msgr2=0x555be315e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.633+0000 7f7b9b7fe640 1 --2- 172.21.15.7:0/3605649920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555be315e010 0x555be315e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.633+0000 7f7b9b7fe640 1 -- 172.21.15.7:0/3605649920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555be3163b70 msgr2=0x555be3166000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.633+0000 7f7b9b7fe640 1 --2- 172.21.15.7:0/3605649920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555be3163b70 0x555be3166000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.633+0000 7f7b9bfff640 1 --2- 172.21.15.7:0/1898186879 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7b840a2b80 0x7f7b840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.633+0000 7f7b98ff9640 1 -- 172.21.15.7:0/1898186879 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f7b8c0cef00 con 0x7f7b9c100910 2024-06-10T01:20:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.633+0000 7f7b9b7fe640 1 -- 172.21.15.7:0/3605649920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555be307b9b0 con 0x555be315ea00 2024-06-10T01:20:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.634+0000 7f7b9b7fe640 1 --2- 172.21.15.7:0/3605649920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555be315ea00 0x555be3163500 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0cc680 tx=0x7f7b8c010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f072980224f2220 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.634+0000 7f7b81ffb640 1 -- 172.21.15.7:0/3605649920 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c021030 con 0x555be315ea00 2024-06-10T01:20:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.634+0000 7f7b81ffb640 1 -- 172.21.15.7:0/3605649920 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b8c009de0 con 0x555be315ea00 2024-06-10T01:20:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.634+0000 7f7b81ffb640 1 -- 172.21.15.7:0/3605649920 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c09b3e0 con 0x555be315ea00 2024-06-10T01:20:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.634+0000 7f7ba4b95740 1 -- 172.21.15.7:0/3605649920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555be315ea00 msgr2=0x555be3163500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.634+0000 7f7ba4b95740 1 --2- 172.21.15.7:0/3605649920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555be315ea00 0x555be3163500 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0cc680 tx=0x7f7b8c010ce0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.634+0000 7f7b9bfff640 1 --2- 172.21.15.7:0/1898186879 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7b840a2b80 0x7f7b840a5040 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7b94006010 tx=0x7f7b9403e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.634+0000 7f7ba4b95740 1 -- 172.21.15.7:0/3605649920 shutdown_connections 2024-06-10T01:20:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.634+0000 7f7ba4b95740 1 --2- 172.21.15.7:0/3605649920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555be3163b70 0x555be3166000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.634+0000 7f7ba4b95740 1 --2- 172.21.15.7:0/3605649920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555be315ea00 0x555be3163500 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.634+0000 7f7ba4b95740 1 --2- 172.21.15.7:0/3605649920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555be315e010 0x555be315e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.634+0000 7f7ba4b95740 1 -- 172.21.15.7:0/3605649920 >> 172.21.15.7:0/3605649920 conn(0x555be3159680 msgr2=0x555be3159a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.635+0000 7f7ba4b95740 1 -- 172.21.15.7:0/3605649920 shutdown_connections 2024-06-10T01:20:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.635+0000 7f7ba4b95740 1 -- 172.21.15.7:0/3605649920 wait complete. 2024-06-10T01:20:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.635+0000 7f7ba4b95740 1 -- start start 2024-06-10T01:20:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.635+0000 7f7ba4b95740 1 Processor -- start 2024-06-10T01:20:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.635+0000 7f7ba4b95740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555be315e010 0x555be31681f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.635+0000 7f7ba4b95740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555be315ea00 0x555be3168730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.635+0000 7f7ba4b95740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555be3163b70 0x555be316f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.635+0000 7f7ba4b95740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555be3166a60 con 0x555be315ea00 2024-06-10T01:20:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.635+0000 7f7ba4b95740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555be304aae0 con 0x555be3163b70 2024-06-10T01:20:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.635+0000 7f7ba4b95740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555be3166be0 con 0x555be315e010 2024-06-10T01:20:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.635+0000 7f7ba4b95740 10 client.0 _ll_register_callbacks cb 0x555be30839b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:20:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.635+0000 7f7b9affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555be315ea00 0x555be3168730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.635+0000 7f7b9affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555be315ea00 0x555be3168730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49244/0 (socket says 172.21.15.7:49244) 2024-06-10T01:20:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.635+0000 7f7b9affd640 1 -- 172.21.15.7:0/504354851 learned_addr learned my addr 172.21.15.7:0/504354851 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.636+0000 7f7b9b7fe640 1 --2- 172.21.15.7:0/504354851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555be315e010 0x555be31681f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.636+0000 7f7b9bfff640 1 --2- 172.21.15.7:0/504354851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555be3163b70 0x555be316f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.636+0000 7f7b9affd640 1 -- 172.21.15.7:0/504354851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555be315e010 msgr2=0x555be31681f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.636+0000 7f7b9affd640 1 --2- 172.21.15.7:0/504354851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555be315e010 0x555be31681f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.636+0000 7f7b9affd640 1 -- 172.21.15.7:0/504354851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555be3163b70 msgr2=0x555be316f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.636+0000 7f7b9affd640 1 --2- 172.21.15.7:0/504354851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555be3163b70 0x555be316f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.636+0000 7f7b9affd640 1 -- 172.21.15.7:0/504354851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555be2dd5210 con 0x555be315ea00 2024-06-10T01:20:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.636+0000 7f7b9bfff640 1 --2- 172.21.15.7:0/504354851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555be3163b70 0x555be316f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:20:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.636+0000 7f7b9affd640 1 --2- 172.21.15.7:0/504354851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555be315ea00 0x555be3168730 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f7b90013790 tx=0x7f7b90013c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b3f02717e781d17 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.636+0000 7f7b7affd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:20:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.636+0000 7f7b7affd640 1 -- 172.21.15.7:0/504354851 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b90020060 con 0x555be315ea00 2024-06-10T01:20:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.636+0000 7f7b7affd640 1 -- 172.21.15.7:0/504354851 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f7b90011d70 con 0x555be315ea00 2024-06-10T01:20:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.636+0000 7f7ba4b95740 1 -- 172.21.15.7:0/504354851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b8c009de0 con 0x555be315ea00 2024-06-10T01:20:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.636+0000 7f7b7affd640 1 -- 172.21.15.7:0/504354851 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f7b90027050 con 0x555be315ea00 2024-06-10T01:20:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.636+0000 7f7ba4b95740 10 client.19164 fetch_fsmap learned FSMap version 44 2024-06-10T01:20:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.636+0000 7f7ba4b95740 1 -- 172.21.15.7:0/504354851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x555be2ea1620 con 0x555be315ea00 2024-06-10T01:20:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.636+0000 7f7b7affd640 1 -- 172.21.15.7:0/504354851 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b8c009de0 con 0x555be315ea00 2024-06-10T01:20:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.637+0000 7f7b7affd640 1 -- 172.21.15.7:0/504354851 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f7b9001a030 con 0x555be315ea00 2024-06-10T01:20:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.637+0000 7f7ba4b95740 10 client.19164 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:20:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.637+0000 7f7ba4b95740 10 client.19164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:20:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.637+0000 7f7ba4b95740 20 client.19164 populate_metadata read hostname 'smithi007' 2024-06-10T01:20:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.637+0000 7f7ba4b95740 1 --2- 172.21.15.7:0/504354851 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555be3174190 0x555be31765c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.637+0000 7f7ba4b95740 4 client.19164 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:20:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.637+0000 7f7ba4b95740 1 -- 172.21.15.7:0/504354851 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555be30d4f10 con 0x555be3174190 2024-06-10T01:20:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.637+0000 7f7b9b7fe640 1 --2- 172.21.15.7:0/504354851 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555be3174190 0x555be31765c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.637+0000 7f7b9b7fe640 1 --2- 172.21.15.7:0/504354851 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555be3174190 0x555be31765c0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf9dbf979ca6fbae server_cookie=31daf8edf6ae753c in_seq=0 out_seq=0 2024-06-10T01:20:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.637+0000 7f7b7affd640 10 client.19164 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.638+0000 7f7b7affd640 1 -- 172.21.15.7:0/504354851 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555be30d4f10 con 0x555be3174190 2024-06-10T01:20:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.638+0000 7f7b7affd640 10 client.19164 handle_command_reply: tid=0 2024-06-10T01:20:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.638+0000 7f7ba4b95740 1 client.19164 shutdown 2024-06-10T01:20:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.638+0000 7f7ba4b95740 1 -- 172.21.15.7:0/504354851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555be315ea00 msgr2=0x555be3168730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.638+0000 7f7ba4b95740 1 --2- 172.21.15.7:0/504354851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555be315ea00 0x555be3168730 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f7b90013790 tx=0x7f7b90013c50 comp rx=0 tx=0).stop 2024-06-10T01:20:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.638+0000 7f7ba4b95740 1 -- 172.21.15.7:0/504354851 shutdown_connections 2024-06-10T01:20:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.638+0000 7f7ba4b95740 1 --2- 172.21.15.7:0/504354851 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555be3174190 0x555be31765c0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.638+0000 7f7ba4b95740 1 --2- 172.21.15.7:0/504354851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555be3163b70 0x555be316f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.638+0000 7f7ba4b95740 1 --2- 172.21.15.7:0/504354851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555be315ea00 0x555be3168730 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.638+0000 7f7b7affd640 0 client.19164 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.638+0000 7f7ba4b95740 1 --2- 172.21.15.7:0/504354851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555be315e010 0x555be31681f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.638+0000 7f7ba4b95740 1 -- 172.21.15.7:0/504354851 >> 172.21.15.7:0/504354851 conn(0x555be3159680 msgr2=0x555be31587e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.638+0000 7f7ba4b95740 1 -- 172.21.15.7:0/504354851 shutdown_connections 2024-06-10T01:20:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.638+0000 7f7ba4b95740 1 -- 172.21.15.7:0/504354851 wait complete. 2024-06-10T01:20:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.639+0000 7f7ba4b95740 20 client.19164 trim_cache size 0 max 16384 2024-06-10T01:20:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.654+0000 7f7ba4b95740 1 Processor -- start 2024-06-10T01:20:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.654+0000 7f7ba4b95740 1 -- start start 2024-06-10T01:20:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.654+0000 7f7ba4b95740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555be315e010 0x555be315c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.654+0000 7f7ba4b95740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555be315ea00 0x555be316f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.654+0000 7f7ba4b95740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555be3163b70 0x555be315c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.654+0000 7f7ba4b95740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c1acf00 con 0x555be3163b70 2024-06-10T01:20:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.654+0000 7f7ba4b95740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c1aa740 con 0x555be315e010 2024-06-10T01:20:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.654+0000 7f7ba4b95740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c1a8590 con 0x555be315ea00 2024-06-10T01:20:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.654+0000 7f7b9bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555be3163b70 0x555be315c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.654+0000 7f7b9bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555be3163b70 0x555be315c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49252/0 (socket says 172.21.15.7:49252) 2024-06-10T01:20:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.654+0000 7f7b9bfff640 1 -- 172.21.15.7:0/2436169356 learned_addr learned my addr 172.21.15.7:0/2436169356 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.655+0000 7f7b9affd640 1 --2- 172.21.15.7:0/2436169356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555be315ea00 0x555be316f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.655+0000 7f7b9b7fe640 1 --2- 172.21.15.7:0/2436169356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555be315e010 0x555be315c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.655+0000 7f7b9bfff640 1 -- 172.21.15.7:0/2436169356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555be315ea00 msgr2=0x555be316f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:52.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.655+0000 7f7b9bfff640 1 --2- 172.21.15.7:0/2436169356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555be315ea00 0x555be316f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.655+0000 7f7b9bfff640 1 -- 172.21.15.7:0/2436169356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555be315e010 msgr2=0x555be315c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:52.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.655+0000 7f7b9bfff640 1 --2- 172.21.15.7:0/2436169356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555be315e010 0x555be315c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.655+0000 7f7b9bfff640 1 -- 172.21.15.7:0/2436169356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555be31594b0 con 0x555be3163b70 2024-06-10T01:20:52.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.655+0000 7f7b9b7fe640 1 --2- 172.21.15.7:0/2436169356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555be315e010 0x555be315c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:20:52.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.655+0000 7f7b9bfff640 1 --2- 172.21.15.7:0/2436169356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555be3163b70 0x555be315c8a0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009ca0 tx=0x7f7b94040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da117c99fbd442b0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:52.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.655+0000 7f7b817fa640 1 -- 172.21.15.7:0/2436169356 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b94003e40 con 0x555be3163b70 2024-06-10T01:20:52.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.655+0000 7f7b817fa640 1 -- 172.21.15.7:0/2436169356 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b94036040 con 0x555be3163b70 2024-06-10T01:20:52.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.655+0000 7f7b817fa640 1 -- 172.21.15.7:0/2436169356 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9402fb10 con 0x555be3163b70 2024-06-10T01:20:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.656+0000 7f7ba4b95740 1 -- 172.21.15.7:0/2436169356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555be3163b70 msgr2=0x555be315c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.656+0000 7f7ba4b95740 1 --2- 172.21.15.7:0/2436169356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555be3163b70 0x555be315c8a0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009ca0 tx=0x7f7b94040000 comp rx=0 tx=0).stop 2024-06-10T01:20:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.656+0000 7f7ba4b95740 1 -- 172.21.15.7:0/2436169356 shutdown_connections 2024-06-10T01:20:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.656+0000 7f7ba4b95740 1 --2- 172.21.15.7:0/2436169356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555be3163b70 0x555be315c8a0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.656+0000 7f7ba4b95740 1 --2- 172.21.15.7:0/2436169356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555be315ea00 0x555be316f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.656+0000 7f7ba4b95740 1 --2- 172.21.15.7:0/2436169356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555be315e010 0x555be315c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.656+0000 7f7ba4b95740 1 -- 172.21.15.7:0/2436169356 >> 172.21.15.7:0/2436169356 conn(0x555be3159680 msgr2=0x555be315fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.656+0000 7f7ba4b95740 1 -- 172.21.15.7:0/2436169356 shutdown_connections 2024-06-10T01:20:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.656+0000 7f7ba4b95740 1 -- 172.21.15.7:0/2436169356 wait complete. 2024-06-10T01:20:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.656+0000 7f7ba4b95740 1 -- start start 2024-06-10T01:20:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.656+0000 7f7ba4b95740 1 Processor -- start 2024-06-10T01:20:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.656+0000 7f7ba4b95740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555be315e010 0x555be316cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.656+0000 7f7ba4b95740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555be315ea00 0x555be316d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.656+0000 7f7ba4b95740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555be3163b70 0x555be316e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.657+0000 7f7ba4b95740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c1a8c00 con 0x555be3163b70 2024-06-10T01:20:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.657+0000 7f7ba4b95740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c1aa070 con 0x555be315e010 2024-06-10T01:20:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.657+0000 7f7ba4b95740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c1a92d0 con 0x555be315ea00 2024-06-10T01:20:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.657+0000 7f7ba4b95740 10 client.0 _ll_register_callbacks cb 0x7f7b900096d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:20:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.657+0000 7f7b9bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555be3163b70 0x555be316e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.657+0000 7f7b9bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555be3163b70 0x555be316e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49262/0 (socket says 172.21.15.7:49262) 2024-06-10T01:20:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.657+0000 7f7b9bfff640 1 -- 172.21.15.7:0/3448844527 learned_addr learned my addr 172.21.15.7:0/3448844527 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.657+0000 7f7b9affd640 1 --2- 172.21.15.7:0/3448844527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555be315ea00 0x555be316d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.657+0000 7f7b9b7fe640 1 --2- 172.21.15.7:0/3448844527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555be315e010 0x555be316cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.657+0000 7f7b9bfff640 1 -- 172.21.15.7:0/3448844527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555be315ea00 msgr2=0x555be316d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.657+0000 7f7b9bfff640 1 --2- 172.21.15.7:0/3448844527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555be315ea00 0x555be316d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.657+0000 7f7b9bfff640 1 -- 172.21.15.7:0/3448844527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555be315e010 msgr2=0x555be316cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.657+0000 7f7b9bfff640 1 --2- 172.21.15.7:0/3448844527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555be315e010 0x555be316cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.657+0000 7f7b9bfff640 1 -- 172.21.15.7:0/3448844527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555be315f6d0 con 0x555be3163b70 2024-06-10T01:20:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.657+0000 7f7b9affd640 1 --2- 172.21.15.7:0/3448844527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555be315ea00 0x555be316d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:20:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.657+0000 7f7b9bfff640 1 --2- 172.21.15.7:0/3448844527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555be3163b70 0x555be316e0c0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009a60 tx=0x7f7b94004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30030c75706f4f8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.657+0000 7f7b7affd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:20:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.657+0000 7f7b7affd640 1 -- 172.21.15.7:0/3448844527 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b94003e40 con 0x555be3163b70 2024-06-10T01:20:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.657+0000 7f7b7affd640 1 -- 172.21.15.7:0/3448844527 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f7b9402fae0 con 0x555be3163b70 2024-06-10T01:20:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.657+0000 7f7ba4b95740 1 -- 172.21.15.7:0/3448844527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b94036040 con 0x555be3163b70 2024-06-10T01:20:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.657+0000 7f7b7affd640 1 -- 172.21.15.7:0/3448844527 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f7b9402fce0 con 0x555be3163b70 2024-06-10T01:20:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.658+0000 7f7ba4b95740 10 client.19170 fetch_fsmap learned FSMap version 44 2024-06-10T01:20:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.658+0000 7f7ba4b95740 1 -- 172.21.15.7:0/3448844527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x555be316f7f0 con 0x555be3163b70 2024-06-10T01:20:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.658+0000 7f7b7affd640 1 -- 172.21.15.7:0/3448844527 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b94036040 con 0x555be3163b70 2024-06-10T01:20:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.658+0000 7f7b7affd640 1 -- 172.21.15.7:0/3448844527 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f7b94050460 con 0x555be3163b70 2024-06-10T01:20:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.658+0000 7f7ba4b95740 10 client.19170 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:20:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.658+0000 7f7ba4b95740 10 client.19170 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:20:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.658+0000 7f7ba4b95740 20 client.19170 populate_metadata read hostname 'smithi007' 2024-06-10T01:20:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.658+0000 7f7ba4b95740 1 --2- 172.21.15.7:0/3448844527 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555be31df4a0 0x555be31e18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.658+0000 7f7ba4b95740 4 client.19170 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:20:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.658+0000 7f7ba4b95740 1 -- 172.21.15.7:0/3448844527 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555be30cec20 con 0x555be31df4a0 2024-06-10T01:20:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.658+0000 7f7b9b7fe640 1 --2- 172.21.15.7:0/3448844527 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555be31df4a0 0x555be31e18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.659+0000 7f7b7affd640 10 client.19170 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.659+0000 7f7b9b7fe640 1 --2- 172.21.15.7:0/3448844527 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555be31df4a0 0x555be31e18c0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40054eb138ce9a6e server_cookie=9a625ba87558221c in_seq=0 out_seq=0 2024-06-10T01:20:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.703+0000 7f7b7affd640 1 -- 172.21.15.7:0/3448844527 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555be30cec20 con 0x555be31df4a0 2024-06-10T01:20:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.703+0000 7f7b7affd640 10 client.19170 handle_command_reply: tid=0 2024-06-10T01:20:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.703+0000 7f7ba4b95740 1 client.19170 shutdown 2024-06-10T01:20:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.703+0000 7f7ba4b95740 1 -- 172.21.15.7:0/3448844527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555be3163b70 msgr2=0x555be316e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.703+0000 7f7ba4b95740 1 --2- 172.21.15.7:0/3448844527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555be3163b70 0x555be316e0c0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009a60 tx=0x7f7b94004750 comp rx=0 tx=0).stop 2024-06-10T01:20:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.703+0000 7f7ba4b95740 1 -- 172.21.15.7:0/3448844527 shutdown_connections 2024-06-10T01:20:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.703+0000 7f7ba4b95740 1 --2- 172.21.15.7:0/3448844527 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555be31df4a0 0x555be31e18c0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.703+0000 7f7b7affd640 0 client.19170 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.703+0000 7f7ba4b95740 1 --2- 172.21.15.7:0/3448844527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555be3163b70 0x555be316e0c0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.703+0000 7f7ba4b95740 1 --2- 172.21.15.7:0/3448844527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555be315ea00 0x555be316d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.703+0000 7f7ba4b95740 1 --2- 172.21.15.7:0/3448844527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555be315e010 0x555be316cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.703+0000 7f7ba4b95740 1 -- 172.21.15.7:0/3448844527 >> 172.21.15.7:0/3448844527 conn(0x555be3159680 msgr2=0x555be31680c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.703+0000 7f7ba4b95740 1 -- 172.21.15.7:0/3448844527 shutdown_connections 2024-06-10T01:20:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.703+0000 7f7ba4b95740 1 -- 172.21.15.7:0/3448844527 wait complete. 2024-06-10T01:20:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.704+0000 7f7ba4b95740 20 client.19170 trim_cache size 0 max 16384 2024-06-10T01:20:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:20:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:20:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:20:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.704+0000 7f7ba1e91640 1 -- 172.21.15.7:0/1898186879 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7b840a2b80 msgr2=0x7f7b840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.704+0000 7f7ba1e91640 1 --2- 172.21.15.7:0/1898186879 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7b840a2b80 0x7f7b840a5040 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7b94006010 tx=0x7f7b9403e040 comp rx=0 tx=0).stop 2024-06-10T01:20:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.704+0000 7f7ba1e91640 1 -- 172.21.15.7:0/1898186879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9c100910 msgr2=0x7f7b9c06abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.704+0000 7f7ba1e91640 1 --2- 172.21.15.7:0/1898186879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9c100910 0x7f7b9c06abd0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00d950 tx=0x7f7b8c00de10 comp rx=0 tx=0).stop 2024-06-10T01:20:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.704+0000 7f7ba1e91640 1 -- 172.21.15.7:0/1898186879 shutdown_connections 2024-06-10T01:20:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.704+0000 7f7ba1e91640 1 --2- 172.21.15.7:0/1898186879 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7b840a2b80 0x7f7b840a5040 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.704+0000 7f7ba1e91640 1 --2- 172.21.15.7:0/1898186879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b9c1024c0 0x7f7b9c19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.704+0000 7f7ba1e91640 1 --2- 172.21.15.7:0/1898186879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b9c101b00 0x7f7b9c19c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.704+0000 7f7ba1e91640 1 --2- 172.21.15.7:0/1898186879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9c100910 0x7f7b9c06abd0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.704+0000 7f7ba1e91640 1 -- 172.21.15.7:0/1898186879 >> 172.21.15.7:0/1898186879 conn(0x7f7b9c0fbe80 msgr2=0x7f7b9c104e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.705+0000 7f7ba1e91640 1 -- 172.21.15.7:0/1898186879 shutdown_connections 2024-06-10T01:20:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:20:52.705+0000 7f7ba1e91640 1 -- 172.21.15.7:0/1898186879 wait complete. 2024-06-10T01:20:52.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:52 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:20:52.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:52 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:20:52.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:52 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:20:52.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:52 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:20:52.773 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:52 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:20:52.773 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:52 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:20:52.773 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:52 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:20:52.773 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:52 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:20:52.773 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:52 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:20:52.773 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:52 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:20:52.773 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:52 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:20:52.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:52 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:20:52.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:52 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:20:52.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:52 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:20:52.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:52 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:20:52.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:52 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:20:52.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:52 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:20:52.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:52 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:20:52.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:52 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:20:52.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:52 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:20:52.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:52 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:20:52.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:52 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:20:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.943+0000 7f0f6b607700 1 -- 172.21.15.7:0/2631355214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f64108500 msgr2=0x7f0f641088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.943+0000 7f0f6b607700 1 --2- 172.21.15.7:0/2631355214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f64108500 0x7f0f641088e0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f0f60009990 tx=0x7f0f60009ca0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.943+0000 7f0f6b607700 1 -- 172.21.15.7:0/2631355214 shutdown_connections 2024-06-10T01:20:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.943+0000 7f0f6b607700 1 --2- 172.21.15.7:0/2631355214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f6406ab30 0x7f0f6410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.943+0000 7f0f6b607700 1 --2- 172.21.15.7:0/2631355214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f64108500 0x7f0f641088e0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.943+0000 7f0f6b607700 1 --2- 172.21.15.7:0/2631355214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f6406a170 0x7f0f6406a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.943+0000 7f0f6b607700 1 -- 172.21.15.7:0/2631355214 >> 172.21.15.7:0/2631355214 conn(0x7f0f64078340 msgr2=0x7f0f64078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.943+0000 7f0f6b607700 1 -- 172.21.15.7:0/2631355214 shutdown_connections 2024-06-10T01:20:52.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.943+0000 7f0f6b607700 1 -- 172.21.15.7:0/2631355214 wait complete. 2024-06-10T01:20:52.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.944+0000 7f0f6b607700 1 Processor -- start 2024-06-10T01:20:52.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.944+0000 7f0f6b607700 1 -- start start 2024-06-10T01:20:52.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.944+0000 7f0f6b607700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f6406a170 0x7f0f64101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:52.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.944+0000 7f0f6b607700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f6406ab30 0x7f0f641019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:52.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.944+0000 7f0f6b607700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f64108500 0x7f0f64101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:52.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.944+0000 7f0f6b607700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0f64112780 con 0x7f0f6406ab30 2024-06-10T01:20:52.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.944+0000 7f0f6b607700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0f64112600 con 0x7f0f64108500 2024-06-10T01:20:52.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.944+0000 7f0f6b607700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0f64112900 con 0x7f0f6406a170 2024-06-10T01:20:52.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.944+0000 7f0f68ba2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f6406ab30 0x7f0f641019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:52.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.944+0000 7f0f68ba2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f6406ab30 0x7f0f641019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49286/0 (socket says 172.21.15.7:49286) 2024-06-10T01:20:52.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.944+0000 7f0f68ba2700 1 -- 172.21.15.7:0/4041174243 learned_addr learned my addr 172.21.15.7:0/4041174243 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:52.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.944+0000 7f0f693a3700 1 --2- 172.21.15.7:0/4041174243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f6406a170 0x7f0f64101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:52.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.945+0000 7f0f69ba4700 1 --2- 172.21.15.7:0/4041174243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f64108500 0x7f0f64101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:52.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.945+0000 7f0f68ba2700 1 -- 172.21.15.7:0/4041174243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f6406a170 msgr2=0x7f0f64101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:52.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.945+0000 7f0f68ba2700 1 --2- 172.21.15.7:0/4041174243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f6406a170 0x7f0f64101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.945+0000 7f0f68ba2700 1 -- 172.21.15.7:0/4041174243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f64108500 msgr2=0x7f0f64101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:52.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.945+0000 7f0f68ba2700 1 --2- 172.21.15.7:0/4041174243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f64108500 0x7f0f64101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:52.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.945+0000 7f0f68ba2700 1 -- 172.21.15.7:0/4041174243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f641a70c0 con 0x7f0f6406ab30 2024-06-10T01:20:52.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.945+0000 7f0f69ba4700 1 --2- 172.21.15.7:0/4041174243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f64108500 0x7f0f64101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:20:52.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.945+0000 7f0f68ba2700 1 --2- 172.21.15.7:0/4041174243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f6406ab30 0x7f0f641019a0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400d980 tx=0x7f0f5400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:52.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.945+0000 7f0f5a7fc700 1 -- 172.21.15.7:0/4041174243 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f54017070 con 0x7f0f6406ab30 2024-06-10T01:20:52.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.945+0000 7f0f6b607700 1 -- 172.21.15.7:0/4041174243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f641a73b0 con 0x7f0f6406ab30 2024-06-10T01:20:52.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.945+0000 7f0f5a7fc700 1 -- 172.21.15.7:0/4041174243 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f54005020 con 0x7f0f6406ab30 2024-06-10T01:20:52.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.945+0000 7f0f5a7fc700 1 -- 172.21.15.7:0/4041174243 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f54009bc0 con 0x7f0f6406ab30 2024-06-10T01:20:52.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.945+0000 7f0f6b607700 1 -- 172.21.15.7:0/4041174243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f641a78a0 con 0x7f0f6406ab30 2024-06-10T01:20:52.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.946+0000 7f0f6b607700 1 -- 172.21.15.7:0/4041174243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f6410a9e0 con 0x7f0f6406ab30 2024-06-10T01:20:52.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.947+0000 7f0f5a7fc700 1 -- 172.21.15.7:0/4041174243 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0f540051d0 con 0x7f0f6406ab30 2024-06-10T01:20:52.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.948+0000 7f0f5a7fc700 1 --2- 172.21.15.7:0/4041174243 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0f500a2c20 0x7f0f500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:52.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.948+0000 7f0f693a3700 1 --2- 172.21.15.7:0/4041174243 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0f500a2c20 0x7f0f500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:52.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.948+0000 7f0f5a7fc700 1 -- 172.21.15.7:0/4041174243 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f0f540cf6f0 con 0x7f0f6406ab30 2024-06-10T01:20:52.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.949+0000 7f0f693a3700 1 --2- 172.21.15.7:0/4041174243 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0f500a2c20 0x7f0f500a50e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0f6000b5c0 tx=0x7f0f60005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:52.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:52.952+0000 7f0f5a7fc700 1 -- 172.21.15.7:0/4041174243 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0f54020020 con 0x7f0f6406ab30 2024-06-10T01:20:52.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:52 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:20:52.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:52 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:20:52.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:52 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:20:52.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:52 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:20:52.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:52 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:20:52.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:52 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:20:52.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:52 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:20:52.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:52 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:20:52.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:52 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:20:52.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:52 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:20:52.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:52 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:20:53.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:53.204+0000 7f0f6b607700 1 -- 172.21.15.7:0/4041174243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f64069180 con 0x7f0f6406ab30 2024-06-10T01:20:53.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:53.204+0000 7f0f5a7fc700 1 -- 172.21.15.7:0/4041174243 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11860 (secure 0 0 0) 0x7f0f5409c2d0 con 0x7f0f6406ab30 2024-06-10T01:20:53.198 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:20:53.198 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:19:47.239285+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:20:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:53.207+0000 7f0f6b607700 1 -- 172.21.15.7:0/4041174243 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0f500a2c20 msgr2=0x7f0f500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:53.207+0000 7f0f6b607700 1 --2- 172.21.15.7:0/4041174243 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0f500a2c20 0x7f0f500a50e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0f6000b5c0 tx=0x7f0f60005960 comp rx=0 tx=0).stop 2024-06-10T01:20:53.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:53.207+0000 7f0f6b607700 1 -- 172.21.15.7:0/4041174243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f6406ab30 msgr2=0x7f0f641019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:53.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:53.207+0000 7f0f6b607700 1 --2- 172.21.15.7:0/4041174243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f6406ab30 0x7f0f641019a0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400d980 tx=0x7f0f5400dc90 comp rx=0 tx=0).stop 2024-06-10T01:20:53.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:53.207+0000 7f0f6b607700 1 -- 172.21.15.7:0/4041174243 shutdown_connections 2024-06-10T01:20:53.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:53.207+0000 7f0f6b607700 1 --2- 172.21.15.7:0/4041174243 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0f500a2c20 0x7f0f500a50e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:53.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:53.207+0000 7f0f6b607700 1 --2- 172.21.15.7:0/4041174243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f64108500 0x7f0f64101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:53.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:53.207+0000 7f0f6b607700 1 --2- 172.21.15.7:0/4041174243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f6406a170 0x7f0f64101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:53.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:53.207+0000 7f0f6b607700 1 --2- 172.21.15.7:0/4041174243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f6406ab30 0x7f0f641019a0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:53.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:53.207+0000 7f0f6b607700 1 -- 172.21.15.7:0/4041174243 >> 172.21.15.7:0/4041174243 conn(0x7f0f64078340 msgr2=0x7f0f640ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:53.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:53.207+0000 7f0f6b607700 1 -- 172.21.15.7:0/4041174243 shutdown_connections 2024-06-10T01:20:53.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:53.207+0000 7f0f6b607700 1 -- 172.21.15.7:0/4041174243 wait complete. 2024-06-10T01:20:53.202 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 44 2024-06-10T01:20:53.473 INFO:tasks.workunit.client.0.smithi007.stdout: 10 19792 13.80 MB/sec warmup 90 sec latency 23069.135 ms 2024-06-10T01:20:53.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:53 smithi007 ceph-mon[32398]: Adjusting osd_memory_target on smithi078 to 176.2M 2024-06-10T01:20:53.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:53 smithi007 ceph-mon[32398]: Unable to set osd_memory_target on smithi078 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-06-10T01:20:53.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:53 smithi007 ceph-mon[32398]: pgmap v878: 129 pgs: 129 active+clean; 383 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 6.0 MiB/s wr, 249 op/s 2024-06-10T01:20:53.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:53 smithi007 ceph-mon[32398]: Adjusting osd_memory_target on smithi007 to 177.2M 2024-06-10T01:20:53.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:53 smithi007 ceph-mon[32398]: Unable to set osd_memory_target on smithi007 to 185842585: error parsing value: Value '185842585' is below minimum 939524096 2024-06-10T01:20:53.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:53 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/4041174243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:20:53.790 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db --set-id 8f7fcf77 --release --await' 2024-06-10T01:20:53.790 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db --set-id 8f7fcf77 --release --await 2024-06-10T01:20:53.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:53 smithi078 ceph-mon[36683]: Adjusting osd_memory_target on smithi078 to 176.2M 2024-06-10T01:20:53.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:53 smithi078 ceph-mon[36683]: Unable to set osd_memory_target on smithi078 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-06-10T01:20:53.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:53 smithi078 ceph-mon[36683]: pgmap v878: 129 pgs: 129 active+clean; 383 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 6.0 MiB/s wr, 249 op/s 2024-06-10T01:20:53.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:53 smithi078 ceph-mon[36683]: Adjusting osd_memory_target on smithi007 to 177.2M 2024-06-10T01:20:53.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:53 smithi078 ceph-mon[36683]: Unable to set osd_memory_target on smithi007 to 185842585: error parsing value: Value '185842585' is below minimum 939524096 2024-06-10T01:20:53.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:53 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/4041174243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:20:53.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:53 smithi175 ceph-mon[36636]: Adjusting osd_memory_target on smithi078 to 176.2M 2024-06-10T01:20:53.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:53 smithi175 ceph-mon[36636]: Unable to set osd_memory_target on smithi078 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-06-10T01:20:53.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:53 smithi175 ceph-mon[36636]: pgmap v878: 129 pgs: 129 active+clean; 383 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 6.0 MiB/s wr, 249 op/s 2024-06-10T01:20:53.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:53 smithi175 ceph-mon[36636]: Adjusting osd_memory_target on smithi007 to 177.2M 2024-06-10T01:20:53.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:53 smithi175 ceph-mon[36636]: Unable to set osd_memory_target on smithi007 to 185842585: error parsing value: Value '185842585' is below minimum 939524096 2024-06-10T01:20:53.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:53 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/4041174243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:20:54.084 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:20:54.473 INFO:tasks.workunit.client.0.smithi007.stdout: 10 19792 13.65 MB/sec warmup 91 sec latency 24069.179 ms 2024-06-10T01:20:55.474 INFO:tasks.workunit.client.0.smithi007.stdout: 10 19792 13.50 MB/sec warmup 92 sec latency 25069.268 ms 2024-06-10T01:20:55.598 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:55 smithi007 ceph-mon[32398]: pgmap v879: 129 pgs: 129 active+clean; 380 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 6.0 MiB/s wr, 235 op/s 2024-06-10T01:20:55.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.718+0000 7f1b4fe80700 1 -- 172.21.15.7:0/2469716333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b48101860 msgr2=0x7f1b4810d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:55.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.718+0000 7f1b4fe80700 1 --2- 172.21.15.7:0/2469716333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b48101860 0x7f1b4810d990 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000b0d0 tx=0x7f1b4000b3e0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.718+0000 7f1b4fe80700 1 -- 172.21.15.7:0/2469716333 shutdown_connections 2024-06-10T01:20:55.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.718+0000 7f1b4fe80700 1 --2- 172.21.15.7:0/2469716333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b48101860 0x7f1b4810d990 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.718+0000 7f1b4fe80700 1 --2- 172.21.15.7:0/2469716333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b48106b30 0x7f1b48106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.718+0000 7f1b4fe80700 1 --2- 172.21.15.7:0/2469716333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b48100ea0 0x7f1b48101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.718+0000 7f1b4fe80700 1 -- 172.21.15.7:0/2469716333 >> 172.21.15.7:0/2469716333 conn(0x7f1b48077590 msgr2=0x7f1b480fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:55.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.718+0000 7f1b4fe80700 1 -- 172.21.15.7:0/2469716333 shutdown_connections 2024-06-10T01:20:55.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.718+0000 7f1b4fe80700 1 -- 172.21.15.7:0/2469716333 wait complete. 2024-06-10T01:20:55.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.719+0000 7f1b4fe80700 1 Processor -- start 2024-06-10T01:20:55.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.719+0000 7f1b4fe80700 1 -- start start 2024-06-10T01:20:55.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.719+0000 7f1b4fe80700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b48100ea0 0x7f1b4819e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:55.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.719+0000 7f1b4fe80700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b48101860 0x7f1b4819ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:55.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.719+0000 7f1b4fe80700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b48106b30 0x7f1b48198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:55.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.719+0000 7f1b4fe80700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1b48110ef0 con 0x7f1b48106b30 2024-06-10T01:20:55.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.719+0000 7f1b4fe80700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1b48110d70 con 0x7f1b48101860 2024-06-10T01:20:55.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.719+0000 7f1b4fe80700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1b48111070 con 0x7f1b48100ea0 2024-06-10T01:20:55.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.719+0000 7f1b4e41d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b48106b30 0x7f1b48198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:55.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.719+0000 7f1b4e41d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b48106b30 0x7f1b48198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49308/0 (socket says 172.21.15.7:49308) 2024-06-10T01:20:55.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.719+0000 7f1b4e41d700 1 -- 172.21.15.7:0/4183999219 learned_addr learned my addr 172.21.15.7:0/4183999219 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:55.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.719+0000 7f1b4dc1c700 1 --2- 172.21.15.7:0/4183999219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b48100ea0 0x7f1b4819e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:55.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.719+0000 7f1b4d41b700 1 --2- 172.21.15.7:0/4183999219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b48101860 0x7f1b4819ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:55.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.719+0000 7f1b4e41d700 1 -- 172.21.15.7:0/4183999219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b48100ea0 msgr2=0x7f1b4819e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:55.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.719+0000 7f1b4e41d700 1 --2- 172.21.15.7:0/4183999219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b48100ea0 0x7f1b4819e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.719+0000 7f1b4e41d700 1 -- 172.21.15.7:0/4183999219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b48101860 msgr2=0x7f1b4819ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:55.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.719+0000 7f1b4e41d700 1 --2- 172.21.15.7:0/4183999219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b48101860 0x7f1b4819ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.719+0000 7f1b4e41d700 1 -- 172.21.15.7:0/4183999219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b48199220 con 0x7f1b48106b30 2024-06-10T01:20:55.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.720+0000 7f1b4e41d700 1 --2- 172.21.15.7:0/4183999219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b48106b30 0x7f1b48198ac0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f1b40009cd0 tx=0x7f1b400079a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:55.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.720+0000 7f1b3effd700 1 -- 172.21.15.7:0/4183999219 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4001f070 con 0x7f1b48106b30 2024-06-10T01:20:55.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.720+0000 7f1b4fe80700 1 -- 172.21.15.7:0/4183999219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b481994b0 con 0x7f1b48106b30 2024-06-10T01:20:55.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.720+0000 7f1b3effd700 1 -- 172.21.15.7:0/4183999219 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b40005070 con 0x7f1b48106b30 2024-06-10T01:20:55.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.720+0000 7f1b3effd700 1 -- 172.21.15.7:0/4183999219 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4001b6a0 con 0x7f1b48106b30 2024-06-10T01:20:55.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.720+0000 7f1b4fe80700 1 -- 172.21.15.7:0/4183999219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b481a57d0 con 0x7f1b48106b30 2024-06-10T01:20:55.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.722+0000 7f1b3effd700 1 -- 172.21.15.7:0/4183999219 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f1b40022070 con 0x7f1b48106b30 2024-06-10T01:20:55.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.723+0000 7f1b3effd700 1 --2- 172.21.15.7:0/4183999219 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1b340a2c20 0x7f1b340a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:55.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.723+0000 7f1b4dc1c700 1 --2- 172.21.15.7:0/4183999219 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1b340a2c20 0x7f1b340a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:55.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.723+0000 7f1b3effd700 1 -- 172.21.15.7:0/4183999219 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f1b400d0460 con 0x7f1b48106b30 2024-06-10T01:20:55.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.724+0000 7f1b4dc1c700 1 --2- 172.21.15.7:0/4183999219 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1b340a2c20 0x7f1b340a50e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1b48199c00 tx=0x7f1b44009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:55.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.724+0000 7f1b5a703b80 1 Processor -- start 2024-06-10T01:20:55.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.724+0000 7f1b5a703b80 1 -- start start 2024-06-10T01:20:55.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.725+0000 7f1b5a703b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559750db7c40 0x559750db8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:55.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.725+0000 7f1b5a703b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559750db8560 0x559750db1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:55.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.725+0000 7f1b5a703b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559750db23d0 0x559750db2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:55.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.725+0000 7f1b5a703b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559750d03910 con 0x559750db23d0 2024-06-10T01:20:55.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.725+0000 7f1b5a703b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559750cab820 con 0x559750db8560 2024-06-10T01:20:55.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.725+0000 7f1b5a703b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559750cff8d0 con 0x559750db7c40 2024-06-10T01:20:55.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.725+0000 7f1b4e41d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559750db23d0 0x559750db2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:55.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.725+0000 7f1b4d41b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559750db8560 0x559750db1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:55.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.725+0000 7f1b4e41d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559750db23d0 0x559750db2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49316/0 (socket says 172.21.15.7:49316) 2024-06-10T01:20:55.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.725+0000 7f1b4e41d700 1 -- 172.21.15.7:0/546490304 learned_addr learned my addr 172.21.15.7:0/546490304 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:55.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.725+0000 7f1b4dc1c700 1 --2- 172.21.15.7:0/546490304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559750db7c40 0x559750db8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:55.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.725+0000 7f1b4d41b700 1 -- 172.21.15.7:0/546490304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559750db7c40 msgr2=0x559750db8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:55.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.725+0000 7f1b4d41b700 1 --2- 172.21.15.7:0/546490304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559750db7c40 0x559750db8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.725+0000 7f1b4d41b700 1 -- 172.21.15.7:0/546490304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559750db23d0 msgr2=0x559750db2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:55.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.725+0000 7f1b4d41b700 1 --2- 172.21.15.7:0/546490304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559750db23d0 0x559750db2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.725+0000 7f1b4d41b700 1 -- 172.21.15.7:0/546490304 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559750ca2050 con 0x559750db8560 2024-06-10T01:20:55.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.725+0000 7f1b4d41b700 1 --2- 172.21.15.7:0/546490304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559750db8560 0x559750db1e90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800ac30 tx=0x7f1b3800af40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ad6ef73feab59c7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:55.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.725+0000 7f1b33fff700 1 -- 172.21.15.7:0/546490304 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3800fde0 con 0x559750db8560 2024-06-10T01:20:55.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.726+0000 7f1b33fff700 1 -- 172.21.15.7:0/546490304 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b38004500 con 0x559750db8560 2024-06-10T01:20:55.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.726+0000 7f1b33fff700 1 -- 172.21.15.7:0/546490304 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b380053c0 con 0x559750db8560 2024-06-10T01:20:55.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.726+0000 7f1b5a703b80 1 -- 172.21.15.7:0/546490304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559750db8560 msgr2=0x559750db1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:55.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.726+0000 7f1b5a703b80 1 --2- 172.21.15.7:0/546490304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559750db8560 0x559750db1e90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800ac30 tx=0x7f1b3800af40 comp rx=0 tx=0).stop 2024-06-10T01:20:55.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.726+0000 7f1b5a703b80 1 -- 172.21.15.7:0/546490304 shutdown_connections 2024-06-10T01:20:55.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.726+0000 7f1b5a703b80 1 --2- 172.21.15.7:0/546490304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559750db23d0 0x559750db2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.726+0000 7f1b5a703b80 1 --2- 172.21.15.7:0/546490304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559750db7c40 0x559750db8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.726+0000 7f1b5a703b80 1 --2- 172.21.15.7:0/546490304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559750db8560 0x559750db1e90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.726+0000 7f1b5a703b80 1 -- 172.21.15.7:0/546490304 >> 172.21.15.7:0/546490304 conn(0x559750cdcdc0 msgr2=0x559750dad120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:55.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.726+0000 7f1b5a703b80 1 -- 172.21.15.7:0/546490304 shutdown_connections 2024-06-10T01:20:55.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.726+0000 7f1b5a703b80 1 -- 172.21.15.7:0/546490304 wait complete. 2024-06-10T01:20:55.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.727+0000 7f1b5a703b80 1 -- start start 2024-06-10T01:20:55.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.727+0000 7f1b5a703b80 1 Processor -- start 2024-06-10T01:20:55.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.727+0000 7f1b5a703b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559750db23d0 0x559750dbe2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:55.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.727+0000 7f1b5a703b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559750db7c40 0x559750dbb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:55.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.727+0000 7f1b5a703b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559750db8560 0x559750dbbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:55.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.727+0000 7f1b5a703b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559750dbefd0 con 0x559750db7c40 2024-06-10T01:20:55.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.727+0000 7f1b5a703b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559750ca3cf0 con 0x559750db23d0 2024-06-10T01:20:55.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.727+0000 7f1b5a703b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559750dbf150 con 0x559750db8560 2024-06-10T01:20:55.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.727+0000 7f1b5a703b80 10 client.0 _ll_register_callbacks cb 0x559750c9da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:20:55.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.727+0000 7f1b4d41b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559750db7c40 0x559750dbb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:55.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.728+0000 7f1b4d41b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559750db7c40 0x559750dbb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49330/0 (socket says 172.21.15.7:49330) 2024-06-10T01:20:55.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.728+0000 7f1b4d41b700 1 -- 172.21.15.7:0/923275716 learned_addr learned my addr 172.21.15.7:0/923275716 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:55.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.728+0000 7f1b4dc1c700 1 --2- 172.21.15.7:0/923275716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559750db23d0 0x559750dbe2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:55.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.728+0000 7f1b4e41d700 1 --2- 172.21.15.7:0/923275716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559750db8560 0x559750dbbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:55.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.728+0000 7f1b4d41b700 1 -- 172.21.15.7:0/923275716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559750db8560 msgr2=0x559750dbbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:55.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.728+0000 7f1b4d41b700 1 --2- 172.21.15.7:0/923275716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559750db8560 0x559750dbbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.728+0000 7f1b4d41b700 1 -- 172.21.15.7:0/923275716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559750db23d0 msgr2=0x559750dbe2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:55.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.728+0000 7f1b4d41b700 1 --2- 172.21.15.7:0/923275716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559750db23d0 0x559750dbe2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.728+0000 7f1b4d41b700 1 -- 172.21.15.7:0/923275716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559750c529d0 con 0x559750db7c40 2024-06-10T01:20:55.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.728+0000 7f1b4d41b700 1 --2- 172.21.15.7:0/923275716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559750db7c40 0x559750dbb5b0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f1b38010010 tx=0x7f1b38004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47117913fc038b47 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:55.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.728+0000 7f1b317fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:20:55.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.728+0000 7f1b317fa700 1 -- 172.21.15.7:0/923275716 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38026020 con 0x559750db7c40 2024-06-10T01:20:55.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.728+0000 7f1b317fa700 1 -- 172.21.15.7:0/923275716 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f1b38005350 con 0x559750db7c40 2024-06-10T01:20:55.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.728+0000 7f1b5a703b80 1 -- 172.21.15.7:0/923275716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b38004500 con 0x559750db7c40 2024-06-10T01:20:55.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.728+0000 7f1b317fa700 1 -- 172.21.15.7:0/923275716 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1b38031050 con 0x559750db7c40 2024-06-10T01:20:55.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.728+0000 7f1b5a703b80 10 client.19188 fetch_fsmap learned FSMap version 44 2024-06-10T01:20:55.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.728+0000 7f1b5a703b80 1 -- 172.21.15.7:0/923275716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x559750dc9870 con 0x559750db7c40 2024-06-10T01:20:55.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.729+0000 7f1b317fa700 1 -- 172.21.15.7:0/923275716 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b38004500 con 0x559750db7c40 2024-06-10T01:20:55.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.729+0000 7f1b317fa700 1 -- 172.21.15.7:0/923275716 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f1b38010430 con 0x559750db7c40 2024-06-10T01:20:55.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.729+0000 7f1b5a703b80 10 client.19188 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:20:55.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.729+0000 7f1b5a703b80 10 client.19188 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:20:55.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.729+0000 7f1b5a703b80 20 client.19188 populate_metadata read hostname 'smithi007' 2024-06-10T01:20:55.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.729+0000 7f1b5a703b80 1 --2- 172.21.15.7:0/923275716 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559750dcbeb0 0x559750dce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:55.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.729+0000 7f1b5a703b80 4 client.19188 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:20:55.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.729+0000 7f1b5a703b80 1 -- 172.21.15.7:0/923275716 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559750d10e10 con 0x559750dcbeb0 2024-06-10T01:20:55.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.729+0000 7f1b4dc1c700 1 --2- 172.21.15.7:0/923275716 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559750dcbeb0 0x559750dce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:55.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.730+0000 7f1b4dc1c700 1 --2- 172.21.15.7:0/923275716 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559750dcbeb0 0x559750dce2a0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7db145d40bfbb61a server_cookie=6f9292a3c169c873 in_seq=0 out_seq=0 2024-06-10T01:20:55.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.730+0000 7f1b317fa700 10 client.19188 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:55.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.730+0000 7f1b317fa700 1 -- 172.21.15.7:0/923275716 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559750d10e10 con 0x559750dcbeb0 2024-06-10T01:20:55.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.730+0000 7f1b317fa700 10 client.19188 handle_command_reply: tid=0 2024-06-10T01:20:55.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.731+0000 7f1b5a703b80 1 client.19188 shutdown 2024-06-10T01:20:55.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.731+0000 7f1b5a703b80 1 -- 172.21.15.7:0/923275716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559750db7c40 msgr2=0x559750dbb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:55.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.731+0000 7f1b5a703b80 1 --2- 172.21.15.7:0/923275716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559750db7c40 0x559750dbb5b0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f1b38010010 tx=0x7f1b38004a00 comp rx=0 tx=0).stop 2024-06-10T01:20:55.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.731+0000 7f1b5a703b80 1 -- 172.21.15.7:0/923275716 shutdown_connections 2024-06-10T01:20:55.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.731+0000 7f1b5a703b80 1 --2- 172.21.15.7:0/923275716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559750db8560 0x559750dbbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.731+0000 7f1b5a703b80 1 --2- 172.21.15.7:0/923275716 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559750dcbeb0 0x559750dce2a0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.731+0000 7f1b5a703b80 1 --2- 172.21.15.7:0/923275716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559750db23d0 0x559750dbe2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.731+0000 7f1b5a703b80 1 --2- 172.21.15.7:0/923275716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559750db7c40 0x559750dbb5b0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.731+0000 7f1b317fa700 0 client.19188 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:55.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.731+0000 7f1b5a703b80 1 -- 172.21.15.7:0/923275716 >> 172.21.15.7:0/923275716 conn(0x559750cdcdc0 msgr2=0x559750db0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:55.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.731+0000 7f1b5a703b80 1 -- 172.21.15.7:0/923275716 shutdown_connections 2024-06-10T01:20:55.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.731+0000 7f1b5a703b80 1 -- 172.21.15.7:0/923275716 wait complete. 2024-06-10T01:20:55.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.731+0000 7f1b5a703b80 20 client.19188 trim_cache size 0 max 16384 2024-06-10T01:20:55.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.751+0000 7f1b5a703b80 1 Processor -- start 2024-06-10T01:20:55.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.752+0000 7f1b5a703b80 1 -- start start 2024-06-10T01:20:55.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.752+0000 7f1b5a703b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559750db23d0 0x559750db8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:55.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.752+0000 7f1b5a703b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559750db7c40 0x559750db8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:55.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.752+0000 7f1b5a703b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559750db8560 0x559750dd3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:55.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.752+0000 7f1b5a703b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1b481b0a70 con 0x559750db7c40 2024-06-10T01:20:55.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.752+0000 7f1b5a703b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1b481ae270 con 0x559750db23d0 2024-06-10T01:20:55.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.752+0000 7f1b5a703b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1b481ac0c0 con 0x559750db8560 2024-06-10T01:20:55.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.752+0000 7f1b4d41b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559750db7c40 0x559750db8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:55.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.752+0000 7f1b4d41b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559750db7c40 0x559750db8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49332/0 (socket says 172.21.15.7:49332) 2024-06-10T01:20:55.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.752+0000 7f1b4d41b700 1 -- 172.21.15.7:0/829914606 learned_addr learned my addr 172.21.15.7:0/829914606 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:55.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.752+0000 7f1b4dc1c700 1 --2- 172.21.15.7:0/829914606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559750db23d0 0x559750db8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:55.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.752+0000 7f1b4d41b700 1 -- 172.21.15.7:0/829914606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559750db8560 msgr2=0x559750dd3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:55.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.752+0000 7f1b4d41b700 1 --2- 172.21.15.7:0/829914606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559750db8560 0x559750dd3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.752+0000 7f1b4d41b700 1 -- 172.21.15.7:0/829914606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559750db23d0 msgr2=0x559750db8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:55.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.752+0000 7f1b4d41b700 1 --2- 172.21.15.7:0/829914606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559750db23d0 0x559750db8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.752+0000 7f1b4d41b700 1 -- 172.21.15.7:0/829914606 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559750db9b60 con 0x559750db7c40 2024-06-10T01:20:55.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.752+0000 7f1b4d41b700 1 --2- 172.21.15.7:0/829914606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559750db7c40 0x559750db8f40 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800ab80 tx=0x7f1b3800ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c05929353a6df3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:55.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.753+0000 7f1b337fe700 1 -- 172.21.15.7:0/829914606 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38026070 con 0x559750db7c40 2024-06-10T01:20:55.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.753+0000 7f1b337fe700 1 -- 172.21.15.7:0/829914606 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b3801d040 con 0x559750db7c40 2024-06-10T01:20:55.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.753+0000 7f1b337fe700 1 -- 172.21.15.7:0/829914606 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3801c070 con 0x559750db7c40 2024-06-10T01:20:55.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.753+0000 7f1b5a703b80 1 -- 172.21.15.7:0/829914606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559750db7c40 msgr2=0x559750db8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:55.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.753+0000 7f1b5a703b80 1 --2- 172.21.15.7:0/829914606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559750db7c40 0x559750db8f40 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800ab80 tx=0x7f1b3800ff60 comp rx=0 tx=0).stop 2024-06-10T01:20:55.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.753+0000 7f1b5a703b80 1 -- 172.21.15.7:0/829914606 shutdown_connections 2024-06-10T01:20:55.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.753+0000 7f1b5a703b80 1 --2- 172.21.15.7:0/829914606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559750db8560 0x559750dd3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.753+0000 7f1b5a703b80 1 --2- 172.21.15.7:0/829914606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559750db23d0 0x559750db8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.753+0000 7f1b5a703b80 1 --2- 172.21.15.7:0/829914606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559750db7c40 0x559750db8f40 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.753+0000 7f1b5a703b80 1 -- 172.21.15.7:0/829914606 >> 172.21.15.7:0/829914606 conn(0x559750cdcdc0 msgr2=0x559750db9ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:55.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.753+0000 7f1b5a703b80 1 -- 172.21.15.7:0/829914606 shutdown_connections 2024-06-10T01:20:55.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.753+0000 7f1b5a703b80 1 -- 172.21.15.7:0/829914606 wait complete. 2024-06-10T01:20:55.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.754+0000 7f1b5a703b80 1 -- start start 2024-06-10T01:20:55.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.754+0000 7f1b5a703b80 1 Processor -- start 2024-06-10T01:20:55.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.754+0000 7f1b5a703b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559750db23d0 0x559750dc4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:55.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.754+0000 7f1b5a703b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559750db7c40 0x559750dc1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:55.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.754+0000 7f1b5a703b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559750db8560 0x559750dc1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:55.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.754+0000 7f1b5a703b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1b481ac730 con 0x559750db23d0 2024-06-10T01:20:55.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.754+0000 7f1b5a703b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1b481adba0 con 0x559750db7c40 2024-06-10T01:20:55.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.754+0000 7f1b5a703b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1b481ace00 con 0x559750db8560 2024-06-10T01:20:55.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.754+0000 7f1b5a703b80 10 client.0 _ll_register_callbacks cb 0x559750c9da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:20:55.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.754+0000 7f1b4dc1c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559750db23d0 0x559750dc4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:55.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.754+0000 7f1b4dc1c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559750db23d0 0x559750dc4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49346/0 (socket says 172.21.15.7:49346) 2024-06-10T01:20:55.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.754+0000 7f1b4dc1c700 1 -- 172.21.15.7:0/3136498989 learned_addr learned my addr 172.21.15.7:0/3136498989 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:55.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.755+0000 7f1b4dc1c700 1 -- 172.21.15.7:0/3136498989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559750db8560 msgr2=0x559750dc1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:55.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.755+0000 7f1b4dc1c700 1 --2- 172.21.15.7:0/3136498989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559750db8560 0x559750dc1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.755+0000 7f1b4dc1c700 1 -- 172.21.15.7:0/3136498989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559750db7c40 msgr2=0x559750dc1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.755+0000 7f1b4d41b700 1 --2- 172.21.15.7:0/3136498989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559750db7c40 0x559750dc1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.755+0000 7f1b4dc1c700 1 --2- 172.21.15.7:0/3136498989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559750db7c40 0x559750dc1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.755+0000 7f1b4dc1c700 1 -- 172.21.15.7:0/3136498989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559750dc21b0 con 0x559750db23d0 2024-06-10T01:20:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.755+0000 7f1b4dc1c700 1 --2- 172.21.15.7:0/3136498989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559750db23d0 0x559750dc4c20 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x559750dce920 tx=0x7f1b44011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168826de2f3ff3f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.755+0000 7f1b317fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:20:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.755+0000 7f1b317fa700 1 -- 172.21.15.7:0/3136498989 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4402e070 con 0x559750db23d0 2024-06-10T01:20:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.755+0000 7f1b5a703b80 1 -- 172.21.15.7:0/3136498989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b3801d040 con 0x559750db23d0 2024-06-10T01:20:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.755+0000 7f1b317fa700 1 -- 172.21.15.7:0/3136498989 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f1b44015a70 con 0x559750db23d0 2024-06-10T01:20:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.755+0000 7f1b317fa700 1 -- 172.21.15.7:0/3136498989 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1b44036050 con 0x559750db23d0 2024-06-10T01:20:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.755+0000 7f1b5a703b80 10 client.19194 fetch_fsmap learned FSMap version 44 2024-06-10T01:20:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.755+0000 7f1b5a703b80 1 -- 172.21.15.7:0/3136498989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x559750e3be40 con 0x559750db23d0 2024-06-10T01:20:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.755+0000 7f1b317fa700 1 -- 172.21.15.7:0/3136498989 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b3801d040 con 0x559750db23d0 2024-06-10T01:20:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.755+0000 7f1b317fa700 1 -- 172.21.15.7:0/3136498989 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f1b44031030 con 0x559750db23d0 2024-06-10T01:20:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.756+0000 7f1b5a703b80 10 client.19194 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:20:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.756+0000 7f1b5a703b80 10 client.19194 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:20:55.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.756+0000 7f1b5a703b80 20 client.19194 populate_metadata read hostname 'smithi007' 2024-06-10T01:20:55.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.756+0000 7f1b5a703b80 1 --2- 172.21.15.7:0/3136498989 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559750e3c590 0x559750e3e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:55.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.756+0000 7f1b5a703b80 4 client.19194 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8f7fcf77", "release": true, "await": true}] 2024-06-10T01:20:55.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.756+0000 7f1b5a703b80 1 -- 172.21.15.7:0/3136498989 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8f7fcf77", "release": true, "await": true}) -- 0x559750d0b390 con 0x559750e3c590 2024-06-10T01:20:55.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.756+0000 7f1b4d41b700 1 --2- 172.21.15.7:0/3136498989 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559750e3c590 0x559750e3e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:55.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.756+0000 7f1b317fa700 10 client.19194 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:55.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.756+0000 7f1b4d41b700 1 --2- 172.21.15.7:0/3136498989 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559750e3c590 0x559750e3e9f0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2c4897b9f2ca459 server_cookie=36bddee97bf4a772 in_seq=0 out_seq=0 2024-06-10T01:20:55.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.802+0000 7f1b317fa700 1 -- 172.21.15.7:0/3136498989 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559750d0b390 con 0x559750e3c590 2024-06-10T01:20:55.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.802+0000 7f1b317fa700 10 client.19194 handle_command_reply: tid=0 2024-06-10T01:20:55.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.802+0000 7f1b5a703b80 1 client.19194 shutdown 2024-06-10T01:20:55.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.802+0000 7f1b5a703b80 1 -- 172.21.15.7:0/3136498989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559750db23d0 msgr2=0x559750dc4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:55.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.802+0000 7f1b5a703b80 1 --2- 172.21.15.7:0/3136498989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559750db23d0 0x559750dc4c20 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x559750dce920 tx=0x7f1b44011000 comp rx=0 tx=0).stop 2024-06-10T01:20:55.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.802+0000 7f1b5a703b80 1 -- 172.21.15.7:0/3136498989 shutdown_connections 2024-06-10T01:20:55.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.802+0000 7f1b5a703b80 1 --2- 172.21.15.7:0/3136498989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559750db8560 0x559750dc1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.802+0000 7f1b5a703b80 1 --2- 172.21.15.7:0/3136498989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559750db23d0 0x559750dc4c20 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.802+0000 7f1b5a703b80 1 --2- 172.21.15.7:0/3136498989 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559750e3c590 0x559750e3e9f0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.802+0000 7f1b5a703b80 1 --2- 172.21.15.7:0/3136498989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559750db7c40 0x559750dc1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.802+0000 7f1b317fa700 0 client.19194 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:55.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.802+0000 7f1b5a703b80 1 -- 172.21.15.7:0/3136498989 >> 172.21.15.7:0/3136498989 conn(0x559750cdcdc0 msgr2=0x559750db89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:55.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.802+0000 7f1b5a703b80 1 -- 172.21.15.7:0/3136498989 shutdown_connections 2024-06-10T01:20:55.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.802+0000 7f1b5a703b80 1 -- 172.21.15.7:0/3136498989 wait complete. 2024-06-10T01:20:55.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.802+0000 7f1b5a703b80 20 client.19194 trim_cache size 0 max 16384 2024-06-10T01:20:55.797 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:20:55.797 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 44, 2024-06-10T01:20:55.797 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:20:55.798 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 52, 2024-06-10T01:20:55.798 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:20:55.798 INFO:teuthology.orchestra.run.smithi007.stdout: "8f7fcf77": { 2024-06-10T01:20:55.798 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 52, 2024-06-10T01:20:55.798 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:20:55.798 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:20:55.798 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:20:55.798 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:20:55.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:20:55.799 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:20:55.799 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 147.7, 2024-06-10T01:20:55.799 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:20:55.799 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:20:55.799 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:20:55.799 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:20:55.799 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:20:55.799 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:20:55.799 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:20:55.799 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:20:55.799 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:20:55.800 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:20:55.800 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:20:55.800 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:20:55.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.803+0000 7f1b4fe80700 1 -- 172.21.15.7:0/4183999219 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1b340a2c20 msgr2=0x7f1b340a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.803+0000 7f1b4fe80700 1 --2- 172.21.15.7:0/4183999219 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1b340a2c20 0x7f1b340a50e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1b48199c00 tx=0x7f1b44009190 comp rx=0 tx=0).stop 2024-06-10T01:20:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.803+0000 7f1b4fe80700 1 -- 172.21.15.7:0/4183999219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b48106b30 msgr2=0x7f1b48198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.803+0000 7f1b4fe80700 1 --2- 172.21.15.7:0/4183999219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b48106b30 0x7f1b48198ac0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f1b40009cd0 tx=0x7f1b400079a0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.803+0000 7f1b4fe80700 1 -- 172.21.15.7:0/4183999219 shutdown_connections 2024-06-10T01:20:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.803+0000 7f1b4fe80700 1 --2- 172.21.15.7:0/4183999219 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1b340a2c20 0x7f1b340a50e0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.803+0000 7f1b4fe80700 1 --2- 172.21.15.7:0/4183999219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b48106b30 0x7f1b48198ac0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.803+0000 7f1b4fe80700 1 --2- 172.21.15.7:0/4183999219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b48100ea0 0x7f1b4819e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.803+0000 7f1b4fe80700 1 --2- 172.21.15.7:0/4183999219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b48101860 0x7f1b4819ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.803+0000 7f1b4fe80700 1 -- 172.21.15.7:0/4183999219 >> 172.21.15.7:0/4183999219 conn(0x7f1b48077590 msgr2=0x7f1b480fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.803+0000 7f1b4fe80700 1 -- 172.21.15.7:0/4183999219 shutdown_connections 2024-06-10T01:20:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:55.803+0000 7f1b4fe80700 1 -- 172.21.15.7:0/4183999219 wait complete. 2024-06-10T01:20:55.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:55 smithi078 ceph-mon[36683]: pgmap v879: 129 pgs: 129 active+clean; 380 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 6.0 MiB/s wr, 235 op/s 2024-06-10T01:20:55.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:55 smithi175 ceph-mon[36636]: pgmap v879: 129 pgs: 129 active+clean; 380 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 6.0 MiB/s wr, 235 op/s 2024-06-10T01:20:56.234 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:20:56.473 INFO:tasks.workunit.client.0.smithi007.stdout: 10 20410 13.69 MB/sec warmup 93 sec latency 25378.095 ms 2024-06-10T01:20:56.512 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8f7fcf77', total seconds elapsed: 33.4 2024-06-10T01:20:56.633 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:20:57.474 INFO:tasks.workunit.client.0.smithi007.stdout: 10 21265 14.02 MB/sec warmup 94 sec latency 69.667 ms 2024-06-10T01:20:57.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:57 smithi007 ceph-mon[32398]: pgmap v880: 129 pgs: 129 active+clean; 380 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 3.4 MiB/s wr, 184 op/s 2024-06-10T01:20:57.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:57 smithi078 ceph-mon[36683]: pgmap v880: 129 pgs: 129 active+clean; 380 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 3.4 MiB/s wr, 184 op/s 2024-06-10T01:20:57.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:57 smithi175 ceph-mon[36636]: pgmap v880: 129 pgs: 129 active+clean; 380 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 3.4 MiB/s wr, 184 op/s 2024-06-10T01:20:58.474 INFO:tasks.workunit.client.0.smithi007.stdout: 10 22068 14.33 MB/sec warmup 95 sec latency 80.388 ms 2024-06-10T01:20:58.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.531+0000 7fbe351c3700 1 -- 172.21.15.7:0/3082260276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe30103ea0 msgr2=0x7fbe3010c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:58.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.531+0000 7fbe351c3700 1 --2- 172.21.15.7:0/3082260276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe30103ea0 0x7fbe3010c390 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fbe24009990 tx=0x7fbe24009ca0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.532+0000 7fbe351c3700 1 -- 172.21.15.7:0/3082260276 shutdown_connections 2024-06-10T01:20:58.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.532+0000 7fbe351c3700 1 --2- 172.21.15.7:0/3082260276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe3010c9b0 0x7fbe3010ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.532+0000 7fbe351c3700 1 --2- 172.21.15.7:0/3082260276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe30103580 0x7fbe30103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.532+0000 7fbe351c3700 1 --2- 172.21.15.7:0/3082260276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe30103ea0 0x7fbe3010c390 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.532+0000 7fbe351c3700 1 -- 172.21.15.7:0/3082260276 >> 172.21.15.7:0/3082260276 conn(0x7fbe300fcdb0 msgr2=0x7fbe300ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:58.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.533+0000 7fbe351c3700 1 -- 172.21.15.7:0/3082260276 shutdown_connections 2024-06-10T01:20:58.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.533+0000 7fbe351c3700 1 -- 172.21.15.7:0/3082260276 wait complete. 2024-06-10T01:20:58.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.534+0000 7fbe351c3700 1 Processor -- start 2024-06-10T01:20:58.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.534+0000 7fbe351c3700 1 -- start start 2024-06-10T01:20:58.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.534+0000 7fbe351c3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe30103580 0x7fbe3010c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:58.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.535+0000 7fbe351c3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe30103ea0 0x7fbe301093b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:58.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.535+0000 7fbe351c3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe3010c9b0 0x7fbe30109930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:58.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.535+0000 7fbe351c3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbe30102990 con 0x7fbe3010c9b0 2024-06-10T01:20:58.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.535+0000 7fbe351c3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbe30102810 con 0x7fbe30103580 2024-06-10T01:20:58.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.535+0000 7fbe351c3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbe30102b10 con 0x7fbe30103ea0 2024-06-10T01:20:58.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.535+0000 7fbe2ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe30103580 0x7fbe3010c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:58.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.535+0000 7fbe2ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe30103580 0x7fbe3010c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:45990/0 (socket says 172.21.15.7:45990) 2024-06-10T01:20:58.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.535+0000 7fbe2ffff700 1 -- 172.21.15.7:0/3488100779 learned_addr learned my addr 172.21.15.7:0/3488100779 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:58.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.535+0000 7fbe349c2700 1 --2- 172.21.15.7:0/3488100779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe3010c9b0 0x7fbe30109930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:58.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.535+0000 7fbe2ffff700 1 -- 172.21.15.7:0/3488100779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe30103ea0 msgr2=0x7fbe301093b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:58.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.536+0000 7fbe2f7fe700 1 --2- 172.21.15.7:0/3488100779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe30103ea0 0x7fbe301093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:58.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.536+0000 7fbe2ffff700 1 --2- 172.21.15.7:0/3488100779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe30103ea0 0x7fbe301093b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.536+0000 7fbe2ffff700 1 -- 172.21.15.7:0/3488100779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe3010c9b0 msgr2=0x7fbe30109930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:58.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.536+0000 7fbe2ffff700 1 --2- 172.21.15.7:0/3488100779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe3010c9b0 0x7fbe30109930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.536+0000 7fbe2ffff700 1 -- 172.21.15.7:0/3488100779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe3011c2d0 con 0x7fbe30103580 2024-06-10T01:20:58.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.536+0000 7fbe349c2700 1 --2- 172.21.15.7:0/3488100779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe3010c9b0 0x7fbe30109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:20:58.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.536+0000 7fbe2ffff700 1 --2- 172.21.15.7:0/3488100779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe30103580 0x7fbe3010c260 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000b550 tx=0x7fbe2000b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:58.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.536+0000 7fbe2d7fa700 1 -- 172.21.15.7:0/3488100779 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe20015020 con 0x7fbe30103580 2024-06-10T01:20:58.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.537+0000 7fbe351c3700 1 -- 172.21.15.7:0/3488100779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe3011c5c0 con 0x7fbe30103580 2024-06-10T01:20:58.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.537+0000 7fbe351c3700 1 -- 172.21.15.7:0/3488100779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe3011cb50 con 0x7fbe30103580 2024-06-10T01:20:58.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.537+0000 7fbe2d7fa700 1 -- 172.21.15.7:0/3488100779 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe20011d10 con 0x7fbe30103580 2024-06-10T01:20:58.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.538+0000 7fbe2d7fa700 1 -- 172.21.15.7:0/3488100779 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2000f2e0 con 0x7fbe30103580 2024-06-10T01:20:58.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.541+0000 7fbe2d7fa700 1 -- 172.21.15.7:0/3488100779 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fbe20018020 con 0x7fbe30103580 2024-06-10T01:20:58.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.543+0000 7fbe40ca8b80 1 Processor -- start 2024-06-10T01:20:58.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.543+0000 7fbe2d7fa700 1 --2- 172.21.15.7:0/3488100779 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fbe180a2ca0 0x7fbe180a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:58.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.543+0000 7fbe40ca8b80 1 -- start start 2024-06-10T01:20:58.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.543+0000 7fbe40ca8b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde2a278f0 0x55fde2a27cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:58.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.543+0000 7fbe40ca8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fde2a28210 0x55fde2a21b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:58.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.543+0000 7fbe40ca8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde2a22080 0x55fde2a224c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:58.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.543+0000 7fbe40ca8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fde2973810 con 0x55fde2a22080 2024-06-10T01:20:58.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.543+0000 7fbe40ca8b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fde291b720 con 0x55fde2a278f0 2024-06-10T01:20:58.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.543+0000 7fbe2d7fa700 1 -- 172.21.15.7:0/3488100779 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fbe200ceaa0 con 0x7fbe30103580 2024-06-10T01:20:58.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.543+0000 7fbe40ca8b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fde296f7d0 con 0x55fde2a28210 2024-06-10T01:20:58.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.544+0000 7fbe2f7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fde2a28210 0x55fde2a21b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:58.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.544+0000 7fbe349c2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde2a22080 0x55fde2a224c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:58.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.544+0000 7fbe2ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde2a278f0 0x55fde2a27cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:58.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.544+0000 7fbe349c2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde2a22080 0x55fde2a224c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49370/0 (socket says 172.21.15.7:49370) 2024-06-10T01:20:58.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.544+0000 7fbe349c2700 1 -- 172.21.15.7:0/805680098 learned_addr learned my addr 172.21.15.7:0/805680098 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:58.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.544+0000 7fbe349c2700 1 --2- 172.21.15.7:0/3488100779 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fbe180a2ca0 0x7fbe180a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:58.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.544+0000 7fbe2f7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fde2a28210 0x55fde2a21b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:57996/0 (socket says 172.21.15.7:57996) 2024-06-10T01:20:58.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.544+0000 7fbe2f7fe700 1 -- 172.21.15.7:0/805680098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde2a278f0 msgr2=0x55fde2a27cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:58.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.544+0000 7fbe2f7fe700 1 --2- 172.21.15.7:0/805680098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde2a278f0 0x55fde2a27cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.544+0000 7fbe2f7fe700 1 -- 172.21.15.7:0/805680098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde2a22080 msgr2=0x55fde2a224c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:58.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.544+0000 7fbe2f7fe700 1 --2- 172.21.15.7:0/805680098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde2a22080 0x55fde2a224c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.544+0000 7fbe2f7fe700 1 -- 172.21.15.7:0/805680098 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fde2911f50 con 0x55fde2a28210 2024-06-10T01:20:58.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.545+0000 7fbe2f7fe700 1 --2- 172.21.15.7:0/805680098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fde2a28210 0x55fde2a21b40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbe3010aad0 tx=0x7fbe24005da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=935c2a11422d8042 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:58.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.545+0000 7fbe349c2700 1 --2- 172.21.15.7:0/3488100779 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fbe180a2ca0 0x7fbe180a5160 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800ddf0 tx=0x7fbe28009b00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:58.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.545+0000 7fbe167fc700 1 -- 172.21.15.7:0/805680098 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe24003700 con 0x55fde2a28210 2024-06-10T01:20:58.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.545+0000 7fbe167fc700 1 -- 172.21.15.7:0/805680098 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe240050c0 con 0x55fde2a28210 2024-06-10T01:20:58.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.545+0000 7fbe167fc700 1 -- 172.21.15.7:0/805680098 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2400ba10 con 0x55fde2a28210 2024-06-10T01:20:58.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.546+0000 7fbe40ca8b80 1 -- 172.21.15.7:0/805680098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fde2a28210 msgr2=0x55fde2a21b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:58.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.546+0000 7fbe40ca8b80 1 --2- 172.21.15.7:0/805680098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fde2a28210 0x55fde2a21b40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbe3010aad0 tx=0x7fbe24005da0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.546+0000 7fbe40ca8b80 1 -- 172.21.15.7:0/805680098 shutdown_connections 2024-06-10T01:20:58.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.546+0000 7fbe40ca8b80 1 --2- 172.21.15.7:0/805680098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde2a22080 0x55fde2a224c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.546+0000 7fbe40ca8b80 1 --2- 172.21.15.7:0/805680098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde2a278f0 0x55fde2a27cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.546+0000 7fbe40ca8b80 1 --2- 172.21.15.7:0/805680098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fde2a28210 0x55fde2a21b40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.547+0000 7fbe40ca8b80 1 -- 172.21.15.7:0/805680098 >> 172.21.15.7:0/805680098 conn(0x55fde294ccc0 msgr2=0x55fde2a1caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:58.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.547+0000 7fbe40ca8b80 1 -- 172.21.15.7:0/805680098 shutdown_connections 2024-06-10T01:20:58.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.547+0000 7fbe40ca8b80 1 -- 172.21.15.7:0/805680098 wait complete. 2024-06-10T01:20:58.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.547+0000 7fbe40ca8b80 1 -- start start 2024-06-10T01:20:58.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.547+0000 7fbe40ca8b80 1 Processor -- start 2024-06-10T01:20:58.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.548+0000 7fbe40ca8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde2a22080 0x55fde2a2dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:58.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.548+0000 7fbe40ca8b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde2a278f0 0x55fde2a2b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:58.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.548+0000 7fbe40ca8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fde2a28210 0x55fde2a2b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:58.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.548+0000 7fbe40ca8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fde2a2eb70 con 0x55fde2a22080 2024-06-10T01:20:58.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.548+0000 7fbe40ca8b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fde2913bf0 con 0x55fde2a278f0 2024-06-10T01:20:58.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.548+0000 7fbe40ca8b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fde2a2ecf0 con 0x55fde2a28210 2024-06-10T01:20:58.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.548+0000 7fbe40ca8b80 10 client.0 _ll_register_callbacks cb 0x55fde290d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:20:58.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.548+0000 7fbe349c2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fde2a28210 0x55fde2a2b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:58.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.548+0000 7fbe349c2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fde2a28210 0x55fde2a2b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:57998/0 (socket says 172.21.15.7:57998) 2024-06-10T01:20:58.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.548+0000 7fbe349c2700 1 -- 172.21.15.7:0/4097335109 learned_addr learned my addr 172.21.15.7:0/4097335109 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:58.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.548+0000 7fbe2f7fe700 1 --2- 172.21.15.7:0/4097335109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde2a278f0 0x55fde2a2b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:58.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.549+0000 7fbe349c2700 1 -- 172.21.15.7:0/4097335109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde2a278f0 msgr2=0x55fde2a2b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:58.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.549+0000 7fbe349c2700 1 --2- 172.21.15.7:0/4097335109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde2a278f0 0x55fde2a2b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.549+0000 7fbe349c2700 1 -- 172.21.15.7:0/4097335109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde2a22080 msgr2=0x55fde2a2dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:58.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.549+0000 7fbe349c2700 1 --2- 172.21.15.7:0/4097335109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde2a22080 0x55fde2a2dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.549+0000 7fbe349c2700 1 -- 172.21.15.7:0/4097335109 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fde28c28d0 con 0x55fde2a28210 2024-06-10T01:20:58.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.549+0000 7fbe349c2700 1 --2- 172.21.15.7:0/4097335109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fde2a28210 0x55fde2a2b6e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800d2b0 tx=0x7fbe28009880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27ccb1243fd14c87 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:58.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.549+0000 7fbe0f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:20:58.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.550+0000 7fbe0f7fe700 1 -- 172.21.15.7:0/4097335109 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe28028060 con 0x55fde2a28210 2024-06-10T01:20:58.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.550+0000 7fbe40ca8b80 1 -- 172.21.15.7:0/4097335109 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe240050c0 con 0x55fde2a28210 2024-06-10T01:20:58.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.550+0000 7fbe0f7fe700 1 -- 172.21.15.7:0/4097335109 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fbe28023440 con 0x55fde2a28210 2024-06-10T01:20:58.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.550+0000 7fbe0f7fe700 1 -- 172.21.15.7:0/4097335109 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fbe28023640 con 0x55fde2a28210 2024-06-10T01:20:58.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.551+0000 7fbe40ca8b80 10 client.28480 fetch_fsmap learned FSMap version 44 2024-06-10T01:20:58.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.551+0000 7fbe40ca8b80 1 -- 172.21.15.7:0/4097335109 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55fde2a394d0 con 0x55fde2a28210 2024-06-10T01:20:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.552+0000 7fbe0f7fe700 1 -- 172.21.15.7:0/4097335109 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe28023cc0 con 0x55fde2a28210 2024-06-10T01:20:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.552+0000 7fbe0f7fe700 1 -- 172.21.15.7:0/4097335109 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fbe2801ba90 con 0x55fde2a28210 2024-06-10T01:20:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.552+0000 7fbe40ca8b80 10 client.28480 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:20:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.552+0000 7fbe40ca8b80 10 client.28480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:20:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.552+0000 7fbe40ca8b80 20 client.28480 populate_metadata read hostname 'smithi007' 2024-06-10T01:20:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.552+0000 7fbe40ca8b80 1 --2- 172.21.15.7:0/4097335109 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fde2a3ba60 0x55fde2a3ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.552+0000 7fbe40ca8b80 4 client.28480 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:20:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.553+0000 7fbe40ca8b80 1 -- 172.21.15.7:0/4097335109 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbe240050c0 con 0x55fde2a3ba60 2024-06-10T01:20:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.553+0000 7fbe2ffff700 1 --2- 172.21.15.7:0/4097335109 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fde2a3ba60 0x55fde2a3ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.561+0000 7fbe2ffff700 1 --2- 172.21.15.7:0/4097335109 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fde2a3ba60 0x55fde2a3ded0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8409351b60d87dc8 server_cookie=6bb5f754402b3102 in_seq=0 out_seq=0 2024-06-10T01:20:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.561+0000 7fbe0f7fe700 10 client.28480 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:58.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.565+0000 7fbe0f7fe700 1 -- 172.21.15.7:0/4097335109 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fbe240050c0 con 0x55fde2a3ba60 2024-06-10T01:20:58.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.565+0000 7fbe0f7fe700 10 client.28480 handle_command_reply: tid=0 2024-06-10T01:20:58.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.565+0000 7fbe40ca8b80 1 client.28480 shutdown 2024-06-10T01:20:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.567+0000 7fbe40ca8b80 1 -- 172.21.15.7:0/4097335109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fde2a28210 msgr2=0x55fde2a2b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.567+0000 7fbe40ca8b80 1 --2- 172.21.15.7:0/4097335109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fde2a28210 0x55fde2a2b6e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800d2b0 tx=0x7fbe28009880 comp rx=0 tx=0).stop 2024-06-10T01:20:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.567+0000 7fbe40ca8b80 1 -- 172.21.15.7:0/4097335109 shutdown_connections 2024-06-10T01:20:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.567+0000 7fbe40ca8b80 1 --2- 172.21.15.7:0/4097335109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fde2a28210 0x55fde2a2b6e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.567+0000 7fbe40ca8b80 1 --2- 172.21.15.7:0/4097335109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde2a22080 0x55fde2a2dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.567+0000 7fbe40ca8b80 1 --2- 172.21.15.7:0/4097335109 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fde2a3ba60 0x55fde2a3ded0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.567+0000 7fbe40ca8b80 1 --2- 172.21.15.7:0/4097335109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde2a278f0 0x55fde2a2b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.567+0000 7fbe40ca8b80 1 -- 172.21.15.7:0/4097335109 >> 172.21.15.7:0/4097335109 conn(0x55fde294ccc0 msgr2=0x55fde2a20b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.567+0000 7fbe0f7fe700 0 client.28480 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.567+0000 7fbe40ca8b80 1 -- 172.21.15.7:0/4097335109 shutdown_connections 2024-06-10T01:20:58.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.568+0000 7fbe40ca8b80 1 -- 172.21.15.7:0/4097335109 wait complete. 2024-06-10T01:20:58.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.568+0000 7fbe40ca8b80 20 client.28480 trim_cache size 0 max 16384 2024-06-10T01:20:58.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.605+0000 7fbe40ca8b80 1 Processor -- start 2024-06-10T01:20:58.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.605+0000 7fbe40ca8b80 1 -- start start 2024-06-10T01:20:58.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.605+0000 7fbe40ca8b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde2a22080 0x55fde2a28da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:58.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.605+0000 7fbe40ca8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde2a278f0 0x55fde2a2aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:58.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.605+0000 7fbe40ca8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fde2a28210 0x55fde2a21380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:58.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.605+0000 7fbe40ca8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c0113e0 con 0x55fde2a278f0 2024-06-10T01:20:58.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.605+0000 7fbe40ca8b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c00ebe0 con 0x55fde2a22080 2024-06-10T01:20:58.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.605+0000 7fbe40ca8b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c00ca30 con 0x55fde2a28210 2024-06-10T01:20:58.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.606+0000 7fbe2ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde2a22080 0x55fde2a28da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:58.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.606+0000 7fbe2f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde2a278f0 0x55fde2a2aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:58.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.606+0000 7fbe2ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde2a22080 0x55fde2a28da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:46018/0 (socket says 172.21.15.7:46018) 2024-06-10T01:20:58.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.606+0000 7fbe2ffff700 1 -- 172.21.15.7:0/705851571 learned_addr learned my addr 172.21.15.7:0/705851571 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:58.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.606+0000 7fbe349c2700 1 --2- 172.21.15.7:0/705851571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fde2a28210 0x55fde2a21380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:58.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.606+0000 7fbe2ffff700 1 -- 172.21.15.7:0/705851571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fde2a28210 msgr2=0x55fde2a21380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:58.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.606+0000 7fbe2ffff700 1 --2- 172.21.15.7:0/705851571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fde2a28210 0x55fde2a21380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.606+0000 7fbe2ffff700 1 -- 172.21.15.7:0/705851571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde2a278f0 msgr2=0x55fde2a2aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:58.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.606+0000 7fbe2ffff700 1 --2- 172.21.15.7:0/705851571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde2a278f0 0x55fde2a2aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.606+0000 7fbe2ffff700 1 -- 172.21.15.7:0/705851571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fde2a29710 con 0x55fde2a22080 2024-06-10T01:20:58.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.606+0000 7fbe2ffff700 1 --2- 172.21.15.7:0/705851571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde2a22080 0x55fde2a28da0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbe200973e0 tx=0x7fbe20011ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f00d8e87d3de93af server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:58.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.607+0000 7fbe15ffb700 1 -- 172.21.15.7:0/705851571 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe200b3070 con 0x55fde2a22080 2024-06-10T01:20:58.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.607+0000 7fbe15ffb700 1 -- 172.21.15.7:0/705851571 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe200b5030 con 0x55fde2a22080 2024-06-10T01:20:58.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.607+0000 7fbe15ffb700 1 -- 172.21.15.7:0/705851571 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2009b440 con 0x55fde2a22080 2024-06-10T01:20:58.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.607+0000 7fbe40ca8b80 1 -- 172.21.15.7:0/705851571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde2a22080 msgr2=0x55fde2a28da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:58.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.607+0000 7fbe40ca8b80 1 --2- 172.21.15.7:0/705851571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde2a22080 0x55fde2a28da0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbe200973e0 tx=0x7fbe20011ec0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.607+0000 7fbe40ca8b80 1 -- 172.21.15.7:0/705851571 shutdown_connections 2024-06-10T01:20:58.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.607+0000 7fbe40ca8b80 1 --2- 172.21.15.7:0/705851571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fde2a28210 0x55fde2a21380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.607+0000 7fbe40ca8b80 1 --2- 172.21.15.7:0/705851571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde2a22080 0x55fde2a28da0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.607+0000 7fbe40ca8b80 1 --2- 172.21.15.7:0/705851571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde2a278f0 0x55fde2a2aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.608+0000 7fbe40ca8b80 1 -- 172.21.15.7:0/705851571 >> 172.21.15.7:0/705851571 conn(0x55fde294ccc0 msgr2=0x55fde2a28690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:58.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.608+0000 7fbe40ca8b80 1 -- 172.21.15.7:0/705851571 shutdown_connections 2024-06-10T01:20:58.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.608+0000 7fbe40ca8b80 1 -- 172.21.15.7:0/705851571 wait complete. 2024-06-10T01:20:58.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.608+0000 7fbe40ca8b80 1 -- start start 2024-06-10T01:20:58.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.608+0000 7fbe40ca8b80 1 Processor -- start 2024-06-10T01:20:58.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.609+0000 7fbe40ca8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde2a22080 0x55fde2a346b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:58.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.609+0000 7fbe40ca8b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde2a278f0 0x55fde2a430a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:58.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.609+0000 7fbe40ca8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fde2a28210 0x55fde2a435e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:58.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.609+0000 7fbe40ca8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c00d0a0 con 0x55fde2a22080 2024-06-10T01:20:58.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.609+0000 7fbe40ca8b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c00e510 con 0x55fde2a278f0 2024-06-10T01:20:58.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.609+0000 7fbe40ca8b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c00d770 con 0x55fde2a28210 2024-06-10T01:20:58.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.609+0000 7fbe40ca8b80 10 client.0 _ll_register_callbacks cb 0x55fde290d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:20:58.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.609+0000 7fbe2ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde2a22080 0x55fde2a346b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:58.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.609+0000 7fbe2f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde2a278f0 0x55fde2a430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:58.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.609+0000 7fbe2ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde2a22080 0x55fde2a346b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49392/0 (socket says 172.21.15.7:49392) 2024-06-10T01:20:58.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.609+0000 7fbe2f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde2a278f0 0x55fde2a430a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:46034/0 (socket says 172.21.15.7:46034) 2024-06-10T01:20:58.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.609+0000 7fbe2ffff700 1 -- 172.21.15.7:0/2563437153 learned_addr learned my addr 172.21.15.7:0/2563437153 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:20:58.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.609+0000 7fbe349c2700 1 --2- 172.21.15.7:0/2563437153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fde2a28210 0x55fde2a435e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:58.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.609+0000 7fbe2ffff700 1 -- 172.21.15.7:0/2563437153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fde2a28210 msgr2=0x55fde2a435e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:58.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.609+0000 7fbe2ffff700 1 --2- 172.21.15.7:0/2563437153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fde2a28210 0x55fde2a435e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.609+0000 7fbe2ffff700 1 -- 172.21.15.7:0/2563437153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde2a278f0 msgr2=0x55fde2a430a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:20:58.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.609+0000 7fbe2ffff700 1 --2- 172.21.15.7:0/2563437153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde2a278f0 0x55fde2a430a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.610+0000 7fbe2ffff700 1 -- 172.21.15.7:0/2563437153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fde2a43e20 con 0x55fde2a22080 2024-06-10T01:20:58.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.610+0000 7fbe2f7fe700 1 --2- 172.21.15.7:0/2563437153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde2a278f0 0x55fde2a430a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:20:58.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.610+0000 7fbe2ffff700 1 --2- 172.21.15.7:0/2563437153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde2a22080 0x55fde2a346b0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fbe2009af10 tx=0x7fbe20009450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d44a84fd814817 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:20:58.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.610+0000 7fbe0f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:20:58.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.610+0000 7fbe0f7fe700 1 -- 172.21.15.7:0/2563437153 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2007e060 con 0x55fde2a22080 2024-06-10T01:20:58.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.610+0000 7fbe0f7fe700 1 -- 172.21.15.7:0/2563437153 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fbe20099280 con 0x55fde2a22080 2024-06-10T01:20:58.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.610+0000 7fbe40ca8b80 1 -- 172.21.15.7:0/2563437153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe200b5030 con 0x55fde2a22080 2024-06-10T01:20:58.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.610+0000 7fbe0f7fe700 1 -- 172.21.15.7:0/2563437153 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fbe200b3070 con 0x55fde2a22080 2024-06-10T01:20:58.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.610+0000 7fbe40ca8b80 10 client.19209 fetch_fsmap learned FSMap version 44 2024-06-10T01:20:58.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.610+0000 7fbe40ca8b80 1 -- 172.21.15.7:0/2563437153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55fde2aa6740 con 0x55fde2a22080 2024-06-10T01:20:58.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.610+0000 7fbe0f7fe700 1 -- 172.21.15.7:0/2563437153 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe200b5030 con 0x55fde2a22080 2024-06-10T01:20:58.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.611+0000 7fbe0f7fe700 1 -- 172.21.15.7:0/2563437153 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fbe2009b270 con 0x55fde2a22080 2024-06-10T01:20:58.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.611+0000 7fbe40ca8b80 10 client.19209 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:20:58.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.611+0000 7fbe40ca8b80 10 client.19209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:20:58.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.611+0000 7fbe40ca8b80 20 client.19209 populate_metadata read hostname 'smithi007' 2024-06-10T01:20:58.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.611+0000 7fbe40ca8b80 1 --2- 172.21.15.7:0/2563437153 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fde2aa6e10 0x55fde2aa9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:20:58.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.611+0000 7fbe40ca8b80 4 client.19209 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:20:58.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.611+0000 7fbe40ca8b80 1 -- 172.21.15.7:0/2563437153 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fde2980d10 con 0x55fde2aa6e10 2024-06-10T01:20:58.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.611+0000 7fbe2f7fe700 1 --2- 172.21.15.7:0/2563437153 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fde2aa6e10 0x55fde2aa9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:20:58.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.612+0000 7fbe2f7fe700 1 --2- 172.21.15.7:0/2563437153 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fde2aa6e10 0x55fde2aa9230 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=602d71670e2c27ac server_cookie=5fa54f6bbd6ea1ed in_seq=0 out_seq=0 2024-06-10T01:20:58.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.612+0000 7fbe0f7fe700 10 client.19209 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:58.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.616+0000 7fbe0f7fe700 1 -- 172.21.15.7:0/2563437153 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fde2980d10 con 0x55fde2aa6e10 2024-06-10T01:20:58.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.616+0000 7fbe0f7fe700 10 client.19209 handle_command_reply: tid=0 2024-06-10T01:20:58.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.616+0000 7fbe40ca8b80 1 client.19209 shutdown 2024-06-10T01:20:58.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.617+0000 7fbe40ca8b80 1 -- 172.21.15.7:0/2563437153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde2a22080 msgr2=0x55fde2a346b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:58.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.617+0000 7fbe40ca8b80 1 --2- 172.21.15.7:0/2563437153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde2a22080 0x55fde2a346b0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fbe2009af10 tx=0x7fbe20009450 comp rx=0 tx=0).stop 2024-06-10T01:20:58.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.617+0000 7fbe40ca8b80 1 -- 172.21.15.7:0/2563437153 shutdown_connections 2024-06-10T01:20:58.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.617+0000 7fbe40ca8b80 1 --2- 172.21.15.7:0/2563437153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fde2a28210 0x55fde2a435e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.617+0000 7fbe40ca8b80 1 --2- 172.21.15.7:0/2563437153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde2a22080 0x55fde2a346b0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.617+0000 7fbe40ca8b80 1 --2- 172.21.15.7:0/2563437153 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fde2aa6e10 0x55fde2aa9230 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.617+0000 7fbe40ca8b80 1 --2- 172.21.15.7:0/2563437153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde2a278f0 0x55fde2a430a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.617+0000 7fbe0f7fe700 0 client.19209 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:20:58.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.617+0000 7fbe40ca8b80 1 -- 172.21.15.7:0/2563437153 >> 172.21.15.7:0/2563437153 conn(0x55fde294ccc0 msgr2=0x55fde2a1c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:58.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.617+0000 7fbe40ca8b80 1 -- 172.21.15.7:0/2563437153 shutdown_connections 2024-06-10T01:20:58.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.617+0000 7fbe40ca8b80 1 -- 172.21.15.7:0/2563437153 wait complete. 2024-06-10T01:20:58.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.617+0000 7fbe40ca8b80 20 client.19209 trim_cache size 0 max 16384 2024-06-10T01:20:58.612 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:20:58.612 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-10T01:20:58.612 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-10T01:20:58.612 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:20:58.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.618+0000 7fbe351c3700 1 -- 172.21.15.7:0/3488100779 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fbe180a2ca0 msgr2=0x7fbe180a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:58.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.618+0000 7fbe351c3700 1 --2- 172.21.15.7:0/3488100779 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fbe180a2ca0 0x7fbe180a5160 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800ddf0 tx=0x7fbe28009b00 comp rx=0 tx=0).stop 2024-06-10T01:20:58.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.618+0000 7fbe351c3700 1 -- 172.21.15.7:0/3488100779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe30103580 msgr2=0x7fbe3010c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:20:58.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.618+0000 7fbe351c3700 1 --2- 172.21.15.7:0/3488100779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe30103580 0x7fbe3010c260 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000b550 tx=0x7fbe2000b910 comp rx=0 tx=0).stop 2024-06-10T01:20:58.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.618+0000 7fbe351c3700 1 -- 172.21.15.7:0/3488100779 shutdown_connections 2024-06-10T01:20:58.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.618+0000 7fbe351c3700 1 --2- 172.21.15.7:0/3488100779 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fbe180a2ca0 0x7fbe180a5160 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.618+0000 7fbe351c3700 1 --2- 172.21.15.7:0/3488100779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe3010c9b0 0x7fbe30109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.618+0000 7fbe351c3700 1 --2- 172.21.15.7:0/3488100779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe30103580 0x7fbe3010c260 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.618+0000 7fbe351c3700 1 --2- 172.21.15.7:0/3488100779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe30103ea0 0x7fbe301093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:20:58.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.618+0000 7fbe351c3700 1 -- 172.21.15.7:0/3488100779 >> 172.21.15.7:0/3488100779 conn(0x7fbe300fcdb0 msgr2=0x7fbe3010e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:20:58.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.618+0000 7fbe351c3700 1 -- 172.21.15.7:0/3488100779 shutdown_connections 2024-06-10T01:20:58.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:20:58.618+0000 7fbe351c3700 1 -- 172.21.15.7:0/3488100779 wait complete. 2024-06-10T01:20:59.333 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:20:59.333 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:20:59.474 INFO:tasks.workunit.client.0.smithi007.stdout: 10 22905 14.67 MB/sec warmup 96 sec latency 52.231 ms 2024-06-10T01:20:59.672 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:20:59.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:20:59 smithi007 ceph-mon[32398]: pgmap v881: 129 pgs: 129 active+clean; 381 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.9 MiB/s wr, 165 op/s 2024-06-10T01:20:59.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:20:59 smithi078 ceph-mon[36683]: pgmap v881: 129 pgs: 129 active+clean; 381 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.9 MiB/s wr, 165 op/s 2024-06-10T01:20:59.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:20:59 smithi175 ceph-mon[36636]: pgmap v881: 129 pgs: 129 active+clean; 381 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.9 MiB/s wr, 165 op/s 2024-06-10T01:21:00.474 INFO:tasks.workunit.client.0.smithi007.stdout: 10 23710 14.94 MB/sec warmup 97 sec latency 49.895 ms 2024-06-10T01:21:01.485 INFO:tasks.workunit.client.0.smithi007.stdout: 10 24539 15.24 MB/sec warmup 98 sec latency 62.868 ms 2024-06-10T01:21:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.685+0000 7f7c3b34f700 1 -- 172.21.15.7:0/459252909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c3406ab10 msgr2=0x7f7c3410f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.685+0000 7f7c3b34f700 1 --2- 172.21.15.7:0/459252909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c3406ab10 0x7f7c3410f320 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00b0d0 tx=0x7f7c2c00b3e0 comp rx=0 tx=0).stop 2024-06-10T01:21:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.685+0000 7f7c3b34f700 1 -- 172.21.15.7:0/459252909 shutdown_connections 2024-06-10T01:21:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.685+0000 7f7c3b34f700 1 --2- 172.21.15.7:0/459252909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c3406ab10 0x7f7c3410f320 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.685+0000 7f7c3b34f700 1 --2- 172.21.15.7:0/459252909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c341084e0 0x7f7c341088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.685+0000 7f7c3b34f700 1 --2- 172.21.15.7:0/459252909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c3406a170 0x7f7c3406a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.685+0000 7f7c3b34f700 1 -- 172.21.15.7:0/459252909 >> 172.21.15.7:0/459252909 conn(0x7f7c340782c0 msgr2=0x7f7c340786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.685+0000 7f7c3b34f700 1 -- 172.21.15.7:0/459252909 shutdown_connections 2024-06-10T01:21:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.685+0000 7f7c3b34f700 1 -- 172.21.15.7:0/459252909 wait complete. 2024-06-10T01:21:01.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.686+0000 7f7c3b34f700 1 Processor -- start 2024-06-10T01:21:01.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.686+0000 7f7c3b34f700 1 -- start start 2024-06-10T01:21:01.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.686+0000 7f7c3b34f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c3406a170 0x7f7c34101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:01.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.686+0000 7f7c3b34f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c3406ab10 0x7f7c341019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:01.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.686+0000 7f7c3b34f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c341084e0 0x7f7c34101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:01.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.686+0000 7f7c3b34f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7c34112780 con 0x7f7c3406a170 2024-06-10T01:21:01.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.686+0000 7f7c3b34f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7c34112600 con 0x7f7c341084e0 2024-06-10T01:21:01.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.686+0000 7f7c3b34f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7c34112900 con 0x7f7c3406ab10 2024-06-10T01:21:01.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.687+0000 7f7c390eb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c3406a170 0x7f7c34101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:01.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.687+0000 7f7c390eb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c3406a170 0x7f7c34101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49404/0 (socket says 172.21.15.7:49404) 2024-06-10T01:21:01.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.687+0000 7f7c398ec700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c341084e0 0x7f7c34101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:01.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.687+0000 7f7c390eb700 1 -- 172.21.15.7:0/3116175066 learned_addr learned my addr 172.21.15.7:0/3116175066 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:01.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.687+0000 7f7c388ea700 1 --2- 172.21.15.7:0/3116175066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c3406ab10 0x7f7c341019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:01.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.687+0000 7f7c390eb700 1 -- 172.21.15.7:0/3116175066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c3406ab10 msgr2=0x7f7c341019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:01.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.687+0000 7f7c390eb700 1 --2- 172.21.15.7:0/3116175066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c3406ab10 0x7f7c341019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:01.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.687+0000 7f7c390eb700 1 -- 172.21.15.7:0/3116175066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c341084e0 msgr2=0x7f7c34101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:01.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.687+0000 7f7c390eb700 1 --2- 172.21.15.7:0/3116175066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c341084e0 0x7f7c34101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:01.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.687+0000 7f7c390eb700 1 -- 172.21.15.7:0/3116175066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c341a7100 con 0x7f7c3406a170 2024-06-10T01:21:01.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.687+0000 7f7c398ec700 1 --2- 172.21.15.7:0/3116175066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c341084e0 0x7f7c34101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:21:01.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.687+0000 7f7c390eb700 1 --2- 172.21.15.7:0/3116175066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c3406a170 0x7f7c34101460 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000e9a0 tx=0x7f7c3000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:01.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.687+0000 7f7c2a7fc700 1 -- 172.21.15.7:0/3116175066 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3000ca10 con 0x7f7c3406a170 2024-06-10T01:21:01.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.687+0000 7f7c3b34f700 1 -- 172.21.15.7:0/3116175066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c341a73f0 con 0x7f7c3406a170 2024-06-10T01:21:01.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.687+0000 7f7c2a7fc700 1 -- 172.21.15.7:0/3116175066 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c3000cbb0 con 0x7f7c3406a170 2024-06-10T01:21:01.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.687+0000 7f7c3b34f700 1 -- 172.21.15.7:0/3116175066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c341a7980 con 0x7f7c3406a170 2024-06-10T01:21:01.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.687+0000 7f7c2a7fc700 1 -- 172.21.15.7:0/3116175066 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30018680 con 0x7f7c3406a170 2024-06-10T01:21:01.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.688+0000 7f7c3b34f700 1 -- 172.21.15.7:0/3116175066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c34069180 con 0x7f7c3406a170 2024-06-10T01:21:01.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.690+0000 7f7c2a7fc700 1 -- 172.21.15.7:0/3116175066 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f7c30010430 con 0x7f7c3406a170 2024-06-10T01:21:01.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.690+0000 7f7c2a7fc700 1 --2- 172.21.15.7:0/3116175066 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7c200a2c20 0x7f7c200a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:01.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.690+0000 7f7c2a7fc700 1 -- 172.21.15.7:0/3116175066 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f7c300cf5b0 con 0x7f7c3406a170 2024-06-10T01:21:01.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.690+0000 7f7c388ea700 1 --2- 172.21.15.7:0/3116175066 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7c200a2c20 0x7f7c200a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:01.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.691+0000 7f7c388ea700 1 --2- 172.21.15.7:0/3116175066 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7c200a2c20 0x7f7c200a50e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7c34078850 tx=0x7f7c24009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:01.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:01.696+0000 7f7c2a7fc700 1 -- 172.21.15.7:0/3116175066 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7c3009c190 con 0x7f7c3406a170 2024-06-10T01:21:01.689 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:01 smithi007 ceph-mon[32398]: pgmap v882: 129 pgs: 129 active+clean; 378 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 908 KiB/s wr, 195 op/s 2024-06-10T01:21:01.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:01 smithi078 ceph-mon[36683]: pgmap v882: 129 pgs: 129 active+clean; 378 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 908 KiB/s wr, 195 op/s 2024-06-10T01:21:01.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:01 smithi175 ceph-mon[36636]: pgmap v882: 129 pgs: 129 active+clean; 378 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 908 KiB/s wr, 195 op/s 2024-06-10T01:21:02.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:02.025+0000 7f7c3b34f700 1 -- 172.21.15.7:0/3116175066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c34102990 con 0x7f7c3406a170 2024-06-10T01:21:02.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:02.026+0000 7f7c2a7fc700 1 -- 172.21.15.7:0/3116175066 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11860 (secure 0 0 0) 0x7f7c3009ba90 con 0x7f7c3406a170 2024-06-10T01:21:02.019 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:21:02.020 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:19:47.239285+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:21:02.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:02.028+0000 7f7c3b34f700 1 -- 172.21.15.7:0/3116175066 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7c200a2c20 msgr2=0x7f7c200a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:02.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:02.028+0000 7f7c3b34f700 1 --2- 172.21.15.7:0/3116175066 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7c200a2c20 0x7f7c200a50e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7c34078850 tx=0x7f7c24009260 comp rx=0 tx=0).stop 2024-06-10T01:21:02.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:02.028+0000 7f7c3b34f700 1 -- 172.21.15.7:0/3116175066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c3406a170 msgr2=0x7f7c34101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:02.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:02.028+0000 7f7c3b34f700 1 --2- 172.21.15.7:0/3116175066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c3406a170 0x7f7c34101460 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000e9a0 tx=0x7f7c3000ecb0 comp rx=0 tx=0).stop 2024-06-10T01:21:02.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:02.028+0000 7f7c3b34f700 1 -- 172.21.15.7:0/3116175066 shutdown_connections 2024-06-10T01:21:02.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:02.028+0000 7f7c3b34f700 1 --2- 172.21.15.7:0/3116175066 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7c200a2c20 0x7f7c200a50e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:02.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:02.028+0000 7f7c3b34f700 1 --2- 172.21.15.7:0/3116175066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c341084e0 0x7f7c34101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:02.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:02.028+0000 7f7c3b34f700 1 --2- 172.21.15.7:0/3116175066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c3406a170 0x7f7c34101460 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:02.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:02.028+0000 7f7c3b34f700 1 --2- 172.21.15.7:0/3116175066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c3406ab10 0x7f7c341019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:02.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:02.028+0000 7f7c3b34f700 1 -- 172.21.15.7:0/3116175066 >> 172.21.15.7:0/3116175066 conn(0x7f7c340782c0 msgr2=0x7f7c340ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:02.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:02.028+0000 7f7c3b34f700 1 -- 172.21.15.7:0/3116175066 shutdown_connections 2024-06-10T01:21:02.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:02.029+0000 7f7c3b34f700 1 -- 172.21.15.7:0/3116175066 wait complete. 2024-06-10T01:21:02.023 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 44 2024-06-10T01:21:02.485 INFO:tasks.workunit.client.0.smithi007.stdout: 10 25324 15.54 MB/sec warmup 99 sec latency 56.033 ms 2024-06-10T01:21:02.743 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:02 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3116175066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:21:02.772 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:21:02.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:02 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3116175066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:21:02.953 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:02 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3116175066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:21:03.047 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:21:03.485 INFO:tasks.workunit.client.0.smithi007.stdout: 10 26155 15.86 MB/sec warmup 100 sec latency 35.133 ms 2024-06-10T01:21:03.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:03 smithi007 ceph-mon[32398]: pgmap v883: 129 pgs: 129 active+clean; 381 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.5 MiB/s wr, 220 op/s 2024-06-10T01:21:03.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:03 smithi078 ceph-mon[36683]: pgmap v883: 129 pgs: 129 active+clean; 381 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.5 MiB/s wr, 220 op/s 2024-06-10T01:21:03.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:03 smithi175 ceph-mon[36636]: pgmap v883: 129 pgs: 129 active+clean; 381 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.5 MiB/s wr, 220 op/s 2024-06-10T01:21:04.485 INFO:tasks.workunit.client.0.smithi007.stdout: 10 26740 16.03 MB/sec warmup 101 sec latency 83.534 ms 2024-06-10T01:21:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.709+0000 7f9fa48ba700 1 -- 172.21.15.7:0/1921197057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fa0107840 msgr2=0x7f9fa010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.709+0000 7f9fa48ba700 1 --2- 172.21.15.7:0/1921197057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fa0107840 0x7f9fa010f3c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9f9800b0d0 tx=0x7f9f9800b3e0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.710+0000 7f9fa48ba700 1 -- 172.21.15.7:0/1921197057 shutdown_connections 2024-06-10T01:21:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.710+0000 7f9fa48ba700 1 --2- 172.21.15.7:0/1921197057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fa0107840 0x7f9fa010f3c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.710+0000 7f9fa48ba700 1 --2- 172.21.15.7:0/1921197057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa0102740 0x7f9fa0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.710+0000 7f9fa48ba700 1 --2- 172.21.15.7:0/1921197057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fa01030f0 0x7f9fa0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.710+0000 7f9fa48ba700 1 -- 172.21.15.7:0/1921197057 >> 172.21.15.7:0/1921197057 conn(0x7f9fa00795f0 msgr2=0x7f9fa0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.710+0000 7f9fa48ba700 1 -- 172.21.15.7:0/1921197057 shutdown_connections 2024-06-10T01:21:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.710+0000 7f9fa48ba700 1 -- 172.21.15.7:0/1921197057 wait complete. 2024-06-10T01:21:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.711+0000 7f9fa48ba700 1 Processor -- start 2024-06-10T01:21:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.711+0000 7f9fa48ba700 1 -- start start 2024-06-10T01:21:05.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.711+0000 7f9fa48ba700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fa0102740 0x7f9fa010f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.711+0000 7f9fa48ba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa01030f0 0x7f9fa010c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.711+0000 7f9fa48ba700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fa0107840 0x7f9fa010c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.711+0000 7f9fa48ba700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa00762f0 con 0x7f9fa01030f0 2024-06-10T01:21:05.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.711+0000 7f9fa48ba700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa0076170 con 0x7f9fa0102740 2024-06-10T01:21:05.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.711+0000 7f9fa48ba700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa0076470 con 0x7f9fa0107840 2024-06-10T01:21:05.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.712+0000 7f9f9e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fa0102740 0x7f9fa010f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:05.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.712+0000 7f9f9e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fa0102740 0x7f9fa010f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:41738/0 (socket says 172.21.15.7:41738) 2024-06-10T01:21:05.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.712+0000 7f9f9e59c700 1 -- 172.21.15.7:0/3743974484 learned_addr learned my addr 172.21.15.7:0/3743974484 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:05.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.712+0000 7f9f9ed9d700 1 --2- 172.21.15.7:0/3743974484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fa0107840 0x7f9fa010c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:05.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.712+0000 7f9f9dd9b700 1 --2- 172.21.15.7:0/3743974484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa01030f0 0x7f9fa010c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:05.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.712+0000 7f9f9e59c700 1 -- 172.21.15.7:0/3743974484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fa0107840 msgr2=0x7f9fa010c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:05.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.712+0000 7f9f9e59c700 1 --2- 172.21.15.7:0/3743974484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fa0107840 0x7f9fa010c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.712+0000 7f9f9e59c700 1 -- 172.21.15.7:0/3743974484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa01030f0 msgr2=0x7f9fa010c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:05.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.712+0000 7f9f9e59c700 1 --2- 172.21.15.7:0/3743974484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa01030f0 0x7f9fa010c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.712+0000 7f9f9e59c700 1 -- 172.21.15.7:0/3743974484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa010cf40 con 0x7f9fa0102740 2024-06-10T01:21:05.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.712+0000 7f9f9dd9b700 1 --2- 172.21.15.7:0/3743974484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa01030f0 0x7f9fa010c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:21:05.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.713+0000 7f9f9e59c700 1 --2- 172.21.15.7:0/3743974484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fa0102740 0x7f9fa010f140 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9f9000e9a0 tx=0x7f9f9000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:05.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.714+0000 7f9f977fe700 1 -- 172.21.15.7:0/3743974484 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9000cbd0 con 0x7f9fa0102740 2024-06-10T01:21:05.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.714+0000 7f9f977fe700 1 -- 172.21.15.7:0/3743974484 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f9000cd70 con 0x7f9fa0102740 2024-06-10T01:21:05.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.715+0000 7f9f977fe700 1 -- 172.21.15.7:0/3743974484 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9000cbd0 con 0x7f9fa0102740 2024-06-10T01:21:05.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.715+0000 7f9fa48ba700 1 -- 172.21.15.7:0/3743974484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fa01ab760 con 0x7f9fa0102740 2024-06-10T01:21:05.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.715+0000 7f9fa48ba700 1 -- 172.21.15.7:0/3743974484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fa01abc20 con 0x7f9fa0102740 2024-06-10T01:21:05.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.720+0000 7f9faf13db80 1 Processor -- start 2024-06-10T01:21:05.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.721+0000 7f9faf13db80 1 -- start start 2024-06-10T01:21:05.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.721+0000 7f9faf13db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55617f782640 0x55617f782a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.721+0000 7f9f977fe700 1 -- 172.21.15.7:0/3743974484 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9f90010430 con 0x7f9fa0102740 2024-06-10T01:21:05.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.721+0000 7f9faf13db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55617f782f60 0x55617f77c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.721+0000 7f9faf13db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55617f77cf20 0x55617f77d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.721+0000 7f9faf13db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55617f6ce810 con 0x55617f782640 2024-06-10T01:21:05.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.721+0000 7f9faf13db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55617f676720 con 0x55617f782f60 2024-06-10T01:21:05.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.721+0000 7f9faf13db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55617f6ca7d0 con 0x55617f77cf20 2024-06-10T01:21:05.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.721+0000 7f9f9ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55617f782640 0x55617f782a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:05.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.721+0000 7f9f9ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55617f782640 0x55617f782a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53138/0 (socket says 172.21.15.7:53138) 2024-06-10T01:21:05.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.721+0000 7f9f9ed9d700 1 -- 172.21.15.7:0/167603550 learned_addr learned my addr 172.21.15.7:0/167603550 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:05.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.722+0000 7f9f9ed9d700 1 -- 172.21.15.7:0/167603550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55617f77cf20 msgr2=0x55617f77d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:05.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.722+0000 7f9f9ed9d700 1 --2- 172.21.15.7:0/167603550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55617f77cf20 0x55617f77d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.722+0000 7f9f9ed9d700 1 -- 172.21.15.7:0/167603550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55617f782f60 msgr2=0x55617f77c9e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T01:21:05.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.722+0000 7f9f9ed9d700 1 --2- 172.21.15.7:0/167603550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55617f782f60 0x55617f77c9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.722+0000 7f9f9ed9d700 1 -- 172.21.15.7:0/167603550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55617f66cf50 con 0x55617f782640 2024-06-10T01:21:05.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.722+0000 7f9f9ed9d700 1 --2- 172.21.15.7:0/167603550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55617f782640 0x55617f782a20 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f9f98015000 tx=0x7f9f98009d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d882f2839bc403 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:05.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.723+0000 7f9f977fe700 1 --2- 172.21.15.7:0/3743974484 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9f8c0a3090 0x7f9f8c0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.723+0000 7f9f977fe700 1 -- 172.21.15.7:0/3743974484 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f9f9000c9d0 con 0x7f9fa0102740 2024-06-10T01:21:05.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.724+0000 7f9f9e59c700 1 --2- 172.21.15.7:0/3743974484 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9f8c0a3090 0x7f9f8c0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:05.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.725+0000 7f9f94ff9700 1 -- 172.21.15.7:0/167603550 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f980094a0 con 0x55617f782640 2024-06-10T01:21:05.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.725+0000 7f9f94ff9700 1 -- 172.21.15.7:0/167603550 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f9800bca0 con 0x55617f782640 2024-06-10T01:21:05.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.726+0000 7f9faf13db80 1 -- 172.21.15.7:0/167603550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55617f782640 msgr2=0x55617f782a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:05.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.726+0000 7f9faf13db80 1 --2- 172.21.15.7:0/167603550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55617f782640 0x55617f782a20 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f9f98015000 tx=0x7f9f98009d90 comp rx=0 tx=0).stop 2024-06-10T01:21:05.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.726+0000 7f9f9e59c700 1 --2- 172.21.15.7:0/3743974484 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9f8c0a3090 0x7f9f8c0a5550 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9f9009b230 tx=0x7f9f90010d70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:05.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.728+0000 7f9faf13db80 1 -- 172.21.15.7:0/167603550 shutdown_connections 2024-06-10T01:21:05.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.728+0000 7f9faf13db80 1 --2- 172.21.15.7:0/167603550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55617f77cf20 0x55617f77d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.728+0000 7f9faf13db80 1 --2- 172.21.15.7:0/167603550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55617f782640 0x55617f782a20 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.728+0000 7f9faf13db80 1 --2- 172.21.15.7:0/167603550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55617f782f60 0x55617f77c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.728+0000 7f9faf13db80 1 -- 172.21.15.7:0/167603550 >> 172.21.15.7:0/167603550 conn(0x55617f6a7cc0 msgr2=0x55617f777850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:05.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.729+0000 7f9faf13db80 1 -- 172.21.15.7:0/167603550 shutdown_connections 2024-06-10T01:21:05.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.729+0000 7f9faf13db80 1 -- 172.21.15.7:0/167603550 wait complete. 2024-06-10T01:21:05.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.730+0000 7f9faf13db80 1 -- start start 2024-06-10T01:21:05.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.730+0000 7f9faf13db80 1 Processor -- start 2024-06-10T01:21:05.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.730+0000 7f9faf13db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55617f77cf20 0x55617f788c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.730+0000 7f9faf13db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55617f782640 0x55617f785f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.730+0000 7f9faf13db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55617f782f60 0x55617f7864c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.730+0000 7f9faf13db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55617f79a9d0 con 0x55617f782f60 2024-06-10T01:21:05.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.730+0000 7f9faf13db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55617f66ebf0 con 0x55617f782640 2024-06-10T01:21:05.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.730+0000 7f9faf13db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55617f79ab50 con 0x55617f77cf20 2024-06-10T01:21:05.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.730+0000 7f9faf13db80 10 client.0 _ll_register_callbacks cb 0x55617f668920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:21:05.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.731+0000 7f9f9dd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55617f782640 0x55617f785f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:05.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.731+0000 7f9f9dd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55617f782640 0x55617f785f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:41752/0 (socket says 172.21.15.7:41752) 2024-06-10T01:21:05.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.731+0000 7f9f9dd9b700 1 -- 172.21.15.7:0/3528693676 learned_addr learned my addr 172.21.15.7:0/3528693676 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:05.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.731+0000 7f9f9ed9d700 1 --2- 172.21.15.7:0/3528693676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55617f77cf20 0x55617f788c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:05.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.731+0000 7f9f9ed9d700 1 --2- 172.21.15.7:0/3528693676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55617f782f60 0x55617f7864c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:05.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.732+0000 7f9f9dd9b700 1 -- 172.21.15.7:0/3528693676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55617f77cf20 msgr2=0x55617f788c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:05.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.732+0000 7f9f9dd9b700 1 --2- 172.21.15.7:0/3528693676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55617f77cf20 0x55617f788c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.732+0000 7f9f9dd9b700 1 -- 172.21.15.7:0/3528693676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55617f782f60 msgr2=0x55617f7864c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:05.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.732+0000 7f9f9dd9b700 1 --2- 172.21.15.7:0/3528693676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55617f782f60 0x55617f7864c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.732+0000 7f9f9dd9b700 1 -- 172.21.15.7:0/3528693676 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55617f61d8d0 con 0x55617f782640 2024-06-10T01:21:05.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.732+0000 7f9f9dd9b700 1 --2- 172.21.15.7:0/3528693676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55617f782640 0x55617f785f80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9f880105f0 tx=0x7f9f880109b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ed554e01419cbad server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:05.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.733+0000 7f9f85ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:21:05.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.734+0000 7f9f85ffb700 1 -- 172.21.15.7:0/3528693676 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f88004230 con 0x55617f782640 2024-06-10T01:21:05.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.734+0000 7f9f85ffb700 1 -- 172.21.15.7:0/3528693676 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f9f880043d0 con 0x55617f782640 2024-06-10T01:21:05.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.734+0000 7f9faf13db80 1 -- 172.21.15.7:0/3528693676 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f9800bca0 con 0x55617f782640 2024-06-10T01:21:05.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.734+0000 7f9f85ffb700 1 -- 172.21.15.7:0/3528693676 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f9f88023050 con 0x55617f782640 2024-06-10T01:21:05.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.734+0000 7f9faf13db80 10 client.28472 fetch_fsmap learned FSMap version 44 2024-06-10T01:21:05.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.734+0000 7f9faf13db80 1 -- 172.21.15.7:0/3528693676 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55617f7a5380 con 0x55617f782640 2024-06-10T01:21:05.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.735+0000 7f9f85ffb700 1 -- 172.21.15.7:0/3528693676 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f9800bca0 con 0x55617f782640 2024-06-10T01:21:05.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.735+0000 7f9f85ffb700 1 -- 172.21.15.7:0/3528693676 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f9f8800f040 con 0x55617f782640 2024-06-10T01:21:05.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.735+0000 7f9faf13db80 10 client.28472 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:21:05.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.735+0000 7f9faf13db80 10 client.28472 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:21:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.735+0000 7f9faf13db80 20 client.28472 populate_metadata read hostname 'smithi007' 2024-06-10T01:21:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.735+0000 7f9faf13db80 1 --2- 172.21.15.7:0/3528693676 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55617f7a7e20 0x55617f7aa240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.735+0000 7f9faf13db80 4 client.28472 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:21:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.735+0000 7f9faf13db80 1 -- 172.21.15.7:0/3528693676 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55617f6dbd10 con 0x55617f7a7e20 2024-06-10T01:21:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.736+0000 7f9f9ed9d700 1 --2- 172.21.15.7:0/3528693676 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55617f7a7e20 0x55617f7aa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.740+0000 7f9f85ffb700 10 client.28472 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:21:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.740+0000 7f9f9ed9d700 1 --2- 172.21.15.7:0/3528693676 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55617f7a7e20 0x55617f7aa240 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9c968b5c4cd70a67 server_cookie=555a321ed3b9b81c in_seq=0 out_seq=0 2024-06-10T01:21:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.744+0000 7f9f85ffb700 1 -- 172.21.15.7:0/3528693676 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55617f6dbd10 con 0x55617f7a7e20 2024-06-10T01:21:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.744+0000 7f9f85ffb700 10 client.28472 handle_command_reply: tid=0 2024-06-10T01:21:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.744+0000 7f9faf13db80 1 client.28472 shutdown 2024-06-10T01:21:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.744+0000 7f9faf13db80 1 -- 172.21.15.7:0/3528693676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55617f782640 msgr2=0x55617f785f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.744+0000 7f9faf13db80 1 --2- 172.21.15.7:0/3528693676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55617f782640 0x55617f785f80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9f880105f0 tx=0x7f9f880109b0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.744+0000 7f9faf13db80 1 -- 172.21.15.7:0/3528693676 shutdown_connections 2024-06-10T01:21:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.744+0000 7f9faf13db80 1 --2- 172.21.15.7:0/3528693676 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55617f7a7e20 0x55617f7aa240 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.744+0000 7f9faf13db80 1 --2- 172.21.15.7:0/3528693676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55617f782f60 0x55617f7864c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.744+0000 7f9faf13db80 1 --2- 172.21.15.7:0/3528693676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55617f77cf20 0x55617f788c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.744+0000 7f9faf13db80 1 --2- 172.21.15.7:0/3528693676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55617f782640 0x55617f785f80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.744+0000 7f9faf13db80 1 -- 172.21.15.7:0/3528693676 >> 172.21.15.7:0/3528693676 conn(0x55617f6a7cc0 msgr2=0x55617f77b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.744+0000 7f9f85ffb700 0 client.28472 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:21:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.744+0000 7f9faf13db80 1 -- 172.21.15.7:0/3528693676 shutdown_connections 2024-06-10T01:21:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.745+0000 7f9faf13db80 1 -- 172.21.15.7:0/3528693676 wait complete. 2024-06-10T01:21:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.745+0000 7f9faf13db80 20 client.28472 trim_cache size 0 max 16384 2024-06-10T01:21:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.770+0000 7f9faf13db80 1 Processor -- start 2024-06-10T01:21:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.770+0000 7f9faf13db80 1 -- start start 2024-06-10T01:21:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.770+0000 7f9faf13db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55617f77cf20 0x55617f77c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.771+0000 7f9faf13db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55617f782640 0x55617f7779f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.771+0000 7f9faf13db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55617f782f60 0x55617f7864f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.771+0000 7f9faf13db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa01ae980 con 0x55617f782640 2024-06-10T01:21:05.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.771+0000 7f9faf13db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa01ac180 con 0x55617f782f60 2024-06-10T01:21:05.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.771+0000 7f9faf13db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa0072750 con 0x55617f77cf20 2024-06-10T01:21:05.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.771+0000 7f9f9dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55617f782640 0x55617f7779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:05.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.771+0000 7f9f9dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55617f782640 0x55617f7779f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53160/0 (socket says 172.21.15.7:53160) 2024-06-10T01:21:05.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.771+0000 7f9f9dd9b700 1 -- 172.21.15.7:0/1534892949 learned_addr learned my addr 172.21.15.7:0/1534892949 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:05.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.771+0000 7f9f9dd9b700 1 -- 172.21.15.7:0/1534892949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55617f77cf20 msgr2=0x55617f77c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:05.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.771+0000 7f9f9dd9b700 1 --2- 172.21.15.7:0/1534892949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55617f77cf20 0x55617f77c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.771+0000 7f9f9dd9b700 1 -- 172.21.15.7:0/1534892949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55617f782f60 msgr2=0x55617f7864f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:05.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.771+0000 7f9f9dd9b700 1 --2- 172.21.15.7:0/1534892949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55617f782f60 0x55617f7864f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.771+0000 7f9f9dd9b700 1 -- 172.21.15.7:0/1534892949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55617f7844f0 con 0x55617f782640 2024-06-10T01:21:05.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.771+0000 7f9f9dd9b700 1 --2- 172.21.15.7:0/1534892949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55617f782640 0x55617f7779f0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f9f88012b10 tx=0x7f9f88005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd9a1563278593d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:05.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.772+0000 7f9f87fff700 1 -- 172.21.15.7:0/1534892949 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f88009670 con 0x55617f782640 2024-06-10T01:21:05.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.772+0000 7f9f87fff700 1 -- 172.21.15.7:0/1534892949 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f88012660 con 0x55617f782640 2024-06-10T01:21:05.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.772+0000 7f9f87fff700 1 -- 172.21.15.7:0/1534892949 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8800d590 con 0x55617f782640 2024-06-10T01:21:05.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.773+0000 7f9faf13db80 1 -- 172.21.15.7:0/1534892949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55617f782640 msgr2=0x55617f7779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:05.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.773+0000 7f9faf13db80 1 --2- 172.21.15.7:0/1534892949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55617f782640 0x55617f7779f0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f9f88012b10 tx=0x7f9f88005900 comp rx=0 tx=0).stop 2024-06-10T01:21:05.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.774+0000 7f9faf13db80 1 -- 172.21.15.7:0/1534892949 shutdown_connections 2024-06-10T01:21:05.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.774+0000 7f9faf13db80 1 --2- 172.21.15.7:0/1534892949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55617f782f60 0x55617f7864f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.774+0000 7f9faf13db80 1 --2- 172.21.15.7:0/1534892949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55617f77cf20 0x55617f77c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.774+0000 7f9faf13db80 1 --2- 172.21.15.7:0/1534892949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55617f782640 0x55617f7779f0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.774+0000 7f9faf13db80 1 -- 172.21.15.7:0/1534892949 >> 172.21.15.7:0/1534892949 conn(0x55617f6a7cc0 msgr2=0x55617f7833a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:05.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.774+0000 7f9faf13db80 1 -- 172.21.15.7:0/1534892949 shutdown_connections 2024-06-10T01:21:05.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.774+0000 7f9faf13db80 1 -- 172.21.15.7:0/1534892949 wait complete. 2024-06-10T01:21:05.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.775+0000 7f9faf13db80 1 -- start start 2024-06-10T01:21:05.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.775+0000 7f9faf13db80 1 Processor -- start 2024-06-10T01:21:05.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.776+0000 7f9faf13db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55617f77cf20 0x55617f79fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.776+0000 7f9faf13db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55617f782640 0x55617f7a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.776+0000 7f9faf13db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55617f782f60 0x55617f7aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.776+0000 7f9faf13db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa0072dc0 con 0x55617f77cf20 2024-06-10T01:21:05.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.776+0000 7f9faf13db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa0074230 con 0x55617f782f60 2024-06-10T01:21:05.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.776+0000 7f9faf13db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa0073490 con 0x55617f782640 2024-06-10T01:21:05.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.776+0000 7f9f9ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55617f77cf20 0x55617f79fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:05.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.776+0000 7f9faf13db80 10 client.0 _ll_register_callbacks cb 0x55617f668920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:21:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.776+0000 7f9f9ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55617f77cf20 0x55617f79fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53174/0 (socket says 172.21.15.7:53174) 2024-06-10T01:21:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.776+0000 7f9f9ed9d700 1 -- 172.21.15.7:0/3094868650 learned_addr learned my addr 172.21.15.7:0/3094868650 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.776+0000 7f9f9ed9d700 1 --2- 172.21.15.7:0/3094868650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55617f782f60 0x55617f7aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.776+0000 7f9f9ed9d700 1 -- 172.21.15.7:0/3094868650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55617f782640 msgr2=0x55617f7a03a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:21:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.776+0000 7f9f9ed9d700 1 --2- 172.21.15.7:0/3094868650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55617f782640 0x55617f7a03a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.776+0000 7f9f9ed9d700 1 -- 172.21.15.7:0/3094868650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55617f782f60 msgr2=0x55617f7aeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.776+0000 7f9f9ed9d700 1 --2- 172.21.15.7:0/3094868650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55617f782f60 0x55617f7aeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.777+0000 7f9f9ed9d700 1 -- 172.21.15.7:0/3094868650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55617f7a0aa0 con 0x55617f77cf20 2024-06-10T01:21:05.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.777+0000 7f9f9ed9d700 1 --2- 172.21.15.7:0/3094868650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55617f77cf20 0x55617f79fe60 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f9f98005150 tx=0x7f9f98005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27106e06b34a937 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:05.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.777+0000 7f9f85ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:21:05.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.777+0000 7f9f85ffb700 1 -- 172.21.15.7:0/3094868650 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f98007880 con 0x55617f77cf20 2024-06-10T01:21:05.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.777+0000 7f9f85ffb700 1 -- 172.21.15.7:0/3094868650 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f9f98007a20 con 0x55617f77cf20 2024-06-10T01:21:05.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.777+0000 7f9faf13db80 1 -- 172.21.15.7:0/3094868650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f88012660 con 0x55617f77cf20 2024-06-10T01:21:05.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.778+0000 7f9f85ffb700 1 -- 172.21.15.7:0/3094868650 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f9f9803b050 con 0x55617f77cf20 2024-06-10T01:21:05.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.778+0000 7f9faf13db80 10 client.19230 fetch_fsmap learned FSMap version 44 2024-06-10T01:21:05.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.778+0000 7f9faf13db80 1 -- 172.21.15.7:0/3094868650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55617f8120a0 con 0x55617f77cf20 2024-06-10T01:21:05.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.778+0000 7f9f85ffb700 1 -- 172.21.15.7:0/3094868650 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f9802b070 con 0x55617f77cf20 2024-06-10T01:21:05.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.778+0000 7f9f85ffb700 1 -- 172.21.15.7:0/3094868650 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f9f98011070 con 0x55617f77cf20 2024-06-10T01:21:05.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.778+0000 7f9faf13db80 10 client.19230 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:21:05.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.778+0000 7f9faf13db80 10 client.19230 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:21:05.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.778+0000 7f9faf13db80 20 client.19230 populate_metadata read hostname 'smithi007' 2024-06-10T01:21:05.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.779+0000 7f9faf13db80 1 --2- 172.21.15.7:0/3094868650 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55617f812330 0x55617f814720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.779+0000 7f9faf13db80 4 client.19230 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:21:05.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.779+0000 7f9faf13db80 1 -- 172.21.15.7:0/3094868650 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f9f88012660 con 0x55617f812330 2024-06-10T01:21:05.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.779+0000 7f9f9dd9b700 1 --2- 172.21.15.7:0/3094868650 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55617f812330 0x55617f814720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:05.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.779+0000 7f9f9dd9b700 1 --2- 172.21.15.7:0/3094868650 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55617f812330 0x55617f814720 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e139197bd28f495b server_cookie=2937c9c676619475 in_seq=0 out_seq=0 2024-06-10T01:21:05.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.779+0000 7f9f85ffb700 10 client.19230 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:21:05.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.780+0000 7f9f85ffb700 1 -- 172.21.15.7:0/3094868650 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f9f88012660 con 0x55617f812330 2024-06-10T01:21:05.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.780+0000 7f9f85ffb700 10 client.19230 handle_command_reply: tid=0 2024-06-10T01:21:05.218 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:21:05.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.780+0000 7f9faf13db80 1 client.19230 shutdown 2024-06-10T01:21:05.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.780+0000 7f9faf13db80 1 -- 172.21.15.7:0/3094868650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55617f77cf20 msgr2=0x55617f79fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:05.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.780+0000 7f9faf13db80 1 --2- 172.21.15.7:0/3094868650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55617f77cf20 0x55617f79fe60 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f9f98005150 tx=0x7f9f98005040 comp rx=0 tx=0).stop 2024-06-10T01:21:05.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.780+0000 7f9faf13db80 1 -- 172.21.15.7:0/3094868650 shutdown_connections 2024-06-10T01:21:05.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.780+0000 7f9faf13db80 1 --2- 172.21.15.7:0/3094868650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55617f782f60 0x55617f7aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.780+0000 7f9faf13db80 1 --2- 172.21.15.7:0/3094868650 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55617f812330 0x55617f814720 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.780+0000 7f9faf13db80 1 --2- 172.21.15.7:0/3094868650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55617f77cf20 0x55617f79fe60 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.780+0000 7f9f85ffb700 0 client.19230 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:21:05.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.780+0000 7f9faf13db80 1 --2- 172.21.15.7:0/3094868650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55617f782640 0x55617f7a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.780+0000 7f9faf13db80 1 -- 172.21.15.7:0/3094868650 >> 172.21.15.7:0/3094868650 conn(0x55617f6a7cc0 msgr2=0x55617f770e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:05.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.780+0000 7f9faf13db80 1 -- 172.21.15.7:0/3094868650 shutdown_connections 2024-06-10T01:21:05.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.780+0000 7f9faf13db80 1 -- 172.21.15.7:0/3094868650 wait complete. 2024-06-10T01:21:05.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.780+0000 7f9faf13db80 20 client.19230 trim_cache size 0 max 16384 2024-06-10T01:21:05.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.781+0000 7f9fa48ba700 1 -- 172.21.15.7:0/3743974484 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9f8c0a3090 msgr2=0x7f9f8c0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:05.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.781+0000 7f9fa48ba700 1 --2- 172.21.15.7:0/3743974484 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9f8c0a3090 0x7f9f8c0a5550 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9f9009b230 tx=0x7f9f90010d70 comp rx=0 tx=0).stop 2024-06-10T01:21:05.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.781+0000 7f9fa48ba700 1 -- 172.21.15.7:0/3743974484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fa0102740 msgr2=0x7f9fa010f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:05.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.781+0000 7f9fa48ba700 1 --2- 172.21.15.7:0/3743974484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fa0102740 0x7f9fa010f140 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9f9000e9a0 tx=0x7f9f9000ecb0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.781+0000 7f9fa48ba700 1 -- 172.21.15.7:0/3743974484 shutdown_connections 2024-06-10T01:21:05.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.781+0000 7f9fa48ba700 1 --2- 172.21.15.7:0/3743974484 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9f8c0a3090 0x7f9f8c0a5550 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.781+0000 7f9fa48ba700 1 --2- 172.21.15.7:0/3743974484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fa0107840 0x7f9fa010c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.781+0000 7f9fa48ba700 1 --2- 172.21.15.7:0/3743974484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fa0102740 0x7f9fa010f140 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.781+0000 7f9fa48ba700 1 --2- 172.21.15.7:0/3743974484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa01030f0 0x7f9fa010c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.781+0000 7f9fa48ba700 1 -- 172.21.15.7:0/3743974484 >> 172.21.15.7:0/3743974484 conn(0x7f9fa00795f0 msgr2=0x7f9fa0100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:05.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.781+0000 7f9fa48ba700 1 -- 172.21.15.7:0/3743974484 shutdown_connections 2024-06-10T01:21:05.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:04.781+0000 7f9fa48ba700 1 -- 172.21.15.7:0/3743974484 wait complete. 2024-06-10T01:21:05.485 INFO:tasks.workunit.client.0.smithi007.stdout: 10 27411 16.13 MB/sec warmup 102 sec latency 83.918 ms 2024-06-10T01:21:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.834+0000 7f0e0a7fc640 1 -- 172.21.15.7:0/204420936 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04030950 con 0x7f0e0c1035e0 2024-06-10T01:21:05.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:05 smithi078 ceph-mon[36683]: pgmap v884: 129 pgs: 129 active+clean; 389 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.4 MiB/s wr, 307 op/s 2024-06-10T01:21:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.835+0000 7f0e113ed640 1 -- 172.21.15.7:0/204420936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e0c1035e0 msgr2=0x7f0e0c109e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.835+0000 7f0e113ed640 1 --2- 172.21.15.7:0/204420936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e0c1035e0 0x7f0e0c109e70 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f0e04009860 tx=0x7f0e0402a340 comp rx=0 tx=0).stop 2024-06-10T01:21:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.835+0000 7f0e113ed640 1 -- 172.21.15.7:0/204420936 shutdown_connections 2024-06-10T01:21:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.835+0000 7f0e113ed640 1 --2- 172.21.15.7:0/204420936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e0c1035e0 0x7f0e0c109e70 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.835+0000 7f0e113ed640 1 --2- 172.21.15.7:0/204420936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e0c102c20 0x7f0e0c1030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.835+0000 7f0e113ed640 1 --2- 172.21.15.7:0/204420936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e0c101a30 0x7f0e0c101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.835+0000 7f0e113ed640 1 -- 172.21.15.7:0/204420936 >> 172.21.15.7:0/204420936 conn(0x7f0e0c0fcfc0 msgr2=0x7f0e0c0ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.835+0000 7f0e113ed640 1 -- 172.21.15.7:0/204420936 shutdown_connections 2024-06-10T01:21:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.835+0000 7f0e113ed640 1 -- 172.21.15.7:0/204420936 wait complete. 2024-06-10T01:21:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.836+0000 7f0e113ed640 1 Processor -- start 2024-06-10T01:21:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.836+0000 7f0e113ed640 1 -- start start 2024-06-10T01:21:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.837+0000 7f0e113ed640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e0c101a30 0x7f0e0c199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.837+0000 7f0e113ed640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e0c102c20 0x7f0e0c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.837+0000 7f0e113ed640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e0c1035e0 0x7f0e0c1a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.837+0000 7f0e113ed640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c0757f0 con 0x7f0e0c101a30 2024-06-10T01:21:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.837+0000 7f0e113ed640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c075670 con 0x7f0e0c1035e0 2024-06-10T01:21:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.837+0000 7f0e113ed640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c075970 con 0x7f0e0c102c20 2024-06-10T01:21:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.837+0000 7f0e027fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e0c102c20 0x7f0e0c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.837+0000 7f0e0b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e0c1035e0 0x7f0e0c1a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.837+0000 7f0e027fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e0c102c20 0x7f0e0c19a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49956/0 (socket says 172.21.15.7:49956) 2024-06-10T01:21:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.837+0000 7f0e0affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e0c101a30 0x7f0e0c199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.837+0000 7f0e027fc640 1 -- 172.21.15.7:0/975140008 learned_addr learned my addr 172.21.15.7:0/975140008 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.837+0000 7f0e0b7fe640 1 --2- 172.21.15.7:0/975140008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e0c1035e0 0x7f0e0c1a1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:41784/0 (socket says 172.21.15.7:41784) 2024-06-10T01:21:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.838+0000 7f0e0b7fe640 1 -- 172.21.15.7:0/975140008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e0c102c20 msgr2=0x7f0e0c19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.838+0000 7f0e0b7fe640 1 --2- 172.21.15.7:0/975140008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e0c102c20 0x7f0e0c19a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.838+0000 7f0e0b7fe640 1 -- 172.21.15.7:0/975140008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e0c101a30 msgr2=0x7f0e0c199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.838+0000 7f0e0b7fe640 1 --2- 172.21.15.7:0/975140008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e0c101a30 0x7f0e0c199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.838+0000 7f0e0b7fe640 1 -- 172.21.15.7:0/975140008 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e0c1a1b90 con 0x7f0e0c1035e0 2024-06-10T01:21:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.838+0000 7f0e0b7fe640 1 --2- 172.21.15.7:0/975140008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e0c1035e0 0x7f0e0c1a1520 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0e04009780 tx=0x7f0e04033d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.838+0000 7f0e08ff9640 1 -- 172.21.15.7:0/975140008 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04037040 con 0x7f0e0c1035e0 2024-06-10T01:21:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.838+0000 7f0e08ff9640 1 -- 172.21.15.7:0/975140008 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e04030850 con 0x7f0e0c1035e0 2024-06-10T01:21:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.838+0000 7f0e08ff9640 1 -- 172.21.15.7:0/975140008 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e040046c0 con 0x7f0e0c1035e0 2024-06-10T01:21:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.838+0000 7f0e113ed640 1 -- 172.21.15.7:0/975140008 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e0c10bb40 con 0x7f0e0c1035e0 2024-06-10T01:21:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.838+0000 7f0e113ed640 1 -- 172.21.15.7:0/975140008 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e0c10c060 con 0x7f0e0c1035e0 2024-06-10T01:21:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.842+0000 7f0e141cf740 1 Processor -- start 2024-06-10T01:21:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.842+0000 7f0e08ff9640 1 -- 172.21.15.7:0/975140008 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0e04004050 con 0x7f0e0c1035e0 2024-06-10T01:21:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.843+0000 7f0e141cf740 1 -- start start 2024-06-10T01:21:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.843+0000 7f0e141cf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f99a0b8010 0x55f99a0b8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.843+0000 7f0e141cf740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f99a0b8a00 0x55f99a0bd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.843+0000 7f0e141cf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f99a0bdb70 0x55f99a0c0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.843+0000 7f0e141cf740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f99a01b4f0 con 0x55f99a0b8a00 2024-06-10T01:21:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.843+0000 7f0e141cf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f999fb9870 con 0x55f99a0bdb70 2024-06-10T01:21:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.843+0000 7f0e141cf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f99a0174b0 con 0x55f99a0b8010 2024-06-10T01:21:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.844+0000 7f0e027fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f99a0bdb70 0x55f99a0c0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.844+0000 7f0e027fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f99a0b8010 0x55f99a0b8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.844+0000 7f0e027fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f99a0b8010 0x55f99a0b8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49972/0 (socket says 172.21.15.7:49972) 2024-06-10T01:21:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.844+0000 7f0e027fc640 1 -- 172.21.15.7:0/3961613762 learned_addr learned my addr 172.21.15.7:0/3961613762 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.844+0000 7f0e08ff9640 1 --2- 172.21.15.7:0/975140008 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0dec0a2cc0 0x7f0dec0a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.844+0000 7f0e027fc640 1 -- 172.21.15.7:0/3961613762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f99a0b8010 msgr2=0x55f99a0b8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.844+0000 7f0e027fc640 1 --2- 172.21.15.7:0/3961613762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f99a0b8010 0x55f99a0b8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.844+0000 7f0e027fc640 1 -- 172.21.15.7:0/3961613762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f99a0b8a00 msgr2=0x55f99a0bd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.844+0000 7f0e0affd640 1 --2- 172.21.15.7:0/3961613762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f99a0b8a00 0x55f99a0bd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.845+0000 7f0e08ff9640 1 -- 172.21.15.7:0/975140008 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f0e040eeea0 con 0x7f0e0c1035e0 2024-06-10T01:21:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.845+0000 7f0e0b7fe640 1 --2- 172.21.15.7:0/975140008 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0dec0a2cc0 0x7f0dec0a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.845+0000 7f0e027fc640 1 --2- 172.21.15.7:0/3961613762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f99a0b8a00 0x55f99a0bd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.845+0000 7f0e027fc640 1 -- 172.21.15.7:0/3961613762 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f999fd59b0 con 0x55f99a0bdb70 2024-06-10T01:21:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.845+0000 7f0e0affd640 1 --2- 172.21.15.7:0/3961613762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f99a0b8a00 0x55f99a0bd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:21:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.845+0000 7f0e0b7fe640 1 --2- 172.21.15.7:0/975140008 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0dec0a2cc0 0x7f0dec0a5180 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0e040b7d80 tx=0x7f0e04005280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.845+0000 7f0e027fc640 1 --2- 172.21.15.7:0/3961613762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f99a0bdb70 0x55f99a0c0000 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0df4009fd0 tx=0x7f0df4007040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c10aef8ee5c28131 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.846+0000 7f0e017fa640 1 -- 172.21.15.7:0/3961613762 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df401f070 con 0x55f99a0bdb70 2024-06-10T01:21:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.846+0000 7f0e017fa640 1 -- 172.21.15.7:0/3961613762 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0df401a460 con 0x55f99a0bdb70 2024-06-10T01:21:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.846+0000 7f0e017fa640 1 -- 172.21.15.7:0/3961613762 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df40185c0 con 0x55f99a0bdb70 2024-06-10T01:21:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.847+0000 7f0e141cf740 1 -- 172.21.15.7:0/3961613762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f99a0bdb70 msgr2=0x55f99a0c0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.847+0000 7f0e141cf740 1 --2- 172.21.15.7:0/3961613762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f99a0bdb70 0x55f99a0c0000 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0df4009fd0 tx=0x7f0df4007040 comp rx=0 tx=0).stop 2024-06-10T01:21:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.848+0000 7f0e141cf740 1 -- 172.21.15.7:0/3961613762 shutdown_connections 2024-06-10T01:21:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.848+0000 7f0e141cf740 1 --2- 172.21.15.7:0/3961613762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f99a0bdb70 0x55f99a0c0000 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.848+0000 7f0e141cf740 1 --2- 172.21.15.7:0/3961613762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f99a0b8a00 0x55f99a0bd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.848+0000 7f0e141cf740 1 --2- 172.21.15.7:0/3961613762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f99a0b8010 0x55f99a0b8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.848+0000 7f0e141cf740 1 -- 172.21.15.7:0/3961613762 >> 172.21.15.7:0/3961613762 conn(0x55f99a0b3680 msgr2=0x55f99a0b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.848+0000 7f0e141cf740 1 -- 172.21.15.7:0/3961613762 shutdown_connections 2024-06-10T01:21:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.848+0000 7f0e141cf740 1 -- 172.21.15.7:0/3961613762 wait complete. 2024-06-10T01:21:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.848+0000 7f0e141cf740 1 -- start start 2024-06-10T01:21:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.848+0000 7f0e141cf740 1 Processor -- start 2024-06-10T01:21:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.849+0000 7f0e141cf740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f99a0b8010 0x55f99a0d3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.849+0000 7f0e141cf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f99a0b8a00 0x55f99a0d37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.849+0000 7f0e141cf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f99a0bdb70 0x55f99a0da7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.849+0000 7f0e027fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f99a0b8010 0x55f99a0d3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.849+0000 7f0e027fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f99a0b8010 0x55f99a0d3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53220/0 (socket says 172.21.15.7:53220) 2024-06-10T01:21:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.849+0000 7f0e027fc640 1 -- 172.21.15.7:0/3729382688 learned_addr learned my addr 172.21.15.7:0/3729382688 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.849+0000 7f0e141cf740 1 -- 172.21.15.7:0/3729382688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f99a0d1b30 con 0x55f99a0b8010 2024-06-10T01:21:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.849+0000 7f0e141cf740 1 -- 172.21.15.7:0/3729382688 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f999fa4ae0 con 0x55f99a0b8a00 2024-06-10T01:21:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.849+0000 7f0e0affd640 1 --2- 172.21.15.7:0/3729382688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f99a0b8a00 0x55f99a0d37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.849+0000 7f0e141cf740 1 -- 172.21.15.7:0/3729382688 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f99a0d1cb0 con 0x55f99a0bdb70 2024-06-10T01:21:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.850+0000 7f0e141cf740 10 client.0 _ll_register_callbacks cb 0x55f999fdd9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:21:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.850+0000 7f0e0affd640 1 -- 172.21.15.7:0/3729382688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f99a0bdb70 msgr2=0x55f99a0da7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.850+0000 7f0e0affd640 1 --2- 172.21.15.7:0/3729382688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f99a0bdb70 0x55f99a0da7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.850+0000 7f0e0affd640 1 -- 172.21.15.7:0/3729382688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f99a0b8010 msgr2=0x55f99a0d3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.850+0000 7f0e0affd640 1 --2- 172.21.15.7:0/3729382688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f99a0b8010 0x55f99a0d3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.850+0000 7f0e0affd640 1 -- 172.21.15.7:0/3729382688 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f999d2f210 con 0x55f99a0b8a00 2024-06-10T01:21:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.850+0000 7f0e027fc640 1 --2- 172.21.15.7:0/3729382688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f99a0b8010 0x55f99a0d3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:21:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.850+0000 7f0e0affd640 1 --2- 172.21.15.7:0/3729382688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f99a0b8a00 0x55f99a0d37a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x55f99a0b8e00 tx=0x7f0dfc00b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7ee4da41ca02743 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.850+0000 7f0dfa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:21:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.851+0000 7f0dfa7fc640 1 -- 172.21.15.7:0/3729382688 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc004230 con 0x55f99a0b8a00 2024-06-10T01:21:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.851+0000 7f0dfa7fc640 1 -- 172.21.15.7:0/3729382688 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f0dfc0043d0 con 0x55f99a0b8a00 2024-06-10T01:21:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.851+0000 7f0e141cf740 1 -- 172.21.15.7:0/3729382688 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0df401a460 con 0x55f99a0b8a00 2024-06-10T01:21:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.851+0000 7f0dfa7fc640 1 -- 172.21.15.7:0/3729382688 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f0dfc02c050 con 0x55f99a0b8a00 2024-06-10T01:21:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.851+0000 7f0e141cf740 10 client.28487 fetch_fsmap learned FSMap version 44 2024-06-10T01:21:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.851+0000 7f0e141cf740 1 -- 172.21.15.7:0/3729382688 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55f999dfb620 con 0x55f99a0b8a00 2024-06-10T01:21:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.852+0000 7f0dfa7fc640 1 -- 172.21.15.7:0/3729382688 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0df401a460 con 0x55f99a0b8a00 2024-06-10T01:21:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.852+0000 7f0dfa7fc640 1 -- 172.21.15.7:0/3729382688 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f0dfc01e630 con 0x55f99a0b8a00 2024-06-10T01:21:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.852+0000 7f0e141cf740 10 client.28487 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:21:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.852+0000 7f0e141cf740 10 client.28487 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:21:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.852+0000 7f0e141cf740 20 client.28487 populate_metadata read hostname 'smithi007' 2024-06-10T01:21:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.852+0000 7f0e141cf740 1 --2- 172.21.15.7:0/3729382688 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f99a0df550 0x55f99a0e1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.852+0000 7f0e141cf740 4 client.28487 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:21:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.852+0000 7f0e141cf740 1 -- 172.21.15.7:0/3729382688 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f99a02ef10 con 0x55f99a0df550 2024-06-10T01:21:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.852+0000 7f0e027fc640 1 --2- 172.21.15.7:0/3729382688 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f99a0df550 0x55f99a0e1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.853+0000 7f0dfa7fc640 10 client.28487 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.853+0000 7f0e027fc640 1 --2- 172.21.15.7:0/3729382688 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f99a0df550 0x55f99a0e1a10 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a48811b9306b0a6 server_cookie=7465774d9162e48a in_seq=0 out_seq=0 2024-06-10T01:21:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.855+0000 7f0dfa7fc640 1 -- 172.21.15.7:0/3729382688 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f99a02ef10 con 0x55f99a0df550 2024-06-10T01:21:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.855+0000 7f0dfa7fc640 10 client.28487 handle_command_reply: tid=0 2024-06-10T01:21:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.855+0000 7f0e141cf740 1 client.28487 shutdown 2024-06-10T01:21:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.855+0000 7f0e141cf740 1 -- 172.21.15.7:0/3729382688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f99a0b8a00 msgr2=0x55f99a0d37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.855+0000 7f0e141cf740 1 --2- 172.21.15.7:0/3729382688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f99a0b8a00 0x55f99a0d37a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x55f99a0b8e00 tx=0x7f0dfc00b690 comp rx=0 tx=0).stop 2024-06-10T01:21:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.855+0000 7f0e141cf740 1 -- 172.21.15.7:0/3729382688 shutdown_connections 2024-06-10T01:21:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.855+0000 7f0e141cf740 1 --2- 172.21.15.7:0/3729382688 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f99a0df550 0x55f99a0e1a10 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.855+0000 7f0e141cf740 1 --2- 172.21.15.7:0/3729382688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f99a0bdb70 0x55f99a0da7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.855+0000 7f0e141cf740 1 --2- 172.21.15.7:0/3729382688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f99a0b8a00 0x55f99a0d37a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.855+0000 7f0e141cf740 1 --2- 172.21.15.7:0/3729382688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f99a0b8010 0x55f99a0d3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.855+0000 7f0e141cf740 1 -- 172.21.15.7:0/3729382688 >> 172.21.15.7:0/3729382688 conn(0x55f99a0b3680 msgr2=0x55f99a0b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.855+0000 7f0dfa7fc640 0 client.28487 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.856+0000 7f0e141cf740 1 -- 172.21.15.7:0/3729382688 shutdown_connections 2024-06-10T01:21:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.856+0000 7f0e141cf740 1 -- 172.21.15.7:0/3729382688 wait complete. 2024-06-10T01:21:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.856+0000 7f0e141cf740 20 client.28487 trim_cache size 0 max 16384 2024-06-10T01:21:05.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.882+0000 7f0e141cf740 1 Processor -- start 2024-06-10T01:21:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.882+0000 7f0e141cf740 1 -- start start 2024-06-10T01:21:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.883+0000 7f0e141cf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f99a0b8010 0x55f99a0bd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.883+0000 7f0e141cf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f99a0b8a00 0x55f99a0da7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.883+0000 7f0e141cf740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f99a0bdb70 0x55f99a0d3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.883+0000 7f0e027fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f99a0b8010 0x55f99a0bd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.883+0000 7f0e027fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f99a0b8010 0x55f99a0bd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:41812/0 (socket says 172.21.15.7:41812) 2024-06-10T01:21:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.883+0000 7f0e027fc640 1 -- 172.21.15.7:0/3146351140 learned_addr learned my addr 172.21.15.7:0/3146351140 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.883+0000 7f0e0affd640 1 --2- 172.21.15.7:0/3146351140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f99a0b8a00 0x55f99a0da7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.883+0000 7f0e141cf740 1 -- 172.21.15.7:0/3146351140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c1a25d0 con 0x55f99a0bdb70 2024-06-10T01:21:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.883+0000 7f0e141cf740 1 -- 172.21.15.7:0/3146351140 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c071c70 con 0x55f99a0b8010 2024-06-10T01:21:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.883+0000 7f0e141cf740 1 -- 172.21.15.7:0/3146351140 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c06fac0 con 0x55f99a0b8a00 2024-06-10T01:21:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.884+0000 7f0e027fc640 1 -- 172.21.15.7:0/3146351140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f99a0b8a00 msgr2=0x55f99a0da7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.884+0000 7f0e027fc640 1 --2- 172.21.15.7:0/3146351140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f99a0b8a00 0x55f99a0da7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.884+0000 7f0e027fc640 1 -- 172.21.15.7:0/3146351140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f99a0bdb70 msgr2=0x55f99a0d3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.884+0000 7f0e027fc640 1 --2- 172.21.15.7:0/3146351140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f99a0bdb70 0x55f99a0d3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.884+0000 7f0e027fc640 1 -- 172.21.15.7:0/3146351140 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f99a0b34b0 con 0x55f99a0b8010 2024-06-10T01:21:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.884+0000 7f0e027fc640 1 --2- 172.21.15.7:0/3146351140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f99a0b8010 0x55f99a0bd250 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0df4009010 tx=0x7f0df4009990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d813308447790e1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.885+0000 7f0e00ff9640 1 -- 172.21.15.7:0/3146351140 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df401f040 con 0x55f99a0b8010 2024-06-10T01:21:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.885+0000 7f0e00ff9640 1 -- 172.21.15.7:0/3146351140 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0df4027940 con 0x55f99a0b8010 2024-06-10T01:21:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.885+0000 7f0e00ff9640 1 -- 172.21.15.7:0/3146351140 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df401a7e0 con 0x55f99a0b8010 2024-06-10T01:21:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.886+0000 7f0e141cf740 1 -- 172.21.15.7:0/3146351140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f99a0b8010 msgr2=0x55f99a0bd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.886+0000 7f0e141cf740 1 --2- 172.21.15.7:0/3146351140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f99a0b8010 0x55f99a0bd250 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0df4009010 tx=0x7f0df4009990 comp rx=0 tx=0).stop 2024-06-10T01:21:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.886+0000 7f0e141cf740 1 -- 172.21.15.7:0/3146351140 shutdown_connections 2024-06-10T01:21:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.886+0000 7f0e141cf740 1 --2- 172.21.15.7:0/3146351140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f99a0bdb70 0x55f99a0d3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.886+0000 7f0e141cf740 1 --2- 172.21.15.7:0/3146351140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f99a0b8a00 0x55f99a0da7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.886+0000 7f0e141cf740 1 --2- 172.21.15.7:0/3146351140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f99a0b8010 0x55f99a0bd250 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.886+0000 7f0e141cf740 1 -- 172.21.15.7:0/3146351140 >> 172.21.15.7:0/3146351140 conn(0x55f99a0b3680 msgr2=0x55f99a0b3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.887+0000 7f0e141cf740 1 -- 172.21.15.7:0/3146351140 shutdown_connections 2024-06-10T01:21:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.887+0000 7f0e141cf740 1 -- 172.21.15.7:0/3146351140 wait complete. 2024-06-10T01:21:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.887+0000 7f0e141cf740 1 -- start start 2024-06-10T01:21:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.887+0000 7f0e141cf740 1 Processor -- start 2024-06-10T01:21:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.888+0000 7f0e141cf740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f99a0b8010 0x55f99a0d63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.888+0000 7f0e141cf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f99a0b8a00 0x55f99a0d6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.888+0000 7f0e141cf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f99a0bdb70 0x55f99a0d9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.888+0000 7f0e027fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f99a0b8010 0x55f99a0d63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:05.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.888+0000 7f0e0affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f99a0b8a00 0x55f99a0d6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:05.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.888+0000 7f0e027fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f99a0b8010 0x55f99a0d63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53228/0 (socket says 172.21.15.7:53228) 2024-06-10T01:21:05.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.888+0000 7f0e141cf740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c070130 con 0x55f99a0b8010 2024-06-10T01:21:05.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.888+0000 7f0e141cf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c0715a0 con 0x55f99a0b8a00 2024-06-10T01:21:05.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.888+0000 7f0e0affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f99a0b8a00 0x55f99a0d6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:41814/0 (socket says 172.21.15.7:41814) 2024-06-10T01:21:05.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.888+0000 7f0e0affd640 1 -- 172.21.15.7:0/788554829 learned_addr learned my addr 172.21.15.7:0/788554829 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:05.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.888+0000 7f0e141cf740 1 -- 172.21.15.7:0/788554829 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c070800 con 0x55f99a0bdb70 2024-06-10T01:21:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.888+0000 7f0e141cf740 10 client.0 _ll_register_callbacks cb 0x7f0dfc009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:21:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.888+0000 7f0e027fc640 1 --2- 172.21.15.7:0/788554829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f99a0bdb70 0x55f99a0d9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.889+0000 7f0e0affd640 1 -- 172.21.15.7:0/788554829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f99a0bdb70 msgr2=0x55f99a0d9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.889+0000 7f0e0affd640 1 --2- 172.21.15.7:0/788554829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f99a0bdb70 0x55f99a0d9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.889+0000 7f0e0affd640 1 -- 172.21.15.7:0/788554829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f99a0b8010 msgr2=0x55f99a0d63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.889+0000 7f0e0affd640 1 --2- 172.21.15.7:0/788554829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f99a0b8010 0x55f99a0d63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.889+0000 7f0e0affd640 1 -- 172.21.15.7:0/788554829 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f99a0d9f30 con 0x55f99a0b8a00 2024-06-10T01:21:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.889+0000 7f0e0affd640 1 --2- 172.21.15.7:0/788554829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f99a0b8a00 0x55f99a0d6920 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc000c00 tx=0x7f0dfc01e6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=709b97682b788a0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.889+0000 7f0dfa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:21:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.889+0000 7f0dfa7fc640 1 -- 172.21.15.7:0/788554829 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc005450 con 0x55f99a0b8a00 2024-06-10T01:21:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.889+0000 7f0dfa7fc640 1 -- 172.21.15.7:0/788554829 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f0dfc0055f0 con 0x55f99a0b8a00 2024-06-10T01:21:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.889+0000 7f0e141cf740 1 -- 172.21.15.7:0/788554829 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0df4027940 con 0x55f99a0b8a00 2024-06-10T01:21:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.890+0000 7f0dfa7fc640 1 -- 172.21.15.7:0/788554829 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f0dfc042050 con 0x55f99a0b8a00 2024-06-10T01:21:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.890+0000 7f0e141cf740 10 client.28493 fetch_fsmap learned FSMap version 44 2024-06-10T01:21:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.890+0000 7f0e141cf740 1 -- 172.21.15.7:0/788554829 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55f99a148820 con 0x55f99a0b8a00 2024-06-10T01:21:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.890+0000 7f0dfa7fc640 1 -- 172.21.15.7:0/788554829 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0df4027940 con 0x55f99a0b8a00 2024-06-10T01:21:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.891+0000 7f0dfa7fc640 1 -- 172.21.15.7:0/788554829 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f0dfc01ece0 con 0x55f99a0b8a00 2024-06-10T01:21:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.891+0000 7f0e141cf740 10 client.28493 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:21:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.891+0000 7f0e141cf740 10 client.28493 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:21:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.891+0000 7f0e141cf740 20 client.28493 populate_metadata read hostname 'smithi007' 2024-06-10T01:21:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.891+0000 7f0e141cf740 1 --2- 172.21.15.7:0/788554829 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f99a149200 0x55f99a14b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.891+0000 7f0e141cf740 4 client.28493 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:21:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.891+0000 7f0e141cf740 1 -- 172.21.15.7:0/788554829 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f99a028c20 con 0x55f99a149200 2024-06-10T01:21:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.891+0000 7f0e027fc640 1 --2- 172.21.15.7:0/788554829 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f99a149200 0x55f99a14b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.892+0000 7f0e027fc640 1 --2- 172.21.15.7:0/788554829 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f99a149200 0x55f99a14b620 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ab742e520363ae7 server_cookie=b84f02853523d556 in_seq=0 out_seq=0 2024-06-10T01:21:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:05.892+0000 7f0dfa7fc640 10 client.28493 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:05.890 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:05 smithi007 ceph-mon[32398]: pgmap v884: 129 pgs: 129 active+clean; 389 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.4 MiB/s wr, 307 op/s 2024-06-10T01:21:05.931 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:21:05.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:05 smithi175 ceph-mon[36636]: pgmap v884: 129 pgs: 129 active+clean; 389 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.4 MiB/s wr, 307 op/s 2024-06-10T01:21:06.298 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:21:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:06.307+0000 7f0dfa7fc640 1 -- 172.21.15.7:0/788554829 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f99a028c20 con 0x55f99a149200 2024-06-10T01:21:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:06.307+0000 7f0dfa7fc640 10 client.28493 handle_command_reply: tid=0 2024-06-10T01:21:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:06.307+0000 7f0e141cf740 1 client.28493 shutdown 2024-06-10T01:21:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:06.308+0000 7f0e141cf740 1 -- 172.21.15.7:0/788554829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f99a0b8a00 msgr2=0x55f99a0d6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:06.308+0000 7f0e141cf740 1 --2- 172.21.15.7:0/788554829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f99a0b8a00 0x55f99a0d6920 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc000c00 tx=0x7f0dfc01e6b0 comp rx=0 tx=0).stop 2024-06-10T01:21:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:06.308+0000 7f0e141cf740 1 -- 172.21.15.7:0/788554829 shutdown_connections 2024-06-10T01:21:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:06.308+0000 7f0e141cf740 1 --2- 172.21.15.7:0/788554829 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f99a149200 0x55f99a14b620 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:06.308+0000 7f0e141cf740 1 --2- 172.21.15.7:0/788554829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f99a0bdb70 0x55f99a0d9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:06.308+0000 7f0e141cf740 1 --2- 172.21.15.7:0/788554829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f99a0b8a00 0x55f99a0d6920 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:06.308+0000 7f0e141cf740 1 --2- 172.21.15.7:0/788554829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f99a0b8010 0x55f99a0d63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:06.308+0000 7f0e141cf740 1 -- 172.21.15.7:0/788554829 >> 172.21.15.7:0/788554829 conn(0x55f99a0b3680 msgr2=0x55f99a0bd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:06.308+0000 7f0dfa7fc640 0 client.28493 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:06.308+0000 7f0e141cf740 1 -- 172.21.15.7:0/788554829 shutdown_connections 2024-06-10T01:21:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:06.308+0000 7f0e141cf740 1 -- 172.21.15.7:0/788554829 wait complete. 2024-06-10T01:21:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:06.308+0000 7f0e141cf740 20 client.28493 trim_cache size 0 max 16384 2024-06-10T01:21:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:21:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:21:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:21:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:06.308+0000 7f0e113ed640 1 -- 172.21.15.7:0/975140008 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0dec0a2cc0 msgr2=0x7f0dec0a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:06.308+0000 7f0e113ed640 1 --2- 172.21.15.7:0/975140008 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0dec0a2cc0 0x7f0dec0a5180 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0e040b7d80 tx=0x7f0e04005280 comp rx=0 tx=0).stop 2024-06-10T01:21:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:06.308+0000 7f0e113ed640 1 -- 172.21.15.7:0/975140008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e0c1035e0 msgr2=0x7f0e0c1a1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:06.308+0000 7f0e113ed640 1 --2- 172.21.15.7:0/975140008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e0c1035e0 0x7f0e0c1a1520 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0e04009780 tx=0x7f0e04033d10 comp rx=0 tx=0).stop 2024-06-10T01:21:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:06.309+0000 7f0e113ed640 1 -- 172.21.15.7:0/975140008 shutdown_connections 2024-06-10T01:21:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:06.309+0000 7f0e113ed640 1 --2- 172.21.15.7:0/975140008 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0dec0a2cc0 0x7f0dec0a5180 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:06.309+0000 7f0e113ed640 1 --2- 172.21.15.7:0/975140008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e0c1035e0 0x7f0e0c1a1520 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:06.309+0000 7f0e113ed640 1 --2- 172.21.15.7:0/975140008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e0c102c20 0x7f0e0c19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:06.309+0000 7f0e113ed640 1 --2- 172.21.15.7:0/975140008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e0c101a30 0x7f0e0c199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:06.309+0000 7f0e113ed640 1 -- 172.21.15.7:0/975140008 >> 172.21.15.7:0/975140008 conn(0x7f0e0c0fcfc0 msgr2=0x7f0e0c105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:06.309+0000 7f0e113ed640 1 -- 172.21.15.7:0/975140008 shutdown_connections 2024-06-10T01:21:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:06.309+0000 7f0e113ed640 1 -- 172.21.15.7:0/975140008 wait complete. 2024-06-10T01:21:06.485 INFO:tasks.workunit.client.0.smithi007.stdout: 10 27930 16.25 MB/sec warmup 103 sec latency 241.680 ms 2024-06-10T01:21:07.485 INFO:tasks.workunit.client.0.smithi007.stdout: 10 28951 16.63 MB/sec warmup 104 sec latency 48.187 ms 2024-06-10T01:21:07.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:07 smithi007 ceph-mon[32398]: pgmap v885: 129 pgs: 129 active+clean; 395 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.1 MiB/s wr, 379 op/s 2024-06-10T01:21:07.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:07 smithi078 ceph-mon[36683]: pgmap v885: 129 pgs: 129 active+clean; 395 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.1 MiB/s wr, 379 op/s 2024-06-10T01:21:07.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:07 smithi175 ceph-mon[36636]: pgmap v885: 129 pgs: 129 active+clean; 395 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.1 MiB/s wr, 379 op/s 2024-06-10T01:21:08.502 INFO:tasks.workunit.client.0.smithi007.stdout: 10 29722 17.01 MB/sec warmup 105 sec latency 83.606 ms 2024-06-10T01:21:08.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.717+0000 7fdd4932d700 1 -- 172.21.15.7:0/779256007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd44101d50 msgr2=0x7fdd4410e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:08.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.717+0000 7fdd4932d700 1 --2- 172.21.15.7:0/779256007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd44101d50 0x7fdd4410e240 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c00b0a0 tx=0x7fdd3c00b3b0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.717+0000 7fdd4932d700 1 -- 172.21.15.7:0/779256007 shutdown_connections 2024-06-10T01:21:08.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.717+0000 7fdd4932d700 1 --2- 172.21.15.7:0/779256007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd44101d50 0x7fdd4410e240 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.717+0000 7fdd4932d700 1 --2- 172.21.15.7:0/779256007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd44107400 0x7fdd441077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.717+0000 7fdd4932d700 1 --2- 172.21.15.7:0/779256007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd44101390 0x7fdd44101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.717+0000 7fdd4932d700 1 -- 172.21.15.7:0/779256007 >> 172.21.15.7:0/779256007 conn(0x7fdd44078340 msgr2=0x7fdd44078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:08.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.717+0000 7fdd4932d700 1 -- 172.21.15.7:0/779256007 shutdown_connections 2024-06-10T01:21:08.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.717+0000 7fdd4932d700 1 -- 172.21.15.7:0/779256007 wait complete. 2024-06-10T01:21:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.718+0000 7fdd4932d700 1 Processor -- start 2024-06-10T01:21:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.718+0000 7fdd4932d700 1 -- start start 2024-06-10T01:21:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.718+0000 7fdd4932d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd44101390 0x7fdd4419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.718+0000 7fdd4932d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd44101d50 0x7fdd4419dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.718+0000 7fdd4932d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd44107400 0x7fdd441978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.718+0000 7fdd4932d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdd44111690 con 0x7fdd44101d50 2024-06-10T01:21:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.718+0000 7fdd4932d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdd44111510 con 0x7fdd44107400 2024-06-10T01:21:08.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.718+0000 7fdd4932d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdd44111810 con 0x7fdd44101390 2024-06-10T01:21:08.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.719+0000 7fdd427fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd44101d50 0x7fdd4419dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:08.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.719+0000 7fdd42ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd44101390 0x7fdd4419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:08.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.719+0000 7fdd427fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd44101d50 0x7fdd4419dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53240/0 (socket says 172.21.15.7:53240) 2024-06-10T01:21:08.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.719+0000 7fdd42ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd44101390 0x7fdd4419d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:50012/0 (socket says 172.21.15.7:50012) 2024-06-10T01:21:08.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.719+0000 7fdd427fc700 1 -- 172.21.15.7:0/3191256796 learned_addr learned my addr 172.21.15.7:0/3191256796 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:08.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.719+0000 7fdd437fe700 1 --2- 172.21.15.7:0/3191256796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd44107400 0x7fdd441978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:08.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.719+0000 7fdd437fe700 1 -- 172.21.15.7:0/3191256796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd44101390 msgr2=0x7fdd4419d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:08.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.719+0000 7fdd437fe700 1 --2- 172.21.15.7:0/3191256796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd44101390 0x7fdd4419d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.719+0000 7fdd437fe700 1 -- 172.21.15.7:0/3191256796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd44101d50 msgr2=0x7fdd4419dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:08.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.719+0000 7fdd437fe700 1 --2- 172.21.15.7:0/3191256796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd44101d50 0x7fdd4419dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.719+0000 7fdd437fe700 1 -- 172.21.15.7:0/3191256796 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd44198030 con 0x7fdd44107400 2024-06-10T01:21:08.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.719+0000 7fdd427fc700 1 --2- 172.21.15.7:0/3191256796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd44101d50 0x7fdd4419dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:21:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.719+0000 7fdd437fe700 1 --2- 172.21.15.7:0/3191256796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd44107400 0x7fdd441978d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c00b0a0 tx=0x7fdd3c004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.719+0000 7fdd2bfff700 1 -- 172.21.15.7:0/3191256796 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c021070 con 0x7fdd44107400 2024-06-10T01:21:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.720+0000 7fdd2bfff700 1 -- 172.21.15.7:0/3191256796 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd3c007b80 con 0x7fdd44107400 2024-06-10T01:21:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.720+0000 7fdd4932d700 1 -- 172.21.15.7:0/3191256796 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd441982c0 con 0x7fdd44107400 2024-06-10T01:21:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.720+0000 7fdd2bfff700 1 -- 172.21.15.7:0/3191256796 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c005040 con 0x7fdd44107400 2024-06-10T01:21:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.720+0000 7fdd4932d700 1 -- 172.21.15.7:0/3191256796 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd441a45e0 con 0x7fdd44107400 2024-06-10T01:21:08.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.723+0000 7fdd2bfff700 1 -- 172.21.15.7:0/3191256796 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fdd3c024070 con 0x7fdd44107400 2024-06-10T01:21:08.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.724+0000 7fdd2bfff700 1 --2- 172.21.15.7:0/3191256796 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdd2c0a2ca0 0x7fdd2c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:08.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.724+0000 7fdd42ffd700 1 --2- 172.21.15.7:0/3191256796 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdd2c0a2ca0 0x7fdd2c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:08.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.724+0000 7fdd2bfff700 1 -- 172.21.15.7:0/3191256796 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fdd3c00f020 con 0x7fdd44107400 2024-06-10T01:21:08.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.724+0000 7fdd53bb0b80 1 Processor -- start 2024-06-10T01:21:08.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.724+0000 7fdd53bb0b80 1 -- start start 2024-06-10T01:21:08.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.724+0000 7fdd42ffd700 1 --2- 172.21.15.7:0/3191256796 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdd2c0a2ca0 0x7fdd2c0a5160 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fdd44198a10 tx=0x7fdd34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:08.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.724+0000 7fdd53bb0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602f576c170 0x5602f576c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:08.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.724+0000 7fdd53bb0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602f576ca90 0x5602f5766470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:08.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.724+0000 7fdd53bb0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602f57669b0 0x5602f5766dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:08.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.724+0000 7fdd53bb0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5602f5656f60 con 0x5602f576c170 2024-06-10T01:21:08.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.724+0000 7fdd53bb0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5602f56b5850 con 0x5602f576ca90 2024-06-10T01:21:08.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.724+0000 7fdd53bb0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5602f561d460 con 0x5602f57669b0 2024-06-10T01:21:08.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.725+0000 7fdd42ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602f576c170 0x5602f576c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:08.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.725+0000 7fdd42ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602f576c170 0x5602f576c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53250/0 (socket says 172.21.15.7:53250) 2024-06-10T01:21:08.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.725+0000 7fdd427fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602f576ca90 0x5602f5766470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:08.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.725+0000 7fdd42ffd700 1 -- 172.21.15.7:0/3342212632 learned_addr learned my addr 172.21.15.7:0/3342212632 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:08.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.725+0000 7fdd437fe700 1 --2- 172.21.15.7:0/3342212632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602f57669b0 0x5602f5766dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:08.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.725+0000 7fdd42ffd700 1 -- 172.21.15.7:0/3342212632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602f57669b0 msgr2=0x5602f5766dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:08.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.725+0000 7fdd42ffd700 1 --2- 172.21.15.7:0/3342212632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602f57669b0 0x5602f5766dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.725+0000 7fdd42ffd700 1 -- 172.21.15.7:0/3342212632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602f576ca90 msgr2=0x5602f5766470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:08.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.725+0000 7fdd42ffd700 1 --2- 172.21.15.7:0/3342212632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602f576ca90 0x5602f5766470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.725+0000 7fdd42ffd700 1 -- 172.21.15.7:0/3342212632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602f56048d0 con 0x5602f576c170 2024-06-10T01:21:08.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.725+0000 7fdd42ffd700 1 --2- 172.21.15.7:0/3342212632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602f576c170 0x5602f576c550 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fdd340046c0 tx=0x7fdd34014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e18a72fb0dfb3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:08.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.725+0000 7fdd297fa700 1 -- 172.21.15.7:0/3342212632 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3400eb20 con 0x5602f576c170 2024-06-10T01:21:08.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.725+0000 7fdd297fa700 1 -- 172.21.15.7:0/3342212632 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd3400ecc0 con 0x5602f576c170 2024-06-10T01:21:08.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.725+0000 7fdd297fa700 1 -- 172.21.15.7:0/3342212632 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3401e5f0 con 0x5602f576c170 2024-06-10T01:21:08.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.726+0000 7fdd53bb0b80 1 -- 172.21.15.7:0/3342212632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602f576c170 msgr2=0x5602f576c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:08.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.726+0000 7fdd53bb0b80 1 --2- 172.21.15.7:0/3342212632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602f576c170 0x5602f576c550 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fdd340046c0 tx=0x7fdd34014000 comp rx=0 tx=0).stop 2024-06-10T01:21:08.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.726+0000 7fdd53bb0b80 1 -- 172.21.15.7:0/3342212632 shutdown_connections 2024-06-10T01:21:08.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.726+0000 7fdd53bb0b80 1 --2- 172.21.15.7:0/3342212632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602f57669b0 0x5602f5766dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.726+0000 7fdd53bb0b80 1 --2- 172.21.15.7:0/3342212632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602f576c170 0x5602f576c550 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.726+0000 7fdd53bb0b80 1 --2- 172.21.15.7:0/3342212632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602f576ca90 0x5602f5766470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.726+0000 7fdd53bb0b80 1 -- 172.21.15.7:0/3342212632 >> 172.21.15.7:0/3342212632 conn(0x5602f5691080 msgr2=0x5602f575f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:08.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.726+0000 7fdd53bb0b80 1 -- 172.21.15.7:0/3342212632 shutdown_connections 2024-06-10T01:21:08.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.726+0000 7fdd53bb0b80 1 -- 172.21.15.7:0/3342212632 wait complete. 2024-06-10T01:21:08.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.727+0000 7fdd53bb0b80 1 -- start start 2024-06-10T01:21:08.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.727+0000 7fdd53bb0b80 1 Processor -- start 2024-06-10T01:21:08.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.727+0000 7fdd53bb0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602f57669b0 0x5602f576df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:08.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.727+0000 7fdd53bb0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602f576c170 0x5602f576e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:08.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.727+0000 7fdd53bb0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602f576ca90 0x5602f576ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:08.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.727+0000 7fdd53bb0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5602f5655fc0 con 0x5602f576c170 2024-06-10T01:21:08.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.727+0000 7fdd53bb0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5602f56b1810 con 0x5602f57669b0 2024-06-10T01:21:08.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.727+0000 7fdd53bb0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5602f5764570 con 0x5602f576ca90 2024-06-10T01:21:08.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.727+0000 7fdd53bb0b80 10 client.0 _ll_register_callbacks cb 0x5602f564b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:21:08.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.727+0000 7fdd427fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602f576c170 0x5602f576e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:08.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.727+0000 7fdd427fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602f576c170 0x5602f576e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53266/0 (socket says 172.21.15.7:53266) 2024-06-10T01:21:08.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.727+0000 7fdd427fc700 1 -- 172.21.15.7:0/815730377 learned_addr learned my addr 172.21.15.7:0/815730377 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:08.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.727+0000 7fdd42ffd700 1 --2- 172.21.15.7:0/815730377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602f57669b0 0x5602f576df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:08.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.727+0000 7fdd437fe700 1 --2- 172.21.15.7:0/815730377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602f576ca90 0x5602f576ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:08.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.728+0000 7fdd427fc700 1 -- 172.21.15.7:0/815730377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602f576ca90 msgr2=0x5602f576ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:08.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.728+0000 7fdd427fc700 1 --2- 172.21.15.7:0/815730377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602f576ca90 0x5602f576ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.728+0000 7fdd427fc700 1 -- 172.21.15.7:0/815730377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602f57669b0 msgr2=0x5602f576df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:08.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.728+0000 7fdd427fc700 1 --2- 172.21.15.7:0/815730377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602f57669b0 0x5602f576df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.728+0000 7fdd427fc700 1 -- 172.21.15.7:0/815730377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602f567b750 con 0x5602f576c170 2024-06-10T01:21:08.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.728+0000 7fdd427fc700 1 --2- 172.21.15.7:0/815730377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602f576c170 0x5602f576e4d0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fdd38012c90 tx=0x7fdd3800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb06c8e468f0d12 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:08.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.728+0000 7fdd26ff5700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:21:08.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.728+0000 7fdd26ff5700 1 -- 172.21.15.7:0/815730377 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3800f960 con 0x5602f576c170 2024-06-10T01:21:08.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.728+0000 7fdd26ff5700 1 -- 172.21.15.7:0/815730377 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fdd3800fb00 con 0x5602f576c170 2024-06-10T01:21:08.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.728+0000 7fdd53bb0b80 1 -- 172.21.15.7:0/815730377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd3400ecc0 con 0x5602f576c170 2024-06-10T01:21:08.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.728+0000 7fdd26ff5700 1 -- 172.21.15.7:0/815730377 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fdd3800fd00 con 0x5602f576c170 2024-06-10T01:21:08.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.728+0000 7fdd53bb0b80 10 client.19254 fetch_fsmap learned FSMap version 44 2024-06-10T01:21:08.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.728+0000 7fdd53bb0b80 1 -- 172.21.15.7:0/815730377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5602f577fee0 con 0x5602f576c170 2024-06-10T01:21:08.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.729+0000 7fdd26ff5700 1 -- 172.21.15.7:0/815730377 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd3400ecc0 con 0x5602f576c170 2024-06-10T01:21:08.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.729+0000 7fdd26ff5700 1 -- 172.21.15.7:0/815730377 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fdd38010430 con 0x5602f576c170 2024-06-10T01:21:08.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.729+0000 7fdd53bb0b80 10 client.19254 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:21:08.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.729+0000 7fdd53bb0b80 10 client.19254 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:21:08.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.729+0000 7fdd53bb0b80 20 client.19254 populate_metadata read hostname 'smithi007' 2024-06-10T01:21:08.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.729+0000 7fdd53bb0b80 1 --2- 172.21.15.7:0/815730377 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5602f5782520 0x5602f5784910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:08.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.729+0000 7fdd53bb0b80 4 client.19254 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:21:08.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.729+0000 7fdd53bb0b80 1 -- 172.21.15.7:0/815730377 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602f56c2d50 con 0x5602f5782520 2024-06-10T01:21:08.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.730+0000 7fdd42ffd700 1 --2- 172.21.15.7:0/815730377 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5602f5782520 0x5602f5784910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:08.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.731+0000 7fdd42ffd700 1 --2- 172.21.15.7:0/815730377 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5602f5782520 0x5602f5784910 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d7ce70828542bb13 server_cookie=57cca8f69ba05b69 in_seq=0 out_seq=0 2024-06-10T01:21:08.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.731+0000 7fdd26ff5700 10 client.19254 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:21:08.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.732+0000 7fdd26ff5700 1 -- 172.21.15.7:0/815730377 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5602f56c2d50 con 0x5602f5782520 2024-06-10T01:21:08.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.732+0000 7fdd26ff5700 10 client.19254 handle_command_reply: tid=0 2024-06-10T01:21:08.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.732+0000 7fdd53bb0b80 1 client.19254 shutdown 2024-06-10T01:21:08.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.733+0000 7fdd53bb0b80 1 -- 172.21.15.7:0/815730377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602f576c170 msgr2=0x5602f576e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:08.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.733+0000 7fdd53bb0b80 1 --2- 172.21.15.7:0/815730377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602f576c170 0x5602f576e4d0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fdd38012c90 tx=0x7fdd3800f3c0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.733+0000 7fdd53bb0b80 1 -- 172.21.15.7:0/815730377 shutdown_connections 2024-06-10T01:21:08.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.733+0000 7fdd53bb0b80 1 --2- 172.21.15.7:0/815730377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602f576ca90 0x5602f576ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.733+0000 7fdd53bb0b80 1 --2- 172.21.15.7:0/815730377 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5602f5782520 0x5602f5784910 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.733+0000 7fdd53bb0b80 1 --2- 172.21.15.7:0/815730377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602f57669b0 0x5602f576df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.733+0000 7fdd53bb0b80 1 --2- 172.21.15.7:0/815730377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602f576c170 0x5602f576e4d0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.733+0000 7fdd53bb0b80 1 -- 172.21.15.7:0/815730377 >> 172.21.15.7:0/815730377 conn(0x5602f5691080 msgr2=0x5602f575e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:08.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.733+0000 7fdd26ff5700 0 client.19254 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:21:08.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.733+0000 7fdd53bb0b80 1 -- 172.21.15.7:0/815730377 shutdown_connections 2024-06-10T01:21:08.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.733+0000 7fdd53bb0b80 1 -- 172.21.15.7:0/815730377 wait complete. 2024-06-10T01:21:08.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.733+0000 7fdd53bb0b80 20 client.19254 trim_cache size 0 max 16384 2024-06-10T01:21:08.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.758+0000 7fdd53bb0b80 1 Processor -- start 2024-06-10T01:21:08.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.758+0000 7fdd53bb0b80 1 -- start start 2024-06-10T01:21:08.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.758+0000 7fdd53bb0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602f57669b0 0x5602f576b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:08.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.758+0000 7fdd53bb0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602f576c170 0x5602f5766d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.758+0000 7fdd53bb0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602f576ca90 0x5602f575ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.758+0000 7fdd53bb0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdd441ae940 con 0x5602f576ca90 2024-06-10T01:21:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.758+0000 7fdd53bb0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdd441ac140 con 0x5602f576c170 2024-06-10T01:21:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.758+0000 7fdd53bb0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdd441a9f90 con 0x5602f57669b0 2024-06-10T01:21:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.758+0000 7fdd427fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602f576c170 0x5602f5766d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.758+0000 7fdd437fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602f576ca90 0x5602f575ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.758+0000 7fdd437fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602f576ca90 0x5602f575ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53274/0 (socket says 172.21.15.7:53274) 2024-06-10T01:21:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.758+0000 7fdd437fe700 1 -- 172.21.15.7:0/1677845519 learned_addr learned my addr 172.21.15.7:0/1677845519 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.758+0000 7fdd427fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602f576c170 0x5602f5766d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:41858/0 (socket says 172.21.15.7:41858) 2024-06-10T01:21:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.759+0000 7fdd437fe700 1 -- 172.21.15.7:0/1677845519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602f57669b0 msgr2=0x5602f576b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.759+0000 7fdd437fe700 1 --2- 172.21.15.7:0/1677845519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602f57669b0 0x5602f576b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.759+0000 7fdd437fe700 1 -- 172.21.15.7:0/1677845519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602f576c170 msgr2=0x5602f5766d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.759+0000 7fdd437fe700 1 --2- 172.21.15.7:0/1677845519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602f576c170 0x5602f5766d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.759+0000 7fdd437fe700 1 -- 172.21.15.7:0/1677845519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602f576b3b0 con 0x5602f576ca90 2024-06-10T01:21:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.759+0000 7fdd437fe700 1 --2- 172.21.15.7:0/1677845519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602f576ca90 0x5602f575ee80 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0995d0 tx=0x7fdd3c00b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154bf67c0718d6f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.759+0000 7fdd28ff9700 1 -- 172.21.15.7:0/1677845519 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c09cc00 con 0x5602f576ca90 2024-06-10T01:21:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.760+0000 7fdd28ff9700 1 -- 172.21.15.7:0/1677845519 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd3c0a0fb0 con 0x5602f576ca90 2024-06-10T01:21:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.760+0000 7fdd28ff9700 1 -- 172.21.15.7:0/1677845519 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c02e350 con 0x5602f576ca90 2024-06-10T01:21:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.761+0000 7fdd53bb0b80 1 -- 172.21.15.7:0/1677845519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602f576ca90 msgr2=0x5602f575ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.761+0000 7fdd53bb0b80 1 --2- 172.21.15.7:0/1677845519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602f576ca90 0x5602f575ee80 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0995d0 tx=0x7fdd3c00b3b0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.762+0000 7fdd53bb0b80 1 -- 172.21.15.7:0/1677845519 shutdown_connections 2024-06-10T01:21:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.762+0000 7fdd53bb0b80 1 --2- 172.21.15.7:0/1677845519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602f576ca90 0x5602f575ee80 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.762+0000 7fdd53bb0b80 1 --2- 172.21.15.7:0/1677845519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602f57669b0 0x5602f576b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.762+0000 7fdd53bb0b80 1 --2- 172.21.15.7:0/1677845519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602f576c170 0x5602f5766d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.762+0000 7fdd53bb0b80 1 -- 172.21.15.7:0/1677845519 >> 172.21.15.7:0/1677845519 conn(0x5602f5691080 msgr2=0x5602f5768540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:08.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.763+0000 7fdd53bb0b80 1 -- 172.21.15.7:0/1677845519 shutdown_connections 2024-06-10T01:21:08.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.764+0000 7fdd53bb0b80 1 -- 172.21.15.7:0/1677845519 wait complete. 2024-06-10T01:21:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.764+0000 7fdd53bb0b80 1 -- start start 2024-06-10T01:21:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.765+0000 7fdd53bb0b80 1 Processor -- start 2024-06-10T01:21:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.765+0000 7fdd53bb0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602f57669b0 0x5602f5776a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.765+0000 7fdd53bb0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602f576c170 0x5602f5776fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.765+0000 7fdd53bb0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602f576ca90 0x5602f5769e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.765+0000 7fdd53bb0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdd441aa600 con 0x5602f576c170 2024-06-10T01:21:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.765+0000 7fdd53bb0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdd441aba70 con 0x5602f576ca90 2024-06-10T01:21:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.765+0000 7fdd53bb0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdd441aacd0 con 0x5602f57669b0 2024-06-10T01:21:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.765+0000 7fdd53bb0b80 10 client.0 _ll_register_callbacks cb 0x5602f564b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:21:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.765+0000 7fdd437fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602f576ca90 0x5602f5769e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.765+0000 7fdd437fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602f576ca90 0x5602f5769e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:41870/0 (socket says 172.21.15.7:41870) 2024-06-10T01:21:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.765+0000 7fdd437fe700 1 -- 172.21.15.7:0/905023754 learned_addr learned my addr 172.21.15.7:0/905023754 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.766+0000 7fdd42ffd700 1 --2- 172.21.15.7:0/905023754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602f57669b0 0x5602f5776a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.766+0000 7fdd427fc700 1 --2- 172.21.15.7:0/905023754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602f576c170 0x5602f5776fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.766+0000 7fdd437fe700 1 -- 172.21.15.7:0/905023754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602f57669b0 msgr2=0x5602f5776a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.766+0000 7fdd437fe700 1 --2- 172.21.15.7:0/905023754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602f57669b0 0x5602f5776a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.766+0000 7fdd437fe700 1 -- 172.21.15.7:0/905023754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602f576c170 msgr2=0x5602f5776fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.766+0000 7fdd437fe700 1 --2- 172.21.15.7:0/905023754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602f576c170 0x5602f5776fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.766+0000 7fdd437fe700 1 -- 172.21.15.7:0/905023754 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602f576a650 con 0x5602f576ca90 2024-06-10T01:21:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.766+0000 7fdd437fe700 1 --2- 172.21.15.7:0/905023754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602f576ca90 0x5602f5769e50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0a52b0 tx=0x7fdd3c004eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75bfe37f3ae3c262 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.766+0000 7fdd26ff5700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:21:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.768+0000 7fdd26ff5700 1 -- 172.21.15.7:0/905023754 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c080060 con 0x5602f576ca90 2024-06-10T01:21:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.768+0000 7fdd26ff5700 1 -- 172.21.15.7:0/905023754 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fdd3c0074e0 con 0x5602f576ca90 2024-06-10T01:21:08.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.770+0000 7fdd53bb0b80 1 -- 172.21.15.7:0/905023754 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd3c0a0fb0 con 0x5602f576ca90 2024-06-10T01:21:08.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.771+0000 7fdd26ff5700 1 -- 172.21.15.7:0/905023754 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fdd3c02e350 con 0x5602f576ca90 2024-06-10T01:21:08.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.771+0000 7fdd53bb0b80 10 client.28511 fetch_fsmap learned FSMap version 44 2024-06-10T01:21:08.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.771+0000 7fdd53bb0b80 1 -- 172.21.15.7:0/905023754 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5602f57e5d50 con 0x5602f576ca90 2024-06-10T01:21:08.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.771+0000 7fdd26ff5700 1 -- 172.21.15.7:0/905023754 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd3c0a0fb0 con 0x5602f576ca90 2024-06-10T01:21:08.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.772+0000 7fdd26ff5700 1 -- 172.21.15.7:0/905023754 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fdd3c011070 con 0x5602f576ca90 2024-06-10T01:21:08.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.772+0000 7fdd53bb0b80 10 client.28511 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:21:08.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.772+0000 7fdd53bb0b80 10 client.28511 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:21:08.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.772+0000 7fdd53bb0b80 20 client.28511 populate_metadata read hostname 'smithi007' 2024-06-10T01:21:08.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.772+0000 7fdd53bb0b80 1 --2- 172.21.15.7:0/905023754 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5602f57e63f0 0x5602f57e8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:08.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.772+0000 7fdd53bb0b80 4 client.28511 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:21:08.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.772+0000 7fdd53bb0b80 1 -- 172.21.15.7:0/905023754 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5602f56bd2d0 con 0x5602f57e63f0 2024-06-10T01:21:08.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.773+0000 7fdd42ffd700 1 --2- 172.21.15.7:0/905023754 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5602f57e63f0 0x5602f57e8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:08.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.775+0000 7fdd42ffd700 1 --2- 172.21.15.7:0/905023754 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5602f57e63f0 0x5602f57e8810 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=898903ff5ea93efa server_cookie=69fd2c85b0f8e40d in_seq=0 out_seq=0 2024-06-10T01:21:08.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.775+0000 7fdd26ff5700 10 client.28511 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:21:08.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.778+0000 7fdd26ff5700 1 -- 172.21.15.7:0/905023754 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5602f56bd2d0 con 0x5602f57e63f0 2024-06-10T01:21:08.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.778+0000 7fdd26ff5700 10 client.28511 handle_command_reply: tid=0 2024-06-10T01:21:08.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.779+0000 7fdd53bb0b80 1 client.28511 shutdown 2024-06-10T01:21:08.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.779+0000 7fdd53bb0b80 1 -- 172.21.15.7:0/905023754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602f576ca90 msgr2=0x5602f5769e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:08.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.779+0000 7fdd53bb0b80 1 --2- 172.21.15.7:0/905023754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602f576ca90 0x5602f5769e50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0a52b0 tx=0x7fdd3c004eb0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.779+0000 7fdd53bb0b80 1 -- 172.21.15.7:0/905023754 shutdown_connections 2024-06-10T01:21:08.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.779+0000 7fdd53bb0b80 1 --2- 172.21.15.7:0/905023754 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5602f57e63f0 0x5602f57e8810 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.779+0000 7fdd53bb0b80 1 --2- 172.21.15.7:0/905023754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602f576ca90 0x5602f5769e50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.779+0000 7fdd26ff5700 0 client.28511 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:21:08.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.779+0000 7fdd53bb0b80 1 --2- 172.21.15.7:0/905023754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602f57669b0 0x5602f5776a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.779+0000 7fdd53bb0b80 1 --2- 172.21.15.7:0/905023754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602f576c170 0x5602f5776fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.779+0000 7fdd53bb0b80 1 -- 172.21.15.7:0/905023754 >> 172.21.15.7:0/905023754 conn(0x5602f5691080 msgr2=0x5602f5767bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:08.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.779+0000 7fdd53bb0b80 1 -- 172.21.15.7:0/905023754 shutdown_connections 2024-06-10T01:21:08.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.779+0000 7fdd53bb0b80 1 -- 172.21.15.7:0/905023754 wait complete. 2024-06-10T01:21:08.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.779+0000 7fdd53bb0b80 20 client.28511 trim_cache size 0 max 16384 2024-06-10T01:21:08.776 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:21:08.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.780+0000 7fdd4932d700 1 -- 172.21.15.7:0/3191256796 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdd2c0a2ca0 msgr2=0x7fdd2c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:08.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.780+0000 7fdd4932d700 1 --2- 172.21.15.7:0/3191256796 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdd2c0a2ca0 0x7fdd2c0a5160 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fdd44198a10 tx=0x7fdd34009260 comp rx=0 tx=0).stop 2024-06-10T01:21:08.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.780+0000 7fdd4932d700 1 -- 172.21.15.7:0/3191256796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd44107400 msgr2=0x7fdd441978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:08.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.780+0000 7fdd4932d700 1 --2- 172.21.15.7:0/3191256796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd44107400 0x7fdd441978d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c00b0a0 tx=0x7fdd3c004d00 comp rx=0 tx=0).stop 2024-06-10T01:21:08.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.780+0000 7fdd4932d700 1 -- 172.21.15.7:0/3191256796 shutdown_connections 2024-06-10T01:21:08.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.780+0000 7fdd4932d700 1 --2- 172.21.15.7:0/3191256796 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdd2c0a2ca0 0x7fdd2c0a5160 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.780+0000 7fdd4932d700 1 --2- 172.21.15.7:0/3191256796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd44107400 0x7fdd441978d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.780+0000 7fdd4932d700 1 --2- 172.21.15.7:0/3191256796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd44101390 0x7fdd4419d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.780+0000 7fdd4932d700 1 --2- 172.21.15.7:0/3191256796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd44101d50 0x7fdd4419dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:08.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.780+0000 7fdd4932d700 1 -- 172.21.15.7:0/3191256796 >> 172.21.15.7:0/3191256796 conn(0x7fdd44078340 msgr2=0x7fdd440ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:08.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.780+0000 7fdd4932d700 1 -- 172.21.15.7:0/3191256796 shutdown_connections 2024-06-10T01:21:08.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:08.781+0000 7fdd4932d700 1 -- 172.21.15.7:0/3191256796 wait complete. 2024-06-10T01:21:09.495 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:21:09.502 INFO:tasks.workunit.client.0.smithi007.stdout: 10 30167 17.05 MB/sec warmup 106 sec latency 159.221 ms 2024-06-10T01:21:09.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:09 smithi007 ceph-mon[32398]: pgmap v886: 129 pgs: 129 active+clean; 399 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.8 MiB/s wr, 486 op/s 2024-06-10T01:21:09.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:09 smithi078 ceph-mon[36683]: pgmap v886: 129 pgs: 129 active+clean; 399 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.8 MiB/s wr, 486 op/s 2024-06-10T01:21:09.832 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:21:09.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:09 smithi175 ceph-mon[36636]: pgmap v886: 129 pgs: 129 active+clean; 399 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.8 MiB/s wr, 486 op/s 2024-06-10T01:21:10.502 INFO:tasks.workunit.client.0.smithi007.stdout: 10 30792 17.09 MB/sec warmup 107 sec latency 77.415 ms 2024-06-10T01:21:11.502 INFO:tasks.workunit.client.0.smithi007.stdout: 10 31360 17.23 MB/sec warmup 108 sec latency 182.342 ms 2024-06-10T01:21:11.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:11 smithi078 ceph-mon[36683]: pgmap v887: 129 pgs: 129 active+clean; 400 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 4.3 MiB/s wr, 588 op/s 2024-06-10T01:21:11.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:11 smithi175 ceph-mon[36636]: pgmap v887: 129 pgs: 129 active+clean; 400 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 4.3 MiB/s wr, 588 op/s 2024-06-10T01:21:12.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:11 smithi007 ceph-mon[32398]: pgmap v887: 129 pgs: 129 active+clean; 400 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 4.3 MiB/s wr, 588 op/s 2024-06-10T01:21:12.502 INFO:tasks.workunit.client.0.smithi007.stdout: 10 31809 17.31 MB/sec warmup 109 sec latency 757.627 ms 2024-06-10T01:21:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.254+0000 7fa411018700 1 -- 172.21.15.7:0/572957386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa40c10b750 msgr2=0x7fa40c113ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.254+0000 7fa411018700 1 --2- 172.21.15.7:0/572957386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa40c10b750 0x7fa40c113ea0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4009990 tx=0x7fa3f4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.254+0000 7fa411018700 1 -- 172.21.15.7:0/572957386 shutdown_connections 2024-06-10T01:21:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.254+0000 7fa411018700 1 --2- 172.21.15.7:0/572957386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa40c1143e0 0x7fa40c1167d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.254+0000 7fa411018700 1 --2- 172.21.15.7:0/572957386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa40c10ae30 0x7fa40c10b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.254+0000 7fa411018700 1 --2- 172.21.15.7:0/572957386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa40c10b750 0x7fa40c113ea0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.254+0000 7fa411018700 1 -- 172.21.15.7:0/572957386 >> 172.21.15.7:0/572957386 conn(0x7fa40c06d600 msgr2=0x7fa40c06da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:13.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.254+0000 7fa411018700 1 -- 172.21.15.7:0/572957386 shutdown_connections 2024-06-10T01:21:13.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.254+0000 7fa411018700 1 -- 172.21.15.7:0/572957386 wait complete. 2024-06-10T01:21:13.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.255+0000 7fa411018700 1 Processor -- start 2024-06-10T01:21:13.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.256+0000 7fa411018700 1 -- start start 2024-06-10T01:21:13.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.256+0000 7fa411018700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa40c10ae30 0x7fa40c071100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:13.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.256+0000 7fa411018700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa40c10b750 0x7fa40c071640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:13.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.256+0000 7fa411018700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa40c1143e0 0x7fa40c071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:13.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.256+0000 7fa411018700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c119f50 con 0x7fa40c10ae30 2024-06-10T01:21:13.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.256+0000 7fa411018700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c119dd0 con 0x7fa40c10b750 2024-06-10T01:21:13.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.256+0000 7fa411018700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c11a0d0 con 0x7fa40c1143e0 2024-06-10T01:21:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.257+0000 7fa403fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa40c10b750 0x7fa40c071640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.257+0000 7fa403fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa40c10b750 0x7fa40c071640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:35126/0 (socket says 172.21.15.7:35126) 2024-06-10T01:21:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.257+0000 7fa403fff700 1 -- 172.21.15.7:0/1972408312 learned_addr learned my addr 172.21.15.7:0/1972408312 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.257+0000 7fa40b59e700 1 --2- 172.21.15.7:0/1972408312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa40c1143e0 0x7fa40c071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.257+0000 7fa40ad9d700 1 --2- 172.21.15.7:0/1972408312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa40c10ae30 0x7fa40c071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.258+0000 7fa40b59e700 1 -- 172.21.15.7:0/1972408312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa40c10b750 msgr2=0x7fa40c071640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.258+0000 7fa40b59e700 1 --2- 172.21.15.7:0/1972408312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa40c10b750 0x7fa40c071640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.258+0000 7fa40b59e700 1 -- 172.21.15.7:0/1972408312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa40c10ae30 msgr2=0x7fa40c071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.258+0000 7fa40b59e700 1 --2- 172.21.15.7:0/1972408312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa40c10ae30 0x7fa40c071100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.258+0000 7fa40b59e700 1 -- 172.21.15.7:0/1972408312 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40c1b3e50 con 0x7fa40c1143e0 2024-06-10T01:21:13.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.258+0000 7fa403fff700 1 --2- 172.21.15.7:0/1972408312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa40c10b750 0x7fa40c071640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:21:13.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.258+0000 7fa40ad9d700 1 --2- 172.21.15.7:0/1972408312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa40c10ae30 0x7fa40c071100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:21:13.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.259+0000 7fa40b59e700 1 --2- 172.21.15.7:0/1972408312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa40c1143e0 0x7fa40c071b80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa40400efc0 tx=0x7fa40400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:13.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.259+0000 7fa408d99700 1 -- 172.21.15.7:0/1972408312 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4040092e0 con 0x7fa40c1143e0 2024-06-10T01:21:13.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.259+0000 7fa411018700 1 -- 172.21.15.7:0/1972408312 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa40c1b40e0 con 0x7fa40c1143e0 2024-06-10T01:21:13.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.259+0000 7fa408d99700 1 -- 172.21.15.7:0/1972408312 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa404004bb0 con 0x7fa40c1143e0 2024-06-10T01:21:13.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.259+0000 7fa411018700 1 -- 172.21.15.7:0/1972408312 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa40c1b4620 con 0x7fa40c1143e0 2024-06-10T01:21:13.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.260+0000 7fa408d99700 1 -- 172.21.15.7:0/1972408312 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404004280 con 0x7fa40c1143e0 2024-06-10T01:21:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.265+0000 7fa41b89bb80 1 Processor -- start 2024-06-10T01:21:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.265+0000 7fa408d99700 1 -- 172.21.15.7:0/1972408312 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa40401b030 con 0x7fa40c1143e0 2024-06-10T01:21:13.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.265+0000 7fa41b89bb80 1 -- start start 2024-06-10T01:21:13.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.266+0000 7fa41b89bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d6fbdf60 0x5583d6fbe340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:13.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.266+0000 7fa41b89bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583d6fbe880 0x5583d6fb8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:13.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.266+0000 7fa41b89bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583d6fb87a0 0x5583d6fb8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:13.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.266+0000 7fa41b89bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5583d6eaaf60 con 0x5583d6fbe880 2024-06-10T01:21:13.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.266+0000 7fa403fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d6fbdf60 0x5583d6fbe340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:13.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.266+0000 7fa403fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d6fbdf60 0x5583d6fbe340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:35136/0 (socket says 172.21.15.7:35136) 2024-06-10T01:21:13.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.266+0000 7fa403fff700 1 -- 172.21.15.7:0/2570201872 learned_addr learned my addr 172.21.15.7:0/2570201872 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:13.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.266+0000 7fa41b89bb80 1 -- 172.21.15.7:0/2570201872 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5583d6f09850 con 0x5583d6fbdf60 2024-06-10T01:21:13.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.267+0000 7fa41b89bb80 1 -- 172.21.15.7:0/2570201872 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5583d6e71460 con 0x5583d6fb87a0 2024-06-10T01:21:13.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.267+0000 7fa40ad9d700 1 --2- 172.21.15.7:0/2570201872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583d6fbe880 0x5583d6fb8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:13.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.267+0000 7fa408d99700 1 --2- 172.21.15.7:0/1972408312 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa3f80a2c20 0x7fa3f80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:13.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.267+0000 7fa403fff700 1 --2- 172.21.15.7:0/2570201872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583d6fb87a0 0x5583d6fb8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:13.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.267+0000 7fa408d99700 1 -- 172.21.15.7:0/1972408312 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa4040cef50 con 0x7fa40c1143e0 2024-06-10T01:21:13.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.268+0000 7fa40ad9d700 1 -- 172.21.15.7:0/2570201872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583d6fb87a0 msgr2=0x5583d6fb8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:13.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.268+0000 7fa40ad9d700 1 --2- 172.21.15.7:0/2570201872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583d6fb87a0 0x5583d6fb8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.268+0000 7fa40ad9d700 1 -- 172.21.15.7:0/2570201872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d6fbdf60 msgr2=0x5583d6fbe340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:13.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.268+0000 7fa40ad9d700 1 --2- 172.21.15.7:0/2570201872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d6fbdf60 0x5583d6fbe340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.268+0000 7fa40ad9d700 1 -- 172.21.15.7:0/2570201872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583d6e588d0 con 0x5583d6fbe880 2024-06-10T01:21:13.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.268+0000 7fa40b59e700 1 --2- 172.21.15.7:0/1972408312 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa3f80a2c20 0x7fa3f80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:13.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.268+0000 7fa40ad9d700 1 --2- 172.21.15.7:0/2570201872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583d6fbe880 0x5583d6fb8260 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc009cc0 tx=0x7fa3fc0113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea06492add615a0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:13.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.269+0000 7fa40b59e700 1 --2- 172.21.15.7:0/1972408312 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa3f80a2c20 0x7fa3f80a50e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa4040090a0 tx=0x7fa40400cf60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:13.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.269+0000 7fa4017fa700 1 -- 172.21.15.7:0/2570201872 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc011ce0 con 0x5583d6fbe880 2024-06-10T01:21:13.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.269+0000 7fa4017fa700 1 -- 172.21.15.7:0/2570201872 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3fc011e80 con 0x5583d6fbe880 2024-06-10T01:21:13.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.270+0000 7fa4017fa700 1 -- 172.21.15.7:0/2570201872 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc00f840 con 0x5583d6fbe880 2024-06-10T01:21:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.270+0000 7fa41b89bb80 1 -- 172.21.15.7:0/2570201872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583d6fbe880 msgr2=0x5583d6fb8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.270+0000 7fa41b89bb80 1 --2- 172.21.15.7:0/2570201872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583d6fbe880 0x5583d6fb8260 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc009cc0 tx=0x7fa3fc0113e0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.271+0000 7fa41b89bb80 1 -- 172.21.15.7:0/2570201872 shutdown_connections 2024-06-10T01:21:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.271+0000 7fa41b89bb80 1 --2- 172.21.15.7:0/2570201872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583d6fb87a0 0x5583d6fb8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.271+0000 7fa41b89bb80 1 --2- 172.21.15.7:0/2570201872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d6fbdf60 0x5583d6fbe340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.271+0000 7fa41b89bb80 1 --2- 172.21.15.7:0/2570201872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583d6fbe880 0x5583d6fb8260 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.271+0000 7fa41b89bb80 1 -- 172.21.15.7:0/2570201872 >> 172.21.15.7:0/2570201872 conn(0x5583d6ee5080 msgr2=0x5583d6fb3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.271+0000 7fa41b89bb80 1 -- 172.21.15.7:0/2570201872 shutdown_connections 2024-06-10T01:21:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.272+0000 7fa41b89bb80 1 -- 172.21.15.7:0/2570201872 wait complete. 2024-06-10T01:21:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.272+0000 7fa41b89bb80 1 -- start start 2024-06-10T01:21:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.272+0000 7fa41b89bb80 1 Processor -- start 2024-06-10T01:21:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.272+0000 7fa41b89bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583d6fb87a0 0x5583d6fc44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.273+0000 7fa41b89bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583d6fbdf60 0x5583d6fc17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.273+0000 7fa41b89bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d6fbe880 0x5583d6fc1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.273+0000 7fa41b89bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5583d6ea9fc0 con 0x5583d6fb87a0 2024-06-10T01:21:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.273+0000 7fa41b89bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5583d6f05810 con 0x5583d6fbe880 2024-06-10T01:21:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.273+0000 7fa41b89bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5583d6fc52b0 con 0x5583d6fbdf60 2024-06-10T01:21:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.273+0000 7fa41b89bb80 10 client.0 _ll_register_callbacks cb 0x5583d6e9f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:21:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.273+0000 7fa40ad9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583d6fbdf60 0x5583d6fc17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.273+0000 7fa40ad9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583d6fbdf60 0x5583d6fc17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:53244/0 (socket says 172.21.15.7:53244) 2024-06-10T01:21:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.273+0000 7fa40ad9d700 1 -- 172.21.15.7:0/2239362306 learned_addr learned my addr 172.21.15.7:0/2239362306 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.273+0000 7fa403fff700 1 --2- 172.21.15.7:0/2239362306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583d6fb87a0 0x5583d6fc44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.273+0000 7fa403fff700 1 --2- 172.21.15.7:0/2239362306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d6fbe880 0x5583d6fc1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.273+0000 7fa40ad9d700 1 -- 172.21.15.7:0/2239362306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d6fbe880 msgr2=0x5583d6fc1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.273+0000 7fa40ad9d700 1 --2- 172.21.15.7:0/2239362306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d6fbe880 0x5583d6fc1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.273+0000 7fa40ad9d700 1 -- 172.21.15.7:0/2239362306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583d6fb87a0 msgr2=0x5583d6fc44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.273+0000 7fa40ad9d700 1 --2- 172.21.15.7:0/2239362306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583d6fb87a0 0x5583d6fc44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.273+0000 7fa40ad9d700 1 -- 172.21.15.7:0/2239362306 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583d6ecf750 con 0x5583d6fbdf60 2024-06-10T01:21:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.274+0000 7fa40ad9d700 1 --2- 172.21.15.7:0/2239362306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583d6fbdf60 0x5583d6fc17a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc000c00 tx=0x7fa3fc005430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d430c1b9c93d97e9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.274+0000 7fa3f27fc700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:21:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.274+0000 7fa3f27fc700 1 -- 172.21.15.7:0/2239362306 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc028020 con 0x5583d6fbdf60 2024-06-10T01:21:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.274+0000 7fa41b89bb80 1 -- 172.21.15.7:0/2239362306 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5583d6f16d50 con 0x5583d6fbdf60 2024-06-10T01:21:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.274+0000 7fa3f27fc700 1 -- 172.21.15.7:0/2239362306 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa3fc004960 con 0x5583d6fbdf60 2024-06-10T01:21:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.274+0000 7fa3f27fc700 1 -- 172.21.15.7:0/2239362306 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa3fc033050 con 0x5583d6fbdf60 2024-06-10T01:21:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.274+0000 7fa41b89bb80 10 client.28537 fetch_fsmap learned FSMap version 44 2024-06-10T01:21:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.275+0000 7fa41b89bb80 1 -- 172.21.15.7:0/2239362306 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5583d6fcf940 con 0x5583d6fbdf60 2024-06-10T01:21:13.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.275+0000 7fa3f27fc700 1 -- 172.21.15.7:0/2239362306 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3fc004b60 con 0x5583d6fbdf60 2024-06-10T01:21:13.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.276+0000 7fa3f27fc700 1 -- 172.21.15.7:0/2239362306 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fa3fc0245e0 con 0x5583d6fbdf60 2024-06-10T01:21:13.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.276+0000 7fa41b89bb80 10 client.28537 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:21:13.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.276+0000 7fa41b89bb80 10 client.28537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:21:13.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.276+0000 7fa41b89bb80 20 client.28537 populate_metadata read hostname 'smithi007' 2024-06-10T01:21:13.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.276+0000 7fa41b89bb80 1 --2- 172.21.15.7:0/2239362306 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5583d6fd2200 0x5583d6fd4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:13.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.276+0000 7fa41b89bb80 4 client.28537 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:21:13.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.276+0000 7fa41b89bb80 1 -- 172.21.15.7:0/2239362306 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583d6f16d50 con 0x5583d6fd2200 2024-06-10T01:21:13.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.276+0000 7fa403fff700 1 --2- 172.21.15.7:0/2239362306 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5583d6fd2200 0x5583d6fd4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:13.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.277+0000 7fa403fff700 1 --2- 172.21.15.7:0/2239362306 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5583d6fd2200 0x5583d6fd4660 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfab3e1da84c7288 server_cookie=246d9de61f7cff26 in_seq=0 out_seq=0 2024-06-10T01:21:13.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.277+0000 7fa3f27fc700 10 client.28537 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.279+0000 7fa3f27fc700 1 -- 172.21.15.7:0/2239362306 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5583d6f16d50 con 0x5583d6fd2200 2024-06-10T01:21:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.279+0000 7fa3f27fc700 10 client.28537 handle_command_reply: tid=0 2024-06-10T01:21:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.280+0000 7fa41b89bb80 1 client.28537 shutdown 2024-06-10T01:21:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.280+0000 7fa41b89bb80 1 -- 172.21.15.7:0/2239362306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583d6fbdf60 msgr2=0x5583d6fc17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.280+0000 7fa41b89bb80 1 --2- 172.21.15.7:0/2239362306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583d6fbdf60 0x5583d6fc17a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc000c00 tx=0x7fa3fc005430 comp rx=0 tx=0).stop 2024-06-10T01:21:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.280+0000 7fa41b89bb80 1 -- 172.21.15.7:0/2239362306 shutdown_connections 2024-06-10T01:21:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.280+0000 7fa41b89bb80 1 --2- 172.21.15.7:0/2239362306 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5583d6fd2200 0x5583d6fd4660 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.280+0000 7fa41b89bb80 1 --2- 172.21.15.7:0/2239362306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d6fbe880 0x5583d6fc1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.280+0000 7fa41b89bb80 1 --2- 172.21.15.7:0/2239362306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583d6fb87a0 0x5583d6fc44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.280+0000 7fa41b89bb80 1 --2- 172.21.15.7:0/2239362306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583d6fbdf60 0x5583d6fc17a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.280+0000 7fa41b89bb80 1 -- 172.21.15.7:0/2239362306 >> 172.21.15.7:0/2239362306 conn(0x5583d6ee5080 msgr2=0x5583d6fb2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.280+0000 7fa3f27fc700 0 client.28537 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.281+0000 7fa41b89bb80 1 -- 172.21.15.7:0/2239362306 shutdown_connections 2024-06-10T01:21:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.281+0000 7fa41b89bb80 1 -- 172.21.15.7:0/2239362306 wait complete. 2024-06-10T01:21:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.281+0000 7fa41b89bb80 20 client.28537 trim_cache size 0 max 16384 2024-06-10T01:21:13.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.315+0000 7fa41b89bb80 1 Processor -- start 2024-06-10T01:21:13.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.315+0000 7fa41b89bb80 1 -- start start 2024-06-10T01:21:13.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.316+0000 7fa41b89bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583d6fb87a0 0x5583d6fc1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:13.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.316+0000 7fa41b89bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583d6fbdf60 0x5583d6fb2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:13.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.316+0000 7fa41b89bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d6fbe880 0x5583d6fb33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:13.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.316+0000 7fa41b89bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c1bfbb0 con 0x5583d6fb87a0 2024-06-10T01:21:13.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.316+0000 7fa41b89bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c1bd3b0 con 0x5583d6fbe880 2024-06-10T01:21:13.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.316+0000 7fa41b89bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c1bb200 con 0x5583d6fbdf60 2024-06-10T01:21:13.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.316+0000 7fa403fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583d6fb87a0 0x5583d6fc1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:13.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.316+0000 7fa403fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d6fbe880 0x5583d6fb33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:13.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.316+0000 7fa403fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d6fbe880 0x5583d6fb33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:35160/0 (socket says 172.21.15.7:35160) 2024-06-10T01:21:13.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.316+0000 7fa403fff700 1 -- 172.21.15.7:0/3011765841 learned_addr learned my addr 172.21.15.7:0/3011765841 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:13.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.316+0000 7fa403fff700 1 -- 172.21.15.7:0/3011765841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583d6fbdf60 msgr2=0x5583d6fb2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:13.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.317+0000 7fa40ad9d700 1 --2- 172.21.15.7:0/3011765841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583d6fbdf60 0x5583d6fb2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:13.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.317+0000 7fa403fff700 1 --2- 172.21.15.7:0/3011765841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583d6fbdf60 0x5583d6fb2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.317+0000 7fa403fff700 1 -- 172.21.15.7:0/3011765841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583d6fb87a0 msgr2=0x5583d6fc1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:13.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.317+0000 7fa403fff700 1 --2- 172.21.15.7:0/3011765841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583d6fb87a0 0x5583d6fc1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.317+0000 7fa403fff700 1 -- 172.21.15.7:0/3011765841 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583d6fbd920 con 0x5583d6fbe880 2024-06-10T01:21:13.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.317+0000 7fa403fff700 1 --2- 172.21.15.7:0/3011765841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d6fbe880 0x5583d6fb33c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa40c1a55e0 tx=0x7fa3f4009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=290ab20c563448dd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:13.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.317+0000 7fa400ff9700 1 -- 172.21.15.7:0/3011765841 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f4033040 con 0x5583d6fbe880 2024-06-10T01:21:13.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.317+0000 7fa400ff9700 1 -- 172.21.15.7:0/3011765841 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3f4003b60 con 0x5583d6fbe880 2024-06-10T01:21:13.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.317+0000 7fa400ff9700 1 -- 172.21.15.7:0/3011765841 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f400fa40 con 0x5583d6fbe880 2024-06-10T01:21:13.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.318+0000 7fa41b89bb80 1 -- 172.21.15.7:0/3011765841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d6fbe880 msgr2=0x5583d6fb33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:13.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.318+0000 7fa41b89bb80 1 --2- 172.21.15.7:0/3011765841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d6fbe880 0x5583d6fb33c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa40c1a55e0 tx=0x7fa3f4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.318+0000 7fa41b89bb80 1 -- 172.21.15.7:0/3011765841 shutdown_connections 2024-06-10T01:21:13.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.318+0000 7fa41b89bb80 1 --2- 172.21.15.7:0/3011765841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d6fbe880 0x5583d6fb33c0 secure :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa40c1a55e0 tx=0x7fa3f4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.318+0000 7fa41b89bb80 1 --2- 172.21.15.7:0/3011765841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583d6fb87a0 0x5583d6fc1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.318+0000 7fa41b89bb80 1 --2- 172.21.15.7:0/3011765841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583d6fbdf60 0x5583d6fb2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.318+0000 7fa41b89bb80 1 -- 172.21.15.7:0/3011765841 >> 172.21.15.7:0/3011765841 conn(0x5583d6ee5080 msgr2=0x5583d6fc1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:13.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.319+0000 7fa41b89bb80 1 -- 172.21.15.7:0/3011765841 shutdown_connections 2024-06-10T01:21:13.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.319+0000 7fa41b89bb80 1 -- 172.21.15.7:0/3011765841 wait complete. 2024-06-10T01:21:13.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.319+0000 7fa41b89bb80 1 -- start start 2024-06-10T01:21:13.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.319+0000 7fa41b89bb80 1 Processor -- start 2024-06-10T01:21:13.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.320+0000 7fa41b89bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583d6fb87a0 0x5583d6fda6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:13.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.320+0000 7fa41b89bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583d6fbdf60 0x5583d6fca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:13.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.320+0000 7fa41b89bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d6fcab30 0x5583d6fd8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:13.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.320+0000 7fa41b89bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c1bb870 con 0x5583d6fb87a0 2024-06-10T01:21:13.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.320+0000 7fa41b89bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c1bcce0 con 0x5583d6fcab30 2024-06-10T01:21:13.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.320+0000 7fa41b89bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c1bbf40 con 0x5583d6fbdf60 2024-06-10T01:21:13.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.320+0000 7fa41b89bb80 10 client.0 _ll_register_callbacks cb 0x5583d6e9f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:21:13.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.320+0000 7fa403fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583d6fb87a0 0x5583d6fda6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:13.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.320+0000 7fa403fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583d6fb87a0 0x5583d6fda6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45582/0 (socket says 172.21.15.7:45582) 2024-06-10T01:21:13.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.320+0000 7fa403fff700 1 -- 172.21.15.7:0/2429165387 learned_addr learned my addr 172.21.15.7:0/2429165387 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:13.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.320+0000 7fa403fff700 1 --2- 172.21.15.7:0/2429165387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d6fcab30 0x5583d6fd8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:13.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.320+0000 7fa40ad9d700 1 --2- 172.21.15.7:0/2429165387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583d6fbdf60 0x5583d6fca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:13.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.321+0000 7fa403fff700 1 -- 172.21.15.7:0/2429165387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583d6fbdf60 msgr2=0x5583d6fca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:13.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.321+0000 7fa403fff700 1 --2- 172.21.15.7:0/2429165387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583d6fbdf60 0x5583d6fca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.321+0000 7fa403fff700 1 -- 172.21.15.7:0/2429165387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583d6fb87a0 msgr2=0x5583d6fda6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:13.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.321+0000 7fa403fff700 1 --2- 172.21.15.7:0/2429165387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583d6fb87a0 0x5583d6fda6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.321+0000 7fa403fff700 1 -- 172.21.15.7:0/2429165387 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583d6fcb220 con 0x5583d6fcab30 2024-06-10T01:21:13.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.321+0000 7fa403fff700 1 --2- 172.21.15.7:0/2429165387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d6fcab30 0x5583d6fd8f60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4005850 tx=0x7fa3f400b9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11ecc6a45817ed17 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:13.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.321+0000 7fa3f27fc700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:21:13.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.321+0000 7fa3f27fc700 1 -- 172.21.15.7:0/2429165387 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f4003730 con 0x5583d6fcab30 2024-06-10T01:21:13.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.321+0000 7fa3f27fc700 1 -- 172.21.15.7:0/2429165387 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa3f40038d0 con 0x5583d6fcab30 2024-06-10T01:21:13.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.321+0000 7fa41b89bb80 1 -- 172.21.15.7:0/2429165387 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3f4003b60 con 0x5583d6fcab30 2024-06-10T01:21:13.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.321+0000 7fa3f27fc700 1 -- 172.21.15.7:0/2429165387 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa3f400f6c0 con 0x5583d6fcab30 2024-06-10T01:21:13.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.321+0000 7fa41b89bb80 10 client.28529 fetch_fsmap learned FSMap version 44 2024-06-10T01:21:13.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.321+0000 7fa41b89bb80 1 -- 172.21.15.7:0/2429165387 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5583d703c4b0 con 0x5583d6fcab30 2024-06-10T01:21:13.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.322+0000 7fa3f27fc700 1 -- 172.21.15.7:0/2429165387 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3f400fd40 con 0x5583d6fcab30 2024-06-10T01:21:13.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.322+0000 7fa3f27fc700 1 -- 172.21.15.7:0/2429165387 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fa3f4027040 con 0x5583d6fcab30 2024-06-10T01:21:13.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.322+0000 7fa41b89bb80 10 client.28529 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:21:13.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.322+0000 7fa41b89bb80 10 client.28529 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:21:13.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.322+0000 7fa41b89bb80 20 client.28529 populate_metadata read hostname 'smithi007' 2024-06-10T01:21:13.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.322+0000 7fa41b89bb80 1 --2- 172.21.15.7:0/2429165387 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5583d703ca50 0x5583d703ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:13.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.322+0000 7fa41b89bb80 4 client.28529 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:21:13.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.322+0000 7fa41b89bb80 1 -- 172.21.15.7:0/2429165387 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa3f4003b60 con 0x5583d703ca50 2024-06-10T01:21:13.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.322+0000 7fa40ad9d700 1 --2- 172.21.15.7:0/2429165387 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5583d703ca50 0x5583d703ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:13.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.323+0000 7fa40ad9d700 1 --2- 172.21.15.7:0/2429165387 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5583d703ca50 0x5583d703ee70 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f5e9c72fb697aaa server_cookie=62a98922462fb746 in_seq=0 out_seq=0 2024-06-10T01:21:13.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.323+0000 7fa3f27fc700 10 client.28529 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:13.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.327+0000 7fa3f27fc700 1 -- 172.21.15.7:0/2429165387 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa3f4003b60 con 0x5583d703ca50 2024-06-10T01:21:13.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.327+0000 7fa3f27fc700 10 client.28529 handle_command_reply: tid=0 2024-06-10T01:21:13.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.327+0000 7fa41b89bb80 1 client.28529 shutdown 2024-06-10T01:21:13.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.327+0000 7fa41b89bb80 1 -- 172.21.15.7:0/2429165387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d6fcab30 msgr2=0x5583d6fd8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:13.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.327+0000 7fa41b89bb80 1 --2- 172.21.15.7:0/2429165387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d6fcab30 0x5583d6fd8f60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4005850 tx=0x7fa3f400b9b0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.327+0000 7fa41b89bb80 1 -- 172.21.15.7:0/2429165387 shutdown_connections 2024-06-10T01:21:13.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.327+0000 7fa41b89bb80 1 --2- 172.21.15.7:0/2429165387 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5583d703ca50 0x5583d703ee70 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.327+0000 7fa41b89bb80 1 --2- 172.21.15.7:0/2429165387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d6fcab30 0x5583d6fd8f60 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.327+0000 7fa41b89bb80 1 --2- 172.21.15.7:0/2429165387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5583d6fb87a0 0x5583d6fda6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.327+0000 7fa41b89bb80 1 --2- 172.21.15.7:0/2429165387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583d6fbdf60 0x5583d6fca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.327+0000 7fa41b89bb80 1 -- 172.21.15.7:0/2429165387 >> 172.21.15.7:0/2429165387 conn(0x5583d6ee5080 msgr2=0x5583d6fc0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:13.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.328+0000 7fa3f27fc700 0 client.28529 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:13.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.328+0000 7fa41b89bb80 1 -- 172.21.15.7:0/2429165387 shutdown_connections 2024-06-10T01:21:13.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.328+0000 7fa41b89bb80 1 -- 172.21.15.7:0/2429165387 wait complete. 2024-06-10T01:21:13.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.328+0000 7fa41b89bb80 20 client.28529 trim_cache size 0 max 16384 2024-06-10T01:21:13.323 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:21:13.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.329+0000 7fa411018700 1 -- 172.21.15.7:0/1972408312 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa3f80a2c20 msgr2=0x7fa3f80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:13.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.329+0000 7fa411018700 1 --2- 172.21.15.7:0/1972408312 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa3f80a2c20 0x7fa3f80a50e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa4040090a0 tx=0x7fa40400cf60 comp rx=0 tx=0).stop 2024-06-10T01:21:13.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.329+0000 7fa411018700 1 -- 172.21.15.7:0/1972408312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa40c1143e0 msgr2=0x7fa40c071b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:13.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.329+0000 7fa411018700 1 --2- 172.21.15.7:0/1972408312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa40c1143e0 0x7fa40c071b80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa40400efc0 tx=0x7fa40400c3a0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.330+0000 7fa411018700 1 -- 172.21.15.7:0/1972408312 shutdown_connections 2024-06-10T01:21:13.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.330+0000 7fa411018700 1 --2- 172.21.15.7:0/1972408312 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa3f80a2c20 0x7fa3f80a50e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.330+0000 7fa411018700 1 --2- 172.21.15.7:0/1972408312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa40c1143e0 0x7fa40c071b80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.330+0000 7fa411018700 1 --2- 172.21.15.7:0/1972408312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa40c10ae30 0x7fa40c071100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.330+0000 7fa411018700 1 --2- 172.21.15.7:0/1972408312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa40c10b750 0x7fa40c071640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:13.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.330+0000 7fa411018700 1 -- 172.21.15.7:0/1972408312 >> 172.21.15.7:0/1972408312 conn(0x7fa40c06d600 msgr2=0x7fa40c10a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:13.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.330+0000 7fa411018700 1 -- 172.21.15.7:0/1972408312 shutdown_connections 2024-06-10T01:21:13.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:13.330+0000 7fa411018700 1 -- 172.21.15.7:0/1972408312 wait complete. 2024-06-10T01:21:13.502 INFO:tasks.workunit.client.0.smithi007.stdout: 10 32766 17.63 MB/sec warmup 110 sec latency 56.921 ms 2024-06-10T01:21:13.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:13 smithi078 ceph-mon[36683]: pgmap v888: 129 pgs: 129 active+clean; 405 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.1 MiB/s wr, 582 op/s 2024-06-10T01:21:13.873 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:13 smithi007 ceph-mon[32398]: pgmap v888: 129 pgs: 129 active+clean; 405 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.1 MiB/s wr, 582 op/s 2024-06-10T01:21:13.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:13 smithi175 ceph-mon[36636]: pgmap v888: 129 pgs: 129 active+clean; 405 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.1 MiB/s wr, 582 op/s 2024-06-10T01:21:14.502 INFO:tasks.workunit.client.0.smithi007.stdout: 10 33585 17.92 MB/sec warmup 111 sec latency 62.867 ms 2024-06-10T01:21:15.068 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:21:15.069 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:21:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:21:15.502 INFO:tasks.workunit.client.0.smithi007.stdout: 10 34334 18.03 MB/sec warmup 112 sec latency 64.493 ms 2024-06-10T01:21:15.799 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:15 smithi007 ceph-mon[32398]: pgmap v889: 129 pgs: 129 active+clean; 413 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.4 MiB/s wr, 590 op/s 2024-06-10T01:21:15.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:15 smithi078 ceph-mon[36683]: pgmap v889: 129 pgs: 129 active+clean; 413 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.4 MiB/s wr, 590 op/s 2024-06-10T01:21:15.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:15 smithi175 ceph-mon[36636]: pgmap v889: 129 pgs: 129 active+clean; 413 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.4 MiB/s wr, 590 op/s 2024-06-10T01:21:16.502 INFO:tasks.workunit.client.0.smithi007.stdout: 10 35198 18.32 MB/sec warmup 113 sec latency 66.828 ms 2024-06-10T01:21:17.502 INFO:tasks.workunit.client.0.smithi007.stdout: 10 35991 18.51 MB/sec warmup 114 sec latency 76.703 ms 2024-06-10T01:21:17.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.738+0000 7fca3759e700 1 -- 172.21.15.7:0/2869669874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca38107790 msgr2=0x7fca3810f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:17.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.738+0000 7fca3759e700 1 --2- 172.21.15.7:0/2869669874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca38107790 0x7fca3810f340 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fca30009f90 tx=0x7fca3000b330 comp rx=0 tx=0).stop 2024-06-10T01:21:17.732 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:17 smithi007 ceph-mon[32398]: pgmap v890: 129 pgs: 129 active+clean; 418 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.9 MiB/s wr, 551 op/s 2024-06-10T01:21:17.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.738+0000 7fca3759e700 1 -- 172.21.15.7:0/2869669874 shutdown_connections 2024-06-10T01:21:17.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.738+0000 7fca3759e700 1 --2- 172.21.15.7:0/2869669874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca38107790 0x7fca3810f340 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:17.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.738+0000 7fca3759e700 1 --2- 172.21.15.7:0/2869669874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca38100f20 0x7fca38101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:17.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.738+0000 7fca3759e700 1 --2- 172.21.15.7:0/2869669874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca381018d0 0x7fca381070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:17.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.738+0000 7fca3759e700 1 -- 172.21.15.7:0/2869669874 >> 172.21.15.7:0/2869669874 conn(0x7fca38076640 msgr2=0x7fca38076a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:17.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.739+0000 7fca3759e700 1 -- 172.21.15.7:0/2869669874 shutdown_connections 2024-06-10T01:21:17.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.740+0000 7fca3759e700 1 -- 172.21.15.7:0/2869669874 wait complete. 2024-06-10T01:21:17.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.741+0000 7fca3759e700 1 Processor -- start 2024-06-10T01:21:17.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.741+0000 7fca3759e700 1 -- start start 2024-06-10T01:21:17.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.741+0000 7fca3759e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca38100f20 0x7fca38101ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:17.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.741+0000 7fca3759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca381018d0 0x7fca381023e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:17.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.741+0000 7fca3759e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca38107790 0x7fca38102920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:17.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.741+0000 7fca3759e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fca38112790 con 0x7fca381018d0 2024-06-10T01:21:17.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.741+0000 7fca3759e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fca38112610 con 0x7fca38107790 2024-06-10T01:21:17.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.741+0000 7fca3759e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fca38112910 con 0x7fca38100f20 2024-06-10T01:21:17.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.741+0000 7fca35d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca381018d0 0x7fca381023e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:17.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.741+0000 7fca35d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca381018d0 0x7fca381023e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45600/0 (socket says 172.21.15.7:45600) 2024-06-10T01:21:17.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.741+0000 7fca35d9b700 1 -- 172.21.15.7:0/913736819 learned_addr learned my addr 172.21.15.7:0/913736819 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:17.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.741+0000 7fca36d9d700 1 --2- 172.21.15.7:0/913736819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca38107790 0x7fca38102920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:17.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.741+0000 7fca3659c700 1 --2- 172.21.15.7:0/913736819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca38100f20 0x7fca38101ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:17.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.742+0000 7fca35d9b700 1 -- 172.21.15.7:0/913736819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca38100f20 msgr2=0x7fca38101ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:17.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.742+0000 7fca35d9b700 1 --2- 172.21.15.7:0/913736819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca38100f20 0x7fca38101ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:17.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.742+0000 7fca35d9b700 1 -- 172.21.15.7:0/913736819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca38107790 msgr2=0x7fca38102920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:17.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.742+0000 7fca35d9b700 1 --2- 172.21.15.7:0/913736819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca38107790 0x7fca38102920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:17.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.742+0000 7fca35d9b700 1 -- 172.21.15.7:0/913736819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca381a7000 con 0x7fca381018d0 2024-06-10T01:21:17.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.742+0000 7fca36d9d700 1 --2- 172.21.15.7:0/913736819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca38107790 0x7fca38102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:21:17.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.742+0000 7fca35d9b700 1 --2- 172.21.15.7:0/913736819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca381018d0 0x7fca381023e0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fca2c00e970 tx=0x7fca2c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:17.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.742+0000 7fca277fe700 1 -- 172.21.15.7:0/913736819 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c004a30 con 0x7fca381018d0 2024-06-10T01:21:17.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.742+0000 7fca3759e700 1 -- 172.21.15.7:0/913736819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca381a72f0 con 0x7fca381018d0 2024-06-10T01:21:17.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.742+0000 7fca3759e700 1 -- 172.21.15.7:0/913736819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca381a7880 con 0x7fca381018d0 2024-06-10T01:21:17.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.742+0000 7fca277fe700 1 -- 172.21.15.7:0/913736819 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca2c00ccc0 con 0x7fca381018d0 2024-06-10T01:21:17.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.743+0000 7fca277fe700 1 -- 172.21.15.7:0/913736819 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c005520 con 0x7fca381018d0 2024-06-10T01:21:17.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.745+0000 7fca277fe700 1 -- 172.21.15.7:0/913736819 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fca2c021440 con 0x7fca381018d0 2024-06-10T01:21:17.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.746+0000 7fca277fe700 1 --2- 172.21.15.7:0/913736819 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fca200a2c20 0x7fca200a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:17.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.747+0000 7fca3659c700 1 --2- 172.21.15.7:0/913736819 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fca200a2c20 0x7fca200a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:17.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.747+0000 7fca277fe700 1 -- 172.21.15.7:0/913736819 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fca2c0cf560 con 0x7fca381018d0 2024-06-10T01:21:17.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.747+0000 7fca471dab80 1 Processor -- start 2024-06-10T01:21:17.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.747+0000 7fca471dab80 1 -- start start 2024-06-10T01:21:17.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.747+0000 7fca471dab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b44b9120 0x5612b44b9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:17.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.747+0000 7fca3659c700 1 --2- 172.21.15.7:0/913736819 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fca200a2c20 0x7fca200a50e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fca28009750 tx=0x7fca28009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:17.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.747+0000 7fca471dab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b44b9a40 0x5612b44b4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:17.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.747+0000 7fca471dab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612b44b4a70 0x5612b44b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:17.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.747+0000 7fca471dab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5612b43a6fe0 con 0x5612b44b9a40 2024-06-10T01:21:17.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.747+0000 7fca471dab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5612b44058d0 con 0x5612b44b9120 2024-06-10T01:21:17.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.747+0000 7fca471dab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5612b436d4e0 con 0x5612b44b4a70 2024-06-10T01:21:17.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.747+0000 7fca3659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b44b9120 0x5612b44b9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:17.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.747+0000 7fca3659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b44b9120 0x5612b44b9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:35210/0 (socket says 172.21.15.7:35210) 2024-06-10T01:21:17.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.747+0000 7fca3659c700 1 -- 172.21.15.7:0/781837481 learned_addr learned my addr 172.21.15.7:0/781837481 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:17.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.748+0000 7fca36d9d700 1 --2- 172.21.15.7:0/781837481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612b44b4a70 0x5612b44b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:17.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.748+0000 7fca35d9b700 1 --2- 172.21.15.7:0/781837481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b44b9a40 0x5612b44b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:17.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.748+0000 7fca3659c700 1 -- 172.21.15.7:0/781837481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612b44b4a70 msgr2=0x5612b44b4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:17.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.748+0000 7fca3659c700 1 --2- 172.21.15.7:0/781837481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612b44b4a70 0x5612b44b4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:17.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.748+0000 7fca3659c700 1 -- 172.21.15.7:0/781837481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b44b9a40 msgr2=0x5612b44b4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:17.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.748+0000 7fca3659c700 1 --2- 172.21.15.7:0/781837481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b44b9a40 0x5612b44b4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:17.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.748+0000 7fca3659c700 1 -- 172.21.15.7:0/781837481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612b4354950 con 0x5612b44b9120 2024-06-10T01:21:17.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.748+0000 7fca3659c700 1 --2- 172.21.15.7:0/781837481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b44b9120 0x5612b44b9500 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fca2800e560 tx=0x7fca28014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3e7efd110c70153 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:17.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.748+0000 7fca24ff9700 1 -- 172.21.15.7:0/781837481 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca28026430 con 0x5612b44b9120 2024-06-10T01:21:17.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.748+0000 7fca24ff9700 1 -- 172.21.15.7:0/781837481 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca2801c8f0 con 0x5612b44b9120 2024-06-10T01:21:17.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.749+0000 7fca24ff9700 1 -- 172.21.15.7:0/781837481 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2801d9f0 con 0x5612b44b9120 2024-06-10T01:21:17.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.749+0000 7fca471dab80 1 -- 172.21.15.7:0/781837481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b44b9120 msgr2=0x5612b44b9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:17.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.749+0000 7fca471dab80 1 --2- 172.21.15.7:0/781837481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b44b9120 0x5612b44b9500 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fca2800e560 tx=0x7fca28014000 comp rx=0 tx=0).stop 2024-06-10T01:21:17.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.750+0000 7fca471dab80 1 -- 172.21.15.7:0/781837481 shutdown_connections 2024-06-10T01:21:17.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.750+0000 7fca471dab80 1 --2- 172.21.15.7:0/781837481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612b44b4a70 0x5612b44b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:17.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.750+0000 7fca471dab80 1 --2- 172.21.15.7:0/781837481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b44b9120 0x5612b44b9500 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:17.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.750+0000 7fca471dab80 1 --2- 172.21.15.7:0/781837481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b44b9a40 0x5612b44b4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:17.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.750+0000 7fca471dab80 1 -- 172.21.15.7:0/781837481 >> 172.21.15.7:0/781837481 conn(0x5612b43e1100 msgr2=0x5612b44af410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:17.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.750+0000 7fca471dab80 1 -- 172.21.15.7:0/781837481 shutdown_connections 2024-06-10T01:21:17.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.750+0000 7fca471dab80 1 -- 172.21.15.7:0/781837481 wait complete. 2024-06-10T01:21:17.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.751+0000 7fca471dab80 1 -- start start 2024-06-10T01:21:17.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.751+0000 7fca471dab80 1 Processor -- start 2024-06-10T01:21:17.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.752+0000 7fca471dab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b44b4a70 0x5612b44c0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:17.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.752+0000 7fca471dab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612b44b9120 0x5612b44bdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:17.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.752+0000 7fca471dab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b44b9a40 0x5612b44be000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:17.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.752+0000 7fca3659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b44b4a70 0x5612b44c0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:17.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.752+0000 7fca3659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b44b4a70 0x5612b44c0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45628/0 (socket says 172.21.15.7:45628) 2024-06-10T01:21:17.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.752+0000 7fca35d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612b44b9120 0x5612b44bdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:17.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.753+0000 7fca35d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612b44b9120 0x5612b44bdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:53302/0 (socket says 172.21.15.7:53302) 2024-06-10T01:21:17.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.753+0000 7fca471dab80 1 -- 172.21.15.7:0/2103573731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5612b43a6040 con 0x5612b44b4a70 2024-06-10T01:21:17.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.753+0000 7fca3659c700 1 -- 172.21.15.7:0/2103573731 learned_addr learned my addr 172.21.15.7:0/2103573731 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:17.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.753+0000 7fca471dab80 1 -- 172.21.15.7:0/2103573731 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5612b4401890 con 0x5612b44b9a40 2024-06-10T01:21:17.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.753+0000 7fca471dab80 1 -- 172.21.15.7:0/2103573731 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5612b44c14d0 con 0x5612b44b9120 2024-06-10T01:21:17.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.753+0000 7fca471dab80 10 client.0 _ll_register_callbacks cb 0x5612b439b630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:21:17.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.753+0000 7fca36d9d700 1 --2- 172.21.15.7:0/2103573731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b44b9a40 0x5612b44be000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:17.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.753+0000 7fca3659c700 1 -- 172.21.15.7:0/2103573731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612b44b9120 msgr2=0x5612b44bdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:17.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.753+0000 7fca3659c700 1 --2- 172.21.15.7:0/2103573731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612b44b9120 0x5612b44bdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:17.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.753+0000 7fca3659c700 1 -- 172.21.15.7:0/2103573731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b44b9a40 msgr2=0x5612b44be000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:17.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.753+0000 7fca3659c700 1 --2- 172.21.15.7:0/2103573731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b44b9a40 0x5612b44be000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:17.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.753+0000 7fca3659c700 1 -- 172.21.15.7:0/2103573731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612b43cb7d0 con 0x5612b44b4a70 2024-06-10T01:21:17.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.753+0000 7fca35d9b700 1 --2- 172.21.15.7:0/2103573731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612b44b9120 0x5612b44bdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:21:17.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.754+0000 7fca3659c700 1 --2- 172.21.15.7:0/2103573731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b44b4a70 0x5612b44c0770 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fca28009860 tx=0x7fca2800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965df9710dd15372 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:17.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.754+0000 7fca1dffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:21:17.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.754+0000 7fca1dffb700 1 -- 172.21.15.7:0/2103573731 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca28034060 con 0x5612b44b4a70 2024-06-10T01:21:17.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.754+0000 7fca1dffb700 1 -- 172.21.15.7:0/2103573731 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fca2801caa0 con 0x5612b44b4a70 2024-06-10T01:21:17.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.754+0000 7fca471dab80 1 -- 172.21.15.7:0/2103573731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca2801c8f0 con 0x5612b44b4a70 2024-06-10T01:21:17.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.754+0000 7fca1dffb700 1 -- 172.21.15.7:0/2103573731 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fca2803b050 con 0x5612b44b4a70 2024-06-10T01:21:17.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.754+0000 7fca471dab80 10 client.19290 fetch_fsmap learned FSMap version 44 2024-06-10T01:21:17.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.754+0000 7fca471dab80 1 -- 172.21.15.7:0/2103573731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5612b44d4350 con 0x5612b44b4a70 2024-06-10T01:21:17.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.754+0000 7fca1dffb700 1 -- 172.21.15.7:0/2103573731 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca2801c8f0 con 0x5612b44b4a70 2024-06-10T01:21:17.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.755+0000 7fca1dffb700 1 -- 172.21.15.7:0/2103573731 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fca28004890 con 0x5612b44b4a70 2024-06-10T01:21:17.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.755+0000 7fca471dab80 10 client.19290 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:21:17.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.755+0000 7fca471dab80 10 client.19290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:21:17.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.755+0000 7fca471dab80 20 client.19290 populate_metadata read hostname 'smithi007' 2024-06-10T01:21:17.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.755+0000 7fca471dab80 1 --2- 172.21.15.7:0/2103573731 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5612b44d6820 0x5612b44d8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:17.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.755+0000 7fca471dab80 4 client.19290 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:21:17.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.755+0000 7fca471dab80 1 -- 172.21.15.7:0/2103573731 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612b4412dd0 con 0x5612b44d6820 2024-06-10T01:21:17.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.755+0000 7fca35d9b700 1 --2- 172.21.15.7:0/2103573731 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5612b44d6820 0x5612b44d8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:17.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.756+0000 7fca35d9b700 1 --2- 172.21.15.7:0/2103573731 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5612b44d6820 0x5612b44d8c10 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f56f41f960ef8c83 server_cookie=d7e0068e0a228d8 in_seq=0 out_seq=0 2024-06-10T01:21:17.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.756+0000 7fca1dffb700 10 client.19290 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:17.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.759+0000 7fca1dffb700 1 -- 172.21.15.7:0/2103573731 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5612b4412dd0 con 0x5612b44d6820 2024-06-10T01:21:17.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.759+0000 7fca1dffb700 10 client.19290 handle_command_reply: tid=0 2024-06-10T01:21:17.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.759+0000 7fca471dab80 1 client.19290 shutdown 2024-06-10T01:21:17.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.759+0000 7fca471dab80 1 -- 172.21.15.7:0/2103573731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b44b4a70 msgr2=0x5612b44c0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:17.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.759+0000 7fca471dab80 1 --2- 172.21.15.7:0/2103573731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b44b4a70 0x5612b44c0770 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fca28009860 tx=0x7fca2800ee00 comp rx=0 tx=0).stop 2024-06-10T01:21:17.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.759+0000 7fca471dab80 1 -- 172.21.15.7:0/2103573731 shutdown_connections 2024-06-10T01:21:17.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.759+0000 7fca471dab80 1 --2- 172.21.15.7:0/2103573731 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5612b44d6820 0x5612b44d8c10 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:17.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.759+0000 7fca471dab80 1 --2- 172.21.15.7:0/2103573731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b44b9a40 0x5612b44be000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:17.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.759+0000 7fca471dab80 1 --2- 172.21.15.7:0/2103573731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b44b4a70 0x5612b44c0770 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:17.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.759+0000 7fca1dffb700 0 client.19290 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:17.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.759+0000 7fca471dab80 1 --2- 172.21.15.7:0/2103573731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612b44b9120 0x5612b44bdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:17.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.759+0000 7fca471dab80 1 -- 172.21.15.7:0/2103573731 >> 172.21.15.7:0/2103573731 conn(0x5612b43e1100 msgr2=0x5612b44b4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:17.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.760+0000 7fca471dab80 1 -- 172.21.15.7:0/2103573731 shutdown_connections 2024-06-10T01:21:17.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.760+0000 7fca471dab80 1 -- 172.21.15.7:0/2103573731 wait complete. 2024-06-10T01:21:17.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.760+0000 7fca471dab80 20 client.19290 trim_cache size 0 max 16384 2024-06-10T01:21:17.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.788+0000 7fca471dab80 1 Processor -- start 2024-06-10T01:21:17.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.788+0000 7fca471dab80 1 -- start start 2024-06-10T01:21:17.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.788+0000 7fca471dab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612b44b4a70 0x5612b44af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:17.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.788+0000 7fca471dab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b44b9120 0x5612b44c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:17.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.788+0000 7fca471dab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b44b9a40 0x5612b44ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:17.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.788+0000 7fca471dab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fca180113d0 con 0x5612b44b9120 2024-06-10T01:21:17.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.788+0000 7fca471dab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fca1800ebd0 con 0x5612b44b9a40 2024-06-10T01:21:17.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.788+0000 7fca471dab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fca1800ca20 con 0x5612b44b4a70 2024-06-10T01:21:17.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.789+0000 7fca3659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612b44b4a70 0x5612b44af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:17.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.789+0000 7fca3659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612b44b4a70 0x5612b44af150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:53318/0 (socket says 172.21.15.7:53318) 2024-06-10T01:21:17.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.789+0000 7fca3659c700 1 -- 172.21.15.7:0/991288359 learned_addr learned my addr 172.21.15.7:0/991288359 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:17.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.789+0000 7fca36d9d700 1 --2- 172.21.15.7:0/991288359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b44b9a40 0x5612b44ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:17.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.789+0000 7fca35d9b700 1 --2- 172.21.15.7:0/991288359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b44b9120 0x5612b44c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:17.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.790+0000 7fca35d9b700 1 -- 172.21.15.7:0/991288359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612b44b4a70 msgr2=0x5612b44af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:17.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.790+0000 7fca35d9b700 1 --2- 172.21.15.7:0/991288359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612b44b4a70 0x5612b44af150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:17.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.790+0000 7fca35d9b700 1 -- 172.21.15.7:0/991288359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b44b9a40 msgr2=0x5612b44ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:17.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.790+0000 7fca35d9b700 1 --2- 172.21.15.7:0/991288359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b44b9a40 0x5612b44ca2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:17.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.790+0000 7fca35d9b700 1 -- 172.21.15.7:0/991288359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612b44bc030 con 0x5612b44b9120 2024-06-10T01:21:17.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.791+0000 7fca35d9b700 1 --2- 172.21.15.7:0/991288359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b44b9120 0x5612b44c9d80 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fca2c005090 tx=0x7fca2c09f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876e5e8b204acba4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:17.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.791+0000 7fca1ffff700 1 -- 172.21.15.7:0/991288359 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c065020 con 0x5612b44b9120 2024-06-10T01:21:17.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.791+0000 7fca1ffff700 1 -- 172.21.15.7:0/991288359 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca2c09d140 con 0x5612b44b9120 2024-06-10T01:21:17.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.792+0000 7fca1ffff700 1 -- 172.21.15.7:0/991288359 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c0cf1c0 con 0x5612b44b9120 2024-06-10T01:21:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.792+0000 7fca471dab80 1 -- 172.21.15.7:0/991288359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b44b9120 msgr2=0x5612b44c9d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.792+0000 7fca471dab80 1 --2- 172.21.15.7:0/991288359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b44b9120 0x5612b44c9d80 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fca2c005090 tx=0x7fca2c09f040 comp rx=0 tx=0).stop 2024-06-10T01:21:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.792+0000 7fca471dab80 1 -- 172.21.15.7:0/991288359 shutdown_connections 2024-06-10T01:21:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.792+0000 7fca471dab80 1 --2- 172.21.15.7:0/991288359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b44b9a40 0x5612b44ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.792+0000 7fca471dab80 1 --2- 172.21.15.7:0/991288359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612b44b4a70 0x5612b44af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.792+0000 7fca471dab80 1 --2- 172.21.15.7:0/991288359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b44b9120 0x5612b44c9d80 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.792+0000 7fca471dab80 1 -- 172.21.15.7:0/991288359 >> 172.21.15.7:0/991288359 conn(0x5612b43e1100 msgr2=0x5612b44c9670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.792+0000 7fca471dab80 1 -- 172.21.15.7:0/991288359 shutdown_connections 2024-06-10T01:21:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.793+0000 7fca471dab80 1 -- 172.21.15.7:0/991288359 wait complete. 2024-06-10T01:21:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.793+0000 7fca471dab80 1 -- start start 2024-06-10T01:21:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.793+0000 7fca471dab80 1 Processor -- start 2024-06-10T01:21:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.794+0000 7fca471dab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b44b4a70 0x5612b44c6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.794+0000 7fca471dab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612b44b9120 0x5612b44c6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.794+0000 7fca471dab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b44b9a40 0x5612b44b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.794+0000 7fca471dab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fca1800d090 con 0x5612b44b9a40 2024-06-10T01:21:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.794+0000 7fca471dab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fca1800e500 con 0x5612b44b4a70 2024-06-10T01:21:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.794+0000 7fca471dab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fca1800d760 con 0x5612b44b9120 2024-06-10T01:21:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.794+0000 7fca471dab80 10 client.0 _ll_register_callbacks cb 0x5612b439b630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:21:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.794+0000 7fca3659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b44b4a70 0x5612b44c6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.794+0000 7fca3659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b44b4a70 0x5612b44c6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:35242/0 (socket says 172.21.15.7:35242) 2024-06-10T01:21:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.794+0000 7fca3659c700 1 -- 172.21.15.7:0/241278717 learned_addr learned my addr 172.21.15.7:0/241278717 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.794+0000 7fca35d9b700 1 --2- 172.21.15.7:0/241278717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612b44b9120 0x5612b44c6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.794+0000 7fca36d9d700 1 --2- 172.21.15.7:0/241278717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b44b9a40 0x5612b44b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.795+0000 7fca3659c700 1 -- 172.21.15.7:0/241278717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612b44b9120 msgr2=0x5612b44c6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.795+0000 7fca3659c700 1 --2- 172.21.15.7:0/241278717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612b44b9120 0x5612b44c6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.795+0000 7fca3659c700 1 -- 172.21.15.7:0/241278717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b44b9a40 msgr2=0x5612b44b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.795+0000 7fca3659c700 1 --2- 172.21.15.7:0/241278717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b44b9a40 0x5612b44b9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.795+0000 7fca3659c700 1 -- 172.21.15.7:0/241278717 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612b44ba660 con 0x5612b44b4a70 2024-06-10T01:21:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.795+0000 7fca3659c700 1 --2- 172.21.15.7:0/241278717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b44b4a70 0x5612b44c6a40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fca28009b50 tx=0x7fca28009400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b03445572e9a5214 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.795+0000 7fca1dffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:21:17.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.795+0000 7fca1dffb700 1 -- 172.21.15.7:0/241278717 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca28034070 con 0x5612b44b4a70 2024-06-10T01:21:17.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.795+0000 7fca471dab80 1 -- 172.21.15.7:0/241278717 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca2c09d140 con 0x5612b44b4a70 2024-06-10T01:21:17.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.795+0000 7fca1dffb700 1 -- 172.21.15.7:0/241278717 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fca280049a0 con 0x5612b44b4a70 2024-06-10T01:21:17.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.796+0000 7fca1dffb700 1 -- 172.21.15.7:0/241278717 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fca2801c760 con 0x5612b44b4a70 2024-06-10T01:21:17.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.796+0000 7fca471dab80 10 client.28547 fetch_fsmap learned FSMap version 44 2024-06-10T01:21:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.796+0000 7fca471dab80 1 -- 172.21.15.7:0/241278717 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5612b44bb9a0 con 0x5612b44b4a70 2024-06-10T01:21:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.797+0000 7fca1dffb700 1 -- 172.21.15.7:0/241278717 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca2c09d140 con 0x5612b44b4a70 2024-06-10T01:21:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.797+0000 7fca1dffb700 1 -- 172.21.15.7:0/241278717 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fca28005420 con 0x5612b44b4a70 2024-06-10T01:21:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.797+0000 7fca471dab80 10 client.28547 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:21:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.797+0000 7fca471dab80 10 client.28547 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:21:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.797+0000 7fca471dab80 20 client.28547 populate_metadata read hostname 'smithi007' 2024-06-10T01:21:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.797+0000 7fca471dab80 1 --2- 172.21.15.7:0/241278717 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5612b4542680 0x5612b4544aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.797+0000 7fca471dab80 4 client.28547 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:21:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.797+0000 7fca471dab80 1 -- 172.21.15.7:0/241278717 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5612b440d350 con 0x5612b4542680 2024-06-10T01:21:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.798+0000 7fca35d9b700 1 --2- 172.21.15.7:0/241278717 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5612b4542680 0x5612b4544aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.800+0000 7fca35d9b700 1 --2- 172.21.15.7:0/241278717 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5612b4542680 0x5612b4544aa0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5bd9da4107e4b9c server_cookie=fdb19811b62b0191 in_seq=0 out_seq=0 2024-06-10T01:21:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.800+0000 7fca1dffb700 10 client.28547 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.805+0000 7fca1dffb700 1 -- 172.21.15.7:0/241278717 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5612b440d350 con 0x5612b4542680 2024-06-10T01:21:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.806+0000 7fca1dffb700 10 client.28547 handle_command_reply: tid=0 2024-06-10T01:21:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.806+0000 7fca471dab80 1 client.28547 shutdown 2024-06-10T01:21:17.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.809+0000 7fca471dab80 1 -- 172.21.15.7:0/241278717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b44b4a70 msgr2=0x5612b44c6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:17.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.809+0000 7fca471dab80 1 --2- 172.21.15.7:0/241278717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b44b4a70 0x5612b44c6a40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fca28009b50 tx=0x7fca28009400 comp rx=0 tx=0).stop 2024-06-10T01:21:17.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.809+0000 7fca471dab80 1 -- 172.21.15.7:0/241278717 shutdown_connections 2024-06-10T01:21:17.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.809+0000 7fca471dab80 1 --2- 172.21.15.7:0/241278717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b44b9a40 0x5612b44b9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:17.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.809+0000 7fca471dab80 1 --2- 172.21.15.7:0/241278717 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5612b4542680 0x5612b4544aa0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:17.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.809+0000 7fca471dab80 1 --2- 172.21.15.7:0/241278717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b44b4a70 0x5612b44c6a40 secure :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fca28009b50 tx=0x7fca28009400 comp rx=0 tx=0).stop 2024-06-10T01:21:17.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.809+0000 7fca471dab80 1 --2- 172.21.15.7:0/241278717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612b44b9120 0x5612b44c6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:17.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.809+0000 7fca471dab80 1 -- 172.21.15.7:0/241278717 >> 172.21.15.7:0/241278717 conn(0x5612b43e1100 msgr2=0x5612b44c9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:17.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.809+0000 7fca1dffb700 0 client.28547 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:17.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.812+0000 7fca471dab80 1 -- 172.21.15.7:0/241278717 shutdown_connections 2024-06-10T01:21:17.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.812+0000 7fca471dab80 1 -- 172.21.15.7:0/241278717 wait complete. 2024-06-10T01:21:17.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.812+0000 7fca471dab80 20 client.28547 trim_cache size 0 max 16384 2024-06-10T01:21:17.806 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:21:17.806 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-10T01:21:17.806 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "9830b421-b118-4340-9f90-b40db12d236a", 2024-06-10T01:21:17.806 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-10T01:21:17.806 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:21:17.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.814+0000 7fca3759e700 1 -- 172.21.15.7:0/913736819 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fca200a2c20 msgr2=0x7fca200a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:17.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.814+0000 7fca3759e700 1 --2- 172.21.15.7:0/913736819 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fca200a2c20 0x7fca200a50e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fca28009750 tx=0x7fca28009310 comp rx=0 tx=0).stop 2024-06-10T01:21:17.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.814+0000 7fca3759e700 1 -- 172.21.15.7:0/913736819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca381018d0 msgr2=0x7fca381023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:17.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.814+0000 7fca3759e700 1 --2- 172.21.15.7:0/913736819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca381018d0 0x7fca381023e0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fca2c00e970 tx=0x7fca2c00ec80 comp rx=0 tx=0).stop 2024-06-10T01:21:17.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.814+0000 7fca3759e700 1 -- 172.21.15.7:0/913736819 shutdown_connections 2024-06-10T01:21:17.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.814+0000 7fca3759e700 1 --2- 172.21.15.7:0/913736819 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fca200a2c20 0x7fca200a50e0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:17.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.814+0000 7fca3759e700 1 --2- 172.21.15.7:0/913736819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca38107790 0x7fca38102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:17.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.814+0000 7fca3759e700 1 --2- 172.21.15.7:0/913736819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca38100f20 0x7fca38101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:17.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.814+0000 7fca3759e700 1 --2- 172.21.15.7:0/913736819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca381018d0 0x7fca381023e0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:17.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.814+0000 7fca3759e700 1 -- 172.21.15.7:0/913736819 >> 172.21.15.7:0/913736819 conn(0x7fca38076640 msgr2=0x7fca380febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:17.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.814+0000 7fca3759e700 1 -- 172.21.15.7:0/913736819 shutdown_connections 2024-06-10T01:21:17.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:17.814+0000 7fca3759e700 1 -- 172.21.15.7:0/913736819 wait complete. 2024-06-10T01:21:17.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:17 smithi078 ceph-mon[36683]: pgmap v890: 129 pgs: 129 active+clean; 418 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.9 MiB/s wr, 551 op/s 2024-06-10T01:21:17.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:17 smithi175 ceph-mon[36636]: pgmap v890: 129 pgs: 129 active+clean; 418 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.9 MiB/s wr, 551 op/s 2024-06-10T01:21:18.502 INFO:tasks.workunit.client.0.smithi007.stdout: 10 36792 18.74 MB/sec warmup 115 sec latency 81.514 ms 2024-06-10T01:21:18.585 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:21:18.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:18 smithi078 ceph-mon[36683]: scrub queued for path: / 2024-06-10T01:21:18.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:18 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:21:18.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:18 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:21:18.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:18 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:21:18.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:18 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:21:18.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:18 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:21:18.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:18 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:21:18.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:18 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:21:18.947 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:21:18.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:18 smithi175 ceph-mon[36636]: scrub queued for path: / 2024-06-10T01:21:18.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:18 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:21:18.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:18 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:21:18.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:18 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:21:18.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:18 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:21:18.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:18 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:21:18.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:18 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:21:18.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:18 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:21:19.012 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:18 smithi007 ceph-mon[32398]: scrub queued for path: / 2024-06-10T01:21:19.012 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:18 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:21:19.012 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:18 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:21:19.012 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:18 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:21:19.012 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:18 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:21:19.012 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:18 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:21:19.013 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:18 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:21:19.013 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:18 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:21:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.416+0000 7fc3070ba640 1 -- 172.21.15.7:0/2790197590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3001024b0 msgr2=0x7fc300108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.416+0000 7fc3070ba640 1 --2- 172.21.15.7:0/2790197590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3001024b0 0x7fc300108d40 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009ca0 tx=0x7fc2f802a8c0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.416+0000 7fc3070ba640 1 -- 172.21.15.7:0/2790197590 shutdown_connections 2024-06-10T01:21:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.416+0000 7fc3070ba640 1 --2- 172.21.15.7:0/2790197590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3001024b0 0x7fc300108d40 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.416+0000 7fc3070ba640 1 --2- 172.21.15.7:0/2790197590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc300101af0 0x7fc300101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.416+0000 7fc3070ba640 1 --2- 172.21.15.7:0/2790197590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc300100900 0x7fc300100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.416+0000 7fc3070ba640 1 -- 172.21.15.7:0/2790197590 >> 172.21.15.7:0/2790197590 conn(0x7fc3000fbe70 msgr2=0x7fc3000fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.416+0000 7fc3070ba640 1 -- 172.21.15.7:0/2790197590 shutdown_connections 2024-06-10T01:21:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.416+0000 7fc3070ba640 1 -- 172.21.15.7:0/2790197590 wait complete. 2024-06-10T01:21:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.417+0000 7fc3070ba640 1 Processor -- start 2024-06-10T01:21:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.417+0000 7fc3070ba640 1 -- start start 2024-06-10T01:21:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.417+0000 7fc3070ba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc300100900 0x7fc300104890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.417+0000 7fc3070ba640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc300101af0 0x7fc300102ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.417+0000 7fc3070ba640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3001024b0 0x7fc300103410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.417+0000 7fc3070ba640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc30010c580 con 0x7fc3001024b0 2024-06-10T01:21:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.417+0000 7fc3070ba640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc30010c400 con 0x7fc300100900 2024-06-10T01:21:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.417+0000 7fc3070ba640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc30010c700 con 0x7fc300101af0 2024-06-10T01:21:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.417+0000 7fc304e56640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc300100900 0x7fc300104890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.417+0000 7fc305657640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3001024b0 0x7fc300103410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.417+0000 7fc304e56640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc300100900 0x7fc300104890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:35264/0 (socket says 172.21.15.7:35264) 2024-06-10T01:21:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.417+0000 7fc304e56640 1 -- 172.21.15.7:0/3105526122 learned_addr learned my addr 172.21.15.7:0/3105526122 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.417+0000 7fc2fffff640 1 --2- 172.21.15.7:0/3105526122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc300101af0 0x7fc300102ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.417+0000 7fc305657640 1 -- 172.21.15.7:0/3105526122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc300101af0 msgr2=0x7fc300102ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.417+0000 7fc305657640 1 --2- 172.21.15.7:0/3105526122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc300101af0 0x7fc300102ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.417+0000 7fc305657640 1 -- 172.21.15.7:0/3105526122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc300100900 msgr2=0x7fc300104890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.417+0000 7fc305657640 1 --2- 172.21.15.7:0/3105526122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc300100900 0x7fc300104890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.417+0000 7fc305657640 1 -- 172.21.15.7:0/3105526122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc300103ca0 con 0x7fc3001024b0 2024-06-10T01:21:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.417+0000 7fc304e56640 1 --2- 172.21.15.7:0/3105526122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc300100900 0x7fc300104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:21:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.417+0000 7fc2fffff640 1 --2- 172.21.15.7:0/3105526122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc300101af0 0x7fc300102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:21:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.418+0000 7fc305657640 1 --2- 172.21.15.7:0/3105526122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3001024b0 0x7fc300103410 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8031ae0 tx=0x7fc2f8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.418+0000 7fc2fdffb640 1 -- 172.21.15.7:0/3105526122 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8003b70 con 0x7fc3001024b0 2024-06-10T01:21:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.418+0000 7fc2fdffb640 1 -- 172.21.15.7:0/3105526122 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f8003d10 con 0x7fc3001024b0 2024-06-10T01:21:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.418+0000 7fc3070ba640 1 -- 172.21.15.7:0/3105526122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3001a1e30 con 0x7fc3001024b0 2024-06-10T01:21:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.418+0000 7fc2fdffb640 1 -- 172.21.15.7:0/3105526122 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f802fc40 con 0x7fc3001024b0 2024-06-10T01:21:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.418+0000 7fc3070ba640 1 -- 172.21.15.7:0/3105526122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3001a2290 con 0x7fc3001024b0 2024-06-10T01:21:19.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.421+0000 7fc309dc7740 1 Processor -- start 2024-06-10T01:21:19.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.421+0000 7fc2fdffb640 1 -- 172.21.15.7:0/3105526122 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fc2f8004050 con 0x7fc3001024b0 2024-06-10T01:21:19.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.421+0000 7fc309dc7740 1 -- start start 2024-06-10T01:21:19.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.421+0000 7fc309dc7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641a3be6010 0x5641a3be6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:19.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.421+0000 7fc309dc7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641a3be6a00 0x5641a3beb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:19.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.421+0000 7fc309dc7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641a3bebb70 0x5641a3bee000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.421+0000 7fc309dc7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5641a3b494f0 con 0x5641a3be6010 2024-06-10T01:21:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.421+0000 7fc309dc7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5641a3ae7870 con 0x5641a3bebb70 2024-06-10T01:21:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.421+0000 7fc309dc7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5641a3b454b0 con 0x5641a3be6a00 2024-06-10T01:21:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.422+0000 7fc2fffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641a3be6010 0x5641a3be6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.422+0000 7fc2fffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641a3be6010 0x5641a3be6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45692/0 (socket says 172.21.15.7:45692) 2024-06-10T01:21:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.422+0000 7fc2fffff640 1 -- 172.21.15.7:0/1964242209 learned_addr learned my addr 172.21.15.7:0/1964242209 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.422+0000 7fc304e56640 1 --2- 172.21.15.7:0/1964242209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641a3be6a00 0x5641a3beb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.422+0000 7fc2fffff640 1 --2- 172.21.15.7:0/1964242209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641a3bebb70 0x5641a3bee000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.422+0000 7fc2fffff640 1 -- 172.21.15.7:0/1964242209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641a3be6a00 msgr2=0x5641a3beb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.422+0000 7fc2fdffb640 1 --2- 172.21.15.7:0/3105526122 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc2f00a2b80 0x7fc2f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.422+0000 7fc2fffff640 1 --2- 172.21.15.7:0/1964242209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641a3be6a00 0x5641a3beb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.422+0000 7fc2fffff640 1 -- 172.21.15.7:0/1964242209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641a3bebb70 msgr2=0x5641a3bee000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.422+0000 7fc2fffff640 1 --2- 172.21.15.7:0/1964242209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641a3bebb70 0x5641a3bee000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.422+0000 7fc2fffff640 1 -- 172.21.15.7:0/1964242209 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641a3b039b0 con 0x5641a3be6010 2024-06-10T01:21:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.422+0000 7fc304e56640 1 --2- 172.21.15.7:0/1964242209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641a3be6a00 0x5641a3beb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:21:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.422+0000 7fc305657640 1 --2- 172.21.15.7:0/3105526122 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc2f00a2b80 0x7fc2f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.422+0000 7fc2fdffb640 1 -- 172.21.15.7:0/3105526122 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc2f80f0420 con 0x7fc3001024b0 2024-06-10T01:21:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.422+0000 7fc2fffff640 1 --2- 172.21.15.7:0/1964242209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641a3be6010 0x5641a3be6430 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec014530 tx=0x7fc2ec009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8147d0621e66b27b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.423+0000 7fc2eaffd640 1 -- 172.21.15.7:0/1964242209 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec01a030 con 0x5641a3be6010 2024-06-10T01:21:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.423+0000 7fc2eaffd640 1 -- 172.21.15.7:0/1964242209 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2ec0140d0 con 0x5641a3be6010 2024-06-10T01:21:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.423+0000 7fc2eaffd640 1 -- 172.21.15.7:0/1964242209 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec012480 con 0x5641a3be6010 2024-06-10T01:21:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.423+0000 7fc305657640 1 --2- 172.21.15.7:0/3105526122 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc2f00a2b80 0x7fc2f00a5040 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80b92d0 tx=0x7fc2f80b1de0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.423+0000 7fc309dc7740 1 -- 172.21.15.7:0/1964242209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641a3be6010 msgr2=0x5641a3be6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.423+0000 7fc309dc7740 1 --2- 172.21.15.7:0/1964242209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641a3be6010 0x5641a3be6430 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec014530 tx=0x7fc2ec009880 comp rx=0 tx=0).stop 2024-06-10T01:21:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.424+0000 7fc309dc7740 1 -- 172.21.15.7:0/1964242209 shutdown_connections 2024-06-10T01:21:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.424+0000 7fc309dc7740 1 --2- 172.21.15.7:0/1964242209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641a3bebb70 0x5641a3bee000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.424+0000 7fc309dc7740 1 --2- 172.21.15.7:0/1964242209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641a3be6a00 0x5641a3beb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.424+0000 7fc309dc7740 1 --2- 172.21.15.7:0/1964242209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641a3be6010 0x5641a3be6430 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.424+0000 7fc309dc7740 1 -- 172.21.15.7:0/1964242209 >> 172.21.15.7:0/1964242209 conn(0x5641a3be1680 msgr2=0x5641a3be1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.424+0000 7fc309dc7740 1 -- 172.21.15.7:0/1964242209 shutdown_connections 2024-06-10T01:21:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.424+0000 7fc309dc7740 1 -- 172.21.15.7:0/1964242209 wait complete. 2024-06-10T01:21:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.424+0000 7fc309dc7740 1 -- start start 2024-06-10T01:21:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.424+0000 7fc309dc7740 1 Processor -- start 2024-06-10T01:21:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.424+0000 7fc309dc7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641a3be6010 0x5641a3bf0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.424+0000 7fc309dc7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641a3be6a00 0x5641a3bf0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.424+0000 7fc309dc7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641a3bebb70 0x5641a3bf7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.425+0000 7fc309dc7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5641a3beeb20 con 0x5641a3bebb70 2024-06-10T01:21:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.425+0000 7fc309dc7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5641a3ad2ae0 con 0x5641a3be6a00 2024-06-10T01:21:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.425+0000 7fc309dc7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5641a3beeca0 con 0x5641a3be6010 2024-06-10T01:21:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.425+0000 7fc309dc7740 10 client.0 _ll_register_callbacks cb 0x5641a3b0b9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:21:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.425+0000 7fc2fffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641a3bebb70 0x5641a3bf7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.425+0000 7fc304e56640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641a3be6a00 0x5641a3bf0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.425+0000 7fc2fffff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641a3be6010 0x5641a3bf0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.425+0000 7fc304e56640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641a3be6a00 0x5641a3bf0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:35284/0 (socket says 172.21.15.7:35284) 2024-06-10T01:21:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.425+0000 7fc304e56640 1 -- 172.21.15.7:0/3483335054 learned_addr learned my addr 172.21.15.7:0/3483335054 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.425+0000 7fc2fffff640 1 -- 172.21.15.7:0/3483335054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641a3be6010 msgr2=0x5641a3bf0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.425+0000 7fc2fffff640 1 --2- 172.21.15.7:0/3483335054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641a3be6010 0x5641a3bf0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.425+0000 7fc2fffff640 1 -- 172.21.15.7:0/3483335054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641a3be6a00 msgr2=0x5641a3bf0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.425+0000 7fc2fffff640 1 --2- 172.21.15.7:0/3483335054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641a3be6a00 0x5641a3bf0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.425+0000 7fc2fffff640 1 -- 172.21.15.7:0/3483335054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641a385d210 con 0x5641a3bebb70 2024-06-10T01:21:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.425+0000 7fc304e56640 1 --2- 172.21.15.7:0/3483335054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641a3be6a00 0x5641a3bf0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:21:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.425+0000 7fc2fffff640 1 --2- 172.21.15.7:0/3483335054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641a3bebb70 0x5641a3bf7890 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0095d0 tx=0x7fc2ec009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6388b20bb8d34b5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.425+0000 7fc2dffff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:21:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.425+0000 7fc2dffff640 1 -- 172.21.15.7:0/3483335054 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec012480 con 0x5641a3bebb70 2024-06-10T01:21:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.426+0000 7fc2dffff640 1 -- 172.21.15.7:0/3483335054 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc2ec012620 con 0x5641a3bebb70 2024-06-10T01:21:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.426+0000 7fc309dc7740 1 -- 172.21.15.7:0/3483335054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2ec0140d0 con 0x5641a3bebb70 2024-06-10T01:21:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.426+0000 7fc2dffff640 1 -- 172.21.15.7:0/3483335054 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc2ec012820 con 0x5641a3bebb70 2024-06-10T01:21:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.426+0000 7fc309dc7740 10 client.19308 fetch_fsmap learned FSMap version 44 2024-06-10T01:21:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.426+0000 7fc309dc7740 1 -- 172.21.15.7:0/3483335054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5641a3929620 con 0x5641a3bebb70 2024-06-10T01:21:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.426+0000 7fc2dffff640 1 -- 172.21.15.7:0/3483335054 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2ec0140d0 con 0x5641a3bebb70 2024-06-10T01:21:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.426+0000 7fc2dffff640 1 -- 172.21.15.7:0/3483335054 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fc2ec00f040 con 0x5641a3bebb70 2024-06-10T01:21:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.426+0000 7fc309dc7740 10 client.19308 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:21:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.426+0000 7fc309dc7740 10 client.19308 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:21:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.426+0000 7fc309dc7740 20 client.19308 populate_metadata read hostname 'smithi007' 2024-06-10T01:21:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.426+0000 7fc309dc7740 1 --2- 172.21.15.7:0/3483335054 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5641a3bfbea0 0x5641a3bfe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.426+0000 7fc309dc7740 4 client.19308 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:21:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.426+0000 7fc309dc7740 1 -- 172.21.15.7:0/3483335054 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641a3b5cf10 con 0x5641a3bfbea0 2024-06-10T01:21:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.427+0000 7fc304e56640 1 --2- 172.21.15.7:0/3483335054 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5641a3bfbea0 0x5641a3bfe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.427+0000 7fc2dffff640 10 client.19308 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.427+0000 7fc304e56640 1 --2- 172.21.15.7:0/3483335054 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5641a3bfbea0 0x5641a3bfe2e0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da9989c3c1e48f60 server_cookie=663772776b9bee23 in_seq=0 out_seq=0 2024-06-10T01:21:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.430+0000 7fc2dffff640 1 -- 172.21.15.7:0/3483335054 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5641a3b5cf10 con 0x5641a3bfbea0 2024-06-10T01:21:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.430+0000 7fc2dffff640 10 client.19308 handle_command_reply: tid=0 2024-06-10T01:21:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.430+0000 7fc309dc7740 1 client.19308 shutdown 2024-06-10T01:21:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.430+0000 7fc309dc7740 1 -- 172.21.15.7:0/3483335054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641a3bebb70 msgr2=0x5641a3bf7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.430+0000 7fc309dc7740 1 --2- 172.21.15.7:0/3483335054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641a3bebb70 0x5641a3bf7890 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0095d0 tx=0x7fc2ec009eb0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.430+0000 7fc309dc7740 1 -- 172.21.15.7:0/3483335054 shutdown_connections 2024-06-10T01:21:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.430+0000 7fc309dc7740 1 --2- 172.21.15.7:0/3483335054 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5641a3bfbea0 0x5641a3bfe2e0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.430+0000 7fc309dc7740 1 --2- 172.21.15.7:0/3483335054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641a3bebb70 0x5641a3bf7890 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.430+0000 7fc309dc7740 1 --2- 172.21.15.7:0/3483335054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641a3be6a00 0x5641a3bf0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.430+0000 7fc2dffff640 0 client.19308 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.430+0000 7fc309dc7740 1 --2- 172.21.15.7:0/3483335054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641a3be6010 0x5641a3bf0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.430+0000 7fc309dc7740 1 -- 172.21.15.7:0/3483335054 >> 172.21.15.7:0/3483335054 conn(0x5641a3be1680 msgr2=0x5641a3be07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.430+0000 7fc309dc7740 1 -- 172.21.15.7:0/3483335054 shutdown_connections 2024-06-10T01:21:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.430+0000 7fc309dc7740 1 -- 172.21.15.7:0/3483335054 wait complete. 2024-06-10T01:21:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.430+0000 7fc309dc7740 20 client.19308 trim_cache size 0 max 16384 2024-06-10T01:21:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.447+0000 7fc309dc7740 1 Processor -- start 2024-06-10T01:21:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.447+0000 7fc309dc7740 1 -- start start 2024-06-10T01:21:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.448+0000 7fc309dc7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641a3be6010 0x5641a3beb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.448+0000 7fc309dc7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641a3be6a00 0x5641a3bf7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.448+0000 7fc309dc7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641a3bebb70 0x5641a3be0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.448+0000 7fc309dc7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc3001af120 con 0x5641a3be6a00 2024-06-10T01:21:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.448+0000 7fc309dc7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc3001ac960 con 0x5641a3bebb70 2024-06-10T01:21:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.448+0000 7fc309dc7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc3001aa7b0 con 0x5641a3be6010 2024-06-10T01:21:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.448+0000 7fc304e56640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641a3be6a00 0x5641a3bf7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.448+0000 7fc304e56640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641a3be6a00 0x5641a3bf7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45724/0 (socket says 172.21.15.7:45724) 2024-06-10T01:21:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.448+0000 7fc304e56640 1 -- 172.21.15.7:0/3797829515 learned_addr learned my addr 172.21.15.7:0/3797829515 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.448+0000 7fc2fffff640 1 --2- 172.21.15.7:0/3797829515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641a3bebb70 0x5641a3be0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.448+0000 7fc2fffff640 1 --2- 172.21.15.7:0/3797829515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641a3be6010 0x5641a3beb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.448+0000 7fc304e56640 1 -- 172.21.15.7:0/3797829515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641a3be6010 msgr2=0x5641a3beb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.448+0000 7fc304e56640 1 --2- 172.21.15.7:0/3797829515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641a3be6010 0x5641a3beb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.448+0000 7fc304e56640 1 -- 172.21.15.7:0/3797829515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641a3bebb70 msgr2=0x5641a3be0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.448+0000 7fc304e56640 1 --2- 172.21.15.7:0/3797829515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641a3bebb70 0x5641a3be0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.448+0000 7fc304e56640 1 -- 172.21.15.7:0/3797829515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641a3bdcfe0 con 0x5641a3be6a00 2024-06-10T01:21:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.448+0000 7fc304e56640 1 --2- 172.21.15.7:0/3797829515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641a3be6a00 0x5641a3bf7850 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x5641a3bfeb20 tx=0x7fc2f4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=656216c0000083bd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.449+0000 7fc2ea7fc640 1 -- 172.21.15.7:0/3797829515 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f400fc80 con 0x5641a3be6a00 2024-06-10T01:21:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.449+0000 7fc2ea7fc640 1 -- 172.21.15.7:0/3797829515 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f400fe20 con 0x5641a3be6a00 2024-06-10T01:21:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.449+0000 7fc2ea7fc640 1 -- 172.21.15.7:0/3797829515 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4010430 con 0x5641a3be6a00 2024-06-10T01:21:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.449+0000 7fc309dc7740 1 -- 172.21.15.7:0/3797829515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641a3be6a00 msgr2=0x5641a3bf7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.449+0000 7fc309dc7740 1 --2- 172.21.15.7:0/3797829515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641a3be6a00 0x5641a3bf7850 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x5641a3bfeb20 tx=0x7fc2f4005f90 comp rx=0 tx=0).stop 2024-06-10T01:21:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.449+0000 7fc309dc7740 1 -- 172.21.15.7:0/3797829515 shutdown_connections 2024-06-10T01:21:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.449+0000 7fc309dc7740 1 --2- 172.21.15.7:0/3797829515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641a3bebb70 0x5641a3be0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.449+0000 7fc309dc7740 1 --2- 172.21.15.7:0/3797829515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641a3be6a00 0x5641a3bf7850 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.449+0000 7fc309dc7740 1 --2- 172.21.15.7:0/3797829515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641a3be6010 0x5641a3beb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.449+0000 7fc309dc7740 1 -- 172.21.15.7:0/3797829515 >> 172.21.15.7:0/3797829515 conn(0x5641a3be1680 msgr2=0x5641a3bef0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.450+0000 7fc309dc7740 1 -- 172.21.15.7:0/3797829515 shutdown_connections 2024-06-10T01:21:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.450+0000 7fc309dc7740 1 -- 172.21.15.7:0/3797829515 wait complete. 2024-06-10T01:21:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.450+0000 7fc309dc7740 1 -- start start 2024-06-10T01:21:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.450+0000 7fc309dc7740 1 Processor -- start 2024-06-10T01:21:19.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.450+0000 7fc309dc7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641a3be6010 0x5641a3bf7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:19.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.450+0000 7fc309dc7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641a3be6a00 0x5641a3bf3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:19.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.450+0000 7fc309dc7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641a3bebb70 0x5641a3bf4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:19.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.450+0000 7fc309dc7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc3001aae20 con 0x5641a3be6010 2024-06-10T01:21:19.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.450+0000 7fc309dc7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc3001ac290 con 0x5641a3bebb70 2024-06-10T01:21:19.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.450+0000 7fc309dc7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc3001ab4f0 con 0x5641a3be6a00 2024-06-10T01:21:19.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.450+0000 7fc309dc7740 10 client.0 _ll_register_callbacks cb 0x7fc2f4009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:21:19.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.450+0000 7fc2fffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641a3be6010 0x5641a3bf7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:19.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.450+0000 7fc2fffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641a3be6010 0x5641a3bf7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45732/0 (socket says 172.21.15.7:45732) 2024-06-10T01:21:19.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.450+0000 7fc2fffff640 1 -- 172.21.15.7:0/1021647391 learned_addr learned my addr 172.21.15.7:0/1021647391 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:19.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.451+0000 7fc2fffff640 1 --2- 172.21.15.7:0/1021647391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641a3bebb70 0x5641a3bf4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:19.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.451+0000 7fc304e56640 1 --2- 172.21.15.7:0/1021647391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641a3be6a00 0x5641a3bf3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:19.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.451+0000 7fc2fffff640 1 -- 172.21.15.7:0/1021647391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641a3be6a00 msgr2=0x5641a3bf3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:19.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.451+0000 7fc2fffff640 1 --2- 172.21.15.7:0/1021647391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641a3be6a00 0x5641a3bf3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.451+0000 7fc2fffff640 1 -- 172.21.15.7:0/1021647391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641a3bebb70 msgr2=0x5641a3bf4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:19.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.451+0000 7fc2fffff640 1 --2- 172.21.15.7:0/1021647391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641a3bebb70 0x5641a3bf4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.451+0000 7fc2fffff640 1 -- 172.21.15.7:0/1021647391 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641a3bf79b0 con 0x5641a3be6010 2024-06-10T01:21:19.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.451+0000 7fc2fffff640 1 --2- 172.21.15.7:0/1021647391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641a3be6010 0x5641a3bf7380 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec01fa50 tx=0x7fc2ec009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4dbbc1338c62570 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:19.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.451+0000 7fc2dffff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:21:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.451+0000 7fc2dffff640 1 -- 172.21.15.7:0/1021647391 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec01a040 con 0x5641a3be6010 2024-06-10T01:21:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.451+0000 7fc2dffff640 1 -- 172.21.15.7:0/1021647391 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc2ec012480 con 0x5641a3be6010 2024-06-10T01:21:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.451+0000 7fc309dc7740 1 -- 172.21.15.7:0/1021647391 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2f400fe20 con 0x5641a3be6010 2024-06-10T01:21:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.451+0000 7fc2dffff640 1 -- 172.21.15.7:0/1021647391 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc2ec012680 con 0x5641a3be6010 2024-06-10T01:21:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.451+0000 7fc309dc7740 10 client.19314 fetch_fsmap learned FSMap version 44 2024-06-10T01:21:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.451+0000 7fc309dc7740 1 -- 172.21.15.7:0/1021647391 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5641a3c65aa0 con 0x5641a3be6010 2024-06-10T01:21:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.451+0000 7fc2dffff640 1 -- 172.21.15.7:0/1021647391 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f400fe20 con 0x5641a3be6010 2024-06-10T01:21:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.452+0000 7fc2dffff640 1 -- 172.21.15.7:0/1021647391 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fc2ec012c90 con 0x5641a3be6010 2024-06-10T01:21:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.452+0000 7fc309dc7740 10 client.19314 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:21:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.452+0000 7fc309dc7740 10 client.19314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:21:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.452+0000 7fc309dc7740 20 client.19314 populate_metadata read hostname 'smithi007' 2024-06-10T01:21:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.452+0000 7fc309dc7740 1 --2- 172.21.15.7:0/1021647391 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5641a3c66420 0x5641a3c68840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.452+0000 7fc309dc7740 4 client.19314 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:21:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.452+0000 7fc309dc7740 1 -- 172.21.15.7:0/1021647391 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5641a3b56c20 con 0x5641a3c66420 2024-06-10T01:21:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.452+0000 7fc304e56640 1 --2- 172.21.15.7:0/1021647391 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5641a3c66420 0x5641a3c68840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.453+0000 7fc2dffff640 10 client.19314 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.453+0000 7fc304e56640 1 --2- 172.21.15.7:0/1021647391 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5641a3c66420 0x5641a3c68840 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60c54bce6ed3ff0d server_cookie=c85c448d21c355c4 in_seq=0 out_seq=0 2024-06-10T01:21:19.502 INFO:tasks.workunit.client.0.smithi007.stdout: 10 37581 18.98 MB/sec warmup 116 sec latency 65.488 ms 2024-06-10T01:21:19.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:19 smithi007 ceph-mon[32398]: pgmap v891: 129 pgs: 129 active+clean; 422 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.7 MiB/s wr, 515 op/s 2024-06-10T01:21:19.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:19 smithi078 ceph-mon[36683]: pgmap v891: 129 pgs: 129 active+clean; 422 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.7 MiB/s wr, 515 op/s 2024-06-10T01:21:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.929+0000 7fc2dffff640 1 -- 172.21.15.7:0/1021647391 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5641a3b56c20 con 0x5641a3c66420 2024-06-10T01:21:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.929+0000 7fc2dffff640 10 client.19314 handle_command_reply: tid=0 2024-06-10T01:21:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.929+0000 7fc309dc7740 1 client.19314 shutdown 2024-06-10T01:21:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.929+0000 7fc309dc7740 1 -- 172.21.15.7:0/1021647391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641a3be6010 msgr2=0x5641a3bf7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.929+0000 7fc309dc7740 1 --2- 172.21.15.7:0/1021647391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641a3be6010 0x5641a3bf7380 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec01fa50 tx=0x7fc2ec009eb0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.929+0000 7fc309dc7740 1 -- 172.21.15.7:0/1021647391 shutdown_connections 2024-06-10T01:21:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.929+0000 7fc309dc7740 1 --2- 172.21.15.7:0/1021647391 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5641a3c66420 0x5641a3c68840 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.929+0000 7fc2dffff640 0 client.19314 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.929+0000 7fc309dc7740 1 --2- 172.21.15.7:0/1021647391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641a3bebb70 0x5641a3bf4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.929+0000 7fc309dc7740 1 --2- 172.21.15.7:0/1021647391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641a3be6a00 0x5641a3bf3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.929+0000 7fc309dc7740 1 --2- 172.21.15.7:0/1021647391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641a3be6010 0x5641a3bf7380 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.929+0000 7fc309dc7740 1 -- 172.21.15.7:0/1021647391 >> 172.21.15.7:0/1021647391 conn(0x5641a3be1680 msgr2=0x5641a3bf02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.930+0000 7fc309dc7740 1 -- 172.21.15.7:0/1021647391 shutdown_connections 2024-06-10T01:21:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.930+0000 7fc309dc7740 1 -- 172.21.15.7:0/1021647391 wait complete. 2024-06-10T01:21:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.930+0000 7fc309dc7740 20 client.19314 trim_cache size 0 max 16384 2024-06-10T01:21:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:21:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:21:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:21:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.931+0000 7fc3070ba640 1 -- 172.21.15.7:0/3105526122 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc2f00a2b80 msgr2=0x7fc2f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.931+0000 7fc3070ba640 1 --2- 172.21.15.7:0/3105526122 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc2f00a2b80 0x7fc2f00a5040 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80b92d0 tx=0x7fc2f80b1de0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.931+0000 7fc3070ba640 1 -- 172.21.15.7:0/3105526122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3001024b0 msgr2=0x7fc300103410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.931+0000 7fc3070ba640 1 --2- 172.21.15.7:0/3105526122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3001024b0 0x7fc300103410 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8031ae0 tx=0x7fc2f8004ab0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.931+0000 7fc3070ba640 1 -- 172.21.15.7:0/3105526122 shutdown_connections 2024-06-10T01:21:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.931+0000 7fc3070ba640 1 --2- 172.21.15.7:0/3105526122 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc2f00a2b80 0x7fc2f00a5040 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.931+0000 7fc3070ba640 1 --2- 172.21.15.7:0/3105526122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3001024b0 0x7fc300103410 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.931+0000 7fc3070ba640 1 --2- 172.21.15.7:0/3105526122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc300101af0 0x7fc300102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.931+0000 7fc3070ba640 1 --2- 172.21.15.7:0/3105526122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc300100900 0x7fc300104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.931+0000 7fc3070ba640 1 -- 172.21.15.7:0/3105526122 >> 172.21.15.7:0/3105526122 conn(0x7fc3000fbe70 msgr2=0x7fc3000fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.931+0000 7fc3070ba640 1 -- 172.21.15.7:0/3105526122 shutdown_connections 2024-06-10T01:21:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:19.931+0000 7fc3070ba640 1 -- 172.21.15.7:0/3105526122 wait complete. 2024-06-10T01:21:19.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:19 smithi175 ceph-mon[36636]: pgmap v891: 129 pgs: 129 active+clean; 422 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.7 MiB/s wr, 515 op/s 2024-06-10T01:21:20.519 INFO:tasks.workunit.client.0.smithi007.stdout: 10 38166 19.05 MB/sec warmup 117 sec latency 280.203 ms 2024-06-10T01:21:20.655 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:20 smithi007 ceph-mon[32398]: pgmap v892: 129 pgs: 129 active+clean; 437 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 4.5 MiB/s wr, 625 op/s 2024-06-10T01:21:20.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.792+0000 7efccd464700 1 -- 172.21.15.7:0/1605043115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcc8102550 msgr2=0x7efcc81029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:20.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.792+0000 7efccd464700 1 --2- 172.21.15.7:0/1605043115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcc8102550 0x7efcc81029d0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7efcb0009990 tx=0x7efcb0009ca0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.793+0000 7efccd464700 1 -- 172.21.15.7:0/1605043115 shutdown_connections 2024-06-10T01:21:20.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.793+0000 7efccd464700 1 --2- 172.21.15.7:0/1605043115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcc8102f10 0x7efcc810f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.793+0000 7efccd464700 1 --2- 172.21.15.7:0/1605043115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcc81084f0 0x7efcc81088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.793+0000 7efccd464700 1 --2- 172.21.15.7:0/1605043115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcc8102550 0x7efcc81029d0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.793+0000 7efccd464700 1 -- 172.21.15.7:0/1605043115 >> 172.21.15.7:0/1605043115 conn(0x7efcc8079650 msgr2=0x7efcc8079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:20.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.793+0000 7efccd464700 1 -- 172.21.15.7:0/1605043115 shutdown_connections 2024-06-10T01:21:20.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.793+0000 7efccd464700 1 -- 172.21.15.7:0/1605043115 wait complete. 2024-06-10T01:21:20.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.793+0000 7efccd464700 1 Processor -- start 2024-06-10T01:21:20.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.793+0000 7efccd464700 1 -- start start 2024-06-10T01:21:20.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.794+0000 7efccd464700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcc8102550 0x7efcc8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:20.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.794+0000 7efccd464700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcc8102f10 0x7efcc8074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:20.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.794+0000 7efccd464700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcc81084f0 0x7efcc8075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:20.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.794+0000 7efccd464700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efcc8112760 con 0x7efcc8102f10 2024-06-10T01:21:20.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.794+0000 7efccd464700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efcc81125e0 con 0x7efcc8102550 2024-06-10T01:21:20.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.794+0000 7efccd464700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efcc81128e0 con 0x7efcc81084f0 2024-06-10T01:21:20.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.794+0000 7efcbffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcc8102f10 0x7efcc8074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:20.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.794+0000 7efcbffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcc8102f10 0x7efcc8074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45756/0 (socket says 172.21.15.7:45756) 2024-06-10T01:21:20.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.794+0000 7efcbffff700 1 -- 172.21.15.7:0/3725532437 learned_addr learned my addr 172.21.15.7:0/3725532437 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:20.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.794+0000 7efcc6ffd700 1 --2- 172.21.15.7:0/3725532437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcc8102550 0x7efcc8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:20.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.794+0000 7efcc77fe700 1 --2- 172.21.15.7:0/3725532437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcc81084f0 0x7efcc8075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:20.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.794+0000 7efcbffff700 1 -- 172.21.15.7:0/3725532437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcc81084f0 msgr2=0x7efcc8075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:20.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.794+0000 7efcbffff700 1 --2- 172.21.15.7:0/3725532437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcc81084f0 0x7efcc8075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.794+0000 7efcbffff700 1 -- 172.21.15.7:0/3725532437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcc8102550 msgr2=0x7efcc8074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:20.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.794+0000 7efcbffff700 1 --2- 172.21.15.7:0/3725532437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcc8102550 0x7efcc8074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.794+0000 7efcbffff700 1 -- 172.21.15.7:0/3725532437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc81a7160 con 0x7efcc8102f10 2024-06-10T01:21:20.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.794+0000 7efcbffff700 1 --2- 172.21.15.7:0/3725532437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcc8102f10 0x7efcc8074cd0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7efcb000bae0 tx=0x7efcb000bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:20.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.795+0000 7efcc4ff9700 1 -- 172.21.15.7:0/3725532437 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0026070 con 0x7efcc8102f10 2024-06-10T01:21:20.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.795+0000 7efcc4ff9700 1 -- 172.21.15.7:0/3725532437 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcb0003d70 con 0x7efcc8102f10 2024-06-10T01:21:20.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.795+0000 7efccd464700 1 -- 172.21.15.7:0/3725532437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc81a73f0 con 0x7efcc8102f10 2024-06-10T01:21:20.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.795+0000 7efcc4ff9700 1 -- 172.21.15.7:0/3725532437 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0021410 con 0x7efcc8102f10 2024-06-10T01:21:20.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.795+0000 7efccd464700 1 -- 172.21.15.7:0/3725532437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc81a78d0 con 0x7efcc8102f10 2024-06-10T01:21:20.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.797+0000 7efcc4ff9700 1 -- 172.21.15.7:0/3725532437 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7efcb0029070 con 0x7efcc8102f10 2024-06-10T01:21:20.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.798+0000 7efcc4ff9700 1 --2- 172.21.15.7:0/3725532437 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efcb40a2c20 0x7efcb40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:20.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.798+0000 7efcc4ff9700 1 -- 172.21.15.7:0/3725532437 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7efcb00d0440 con 0x7efcc8102f10 2024-06-10T01:21:20.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.798+0000 7efcc6ffd700 1 --2- 172.21.15.7:0/3725532437 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efcb40a2c20 0x7efcb40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:20.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.798+0000 7efcc6ffd700 1 --2- 172.21.15.7:0/3725532437 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efcb40a2c20 0x7efcb40a50e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7efcb8007b60 tx=0x7efcb80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:20.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.799+0000 7efcd7ce7b80 1 Processor -- start 2024-06-10T01:21:20.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.799+0000 7efcd7ce7b80 1 -- start start 2024-06-10T01:21:20.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.800+0000 7efcd7ce7b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffef18d640 0x55ffef18da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:20.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.800+0000 7efcd7ce7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ffef18df60 0x55ffef1879e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:20.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.800+0000 7efcd7ce7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ffef187f20 0x55ffef188340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:20.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.800+0000 7efcd7ce7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ffef0d9810 con 0x55ffef18df60 2024-06-10T01:21:20.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.800+0000 7efcd7ce7b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ffef081720 con 0x55ffef18d640 2024-06-10T01:21:20.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.800+0000 7efcd7ce7b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ffef0d57d0 con 0x55ffef187f20 2024-06-10T01:21:20.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.800+0000 7efcbffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ffef18df60 0x55ffef1879e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:20.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.800+0000 7efcbffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ffef18df60 0x55ffef1879e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45766/0 (socket says 172.21.15.7:45766) 2024-06-10T01:21:20.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.800+0000 7efcbffff700 1 -- 172.21.15.7:0/2139817586 learned_addr learned my addr 172.21.15.7:0/2139817586 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:20.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.800+0000 7efcc6ffd700 1 --2- 172.21.15.7:0/2139817586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffef18d640 0x55ffef18da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:20.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.800+0000 7efcc77fe700 1 --2- 172.21.15.7:0/2139817586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ffef187f20 0x55ffef188340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:20.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.800+0000 7efcbffff700 1 -- 172.21.15.7:0/2139817586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ffef187f20 msgr2=0x55ffef188340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:20.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.800+0000 7efcbffff700 1 --2- 172.21.15.7:0/2139817586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ffef187f20 0x55ffef188340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.800+0000 7efcbffff700 1 -- 172.21.15.7:0/2139817586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffef18d640 msgr2=0x55ffef18da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:20.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.800+0000 7efcbffff700 1 --2- 172.21.15.7:0/2139817586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffef18d640 0x55ffef18da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.800+0000 7efcbffff700 1 -- 172.21.15.7:0/2139817586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffef077f50 con 0x55ffef18df60 2024-06-10T01:21:20.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.800+0000 7efcc6ffd700 1 --2- 172.21.15.7:0/2139817586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffef18d640 0x55ffef18da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:21:20.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.800+0000 7efcbffff700 1 --2- 172.21.15.7:0/2139817586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ffef18df60 0x55ffef1879e0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7efcb0098e10 tx=0x7efcb0005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef567b1c70363e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:20.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.801+0000 7efcbd7fa700 1 -- 172.21.15.7:0/2139817586 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb009c310 con 0x55ffef18df60 2024-06-10T01:21:20.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.801+0000 7efcbd7fa700 1 -- 172.21.15.7:0/2139817586 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcb009d0b0 con 0x55ffef18df60 2024-06-10T01:21:20.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.801+0000 7efcbd7fa700 1 -- 172.21.15.7:0/2139817586 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb00d0180 con 0x55ffef18df60 2024-06-10T01:21:20.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.801+0000 7efcd7ce7b80 1 -- 172.21.15.7:0/2139817586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ffef18df60 msgr2=0x55ffef1879e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:20.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.801+0000 7efcd7ce7b80 1 --2- 172.21.15.7:0/2139817586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ffef18df60 0x55ffef1879e0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7efcb0098e10 tx=0x7efcb0005a00 comp rx=0 tx=0).stop 2024-06-10T01:21:20.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.802+0000 7efcd7ce7b80 1 -- 172.21.15.7:0/2139817586 shutdown_connections 2024-06-10T01:21:20.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.802+0000 7efcd7ce7b80 1 --2- 172.21.15.7:0/2139817586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ffef187f20 0x55ffef188340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.802+0000 7efcd7ce7b80 1 --2- 172.21.15.7:0/2139817586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffef18d640 0x55ffef18da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.802+0000 7efcd7ce7b80 1 --2- 172.21.15.7:0/2139817586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ffef18df60 0x55ffef1879e0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.802+0000 7efcd7ce7b80 1 -- 172.21.15.7:0/2139817586 >> 172.21.15.7:0/2139817586 conn(0x55ffef0b2cc0 msgr2=0x55ffef182850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:20.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.802+0000 7efcd7ce7b80 1 -- 172.21.15.7:0/2139817586 shutdown_connections 2024-06-10T01:21:20.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.802+0000 7efcd7ce7b80 1 -- 172.21.15.7:0/2139817586 wait complete. 2024-06-10T01:21:20.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.802+0000 7efcd7ce7b80 1 -- start start 2024-06-10T01:21:20.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.802+0000 7efcd7ce7b80 1 Processor -- start 2024-06-10T01:21:20.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.802+0000 7efcd7ce7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ffef187f20 0x55ffef193ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:20.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.802+0000 7efcd7ce7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ffef18d640 0x55ffef191030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:20.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.802+0000 7efcd7ce7b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffef18df60 0x55ffef191570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:20.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.802+0000 7efcd7ce7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ffef1946f0 con 0x55ffef187f20 2024-06-10T01:21:20.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.802+0000 7efcd7ce7b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ffef079bf0 con 0x55ffef18df60 2024-06-10T01:21:20.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.802+0000 7efcd7ce7b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ffef194870 con 0x55ffef18d640 2024-06-10T01:21:20.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.802+0000 7efcd7ce7b80 10 client.0 _ll_register_callbacks cb 0x55ffef073920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:21:20.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.803+0000 7efcc6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ffef187f20 0x55ffef193ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:20.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.803+0000 7efcc6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ffef187f20 0x55ffef193ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45782/0 (socket says 172.21.15.7:45782) 2024-06-10T01:21:20.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.803+0000 7efcc6ffd700 1 -- 172.21.15.7:0/2044565506 learned_addr learned my addr 172.21.15.7:0/2044565506 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:20.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.803+0000 7efcbffff700 1 --2- 172.21.15.7:0/2044565506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ffef18d640 0x55ffef191030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:20.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.803+0000 7efcc77fe700 1 --2- 172.21.15.7:0/2044565506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffef18df60 0x55ffef191570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:20.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.803+0000 7efcc6ffd700 1 -- 172.21.15.7:0/2044565506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ffef18d640 msgr2=0x55ffef191030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:20.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.803+0000 7efcc6ffd700 1 --2- 172.21.15.7:0/2044565506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ffef18d640 0x55ffef191030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.803+0000 7efcc6ffd700 1 -- 172.21.15.7:0/2044565506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffef18df60 msgr2=0x55ffef191570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:20.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.803+0000 7efcc6ffd700 1 --2- 172.21.15.7:0/2044565506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffef18df60 0x55ffef191570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.803+0000 7efcc6ffd700 1 -- 172.21.15.7:0/2044565506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffef0288d0 con 0x55ffef187f20 2024-06-10T01:21:20.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.803+0000 7efcc6ffd700 1 --2- 172.21.15.7:0/2044565506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ffef187f20 0x55ffef193ce0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7efcb80116e0 tx=0x7efcb80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29adbddcf461a6ef server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:20.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.803+0000 7efcae7fc700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:21:20.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.803+0000 7efcae7fc700 1 -- 172.21.15.7:0/2044565506 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb8028470 con 0x55ffef187f20 2024-06-10T01:21:20.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.803+0000 7efcae7fc700 1 -- 172.21.15.7:0/2044565506 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7efcb8028610 con 0x55ffef187f20 2024-06-10T01:21:20.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.803+0000 7efcd7ce7b80 1 -- 172.21.15.7:0/2044565506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcb009d0b0 con 0x55ffef187f20 2024-06-10T01:21:20.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.804+0000 7efcae7fc700 1 -- 172.21.15.7:0/2044565506 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7efcb8028810 con 0x55ffef187f20 2024-06-10T01:21:20.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.804+0000 7efcd7ce7b80 10 client.19326 fetch_fsmap learned FSMap version 44 2024-06-10T01:21:20.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.804+0000 7efcd7ce7b80 1 -- 172.21.15.7:0/2044565506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ffef19f090 con 0x55ffef187f20 2024-06-10T01:21:20.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.804+0000 7efcae7fc700 1 -- 172.21.15.7:0/2044565506 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcb8023070 con 0x55ffef187f20 2024-06-10T01:21:20.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.804+0000 7efcae7fc700 1 -- 172.21.15.7:0/2044565506 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7efcb801fc10 con 0x55ffef187f20 2024-06-10T01:21:20.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.804+0000 7efcd7ce7b80 10 client.19326 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:21:20.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.804+0000 7efcd7ce7b80 10 client.19326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:21:20.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.804+0000 7efcd7ce7b80 20 client.19326 populate_metadata read hostname 'smithi007' 2024-06-10T01:21:20.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.804+0000 7efcd7ce7b80 1 --2- 172.21.15.7:0/2044565506 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ffef1a16d0 0x55ffef1a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:20.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.804+0000 7efcd7ce7b80 4 client.19326 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:21:20.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.804+0000 7efcd7ce7b80 1 -- 172.21.15.7:0/2044565506 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efcb009d0b0 con 0x55ffef1a16d0 2024-06-10T01:21:20.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.804+0000 7efcbffff700 1 --2- 172.21.15.7:0/2044565506 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ffef1a16d0 0x55ffef1a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:20.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.805+0000 7efcbffff700 1 --2- 172.21.15.7:0/2044565506 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ffef1a16d0 0x55ffef1a3ac0 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eda77e9be8383301 server_cookie=700aeda6fac6e272 in_seq=0 out_seq=0 2024-06-10T01:21:20.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.805+0000 7efcae7fc700 10 client.19326 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:20.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.808+0000 7efcae7fc700 1 -- 172.21.15.7:0/2044565506 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7efcb009d0b0 con 0x55ffef1a16d0 2024-06-10T01:21:20.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.808+0000 7efcae7fc700 10 client.19326 handle_command_reply: tid=0 2024-06-10T01:21:20.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.808+0000 7efcd7ce7b80 1 client.19326 shutdown 2024-06-10T01:21:20.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.808+0000 7efcd7ce7b80 1 -- 172.21.15.7:0/2044565506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ffef187f20 msgr2=0x55ffef193ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:20.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.808+0000 7efcd7ce7b80 1 --2- 172.21.15.7:0/2044565506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ffef187f20 0x55ffef193ce0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7efcb80116e0 tx=0x7efcb80047c0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.808+0000 7efcd7ce7b80 1 -- 172.21.15.7:0/2044565506 shutdown_connections 2024-06-10T01:21:20.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.808+0000 7efcd7ce7b80 1 --2- 172.21.15.7:0/2044565506 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ffef1a16d0 0x55ffef1a3ac0 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.808+0000 7efcd7ce7b80 1 --2- 172.21.15.7:0/2044565506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffef18df60 0x55ffef191570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.808+0000 7efcae7fc700 0 client.19326 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:20.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.808+0000 7efcd7ce7b80 1 --2- 172.21.15.7:0/2044565506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ffef187f20 0x55ffef193ce0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.808+0000 7efcd7ce7b80 1 --2- 172.21.15.7:0/2044565506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ffef18d640 0x55ffef191030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.808+0000 7efcd7ce7b80 1 -- 172.21.15.7:0/2044565506 >> 172.21.15.7:0/2044565506 conn(0x55ffef0b2cc0 msgr2=0x55ffef1825f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:20.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.809+0000 7efcd7ce7b80 1 -- 172.21.15.7:0/2044565506 shutdown_connections 2024-06-10T01:21:20.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.809+0000 7efcd7ce7b80 1 -- 172.21.15.7:0/2044565506 wait complete. 2024-06-10T01:21:20.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.809+0000 7efcd7ce7b80 20 client.19326 trim_cache size 0 max 16384 2024-06-10T01:21:20.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.837+0000 7efcd7ce7b80 1 Processor -- start 2024-06-10T01:21:20.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.837+0000 7efcd7ce7b80 1 -- start start 2024-06-10T01:21:20.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.838+0000 7efcd7ce7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ffef187f20 0x55ffef1879e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:20.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.838+0000 7efcd7ce7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ffef18d640 0x55ffef1a9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:20.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.838+0000 7efcd7ce7b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffef18df60 0x55ffef1a98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:20.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.838+0000 7efcd7ce7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efcc81b2e30 con 0x55ffef18d640 2024-06-10T01:21:20.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.838+0000 7efcd7ce7b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efcc81b0630 con 0x55ffef18df60 2024-06-10T01:21:20.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.838+0000 7efcd7ce7b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efcc81ae480 con 0x55ffef187f20 2024-06-10T01:21:20.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.839+0000 7efcc77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffef18df60 0x55ffef1a98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:20.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.839+0000 7efcc6ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ffef187f20 0x55ffef1879e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:20.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.839+0000 7efcbffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ffef18d640 0x55ffef1a9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:20.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.839+0000 7efcc77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffef18df60 0x55ffef1a98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:35360/0 (socket says 172.21.15.7:35360) 2024-06-10T01:21:20.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.839+0000 7efcc77fe700 1 -- 172.21.15.7:0/711905072 learned_addr learned my addr 172.21.15.7:0/711905072 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:20.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.839+0000 7efcc6ffd700 1 --2- 172.21.15.7:0/711905072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ffef187f20 0x55ffef1879e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:53412/0 (socket says 172.21.15.7:53412) 2024-06-10T01:21:20.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.839+0000 7efcc77fe700 1 -- 172.21.15.7:0/711905072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ffef187f20 msgr2=0x55ffef1879e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:20.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.839+0000 7efcc77fe700 1 --2- 172.21.15.7:0/711905072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ffef187f20 0x55ffef1879e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.839+0000 7efcc77fe700 1 -- 172.21.15.7:0/711905072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ffef18d640 msgr2=0x55ffef1a9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:20.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.839+0000 7efcc77fe700 1 --2- 172.21.15.7:0/711905072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ffef18d640 0x55ffef1a9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.839+0000 7efcc77fe700 1 -- 172.21.15.7:0/711905072 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffef18dd40 con 0x55ffef18df60 2024-06-10T01:21:20.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.840+0000 7efcc77fe700 1 --2- 172.21.15.7:0/711905072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffef18df60 0x55ffef1a98c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efcc00119d0 tx=0x7efcc0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=564186b4f6e0ee65 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:20.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.840+0000 7efcbcff9700 1 -- 172.21.15.7:0/711905072 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc0017880 con 0x55ffef18df60 2024-06-10T01:21:20.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.840+0000 7efcbcff9700 1 -- 172.21.15.7:0/711905072 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcc0007500 con 0x55ffef18df60 2024-06-10T01:21:20.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.840+0000 7efcbcff9700 1 -- 172.21.15.7:0/711905072 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc001f640 con 0x55ffef18df60 2024-06-10T01:21:20.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.841+0000 7efcd7ce7b80 1 -- 172.21.15.7:0/711905072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffef18df60 msgr2=0x55ffef1a98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:20.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.841+0000 7efcd7ce7b80 1 --2- 172.21.15.7:0/711905072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffef18df60 0x55ffef1a98c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efcc00119d0 tx=0x7efcc0009920 comp rx=0 tx=0).stop 2024-06-10T01:21:20.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.842+0000 7efcd7ce7b80 1 -- 172.21.15.7:0/711905072 shutdown_connections 2024-06-10T01:21:20.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.842+0000 7efcd7ce7b80 1 --2- 172.21.15.7:0/711905072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffef18df60 0x55ffef1a98c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.842+0000 7efcd7ce7b80 1 --2- 172.21.15.7:0/711905072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ffef187f20 0x55ffef1879e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.842+0000 7efcd7ce7b80 1 --2- 172.21.15.7:0/711905072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ffef18d640 0x55ffef1a9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.842+0000 7efcd7ce7b80 1 -- 172.21.15.7:0/711905072 >> 172.21.15.7:0/711905072 conn(0x55ffef0b2cc0 msgr2=0x55ffef18b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:20.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.842+0000 7efcd7ce7b80 1 -- 172.21.15.7:0/711905072 shutdown_connections 2024-06-10T01:21:20.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.842+0000 7efcd7ce7b80 1 -- 172.21.15.7:0/711905072 wait complete. 2024-06-10T01:21:20.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.842+0000 7efcd7ce7b80 1 -- start start 2024-06-10T01:21:20.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.843+0000 7efcd7ce7b80 1 Processor -- start 2024-06-10T01:21:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.843+0000 7efcd7ce7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ffef187f20 0x55ffef1a9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.843+0000 7efcd7ce7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ffef18d640 0x55ffef1a95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.843+0000 7efcd7ce7b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffef18df60 0x55ffef1a9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.843+0000 7efcc6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ffef187f20 0x55ffef1a9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.844+0000 7efcbffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ffef18d640 0x55ffef1a95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.844+0000 7efcc6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ffef187f20 0x55ffef1a9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45800/0 (socket says 172.21.15.7:45800) 2024-06-10T01:21:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.844+0000 7efcbffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ffef18d640 0x55ffef1a95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:53426/0 (socket says 172.21.15.7:53426) 2024-06-10T01:21:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.844+0000 7efcc6ffd700 1 -- 172.21.15.7:0/3560878006 learned_addr learned my addr 172.21.15.7:0/3560878006 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.844+0000 7efcc77fe700 1 --2- 172.21.15.7:0/3560878006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffef18df60 0x55ffef1a9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.844+0000 7efcd7ce7b80 1 -- 172.21.15.7:0/3560878006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efcc81aeaf0 con 0x55ffef187f20 2024-06-10T01:21:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.844+0000 7efcd7ce7b80 1 -- 172.21.15.7:0/3560878006 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efcc81aff60 con 0x55ffef18df60 2024-06-10T01:21:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.844+0000 7efcd7ce7b80 1 -- 172.21.15.7:0/3560878006 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efcc81af1c0 con 0x55ffef18d640 2024-06-10T01:21:20.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.845+0000 7efcd7ce7b80 10 client.0 _ll_register_callbacks cb 0x55ffef073920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:21:20.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.845+0000 7efcc77fe700 1 -- 172.21.15.7:0/3560878006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ffef18d640 msgr2=0x55ffef1a95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:20.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.845+0000 7efcc77fe700 1 --2- 172.21.15.7:0/3560878006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ffef18d640 0x55ffef1a95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.845+0000 7efcc77fe700 1 -- 172.21.15.7:0/3560878006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ffef187f20 msgr2=0x55ffef1a9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:20.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.845+0000 7efcc77fe700 1 --2- 172.21.15.7:0/3560878006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ffef187f20 0x55ffef1a9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.845+0000 7efcc77fe700 1 -- 172.21.15.7:0/3560878006 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffef20b450 con 0x55ffef18df60 2024-06-10T01:21:20.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.845+0000 7efcc77fe700 1 --2- 172.21.15.7:0/3560878006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffef18df60 0x55ffef1a9af0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7efcc00119d0 tx=0x7efcc0009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=590177097a57a724 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:20.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.845+0000 7efcae7fc700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:21:20.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.846+0000 7efcae7fc700 1 -- 172.21.15.7:0/3560878006 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc0004ad0 con 0x55ffef18df60 2024-06-10T01:21:20.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.846+0000 7efcae7fc700 1 -- 172.21.15.7:0/3560878006 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7efcc0004c70 con 0x55ffef18df60 2024-06-10T01:21:20.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.846+0000 7efcd7ce7b80 1 -- 172.21.15.7:0/3560878006 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcc0007500 con 0x55ffef18df60 2024-06-10T01:21:20.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.846+0000 7efcae7fc700 1 -- 172.21.15.7:0/3560878006 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7efcc0033050 con 0x55ffef18df60 2024-06-10T01:21:20.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.847+0000 7efcd7ce7b80 10 client.28583 fetch_fsmap learned FSMap version 44 2024-06-10T01:21:20.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.847+0000 7efcd7ce7b80 1 -- 172.21.15.7:0/3560878006 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ffef20c720 con 0x55ffef18df60 2024-06-10T01:21:20.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.848+0000 7efcae7fc700 1 -- 172.21.15.7:0/3560878006 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcc0007500 con 0x55ffef18df60 2024-06-10T01:21:20.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.848+0000 7efcae7fc700 1 -- 172.21.15.7:0/3560878006 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7efcc001e070 con 0x55ffef18df60 2024-06-10T01:21:20.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.848+0000 7efcd7ce7b80 10 client.28583 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:21:20.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.848+0000 7efcd7ce7b80 10 client.28583 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:21:20.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.848+0000 7efcd7ce7b80 20 client.28583 populate_metadata read hostname 'smithi007' 2024-06-10T01:21:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.848+0000 7efcd7ce7b80 1 --2- 172.21.15.7:0/3560878006 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ffef1a16d0 0x55ffef21fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.848+0000 7efcd7ce7b80 4 client.28583 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:21:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.848+0000 7efcd7ce7b80 1 -- 172.21.15.7:0/3560878006 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ffef0e6d10 con 0x55ffef1a16d0 2024-06-10T01:21:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.850+0000 7efcc6ffd700 1 --2- 172.21.15.7:0/3560878006 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ffef1a16d0 0x55ffef21fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:20.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.854+0000 7efcc6ffd700 1 --2- 172.21.15.7:0/3560878006 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ffef1a16d0 0x55ffef21fe30 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=185e1c90b3d2cf90 server_cookie=19c4cfd8a9407bd0 in_seq=0 out_seq=0 2024-06-10T01:21:20.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.854+0000 7efcae7fc700 10 client.28583 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:20.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.861+0000 7efcae7fc700 1 -- 172.21.15.7:0/3560878006 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ffef0e6d10 con 0x55ffef1a16d0 2024-06-10T01:21:20.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.861+0000 7efcae7fc700 10 client.28583 handle_command_reply: tid=0 2024-06-10T01:21:20.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.861+0000 7efcd7ce7b80 1 client.28583 shutdown 2024-06-10T01:21:20.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.863+0000 7efcd7ce7b80 1 -- 172.21.15.7:0/3560878006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffef18df60 msgr2=0x55ffef1a9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:20.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.863+0000 7efcd7ce7b80 1 --2- 172.21.15.7:0/3560878006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffef18df60 0x55ffef1a9af0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7efcc00119d0 tx=0x7efcc0009510 comp rx=0 tx=0).stop 2024-06-10T01:21:20.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.864+0000 7efcd7ce7b80 1 -- 172.21.15.7:0/3560878006 shutdown_connections 2024-06-10T01:21:20.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.864+0000 7efcd7ce7b80 1 --2- 172.21.15.7:0/3560878006 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ffef1a16d0 0x55ffef21fe30 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.864+0000 7efcd7ce7b80 1 --2- 172.21.15.7:0/3560878006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffef18df60 0x55ffef1a9af0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.864+0000 7efcae7fc700 0 client.28583 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:20.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.864+0000 7efcd7ce7b80 1 --2- 172.21.15.7:0/3560878006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ffef187f20 0x55ffef1a9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.864+0000 7efcd7ce7b80 1 --2- 172.21.15.7:0/3560878006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ffef18d640 0x55ffef1a95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.864+0000 7efcd7ce7b80 1 -- 172.21.15.7:0/3560878006 >> 172.21.15.7:0/3560878006 conn(0x55ffef0b2cc0 msgr2=0x55ffef17be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:20.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.867+0000 7efcd7ce7b80 1 -- 172.21.15.7:0/3560878006 shutdown_connections 2024-06-10T01:21:20.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.867+0000 7efcd7ce7b80 1 -- 172.21.15.7:0/3560878006 wait complete. 2024-06-10T01:21:20.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.867+0000 7efcd7ce7b80 20 client.28583 trim_cache size 0 max 16384 2024-06-10T01:21:20.861 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:21:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T01:21:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-10T01:21:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "9830b421-b118-4340-9f90-b40db12d236a": { 2024-06-10T01:21:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-10T01:21:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "9830b421-b118-4340-9f90-b40db12d236a", 2024-06-10T01:21:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-10T01:21:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:21:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:21:20.861 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:21:20.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.868+0000 7efccd464700 1 -- 172.21.15.7:0/3725532437 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efcb40a2c20 msgr2=0x7efcb40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:20.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.868+0000 7efccd464700 1 --2- 172.21.15.7:0/3725532437 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efcb40a2c20 0x7efcb40a50e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7efcb8007b60 tx=0x7efcb80073d0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.868+0000 7efccd464700 1 -- 172.21.15.7:0/3725532437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcc8102f10 msgr2=0x7efcc8074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:20.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.868+0000 7efccd464700 1 --2- 172.21.15.7:0/3725532437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcc8102f10 0x7efcc8074cd0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7efcb000bae0 tx=0x7efcb000bb10 comp rx=0 tx=0).stop 2024-06-10T01:21:20.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.868+0000 7efccd464700 1 -- 172.21.15.7:0/3725532437 shutdown_connections 2024-06-10T01:21:20.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.868+0000 7efccd464700 1 --2- 172.21.15.7:0/3725532437 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efcb40a2c20 0x7efcb40a50e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.868+0000 7efccd464700 1 --2- 172.21.15.7:0/3725532437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcc81084f0 0x7efcc8075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.868+0000 7efccd464700 1 --2- 172.21.15.7:0/3725532437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcc8102550 0x7efcc8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.869+0000 7efccd464700 1 --2- 172.21.15.7:0/3725532437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcc8102f10 0x7efcc8074cd0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:20.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.869+0000 7efccd464700 1 -- 172.21.15.7:0/3725532437 >> 172.21.15.7:0/3725532437 conn(0x7efcc8079650 msgr2=0x7efcc81005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:20.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.869+0000 7efccd464700 1 -- 172.21.15.7:0/3725532437 shutdown_connections 2024-06-10T01:21:20.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:20.869+0000 7efccd464700 1 -- 172.21.15.7:0/3725532437 wait complete. 2024-06-10T01:21:20.954 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:20 smithi175 ceph-mon[36636]: pgmap v892: 129 pgs: 129 active+clean; 437 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 4.5 MiB/s wr, 625 op/s 2024-06-10T01:21:21.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:20 smithi078 ceph-mon[36683]: pgmap v892: 129 pgs: 129 active+clean; 437 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 4.5 MiB/s wr, 625 op/s 2024-06-10T01:21:21.519 INFO:tasks.workunit.client.0.smithi007.stdout: 10 38859 19.21 MB/sec warmup 118 sec latency 78.679 ms 2024-06-10T01:21:21.789 INFO:tasks.cephfs.filesystem:scrub status for tag:9830b421-b118-4340-9f90-b40db12d236a - {'path': '/', 'tag': '9830b421-b118-4340-9f90-b40db12d236a', 'options': 'recursive,force'} 2024-06-10T01:21:22.519 INFO:tasks.workunit.client.0.smithi007.stdout: 10 39709 19.39 MB/sec warmup 119 sec latency 85.073 ms 2024-06-10T01:21:23.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:22 smithi175 ceph-mon[36636]: pgmap v893: 129 pgs: 129 active+clean; 439 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 4.0 MiB/s wr, 571 op/s 2024-06-10T01:21:23.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:22 smithi007 ceph-mon[32398]: pgmap v893: 129 pgs: 129 active+clean; 439 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 4.0 MiB/s wr, 571 op/s 2024-06-10T01:21:23.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:22 smithi078 ceph-mon[36683]: pgmap v893: 129 pgs: 129 active+clean; 439 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 4.0 MiB/s wr, 571 op/s 2024-06-10T01:21:24.519 INFO:tasks.workunit.client.0.smithi007.stdout: 10 41175 27.96 MB/sec execute 1 sec latency 98.630 ms 2024-06-10T01:21:25.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:24 smithi175 ceph-mon[36636]: pgmap v894: 129 pgs: 129 active+clean; 446 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 4.7 MiB/s wr, 707 op/s 2024-06-10T01:21:25.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:24 smithi007 ceph-mon[32398]: pgmap v894: 129 pgs: 129 active+clean; 446 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 4.7 MiB/s wr, 707 op/s 2024-06-10T01:21:25.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:24 smithi078 ceph-mon[36683]: pgmap v894: 129 pgs: 129 active+clean; 446 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 4.7 MiB/s wr, 707 op/s 2024-06-10T01:21:25.519 INFO:tasks.workunit.client.0.smithi007.stdout: 10 42004 36.11 MB/sec execute 2 sec latency 116.193 ms 2024-06-10T01:21:26.519 INFO:tasks.workunit.client.0.smithi007.stdout: 10 42860 37.87 MB/sec execute 3 sec latency 70.864 ms 2024-06-10T01:21:27.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:26 smithi175 ceph-mon[36636]: pgmap v895: 129 pgs: 129 active+clean; 446 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 4.3 MiB/s wr, 694 op/s 2024-06-10T01:21:27.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:26 smithi007 ceph-mon[32398]: pgmap v895: 129 pgs: 129 active+clean; 446 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 4.3 MiB/s wr, 694 op/s 2024-06-10T01:21:27.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:26 smithi078 ceph-mon[36683]: pgmap v895: 129 pgs: 129 active+clean; 446 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 4.3 MiB/s wr, 694 op/s 2024-06-10T01:21:27.519 INFO:tasks.workunit.client.0.smithi007.stdout: 10 43492 38.26 MB/sec execute 4 sec latency 109.348 ms 2024-06-10T01:21:28.519 INFO:tasks.workunit.client.0.smithi007.stdout: 10 44198 39.03 MB/sec execute 5 sec latency 59.323 ms 2024-06-10T01:21:29.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:28 smithi175 ceph-mon[36636]: pgmap v896: 129 pgs: 129 active+clean; 449 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.2 MiB/s wr, 666 op/s 2024-06-10T01:21:29.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:28 smithi007 ceph-mon[32398]: pgmap v896: 129 pgs: 129 active+clean; 449 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.2 MiB/s wr, 666 op/s 2024-06-10T01:21:29.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:28 smithi078 ceph-mon[36683]: pgmap v896: 129 pgs: 129 active+clean; 449 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.2 MiB/s wr, 666 op/s 2024-06-10T01:21:29.519 INFO:tasks.workunit.client.0.smithi007.stdout: 10 44897 37.74 MB/sec execute 6 sec latency 72.811 ms 2024-06-10T01:21:30.519 INFO:tasks.workunit.client.0.smithi007.stdout: 10 45649 38.62 MB/sec execute 7 sec latency 56.146 ms 2024-06-10T01:21:31.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:30 smithi175 ceph-mon[36636]: pgmap v897: 129 pgs: 129 active+clean; 463 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.9 MiB/s wr, 837 op/s 2024-06-10T01:21:31.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:30 smithi007 ceph-mon[32398]: pgmap v897: 129 pgs: 129 active+clean; 463 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.9 MiB/s wr, 837 op/s 2024-06-10T01:21:31.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:30 smithi078 ceph-mon[36683]: pgmap v897: 129 pgs: 129 active+clean; 463 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.9 MiB/s wr, 837 op/s 2024-06-10T01:21:31.519 INFO:tasks.workunit.client.0.smithi007.stdout: 10 46572 40.16 MB/sec execute 8 sec latency 49.510 ms 2024-06-10T01:21:32.519 INFO:tasks.workunit.client.0.smithi007.stdout: 10 46953 38.04 MB/sec execute 9 sec latency 603.014 ms 2024-06-10T01:21:33.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.087+0000 7fc825a93640 1 -- 172.21.15.7:0/3862152819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc820103c30 msgr2=0x7fc820104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:33.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.087+0000 7fc825a93640 1 --2- 172.21.15.7:0/3862152819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc820103c30 0x7fc820104050 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc810009860 tx=0x7fc81002a380 comp rx=0 tx=0).stop 2024-06-10T01:21:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.094+0000 7fc825a93640 1 -- 172.21.15.7:0/3862152819 shutdown_connections 2024-06-10T01:21:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.094+0000 7fc825a93640 1 --2- 172.21.15.7:0/3862152819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8201057e0 0x7fc82010c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.094+0000 7fc825a93640 1 --2- 172.21.15.7:0/3862152819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc820104e20 0x7fc8201052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.094+0000 7fc825a93640 1 --2- 172.21.15.7:0/3862152819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc820103c30 0x7fc820104050 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.094+0000 7fc825a93640 1 -- 172.21.15.7:0/3862152819 >> 172.21.15.7:0/3862152819 conn(0x7fc8200ff1c0 msgr2=0x7fc8201015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.095+0000 7fc825a93640 1 -- 172.21.15.7:0/3862152819 shutdown_connections 2024-06-10T01:21:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.095+0000 7fc825a93640 1 -- 172.21.15.7:0/3862152819 wait complete. 2024-06-10T01:21:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.097+0000 7fc825a93640 1 Processor -- start 2024-06-10T01:21:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.097+0000 7fc825a93640 1 -- start start 2024-06-10T01:21:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.097+0000 7fc825a93640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc820103c30 0x7fc820107f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.097+0000 7fc825a93640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc820104e20 0x7fc8201084a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.097+0000 7fc825a93640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8201057e0 0x7fc82010b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.097+0000 7fc825a93640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc82006e010 con 0x7fc820103c30 2024-06-10T01:21:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.097+0000 7fc825a93640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc82006de90 con 0x7fc820104e20 2024-06-10T01:21:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.097+0000 7fc825a93640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc82006e190 con 0x7fc8201057e0 2024-06-10T01:21:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.098+0000 7fc81ffff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8201057e0 0x7fc82010b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.098+0000 7fc81ffff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8201057e0 0x7fc82010b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:37364/0 (socket says 172.21.15.7:37364) 2024-06-10T01:21:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.098+0000 7fc81ffff640 1 -- 172.21.15.7:0/2108425244 learned_addr learned my addr 172.21.15.7:0/2108425244 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.098+0000 7fc81f7fe640 1 --2- 172.21.15.7:0/2108425244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc820103c30 0x7fc820107f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.098+0000 7fc81ffff640 1 -- 172.21.15.7:0/2108425244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc820104e20 msgr2=0x7fc8201084a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T01:21:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.098+0000 7fc81ffff640 1 --2- 172.21.15.7:0/2108425244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc820104e20 0x7fc8201084a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.098+0000 7fc81ffff640 1 -- 172.21.15.7:0/2108425244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc820103c30 msgr2=0x7fc820107f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.098+0000 7fc81ffff640 1 --2- 172.21.15.7:0/2108425244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc820103c30 0x7fc820107f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.098+0000 7fc81ffff640 1 -- 172.21.15.7:0/2108425244 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc82010bb60 con 0x7fc8201057e0 2024-06-10T01:21:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.099+0000 7fc81ffff640 1 --2- 172.21.15.7:0/2108425244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8201057e0 0x7fc82010b4f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc81801fd90 tx=0x7fc81801d750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.099+0000 7fc81cff9640 1 -- 172.21.15.7:0/2108425244 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81801de60 con 0x7fc8201057e0 2024-06-10T01:21:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.100+0000 7fc825a93640 1 -- 172.21.15.7:0/2108425244 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc82010bdf0 con 0x7fc8201057e0 2024-06-10T01:21:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.100+0000 7fc825a93640 1 -- 172.21.15.7:0/2108425244 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc82010c340 con 0x7fc8201057e0 2024-06-10T01:21:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.100+0000 7fc81cff9640 1 -- 172.21.15.7:0/2108425244 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc818003970 con 0x7fc8201057e0 2024-06-10T01:21:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.100+0000 7fc81cff9640 1 -- 172.21.15.7:0/2108425244 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc818004400 con 0x7fc8201057e0 2024-06-10T01:21:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.103+0000 7fc82879d740 1 Processor -- start 2024-06-10T01:21:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.107+0000 7fc81cff9640 1 -- 172.21.15.7:0/2108425244 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fc81802e020 con 0x7fc8201057e0 2024-06-10T01:21:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.108+0000 7fc81cff9640 1 --2- 172.21.15.7:0/2108425244 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc8080a2d90 0x7fc8080a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.108+0000 7fc81cff9640 1 -- 172.21.15.7:0/2108425244 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc8180dfdc0 con 0x7fc8201057e0 2024-06-10T01:21:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.109+0000 7fc82879d740 1 -- start start 2024-06-10T01:21:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.109+0000 7fc82879d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdaa62010 0x564bdaa62430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.109+0000 7fc82879d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bdaa62a00 0x564bdaa67500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.109+0000 7fc82879d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bdaa67b70 0x564bdaa6a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.109+0000 7fc82879d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564bda9c54f0 con 0x564bdaa62010 2024-06-10T01:21:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.109+0000 7fc82879d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564bda963870 con 0x564bdaa62a00 2024-06-10T01:21:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.109+0000 7fc82879d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564bda9c14b0 con 0x564bdaa67b70 2024-06-10T01:21:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.110+0000 7fc81f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdaa62010 0x564bdaa62430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.110+0000 7fc81effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bdaa62a00 0x564bdaa67500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.110+0000 7fc81f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdaa62010 0x564bdaa62430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40818/0 (socket says 172.21.15.7:40818) 2024-06-10T01:21:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.110+0000 7fc81f7fe640 1 -- 172.21.15.7:0/4110532587 learned_addr learned my addr 172.21.15.7:0/4110532587 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.110+0000 7fc81effd640 1 --2- 172.21.15.7:0/2108425244 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc8080a2d90 0x7fc8080a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.111+0000 7fc81ffff640 1 --2- 172.21.15.7:0/4110532587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bdaa67b70 0x564bdaa6a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.111+0000 7fc81f7fe640 1 -- 172.21.15.7:0/4110532587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bdaa67b70 msgr2=0x564bdaa6a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.111+0000 7fc81f7fe640 1 --2- 172.21.15.7:0/4110532587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bdaa67b70 0x564bdaa6a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.111+0000 7fc81f7fe640 1 -- 172.21.15.7:0/4110532587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bdaa62a00 msgr2=0x564bdaa67500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.111+0000 7fc81f7fe640 1 --2- 172.21.15.7:0/4110532587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bdaa62a00 0x564bdaa67500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.111+0000 7fc81f7fe640 1 -- 172.21.15.7:0/4110532587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bda97f9b0 con 0x564bdaa62010 2024-06-10T01:21:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.111+0000 7fc81f7fe640 1 --2- 172.21.15.7:0/4110532587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdaa62010 0x564bdaa62430 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fc810009860 tx=0x7fc810030340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2539e50962f07915 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.112+0000 7fc805ffb640 1 -- 172.21.15.7:0/4110532587 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc810037040 con 0x564bdaa62010 2024-06-10T01:21:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.112+0000 7fc805ffb640 1 -- 172.21.15.7:0/4110532587 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc810005030 con 0x564bdaa62010 2024-06-10T01:21:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.112+0000 7fc805ffb640 1 -- 172.21.15.7:0/4110532587 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8100049b0 con 0x564bdaa62010 2024-06-10T01:21:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.114+0000 7fc82879d740 1 -- 172.21.15.7:0/4110532587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdaa62010 msgr2=0x564bdaa62430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.114+0000 7fc82879d740 1 --2- 172.21.15.7:0/4110532587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdaa62010 0x564bdaa62430 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fc810009860 tx=0x7fc810030340 comp rx=0 tx=0).stop 2024-06-10T01:21:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.114+0000 7fc81effd640 1 --2- 172.21.15.7:0/2108425244 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc8080a2d90 0x7fc8080a5250 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x564bdaa62e00 tx=0x7fc8140046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.114+0000 7fc82879d740 1 -- 172.21.15.7:0/4110532587 shutdown_connections 2024-06-10T01:21:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.114+0000 7fc82879d740 1 --2- 172.21.15.7:0/4110532587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bdaa67b70 0x564bdaa6a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.114+0000 7fc82879d740 1 --2- 172.21.15.7:0/4110532587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bdaa62a00 0x564bdaa67500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.114+0000 7fc82879d740 1 --2- 172.21.15.7:0/4110532587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdaa62010 0x564bdaa62430 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.114+0000 7fc82879d740 1 -- 172.21.15.7:0/4110532587 >> 172.21.15.7:0/4110532587 conn(0x564bdaa5d680 msgr2=0x564bdaa5da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.115+0000 7fc82879d740 1 -- 172.21.15.7:0/4110532587 shutdown_connections 2024-06-10T01:21:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.116+0000 7fc82879d740 1 -- 172.21.15.7:0/4110532587 wait complete. 2024-06-10T01:21:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.116+0000 7fc82879d740 1 -- start start 2024-06-10T01:21:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.117+0000 7fc82879d740 1 Processor -- start 2024-06-10T01:21:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.117+0000 7fc82879d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bdaa62010 0x564bdaa6c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.117+0000 7fc82879d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bdaa62a00 0x564bdaa6c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.117+0000 7fc82879d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdaa67b70 0x564bdaa73800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.117+0000 7fc81effd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bdaa62a00 0x564bdaa6c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.117+0000 7fc81effd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bdaa62a00 0x564bdaa6c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:37374/0 (socket says 172.21.15.7:37374) 2024-06-10T01:21:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.117+0000 7fc81effd640 1 -- 172.21.15.7:0/3754823350 learned_addr learned my addr 172.21.15.7:0/3754823350 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.117+0000 7fc82879d740 1 -- 172.21.15.7:0/3754823350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564bdaa6aac0 con 0x564bdaa67b70 2024-06-10T01:21:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.117+0000 7fc82879d740 1 -- 172.21.15.7:0/3754823350 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564bda94eae0 con 0x564bdaa62010 2024-06-10T01:21:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.118+0000 7fc82879d740 1 -- 172.21.15.7:0/3754823350 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564bdaa6ac40 con 0x564bdaa62a00 2024-06-10T01:21:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.118+0000 7fc82879d740 10 client.0 _ll_register_callbacks cb 0x564bda9879b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:21:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.118+0000 7fc81effd640 1 -- 172.21.15.7:0/3754823350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bdaa62010 msgr2=0x564bdaa6c280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T01:21:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.118+0000 7fc81effd640 1 --2- 172.21.15.7:0/3754823350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bdaa62010 0x564bdaa6c280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.118+0000 7fc81effd640 1 -- 172.21.15.7:0/3754823350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdaa67b70 msgr2=0x564bdaa73800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.118+0000 7fc81effd640 1 --2- 172.21.15.7:0/3754823350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdaa67b70 0x564bdaa73800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.118+0000 7fc81effd640 1 -- 172.21.15.7:0/3754823350 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bda6d9210 con 0x564bdaa62a00 2024-06-10T01:21:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.118+0000 7fc81effd640 1 --2- 172.21.15.7:0/3754823350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bdaa62a00 0x564bdaa6c7c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc820109470 tx=0x7fc814004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c52a918e46cd17c7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.119+0000 7fc7feffd640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:21:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.122+0000 7fc7feffd640 1 -- 172.21.15.7:0/3754823350 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81404e060 con 0x564bdaa62a00 2024-06-10T01:21:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.122+0000 7fc82879d740 1 -- 172.21.15.7:0/3754823350 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc810005030 con 0x564bdaa62a00 2024-06-10T01:21:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.122+0000 7fc7feffd640 1 -- 172.21.15.7:0/3754823350 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc814049420 con 0x564bdaa62a00 2024-06-10T01:21:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.122+0000 7fc7feffd640 1 -- 172.21.15.7:0/3754823350 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc81403fe10 con 0x564bdaa62a00 2024-06-10T01:21:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.122+0000 7fc82879d740 10 client.28603 fetch_fsmap learned FSMap version 44 2024-06-10T01:21:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.122+0000 7fc82879d740 1 -- 172.21.15.7:0/3754823350 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x564bda7a5620 con 0x564bdaa62a00 2024-06-10T01:21:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.123+0000 7fc7feffd640 1 -- 172.21.15.7:0/3754823350 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc814049b00 con 0x564bdaa62a00 2024-06-10T01:21:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.123+0000 7fc7feffd640 1 -- 172.21.15.7:0/3754823350 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fc814010680 con 0x564bdaa62a00 2024-06-10T01:21:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.123+0000 7fc82879d740 10 client.28603 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:21:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.123+0000 7fc82879d740 10 client.28603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:21:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.123+0000 7fc82879d740 20 client.28603 populate_metadata read hostname 'smithi007' 2024-06-10T01:21:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.123+0000 7fc82879d740 1 --2- 172.21.15.7:0/3754823350 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564bdaa78400 0x564bdaa7a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.123+0000 7fc82879d740 4 client.28603 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:21:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.123+0000 7fc82879d740 1 -- 172.21.15.7:0/3754823350 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc810005030 con 0x564bdaa78400 2024-06-10T01:21:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.126+0000 7fc81f7fe640 1 --2- 172.21.15.7:0/3754823350 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564bdaa78400 0x564bdaa7a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.126+0000 7fc81f7fe640 1 --2- 172.21.15.7:0/3754823350 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564bdaa78400 0x564bdaa7a8a0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3cb44b85cf0c7a server_cookie=d2cfdcad4e05fb62 in_seq=0 out_seq=0 2024-06-10T01:21:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.126+0000 7fc7feffd640 10 client.28603 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.127+0000 7fc7feffd640 1 -- 172.21.15.7:0/3754823350 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fc810005030 con 0x564bdaa78400 2024-06-10T01:21:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.127+0000 7fc7feffd640 10 client.28603 handle_command_reply: tid=0 2024-06-10T01:21:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.128+0000 7fc82879d740 1 client.28603 shutdown 2024-06-10T01:21:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.129+0000 7fc82879d740 1 -- 172.21.15.7:0/3754823350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bdaa62a00 msgr2=0x564bdaa6c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.129+0000 7fc82879d740 1 --2- 172.21.15.7:0/3754823350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bdaa62a00 0x564bdaa6c7c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc820109470 tx=0x7fc814004730 comp rx=0 tx=0).stop 2024-06-10T01:21:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.129+0000 7fc82879d740 1 -- 172.21.15.7:0/3754823350 shutdown_connections 2024-06-10T01:21:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.129+0000 7fc82879d740 1 --2- 172.21.15.7:0/3754823350 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564bdaa78400 0x564bdaa7a8a0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.129+0000 7fc82879d740 1 --2- 172.21.15.7:0/3754823350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdaa67b70 0x564bdaa73800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.129+0000 7fc82879d740 1 --2- 172.21.15.7:0/3754823350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bdaa62a00 0x564bdaa6c7c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.129+0000 7fc82879d740 1 --2- 172.21.15.7:0/3754823350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bdaa62010 0x564bdaa6c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.129+0000 7fc82879d740 1 -- 172.21.15.7:0/3754823350 >> 172.21.15.7:0/3754823350 conn(0x564bdaa5d680 msgr2=0x564bdaa5c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.130+0000 7fc7feffd640 0 client.28603 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.132+0000 7fc82879d740 1 -- 172.21.15.7:0/3754823350 shutdown_connections 2024-06-10T01:21:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.132+0000 7fc82879d740 1 -- 172.21.15.7:0/3754823350 wait complete. 2024-06-10T01:21:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.133+0000 7fc82879d740 20 client.28603 trim_cache size 0 max 16384 2024-06-10T01:21:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.163+0000 7fc82879d740 1 Processor -- start 2024-06-10T01:21:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.164+0000 7fc82879d740 1 -- start start 2024-06-10T01:21:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.164+0000 7fc82879d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bdaa62010 0x564bdaa60d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.164+0000 7fc82879d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdaa62a00 0x564bdaa737c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.164+0000 7fc82879d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bdaa67b70 0x564bdaa5bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.164+0000 7fc82879d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c0125c0 con 0x564bdaa62a00 2024-06-10T01:21:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.164+0000 7fc82879d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c00fe00 con 0x564bdaa67b70 2024-06-10T01:21:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.164+0000 7fc82879d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c00dc50 con 0x564bdaa62010 2024-06-10T01:21:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.165+0000 7fc81ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bdaa67b70 0x564bdaa5bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.165+0000 7fc81ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bdaa67b70 0x564bdaa5bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:60320/0 (socket says 172.21.15.7:60320) 2024-06-10T01:21:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.165+0000 7fc81ffff640 1 -- 172.21.15.7:0/4022122546 learned_addr learned my addr 172.21.15.7:0/4022122546 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.165+0000 7fc81ffff640 1 -- 172.21.15.7:0/4022122546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bdaa62010 msgr2=0x564bdaa60d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.165+0000 7fc81ffff640 1 --2- 172.21.15.7:0/4022122546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bdaa62010 0x564bdaa60d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.165+0000 7fc81ffff640 1 -- 172.21.15.7:0/4022122546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdaa62a00 msgr2=0x564bdaa737c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.165+0000 7fc81ffff640 1 --2- 172.21.15.7:0/4022122546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdaa62a00 0x564bdaa737c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.166+0000 7fc81ffff640 1 -- 172.21.15.7:0/4022122546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bdaa76820 con 0x564bdaa67b70 2024-06-10T01:21:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.166+0000 7fc81ffff640 1 --2- 172.21.15.7:0/4022122546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bdaa67b70 0x564bdaa5bf50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc8180dd4e0 tx=0x7fc8180aca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4be68f64ec623f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.167+0000 7fc8057fa640 1 -- 172.21.15.7:0/4022122546 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8180c5070 con 0x564bdaa67b70 2024-06-10T01:21:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.167+0000 7fc8057fa640 1 -- 172.21.15.7:0/4022122546 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8180c5210 con 0x564bdaa67b70 2024-06-10T01:21:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.167+0000 7fc8057fa640 1 -- 172.21.15.7:0/4022122546 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81802b040 con 0x564bdaa67b70 2024-06-10T01:21:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.168+0000 7fc82879d740 1 -- 172.21.15.7:0/4022122546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bdaa67b70 msgr2=0x564bdaa5bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.168+0000 7fc82879d740 1 --2- 172.21.15.7:0/4022122546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bdaa67b70 0x564bdaa5bf50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc8180dd4e0 tx=0x7fc8180aca00 comp rx=0 tx=0).stop 2024-06-10T01:21:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.169+0000 7fc82879d740 1 -- 172.21.15.7:0/4022122546 shutdown_connections 2024-06-10T01:21:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.169+0000 7fc82879d740 1 --2- 172.21.15.7:0/4022122546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bdaa67b70 0x564bdaa5bf50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.169+0000 7fc82879d740 1 --2- 172.21.15.7:0/4022122546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdaa62a00 0x564bdaa737c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.169+0000 7fc82879d740 1 --2- 172.21.15.7:0/4022122546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bdaa62010 0x564bdaa60d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.169+0000 7fc82879d740 1 -- 172.21.15.7:0/4022122546 >> 172.21.15.7:0/4022122546 conn(0x564bdaa5d680 msgr2=0x564bdaa60650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.170+0000 7fc82879d740 1 -- 172.21.15.7:0/4022122546 shutdown_connections 2024-06-10T01:21:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.171+0000 7fc82879d740 1 -- 172.21.15.7:0/4022122546 wait complete. 2024-06-10T01:21:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.172+0000 7fc82879d740 1 -- start start 2024-06-10T01:21:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.172+0000 7fc82879d740 1 Processor -- start 2024-06-10T01:21:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.173+0000 7fc82879d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bdaa62a00 0x564bdaa70480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.173+0000 7fc82879d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bdaa67b70 0x564bdaa709c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.173+0000 7fc82879d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdaadf160 0x564bdaae1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.173+0000 7fc82879d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c00e2c0 con 0x564bdaadf160 2024-06-10T01:21:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.173+0000 7fc82879d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c00f730 con 0x564bdaa67b70 2024-06-10T01:21:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.173+0000 7fc82879d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c00e990 con 0x564bdaa62a00 2024-06-10T01:21:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.173+0000 7fc82879d740 10 client.0 _ll_register_callbacks cb 0x7fc81400fbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:21:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.174+0000 7fc81f7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bdaa62a00 0x564bdaa70480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.174+0000 7fc81f7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bdaa62a00 0x564bdaa70480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:37382/0 (socket says 172.21.15.7:37382) 2024-06-10T01:21:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.174+0000 7fc81f7fe640 1 -- 172.21.15.7:0/2123861538 learned_addr learned my addr 172.21.15.7:0/2123861538 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.174+0000 7fc81ffff640 1 --2- 172.21.15.7:0/2123861538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdaadf160 0x564bdaae1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.175+0000 7fc81effd640 1 --2- 172.21.15.7:0/2123861538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bdaa67b70 0x564bdaa709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.176+0000 7fc81ffff640 1 -- 172.21.15.7:0/2123861538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bdaa62a00 msgr2=0x564bdaa70480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.176+0000 7fc81ffff640 1 --2- 172.21.15.7:0/2123861538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bdaa62a00 0x564bdaa70480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.176+0000 7fc81ffff640 1 -- 172.21.15.7:0/2123861538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bdaa67b70 msgr2=0x564bdaa709c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.176+0000 7fc81ffff640 1 --2- 172.21.15.7:0/2123861538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bdaa67b70 0x564bdaa709c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.176+0000 7fc81ffff640 1 -- 172.21.15.7:0/2123861538 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bdaa73a90 con 0x564bdaadf160 2024-06-10T01:21:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.178+0000 7fc81ffff640 1 --2- 172.21.15.7:0/2123861538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdaadf160 0x564bdaae1550 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fc8180dd4e0 tx=0x7fc8180aa750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b4037b97d4f401 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.178+0000 7fc7feffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:21:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.188+0000 7fc7feffd640 1 -- 172.21.15.7:0/2123861538 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8180c9020 con 0x564bdaadf160 2024-06-10T01:21:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.188+0000 7fc7feffd640 1 -- 172.21.15.7:0/2123861538 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc8180ac6a0 con 0x564bdaadf160 2024-06-10T01:21:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.190+0000 7fc82879d740 1 -- 172.21.15.7:0/2123861538 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8180c5210 con 0x564bdaadf160 2024-06-10T01:21:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.191+0000 7fc7feffd640 1 -- 172.21.15.7:0/2123861538 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc8180c5040 con 0x564bdaadf160 2024-06-10T01:21:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.191+0000 7fc82879d740 10 client.19344 fetch_fsmap learned FSMap version 44 2024-06-10T01:21:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.191+0000 7fc82879d740 1 -- 172.21.15.7:0/2123861538 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x564bdaae2ae0 con 0x564bdaadf160 2024-06-10T01:21:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.192+0000 7fc7feffd640 1 -- 172.21.15.7:0/2123861538 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8180c5210 con 0x564bdaadf160 2024-06-10T01:21:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.192+0000 7fc7feffd640 1 -- 172.21.15.7:0/2123861538 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fc8180ab690 con 0x564bdaadf160 2024-06-10T01:21:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.193+0000 7fc82879d740 10 client.19344 fetch_fsmap finished waiting for FSMap version 44 2024-06-10T01:21:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.193+0000 7fc82879d740 10 client.19344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:21:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.193+0000 7fc82879d740 20 client.19344 populate_metadata read hostname 'smithi007' 2024-06-10T01:21:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.193+0000 7fc82879d740 1 --2- 172.21.15.7:0/2123861538 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564bdaae3550 0x564bdaae5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.193+0000 7fc82879d740 4 client.19344 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:21:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.193+0000 7fc82879d740 1 -- 172.21.15.7:0/2123861538 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564bda9d8f10 con 0x564bdaae3550 2024-06-10T01:21:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.197+0000 7fc81f7fe640 1 --2- 172.21.15.7:0/2123861538 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564bdaae3550 0x564bdaae5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:33.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:32 smithi175 ceph-mon[36636]: pgmap v898: 129 pgs: 129 active+clean; 465 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.8 MiB/s wr, 638 op/s 2024-06-10T01:21:33.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:32 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:21:33.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:32 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:21:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.209+0000 7fc81f7fe640 1 --2- 172.21.15.7:0/2123861538 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564bdaae3550 0x564bdaae5970 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cfb6724c6758824 server_cookie=6d5de8acab373fca in_seq=0 out_seq=0 2024-06-10T01:21:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.209+0000 7fc7feffd640 10 client.19344 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:33.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:32 smithi007 ceph-mon[32398]: pgmap v898: 129 pgs: 129 active+clean; 465 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.8 MiB/s wr, 638 op/s 2024-06-10T01:21:33.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:32 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:21:33.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:32 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:21:33.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:32 smithi078 ceph-mon[36683]: pgmap v898: 129 pgs: 129 active+clean; 465 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.8 MiB/s wr, 638 op/s 2024-06-10T01:21:33.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:32 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:21:33.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:32 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:21:33.519 INFO:tasks.workunit.client.0.smithi007.stdout: 10 47300 36.76 MB/sec execute 10 sec latency 338.803 ms 2024-06-10T01:21:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.982+0000 7fc7feffd640 1 -- 172.21.15.7:0/2123861538 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564bda9d8f10 con 0x564bdaae3550 2024-06-10T01:21:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.982+0000 7fc7feffd640 10 client.19344 handle_command_reply: tid=0 2024-06-10T01:21:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.982+0000 7fc82879d740 1 client.19344 shutdown 2024-06-10T01:21:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.982+0000 7fc82879d740 1 -- 172.21.15.7:0/2123861538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdaadf160 msgr2=0x564bdaae1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.982+0000 7fc82879d740 1 --2- 172.21.15.7:0/2123861538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdaadf160 0x564bdaae1550 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fc8180dd4e0 tx=0x7fc8180aa750 comp rx=0 tx=0).stop 2024-06-10T01:21:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.982+0000 7fc82879d740 1 -- 172.21.15.7:0/2123861538 shutdown_connections 2024-06-10T01:21:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.982+0000 7fc82879d740 1 --2- 172.21.15.7:0/2123861538 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564bdaae3550 0x564bdaae5970 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.982+0000 7fc82879d740 1 --2- 172.21.15.7:0/2123861538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdaadf160 0x564bdaae1550 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.982+0000 7fc82879d740 1 --2- 172.21.15.7:0/2123861538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bdaa67b70 0x564bdaa709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.982+0000 7fc82879d740 1 --2- 172.21.15.7:0/2123861538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564bdaa62a00 0x564bdaa70480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.982+0000 7fc82879d740 1 -- 172.21.15.7:0/2123861538 >> 172.21.15.7:0/2123861538 conn(0x564bdaa5d680 msgr2=0x564bdaa60650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.982+0000 7fc7feffd640 0 client.19344 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.982+0000 7fc82879d740 1 -- 172.21.15.7:0/2123861538 shutdown_connections 2024-06-10T01:21:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.983+0000 7fc82879d740 1 -- 172.21.15.7:0/2123861538 wait complete. 2024-06-10T01:21:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.983+0000 7fc82879d740 20 client.19344 trim_cache size 0 max 16384 2024-06-10T01:21:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:21:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:21:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:21:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.983+0000 7fc825a93640 1 -- 172.21.15.7:0/2108425244 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc8080a2d90 msgr2=0x7fc8080a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.983+0000 7fc825a93640 1 --2- 172.21.15.7:0/2108425244 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc8080a2d90 0x7fc8080a5250 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x564bdaa62e00 tx=0x7fc8140046c0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.983+0000 7fc825a93640 1 -- 172.21.15.7:0/2108425244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8201057e0 msgr2=0x7fc82010b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.983+0000 7fc825a93640 1 --2- 172.21.15.7:0/2108425244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8201057e0 0x7fc82010b4f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc81801fd90 tx=0x7fc81801d750 comp rx=0 tx=0).stop 2024-06-10T01:21:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.984+0000 7fc825a93640 1 -- 172.21.15.7:0/2108425244 shutdown_connections 2024-06-10T01:21:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.984+0000 7fc825a93640 1 --2- 172.21.15.7:0/2108425244 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc8080a2d90 0x7fc8080a5250 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.984+0000 7fc825a93640 1 --2- 172.21.15.7:0/2108425244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8201057e0 0x7fc82010b4f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.984+0000 7fc825a93640 1 --2- 172.21.15.7:0/2108425244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc820104e20 0x7fc8201084a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.984+0000 7fc825a93640 1 --2- 172.21.15.7:0/2108425244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc820103c30 0x7fc820107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.984+0000 7fc825a93640 1 -- 172.21.15.7:0/2108425244 >> 172.21.15.7:0/2108425244 conn(0x7fc8200ff1c0 msgr2=0x7fc820077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.984+0000 7fc825a93640 1 -- 172.21.15.7:0/2108425244 shutdown_connections 2024-06-10T01:21:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:33.984+0000 7fc825a93640 1 -- 172.21.15.7:0/2108425244 wait complete. 2024-06-10T01:21:34.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:33 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:21:34.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:33 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:21:34.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:33 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:21:34.519 INFO:tasks.workunit.client.0.smithi007.stdout: 10 47673 34.51 MB/sec execute 11 sec latency 674.725 ms 2024-06-10T01:21:35.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:34 smithi175 ceph-mon[36636]: pgmap v899: 129 pgs: 129 active+clean; 475 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.7 MiB/s wr, 712 op/s 2024-06-10T01:21:35.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:34 smithi007 ceph-mon[32398]: pgmap v899: 129 pgs: 129 active+clean; 475 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.7 MiB/s wr, 712 op/s 2024-06-10T01:21:35.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:34 smithi078 ceph-mon[36683]: pgmap v899: 129 pgs: 129 active+clean; 475 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.7 MiB/s wr, 712 op/s 2024-06-10T01:21:35.519 INFO:tasks.workunit.client.0.smithi007.stdout: 10 48346 34.00 MB/sec execute 12 sec latency 78.475 ms 2024-06-10T01:21:36.519 INFO:tasks.workunit.client.0.smithi007.stdout: 10 49147 34.67 MB/sec execute 13 sec latency 112.724 ms 2024-06-10T01:21:37.520 INFO:tasks.workunit.client.0.smithi007.stdout: 10 49778 34.60 MB/sec execute 14 sec latency 320.805 ms 2024-06-10T01:21:37.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:37 smithi175 ceph-mon[36636]: pgmap v900: 129 pgs: 129 active+clean; 477 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.3 MiB/s wr, 710 op/s 2024-06-10T01:21:37.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:37 smithi007 ceph-mon[32398]: pgmap v900: 129 pgs: 129 active+clean; 477 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.3 MiB/s wr, 710 op/s 2024-06-10T01:21:37.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:37 smithi078 ceph-mon[36683]: pgmap v900: 129 pgs: 129 active+clean; 477 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.3 MiB/s wr, 710 op/s 2024-06-10T01:21:38.520 INFO:tasks.workunit.client.0.smithi007.stdout: 10 50312 34.69 MB/sec execute 15 sec latency 83.493 ms 2024-06-10T01:21:39.520 INFO:tasks.workunit.client.0.smithi007.stdout: 10 50997 35.33 MB/sec execute 16 sec latency 169.692 ms 2024-06-10T01:21:39.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:39 smithi175 ceph-mon[36636]: pgmap v901: 129 pgs: 129 active+clean; 473 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.1 MiB/s wr, 749 op/s 2024-06-10T01:21:39.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:39 smithi007 ceph-mon[32398]: pgmap v901: 129 pgs: 129 active+clean; 473 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.1 MiB/s wr, 749 op/s 2024-06-10T01:21:39.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:39 smithi078 ceph-mon[36683]: pgmap v901: 129 pgs: 129 active+clean; 473 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.1 MiB/s wr, 749 op/s 2024-06-10T01:21:40.519 INFO:tasks.workunit.client.0.smithi007.stdout: 10 51656 34.73 MB/sec execute 17 sec latency 61.240 ms 2024-06-10T01:21:41.519 INFO:tasks.workunit.client.0.smithi007.stdout: 10 52356 34.76 MB/sec execute 18 sec latency 93.976 ms 2024-06-10T01:21:41.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:41 smithi175 ceph-mon[36636]: pgmap v902: 129 pgs: 129 active+clean; 477 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.9 MiB/s wr, 896 op/s 2024-06-10T01:21:41.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:41 smithi007 ceph-mon[32398]: pgmap v902: 129 pgs: 129 active+clean; 477 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.9 MiB/s wr, 896 op/s 2024-06-10T01:21:41.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:41 smithi078 ceph-mon[36683]: pgmap v902: 129 pgs: 129 active+clean; 477 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.9 MiB/s wr, 896 op/s 2024-06-10T01:21:42.520 INFO:tasks.workunit.client.0.smithi007.stdout: 10 52987 34.82 MB/sec execute 19 sec latency 191.616 ms 2024-06-10T01:21:43.520 INFO:tasks.workunit.client.0.smithi007.stdout: 10 53538 34.72 MB/sec execute 20 sec latency 139.458 ms 2024-06-10T01:21:43.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:43 smithi175 ceph-mon[36636]: pgmap v903: 129 pgs: 129 active+clean; 477 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 8.7 MiB/s wr, 736 op/s 2024-06-10T01:21:43.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:43 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:21:43.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:43 smithi007 ceph-mon[32398]: pgmap v903: 129 pgs: 129 active+clean; 477 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 8.7 MiB/s wr, 736 op/s 2024-06-10T01:21:43.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:43 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:21:43.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:43 smithi078 ceph-mon[36683]: pgmap v903: 129 pgs: 129 active+clean; 477 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 8.7 MiB/s wr, 736 op/s 2024-06-10T01:21:43.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:43 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:21:44.520 INFO:tasks.workunit.client.0.smithi007.stdout: 10 54181 34.94 MB/sec execute 21 sec latency 120.119 ms 2024-06-10T01:21:45.520 INFO:tasks.workunit.client.0.smithi007.stdout: 10 54852 34.91 MB/sec execute 22 sec latency 44.022 ms 2024-06-10T01:21:45.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:45 smithi175 ceph-mon[36636]: pgmap v904: 129 pgs: 129 active+clean; 485 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 872 op/s 2024-06-10T01:21:45.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:45 smithi007 ceph-mon[32398]: pgmap v904: 129 pgs: 129 active+clean; 485 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 872 op/s 2024-06-10T01:21:45.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:45 smithi078 ceph-mon[36683]: pgmap v904: 129 pgs: 129 active+clean; 485 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 872 op/s 2024-06-10T01:21:46.520 INFO:tasks.workunit.client.0.smithi007.stdout: 10 55574 34.82 MB/sec execute 23 sec latency 78.232 ms 2024-06-10T01:21:47.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.094+0000 7fdbf34b3640 1 -- 172.21.15.7:0/3931737445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbec1024a0 msgr2=0x7fdbec104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:47.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.095+0000 7fdbdffff640 1 -- 172.21.15.7:0/3931737445 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4004230 con 0x7fdbec1024a0 2024-06-10T01:21:47.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.095+0000 7fdbf34b3640 1 --2- 172.21.15.7:0/3931737445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbec1024a0 0x7fdbec104890 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009900 tx=0x7fdbe402a410 comp rx=0 tx=0).stop 2024-06-10T01:21:47.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.095+0000 7fdbf34b3640 1 -- 172.21.15.7:0/3931737445 shutdown_connections 2024-06-10T01:21:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.095+0000 7fdbf34b3640 1 --2- 172.21.15.7:0/3931737445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbec107790 0x7fdbec109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.095+0000 7fdbf34b3640 1 --2- 172.21.15.7:0/3931737445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbec104dd0 0x7fdbec1071c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.095+0000 7fdbf34b3640 1 --2- 172.21.15.7:0/3931737445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbec1024a0 0x7fdbec104890 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.095+0000 7fdbf34b3640 1 -- 172.21.15.7:0/3931737445 >> 172.21.15.7:0/3931737445 conn(0x7fdbec0fbeb0 msgr2=0x7fdbec0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.096+0000 7fdbf34b3640 1 -- 172.21.15.7:0/3931737445 shutdown_connections 2024-06-10T01:21:47.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.097+0000 7fdbf34b3640 1 -- 172.21.15.7:0/3931737445 wait complete. 2024-06-10T01:21:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.098+0000 7fdbf34b3640 1 Processor -- start 2024-06-10T01:21:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.098+0000 7fdbf34b3640 1 -- start start 2024-06-10T01:21:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.098+0000 7fdbf34b3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbec1024a0 0x7fdbec070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.098+0000 7fdbf34b3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbec104dd0 0x7fdbec0708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.098+0000 7fdbf34b3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbec107790 0x7fdbec196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.098+0000 7fdbf34b3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec10d8e0 con 0x7fdbec1024a0 2024-06-10T01:21:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.098+0000 7fdbf34b3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec10d760 con 0x7fdbec107790 2024-06-10T01:21:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.098+0000 7fdbf34b3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec10da60 con 0x7fdbec104dd0 2024-06-10T01:21:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.099+0000 7fdbf124f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbec1024a0 0x7fdbec070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.099+0000 7fdbf124f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbec1024a0 0x7fdbec070360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54742/0 (socket says 172.21.15.7:54742) 2024-06-10T01:21:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.099+0000 7fdbf124f640 1 -- 172.21.15.7:0/1869726311 learned_addr learned my addr 172.21.15.7:0/1869726311 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.099+0000 7fdbf1a50640 1 --2- 172.21.15.7:0/1869726311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbec107790 0x7fdbec196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.099+0000 7fdbf124f640 1 -- 172.21.15.7:0/1869726311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbec104dd0 msgr2=0x7fdbec0708a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-10T01:21:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.099+0000 7fdbf124f640 1 --2- 172.21.15.7:0/1869726311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbec104dd0 0x7fdbec0708a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.099+0000 7fdbf124f640 1 -- 172.21.15.7:0/1869726311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbec107790 msgr2=0x7fdbec196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.099+0000 7fdbf124f640 1 --2- 172.21.15.7:0/1869726311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbec107790 0x7fdbec196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.099+0000 7fdbf124f640 1 -- 172.21.15.7:0/1869726311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbec197520 con 0x7fdbec1024a0 2024-06-10T01:21:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.100+0000 7fdbf124f640 1 --2- 172.21.15.7:0/1869726311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbec1024a0 0x7fdbec070360 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4004cb0 tx=0x7fdbe4005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.109+0000 7fdbde7fc640 1 -- 172.21.15.7:0/1869726311 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4040030 con 0x7fdbec1024a0 2024-06-10T01:21:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.109+0000 7fdbde7fc640 1 -- 172.21.15.7:0/1869726311 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbe4031c70 con 0x7fdbec1024a0 2024-06-10T01:21:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.109+0000 7fdbde7fc640 1 -- 172.21.15.7:0/1869726311 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe403ce10 con 0x7fdbec1024a0 2024-06-10T01:21:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.109+0000 7fdbf34b3640 1 -- 172.21.15.7:0/1869726311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbec100290 con 0x7fdbec1024a0 2024-06-10T01:21:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.109+0000 7fdbf34b3640 1 -- 172.21.15.7:0/1869726311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbec1007b0 con 0x7fdbec1024a0 2024-06-10T01:21:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.114+0000 7fdbf61c4740 1 Processor -- start 2024-06-10T01:21:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.123+0000 7fdbf61c4740 1 -- start start 2024-06-10T01:21:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.123+0000 7fdbf61c4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9b5b7010 0x55ea9b5b7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.123+0000 7fdbf61c4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9b5b7a00 0x55ea9b5bc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.124+0000 7fdbf61c4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea9b5bcb70 0x55ea9b5bf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.124+0000 7fdbf61c4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ea9b51a4f0 con 0x55ea9b5b7010 2024-06-10T01:21:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.124+0000 7fdbf1a50640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9b5b7010 0x55ea9b5b7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.124+0000 7fdbf1a50640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9b5b7010 0x55ea9b5b7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54758/0 (socket says 172.21.15.7:54758) 2024-06-10T01:21:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.124+0000 7fdbf1a50640 1 -- 172.21.15.7:0/645671524 learned_addr learned my addr 172.21.15.7:0/645671524 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.124+0000 7fdbf124f640 1 --2- 172.21.15.7:0/645671524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9b5b7a00 0x55ea9b5bc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.124+0000 7fdbf61c4740 1 -- 172.21.15.7:0/645671524 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ea9b4b8870 con 0x55ea9b5bcb70 2024-06-10T01:21:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.124+0000 7fdbf61c4740 1 -- 172.21.15.7:0/645671524 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ea9b5164b0 con 0x55ea9b5b7a00 2024-06-10T01:21:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.125+0000 7fdbde7fc640 1 -- 172.21.15.7:0/1869726311 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fdbe4043030 con 0x7fdbec1024a0 2024-06-10T01:21:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.126+0000 7fdbde7fc640 1 --2- 172.21.15.7:0/1869726311 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdbd80a2da0 0x7fdbd80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.126+0000 7fdbf1a50640 1 -- 172.21.15.7:0/645671524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9b5b7a00 msgr2=0x55ea9b5bc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.126+0000 7fdbf1a50640 1 --2- 172.21.15.7:0/645671524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9b5b7a00 0x55ea9b5bc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.126+0000 7fdbf1a50640 1 -- 172.21.15.7:0/645671524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea9b5bcb70 msgr2=0x55ea9b5bf000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T01:21:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.126+0000 7fdbf1a50640 1 --2- 172.21.15.7:0/645671524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea9b5bcb70 0x55ea9b5bf000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.126+0000 7fdbf1a50640 1 -- 172.21.15.7:0/645671524 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea9b4d49b0 con 0x55ea9b5b7010 2024-06-10T01:21:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.126+0000 7fdbde7fc640 1 -- 172.21.15.7:0/1869726311 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fdbe40effe0 con 0x7fdbec1024a0 2024-06-10T01:21:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.127+0000 7fdbf1a50640 1 --2- 172.21.15.7:0/645671524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9b5b7010 0x55ea9b5b7430 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800b7a0 tx=0x7fdbe800bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e6891df90076 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.127+0000 7fdbf1a50640 1 --2- 172.21.15.7:0/1869726311 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdbd80a2da0 0x7fdbd80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.127+0000 7fdbd77fe640 1 -- 172.21.15.7:0/645671524 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe800de10 con 0x55ea9b5b7010 2024-06-10T01:21:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.128+0000 7fdbd77fe640 1 -- 172.21.15.7:0/645671524 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbe8012900 con 0x55ea9b5b7010 2024-06-10T01:21:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.128+0000 7fdbd77fe640 1 -- 172.21.15.7:0/645671524 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8015bc0 con 0x55ea9b5b7010 2024-06-10T01:21:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.128+0000 7fdbf124f640 1 --2- 172.21.15.7:0/645671524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9b5b7a00 0x55ea9b5bc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:21:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.129+0000 7fdbf61c4740 1 -- 172.21.15.7:0/645671524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9b5b7010 msgr2=0x55ea9b5b7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.129+0000 7fdbf61c4740 1 --2- 172.21.15.7:0/645671524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9b5b7010 0x55ea9b5b7430 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800b7a0 tx=0x7fdbe800bc60 comp rx=0 tx=0).stop 2024-06-10T01:21:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.132+0000 7fdbf61c4740 1 -- 172.21.15.7:0/645671524 shutdown_connections 2024-06-10T01:21:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.132+0000 7fdbf61c4740 1 --2- 172.21.15.7:0/645671524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea9b5bcb70 0x55ea9b5bf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.132+0000 7fdbf61c4740 1 --2- 172.21.15.7:0/645671524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9b5b7a00 0x55ea9b5bc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.132+0000 7fdbf61c4740 1 --2- 172.21.15.7:0/645671524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9b5b7010 0x55ea9b5b7430 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.132+0000 7fdbf61c4740 1 -- 172.21.15.7:0/645671524 >> 172.21.15.7:0/645671524 conn(0x55ea9b5b2680 msgr2=0x55ea9b5b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.136+0000 7fdbf1a50640 1 --2- 172.21.15.7:0/1869726311 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdbd80a2da0 0x7fdbd80a5260 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fdbe801c000 tx=0x7fdbe801f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.139+0000 7fdbf61c4740 1 -- 172.21.15.7:0/645671524 shutdown_connections 2024-06-10T01:21:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.141+0000 7fdbf61c4740 1 -- 172.21.15.7:0/645671524 wait complete. 2024-06-10T01:21:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.142+0000 7fdbf61c4740 1 -- start start 2024-06-10T01:21:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.142+0000 7fdbf61c4740 1 Processor -- start 2024-06-10T01:21:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.145+0000 7fdbf61c4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9b5b7010 0x55ea9b5c1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.145+0000 7fdbf61c4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea9b5b7a00 0x55ea9b5c1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.145+0000 7fdbf61c4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9b5bcb70 0x55ea9b5c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.145+0000 7fdbf61c4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ea9b5bfb20 con 0x55ea9b5bcb70 2024-06-10T01:21:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.145+0000 7fdbf61c4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ea9b4a3ae0 con 0x55ea9b5b7a00 2024-06-10T01:21:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.145+0000 7fdbf61c4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ea9b5bfca0 con 0x55ea9b5b7010 2024-06-10T01:21:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.145+0000 7fdbf1a50640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9b5bcb70 0x55ea9b5c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.145+0000 7fdbf1a50640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9b5bcb70 0x55ea9b5c87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54772/0 (socket says 172.21.15.7:54772) 2024-06-10T01:21:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.145+0000 7fdbf1a50640 1 -- 172.21.15.7:0/313858010 learned_addr learned my addr 172.21.15.7:0/313858010 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.145+0000 7fdbf61c4740 10 client.0 _ll_register_callbacks cb 0x55ea9b4dc9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:21:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.146+0000 7fdbf1a50640 1 -- 172.21.15.7:0/313858010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9b5b7010 msgr2=0x55ea9b5c1220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:21:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.146+0000 7fdbf1a50640 1 --2- 172.21.15.7:0/313858010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9b5b7010 0x55ea9b5c1220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.146+0000 7fdbf1a50640 1 -- 172.21.15.7:0/313858010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea9b5b7a00 msgr2=0x55ea9b5c1760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:21:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.146+0000 7fdbf1a50640 1 --2- 172.21.15.7:0/313858010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea9b5b7a00 0x55ea9b5c1760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.146+0000 7fdbf1a50640 1 -- 172.21.15.7:0/313858010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea9b22e210 con 0x55ea9b5bcb70 2024-06-10T01:21:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.149+0000 7fdbf1a50640 1 --2- 172.21.15.7:0/313858010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9b5bcb70 0x55ea9b5c87a0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8015b50 tx=0x7fdbe8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c218f9d2e3335aa server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.150+0000 7fdbd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:21:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.151+0000 7fdbd4ff9640 1 -- 172.21.15.7:0/313858010 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe800ddd0 con 0x55ea9b5bcb70 2024-06-10T01:21:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.151+0000 7fdbd4ff9640 1 -- 172.21.15.7:0/313858010 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fdbe8015db0 con 0x55ea9b5bcb70 2024-06-10T01:21:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.151+0000 7fdbf61c4740 1 -- 172.21.15.7:0/313858010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbe8012900 con 0x55ea9b5bcb70 2024-06-10T01:21:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.152+0000 7fdbd4ff9640 1 -- 172.21.15.7:0/313858010 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdbe8012b20 con 0x55ea9b5bcb70 2024-06-10T01:21:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.155+0000 7fdbf61c4740 10 client.19356 fetch_fsmap learned FSMap version 45 2024-06-10T01:21:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.155+0000 7fdbf61c4740 1 -- 172.21.15.7:0/313858010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ea9b2fa620 con 0x55ea9b5bcb70 2024-06-10T01:21:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.161+0000 7fdbd4ff9640 1 -- 172.21.15.7:0/313858010 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbe8012900 con 0x55ea9b5bcb70 2024-06-10T01:21:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.162+0000 7fdbd4ff9640 1 -- 172.21.15.7:0/313858010 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 45) ==== 8358+0+0 (secure 0 0 0) 0x7fdbe8013070 con 0x55ea9b5bcb70 2024-06-10T01:21:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.162+0000 7fdbf61c4740 10 client.19356 fetch_fsmap finished waiting for FSMap version 45 2024-06-10T01:21:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.162+0000 7fdbf61c4740 10 client.19356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:21:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.162+0000 7fdbf61c4740 20 client.19356 populate_metadata read hostname 'smithi007' 2024-06-10T01:21:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.162+0000 7fdbf61c4740 1 --2- 172.21.15.7:0/313858010 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ea9b5dde10 0x55ea9b5e0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.162+0000 7fdbf61c4740 4 client.19356 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:21:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.162+0000 7fdbf61c4740 1 -- 172.21.15.7:0/313858010 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea9b52df10 con 0x55ea9b5dde10 2024-06-10T01:21:47.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.166+0000 7fdbf124f640 1 --2- 172.21.15.7:0/313858010 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ea9b5dde10 0x55ea9b5e0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.174+0000 7fdbf124f640 1 --2- 172.21.15.7:0/313858010 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ea9b5dde10 0x55ea9b5e0290 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=640085d92ca66a8c server_cookie=48dd8c456ed36dee in_seq=0 out_seq=0 2024-06-10T01:21:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.174+0000 7fdbd4ff9640 10 client.19356 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.178+0000 7fdbd4ff9640 1 -- 172.21.15.7:0/313858010 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ea9b52df10 con 0x55ea9b5dde10 2024-06-10T01:21:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.178+0000 7fdbd4ff9640 10 client.19356 handle_command_reply: tid=0 2024-06-10T01:21:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.178+0000 7fdbf61c4740 1 client.19356 shutdown 2024-06-10T01:21:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.180+0000 7fdbf61c4740 1 -- 172.21.15.7:0/313858010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9b5bcb70 msgr2=0x55ea9b5c87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.180+0000 7fdbf61c4740 1 --2- 172.21.15.7:0/313858010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9b5bcb70 0x55ea9b5c87a0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8015b50 tx=0x7fdbe8011000 comp rx=0 tx=0).stop 2024-06-10T01:21:47.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.188+0000 7fdbf61c4740 1 -- 172.21.15.7:0/313858010 shutdown_connections 2024-06-10T01:21:47.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.188+0000 7fdbf61c4740 1 --2- 172.21.15.7:0/313858010 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ea9b5dde10 0x55ea9b5e0290 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:47.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.188+0000 7fdbf61c4740 1 --2- 172.21.15.7:0/313858010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9b5bcb70 0x55ea9b5c87a0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:47.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.188+0000 7fdbf61c4740 1 --2- 172.21.15.7:0/313858010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea9b5b7a00 0x55ea9b5c1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:47.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.188+0000 7fdbf61c4740 1 --2- 172.21.15.7:0/313858010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9b5b7010 0x55ea9b5c1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:47.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.188+0000 7fdbf61c4740 1 -- 172.21.15.7:0/313858010 >> 172.21.15.7:0/313858010 conn(0x55ea9b5b2680 msgr2=0x55ea9b5b17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:47.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.189+0000 7fdbd4ff9640 0 client.19356 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.194+0000 7fdbf61c4740 1 -- 172.21.15.7:0/313858010 shutdown_connections 2024-06-10T01:21:47.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.198+0000 7fdbf61c4740 1 -- 172.21.15.7:0/313858010 wait complete. 2024-06-10T01:21:47.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.198+0000 7fdbf61c4740 20 client.19356 trim_cache size 0 max 16384 2024-06-10T01:21:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.231+0000 7fdbf61c4740 1 Processor -- start 2024-06-10T01:21:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.232+0000 7fdbf61c4740 1 -- start start 2024-06-10T01:21:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.232+0000 7fdbf61c4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9b5b7010 0x55ea9b5b8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.232+0000 7fdbf61c4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea9b5b7a00 0x55ea9b5c8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.232+0000 7fdbf61c4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9b5bcb70 0x55ea9b5b9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.232+0000 7fdbf61c4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd0012460 con 0x55ea9b5bcb70 2024-06-10T01:21:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.232+0000 7fdbf61c4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd000fcd0 con 0x55ea9b5b7a00 2024-06-10T01:21:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.232+0000 7fdbf61c4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd000db20 con 0x55ea9b5b7010 2024-06-10T01:21:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.232+0000 7fdbf1a50640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9b5bcb70 0x55ea9b5b9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.232+0000 7fdbf124f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9b5b7010 0x55ea9b5b8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.232+0000 7fdbf1a50640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9b5bcb70 0x55ea9b5b9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54774/0 (socket says 172.21.15.7:54774) 2024-06-10T01:21:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.232+0000 7fdbf1a50640 1 -- 172.21.15.7:0/3082631451 learned_addr learned my addr 172.21.15.7:0/3082631451 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.232+0000 7fdbf124f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9b5b7010 0x55ea9b5b8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:38614/0 (socket says 172.21.15.7:38614) 2024-06-10T01:21:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.233+0000 7fdbf1a50640 1 -- 172.21.15.7:0/3082631451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9b5b7010 msgr2=0x55ea9b5b8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.233+0000 7fdbf1a50640 1 --2- 172.21.15.7:0/3082631451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9b5b7010 0x55ea9b5b8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.233+0000 7fdbf1a50640 1 -- 172.21.15.7:0/3082631451 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea9b5b7a00 msgr2=0x55ea9b5c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.233+0000 7fdbf1a50640 1 --2- 172.21.15.7:0/3082631451 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea9b5b7a00 0x55ea9b5c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.233+0000 7fdbf1a50640 1 -- 172.21.15.7:0/3082631451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea9b5b24b0 con 0x55ea9b5bcb70 2024-06-10T01:21:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.234+0000 7fdbf124f640 1 --2- 172.21.15.7:0/3082631451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9b5b7010 0x55ea9b5b8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:21:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.235+0000 7fdbf1a50640 1 --2- 172.21.15.7:0/3082631451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9b5bcb70 0x55ea9b5b9080 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80208c0 tx=0x7fdbe800d950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f8bae37f89d21f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:47.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.236+0000 7fdbd6ffd640 1 -- 172.21.15.7:0/3082631451 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8038070 con 0x55ea9b5bcb70 2024-06-10T01:21:47.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.236+0000 7fdbd6ffd640 1 -- 172.21.15.7:0/3082631451 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbe8020e20 con 0x55ea9b5bcb70 2024-06-10T01:21:47.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.236+0000 7fdbd6ffd640 1 -- 172.21.15.7:0/3082631451 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8012cc0 con 0x55ea9b5bcb70 2024-06-10T01:21:47.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.237+0000 7fdbf61c4740 1 -- 172.21.15.7:0/3082631451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9b5bcb70 msgr2=0x55ea9b5b9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:47.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.237+0000 7fdbf61c4740 1 --2- 172.21.15.7:0/3082631451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9b5bcb70 0x55ea9b5b9080 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80208c0 tx=0x7fdbe800d950 comp rx=0 tx=0).stop 2024-06-10T01:21:47.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.238+0000 7fdbf61c4740 1 -- 172.21.15.7:0/3082631451 shutdown_connections 2024-06-10T01:21:47.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.238+0000 7fdbf61c4740 1 --2- 172.21.15.7:0/3082631451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9b5bcb70 0x55ea9b5b9080 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:47.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.238+0000 7fdbf61c4740 1 --2- 172.21.15.7:0/3082631451 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea9b5b7a00 0x55ea9b5c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:47.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.238+0000 7fdbf61c4740 1 --2- 172.21.15.7:0/3082631451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9b5b7010 0x55ea9b5b8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:47.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.238+0000 7fdbf61c4740 1 -- 172.21.15.7:0/3082631451 >> 172.21.15.7:0/3082631451 conn(0x55ea9b5b2680 msgr2=0x55ea9b5c1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.241+0000 7fdbf61c4740 1 -- 172.21.15.7:0/3082631451 shutdown_connections 2024-06-10T01:21:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.243+0000 7fdbf61c4740 1 -- 172.21.15.7:0/3082631451 wait complete. 2024-06-10T01:21:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.244+0000 7fdbf61c4740 1 -- start start 2024-06-10T01:21:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.244+0000 7fdbf61c4740 1 Processor -- start 2024-06-10T01:21:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.244+0000 7fdbf61c4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9b5b7010 0x55ea9b5c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.244+0000 7fdbf61c4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9b5b7a00 0x55ea9b5c4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.244+0000 7fdbf61c4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea9b5bcb70 0x55ea9b5c5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.244+0000 7fdbf61c4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd000e190 con 0x55ea9b5b7a00 2024-06-10T01:21:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.244+0000 7fdbf61c4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd000f600 con 0x55ea9b5bcb70 2024-06-10T01:21:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.244+0000 7fdbf61c4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd000e860 con 0x55ea9b5b7010 2024-06-10T01:21:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.244+0000 7fdbf61c4740 10 client.0 _ll_register_callbacks cb 0x7fdbe800b5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:21:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.245+0000 7fdbf0a4e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9b5b7a00 0x55ea9b5c4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.245+0000 7fdbf0a4e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9b5b7a00 0x55ea9b5c4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54776/0 (socket says 172.21.15.7:54776) 2024-06-10T01:21:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.245+0000 7fdbf0a4e640 1 -- 172.21.15.7:0/2593657424 learned_addr learned my addr 172.21.15.7:0/2593657424 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.246+0000 7fdbf1a50640 1 --2- 172.21.15.7:0/2593657424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea9b5bcb70 0x55ea9b5c5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.246+0000 7fdbf124f640 1 --2- 172.21.15.7:0/2593657424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9b5b7010 0x55ea9b5c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.249+0000 7fdbf0a4e640 1 -- 172.21.15.7:0/2593657424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9b5b7010 msgr2=0x55ea9b5c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.249+0000 7fdbf0a4e640 1 --2- 172.21.15.7:0/2593657424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9b5b7010 0x55ea9b5c83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.249+0000 7fdbf0a4e640 1 -- 172.21.15.7:0/2593657424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea9b5bcb70 msgr2=0x55ea9b5c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.249+0000 7fdbf0a4e640 1 --2- 172.21.15.7:0/2593657424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea9b5bcb70 0x55ea9b5c5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.249+0000 7fdbf0a4e640 1 -- 172.21.15.7:0/2593657424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea9b5c58f0 con 0x55ea9b5b7a00 2024-06-10T01:21:47.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.251+0000 7fdbf0a4e640 1 --2- 172.21.15.7:0/2593657424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9b5b7a00 0x55ea9b5c4e40 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000b6b0 tx=0x7fdbe000bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a47ac6525f1dd94 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:47.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.252+0000 7fdbd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:21:47.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.252+0000 7fdbd4ff9640 1 -- 172.21.15.7:0/2593657424 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0019060 con 0x55ea9b5b7a00 2024-06-10T01:21:47.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.252+0000 7fdbf61c4740 1 -- 172.21.15.7:0/2593657424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbe8020e20 con 0x55ea9b5b7a00 2024-06-10T01:21:47.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.252+0000 7fdbd4ff9640 1 -- 172.21.15.7:0/2593657424 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fdbe000c820 con 0x55ea9b5b7a00 2024-06-10T01:21:47.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.253+0000 7fdbd4ff9640 1 -- 172.21.15.7:0/2593657424 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdbe0020050 con 0x55ea9b5b7a00 2024-06-10T01:21:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.253+0000 7fdbf61c4740 10 client.19362 fetch_fsmap learned FSMap version 45 2024-06-10T01:21:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.253+0000 7fdbf61c4740 1 -- 172.21.15.7:0/2593657424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ea9b5c8fb0 con 0x55ea9b5b7a00 2024-06-10T01:21:47.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.255+0000 7fdbd4ff9640 1 -- 172.21.15.7:0/2593657424 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbe8020e20 con 0x55ea9b5b7a00 2024-06-10T01:21:47.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.256+0000 7fdbd4ff9640 1 -- 172.21.15.7:0/2593657424 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 45) ==== 8358+0+0 (secure 0 0 0) 0x7fdbe0012cd0 con 0x55ea9b5b7a00 2024-06-10T01:21:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.256+0000 7fdbf61c4740 10 client.19362 fetch_fsmap finished waiting for FSMap version 45 2024-06-10T01:21:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.256+0000 7fdbf61c4740 10 client.19362 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:21:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.256+0000 7fdbf61c4740 20 client.19362 populate_metadata read hostname 'smithi007' 2024-06-10T01:21:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.256+0000 7fdbf61c4740 1 --2- 172.21.15.7:0/2593657424 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ea9b5c99f0 0x55ea9b5cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.256+0000 7fdbf61c4740 4 client.19362 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:21:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.256+0000 7fdbf61c4740 1 -- 172.21.15.7:0/2593657424 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ea9b527c20 con 0x55ea9b5c99f0 2024-06-10T01:21:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.261+0000 7fdbf124f640 1 --2- 172.21.15.7:0/2593657424 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ea9b5c99f0 0x55ea9b5cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.270+0000 7fdbf124f640 1 --2- 172.21.15.7:0/2593657424 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ea9b5c99f0 0x55ea9b5cbe10 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9d955374f7188e7 server_cookie=6c372af202edf733 in_seq=0 out_seq=0 2024-06-10T01:21:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:47.270+0000 7fdbd4ff9640 10 client.19362 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:47.520 INFO:tasks.workunit.client.0.smithi007.stdout: 10 56018 34.65 MB/sec execute 24 sec latency 394.817 ms 2024-06-10T01:21:47.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:47 smithi175 ceph-mon[36636]: pgmap v905: 129 pgs: 129 active+clean; 488 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 9.5 MiB/s wr, 819 op/s 2024-06-10T01:21:47.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:47 smithi007 ceph-mon[32398]: pgmap v905: 129 pgs: 129 active+clean; 488 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 9.5 MiB/s wr, 819 op/s 2024-06-10T01:21:47.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:47 smithi078 ceph-mon[36683]: pgmap v905: 129 pgs: 129 active+clean; 488 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 9.5 MiB/s wr, 819 op/s 2024-06-10T01:21:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:48.119+0000 7fdbd4ff9640 1 -- 172.21.15.7:0/2593657424 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ea9b527c20 con 0x55ea9b5c99f0 2024-06-10T01:21:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:48.119+0000 7fdbd4ff9640 10 client.19362 handle_command_reply: tid=0 2024-06-10T01:21:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:48.119+0000 7fdbf61c4740 1 client.19362 shutdown 2024-06-10T01:21:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:48.119+0000 7fdbf61c4740 1 -- 172.21.15.7:0/2593657424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9b5b7a00 msgr2=0x55ea9b5c4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:48.119+0000 7fdbf61c4740 1 --2- 172.21.15.7:0/2593657424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9b5b7a00 0x55ea9b5c4e40 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000b6b0 tx=0x7fdbe000bb70 comp rx=0 tx=0).stop 2024-06-10T01:21:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:48.119+0000 7fdbf61c4740 1 -- 172.21.15.7:0/2593657424 shutdown_connections 2024-06-10T01:21:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:48.119+0000 7fdbf61c4740 1 --2- 172.21.15.7:0/2593657424 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ea9b5c99f0 0x55ea9b5cbe10 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:48.119+0000 7fdbf61c4740 1 --2- 172.21.15.7:0/2593657424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea9b5bcb70 0x55ea9b5c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:48.119+0000 7fdbf61c4740 1 --2- 172.21.15.7:0/2593657424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9b5b7a00 0x55ea9b5c4e40 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:48.119+0000 7fdbf61c4740 1 --2- 172.21.15.7:0/2593657424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9b5b7010 0x55ea9b5c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:48.119+0000 7fdbf61c4740 1 -- 172.21.15.7:0/2593657424 >> 172.21.15.7:0/2593657424 conn(0x55ea9b5b2680 msgr2=0x55ea9b5b5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:48.120+0000 7fdbd4ff9640 0 client.19362 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:48.120+0000 7fdbf61c4740 1 -- 172.21.15.7:0/2593657424 shutdown_connections 2024-06-10T01:21:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:48.120+0000 7fdbf61c4740 1 -- 172.21.15.7:0/2593657424 wait complete. 2024-06-10T01:21:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:48.120+0000 7fdbf61c4740 20 client.19362 trim_cache size 0 max 16384 2024-06-10T01:21:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:21:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:21:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:21:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:48.121+0000 7fdbf34b3640 1 -- 172.21.15.7:0/1869726311 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdbd80a2da0 msgr2=0x7fdbd80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:48.121+0000 7fdbf34b3640 1 --2- 172.21.15.7:0/1869726311 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdbd80a2da0 0x7fdbd80a5260 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fdbe801c000 tx=0x7fdbe801f000 comp rx=0 tx=0).stop 2024-06-10T01:21:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:48.121+0000 7fdbf34b3640 1 -- 172.21.15.7:0/1869726311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbec1024a0 msgr2=0x7fdbec070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:48.121+0000 7fdbf34b3640 1 --2- 172.21.15.7:0/1869726311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbec1024a0 0x7fdbec070360 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4004cb0 tx=0x7fdbe4005620 comp rx=0 tx=0).stop 2024-06-10T01:21:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:48.121+0000 7fdbf34b3640 1 -- 172.21.15.7:0/1869726311 shutdown_connections 2024-06-10T01:21:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:48.121+0000 7fdbf34b3640 1 --2- 172.21.15.7:0/1869726311 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdbd80a2da0 0x7fdbd80a5260 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:48.121+0000 7fdbf34b3640 1 --2- 172.21.15.7:0/1869726311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbec107790 0x7fdbec196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:48.121+0000 7fdbf34b3640 1 --2- 172.21.15.7:0/1869726311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbec104dd0 0x7fdbec0708a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:48.121+0000 7fdbf34b3640 1 --2- 172.21.15.7:0/1869726311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbec1024a0 0x7fdbec070360 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:48.121+0000 7fdbf34b3640 1 -- 172.21.15.7:0/1869726311 >> 172.21.15.7:0/1869726311 conn(0x7fdbec0fbeb0 msgr2=0x7fdbec102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:48.121+0000 7fdbf34b3640 1 -- 172.21.15.7:0/1869726311 shutdown_connections 2024-06-10T01:21:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:21:48.121+0000 7fdbf34b3640 1 -- 172.21.15.7:0/1869726311 wait complete. 2024-06-10T01:21:48.520 INFO:tasks.workunit.client.0.smithi007.stdout: 10 56302 33.76 MB/sec execute 25 sec latency 384.523 ms 2024-06-10T01:21:49.520 INFO:tasks.workunit.client.0.smithi007.stdout: 10 56950 33.76 MB/sec execute 26 sec latency 81.986 ms 2024-06-10T01:21:49.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:49 smithi175 ceph-mon[36636]: pgmap v906: 129 pgs: 129 active+clean; 491 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 8.8 MiB/s wr, 682 op/s 2024-06-10T01:21:49.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:49 smithi007 ceph-mon[32398]: pgmap v906: 129 pgs: 129 active+clean; 491 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 8.8 MiB/s wr, 682 op/s 2024-06-10T01:21:49.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:49 smithi078 ceph-mon[36683]: pgmap v906: 129 pgs: 129 active+clean; 491 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 8.8 MiB/s wr, 682 op/s 2024-06-10T01:21:50.520 INFO:tasks.workunit.client.0.smithi007.stdout: 10 57662 34.06 MB/sec execute 27 sec latency 78.893 ms 2024-06-10T01:21:51.520 INFO:tasks.workunit.client.0.smithi007.stdout: 10 58526 34.01 MB/sec execute 28 sec latency 56.688 ms 2024-06-10T01:21:51.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:51 smithi007 ceph-mon[32398]: pgmap v907: 129 pgs: 129 active+clean; 492 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 817 op/s 2024-06-10T01:21:51.790 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:21:51.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:51 smithi078 ceph-mon[36683]: pgmap v907: 129 pgs: 129 active+clean; 492 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 817 op/s 2024-06-10T01:21:51.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:51 smithi175 ceph-mon[36636]: pgmap v907: 129 pgs: 129 active+clean; 492 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 817 op/s 2024-06-10T01:21:52.187 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:21:52.520 INFO:tasks.workunit.client.0.smithi007.stdout: 10 58943 33.75 MB/sec execute 29 sec latency 329.588 ms 2024-06-10T01:21:52.653 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:52 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:21:52.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:52 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:21:52.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:52 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:21:53.520 INFO:tasks.workunit.client.0.smithi007.stdout: 10 59457 33.77 MB/sec execute 30 sec latency 351.300 ms 2024-06-10T01:21:53.759 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:53 smithi007 ceph-mon[32398]: pgmap v908: 129 pgs: 129 active+clean; 495 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.4 MiB/s wr, 759 op/s 2024-06-10T01:21:53.759 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:53 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:21:53.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:53 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:21:53.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:53 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:21:53.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:53 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:21:53.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:53 smithi078 ceph-mon[36683]: pgmap v908: 129 pgs: 129 active+clean; 495 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.4 MiB/s wr, 759 op/s 2024-06-10T01:21:53.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:53 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:21:53.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:53 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:21:53.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:53 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:21:53.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:53 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:21:53.852 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:53 smithi175 ceph-mon[36636]: pgmap v908: 129 pgs: 129 active+clean; 495 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.4 MiB/s wr, 759 op/s 2024-06-10T01:21:53.852 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:53 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:21:53.852 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:53 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:21:53.852 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:53 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:21:53.853 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:53 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:21:54.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.162+0000 7f6374703700 1 -- 172.21.15.7:0/2479975800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f636c107870 msgr2=0x7f636c10f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:54.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.162+0000 7f6374703700 1 --2- 172.21.15.7:0/2479975800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f636c107870 0x7f636c10f3f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f636400b0d0 tx=0x7f636400b3e0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.162+0000 7f6374703700 1 -- 172.21.15.7:0/2479975800 shutdown_connections 2024-06-10T01:21:54.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.162+0000 7f6374703700 1 --2- 172.21.15.7:0/2479975800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f636c107870 0x7f636c10f3f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.163+0000 7f6374703700 1 --2- 172.21.15.7:0/2479975800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f636c1027a0 0x7f636c102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.163+0000 7f6374703700 1 --2- 172.21.15.7:0/2479975800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f636c103150 0x7f636c107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.163+0000 7f6374703700 1 -- 172.21.15.7:0/2479975800 >> 172.21.15.7:0/2479975800 conn(0x7f636c079570 msgr2=0x7f636c079980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:54.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.163+0000 7f6374703700 1 -- 172.21.15.7:0/2479975800 shutdown_connections 2024-06-10T01:21:54.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.163+0000 7f6374703700 1 -- 172.21.15.7:0/2479975800 wait complete. 2024-06-10T01:21:54.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.164+0000 7f6374703700 1 Processor -- start 2024-06-10T01:21:54.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.164+0000 7f6374703700 1 -- start start 2024-06-10T01:21:54.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.165+0000 7f6374703700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f636c1027a0 0x7f636c19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:54.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.165+0000 7f6374703700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f636c103150 0x7f636c19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:54.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.165+0000 7f6374703700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f636c107870 0x7f636c1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:54.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.165+0000 7f6371c9e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f636c103150 0x7f636c19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:54.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.165+0000 7f637249f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f636c1027a0 0x7f636c19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:54.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.165+0000 7f6371c9e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f636c103150 0x7f636c19ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:52220/0 (socket says 172.21.15.7:52220) 2024-06-10T01:21:54.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.165+0000 7f6371c9e700 1 -- 172.21.15.7:0/226877427 learned_addr learned my addr 172.21.15.7:0/226877427 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:54.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.166+0000 7f6374703700 1 -- 172.21.15.7:0/226877427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f636c112800 con 0x7f636c107870 2024-06-10T01:21:54.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.166+0000 7f6374703700 1 -- 172.21.15.7:0/226877427 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f636c112680 con 0x7f636c103150 2024-06-10T01:21:54.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.166+0000 7f6374703700 1 -- 172.21.15.7:0/226877427 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f636c112980 con 0x7f636c1027a0 2024-06-10T01:21:54.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.166+0000 7f6372ca0700 1 --2- 172.21.15.7:0/226877427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f636c107870 0x7f636c1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:54.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.166+0000 7f6372ca0700 1 -- 172.21.15.7:0/226877427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f636c1027a0 msgr2=0x7f636c19e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:54.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.166+0000 7f6372ca0700 1 --2- 172.21.15.7:0/226877427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f636c1027a0 0x7f636c19e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.166+0000 7f6372ca0700 1 -- 172.21.15.7:0/226877427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f636c103150 msgr2=0x7f636c19ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:54.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.166+0000 7f6372ca0700 1 --2- 172.21.15.7:0/226877427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f636c103150 0x7f636c19ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.166+0000 7f6372ca0700 1 -- 172.21.15.7:0/226877427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f636c199120 con 0x7f636c107870 2024-06-10T01:21:54.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.166+0000 7f6371c9e700 1 --2- 172.21.15.7:0/226877427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f636c103150 0x7f636c19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:21:54.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.166+0000 7f6372ca0700 1 --2- 172.21.15.7:0/226877427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f636c107870 0x7f636c1989c0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f636400d010 tx=0x7f6364009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:54.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.167+0000 7f63637fe700 1 -- 172.21.15.7:0/226877427 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364003c00 con 0x7f636c107870 2024-06-10T01:21:54.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.167+0000 7f63637fe700 1 -- 172.21.15.7:0/226877427 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6364020070 con 0x7f636c107870 2024-06-10T01:21:54.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.167+0000 7f63637fe700 1 -- 172.21.15.7:0/226877427 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364005510 con 0x7f636c107870 2024-06-10T01:21:54.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.167+0000 7f6374703700 1 -- 172.21.15.7:0/226877427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f636c1993b0 con 0x7f636c107870 2024-06-10T01:21:54.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.167+0000 7f6374703700 1 -- 172.21.15.7:0/226877427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f636c0746c0 con 0x7f636c107870 2024-06-10T01:21:54.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.171+0000 7f63637fe700 1 -- 172.21.15.7:0/226877427 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f63640056b0 con 0x7f636c107870 2024-06-10T01:21:54.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.172+0000 7f63637fe700 1 --2- 172.21.15.7:0/226877427 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f63580a2ca0 0x7f63580a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:54.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.173+0000 7f63637fe700 1 -- 172.21.15.7:0/226877427 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f636400f020 con 0x7f636c107870 2024-06-10T01:21:54.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.173+0000 7f637249f700 1 --2- 172.21.15.7:0/226877427 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f63580a2ca0 0x7f63580a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:54.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.173+0000 7f637ef86b80 1 Processor -- start 2024-06-10T01:21:54.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.173+0000 7f637ef86b80 1 -- start start 2024-06-10T01:21:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.174+0000 7f637ef86b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c626b19f60 0x55c626b1a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:54.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.174+0000 7f637ef86b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c626b1a880 0x55c626b14260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:54.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.174+0000 7f637ef86b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c626b147a0 0x55c626b14bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:54.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.174+0000 7f6371c9e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c626b1a880 0x55c626b14260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:54.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.174+0000 7f637ef86b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c626a06f60 con 0x55c626b1a880 2024-06-10T01:21:54.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.174+0000 7f6371c9e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c626b1a880 0x55c626b14260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52394/0 (socket says 172.21.15.7:52394) 2024-06-10T01:21:54.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.174+0000 7f637ef86b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c626a65850 con 0x55c626b147a0 2024-06-10T01:21:54.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.174+0000 7f6371c9e700 1 -- 172.21.15.7:0/1219663792 learned_addr learned my addr 172.21.15.7:0/1219663792 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:54.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.174+0000 7f637ef86b80 1 -- 172.21.15.7:0/1219663792 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c6269cd460 con 0x55c626b19f60 2024-06-10T01:21:54.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.174+0000 7f6372ca0700 1 --2- 172.21.15.7:0/1219663792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c626b147a0 0x55c626b14bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:54.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.175+0000 7f6372ca0700 1 -- 172.21.15.7:0/1219663792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c626b19f60 msgr2=0x55c626b1a340 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:21:54.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.175+0000 7f6372ca0700 1 --2- 172.21.15.7:0/1219663792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c626b19f60 0x55c626b1a340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.175+0000 7f6372ca0700 1 -- 172.21.15.7:0/1219663792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c626b1a880 msgr2=0x55c626b14260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:54.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.175+0000 7f6372ca0700 1 --2- 172.21.15.7:0/1219663792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c626b1a880 0x55c626b14260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.175+0000 7f6372ca0700 1 -- 172.21.15.7:0/1219663792 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6269b48d0 con 0x55c626b147a0 2024-06-10T01:21:54.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.175+0000 7f6372ca0700 1 --2- 172.21.15.7:0/1219663792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c626b147a0 0x55c626b14bc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f636409dbc0 tx=0x7f636409d2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=349ad409ef432226 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:54.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.175+0000 7f6360ff9700 1 -- 172.21.15.7:0/1219663792 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63640ba070 con 0x55c626b147a0 2024-06-10T01:21:54.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.175+0000 7f6360ff9700 1 -- 172.21.15.7:0/1219663792 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63640a5b10 con 0x55c626b147a0 2024-06-10T01:21:54.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.175+0000 7f6360ff9700 1 -- 172.21.15.7:0/1219663792 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636409fa60 con 0x55c626b147a0 2024-06-10T01:21:54.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.175+0000 7f637249f700 1 --2- 172.21.15.7:0/226877427 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f63580a2ca0 0x7f63580a5160 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f636c199b00 tx=0x7f6368009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:54.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.176+0000 7f637ef86b80 1 -- 172.21.15.7:0/1219663792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c626b147a0 msgr2=0x55c626b14bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:54.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.176+0000 7f637ef86b80 1 --2- 172.21.15.7:0/1219663792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c626b147a0 0x55c626b14bc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f636409dbc0 tx=0x7f636409d2e0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.176+0000 7f637ef86b80 1 -- 172.21.15.7:0/1219663792 shutdown_connections 2024-06-10T01:21:54.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.176+0000 7f637ef86b80 1 --2- 172.21.15.7:0/1219663792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c626b147a0 0x55c626b14bc0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.176+0000 7f637ef86b80 1 --2- 172.21.15.7:0/1219663792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c626b19f60 0x55c626b1a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.176+0000 7f637ef86b80 1 --2- 172.21.15.7:0/1219663792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c626b1a880 0x55c626b14260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.176+0000 7f637ef86b80 1 -- 172.21.15.7:0/1219663792 >> 172.21.15.7:0/1219663792 conn(0x55c626a41080 msgr2=0x55c626b0f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:54.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.176+0000 7f637ef86b80 1 -- 172.21.15.7:0/1219663792 shutdown_connections 2024-06-10T01:21:54.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.176+0000 7f637ef86b80 1 -- 172.21.15.7:0/1219663792 wait complete. 2024-06-10T01:21:54.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.177+0000 7f637ef86b80 1 -- start start 2024-06-10T01:21:54.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.177+0000 7f637ef86b80 1 Processor -- start 2024-06-10T01:21:54.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.177+0000 7f637ef86b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c626b147a0 0x55c626b20640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:54.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.177+0000 7f637ef86b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c626b19f60 0x55c626b1d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:54.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.177+0000 7f637ef86b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c626b1a880 0x55c626b1de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:54.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.177+0000 7f637ef86b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c626a05fc0 con 0x55c626b147a0 2024-06-10T01:21:54.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.177+0000 7f637ef86b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c626a61810 con 0x55c626b19f60 2024-06-10T01:21:54.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.177+0000 7f6371c9e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c626b19f60 0x55c626b1d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:54.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.177+0000 7f637249f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c626b147a0 0x55c626b20640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:54.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.178+0000 7f6372ca0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c626b1a880 0x55c626b1de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:54.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.178+0000 7f6371c9e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c626b19f60 0x55c626b1d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:52240/0 (socket says 172.21.15.7:52240) 2024-06-10T01:21:54.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.178+0000 7f6371c9e700 1 -- 172.21.15.7:0/375919102 learned_addr learned my addr 172.21.15.7:0/375919102 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:54.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.178+0000 7f6372ca0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c626b1a880 0x55c626b1de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:60980/0 (socket says 172.21.15.7:60980) 2024-06-10T01:21:54.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.177+0000 7f637ef86b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c626b21010 con 0x55c626b1a880 2024-06-10T01:21:54.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.178+0000 7f637249f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c626b147a0 0x55c626b20640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52404/0 (socket says 172.21.15.7:52404) 2024-06-10T01:21:54.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.178+0000 7f637ef86b80 10 client.0 _ll_register_callbacks cb 0x55c6269fb5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:21:54.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.178+0000 7f637249f700 1 -- 172.21.15.7:0/375919102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c626b1a880 msgr2=0x55c626b1de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:54.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.178+0000 7f637249f700 1 --2- 172.21.15.7:0/375919102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c626b1a880 0x55c626b1de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.178+0000 7f637249f700 1 -- 172.21.15.7:0/375919102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c626b19f60 msgr2=0x55c626b1d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:54.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.178+0000 7f637249f700 1 --2- 172.21.15.7:0/375919102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c626b19f60 0x55c626b1d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.178+0000 7f637249f700 1 -- 172.21.15.7:0/375919102 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c626a2b750 con 0x55c626b147a0 2024-06-10T01:21:54.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.178+0000 7f637249f700 1 --2- 172.21.15.7:0/375919102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c626b147a0 0x55c626b20640 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f6368009ab0 tx=0x7f63680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea30bffca47e644 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:54.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.178+0000 7f6355ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:21:54.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.178+0000 7f6355ffb700 1 -- 172.21.15.7:0/375919102 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636801f830 con 0x55c626b147a0 2024-06-10T01:21:54.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.178+0000 7f6355ffb700 1 -- 172.21.15.7:0/375919102 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f636801f9d0 con 0x55c626b147a0 2024-06-10T01:21:54.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.178+0000 7f637ef86b80 1 -- 172.21.15.7:0/375919102 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63640a5b10 con 0x55c626b147a0 2024-06-10T01:21:54.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.179+0000 7f6355ffb700 1 -- 172.21.15.7:0/375919102 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f636802f050 con 0x55c626b147a0 2024-06-10T01:21:54.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.179+0000 7f637ef86b80 10 client.19374 fetch_fsmap learned FSMap version 45 2024-06-10T01:21:54.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.179+0000 7f637ef86b80 1 -- 172.21.15.7:0/375919102 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55c626b2b7b0 con 0x55c626b147a0 2024-06-10T01:21:54.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.179+0000 7f6355ffb700 1 -- 172.21.15.7:0/375919102 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63640a5b10 con 0x55c626b147a0 2024-06-10T01:21:54.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.179+0000 7f6355ffb700 1 -- 172.21.15.7:0/375919102 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 45) ==== 8358+0+0 (secure 0 0 0) 0x7f636801d3b0 con 0x55c626b147a0 2024-06-10T01:21:54.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.179+0000 7f637ef86b80 10 client.19374 fetch_fsmap finished waiting for FSMap version 45 2024-06-10T01:21:54.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.179+0000 7f637ef86b80 10 client.19374 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:21:54.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.179+0000 7f637ef86b80 20 client.19374 populate_metadata read hostname 'smithi007' 2024-06-10T01:21:54.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.179+0000 7f637ef86b80 1 --2- 172.21.15.7:0/375919102 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c626b2ddf0 0x55c626b30280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:54.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.179+0000 7f637ef86b80 4 client.19374 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:21:54.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.179+0000 7f637ef86b80 1 -- 172.21.15.7:0/375919102 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c626a72d50 con 0x55c626b2ddf0 2024-06-10T01:21:54.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.316+0000 7f6371c9e700 1 --2- 172.21.15.7:0/375919102 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c626b2ddf0 0x55c626b30280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:54.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.316+0000 7f6355ffb700 10 client.19374 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:54.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.316+0000 7f6371c9e700 1 --2- 172.21.15.7:0/375919102 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c626b2ddf0 0x55c626b30280 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a91888386ed0d01 server_cookie=822bd8751072c1e3 in_seq=0 out_seq=0 2024-06-10T01:21:54.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.320+0000 7f6355ffb700 1 -- 172.21.15.7:0/375919102 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c626a72d50 con 0x55c626b2ddf0 2024-06-10T01:21:54.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.320+0000 7f6355ffb700 10 client.19374 handle_command_reply: tid=0 2024-06-10T01:21:54.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.320+0000 7f637ef86b80 1 client.19374 shutdown 2024-06-10T01:21:54.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.320+0000 7f637ef86b80 1 -- 172.21.15.7:0/375919102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c626b147a0 msgr2=0x55c626b20640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:54.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.320+0000 7f637ef86b80 1 --2- 172.21.15.7:0/375919102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c626b147a0 0x55c626b20640 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f6368009ab0 tx=0x7f63680047c0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.320+0000 7f637ef86b80 1 -- 172.21.15.7:0/375919102 shutdown_connections 2024-06-10T01:21:54.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.320+0000 7f637ef86b80 1 --2- 172.21.15.7:0/375919102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c626b1a880 0x55c626b1de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.320+0000 7f637ef86b80 1 --2- 172.21.15.7:0/375919102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c626b147a0 0x55c626b20640 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.320+0000 7f637ef86b80 1 --2- 172.21.15.7:0/375919102 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c626b2ddf0 0x55c626b30280 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.320+0000 7f637ef86b80 1 --2- 172.21.15.7:0/375919102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c626b19f60 0x55c626b1d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.321+0000 7f637ef86b80 1 -- 172.21.15.7:0/375919102 >> 172.21.15.7:0/375919102 conn(0x55c626a41080 msgr2=0x55c626b0e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:54.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.321+0000 7f6355ffb700 0 client.19374 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:54.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.321+0000 7f637ef86b80 1 -- 172.21.15.7:0/375919102 shutdown_connections 2024-06-10T01:21:54.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.321+0000 7f637ef86b80 1 -- 172.21.15.7:0/375919102 wait complete. 2024-06-10T01:21:54.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.321+0000 7f637ef86b80 20 client.19374 trim_cache size 0 max 16384 2024-06-10T01:21:54.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.350+0000 7f637ef86b80 1 Processor -- start 2024-06-10T01:21:54.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.351+0000 7f637ef86b80 1 -- start start 2024-06-10T01:21:54.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.351+0000 7f637ef86b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c626b147a0 0x55c626b14260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:54.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.351+0000 7f637ef86b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c626b19f60 0x55c626b19330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:54.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.351+0000 7f637ef86b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c626b1a880 0x55c626b20180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:54.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.351+0000 7f637ef86b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f636c1ae870 con 0x55c626b147a0 2024-06-10T01:21:54.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.351+0000 7f637ef86b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f636c1ac0a0 con 0x55c626b1a880 2024-06-10T01:21:54.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.351+0000 7f637ef86b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f636c1a9f10 con 0x55c626b19f60 2024-06-10T01:21:54.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.351+0000 7f637249f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c626b147a0 0x55c626b14260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:54.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.351+0000 7f637249f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c626b147a0 0x55c626b14260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52414/0 (socket says 172.21.15.7:52414) 2024-06-10T01:21:54.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.351+0000 7f637249f700 1 -- 172.21.15.7:0/779769291 learned_addr learned my addr 172.21.15.7:0/779769291 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:54.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.351+0000 7f6372ca0700 1 --2- 172.21.15.7:0/779769291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c626b1a880 0x55c626b20180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:54.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.352+0000 7f6371c9e700 1 --2- 172.21.15.7:0/779769291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c626b19f60 0x55c626b19330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:54.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.352+0000 7f637249f700 1 -- 172.21.15.7:0/779769291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c626b19f60 msgr2=0x55c626b19330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:54.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.352+0000 7f637249f700 1 --2- 172.21.15.7:0/779769291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c626b19f60 0x55c626b19330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.352+0000 7f637249f700 1 -- 172.21.15.7:0/779769291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c626b1a880 msgr2=0x55c626b20180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.352+0000 7f637249f700 1 --2- 172.21.15.7:0/779769291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c626b1a880 0x55c626b20180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.352+0000 7f637249f700 1 -- 172.21.15.7:0/779769291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c626b2b5e0 con 0x55c626b147a0 2024-06-10T01:21:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.352+0000 7f637249f700 1 --2- 172.21.15.7:0/779769291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c626b147a0 0x55c626b14260 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f636801d7d0 tx=0x7f6368005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d46d872073d21b48 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.352+0000 7f6357fff700 1 -- 172.21.15.7:0/779769291 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636801da40 con 0x55c626b147a0 2024-06-10T01:21:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.353+0000 7f6357fff700 1 -- 172.21.15.7:0/779769291 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6368037030 con 0x55c626b147a0 2024-06-10T01:21:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.353+0000 7f6357fff700 1 -- 172.21.15.7:0/779769291 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6368003a00 con 0x55c626b147a0 2024-06-10T01:21:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.353+0000 7f6371c9e700 1 --2- 172.21.15.7:0/779769291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c626b19f60 0x55c626b19330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:21:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.354+0000 7f637ef86b80 1 -- 172.21.15.7:0/779769291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c626b147a0 msgr2=0x55c626b14260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.354+0000 7f637ef86b80 1 --2- 172.21.15.7:0/779769291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c626b147a0 0x55c626b14260 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f636801d7d0 tx=0x7f6368005ce0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.354+0000 7f637ef86b80 1 -- 172.21.15.7:0/779769291 shutdown_connections 2024-06-10T01:21:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.354+0000 7f637ef86b80 1 --2- 172.21.15.7:0/779769291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c626b1a880 0x55c626b20180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.354+0000 7f637ef86b80 1 --2- 172.21.15.7:0/779769291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c626b147a0 0x55c626b14260 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.354+0000 7f637ef86b80 1 --2- 172.21.15.7:0/779769291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c626b19f60 0x55c626b19330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.354+0000 7f637ef86b80 1 -- 172.21.15.7:0/779769291 >> 172.21.15.7:0/779769291 conn(0x55c626a41080 msgr2=0x55c626b1b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.354+0000 7f637ef86b80 1 -- 172.21.15.7:0/779769291 shutdown_connections 2024-06-10T01:21:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.354+0000 7f637ef86b80 1 -- 172.21.15.7:0/779769291 wait complete. 2024-06-10T01:21:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.355+0000 7f637ef86b80 1 -- start start 2024-06-10T01:21:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.355+0000 7f637ef86b80 1 Processor -- start 2024-06-10T01:21:54.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.355+0000 7f637ef86b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c626b147a0 0x55c626b26840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:54.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.355+0000 7f637ef86b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c626b19f60 0x55c626b35370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:54.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.355+0000 7f637ef86b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c626b1a880 0x55c626b358b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:54.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.355+0000 7f637ef86b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f636c1aa560 con 0x55c626b19f60 2024-06-10T01:21:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.355+0000 7f637ef86b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f636c1ab9d0 con 0x55c626b1a880 2024-06-10T01:21:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.355+0000 7f637ef86b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f636c1aac30 con 0x55c626b147a0 2024-06-10T01:21:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.355+0000 7f637ef86b80 10 client.0 _ll_register_callbacks cb 0x55c6269fb5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:21:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.355+0000 7f637249f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c626b147a0 0x55c626b26840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.355+0000 7f637249f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c626b147a0 0x55c626b26840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:32774/0 (socket says 172.21.15.7:32774) 2024-06-10T01:21:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.355+0000 7f637249f700 1 -- 172.21.15.7:0/3978507627 learned_addr learned my addr 172.21.15.7:0/3978507627 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.356+0000 7f6371c9e700 1 --2- 172.21.15.7:0/3978507627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c626b19f60 0x55c626b35370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.356+0000 7f6372ca0700 1 --2- 172.21.15.7:0/3978507627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c626b1a880 0x55c626b358b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.356+0000 7f6372ca0700 1 -- 172.21.15.7:0/3978507627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c626b147a0 msgr2=0x55c626b26840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.356+0000 7f6372ca0700 1 --2- 172.21.15.7:0/3978507627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c626b147a0 0x55c626b26840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.356+0000 7f6372ca0700 1 -- 172.21.15.7:0/3978507627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c626b19f60 msgr2=0x55c626b35370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:21:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.356+0000 7f6372ca0700 1 --2- 172.21.15.7:0/3978507627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c626b19f60 0x55c626b35370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.356+0000 7f6372ca0700 1 -- 172.21.15.7:0/3978507627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c626b35f20 con 0x55c626b1a880 2024-06-10T01:21:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.356+0000 7f637249f700 1 --2- 172.21.15.7:0/3978507627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c626b147a0 0x55c626b26840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:21:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.356+0000 7f6372ca0700 1 --2- 172.21.15.7:0/3978507627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c626b1a880 0x55c626b358b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55c626b20ea0 tx=0x7f63640a1820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67c1dfc2b38ce865 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.356+0000 7f6355ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:21:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.356+0000 7f6355ffb700 1 -- 172.21.15.7:0/3978507627 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63640ba040 con 0x55c626b1a880 2024-06-10T01:21:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.357+0000 7f6355ffb700 1 -- 172.21.15.7:0/3978507627 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f6364003da0 con 0x55c626b1a880 2024-06-10T01:21:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.357+0000 7f637ef86b80 1 -- 172.21.15.7:0/3978507627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6368037030 con 0x55c626b1a880 2024-06-10T01:21:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.357+0000 7f6355ffb700 1 -- 172.21.15.7:0/3978507627 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f63640a5b10 con 0x55c626b1a880 2024-06-10T01:21:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.357+0000 7f637ef86b80 10 client.28616 fetch_fsmap learned FSMap version 45 2024-06-10T01:21:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.357+0000 7f637ef86b80 1 -- 172.21.15.7:0/3978507627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55c626b989a0 con 0x55c626b1a880 2024-06-10T01:21:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.357+0000 7f6355ffb700 1 -- 172.21.15.7:0/3978507627 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6368037030 con 0x55c626b1a880 2024-06-10T01:21:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.357+0000 7f6355ffb700 1 -- 172.21.15.7:0/3978507627 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 45) ==== 8358+0+0 (secure 0 0 0) 0x7f6364011070 con 0x55c626b1a880 2024-06-10T01:21:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.357+0000 7f637ef86b80 10 client.28616 fetch_fsmap finished waiting for FSMap version 45 2024-06-10T01:21:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.357+0000 7f637ef86b80 10 client.28616 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:21:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.357+0000 7f637ef86b80 20 client.28616 populate_metadata read hostname 'smithi007' 2024-06-10T01:21:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.357+0000 7f637ef86b80 1 --2- 172.21.15.7:0/3978507627 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c626b98d20 0x55c626b9b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.357+0000 7f637ef86b80 4 client.28616 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:21:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.357+0000 7f637ef86b80 1 -- 172.21.15.7:0/3978507627 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c626a6d2d0 con 0x55c626b98d20 2024-06-10T01:21:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.358+0000 7f637249f700 1 --2- 172.21.15.7:0/3978507627 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c626b98d20 0x55c626b9b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.358+0000 7f637249f700 1 --2- 172.21.15.7:0/3978507627 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c626b98d20 0x55c626b9b140 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c2d9c0b1c096ccb server_cookie=1d1aa0f60a10c89d in_seq=0 out_seq=0 2024-06-10T01:21:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.358+0000 7f6355ffb700 10 client.28616 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:54.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.363+0000 7f6355ffb700 1 -- 172.21.15.7:0/3978507627 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c626a6d2d0 con 0x55c626b98d20 2024-06-10T01:21:54.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.363+0000 7f6355ffb700 10 client.28616 handle_command_reply: tid=0 2024-06-10T01:21:54.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.363+0000 7f637ef86b80 1 client.28616 shutdown 2024-06-10T01:21:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.363+0000 7f637ef86b80 1 -- 172.21.15.7:0/3978507627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c626b1a880 msgr2=0x55c626b358b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.363+0000 7f637ef86b80 1 --2- 172.21.15.7:0/3978507627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c626b1a880 0x55c626b358b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55c626b20ea0 tx=0x7f63640a1820 comp rx=0 tx=0).stop 2024-06-10T01:21:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.363+0000 7f637ef86b80 1 -- 172.21.15.7:0/3978507627 shutdown_connections 2024-06-10T01:21:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.363+0000 7f637ef86b80 1 --2- 172.21.15.7:0/3978507627 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c626b98d20 0x55c626b9b140 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.363+0000 7f637ef86b80 1 --2- 172.21.15.7:0/3978507627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c626b1a880 0x55c626b358b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.364+0000 7f637ef86b80 1 --2- 172.21.15.7:0/3978507627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c626b147a0 0x55c626b26840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.364+0000 7f637ef86b80 1 --2- 172.21.15.7:0/3978507627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c626b19f60 0x55c626b35370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.364+0000 7f637ef86b80 1 -- 172.21.15.7:0/3978507627 >> 172.21.15.7:0/3978507627 conn(0x55c626a41080 msgr2=0x55c626b19330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.364+0000 7f6355ffb700 0 client.28616 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:21:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.364+0000 7f637ef86b80 1 -- 172.21.15.7:0/3978507627 shutdown_connections 2024-06-10T01:21:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.364+0000 7f637ef86b80 1 -- 172.21.15.7:0/3978507627 wait complete. 2024-06-10T01:21:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.364+0000 7f637ef86b80 20 client.28616 trim_cache size 0 max 16384 2024-06-10T01:21:54.360 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:21:54.360 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-10T01:21:54.360 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-10T01:21:54.360 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:21:54.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.365+0000 7f6374703700 1 -- 172.21.15.7:0/226877427 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f63580a2ca0 msgr2=0x7f63580a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:54.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.365+0000 7f6374703700 1 --2- 172.21.15.7:0/226877427 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f63580a2ca0 0x7f63580a5160 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f636c199b00 tx=0x7f6368009260 comp rx=0 tx=0).stop 2024-06-10T01:21:54.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.365+0000 7f6374703700 1 -- 172.21.15.7:0/226877427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f636c107870 msgr2=0x7f636c1989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:54.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.365+0000 7f6374703700 1 --2- 172.21.15.7:0/226877427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f636c107870 0x7f636c1989c0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f636400d010 tx=0x7f6364009810 comp rx=0 tx=0).stop 2024-06-10T01:21:54.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.365+0000 7f6374703700 1 -- 172.21.15.7:0/226877427 shutdown_connections 2024-06-10T01:21:54.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.365+0000 7f6374703700 1 --2- 172.21.15.7:0/226877427 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f63580a2ca0 0x7f63580a5160 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.365+0000 7f6374703700 1 --2- 172.21.15.7:0/226877427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f636c107870 0x7f636c1989c0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.365+0000 7f6374703700 1 --2- 172.21.15.7:0/226877427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f636c1027a0 0x7f636c19e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.365+0000 7f6374703700 1 --2- 172.21.15.7:0/226877427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f636c103150 0x7f636c19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:54.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.365+0000 7f6374703700 1 -- 172.21.15.7:0/226877427 >> 172.21.15.7:0/226877427 conn(0x7f636c079570 msgr2=0x7f636c1004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:54.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.365+0000 7f6374703700 1 -- 172.21.15.7:0/226877427 shutdown_connections 2024-06-10T01:21:54.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:54.365+0000 7f6374703700 1 -- 172.21.15.7:0/226877427 wait complete. 2024-06-10T01:21:54.520 INFO:tasks.workunit.client.0.smithi007.stdout: 10 60104 34.00 MB/sec execute 31 sec latency 171.040 ms 2024-06-10T01:21:54.591 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:54 smithi007 ceph-mon[32398]: Adjusting osd_memory_target on smithi175 to 176.2M 2024-06-10T01:21:54.591 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:54 smithi007 ceph-mon[32398]: Unable to set osd_memory_target on smithi175 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-10T01:21:54.591 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:54 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:21:54.591 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:54 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:21:54.591 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:54 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:21:54.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:54 smithi078 ceph-mon[36683]: Adjusting osd_memory_target on smithi175 to 176.2M 2024-06-10T01:21:54.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:54 smithi078 ceph-mon[36683]: Unable to set osd_memory_target on smithi175 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-10T01:21:54.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:54 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:21:54.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:54 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:21:54.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:54 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:21:54.849 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:54 smithi175 ceph-mon[36636]: Adjusting osd_memory_target on smithi175 to 176.2M 2024-06-10T01:21:55.157 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:21:55.157 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:21:55.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:54 smithi175 ceph-mon[36636]: Unable to set osd_memory_target on smithi175 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-10T01:21:55.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:54 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:21:55.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:54 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:21:55.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:54 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:21:55.520 INFO:tasks.workunit.client.0.smithi007.stdout: 10 60609 33.79 MB/sec execute 32 sec latency 377.554 ms 2024-06-10T01:21:55.522 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:21:55.728 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:55 smithi078 ceph-mon[36683]: pgmap v909: 129 pgs: 129 active+clean; 491 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.8 MiB/s wr, 789 op/s 2024-06-10T01:21:55.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:55 smithi007 ceph-mon[32398]: pgmap v909: 129 pgs: 129 active+clean; 491 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.8 MiB/s wr, 789 op/s 2024-06-10T01:21:55.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:55 smithi175 ceph-mon[36636]: pgmap v909: 129 pgs: 129 active+clean; 491 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.8 MiB/s wr, 789 op/s 2024-06-10T01:21:56.520 INFO:tasks.workunit.client.0.smithi007.stdout: 10 61203 33.64 MB/sec execute 33 sec latency 79.338 ms 2024-06-10T01:21:57.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.223+0000 7f630082f700 1 -- 172.21.15.7:0/3817787627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62fc101d50 msgr2=0x7f62fc06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:57.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.223+0000 7f630082f700 1 --2- 172.21.15.7:0/3817787627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62fc101d50 0x7f62fc06a170 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f62f400b0a0 tx=0x7f62f400b3b0 comp rx=0 tx=0).stop 2024-06-10T01:21:57.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.223+0000 7f630082f700 1 -- 172.21.15.7:0/3817787627 shutdown_connections 2024-06-10T01:21:57.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.223+0000 7f630082f700 1 --2- 172.21.15.7:0/3817787627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62fc101d50 0x7f62fc06a170 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:57.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.223+0000 7f630082f700 1 --2- 172.21.15.7:0/3817787627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62fc107400 0x7f62fc1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:57.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.223+0000 7f630082f700 1 --2- 172.21.15.7:0/3817787627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62fc101390 0x7f62fc101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:57.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.223+0000 7f630082f700 1 -- 172.21.15.7:0/3817787627 >> 172.21.15.7:0/3817787627 conn(0x7f62fc078340 msgr2=0x7f62fc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:57.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.223+0000 7f630082f700 1 -- 172.21.15.7:0/3817787627 shutdown_connections 2024-06-10T01:21:57.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.223+0000 7f630082f700 1 -- 172.21.15.7:0/3817787627 wait complete. 2024-06-10T01:21:57.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.224+0000 7f630082f700 1 Processor -- start 2024-06-10T01:21:57.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.224+0000 7f630082f700 1 -- start start 2024-06-10T01:21:57.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.225+0000 7f630082f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62fc101390 0x7f62fc10cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:57.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.225+0000 7f630082f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62fc101d50 0x7f62fc10d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:57.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.225+0000 7f630082f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62fc107400 0x7f62fc19f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:57.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.225+0000 7f630082f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f62fc1127e0 con 0x7f62fc101390 2024-06-10T01:21:57.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.225+0000 7f630082f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f62fc112660 con 0x7f62fc101d50 2024-06-10T01:21:57.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.225+0000 7f630082f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f62fc112960 con 0x7f62fc107400 2024-06-10T01:21:57.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.225+0000 7f62fa59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62fc101390 0x7f62fc10cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:57.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.225+0000 7f62fa59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62fc101390 0x7f62fc10cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52440/0 (socket says 172.21.15.7:52440) 2024-06-10T01:21:57.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.225+0000 7f62fa59c700 1 -- 172.21.15.7:0/1893782475 learned_addr learned my addr 172.21.15.7:0/1893782475 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:21:57.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.225+0000 7f62fad9d700 1 --2- 172.21.15.7:0/1893782475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62fc107400 0x7f62fc19f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:57.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.226+0000 7f62fad9d700 1 -- 172.21.15.7:0/1893782475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62fc101d50 msgr2=0x7f62fc10d4f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T01:21:57.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.226+0000 7f62fad9d700 1 --2- 172.21.15.7:0/1893782475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62fc101d50 0x7f62fc10d4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:57.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.226+0000 7f62fad9d700 1 -- 172.21.15.7:0/1893782475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62fc101390 msgr2=0x7f62fc10cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:57.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.226+0000 7f62fad9d700 1 --2- 172.21.15.7:0/1893782475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62fc101390 0x7f62fc10cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:57.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.226+0000 7f62fad9d700 1 -- 172.21.15.7:0/1893782475 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62fc19fc00 con 0x7f62fc107400 2024-06-10T01:21:57.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.226+0000 7f62fad9d700 1 --2- 172.21.15.7:0/1893782475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62fc107400 0x7f62fc19f500 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f62f400b0a0 tx=0x7f62f40095a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:57.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.226+0000 7f62eb7fe700 1 -- 172.21.15.7:0/1893782475 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f4021070 con 0x7f62fc107400 2024-06-10T01:21:57.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.226+0000 7f62eb7fe700 1 -- 172.21.15.7:0/1893782475 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62f4007cc0 con 0x7f62fc107400 2024-06-10T01:21:57.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.226+0000 7f62eb7fe700 1 -- 172.21.15.7:0/1893782475 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f40053b0 con 0x7f62fc107400 2024-06-10T01:21:57.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.226+0000 7f630082f700 1 -- 172.21.15.7:0/1893782475 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62fc19fe90 con 0x7f62fc107400 2024-06-10T01:21:57.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.227+0000 7f630082f700 1 -- 172.21.15.7:0/1893782475 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62fc1a02c0 con 0x7f62fc107400 2024-06-10T01:21:57.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.228+0000 7f630082f700 1 -- 172.21.15.7:0/1893782475 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62fc069180 con 0x7f62fc107400 2024-06-10T01:21:57.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.229+0000 7f62eb7fe700 1 -- 172.21.15.7:0/1893782475 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f62f4024070 con 0x7f62fc107400 2024-06-10T01:21:57.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.230+0000 7f62eb7fe700 1 --2- 172.21.15.7:0/1893782475 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f62e40a2ca0 0x7f62e40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:21:57.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.230+0000 7f62fa59c700 1 --2- 172.21.15.7:0/1893782475 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f62e40a2ca0 0x7f62e40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:21:57.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.230+0000 7f62eb7fe700 1 -- 172.21.15.7:0/1893782475 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f62f40d0900 con 0x7f62fc107400 2024-06-10T01:21:57.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.231+0000 7f62fa59c700 1 --2- 172.21.15.7:0/1893782475 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f62e40a2ca0 0x7f62e40a5160 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f62ec009680 tx=0x7f62ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:21:57.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.238+0000 7f62eb7fe700 1 -- 172.21.15.7:0/1893782475 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f62f409d510 con 0x7f62fc107400 2024-06-10T01:21:57.521 INFO:tasks.workunit.client.0.smithi007.stdout: 10 61815 33.43 MB/sec execute 34 sec latency 283.566 ms 2024-06-10T01:21:57.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.688+0000 7f630082f700 1 -- 172.21.15.7:0/1893782475 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62fc02de60 con 0x7f62fc107400 2024-06-10T01:21:57.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.691+0000 7f62eb7fe700 1 -- 172.21.15.7:0/1893782475 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11859 (secure 0 0 0) 0x7f62f409ce10 con 0x7f62fc107400 2024-06-10T01:21:57.688 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:21:57.688 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:21:41.557407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24434,24436,24451]},"id":1}]} 2024-06-10T01:21:57.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.698+0000 7f630082f700 1 -- 172.21.15.7:0/1893782475 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f62e40a2ca0 msgr2=0x7f62e40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:57.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.698+0000 7f630082f700 1 --2- 172.21.15.7:0/1893782475 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f62e40a2ca0 0x7f62e40a5160 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f62ec009680 tx=0x7f62ec009260 comp rx=0 tx=0).stop 2024-06-10T01:21:57.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.698+0000 7f630082f700 1 -- 172.21.15.7:0/1893782475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62fc107400 msgr2=0x7f62fc19f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:21:57.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.699+0000 7f630082f700 1 --2- 172.21.15.7:0/1893782475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62fc107400 0x7f62fc19f500 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f62f400b0a0 tx=0x7f62f40095a0 comp rx=0 tx=0).stop 2024-06-10T01:21:57.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.699+0000 7f630082f700 1 -- 172.21.15.7:0/1893782475 shutdown_connections 2024-06-10T01:21:57.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.699+0000 7f630082f700 1 --2- 172.21.15.7:0/1893782475 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f62e40a2ca0 0x7f62e40a5160 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:57.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.699+0000 7f630082f700 1 --2- 172.21.15.7:0/1893782475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62fc107400 0x7f62fc19f500 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:57.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.699+0000 7f630082f700 1 --2- 172.21.15.7:0/1893782475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62fc101390 0x7f62fc10cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:57.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.699+0000 7f630082f700 1 --2- 172.21.15.7:0/1893782475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62fc101d50 0x7f62fc10d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:21:57.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.699+0000 7f630082f700 1 -- 172.21.15.7:0/1893782475 >> 172.21.15.7:0/1893782475 conn(0x7f62fc078340 msgr2=0x7f62fc102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:21:57.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.699+0000 7f630082f700 1 -- 172.21.15.7:0/1893782475 shutdown_connections 2024-06-10T01:21:57.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:21:57.701+0000 7f630082f700 1 -- 172.21.15.7:0/1893782475 wait complete. 2024-06-10T01:21:57.696 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 45 2024-06-10T01:21:57.719 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:57 smithi007 ceph-mon[32398]: pgmap v910: 129 pgs: 129 active+clean; 501 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.8 MiB/s wr, 807 op/s 2024-06-10T01:21:57.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:57 smithi078 ceph-mon[36683]: pgmap v910: 129 pgs: 129 active+clean; 501 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.8 MiB/s wr, 807 op/s 2024-06-10T01:21:57.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:57 smithi175 ceph-mon[36636]: pgmap v910: 129 pgs: 129 active+clean; 501 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.8 MiB/s wr, 807 op/s 2024-06-10T01:21:58.521 INFO:tasks.workunit.client.0.smithi007.stdout: 10 62273 33.06 MB/sec execute 35 sec latency 290.634 ms 2024-06-10T01:21:58.596 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:21:58.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:58 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1893782475' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:21:58.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:58 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1893782475' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:21:58.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:58 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1893782475' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:21:58.991 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:21:59.521 INFO:tasks.workunit.client.0.smithi007.stdout: 10 63127 33.76 MB/sec execute 36 sec latency 130.862 ms 2024-06-10T01:21:59.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:21:59 smithi007 ceph-mon[32398]: pgmap v911: 129 pgs: 129 active+clean; 498 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 9.2 MiB/s wr, 763 op/s 2024-06-10T01:21:59.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:21:59 smithi078 ceph-mon[36683]: pgmap v911: 129 pgs: 129 active+clean; 498 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 9.2 MiB/s wr, 763 op/s 2024-06-10T01:21:59.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:21:59 smithi175 ceph-mon[36636]: pgmap v911: 129 pgs: 129 active+clean; 498 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 9.2 MiB/s wr, 763 op/s 2024-06-10T01:22:00.521 INFO:tasks.workunit.client.0.smithi007.stdout: 10 63890 33.87 MB/sec execute 37 sec latency 52.853 ms 2024-06-10T01:22:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.277+0000 7f4ebc815640 1 -- 172.21.15.7:0/1848850689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4eb81076e0 msgr2=0x7f4eb8109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.277+0000 7f4ebc815640 1 --2- 172.21.15.7:0/1848850689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4eb81076e0 0x7f4eb8109b30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009860 tx=0x7f4eb002a340 comp rx=0 tx=0).stop 2024-06-10T01:22:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.277+0000 7f4ebc815640 1 -- 172.21.15.7:0/1848850689 shutdown_connections 2024-06-10T01:22:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.277+0000 7f4ebc815640 1 --2- 172.21.15.7:0/1848850689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4eb81076e0 0x7f4eb8109b30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.277+0000 7f4ebc815640 1 --2- 172.21.15.7:0/1848850689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4eb8104db0 0x7f4eb81071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.277+0000 7f4ebc815640 1 --2- 172.21.15.7:0/1848850689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4eb8102480 0x7f4eb8104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.277+0000 7f4ebc815640 1 -- 172.21.15.7:0/1848850689 >> 172.21.15.7:0/1848850689 conn(0x7f4eb80fbe70 msgr2=0x7f4eb80fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.277+0000 7f4ebc815640 1 -- 172.21.15.7:0/1848850689 shutdown_connections 2024-06-10T01:22:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.277+0000 7f4ebc815640 1 -- 172.21.15.7:0/1848850689 wait complete. 2024-06-10T01:22:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.278+0000 7f4ebc815640 1 Processor -- start 2024-06-10T01:22:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.279+0000 7f4ebc815640 1 -- start start 2024-06-10T01:22:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.279+0000 7f4ebc815640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4eb8102480 0x7f4eb819e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.279+0000 7f4ebc815640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4eb8104db0 0x7f4eb819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.279+0000 7f4ebc815640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4eb81076e0 0x7f4eb81a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.279+0000 7f4ebc815640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb810b490 con 0x7f4eb8102480 2024-06-10T01:22:01.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.279+0000 7f4ebc815640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb810b310 con 0x7f4eb81076e0 2024-06-10T01:22:01.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.279+0000 7f4ebc815640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb810b610 con 0x7f4eb8104db0 2024-06-10T01:22:01.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.279+0000 7f4eb5d9b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4eb8104db0 0x7f4eb819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.279+0000 7f4eb659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4eb8102480 0x7f4eb819e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.279+0000 7f4eb5d9b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4eb8104db0 0x7f4eb819e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:32854/0 (socket says 172.21.15.7:32854) 2024-06-10T01:22:01.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.279+0000 7f4eb659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4eb8102480 0x7f4eb819e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52504/0 (socket says 172.21.15.7:52504) 2024-06-10T01:22:01.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.279+0000 7f4eb5d9b640 1 -- 172.21.15.7:0/1652514906 learned_addr learned my addr 172.21.15.7:0/1652514906 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:01.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.280+0000 7f4eb6d9d640 1 --2- 172.21.15.7:0/1652514906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4eb81076e0 0x7f4eb81a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.280+0000 7f4eb6d9d640 1 -- 172.21.15.7:0/1652514906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4eb8104db0 msgr2=0x7f4eb819e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:01.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.280+0000 7f4eb6d9d640 1 --2- 172.21.15.7:0/1652514906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4eb8104db0 0x7f4eb819e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.280+0000 7f4eb6d9d640 1 -- 172.21.15.7:0/1652514906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4eb8102480 msgr2=0x7f4eb819e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.280+0000 7f4eb6d9d640 1 --2- 172.21.15.7:0/1652514906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4eb8102480 0x7f4eb819e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.280+0000 7f4eb6d9d640 1 -- 172.21.15.7:0/1652514906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb81a5ef0 con 0x7f4eb81076e0 2024-06-10T01:22:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.280+0000 7f4eb6d9d640 1 --2- 172.21.15.7:0/1652514906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4eb81076e0 0x7f4eb81a5880 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4eb002a930 tx=0x7f4eb0033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.280+0000 7f4ea77fe640 1 -- 172.21.15.7:0/1652514906 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0037040 con 0x7f4eb81076e0 2024-06-10T01:22:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.280+0000 7f4ebc815640 1 -- 172.21.15.7:0/1652514906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb81a6180 con 0x7f4eb81076e0 2024-06-10T01:22:01.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.281+0000 7f4ea77fe640 1 -- 172.21.15.7:0/1652514906 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4eb0040070 con 0x7f4eb81076e0 2024-06-10T01:22:01.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.281+0000 7f4ebc815640 1 -- 172.21.15.7:0/1652514906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb81a6690 con 0x7f4eb81076e0 2024-06-10T01:22:01.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.281+0000 7f4ea77fe640 1 -- 172.21.15.7:0/1652514906 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb00055f0 con 0x7f4eb81076e0 2024-06-10T01:22:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.284+0000 7f4ea77fe640 1 -- 172.21.15.7:0/1652514906 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f4eb0030850 con 0x7f4eb81076e0 2024-06-10T01:22:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.284+0000 7f4ebf5ff740 1 Processor -- start 2024-06-10T01:22:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.285+0000 7f4ebf5ff740 1 -- start start 2024-06-10T01:22:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.285+0000 7f4ebf5ff740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78997a010 0x55d78997a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.285+0000 7f4ebf5ff740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78997aa00 0x55d78997f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.285+0000 7f4ebf5ff740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78997fb70 0x55d789982000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.285+0000 7f4ebf5ff740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d7898dd4f0 con 0x55d78997aa00 2024-06-10T01:22:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.285+0000 7f4ebf5ff740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d78987b870 con 0x55d78997a010 2024-06-10T01:22:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.285+0000 7f4ebf5ff740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d7898d94b0 con 0x55d78997fb70 2024-06-10T01:22:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.285+0000 7f4eb5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78997a010 0x55d78997a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.285+0000 7f4eb5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78997a010 0x55d78997a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:52334/0 (socket says 172.21.15.7:52334) 2024-06-10T01:22:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.285+0000 7f4eb5d9b640 1 -- 172.21.15.7:0/2413800263 learned_addr learned my addr 172.21.15.7:0/2413800263 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.285+0000 7f4eb5d9b640 1 --2- 172.21.15.7:0/2413800263 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78997fb70 0x55d789982000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.285+0000 7f4eb5d9b640 1 -- 172.21.15.7:0/2413800263 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78997fb70 msgr2=0x55d789982000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.285+0000 7f4eb5d9b640 1 --2- 172.21.15.7:0/2413800263 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78997fb70 0x55d789982000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.285+0000 7f4eb5d9b640 1 -- 172.21.15.7:0/2413800263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78997aa00 msgr2=0x55d78997f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.285+0000 7f4eb5d9b640 1 --2- 172.21.15.7:0/2413800263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78997aa00 0x55d78997f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.285+0000 7f4eb5d9b640 1 -- 172.21.15.7:0/2413800263 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7898979b0 con 0x55d78997a010 2024-06-10T01:22:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.285+0000 7f4ea77fe640 1 --2- 172.21.15.7:0/1652514906 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4ea00a2c00 0x7f4ea00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.286+0000 7f4ea77fe640 1 -- 172.21.15.7:0/1652514906 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f4eb00ef280 con 0x7f4eb81076e0 2024-06-10T01:22:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.286+0000 7f4eb5d9b640 1 --2- 172.21.15.7:0/2413800263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78997a010 0x55d78997a430 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4eac011880 tx=0x7f4eac0097f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55956be633698284 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.286+0000 7f4eb5d9b640 1 --2- 172.21.15.7:0/1652514906 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4ea00a2c00 0x7f4ea00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.286+0000 7f4ea4ff9640 1 -- 172.21.15.7:0/2413800263 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac026070 con 0x55d78997a010 2024-06-10T01:22:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.286+0000 7f4ea4ff9640 1 -- 172.21.15.7:0/2413800263 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4eac020d90 con 0x55d78997a010 2024-06-10T01:22:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.286+0000 7f4ea4ff9640 1 -- 172.21.15.7:0/2413800263 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac021b60 con 0x55d78997a010 2024-06-10T01:22:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.286+0000 7f4eb5d9b640 1 --2- 172.21.15.7:0/1652514906 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4ea00a2c00 0x7f4ea00a50c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4eac023000 tx=0x7f4eac024000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.287+0000 7f4ebf5ff740 1 -- 172.21.15.7:0/2413800263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78997a010 msgr2=0x55d78997a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:01.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.287+0000 7f4ebf5ff740 1 --2- 172.21.15.7:0/2413800263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78997a010 0x55d78997a430 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4eac011880 tx=0x7f4eac0097f0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.288+0000 7f4ebf5ff740 1 -- 172.21.15.7:0/2413800263 shutdown_connections 2024-06-10T01:22:01.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.289+0000 7f4ebf5ff740 1 --2- 172.21.15.7:0/2413800263 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78997fb70 0x55d789982000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.289+0000 7f4ebf5ff740 1 --2- 172.21.15.7:0/2413800263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78997aa00 0x55d78997f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.289+0000 7f4ebf5ff740 1 --2- 172.21.15.7:0/2413800263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78997a010 0x55d78997a430 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.289+0000 7f4ebf5ff740 1 -- 172.21.15.7:0/2413800263 >> 172.21.15.7:0/2413800263 conn(0x55d789975680 msgr2=0x55d789975a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.289+0000 7f4ebf5ff740 1 -- 172.21.15.7:0/2413800263 shutdown_connections 2024-06-10T01:22:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.290+0000 7f4ebf5ff740 1 -- 172.21.15.7:0/2413800263 wait complete. 2024-06-10T01:22:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.291+0000 7f4ebf5ff740 1 -- start start 2024-06-10T01:22:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.291+0000 7f4ebf5ff740 1 Processor -- start 2024-06-10T01:22:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.291+0000 7f4ebf5ff740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78997a010 0x55d789995300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.291+0000 7f4ebf5ff740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78997aa00 0x55d789995840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.291+0000 7f4ebf5ff740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78997fb70 0x55d78999c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.291+0000 7f4ebf5ff740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d789993b30 con 0x55d78997a010 2024-06-10T01:22:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.291+0000 7f4ebf5ff740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d789866ae0 con 0x55d78997fb70 2024-06-10T01:22:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.291+0000 7f4ebf5ff740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d789993cb0 con 0x55d78997aa00 2024-06-10T01:22:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.291+0000 7f4eb659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78997a010 0x55d789995300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.291+0000 7f4eb659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78997a010 0x55d789995300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52512/0 (socket says 172.21.15.7:52512) 2024-06-10T01:22:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.291+0000 7f4eb659c640 1 -- 172.21.15.7:0/2371876599 learned_addr learned my addr 172.21.15.7:0/2371876599 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.291+0000 7f4eb6d9d640 1 --2- 172.21.15.7:0/2371876599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78997fb70 0x55d78999c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.292+0000 7f4eb5d9b640 1 --2- 172.21.15.7:0/2371876599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78997aa00 0x55d789995840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.292+0000 7f4ebf5ff740 10 client.0 _ll_register_callbacks cb 0x55d78989f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:22:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.292+0000 7f4eb6d9d640 1 -- 172.21.15.7:0/2371876599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78997aa00 msgr2=0x55d789995840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.292+0000 7f4eb6d9d640 1 --2- 172.21.15.7:0/2371876599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78997aa00 0x55d789995840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.292+0000 7f4eb6d9d640 1 -- 172.21.15.7:0/2371876599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78997a010 msgr2=0x55d789995300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.292+0000 7f4eb6d9d640 1 --2- 172.21.15.7:0/2371876599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78997a010 0x55d789995300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.292+0000 7f4eb6d9d640 1 -- 172.21.15.7:0/2371876599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7895f1210 con 0x55d78997fb70 2024-06-10T01:22:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.292+0000 7f4eb659c640 1 --2- 172.21.15.7:0/2371876599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78997a010 0x55d789995300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:22:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.292+0000 7f4eb6d9d640 1 --2- 172.21.15.7:0/2371876599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78997fb70 0x55d78999c880 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00bb690 tx=0x7f4eb0030b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=633b064ae3499f10 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.292+0000 7f4e9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:22:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.292+0000 7f4e9dffb640 1 -- 172.21.15.7:0/2371876599 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb00ece60 con 0x55d78997fb70 2024-06-10T01:22:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.292+0000 7f4ebf5ff740 1 -- 172.21.15.7:0/2371876599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4eac020d90 con 0x55d78997fb70 2024-06-10T01:22:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.292+0000 7f4e9dffb640 1 -- 172.21.15.7:0/2371876599 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f4eb00c3ee0 con 0x55d78997fb70 2024-06-10T01:22:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.293+0000 7f4e9dffb640 1 -- 172.21.15.7:0/2371876599 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f4eb00bb700 con 0x55d78997fb70 2024-06-10T01:22:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.293+0000 7f4ebf5ff740 10 client.28649 fetch_fsmap learned FSMap version 45 2024-06-10T01:22:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.293+0000 7f4ebf5ff740 1 -- 172.21.15.7:0/2371876599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55d7896bd620 con 0x55d78997fb70 2024-06-10T01:22:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.293+0000 7f4e9dffb640 1 -- 172.21.15.7:0/2371876599 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4eb00bfab0 con 0x55d78997fb70 2024-06-10T01:22:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.293+0000 7f4e9dffb640 1 -- 172.21.15.7:0/2371876599 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 45) ==== 8358+0+0 (secure 0 0 0) 0x7f4eb003a070 con 0x55d78997fb70 2024-06-10T01:22:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.293+0000 7f4ebf5ff740 10 client.28649 fetch_fsmap finished waiting for FSMap version 45 2024-06-10T01:22:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.293+0000 7f4ebf5ff740 10 client.28649 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:22:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.293+0000 7f4ebf5ff740 20 client.28649 populate_metadata read hostname 'smithi007' 2024-06-10T01:22:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.293+0000 7f4ebf5ff740 1 --2- 172.21.15.7:0/2371876599 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d7899a1090 0x55d7899a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.293+0000 7f4ebf5ff740 4 client.28649 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:22:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.293+0000 7f4ebf5ff740 1 -- 172.21.15.7:0/2371876599 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4eac020d90 con 0x55d7899a1090 2024-06-10T01:22:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.295+0000 7f4eb659c640 1 --2- 172.21.15.7:0/2371876599 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d7899a1090 0x55d7899a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.296+0000 7f4eb659c640 1 --2- 172.21.15.7:0/2371876599 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d7899a1090 0x55d7899a34e0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdf613a1d5316977 server_cookie=e5eb4b03dd6a377c in_seq=0 out_seq=0 2024-06-10T01:22:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.296+0000 7f4e9dffb640 10 client.28649 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.301+0000 7f4e9dffb640 1 -- 172.21.15.7:0/2371876599 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f4eac020d90 con 0x55d7899a1090 2024-06-10T01:22:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.301+0000 7f4e9dffb640 10 client.28649 handle_command_reply: tid=0 2024-06-10T01:22:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.301+0000 7f4ebf5ff740 1 client.28649 shutdown 2024-06-10T01:22:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.301+0000 7f4ebf5ff740 1 -- 172.21.15.7:0/2371876599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78997fb70 msgr2=0x55d78999c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.301+0000 7f4ebf5ff740 1 --2- 172.21.15.7:0/2371876599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78997fb70 0x55d78999c880 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00bb690 tx=0x7f4eb0030b40 comp rx=0 tx=0).stop 2024-06-10T01:22:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.302+0000 7f4ebf5ff740 1 -- 172.21.15.7:0/2371876599 shutdown_connections 2024-06-10T01:22:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.302+0000 7f4ebf5ff740 1 --2- 172.21.15.7:0/2371876599 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d7899a1090 0x55d7899a34e0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.302+0000 7f4e9dffb640 0 client.28649 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.302+0000 7f4ebf5ff740 1 --2- 172.21.15.7:0/2371876599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78997fb70 0x55d78999c880 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.302+0000 7f4ebf5ff740 1 --2- 172.21.15.7:0/2371876599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78997aa00 0x55d789995840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.302+0000 7f4ebf5ff740 1 --2- 172.21.15.7:0/2371876599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78997a010 0x55d789995300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.302+0000 7f4ebf5ff740 1 -- 172.21.15.7:0/2371876599 >> 172.21.15.7:0/2371876599 conn(0x55d789975680 msgr2=0x55d7899747e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.302+0000 7f4ebf5ff740 1 -- 172.21.15.7:0/2371876599 shutdown_connections 2024-06-10T01:22:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.302+0000 7f4ebf5ff740 1 -- 172.21.15.7:0/2371876599 wait complete. 2024-06-10T01:22:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.302+0000 7f4ebf5ff740 20 client.28649 trim_cache size 0 max 16384 2024-06-10T01:22:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.321+0000 7f4ebf5ff740 1 Processor -- start 2024-06-10T01:22:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.321+0000 7f4ebf5ff740 1 -- start start 2024-06-10T01:22:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.321+0000 7f4ebf5ff740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78997a010 0x55d78997bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.321+0000 7f4ebf5ff740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78997aa00 0x55d78999c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.321+0000 7f4ebf5ff740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78997fb70 0x55d78997c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.321+0000 7f4ebf5ff740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb81aac30 con 0x55d78997fb70 2024-06-10T01:22:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.321+0000 7f4ebf5ff740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb81a84a0 con 0x55d78997aa00 2024-06-10T01:22:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.321+0000 7f4ebf5ff740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb8116f60 con 0x55d78997a010 2024-06-10T01:22:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.321+0000 7f4eb5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78997aa00 0x55d78999c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.321+0000 7f4eb6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78997fb70 0x55d78997c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.321+0000 7f4eb5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78997aa00 0x55d78999c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:52364/0 (socket says 172.21.15.7:52364) 2024-06-10T01:22:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.321+0000 7f4eb6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78997fb70 0x55d78997c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52522/0 (socket says 172.21.15.7:52522) 2024-06-10T01:22:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.321+0000 7f4eb5d9b640 1 -- 172.21.15.7:0/2707166629 learned_addr learned my addr 172.21.15.7:0/2707166629 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.321+0000 7f4eb659c640 1 --2- 172.21.15.7:0/2707166629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78997a010 0x55d78997bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.322+0000 7f4eb5d9b640 1 -- 172.21.15.7:0/2707166629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78997a010 msgr2=0x55d78997bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.322+0000 7f4eb5d9b640 1 --2- 172.21.15.7:0/2707166629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78997a010 0x55d78997bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.322+0000 7f4eb5d9b640 1 -- 172.21.15.7:0/2707166629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78997fb70 msgr2=0x55d78997c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.322+0000 7f4eb5d9b640 1 --2- 172.21.15.7:0/2707166629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78997fb70 0x55d78997c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.322+0000 7f4eb5d9b640 1 -- 172.21.15.7:0/2707166629 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7899a3b60 con 0x55d78997aa00 2024-06-10T01:22:01.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.322+0000 7f4eb5d9b640 1 --2- 172.21.15.7:0/2707166629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78997aa00 0x55d78999c840 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4eac009d80 tx=0x7f4eac029000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70176d0d639529a0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:01.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.322+0000 7f4e9ffff640 1 -- 172.21.15.7:0/2707166629 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac026030 con 0x55d78997aa00 2024-06-10T01:22:01.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.322+0000 7f4e9ffff640 1 -- 172.21.15.7:0/2707166629 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4eac0055f0 con 0x55d78997aa00 2024-06-10T01:22:01.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.322+0000 7f4e9ffff640 1 -- 172.21.15.7:0/2707166629 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac021540 con 0x55d78997aa00 2024-06-10T01:22:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.323+0000 7f4ebf5ff740 1 -- 172.21.15.7:0/2707166629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78997aa00 msgr2=0x55d78999c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.323+0000 7f4ebf5ff740 1 --2- 172.21.15.7:0/2707166629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78997aa00 0x55d78999c840 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4eac009d80 tx=0x7f4eac029000 comp rx=0 tx=0).stop 2024-06-10T01:22:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.323+0000 7f4ebf5ff740 1 -- 172.21.15.7:0/2707166629 shutdown_connections 2024-06-10T01:22:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.323+0000 7f4ebf5ff740 1 --2- 172.21.15.7:0/2707166629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78997fb70 0x55d78997c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.323+0000 7f4ebf5ff740 1 --2- 172.21.15.7:0/2707166629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78997aa00 0x55d78999c840 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.323+0000 7f4ebf5ff740 1 --2- 172.21.15.7:0/2707166629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78997a010 0x55d78997bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.323+0000 7f4ebf5ff740 1 -- 172.21.15.7:0/2707166629 >> 172.21.15.7:0/2707166629 conn(0x55d789975680 msgr2=0x55d78997f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.323+0000 7f4ebf5ff740 1 -- 172.21.15.7:0/2707166629 shutdown_connections 2024-06-10T01:22:01.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.323+0000 7f4ebf5ff740 1 -- 172.21.15.7:0/2707166629 wait complete. 2024-06-10T01:22:01.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.323+0000 7f4ebf5ff740 1 -- start start 2024-06-10T01:22:01.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.324+0000 7f4ebf5ff740 1 Processor -- start 2024-06-10T01:22:01.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.324+0000 7f4ebf5ff740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78997a010 0x55d789999d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.324+0000 7f4ebf5ff740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78997aa00 0x55d78999a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.324+0000 7f4ebf5ff740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78997fb70 0x55d78999b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.324+0000 7f4ebf5ff740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb81a6a00 con 0x55d78997aa00 2024-06-10T01:22:01.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.324+0000 7f4ebf5ff740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb81a7dd0 con 0x55d78997fb70 2024-06-10T01:22:01.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.324+0000 7f4ebf5ff740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb81a7030 con 0x55d78997a010 2024-06-10T01:22:01.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.324+0000 7f4ebf5ff740 10 client.0 _ll_register_callbacks cb 0x7f4eb00097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:22:01.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.324+0000 7f4eb659c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78997a010 0x55d789999d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.324+0000 7f4eb5d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78997aa00 0x55d78999a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.324+0000 7f4eb6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78997fb70 0x55d78999b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.324+0000 7f4eb659c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78997a010 0x55d789999d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:32882/0 (socket says 172.21.15.7:32882) 2024-06-10T01:22:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.324+0000 7f4eb659c640 1 -- 172.21.15.7:0/4024219259 learned_addr learned my addr 172.21.15.7:0/4024219259 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.324+0000 7f4eb659c640 1 -- 172.21.15.7:0/4024219259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78997fb70 msgr2=0x55d78999b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.324+0000 7f4eb659c640 1 --2- 172.21.15.7:0/4024219259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78997fb70 0x55d78999b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.324+0000 7f4eb659c640 1 -- 172.21.15.7:0/4024219259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78997aa00 msgr2=0x55d78999a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.324+0000 7f4eb659c640 1 --2- 172.21.15.7:0/4024219259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78997aa00 0x55d78999a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.324+0000 7f4eb659c640 1 -- 172.21.15.7:0/4024219259 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7899754b0 con 0x55d78997a010 2024-06-10T01:22:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.324+0000 7f4eb5d9b640 1 --2- 172.21.15.7:0/4024219259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78997aa00 0x55d78999a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:22:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.325+0000 7f4eb659c640 1 --2- 172.21.15.7:0/4024219259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78997a010 0x55d789999d50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8011710 tx=0x7f4ea8011f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe8539733eec8d19 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.325+0000 7f4e9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:22:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.325+0000 7f4e9dffb640 1 -- 172.21.15.7:0/4024219259 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8022070 con 0x55d78997a010 2024-06-10T01:22:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.325+0000 7f4ebf5ff740 1 -- 172.21.15.7:0/4024219259 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4eac0055f0 con 0x55d78997a010 2024-06-10T01:22:01.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.325+0000 7f4e9dffb640 1 -- 172.21.15.7:0/4024219259 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f4ea8003970 con 0x55d78997a010 2024-06-10T01:22:01.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.325+0000 7f4e9dffb640 1 -- 172.21.15.7:0/4024219259 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f4ea8003b70 con 0x55d78997a010 2024-06-10T01:22:01.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.325+0000 7f4ebf5ff740 10 client.28663 fetch_fsmap learned FSMap version 45 2024-06-10T01:22:01.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.325+0000 7f4ebf5ff740 1 -- 172.21.15.7:0/4024219259 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55d78999caf0 con 0x55d78997a010 2024-06-10T01:22:01.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.326+0000 7f4e9dffb640 1 -- 172.21.15.7:0/4024219259 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4eac0055f0 con 0x55d78997a010 2024-06-10T01:22:01.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.326+0000 7f4e9dffb640 1 -- 172.21.15.7:0/4024219259 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 45) ==== 8358+0+0 (secure 0 0 0) 0x7f4ea801e420 con 0x55d78997a010 2024-06-10T01:22:01.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.326+0000 7f4ebf5ff740 10 client.28663 fetch_fsmap finished waiting for FSMap version 45 2024-06-10T01:22:01.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.326+0000 7f4ebf5ff740 10 client.28663 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:22:01.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.326+0000 7f4ebf5ff740 20 client.28663 populate_metadata read hostname 'smithi007' 2024-06-10T01:22:01.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.326+0000 7f4ebf5ff740 1 --2- 172.21.15.7:0/4024219259 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d789a0c5c0 0x55d789a0e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.326+0000 7f4ebf5ff740 4 client.28663 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:22:01.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.326+0000 7f4ebf5ff740 1 -- 172.21.15.7:0/4024219259 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d7898f0f10 con 0x55d789a0c5c0 2024-06-10T01:22:01.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.327+0000 7f4eb5d9b640 1 --2- 172.21.15.7:0/4024219259 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d789a0c5c0 0x55d789a0e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.327+0000 7f4e9dffb640 10 client.28663 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:01.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.327+0000 7f4eb5d9b640 1 --2- 172.21.15.7:0/4024219259 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d789a0c5c0 0x55d789a0e9e0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c17f50a8151e510 server_cookie=ec060bc0d1c6ed07 in_seq=0 out_seq=0 2024-06-10T01:22:01.521 INFO:tasks.workunit.client.0.smithi007.stdout: 10 64378 33.76 MB/sec execute 38 sec latency 189.498 ms 2024-06-10T01:22:01.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.008+0000 7f328a8fb700 1 -- 172.21.15.7:0/3248951098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f328406b3c0 msgr2=0x7f328410f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:01.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.008+0000 7f328a8fb700 1 --2- 172.21.15.7:0/3248951098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f328406b3c0 0x7f328410f310 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f327c009f90 tx=0x7f327c00b330 comp rx=0 tx=0).stop 2024-06-10T01:22:01.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.008+0000 7f328a8fb700 1 -- 172.21.15.7:0/3248951098 shutdown_connections 2024-06-10T01:22:01.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.008+0000 7f328a8fb700 1 --2- 172.21.15.7:0/3248951098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f328406b3c0 0x7f328410f310 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.008+0000 7f328a8fb700 1 --2- 172.21.15.7:0/3248951098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32841084f0 0x7f32841088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.008+0000 7f328a8fb700 1 --2- 172.21.15.7:0/3248951098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f328406aa00 0x7f328406ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.008+0000 7f328a8fb700 1 -- 172.21.15.7:0/3248951098 >> 172.21.15.7:0/3248951098 conn(0x7f3284078300 msgr2=0x7f3284078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:01.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.008+0000 7f328a8fb700 1 -- 172.21.15.7:0/3248951098 shutdown_connections 2024-06-10T01:22:01.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.008+0000 7f328a8fb700 1 -- 172.21.15.7:0/3248951098 wait complete. 2024-06-10T01:22:01.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.009+0000 7f328a8fb700 1 Processor -- start 2024-06-10T01:22:01.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.009+0000 7f328a8fb700 1 -- start start 2024-06-10T01:22:01.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.009+0000 7f328a8fb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f328406aa00 0x7f328419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.009+0000 7f328a8fb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f328406b3c0 0x7f328419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.009+0000 7f328a8fb700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32841084f0 0x7f32841989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.009+0000 7f328a8fb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3284112750 con 0x7f328406b3c0 2024-06-10T01:22:01.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.009+0000 7f328a8fb700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f32841125d0 con 0x7f328406aa00 2024-06-10T01:22:01.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.009+0000 7f328a8fb700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f32841128d0 con 0x7f32841084f0 2024-06-10T01:22:01.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.009+0000 7f327b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f328406b3c0 0x7f328419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.009+0000 7f327b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f328406b3c0 0x7f328419ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52452/0 (socket says 172.21.15.7:52452) 2024-06-10T01:22:01.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.009+0000 7f327b7fe700 1 -- 172.21.15.7:0/1335138832 learned_addr learned my addr 172.21.15.7:0/1335138832 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:01.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.009+0000 7f3288e98700 1 --2- 172.21.15.7:0/1335138832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32841084f0 0x7f32841989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.009+0000 7f3283fff700 1 --2- 172.21.15.7:0/1335138832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f328406aa00 0x7f328419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.010+0000 7f327b7fe700 1 -- 172.21.15.7:0/1335138832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32841084f0 msgr2=0x7f32841989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:01.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.010+0000 7f327b7fe700 1 --2- 172.21.15.7:0/1335138832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32841084f0 0x7f32841989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.010+0000 7f327b7fe700 1 -- 172.21.15.7:0/1335138832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f328406aa00 msgr2=0x7f328419e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:01.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.010+0000 7f327b7fe700 1 --2- 172.21.15.7:0/1335138832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f328406aa00 0x7f328419e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.010+0000 7f327b7fe700 1 -- 172.21.15.7:0/1335138832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3284199150 con 0x7f328406b3c0 2024-06-10T01:22:01.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.010+0000 7f327b7fe700 1 --2- 172.21.15.7:0/1335138832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f328406b3c0 0x7f328419ed70 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f326c00eb30 tx=0x7f326c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:01.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.010+0000 7f3281ffb700 1 -- 172.21.15.7:0/1335138832 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c00cbd0 con 0x7f328406b3c0 2024-06-10T01:22:01.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.010+0000 7f3281ffb700 1 -- 172.21.15.7:0/1335138832 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f326c004500 con 0x7f328406b3c0 2024-06-10T01:22:01.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.010+0000 7f3281ffb700 1 -- 172.21.15.7:0/1335138832 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c010430 con 0x7f328406b3c0 2024-06-10T01:22:01.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.010+0000 7f328a8fb700 1 -- 172.21.15.7:0/1335138832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3284199440 con 0x7f328406b3c0 2024-06-10T01:22:01.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.010+0000 7f328a8fb700 1 -- 172.21.15.7:0/1335138832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32841a5700 con 0x7f328406b3c0 2024-06-10T01:22:01.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.012+0000 7f3281ffb700 1 -- 172.21.15.7:0/1335138832 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f326c003f00 con 0x7f328406b3c0 2024-06-10T01:22:01.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.013+0000 7f3281ffb700 1 --2- 172.21.15.7:0/1335138832 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f32700a2ca0 0x7f32700a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.013+0000 7f3283fff700 1 --2- 172.21.15.7:0/1335138832 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f32700a2ca0 0x7f32700a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.013+0000 7f3281ffb700 1 -- 172.21.15.7:0/1335138832 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f326c0cf550 con 0x7f328406b3c0 2024-06-10T01:22:01.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.014+0000 7f3283fff700 1 --2- 172.21.15.7:0/1335138832 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f32700a2ca0 0x7f32700a5160 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3284199b30 tx=0x7f3274009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:01.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.014+0000 7f329517eb80 1 Processor -- start 2024-06-10T01:22:01.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.014+0000 7f329517eb80 1 -- start start 2024-06-10T01:22:01.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.015+0000 7f329517eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5edc1f60 0x55da5edc2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.015+0000 7f329517eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da5edc2880 0x55da5edbc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.015+0000 7f329517eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da5edbc7a0 0x55da5edbcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.015+0000 7f329517eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55da5ecaef60 con 0x55da5edc2880 2024-06-10T01:22:01.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.015+0000 7f329517eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55da5ed0d850 con 0x55da5edbc7a0 2024-06-10T01:22:01.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.015+0000 7f329517eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55da5ec75460 con 0x55da5edc1f60 2024-06-10T01:22:01.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.015+0000 7f3283fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5edc1f60 0x55da5edc2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.015+0000 7f3283fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5edc1f60 0x55da5edc2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:32814/0 (socket says 172.21.15.7:32814) 2024-06-10T01:22:01.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.015+0000 7f3283fff700 1 -- 172.21.15.7:0/3020342809 learned_addr learned my addr 172.21.15.7:0/3020342809 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:01.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.015+0000 7f3288e98700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da5edbc7a0 0x55da5edbcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.015+0000 7f327b7fe700 1 --2- 172.21.15.7:0/3020342809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da5edc2880 0x55da5edbc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.015+0000 7f3283fff700 1 -- 172.21.15.7:0/3020342809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da5edbc7a0 msgr2=0x55da5edbcbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:01.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.015+0000 7f3283fff700 1 --2- 172.21.15.7:0/3020342809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da5edbc7a0 0x55da5edbcbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.015+0000 7f3283fff700 1 -- 172.21.15.7:0/3020342809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da5edc2880 msgr2=0x55da5edbc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:01.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.015+0000 7f3283fff700 1 --2- 172.21.15.7:0/3020342809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da5edc2880 0x55da5edbc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.015+0000 7f3283fff700 1 -- 172.21.15.7:0/3020342809 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da5ec5c8d0 con 0x55da5edc1f60 2024-06-10T01:22:01.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.015+0000 7f3283fff700 1 --2- 172.21.15.7:0/3020342809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5edc1f60 0x55da5edc2340 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3274011d40 tx=0x7f3274014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13fd29c9d390c88c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:01.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.016+0000 7f327a7fc700 1 -- 172.21.15.7:0/3020342809 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3274004a30 con 0x55da5edc1f60 2024-06-10T01:22:01.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.016+0000 7f327a7fc700 1 -- 172.21.15.7:0/3020342809 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3274005500 con 0x55da5edc1f60 2024-06-10T01:22:01.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.016+0000 7f327a7fc700 1 -- 172.21.15.7:0/3020342809 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3274003cd0 con 0x55da5edc1f60 2024-06-10T01:22:01.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.016+0000 7f329517eb80 1 -- 172.21.15.7:0/3020342809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5edc1f60 msgr2=0x55da5edc2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:01.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.016+0000 7f329517eb80 1 --2- 172.21.15.7:0/3020342809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5edc1f60 0x55da5edc2340 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3274011d40 tx=0x7f3274014000 comp rx=0 tx=0).stop 2024-06-10T01:22:01.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.016+0000 7f329517eb80 1 -- 172.21.15.7:0/3020342809 shutdown_connections 2024-06-10T01:22:01.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.016+0000 7f329517eb80 1 --2- 172.21.15.7:0/3020342809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da5edbc7a0 0x55da5edbcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.016+0000 7f329517eb80 1 --2- 172.21.15.7:0/3020342809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5edc1f60 0x55da5edc2340 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.016+0000 7f329517eb80 1 --2- 172.21.15.7:0/3020342809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da5edc2880 0x55da5edbc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.016+0000 7f329517eb80 1 -- 172.21.15.7:0/3020342809 >> 172.21.15.7:0/3020342809 conn(0x55da5ece9080 msgr2=0x55da5edb7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:01.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.016+0000 7f329517eb80 1 -- 172.21.15.7:0/3020342809 shutdown_connections 2024-06-10T01:22:01.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.016+0000 7f329517eb80 1 -- 172.21.15.7:0/3020342809 wait complete. 2024-06-10T01:22:01.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.017+0000 7f329517eb80 1 -- start start 2024-06-10T01:22:01.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.017+0000 7f329517eb80 1 Processor -- start 2024-06-10T01:22:01.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.017+0000 7f329517eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da5edbc7a0 0x55da5edc85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.017+0000 7f329517eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5edc1f60 0x55da5edc58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.017+0000 7f329517eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da5edc2880 0x55da5edc5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.017+0000 7f329517eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55da5ecadfc0 con 0x55da5edc2880 2024-06-10T01:22:01.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.017+0000 7f329517eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55da5ed09810 con 0x55da5edbc7a0 2024-06-10T01:22:01.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.017+0000 7f329517eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55da5edc9260 con 0x55da5edc1f60 2024-06-10T01:22:01.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.017+0000 7f329517eb80 10 client.0 _ll_register_callbacks cb 0x55da5eca35b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:22:01.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.017+0000 7f3283fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da5edbc7a0 0x55da5edc85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.017+0000 7f3283fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da5edbc7a0 0x55da5edc85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:52280/0 (socket says 172.21.15.7:52280) 2024-06-10T01:22:01.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.017+0000 7f3283fff700 1 -- 172.21.15.7:0/4294574030 learned_addr learned my addr 172.21.15.7:0/4294574030 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:01.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.017+0000 7f327b7fe700 1 --2- 172.21.15.7:0/4294574030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5edc1f60 0x55da5edc58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.017+0000 7f3288e98700 1 --2- 172.21.15.7:0/4294574030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da5edc2880 0x55da5edc5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.018+0000 7f327b7fe700 1 -- 172.21.15.7:0/4294574030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da5edbc7a0 msgr2=0x55da5edc85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:01.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.018+0000 7f327b7fe700 1 --2- 172.21.15.7:0/4294574030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da5edbc7a0 0x55da5edc85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.018+0000 7f327b7fe700 1 -- 172.21.15.7:0/4294574030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da5edc2880 msgr2=0x55da5edc5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:01.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.018+0000 7f327b7fe700 1 --2- 172.21.15.7:0/4294574030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da5edc2880 0x55da5edc5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.018+0000 7f327b7fe700 1 -- 172.21.15.7:0/4294574030 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da5ecd3750 con 0x55da5edc1f60 2024-06-10T01:22:01.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.018+0000 7f3283fff700 1 --2- 172.21.15.7:0/4294574030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da5edbc7a0 0x55da5edc85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:22:01.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.018+0000 7f3288e98700 1 --2- 172.21.15.7:0/4294574030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da5edc2880 0x55da5edc5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:22:01.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.018+0000 7f327b7fe700 1 --2- 172.21.15.7:0/4294574030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5edc1f60 0x55da5edc58c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f326c00c5b0 tx=0x7f326c09b570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc1c6cfa2bff63e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:01.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.018+0000 7f32637fe700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:22:01.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.018+0000 7f32637fe700 1 -- 172.21.15.7:0/4294574030 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c09c130 con 0x55da5edc1f60 2024-06-10T01:22:01.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.018+0000 7f329517eb80 1 -- 172.21.15.7:0/4294574030 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3274005500 con 0x55da5edc1f60 2024-06-10T01:22:01.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.018+0000 7f32637fe700 1 -- 172.21.15.7:0/4294574030 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f326c0cf270 con 0x55da5edc1f60 2024-06-10T01:22:01.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.018+0000 7f32637fe700 1 -- 172.21.15.7:0/4294574030 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f326c09ba30 con 0x55da5edc1f60 2024-06-10T01:22:01.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.018+0000 7f329517eb80 10 client.28639 fetch_fsmap learned FSMap version 45 2024-06-10T01:22:01.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.018+0000 7f329517eb80 1 -- 172.21.15.7:0/4294574030 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55da5edd3a70 con 0x55da5edc1f60 2024-06-10T01:22:01.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.018+0000 7f32637fe700 1 -- 172.21.15.7:0/4294574030 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3274005500 con 0x55da5edc1f60 2024-06-10T01:22:01.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.019+0000 7f32637fe700 1 -- 172.21.15.7:0/4294574030 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 45) ==== 8358+0+0 (secure 0 0 0) 0x7f326c0a4210 con 0x55da5edc1f60 2024-06-10T01:22:01.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.019+0000 7f329517eb80 10 client.28639 fetch_fsmap finished waiting for FSMap version 45 2024-06-10T01:22:01.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.019+0000 7f329517eb80 10 client.28639 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:22:01.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.019+0000 7f329517eb80 20 client.28639 populate_metadata read hostname 'smithi007' 2024-06-10T01:22:01.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.019+0000 7f329517eb80 1 --2- 172.21.15.7:0/4294574030 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55da5edd6410 0x55da5edd8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.019+0000 7f329517eb80 4 client.28639 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:22:01.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.019+0000 7f329517eb80 1 -- 172.21.15.7:0/4294574030 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da5ed1ad50 con 0x55da5edd6410 2024-06-10T01:22:01.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.019+0000 7f3283fff700 1 --2- 172.21.15.7:0/4294574030 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55da5edd6410 0x55da5edd8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.019+0000 7f3283fff700 1 --2- 172.21.15.7:0/4294574030 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55da5edd6410 0x55da5edd8800 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b25df1f621fb9742 server_cookie=989d90d7beacce87 in_seq=0 out_seq=0 2024-06-10T01:22:01.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.019+0000 7f32637fe700 10 client.28639 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:22:01.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.024+0000 7f32637fe700 1 -- 172.21.15.7:0/4294574030 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55da5ed1ad50 con 0x55da5edd6410 2024-06-10T01:22:01.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.024+0000 7f32637fe700 10 client.28639 handle_command_reply: tid=0 2024-06-10T01:22:01.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.024+0000 7f329517eb80 1 client.28639 shutdown 2024-06-10T01:22:01.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.024+0000 7f329517eb80 1 -- 172.21.15.7:0/4294574030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5edc1f60 msgr2=0x55da5edc58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:01.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.024+0000 7f329517eb80 1 --2- 172.21.15.7:0/4294574030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5edc1f60 0x55da5edc58c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f326c00c5b0 tx=0x7f326c09b570 comp rx=0 tx=0).stop 2024-06-10T01:22:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.024+0000 7f329517eb80 1 -- 172.21.15.7:0/4294574030 shutdown_connections 2024-06-10T01:22:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.024+0000 7f329517eb80 1 --2- 172.21.15.7:0/4294574030 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55da5edd6410 0x55da5edd8800 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.024+0000 7f32637fe700 0 client.28639 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:22:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.024+0000 7f329517eb80 1 --2- 172.21.15.7:0/4294574030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da5edc2880 0x55da5edc5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.024+0000 7f329517eb80 1 --2- 172.21.15.7:0/4294574030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da5edbc7a0 0x55da5edc85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.024+0000 7f329517eb80 1 --2- 172.21.15.7:0/4294574030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5edc1f60 0x55da5edc58c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.024+0000 7f329517eb80 1 -- 172.21.15.7:0/4294574030 >> 172.21.15.7:0/4294574030 conn(0x55da5ece9080 msgr2=0x55da5edb6700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.024+0000 7f329517eb80 1 -- 172.21.15.7:0/4294574030 shutdown_connections 2024-06-10T01:22:01.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.024+0000 7f329517eb80 1 -- 172.21.15.7:0/4294574030 wait complete. 2024-06-10T01:22:01.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.024+0000 7f329517eb80 20 client.28639 trim_cache size 0 max 16384 2024-06-10T01:22:01.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.041+0000 7f329517eb80 1 Processor -- start 2024-06-10T01:22:01.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.041+0000 7f329517eb80 1 -- start start 2024-06-10T01:22:01.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.041+0000 7f329517eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da5edbc7a0 0x55da5edbc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.041+0000 7f329517eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5edc1f60 0x55da5edc4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.041+0000 7f329517eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da5edc2880 0x55da5edb6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.041+0000 7f329517eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f32841aea30 con 0x55da5edbc7a0 2024-06-10T01:22:01.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.041+0000 7f329517eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f32841ac260 con 0x55da5edc2880 2024-06-10T01:22:01.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.041+0000 7f329517eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f32841aa0d0 con 0x55da5edc1f60 2024-06-10T01:22:01.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.041+0000 7f3283fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da5edbc7a0 0x55da5edbc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.041+0000 7f3283fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da5edbc7a0 0x55da5edbc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52478/0 (socket says 172.21.15.7:52478) 2024-06-10T01:22:01.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.041+0000 7f3283fff700 1 -- 172.21.15.7:0/3449183019 learned_addr learned my addr 172.21.15.7:0/3449183019 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:01.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.041+0000 7f327b7fe700 1 --2- 172.21.15.7:0/3449183019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5edc1f60 0x55da5edc4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.041+0000 7f3288e98700 1 --2- 172.21.15.7:0/3449183019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da5edc2880 0x55da5edb6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.041+0000 7f3283fff700 1 -- 172.21.15.7:0/3449183019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5edc1f60 msgr2=0x55da5edc4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:01.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.041+0000 7f3283fff700 1 --2- 172.21.15.7:0/3449183019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5edc1f60 0x55da5edc4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.041+0000 7f3283fff700 1 -- 172.21.15.7:0/3449183019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da5edc2880 msgr2=0x55da5edb6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:01.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.041+0000 7f3283fff700 1 --2- 172.21.15.7:0/3449183019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da5edc2880 0x55da5edb6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.041+0000 7f3283fff700 1 -- 172.21.15.7:0/3449183019 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da5edc3e30 con 0x55da5edbc7a0 2024-06-10T01:22:01.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.041+0000 7f3288e98700 1 --2- 172.21.15.7:0/3449183019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da5edc2880 0x55da5edb6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:22:01.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.042+0000 7f3283fff700 1 --2- 172.21.15.7:0/3449183019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da5edbc7a0 0x55da5edbc260 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f3274011d10 tx=0x7f3274017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445e0982e20dd3f2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:01.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.042+0000 7f3279ffb700 1 -- 172.21.15.7:0/3449183019 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3274005450 con 0x55da5edbc7a0 2024-06-10T01:22:01.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.042+0000 7f3279ffb700 1 -- 172.21.15.7:0/3449183019 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32740055f0 con 0x55da5edbc7a0 2024-06-10T01:22:01.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.042+0000 7f3279ffb700 1 -- 172.21.15.7:0/3449183019 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3274004ce0 con 0x55da5edbc7a0 2024-06-10T01:22:01.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.042+0000 7f329517eb80 1 -- 172.21.15.7:0/3449183019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da5edbc7a0 msgr2=0x55da5edbc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:01.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.042+0000 7f329517eb80 1 --2- 172.21.15.7:0/3449183019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da5edbc7a0 0x55da5edbc260 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f3274011d10 tx=0x7f3274017000 comp rx=0 tx=0).stop 2024-06-10T01:22:01.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.042+0000 7f329517eb80 1 -- 172.21.15.7:0/3449183019 shutdown_connections 2024-06-10T01:22:01.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.042+0000 7f329517eb80 1 --2- 172.21.15.7:0/3449183019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da5edc2880 0x55da5edb6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.042+0000 7f329517eb80 1 --2- 172.21.15.7:0/3449183019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da5edbc7a0 0x55da5edbc260 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.042+0000 7f329517eb80 1 --2- 172.21.15.7:0/3449183019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5edc1f60 0x55da5edc4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.042+0000 7f329517eb80 1 -- 172.21.15.7:0/3449183019 >> 172.21.15.7:0/3449183019 conn(0x55da5ece9080 msgr2=0x55da5edc32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:01.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.043+0000 7f329517eb80 1 -- 172.21.15.7:0/3449183019 shutdown_connections 2024-06-10T01:22:01.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.043+0000 7f329517eb80 1 -- 172.21.15.7:0/3449183019 wait complete. 2024-06-10T01:22:01.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.043+0000 7f329517eb80 1 -- start start 2024-06-10T01:22:01.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.043+0000 7f329517eb80 1 Processor -- start 2024-06-10T01:22:01.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.043+0000 7f329517eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da5edbc7a0 0x55da5edd9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.043+0000 7f329517eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da5edc1f60 0x55da5edd9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.043+0000 7f329517eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5edc2880 0x55da5eddb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.043+0000 7f329517eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f32841aa720 con 0x55da5edc1f60 2024-06-10T01:22:01.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.043+0000 7f329517eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f32841abb90 con 0x55da5edbc7a0 2024-06-10T01:22:01.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.043+0000 7f329517eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f32841aadf0 con 0x55da5edc2880 2024-06-10T01:22:01.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.043+0000 7f329517eb80 10 client.0 _ll_register_callbacks cb 0x55da5eca35b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:22:01.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.043+0000 7f327b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da5edc1f60 0x55da5edd9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.043+0000 7f327b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da5edc1f60 0x55da5edd9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52482/0 (socket says 172.21.15.7:52482) 2024-06-10T01:22:01.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.043+0000 7f327b7fe700 1 -- 172.21.15.7:0/770326287 learned_addr learned my addr 172.21.15.7:0/770326287 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:01.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.043+0000 7f3283fff700 1 --2- 172.21.15.7:0/770326287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da5edbc7a0 0x55da5edd9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.043+0000 7f3288e98700 1 --2- 172.21.15.7:0/770326287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5edc2880 0x55da5eddb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.044+0000 7f327b7fe700 1 -- 172.21.15.7:0/770326287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5edc2880 msgr2=0x55da5eddb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:01.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.044+0000 7f327b7fe700 1 --2- 172.21.15.7:0/770326287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5edc2880 0x55da5eddb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.044+0000 7f327b7fe700 1 -- 172.21.15.7:0/770326287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da5edbc7a0 msgr2=0x55da5edd9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:01.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.044+0000 7f327b7fe700 1 --2- 172.21.15.7:0/770326287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da5edbc7a0 0x55da5edd9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.044+0000 7f327b7fe700 1 -- 172.21.15.7:0/770326287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da5eddbc80 con 0x55da5edc1f60 2024-06-10T01:22:01.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.044+0000 7f327b7fe700 1 --2- 172.21.15.7:0/770326287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da5edc1f60 0x55da5edd9590 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f326c097660 tx=0x7f326c0098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2440274f4ea97251 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:01.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.044+0000 7f32637fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:22:01.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.044+0000 7f32637fe700 1 -- 172.21.15.7:0/770326287 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c09c130 con 0x55da5edc1f60 2024-06-10T01:22:01.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.044+0000 7f32637fe700 1 -- 172.21.15.7:0/770326287 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f326c09ae80 con 0x55da5edc1f60 2024-06-10T01:22:01.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.044+0000 7f329517eb80 1 -- 172.21.15.7:0/770326287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32740055f0 con 0x55da5edc1f60 2024-06-10T01:22:01.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.044+0000 7f32637fe700 1 -- 172.21.15.7:0/770326287 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f326c08b050 con 0x55da5edc1f60 2024-06-10T01:22:01.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.044+0000 7f329517eb80 10 client.19404 fetch_fsmap learned FSMap version 45 2024-06-10T01:22:01.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.044+0000 7f329517eb80 1 -- 172.21.15.7:0/770326287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55da5ee408e0 con 0x55da5edc1f60 2024-06-10T01:22:01.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.044+0000 7f32637fe700 1 -- 172.21.15.7:0/770326287 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32740055f0 con 0x55da5edc1f60 2024-06-10T01:22:01.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.045+0000 7f32637fe700 1 -- 172.21.15.7:0/770326287 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 45) ==== 8358+0+0 (secure 0 0 0) 0x7f326c09a8f0 con 0x55da5edc1f60 2024-06-10T01:22:01.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.045+0000 7f329517eb80 10 client.19404 fetch_fsmap finished waiting for FSMap version 45 2024-06-10T01:22:01.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.045+0000 7f329517eb80 10 client.19404 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:22:01.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.045+0000 7f329517eb80 20 client.19404 populate_metadata read hostname 'smithi007' 2024-06-10T01:22:01.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.045+0000 7f329517eb80 1 --2- 172.21.15.7:0/770326287 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55da5edd6410 0x55da5ee43060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:01.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.045+0000 7f329517eb80 4 client.19404 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:22:01.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.045+0000 7f329517eb80 1 -- 172.21.15.7:0/770326287 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55da5ed152d0 con 0x55da5edd6410 2024-06-10T01:22:01.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.045+0000 7f3283fff700 1 --2- 172.21.15.7:0/770326287 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55da5edd6410 0x55da5ee43060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:01.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.045+0000 7f3283fff700 1 --2- 172.21.15.7:0/770326287 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55da5edd6410 0x55da5ee43060 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=43a57fda3e44cf00 server_cookie=b1cf0100535e1ad in_seq=0 out_seq=0 2024-06-10T01:22:01.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.045+0000 7f32637fe700 10 client.19404 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:22:01.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.049+0000 7f32637fe700 1 -- 172.21.15.7:0/770326287 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55da5ed152d0 con 0x55da5edd6410 2024-06-10T01:22:01.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.049+0000 7f32637fe700 10 client.19404 handle_command_reply: tid=0 2024-06-10T01:22:01.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.049+0000 7f329517eb80 1 client.19404 shutdown 2024-06-10T01:22:01.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.049+0000 7f329517eb80 1 -- 172.21.15.7:0/770326287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da5edc1f60 msgr2=0x55da5edd9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:01.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.049+0000 7f329517eb80 1 --2- 172.21.15.7:0/770326287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da5edc1f60 0x55da5edd9590 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f326c097660 tx=0x7f326c0098f0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.049+0000 7f329517eb80 1 -- 172.21.15.7:0/770326287 shutdown_connections 2024-06-10T01:22:01.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.049+0000 7f329517eb80 1 --2- 172.21.15.7:0/770326287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5edc2880 0x55da5eddb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.049+0000 7f329517eb80 1 --2- 172.21.15.7:0/770326287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da5edbc7a0 0x55da5edd9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.049+0000 7f329517eb80 1 --2- 172.21.15.7:0/770326287 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55da5edd6410 0x55da5ee43060 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.049+0000 7f329517eb80 1 --2- 172.21.15.7:0/770326287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da5edc1f60 0x55da5edd9590 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.049+0000 7f329517eb80 1 -- 172.21.15.7:0/770326287 >> 172.21.15.7:0/770326287 conn(0x55da5ece9080 msgr2=0x55da5edc7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:01.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.049+0000 7f32637fe700 0 client.19404 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:22:01.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.049+0000 7f329517eb80 1 -- 172.21.15.7:0/770326287 shutdown_connections 2024-06-10T01:22:01.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.049+0000 7f329517eb80 1 -- 172.21.15.7:0/770326287 wait complete. 2024-06-10T01:22:01.564 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:22:01.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.049+0000 7f329517eb80 20 client.19404 trim_cache size 0 max 16384 2024-06-10T01:22:01.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.050+0000 7f328a8fb700 1 -- 172.21.15.7:0/1335138832 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f32700a2ca0 msgr2=0x7f32700a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:01.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.050+0000 7f328a8fb700 1 --2- 172.21.15.7:0/1335138832 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f32700a2ca0 0x7f32700a5160 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3284199b30 tx=0x7f3274009310 comp rx=0 tx=0).stop 2024-06-10T01:22:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.050+0000 7f328a8fb700 1 -- 172.21.15.7:0/1335138832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f328406b3c0 msgr2=0x7f328419ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.050+0000 7f328a8fb700 1 --2- 172.21.15.7:0/1335138832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f328406b3c0 0x7f328419ed70 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f326c00eb30 tx=0x7f326c00ee40 comp rx=0 tx=0).stop 2024-06-10T01:22:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.050+0000 7f328a8fb700 1 -- 172.21.15.7:0/1335138832 shutdown_connections 2024-06-10T01:22:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.050+0000 7f328a8fb700 1 --2- 172.21.15.7:0/1335138832 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f32700a2ca0 0x7f32700a5160 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.050+0000 7f328a8fb700 1 --2- 172.21.15.7:0/1335138832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32841084f0 0x7f32841989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.050+0000 7f328a8fb700 1 --2- 172.21.15.7:0/1335138832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f328406aa00 0x7f328419e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.050+0000 7f328a8fb700 1 --2- 172.21.15.7:0/1335138832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f328406b3c0 0x7f328419ed70 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.050+0000 7f328a8fb700 1 -- 172.21.15.7:0/1335138832 >> 172.21.15.7:0/1335138832 conn(0x7f3284078300 msgr2=0x7f32840fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.050+0000 7f328a8fb700 1 -- 172.21.15.7:0/1335138832 shutdown_connections 2024-06-10T01:22:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:01.050+0000 7f328a8fb700 1 -- 172.21.15.7:0/1335138832 wait complete. 2024-06-10T01:22:01.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:01 smithi007 ceph-mon[32398]: pgmap v912: 129 pgs: 129 active+clean; 509 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 11 MiB/s wr, 890 op/s 2024-06-10T01:22:01.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:01 smithi078 ceph-mon[36683]: pgmap v912: 129 pgs: 129 active+clean; 509 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 11 MiB/s wr, 890 op/s 2024-06-10T01:22:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.834+0000 7f4e9dffb640 1 -- 172.21.15.7:0/4024219259 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d7898f0f10 con 0x55d789a0c5c0 2024-06-10T01:22:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.834+0000 7f4e9dffb640 10 client.28663 handle_command_reply: tid=0 2024-06-10T01:22:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.834+0000 7f4ebf5ff740 1 client.28663 shutdown 2024-06-10T01:22:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.834+0000 7f4ebf5ff740 1 -- 172.21.15.7:0/4024219259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78997a010 msgr2=0x55d789999d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.834+0000 7f4ebf5ff740 1 --2- 172.21.15.7:0/4024219259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78997a010 0x55d789999d50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8011710 tx=0x7f4ea8011f60 comp rx=0 tx=0).stop 2024-06-10T01:22:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.834+0000 7f4ebf5ff740 1 -- 172.21.15.7:0/4024219259 shutdown_connections 2024-06-10T01:22:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.834+0000 7f4ebf5ff740 1 --2- 172.21.15.7:0/4024219259 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d789a0c5c0 0x55d789a0e9e0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.834+0000 7f4ebf5ff740 1 --2- 172.21.15.7:0/4024219259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78997fb70 0x55d78999b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.834+0000 7f4e9dffb640 0 client.28663 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.834+0000 7f4ebf5ff740 1 --2- 172.21.15.7:0/4024219259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78997aa00 0x55d78999a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.834+0000 7f4ebf5ff740 1 --2- 172.21.15.7:0/4024219259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78997a010 0x55d789999d50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.834+0000 7f4ebf5ff740 1 -- 172.21.15.7:0/4024219259 >> 172.21.15.7:0/4024219259 conn(0x55d789975680 msgr2=0x55d78997eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.834+0000 7f4ebf5ff740 1 -- 172.21.15.7:0/4024219259 shutdown_connections 2024-06-10T01:22:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.834+0000 7f4ebf5ff740 1 -- 172.21.15.7:0/4024219259 wait complete. 2024-06-10T01:22:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.834+0000 7f4ebf5ff740 20 client.28663 trim_cache size 0 max 16384 2024-06-10T01:22:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:22:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:22:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:22:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.835+0000 7f4ebc815640 1 -- 172.21.15.7:0/1652514906 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4ea00a2c00 msgr2=0x7f4ea00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.835+0000 7f4ebc815640 1 --2- 172.21.15.7:0/1652514906 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4ea00a2c00 0x7f4ea00a50c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4eac023000 tx=0x7f4eac024000 comp rx=0 tx=0).stop 2024-06-10T01:22:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.835+0000 7f4ebc815640 1 -- 172.21.15.7:0/1652514906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4eb81076e0 msgr2=0x7f4eb81a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.835+0000 7f4ebc815640 1 --2- 172.21.15.7:0/1652514906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4eb81076e0 0x7f4eb81a5880 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4eb002a930 tx=0x7f4eb0033d90 comp rx=0 tx=0).stop 2024-06-10T01:22:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.835+0000 7f4ebc815640 1 -- 172.21.15.7:0/1652514906 shutdown_connections 2024-06-10T01:22:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.835+0000 7f4ebc815640 1 --2- 172.21.15.7:0/1652514906 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4ea00a2c00 0x7f4ea00a50c0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.835+0000 7f4ebc815640 1 --2- 172.21.15.7:0/1652514906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4eb81076e0 0x7f4eb81a5880 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.835+0000 7f4ebc815640 1 --2- 172.21.15.7:0/1652514906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4eb8104db0 0x7f4eb819e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.835+0000 7f4ebc815640 1 --2- 172.21.15.7:0/1652514906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4eb8102480 0x7f4eb819e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.835+0000 7f4ebc815640 1 -- 172.21.15.7:0/1652514906 >> 172.21.15.7:0/1652514906 conn(0x7f4eb80fbe70 msgr2=0x7f4eb8102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.835+0000 7f4ebc815640 1 -- 172.21.15.7:0/1652514906 shutdown_connections 2024-06-10T01:22:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:01.835+0000 7f4ebc815640 1 -- 172.21.15.7:0/1652514906 wait complete. 2024-06-10T01:22:01.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:01 smithi175 ceph-mon[36636]: pgmap v912: 129 pgs: 129 active+clean; 509 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 11 MiB/s wr, 890 op/s 2024-06-10T01:22:02.521 INFO:tasks.workunit.client.0.smithi007.stdout: 10 64890 33.47 MB/sec execute 39 sec latency 424.154 ms 2024-06-10T01:22:02.531 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:22:02.959 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:22:03.522 INFO:tasks.workunit.client.0.smithi007.stdout: 10 65197 33.13 MB/sec execute 40 sec latency 227.082 ms 2024-06-10T01:22:03.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:03 smithi007 ceph-mon[32398]: pgmap v913: 129 pgs: 129 active+clean; 508 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 9.0 MiB/s wr, 683 op/s 2024-06-10T01:22:03.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:03 smithi078 ceph-mon[36683]: pgmap v913: 129 pgs: 129 active+clean; 508 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 9.0 MiB/s wr, 683 op/s 2024-06-10T01:22:03.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:03 smithi175 ceph-mon[36636]: pgmap v913: 129 pgs: 129 active+clean; 508 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 9.0 MiB/s wr, 683 op/s 2024-06-10T01:22:04.522 INFO:tasks.workunit.client.0.smithi007.stdout: 10 65762 32.90 MB/sec execute 41 sec latency 208.609 ms 2024-06-10T01:22:04.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.946+0000 7f08f94eb700 1 -- 172.21.15.7:0/2041808648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08f41073c0 msgr2=0x7f08f41077a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:04.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.946+0000 7f08f94eb700 1 --2- 172.21.15.7:0/2041808648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08f41073c0 0x7f08f41077a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f08e4009990 tx=0x7f08e4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:22:04.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.946+0000 7f08f94eb700 1 -- 172.21.15.7:0/2041808648 shutdown_connections 2024-06-10T01:22:04.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.946+0000 7f08f94eb700 1 --2- 172.21.15.7:0/2041808648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08f4101d40 0x7f08f410e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:04.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.946+0000 7f08f94eb700 1 --2- 172.21.15.7:0/2041808648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08f41073c0 0x7f08f41077a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:04.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.946+0000 7f08f94eb700 1 --2- 172.21.15.7:0/2041808648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08f4101380 0x7f08f4101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:04.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.946+0000 7f08f94eb700 1 -- 172.21.15.7:0/2041808648 >> 172.21.15.7:0/2041808648 conn(0x7f08f4078330 msgr2=0x7f08f4078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:04.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.946+0000 7f08f94eb700 1 -- 172.21.15.7:0/2041808648 shutdown_connections 2024-06-10T01:22:04.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.946+0000 7f08f94eb700 1 -- 172.21.15.7:0/2041808648 wait complete. 2024-06-10T01:22:04.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.947+0000 7f08f94eb700 1 Processor -- start 2024-06-10T01:22:04.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.947+0000 7f08f94eb700 1 -- start start 2024-06-10T01:22:04.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.948+0000 7f08f94eb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08f4101380 0x7f08f4103030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:04.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.948+0000 7f08f94eb700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08f4101d40 0x7f08f4103570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:04.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.948+0000 7f08f94eb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08f41073c0 0x7f08f4103ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:04.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.948+0000 7f08f94eb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f08f4111660 con 0x7f08f4101380 2024-06-10T01:22:04.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.948+0000 7f08f94eb700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f08f41114e0 con 0x7f08f41073c0 2024-06-10T01:22:04.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.948+0000 7f08f94eb700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f08f41117e0 con 0x7f08f4101d40 2024-06-10T01:22:04.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.948+0000 7f08f27fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08f4101d40 0x7f08f4103570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:04.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.948+0000 7f08f27fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08f4101d40 0x7f08f4103570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:33804/0 (socket says 172.21.15.7:33804) 2024-06-10T01:22:04.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.948+0000 7f08f27fc700 1 -- 172.21.15.7:0/4291886417 learned_addr learned my addr 172.21.15.7:0/4291886417 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:04.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.948+0000 7f08f2ffd700 1 --2- 172.21.15.7:0/4291886417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08f4101380 0x7f08f4103030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:04.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.948+0000 7f08f37fe700 1 --2- 172.21.15.7:0/4291886417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08f41073c0 0x7f08f4103ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:04.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.948+0000 7f08f2ffd700 1 -- 172.21.15.7:0/4291886417 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08f4101d40 msgr2=0x7f08f4103570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:04.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.948+0000 7f08f2ffd700 1 --2- 172.21.15.7:0/4291886417 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08f4101d40 0x7f08f4103570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:04.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.948+0000 7f08f2ffd700 1 -- 172.21.15.7:0/4291886417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08f41073c0 msgr2=0x7f08f4103ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:04.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.948+0000 7f08f2ffd700 1 --2- 172.21.15.7:0/4291886417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08f41073c0 0x7f08f4103ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:04.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.948+0000 7f08f2ffd700 1 -- 172.21.15.7:0/4291886417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08f41a7130 con 0x7f08f4101380 2024-06-10T01:22:04.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.949+0000 7f08f2ffd700 1 --2- 172.21.15.7:0/4291886417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08f4101380 0x7f08f4103030 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f08e4006010 tx=0x7f08e400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:04.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.949+0000 7f08dbfff700 1 -- 172.21.15.7:0/4291886417 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e4027070 con 0x7f08f4101380 2024-06-10T01:22:04.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.949+0000 7f08dbfff700 1 -- 172.21.15.7:0/4291886417 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08e40056d0 con 0x7f08f4101380 2024-06-10T01:22:04.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.949+0000 7f08f94eb700 1 -- 172.21.15.7:0/4291886417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08f41a73c0 con 0x7f08f4101380 2024-06-10T01:22:04.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.949+0000 7f08dbfff700 1 -- 172.21.15.7:0/4291886417 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e4022cf0 con 0x7f08f4101380 2024-06-10T01:22:04.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.949+0000 7f08f94eb700 1 -- 172.21.15.7:0/4291886417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08f41a7870 con 0x7f08f4101380 2024-06-10T01:22:04.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.952+0000 7f08dbfff700 1 -- 172.21.15.7:0/4291886417 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f08e402a070 con 0x7f08f4101380 2024-06-10T01:22:04.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.953+0000 7f08dbfff700 1 --2- 172.21.15.7:0/4291886417 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f08dc0a2ca0 0x7f08dc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:04.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.954+0000 7f08dbfff700 1 -- 172.21.15.7:0/4291886417 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f08e40d0300 con 0x7f08f4101380 2024-06-10T01:22:04.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.954+0000 7f08f27fc700 1 --2- 172.21.15.7:0/4291886417 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f08dc0a2ca0 0x7f08dc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:04.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.955+0000 7f0903d6eb80 1 Processor -- start 2024-06-10T01:22:04.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.955+0000 7f08f27fc700 1 --2- 172.21.15.7:0/4291886417 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f08dc0a2ca0 0x7f08dc0a5160 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f08f4078910 tx=0x7f08e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:04.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.955+0000 7f0903d6eb80 1 -- start start 2024-06-10T01:22:04.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.956+0000 7f0903d6eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945ae82640 0x55945ae82a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:04.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.956+0000 7f0903d6eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945ae82f60 0x55945ae7c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:04.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.956+0000 7f0903d6eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55945ae7cf20 0x55945ae7d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:04.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.956+0000 7f0903d6eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55945adce810 con 0x55945ae7cf20 2024-06-10T01:22:04.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.956+0000 7f0903d6eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55945ad76720 con 0x55945ae82f60 2024-06-10T01:22:04.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.956+0000 7f0903d6eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55945adca7d0 con 0x55945ae82640 2024-06-10T01:22:04.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.956+0000 7f08f2ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945ae82640 0x55945ae82a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:04.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.956+0000 7f08f37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55945ae7cf20 0x55945ae7d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:04.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.956+0000 7f08f27fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945ae82f60 0x55945ae7c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:04.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.956+0000 7f08f37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55945ae7cf20 0x55945ae7d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58050/0 (socket says 172.21.15.7:58050) 2024-06-10T01:22:04.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.956+0000 7f08f27fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945ae82f60 0x55945ae7c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:53954/0 (socket says 172.21.15.7:53954) 2024-06-10T01:22:04.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.956+0000 7f08f37fe700 1 -- 172.21.15.7:0/2513677998 learned_addr learned my addr 172.21.15.7:0/2513677998 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:04.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.956+0000 7f08f37fe700 1 -- 172.21.15.7:0/2513677998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945ae82640 msgr2=0x55945ae82a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:04.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.956+0000 7f08f37fe700 1 --2- 172.21.15.7:0/2513677998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945ae82640 0x55945ae82a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:04.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.956+0000 7f08f37fe700 1 -- 172.21.15.7:0/2513677998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945ae82f60 msgr2=0x55945ae7c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:04.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.956+0000 7f08f37fe700 1 --2- 172.21.15.7:0/2513677998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945ae82f60 0x55945ae7c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:04.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.956+0000 7f08f37fe700 1 -- 172.21.15.7:0/2513677998 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55945ad6cf50 con 0x55945ae7cf20 2024-06-10T01:22:04.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.957+0000 7f08f37fe700 1 --2- 172.21.15.7:0/2513677998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55945ae7cf20 0x55945ae7d340 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f08ec011630 tx=0x7f08ec009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b2d85b646957832 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:04.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.957+0000 7f08f27fc700 1 --2- 172.21.15.7:0/2513677998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945ae82f60 0x55945ae7c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-06-10T01:22:04.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.957+0000 7f08d97fa700 1 -- 172.21.15.7:0/2513677998 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec003a00 con 0x55945ae7cf20 2024-06-10T01:22:04.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.957+0000 7f08d97fa700 1 -- 172.21.15.7:0/2513677998 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08ec01a070 con 0x55945ae7cf20 2024-06-10T01:22:04.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.957+0000 7f08d97fa700 1 -- 172.21.15.7:0/2513677998 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec007760 con 0x55945ae7cf20 2024-06-10T01:22:04.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.958+0000 7f0903d6eb80 1 -- 172.21.15.7:0/2513677998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55945ae7cf20 msgr2=0x55945ae7d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:04.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.958+0000 7f0903d6eb80 1 --2- 172.21.15.7:0/2513677998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55945ae7cf20 0x55945ae7d340 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f08ec011630 tx=0x7f08ec009920 comp rx=0 tx=0).stop 2024-06-10T01:22:04.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.958+0000 7f0903d6eb80 1 -- 172.21.15.7:0/2513677998 shutdown_connections 2024-06-10T01:22:04.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.958+0000 7f0903d6eb80 1 --2- 172.21.15.7:0/2513677998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55945ae7cf20 0x55945ae7d340 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:04.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.958+0000 7f0903d6eb80 1 --2- 172.21.15.7:0/2513677998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945ae82640 0x55945ae82a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:04.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.958+0000 7f0903d6eb80 1 --2- 172.21.15.7:0/2513677998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945ae82f60 0x55945ae7c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:04.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.958+0000 7f0903d6eb80 1 -- 172.21.15.7:0/2513677998 >> 172.21.15.7:0/2513677998 conn(0x55945ada7cc0 msgr2=0x55945ae77850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.958+0000 7f0903d6eb80 1 -- 172.21.15.7:0/2513677998 shutdown_connections 2024-06-10T01:22:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.959+0000 7f0903d6eb80 1 -- 172.21.15.7:0/2513677998 wait complete. 2024-06-10T01:22:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.959+0000 7f0903d6eb80 1 -- start start 2024-06-10T01:22:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.959+0000 7f0903d6eb80 1 Processor -- start 2024-06-10T01:22:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.960+0000 7f0903d6eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945ae7cf20 0x55945ae88bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.960+0000 7f0903d6eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945ae82640 0x55945ae85ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.960+0000 7f0903d6eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55945ae82f60 0x55945ae86410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.960+0000 7f08f37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55945ae82f60 0x55945ae86410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.960+0000 7f08f2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945ae7cf20 0x55945ae88bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.960+0000 7f08f37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55945ae82f60 0x55945ae86410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58060/0 (socket says 172.21.15.7:58060) 2024-06-10T01:22:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.960+0000 7f08f2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945ae7cf20 0x55945ae88bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:53962/0 (socket says 172.21.15.7:53962) 2024-06-10T01:22:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.960+0000 7f08f27fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945ae82640 0x55945ae85ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:04.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.960+0000 7f08f27fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945ae82640 0x55945ae85ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:33834/0 (socket says 172.21.15.7:33834) 2024-06-10T01:22:04.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.960+0000 7f08f37fe700 1 -- 172.21.15.7:0/2805773556 learned_addr learned my addr 172.21.15.7:0/2805773556 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:04.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.960+0000 7f0903d6eb80 1 -- 172.21.15.7:0/2805773556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55945ae899c0 con 0x55945ae82f60 2024-06-10T01:22:04.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.960+0000 7f0903d6eb80 1 -- 172.21.15.7:0/2805773556 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55945ad6ebf0 con 0x55945ae7cf20 2024-06-10T01:22:04.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.960+0000 7f0903d6eb80 1 -- 172.21.15.7:0/2805773556 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55945ae89b40 con 0x55945ae82640 2024-06-10T01:22:04.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.961+0000 7f0903d6eb80 10 client.0 _ll_register_callbacks cb 0x55945ad68920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:22:04.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.961+0000 7f08f37fe700 1 -- 172.21.15.7:0/2805773556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945ae82640 msgr2=0x55945ae85ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:04.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.961+0000 7f08f37fe700 1 --2- 172.21.15.7:0/2805773556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945ae82640 0x55945ae85ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:04.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.961+0000 7f08f37fe700 1 -- 172.21.15.7:0/2805773556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945ae7cf20 msgr2=0x55945ae88bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:04.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.961+0000 7f08f37fe700 1 --2- 172.21.15.7:0/2805773556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945ae7cf20 0x55945ae88bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:04.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.961+0000 7f08f37fe700 1 -- 172.21.15.7:0/2805773556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55945ad1d8d0 con 0x55945ae82f60 2024-06-10T01:22:04.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.961+0000 7f08f37fe700 1 --2- 172.21.15.7:0/2805773556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55945ae82f60 0x55945ae86410 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0112e0 tx=0x7f08ec009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2c9225f1ac64ac server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:04.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.961+0000 7f08d6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:22:04.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.962+0000 7f08d6ff5700 1 -- 172.21.15.7:0/2805773556 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec004050 con 0x55945ae82f60 2024-06-10T01:22:04.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.962+0000 7f08d6ff5700 1 -- 172.21.15.7:0/2805773556 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f08ec0041f0 con 0x55945ae82f60 2024-06-10T01:22:04.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.962+0000 7f0903d6eb80 1 -- 172.21.15.7:0/2805773556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08ec01a070 con 0x55945ae82f60 2024-06-10T01:22:04.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.962+0000 7f08d6ff5700 1 -- 172.21.15.7:0/2805773556 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f08ec0043f0 con 0x55945ae82f60 2024-06-10T01:22:04.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.962+0000 7f0903d6eb80 10 client.19431 fetch_fsmap learned FSMap version 45 2024-06-10T01:22:04.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.962+0000 7f0903d6eb80 1 -- 172.21.15.7:0/2805773556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55945ae94180 con 0x55945ae82f60 2024-06-10T01:22:04.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.962+0000 7f08d6ff5700 1 -- 172.21.15.7:0/2805773556 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08ec01a070 con 0x55945ae82f60 2024-06-10T01:22:04.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.962+0000 7f08d6ff5700 1 -- 172.21.15.7:0/2805773556 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 45) ==== 8358+0+0 (secure 0 0 0) 0x7f08ec015750 con 0x55945ae82f60 2024-06-10T01:22:04.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.963+0000 7f0903d6eb80 10 client.19431 fetch_fsmap finished waiting for FSMap version 45 2024-06-10T01:22:04.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.963+0000 7f0903d6eb80 10 client.19431 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:22:04.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.963+0000 7f0903d6eb80 20 client.19431 populate_metadata read hostname 'smithi007' 2024-06-10T01:22:04.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.963+0000 7f0903d6eb80 1 --2- 172.21.15.7:0/2805773556 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55945ae96760 0x55945ae98b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:04.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.963+0000 7f0903d6eb80 4 client.19431 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:22:04.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.963+0000 7f0903d6eb80 1 -- 172.21.15.7:0/2805773556 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55945addbd10 con 0x55945ae96760 2024-06-10T01:22:04.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.963+0000 7f08f2ffd700 1 --2- 172.21.15.7:0/2805773556 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55945ae96760 0x55945ae98b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:04.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.964+0000 7f08d6ff5700 10 client.19431 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:22:04.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.964+0000 7f08f2ffd700 1 --2- 172.21.15.7:0/2805773556 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55945ae96760 0x55945ae98b50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9dbeea354b0b8151 server_cookie=d6d417bda1784220 in_seq=0 out_seq=0 2024-06-10T01:22:04.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.966+0000 7f08d6ff5700 1 -- 172.21.15.7:0/2805773556 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55945addbd10 con 0x55945ae96760 2024-06-10T01:22:04.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.966+0000 7f08d6ff5700 10 client.19431 handle_command_reply: tid=0 2024-06-10T01:22:04.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.966+0000 7f0903d6eb80 1 client.19431 shutdown 2024-06-10T01:22:04.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.966+0000 7f0903d6eb80 1 -- 172.21.15.7:0/2805773556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55945ae82f60 msgr2=0x55945ae86410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:04.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.966+0000 7f0903d6eb80 1 --2- 172.21.15.7:0/2805773556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55945ae82f60 0x55945ae86410 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0112e0 tx=0x7f08ec009c40 comp rx=0 tx=0).stop 2024-06-10T01:22:04.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.967+0000 7f0903d6eb80 1 -- 172.21.15.7:0/2805773556 shutdown_connections 2024-06-10T01:22:04.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.967+0000 7f0903d6eb80 1 --2- 172.21.15.7:0/2805773556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55945ae82f60 0x55945ae86410 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:04.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.967+0000 7f0903d6eb80 1 --2- 172.21.15.7:0/2805773556 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55945ae96760 0x55945ae98b50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:04.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.967+0000 7f08d6ff5700 0 client.19431 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:22:04.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.967+0000 7f0903d6eb80 1 --2- 172.21.15.7:0/2805773556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945ae7cf20 0x55945ae88bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:04.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.967+0000 7f0903d6eb80 1 --2- 172.21.15.7:0/2805773556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945ae82640 0x55945ae85ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:04.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.967+0000 7f0903d6eb80 1 -- 172.21.15.7:0/2805773556 >> 172.21.15.7:0/2805773556 conn(0x55945ada7cc0 msgr2=0x55945ae7b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:04.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.967+0000 7f0903d6eb80 1 -- 172.21.15.7:0/2805773556 shutdown_connections 2024-06-10T01:22:04.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.967+0000 7f0903d6eb80 1 -- 172.21.15.7:0/2805773556 wait complete. 2024-06-10T01:22:04.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.967+0000 7f0903d6eb80 20 client.19431 trim_cache size 0 max 16384 2024-06-10T01:22:04.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.990+0000 7f0903d6eb80 1 Processor -- start 2024-06-10T01:22:04.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.990+0000 7f0903d6eb80 1 -- start start 2024-06-10T01:22:04.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.990+0000 7f0903d6eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945ae7cf20 0x55945ae7c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:04.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.990+0000 7f0903d6eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55945ae82640 0x55945ae83a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:04.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.990+0000 7f0903d6eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945ae82f60 0x55945ae9e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:04.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.990+0000 7f0903d6eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f08f41b2ce0 con 0x55945ae82640 2024-06-10T01:22:04.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.990+0000 7f0903d6eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f08f41b0510 con 0x55945ae7cf20 2024-06-10T01:22:04.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.990+0000 7f0903d6eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f08f41ae360 con 0x55945ae82f60 2024-06-10T01:22:04.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.990+0000 7f08f2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945ae7cf20 0x55945ae7c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:04.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.990+0000 7f08f2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945ae7cf20 0x55945ae7c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:53978/0 (socket says 172.21.15.7:53978) 2024-06-10T01:22:04.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.990+0000 7f08f2ffd700 1 -- 172.21.15.7:0/802726515 learned_addr learned my addr 172.21.15.7:0/802726515 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:04.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.990+0000 7f08f27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55945ae82640 0x55945ae83a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:04.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.990+0000 7f08f37fe700 1 --2- 172.21.15.7:0/802726515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945ae82f60 0x55945ae9e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:04.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.990+0000 7f08f2ffd700 1 -- 172.21.15.7:0/802726515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945ae82f60 msgr2=0x55945ae9e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:04.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.990+0000 7f08f2ffd700 1 --2- 172.21.15.7:0/802726515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945ae82f60 0x55945ae9e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:04.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.990+0000 7f08f2ffd700 1 -- 172.21.15.7:0/802726515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55945ae82640 msgr2=0x55945ae83a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:04.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.990+0000 7f08f2ffd700 1 --2- 172.21.15.7:0/802726515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55945ae82640 0x55945ae83a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:04.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.990+0000 7f08f2ffd700 1 -- 172.21.15.7:0/802726515 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55945ae78520 con 0x55945ae7cf20 2024-06-10T01:22:04.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.991+0000 7f08f2ffd700 1 --2- 172.21.15.7:0/802726515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945ae7cf20 0x55945ae7c9e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f08e4009550 tx=0x7f08e401a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac7bbc522d2523d1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:04.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.991+0000 7f08d8ff9700 1 -- 172.21.15.7:0/802726515 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e409c1d0 con 0x55945ae7cf20 2024-06-10T01:22:04.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.991+0000 7f08d8ff9700 1 -- 172.21.15.7:0/802726515 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08e409cf70 con 0x55945ae7cf20 2024-06-10T01:22:04.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.991+0000 7f08d8ff9700 1 -- 172.21.15.7:0/802726515 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e40a30b0 con 0x55945ae7cf20 2024-06-10T01:22:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.992+0000 7f0903d6eb80 1 -- 172.21.15.7:0/802726515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945ae7cf20 msgr2=0x55945ae7c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.992+0000 7f0903d6eb80 1 --2- 172.21.15.7:0/802726515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945ae7cf20 0x55945ae7c9e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f08e4009550 tx=0x7f08e401a040 comp rx=0 tx=0).stop 2024-06-10T01:22:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.992+0000 7f0903d6eb80 1 -- 172.21.15.7:0/802726515 shutdown_connections 2024-06-10T01:22:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.992+0000 7f0903d6eb80 1 --2- 172.21.15.7:0/802726515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945ae82f60 0x55945ae9e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.992+0000 7f0903d6eb80 1 --2- 172.21.15.7:0/802726515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945ae7cf20 0x55945ae7c9e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.992+0000 7f0903d6eb80 1 --2- 172.21.15.7:0/802726515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55945ae82640 0x55945ae83a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.992+0000 7f0903d6eb80 1 -- 172.21.15.7:0/802726515 >> 172.21.15.7:0/802726515 conn(0x55945ada7cc0 msgr2=0x55945ae83340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:04.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.992+0000 7f0903d6eb80 1 -- 172.21.15.7:0/802726515 shutdown_connections 2024-06-10T01:22:04.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.992+0000 7f0903d6eb80 1 -- 172.21.15.7:0/802726515 wait complete. 2024-06-10T01:22:04.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.992+0000 7f0903d6eb80 1 -- start start 2024-06-10T01:22:04.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.993+0000 7f0903d6eb80 1 Processor -- start 2024-06-10T01:22:04.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.993+0000 7f0903d6eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945ae7cf20 0x55945ae8ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:04.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.993+0000 7f0903d6eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945ae82640 0x55945ae8f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:04.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.993+0000 7f0903d6eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55945ae82f60 0x55945ae9d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:04.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.993+0000 7f0903d6eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f08f41ae9d0 con 0x55945ae82f60 2024-06-10T01:22:04.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.993+0000 7f0903d6eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f08f41afe40 con 0x55945ae7cf20 2024-06-10T01:22:04.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.993+0000 7f0903d6eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f08f41af0a0 con 0x55945ae82640 2024-06-10T01:22:04.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.993+0000 7f0903d6eb80 10 client.0 _ll_register_callbacks cb 0x55945ad68920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:22:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.993+0000 7f08f2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945ae7cf20 0x55945ae8ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.993+0000 7f08f2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945ae7cf20 0x55945ae8ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:53986/0 (socket says 172.21.15.7:53986) 2024-06-10T01:22:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.993+0000 7f08f2ffd700 1 -- 172.21.15.7:0/478984552 learned_addr learned my addr 172.21.15.7:0/478984552 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.993+0000 7f08f27fc700 1 --2- 172.21.15.7:0/478984552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945ae82640 0x55945ae8f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.994+0000 7f08f2ffd700 1 -- 172.21.15.7:0/478984552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945ae82640 msgr2=0x55945ae8f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:04.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.994+0000 7f08f2ffd700 1 --2- 172.21.15.7:0/478984552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945ae82640 0x55945ae8f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:04.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.994+0000 7f08f2ffd700 1 -- 172.21.15.7:0/478984552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55945ae82f60 msgr2=0x55945ae9d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:04.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.994+0000 7f08f2ffd700 1 --2- 172.21.15.7:0/478984552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55945ae82f60 0x55945ae9d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:04.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.994+0000 7f08f2ffd700 1 -- 172.21.15.7:0/478984552 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55945ae9dac0 con 0x55945ae7cf20 2024-06-10T01:22:04.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.994+0000 7f08f2ffd700 1 --2- 172.21.15.7:0/478984552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945ae7cf20 0x55945ae8ee60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f08e40098e0 tx=0x7f08e4003e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16d4acff6e01196d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:04.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.994+0000 7f08d6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:22:04.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.994+0000 7f08d6ff5700 1 -- 172.21.15.7:0/478984552 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e409c870 con 0x55945ae7cf20 2024-06-10T01:22:04.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.994+0000 7f0903d6eb80 1 -- 172.21.15.7:0/478984552 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08e409cf70 con 0x55945ae7cf20 2024-06-10T01:22:04.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.994+0000 7f08d6ff5700 1 -- 172.21.15.7:0/478984552 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f08e40a30e0 con 0x55945ae7cf20 2024-06-10T01:22:04.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.994+0000 7f08d6ff5700 1 -- 172.21.15.7:0/478984552 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f08e40b4020 con 0x55945ae7cf20 2024-06-10T01:22:04.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.994+0000 7f0903d6eb80 10 client.28670 fetch_fsmap learned FSMap version 45 2024-06-10T01:22:04.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.994+0000 7f0903d6eb80 1 -- 172.21.15.7:0/478984552 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55945af00bf0 con 0x55945ae7cf20 2024-06-10T01:22:04.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.994+0000 7f08d6ff5700 1 -- 172.21.15.7:0/478984552 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08e409cf70 con 0x55945ae7cf20 2024-06-10T01:22:04.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.995+0000 7f08d6ff5700 1 -- 172.21.15.7:0/478984552 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 45) ==== 8358+0+0 (secure 0 0 0) 0x7f08e409cb60 con 0x55945ae7cf20 2024-06-10T01:22:04.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.995+0000 7f0903d6eb80 10 client.28670 fetch_fsmap finished waiting for FSMap version 45 2024-06-10T01:22:04.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.995+0000 7f0903d6eb80 10 client.28670 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:22:04.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.995+0000 7f0903d6eb80 20 client.28670 populate_metadata read hostname 'smithi007' 2024-06-10T01:22:04.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.995+0000 7f0903d6eb80 1 --2- 172.21.15.7:0/478984552 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55945ae96760 0x55945af031f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:04.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.995+0000 7f0903d6eb80 4 client.28670 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:22:04.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.995+0000 7f0903d6eb80 1 -- 172.21.15.7:0/478984552 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55945add6290 con 0x55945ae96760 2024-06-10T01:22:04.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.995+0000 7f08f27fc700 1 --2- 172.21.15.7:0/478984552 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55945ae96760 0x55945af031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:04.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.995+0000 7f08f27fc700 1 --2- 172.21.15.7:0/478984552 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55945ae96760 0x55945af031f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c9984a3eb406d0d7 server_cookie=e81ba8fdbf7c51ff in_seq=0 out_seq=0 2024-06-10T01:22:05.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.995+0000 7f08d6ff5700 10 client.28670 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:22:05.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.999+0000 7f08d6ff5700 1 -- 172.21.15.7:0/478984552 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55945add6290 con 0x55945ae96760 2024-06-10T01:22:05.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.999+0000 7f08d6ff5700 10 client.28670 handle_command_reply: tid=0 2024-06-10T01:22:05.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.999+0000 7f0903d6eb80 1 client.28670 shutdown 2024-06-10T01:22:05.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.999+0000 7f0903d6eb80 1 -- 172.21.15.7:0/478984552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945ae7cf20 msgr2=0x55945ae8ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:05.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.999+0000 7f0903d6eb80 1 --2- 172.21.15.7:0/478984552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945ae7cf20 0x55945ae8ee60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f08e40098e0 tx=0x7f08e4003e90 comp rx=0 tx=0).stop 2024-06-10T01:22:05.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.999+0000 7f0903d6eb80 1 -- 172.21.15.7:0/478984552 shutdown_connections 2024-06-10T01:22:05.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.999+0000 7f0903d6eb80 1 --2- 172.21.15.7:0/478984552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55945ae82f60 0x55945ae9d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:05.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.999+0000 7f0903d6eb80 1 --2- 172.21.15.7:0/478984552 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55945ae96760 0x55945af031f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:05.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.999+0000 7f0903d6eb80 1 --2- 172.21.15.7:0/478984552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945ae7cf20 0x55945ae8ee60 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:05.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.999+0000 7f08d6ff5700 0 client.28670 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:22:05.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.999+0000 7f0903d6eb80 1 --2- 172.21.15.7:0/478984552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55945ae82640 0x55945ae8f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:05.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.999+0000 7f0903d6eb80 1 -- 172.21.15.7:0/478984552 >> 172.21.15.7:0/478984552 conn(0x55945ada7cc0 msgr2=0x55945ae70e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:05.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.999+0000 7f0903d6eb80 1 -- 172.21.15.7:0/478984552 shutdown_connections 2024-06-10T01:22:05.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.999+0000 7f0903d6eb80 1 -- 172.21.15.7:0/478984552 wait complete. 2024-06-10T01:22:05.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:04.999+0000 7f0903d6eb80 20 client.28670 trim_cache size 0 max 16384 2024-06-10T01:22:05.005 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:22:05.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:05.000+0000 7f08f94eb700 1 -- 172.21.15.7:0/4291886417 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f08dc0a2ca0 msgr2=0x7f08dc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:05.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:05.000+0000 7f08f94eb700 1 --2- 172.21.15.7:0/4291886417 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f08dc0a2ca0 0x7f08dc0a5160 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f08f4078910 tx=0x7f08e8009260 comp rx=0 tx=0).stop 2024-06-10T01:22:05.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:05.000+0000 7f08f94eb700 1 -- 172.21.15.7:0/4291886417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08f4101380 msgr2=0x7f08f4103030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:05.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:05.000+0000 7f08f94eb700 1 --2- 172.21.15.7:0/4291886417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08f4101380 0x7f08f4103030 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f08e4006010 tx=0x7f08e400bab0 comp rx=0 tx=0).stop 2024-06-10T01:22:05.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:05.000+0000 7f08f94eb700 1 -- 172.21.15.7:0/4291886417 shutdown_connections 2024-06-10T01:22:05.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:05.000+0000 7f08f94eb700 1 --2- 172.21.15.7:0/4291886417 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f08dc0a2ca0 0x7f08dc0a5160 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:05.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:05.000+0000 7f08f94eb700 1 --2- 172.21.15.7:0/4291886417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08f41073c0 0x7f08f4103ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:05.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:05.000+0000 7f08f94eb700 1 --2- 172.21.15.7:0/4291886417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08f4101380 0x7f08f4103030 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:05.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:05.000+0000 7f08f94eb700 1 --2- 172.21.15.7:0/4291886417 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08f4101d40 0x7f08f4103570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:05.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:05.000+0000 7f08f94eb700 1 -- 172.21.15.7:0/4291886417 >> 172.21.15.7:0/4291886417 conn(0x7f08f4078330 msgr2=0x7f08f4100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:05.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:05.000+0000 7f08f94eb700 1 -- 172.21.15.7:0/4291886417 shutdown_connections 2024-06-10T01:22:05.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:05.000+0000 7f08f94eb700 1 -- 172.21.15.7:0/4291886417 wait complete. 2024-06-10T01:22:05.522 INFO:tasks.workunit.client.0.smithi007.stdout: 10 66624 33.43 MB/sec execute 42 sec latency 211.883 ms 2024-06-10T01:22:05.690 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:05 smithi007 ceph-mon[32398]: pgmap v914: 129 pgs: 129 active+clean; 510 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 10 MiB/s wr, 750 op/s 2024-06-10T01:22:05.747 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:22:05.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:05 smithi078 ceph-mon[36683]: pgmap v914: 129 pgs: 129 active+clean; 510 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 10 MiB/s wr, 750 op/s 2024-06-10T01:22:05.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:05 smithi175 ceph-mon[36636]: pgmap v914: 129 pgs: 129 active+clean; 510 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 10 MiB/s wr, 750 op/s 2024-06-10T01:22:06.100 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:22:06.524 INFO:tasks.workunit.client.0.smithi007.stdout: 10 67201 33.25 MB/sec execute 43 sec latency 184.032 ms 2024-06-10T01:22:07.522 INFO:tasks.workunit.client.0.smithi007.stdout: 10 67878 33.40 MB/sec execute 44 sec latency 123.156 ms 2024-06-10T01:22:07.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:07 smithi007 ceph-mon[32398]: pgmap v915: 129 pgs: 129 active+clean; 505 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 11 MiB/s wr, 815 op/s 2024-06-10T01:22:07.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:07 smithi078 ceph-mon[36683]: pgmap v915: 129 pgs: 129 active+clean; 505 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 11 MiB/s wr, 815 op/s 2024-06-10T01:22:07.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:07 smithi175 ceph-mon[36636]: pgmap v915: 129 pgs: 129 active+clean; 505 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 11 MiB/s wr, 815 op/s 2024-06-10T01:22:08.522 INFO:tasks.workunit.client.0.smithi007.stdout: 10 68154 32.91 MB/sec execute 45 sec latency 517.672 ms 2024-06-10T01:22:09.522 INFO:tasks.workunit.client.0.smithi007.stdout: 10 68667 32.68 MB/sec execute 46 sec latency 158.871 ms 2024-06-10T01:22:09.697 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:09 smithi007 ceph-mon[32398]: pgmap v916: 129 pgs: 129 active+clean; 509 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 9.7 MiB/s wr, 740 op/s 2024-06-10T01:22:09.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:09 smithi078 ceph-mon[36683]: pgmap v916: 129 pgs: 129 active+clean; 509 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 9.7 MiB/s wr, 740 op/s 2024-06-10T01:22:09.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:09 smithi175 ceph-mon[36636]: pgmap v916: 129 pgs: 129 active+clean; 509 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 9.7 MiB/s wr, 740 op/s 2024-06-10T01:22:10.522 INFO:tasks.workunit.client.0.smithi007.stdout: 10 69561 33.22 MB/sec execute 47 sec latency 119.026 ms 2024-06-10T01:22:11.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.210+0000 7f356b522700 1 -- 172.21.15.7:0/1575016189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f356411bc70 msgr2=0x7f356411e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:11.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.210+0000 7f356b522700 1 --2- 172.21.15.7:0/1575016189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f356411bc70 0x7f356411e060 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f355c00b0a0 tx=0x7f355c00b3b0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.210+0000 7f356b522700 1 -- 172.21.15.7:0/1575016189 shutdown_connections 2024-06-10T01:22:11.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.210+0000 7f356b522700 1 --2- 172.21.15.7:0/1575016189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f356411bc70 0x7f356411e060 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.210+0000 7f356b522700 1 --2- 172.21.15.7:0/1575016189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3564110c70 0x7f3564109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.210+0000 7f356b522700 1 --2- 172.21.15.7:0/1575016189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35641099d0 0x7f3564109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.210+0000 7f356b522700 1 -- 172.21.15.7:0/1575016189 >> 172.21.15.7:0/1575016189 conn(0x7f356406d600 msgr2=0x7f356406da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:11.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.210+0000 7f356b522700 1 -- 172.21.15.7:0/1575016189 shutdown_connections 2024-06-10T01:22:11.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.210+0000 7f356b522700 1 -- 172.21.15.7:0/1575016189 wait complete. 2024-06-10T01:22:11.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.211+0000 7f356b522700 1 Processor -- start 2024-06-10T01:22:11.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.211+0000 7f356b522700 1 -- start start 2024-06-10T01:22:11.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.212+0000 7f356b522700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35641099d0 0x7f35641ad750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:11.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.212+0000 7f356b522700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3564110c70 0x7f35641adc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:11.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.212+0000 7f356b522700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f356411bc70 0x7f35641a7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:11.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.212+0000 7f356b522700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f35641216d0 con 0x7f3564110c70 2024-06-10T01:22:11.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.212+0000 7f356b522700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3564121550 con 0x7f35641099d0 2024-06-10T01:22:11.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.212+0000 7f356b522700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3564121850 con 0x7f356411bc70 2024-06-10T01:22:11.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.212+0000 7f35692be700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35641099d0 0x7f35641ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:11.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.212+0000 7f35692be700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35641099d0 0x7f35641ad750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:54010/0 (socket says 172.21.15.7:54010) 2024-06-10T01:22:11.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.212+0000 7f35692be700 1 -- 172.21.15.7:0/1776910678 learned_addr learned my addr 172.21.15.7:0/1776910678 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:11.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.213+0000 7f3568abd700 1 --2- 172.21.15.7:0/1776910678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3564110c70 0x7f35641adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:11.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.213+0000 7f35692be700 1 -- 172.21.15.7:0/1776910678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f356411bc70 msgr2=0x7f35641a7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:11.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.213+0000 7f35692be700 1 --2- 172.21.15.7:0/1776910678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f356411bc70 0x7f35641a7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.213+0000 7f35692be700 1 -- 172.21.15.7:0/1776910678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3564110c70 msgr2=0x7f35641adc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:11.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.213+0000 7f35692be700 1 --2- 172.21.15.7:0/1776910678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3564110c70 0x7f35641adc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.213+0000 7f35692be700 1 -- 172.21.15.7:0/1776910678 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35641a8070 con 0x7f35641099d0 2024-06-10T01:22:11.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.213+0000 7f35692be700 1 --2- 172.21.15.7:0/1776910678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35641099d0 0x7f35641ad750 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f356000b530 tx=0x7f356000b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:11.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.215+0000 7f355a7fc700 1 -- 172.21.15.7:0/1776910678 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3560015020 con 0x7f35641099d0 2024-06-10T01:22:11.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.215+0000 7f355a7fc700 1 -- 172.21.15.7:0/1776910678 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3560011d10 con 0x7f35641099d0 2024-06-10T01:22:11.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.215+0000 7f355a7fc700 1 -- 172.21.15.7:0/1776910678 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356000f2e0 con 0x7f35641099d0 2024-06-10T01:22:11.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.215+0000 7f356b522700 1 -- 172.21.15.7:0/1776910678 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35641a8360 con 0x7f35641099d0 2024-06-10T01:22:11.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.215+0000 7f356b522700 1 -- 172.21.15.7:0/1776910678 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35641b4620 con 0x7f35641099d0 2024-06-10T01:22:11.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.218+0000 7f355a7fc700 1 -- 172.21.15.7:0/1776910678 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f3560018020 con 0x7f35641099d0 2024-06-10T01:22:11.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.220+0000 7f355a7fc700 1 --2- 172.21.15.7:0/1776910678 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35500a2da0 0x7f35500a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:11.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.221+0000 7f355a7fc700 1 -- 172.21.15.7:0/1776910678 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f35600ceae0 con 0x7f35641099d0 2024-06-10T01:22:11.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.221+0000 7f3568abd700 1 --2- 172.21.15.7:0/1776910678 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35500a2da0 0x7f35500a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:11.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.221+0000 7f3568abd700 1 --2- 172.21.15.7:0/1776910678 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35500a2da0 0x7f35500a5260 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3554005950 tx=0x7f35540058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:11.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.225+0000 7f3575da5b80 1 Processor -- start 2024-06-10T01:22:11.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.225+0000 7f3575da5b80 1 -- start start 2024-06-10T01:22:11.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.226+0000 7f3575da5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e11f61640 0x562e11f61a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:11.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.226+0000 7f3575da5b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e11f61f60 0x562e11f5b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:11.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.226+0000 7f3575da5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562e11f5bf20 0x562e11f5c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:11.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.226+0000 7f35692be700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e11f61640 0x562e11f61a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:11.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.226+0000 7f35692be700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e11f61640 0x562e11f61a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:54020/0 (socket says 172.21.15.7:54020) 2024-06-10T01:22:11.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.226+0000 7f35692be700 1 -- 172.21.15.7:0/3133968032 learned_addr learned my addr 172.21.15.7:0/3133968032 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:11.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.226+0000 7f3575da5b80 1 -- 172.21.15.7:0/3133968032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562e11ead810 con 0x562e11f61f60 2024-06-10T01:22:11.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.226+0000 7f3575da5b80 1 -- 172.21.15.7:0/3133968032 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562e11e55720 con 0x562e11f61640 2024-06-10T01:22:11.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.226+0000 7f3575da5b80 1 -- 172.21.15.7:0/3133968032 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562e11ea97d0 con 0x562e11f5bf20 2024-06-10T01:22:11.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.226+0000 7f3568abd700 1 --2- 172.21.15.7:0/3133968032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e11f61f60 0x562e11f5b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:11.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.226+0000 7f3569abf700 1 --2- 172.21.15.7:0/3133968032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562e11f5bf20 0x562e11f5c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.227+0000 7f35692be700 1 -- 172.21.15.7:0/3133968032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562e11f5bf20 msgr2=0x562e11f5c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.227+0000 7f35692be700 1 --2- 172.21.15.7:0/3133968032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562e11f5bf20 0x562e11f5c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.227+0000 7f35692be700 1 -- 172.21.15.7:0/3133968032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e11f61f60 msgr2=0x562e11f5b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.227+0000 7f35692be700 1 --2- 172.21.15.7:0/3133968032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e11f61f60 0x562e11f5b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.227+0000 7f35692be700 1 -- 172.21.15.7:0/3133968032 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e11e4bf50 con 0x562e11f61640 2024-06-10T01:22:11.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.227+0000 7f35692be700 1 --2- 172.21.15.7:0/3133968032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e11f61640 0x562e11f61a20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f35600973c0 tx=0x7f356009a9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57a3727778d39d89 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:11.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.228+0000 7f354f7fe700 1 -- 172.21.15.7:0/3133968032 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3560009300 con 0x562e11f61640 2024-06-10T01:22:11.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.228+0000 7f354f7fe700 1 -- 172.21.15.7:0/3133968032 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f356009af60 con 0x562e11f61640 2024-06-10T01:22:11.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.228+0000 7f354f7fe700 1 -- 172.21.15.7:0/3133968032 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35600a3740 con 0x562e11f61640 2024-06-10T01:22:11.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.229+0000 7f3575da5b80 1 -- 172.21.15.7:0/3133968032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e11f61640 msgr2=0x562e11f61a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:11.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.229+0000 7f3575da5b80 1 --2- 172.21.15.7:0/3133968032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e11f61640 0x562e11f61a20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f35600973c0 tx=0x7f356009a9b0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.229+0000 7f3575da5b80 1 -- 172.21.15.7:0/3133968032 shutdown_connections 2024-06-10T01:22:11.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.229+0000 7f3575da5b80 1 --2- 172.21.15.7:0/3133968032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562e11f5bf20 0x562e11f5c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.229+0000 7f3575da5b80 1 --2- 172.21.15.7:0/3133968032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e11f61640 0x562e11f61a20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.230+0000 7f3575da5b80 1 --2- 172.21.15.7:0/3133968032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e11f61f60 0x562e11f5b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.230+0000 7f3575da5b80 1 -- 172.21.15.7:0/3133968032 >> 172.21.15.7:0/3133968032 conn(0x562e11e86cc0 msgr2=0x562e11f56850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:11.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.230+0000 7f3575da5b80 1 -- 172.21.15.7:0/3133968032 shutdown_connections 2024-06-10T01:22:11.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.231+0000 7f3575da5b80 1 -- 172.21.15.7:0/3133968032 wait complete. 2024-06-10T01:22:11.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.232+0000 7f3575da5b80 1 -- start start 2024-06-10T01:22:11.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.232+0000 7f3575da5b80 1 Processor -- start 2024-06-10T01:22:11.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.233+0000 7f3575da5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e11f5bf20 0x562e11f67d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:11.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.233+0000 7f3575da5b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e11f61640 0x562e11f65030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:11.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.233+0000 7f35692be700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e11f5bf20 0x562e11f67d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:11.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.233+0000 7f35692be700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e11f5bf20 0x562e11f67d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:54022/0 (socket says 172.21.15.7:54022) 2024-06-10T01:22:11.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.233+0000 7f35692be700 1 -- 172.21.15.7:0/3271587455 learned_addr learned my addr 172.21.15.7:0/3271587455 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:11.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.233+0000 7f3575da5b80 1 --2- 172.21.15.7:0/3271587455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562e11f61f60 0x562e11f65570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:11.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.234+0000 7f3575da5b80 1 -- 172.21.15.7:0/3271587455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562e11f68a70 con 0x562e11f61640 2024-06-10T01:22:11.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.234+0000 7f3575da5b80 1 -- 172.21.15.7:0/3271587455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562e11e4dbf0 con 0x562e11f5bf20 2024-06-10T01:22:11.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.234+0000 7f3575da5b80 1 -- 172.21.15.7:0/3271587455 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562e11f68bf0 con 0x562e11f61f60 2024-06-10T01:22:11.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.234+0000 7f3575da5b80 10 client.0 _ll_register_callbacks cb 0x562e11e47920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:22:11.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.235+0000 7f3568abd700 1 --2- 172.21.15.7:0/3271587455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e11f61640 0x562e11f65030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:11.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.236+0000 7f3568abd700 1 -- 172.21.15.7:0/3271587455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562e11f61f60 msgr2=0x562e11f65570 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:22:11.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.236+0000 7f3568abd700 1 --2- 172.21.15.7:0/3271587455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562e11f61f60 0x562e11f65570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.236+0000 7f3568abd700 1 -- 172.21.15.7:0/3271587455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e11f5bf20 msgr2=0x562e11f67d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:11.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.236+0000 7f3568abd700 1 --2- 172.21.15.7:0/3271587455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e11f5bf20 0x562e11f67d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.236+0000 7f3568abd700 1 -- 172.21.15.7:0/3271587455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e11dfc8d0 con 0x562e11f61640 2024-06-10T01:22:11.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.237+0000 7f3568abd700 1 --2- 172.21.15.7:0/3271587455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e11f61640 0x562e11f65030 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f355400b570 tx=0x7f3554004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81044247f9280f0c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:11.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.237+0000 7f354cff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:22:11.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.237+0000 7f354cff9700 1 -- 172.21.15.7:0/3271587455 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3554011e30 con 0x562e11f61640 2024-06-10T01:22:11.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.237+0000 7f354cff9700 1 -- 172.21.15.7:0/3271587455 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f355401f430 con 0x562e11f61640 2024-06-10T01:22:11.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.238+0000 7f3575da5b80 1 -- 172.21.15.7:0/3271587455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f356009af60 con 0x562e11f61640 2024-06-10T01:22:11.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.238+0000 7f354cff9700 1 -- 172.21.15.7:0/3271587455 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f355401f630 con 0x562e11f61640 2024-06-10T01:22:11.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.238+0000 7f3575da5b80 10 client.19446 fetch_fsmap learned FSMap version 45 2024-06-10T01:22:11.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.238+0000 7f3575da5b80 1 -- 172.21.15.7:0/3271587455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x562e11f73220 con 0x562e11f61640 2024-06-10T01:22:11.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.238+0000 7f354cff9700 1 -- 172.21.15.7:0/3271587455 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f356009af60 con 0x562e11f61640 2024-06-10T01:22:11.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.239+0000 7f354cff9700 1 -- 172.21.15.7:0/3271587455 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 45) ==== 8358+0+0 (secure 0 0 0) 0x7f3554031a40 con 0x562e11f61640 2024-06-10T01:22:11.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.239+0000 7f3575da5b80 10 client.19446 fetch_fsmap finished waiting for FSMap version 45 2024-06-10T01:22:11.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.239+0000 7f3575da5b80 10 client.19446 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:22:11.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.239+0000 7f3575da5b80 20 client.19446 populate_metadata read hostname 'smithi007' 2024-06-10T01:22:11.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.239+0000 7f3575da5b80 1 --2- 172.21.15.7:0/3271587455 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562e11f75860 0x562e11f77c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:11.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.239+0000 7f3575da5b80 4 client.19446 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:22:11.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.239+0000 7f3575da5b80 1 -- 172.21.15.7:0/3271587455 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e11ebad10 con 0x562e11f75860 2024-06-10T01:22:11.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.239+0000 7f35692be700 1 --2- 172.21.15.7:0/3271587455 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562e11f75860 0x562e11f77c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:11.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.240+0000 7f354cff9700 10 client.19446 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:11.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.240+0000 7f35692be700 1 --2- 172.21.15.7:0/3271587455 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562e11f75860 0x562e11f77c50 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc33954eec53f7c9 server_cookie=ee46116ebb305bfc in_seq=0 out_seq=0 2024-06-10T01:22:11.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.241+0000 7f354cff9700 1 -- 172.21.15.7:0/3271587455 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562e11ebad10 con 0x562e11f75860 2024-06-10T01:22:11.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.241+0000 7f354cff9700 10 client.19446 handle_command_reply: tid=0 2024-06-10T01:22:11.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.241+0000 7f3575da5b80 1 client.19446 shutdown 2024-06-10T01:22:11.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.242+0000 7f3575da5b80 1 -- 172.21.15.7:0/3271587455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e11f61640 msgr2=0x562e11f65030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:11.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.242+0000 7f3575da5b80 1 --2- 172.21.15.7:0/3271587455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e11f61640 0x562e11f65030 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f355400b570 tx=0x7f3554004750 comp rx=0 tx=0).stop 2024-06-10T01:22:11.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.242+0000 7f3575da5b80 1 -- 172.21.15.7:0/3271587455 shutdown_connections 2024-06-10T01:22:11.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.242+0000 7f3575da5b80 1 --2- 172.21.15.7:0/3271587455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562e11f61f60 0x562e11f65570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.242+0000 7f3575da5b80 1 --2- 172.21.15.7:0/3271587455 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562e11f75860 0x562e11f77c50 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.242+0000 7f3575da5b80 1 --2- 172.21.15.7:0/3271587455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e11f5bf20 0x562e11f67d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.242+0000 7f354cff9700 0 client.19446 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:11.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.242+0000 7f3575da5b80 1 --2- 172.21.15.7:0/3271587455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e11f61640 0x562e11f65030 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.242+0000 7f3575da5b80 1 -- 172.21.15.7:0/3271587455 >> 172.21.15.7:0/3271587455 conn(0x562e11e86cc0 msgr2=0x562e11f565f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:11.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.242+0000 7f3575da5b80 1 -- 172.21.15.7:0/3271587455 shutdown_connections 2024-06-10T01:22:11.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.243+0000 7f3575da5b80 1 -- 172.21.15.7:0/3271587455 wait complete. 2024-06-10T01:22:11.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.243+0000 7f3575da5b80 20 client.19446 trim_cache size 0 max 16384 2024-06-10T01:22:11.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.266+0000 7f3575da5b80 1 Processor -- start 2024-06-10T01:22:11.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.266+0000 7f3575da5b80 1 -- start start 2024-06-10T01:22:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.267+0000 7f3575da5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e11f5bf20 0x562e11f5b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.267+0000 7f3575da5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562e11f61640 0x562e11f63920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.267+0000 7f3575da5b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e11f61f60 0x562e11f63e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.267+0000 7f3575da5b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f35641bfb70 con 0x562e11f61f60 2024-06-10T01:22:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.267+0000 7f3575da5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f35641bd3a0 con 0x562e11f5bf20 2024-06-10T01:22:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.267+0000 7f3575da5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f35641bb1f0 con 0x562e11f61640 2024-06-10T01:22:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.267+0000 7f3569abf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e11f61f60 0x562e11f63e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.267+0000 7f35692be700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e11f5bf20 0x562e11f5b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.267+0000 7f35692be700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e11f5bf20 0x562e11f5b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:54032/0 (socket says 172.21.15.7:54032) 2024-06-10T01:22:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.267+0000 7f3569abf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e11f61f60 0x562e11f63e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58136/0 (socket says 172.21.15.7:58136) 2024-06-10T01:22:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.267+0000 7f35692be700 1 -- 172.21.15.7:0/1102288594 learned_addr learned my addr 172.21.15.7:0/1102288594 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.267+0000 7f3568abd700 1 --2- 172.21.15.7:0/1102288594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562e11f61640 0x562e11f63920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.268+0000 7f3569abf700 1 -- 172.21.15.7:0/1102288594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562e11f61640 msgr2=0x562e11f63920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.268+0000 7f3569abf700 1 --2- 172.21.15.7:0/1102288594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562e11f61640 0x562e11f63920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.268+0000 7f3569abf700 1 -- 172.21.15.7:0/1102288594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e11f5bf20 msgr2=0x562e11f5b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.268+0000 7f3569abf700 1 --2- 172.21.15.7:0/1102288594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e11f5bf20 0x562e11f5b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.268+0000 7f3569abf700 1 -- 172.21.15.7:0/1102288594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e11f57470 con 0x562e11f61f60 2024-06-10T01:22:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.268+0000 7f35692be700 1 --2- 172.21.15.7:0/1102288594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e11f5bf20 0x562e11f5b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:22:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.268+0000 7f3569abf700 1 --2- 172.21.15.7:0/1102288594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e11f61f60 0x562e11f63e60 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f355c009fd0 tx=0x7f355c00cfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a22cceb9cecbf27 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.268+0000 7f354effd700 1 -- 172.21.15.7:0/1102288594 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c020070 con 0x562e11f61f60 2024-06-10T01:22:11.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.268+0000 7f354effd700 1 -- 172.21.15.7:0/1102288594 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f355c0093d0 con 0x562e11f61f60 2024-06-10T01:22:11.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.269+0000 7f354effd700 1 -- 172.21.15.7:0/1102288594 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c01c440 con 0x562e11f61f60 2024-06-10T01:22:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.269+0000 7f3575da5b80 1 -- 172.21.15.7:0/1102288594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e11f61f60 msgr2=0x562e11f63e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.269+0000 7f3575da5b80 1 --2- 172.21.15.7:0/1102288594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e11f61f60 0x562e11f63e60 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f355c009fd0 tx=0x7f355c00cfc0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.269+0000 7f3575da5b80 1 -- 172.21.15.7:0/1102288594 shutdown_connections 2024-06-10T01:22:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.269+0000 7f3575da5b80 1 --2- 172.21.15.7:0/1102288594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e11f61f60 0x562e11f63e60 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.269+0000 7f3575da5b80 1 --2- 172.21.15.7:0/1102288594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e11f5bf20 0x562e11f5b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.269+0000 7f3575da5b80 1 --2- 172.21.15.7:0/1102288594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562e11f61640 0x562e11f63920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.269+0000 7f3575da5b80 1 -- 172.21.15.7:0/1102288594 >> 172.21.15.7:0/1102288594 conn(0x562e11e86cc0 msgr2=0x562e11f64ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.270+0000 7f3575da5b80 1 -- 172.21.15.7:0/1102288594 shutdown_connections 2024-06-10T01:22:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.270+0000 7f3575da5b80 1 -- 172.21.15.7:0/1102288594 wait complete. 2024-06-10T01:22:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.270+0000 7f3575da5b80 1 -- start start 2024-06-10T01:22:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.270+0000 7f3575da5b80 1 Processor -- start 2024-06-10T01:22:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.271+0000 7f3575da5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e11f5bf20 0x562e11f7ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.271+0000 7f3575da5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562e11f61640 0x562e11f6ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.271+0000 7f3575da5b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e11f61f60 0x562e11f6e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.271+0000 7f35692be700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e11f5bf20 0x562e11f7ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.271+0000 7f35692be700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e11f5bf20 0x562e11f7ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:54048/0 (socket says 172.21.15.7:54048) 2024-06-10T01:22:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.271+0000 7f35692be700 1 -- 172.21.15.7:0/1524862704 learned_addr learned my addr 172.21.15.7:0/1524862704 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.271+0000 7f3569abf700 1 --2- 172.21.15.7:0/1524862704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e11f61f60 0x562e11f6e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.271+0000 7f3568abd700 1 --2- 172.21.15.7:0/1524862704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562e11f61640 0x562e11f6ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.271+0000 7f3575da5b80 1 -- 172.21.15.7:0/1524862704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f35641bb860 con 0x562e11f61f60 2024-06-10T01:22:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.271+0000 7f3575da5b80 1 -- 172.21.15.7:0/1524862704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f35641bccd0 con 0x562e11f5bf20 2024-06-10T01:22:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.271+0000 7f3575da5b80 1 -- 172.21.15.7:0/1524862704 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f35641bbf30 con 0x562e11f61640 2024-06-10T01:22:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.272+0000 7f3575da5b80 10 client.0 _ll_register_callbacks cb 0x562e11e47920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:22:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.272+0000 7f35692be700 1 -- 172.21.15.7:0/1524862704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562e11f61640 msgr2=0x562e11f6ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.272+0000 7f35692be700 1 --2- 172.21.15.7:0/1524862704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562e11f61640 0x562e11f6ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.272+0000 7f35692be700 1 -- 172.21.15.7:0/1524862704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e11f61f60 msgr2=0x562e11f6e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:11.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.272+0000 7f35692be700 1 --2- 172.21.15.7:0/1524862704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e11f61f60 0x562e11f6e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.272+0000 7f35692be700 1 -- 172.21.15.7:0/1524862704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e11f61140 con 0x562e11f5bf20 2024-06-10T01:22:11.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.272+0000 7f35692be700 1 --2- 172.21.15.7:0/1524862704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e11f5bf20 0x562e11f7ddd0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f35600a3d90 tx=0x7f35600cc520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b5d1ea3af99358a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:11.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.272+0000 7f354cff9700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:22:11.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.272+0000 7f354cff9700 1 -- 172.21.15.7:0/1524862704 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356009af60 con 0x562e11f5bf20 2024-06-10T01:22:11.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.272+0000 7f354cff9700 1 -- 172.21.15.7:0/1524862704 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f3560099260 con 0x562e11f5bf20 2024-06-10T01:22:11.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.272+0000 7f3575da5b80 1 -- 172.21.15.7:0/1524862704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f355c0093d0 con 0x562e11f5bf20 2024-06-10T01:22:11.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.272+0000 7f354cff9700 1 -- 172.21.15.7:0/1524862704 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f3560010510 con 0x562e11f5bf20 2024-06-10T01:22:11.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.273+0000 7f3575da5b80 10 client.28688 fetch_fsmap learned FSMap version 45 2024-06-10T01:22:11.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.273+0000 7f3575da5b80 1 -- 172.21.15.7:0/1524862704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x562e11fdf680 con 0x562e11f5bf20 2024-06-10T01:22:11.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.273+0000 7f354cff9700 1 -- 172.21.15.7:0/1524862704 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3560079e50 con 0x562e11f5bf20 2024-06-10T01:22:11.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.273+0000 7f354cff9700 1 -- 172.21.15.7:0/1524862704 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 45) ==== 8358+0+0 (secure 0 0 0) 0x7f35600a37d0 con 0x562e11f5bf20 2024-06-10T01:22:11.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.273+0000 7f3575da5b80 10 client.28688 fetch_fsmap finished waiting for FSMap version 45 2024-06-10T01:22:11.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.273+0000 7f3575da5b80 10 client.28688 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:22:11.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.273+0000 7f3575da5b80 20 client.28688 populate_metadata read hostname 'smithi007' 2024-06-10T01:22:11.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.273+0000 7f3575da5b80 1 --2- 172.21.15.7:0/1524862704 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562e11fdf970 0x562e11fe1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:11.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.273+0000 7f3575da5b80 4 client.28688 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:22:11.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.273+0000 7f3575da5b80 1 -- 172.21.15.7:0/1524862704 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f355c0093d0 con 0x562e11fdf970 2024-06-10T01:22:11.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.274+0000 7f3568abd700 1 --2- 172.21.15.7:0/1524862704 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562e11fdf970 0x562e11fe1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:11.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.274+0000 7f3568abd700 1 --2- 172.21.15.7:0/1524862704 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562e11fdf970 0x562e11fe1d60 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9c97fc9f0a6addd server_cookie=bb153255ece55bf3 in_seq=0 out_seq=0 2024-06-10T01:22:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.274+0000 7f354cff9700 10 client.28688 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.274+0000 7f354cff9700 1 -- 172.21.15.7:0/1524862704 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f355c0093d0 con 0x562e11fdf970 2024-06-10T01:22:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.274+0000 7f354cff9700 10 client.28688 handle_command_reply: tid=0 2024-06-10T01:22:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.274+0000 7f3575da5b80 1 client.28688 shutdown 2024-06-10T01:22:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.275+0000 7f3575da5b80 1 -- 172.21.15.7:0/1524862704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e11f5bf20 msgr2=0x562e11f7ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.275+0000 7f3575da5b80 1 --2- 172.21.15.7:0/1524862704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e11f5bf20 0x562e11f7ddd0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f35600a3d90 tx=0x7f35600cc520 comp rx=0 tx=0).stop 2024-06-10T01:22:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.275+0000 7f3575da5b80 1 -- 172.21.15.7:0/1524862704 shutdown_connections 2024-06-10T01:22:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.275+0000 7f3575da5b80 1 --2- 172.21.15.7:0/1524862704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e11f61f60 0x562e11f6e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.275+0000 7f3575da5b80 1 --2- 172.21.15.7:0/1524862704 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562e11fdf970 0x562e11fe1d60 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.275+0000 7f354cff9700 0 client.28688 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.275+0000 7f3575da5b80 1 --2- 172.21.15.7:0/1524862704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e11f5bf20 0x562e11f7ddd0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.275+0000 7f3575da5b80 1 --2- 172.21.15.7:0/1524862704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562e11f61640 0x562e11f6ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.275+0000 7f3575da5b80 1 -- 172.21.15.7:0/1524862704 >> 172.21.15.7:0/1524862704 conn(0x562e11e86cc0 msgr2=0x562e11f63860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.275+0000 7f3575da5b80 1 -- 172.21.15.7:0/1524862704 shutdown_connections 2024-06-10T01:22:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.275+0000 7f3575da5b80 1 -- 172.21.15.7:0/1524862704 wait complete. 2024-06-10T01:22:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.275+0000 7f3575da5b80 20 client.28688 trim_cache size 0 max 16384 2024-06-10T01:22:11.273 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:22:11.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.276+0000 7f356b522700 1 -- 172.21.15.7:0/1776910678 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35500a2da0 msgr2=0x7f35500a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:11.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.276+0000 7f356b522700 1 --2- 172.21.15.7:0/1776910678 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35500a2da0 0x7f35500a5260 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3554005950 tx=0x7f35540058e0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.276+0000 7f356b522700 1 -- 172.21.15.7:0/1776910678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35641099d0 msgr2=0x7f35641ad750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:11.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.276+0000 7f356b522700 1 --2- 172.21.15.7:0/1776910678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35641099d0 0x7f35641ad750 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f356000b530 tx=0x7f356000b8f0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.276+0000 7f356b522700 1 -- 172.21.15.7:0/1776910678 shutdown_connections 2024-06-10T01:22:11.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.276+0000 7f356b522700 1 --2- 172.21.15.7:0/1776910678 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35500a2da0 0x7f35500a5260 secure :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3554005950 tx=0x7f35540058e0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.276+0000 7f356b522700 1 --2- 172.21.15.7:0/1776910678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f356411bc70 0x7f35641a7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.276+0000 7f356b522700 1 --2- 172.21.15.7:0/1776910678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35641099d0 0x7f35641ad750 secure :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f356000b530 tx=0x7f356000b8f0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.276+0000 7f356b522700 1 --2- 172.21.15.7:0/1776910678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3564110c70 0x7f35641adc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:11.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.276+0000 7f356b522700 1 -- 172.21.15.7:0/1776910678 >> 172.21.15.7:0/1776910678 conn(0x7f356406d600 msgr2=0x7f3564119c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:11.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.277+0000 7f356b522700 1 -- 172.21.15.7:0/1776910678 shutdown_connections 2024-06-10T01:22:11.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:11.277+0000 7f356b522700 1 -- 172.21.15.7:0/1776910678 wait complete. 2024-06-10T01:22:11.522 INFO:tasks.workunit.client.0.smithi007.stdout: 10 70286 33.38 MB/sec execute 48 sec latency 69.689 ms 2024-06-10T01:22:11.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:11 smithi007 ceph-mon[32398]: pgmap v917: 129 pgs: 129 active+clean; 523 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 11 MiB/s wr, 846 op/s 2024-06-10T01:22:11.839 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:11 smithi078 ceph-mon[36683]: pgmap v917: 129 pgs: 129 active+clean; 523 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 11 MiB/s wr, 846 op/s 2024-06-10T01:22:11.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:11 smithi175 ceph-mon[36636]: pgmap v917: 129 pgs: 129 active+clean; 523 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 11 MiB/s wr, 846 op/s 2024-06-10T01:22:12.522 INFO:tasks.workunit.client.0.smithi007.stdout: 10 70921 33.36 MB/sec execute 49 sec latency 72.423 ms 2024-06-10T01:22:13.092 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:22:13.093 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:22:13.522 INFO:tasks.workunit.client.0.smithi007.stdout: 10 71268 33.04 MB/sec execute 50 sec latency 449.714 ms 2024-06-10T01:22:13.561 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:22:13.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:13 smithi078 ceph-mon[36683]: pgmap v918: 129 pgs: 129 active+clean; 524 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.5 MiB/s wr, 695 op/s 2024-06-10T01:22:13.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:13 smithi175 ceph-mon[36636]: pgmap v918: 129 pgs: 129 active+clean; 524 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.5 MiB/s wr, 695 op/s 2024-06-10T01:22:14.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:13 smithi007 ceph-mon[32398]: pgmap v918: 129 pgs: 129 active+clean; 524 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.5 MiB/s wr, 695 op/s 2024-06-10T01:22:14.522 INFO:tasks.workunit.client.0.smithi007.stdout: 10 71856 32.86 MB/sec execute 51 sec latency 199.902 ms 2024-06-10T01:22:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.963+0000 7faad3b48640 1 -- 172.21.15.7:0/918203609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faacc1036b0 msgr2=0x7faacc109f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.963+0000 7faad3b48640 1 --2- 172.21.15.7:0/918203609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faacc1036b0 0x7faacc109f40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faac4009ca0 tx=0x7faac402a8c0 comp rx=0 tx=0).stop 2024-06-10T01:22:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.964+0000 7faad3b48640 1 -- 172.21.15.7:0/918203609 shutdown_connections 2024-06-10T01:22:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.964+0000 7faad3b48640 1 --2- 172.21.15.7:0/918203609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faacc1036b0 0x7faacc109f40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.964+0000 7faad3b48640 1 --2- 172.21.15.7:0/918203609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faacc102cf0 0x7faacc103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.964+0000 7faad3b48640 1 --2- 172.21.15.7:0/918203609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faacc0ffd20 0x7faacc100140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.964+0000 7faad3b48640 1 -- 172.21.15.7:0/918203609 >> 172.21.15.7:0/918203609 conn(0x7faacc0fb650 msgr2=0x7faacc0fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.964+0000 7faad3b48640 1 -- 172.21.15.7:0/918203609 shutdown_connections 2024-06-10T01:22:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.964+0000 7faad3b48640 1 -- 172.21.15.7:0/918203609 wait complete. 2024-06-10T01:22:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.965+0000 7faad3b48640 1 Processor -- start 2024-06-10T01:22:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.965+0000 7faad3b48640 1 -- start start 2024-06-10T01:22:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.965+0000 7faad3b48640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faacc0ffd20 0x7faacc102660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.965+0000 7faad3b48640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faacc102cf0 0x7faacc100ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.965+0000 7faad3b48640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faacc1036b0 0x7faacc1011e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.965+0000 7faad3b48640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faacc10d6f0 con 0x7faacc0ffd20 2024-06-10T01:22:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.965+0000 7faad3b48640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faacc10d570 con 0x7faacc1036b0 2024-06-10T01:22:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.965+0000 7faad3b48640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faacc10d870 con 0x7faacc102cf0 2024-06-10T01:22:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.965+0000 7faad18e4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faacc0ffd20 0x7faacc102660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.965+0000 7faad10e3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faacc102cf0 0x7faacc100ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.965+0000 7faad10e3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faacc102cf0 0x7faacc100ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49136/0 (socket says 172.21.15.7:49136) 2024-06-10T01:22:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.965+0000 7faad18e4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faacc0ffd20 0x7faacc102660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45472/0 (socket says 172.21.15.7:45472) 2024-06-10T01:22:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.965+0000 7faad10e3640 1 -- 172.21.15.7:0/2664195711 learned_addr learned my addr 172.21.15.7:0/2664195711 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.965+0000 7faad20e5640 1 --2- 172.21.15.7:0/2664195711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faacc1036b0 0x7faacc1011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.966+0000 7faad18e4640 1 -- 172.21.15.7:0/2664195711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faacc102cf0 msgr2=0x7faacc100ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.966+0000 7faad18e4640 1 --2- 172.21.15.7:0/2664195711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faacc102cf0 0x7faacc100ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.966+0000 7faad18e4640 1 -- 172.21.15.7:0/2664195711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faacc1036b0 msgr2=0x7faacc1011e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.966+0000 7faad18e4640 1 --2- 172.21.15.7:0/2664195711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faacc1036b0 0x7faacc1011e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.966+0000 7faad18e4640 1 -- 172.21.15.7:0/2664195711 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faacc101a70 con 0x7faacc0ffd20 2024-06-10T01:22:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.966+0000 7faad10e3640 1 --2- 172.21.15.7:0/2664195711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faacc102cf0 0x7faacc100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:22:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.966+0000 7faad18e4640 1 --2- 172.21.15.7:0/2664195711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faacc0ffd20 0x7faacc102660 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7faac800e9c0 tx=0x7faac800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.966+0000 7faac2ffd640 1 -- 172.21.15.7:0/2664195711 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac800ccc0 con 0x7faacc0ffd20 2024-06-10T01:22:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.966+0000 7faac2ffd640 1 -- 172.21.15.7:0/2664195711 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faac80035f0 con 0x7faacc0ffd20 2024-06-10T01:22:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.966+0000 7faad3b48640 1 -- 172.21.15.7:0/2664195711 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faacc1a1e00 con 0x7faacc0ffd20 2024-06-10T01:22:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.966+0000 7faac2ffd640 1 -- 172.21.15.7:0/2664195711 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac800ce60 con 0x7faacc0ffd20 2024-06-10T01:22:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.966+0000 7faad3b48640 1 -- 172.21.15.7:0/2664195711 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faacc1a2210 con 0x7faacc0ffd20 2024-06-10T01:22:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.969+0000 7faac2ffd640 1 -- 172.21.15.7:0/2664195711 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7faac80037a0 con 0x7faacc0ffd20 2024-06-10T01:22:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.969+0000 7faad684c740 1 Processor -- start 2024-06-10T01:22:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.969+0000 7faad684c740 1 -- start start 2024-06-10T01:22:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.970+0000 7faad684c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aea5340010 0x55aea5340430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.970+0000 7faad684c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aea5340a00 0x55aea5345500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.970+0000 7faad684c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aea5345b70 0x55aea5348000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.970+0000 7faad684c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55aea52a34f0 con 0x55aea5345b70 2024-06-10T01:22:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.970+0000 7faad684c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55aea5241870 con 0x55aea5340a00 2024-06-10T01:22:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.970+0000 7faad684c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55aea529f4b0 con 0x55aea5340010 2024-06-10T01:22:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.970+0000 7faad10e3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aea5345b70 0x55aea5348000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.970+0000 7faad18e4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aea5340a00 0x55aea5345500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.970+0000 7faad10e3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aea5345b70 0x55aea5348000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45486/0 (socket says 172.21.15.7:45486) 2024-06-10T01:22:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.970+0000 7faad20e5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aea5340010 0x55aea5340430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.970+0000 7faad10e3640 1 -- 172.21.15.7:0/34317956 learned_addr learned my addr 172.21.15.7:0/34317956 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.970+0000 7faac2ffd640 1 --2- 172.21.15.7:0/2664195711 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7faab80a2b80 0x7faab80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.970+0000 7faad10e3640 1 -- 172.21.15.7:0/34317956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aea5340010 msgr2=0x55aea5340430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.970+0000 7faad10e3640 1 --2- 172.21.15.7:0/34317956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aea5340010 0x55aea5340430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.970+0000 7faad10e3640 1 -- 172.21.15.7:0/34317956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aea5340a00 msgr2=0x55aea5345500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.970+0000 7faad10e3640 1 --2- 172.21.15.7:0/34317956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aea5340a00 0x55aea5345500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.970+0000 7faad20e5640 1 --2- 172.21.15.7:0/2664195711 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7faab80a2b80 0x7faab80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.970+0000 7faad10e3640 1 -- 172.21.15.7:0/34317956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aea525d9b0 con 0x55aea5345b70 2024-06-10T01:22:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.970+0000 7faac2ffd640 1 -- 172.21.15.7:0/2664195711 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7faac80cf8a0 con 0x7faacc0ffd20 2024-06-10T01:22:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.971+0000 7faad10e3640 1 --2- 172.21.15.7:0/34317956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aea5345b70 0x55aea5348000 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7faabc0115f0 tx=0x7faabc011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f88988b239ea5a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.971+0000 7faab7fff640 1 -- 172.21.15.7:0/34317956 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc01e070 con 0x55aea5345b70 2024-06-10T01:22:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.971+0000 7faab7fff640 1 -- 172.21.15.7:0/34317956 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faabc00f660 con 0x55aea5345b70 2024-06-10T01:22:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.971+0000 7faab7fff640 1 -- 172.21.15.7:0/34317956 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc003850 con 0x55aea5345b70 2024-06-10T01:22:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.972+0000 7faad684c740 1 -- 172.21.15.7:0/34317956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aea5345b70 msgr2=0x55aea5348000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.972+0000 7faad20e5640 1 --2- 172.21.15.7:0/2664195711 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7faab80a2b80 0x7faab80a5040 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x55aea5341260 tx=0x7faac402a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.972+0000 7faad684c740 1 --2- 172.21.15.7:0/34317956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aea5345b70 0x55aea5348000 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7faabc0115f0 tx=0x7faabc011ab0 comp rx=0 tx=0).stop 2024-06-10T01:22:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.972+0000 7faad684c740 1 -- 172.21.15.7:0/34317956 shutdown_connections 2024-06-10T01:22:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.972+0000 7faad684c740 1 --2- 172.21.15.7:0/34317956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aea5345b70 0x55aea5348000 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.972+0000 7faad684c740 1 --2- 172.21.15.7:0/34317956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aea5340a00 0x55aea5345500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.972+0000 7faad684c740 1 --2- 172.21.15.7:0/34317956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aea5340010 0x55aea5340430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.972+0000 7faad684c740 1 -- 172.21.15.7:0/34317956 >> 172.21.15.7:0/34317956 conn(0x55aea533b680 msgr2=0x55aea533ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.972+0000 7faad684c740 1 -- 172.21.15.7:0/34317956 shutdown_connections 2024-06-10T01:22:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.972+0000 7faad684c740 1 -- 172.21.15.7:0/34317956 wait complete. 2024-06-10T01:22:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.972+0000 7faad684c740 1 -- start start 2024-06-10T01:22:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.972+0000 7faad684c740 1 Processor -- start 2024-06-10T01:22:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.973+0000 7faad684c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aea5340010 0x55aea534a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.973+0000 7faad684c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aea5340a00 0x55aea534a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.973+0000 7faad684c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aea5345b70 0x55aea53517a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.973+0000 7faad684c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55aea5348b20 con 0x55aea5340010 2024-06-10T01:22:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.973+0000 7faad684c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55aea522cae0 con 0x55aea5340a00 2024-06-10T01:22:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.973+0000 7faad684c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55aea5348ca0 con 0x55aea5345b70 2024-06-10T01:22:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.973+0000 7faad18e4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aea5340010 0x55aea534a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.973+0000 7faad684c740 10 client.0 _ll_register_callbacks cb 0x55aea52659b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:22:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.973+0000 7faad18e4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aea5340010 0x55aea534a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45490/0 (socket says 172.21.15.7:45490) 2024-06-10T01:22:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.973+0000 7faad18e4640 1 -- 172.21.15.7:0/1480140936 learned_addr learned my addr 172.21.15.7:0/1480140936 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.973+0000 7faad20e5640 1 --2- 172.21.15.7:0/1480140936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aea5345b70 0x55aea53517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.973+0000 7faad10e3640 1 --2- 172.21.15.7:0/1480140936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aea5340a00 0x55aea534a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.973+0000 7faad18e4640 1 -- 172.21.15.7:0/1480140936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aea5345b70 msgr2=0x55aea53517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.973+0000 7faad18e4640 1 --2- 172.21.15.7:0/1480140936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aea5345b70 0x55aea53517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.973+0000 7faad18e4640 1 -- 172.21.15.7:0/1480140936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aea5340a00 msgr2=0x55aea534a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.973+0000 7faad18e4640 1 --2- 172.21.15.7:0/1480140936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aea5340a00 0x55aea534a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.973+0000 7faad18e4640 1 -- 172.21.15.7:0/1480140936 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aea4fb7210 con 0x55aea5340010 2024-06-10T01:22:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.973+0000 7faad18e4640 1 --2- 172.21.15.7:0/1480140936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aea5340010 0x55aea534a290 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7faac8009fd0 tx=0x7faac809b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64089e64d38ce567 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.973+0000 7faab57fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:22:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.974+0000 7faab57fa640 1 -- 172.21.15.7:0/1480140936 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac809bd80 con 0x55aea5340010 2024-06-10T01:22:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.974+0000 7faab57fa640 1 -- 172.21.15.7:0/1480140936 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7faac8003b10 con 0x55aea5340010 2024-06-10T01:22:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.974+0000 7faad684c740 1 -- 172.21.15.7:0/1480140936 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faabc00f660 con 0x55aea5340010 2024-06-10T01:22:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.974+0000 7faab57fa640 1 -- 172.21.15.7:0/1480140936 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7faac80a0130 con 0x55aea5340010 2024-06-10T01:22:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.974+0000 7faad684c740 10 client.19464 fetch_fsmap learned FSMap version 45 2024-06-10T01:22:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.974+0000 7faad684c740 1 -- 172.21.15.7:0/1480140936 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55aea5083620 con 0x55aea5340010 2024-06-10T01:22:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.974+0000 7faab57fa640 1 -- 172.21.15.7:0/1480140936 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faabc00f660 con 0x55aea5340010 2024-06-10T01:22:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.974+0000 7faab57fa640 1 -- 172.21.15.7:0/1480140936 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 45) ==== 8358+0+0 (secure 0 0 0) 0x7faac8097bf0 con 0x55aea5340010 2024-06-10T01:22:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.974+0000 7faad684c740 10 client.19464 fetch_fsmap finished waiting for FSMap version 45 2024-06-10T01:22:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.974+0000 7faad684c740 10 client.19464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:22:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.974+0000 7faad684c740 20 client.19464 populate_metadata read hostname 'smithi007' 2024-06-10T01:22:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.974+0000 7faad684c740 1 --2- 172.21.15.7:0/1480140936 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55aea53560f0 0x55aea5358540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.974+0000 7faad684c740 4 client.19464 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:22:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.974+0000 7faad684c740 1 -- 172.21.15.7:0/1480140936 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aea52b6f10 con 0x55aea53560f0 2024-06-10T01:22:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.975+0000 7faad10e3640 1 --2- 172.21.15.7:0/1480140936 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55aea53560f0 0x55aea5358540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.975+0000 7faab57fa640 10 client.19464 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.975+0000 7faad10e3640 1 --2- 172.21.15.7:0/1480140936 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55aea53560f0 0x55aea5358540 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e780d07dd5f03c60 server_cookie=16a7d4c76e65f10f in_seq=0 out_seq=0 2024-06-10T01:22:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.976+0000 7faab57fa640 1 -- 172.21.15.7:0/1480140936 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55aea52b6f10 con 0x55aea53560f0 2024-06-10T01:22:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.976+0000 7faab57fa640 10 client.19464 handle_command_reply: tid=0 2024-06-10T01:22:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.976+0000 7faad684c740 1 client.19464 shutdown 2024-06-10T01:22:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.976+0000 7faad684c740 1 -- 172.21.15.7:0/1480140936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aea5340010 msgr2=0x55aea534a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.976+0000 7faad684c740 1 --2- 172.21.15.7:0/1480140936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aea5340010 0x55aea534a290 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7faac8009fd0 tx=0x7faac809b9b0 comp rx=0 tx=0).stop 2024-06-10T01:22:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.977+0000 7faad684c740 1 -- 172.21.15.7:0/1480140936 shutdown_connections 2024-06-10T01:22:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.977+0000 7faad684c740 1 --2- 172.21.15.7:0/1480140936 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55aea53560f0 0x55aea5358540 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.977+0000 7faad684c740 1 --2- 172.21.15.7:0/1480140936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aea5345b70 0x55aea53517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.977+0000 7faad684c740 1 --2- 172.21.15.7:0/1480140936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aea5340a00 0x55aea534a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.977+0000 7faad684c740 1 --2- 172.21.15.7:0/1480140936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aea5340010 0x55aea534a290 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.977+0000 7faad684c740 1 -- 172.21.15.7:0/1480140936 >> 172.21.15.7:0/1480140936 conn(0x55aea533b680 msgr2=0x55aea533a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.977+0000 7faab57fa640 0 client.19464 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.977+0000 7faad684c740 1 -- 172.21.15.7:0/1480140936 shutdown_connections 2024-06-10T01:22:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.977+0000 7faad684c740 1 -- 172.21.15.7:0/1480140936 wait complete. 2024-06-10T01:22:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.977+0000 7faad684c740 20 client.19464 trim_cache size 0 max 16384 2024-06-10T01:22:14.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.998+0000 7faad684c740 1 Processor -- start 2024-06-10T01:22:14.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.998+0000 7faad684c740 1 -- start start 2024-06-10T01:22:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.998+0000 7faad684c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aea5340010 0x55aea53452c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.998+0000 7faad684c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aea5340a00 0x55aea5351760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.998+0000 7faad684c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aea5345b70 0x55aea533e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.998+0000 7faad684c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faacc1af0e0 con 0x55aea5345b70 2024-06-10T01:22:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.998+0000 7faad684c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faacc1ac950 con 0x55aea5340a00 2024-06-10T01:22:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.998+0000 7faad684c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faacc1aa7a0 con 0x55aea5340010 2024-06-10T01:22:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.998+0000 7faad20e5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aea5345b70 0x55aea533e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.998+0000 7faad20e5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aea5345b70 0x55aea533e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45496/0 (socket says 172.21.15.7:45496) 2024-06-10T01:22:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.998+0000 7faad20e5640 1 -- 172.21.15.7:0/2757980681 learned_addr learned my addr 172.21.15.7:0/2757980681 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.998+0000 7faad10e3640 1 --2- 172.21.15.7:0/2757980681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aea5340a00 0x55aea5351760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.998+0000 7faad18e4640 1 --2- 172.21.15.7:0/2757980681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aea5340010 0x55aea53452c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.999+0000 7faad10e3640 1 -- 172.21.15.7:0/2757980681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aea5340010 msgr2=0x55aea53452c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.999+0000 7faad10e3640 1 --2- 172.21.15.7:0/2757980681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aea5340010 0x55aea53452c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.999+0000 7faad10e3640 1 -- 172.21.15.7:0/2757980681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aea5345b70 msgr2=0x55aea533e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.999+0000 7faad10e3640 1 --2- 172.21.15.7:0/2757980681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aea5345b70 0x55aea533e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.999+0000 7faad10e3640 1 -- 172.21.15.7:0/2757980681 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aea533b4b0 con 0x55aea5340a00 2024-06-10T01:22:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.999+0000 7faad10e3640 1 --2- 172.21.15.7:0/2757980681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aea5340a00 0x55aea5351760 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7faabc009620 tx=0x7faabc011550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=364f2a13356adf25 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.999+0000 7faab77fe640 1 -- 172.21.15.7:0/2757980681 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc028040 con 0x55aea5340a00 2024-06-10T01:22:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.999+0000 7faab77fe640 1 -- 172.21.15.7:0/2757980681 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faabc005500 con 0x55aea5340a00 2024-06-10T01:22:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:14.999+0000 7faab77fe640 1 -- 172.21.15.7:0/2757980681 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc01e040 con 0x55aea5340a00 2024-06-10T01:22:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.000+0000 7faad684c740 1 -- 172.21.15.7:0/2757980681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aea5340a00 msgr2=0x55aea5351760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.000+0000 7faad684c740 1 --2- 172.21.15.7:0/2757980681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aea5340a00 0x55aea5351760 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7faabc009620 tx=0x7faabc011550 comp rx=0 tx=0).stop 2024-06-10T01:22:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.000+0000 7faad684c740 1 -- 172.21.15.7:0/2757980681 shutdown_connections 2024-06-10T01:22:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.000+0000 7faad684c740 1 --2- 172.21.15.7:0/2757980681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aea5345b70 0x55aea533e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.000+0000 7faad684c740 1 --2- 172.21.15.7:0/2757980681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aea5340a00 0x55aea5351760 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.000+0000 7faad684c740 1 --2- 172.21.15.7:0/2757980681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aea5340010 0x55aea53452c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.000+0000 7faad684c740 1 -- 172.21.15.7:0/2757980681 >> 172.21.15.7:0/2757980681 conn(0x55aea533b680 msgr2=0x55aea533ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.000+0000 7faad684c740 1 -- 172.21.15.7:0/2757980681 shutdown_connections 2024-06-10T01:22:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.000+0000 7faad684c740 1 -- 172.21.15.7:0/2757980681 wait complete. 2024-06-10T01:22:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.001+0000 7faad684c740 1 -- start start 2024-06-10T01:22:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.001+0000 7faad684c740 1 Processor -- start 2024-06-10T01:22:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.001+0000 7faad684c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aea5340010 0x55aea534e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.001+0000 7faad684c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aea5340a00 0x55aea534ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.001+0000 7faad684c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aea5345b70 0x55aea53bee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.001+0000 7faad684c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faacc1aae10 con 0x55aea5340a00 2024-06-10T01:22:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.001+0000 7faad684c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faacc1ac280 con 0x55aea5345b70 2024-06-10T01:22:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.001+0000 7faad684c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faacc1ab4e0 con 0x55aea5340010 2024-06-10T01:22:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.001+0000 7faad684c740 10 client.0 _ll_register_callbacks cb 0x7faac809bc40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:22:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.001+0000 7faad18e4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aea5340010 0x55aea534e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.001+0000 7faad18e4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aea5340010 0x55aea534e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49166/0 (socket says 172.21.15.7:49166) 2024-06-10T01:22:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.001+0000 7faad18e4640 1 -- 172.21.15.7:0/3882363090 learned_addr learned my addr 172.21.15.7:0/3882363090 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.002+0000 7faad20e5640 1 --2- 172.21.15.7:0/3882363090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aea5345b70 0x55aea53bee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.002+0000 7faad10e3640 1 --2- 172.21.15.7:0/3882363090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aea5340a00 0x55aea534ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.002+0000 7faad18e4640 1 -- 172.21.15.7:0/3882363090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aea5345b70 msgr2=0x55aea53bee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.002+0000 7faad18e4640 1 --2- 172.21.15.7:0/3882363090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aea5345b70 0x55aea53bee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.002+0000 7faad18e4640 1 -- 172.21.15.7:0/3882363090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aea5340a00 msgr2=0x55aea534ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.002+0000 7faad18e4640 1 --2- 172.21.15.7:0/3882363090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aea5340a00 0x55aea534ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.002+0000 7faad18e4640 1 -- 172.21.15.7:0/3882363090 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aea534f0a0 con 0x55aea5340010 2024-06-10T01:22:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.002+0000 7faad10e3640 1 --2- 172.21.15.7:0/3882363090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aea5340a00 0x55aea534ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:22:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.002+0000 7faad18e4640 1 --2- 172.21.15.7:0/3882363090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aea5340010 0x55aea534e4f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7faac809ac40 tx=0x7faac8003d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd0f35a96ede0e9d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.002+0000 7faab57fa640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:22:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.002+0000 7faab57fa640 1 -- 172.21.15.7:0/3882363090 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac809b3d0 con 0x55aea5340010 2024-06-10T01:22:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.002+0000 7faab57fa640 1 -- 172.21.15.7:0/3882363090 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7faac8004000 con 0x55aea5340010 2024-06-10T01:22:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.002+0000 7faad684c740 1 -- 172.21.15.7:0/3882363090 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faabc005500 con 0x55aea5340010 2024-06-10T01:22:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.003+0000 7faab57fa640 1 -- 172.21.15.7:0/3882363090 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7faac809bd80 con 0x55aea5340010 2024-06-10T01:22:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.003+0000 7faad684c740 10 client.28708 fetch_fsmap learned FSMap version 45 2024-06-10T01:22:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.003+0000 7faad684c740 1 -- 172.21.15.7:0/3882363090 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55aea53c04d0 con 0x55aea5340010 2024-06-10T01:22:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.003+0000 7faab57fa640 1 -- 172.21.15.7:0/3882363090 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faabc005500 con 0x55aea5340010 2024-06-10T01:22:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.003+0000 7faab57fa640 1 -- 172.21.15.7:0/3882363090 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 45) ==== 8358+0+0 (secure 0 0 0) 0x7faac8086380 con 0x55aea5340010 2024-06-10T01:22:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.004+0000 7faad684c740 10 client.28708 fetch_fsmap finished waiting for FSMap version 45 2024-06-10T01:22:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.004+0000 7faad684c740 10 client.28708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:22:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.004+0000 7faad684c740 20 client.28708 populate_metadata read hostname 'smithi007' 2024-06-10T01:22:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.004+0000 7faad684c740 1 --2- 172.21.15.7:0/3882363090 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55aea53c0e60 0x55aea53c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.004+0000 7faad684c740 4 client.28708 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:22:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.004+0000 7faad684c740 1 -- 172.21.15.7:0/3882363090 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aea52b0c20 con 0x55aea53c0e60 2024-06-10T01:22:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.004+0000 7faad10e3640 1 --2- 172.21.15.7:0/3882363090 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55aea53c0e60 0x55aea53c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.005+0000 7faad10e3640 1 --2- 172.21.15.7:0/3882363090 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55aea53c0e60 0x55aea53c3280 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e0d8ec11ff8a8ea server_cookie=a0aefb4ffd31057d in_seq=0 out_seq=0 2024-06-10T01:22:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.005+0000 7faab57fa640 10 client.28708 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.496+0000 7faab57fa640 1 -- 172.21.15.7:0/3882363090 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55aea52b0c20 con 0x55aea53c0e60 2024-06-10T01:22:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.496+0000 7faab57fa640 10 client.28708 handle_command_reply: tid=0 2024-06-10T01:22:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.497+0000 7faad684c740 1 client.28708 shutdown 2024-06-10T01:22:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.497+0000 7faad684c740 1 -- 172.21.15.7:0/3882363090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aea5340010 msgr2=0x55aea534e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.497+0000 7faad684c740 1 --2- 172.21.15.7:0/3882363090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aea5340010 0x55aea534e4f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7faac809ac40 tx=0x7faac8003d40 comp rx=0 tx=0).stop 2024-06-10T01:22:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.497+0000 7faad684c740 1 -- 172.21.15.7:0/3882363090 shutdown_connections 2024-06-10T01:22:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.497+0000 7faad684c740 1 --2- 172.21.15.7:0/3882363090 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55aea53c0e60 0x55aea53c3280 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.497+0000 7faad684c740 1 --2- 172.21.15.7:0/3882363090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aea5345b70 0x55aea53bee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.497+0000 7faab57fa640 0 client.28708 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.498+0000 7faad684c740 1 --2- 172.21.15.7:0/3882363090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aea5340a00 0x55aea534ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.498+0000 7faad684c740 1 --2- 172.21.15.7:0/3882363090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aea5340010 0x55aea534e4f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.498+0000 7faad684c740 1 -- 172.21.15.7:0/3882363090 >> 172.21.15.7:0/3882363090 conn(0x55aea533b680 msgr2=0x55aea5345280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.498+0000 7faad684c740 1 -- 172.21.15.7:0/3882363090 shutdown_connections 2024-06-10T01:22:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.498+0000 7faad684c740 1 -- 172.21.15.7:0/3882363090 wait complete. 2024-06-10T01:22:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.498+0000 7faad684c740 20 client.28708 trim_cache size 0 max 16384 2024-06-10T01:22:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:22:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:22:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:22:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.499+0000 7faad3b48640 1 -- 172.21.15.7:0/2664195711 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7faab80a2b80 msgr2=0x7faab80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.499+0000 7faad3b48640 1 --2- 172.21.15.7:0/2664195711 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7faab80a2b80 0x7faab80a5040 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x55aea5341260 tx=0x7faac402a8c0 comp rx=0 tx=0).stop 2024-06-10T01:22:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.499+0000 7faad3b48640 1 -- 172.21.15.7:0/2664195711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faacc0ffd20 msgr2=0x7faacc102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.499+0000 7faad3b48640 1 --2- 172.21.15.7:0/2664195711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faacc0ffd20 0x7faacc102660 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7faac800e9c0 tx=0x7faac800ee80 comp rx=0 tx=0).stop 2024-06-10T01:22:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.499+0000 7faad3b48640 1 -- 172.21.15.7:0/2664195711 shutdown_connections 2024-06-10T01:22:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.499+0000 7faad3b48640 1 --2- 172.21.15.7:0/2664195711 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7faab80a2b80 0x7faab80a5040 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.499+0000 7faad3b48640 1 --2- 172.21.15.7:0/2664195711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faacc1036b0 0x7faacc1011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.499+0000 7faad3b48640 1 --2- 172.21.15.7:0/2664195711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faacc102cf0 0x7faacc100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.499+0000 7faad3b48640 1 --2- 172.21.15.7:0/2664195711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faacc0ffd20 0x7faacc102660 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.499+0000 7faad3b48640 1 -- 172.21.15.7:0/2664195711 >> 172.21.15.7:0/2664195711 conn(0x7faacc0fb650 msgr2=0x7faacc105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.499+0000 7faad3b48640 1 -- 172.21.15.7:0/2664195711 shutdown_connections 2024-06-10T01:22:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:15.499+0000 7faad3b48640 1 -- 172.21.15.7:0/2664195711 wait complete. 2024-06-10T01:22:15.522 INFO:tasks.workunit.client.0.smithi007.stdout: 10 72414 32.82 MB/sec execute 52 sec latency 337.240 ms 2024-06-10T01:22:15.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:15 smithi007 ceph-mon[32398]: pgmap v919: 129 pgs: 129 active+clean; 526 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.7 MiB/s wr, 764 op/s 2024-06-10T01:22:15.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:15 smithi078 ceph-mon[36683]: pgmap v919: 129 pgs: 129 active+clean; 526 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.7 MiB/s wr, 764 op/s 2024-06-10T01:22:15.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:15 smithi175 ceph-mon[36636]: pgmap v919: 129 pgs: 129 active+clean; 526 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.7 MiB/s wr, 764 op/s 2024-06-10T01:22:16.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.120+0000 7fb359122700 1 -- 172.21.15.7:0/1671390288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb354101d30 msgr2=0x7fb35410e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:16.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.120+0000 7fb359122700 1 --2- 172.21.15.7:0/1671390288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb354101d30 0x7fb35410e210 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00f4c0 tx=0x7fb34c00f7d0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.121+0000 7fb359122700 1 -- 172.21.15.7:0/1671390288 shutdown_connections 2024-06-10T01:22:16.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.121+0000 7fb359122700 1 --2- 172.21.15.7:0/1671390288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb354101d30 0x7fb35410e210 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.121+0000 7fb359122700 1 --2- 172.21.15.7:0/1671390288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3541073d0 0x7fb3541077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.121+0000 7fb359122700 1 --2- 172.21.15.7:0/1671390288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb354101390 0x7fb3541017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.121+0000 7fb359122700 1 -- 172.21.15.7:0/1671390288 >> 172.21.15.7:0/1671390288 conn(0x7fb3540782c0 msgr2=0x7fb3540786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:16.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.121+0000 7fb359122700 1 -- 172.21.15.7:0/1671390288 shutdown_connections 2024-06-10T01:22:16.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.121+0000 7fb359122700 1 -- 172.21.15.7:0/1671390288 wait complete. 2024-06-10T01:22:16.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.122+0000 7fb359122700 1 Processor -- start 2024-06-10T01:22:16.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.123+0000 7fb359122700 1 -- start start 2024-06-10T01:22:16.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.123+0000 7fb359122700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb354101390 0x7fb354116e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:16.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.123+0000 7fb359122700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb354101d30 0x7fb354117350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:16.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.123+0000 7fb359122700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3541073d0 0x7fb35410fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:16.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.124+0000 7fb35259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb354101d30 0x7fb354117350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:16.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.124+0000 7fb35259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb354101d30 0x7fb354117350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:56770/0 (socket says 172.21.15.7:56770) 2024-06-10T01:22:16.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.124+0000 7fb35259c700 1 -- 172.21.15.7:0/3364416963 learned_addr learned my addr 172.21.15.7:0/3364416963 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:16.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.124+0000 7fb35359e700 1 --2- 172.21.15.7:0/3364416963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3541073d0 0x7fb35410fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:16.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.124+0000 7fb352d9d700 1 --2- 172.21.15.7:0/3364416963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb354101390 0x7fb354116e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:16.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.124+0000 7fb359122700 1 -- 172.21.15.7:0/3364416963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb354118e00 con 0x7fb3541073d0 2024-06-10T01:22:16.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.124+0000 7fb359122700 1 -- 172.21.15.7:0/3364416963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb354118c80 con 0x7fb354101d30 2024-06-10T01:22:16.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.124+0000 7fb359122700 1 -- 172.21.15.7:0/3364416963 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb354118f80 con 0x7fb354101390 2024-06-10T01:22:16.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.124+0000 7fb35359e700 1 -- 172.21.15.7:0/3364416963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb354101390 msgr2=0x7fb354116e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:16.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.124+0000 7fb35359e700 1 --2- 172.21.15.7:0/3364416963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb354101390 0x7fb354116e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.124+0000 7fb35359e700 1 -- 172.21.15.7:0/3364416963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb354101d30 msgr2=0x7fb354117350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:16.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.124+0000 7fb35359e700 1 --2- 172.21.15.7:0/3364416963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb354101d30 0x7fb354117350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.124+0000 7fb35359e700 1 -- 172.21.15.7:0/3364416963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3541107f0 con 0x7fb3541073d0 2024-06-10T01:22:16.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.125+0000 7fb35359e700 1 --2- 172.21.15.7:0/3364416963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3541073d0 0x7fb35410fef0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00fc50 tx=0x7fb34c00d760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:16.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.125+0000 7fb33bfff700 1 -- 172.21.15.7:0/3364416963 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c00b510 con 0x7fb3541073d0 2024-06-10T01:22:16.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.125+0000 7fb33bfff700 1 -- 172.21.15.7:0/3364416963 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb34c02c070 con 0x7fb3541073d0 2024-06-10T01:22:16.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.125+0000 7fb33bfff700 1 -- 172.21.15.7:0/3364416963 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c005510 con 0x7fb3541073d0 2024-06-10T01:22:16.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.126+0000 7fb359122700 1 -- 172.21.15.7:0/3364416963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb354110a80 con 0x7fb3541073d0 2024-06-10T01:22:16.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.126+0000 7fb359122700 1 -- 172.21.15.7:0/3364416963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb354114940 con 0x7fb3541073d0 2024-06-10T01:22:16.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.130+0000 7fb33bfff700 1 -- 172.21.15.7:0/3364416963 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb34c037060 con 0x7fb3541073d0 2024-06-10T01:22:16.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.132+0000 7fb33bfff700 1 --2- 172.21.15.7:0/3364416963 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb33c0a2da0 0x7fb33c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:16.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.132+0000 7fb33bfff700 1 -- 172.21.15.7:0/3364416963 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb34c0e07a0 con 0x7fb3541073d0 2024-06-10T01:22:16.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.132+0000 7fb352d9d700 1 --2- 172.21.15.7:0/3364416963 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb33c0a2da0 0x7fb33c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:16.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.134+0000 7fb352d9d700 1 --2- 172.21.15.7:0/3364416963 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb33c0a2da0 0x7fb33c0a5260 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb344005fd0 tx=0x7fb34400af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:16.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.137+0000 7fb3639a5b80 1 Processor -- start 2024-06-10T01:22:16.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.137+0000 7fb3639a5b80 1 -- start start 2024-06-10T01:22:16.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.137+0000 7fb3639a5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b335c906c0 0x55b335c90aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:16.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.137+0000 7fb3639a5b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b335c90fe0 0x55b335c8aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:16.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.137+0000 7fb3639a5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b335c8afa0 0x55b335c8b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:16.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.137+0000 7fb35259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b335c90fe0 0x55b335c8aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:16.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.137+0000 7fb35259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b335c90fe0 0x55b335c8aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45552/0 (socket says 172.21.15.7:45552) 2024-06-10T01:22:16.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.137+0000 7fb35259c700 1 -- 172.21.15.7:0/4000057675 learned_addr learned my addr 172.21.15.7:0/4000057675 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:16.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.137+0000 7fb352d9d700 1 --2- 172.21.15.7:0/4000057675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b335c906c0 0x55b335c90aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:16.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.138+0000 7fb3639a5b80 1 -- 172.21.15.7:0/4000057675 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b335bdc890 con 0x55b335c90fe0 2024-06-10T01:22:16.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.138+0000 7fb3639a5b80 1 -- 172.21.15.7:0/4000057675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b335b847a0 con 0x55b335c906c0 2024-06-10T01:22:16.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.138+0000 7fb3639a5b80 1 -- 172.21.15.7:0/4000057675 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b335bd8850 con 0x55b335c8afa0 2024-06-10T01:22:16.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.138+0000 7fb35359e700 1 --2- 172.21.15.7:0/4000057675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b335c8afa0 0x55b335c8b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:16.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.138+0000 7fb35259c700 1 -- 172.21.15.7:0/4000057675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b335c8afa0 msgr2=0x55b335c8b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:16.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.138+0000 7fb35259c700 1 --2- 172.21.15.7:0/4000057675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b335c8afa0 0x55b335c8b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.138+0000 7fb35259c700 1 -- 172.21.15.7:0/4000057675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b335c906c0 msgr2=0x55b335c90aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:16.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.138+0000 7fb35259c700 1 --2- 172.21.15.7:0/4000057675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b335c906c0 0x55b335c90aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.138+0000 7fb35259c700 1 -- 172.21.15.7:0/4000057675 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b335b7afd0 con 0x55b335c90fe0 2024-06-10T01:22:16.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.141+0000 7fb35259c700 1 --2- 172.21.15.7:0/4000057675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b335c90fe0 0x55b335c8aa60 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fb348003b10 tx=0x7fb348003e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24455359f5e59b9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:16.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.141+0000 7fb3397fa700 1 -- 172.21.15.7:0/4000057675 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb348015070 con 0x55b335c90fe0 2024-06-10T01:22:16.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.141+0000 7fb3397fa700 1 -- 172.21.15.7:0/4000057675 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb348011440 con 0x55b335c90fe0 2024-06-10T01:22:16.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.142+0000 7fb3397fa700 1 -- 172.21.15.7:0/4000057675 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb348019660 con 0x55b335c90fe0 2024-06-10T01:22:16.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.142+0000 7fb3639a5b80 1 -- 172.21.15.7:0/4000057675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b335c90fe0 msgr2=0x55b335c8aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:16.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.142+0000 7fb3639a5b80 1 --2- 172.21.15.7:0/4000057675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b335c90fe0 0x55b335c8aa60 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fb348003b10 tx=0x7fb348003e20 comp rx=0 tx=0).stop 2024-06-10T01:22:16.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.143+0000 7fb3639a5b80 1 -- 172.21.15.7:0/4000057675 shutdown_connections 2024-06-10T01:22:16.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.143+0000 7fb3639a5b80 1 --2- 172.21.15.7:0/4000057675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b335c8afa0 0x55b335c8b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.143+0000 7fb3639a5b80 1 --2- 172.21.15.7:0/4000057675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b335c906c0 0x55b335c90aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.143+0000 7fb3639a5b80 1 --2- 172.21.15.7:0/4000057675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b335c90fe0 0x55b335c8aa60 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.143+0000 7fb3639a5b80 1 -- 172.21.15.7:0/4000057675 >> 172.21.15.7:0/4000057675 conn(0x55b335bb5d40 msgr2=0x55b335c858d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:16.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.143+0000 7fb3639a5b80 1 -- 172.21.15.7:0/4000057675 shutdown_connections 2024-06-10T01:22:16.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.144+0000 7fb3639a5b80 1 -- 172.21.15.7:0/4000057675 wait complete. 2024-06-10T01:22:16.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.144+0000 7fb3639a5b80 1 -- start start 2024-06-10T01:22:16.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.144+0000 7fb3639a5b80 1 Processor -- start 2024-06-10T01:22:16.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.145+0000 7fb3639a5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b335c8afa0 0x55b335c96d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:16.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.145+0000 7fb3639a5b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b335c906c0 0x55b335c94020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:16.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.145+0000 7fb3639a5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b335c90fe0 0x55b335c94560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:16.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.146+0000 7fb3639a5b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b335c97aa0 con 0x55b335c906c0 2024-06-10T01:22:16.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.146+0000 7fb352d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b335c8afa0 0x55b335c96d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:16.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.146+0000 7fb352d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b335c8afa0 0x55b335c96d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49206/0 (socket says 172.21.15.7:49206) 2024-06-10T01:22:16.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.146+0000 7fb352d9d700 1 -- 172.21.15.7:0/107963759 learned_addr learned my addr 172.21.15.7:0/107963759 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:16.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.146+0000 7fb3639a5b80 1 -- 172.21.15.7:0/107963759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b335b7cc70 con 0x55b335c90fe0 2024-06-10T01:22:16.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.146+0000 7fb35359e700 1 --2- 172.21.15.7:0/107963759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b335c90fe0 0x55b335c94560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:16.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.146+0000 7fb35259c700 1 --2- 172.21.15.7:0/107963759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b335c906c0 0x55b335c94020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:16.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.146+0000 7fb3639a5b80 1 -- 172.21.15.7:0/107963759 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b335c97c20 con 0x55b335c8afa0 2024-06-10T01:22:16.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.146+0000 7fb3639a5b80 10 client.0 _ll_register_callbacks cb 0x55b335b769a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:22:16.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.147+0000 7fb35259c700 1 -- 172.21.15.7:0/107963759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b335c8afa0 msgr2=0x55b335c96d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:16.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.147+0000 7fb35259c700 1 --2- 172.21.15.7:0/107963759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b335c8afa0 0x55b335c96d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.147+0000 7fb35259c700 1 -- 172.21.15.7:0/107963759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b335c90fe0 msgr2=0x55b335c94560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:16.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.147+0000 7fb35259c700 1 --2- 172.21.15.7:0/107963759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b335c90fe0 0x55b335c94560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.147+0000 7fb35259c700 1 -- 172.21.15.7:0/107963759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b335b2b950 con 0x55b335c906c0 2024-06-10T01:22:16.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.147+0000 7fb35259c700 1 --2- 172.21.15.7:0/107963759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b335c906c0 0x55b335c94020 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fb348018000 tx=0x7fb348011800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dec098aef1fd41 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:16.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.148+0000 7fb336ff5700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:22:16.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.148+0000 7fb336ff5700 1 -- 172.21.15.7:0/107963759 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb348028060 con 0x55b335c906c0 2024-06-10T01:22:16.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.148+0000 7fb3639a5b80 1 -- 172.21.15.7:0/107963759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb348011440 con 0x55b335c906c0 2024-06-10T01:22:16.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.148+0000 7fb336ff5700 1 -- 172.21.15.7:0/107963759 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb34800fa50 con 0x55b335c906c0 2024-06-10T01:22:16.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.148+0000 7fb336ff5700 1 -- 172.21.15.7:0/107963759 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fb34802f050 con 0x55b335c906c0 2024-06-10T01:22:16.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.148+0000 7fb3639a5b80 10 client.19482 fetch_fsmap learned FSMap version 45 2024-06-10T01:22:16.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.149+0000 7fb3639a5b80 1 -- 172.21.15.7:0/107963759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55b335ca21a0 con 0x55b335c906c0 2024-06-10T01:22:16.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.150+0000 7fb336ff5700 1 -- 172.21.15.7:0/107963759 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb348011440 con 0x55b335c906c0 2024-06-10T01:22:16.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.150+0000 7fb336ff5700 1 -- 172.21.15.7:0/107963759 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 45) ==== 8358+0+0 (secure 0 0 0) 0x7fb348019400 con 0x55b335c906c0 2024-06-10T01:22:16.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.150+0000 7fb3639a5b80 10 client.19482 fetch_fsmap finished waiting for FSMap version 45 2024-06-10T01:22:16.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.150+0000 7fb3639a5b80 10 client.19482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:22:16.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.151+0000 7fb3639a5b80 20 client.19482 populate_metadata read hostname 'smithi007' 2024-06-10T01:22:16.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.151+0000 7fb3639a5b80 1 --2- 172.21.15.7:0/107963759 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b335ca4670 0x55b335ca6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:16.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.151+0000 7fb3639a5b80 4 client.19482 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:22:16.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.151+0000 7fb3639a5b80 1 -- 172.21.15.7:0/107963759 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b335be9d90 con 0x55b335ca4670 2024-06-10T01:22:16.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.151+0000 7fb352d9d700 1 --2- 172.21.15.7:0/107963759 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b335ca4670 0x55b335ca6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:16.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.152+0000 7fb352d9d700 1 --2- 172.21.15.7:0/107963759 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b335ca4670 0x55b335ca6a60 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=735601d359cd8030 server_cookie=7749736a914c9ee3 in_seq=0 out_seq=0 2024-06-10T01:22:16.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.152+0000 7fb336ff5700 10 client.19482 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:16.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.155+0000 7fb336ff5700 1 -- 172.21.15.7:0/107963759 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b335be9d90 con 0x55b335ca4670 2024-06-10T01:22:16.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.155+0000 7fb336ff5700 10 client.19482 handle_command_reply: tid=0 2024-06-10T01:22:16.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.155+0000 7fb3639a5b80 1 client.19482 shutdown 2024-06-10T01:22:16.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.155+0000 7fb3639a5b80 1 -- 172.21.15.7:0/107963759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b335c906c0 msgr2=0x55b335c94020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:16.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.155+0000 7fb3639a5b80 1 --2- 172.21.15.7:0/107963759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b335c906c0 0x55b335c94020 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fb348018000 tx=0x7fb348011800 comp rx=0 tx=0).stop 2024-06-10T01:22:16.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.157+0000 7fb3639a5b80 1 -- 172.21.15.7:0/107963759 shutdown_connections 2024-06-10T01:22:16.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.157+0000 7fb3639a5b80 1 --2- 172.21.15.7:0/107963759 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b335ca4670 0x55b335ca6a60 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.157+0000 7fb3639a5b80 1 --2- 172.21.15.7:0/107963759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b335c90fe0 0x55b335c94560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.157+0000 7fb3639a5b80 1 --2- 172.21.15.7:0/107963759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b335c8afa0 0x55b335c96d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.157+0000 7fb3639a5b80 1 --2- 172.21.15.7:0/107963759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b335c906c0 0x55b335c94020 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.157+0000 7fb3639a5b80 1 -- 172.21.15.7:0/107963759 >> 172.21.15.7:0/107963759 conn(0x55b335bb5d40 msgr2=0x55b335c89a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:16.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.158+0000 7fb336ff5700 0 client.19482 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:16.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.159+0000 7fb3639a5b80 1 -- 172.21.15.7:0/107963759 shutdown_connections 2024-06-10T01:22:16.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.159+0000 7fb3639a5b80 1 -- 172.21.15.7:0/107963759 wait complete. 2024-06-10T01:22:16.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.159+0000 7fb3639a5b80 20 client.19482 trim_cache size 0 max 16384 2024-06-10T01:22:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.205+0000 7fb3639a5b80 1 Processor -- start 2024-06-10T01:22:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.206+0000 7fb3639a5b80 1 -- start start 2024-06-10T01:22:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.206+0000 7fb3639a5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b335c8afa0 0x55b335c8aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.206+0000 7fb3639a5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b335c906c0 0x55b335c7f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.206+0000 7fb3639a5b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b335c90fe0 0x55b335c91af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.206+0000 7fb3639a5b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb3541b6e10 con 0x55b335c90fe0 2024-06-10T01:22:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.206+0000 7fb3639a5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb354073a50 con 0x55b335c8afa0 2024-06-10T01:22:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.206+0000 7fb3639a5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb3540718a0 con 0x55b335c906c0 2024-06-10T01:22:16.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.207+0000 7fb35359e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b335c90fe0 0x55b335c91af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:16.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.207+0000 7fb35259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b335c906c0 0x55b335c7f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:16.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.207+0000 7fb35259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b335c906c0 0x55b335c7f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49222/0 (socket says 172.21.15.7:49222) 2024-06-10T01:22:16.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.207+0000 7fb35259c700 1 -- 172.21.15.7:0/271775467 learned_addr learned my addr 172.21.15.7:0/271775467 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:16.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.207+0000 7fb35259c700 1 -- 172.21.15.7:0/271775467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b335c8afa0 msgr2=0x55b335c8aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:16.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.207+0000 7fb35259c700 1 --2- 172.21.15.7:0/271775467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b335c8afa0 0x55b335c8aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.207+0000 7fb35259c700 1 -- 172.21.15.7:0/271775467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b335c90fe0 msgr2=0x55b335c91af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:16.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.207+0000 7fb35259c700 1 --2- 172.21.15.7:0/271775467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b335c90fe0 0x55b335c91af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.207+0000 7fb35259c700 1 -- 172.21.15.7:0/271775467 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b335c85e10 con 0x55b335c906c0 2024-06-10T01:22:16.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.208+0000 7fb35259c700 1 --2- 172.21.15.7:0/271775467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b335c906c0 0x55b335c7f710 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb3480057c0 tx=0x7fb348005910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9aa1a329e93a124c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:16.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.210+0000 7fb338ff9700 1 -- 172.21.15.7:0/271775467 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb348028070 con 0x55b335c906c0 2024-06-10T01:22:16.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.210+0000 7fb338ff9700 1 -- 172.21.15.7:0/271775467 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb348011d60 con 0x55b335c906c0 2024-06-10T01:22:16.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.210+0000 7fb338ff9700 1 -- 172.21.15.7:0/271775467 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb348015070 con 0x55b335c906c0 2024-06-10T01:22:16.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.212+0000 7fb3639a5b80 1 -- 172.21.15.7:0/271775467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b335c906c0 msgr2=0x55b335c7f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:16.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.212+0000 7fb3639a5b80 1 --2- 172.21.15.7:0/271775467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b335c906c0 0x55b335c7f710 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb3480057c0 tx=0x7fb348005910 comp rx=0 tx=0).stop 2024-06-10T01:22:16.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.212+0000 7fb3639a5b80 1 -- 172.21.15.7:0/271775467 shutdown_connections 2024-06-10T01:22:16.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.212+0000 7fb3639a5b80 1 --2- 172.21.15.7:0/271775467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b335c90fe0 0x55b335c91af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.212+0000 7fb3639a5b80 1 --2- 172.21.15.7:0/271775467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b335c8afa0 0x55b335c8aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.212+0000 7fb3639a5b80 1 --2- 172.21.15.7:0/271775467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b335c906c0 0x55b335c7f710 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.212+0000 7fb3639a5b80 1 -- 172.21.15.7:0/271775467 >> 172.21.15.7:0/271775467 conn(0x55b335bb5d40 msgr2=0x55b335c92910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:16.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.214+0000 7fb3639a5b80 1 -- 172.21.15.7:0/271775467 shutdown_connections 2024-06-10T01:22:16.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.214+0000 7fb3639a5b80 1 -- 172.21.15.7:0/271775467 wait complete. 2024-06-10T01:22:16.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.215+0000 7fb3639a5b80 1 -- start start 2024-06-10T01:22:16.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.215+0000 7fb3639a5b80 1 Processor -- start 2024-06-10T01:22:16.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.215+0000 7fb3639a5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b335c906c0 0x55b335ca9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:16.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.215+0000 7fb3639a5b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b335c90fe0 0x55b335ca81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:16.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.215+0000 7fb3639a5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b335ca8700 0x55b335ca8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:16.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.215+0000 7fb3639a5b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb354071f10 con 0x55b335c90fe0 2024-06-10T01:22:16.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.215+0000 7fb3639a5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb354073380 con 0x55b335ca8700 2024-06-10T01:22:16.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.215+0000 7fb3639a5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb3540725e0 con 0x55b335c906c0 2024-06-10T01:22:16.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.215+0000 7fb3639a5b80 10 client.0 _ll_register_callbacks cb 0x55b335b769a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:22:16.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.216+0000 7fb35259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b335c90fe0 0x55b335ca81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:16.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.216+0000 7fb35259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b335c90fe0 0x55b335ca81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45574/0 (socket says 172.21.15.7:45574) 2024-06-10T01:22:16.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.216+0000 7fb35259c700 1 -- 172.21.15.7:0/2772940061 learned_addr learned my addr 172.21.15.7:0/2772940061 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:16.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.217+0000 7fb35359e700 1 --2- 172.21.15.7:0/2772940061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b335ca8700 0x55b335ca8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:16.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.217+0000 7fb35259c700 1 -- 172.21.15.7:0/2772940061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b335c906c0 msgr2=0x55b335ca9eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T01:22:16.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.217+0000 7fb35259c700 1 --2- 172.21.15.7:0/2772940061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b335c906c0 0x55b335ca9eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.217+0000 7fb35259c700 1 -- 172.21.15.7:0/2772940061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b335ca8700 msgr2=0x55b335ca8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:16.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.217+0000 7fb35259c700 1 --2- 172.21.15.7:0/2772940061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b335ca8700 0x55b335ca8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.217+0000 7fb35259c700 1 -- 172.21.15.7:0/2772940061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b335d0efc0 con 0x55b335c90fe0 2024-06-10T01:22:16.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.217+0000 7fb35259c700 1 --2- 172.21.15.7:0/2772940061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b335c90fe0 0x55b335ca81c0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fb3480199c0 tx=0x7fb34800fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c94b8263d6b3eb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:16.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.217+0000 7fb336ff5700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:22:16.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.218+0000 7fb336ff5700 1 -- 172.21.15.7:0/2772940061 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb348037020 con 0x55b335c90fe0 2024-06-10T01:22:16.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.219+0000 7fb336ff5700 1 -- 172.21.15.7:0/2772940061 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb348019600 con 0x55b335c90fe0 2024-06-10T01:22:16.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.219+0000 7fb3639a5b80 1 -- 172.21.15.7:0/2772940061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb348011d60 con 0x55b335c90fe0 2024-06-10T01:22:16.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.220+0000 7fb336ff5700 1 -- 172.21.15.7:0/2772940061 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fb348015070 con 0x55b335c90fe0 2024-06-10T01:22:16.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.221+0000 7fb3639a5b80 10 client.19485 fetch_fsmap learned FSMap version 45 2024-06-10T01:22:16.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.221+0000 7fb3639a5b80 1 -- 172.21.15.7:0/2772940061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55b335d10140 con 0x55b335c90fe0 2024-06-10T01:22:16.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.221+0000 7fb336ff5700 1 -- 172.21.15.7:0/2772940061 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb34800fb50 con 0x55b335c90fe0 2024-06-10T01:22:16.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.222+0000 7fb336ff5700 1 -- 172.21.15.7:0/2772940061 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 45) ==== 8358+0+0 (secure 0 0 0) 0x7fb348033690 con 0x55b335c90fe0 2024-06-10T01:22:16.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.222+0000 7fb3639a5b80 10 client.19485 fetch_fsmap finished waiting for FSMap version 45 2024-06-10T01:22:16.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.222+0000 7fb3639a5b80 10 client.19485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:22:16.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.222+0000 7fb3639a5b80 20 client.19485 populate_metadata read hostname 'smithi007' 2024-06-10T01:22:16.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.222+0000 7fb3639a5b80 1 --2- 172.21.15.7:0/2772940061 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b335d10750 0x55b335d12b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:16.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.222+0000 7fb3639a5b80 4 client.19485 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:22:16.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.222+0000 7fb3639a5b80 1 -- 172.21.15.7:0/2772940061 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fb348011d60 con 0x55b335d10750 2024-06-10T01:22:16.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.228+0000 7fb352d9d700 1 --2- 172.21.15.7:0/2772940061 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b335d10750 0x55b335d12b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:16.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.238+0000 7fb352d9d700 1 --2- 172.21.15.7:0/2772940061 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b335d10750 0x55b335d12b70 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3443587b91bf7f1b server_cookie=73dd560bc104c290 in_seq=0 out_seq=0 2024-06-10T01:22:16.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.238+0000 7fb336ff5700 10 client.19485 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:16.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.243+0000 7fb336ff5700 1 -- 172.21.15.7:0/2772940061 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fb348011d60 con 0x55b335d10750 2024-06-10T01:22:16.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.243+0000 7fb336ff5700 10 client.19485 handle_command_reply: tid=0 2024-06-10T01:22:16.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.243+0000 7fb3639a5b80 1 client.19485 shutdown 2024-06-10T01:22:16.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.244+0000 7fb3639a5b80 1 -- 172.21.15.7:0/2772940061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b335c90fe0 msgr2=0x55b335ca81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:16.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.244+0000 7fb3639a5b80 1 --2- 172.21.15.7:0/2772940061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b335c90fe0 0x55b335ca81c0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fb3480199c0 tx=0x7fb34800fd60 comp rx=0 tx=0).stop 2024-06-10T01:22:16.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.246+0000 7fb3639a5b80 1 -- 172.21.15.7:0/2772940061 shutdown_connections 2024-06-10T01:22:16.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.246+0000 7fb3639a5b80 1 --2- 172.21.15.7:0/2772940061 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b335d10750 0x55b335d12b70 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.246+0000 7fb3639a5b80 1 --2- 172.21.15.7:0/2772940061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b335ca8700 0x55b335ca8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.246+0000 7fb3639a5b80 1 --2- 172.21.15.7:0/2772940061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b335c906c0 0x55b335ca9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.246+0000 7fb3639a5b80 1 --2- 172.21.15.7:0/2772940061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b335c90fe0 0x55b335ca81c0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.246+0000 7fb3639a5b80 1 -- 172.21.15.7:0/2772940061 >> 172.21.15.7:0/2772940061 conn(0x55b335bb5d40 msgr2=0x55b335c92910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:16.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.246+0000 7fb336ff5700 0 client.19485 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:16.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.246+0000 7fb3639a5b80 1 -- 172.21.15.7:0/2772940061 shutdown_connections 2024-06-10T01:22:16.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.246+0000 7fb3639a5b80 1 -- 172.21.15.7:0/2772940061 wait complete. 2024-06-10T01:22:16.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.246+0000 7fb3639a5b80 20 client.19485 trim_cache size 0 max 16384 2024-06-10T01:22:16.241 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:22:16.241 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-10T01:22:16.241 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "d7e8820b-9008-4e4b-925a-8bb18bfacaa5", 2024-06-10T01:22:16.241 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-10T01:22:16.241 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:22:16.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.247+0000 7fb359122700 1 -- 172.21.15.7:0/3364416963 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb33c0a2da0 msgr2=0x7fb33c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:16.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.247+0000 7fb359122700 1 --2- 172.21.15.7:0/3364416963 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb33c0a2da0 0x7fb33c0a5260 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb344005fd0 tx=0x7fb34400af20 comp rx=0 tx=0).stop 2024-06-10T01:22:16.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.247+0000 7fb359122700 1 -- 172.21.15.7:0/3364416963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3541073d0 msgr2=0x7fb35410fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:16.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.247+0000 7fb359122700 1 --2- 172.21.15.7:0/3364416963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3541073d0 0x7fb35410fef0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00fc50 tx=0x7fb34c00d760 comp rx=0 tx=0).stop 2024-06-10T01:22:16.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.247+0000 7fb359122700 1 -- 172.21.15.7:0/3364416963 shutdown_connections 2024-06-10T01:22:16.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.247+0000 7fb359122700 1 --2- 172.21.15.7:0/3364416963 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb33c0a2da0 0x7fb33c0a5260 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.247+0000 7fb359122700 1 --2- 172.21.15.7:0/3364416963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3541073d0 0x7fb35410fef0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.247+0000 7fb359122700 1 --2- 172.21.15.7:0/3364416963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb354101390 0x7fb354116e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.247+0000 7fb359122700 1 --2- 172.21.15.7:0/3364416963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb354101d30 0x7fb354117350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:16.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.247+0000 7fb359122700 1 -- 172.21.15.7:0/3364416963 >> 172.21.15.7:0/3364416963 conn(0x7fb3540782c0 msgr2=0x7fb3540ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:16.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.247+0000 7fb359122700 1 -- 172.21.15.7:0/3364416963 shutdown_connections 2024-06-10T01:22:16.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:16.247+0000 7fb359122700 1 -- 172.21.15.7:0/3364416963 wait complete. 2024-06-10T01:22:16.522 INFO:tasks.workunit.client.0.smithi007.stdout: 10 73184 33.16 MB/sec execute 53 sec latency 110.451 ms 2024-06-10T01:22:16.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:16 smithi078 ceph-mon[36683]: scrub queued for path: / 2024-06-10T01:22:16.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:16 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:22:16.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:16 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:22:16.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:16 smithi175 ceph-mon[36636]: scrub queued for path: / 2024-06-10T01:22:16.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:16 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:22:16.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:16 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:22:17.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:16 smithi007 ceph-mon[32398]: scrub queued for path: / 2024-06-10T01:22:17.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:16 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:22:17.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:16 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:22:17.093 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:22:17.438 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:22:17.522 INFO:tasks.workunit.client.0.smithi007.stdout: 10 73915 33.19 MB/sec execute 54 sec latency 44.257 ms 2024-06-10T01:22:17.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:17 smithi078 ceph-mon[36683]: pgmap v920: 129 pgs: 129 active+clean; 532 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.1 MiB/s wr, 722 op/s 2024-06-10T01:22:17.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:17 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:22:17.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:17 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:22:17.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:17 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:22:17.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:17 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:22:17.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:17 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:22:17.885 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:17 smithi007 ceph-mon[32398]: pgmap v920: 129 pgs: 129 active+clean; 532 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.1 MiB/s wr, 722 op/s 2024-06-10T01:22:17.885 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:17 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:22:17.885 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:17 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:22:17.886 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:17 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:22:17.886 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:17 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:22:17.886 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:17 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:22:17.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:17 smithi175 ceph-mon[36636]: pgmap v920: 129 pgs: 129 active+clean; 532 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.1 MiB/s wr, 722 op/s 2024-06-10T01:22:17.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:17 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:22:17.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:17 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:22:17.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:17 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:22:17.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:17 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:22:17.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:17 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:22:18.523 INFO:tasks.workunit.client.0.smithi007.stdout: 10 74285 32.90 MB/sec execute 55 sec latency 395.177 ms 2024-06-10T01:22:19.522 INFO:tasks.workunit.client.0.smithi007.stdout: 10 75044 33.03 MB/sec execute 56 sec latency 53.284 ms 2024-06-10T01:22:19.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:19 smithi078 ceph-mon[36683]: pgmap v921: 129 pgs: 129 active+clean; 533 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.8 MiB/s wr, 628 op/s 2024-06-10T01:22:19.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:19 smithi175 ceph-mon[36636]: pgmap v921: 129 pgs: 129 active+clean; 533 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.8 MiB/s wr, 628 op/s 2024-06-10T01:22:20.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:19 smithi007 ceph-mon[32398]: pgmap v921: 129 pgs: 129 active+clean; 533 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.8 MiB/s wr, 628 op/s 2024-06-10T01:22:20.522 INFO:tasks.workunit.client.0.smithi007.stdout: 10 75756 33.20 MB/sec execute 57 sec latency 132.634 ms 2024-06-10T01:22:21.523 INFO:tasks.workunit.client.0.smithi007.stdout: 10 76470 33.34 MB/sec execute 58 sec latency 121.403 ms 2024-06-10T01:22:21.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.740+0000 7fdf72d78700 1 -- 172.21.15.7:0/3137896581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf6c06b3c0 msgr2=0x7fdf6c10f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:21.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.740+0000 7fdf72d78700 1 --2- 172.21.15.7:0/3137896581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf6c06b3c0 0x7fdf6c10f310 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009990 tx=0x7fdf64009ca0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.740+0000 7fdf72d78700 1 -- 172.21.15.7:0/3137896581 shutdown_connections 2024-06-10T01:22:21.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.740+0000 7fdf72d78700 1 --2- 172.21.15.7:0/3137896581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf6c06b3c0 0x7fdf6c10f310 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.740+0000 7fdf72d78700 1 --2- 172.21.15.7:0/3137896581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf6c1084f0 0x7fdf6c1088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.740+0000 7fdf72d78700 1 --2- 172.21.15.7:0/3137896581 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf6c06aa00 0x7fdf6c06ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.740+0000 7fdf72d78700 1 -- 172.21.15.7:0/3137896581 >> 172.21.15.7:0/3137896581 conn(0x7fdf6c078300 msgr2=0x7fdf6c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:21.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.740+0000 7fdf72d78700 1 -- 172.21.15.7:0/3137896581 shutdown_connections 2024-06-10T01:22:21.734 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:21 smithi007 ceph-mon[32398]: pgmap v922: 129 pgs: 129 active+clean; 535 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.3 MiB/s wr, 866 op/s 2024-06-10T01:22:21.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.740+0000 7fdf72d78700 1 -- 172.21.15.7:0/3137896581 wait complete. 2024-06-10T01:22:21.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.741+0000 7fdf72d78700 1 Processor -- start 2024-06-10T01:22:21.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.741+0000 7fdf72d78700 1 -- start start 2024-06-10T01:22:21.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.741+0000 7fdf72d78700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf6c06aa00 0x7fdf6c19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:21.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.741+0000 7fdf72d78700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf6c06b3c0 0x7fdf6c19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:21.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.741+0000 7fdf72d78700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf6c1084f0 0x7fdf6c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:21.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.741+0000 7fdf72d78700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c1127d0 con 0x7fdf6c1084f0 2024-06-10T01:22:21.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.741+0000 7fdf72d78700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c112650 con 0x7fdf6c06aa00 2024-06-10T01:22:21.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.741+0000 7fdf72d78700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c112950 con 0x7fdf6c06b3c0 2024-06-10T01:22:21.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.741+0000 7fdf71315700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf6c1084f0 0x7fdf6c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:21.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.741+0000 7fdf71315700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf6c1084f0 0x7fdf6c198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45584/0 (socket says 172.21.15.7:45584) 2024-06-10T01:22:21.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.741+0000 7fdf71315700 1 -- 172.21.15.7:0/1647374109 learned_addr learned my addr 172.21.15.7:0/1647374109 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:21.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.741+0000 7fdf63fff700 1 --2- 172.21.15.7:0/1647374109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf6c06b3c0 0x7fdf6c19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:21.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.741+0000 7fdf70b14700 1 --2- 172.21.15.7:0/1647374109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf6c06aa00 0x7fdf6c19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:21.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.742+0000 7fdf71315700 1 -- 172.21.15.7:0/1647374109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf6c06b3c0 msgr2=0x7fdf6c19ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:21.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.742+0000 7fdf71315700 1 --2- 172.21.15.7:0/1647374109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf6c06b3c0 0x7fdf6c19ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.742+0000 7fdf71315700 1 -- 172.21.15.7:0/1647374109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf6c06aa00 msgr2=0x7fdf6c19e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:21.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.742+0000 7fdf71315700 1 --2- 172.21.15.7:0/1647374109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf6c06aa00 0x7fdf6c19e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.742+0000 7fdf71315700 1 -- 172.21.15.7:0/1647374109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf6c199250 con 0x7fdf6c1084f0 2024-06-10T01:22:21.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.742+0000 7fdf71315700 1 --2- 172.21.15.7:0/1647374109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf6c1084f0 0x7fdf6c198af0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400b820 tx=0x7fdf6400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:21.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.742+0000 7fdf6a7fc700 1 -- 172.21.15.7:0/1647374109 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf64027070 con 0x7fdf6c1084f0 2024-06-10T01:22:21.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.742+0000 7fdf6a7fc700 1 -- 172.21.15.7:0/1647374109 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf64003d70 con 0x7fdf6c1084f0 2024-06-10T01:22:21.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.742+0000 7fdf72d78700 1 -- 172.21.15.7:0/1647374109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf6c1994e0 con 0x7fdf6c1084f0 2024-06-10T01:22:21.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.742+0000 7fdf6a7fc700 1 -- 172.21.15.7:0/1647374109 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6400f460 con 0x7fdf6c1084f0 2024-06-10T01:22:21.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.742+0000 7fdf72d78700 1 -- 172.21.15.7:0/1647374109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf6c1a57b0 con 0x7fdf6c1084f0 2024-06-10T01:22:21.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.745+0000 7fdf6a7fc700 1 -- 172.21.15.7:0/1647374109 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fdf6402a070 con 0x7fdf6c1084f0 2024-06-10T01:22:21.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.746+0000 7fdf6a7fc700 1 --2- 172.21.15.7:0/1647374109 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdf580a2ca0 0x7fdf580a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:21.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.746+0000 7fdf70b14700 1 --2- 172.21.15.7:0/1647374109 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdf580a2ca0 0x7fdf580a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:21.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.746+0000 7fdf6a7fc700 1 -- 172.21.15.7:0/1647374109 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fdf640d01d0 con 0x7fdf6c1084f0 2024-06-10T01:22:21.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.746+0000 7fdf70b14700 1 --2- 172.21.15.7:0/1647374109 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdf580a2ca0 0x7fdf580a5160 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c0ffb40 tx=0x7fdf5c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:21.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.747+0000 7fdf7d5fbb80 1 Processor -- start 2024-06-10T01:22:21.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.747+0000 7fdf7d5fbb80 1 -- start start 2024-06-10T01:22:21.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.747+0000 7fdf7d5fbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e996834a70 0x55e996834e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:21.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.747+0000 7fdf7d5fbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e996835390 0x55e99682edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:21.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.747+0000 7fdf7d5fbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e99682f330 0x55e99682f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:21.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.747+0000 7fdf7d5fbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e99677c810 con 0x55e99682f330 2024-06-10T01:22:21.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.747+0000 7fdf7d5fbb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e996724720 con 0x55e996835390 2024-06-10T01:22:21.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.747+0000 7fdf7d5fbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e9967787d0 con 0x55e996834a70 2024-06-10T01:22:21.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.747+0000 7fdf71315700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e99682f330 0x55e99682f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:21.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.747+0000 7fdf70b14700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e996834a70 0x55e996834e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:21.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.747+0000 7fdf71315700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e99682f330 0x55e99682f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45592/0 (socket says 172.21.15.7:45592) 2024-06-10T01:22:21.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.747+0000 7fdf71315700 1 -- 172.21.15.7:0/1174271760 learned_addr learned my addr 172.21.15.7:0/1174271760 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:21.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.747+0000 7fdf63fff700 1 --2- 172.21.15.7:0/1174271760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e996835390 0x55e99682edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:21.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.747+0000 7fdf70b14700 1 -- 172.21.15.7:0/1174271760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e996835390 msgr2=0x55e99682edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:21.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.747+0000 7fdf70b14700 1 --2- 172.21.15.7:0/1174271760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e996835390 0x55e99682edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.747+0000 7fdf70b14700 1 -- 172.21.15.7:0/1174271760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e99682f330 msgr2=0x55e99682f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:21.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.747+0000 7fdf70b14700 1 --2- 172.21.15.7:0/1174271760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e99682f330 0x55e99682f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.747+0000 7fdf70b14700 1 -- 172.21.15.7:0/1174271760 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e99671af50 con 0x55e996834a70 2024-06-10T01:22:21.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.747+0000 7fdf71315700 1 --2- 172.21.15.7:0/1174271760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e99682f330 0x55e99682f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:22:21.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.748+0000 7fdf70b14700 1 --2- 172.21.15.7:0/1174271760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e996834a70 0x55e996834e50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00fc60 tx=0x7fdf5c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17f7a5e2a77bc5c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:21.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.748+0000 7fdf62ffd700 1 -- 172.21.15.7:0/1174271760 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c003c00 con 0x55e996834a70 2024-06-10T01:22:21.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.748+0000 7fdf62ffd700 1 -- 172.21.15.7:0/1174271760 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf5c01b070 con 0x55e996834a70 2024-06-10T01:22:21.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.748+0000 7fdf62ffd700 1 -- 172.21.15.7:0/1174271760 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c004260 con 0x55e996834a70 2024-06-10T01:22:21.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.749+0000 7fdf7d5fbb80 1 -- 172.21.15.7:0/1174271760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e996834a70 msgr2=0x55e996834e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:21.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.749+0000 7fdf7d5fbb80 1 --2- 172.21.15.7:0/1174271760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e996834a70 0x55e996834e50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00fc60 tx=0x7fdf5c014000 comp rx=0 tx=0).stop 2024-06-10T01:22:21.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.749+0000 7fdf7d5fbb80 1 -- 172.21.15.7:0/1174271760 shutdown_connections 2024-06-10T01:22:21.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.749+0000 7fdf7d5fbb80 1 --2- 172.21.15.7:0/1174271760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e99682f330 0x55e99682f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.749+0000 7fdf7d5fbb80 1 --2- 172.21.15.7:0/1174271760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e996834a70 0x55e996834e50 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.749+0000 7fdf7d5fbb80 1 --2- 172.21.15.7:0/1174271760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e996835390 0x55e99682edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.749+0000 7fdf7d5fbb80 1 -- 172.21.15.7:0/1174271760 >> 172.21.15.7:0/1174271760 conn(0x55e996755cc0 msgr2=0x55e996825850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:21.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.749+0000 7fdf7d5fbb80 1 -- 172.21.15.7:0/1174271760 shutdown_connections 2024-06-10T01:22:21.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.749+0000 7fdf7d5fbb80 1 -- 172.21.15.7:0/1174271760 wait complete. 2024-06-10T01:22:21.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.749+0000 7fdf7d5fbb80 1 -- start start 2024-06-10T01:22:21.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.749+0000 7fdf7d5fbb80 1 Processor -- start 2024-06-10T01:22:21.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.750+0000 7fdf7d5fbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e99682f330 0x55e99683b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:21.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.750+0000 7fdf7d5fbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e996834a70 0x55e9968383d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:21.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.750+0000 7fdf7d5fbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e996835390 0x55e996838910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:21.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.750+0000 7fdf7d5fbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e99683bdd0 con 0x55e996835390 2024-06-10T01:22:21.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.750+0000 7fdf7d5fbb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e99671cbf0 con 0x55e996834a70 2024-06-10T01:22:21.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.750+0000 7fdf7d5fbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e99683bf50 con 0x55e99682f330 2024-06-10T01:22:21.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.750+0000 7fdf7d5fbb80 10 client.0 _ll_register_callbacks cb 0x55e996716920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:22:21.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.750+0000 7fdf71315700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e996835390 0x55e996838910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:21.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.750+0000 7fdf70b14700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e99682f330 0x55e99683b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:21.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.750+0000 7fdf71315700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e996835390 0x55e996838910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45608/0 (socket says 172.21.15.7:45608) 2024-06-10T01:22:21.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.750+0000 7fdf71315700 1 -- 172.21.15.7:0/483718336 learned_addr learned my addr 172.21.15.7:0/483718336 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:21.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.750+0000 7fdf63fff700 1 --2- 172.21.15.7:0/483718336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e996834a70 0x55e9968383d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:21.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.750+0000 7fdf71315700 1 -- 172.21.15.7:0/483718336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e99682f330 msgr2=0x55e99683b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:21.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.750+0000 7fdf71315700 1 --2- 172.21.15.7:0/483718336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e99682f330 0x55e99683b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.750+0000 7fdf71315700 1 -- 172.21.15.7:0/483718336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e996834a70 msgr2=0x55e9968383d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:21.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.750+0000 7fdf71315700 1 --2- 172.21.15.7:0/483718336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e996834a70 0x55e9968383d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.750+0000 7fdf71315700 1 -- 172.21.15.7:0/483718336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9966cb8d0 con 0x55e996835390 2024-06-10T01:22:21.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.750+0000 7fdf71315700 1 --2- 172.21.15.7:0/483718336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e996835390 0x55e996838910 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fdf64099110 tx=0x7fdf640a3180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3675ce9a86c78352 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:21.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.750+0000 7fdf4bfff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:22:21.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.751+0000 7fdf4bfff700 1 -- 172.21.15.7:0/483718336 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6409c510 con 0x55e996835390 2024-06-10T01:22:21.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.751+0000 7fdf4bfff700 1 -- 172.21.15.7:0/483718336 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fdf640a4e90 con 0x55e996835390 2024-06-10T01:22:21.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.751+0000 7fdf7d5fbb80 1 -- 172.21.15.7:0/483718336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf5c01b070 con 0x55e996835390 2024-06-10T01:22:21.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.751+0000 7fdf4bfff700 1 -- 172.21.15.7:0/483718336 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdf6409c0a0 con 0x55e996835390 2024-06-10T01:22:21.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.751+0000 7fdf7d5fbb80 10 client.19497 fetch_fsmap learned FSMap version 45 2024-06-10T01:22:21.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.751+0000 7fdf7d5fbb80 1 -- 172.21.15.7:0/483718336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55e9968466a0 con 0x55e996835390 2024-06-10T01:22:21.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.751+0000 7fdf4bfff700 1 -- 172.21.15.7:0/483718336 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf640043d0 con 0x55e996835390 2024-06-10T01:22:21.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.751+0000 7fdf4bfff700 1 -- 172.21.15.7:0/483718336 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 45) ==== 8358+0+0 (secure 0 0 0) 0x7fdf640bc040 con 0x55e996835390 2024-06-10T01:22:21.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.751+0000 7fdf7d5fbb80 10 client.19497 fetch_fsmap finished waiting for FSMap version 45 2024-06-10T01:22:21.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.751+0000 7fdf7d5fbb80 10 client.19497 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:22:21.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.752+0000 7fdf7d5fbb80 20 client.19497 populate_metadata read hostname 'smithi007' 2024-06-10T01:22:21.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.752+0000 7fdf7d5fbb80 1 --2- 172.21.15.7:0/483718336 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e996848d10 0x55e99684b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:21.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.752+0000 7fdf7d5fbb80 4 client.19497 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:22:21.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.752+0000 7fdf7d5fbb80 1 -- 172.21.15.7:0/483718336 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdf5c01b070 con 0x55e996848d10 2024-06-10T01:22:21.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.752+0000 7fdf70b14700 1 --2- 172.21.15.7:0/483718336 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e996848d10 0x55e99684b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:21.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.752+0000 7fdf4bfff700 10 client.19497 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:21.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.752+0000 7fdf70b14700 1 --2- 172.21.15.7:0/483718336 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e996848d10 0x55e99684b130 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e5590a62dc27b67 server_cookie=b85973b7901523d4 in_seq=0 out_seq=0 2024-06-10T01:22:21.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.753+0000 7fdf4bfff700 1 -- 172.21.15.7:0/483718336 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fdf5c01b070 con 0x55e996848d10 2024-06-10T01:22:21.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.753+0000 7fdf4bfff700 10 client.19497 handle_command_reply: tid=0 2024-06-10T01:22:21.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.753+0000 7fdf7d5fbb80 1 client.19497 shutdown 2024-06-10T01:22:21.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.753+0000 7fdf7d5fbb80 1 -- 172.21.15.7:0/483718336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e996835390 msgr2=0x55e996838910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:21.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.753+0000 7fdf7d5fbb80 1 --2- 172.21.15.7:0/483718336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e996835390 0x55e996838910 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fdf64099110 tx=0x7fdf640a3180 comp rx=0 tx=0).stop 2024-06-10T01:22:21.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.754+0000 7fdf7d5fbb80 1 -- 172.21.15.7:0/483718336 shutdown_connections 2024-06-10T01:22:21.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.754+0000 7fdf7d5fbb80 1 --2- 172.21.15.7:0/483718336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e996835390 0x55e996838910 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.754+0000 7fdf7d5fbb80 1 --2- 172.21.15.7:0/483718336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e99682f330 0x55e99683b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.754+0000 7fdf7d5fbb80 1 --2- 172.21.15.7:0/483718336 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e996848d10 0x55e99684b130 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.754+0000 7fdf7d5fbb80 1 --2- 172.21.15.7:0/483718336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e996834a70 0x55e9968383d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.754+0000 7fdf7d5fbb80 1 -- 172.21.15.7:0/483718336 >> 172.21.15.7:0/483718336 conn(0x55e996755cc0 msgr2=0x55e9968299d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:21.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.754+0000 7fdf4bfff700 0 client.19497 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:21.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.754+0000 7fdf7d5fbb80 1 -- 172.21.15.7:0/483718336 shutdown_connections 2024-06-10T01:22:21.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.754+0000 7fdf7d5fbb80 1 -- 172.21.15.7:0/483718336 wait complete. 2024-06-10T01:22:21.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.754+0000 7fdf7d5fbb80 20 client.19497 trim_cache size 0 max 16384 2024-06-10T01:22:21.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.776+0000 7fdf7d5fbb80 1 Processor -- start 2024-06-10T01:22:21.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.776+0000 7fdf7d5fbb80 1 -- start start 2024-06-10T01:22:21.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.776+0000 7fdf7d5fbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e99682f330 0x55e996835e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:21.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.776+0000 7fdf7d5fbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e996834a70 0x55e996850a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:21.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.776+0000 7fdf7d5fbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e996835390 0x55e996850f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:21.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.776+0000 7fdf7d5fbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c1ae8d0 con 0x55e99682f330 2024-06-10T01:22:21.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.776+0000 7fdf7d5fbb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c1ac100 con 0x55e996835390 2024-06-10T01:22:21.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.776+0000 7fdf7d5fbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c1a9f50 con 0x55e996834a70 2024-06-10T01:22:21.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.777+0000 7fdf70b14700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e99682f330 0x55e996835e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:21.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.777+0000 7fdf70b14700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e99682f330 0x55e996835e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45614/0 (socket says 172.21.15.7:45614) 2024-06-10T01:22:21.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.777+0000 7fdf70b14700 1 -- 172.21.15.7:0/1746477158 learned_addr learned my addr 172.21.15.7:0/1746477158 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:21.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.777+0000 7fdf63fff700 1 --2- 172.21.15.7:0/1746477158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e996834a70 0x55e996850a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:21.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.777+0000 7fdf71315700 1 --2- 172.21.15.7:0/1746477158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e996835390 0x55e996850f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:21.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.777+0000 7fdf70b14700 1 -- 172.21.15.7:0/1746477158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e996834a70 msgr2=0x55e996850a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:21.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.777+0000 7fdf70b14700 1 --2- 172.21.15.7:0/1746477158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e996834a70 0x55e996850a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.777+0000 7fdf70b14700 1 -- 172.21.15.7:0/1746477158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e996835390 msgr2=0x55e996850f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:21.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.777+0000 7fdf70b14700 1 --2- 172.21.15.7:0/1746477158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e996835390 0x55e996850f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.777+0000 7fdf70b14700 1 -- 172.21.15.7:0/1746477158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9968392d0 con 0x55e99682f330 2024-06-10T01:22:21.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.777+0000 7fdf70b14700 1 --2- 172.21.15.7:0/1746477158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e99682f330 0x55e996835e80 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c015e90 tx=0x7fdf5c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba2420dc4b45215d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:21.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.777+0000 7fdf627fc700 1 -- 172.21.15.7:0/1746477158 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c015620 con 0x55e99682f330 2024-06-10T01:22:21.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.777+0000 7fdf627fc700 1 -- 172.21.15.7:0/1746477158 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf5c01b070 con 0x55e99682f330 2024-06-10T01:22:21.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.777+0000 7fdf627fc700 1 -- 172.21.15.7:0/1746477158 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c033ae0 con 0x55e99682f330 2024-06-10T01:22:21.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.778+0000 7fdf7d5fbb80 1 -- 172.21.15.7:0/1746477158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e99682f330 msgr2=0x55e996835e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:21.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.778+0000 7fdf7d5fbb80 1 --2- 172.21.15.7:0/1746477158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e99682f330 0x55e996835e80 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c015e90 tx=0x7fdf5c009920 comp rx=0 tx=0).stop 2024-06-10T01:22:21.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.778+0000 7fdf7d5fbb80 1 -- 172.21.15.7:0/1746477158 shutdown_connections 2024-06-10T01:22:21.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.778+0000 7fdf7d5fbb80 1 --2- 172.21.15.7:0/1746477158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e996835390 0x55e996850f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.778+0000 7fdf7d5fbb80 1 --2- 172.21.15.7:0/1746477158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e99682f330 0x55e996835e80 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.778+0000 7fdf7d5fbb80 1 --2- 172.21.15.7:0/1746477158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e996834a70 0x55e996850a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.778+0000 7fdf7d5fbb80 1 -- 172.21.15.7:0/1746477158 >> 172.21.15.7:0/1746477158 conn(0x55e996755cc0 msgr2=0x55e996835770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:21.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.778+0000 7fdf7d5fbb80 1 -- 172.21.15.7:0/1746477158 shutdown_connections 2024-06-10T01:22:21.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.778+0000 7fdf7d5fbb80 1 -- 172.21.15.7:0/1746477158 wait complete. 2024-06-10T01:22:21.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.779+0000 7fdf7d5fbb80 1 -- start start 2024-06-10T01:22:21.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.779+0000 7fdf7d5fbb80 1 Processor -- start 2024-06-10T01:22:21.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.779+0000 7fdf7d5fbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e99682f330 0x55e99683d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:21.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.779+0000 7fdf7d5fbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e996834a70 0x55e99683d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:21.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.779+0000 7fdf7d5fbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e996835390 0x55e9968b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:21.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.779+0000 7fdf7d5fbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c1aa5c0 con 0x55e996835390 2024-06-10T01:22:21.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.779+0000 7fdf7d5fbb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c1aba30 con 0x55e99682f330 2024-06-10T01:22:21.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.779+0000 7fdf7d5fbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c1aac90 con 0x55e996834a70 2024-06-10T01:22:21.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.779+0000 7fdf7d5fbb80 10 client.0 _ll_register_callbacks cb 0x55e996716920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:22:21.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.779+0000 7fdf71315700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e996835390 0x55e9968b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:21.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.780+0000 7fdf63fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e996834a70 0x55e99683d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:21.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.780+0000 7fdf71315700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e996835390 0x55e9968b29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45616/0 (socket says 172.21.15.7:45616) 2024-06-10T01:22:21.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.780+0000 7fdf71315700 1 -- 172.21.15.7:0/3981406269 learned_addr learned my addr 172.21.15.7:0/3981406269 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:21.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.780+0000 7fdf70b14700 1 --2- 172.21.15.7:0/3981406269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e99682f330 0x55e99683d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:21.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.780+0000 7fdf71315700 1 -- 172.21.15.7:0/3981406269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e996834a70 msgr2=0x55e99683d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:21.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.780+0000 7fdf71315700 1 --2- 172.21.15.7:0/3981406269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e996834a70 0x55e99683d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.780+0000 7fdf71315700 1 -- 172.21.15.7:0/3981406269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e99682f330 msgr2=0x55e99683d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:21.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.780+0000 7fdf71315700 1 --2- 172.21.15.7:0/3981406269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e99682f330 0x55e99683d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.780+0000 7fdf71315700 1 -- 172.21.15.7:0/3981406269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9968b2f10 con 0x55e996835390 2024-06-10T01:22:21.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.780+0000 7fdf71315700 1 --2- 172.21.15.7:0/3981406269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e996835390 0x55e9968b29d0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fdf640d04f0 tx=0x7fdf640d0160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15cbdc019eacc35 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.780+0000 7fdf4bfff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:22:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.780+0000 7fdf4bfff700 1 -- 172.21.15.7:0/3981406269 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6409c280 con 0x55e996835390 2024-06-10T01:22:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.780+0000 7fdf7d5fbb80 1 -- 172.21.15.7:0/3981406269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf5c01b070 con 0x55e996835390 2024-06-10T01:22:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.780+0000 7fdf4bfff700 1 -- 172.21.15.7:0/3981406269 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fdf640984e0 con 0x55e996835390 2024-06-10T01:22:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.781+0000 7fdf4bfff700 1 -- 172.21.15.7:0/3981406269 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdf6409c510 con 0x55e996835390 2024-06-10T01:22:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.781+0000 7fdf7d5fbb80 10 client.19503 fetch_fsmap learned FSMap version 45 2024-06-10T01:22:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.781+0000 7fdf7d5fbb80 1 -- 172.21.15.7:0/3981406269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55e9968b2f10 con 0x55e996835390 2024-06-10T01:22:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.781+0000 7fdf4bfff700 1 -- 172.21.15.7:0/3981406269 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf5c01b070 con 0x55e996835390 2024-06-10T01:22:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.781+0000 7fdf4bfff700 1 -- 172.21.15.7:0/3981406269 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 45) ==== 8358+0+0 (secure 0 0 0) 0x7fdf640bc040 con 0x55e996835390 2024-06-10T01:22:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.781+0000 7fdf7d5fbb80 10 client.19503 fetch_fsmap finished waiting for FSMap version 45 2024-06-10T01:22:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.781+0000 7fdf7d5fbb80 10 client.19503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:22:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.781+0000 7fdf7d5fbb80 20 client.19503 populate_metadata read hostname 'smithi007' 2024-06-10T01:22:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.781+0000 7fdf7d5fbb80 1 --2- 172.21.15.7:0/3981406269 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e9968b4520 0x55e9968b6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.781+0000 7fdf7d5fbb80 4 client.19503 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:22:21.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.781+0000 7fdf7d5fbb80 1 -- 172.21.15.7:0/3981406269 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e996789d10 con 0x55e9968b4520 2024-06-10T01:22:21.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.782+0000 7fdf70b14700 1 --2- 172.21.15.7:0/3981406269 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e9968b4520 0x55e9968b6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:21.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.782+0000 7fdf4bfff700 10 client.19503 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:21.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.782+0000 7fdf70b14700 1 --2- 172.21.15.7:0/3981406269 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e9968b4520 0x55e9968b6940 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec3dbb67e0c22886 server_cookie=b3e9338bd6cc5034 in_seq=0 out_seq=0 2024-06-10T01:22:21.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.783+0000 7fdf4bfff700 1 -- 172.21.15.7:0/3981406269 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e996789d10 con 0x55e9968b4520 2024-06-10T01:22:21.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.783+0000 7fdf4bfff700 10 client.19503 handle_command_reply: tid=0 2024-06-10T01:22:21.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.783+0000 7fdf7d5fbb80 1 client.19503 shutdown 2024-06-10T01:22:21.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.783+0000 7fdf7d5fbb80 1 -- 172.21.15.7:0/3981406269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e996835390 msgr2=0x55e9968b29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:21.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.783+0000 7fdf7d5fbb80 1 --2- 172.21.15.7:0/3981406269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e996835390 0x55e9968b29d0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fdf640d04f0 tx=0x7fdf640d0160 comp rx=0 tx=0).stop 2024-06-10T01:22:21.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.783+0000 7fdf7d5fbb80 1 -- 172.21.15.7:0/3981406269 shutdown_connections 2024-06-10T01:22:21.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.783+0000 7fdf7d5fbb80 1 --2- 172.21.15.7:0/3981406269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e996835390 0x55e9968b29d0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.783+0000 7fdf7d5fbb80 1 --2- 172.21.15.7:0/3981406269 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55e9968b4520 0x55e9968b6940 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.783+0000 7fdf4bfff700 0 client.19503 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:21.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.783+0000 7fdf7d5fbb80 1 --2- 172.21.15.7:0/3981406269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e99682f330 0x55e99683d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.783+0000 7fdf7d5fbb80 1 --2- 172.21.15.7:0/3981406269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e996834a70 0x55e99683d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.783+0000 7fdf7d5fbb80 1 -- 172.21.15.7:0/3981406269 >> 172.21.15.7:0/3981406269 conn(0x55e996755cc0 msgr2=0x55e99681ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:21.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.783+0000 7fdf7d5fbb80 1 -- 172.21.15.7:0/3981406269 shutdown_connections 2024-06-10T01:22:21.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.783+0000 7fdf7d5fbb80 1 -- 172.21.15.7:0/3981406269 wait complete. 2024-06-10T01:22:21.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.783+0000 7fdf7d5fbb80 20 client.19503 trim_cache size 0 max 16384 2024-06-10T01:22:21.780 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:22:21.780 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T01:22:21.780 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-10T01:22:21.780 INFO:teuthology.orchestra.run.smithi007.stdout: "d7e8820b-9008-4e4b-925a-8bb18bfacaa5": { 2024-06-10T01:22:21.781 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-10T01:22:21.781 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "d7e8820b-9008-4e4b-925a-8bb18bfacaa5", 2024-06-10T01:22:21.781 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-10T01:22:21.781 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:22:21.781 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:22:21.781 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:22:21.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.784+0000 7fdf72d78700 1 -- 172.21.15.7:0/1647374109 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdf580a2ca0 msgr2=0x7fdf580a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:21.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.784+0000 7fdf72d78700 1 --2- 172.21.15.7:0/1647374109 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdf580a2ca0 0x7fdf580a5160 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c0ffb40 tx=0x7fdf5c00b040 comp rx=0 tx=0).stop 2024-06-10T01:22:21.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.784+0000 7fdf72d78700 1 -- 172.21.15.7:0/1647374109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf6c1084f0 msgr2=0x7fdf6c198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:21.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.784+0000 7fdf72d78700 1 --2- 172.21.15.7:0/1647374109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf6c1084f0 0x7fdf6c198af0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400b820 tx=0x7fdf6400bab0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.784+0000 7fdf72d78700 1 -- 172.21.15.7:0/1647374109 shutdown_connections 2024-06-10T01:22:21.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.784+0000 7fdf72d78700 1 --2- 172.21.15.7:0/1647374109 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdf580a2ca0 0x7fdf580a5160 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.784+0000 7fdf72d78700 1 --2- 172.21.15.7:0/1647374109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf6c1084f0 0x7fdf6c198af0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.784+0000 7fdf72d78700 1 --2- 172.21.15.7:0/1647374109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf6c06aa00 0x7fdf6c19e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.784+0000 7fdf72d78700 1 --2- 172.21.15.7:0/1647374109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf6c06b3c0 0x7fdf6c19ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:21.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.784+0000 7fdf72d78700 1 -- 172.21.15.7:0/1647374109 >> 172.21.15.7:0/1647374109 conn(0x7fdf6c078300 msgr2=0x7fdf6c0fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:21.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.784+0000 7fdf72d78700 1 -- 172.21.15.7:0/1647374109 shutdown_connections 2024-06-10T01:22:21.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:21.784+0000 7fdf72d78700 1 -- 172.21.15.7:0/1647374109 wait complete. 2024-06-10T01:22:21.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:21 smithi078 ceph-mon[36683]: pgmap v922: 129 pgs: 129 active+clean; 535 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.3 MiB/s wr, 866 op/s 2024-06-10T01:22:21.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:21 smithi175 ceph-mon[36636]: pgmap v922: 129 pgs: 129 active+clean; 535 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.3 MiB/s wr, 866 op/s 2024-06-10T01:22:22.523 INFO:tasks.workunit.client.0.smithi007.stdout: 10 77161 33.35 MB/sec execute 59 sec latency 67.828 ms 2024-06-10T01:22:22.602 INFO:tasks.cephfs.filesystem:scrub status for tag:d7e8820b-9008-4e4b-925a-8bb18bfacaa5 - {'path': '/', 'tag': 'd7e8820b-9008-4e4b-925a-8bb18bfacaa5', 'options': 'recursive,force'} 2024-06-10T01:22:23.523 INFO:tasks.workunit.client.0.smithi007.stdout: 10 77565 33.22 MB/sec execute 60 sec latency 325.907 ms 2024-06-10T01:22:23.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:23 smithi078 ceph-mon[36683]: pgmap v923: 129 pgs: 129 active+clean; 539 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.4 MiB/s wr, 809 op/s 2024-06-10T01:22:23.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:23 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:22:23.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:23 smithi175 ceph-mon[36636]: pgmap v923: 129 pgs: 129 active+clean; 539 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.4 MiB/s wr, 809 op/s 2024-06-10T01:22:23.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:23 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:22:24.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:23 smithi007 ceph-mon[32398]: pgmap v923: 129 pgs: 129 active+clean; 539 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.4 MiB/s wr, 809 op/s 2024-06-10T01:22:24.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:23 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:22:24.523 INFO:tasks.workunit.client.0.smithi007.stdout: 10 78095 33.08 MB/sec execute 61 sec latency 120.619 ms 2024-06-10T01:22:25.523 INFO:tasks.workunit.client.0.smithi007.stdout: 10 78858 33.15 MB/sec execute 62 sec latency 76.904 ms 2024-06-10T01:22:25.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:25 smithi078 ceph-mon[36683]: pgmap v924: 129 pgs: 129 active+clean; 546 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.5 MiB/s wr, 893 op/s 2024-06-10T01:22:25.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:25 smithi175 ceph-mon[36636]: pgmap v924: 129 pgs: 129 active+clean; 546 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.5 MiB/s wr, 893 op/s 2024-06-10T01:22:26.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:25 smithi007 ceph-mon[32398]: pgmap v924: 129 pgs: 129 active+clean; 546 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.5 MiB/s wr, 893 op/s 2024-06-10T01:22:26.523 INFO:tasks.workunit.client.0.smithi007.stdout: 10 79784 33.48 MB/sec execute 63 sec latency 45.652 ms 2024-06-10T01:22:27.523 INFO:tasks.workunit.client.0.smithi007.stdout: 10 80466 33.57 MB/sec execute 64 sec latency 63.232 ms 2024-06-10T01:22:27.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:27 smithi078 ceph-mon[36683]: pgmap v925: 129 pgs: 129 active+clean; 548 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 8.6 MiB/s wr, 834 op/s 2024-06-10T01:22:27.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:27 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:22:27.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:27 smithi175 ceph-mon[36636]: pgmap v925: 129 pgs: 129 active+clean; 548 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 8.6 MiB/s wr, 834 op/s 2024-06-10T01:22:27.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:27 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:22:28.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:27 smithi007 ceph-mon[32398]: pgmap v925: 129 pgs: 129 active+clean; 548 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 8.6 MiB/s wr, 834 op/s 2024-06-10T01:22:28.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:27 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:22:28.523 INFO:tasks.workunit.client.0.smithi007.stdout: 10 80947 33.40 MB/sec execute 65 sec latency 212.794 ms 2024-06-10T01:22:28.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.626+0000 7ff538534640 1 -- 172.21.15.7:0/3802785558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff530102480 msgr2=0x7ff530104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.626+0000 7ff538534640 1 --2- 172.21.15.7:0/3802785558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff530102480 0x7ff530104870 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7ff52c009900 tx=0x7ff52c02a410 comp rx=0 tx=0).stop 2024-06-10T01:22:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.626+0000 7ff538534640 1 -- 172.21.15.7:0/3802785558 shutdown_connections 2024-06-10T01:22:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.626+0000 7ff538534640 1 --2- 172.21.15.7:0/3802785558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5301076e0 0x7ff530109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.626+0000 7ff538534640 1 --2- 172.21.15.7:0/3802785558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff530104db0 0x7ff5301071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.626+0000 7ff538534640 1 --2- 172.21.15.7:0/3802785558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff530102480 0x7ff530104870 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.626+0000 7ff538534640 1 -- 172.21.15.7:0/3802785558 >> 172.21.15.7:0/3802785558 conn(0x7ff5300fbe90 msgr2=0x7ff5300fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.626+0000 7ff538534640 1 -- 172.21.15.7:0/3802785558 shutdown_connections 2024-06-10T01:22:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.626+0000 7ff538534640 1 -- 172.21.15.7:0/3802785558 wait complete. 2024-06-10T01:22:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.627+0000 7ff538534640 1 Processor -- start 2024-06-10T01:22:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.627+0000 7ff538534640 1 -- start start 2024-06-10T01:22:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.627+0000 7ff538534640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff530102480 0x7ff530101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.627+0000 7ff538534640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff530104db0 0x7ff530100430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.627+0000 7ff538534640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5301076e0 0x7ff530100970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.627+0000 7ff538534640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff53010d750 con 0x7ff5301076e0 2024-06-10T01:22:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.627+0000 7ff538534640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff53010d5d0 con 0x7ff530104db0 2024-06-10T01:22:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.627+0000 7ff538534640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff53010d8d0 con 0x7ff530102480 2024-06-10T01:22:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.628+0000 7ff536ad1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5301076e0 0x7ff530100970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.628+0000 7ff536ad1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5301076e0 0x7ff530100970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57466/0 (socket says 172.21.15.7:57466) 2024-06-10T01:22:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.628+0000 7ff536ad1640 1 -- 172.21.15.7:0/1602810159 learned_addr learned my addr 172.21.15.7:0/1602810159 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.628+0000 7ff535acf640 1 --2- 172.21.15.7:0/1602810159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff530104db0 0x7ff530100430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.628+0000 7ff5362d0640 1 --2- 172.21.15.7:0/1602810159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff530102480 0x7ff530101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.628+0000 7ff536ad1640 1 -- 172.21.15.7:0/1602810159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff530102480 msgr2=0x7ff530101df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.628+0000 7ff536ad1640 1 --2- 172.21.15.7:0/1602810159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff530102480 0x7ff530101df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.628+0000 7ff536ad1640 1 -- 172.21.15.7:0/1602810159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff530104db0 msgr2=0x7ff530100430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.628+0000 7ff536ad1640 1 --2- 172.21.15.7:0/1602810159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff530104db0 0x7ff530100430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.628+0000 7ff536ad1640 1 -- 172.21.15.7:0/1602810159 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff530101200 con 0x7ff5301076e0 2024-06-10T01:22:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.628+0000 7ff535acf640 1 --2- 172.21.15.7:0/1602810159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff530104db0 0x7ff530100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:22:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.628+0000 7ff536ad1640 1 --2- 172.21.15.7:0/1602810159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5301076e0 0x7ff530100970 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7ff52800efd0 tx=0x7ff52800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.628+0000 7ff5277fe640 1 -- 172.21.15.7:0/1602810159 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff528003b70 con 0x7ff5301076e0 2024-06-10T01:22:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.629+0000 7ff5277fe640 1 -- 172.21.15.7:0/1602810159 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff528003d10 con 0x7ff5301076e0 2024-06-10T01:22:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.629+0000 7ff538534640 1 -- 172.21.15.7:0/1602810159 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5301a1ec0 con 0x7ff5301076e0 2024-06-10T01:22:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.629+0000 7ff538534640 1 -- 172.21.15.7:0/1602810159 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5301a23b0 con 0x7ff5301076e0 2024-06-10T01:22:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.629+0000 7ff5277fe640 1 -- 172.21.15.7:0/1602810159 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5280108d0 con 0x7ff5301076e0 2024-06-10T01:22:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.633+0000 7ff53b243740 1 Processor -- start 2024-06-10T01:22:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.633+0000 7ff5277fe640 1 -- 172.21.15.7:0/1602810159 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7ff528004020 con 0x7ff5301076e0 2024-06-10T01:22:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.633+0000 7ff53b243740 1 -- start start 2024-06-10T01:22:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.634+0000 7ff53b243740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f3552c820 0x557f3552cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.634+0000 7ff53b243740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f3552d210 0x557f35531d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.634+0000 7ff53b243740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f35532380 0x557f35534810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.634+0000 7ff53b243740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557f354874f0 con 0x557f3552c820 2024-06-10T01:22:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.634+0000 7ff53b243740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557f35425870 con 0x557f35532380 2024-06-10T01:22:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.634+0000 7ff53b243740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557f354834b0 con 0x557f3552d210 2024-06-10T01:22:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.634+0000 7ff535acf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f3552c820 0x557f3552cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.634+0000 7ff5277fe640 1 --2- 172.21.15.7:0/1602810159 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff51c0a2cd0 0x7ff51c0a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.634+0000 7ff535acf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f3552c820 0x557f3552cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57476/0 (socket says 172.21.15.7:57476) 2024-06-10T01:22:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.634+0000 7ff535acf640 1 -- 172.21.15.7:0/2155254070 learned_addr learned my addr 172.21.15.7:0/2155254070 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.634+0000 7ff535acf640 1 --2- 172.21.15.7:0/2155254070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f35532380 0x557f35534810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.634+0000 7ff5362d0640 1 --2- 172.21.15.7:0/2155254070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f3552d210 0x557f35531d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.634+0000 7ff5277fe640 1 -- 172.21.15.7:0/1602810159 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7ff5280cfa20 con 0x7ff5301076e0 2024-06-10T01:22:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.635+0000 7ff536ad1640 1 --2- 172.21.15.7:0/1602810159 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff51c0a2cd0 0x7ff51c0a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.635+0000 7ff535acf640 1 -- 172.21.15.7:0/2155254070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f3552d210 msgr2=0x557f35531d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.635+0000 7ff535acf640 1 --2- 172.21.15.7:0/2155254070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f3552d210 0x557f35531d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.635+0000 7ff535acf640 1 -- 172.21.15.7:0/2155254070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f3552c820 msgr2=0x557f3552cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.635+0000 7ff535acf640 1 --2- 172.21.15.7:0/2155254070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f3552c820 0x557f3552cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.635+0000 7ff535acf640 1 -- 172.21.15.7:0/2155254070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f354419b0 con 0x557f35532380 2024-06-10T01:22:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.635+0000 7ff535acf640 1 --2- 172.21.15.7:0/2155254070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f35532380 0x557f35534810 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff520015a40 tx=0x7ff520009110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36d4d5eec2ffdfb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.635+0000 7ff524bfe640 1 -- 172.21.15.7:0/2155254070 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5200139b0 con 0x557f35532380 2024-06-10T01:22:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.635+0000 7ff524bfe640 1 -- 172.21.15.7:0/2155254070 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff520018450 con 0x557f35532380 2024-06-10T01:22:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.635+0000 7ff524bfe640 1 -- 172.21.15.7:0/2155254070 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff520017440 con 0x557f35532380 2024-06-10T01:22:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.636+0000 7ff53b243740 1 -- 172.21.15.7:0/2155254070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f35532380 msgr2=0x557f35534810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.636+0000 7ff536ad1640 1 --2- 172.21.15.7:0/1602810159 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff51c0a2cd0 0x7ff51c0a5190 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff52809bba0 tx=0x7ff52800cd50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.636+0000 7ff53b243740 1 --2- 172.21.15.7:0/2155254070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f35532380 0x557f35534810 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff520015a40 tx=0x7ff520009110 comp rx=0 tx=0).stop 2024-06-10T01:22:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.637+0000 7ff53b243740 1 -- 172.21.15.7:0/2155254070 shutdown_connections 2024-06-10T01:22:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.637+0000 7ff53b243740 1 --2- 172.21.15.7:0/2155254070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f35532380 0x557f35534810 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.637+0000 7ff53b243740 1 --2- 172.21.15.7:0/2155254070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f3552d210 0x557f35531d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.637+0000 7ff53b243740 1 --2- 172.21.15.7:0/2155254070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f3552c820 0x557f3552cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.637+0000 7ff53b243740 1 -- 172.21.15.7:0/2155254070 >> 172.21.15.7:0/2155254070 conn(0x557f35527e90 msgr2=0x557f35528290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.637+0000 7ff53b243740 1 -- 172.21.15.7:0/2155254070 shutdown_connections 2024-06-10T01:22:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.637+0000 7ff53b243740 1 -- 172.21.15.7:0/2155254070 wait complete. 2024-06-10T01:22:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.637+0000 7ff53b243740 1 -- start start 2024-06-10T01:22:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.638+0000 7ff53b243740 1 Processor -- start 2024-06-10T01:22:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.638+0000 7ff53b243740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f3552c820 0x557f355369a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.638+0000 7ff53b243740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f3552d210 0x557f35536ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.638+0000 7ff53b243740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f35532380 0x557f3553df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.638+0000 7ff53b243740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557f355352d0 con 0x557f3552c820 2024-06-10T01:22:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.638+0000 7ff53b243740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557f35410ae0 con 0x557f3552d210 2024-06-10T01:22:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.638+0000 7ff53b243740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557f35535450 con 0x557f35532380 2024-06-10T01:22:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.638+0000 7ff53b243740 10 client.0 _ll_register_callbacks cb 0x557f354499b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:22:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.638+0000 7ff535acf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f3552c820 0x557f355369a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.638+0000 7ff535acf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f3552c820 0x557f355369a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57484/0 (socket says 172.21.15.7:57484) 2024-06-10T01:22:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.638+0000 7ff535acf640 1 -- 172.21.15.7:0/3619088572 learned_addr learned my addr 172.21.15.7:0/3619088572 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.638+0000 7ff5362d0640 1 --2- 172.21.15.7:0/3619088572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f3552d210 0x557f35536ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.638+0000 7ff535acf640 1 --2- 172.21.15.7:0/3619088572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f35532380 0x557f3553df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.639+0000 7ff535acf640 1 -- 172.21.15.7:0/3619088572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f35532380 msgr2=0x557f3553df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.639+0000 7ff535acf640 1 --2- 172.21.15.7:0/3619088572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f35532380 0x557f3553df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.639+0000 7ff535acf640 1 -- 172.21.15.7:0/3619088572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f3552d210 msgr2=0x557f35536ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.639+0000 7ff535acf640 1 --2- 172.21.15.7:0/3619088572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f3552d210 0x557f35536ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.639+0000 7ff535acf640 1 -- 172.21.15.7:0/3619088572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f3519b210 con 0x557f3552c820 2024-06-10T01:22:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.639+0000 7ff535acf640 1 --2- 172.21.15.7:0/3619088572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f3552c820 0x557f355369a0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7ff520018d80 tx=0x7ff5200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9450a3992308f8e7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.639+0000 7ff519ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:22:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.639+0000 7ff519ffb640 1 -- 172.21.15.7:0/3619088572 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff520013bc0 con 0x557f3552c820 2024-06-10T01:22:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.639+0000 7ff53b243740 1 -- 172.21.15.7:0/3619088572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff520018450 con 0x557f3552c820 2024-06-10T01:22:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.640+0000 7ff519ffb640 1 -- 172.21.15.7:0/3619088572 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7ff520013d60 con 0x557f3552c820 2024-06-10T01:22:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.640+0000 7ff519ffb640 1 -- 172.21.15.7:0/3619088572 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ff52002c510 con 0x557f3552c820 2024-06-10T01:22:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.640+0000 7ff53b243740 10 client.19515 fetch_fsmap learned FSMap version 46 2024-06-10T01:22:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.640+0000 7ff53b243740 1 -- 172.21.15.7:0/3619088572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x557f35267620 con 0x557f3552c820 2024-06-10T01:22:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.640+0000 7ff519ffb640 1 -- 172.21.15.7:0/3619088572 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff520018450 con 0x557f3552c820 2024-06-10T01:22:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.640+0000 7ff519ffb640 1 -- 172.21.15.7:0/3619088572 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 46) ==== 8354+0+0 (secure 0 0 0) 0x7ff520025c10 con 0x557f3552c820 2024-06-10T01:22:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.640+0000 7ff53b243740 10 client.19515 fetch_fsmap finished waiting for FSMap version 46 2024-06-10T01:22:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.640+0000 7ff53b243740 10 client.19515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:22:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.640+0000 7ff53b243740 20 client.19515 populate_metadata read hostname 'smithi007' 2024-06-10T01:22:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.640+0000 7ff53b243740 1 --2- 172.21.15.7:0/3619088572 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f35542370 0x557f355447a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.640+0000 7ff53b243740 4 client.19515 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:22:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.640+0000 7ff53b243740 1 -- 172.21.15.7:0/3619088572 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f3549af10 con 0x557f35542370 2024-06-10T01:22:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.641+0000 7ff5362d0640 1 --2- 172.21.15.7:0/3619088572 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f35542370 0x557f355447a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.641+0000 7ff5362d0640 1 --2- 172.21.15.7:0/3619088572 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f35542370 0x557f355447a0 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf1f9e9f0d529be8 server_cookie=9332861267389b61 in_seq=0 out_seq=0 2024-06-10T01:22:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.641+0000 7ff519ffb640 10 client.19515 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:28.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.643+0000 7ff519ffb640 1 -- 172.21.15.7:0/3619088572 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557f3549af10 con 0x557f35542370 2024-06-10T01:22:28.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.643+0000 7ff519ffb640 10 client.19515 handle_command_reply: tid=0 2024-06-10T01:22:28.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.643+0000 7ff53b243740 1 client.19515 shutdown 2024-06-10T01:22:28.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.643+0000 7ff53b243740 1 -- 172.21.15.7:0/3619088572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f3552c820 msgr2=0x557f355369a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:28.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.643+0000 7ff53b243740 1 --2- 172.21.15.7:0/3619088572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f3552c820 0x557f355369a0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7ff520018d80 tx=0x7ff5200046c0 comp rx=0 tx=0).stop 2024-06-10T01:22:28.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.643+0000 7ff53b243740 1 -- 172.21.15.7:0/3619088572 shutdown_connections 2024-06-10T01:22:28.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.643+0000 7ff53b243740 1 --2- 172.21.15.7:0/3619088572 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f35542370 0x557f355447a0 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:28.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.643+0000 7ff53b243740 1 --2- 172.21.15.7:0/3619088572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f35532380 0x557f3553df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:28.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.643+0000 7ff53b243740 1 --2- 172.21.15.7:0/3619088572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f3552d210 0x557f35536ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:28.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.643+0000 7ff53b243740 1 --2- 172.21.15.7:0/3619088572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f3552c820 0x557f355369a0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:28.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.643+0000 7ff519ffb640 0 client.19515 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:28.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.643+0000 7ff53b243740 1 -- 172.21.15.7:0/3619088572 >> 172.21.15.7:0/3619088572 conn(0x557f35527e90 msgr2=0x557f3551e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:28.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.643+0000 7ff53b243740 1 -- 172.21.15.7:0/3619088572 shutdown_connections 2024-06-10T01:22:28.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.643+0000 7ff53b243740 1 -- 172.21.15.7:0/3619088572 wait complete. 2024-06-10T01:22:28.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.644+0000 7ff53b243740 20 client.19515 trim_cache size 0 max 16384 2024-06-10T01:22:28.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.667+0000 7ff53b243740 1 Processor -- start 2024-06-10T01:22:28.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.667+0000 7ff53b243740 1 -- start start 2024-06-10T01:22:28.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.667+0000 7ff53b243740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f3552c820 0x557f355365b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:28.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.667+0000 7ff53b243740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f3552d210 0x557f3553dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:28.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.667+0000 7ff53b243740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f35532380 0x557f35536af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:28.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.667+0000 7ff53b243740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff514012440 con 0x557f3552c820 2024-06-10T01:22:28.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.667+0000 7ff53b243740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff51400fce0 con 0x557f3552d210 2024-06-10T01:22:28.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.667+0000 7ff53b243740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff51400db30 con 0x557f35532380 2024-06-10T01:22:28.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.667+0000 7ff535acf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f3552c820 0x557f355365b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:28.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.667+0000 7ff535acf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f3552c820 0x557f355365b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57486/0 (socket says 172.21.15.7:57486) 2024-06-10T01:22:28.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.667+0000 7ff535acf640 1 -- 172.21.15.7:0/2701081436 learned_addr learned my addr 172.21.15.7:0/2701081436 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:28.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.667+0000 7ff5362d0640 1 --2- 172.21.15.7:0/2701081436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f3552d210 0x557f3553dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:28.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.668+0000 7ff535acf640 1 --2- 172.21.15.7:0/2701081436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f35532380 0x557f35536af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:28.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.668+0000 7ff535acf640 1 -- 172.21.15.7:0/2701081436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f35532380 msgr2=0x557f35536af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:28.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.668+0000 7ff535acf640 1 --2- 172.21.15.7:0/2701081436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f35532380 0x557f35536af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:28.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.668+0000 7ff535acf640 1 -- 172.21.15.7:0/2701081436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f3552d210 msgr2=0x557f3553dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:28.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.668+0000 7ff535acf640 1 --2- 172.21.15.7:0/2701081436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f3552d210 0x557f3553dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:28.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.668+0000 7ff535acf640 1 -- 172.21.15.7:0/2701081436 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f35527cc0 con 0x557f3552c820 2024-06-10T01:22:28.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.668+0000 7ff535acf640 1 --2- 172.21.15.7:0/2701081436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f3552c820 0x557f355365b0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7ff520009930 tx=0x7ff5200047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de77ac500bc87575 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:28.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.668+0000 7ff51bfff640 1 -- 172.21.15.7:0/2701081436 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff520009bc0 con 0x557f3552c820 2024-06-10T01:22:28.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.668+0000 7ff51bfff640 1 -- 172.21.15.7:0/2701081436 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff52002c0c0 con 0x557f3552c820 2024-06-10T01:22:28.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.668+0000 7ff51bfff640 1 -- 172.21.15.7:0/2701081436 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff520013e20 con 0x557f3552c820 2024-06-10T01:22:28.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.669+0000 7ff53b243740 1 -- 172.21.15.7:0/2701081436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f3552c820 msgr2=0x557f355365b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:28.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.669+0000 7ff53b243740 1 --2- 172.21.15.7:0/2701081436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f3552c820 0x557f355365b0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7ff520009930 tx=0x7ff5200047d0 comp rx=0 tx=0).stop 2024-06-10T01:22:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.669+0000 7ff53b243740 1 -- 172.21.15.7:0/2701081436 shutdown_connections 2024-06-10T01:22:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.669+0000 7ff53b243740 1 --2- 172.21.15.7:0/2701081436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f35532380 0x557f35536af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.669+0000 7ff53b243740 1 --2- 172.21.15.7:0/2701081436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f3552d210 0x557f3553dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.669+0000 7ff53b243740 1 --2- 172.21.15.7:0/2701081436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f3552c820 0x557f355365b0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.669+0000 7ff53b243740 1 -- 172.21.15.7:0/2701081436 >> 172.21.15.7:0/2701081436 conn(0x557f35527e90 msgr2=0x557f35528270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.669+0000 7ff53b243740 1 -- 172.21.15.7:0/2701081436 shutdown_connections 2024-06-10T01:22:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.669+0000 7ff53b243740 1 -- 172.21.15.7:0/2701081436 wait complete. 2024-06-10T01:22:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.669+0000 7ff53b243740 1 -- start start 2024-06-10T01:22:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.669+0000 7ff53b243740 1 Processor -- start 2024-06-10T01:22:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.670+0000 7ff53b243740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f3552c820 0x557f3553aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.670+0000 7ff53b243740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f3552d210 0x557f3553b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.670+0000 7ff53b243740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f35532380 0x557f355ab030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.670+0000 7ff53b243740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff51400e1a0 con 0x557f3552c820 2024-06-10T01:22:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.670+0000 7ff53b243740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff51400f610 con 0x557f3552d210 2024-06-10T01:22:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.670+0000 7ff53b243740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff51400e870 con 0x557f35532380 2024-06-10T01:22:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.670+0000 7ff535acf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f3552c820 0x557f3553aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.670+0000 7ff535acf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f3552c820 0x557f3553aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57494/0 (socket says 172.21.15.7:57494) 2024-06-10T01:22:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.670+0000 7ff535acf640 1 -- 172.21.15.7:0/3605540101 learned_addr learned my addr 172.21.15.7:0/3605540101 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.670+0000 7ff53b243740 10 client.0 _ll_register_callbacks cb 0x7ff5200153a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:22:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.670+0000 7ff535acf640 1 --2- 172.21.15.7:0/3605540101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f35532380 0x557f355ab030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.670+0000 7ff5362d0640 1 --2- 172.21.15.7:0/3605540101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f3552d210 0x557f3553b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.670+0000 7ff535acf640 1 -- 172.21.15.7:0/3605540101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f35532380 msgr2=0x557f355ab030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.670+0000 7ff535acf640 1 --2- 172.21.15.7:0/3605540101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f35532380 0x557f355ab030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.670+0000 7ff535acf640 1 -- 172.21.15.7:0/3605540101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f3552d210 msgr2=0x557f3553b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.670+0000 7ff535acf640 1 --2- 172.21.15.7:0/3605540101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f3552d210 0x557f3553b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.670+0000 7ff535acf640 1 -- 172.21.15.7:0/3605540101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f3553e0d0 con 0x557f3552c820 2024-06-10T01:22:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.671+0000 7ff535acf640 1 --2- 172.21.15.7:0/3605540101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f3552c820 0x557f3553aac0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7ff5200188d0 tx=0x7ff520009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf68954538d0ef5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.671+0000 7ff519ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:22:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.671+0000 7ff519ffb640 1 -- 172.21.15.7:0/3605540101 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff520018450 con 0x557f3552c820 2024-06-10T01:22:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.671+0000 7ff519ffb640 1 -- 172.21.15.7:0/3605540101 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7ff5200059c0 con 0x557f3552c820 2024-06-10T01:22:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.671+0000 7ff53b243740 1 -- 172.21.15.7:0/3605540101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff52002c0c0 con 0x557f3552c820 2024-06-10T01:22:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.671+0000 7ff519ffb640 1 -- 172.21.15.7:0/3605540101 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ff520005be0 con 0x557f3552c820 2024-06-10T01:22:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.671+0000 7ff53b243740 10 client.19521 fetch_fsmap learned FSMap version 46 2024-06-10T01:22:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.671+0000 7ff53b243740 1 -- 172.21.15.7:0/3605540101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x557f355ac510 con 0x557f3552c820 2024-06-10T01:22:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.671+0000 7ff519ffb640 1 -- 172.21.15.7:0/3605540101 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff52002c0c0 con 0x557f3552c820 2024-06-10T01:22:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.672+0000 7ff519ffb640 1 -- 172.21.15.7:0/3605540101 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 46) ==== 8354+0+0 (secure 0 0 0) 0x7ff520024770 con 0x557f3552c820 2024-06-10T01:22:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.672+0000 7ff53b243740 10 client.19521 fetch_fsmap finished waiting for FSMap version 46 2024-06-10T01:22:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.672+0000 7ff53b243740 10 client.19521 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:22:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.672+0000 7ff53b243740 20 client.19521 populate_metadata read hostname 'smithi007' 2024-06-10T01:22:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.672+0000 7ff53b243740 1 --2- 172.21.15.7:0/3605540101 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f35542370 0x557f355aed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.672+0000 7ff53b243740 4 client.19521 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:22:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.672+0000 7ff53b243740 1 -- 172.21.15.7:0/3605540101 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557f35494c20 con 0x557f35542370 2024-06-10T01:22:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.672+0000 7ff5362d0640 1 --2- 172.21.15.7:0/3605540101 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f35542370 0x557f355aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.672+0000 7ff5362d0640 1 --2- 172.21.15.7:0/3605540101 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f35542370 0x557f355aed10 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdc561fcdc36a093 server_cookie=40dd8e361fd206be in_seq=0 out_seq=0 2024-06-10T01:22:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:28.672+0000 7ff519ffb640 10 client.19521 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:28.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:28 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:22:28.831 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:28 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:22:28.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:28 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:22:28.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:28 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:22:29.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:28 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:22:29.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:28 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:22:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:29.278+0000 7ff519ffb640 1 -- 172.21.15.7:0/3605540101 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557f35494c20 con 0x557f35542370 2024-06-10T01:22:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:29.278+0000 7ff519ffb640 10 client.19521 handle_command_reply: tid=0 2024-06-10T01:22:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:29.278+0000 7ff53b243740 1 client.19521 shutdown 2024-06-10T01:22:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:29.279+0000 7ff53b243740 1 -- 172.21.15.7:0/3605540101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f3552c820 msgr2=0x557f3553aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:29.279+0000 7ff53b243740 1 --2- 172.21.15.7:0/3605540101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f3552c820 0x557f3553aac0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7ff5200188d0 tx=0x7ff520009dd0 comp rx=0 tx=0).stop 2024-06-10T01:22:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:29.279+0000 7ff53b243740 1 -- 172.21.15.7:0/3605540101 shutdown_connections 2024-06-10T01:22:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:29.279+0000 7ff53b243740 1 --2- 172.21.15.7:0/3605540101 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f35542370 0x557f355aed10 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:29.279+0000 7ff519ffb640 0 client.19521 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:29.279+0000 7ff53b243740 1 --2- 172.21.15.7:0/3605540101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f35532380 0x557f355ab030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:29.279+0000 7ff53b243740 1 --2- 172.21.15.7:0/3605540101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f3552d210 0x557f3553b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:29.280+0000 7ff53b243740 1 --2- 172.21.15.7:0/3605540101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f3552c820 0x557f3553aac0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:29.280+0000 7ff53b243740 1 -- 172.21.15.7:0/3605540101 >> 172.21.15.7:0/3605540101 conn(0x557f35527e90 msgr2=0x557f355365b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:29.280+0000 7ff53b243740 1 -- 172.21.15.7:0/3605540101 shutdown_connections 2024-06-10T01:22:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:29.280+0000 7ff53b243740 1 -- 172.21.15.7:0/3605540101 wait complete. 2024-06-10T01:22:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:29.280+0000 7ff53b243740 20 client.19521 trim_cache size 0 max 16384 2024-06-10T01:22:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:22:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:22:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:22:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:29.281+0000 7ff538534640 1 -- 172.21.15.7:0/1602810159 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff51c0a2cd0 msgr2=0x7ff51c0a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:29.281+0000 7ff538534640 1 --2- 172.21.15.7:0/1602810159 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff51c0a2cd0 0x7ff51c0a5190 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff52809bba0 tx=0x7ff52800cd50 comp rx=0 tx=0).stop 2024-06-10T01:22:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:29.281+0000 7ff538534640 1 -- 172.21.15.7:0/1602810159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5301076e0 msgr2=0x7ff530100970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:29.281+0000 7ff538534640 1 --2- 172.21.15.7:0/1602810159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5301076e0 0x7ff530100970 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7ff52800efd0 tx=0x7ff52800c6a0 comp rx=0 tx=0).stop 2024-06-10T01:22:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:29.281+0000 7ff538534640 1 -- 172.21.15.7:0/1602810159 shutdown_connections 2024-06-10T01:22:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:29.281+0000 7ff538534640 1 --2- 172.21.15.7:0/1602810159 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff51c0a2cd0 0x7ff51c0a5190 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:29.281+0000 7ff538534640 1 --2- 172.21.15.7:0/1602810159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5301076e0 0x7ff530100970 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:29.281+0000 7ff538534640 1 --2- 172.21.15.7:0/1602810159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff530104db0 0x7ff530100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:29.281+0000 7ff538534640 1 --2- 172.21.15.7:0/1602810159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff530102480 0x7ff530101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:29.281+0000 7ff538534640 1 -- 172.21.15.7:0/1602810159 >> 172.21.15.7:0/1602810159 conn(0x7ff5300fbe90 msgr2=0x7ff530102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:29.281+0000 7ff538534640 1 -- 172.21.15.7:0/1602810159 shutdown_connections 2024-06-10T01:22:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:29.281+0000 7ff538534640 1 -- 172.21.15.7:0/1602810159 wait complete. 2024-06-10T01:22:29.523 INFO:tasks.workunit.client.0.smithi007.stdout: 10 81342 33.27 MB/sec execute 66 sec latency 419.043 ms 2024-06-10T01:22:29.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:29 smithi078 ceph-mon[36683]: pgmap v926: 129 pgs: 129 active+clean; 552 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.8 MiB/s wr, 743 op/s 2024-06-10T01:22:29.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:29 smithi175 ceph-mon[36636]: pgmap v926: 129 pgs: 129 active+clean; 552 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.8 MiB/s wr, 743 op/s 2024-06-10T01:22:30.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:29 smithi007 ceph-mon[32398]: pgmap v926: 129 pgs: 129 active+clean; 552 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.8 MiB/s wr, 743 op/s 2024-06-10T01:22:30.523 INFO:tasks.workunit.client.0.smithi007.stdout: 10 82000 33.20 MB/sec execute 67 sec latency 145.761 ms 2024-06-10T01:22:31.523 INFO:tasks.workunit.client.0.smithi007.stdout: 10 82816 33.55 MB/sec execute 68 sec latency 73.327 ms 2024-06-10T01:22:31.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:31 smithi078 ceph-mon[36683]: pgmap v927: 129 pgs: 129 active+clean; 560 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 930 op/s 2024-06-10T01:22:31.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:31 smithi175 ceph-mon[36636]: pgmap v927: 129 pgs: 129 active+clean; 560 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 930 op/s 2024-06-10T01:22:32.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:31 smithi007 ceph-mon[32398]: pgmap v927: 129 pgs: 129 active+clean; 560 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 930 op/s 2024-06-10T01:22:32.523 INFO:tasks.workunit.client.0.smithi007.stdout: 10 83419 33.44 MB/sec execute 69 sec latency 90.591 ms 2024-06-10T01:22:33.523 INFO:tasks.workunit.client.0.smithi007.stdout: 10 84019 33.33 MB/sec execute 70 sec latency 267.068 ms 2024-06-10T01:22:33.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:33 smithi078 ceph-mon[36683]: pgmap v928: 129 pgs: 129 active+clean; 563 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.3 MiB/s wr, 705 op/s 2024-06-10T01:22:33.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:33 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:22:33.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:33 smithi175 ceph-mon[36636]: pgmap v928: 129 pgs: 129 active+clean; 563 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.3 MiB/s wr, 705 op/s 2024-06-10T01:22:33.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:33 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:22:34.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:33 smithi007 ceph-mon[32398]: pgmap v928: 129 pgs: 129 active+clean; 563 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.3 MiB/s wr, 705 op/s 2024-06-10T01:22:34.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:33 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:22:34.523 INFO:tasks.workunit.client.0.smithi007.stdout: 10 84614 33.34 MB/sec execute 71 sec latency 166.820 ms 2024-06-10T01:22:35.524 INFO:tasks.workunit.client.0.smithi007.stdout: 10 85515 33.51 MB/sec execute 72 sec latency 186.851 ms 2024-06-10T01:22:35.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:35 smithi078 ceph-mon[36683]: pgmap v929: 129 pgs: 129 active+clean; 560 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.2 MiB/s wr, 716 op/s 2024-06-10T01:22:35.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:35 smithi175 ceph-mon[36636]: pgmap v929: 129 pgs: 129 active+clean; 560 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.2 MiB/s wr, 716 op/s 2024-06-10T01:22:36.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:35 smithi007 ceph-mon[32398]: pgmap v929: 129 pgs: 129 active+clean; 560 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.2 MiB/s wr, 716 op/s 2024-06-10T01:22:36.523 INFO:tasks.workunit.client.0.smithi007.stdout: 10 86253 33.77 MB/sec execute 73 sec latency 168.802 ms 2024-06-10T01:22:37.523 INFO:tasks.workunit.client.0.smithi007.stdout: 10 86836 33.68 MB/sec execute 74 sec latency 72.866 ms 2024-06-10T01:22:37.814 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:37 smithi007 ceph-mon[32398]: pgmap v930: 129 pgs: 129 active+clean; 570 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 8.6 MiB/s wr, 764 op/s 2024-06-10T01:22:37.829 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:37 smithi078 ceph-mon[36683]: pgmap v930: 129 pgs: 129 active+clean; 570 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 8.6 MiB/s wr, 764 op/s 2024-06-10T01:22:37.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:37 smithi175 ceph-mon[36636]: pgmap v930: 129 pgs: 129 active+clean; 570 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 8.6 MiB/s wr, 764 op/s 2024-06-10T01:22:38.524 INFO:tasks.workunit.client.0.smithi007.stdout: 10 87343 33.51 MB/sec execute 75 sec latency 249.904 ms 2024-06-10T01:22:38.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:38 smithi175 ceph-mon[36636]: pgmap v931: 129 pgs: 129 active+clean; 571 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 8.2 MiB/s wr, 741 op/s 2024-06-10T01:22:39.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:38 smithi007 ceph-mon[32398]: pgmap v931: 129 pgs: 129 active+clean; 571 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 8.2 MiB/s wr, 741 op/s 2024-06-10T01:22:39.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:38 smithi078 ceph-mon[36683]: pgmap v931: 129 pgs: 129 active+clean; 571 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 8.2 MiB/s wr, 741 op/s 2024-06-10T01:22:39.213 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-06-10T01:22:39.213 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-06-10T01:22:39.214 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:22:39.525 INFO:tasks.workunit.client.0.smithi007.stdout: 10 87779 33.33 MB/sec execute 76 sec latency 418.943 ms 2024-06-10T01:22:39.726 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:22:40.525 INFO:tasks.workunit.client.0.smithi007.stdout: 10 88271 33.34 MB/sec execute 77 sec latency 167.739 ms 2024-06-10T01:22:41.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:40 smithi175 ceph-mon[36636]: pgmap v932: 129 pgs: 129 active+clean; 580 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 862 op/s 2024-06-10T01:22:41.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:40 smithi007 ceph-mon[32398]: pgmap v932: 129 pgs: 129 active+clean; 580 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 862 op/s 2024-06-10T01:22:41.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:40 smithi078 ceph-mon[36683]: pgmap v932: 129 pgs: 129 active+clean; 580 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 862 op/s 2024-06-10T01:22:41.525 INFO:tasks.workunit.client.0.smithi007.stdout: 10 88997 33.41 MB/sec execute 78 sec latency 101.369 ms 2024-06-10T01:22:41.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.722+0000 7f2dd8e90700 1 -- 172.21.15.7:0/2986934426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dd41084e0 msgr2=0x7f2dd41088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:41.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.722+0000 7f2dd8e90700 1 --2- 172.21.15.7:0/2986934426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dd41084e0 0x7f2dd41088c0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc013960 tx=0x7f2dcc013c70 comp rx=0 tx=0).stop 2024-06-10T01:22:41.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.723+0000 7f2dd8e90700 1 -- 172.21.15.7:0/2986934426 shutdown_connections 2024-06-10T01:22:41.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.723+0000 7f2dd8e90700 1 --2- 172.21.15.7:0/2986934426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dd406ab10 0x7f2dd410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:41.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.723+0000 7f2dd8e90700 1 --2- 172.21.15.7:0/2986934426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dd41084e0 0x7f2dd41088c0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:41.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.723+0000 7f2dd8e90700 1 --2- 172.21.15.7:0/2986934426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dd406a170 0x7f2dd406a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:41.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.723+0000 7f2dd8e90700 1 -- 172.21.15.7:0/2986934426 >> 172.21.15.7:0/2986934426 conn(0x7f2dd40782c0 msgr2=0x7f2dd40786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:41.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.723+0000 7f2dd8e90700 1 -- 172.21.15.7:0/2986934426 shutdown_connections 2024-06-10T01:22:41.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.723+0000 7f2dd8e90700 1 -- 172.21.15.7:0/2986934426 wait complete. 2024-06-10T01:22:41.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.724+0000 7f2dd8e90700 1 Processor -- start 2024-06-10T01:22:41.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.724+0000 7f2dd8e90700 1 -- start start 2024-06-10T01:22:41.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.725+0000 7f2dd8e90700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dd406a170 0x7f2dd419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:41.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.725+0000 7f2dd8e90700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dd406ab10 0x7f2dd419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:41.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.725+0000 7f2dd8e90700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dd41084e0 0x7f2dd4196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:41.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.725+0000 7f2dd8e90700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd4110660 con 0x7f2dd41084e0 2024-06-10T01:22:41.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.725+0000 7f2dd8e90700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd41104e0 con 0x7f2dd406ab10 2024-06-10T01:22:41.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.725+0000 7f2dd8e90700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd41107e0 con 0x7f2dd406a170 2024-06-10T01:22:41.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.726+0000 7f2dd2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dd41084e0 0x7f2dd4196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:41.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.726+0000 7f2dd2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dd41084e0 0x7f2dd4196840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40006/0 (socket says 172.21.15.7:40006) 2024-06-10T01:22:41.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.726+0000 7f2dd2d9d700 1 -- 172.21.15.7:0/2193671918 learned_addr learned my addr 172.21.15.7:0/2193671918 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:41.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.726+0000 7f2dd1d9b700 1 --2- 172.21.15.7:0/2193671918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dd406ab10 0x7f2dd419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:41.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.726+0000 7f2dd259c700 1 --2- 172.21.15.7:0/2193671918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dd406a170 0x7f2dd419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:41.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.726+0000 7f2dd2d9d700 1 -- 172.21.15.7:0/2193671918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dd406a170 msgr2=0x7f2dd419c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:41.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.726+0000 7f2dd2d9d700 1 --2- 172.21.15.7:0/2193671918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dd406a170 0x7f2dd419c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:41.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.726+0000 7f2dd2d9d700 1 -- 172.21.15.7:0/2193671918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dd406ab10 msgr2=0x7f2dd419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:41.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.726+0000 7f2dd2d9d700 1 --2- 172.21.15.7:0/2193671918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dd406ab10 0x7f2dd419cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:41.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.726+0000 7f2dd2d9d700 1 -- 172.21.15.7:0/2193671918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd4197020 con 0x7f2dd41084e0 2024-06-10T01:22:41.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.726+0000 7f2dd1d9b700 1 --2- 172.21.15.7:0/2193671918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dd406ab10 0x7f2dd419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:22:41.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.727+0000 7f2dd259c700 1 --2- 172.21.15.7:0/2193671918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dd406a170 0x7f2dd419c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:22:41.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.727+0000 7f2dd2d9d700 1 --2- 172.21.15.7:0/2193671918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dd41084e0 0x7f2dd4196840 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f2dc400e940 tx=0x7f2dc400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:41.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.727+0000 7f2dc37fe700 1 -- 172.21.15.7:0/2193671918 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc400cbd0 con 0x7f2dd41084e0 2024-06-10T01:22:41.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.728+0000 7f2dd8e90700 1 -- 172.21.15.7:0/2193671918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dd4197310 con 0x7f2dd41084e0 2024-06-10T01:22:41.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.728+0000 7f2dd8e90700 1 -- 172.21.15.7:0/2193671918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dd41a3520 con 0x7f2dd41084e0 2024-06-10T01:22:41.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.728+0000 7f2dc37fe700 1 -- 172.21.15.7:0/2193671918 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dc400cd70 con 0x7f2dd41084e0 2024-06-10T01:22:41.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.729+0000 7f2dc37fe700 1 -- 172.21.15.7:0/2193671918 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc400cbd0 con 0x7f2dd41084e0 2024-06-10T01:22:41.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.729+0000 7f2dd8e90700 1 -- 172.21.15.7:0/2193671918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dd4103b90 con 0x7f2dd41084e0 2024-06-10T01:22:41.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.732+0000 7f2dc37fe700 1 -- 172.21.15.7:0/2193671918 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f2dc4004da0 con 0x7f2dd41084e0 2024-06-10T01:22:41.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.733+0000 7f2dc37fe700 1 --2- 172.21.15.7:0/2193671918 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2dbc0a2e20 0x7f2dbc0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:41.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.734+0000 7f2dc37fe700 1 -- 172.21.15.7:0/2193671918 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f2dc40cf480 con 0x7f2dd41084e0 2024-06-10T01:22:41.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.734+0000 7f2dd259c700 1 --2- 172.21.15.7:0/2193671918 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2dbc0a2e20 0x7f2dbc0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:41.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.734+0000 7f2dd259c700 1 --2- 172.21.15.7:0/2193671918 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2dbc0a2e20 0x7f2dbc0a52e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc00a010 tx=0x7f2dcc009be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:41.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:41.737+0000 7f2dc37fe700 1 -- 172.21.15.7:0/2193671918 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2dc4053850 con 0x7f2dd41084e0 2024-06-10T01:22:42.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:42.109+0000 7f2dd8e90700 1 -- 172.21.15.7:0/2193671918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2dd402dca0 con 0x7f2dd41084e0 2024-06-10T01:22:42.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:42.115+0000 7f2dc37fe700 1 -- 172.21.15.7:0/2193671918 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11856 (secure 0 0 0) 0x7f2dc409c060 con 0x7f2dd41084e0 2024-06-10T01:22:42.111 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:22:42.111 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:22:32.264727+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:22:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:42.120+0000 7f2dd8e90700 1 -- 172.21.15.7:0/2193671918 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2dbc0a2e20 msgr2=0x7f2dbc0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:42.120+0000 7f2dd8e90700 1 --2- 172.21.15.7:0/2193671918 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2dbc0a2e20 0x7f2dbc0a52e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc00a010 tx=0x7f2dcc009be0 comp rx=0 tx=0).stop 2024-06-10T01:22:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:42.120+0000 7f2dd8e90700 1 -- 172.21.15.7:0/2193671918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dd41084e0 msgr2=0x7f2dd4196840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:42.120+0000 7f2dd8e90700 1 --2- 172.21.15.7:0/2193671918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dd41084e0 0x7f2dd4196840 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f2dc400e940 tx=0x7f2dc400ed00 comp rx=0 tx=0).stop 2024-06-10T01:22:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:42.120+0000 7f2dd8e90700 1 -- 172.21.15.7:0/2193671918 shutdown_connections 2024-06-10T01:22:42.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:42.120+0000 7f2dd8e90700 1 --2- 172.21.15.7:0/2193671918 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2dbc0a2e20 0x7f2dbc0a52e0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:42.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:42.120+0000 7f2dd8e90700 1 --2- 172.21.15.7:0/2193671918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dd41084e0 0x7f2dd4196840 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:42.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:42.121+0000 7f2dd8e90700 1 --2- 172.21.15.7:0/2193671918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dd406a170 0x7f2dd419c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:42.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:42.121+0000 7f2dd8e90700 1 --2- 172.21.15.7:0/2193671918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dd406ab10 0x7f2dd419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:42.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:42.121+0000 7f2dd8e90700 1 -- 172.21.15.7:0/2193671918 >> 172.21.15.7:0/2193671918 conn(0x7f2dd40782c0 msgr2=0x7f2dd4100260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:42.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:42.121+0000 7f2dd8e90700 1 -- 172.21.15.7:0/2193671918 shutdown_connections 2024-06-10T01:22:42.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:42.121+0000 7f2dd8e90700 1 -- 172.21.15.7:0/2193671918 wait complete. 2024-06-10T01:22:42.115 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 47 2024-06-10T01:22:42.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.446+0000 7f9a4e89a640 1 -- 172.21.15.7:0/3235156436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a48101b10 msgr2=0x7f9a48101f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:42.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.446+0000 7f9a4e89a640 1 --2- 172.21.15.7:0/3235156436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a48101b10 0x7f9a48101f90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9a38009900 tx=0x7f9a3802a420 comp rx=0 tx=0).stop 2024-06-10T01:22:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.448+0000 7f9a4e89a640 1 -- 172.21.15.7:0/3235156436 shutdown_connections 2024-06-10T01:22:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.448+0000 7f9a4e89a640 1 --2- 172.21.15.7:0/3235156436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a481024d0 0x7f9a48108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.448+0000 7f9a4e89a640 1 --2- 172.21.15.7:0/3235156436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a48101b10 0x7f9a48101f90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.448+0000 7f9a4e89a640 1 --2- 172.21.15.7:0/3235156436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a48100920 0x7f9a48100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.448+0000 7f9a4e89a640 1 -- 172.21.15.7:0/3235156436 >> 172.21.15.7:0/3235156436 conn(0x7f9a480fbeb0 msgr2=0x7f9a480fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.449+0000 7f9a4e89a640 1 -- 172.21.15.7:0/3235156436 shutdown_connections 2024-06-10T01:22:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.449+0000 7f9a4e89a640 1 -- 172.21.15.7:0/3235156436 wait complete. 2024-06-10T01:22:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.450+0000 7f9a4e89a640 1 Processor -- start 2024-06-10T01:22:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.450+0000 7f9a4e89a640 1 -- start start 2024-06-10T01:22:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.450+0000 7f9a4e89a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a48100920 0x7f9a48195bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.450+0000 7f9a4e89a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a48101b10 0x7f9a48196100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.450+0000 7f9a4e89a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a481024d0 0x7f9a4819d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.451+0000 7f9a4e89a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4810b550 con 0x7f9a48100920 2024-06-10T01:22:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.451+0000 7f9a4e89a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4810b3d0 con 0x7f9a481024d0 2024-06-10T01:22:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.451+0000 7f9a4e89a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4810b6d0 con 0x7f9a48101b10 2024-06-10T01:22:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.451+0000 7f9a47fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a48100920 0x7f9a48195bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.451+0000 7f9a47fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a48100920 0x7f9a48195bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38068/0 (socket says 172.21.15.7:38068) 2024-06-10T01:22:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.451+0000 7f9a47fff640 1 -- 172.21.15.7:0/327908091 learned_addr learned my addr 172.21.15.7:0/327908091 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.451+0000 7f9a3ffff640 1 --2- 172.21.15.7:0/327908091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a48101b10 0x7f9a48196100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.451+0000 7f9a4ce37640 1 --2- 172.21.15.7:0/327908091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a481024d0 0x7f9a4819d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.451+0000 7f9a3ffff640 1 -- 172.21.15.7:0/327908091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a481024d0 msgr2=0x7f9a4819d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.451+0000 7f9a3ffff640 1 --2- 172.21.15.7:0/327908091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a481024d0 0x7f9a4819d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.451+0000 7f9a3ffff640 1 -- 172.21.15.7:0/327908091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a48100920 msgr2=0x7f9a48195bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.451+0000 7f9a3ffff640 1 --2- 172.21.15.7:0/327908091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a48100920 0x7f9a48195bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.451+0000 7f9a3ffff640 1 -- 172.21.15.7:0/327908091 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a4819d7e0 con 0x7f9a48101b10 2024-06-10T01:22:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.451+0000 7f9a47fff640 1 --2- 172.21.15.7:0/327908091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a48100920 0x7f9a48195bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:22:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.451+0000 7f9a4ce37640 1 --2- 172.21.15.7:0/327908091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a481024d0 0x7f9a4819d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:22:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.452+0000 7f9a3ffff640 1 --2- 172.21.15.7:0/327908091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a48101b10 0x7f9a48196100 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9a3802a940 tx=0x7f9a38005000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.452+0000 7f9a45ffb640 1 -- 172.21.15.7:0/327908091 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3803f070 con 0x7f9a48101b10 2024-06-10T01:22:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.452+0000 7f9a45ffb640 1 -- 172.21.15.7:0/327908091 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a38005600 con 0x7f9a48101b10 2024-06-10T01:22:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.452+0000 7f9a4e89a640 1 -- 172.21.15.7:0/327908091 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a4819da70 con 0x7f9a48101b10 2024-06-10T01:22:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.452+0000 7f9a45ffb640 1 -- 172.21.15.7:0/327908091 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a38035030 con 0x7f9a48101b10 2024-06-10T01:22:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.452+0000 7f9a4e89a640 1 -- 172.21.15.7:0/327908091 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a4819df30 con 0x7f9a48101b10 2024-06-10T01:22:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.455+0000 7f9a45ffb640 1 -- 172.21.15.7:0/327908091 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9a38042070 con 0x7f9a48101b10 2024-06-10T01:22:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.455+0000 7f9a515a1740 1 Processor -- start 2024-06-10T01:22:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.455+0000 7f9a515a1740 1 -- start start 2024-06-10T01:22:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.455+0000 7f9a515a1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d36e6fd820 0x55d36e6fdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.455+0000 7f9a515a1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d36e6fe210 0x55d36e702d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.455+0000 7f9a515a1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d36e703380 0x55d36e705810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.455+0000 7f9a515a1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d36e6584f0 con 0x55d36e703380 2024-06-10T01:22:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.455+0000 7f9a515a1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d36e5f6870 con 0x55d36e6fd820 2024-06-10T01:22:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.455+0000 7f9a515a1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d36e6544b0 con 0x55d36e6fe210 2024-06-10T01:22:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.456+0000 7f9a47fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d36e6fe210 0x55d36e702d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.456+0000 7f9a47fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d36e6fe210 0x55d36e702d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:54176/0 (socket says 172.21.15.7:54176) 2024-06-10T01:22:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.456+0000 7f9a47fff640 1 -- 172.21.15.7:0/2696434646 learned_addr learned my addr 172.21.15.7:0/2696434646 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.456+0000 7f9a3ffff640 1 --2- 172.21.15.7:0/2696434646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d36e703380 0x55d36e705810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.456+0000 7f9a4ce37640 1 --2- 172.21.15.7:0/2696434646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d36e6fd820 0x55d36e6fdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.456+0000 7f9a4ce37640 1 -- 172.21.15.7:0/2696434646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d36e6fe210 msgr2=0x55d36e702d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.456+0000 7f9a4ce37640 1 --2- 172.21.15.7:0/2696434646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d36e6fe210 0x55d36e702d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.456+0000 7f9a4ce37640 1 -- 172.21.15.7:0/2696434646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d36e703380 msgr2=0x55d36e705810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.456+0000 7f9a4ce37640 1 --2- 172.21.15.7:0/2696434646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d36e703380 0x55d36e705810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.456+0000 7f9a4ce37640 1 -- 172.21.15.7:0/2696434646 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d36e6129b0 con 0x55d36e6fd820 2024-06-10T01:22:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.456+0000 7f9a47fff640 1 --2- 172.21.15.7:0/2696434646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d36e6fe210 0x55d36e702d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:22:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.456+0000 7f9a4ce37640 1 --2- 172.21.15.7:0/2696434646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d36e6fd820 0x55d36e6fdc40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9a40010690 tx=0x7f9a40010b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15bed0186da5088a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.456+0000 7f9a45ffb640 1 --2- 172.21.15.7:0/327908091 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9a340a2cd0 0x7f9a340a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.456+0000 7f9a3e7fc640 1 -- 172.21.15.7:0/2696434646 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a40004b20 con 0x55d36e6fd820 2024-06-10T01:22:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.457+0000 7f9a3e7fc640 1 -- 172.21.15.7:0/2696434646 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a4000edb0 con 0x55d36e6fd820 2024-06-10T01:22:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.457+0000 7f9a3e7fc640 1 -- 172.21.15.7:0/2696434646 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a400057f0 con 0x55d36e6fd820 2024-06-10T01:22:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.457+0000 7f9a47fff640 1 --2- 172.21.15.7:0/327908091 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9a340a2cd0 0x7f9a340a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.457+0000 7f9a45ffb640 1 -- 172.21.15.7:0/327908091 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f9a380efb40 con 0x7f9a48101b10 2024-06-10T01:22:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.457+0000 7f9a515a1740 1 -- 172.21.15.7:0/2696434646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d36e6fd820 msgr2=0x55d36e6fdc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.457+0000 7f9a47fff640 1 --2- 172.21.15.7:0/327908091 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9a340a2cd0 0x7f9a340a5190 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9a30009fd0 tx=0x7f9a30009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.457+0000 7f9a515a1740 1 --2- 172.21.15.7:0/2696434646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d36e6fd820 0x55d36e6fdc40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9a40010690 tx=0x7f9a40010b50 comp rx=0 tx=0).stop 2024-06-10T01:22:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.457+0000 7f9a515a1740 1 -- 172.21.15.7:0/2696434646 shutdown_connections 2024-06-10T01:22:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.458+0000 7f9a515a1740 1 --2- 172.21.15.7:0/2696434646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d36e703380 0x55d36e705810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.458+0000 7f9a515a1740 1 --2- 172.21.15.7:0/2696434646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d36e6fe210 0x55d36e702d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.458+0000 7f9a515a1740 1 --2- 172.21.15.7:0/2696434646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d36e6fd820 0x55d36e6fdc40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.458+0000 7f9a515a1740 1 -- 172.21.15.7:0/2696434646 >> 172.21.15.7:0/2696434646 conn(0x55d36e6f8e90 msgr2=0x55d36e6f9290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.458+0000 7f9a515a1740 1 -- 172.21.15.7:0/2696434646 shutdown_connections 2024-06-10T01:22:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.458+0000 7f9a515a1740 1 -- 172.21.15.7:0/2696434646 wait complete. 2024-06-10T01:22:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.458+0000 7f9a515a1740 1 -- start start 2024-06-10T01:22:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.458+0000 7f9a515a1740 1 Processor -- start 2024-06-10T01:22:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.458+0000 7f9a515a1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d36e6fd820 0x55d36e707af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.458+0000 7f9a515a1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d36e6fe210 0x55d36e708030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.458+0000 7f9a515a1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d36e703380 0x55d36e70f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.458+0000 7f9a515a1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d36e706330 con 0x55d36e6fd820 2024-06-10T01:22:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.458+0000 7f9a515a1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d36e5e1ae0 con 0x55d36e703380 2024-06-10T01:22:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.458+0000 7f9a515a1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d36e7064b0 con 0x55d36e6fe210 2024-06-10T01:22:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.458+0000 7f9a515a1740 10 client.0 _ll_register_callbacks cb 0x55d36e61a9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:22:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.459+0000 7f9a47fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d36e6fd820 0x55d36e707af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.459+0000 7f9a3ffff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d36e6fe210 0x55d36e708030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.459+0000 7f9a47fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d36e6fd820 0x55d36e707af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38084/0 (socket says 172.21.15.7:38084) 2024-06-10T01:22:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.459+0000 7f9a47fff640 1 -- 172.21.15.7:0/2580984143 learned_addr learned my addr 172.21.15.7:0/2580984143 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.459+0000 7f9a4ce37640 1 --2- 172.21.15.7:0/2580984143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d36e703380 0x55d36e70f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.459+0000 7f9a47fff640 1 -- 172.21.15.7:0/2580984143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d36e6fe210 msgr2=0x55d36e708030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.459+0000 7f9a47fff640 1 --2- 172.21.15.7:0/2580984143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d36e6fe210 0x55d36e708030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.459+0000 7f9a47fff640 1 -- 172.21.15.7:0/2580984143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d36e703380 msgr2=0x55d36e70f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.459+0000 7f9a47fff640 1 --2- 172.21.15.7:0/2580984143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d36e703380 0x55d36e70f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.459+0000 7f9a47fff640 1 -- 172.21.15.7:0/2580984143 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d36e36c210 con 0x55d36e6fd820 2024-06-10T01:22:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.459+0000 7f9a47fff640 1 --2- 172.21.15.7:0/2580984143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d36e6fd820 0x55d36e707af0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f9a30012010 tx=0x7f9a300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c80e8061415035 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.459+0000 7f9a277fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:22:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.459+0000 7f9a277fe640 1 -- 172.21.15.7:0/2580984143 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a30025070 con 0x55d36e6fd820 2024-06-10T01:22:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.459+0000 7f9a277fe640 1 -- 172.21.15.7:0/2580984143 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f9a30014b50 con 0x55d36e6fd820 2024-06-10T01:22:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.459+0000 7f9a515a1740 1 -- 172.21.15.7:0/2580984143 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a4000edb0 con 0x55d36e6fd820 2024-06-10T01:22:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.459+0000 7f9a277fe640 1 -- 172.21.15.7:0/2580984143 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f9a30031050 con 0x55d36e6fd820 2024-06-10T01:22:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.459+0000 7f9a515a1740 10 client.19536 fetch_fsmap learned FSMap version 47 2024-06-10T01:22:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.460+0000 7f9a515a1740 1 -- 172.21.15.7:0/2580984143 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55d36e438620 con 0x55d36e6fd820 2024-06-10T01:22:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.460+0000 7f9a277fe640 1 -- 172.21.15.7:0/2580984143 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a4000edb0 con 0x55d36e6fd820 2024-06-10T01:22:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.460+0000 7f9a277fe640 1 -- 172.21.15.7:0/2580984143 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 47) ==== 8350+0+0 (secure 0 0 0) 0x7f9a30020070 con 0x55d36e6fd820 2024-06-10T01:22:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.460+0000 7f9a515a1740 10 client.19536 fetch_fsmap finished waiting for FSMap version 47 2024-06-10T01:22:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.460+0000 7f9a515a1740 10 client.19536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:22:42.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.460+0000 7f9a515a1740 20 client.19536 populate_metadata read hostname 'smithi007' 2024-06-10T01:22:42.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.460+0000 7f9a515a1740 1 --2- 172.21.15.7:0/2580984143 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d36e7137b0 0x55d36e715bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:42.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.460+0000 7f9a515a1740 4 client.19536 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:22:42.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.460+0000 7f9a515a1740 1 -- 172.21.15.7:0/2580984143 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d36e66bf10 con 0x55d36e7137b0 2024-06-10T01:22:42.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.460+0000 7f9a3ffff640 1 --2- 172.21.15.7:0/2580984143 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d36e7137b0 0x55d36e715bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:42.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.460+0000 7f9a277fe640 10 client.19536 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:42.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.460+0000 7f9a3ffff640 1 --2- 172.21.15.7:0/2580984143 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d36e7137b0 0x55d36e715bc0 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2dd03b38ccb552b server_cookie=f2666c73fb1e804f in_seq=0 out_seq=0 2024-06-10T01:22:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.462+0000 7f9a277fe640 1 -- 172.21.15.7:0/2580984143 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d36e66bf10 con 0x55d36e7137b0 2024-06-10T01:22:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.462+0000 7f9a277fe640 10 client.19536 handle_command_reply: tid=0 2024-06-10T01:22:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.462+0000 7f9a515a1740 1 client.19536 shutdown 2024-06-10T01:22:42.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.462+0000 7f9a515a1740 1 -- 172.21.15.7:0/2580984143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d36e6fd820 msgr2=0x55d36e707af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:42.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.462+0000 7f9a515a1740 1 --2- 172.21.15.7:0/2580984143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d36e6fd820 0x55d36e707af0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f9a30012010 tx=0x7f9a300046c0 comp rx=0 tx=0).stop 2024-06-10T01:22:42.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.462+0000 7f9a515a1740 1 -- 172.21.15.7:0/2580984143 shutdown_connections 2024-06-10T01:22:42.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.462+0000 7f9a515a1740 1 --2- 172.21.15.7:0/2580984143 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d36e7137b0 0x55d36e715bc0 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:42.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.462+0000 7f9a515a1740 1 --2- 172.21.15.7:0/2580984143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d36e703380 0x55d36e70f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:42.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.462+0000 7f9a277fe640 0 client.19536 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:42.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.462+0000 7f9a515a1740 1 --2- 172.21.15.7:0/2580984143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d36e6fe210 0x55d36e708030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:42.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.462+0000 7f9a515a1740 1 --2- 172.21.15.7:0/2580984143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d36e6fd820 0x55d36e707af0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:42.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.462+0000 7f9a515a1740 1 -- 172.21.15.7:0/2580984143 >> 172.21.15.7:0/2580984143 conn(0x55d36e6f8e90 msgr2=0x55d36e6ef7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:42.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.462+0000 7f9a515a1740 1 -- 172.21.15.7:0/2580984143 shutdown_connections 2024-06-10T01:22:42.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.462+0000 7f9a515a1740 1 -- 172.21.15.7:0/2580984143 wait complete. 2024-06-10T01:22:42.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.462+0000 7f9a515a1740 20 client.19536 trim_cache size 0 max 16384 2024-06-10T01:22:42.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.478+0000 7f9a515a1740 1 Processor -- start 2024-06-10T01:22:42.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.478+0000 7f9a515a1740 1 -- start start 2024-06-10T01:22:42.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.478+0000 7f9a515a1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d36e6fd820 0x55d36e707700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:42.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.478+0000 7f9a515a1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d36e6fe210 0x55d36e70f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.478+0000 7f9a515a1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d36e703380 0x55d36e707c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.478+0000 7f9a515a1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9a481aac20 con 0x55d36e703380 2024-06-10T01:22:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.478+0000 7f9a515a1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9a481a84f0 con 0x55d36e6fd820 2024-06-10T01:22:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.478+0000 7f9a515a1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9a481a6340 con 0x55d36e6fe210 2024-06-10T01:22:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.478+0000 7f9a4ce37640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d36e703380 0x55d36e707c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.478+0000 7f9a4ce37640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d36e703380 0x55d36e707c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38092/0 (socket says 172.21.15.7:38092) 2024-06-10T01:22:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.478+0000 7f9a4ce37640 1 -- 172.21.15.7:0/897980561 learned_addr learned my addr 172.21.15.7:0/897980561 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.479+0000 7f9a47fff640 1 --2- 172.21.15.7:0/897980561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d36e6fd820 0x55d36e707700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.479+0000 7f9a3ffff640 1 --2- 172.21.15.7:0/897980561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d36e6fe210 0x55d36e70f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.479+0000 7f9a4ce37640 1 -- 172.21.15.7:0/897980561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d36e6fe210 msgr2=0x55d36e70f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.479+0000 7f9a4ce37640 1 --2- 172.21.15.7:0/897980561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d36e6fe210 0x55d36e70f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.479+0000 7f9a4ce37640 1 -- 172.21.15.7:0/897980561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d36e6fd820 msgr2=0x55d36e707700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.479+0000 7f9a4ce37640 1 --2- 172.21.15.7:0/897980561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d36e6fd820 0x55d36e707700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:42.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.479+0000 7f9a4ce37640 1 -- 172.21.15.7:0/897980561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d36e7120e0 con 0x55d36e703380 2024-06-10T01:22:42.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.479+0000 7f9a4ce37640 1 --2- 172.21.15.7:0/897980561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d36e703380 0x55d36e707c40 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f9a4001a000 tx=0x7f9a40005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28574938d3b3c082 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:42.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.479+0000 7f9a3dffb640 1 -- 172.21.15.7:0/897980561 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4002d070 con 0x55d36e703380 2024-06-10T01:22:42.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.479+0000 7f9a3dffb640 1 -- 172.21.15.7:0/897980561 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a40005860 con 0x55d36e703380 2024-06-10T01:22:42.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.479+0000 7f9a3dffb640 1 -- 172.21.15.7:0/897980561 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a40004fc0 con 0x55d36e703380 2024-06-10T01:22:42.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.480+0000 7f9a515a1740 1 -- 172.21.15.7:0/897980561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d36e703380 msgr2=0x55d36e707c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:42.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.480+0000 7f9a515a1740 1 --2- 172.21.15.7:0/897980561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d36e703380 0x55d36e707c40 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f9a4001a000 tx=0x7f9a40005bd0 comp rx=0 tx=0).stop 2024-06-10T01:22:42.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.480+0000 7f9a515a1740 1 -- 172.21.15.7:0/897980561 shutdown_connections 2024-06-10T01:22:42.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.480+0000 7f9a515a1740 1 --2- 172.21.15.7:0/897980561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d36e703380 0x55d36e707c40 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.480+0000 7f9a515a1740 1 --2- 172.21.15.7:0/897980561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d36e6fe210 0x55d36e70f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.480+0000 7f9a515a1740 1 --2- 172.21.15.7:0/897980561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d36e6fd820 0x55d36e707700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.480+0000 7f9a515a1740 1 -- 172.21.15.7:0/897980561 >> 172.21.15.7:0/897980561 conn(0x55d36e6f8e90 msgr2=0x55d36e6f9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.480+0000 7f9a515a1740 1 -- 172.21.15.7:0/897980561 shutdown_connections 2024-06-10T01:22:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.480+0000 7f9a515a1740 1 -- 172.21.15.7:0/897980561 wait complete. 2024-06-10T01:22:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.480+0000 7f9a515a1740 1 -- start start 2024-06-10T01:22:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.480+0000 7f9a515a1740 1 Processor -- start 2024-06-10T01:22:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.481+0000 7f9a515a1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d36e6fd820 0x55d36e70c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.481+0000 7f9a515a1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d36e6fe210 0x55d36e70c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.481+0000 7f9a515a1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d36e703380 0x55d36e70d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.481+0000 7f9a515a1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9a481a69b0 con 0x55d36e6fe210 2024-06-10T01:22:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.481+0000 7f9a515a1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9a481a7e20 con 0x55d36e6fd820 2024-06-10T01:22:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.481+0000 7f9a515a1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9a481a7080 con 0x55d36e703380 2024-06-10T01:22:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.481+0000 7f9a515a1740 10 client.0 _ll_register_callbacks cb 0x7f9a30009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:22:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.481+0000 7f9a3ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d36e6fe210 0x55d36e70c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.481+0000 7f9a47fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d36e6fd820 0x55d36e70c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.481+0000 7f9a47fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d36e6fd820 0x55d36e70c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:37832/0 (socket says 172.21.15.7:37832) 2024-06-10T01:22:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.481+0000 7f9a47fff640 1 -- 172.21.15.7:0/1209523937 learned_addr learned my addr 172.21.15.7:0/1209523937 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.481+0000 7f9a3ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d36e6fe210 0x55d36e70c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38096/0 (socket says 172.21.15.7:38096) 2024-06-10T01:22:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.481+0000 7f9a4ce37640 1 --2- 172.21.15.7:0/1209523937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d36e703380 0x55d36e70d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.481+0000 7f9a3ffff640 1 -- 172.21.15.7:0/1209523937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d36e703380 msgr2=0x55d36e70d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.481+0000 7f9a3ffff640 1 --2- 172.21.15.7:0/1209523937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d36e703380 0x55d36e70d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.481+0000 7f9a3ffff640 1 -- 172.21.15.7:0/1209523937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d36e6fd820 msgr2=0x55d36e70c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.481+0000 7f9a3ffff640 1 --2- 172.21.15.7:0/1209523937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d36e6fd820 0x55d36e70c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.481+0000 7f9a3ffff640 1 -- 172.21.15.7:0/1209523937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d36e70dda0 con 0x55d36e6fe210 2024-06-10T01:22:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.481+0000 7f9a47fff640 1 --2- 172.21.15.7:0/1209523937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d36e6fd820 0x55d36e70c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:22:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.481+0000 7f9a3ffff640 1 --2- 172.21.15.7:0/1209523937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d36e6fe210 0x55d36e70c860 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f9a38009900 tx=0x7f9a380dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a89655b5a9da03 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.481+0000 7f9a277fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:22:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.481+0000 7f9a277fe640 1 -- 172.21.15.7:0/1209523937 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a38049040 con 0x55d36e6fe210 2024-06-10T01:22:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.481+0000 7f9a277fe640 1 -- 172.21.15.7:0/1209523937 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f9a380ed370 con 0x55d36e6fe210 2024-06-10T01:22:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.482+0000 7f9a515a1740 1 -- 172.21.15.7:0/1209523937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a40005860 con 0x55d36e6fe210 2024-06-10T01:22:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.482+0000 7f9a277fe640 1 -- 172.21.15.7:0/1209523937 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f9a380bb9d0 con 0x55d36e6fe210 2024-06-10T01:22:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.482+0000 7f9a515a1740 10 client.19542 fetch_fsmap learned FSMap version 47 2024-06-10T01:22:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.482+0000 7f9a515a1740 1 -- 172.21.15.7:0/1209523937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55d36e70f120 con 0x55d36e6fe210 2024-06-10T01:22:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.482+0000 7f9a277fe640 1 -- 172.21.15.7:0/1209523937 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a40005860 con 0x55d36e6fe210 2024-06-10T01:22:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.482+0000 7f9a277fe640 1 -- 172.21.15.7:0/1209523937 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 47) ==== 8350+0+0 (secure 0 0 0) 0x7f9a380be070 con 0x55d36e6fe210 2024-06-10T01:22:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.482+0000 7f9a515a1740 10 client.19542 fetch_fsmap finished waiting for FSMap version 47 2024-06-10T01:22:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.482+0000 7f9a515a1740 10 client.19542 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:22:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.482+0000 7f9a515a1740 20 client.19542 populate_metadata read hostname 'smithi007' 2024-06-10T01:22:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.482+0000 7f9a515a1740 1 --2- 172.21.15.7:0/1209523937 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d36e7137b0 0x55d36e780a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.482+0000 7f9a515a1740 4 client.19542 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:22:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.482+0000 7f9a515a1740 1 -- 172.21.15.7:0/1209523937 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d36e665c20 con 0x55d36e7137b0 2024-06-10T01:22:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.482+0000 7f9a47fff640 1 --2- 172.21.15.7:0/1209523937 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d36e7137b0 0x55d36e780a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.483+0000 7f9a47fff640 1 --2- 172.21.15.7:0/1209523937 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d36e7137b0 0x55d36e780a60 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9a0a6795da31738 server_cookie=1bb6bc1d05159576 in_seq=0 out_seq=0 2024-06-10T01:22:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:42.483+0000 7f9a277fe640 10 client.19542 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:42.525 INFO:tasks.workunit.client.0.smithi007.stdout: 10 89506 33.33 MB/sec execute 79 sec latency 195.715 ms 2024-06-10T01:22:43.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:43.139+0000 7f9a277fe640 1 -- 172.21.15.7:0/1209523937 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d36e665c20 con 0x55d36e7137b0 2024-06-10T01:22:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:43.139+0000 7f9a277fe640 10 client.19542 handle_command_reply: tid=0 2024-06-10T01:22:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:43.139+0000 7f9a515a1740 1 client.19542 shutdown 2024-06-10T01:22:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:43.139+0000 7f9a515a1740 1 -- 172.21.15.7:0/1209523937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d36e6fe210 msgr2=0x55d36e70c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:43.139+0000 7f9a515a1740 1 --2- 172.21.15.7:0/1209523937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d36e6fe210 0x55d36e70c860 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f9a38009900 tx=0x7f9a380dc040 comp rx=0 tx=0).stop 2024-06-10T01:22:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:43.139+0000 7f9a515a1740 1 -- 172.21.15.7:0/1209523937 shutdown_connections 2024-06-10T01:22:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:43.139+0000 7f9a515a1740 1 --2- 172.21.15.7:0/1209523937 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d36e7137b0 0x55d36e780a60 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:43.139+0000 7f9a277fe640 0 client.19542 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:43.139+0000 7f9a515a1740 1 --2- 172.21.15.7:0/1209523937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d36e703380 0x55d36e70d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:43.139+0000 7f9a515a1740 1 --2- 172.21.15.7:0/1209523937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d36e6fe210 0x55d36e70c860 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:43.139+0000 7f9a515a1740 1 --2- 172.21.15.7:0/1209523937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d36e6fd820 0x55d36e70c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:43.139+0000 7f9a515a1740 1 -- 172.21.15.7:0/1209523937 >> 172.21.15.7:0/1209523937 conn(0x55d36e6f8e90 msgr2=0x55d36e707700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:43.139+0000 7f9a515a1740 1 -- 172.21.15.7:0/1209523937 shutdown_connections 2024-06-10T01:22:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:43.139+0000 7f9a515a1740 1 -- 172.21.15.7:0/1209523937 wait complete. 2024-06-10T01:22:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:43.139+0000 7f9a515a1740 20 client.19542 trim_cache size 0 max 16384 2024-06-10T01:22:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:22:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:22:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:22:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:43.140+0000 7f9a4e89a640 1 -- 172.21.15.7:0/327908091 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9a340a2cd0 msgr2=0x7f9a340a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:43.140+0000 7f9a4e89a640 1 --2- 172.21.15.7:0/327908091 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9a340a2cd0 0x7f9a340a5190 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9a30009fd0 tx=0x7f9a30009190 comp rx=0 tx=0).stop 2024-06-10T01:22:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:43.140+0000 7f9a4e89a640 1 -- 172.21.15.7:0/327908091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a48101b10 msgr2=0x7f9a48196100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:43.140+0000 7f9a4e89a640 1 --2- 172.21.15.7:0/327908091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a48101b10 0x7f9a48196100 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9a3802a940 tx=0x7f9a38005000 comp rx=0 tx=0).stop 2024-06-10T01:22:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:43.140+0000 7f9a4e89a640 1 -- 172.21.15.7:0/327908091 shutdown_connections 2024-06-10T01:22:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:43.140+0000 7f9a4e89a640 1 --2- 172.21.15.7:0/327908091 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9a340a2cd0 0x7f9a340a5190 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:43.140+0000 7f9a4e89a640 1 --2- 172.21.15.7:0/327908091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a481024d0 0x7f9a4819d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:43.140+0000 7f9a4e89a640 1 --2- 172.21.15.7:0/327908091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a48101b10 0x7f9a48196100 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:43.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:43.140+0000 7f9a4e89a640 1 --2- 172.21.15.7:0/327908091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a48100920 0x7f9a48195bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:43.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:43.140+0000 7f9a4e89a640 1 -- 172.21.15.7:0/327908091 >> 172.21.15.7:0/327908091 conn(0x7f9a480fbeb0 msgr2=0x7f9a48104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:43.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:43.140+0000 7f9a4e89a640 1 -- 172.21.15.7:0/327908091 shutdown_connections 2024-06-10T01:22:43.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:43.140+0000 7f9a4e89a640 1 -- 172.21.15.7:0/327908091 wait complete. 2024-06-10T01:22:43.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:42 smithi175 ceph-mon[36636]: pgmap v933: 129 pgs: 129 active+clean; 585 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 8.6 MiB/s wr, 719 op/s 2024-06-10T01:22:43.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:42 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2193671918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:22:43.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:42 smithi007 ceph-mon[32398]: pgmap v933: 129 pgs: 129 active+clean; 585 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 8.6 MiB/s wr, 719 op/s 2024-06-10T01:22:43.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:42 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2193671918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:22:43.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:42 smithi078 ceph-mon[36683]: pgmap v933: 129 pgs: 129 active+clean; 585 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 8.6 MiB/s wr, 719 op/s 2024-06-10T01:22:43.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:42 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2193671918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:22:43.528 INFO:tasks.workunit.client.0.smithi007.stdout: 10 89767 33.09 MB/sec execute 80 sec latency 470.746 ms 2024-06-10T01:22:44.528 INFO:tasks.workunit.client.0.smithi007.stdout: 10 89993 32.81 MB/sec execute 81 sec latency 445.870 ms 2024-06-10T01:22:45.135 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-06-10T01:22:45.135 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-06-10T01:22:45.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:44 smithi007 ceph-mon[32398]: pgmap v934: 129 pgs: 129 active+clean; 580 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 8.8 MiB/s wr, 696 op/s 2024-06-10T01:22:45.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:44 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:22:45.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:44 smithi078 ceph-mon[36683]: pgmap v934: 129 pgs: 129 active+clean; 580 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 8.8 MiB/s wr, 696 op/s 2024-06-10T01:22:45.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:44 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:22:45.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:44 smithi175 ceph-mon[36636]: pgmap v934: 129 pgs: 129 active+clean; 580 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 8.8 MiB/s wr, 696 op/s 2024-06-10T01:22:45.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:44 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:22:45.505 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:22:45.528 INFO:tasks.workunit.client.0.smithi007.stdout: 10 90682 32.82 MB/sec execute 82 sec latency 93.054 ms 2024-06-10T01:22:46.529 INFO:tasks.workunit.client.0.smithi007.stdout: 10 91729 33.19 MB/sec execute 83 sec latency 38.325 ms 2024-06-10T01:22:47.263 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:46 smithi007 ceph-mon[32398]: pgmap v935: 129 pgs: 129 active+clean; 581 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.6 MiB/s wr, 758 op/s 2024-06-10T01:22:47.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:46 smithi078 ceph-mon[36683]: pgmap v935: 129 pgs: 129 active+clean; 581 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.6 MiB/s wr, 758 op/s 2024-06-10T01:22:47.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.350+0000 7f6213c50700 1 -- 172.21.15.7:0/2735505874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f620c06b420 msgr2=0x7f620c10f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:47.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.350+0000 7f6213c50700 1 --2- 172.21.15.7:0/2735505874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f620c06b420 0x7f620c10f340 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f620400b0a0 tx=0x7f620400b3b0 comp rx=0 tx=0).stop 2024-06-10T01:22:47.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.351+0000 7f6213c50700 1 -- 172.21.15.7:0/2735505874 shutdown_connections 2024-06-10T01:22:47.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.351+0000 7f6213c50700 1 --2- 172.21.15.7:0/2735505874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f620c06b420 0x7f620c10f340 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:47.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.351+0000 7f6213c50700 1 --2- 172.21.15.7:0/2735505874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f620c108520 0x7f620c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:47.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.351+0000 7f6213c50700 1 --2- 172.21.15.7:0/2735505874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f620c06aa60 0x7f620c06aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:47.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.351+0000 7f6213c50700 1 -- 172.21.15.7:0/2735505874 >> 172.21.15.7:0/2735505874 conn(0x7f620c0783d0 msgr2=0x7f620c0787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:47.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.351+0000 7f6213c50700 1 -- 172.21.15.7:0/2735505874 shutdown_connections 2024-06-10T01:22:47.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.351+0000 7f6213c50700 1 -- 172.21.15.7:0/2735505874 wait complete. 2024-06-10T01:22:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.351+0000 7f6213c50700 1 Processor -- start 2024-06-10T01:22:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.351+0000 7f6213c50700 1 -- start start 2024-06-10T01:22:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.352+0000 7f6213c50700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f620c06aa60 0x7f620c19e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.352+0000 7f6213c50700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f620c06b420 0x7f620c19ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.352+0000 7f6213c50700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f620c108520 0x7f620c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.352+0000 7f6213c50700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f620c1127a0 con 0x7f620c06b420 2024-06-10T01:22:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.352+0000 7f6213c50700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f620c112620 con 0x7f620c06aa60 2024-06-10T01:22:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.352+0000 7f62111eb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f620c06b420 0x7f620c19ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.352+0000 7f62121ed700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f620c108520 0x7f620c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.352+0000 7f62119ec700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f620c06aa60 0x7f620c19e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:47.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.352+0000 7f6213c50700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f620c112920 con 0x7f620c108520 2024-06-10T01:22:47.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.352+0000 7f62119ec700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f620c06aa60 0x7f620c19e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:37862/0 (socket says 172.21.15.7:37862) 2024-06-10T01:22:47.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.352+0000 7f62121ed700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f620c108520 0x7f620c198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:54204/0 (socket says 172.21.15.7:54204) 2024-06-10T01:22:47.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.352+0000 7f62119ec700 1 -- 172.21.15.7:0/2578034388 learned_addr learned my addr 172.21.15.7:0/2578034388 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:47.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.352+0000 7f62111eb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f620c06b420 0x7f620c19ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38102/0 (socket says 172.21.15.7:38102) 2024-06-10T01:22:47.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.352+0000 7f62111eb700 1 -- 172.21.15.7:0/2578034388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f620c108520 msgr2=0x7f620c198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:47.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.352+0000 7f62111eb700 1 --2- 172.21.15.7:0/2578034388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f620c108520 0x7f620c198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:47.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.352+0000 7f62111eb700 1 -- 172.21.15.7:0/2578034388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f620c06aa60 msgr2=0x7f620c19e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:47.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.352+0000 7f62111eb700 1 --2- 172.21.15.7:0/2578034388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f620c06aa60 0x7f620c19e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:47.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.352+0000 7f62111eb700 1 -- 172.21.15.7:0/2578034388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620c199060 con 0x7f620c06b420 2024-06-10T01:22:47.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.353+0000 7f62111eb700 1 --2- 172.21.15.7:0/2578034388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f620c06b420 0x7f620c19ec80 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00e9f0 tx=0x7f61fc00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:47.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.353+0000 7f6202ffd700 1 -- 172.21.15.7:0/2578034388 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc004220 con 0x7f620c06b420 2024-06-10T01:22:47.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.353+0000 7f6213c50700 1 -- 172.21.15.7:0/2578034388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f620c199350 con 0x7f620c06b420 2024-06-10T01:22:47.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.353+0000 7f6213c50700 1 -- 172.21.15.7:0/2578034388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f620c101380 con 0x7f620c06b420 2024-06-10T01:22:47.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.353+0000 7f6202ffd700 1 -- 172.21.15.7:0/2578034388 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61fc004d10 con 0x7f620c06b420 2024-06-10T01:22:47.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.353+0000 7f6202ffd700 1 -- 172.21.15.7:0/2578034388 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc005020 con 0x7f620c06b420 2024-06-10T01:22:47.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.356+0000 7f6202ffd700 1 -- 172.21.15.7:0/2578034388 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f61fc00c7c0 con 0x7f620c06b420 2024-06-10T01:22:47.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.358+0000 7f6202ffd700 1 --2- 172.21.15.7:0/2578034388 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f61f80a2d40 0x7f61f80a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:47.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.358+0000 7f6202ffd700 1 -- 172.21.15.7:0/2578034388 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f61fc0cf1c0 con 0x7f620c06b420 2024-06-10T01:22:47.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.358+0000 7f62119ec700 1 --2- 172.21.15.7:0/2578034388 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f61f80a2d40 0x7f61f80a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:47.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.359+0000 7f62119ec700 1 --2- 172.21.15.7:0/2578034388 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f61f80a2d40 0x7f61f80a5200 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f6208009680 tx=0x7f6208012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:47.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.359+0000 7f621e4d3b80 1 Processor -- start 2024-06-10T01:22:47.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.359+0000 7f621e4d3b80 1 -- start start 2024-06-10T01:22:47.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.360+0000 7f621e4d3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e9b999590 0x556e9b999970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:47.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.360+0000 7f621e4d3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e9b999eb0 0x556e9b9937a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:47.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.360+0000 7f621e4d3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e9b993ce0 0x556e9b994100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.360+0000 7f621e4d3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556e9b8860f0 con 0x556e9b999590 2024-06-10T01:22:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.360+0000 7f621e4d3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556e9b8e49e0 con 0x556e9b999eb0 2024-06-10T01:22:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.360+0000 7f621e4d3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556e9b84c5f0 con 0x556e9b993ce0 2024-06-10T01:22:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.360+0000 7f62121ed700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e9b993ce0 0x556e9b994100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.360+0000 7f62121ed700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e9b993ce0 0x556e9b994100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:54218/0 (socket says 172.21.15.7:54218) 2024-06-10T01:22:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.360+0000 7f62121ed700 1 -- 172.21.15.7:0/2313262827 learned_addr learned my addr 172.21.15.7:0/2313262827 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.360+0000 7f62111eb700 1 --2- 172.21.15.7:0/2313262827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e9b999eb0 0x556e9b9937a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.360+0000 7f62119ec700 1 --2- 172.21.15.7:0/2313262827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e9b999590 0x556e9b999970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.361+0000 7f62121ed700 1 -- 172.21.15.7:0/2313262827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e9b999eb0 msgr2=0x556e9b9937a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.361+0000 7f62121ed700 1 --2- 172.21.15.7:0/2313262827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e9b999eb0 0x556e9b9937a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.361+0000 7f62121ed700 1 -- 172.21.15.7:0/2313262827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e9b999590 msgr2=0x556e9b999970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.361+0000 7f62121ed700 1 --2- 172.21.15.7:0/2313262827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e9b999590 0x556e9b999970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.361+0000 7f62121ed700 1 -- 172.21.15.7:0/2313262827 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e9b833a60 con 0x556e9b993ce0 2024-06-10T01:22:47.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.361+0000 7f62121ed700 1 --2- 172.21.15.7:0/2313262827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e9b993ce0 0x556e9b994100 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6204009de0 tx=0x7f6204009e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d0da64702091709 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:47.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.362+0000 7f61f7fff700 1 -- 172.21.15.7:0/2313262827 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6204009490 con 0x556e9b993ce0 2024-06-10T01:22:47.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.362+0000 7f61f7fff700 1 -- 172.21.15.7:0/2313262827 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6204009490 con 0x556e9b993ce0 2024-06-10T01:22:47.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.362+0000 7f61f7fff700 1 -- 172.21.15.7:0/2313262827 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6204003e30 con 0x556e9b993ce0 2024-06-10T01:22:47.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.363+0000 7f621e4d3b80 1 -- 172.21.15.7:0/2313262827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e9b993ce0 msgr2=0x556e9b994100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:47.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.363+0000 7f621e4d3b80 1 --2- 172.21.15.7:0/2313262827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e9b993ce0 0x556e9b994100 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6204009de0 tx=0x7f6204009e10 comp rx=0 tx=0).stop 2024-06-10T01:22:47.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.363+0000 7f621e4d3b80 1 -- 172.21.15.7:0/2313262827 shutdown_connections 2024-06-10T01:22:47.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.363+0000 7f621e4d3b80 1 --2- 172.21.15.7:0/2313262827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e9b993ce0 0x556e9b994100 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:47.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.363+0000 7f621e4d3b80 1 --2- 172.21.15.7:0/2313262827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e9b999590 0x556e9b999970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:47.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.363+0000 7f621e4d3b80 1 --2- 172.21.15.7:0/2313262827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e9b999eb0 0x556e9b9937a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:47.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.363+0000 7f621e4d3b80 1 -- 172.21.15.7:0/2313262827 >> 172.21.15.7:0/2313262827 conn(0x556e9b8c0210 msgr2=0x556e9b98ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:47.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.363+0000 7f621e4d3b80 1 -- 172.21.15.7:0/2313262827 shutdown_connections 2024-06-10T01:22:47.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.363+0000 7f621e4d3b80 1 -- 172.21.15.7:0/2313262827 wait complete. 2024-06-10T01:22:47.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.364+0000 7f621e4d3b80 1 -- start start 2024-06-10T01:22:47.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.364+0000 7f621e4d3b80 1 Processor -- start 2024-06-10T01:22:47.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.365+0000 7f621e4d3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e9b993ce0 0x556e9b99fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:47.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.365+0000 7f621e4d3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e9b999590 0x556e9b99cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:47.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.365+0000 7f62119ec700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e9b993ce0 0x556e9b99fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:47.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.365+0000 7f62119ec700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e9b993ce0 0x556e9b99fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:37880/0 (socket says 172.21.15.7:37880) 2024-06-10T01:22:47.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.365+0000 7f62119ec700 1 -- 172.21.15.7:0/1908286453 learned_addr learned my addr 172.21.15.7:0/1908286453 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:47.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.365+0000 7f621e4d3b80 1 --2- 172.21.15.7:0/1908286453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e9b999eb0 0x556e9b99d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:47.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.365+0000 7f62111eb700 1 --2- 172.21.15.7:0/1908286453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e9b999590 0x556e9b99cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:47.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.365+0000 7f621e4d3b80 1 -- 172.21.15.7:0/1908286453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556e9b885150 con 0x556e9b999eb0 2024-06-10T01:22:47.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.365+0000 7f621e4d3b80 1 -- 172.21.15.7:0/1908286453 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556e9b8e09a0 con 0x556e9b993ce0 2024-06-10T01:22:47.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.365+0000 7f621e4d3b80 1 -- 172.21.15.7:0/1908286453 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556e9b9a0830 con 0x556e9b999590 2024-06-10T01:22:47.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.366+0000 7f62121ed700 1 --2- 172.21.15.7:0/1908286453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e9b999eb0 0x556e9b99d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:47.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.366+0000 7f62119ec700 1 -- 172.21.15.7:0/1908286453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e9b999590 msgr2=0x556e9b99cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:47.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.366+0000 7f62119ec700 1 --2- 172.21.15.7:0/1908286453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e9b999590 0x556e9b99cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:47.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.366+0000 7f62119ec700 1 -- 172.21.15.7:0/1908286453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e9b999eb0 msgr2=0x556e9b99d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:47.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.366+0000 7f62119ec700 1 --2- 172.21.15.7:0/1908286453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e9b999eb0 0x556e9b99d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:47.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.366+0000 7f62119ec700 1 -- 172.21.15.7:0/1908286453 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e9b8aa8e0 con 0x556e9b993ce0 2024-06-10T01:22:47.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.366+0000 7f621e4d3b80 10 client.0 _ll_register_callbacks cb 0x556e9b87a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:22:47.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.366+0000 7f62121ed700 1 --2- 172.21.15.7:0/1908286453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e9b999eb0 0x556e9b99d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:22:47.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.366+0000 7f621e4d3b80 1 -- 172.21.15.7:0/1908286453 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6204009490 con 0x556e9b993ce0 2024-06-10T01:22:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.367+0000 7f62119ec700 1 --2- 172.21.15.7:0/1908286453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e9b993ce0 0x556e9b99fbe0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f620801a660 tx=0x7f6208004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3041348250b8aa05 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.367+0000 7f61f57fa700 10 client.28790 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:22:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.367+0000 7f61f57fa700 1 -- 172.21.15.7:0/1908286453 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620802b060 con 0x556e9b993ce0 2024-06-10T01:22:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.367+0000 7f61f57fa700 1 -- 172.21.15.7:0/1908286453 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f62080255d0 con 0x556e9b993ce0 2024-06-10T01:22:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.367+0000 7f61f57fa700 1 -- 172.21.15.7:0/1908286453 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f62080257f0 con 0x556e9b993ce0 2024-06-10T01:22:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.367+0000 7f621e4d3b80 10 client.28790 fetch_fsmap learned FSMap version 48 2024-06-10T01:22:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.367+0000 7f621e4d3b80 1 -- 172.21.15.7:0/1908286453 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=48}) -- 0x556e9b9aafc0 con 0x556e9b993ce0 2024-06-10T01:22:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.367+0000 7f61f57fa700 1 -- 172.21.15.7:0/1908286453 <== mon.2 v2:172.21.15.78:3300/0 4 ==== fsmap(e 48) ==== 8354+0+0 (secure 0 0 0) 0x7f6208019070 con 0x556e9b993ce0 2024-06-10T01:22:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.368+0000 7f621e4d3b80 10 client.28790 fetch_fsmap finished waiting for FSMap version 48 2024-06-10T01:22:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.368+0000 7f621e4d3b80 10 client.28790 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:22:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.368+0000 7f621e4d3b80 20 client.28790 populate_metadata read hostname 'smithi007' 2024-06-10T01:22:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.368+0000 7f621e4d3b80 1 --2- 172.21.15.7:0/1908286453 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556e9b9ad490 0x556e9b9af880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.368+0000 7f621e4d3b80 4 client.28790 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:22:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.368+0000 7f621e4d3b80 1 -- 172.21.15.7:0/1908286453 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e9b8f1ee0 con 0x556e9b9ad490 2024-06-10T01:22:47.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.368+0000 7f62111eb700 1 --2- 172.21.15.7:0/1908286453 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556e9b9ad490 0x556e9b9af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:47.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.369+0000 7f62111eb700 1 --2- 172.21.15.7:0/1908286453 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556e9b9ad490 0x556e9b9af880 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b022c68335a52cba server_cookie=a25098c029791b2f in_seq=0 out_seq=0 2024-06-10T01:22:47.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.369+0000 7f61f57fa700 10 client.28790 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:47.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.370+0000 7f61f57fa700 1 -- 172.21.15.7:0/1908286453 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556e9b8f1ee0 con 0x556e9b9ad490 2024-06-10T01:22:47.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.370+0000 7f61f57fa700 10 client.28790 handle_command_reply: tid=0 2024-06-10T01:22:47.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.370+0000 7f621e4d3b80 1 client.28790 shutdown 2024-06-10T01:22:47.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.370+0000 7f621e4d3b80 1 -- 172.21.15.7:0/1908286453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e9b993ce0 msgr2=0x556e9b99fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:47.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.370+0000 7f621e4d3b80 1 --2- 172.21.15.7:0/1908286453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e9b993ce0 0x556e9b99fbe0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f620801a660 tx=0x7f6208004750 comp rx=0 tx=0).stop 2024-06-10T01:22:47.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.371+0000 7f621e4d3b80 1 -- 172.21.15.7:0/1908286453 shutdown_connections 2024-06-10T01:22:47.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.371+0000 7f621e4d3b80 1 --2- 172.21.15.7:0/1908286453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e9b999eb0 0x556e9b99d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:47.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.371+0000 7f621e4d3b80 1 --2- 172.21.15.7:0/1908286453 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556e9b9ad490 0x556e9b9af880 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:47.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.371+0000 7f621e4d3b80 1 --2- 172.21.15.7:0/1908286453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e9b993ce0 0x556e9b99fbe0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:47.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.371+0000 7f61f57fa700 0 client.28790 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:47.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.371+0000 7f621e4d3b80 1 --2- 172.21.15.7:0/1908286453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e9b999590 0x556e9b99cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:47.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.371+0000 7f621e4d3b80 1 -- 172.21.15.7:0/1908286453 >> 172.21.15.7:0/1908286453 conn(0x556e9b8c0210 msgr2=0x556e9b98d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:47.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.371+0000 7f621e4d3b80 1 -- 172.21.15.7:0/1908286453 shutdown_connections 2024-06-10T01:22:47.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.371+0000 7f621e4d3b80 1 -- 172.21.15.7:0/1908286453 wait complete. 2024-06-10T01:22:47.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.371+0000 7f621e4d3b80 20 client.28790 trim_cache size 0 max 16384 2024-06-10T01:22:47.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.395+0000 7f621e4d3b80 1 Processor -- start 2024-06-10T01:22:47.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.395+0000 7f621e4d3b80 1 -- start start 2024-06-10T01:22:47.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.395+0000 7f621e4d3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e9b8c0210 0x556e9b9937a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:47.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.395+0000 7f621e4d3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e9b993ce0 0x556e9b99cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:47.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.395+0000 7f621e4d3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e9b999590 0x556e9b99d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:47.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.395+0000 7f621e4d3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f61f0011300 con 0x556e9b999590 2024-06-10T01:22:47.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.395+0000 7f621e4d3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f61f000eb90 con 0x556e9b8c0210 2024-06-10T01:22:47.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.395+0000 7f621e4d3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f61f000c9e0 con 0x556e9b993ce0 2024-06-10T01:22:47.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.395+0000 7f62119ec700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e9b8c0210 0x556e9b9937a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:47.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.395+0000 7f62121ed700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e9b999590 0x556e9b99d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:47.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.395+0000 7f62119ec700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e9b8c0210 0x556e9b9937a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:37890/0 (socket says 172.21.15.7:37890) 2024-06-10T01:22:47.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.395+0000 7f62111eb700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e9b993ce0 0x556e9b99cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:47.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.395+0000 7f62119ec700 1 -- 172.21.15.7:0/1344003076 learned_addr learned my addr 172.21.15.7:0/1344003076 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:47.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.395+0000 7f62121ed700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e9b999590 0x556e9b99d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38134/0 (socket says 172.21.15.7:38134) 2024-06-10T01:22:47.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.396+0000 7f62119ec700 1 -- 172.21.15.7:0/1344003076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e9b993ce0 msgr2=0x556e9b99cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:47.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.396+0000 7f62119ec700 1 --2- 172.21.15.7:0/1344003076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e9b993ce0 0x556e9b99cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:47.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.396+0000 7f62119ec700 1 -- 172.21.15.7:0/1344003076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e9b999590 msgr2=0x556e9b99d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:47.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.396+0000 7f62119ec700 1 --2- 172.21.15.7:0/1344003076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e9b999590 0x556e9b99d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:47.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.396+0000 7f62119ec700 1 -- 172.21.15.7:0/1344003076 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e9b99b4a0 con 0x556e9b8c0210 2024-06-10T01:22:47.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.396+0000 7f62119ec700 1 --2- 172.21.15.7:0/1344003076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e9b8c0210 0x556e9b9937a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6208025a60 tx=0x7f620801d070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e14edabafabc62a7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:47.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.396+0000 7f61f77fe700 1 -- 172.21.15.7:0/1344003076 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620803d070 con 0x556e9b8c0210 2024-06-10T01:22:47.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.396+0000 7f61f77fe700 1 -- 172.21.15.7:0/1344003076 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6204009490 con 0x556e9b8c0210 2024-06-10T01:22:47.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.396+0000 7f61f77fe700 1 -- 172.21.15.7:0/1344003076 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208030420 con 0x556e9b8c0210 2024-06-10T01:22:47.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.397+0000 7f621e4d3b80 1 -- 172.21.15.7:0/1344003076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e9b8c0210 msgr2=0x556e9b9937a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:47.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.397+0000 7f621e4d3b80 1 --2- 172.21.15.7:0/1344003076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e9b8c0210 0x556e9b9937a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6208025a60 tx=0x7f620801d070 comp rx=0 tx=0).stop 2024-06-10T01:22:47.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.397+0000 7f621e4d3b80 1 -- 172.21.15.7:0/1344003076 shutdown_connections 2024-06-10T01:22:47.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.397+0000 7f621e4d3b80 1 --2- 172.21.15.7:0/1344003076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e9b999590 0x556e9b99d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:47.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.397+0000 7f621e4d3b80 1 --2- 172.21.15.7:0/1344003076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e9b8c0210 0x556e9b9937a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:47.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.397+0000 7f621e4d3b80 1 --2- 172.21.15.7:0/1344003076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e9b993ce0 0x556e9b99cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:47.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.397+0000 7f621e4d3b80 1 -- 172.21.15.7:0/1344003076 >> 172.21.15.7:0/1344003076 conn(0x556e9b99fd40 msgr2=0x556e9b99c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:47.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.397+0000 7f621e4d3b80 1 -- 172.21.15.7:0/1344003076 shutdown_connections 2024-06-10T01:22:47.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.397+0000 7f621e4d3b80 1 -- 172.21.15.7:0/1344003076 wait complete. 2024-06-10T01:22:47.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.397+0000 7f621e4d3b80 1 -- start start 2024-06-10T01:22:47.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.397+0000 7f621e4d3b80 1 Processor -- start 2024-06-10T01:22:47.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.398+0000 7f621e4d3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e9b8c0210 0x556e9b9a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:47.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.398+0000 7f621e4d3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e9b993ce0 0x556e9b9a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:47.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.398+0000 7f62119ec700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e9b8c0210 0x556e9b9a60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:47.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.398+0000 7f62119ec700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e9b8c0210 0x556e9b9a60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:37894/0 (socket says 172.21.15.7:37894) 2024-06-10T01:22:47.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.398+0000 7f62111eb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e9b993ce0 0x556e9b9a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:47.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.398+0000 7f62119ec700 1 -- 172.21.15.7:0/3685173226 learned_addr learned my addr 172.21.15.7:0/3685173226 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:47.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.398+0000 7f62111eb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e9b993ce0 0x556e9b9a2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38136/0 (socket says 172.21.15.7:38136) 2024-06-10T01:22:47.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.398+0000 7f621e4d3b80 1 --2- 172.21.15.7:0/3685173226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e9b999590 0x556e9b9a3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:47.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.398+0000 7f621e4d3b80 1 -- 172.21.15.7:0/3685173226 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f61f000d050 con 0x556e9b993ce0 2024-06-10T01:22:47.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.398+0000 7f621e4d3b80 1 -- 172.21.15.7:0/3685173226 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f61f000e4c0 con 0x556e9b8c0210 2024-06-10T01:22:47.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.398+0000 7f621e4d3b80 1 -- 172.21.15.7:0/3685173226 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f61f000d720 con 0x556e9b999590 2024-06-10T01:22:47.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.398+0000 7f621e4d3b80 10 client.0 _ll_register_callbacks cb 0x556e9b87a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:22:47.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.398+0000 7f62121ed700 1 --2- 172.21.15.7:0/3685173226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e9b999590 0x556e9b9a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:47.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.398+0000 7f62119ec700 1 -- 172.21.15.7:0/3685173226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e9b999590 msgr2=0x556e9b9a3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:47.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.398+0000 7f62119ec700 1 --2- 172.21.15.7:0/3685173226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e9b999590 0x556e9b9a3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:47.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.398+0000 7f62119ec700 1 -- 172.21.15.7:0/3685173226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e9b993ce0 msgr2=0x556e9b9a2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:47.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.398+0000 7f62119ec700 1 --2- 172.21.15.7:0/3685173226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e9b993ce0 0x556e9b9a2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:47.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.398+0000 7f62119ec700 1 -- 172.21.15.7:0/3685173226 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e9b9a35a0 con 0x556e9b8c0210 2024-06-10T01:22:47.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.399+0000 7f62121ed700 1 --2- 172.21.15.7:0/3685173226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e9b999590 0x556e9b9a3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:22:47.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.399+0000 7f62119ec700 1 --2- 172.21.15.7:0/3685173226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e9b8c0210 0x556e9b9a60b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6208025c00 tx=0x7f6208009390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdb7b6a159923efa server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:47.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.400+0000 7f61f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:22:47.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.400+0000 7f61f57fa700 1 -- 172.21.15.7:0/3685173226 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620803d030 con 0x556e9b8c0210 2024-06-10T01:22:47.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.400+0000 7f61f57fa700 1 -- 172.21.15.7:0/3685173226 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f6208026460 con 0x556e9b8c0210 2024-06-10T01:22:47.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.400+0000 7f621e4d3b80 1 -- 172.21.15.7:0/3685173226 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6204009490 con 0x556e9b8c0210 2024-06-10T01:22:47.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.402+0000 7f61f57fa700 1 -- 172.21.15.7:0/3685173226 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f6208026660 con 0x556e9b8c0210 2024-06-10T01:22:47.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.402+0000 7f621e4d3b80 10 client.28796 fetch_fsmap learned FSMap version 48 2024-06-10T01:22:47.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.402+0000 7f621e4d3b80 1 -- 172.21.15.7:0/3685173226 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x556e9ba18e80 con 0x556e9b8c0210 2024-06-10T01:22:47.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.403+0000 7f61f57fa700 1 -- 172.21.15.7:0/3685173226 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f620803f050 con 0x556e9b8c0210 2024-06-10T01:22:47.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.403+0000 7f61f57fa700 1 -- 172.21.15.7:0/3685173226 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 48) ==== 8354+0+0 (secure 0 0 0) 0x7f6208019040 con 0x556e9b8c0210 2024-06-10T01:22:47.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.403+0000 7f621e4d3b80 10 client.28796 fetch_fsmap finished waiting for FSMap version 48 2024-06-10T01:22:47.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.403+0000 7f621e4d3b80 10 client.28796 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:22:47.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.403+0000 7f621e4d3b80 20 client.28796 populate_metadata read hostname 'smithi007' 2024-06-10T01:22:47.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.404+0000 7f621e4d3b80 1 --2- 172.21.15.7:0/3685173226 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556e9b999eb0 0x556e9ba1b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:47.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.404+0000 7f621e4d3b80 4 client.28796 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-06-10T01:22:47.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.404+0000 7f621e4d3b80 1 -- 172.21.15.7:0/3685173226 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x7f6204009490 con 0x556e9b999eb0 2024-06-10T01:22:47.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.405+0000 7f62111eb700 1 --2- 172.21.15.7:0/3685173226 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556e9b999eb0 0x556e9ba1b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:47.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.406+0000 7f61f57fa700 10 client.28796 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:47.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:47.406+0000 7f62111eb700 1 --2- 172.21.15.7:0/3685173226 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556e9b999eb0 0x556e9ba1b4e0 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf57bec0b5fcb840 server_cookie=abb68b1c8bf68910 in_seq=0 out_seq=0 2024-06-10T01:22:47.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:46 smithi175 ceph-mon[36636]: pgmap v935: 129 pgs: 129 active+clean; 581 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.6 MiB/s wr, 758 op/s 2024-06-10T01:22:47.529 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92360 33.29 MB/sec execute 84 sec latency 126.877 ms 2024-06-10T01:22:48.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:48.036+0000 7f61f57fa700 1 -- 172.21.15.7:0/3685173226 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x7f6204009490 con 0x556e9b999eb0 2024-06-10T01:22:48.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:48.037+0000 7f61f57fa700 10 client.28796 handle_command_reply: tid=0 2024-06-10T01:22:48.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:48.037+0000 7f621e4d3b80 1 client.28796 shutdown 2024-06-10T01:22:48.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:48.037+0000 7f621e4d3b80 1 -- 172.21.15.7:0/3685173226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e9b8c0210 msgr2=0x556e9b9a60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:48.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:48.037+0000 7f621e4d3b80 1 --2- 172.21.15.7:0/3685173226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e9b8c0210 0x556e9b9a60b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6208025c00 tx=0x7f6208009390 comp rx=0 tx=0).stop 2024-06-10T01:22:48.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:48.037+0000 7f621e4d3b80 1 -- 172.21.15.7:0/3685173226 shutdown_connections 2024-06-10T01:22:48.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:48.037+0000 7f621e4d3b80 1 --2- 172.21.15.7:0/3685173226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e9b999590 0x556e9b9a3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:48.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:48.037+0000 7f621e4d3b80 1 --2- 172.21.15.7:0/3685173226 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556e9b999eb0 0x556e9ba1b4e0 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:48.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:48.037+0000 7f61f57fa700 0 client.28796 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:48.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:48.037+0000 7f621e4d3b80 1 --2- 172.21.15.7:0/3685173226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e9b8c0210 0x556e9b9a60b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:48.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:48.037+0000 7f621e4d3b80 1 --2- 172.21.15.7:0/3685173226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e9b993ce0 0x556e9b9a2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:48.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:48.037+0000 7f621e4d3b80 1 -- 172.21.15.7:0/3685173226 >> 172.21.15.7:0/3685173226 conn(0x556e9b99fd40 msgr2=0x556e9b998840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:48.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:48.037+0000 7f621e4d3b80 1 -- 172.21.15.7:0/3685173226 shutdown_connections 2024-06-10T01:22:48.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:48.037+0000 7f621e4d3b80 1 -- 172.21.15.7:0/3685173226 wait complete. 2024-06-10T01:22:48.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:48.037+0000 7f621e4d3b80 20 client.28796 trim_cache size 0 max 16384 2024-06-10T01:22:48.032 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:22:48.032 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 48, 2024-06-10T01:22:48.032 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:22:48.032 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 54, 2024-06-10T01:22:48.032 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:22:48.032 INFO:teuthology.orchestra.run.smithi007.stdout: "f064d2e": { 2024-06-10T01:22:48.032 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 54, 2024-06-10T01:22:48.033 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:22:48.033 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:22:48.033 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:22:48.033 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:22:48.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:22:48.033 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:22:48.033 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 150.0, 2024-06-10T01:22:48.033 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:22:48.033 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:22:48.033 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:22:48.033 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:22:48.033 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:22:48.033 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:22:48.033 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:22:48.033 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:22:48.034 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:22:48.034 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:22:48.034 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:22:48.034 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:22:48.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:48.038+0000 7f6213c50700 1 -- 172.21.15.7:0/2578034388 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f61f80a2d40 msgr2=0x7f61f80a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:48.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:48.038+0000 7f6213c50700 1 --2- 172.21.15.7:0/2578034388 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f61f80a2d40 0x7f61f80a5200 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f6208009680 tx=0x7f6208012040 comp rx=0 tx=0).stop 2024-06-10T01:22:48.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:48.038+0000 7f6213c50700 1 -- 172.21.15.7:0/2578034388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f620c06b420 msgr2=0x7f620c19ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:48.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:48.038+0000 7f6213c50700 1 --2- 172.21.15.7:0/2578034388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f620c06b420 0x7f620c19ec80 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00e9f0 tx=0x7f61fc00ed00 comp rx=0 tx=0).stop 2024-06-10T01:22:48.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:48.038+0000 7f6213c50700 1 -- 172.21.15.7:0/2578034388 shutdown_connections 2024-06-10T01:22:48.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:48.038+0000 7f6213c50700 1 --2- 172.21.15.7:0/2578034388 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f61f80a2d40 0x7f61f80a5200 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:48.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:48.038+0000 7f6213c50700 1 --2- 172.21.15.7:0/2578034388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f620c108520 0x7f620c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:48.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:48.038+0000 7f6213c50700 1 --2- 172.21.15.7:0/2578034388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f620c06aa60 0x7f620c19e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:48.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:48.038+0000 7f6213c50700 1 --2- 172.21.15.7:0/2578034388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f620c06b420 0x7f620c19ec80 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:48.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:48.038+0000 7f6213c50700 1 -- 172.21.15.7:0/2578034388 >> 172.21.15.7:0/2578034388 conn(0x7f620c0783d0 msgr2=0x7f620c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:48.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:48.038+0000 7f6213c50700 1 -- 172.21.15.7:0/2578034388 shutdown_connections 2024-06-10T01:22:48.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:48.038+0000 7f6213c50700 1 -- 172.21.15.7:0/2578034388 wait complete. 2024-06-10T01:22:48.529 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 32.90 MB/sec execute 85 sec latency 1126.921 ms 2024-06-10T01:22:48.876 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f064d2e', quiesce took 9.7 seconds. Will release after: 20.3 2024-06-10T01:22:49.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:48 smithi007 ceph-mon[32398]: pgmap v936: 129 pgs: 129 active+clean; 584 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.1 MiB/s wr, 668 op/s 2024-06-10T01:22:49.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:48 smithi078 ceph-mon[36683]: pgmap v936: 129 pgs: 129 active+clean; 584 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.1 MiB/s wr, 668 op/s 2024-06-10T01:22:49.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:48 smithi175 ceph-mon[36636]: pgmap v936: 129 pgs: 129 active+clean; 584 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.1 MiB/s wr, 668 op/s 2024-06-10T01:22:49.529 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 32.52 MB/sec execute 86 sec latency 2126.964 ms 2024-06-10T01:22:50.529 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 32.14 MB/sec execute 87 sec latency 3127.071 ms 2024-06-10T01:22:51.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:50 smithi007 ceph-mon[32398]: pgmap v937: 129 pgs: 129 active+clean; 589 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 846 op/s 2024-06-10T01:22:51.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:50 smithi078 ceph-mon[36683]: pgmap v937: 129 pgs: 129 active+clean; 589 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 846 op/s 2024-06-10T01:22:51.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:50 smithi175 ceph-mon[36636]: pgmap v937: 129 pgs: 129 active+clean; 589 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 846 op/s 2024-06-10T01:22:51.529 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 31.78 MB/sec execute 88 sec latency 4127.187 ms 2024-06-10T01:22:52.529 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 31.42 MB/sec execute 89 sec latency 5127.304 ms 2024-06-10T01:22:52.603 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:22:52.855 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:22:53.253 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:52 smithi175 ceph-mon[36636]: pgmap v938: 129 pgs: 129 active+clean; 593 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 7.6 MiB/s wr, 684 op/s 2024-06-10T01:22:53.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:52 smithi007 ceph-mon[32398]: pgmap v938: 129 pgs: 129 active+clean; 593 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 7.6 MiB/s wr, 684 op/s 2024-06-10T01:22:53.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:52 smithi078 ceph-mon[36683]: pgmap v938: 129 pgs: 129 active+clean; 593 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 7.6 MiB/s wr, 684 op/s 2024-06-10T01:22:53.529 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 31.07 MB/sec execute 90 sec latency 6127.400 ms 2024-06-10T01:22:54.529 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 30.73 MB/sec execute 91 sec latency 7127.466 ms 2024-06-10T01:22:54.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.674+0000 7f931abac700 1 -- 172.21.15.7:0/2768088121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f931406b3a0 msgr2=0x7f931410f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:54.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.674+0000 7f931abac700 1 --2- 172.21.15.7:0/2768088121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f931406b3a0 0x7f931410f310 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f930c00b0a0 tx=0x7f930c00b3b0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.674+0000 7f931abac700 1 -- 172.21.15.7:0/2768088121 shutdown_connections 2024-06-10T01:22:54.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.674+0000 7f931abac700 1 --2- 172.21.15.7:0/2768088121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f931406b3a0 0x7f931410f310 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.674+0000 7f931abac700 1 --2- 172.21.15.7:0/2768088121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93141084d0 0x7f93141088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.674+0000 7f931abac700 1 --2- 172.21.15.7:0/2768088121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f931406aa00 0x7f931406ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.674+0000 7f931abac700 1 -- 172.21.15.7:0/2768088121 >> 172.21.15.7:0/2768088121 conn(0x7f9314078330 msgr2=0x7f9314078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:54.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.674+0000 7f931abac700 1 -- 172.21.15.7:0/2768088121 shutdown_connections 2024-06-10T01:22:54.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.674+0000 7f931abac700 1 -- 172.21.15.7:0/2768088121 wait complete. 2024-06-10T01:22:54.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.675+0000 7f931abac700 1 Processor -- start 2024-06-10T01:22:54.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.675+0000 7f931abac700 1 -- start start 2024-06-10T01:22:54.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.675+0000 7f931abac700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f931406aa00 0x7f931419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:54.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.675+0000 7f931abac700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f931406b3a0 0x7f931419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:54.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.675+0000 7f931abac700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93141084d0 0x7f93141989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:54.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.675+0000 7f931abac700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9314112750 con 0x7f931406b3a0 2024-06-10T01:22:54.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.675+0000 7f931abac700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f93141125d0 con 0x7f931406aa00 2024-06-10T01:22:54.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.675+0000 7f931abac700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f93141128d0 con 0x7f93141084d0 2024-06-10T01:22:54.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.675+0000 7f9313fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f931406b3a0 0x7f931419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:54.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.675+0000 7f9313fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f931406b3a0 0x7f931419ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35514/0 (socket says 172.21.15.7:35514) 2024-06-10T01:22:54.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.675+0000 7f9313fff700 1 -- 172.21.15.7:0/2022948881 learned_addr learned my addr 172.21.15.7:0/2022948881 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:54.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.675+0000 7f9318948700 1 --2- 172.21.15.7:0/2022948881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f931406aa00 0x7f931419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:54.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.675+0000 7f9319149700 1 --2- 172.21.15.7:0/2022948881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93141084d0 0x7f93141989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:54.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.676+0000 7f9313fff700 1 -- 172.21.15.7:0/2022948881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93141084d0 msgr2=0x7f93141989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:54.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.676+0000 7f9313fff700 1 --2- 172.21.15.7:0/2022948881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93141084d0 0x7f93141989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.676+0000 7f9313fff700 1 -- 172.21.15.7:0/2022948881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f931406aa00 msgr2=0x7f931419e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:54.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.676+0000 7f9313fff700 1 --2- 172.21.15.7:0/2022948881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f931406aa00 0x7f931419e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.676+0000 7f9313fff700 1 -- 172.21.15.7:0/2022948881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9314199130 con 0x7f931406b3a0 2024-06-10T01:22:54.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.676+0000 7f9319149700 1 --2- 172.21.15.7:0/2022948881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93141084d0 0x7f93141989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:22:54.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.676+0000 7f9318948700 1 --2- 172.21.15.7:0/2022948881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f931406aa00 0x7f931419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:22:54.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.676+0000 7f9313fff700 1 --2- 172.21.15.7:0/2022948881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f931406b3a0 0x7f931419ed50 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f930000d980 tx=0x7f930000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:54.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.676+0000 7f9311ffb700 1 -- 172.21.15.7:0/2022948881 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9300017070 con 0x7f931406b3a0 2024-06-10T01:22:54.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.676+0000 7f9311ffb700 1 -- 172.21.15.7:0/2022948881 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9300005020 con 0x7f931406b3a0 2024-06-10T01:22:54.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.676+0000 7f931abac700 1 -- 172.21.15.7:0/2022948881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9314199420 con 0x7f931406b3a0 2024-06-10T01:22:54.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.676+0000 7f9311ffb700 1 -- 172.21.15.7:0/2022948881 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930000fbf0 con 0x7f931406b3a0 2024-06-10T01:22:54.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.676+0000 7f931abac700 1 -- 172.21.15.7:0/2022948881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9314101380 con 0x7f931406b3a0 2024-06-10T01:22:54.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.679+0000 7f9311ffb700 1 -- 172.21.15.7:0/2022948881 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f93000051d0 con 0x7f931406b3a0 2024-06-10T01:22:54.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.680+0000 7f9311ffb700 1 --2- 172.21.15.7:0/2022948881 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f93040a2ca0 0x7f93040a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:54.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.680+0000 7f9311ffb700 1 -- 172.21.15.7:0/2022948881 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f93000cf190 con 0x7f931406b3a0 2024-06-10T01:22:54.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.680+0000 7f9318948700 1 --2- 172.21.15.7:0/2022948881 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f93040a2ca0 0x7f93040a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:54.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.680+0000 7f9318948700 1 --2- 172.21.15.7:0/2022948881 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f93040a2ca0 0x7f93040a5160 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9314199b10 tx=0x7f9308009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:54.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.682+0000 7f932542fb80 1 Processor -- start 2024-06-10T01:22:54.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.682+0000 7f932542fb80 1 -- start start 2024-06-10T01:22:54.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.682+0000 7f932542fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a40c1df60 0x555a40c1e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:54.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.682+0000 7f932542fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a40c1e880 0x555a40c18260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:54.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.682+0000 7f932542fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a40c187a0 0x555a40c18bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:54.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.682+0000 7f932542fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555a40b0af60 con 0x555a40c187a0 2024-06-10T01:22:54.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.682+0000 7f932542fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555a40b69850 con 0x555a40c1df60 2024-06-10T01:22:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.682+0000 7f932542fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555a40ad1460 con 0x555a40c1e880 2024-06-10T01:22:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.682+0000 7f9319149700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a40c187a0 0x555a40c18bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.682+0000 7f9319149700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a40c187a0 0x555a40c18bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35528/0 (socket says 172.21.15.7:35528) 2024-06-10T01:22:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.682+0000 7f9319149700 1 -- 172.21.15.7:0/3719738857 learned_addr learned my addr 172.21.15.7:0/3719738857 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.682+0000 7f9318948700 1 --2- 172.21.15.7:0/3719738857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a40c1df60 0x555a40c1e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.683+0000 7f9313fff700 1 --2- 172.21.15.7:0/3719738857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a40c1e880 0x555a40c18260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.683+0000 7f9318948700 1 -- 172.21.15.7:0/3719738857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a40c1e880 msgr2=0x555a40c18260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.683+0000 7f9318948700 1 --2- 172.21.15.7:0/3719738857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a40c1e880 0x555a40c18260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.683+0000 7f9318948700 1 -- 172.21.15.7:0/3719738857 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a40c187a0 msgr2=0x555a40c18bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.683+0000 7f9318948700 1 --2- 172.21.15.7:0/3719738857 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a40c187a0 0x555a40c18bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.683+0000 7f9318948700 1 -- 172.21.15.7:0/3719738857 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a40ab88d0 con 0x555a40c1df60 2024-06-10T01:22:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.683+0000 7f9319149700 1 --2- 172.21.15.7:0/3719738857 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a40c187a0 0x555a40c18bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:22:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.683+0000 7f9318948700 1 --2- 172.21.15.7:0/3719738857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a40c1df60 0x555a40c1e340 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9308004300 tx=0x7f9308014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a7ec6a4684cec10 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:54.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.683+0000 7f92feffd700 1 -- 172.21.15.7:0/3719738857 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9308003d10 con 0x555a40c1df60 2024-06-10T01:22:54.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.683+0000 7f92feffd700 1 -- 172.21.15.7:0/3719738857 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f930801e900 con 0x555a40c1df60 2024-06-10T01:22:54.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.683+0000 7f92feffd700 1 -- 172.21.15.7:0/3719738857 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9308026890 con 0x555a40c1df60 2024-06-10T01:22:54.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.684+0000 7f932542fb80 1 -- 172.21.15.7:0/3719738857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a40c1df60 msgr2=0x555a40c1e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:54.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.684+0000 7f932542fb80 1 --2- 172.21.15.7:0/3719738857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a40c1df60 0x555a40c1e340 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9308004300 tx=0x7f9308014000 comp rx=0 tx=0).stop 2024-06-10T01:22:54.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.684+0000 7f932542fb80 1 -- 172.21.15.7:0/3719738857 shutdown_connections 2024-06-10T01:22:54.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.684+0000 7f932542fb80 1 --2- 172.21.15.7:0/3719738857 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a40c187a0 0x555a40c18bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.684+0000 7f932542fb80 1 --2- 172.21.15.7:0/3719738857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a40c1df60 0x555a40c1e340 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.684+0000 7f932542fb80 1 --2- 172.21.15.7:0/3719738857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a40c1e880 0x555a40c18260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.684+0000 7f932542fb80 1 -- 172.21.15.7:0/3719738857 >> 172.21.15.7:0/3719738857 conn(0x555a40b45080 msgr2=0x555a40c13220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:54.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.684+0000 7f932542fb80 1 -- 172.21.15.7:0/3719738857 shutdown_connections 2024-06-10T01:22:54.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.684+0000 7f932542fb80 1 -- 172.21.15.7:0/3719738857 wait complete. 2024-06-10T01:22:54.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.685+0000 7f932542fb80 1 -- start start 2024-06-10T01:22:54.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.685+0000 7f932542fb80 1 Processor -- start 2024-06-10T01:22:54.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.685+0000 7f932542fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a40c187a0 0x555a40c24670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:54.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.685+0000 7f932542fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a40c1df60 0x555a40c21970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:54.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.685+0000 7f932542fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a40c1e880 0x555a40c21eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:54.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.685+0000 7f932542fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555a40b09fc0 con 0x555a40c1e880 2024-06-10T01:22:54.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.685+0000 7f932542fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555a40b65810 con 0x555a40c1df60 2024-06-10T01:22:54.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.685+0000 7f932542fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555a40c25260 con 0x555a40c187a0 2024-06-10T01:22:54.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.685+0000 7f932542fb80 10 client.0 _ll_register_callbacks cb 0x555a40aff5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:22:54.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.685+0000 7f9319149700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a40c1e880 0x555a40c21eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:54.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.685+0000 7f9318948700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a40c187a0 0x555a40c24670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:54.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.685+0000 7f9319149700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a40c1e880 0x555a40c21eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35544/0 (socket says 172.21.15.7:35544) 2024-06-10T01:22:54.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.685+0000 7f9319149700 1 -- 172.21.15.7:0/934551553 learned_addr learned my addr 172.21.15.7:0/934551553 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:54.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.685+0000 7f9313fff700 1 --2- 172.21.15.7:0/934551553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a40c1df60 0x555a40c21970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:54.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.686+0000 7f9319149700 1 -- 172.21.15.7:0/934551553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a40c187a0 msgr2=0x555a40c24670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:54.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.686+0000 7f9319149700 1 --2- 172.21.15.7:0/934551553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a40c187a0 0x555a40c24670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.686+0000 7f9319149700 1 -- 172.21.15.7:0/934551553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a40c1df60 msgr2=0x555a40c21970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:54.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.686+0000 7f9319149700 1 --2- 172.21.15.7:0/934551553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a40c1df60 0x555a40c21970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.686+0000 7f9319149700 1 -- 172.21.15.7:0/934551553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a40b2f750 con 0x555a40c1e880 2024-06-10T01:22:54.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.686+0000 7f9313fff700 1 --2- 172.21.15.7:0/934551553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a40c1df60 0x555a40c21970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:22:54.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.686+0000 7f9319149700 1 --2- 172.21.15.7:0/934551553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a40c1e880 0x555a40c21eb0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f930c004c60 tx=0x7f930c004c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe5ae70b8abc220 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:54.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.686+0000 7f92f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:22:54.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.686+0000 7f92f3fff700 1 -- 172.21.15.7:0/934551553 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c01cad0 con 0x555a40c1e880 2024-06-10T01:22:54.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.686+0000 7f932542fb80 1 -- 172.21.15.7:0/934551553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f930801e900 con 0x555a40c1e880 2024-06-10T01:22:54.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.686+0000 7f92f3fff700 1 -- 172.21.15.7:0/934551553 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f930c01cc70 con 0x555a40c1e880 2024-06-10T01:22:54.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.686+0000 7f92f3fff700 1 -- 172.21.15.7:0/934551553 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f930c034050 con 0x555a40c1e880 2024-06-10T01:22:54.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.686+0000 7f932542fb80 10 client.19569 fetch_fsmap learned FSMap version 48 2024-06-10T01:22:54.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.686+0000 7f932542fb80 1 -- 172.21.15.7:0/934551553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x555a40c2fab0 con 0x555a40c1e880 2024-06-10T01:22:54.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.687+0000 7f92f3fff700 1 -- 172.21.15.7:0/934551553 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f930801e900 con 0x555a40c1e880 2024-06-10T01:22:54.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.687+0000 7f92f3fff700 1 -- 172.21.15.7:0/934551553 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 48) ==== 8354+0+0 (secure 0 0 0) 0x7f930c02f030 con 0x555a40c1e880 2024-06-10T01:22:54.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.687+0000 7f932542fb80 10 client.19569 fetch_fsmap finished waiting for FSMap version 48 2024-06-10T01:22:54.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.687+0000 7f932542fb80 10 client.19569 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:22:54.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.687+0000 7f932542fb80 20 client.19569 populate_metadata read hostname 'smithi007' 2024-06-10T01:22:54.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.687+0000 7f932542fb80 1 --2- 172.21.15.7:0/934551553 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555a40c32090 0x555a40c34480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:54.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.687+0000 7f932542fb80 4 client.19569 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:22:54.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.687+0000 7f932542fb80 1 -- 172.21.15.7:0/934551553 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a40b76d50 con 0x555a40c32090 2024-06-10T01:22:54.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.687+0000 7f9318948700 1 --2- 172.21.15.7:0/934551553 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555a40c32090 0x555a40c34480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:54.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.688+0000 7f9318948700 1 --2- 172.21.15.7:0/934551553 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555a40c32090 0x555a40c34480 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=966bb3c998247a95 server_cookie=d93093ba23a692af in_seq=0 out_seq=0 2024-06-10T01:22:54.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.688+0000 7f92f3fff700 10 client.19569 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:54.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.688+0000 7f92f3fff700 1 -- 172.21.15.7:0/934551553 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555a40b76d50 con 0x555a40c32090 2024-06-10T01:22:54.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.688+0000 7f92f3fff700 10 client.19569 handle_command_reply: tid=0 2024-06-10T01:22:54.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.688+0000 7f932542fb80 1 client.19569 shutdown 2024-06-10T01:22:54.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.688+0000 7f932542fb80 1 -- 172.21.15.7:0/934551553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a40c1e880 msgr2=0x555a40c21eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:54.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.688+0000 7f932542fb80 1 --2- 172.21.15.7:0/934551553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a40c1e880 0x555a40c21eb0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f930c004c60 tx=0x7f930c004c90 comp rx=0 tx=0).stop 2024-06-10T01:22:54.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.689+0000 7f932542fb80 1 -- 172.21.15.7:0/934551553 shutdown_connections 2024-06-10T01:22:54.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.689+0000 7f932542fb80 1 --2- 172.21.15.7:0/934551553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a40c1e880 0x555a40c21eb0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.689+0000 7f932542fb80 1 --2- 172.21.15.7:0/934551553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a40c187a0 0x555a40c24670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.689+0000 7f932542fb80 1 --2- 172.21.15.7:0/934551553 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555a40c32090 0x555a40c34480 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.689+0000 7f932542fb80 1 --2- 172.21.15.7:0/934551553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a40c1df60 0x555a40c21970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.689+0000 7f92f3fff700 0 client.19569 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:54.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.689+0000 7f932542fb80 1 -- 172.21.15.7:0/934551553 >> 172.21.15.7:0/934551553 conn(0x555a40b45080 msgr2=0x555a40c12700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:54.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.689+0000 7f932542fb80 1 -- 172.21.15.7:0/934551553 shutdown_connections 2024-06-10T01:22:54.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.689+0000 7f932542fb80 1 -- 172.21.15.7:0/934551553 wait complete. 2024-06-10T01:22:54.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.689+0000 7f932542fb80 20 client.19569 trim_cache size 0 max 16384 2024-06-10T01:22:54.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.710+0000 7f932542fb80 1 Processor -- start 2024-06-10T01:22:54.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.710+0000 7f932542fb80 1 -- start start 2024-06-10T01:22:54.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.710+0000 7f932542fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a40b45080 0x555a40c1f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:54.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.710+0000 7f932542fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a40c187a0 0x555a40c21270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:54.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.710+0000 7f932542fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a40c1df60 0x555a40c238b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:54.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.710+0000 7f932542fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f93141ae920 con 0x555a40b45080 2024-06-10T01:22:54.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.710+0000 7f932542fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f93141ac1b0 con 0x555a40c1df60 2024-06-10T01:22:54.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.710+0000 7f932542fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f93141aa000 con 0x555a40c187a0 2024-06-10T01:22:54.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.710+0000 7f9318948700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a40b45080 0x555a40c1f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:54.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.710+0000 7f9318948700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a40b45080 0x555a40c1f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35560/0 (socket says 172.21.15.7:35560) 2024-06-10T01:22:54.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.710+0000 7f9318948700 1 -- 172.21.15.7:0/4248482527 learned_addr learned my addr 172.21.15.7:0/4248482527 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:54.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.710+0000 7f9313fff700 1 --2- 172.21.15.7:0/4248482527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a40c187a0 0x555a40c21270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:54.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.710+0000 7f9319149700 1 --2- 172.21.15.7:0/4248482527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a40c1df60 0x555a40c238b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:54.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.710+0000 7f9318948700 1 -- 172.21.15.7:0/4248482527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a40c187a0 msgr2=0x555a40c21270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:54.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.710+0000 7f9318948700 1 --2- 172.21.15.7:0/4248482527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a40c187a0 0x555a40c21270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.710+0000 7f9318948700 1 -- 172.21.15.7:0/4248482527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a40c1df60 msgr2=0x555a40c238b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:54.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.710+0000 7f9318948700 1 --2- 172.21.15.7:0/4248482527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a40c1df60 0x555a40c238b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.710+0000 7f9318948700 1 -- 172.21.15.7:0/4248482527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a40c347c0 con 0x555a40b45080 2024-06-10T01:22:54.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.711+0000 7f9318948700 1 --2- 172.21.15.7:0/4248482527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a40b45080 0x555a40c1f4f0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f93080042d0 tx=0x7f9308017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ad1fa11d3b2e68 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:54.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.711+0000 7f92fe7fc700 1 -- 172.21.15.7:0/4248482527 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9308003d10 con 0x555a40b45080 2024-06-10T01:22:54.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.711+0000 7f92fe7fc700 1 -- 172.21.15.7:0/4248482527 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f930801ede0 con 0x555a40b45080 2024-06-10T01:22:54.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.711+0000 7f92fe7fc700 1 -- 172.21.15.7:0/4248482527 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93080046d0 con 0x555a40b45080 2024-06-10T01:22:54.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.712+0000 7f932542fb80 1 -- 172.21.15.7:0/4248482527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a40b45080 msgr2=0x555a40c1f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:54.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.712+0000 7f932542fb80 1 --2- 172.21.15.7:0/4248482527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a40b45080 0x555a40c1f4f0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f93080042d0 tx=0x7f9308017000 comp rx=0 tx=0).stop 2024-06-10T01:22:54.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.712+0000 7f932542fb80 1 -- 172.21.15.7:0/4248482527 shutdown_connections 2024-06-10T01:22:54.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.712+0000 7f932542fb80 1 --2- 172.21.15.7:0/4248482527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a40c1df60 0x555a40c238b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.712+0000 7f932542fb80 1 --2- 172.21.15.7:0/4248482527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a40b45080 0x555a40c1f4f0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.712+0000 7f932542fb80 1 --2- 172.21.15.7:0/4248482527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a40c187a0 0x555a40c21270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.712+0000 7f932542fb80 1 -- 172.21.15.7:0/4248482527 >> 172.21.15.7:0/4248482527 conn(0x555a40c1c270 msgr2=0x555a40c231a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:54.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.712+0000 7f932542fb80 1 -- 172.21.15.7:0/4248482527 shutdown_connections 2024-06-10T01:22:54.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.712+0000 7f932542fb80 1 -- 172.21.15.7:0/4248482527 wait complete. 2024-06-10T01:22:54.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.712+0000 7f932542fb80 1 -- start start 2024-06-10T01:22:54.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.712+0000 7f932542fb80 1 Processor -- start 2024-06-10T01:22:54.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.713+0000 7f932542fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a40b45080 0x555a40c2a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:54.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.713+0000 7f932542fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a40c187a0 0x555a40c2ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:54.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.713+0000 7f932542fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a40c1df60 0x555a40c39120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:54.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.713+0000 7f932542fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f93141aa670 con 0x555a40c1df60 2024-06-10T01:22:54.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.713+0000 7f932542fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f93141abae0 con 0x555a40c187a0 2024-06-10T01:22:54.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.713+0000 7f932542fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f93141aad40 con 0x555a40b45080 2024-06-10T01:22:54.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.713+0000 7f932542fb80 10 client.0 _ll_register_callbacks cb 0x555a40c22550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:22:54.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.713+0000 7f9313fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a40c187a0 0x555a40c2ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:54.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.713+0000 7f9318948700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a40b45080 0x555a40c2a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:54.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.713+0000 7f9313fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a40c187a0 0x555a40c2ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:37854/0 (socket says 172.21.15.7:37854) 2024-06-10T01:22:54.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.713+0000 7f9313fff700 1 -- 172.21.15.7:0/2882235322 learned_addr learned my addr 172.21.15.7:0/2882235322 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:54.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.713+0000 7f9319149700 1 --2- 172.21.15.7:0/2882235322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a40c1df60 0x555a40c39120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:54.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.713+0000 7f9318948700 1 -- 172.21.15.7:0/2882235322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a40c187a0 msgr2=0x555a40c2ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:54.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.713+0000 7f9318948700 1 --2- 172.21.15.7:0/2882235322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a40c187a0 0x555a40c2ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.713+0000 7f9318948700 1 -- 172.21.15.7:0/2882235322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a40c1df60 msgr2=0x555a40c39120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:54.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.713+0000 7f9318948700 1 --2- 172.21.15.7:0/2882235322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a40c1df60 0x555a40c39120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.713+0000 7f9318948700 1 -- 172.21.15.7:0/2882235322 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a40c39960 con 0x555a40b45080 2024-06-10T01:22:54.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.713+0000 7f9313fff700 1 --2- 172.21.15.7:0/2882235322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a40c187a0 0x555a40c2ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:22:54.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.713+0000 7f9318948700 1 --2- 172.21.15.7:0/2882235322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a40b45080 0x555a40c2a700 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9308004300 tx=0x7f9308009440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4c26e16fe8d0260 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:54.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.713+0000 7f92f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:22:54.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.714+0000 7f92f3fff700 1 -- 172.21.15.7:0/2882235322 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9308005600 con 0x555a40b45080 2024-06-10T01:22:54.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.714+0000 7f92f3fff700 1 -- 172.21.15.7:0/2882235322 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f930801e880 con 0x555a40b45080 2024-06-10T01:22:54.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.714+0000 7f932542fb80 1 -- 172.21.15.7:0/2882235322 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f930801ede0 con 0x555a40b45080 2024-06-10T01:22:54.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.714+0000 7f92f3fff700 1 -- 172.21.15.7:0/2882235322 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f930804d050 con 0x555a40b45080 2024-06-10T01:22:54.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.714+0000 7f932542fb80 10 client.28807 fetch_fsmap learned FSMap version 48 2024-06-10T01:22:54.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.714+0000 7f932542fb80 1 -- 172.21.15.7:0/2882235322 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x555a40c9c7f0 con 0x555a40b45080 2024-06-10T01:22:54.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.714+0000 7f92f3fff700 1 -- 172.21.15.7:0/2882235322 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f930801ede0 con 0x555a40b45080 2024-06-10T01:22:54.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.714+0000 7f92f3fff700 1 -- 172.21.15.7:0/2882235322 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 48) ==== 8354+0+0 (secure 0 0 0) 0x7f9308004940 con 0x555a40b45080 2024-06-10T01:22:54.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.714+0000 7f932542fb80 10 client.28807 fetch_fsmap finished waiting for FSMap version 48 2024-06-10T01:22:54.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.714+0000 7f932542fb80 10 client.28807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:22:54.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.714+0000 7f932542fb80 20 client.28807 populate_metadata read hostname 'smithi007' 2024-06-10T01:22:54.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.714+0000 7f932542fb80 1 --2- 172.21.15.7:0/2882235322 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555a40c1e880 0x555a40c9eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:54.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.714+0000 7f932542fb80 4 client.28807 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:22:54.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.714+0000 7f932542fb80 1 -- 172.21.15.7:0/2882235322 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555a40b712d0 con 0x555a40c1e880 2024-06-10T01:22:54.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.715+0000 7f9313fff700 1 --2- 172.21.15.7:0/2882235322 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555a40c1e880 0x555a40c9eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:54.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.715+0000 7f9313fff700 1 --2- 172.21.15.7:0/2882235322 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555a40c1e880 0x555a40c9eb10 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92c2d35a9e7ac9be server_cookie=4f727ede8e05dfe8 in_seq=0 out_seq=0 2024-06-10T01:22:54.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.715+0000 7f92f3fff700 10 client.28807 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:54.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.715+0000 7f92f3fff700 1 -- 172.21.15.7:0/2882235322 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555a40b712d0 con 0x555a40c1e880 2024-06-10T01:22:54.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.715+0000 7f92f3fff700 10 client.28807 handle_command_reply: tid=0 2024-06-10T01:22:54.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.715+0000 7f932542fb80 1 client.28807 shutdown 2024-06-10T01:22:54.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.715+0000 7f932542fb80 1 -- 172.21.15.7:0/2882235322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a40b45080 msgr2=0x555a40c2a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:54.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.715+0000 7f932542fb80 1 --2- 172.21.15.7:0/2882235322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a40b45080 0x555a40c2a700 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9308004300 tx=0x7f9308009440 comp rx=0 tx=0).stop 2024-06-10T01:22:54.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.715+0000 7f932542fb80 1 -- 172.21.15.7:0/2882235322 shutdown_connections 2024-06-10T01:22:54.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.715+0000 7f932542fb80 1 --2- 172.21.15.7:0/2882235322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a40c1df60 0x555a40c39120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.715+0000 7f932542fb80 1 --2- 172.21.15.7:0/2882235322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555a40b45080 0x555a40c2a700 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.715+0000 7f932542fb80 1 --2- 172.21.15.7:0/2882235322 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555a40c1e880 0x555a40c9eb10 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.715+0000 7f932542fb80 1 --2- 172.21.15.7:0/2882235322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a40c187a0 0x555a40c2ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.715+0000 7f92f3fff700 0 client.28807 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:54.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.715+0000 7f932542fb80 1 -- 172.21.15.7:0/2882235322 >> 172.21.15.7:0/2882235322 conn(0x555a40c1c270 msgr2=0x555a40c1d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:54.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.716+0000 7f932542fb80 1 -- 172.21.15.7:0/2882235322 shutdown_connections 2024-06-10T01:22:54.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.716+0000 7f932542fb80 1 -- 172.21.15.7:0/2882235322 wait complete. 2024-06-10T01:22:54.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.716+0000 7f932542fb80 20 client.28807 trim_cache size 0 max 16384 2024-06-10T01:22:54.713 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:22:54.714 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-10T01:22:54.714 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-10T01:22:54.714 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:22:54.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.716+0000 7f931abac700 1 -- 172.21.15.7:0/2022948881 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f93040a2ca0 msgr2=0x7f93040a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:54.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.716+0000 7f931abac700 1 --2- 172.21.15.7:0/2022948881 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f93040a2ca0 0x7f93040a5160 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9314199b10 tx=0x7f9308009260 comp rx=0 tx=0).stop 2024-06-10T01:22:54.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.716+0000 7f931abac700 1 -- 172.21.15.7:0/2022948881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f931406b3a0 msgr2=0x7f931419ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:54.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.716+0000 7f931abac700 1 --2- 172.21.15.7:0/2022948881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f931406b3a0 0x7f931419ed50 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f930000d980 tx=0x7f930000dc90 comp rx=0 tx=0).stop 2024-06-10T01:22:54.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.716+0000 7f931abac700 1 -- 172.21.15.7:0/2022948881 shutdown_connections 2024-06-10T01:22:54.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.716+0000 7f931abac700 1 --2- 172.21.15.7:0/2022948881 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f93040a2ca0 0x7f93040a5160 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.716+0000 7f931abac700 1 --2- 172.21.15.7:0/2022948881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93141084d0 0x7f93141989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.716+0000 7f931abac700 1 --2- 172.21.15.7:0/2022948881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f931406aa00 0x7f931419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.716+0000 7f931abac700 1 --2- 172.21.15.7:0/2022948881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f931406b3a0 0x7f931419ed50 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:54.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.716+0000 7f931abac700 1 -- 172.21.15.7:0/2022948881 >> 172.21.15.7:0/2022948881 conn(0x7f9314078330 msgr2=0x7f93140ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:54.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.716+0000 7f931abac700 1 -- 172.21.15.7:0/2022948881 shutdown_connections 2024-06-10T01:22:54.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:54.716+0000 7f931abac700 1 -- 172.21.15.7:0/2022948881 wait complete. 2024-06-10T01:22:55.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:54 smithi007 ceph-mon[32398]: pgmap v939: 129 pgs: 129 active+clean; 589 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.9 MiB/s wr, 605 op/s 2024-06-10T01:22:55.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:54 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:22:55.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:54 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:22:55.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:54 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:22:55.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:54 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:22:55.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:54 smithi078 ceph-mon[36683]: pgmap v939: 129 pgs: 129 active+clean; 589 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.9 MiB/s wr, 605 op/s 2024-06-10T01:22:55.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:54 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:22:55.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:54 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:22:55.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:54 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:22:55.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:54 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:22:55.397 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:22:55.397 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:22:55.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:54 smithi175 ceph-mon[36636]: pgmap v939: 129 pgs: 129 active+clean; 589 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.9 MiB/s wr, 605 op/s 2024-06-10T01:22:55.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:54 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:22:55.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:54 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:22:55.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:54 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:22:55.452 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:54 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:22:55.529 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 30.39 MB/sec execute 92 sec latency 8127.516 ms 2024-06-10T01:22:55.647 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:22:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.236+0000 7f5e63363640 1 -- 172.21.15.7:0/561800701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e5c1035e0 msgr2=0x7f5e5c109e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.236+0000 7f5e63363640 1 --2- 172.21.15.7:0/561800701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e5c1035e0 0x7f5e5c109e70 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009ca0 tx=0x7f5e5402a8c0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.236+0000 7f5e63363640 1 -- 172.21.15.7:0/561800701 shutdown_connections 2024-06-10T01:22:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.236+0000 7f5e63363640 1 --2- 172.21.15.7:0/561800701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e5c1035e0 0x7f5e5c109e70 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.236+0000 7f5e63363640 1 --2- 172.21.15.7:0/561800701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e5c102c20 0x7f5e5c1030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.236+0000 7f5e63363640 1 --2- 172.21.15.7:0/561800701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e5c101a30 0x7f5e5c101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.236+0000 7f5e63363640 1 -- 172.21.15.7:0/561800701 >> 172.21.15.7:0/561800701 conn(0x7f5e5c0fcfc0 msgr2=0x7f5e5c0ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.236+0000 7f5e63363640 1 -- 172.21.15.7:0/561800701 shutdown_connections 2024-06-10T01:22:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.236+0000 7f5e63363640 1 -- 172.21.15.7:0/561800701 wait complete. 2024-06-10T01:22:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.237+0000 7f5e63363640 1 Processor -- start 2024-06-10T01:22:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.237+0000 7f5e63363640 1 -- start start 2024-06-10T01:22:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.237+0000 7f5e63363640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e5c101a30 0x7f5e5c197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.237+0000 7f5e63363640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e5c102c20 0x7f5e5c198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.237+0000 7f5e63363640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e5c1035e0 0x7f5e5c19f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.237+0000 7f5e63363640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5c10d710 con 0x7f5e5c101a30 2024-06-10T01:22:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.237+0000 7f5e63363640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5c10d590 con 0x7f5e5c102c20 2024-06-10T01:22:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.237+0000 7f5e63363640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5c10d890 con 0x7f5e5c1035e0 2024-06-10T01:22:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.237+0000 7f5e610ff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e5c101a30 0x7f5e5c197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.237+0000 7f5e610ff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e5c101a30 0x7f5e5c197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35586/0 (socket says 172.21.15.7:35586) 2024-06-10T01:22:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.237+0000 7f5e610ff640 1 -- 172.21.15.7:0/2968800849 learned_addr learned my addr 172.21.15.7:0/2968800849 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.237+0000 7f5e61900640 1 --2- 172.21.15.7:0/2968800849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e5c1035e0 0x7f5e5c19f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.237+0000 7f5e608fe640 1 --2- 172.21.15.7:0/2968800849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e5c102c20 0x7f5e5c198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.237+0000 7f5e610ff640 1 -- 172.21.15.7:0/2968800849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e5c1035e0 msgr2=0x7f5e5c19f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.237+0000 7f5e610ff640 1 --2- 172.21.15.7:0/2968800849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e5c1035e0 0x7f5e5c19f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.237+0000 7f5e610ff640 1 -- 172.21.15.7:0/2968800849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e5c102c20 msgr2=0x7f5e5c198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.237+0000 7f5e610ff640 1 --2- 172.21.15.7:0/2968800849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e5c102c20 0x7f5e5c198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.237+0000 7f5e610ff640 1 -- 172.21.15.7:0/2968800849 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e5c19f900 con 0x7f5e5c101a30 2024-06-10T01:22:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.237+0000 7f5e610ff640 1 --2- 172.21.15.7:0/2968800849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e5c101a30 0x7f5e5c197cf0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800c970 tx=0x7f5e5800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.238+0000 7f5e527fc640 1 -- 172.21.15.7:0/2968800849 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58007c10 con 0x7f5e5c101a30 2024-06-10T01:22:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.238+0000 7f5e527fc640 1 -- 172.21.15.7:0/2968800849 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e58007db0 con 0x7f5e5c101a30 2024-06-10T01:22:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.238+0000 7f5e63363640 1 -- 172.21.15.7:0/2968800849 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e5c19fbf0 con 0x7f5e5c101a30 2024-06-10T01:22:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.238+0000 7f5e527fc640 1 -- 172.21.15.7:0/2968800849 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5800f940 con 0x7f5e5c101a30 2024-06-10T01:22:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.238+0000 7f5e63363640 1 -- 172.21.15.7:0/2968800849 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e5c1a0190 con 0x7f5e5c101a30 2024-06-10T01:22:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.240+0000 7f5e527fc640 1 -- 172.21.15.7:0/2968800849 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f5e5800fae0 con 0x7f5e5c101a30 2024-06-10T01:22:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.240+0000 7f5e6607b740 1 Processor -- start 2024-06-10T01:22:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.240+0000 7f5e6607b740 1 -- start start 2024-06-10T01:22:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.241+0000 7f5e6607b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558745449010 0x558745449430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.241+0000 7f5e6607b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558745449a00 0x55874544e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.241+0000 7f5e6607b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874544eb70 0x558745451000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.241+0000 7f5e6607b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5587453ac4f0 con 0x55874544eb70 2024-06-10T01:22:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.241+0000 7f5e6607b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55874534a870 con 0x558745449a00 2024-06-10T01:22:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.241+0000 7f5e6607b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5587453a84b0 con 0x558745449010 2024-06-10T01:22:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.241+0000 7f5e527fc640 1 --2- 172.21.15.7:0/2968800849 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5e480a2c00 0x7f5e480a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.241+0000 7f5e608fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874544eb70 0x558745451000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.241+0000 7f5e610ff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558745449a00 0x55874544e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.241+0000 7f5e61900640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558745449010 0x558745449430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.241+0000 7f5e608fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874544eb70 0x558745451000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35594/0 (socket says 172.21.15.7:35594) 2024-06-10T01:22:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.241+0000 7f5e608fe640 1 -- 172.21.15.7:0/1317907405 learned_addr learned my addr 172.21.15.7:0/1317907405 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.241+0000 7f5e61900640 1 --2- 172.21.15.7:0/2968800849 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5e480a2c00 0x7f5e480a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.241+0000 7f5e527fc640 1 -- 172.21.15.7:0/2968800849 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f5e580cf870 con 0x7f5e5c101a30 2024-06-10T01:22:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.241+0000 7f5e608fe640 1 -- 172.21.15.7:0/1317907405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558745449010 msgr2=0x558745449430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.241+0000 7f5e608fe640 1 --2- 172.21.15.7:0/1317907405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558745449010 0x558745449430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.241+0000 7f5e608fe640 1 -- 172.21.15.7:0/1317907405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558745449a00 msgr2=0x55874544e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.241+0000 7f5e608fe640 1 --2- 172.21.15.7:0/1317907405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558745449a00 0x55874544e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.241+0000 7f5e608fe640 1 -- 172.21.15.7:0/1317907405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587453669b0 con 0x55874544eb70 2024-06-10T01:22:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.241+0000 7f5e610ff640 1 --2- 172.21.15.7:0/1317907405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558745449a00 0x55874544e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:22:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.241+0000 7f5e61900640 1 --2- 172.21.15.7:0/2968800849 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5e480a2c00 0x7f5e480a50c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5e54031c10 tx=0x7f5e54005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.241+0000 7f5e608fe640 1 --2- 172.21.15.7:0/1317907405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874544eb70 0x558745451000 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00a920 tx=0x7f5e4c00ade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0269144277e8b88 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.242+0000 7f5e477fe640 1 -- 172.21.15.7:0/1317907405 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c015070 con 0x55874544eb70 2024-06-10T01:22:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.242+0000 7f5e477fe640 1 -- 172.21.15.7:0/1317907405 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e4c00cc00 con 0x55874544eb70 2024-06-10T01:22:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.242+0000 7f5e477fe640 1 -- 172.21.15.7:0/1317907405 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c0185c0 con 0x55874544eb70 2024-06-10T01:22:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.242+0000 7f5e6607b740 1 -- 172.21.15.7:0/1317907405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874544eb70 msgr2=0x558745451000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.242+0000 7f5e6607b740 1 --2- 172.21.15.7:0/1317907405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874544eb70 0x558745451000 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00a920 tx=0x7f5e4c00ade0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.242+0000 7f5e6607b740 1 -- 172.21.15.7:0/1317907405 shutdown_connections 2024-06-10T01:22:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.242+0000 7f5e6607b740 1 --2- 172.21.15.7:0/1317907405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874544eb70 0x558745451000 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.242+0000 7f5e6607b740 1 --2- 172.21.15.7:0/1317907405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558745449a00 0x55874544e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.242+0000 7f5e6607b740 1 --2- 172.21.15.7:0/1317907405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558745449010 0x558745449430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.242+0000 7f5e6607b740 1 -- 172.21.15.7:0/1317907405 >> 172.21.15.7:0/1317907405 conn(0x558745444680 msgr2=0x558745444a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.242+0000 7f5e6607b740 1 -- 172.21.15.7:0/1317907405 shutdown_connections 2024-06-10T01:22:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.242+0000 7f5e6607b740 1 -- 172.21.15.7:0/1317907405 wait complete. 2024-06-10T01:22:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.243+0000 7f5e6607b740 1 -- start start 2024-06-10T01:22:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.243+0000 7f5e6607b740 1 Processor -- start 2024-06-10T01:22:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.243+0000 7f5e6607b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558745449010 0x5587454531f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.243+0000 7f5e6607b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558745449a00 0x558745453730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.243+0000 7f5e6607b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874544eb70 0x55874545a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.243+0000 7f5e6607b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558745451b20 con 0x558745449a00 2024-06-10T01:22:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.243+0000 7f5e6607b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558745335ae0 con 0x55874544eb70 2024-06-10T01:22:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.243+0000 7f5e6607b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558745451ca0 con 0x558745449010 2024-06-10T01:22:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.243+0000 7f5e6607b740 10 client.0 _ll_register_callbacks cb 0x55874536e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:22:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.243+0000 7f5e608fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558745449a00 0x558745453730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.243+0000 7f5e608fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558745449a00 0x558745453730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35606/0 (socket says 172.21.15.7:35606) 2024-06-10T01:22:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.243+0000 7f5e608fe640 1 -- 172.21.15.7:0/1288959490 learned_addr learned my addr 172.21.15.7:0/1288959490 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.243+0000 7f5e610ff640 1 --2- 172.21.15.7:0/1288959490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558745449010 0x5587454531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.244+0000 7f5e61900640 1 --2- 172.21.15.7:0/1288959490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874544eb70 0x55874545a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:56.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.244+0000 7f5e608fe640 1 -- 172.21.15.7:0/1288959490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558745449010 msgr2=0x5587454531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:56.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.244+0000 7f5e608fe640 1 --2- 172.21.15.7:0/1288959490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558745449010 0x5587454531f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.244+0000 7f5e608fe640 1 -- 172.21.15.7:0/1288959490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874544eb70 msgr2=0x55874545a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:56.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.244+0000 7f5e608fe640 1 --2- 172.21.15.7:0/1288959490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874544eb70 0x55874545a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.244+0000 7f5e608fe640 1 -- 172.21.15.7:0/1288959490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587450c0210 con 0x558745449a00 2024-06-10T01:22:56.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.244+0000 7f5e608fe640 1 --2- 172.21.15.7:0/1288959490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558745449a00 0x558745453730 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c008010 tx=0x7f5e4c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=641aaee8973937a2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:56.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.244+0000 7f5e44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:22:56.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.244+0000 7f5e44ff9640 1 -- 172.21.15.7:0/1288959490 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c029060 con 0x558745449a00 2024-06-10T01:22:56.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.244+0000 7f5e44ff9640 1 -- 172.21.15.7:0/1288959490 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f5e4c00cdb0 con 0x558745449a00 2024-06-10T01:22:56.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.244+0000 7f5e6607b740 1 -- 172.21.15.7:0/1288959490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e4c00cc00 con 0x558745449a00 2024-06-10T01:22:56.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.244+0000 7f5e44ff9640 1 -- 172.21.15.7:0/1288959490 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f5e4c030050 con 0x558745449a00 2024-06-10T01:22:56.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.244+0000 7f5e6607b740 10 client.19587 fetch_fsmap learned FSMap version 48 2024-06-10T01:22:56.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.244+0000 7f5e6607b740 1 -- 172.21.15.7:0/1288959490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55874518c620 con 0x558745449a00 2024-06-10T01:22:56.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.244+0000 7f5e44ff9640 1 -- 172.21.15.7:0/1288959490 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e4c00cc00 con 0x558745449a00 2024-06-10T01:22:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.245+0000 7f5e44ff9640 1 -- 172.21.15.7:0/1288959490 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 48) ==== 8354+0+0 (secure 0 0 0) 0x7f5e4c024420 con 0x558745449a00 2024-06-10T01:22:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.245+0000 7f5e6607b740 10 client.19587 fetch_fsmap finished waiting for FSMap version 48 2024-06-10T01:22:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.245+0000 7f5e6607b740 10 client.19587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:22:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.245+0000 7f5e6607b740 20 client.19587 populate_metadata read hostname 'smithi007' 2024-06-10T01:22:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.245+0000 7f5e6607b740 1 --2- 172.21.15.7:0/1288959490 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55874545ecb0 0x5587454610a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.245+0000 7f5e6607b740 4 client.19587 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:22:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.245+0000 7f5e6607b740 1 -- 172.21.15.7:0/1288959490 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587453bff10 con 0x55874545ecb0 2024-06-10T01:22:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.245+0000 7f5e610ff640 1 --2- 172.21.15.7:0/1288959490 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55874545ecb0 0x5587454610a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.245+0000 7f5e44ff9640 10 client.19587 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.245+0000 7f5e610ff640 1 --2- 172.21.15.7:0/1288959490 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55874545ecb0 0x5587454610a0 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e83c2af9eb665c5 server_cookie=a3cfc26a2a8d25dc in_seq=0 out_seq=0 2024-06-10T01:22:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.246+0000 7f5e44ff9640 1 -- 172.21.15.7:0/1288959490 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5587453bff10 con 0x55874545ecb0 2024-06-10T01:22:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.246+0000 7f5e44ff9640 10 client.19587 handle_command_reply: tid=0 2024-06-10T01:22:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.246+0000 7f5e6607b740 1 client.19587 shutdown 2024-06-10T01:22:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.246+0000 7f5e6607b740 1 -- 172.21.15.7:0/1288959490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558745449a00 msgr2=0x558745453730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.246+0000 7f5e6607b740 1 --2- 172.21.15.7:0/1288959490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558745449a00 0x558745453730 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c008010 tx=0x7f5e4c011ad0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.246+0000 7f5e6607b740 1 -- 172.21.15.7:0/1288959490 shutdown_connections 2024-06-10T01:22:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.246+0000 7f5e6607b740 1 --2- 172.21.15.7:0/1288959490 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55874545ecb0 0x5587454610a0 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.246+0000 7f5e6607b740 1 --2- 172.21.15.7:0/1288959490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874544eb70 0x55874545a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.246+0000 7f5e6607b740 1 --2- 172.21.15.7:0/1288959490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558745449a00 0x558745453730 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.246+0000 7f5e6607b740 1 --2- 172.21.15.7:0/1288959490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558745449010 0x5587454531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.246+0000 7f5e44ff9640 0 client.19587 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.246+0000 7f5e6607b740 1 -- 172.21.15.7:0/1288959490 >> 172.21.15.7:0/1288959490 conn(0x558745444680 msgr2=0x5587454437e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.247+0000 7f5e6607b740 1 -- 172.21.15.7:0/1288959490 shutdown_connections 2024-06-10T01:22:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.247+0000 7f5e6607b740 1 -- 172.21.15.7:0/1288959490 wait complete. 2024-06-10T01:22:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.247+0000 7f5e6607b740 20 client.19587 trim_cache size 0 max 16384 2024-06-10T01:22:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.262+0000 7f5e6607b740 1 Processor -- start 2024-06-10T01:22:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.262+0000 7f5e6607b740 1 -- start start 2024-06-10T01:22:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.262+0000 7f5e6607b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558745449010 0x558745450f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.262+0000 7f5e6607b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558745449a00 0x55874545a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.262+0000 7f5e6607b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874544eb70 0x5587454531f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.262+0000 7f5e6607b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5c1aceb0 con 0x55874544eb70 2024-06-10T01:22:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.262+0000 7f5e6607b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5c1aa780 con 0x558745449010 2024-06-10T01:22:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.262+0000 7f5e6607b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5c1a85d0 con 0x558745449a00 2024-06-10T01:22:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.262+0000 7f5e61900640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874544eb70 0x5587454531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.262+0000 7f5e61900640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874544eb70 0x5587454531f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35612/0 (socket says 172.21.15.7:35612) 2024-06-10T01:22:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.262+0000 7f5e61900640 1 -- 172.21.15.7:0/4208572094 learned_addr learned my addr 172.21.15.7:0/4208572094 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.263+0000 7f5e610ff640 1 --2- 172.21.15.7:0/4208572094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558745449010 0x558745450f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.263+0000 7f5e608fe640 1 --2- 172.21.15.7:0/4208572094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558745449a00 0x55874545a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.263+0000 7f5e61900640 1 -- 172.21.15.7:0/4208572094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558745449a00 msgr2=0x55874545a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.263+0000 7f5e61900640 1 --2- 172.21.15.7:0/4208572094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558745449a00 0x55874545a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.263+0000 7f5e61900640 1 -- 172.21.15.7:0/4208572094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558745449010 msgr2=0x558745450f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.263+0000 7f5e61900640 1 --2- 172.21.15.7:0/4208572094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558745449010 0x558745450f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.263+0000 7f5e61900640 1 -- 172.21.15.7:0/4208572094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587454444b0 con 0x55874544eb70 2024-06-10T01:22:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.263+0000 7f5e610ff640 1 --2- 172.21.15.7:0/4208572094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558745449010 0x558745450f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:22:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.263+0000 7f5e61900640 1 --2- 172.21.15.7:0/4208572094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874544eb70 0x5587454531f0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f5e54035000 tx=0x7f5e54040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4388a77e733771f0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.263+0000 7f5e46ffd640 1 -- 172.21.15.7:0/4208572094 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e54003970 con 0x55874544eb70 2024-06-10T01:22:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.263+0000 7f5e46ffd640 1 -- 172.21.15.7:0/4208572094 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e54050070 con 0x55874544eb70 2024-06-10T01:22:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.263+0000 7f5e46ffd640 1 -- 172.21.15.7:0/4208572094 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5402f870 con 0x55874544eb70 2024-06-10T01:22:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.264+0000 7f5e6607b740 1 -- 172.21.15.7:0/4208572094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874544eb70 msgr2=0x5587454531f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.264+0000 7f5e6607b740 1 --2- 172.21.15.7:0/4208572094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874544eb70 0x5587454531f0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f5e54035000 tx=0x7f5e54040000 comp rx=0 tx=0).stop 2024-06-10T01:22:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.264+0000 7f5e6607b740 1 -- 172.21.15.7:0/4208572094 shutdown_connections 2024-06-10T01:22:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.264+0000 7f5e6607b740 1 --2- 172.21.15.7:0/4208572094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874544eb70 0x5587454531f0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.264+0000 7f5e6607b740 1 --2- 172.21.15.7:0/4208572094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558745449a00 0x55874545a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.264+0000 7f5e6607b740 1 --2- 172.21.15.7:0/4208572094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558745449010 0x558745450f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.264+0000 7f5e6607b740 1 -- 172.21.15.7:0/4208572094 >> 172.21.15.7:0/4208572094 conn(0x558745444680 msgr2=0x55874544e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.264+0000 7f5e6607b740 1 -- 172.21.15.7:0/4208572094 shutdown_connections 2024-06-10T01:22:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.264+0000 7f5e6607b740 1 -- 172.21.15.7:0/4208572094 wait complete. 2024-06-10T01:22:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.264+0000 7f5e6607b740 1 -- start start 2024-06-10T01:22:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.264+0000 7f5e6607b740 1 Processor -- start 2024-06-10T01:22:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.264+0000 7f5e6607b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558745449010 0x55874545a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.264+0000 7f5e6607b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558745449a00 0x558745457160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.264+0000 7f5e6607b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874544eb70 0x5587454576a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.264+0000 7f5e6607b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5c1a8c40 con 0x558745449010 2024-06-10T01:22:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.264+0000 7f5e6607b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5c1aa0b0 con 0x55874544eb70 2024-06-10T01:22:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.264+0000 7f5e6607b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5c1a9310 con 0x558745449a00 2024-06-10T01:22:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.264+0000 7f5e6607b740 10 client.0 _ll_register_callbacks cb 0x7f5e58018e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:22:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.265+0000 7f5e610ff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558745449010 0x55874545a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.265+0000 7f5e610ff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558745449010 0x55874545a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35624/0 (socket says 172.21.15.7:35624) 2024-06-10T01:22:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.265+0000 7f5e610ff640 1 -- 172.21.15.7:0/97655486 learned_addr learned my addr 172.21.15.7:0/97655486 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.265+0000 7f5e61900640 1 --2- 172.21.15.7:0/97655486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874544eb70 0x5587454576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.265+0000 7f5e610ff640 1 -- 172.21.15.7:0/97655486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558745449a00 msgr2=0x558745457160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.265+0000 7f5e610ff640 1 --2- 172.21.15.7:0/97655486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558745449a00 0x558745457160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.265+0000 7f5e610ff640 1 -- 172.21.15.7:0/97655486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874544eb70 msgr2=0x5587454576a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:22:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.265+0000 7f5e610ff640 1 --2- 172.21.15.7:0/97655486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874544eb70 0x5587454576a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.265+0000 7f5e610ff640 1 -- 172.21.15.7:0/97655486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587454c7950 con 0x558745449010 2024-06-10T01:22:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.265+0000 7f5e61900640 1 --2- 172.21.15.7:0/97655486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874544eb70 0x5587454576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:22:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.265+0000 7f5e610ff640 1 --2- 172.21.15.7:0/97655486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558745449010 0x55874545a6c0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f5e58018f30 tx=0x7f5e5806bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e07930d8e7319b1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.265+0000 7f5e44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:22:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.265+0000 7f5e44ff9640 1 -- 172.21.15.7:0/97655486 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e580a49c0 con 0x558745449010 2024-06-10T01:22:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.265+0000 7f5e44ff9640 1 -- 172.21.15.7:0/97655486 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f5e58099df0 con 0x558745449010 2024-06-10T01:22:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.265+0000 7f5e6607b740 1 -- 172.21.15.7:0/97655486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e54050070 con 0x558745449010 2024-06-10T01:22:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.265+0000 7f5e44ff9640 1 -- 172.21.15.7:0/97655486 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f5e5809a050 con 0x558745449010 2024-06-10T01:22:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.265+0000 7f5e6607b740 10 client.19593 fetch_fsmap learned FSMap version 48 2024-06-10T01:22:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.265+0000 7f5e6607b740 1 -- 172.21.15.7:0/97655486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5587454c8c70 con 0x558745449010 2024-06-10T01:22:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.265+0000 7f5e44ff9640 1 -- 172.21.15.7:0/97655486 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e54050070 con 0x558745449010 2024-06-10T01:22:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.266+0000 7f5e44ff9640 1 -- 172.21.15.7:0/97655486 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 48) ==== 8354+0+0 (secure 0 0 0) 0x7f5e5809b1a0 con 0x558745449010 2024-06-10T01:22:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.266+0000 7f5e6607b740 10 client.19593 fetch_fsmap finished waiting for FSMap version 48 2024-06-10T01:22:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.266+0000 7f5e6607b740 10 client.19593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:22:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.266+0000 7f5e6607b740 20 client.19593 populate_metadata read hostname 'smithi007' 2024-06-10T01:22:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.266+0000 7f5e6607b740 1 --2- 172.21.15.7:0/97655486 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5587454c95a0 0x5587454cb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.266+0000 7f5e6607b740 4 client.19593 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:22:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.266+0000 7f5e6607b740 1 -- 172.21.15.7:0/97655486 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5587453b9c20 con 0x5587454c95a0 2024-06-10T01:22:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.266+0000 7f5e608fe640 1 --2- 172.21.15.7:0/97655486 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5587454c95a0 0x5587454cb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.266+0000 7f5e44ff9640 10 client.19593 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.266+0000 7f5e608fe640 1 --2- 172.21.15.7:0/97655486 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5587454c95a0 0x5587454cb9c0 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e6defd8ed87475 server_cookie=2a26994cef9b65cb in_seq=0 out_seq=0 2024-06-10T01:22:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.529+0000 7f5e44ff9640 1 -- 172.21.15.7:0/97655486 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5587453b9c20 con 0x5587454c95a0 2024-06-10T01:22:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.529+0000 7f5e44ff9640 10 client.19593 handle_command_reply: tid=0 2024-06-10T01:22:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.529+0000 7f5e6607b740 1 client.19593 shutdown 2024-06-10T01:22:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.529+0000 7f5e6607b740 1 -- 172.21.15.7:0/97655486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558745449010 msgr2=0x55874545a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.529+0000 7f5e6607b740 1 --2- 172.21.15.7:0/97655486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558745449010 0x55874545a6c0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f5e58018f30 tx=0x7f5e5806bc00 comp rx=0 tx=0).stop 2024-06-10T01:22:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.529+0000 7f5e6607b740 1 -- 172.21.15.7:0/97655486 shutdown_connections 2024-06-10T01:22:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.529+0000 7f5e6607b740 1 --2- 172.21.15.7:0/97655486 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5587454c95a0 0x5587454cb9c0 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.529+0000 7f5e6607b740 1 --2- 172.21.15.7:0/97655486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874544eb70 0x5587454576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.529+0000 7f5e6607b740 1 --2- 172.21.15.7:0/97655486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558745449a00 0x558745457160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.529+0000 7f5e6607b740 1 --2- 172.21.15.7:0/97655486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558745449010 0x55874545a6c0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.529+0000 7f5e6607b740 1 -- 172.21.15.7:0/97655486 >> 172.21.15.7:0/97655486 conn(0x558745444680 msgr2=0x55874544abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.529+0000 7f5e44ff9640 0 client.19593 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:22:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.530+0000 7f5e6607b740 1 -- 172.21.15.7:0/97655486 shutdown_connections 2024-06-10T01:22:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.530+0000 7f5e6607b740 1 -- 172.21.15.7:0/97655486 wait complete. 2024-06-10T01:22:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.530+0000 7f5e6607b740 20 client.19593 trim_cache size 0 max 16384 2024-06-10T01:22:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:22:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:22:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:22:56.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.531+0000 7f5e63363640 1 -- 172.21.15.7:0/2968800849 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5e480a2c00 msgr2=0x7f5e480a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:56.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.531+0000 7f5e63363640 1 --2- 172.21.15.7:0/2968800849 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5e480a2c00 0x7f5e480a50c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5e54031c10 tx=0x7f5e54005ef0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.531+0000 7f5e63363640 1 -- 172.21.15.7:0/2968800849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e5c101a30 msgr2=0x7f5e5c197cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:56.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.531+0000 7f5e63363640 1 --2- 172.21.15.7:0/2968800849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e5c101a30 0x7f5e5c197cf0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800c970 tx=0x7f5e5800ce30 comp rx=0 tx=0).stop 2024-06-10T01:22:56.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.531+0000 7f5e63363640 1 -- 172.21.15.7:0/2968800849 shutdown_connections 2024-06-10T01:22:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.531+0000 7f5e63363640 1 --2- 172.21.15.7:0/2968800849 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5e480a2c00 0x7f5e480a50c0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.531+0000 7f5e63363640 1 --2- 172.21.15.7:0/2968800849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e5c1035e0 0x7f5e5c19f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.531+0000 7f5e63363640 1 --2- 172.21.15.7:0/2968800849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e5c102c20 0x7f5e5c198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.531+0000 7f5e63363640 1 --2- 172.21.15.7:0/2968800849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e5c101a30 0x7f5e5c197cf0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.531+0000 7f5e63363640 1 -- 172.21.15.7:0/2968800849 >> 172.21.15.7:0/2968800849 conn(0x7f5e5c0fcfc0 msgr2=0x7f5e5c105ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.531+0000 7f5e63363640 1 -- 172.21.15.7:0/2968800849 shutdown_connections 2024-06-10T01:22:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:22:56.531+0000 7f5e63363640 1 -- 172.21.15.7:0/2968800849 wait complete. 2024-06-10T01:22:56.529 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 30.07 MB/sec execute 93 sec latency 9127.553 ms 2024-06-10T01:22:57.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:56 smithi007 ceph-mon[32398]: pgmap v940: 129 pgs: 129 active+clean; 591 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.2 MiB/s wr, 569 op/s 2024-06-10T01:22:57.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:56 smithi078 ceph-mon[36683]: pgmap v940: 129 pgs: 129 active+clean; 591 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.2 MiB/s wr, 569 op/s 2024-06-10T01:22:57.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:56 smithi175 ceph-mon[36636]: pgmap v940: 129 pgs: 129 active+clean; 591 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.2 MiB/s wr, 569 op/s 2024-06-10T01:22:57.451 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:22:57 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-f[66185]: ignoring --setuser ceph since I am not root 2024-06-10T01:22:57.451 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:22:57 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-f[66185]: ignoring --setgroup ceph since I am not root 2024-06-10T01:22:57.452 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:22:57 smithi175 ceph-mds[66189]: -- 172.21.15.175:0/2001799417 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b2e5743a40 con 0x55b2e5716800 2024-06-10T01:22:57.452 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:22:57 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-f[66185]: starting mds.f at 2024-06-10T01:22:57.530 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 29.75 MB/sec execute 94 sec latency 10127.612 ms 2024-06-10T01:22:57.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.314+0000 7fd49413a700 1 -- 172.21.15.7:0/3045536351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd48c101380 msgr2=0x7fd48c101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:57.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.314+0000 7fd490ed4700 1 -- 172.21.15.7:0/3045536351 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c01d070 con 0x7fd48c101380 2024-06-10T01:22:57.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.314+0000 7fd49413a700 1 --2- 172.21.15.7:0/3045536351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd48c101380 0x7fd48c101800 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00b970 tx=0x7fd47c00bc80 comp rx=0 tx=0).stop 2024-06-10T01:22:57.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.315+0000 7fd49413a700 1 -- 172.21.15.7:0/3045536351 shutdown_connections 2024-06-10T01:22:57.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.315+0000 7fd49413a700 1 --2- 172.21.15.7:0/3045536351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd48c101d40 0x7fd48c10e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:57.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.315+0000 7fd49413a700 1 --2- 172.21.15.7:0/3045536351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd48c1073f0 0x7fd48c1077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:57.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.315+0000 7fd49413a700 1 --2- 172.21.15.7:0/3045536351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd48c101380 0x7fd48c101800 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:57.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.315+0000 7fd49413a700 1 -- 172.21.15.7:0/3045536351 >> 172.21.15.7:0/3045536351 conn(0x7fd48c078300 msgr2=0x7fd48c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:57.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.318+0000 7fd49413a700 1 -- 172.21.15.7:0/3045536351 shutdown_connections 2024-06-10T01:22:57.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.318+0000 7fd49413a700 1 -- 172.21.15.7:0/3045536351 wait complete. 2024-06-10T01:22:57.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.320+0000 7fd49413a700 1 Processor -- start 2024-06-10T01:22:57.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.320+0000 7fd49413a700 1 -- start start 2024-06-10T01:22:57.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.320+0000 7fd49413a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd48c101380 0x7fd48c1ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:57.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.321+0000 7fd49413a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd48c101d40 0x7fd48c1adc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:57.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.321+0000 7fd49413a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd48c1073f0 0x7fd48c1a7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:57.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.321+0000 7fd4916d5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd48c101d40 0x7fd48c1adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:57.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.321+0000 7fd4916d5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd48c101d40 0x7fd48c1adc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:54098/0 (socket says 172.21.15.7:54098) 2024-06-10T01:22:57.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.321+0000 7fd491ed6700 1 --2- 172.21.15.7:0/901859330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd48c101380 0x7fd48c1ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:57.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.321+0000 7fd4916d5700 1 -- 172.21.15.7:0/901859330 learned_addr learned my addr 172.21.15.7:0/901859330 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:22:57.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.321+0000 7fd4926d7700 1 --2- 172.21.15.7:0/901859330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd48c1073f0 0x7fd48c1a7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:57.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.321+0000 7fd49413a700 1 -- 172.21.15.7:0/901859330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c1216a0 con 0x7fd48c101380 2024-06-10T01:22:57.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.321+0000 7fd49413a700 1 -- 172.21.15.7:0/901859330 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c121520 con 0x7fd48c1073f0 2024-06-10T01:22:57.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.322+0000 7fd49413a700 1 -- 172.21.15.7:0/901859330 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c121820 con 0x7fd48c101d40 2024-06-10T01:22:57.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.326+0000 7fd4926d7700 1 -- 172.21.15.7:0/901859330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd48c101d40 msgr2=0x7fd48c1adc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:57.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.326+0000 7fd4926d7700 1 --2- 172.21.15.7:0/901859330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd48c101d40 0x7fd48c1adc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:57.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.326+0000 7fd4926d7700 1 -- 172.21.15.7:0/901859330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd48c101380 msgr2=0x7fd48c1ad6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:57.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.326+0000 7fd4926d7700 1 --2- 172.21.15.7:0/901859330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd48c101380 0x7fd48c1ad6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:57.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.326+0000 7fd4926d7700 1 -- 172.21.15.7:0/901859330 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48c1a8020 con 0x7fd48c1073f0 2024-06-10T01:22:57.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.326+0000 7fd4926d7700 1 --2- 172.21.15.7:0/901859330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd48c1073f0 0x7fd48c1a7840 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd48400c7a0 tx=0x7fd48400cab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:57.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.326+0000 7fd482ffd700 1 -- 172.21.15.7:0/901859330 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd484004a30 con 0x7fd48c1073f0 2024-06-10T01:22:57.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.326+0000 7fd482ffd700 1 -- 172.21.15.7:0/901859330 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd484005500 con 0x7fd48c1073f0 2024-06-10T01:22:57.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.326+0000 7fd482ffd700 1 -- 172.21.15.7:0/901859330 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48400ed80 con 0x7fd48c1073f0 2024-06-10T01:22:57.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.326+0000 7fd49413a700 1 -- 172.21.15.7:0/901859330 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48c1a8310 con 0x7fd48c1073f0 2024-06-10T01:22:57.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.326+0000 7fd49413a700 1 -- 172.21.15.7:0/901859330 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48c1b4520 con 0x7fd48c1073f0 2024-06-10T01:22:57.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.329+0000 7fd49413a700 1 -- 172.21.15.7:0/901859330 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd48c102aa0 con 0x7fd48c1073f0 2024-06-10T01:22:57.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.330+0000 7fd482ffd700 1 -- 172.21.15.7:0/901859330 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fd484005020 con 0x7fd48c1073f0 2024-06-10T01:22:57.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.331+0000 7fd482ffd700 1 --2- 172.21.15.7:0/901859330 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd4780a2e30 0x7fd4780a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:22:57.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.331+0000 7fd482ffd700 1 -- 172.21.15.7:0/901859330 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fd4840cedc0 con 0x7fd48c1073f0 2024-06-10T01:22:57.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.331+0000 7fd491ed6700 1 --2- 172.21.15.7:0/901859330 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd4780a2e30 0x7fd4780a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:22:57.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.332+0000 7fd491ed6700 1 --2- 172.21.15.7:0/901859330 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd4780a2e30 0x7fd4780a52f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd488009bd0 tx=0x7fd488009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:22:57.857 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:22:57.857 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:22:43.821503+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25498":{"gid":25498,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/4278336693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4278336693},{"type":"v1","addr":"172.21.15.175:6835","nonce":4278336693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24434,24436,24451]},"id":1}]} 2024-06-10T01:22:57.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.336+0000 7fd482ffd700 1 -- 172.21.15.7:0/901859330 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd484014030 con 0x7fd48c1073f0 2024-06-10T01:22:57.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.606+0000 7fd49413a700 1 -- 172.21.15.7:0/901859330 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd48c1b4750 con 0x7fd48c1073f0 2024-06-10T01:22:57.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.607+0000 7fd482ffd700 1 -- 172.21.15.7:0/901859330 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11858 (secure 0 0 0) 0x7fd48409b940 con 0x7fd48c1073f0 2024-06-10T01:22:57.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.609+0000 7fd49413a700 1 -- 172.21.15.7:0/901859330 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd4780a2e30 msgr2=0x7fd4780a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:57.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.609+0000 7fd49413a700 1 --2- 172.21.15.7:0/901859330 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd4780a2e30 0x7fd4780a52f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd488009bd0 tx=0x7fd488009040 comp rx=0 tx=0).stop 2024-06-10T01:22:57.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.609+0000 7fd49413a700 1 -- 172.21.15.7:0/901859330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd48c1073f0 msgr2=0x7fd48c1a7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:22:57.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.609+0000 7fd49413a700 1 --2- 172.21.15.7:0/901859330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd48c1073f0 0x7fd48c1a7840 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd48400c7a0 tx=0x7fd48400cab0 comp rx=0 tx=0).stop 2024-06-10T01:22:57.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.609+0000 7fd49413a700 1 -- 172.21.15.7:0/901859330 shutdown_connections 2024-06-10T01:22:57.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.609+0000 7fd49413a700 1 --2- 172.21.15.7:0/901859330 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd4780a2e30 0x7fd4780a52f0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:57.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.609+0000 7fd49413a700 1 --2- 172.21.15.7:0/901859330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd48c1073f0 0x7fd48c1a7840 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:57.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.609+0000 7fd49413a700 1 --2- 172.21.15.7:0/901859330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd48c101380 0x7fd48c1ad6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.609+0000 7fd49413a700 1 --2- 172.21.15.7:0/901859330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd48c101d40 0x7fd48c1adc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:22:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.609+0000 7fd49413a700 1 -- 172.21.15.7:0/901859330 >> 172.21.15.7:0/901859330 conn(0x7fd48c078300 msgr2=0x7fd48c0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:22:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.609+0000 7fd49413a700 1 -- 172.21.15.7:0/901859330 shutdown_connections 2024-06-10T01:22:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:22:57.609+0000 7fd49413a700 1 -- 172.21.15.7:0/901859330 wait complete. 2024-06-10T01:22:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 48 2024-06-10T01:22:58.175 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:57 smithi007 ceph-mon[32398]: daemon mds.f restarted 2024-06-10T01:22:58.175 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:57 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/901859330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:22:58.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:57 smithi078 ceph-mon[36683]: daemon mds.f restarted 2024-06-10T01:22:58.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:57 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/901859330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:22:58.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:57 smithi175 ceph-mon[36636]: daemon mds.f restarted 2024-06-10T01:22:58.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:57 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/901859330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:22:58.470 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:22:58.529 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 29.43 MB/sec execute 95 sec latency 11127.648 ms 2024-06-10T01:22:58.801 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:22:59.247 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:58 smithi007 ceph-mon[32398]: pgmap v941: 129 pgs: 129 active+clean; 590 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.8 MiB/s wr, 420 op/s 2024-06-10T01:22:59.247 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:58 smithi007 ceph-mon[32398]: osdmap e101: 12 total, 12 up, 12 in 2024-06-10T01:22:59.247 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:58 smithi007 ceph-mon[32398]: mds.? [v2:172.21.15.175:6834/1972426673,v1:172.21.15.175:6835/1972426673] up:boot 2024-06-10T01:22:59.247 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:58 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 2 up:standby-replay 7 up:standby 2024-06-10T01:22:59.247 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:58 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T01:22:59.247 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:22:58 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:22:59.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:58 smithi078 ceph-mon[36683]: pgmap v941: 129 pgs: 129 active+clean; 590 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.8 MiB/s wr, 420 op/s 2024-06-10T01:22:59.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:58 smithi078 ceph-mon[36683]: osdmap e101: 12 total, 12 up, 12 in 2024-06-10T01:22:59.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:58 smithi078 ceph-mon[36683]: mds.? [v2:172.21.15.175:6834/1972426673,v1:172.21.15.175:6835/1972426673] up:boot 2024-06-10T01:22:59.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:58 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 2 up:standby-replay 7 up:standby 2024-06-10T01:22:59.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:58 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T01:22:59.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:22:58 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:22:59.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:58 smithi175 ceph-mon[36636]: pgmap v941: 129 pgs: 129 active+clean; 590 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.8 MiB/s wr, 420 op/s 2024-06-10T01:22:59.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:58 smithi175 ceph-mon[36636]: osdmap e101: 12 total, 12 up, 12 in 2024-06-10T01:22:59.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:58 smithi175 ceph-mon[36636]: mds.? [v2:172.21.15.175:6834/1972426673,v1:172.21.15.175:6835/1972426673] up:boot 2024-06-10T01:22:59.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:58 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 2 up:standby-replay 7 up:standby 2024-06-10T01:22:59.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:58 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T01:22:59.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:22:58 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:22:59.529 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 29.13 MB/sec execute 96 sec latency 12127.710 ms 2024-06-10T01:23:00.530 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 28.83 MB/sec execute 97 sec latency 13127.757 ms 2024-06-10T01:23:00.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.555+0000 7fdeafa06700 1 -- 172.21.15.7:0/2231580037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdea8100b30 msgr2=0x7fdea8100fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:00.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.555+0000 7fdeafa06700 1 --2- 172.21.15.7:0/2231580037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdea8100b30 0x7fdea8100fb0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fde980099c0 tx=0x7fde98009cd0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.555+0000 7fdeafa06700 1 -- 172.21.15.7:0/2231580037 shutdown_connections 2024-06-10T01:23:00.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.555+0000 7fdeafa06700 1 --2- 172.21.15.7:0/2231580037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdea81014f0 0x7fdea810d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.555+0000 7fdeafa06700 1 --2- 172.21.15.7:0/2231580037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdea8106be0 0x7fdea8106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.555+0000 7fdeafa06700 1 --2- 172.21.15.7:0/2231580037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdea8100b30 0x7fdea8100fb0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.555+0000 7fdeafa06700 1 -- 172.21.15.7:0/2231580037 >> 172.21.15.7:0/2231580037 conn(0x7fdea8077990 msgr2=0x7fdea80fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:00.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.555+0000 7fdeafa06700 1 -- 172.21.15.7:0/2231580037 shutdown_connections 2024-06-10T01:23:00.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.555+0000 7fdeafa06700 1 -- 172.21.15.7:0/2231580037 wait complete. 2024-06-10T01:23:00.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.555+0000 7fdeafa06700 1 Processor -- start 2024-06-10T01:23:00.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.555+0000 7fdeafa06700 1 -- start start 2024-06-10T01:23:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.556+0000 7fdeafa06700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdea8100b30 0x7fdea819d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.556+0000 7fdeafa06700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdea81014f0 0x7fdea819dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.556+0000 7fdeafa06700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdea8106be0 0x7fdea81978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.556+0000 7fdeafa06700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdea8111710 con 0x7fdea8106be0 2024-06-10T01:23:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.556+0000 7fdeafa06700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdea8111590 con 0x7fdea8100b30 2024-06-10T01:23:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.556+0000 7fdeafa06700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdea8111890 con 0x7fdea81014f0 2024-06-10T01:23:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.556+0000 7fdeadfa3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdea8106be0 0x7fdea81978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.556+0000 7fdeadfa3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdea8106be0 0x7fdea81978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35662/0 (socket says 172.21.15.7:35662) 2024-06-10T01:23:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.556+0000 7fdeadfa3700 1 -- 172.21.15.7:0/2899486319 learned_addr learned my addr 172.21.15.7:0/2899486319 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.556+0000 7fdeacfa1700 1 --2- 172.21.15.7:0/2899486319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdea81014f0 0x7fdea819dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.556+0000 7fdead7a2700 1 --2- 172.21.15.7:0/2899486319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdea8100b30 0x7fdea819d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.556+0000 7fdeadfa3700 1 -- 172.21.15.7:0/2899486319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdea81014f0 msgr2=0x7fdea819dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.556+0000 7fdeadfa3700 1 --2- 172.21.15.7:0/2899486319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdea81014f0 0x7fdea819dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.556+0000 7fdeadfa3700 1 -- 172.21.15.7:0/2899486319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdea8100b30 msgr2=0x7fdea819d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.556+0000 7fdeadfa3700 1 --2- 172.21.15.7:0/2899486319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdea8100b30 0x7fdea819d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.556+0000 7fdeadfa3700 1 -- 172.21.15.7:0/2899486319 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea8198050 con 0x7fdea8106be0 2024-06-10T01:23:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.556+0000 7fdeadfa3700 1 --2- 172.21.15.7:0/2899486319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdea8106be0 0x7fdea81978f0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fdea000b970 tx=0x7fdea000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.557+0000 7fde9e7fc700 1 -- 172.21.15.7:0/2899486319 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0019070 con 0x7fdea8106be0 2024-06-10T01:23:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.557+0000 7fdeafa06700 1 -- 172.21.15.7:0/2899486319 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdea8198340 con 0x7fdea8106be0 2024-06-10T01:23:00.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.557+0000 7fde9e7fc700 1 -- 172.21.15.7:0/2899486319 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdea000ce10 con 0x7fdea8106be0 2024-06-10T01:23:00.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.557+0000 7fde9e7fc700 1 -- 172.21.15.7:0/2899486319 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0007370 con 0x7fdea8106be0 2024-06-10T01:23:00.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.557+0000 7fdeafa06700 1 -- 172.21.15.7:0/2899486319 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdea810e370 con 0x7fdea8106be0 2024-06-10T01:23:00.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.559+0000 7fde9e7fc700 1 -- 172.21.15.7:0/2899486319 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fdea001c070 con 0x7fdea8106be0 2024-06-10T01:23:00.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.560+0000 7fde9e7fc700 1 --2- 172.21.15.7:0/2899486319 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fde940a2c20 0x7fde940a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:00.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.560+0000 7fdead7a2700 1 --2- 172.21.15.7:0/2899486319 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fde940a2c20 0x7fde940a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:00.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.560+0000 7fde9e7fc700 1 -- 172.21.15.7:0/2899486319 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fdea00ce790 con 0x7fdea8106be0 2024-06-10T01:23:00.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.560+0000 7fdead7a2700 1 --2- 172.21.15.7:0/2899486319 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fde940a2c20 0x7fde940a50e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdea4005ea0 tx=0x7fdea4005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:00.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.561+0000 7fdeba289b80 1 Processor -- start 2024-06-10T01:23:00.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.561+0000 7fdeba289b80 1 -- start start 2024-06-10T01:23:00.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.561+0000 7fdeba289b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1954b5370 0x55d1954b5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:00.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.561+0000 7fdeba289b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1954b5c90 0x55d1954af670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:00.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.561+0000 7fdeba289b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1954afbb0 0x55d1954afff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:00.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.561+0000 7fdeba289b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d19539df60 con 0x55d1954b5370 2024-06-10T01:23:00.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.561+0000 7fdeba289b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d1953fc850 con 0x55d1954afbb0 2024-06-10T01:23:00.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.561+0000 7fdeba289b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d195364460 con 0x55d1954b5c90 2024-06-10T01:23:00.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.561+0000 7fdead7a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1954b5370 0x55d1954b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:00.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.561+0000 7fdead7a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1954b5370 0x55d1954b5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35690/0 (socket says 172.21.15.7:35690) 2024-06-10T01:23:00.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.561+0000 7fdead7a2700 1 -- 172.21.15.7:0/3647954703 learned_addr learned my addr 172.21.15.7:0/3647954703 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:00.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.561+0000 7fdeadfa3700 1 --2- 172.21.15.7:0/3647954703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1954afbb0 0x55d1954afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:00.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.561+0000 7fdeacfa1700 1 --2- 172.21.15.7:0/3647954703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1954b5c90 0x55d1954af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:00.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.561+0000 7fdead7a2700 1 -- 172.21.15.7:0/3647954703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1954b5c90 msgr2=0x55d1954af670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:00.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.561+0000 7fdead7a2700 1 --2- 172.21.15.7:0/3647954703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1954b5c90 0x55d1954af670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.561+0000 7fdead7a2700 1 -- 172.21.15.7:0/3647954703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1954afbb0 msgr2=0x55d1954afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:00.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.561+0000 7fdead7a2700 1 --2- 172.21.15.7:0/3647954703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1954afbb0 0x55d1954afff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.561+0000 7fdead7a2700 1 -- 172.21.15.7:0/3647954703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d19534b8d0 con 0x55d1954b5370 2024-06-10T01:23:00.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.562+0000 7fdead7a2700 1 --2- 172.21.15.7:0/3647954703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1954b5370 0x55d1954b5750 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fdea400a520 tx=0x7fdea4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c81205a311d34f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:00.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.562+0000 7fde937fe700 1 -- 172.21.15.7:0/3647954703 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea4021070 con 0x55d1954b5370 2024-06-10T01:23:00.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.562+0000 7fde937fe700 1 -- 172.21.15.7:0/3647954703 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdea4010cd0 con 0x55d1954b5370 2024-06-10T01:23:00.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.562+0000 7fde937fe700 1 -- 172.21.15.7:0/3647954703 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea400f740 con 0x55d1954b5370 2024-06-10T01:23:00.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.563+0000 7fdeba289b80 1 -- 172.21.15.7:0/3647954703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1954b5370 msgr2=0x55d1954b5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:00.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.563+0000 7fdeba289b80 1 --2- 172.21.15.7:0/3647954703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1954b5370 0x55d1954b5750 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fdea400a520 tx=0x7fdea4011000 comp rx=0 tx=0).stop 2024-06-10T01:23:00.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.563+0000 7fdeba289b80 1 -- 172.21.15.7:0/3647954703 shutdown_connections 2024-06-10T01:23:00.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.563+0000 7fdeba289b80 1 --2- 172.21.15.7:0/3647954703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1954afbb0 0x55d1954afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.563+0000 7fdeba289b80 1 --2- 172.21.15.7:0/3647954703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1954b5370 0x55d1954b5750 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.563+0000 7fdeba289b80 1 --2- 172.21.15.7:0/3647954703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1954b5c90 0x55d1954af670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.563+0000 7fdeba289b80 1 -- 172.21.15.7:0/3647954703 >> 172.21.15.7:0/3647954703 conn(0x55d1953d8080 msgr2=0x55d1954a6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:00.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.563+0000 7fdeba289b80 1 -- 172.21.15.7:0/3647954703 shutdown_connections 2024-06-10T01:23:00.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.563+0000 7fdeba289b80 1 -- 172.21.15.7:0/3647954703 wait complete. 2024-06-10T01:23:00.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.563+0000 7fdeba289b80 1 -- start start 2024-06-10T01:23:00.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.563+0000 7fdeba289b80 1 Processor -- start 2024-06-10T01:23:00.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.563+0000 7fdeba289b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1954afbb0 0x55d1954bb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:00.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.563+0000 7fdeba289b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1954b5370 0x55d1954b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:00.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.564+0000 7fdeba289b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1954b5c90 0x55d1954b9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:00.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.564+0000 7fdeba289b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d19539cfc0 con 0x55d1954afbb0 2024-06-10T01:23:00.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.564+0000 7fdeba289b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d1953f8810 con 0x55d1954b5370 2024-06-10T01:23:00.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.564+0000 7fdeba289b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d1954bc680 con 0x55d1954b5c90 2024-06-10T01:23:00.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.564+0000 7fdeba289b80 10 client.0 _ll_register_callbacks cb 0x55d1953925b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:23:00.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.564+0000 7fdead7a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1954afbb0 0x55d1954bb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:00.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.564+0000 7fdead7a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1954afbb0 0x55d1954bb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35692/0 (socket says 172.21.15.7:35692) 2024-06-10T01:23:00.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.564+0000 7fdead7a2700 1 -- 172.21.15.7:0/862307470 learned_addr learned my addr 172.21.15.7:0/862307470 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:00.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.564+0000 7fdeacfa1700 1 --2- 172.21.15.7:0/862307470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1954b5370 0x55d1954b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:00.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.564+0000 7fdeadfa3700 1 --2- 172.21.15.7:0/862307470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1954b5c90 0x55d1954b9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:00.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.564+0000 7fdead7a2700 1 -- 172.21.15.7:0/862307470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1954b5c90 msgr2=0x55d1954b9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:00.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.564+0000 7fdead7a2700 1 --2- 172.21.15.7:0/862307470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1954b5c90 0x55d1954b9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.564+0000 7fdead7a2700 1 -- 172.21.15.7:0/862307470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1954b5370 msgr2=0x55d1954b8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:00.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.564+0000 7fdead7a2700 1 --2- 172.21.15.7:0/862307470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1954b5370 0x55d1954b8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.564+0000 7fdead7a2700 1 -- 172.21.15.7:0/862307470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1953c2750 con 0x55d1954afbb0 2024-06-10T01:23:00.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.564+0000 7fdead7a2700 1 --2- 172.21.15.7:0/862307470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1954afbb0 0x55d1954bb9e0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fdea401f040 tx=0x7fdea400a9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e327b70f7c2b941f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:00.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.564+0000 7fde90ff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:23:00.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.564+0000 7fde90ff9700 1 -- 172.21.15.7:0/862307470 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea4035060 con 0x55d1954afbb0 2024-06-10T01:23:00.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.564+0000 7fde90ff9700 1 -- 172.21.15.7:0/862307470 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fdea4010ad0 con 0x55d1954afbb0 2024-06-10T01:23:00.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.564+0000 7fdeba289b80 1 -- 172.21.15.7:0/862307470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdea4010cd0 con 0x55d1954afbb0 2024-06-10T01:23:00.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.565+0000 7fde90ff9700 1 -- 172.21.15.7:0/862307470 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fdea403c050 con 0x55d1954afbb0 2024-06-10T01:23:00.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.565+0000 7fdeba289b80 10 client.19617 fetch_fsmap learned FSMap version 50 2024-06-10T01:23:00.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.565+0000 7fdeba289b80 1 -- 172.21.15.7:0/862307470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55d1954c6e00 con 0x55d1954afbb0 2024-06-10T01:23:00.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.565+0000 7fde90ff9700 1 -- 172.21.15.7:0/862307470 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdea4010cd0 con 0x55d1954afbb0 2024-06-10T01:23:00.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.565+0000 7fde90ff9700 1 -- 172.21.15.7:0/862307470 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7fdea4026450 con 0x55d1954afbb0 2024-06-10T01:23:00.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.565+0000 7fdeba289b80 10 client.19617 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:23:00.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.565+0000 7fdeba289b80 10 client.19617 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:23:00.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.565+0000 7fdeba289b80 20 client.19617 populate_metadata read hostname 'smithi007' 2024-06-10T01:23:00.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.565+0000 7fdeba289b80 1 --2- 172.21.15.7:0/862307470 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55d1954c9380 0x55d1954cb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:00.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.565+0000 7fdeba289b80 4 client.19617 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:23:00.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.565+0000 7fdeba289b80 1 -- 172.21.15.7:0/862307470 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d195409d50 con 0x55d1954c9380 2024-06-10T01:23:00.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.565+0000 7fdeacfa1700 1 --2- 172.21.15.7:0/862307470 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55d1954c9380 0x55d1954cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:00.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.566+0000 7fdeacfa1700 1 --2- 172.21.15.7:0/862307470 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55d1954c9380 0x55d1954cb770 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7e01f58c6d39a99c server_cookie=ef61e464ca2f7be1 in_seq=0 out_seq=0 2024-06-10T01:23:00.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.566+0000 7fde90ff9700 10 client.19617 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:23:00.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.566+0000 7fde90ff9700 1 -- 172.21.15.7:0/862307470 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d195409d50 con 0x55d1954c9380 2024-06-10T01:23:00.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.566+0000 7fde90ff9700 10 client.19617 handle_command_reply: tid=0 2024-06-10T01:23:00.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.566+0000 7fdeba289b80 1 client.19617 shutdown 2024-06-10T01:23:00.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.567+0000 7fdeba289b80 1 -- 172.21.15.7:0/862307470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1954afbb0 msgr2=0x55d1954bb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:00.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.567+0000 7fdeba289b80 1 --2- 172.21.15.7:0/862307470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1954afbb0 0x55d1954bb9e0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fdea401f040 tx=0x7fdea400a9c0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.567+0000 7fdeba289b80 1 -- 172.21.15.7:0/862307470 shutdown_connections 2024-06-10T01:23:00.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.567+0000 7fdeba289b80 1 --2- 172.21.15.7:0/862307470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1954b5c90 0x55d1954b9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.567+0000 7fdeba289b80 1 --2- 172.21.15.7:0/862307470 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55d1954c9380 0x55d1954cb770 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.567+0000 7fdeba289b80 1 --2- 172.21.15.7:0/862307470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1954afbb0 0x55d1954bb9e0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.567+0000 7fdeba289b80 1 --2- 172.21.15.7:0/862307470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1954b5370 0x55d1954b8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.567+0000 7fdeba289b80 1 -- 172.21.15.7:0/862307470 >> 172.21.15.7:0/862307470 conn(0x55d1953d8080 msgr2=0x55d1954a5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:00.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.567+0000 7fde90ff9700 0 client.19617 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:23:00.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.567+0000 7fdeba289b80 1 -- 172.21.15.7:0/862307470 shutdown_connections 2024-06-10T01:23:00.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.567+0000 7fdeba289b80 1 -- 172.21.15.7:0/862307470 wait complete. 2024-06-10T01:23:00.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.567+0000 7fdeba289b80 20 client.19617 trim_cache size 0 max 16384 2024-06-10T01:23:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.582+0000 7fdeba289b80 1 Processor -- start 2024-06-10T01:23:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.582+0000 7fdeba289b80 1 -- start start 2024-06-10T01:23:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.582+0000 7fdeba289b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1954afbb0 0x55d1954b6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.582+0000 7fdeba289b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1954b5370 0x55d1954a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.583+0000 7fdeba289b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1954b5c90 0x55d1954a63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.583+0000 7fdeba289b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdea81ae8e0 con 0x55d1954b5370 2024-06-10T01:23:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.583+0000 7fdeba289b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdea81ac170 con 0x55d1954afbb0 2024-06-10T01:23:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.583+0000 7fdeba289b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdea81a9fc0 con 0x55d1954b5c90 2024-06-10T01:23:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.583+0000 7fdeacfa1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1954b5370 0x55d1954a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.583+0000 7fdeacfa1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1954b5370 0x55d1954a5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35694/0 (socket says 172.21.15.7:35694) 2024-06-10T01:23:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.583+0000 7fdeacfa1700 1 -- 172.21.15.7:0/2409604304 learned_addr learned my addr 172.21.15.7:0/2409604304 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.583+0000 7fdead7a2700 1 --2- 172.21.15.7:0/2409604304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1954afbb0 0x55d1954b6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.583+0000 7fdeadfa3700 1 --2- 172.21.15.7:0/2409604304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1954b5c90 0x55d1954a63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.583+0000 7fdeacfa1700 1 -- 172.21.15.7:0/2409604304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1954b5c90 msgr2=0x55d1954a63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.583+0000 7fdeacfa1700 1 --2- 172.21.15.7:0/2409604304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1954b5c90 0x55d1954a63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.583+0000 7fdeacfa1700 1 -- 172.21.15.7:0/2409604304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1954afbb0 msgr2=0x55d1954b6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.583+0000 7fdeacfa1700 1 --2- 172.21.15.7:0/2409604304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1954afbb0 0x55d1954b6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.583+0000 7fdeacfa1700 1 -- 172.21.15.7:0/2409604304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1954b0520 con 0x55d1954b5370 2024-06-10T01:23:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.583+0000 7fdead7a2700 1 --2- 172.21.15.7:0/2409604304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1954afbb0 0x55d1954b6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:23:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.583+0000 7fdeacfa1700 1 --2- 172.21.15.7:0/2409604304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1954b5370 0x55d1954a5e80 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fde980098e0 tx=0x7fde98005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d94cae230d6fa28 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.583+0000 7fde92ffd700 1 -- 172.21.15.7:0/2409604304 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde980050b0 con 0x55d1954b5370 2024-06-10T01:23:00.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.584+0000 7fde92ffd700 1 -- 172.21.15.7:0/2409604304 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde9800b730 con 0x55d1954b5370 2024-06-10T01:23:00.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.584+0000 7fde92ffd700 1 -- 172.21.15.7:0/2409604304 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9800f6c0 con 0x55d1954b5370 2024-06-10T01:23:00.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.584+0000 7fdeba289b80 1 -- 172.21.15.7:0/2409604304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1954b5370 msgr2=0x55d1954a5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:00.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.584+0000 7fdeba289b80 1 --2- 172.21.15.7:0/2409604304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1954b5370 0x55d1954a5e80 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fde980098e0 tx=0x7fde98005040 comp rx=0 tx=0).stop 2024-06-10T01:23:00.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.585+0000 7fdeba289b80 1 -- 172.21.15.7:0/2409604304 shutdown_connections 2024-06-10T01:23:00.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.585+0000 7fdeba289b80 1 --2- 172.21.15.7:0/2409604304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1954b5c90 0x55d1954a63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.585+0000 7fdeba289b80 1 --2- 172.21.15.7:0/2409604304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1954afbb0 0x55d1954b6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.585+0000 7fdeba289b80 1 --2- 172.21.15.7:0/2409604304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1954b5370 0x55d1954a5e80 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.585+0000 7fdeba289b80 1 -- 172.21.15.7:0/2409604304 >> 172.21.15.7:0/2409604304 conn(0x55d1953d8080 msgr2=0x55d1954b6070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:00.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.585+0000 7fdeba289b80 1 -- 172.21.15.7:0/2409604304 shutdown_connections 2024-06-10T01:23:00.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.585+0000 7fdeba289b80 1 -- 172.21.15.7:0/2409604304 wait complete. 2024-06-10T01:23:00.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.585+0000 7fdeba289b80 1 -- start start 2024-06-10T01:23:00.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.585+0000 7fdeba289b80 1 Processor -- start 2024-06-10T01:23:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.586+0000 7fdeba289b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1954afbb0 0x55d1954ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.586+0000 7fdeba289b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1954b5370 0x55d1954cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.586+0000 7fdeba289b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1954b5c90 0x55d1954cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.586+0000 7fdeba289b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdea81aa630 con 0x55d1954b5370 2024-06-10T01:23:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.586+0000 7fdeba289b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdea81abaa0 con 0x55d1954b5c90 2024-06-10T01:23:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.586+0000 7fdeba289b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdea81aad00 con 0x55d1954afbb0 2024-06-10T01:23:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.586+0000 7fdeba289b80 10 client.0 _ll_register_callbacks cb 0x55d1953925b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:23:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.586+0000 7fdeacfa1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1954b5370 0x55d1954cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.586+0000 7fdeacfa1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1954b5370 0x55d1954cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35706/0 (socket says 172.21.15.7:35706) 2024-06-10T01:23:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.586+0000 7fdeacfa1700 1 -- 172.21.15.7:0/515412823 learned_addr learned my addr 172.21.15.7:0/515412823 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.586+0000 7fdead7a2700 1 --2- 172.21.15.7:0/515412823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1954afbb0 0x55d1954ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.586+0000 7fdeadfa3700 1 --2- 172.21.15.7:0/515412823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1954b5c90 0x55d1954cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.586+0000 7fdeacfa1700 1 -- 172.21.15.7:0/515412823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1954afbb0 msgr2=0x55d1954ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.586+0000 7fdeacfa1700 1 --2- 172.21.15.7:0/515412823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1954afbb0 0x55d1954ce720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.586+0000 7fdeacfa1700 1 -- 172.21.15.7:0/515412823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1954b5c90 msgr2=0x55d1954cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.586+0000 7fdeacfa1700 1 --2- 172.21.15.7:0/515412823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1954b5c90 0x55d1954cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.586+0000 7fdeacfa1700 1 -- 172.21.15.7:0/515412823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1954b4e50 con 0x55d1954b5370 2024-06-10T01:23:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.586+0000 7fdeadfa3700 1 --2- 172.21.15.7:0/515412823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1954b5c90 0x55d1954cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:23:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.586+0000 7fdead7a2700 1 --2- 172.21.15.7:0/515412823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1954afbb0 0x55d1954ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:23:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.586+0000 7fdeacfa1700 1 --2- 172.21.15.7:0/515412823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1954b5370 0x55d1954cec60 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fde98009d10 tx=0x7fde98005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ceb5c5ecc4fbcf7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.586+0000 7fde90ff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:23:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.587+0000 7fde90ff9700 1 -- 172.21.15.7:0/515412823 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9803a060 con 0x55d1954b5370 2024-06-10T01:23:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.587+0000 7fdeba289b80 1 -- 172.21.15.7:0/515412823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde9800b730 con 0x55d1954b5370 2024-06-10T01:23:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.587+0000 7fde90ff9700 1 -- 172.21.15.7:0/515412823 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fde9800bd70 con 0x55d1954b5370 2024-06-10T01:23:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.587+0000 7fde90ff9700 1 -- 172.21.15.7:0/515412823 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fde98041050 con 0x55d1954b5370 2024-06-10T01:23:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.587+0000 7fdeba289b80 10 client.19623 fetch_fsmap learned FSMap version 50 2024-06-10T01:23:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.587+0000 7fdeba289b80 1 -- 172.21.15.7:0/515412823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55d195533720 con 0x55d1954b5370 2024-06-10T01:23:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.587+0000 7fde90ff9700 1 -- 172.21.15.7:0/515412823 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde9800b730 con 0x55d1954b5370 2024-06-10T01:23:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.587+0000 7fde90ff9700 1 -- 172.21.15.7:0/515412823 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7fde9800fa60 con 0x55d1954b5370 2024-06-10T01:23:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.587+0000 7fdeba289b80 10 client.19623 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:23:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.587+0000 7fdeba289b80 10 client.19623 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:23:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.587+0000 7fdeba289b80 20 client.19623 populate_metadata read hostname 'smithi007' 2024-06-10T01:23:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.587+0000 7fdeba289b80 1 --2- 172.21.15.7:0/515412823 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55d195533de0 0x55d195536200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.587+0000 7fdeba289b80 4 client.19623 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:23:00.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.587+0000 7fdeba289b80 1 -- 172.21.15.7:0/515412823 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d1954042d0 con 0x55d195533de0 2024-06-10T01:23:00.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.588+0000 7fdead7a2700 1 --2- 172.21.15.7:0/515412823 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55d195533de0 0x55d195536200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:00.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.588+0000 7fdead7a2700 1 --2- 172.21.15.7:0/515412823 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55d195533de0 0x55d195536200 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=67530ba15cf671c4 server_cookie=1c7498cce3e57119 in_seq=0 out_seq=0 2024-06-10T01:23:00.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.588+0000 7fde90ff9700 10 client.19623 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:23:00.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.588+0000 7fde90ff9700 1 -- 172.21.15.7:0/515412823 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d1954042d0 con 0x55d195533de0 2024-06-10T01:23:00.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.588+0000 7fde90ff9700 10 client.19623 handle_command_reply: tid=0 2024-06-10T01:23:00.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.588+0000 7fdeba289b80 1 client.19623 shutdown 2024-06-10T01:23:00.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.588+0000 7fdeba289b80 1 -- 172.21.15.7:0/515412823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1954b5370 msgr2=0x55d1954cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:00.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.588+0000 7fdeba289b80 1 --2- 172.21.15.7:0/515412823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1954b5370 0x55d1954cec60 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fde98009d10 tx=0x7fde98005cd0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.588+0000 7fdeba289b80 1 -- 172.21.15.7:0/515412823 shutdown_connections 2024-06-10T01:23:00.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.589+0000 7fdeba289b80 1 --2- 172.21.15.7:0/515412823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1954b5c90 0x55d1954cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.589+0000 7fdeba289b80 1 --2- 172.21.15.7:0/515412823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1954afbb0 0x55d1954ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.589+0000 7fdeba289b80 1 --2- 172.21.15.7:0/515412823 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55d195533de0 0x55d195536200 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.589+0000 7fdeba289b80 1 --2- 172.21.15.7:0/515412823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1954b5370 0x55d1954cec60 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.589+0000 7fdeba289b80 1 -- 172.21.15.7:0/515412823 >> 172.21.15.7:0/515412823 conn(0x55d1953d8080 msgr2=0x55d1954aa250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:00.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.589+0000 7fde90ff9700 0 client.19623 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:23:00.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.589+0000 7fdeba289b80 1 -- 172.21.15.7:0/515412823 shutdown_connections 2024-06-10T01:23:00.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.589+0000 7fdeba289b80 1 -- 172.21.15.7:0/515412823 wait complete. 2024-06-10T01:23:00.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.589+0000 7fdeba289b80 20 client.19623 trim_cache size 0 max 16384 2024-06-10T01:23:00.588 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:23:00.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.589+0000 7fdeafa06700 1 -- 172.21.15.7:0/2899486319 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fde940a2c20 msgr2=0x7fde940a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:00.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.589+0000 7fdeafa06700 1 --2- 172.21.15.7:0/2899486319 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fde940a2c20 0x7fde940a50e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdea4005ea0 tx=0x7fdea4005e30 comp rx=0 tx=0).stop 2024-06-10T01:23:00.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.589+0000 7fdeafa06700 1 -- 172.21.15.7:0/2899486319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdea8106be0 msgr2=0x7fdea81978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:00.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.589+0000 7fdeafa06700 1 --2- 172.21.15.7:0/2899486319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdea8106be0 0x7fdea81978f0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fdea000b970 tx=0x7fdea000bc80 comp rx=0 tx=0).stop 2024-06-10T01:23:00.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.590+0000 7fdeafa06700 1 -- 172.21.15.7:0/2899486319 shutdown_connections 2024-06-10T01:23:00.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.590+0000 7fdeafa06700 1 --2- 172.21.15.7:0/2899486319 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fde940a2c20 0x7fde940a50e0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.590+0000 7fdeafa06700 1 --2- 172.21.15.7:0/2899486319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdea8106be0 0x7fdea81978f0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.590+0000 7fdeafa06700 1 --2- 172.21.15.7:0/2899486319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdea8100b30 0x7fdea819d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.590+0000 7fdeafa06700 1 --2- 172.21.15.7:0/2899486319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdea81014f0 0x7fdea819dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:00.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.590+0000 7fdeafa06700 1 -- 172.21.15.7:0/2899486319 >> 172.21.15.7:0/2899486319 conn(0x7fdea8077990 msgr2=0x7fdea80feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:00.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.590+0000 7fdeafa06700 1 -- 172.21.15.7:0/2899486319 shutdown_connections 2024-06-10T01:23:00.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:00.590+0000 7fdeafa06700 1 -- 172.21.15.7:0/2899486319 wait complete. 2024-06-10T01:23:01.252 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:00 smithi007 ceph-mon[32398]: pgmap v943: 129 pgs: 129 active+clean; 593 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 5.4 MiB/s wr, 274 op/s 2024-06-10T01:23:01.253 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:23:01.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:00 smithi078 ceph-mon[36683]: pgmap v943: 129 pgs: 129 active+clean; 593 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 5.4 MiB/s wr, 274 op/s 2024-06-10T01:23:01.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:00 smithi175 ceph-mon[36636]: pgmap v943: 129 pgs: 129 active+clean; 593 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 5.4 MiB/s wr, 274 op/s 2024-06-10T01:23:01.530 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 28.53 MB/sec execute 98 sec latency 14128.034 ms 2024-06-10T01:23:01.552 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:23:02.530 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 28.25 MB/sec execute 99 sec latency 15128.075 ms 2024-06-10T01:23:03.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:03 smithi007 ceph-mon[32398]: pgmap v944: 129 pgs: 129 active+clean; 587 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 5.4 MiB/s wr, 292 op/s 2024-06-10T01:23:03.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.274+0000 7f9afb8b1700 1 -- 172.21.15.7:0/2883238720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9af41024e0 msgr2=0x7f9af4102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:03.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.274+0000 7f9afb8b1700 1 --2- 172.21.15.7:0/2883238720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9af41024e0 0x7f9af4102940 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4009990 tx=0x7f9ae4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.274+0000 7f9afb8b1700 1 -- 172.21.15.7:0/2883238720 shutdown_connections 2024-06-10T01:23:03.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.274+0000 7f9afb8b1700 1 --2- 172.21.15.7:0/2883238720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9af4102e80 0x7f9af410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.274+0000 7f9afb8b1700 1 --2- 172.21.15.7:0/2883238720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9af41084f0 0x7f9af41088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.274+0000 7f9afb8b1700 1 --2- 172.21.15.7:0/2883238720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9af41024e0 0x7f9af4102940 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.274+0000 7f9afb8b1700 1 -- 172.21.15.7:0/2883238720 >> 172.21.15.7:0/2883238720 conn(0x7f9af4079530 msgr2=0x7f9af4079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:03.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.274+0000 7f9afb8b1700 1 -- 172.21.15.7:0/2883238720 shutdown_connections 2024-06-10T01:23:03.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.274+0000 7f9afb8b1700 1 -- 172.21.15.7:0/2883238720 wait complete. 2024-06-10T01:23:03.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.275+0000 7f9afb8b1700 1 Processor -- start 2024-06-10T01:23:03.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.275+0000 7f9afb8b1700 1 -- start start 2024-06-10T01:23:03.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.275+0000 7f9afb8b1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9af41024e0 0x7f9af419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:03.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.275+0000 7f9afb8b1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9af4102e80 0x7f9af419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:03.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.275+0000 7f9afb8b1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9af41084f0 0x7f9af4198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:03.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.275+0000 7f9afb8b1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9af4112750 con 0x7f9af41024e0 2024-06-10T01:23:03.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.275+0000 7f9afb8b1700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41125d0 con 0x7f9af41084f0 2024-06-10T01:23:03.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.275+0000 7f9afb8b1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41128d0 con 0x7f9af4102e80 2024-06-10T01:23:03.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.275+0000 7f9af964d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9af41024e0 0x7f9af419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:03.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.275+0000 7f9af964d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9af41024e0 0x7f9af419e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38832/0 (socket says 172.21.15.7:38832) 2024-06-10T01:23:03.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.275+0000 7f9af964d700 1 -- 172.21.15.7:0/885451923 learned_addr learned my addr 172.21.15.7:0/885451923 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:03.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.275+0000 7f9af8e4c700 1 --2- 172.21.15.7:0/885451923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9af4102e80 0x7f9af419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:03.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.275+0000 7f9af964d700 1 -- 172.21.15.7:0/885451923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9af4102e80 msgr2=0x7f9af419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:03.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.275+0000 7f9af964d700 1 --2- 172.21.15.7:0/885451923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9af4102e80 0x7f9af419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.275+0000 7f9af964d700 1 -- 172.21.15.7:0/885451923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9af41084f0 msgr2=0x7f9af4198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:03.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.275+0000 7f9af964d700 1 --2- 172.21.15.7:0/885451923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9af41084f0 0x7f9af4198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.275+0000 7f9af964d700 1 -- 172.21.15.7:0/885451923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af41991b0 con 0x7f9af41024e0 2024-06-10T01:23:03.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.276+0000 7f9af964d700 1 --2- 172.21.15.7:0/885451923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9af41024e0 0x7f9af419e840 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f9af000d950 tx=0x7f9af000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:03.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.276+0000 7f9aea7fc700 1 -- 172.21.15.7:0/885451923 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af0016070 con 0x7f9af41024e0 2024-06-10T01:23:03.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.276+0000 7f9aea7fc700 1 -- 172.21.15.7:0/885451923 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9af0009da0 con 0x7f9af41024e0 2024-06-10T01:23:03.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.276+0000 7f9afb8b1700 1 -- 172.21.15.7:0/885451923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9af41994a0 con 0x7f9af41024e0 2024-06-10T01:23:03.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.276+0000 7f9aea7fc700 1 -- 172.21.15.7:0/885451923 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af000f620 con 0x7f9af41024e0 2024-06-10T01:23:03.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.276+0000 7f9afb8b1700 1 -- 172.21.15.7:0/885451923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9af41a5710 con 0x7f9af41024e0 2024-06-10T01:23:03.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.278+0000 7f9aea7fc700 1 -- 172.21.15.7:0/885451923 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9af0019070 con 0x7f9af41024e0 2024-06-10T01:23:03.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.279+0000 7f9aea7fc700 1 --2- 172.21.15.7:0/885451923 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9ae00a2ca0 0x7f9ae00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:03.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.279+0000 7f9af8e4c700 1 --2- 172.21.15.7:0/885451923 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9ae00a2ca0 0x7f9ae00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:03.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.279+0000 7f9aea7fc700 1 -- 172.21.15.7:0/885451923 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f9af00cee80 con 0x7f9af41024e0 2024-06-10T01:23:03.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.279+0000 7f9af8e4c700 1 --2- 172.21.15.7:0/885451923 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9ae00a2ca0 0x7f9ae00a5160 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9ae400b2e0 tx=0x7f9ae4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:03.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.280+0000 7f9b06134b80 1 Processor -- start 2024-06-10T01:23:03.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.280+0000 7f9b06134b80 1 -- start start 2024-06-10T01:23:03.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.280+0000 7f9b06134b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56031aa13f60 0x56031aa14340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:03.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.280+0000 7f9b06134b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56031aa14880 0x56031aa0e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:03.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.280+0000 7f9b06134b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56031aa0e7a0 0x56031aa0ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:03.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.280+0000 7f9b06134b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56031a900f60 con 0x56031aa0e7a0 2024-06-10T01:23:03.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.280+0000 7f9b06134b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56031a95f850 con 0x56031aa14880 2024-06-10T01:23:03.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.280+0000 7f9b06134b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56031a8c7460 con 0x56031aa13f60 2024-06-10T01:23:03.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.280+0000 7f9af9e4e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56031aa0e7a0 0x56031aa0ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:03.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.280+0000 7f9af9e4e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56031aa0e7a0 0x56031aa0ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38842/0 (socket says 172.21.15.7:38842) 2024-06-10T01:23:03.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.280+0000 7f9af9e4e700 1 -- 172.21.15.7:0/3319144361 learned_addr learned my addr 172.21.15.7:0/3319144361 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:03.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.280+0000 7f9af964d700 1 --2- 172.21.15.7:0/3319144361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56031aa13f60 0x56031aa14340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:03.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.280+0000 7f9af8e4c700 1 --2- 172.21.15.7:0/3319144361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56031aa14880 0x56031aa0e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:03.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.280+0000 7f9af9e4e700 1 -- 172.21.15.7:0/3319144361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56031aa13f60 msgr2=0x56031aa14340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:03.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.280+0000 7f9af9e4e700 1 --2- 172.21.15.7:0/3319144361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56031aa13f60 0x56031aa14340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.280+0000 7f9af9e4e700 1 -- 172.21.15.7:0/3319144361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56031aa14880 msgr2=0x56031aa0e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:03.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.280+0000 7f9af9e4e700 1 --2- 172.21.15.7:0/3319144361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56031aa14880 0x56031aa0e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.281+0000 7f9af9e4e700 1 -- 172.21.15.7:0/3319144361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56031a8ae8d0 con 0x56031aa0e7a0 2024-06-10T01:23:03.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.281+0000 7f9af9e4e700 1 --2- 172.21.15.7:0/3319144361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56031aa0e7a0 0x56031aa0ebc0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00e400 tx=0x7f9aec00e710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ed76d33a75ad81 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:03.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.281+0000 7f9adf7fe700 1 -- 172.21.15.7:0/3319144361 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec004bb0 con 0x56031aa0e7a0 2024-06-10T01:23:03.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.281+0000 7f9adf7fe700 1 -- 172.21.15.7:0/3319144361 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9aec004d50 con 0x56031aa0e7a0 2024-06-10T01:23:03.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.281+0000 7f9adf7fe700 1 -- 172.21.15.7:0/3319144361 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec0044f0 con 0x56031aa0e7a0 2024-06-10T01:23:03.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.282+0000 7f9b06134b80 1 -- 172.21.15.7:0/3319144361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56031aa0e7a0 msgr2=0x56031aa0ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:03.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.282+0000 7f9b06134b80 1 --2- 172.21.15.7:0/3319144361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56031aa0e7a0 0x56031aa0ebc0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00e400 tx=0x7f9aec00e710 comp rx=0 tx=0).stop 2024-06-10T01:23:03.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.282+0000 7f9b06134b80 1 -- 172.21.15.7:0/3319144361 shutdown_connections 2024-06-10T01:23:03.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.282+0000 7f9b06134b80 1 --2- 172.21.15.7:0/3319144361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56031aa0e7a0 0x56031aa0ebc0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.282+0000 7f9b06134b80 1 --2- 172.21.15.7:0/3319144361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56031aa13f60 0x56031aa14340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.282+0000 7f9b06134b80 1 --2- 172.21.15.7:0/3319144361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56031aa14880 0x56031aa0e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.282+0000 7f9b06134b80 1 -- 172.21.15.7:0/3319144361 >> 172.21.15.7:0/3319144361 conn(0x56031a93b080 msgr2=0x56031aa09220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:03.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.282+0000 7f9b06134b80 1 -- 172.21.15.7:0/3319144361 shutdown_connections 2024-06-10T01:23:03.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.282+0000 7f9b06134b80 1 -- 172.21.15.7:0/3319144361 wait complete. 2024-06-10T01:23:03.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.282+0000 7f9b06134b80 1 -- start start 2024-06-10T01:23:03.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.282+0000 7f9b06134b80 1 Processor -- start 2024-06-10T01:23:03.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.283+0000 7f9b06134b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56031aa0e7a0 0x56031aa1a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:03.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.283+0000 7f9b06134b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56031aa13f60 0x56031aa178c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:03.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.283+0000 7f9b06134b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56031aa14880 0x56031aa17e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:03.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.283+0000 7f9b06134b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56031a8fffc0 con 0x56031aa13f60 2024-06-10T01:23:03.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.283+0000 7f9b06134b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56031a95b810 con 0x56031aa0e7a0 2024-06-10T01:23:03.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.283+0000 7f9b06134b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56031aa1f610 con 0x56031aa14880 2024-06-10T01:23:03.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.283+0000 7f9b06134b80 10 client.0 _ll_register_callbacks cb 0x56031a8f55b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:23:03.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.283+0000 7f9af8e4c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56031aa13f60 0x56031aa178c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:03.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.283+0000 7f9af8e4c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56031aa13f60 0x56031aa178c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38850/0 (socket says 172.21.15.7:38850) 2024-06-10T01:23:03.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.283+0000 7f9af8e4c700 1 -- 172.21.15.7:0/559183493 learned_addr learned my addr 172.21.15.7:0/559183493 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:03.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.283+0000 7f9af9e4e700 1 --2- 172.21.15.7:0/559183493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56031aa14880 0x56031aa17e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:03.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.283+0000 7f9af964d700 1 --2- 172.21.15.7:0/559183493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56031aa0e7a0 0x56031aa1a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:03.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.283+0000 7f9af8e4c700 1 -- 172.21.15.7:0/559183493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56031aa14880 msgr2=0x56031aa17e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:03.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.283+0000 7f9af8e4c700 1 --2- 172.21.15.7:0/559183493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56031aa14880 0x56031aa17e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.283+0000 7f9af8e4c700 1 -- 172.21.15.7:0/559183493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56031aa0e7a0 msgr2=0x56031aa1a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:03.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.283+0000 7f9af8e4c700 1 --2- 172.21.15.7:0/559183493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56031aa0e7a0 0x56031aa1a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.283+0000 7f9af8e4c700 1 -- 172.21.15.7:0/559183493 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56031a925750 con 0x56031aa13f60 2024-06-10T01:23:03.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.283+0000 7f9af8e4c700 1 --2- 172.21.15.7:0/559183493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56031aa13f60 0x56031aa178c0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4015000 tx=0x7f9ae4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29afe1a40d4ed377 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:03.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.283+0000 7f9adcff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:23:03.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.284+0000 7f9adcff9700 1 -- 172.21.15.7:0/559183493 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4032060 con 0x56031aa13f60 2024-06-10T01:23:03.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.284+0000 7f9adcff9700 1 -- 172.21.15.7:0/559183493 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f9ae400f460 con 0x56031aa13f60 2024-06-10T01:23:03.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.284+0000 7f9b06134b80 1 -- 172.21.15.7:0/559183493 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9aec004d50 con 0x56031aa13f60 2024-06-10T01:23:03.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.284+0000 7f9adcff9700 1 -- 172.21.15.7:0/559183493 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f9ae4039050 con 0x56031aa13f60 2024-06-10T01:23:03.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.284+0000 7f9b06134b80 10 client.19635 fetch_fsmap learned FSMap version 50 2024-06-10T01:23:03.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.284+0000 7f9b06134b80 1 -- 172.21.15.7:0/559183493 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x56031aa29dc0 con 0x56031aa13f60 2024-06-10T01:23:03.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.284+0000 7f9adcff9700 1 -- 172.21.15.7:0/559183493 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9aec004d50 con 0x56031aa13f60 2024-06-10T01:23:03.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.284+0000 7f9adcff9700 1 -- 172.21.15.7:0/559183493 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7f9ae402d600 con 0x56031aa13f60 2024-06-10T01:23:03.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.284+0000 7f9b06134b80 10 client.19635 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:23:03.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.284+0000 7f9b06134b80 10 client.19635 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:23:03.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.284+0000 7f9b06134b80 20 client.19635 populate_metadata read hostname 'smithi007' 2024-06-10T01:23:03.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.284+0000 7f9b06134b80 1 --2- 172.21.15.7:0/559183493 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x56031aa2c400 0x56031aa2e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:03.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.284+0000 7f9b06134b80 4 client.19635 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:23:03.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.284+0000 7f9b06134b80 1 -- 172.21.15.7:0/559183493 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56031a96cd50 con 0x56031aa2c400 2024-06-10T01:23:03.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.285+0000 7f9af964d700 1 --2- 172.21.15.7:0/559183493 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x56031aa2c400 0x56031aa2e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:03.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.285+0000 7f9af964d700 1 --2- 172.21.15.7:0/559183493 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x56031aa2c400 0x56031aa2e810 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=80e45a0e9c19c372 server_cookie=b587174399f70b67 in_seq=0 out_seq=0 2024-06-10T01:23:03.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.285+0000 7f9adcff9700 10 client.19635 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:23:03.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.286+0000 7f9adcff9700 1 -- 172.21.15.7:0/559183493 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56031a96cd50 con 0x56031aa2c400 2024-06-10T01:23:03.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.286+0000 7f9adcff9700 10 client.19635 handle_command_reply: tid=0 2024-06-10T01:23:03.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.286+0000 7f9b06134b80 1 client.19635 shutdown 2024-06-10T01:23:03.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.286+0000 7f9b06134b80 1 -- 172.21.15.7:0/559183493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56031aa13f60 msgr2=0x56031aa178c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:03.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.286+0000 7f9b06134b80 1 --2- 172.21.15.7:0/559183493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56031aa13f60 0x56031aa178c0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4015000 tx=0x7f9ae4009310 comp rx=0 tx=0).stop 2024-06-10T01:23:03.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.286+0000 7f9b06134b80 1 -- 172.21.15.7:0/559183493 shutdown_connections 2024-06-10T01:23:03.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.286+0000 7f9b06134b80 1 --2- 172.21.15.7:0/559183493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56031aa14880 0x56031aa17e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.286+0000 7f9b06134b80 1 --2- 172.21.15.7:0/559183493 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x56031aa2c400 0x56031aa2e810 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.286+0000 7f9adcff9700 0 client.19635 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:23:03.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.286+0000 7f9b06134b80 1 --2- 172.21.15.7:0/559183493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56031aa0e7a0 0x56031aa1a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.286+0000 7f9b06134b80 1 --2- 172.21.15.7:0/559183493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56031aa13f60 0x56031aa178c0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.286+0000 7f9b06134b80 1 -- 172.21.15.7:0/559183493 >> 172.21.15.7:0/559183493 conn(0x56031a93b080 msgr2=0x56031aa08700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:03.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.286+0000 7f9b06134b80 1 -- 172.21.15.7:0/559183493 shutdown_connections 2024-06-10T01:23:03.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.286+0000 7f9b06134b80 1 -- 172.21.15.7:0/559183493 wait complete. 2024-06-10T01:23:03.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.286+0000 7f9b06134b80 20 client.19635 trim_cache size 0 max 16384 2024-06-10T01:23:03.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.301+0000 7f9b06134b80 1 Processor -- start 2024-06-10T01:23:03.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.301+0000 7f9b06134b80 1 -- start start 2024-06-10T01:23:03.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.301+0000 7f9b06134b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56031aa0e7a0 0x56031aa0e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:03.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.301+0000 7f9b06134b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56031aa13f60 0x56031aa15430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:03.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.301+0000 7f9b06134b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56031aa14880 0x56031aa08e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:03.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.301+0000 7f9b06134b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41ae8d0 con 0x56031aa14880 2024-06-10T01:23:03.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.301+0000 7f9b06134b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41ac160 con 0x56031aa0e7a0 2024-06-10T01:23:03.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.301+0000 7f9b06134b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41a9fb0 con 0x56031aa13f60 2024-06-10T01:23:03.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.302+0000 7f9af9e4e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56031aa14880 0x56031aa08e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:03.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.302+0000 7f9af9e4e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56031aa14880 0x56031aa08e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38864/0 (socket says 172.21.15.7:38864) 2024-06-10T01:23:03.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.302+0000 7f9af9e4e700 1 -- 172.21.15.7:0/3947839217 learned_addr learned my addr 172.21.15.7:0/3947839217 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:03.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.302+0000 7f9af8e4c700 1 --2- 172.21.15.7:0/3947839217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56031aa13f60 0x56031aa15430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:03.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.302+0000 7f9af964d700 1 --2- 172.21.15.7:0/3947839217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56031aa0e7a0 0x56031aa0e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:03.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.302+0000 7f9af9e4e700 1 -- 172.21.15.7:0/3947839217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56031aa13f60 msgr2=0x56031aa15430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:03.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.302+0000 7f9af9e4e700 1 --2- 172.21.15.7:0/3947839217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56031aa13f60 0x56031aa15430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.302+0000 7f9af9e4e700 1 -- 172.21.15.7:0/3947839217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56031aa0e7a0 msgr2=0x56031aa0e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:03.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.302+0000 7f9af9e4e700 1 --2- 172.21.15.7:0/3947839217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56031aa0e7a0 0x56031aa0e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.302+0000 7f9af9e4e700 1 -- 172.21.15.7:0/3947839217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56031aa15e30 con 0x56031aa14880 2024-06-10T01:23:03.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.302+0000 7f9af8e4c700 1 --2- 172.21.15.7:0/3947839217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56031aa13f60 0x56031aa15430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:23:03.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.302+0000 7f9af9e4e700 1 --2- 172.21.15.7:0/3947839217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56031aa14880 0x56031aa08e80 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00e3a0 tx=0x7f9aec009180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b735841d583a3aa server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:03.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.302+0000 7f9adeffd700 1 -- 172.21.15.7:0/3947839217 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec023070 con 0x56031aa14880 2024-06-10T01:23:03.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.302+0000 7f9adeffd700 1 -- 172.21.15.7:0/3947839217 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9aec004bb0 con 0x56031aa14880 2024-06-10T01:23:03.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.302+0000 7f9adeffd700 1 -- 172.21.15.7:0/3947839217 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec0045f0 con 0x56031aa14880 2024-06-10T01:23:03.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.303+0000 7f9b06134b80 1 -- 172.21.15.7:0/3947839217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56031aa14880 msgr2=0x56031aa08e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:03.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.303+0000 7f9b06134b80 1 --2- 172.21.15.7:0/3947839217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56031aa14880 0x56031aa08e80 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00e3a0 tx=0x7f9aec009180 comp rx=0 tx=0).stop 2024-06-10T01:23:03.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.303+0000 7f9b06134b80 1 -- 172.21.15.7:0/3947839217 shutdown_connections 2024-06-10T01:23:03.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.303+0000 7f9b06134b80 1 --2- 172.21.15.7:0/3947839217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56031aa14880 0x56031aa08e80 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.303+0000 7f9b06134b80 1 --2- 172.21.15.7:0/3947839217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56031aa0e7a0 0x56031aa0e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.303+0000 7f9b06134b80 1 --2- 172.21.15.7:0/3947839217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56031aa13f60 0x56031aa15430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.303+0000 7f9b06134b80 1 -- 172.21.15.7:0/3947839217 >> 172.21.15.7:0/3947839217 conn(0x56031a93b080 msgr2=0x56031aa14d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:03.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.303+0000 7f9b06134b80 1 -- 172.21.15.7:0/3947839217 shutdown_connections 2024-06-10T01:23:03.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.303+0000 7f9b06134b80 1 -- 172.21.15.7:0/3947839217 wait complete. 2024-06-10T01:23:03.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.303+0000 7f9b06134b80 1 -- start start 2024-06-10T01:23:03.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.303+0000 7f9b06134b80 1 Processor -- start 2024-06-10T01:23:03.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.304+0000 7f9b06134b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56031aa0e7a0 0x56031aa24ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:03.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.304+0000 7f9b06134b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56031aa13f60 0x56031aa24ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:03.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.304+0000 7f9b06134b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56031aa14880 0x56031aa33510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:03.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.304+0000 7f9b06134b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41aa620 con 0x56031aa13f60 2024-06-10T01:23:03.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.304+0000 7f9b06134b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41aba90 con 0x56031aa14880 2024-06-10T01:23:03.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.304+0000 7f9b06134b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41aacf0 con 0x56031aa0e7a0 2024-06-10T01:23:03.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.304+0000 7f9b06134b80 10 client.0 _ll_register_callbacks cb 0x56031a8f55b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:23:03.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.304+0000 7f9af8e4c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56031aa13f60 0x56031aa24ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:03.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.304+0000 7f9af8e4c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56031aa13f60 0x56031aa24ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38874/0 (socket says 172.21.15.7:38874) 2024-06-10T01:23:03.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.304+0000 7f9af8e4c700 1 -- 172.21.15.7:0/3104805109 learned_addr learned my addr 172.21.15.7:0/3104805109 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:03.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.304+0000 7f9af964d700 1 --2- 172.21.15.7:0/3104805109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56031aa0e7a0 0x56031aa24ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:03.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.304+0000 7f9af9e4e700 1 --2- 172.21.15.7:0/3104805109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56031aa14880 0x56031aa33510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:03.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.304+0000 7f9af8e4c700 1 -- 172.21.15.7:0/3104805109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56031aa0e7a0 msgr2=0x56031aa24ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:03.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.304+0000 7f9af8e4c700 1 --2- 172.21.15.7:0/3104805109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56031aa0e7a0 0x56031aa24ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.304+0000 7f9af8e4c700 1 -- 172.21.15.7:0/3104805109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56031aa14880 msgr2=0x56031aa33510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:03.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.304+0000 7f9af8e4c700 1 --2- 172.21.15.7:0/3104805109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56031aa14880 0x56031aa33510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.304+0000 7f9af8e4c700 1 -- 172.21.15.7:0/3104805109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56031aa33d50 con 0x56031aa13f60 2024-06-10T01:23:03.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.304+0000 7f9af9e4e700 1 --2- 172.21.15.7:0/3104805109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56031aa14880 0x56031aa33510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:23:03.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.304+0000 7f9af8e4c700 1 --2- 172.21.15.7:0/3104805109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56031aa13f60 0x56031aa24ff0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4009ce0 tx=0x7f9ae40039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368d7fc1ad47ef56 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:03.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.304+0000 7f9adcff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:23:03.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.305+0000 7f9adcff9700 1 -- 172.21.15.7:0/3104805109 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4042060 con 0x56031aa13f60 2024-06-10T01:23:03.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.305+0000 7f9b06134b80 1 -- 172.21.15.7:0/3104805109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9aec004bb0 con 0x56031aa13f60 2024-06-10T01:23:03.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.305+0000 7f9adcff9700 1 -- 172.21.15.7:0/3104805109 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f9ae402d440 con 0x56031aa13f60 2024-06-10T01:23:03.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.305+0000 7f9adcff9700 1 -- 172.21.15.7:0/3104805109 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f9ae4049050 con 0x56031aa13f60 2024-06-10T01:23:03.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.305+0000 7f9b06134b80 10 client.19641 fetch_fsmap learned FSMap version 50 2024-06-10T01:23:03.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.305+0000 7f9b06134b80 1 -- 172.21.15.7:0/3104805109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x56031aa96bd0 con 0x56031aa13f60 2024-06-10T01:23:03.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.305+0000 7f9adcff9700 1 -- 172.21.15.7:0/3104805109 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9aec004bb0 con 0x56031aa13f60 2024-06-10T01:23:03.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.305+0000 7f9adcff9700 1 -- 172.21.15.7:0/3104805109 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7f9ae400f460 con 0x56031aa13f60 2024-06-10T01:23:03.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.305+0000 7f9b06134b80 10 client.19641 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:23:03.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.305+0000 7f9b06134b80 10 client.19641 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:23:03.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.305+0000 7f9b06134b80 20 client.19641 populate_metadata read hostname 'smithi007' 2024-06-10T01:23:03.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.305+0000 7f9b06134b80 1 --2- 172.21.15.7:0/3104805109 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x56031aa2c400 0x56031aa992c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:03.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.305+0000 7f9b06134b80 4 client.19641 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:23:03.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.305+0000 7f9b06134b80 1 -- 172.21.15.7:0/3104805109 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56031a9672d0 con 0x56031aa2c400 2024-06-10T01:23:03.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.306+0000 7f9af964d700 1 --2- 172.21.15.7:0/3104805109 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x56031aa2c400 0x56031aa992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:03.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.306+0000 7f9af964d700 1 --2- 172.21.15.7:0/3104805109 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x56031aa2c400 0x56031aa992c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7256a952e5b42885 server_cookie=206d1355b06cfbc6 in_seq=0 out_seq=0 2024-06-10T01:23:03.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.306+0000 7f9adcff9700 10 client.19641 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:23:03.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.306+0000 7f9adcff9700 1 -- 172.21.15.7:0/3104805109 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56031a9672d0 con 0x56031aa2c400 2024-06-10T01:23:03.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.306+0000 7f9adcff9700 10 client.19641 handle_command_reply: tid=0 2024-06-10T01:23:03.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.306+0000 7f9b06134b80 1 client.19641 shutdown 2024-06-10T01:23:03.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.306+0000 7f9b06134b80 1 -- 172.21.15.7:0/3104805109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56031aa13f60 msgr2=0x56031aa24ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:03.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.306+0000 7f9b06134b80 1 --2- 172.21.15.7:0/3104805109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56031aa13f60 0x56031aa24ff0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4009ce0 tx=0x7f9ae40039d0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.306+0000 7f9b06134b80 1 -- 172.21.15.7:0/3104805109 shutdown_connections 2024-06-10T01:23:03.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.306+0000 7f9b06134b80 1 --2- 172.21.15.7:0/3104805109 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x56031aa2c400 0x56031aa992c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.306+0000 7f9b06134b80 1 --2- 172.21.15.7:0/3104805109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56031aa14880 0x56031aa33510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.306+0000 7f9b06134b80 1 --2- 172.21.15.7:0/3104805109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56031aa0e7a0 0x56031aa24ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.306+0000 7f9adcff9700 0 client.19641 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:23:03.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.306+0000 7f9b06134b80 1 --2- 172.21.15.7:0/3104805109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56031aa13f60 0x56031aa24ff0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.306+0000 7f9b06134b80 1 -- 172.21.15.7:0/3104805109 >> 172.21.15.7:0/3104805109 conn(0x56031a93b080 msgr2=0x56031aa190f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:03.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.307+0000 7f9b06134b80 1 -- 172.21.15.7:0/3104805109 shutdown_connections 2024-06-10T01:23:03.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.307+0000 7f9b06134b80 1 -- 172.21.15.7:0/3104805109 wait complete. 2024-06-10T01:23:03.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.307+0000 7f9b06134b80 20 client.19641 trim_cache size 0 max 16384 2024-06-10T01:23:03.306 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:23:03.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.307+0000 7f9afb8b1700 1 -- 172.21.15.7:0/885451923 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9ae00a2ca0 msgr2=0x7f9ae00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:03.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.307+0000 7f9afb8b1700 1 --2- 172.21.15.7:0/885451923 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9ae00a2ca0 0x7f9ae00a5160 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9ae400b2e0 tx=0x7f9ae4005960 comp rx=0 tx=0).stop 2024-06-10T01:23:03.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.307+0000 7f9afb8b1700 1 -- 172.21.15.7:0/885451923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9af41024e0 msgr2=0x7f9af419e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:03.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.307+0000 7f9afb8b1700 1 --2- 172.21.15.7:0/885451923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9af41024e0 0x7f9af419e840 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f9af000d950 tx=0x7f9af000dc60 comp rx=0 tx=0).stop 2024-06-10T01:23:03.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.307+0000 7f9afb8b1700 1 -- 172.21.15.7:0/885451923 shutdown_connections 2024-06-10T01:23:03.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.307+0000 7f9afb8b1700 1 --2- 172.21.15.7:0/885451923 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9ae00a2ca0 0x7f9ae00a5160 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.307+0000 7f9afb8b1700 1 --2- 172.21.15.7:0/885451923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9af41084f0 0x7f9af4198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.307+0000 7f9afb8b1700 1 --2- 172.21.15.7:0/885451923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9af41024e0 0x7f9af419e840 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.307+0000 7f9afb8b1700 1 --2- 172.21.15.7:0/885451923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9af4102e80 0x7f9af419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:03.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.307+0000 7f9afb8b1700 1 -- 172.21.15.7:0/885451923 >> 172.21.15.7:0/885451923 conn(0x7f9af4079530 msgr2=0x7f9af41005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:03.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.307+0000 7f9afb8b1700 1 -- 172.21.15.7:0/885451923 shutdown_connections 2024-06-10T01:23:03.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:03.307+0000 7f9afb8b1700 1 -- 172.21.15.7:0/885451923 wait complete. 2024-06-10T01:23:03.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:03 smithi078 ceph-mon[36683]: pgmap v944: 129 pgs: 129 active+clean; 587 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 5.4 MiB/s wr, 292 op/s 2024-06-10T01:23:03.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:03 smithi175 ceph-mon[36636]: pgmap v944: 129 pgs: 129 active+clean; 587 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 5.4 MiB/s wr, 292 op/s 2024-06-10T01:23:03.530 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 27.96 MB/sec execute 100 sec latency 16128.158 ms 2024-06-10T01:23:04.082 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:23:04.501 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:23:04.530 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 27.69 MB/sec execute 101 sec latency 17128.204 ms 2024-06-10T01:23:05.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:05 smithi007 ceph-mon[32398]: pgmap v945: 129 pgs: 129 active+clean; 587 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 5.0 MiB/s wr, 272 op/s 2024-06-10T01:23:05.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:05 smithi078 ceph-mon[36683]: pgmap v945: 129 pgs: 129 active+clean; 587 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 5.0 MiB/s wr, 272 op/s 2024-06-10T01:23:05.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:05 smithi175 ceph-mon[36636]: pgmap v945: 129 pgs: 129 active+clean; 587 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 5.0 MiB/s wr, 272 op/s 2024-06-10T01:23:05.530 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 27.41 MB/sec execute 102 sec latency 18128.288 ms 2024-06-10T01:23:06.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.430+0000 7f98b6821700 1 -- 172.21.15.7:0/4228232100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98b0101610 msgr2=0x7f98b01019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:06.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.430+0000 7f98b6821700 1 --2- 172.21.15.7:0/4228232100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98b0101610 0x7f98b01019f0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f98a00099c0 tx=0x7f98a0009cd0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.430+0000 7f98b6821700 1 -- 172.21.15.7:0/4228232100 shutdown_connections 2024-06-10T01:23:06.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.430+0000 7f98b6821700 1 --2- 172.21.15.7:0/4228232100 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98b01066f0 0x7f98b010e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.430+0000 7f98b6821700 1 --2- 172.21.15.7:0/4228232100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98b0101610 0x7f98b01019f0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.430+0000 7f98b6821700 1 --2- 172.21.15.7:0/4228232100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98b0101fc0 0x7f98b0106010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.430+0000 7f98b6821700 1 -- 172.21.15.7:0/4228232100 >> 172.21.15.7:0/4228232100 conn(0x7f98b0078430 msgr2=0x7f98b0078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:06.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.430+0000 7f98b6821700 1 -- 172.21.15.7:0/4228232100 shutdown_connections 2024-06-10T01:23:06.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.430+0000 7f98b6821700 1 -- 172.21.15.7:0/4228232100 wait complete. 2024-06-10T01:23:06.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.430+0000 7f98b6821700 1 Processor -- start 2024-06-10T01:23:06.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.431+0000 7f98b6821700 1 -- start start 2024-06-10T01:23:06.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.431+0000 7f98b6821700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98b0101610 0x7f98b01030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:06.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.431+0000 7f98b6821700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98b0101fc0 0x7f98b01035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:06.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.431+0000 7f98b6821700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98b01066f0 0x7f98b0103b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:06.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.431+0000 7f98b6821700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f98b01116d0 con 0x7f98b0101610 2024-06-10T01:23:06.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.431+0000 7f98b6821700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f98b0111550 con 0x7f98b0101fc0 2024-06-10T01:23:06.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.431+0000 7f98b6821700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f98b0111850 con 0x7f98b01066f0 2024-06-10T01:23:06.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.431+0000 7f98affff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98b0101610 0x7f98b01030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:06.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.431+0000 7f98affff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98b0101610 0x7f98b01030b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38892/0 (socket says 172.21.15.7:38892) 2024-06-10T01:23:06.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.431+0000 7f98b4dbe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98b01066f0 0x7f98b0103b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:06.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.431+0000 7f98affff700 1 -- 172.21.15.7:0/2081053370 learned_addr learned my addr 172.21.15.7:0/2081053370 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:06.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.431+0000 7f98af7fe700 1 --2- 172.21.15.7:0/2081053370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98b0101fc0 0x7f98b01035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:06.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.432+0000 7f98affff700 1 -- 172.21.15.7:0/2081053370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98b01066f0 msgr2=0x7f98b0103b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:06.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.432+0000 7f98affff700 1 --2- 172.21.15.7:0/2081053370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98b01066f0 0x7f98b0103b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.432+0000 7f98affff700 1 -- 172.21.15.7:0/2081053370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98b0101fc0 msgr2=0x7f98b01035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:06.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.432+0000 7f98affff700 1 --2- 172.21.15.7:0/2081053370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98b0101fc0 0x7f98b01035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.432+0000 7f98affff700 1 -- 172.21.15.7:0/2081053370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b01a71b0 con 0x7f98b0101610 2024-06-10T01:23:06.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.432+0000 7f98af7fe700 1 --2- 172.21.15.7:0/2081053370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98b0101fc0 0x7f98b01035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:23:06.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.432+0000 7f98b4dbe700 1 --2- 172.21.15.7:0/2081053370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98b01066f0 0x7f98b0103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:23:06.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.432+0000 7f98affff700 1 --2- 172.21.15.7:0/2081053370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98b0101610 0x7f98b01030b0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f98a00059d0 tx=0x7f98a00038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:06.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.432+0000 7f98ad7fa700 1 -- 172.21.15.7:0/2081053370 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a0027070 con 0x7f98b0101610 2024-06-10T01:23:06.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.432+0000 7f98b6821700 1 -- 172.21.15.7:0/2081053370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98b01a7440 con 0x7f98b0101610 2024-06-10T01:23:06.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.432+0000 7f98ad7fa700 1 -- 172.21.15.7:0/2081053370 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98a0004050 con 0x7f98b0101610 2024-06-10T01:23:06.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.432+0000 7f98ad7fa700 1 -- 172.21.15.7:0/2081053370 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a0022db0 con 0x7f98b0101610 2024-06-10T01:23:06.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.432+0000 7f98b6821700 1 -- 172.21.15.7:0/2081053370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98b01a78f0 con 0x7f98b0101610 2024-06-10T01:23:06.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.434+0000 7f98ad7fa700 1 -- 172.21.15.7:0/2081053370 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f98a002a070 con 0x7f98b0101610 2024-06-10T01:23:06.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.435+0000 7f98ad7fa700 1 --2- 172.21.15.7:0/2081053370 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f98980a2ca0 0x7f98980a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:06.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.435+0000 7f98ad7fa700 1 -- 172.21.15.7:0/2081053370 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f98a00d0010 con 0x7f98b0101610 2024-06-10T01:23:06.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.435+0000 7f98af7fe700 1 --2- 172.21.15.7:0/2081053370 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f98980a2ca0 0x7f98980a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:06.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.436+0000 7f98af7fe700 1 --2- 172.21.15.7:0/2081053370 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f98980a2ca0 0x7f98980a5160 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f98a4005950 tx=0x7f98a40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:06.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.436+0000 7f98c10a4b80 1 Processor -- start 2024-06-10T01:23:06.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.436+0000 7f98c10a4b80 1 -- start start 2024-06-10T01:23:06.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.436+0000 7f98c10a4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5623b990b640 0x5623b990ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:06.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.436+0000 7f98c10a4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623b990bf60 0x5623b99059e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:06.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.436+0000 7f98c10a4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623b9905f20 0x5623b9906340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:06.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.436+0000 7f98c10a4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5623b9857810 con 0x5623b9905f20 2024-06-10T01:23:06.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.436+0000 7f98c10a4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5623b97ff720 con 0x5623b990bf60 2024-06-10T01:23:06.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.436+0000 7f98c10a4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5623b98537d0 con 0x5623b990b640 2024-06-10T01:23:06.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.436+0000 7f98b4dbe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623b9905f20 0x5623b9906340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:06.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.436+0000 7f98b4dbe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623b9905f20 0x5623b9906340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38902/0 (socket says 172.21.15.7:38902) 2024-06-10T01:23:06.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.436+0000 7f98b4dbe700 1 -- 172.21.15.7:0/3880095250 learned_addr learned my addr 172.21.15.7:0/3880095250 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:06.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.436+0000 7f98af7fe700 1 --2- 172.21.15.7:0/3880095250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623b990bf60 0x5623b99059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:06.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.436+0000 7f98affff700 1 --2- 172.21.15.7:0/3880095250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5623b990b640 0x5623b990ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:06.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.436+0000 7f98b4dbe700 1 -- 172.21.15.7:0/3880095250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5623b990b640 msgr2=0x5623b990ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:06.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.437+0000 7f98b4dbe700 1 --2- 172.21.15.7:0/3880095250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5623b990b640 0x5623b990ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.437+0000 7f98b4dbe700 1 -- 172.21.15.7:0/3880095250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623b990bf60 msgr2=0x5623b99059e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:06.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.437+0000 7f98b4dbe700 1 --2- 172.21.15.7:0/3880095250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623b990bf60 0x5623b99059e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.437+0000 7f98b4dbe700 1 -- 172.21.15.7:0/3880095250 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623b97f5f50 con 0x5623b9905f20 2024-06-10T01:23:06.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.437+0000 7f98affff700 1 --2- 172.21.15.7:0/3880095250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5623b990b640 0x5623b990ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:23:06.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.437+0000 7f98b4dbe700 1 --2- 172.21.15.7:0/3880095250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623b9905f20 0x5623b9906340 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f98a800a730 tx=0x7f98a80078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f7433120dbaa35c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:06.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.437+0000 7f98967fc700 1 -- 172.21.15.7:0/3880095250 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a8014600 con 0x5623b9905f20 2024-06-10T01:23:06.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.437+0000 7f98967fc700 1 -- 172.21.15.7:0/3880095250 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98a8019070 con 0x5623b9905f20 2024-06-10T01:23:06.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.437+0000 7f98967fc700 1 -- 172.21.15.7:0/3880095250 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a8008ac0 con 0x5623b9905f20 2024-06-10T01:23:06.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.438+0000 7f98c10a4b80 1 -- 172.21.15.7:0/3880095250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623b9905f20 msgr2=0x5623b9906340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:06.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.438+0000 7f98c10a4b80 1 --2- 172.21.15.7:0/3880095250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623b9905f20 0x5623b9906340 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f98a800a730 tx=0x7f98a80078d0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.438+0000 7f98c10a4b80 1 -- 172.21.15.7:0/3880095250 shutdown_connections 2024-06-10T01:23:06.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.438+0000 7f98c10a4b80 1 --2- 172.21.15.7:0/3880095250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623b9905f20 0x5623b9906340 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.438+0000 7f98c10a4b80 1 --2- 172.21.15.7:0/3880095250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5623b990b640 0x5623b990ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.438+0000 7f98c10a4b80 1 --2- 172.21.15.7:0/3880095250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623b990bf60 0x5623b99059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.438+0000 7f98c10a4b80 1 -- 172.21.15.7:0/3880095250 >> 172.21.15.7:0/3880095250 conn(0x5623b9830cc0 msgr2=0x5623b9900850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:06.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.438+0000 7f98c10a4b80 1 -- 172.21.15.7:0/3880095250 shutdown_connections 2024-06-10T01:23:06.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.438+0000 7f98c10a4b80 1 -- 172.21.15.7:0/3880095250 wait complete. 2024-06-10T01:23:06.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.438+0000 7f98c10a4b80 1 -- start start 2024-06-10T01:23:06.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.439+0000 7f98c10a4b80 1 Processor -- start 2024-06-10T01:23:06.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.439+0000 7f98c10a4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623b9905f20 0x5623b9911d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:06.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.439+0000 7f98c10a4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623b990b640 0x5623b990f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:06.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.439+0000 7f98c10a4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5623b990bf60 0x5623b990f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:06.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.439+0000 7f98affff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623b9905f20 0x5623b9911d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:06.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.439+0000 7f98c10a4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5623b9912970 con 0x5623b9905f20 2024-06-10T01:23:06.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.439+0000 7f98affff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623b9905f20 0x5623b9911d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38918/0 (socket says 172.21.15.7:38918) 2024-06-10T01:23:06.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.439+0000 7f98c10a4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5623b97f7bf0 con 0x5623b990b640 2024-06-10T01:23:06.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.439+0000 7f98affff700 1 -- 172.21.15.7:0/2519096275 learned_addr learned my addr 172.21.15.7:0/2519096275 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:06.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.439+0000 7f98c10a4b80 1 -- 172.21.15.7:0/2519096275 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5623b9912af0 con 0x5623b990bf60 2024-06-10T01:23:06.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.439+0000 7f98c10a4b80 10 client.0 _ll_register_callbacks cb 0x5623b97f1920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:23:06.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.439+0000 7f98af7fe700 1 --2- 172.21.15.7:0/2519096275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623b990b640 0x5623b990f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:06.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.439+0000 7f98b4dbe700 1 --2- 172.21.15.7:0/2519096275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5623b990bf60 0x5623b990f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:06.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.439+0000 7f98affff700 1 -- 172.21.15.7:0/2519096275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5623b990bf60 msgr2=0x5623b990f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:06.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.439+0000 7f98affff700 1 --2- 172.21.15.7:0/2519096275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5623b990bf60 0x5623b990f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.439+0000 7f98affff700 1 -- 172.21.15.7:0/2519096275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623b990b640 msgr2=0x5623b990f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:06.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.440+0000 7f98affff700 1 --2- 172.21.15.7:0/2519096275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623b990b640 0x5623b990f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.440+0000 7f98affff700 1 -- 172.21.15.7:0/2519096275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623b97a68d0 con 0x5623b9905f20 2024-06-10T01:23:06.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.440+0000 7f98affff700 1 --2- 172.21.15.7:0/2519096275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623b9905f20 0x5623b9911d30 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f98a009c220 tx=0x7f98a0022750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395f80f474992253 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:06.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.440+0000 7f988f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:23:06.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.440+0000 7f988f7fe700 1 -- 172.21.15.7:0/2519096275 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a00b5060 con 0x5623b9905f20 2024-06-10T01:23:06.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.440+0000 7f988f7fe700 1 -- 172.21.15.7:0/2519096275 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f98a009c980 con 0x5623b9905f20 2024-06-10T01:23:06.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.440+0000 7f98c10a4b80 1 -- 172.21.15.7:0/2519096275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98a8019070 con 0x5623b9905f20 2024-06-10T01:23:06.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.440+0000 7f988f7fe700 1 -- 172.21.15.7:0/2519096275 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f98a009c520 con 0x5623b9905f20 2024-06-10T01:23:06.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.440+0000 7f98c10a4b80 10 client.19653 fetch_fsmap learned FSMap version 50 2024-06-10T01:23:06.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.440+0000 7f98c10a4b80 1 -- 172.21.15.7:0/2519096275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5623b991d2d0 con 0x5623b9905f20 2024-06-10T01:23:06.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.440+0000 7f988f7fe700 1 -- 172.21.15.7:0/2519096275 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98a8019070 con 0x5623b9905f20 2024-06-10T01:23:06.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.441+0000 7f988f7fe700 1 -- 172.21.15.7:0/2519096275 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7f98a00a4c70 con 0x5623b9905f20 2024-06-10T01:23:06.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.441+0000 7f98c10a4b80 10 client.19653 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:23:06.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.441+0000 7f98c10a4b80 10 client.19653 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:23:06.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.441+0000 7f98c10a4b80 20 client.19653 populate_metadata read hostname 'smithi007' 2024-06-10T01:23:06.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.441+0000 7f98c10a4b80 1 --2- 172.21.15.7:0/2519096275 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5623b991f890 0x5623b9921d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:06.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.441+0000 7f98c10a4b80 4 client.19653 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:23:06.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.441+0000 7f98c10a4b80 1 -- 172.21.15.7:0/2519096275 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623b9864d10 con 0x5623b991f890 2024-06-10T01:23:06.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.441+0000 7f98af7fe700 1 --2- 172.21.15.7:0/2519096275 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5623b991f890 0x5623b9921d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:06.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.441+0000 7f988f7fe700 10 client.19653 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:06.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.441+0000 7f98af7fe700 1 --2- 172.21.15.7:0/2519096275 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5623b991f890 0x5623b9921d10 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c4ba0cf61270928 server_cookie=a5937c3d5537d7a9 in_seq=0 out_seq=0 2024-06-10T01:23:06.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.442+0000 7f988f7fe700 1 -- 172.21.15.7:0/2519096275 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5623b9864d10 con 0x5623b991f890 2024-06-10T01:23:06.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.442+0000 7f988f7fe700 10 client.19653 handle_command_reply: tid=0 2024-06-10T01:23:06.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.442+0000 7f98c10a4b80 1 client.19653 shutdown 2024-06-10T01:23:06.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.442+0000 7f98c10a4b80 1 -- 172.21.15.7:0/2519096275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623b9905f20 msgr2=0x5623b9911d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:06.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.442+0000 7f98c10a4b80 1 --2- 172.21.15.7:0/2519096275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623b9905f20 0x5623b9911d30 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f98a009c220 tx=0x7f98a0022750 comp rx=0 tx=0).stop 2024-06-10T01:23:06.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.442+0000 7f98c10a4b80 1 -- 172.21.15.7:0/2519096275 shutdown_connections 2024-06-10T01:23:06.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.442+0000 7f98c10a4b80 1 --2- 172.21.15.7:0/2519096275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5623b990bf60 0x5623b990f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.442+0000 7f98c10a4b80 1 --2- 172.21.15.7:0/2519096275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623b9905f20 0x5623b9911d30 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.442+0000 7f98c10a4b80 1 --2- 172.21.15.7:0/2519096275 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5623b991f890 0x5623b9921d10 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.442+0000 7f98c10a4b80 1 --2- 172.21.15.7:0/2519096275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623b990b640 0x5623b990f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.442+0000 7f988f7fe700 0 client.19653 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:06.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.442+0000 7f98c10a4b80 1 -- 172.21.15.7:0/2519096275 >> 172.21.15.7:0/2519096275 conn(0x5623b9830cc0 msgr2=0x5623b99049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:06.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.443+0000 7f98c10a4b80 1 -- 172.21.15.7:0/2519096275 shutdown_connections 2024-06-10T01:23:06.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.443+0000 7f98c10a4b80 1 -- 172.21.15.7:0/2519096275 wait complete. 2024-06-10T01:23:06.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.443+0000 7f98c10a4b80 20 client.19653 trim_cache size 0 max 16384 2024-06-10T01:23:06.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.459+0000 7f98c10a4b80 1 Processor -- start 2024-06-10T01:23:06.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.459+0000 7f98c10a4b80 1 -- start start 2024-06-10T01:23:06.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.459+0000 7f98c10a4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5623b9905f20 0x5623b991bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:06.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.459+0000 7f98c10a4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623b990b640 0x5623b99059e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:06.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.459+0000 7f98c10a4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623b990bf60 0x5623b990ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:06.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.459+0000 7f98c10a4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f98b01b2d10 con 0x5623b990b640 2024-06-10T01:23:06.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.459+0000 7f98c10a4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f98b01b05a0 con 0x5623b990bf60 2024-06-10T01:23:06.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.459+0000 7f98c10a4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f98b01ae3f0 con 0x5623b9905f20 2024-06-10T01:23:06.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.459+0000 7f98af7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623b990b640 0x5623b99059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:06.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.459+0000 7f98af7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623b990b640 0x5623b99059e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38920/0 (socket says 172.21.15.7:38920) 2024-06-10T01:23:06.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.459+0000 7f98af7fe700 1 -- 172.21.15.7:0/4199683036 learned_addr learned my addr 172.21.15.7:0/4199683036 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:06.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.459+0000 7f98b4dbe700 1 --2- 172.21.15.7:0/4199683036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623b990bf60 0x5623b990ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:06.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.459+0000 7f98affff700 1 --2- 172.21.15.7:0/4199683036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5623b9905f20 0x5623b991bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:06.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.459+0000 7f98af7fe700 1 -- 172.21.15.7:0/4199683036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5623b9905f20 msgr2=0x5623b991bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:06.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.460+0000 7f98af7fe700 1 --2- 172.21.15.7:0/4199683036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5623b9905f20 0x5623b991bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.460+0000 7f98af7fe700 1 -- 172.21.15.7:0/4199683036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623b990bf60 msgr2=0x5623b990ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:06.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.460+0000 7f98af7fe700 1 --2- 172.21.15.7:0/4199683036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623b990bf60 0x5623b990ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.460+0000 7f98af7fe700 1 -- 172.21.15.7:0/4199683036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623b990d5a0 con 0x5623b990b640 2024-06-10T01:23:06.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.460+0000 7f98affff700 1 --2- 172.21.15.7:0/4199683036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5623b9905f20 0x5623b991bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:23:06.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.460+0000 7f98af7fe700 1 --2- 172.21.15.7:0/4199683036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623b990b640 0x5623b99059e0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f98b0104b40 tx=0x7f98a4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f64feca21a714d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:06.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.460+0000 7f9895ffb700 1 -- 172.21.15.7:0/4199683036 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a402e070 con 0x5623b990b640 2024-06-10T01:23:06.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.460+0000 7f9895ffb700 1 -- 172.21.15.7:0/4199683036 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98a4003d60 con 0x5623b990b640 2024-06-10T01:23:06.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.460+0000 7f9895ffb700 1 -- 172.21.15.7:0/4199683036 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a4011440 con 0x5623b990b640 2024-06-10T01:23:06.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.461+0000 7f98c10a4b80 1 -- 172.21.15.7:0/4199683036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623b990b640 msgr2=0x5623b99059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.461+0000 7f98c10a4b80 1 --2- 172.21.15.7:0/4199683036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623b990b640 0x5623b99059e0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f98b0104b40 tx=0x7f98a4015000 comp rx=0 tx=0).stop 2024-06-10T01:23:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.461+0000 7f98c10a4b80 1 -- 172.21.15.7:0/4199683036 shutdown_connections 2024-06-10T01:23:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.461+0000 7f98c10a4b80 1 --2- 172.21.15.7:0/4199683036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623b990bf60 0x5623b990ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.461+0000 7f98c10a4b80 1 --2- 172.21.15.7:0/4199683036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5623b9905f20 0x5623b991bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.461+0000 7f98c10a4b80 1 --2- 172.21.15.7:0/4199683036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623b990b640 0x5623b99059e0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.461+0000 7f98c10a4b80 1 -- 172.21.15.7:0/4199683036 >> 172.21.15.7:0/4199683036 conn(0x5623b9830cc0 msgr2=0x5623b990c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.461+0000 7f98c10a4b80 1 -- 172.21.15.7:0/4199683036 shutdown_connections 2024-06-10T01:23:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.461+0000 7f98c10a4b80 1 -- 172.21.15.7:0/4199683036 wait complete. 2024-06-10T01:23:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.461+0000 7f98c10a4b80 1 -- start start 2024-06-10T01:23:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.461+0000 7f98c10a4b80 1 Processor -- start 2024-06-10T01:23:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.462+0000 7f98c10a4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5623b9905f20 0x5623b9918580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.462+0000 7f98c10a4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623b990b640 0x5623b9926ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.462+0000 7f98c10a4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623b990bf60 0x5623b9927570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.462+0000 7f98c10a4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f98b01aea60 con 0x5623b990b640 2024-06-10T01:23:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.462+0000 7f98c10a4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f98b01afed0 con 0x5623b990bf60 2024-06-10T01:23:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.462+0000 7f98c10a4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f98b01af130 con 0x5623b9905f20 2024-06-10T01:23:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.462+0000 7f98c10a4b80 10 client.0 _ll_register_callbacks cb 0x5623b97f1920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:23:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.462+0000 7f98af7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623b990b640 0x5623b9926ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.462+0000 7f98af7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623b990b640 0x5623b9926ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38936/0 (socket says 172.21.15.7:38936) 2024-06-10T01:23:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.462+0000 7f98af7fe700 1 -- 172.21.15.7:0/1001868697 learned_addr learned my addr 172.21.15.7:0/1001868697 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.462+0000 7f98affff700 1 --2- 172.21.15.7:0/1001868697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5623b9905f20 0x5623b9918580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.462+0000 7f98b4dbe700 1 --2- 172.21.15.7:0/1001868697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623b990bf60 0x5623b9927570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.462+0000 7f98af7fe700 1 -- 172.21.15.7:0/1001868697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5623b9905f20 msgr2=0x5623b9918580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.462+0000 7f98af7fe700 1 --2- 172.21.15.7:0/1001868697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5623b9905f20 0x5623b9918580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.462+0000 7f98af7fe700 1 -- 172.21.15.7:0/1001868697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623b990bf60 msgr2=0x5623b9927570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.462+0000 7f98af7fe700 1 --2- 172.21.15.7:0/1001868697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623b990bf60 0x5623b9927570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.462+0000 7f98af7fe700 1 -- 172.21.15.7:0/1001868697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623b9927db0 con 0x5623b990b640 2024-06-10T01:23:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.462+0000 7f98af7fe700 1 --2- 172.21.15.7:0/1001868697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623b990b640 0x5623b9926ff0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f98a402c040 tx=0x7f98a400fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c610550b1c4091d5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.462+0000 7f988f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:23:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.463+0000 7f988f7fe700 1 -- 172.21.15.7:0/1001868697 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a402e040 con 0x5623b990b640 2024-06-10T01:23:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.463+0000 7f988f7fe700 1 -- 172.21.15.7:0/1001868697 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f98a40299d0 con 0x5623b990b640 2024-06-10T01:23:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.463+0000 7f98c10a4b80 1 -- 172.21.15.7:0/1001868697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98a4003d60 con 0x5623b990b640 2024-06-10T01:23:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.463+0000 7f988f7fe700 1 -- 172.21.15.7:0/1001868697 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f98a4041050 con 0x5623b990b640 2024-06-10T01:23:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.463+0000 7f98c10a4b80 10 client.19659 fetch_fsmap learned FSMap version 50 2024-06-10T01:23:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.463+0000 7f98c10a4b80 1 -- 172.21.15.7:0/1001868697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5623b998a610 con 0x5623b990b640 2024-06-10T01:23:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.463+0000 7f988f7fe700 1 -- 172.21.15.7:0/1001868697 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98a4003d60 con 0x5623b990b640 2024-06-10T01:23:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.463+0000 7f988f7fe700 1 -- 172.21.15.7:0/1001868697 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7f98a4011710 con 0x5623b990b640 2024-06-10T01:23:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.463+0000 7f98c10a4b80 10 client.19659 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:23:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.463+0000 7f98c10a4b80 10 client.19659 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:23:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.463+0000 7f98c10a4b80 20 client.19659 populate_metadata read hostname 'smithi007' 2024-06-10T01:23:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.463+0000 7f98c10a4b80 1 --2- 172.21.15.7:0/1001868697 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5623b998acb0 0x5623b998d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.463+0000 7f98c10a4b80 4 client.19659 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:23:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.463+0000 7f98c10a4b80 1 -- 172.21.15.7:0/1001868697 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5623b985f290 con 0x5623b998acb0 2024-06-10T01:23:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.464+0000 7f98affff700 1 --2- 172.21.15.7:0/1001868697 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5623b998acb0 0x5623b998d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.464+0000 7f98affff700 1 --2- 172.21.15.7:0/1001868697 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5623b998acb0 0x5623b998d0d0 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2bec5858a3bd1c0 server_cookie=611334ba638a1795 in_seq=0 out_seq=0 2024-06-10T01:23:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.464+0000 7f988f7fe700 10 client.19659 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.464+0000 7f988f7fe700 1 -- 172.21.15.7:0/1001868697 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5623b985f290 con 0x5623b998acb0 2024-06-10T01:23:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.464+0000 7f988f7fe700 10 client.19659 handle_command_reply: tid=0 2024-06-10T01:23:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.464+0000 7f98c10a4b80 1 client.19659 shutdown 2024-06-10T01:23:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.464+0000 7f98c10a4b80 1 -- 172.21.15.7:0/1001868697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623b990b640 msgr2=0x5623b9926ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.464+0000 7f98c10a4b80 1 --2- 172.21.15.7:0/1001868697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623b990b640 0x5623b9926ff0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f98a402c040 tx=0x7f98a400fae0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.465+0000 7f98c10a4b80 1 -- 172.21.15.7:0/1001868697 shutdown_connections 2024-06-10T01:23:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.465+0000 7f98c10a4b80 1 --2- 172.21.15.7:0/1001868697 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5623b998acb0 0x5623b998d0d0 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.465+0000 7f98c10a4b80 1 --2- 172.21.15.7:0/1001868697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623b990bf60 0x5623b9927570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.465+0000 7f98c10a4b80 1 --2- 172.21.15.7:0/1001868697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5623b9905f20 0x5623b9918580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.465+0000 7f98c10a4b80 1 --2- 172.21.15.7:0/1001868697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623b990b640 0x5623b9926ff0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.465+0000 7f988f7fe700 0 client.19659 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.465+0000 7f98c10a4b80 1 -- 172.21.15.7:0/1001868697 >> 172.21.15.7:0/1001868697 conn(0x5623b9830cc0 msgr2=0x5623b990c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.465+0000 7f98c10a4b80 1 -- 172.21.15.7:0/1001868697 shutdown_connections 2024-06-10T01:23:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.465+0000 7f98c10a4b80 1 -- 172.21.15.7:0/1001868697 wait complete. 2024-06-10T01:23:06.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.465+0000 7f98c10a4b80 20 client.19659 trim_cache size 0 max 16384 2024-06-10T01:23:06.464 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:23:06.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.465+0000 7f98b6821700 1 -- 172.21.15.7:0/2081053370 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f98980a2ca0 msgr2=0x7f98980a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.465+0000 7f98b6821700 1 --2- 172.21.15.7:0/2081053370 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f98980a2ca0 0x7f98980a5160 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f98a4005950 tx=0x7f98a40058e0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.465+0000 7f98b6821700 1 -- 172.21.15.7:0/2081053370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98b0101610 msgr2=0x7f98b01030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.465+0000 7f98b6821700 1 --2- 172.21.15.7:0/2081053370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98b0101610 0x7f98b01030b0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f98a00059d0 tx=0x7f98a00038b0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.466+0000 7f98b6821700 1 -- 172.21.15.7:0/2081053370 shutdown_connections 2024-06-10T01:23:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.466+0000 7f98b6821700 1 --2- 172.21.15.7:0/2081053370 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f98980a2ca0 0x7f98980a5160 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.466+0000 7f98b6821700 1 --2- 172.21.15.7:0/2081053370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98b01066f0 0x7f98b0103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.466+0000 7f98b6821700 1 --2- 172.21.15.7:0/2081053370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98b0101610 0x7f98b01030b0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.466+0000 7f98b6821700 1 --2- 172.21.15.7:0/2081053370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98b0101fc0 0x7f98b01035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.466+0000 7f98b6821700 1 -- 172.21.15.7:0/2081053370 >> 172.21.15.7:0/2081053370 conn(0x7f98b0078430 msgr2=0x7f98b006aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.466+0000 7f98b6821700 1 -- 172.21.15.7:0/2081053370 shutdown_connections 2024-06-10T01:23:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:06.466+0000 7f98b6821700 1 -- 172.21.15.7:0/2081053370 wait complete. 2024-06-10T01:23:06.530 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 27.15 MB/sec execute 103 sec latency 19128.383 ms 2024-06-10T01:23:07.196 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:07 smithi007 ceph-mon[32398]: pgmap v946: 129 pgs: 129 active+clean; 580 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 5.5 MiB/s wr, 296 op/s 2024-06-10T01:23:07.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:07 smithi078 ceph-mon[36683]: pgmap v946: 129 pgs: 129 active+clean; 580 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 5.5 MiB/s wr, 296 op/s 2024-06-10T01:23:07.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:07 smithi175 ceph-mon[36636]: pgmap v946: 129 pgs: 129 active+clean; 580 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 5.5 MiB/s wr, 296 op/s 2024-06-10T01:23:07.530 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 26.89 MB/sec execute 104 sec latency 20128.440 ms 2024-06-10T01:23:08.226 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:23:08.226 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:23:08.502 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:23:08.530 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 26.63 MB/sec execute 105 sec latency 21128.505 ms 2024-06-10T01:23:09.177 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:23:09.237 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:09 smithi007 ceph-mon[32398]: pgmap v947: 129 pgs: 129 active+clean; 579 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 5.3 MiB/s wr, 291 op/s 2024-06-10T01:23:09.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:09 smithi078 ceph-mon[36683]: pgmap v947: 129 pgs: 129 active+clean; 579 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 5.3 MiB/s wr, 291 op/s 2024-06-10T01:23:09.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:09 smithi175 ceph-mon[36636]: pgmap v947: 129 pgs: 129 active+clean; 579 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 5.3 MiB/s wr, 291 op/s 2024-06-10T01:23:09.531 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 26.38 MB/sec execute 106 sec latency 22128.619 ms 2024-06-10T01:23:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.677+0000 7f221b390640 1 -- 172.21.15.7:0/677846106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22141024b0 msgr2=0x7f2214108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.677+0000 7f221b390640 1 --2- 172.21.15.7:0/677846106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22141024b0 0x7f2214108d40 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f220c009be0 tx=0x7f220c02a6c0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.678+0000 7f221b390640 1 -- 172.21.15.7:0/677846106 shutdown_connections 2024-06-10T01:23:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.678+0000 7f221b390640 1 --2- 172.21.15.7:0/677846106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22141024b0 0x7f2214108d40 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.678+0000 7f221b390640 1 --2- 172.21.15.7:0/677846106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2214101af0 0x7f2214101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.678+0000 7f221b390640 1 --2- 172.21.15.7:0/677846106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2214100900 0x7f2214100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.678+0000 7f221b390640 1 -- 172.21.15.7:0/677846106 >> 172.21.15.7:0/677846106 conn(0x7f22140fbe70 msgr2=0x7f22140fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.678+0000 7f221b390640 1 -- 172.21.15.7:0/677846106 shutdown_connections 2024-06-10T01:23:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.678+0000 7f221b390640 1 -- 172.21.15.7:0/677846106 wait complete. 2024-06-10T01:23:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.678+0000 7f221b390640 1 Processor -- start 2024-06-10T01:23:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.678+0000 7f221b390640 1 -- start start 2024-06-10T01:23:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.678+0000 7f221b390640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2214100900 0x7f2214195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.678+0000 7f221b390640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2214101af0 0x7f2214195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.678+0000 7f221b390640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22141024b0 0x7f221419d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.678+0000 7f221b390640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f221410b480 con 0x7f22141024b0 2024-06-10T01:23:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.678+0000 7f221b390640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f221410b300 con 0x7f2214100900 2024-06-10T01:23:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.679+0000 7f221b390640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f221410b600 con 0x7f2214101af0 2024-06-10T01:23:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.679+0000 7f221992d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22141024b0 0x7f221419d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.679+0000 7f221892b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2214101af0 0x7f2214195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.679+0000 7f221892b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2214101af0 0x7f2214195fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:40792/0 (socket says 172.21.15.7:40792) 2024-06-10T01:23:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.679+0000 7f221992d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22141024b0 0x7f221419d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38952/0 (socket says 172.21.15.7:38952) 2024-06-10T01:23:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.679+0000 7f221892b640 1 -- 172.21.15.7:0/3842163311 learned_addr learned my addr 172.21.15.7:0/3842163311 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.679+0000 7f221912c640 1 --2- 172.21.15.7:0/3842163311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2214100900 0x7f2214195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.679+0000 7f221992d640 1 -- 172.21.15.7:0/3842163311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2214101af0 msgr2=0x7f2214195fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.679+0000 7f221992d640 1 --2- 172.21.15.7:0/3842163311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2214101af0 0x7f2214195fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.679+0000 7f221992d640 1 -- 172.21.15.7:0/3842163311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2214100900 msgr2=0x7f2214195a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.679+0000 7f221992d640 1 --2- 172.21.15.7:0/3842163311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2214100900 0x7f2214195a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.679+0000 7f221992d640 1 -- 172.21.15.7:0/3842163311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f221419d6b0 con 0x7f22141024b0 2024-06-10T01:23:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.679+0000 7f221912c640 1 --2- 172.21.15.7:0/3842163311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2214100900 0x7f2214195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:23:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.679+0000 7f221992d640 1 --2- 172.21.15.7:0/3842163311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22141024b0 0x7f221419d040 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f220c034040 tx=0x7f220c0305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.679+0000 7f220a7fc640 1 -- 172.21.15.7:0/3842163311 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c041020 con 0x7f22141024b0 2024-06-10T01:23:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.679+0000 7f220a7fc640 1 -- 172.21.15.7:0/3842163311 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f220c030740 con 0x7f22141024b0 2024-06-10T01:23:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.679+0000 7f220a7fc640 1 -- 172.21.15.7:0/3842163311 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c036040 con 0x7f22141024b0 2024-06-10T01:23:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.679+0000 7f221b390640 1 -- 172.21.15.7:0/3842163311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f221419d940 con 0x7f22141024b0 2024-06-10T01:23:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.680+0000 7f221b390640 1 -- 172.21.15.7:0/3842163311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f221419ddb0 con 0x7f22141024b0 2024-06-10T01:23:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.682+0000 7f220a7fc640 1 -- 172.21.15.7:0/3842163311 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f220c044020 con 0x7f22141024b0 2024-06-10T01:23:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.682+0000 7f220a7fc640 1 --2- 172.21.15.7:0/3842163311 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f22000a2c00 0x7f22000a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.683+0000 7f220a7fc640 1 -- 172.21.15.7:0/3842163311 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f220c0effc0 con 0x7f22141024b0 2024-06-10T01:23:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.683+0000 7f221912c640 1 --2- 172.21.15.7:0/3842163311 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f22000a2c00 0x7f22000a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.683+0000 7f221912c640 1 --2- 172.21.15.7:0/3842163311 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f22000a2c00 0x7f22000a50c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2214101950 tx=0x7f2210005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.683+0000 7f221e17f740 1 Processor -- start 2024-06-10T01:23:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.683+0000 7f221e17f740 1 -- start start 2024-06-10T01:23:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.683+0000 7f221e17f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c07f1e010 0x556c07f1e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.683+0000 7f221e17f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c07f1ea00 0x556c07f23500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.683+0000 7f221e17f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c07f23b70 0x556c07f26000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.683+0000 7f221e17f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556c07e814f0 con 0x556c07f1e010 2024-06-10T01:23:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.683+0000 7f221e17f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556c07e1f870 con 0x556c07f23b70 2024-06-10T01:23:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.683+0000 7f221e17f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556c07e7d4b0 con 0x556c07f1ea00 2024-06-10T01:23:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.684+0000 7f221912c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c07f1e010 0x556c07f1e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.684+0000 7f221912c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c07f1e010 0x556c07f1e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38966/0 (socket says 172.21.15.7:38966) 2024-06-10T01:23:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.684+0000 7f221912c640 1 -- 172.21.15.7:0/757354922 learned_addr learned my addr 172.21.15.7:0/757354922 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.684+0000 7f221992d640 1 --2- 172.21.15.7:0/757354922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c07f23b70 0x556c07f26000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.684+0000 7f221892b640 1 --2- 172.21.15.7:0/757354922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c07f1ea00 0x556c07f23500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.684+0000 7f221912c640 1 -- 172.21.15.7:0/757354922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c07f1ea00 msgr2=0x556c07f23500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.684+0000 7f221912c640 1 --2- 172.21.15.7:0/757354922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c07f1ea00 0x556c07f23500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.684+0000 7f221912c640 1 -- 172.21.15.7:0/757354922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c07f23b70 msgr2=0x556c07f26000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.684+0000 7f221912c640 1 --2- 172.21.15.7:0/757354922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c07f23b70 0x556c07f26000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.684+0000 7f221912c640 1 -- 172.21.15.7:0/757354922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c07e3b9b0 con 0x556c07f1e010 2024-06-10T01:23:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.684+0000 7f221912c640 1 --2- 172.21.15.7:0/757354922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c07f1e010 0x556c07f1e430 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f2210012480 tx=0x7f2210014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc5935f618698b25 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.684+0000 7f21ff7fe640 1 -- 172.21.15.7:0/757354922 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221001f620 con 0x556c07f1e010 2024-06-10T01:23:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.684+0000 7f21ff7fe640 1 -- 172.21.15.7:0/757354922 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f221001fca0 con 0x556c07f1e010 2024-06-10T01:23:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.684+0000 7f21ff7fe640 1 -- 172.21.15.7:0/757354922 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2210027df0 con 0x556c07f1e010 2024-06-10T01:23:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.685+0000 7f221e17f740 1 -- 172.21.15.7:0/757354922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c07f1e010 msgr2=0x556c07f1e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.685+0000 7f221e17f740 1 --2- 172.21.15.7:0/757354922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c07f1e010 0x556c07f1e430 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f2210012480 tx=0x7f2210014000 comp rx=0 tx=0).stop 2024-06-10T01:23:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.685+0000 7f221e17f740 1 -- 172.21.15.7:0/757354922 shutdown_connections 2024-06-10T01:23:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.685+0000 7f221e17f740 1 --2- 172.21.15.7:0/757354922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c07f23b70 0x556c07f26000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.685+0000 7f221e17f740 1 --2- 172.21.15.7:0/757354922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c07f1ea00 0x556c07f23500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.685+0000 7f221e17f740 1 --2- 172.21.15.7:0/757354922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c07f1e010 0x556c07f1e430 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.685+0000 7f221e17f740 1 -- 172.21.15.7:0/757354922 >> 172.21.15.7:0/757354922 conn(0x556c07f19680 msgr2=0x556c07f19a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.685+0000 7f221e17f740 1 -- 172.21.15.7:0/757354922 shutdown_connections 2024-06-10T01:23:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.685+0000 7f221e17f740 1 -- 172.21.15.7:0/757354922 wait complete. 2024-06-10T01:23:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.686+0000 7f221e17f740 1 -- start start 2024-06-10T01:23:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.686+0000 7f221e17f740 1 Processor -- start 2024-06-10T01:23:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.687+0000 7f221e17f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c07f1e010 0x556c07f28370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.687+0000 7f221e17f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c07f1ea00 0x556c07f288b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.687+0000 7f221e17f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c07f23b70 0x556c07f2f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.687+0000 7f221e17f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556c07f26b20 con 0x556c07f1e010 2024-06-10T01:23:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.687+0000 7f221912c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c07f1e010 0x556c07f28370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.687+0000 7f221912c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c07f1e010 0x556c07f28370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38976/0 (socket says 172.21.15.7:38976) 2024-06-10T01:23:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.687+0000 7f221912c640 1 -- 172.21.15.7:0/2358364259 learned_addr learned my addr 172.21.15.7:0/2358364259 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.687+0000 7f221e17f740 1 -- 172.21.15.7:0/2358364259 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556c07e0aae0 con 0x556c07f23b70 2024-06-10T01:23:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.687+0000 7f221e17f740 1 -- 172.21.15.7:0/2358364259 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556c07f26ca0 con 0x556c07f1ea00 2024-06-10T01:23:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.687+0000 7f221892b640 1 --2- 172.21.15.7:0/2358364259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c07f1ea00 0x556c07f288b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.687+0000 7f221e17f740 10 client.0 _ll_register_callbacks cb 0x556c07e439b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:23:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.687+0000 7f221992d640 1 --2- 172.21.15.7:0/2358364259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c07f23b70 0x556c07f2f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.687+0000 7f221912c640 1 -- 172.21.15.7:0/2358364259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c07f1ea00 msgr2=0x556c07f288b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.687+0000 7f221912c640 1 --2- 172.21.15.7:0/2358364259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c07f1ea00 0x556c07f288b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.687+0000 7f221912c640 1 -- 172.21.15.7:0/2358364259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c07f23b70 msgr2=0x556c07f2f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.687+0000 7f221912c640 1 --2- 172.21.15.7:0/2358364259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c07f23b70 0x556c07f2f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.687+0000 7f221912c640 1 -- 172.21.15.7:0/2358364259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c07b95210 con 0x556c07f1e010 2024-06-10T01:23:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.687+0000 7f221992d640 1 --2- 172.21.15.7:0/2358364259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c07f23b70 0x556c07f2f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:23:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.687+0000 7f221912c640 1 --2- 172.21.15.7:0/2358364259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c07f1e010 0x556c07f28370 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f2210028e10 tx=0x7f2210028e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c6169876d65ab1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.688+0000 7f21fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:23:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.688+0000 7f21fcff9640 1 -- 172.21.15.7:0/2358364259 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2210005030 con 0x556c07f1e010 2024-06-10T01:23:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.688+0000 7f221e17f740 1 -- 172.21.15.7:0/2358364259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f221001fca0 con 0x556c07f1e010 2024-06-10T01:23:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.688+0000 7f21fcff9640 1 -- 172.21.15.7:0/2358364259 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f22100051d0 con 0x556c07f1e010 2024-06-10T01:23:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.688+0000 7f21fcff9640 1 -- 172.21.15.7:0/2358364259 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f22100053d0 con 0x556c07f1e010 2024-06-10T01:23:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.688+0000 7f221e17f740 10 client.19671 fetch_fsmap learned FSMap version 50 2024-06-10T01:23:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.688+0000 7f221e17f740 1 -- 172.21.15.7:0/2358364259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x556c07c61620 con 0x556c07f1e010 2024-06-10T01:23:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.688+0000 7f21fcff9640 1 -- 172.21.15.7:0/2358364259 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f221001fca0 con 0x556c07f1e010 2024-06-10T01:23:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.688+0000 7f21fcff9640 1 -- 172.21.15.7:0/2358364259 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7f2210004900 con 0x556c07f1e010 2024-06-10T01:23:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.688+0000 7f221e17f740 10 client.19671 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:23:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.688+0000 7f221e17f740 10 client.19671 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:23:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.688+0000 7f221e17f740 20 client.19671 populate_metadata read hostname 'smithi007' 2024-06-10T01:23:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.689+0000 7f221e17f740 1 --2- 172.21.15.7:0/2358364259 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556c07f33f80 0x556c07f36370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.689+0000 7f221e17f740 4 client.19671 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:23:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.689+0000 7f221e17f740 1 -- 172.21.15.7:0/2358364259 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c07e94f10 con 0x556c07f33f80 2024-06-10T01:23:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.689+0000 7f221892b640 1 --2- 172.21.15.7:0/2358364259 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556c07f33f80 0x556c07f36370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.689+0000 7f221892b640 1 --2- 172.21.15.7:0/2358364259 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556c07f33f80 0x556c07f36370 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d0652be202ccc96 server_cookie=dccf8ddb8999c6ef in_seq=0 out_seq=0 2024-06-10T01:23:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.689+0000 7f21fcff9640 10 client.19671 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.690+0000 7f21fcff9640 1 -- 172.21.15.7:0/2358364259 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556c07e94f10 con 0x556c07f33f80 2024-06-10T01:23:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.690+0000 7f21fcff9640 10 client.19671 handle_command_reply: tid=0 2024-06-10T01:23:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.690+0000 7f221e17f740 1 client.19671 shutdown 2024-06-10T01:23:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.690+0000 7f221e17f740 1 -- 172.21.15.7:0/2358364259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c07f1e010 msgr2=0x556c07f28370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.690+0000 7f221e17f740 1 --2- 172.21.15.7:0/2358364259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c07f1e010 0x556c07f28370 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f2210028e10 tx=0x7f2210028e40 comp rx=0 tx=0).stop 2024-06-10T01:23:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.690+0000 7f221e17f740 1 -- 172.21.15.7:0/2358364259 shutdown_connections 2024-06-10T01:23:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.690+0000 7f221e17f740 1 --2- 172.21.15.7:0/2358364259 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556c07f33f80 0x556c07f36370 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.690+0000 7f21fcff9640 0 client.19671 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.690+0000 7f221e17f740 1 --2- 172.21.15.7:0/2358364259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c07f23b70 0x556c07f2f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.690+0000 7f221e17f740 1 --2- 172.21.15.7:0/2358364259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c07f1ea00 0x556c07f288b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.690+0000 7f221e17f740 1 --2- 172.21.15.7:0/2358364259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c07f1e010 0x556c07f28370 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.690+0000 7f221e17f740 1 -- 172.21.15.7:0/2358364259 >> 172.21.15.7:0/2358364259 conn(0x556c07f19680 msgr2=0x556c07f187e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.690+0000 7f221e17f740 1 -- 172.21.15.7:0/2358364259 shutdown_connections 2024-06-10T01:23:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.690+0000 7f221e17f740 1 -- 172.21.15.7:0/2358364259 wait complete. 2024-06-10T01:23:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.690+0000 7f221e17f740 20 client.19671 trim_cache size 0 max 16384 2024-06-10T01:23:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.709+0000 7f221e17f740 1 Processor -- start 2024-06-10T01:23:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.709+0000 7f221e17f740 1 -- start start 2024-06-10T01:23:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.709+0000 7f221e17f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c07f1e010 0x556c07f27040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.710+0000 7f221e17f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c07f1ea00 0x556c07f2f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.710+0000 7f221e17f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c07f23b70 0x556c07f27580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.710+0000 7f221e17f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f22141aabf0 con 0x556c07f1e010 2024-06-10T01:23:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.710+0000 7f221e17f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f22141a84c0 con 0x556c07f23b70 2024-06-10T01:23:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.710+0000 7f221e17f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f22141a6310 con 0x556c07f1ea00 2024-06-10T01:23:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.710+0000 7f221912c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c07f1e010 0x556c07f27040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.710+0000 7f221912c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c07f1e010 0x556c07f27040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38980/0 (socket says 172.21.15.7:38980) 2024-06-10T01:23:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.710+0000 7f221912c640 1 -- 172.21.15.7:0/2713850574 learned_addr learned my addr 172.21.15.7:0/2713850574 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.710+0000 7f221992d640 1 --2- 172.21.15.7:0/2713850574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c07f23b70 0x556c07f27580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.710+0000 7f221892b640 1 --2- 172.21.15.7:0/2713850574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c07f1ea00 0x556c07f2f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.710+0000 7f221912c640 1 -- 172.21.15.7:0/2713850574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c07f1ea00 msgr2=0x556c07f2f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.710+0000 7f221912c640 1 --2- 172.21.15.7:0/2713850574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c07f1ea00 0x556c07f2f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.710+0000 7f221912c640 1 -- 172.21.15.7:0/2713850574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c07f23b70 msgr2=0x556c07f27580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.710+0000 7f221912c640 1 --2- 172.21.15.7:0/2713850574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c07f23b70 0x556c07f27580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.710+0000 7f221912c640 1 -- 172.21.15.7:0/2713850574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c07f1e3f0 con 0x556c07f1e010 2024-06-10T01:23:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.710+0000 7f221992d640 1 --2- 172.21.15.7:0/2713850574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c07f23b70 0x556c07f27580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:23:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.710+0000 7f221892b640 1 --2- 172.21.15.7:0/2713850574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c07f1ea00 0x556c07f2f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:23:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.710+0000 7f221912c640 1 --2- 172.21.15.7:0/2713850574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c07f1e010 0x556c07f27040 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f2210011910 tx=0x7f2210011fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5433f46fed1b5558 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.711+0000 7f21feffd640 1 -- 172.21.15.7:0/2713850574 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2210047070 con 0x556c07f1e010 2024-06-10T01:23:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.711+0000 7f21feffd640 1 -- 172.21.15.7:0/2713850574 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f221001fe50 con 0x556c07f1e010 2024-06-10T01:23:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.711+0000 7f21feffd640 1 -- 172.21.15.7:0/2713850574 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2210027ac0 con 0x556c07f1e010 2024-06-10T01:23:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.712+0000 7f221e17f740 1 -- 172.21.15.7:0/2713850574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c07f1e010 msgr2=0x556c07f27040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.712+0000 7f221e17f740 1 --2- 172.21.15.7:0/2713850574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c07f1e010 0x556c07f27040 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f2210011910 tx=0x7f2210011fd0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.712+0000 7f221e17f740 1 -- 172.21.15.7:0/2713850574 shutdown_connections 2024-06-10T01:23:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.712+0000 7f221e17f740 1 --2- 172.21.15.7:0/2713850574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c07f23b70 0x556c07f27580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.713+0000 7f221e17f740 1 --2- 172.21.15.7:0/2713850574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c07f1ea00 0x556c07f2f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.713+0000 7f221e17f740 1 --2- 172.21.15.7:0/2713850574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c07f1e010 0x556c07f27040 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.713+0000 7f221e17f740 1 -- 172.21.15.7:0/2713850574 >> 172.21.15.7:0/2713850574 conn(0x556c07f19680 msgr2=0x556c07f27f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.713+0000 7f221e17f740 1 -- 172.21.15.7:0/2713850574 shutdown_connections 2024-06-10T01:23:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.713+0000 7f221e17f740 1 -- 172.21.15.7:0/2713850574 wait complete. 2024-06-10T01:23:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.713+0000 7f221e17f740 1 -- start start 2024-06-10T01:23:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.714+0000 7f221e17f740 1 Processor -- start 2024-06-10T01:23:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.714+0000 7f221e17f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c07f1e010 0x556c07f2a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.715+0000 7f221e17f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c07f1ea00 0x556c07f2f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.715+0000 7f221e17f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c07f23b70 0x556c07f2a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.715+0000 7f221e17f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f22141a6980 con 0x556c07f1e010 2024-06-10T01:23:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.715+0000 7f221e17f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f22141a7df0 con 0x556c07f1ea00 2024-06-10T01:23:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.715+0000 7f221912c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c07f1e010 0x556c07f2a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.715+0000 7f221892b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c07f1ea00 0x556c07f2f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.715+0000 7f221892b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c07f1ea00 0x556c07f2f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:55924/0 (socket says 172.21.15.7:55924) 2024-06-10T01:23:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.715+0000 7f221892b640 1 -- 172.21.15.7:0/3818164413 learned_addr learned my addr 172.21.15.7:0/3818164413 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.715+0000 7f221912c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c07f1e010 0x556c07f2a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38996/0 (socket says 172.21.15.7:38996) 2024-06-10T01:23:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.715+0000 7f221e17f740 1 -- 172.21.15.7:0/3818164413 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f22141a7050 con 0x556c07f23b70 2024-06-10T01:23:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.715+0000 7f221992d640 1 --2- 172.21.15.7:0/3818164413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c07f23b70 0x556c07f2a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.715+0000 7f221e17f740 10 client.0 _ll_register_callbacks cb 0x7f220c0b8020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:23:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.715+0000 7f221892b640 1 -- 172.21.15.7:0/3818164413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c07f23b70 msgr2=0x556c07f2a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.715+0000 7f221892b640 1 --2- 172.21.15.7:0/3818164413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c07f23b70 0x556c07f2a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.715+0000 7f221892b640 1 -- 172.21.15.7:0/3818164413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c07f1e010 msgr2=0x556c07f2a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.715+0000 7f221892b640 1 --2- 172.21.15.7:0/3818164413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c07f1e010 0x556c07f2a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.715+0000 7f221892b640 1 -- 172.21.15.7:0/3818164413 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c07f2ac00 con 0x556c07f1ea00 2024-06-10T01:23:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.715+0000 7f221992d640 1 --2- 172.21.15.7:0/3818164413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c07f23b70 0x556c07f2a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:23:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.716+0000 7f221892b640 1 --2- 172.21.15.7:0/3818164413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c07f1ea00 0x556c07f2f5c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2204005ab0 tx=0x7f2204009610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e249f0c33f971b9e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.716+0000 7f21fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:23:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.716+0000 7f21fcff9640 1 -- 172.21.15.7:0/3818164413 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204029060 con 0x556c07f1ea00 2024-06-10T01:23:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.716+0000 7f21fcff9640 1 -- 172.21.15.7:0/3818164413 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f2204004460 con 0x556c07f1ea00 2024-06-10T01:23:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.716+0000 7f221e17f740 1 -- 172.21.15.7:0/3818164413 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f221001fe50 con 0x556c07f1ea00 2024-06-10T01:23:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.716+0000 7f21fcff9640 1 -- 172.21.15.7:0/3818164413 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f2204030050 con 0x556c07f1ea00 2024-06-10T01:23:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.716+0000 7f221e17f740 10 client.28898 fetch_fsmap learned FSMap version 50 2024-06-10T01:23:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.716+0000 7f221e17f740 1 -- 172.21.15.7:0/3818164413 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x556c07fa20d0 con 0x556c07f1ea00 2024-06-10T01:23:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.716+0000 7f21fcff9640 1 -- 172.21.15.7:0/3818164413 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f221001fe50 con 0x556c07f1ea00 2024-06-10T01:23:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.716+0000 7f21fcff9640 1 -- 172.21.15.7:0/3818164413 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7f2204023710 con 0x556c07f1ea00 2024-06-10T01:23:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.717+0000 7f221e17f740 10 client.28898 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:23:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.717+0000 7f221e17f740 10 client.28898 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:23:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.717+0000 7f221e17f740 20 client.28898 populate_metadata read hostname 'smithi007' 2024-06-10T01:23:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.717+0000 7f221e17f740 1 --2- 172.21.15.7:0/3818164413 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556c07fa2a30 0x556c07fa4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.717+0000 7f221e17f740 4 client.28898 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:23:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.717+0000 7f221e17f740 1 -- 172.21.15.7:0/3818164413 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556c07e8ec20 con 0x556c07fa2a30 2024-06-10T01:23:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.717+0000 7f221912c640 1 --2- 172.21.15.7:0/3818164413 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556c07fa2a30 0x556c07fa4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.717+0000 7f21fcff9640 10 client.28898 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.717+0000 7f221912c640 1 --2- 172.21.15.7:0/3818164413 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556c07fa2a30 0x556c07fa4e50 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bdbecdff7cb73c1 server_cookie=a1961e9b0710dcbe in_seq=0 out_seq=0 2024-06-10T01:23:09.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.785+0000 7f21fcff9640 1 -- 172.21.15.7:0/3818164413 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556c07e8ec20 con 0x556c07fa2a30 2024-06-10T01:23:09.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.786+0000 7f21fcff9640 10 client.28898 handle_command_reply: tid=0 2024-06-10T01:23:09.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.786+0000 7f221e17f740 1 client.28898 shutdown 2024-06-10T01:23:09.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.786+0000 7f221e17f740 1 -- 172.21.15.7:0/3818164413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c07f1ea00 msgr2=0x556c07f2f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:09.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.786+0000 7f221e17f740 1 --2- 172.21.15.7:0/3818164413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c07f1ea00 0x556c07f2f5c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2204005ab0 tx=0x7f2204009610 comp rx=0 tx=0).stop 2024-06-10T01:23:09.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.786+0000 7f221e17f740 1 -- 172.21.15.7:0/3818164413 shutdown_connections 2024-06-10T01:23:09.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.786+0000 7f221e17f740 1 --2- 172.21.15.7:0/3818164413 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556c07fa2a30 0x556c07fa4e50 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.786+0000 7f221e17f740 1 --2- 172.21.15.7:0/3818164413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c07f23b70 0x556c07f2a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.786+0000 7f221e17f740 1 --2- 172.21.15.7:0/3818164413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c07f1ea00 0x556c07f2f5c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.786+0000 7f221e17f740 1 --2- 172.21.15.7:0/3818164413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c07f1e010 0x556c07f2a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.786+0000 7f221e17f740 1 -- 172.21.15.7:0/3818164413 >> 172.21.15.7:0/3818164413 conn(0x556c07f19680 msgr2=0x556c07f17f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:09.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.786+0000 7f21fcff9640 0 client.28898 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:09.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.786+0000 7f221e17f740 1 -- 172.21.15.7:0/3818164413 shutdown_connections 2024-06-10T01:23:09.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.786+0000 7f221e17f740 1 -- 172.21.15.7:0/3818164413 wait complete. 2024-06-10T01:23:09.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.786+0000 7f221e17f740 20 client.28898 trim_cache size 0 max 16384 2024-06-10T01:23:09.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:23:09.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:23:09.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:23:09.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.787+0000 7f221b390640 1 -- 172.21.15.7:0/3842163311 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f22000a2c00 msgr2=0x7f22000a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:09.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.787+0000 7f221b390640 1 --2- 172.21.15.7:0/3842163311 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f22000a2c00 0x7f22000a50c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2214101950 tx=0x7f2210005f90 comp rx=0 tx=0).stop 2024-06-10T01:23:09.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.787+0000 7f221b390640 1 -- 172.21.15.7:0/3842163311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22141024b0 msgr2=0x7f221419d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:09.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.787+0000 7f221b390640 1 --2- 172.21.15.7:0/3842163311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22141024b0 0x7f221419d040 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f220c034040 tx=0x7f220c0305d0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.787+0000 7f221b390640 1 -- 172.21.15.7:0/3842163311 shutdown_connections 2024-06-10T01:23:09.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.787+0000 7f221b390640 1 --2- 172.21.15.7:0/3842163311 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f22000a2c00 0x7f22000a50c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.787+0000 7f221b390640 1 --2- 172.21.15.7:0/3842163311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22141024b0 0x7f221419d040 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.787+0000 7f221b390640 1 --2- 172.21.15.7:0/3842163311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2214101af0 0x7f2214195fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.787+0000 7f221b390640 1 --2- 172.21.15.7:0/3842163311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2214100900 0x7f2214195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.787+0000 7f221b390640 1 -- 172.21.15.7:0/3842163311 >> 172.21.15.7:0/3842163311 conn(0x7f22140fbe70 msgr2=0x7f2214104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.787+0000 7f221b390640 1 -- 172.21.15.7:0/3842163311 shutdown_connections 2024-06-10T01:23:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:09.787+0000 7f221b390640 1 -- 172.21.15.7:0/3842163311 wait complete. 2024-06-10T01:23:09.804 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:23:10.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.476+0000 7f53ef0f6700 1 -- 172.21.15.7:0/1608453957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53e8106b90 msgr2=0x7f53e8106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:10.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.476+0000 7f53ef0f6700 1 --2- 172.21.15.7:0/1608453957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53e8106b90 0x7f53e8106f70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f53e400b0a0 tx=0x7f53e400b3b0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.477+0000 7f53ef0f6700 1 -- 172.21.15.7:0/1608453957 shutdown_connections 2024-06-10T01:23:10.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.477+0000 7f53ef0f6700 1 --2- 172.21.15.7:0/1608453957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53e8101510 0x7f53e810d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.477+0000 7f53ef0f6700 1 --2- 172.21.15.7:0/1608453957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53e8106b90 0x7f53e8106f70 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.477+0000 7f53ef0f6700 1 --2- 172.21.15.7:0/1608453957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53e8100b50 0x7f53e8100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.477+0000 7f53ef0f6700 1 -- 172.21.15.7:0/1608453957 >> 172.21.15.7:0/1608453957 conn(0x7f53e8077a00 msgr2=0x7f53e80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:10.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.478+0000 7f53ef0f6700 1 -- 172.21.15.7:0/1608453957 shutdown_connections 2024-06-10T01:23:10.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.478+0000 7f53ef0f6700 1 -- 172.21.15.7:0/1608453957 wait complete. 2024-06-10T01:23:10.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.479+0000 7f53ef0f6700 1 Processor -- start 2024-06-10T01:23:10.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.479+0000 7f53ef0f6700 1 -- start start 2024-06-10T01:23:10.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.479+0000 7f53ef0f6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53e8100b50 0x7f53e810cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:10.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.479+0000 7f53ef0f6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53e8101510 0x7f53e810d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:10.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.479+0000 7f53ef0f6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53e8106b90 0x7f53e81a3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:10.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.479+0000 7f53ef0f6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f53e8112880 con 0x7f53e8101510 2024-06-10T01:23:10.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.479+0000 7f53ef0f6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f53e8112700 con 0x7f53e8100b50 2024-06-10T01:23:10.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.479+0000 7f53ef0f6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f53e8112a00 con 0x7f53e8106b90 2024-06-10T01:23:10.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.480+0000 7f53ece92700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53e8100b50 0x7f53e810cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:10.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.480+0000 7f53ece92700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53e8100b50 0x7f53e810cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:55956/0 (socket says 172.21.15.7:55956) 2024-06-10T01:23:10.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.480+0000 7f53ece92700 1 -- 172.21.15.7:0/1415741048 learned_addr learned my addr 172.21.15.7:0/1415741048 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:10.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.480+0000 7f53dffff700 1 --2- 172.21.15.7:0/1415741048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53e8101510 0x7f53e810d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:10.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.482+0000 7f53ed693700 1 --2- 172.21.15.7:0/1415741048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53e8106b90 0x7f53e81a3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:10.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.483+0000 7f53ece92700 1 -- 172.21.15.7:0/1415741048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53e8106b90 msgr2=0x7f53e81a3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:10.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.483+0000 7f53ece92700 1 --2- 172.21.15.7:0/1415741048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53e8106b90 0x7f53e81a3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.483+0000 7f53ece92700 1 -- 172.21.15.7:0/1415741048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53e8101510 msgr2=0x7f53e810d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:10.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.483+0000 7f53ece92700 1 --2- 172.21.15.7:0/1415741048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53e8101510 0x7f53e810d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.483+0000 7f53ece92700 1 -- 172.21.15.7:0/1415741048 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e81a41b0 con 0x7f53e8100b50 2024-06-10T01:23:10.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.483+0000 7f53dffff700 1 --2- 172.21.15.7:0/1415741048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53e8101510 0x7f53e810d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:23:10.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.484+0000 7f53ece92700 1 --2- 172.21.15.7:0/1415741048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53e8100b50 0x7f53e810cc50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f53e4003d00 tx=0x7f53e4003d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:10.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.484+0000 7f53ddffb700 1 -- 172.21.15.7:0/1415741048 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e401f070 con 0x7f53e8100b50 2024-06-10T01:23:10.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.484+0000 7f53ddffb700 1 -- 172.21.15.7:0/1415741048 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53e4012d10 con 0x7f53e8100b50 2024-06-10T01:23:10.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.484+0000 7f53ddffb700 1 -- 172.21.15.7:0/1415741048 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e40057c0 con 0x7f53e8100b50 2024-06-10T01:23:10.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.485+0000 7f53ef0f6700 1 -- 172.21.15.7:0/1415741048 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e81a4440 con 0x7f53e8100b50 2024-06-10T01:23:10.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.485+0000 7f53ef0f6700 1 -- 172.21.15.7:0/1415741048 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e81abd20 con 0x7f53e8100b50 2024-06-10T01:23:10.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.493+0000 7f53f9979b80 1 Processor -- start 2024-06-10T01:23:10.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.493+0000 7f53f9979b80 1 -- start start 2024-06-10T01:23:10.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.494+0000 7f53f9979b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a567f596f0 0x55a567f59ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:10.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.494+0000 7f53f9979b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a567f5a010 0x55a567f53a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:10.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.494+0000 7f53f9979b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a567f53fb0 0x55a567f543d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:10.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.494+0000 7f53f9979b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a567e94890 con 0x55a567f5a010 2024-06-10T01:23:10.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.494+0000 7f53f9979b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a567e3c7a0 con 0x55a567f53fb0 2024-06-10T01:23:10.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.494+0000 7f53f9979b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a567e90850 con 0x55a567f596f0 2024-06-10T01:23:10.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.495+0000 7f53ddffb700 1 -- 172.21.15.7:0/1415741048 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f53e4022070 con 0x7f53e8100b50 2024-06-10T01:23:10.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.496+0000 7f53dffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a567f53fb0 0x55a567f543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:10.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.496+0000 7f53ece92700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a567f5a010 0x55a567f53a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:10.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.496+0000 7f53ed693700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a567f596f0 0x55a567f59ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:10.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.496+0000 7f53ed693700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a567f596f0 0x55a567f59ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:40854/0 (socket says 172.21.15.7:40854) 2024-06-10T01:23:10.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.496+0000 7f53ed693700 1 -- 172.21.15.7:0/1560891746 learned_addr learned my addr 172.21.15.7:0/1560891746 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:10.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.497+0000 7f53ed693700 1 -- 172.21.15.7:0/1560891746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a567f53fb0 msgr2=0x55a567f543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:10.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.497+0000 7f53ddffb700 1 --2- 172.21.15.7:0/1415741048 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f53d00a2db0 0x7f53d00a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:10.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.497+0000 7f53ed693700 1 --2- 172.21.15.7:0/1560891746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a567f53fb0 0x55a567f543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.497+0000 7f53ed693700 1 -- 172.21.15.7:0/1560891746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a567f5a010 msgr2=0x55a567f53a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:10.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.497+0000 7f53ed693700 1 --2- 172.21.15.7:0/1560891746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a567f5a010 0x55a567f53a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.497+0000 7f53ed693700 1 -- 172.21.15.7:0/1560891746 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a567e32fd0 con 0x55a567f596f0 2024-06-10T01:23:10.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.497+0000 7f53ece92700 1 --2- 172.21.15.7:0/1560891746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a567f5a010 0x55a567f53a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:23:10.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.498+0000 7f53ddffb700 1 -- 172.21.15.7:0/1415741048 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f53e400f020 con 0x7f53e8100b50 2024-06-10T01:23:10.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.498+0000 7f53ed693700 1 --2- 172.21.15.7:0/1560891746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a567f596f0 0x55a567f59ad0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f53e00046d0 tx=0x7f53e000b970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39ab39aa4a01a93d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:10.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.498+0000 7f53ed693700 1 --2- 172.21.15.7:0/1415741048 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f53d00a2db0 0x7f53d00a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:10.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.498+0000 7f53ceffd700 1 -- 172.21.15.7:0/1560891746 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0003b60 con 0x55a567f596f0 2024-06-10T01:23:10.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.498+0000 7f53ceffd700 1 -- 172.21.15.7:0/1560891746 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53e0015070 con 0x55a567f596f0 2024-06-10T01:23:10.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.498+0000 7f53ceffd700 1 -- 172.21.15.7:0/1560891746 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e001c800 con 0x55a567f596f0 2024-06-10T01:23:10.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.500+0000 7f53f9979b80 1 -- 172.21.15.7:0/1560891746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a567f596f0 msgr2=0x55a567f59ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:10.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.500+0000 7f53f9979b80 1 --2- 172.21.15.7:0/1560891746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a567f596f0 0x55a567f59ad0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f53e00046d0 tx=0x7f53e000b970 comp rx=0 tx=0).stop 2024-06-10T01:23:10.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.500+0000 7f53ed693700 1 --2- 172.21.15.7:0/1415741048 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f53d00a2db0 0x7f53d00a5270 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f53e001ca30 tx=0x7f53e001c9a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:10.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.501+0000 7f53f9979b80 1 -- 172.21.15.7:0/1560891746 shutdown_connections 2024-06-10T01:23:10.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.501+0000 7f53f9979b80 1 --2- 172.21.15.7:0/1560891746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a567f53fb0 0x55a567f543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.501+0000 7f53f9979b80 1 --2- 172.21.15.7:0/1560891746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a567f596f0 0x55a567f59ad0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.501+0000 7f53f9979b80 1 --2- 172.21.15.7:0/1560891746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a567f5a010 0x55a567f53a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.501+0000 7f53f9979b80 1 -- 172.21.15.7:0/1560891746 >> 172.21.15.7:0/1560891746 conn(0x55a567e6dd40 msgr2=0x55a567f3d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:10.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.503+0000 7f53f9979b80 1 -- 172.21.15.7:0/1560891746 shutdown_connections 2024-06-10T01:23:10.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.503+0000 7f53f9979b80 1 -- 172.21.15.7:0/1560891746 wait complete. 2024-06-10T01:23:10.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.504+0000 7f53f9979b80 1 -- start start 2024-06-10T01:23:10.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.504+0000 7f53f9979b80 1 Processor -- start 2024-06-10T01:23:10.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.504+0000 7f53f9979b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a567f53fb0 0x55a567f5fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:10.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.504+0000 7f53f9979b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a567f596f0 0x55a567f5d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:10.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.504+0000 7f53f9979b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a567f5a010 0x55a567f5d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:10.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.504+0000 7f53f9979b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a567f60a50 con 0x55a567f596f0 2024-06-10T01:23:10.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.504+0000 7f53f9979b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a567e34c70 con 0x55a567f53fb0 2024-06-10T01:23:10.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.505+0000 7f53f9979b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a567f60bd0 con 0x55a567f5a010 2024-06-10T01:23:10.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.505+0000 7f53f9979b80 10 client.0 _ll_register_callbacks cb 0x55a567e2e9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:23:10.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.505+0000 7f53dffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a567f596f0 0x55a567f5d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:10.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.505+0000 7f53dffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a567f596f0 0x55a567f5d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39016/0 (socket says 172.21.15.7:39016) 2024-06-10T01:23:10.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.505+0000 7f53dffff700 1 -- 172.21.15.7:0/2441017442 learned_addr learned my addr 172.21.15.7:0/2441017442 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:10.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.505+0000 7f53dffff700 1 -- 172.21.15.7:0/2441017442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a567f5a010 msgr2=0x55a567f5d590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:23:10.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.505+0000 7f53dffff700 1 --2- 172.21.15.7:0/2441017442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a567f5a010 0x55a567f5d590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.505+0000 7f53dffff700 1 -- 172.21.15.7:0/2441017442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a567f53fb0 msgr2=0x55a567f5fd50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:23:10.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.505+0000 7f53dffff700 1 --2- 172.21.15.7:0/2441017442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a567f53fb0 0x55a567f5fd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.505+0000 7f53dffff700 1 -- 172.21.15.7:0/2441017442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a567de3950 con 0x55a567f596f0 2024-06-10T01:23:10.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.506+0000 7f53dffff700 1 --2- 172.21.15.7:0/2441017442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a567f596f0 0x55a567f5d050 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f53d800bfa0 tx=0x7f53d8012bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4644aa3cf2f7bb4f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:10.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.507+0000 7f53c7fff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:23:10.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.507+0000 7f53c7fff700 1 -- 172.21.15.7:0/2441017442 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d803a050 con 0x55a567f596f0 2024-06-10T01:23:10.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.507+0000 7f53f9979b80 1 -- 172.21.15.7:0/2441017442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53e0015070 con 0x55a567f596f0 2024-06-10T01:23:10.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.507+0000 7f53c7fff700 1 -- 172.21.15.7:0/2441017442 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f53d8010ae0 con 0x55a567f596f0 2024-06-10T01:23:10.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.507+0000 7f53c7fff700 1 -- 172.21.15.7:0/2441017442 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f53d8049050 con 0x55a567f596f0 2024-06-10T01:23:10.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.507+0000 7f53f9979b80 10 client.19686 fetch_fsmap learned FSMap version 50 2024-06-10T01:23:10.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.507+0000 7f53f9979b80 1 -- 172.21.15.7:0/2441017442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55a567f6a020 con 0x55a567f596f0 2024-06-10T01:23:10.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.508+0000 7f53c7fff700 1 -- 172.21.15.7:0/2441017442 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53e0015070 con 0x55a567f596f0 2024-06-10T01:23:10.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.508+0000 7f53c7fff700 1 -- 172.21.15.7:0/2441017442 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7f53d803d390 con 0x55a567f596f0 2024-06-10T01:23:10.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.508+0000 7f53f9979b80 10 client.19686 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:23:10.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.508+0000 7f53f9979b80 10 client.19686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:23:10.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.508+0000 7f53f9979b80 20 client.19686 populate_metadata read hostname 'smithi007' 2024-06-10T01:23:10.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.508+0000 7f53f9979b80 1 --2- 172.21.15.7:0/2441017442 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a567f6cb90 0x55a567f6f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:10.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.508+0000 7f53f9979b80 4 client.19686 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:23:10.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.508+0000 7f53f9979b80 1 -- 172.21.15.7:0/2441017442 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a567ea1d90 con 0x55a567f6cb90 2024-06-10T01:23:10.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.511+0000 7f53ece92700 1 --2- 172.21.15.7:0/2441017442 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a567f6cb90 0x55a567f6f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:10.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.513+0000 7f53ece92700 1 --2- 172.21.15.7:0/2441017442 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a567f6cb90 0x55a567f6f050 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff51cfd82b05b61d server_cookie=ca4b922f25abbbd in_seq=0 out_seq=0 2024-06-10T01:23:10.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.514+0000 7f53c7fff700 10 client.19686 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:10.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.515+0000 7f53c7fff700 1 -- 172.21.15.7:0/2441017442 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a567ea1d90 con 0x55a567f6cb90 2024-06-10T01:23:10.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.515+0000 7f53c7fff700 10 client.19686 handle_command_reply: tid=0 2024-06-10T01:23:10.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.515+0000 7f53f9979b80 1 client.19686 shutdown 2024-06-10T01:23:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.517+0000 7f53f9979b80 1 -- 172.21.15.7:0/2441017442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a567f596f0 msgr2=0x55a567f5d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.517+0000 7f53f9979b80 1 --2- 172.21.15.7:0/2441017442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a567f596f0 0x55a567f5d050 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f53d800bfa0 tx=0x7f53d8012bd0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.517+0000 7f53f9979b80 1 -- 172.21.15.7:0/2441017442 shutdown_connections 2024-06-10T01:23:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.517+0000 7f53f9979b80 1 --2- 172.21.15.7:0/2441017442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a567f5a010 0x55a567f5d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.517+0000 7f53f9979b80 1 --2- 172.21.15.7:0/2441017442 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a567f6cb90 0x55a567f6f050 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.517+0000 7f53f9979b80 1 --2- 172.21.15.7:0/2441017442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a567f53fb0 0x55a567f5fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.517+0000 7f53f9979b80 1 --2- 172.21.15.7:0/2441017442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a567f596f0 0x55a567f5d050 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.517+0000 7f53f9979b80 1 -- 172.21.15.7:0/2441017442 >> 172.21.15.7:0/2441017442 conn(0x55a567e6dd40 msgr2=0x55a567f3d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:10.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.518+0000 7f53c7fff700 0 client.19686 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:10.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.520+0000 7f53f9979b80 1 -- 172.21.15.7:0/2441017442 shutdown_connections 2024-06-10T01:23:10.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.520+0000 7f53f9979b80 1 -- 172.21.15.7:0/2441017442 wait complete. 2024-06-10T01:23:10.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.520+0000 7f53f9979b80 20 client.19686 trim_cache size 0 max 16384 2024-06-10T01:23:10.531 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 26.13 MB/sec execute 107 sec latency 23129.092 ms 2024-06-10T01:23:10.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.565+0000 7f53f9979b80 1 Processor -- start 2024-06-10T01:23:10.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.565+0000 7f53f9979b80 1 -- start start 2024-06-10T01:23:10.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.566+0000 7f53f9979b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a567f53fb0 0x55a567f3da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:10.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.566+0000 7f53f9979b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a567f596f0 0x55a567f5c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:10.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.566+0000 7f53ece92700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a567f53fb0 0x55a567f3da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:10.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.566+0000 7f53ece92700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a567f53fb0 0x55a567f3da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:55968/0 (socket says 172.21.15.7:55968) 2024-06-10T01:23:10.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.566+0000 7f53ece92700 1 -- 172.21.15.7:0/3978549729 learned_addr learned my addr 172.21.15.7:0/3978549729 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:10.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.566+0000 7f53f9979b80 1 --2- 172.21.15.7:0/3978549729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a567f5a010 0x55a567f37590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:10.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.567+0000 7f53f9979b80 1 -- 172.21.15.7:0/3978549729 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f53d4011440 con 0x55a567f596f0 2024-06-10T01:23:10.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.567+0000 7f53f9979b80 1 -- 172.21.15.7:0/3978549729 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f53d400ecd0 con 0x55a567f53fb0 2024-06-10T01:23:10.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.567+0000 7f53ed693700 1 --2- 172.21.15.7:0/3978549729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a567f5a010 0x55a567f37590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:10.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.567+0000 7f53f9979b80 1 -- 172.21.15.7:0/3978549729 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f53d400cb20 con 0x55a567f5a010 2024-06-10T01:23:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.567+0000 7f53dffff700 1 --2- 172.21.15.7:0/3978549729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a567f596f0 0x55a567f5c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.568+0000 7f53ece92700 1 -- 172.21.15.7:0/3978549729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a567f5a010 msgr2=0x55a567f37590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:10.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.568+0000 7f53ece92700 1 --2- 172.21.15.7:0/3978549729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a567f5a010 0x55a567f37590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.568+0000 7f53ece92700 1 -- 172.21.15.7:0/3978549729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a567f596f0 msgr2=0x55a567f5c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:10.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.568+0000 7f53ece92700 1 --2- 172.21.15.7:0/3978549729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a567f596f0 0x55a567f5c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.568+0000 7f53ece92700 1 -- 172.21.15.7:0/3978549729 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a567f3e3a0 con 0x55a567f53fb0 2024-06-10T01:23:10.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.568+0000 7f53ece92700 1 --2- 172.21.15.7:0/3978549729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a567f53fb0 0x55a567f3da70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f53e4098e00 tx=0x7f53e4005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c209031eb336d9d6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:10.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.568+0000 7f53ce7fc700 1 -- 172.21.15.7:0/3978549729 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e409c300 con 0x55a567f53fb0 2024-06-10T01:23:10.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.568+0000 7f53ce7fc700 1 -- 172.21.15.7:0/3978549729 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53e409d0a0 con 0x55a567f53fb0 2024-06-10T01:23:10.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.569+0000 7f53ce7fc700 1 -- 172.21.15.7:0/3978549729 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e40050b0 con 0x55a567f53fb0 2024-06-10T01:23:10.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.569+0000 7f53f9979b80 1 -- 172.21.15.7:0/3978549729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a567f53fb0 msgr2=0x55a567f3da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:10.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.570+0000 7f53f9979b80 1 --2- 172.21.15.7:0/3978549729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a567f53fb0 0x55a567f3da70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f53e4098e00 tx=0x7f53e4005040 comp rx=0 tx=0).stop 2024-06-10T01:23:10.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.570+0000 7f53f9979b80 1 -- 172.21.15.7:0/3978549729 shutdown_connections 2024-06-10T01:23:10.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.570+0000 7f53f9979b80 1 --2- 172.21.15.7:0/3978549729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a567f5a010 0x55a567f37590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.570+0000 7f53f9979b80 1 --2- 172.21.15.7:0/3978549729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a567f53fb0 0x55a567f3da70 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.570+0000 7f53f9979b80 1 --2- 172.21.15.7:0/3978549729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a567f596f0 0x55a567f5c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.570+0000 7f53f9979b80 1 -- 172.21.15.7:0/3978549729 >> 172.21.15.7:0/3978549729 conn(0x55a567e6dd40 msgr2=0x55a567f36e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:10.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.570+0000 7f53f9979b80 1 -- 172.21.15.7:0/3978549729 shutdown_connections 2024-06-10T01:23:10.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.570+0000 7f53f9979b80 1 -- 172.21.15.7:0/3978549729 wait complete. 2024-06-10T01:23:10.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.570+0000 7f53f9979b80 1 -- start start 2024-06-10T01:23:10.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.570+0000 7f53f9979b80 1 Processor -- start 2024-06-10T01:23:10.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.571+0000 7f53f9979b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a567f53fb0 0x55a567f666c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:10.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.571+0000 7f53f9979b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a567f596f0 0x55a567f62ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:10.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.571+0000 7f53f9979b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a567f5a010 0x55a567f63440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:10.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.571+0000 7f53f9979b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f53d400d190 con 0x55a567f5a010 2024-06-10T01:23:10.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.571+0000 7f53f9979b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f53d400e600 con 0x55a567f53fb0 2024-06-10T01:23:10.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.571+0000 7f53f9979b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f53d400d860 con 0x55a567f596f0 2024-06-10T01:23:10.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.571+0000 7f53f9979b80 10 client.0 _ll_register_callbacks cb 0x7f53d800be60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:23:10.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.571+0000 7f53ed693700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a567f5a010 0x55a567f63440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:10.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.572+0000 7f53ed693700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a567f5a010 0x55a567f63440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39020/0 (socket says 172.21.15.7:39020) 2024-06-10T01:23:10.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.572+0000 7f53ed693700 1 -- 172.21.15.7:0/1204244143 learned_addr learned my addr 172.21.15.7:0/1204244143 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:10.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.572+0000 7f53dffff700 1 --2- 172.21.15.7:0/1204244143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a567f596f0 0x55a567f62ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:10.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.572+0000 7f53ece92700 1 --2- 172.21.15.7:0/1204244143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a567f53fb0 0x55a567f666c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:10.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.572+0000 7f53ed693700 1 -- 172.21.15.7:0/1204244143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a567f596f0 msgr2=0x55a567f62ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:10.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.572+0000 7f53ed693700 1 --2- 172.21.15.7:0/1204244143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a567f596f0 0x55a567f62ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.572+0000 7f53ed693700 1 -- 172.21.15.7:0/1204244143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a567f53fb0 msgr2=0x55a567f666c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:10.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.572+0000 7f53ed693700 1 --2- 172.21.15.7:0/1204244143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a567f53fb0 0x55a567f666c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.572+0000 7f53ed693700 1 -- 172.21.15.7:0/1204244143 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a567f63c80 con 0x55a567f5a010 2024-06-10T01:23:10.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.572+0000 7f53dffff700 1 --2- 172.21.15.7:0/1204244143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a567f596f0 0x55a567f62ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:23:10.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.573+0000 7f53ed693700 1 --2- 172.21.15.7:0/1204244143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a567f5a010 0x55a567f63440 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f53e0011fd0 tx=0x7f53e0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=495daf7a25e1791e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:10.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.573+0000 7f53c7fff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:23:10.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.574+0000 7f53c7fff700 1 -- 172.21.15.7:0/1204244143 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0036060 con 0x55a567f5a010 2024-06-10T01:23:10.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.574+0000 7f53c7fff700 1 -- 172.21.15.7:0/1204244143 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f53e001c690 con 0x55a567f5a010 2024-06-10T01:23:10.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.574+0000 7f53f9979b80 1 -- 172.21.15.7:0/1204244143 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53e409d0a0 con 0x55a567f5a010 2024-06-10T01:23:10.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.574+0000 7f53c7fff700 1 -- 172.21.15.7:0/1204244143 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f53e0005050 con 0x55a567f5a010 2024-06-10T01:23:10.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.574+0000 7f53f9979b80 10 client.19692 fetch_fsmap learned FSMap version 50 2024-06-10T01:23:10.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.574+0000 7f53f9979b80 1 -- 172.21.15.7:0/1204244143 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55a567fd9580 con 0x55a567f5a010 2024-06-10T01:23:10.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.575+0000 7f53c7fff700 1 -- 172.21.15.7:0/1204244143 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53e409d0a0 con 0x55a567f5a010 2024-06-10T01:23:10.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.575+0000 7f53c7fff700 1 -- 172.21.15.7:0/1204244143 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7f53e0030030 con 0x55a567f5a010 2024-06-10T01:23:10.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.575+0000 7f53f9979b80 10 client.19692 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:23:10.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.575+0000 7f53f9979b80 10 client.19692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:23:10.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.575+0000 7f53f9979b80 20 client.19692 populate_metadata read hostname 'smithi007' 2024-06-10T01:23:10.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.575+0000 7f53f9979b80 1 --2- 172.21.15.7:0/1204244143 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a567fd9a50 0x55a567fdbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:10.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.575+0000 7f53f9979b80 4 client.19692 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:23:10.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.575+0000 7f53f9979b80 1 -- 172.21.15.7:0/1204244143 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a567e9c310 con 0x55a567fd9a50 2024-06-10T01:23:10.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.576+0000 7f53ece92700 1 --2- 172.21.15.7:0/1204244143 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a567fd9a50 0x55a567fdbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:10.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.576+0000 7f53c7fff700 10 client.19692 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:10.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.576+0000 7f53ece92700 1 --2- 172.21.15.7:0/1204244143 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a567fd9a50 0x55a567fdbe70 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3080b77be35320e server_cookie=1086501081cbe8df in_seq=0 out_seq=0 2024-06-10T01:23:10.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.577+0000 7f53c7fff700 1 -- 172.21.15.7:0/1204244143 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a567e9c310 con 0x55a567fd9a50 2024-06-10T01:23:10.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.577+0000 7f53c7fff700 10 client.19692 handle_command_reply: tid=0 2024-06-10T01:23:10.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.577+0000 7f53f9979b80 1 client.19692 shutdown 2024-06-10T01:23:10.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.578+0000 7f53f9979b80 1 -- 172.21.15.7:0/1204244143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a567f5a010 msgr2=0x55a567f63440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:10.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.578+0000 7f53f9979b80 1 --2- 172.21.15.7:0/1204244143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a567f5a010 0x55a567f63440 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f53e0011fd0 tx=0x7f53e0017000 comp rx=0 tx=0).stop 2024-06-10T01:23:10.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.578+0000 7f53f9979b80 1 -- 172.21.15.7:0/1204244143 shutdown_connections 2024-06-10T01:23:10.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.578+0000 7f53f9979b80 1 --2- 172.21.15.7:0/1204244143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a567f5a010 0x55a567f63440 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.578+0000 7f53f9979b80 1 --2- 172.21.15.7:0/1204244143 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a567fd9a50 0x55a567fdbe70 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.578+0000 7f53f9979b80 1 --2- 172.21.15.7:0/1204244143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a567f53fb0 0x55a567f666c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.578+0000 7f53f9979b80 1 --2- 172.21.15.7:0/1204244143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a567f596f0 0x55a567f62ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.578+0000 7f53f9979b80 1 -- 172.21.15.7:0/1204244143 >> 172.21.15.7:0/1204244143 conn(0x55a567e6dd40 msgr2=0x55a567f41a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:10.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.578+0000 7f53c7fff700 0 client.19692 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:10.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.580+0000 7f53f9979b80 1 -- 172.21.15.7:0/1204244143 shutdown_connections 2024-06-10T01:23:10.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.580+0000 7f53f9979b80 1 -- 172.21.15.7:0/1204244143 wait complete. 2024-06-10T01:23:10.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.581+0000 7f53f9979b80 20 client.19692 trim_cache size 0 max 16384 2024-06-10T01:23:10.578 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:23:10.578 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-10T01:23:10.578 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "86ddbadf-cafa-40e1-a1d0-90cf09b01c1b", 2024-06-10T01:23:10.579 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-10T01:23:10.579 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:23:10.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.582+0000 7f53ef0f6700 1 -- 172.21.15.7:0/1415741048 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f53d00a2db0 msgr2=0x7f53d00a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:10.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.582+0000 7f53ef0f6700 1 --2- 172.21.15.7:0/1415741048 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f53d00a2db0 0x7f53d00a5270 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f53e001ca30 tx=0x7f53e001c9a0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.582+0000 7f53ef0f6700 1 -- 172.21.15.7:0/1415741048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53e8100b50 msgr2=0x7f53e810cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:10.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.582+0000 7f53ef0f6700 1 --2- 172.21.15.7:0/1415741048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53e8100b50 0x7f53e810cc50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f53e4003d00 tx=0x7f53e4003d30 comp rx=0 tx=0).stop 2024-06-10T01:23:10.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.582+0000 7f53ef0f6700 1 -- 172.21.15.7:0/1415741048 shutdown_connections 2024-06-10T01:23:10.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.582+0000 7f53ef0f6700 1 --2- 172.21.15.7:0/1415741048 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f53d00a2db0 0x7f53d00a5270 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.582+0000 7f53ef0f6700 1 --2- 172.21.15.7:0/1415741048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53e8106b90 0x7f53e81a3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.582+0000 7f53ef0f6700 1 --2- 172.21.15.7:0/1415741048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53e8100b50 0x7f53e810cc50 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.582+0000 7f53ef0f6700 1 --2- 172.21.15.7:0/1415741048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53e8101510 0x7f53e810d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:10.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.582+0000 7f53ef0f6700 1 -- 172.21.15.7:0/1415741048 >> 172.21.15.7:0/1415741048 conn(0x7f53e8077a00 msgr2=0x7f53e8103710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:10.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.582+0000 7f53ef0f6700 1 -- 172.21.15.7:0/1415741048 shutdown_connections 2024-06-10T01:23:10.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:10.582+0000 7f53ef0f6700 1 -- 172.21.15.7:0/1415741048 wait complete. 2024-06-10T01:23:11.166 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:11 smithi007 ceph-mon[32398]: pgmap v948: 129 pgs: 129 active+clean; 576 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 6.0 MiB/s wr, 308 op/s 2024-06-10T01:23:11.166 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:11 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:23:11.166 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:11 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:23:11.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:11 smithi078 ceph-mon[36683]: pgmap v948: 129 pgs: 129 active+clean; 576 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 6.0 MiB/s wr, 308 op/s 2024-06-10T01:23:11.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:11 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:23:11.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:11 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:23:11.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:11 smithi175 ceph-mon[36636]: pgmap v948: 129 pgs: 129 active+clean; 576 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 6.0 MiB/s wr, 308 op/s 2024-06-10T01:23:11.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:11 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:23:11.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:11 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:23:11.531 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 25.89 MB/sec execute 108 sec latency 24129.207 ms 2024-06-10T01:23:12.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:12 smithi078 ceph-mon[36683]: scrub queued for path: / 2024-06-10T01:23:12.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:12 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:23:12.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:12 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:23:12.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:12 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:23:12.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:12 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:23:12.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:12 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:23:12.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:12 smithi007 ceph-mon[32398]: scrub queued for path: / 2024-06-10T01:23:12.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:12 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:23:12.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:12 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:23:12.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:12 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:23:12.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:12 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:23:12.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:12 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:23:12.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:12 smithi175 ceph-mon[36636]: scrub queued for path: / 2024-06-10T01:23:12.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:12 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:23:12.452 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:12 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:23:12.452 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:12 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:23:12.452 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:12 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:23:12.452 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:12 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:23:12.531 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 25.65 MB/sec execute 109 sec latency 25129.272 ms 2024-06-10T01:23:12.790 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:23:13.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:13 smithi078 ceph-mon[36683]: pgmap v949: 129 pgs: 129 active+clean; 570 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 4.8 MiB/s wr, 246 op/s 2024-06-10T01:23:13.370 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:13 smithi007 ceph-mon[32398]: pgmap v949: 129 pgs: 129 active+clean; 570 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 4.8 MiB/s wr, 246 op/s 2024-06-10T01:23:13.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:13 smithi175 ceph-mon[36636]: pgmap v949: 129 pgs: 129 active+clean; 570 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 4.8 MiB/s wr, 246 op/s 2024-06-10T01:23:13.531 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 25.42 MB/sec execute 110 sec latency 26129.320 ms 2024-06-10T01:23:13.713 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:23:14.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.346+0000 7fb64cb4d700 1 -- 172.21.15.7:0/349779787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6481013d0 msgr2=0x7fb648101830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:14.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.346+0000 7fb64cb4d700 1 --2- 172.21.15.7:0/349779787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6481013d0 0x7fb648101830 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb638009990 tx=0x7fb638009ca0 comp rx=0 tx=0).stop 2024-06-10T01:23:14.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.354+0000 7fb64cb4d700 1 -- 172.21.15.7:0/349779787 shutdown_connections 2024-06-10T01:23:14.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.354+0000 7fb64cb4d700 1 --2- 172.21.15.7:0/349779787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb648101d70 0x7fb64810e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:14.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.354+0000 7fb64cb4d700 1 --2- 172.21.15.7:0/349779787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb648107410 0x7fb6481077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:14.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.354+0000 7fb64cb4d700 1 --2- 172.21.15.7:0/349779787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6481013d0 0x7fb648101830 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:14.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.354+0000 7fb64cb4d700 1 -- 172.21.15.7:0/349779787 >> 172.21.15.7:0/349779787 conn(0x7fb6480782c0 msgr2=0x7fb6480786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:14.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.356+0000 7fb64cb4d700 1 -- 172.21.15.7:0/349779787 shutdown_connections 2024-06-10T01:23:14.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.358+0000 7fb64cb4d700 1 -- 172.21.15.7:0/349779787 wait complete. 2024-06-10T01:23:14.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.360+0000 7fb64cb4d700 1 Processor -- start 2024-06-10T01:23:14.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.360+0000 7fb64cb4d700 1 -- start start 2024-06-10T01:23:14.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.360+0000 7fb64cb4d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6481013d0 0x7fb6481119d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:14.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.360+0000 7fb64cb4d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb648101d70 0x7fb648111f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:14.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.360+0000 7fb64cb4d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb648107410 0x7fb6481b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:14.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.360+0000 7fb64cb4d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb648110290 con 0x7fb6481013d0 2024-06-10T01:23:14.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.360+0000 7fb64cb4d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb648110110 con 0x7fb648101d70 2024-06-10T01:23:14.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.360+0000 7fb64cb4d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb648110410 con 0x7fb648107410 2024-06-10T01:23:14.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.360+0000 7fb645d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb648101d70 0x7fb648111f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:14.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.360+0000 7fb645d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb648101d70 0x7fb648111f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:56772/0 (socket says 172.21.15.7:56772) 2024-06-10T01:23:14.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.360+0000 7fb645d9b700 1 -- 172.21.15.7:0/1879946892 learned_addr learned my addr 172.21.15.7:0/1879946892 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:14.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.360+0000 7fb64659c700 1 --2- 172.21.15.7:0/1879946892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6481013d0 0x7fb6481119d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:14.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.361+0000 7fb646d9d700 1 --2- 172.21.15.7:0/1879946892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb648107410 0x7fb6481b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:14.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.361+0000 7fb64659c700 1 -- 172.21.15.7:0/1879946892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb648107410 msgr2=0x7fb6481b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:14.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.361+0000 7fb64659c700 1 --2- 172.21.15.7:0/1879946892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb648107410 0x7fb6481b37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:14.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.361+0000 7fb64659c700 1 -- 172.21.15.7:0/1879946892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb648101d70 msgr2=0x7fb648111f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:14.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.361+0000 7fb64659c700 1 --2- 172.21.15.7:0/1879946892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb648101d70 0x7fb648111f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:14.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.361+0000 7fb64659c700 1 -- 172.21.15.7:0/1879946892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6481b3e50 con 0x7fb6481013d0 2024-06-10T01:23:14.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.362+0000 7fb64659c700 1 --2- 172.21.15.7:0/1879946892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6481013d0 0x7fb6481119d0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fb64000ada0 tx=0x7fb64000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:14.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.372+0000 7fb6377fe700 1 -- 172.21.15.7:0/1879946892 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640017020 con 0x7fb6481013d0 2024-06-10T01:23:14.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.372+0000 7fb6377fe700 1 -- 172.21.15.7:0/1879946892 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb64000cdb0 con 0x7fb6481013d0 2024-06-10T01:23:14.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.372+0000 7fb64cb4d700 1 -- 172.21.15.7:0/1879946892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6481b4140 con 0x7fb6481013d0 2024-06-10T01:23:14.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.372+0000 7fb6377fe700 1 -- 172.21.15.7:0/1879946892 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640005280 con 0x7fb6481013d0 2024-06-10T01:23:14.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.372+0000 7fb64cb4d700 1 -- 172.21.15.7:0/1879946892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6481b4650 con 0x7fb6481013d0 2024-06-10T01:23:14.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.375+0000 7fb64cb4d700 1 -- 172.21.15.7:0/1879946892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb628005330 con 0x7fb6481013d0 2024-06-10T01:23:14.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.387+0000 7fb6377fe700 1 -- 172.21.15.7:0/1879946892 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb64000c8d0 con 0x7fb6481013d0 2024-06-10T01:23:14.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.388+0000 7fb6377fe700 1 --2- 172.21.15.7:0/1879946892 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb6300a2e30 0x7fb6300a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:14.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.389+0000 7fb6377fe700 1 -- 172.21.15.7:0/1879946892 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fb6400ce840 con 0x7fb6481013d0 2024-06-10T01:23:14.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.392+0000 7fb645d9b700 1 --2- 172.21.15.7:0/1879946892 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb6300a2e30 0x7fb6300a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:14.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.392+0000 7fb645d9b700 1 --2- 172.21.15.7:0/1879946892 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb6300a2e30 0x7fb6300a52f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb648113050 tx=0x7fb638011040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:14.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.401+0000 7fb6377fe700 1 -- 172.21.15.7:0/1879946892 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb640021020 con 0x7fb6481013d0 2024-06-10T01:23:14.531 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 25.19 MB/sec execute 111 sec latency 27129.361 ms 2024-06-10T01:23:14.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.675+0000 7fb64cb4d700 1 -- 172.21.15.7:0/1879946892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb628005620 con 0x7fb6481013d0 2024-06-10T01:23:14.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.676+0000 7fb6377fe700 1 -- 172.21.15.7:0/1879946892 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11858 (secure 0 0 0) 0x7fb640004ad0 con 0x7fb6481013d0 2024-06-10T01:23:14.671 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:23:14.671 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":50,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:22:57.989337+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28825":{"gid":28825,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/1972426673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1972426673},{"type":"v1","addr":"172.21.15.175:6835","nonce":1972426673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:23:14.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.678+0000 7fb64cb4d700 1 -- 172.21.15.7:0/1879946892 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb6300a2e30 msgr2=0x7fb6300a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:14.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.678+0000 7fb64cb4d700 1 --2- 172.21.15.7:0/1879946892 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb6300a2e30 0x7fb6300a52f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb648113050 tx=0x7fb638011040 comp rx=0 tx=0).stop 2024-06-10T01:23:14.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.678+0000 7fb64cb4d700 1 -- 172.21.15.7:0/1879946892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6481013d0 msgr2=0x7fb6481119d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:14.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.678+0000 7fb64cb4d700 1 --2- 172.21.15.7:0/1879946892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6481013d0 0x7fb6481119d0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fb64000ada0 tx=0x7fb64000c330 comp rx=0 tx=0).stop 2024-06-10T01:23:14.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.679+0000 7fb64cb4d700 1 -- 172.21.15.7:0/1879946892 shutdown_connections 2024-06-10T01:23:14.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.679+0000 7fb64cb4d700 1 --2- 172.21.15.7:0/1879946892 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb6300a2e30 0x7fb6300a52f0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:14.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.679+0000 7fb64cb4d700 1 --2- 172.21.15.7:0/1879946892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb648107410 0x7fb6481b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:14.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.679+0000 7fb64cb4d700 1 --2- 172.21.15.7:0/1879946892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6481013d0 0x7fb6481119d0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:14.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.679+0000 7fb64cb4d700 1 --2- 172.21.15.7:0/1879946892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb648101d70 0x7fb648111f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:14.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.679+0000 7fb64cb4d700 1 -- 172.21.15.7:0/1879946892 >> 172.21.15.7:0/1879946892 conn(0x7fb6480782c0 msgr2=0x7fb6480fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:14.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.679+0000 7fb64cb4d700 1 -- 172.21.15.7:0/1879946892 shutdown_connections 2024-06-10T01:23:14.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:14.679+0000 7fb64cb4d700 1 -- 172.21.15.7:0/1879946892 wait complete. 2024-06-10T01:23:14.674 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 50 2024-06-10T01:23:15.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:15 smithi007 ceph-mon[32398]: pgmap v950: 129 pgs: 129 active+clean; 570 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 4.3 MiB/s wr, 216 op/s 2024-06-10T01:23:15.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:15 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1879946892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:23:15.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:15 smithi078 ceph-mon[36683]: pgmap v950: 129 pgs: 129 active+clean; 570 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 4.3 MiB/s wr, 216 op/s 2024-06-10T01:23:15.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:15 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1879946892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:23:15.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:15 smithi175 ceph-mon[36636]: pgmap v950: 129 pgs: 129 active+clean; 570 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 4.3 MiB/s wr, 216 op/s 2024-06-10T01:23:15.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:15 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1879946892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:23:15.531 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 24.97 MB/sec execute 112 sec latency 28129.449 ms 2024-06-10T01:23:15.700 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db --set-id f064d2e --release --await' 2024-06-10T01:23:15.701 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db --set-id f064d2e --release --await 2024-06-10T01:23:16.531 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 24.75 MB/sec execute 113 sec latency 29129.549 ms 2024-06-10T01:23:16.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.912+0000 7f0f4e89b700 1 -- 172.21.15.7:0/2901023604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f48102740 msgr2=0x7f0f48102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:16.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.912+0000 7f0f4e89b700 1 --2- 172.21.15.7:0/2901023604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f48102740 0x7f0f48102b20 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f0f380099c0 tx=0x7f0f38009cd0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.912+0000 7f0f4e89b700 1 -- 172.21.15.7:0/2901023604 shutdown_connections 2024-06-10T01:23:16.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.912+0000 7f0f4e89b700 1 --2- 172.21.15.7:0/2901023604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f48107840 0x7f0f4810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.912+0000 7f0f4e89b700 1 --2- 172.21.15.7:0/2901023604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f48102740 0x7f0f48102b20 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.912+0000 7f0f4e89b700 1 --2- 172.21.15.7:0/2901023604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f481030f0 0x7f0f48107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.912+0000 7f0f4e89b700 1 -- 172.21.15.7:0/2901023604 >> 172.21.15.7:0/2901023604 conn(0x7f0f480795f0 msgr2=0x7f0f48079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:16.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.912+0000 7f0f4e89b700 1 -- 172.21.15.7:0/2901023604 shutdown_connections 2024-06-10T01:23:16.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.912+0000 7f0f4e89b700 1 -- 172.21.15.7:0/2901023604 wait complete. 2024-06-10T01:23:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.913+0000 7f0f4e89b700 1 Processor -- start 2024-06-10T01:23:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.913+0000 7f0f4e89b700 1 -- start start 2024-06-10T01:23:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.913+0000 7f0f4e89b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f48102740 0x7f0f4819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.913+0000 7f0f4e89b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f481030f0 0x7f0f4819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.913+0000 7f0f4e89b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f48107840 0x7f0f48198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.913+0000 7f0f4e89b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0f48112780 con 0x7f0f48102740 2024-06-10T01:23:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.913+0000 7f0f4e89b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0f48112600 con 0x7f0f481030f0 2024-06-10T01:23:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.913+0000 7f0f4e89b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0f48112900 con 0x7f0f48107840 2024-06-10T01:23:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.913+0000 7f0f47fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f48102740 0x7f0f4819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.913+0000 7f0f47fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f48102740 0x7f0f4819e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36468/0 (socket says 172.21.15.7:36468) 2024-06-10T01:23:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.913+0000 7f0f47fff700 1 -- 172.21.15.7:0/3859194939 learned_addr learned my addr 172.21.15.7:0/3859194939 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.914+0000 7f0f3ffff700 1 --2- 172.21.15.7:0/3859194939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f481030f0 0x7f0f4819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.914+0000 7f0f4ce38700 1 --2- 172.21.15.7:0/3859194939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f48107840 0x7f0f48198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.914+0000 7f0f47fff700 1 -- 172.21.15.7:0/3859194939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f48107840 msgr2=0x7f0f48198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.914+0000 7f0f47fff700 1 --2- 172.21.15.7:0/3859194939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f48107840 0x7f0f48198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.914+0000 7f0f47fff700 1 -- 172.21.15.7:0/3859194939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f481030f0 msgr2=0x7f0f4819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.914+0000 7f0f47fff700 1 --2- 172.21.15.7:0/3859194939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f481030f0 0x7f0f4819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.914+0000 7f0f47fff700 1 -- 172.21.15.7:0/3859194939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f48199160 con 0x7f0f48102740 2024-06-10T01:23:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.914+0000 7f0f47fff700 1 --2- 172.21.15.7:0/3859194939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f48102740 0x7f0f4819e840 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f0f38006010 tx=0x7f0f38009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.915+0000 7f0f45ffb700 1 -- 172.21.15.7:0/3859194939 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f38027070 con 0x7f0f48102740 2024-06-10T01:23:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.915+0000 7f0f4e89b700 1 -- 172.21.15.7:0/3859194939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f481993f0 con 0x7f0f48102740 2024-06-10T01:23:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.915+0000 7f0f45ffb700 1 -- 172.21.15.7:0/3859194939 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f38003780 con 0x7f0f48102740 2024-06-10T01:23:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.915+0000 7f0f45ffb700 1 -- 172.21.15.7:0/3859194939 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f38022da0 con 0x7f0f48102740 2024-06-10T01:23:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.915+0000 7f0f4e89b700 1 -- 172.21.15.7:0/3859194939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f480746c0 con 0x7f0f48102740 2024-06-10T01:23:16.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.918+0000 7f0f45ffb700 1 -- 172.21.15.7:0/3859194939 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0f3802a070 con 0x7f0f48102740 2024-06-10T01:23:16.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.919+0000 7f0f5911eb80 1 Processor -- start 2024-06-10T01:23:16.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.919+0000 7f0f5911eb80 1 -- start start 2024-06-10T01:23:16.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.919+0000 7f0f45ffb700 1 --2- 172.21.15.7:0/3859194939 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0f340a2ca0 0x7f0f340a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:16.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.919+0000 7f0f4ce38700 1 --2- 172.21.15.7:0/3859194939 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0f340a2ca0 0x7f0f340a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:16.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.919+0000 7f0f5911eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a280fd5640 0x55a280fd5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:16.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.919+0000 7f0f45ffb700 1 -- 172.21.15.7:0/3859194939 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f0f380d0150 con 0x7f0f48102740 2024-06-10T01:23:16.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.919+0000 7f0f5911eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a280fd5f60 0x55a280fcf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:16.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.919+0000 7f0f5911eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a280fcff20 0x55a280fd0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:16.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.919+0000 7f0f5911eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a280f21810 con 0x55a280fd5f60 2024-06-10T01:23:16.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.919+0000 7f0f5911eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a280ec9720 con 0x55a280fcff20 2024-06-10T01:23:16.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.919+0000 7f0f5911eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a280f1d7d0 con 0x55a280fd5640 2024-06-10T01:23:16.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.919+0000 7f0f3ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a280fd5f60 0x55a280fcf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:16.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.919+0000 7f0f3ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a280fd5f60 0x55a280fcf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36470/0 (socket says 172.21.15.7:36470) 2024-06-10T01:23:16.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.919+0000 7f0f3ffff700 1 -- 172.21.15.7:0/3018880119 learned_addr learned my addr 172.21.15.7:0/3018880119 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:16.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.919+0000 7f0f47fff700 1 --2- 172.21.15.7:0/3018880119 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a280fd5640 0x55a280fd5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:16.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.920+0000 7f0f3ffff700 1 -- 172.21.15.7:0/3018880119 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a280fd5640 msgr2=0x55a280fd5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:16.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.920+0000 7f0f3ffff700 1 --2- 172.21.15.7:0/3018880119 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a280fd5640 0x55a280fd5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.920+0000 7f0f4ce38700 1 --2- 172.21.15.7:0/3018880119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a280fcff20 0x55a280fd0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:16.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.920+0000 7f0f3ffff700 1 -- 172.21.15.7:0/3018880119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a280fcff20 msgr2=0x55a280fd0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:16.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.920+0000 7f0f3ffff700 1 --2- 172.21.15.7:0/3018880119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a280fcff20 0x55a280fd0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.920+0000 7f0f3ffff700 1 -- 172.21.15.7:0/3018880119 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a280ebff50 con 0x55a280fd5f60 2024-06-10T01:23:16.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.920+0000 7f0f4ce38700 1 --2- 172.21.15.7:0/3018880119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a280fcff20 0x55a280fd0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:23:16.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.920+0000 7f0f4ce38700 1 --2- 172.21.15.7:0/3859194939 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0f340a2ca0 0x7f0f340a5160 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009b90 tx=0x7f0f40009f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:16.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.920+0000 7f0f3ffff700 1 --2- 172.21.15.7:0/3018880119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a280fd5f60 0x55a280fcf9e0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000ac70 tx=0x7f0f3000af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e60c86836877ea9f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:16.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.920+0000 7f0f3e7fc700 1 -- 172.21.15.7:0/3018880119 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3001d070 con 0x55a280fd5f60 2024-06-10T01:23:16.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.920+0000 7f0f3e7fc700 1 -- 172.21.15.7:0/3018880119 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f3000fde0 con 0x55a280fd5f60 2024-06-10T01:23:16.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.920+0000 7f0f3e7fc700 1 -- 172.21.15.7:0/3018880119 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f300107f0 con 0x55a280fd5f60 2024-06-10T01:23:16.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.921+0000 7f0f5911eb80 1 -- 172.21.15.7:0/3018880119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a280fd5f60 msgr2=0x55a280fcf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:16.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.921+0000 7f0f5911eb80 1 --2- 172.21.15.7:0/3018880119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a280fd5f60 0x55a280fcf9e0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000ac70 tx=0x7f0f3000af80 comp rx=0 tx=0).stop 2024-06-10T01:23:16.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.921+0000 7f0f5911eb80 1 -- 172.21.15.7:0/3018880119 shutdown_connections 2024-06-10T01:23:16.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.921+0000 7f0f5911eb80 1 --2- 172.21.15.7:0/3018880119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a280fcff20 0x55a280fd0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.921+0000 7f0f5911eb80 1 --2- 172.21.15.7:0/3018880119 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a280fd5640 0x55a280fd5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.921+0000 7f0f5911eb80 1 --2- 172.21.15.7:0/3018880119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a280fd5f60 0x55a280fcf9e0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.921+0000 7f0f5911eb80 1 -- 172.21.15.7:0/3018880119 >> 172.21.15.7:0/3018880119 conn(0x55a280efacc0 msgr2=0x55a280fca850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:16.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.921+0000 7f0f5911eb80 1 -- 172.21.15.7:0/3018880119 shutdown_connections 2024-06-10T01:23:16.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.921+0000 7f0f5911eb80 1 -- 172.21.15.7:0/3018880119 wait complete. 2024-06-10T01:23:16.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.921+0000 7f0f5911eb80 1 -- start start 2024-06-10T01:23:16.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.922+0000 7f0f5911eb80 1 Processor -- start 2024-06-10T01:23:16.921 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:23:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.922+0000 7f0f5911eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a280fcff20 0x55a280fdbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.922+0000 7f0f5911eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a280fd5640 0x55a280fd8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.922+0000 7f0f5911eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a280fd5f60 0x55a280fd9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.922+0000 7f0f5911eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a280fdc9d0 con 0x55a280fcff20 2024-06-10T01:23:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.922+0000 7f0f5911eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a280ec1bf0 con 0x55a280fd5640 2024-06-10T01:23:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.922+0000 7f0f5911eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a280fdcb50 con 0x55a280fd5f60 2024-06-10T01:23:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.922+0000 7f0f5911eb80 10 client.0 _ll_register_callbacks cb 0x55a280ebb920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:23:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.922+0000 7f0f47fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a280fcff20 0x55a280fdbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.922+0000 7f0f47fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a280fcff20 0x55a280fdbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36472/0 (socket says 172.21.15.7:36472) 2024-06-10T01:23:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.922+0000 7f0f47fff700 1 -- 172.21.15.7:0/748999371 learned_addr learned my addr 172.21.15.7:0/748999371 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.922+0000 7f0f4ce38700 1 --2- 172.21.15.7:0/748999371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a280fd5f60 0x55a280fd9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.923+0000 7f0f3ffff700 1 --2- 172.21.15.7:0/748999371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a280fd5640 0x55a280fd8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.923+0000 7f0f47fff700 1 -- 172.21.15.7:0/748999371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a280fd5f60 msgr2=0x55a280fd9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.923+0000 7f0f47fff700 1 --2- 172.21.15.7:0/748999371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a280fd5f60 0x55a280fd9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.923+0000 7f0f47fff700 1 -- 172.21.15.7:0/748999371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a280fd5640 msgr2=0x55a280fd8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.923+0000 7f0f47fff700 1 --2- 172.21.15.7:0/748999371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a280fd5640 0x55a280fd8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.923+0000 7f0f47fff700 1 -- 172.21.15.7:0/748999371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a280e708d0 con 0x55a280fcff20 2024-06-10T01:23:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.923+0000 7f0f3ffff700 1 --2- 172.21.15.7:0/748999371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a280fd5640 0x55a280fd8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:23:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.923+0000 7f0f47fff700 1 --2- 172.21.15.7:0/748999371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a280fcff20 0x55a280fdbcc0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f0f380cdc40 tx=0x7f0f38009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2a1512e2f53b32 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.923+0000 7f0f277fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:23:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.923+0000 7f0f277fe700 1 -- 172.21.15.7:0/748999371 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f380b5060 con 0x55a280fcff20 2024-06-10T01:23:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.923+0000 7f0f277fe700 1 -- 172.21.15.7:0/748999371 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f0f380b0b50 con 0x55a280fcff20 2024-06-10T01:23:16.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.923+0000 7f0f5911eb80 1 -- 172.21.15.7:0/748999371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f3000fde0 con 0x55a280fcff20 2024-06-10T01:23:16.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.923+0000 7f0f277fe700 1 -- 172.21.15.7:0/748999371 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f0f3809c6c0 con 0x55a280fcff20 2024-06-10T01:23:16.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.923+0000 7f0f5911eb80 10 client.19707 fetch_fsmap learned FSMap version 50 2024-06-10T01:23:16.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.923+0000 7f0f5911eb80 1 -- 172.21.15.7:0/748999371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55a280fe7240 con 0x55a280fcff20 2024-06-10T01:23:16.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.924+0000 7f0f277fe700 1 -- 172.21.15.7:0/748999371 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f3000fde0 con 0x55a280fcff20 2024-06-10T01:23:16.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.924+0000 7f0f277fe700 1 -- 172.21.15.7:0/748999371 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7f0f3809bb10 con 0x55a280fcff20 2024-06-10T01:23:16.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.924+0000 7f0f5911eb80 10 client.19707 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:23:16.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.924+0000 7f0f5911eb80 10 client.19707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:23:16.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.924+0000 7f0f5911eb80 20 client.19707 populate_metadata read hostname 'smithi007' 2024-06-10T01:23:16.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.924+0000 7f0f5911eb80 1 --2- 172.21.15.7:0/748999371 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a280fe97b0 0x55a280febc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:16.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.924+0000 7f0f5911eb80 4 client.19707 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:23:16.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.924+0000 7f0f5911eb80 1 -- 172.21.15.7:0/748999371 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a280f2ed10 con 0x55a280fe97b0 2024-06-10T01:23:16.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.924+0000 7f0f3ffff700 1 --2- 172.21.15.7:0/748999371 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a280fe97b0 0x55a280febc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:16.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.925+0000 7f0f3ffff700 1 --2- 172.21.15.7:0/748999371 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a280fe97b0 0x55a280febc70 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64c497953a66bfad server_cookie=28d8ffc7d99db35d in_seq=0 out_seq=0 2024-06-10T01:23:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.925+0000 7f0f277fe700 10 client.19707 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.925+0000 7f0f277fe700 1 -- 172.21.15.7:0/748999371 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a280f2ed10 con 0x55a280fe97b0 2024-06-10T01:23:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.925+0000 7f0f277fe700 10 client.19707 handle_command_reply: tid=0 2024-06-10T01:23:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.925+0000 7f0f5911eb80 1 client.19707 shutdown 2024-06-10T01:23:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.925+0000 7f0f5911eb80 1 -- 172.21.15.7:0/748999371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a280fcff20 msgr2=0x55a280fdbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.925+0000 7f0f5911eb80 1 --2- 172.21.15.7:0/748999371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a280fcff20 0x55a280fdbcc0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f0f380cdc40 tx=0x7f0f38009310 comp rx=0 tx=0).stop 2024-06-10T01:23:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.926+0000 7f0f5911eb80 1 -- 172.21.15.7:0/748999371 shutdown_connections 2024-06-10T01:23:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.926+0000 7f0f5911eb80 1 --2- 172.21.15.7:0/748999371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a280fd5f60 0x55a280fd9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.926+0000 7f0f5911eb80 1 --2- 172.21.15.7:0/748999371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a280fcff20 0x55a280fdbcc0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.926+0000 7f0f5911eb80 1 --2- 172.21.15.7:0/748999371 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a280fe97b0 0x55a280febc70 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.926+0000 7f0f5911eb80 1 --2- 172.21.15.7:0/748999371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a280fd5640 0x55a280fd8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.926+0000 7f0f5911eb80 1 -- 172.21.15.7:0/748999371 >> 172.21.15.7:0/748999371 conn(0x55a280efacc0 msgr2=0x55a280fce9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.926+0000 7f0f277fe700 0 client.19707 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.926+0000 7f0f5911eb80 1 -- 172.21.15.7:0/748999371 shutdown_connections 2024-06-10T01:23:16.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.926+0000 7f0f5911eb80 1 -- 172.21.15.7:0/748999371 wait complete. 2024-06-10T01:23:16.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.926+0000 7f0f5911eb80 20 client.19707 trim_cache size 0 max 16384 2024-06-10T01:23:16.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.952+0000 7f0f5911eb80 1 Processor -- start 2024-06-10T01:23:16.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.952+0000 7f0f5911eb80 1 -- start start 2024-06-10T01:23:16.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.952+0000 7f0f5911eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a280fcff20 0x55a280fcf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:16.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.952+0000 7f0f5911eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a280fd5640 0x55a280fd6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:16.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.952+0000 7f0f5911eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a280fd5f60 0x55a280fd7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:16.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.952+0000 7f0f5911eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481b2b50 con 0x55a280fd5640 2024-06-10T01:23:16.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.952+0000 7f0f5911eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481b03e0 con 0x55a280fcff20 2024-06-10T01:23:16.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.952+0000 7f0f5911eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481ae230 con 0x55a280fd5f60 2024-06-10T01:23:16.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.952+0000 7f0f3ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a280fd5640 0x55a280fd6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:16.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.952+0000 7f0f3ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a280fd5640 0x55a280fd6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36488/0 (socket says 172.21.15.7:36488) 2024-06-10T01:23:16.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.952+0000 7f0f3ffff700 1 -- 172.21.15.7:0/3204644767 learned_addr learned my addr 172.21.15.7:0/3204644767 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:16.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.952+0000 7f0f47fff700 1 --2- 172.21.15.7:0/3204644767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a280fcff20 0x55a280fcf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:16.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.952+0000 7f0f4ce38700 1 --2- 172.21.15.7:0/3204644767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a280fd5f60 0x55a280fd7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:16.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.953+0000 7f0f47fff700 1 -- 172.21.15.7:0/3204644767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a280fd5f60 msgr2=0x55a280fd7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:16.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.953+0000 7f0f47fff700 1 --2- 172.21.15.7:0/3204644767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a280fd5f60 0x55a280fd7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.953+0000 7f0f47fff700 1 -- 172.21.15.7:0/3204644767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a280fd5640 msgr2=0x55a280fd6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:16.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.953+0000 7f0f47fff700 1 --2- 172.21.15.7:0/3204644767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a280fd5640 0x55a280fd6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.953+0000 7f0f47fff700 1 -- 172.21.15.7:0/3204644767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a280fd7530 con 0x55a280fcff20 2024-06-10T01:23:16.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.953+0000 7f0f3ffff700 1 --2- 172.21.15.7:0/3204644767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a280fd5640 0x55a280fd6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:23:16.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.953+0000 7f0f47fff700 1 --2- 172.21.15.7:0/3204644767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a280fcff20 0x55a280fcf9e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0f38098b20 tx=0x7f0f3809abd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ceea47f7b2a3147 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:16.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.953+0000 7f0f3dffb700 1 -- 172.21.15.7:0/3204644767 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f380a52a0 con 0x55a280fcff20 2024-06-10T01:23:16.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.953+0000 7f0f3dffb700 1 -- 172.21.15.7:0/3204644767 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f38022590 con 0x55a280fcff20 2024-06-10T01:23:16.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.953+0000 7f0f3dffb700 1 -- 172.21.15.7:0/3204644767 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f380bc2e0 con 0x55a280fcff20 2024-06-10T01:23:16.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.954+0000 7f0f5911eb80 1 -- 172.21.15.7:0/3204644767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a280fcff20 msgr2=0x55a280fcf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:16.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.954+0000 7f0f5911eb80 1 --2- 172.21.15.7:0/3204644767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a280fcff20 0x55a280fcf9e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0f38098b20 tx=0x7f0f3809abd0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.954+0000 7f0f5911eb80 1 -- 172.21.15.7:0/3204644767 shutdown_connections 2024-06-10T01:23:16.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.954+0000 7f0f5911eb80 1 --2- 172.21.15.7:0/3204644767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a280fd5f60 0x55a280fd7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.954+0000 7f0f5911eb80 1 --2- 172.21.15.7:0/3204644767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a280fcff20 0x55a280fcf9e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.954+0000 7f0f5911eb80 1 --2- 172.21.15.7:0/3204644767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a280fd5640 0x55a280fd6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.954+0000 7f0f5911eb80 1 -- 172.21.15.7:0/3204644767 >> 172.21.15.7:0/3204644767 conn(0x55a280efacc0 msgr2=0x55a280fc3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:16.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.954+0000 7f0f5911eb80 1 -- 172.21.15.7:0/3204644767 shutdown_connections 2024-06-10T01:23:16.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.954+0000 7f0f5911eb80 1 -- 172.21.15.7:0/3204644767 wait complete. 2024-06-10T01:23:16.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.954+0000 7f0f5911eb80 1 -- start start 2024-06-10T01:23:16.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.955+0000 7f0f5911eb80 1 Processor -- start 2024-06-10T01:23:16.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.955+0000 7f0f5911eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a280fcff20 0x55a280fe1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:16.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.955+0000 7f0f5911eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a280fd5640 0x55a280fe2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:16.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.955+0000 7f0f5911eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a280fd5f60 0x55a280fdede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:16.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.955+0000 7f0f5911eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481ae8a0 con 0x55a280fcff20 2024-06-10T01:23:16.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.955+0000 7f0f5911eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481afd10 con 0x55a280fd5640 2024-06-10T01:23:16.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.955+0000 7f0f5911eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481aef70 con 0x55a280fd5f60 2024-06-10T01:23:16.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.955+0000 7f0f5911eb80 10 client.0 _ll_register_callbacks cb 0x55a280ebb920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:23:16.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.956+0000 7f0f47fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a280fcff20 0x55a280fe1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:16.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.956+0000 7f0f47fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a280fcff20 0x55a280fe1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36494/0 (socket says 172.21.15.7:36494) 2024-06-10T01:23:16.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.956+0000 7f0f47fff700 1 -- 172.21.15.7:0/300254339 learned_addr learned my addr 172.21.15.7:0/300254339 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:16.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.956+0000 7f0f3ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a280fd5640 0x55a280fe2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:16.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.956+0000 7f0f4ce38700 1 --2- 172.21.15.7:0/300254339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a280fd5f60 0x55a280fdede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:16.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.956+0000 7f0f3ffff700 1 -- 172.21.15.7:0/300254339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a280fd5f60 msgr2=0x55a280fdede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:16.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.956+0000 7f0f3ffff700 1 --2- 172.21.15.7:0/300254339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a280fd5f60 0x55a280fdede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.956+0000 7f0f3ffff700 1 -- 172.21.15.7:0/300254339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a280fcff20 msgr2=0x55a280fe1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:16.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.956+0000 7f0f3ffff700 1 --2- 172.21.15.7:0/300254339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a280fcff20 0x55a280fe1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.956+0000 7f0f3ffff700 1 -- 172.21.15.7:0/300254339 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a280fe29d0 con 0x55a280fd5640 2024-06-10T01:23:16.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.956+0000 7f0f4ce38700 1 --2- 172.21.15.7:0/300254339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a280fd5f60 0x55a280fdede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:23:16.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.956+0000 7f0f3ffff700 1 --2- 172.21.15.7:0/300254339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a280fd5640 0x55a280fe2350 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0f30000c00 tx=0x7f0f30018ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c58a8566fc519fe4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:16.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.956+0000 7f0f277fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:23:16.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.956+0000 7f0f277fe700 1 -- 172.21.15.7:0/300254339 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f30031020 con 0x55a280fd5640 2024-06-10T01:23:16.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.956+0000 7f0f5911eb80 1 -- 172.21.15.7:0/300254339 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f38022590 con 0x55a280fd5640 2024-06-10T01:23:16.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.957+0000 7f0f277fe700 1 -- 172.21.15.7:0/300254339 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f0f30005400 con 0x55a280fd5640 2024-06-10T01:23:16.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.957+0000 7f0f277fe700 1 -- 172.21.15.7:0/300254339 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f0f3001d040 con 0x55a280fd5640 2024-06-10T01:23:16.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.957+0000 7f0f5911eb80 10 client.28931 fetch_fsmap learned FSMap version 50 2024-06-10T01:23:16.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.957+0000 7f0f5911eb80 1 -- 172.21.15.7:0/300254339 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55a28105d2f0 con 0x55a280fd5640 2024-06-10T01:23:16.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.957+0000 7f0f277fe700 1 -- 172.21.15.7:0/300254339 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f38022590 con 0x55a280fd5640 2024-06-10T01:23:16.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.957+0000 7f0f277fe700 1 -- 172.21.15.7:0/300254339 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7f0f30004a60 con 0x55a280fd5640 2024-06-10T01:23:16.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.957+0000 7f0f5911eb80 10 client.28931 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:23:16.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.957+0000 7f0f5911eb80 10 client.28931 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:23:16.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.957+0000 7f0f5911eb80 20 client.28931 populate_metadata read hostname 'smithi007' 2024-06-10T01:23:16.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.957+0000 7f0f5911eb80 1 --2- 172.21.15.7:0/300254339 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a28105d970 0x55a28105fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:16.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.957+0000 7f0f5911eb80 4 client.28931 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:23:16.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.957+0000 7f0f5911eb80 1 -- 172.21.15.7:0/300254339 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a280f29290 con 0x55a28105d970 2024-06-10T01:23:16.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.958+0000 7f0f47fff700 1 --2- 172.21.15.7:0/300254339 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a28105d970 0x55a28105fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:16.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.958+0000 7f0f47fff700 1 --2- 172.21.15.7:0/300254339 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a28105d970 0x55a28105fd90 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7de6b79174d9b3 server_cookie=c8cf7a32b7848891 in_seq=0 out_seq=0 2024-06-10T01:23:16.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.958+0000 7f0f277fe700 10 client.28931 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:16.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.959+0000 7f0f277fe700 1 -- 172.21.15.7:0/300254339 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a280f29290 con 0x55a28105d970 2024-06-10T01:23:16.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.959+0000 7f0f277fe700 10 client.28931 handle_command_reply: tid=0 2024-06-10T01:23:16.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.959+0000 7f0f5911eb80 1 client.28931 shutdown 2024-06-10T01:23:16.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.959+0000 7f0f5911eb80 1 -- 172.21.15.7:0/300254339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a280fd5640 msgr2=0x55a280fe2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:16.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.959+0000 7f0f5911eb80 1 --2- 172.21.15.7:0/300254339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a280fd5640 0x55a280fe2350 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0f30000c00 tx=0x7f0f30018ef0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.959+0000 7f0f5911eb80 1 -- 172.21.15.7:0/300254339 shutdown_connections 2024-06-10T01:23:16.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.959+0000 7f0f5911eb80 1 --2- 172.21.15.7:0/300254339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a280fd5f60 0x55a280fdede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.959+0000 7f0f5911eb80 1 --2- 172.21.15.7:0/300254339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a280fcff20 0x55a280fe1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.959+0000 7f0f5911eb80 1 --2- 172.21.15.7:0/300254339 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a28105d970 0x55a28105fd90 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.959+0000 7f0f277fe700 0 client.28931 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:16.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.959+0000 7f0f5911eb80 1 --2- 172.21.15.7:0/300254339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a280fd5640 0x55a280fe2350 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.959+0000 7f0f5911eb80 1 -- 172.21.15.7:0/300254339 >> 172.21.15.7:0/300254339 conn(0x55a280efacc0 msgr2=0x55a280fd8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:16.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.959+0000 7f0f5911eb80 1 -- 172.21.15.7:0/300254339 shutdown_connections 2024-06-10T01:23:16.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.959+0000 7f0f5911eb80 1 -- 172.21.15.7:0/300254339 wait complete. 2024-06-10T01:23:16.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.959+0000 7f0f5911eb80 20 client.28931 trim_cache size 0 max 16384 2024-06-10T01:23:16.957 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:23:16.958 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T01:23:16.958 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-10T01:23:16.958 INFO:teuthology.orchestra.run.smithi007.stdout: "86ddbadf-cafa-40e1-a1d0-90cf09b01c1b": { 2024-06-10T01:23:16.958 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-10T01:23:16.958 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "86ddbadf-cafa-40e1-a1d0-90cf09b01c1b", 2024-06-10T01:23:16.958 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-10T01:23:16.958 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:23:16.958 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:23:16.958 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:23:16.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.960+0000 7f0f4e89b700 1 -- 172.21.15.7:0/3859194939 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0f340a2ca0 msgr2=0x7f0f340a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:16.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.960+0000 7f0f4e89b700 1 --2- 172.21.15.7:0/3859194939 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0f340a2ca0 0x7f0f340a5160 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009b90 tx=0x7f0f40009f50 comp rx=0 tx=0).stop 2024-06-10T01:23:16.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.960+0000 7f0f4e89b700 1 -- 172.21.15.7:0/3859194939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f48102740 msgr2=0x7f0f4819e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:16.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.960+0000 7f0f4e89b700 1 --2- 172.21.15.7:0/3859194939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f48102740 0x7f0f4819e840 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f0f38006010 tx=0x7f0f38009f80 comp rx=0 tx=0).stop 2024-06-10T01:23:16.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.960+0000 7f0f4e89b700 1 -- 172.21.15.7:0/3859194939 shutdown_connections 2024-06-10T01:23:16.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.960+0000 7f0f4e89b700 1 --2- 172.21.15.7:0/3859194939 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0f340a2ca0 0x7f0f340a5160 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.960+0000 7f0f4e89b700 1 --2- 172.21.15.7:0/3859194939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f48107840 0x7f0f48198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.960+0000 7f0f4e89b700 1 --2- 172.21.15.7:0/3859194939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f48102740 0x7f0f4819e840 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.960+0000 7f0f4e89b700 1 --2- 172.21.15.7:0/3859194939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f481030f0 0x7f0f4819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:16.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.960+0000 7f0f4e89b700 1 -- 172.21.15.7:0/3859194939 >> 172.21.15.7:0/3859194939 conn(0x7f0f480795f0 msgr2=0x7f0f48101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:16.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.961+0000 7f0f4e89b700 1 -- 172.21.15.7:0/3859194939 shutdown_connections 2024-06-10T01:23:16.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:16.961+0000 7f0f4e89b700 1 -- 172.21.15.7:0/3859194939 wait complete. 2024-06-10T01:23:17.145 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:17 smithi007 ceph-mon[32398]: pgmap v951: 129 pgs: 129 active+clean; 557 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 6.0 MiB/s wr, 349 op/s 2024-06-10T01:23:17.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:17 smithi078 ceph-mon[36683]: pgmap v951: 129 pgs: 129 active+clean; 557 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 6.0 MiB/s wr, 349 op/s 2024-06-10T01:23:17.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:17 smithi175 ceph-mon[36636]: pgmap v951: 129 pgs: 129 active+clean; 557 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 6.0 MiB/s wr, 349 op/s 2024-06-10T01:23:17.532 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 24.53 MB/sec execute 114 sec latency 30129.647 ms 2024-06-10T01:23:18.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:18 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:23:18.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:18 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:23:18.512 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:18 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:23:18.512 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:18 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:23:18.531 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 24.32 MB/sec execute 115 sec latency 31129.710 ms 2024-06-10T01:23:18.550 INFO:tasks.cephfs.filesystem:scrub status for tag:86ddbadf-cafa-40e1-a1d0-90cf09b01c1b - {'path': '/', 'tag': '86ddbadf-cafa-40e1-a1d0-90cf09b01c1b', 'options': 'recursive,force'} 2024-06-10T01:23:18.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:18 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:23:18.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:18 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:23:19.218 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:19 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:23:19.218 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:19 smithi007 ceph-mon[32398]: pgmap v952: 129 pgs: 129 active+clean; 559 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 4.5 MiB/s wr, 279 op/s 2024-06-10T01:23:19.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:19 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:23:19.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:19 smithi175 ceph-mon[36636]: pgmap v952: 129 pgs: 129 active+clean; 559 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 4.5 MiB/s wr, 279 op/s 2024-06-10T01:23:19.531 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92367 24.11 MB/sec execute 116 sec latency 32129.750 ms 2024-06-10T01:23:19.578 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:19 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:23:19.578 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:19 smithi078 ceph-mon[36683]: pgmap v952: 129 pgs: 129 active+clean; 559 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 4.5 MiB/s wr, 279 op/s 2024-06-10T01:23:19.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.590+0000 7f9b994d2700 1 -- 172.21.15.7:0/2885465878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b94108b00 msgr2=0x7f9b94108f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:19.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.590+0000 7f9b994d2700 1 --2- 172.21.15.7:0/2885465878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b94108b00 0x7f9b94108f80 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f9b88009990 tx=0x7f9b88009ca0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.596+0000 7f9b994d2700 1 -- 172.21.15.7:0/2885465878 shutdown_connections 2024-06-10T01:23:19.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.596+0000 7f9b994d2700 1 --2- 172.21.15.7:0/2885465878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b941094c0 0x7f9b94115980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.596+0000 7f9b994d2700 1 --2- 172.21.15.7:0/2885465878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b9410eab0 0x7f9b9410ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.596+0000 7f9b994d2700 1 --2- 172.21.15.7:0/2885465878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b94108b00 0x7f9b94108f80 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.596+0000 7f9b994d2700 1 -- 172.21.15.7:0/2885465878 >> 172.21.15.7:0/2885465878 conn(0x7f9b94076a50 msgr2=0x7f9b94078e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:19.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.598+0000 7f9b994d2700 1 -- 172.21.15.7:0/2885465878 shutdown_connections 2024-06-10T01:23:19.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.599+0000 7f9b994d2700 1 -- 172.21.15.7:0/2885465878 wait complete. 2024-06-10T01:23:19.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.600+0000 7f9b994d2700 1 Processor -- start 2024-06-10T01:23:19.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.600+0000 7f9b994d2700 1 -- start start 2024-06-10T01:23:19.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.600+0000 7f9b994d2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b94108b00 0x7f9b9419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:19.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.600+0000 7f9b994d2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b941094c0 0x7f9b9419f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:19.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.600+0000 7f9b994d2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b9410eab0 0x7f9b941a3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:19.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.600+0000 7f9b994d2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9b94118ee0 con 0x7f9b941094c0 2024-06-10T01:23:19.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.600+0000 7f9b994d2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9b94118d60 con 0x7f9b9410eab0 2024-06-10T01:23:19.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.600+0000 7f9b994d2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9b94119060 con 0x7f9b94108b00 2024-06-10T01:23:19.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.601+0000 7f9b937fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b9410eab0 0x7f9b941a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:19.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.602+0000 7f9b937fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b9410eab0 0x7f9b941a3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:56830/0 (socket says 172.21.15.7:56830) 2024-06-10T01:23:19.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.602+0000 7f9b937fe700 1 -- 172.21.15.7:0/444532150 learned_addr learned my addr 172.21.15.7:0/444532150 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:19.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.602+0000 7f9b92ffd700 1 --2- 172.21.15.7:0/444532150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b94108b00 0x7f9b9419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:19.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.605+0000 7f9b937fe700 1 -- 172.21.15.7:0/444532150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b94108b00 msgr2=0x7f9b9419f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:19.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.605+0000 7f9b937fe700 1 --2- 172.21.15.7:0/444532150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b94108b00 0x7f9b9419f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.605+0000 7f9b937fe700 1 -- 172.21.15.7:0/444532150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b941094c0 msgr2=0x7f9b9419f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:19.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.605+0000 7f9b937fe700 1 --2- 172.21.15.7:0/444532150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b941094c0 0x7f9b9419f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.605+0000 7f9b937fe700 1 -- 172.21.15.7:0/444532150 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b941a40b0 con 0x7f9b9410eab0 2024-06-10T01:23:19.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.607+0000 7f9b937fe700 1 --2- 172.21.15.7:0/444532150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b9410eab0 0x7f9b941a3970 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00b780 tx=0x7f9b8c00bb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:19.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.607+0000 7f9b7bfff700 1 -- 172.21.15.7:0/444532150 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c00f840 con 0x7f9b9410eab0 2024-06-10T01:23:19.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.607+0000 7f9b994d2700 1 -- 172.21.15.7:0/444532150 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b941a43a0 con 0x7f9b9410eab0 2024-06-10T01:23:19.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.607+0000 7f9b994d2700 1 -- 172.21.15.7:0/444532150 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b9406beb0 con 0x7f9b9410eab0 2024-06-10T01:23:19.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.607+0000 7f9b7bfff700 1 -- 172.21.15.7:0/444532150 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b8c017400 con 0x7f9b9410eab0 2024-06-10T01:23:19.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.607+0000 7f9b7bfff700 1 -- 172.21.15.7:0/444532150 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c00d630 con 0x7f9b9410eab0 2024-06-10T01:23:19.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.612+0000 7f9ba3d55b80 1 Processor -- start 2024-06-10T01:23:19.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.612+0000 7f9ba3d55b80 1 -- start start 2024-06-10T01:23:19.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.612+0000 7f9ba3d55b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db6ec46c40 0x55db6ec47020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:19.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.612+0000 7f9b7bfff700 1 -- 172.21.15.7:0/444532150 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9b8c00d890 con 0x7f9b9410eab0 2024-06-10T01:23:19.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.612+0000 7f9ba3d55b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db6ec47560 0x55db6ec40e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:19.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.612+0000 7f9ba3d55b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db6ec413d0 0x55db6ec41810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:19.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.612+0000 7f9ba3d55b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55db6eb92910 con 0x55db6ec46c40 2024-06-10T01:23:19.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.612+0000 7f9ba3d55b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55db6eb3a820 con 0x55db6ec47560 2024-06-10T01:23:19.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.612+0000 7f9ba3d55b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55db6eb8e8d0 con 0x55db6ec413d0 2024-06-10T01:23:19.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.612+0000 7f9b92ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db6ec47560 0x55db6ec40e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:19.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.612+0000 7f9b92ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db6ec47560 0x55db6ec40e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:56842/0 (socket says 172.21.15.7:56842) 2024-06-10T01:23:19.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.612+0000 7f9b92ffd700 1 -- 172.21.15.7:0/3892943298 learned_addr learned my addr 172.21.15.7:0/3892943298 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:19.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.613+0000 7f9b927fc700 1 --2- 172.21.15.7:0/3892943298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db6ec46c40 0x55db6ec47020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:19.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.613+0000 7f9b92ffd700 1 -- 172.21.15.7:0/3892943298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db6ec413d0 msgr2=0x55db6ec41810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:23:19.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.613+0000 7f9b92ffd700 1 --2- 172.21.15.7:0/3892943298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db6ec413d0 0x55db6ec41810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.613+0000 7f9b92ffd700 1 -- 172.21.15.7:0/3892943298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db6ec46c40 msgr2=0x55db6ec47020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:19.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.613+0000 7f9b92ffd700 1 --2- 172.21.15.7:0/3892943298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db6ec46c40 0x55db6ec47020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.613+0000 7f9b92ffd700 1 -- 172.21.15.7:0/3892943298 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db6eb31050 con 0x55db6ec47560 2024-06-10T01:23:19.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.613+0000 7f9b927fc700 1 --2- 172.21.15.7:0/3892943298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db6ec46c40 0x55db6ec47020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:23:19.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.613+0000 7f9b7bfff700 1 --2- 172.21.15.7:0/444532150 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9b7c0a2d70 0x7f9b7c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:19.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.613+0000 7f9b7bfff700 1 -- 172.21.15.7:0/444532150 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f9b8c0cf1c0 con 0x7f9b9410eab0 2024-06-10T01:23:19.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.613+0000 7f9b927fc700 1 --2- 172.21.15.7:0/444532150 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9b7c0a2d70 0x7f9b7c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:19.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.614+0000 7f9b92ffd700 1 --2- 172.21.15.7:0/3892943298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db6ec47560 0x55db6ec40e90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9b84005e20 tx=0x7f9b8400cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efbcf58efe4af062 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:19.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.614+0000 7f9b927fc700 1 --2- 172.21.15.7:0/444532150 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9b7c0a2d70 0x7f9b7c0a5230 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800b5c0 tx=0x7f9b88005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:19.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.616+0000 7f9b797fa700 1 -- 172.21.15.7:0/3892943298 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8400eda0 con 0x55db6ec47560 2024-06-10T01:23:19.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.616+0000 7f9b797fa700 1 -- 172.21.15.7:0/3892943298 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b84010910 con 0x55db6ec47560 2024-06-10T01:23:19.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.616+0000 7f9b797fa700 1 -- 172.21.15.7:0/3892943298 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8401aa70 con 0x55db6ec47560 2024-06-10T01:23:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.617+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/3892943298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db6ec47560 msgr2=0x55db6ec40e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.617+0000 7f9ba3d55b80 1 --2- 172.21.15.7:0/3892943298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db6ec47560 0x55db6ec40e90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9b84005e20 tx=0x7f9b8400cf90 comp rx=0 tx=0).stop 2024-06-10T01:23:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.617+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/3892943298 shutdown_connections 2024-06-10T01:23:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.617+0000 7f9ba3d55b80 1 --2- 172.21.15.7:0/3892943298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db6ec413d0 0x55db6ec41810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.617+0000 7f9ba3d55b80 1 --2- 172.21.15.7:0/3892943298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db6ec46c40 0x55db6ec47020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.618+0000 7f9ba3d55b80 1 --2- 172.21.15.7:0/3892943298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db6ec47560 0x55db6ec40e90 secure :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9b84005e20 tx=0x7f9b8400cf90 comp rx=0 tx=0).stop 2024-06-10T01:23:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.618+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/3892943298 >> 172.21.15.7:0/3892943298 conn(0x55db6eb6bdc0 msgr2=0x55db6ec3c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.618+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/3892943298 shutdown_connections 2024-06-10T01:23:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.618+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/3892943298 wait complete. 2024-06-10T01:23:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.619+0000 7f9ba3d55b80 1 -- start start 2024-06-10T01:23:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.619+0000 7f9ba3d55b80 1 Processor -- start 2024-06-10T01:23:19.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.619+0000 7f9ba3d55b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db6ec413d0 0x55db6ec4d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:19.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.619+0000 7f9ba3d55b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db6ec46c40 0x55db6ec4a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:19.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.619+0000 7f9ba3d55b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db6ec4adb0 0x55db6ec4b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:19.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.619+0000 7f9ba3d55b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55db6ec4dfd0 con 0x55db6ec4adb0 2024-06-10T01:23:19.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.619+0000 7f9ba3d55b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55db6eb32cf0 con 0x55db6ec413d0 2024-06-10T01:23:19.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.619+0000 7f9ba3d55b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55db6ec4e150 con 0x55db6ec46c40 2024-06-10T01:23:19.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.619+0000 7f9ba3d55b80 10 client.0 _ll_register_callbacks cb 0x55db6eb2ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:23:19.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.619+0000 7f9b937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db6ec4adb0 0x55db6ec4b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:19.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.619+0000 7f9b937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db6ec4adb0 0x55db6ec4b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36528/0 (socket says 172.21.15.7:36528) 2024-06-10T01:23:19.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.620+0000 7f9b937fe700 1 -- 172.21.15.7:0/3636756380 learned_addr learned my addr 172.21.15.7:0/3636756380 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:19.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.620+0000 7f9b927fc700 1 --2- 172.21.15.7:0/3636756380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db6ec46c40 0x55db6ec4a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:19.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.620+0000 7f9b937fe700 1 -- 172.21.15.7:0/3636756380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db6ec46c40 msgr2=0x55db6ec4a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:19.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.620+0000 7f9b937fe700 1 --2- 172.21.15.7:0/3636756380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db6ec46c40 0x55db6ec4a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.620+0000 7f9b937fe700 1 -- 172.21.15.7:0/3636756380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db6ec413d0 msgr2=0x55db6ec4d520 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:23:19.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.620+0000 7f9b937fe700 1 --2- 172.21.15.7:0/3636756380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db6ec413d0 0x55db6ec4d520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.620+0000 7f9b937fe700 1 -- 172.21.15.7:0/3636756380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db6eae19d0 con 0x55db6ec4adb0 2024-06-10T01:23:19.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.620+0000 7f9b937fe700 1 --2- 172.21.15.7:0/3636756380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db6ec4adb0 0x55db6ec4b210 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0a3d80 tx=0x7f9b8c0ccaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f02c01bbf1af217 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:19.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.621+0000 7f9b76ff5700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:23:19.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.621+0000 7f9b76ff5700 1 -- 172.21.15.7:0/3636756380 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c0862e0 con 0x55db6ec4adb0 2024-06-10T01:23:19.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.621+0000 7f9b76ff5700 1 -- 172.21.15.7:0/3636756380 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f9b8c0afb60 con 0x55db6ec4adb0 2024-06-10T01:23:19.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.621+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/3636756380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b84010910 con 0x55db6ec4adb0 2024-06-10T01:23:19.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.621+0000 7f9b76ff5700 1 -- 172.21.15.7:0/3636756380 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f9b8c0afd60 con 0x55db6ec4adb0 2024-06-10T01:23:19.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.621+0000 7f9ba3d55b80 10 client.19719 fetch_fsmap learned FSMap version 50 2024-06-10T01:23:19.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.621+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/3636756380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55db6ec58f40 con 0x55db6ec4adb0 2024-06-10T01:23:19.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.622+0000 7f9b76ff5700 1 -- 172.21.15.7:0/3636756380 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b84010910 con 0x55db6ec4adb0 2024-06-10T01:23:19.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.622+0000 7f9b76ff5700 1 -- 172.21.15.7:0/3636756380 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7f9b8c01f030 con 0x55db6ec4adb0 2024-06-10T01:23:19.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.622+0000 7f9ba3d55b80 10 client.19719 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:23:19.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.622+0000 7f9ba3d55b80 10 client.19719 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:23:19.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.622+0000 7f9ba3d55b80 20 client.19719 populate_metadata read hostname 'smithi007' 2024-06-10T01:23:19.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.622+0000 7f9ba3d55b80 1 --2- 172.21.15.7:0/3636756380 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55db6ec5b580 0x55db6ec5da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:19.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.623+0000 7f9ba3d55b80 4 client.19719 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:23:19.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.623+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/3636756380 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db6eb9fe10 con 0x55db6ec5b580 2024-06-10T01:23:19.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.625+0000 7f9b92ffd700 1 --2- 172.21.15.7:0/3636756380 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55db6ec5b580 0x55db6ec5da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:19.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.632+0000 7f9b76ff5700 10 client.19719 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:19.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.633+0000 7f9b92ffd700 1 --2- 172.21.15.7:0/3636756380 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55db6ec5b580 0x55db6ec5da20 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de792cedd1083b7d server_cookie=23642680faa7cbe4 in_seq=0 out_seq=0 2024-06-10T01:23:19.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.634+0000 7f9b76ff5700 1 -- 172.21.15.7:0/3636756380 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55db6eb9fe10 con 0x55db6ec5b580 2024-06-10T01:23:19.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.634+0000 7f9b76ff5700 10 client.19719 handle_command_reply: tid=0 2024-06-10T01:23:19.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.634+0000 7f9ba3d55b80 1 client.19719 shutdown 2024-06-10T01:23:19.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.634+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/3636756380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db6ec4adb0 msgr2=0x55db6ec4b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:19.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.634+0000 7f9ba3d55b80 1 --2- 172.21.15.7:0/3636756380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db6ec4adb0 0x55db6ec4b210 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0a3d80 tx=0x7f9b8c0ccaa0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.636+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/3636756380 shutdown_connections 2024-06-10T01:23:19.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.636+0000 7f9ba3d55b80 1 --2- 172.21.15.7:0/3636756380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db6ec4adb0 0x55db6ec4b210 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.636+0000 7f9ba3d55b80 1 --2- 172.21.15.7:0/3636756380 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55db6ec5b580 0x55db6ec5da20 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.636+0000 7f9ba3d55b80 1 --2- 172.21.15.7:0/3636756380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db6ec413d0 0x55db6ec4d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.636+0000 7f9ba3d55b80 1 --2- 172.21.15.7:0/3636756380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db6ec46c40 0x55db6ec4a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.636+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/3636756380 >> 172.21.15.7:0/3636756380 conn(0x55db6eb6bdc0 msgr2=0x55db6ec3fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:19.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.636+0000 7f9b76ff5700 0 client.19719 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:19.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.639+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/3636756380 shutdown_connections 2024-06-10T01:23:19.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.639+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/3636756380 wait complete. 2024-06-10T01:23:19.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.640+0000 7f9ba3d55b80 20 client.19719 trim_cache size 0 max 16384 2024-06-10T01:23:19.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.684+0000 7f9ba3d55b80 1 Processor -- start 2024-06-10T01:23:19.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.685+0000 7f9ba3d55b80 1 -- start start 2024-06-10T01:23:19.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.686+0000 7f9ba3d55b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db6ec413d0 0x55db6ec49130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:19.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.686+0000 7f9ba3d55b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db6ec46c40 0x55db6ec49670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:19.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.686+0000 7f9b92ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db6ec413d0 0x55db6ec49130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:19.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.687+0000 7f9b92ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db6ec413d0 0x55db6ec49130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:56852/0 (socket says 172.21.15.7:56852) 2024-06-10T01:23:19.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.687+0000 7f9b92ffd700 1 -- 172.21.15.7:0/3698337384 learned_addr learned my addr 172.21.15.7:0/3698337384 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:19.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.687+0000 7f9ba3d55b80 1 --2- 172.21.15.7:0/3698337384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db6ec4adb0 0x55db6ec44790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:19.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.687+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/3698337384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9b80011300 con 0x55db6ec4adb0 2024-06-10T01:23:19.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.687+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/3698337384 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8000eb90 con 0x55db6ec413d0 2024-06-10T01:23:19.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.687+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/3698337384 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8000c9e0 con 0x55db6ec46c40 2024-06-10T01:23:19.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.690+0000 7f9b937fe700 1 --2- 172.21.15.7:0/3698337384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db6ec4adb0 0x55db6ec44790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:19.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.690+0000 7f9b927fc700 1 --2- 172.21.15.7:0/3698337384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db6ec46c40 0x55db6ec49670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:19.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.690+0000 7f9b927fc700 1 -- 172.21.15.7:0/3698337384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db6ec413d0 msgr2=0x55db6ec49130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:19.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.690+0000 7f9b927fc700 1 --2- 172.21.15.7:0/3698337384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db6ec413d0 0x55db6ec49130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.690+0000 7f9b927fc700 1 -- 172.21.15.7:0/3698337384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db6ec4adb0 msgr2=0x55db6ec44790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:19.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.690+0000 7f9b927fc700 1 --2- 172.21.15.7:0/3698337384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db6ec4adb0 0x55db6ec44790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.690+0000 7f9b927fc700 1 -- 172.21.15.7:0/3698337384 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db6ec4de10 con 0x55db6ec46c40 2024-06-10T01:23:19.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.691+0000 7f9b937fe700 1 --2- 172.21.15.7:0/3698337384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db6ec4adb0 0x55db6ec44790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:23:19.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.693+0000 7f9b927fc700 1 --2- 172.21.15.7:0/3698337384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db6ec46c40 0x55db6ec49670 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9b88005b40 tx=0x7f9b8801d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fddd83d3bca69ce server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:19.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.693+0000 7f9b78ff9700 1 -- 172.21.15.7:0/3698337384 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b880056b0 con 0x55db6ec46c40 2024-06-10T01:23:19.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.693+0000 7f9b78ff9700 1 -- 172.21.15.7:0/3698337384 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b8800bcb0 con 0x55db6ec46c40 2024-06-10T01:23:19.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.693+0000 7f9b78ff9700 1 -- 172.21.15.7:0/3698337384 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8800fcd0 con 0x55db6ec46c40 2024-06-10T01:23:19.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.694+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/3698337384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db6ec46c40 msgr2=0x55db6ec49670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:19.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.695+0000 7f9ba3d55b80 1 --2- 172.21.15.7:0/3698337384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db6ec46c40 0x55db6ec49670 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9b88005b40 tx=0x7f9b8801d000 comp rx=0 tx=0).stop 2024-06-10T01:23:19.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.695+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/3698337384 shutdown_connections 2024-06-10T01:23:19.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.695+0000 7f9ba3d55b80 1 --2- 172.21.15.7:0/3698337384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db6ec4adb0 0x55db6ec44790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.695+0000 7f9ba3d55b80 1 --2- 172.21.15.7:0/3698337384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db6ec413d0 0x55db6ec49130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.695+0000 7f9ba3d55b80 1 --2- 172.21.15.7:0/3698337384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db6ec46c40 0x55db6ec49670 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.695+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/3698337384 >> 172.21.15.7:0/3698337384 conn(0x55db6eb6bdc0 msgr2=0x55db6ec3bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:19.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.696+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/3698337384 shutdown_connections 2024-06-10T01:23:19.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.697+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/3698337384 wait complete. 2024-06-10T01:23:19.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.697+0000 7f9ba3d55b80 1 -- start start 2024-06-10T01:23:19.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.697+0000 7f9ba3d55b80 1 Processor -- start 2024-06-10T01:23:19.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.698+0000 7f9ba3d55b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db6ec413d0 0x55db6ec53c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:19.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.698+0000 7f9ba3d55b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db6ec46c40 0x55db6ec62de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:19.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.698+0000 7f9b92ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db6ec413d0 0x55db6ec53c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:19.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.699+0000 7f9b92ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db6ec413d0 0x55db6ec53c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36536/0 (socket says 172.21.15.7:36536) 2024-06-10T01:23:19.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.699+0000 7f9b927fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db6ec46c40 0x55db6ec62de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:19.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.699+0000 7f9b927fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db6ec46c40 0x55db6ec62de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:56860/0 (socket says 172.21.15.7:56860) 2024-06-10T01:23:19.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.699+0000 7f9b92ffd700 1 -- 172.21.15.7:0/2243651860 learned_addr learned my addr 172.21.15.7:0/2243651860 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:19.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.699+0000 7f9ba3d55b80 1 --2- 172.21.15.7:0/2243651860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db6ec4adb0 0x55db6ec63380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:19.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.699+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/2243651860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8000d050 con 0x55db6ec413d0 2024-06-10T01:23:19.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.699+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/2243651860 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8000e4c0 con 0x55db6ec46c40 2024-06-10T01:23:19.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.699+0000 7f9b937fe700 1 --2- 172.21.15.7:0/2243651860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db6ec4adb0 0x55db6ec63380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:19.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.700+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/2243651860 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8000d720 con 0x55db6ec4adb0 2024-06-10T01:23:19.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.700+0000 7f9ba3d55b80 10 client.0 _ll_register_callbacks cb 0x55db6eb2ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:23:19.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.700+0000 7f9b927fc700 1 -- 172.21.15.7:0/2243651860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db6ec4adb0 msgr2=0x55db6ec63380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:19.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.700+0000 7f9b927fc700 1 --2- 172.21.15.7:0/2243651860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db6ec4adb0 0x55db6ec63380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.700+0000 7f9b927fc700 1 -- 172.21.15.7:0/2243651860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db6ec413d0 msgr2=0x55db6ec53c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:19.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.700+0000 7f9b927fc700 1 --2- 172.21.15.7:0/2243651860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db6ec413d0 0x55db6ec53c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.700+0000 7f9b927fc700 1 -- 172.21.15.7:0/2243651860 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db6ec63bf0 con 0x55db6ec46c40 2024-06-10T01:23:19.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.700+0000 7f9b937fe700 1 --2- 172.21.15.7:0/2243651860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db6ec4adb0 0x55db6ec63380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:23:19.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.701+0000 7f9b927fc700 1 --2- 172.21.15.7:0/2243651860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db6ec46c40 0x55db6ec62de0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x55db6ec58e90 tx=0x7f9b8802f9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fff63bb6fdc292c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:19.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.701+0000 7f9b76ff5700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:23:19.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.701+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/2243651860 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b8800bcb0 con 0x55db6ec46c40 2024-06-10T01:23:19.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.702+0000 7f9b76ff5700 1 -- 172.21.15.7:0/2243651860 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b880056b0 con 0x55db6ec46c40 2024-06-10T01:23:19.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.702+0000 7f9b76ff5700 1 -- 172.21.15.7:0/2243651860 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f9b8800f460 con 0x55db6ec46c40 2024-06-10T01:23:19.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.704+0000 7f9b76ff5700 1 -- 172.21.15.7:0/2243651860 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f9b88040050 con 0x55db6ec46c40 2024-06-10T01:23:19.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.705+0000 7f9ba3d55b80 10 client.28943 fetch_fsmap learned FSMap version 50 2024-06-10T01:23:19.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.705+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/2243651860 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=50}) -- 0x55db6ecc62d0 con 0x55db6ec46c40 2024-06-10T01:23:19.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.706+0000 7f9b76ff5700 1 -- 172.21.15.7:0/2243651860 <== mon.2 v2:172.21.15.78:3300/0 4 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7f9b8802d070 con 0x55db6ec46c40 2024-06-10T01:23:19.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.706+0000 7f9ba3d55b80 10 client.28943 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:23:19.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.706+0000 7f9ba3d55b80 10 client.28943 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:23:19.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.706+0000 7f9ba3d55b80 20 client.28943 populate_metadata read hostname 'smithi007' 2024-06-10T01:23:19.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.706+0000 7f9ba3d55b80 1 --2- 172.21.15.7:0/2243651860 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55db6ecc67b0 0x55db6ecc8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:19.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.706+0000 7f9ba3d55b80 4 client.28943 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f064d2e", "release": true, "await": true}] 2024-06-10T01:23:19.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.706+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/2243651860 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f064d2e", "release": true, "await": true}) -- 0x55db6eb9a390 con 0x55db6ecc67b0 2024-06-10T01:23:19.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.706+0000 7f9b92ffd700 1 --2- 172.21.15.7:0/2243651860 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55db6ecc67b0 0x55db6ecc8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:19.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.707+0000 7f9b92ffd700 1 --2- 172.21.15.7:0/2243651860 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55db6ecc67b0 0x55db6ecc8bd0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d35a47584aa5c23 server_cookie=6b6fe20a4f042b56 in_seq=0 out_seq=0 2024-06-10T01:23:19.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.707+0000 7f9b76ff5700 10 client.28943 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:19.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.761+0000 7f9b76ff5700 1 -- 172.21.15.7:0/2243651860 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55db6eb9a390 con 0x55db6ecc67b0 2024-06-10T01:23:19.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.761+0000 7f9b76ff5700 10 client.28943 handle_command_reply: tid=0 2024-06-10T01:23:19.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.761+0000 7f9ba3d55b80 1 client.28943 shutdown 2024-06-10T01:23:19.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.762+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/2243651860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db6ec46c40 msgr2=0x55db6ec62de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:19.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.762+0000 7f9ba3d55b80 1 --2- 172.21.15.7:0/2243651860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db6ec46c40 0x55db6ec62de0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x55db6ec58e90 tx=0x7f9b8802f9e0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.762+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/2243651860 shutdown_connections 2024-06-10T01:23:19.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.762+0000 7f9ba3d55b80 1 --2- 172.21.15.7:0/2243651860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db6ec4adb0 0x55db6ec63380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.762+0000 7f9ba3d55b80 1 --2- 172.21.15.7:0/2243651860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db6ec413d0 0x55db6ec53c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.762+0000 7f9ba3d55b80 1 --2- 172.21.15.7:0/2243651860 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55db6ecc67b0 0x55db6ecc8bd0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.762+0000 7f9ba3d55b80 1 --2- 172.21.15.7:0/2243651860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db6ec46c40 0x55db6ec62de0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.762+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/2243651860 >> 172.21.15.7:0/2243651860 conn(0x55db6eb6bdc0 msgr2=0x55db6ec490f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:19.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.762+0000 7f9b76ff5700 0 client.28943 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:19.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.762+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/2243651860 shutdown_connections 2024-06-10T01:23:19.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.762+0000 7f9ba3d55b80 1 -- 172.21.15.7:0/2243651860 wait complete. 2024-06-10T01:23:19.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.762+0000 7f9ba3d55b80 20 client.28943 trim_cache size 0 max 16384 2024-06-10T01:23:19.756 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:23:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 50, 2024-06-10T01:23:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:23:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 56, 2024-06-10T01:23:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:23:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "f064d2e": { 2024-06-10T01:23:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 56, 2024-06-10T01:23:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:23:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:23:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:23:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:23:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:23:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:23:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 150.0, 2024-06-10T01:23:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:23:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:23:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:23:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:23:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:23:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:23:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:23:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:23:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:23:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:23:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:23:19.758 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:23:19.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.763+0000 7f9b994d2700 1 -- 172.21.15.7:0/444532150 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9b7c0a2d70 msgr2=0x7f9b7c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:19.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.763+0000 7f9b994d2700 1 --2- 172.21.15.7:0/444532150 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9b7c0a2d70 0x7f9b7c0a5230 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800b5c0 tx=0x7f9b88005a90 comp rx=0 tx=0).stop 2024-06-10T01:23:19.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.763+0000 7f9b994d2700 1 -- 172.21.15.7:0/444532150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b9410eab0 msgr2=0x7f9b941a3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:19.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.763+0000 7f9b994d2700 1 --2- 172.21.15.7:0/444532150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b9410eab0 0x7f9b941a3970 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00b780 tx=0x7f9b8c00bb40 comp rx=0 tx=0).stop 2024-06-10T01:23:19.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.763+0000 7f9b994d2700 1 -- 172.21.15.7:0/444532150 shutdown_connections 2024-06-10T01:23:19.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.763+0000 7f9b994d2700 1 --2- 172.21.15.7:0/444532150 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9b7c0a2d70 0x7f9b7c0a5230 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.763+0000 7f9b994d2700 1 --2- 172.21.15.7:0/444532150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b9410eab0 0x7f9b941a3970 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.763+0000 7f9b994d2700 1 --2- 172.21.15.7:0/444532150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b94108b00 0x7f9b9419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.763+0000 7f9b994d2700 1 --2- 172.21.15.7:0/444532150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b941094c0 0x7f9b9419f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:19.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.763+0000 7f9b994d2700 1 -- 172.21.15.7:0/444532150 >> 172.21.15.7:0/444532150 conn(0x7f9b94076a50 msgr2=0x7f9b94076e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:19.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.763+0000 7f9b994d2700 1 -- 172.21.15.7:0/444532150 shutdown_connections 2024-06-10T01:23:19.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:19.763+0000 7f9b994d2700 1 -- 172.21.15.7:0/444532150 wait complete. 2024-06-10T01:23:20.532 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92697 23.99 MB/sec execute 117 sec latency 32315.740 ms 2024-06-10T01:23:20.703 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f064d2e', total seconds elapsed: 41.5 2024-06-10T01:23:20.703 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-06-10T01:23:20.704 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:23:21.084 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:23:21.408 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:21 smithi007 ceph-mon[32398]: pgmap v953: 129 pgs: 129 active+clean; 558 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 5.2 MiB/s wr, 303 op/s 2024-06-10T01:23:21.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:21 smithi175 ceph-mon[36636]: pgmap v953: 129 pgs: 129 active+clean; 558 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 5.2 MiB/s wr, 303 op/s 2024-06-10T01:23:21.532 INFO:tasks.workunit.client.0.smithi007.stdout: 10 93208 24.00 MB/sec execute 118 sec latency 90.108 ms 2024-06-10T01:23:21.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:21 smithi078 ceph-mon[36683]: pgmap v953: 129 pgs: 129 active+clean; 558 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 5.2 MiB/s wr, 303 op/s 2024-06-10T01:23:22.532 INFO:tasks.workunit.client.0.smithi007.stdout: 10 94051 24.13 MB/sec execute 119 sec latency 91.511 ms 2024-06-10T01:23:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.902+0000 7f82a5fb0640 1 -- 172.21.15.7:0/107876071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82a01036a0 msgr2=0x7f82a0109f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.902+0000 7f82a5fb0640 1 --2- 172.21.15.7:0/107876071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82a01036a0 0x7f82a0109f30 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f8298009cd0 tx=0x7f829802a8e0 comp rx=0 tx=0).stop 2024-06-10T01:23:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.902+0000 7f82a5fb0640 1 -- 172.21.15.7:0/107876071 shutdown_connections 2024-06-10T01:23:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.902+0000 7f82a5fb0640 1 --2- 172.21.15.7:0/107876071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82a01036a0 0x7f82a0109f30 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.902+0000 7f82a5fb0640 1 --2- 172.21.15.7:0/107876071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82a0102ce0 0x7f82a0103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.902+0000 7f82a5fb0640 1 --2- 172.21.15.7:0/107876071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82a00ffd10 0x7f82a0100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.902+0000 7f82a5fb0640 1 -- 172.21.15.7:0/107876071 >> 172.21.15.7:0/107876071 conn(0x7f82a00fb640 msgr2=0x7f82a00fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.903+0000 7f82a5fb0640 1 -- 172.21.15.7:0/107876071 shutdown_connections 2024-06-10T01:23:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.903+0000 7f82a5fb0640 1 -- 172.21.15.7:0/107876071 wait complete. 2024-06-10T01:23:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.904+0000 7f82a5fb0640 1 Processor -- start 2024-06-10T01:23:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.904+0000 7f82a5fb0640 1 -- start start 2024-06-10T01:23:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.904+0000 7f82a5fb0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82a00ffd10 0x7f82a0071790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.904+0000 7f82a5fb0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82a0102ce0 0x7f82a0071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.904+0000 7f82a5fb0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82a01036a0 0x7f82a0072ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.904+0000 7f82a5fb0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f82a006be10 con 0x7f82a00ffd10 2024-06-10T01:23:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.904+0000 7f82a5fb0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f82a006bc90 con 0x7f82a01036a0 2024-06-10T01:23:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.904+0000 7f829effd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82a0102ce0 0x7f82a0071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.905+0000 7f82a5fb0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f82a006bf90 con 0x7f82a0102ce0 2024-06-10T01:23:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.905+0000 7f829effd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82a0102ce0 0x7f82a0071cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:57408/0 (socket says 172.21.15.7:57408) 2024-06-10T01:23:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.905+0000 7f829effd640 1 -- 172.21.15.7:0/157691310 learned_addr learned my addr 172.21.15.7:0/157691310 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.905+0000 7f829f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82a00ffd10 0x7f82a0071790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.905+0000 7f829ffff640 1 --2- 172.21.15.7:0/157691310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82a01036a0 0x7f82a0072ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.905+0000 7f829f7fe640 1 -- 172.21.15.7:0/157691310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82a0102ce0 msgr2=0x7f82a0071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.905+0000 7f829f7fe640 1 --2- 172.21.15.7:0/157691310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82a0102ce0 0x7f82a0071cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.905+0000 7f829f7fe640 1 -- 172.21.15.7:0/157691310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82a01036a0 msgr2=0x7f82a0072ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.905+0000 7f829f7fe640 1 --2- 172.21.15.7:0/157691310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82a01036a0 0x7f82a0072ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.905+0000 7f829f7fe640 1 -- 172.21.15.7:0/157691310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a00732d0 con 0x7f82a00ffd10 2024-06-10T01:23:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.905+0000 7f829f7fe640 1 --2- 172.21.15.7:0/157691310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82a00ffd10 0x7f82a0071790 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f8290009e60 tx=0x7f829000ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.905+0000 7f829cff9640 1 -- 172.21.15.7:0/157691310 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829000ccc0 con 0x7f82a00ffd10 2024-06-10T01:23:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.906+0000 7f82a5fb0640 1 -- 172.21.15.7:0/157691310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82a00735c0 con 0x7f82a00ffd10 2024-06-10T01:23:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.906+0000 7f82a5fb0640 1 -- 172.21.15.7:0/157691310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82a0100ad0 con 0x7f82a00ffd10 2024-06-10T01:23:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.906+0000 7f829cff9640 1 -- 172.21.15.7:0/157691310 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82900137d0 con 0x7f82a00ffd10 2024-06-10T01:23:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.906+0000 7f829cff9640 1 -- 172.21.15.7:0/157691310 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8290003850 con 0x7f82a00ffd10 2024-06-10T01:23:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.908+0000 7f82a8d9d740 1 Processor -- start 2024-06-10T01:23:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.908+0000 7f82a8d9d740 1 -- start start 2024-06-10T01:23:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.909+0000 7f82a8d9d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563af2506010 0x563af2506430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.909+0000 7f82a8d9d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af2506a00 0x563af250b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.909+0000 7f82a8d9d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563af250bb70 0x563af250e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.909+0000 7f82a8d9d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563af24694f0 con 0x563af250bb70 2024-06-10T01:23:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.909+0000 7f82a8d9d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563af2407870 con 0x563af2506010 2024-06-10T01:23:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.909+0000 7f82a8d9d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563af24654b0 con 0x563af2506a00 2024-06-10T01:23:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.909+0000 7f829cff9640 1 -- 172.21.15.7:0/157691310 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f8290022020 con 0x7f82a00ffd10 2024-06-10T01:23:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.909+0000 7f829ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563af2506010 0x563af2506430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.909+0000 7f829effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563af250bb70 0x563af250e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.909+0000 7f829ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563af2506010 0x563af2506430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:45260/0 (socket says 172.21.15.7:45260) 2024-06-10T01:23:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.909+0000 7f829ffff640 1 -- 172.21.15.7:0/142502603 learned_addr learned my addr 172.21.15.7:0/142502603 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.909+0000 7f829f7fe640 1 --2- 172.21.15.7:0/142502603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af2506a00 0x563af250b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.909+0000 7f829ffff640 1 -- 172.21.15.7:0/142502603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af2506a00 msgr2=0x563af250b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.909+0000 7f829ffff640 1 --2- 172.21.15.7:0/142502603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af2506a00 0x563af250b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.909+0000 7f829ffff640 1 -- 172.21.15.7:0/142502603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563af250bb70 msgr2=0x563af250e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.909+0000 7f829ffff640 1 --2- 172.21.15.7:0/142502603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563af250bb70 0x563af250e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.909+0000 7f829ffff640 1 -- 172.21.15.7:0/142502603 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563af24239b0 con 0x563af2506010 2024-06-10T01:23:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.910+0000 7f829ffff640 1 --2- 172.21.15.7:0/142502603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563af2506010 0x563af2506430 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f829802aed0 tx=0x7f8298009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54bf240cceb47049 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.910+0000 7f8285ffb640 1 -- 172.21.15.7:0/142502603 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8298049070 con 0x563af2506010 2024-06-10T01:23:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.910+0000 7f8285ffb640 1 -- 172.21.15.7:0/142502603 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8298036040 con 0x563af2506010 2024-06-10T01:23:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.910+0000 7f829cff9640 1 --2- 172.21.15.7:0/157691310 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f82880a2c00 0x7f82880a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.910+0000 7f829cff9640 1 -- 172.21.15.7:0/157691310 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f82900d3470 con 0x7f82a00ffd10 2024-06-10T01:23:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.910+0000 7f8285ffb640 1 -- 172.21.15.7:0/142502603 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82980042a0 con 0x563af2506010 2024-06-10T01:23:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.910+0000 7f829f7fe640 1 --2- 172.21.15.7:0/157691310 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f82880a2c00 0x7f82880a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.911+0000 7f82a8d9d740 1 -- 172.21.15.7:0/142502603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563af2506010 msgr2=0x563af2506430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.911+0000 7f82a8d9d740 1 --2- 172.21.15.7:0/142502603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563af2506010 0x563af2506430 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f829802aed0 tx=0x7f8298009a20 comp rx=0 tx=0).stop 2024-06-10T01:23:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.911+0000 7f829f7fe640 1 --2- 172.21.15.7:0/157691310 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f82880a2c00 0x7f82880a50c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x563af2506e00 tx=0x7f8290000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.911+0000 7f82a8d9d740 1 -- 172.21.15.7:0/142502603 shutdown_connections 2024-06-10T01:23:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.911+0000 7f82a8d9d740 1 --2- 172.21.15.7:0/142502603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563af250bb70 0x563af250e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.911+0000 7f82a8d9d740 1 --2- 172.21.15.7:0/142502603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af2506a00 0x563af250b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.911+0000 7f82a8d9d740 1 --2- 172.21.15.7:0/142502603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563af2506010 0x563af2506430 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.911+0000 7f82a8d9d740 1 -- 172.21.15.7:0/142502603 >> 172.21.15.7:0/142502603 conn(0x563af2501680 msgr2=0x563af2501a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.911+0000 7f82a8d9d740 1 -- 172.21.15.7:0/142502603 shutdown_connections 2024-06-10T01:23:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.911+0000 7f82a8d9d740 1 -- 172.21.15.7:0/142502603 wait complete. 2024-06-10T01:23:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.911+0000 7f82a8d9d740 1 -- start start 2024-06-10T01:23:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.912+0000 7f82a8d9d740 1 Processor -- start 2024-06-10T01:23:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.912+0000 7f82a8d9d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563af2506010 0x563af2510300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.912+0000 7f82a8d9d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563af2506a00 0x563af2510840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.912+0000 7f82a8d9d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af250bb70 0x563af2517810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.912+0000 7f829ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563af2506010 0x563af2510300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.912+0000 7f829ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563af2506010 0x563af2510300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:45266/0 (socket says 172.21.15.7:45266) 2024-06-10T01:23:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.913+0000 7f829ffff640 1 -- 172.21.15.7:0/1020955172 learned_addr learned my addr 172.21.15.7:0/1020955172 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.913+0000 7f82a8d9d740 1 -- 172.21.15.7:0/1020955172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563af250eb10 con 0x563af2506a00 2024-06-10T01:23:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.913+0000 7f82a8d9d740 1 -- 172.21.15.7:0/1020955172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563af23f2ae0 con 0x563af2506010 2024-06-10T01:23:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.913+0000 7f82a8d9d740 1 -- 172.21.15.7:0/1020955172 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563af250ec90 con 0x563af250bb70 2024-06-10T01:23:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.913+0000 7f82a8d9d740 10 client.0 _ll_register_callbacks cb 0x563af242b9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:23:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.913+0000 7f829effd640 1 --2- 172.21.15.7:0/1020955172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563af2506a00 0x563af2510840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.913+0000 7f829ffff640 1 --2- 172.21.15.7:0/1020955172 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af250bb70 0x563af2517810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.913+0000 7f829ffff640 1 -- 172.21.15.7:0/1020955172 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af250bb70 msgr2=0x563af2517810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.913+0000 7f829ffff640 1 --2- 172.21.15.7:0/1020955172 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af250bb70 0x563af2517810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.913+0000 7f829ffff640 1 -- 172.21.15.7:0/1020955172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563af2506a00 msgr2=0x563af2510840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.913+0000 7f829ffff640 1 --2- 172.21.15.7:0/1020955172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563af2506a00 0x563af2510840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.913+0000 7f829ffff640 1 -- 172.21.15.7:0/1020955172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563af217d210 con 0x563af2506010 2024-06-10T01:23:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.913+0000 7f829effd640 1 --2- 172.21.15.7:0/1020955172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563af2506a00 0x563af2510840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:23:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.913+0000 7f829ffff640 1 --2- 172.21.15.7:0/1020955172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563af2506010 0x563af2510300 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8298047040 tx=0x7f829802a8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52dd5eadbb71c3e5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.913+0000 7f827effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:23:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.913+0000 7f827effd640 1 -- 172.21.15.7:0/1020955172 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8298055060 con 0x563af2506010 2024-06-10T01:23:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.914+0000 7f82a8d9d740 1 -- 172.21.15.7:0/1020955172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8298036040 con 0x563af2506010 2024-06-10T01:23:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.914+0000 7f827effd640 1 -- 172.21.15.7:0/1020955172 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f8298041040 con 0x563af2506010 2024-06-10T01:23:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.914+0000 7f827effd640 1 -- 172.21.15.7:0/1020955172 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f829805c050 con 0x563af2506010 2024-06-10T01:23:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.914+0000 7f82a8d9d740 10 client.28955 fetch_fsmap learned FSMap version 50 2024-06-10T01:23:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.914+0000 7f82a8d9d740 1 -- 172.21.15.7:0/1020955172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x563af2249620 con 0x563af2506010 2024-06-10T01:23:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.914+0000 7f827effd640 1 -- 172.21.15.7:0/1020955172 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8298036040 con 0x563af2506010 2024-06-10T01:23:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.914+0000 7f827effd640 1 -- 172.21.15.7:0/1020955172 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7f829804f070 con 0x563af2506010 2024-06-10T01:23:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.914+0000 7f82a8d9d740 10 client.28955 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:23:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.914+0000 7f82a8d9d740 10 client.28955 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:23:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.914+0000 7f82a8d9d740 20 client.28955 populate_metadata read hostname 'smithi007' 2024-06-10T01:23:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.914+0000 7f82a8d9d740 1 --2- 172.21.15.7:0/1020955172 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563af251bff0 0x563af251e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.914+0000 7f82a8d9d740 4 client.28955 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:23:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.914+0000 7f82a8d9d740 1 -- 172.21.15.7:0/1020955172 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563af247cf10 con 0x563af251bff0 2024-06-10T01:23:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.914+0000 7f829effd640 1 --2- 172.21.15.7:0/1020955172 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563af251bff0 0x563af251e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.915+0000 7f827effd640 10 client.28955 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.915+0000 7f829effd640 1 --2- 172.21.15.7:0/1020955172 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563af251bff0 0x563af251e410 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a265c06602e08341 server_cookie=15d4972b98cf4d1e in_seq=0 out_seq=0 2024-06-10T01:23:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.918+0000 7f827effd640 1 -- 172.21.15.7:0/1020955172 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563af247cf10 con 0x563af251bff0 2024-06-10T01:23:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.918+0000 7f827effd640 10 client.28955 handle_command_reply: tid=0 2024-06-10T01:23:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.918+0000 7f82a8d9d740 1 client.28955 shutdown 2024-06-10T01:23:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.918+0000 7f82a8d9d740 1 -- 172.21.15.7:0/1020955172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563af2506010 msgr2=0x563af2510300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.918+0000 7f82a8d9d740 1 --2- 172.21.15.7:0/1020955172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563af2506010 0x563af2510300 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8298047040 tx=0x7f829802a8e0 comp rx=0 tx=0).stop 2024-06-10T01:23:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.918+0000 7f82a8d9d740 1 -- 172.21.15.7:0/1020955172 shutdown_connections 2024-06-10T01:23:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.918+0000 7f82a8d9d740 1 --2- 172.21.15.7:0/1020955172 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563af251bff0 0x563af251e410 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.918+0000 7f82a8d9d740 1 --2- 172.21.15.7:0/1020955172 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af250bb70 0x563af2517810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.918+0000 7f82a8d9d740 1 --2- 172.21.15.7:0/1020955172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563af2506a00 0x563af2510840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.918+0000 7f82a8d9d740 1 --2- 172.21.15.7:0/1020955172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563af2506010 0x563af2510300 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.918+0000 7f82a8d9d740 1 -- 172.21.15.7:0/1020955172 >> 172.21.15.7:0/1020955172 conn(0x563af2501680 msgr2=0x563af25007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.918+0000 7f827effd640 0 client.28955 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.918+0000 7f82a8d9d740 1 -- 172.21.15.7:0/1020955172 shutdown_connections 2024-06-10T01:23:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.918+0000 7f82a8d9d740 1 -- 172.21.15.7:0/1020955172 wait complete. 2024-06-10T01:23:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.918+0000 7f82a8d9d740 20 client.28955 trim_cache size 0 max 16384 2024-06-10T01:23:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.942+0000 7f82a8d9d740 1 Processor -- start 2024-06-10T01:23:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.943+0000 7f82a8d9d740 1 -- start start 2024-06-10T01:23:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.943+0000 7f82a8d9d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563af2506010 0x563af24fc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.943+0000 7f82a8d9d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563af2506a00 0x563af25177a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.943+0000 7f82a8d9d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af250bb70 0x563af2507c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.943+0000 7f82a8d9d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f828c012410 con 0x563af2506a00 2024-06-10T01:23:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.943+0000 7f82a8d9d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f828c00fce0 con 0x563af2506010 2024-06-10T01:23:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.943+0000 7f82a8d9d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f828c00db30 con 0x563af250bb70 2024-06-10T01:23:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.943+0000 7f829ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563af2506010 0x563af24fc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.943+0000 7f829ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563af2506010 0x563af24fc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:45272/0 (socket says 172.21.15.7:45272) 2024-06-10T01:23:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.943+0000 7f829effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563af2506a00 0x563af25177a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.943+0000 7f829ffff640 1 -- 172.21.15.7:0/4096795275 learned_addr learned my addr 172.21.15.7:0/4096795275 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.943+0000 7f829ffff640 1 --2- 172.21.15.7:0/4096795275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af250bb70 0x563af2507c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.943+0000 7f829ffff640 1 -- 172.21.15.7:0/4096795275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af250bb70 msgr2=0x563af2507c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.943+0000 7f829ffff640 1 --2- 172.21.15.7:0/4096795275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af250bb70 0x563af2507c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.943+0000 7f829ffff640 1 -- 172.21.15.7:0/4096795275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563af2506a00 msgr2=0x563af25177a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.943+0000 7f829ffff640 1 --2- 172.21.15.7:0/4096795275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563af2506a00 0x563af25177a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.944+0000 7f829ffff640 1 -- 172.21.15.7:0/4096795275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563af25014b0 con 0x563af2506010 2024-06-10T01:23:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.944+0000 7f829ffff640 1 --2- 172.21.15.7:0/4096795275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563af2506010 0x563af24fc560 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f829802aed0 tx=0x7f8298009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89d8b6353de42342 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.944+0000 7f82857fa640 1 -- 172.21.15.7:0/4096795275 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8298055070 con 0x563af2506010 2024-06-10T01:23:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.944+0000 7f82857fa640 1 -- 172.21.15.7:0/4096795275 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8298036040 con 0x563af2506010 2024-06-10T01:23:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.944+0000 7f82857fa640 1 -- 172.21.15.7:0/4096795275 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8298049040 con 0x563af2506010 2024-06-10T01:23:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.945+0000 7f82a8d9d740 1 -- 172.21.15.7:0/4096795275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563af2506010 msgr2=0x563af24fc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.945+0000 7f82a8d9d740 1 --2- 172.21.15.7:0/4096795275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563af2506010 0x563af24fc560 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f829802aed0 tx=0x7f8298009a20 comp rx=0 tx=0).stop 2024-06-10T01:23:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.945+0000 7f82a8d9d740 1 -- 172.21.15.7:0/4096795275 shutdown_connections 2024-06-10T01:23:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.945+0000 7f82a8d9d740 1 --2- 172.21.15.7:0/4096795275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af250bb70 0x563af2507c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.945+0000 7f82a8d9d740 1 --2- 172.21.15.7:0/4096795275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563af2506a00 0x563af25177a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.945+0000 7f82a8d9d740 1 --2- 172.21.15.7:0/4096795275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563af2506010 0x563af24fc560 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.945+0000 7f82a8d9d740 1 -- 172.21.15.7:0/4096795275 >> 172.21.15.7:0/4096795275 conn(0x563af2501680 msgr2=0x563af250b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.945+0000 7f82a8d9d740 1 -- 172.21.15.7:0/4096795275 shutdown_connections 2024-06-10T01:23:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.945+0000 7f82a8d9d740 1 -- 172.21.15.7:0/4096795275 wait complete. 2024-06-10T01:23:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.945+0000 7f82a8d9d740 1 -- start start 2024-06-10T01:23:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.946+0000 7f82a8d9d740 1 Processor -- start 2024-06-10T01:23:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.946+0000 7f82a8d9d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af2506010 0x563af25170b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.946+0000 7f82a8d9d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563af2506a00 0x563af25175f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.946+0000 7f82a8d9d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563af250bb70 0x563af2513ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.946+0000 7f82a8d9d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f828c00e1a0 con 0x563af250bb70 2024-06-10T01:23:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.946+0000 7f82a8d9d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f828c00f610 con 0x563af2506a00 2024-06-10T01:23:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.946+0000 7f82a8d9d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f828c00e870 con 0x563af2506010 2024-06-10T01:23:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.946+0000 7f82a8d9d740 10 client.0 _ll_register_callbacks cb 0x7f8298047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:23:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.946+0000 7f829ffff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af2506010 0x563af25170b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.946+0000 7f829ffff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af2506010 0x563af25170b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:57450/0 (socket says 172.21.15.7:57450) 2024-06-10T01:23:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.946+0000 7f829ffff640 1 -- 172.21.15.7:0/1382506163 learned_addr learned my addr 172.21.15.7:0/1382506163 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.946+0000 7f829ffff640 1 --2- 172.21.15.7:0/1382506163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563af250bb70 0x563af2513ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.947+0000 7f829effd640 1 --2- 172.21.15.7:0/1382506163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563af2506a00 0x563af25175f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.947+0000 7f829ffff640 1 -- 172.21.15.7:0/1382506163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563af2506a00 msgr2=0x563af25175f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.947+0000 7f829ffff640 1 --2- 172.21.15.7:0/1382506163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563af2506a00 0x563af25175f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.947+0000 7f829ffff640 1 -- 172.21.15.7:0/1382506163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563af250bb70 msgr2=0x563af2513ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.947+0000 7f829ffff640 1 --2- 172.21.15.7:0/1382506163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563af250bb70 0x563af2513ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.947+0000 7f829ffff640 1 -- 172.21.15.7:0/1382506163 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563af2514310 con 0x563af2506010 2024-06-10T01:23:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.947+0000 7f829effd640 1 --2- 172.21.15.7:0/1382506163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563af2506a00 0x563af25175f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:23:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.947+0000 7f829ffff640 1 --2- 172.21.15.7:0/1382506163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af2506010 0x563af25170b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8298005e30 tx=0x7f8298000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=731608df0aea6bc1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.947+0000 7f827effd640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:23:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.947+0000 7f827effd640 1 -- 172.21.15.7:0/1382506163 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8298055070 con 0x563af2506010 2024-06-10T01:23:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.948+0000 7f827effd640 1 -- 172.21.15.7:0/1382506163 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f829802e050 con 0x563af2506010 2024-06-10T01:23:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.948+0000 7f82a8d9d740 1 -- 172.21.15.7:0/1382506163 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8298036040 con 0x563af2506010 2024-06-10T01:23:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.948+0000 7f827effd640 1 -- 172.21.15.7:0/1382506163 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f8298049070 con 0x563af2506010 2024-06-10T01:23:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.948+0000 7f82a8d9d740 10 client.28939 fetch_fsmap learned FSMap version 50 2024-06-10T01:23:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.948+0000 7f82a8d9d740 1 -- 172.21.15.7:0/1382506163 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x563af258e1b0 con 0x563af2506010 2024-06-10T01:23:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.948+0000 7f827effd640 1 -- 172.21.15.7:0/1382506163 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8298036040 con 0x563af2506010 2024-06-10T01:23:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.948+0000 7f827effd640 1 -- 172.21.15.7:0/1382506163 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7f829804f070 con 0x563af2506010 2024-06-10T01:23:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.948+0000 7f82a8d9d740 10 client.28939 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:23:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.948+0000 7f82a8d9d740 10 client.28939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:23:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.948+0000 7f82a8d9d740 20 client.28939 populate_metadata read hostname 'smithi007' 2024-06-10T01:23:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.948+0000 7f82a8d9d740 1 --2- 172.21.15.7:0/1382506163 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563af258ea20 0x563af2590e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.948+0000 7f82a8d9d740 4 client.28939 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:23:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.948+0000 7f82a8d9d740 1 -- 172.21.15.7:0/1382506163 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563af2476c20 con 0x563af258ea20 2024-06-10T01:23:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.949+0000 7f829effd640 1 --2- 172.21.15.7:0/1382506163 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563af258ea20 0x563af2590e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.949+0000 7f829effd640 1 --2- 172.21.15.7:0/1382506163 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563af258ea20 0x563af2590e40 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=914f38948adcb905 server_cookie=77b87914814a4079 in_seq=0 out_seq=0 2024-06-10T01:23:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:22.949+0000 7f827effd640 10 client.28939 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:23.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.218+0000 7f27c09b0700 1 -- 172.21.15.7:0/4201242073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27bc100870 msgr2=0x7f27bc100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:23.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.218+0000 7f27c09b0700 1 --2- 172.21.15.7:0/4201242073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27bc100870 0x7f27bc100c50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f27ac009990 tx=0x7f27ac009ca0 comp rx=0 tx=0).stop 2024-06-10T01:23:23.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.218+0000 7f27c09b0700 1 -- 172.21.15.7:0/4201242073 shutdown_connections 2024-06-10T01:23:23.212 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:23 smithi007 ceph-mon[32398]: pgmap v954: 129 pgs: 129 active+clean; 560 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 4.5 MiB/s wr, 274 op/s 2024-06-10T01:23:23.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.218+0000 7f27c09b0700 1 --2- 172.21.15.7:0/4201242073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27bc10ec80 0x7f27bc111120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:23.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.218+0000 7f27c09b0700 1 --2- 172.21.15.7:0/4201242073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27bc100870 0x7f27bc100c50 secure :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f27ac009990 tx=0x7f27ac009ca0 comp rx=0 tx=0).stop 2024-06-10T01:23:23.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.218+0000 7f27c09b0700 1 --2- 172.21.15.7:0/4201242073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27bc101190 0x7f27bc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:23.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.218+0000 7f27c09b0700 1 -- 172.21.15.7:0/4201242073 >> 172.21.15.7:0/4201242073 conn(0x7f27bc077590 msgr2=0x7f27bc0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:23.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.218+0000 7f27c09b0700 1 -- 172.21.15.7:0/4201242073 shutdown_connections 2024-06-10T01:23:23.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.218+0000 7f27c09b0700 1 -- 172.21.15.7:0/4201242073 wait complete. 2024-06-10T01:23:23.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.220+0000 7f27c09b0700 1 Processor -- start 2024-06-10T01:23:23.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.220+0000 7f27c09b0700 1 -- start start 2024-06-10T01:23:23.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.220+0000 7f27c09b0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27bc101190 0x7f27bc19b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:23.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.220+0000 7f27c09b0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27bc10ec80 0x7f27bc19b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:23.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.220+0000 7f27c09b0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27bc19f920 0x7f27bc19fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:23.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.220+0000 7f27c09b0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc114a00 con 0x7f27bc10ec80 2024-06-10T01:23:23.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.220+0000 7f27c09b0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc114880 con 0x7f27bc19f920 2024-06-10T01:23:23.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.221+0000 7f27c09b0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc114b80 con 0x7f27bc101190 2024-06-10T01:23:23.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.221+0000 7f27bad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27bc19f920 0x7f27bc19fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:23.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.221+0000 7f27bad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27bc19f920 0x7f27bc19fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:45298/0 (socket says 172.21.15.7:45298) 2024-06-10T01:23:23.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.221+0000 7f27bad9d700 1 -- 172.21.15.7:0/2504122743 learned_addr learned my addr 172.21.15.7:0/2504122743 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:23.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.221+0000 7f27ba59c700 1 --2- 172.21.15.7:0/2504122743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27bc101190 0x7f27bc19b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:23.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.221+0000 7f27b9d9b700 1 --2- 172.21.15.7:0/2504122743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27bc10ec80 0x7f27bc19b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:23.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.221+0000 7f27bad9d700 1 -- 172.21.15.7:0/2504122743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27bc101190 msgr2=0x7f27bc19b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:23.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.221+0000 7f27bad9d700 1 --2- 172.21.15.7:0/2504122743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27bc101190 0x7f27bc19b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:23.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.221+0000 7f27bad9d700 1 -- 172.21.15.7:0/2504122743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27bc10ec80 msgr2=0x7f27bc19b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:23.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.221+0000 7f27bad9d700 1 --2- 172.21.15.7:0/2504122743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27bc10ec80 0x7f27bc19b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:23.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.221+0000 7f27bad9d700 1 -- 172.21.15.7:0/2504122743 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27bc1a0440 con 0x7f27bc19f920 2024-06-10T01:23:23.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.221+0000 7f27bad9d700 1 --2- 172.21.15.7:0/2504122743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27bc19f920 0x7f27bc19fda0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f27b400ef30 tx=0x7f27b400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:23.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.221+0000 7f27ab7fe700 1 -- 172.21.15.7:0/2504122743 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b40092e0 con 0x7f27bc19f920 2024-06-10T01:23:23.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.222+0000 7f27c09b0700 1 -- 172.21.15.7:0/2504122743 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27bc1a06d0 con 0x7f27bc19f920 2024-06-10T01:23:23.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.222+0000 7f27b9d9b700 1 --2- 172.21.15.7:0/2504122743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27bc10ec80 0x7f27bc19b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:23:23.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.222+0000 7f27ab7fe700 1 -- 172.21.15.7:0/2504122743 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27b4004bb0 con 0x7f27bc19f920 2024-06-10T01:23:23.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.222+0000 7f27ab7fe700 1 -- 172.21.15.7:0/2504122743 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b4004280 con 0x7f27bc19f920 2024-06-10T01:23:23.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.222+0000 7f27c09b0700 1 -- 172.21.15.7:0/2504122743 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27bc1a7eb0 con 0x7f27bc19f920 2024-06-10T01:23:23.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.223+0000 7f27c09b0700 1 -- 172.21.15.7:0/2504122743 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27bc1019a0 con 0x7f27bc19f920 2024-06-10T01:23:23.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.225+0000 7f27ab7fe700 1 -- 172.21.15.7:0/2504122743 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f27b401b030 con 0x7f27bc19f920 2024-06-10T01:23:23.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.225+0000 7f27ab7fe700 1 --2- 172.21.15.7:0/2504122743 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f27a40a2d20 0x7f27a40a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:23.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.226+0000 7f27ab7fe700 1 -- 172.21.15.7:0/2504122743 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f27b40cf8e0 con 0x7f27bc19f920 2024-06-10T01:23:23.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.226+0000 7f27ba59c700 1 --2- 172.21.15.7:0/2504122743 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f27a40a2d20 0x7f27a40a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:23.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.226+0000 7f27ba59c700 1 --2- 172.21.15.7:0/2504122743 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f27a40a2d20 0x7f27a40a51e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f27ac000c00 tx=0x7f27ac005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:23.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.231+0000 7f27ab7fe700 1 -- 172.21.15.7:0/2504122743 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f27b4022020 con 0x7f27bc19f920 2024-06-10T01:23:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:23.243+0000 7f827effd640 1 -- 172.21.15.7:0/1382506163 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563af2476c20 con 0x563af258ea20 2024-06-10T01:23:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:23.243+0000 7f827effd640 10 client.28939 handle_command_reply: tid=0 2024-06-10T01:23:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:23.243+0000 7f82a8d9d740 1 client.28939 shutdown 2024-06-10T01:23:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:23.244+0000 7f82a8d9d740 1 -- 172.21.15.7:0/1382506163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af2506010 msgr2=0x563af25170b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:23.244+0000 7f82a8d9d740 1 --2- 172.21.15.7:0/1382506163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af2506010 0x563af25170b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8298005e30 tx=0x7f8298000f40 comp rx=0 tx=0).stop 2024-06-10T01:23:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:23.244+0000 7f82a8d9d740 1 -- 172.21.15.7:0/1382506163 shutdown_connections 2024-06-10T01:23:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:23.244+0000 7f82a8d9d740 1 --2- 172.21.15.7:0/1382506163 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563af258ea20 0x563af2590e40 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:23.244+0000 7f82a8d9d740 1 --2- 172.21.15.7:0/1382506163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563af250bb70 0x563af2513ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:23.244+0000 7f827effd640 0 client.28939 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:23.244+0000 7f82a8d9d740 1 --2- 172.21.15.7:0/1382506163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563af2506a00 0x563af25175f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:23.244+0000 7f82a8d9d740 1 --2- 172.21.15.7:0/1382506163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563af2506010 0x563af25170b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:23.244+0000 7f82a8d9d740 1 -- 172.21.15.7:0/1382506163 >> 172.21.15.7:0/1382506163 conn(0x563af2501680 msgr2=0x563af24fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:23.244+0000 7f82a8d9d740 1 -- 172.21.15.7:0/1382506163 shutdown_connections 2024-06-10T01:23:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:23.244+0000 7f82a8d9d740 1 -- 172.21.15.7:0/1382506163 wait complete. 2024-06-10T01:23:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:23.244+0000 7f82a8d9d740 20 client.28939 trim_cache size 0 max 16384 2024-06-10T01:23:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:23:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:23:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:23:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:23.245+0000 7f82a5fb0640 1 -- 172.21.15.7:0/157691310 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f82880a2c00 msgr2=0x7f82880a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:23.245+0000 7f82a5fb0640 1 --2- 172.21.15.7:0/157691310 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f82880a2c00 0x7f82880a50c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x563af2506e00 tx=0x7f8290000bc0 comp rx=0 tx=0).stop 2024-06-10T01:23:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:23.245+0000 7f82a5fb0640 1 -- 172.21.15.7:0/157691310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82a00ffd10 msgr2=0x7f82a0071790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:23.245+0000 7f82a5fb0640 1 --2- 172.21.15.7:0/157691310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82a00ffd10 0x7f82a0071790 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f8290009e60 tx=0x7f829000ed90 comp rx=0 tx=0).stop 2024-06-10T01:23:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:23.245+0000 7f82a5fb0640 1 -- 172.21.15.7:0/157691310 shutdown_connections 2024-06-10T01:23:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:23.245+0000 7f82a5fb0640 1 --2- 172.21.15.7:0/157691310 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f82880a2c00 0x7f82880a50c0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:23.245+0000 7f82a5fb0640 1 --2- 172.21.15.7:0/157691310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82a01036a0 0x7f82a0072ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:23.245+0000 7f82a5fb0640 1 --2- 172.21.15.7:0/157691310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82a0102ce0 0x7f82a0071cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:23.245+0000 7f82a5fb0640 1 --2- 172.21.15.7:0/157691310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82a00ffd10 0x7f82a0071790 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:23.245+0000 7f82a5fb0640 1 -- 172.21.15.7:0/157691310 >> 172.21.15.7:0/157691310 conn(0x7f82a00fb640 msgr2=0x7f82a0105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:23.245+0000 7f82a5fb0640 1 -- 172.21.15.7:0/157691310 shutdown_connections 2024-06-10T01:23:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:23.245+0000 7f82a5fb0640 1 -- 172.21.15.7:0/157691310 wait complete. 2024-06-10T01:23:23.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:23 smithi175 ceph-mon[36636]: pgmap v954: 129 pgs: 129 active+clean; 560 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 4.5 MiB/s wr, 274 op/s 2024-06-10T01:23:23.532 INFO:tasks.workunit.client.0.smithi007.stdout: 10 94913 24.35 MB/sec execute 120 sec latency 184.393 ms 2024-06-10T01:23:23.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:23 smithi078 ceph-mon[36683]: pgmap v954: 129 pgs: 129 active+clean; 560 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 4.5 MiB/s wr, 274 op/s 2024-06-10T01:23:23.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.595+0000 7f27c09b0700 1 -- 172.21.15.7:0/2504122743 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27bc02dca0 con 0x7f27bc19f920 2024-06-10T01:23:23.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.596+0000 7f27ab7fe700 1 -- 172.21.15.7:0/2504122743 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11858 (secure 0 0 0) 0x7f27b4010040 con 0x7f27bc19f920 2024-06-10T01:23:23.589 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:23:23.589 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":50,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:22:57.989337+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28825":{"gid":28825,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/1972426673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1972426673},{"type":"v1","addr":"172.21.15.175:6835","nonce":1972426673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:23:23.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.598+0000 7f27c09b0700 1 -- 172.21.15.7:0/2504122743 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f27a40a2d20 msgr2=0x7f27a40a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:23.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.598+0000 7f27c09b0700 1 --2- 172.21.15.7:0/2504122743 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f27a40a2d20 0x7f27a40a51e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f27ac000c00 tx=0x7f27ac005fd0 comp rx=0 tx=0).stop 2024-06-10T01:23:23.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.598+0000 7f27c09b0700 1 -- 172.21.15.7:0/2504122743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27bc19f920 msgr2=0x7f27bc19fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:23.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.598+0000 7f27c09b0700 1 --2- 172.21.15.7:0/2504122743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27bc19f920 0x7f27bc19fda0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f27b400ef30 tx=0x7f27b400c3a0 comp rx=0 tx=0).stop 2024-06-10T01:23:23.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.598+0000 7f27c09b0700 1 -- 172.21.15.7:0/2504122743 shutdown_connections 2024-06-10T01:23:23.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.598+0000 7f27c09b0700 1 --2- 172.21.15.7:0/2504122743 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f27a40a2d20 0x7f27a40a51e0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:23.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.598+0000 7f27c09b0700 1 --2- 172.21.15.7:0/2504122743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27bc19f920 0x7f27bc19fda0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:23.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.599+0000 7f27c09b0700 1 --2- 172.21.15.7:0/2504122743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27bc101190 0x7f27bc19b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:23.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.599+0000 7f27c09b0700 1 --2- 172.21.15.7:0/2504122743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27bc10ec80 0x7f27bc19b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:23.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.599+0000 7f27c09b0700 1 -- 172.21.15.7:0/2504122743 >> 172.21.15.7:0/2504122743 conn(0x7f27bc077590 msgr2=0x7f27bc0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:23.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.599+0000 7f27c09b0700 1 -- 172.21.15.7:0/2504122743 shutdown_connections 2024-06-10T01:23:23.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:23.599+0000 7f27c09b0700 1 -- 172.21.15.7:0/2504122743 wait complete. 2024-06-10T01:23:23.593 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 50 2024-06-10T01:23:24.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:24 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2504122743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:23:24.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:24 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2504122743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:23:24.532 INFO:tasks.workunit.client.0.smithi007.stdout: 10 95647 24.54 MB/sec execute 121 sec latency 45.291 ms 2024-06-10T01:23:24.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:24 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2504122743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:23:24.630 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-06-10T01:23:24.630 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-06-10T01:23:24.934 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:23:25.319 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:25 smithi007 ceph-mon[32398]: pgmap v955: 129 pgs: 129 active+clean; 560 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 4.0 MiB/s wr, 240 op/s 2024-06-10T01:23:25.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:25 smithi175 ceph-mon[36636]: pgmap v955: 129 pgs: 129 active+clean; 560 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 4.0 MiB/s wr, 240 op/s 2024-06-10T01:23:25.478 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:25 smithi078 ceph-mon[36683]: pgmap v955: 129 pgs: 129 active+clean; 560 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 4.0 MiB/s wr, 240 op/s 2024-06-10T01:23:25.534 INFO:tasks.workunit.client.0.smithi007.stdout: 10 96262 24.58 MB/sec execute 122 sec latency 58.837 ms 2024-06-10T01:23:26.532 INFO:tasks.workunit.client.0.smithi007.stdout: 10 96972 24.68 MB/sec execute 123 sec latency 48.915 ms 2024-06-10T01:23:26.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.990+0000 7f3bea7fc700 1 -- 172.21.15.7:0/3038319131 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc012650 con 0x7f3bec0748c0 2024-06-10T01:23:26.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.991+0000 7f3bf0bf9700 1 -- 172.21.15.7:0/3038319131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bec0748c0 msgr2=0x7f3bec074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:26.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.991+0000 7f3bf0bf9700 1 --2- 172.21.15.7:0/3038319131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bec0748c0 0x7f3bec074ca0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc009f90 tx=0x7f3bdc00b330 comp rx=0 tx=0).stop 2024-06-10T01:23:26.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.991+0000 7f3bf0bf9700 1 -- 172.21.15.7:0/3038319131 shutdown_connections 2024-06-10T01:23:26.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.991+0000 7f3bf0bf9700 1 --2- 172.21.15.7:0/3038319131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bec10eb20 0x7f3bec110f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:26.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.991+0000 7f3bf0bf9700 1 --2- 172.21.15.7:0/3038319131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bec0748c0 0x7f3bec074ca0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:26.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.991+0000 7f3bf0bf9700 1 --2- 172.21.15.7:0/3038319131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3bec0751e0 0x7f3bec10e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:26.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.991+0000 7f3bf0bf9700 1 -- 172.21.15.7:0/3038319131 >> 172.21.15.7:0/3038319131 conn(0x7f3bec0fde70 msgr2=0x7f3bec100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:26.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.991+0000 7f3bf0bf9700 1 -- 172.21.15.7:0/3038319131 shutdown_connections 2024-06-10T01:23:26.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.991+0000 7f3bf0bf9700 1 -- 172.21.15.7:0/3038319131 wait complete. 2024-06-10T01:23:26.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.992+0000 7f3bf0bf9700 1 Processor -- start 2024-06-10T01:23:26.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.992+0000 7f3bf0bf9700 1 -- start start 2024-06-10T01:23:26.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.992+0000 7f3bf0bf9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bec0748c0 0x7f3bec1a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:26.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.992+0000 7f3bf0bf9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bec0751e0 0x7f3bec1a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:26.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.992+0000 7f3bf0bf9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3bec10eb20 0x7f3bec19a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:26.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.992+0000 7f3bf0bf9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec114890 con 0x7f3bec0748c0 2024-06-10T01:23:26.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.992+0000 7f3bf0bf9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec114710 con 0x7f3bec10eb20 2024-06-10T01:23:26.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.992+0000 7f3bf0bf9700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec114a10 con 0x7f3bec0751e0 2024-06-10T01:23:26.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.993+0000 7f3bebfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3bec10eb20 0x7f3bec19a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:26.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.993+0000 7f3beaffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bec0751e0 0x7f3bec1a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:26.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.993+0000 7f3bebfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3bec10eb20 0x7f3bec19a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:45316/0 (socket says 172.21.15.7:45316) 2024-06-10T01:23:26.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.993+0000 7f3bebfff700 1 -- 172.21.15.7:0/3723475097 learned_addr learned my addr 172.21.15.7:0/3723475097 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:26.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.993+0000 7f3beb7fe700 1 --2- 172.21.15.7:0/3723475097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bec0748c0 0x7f3bec1a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:26.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.993+0000 7f3bebfff700 1 -- 172.21.15.7:0/3723475097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bec0751e0 msgr2=0x7f3bec1a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:26.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.993+0000 7f3bebfff700 1 --2- 172.21.15.7:0/3723475097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bec0751e0 0x7f3bec1a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:26.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.993+0000 7f3bebfff700 1 -- 172.21.15.7:0/3723475097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bec0748c0 msgr2=0x7f3bec1a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:26.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.993+0000 7f3bebfff700 1 --2- 172.21.15.7:0/3723475097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bec0748c0 0x7f3bec1a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:26.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.993+0000 7f3bebfff700 1 -- 172.21.15.7:0/3723475097 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bec19b0b0 con 0x7f3bec10eb20 2024-06-10T01:23:26.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.993+0000 7f3bebfff700 1 --2- 172.21.15.7:0/3723475097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3bec10eb20 0x7f3bec19a950 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3be400e970 tx=0x7f3be400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:26.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.994+0000 7f3be8ff9700 1 -- 172.21.15.7:0/3723475097 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be4004a30 con 0x7f3bec10eb20 2024-06-10T01:23:26.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.994+0000 7f3be8ff9700 1 -- 172.21.15.7:0/3723475097 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3be4005500 con 0x7f3bec10eb20 2024-06-10T01:23:26.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.994+0000 7f3bf0bf9700 1 -- 172.21.15.7:0/3723475097 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bec19b3a0 con 0x7f3bec10eb20 2024-06-10T01:23:26.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.994+0000 7f3bf0bf9700 1 -- 172.21.15.7:0/3723475097 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bec102170 con 0x7f3bec10eb20 2024-06-10T01:23:26.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.995+0000 7f3be8ff9700 1 -- 172.21.15.7:0/3723475097 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be4010d30 con 0x7f3bec10eb20 2024-06-10T01:23:26.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.998+0000 7f3be8ff9700 1 -- 172.21.15.7:0/3723475097 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f3be4021440 con 0x7f3bec10eb20 2024-06-10T01:23:26.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.999+0000 7f3be8ff9700 1 --2- 172.21.15.7:0/3723475097 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3bd40a2e30 0x7f3bd40a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:26.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.999+0000 7f3beb7fe700 1 --2- 172.21.15.7:0/3723475097 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3bd40a2e30 0x7f3bd40a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:26.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:26.999+0000 7f3be8ff9700 1 -- 172.21.15.7:0/3723475097 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f3be40cf8e0 con 0x7f3bec10eb20 2024-06-10T01:23:26.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.000+0000 7f3bfc6deb80 1 Processor -- start 2024-06-10T01:23:26.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.000+0000 7f3bfc6deb80 1 -- start start 2024-06-10T01:23:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.000+0000 7f3beb7fe700 1 --2- 172.21.15.7:0/3723475097 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3bd40a2e30 0x7f3bd40a52f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc009da0 tx=0x7f3bdc009720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.000+0000 7f3bfc6deb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae723db870 0x55ae723dbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.000+0000 7f3bfc6deb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae723dc190 0x55ae723e6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.000+0000 7f3bfc6deb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae723e7040 0x55ae723e9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.000+0000 7f3bfc6deb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ae722cf0f0 con 0x55ae723dc190 2024-06-10T01:23:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.000+0000 7f3bfc6deb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ae7232d9e0 con 0x55ae723db870 2024-06-10T01:23:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.000+0000 7f3bfc6deb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ae722955f0 con 0x55ae723e7040 2024-06-10T01:23:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.001+0000 7f3beb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae723db870 0x55ae723dbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.001+0000 7f3bebfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae723e7040 0x55ae723e9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.001+0000 7f3beb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae723db870 0x55ae723dbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:45332/0 (socket says 172.21.15.7:45332) 2024-06-10T01:23:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.001+0000 7f3beb7fe700 1 -- 172.21.15.7:0/1568678478 learned_addr learned my addr 172.21.15.7:0/1568678478 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.001+0000 7f3beaffd700 1 --2- 172.21.15.7:0/1568678478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae723dc190 0x55ae723e6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:26.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.001+0000 7f3bebfff700 1 -- 172.21.15.7:0/1568678478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae723db870 msgr2=0x55ae723dbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:26.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.001+0000 7f3bebfff700 1 --2- 172.21.15.7:0/1568678478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae723db870 0x55ae723dbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:26.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.001+0000 7f3bebfff700 1 -- 172.21.15.7:0/1568678478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae723dc190 msgr2=0x55ae723e6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:26.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.001+0000 7f3bebfff700 1 --2- 172.21.15.7:0/1568678478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae723dc190 0x55ae723e6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:26.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.001+0000 7f3bebfff700 1 -- 172.21.15.7:0/1568678478 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae7227ca60 con 0x55ae723e7040 2024-06-10T01:23:26.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.001+0000 7f3bebfff700 1 --2- 172.21.15.7:0/1568678478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae723e7040 0x55ae723e9430 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3be40cd060 tx=0x7f3be4098150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2edf1895b63a0ea server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:26.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.001+0000 7f3bd1ffb700 1 -- 172.21.15.7:0/1568678478 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be409c4c0 con 0x55ae723e7040 2024-06-10T01:23:26.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.001+0000 7f3bd1ffb700 1 -- 172.21.15.7:0/1568678478 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3be409bdc0 con 0x55ae723e7040 2024-06-10T01:23:26.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.001+0000 7f3bd1ffb700 1 -- 172.21.15.7:0/1568678478 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be409c020 con 0x55ae723e7040 2024-06-10T01:23:26.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.002+0000 7f3bfc6deb80 1 -- 172.21.15.7:0/1568678478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae723e7040 msgr2=0x55ae723e9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:26.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.002+0000 7f3bfc6deb80 1 --2- 172.21.15.7:0/1568678478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae723e7040 0x55ae723e9430 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3be40cd060 tx=0x7f3be4098150 comp rx=0 tx=0).stop 2024-06-10T01:23:26.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.002+0000 7f3bfc6deb80 1 -- 172.21.15.7:0/1568678478 shutdown_connections 2024-06-10T01:23:26.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.002+0000 7f3bfc6deb80 1 --2- 172.21.15.7:0/1568678478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae723e7040 0x55ae723e9430 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:26.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.002+0000 7f3bfc6deb80 1 --2- 172.21.15.7:0/1568678478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae723db870 0x55ae723dbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:26.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.002+0000 7f3bfc6deb80 1 --2- 172.21.15.7:0/1568678478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae723dc190 0x55ae723e6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:26.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.002+0000 7f3bfc6deb80 1 -- 172.21.15.7:0/1568678478 >> 172.21.15.7:0/1568678478 conn(0x55ae72309210 msgr2=0x55ae723d7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:26.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.003+0000 7f3bfc6deb80 1 -- 172.21.15.7:0/1568678478 shutdown_connections 2024-06-10T01:23:26.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.003+0000 7f3bfc6deb80 1 -- 172.21.15.7:0/1568678478 wait complete. 2024-06-10T01:23:27.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.003+0000 7f3bfc6deb80 1 -- start start 2024-06-10T01:23:27.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.004+0000 7f3bfc6deb80 1 Processor -- start 2024-06-10T01:23:27.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.004+0000 7f3bfc6deb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae723db870 0x55ae723e6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:27.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.004+0000 7f3bfc6deb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae723dc190 0x55ae723e3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.004+0000 7f3bfc6deb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae723e7040 0x55ae723e3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.004+0000 7f3bfc6deb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ae722ce150 con 0x55ae723dc190 2024-06-10T01:23:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.004+0000 7f3beb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae723db870 0x55ae723e6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.004+0000 7f3beb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae723db870 0x55ae723e6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:45348/0 (socket says 172.21.15.7:45348) 2024-06-10T01:23:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.004+0000 7f3beb7fe700 1 -- 172.21.15.7:0/501547220 learned_addr learned my addr 172.21.15.7:0/501547220 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.004+0000 7f3beaffd700 1 --2- 172.21.15.7:0/501547220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae723dc190 0x55ae723e3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.004+0000 7f3bfc6deb80 1 -- 172.21.15.7:0/501547220 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ae723299a0 con 0x55ae723db870 2024-06-10T01:23:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.004+0000 7f3bfc6deb80 1 -- 172.21.15.7:0/501547220 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ae723e9b60 con 0x55ae723e7040 2024-06-10T01:23:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.004+0000 7f3bebfff700 1 --2- 172.21.15.7:0/501547220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae723e7040 0x55ae723e3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.004+0000 7f3bfc6deb80 10 client.0 _ll_register_callbacks cb 0x55ae722c3740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:23:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.005+0000 7f3beb7fe700 1 -- 172.21.15.7:0/501547220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae723e7040 msgr2=0x55ae723e3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.005+0000 7f3beb7fe700 1 --2- 172.21.15.7:0/501547220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae723e7040 0x55ae723e3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.005+0000 7f3beb7fe700 1 -- 172.21.15.7:0/501547220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae723dc190 msgr2=0x55ae723e3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:27.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.005+0000 7f3beb7fe700 1 --2- 172.21.15.7:0/501547220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae723dc190 0x55ae723e3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:27.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.005+0000 7f3beb7fe700 1 -- 172.21.15.7:0/501547220 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae722f38e0 con 0x55ae723db870 2024-06-10T01:23:27.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.005+0000 7f3beb7fe700 1 --2- 172.21.15.7:0/501547220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae723db870 0x55ae723e6770 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00b6c0 tx=0x7f3bdc009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9639515fc1380b63 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:27.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.005+0000 7f3bcaffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:23:27.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.006+0000 7f3bcaffd700 1 -- 172.21.15.7:0/501547220 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc029420 con 0x55ae723db870 2024-06-10T01:23:27.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.006+0000 7f3bfc6deb80 1 -- 172.21.15.7:0/501547220 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3be409bdc0 con 0x55ae723db870 2024-06-10T01:23:27.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.006+0000 7f3bcaffd700 1 -- 172.21.15.7:0/501547220 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f3bdc0295c0 con 0x55ae723db870 2024-06-10T01:23:27.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.006+0000 7f3bcaffd700 1 -- 172.21.15.7:0/501547220 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f3bdc03a050 con 0x55ae723db870 2024-06-10T01:23:27.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.006+0000 7f3bfc6deb80 10 client.28973 fetch_fsmap learned FSMap version 50 2024-06-10T01:23:27.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.006+0000 7f3bfc6deb80 1 -- 172.21.15.7:0/501547220 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55ae723f42a0 con 0x55ae723db870 2024-06-10T01:23:27.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.006+0000 7f3bcaffd700 1 -- 172.21.15.7:0/501547220 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3be409bdc0 con 0x55ae723db870 2024-06-10T01:23:27.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.007+0000 7f3bcaffd700 1 -- 172.21.15.7:0/501547220 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7f3bdc011070 con 0x55ae723db870 2024-06-10T01:23:27.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.007+0000 7f3bfc6deb80 10 client.28973 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:23:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.007+0000 7f3bfc6deb80 10 client.28973 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:23:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.007+0000 7f3bfc6deb80 20 client.28973 populate_metadata read hostname 'smithi007' 2024-06-10T01:23:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.007+0000 7f3bfc6deb80 1 --2- 172.21.15.7:0/501547220 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ae723f68d0 0x55ae723f8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.007+0000 7f3bfc6deb80 4 client.28973 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:23:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.007+0000 7f3bfc6deb80 1 -- 172.21.15.7:0/501547220 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae7233aee0 con 0x55ae723f68d0 2024-06-10T01:23:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.007+0000 7f3beaffd700 1 --2- 172.21.15.7:0/501547220 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ae723f68d0 0x55ae723f8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.008+0000 7f3bcaffd700 10 client.28973 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.008+0000 7f3beaffd700 1 --2- 172.21.15.7:0/501547220 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ae723f68d0 0x55ae723f8d30 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c5e658ce142dd49 server_cookie=736c6f40da482a6e in_seq=0 out_seq=0 2024-06-10T01:23:27.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.010+0000 7f3bcaffd700 1 -- 172.21.15.7:0/501547220 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ae7233aee0 con 0x55ae723f68d0 2024-06-10T01:23:27.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.010+0000 7f3bcaffd700 10 client.28973 handle_command_reply: tid=0 2024-06-10T01:23:27.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.010+0000 7f3bfc6deb80 1 client.28973 shutdown 2024-06-10T01:23:27.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.011+0000 7f3bfc6deb80 1 -- 172.21.15.7:0/501547220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae723db870 msgr2=0x55ae723e6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:27.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.011+0000 7f3bfc6deb80 1 --2- 172.21.15.7:0/501547220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae723db870 0x55ae723e6770 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00b6c0 tx=0x7f3bdc009920 comp rx=0 tx=0).stop 2024-06-10T01:23:27.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.011+0000 7f3bfc6deb80 1 -- 172.21.15.7:0/501547220 shutdown_connections 2024-06-10T01:23:27.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.011+0000 7f3bfc6deb80 1 --2- 172.21.15.7:0/501547220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae723e7040 0x55ae723e3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:27.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.011+0000 7f3bfc6deb80 1 --2- 172.21.15.7:0/501547220 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ae723f68d0 0x55ae723f8d30 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:27.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.011+0000 7f3bfc6deb80 1 --2- 172.21.15.7:0/501547220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae723db870 0x55ae723e6770 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:27.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.011+0000 7f3bfc6deb80 1 --2- 172.21.15.7:0/501547220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae723dc190 0x55ae723e3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:27.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.011+0000 7f3bfc6deb80 1 -- 172.21.15.7:0/501547220 >> 172.21.15.7:0/501547220 conn(0x55ae72309210 msgr2=0x55ae723d7910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:27.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.011+0000 7f3bcaffd700 0 client.28973 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:27.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.011+0000 7f3bfc6deb80 1 -- 172.21.15.7:0/501547220 shutdown_connections 2024-06-10T01:23:27.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.011+0000 7f3bfc6deb80 1 -- 172.21.15.7:0/501547220 wait complete. 2024-06-10T01:23:27.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.011+0000 7f3bfc6deb80 20 client.28973 trim_cache size 0 max 16384 2024-06-10T01:23:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.034+0000 7f3bfc6deb80 1 Processor -- start 2024-06-10T01:23:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.034+0000 7f3bfc6deb80 1 -- start start 2024-06-10T01:23:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.034+0000 7f3bfc6deb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae723db870 0x55ae723fe600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.034+0000 7f3bfc6deb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae723dc190 0x55ae723feb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.034+0000 7f3bfc6deb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae723e7040 0x55ae723e0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.034+0000 7f3bfc6deb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec1ae660 con 0x55ae723dc190 2024-06-10T01:23:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.034+0000 7f3bfc6deb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec1abef0 con 0x55ae723e7040 2024-06-10T01:23:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.034+0000 7f3bfc6deb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec1a9d40 con 0x55ae723db870 2024-06-10T01:23:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.034+0000 7f3beaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae723dc190 0x55ae723feb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.034+0000 7f3beaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae723dc190 0x55ae723feb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55194/0 (socket says 172.21.15.7:55194) 2024-06-10T01:23:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.034+0000 7f3beaffd700 1 -- 172.21.15.7:0/2069476979 learned_addr learned my addr 172.21.15.7:0/2069476979 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.034+0000 7f3bebfff700 1 --2- 172.21.15.7:0/2069476979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae723e7040 0x55ae723e0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.034+0000 7f3beb7fe700 1 --2- 172.21.15.7:0/2069476979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae723db870 0x55ae723fe600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.035+0000 7f3beaffd700 1 -- 172.21.15.7:0/2069476979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae723db870 msgr2=0x55ae723fe600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.035+0000 7f3beaffd700 1 --2- 172.21.15.7:0/2069476979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae723db870 0x55ae723fe600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.035+0000 7f3beaffd700 1 -- 172.21.15.7:0/2069476979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae723e7040 msgr2=0x55ae723e0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.035+0000 7f3beaffd700 1 --2- 172.21.15.7:0/2069476979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae723e7040 0x55ae723e0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.035+0000 7f3beaffd700 1 -- 172.21.15.7:0/2069476979 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae723dc570 con 0x55ae723dc190 2024-06-10T01:23:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.035+0000 7f3bebfff700 1 --2- 172.21.15.7:0/2069476979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae723e7040 0x55ae723e0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:23:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.035+0000 7f3beaffd700 1 --2- 172.21.15.7:0/2069476979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae723dc190 0x55ae723feb40 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f3be0012fd0 tx=0x7f3be0012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5b906ff9637a40 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:27.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.035+0000 7f3bd17fa700 1 -- 172.21.15.7:0/2069476979 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0004830 con 0x55ae723dc190 2024-06-10T01:23:27.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.035+0000 7f3bd17fa700 1 -- 172.21.15.7:0/2069476979 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3be0012250 con 0x55ae723dc190 2024-06-10T01:23:27.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.035+0000 7f3bd17fa700 1 -- 172.21.15.7:0/2069476979 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0005240 con 0x55ae723dc190 2024-06-10T01:23:27.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.036+0000 7f3bfc6deb80 1 -- 172.21.15.7:0/2069476979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae723dc190 msgr2=0x55ae723feb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:27.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.036+0000 7f3bfc6deb80 1 --2- 172.21.15.7:0/2069476979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae723dc190 0x55ae723feb40 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f3be0012fd0 tx=0x7f3be0012040 comp rx=0 tx=0).stop 2024-06-10T01:23:27.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.036+0000 7f3bfc6deb80 1 -- 172.21.15.7:0/2069476979 shutdown_connections 2024-06-10T01:23:27.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.036+0000 7f3bfc6deb80 1 --2- 172.21.15.7:0/2069476979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae723e7040 0x55ae723e0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:27.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.036+0000 7f3bfc6deb80 1 --2- 172.21.15.7:0/2069476979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae723db870 0x55ae723fe600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:27.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.036+0000 7f3bfc6deb80 1 --2- 172.21.15.7:0/2069476979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae723dc190 0x55ae723feb40 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:27.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.036+0000 7f3bfc6deb80 1 -- 172.21.15.7:0/2069476979 >> 172.21.15.7:0/2069476979 conn(0x55ae72309210 msgr2=0x55ae723e3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:27.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.036+0000 7f3bfc6deb80 1 -- 172.21.15.7:0/2069476979 shutdown_connections 2024-06-10T01:23:27.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.036+0000 7f3bfc6deb80 1 -- 172.21.15.7:0/2069476979 wait complete. 2024-06-10T01:23:27.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.037+0000 7f3bfc6deb80 1 -- start start 2024-06-10T01:23:27.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.037+0000 7f3bfc6deb80 1 Processor -- start 2024-06-10T01:23:27.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.037+0000 7f3bfc6deb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae723db870 0x55ae723eee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.037+0000 7f3bfc6deb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae723dc190 0x55ae723ef3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.037+0000 7f3bfc6deb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae723e7040 0x55ae723ed330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.037+0000 7f3bfc6deb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec1aa3b0 con 0x55ae723dc190 2024-06-10T01:23:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.037+0000 7f3bfc6deb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec1ab820 con 0x55ae723db870 2024-06-10T01:23:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.037+0000 7f3bfc6deb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec1aaa80 con 0x55ae723e7040 2024-06-10T01:23:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.037+0000 7f3bfc6deb80 10 client.0 _ll_register_callbacks cb 0x55ae722c3740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:23:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.038+0000 7f3beb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae723db870 0x55ae723eee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.038+0000 7f3bebfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae723e7040 0x55ae723ed330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.038+0000 7f3beaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae723dc190 0x55ae723ef3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.038+0000 7f3bebfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae723e7040 0x55ae723ed330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:57522/0 (socket says 172.21.15.7:57522) 2024-06-10T01:23:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.038+0000 7f3bebfff700 1 -- 172.21.15.7:0/2632134298 learned_addr learned my addr 172.21.15.7:0/2632134298 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.038+0000 7f3beb7fe700 1 -- 172.21.15.7:0/2632134298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae723e7040 msgr2=0x55ae723ed330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.038+0000 7f3beb7fe700 1 --2- 172.21.15.7:0/2632134298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae723e7040 0x55ae723ed330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.038+0000 7f3beb7fe700 1 -- 172.21.15.7:0/2632134298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae723dc190 msgr2=0x55ae723ef3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.038+0000 7f3beb7fe700 1 --2- 172.21.15.7:0/2632134298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae723dc190 0x55ae723ef3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.038+0000 7f3beb7fe700 1 -- 172.21.15.7:0/2632134298 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae723edb40 con 0x55ae723db870 2024-06-10T01:23:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.038+0000 7f3beb7fe700 1 --2- 172.21.15.7:0/2632134298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae723db870 0x55ae723eee80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc0078b0 tx=0x7f3bdc007610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f00bb433609c39c0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.038+0000 7f3bcaffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:23:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.038+0000 7f3bcaffd700 1 -- 172.21.15.7:0/2632134298 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc005e70 con 0x55ae723db870 2024-06-10T01:23:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.038+0000 7f3bcaffd700 1 -- 172.21.15.7:0/2632134298 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f3bdc006010 con 0x55ae723db870 2024-06-10T01:23:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.038+0000 7f3bfc6deb80 1 -- 172.21.15.7:0/2632134298 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3be0012250 con 0x55ae723db870 2024-06-10T01:23:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.039+0000 7f3bcaffd700 1 -- 172.21.15.7:0/2632134298 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f3bdc04b050 con 0x55ae723db870 2024-06-10T01:23:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.039+0000 7f3bfc6deb80 10 client.28979 fetch_fsmap learned FSMap version 50 2024-06-10T01:23:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.039+0000 7f3bfc6deb80 1 -- 172.21.15.7:0/2632134298 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55ae72462070 con 0x55ae723db870 2024-06-10T01:23:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.039+0000 7f3bcaffd700 1 -- 172.21.15.7:0/2632134298 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3be0012250 con 0x55ae723db870 2024-06-10T01:23:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.039+0000 7f3bcaffd700 1 -- 172.21.15.7:0/2632134298 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7f3bdc046030 con 0x55ae723db870 2024-06-10T01:23:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.039+0000 7f3bfc6deb80 10 client.28979 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:23:27.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.039+0000 7f3bfc6deb80 10 client.28979 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:23:27.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.039+0000 7f3bfc6deb80 20 client.28979 populate_metadata read hostname 'smithi007' 2024-06-10T01:23:27.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.039+0000 7f3bfc6deb80 1 --2- 172.21.15.7:0/2632134298 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ae724627e0 0x55ae72464c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:27.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.039+0000 7f3bfc6deb80 4 client.28979 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-06-10T01:23:27.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.039+0000 7f3bfc6deb80 1 -- 172.21.15.7:0/2632134298 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55ae72335460 con 0x55ae724627e0 2024-06-10T01:23:27.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.039+0000 7f3beaffd700 1 --2- 172.21.15.7:0/2632134298 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ae724627e0 0x55ae72464c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:27.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.040+0000 7f3beaffd700 1 --2- 172.21.15.7:0/2632134298 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ae724627e0 0x55ae72464c00 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c089b01e1d0292b server_cookie=22231e8e56ca255c in_seq=0 out_seq=0 2024-06-10T01:23:27.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.040+0000 7f3bcaffd700 10 client.28979 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:27.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:27 smithi007 ceph-mon[32398]: pgmap v956: 129 pgs: 129 active+clean; 556 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.3 MiB/s wr, 515 op/s 2024-06-10T01:23:27.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:27 smithi175 ceph-mon[36636]: pgmap v956: 129 pgs: 129 active+clean; 556 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.3 MiB/s wr, 515 op/s 2024-06-10T01:23:27.532 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97412 24.67 MB/sec execute 124 sec latency 474.151 ms 2024-06-10T01:23:27.578 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:27 smithi078 ceph-mon[36683]: pgmap v956: 129 pgs: 129 active+clean; 556 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.3 MiB/s wr, 515 op/s 2024-06-10T01:23:27.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.619+0000 7f3bcaffd700 1 -- 172.21.15.7:0/2632134298 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ae72335460 con 0x55ae724627e0 2024-06-10T01:23:27.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.619+0000 7f3bcaffd700 10 client.28979 handle_command_reply: tid=0 2024-06-10T01:23:27.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.619+0000 7f3bfc6deb80 1 client.28979 shutdown 2024-06-10T01:23:27.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.619+0000 7f3bfc6deb80 1 -- 172.21.15.7:0/2632134298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae723db870 msgr2=0x55ae723eee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:27.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.619+0000 7f3bfc6deb80 1 --2- 172.21.15.7:0/2632134298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae723db870 0x55ae723eee80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc0078b0 tx=0x7f3bdc007610 comp rx=0 tx=0).stop 2024-06-10T01:23:27.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.619+0000 7f3bfc6deb80 1 -- 172.21.15.7:0/2632134298 shutdown_connections 2024-06-10T01:23:27.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.619+0000 7f3bfc6deb80 1 --2- 172.21.15.7:0/2632134298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ae723e7040 0x55ae723ed330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:27.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.619+0000 7f3bfc6deb80 1 --2- 172.21.15.7:0/2632134298 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ae724627e0 0x55ae72464c00 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:27.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.619+0000 7f3bfc6deb80 1 --2- 172.21.15.7:0/2632134298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae723db870 0x55ae723eee80 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:27.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.619+0000 7f3bcaffd700 0 client.28979 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:27.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.619+0000 7f3bfc6deb80 1 --2- 172.21.15.7:0/2632134298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae723dc190 0x55ae723ef3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:27.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.619+0000 7f3bfc6deb80 1 -- 172.21.15.7:0/2632134298 >> 172.21.15.7:0/2632134298 conn(0x55ae72309210 msgr2=0x55ae723f1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:27.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.619+0000 7f3bfc6deb80 1 -- 172.21.15.7:0/2632134298 shutdown_connections 2024-06-10T01:23:27.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.619+0000 7f3bfc6deb80 1 -- 172.21.15.7:0/2632134298 wait complete. 2024-06-10T01:23:27.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.619+0000 7f3bfc6deb80 20 client.28979 trim_cache size 0 max 16384 2024-06-10T01:23:27.614 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:23:27.614 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 50, 2024-06-10T01:23:27.614 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:23:27.614 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 58, 2024-06-10T01:23:27.614 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:23:27.614 INFO:teuthology.orchestra.run.smithi007.stdout: "f8806db1": { 2024-06-10T01:23:27.614 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 58, 2024-06-10T01:23:27.615 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:23:27.615 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:23:27.615 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:23:27.615 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:23:27.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:23:27.615 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:23:27.615 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 150.0, 2024-06-10T01:23:27.615 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:23:27.615 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:23:27.615 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:23:27.615 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:23:27.615 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:23:27.615 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:23:27.615 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:23:27.615 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:23:27.616 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:23:27.616 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:23:27.616 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:23:27.616 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:23:27.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.620+0000 7f3bf0bf9700 1 -- 172.21.15.7:0/3723475097 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3bd40a2e30 msgr2=0x7f3bd40a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:27.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.620+0000 7f3bf0bf9700 1 --2- 172.21.15.7:0/3723475097 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3bd40a2e30 0x7f3bd40a52f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc009da0 tx=0x7f3bdc009720 comp rx=0 tx=0).stop 2024-06-10T01:23:27.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.620+0000 7f3bf0bf9700 1 -- 172.21.15.7:0/3723475097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3bec10eb20 msgr2=0x7f3bec19a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:27.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.620+0000 7f3bf0bf9700 1 --2- 172.21.15.7:0/3723475097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3bec10eb20 0x7f3bec19a950 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3be400e970 tx=0x7f3be400ec80 comp rx=0 tx=0).stop 2024-06-10T01:23:27.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.620+0000 7f3bf0bf9700 1 -- 172.21.15.7:0/3723475097 shutdown_connections 2024-06-10T01:23:27.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.620+0000 7f3bf0bf9700 1 --2- 172.21.15.7:0/3723475097 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3bd40a2e30 0x7f3bd40a52f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:27.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.620+0000 7f3bf0bf9700 1 --2- 172.21.15.7:0/3723475097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3bec10eb20 0x7f3bec19a950 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:27.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.620+0000 7f3bf0bf9700 1 --2- 172.21.15.7:0/3723475097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bec0748c0 0x7f3bec1a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:27.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.620+0000 7f3bf0bf9700 1 --2- 172.21.15.7:0/3723475097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bec0751e0 0x7f3bec1a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:27.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.620+0000 7f3bf0bf9700 1 -- 172.21.15.7:0/3723475097 >> 172.21.15.7:0/3723475097 conn(0x7f3bec0fde70 msgr2=0x7f3bec110da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:27.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.620+0000 7f3bf0bf9700 1 -- 172.21.15.7:0/3723475097 shutdown_connections 2024-06-10T01:23:27.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:27.620+0000 7f3bf0bf9700 1 -- 172.21.15.7:0/3723475097 wait complete. 2024-06-10T01:23:28.364 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f8806db1', quiesce took 7.7 seconds. Will release after: 22.3 2024-06-10T01:23:28.532 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 24.49 MB/sec execute 125 sec latency 1469.439 ms 2024-06-10T01:23:29.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:29 smithi175 ceph-mon[36636]: pgmap v957: 129 pgs: 129 active+clean; 554 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.7 MiB/s wr, 407 op/s 2024-06-10T01:23:29.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:29 smithi007 ceph-mon[32398]: pgmap v957: 129 pgs: 129 active+clean; 554 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.7 MiB/s wr, 407 op/s 2024-06-10T01:23:29.532 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 24.30 MB/sec execute 126 sec latency 2469.529 ms 2024-06-10T01:23:29.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:29 smithi078 ceph-mon[36683]: pgmap v957: 129 pgs: 129 active+clean; 554 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.7 MiB/s wr, 407 op/s 2024-06-10T01:23:30.532 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 24.11 MB/sec execute 127 sec latency 3469.607 ms 2024-06-10T01:23:31.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:31 smithi175 ceph-mon[36636]: pgmap v958: 129 pgs: 129 active+clean; 550 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 5.4 MiB/s wr, 542 op/s 2024-06-10T01:23:31.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:31 smithi007 ceph-mon[32398]: pgmap v958: 129 pgs: 129 active+clean; 550 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 5.4 MiB/s wr, 542 op/s 2024-06-10T01:23:31.532 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 23.92 MB/sec execute 128 sec latency 4469.742 ms 2024-06-10T01:23:31.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:31 smithi078 ceph-mon[36683]: pgmap v958: 129 pgs: 129 active+clean; 550 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 5.4 MiB/s wr, 542 op/s 2024-06-10T01:23:32.532 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 23.74 MB/sec execute 129 sec latency 5469.859 ms 2024-06-10T01:23:33.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:33 smithi175 ceph-mon[36636]: pgmap v959: 129 pgs: 129 active+clean; 551 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 4.4 MiB/s wr, 508 op/s 2024-06-10T01:23:33.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:33 smithi007 ceph-mon[32398]: pgmap v959: 129 pgs: 129 active+clean; 551 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 4.4 MiB/s wr, 508 op/s 2024-06-10T01:23:33.532 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 23.55 MB/sec execute 130 sec latency 6469.982 ms 2024-06-10T01:23:33.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:33 smithi078 ceph-mon[36683]: pgmap v959: 129 pgs: 129 active+clean; 551 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 4.4 MiB/s wr, 508 op/s 2024-06-10T01:23:34.533 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 23.37 MB/sec execute 131 sec latency 7470.101 ms 2024-06-10T01:23:35.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:35 smithi175 ceph-mon[36636]: pgmap v960: 129 pgs: 129 active+clean; 551 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.6 MiB/s wr, 472 op/s 2024-06-10T01:23:35.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:35 smithi007 ceph-mon[32398]: pgmap v960: 129 pgs: 129 active+clean; 551 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.6 MiB/s wr, 472 op/s 2024-06-10T01:23:35.532 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 23.20 MB/sec execute 132 sec latency 8470.151 ms 2024-06-10T01:23:35.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:35 smithi078 ceph-mon[36683]: pgmap v960: 129 pgs: 129 active+clean; 551 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.6 MiB/s wr, 472 op/s 2024-06-10T01:23:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.352+0000 7fbaaab09640 1 -- 172.21.15.7:0/3362379236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaa41076e0 msgr2=0x7fbaa4109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.352+0000 7fbaaab09640 1 --2- 172.21.15.7:0/3362379236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaa41076e0 0x7fbaa4109b30 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fba9c009ca0 tx=0x7fba9c02a8c0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.352+0000 7fbaaab09640 1 -- 172.21.15.7:0/3362379236 shutdown_connections 2024-06-10T01:23:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.352+0000 7fbaaab09640 1 --2- 172.21.15.7:0/3362379236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaa41076e0 0x7fbaa4109b30 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.352+0000 7fbaaab09640 1 --2- 172.21.15.7:0/3362379236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaa4104db0 0x7fbaa41071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.352+0000 7fbaaab09640 1 --2- 172.21.15.7:0/3362379236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbaa4102480 0x7fbaa4104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.352+0000 7fbaaab09640 1 -- 172.21.15.7:0/3362379236 >> 172.21.15.7:0/3362379236 conn(0x7fbaa40fbe70 msgr2=0x7fbaa40fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.352+0000 7fbaaab09640 1 -- 172.21.15.7:0/3362379236 shutdown_connections 2024-06-10T01:23:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.352+0000 7fbaaab09640 1 -- 172.21.15.7:0/3362379236 wait complete. 2024-06-10T01:23:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.353+0000 7fbaaab09640 1 Processor -- start 2024-06-10T01:23:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.353+0000 7fbaaab09640 1 -- start start 2024-06-10T01:23:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.353+0000 7fbaaab09640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaa4102480 0x7fbaa4197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.353+0000 7fbaaab09640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbaa4104db0 0x7fbaa4198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.353+0000 7fbaaab09640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaa41076e0 0x7fbaa419f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.353+0000 7fbaaab09640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa410d6a0 con 0x7fbaa4102480 2024-06-10T01:23:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.353+0000 7fbaaab09640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa410d520 con 0x7fbaa4104db0 2024-06-10T01:23:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.353+0000 7fbaaab09640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa410d820 con 0x7fbaa41076e0 2024-06-10T01:23:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.353+0000 7fbaa88a5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaa4102480 0x7fbaa4197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.353+0000 7fbaa88a5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaa4102480 0x7fbaa4197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54216/0 (socket says 172.21.15.7:54216) 2024-06-10T01:23:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.353+0000 7fbaa88a5640 1 -- 172.21.15.7:0/1868277519 learned_addr learned my addr 172.21.15.7:0/1868277519 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.354+0000 7fbaa3fff640 1 --2- 172.21.15.7:0/1868277519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbaa4104db0 0x7fbaa4198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.354+0000 7fbaa90a6640 1 --2- 172.21.15.7:0/1868277519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaa41076e0 0x7fbaa419f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.354+0000 7fbaa88a5640 1 -- 172.21.15.7:0/1868277519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaa41076e0 msgr2=0x7fbaa419f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.354+0000 7fbaa88a5640 1 --2- 172.21.15.7:0/1868277519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaa41076e0 0x7fbaa419f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.354+0000 7fbaa88a5640 1 -- 172.21.15.7:0/1868277519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbaa4104db0 msgr2=0x7fbaa4198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.354+0000 7fbaa88a5640 1 --2- 172.21.15.7:0/1868277519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbaa4104db0 0x7fbaa4198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.354+0000 7fbaa88a5640 1 -- 172.21.15.7:0/1868277519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa419f910 con 0x7fbaa4102480 2024-06-10T01:23:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.354+0000 7fbaa88a5640 1 --2- 172.21.15.7:0/1868277519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaa4102480 0x7fbaa4197d00 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fba9400d950 tx=0x7fba9400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.354+0000 7fbaa1ffb640 1 -- 172.21.15.7:0/1868277519 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba94017070 con 0x7fbaa4102480 2024-06-10T01:23:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.354+0000 7fbaa1ffb640 1 -- 172.21.15.7:0/1868277519 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba94009c30 con 0x7fbaa4102480 2024-06-10T01:23:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.354+0000 7fbaa1ffb640 1 -- 172.21.15.7:0/1868277519 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9400f6b0 con 0x7fbaa4102480 2024-06-10T01:23:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.354+0000 7fbaaab09640 1 -- 172.21.15.7:0/1868277519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaa419fc00 con 0x7fbaa4102480 2024-06-10T01:23:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.354+0000 7fbaaab09640 1 -- 172.21.15.7:0/1868277519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaa41a01a0 con 0x7fbaa4102480 2024-06-10T01:23:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.357+0000 7fbaa1ffb640 1 -- 172.21.15.7:0/1868277519 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fba940109d0 con 0x7fbaa4102480 2024-06-10T01:23:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.357+0000 7fbaad812740 1 Processor -- start 2024-06-10T01:23:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.357+0000 7fbaad812740 1 -- start start 2024-06-10T01:23:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.357+0000 7fbaad812740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2f767d420 0x55f2f767d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.357+0000 7fbaad812740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2f767de10 0x55f2f7682910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.357+0000 7fbaa1ffb640 1 --2- 172.21.15.7:0/1868277519 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fba900a2c00 0x7fba900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.357+0000 7fbaad812740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2f7682f80 0x55f2f7685410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.357+0000 7fbaad812740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f2f75dc4f0 con 0x55f2f767d420 2024-06-10T01:23:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.357+0000 7fbaad812740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f2f757a870 con 0x55f2f7682f80 2024-06-10T01:23:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.357+0000 7fbaad812740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f2f75d84b0 con 0x55f2f767de10 2024-06-10T01:23:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.357+0000 7fbaa90a6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2f767d420 0x55f2f767d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.357+0000 7fbaa90a6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2f767d420 0x55f2f767d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54230/0 (socket says 172.21.15.7:54230) 2024-06-10T01:23:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.358+0000 7fbaa90a6640 1 -- 172.21.15.7:0/384588413 learned_addr learned my addr 172.21.15.7:0/384588413 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.358+0000 7fbaa3fff640 1 --2- 172.21.15.7:0/1868277519 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fba900a2c00 0x7fba900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.358+0000 7fbaa1ffb640 1 -- 172.21.15.7:0/1868277519 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fba940cf040 con 0x7fbaa4102480 2024-06-10T01:23:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.358+0000 7fbaa90a6640 1 --2- 172.21.15.7:0/384588413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2f7682f80 0x55f2f7685410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.358+0000 7fbaa88a5640 1 --2- 172.21.15.7:0/384588413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2f767de10 0x55f2f7682910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.358+0000 7fbaa90a6640 1 -- 172.21.15.7:0/384588413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2f767de10 msgr2=0x55f2f7682910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.358+0000 7fbaa90a6640 1 --2- 172.21.15.7:0/384588413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2f767de10 0x55f2f7682910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.358+0000 7fbaa90a6640 1 -- 172.21.15.7:0/384588413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2f7682f80 msgr2=0x55f2f7685410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.358+0000 7fbaa90a6640 1 --2- 172.21.15.7:0/384588413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2f7682f80 0x55f2f7685410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.358+0000 7fbaa90a6640 1 -- 172.21.15.7:0/384588413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2f75969b0 con 0x55f2f767d420 2024-06-10T01:23:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.358+0000 7fbaa3fff640 1 --2- 172.21.15.7:0/1868277519 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fba900a2c00 0x7fba900a50c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fba8c005fd0 tx=0x7fba8c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.358+0000 7fbaa90a6640 1 --2- 172.21.15.7:0/384588413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2f767d420 0x55f2f767d840 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fba9c006010 tx=0x7fba9c005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bc14f0671518980 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.358+0000 7fba9affd640 1 -- 172.21.15.7:0/384588413 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c031450 con 0x55f2f767d420 2024-06-10T01:23:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.358+0000 7fba9affd640 1 -- 172.21.15.7:0/384588413 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba9c036070 con 0x55f2f767d420 2024-06-10T01:23:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.358+0000 7fba9affd640 1 -- 172.21.15.7:0/384588413 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c004300 con 0x55f2f767d420 2024-06-10T01:23:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.359+0000 7fbaad812740 1 -- 172.21.15.7:0/384588413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2f767d420 msgr2=0x55f2f767d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.359+0000 7fbaad812740 1 --2- 172.21.15.7:0/384588413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2f767d420 0x55f2f767d840 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fba9c006010 tx=0x7fba9c005c30 comp rx=0 tx=0).stop 2024-06-10T01:23:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.359+0000 7fbaad812740 1 -- 172.21.15.7:0/384588413 shutdown_connections 2024-06-10T01:23:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.359+0000 7fbaad812740 1 --2- 172.21.15.7:0/384588413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2f7682f80 0x55f2f7685410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.359+0000 7fbaad812740 1 --2- 172.21.15.7:0/384588413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2f767de10 0x55f2f7682910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.359+0000 7fbaad812740 1 --2- 172.21.15.7:0/384588413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2f767d420 0x55f2f767d840 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.359+0000 7fbaad812740 1 -- 172.21.15.7:0/384588413 >> 172.21.15.7:0/384588413 conn(0x55f2f7674680 msgr2=0x55f2f7674a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.359+0000 7fbaad812740 1 -- 172.21.15.7:0/384588413 shutdown_connections 2024-06-10T01:23:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.359+0000 7fbaad812740 1 -- 172.21.15.7:0/384588413 wait complete. 2024-06-10T01:23:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.359+0000 7fbaad812740 1 -- start start 2024-06-10T01:23:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.360+0000 7fbaad812740 1 Processor -- start 2024-06-10T01:23:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.360+0000 7fbaad812740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2f767d420 0x55f2f76874f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.360+0000 7fbaad812740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2f767de10 0x55f2f7687a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.360+0000 7fbaad812740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2f7682f80 0x55f2f768ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.360+0000 7fbaad812740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f2f7685d50 con 0x55f2f7682f80 2024-06-10T01:23:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.360+0000 7fbaad812740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f2f7565ae0 con 0x55f2f767d420 2024-06-10T01:23:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.360+0000 7fbaad812740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f2f7685ed0 con 0x55f2f767de10 2024-06-10T01:23:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.360+0000 7fbaad812740 10 client.0 _ll_register_callbacks cb 0x55f2f759e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:23:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.360+0000 7fbaa90a6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2f7682f80 0x55f2f768ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.360+0000 7fbaa90a6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2f7682f80 0x55f2f768ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54232/0 (socket says 172.21.15.7:54232) 2024-06-10T01:23:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.360+0000 7fbaa90a6640 1 -- 172.21.15.7:0/2541996952 learned_addr learned my addr 172.21.15.7:0/2541996952 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.360+0000 7fbaa88a5640 1 --2- 172.21.15.7:0/2541996952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2f767d420 0x55f2f76874f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.360+0000 7fbaa3fff640 1 --2- 172.21.15.7:0/2541996952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2f767de10 0x55f2f7687a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.361+0000 7fbaa90a6640 1 -- 172.21.15.7:0/2541996952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2f767de10 msgr2=0x55f2f7687a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.361+0000 7fbaa90a6640 1 --2- 172.21.15.7:0/2541996952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2f767de10 0x55f2f7687a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.361+0000 7fbaa90a6640 1 -- 172.21.15.7:0/2541996952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2f767d420 msgr2=0x55f2f76874f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.361+0000 7fbaa90a6640 1 --2- 172.21.15.7:0/2541996952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2f767d420 0x55f2f76874f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.361+0000 7fbaa90a6640 1 -- 172.21.15.7:0/2541996952 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2f72f0210 con 0x55f2f7682f80 2024-06-10T01:23:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.361+0000 7fbaa88a5640 1 --2- 172.21.15.7:0/2541996952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2f767d420 0x55f2f76874f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:23:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.361+0000 7fbaa90a6640 1 --2- 172.21.15.7:0/2541996952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2f7682f80 0x55f2f768ea70 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fba9c009bf0 tx=0x7fba9c004df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708f793e7485c345 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.361+0000 7fba83fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:23:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.361+0000 7fba83fff640 1 -- 172.21.15.7:0/2541996952 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c003d80 con 0x55f2f7682f80 2024-06-10T01:23:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.361+0000 7fbaad812740 1 -- 172.21.15.7:0/2541996952 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba9c036070 con 0x55f2f7682f80 2024-06-10T01:23:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.361+0000 7fba83fff640 1 -- 172.21.15.7:0/2541996952 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fba9c0040c0 con 0x55f2f7682f80 2024-06-10T01:23:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.361+0000 7fba83fff640 1 -- 172.21.15.7:0/2541996952 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fba9c057050 con 0x55f2f7682f80 2024-06-10T01:23:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.361+0000 7fbaad812740 10 client.19779 fetch_fsmap learned FSMap version 50 2024-06-10T01:23:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.361+0000 7fbaad812740 1 -- 172.21.15.7:0/2541996952 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55f2f73bc620 con 0x55f2f7682f80 2024-06-10T01:23:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.361+0000 7fba83fff640 1 -- 172.21.15.7:0/2541996952 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba9c036070 con 0x55f2f7682f80 2024-06-10T01:23:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.361+0000 7fba83fff640 1 -- 172.21.15.7:0/2541996952 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7fba9c02f980 con 0x55f2f7682f80 2024-06-10T01:23:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.362+0000 7fbaad812740 10 client.19779 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:23:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.362+0000 7fbaad812740 10 client.19779 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:23:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.362+0000 7fbaad812740 20 client.19779 populate_metadata read hostname 'smithi007' 2024-06-10T01:23:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.362+0000 7fbaad812740 1 --2- 172.21.15.7:0/2541996952 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f2f7692f60 0x55f2f7695350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.362+0000 7fbaad812740 4 client.19779 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:23:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.362+0000 7fbaad812740 1 -- 172.21.15.7:0/2541996952 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2f75eff10 con 0x55f2f7692f60 2024-06-10T01:23:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.362+0000 7fbaa88a5640 1 --2- 172.21.15.7:0/2541996952 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f2f7692f60 0x55f2f7695350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.362+0000 7fba83fff640 10 client.19779 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.362+0000 7fbaa88a5640 1 --2- 172.21.15.7:0/2541996952 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f2f7692f60 0x55f2f7695350 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a593ed668f40ca29 server_cookie=5a514e6e36a49aaf in_seq=0 out_seq=0 2024-06-10T01:23:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.363+0000 7fba83fff640 1 -- 172.21.15.7:0/2541996952 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f2f75eff10 con 0x55f2f7692f60 2024-06-10T01:23:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.363+0000 7fba83fff640 10 client.19779 handle_command_reply: tid=0 2024-06-10T01:23:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.363+0000 7fbaad812740 1 client.19779 shutdown 2024-06-10T01:23:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.363+0000 7fbaad812740 1 -- 172.21.15.7:0/2541996952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2f7682f80 msgr2=0x55f2f768ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.363+0000 7fbaad812740 1 --2- 172.21.15.7:0/2541996952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2f7682f80 0x55f2f768ea70 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fba9c009bf0 tx=0x7fba9c004df0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.363+0000 7fbaad812740 1 -- 172.21.15.7:0/2541996952 shutdown_connections 2024-06-10T01:23:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.363+0000 7fbaad812740 1 --2- 172.21.15.7:0/2541996952 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f2f7692f60 0x55f2f7695350 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.364+0000 7fbaad812740 1 --2- 172.21.15.7:0/2541996952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2f7682f80 0x55f2f768ea70 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.364+0000 7fbaad812740 1 --2- 172.21.15.7:0/2541996952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2f767de10 0x55f2f7687a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.364+0000 7fba83fff640 0 client.19779 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.364+0000 7fbaad812740 1 --2- 172.21.15.7:0/2541996952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2f767d420 0x55f2f76874f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.364+0000 7fbaad812740 1 -- 172.21.15.7:0/2541996952 >> 172.21.15.7:0/2541996952 conn(0x55f2f7674680 msgr2=0x55f2f76737e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.364+0000 7fbaad812740 1 -- 172.21.15.7:0/2541996952 shutdown_connections 2024-06-10T01:23:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.364+0000 7fbaad812740 1 -- 172.21.15.7:0/2541996952 wait complete. 2024-06-10T01:23:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.364+0000 7fbaad812740 20 client.19779 trim_cache size 0 max 16384 2024-06-10T01:23:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.379+0000 7fbaad812740 1 Processor -- start 2024-06-10T01:23:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.379+0000 7fbaad812740 1 -- start start 2024-06-10T01:23:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.379+0000 7fbaad812740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2f767d420 0x55f2f7674a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.379+0000 7fbaad812740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2f767de10 0x55f2f768ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.379+0000 7fbaad812740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2f7682f80 0x55f2f76874f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.379+0000 7fbaad812740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa41aacb0 con 0x55f2f7682f80 2024-06-10T01:23:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.379+0000 7fbaad812740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa41a8580 con 0x55f2f767d420 2024-06-10T01:23:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.379+0000 7fbaad812740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa41a63d0 con 0x55f2f767de10 2024-06-10T01:23:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.380+0000 7fbaa90a6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2f7682f80 0x55f2f76874f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.380+0000 7fbaa90a6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2f7682f80 0x55f2f76874f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54242/0 (socket says 172.21.15.7:54242) 2024-06-10T01:23:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.380+0000 7fbaa90a6640 1 -- 172.21.15.7:0/62046856 learned_addr learned my addr 172.21.15.7:0/62046856 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.380+0000 7fbaa88a5640 1 --2- 172.21.15.7:0/62046856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2f767d420 0x55f2f7674a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.380+0000 7fbaa90a6640 1 -- 172.21.15.7:0/62046856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2f767de10 msgr2=0x55f2f768ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.380+0000 7fbaa3fff640 1 --2- 172.21.15.7:0/62046856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2f767de10 0x55f2f768ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.380+0000 7fbaa90a6640 1 --2- 172.21.15.7:0/62046856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2f767de10 0x55f2f768ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.380+0000 7fbaa90a6640 1 -- 172.21.15.7:0/62046856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2f767d420 msgr2=0x55f2f7674a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.380+0000 7fbaa90a6640 1 --2- 172.21.15.7:0/62046856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2f767d420 0x55f2f7674a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.380+0000 7fbaa90a6640 1 -- 172.21.15.7:0/62046856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2f767d800 con 0x55f2f7682f80 2024-06-10T01:23:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.380+0000 7fbaa88a5640 1 --2- 172.21.15.7:0/62046856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2f767d420 0x55f2f7674a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:23:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.380+0000 7fbaa90a6640 1 --2- 172.21.15.7:0/62046856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2f7682f80 0x55f2f76874f0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fba9c02fd50 tx=0x7fba9c02f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e61ae27e46582c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.380+0000 7fba9a7fc640 1 -- 172.21.15.7:0/62046856 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c031dd0 con 0x55f2f7682f80 2024-06-10T01:23:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.380+0000 7fba9a7fc640 1 -- 172.21.15.7:0/62046856 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba9c036070 con 0x55f2f7682f80 2024-06-10T01:23:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.380+0000 7fba9a7fc640 1 -- 172.21.15.7:0/62046856 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c030d50 con 0x55f2f7682f80 2024-06-10T01:23:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.381+0000 7fbaad812740 1 -- 172.21.15.7:0/62046856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2f7682f80 msgr2=0x55f2f76874f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.381+0000 7fbaad812740 1 --2- 172.21.15.7:0/62046856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2f7682f80 0x55f2f76874f0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fba9c02fd50 tx=0x7fba9c02f390 comp rx=0 tx=0).stop 2024-06-10T01:23:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.381+0000 7fbaad812740 1 -- 172.21.15.7:0/62046856 shutdown_connections 2024-06-10T01:23:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.381+0000 7fbaad812740 1 --2- 172.21.15.7:0/62046856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2f7682f80 0x55f2f76874f0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.381+0000 7fbaad812740 1 --2- 172.21.15.7:0/62046856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2f767de10 0x55f2f768ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.381+0000 7fbaad812740 1 --2- 172.21.15.7:0/62046856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2f767d420 0x55f2f7674a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.381+0000 7fbaad812740 1 -- 172.21.15.7:0/62046856 >> 172.21.15.7:0/62046856 conn(0x55f2f7674680 msgr2=0x55f2f7682700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.381+0000 7fbaad812740 1 -- 172.21.15.7:0/62046856 shutdown_connections 2024-06-10T01:23:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.381+0000 7fbaad812740 1 -- 172.21.15.7:0/62046856 wait complete. 2024-06-10T01:23:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.381+0000 7fbaad812740 1 -- start start 2024-06-10T01:23:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.381+0000 7fbaad812740 1 Processor -- start 2024-06-10T01:23:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.382+0000 7fbaad812740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2f767d420 0x55f2f768e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.382+0000 7fbaad812740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2f767de10 0x55f2f768b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.382+0000 7fbaad812740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2f7682f80 0x55f2f768b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.382+0000 7fbaad812740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa41a6a40 con 0x55f2f7682f80 2024-06-10T01:23:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.382+0000 7fbaad812740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa41a7eb0 con 0x55f2f767d420 2024-06-10T01:23:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.382+0000 7fbaad812740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa41a7110 con 0x55f2f767de10 2024-06-10T01:23:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.382+0000 7fbaad812740 10 client.0 _ll_register_callbacks cb 0x7fba9c009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:23:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.382+0000 7fbaa90a6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2f7682f80 0x55f2f768b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.382+0000 7fbaa90a6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2f7682f80 0x55f2f768b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54248/0 (socket says 172.21.15.7:54248) 2024-06-10T01:23:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.382+0000 7fbaa90a6640 1 -- 172.21.15.7:0/135324991 learned_addr learned my addr 172.21.15.7:0/135324991 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.382+0000 7fbaa88a5640 1 --2- 172.21.15.7:0/135324991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2f767d420 0x55f2f768e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.382+0000 7fbaa3fff640 1 --2- 172.21.15.7:0/135324991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2f767de10 0x55f2f768b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.382+0000 7fbaa90a6640 1 -- 172.21.15.7:0/135324991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2f767de10 msgr2=0x55f2f768b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.382+0000 7fbaa90a6640 1 --2- 172.21.15.7:0/135324991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2f767de10 0x55f2f768b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.382+0000 7fbaa90a6640 1 -- 172.21.15.7:0/135324991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2f767d420 msgr2=0x55f2f768e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.382+0000 7fbaa90a6640 1 --2- 172.21.15.7:0/135324991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2f767d420 0x55f2f768e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.382+0000 7fbaa90a6640 1 -- 172.21.15.7:0/135324991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2f76fbbe0 con 0x55f2f7682f80 2024-06-10T01:23:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.382+0000 7fbaa90a6640 1 --2- 172.21.15.7:0/135324991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2f7682f80 0x55f2f768b990 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fba9c02fdd0 tx=0x7fba9c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=442a61fd646616a6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.382+0000 7fba83fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:23:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.382+0000 7fba83fff640 1 -- 172.21.15.7:0/135324991 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c060070 con 0x55f2f7682f80 2024-06-10T01:23:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.382+0000 7fba83fff640 1 -- 172.21.15.7:0/135324991 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fba9c030c50 con 0x55f2f7682f80 2024-06-10T01:23:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.383+0000 7fbaad812740 1 -- 172.21.15.7:0/135324991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba9c036070 con 0x55f2f7682f80 2024-06-10T01:23:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.383+0000 7fba83fff640 1 -- 172.21.15.7:0/135324991 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fba9c06c050 con 0x55f2f7682f80 2024-06-10T01:23:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.383+0000 7fbaad812740 10 client.19785 fetch_fsmap learned FSMap version 50 2024-06-10T01:23:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.383+0000 7fbaad812740 1 -- 172.21.15.7:0/135324991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55f2f76fce50 con 0x55f2f7682f80 2024-06-10T01:23:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.383+0000 7fba83fff640 1 -- 172.21.15.7:0/135324991 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba9c036070 con 0x55f2f7682f80 2024-06-10T01:23:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.383+0000 7fba83fff640 1 -- 172.21.15.7:0/135324991 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7fba9c05c640 con 0x55f2f7682f80 2024-06-10T01:23:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.383+0000 7fbaad812740 10 client.19785 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:23:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.383+0000 7fbaad812740 10 client.19785 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:23:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.383+0000 7fbaad812740 20 client.19785 populate_metadata read hostname 'smithi007' 2024-06-10T01:23:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.383+0000 7fbaad812740 1 --2- 172.21.15.7:0/135324991 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f2f76fd830 0x55f2f76ffc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.383+0000 7fbaad812740 4 client.19785 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:23:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.383+0000 7fbaad812740 1 -- 172.21.15.7:0/135324991 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f2f75e9c20 con 0x55f2f76fd830 2024-06-10T01:23:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.383+0000 7fbaa88a5640 1 --2- 172.21.15.7:0/135324991 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f2f76fd830 0x55f2f76ffc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.384+0000 7fbaa88a5640 1 --2- 172.21.15.7:0/135324991 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f2f76fd830 0x55f2f76ffc50 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee7e660e1ec670d1 server_cookie=76492862a0d1d58c in_seq=0 out_seq=0 2024-06-10T01:23:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.384+0000 7fba83fff640 10 client.19785 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:36.533 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 23.02 MB/sec execute 133 sec latency 9470.188 ms 2024-06-10T01:23:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.581+0000 7fba83fff640 1 -- 172.21.15.7:0/135324991 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f2f75e9c20 con 0x55f2f76fd830 2024-06-10T01:23:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.581+0000 7fba83fff640 10 client.19785 handle_command_reply: tid=0 2024-06-10T01:23:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.581+0000 7fbaad812740 1 client.19785 shutdown 2024-06-10T01:23:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.581+0000 7fbaad812740 1 -- 172.21.15.7:0/135324991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2f7682f80 msgr2=0x55f2f768b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.581+0000 7fbaad812740 1 --2- 172.21.15.7:0/135324991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2f7682f80 0x55f2f768b990 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fba9c02fdd0 tx=0x7fba9c000f40 comp rx=0 tx=0).stop 2024-06-10T01:23:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.581+0000 7fbaad812740 1 -- 172.21.15.7:0/135324991 shutdown_connections 2024-06-10T01:23:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.581+0000 7fbaad812740 1 --2- 172.21.15.7:0/135324991 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f2f76fd830 0x55f2f76ffc50 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.581+0000 7fbaad812740 1 --2- 172.21.15.7:0/135324991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2f7682f80 0x55f2f768b990 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.581+0000 7fbaad812740 1 --2- 172.21.15.7:0/135324991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2f767de10 0x55f2f768b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.581+0000 7fba83fff640 0 client.19785 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.581+0000 7fbaad812740 1 --2- 172.21.15.7:0/135324991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2f767d420 0x55f2f768e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.581+0000 7fbaad812740 1 -- 172.21.15.7:0/135324991 >> 172.21.15.7:0/135324991 conn(0x55f2f7674680 msgr2=0x55f2f7674a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.581+0000 7fbaad812740 1 -- 172.21.15.7:0/135324991 shutdown_connections 2024-06-10T01:23:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.581+0000 7fbaad812740 1 -- 172.21.15.7:0/135324991 wait complete. 2024-06-10T01:23:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.581+0000 7fbaad812740 20 client.19785 trim_cache size 0 max 16384 2024-06-10T01:23:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:23:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:23:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:23:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.582+0000 7fbaaab09640 1 -- 172.21.15.7:0/1868277519 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fba900a2c00 msgr2=0x7fba900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.582+0000 7fbaaab09640 1 --2- 172.21.15.7:0/1868277519 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fba900a2c00 0x7fba900a50c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fba8c005fd0 tx=0x7fba8c00b390 comp rx=0 tx=0).stop 2024-06-10T01:23:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.582+0000 7fbaaab09640 1 -- 172.21.15.7:0/1868277519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaa4102480 msgr2=0x7fbaa4197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.582+0000 7fbaaab09640 1 --2- 172.21.15.7:0/1868277519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaa4102480 0x7fbaa4197d00 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fba9400d950 tx=0x7fba9400de10 comp rx=0 tx=0).stop 2024-06-10T01:23:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.582+0000 7fbaaab09640 1 -- 172.21.15.7:0/1868277519 shutdown_connections 2024-06-10T01:23:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.582+0000 7fbaaab09640 1 --2- 172.21.15.7:0/1868277519 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fba900a2c00 0x7fba900a50c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.582+0000 7fbaaab09640 1 --2- 172.21.15.7:0/1868277519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaa41076e0 0x7fbaa419f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.582+0000 7fbaaab09640 1 --2- 172.21.15.7:0/1868277519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbaa4104db0 0x7fbaa4198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.582+0000 7fbaaab09640 1 --2- 172.21.15.7:0/1868277519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaa4102480 0x7fbaa4197d00 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.582+0000 7fbaaab09640 1 -- 172.21.15.7:0/1868277519 >> 172.21.15.7:0/1868277519 conn(0x7fbaa40fbe70 msgr2=0x7fbaa4102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.582+0000 7fbaaab09640 1 -- 172.21.15.7:0/1868277519 shutdown_connections 2024-06-10T01:23:36.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:36.582+0000 7fbaaab09640 1 -- 172.21.15.7:0/1868277519 wait complete. 2024-06-10T01:23:37.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:37 smithi175 ceph-mon[36636]: pgmap v961: 129 pgs: 129 active+clean; 551 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.5 MiB/s wr, 470 op/s 2024-06-10T01:23:37.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:37 smithi007 ceph-mon[32398]: pgmap v961: 129 pgs: 129 active+clean; 551 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.5 MiB/s wr, 470 op/s 2024-06-10T01:23:37.533 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 22.85 MB/sec execute 134 sec latency 10470.256 ms 2024-06-10T01:23:37.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:37 smithi078 ceph-mon[36683]: pgmap v961: 129 pgs: 129 active+clean; 551 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.5 MiB/s wr, 470 op/s 2024-06-10T01:23:38.533 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 22.68 MB/sec execute 135 sec latency 11470.319 ms 2024-06-10T01:23:39.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:39 smithi175 ceph-mon[36636]: pgmap v962: 129 pgs: 129 active+clean; 551 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.2 MiB/s wr, 192 op/s 2024-06-10T01:23:39.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:39 smithi007 ceph-mon[32398]: pgmap v962: 129 pgs: 129 active+clean; 551 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.2 MiB/s wr, 192 op/s 2024-06-10T01:23:39.533 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 22.51 MB/sec execute 136 sec latency 12470.351 ms 2024-06-10T01:23:39.578 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:39 smithi078 ceph-mon[36683]: pgmap v962: 129 pgs: 129 active+clean; 551 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.2 MiB/s wr, 192 op/s 2024-06-10T01:23:40.533 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 22.35 MB/sec execute 137 sec latency 13470.385 ms 2024-06-10T01:23:41.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:41 smithi175 ceph-mon[36636]: pgmap v963: 129 pgs: 129 active+clean; 551 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.1 MiB/s wr, 208 op/s 2024-06-10T01:23:41.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:41 smithi007 ceph-mon[32398]: pgmap v963: 129 pgs: 129 active+clean; 551 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.1 MiB/s wr, 208 op/s 2024-06-10T01:23:41.533 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 22.19 MB/sec execute 138 sec latency 14470.471 ms 2024-06-10T01:23:41.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:41 smithi078 ceph-mon[36683]: pgmap v963: 129 pgs: 129 active+clean; 551 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.1 MiB/s wr, 208 op/s 2024-06-10T01:23:42.533 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 22.03 MB/sec execute 139 sec latency 15470.521 ms 2024-06-10T01:23:43.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:43 smithi175 ceph-mon[36636]: pgmap v964: 129 pgs: 129 active+clean; 559 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-06-10T01:23:43.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:43 smithi007 ceph-mon[32398]: pgmap v964: 129 pgs: 129 active+clean; 559 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-06-10T01:23:43.533 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 21.87 MB/sec execute 140 sec latency 16470.616 ms 2024-06-10T01:23:43.578 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:43 smithi078 ceph-mon[36683]: pgmap v964: 129 pgs: 129 active+clean; 559 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-06-10T01:23:44.533 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 21.72 MB/sec execute 141 sec latency 17470.667 ms 2024-06-10T01:23:45.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:45 smithi175 ceph-mon[36636]: pgmap v965: 129 pgs: 129 active+clean; 560 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 1.1 MiB/s wr, 91 op/s 2024-06-10T01:23:45.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:45 smithi007 ceph-mon[32398]: pgmap v965: 129 pgs: 129 active+clean; 560 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 1.1 MiB/s wr, 91 op/s 2024-06-10T01:23:45.533 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 21.56 MB/sec execute 142 sec latency 18470.750 ms 2024-06-10T01:23:45.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:45 smithi078 ceph-mon[36683]: pgmap v965: 129 pgs: 129 active+clean; 560 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 1.1 MiB/s wr, 91 op/s 2024-06-10T01:23:46.533 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 21.41 MB/sec execute 143 sec latency 19470.834 ms 2024-06-10T01:23:47.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:47 smithi175 ceph-mon[36636]: pgmap v966: 129 pgs: 129 active+clean; 564 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 4.0 MiB/s wr, 193 op/s 2024-06-10T01:23:47.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:47 smithi007 ceph-mon[32398]: pgmap v966: 129 pgs: 129 active+clean; 564 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 4.0 MiB/s wr, 193 op/s 2024-06-10T01:23:47.533 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 21.26 MB/sec execute 144 sec latency 20470.932 ms 2024-06-10T01:23:47.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:47 smithi078 ceph-mon[36683]: pgmap v966: 129 pgs: 129 active+clean; 564 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 4.0 MiB/s wr, 193 op/s 2024-06-10T01:23:48.534 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 21.12 MB/sec execute 145 sec latency 21471.034 ms 2024-06-10T01:23:48.551 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:23:48.816 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:23:49.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:49 smithi007 ceph-mon[32398]: pgmap v967: 129 pgs: 129 active+clean; 559 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 4.0 MiB/s wr, 199 op/s 2024-06-10T01:23:49.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:49 smithi175 ceph-mon[36636]: pgmap v967: 129 pgs: 129 active+clean; 559 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 4.0 MiB/s wr, 199 op/s 2024-06-10T01:23:49.534 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 20.97 MB/sec execute 146 sec latency 22471.150 ms 2024-06-10T01:23:49.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:49 smithi078 ceph-mon[36683]: pgmap v967: 129 pgs: 129 active+clean; 559 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 4.0 MiB/s wr, 199 op/s 2024-06-10T01:23:49.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.678+0000 7fe9a0989640 1 -- 172.21.15.7:0/1319628239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe99c1035c0 msgr2=0x7fe99c109e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.678+0000 7fe9a0989640 1 --2- 172.21.15.7:0/1319628239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe99c1035c0 0x7fe99c109e50 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fe994009ca0 tx=0x7fe99402a8c0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.678+0000 7fe9a0989640 1 -- 172.21.15.7:0/1319628239 shutdown_connections 2024-06-10T01:23:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.678+0000 7fe9a0989640 1 --2- 172.21.15.7:0/1319628239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe99c1035c0 0x7fe99c109e50 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.678+0000 7fe9a0989640 1 --2- 172.21.15.7:0/1319628239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe99c102c00 0x7fe99c103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.678+0000 7fe9a0989640 1 --2- 172.21.15.7:0/1319628239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe99c101a10 0x7fe99c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.678+0000 7fe9a0989640 1 -- 172.21.15.7:0/1319628239 >> 172.21.15.7:0/1319628239 conn(0x7fe99c0fcf80 msgr2=0x7fe99c0ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.678+0000 7fe9a0989640 1 -- 172.21.15.7:0/1319628239 shutdown_connections 2024-06-10T01:23:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.678+0000 7fe9a0989640 1 -- 172.21.15.7:0/1319628239 wait complete. 2024-06-10T01:23:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.679+0000 7fe9a0989640 1 Processor -- start 2024-06-10T01:23:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.679+0000 7fe9a0989640 1 -- start start 2024-06-10T01:23:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.679+0000 7fe9a0989640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe99c101a10 0x7fe99c197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.679+0000 7fe9a0989640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe99c102c00 0x7fe99c198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.679+0000 7fe9a0989640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe99c1035c0 0x7fe99c19f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.679+0000 7fe9a0989640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c10d690 con 0x7fe99c101a10 2024-06-10T01:23:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.679+0000 7fe9a0989640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c10d510 con 0x7fe99c1035c0 2024-06-10T01:23:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.679+0000 7fe9a0989640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c10d810 con 0x7fe99c102c00 2024-06-10T01:23:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.679+0000 7fe99a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe99c101a10 0x7fe99c197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.679+0000 7fe99a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe99c101a10 0x7fe99c197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34810/0 (socket says 172.21.15.7:34810) 2024-06-10T01:23:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.679+0000 7fe99a59c640 1 -- 172.21.15.7:0/4142995687 learned_addr learned my addr 172.21.15.7:0/4142995687 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.679+0000 7fe993fff640 1 --2- 172.21.15.7:0/4142995687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe99c102c00 0x7fe99c198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.679+0000 7fe99ad9d640 1 --2- 172.21.15.7:0/4142995687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe99c1035c0 0x7fe99c19f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.680+0000 7fe99a59c640 1 -- 172.21.15.7:0/4142995687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe99c102c00 msgr2=0x7fe99c198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.680+0000 7fe99a59c640 1 --2- 172.21.15.7:0/4142995687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe99c102c00 0x7fe99c198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.680+0000 7fe99a59c640 1 -- 172.21.15.7:0/4142995687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe99c1035c0 msgr2=0x7fe99c19f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.680+0000 7fe99a59c640 1 --2- 172.21.15.7:0/4142995687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe99c1035c0 0x7fe99c19f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.680+0000 7fe99a59c640 1 -- 172.21.15.7:0/4142995687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe99c19f900 con 0x7fe99c101a10 2024-06-10T01:23:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.680+0000 7fe993fff640 1 --2- 172.21.15.7:0/4142995687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe99c102c00 0x7fe99c198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:23:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.680+0000 7fe99a59c640 1 --2- 172.21.15.7:0/4142995687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe99c101a10 0x7fe99c197cf0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fe98c00e9c0 tx=0x7fe98c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.680+0000 7fe9937fe640 1 -- 172.21.15.7:0/4142995687 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c00cd80 con 0x7fe99c101a10 2024-06-10T01:23:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.680+0000 7fe9a0989640 1 -- 172.21.15.7:0/4142995687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe99c19fbf0 con 0x7fe99c101a10 2024-06-10T01:23:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.680+0000 7fe9937fe640 1 -- 172.21.15.7:0/4142995687 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98c0035f0 con 0x7fe99c101a10 2024-06-10T01:23:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.680+0000 7fe9937fe640 1 -- 172.21.15.7:0/4142995687 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c010690 con 0x7fe99c101a10 2024-06-10T01:23:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.680+0000 7fe9a0989640 1 -- 172.21.15.7:0/4142995687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe99c1a0140 con 0x7fe99c101a10 2024-06-10T01:23:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.682+0000 7fe9937fe640 1 -- 172.21.15.7:0/4142995687 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fe98c0037a0 con 0x7fe99c101a10 2024-06-10T01:23:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.683+0000 7fe9a3772740 1 Processor -- start 2024-06-10T01:23:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.683+0000 7fe9a3772740 1 -- start start 2024-06-10T01:23:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.683+0000 7fe9a3772740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55637b55e420 0x55637b55e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.683+0000 7fe9a3772740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55637b55ee10 0x55637b563910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.683+0000 7fe9a3772740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55637b563f80 0x55637b566410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.683+0000 7fe9a3772740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55637b4bd4f0 con 0x55637b563f80 2024-06-10T01:23:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.683+0000 7fe9a3772740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55637b45b870 con 0x55637b55ee10 2024-06-10T01:23:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.683+0000 7fe9a3772740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55637b4b94b0 con 0x55637b55e420 2024-06-10T01:23:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.683+0000 7fe993fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55637b563f80 0x55637b566410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.683+0000 7fe993fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55637b563f80 0x55637b566410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34814/0 (socket says 172.21.15.7:34814) 2024-06-10T01:23:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.683+0000 7fe993fff640 1 -- 172.21.15.7:0/702321216 learned_addr learned my addr 172.21.15.7:0/702321216 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.683+0000 7fe99a59c640 1 --2- 172.21.15.7:0/702321216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55637b55ee10 0x55637b563910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:49.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.683+0000 7fe99ad9d640 1 --2- 172.21.15.7:0/702321216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55637b55e420 0x55637b55e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:49.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.683+0000 7fe9937fe640 1 --2- 172.21.15.7:0/4142995687 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe9880a2b80 0x7fe9880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:49.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.684+0000 7fe99ad9d640 1 --2- 172.21.15.7:0/4142995687 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe9880a2b80 0x7fe9880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:49.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.684+0000 7fe993fff640 1 -- 172.21.15.7:0/702321216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55637b55e420 msgr2=0x55637b55e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:49.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.684+0000 7fe993fff640 1 --2- 172.21.15.7:0/702321216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55637b55e420 0x55637b55e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.684+0000 7fe993fff640 1 -- 172.21.15.7:0/702321216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55637b55ee10 msgr2=0x55637b563910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:49.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.684+0000 7fe993fff640 1 --2- 172.21.15.7:0/702321216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55637b55ee10 0x55637b563910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.684+0000 7fe993fff640 1 -- 172.21.15.7:0/702321216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55637b4779b0 con 0x55637b563f80 2024-06-10T01:23:49.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.684+0000 7fe9937fe640 1 -- 172.21.15.7:0/4142995687 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fe98c0cfa00 con 0x7fe99c101a10 2024-06-10T01:23:49.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.684+0000 7fe993fff640 1 --2- 172.21.15.7:0/702321216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55637b563f80 0x55637b566410 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fe984009d40 tx=0x7fe9840117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b0d85fb1d9b9a7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:49.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.684+0000 7fe990ff9640 1 -- 172.21.15.7:0/702321216 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe984012650 con 0x55637b563f80 2024-06-10T01:23:49.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.684+0000 7fe99ad9d640 1 --2- 172.21.15.7:0/4142995687 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe9880a2b80 0x7fe9880a5040 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe994005e30 tx=0x7fe99402a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:49.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.684+0000 7fe990ff9640 1 -- 172.21.15.7:0/702321216 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe984009e30 con 0x55637b563f80 2024-06-10T01:23:49.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.684+0000 7fe990ff9640 1 -- 172.21.15.7:0/702321216 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98400fcc0 con 0x55637b563f80 2024-06-10T01:23:49.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.685+0000 7fe9a3772740 1 -- 172.21.15.7:0/702321216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55637b563f80 msgr2=0x55637b566410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:49.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.685+0000 7fe9a3772740 1 --2- 172.21.15.7:0/702321216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55637b563f80 0x55637b566410 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fe984009d40 tx=0x7fe9840117b0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.685+0000 7fe9a3772740 1 -- 172.21.15.7:0/702321216 shutdown_connections 2024-06-10T01:23:49.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.685+0000 7fe9a3772740 1 --2- 172.21.15.7:0/702321216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55637b563f80 0x55637b566410 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.685+0000 7fe9a3772740 1 --2- 172.21.15.7:0/702321216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55637b55ee10 0x55637b563910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.685+0000 7fe9a3772740 1 --2- 172.21.15.7:0/702321216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55637b55e420 0x55637b55e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.685+0000 7fe9a3772740 1 -- 172.21.15.7:0/702321216 >> 172.21.15.7:0/702321216 conn(0x55637b555680 msgr2=0x55637b555a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:49.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.685+0000 7fe9a3772740 1 -- 172.21.15.7:0/702321216 shutdown_connections 2024-06-10T01:23:49.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.685+0000 7fe9a3772740 1 -- 172.21.15.7:0/702321216 wait complete. 2024-06-10T01:23:49.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.685+0000 7fe9a3772740 1 -- start start 2024-06-10T01:23:49.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.685+0000 7fe9a3772740 1 Processor -- start 2024-06-10T01:23:49.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.686+0000 7fe9a3772740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55637b55e420 0x55637b568630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:49.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.686+0000 7fe9a3772740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55637b55ee10 0x55637b568b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:49.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.686+0000 7fe9a3772740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55637b563f80 0x55637b56fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:49.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.686+0000 7fe9a3772740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55637b566f30 con 0x55637b563f80 2024-06-10T01:23:49.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.686+0000 7fe9a3772740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55637b446ae0 con 0x55637b55ee10 2024-06-10T01:23:49.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.686+0000 7fe9a3772740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55637b5670b0 con 0x55637b55e420 2024-06-10T01:23:49.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.686+0000 7fe9a3772740 10 client.0 _ll_register_callbacks cb 0x55637b47f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:23:49.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.686+0000 7fe99ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55637b563f80 0x55637b56fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.686+0000 7fe99ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55637b563f80 0x55637b56fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34824/0 (socket says 172.21.15.7:34824) 2024-06-10T01:23:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.686+0000 7fe99ad9d640 1 -- 172.21.15.7:0/2087186569 learned_addr learned my addr 172.21.15.7:0/2087186569 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.686+0000 7fe993fff640 1 --2- 172.21.15.7:0/2087186569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55637b55ee10 0x55637b568b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.686+0000 7fe99a59c640 1 --2- 172.21.15.7:0/2087186569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55637b55e420 0x55637b568630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.686+0000 7fe99ad9d640 1 -- 172.21.15.7:0/2087186569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55637b55e420 msgr2=0x55637b568630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.686+0000 7fe99ad9d640 1 --2- 172.21.15.7:0/2087186569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55637b55e420 0x55637b568630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.686+0000 7fe99ad9d640 1 -- 172.21.15.7:0/2087186569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55637b55ee10 msgr2=0x55637b568b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.686+0000 7fe99ad9d640 1 --2- 172.21.15.7:0/2087186569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55637b55ee10 0x55637b568b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.686+0000 7fe99ad9d640 1 -- 172.21.15.7:0/2087186569 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55637b1d1210 con 0x55637b563f80 2024-06-10T01:23:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.686+0000 7fe993fff640 1 --2- 172.21.15.7:0/2087186569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55637b55ee10 0x55637b568b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:23:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.686+0000 7fe99ad9d640 1 --2- 172.21.15.7:0/2087186569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55637b563f80 0x55637b56fbb0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fe994035000 tx=0x7fe9940095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cde57e19f3f7330 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.686+0000 7fe981ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:23:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.686+0000 7fe981ffb640 1 -- 172.21.15.7:0/2087186569 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99404e020 con 0x55637b563f80 2024-06-10T01:23:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.686+0000 7fe981ffb640 1 -- 172.21.15.7:0/2087186569 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fe9940047c0 con 0x55637b563f80 2024-06-10T01:23:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.686+0000 7fe9a3772740 1 -- 172.21.15.7:0/2087186569 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe984009e30 con 0x55637b563f80 2024-06-10T01:23:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.687+0000 7fe981ffb640 1 -- 172.21.15.7:0/2087186569 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fe994059050 con 0x55637b563f80 2024-06-10T01:23:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.687+0000 7fe9a3772740 10 client.19797 fetch_fsmap learned FSMap version 50 2024-06-10T01:23:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.687+0000 7fe9a3772740 1 -- 172.21.15.7:0/2087186569 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55637b29d620 con 0x55637b563f80 2024-06-10T01:23:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.687+0000 7fe981ffb640 1 -- 172.21.15.7:0/2087186569 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe984009e30 con 0x55637b563f80 2024-06-10T01:23:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.687+0000 7fe981ffb640 1 -- 172.21.15.7:0/2087186569 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7fe994049040 con 0x55637b563f80 2024-06-10T01:23:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.687+0000 7fe9a3772740 10 client.19797 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:23:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.687+0000 7fe9a3772740 10 client.19797 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:23:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.687+0000 7fe9a3772740 20 client.19797 populate_metadata read hostname 'smithi007' 2024-06-10T01:23:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.687+0000 7fe9a3772740 1 --2- 172.21.15.7:0/2087186569 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55637b5745c0 0x55637b576a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.687+0000 7fe9a3772740 4 client.19797 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:23:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.687+0000 7fe9a3772740 1 -- 172.21.15.7:0/2087186569 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55637b4d0f10 con 0x55637b5745c0 2024-06-10T01:23:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.687+0000 7fe99a59c640 1 --2- 172.21.15.7:0/2087186569 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55637b5745c0 0x55637b576a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.688+0000 7fe99a59c640 1 --2- 172.21.15.7:0/2087186569 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55637b5745c0 0x55637b576a20 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44848e0722ffefe7 server_cookie=44bab439e832d237 in_seq=0 out_seq=0 2024-06-10T01:23:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.688+0000 7fe981ffb640 10 client.19797 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.688+0000 7fe981ffb640 1 -- 172.21.15.7:0/2087186569 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55637b4d0f10 con 0x55637b5745c0 2024-06-10T01:23:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.688+0000 7fe981ffb640 10 client.19797 handle_command_reply: tid=0 2024-06-10T01:23:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.688+0000 7fe9a3772740 1 client.19797 shutdown 2024-06-10T01:23:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.688+0000 7fe9a3772740 1 -- 172.21.15.7:0/2087186569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55637b563f80 msgr2=0x55637b56fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.688+0000 7fe9a3772740 1 --2- 172.21.15.7:0/2087186569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55637b563f80 0x55637b56fbb0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fe994035000 tx=0x7fe9940095c0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.689+0000 7fe9a3772740 1 -- 172.21.15.7:0/2087186569 shutdown_connections 2024-06-10T01:23:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.689+0000 7fe9a3772740 1 --2- 172.21.15.7:0/2087186569 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55637b5745c0 0x55637b576a20 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.689+0000 7fe9a3772740 1 --2- 172.21.15.7:0/2087186569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55637b563f80 0x55637b56fbb0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.689+0000 7fe9a3772740 1 --2- 172.21.15.7:0/2087186569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55637b55ee10 0x55637b568b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.689+0000 7fe981ffb640 0 client.19797 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.689+0000 7fe9a3772740 1 --2- 172.21.15.7:0/2087186569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55637b55e420 0x55637b568630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.689+0000 7fe9a3772740 1 -- 172.21.15.7:0/2087186569 >> 172.21.15.7:0/2087186569 conn(0x55637b555680 msgr2=0x55637b5547e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.689+0000 7fe9a3772740 1 -- 172.21.15.7:0/2087186569 shutdown_connections 2024-06-10T01:23:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.689+0000 7fe9a3772740 1 -- 172.21.15.7:0/2087186569 wait complete. 2024-06-10T01:23:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.689+0000 7fe9a3772740 20 client.19797 trim_cache size 0 max 16384 2024-06-10T01:23:49.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.704+0000 7fe9a3772740 1 Processor -- start 2024-06-10T01:23:49.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.704+0000 7fe9a3772740 1 -- start start 2024-06-10T01:23:49.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.704+0000 7fe9a3772740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55637b55e420 0x55637b568630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:49.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.704+0000 7fe9a3772740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55637b55ee10 0x55637b568b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:49.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.704+0000 7fe9a3772740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55637b563f80 0x55637b56fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:49.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.704+0000 7fe9a3772740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c1aace0 con 0x55637b55ee10 2024-06-10T01:23:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.704+0000 7fe9a3772740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c1a85b0 con 0x55637b55e420 2024-06-10T01:23:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.704+0000 7fe9a3772740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c1a6400 con 0x55637b563f80 2024-06-10T01:23:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.704+0000 7fe993fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55637b55ee10 0x55637b568b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.704+0000 7fe993fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55637b55ee10 0x55637b568b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34838/0 (socket says 172.21.15.7:34838) 2024-06-10T01:23:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.704+0000 7fe993fff640 1 -- 172.21.15.7:0/2994317485 learned_addr learned my addr 172.21.15.7:0/2994317485 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.704+0000 7fe99a59c640 1 --2- 172.21.15.7:0/2994317485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55637b55e420 0x55637b568630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.704+0000 7fe99ad9d640 1 --2- 172.21.15.7:0/2994317485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55637b563f80 0x55637b56fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.704+0000 7fe993fff640 1 -- 172.21.15.7:0/2994317485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55637b563f80 msgr2=0x55637b56fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.704+0000 7fe993fff640 1 --2- 172.21.15.7:0/2994317485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55637b563f80 0x55637b56fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.705+0000 7fe993fff640 1 -- 172.21.15.7:0/2994317485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55637b55e420 msgr2=0x55637b568630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.705+0000 7fe993fff640 1 --2- 172.21.15.7:0/2994317485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55637b55e420 0x55637b568630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.705+0000 7fe993fff640 1 -- 172.21.15.7:0/2994317485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55637b5554b0 con 0x55637b55ee10 2024-06-10T01:23:49.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.705+0000 7fe993fff640 1 --2- 172.21.15.7:0/2994317485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55637b55ee10 0x55637b568b70 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x55637b556550 tx=0x7fe984009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d452792b9418fe server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:49.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.705+0000 7fe983fff640 1 -- 172.21.15.7:0/2994317485 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98402a070 con 0x55637b55ee10 2024-06-10T01:23:49.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.705+0000 7fe983fff640 1 -- 172.21.15.7:0/2994317485 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe984005520 con 0x55637b55ee10 2024-06-10T01:23:49.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.705+0000 7fe983fff640 1 -- 172.21.15.7:0/2994317485 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe984004660 con 0x55637b55ee10 2024-06-10T01:23:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.705+0000 7fe9a3772740 1 -- 172.21.15.7:0/2994317485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55637b55ee10 msgr2=0x55637b568b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.705+0000 7fe9a3772740 1 --2- 172.21.15.7:0/2994317485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55637b55ee10 0x55637b568b70 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x55637b556550 tx=0x7fe984009550 comp rx=0 tx=0).stop 2024-06-10T01:23:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.705+0000 7fe9a3772740 1 -- 172.21.15.7:0/2994317485 shutdown_connections 2024-06-10T01:23:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.706+0000 7fe9a3772740 1 --2- 172.21.15.7:0/2994317485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55637b563f80 0x55637b56fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.706+0000 7fe9a3772740 1 --2- 172.21.15.7:0/2994317485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55637b55ee10 0x55637b568b70 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.706+0000 7fe9a3772740 1 --2- 172.21.15.7:0/2994317485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55637b55e420 0x55637b568630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.706+0000 7fe9a3772740 1 -- 172.21.15.7:0/2994317485 >> 172.21.15.7:0/2994317485 conn(0x55637b555680 msgr2=0x55637b563660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.706+0000 7fe9a3772740 1 -- 172.21.15.7:0/2994317485 shutdown_connections 2024-06-10T01:23:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.706+0000 7fe9a3772740 1 -- 172.21.15.7:0/2994317485 wait complete. 2024-06-10T01:23:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.706+0000 7fe9a3772740 1 -- start start 2024-06-10T01:23:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.706+0000 7fe9a3772740 1 Processor -- start 2024-06-10T01:23:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.706+0000 7fe9a3772740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55637b55e420 0x55637b56f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.706+0000 7fe9a3772740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55637b55ee10 0x55637b56c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.706+0000 7fe9a3772740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55637b563f80 0x55637b56c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.706+0000 7fe9a3772740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c1a6a70 con 0x55637b563f80 2024-06-10T01:23:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.706+0000 7fe9a3772740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c1a7ee0 con 0x55637b55ee10 2024-06-10T01:23:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.706+0000 7fe9a3772740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c1a7140 con 0x55637b55e420 2024-06-10T01:23:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.706+0000 7fe9a3772740 10 client.0 _ll_register_callbacks cb 0x7fe984005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:23:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.706+0000 7fe99ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55637b563f80 0x55637b56c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.706+0000 7fe99ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55637b563f80 0x55637b56c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34854/0 (socket says 172.21.15.7:34854) 2024-06-10T01:23:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.706+0000 7fe99ad9d640 1 -- 172.21.15.7:0/3593957208 learned_addr learned my addr 172.21.15.7:0/3593957208 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.706+0000 7fe993fff640 1 --2- 172.21.15.7:0/3593957208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55637b55ee10 0x55637b56c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.706+0000 7fe99a59c640 1 --2- 172.21.15.7:0/3593957208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55637b55e420 0x55637b56f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.706+0000 7fe99ad9d640 1 -- 172.21.15.7:0/3593957208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55637b55e420 msgr2=0x55637b56f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.706+0000 7fe99ad9d640 1 --2- 172.21.15.7:0/3593957208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55637b55e420 0x55637b56f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.707+0000 7fe99ad9d640 1 -- 172.21.15.7:0/3593957208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55637b55ee10 msgr2=0x55637b56c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.707+0000 7fe99ad9d640 1 --2- 172.21.15.7:0/3593957208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55637b55ee10 0x55637b56c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.707+0000 7fe99ad9d640 1 -- 172.21.15.7:0/3593957208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55637b5671f0 con 0x55637b563f80 2024-06-10T01:23:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.707+0000 7fe993fff640 1 --2- 172.21.15.7:0/3593957208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55637b55ee10 0x55637b56c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:23:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.707+0000 7fe99a59c640 1 --2- 172.21.15.7:0/3593957208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55637b55e420 0x55637b56f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:23:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.707+0000 7fe99ad9d640 1 --2- 172.21.15.7:0/3593957208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55637b563f80 0x55637b56c8f0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fe994031ae0 tx=0x7fe994004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d5b0177ee4aa28 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.707+0000 7fe981ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:23:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.707+0000 7fe981ffb640 1 -- 172.21.15.7:0/3593957208 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe994062060 con 0x55637b563f80 2024-06-10T01:23:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.707+0000 7fe981ffb640 1 -- 172.21.15.7:0/3593957208 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fe99402fa20 con 0x55637b563f80 2024-06-10T01:23:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.707+0000 7fe9a3772740 1 -- 172.21.15.7:0/3593957208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe984005520 con 0x55637b563f80 2024-06-10T01:23:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.707+0000 7fe981ffb640 1 -- 172.21.15.7:0/3593957208 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fe994069050 con 0x55637b563f80 2024-06-10T01:23:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.707+0000 7fe9a3772740 10 client.19803 fetch_fsmap learned FSMap version 50 2024-06-10T01:23:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.707+0000 7fe9a3772740 1 -- 172.21.15.7:0/3593957208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55637b5de180 con 0x55637b563f80 2024-06-10T01:23:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.707+0000 7fe981ffb640 1 -- 172.21.15.7:0/3593957208 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe984005520 con 0x55637b563f80 2024-06-10T01:23:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.707+0000 7fe981ffb640 1 -- 172.21.15.7:0/3593957208 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7fe994049040 con 0x55637b563f80 2024-06-10T01:23:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.707+0000 7fe9a3772740 10 client.19803 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:23:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.707+0000 7fe9a3772740 10 client.19803 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:23:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.707+0000 7fe9a3772740 20 client.19803 populate_metadata read hostname 'smithi007' 2024-06-10T01:23:49.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.707+0000 7fe9a3772740 1 --2- 172.21.15.7:0/3593957208 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55637b5deb60 0x55637b5e0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:49.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.707+0000 7fe9a3772740 4 client.19803 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:23:49.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.707+0000 7fe9a3772740 1 -- 172.21.15.7:0/3593957208 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55637b4cac20 con 0x55637b5deb60 2024-06-10T01:23:49.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.708+0000 7fe99a59c640 1 --2- 172.21.15.7:0/3593957208 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55637b5deb60 0x55637b5e0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:49.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.708+0000 7fe981ffb640 10 client.19803 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:49.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.708+0000 7fe99a59c640 1 --2- 172.21.15.7:0/3593957208 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55637b5deb60 0x55637b5e0f80 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b94b6d0d28de15 server_cookie=cdd1e752e80fba9a in_seq=0 out_seq=0 2024-06-10T01:23:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.774+0000 7fe981ffb640 1 -- 172.21.15.7:0/3593957208 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55637b4cac20 con 0x55637b5deb60 2024-06-10T01:23:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.774+0000 7fe981ffb640 10 client.19803 handle_command_reply: tid=0 2024-06-10T01:23:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.774+0000 7fe9a3772740 1 client.19803 shutdown 2024-06-10T01:23:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.774+0000 7fe9a3772740 1 -- 172.21.15.7:0/3593957208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55637b563f80 msgr2=0x55637b56c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.774+0000 7fe9a3772740 1 --2- 172.21.15.7:0/3593957208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55637b563f80 0x55637b56c8f0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fe994031ae0 tx=0x7fe994004640 comp rx=0 tx=0).stop 2024-06-10T01:23:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.774+0000 7fe9a3772740 1 -- 172.21.15.7:0/3593957208 shutdown_connections 2024-06-10T01:23:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.774+0000 7fe9a3772740 1 --2- 172.21.15.7:0/3593957208 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55637b5deb60 0x55637b5e0f80 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.774+0000 7fe9a3772740 1 --2- 172.21.15.7:0/3593957208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55637b563f80 0x55637b56c8f0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.774+0000 7fe9a3772740 1 --2- 172.21.15.7:0/3593957208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55637b55ee10 0x55637b56c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.774+0000 7fe981ffb640 0 client.19803 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.774+0000 7fe9a3772740 1 --2- 172.21.15.7:0/3593957208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55637b55e420 0x55637b56f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.774+0000 7fe9a3772740 1 -- 172.21.15.7:0/3593957208 >> 172.21.15.7:0/3593957208 conn(0x55637b555680 msgr2=0x55637b568630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.774+0000 7fe9a3772740 1 -- 172.21.15.7:0/3593957208 shutdown_connections 2024-06-10T01:23:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.774+0000 7fe9a3772740 1 -- 172.21.15.7:0/3593957208 wait complete. 2024-06-10T01:23:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.774+0000 7fe9a3772740 20 client.19803 trim_cache size 0 max 16384 2024-06-10T01:23:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:23:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:23:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:23:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.775+0000 7fe9a0989640 1 -- 172.21.15.7:0/4142995687 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe9880a2b80 msgr2=0x7fe9880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.775+0000 7fe9a0989640 1 --2- 172.21.15.7:0/4142995687 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe9880a2b80 0x7fe9880a5040 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe994005e30 tx=0x7fe99402a8c0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.775+0000 7fe9a0989640 1 -- 172.21.15.7:0/4142995687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe99c101a10 msgr2=0x7fe99c197cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.775+0000 7fe9a0989640 1 --2- 172.21.15.7:0/4142995687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe99c101a10 0x7fe99c197cf0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fe98c00e9c0 tx=0x7fe98c00ee80 comp rx=0 tx=0).stop 2024-06-10T01:23:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.775+0000 7fe9a0989640 1 -- 172.21.15.7:0/4142995687 shutdown_connections 2024-06-10T01:23:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.775+0000 7fe9a0989640 1 --2- 172.21.15.7:0/4142995687 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe9880a2b80 0x7fe9880a5040 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.775+0000 7fe9a0989640 1 --2- 172.21.15.7:0/4142995687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe99c1035c0 0x7fe99c19f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.775+0000 7fe9a0989640 1 --2- 172.21.15.7:0/4142995687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe99c102c00 0x7fe99c198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.775+0000 7fe9a0989640 1 --2- 172.21.15.7:0/4142995687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe99c101a10 0x7fe99c197cf0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.775+0000 7fe9a0989640 1 -- 172.21.15.7:0/4142995687 >> 172.21.15.7:0/4142995687 conn(0x7fe99c0fcf80 msgr2=0x7fe99c105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.775+0000 7fe9a0989640 1 -- 172.21.15.7:0/4142995687 shutdown_connections 2024-06-10T01:23:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:23:49.775+0000 7fe9a0989640 1 -- 172.21.15.7:0/4142995687 wait complete. 2024-06-10T01:23:50.534 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 20.83 MB/sec execute 147 sec latency 23471.199 ms 2024-06-10T01:23:50.665 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:23:50.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.881+0000 7f30b3b9a700 1 -- 172.21.15.7:0/1169747932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30ac101690 msgr2=0x7f30ac1129d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:50.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.881+0000 7f30b3b9a700 1 --2- 172.21.15.7:0/1169747932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30ac101690 0x7f30ac1129d0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f309c009990 tx=0x7f309c009ca0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.881+0000 7f30b3b9a700 1 -- 172.21.15.7:0/1169747932 shutdown_connections 2024-06-10T01:23:50.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.881+0000 7f30b3b9a700 1 --2- 172.21.15.7:0/1169747932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30ac113090 0x7f30ac115530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.881+0000 7f30b3b9a700 1 --2- 172.21.15.7:0/1169747932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30ac100ce0 0x7f30ac1010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.881+0000 7f30b3b9a700 1 --2- 172.21.15.7:0/1169747932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30ac101690 0x7f30ac1129d0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.881+0000 7f30b3b9a700 1 -- 172.21.15.7:0/1169747932 >> 172.21.15.7:0/1169747932 conn(0x7f30ac077a00 msgr2=0x7f30ac0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:50.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.881+0000 7f30b3b9a700 1 -- 172.21.15.7:0/1169747932 shutdown_connections 2024-06-10T01:23:50.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.881+0000 7f30b3b9a700 1 -- 172.21.15.7:0/1169747932 wait complete. 2024-06-10T01:23:50.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.881+0000 7f30b3b9a700 1 Processor -- start 2024-06-10T01:23:50.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.881+0000 7f30b3b9a700 1 -- start start 2024-06-10T01:23:50.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.882+0000 7f30b3b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30ac100ce0 0x7f30ac19f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:50.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.882+0000 7f30b3b9a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30ac101690 0x7f30ac19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:50.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.882+0000 7f30b3b9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30ac113090 0x7f30ac1a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:50.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.882+0000 7f30b3b9a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac118e20 con 0x7f30ac100ce0 2024-06-10T01:23:50.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.882+0000 7f30b3b9a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac118ca0 con 0x7f30ac101690 2024-06-10T01:23:50.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.882+0000 7f30b3b9a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac118fa0 con 0x7f30ac113090 2024-06-10T01:23:50.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.882+0000 7f30b1936700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30ac100ce0 0x7f30ac19f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:50.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.882+0000 7f30b1936700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30ac100ce0 0x7f30ac19f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34866/0 (socket says 172.21.15.7:34866) 2024-06-10T01:23:50.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.882+0000 7f30b1936700 1 -- 172.21.15.7:0/1672928102 learned_addr learned my addr 172.21.15.7:0/1672928102 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:50.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.882+0000 7f30b1135700 1 --2- 172.21.15.7:0/1672928102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30ac101690 0x7f30ac19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:50.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.882+0000 7f30b1936700 1 -- 172.21.15.7:0/1672928102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30ac113090 msgr2=0x7f30ac1a3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:50.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.882+0000 7f30b1936700 1 --2- 172.21.15.7:0/1672928102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30ac113090 0x7f30ac1a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.882+0000 7f30b1936700 1 -- 172.21.15.7:0/1672928102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30ac101690 msgr2=0x7f30ac19f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:50.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.882+0000 7f30b1936700 1 --2- 172.21.15.7:0/1672928102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30ac101690 0x7f30ac19f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.882+0000 7f30b1936700 1 -- 172.21.15.7:0/1672928102 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30ac1a42a0 con 0x7f30ac100ce0 2024-06-10T01:23:50.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.883+0000 7f30b1936700 1 --2- 172.21.15.7:0/1672928102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30ac100ce0 0x7f30ac19f290 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f30a800ba50 tx=0x7f30a800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:50.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.883+0000 7f30a2ffd700 1 -- 172.21.15.7:0/1672928102 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a8019070 con 0x7f30ac100ce0 2024-06-10T01:23:50.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.883+0000 7f30a2ffd700 1 -- 172.21.15.7:0/1672928102 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30a8014440 con 0x7f30ac100ce0 2024-06-10T01:23:50.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.883+0000 7f30a2ffd700 1 -- 172.21.15.7:0/1672928102 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a80075d0 con 0x7f30ac100ce0 2024-06-10T01:23:50.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.883+0000 7f30b3b9a700 1 -- 172.21.15.7:0/1672928102 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30ac1a4590 con 0x7f30ac100ce0 2024-06-10T01:23:50.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.883+0000 7f30b3b9a700 1 -- 172.21.15.7:0/1672928102 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30ac1abe10 con 0x7f30ac100ce0 2024-06-10T01:23:50.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.885+0000 7f30a2ffd700 1 -- 172.21.15.7:0/1672928102 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f30a801c070 con 0x7f30ac100ce0 2024-06-10T01:23:50.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.886+0000 7f30a2ffd700 1 --2- 172.21.15.7:0/1672928102 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f30980a2c20 0x7f30980a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:50.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.886+0000 7f30a2ffd700 1 -- 172.21.15.7:0/1672928102 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f30a80ce7a0 con 0x7f30ac100ce0 2024-06-10T01:23:50.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.886+0000 7f30b1135700 1 --2- 172.21.15.7:0/1672928102 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f30980a2c20 0x7f30980a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:50.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.887+0000 7f30b1135700 1 --2- 172.21.15.7:0/1672928102 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f30980a2c20 0x7f30980a50e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f309c006010 tx=0x7f309c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:50.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.887+0000 7f30be41db80 1 Processor -- start 2024-06-10T01:23:50.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.887+0000 7f30be41db80 1 -- start start 2024-06-10T01:23:50.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.887+0000 7f30be41db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8a2377f60 0x55b8a2378340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:50.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.887+0000 7f30be41db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8a2378880 0x55b8a2372260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:50.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.887+0000 7f30be41db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8a23727a0 0x55b8a2372bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:50.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.887+0000 7f30be41db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b8a2264f60 con 0x55b8a23727a0 2024-06-10T01:23:50.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.887+0000 7f30be41db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b8a22c3850 con 0x55b8a2378880 2024-06-10T01:23:50.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.887+0000 7f30be41db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b8a222b460 con 0x55b8a2377f60 2024-06-10T01:23:50.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.887+0000 7f30b2137700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8a23727a0 0x55b8a2372bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:50.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.888+0000 7f30b2137700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8a23727a0 0x55b8a2372bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34874/0 (socket says 172.21.15.7:34874) 2024-06-10T01:23:50.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.888+0000 7f30b2137700 1 -- 172.21.15.7:0/1381264800 learned_addr learned my addr 172.21.15.7:0/1381264800 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:50.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.888+0000 7f30b1936700 1 --2- 172.21.15.7:0/1381264800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8a2377f60 0x55b8a2378340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:50.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.888+0000 7f30b1135700 1 --2- 172.21.15.7:0/1381264800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8a2378880 0x55b8a2372260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:50.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.888+0000 7f30b2137700 1 -- 172.21.15.7:0/1381264800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8a2377f60 msgr2=0x55b8a2378340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:50.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.888+0000 7f30b2137700 1 --2- 172.21.15.7:0/1381264800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8a2377f60 0x55b8a2378340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.888+0000 7f30b2137700 1 -- 172.21.15.7:0/1381264800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8a2378880 msgr2=0x55b8a2372260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:50.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.888+0000 7f30b2137700 1 --2- 172.21.15.7:0/1381264800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8a2378880 0x55b8a2372260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.888+0000 7f30b2137700 1 -- 172.21.15.7:0/1381264800 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8a22128d0 con 0x55b8a23727a0 2024-06-10T01:23:50.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.888+0000 7f30b1135700 1 --2- 172.21.15.7:0/1381264800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8a2378880 0x55b8a2372260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:23:50.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.888+0000 7f30b2137700 1 --2- 172.21.15.7:0/1381264800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8a23727a0 0x55b8a2372bc0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f30a400a9f0 tx=0x7f30a400ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c368ffc4373e9396 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:50.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.888+0000 7f3097fff700 1 -- 172.21.15.7:0/1381264800 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a400cc10 con 0x55b8a23727a0 2024-06-10T01:23:50.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.888+0000 7f3097fff700 1 -- 172.21.15.7:0/1381264800 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30a400cdb0 con 0x55b8a23727a0 2024-06-10T01:23:50.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.888+0000 7f3097fff700 1 -- 172.21.15.7:0/1381264800 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a40186a0 con 0x55b8a23727a0 2024-06-10T01:23:50.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.889+0000 7f30be41db80 1 -- 172.21.15.7:0/1381264800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8a23727a0 msgr2=0x55b8a2372bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:50.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.889+0000 7f30be41db80 1 --2- 172.21.15.7:0/1381264800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8a23727a0 0x55b8a2372bc0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f30a400a9f0 tx=0x7f30a400ad00 comp rx=0 tx=0).stop 2024-06-10T01:23:50.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.889+0000 7f30be41db80 1 -- 172.21.15.7:0/1381264800 shutdown_connections 2024-06-10T01:23:50.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.889+0000 7f30be41db80 1 --2- 172.21.15.7:0/1381264800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8a23727a0 0x55b8a2372bc0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.889+0000 7f30be41db80 1 --2- 172.21.15.7:0/1381264800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8a2377f60 0x55b8a2378340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.889+0000 7f30be41db80 1 --2- 172.21.15.7:0/1381264800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8a2378880 0x55b8a2372260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.889+0000 7f30be41db80 1 -- 172.21.15.7:0/1381264800 >> 172.21.15.7:0/1381264800 conn(0x55b8a229f080 msgr2=0x55b8a236d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:50.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.889+0000 7f30be41db80 1 -- 172.21.15.7:0/1381264800 shutdown_connections 2024-06-10T01:23:50.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.889+0000 7f30be41db80 1 -- 172.21.15.7:0/1381264800 wait complete. 2024-06-10T01:23:50.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.890+0000 7f30be41db80 1 -- start start 2024-06-10T01:23:50.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.890+0000 7f30be41db80 1 Processor -- start 2024-06-10T01:23:50.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.890+0000 7f30be41db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8a23727a0 0x55b8a237e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:50.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.890+0000 7f30be41db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8a2377f60 0x55b8a237b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:50.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.890+0000 7f30be41db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8a2378880 0x55b8a237be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:50.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.890+0000 7f30be41db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b8a2263fc0 con 0x55b8a23727a0 2024-06-10T01:23:50.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.890+0000 7f30b1936700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8a23727a0 0x55b8a237e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:50.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.890+0000 7f30be41db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b8a22bf810 con 0x55b8a2378880 2024-06-10T01:23:50.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.890+0000 7f30be41db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b8a237f260 con 0x55b8a2377f60 2024-06-10T01:23:50.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.890+0000 7f30b1936700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8a23727a0 0x55b8a237e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34882/0 (socket says 172.21.15.7:34882) 2024-06-10T01:23:50.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.890+0000 7f30b1936700 1 -- 172.21.15.7:0/3713960208 learned_addr learned my addr 172.21.15.7:0/3713960208 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:50.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.891+0000 7f30be41db80 10 client.0 _ll_register_callbacks cb 0x55b8a22595b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:23:50.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.891+0000 7f30b1135700 1 --2- 172.21.15.7:0/3713960208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8a2377f60 0x55b8a237b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:50.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.891+0000 7f30b2137700 1 --2- 172.21.15.7:0/3713960208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8a2378880 0x55b8a237be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:50.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.891+0000 7f30b1936700 1 -- 172.21.15.7:0/3713960208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8a2377f60 msgr2=0x55b8a237b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:50.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.891+0000 7f30b1936700 1 --2- 172.21.15.7:0/3713960208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8a2377f60 0x55b8a237b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.891+0000 7f30b1936700 1 -- 172.21.15.7:0/3713960208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8a2378880 msgr2=0x55b8a237be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:50.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.891+0000 7f30b1936700 1 --2- 172.21.15.7:0/3713960208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8a2378880 0x55b8a237be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.891+0000 7f30b1936700 1 -- 172.21.15.7:0/3713960208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8a2289750 con 0x55b8a23727a0 2024-06-10T01:23:50.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.891+0000 7f30b1135700 1 --2- 172.21.15.7:0/3713960208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8a2377f60 0x55b8a237b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:23:50.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.891+0000 7f30b1936700 1 --2- 172.21.15.7:0/3713960208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8a23727a0 0x55b8a237e5f0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f30a80cbf70 tx=0x7f30a8004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6adca149ca8a62e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:50.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.891+0000 7f30957fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:23:50.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.891+0000 7f30957fa700 1 -- 172.21.15.7:0/3713960208 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a80a3460 con 0x55b8a23727a0 2024-06-10T01:23:50.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.891+0000 7f30be41db80 1 -- 172.21.15.7:0/3713960208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30a400cdb0 con 0x55b8a23727a0 2024-06-10T01:23:50.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.891+0000 7f30957fa700 1 -- 172.21.15.7:0/3713960208 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f30a80a3600 con 0x55b8a23727a0 2024-06-10T01:23:50.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.891+0000 7f30957fa700 1 -- 172.21.15.7:0/3713960208 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f30a809a0d0 con 0x55b8a23727a0 2024-06-10T01:23:50.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.891+0000 7f30be41db80 10 client.19815 fetch_fsmap learned FSMap version 50 2024-06-10T01:23:50.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.891+0000 7f30be41db80 1 -- 172.21.15.7:0/3713960208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55b8a2389a90 con 0x55b8a23727a0 2024-06-10T01:23:50.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.892+0000 7f30957fa700 1 -- 172.21.15.7:0/3713960208 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30a400cdb0 con 0x55b8a23727a0 2024-06-10T01:23:50.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.892+0000 7f30957fa700 1 -- 172.21.15.7:0/3713960208 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7f30a80b3070 con 0x55b8a23727a0 2024-06-10T01:23:50.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.892+0000 7f30be41db80 10 client.19815 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:23:50.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.892+0000 7f30be41db80 10 client.19815 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:23:50.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.892+0000 7f30be41db80 20 client.19815 populate_metadata read hostname 'smithi007' 2024-06-10T01:23:50.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.892+0000 7f30be41db80 1 --2- 172.21.15.7:0/3713960208 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b8a238c020 0x55b8a238e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:50.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.892+0000 7f30be41db80 4 client.19815 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:23:50.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.892+0000 7f30be41db80 1 -- 172.21.15.7:0/3713960208 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8a22d0d50 con 0x55b8a238c020 2024-06-10T01:23:50.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.892+0000 7f30b1135700 1 --2- 172.21.15.7:0/3713960208 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b8a238c020 0x55b8a238e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:50.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.893+0000 7f30b1135700 1 --2- 172.21.15.7:0/3713960208 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b8a238c020 0x55b8a238e410 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0325cf7b42447f7 server_cookie=79c8ff1f8ea8a2c5 in_seq=0 out_seq=0 2024-06-10T01:23:50.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.893+0000 7f30957fa700 10 client.19815 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:50.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.893+0000 7f30957fa700 1 -- 172.21.15.7:0/3713960208 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b8a22d0d50 con 0x55b8a238c020 2024-06-10T01:23:50.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.893+0000 7f30957fa700 10 client.19815 handle_command_reply: tid=0 2024-06-10T01:23:50.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.893+0000 7f30be41db80 1 client.19815 shutdown 2024-06-10T01:23:50.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.894+0000 7f30be41db80 1 -- 172.21.15.7:0/3713960208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8a23727a0 msgr2=0x55b8a237e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:50.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.894+0000 7f30be41db80 1 --2- 172.21.15.7:0/3713960208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8a23727a0 0x55b8a237e5f0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f30a80cbf70 tx=0x7f30a8004be0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.894+0000 7f30be41db80 1 -- 172.21.15.7:0/3713960208 shutdown_connections 2024-06-10T01:23:50.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.894+0000 7f30be41db80 1 --2- 172.21.15.7:0/3713960208 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b8a238c020 0x55b8a238e410 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.894+0000 7f30957fa700 0 client.19815 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:50.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.894+0000 7f30be41db80 1 --2- 172.21.15.7:0/3713960208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8a2378880 0x55b8a237be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.894+0000 7f30be41db80 1 --2- 172.21.15.7:0/3713960208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8a23727a0 0x55b8a237e5f0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.894+0000 7f30be41db80 1 --2- 172.21.15.7:0/3713960208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8a2377f60 0x55b8a237b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.894+0000 7f30be41db80 1 -- 172.21.15.7:0/3713960208 >> 172.21.15.7:0/3713960208 conn(0x55b8a229f080 msgr2=0x55b8a236c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:50.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.894+0000 7f30be41db80 1 -- 172.21.15.7:0/3713960208 shutdown_connections 2024-06-10T01:23:50.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.894+0000 7f30be41db80 1 -- 172.21.15.7:0/3713960208 wait complete. 2024-06-10T01:23:50.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.894+0000 7f30be41db80 20 client.19815 trim_cache size 0 max 16384 2024-06-10T01:23:50.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.917+0000 7f30be41db80 1 Processor -- start 2024-06-10T01:23:50.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.917+0000 7f30be41db80 1 -- start start 2024-06-10T01:23:50.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.917+0000 7f30be41db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8a23727a0 0x55b8a2372260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:50.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.917+0000 7f30be41db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8a2377f60 0x55b8a236ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:50.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.917+0000 7f30be41db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8a2378880 0x55b8a236d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:50.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.917+0000 7f30be41db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac1b71f0 con 0x55b8a23727a0 2024-06-10T01:23:50.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.917+0000 7f30be41db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac1b4a80 con 0x55b8a2378880 2024-06-10T01:23:50.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.917+0000 7f30be41db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac1b28d0 con 0x55b8a2377f60 2024-06-10T01:23:50.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.917+0000 7f30b1936700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8a23727a0 0x55b8a2372260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:50.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.917+0000 7f30b1936700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8a23727a0 0x55b8a2372260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34884/0 (socket says 172.21.15.7:34884) 2024-06-10T01:23:50.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.917+0000 7f30b1936700 1 -- 172.21.15.7:0/2735280162 learned_addr learned my addr 172.21.15.7:0/2735280162 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:50.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.917+0000 7f30b2137700 1 --2- 172.21.15.7:0/2735280162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8a2378880 0x55b8a236d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:50.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.917+0000 7f30b1936700 1 -- 172.21.15.7:0/2735280162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8a2377f60 msgr2=0x55b8a236ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:50.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.917+0000 7f30b1936700 1 --2- 172.21.15.7:0/2735280162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8a2377f60 0x55b8a236ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.918+0000 7f30b1936700 1 -- 172.21.15.7:0/2735280162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8a2378880 msgr2=0x55b8a236d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:50.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.918+0000 7f30b1936700 1 --2- 172.21.15.7:0/2735280162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8a2378880 0x55b8a236d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.918+0000 7f30b1936700 1 -- 172.21.15.7:0/2735280162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8a2379e60 con 0x55b8a23727a0 2024-06-10T01:23:50.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.918+0000 7f30b2137700 1 --2- 172.21.15.7:0/2735280162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8a2378880 0x55b8a236d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:23:50.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.918+0000 7f30b1936700 1 --2- 172.21.15.7:0/2735280162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8a23727a0 0x55b8a2372260 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f30a8097600 tx=0x7f30a8099b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e64f87f745b815f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:50.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.918+0000 7f30977fe700 1 -- 172.21.15.7:0/2735280162 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a80149a0 con 0x55b8a23727a0 2024-06-10T01:23:50.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.918+0000 7f30977fe700 1 -- 172.21.15.7:0/2735280162 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30a809a270 con 0x55b8a23727a0 2024-06-10T01:23:50.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.918+0000 7f30977fe700 1 -- 172.21.15.7:0/2735280162 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a80091e0 con 0x55b8a23727a0 2024-06-10T01:23:50.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.919+0000 7f30be41db80 1 -- 172.21.15.7:0/2735280162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8a23727a0 msgr2=0x55b8a2372260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:50.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.919+0000 7f30be41db80 1 --2- 172.21.15.7:0/2735280162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8a23727a0 0x55b8a2372260 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f30a8097600 tx=0x7f30a8099b40 comp rx=0 tx=0).stop 2024-06-10T01:23:50.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.919+0000 7f30be41db80 1 -- 172.21.15.7:0/2735280162 shutdown_connections 2024-06-10T01:23:50.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.919+0000 7f30be41db80 1 --2- 172.21.15.7:0/2735280162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8a2378880 0x55b8a236d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.919+0000 7f30be41db80 1 --2- 172.21.15.7:0/2735280162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8a23727a0 0x55b8a2372260 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.919+0000 7f30be41db80 1 --2- 172.21.15.7:0/2735280162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8a2377f60 0x55b8a236ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.919+0000 7f30be41db80 1 -- 172.21.15.7:0/2735280162 >> 172.21.15.7:0/2735280162 conn(0x55b8a229f080 msgr2=0x55b8a237a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:50.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.919+0000 7f30be41db80 1 -- 172.21.15.7:0/2735280162 shutdown_connections 2024-06-10T01:23:50.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.920+0000 7f30be41db80 1 -- 172.21.15.7:0/2735280162 wait complete. 2024-06-10T01:23:50.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.920+0000 7f30be41db80 1 -- start start 2024-06-10T01:23:50.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.920+0000 7f30be41db80 1 Processor -- start 2024-06-10T01:23:50.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.921+0000 7f30be41db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8a23727a0 0x55b8a2384a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:50.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.921+0000 7f30be41db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8a2377f60 0x55b8a23814f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:50.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.921+0000 7f30be41db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8a2378880 0x55b8a2381a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:50.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.921+0000 7f30be41db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac1b2f40 con 0x55b8a23727a0 2024-06-10T01:23:50.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.921+0000 7f30be41db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac1b43b0 con 0x55b8a2378880 2024-06-10T01:23:50.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.921+0000 7f30be41db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac1b3610 con 0x55b8a2377f60 2024-06-10T01:23:50.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.921+0000 7f30be41db80 10 client.0 _ll_register_callbacks cb 0x55b8a22595b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:23:50.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.921+0000 7f30b1936700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8a23727a0 0x55b8a2384a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:50.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.921+0000 7f30b1936700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8a23727a0 0x55b8a2384a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34898/0 (socket says 172.21.15.7:34898) 2024-06-10T01:23:50.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.921+0000 7f30b1936700 1 -- 172.21.15.7:0/3844069045 learned_addr learned my addr 172.21.15.7:0/3844069045 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:50.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.921+0000 7f30b2137700 1 --2- 172.21.15.7:0/3844069045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8a2378880 0x55b8a2381a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:50.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.921+0000 7f30b1135700 1 --2- 172.21.15.7:0/3844069045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8a2377f60 0x55b8a23814f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:50.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.922+0000 7f30b1936700 1 -- 172.21.15.7:0/3844069045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8a2377f60 msgr2=0x55b8a23814f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:50.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.922+0000 7f30b1936700 1 --2- 172.21.15.7:0/3844069045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8a2377f60 0x55b8a23814f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.922+0000 7f30b1936700 1 -- 172.21.15.7:0/3844069045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8a2378880 msgr2=0x55b8a2381a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:50.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.922+0000 7f30b1936700 1 --2- 172.21.15.7:0/3844069045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8a2378880 0x55b8a2381a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.922+0000 7f30b1936700 1 -- 172.21.15.7:0/3844069045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8a2381ff0 con 0x55b8a23727a0 2024-06-10T01:23:50.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.922+0000 7f30b1936700 1 --2- 172.21.15.7:0/3844069045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8a23727a0 0x55b8a2384a20 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f30a809a930 tx=0x7f30a8014d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa170ef6177f513 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:50.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.922+0000 7f30957fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:23:50.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.922+0000 7f30957fa700 1 -- 172.21.15.7:0/3844069045 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a80a7070 con 0x55b8a23727a0 2024-06-10T01:23:50.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.922+0000 7f30957fa700 1 -- 172.21.15.7:0/3844069045 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f30a80a3670 con 0x55b8a23727a0 2024-06-10T01:23:50.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.922+0000 7f30be41db80 1 -- 172.21.15.7:0/3844069045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30a809a270 con 0x55b8a23727a0 2024-06-10T01:23:50.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.922+0000 7f30957fa700 1 -- 172.21.15.7:0/3844069045 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f30a8099be0 con 0x55b8a23727a0 2024-06-10T01:23:50.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.922+0000 7f30be41db80 10 client.19821 fetch_fsmap learned FSMap version 50 2024-06-10T01:23:50.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.922+0000 7f30be41db80 1 -- 172.21.15.7:0/3844069045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55b8a2381ff0 con 0x55b8a23727a0 2024-06-10T01:23:50.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.922+0000 7f30957fa700 1 -- 172.21.15.7:0/3844069045 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30a809a270 con 0x55b8a23727a0 2024-06-10T01:23:50.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.923+0000 7f30957fa700 1 -- 172.21.15.7:0/3844069045 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7f30a80aa020 con 0x55b8a23727a0 2024-06-10T01:23:50.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.923+0000 7f30be41db80 10 client.19821 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:23:50.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.923+0000 7f30be41db80 10 client.19821 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:23:50.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.923+0000 7f30be41db80 20 client.19821 populate_metadata read hostname 'smithi007' 2024-06-10T01:23:50.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.923+0000 7f30be41db80 1 --2- 172.21.15.7:0/3844069045 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b8a238c020 0x55b8a23f9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:50.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.923+0000 7f30be41db80 4 client.19821 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:23:50.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.923+0000 7f30be41db80 1 -- 172.21.15.7:0/3844069045 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b8a22cb2d0 con 0x55b8a238c020 2024-06-10T01:23:50.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.923+0000 7f30b1135700 1 --2- 172.21.15.7:0/3844069045 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b8a238c020 0x55b8a23f9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:50.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.924+0000 7f30957fa700 10 client.19821 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:50.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.924+0000 7f30b1135700 1 --2- 172.21.15.7:0/3844069045 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b8a238c020 0x55b8a23f9c20 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1004351ac89e2fd server_cookie=c41e9cf75b2f1e0e in_seq=0 out_seq=0 2024-06-10T01:23:50.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.924+0000 7f30957fa700 1 -- 172.21.15.7:0/3844069045 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b8a22cb2d0 con 0x55b8a238c020 2024-06-10T01:23:50.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.924+0000 7f30957fa700 10 client.19821 handle_command_reply: tid=0 2024-06-10T01:23:50.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.924+0000 7f30be41db80 1 client.19821 shutdown 2024-06-10T01:23:50.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.924+0000 7f30be41db80 1 -- 172.21.15.7:0/3844069045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8a23727a0 msgr2=0x55b8a2384a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:50.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.924+0000 7f30be41db80 1 --2- 172.21.15.7:0/3844069045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8a23727a0 0x55b8a2384a20 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f30a809a930 tx=0x7f30a8014d70 comp rx=0 tx=0).stop 2024-06-10T01:23:50.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.924+0000 7f30be41db80 1 -- 172.21.15.7:0/3844069045 shutdown_connections 2024-06-10T01:23:50.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.924+0000 7f30be41db80 1 --2- 172.21.15.7:0/3844069045 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b8a238c020 0x55b8a23f9c20 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.924+0000 7f30957fa700 0 client.19821 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:23:50.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.924+0000 7f30be41db80 1 --2- 172.21.15.7:0/3844069045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8a2378880 0x55b8a2381a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.924+0000 7f30be41db80 1 --2- 172.21.15.7:0/3844069045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b8a23727a0 0x55b8a2384a20 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.924+0000 7f30be41db80 1 --2- 172.21.15.7:0/3844069045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b8a2377f60 0x55b8a23814f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.924+0000 7f30be41db80 1 -- 172.21.15.7:0/3844069045 >> 172.21.15.7:0/3844069045 conn(0x55b8a229f080 msgr2=0x55b8a2376a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.924+0000 7f30be41db80 1 -- 172.21.15.7:0/3844069045 shutdown_connections 2024-06-10T01:23:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.924+0000 7f30be41db80 1 -- 172.21.15.7:0/3844069045 wait complete. 2024-06-10T01:23:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.924+0000 7f30be41db80 20 client.19821 trim_cache size 0 max 16384 2024-06-10T01:23:50.923 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:23:50.923 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-10T01:23:50.923 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-10T01:23:50.924 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:23:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.925+0000 7f30b3b9a700 1 -- 172.21.15.7:0/1672928102 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f30980a2c20 msgr2=0x7f30980a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.925+0000 7f30b3b9a700 1 --2- 172.21.15.7:0/1672928102 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f30980a2c20 0x7f30980a50e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f309c006010 tx=0x7f309c005960 comp rx=0 tx=0).stop 2024-06-10T01:23:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.925+0000 7f30b3b9a700 1 -- 172.21.15.7:0/1672928102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30ac100ce0 msgr2=0x7f30ac19f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.925+0000 7f30b3b9a700 1 --2- 172.21.15.7:0/1672928102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30ac100ce0 0x7f30ac19f290 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f30a800ba50 tx=0x7f30a800bd60 comp rx=0 tx=0).stop 2024-06-10T01:23:50.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.925+0000 7f30b3b9a700 1 -- 172.21.15.7:0/1672928102 shutdown_connections 2024-06-10T01:23:50.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.925+0000 7f30b3b9a700 1 --2- 172.21.15.7:0/1672928102 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f30980a2c20 0x7f30980a50e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.925+0000 7f30b3b9a700 1 --2- 172.21.15.7:0/1672928102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30ac113090 0x7f30ac1a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.925+0000 7f30b3b9a700 1 --2- 172.21.15.7:0/1672928102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30ac100ce0 0x7f30ac19f290 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.925+0000 7f30b3b9a700 1 --2- 172.21.15.7:0/1672928102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30ac101690 0x7f30ac19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:50.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.925+0000 7f30b3b9a700 1 -- 172.21.15.7:0/1672928102 >> 172.21.15.7:0/1672928102 conn(0x7f30ac077a00 msgr2=0x7f30ac111940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:50.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.925+0000 7f30b3b9a700 1 -- 172.21.15.7:0/1672928102 shutdown_connections 2024-06-10T01:23:50.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:50.925+0000 7f30b3b9a700 1 -- 172.21.15.7:0/1672928102 wait complete. 2024-06-10T01:23:51.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:51 smithi007 ceph-mon[32398]: pgmap v968: 129 pgs: 129 active+clean; 554 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 4.0 MiB/s wr, 209 op/s 2024-06-10T01:23:51.276 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:23:51.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:51 smithi175 ceph-mon[36636]: pgmap v968: 129 pgs: 129 active+clean; 554 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 4.0 MiB/s wr, 209 op/s 2024-06-10T01:23:51.534 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 20.69 MB/sec execute 148 sec latency 24471.241 ms 2024-06-10T01:23:51.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:51 smithi078 ceph-mon[36683]: pgmap v968: 129 pgs: 129 active+clean; 554 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 4.0 MiB/s wr, 209 op/s 2024-06-10T01:23:51.670 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:23:51.671 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:23:52.497 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:23:52.534 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 20.55 MB/sec execute 149 sec latency 25471.284 ms 2024-06-10T01:23:53.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:53 smithi175 ceph-mon[36636]: pgmap v969: 129 pgs: 129 active+clean; 554 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 4.0 MiB/s wr, 169 op/s 2024-06-10T01:23:53.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:53 smithi007 ceph-mon[32398]: pgmap v969: 129 pgs: 129 active+clean; 554 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 4.0 MiB/s wr, 169 op/s 2024-06-10T01:23:53.534 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 20.41 MB/sec execute 150 sec latency 26471.332 ms 2024-06-10T01:23:53.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:53 smithi078 ceph-mon[36683]: pgmap v969: 129 pgs: 129 active+clean; 554 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 4.0 MiB/s wr, 169 op/s 2024-06-10T01:23:54.534 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 20.28 MB/sec execute 151 sec latency 27471.400 ms 2024-06-10T01:23:54.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.590+0000 7fb923bc6700 1 -- 172.21.15.7:0/2081542093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb91c108580 msgr2=0x7fb91c108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:54.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.590+0000 7fb923bc6700 1 --2- 172.21.15.7:0/2081542093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb91c108580 0x7fb91c108960 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fb9180099c0 tx=0x7fb918009cd0 comp rx=0 tx=0).stop 2024-06-10T01:23:54.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.590+0000 7fb923bc6700 1 -- 172.21.15.7:0/2081542093 shutdown_connections 2024-06-10T01:23:54.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.590+0000 7fb923bc6700 1 --2- 172.21.15.7:0/2081542093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb91c06b620 0x7fb91c10f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:54.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.590+0000 7fb923bc6700 1 --2- 172.21.15.7:0/2081542093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb91c108580 0x7fb91c108960 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:54.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.590+0000 7fb923bc6700 1 --2- 172.21.15.7:0/2081542093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb91c06ac60 0x7fb91c06b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:54.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.590+0000 7fb923bc6700 1 -- 172.21.15.7:0/2081542093 >> 172.21.15.7:0/2081542093 conn(0x7fb91c078430 msgr2=0x7fb91c078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:54.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.590+0000 7fb923bc6700 1 -- 172.21.15.7:0/2081542093 shutdown_connections 2024-06-10T01:23:54.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.590+0000 7fb923bc6700 1 -- 172.21.15.7:0/2081542093 wait complete. 2024-06-10T01:23:54.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.590+0000 7fb923bc6700 1 Processor -- start 2024-06-10T01:23:54.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.591+0000 7fb923bc6700 1 -- start start 2024-06-10T01:23:54.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.591+0000 7fb923bc6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb91c06ac60 0x7fb91c101400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:54.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.591+0000 7fb923bc6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb91c06b620 0x7fb91c101940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:54.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.591+0000 7fb923bc6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb91c108580 0x7fb91c101e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:54.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.591+0000 7fb923bc6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb91c112840 con 0x7fb91c06b620 2024-06-10T01:23:54.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.591+0000 7fb923bc6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb91c1126c0 con 0x7fb91c108580 2024-06-10T01:23:54.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.591+0000 7fb923bc6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb91c1129c0 con 0x7fb91c06ac60 2024-06-10T01:23:54.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.591+0000 7fb921161700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb91c06b620 0x7fb91c101940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:54.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.591+0000 7fb921161700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb91c06b620 0x7fb91c101940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35134/0 (socket says 172.21.15.7:35134) 2024-06-10T01:23:54.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.591+0000 7fb921161700 1 -- 172.21.15.7:0/2717241785 learned_addr learned my addr 172.21.15.7:0/2717241785 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:54.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.591+0000 7fb921962700 1 --2- 172.21.15.7:0/2717241785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb91c06ac60 0x7fb91c101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:54.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.591+0000 7fb922163700 1 --2- 172.21.15.7:0/2717241785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb91c108580 0x7fb91c101e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:54.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.591+0000 7fb921161700 1 -- 172.21.15.7:0/2717241785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb91c06ac60 msgr2=0x7fb91c101400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:54.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.591+0000 7fb921161700 1 --2- 172.21.15.7:0/2717241785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb91c06ac60 0x7fb91c101400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:54.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.591+0000 7fb921161700 1 -- 172.21.15.7:0/2717241785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb91c108580 msgr2=0x7fb91c101e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:54.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.591+0000 7fb921161700 1 --2- 172.21.15.7:0/2717241785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb91c108580 0x7fb91c101e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:54.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.591+0000 7fb921161700 1 -- 172.21.15.7:0/2717241785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91c1a7180 con 0x7fb91c06b620 2024-06-10T01:23:54.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.591+0000 7fb922163700 1 --2- 172.21.15.7:0/2717241785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb91c108580 0x7fb91c101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:23:54.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.592+0000 7fb921161700 1 --2- 172.21.15.7:0/2717241785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb91c06b620 0x7fb91c101940 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fb90c00b530 tx=0x7fb90c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:54.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.592+0000 7fb912ffd700 1 -- 172.21.15.7:0/2717241785 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c015020 con 0x7fb91c06b620 2024-06-10T01:23:54.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.592+0000 7fb912ffd700 1 -- 172.21.15.7:0/2717241785 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb90c011d10 con 0x7fb91c06b620 2024-06-10T01:23:54.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.592+0000 7fb923bc6700 1 -- 172.21.15.7:0/2717241785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb91c1a7470 con 0x7fb91c06b620 2024-06-10T01:23:54.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.592+0000 7fb912ffd700 1 -- 172.21.15.7:0/2717241785 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c00f2e0 con 0x7fb91c06b620 2024-06-10T01:23:54.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.592+0000 7fb923bc6700 1 -- 172.21.15.7:0/2717241785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb91c1a79b0 con 0x7fb91c06b620 2024-06-10T01:23:54.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.593+0000 7fb923bc6700 1 -- 172.21.15.7:0/2717241785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb91c10aa10 con 0x7fb91c06b620 2024-06-10T01:23:54.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.594+0000 7fb912ffd700 1 -- 172.21.15.7:0/2717241785 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb90c018020 con 0x7fb91c06b620 2024-06-10T01:23:54.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.594+0000 7fb912ffd700 1 --2- 172.21.15.7:0/2717241785 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb9080a2c20 0x7fb9080a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:54.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.595+0000 7fb921962700 1 --2- 172.21.15.7:0/2717241785 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb9080a2c20 0x7fb9080a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:54.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.595+0000 7fb912ffd700 1 -- 172.21.15.7:0/2717241785 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fb90c0cec80 con 0x7fb91c06b620 2024-06-10T01:23:54.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.595+0000 7fb921962700 1 --2- 172.21.15.7:0/2717241785 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb9080a2c20 0x7fb9080a50e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb918009f50 tx=0x7fb918005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:54.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.599+0000 7fb912ffd700 1 -- 172.21.15.7:0/2717241785 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb90c01f020 con 0x7fb91c06b620 2024-06-10T01:23:54.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.856+0000 7fb923bc6700 1 -- 172.21.15.7:0/2717241785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb91c069180 con 0x7fb91c06b620 2024-06-10T01:23:54.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.857+0000 7fb912ffd700 1 -- 172.21.15.7:0/2717241785 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11858 (secure 0 0 0) 0x7fb90c09b800 con 0x7fb91c06b620 2024-06-10T01:23:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:23:54.850 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":50,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:22:57.989337+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28825":{"gid":28825,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/1972426673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1972426673},{"type":"v1","addr":"172.21.15.175:6835","nonce":1972426673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:23:54.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.859+0000 7fb923bc6700 1 -- 172.21.15.7:0/2717241785 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb9080a2c20 msgr2=0x7fb9080a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:54.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.859+0000 7fb923bc6700 1 --2- 172.21.15.7:0/2717241785 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb9080a2c20 0x7fb9080a50e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb918009f50 tx=0x7fb918005fb0 comp rx=0 tx=0).stop 2024-06-10T01:23:54.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.859+0000 7fb923bc6700 1 -- 172.21.15.7:0/2717241785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb91c06b620 msgr2=0x7fb91c101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:54.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.859+0000 7fb923bc6700 1 --2- 172.21.15.7:0/2717241785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb91c06b620 0x7fb91c101940 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fb90c00b530 tx=0x7fb90c00b840 comp rx=0 tx=0).stop 2024-06-10T01:23:54.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.860+0000 7fb923bc6700 1 -- 172.21.15.7:0/2717241785 shutdown_connections 2024-06-10T01:23:54.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.860+0000 7fb923bc6700 1 --2- 172.21.15.7:0/2717241785 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb9080a2c20 0x7fb9080a50e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:54.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.860+0000 7fb923bc6700 1 --2- 172.21.15.7:0/2717241785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb91c108580 0x7fb91c101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:54.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.860+0000 7fb923bc6700 1 --2- 172.21.15.7:0/2717241785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb91c06ac60 0x7fb91c101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:54.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.860+0000 7fb923bc6700 1 --2- 172.21.15.7:0/2717241785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb91c06b620 0x7fb91c101940 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:54.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.860+0000 7fb923bc6700 1 -- 172.21.15.7:0/2717241785 >> 172.21.15.7:0/2717241785 conn(0x7fb91c078430 msgr2=0x7fb91c0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:54.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.860+0000 7fb923bc6700 1 -- 172.21.15.7:0/2717241785 shutdown_connections 2024-06-10T01:23:54.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.860+0000 7fb923bc6700 1 -- 172.21.15.7:0/2717241785 wait complete. 2024-06-10T01:23:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 50 2024-06-10T01:23:54.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.968+0000 7fec8887e700 1 -- 172.21.15.7:0/1308606424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec84101d50 msgr2=0x7fec8410e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:54.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.968+0000 7fec8887e700 1 --2- 172.21.15.7:0/1308606424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec84101d50 0x7fec8410e240 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fec7c00b0a0 tx=0x7fec7c00b3b0 comp rx=0 tx=0).stop 2024-06-10T01:23:54.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.968+0000 7fec8887e700 1 -- 172.21.15.7:0/1308606424 shutdown_connections 2024-06-10T01:23:54.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.968+0000 7fec8887e700 1 --2- 172.21.15.7:0/1308606424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec84101d50 0x7fec8410e240 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:54.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.968+0000 7fec8887e700 1 --2- 172.21.15.7:0/1308606424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec84107400 0x7fec841077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:54.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.968+0000 7fec8887e700 1 --2- 172.21.15.7:0/1308606424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec84101390 0x7fec84101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:54.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.968+0000 7fec8887e700 1 -- 172.21.15.7:0/1308606424 >> 172.21.15.7:0/1308606424 conn(0x7fec84078340 msgr2=0x7fec84078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:54.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.968+0000 7fec8887e700 1 -- 172.21.15.7:0/1308606424 shutdown_connections 2024-06-10T01:23:54.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.968+0000 7fec8887e700 1 -- 172.21.15.7:0/1308606424 wait complete. 2024-06-10T01:23:54.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.969+0000 7fec8887e700 1 Processor -- start 2024-06-10T01:23:54.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.969+0000 7fec8887e700 1 -- start start 2024-06-10T01:23:54.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.969+0000 7fec8887e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec84101390 0x7fec84103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.969+0000 7fec8887e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec84101d50 0x7fec841035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.969+0000 7fec8887e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec84107400 0x7fec84103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.969+0000 7fec8887e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fec84111690 con 0x7fec84101390 2024-06-10T01:23:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.969+0000 7fec8887e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fec84111510 con 0x7fec84107400 2024-06-10T01:23:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.969+0000 7fec8887e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fec84111810 con 0x7fec84101d50 2024-06-10T01:23:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.970+0000 7fec8259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec84101390 0x7fec84103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.970+0000 7fec8259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec84101390 0x7fec84103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35154/0 (socket says 172.21.15.7:35154) 2024-06-10T01:23:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.970+0000 7fec8259c700 1 -- 172.21.15.7:0/1739764716 learned_addr learned my addr 172.21.15.7:0/1739764716 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.970+0000 7fec82d9d700 1 --2- 172.21.15.7:0/1739764716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec84107400 0x7fec84103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.970+0000 7fec79d9b700 1 --2- 172.21.15.7:0/1739764716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec84101d50 0x7fec841035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.970+0000 7fec8259c700 1 -- 172.21.15.7:0/1739764716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec84101d50 msgr2=0x7fec841035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.970+0000 7fec8259c700 1 --2- 172.21.15.7:0/1739764716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec84101d50 0x7fec841035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.970+0000 7fec8259c700 1 -- 172.21.15.7:0/1739764716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec84107400 msgr2=0x7fec84103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:54.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.970+0000 7fec8259c700 1 --2- 172.21.15.7:0/1739764716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec84107400 0x7fec84103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:54.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.970+0000 7fec8259c700 1 -- 172.21.15.7:0/1739764716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec841a70d0 con 0x7fec84101390 2024-06-10T01:23:54.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.970+0000 7fec8259c700 1 --2- 172.21.15.7:0/1739764716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec84101390 0x7fec84103070 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fec7400d980 tx=0x7fec7400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:54.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.970+0000 7fec7bfff700 1 -- 172.21.15.7:0/1739764716 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec74017070 con 0x7fec84101390 2024-06-10T01:23:54.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.970+0000 7fec8887e700 1 -- 172.21.15.7:0/1739764716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec841a73c0 con 0x7fec84101390 2024-06-10T01:23:54.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.970+0000 7fec7bfff700 1 -- 172.21.15.7:0/1739764716 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec74005020 con 0x7fec84101390 2024-06-10T01:23:54.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.970+0000 7fec8887e700 1 -- 172.21.15.7:0/1739764716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec841a7900 con 0x7fec84101390 2024-06-10T01:23:54.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.971+0000 7fec7bfff700 1 -- 172.21.15.7:0/1739764716 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7400fbf0 con 0x7fec84101390 2024-06-10T01:23:54.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.971+0000 7fec8887e700 1 -- 172.21.15.7:0/1739764716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec64005330 con 0x7fec84101390 2024-06-10T01:23:54.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.973+0000 7fec7bfff700 1 -- 172.21.15.7:0/1739764716 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fec74004a30 con 0x7fec84101390 2024-06-10T01:23:54.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.974+0000 7fec7bfff700 1 --2- 172.21.15.7:0/1739764716 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fec700a2ca0 0x7fec700a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:54.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.974+0000 7fec79d9b700 1 --2- 172.21.15.7:0/1739764716 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fec700a2ca0 0x7fec700a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:54.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.974+0000 7fec7bfff700 1 -- 172.21.15.7:0/1739764716 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fec740cf450 con 0x7fec84101390 2024-06-10T01:23:54.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.975+0000 7fec79d9b700 1 --2- 172.21.15.7:0/1739764716 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fec700a2ca0 0x7fec700a5160 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fec84078940 tx=0x7fec6c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:54.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:54.978+0000 7fec7bfff700 1 -- 172.21.15.7:0/1739764716 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fec74020020 con 0x7fec84101390 2024-06-10T01:23:55.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:55.260+0000 7fec8887e700 1 -- 172.21.15.7:0/1739764716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec64005620 con 0x7fec84101390 2024-06-10T01:23:55.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:55.260+0000 7fec7bfff700 1 -- 172.21.15.7:0/1739764716 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11858 (secure 0 0 0) 0x7fec7409c050 con 0x7fec84101390 2024-06-10T01:23:55.254 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:23:55.255 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":50,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:22:57.989337+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28825":{"gid":28825,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/1972426673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1972426673},{"type":"v1","addr":"172.21.15.175:6835","nonce":1972426673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:23:55.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:55.263+0000 7fec8887e700 1 -- 172.21.15.7:0/1739764716 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fec700a2ca0 msgr2=0x7fec700a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:55.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:55.263+0000 7fec8887e700 1 --2- 172.21.15.7:0/1739764716 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fec700a2ca0 0x7fec700a5160 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fec84078940 tx=0x7fec6c0073d0 comp rx=0 tx=0).stop 2024-06-10T01:23:55.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:55.263+0000 7fec8887e700 1 -- 172.21.15.7:0/1739764716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec84101390 msgr2=0x7fec84103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:55.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:55.263+0000 7fec8887e700 1 --2- 172.21.15.7:0/1739764716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec84101390 0x7fec84103070 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fec7400d980 tx=0x7fec7400dc90 comp rx=0 tx=0).stop 2024-06-10T01:23:55.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:55.263+0000 7fec8887e700 1 -- 172.21.15.7:0/1739764716 shutdown_connections 2024-06-10T01:23:55.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:55.263+0000 7fec8887e700 1 --2- 172.21.15.7:0/1739764716 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fec700a2ca0 0x7fec700a5160 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:55.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:55.263+0000 7fec8887e700 1 --2- 172.21.15.7:0/1739764716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec84107400 0x7fec84103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:55.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:55.263+0000 7fec8887e700 1 --2- 172.21.15.7:0/1739764716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec84101390 0x7fec84103070 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:55.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:55.263+0000 7fec8887e700 1 --2- 172.21.15.7:0/1739764716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec84101d50 0x7fec841035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:55.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:55.263+0000 7fec8887e700 1 -- 172.21.15.7:0/1739764716 >> 172.21.15.7:0/1739764716 conn(0x7fec84078340 msgr2=0x7fec841005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:55.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:55.263+0000 7fec8887e700 1 -- 172.21.15.7:0/1739764716 shutdown_connections 2024-06-10T01:23:55.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:55.263+0000 7fec8887e700 1 -- 172.21.15.7:0/1739764716 wait complete. 2024-06-10T01:23:55.257 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 50 2024-06-10T01:23:55.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:55 smithi175 ceph-mon[36636]: pgmap v970: 129 pgs: 129 active+clean; 554 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 3.1 MiB/s wr, 141 op/s 2024-06-10T01:23:55.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:55 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2717241785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:23:55.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:55 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:23:55.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:55 smithi007 ceph-mon[32398]: pgmap v970: 129 pgs: 129 active+clean; 554 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 3.1 MiB/s wr, 141 op/s 2024-06-10T01:23:55.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:55 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2717241785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:23:55.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:55 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:23:55.534 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 20.14 MB/sec execute 152 sec latency 28471.428 ms 2024-06-10T01:23:55.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:55 smithi078 ceph-mon[36683]: pgmap v970: 129 pgs: 129 active+clean; 554 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 3.1 MiB/s wr, 141 op/s 2024-06-10T01:23:55.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:55 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2717241785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:23:55.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:55 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:23:55.635 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:23:56.248 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:56 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1739764716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:23:56.248 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:56 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:23:56.248 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:56 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:23:56.249 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:56 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:23:56.279 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db --set-id f8806db1 --release --await' 2024-06-10T01:23:56.280 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db --set-id f8806db1 --release --await 2024-06-10T01:23:56.347 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:23:56.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:56 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1739764716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:23:56.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:56 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:23:56.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:56 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:23:56.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:56 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:23:56.534 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 20.01 MB/sec execute 153 sec latency 29471.524 ms 2024-06-10T01:23:56.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:56 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1739764716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:23:56.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:56 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:23:56.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:56 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:23:56.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:56 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:23:57.406 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:57 smithi007 ceph-mon[32398]: pgmap v971: 129 pgs: 129 active+clean; 550 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 2.9 MiB/s wr, 150 op/s 2024-06-10T01:23:57.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:57 smithi175 ceph-mon[36636]: pgmap v971: 129 pgs: 129 active+clean; 550 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 2.9 MiB/s wr, 150 op/s 2024-06-10T01:23:57.534 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 19.88 MB/sec execute 154 sec latency 30471.624 ms 2024-06-10T01:23:57.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:57 smithi078 ceph-mon[36683]: pgmap v971: 129 pgs: 129 active+clean; 550 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 2.9 MiB/s wr, 150 op/s 2024-06-10T01:23:57.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.943+0000 7f011c56a700 1 -- 172.21.15.7:0/1677903466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f011406b3a0 msgr2=0x7f011410f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:57.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.943+0000 7f011c56a700 1 --2- 172.21.15.7:0/1677903466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f011406b3a0 0x7f011410f310 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f010c00b0a0 tx=0x7f010c00b3b0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.943+0000 7f011c56a700 1 -- 172.21.15.7:0/1677903466 shutdown_connections 2024-06-10T01:23:57.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.943+0000 7f011c56a700 1 --2- 172.21.15.7:0/1677903466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f011406b3a0 0x7f011410f310 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.943+0000 7f011c56a700 1 --2- 172.21.15.7:0/1677903466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01141084d0 0x7f01141088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.943+0000 7f011c56a700 1 --2- 172.21.15.7:0/1677903466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f011406aa00 0x7f011406ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.943+0000 7f011c56a700 1 -- 172.21.15.7:0/1677903466 >> 172.21.15.7:0/1677903466 conn(0x7f0114078330 msgr2=0x7f0114078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:57.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.943+0000 7f011c56a700 1 -- 172.21.15.7:0/1677903466 shutdown_connections 2024-06-10T01:23:57.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.943+0000 7f011c56a700 1 -- 172.21.15.7:0/1677903466 wait complete. 2024-06-10T01:23:57.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.944+0000 7f011c56a700 1 Processor -- start 2024-06-10T01:23:57.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.944+0000 7f011c56a700 1 -- start start 2024-06-10T01:23:57.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.944+0000 7f011c56a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f011406aa00 0x7f011419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:57.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.944+0000 7f011c56a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f011406b3a0 0x7f011419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:57.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.944+0000 7f011c56a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01141084d0 0x7f01141989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:57.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.944+0000 7f011c56a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0114112750 con 0x7f011406b3a0 2024-06-10T01:23:57.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.944+0000 7f011c56a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f01141125d0 con 0x7f01141084d0 2024-06-10T01:23:57.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.944+0000 7f011c56a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f01141128d0 con 0x7f011406aa00 2024-06-10T01:23:57.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.944+0000 7f0119b05700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f011406b3a0 0x7f011419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:57.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.944+0000 7f0119b05700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f011406b3a0 0x7f011419ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35166/0 (socket says 172.21.15.7:35166) 2024-06-10T01:23:57.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.944+0000 7f0119b05700 1 -- 172.21.15.7:0/3757786560 learned_addr learned my addr 172.21.15.7:0/3757786560 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:57.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.944+0000 7f011a306700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f011406aa00 0x7f011419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:57.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.944+0000 7f011ab07700 1 --2- 172.21.15.7:0/3757786560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01141084d0 0x7f01141989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:57.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.945+0000 7f0119b05700 1 -- 172.21.15.7:0/3757786560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f011406aa00 msgr2=0x7f011419e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:57.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.945+0000 7f0119b05700 1 --2- 172.21.15.7:0/3757786560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f011406aa00 0x7f011419e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.945+0000 7f0119b05700 1 -- 172.21.15.7:0/3757786560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01141084d0 msgr2=0x7f01141989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:57.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.945+0000 7f0119b05700 1 --2- 172.21.15.7:0/3757786560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01141084d0 0x7f01141989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.945+0000 7f0119b05700 1 -- 172.21.15.7:0/3757786560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0114199130 con 0x7f011406b3a0 2024-06-10T01:23:57.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.945+0000 7f011ab07700 1 --2- 172.21.15.7:0/3757786560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01141084d0 0x7f01141989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:23:57.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.945+0000 7f0119b05700 1 --2- 172.21.15.7:0/3757786560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f011406b3a0 0x7f011419ed50 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f010400e9c0 tx=0x7f010400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:57.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.945+0000 7f010b7fe700 1 -- 172.21.15.7:0/3757786560 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010400cbd0 con 0x7f011406b3a0 2024-06-10T01:23:57.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.945+0000 7f010b7fe700 1 -- 172.21.15.7:0/3757786560 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f010400cd70 con 0x7f011406b3a0 2024-06-10T01:23:57.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.945+0000 7f011c56a700 1 -- 172.21.15.7:0/3757786560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0114199420 con 0x7f011406b3a0 2024-06-10T01:23:57.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.945+0000 7f011c56a700 1 -- 172.21.15.7:0/3757786560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01141a56e0 con 0x7f011406b3a0 2024-06-10T01:23:57.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.945+0000 7f010b7fe700 1 -- 172.21.15.7:0/3757786560 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104010430 con 0x7f011406b3a0 2024-06-10T01:23:57.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.948+0000 7f010b7fe700 1 -- 172.21.15.7:0/3757786560 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0104004750 con 0x7f011406b3a0 2024-06-10T01:23:57.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.948+0000 7f010b7fe700 1 --2- 172.21.15.7:0/3757786560 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f01000a2c20 0x7f01000a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:57.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.949+0000 7f010b7fe700 1 -- 172.21.15.7:0/3757786560 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f01040cf580 con 0x7f011406b3a0 2024-06-10T01:23:57.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.949+0000 7f011a306700 1 --2- 172.21.15.7:0/3757786560 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f01000a2c20 0x7f01000a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:57.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.949+0000 7f011a306700 1 --2- 172.21.15.7:0/3757786560 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f01000a2c20 0x7f01000a50e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0114199b10 tx=0x7f011000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:57.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.950+0000 7f0126dedb80 1 Processor -- start 2024-06-10T01:23:57.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.950+0000 7f0126dedb80 1 -- start start 2024-06-10T01:23:57.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.950+0000 7f0126dedb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ee19b640 0x55a1ee19ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:57.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.950+0000 7f0126dedb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ee19bf60 0x55a1ee1959e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:57.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.951+0000 7f0126dedb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ee195f20 0x55a1ee196340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:57.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.951+0000 7f0126dedb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a1ee0e7810 con 0x55a1ee195f20 2024-06-10T01:23:57.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.951+0000 7f0126dedb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a1ee08f720 con 0x55a1ee19bf60 2024-06-10T01:23:57.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.951+0000 7f0126dedb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a1ee0e37d0 con 0x55a1ee19b640 2024-06-10T01:23:57.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.951+0000 7f0119b05700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ee19bf60 0x55a1ee1959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:57.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.951+0000 7f0119b05700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ee19bf60 0x55a1ee1959e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:59736/0 (socket says 172.21.15.7:59736) 2024-06-10T01:23:57.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.951+0000 7f0119b05700 1 -- 172.21.15.7:0/2833950848 learned_addr learned my addr 172.21.15.7:0/2833950848 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:57.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.951+0000 7f011ab07700 1 --2- 172.21.15.7:0/2833950848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ee195f20 0x55a1ee196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:57.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.951+0000 7f011a306700 1 --2- 172.21.15.7:0/2833950848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ee19b640 0x55a1ee19ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:57.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.951+0000 7f0119b05700 1 -- 172.21.15.7:0/2833950848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ee19b640 msgr2=0x55a1ee19ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:57.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.951+0000 7f0119b05700 1 --2- 172.21.15.7:0/2833950848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ee19b640 0x55a1ee19ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.951+0000 7f0119b05700 1 -- 172.21.15.7:0/2833950848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ee195f20 msgr2=0x55a1ee196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:57.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.951+0000 7f0119b05700 1 --2- 172.21.15.7:0/2833950848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ee195f20 0x55a1ee196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.951+0000 7f0119b05700 1 -- 172.21.15.7:0/2833950848 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1ee085f50 con 0x55a1ee19bf60 2024-06-10T01:23:57.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.951+0000 7f011ab07700 1 --2- 172.21.15.7:0/2833950848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ee195f20 0x55a1ee196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:23:57.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.951+0000 7f0119b05700 1 --2- 172.21.15.7:0/2833950848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ee19bf60 0x55a1ee1959e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0104009fd0 tx=0x7f010409b620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e16aef9ae1890b84 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:57.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.952+0000 7f0108ff9700 1 -- 172.21.15.7:0/2833950848 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010409ba60 con 0x55a1ee19bf60 2024-06-10T01:23:57.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.952+0000 7f0108ff9700 1 -- 172.21.15.7:0/2833950848 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f010409fe10 con 0x55a1ee19bf60 2024-06-10T01:23:57.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.952+0000 7f0108ff9700 1 -- 172.21.15.7:0/2833950848 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01040a4240 con 0x55a1ee19bf60 2024-06-10T01:23:57.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.952+0000 7f0126dedb80 1 -- 172.21.15.7:0/2833950848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ee19bf60 msgr2=0x55a1ee1959e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:57.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.952+0000 7f0126dedb80 1 --2- 172.21.15.7:0/2833950848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ee19bf60 0x55a1ee1959e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0104009fd0 tx=0x7f010409b620 comp rx=0 tx=0).stop 2024-06-10T01:23:57.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.953+0000 7f0126dedb80 1 -- 172.21.15.7:0/2833950848 shutdown_connections 2024-06-10T01:23:57.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.953+0000 7f0126dedb80 1 --2- 172.21.15.7:0/2833950848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ee195f20 0x55a1ee196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.953+0000 7f0126dedb80 1 --2- 172.21.15.7:0/2833950848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ee19b640 0x55a1ee19ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.953+0000 7f0126dedb80 1 --2- 172.21.15.7:0/2833950848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ee19bf60 0x55a1ee1959e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.953+0000 7f0126dedb80 1 -- 172.21.15.7:0/2833950848 >> 172.21.15.7:0/2833950848 conn(0x55a1ee0c0cc0 msgr2=0x55a1ee190850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:57.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.953+0000 7f0126dedb80 1 -- 172.21.15.7:0/2833950848 shutdown_connections 2024-06-10T01:23:57.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.953+0000 7f0126dedb80 1 -- 172.21.15.7:0/2833950848 wait complete. 2024-06-10T01:23:57.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.953+0000 7f0126dedb80 1 -- start start 2024-06-10T01:23:57.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.954+0000 7f0126dedb80 1 Processor -- start 2024-06-10T01:23:57.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.954+0000 7f0126dedb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ee195f20 0x55a1ee1a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:57.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.954+0000 7f0126dedb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ee19b640 0x55a1ee19f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:57.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.954+0000 7f0126dedb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ee19bf60 0x55a1ee19f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:57.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.954+0000 7f0126dedb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a1ee1a2970 con 0x55a1ee19bf60 2024-06-10T01:23:57.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.954+0000 7f0126dedb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a1ee087bf0 con 0x55a1ee195f20 2024-06-10T01:23:57.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.954+0000 7f0126dedb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a1ee1a2af0 con 0x55a1ee19b640 2024-06-10T01:23:57.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.954+0000 7f011a306700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ee195f20 0x55a1ee1a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:57.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.954+0000 7f011a306700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ee195f20 0x55a1ee1a1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:59738/0 (socket says 172.21.15.7:59738) 2024-06-10T01:23:57.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.954+0000 7f011a306700 1 -- 172.21.15.7:0/20300437 learned_addr learned my addr 172.21.15.7:0/20300437 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:57.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.954+0000 7f011ab07700 1 --2- 172.21.15.7:0/20300437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ee19bf60 0x55a1ee19f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:57.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.954+0000 7f0126dedb80 10 client.0 _ll_register_callbacks cb 0x55a1ee081920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:23:57.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.954+0000 7f011a306700 1 -- 172.21.15.7:0/20300437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ee19b640 msgr2=0x55a1ee19f030 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:23:57.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.954+0000 7f011a306700 1 --2- 172.21.15.7:0/20300437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ee19b640 0x55a1ee19f030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.954+0000 7f011a306700 1 -- 172.21.15.7:0/20300437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ee19bf60 msgr2=0x55a1ee19f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:57.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.954+0000 7f011a306700 1 --2- 172.21.15.7:0/20300437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ee19bf60 0x55a1ee19f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.955+0000 7f011a306700 1 -- 172.21.15.7:0/20300437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1ee0368d0 con 0x55a1ee195f20 2024-06-10T01:23:57.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.955+0000 7f011ab07700 1 --2- 172.21.15.7:0/20300437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ee19bf60 0x55a1ee19f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:23:57.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.955+0000 7f011a306700 1 --2- 172.21.15.7:0/20300437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ee195f20 0x55a1ee1a1d30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f01100116c0 tx=0x7f01100047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7438d26a2c1ad019 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:57.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.955+0000 7f00fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:23:57.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.955+0000 7f00fdffb700 1 -- 172.21.15.7:0/20300437 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110028670 con 0x55a1ee195f20 2024-06-10T01:23:57.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.955+0000 7f00fdffb700 1 -- 172.21.15.7:0/20300437 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f0110028810 con 0x55a1ee195f20 2024-06-10T01:23:57.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.955+0000 7f0126dedb80 1 -- 172.21.15.7:0/20300437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f010409fe10 con 0x55a1ee195f20 2024-06-10T01:23:57.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.956+0000 7f00fdffb700 1 -- 172.21.15.7:0/20300437 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f0110028a10 con 0x55a1ee195f20 2024-06-10T01:23:57.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.956+0000 7f0126dedb80 10 client.29048 fetch_fsmap learned FSMap version 50 2024-06-10T01:23:57.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.956+0000 7f0126dedb80 1 -- 172.21.15.7:0/20300437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55a1ee1ad160 con 0x55a1ee195f20 2024-06-10T01:23:57.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.956+0000 7f00fdffb700 1 -- 172.21.15.7:0/20300437 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f010409fe10 con 0x55a1ee195f20 2024-06-10T01:23:57.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.956+0000 7f00fdffb700 1 -- 172.21.15.7:0/20300437 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7f0110032540 con 0x55a1ee195f20 2024-06-10T01:23:57.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.956+0000 7f0126dedb80 10 client.29048 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:23:57.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.956+0000 7f0126dedb80 10 client.29048 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:23:57.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.956+0000 7f0126dedb80 20 client.29048 populate_metadata read hostname 'smithi007' 2024-06-10T01:23:57.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.956+0000 7f0126dedb80 1 --2- 172.21.15.7:0/20300437 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55a1ee1af780 0x55a1ee1b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:57.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.956+0000 7f0126dedb80 4 client.29048 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:23:57.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.956+0000 7f0126dedb80 1 -- 172.21.15.7:0/20300437 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1ee0f4d10 con 0x55a1ee1af780 2024-06-10T01:23:57.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.956+0000 7f0119b05700 1 --2- 172.21.15.7:0/20300437 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55a1ee1af780 0x55a1ee1b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:57.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.957+0000 7f00fdffb700 10 client.29048 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:23:57.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.957+0000 7f0119b05700 1 --2- 172.21.15.7:0/20300437 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55a1ee1af780 0x55a1ee1b1bc0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a258499c9b331074 server_cookie=15e65fea37ad1167 in_seq=0 out_seq=0 2024-06-10T01:23:57.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.958+0000 7f00fdffb700 1 -- 172.21.15.7:0/20300437 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a1ee0f4d10 con 0x55a1ee1af780 2024-06-10T01:23:57.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.958+0000 7f00fdffb700 10 client.29048 handle_command_reply: tid=0 2024-06-10T01:23:57.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.958+0000 7f0126dedb80 1 client.29048 shutdown 2024-06-10T01:23:57.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.958+0000 7f0126dedb80 1 -- 172.21.15.7:0/20300437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ee195f20 msgr2=0x55a1ee1a1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:57.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.958+0000 7f0126dedb80 1 --2- 172.21.15.7:0/20300437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ee195f20 0x55a1ee1a1d30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f01100116c0 tx=0x7f01100047c0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.958+0000 7f0126dedb80 1 -- 172.21.15.7:0/20300437 shutdown_connections 2024-06-10T01:23:57.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.958+0000 7f0126dedb80 1 --2- 172.21.15.7:0/20300437 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55a1ee1af780 0x55a1ee1b1bc0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.958+0000 7f0126dedb80 1 --2- 172.21.15.7:0/20300437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ee19bf60 0x55a1ee19f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.958+0000 7f00fdffb700 0 client.29048 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:23:57.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.958+0000 7f0126dedb80 1 --2- 172.21.15.7:0/20300437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ee195f20 0x55a1ee1a1d30 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.958+0000 7f0126dedb80 1 --2- 172.21.15.7:0/20300437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ee19b640 0x55a1ee19f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.958+0000 7f0126dedb80 1 -- 172.21.15.7:0/20300437 >> 172.21.15.7:0/20300437 conn(0x55a1ee0c0cc0 msgr2=0x55a1ee1949d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:57.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.958+0000 7f0126dedb80 1 -- 172.21.15.7:0/20300437 shutdown_connections 2024-06-10T01:23:57.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.958+0000 7f0126dedb80 1 -- 172.21.15.7:0/20300437 wait complete. 2024-06-10T01:23:57.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.958+0000 7f0126dedb80 20 client.29048 trim_cache size 0 max 16384 2024-06-10T01:23:57.962 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:23:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.974+0000 7f0126dedb80 1 Processor -- start 2024-06-10T01:23:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.974+0000 7f0126dedb80 1 -- start start 2024-06-10T01:23:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.974+0000 7f0126dedb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ee195f20 0x55a1ee1959e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.974+0000 7f0126dedb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ee19b640 0x55a1ee1909f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:57.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.974+0000 7f0126dedb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ee19bf60 0x55a1ee19cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:57.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.974+0000 7f0126dedb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f00f8011300 con 0x55a1ee195f20 2024-06-10T01:23:57.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.974+0000 7f0126dedb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f00f800eb90 con 0x55a1ee19b640 2024-06-10T01:23:57.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.974+0000 7f0126dedb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f00f800c9e0 con 0x55a1ee19bf60 2024-06-10T01:23:57.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.975+0000 7f011a306700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ee195f20 0x55a1ee1959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:57.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.975+0000 7f011a306700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ee195f20 0x55a1ee1959e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35190/0 (socket says 172.21.15.7:35190) 2024-06-10T01:23:57.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.975+0000 7f011a306700 1 -- 172.21.15.7:0/1517096366 learned_addr learned my addr 172.21.15.7:0/1517096366 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:57.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.975+0000 7f0119b05700 1 --2- 172.21.15.7:0/1517096366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ee19b640 0x55a1ee1909f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:57.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.975+0000 7f011ab07700 1 --2- 172.21.15.7:0/1517096366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ee19bf60 0x55a1ee19cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:57.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.975+0000 7f011a306700 1 -- 172.21.15.7:0/1517096366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ee19bf60 msgr2=0x55a1ee19cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:57.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.975+0000 7f011a306700 1 --2- 172.21.15.7:0/1517096366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ee19bf60 0x55a1ee19cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.975+0000 7f011a306700 1 -- 172.21.15.7:0/1517096366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ee19b640 msgr2=0x55a1ee1909f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:57.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.975+0000 7f011a306700 1 --2- 172.21.15.7:0/1517096366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ee19b640 0x55a1ee1909f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.975+0000 7f011a306700 1 -- 172.21.15.7:0/1517096366 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1ee1acf90 con 0x55a1ee195f20 2024-06-10T01:23:57.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.975+0000 7f0119b05700 1 --2- 172.21.15.7:0/1517096366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ee19b640 0x55a1ee1909f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:23:57.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.975+0000 7f011ab07700 1 --2- 172.21.15.7:0/1517096366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ee19bf60 0x55a1ee19cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:23:57.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.975+0000 7f011a306700 1 --2- 172.21.15.7:0/1517096366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ee195f20 0x55a1ee1959e0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f011000b510 tx=0x7f0110010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb3c42fdf60314 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:57.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.975+0000 7f00fffff700 1 -- 172.21.15.7:0/1517096366 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110011b30 con 0x55a1ee195f20 2024-06-10T01:23:57.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.975+0000 7f00fffff700 1 -- 172.21.15.7:0/1517096366 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0110023070 con 0x55a1ee195f20 2024-06-10T01:23:57.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.975+0000 7f00fffff700 1 -- 172.21.15.7:0/1517096366 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01100287a0 con 0x55a1ee195f20 2024-06-10T01:23:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.976+0000 7f0126dedb80 1 -- 172.21.15.7:0/1517096366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ee195f20 msgr2=0x55a1ee1959e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.976+0000 7f0126dedb80 1 --2- 172.21.15.7:0/1517096366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ee195f20 0x55a1ee1959e0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f011000b510 tx=0x7f0110010860 comp rx=0 tx=0).stop 2024-06-10T01:23:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.976+0000 7f0126dedb80 1 -- 172.21.15.7:0/1517096366 shutdown_connections 2024-06-10T01:23:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.976+0000 7f0126dedb80 1 --2- 172.21.15.7:0/1517096366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ee19bf60 0x55a1ee19cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.976+0000 7f0126dedb80 1 --2- 172.21.15.7:0/1517096366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ee195f20 0x55a1ee1959e0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.976+0000 7f0126dedb80 1 --2- 172.21.15.7:0/1517096366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ee19b640 0x55a1ee1909f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.976+0000 7f0126dedb80 1 -- 172.21.15.7:0/1517096366 >> 172.21.15.7:0/1517096366 conn(0x55a1ee0c0cc0 msgr2=0x55a1ee19c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.976+0000 7f0126dedb80 1 -- 172.21.15.7:0/1517096366 shutdown_connections 2024-06-10T01:23:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.976+0000 7f0126dedb80 1 -- 172.21.15.7:0/1517096366 wait complete. 2024-06-10T01:23:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.977+0000 7f0126dedb80 1 -- start start 2024-06-10T01:23:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.977+0000 7f0126dedb80 1 Processor -- start 2024-06-10T01:23:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.977+0000 7f0126dedb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ee195f20 0x55a1ee1a81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.977+0000 7f0126dedb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ee19b640 0x55a1ee1b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.977+0000 7f0126dedb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ee19bf60 0x55a1ee1b6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.977+0000 7f0126dedb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f00f800d050 con 0x55a1ee19bf60 2024-06-10T01:23:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.977+0000 7f0126dedb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f00f800e4c0 con 0x55a1ee19b640 2024-06-10T01:23:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.977+0000 7f0126dedb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f00f800d720 con 0x55a1ee195f20 2024-06-10T01:23:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.977+0000 7f0126dedb80 10 client.0 _ll_register_callbacks cb 0x55a1ee081920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:23:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.977+0000 7f011ab07700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ee19bf60 0x55a1ee1b6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.977+0000 7f011ab07700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ee19bf60 0x55a1ee1b6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35202/0 (socket says 172.21.15.7:35202) 2024-06-10T01:23:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.977+0000 7f011ab07700 1 -- 172.21.15.7:0/1833670888 learned_addr learned my addr 172.21.15.7:0/1833670888 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:23:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.977+0000 7f0119b05700 1 --2- 172.21.15.7:0/1833670888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ee19b640 0x55a1ee1b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.977+0000 7f011a306700 1 --2- 172.21.15.7:0/1833670888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ee195f20 0x55a1ee1a81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.978+0000 7f0119b05700 1 -- 172.21.15.7:0/1833670888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ee195f20 msgr2=0x55a1ee1a81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.978+0000 7f0119b05700 1 --2- 172.21.15.7:0/1833670888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ee195f20 0x55a1ee1a81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.978+0000 7f0119b05700 1 -- 172.21.15.7:0/1833670888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ee19bf60 msgr2=0x55a1ee1b6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:23:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.978+0000 7f0119b05700 1 --2- 172.21.15.7:0/1833670888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ee19bf60 0x55a1ee1b6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.978+0000 7f0119b05700 1 -- 172.21.15.7:0/1833670888 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1ee1b7500 con 0x55a1ee19b640 2024-06-10T01:23:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.978+0000 7f011ab07700 1 --2- 172.21.15.7:0/1833670888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ee19bf60 0x55a1ee1b6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:23:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.978+0000 7f0119b05700 1 --2- 172.21.15.7:0/1833670888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ee19b640 0x55a1ee1b6a80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x55a1ee15b2e0 tx=0x7f010409b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3c4c246e64de73c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:23:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.978+0000 7f00fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:23:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.978+0000 7f00fdffb700 1 -- 172.21.15.7:0/1833670888 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010409fe10 con 0x55a1ee19b640 2024-06-10T01:23:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.978+0000 7f00fdffb700 1 -- 172.21.15.7:0/1833670888 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f0104099d60 con 0x55a1ee19b640 2024-06-10T01:23:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.978+0000 7f0126dedb80 1 -- 172.21.15.7:0/1833670888 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0110023070 con 0x55a1ee19b640 2024-06-10T01:23:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.978+0000 7f00fdffb700 1 -- 172.21.15.7:0/1833670888 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f0104005730 con 0x55a1ee19b640 2024-06-10T01:23:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.978+0000 7f0126dedb80 10 client.29054 fetch_fsmap learned FSMap version 50 2024-06-10T01:23:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.978+0000 7f0126dedb80 1 -- 172.21.15.7:0/1833670888 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55a1ee21a0a0 con 0x55a1ee19b640 2024-06-10T01:23:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.979+0000 7f00fdffb700 1 -- 172.21.15.7:0/1833670888 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0110023070 con 0x55a1ee19b640 2024-06-10T01:23:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.979+0000 7f00fdffb700 1 -- 172.21.15.7:0/1833670888 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7f01040ba1f0 con 0x55a1ee19b640 2024-06-10T01:23:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.979+0000 7f0126dedb80 10 client.29054 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:23:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.979+0000 7f0126dedb80 10 client.29054 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:23:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.979+0000 7f0126dedb80 20 client.29054 populate_metadata read hostname 'smithi007' 2024-06-10T01:23:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.979+0000 7f0126dedb80 1 --2- 172.21.15.7:0/1833670888 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55a1ee1af780 0x55a1ee21c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:23:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.979+0000 7f0126dedb80 4 client.29054 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:23:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.979+0000 7f0126dedb80 1 -- 172.21.15.7:0/1833670888 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a1ee0ef290 con 0x55a1ee1af780 2024-06-10T01:23:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.979+0000 7f011a306700 1 --2- 172.21.15.7:0/1833670888 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55a1ee1af780 0x55a1ee21c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:23:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.980+0000 7f00fdffb700 10 client.29054 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:23:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.980+0000 7f011a306700 1 --2- 172.21.15.7:0/1833670888 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55a1ee1af780 0x55a1ee21c3f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=71f73370f0c96ecf server_cookie=da42670c91f09f7b in_seq=0 out_seq=0 2024-06-10T01:23:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.980+0000 7f00fdffb700 1 -- 172.21.15.7:0/1833670888 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a1ee0ef290 con 0x55a1ee1af780 2024-06-10T01:23:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.980+0000 7f00fdffb700 10 client.29054 handle_command_reply: tid=0 2024-06-10T01:23:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.980+0000 7f0126dedb80 1 client.29054 shutdown 2024-06-10T01:23:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.980+0000 7f0126dedb80 1 -- 172.21.15.7:0/1833670888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ee19b640 msgr2=0x55a1ee1b6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.980+0000 7f0126dedb80 1 --2- 172.21.15.7:0/1833670888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ee19b640 0x55a1ee1b6a80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x55a1ee15b2e0 tx=0x7f010409b690 comp rx=0 tx=0).stop 2024-06-10T01:23:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.980+0000 7f0126dedb80 1 -- 172.21.15.7:0/1833670888 shutdown_connections 2024-06-10T01:23:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.980+0000 7f0126dedb80 1 --2- 172.21.15.7:0/1833670888 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55a1ee1af780 0x55a1ee21c3f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.980+0000 7f0126dedb80 1 --2- 172.21.15.7:0/1833670888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1ee19bf60 0x55a1ee1b6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.980+0000 7f00fdffb700 0 client.29054 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:23:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.980+0000 7f0126dedb80 1 --2- 172.21.15.7:0/1833670888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1ee195f20 0x55a1ee1a81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.980+0000 7f0126dedb80 1 --2- 172.21.15.7:0/1833670888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1ee19b640 0x55a1ee1b6a80 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.980+0000 7f0126dedb80 1 -- 172.21.15.7:0/1833670888 >> 172.21.15.7:0/1833670888 conn(0x55a1ee0c0cc0 msgr2=0x55a1ee18a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.981+0000 7f0126dedb80 1 -- 172.21.15.7:0/1833670888 shutdown_connections 2024-06-10T01:23:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.981+0000 7f0126dedb80 1 -- 172.21.15.7:0/1833670888 wait complete. 2024-06-10T01:23:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.981+0000 7f0126dedb80 20 client.29054 trim_cache size 0 max 16384 2024-06-10T01:23:57.980 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:23:57.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.981+0000 7f011c56a700 1 -- 172.21.15.7:0/3757786560 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f01000a2c20 msgr2=0x7f01000a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:57.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.981+0000 7f011c56a700 1 --2- 172.21.15.7:0/3757786560 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f01000a2c20 0x7f01000a50e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0114199b10 tx=0x7f011000b410 comp rx=0 tx=0).stop 2024-06-10T01:23:57.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.981+0000 7f011c56a700 1 -- 172.21.15.7:0/3757786560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f011406b3a0 msgr2=0x7f011419ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:23:57.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.981+0000 7f011c56a700 1 --2- 172.21.15.7:0/3757786560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f011406b3a0 0x7f011419ed50 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f010400e9c0 tx=0x7f010400ecd0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.981+0000 7f011c56a700 1 -- 172.21.15.7:0/3757786560 shutdown_connections 2024-06-10T01:23:57.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.981+0000 7f011c56a700 1 --2- 172.21.15.7:0/3757786560 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f01000a2c20 0x7f01000a50e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.981+0000 7f011c56a700 1 --2- 172.21.15.7:0/3757786560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01141084d0 0x7f01141989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.981+0000 7f011c56a700 1 --2- 172.21.15.7:0/3757786560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f011406aa00 0x7f011419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.981+0000 7f011c56a700 1 --2- 172.21.15.7:0/3757786560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f011406b3a0 0x7f011419ed50 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:23:57.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.981+0000 7f011c56a700 1 -- 172.21.15.7:0/3757786560 >> 172.21.15.7:0/3757786560 conn(0x7f0114078330 msgr2=0x7f01140ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:23:57.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.982+0000 7f011c56a700 1 -- 172.21.15.7:0/3757786560 shutdown_connections 2024-06-10T01:23:57.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:23:57.982+0000 7f011c56a700 1 -- 172.21.15.7:0/3757786560 wait complete. 2024-06-10T01:23:58.534 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 19.75 MB/sec execute 155 sec latency 31471.686 ms 2024-06-10T01:23:59.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:23:59 smithi175 ceph-mon[36636]: pgmap v972: 129 pgs: 129 active+clean; 550 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 57 KiB/s wr, 48 op/s 2024-06-10T01:23:59.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:23:59 smithi007 ceph-mon[32398]: pgmap v972: 129 pgs: 129 active+clean; 550 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 57 KiB/s wr, 48 op/s 2024-06-10T01:23:59.534 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97431 19.63 MB/sec execute 156 sec latency 32471.743 ms 2024-06-10T01:23:59.578 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:23:59 smithi078 ceph-mon[36683]: pgmap v972: 129 pgs: 129 active+clean; 550 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 57 KiB/s wr, 48 op/s 2024-06-10T01:23:59.960 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:24:00.409 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:24:00.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.418+0000 7f0c135ec700 1 -- 172.21.15.7:0/2186071164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c06ab30 msgr2=0x7f0c0c10f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:00.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.418+0000 7f0c135ec700 1 --2- 172.21.15.7:0/2186071164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c06ab30 0x7f0c0c10f320 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400b0a0 tx=0x7f0c0400b3b0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.418+0000 7f0c135ec700 1 -- 172.21.15.7:0/2186071164 shutdown_connections 2024-06-10T01:24:00.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.418+0000 7f0c135ec700 1 --2- 172.21.15.7:0/2186071164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c06ab30 0x7f0c0c10f320 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.418+0000 7f0c135ec700 1 --2- 172.21.15.7:0/2186071164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c0c108500 0x7f0c0c1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.418+0000 7f0c135ec700 1 --2- 172.21.15.7:0/2186071164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c0c06a170 0x7f0c0c06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.418+0000 7f0c135ec700 1 -- 172.21.15.7:0/2186071164 >> 172.21.15.7:0/2186071164 conn(0x7f0c0c078340 msgr2=0x7f0c0c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:00.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.419+0000 7f0c135ec700 1 -- 172.21.15.7:0/2186071164 shutdown_connections 2024-06-10T01:24:00.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.419+0000 7f0c135ec700 1 -- 172.21.15.7:0/2186071164 wait complete. 2024-06-10T01:24:00.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.419+0000 7f0c135ec700 1 Processor -- start 2024-06-10T01:24:00.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.419+0000 7f0c135ec700 1 -- start start 2024-06-10T01:24:00.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.420+0000 7f0c135ec700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c06a170 0x7f0c0c101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.420+0000 7f0c135ec700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c0c06ab30 0x7f0c0c1019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.420+0000 7f0c135ec700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c0c108500 0x7f0c0c101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.420+0000 7f0c135ec700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0c112760 con 0x7f0c0c06a170 2024-06-10T01:24:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.420+0000 7f0c135ec700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0c1125e0 con 0x7f0c0c06ab30 2024-06-10T01:24:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.420+0000 7f0c135ec700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0c1128e0 con 0x7f0c0c108500 2024-06-10T01:24:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.420+0000 7f0c11388700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c06a170 0x7f0c0c101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.420+0000 7f0c11388700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c06a170 0x7f0c0c101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35210/0 (socket says 172.21.15.7:35210) 2024-06-10T01:24:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.420+0000 7f0c11388700 1 -- 172.21.15.7:0/4088449773 learned_addr learned my addr 172.21.15.7:0/4088449773 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.420+0000 7f0c10b87700 1 --2- 172.21.15.7:0/4088449773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c0c06ab30 0x7f0c0c1019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.420+0000 7f0c11b89700 1 --2- 172.21.15.7:0/4088449773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c0c108500 0x7f0c0c101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.420+0000 7f0c11388700 1 -- 172.21.15.7:0/4088449773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c0c108500 msgr2=0x7f0c0c101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.420+0000 7f0c11388700 1 --2- 172.21.15.7:0/4088449773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c0c108500 0x7f0c0c101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.420+0000 7f0c11388700 1 -- 172.21.15.7:0/4088449773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c0c06ab30 msgr2=0x7f0c0c1019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.420+0000 7f0c11388700 1 --2- 172.21.15.7:0/4088449773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c0c06ab30 0x7f0c0c1019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.420+0000 7f0c11388700 1 -- 172.21.15.7:0/4088449773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c0c1a70e0 con 0x7f0c0c06a170 2024-06-10T01:24:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.420+0000 7f0c11b89700 1 --2- 172.21.15.7:0/4088449773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c0c108500 0x7f0c0c101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:24:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.420+0000 7f0c10b87700 1 --2- 172.21.15.7:0/4088449773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c0c06ab30 0x7f0c0c1019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:24:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.420+0000 7f0c11388700 1 --2- 172.21.15.7:0/4088449773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c06a170 0x7f0c0c101460 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f0c0800d950 tx=0x7f0c0800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.421+0000 7f0c027fc700 1 -- 172.21.15.7:0/4088449773 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08016070 con 0x7f0c0c06a170 2024-06-10T01:24:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.421+0000 7f0c027fc700 1 -- 172.21.15.7:0/4088449773 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c08009e20 con 0x7f0c0c06a170 2024-06-10T01:24:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.421+0000 7f0c135ec700 1 -- 172.21.15.7:0/4088449773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c0c1a73d0 con 0x7f0c0c06a170 2024-06-10T01:24:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.421+0000 7f0c027fc700 1 -- 172.21.15.7:0/4088449773 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0800f620 con 0x7f0c0c06a170 2024-06-10T01:24:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.421+0000 7f0c135ec700 1 -- 172.21.15.7:0/4088449773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c0c1a7910 con 0x7f0c0c06a170 2024-06-10T01:24:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.423+0000 7f0c027fc700 1 -- 172.21.15.7:0/4088449773 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0c08019070 con 0x7f0c0c06a170 2024-06-10T01:24:00.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.424+0000 7f0c027fc700 1 --2- 172.21.15.7:0/4088449773 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0bf80a2c20 0x7f0bf80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:00.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.424+0000 7f0c027fc700 1 -- 172.21.15.7:0/4088449773 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f0c080cf2e0 con 0x7f0c0c06a170 2024-06-10T01:24:00.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.424+0000 7f0c10b87700 1 --2- 172.21.15.7:0/4088449773 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0bf80a2c20 0x7f0bf80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:00.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.425+0000 7f0c1de6fb80 1 Processor -- start 2024-06-10T01:24:00.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.425+0000 7f0c1de6fb80 1 -- start start 2024-06-10T01:24:00.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.425+0000 7f0c10b87700 1 --2- 172.21.15.7:0/4088449773 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0bf80a2c20 0x7f0bf80a50e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c078940 tx=0x7f0bfc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:00.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.425+0000 7f0c1de6fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0a2f97c40 0x55d0a2f98020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:00.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.425+0000 7f0c1de6fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0a2f98560 0x55d0a2f91e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:00.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.425+0000 7f0c1de6fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0a2f923d0 0x55d0a2f92810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:00.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.425+0000 7f0c1de6fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d0a2ee3910 con 0x55d0a2f923d0 2024-06-10T01:24:00.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.425+0000 7f0c1de6fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d0a2e8b820 con 0x55d0a2f98560 2024-06-10T01:24:00.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.425+0000 7f0c1de6fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d0a2edf8d0 con 0x55d0a2f97c40 2024-06-10T01:24:00.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.425+0000 7f0c11b89700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0a2f923d0 0x55d0a2f92810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.425+0000 7f0c11b89700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0a2f923d0 0x55d0a2f92810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35222/0 (socket says 172.21.15.7:35222) 2024-06-10T01:24:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.425+0000 7f0c11b89700 1 -- 172.21.15.7:0/511672900 learned_addr learned my addr 172.21.15.7:0/511672900 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.425+0000 7f0c11388700 1 --2- 172.21.15.7:0/511672900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0a2f97c40 0x55d0a2f98020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.425+0000 7f0c10b87700 1 --2- 172.21.15.7:0/511672900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0a2f98560 0x55d0a2f91e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.426+0000 7f0c11388700 1 -- 172.21.15.7:0/511672900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0a2f98560 msgr2=0x55d0a2f91e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.426+0000 7f0c11388700 1 --2- 172.21.15.7:0/511672900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0a2f98560 0x55d0a2f91e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.426+0000 7f0c11388700 1 -- 172.21.15.7:0/511672900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0a2f923d0 msgr2=0x55d0a2f92810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.426+0000 7f0c11388700 1 --2- 172.21.15.7:0/511672900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0a2f923d0 0x55d0a2f92810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.426+0000 7f0c11388700 1 -- 172.21.15.7:0/511672900 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0a2e82050 con 0x55d0a2f97c40 2024-06-10T01:24:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.426+0000 7f0c11b89700 1 --2- 172.21.15.7:0/511672900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0a2f923d0 0x55d0a2f92810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:24:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.426+0000 7f0c11388700 1 --2- 172.21.15.7:0/511672900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0a2f97c40 0x55d0a2f98020 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0c080ccab0 tx=0x7f0c08010b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56c8d53bdb2cb6f6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.426+0000 7f0bf77fe700 1 -- 172.21.15.7:0/511672900 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08020070 con 0x55d0a2f97c40 2024-06-10T01:24:00.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.426+0000 7f0bf77fe700 1 -- 172.21.15.7:0/511672900 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c080a4430 con 0x55d0a2f97c40 2024-06-10T01:24:00.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.426+0000 7f0bf77fe700 1 -- 172.21.15.7:0/511672900 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08097e30 con 0x55d0a2f97c40 2024-06-10T01:24:00.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.427+0000 7f0c1de6fb80 1 -- 172.21.15.7:0/511672900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0a2f97c40 msgr2=0x55d0a2f98020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:00.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.427+0000 7f0c1de6fb80 1 --2- 172.21.15.7:0/511672900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0a2f97c40 0x55d0a2f98020 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0c080ccab0 tx=0x7f0c08010b50 comp rx=0 tx=0).stop 2024-06-10T01:24:00.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.427+0000 7f0c1de6fb80 1 -- 172.21.15.7:0/511672900 shutdown_connections 2024-06-10T01:24:00.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.427+0000 7f0c1de6fb80 1 --2- 172.21.15.7:0/511672900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0a2f923d0 0x55d0a2f92810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.427+0000 7f0c1de6fb80 1 --2- 172.21.15.7:0/511672900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0a2f97c40 0x55d0a2f98020 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.427+0000 7f0c1de6fb80 1 --2- 172.21.15.7:0/511672900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0a2f98560 0x55d0a2f91e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.427+0000 7f0c1de6fb80 1 -- 172.21.15.7:0/511672900 >> 172.21.15.7:0/511672900 conn(0x55d0a2ebcdc0 msgr2=0x55d0a2f8d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:00.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.427+0000 7f0c1de6fb80 1 -- 172.21.15.7:0/511672900 shutdown_connections 2024-06-10T01:24:00.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.427+0000 7f0c1de6fb80 1 -- 172.21.15.7:0/511672900 wait complete. 2024-06-10T01:24:00.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.427+0000 7f0c1de6fb80 1 -- start start 2024-06-10T01:24:00.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.427+0000 7f0c1de6fb80 1 Processor -- start 2024-06-10T01:24:00.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.428+0000 7f0c1de6fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0a2f923d0 0x55d0a2f9e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:00.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.428+0000 7f0c1de6fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0a2f97c40 0x55d0a2f9b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:00.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.428+0000 7f0c1de6fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0a2f98560 0x55d0a2f9bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:00.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.428+0000 7f0c1de6fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d0a2f9ef10 con 0x55d0a2f97c40 2024-06-10T01:24:00.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.428+0000 7f0c1de6fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d0a2e83cf0 con 0x55d0a2f98560 2024-06-10T01:24:00.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.428+0000 7f0c1de6fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d0a2f9f090 con 0x55d0a2f923d0 2024-06-10T01:24:00.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.428+0000 7f0c1de6fb80 10 client.0 _ll_register_callbacks cb 0x55d0a2e7da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:24:00.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.428+0000 7f0c10b87700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0a2f97c40 0x55d0a2f9b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:00.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.428+0000 7f0c11b89700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0a2f98560 0x55d0a2f9bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:00.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.428+0000 7f0c11b89700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0a2f98560 0x55d0a2f9bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:59804/0 (socket says 172.21.15.7:59804) 2024-06-10T01:24:00.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.428+0000 7f0c11b89700 1 -- 172.21.15.7:0/1470898324 learned_addr learned my addr 172.21.15.7:0/1470898324 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:00.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.428+0000 7f0c10b87700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0a2f97c40 0x55d0a2f9b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35230/0 (socket says 172.21.15.7:35230) 2024-06-10T01:24:00.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.428+0000 7f0c11388700 1 --2- 172.21.15.7:0/1470898324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0a2f923d0 0x55d0a2f9e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:00.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.428+0000 7f0c11b89700 1 -- 172.21.15.7:0/1470898324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0a2f923d0 msgr2=0x55d0a2f9e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:00.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.428+0000 7f0c11b89700 1 --2- 172.21.15.7:0/1470898324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0a2f923d0 0x55d0a2f9e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.428+0000 7f0c11b89700 1 -- 172.21.15.7:0/1470898324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0a2f97c40 msgr2=0x55d0a2f9b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:00.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.428+0000 7f0c11b89700 1 --2- 172.21.15.7:0/1470898324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0a2f97c40 0x55d0a2f9b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.428+0000 7f0c11b89700 1 -- 172.21.15.7:0/1470898324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0a2e329d0 con 0x55d0a2f98560 2024-06-10T01:24:00.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.429+0000 7f0c11b89700 1 --2- 172.21.15.7:0/1470898324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0a2f98560 0x55d0a2f9bbc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0c04006100 tx=0x7f0c0400beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3820e37b0dea6f6d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:00.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.429+0000 7f0bf4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:24:00.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.429+0000 7f0bf4ff9700 1 -- 172.21.15.7:0/1470898324 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c04020070 con 0x55d0a2f98560 2024-06-10T01:24:00.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.429+0000 7f0c1de6fb80 1 -- 172.21.15.7:0/1470898324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c080a4430 con 0x55d0a2f98560 2024-06-10T01:24:00.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.429+0000 7f0bf4ff9700 1 -- 172.21.15.7:0/1470898324 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f0c040076e0 con 0x55d0a2f98560 2024-06-10T01:24:00.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.429+0000 7f0bf4ff9700 1 -- 172.21.15.7:0/1470898324 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f0c0402e050 con 0x55d0a2f98560 2024-06-10T01:24:00.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.429+0000 7f0c1de6fb80 10 client.29066 fetch_fsmap learned FSMap version 50 2024-06-10T01:24:00.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.429+0000 7f0c1de6fb80 1 -- 172.21.15.7:0/1470898324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55d0a2fa9910 con 0x55d0a2f98560 2024-06-10T01:24:00.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.429+0000 7f0bf4ff9700 1 -- 172.21.15.7:0/1470898324 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c080a4430 con 0x55d0a2f98560 2024-06-10T01:24:00.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.430+0000 7f0bf4ff9700 1 -- 172.21.15.7:0/1470898324 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7f0c04011070 con 0x55d0a2f98560 2024-06-10T01:24:00.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.430+0000 7f0c1de6fb80 10 client.29066 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:24:00.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.430+0000 7f0c1de6fb80 10 client.29066 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:24:00.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.430+0000 7f0c1de6fb80 20 client.29066 populate_metadata read hostname 'smithi007' 2024-06-10T01:24:00.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.430+0000 7f0c1de6fb80 1 --2- 172.21.15.7:0/1470898324 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d0a2fabea0 0x55d0a2fae290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:00.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.430+0000 7f0c1de6fb80 4 client.29066 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:24:00.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.430+0000 7f0c1de6fb80 1 -- 172.21.15.7:0/1470898324 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0a2ef0e10 con 0x55d0a2fabea0 2024-06-10T01:24:00.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.430+0000 7f0c11388700 1 --2- 172.21.15.7:0/1470898324 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d0a2fabea0 0x55d0a2fae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:00.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.430+0000 7f0c11388700 1 --2- 172.21.15.7:0/1470898324 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d0a2fabea0 0x55d0a2fae290 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b06791c97d8d02e2 server_cookie=9bc0c3ef29fe9d0a in_seq=0 out_seq=0 2024-06-10T01:24:00.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.430+0000 7f0bf4ff9700 10 client.29066 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:00.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.431+0000 7f0bf4ff9700 1 -- 172.21.15.7:0/1470898324 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d0a2ef0e10 con 0x55d0a2fabea0 2024-06-10T01:24:00.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.431+0000 7f0bf4ff9700 10 client.29066 handle_command_reply: tid=0 2024-06-10T01:24:00.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.431+0000 7f0c1de6fb80 1 client.29066 shutdown 2024-06-10T01:24:00.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.431+0000 7f0c1de6fb80 1 -- 172.21.15.7:0/1470898324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0a2f98560 msgr2=0x55d0a2f9bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:00.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.431+0000 7f0c1de6fb80 1 --2- 172.21.15.7:0/1470898324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0a2f98560 0x55d0a2f9bbc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0c04006100 tx=0x7f0c0400beb0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.431+0000 7f0c1de6fb80 1 -- 172.21.15.7:0/1470898324 shutdown_connections 2024-06-10T01:24:00.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.431+0000 7f0c1de6fb80 1 --2- 172.21.15.7:0/1470898324 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d0a2fabea0 0x55d0a2fae290 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.431+0000 7f0c1de6fb80 1 --2- 172.21.15.7:0/1470898324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0a2f98560 0x55d0a2f9bbc0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.431+0000 7f0c1de6fb80 1 --2- 172.21.15.7:0/1470898324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0a2f923d0 0x55d0a2f9e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.431+0000 7f0bf4ff9700 0 client.29066 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:00.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.431+0000 7f0c1de6fb80 1 --2- 172.21.15.7:0/1470898324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0a2f97c40 0x55d0a2f9b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.431+0000 7f0c1de6fb80 1 -- 172.21.15.7:0/1470898324 >> 172.21.15.7:0/1470898324 conn(0x55d0a2ebcdc0 msgr2=0x55d0a2f927f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:00.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.432+0000 7f0c1de6fb80 1 -- 172.21.15.7:0/1470898324 shutdown_connections 2024-06-10T01:24:00.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.432+0000 7f0c1de6fb80 1 -- 172.21.15.7:0/1470898324 wait complete. 2024-06-10T01:24:00.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.432+0000 7f0c1de6fb80 20 client.29066 trim_cache size 0 max 16384 2024-06-10T01:24:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.454+0000 7f0c1de6fb80 1 Processor -- start 2024-06-10T01:24:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.454+0000 7f0c1de6fb80 1 -- start start 2024-06-10T01:24:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.454+0000 7f0c1de6fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0a2f923d0 0x55d0a2fa81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.454+0000 7f0c1de6fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0a2f97c40 0x55d0a2f91e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.454+0000 7f0c1de6fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0a2f98560 0x55d0a2f991d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.454+0000 7f0c1de6fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0c1b2de0 con 0x55d0a2f923d0 2024-06-10T01:24:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.454+0000 7f0c1de6fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0c1b0670 con 0x55d0a2f97c40 2024-06-10T01:24:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.454+0000 7f0c1de6fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0c1ae4c0 con 0x55d0a2f98560 2024-06-10T01:24:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.454+0000 7f0c11388700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0a2f923d0 0x55d0a2fa81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.454+0000 7f0c11388700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0a2f923d0 0x55d0a2fa81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35232/0 (socket says 172.21.15.7:35232) 2024-06-10T01:24:00.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.454+0000 7f0c11388700 1 -- 172.21.15.7:0/3769560531 learned_addr learned my addr 172.21.15.7:0/3769560531 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:00.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.454+0000 7f0c10b87700 1 --2- 172.21.15.7:0/3769560531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0a2f97c40 0x55d0a2f91e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:00.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.454+0000 7f0c11b89700 1 --2- 172.21.15.7:0/3769560531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0a2f98560 0x55d0a2f991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:00.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.454+0000 7f0c11388700 1 -- 172.21.15.7:0/3769560531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0a2f98560 msgr2=0x55d0a2f991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:00.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.454+0000 7f0c11388700 1 --2- 172.21.15.7:0/3769560531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0a2f98560 0x55d0a2f991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.454+0000 7f0c11388700 1 -- 172.21.15.7:0/3769560531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0a2f97c40 msgr2=0x55d0a2f91e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:00.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.454+0000 7f0c11388700 1 --2- 172.21.15.7:0/3769560531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0a2f97c40 0x55d0a2f91e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.454+0000 7f0c11388700 1 -- 172.21.15.7:0/3769560531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0a2f99c10 con 0x55d0a2f923d0 2024-06-10T01:24:00.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.455+0000 7f0c11388700 1 --2- 172.21.15.7:0/3769560531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0a2f923d0 0x55d0a2fa81b0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f0c08009fd0 tx=0x7f0c08086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407664fbb7e72beb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:00.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.455+0000 7f0bf6ffd700 1 -- 172.21.15.7:0/3769560531 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08020040 con 0x55d0a2f923d0 2024-06-10T01:24:00.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.455+0000 7f0bf6ffd700 1 -- 172.21.15.7:0/3769560531 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c080b9070 con 0x55d0a2f923d0 2024-06-10T01:24:00.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.455+0000 7f0bf6ffd700 1 -- 172.21.15.7:0/3769560531 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08097e30 con 0x55d0a2f923d0 2024-06-10T01:24:00.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.456+0000 7f0c1de6fb80 1 -- 172.21.15.7:0/3769560531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0a2f923d0 msgr2=0x55d0a2fa81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:00.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.456+0000 7f0c1de6fb80 1 --2- 172.21.15.7:0/3769560531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0a2f923d0 0x55d0a2fa81b0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f0c08009fd0 tx=0x7f0c08086040 comp rx=0 tx=0).stop 2024-06-10T01:24:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.456+0000 7f0c1de6fb80 1 -- 172.21.15.7:0/3769560531 shutdown_connections 2024-06-10T01:24:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.456+0000 7f0c1de6fb80 1 --2- 172.21.15.7:0/3769560531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0a2f98560 0x55d0a2f991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.456+0000 7f0c1de6fb80 1 --2- 172.21.15.7:0/3769560531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0a2f923d0 0x55d0a2fa81b0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.456+0000 7f0c1de6fb80 1 --2- 172.21.15.7:0/3769560531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0a2f97c40 0x55d0a2f91e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.456+0000 7f0c1de6fb80 1 -- 172.21.15.7:0/3769560531 >> 172.21.15.7:0/3769560531 conn(0x55d0a2ebcdc0 msgr2=0x55d0a2f95b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.456+0000 7f0c1de6fb80 1 -- 172.21.15.7:0/3769560531 shutdown_connections 2024-06-10T01:24:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.456+0000 7f0c1de6fb80 1 -- 172.21.15.7:0/3769560531 wait complete. 2024-06-10T01:24:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.456+0000 7f0c1de6fb80 1 -- start start 2024-06-10T01:24:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.456+0000 7f0c1de6fb80 1 Processor -- start 2024-06-10T01:24:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.456+0000 7f0c1de6fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0a2f923d0 0x55d0a2fa12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.456+0000 7f0c1de6fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0a2f97c40 0x55d0a2fa1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.456+0000 7f0c1de6fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0a2f98560 0x55d0a2fa1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.457+0000 7f0c1de6fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0c1aeb30 con 0x55d0a2f97c40 2024-06-10T01:24:00.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.457+0000 7f0c1de6fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0c1affa0 con 0x55d0a2f98560 2024-06-10T01:24:00.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.457+0000 7f0c1de6fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0c1af200 con 0x55d0a2f923d0 2024-06-10T01:24:00.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.457+0000 7f0c1de6fb80 10 client.0 _ll_register_callbacks cb 0x55d0a2e7da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:24:00.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.457+0000 7f0c10b87700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0a2f97c40 0x55d0a2fa1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:00.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.457+0000 7f0c10b87700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0a2f97c40 0x55d0a2fa1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35240/0 (socket says 172.21.15.7:35240) 2024-06-10T01:24:00.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.457+0000 7f0c10b87700 1 -- 172.21.15.7:0/1154303070 learned_addr learned my addr 172.21.15.7:0/1154303070 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:00.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.457+0000 7f0c11388700 1 --2- 172.21.15.7:0/1154303070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0a2f923d0 0x55d0a2fa12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:00.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.457+0000 7f0c11b89700 1 --2- 172.21.15.7:0/1154303070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0a2f98560 0x55d0a2fa1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:00.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.457+0000 7f0c10b87700 1 -- 172.21.15.7:0/1154303070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0a2f923d0 msgr2=0x55d0a2fa12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:00.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.457+0000 7f0c10b87700 1 --2- 172.21.15.7:0/1154303070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0a2f923d0 0x55d0a2fa12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.457+0000 7f0c10b87700 1 -- 172.21.15.7:0/1154303070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0a2f98560 msgr2=0x55d0a2fa1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:00.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.457+0000 7f0c10b87700 1 --2- 172.21.15.7:0/1154303070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0a2f98560 0x55d0a2fa1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.457+0000 7f0c10b87700 1 -- 172.21.15.7:0/1154303070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0a3016b20 con 0x55d0a2f97c40 2024-06-10T01:24:00.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.457+0000 7f0c11b89700 1 --2- 172.21.15.7:0/1154303070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0a2f98560 0x55d0a2fa1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:24:00.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.457+0000 7f0c10b87700 1 --2- 172.21.15.7:0/1154303070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0a2f97c40 0x55d0a2fa1830 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc0094f0 tx=0x7f0bfc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=734b835e780f1353 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:00.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.457+0000 7f0bf4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:24:00.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.457+0000 7f0bf4ff9700 1 -- 172.21.15.7:0/1154303070 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc02e060 con 0x55d0a2f97c40 2024-06-10T01:24:00.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.457+0000 7f0bf4ff9700 1 -- 172.21.15.7:0/1154303070 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f0bfc01e940 con 0x55d0a2f97c40 2024-06-10T01:24:00.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.457+0000 7f0c1de6fb80 1 -- 172.21.15.7:0/1154303070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c080b9070 con 0x55d0a2f97c40 2024-06-10T01:24:00.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.458+0000 7f0bf4ff9700 1 -- 172.21.15.7:0/1154303070 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f0bfc035050 con 0x55d0a2f97c40 2024-06-10T01:24:00.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.458+0000 7f0c1de6fb80 10 client.19869 fetch_fsmap learned FSMap version 50 2024-06-10T01:24:00.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.458+0000 7f0c1de6fb80 1 -- 172.21.15.7:0/1154303070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55d0a3017d00 con 0x55d0a2f97c40 2024-06-10T01:24:00.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.458+0000 7f0bf4ff9700 1 -- 172.21.15.7:0/1154303070 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c080b9070 con 0x55d0a2f97c40 2024-06-10T01:24:00.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.458+0000 7f0bf4ff9700 1 -- 172.21.15.7:0/1154303070 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7f0bfc028070 con 0x55d0a2f97c40 2024-06-10T01:24:00.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.458+0000 7f0c1de6fb80 10 client.19869 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:24:00.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.458+0000 7f0c1de6fb80 10 client.19869 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:24:00.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.458+0000 7f0c1de6fb80 20 client.19869 populate_metadata read hostname 'smithi007' 2024-06-10T01:24:00.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.458+0000 7f0c1de6fb80 1 --2- 172.21.15.7:0/1154303070 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d0a3018470 0x55d0a301a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:00.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.458+0000 7f0c1de6fb80 4 client.19869 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f8806db1", "release": true, "await": true}] 2024-06-10T01:24:00.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.458+0000 7f0c1de6fb80 1 -- 172.21.15.7:0/1154303070 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f8806db1", "release": true, "await": true}) -- 0x55d0a2eeb390 con 0x55d0a3018470 2024-06-10T01:24:00.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.458+0000 7f0c11388700 1 --2- 172.21.15.7:0/1154303070 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d0a3018470 0x55d0a301a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:00.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.459+0000 7f0bf4ff9700 10 client.19869 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:00.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.459+0000 7f0c11388700 1 --2- 172.21.15.7:0/1154303070 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d0a3018470 0x55d0a301a890 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d48d1ed82796dcf3 server_cookie=b2726fd82452a0ab in_seq=0 out_seq=0 2024-06-10T01:24:00.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.501+0000 7f0bf4ff9700 1 -- 172.21.15.7:0/1154303070 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d0a2eeb390 con 0x55d0a3018470 2024-06-10T01:24:00.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.501+0000 7f0bf4ff9700 10 client.19869 handle_command_reply: tid=0 2024-06-10T01:24:00.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.501+0000 7f0c1de6fb80 1 client.19869 shutdown 2024-06-10T01:24:00.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.502+0000 7f0c1de6fb80 1 -- 172.21.15.7:0/1154303070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0a2f97c40 msgr2=0x55d0a2fa1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:00.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.502+0000 7f0c1de6fb80 1 --2- 172.21.15.7:0/1154303070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0a2f97c40 0x55d0a2fa1830 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc0094f0 tx=0x7f0bfc0047c0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.502+0000 7f0c1de6fb80 1 -- 172.21.15.7:0/1154303070 shutdown_connections 2024-06-10T01:24:00.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.502+0000 7f0c1de6fb80 1 --2- 172.21.15.7:0/1154303070 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d0a3018470 0x55d0a301a890 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.502+0000 7f0c1de6fb80 1 --2- 172.21.15.7:0/1154303070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0a2f98560 0x55d0a2fa1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.502+0000 7f0c1de6fb80 1 --2- 172.21.15.7:0/1154303070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0a2f923d0 0x55d0a2fa12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.502+0000 7f0c1de6fb80 1 --2- 172.21.15.7:0/1154303070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0a2f97c40 0x55d0a2fa1830 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.502+0000 7f0c1de6fb80 1 -- 172.21.15.7:0/1154303070 >> 172.21.15.7:0/1154303070 conn(0x55d0a2ebcdc0 msgr2=0x55d0a2f99f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:00.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.502+0000 7f0bf4ff9700 0 client.19869 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:00.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.502+0000 7f0c1de6fb80 1 -- 172.21.15.7:0/1154303070 shutdown_connections 2024-06-10T01:24:00.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.502+0000 7f0c1de6fb80 1 -- 172.21.15.7:0/1154303070 wait complete. 2024-06-10T01:24:00.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.502+0000 7f0c1de6fb80 20 client.19869 trim_cache size 0 max 16384 2024-06-10T01:24:00.496 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:24:00.497 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 50, 2024-06-10T01:24:00.497 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:24:00.497 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 60, 2024-06-10T01:24:00.497 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:24:00.497 INFO:teuthology.orchestra.run.smithi007.stdout: "f8806db1": { 2024-06-10T01:24:00.497 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 60, 2024-06-10T01:24:00.497 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:24:00.497 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:24:00.497 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:24:00.497 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:24:00.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:24:00.497 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:24:00.497 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 150.0, 2024-06-10T01:24:00.498 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:24:00.498 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:24:00.498 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:24:00.498 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:24:00.498 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:24:00.498 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:24:00.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:24:00.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:24:00.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:24:00.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:24:00.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:24:00.498 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:24:00.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.503+0000 7f0c135ec700 1 -- 172.21.15.7:0/4088449773 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0bf80a2c20 msgr2=0x7f0bf80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:00.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.503+0000 7f0c135ec700 1 --2- 172.21.15.7:0/4088449773 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0bf80a2c20 0x7f0bf80a50e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c078940 tx=0x7f0bfc009260 comp rx=0 tx=0).stop 2024-06-10T01:24:00.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.503+0000 7f0c135ec700 1 -- 172.21.15.7:0/4088449773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c06a170 msgr2=0x7f0c0c101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:00.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.503+0000 7f0c135ec700 1 --2- 172.21.15.7:0/4088449773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c06a170 0x7f0c0c101460 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f0c0800d950 tx=0x7f0c0800dc60 comp rx=0 tx=0).stop 2024-06-10T01:24:00.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.503+0000 7f0c135ec700 1 -- 172.21.15.7:0/4088449773 shutdown_connections 2024-06-10T01:24:00.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.503+0000 7f0c135ec700 1 --2- 172.21.15.7:0/4088449773 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0bf80a2c20 0x7f0bf80a50e0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.503+0000 7f0c135ec700 1 --2- 172.21.15.7:0/4088449773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c0c108500 0x7f0c0c101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.503+0000 7f0c135ec700 1 --2- 172.21.15.7:0/4088449773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c06a170 0x7f0c0c101460 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.503+0000 7f0c135ec700 1 --2- 172.21.15.7:0/4088449773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c0c06ab30 0x7f0c0c1019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:00.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.503+0000 7f0c135ec700 1 -- 172.21.15.7:0/4088449773 >> 172.21.15.7:0/4088449773 conn(0x7f0c0c078340 msgr2=0x7f0c0c0ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:00.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.503+0000 7f0c135ec700 1 -- 172.21.15.7:0/4088449773 shutdown_connections 2024-06-10T01:24:00.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:00.503+0000 7f0c135ec700 1 -- 172.21.15.7:0/4088449773 wait complete. 2024-06-10T01:24:00.534 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97468 19.51 MB/sec execute 157 sec latency 33415.786 ms 2024-06-10T01:24:01.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:01 smithi175 ceph-mon[36636]: pgmap v973: 129 pgs: 129 active+clean; 550 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 59 KiB/s wr, 39 op/s 2024-06-10T01:24:01.512 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:01 smithi007 ceph-mon[32398]: pgmap v973: 129 pgs: 129 active+clean; 550 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 59 KiB/s wr, 39 op/s 2024-06-10T01:24:01.534 INFO:tasks.workunit.client.0.smithi007.stdout: 10 98193 19.63 MB/sec execute 158 sec latency 85.647 ms 2024-06-10T01:24:01.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:01 smithi078 ceph-mon[36683]: pgmap v973: 129 pgs: 129 active+clean; 550 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 59 KiB/s wr, 39 op/s 2024-06-10T01:24:02.534 INFO:tasks.workunit.client.0.smithi007.stdout: 10 98952 19.77 MB/sec execute 159 sec latency 68.304 ms 2024-06-10T01:24:02.629 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f8806db1', total seconds elapsed: 41.9 2024-06-10T01:24:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.896+0000 7fa81a963640 1 -- 172.21.15.7:0/661000707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa814103740 msgr2=0x7fa814109fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.896+0000 7fa81a963640 1 --2- 172.21.15.7:0/661000707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa814103740 0x7fa814109fd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa80c009c10 tx=0x7fa80c02a6f0 comp rx=0 tx=0).stop 2024-06-10T01:24:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.896+0000 7fa81a963640 1 -- 172.21.15.7:0/661000707 shutdown_connections 2024-06-10T01:24:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.896+0000 7fa81a963640 1 --2- 172.21.15.7:0/661000707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa814103740 0x7fa814109fd0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.896+0000 7fa81a963640 1 --2- 172.21.15.7:0/661000707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa814102d80 0x7fa814103200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.896+0000 7fa81a963640 1 --2- 172.21.15.7:0/661000707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8140ffd10 0x7fa814100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.896+0000 7fa81a963640 1 -- 172.21.15.7:0/661000707 >> 172.21.15.7:0/661000707 conn(0x7fa8140fb620 msgr2=0x7fa8140fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.896+0000 7fa81a963640 1 -- 172.21.15.7:0/661000707 shutdown_connections 2024-06-10T01:24:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.896+0000 7fa81a963640 1 -- 172.21.15.7:0/661000707 wait complete. 2024-06-10T01:24:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.896+0000 7fa81a963640 1 Processor -- start 2024-06-10T01:24:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.897+0000 7fa81a963640 1 -- start start 2024-06-10T01:24:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.897+0000 7fa81a963640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8140ffd10 0x7fa814197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.897+0000 7fa81a963640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa814102d80 0x7fa814198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.897+0000 7fa81a963640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa814103740 0x7fa81419f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.897+0000 7fa8137fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa814102d80 0x7fa814198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.897+0000 7fa813fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8140ffd10 0x7fa814197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.897+0000 7fa813fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8140ffd10 0x7fa814197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:35690/0 (socket says 172.21.15.7:35690) 2024-06-10T01:24:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.897+0000 7fa8137fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa814102d80 0x7fa814198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51596/0 (socket says 172.21.15.7:51596) 2024-06-10T01:24:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.897+0000 7fa813fff640 1 -- 172.21.15.7:0/1396396578 learned_addr learned my addr 172.21.15.7:0/1396396578 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.897+0000 7fa81a963640 1 -- 172.21.15.7:0/1396396578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa81410d640 con 0x7fa814102d80 2024-06-10T01:24:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.897+0000 7fa81a963640 1 -- 172.21.15.7:0/1396396578 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa81410d4c0 con 0x7fa814103740 2024-06-10T01:24:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.897+0000 7fa81a963640 1 -- 172.21.15.7:0/1396396578 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa81410d7c0 con 0x7fa8140ffd10 2024-06-10T01:24:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.897+0000 7fa818f00640 1 --2- 172.21.15.7:0/1396396578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa814103740 0x7fa81419f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.898+0000 7fa813fff640 1 -- 172.21.15.7:0/1396396578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa814103740 msgr2=0x7fa81419f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.898+0000 7fa813fff640 1 --2- 172.21.15.7:0/1396396578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa814103740 0x7fa81419f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.898+0000 7fa813fff640 1 -- 172.21.15.7:0/1396396578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa814102d80 msgr2=0x7fa814198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.898+0000 7fa813fff640 1 --2- 172.21.15.7:0/1396396578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa814102d80 0x7fa814198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.898+0000 7fa813fff640 1 -- 172.21.15.7:0/1396396578 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa81419f910 con 0x7fa8140ffd10 2024-06-10T01:24:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.898+0000 7fa813fff640 1 --2- 172.21.15.7:0/1396396578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8140ffd10 0x7fa814197d00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa80400e9c0 tx=0x7fa80400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.898+0000 7fa8117fa640 1 -- 172.21.15.7:0/1396396578 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80400cd80 con 0x7fa8140ffd10 2024-06-10T01:24:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.898+0000 7fa81a963640 1 -- 172.21.15.7:0/1396396578 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa81419fc00 con 0x7fa8140ffd10 2024-06-10T01:24:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.898+0000 7fa8117fa640 1 -- 172.21.15.7:0/1396396578 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8040035f0 con 0x7fa8140ffd10 2024-06-10T01:24:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.898+0000 7fa8117fa640 1 -- 172.21.15.7:0/1396396578 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa804010690 con 0x7fa8140ffd10 2024-06-10T01:24:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.898+0000 7fa81a963640 1 -- 172.21.15.7:0/1396396578 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8141a0150 con 0x7fa8140ffd10 2024-06-10T01:24:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.901+0000 7fa8117fa640 1 -- 172.21.15.7:0/1396396578 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa8040037a0 con 0x7fa8140ffd10 2024-06-10T01:24:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.901+0000 7fa81d679740 1 Processor -- start 2024-06-10T01:24:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.901+0000 7fa81d679740 1 -- start start 2024-06-10T01:24:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.901+0000 7fa81d679740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560958138e70 0x560958139290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.901+0000 7fa81d679740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560958139860 0x56095813dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.901+0000 7fa81d679740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56095813e3b0 0x560958140800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.901+0000 7fa81d679740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5609580934f0 con 0x56095813e3b0 2024-06-10T01:24:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.901+0000 7fa81d679740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560958031870 con 0x560958139860 2024-06-10T01:24:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.901+0000 7fa81d679740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56095808f4b0 con 0x560958138e70 2024-06-10T01:24:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.901+0000 7fa818f00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560958138e70 0x560958139290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.901+0000 7fa818f00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560958138e70 0x560958139290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:35698/0 (socket says 172.21.15.7:35698) 2024-06-10T01:24:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.901+0000 7fa818f00640 1 -- 172.21.15.7:0/1426162230 learned_addr learned my addr 172.21.15.7:0/1426162230 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.901+0000 7fa8137fe640 1 --2- 172.21.15.7:0/1426162230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56095813e3b0 0x560958140800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.901+0000 7fa813fff640 1 --2- 172.21.15.7:0/1426162230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560958139860 0x56095813dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.902+0000 7fa818f00640 1 -- 172.21.15.7:0/1426162230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560958139860 msgr2=0x56095813dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.902+0000 7fa818f00640 1 --2- 172.21.15.7:0/1426162230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560958139860 0x56095813dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.902+0000 7fa818f00640 1 -- 172.21.15.7:0/1426162230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56095813e3b0 msgr2=0x560958140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.902+0000 7fa818f00640 1 --2- 172.21.15.7:0/1426162230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56095813e3b0 0x560958140800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.902+0000 7fa818f00640 1 -- 172.21.15.7:0/1426162230 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56095804d9b0 con 0x560958138e70 2024-06-10T01:24:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.902+0000 7fa8117fa640 1 --2- 172.21.15.7:0/1396396578 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa7fc0a2c00 0x7fa7fc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.902+0000 7fa813fff640 1 --2- 172.21.15.7:0/1396396578 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa7fc0a2c00 0x7fa7fc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.902+0000 7fa818f00640 1 --2- 172.21.15.7:0/1426162230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560958138e70 0x560958139290 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa80c009be0 tx=0x7fa80c005ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ba3ed6f5fe41554 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.902+0000 7fa8117fa640 1 -- 172.21.15.7:0/1396396578 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fa8040cf880 con 0x7fa8140ffd10 2024-06-10T01:24:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.902+0000 7fa7fa7fc640 1 -- 172.21.15.7:0/1426162230 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c042030 con 0x560958138e70 2024-06-10T01:24:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.902+0000 7fa7fa7fc640 1 -- 172.21.15.7:0/1426162230 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa80c037040 con 0x560958138e70 2024-06-10T01:24:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.902+0000 7fa7fa7fc640 1 -- 172.21.15.7:0/1426162230 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c036070 con 0x560958138e70 2024-06-10T01:24:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.903+0000 7fa81d679740 1 -- 172.21.15.7:0/1426162230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560958138e70 msgr2=0x560958139290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.903+0000 7fa813fff640 1 --2- 172.21.15.7:0/1396396578 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa7fc0a2c00 0x7fa7fc0a50c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa8040a4440 tx=0x7fa804097f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.903+0000 7fa81d679740 1 --2- 172.21.15.7:0/1426162230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560958138e70 0x560958139290 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa80c009be0 tx=0x7fa80c005ec0 comp rx=0 tx=0).stop 2024-06-10T01:24:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.903+0000 7fa81d679740 1 -- 172.21.15.7:0/1426162230 shutdown_connections 2024-06-10T01:24:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.903+0000 7fa81d679740 1 --2- 172.21.15.7:0/1426162230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56095813e3b0 0x560958140800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.903+0000 7fa81d679740 1 --2- 172.21.15.7:0/1426162230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560958139860 0x56095813dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.903+0000 7fa81d679740 1 --2- 172.21.15.7:0/1426162230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560958138e70 0x560958139290 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.903+0000 7fa81d679740 1 -- 172.21.15.7:0/1426162230 >> 172.21.15.7:0/1426162230 conn(0x56095812b600 msgr2=0x56095812ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.903+0000 7fa81d679740 1 -- 172.21.15.7:0/1426162230 shutdown_connections 2024-06-10T01:24:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.903+0000 7fa81d679740 1 -- 172.21.15.7:0/1426162230 wait complete. 2024-06-10T01:24:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.903+0000 7fa81d679740 1 -- start start 2024-06-10T01:24:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.904+0000 7fa81d679740 1 Processor -- start 2024-06-10T01:24:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.904+0000 7fa81d679740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560958138e70 0x560958142a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.904+0000 7fa81d679740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560958139860 0x560958142fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.904+0000 7fa81d679740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56095813e3b0 0x560958149f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.904+0000 7fa81d679740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5609581412c0 con 0x560958139860 2024-06-10T01:24:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.904+0000 7fa81d679740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56095801cae0 con 0x560958138e70 2024-06-10T01:24:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.904+0000 7fa81d679740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560958141440 con 0x56095813e3b0 2024-06-10T01:24:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.904+0000 7fa8137fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560958139860 0x560958142fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.905+0000 7fa8137fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560958139860 0x560958142fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51606/0 (socket says 172.21.15.7:51606) 2024-06-10T01:24:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.905+0000 7fa8137fe640 1 -- 172.21.15.7:0/2222046681 learned_addr learned my addr 172.21.15.7:0/2222046681 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.905+0000 7fa81d679740 10 client.0 _ll_register_callbacks cb 0x5609580559b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:24:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.905+0000 7fa818f00640 1 --2- 172.21.15.7:0/2222046681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560958138e70 0x560958142a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.905+0000 7fa818f00640 1 --2- 172.21.15.7:0/2222046681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56095813e3b0 0x560958149f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.905+0000 7fa8137fe640 1 -- 172.21.15.7:0/2222046681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56095813e3b0 msgr2=0x560958149f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.905+0000 7fa8137fe640 1 --2- 172.21.15.7:0/2222046681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56095813e3b0 0x560958149f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.905+0000 7fa8137fe640 1 -- 172.21.15.7:0/2222046681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560958138e70 msgr2=0x560958142a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.905+0000 7fa8137fe640 1 --2- 172.21.15.7:0/2222046681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560958138e70 0x560958142a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.905+0000 7fa8137fe640 1 -- 172.21.15.7:0/2222046681 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560957da7210 con 0x560958139860 2024-06-10T01:24:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.905+0000 7fa8137fe640 1 --2- 172.21.15.7:0/2222046681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560958139860 0x560958142fc0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fa808012740 tx=0x7fa808012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36ceeb25ba5c075 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.905+0000 7fa7f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:24:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.906+0000 7fa7f37fe640 1 -- 172.21.15.7:0/2222046681 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808004020 con 0x560958139860 2024-06-10T01:24:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.906+0000 7fa81d679740 1 -- 172.21.15.7:0/2222046681 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa80c037040 con 0x560958139860 2024-06-10T01:24:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.906+0000 7fa7f37fe640 1 -- 172.21.15.7:0/2222046681 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fa8080041c0 con 0x560958139860 2024-06-10T01:24:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.906+0000 7fa7f37fe640 1 -- 172.21.15.7:0/2222046681 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fa808025050 con 0x560958139860 2024-06-10T01:24:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.906+0000 7fa81d679740 10 client.19881 fetch_fsmap learned FSMap version 50 2024-06-10T01:24:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.906+0000 7fa81d679740 1 -- 172.21.15.7:0/2222046681 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x560957e73620 con 0x560958139860 2024-06-10T01:24:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.906+0000 7fa7f37fe640 1 -- 172.21.15.7:0/2222046681 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa80c037040 con 0x560958139860 2024-06-10T01:24:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.906+0000 7fa7f37fe640 1 -- 172.21.15.7:0/2222046681 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7fa808003960 con 0x560958139860 2024-06-10T01:24:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.906+0000 7fa81d679740 10 client.19881 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:24:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.906+0000 7fa81d679740 10 client.19881 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:24:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.907+0000 7fa81d679740 20 client.19881 populate_metadata read hostname 'smithi007' 2024-06-10T01:24:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.907+0000 7fa81d679740 1 --2- 172.21.15.7:0/2222046681 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56095814ed40 0x5609581511e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.907+0000 7fa81d679740 4 client.19881 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:24:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.907+0000 7fa81d679740 1 -- 172.21.15.7:0/2222046681 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609580a6f10 con 0x56095814ed40 2024-06-10T01:24:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.907+0000 7fa818f00640 1 --2- 172.21.15.7:0/2222046681 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56095814ed40 0x5609581511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.907+0000 7fa7f37fe640 10 client.19881 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.907+0000 7fa818f00640 1 --2- 172.21.15.7:0/2222046681 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56095814ed40 0x5609581511e0 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3efdebeca5213d6c server_cookie=234285f53bc7bf49 in_seq=0 out_seq=0 2024-06-10T01:24:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.910+0000 7fa7f37fe640 1 -- 172.21.15.7:0/2222046681 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5609580a6f10 con 0x56095814ed40 2024-06-10T01:24:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.910+0000 7fa7f37fe640 10 client.19881 handle_command_reply: tid=0 2024-06-10T01:24:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.910+0000 7fa81d679740 1 client.19881 shutdown 2024-06-10T01:24:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.911+0000 7fa81d679740 1 -- 172.21.15.7:0/2222046681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560958139860 msgr2=0x560958142fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.911+0000 7fa81d679740 1 --2- 172.21.15.7:0/2222046681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560958139860 0x560958142fc0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fa808012740 tx=0x7fa808012c00 comp rx=0 tx=0).stop 2024-06-10T01:24:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.911+0000 7fa81d679740 1 -- 172.21.15.7:0/2222046681 shutdown_connections 2024-06-10T01:24:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.911+0000 7fa81d679740 1 --2- 172.21.15.7:0/2222046681 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56095814ed40 0x5609581511e0 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.911+0000 7fa81d679740 1 --2- 172.21.15.7:0/2222046681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56095813e3b0 0x560958149f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.911+0000 7fa7f37fe640 0 client.19881 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.911+0000 7fa81d679740 1 --2- 172.21.15.7:0/2222046681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560958139860 0x560958142fc0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.911+0000 7fa81d679740 1 --2- 172.21.15.7:0/2222046681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560958138e70 0x560958142a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.911+0000 7fa81d679740 1 -- 172.21.15.7:0/2222046681 >> 172.21.15.7:0/2222046681 conn(0x56095812b600 msgr2=0x560958138500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.911+0000 7fa81d679740 1 -- 172.21.15.7:0/2222046681 shutdown_connections 2024-06-10T01:24:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.911+0000 7fa81d679740 1 -- 172.21.15.7:0/2222046681 wait complete. 2024-06-10T01:24:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.912+0000 7fa81d679740 20 client.19881 trim_cache size 0 max 16384 2024-06-10T01:24:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.929+0000 7fa81d679740 1 Processor -- start 2024-06-10T01:24:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.929+0000 7fa81d679740 1 -- start start 2024-06-10T01:24:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.929+0000 7fa81d679740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560958138e70 0x5609581420a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.930+0000 7fa81d679740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560958139860 0x560958149f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.930+0000 7fa81d679740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56095813e3b0 0x56095813a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.930+0000 7fa81d679740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa8141aabc0 con 0x56095813e3b0 2024-06-10T01:24:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.930+0000 7fa81d679740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa8141a8490 con 0x560958138e70 2024-06-10T01:24:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.930+0000 7fa81d679740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa8141a62e0 con 0x560958139860 2024-06-10T01:24:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.930+0000 7fa818f00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56095813e3b0 0x56095813a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.930+0000 7fa818f00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560958138e70 0x5609581420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.930+0000 7fa8137fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560958139860 0x560958149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.930+0000 7fa818f00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560958138e70 0x5609581420a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:40662/0 (socket says 172.21.15.7:40662) 2024-06-10T01:24:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.930+0000 7fa818f00640 1 -- 172.21.15.7:0/1708918707 learned_addr learned my addr 172.21.15.7:0/1708918707 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.930+0000 7fa8137fe640 1 -- 172.21.15.7:0/1708918707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560958138e70 msgr2=0x5609581420a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.930+0000 7fa8137fe640 1 --2- 172.21.15.7:0/1708918707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560958138e70 0x5609581420a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.930+0000 7fa8137fe640 1 -- 172.21.15.7:0/1708918707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56095813e3b0 msgr2=0x56095813a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.930+0000 7fa8137fe640 1 --2- 172.21.15.7:0/1708918707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56095813e3b0 0x56095813a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.930+0000 7fa8137fe640 1 -- 172.21.15.7:0/1708918707 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56095812b430 con 0x560958139860 2024-06-10T01:24:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.930+0000 7fa8137fe640 1 --2- 172.21.15.7:0/1708918707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560958139860 0x560958149f50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa808001eb0 tx=0x7fa808000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88fae90afdca563b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.931+0000 7fa7f9ffb640 1 -- 172.21.15.7:0/1708918707 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808022040 con 0x560958139860 2024-06-10T01:24:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.931+0000 7fa7f9ffb640 1 -- 172.21.15.7:0/1708918707 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa808012e50 con 0x560958139860 2024-06-10T01:24:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.931+0000 7fa7f9ffb640 1 -- 172.21.15.7:0/1708918707 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80801c780 con 0x560958139860 2024-06-10T01:24:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.931+0000 7fa81d679740 1 -- 172.21.15.7:0/1708918707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560958139860 msgr2=0x560958149f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.931+0000 7fa81d679740 1 --2- 172.21.15.7:0/1708918707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560958139860 0x560958149f50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa808001eb0 tx=0x7fa808000bc0 comp rx=0 tx=0).stop 2024-06-10T01:24:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.931+0000 7fa81d679740 1 -- 172.21.15.7:0/1708918707 shutdown_connections 2024-06-10T01:24:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.931+0000 7fa81d679740 1 --2- 172.21.15.7:0/1708918707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56095813e3b0 0x56095813a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.931+0000 7fa81d679740 1 --2- 172.21.15.7:0/1708918707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560958139860 0x560958149f50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.931+0000 7fa81d679740 1 --2- 172.21.15.7:0/1708918707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560958138e70 0x5609581420a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.931+0000 7fa81d679740 1 -- 172.21.15.7:0/1708918707 >> 172.21.15.7:0/1708918707 conn(0x56095812b600 msgr2=0x56095812b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.932+0000 7fa81d679740 1 -- 172.21.15.7:0/1708918707 shutdown_connections 2024-06-10T01:24:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.932+0000 7fa81d679740 1 -- 172.21.15.7:0/1708918707 wait complete. 2024-06-10T01:24:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.932+0000 7fa81d679740 1 -- start start 2024-06-10T01:24:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.932+0000 7fa81d679740 1 Processor -- start 2024-06-10T01:24:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.932+0000 7fa81d679740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560958138e70 0x560958149b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.932+0000 7fa81d679740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560958139860 0x5609581466a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.932+0000 7fa81d679740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56095813e3b0 0x560958146be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.932+0000 7fa81d679740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa8141a6950 con 0x56095813e3b0 2024-06-10T01:24:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.932+0000 7fa81d679740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa8141a7dc0 con 0x560958139860 2024-06-10T01:24:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.932+0000 7fa81d679740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa8141a7020 con 0x560958138e70 2024-06-10T01:24:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.933+0000 7fa81d679740 10 client.0 _ll_register_callbacks cb 0x7fa808009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:24:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.933+0000 7fa8137fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560958139860 0x5609581466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.933+0000 7fa8137fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560958139860 0x5609581466a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:40670/0 (socket says 172.21.15.7:40670) 2024-06-10T01:24:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.933+0000 7fa8137fe640 1 -- 172.21.15.7:0/2020865054 learned_addr learned my addr 172.21.15.7:0/2020865054 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.933+0000 7fa818f00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56095813e3b0 0x560958146be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.933+0000 7fa818f00640 1 --2- 172.21.15.7:0/2020865054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560958138e70 0x560958149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.933+0000 7fa8137fe640 1 -- 172.21.15.7:0/2020865054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560958138e70 msgr2=0x560958149b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.933+0000 7fa8137fe640 1 --2- 172.21.15.7:0/2020865054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560958138e70 0x560958149b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.933+0000 7fa8137fe640 1 -- 172.21.15.7:0/2020865054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56095813e3b0 msgr2=0x560958146be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.933+0000 7fa8137fe640 1 --2- 172.21.15.7:0/2020865054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56095813e3b0 0x560958146be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.933+0000 7fa8137fe640 1 -- 172.21.15.7:0/2020865054 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560958147150 con 0x560958139860 2024-06-10T01:24:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.933+0000 7fa818f00640 1 --2- 172.21.15.7:0/2020865054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560958138e70 0x560958149b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:24:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.933+0000 7fa8137fe640 1 --2- 172.21.15.7:0/2020865054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560958139860 0x5609581466a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x56095812ef40 tx=0x7fa808003b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8d1a52cf42334e6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.933+0000 7fa7f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:24:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.933+0000 7fa7f37fe640 1 -- 172.21.15.7:0/2020865054 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808036060 con 0x560958139860 2024-06-10T01:24:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.933+0000 7fa7f37fe640 1 -- 172.21.15.7:0/2020865054 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fa808004180 con 0x560958139860 2024-06-10T01:24:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.933+0000 7fa81d679740 1 -- 172.21.15.7:0/2020865054 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa808012e50 con 0x560958139860 2024-06-10T01:24:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.934+0000 7fa7f37fe640 1 -- 172.21.15.7:0/2020865054 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fa8080122e0 con 0x560958139860 2024-06-10T01:24:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.934+0000 7fa81d679740 10 client.29090 fetch_fsmap learned FSMap version 50 2024-06-10T01:24:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.934+0000 7fa81d679740 1 -- 172.21.15.7:0/2020865054 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5609581b8ab0 con 0x560958139860 2024-06-10T01:24:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.934+0000 7fa7f37fe640 1 -- 172.21.15.7:0/2020865054 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa808012e50 con 0x560958139860 2024-06-10T01:24:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.934+0000 7fa7f37fe640 1 -- 172.21.15.7:0/2020865054 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7fa808029430 con 0x560958139860 2024-06-10T01:24:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.935+0000 7fa81d679740 10 client.29090 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:24:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.935+0000 7fa81d679740 10 client.29090 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:24:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.935+0000 7fa81d679740 20 client.29090 populate_metadata read hostname 'smithi007' 2024-06-10T01:24:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.935+0000 7fa81d679740 1 --2- 172.21.15.7:0/2020865054 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56095814ed40 0x5609581bb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.935+0000 7fa81d679740 4 client.29090 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:24:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.935+0000 7fa81d679740 1 -- 172.21.15.7:0/2020865054 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5609580a0c20 con 0x56095814ed40 2024-06-10T01:24:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.935+0000 7fa818f00640 1 --2- 172.21.15.7:0/2020865054 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56095814ed40 0x5609581bb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.935+0000 7fa7f37fe640 10 client.29090 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:02.935+0000 7fa818f00640 1 --2- 172.21.15.7:0/2020865054 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56095814ed40 0x5609581bb490 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=179463ef4008cb92 server_cookie=b2e352b20909c223 in_seq=0 out_seq=0 2024-06-10T01:24:03.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:03.182+0000 7fa7f37fe640 1 -- 172.21.15.7:0/2020865054 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5609580a0c20 con 0x56095814ed40 2024-06-10T01:24:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:03.183+0000 7fa7f37fe640 10 client.29090 handle_command_reply: tid=0 2024-06-10T01:24:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:03.183+0000 7fa81d679740 1 client.29090 shutdown 2024-06-10T01:24:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:03.183+0000 7fa81d679740 1 -- 172.21.15.7:0/2020865054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560958139860 msgr2=0x5609581466a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:03.183+0000 7fa81d679740 1 --2- 172.21.15.7:0/2020865054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560958139860 0x5609581466a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x56095812ef40 tx=0x7fa808003b40 comp rx=0 tx=0).stop 2024-06-10T01:24:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:03.183+0000 7fa81d679740 1 -- 172.21.15.7:0/2020865054 shutdown_connections 2024-06-10T01:24:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:03.183+0000 7fa81d679740 1 --2- 172.21.15.7:0/2020865054 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56095814ed40 0x5609581bb490 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:03.183+0000 7fa7f37fe640 0 client.29090 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:03.183+0000 7fa81d679740 1 --2- 172.21.15.7:0/2020865054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56095813e3b0 0x560958146be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:03.183+0000 7fa81d679740 1 --2- 172.21.15.7:0/2020865054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560958139860 0x5609581466a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:03.183+0000 7fa81d679740 1 --2- 172.21.15.7:0/2020865054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560958138e70 0x560958149b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:03.183+0000 7fa81d679740 1 -- 172.21.15.7:0/2020865054 >> 172.21.15.7:0/2020865054 conn(0x56095812b600 msgr2=0x560958142a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:03.184+0000 7fa81d679740 1 -- 172.21.15.7:0/2020865054 shutdown_connections 2024-06-10T01:24:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:03.184+0000 7fa81d679740 1 -- 172.21.15.7:0/2020865054 wait complete. 2024-06-10T01:24:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:03.184+0000 7fa81d679740 20 client.29090 trim_cache size 0 max 16384 2024-06-10T01:24:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:24:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:24:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:24:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:03.185+0000 7fa81a963640 1 -- 172.21.15.7:0/1396396578 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa7fc0a2c00 msgr2=0x7fa7fc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:03.185+0000 7fa81a963640 1 --2- 172.21.15.7:0/1396396578 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa7fc0a2c00 0x7fa7fc0a50c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa8040a4440 tx=0x7fa804097f60 comp rx=0 tx=0).stop 2024-06-10T01:24:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:03.185+0000 7fa81a963640 1 -- 172.21.15.7:0/1396396578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8140ffd10 msgr2=0x7fa814197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:03.185+0000 7fa81a963640 1 --2- 172.21.15.7:0/1396396578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8140ffd10 0x7fa814197d00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa80400e9c0 tx=0x7fa80400ee80 comp rx=0 tx=0).stop 2024-06-10T01:24:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:03.186+0000 7fa81a963640 1 -- 172.21.15.7:0/1396396578 shutdown_connections 2024-06-10T01:24:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:03.186+0000 7fa81a963640 1 --2- 172.21.15.7:0/1396396578 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa7fc0a2c00 0x7fa7fc0a50c0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:03.186+0000 7fa81a963640 1 --2- 172.21.15.7:0/1396396578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa814103740 0x7fa81419f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:03.186+0000 7fa81a963640 1 --2- 172.21.15.7:0/1396396578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa814102d80 0x7fa814198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:03.186+0000 7fa81a963640 1 --2- 172.21.15.7:0/1396396578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8140ffd10 0x7fa814197d00 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:03.186+0000 7fa81a963640 1 -- 172.21.15.7:0/1396396578 >> 172.21.15.7:0/1396396578 conn(0x7fa8140fb620 msgr2=0x7fa814106080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:03.186+0000 7fa81a963640 1 -- 172.21.15.7:0/1396396578 shutdown_connections 2024-06-10T01:24:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:03.186+0000 7fa81a963640 1 -- 172.21.15.7:0/1396396578 wait complete. 2024-06-10T01:24:03.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.200+0000 7f0808492700 1 -- 172.21.15.7:0/186086762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0800107570 msgr2=0x7f0800107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:03.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.200+0000 7f0808492700 1 --2- 172.21.15.7:0/186086762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0800107570 0x7f0800107950 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f07fc009990 tx=0x7f07fc009ca0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.201+0000 7f0808492700 1 -- 172.21.15.7:0/186086762 shutdown_connections 2024-06-10T01:24:03.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.201+0000 7f0808492700 1 --2- 172.21.15.7:0/186086762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0800101810 0x7f080010f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.201+0000 7f0808492700 1 --2- 172.21.15.7:0/186086762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0800107570 0x7f0800107950 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.201+0000 7f0808492700 1 --2- 172.21.15.7:0/186086762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0800100e50 0x7f08001012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.201+0000 7f0808492700 1 -- 172.21.15.7:0/186086762 >> 172.21.15.7:0/186086762 conn(0x7f08000766b0 msgr2=0x7f0800076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:03.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.201+0000 7f0808492700 1 -- 172.21.15.7:0/186086762 shutdown_connections 2024-06-10T01:24:03.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.201+0000 7f0808492700 1 -- 172.21.15.7:0/186086762 wait complete. 2024-06-10T01:24:03.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.201+0000 7f0808492700 1 Processor -- start 2024-06-10T01:24:03.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.201+0000 7f0808492700 1 -- start start 2024-06-10T01:24:03.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.202+0000 7f0808492700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0800100e50 0x7f080019e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:03.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.202+0000 7f0808492700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0800101810 0x7f080019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:03.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.202+0000 7f0808492700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0800107570 0x7f0800198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:03.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.202+0000 7f0808492700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0800112800 con 0x7f0800101810 2024-06-10T01:24:03.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.202+0000 7f0808492700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0800112680 con 0x7f0800107570 2024-06-10T01:24:03.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.202+0000 7f0808492700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0800112980 con 0x7f0800100e50 2024-06-10T01:24:03.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.202+0000 7f0805a2d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0800101810 0x7f080019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:03.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.202+0000 7f0805a2d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0800101810 0x7f080019ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51624/0 (socket says 172.21.15.7:51624) 2024-06-10T01:24:03.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.202+0000 7f080622e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0800100e50 0x7f080019e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:03.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.202+0000 7f0805a2d700 1 -- 172.21.15.7:0/957519247 learned_addr learned my addr 172.21.15.7:0/957519247 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:03.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.202+0000 7f0806a2f700 1 --2- 172.21.15.7:0/957519247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0800107570 0x7f0800198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:03.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.202+0000 7f0805a2d700 1 -- 172.21.15.7:0/957519247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0800100e50 msgr2=0x7f080019e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:03.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.202+0000 7f0805a2d700 1 --2- 172.21.15.7:0/957519247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0800100e50 0x7f080019e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.202+0000 7f0805a2d700 1 -- 172.21.15.7:0/957519247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0800107570 msgr2=0x7f0800198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:03.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.202+0000 7f0805a2d700 1 --2- 172.21.15.7:0/957519247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0800107570 0x7f0800198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.202+0000 7f0805a2d700 1 -- 172.21.15.7:0/957519247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0800199220 con 0x7f0800101810 2024-06-10T01:24:03.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.202+0000 7f0805a2d700 1 --2- 172.21.15.7:0/957519247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0800101810 0x7f080019ee40 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f07f000e9a0 tx=0x7f07f000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:03.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.203+0000 7f07f77fe700 1 -- 172.21.15.7:0/957519247 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f000ca10 con 0x7f0800101810 2024-06-10T01:24:03.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.203+0000 7f07f77fe700 1 -- 172.21.15.7:0/957519247 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07f000cbb0 con 0x7f0800101810 2024-06-10T01:24:03.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.203+0000 7f0808492700 1 -- 172.21.15.7:0/957519247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0800199510 con 0x7f0800101810 2024-06-10T01:24:03.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.203+0000 7f07f77fe700 1 -- 172.21.15.7:0/957519247 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f0018680 con 0x7f0800101810 2024-06-10T01:24:03.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.203+0000 7f0808492700 1 -- 172.21.15.7:0/957519247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0800101f60 con 0x7f0800101810 2024-06-10T01:24:03.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.206+0000 7f07f77fe700 1 -- 172.21.15.7:0/957519247 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f07f0010430 con 0x7f0800101810 2024-06-10T01:24:03.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.206+0000 7f07f77fe700 1 --2- 172.21.15.7:0/957519247 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f07ec0a2c20 0x7f07ec0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:03.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.207+0000 7f07f77fe700 1 -- 172.21.15.7:0/957519247 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f07f00cf600 con 0x7f0800101810 2024-06-10T01:24:03.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.207+0000 7f080622e700 1 --2- 172.21.15.7:0/957519247 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f07ec0a2c20 0x7f07ec0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:03.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.207+0000 7f0812d15b80 1 Processor -- start 2024-06-10T01:24:03.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.207+0000 7f080622e700 1 --2- 172.21.15.7:0/957519247 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f07ec0a2c20 0x7f07ec0a50e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0800199c00 tx=0x7f07fc01e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:03.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.208+0000 7f0812d15b80 1 -- start start 2024-06-10T01:24:03.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.208+0000 7f0812d15b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556861564640 0x556861564a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:03.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.208+0000 7f0812d15b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556861564f60 0x55686155e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:03.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.208+0000 7f0812d15b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55686155ef20 0x55686155f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:03.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.208+0000 7f0812d15b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5568614b0810 con 0x55686155ef20 2024-06-10T01:24:03.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.208+0000 7f0812d15b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556861458720 con 0x556861564640 2024-06-10T01:24:03.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.208+0000 7f0812d15b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5568614ac7d0 con 0x556861564f60 2024-06-10T01:24:03.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.208+0000 7f080622e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556861564640 0x556861564a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:03.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.208+0000 7f080622e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556861564640 0x556861564a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:40688/0 (socket says 172.21.15.7:40688) 2024-06-10T01:24:03.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.208+0000 7f0806a2f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55686155ef20 0x55686155f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:03.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.208+0000 7f080622e700 1 -- 172.21.15.7:0/428839934 learned_addr learned my addr 172.21.15.7:0/428839934 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:03.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.208+0000 7f0805a2d700 1 --2- 172.21.15.7:0/428839934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556861564f60 0x55686155e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:03.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.208+0000 7f0806a2f700 1 -- 172.21.15.7:0/428839934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556861564f60 msgr2=0x55686155e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:03.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.208+0000 7f0806a2f700 1 --2- 172.21.15.7:0/428839934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556861564f60 0x55686155e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.209+0000 7f0806a2f700 1 -- 172.21.15.7:0/428839934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556861564640 msgr2=0x556861564a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:03.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.209+0000 7f0806a2f700 1 --2- 172.21.15.7:0/428839934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556861564640 0x556861564a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.209+0000 7f0806a2f700 1 -- 172.21.15.7:0/428839934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55686144ef50 con 0x55686155ef20 2024-06-10T01:24:03.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.209+0000 7f080622e700 1 --2- 172.21.15.7:0/428839934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556861564640 0x556861564a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:24:03.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.209+0000 7f0806a2f700 1 --2- 172.21.15.7:0/428839934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55686155ef20 0x55686155f340 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f07f8011580 tx=0x7f07f8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db52f328d8886ef server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:03.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.209+0000 7f07f4d70700 1 -- 172.21.15.7:0/428839934 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f8011e80 con 0x55686155ef20 2024-06-10T01:24:03.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.209+0000 7f07f4d70700 1 -- 172.21.15.7:0/428839934 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07f8004bb0 con 0x55686155ef20 2024-06-10T01:24:03.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.209+0000 7f07f4d70700 1 -- 172.21.15.7:0/428839934 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f8004840 con 0x55686155ef20 2024-06-10T01:24:03.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.210+0000 7f0812d15b80 1 -- 172.21.15.7:0/428839934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55686155ef20 msgr2=0x55686155f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:03.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.210+0000 7f0812d15b80 1 --2- 172.21.15.7:0/428839934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55686155ef20 0x55686155f340 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f07f8011580 tx=0x7f07f8009920 comp rx=0 tx=0).stop 2024-06-10T01:24:03.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.210+0000 7f0812d15b80 1 -- 172.21.15.7:0/428839934 shutdown_connections 2024-06-10T01:24:03.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.210+0000 7f0812d15b80 1 --2- 172.21.15.7:0/428839934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55686155ef20 0x55686155f340 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.210+0000 7f0812d15b80 1 --2- 172.21.15.7:0/428839934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556861564640 0x556861564a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.210+0000 7f0812d15b80 1 --2- 172.21.15.7:0/428839934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556861564f60 0x55686155e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.210+0000 7f0812d15b80 1 -- 172.21.15.7:0/428839934 >> 172.21.15.7:0/428839934 conn(0x556861489cc0 msgr2=0x556861559850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:03.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.210+0000 7f0812d15b80 1 -- 172.21.15.7:0/428839934 shutdown_connections 2024-06-10T01:24:03.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.210+0000 7f0812d15b80 1 -- 172.21.15.7:0/428839934 wait complete. 2024-06-10T01:24:03.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.211+0000 7f0812d15b80 1 -- start start 2024-06-10T01:24:03.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.211+0000 7f0812d15b80 1 Processor -- start 2024-06-10T01:24:03.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.211+0000 7f0812d15b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55686155ef20 0x55686156ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:03.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.211+0000 7f0812d15b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556861564640 0x556861568000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:03.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.211+0000 7f0812d15b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556861564f60 0x556861568540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:03.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.212+0000 7f0812d15b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55686156b9d0 con 0x556861564f60 2024-06-10T01:24:03.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.212+0000 7f0812d15b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556861450bf0 con 0x556861564640 2024-06-10T01:24:03.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.212+0000 7f0812d15b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55686156bb50 con 0x55686155ef20 2024-06-10T01:24:03.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.212+0000 7f0806a2f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556861564f60 0x556861568540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:03.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.212+0000 7f0806a2f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556861564f60 0x556861568540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51650/0 (socket says 172.21.15.7:51650) 2024-06-10T01:24:03.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.212+0000 7f0806a2f700 1 -- 172.21.15.7:0/4174952086 learned_addr learned my addr 172.21.15.7:0/4174952086 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:03.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.212+0000 7f0805a2d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556861564640 0x556861568000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:03.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.212+0000 7f080622e700 1 --2- 172.21.15.7:0/4174952086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55686155ef20 0x55686156ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:03.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.212+0000 7f0812d15b80 10 client.0 _ll_register_callbacks cb 0x55686144a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:24:03.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.212+0000 7f0805a2d700 1 -- 172.21.15.7:0/4174952086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55686155ef20 msgr2=0x55686156ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:03.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.212+0000 7f0805a2d700 1 --2- 172.21.15.7:0/4174952086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55686155ef20 0x55686156ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.212+0000 7f0805a2d700 1 -- 172.21.15.7:0/4174952086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556861564f60 msgr2=0x556861568540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:03.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.212+0000 7f0805a2d700 1 --2- 172.21.15.7:0/4174952086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556861564f60 0x556861568540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.212+0000 7f0805a2d700 1 -- 172.21.15.7:0/4174952086 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568613ff8d0 con 0x556861564640 2024-06-10T01:24:03.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.213+0000 7f0805a2d700 1 --2- 172.21.15.7:0/4174952086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556861564640 0x556861568000 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f07f00ccdd0 tx=0x7f07f009c320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56cabddb55c39393 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:03.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.213+0000 7f07e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:24:03.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.213+0000 7f07e9ffb700 1 -- 172.21.15.7:0/4174952086 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f00a4750 con 0x556861564640 2024-06-10T01:24:03.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.213+0000 7f0812d15b80 1 -- 172.21.15.7:0/4174952086 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07f8004bb0 con 0x556861564640 2024-06-10T01:24:03.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.213+0000 7f07e9ffb700 1 -- 172.21.15.7:0/4174952086 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f07f0099de0 con 0x556861564640 2024-06-10T01:24:03.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.213+0000 7f07e9ffb700 1 -- 172.21.15.7:0/4174952086 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f07f009bd40 con 0x556861564640 2024-06-10T01:24:03.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.213+0000 7f0812d15b80 10 client.29102 fetch_fsmap learned FSMap version 50 2024-06-10T01:24:03.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.213+0000 7f0812d15b80 1 -- 172.21.15.7:0/4174952086 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x556861576390 con 0x556861564640 2024-06-10T01:24:03.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.213+0000 7f07e9ffb700 1 -- 172.21.15.7:0/4174952086 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07f8004bb0 con 0x556861564640 2024-06-10T01:24:03.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.214+0000 7f07e9ffb700 1 -- 172.21.15.7:0/4174952086 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7f07f00b4ce0 con 0x556861564640 2024-06-10T01:24:03.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.214+0000 7f0812d15b80 10 client.29102 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:24:03.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.214+0000 7f0812d15b80 10 client.29102 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:24:03.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.214+0000 7f0812d15b80 20 client.29102 populate_metadata read hostname 'smithi007' 2024-06-10T01:24:03.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.214+0000 7f0812d15b80 1 --2- 172.21.15.7:0/4174952086 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5568615789d0 0x55686157adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:03.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.214+0000 7f0812d15b80 4 client.29102 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:24:03.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.214+0000 7f0812d15b80 1 -- 172.21.15.7:0/4174952086 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568614bdd10 con 0x5568615789d0 2024-06-10T01:24:03.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.214+0000 7f080622e700 1 --2- 172.21.15.7:0/4174952086 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5568615789d0 0x55686157adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:03.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.215+0000 7f07e9ffb700 10 client.29102 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:24:03.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.215+0000 7f080622e700 1 --2- 172.21.15.7:0/4174952086 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5568615789d0 0x55686157adc0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e2592b8f0f147ea6 server_cookie=60c9588b8284a659 in_seq=0 out_seq=0 2024-06-10T01:24:03.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.216+0000 7f07e9ffb700 1 -- 172.21.15.7:0/4174952086 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5568614bdd10 con 0x5568615789d0 2024-06-10T01:24:03.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.216+0000 7f07e9ffb700 10 client.29102 handle_command_reply: tid=0 2024-06-10T01:24:03.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.216+0000 7f0812d15b80 1 client.29102 shutdown 2024-06-10T01:24:03.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.216+0000 7f0812d15b80 1 -- 172.21.15.7:0/4174952086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556861564640 msgr2=0x556861568000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:03.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.216+0000 7f0812d15b80 1 --2- 172.21.15.7:0/4174952086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556861564640 0x556861568000 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f07f00ccdd0 tx=0x7f07f009c320 comp rx=0 tx=0).stop 2024-06-10T01:24:03.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.216+0000 7f0812d15b80 1 -- 172.21.15.7:0/4174952086 shutdown_connections 2024-06-10T01:24:03.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.216+0000 7f0812d15b80 1 --2- 172.21.15.7:0/4174952086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556861564f60 0x556861568540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.216+0000 7f0812d15b80 1 --2- 172.21.15.7:0/4174952086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55686155ef20 0x55686156ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.216+0000 7f0812d15b80 1 --2- 172.21.15.7:0/4174952086 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5568615789d0 0x55686157adc0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.216+0000 7f07e9ffb700 0 client.29102 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:24:03.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.216+0000 7f0812d15b80 1 --2- 172.21.15.7:0/4174952086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556861564640 0x556861568000 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.216+0000 7f0812d15b80 1 -- 172.21.15.7:0/4174952086 >> 172.21.15.7:0/4174952086 conn(0x556861489cc0 msgr2=0x55686155d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:03.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.217+0000 7f0812d15b80 1 -- 172.21.15.7:0/4174952086 shutdown_connections 2024-06-10T01:24:03.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.217+0000 7f0812d15b80 1 -- 172.21.15.7:0/4174952086 wait complete. 2024-06-10T01:24:03.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.217+0000 7f0812d15b80 20 client.29102 trim_cache size 0 max 16384 2024-06-10T01:24:03.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.235+0000 7f0812d15b80 1 Processor -- start 2024-06-10T01:24:03.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.235+0000 7f0812d15b80 1 -- start start 2024-06-10T01:24:03.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.235+0000 7f0812d15b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55686155ef20 0x55686155e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:03.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.235+0000 7f0812d15b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556861564640 0x556861574ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:03.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.235+0000 7f0812d15b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556861564f60 0x556861565b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:03.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.235+0000 7f0812d15b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f08001aea50 con 0x55686155ef20 2024-06-10T01:24:03.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.235+0000 7f0812d15b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f08001ac2e0 con 0x556861564f60 2024-06-10T01:24:03.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.235+0000 7f0812d15b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f08001aa130 con 0x556861564640 2024-06-10T01:24:03.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.235+0000 7f0805a2d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556861564640 0x556861574ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:03.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.235+0000 7f080622e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55686155ef20 0x55686155e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:03.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.235+0000 7f0806a2f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556861564f60 0x556861565b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:03.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.235+0000 7f0805a2d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556861564640 0x556861574ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:35770/0 (socket says 172.21.15.7:35770) 2024-06-10T01:24:03.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.235+0000 7f0805a2d700 1 -- 172.21.15.7:0/3410400360 learned_addr learned my addr 172.21.15.7:0/3410400360 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:03.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.235+0000 7f080622e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55686155ef20 0x55686155e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51658/0 (socket says 172.21.15.7:51658) 2024-06-10T01:24:03.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.236+0000 7f080622e700 1 -- 172.21.15.7:0/3410400360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556861564640 msgr2=0x556861574ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:03.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.236+0000 7f080622e700 1 --2- 172.21.15.7:0/3410400360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556861564640 0x556861574ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.236+0000 7f080622e700 1 -- 172.21.15.7:0/3410400360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556861564f60 msgr2=0x556861565b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:03.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.236+0000 7f080622e700 1 --2- 172.21.15.7:0/3410400360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556861564f60 0x556861565b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.236+0000 7f080622e700 1 -- 172.21.15.7:0/3410400360 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556861566570 con 0x55686155ef20 2024-06-10T01:24:03.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.236+0000 7f0805a2d700 1 --2- 172.21.15.7:0/3410400360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556861564640 0x556861574ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:24:03.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.236+0000 7f0806a2f700 1 --2- 172.21.15.7:0/3410400360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556861564f60 0x556861565b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:24:03.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.236+0000 7f080622e700 1 --2- 172.21.15.7:0/3410400360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55686155ef20 0x55686155e9e0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f07fc0098e0 tx=0x7f07fc023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d12194baa6c675 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:03.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.236+0000 7f07ebfff700 1 -- 172.21.15.7:0/3410400360 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc035900 con 0x55686155ef20 2024-06-10T01:24:03.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.236+0000 7f07ebfff700 1 -- 172.21.15.7:0/3410400360 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07fc003680 con 0x55686155ef20 2024-06-10T01:24:03.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.236+0000 7f07ebfff700 1 -- 172.21.15.7:0/3410400360 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc03d630 con 0x55686155ef20 2024-06-10T01:24:03.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.237+0000 7f0812d15b80 1 -- 172.21.15.7:0/3410400360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55686155ef20 msgr2=0x55686155e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:03.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.237+0000 7f0812d15b80 1 --2- 172.21.15.7:0/3410400360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55686155ef20 0x55686155e9e0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f07fc0098e0 tx=0x7f07fc023000 comp rx=0 tx=0).stop 2024-06-10T01:24:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.237+0000 7f0812d15b80 1 -- 172.21.15.7:0/3410400360 shutdown_connections 2024-06-10T01:24:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.237+0000 7f0812d15b80 1 --2- 172.21.15.7:0/3410400360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556861564f60 0x556861565b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.237+0000 7f0812d15b80 1 --2- 172.21.15.7:0/3410400360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55686155ef20 0x55686155e9e0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.237+0000 7f0812d15b80 1 --2- 172.21.15.7:0/3410400360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556861564640 0x556861574ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.237+0000 7f0812d15b80 1 -- 172.21.15.7:0/3410400360 >> 172.21.15.7:0/3410400360 conn(0x556861489cc0 msgr2=0x556861565400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.237+0000 7f0812d15b80 1 -- 172.21.15.7:0/3410400360 shutdown_connections 2024-06-10T01:24:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.237+0000 7f0812d15b80 1 -- 172.21.15.7:0/3410400360 wait complete. 2024-06-10T01:24:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.238+0000 7f0812d15b80 1 -- start start 2024-06-10T01:24:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.238+0000 7f0812d15b80 1 Processor -- start 2024-06-10T01:24:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.238+0000 7f0812d15b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55686155ef20 0x556861580190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.238+0000 7f080622e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55686155ef20 0x556861580190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:03.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.238+0000 7f080622e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55686155ef20 0x556861580190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51668/0 (socket says 172.21.15.7:51668) 2024-06-10T01:24:03.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.238+0000 7f0812d15b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556861564640 0x5568615806d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:03.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.238+0000 7f080622e700 1 -- 172.21.15.7:0/3789435568 learned_addr learned my addr 172.21.15.7:0/3789435568 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:03.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.238+0000 7f0812d15b80 1 --2- 172.21.15.7:0/3789435568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556861564f60 0x556861580c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:03.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.238+0000 7f0805a2d700 1 --2- 172.21.15.7:0/3789435568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556861564640 0x5568615806d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:03.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.238+0000 7f0806a2f700 1 --2- 172.21.15.7:0/3789435568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556861564f60 0x556861580c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:03.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.238+0000 7f0812d15b80 1 -- 172.21.15.7:0/3789435568 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f08001aa7a0 con 0x55686155ef20 2024-06-10T01:24:03.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.239+0000 7f0812d15b80 1 -- 172.21.15.7:0/3789435568 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f08001abc10 con 0x556861564640 2024-06-10T01:24:03.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.239+0000 7f0812d15b80 1 -- 172.21.15.7:0/3789435568 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f08001aae70 con 0x556861564f60 2024-06-10T01:24:03.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.239+0000 7f0812d15b80 10 client.0 _ll_register_callbacks cb 0x55686144a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:24:03.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.239+0000 7f080622e700 1 -- 172.21.15.7:0/3789435568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556861564f60 msgr2=0x556861580c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:03.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.239+0000 7f080622e700 1 --2- 172.21.15.7:0/3789435568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556861564f60 0x556861580c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.239+0000 7f080622e700 1 -- 172.21.15.7:0/3789435568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556861564640 msgr2=0x5568615806d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:03.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.239+0000 7f080622e700 1 --2- 172.21.15.7:0/3789435568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556861564640 0x5568615806d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.239+0000 7f080622e700 1 -- 172.21.15.7:0/3789435568 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568615e2640 con 0x55686155ef20 2024-06-10T01:24:03.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.239+0000 7f0805a2d700 1 --2- 172.21.15.7:0/3789435568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556861564640 0x5568615806d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:24:03.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.239+0000 7f0806a2f700 1 --2- 172.21.15.7:0/3789435568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556861564f60 0x556861580c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:24:03.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.239+0000 7f080622e700 1 --2- 172.21.15.7:0/3789435568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55686155ef20 0x556861580190 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f07fc009ce0 tx=0x7f07fc0093b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6851f1d71672a066 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:03.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.239+0000 7f07e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:24:03.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.240+0000 7f07e9ffb700 1 -- 172.21.15.7:0/3789435568 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc043070 con 0x55686155ef20 2024-06-10T01:24:03.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.240+0000 7f07e9ffb700 1 -- 172.21.15.7:0/3789435568 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f07fc035900 con 0x55686155ef20 2024-06-10T01:24:03.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.240+0000 7f0812d15b80 1 -- 172.21.15.7:0/3789435568 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07fc003680 con 0x55686155ef20 2024-06-10T01:24:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.240+0000 7f07e9ffb700 1 -- 172.21.15.7:0/3789435568 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f07fc052050 con 0x55686155ef20 2024-06-10T01:24:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.240+0000 7f0812d15b80 10 client.19905 fetch_fsmap learned FSMap version 50 2024-06-10T01:24:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.240+0000 7f0812d15b80 1 -- 172.21.15.7:0/3789435568 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5568615e3840 con 0x55686155ef20 2024-06-10T01:24:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.240+0000 7f07e9ffb700 1 -- 172.21.15.7:0/3789435568 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07fc003d10 con 0x55686155ef20 2024-06-10T01:24:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.241+0000 7f07e9ffb700 1 -- 172.21.15.7:0/3789435568 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7f07fc03d640 con 0x55686155ef20 2024-06-10T01:24:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.241+0000 7f0812d15b80 10 client.19905 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:24:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.241+0000 7f0812d15b80 10 client.19905 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:24:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.241+0000 7f0812d15b80 20 client.19905 populate_metadata read hostname 'smithi007' 2024-06-10T01:24:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.241+0000 7f0812d15b80 1 --2- 172.21.15.7:0/3789435568 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5568615e3e80 0x5568615e62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.241+0000 7f0812d15b80 4 client.19905 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:24:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.241+0000 7f0812d15b80 1 -- 172.21.15.7:0/3789435568 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f07fc003680 con 0x5568615e3e80 2024-06-10T01:24:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.241+0000 7f0805a2d700 1 --2- 172.21.15.7:0/3789435568 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5568615e3e80 0x5568615e62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.241+0000 7f07e9ffb700 10 client.19905 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:24:03.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.241+0000 7f0805a2d700 1 --2- 172.21.15.7:0/3789435568 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5568615e3e80 0x5568615e62a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c0ee2e7d26a6bd85 server_cookie=e8cf5d50e8b52d5b in_seq=0 out_seq=0 2024-06-10T01:24:03.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.244+0000 7f07e9ffb700 1 -- 172.21.15.7:0/3789435568 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f07fc003680 con 0x5568615e3e80 2024-06-10T01:24:03.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.244+0000 7f07e9ffb700 10 client.19905 handle_command_reply: tid=0 2024-06-10T01:24:03.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.244+0000 7f0812d15b80 1 client.19905 shutdown 2024-06-10T01:24:03.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.244+0000 7f0812d15b80 1 -- 172.21.15.7:0/3789435568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55686155ef20 msgr2=0x556861580190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:03.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.244+0000 7f0812d15b80 1 --2- 172.21.15.7:0/3789435568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55686155ef20 0x556861580190 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f07fc009ce0 tx=0x7f07fc0093b0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.244+0000 7f0812d15b80 1 -- 172.21.15.7:0/3789435568 shutdown_connections 2024-06-10T01:24:03.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.244+0000 7f0812d15b80 1 --2- 172.21.15.7:0/3789435568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556861564f60 0x556861580c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.244+0000 7f0812d15b80 1 --2- 172.21.15.7:0/3789435568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55686155ef20 0x556861580190 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.244+0000 7f0812d15b80 1 --2- 172.21.15.7:0/3789435568 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5568615e3e80 0x5568615e62a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.244+0000 7f0812d15b80 1 --2- 172.21.15.7:0/3789435568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556861564640 0x5568615806d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.244+0000 7f0812d15b80 1 -- 172.21.15.7:0/3789435568 >> 172.21.15.7:0/3789435568 conn(0x556861489cc0 msgr2=0x556861552e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:03.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.244+0000 7f07e9ffb700 0 client.19905 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:24:03.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.244+0000 7f0812d15b80 1 -- 172.21.15.7:0/3789435568 shutdown_connections 2024-06-10T01:24:03.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.244+0000 7f0812d15b80 1 -- 172.21.15.7:0/3789435568 wait complete. 2024-06-10T01:24:03.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.244+0000 7f0812d15b80 20 client.19905 trim_cache size 0 max 16384 2024-06-10T01:24:03.242 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:24:03.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.245+0000 7f0808492700 1 -- 172.21.15.7:0/957519247 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f07ec0a2c20 msgr2=0x7f07ec0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:03.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.245+0000 7f0808492700 1 --2- 172.21.15.7:0/957519247 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f07ec0a2c20 0x7f07ec0a50e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0800199c00 tx=0x7f07fc01e040 comp rx=0 tx=0).stop 2024-06-10T01:24:03.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.245+0000 7f0808492700 1 -- 172.21.15.7:0/957519247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0800101810 msgr2=0x7f080019ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:03.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.245+0000 7f0808492700 1 --2- 172.21.15.7:0/957519247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0800101810 0x7f080019ee40 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f07f000e9a0 tx=0x7f07f000ecb0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.245+0000 7f0808492700 1 -- 172.21.15.7:0/957519247 shutdown_connections 2024-06-10T01:24:03.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.245+0000 7f0808492700 1 --2- 172.21.15.7:0/957519247 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f07ec0a2c20 0x7f07ec0a50e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.245+0000 7f0808492700 1 --2- 172.21.15.7:0/957519247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0800107570 0x7f0800198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.245+0000 7f0808492700 1 --2- 172.21.15.7:0/957519247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0800100e50 0x7f080019e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.245+0000 7f0808492700 1 --2- 172.21.15.7:0/957519247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0800101810 0x7f080019ee40 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:03.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.245+0000 7f0808492700 1 -- 172.21.15.7:0/957519247 >> 172.21.15.7:0/957519247 conn(0x7f08000766b0 msgr2=0x7f08000ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:03.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.245+0000 7f0808492700 1 -- 172.21.15.7:0/957519247 shutdown_connections 2024-06-10T01:24:03.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:03.245+0000 7f0808492700 1 -- 172.21.15.7:0/957519247 wait complete. 2024-06-10T01:24:03.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:03 smithi175 ceph-mon[36636]: pgmap v974: 129 pgs: 129 active+clean; 550 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 29 KiB/s wr, 29 op/s 2024-06-10T01:24:03.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:03 smithi007 ceph-mon[32398]: pgmap v974: 129 pgs: 129 active+clean; 550 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 29 KiB/s wr, 29 op/s 2024-06-10T01:24:03.534 INFO:tasks.workunit.client.0.smithi007.stdout: 10 99525 19.83 MB/sec execute 160 sec latency 196.531 ms 2024-06-10T01:24:03.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:03 smithi078 ceph-mon[36683]: pgmap v974: 129 pgs: 129 active+clean; 550 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 29 KiB/s wr, 29 op/s 2024-06-10T01:24:04.051 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:24:04.496 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:24:04.534 INFO:tasks.workunit.client.0.smithi007.stdout: 10 100220 19.96 MB/sec execute 161 sec latency 41.152 ms 2024-06-10T01:24:05.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:04 smithi175 ceph-mon[36636]: pgmap v975: 129 pgs: 129 active+clean; 555 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 448 KiB/s wr, 87 op/s 2024-06-10T01:24:05.204 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:04 smithi007 ceph-mon[32398]: pgmap v975: 129 pgs: 129 active+clean; 555 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 448 KiB/s wr, 87 op/s 2024-06-10T01:24:05.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:04 smithi078 ceph-mon[36683]: pgmap v975: 129 pgs: 129 active+clean; 555 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 448 KiB/s wr, 87 op/s 2024-06-10T01:24:05.534 INFO:tasks.workunit.client.0.smithi007.stdout: 10 100989 20.04 MB/sec execute 162 sec latency 94.593 ms 2024-06-10T01:24:06.535 INFO:tasks.workunit.client.0.smithi007.stdout: 10 101635 20.13 MB/sec execute 163 sec latency 109.296 ms 2024-06-10T01:24:06.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.991+0000 7fd9c292d700 1 -- 172.21.15.7:0/1889255361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9bc108500 msgr2=0x7fd9bc1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:06.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.991+0000 7fd9c292d700 1 --2- 172.21.15.7:0/1889255361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9bc108500 0x7fd9bc1088e0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac009990 tx=0x7fd9ac009ca0 comp rx=0 tx=0).stop 2024-06-10T01:24:06.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.991+0000 7fd9c292d700 1 -- 172.21.15.7:0/1889255361 shutdown_connections 2024-06-10T01:24:06.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.991+0000 7fd9c292d700 1 --2- 172.21.15.7:0/1889255361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9bc06ab30 0x7fd9bc10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:06.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.991+0000 7fd9c292d700 1 --2- 172.21.15.7:0/1889255361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9bc108500 0x7fd9bc1088e0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:06.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.991+0000 7fd9c292d700 1 --2- 172.21.15.7:0/1889255361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9bc06a170 0x7fd9bc06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:06.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.991+0000 7fd9c292d700 1 -- 172.21.15.7:0/1889255361 >> 172.21.15.7:0/1889255361 conn(0x7fd9bc078340 msgr2=0x7fd9bc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:06.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.991+0000 7fd9c292d700 1 -- 172.21.15.7:0/1889255361 shutdown_connections 2024-06-10T01:24:06.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.991+0000 7fd9c292d700 1 -- 172.21.15.7:0/1889255361 wait complete. 2024-06-10T01:24:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.992+0000 7fd9c292d700 1 Processor -- start 2024-06-10T01:24:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.992+0000 7fd9c292d700 1 -- start start 2024-06-10T01:24:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.992+0000 7fd9c292d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9bc06a170 0x7fd9bc19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.992+0000 7fd9c292d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9bc06ab30 0x7fd9bc19ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.992+0000 7fd9c292d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9bc108500 0x7fd9bc198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.993+0000 7fd9c292d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd9bc1127d0 con 0x7fd9bc06ab30 2024-06-10T01:24:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.993+0000 7fd9c292d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd9bc112650 con 0x7fd9bc06a170 2024-06-10T01:24:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.993+0000 7fd9c292d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd9bc112950 con 0x7fd9bc108500 2024-06-10T01:24:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.993+0000 7fd9bbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9bc06a170 0x7fd9bc19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.993+0000 7fd9b3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9bc06ab30 0x7fd9bc19ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.993+0000 7fd9bbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9bc06a170 0x7fd9bc19e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:40740/0 (socket says 172.21.15.7:40740) 2024-06-10T01:24:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.993+0000 7fd9bbfff700 1 -- 172.21.15.7:0/1398431324 learned_addr learned my addr 172.21.15.7:0/1398431324 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.993+0000 7fd9c0eca700 1 --2- 172.21.15.7:0/1398431324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9bc108500 0x7fd9bc198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.993+0000 7fd9bbfff700 1 -- 172.21.15.7:0/1398431324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9bc108500 msgr2=0x7fd9bc198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.993+0000 7fd9bbfff700 1 --2- 172.21.15.7:0/1398431324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9bc108500 0x7fd9bc198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.993+0000 7fd9bbfff700 1 -- 172.21.15.7:0/1398431324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9bc06ab30 msgr2=0x7fd9bc19ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.993+0000 7fd9bbfff700 1 --2- 172.21.15.7:0/1398431324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9bc06ab30 0x7fd9bc19ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.993+0000 7fd9bbfff700 1 -- 172.21.15.7:0/1398431324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9bc199250 con 0x7fd9bc06a170 2024-06-10T01:24:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.993+0000 7fd9bbfff700 1 --2- 172.21.15.7:0/1398431324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9bc06a170 0x7fd9bc19e930 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac000c00 tx=0x7fd9ac00bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.994+0000 7fd9b9ffb700 1 -- 172.21.15.7:0/1398431324 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac027070 con 0x7fd9bc06a170 2024-06-10T01:24:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.994+0000 7fd9c292d700 1 -- 172.21.15.7:0/1398431324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9bc1994e0 con 0x7fd9bc06a170 2024-06-10T01:24:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.994+0000 7fd9b9ffb700 1 -- 172.21.15.7:0/1398431324 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9ac0056d0 con 0x7fd9bc06a170 2024-06-10T01:24:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.994+0000 7fd9b9ffb700 1 -- 172.21.15.7:0/1398431324 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac022e10 con 0x7fd9bc06a170 2024-06-10T01:24:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.994+0000 7fd9c292d700 1 -- 172.21.15.7:0/1398431324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9bc101390 con 0x7fd9bc06a170 2024-06-10T01:24:06.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.997+0000 7fd9b9ffb700 1 -- 172.21.15.7:0/1398431324 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fd9ac02a070 con 0x7fd9bc06a170 2024-06-10T01:24:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.998+0000 7fd9b9ffb700 1 --2- 172.21.15.7:0/1398431324 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd9a80a2ca0 0x7fd9a80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.999+0000 7fd9b9ffb700 1 -- 172.21.15.7:0/1398431324 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fd9ac0cff80 con 0x7fd9bc06a170 2024-06-10T01:24:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.999+0000 7fd9b3fff700 1 --2- 172.21.15.7:0/1398431324 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd9a80a2ca0 0x7fd9a80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.999+0000 7fd9cd1b0b80 1 Processor -- start 2024-06-10T01:24:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:06.999+0000 7fd9cd1b0b80 1 -- start start 2024-06-10T01:24:06.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.000+0000 7fd9b3fff700 1 --2- 172.21.15.7:0/1398431324 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd9a80a2ca0 0x7fd9a80a5160 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0789d0 tx=0x7fd9a4012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:06.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.000+0000 7fd9cd1b0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fc1af3640 0x564fc1af3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:06.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.000+0000 7fd9cd1b0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc1af3f60 0x564fc1aed9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:06.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.000+0000 7fd9cd1b0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fc1aedf20 0x564fc1aee340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:06.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.000+0000 7fd9bbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fc1af3640 0x564fc1af3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:06.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.000+0000 7fd9bbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fc1af3640 0x564fc1af3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:40756/0 (socket says 172.21.15.7:40756) 2024-06-10T01:24:06.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.000+0000 7fd9cd1b0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564fc1a3f810 con 0x564fc1aedf20 2024-06-10T01:24:06.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.000+0000 7fd9cd1b0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564fc19e7720 con 0x564fc1af3640 2024-06-10T01:24:06.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.000+0000 7fd9cd1b0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564fc1a3b7d0 con 0x564fc1af3f60 2024-06-10T01:24:06.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.000+0000 7fd9bbfff700 1 -- 172.21.15.7:0/1525076974 learned_addr learned my addr 172.21.15.7:0/1525076974 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:06.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.001+0000 7fd9b3fff700 1 --2- 172.21.15.7:0/1525076974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc1af3f60 0x564fc1aed9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:06.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.001+0000 7fd9c0eca700 1 --2- 172.21.15.7:0/1525076974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fc1aedf20 0x564fc1aee340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:06.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.001+0000 7fd9bbfff700 1 -- 172.21.15.7:0/1525076974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc1af3f60 msgr2=0x564fc1aed9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:06.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.001+0000 7fd9bbfff700 1 --2- 172.21.15.7:0/1525076974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc1af3f60 0x564fc1aed9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:06.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.001+0000 7fd9bbfff700 1 -- 172.21.15.7:0/1525076974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fc1aedf20 msgr2=0x564fc1aee340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:06.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.001+0000 7fd9bbfff700 1 --2- 172.21.15.7:0/1525076974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fc1aedf20 0x564fc1aee340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:06.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.001+0000 7fd9bbfff700 1 -- 172.21.15.7:0/1525076974 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fc19ddf50 con 0x564fc1af3640 2024-06-10T01:24:06.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.001+0000 7fd9b3fff700 1 --2- 172.21.15.7:0/1525076974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc1af3f60 0x564fc1aed9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:24:06.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.001+0000 7fd9c0eca700 1 --2- 172.21.15.7:0/1525076974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fc1aedf20 0x564fc1aee340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:24:06.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.001+0000 7fd9bbfff700 1 --2- 172.21.15.7:0/1525076974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fc1af3640 0x564fc1af3a20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac0a4ba0 tx=0x7fd9ac09c2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36d8a9773959de9a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:06.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.001+0000 7fd9b27fc700 1 -- 172.21.15.7:0/1525076974 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac09cbf0 con 0x564fc1af3640 2024-06-10T01:24:06.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.002+0000 7fd9b27fc700 1 -- 172.21.15.7:0/1525076974 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9ac09c4f0 con 0x564fc1af3640 2024-06-10T01:24:06.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.002+0000 7fd9b27fc700 1 -- 172.21.15.7:0/1525076974 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac0226d0 con 0x564fc1af3640 2024-06-10T01:24:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.002+0000 7fd9cd1b0b80 1 -- 172.21.15.7:0/1525076974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fc1af3640 msgr2=0x564fc1af3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.002+0000 7fd9cd1b0b80 1 --2- 172.21.15.7:0/1525076974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fc1af3640 0x564fc1af3a20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac0a4ba0 tx=0x7fd9ac09c2c0 comp rx=0 tx=0).stop 2024-06-10T01:24:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.002+0000 7fd9cd1b0b80 1 -- 172.21.15.7:0/1525076974 shutdown_connections 2024-06-10T01:24:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.003+0000 7fd9cd1b0b80 1 --2- 172.21.15.7:0/1525076974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fc1aedf20 0x564fc1aee340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.003+0000 7fd9cd1b0b80 1 --2- 172.21.15.7:0/1525076974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fc1af3640 0x564fc1af3a20 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.003+0000 7fd9cd1b0b80 1 --2- 172.21.15.7:0/1525076974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc1af3f60 0x564fc1aed9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.003+0000 7fd9cd1b0b80 1 -- 172.21.15.7:0/1525076974 >> 172.21.15.7:0/1525076974 conn(0x564fc1a18cc0 msgr2=0x564fc1ae8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.003+0000 7fd9cd1b0b80 1 -- 172.21.15.7:0/1525076974 shutdown_connections 2024-06-10T01:24:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.003+0000 7fd9cd1b0b80 1 -- 172.21.15.7:0/1525076974 wait complete. 2024-06-10T01:24:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.003+0000 7fd9cd1b0b80 1 -- start start 2024-06-10T01:24:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.003+0000 7fd9cd1b0b80 1 Processor -- start 2024-06-10T01:24:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.004+0000 7fd9cd1b0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fc1aedf20 0x564fc1af9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.004+0000 7fd9cd1b0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc1af3640 0x564fc1af7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.004+0000 7fd9cd1b0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fc1af3f60 0x564fc1af7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.004+0000 7fd9cd1b0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564fc1afa6f0 con 0x564fc1af3f60 2024-06-10T01:24:07.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.004+0000 7fd9cd1b0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564fc19dfbf0 con 0x564fc1aedf20 2024-06-10T01:24:07.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.004+0000 7fd9cd1b0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564fc1afa870 con 0x564fc1af3640 2024-06-10T01:24:07.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.004+0000 7fd9c0eca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fc1af3f60 0x564fc1af7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:07.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.004+0000 7fd9c0eca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fc1af3f60 0x564fc1af7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51694/0 (socket says 172.21.15.7:51694) 2024-06-10T01:24:07.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.004+0000 7fd9c0eca700 1 -- 172.21.15.7:0/898616715 learned_addr learned my addr 172.21.15.7:0/898616715 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:07.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.004+0000 7fd9bbfff700 1 --2- 172.21.15.7:0/898616715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fc1aedf20 0x564fc1af9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:07.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.004+0000 7fd9b3fff700 1 --2- 172.21.15.7:0/898616715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc1af3640 0x564fc1af7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:07.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.005+0000 7fd9b3fff700 1 -- 172.21.15.7:0/898616715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fc1aedf20 msgr2=0x564fc1af9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:07.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.005+0000 7fd9b3fff700 1 --2- 172.21.15.7:0/898616715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fc1aedf20 0x564fc1af9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:07.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.005+0000 7fd9b3fff700 1 -- 172.21.15.7:0/898616715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fc1af3f60 msgr2=0x564fc1af7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:07.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.005+0000 7fd9b3fff700 1 --2- 172.21.15.7:0/898616715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fc1af3f60 0x564fc1af7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:07.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.005+0000 7fd9b3fff700 1 -- 172.21.15.7:0/898616715 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fc198e8d0 con 0x564fc1af3640 2024-06-10T01:24:07.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.005+0000 7fd9cd1b0b80 10 client.0 _ll_register_callbacks cb 0x564fc19d9920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:24:07.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.005+0000 7fd9cd1b0b80 1 -- 172.21.15.7:0/898616715 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9ac09c4f0 con 0x564fc1af3640 2024-06-10T01:24:07.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.006+0000 7fd9b3fff700 1 --2- 172.21.15.7:0/898616715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc1af3640 0x564fc1af7000 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009c30 tx=0x7fd9a4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ade300f56d42cf8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:07.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.006+0000 7fd99b7fe700 10 client.29083 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:24:07.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.006+0000 7fd99b7fe700 1 -- 172.21.15.7:0/898616715 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4022050 con 0x564fc1af3640 2024-06-10T01:24:07.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.006+0000 7fd99b7fe700 1 -- 172.21.15.7:0/898616715 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fd9a4010270 con 0x564fc1af3640 2024-06-10T01:24:07.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.006+0000 7fd99b7fe700 1 -- 172.21.15.7:0/898616715 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fd9a40104b0 con 0x564fc1af3640 2024-06-10T01:24:07.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.007+0000 7fd9cd1b0b80 10 client.29083 fetch_fsmap learned FSMap version 50 2024-06-10T01:24:07.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.007+0000 7fd9cd1b0b80 1 -- 172.21.15.7:0/898616715 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({fsmap=50}) -- 0x564fc1b15e80 con 0x564fc1af3640 2024-06-10T01:24:07.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.007+0000 7fd99b7fe700 1 -- 172.21.15.7:0/898616715 <== mon.1 v2:172.21.15.175:3300/0 4 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7fd9a4026020 con 0x564fc1af3640 2024-06-10T01:24:07.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.008+0000 7fd9cd1b0b80 10 client.29083 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:24:07.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.008+0000 7fd9cd1b0b80 10 client.29083 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:24:07.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.008+0000 7fd9cd1b0b80 20 client.29083 populate_metadata read hostname 'smithi007' 2024-06-10T01:24:07.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.008+0000 7fd9cd1b0b80 1 --2- 172.21.15.7:0/898616715 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564fc1b183b0 0x564fc1b1a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:07.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.008+0000 7fd9cd1b0b80 4 client.29083 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:24:07.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.008+0000 7fd9cd1b0b80 1 -- 172.21.15.7:0/898616715 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fc1a4cd10 con 0x564fc1b183b0 2024-06-10T01:24:07.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.008+0000 7fd9bbfff700 1 --2- 172.21.15.7:0/898616715 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564fc1b183b0 0x564fc1b1a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:07.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.009+0000 7fd9bbfff700 1 --2- 172.21.15.7:0/898616715 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564fc1b183b0 0x564fc1b1a7a0 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6a581c81d45c20 server_cookie=58894dffda9f014 in_seq=0 out_seq=0 2024-06-10T01:24:07.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.009+0000 7fd99b7fe700 10 client.29083 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.013+0000 7fd99b7fe700 1 -- 172.21.15.7:0/898616715 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564fc1a4cd10 con 0x564fc1b183b0 2024-06-10T01:24:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.013+0000 7fd99b7fe700 10 client.29083 handle_command_reply: tid=0 2024-06-10T01:24:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.013+0000 7fd9cd1b0b80 1 client.29083 shutdown 2024-06-10T01:24:07.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.014+0000 7fd9cd1b0b80 1 -- 172.21.15.7:0/898616715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc1af3640 msgr2=0x564fc1af7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:07.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.014+0000 7fd9cd1b0b80 1 --2- 172.21.15.7:0/898616715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc1af3640 0x564fc1af7000 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009c30 tx=0x7fd9a4004750 comp rx=0 tx=0).stop 2024-06-10T01:24:07.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.014+0000 7fd9cd1b0b80 1 -- 172.21.15.7:0/898616715 shutdown_connections 2024-06-10T01:24:07.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.014+0000 7fd9cd1b0b80 1 --2- 172.21.15.7:0/898616715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fc1af3f60 0x564fc1af7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:07.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.014+0000 7fd9cd1b0b80 1 --2- 172.21.15.7:0/898616715 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564fc1b183b0 0x564fc1b1a7a0 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:07.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.014+0000 7fd9cd1b0b80 1 --2- 172.21.15.7:0/898616715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fc1aedf20 0x564fc1af9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:07.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.014+0000 7fd9cd1b0b80 1 --2- 172.21.15.7:0/898616715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc1af3640 0x564fc1af7000 secure :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009c30 tx=0x7fd9a4004750 comp rx=0 tx=0).stop 2024-06-10T01:24:07.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.014+0000 7fd9cd1b0b80 1 -- 172.21.15.7:0/898616715 >> 172.21.15.7:0/898616715 conn(0x564fc1a18cc0 msgr2=0x564fc1aec9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:07.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.014+0000 7fd99b7fe700 0 client.29083 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:07.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.014+0000 7fd9cd1b0b80 1 -- 172.21.15.7:0/898616715 shutdown_connections 2024-06-10T01:24:07.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.015+0000 7fd9cd1b0b80 1 -- 172.21.15.7:0/898616715 wait complete. 2024-06-10T01:24:07.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.015+0000 7fd9cd1b0b80 20 client.29083 trim_cache size 0 max 16384 2024-06-10T01:24:07.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.035+0000 7fd9cd1b0b80 1 Processor -- start 2024-06-10T01:24:07.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.035+0000 7fd9cd1b0b80 1 -- start start 2024-06-10T01:24:07.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.036+0000 7fd9cd1b0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fc1aedf20 0x564fc1aed9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:07.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.036+0000 7fd9cd1b0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc1af3f60 0x564fc1af17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:07.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.036+0000 7fd9cd1b0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fc1b039f0 0x564fc1ae8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:07.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.036+0000 7fd9cd1b0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c011370 con 0x564fc1b039f0 2024-06-10T01:24:07.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.036+0000 7fd9cd1b0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c00ec00 con 0x564fc1aedf20 2024-06-10T01:24:07.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.036+0000 7fd9cd1b0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c00ca50 con 0x564fc1af3f60 2024-06-10T01:24:07.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.036+0000 7fd9c0eca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fc1b039f0 0x564fc1ae8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:07.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.036+0000 7fd9c0eca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fc1b039f0 0x564fc1ae8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51698/0 (socket says 172.21.15.7:51698) 2024-06-10T01:24:07.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.036+0000 7fd9c0eca700 1 -- 172.21.15.7:0/169892628 learned_addr learned my addr 172.21.15.7:0/169892628 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:07.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.036+0000 7fd9bbfff700 1 --2- 172.21.15.7:0/169892628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fc1aedf20 0x564fc1aed9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:07.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.036+0000 7fd9b3fff700 1 --2- 172.21.15.7:0/169892628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc1af3f60 0x564fc1af17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:07.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.036+0000 7fd9c0eca700 1 -- 172.21.15.7:0/169892628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc1af3f60 msgr2=0x564fc1af17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:07.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.036+0000 7fd9c0eca700 1 --2- 172.21.15.7:0/169892628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc1af3f60 0x564fc1af17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:07.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.036+0000 7fd9c0eca700 1 -- 172.21.15.7:0/169892628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fc1aedf20 msgr2=0x564fc1aed9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:07.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.036+0000 7fd9c0eca700 1 --2- 172.21.15.7:0/169892628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fc1aedf20 0x564fc1aed9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:07.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.036+0000 7fd9c0eca700 1 -- 172.21.15.7:0/169892628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fc1af5590 con 0x564fc1b039f0 2024-06-10T01:24:07.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.036+0000 7fd9c0eca700 1 --2- 172.21.15.7:0/169892628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fc1b039f0 0x564fc1ae8940 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4003d50 tx=0x7fd9b4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6dadcdf1a4d6256 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:07.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.037+0000 7fd9b1ffb700 1 -- 172.21.15.7:0/169892628 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b40046b0 con 0x564fc1b039f0 2024-06-10T01:24:07.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.037+0000 7fd9b1ffb700 1 -- 172.21.15.7:0/169892628 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9b4004850 con 0x564fc1b039f0 2024-06-10T01:24:07.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.037+0000 7fd9b1ffb700 1 -- 172.21.15.7:0/169892628 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b40178d0 con 0x564fc1b039f0 2024-06-10T01:24:07.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.037+0000 7fd9cd1b0b80 1 -- 172.21.15.7:0/169892628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fc1b039f0 msgr2=0x564fc1ae8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:07.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.037+0000 7fd9cd1b0b80 1 --2- 172.21.15.7:0/169892628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fc1b039f0 0x564fc1ae8940 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4003d50 tx=0x7fd9b4009920 comp rx=0 tx=0).stop 2024-06-10T01:24:07.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.038+0000 7fd9cd1b0b80 1 -- 172.21.15.7:0/169892628 shutdown_connections 2024-06-10T01:24:07.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.038+0000 7fd9cd1b0b80 1 --2- 172.21.15.7:0/169892628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fc1b039f0 0x564fc1ae8940 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:07.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.038+0000 7fd9cd1b0b80 1 --2- 172.21.15.7:0/169892628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fc1aedf20 0x564fc1aed9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:07.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.038+0000 7fd9cd1b0b80 1 --2- 172.21.15.7:0/169892628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc1af3f60 0x564fc1af17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:07.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.038+0000 7fd9cd1b0b80 1 -- 172.21.15.7:0/169892628 >> 172.21.15.7:0/169892628 conn(0x564fc1a18cc0 msgr2=0x564fc1af10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:07.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.038+0000 7fd9cd1b0b80 1 -- 172.21.15.7:0/169892628 shutdown_connections 2024-06-10T01:24:07.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.038+0000 7fd9cd1b0b80 1 -- 172.21.15.7:0/169892628 wait complete. 2024-06-10T01:24:07.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.038+0000 7fd9cd1b0b80 1 -- start start 2024-06-10T01:24:07.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.038+0000 7fd9cd1b0b80 1 Processor -- start 2024-06-10T01:24:07.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.038+0000 7fd9cd1b0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fc1aedf20 0x564fc1b002f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:07.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.039+0000 7fd9cd1b0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fc1af3f60 0x564fc1b1fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:07.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.039+0000 7fd9cd1b0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc1b039f0 0x564fc1b1ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:07.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.039+0000 7fd9cd1b0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c00d0c0 con 0x564fc1aedf20 2024-06-10T01:24:07.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.039+0000 7fd9cd1b0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c00e530 con 0x564fc1af3f60 2024-06-10T01:24:07.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.039+0000 7fd9cd1b0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c00d790 con 0x564fc1b039f0 2024-06-10T01:24:07.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.039+0000 7fd9cd1b0b80 10 client.0 _ll_register_callbacks cb 0x564fc19d9920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:24:07.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.039+0000 7fd9bbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fc1aedf20 0x564fc1b002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:07.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.039+0000 7fd9bbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fc1aedf20 0x564fc1b002f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51706/0 (socket says 172.21.15.7:51706) 2024-06-10T01:24:07.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.039+0000 7fd9bbfff700 1 -- 172.21.15.7:0/874649216 learned_addr learned my addr 172.21.15.7:0/874649216 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:07.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.039+0000 7fd9b3fff700 1 --2- 172.21.15.7:0/874649216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fc1af3f60 0x564fc1b1fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:07.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.039+0000 7fd9c0eca700 1 --2- 172.21.15.7:0/874649216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc1b039f0 0x564fc1b1ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:07.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.039+0000 7fd9bbfff700 1 -- 172.21.15.7:0/874649216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc1b039f0 msgr2=0x564fc1b1ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:07.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.039+0000 7fd9bbfff700 1 --2- 172.21.15.7:0/874649216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc1b039f0 0x564fc1b1ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:07.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.039+0000 7fd9bbfff700 1 -- 172.21.15.7:0/874649216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fc1af3f60 msgr2=0x564fc1b1fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:07.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.039+0000 7fd9bbfff700 1 --2- 172.21.15.7:0/874649216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fc1af3f60 0x564fc1b1fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:07.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.039+0000 7fd9bbfff700 1 -- 172.21.15.7:0/874649216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fc1b207a0 con 0x564fc1aedf20 2024-06-10T01:24:07.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.039+0000 7fd99b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:24:07.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.039+0000 7fd9bbfff700 1 --2- 172.21.15.7:0/874649216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fc1aedf20 0x564fc1b002f0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac09c1f0 tx=0x7fd9ac09c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff86c3bfd35c5a1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:07.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.040+0000 7fd99b7fe700 1 -- 172.21.15.7:0/874649216 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac09cbf0 con 0x564fc1aedf20 2024-06-10T01:24:07.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.040+0000 7fd99b7fe700 1 -- 172.21.15.7:0/874649216 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fd9ac0a4c40 con 0x564fc1aedf20 2024-06-10T01:24:07.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.040+0000 7fd9cd1b0b80 1 -- 172.21.15.7:0/874649216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9b4004850 con 0x564fc1aedf20 2024-06-10T01:24:07.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.040+0000 7fd99b7fe700 1 -- 172.21.15.7:0/874649216 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fd9ac0a08a0 con 0x564fc1aedf20 2024-06-10T01:24:07.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.040+0000 7fd9cd1b0b80 10 client.19920 fetch_fsmap learned FSMap version 50 2024-06-10T01:24:07.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.040+0000 7fd9cd1b0b80 1 -- 172.21.15.7:0/874649216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x564fc1b83120 con 0x564fc1aedf20 2024-06-10T01:24:07.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.040+0000 7fd99b7fe700 1 -- 172.21.15.7:0/874649216 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9ac0a4e40 con 0x564fc1aedf20 2024-06-10T01:24:07.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.040+0000 7fd99b7fe700 1 -- 172.21.15.7:0/874649216 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7fd9ac022500 con 0x564fc1aedf20 2024-06-10T01:24:07.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.040+0000 7fd9cd1b0b80 10 client.19920 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:24:07.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.040+0000 7fd9cd1b0b80 10 client.19920 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:24:07.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.040+0000 7fd9cd1b0b80 20 client.19920 populate_metadata read hostname 'smithi007' 2024-06-10T01:24:07.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.040+0000 7fd9cd1b0b80 1 --2- 172.21.15.7:0/874649216 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564fc1b835e0 0x564fc1b859d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:07.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.040+0000 7fd9cd1b0b80 4 client.19920 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:24:07.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.040+0000 7fd9cd1b0b80 1 -- 172.21.15.7:0/874649216 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd9b4004850 con 0x564fc1b835e0 2024-06-10T01:24:07.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.041+0000 7fd9b3fff700 1 --2- 172.21.15.7:0/874649216 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564fc1b835e0 0x564fc1b859d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:07.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.041+0000 7fd99b7fe700 10 client.19920 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:07.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.041+0000 7fd9b3fff700 1 --2- 172.21.15.7:0/874649216 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564fc1b835e0 0x564fc1b859d0 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45582f59abb9ca19 server_cookie=9576e8ae0d72b127 in_seq=0 out_seq=0 2024-06-10T01:24:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.046+0000 7fd99b7fe700 1 -- 172.21.15.7:0/874649216 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd9b4004850 con 0x564fc1b835e0 2024-06-10T01:24:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.046+0000 7fd99b7fe700 10 client.19920 handle_command_reply: tid=0 2024-06-10T01:24:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.046+0000 7fd9cd1b0b80 1 client.19920 shutdown 2024-06-10T01:24:07.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.046+0000 7fd9cd1b0b80 1 -- 172.21.15.7:0/874649216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fc1aedf20 msgr2=0x564fc1b002f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:07.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.046+0000 7fd9cd1b0b80 1 --2- 172.21.15.7:0/874649216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fc1aedf20 0x564fc1b002f0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac09c1f0 tx=0x7fd9ac09c3e0 comp rx=0 tx=0).stop 2024-06-10T01:24:07.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.046+0000 7fd9cd1b0b80 1 -- 172.21.15.7:0/874649216 shutdown_connections 2024-06-10T01:24:07.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.046+0000 7fd9cd1b0b80 1 --2- 172.21.15.7:0/874649216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc1b039f0 0x564fc1b1ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:07.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.046+0000 7fd9cd1b0b80 1 --2- 172.21.15.7:0/874649216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fc1aedf20 0x564fc1b002f0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:07.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.046+0000 7fd9cd1b0b80 1 --2- 172.21.15.7:0/874649216 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564fc1b835e0 0x564fc1b859d0 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:07.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.046+0000 7fd99b7fe700 0 client.19920 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:07.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.046+0000 7fd9cd1b0b80 1 --2- 172.21.15.7:0/874649216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fc1af3f60 0x564fc1b1fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:07.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.046+0000 7fd9cd1b0b80 1 -- 172.21.15.7:0/874649216 >> 172.21.15.7:0/874649216 conn(0x564fc1a18cc0 msgr2=0x564fc1ae1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:07.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.047+0000 7fd9cd1b0b80 1 -- 172.21.15.7:0/874649216 shutdown_connections 2024-06-10T01:24:07.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.047+0000 7fd9cd1b0b80 1 -- 172.21.15.7:0/874649216 wait complete. 2024-06-10T01:24:07.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.047+0000 7fd9cd1b0b80 20 client.19920 trim_cache size 0 max 16384 2024-06-10T01:24:07.041 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:24:07.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.047+0000 7fd9c292d700 1 -- 172.21.15.7:0/1398431324 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd9a80a2ca0 msgr2=0x7fd9a80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:07.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.047+0000 7fd9c292d700 1 --2- 172.21.15.7:0/1398431324 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd9a80a2ca0 0x7fd9a80a5160 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0789d0 tx=0x7fd9a4012040 comp rx=0 tx=0).stop 2024-06-10T01:24:07.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.047+0000 7fd9c292d700 1 -- 172.21.15.7:0/1398431324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9bc06a170 msgr2=0x7fd9bc19e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:07.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.047+0000 7fd9c292d700 1 --2- 172.21.15.7:0/1398431324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9bc06a170 0x7fd9bc19e930 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac000c00 tx=0x7fd9ac00bab0 comp rx=0 tx=0).stop 2024-06-10T01:24:07.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.048+0000 7fd9c292d700 1 -- 172.21.15.7:0/1398431324 shutdown_connections 2024-06-10T01:24:07.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.048+0000 7fd9c292d700 1 --2- 172.21.15.7:0/1398431324 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd9a80a2ca0 0x7fd9a80a5160 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.048+0000 7fd9c292d700 1 --2- 172.21.15.7:0/1398431324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9bc108500 0x7fd9bc198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.048+0000 7fd9c292d700 1 --2- 172.21.15.7:0/1398431324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9bc06a170 0x7fd9bc19e930 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.048+0000 7fd9c292d700 1 --2- 172.21.15.7:0/1398431324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9bc06ab30 0x7fd9bc19ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.048+0000 7fd9c292d700 1 -- 172.21.15.7:0/1398431324 >> 172.21.15.7:0/1398431324 conn(0x7fd9bc078340 msgr2=0x7fd9bc100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.048+0000 7fd9c292d700 1 -- 172.21.15.7:0/1398431324 shutdown_connections 2024-06-10T01:24:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:07.048+0000 7fd9c292d700 1 -- 172.21.15.7:0/1398431324 wait complete. 2024-06-10T01:24:07.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:06 smithi007 ceph-mon[32398]: pgmap v976: 129 pgs: 129 active+clean; 557 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.3 MiB/s wr, 240 op/s 2024-06-10T01:24:07.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:06 smithi078 ceph-mon[36683]: pgmap v976: 129 pgs: 129 active+clean; 557 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.3 MiB/s wr, 240 op/s 2024-06-10T01:24:07.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:06 smithi175 ceph-mon[36636]: pgmap v976: 129 pgs: 129 active+clean; 557 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.3 MiB/s wr, 240 op/s 2024-06-10T01:24:07.535 INFO:tasks.workunit.client.0.smithi007.stdout: 10 102488 20.34 MB/sec execute 164 sec latency 109.211 ms 2024-06-10T01:24:08.535 INFO:tasks.workunit.client.0.smithi007.stdout: 10 103219 20.42 MB/sec execute 165 sec latency 112.015 ms 2024-06-10T01:24:08.821 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:24:08.821 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:24:09.136 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:24:09.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:08 smithi007 ceph-mon[32398]: pgmap v977: 129 pgs: 129 active+clean; 555 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.5 MiB/s wr, 262 op/s 2024-06-10T01:24:09.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:08 smithi078 ceph-mon[36683]: pgmap v977: 129 pgs: 129 active+clean; 555 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.5 MiB/s wr, 262 op/s 2024-06-10T01:24:09.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:08 smithi175 ceph-mon[36636]: pgmap v977: 129 pgs: 129 active+clean; 555 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.5 MiB/s wr, 262 op/s 2024-06-10T01:24:09.535 INFO:tasks.workunit.client.0.smithi007.stdout: 10 103977 20.61 MB/sec execute 166 sec latency 54.116 ms 2024-06-10T01:24:10.535 INFO:tasks.workunit.client.0.smithi007.stdout: 10 104565 20.62 MB/sec execute 167 sec latency 94.110 ms 2024-06-10T01:24:11.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:10 smithi007 ceph-mon[32398]: pgmap v978: 129 pgs: 129 active+clean; 567 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.8 MiB/s wr, 406 op/s 2024-06-10T01:24:11.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:10 smithi078 ceph-mon[36683]: pgmap v978: 129 pgs: 129 active+clean; 567 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.8 MiB/s wr, 406 op/s 2024-06-10T01:24:11.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.451+0000 7fdd7df70700 1 -- 172.21.15.7:0/1773619363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd78108500 msgr2=0x7fdd781088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:11.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.451+0000 7fdd7df70700 1 --2- 172.21.15.7:0/1773619363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd78108500 0x7fdd781088e0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fdd68009990 tx=0x7fdd68009ca0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.451+0000 7fdd7df70700 1 -- 172.21.15.7:0/1773619363 shutdown_connections 2024-06-10T01:24:11.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.451+0000 7fdd7df70700 1 --2- 172.21.15.7:0/1773619363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd7806ab30 0x7fdd7810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.451+0000 7fdd7df70700 1 --2- 172.21.15.7:0/1773619363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd78108500 0x7fdd781088e0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.451+0000 7fdd7df70700 1 --2- 172.21.15.7:0/1773619363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd7806a170 0x7fdd7806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.451+0000 7fdd7df70700 1 -- 172.21.15.7:0/1773619363 >> 172.21.15.7:0/1773619363 conn(0x7fdd78078340 msgr2=0x7fdd78078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:11.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.451+0000 7fdd7df70700 1 -- 172.21.15.7:0/1773619363 shutdown_connections 2024-06-10T01:24:11.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.451+0000 7fdd7df70700 1 -- 172.21.15.7:0/1773619363 wait complete. 2024-06-10T01:24:11.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.452+0000 7fdd7df70700 1 Processor -- start 2024-06-10T01:24:11.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.452+0000 7fdd7df70700 1 -- start start 2024-06-10T01:24:11.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.453+0000 7fdd7df70700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd7806a170 0x7fdd78101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:11.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.453+0000 7fdd7df70700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd7806ab30 0x7fdd781019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:11.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.453+0000 7fdd7df70700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd78108500 0x7fdd78101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:11.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.453+0000 7fdd777fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd7806a170 0x7fdd78101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:11.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.453+0000 7fdd777fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd7806a170 0x7fdd78101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51730/0 (socket says 172.21.15.7:51730) 2024-06-10T01:24:11.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.453+0000 7fdd777fe700 1 -- 172.21.15.7:0/3548695434 learned_addr learned my addr 172.21.15.7:0/3548695434 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:11.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.453+0000 7fdd7df70700 1 -- 172.21.15.7:0/3548695434 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdd78112760 con 0x7fdd7806a170 2024-06-10T01:24:11.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.453+0000 7fdd76ffd700 1 --2- 172.21.15.7:0/3548695434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd7806ab30 0x7fdd781019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:11.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.453+0000 7fdd7df70700 1 -- 172.21.15.7:0/3548695434 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdd781125e0 con 0x7fdd78108500 2024-06-10T01:24:11.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.453+0000 7fdd7df70700 1 -- 172.21.15.7:0/3548695434 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdd781128e0 con 0x7fdd7806ab30 2024-06-10T01:24:11.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.453+0000 7fdd77fff700 1 --2- 172.21.15.7:0/3548695434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd78108500 0x7fdd78101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:11.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.453+0000 7fdd77fff700 1 -- 172.21.15.7:0/3548695434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd7806ab30 msgr2=0x7fdd781019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:11.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.453+0000 7fdd77fff700 1 --2- 172.21.15.7:0/3548695434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd7806ab30 0x7fdd781019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.453+0000 7fdd77fff700 1 -- 172.21.15.7:0/3548695434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd7806a170 msgr2=0x7fdd78101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:11.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.453+0000 7fdd77fff700 1 --2- 172.21.15.7:0/3548695434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd7806a170 0x7fdd78101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.453+0000 7fdd77fff700 1 -- 172.21.15.7:0/3548695434 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd781a7130 con 0x7fdd78108500 2024-06-10T01:24:11.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.453+0000 7fdd76ffd700 1 --2- 172.21.15.7:0/3548695434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd7806ab30 0x7fdd781019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:24:11.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.454+0000 7fdd77fff700 1 --2- 172.21.15.7:0/3548695434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd78108500 0x7fdd78101ee0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000efc0 tx=0x7fdd7000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:11.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.454+0000 7fdd74ff9700 1 -- 172.21.15.7:0/3548695434 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd700092e0 con 0x7fdd78108500 2024-06-10T01:24:11.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.454+0000 7fdd74ff9700 1 -- 172.21.15.7:0/3548695434 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd70004bb0 con 0x7fdd78108500 2024-06-10T01:24:11.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.454+0000 7fdd74ff9700 1 -- 172.21.15.7:0/3548695434 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd70004280 con 0x7fdd78108500 2024-06-10T01:24:11.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.454+0000 7fdd7df70700 1 -- 172.21.15.7:0/3548695434 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd781a73c0 con 0x7fdd78108500 2024-06-10T01:24:11.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.454+0000 7fdd7df70700 1 -- 172.21.15.7:0/3548695434 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd781a78b0 con 0x7fdd78108500 2024-06-10T01:24:11.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.457+0000 7fdd74ff9700 1 -- 172.21.15.7:0/3548695434 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fdd7001b030 con 0x7fdd78108500 2024-06-10T01:24:11.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:10 smithi175 ceph-mon[36636]: pgmap v978: 129 pgs: 129 active+clean; 567 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.8 MiB/s wr, 406 op/s 2024-06-10T01:24:11.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.458+0000 7fdd74ff9700 1 --2- 172.21.15.7:0/3548695434 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdd600a2ca0 0x7fdd600a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:11.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.458+0000 7fdd777fe700 1 --2- 172.21.15.7:0/3548695434 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdd600a2ca0 0x7fdd600a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:11.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.458+0000 7fdd74ff9700 1 -- 172.21.15.7:0/3548695434 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fdd700cee60 con 0x7fdd78108500 2024-06-10T01:24:11.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.459+0000 7fdd777fe700 1 --2- 172.21.15.7:0/3548695434 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdd600a2ca0 0x7fdd600a5160 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800b2e0 tx=0x7fdd68009590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:11.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.459+0000 7fdd887f3b80 1 Processor -- start 2024-06-10T01:24:11.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.460+0000 7fdd887f3b80 1 -- start start 2024-06-10T01:24:11.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.460+0000 7fdd887f3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca8c237fe0 0x55ca8c2383c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:11.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.460+0000 7fdd887f3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca8c238900 0x55ca8c2322e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:11.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.460+0000 7fdd887f3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca8c232820 0x55ca8c232c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:11.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.460+0000 7fdd887f3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ca8c124fe0 con 0x55ca8c237fe0 2024-06-10T01:24:11.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.460+0000 7fdd887f3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ca8c1838d0 con 0x55ca8c238900 2024-06-10T01:24:11.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.460+0000 7fdd887f3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ca8c0eb4e0 con 0x55ca8c232820 2024-06-10T01:24:11.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.460+0000 7fdd76ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca8c238900 0x55ca8c2322e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:11.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.460+0000 7fdd76ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca8c238900 0x55ca8c2322e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:40820/0 (socket says 172.21.15.7:40820) 2024-06-10T01:24:11.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.460+0000 7fdd76ffd700 1 -- 172.21.15.7:0/2906619717 learned_addr learned my addr 172.21.15.7:0/2906619717 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:11.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.460+0000 7fdd777fe700 1 --2- 172.21.15.7:0/2906619717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca8c237fe0 0x55ca8c2383c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:11.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.460+0000 7fdd77fff700 1 --2- 172.21.15.7:0/2906619717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca8c232820 0x55ca8c232c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:11.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.460+0000 7fdd76ffd700 1 -- 172.21.15.7:0/2906619717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca8c232820 msgr2=0x55ca8c232c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:11.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.460+0000 7fdd76ffd700 1 --2- 172.21.15.7:0/2906619717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca8c232820 0x55ca8c232c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.460+0000 7fdd76ffd700 1 -- 172.21.15.7:0/2906619717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca8c237fe0 msgr2=0x55ca8c2383c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:11.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.461+0000 7fdd76ffd700 1 --2- 172.21.15.7:0/2906619717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca8c237fe0 0x55ca8c2383c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.461+0000 7fdd76ffd700 1 -- 172.21.15.7:0/2906619717 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca8c0d2950 con 0x55ca8c238900 2024-06-10T01:24:11.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.461+0000 7fdd77fff700 1 --2- 172.21.15.7:0/2906619717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca8c232820 0x55ca8c232c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:24:11.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.461+0000 7fdd777fe700 1 --2- 172.21.15.7:0/2906619717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca8c237fe0 0x55ca8c2383c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:24:11.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.461+0000 7fdd76ffd700 1 --2- 172.21.15.7:0/2906619717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca8c238900 0x55ca8c2322e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c011540 tx=0x7fdd6c011850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c71b004a5ef7501c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:11.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.461+0000 7fdd5dffb700 1 -- 172.21.15.7:0/2906619717 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c004a30 con 0x55ca8c238900 2024-06-10T01:24:11.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.461+0000 7fdd5dffb700 1 -- 172.21.15.7:0/2906619717 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd6c005500 con 0x55ca8c238900 2024-06-10T01:24:11.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.461+0000 7fdd5dffb700 1 -- 172.21.15.7:0/2906619717 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c00e8f0 con 0x55ca8c238900 2024-06-10T01:24:11.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.462+0000 7fdd887f3b80 1 -- 172.21.15.7:0/2906619717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca8c238900 msgr2=0x55ca8c2322e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:11.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.462+0000 7fdd887f3b80 1 --2- 172.21.15.7:0/2906619717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca8c238900 0x55ca8c2322e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c011540 tx=0x7fdd6c011850 comp rx=0 tx=0).stop 2024-06-10T01:24:11.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.462+0000 7fdd887f3b80 1 -- 172.21.15.7:0/2906619717 shutdown_connections 2024-06-10T01:24:11.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.462+0000 7fdd887f3b80 1 --2- 172.21.15.7:0/2906619717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca8c232820 0x55ca8c232c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.462+0000 7fdd887f3b80 1 --2- 172.21.15.7:0/2906619717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca8c237fe0 0x55ca8c2383c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.462+0000 7fdd887f3b80 1 --2- 172.21.15.7:0/2906619717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca8c238900 0x55ca8c2322e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.462+0000 7fdd887f3b80 1 -- 172.21.15.7:0/2906619717 >> 172.21.15.7:0/2906619717 conn(0x55ca8c15f100 msgr2=0x55ca8c22d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:11.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.462+0000 7fdd887f3b80 1 -- 172.21.15.7:0/2906619717 shutdown_connections 2024-06-10T01:24:11.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.462+0000 7fdd887f3b80 1 -- 172.21.15.7:0/2906619717 wait complete. 2024-06-10T01:24:11.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.463+0000 7fdd887f3b80 1 -- start start 2024-06-10T01:24:11.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.463+0000 7fdd887f3b80 1 Processor -- start 2024-06-10T01:24:11.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.463+0000 7fdd887f3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca8c232820 0x55ca8c23e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:11.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.463+0000 7fdd887f3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca8c237fe0 0x55ca8c23b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:11.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.464+0000 7fdd887f3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca8c238900 0x55ca8c23bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:11.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.464+0000 7fdd887f3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ca8c124040 con 0x55ca8c232820 2024-06-10T01:24:11.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.464+0000 7fdd887f3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ca8c17f890 con 0x55ca8c238900 2024-06-10T01:24:11.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.464+0000 7fdd887f3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ca8c23f2e0 con 0x55ca8c237fe0 2024-06-10T01:24:11.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.464+0000 7fdd887f3b80 10 client.0 _ll_register_callbacks cb 0x55ca8c119630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:24:11.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.464+0000 7fdd76ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca8c237fe0 0x55ca8c23b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:11.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.464+0000 7fdd76ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca8c237fe0 0x55ca8c23b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:35874/0 (socket says 172.21.15.7:35874) 2024-06-10T01:24:11.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.464+0000 7fdd76ffd700 1 -- 172.21.15.7:0/2638375576 learned_addr learned my addr 172.21.15.7:0/2638375576 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:11.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.464+0000 7fdd77fff700 1 --2- 172.21.15.7:0/2638375576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca8c238900 0x55ca8c23bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:11.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.464+0000 7fdd777fe700 1 --2- 172.21.15.7:0/2638375576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca8c232820 0x55ca8c23e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:11.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.464+0000 7fdd76ffd700 1 -- 172.21.15.7:0/2638375576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca8c238900 msgr2=0x55ca8c23bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:11.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.464+0000 7fdd76ffd700 1 --2- 172.21.15.7:0/2638375576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca8c238900 0x55ca8c23bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.464+0000 7fdd76ffd700 1 -- 172.21.15.7:0/2638375576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca8c232820 msgr2=0x55ca8c23e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:11.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.464+0000 7fdd76ffd700 1 --2- 172.21.15.7:0/2638375576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca8c232820 0x55ca8c23e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.464+0000 7fdd76ffd700 1 -- 172.21.15.7:0/2638375576 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca8c1497d0 con 0x55ca8c237fe0 2024-06-10T01:24:11.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.464+0000 7fdd777fe700 1 --2- 172.21.15.7:0/2638375576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca8c232820 0x55ca8c23e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:24:11.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.465+0000 7fdd76ffd700 1 --2- 172.21.15.7:0/2638375576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca8c237fe0 0x55ca8c23b990 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c0095f0 tx=0x7fdd6c009750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cce1559c881a667 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:11.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.465+0000 7fdd56ffd700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:24:11.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.465+0000 7fdd56ffd700 1 -- 172.21.15.7:0/2638375576 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c027070 con 0x55ca8c237fe0 2024-06-10T01:24:11.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.465+0000 7fdd56ffd700 1 -- 172.21.15.7:0/2638375576 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fdd6c004b60 con 0x55ca8c237fe0 2024-06-10T01:24:11.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.465+0000 7fdd887f3b80 1 -- 172.21.15.7:0/2638375576 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd6c005500 con 0x55ca8c237fe0 2024-06-10T01:24:11.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.465+0000 7fdd56ffd700 1 -- 172.21.15.7:0/2638375576 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fdd6c033050 con 0x55ca8c237fe0 2024-06-10T01:24:11.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.465+0000 7fdd887f3b80 10 client.29101 fetch_fsmap learned FSMap version 50 2024-06-10T01:24:11.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.465+0000 7fdd887f3b80 1 -- 172.21.15.7:0/2638375576 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55ca8c2497f0 con 0x55ca8c237fe0 2024-06-10T01:24:11.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.466+0000 7fdd56ffd700 1 -- 172.21.15.7:0/2638375576 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd6c005500 con 0x55ca8c237fe0 2024-06-10T01:24:11.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.466+0000 7fdd56ffd700 1 -- 172.21.15.7:0/2638375576 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7fdd6c00e6d0 con 0x55ca8c237fe0 2024-06-10T01:24:11.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.466+0000 7fdd887f3b80 10 client.29101 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:24:11.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.466+0000 7fdd887f3b80 10 client.29101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:24:11.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.466+0000 7fdd887f3b80 20 client.29101 populate_metadata read hostname 'smithi007' 2024-06-10T01:24:11.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.466+0000 7fdd887f3b80 1 --2- 172.21.15.7:0/2638375576 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ca8c24bd20 0x55ca8c24e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:11.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.466+0000 7fdd887f3b80 4 client.29101 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:24:11.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.466+0000 7fdd887f3b80 1 -- 172.21.15.7:0/2638375576 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca8c190dd0 con 0x55ca8c24bd20 2024-06-10T01:24:11.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.466+0000 7fdd777fe700 1 --2- 172.21.15.7:0/2638375576 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ca8c24bd20 0x55ca8c24e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:11.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.467+0000 7fdd777fe700 1 --2- 172.21.15.7:0/2638375576 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ca8c24bd20 0x55ca8c24e110 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f93ba9fd933fad90 server_cookie=6df4a8cdfb1bee94 in_seq=0 out_seq=0 2024-06-10T01:24:11.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.467+0000 7fdd56ffd700 10 client.29101 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:11.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.473+0000 7fdd56ffd700 1 -- 172.21.15.7:0/2638375576 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ca8c190dd0 con 0x55ca8c24bd20 2024-06-10T01:24:11.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.473+0000 7fdd56ffd700 10 client.29101 handle_command_reply: tid=0 2024-06-10T01:24:11.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.473+0000 7fdd887f3b80 1 client.29101 shutdown 2024-06-10T01:24:11.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.473+0000 7fdd887f3b80 1 -- 172.21.15.7:0/2638375576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca8c237fe0 msgr2=0x55ca8c23b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:11.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.473+0000 7fdd887f3b80 1 --2- 172.21.15.7:0/2638375576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca8c237fe0 0x55ca8c23b990 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c0095f0 tx=0x7fdd6c009750 comp rx=0 tx=0).stop 2024-06-10T01:24:11.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.474+0000 7fdd887f3b80 1 -- 172.21.15.7:0/2638375576 shutdown_connections 2024-06-10T01:24:11.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.474+0000 7fdd887f3b80 1 --2- 172.21.15.7:0/2638375576 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ca8c24bd20 0x55ca8c24e110 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.474+0000 7fdd887f3b80 1 --2- 172.21.15.7:0/2638375576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca8c238900 0x55ca8c23bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.474+0000 7fdd887f3b80 1 --2- 172.21.15.7:0/2638375576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca8c232820 0x55ca8c23e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.474+0000 7fdd887f3b80 1 --2- 172.21.15.7:0/2638375576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca8c237fe0 0x55ca8c23b990 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.474+0000 7fdd887f3b80 1 -- 172.21.15.7:0/2638375576 >> 172.21.15.7:0/2638375576 conn(0x55ca8c15f100 msgr2=0x55ca8c22c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:11.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.474+0000 7fdd56ffd700 0 client.29101 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:11.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.474+0000 7fdd887f3b80 1 -- 172.21.15.7:0/2638375576 shutdown_connections 2024-06-10T01:24:11.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.474+0000 7fdd887f3b80 1 -- 172.21.15.7:0/2638375576 wait complete. 2024-06-10T01:24:11.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.474+0000 7fdd887f3b80 20 client.29101 trim_cache size 0 max 16384 2024-06-10T01:24:11.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.507+0000 7fdd887f3b80 1 Processor -- start 2024-06-10T01:24:11.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.508+0000 7fdd887f3b80 1 -- start start 2024-06-10T01:24:11.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.508+0000 7fdd887f3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca8c232820 0x55ca8c2394b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:11.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.508+0000 7fdd887f3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca8c237fe0 0x55ca8c22cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:11.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.508+0000 7fdd887f3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca8c238900 0x55ca8c22d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:11.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.508+0000 7fdd887f3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdd781b2d50 con 0x55ca8c238900 2024-06-10T01:24:11.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.508+0000 7fdd887f3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdd781b05e0 con 0x55ca8c232820 2024-06-10T01:24:11.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.508+0000 7fdd887f3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdd781ae430 con 0x55ca8c237fe0 2024-06-10T01:24:11.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.508+0000 7fdd77fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca8c238900 0x55ca8c22d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:11.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.508+0000 7fdd77fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca8c238900 0x55ca8c22d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51750/0 (socket says 172.21.15.7:51750) 2024-06-10T01:24:11.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.508+0000 7fdd77fff700 1 -- 172.21.15.7:0/4049615409 learned_addr learned my addr 172.21.15.7:0/4049615409 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:11.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.509+0000 7fdd76ffd700 1 --2- 172.21.15.7:0/4049615409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca8c237fe0 0x55ca8c22cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:11.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.509+0000 7fdd77fff700 1 -- 172.21.15.7:0/4049615409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca8c237fe0 msgr2=0x55ca8c22cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:11.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.509+0000 7fdd77fff700 1 --2- 172.21.15.7:0/4049615409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca8c237fe0 0x55ca8c22cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.509+0000 7fdd77fff700 1 -- 172.21.15.7:0/4049615409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca8c232820 msgr2=0x55ca8c2394b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:11.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.509+0000 7fdd77fff700 1 --2- 172.21.15.7:0/4049615409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca8c232820 0x55ca8c2394b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.509+0000 7fdd77fff700 1 -- 172.21.15.7:0/4049615409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca8c239f40 con 0x55ca8c238900 2024-06-10T01:24:11.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.510+0000 7fdd77fff700 1 --2- 172.21.15.7:0/4049615409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca8c238900 0x55ca8c22d440 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fdd700090a0 tx=0x7fdd70004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a7e22c6b46ede3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:11.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.510+0000 7fdd5d7fa700 1 -- 172.21.15.7:0/4049615409 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7009ba70 con 0x55ca8c238900 2024-06-10T01:24:11.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.510+0000 7fdd5d7fa700 1 -- 172.21.15.7:0/4049615409 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd7009b370 con 0x55ca8c238900 2024-06-10T01:24:11.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.510+0000 7fdd5d7fa700 1 -- 172.21.15.7:0/4049615409 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd70004420 con 0x55ca8c238900 2024-06-10T01:24:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.511+0000 7fdd887f3b80 1 -- 172.21.15.7:0/4049615409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca8c238900 msgr2=0x55ca8c22d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.511+0000 7fdd887f3b80 1 --2- 172.21.15.7:0/4049615409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca8c238900 0x55ca8c22d440 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fdd700090a0 tx=0x7fdd70004ad0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.511+0000 7fdd887f3b80 1 -- 172.21.15.7:0/4049615409 shutdown_connections 2024-06-10T01:24:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.511+0000 7fdd887f3b80 1 --2- 172.21.15.7:0/4049615409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca8c238900 0x55ca8c22d440 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.511+0000 7fdd887f3b80 1 --2- 172.21.15.7:0/4049615409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca8c232820 0x55ca8c2394b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.511+0000 7fdd887f3b80 1 --2- 172.21.15.7:0/4049615409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca8c237fe0 0x55ca8c22cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.511+0000 7fdd887f3b80 1 -- 172.21.15.7:0/4049615409 >> 172.21.15.7:0/4049615409 conn(0x55ca8c15f100 msgr2=0x55ca8c238da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.512+0000 7fdd887f3b80 1 -- 172.21.15.7:0/4049615409 shutdown_connections 2024-06-10T01:24:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.512+0000 7fdd887f3b80 1 -- 172.21.15.7:0/4049615409 wait complete. 2024-06-10T01:24:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.512+0000 7fdd887f3b80 1 -- start start 2024-06-10T01:24:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.512+0000 7fdd887f3b80 1 Processor -- start 2024-06-10T01:24:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.513+0000 7fdd887f3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca8c232820 0x55ca8c244980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.513+0000 7fdd887f3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca8c237fe0 0x55ca8c244ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.513+0000 7fdd887f3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca8c238900 0x55ca8c241770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.513+0000 7fdd76ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca8c237fe0 0x55ca8c244ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.513+0000 7fdd76ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca8c237fe0 0x55ca8c244ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:40852/0 (socket says 172.21.15.7:40852) 2024-06-10T01:24:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.513+0000 7fdd777fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca8c232820 0x55ca8c244980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.513+0000 7fdd777fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca8c232820 0x55ca8c244980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:35898/0 (socket says 172.21.15.7:35898) 2024-06-10T01:24:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.514+0000 7fdd76ffd700 1 -- 172.21.15.7:0/1752673929 learned_addr learned my addr 172.21.15.7:0/1752673929 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.514+0000 7fdd887f3b80 1 -- 172.21.15.7:0/1752673929 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdd781aeaa0 con 0x55ca8c238900 2024-06-10T01:24:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.514+0000 7fdd887f3b80 1 -- 172.21.15.7:0/1752673929 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdd781aff10 con 0x55ca8c237fe0 2024-06-10T01:24:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.514+0000 7fdd887f3b80 1 -- 172.21.15.7:0/1752673929 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdd781af170 con 0x55ca8c232820 2024-06-10T01:24:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.514+0000 7fdd887f3b80 10 client.0 _ll_register_callbacks cb 0x55ca8c24e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:24:11.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.514+0000 7fdd76ffd700 1 -- 172.21.15.7:0/1752673929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca8c232820 msgr2=0x55ca8c244980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:11.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.514+0000 7fdd76ffd700 1 --2- 172.21.15.7:0/1752673929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca8c232820 0x55ca8c244980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.514+0000 7fdd76ffd700 1 -- 172.21.15.7:0/1752673929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca8c238900 msgr2=0x55ca8c241770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:11.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.514+0000 7fdd76ffd700 1 --2- 172.21.15.7:0/1752673929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca8c238900 0x55ca8c241770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.514+0000 7fdd76ffd700 1 -- 172.21.15.7:0/1752673929 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca8c241f80 con 0x55ca8c237fe0 2024-06-10T01:24:11.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.515+0000 7fdd76ffd700 1 --2- 172.21.15.7:0/1752673929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca8c237fe0 0x55ca8c244ec0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c001eb0 tx=0x7fdd6c004530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbb8ca5e92fc8e55 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:11.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.515+0000 7fdd887f3b80 1 -- 172.21.15.7:0/1752673929 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd7009b370 con 0x55ca8c237fe0 2024-06-10T01:24:11.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.516+0000 7fdd56ffd700 10 client.29141 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:24:11.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.518+0000 7fdd56ffd700 1 -- 172.21.15.7:0/1752673929 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c027070 con 0x55ca8c237fe0 2024-06-10T01:24:11.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.518+0000 7fdd56ffd700 1 -- 172.21.15.7:0/1752673929 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fdd6c005510 con 0x55ca8c237fe0 2024-06-10T01:24:11.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.518+0000 7fdd56ffd700 1 -- 172.21.15.7:0/1752673929 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fdd6c005b40 con 0x55ca8c237fe0 2024-06-10T01:24:11.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.519+0000 7fdd887f3b80 10 client.29141 fetch_fsmap learned FSMap version 50 2024-06-10T01:24:11.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.519+0000 7fdd887f3b80 1 -- 172.21.15.7:0/1752673929 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=50}) -- 0x55ca8c2b73a0 con 0x55ca8c237fe0 2024-06-10T01:24:11.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.520+0000 7fdd56ffd700 1 -- 172.21.15.7:0/1752673929 <== mon.2 v2:172.21.15.78:3300/0 4 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7fdd6c004960 con 0x55ca8c237fe0 2024-06-10T01:24:11.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.520+0000 7fdd887f3b80 10 client.29141 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:24:11.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.520+0000 7fdd887f3b80 10 client.29141 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:24:11.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.520+0000 7fdd887f3b80 20 client.29141 populate_metadata read hostname 'smithi007' 2024-06-10T01:24:11.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.520+0000 7fdd887f3b80 1 --2- 172.21.15.7:0/1752673929 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ca8c2b7970 0x55ca8c2b9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:11.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.520+0000 7fdd887f3b80 4 client.29141 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:24:11.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.520+0000 7fdd887f3b80 1 -- 172.21.15.7:0/1752673929 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fdd7009b370 con 0x55ca8c2b7970 2024-06-10T01:24:11.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.521+0000 7fdd777fe700 1 --2- 172.21.15.7:0/1752673929 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ca8c2b7970 0x55ca8c2b9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:11.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.523+0000 7fdd777fe700 1 --2- 172.21.15.7:0/1752673929 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ca8c2b7970 0x55ca8c2b9d90 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e4483e68f4f1f8a server_cookie=85eff599820cdfcd in_seq=0 out_seq=0 2024-06-10T01:24:11.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.523+0000 7fdd56ffd700 10 client.29141 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:11.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.535+0000 7fdd56ffd700 1 -- 172.21.15.7:0/1752673929 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fdd7009b370 con 0x55ca8c2b7970 2024-06-10T01:24:11.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.535+0000 7fdd56ffd700 10 client.29141 handle_command_reply: tid=0 2024-06-10T01:24:11.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.535+0000 7fdd887f3b80 1 client.29141 shutdown 2024-06-10T01:24:11.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.536+0000 7fdd887f3b80 1 -- 172.21.15.7:0/1752673929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca8c237fe0 msgr2=0x55ca8c244ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:11.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.536+0000 7fdd887f3b80 1 --2- 172.21.15.7:0/1752673929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca8c237fe0 0x55ca8c244ec0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c001eb0 tx=0x7fdd6c004530 comp rx=0 tx=0).stop 2024-06-10T01:24:11.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.536+0000 7fdd887f3b80 1 -- 172.21.15.7:0/1752673929 shutdown_connections 2024-06-10T01:24:11.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.536+0000 7fdd887f3b80 1 --2- 172.21.15.7:0/1752673929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca8c238900 0x55ca8c241770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.536+0000 7fdd887f3b80 1 --2- 172.21.15.7:0/1752673929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca8c232820 0x55ca8c244980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.536+0000 7fdd887f3b80 1 --2- 172.21.15.7:0/1752673929 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ca8c2b7970 0x55ca8c2b9d90 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.536+0000 7fdd56ffd700 0 client.29141 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:11.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.536+0000 7fdd887f3b80 1 --2- 172.21.15.7:0/1752673929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca8c237fe0 0x55ca8c244ec0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.536+0000 7fdd887f3b80 1 -- 172.21.15.7:0/1752673929 >> 172.21.15.7:0/1752673929 conn(0x55ca8c15f100 msgr2=0x55ca8c23d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:11.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.536+0000 7fdd887f3b80 1 -- 172.21.15.7:0/1752673929 shutdown_connections 2024-06-10T01:24:11.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.536+0000 7fdd887f3b80 1 -- 172.21.15.7:0/1752673929 wait complete. 2024-06-10T01:24:11.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.536+0000 7fdd887f3b80 20 client.29141 trim_cache size 0 max 16384 2024-06-10T01:24:11.531 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:24:11.531 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-10T01:24:11.531 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "1020e3d0-b7e6-46dd-9c2a-fcddc910c096", 2024-06-10T01:24:11.532 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-10T01:24:11.532 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:24:11.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.537+0000 7fdd7df70700 1 -- 172.21.15.7:0/3548695434 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdd600a2ca0 msgr2=0x7fdd600a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:11.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.537+0000 7fdd7df70700 1 --2- 172.21.15.7:0/3548695434 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdd600a2ca0 0x7fdd600a5160 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800b2e0 tx=0x7fdd68009590 comp rx=0 tx=0).stop 2024-06-10T01:24:11.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.537+0000 7fdd7df70700 1 -- 172.21.15.7:0/3548695434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd78108500 msgr2=0x7fdd78101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:11.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.537+0000 7fdd7df70700 1 --2- 172.21.15.7:0/3548695434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd78108500 0x7fdd78101ee0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000efc0 tx=0x7fdd7000c3a0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.537+0000 7fdd7df70700 1 -- 172.21.15.7:0/3548695434 shutdown_connections 2024-06-10T01:24:11.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.537+0000 7fdd7df70700 1 --2- 172.21.15.7:0/3548695434 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdd600a2ca0 0x7fdd600a5160 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.537+0000 7fdd7df70700 1 --2- 172.21.15.7:0/3548695434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd78108500 0x7fdd78101ee0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.537+0000 7fdd7df70700 1 --2- 172.21.15.7:0/3548695434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd7806a170 0x7fdd78101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.537+0000 7fdd7df70700 1 --2- 172.21.15.7:0/3548695434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd7806ab30 0x7fdd781019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:11.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.537+0000 7fdd7df70700 1 -- 172.21.15.7:0/3548695434 >> 172.21.15.7:0/3548695434 conn(0x7fdd78078340 msgr2=0x7fdd780ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:11.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.537+0000 7fdd7df70700 1 -- 172.21.15.7:0/3548695434 shutdown_connections 2024-06-10T01:24:11.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:11.538+0000 7fdd7df70700 1 -- 172.21.15.7:0/3548695434 wait complete. 2024-06-10T01:24:11.535 INFO:tasks.workunit.client.0.smithi007.stdout: 10 105269 20.73 MB/sec execute 168 sec latency 79.821 ms 2024-06-10T01:24:12.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:11 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:24:12.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:11 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:24:12.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:11 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:24:12.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:11 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:24:12.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:11 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:24:12.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:11 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:24:12.535 INFO:tasks.workunit.client.0.smithi007.stdout: 10 106004 20.82 MB/sec execute 169 sec latency 70.714 ms 2024-06-10T01:24:12.588 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:24:12.912 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:24:13.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:12 smithi007 ceph-mon[32398]: scrub queued for path: / 2024-06-10T01:24:13.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:12 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:24:13.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:12 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:24:13.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:12 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:24:13.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:12 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:24:13.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:12 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:24:13.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:12 smithi007 ceph-mon[32398]: pgmap v979: 129 pgs: 129 active+clean; 573 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.4 MiB/s wr, 471 op/s 2024-06-10T01:24:13.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:12 smithi078 ceph-mon[36683]: scrub queued for path: / 2024-06-10T01:24:13.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:12 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:24:13.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:12 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:24:13.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:12 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:24:13.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:12 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:24:13.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:12 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:24:13.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:12 smithi078 ceph-mon[36683]: pgmap v979: 129 pgs: 129 active+clean; 573 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.4 MiB/s wr, 471 op/s 2024-06-10T01:24:13.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:12 smithi175 ceph-mon[36636]: scrub queued for path: / 2024-06-10T01:24:13.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:12 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:24:13.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:12 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:24:13.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:12 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:24:13.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:12 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:24:13.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:12 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:24:13.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:12 smithi175 ceph-mon[36636]: pgmap v979: 129 pgs: 129 active+clean; 573 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.4 MiB/s wr, 471 op/s 2024-06-10T01:24:13.535 INFO:tasks.workunit.client.0.smithi007.stdout: 10 106907 20.98 MB/sec execute 170 sec latency 44.443 ms 2024-06-10T01:24:14.535 INFO:tasks.workunit.client.0.smithi007.stdout: 10 107695 21.14 MB/sec execute 171 sec latency 67.526 ms 2024-06-10T01:24:14.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.599+0000 7f19a2824700 1 -- 172.21.15.7:0/187984673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f199c06b520 msgr2=0x7f199c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:14.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.599+0000 7f19a2824700 1 --2- 172.21.15.7:0/187984673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f199c06b520 0x7f199c10ea90 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f1990009990 tx=0x7f1990009ca0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.599+0000 7f19a2824700 1 -- 172.21.15.7:0/187984673 shutdown_connections 2024-06-10T01:24:14.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.599+0000 7f19a2824700 1 --2- 172.21.15.7:0/187984673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f199c10efd0 0x7f199c1113c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.599+0000 7f19a2824700 1 --2- 172.21.15.7:0/187984673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f199c06ac00 0x7f199c06afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.599+0000 7f19a2824700 1 --2- 172.21.15.7:0/187984673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f199c06b520 0x7f199c10ea90 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.599+0000 7f19a2824700 1 -- 172.21.15.7:0/187984673 >> 172.21.15.7:0/187984673 conn(0x7f199c0773e0 msgr2=0x7f199c0777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:14.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.599+0000 7f19a2824700 1 -- 172.21.15.7:0/187984673 shutdown_connections 2024-06-10T01:24:14.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.599+0000 7f19a2824700 1 -- 172.21.15.7:0/187984673 wait complete. 2024-06-10T01:24:14.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.600+0000 7f19a2824700 1 Processor -- start 2024-06-10T01:24:14.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.600+0000 7f19a2824700 1 -- start start 2024-06-10T01:24:14.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.601+0000 7f19a2824700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f199c06ac00 0x7f199c107fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:14.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.601+0000 7f19a2824700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f199c06b520 0x7f199c108510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:14.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.601+0000 7f19a2824700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f199c10efd0 0x7f199c103670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:14.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.601+0000 7f19a2824700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f199c114a20 con 0x7f199c10efd0 2024-06-10T01:24:14.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.601+0000 7f19a2824700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f199c1148a0 con 0x7f199c06ac00 2024-06-10T01:24:14.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.601+0000 7f19a2824700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f199c114ba0 con 0x7f199c06b520 2024-06-10T01:24:14.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.601+0000 7f199bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f199c06ac00 0x7f199c107fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:14.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.601+0000 7f199bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f199c06ac00 0x7f199c107fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:52888/0 (socket says 172.21.15.7:52888) 2024-06-10T01:24:14.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.601+0000 7f199b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f199c06b520 0x7f199c108510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:14.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.601+0000 7f199bfff700 1 -- 172.21.15.7:0/3504406322 learned_addr learned my addr 172.21.15.7:0/3504406322 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:14.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.601+0000 7f19a0dc1700 1 --2- 172.21.15.7:0/3504406322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f199c10efd0 0x7f199c103670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:14.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.601+0000 7f199bfff700 1 -- 172.21.15.7:0/3504406322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f199c06b520 msgr2=0x7f199c108510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:14.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.601+0000 7f199bfff700 1 --2- 172.21.15.7:0/3504406322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f199c06b520 0x7f199c108510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.601+0000 7f199bfff700 1 -- 172.21.15.7:0/3504406322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f199c10efd0 msgr2=0x7f199c103670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:14.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.601+0000 7f199bfff700 1 --2- 172.21.15.7:0/3504406322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f199c10efd0 0x7f199c103670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.601+0000 7f199bfff700 1 -- 172.21.15.7:0/3504406322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f199c103f70 con 0x7f199c06ac00 2024-06-10T01:24:14.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.602+0000 7f19a0dc1700 1 --2- 172.21.15.7:0/3504406322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f199c10efd0 0x7f199c103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:24:14.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.602+0000 7f199bfff700 1 --2- 172.21.15.7:0/3504406322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f199c06ac00 0x7f199c107fd0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f198c00e9a0 tx=0x7f198c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:14.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.602+0000 7f19997fa700 1 -- 172.21.15.7:0/3504406322 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c00cbd0 con 0x7f199c06ac00 2024-06-10T01:24:14.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.602+0000 7f19997fa700 1 -- 172.21.15.7:0/3504406322 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f198c00cd70 con 0x7f199c06ac00 2024-06-10T01:24:14.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.602+0000 7f19a2824700 1 -- 172.21.15.7:0/3504406322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f199c104260 con 0x7f199c06ac00 2024-06-10T01:24:14.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.602+0000 7f19a2824700 1 -- 172.21.15.7:0/3504406322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f199c10dee0 con 0x7f199c06ac00 2024-06-10T01:24:14.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.603+0000 7f19997fa700 1 -- 172.21.15.7:0/3504406322 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c018750 con 0x7f199c06ac00 2024-06-10T01:24:14.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.605+0000 7f19997fa700 1 -- 172.21.15.7:0/3504406322 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f198c0188f0 con 0x7f199c06ac00 2024-06-10T01:24:14.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.606+0000 7f19997fa700 1 --2- 172.21.15.7:0/3504406322 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f19840a2d70 0x7f19840a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:14.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.606+0000 7f19997fa700 1 -- 172.21.15.7:0/3504406322 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f198c0cf560 con 0x7f199c06ac00 2024-06-10T01:24:14.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.606+0000 7f199b7fe700 1 --2- 172.21.15.7:0/3504406322 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f19840a2d70 0x7f19840a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:14.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.607+0000 7f199b7fe700 1 --2- 172.21.15.7:0/3504406322 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f19840a2d70 0x7f19840a5230 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f199000b2e0 tx=0x7f1990005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:14.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.607+0000 7f19ad0a7b80 1 Processor -- start 2024-06-10T01:24:14.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.607+0000 7f19ad0a7b80 1 -- start start 2024-06-10T01:24:14.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.608+0000 7f19ad0a7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b511334640 0x55b511334a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:14.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.608+0000 7f19ad0a7b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b511334f60 0x55b51132e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:14.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.608+0000 7f19ad0a7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b51132ef20 0x55b51132f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:14.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.608+0000 7f19ad0a7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b511280810 con 0x55b511334640 2024-06-10T01:24:14.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.608+0000 7f19ad0a7b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b511228720 con 0x55b511334f60 2024-06-10T01:24:14.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.608+0000 7f19ad0a7b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b51127c7d0 con 0x55b51132ef20 2024-06-10T01:24:14.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.608+0000 7f199bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b511334640 0x55b511334a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:14.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.608+0000 7f199b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b511334f60 0x55b51132e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:14.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.608+0000 7f199bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b511334640 0x55b511334a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34192/0 (socket says 172.21.15.7:34192) 2024-06-10T01:24:14.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.608+0000 7f199bfff700 1 -- 172.21.15.7:0/3532689059 learned_addr learned my addr 172.21.15.7:0/3532689059 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:14.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.608+0000 7f19a0dc1700 1 --2- 172.21.15.7:0/3532689059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b51132ef20 0x55b51132f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:14.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.608+0000 7f199bfff700 1 -- 172.21.15.7:0/3532689059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b51132ef20 msgr2=0x55b51132f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:14.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.608+0000 7f199bfff700 1 --2- 172.21.15.7:0/3532689059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b51132ef20 0x55b51132f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.608+0000 7f199bfff700 1 -- 172.21.15.7:0/3532689059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b511334f60 msgr2=0x55b51132e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:14.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.608+0000 7f199bfff700 1 --2- 172.21.15.7:0/3532689059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b511334f60 0x55b51132e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.608+0000 7f199bfff700 1 -- 172.21.15.7:0/3532689059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b51121ef50 con 0x55b511334640 2024-06-10T01:24:14.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.608+0000 7f199bfff700 1 --2- 172.21.15.7:0/3532689059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b511334640 0x55b511334a20 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f198c010b60 tx=0x7f198c09c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a530a391a6336 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:14.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.609+0000 7f19827fc700 1 -- 172.21.15.7:0/3532689059 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c0a46b0 con 0x55b511334640 2024-06-10T01:24:14.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.609+0000 7f19827fc700 1 -- 172.21.15.7:0/3532689059 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f198c09ad90 con 0x55b511334640 2024-06-10T01:24:14.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.609+0000 7f19827fc700 1 -- 172.21.15.7:0/3532689059 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c09b950 con 0x55b511334640 2024-06-10T01:24:14.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.609+0000 7f19ad0a7b80 1 -- 172.21.15.7:0/3532689059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b511334640 msgr2=0x55b511334a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:14.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.609+0000 7f19ad0a7b80 1 --2- 172.21.15.7:0/3532689059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b511334640 0x55b511334a20 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f198c010b60 tx=0x7f198c09c440 comp rx=0 tx=0).stop 2024-06-10T01:24:14.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.610+0000 7f19ad0a7b80 1 -- 172.21.15.7:0/3532689059 shutdown_connections 2024-06-10T01:24:14.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.610+0000 7f19ad0a7b80 1 --2- 172.21.15.7:0/3532689059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b51132ef20 0x55b51132f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.610+0000 7f19ad0a7b80 1 --2- 172.21.15.7:0/3532689059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b511334640 0x55b511334a20 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.610+0000 7f19ad0a7b80 1 --2- 172.21.15.7:0/3532689059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b511334f60 0x55b51132e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.610+0000 7f19ad0a7b80 1 -- 172.21.15.7:0/3532689059 >> 172.21.15.7:0/3532689059 conn(0x55b511259cc0 msgr2=0x55b511329850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:14.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.610+0000 7f19ad0a7b80 1 -- 172.21.15.7:0/3532689059 shutdown_connections 2024-06-10T01:24:14.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.610+0000 7f19ad0a7b80 1 -- 172.21.15.7:0/3532689059 wait complete. 2024-06-10T01:24:14.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.610+0000 7f19ad0a7b80 1 -- start start 2024-06-10T01:24:14.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.610+0000 7f19ad0a7b80 1 Processor -- start 2024-06-10T01:24:14.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.611+0000 7f19ad0a7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51132ef20 0x55b51133acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:14.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.611+0000 7f19ad0a7b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b511334640 0x55b511337ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:14.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.611+0000 7f19ad0a7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b511334f60 0x55b511338530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:14.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.611+0000 7f19ad0a7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b51133b970 con 0x55b51132ef20 2024-06-10T01:24:14.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.611+0000 7f19ad0a7b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b511220bf0 con 0x55b511334640 2024-06-10T01:24:14.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.611+0000 7f19ad0a7b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b51133baf0 con 0x55b511334f60 2024-06-10T01:24:14.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.611+0000 7f19ad0a7b80 10 client.0 _ll_register_callbacks cb 0x55b51121a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:24:14.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.611+0000 7f199b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b511334640 0x55b511337ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:14.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.611+0000 7f199b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b511334640 0x55b511337ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:52916/0 (socket says 172.21.15.7:52916) 2024-06-10T01:24:14.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.611+0000 7f199b7fe700 1 -- 172.21.15.7:0/1327450831 learned_addr learned my addr 172.21.15.7:0/1327450831 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:14.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.611+0000 7f199bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51132ef20 0x55b51133acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:14.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.611+0000 7f199b7fe700 1 -- 172.21.15.7:0/1327450831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b511334f60 msgr2=0x55b511338530 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:24:14.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.612+0000 7f199b7fe700 1 --2- 172.21.15.7:0/1327450831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b511334f60 0x55b511338530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.612+0000 7f199b7fe700 1 -- 172.21.15.7:0/1327450831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51132ef20 msgr2=0x55b51133acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:14.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.612+0000 7f199b7fe700 1 --2- 172.21.15.7:0/1327450831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51132ef20 0x55b51133acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.612+0000 7f199b7fe700 1 -- 172.21.15.7:0/1327450831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5111cf8d0 con 0x55b511334640 2024-06-10T01:24:14.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.612+0000 7f199b7fe700 1 --2- 172.21.15.7:0/1327450831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b511334640 0x55b511337ff0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1990009990 tx=0x7f1990009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab850e642ad61208 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:14.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.612+0000 7f197b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:24:14.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.612+0000 7f197b7fe700 1 -- 172.21.15.7:0/1327450831 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199000f460 con 0x55b511334640 2024-06-10T01:24:14.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.612+0000 7f19ad0a7b80 1 -- 172.21.15.7:0/1327450831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f198c09ad90 con 0x55b511334640 2024-06-10T01:24:14.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.612+0000 7f197b7fe700 1 -- 172.21.15.7:0/1327450831 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f199000f600 con 0x55b511334640 2024-06-10T01:24:14.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.612+0000 7f197b7fe700 1 -- 172.21.15.7:0/1327450831 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f199003c050 con 0x55b511334640 2024-06-10T01:24:14.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.612+0000 7f19ad0a7b80 10 client.29153 fetch_fsmap learned FSMap version 50 2024-06-10T01:24:14.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.612+0000 7f19ad0a7b80 1 -- 172.21.15.7:0/1327450831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55b511346360 con 0x55b511334640 2024-06-10T01:24:14.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.613+0000 7f197b7fe700 1 -- 172.21.15.7:0/1327450831 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f198c09ad90 con 0x55b511334640 2024-06-10T01:24:14.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.613+0000 7f197b7fe700 1 -- 172.21.15.7:0/1327450831 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7f1990022850 con 0x55b511334640 2024-06-10T01:24:14.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.613+0000 7f19ad0a7b80 10 client.29153 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:24:14.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.613+0000 7f19ad0a7b80 10 client.29153 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:24:14.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.613+0000 7f19ad0a7b80 20 client.29153 populate_metadata read hostname 'smithi007' 2024-06-10T01:24:14.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.613+0000 7f19ad0a7b80 1 --2- 172.21.15.7:0/1327450831 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b511348940 0x55b51134ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:14.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.613+0000 7f19ad0a7b80 4 client.29153 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:24:14.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.613+0000 7f19ad0a7b80 1 -- 172.21.15.7:0/1327450831 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b51128dd10 con 0x55b511348940 2024-06-10T01:24:14.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.613+0000 7f199bfff700 1 --2- 172.21.15.7:0/1327450831 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b511348940 0x55b51134ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:14.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.614+0000 7f199bfff700 1 --2- 172.21.15.7:0/1327450831 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b511348940 0x55b51134ad90 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3329afbf1f329f8 server_cookie=d9dbd83379324958 in_seq=0 out_seq=0 2024-06-10T01:24:14.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.614+0000 7f197b7fe700 10 client.29153 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:14.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.615+0000 7f197b7fe700 1 -- 172.21.15.7:0/1327450831 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b51128dd10 con 0x55b511348940 2024-06-10T01:24:14.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.615+0000 7f197b7fe700 10 client.29153 handle_command_reply: tid=0 2024-06-10T01:24:14.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.615+0000 7f19ad0a7b80 1 client.29153 shutdown 2024-06-10T01:24:14.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.616+0000 7f19ad0a7b80 1 -- 172.21.15.7:0/1327450831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b511334640 msgr2=0x55b511337ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:14.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.616+0000 7f19ad0a7b80 1 --2- 172.21.15.7:0/1327450831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b511334640 0x55b511337ff0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1990009990 tx=0x7f1990009ca0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.616+0000 7f19ad0a7b80 1 -- 172.21.15.7:0/1327450831 shutdown_connections 2024-06-10T01:24:14.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.616+0000 7f19ad0a7b80 1 --2- 172.21.15.7:0/1327450831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b511334f60 0x55b511338530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.616+0000 7f19ad0a7b80 1 --2- 172.21.15.7:0/1327450831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51132ef20 0x55b51133acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.616+0000 7f19ad0a7b80 1 --2- 172.21.15.7:0/1327450831 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b511348940 0x55b51134ad90 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.616+0000 7f19ad0a7b80 1 --2- 172.21.15.7:0/1327450831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b511334640 0x55b511337ff0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.616+0000 7f197b7fe700 0 client.29153 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:14.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.616+0000 7f19ad0a7b80 1 -- 172.21.15.7:0/1327450831 >> 172.21.15.7:0/1327450831 conn(0x55b511259cc0 msgr2=0x55b5113295f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:14.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.616+0000 7f19ad0a7b80 1 -- 172.21.15.7:0/1327450831 shutdown_connections 2024-06-10T01:24:14.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.616+0000 7f19ad0a7b80 1 -- 172.21.15.7:0/1327450831 wait complete. 2024-06-10T01:24:14.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.616+0000 7f19ad0a7b80 20 client.29153 trim_cache size 0 max 16384 2024-06-10T01:24:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.642+0000 7f19ad0a7b80 1 Processor -- start 2024-06-10T01:24:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.642+0000 7f19ad0a7b80 1 -- start start 2024-06-10T01:24:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.642+0000 7f19ad0a7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51132ef20 0x55b51132e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.642+0000 7f19ad0a7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b511334640 0x55b511337fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.642+0000 7f19ad0a7b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b511334f60 0x55b5113384f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.642+0000 7f19ad0a7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1988011300 con 0x55b51132ef20 2024-06-10T01:24:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.642+0000 7f19ad0a7b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f198800eb90 con 0x55b511334f60 2024-06-10T01:24:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.642+0000 7f19ad0a7b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f198800c9e0 con 0x55b511334640 2024-06-10T01:24:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.642+0000 7f199bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51132ef20 0x55b51132e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.642+0000 7f199bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51132ef20 0x55b51132e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34210/0 (socket says 172.21.15.7:34210) 2024-06-10T01:24:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.642+0000 7f199bfff700 1 -- 172.21.15.7:0/2218244135 learned_addr learned my addr 172.21.15.7:0/2218244135 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.642+0000 7f19a0dc1700 1 --2- 172.21.15.7:0/2218244135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b511334f60 0x55b5113384f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.642+0000 7f199b7fe700 1 --2- 172.21.15.7:0/2218244135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b511334640 0x55b511337fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.643+0000 7f199bfff700 1 -- 172.21.15.7:0/2218244135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b511334640 msgr2=0x55b511337fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.643+0000 7f199bfff700 1 --2- 172.21.15.7:0/2218244135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b511334640 0x55b511337fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.643+0000 7f199bfff700 1 -- 172.21.15.7:0/2218244135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b511334f60 msgr2=0x55b5113384f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.643+0000 7f199bfff700 1 --2- 172.21.15.7:0/2218244135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b511334f60 0x55b5113384f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.643+0000 7f199bfff700 1 -- 172.21.15.7:0/2218244135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5113243b0 con 0x55b51132ef20 2024-06-10T01:24:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.643+0000 7f19a0dc1700 1 --2- 172.21.15.7:0/2218244135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b511334f60 0x55b5113384f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:24:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.643+0000 7f199bfff700 1 --2- 172.21.15.7:0/2218244135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51132ef20 0x55b51132e9e0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f198c010e10 tx=0x7f198c009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc540ba812735f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.643+0000 7f1981ffb700 1 -- 172.21.15.7:0/2218244135 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c0b3070 con 0x55b51132ef20 2024-06-10T01:24:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.643+0000 7f1981ffb700 1 -- 172.21.15.7:0/2218244135 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f198c0cf280 con 0x55b51132ef20 2024-06-10T01:24:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.643+0000 7f1981ffb700 1 -- 172.21.15.7:0/2218244135 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c0987e0 con 0x55b51132ef20 2024-06-10T01:24:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.644+0000 7f19ad0a7b80 1 -- 172.21.15.7:0/2218244135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51132ef20 msgr2=0x55b51132e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.644+0000 7f19ad0a7b80 1 --2- 172.21.15.7:0/2218244135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51132ef20 0x55b51132e9e0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f198c010e10 tx=0x7f198c009880 comp rx=0 tx=0).stop 2024-06-10T01:24:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.644+0000 7f19ad0a7b80 1 -- 172.21.15.7:0/2218244135 shutdown_connections 2024-06-10T01:24:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.644+0000 7f19ad0a7b80 1 --2- 172.21.15.7:0/2218244135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b511334f60 0x55b5113384f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.644+0000 7f19ad0a7b80 1 --2- 172.21.15.7:0/2218244135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51132ef20 0x55b51132e9e0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.644+0000 7f19ad0a7b80 1 --2- 172.21.15.7:0/2218244135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b511334640 0x55b511337fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.644+0000 7f19ad0a7b80 1 -- 172.21.15.7:0/2218244135 >> 172.21.15.7:0/2218244135 conn(0x55b511259cc0 msgr2=0x55b511329380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.644+0000 7f19ad0a7b80 1 -- 172.21.15.7:0/2218244135 shutdown_connections 2024-06-10T01:24:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.644+0000 7f19ad0a7b80 1 -- 172.21.15.7:0/2218244135 wait complete. 2024-06-10T01:24:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.644+0000 7f19ad0a7b80 1 -- start start 2024-06-10T01:24:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.644+0000 7f19ad0a7b80 1 Processor -- start 2024-06-10T01:24:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.645+0000 7f19ad0a7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b51132ef20 0x55b511340de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.645+0000 7f19ad0a7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b511334640 0x55b511341320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.645+0000 7f19ad0a7b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b511334f60 0x55b51134fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.645+0000 7f199bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b51132ef20 0x55b511340de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.645+0000 7f199bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b51132ef20 0x55b511340de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:39702/0 (socket says 172.21.15.7:39702) 2024-06-10T01:24:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.645+0000 7f199bfff700 1 -- 172.21.15.7:0/2147135657 learned_addr learned my addr 172.21.15.7:0/2147135657 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.645+0000 7f19ad0a7b80 1 -- 172.21.15.7:0/2147135657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f198800d050 con 0x55b511334640 2024-06-10T01:24:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.645+0000 7f19ad0a7b80 1 -- 172.21.15.7:0/2147135657 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f198800e4c0 con 0x55b511334f60 2024-06-10T01:24:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.645+0000 7f19ad0a7b80 1 -- 172.21.15.7:0/2147135657 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f198800d720 con 0x55b51132ef20 2024-06-10T01:24:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.645+0000 7f199b7fe700 1 --2- 172.21.15.7:0/2147135657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b511334640 0x55b511341320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.645+0000 7f19ad0a7b80 10 client.0 _ll_register_callbacks cb 0x55b51121a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:24:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.645+0000 7f19a0dc1700 1 --2- 172.21.15.7:0/2147135657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b511334f60 0x55b51134fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.645+0000 7f199bfff700 1 -- 172.21.15.7:0/2147135657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b511334f60 msgr2=0x55b51134fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.645+0000 7f199bfff700 1 --2- 172.21.15.7:0/2147135657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b511334f60 0x55b51134fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.645+0000 7f199bfff700 1 -- 172.21.15.7:0/2147135657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b511334640 msgr2=0x55b511341320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.645+0000 7f199bfff700 1 --2- 172.21.15.7:0/2147135657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b511334640 0x55b511341320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.645+0000 7f199bfff700 1 -- 172.21.15.7:0/2147135657 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5113500d0 con 0x55b51132ef20 2024-06-10T01:24:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.645+0000 7f199b7fe700 1 --2- 172.21.15.7:0/2147135657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b511334640 0x55b511341320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:24:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.646+0000 7f199bfff700 1 --2- 172.21.15.7:0/2147135657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b51132ef20 0x55b511340de0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f198c010eb0 tx=0x7f198c009810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50c7d46d9b8141 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.646+0000 7f197b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:24:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.646+0000 7f197b7fe700 1 -- 172.21.15.7:0/2147135657 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c0b3040 con 0x55b51132ef20 2024-06-10T01:24:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.646+0000 7f19ad0a7b80 1 -- 172.21.15.7:0/2147135657 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f198c0cf280 con 0x55b51132ef20 2024-06-10T01:24:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.646+0000 7f197b7fe700 1 -- 172.21.15.7:0/2147135657 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f198c018de0 con 0x55b51132ef20 2024-06-10T01:24:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.646+0000 7f197b7fe700 1 -- 172.21.15.7:0/2147135657 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f198c09c200 con 0x55b51132ef20 2024-06-10T01:24:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.646+0000 7f19ad0a7b80 10 client.29119 fetch_fsmap learned FSMap version 50 2024-06-10T01:24:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.646+0000 7f19ad0a7b80 1 -- 172.21.15.7:0/2147135657 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55b5113b2f80 con 0x55b51132ef20 2024-06-10T01:24:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.646+0000 7f197b7fe700 1 -- 172.21.15.7:0/2147135657 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f198c0cf280 con 0x55b51132ef20 2024-06-10T01:24:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.646+0000 7f197b7fe700 1 -- 172.21.15.7:0/2147135657 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7f198c09b950 con 0x55b51132ef20 2024-06-10T01:24:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.647+0000 7f19ad0a7b80 10 client.29119 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:24:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.647+0000 7f19ad0a7b80 10 client.29119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:24:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.647+0000 7f19ad0a7b80 20 client.29119 populate_metadata read hostname 'smithi007' 2024-06-10T01:24:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.647+0000 7f19ad0a7b80 1 --2- 172.21.15.7:0/2147135657 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b5113b3210 0x55b5113b5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.647+0000 7f19ad0a7b80 4 client.29119 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:24:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.647+0000 7f19ad0a7b80 1 -- 172.21.15.7:0/2147135657 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b511288290 con 0x55b5113b3210 2024-06-10T01:24:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.647+0000 7f199b7fe700 1 --2- 172.21.15.7:0/2147135657 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b5113b3210 0x55b5113b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.647+0000 7f197b7fe700 10 client.29119 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.647+0000 7f199b7fe700 1 --2- 172.21.15.7:0/2147135657 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b5113b3210 0x55b5113b5600 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be45849692f46903 server_cookie=5a13e8fa39fcb68 in_seq=0 out_seq=0 2024-06-10T01:24:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.649+0000 7f197b7fe700 1 -- 172.21.15.7:0/2147135657 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b511288290 con 0x55b5113b3210 2024-06-10T01:24:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.649+0000 7f197b7fe700 10 client.29119 handle_command_reply: tid=0 2024-06-10T01:24:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.649+0000 7f19ad0a7b80 1 client.29119 shutdown 2024-06-10T01:24:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.649+0000 7f19ad0a7b80 1 -- 172.21.15.7:0/2147135657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b51132ef20 msgr2=0x55b511340de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.649+0000 7f19ad0a7b80 1 --2- 172.21.15.7:0/2147135657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b51132ef20 0x55b511340de0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f198c010eb0 tx=0x7f198c009810 comp rx=0 tx=0).stop 2024-06-10T01:24:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.649+0000 7f19ad0a7b80 1 -- 172.21.15.7:0/2147135657 shutdown_connections 2024-06-10T01:24:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.649+0000 7f19ad0a7b80 1 --2- 172.21.15.7:0/2147135657 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b5113b3210 0x55b5113b5600 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.649+0000 7f19ad0a7b80 1 --2- 172.21.15.7:0/2147135657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b511334f60 0x55b51134fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.649+0000 7f197b7fe700 0 client.29119 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.649+0000 7f19ad0a7b80 1 --2- 172.21.15.7:0/2147135657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b51132ef20 0x55b511340de0 secure :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f198c010eb0 tx=0x7f198c009810 comp rx=0 tx=0).stop 2024-06-10T01:24:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.649+0000 7f19ad0a7b80 1 --2- 172.21.15.7:0/2147135657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b511334640 0x55b511341320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.649+0000 7f19ad0a7b80 1 -- 172.21.15.7:0/2147135657 >> 172.21.15.7:0/2147135657 conn(0x55b511259cc0 msgr2=0x55b511337f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.649+0000 7f19ad0a7b80 1 -- 172.21.15.7:0/2147135657 shutdown_connections 2024-06-10T01:24:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.649+0000 7f19ad0a7b80 1 -- 172.21.15.7:0/2147135657 wait complete. 2024-06-10T01:24:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.649+0000 7f19ad0a7b80 20 client.29119 trim_cache size 0 max 16384 2024-06-10T01:24:14.647 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:24:14.647 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T01:24:14.647 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-10T01:24:14.647 INFO:teuthology.orchestra.run.smithi007.stdout: "1020e3d0-b7e6-46dd-9c2a-fcddc910c096": { 2024-06-10T01:24:14.647 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-10T01:24:14.647 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "1020e3d0-b7e6-46dd-9c2a-fcddc910c096", 2024-06-10T01:24:14.647 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-10T01:24:14.647 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:24:14.647 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:24:14.647 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:24:14.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.650+0000 7f19a2824700 1 -- 172.21.15.7:0/3504406322 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f19840a2d70 msgr2=0x7f19840a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:14.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.650+0000 7f19a2824700 1 --2- 172.21.15.7:0/3504406322 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f19840a2d70 0x7f19840a5230 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f199000b2e0 tx=0x7f1990005960 comp rx=0 tx=0).stop 2024-06-10T01:24:14.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.650+0000 7f19a2824700 1 -- 172.21.15.7:0/3504406322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f199c06ac00 msgr2=0x7f199c107fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:14.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.650+0000 7f19a2824700 1 --2- 172.21.15.7:0/3504406322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f199c06ac00 0x7f199c107fd0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f198c00e9a0 tx=0x7f198c00ecb0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.650+0000 7f19a2824700 1 -- 172.21.15.7:0/3504406322 shutdown_connections 2024-06-10T01:24:14.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.650+0000 7f19a2824700 1 --2- 172.21.15.7:0/3504406322 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f19840a2d70 0x7f19840a5230 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.650+0000 7f19a2824700 1 --2- 172.21.15.7:0/3504406322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f199c10efd0 0x7f199c103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.650+0000 7f19a2824700 1 --2- 172.21.15.7:0/3504406322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f199c06ac00 0x7f199c107fd0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.650+0000 7f19a2824700 1 --2- 172.21.15.7:0/3504406322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f199c06b520 0x7f199c108510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.650+0000 7f19a2824700 1 -- 172.21.15.7:0/3504406322 >> 172.21.15.7:0/3504406322 conn(0x7f199c0773e0 msgr2=0x7f199c1104b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.650+0000 7f19a2824700 1 -- 172.21.15.7:0/3504406322 shutdown_connections 2024-06-10T01:24:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:14.650+0000 7f19a2824700 1 -- 172.21.15.7:0/3504406322 wait complete. 2024-06-10T01:24:15.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:14 smithi007 ceph-mon[32398]: pgmap v980: 129 pgs: 129 active+clean; 575 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.6 MiB/s wr, 499 op/s 2024-06-10T01:24:15.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:14 smithi078 ceph-mon[36683]: pgmap v980: 129 pgs: 129 active+clean; 575 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.6 MiB/s wr, 499 op/s 2024-06-10T01:24:15.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:14 smithi175 ceph-mon[36636]: pgmap v980: 129 pgs: 129 active+clean; 575 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.6 MiB/s wr, 499 op/s 2024-06-10T01:24:15.535 INFO:tasks.workunit.client.0.smithi007.stdout: 10 108508 21.29 MB/sec execute 172 sec latency 53.006 ms 2024-06-10T01:24:16.188 INFO:tasks.cephfs.filesystem:scrub status for tag:1020e3d0-b7e6-46dd-9c2a-fcddc910c096 - {'path': '/', 'tag': '1020e3d0-b7e6-46dd-9c2a-fcddc910c096', 'options': 'recursive,force'} 2024-06-10T01:24:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.339+0000 7f0e809fe640 1 -- 172.21.15.7:0/1362596258 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7402f320 con 0x7f0e7c1089a0 2024-06-10T01:24:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.339+0000 7f0e83c64640 1 -- 172.21.15.7:0/1362596258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e7c1089a0 msgr2=0x7f0e7c108dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.339+0000 7f0e83c64640 1 --2- 172.21.15.7:0/1362596258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e7c1089a0 0x7f0e7c108dc0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f0e74009cd0 tx=0x7f0e7402a910 comp rx=0 tx=0).stop 2024-06-10T01:24:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.339+0000 7f0e83c64640 1 -- 172.21.15.7:0/1362596258 shutdown_connections 2024-06-10T01:24:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.339+0000 7f0e83c64640 1 --2- 172.21.15.7:0/1362596258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e7c110220 0x7f0e7c112610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.339+0000 7f0e83c64640 1 --2- 172.21.15.7:0/1362596258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e7c109300 0x7f0e7c10fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.339+0000 7f0e83c64640 1 --2- 172.21.15.7:0/1362596258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e7c1089a0 0x7f0e7c108dc0 secure :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f0e74009cd0 tx=0x7f0e7402a910 comp rx=0 tx=0).stop 2024-06-10T01:24:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.339+0000 7f0e83c64640 1 -- 172.21.15.7:0/1362596258 >> 172.21.15.7:0/1362596258 conn(0x7f0e7c06e9e0 msgr2=0x7f0e7c070e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.340+0000 7f0e83c64640 1 -- 172.21.15.7:0/1362596258 shutdown_connections 2024-06-10T01:24:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.340+0000 7f0e83c64640 1 -- 172.21.15.7:0/1362596258 wait complete. 2024-06-10T01:24:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.341+0000 7f0e83c64640 1 Processor -- start 2024-06-10T01:24:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.341+0000 7f0e83c64640 1 -- start start 2024-06-10T01:24:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.341+0000 7f0e83c64640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e7c109300 0x7f0e7c10b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.341+0000 7f0e83c64640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e7c110220 0x7f0e7c109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.341+0000 7f0e83c64640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e7c1a4440 0x7f0e7c10a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.341+0000 7f0e83c64640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c115f40 con 0x7f0e7c109300 2024-06-10T01:24:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.341+0000 7f0e83c64640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c115dc0 con 0x7f0e7c1a4440 2024-06-10T01:24:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.341+0000 7f0e83c64640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c1160c0 con 0x7f0e7c110220 2024-06-10T01:24:16.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.341+0000 7f0e81a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e7c109300 0x7f0e7c10b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:16.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.342+0000 7f0e81a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e7c109300 0x7f0e7c10b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34236/0 (socket says 172.21.15.7:34236) 2024-06-10T01:24:16.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.342+0000 7f0e81a00640 1 -- 172.21.15.7:0/2177987993 learned_addr learned my addr 172.21.15.7:0/2177987993 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:16.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.342+0000 7f0e811ff640 1 --2- 172.21.15.7:0/2177987993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e7c110220 0x7f0e7c109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:16.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.342+0000 7f0e82201640 1 --2- 172.21.15.7:0/2177987993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e7c1a4440 0x7f0e7c10a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:16.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.342+0000 7f0e811ff640 1 -- 172.21.15.7:0/2177987993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e7c1a4440 msgr2=0x7f0e7c10a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:16.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.342+0000 7f0e811ff640 1 --2- 172.21.15.7:0/2177987993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e7c1a4440 0x7f0e7c10a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:16.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.342+0000 7f0e811ff640 1 -- 172.21.15.7:0/2177987993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e7c109300 msgr2=0x7f0e7c10b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:16.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.342+0000 7f0e811ff640 1 --2- 172.21.15.7:0/2177987993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e7c109300 0x7f0e7c10b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:16.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.342+0000 7f0e811ff640 1 -- 172.21.15.7:0/2177987993 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e7c10aa40 con 0x7f0e7c110220 2024-06-10T01:24:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.343+0000 7f0e82201640 1 --2- 172.21.15.7:0/2177987993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e7c1a4440 0x7f0e7c10a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:24:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.343+0000 7f0e811ff640 1 --2- 172.21.15.7:0/2177987993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e7c110220 0x7f0e7c109c70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0e70009fd0 tx=0x7f0e7000b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.343+0000 7f0e7affd640 1 -- 172.21.15.7:0/2177987993 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e70015020 con 0x7f0e7c110220 2024-06-10T01:24:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.343+0000 7f0e7affd640 1 -- 172.21.15.7:0/2177987993 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e70011d10 con 0x7f0e7c110220 2024-06-10T01:24:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.343+0000 7f0e7affd640 1 -- 172.21.15.7:0/2177987993 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e70015020 con 0x7f0e7c110220 2024-06-10T01:24:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.344+0000 7f0e83c64640 1 -- 172.21.15.7:0/2177987993 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e7c1a48f0 con 0x7f0e7c110220 2024-06-10T01:24:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.344+0000 7f0e83c64640 1 -- 172.21.15.7:0/2177987993 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e7c1a4e70 con 0x7f0e7c110220 2024-06-10T01:24:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.349+0000 7f0e86974740 1 Processor -- start 2024-06-10T01:24:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.349+0000 7f0e86974740 1 -- start start 2024-06-10T01:24:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.350+0000 7f0e86974740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556619597010 0x556619597430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.350+0000 7f0e86974740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556619597a00 0x55661959c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.350+0000 7f0e86974740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55661959cb70 0x55661959f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.350+0000 7f0e86974740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5566194fa4f0 con 0x55661959cb70 2024-06-10T01:24:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.350+0000 7f0e86974740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556619498870 con 0x556619597010 2024-06-10T01:24:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.350+0000 7f0e86974740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5566194f64b0 con 0x556619597a00 2024-06-10T01:24:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.350+0000 7f0e82201640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556619597010 0x556619597430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.350+0000 7f0e82201640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556619597010 0x556619597430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:52976/0 (socket says 172.21.15.7:52976) 2024-06-10T01:24:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.350+0000 7f0e82201640 1 -- 172.21.15.7:0/1571159888 learned_addr learned my addr 172.21.15.7:0/1571159888 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.351+0000 7f0e7affd640 1 -- 172.21.15.7:0/2177987993 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0e70018020 con 0x7f0e7c110220 2024-06-10T01:24:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.351+0000 7f0e82201640 1 -- 172.21.15.7:0/1571159888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556619597a00 msgr2=0x55661959c500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T01:24:16.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.351+0000 7f0e82201640 1 --2- 172.21.15.7:0/1571159888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556619597a00 0x55661959c500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:16.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.351+0000 7f0e82201640 1 -- 172.21.15.7:0/1571159888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55661959cb70 msgr2=0x55661959f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:16.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.351+0000 7f0e82201640 1 --2- 172.21.15.7:0/1571159888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55661959cb70 0x55661959f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:16.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.351+0000 7f0e82201640 1 -- 172.21.15.7:0/1571159888 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566194b49b0 con 0x556619597010 2024-06-10T01:24:16.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.351+0000 7f0e82201640 1 --2- 172.21.15.7:0/1571159888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556619597010 0x556619597430 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009cc0 tx=0x7f0e6c0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1eb842482b2ea39f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.352+0000 7f0e63fff640 1 -- 172.21.15.7:0/1571159888 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c012450 con 0x556619597010 2024-06-10T01:24:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.352+0000 7f0e63fff640 1 -- 172.21.15.7:0/1571159888 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e6c012ad0 con 0x556619597010 2024-06-10T01:24:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.352+0000 7f0e63fff640 1 -- 172.21.15.7:0/1571159888 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c00fc90 con 0x556619597010 2024-06-10T01:24:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.352+0000 7f0e7affd640 1 --2- 172.21.15.7:0/2177987993 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0e640a2fc0 0x7f0e640a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.353+0000 7f0e7affd640 1 -- 172.21.15.7:0/2177987993 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f0e70011630 con 0x7f0e7c110220 2024-06-10T01:24:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.353+0000 7f0e81a00640 1 --2- 172.21.15.7:0/2177987993 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0e640a2fc0 0x7f0e640a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.353+0000 7f0e86974740 1 -- 172.21.15.7:0/1571159888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556619597010 msgr2=0x556619597430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.353+0000 7f0e86974740 1 --2- 172.21.15.7:0/1571159888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556619597010 0x556619597430 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009cc0 tx=0x7f0e6c0117b0 comp rx=0 tx=0).stop 2024-06-10T01:24:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.354+0000 7f0e86974740 1 -- 172.21.15.7:0/1571159888 shutdown_connections 2024-06-10T01:24:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.354+0000 7f0e86974740 1 --2- 172.21.15.7:0/1571159888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55661959cb70 0x55661959f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.354+0000 7f0e86974740 1 --2- 172.21.15.7:0/1571159888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556619597a00 0x55661959c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:16.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.354+0000 7f0e86974740 1 --2- 172.21.15.7:0/1571159888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556619597010 0x556619597430 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:16.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.354+0000 7f0e86974740 1 -- 172.21.15.7:0/1571159888 >> 172.21.15.7:0/1571159888 conn(0x556619592680 msgr2=0x556619592a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:16.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.354+0000 7f0e86974740 1 -- 172.21.15.7:0/1571159888 shutdown_connections 2024-06-10T01:24:16.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.354+0000 7f0e86974740 1 -- 172.21.15.7:0/1571159888 wait complete. 2024-06-10T01:24:16.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.355+0000 7f0e81a00640 1 --2- 172.21.15.7:0/2177987993 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0e640a2fc0 0x7f0e640a5480 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x556619598470 tx=0x7f0e74000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.355+0000 7f0e86974740 1 -- start start 2024-06-10T01:24:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.355+0000 7f0e86974740 1 Processor -- start 2024-06-10T01:24:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.355+0000 7f0e86974740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556619597010 0x5566195a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.355+0000 7f0e86974740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556619597a00 0x5566195a1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.355+0000 7f0e86974740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55661959cb70 0x5566195a8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.355+0000 7f0e86974740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55661959fb20 con 0x556619597010 2024-06-10T01:24:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.356+0000 7f0e86974740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556619483ae0 con 0x556619597a00 2024-06-10T01:24:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.356+0000 7f0e86974740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55661959fca0 con 0x55661959cb70 2024-06-10T01:24:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.356+0000 7f0e81a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556619597010 0x5566195a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.356+0000 7f0e81a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556619597010 0x5566195a12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34258/0 (socket says 172.21.15.7:34258) 2024-06-10T01:24:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.356+0000 7f0e81a00640 1 -- 172.21.15.7:0/4039291503 learned_addr learned my addr 172.21.15.7:0/4039291503 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.356+0000 7f0e86974740 10 client.0 _ll_register_callbacks cb 0x5566194bc9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:24:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.356+0000 7f0e82201640 1 --2- 172.21.15.7:0/4039291503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55661959cb70 0x5566195a8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.356+0000 7f0e81a00640 1 -- 172.21.15.7:0/4039291503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55661959cb70 msgr2=0x5566195a8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.356+0000 7f0e81a00640 1 --2- 172.21.15.7:0/4039291503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55661959cb70 0x5566195a8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.356+0000 7f0e81a00640 1 -- 172.21.15.7:0/4039291503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556619597a00 msgr2=0x5566195a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.356+0000 7f0e81a00640 1 --2- 172.21.15.7:0/4039291503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556619597a00 0x5566195a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.356+0000 7f0e81a00640 1 -- 172.21.15.7:0/4039291503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55661920e210 con 0x556619597010 2024-06-10T01:24:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.357+0000 7f0e617fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:24:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.357+0000 7f0e81a00640 1 --2- 172.21.15.7:0/4039291503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556619597010 0x5566195a12f0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f0e7402af00 tx=0x7f0e7402a910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71daab724ef2e3a1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.357+0000 7f0e617fa640 1 -- 172.21.15.7:0/4039291503 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e74003b70 con 0x556619597010 2024-06-10T01:24:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.357+0000 7f0e86974740 1 -- 172.21.15.7:0/4039291503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e6c012ad0 con 0x556619597010 2024-06-10T01:24:16.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.357+0000 7f0e617fa640 1 -- 172.21.15.7:0/4039291503 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f0e74003970 con 0x556619597010 2024-06-10T01:24:16.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.358+0000 7f0e617fa640 1 -- 172.21.15.7:0/4039291503 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f0e74059050 con 0x556619597010 2024-06-10T01:24:16.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.358+0000 7f0e86974740 10 client.19959 fetch_fsmap learned FSMap version 50 2024-06-10T01:24:16.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.358+0000 7f0e86974740 1 -- 172.21.15.7:0/4039291503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5566192da620 con 0x556619597010 2024-06-10T01:24:16.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.359+0000 7f0e617fa640 1 -- 172.21.15.7:0/4039291503 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e6c012ad0 con 0x556619597010 2024-06-10T01:24:16.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.359+0000 7f0e617fa640 1 -- 172.21.15.7:0/4039291503 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7f0e74049540 con 0x556619597010 2024-06-10T01:24:16.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.360+0000 7f0e86974740 10 client.19959 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:24:16.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.360+0000 7f0e86974740 10 client.19959 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:24:16.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.360+0000 7f0e86974740 20 client.19959 populate_metadata read hostname 'smithi007' 2024-06-10T01:24:16.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.360+0000 7f0e86974740 1 --2- 172.21.15.7:0/4039291503 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5566195ad260 0x5566195af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:16.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.360+0000 7f0e86974740 4 client.19959 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:24:16.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.360+0000 7f0e86974740 1 -- 172.21.15.7:0/4039291503 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55661950df10 con 0x5566195ad260 2024-06-10T01:24:16.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.361+0000 7f0e811ff640 1 --2- 172.21.15.7:0/4039291503 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5566195ad260 0x5566195af6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.366+0000 7f0e811ff640 1 --2- 172.21.15.7:0/4039291503 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5566195ad260 0x5566195af6b0 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21567ab0e3cbc38 server_cookie=98e7cb18cff0e89d in_seq=0 out_seq=0 2024-06-10T01:24:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.368+0000 7f0e617fa640 10 client.19959 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.369+0000 7f0e617fa640 1 -- 172.21.15.7:0/4039291503 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55661950df10 con 0x5566195ad260 2024-06-10T01:24:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.369+0000 7f0e617fa640 10 client.19959 handle_command_reply: tid=0 2024-06-10T01:24:16.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.370+0000 7f0e86974740 1 client.19959 shutdown 2024-06-10T01:24:16.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.370+0000 7f0e86974740 1 -- 172.21.15.7:0/4039291503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556619597010 msgr2=0x5566195a12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:16.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.370+0000 7f0e86974740 1 --2- 172.21.15.7:0/4039291503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556619597010 0x5566195a12f0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f0e7402af00 tx=0x7f0e7402a910 comp rx=0 tx=0).stop 2024-06-10T01:24:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.370+0000 7f0e86974740 1 -- 172.21.15.7:0/4039291503 shutdown_connections 2024-06-10T01:24:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.370+0000 7f0e86974740 1 --2- 172.21.15.7:0/4039291503 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5566195ad260 0x5566195af6b0 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.370+0000 7f0e617fa640 0 client.19959 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.370+0000 7f0e86974740 1 --2- 172.21.15.7:0/4039291503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55661959cb70 0x5566195a8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.370+0000 7f0e86974740 1 --2- 172.21.15.7:0/4039291503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556619597a00 0x5566195a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.370+0000 7f0e86974740 1 --2- 172.21.15.7:0/4039291503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556619597010 0x5566195a12f0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.371+0000 7f0e86974740 1 -- 172.21.15.7:0/4039291503 >> 172.21.15.7:0/4039291503 conn(0x556619592680 msgr2=0x5566195917e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.371+0000 7f0e86974740 1 -- 172.21.15.7:0/4039291503 shutdown_connections 2024-06-10T01:24:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.371+0000 7f0e86974740 1 -- 172.21.15.7:0/4039291503 wait complete. 2024-06-10T01:24:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.372+0000 7f0e86974740 20 client.19959 trim_cache size 0 max 16384 2024-06-10T01:24:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.404+0000 7f0e86974740 1 Processor -- start 2024-06-10T01:24:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.405+0000 7f0e86974740 1 -- start start 2024-06-10T01:24:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.405+0000 7f0e86974740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556619597010 0x556619590f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.405+0000 7f0e86974740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556619597a00 0x5566195a8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.405+0000 7f0e86974740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55661959cb70 0x556619591490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.405+0000 7f0e86974740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0e68012640 con 0x55661959cb70 2024-06-10T01:24:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.405+0000 7f0e86974740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6800ff10 con 0x556619597a00 2024-06-10T01:24:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.405+0000 7f0e86974740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6800dd20 con 0x556619597010 2024-06-10T01:24:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.405+0000 7f0e82201640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55661959cb70 0x556619591490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.405+0000 7f0e82201640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55661959cb70 0x556619591490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34264/0 (socket says 172.21.15.7:34264) 2024-06-10T01:24:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.405+0000 7f0e82201640 1 -- 172.21.15.7:0/493696175 learned_addr learned my addr 172.21.15.7:0/493696175 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.406+0000 7f0e811ff640 1 --2- 172.21.15.7:0/493696175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556619597a00 0x5566195a8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.406+0000 7f0e82201640 1 -- 172.21.15.7:0/493696175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556619597010 msgr2=0x556619590f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.406+0000 7f0e82201640 1 --2- 172.21.15.7:0/493696175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556619597010 0x556619590f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.406+0000 7f0e82201640 1 -- 172.21.15.7:0/493696175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556619597a00 msgr2=0x5566195a8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.406+0000 7f0e82201640 1 --2- 172.21.15.7:0/493696175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556619597a00 0x5566195a8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.406+0000 7f0e82201640 1 -- 172.21.15.7:0/493696175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566195a03d0 con 0x55661959cb70 2024-06-10T01:24:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.407+0000 7f0e82201640 1 --2- 172.21.15.7:0/493696175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55661959cb70 0x556619591490 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009be0 tx=0x7f0e6c009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd0a8fdd0cd3ecb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:16.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.407+0000 7f0e637fe640 1 -- 172.21.15.7:0/493696175 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c029030 con 0x55661959cb70 2024-06-10T01:24:16.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.407+0000 7f0e637fe640 1 -- 172.21.15.7:0/493696175 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e6c005520 con 0x55661959cb70 2024-06-10T01:24:16.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.407+0000 7f0e637fe640 1 -- 172.21.15.7:0/493696175 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c004e50 con 0x55661959cb70 2024-06-10T01:24:16.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.408+0000 7f0e86974740 1 -- 172.21.15.7:0/493696175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55661959cb70 msgr2=0x556619591490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:16.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.408+0000 7f0e86974740 1 --2- 172.21.15.7:0/493696175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55661959cb70 0x556619591490 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009be0 tx=0x7f0e6c009e60 comp rx=0 tx=0).stop 2024-06-10T01:24:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.411+0000 7f0e86974740 1 -- 172.21.15.7:0/493696175 shutdown_connections 2024-06-10T01:24:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.411+0000 7f0e86974740 1 --2- 172.21.15.7:0/493696175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55661959cb70 0x556619591490 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.411+0000 7f0e86974740 1 --2- 172.21.15.7:0/493696175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556619597a00 0x5566195a8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.411+0000 7f0e86974740 1 --2- 172.21.15.7:0/493696175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556619597010 0x556619590f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.411+0000 7f0e86974740 1 -- 172.21.15.7:0/493696175 >> 172.21.15.7:0/493696175 conn(0x556619592680 msgr2=0x556619592a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.414+0000 7f0e86974740 1 -- 172.21.15.7:0/493696175 shutdown_connections 2024-06-10T01:24:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.414+0000 7f0e86974740 1 -- 172.21.15.7:0/493696175 wait complete. 2024-06-10T01:24:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.415+0000 7f0e86974740 1 -- start start 2024-06-10T01:24:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.415+0000 7f0e86974740 1 Processor -- start 2024-06-10T01:24:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.415+0000 7f0e86974740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556619597010 0x5566195a5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.416+0000 7f0e86974740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556619597a00 0x5566195a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.416+0000 7f0e86974740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55661959cb70 0x556619615f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.416+0000 7f0e86974740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6800e3d0 con 0x556619597010 2024-06-10T01:24:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.416+0000 7f0e86974740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6800f840 con 0x55661959cb70 2024-06-10T01:24:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.416+0000 7f0e86974740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6800eaa0 con 0x556619597a00 2024-06-10T01:24:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.416+0000 7f0e86974740 10 client.0 _ll_register_callbacks cb 0x7f0e6c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:24:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.416+0000 7f0e811ff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556619597a00 0x5566195a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.416+0000 7f0e811ff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556619597a00 0x5566195a5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:39748/0 (socket says 172.21.15.7:39748) 2024-06-10T01:24:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.416+0000 7f0e811ff640 1 -- 172.21.15.7:0/3338180271 learned_addr learned my addr 172.21.15.7:0/3338180271 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.416+0000 7f0e81a00640 1 --2- 172.21.15.7:0/3338180271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556619597010 0x5566195a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.416+0000 7f0e811ff640 1 -- 172.21.15.7:0/3338180271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55661959cb70 msgr2=0x556619615f80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:24:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.416+0000 7f0e811ff640 1 --2- 172.21.15.7:0/3338180271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55661959cb70 0x556619615f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.416+0000 7f0e811ff640 1 -- 172.21.15.7:0/3338180271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556619597010 msgr2=0x5566195a5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.416+0000 7f0e811ff640 1 --2- 172.21.15.7:0/3338180271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556619597010 0x5566195a5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.416+0000 7f0e811ff640 1 -- 172.21.15.7:0/3338180271 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566196164f0 con 0x556619597a00 2024-06-10T01:24:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.417+0000 7f0e811ff640 1 --2- 172.21.15.7:0/3338180271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556619597a00 0x5566195a5cc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0e70096f50 tx=0x7f0e7009a570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffd77569b5610735 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.417+0000 7f0e617fa640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:24:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.417+0000 7f0e617fa640 1 -- 172.21.15.7:0/3338180271 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e700b8060 con 0x556619597a00 2024-06-10T01:24:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.417+0000 7f0e617fa640 1 -- 172.21.15.7:0/3338180271 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f0e700ce330 con 0x556619597a00 2024-06-10T01:24:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.417+0000 7f0e86974740 1 -- 172.21.15.7:0/3338180271 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e6c005520 con 0x556619597a00 2024-06-10T01:24:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.418+0000 7f0e617fa640 1 -- 172.21.15.7:0/3338180271 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f0e70015040 con 0x556619597a00 2024-06-10T01:24:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.418+0000 7f0e86974740 10 client.29128 fetch_fsmap learned FSMap version 50 2024-06-10T01:24:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.418+0000 7f0e86974740 1 -- 172.21.15.7:0/3338180271 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5566196178d0 con 0x556619597a00 2024-06-10T01:24:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.418+0000 7f0e617fa640 1 -- 172.21.15.7:0/3338180271 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e6c005520 con 0x556619597a00 2024-06-10T01:24:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.419+0000 7f0e617fa640 1 -- 172.21.15.7:0/3338180271 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7f0e700b9250 con 0x556619597a00 2024-06-10T01:24:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.419+0000 7f0e86974740 10 client.29128 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:24:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.419+0000 7f0e86974740 10 client.29128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:24:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.419+0000 7f0e86974740 20 client.29128 populate_metadata read hostname 'smithi007' 2024-06-10T01:24:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.419+0000 7f0e86974740 1 --2- 172.21.15.7:0/3338180271 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556619618260 0x55661961a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.419+0000 7f0e86974740 4 client.29128 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:24:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.419+0000 7f0e86974740 1 -- 172.21.15.7:0/3338180271 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556619507c20 con 0x556619618260 2024-06-10T01:24:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.419+0000 7f0e81a00640 1 --2- 172.21.15.7:0/3338180271 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556619618260 0x55661961a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.420+0000 7f0e617fa640 10 client.29128 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:16.420+0000 7f0e81a00640 1 --2- 172.21.15.7:0/3338180271 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556619618260 0x55661961a680 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=128383c3a67175a7 server_cookie=addb13585c3ca09b in_seq=0 out_seq=0 2024-06-10T01:24:16.535 INFO:tasks.workunit.client.0.smithi007.stdout: 10 109137 21.37 MB/sec execute 173 sec latency 99.139 ms 2024-06-10T01:24:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:17.146+0000 7f0e617fa640 1 -- 172.21.15.7:0/3338180271 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556619507c20 con 0x556619618260 2024-06-10T01:24:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:17.146+0000 7f0e617fa640 10 client.29128 handle_command_reply: tid=0 2024-06-10T01:24:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:17.146+0000 7f0e86974740 1 client.29128 shutdown 2024-06-10T01:24:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:17.146+0000 7f0e86974740 1 -- 172.21.15.7:0/3338180271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556619597a00 msgr2=0x5566195a5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:17.146+0000 7f0e86974740 1 --2- 172.21.15.7:0/3338180271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556619597a00 0x5566195a5cc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0e70096f50 tx=0x7f0e7009a570 comp rx=0 tx=0).stop 2024-06-10T01:24:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:17.146+0000 7f0e86974740 1 -- 172.21.15.7:0/3338180271 shutdown_connections 2024-06-10T01:24:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:17.146+0000 7f0e86974740 1 --2- 172.21.15.7:0/3338180271 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556619618260 0x55661961a680 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:17.146+0000 7f0e617fa640 0 client.29128 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:17.146+0000 7f0e86974740 1 --2- 172.21.15.7:0/3338180271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55661959cb70 0x556619615f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:17.146+0000 7f0e86974740 1 --2- 172.21.15.7:0/3338180271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556619597a00 0x5566195a5cc0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:17.146+0000 7f0e86974740 1 --2- 172.21.15.7:0/3338180271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556619597010 0x5566195a5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:17.146+0000 7f0e86974740 1 -- 172.21.15.7:0/3338180271 >> 172.21.15.7:0/3338180271 conn(0x556619592680 msgr2=0x556619595360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:17.146+0000 7f0e86974740 1 -- 172.21.15.7:0/3338180271 shutdown_connections 2024-06-10T01:24:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:17.146+0000 7f0e86974740 1 -- 172.21.15.7:0/3338180271 wait complete. 2024-06-10T01:24:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:17.147+0000 7f0e86974740 20 client.29128 trim_cache size 0 max 16384 2024-06-10T01:24:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:24:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:24:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:24:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:17.147+0000 7f0e83c64640 1 -- 172.21.15.7:0/2177987993 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0e640a2fc0 msgr2=0x7f0e640a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:17.147+0000 7f0e83c64640 1 --2- 172.21.15.7:0/2177987993 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0e640a2fc0 0x7f0e640a5480 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x556619598470 tx=0x7f0e74000f40 comp rx=0 tx=0).stop 2024-06-10T01:24:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:17.147+0000 7f0e83c64640 1 -- 172.21.15.7:0/2177987993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e7c110220 msgr2=0x7f0e7c109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:17.147+0000 7f0e83c64640 1 --2- 172.21.15.7:0/2177987993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e7c110220 0x7f0e7c109c70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0e70009fd0 tx=0x7f0e7000b870 comp rx=0 tx=0).stop 2024-06-10T01:24:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:17.147+0000 7f0e83c64640 1 -- 172.21.15.7:0/2177987993 shutdown_connections 2024-06-10T01:24:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:17.147+0000 7f0e83c64640 1 --2- 172.21.15.7:0/2177987993 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0e640a2fc0 0x7f0e640a5480 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:17.147+0000 7f0e83c64640 1 --2- 172.21.15.7:0/2177987993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e7c1a4440 0x7f0e7c10a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:17.147+0000 7f0e83c64640 1 --2- 172.21.15.7:0/2177987993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e7c110220 0x7f0e7c109c70 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:17.147+0000 7f0e83c64640 1 --2- 172.21.15.7:0/2177987993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e7c109300 0x7f0e7c10b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:17.147+0000 7f0e83c64640 1 -- 172.21.15.7:0/2177987993 >> 172.21.15.7:0/2177987993 conn(0x7f0e7c06e9e0 msgr2=0x7f0e7c070730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:17.147+0000 7f0e83c64640 1 -- 172.21.15.7:0/2177987993 shutdown_connections 2024-06-10T01:24:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:17.148+0000 7f0e83c64640 1 -- 172.21.15.7:0/2177987993 wait complete. 2024-06-10T01:24:17.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:16 smithi007 ceph-mon[32398]: pgmap v981: 129 pgs: 129 active+clean; 586 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.3 MiB/s wr, 609 op/s 2024-06-10T01:24:17.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:16 smithi078 ceph-mon[36683]: pgmap v981: 129 pgs: 129 active+clean; 586 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.3 MiB/s wr, 609 op/s 2024-06-10T01:24:17.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:16 smithi175 ceph-mon[36636]: pgmap v981: 129 pgs: 129 active+clean; 586 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.3 MiB/s wr, 609 op/s 2024-06-10T01:24:17.535 INFO:tasks.workunit.client.0.smithi007.stdout: 10 109531 21.35 MB/sec execute 174 sec latency 458.549 ms 2024-06-10T01:24:18.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:17 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:24:18.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:17 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:24:18.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:17 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:24:18.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:17 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:24:18.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:17 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:24:18.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:17 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:24:18.535 INFO:tasks.workunit.client.0.smithi007.stdout: 10 110226 21.39 MB/sec execute 175 sec latency 47.003 ms 2024-06-10T01:24:19.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:19 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:24:19.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:19 smithi007 ceph-mon[32398]: pgmap v982: 129 pgs: 129 active+clean; 591 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.0 MiB/s wr, 522 op/s 2024-06-10T01:24:19.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:19 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:24:19.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:19 smithi078 ceph-mon[36683]: pgmap v982: 129 pgs: 129 active+clean; 591 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.0 MiB/s wr, 522 op/s 2024-06-10T01:24:19.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:19 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:24:19.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:19 smithi175 ceph-mon[36636]: pgmap v982: 129 pgs: 129 active+clean; 591 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.0 MiB/s wr, 522 op/s 2024-06-10T01:24:19.535 INFO:tasks.workunit.client.0.smithi007.stdout: 10 111221 21.63 MB/sec execute 176 sec latency 48.720 ms 2024-06-10T01:24:20.536 INFO:tasks.workunit.client.0.smithi007.stdout: 10 112011 21.76 MB/sec execute 177 sec latency 81.932 ms 2024-06-10T01:24:21.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:21 smithi007 ceph-mon[32398]: pgmap v983: 129 pgs: 129 active+clean; 593 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.9 MiB/s wr, 732 op/s 2024-06-10T01:24:21.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:21 smithi078 ceph-mon[36683]: pgmap v983: 129 pgs: 129 active+clean; 593 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.9 MiB/s wr, 732 op/s 2024-06-10T01:24:21.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:21 smithi175 ceph-mon[36636]: pgmap v983: 129 pgs: 129 active+clean; 593 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.9 MiB/s wr, 732 op/s 2024-06-10T01:24:21.536 INFO:tasks.workunit.client.0.smithi007.stdout: 10 112676 21.83 MB/sec execute 178 sec latency 67.718 ms 2024-06-10T01:24:22.536 INFO:tasks.workunit.client.0.smithi007.stdout: 10 113309 21.86 MB/sec execute 179 sec latency 64.368 ms 2024-06-10T01:24:23.253 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:23 smithi175 ceph-mon[36636]: pgmap v984: 129 pgs: 129 active+clean; 593 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.1 MiB/s wr, 679 op/s 2024-06-10T01:24:23.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:23 smithi007 ceph-mon[32398]: pgmap v984: 129 pgs: 129 active+clean; 593 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.1 MiB/s wr, 679 op/s 2024-06-10T01:24:23.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:23 smithi078 ceph-mon[36683]: pgmap v984: 129 pgs: 129 active+clean; 593 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.1 MiB/s wr, 679 op/s 2024-06-10T01:24:23.536 INFO:tasks.workunit.client.0.smithi007.stdout: 10 114282 22.05 MB/sec execute 180 sec latency 54.949 ms 2024-06-10T01:24:24.536 INFO:tasks.workunit.client.0.smithi007.stdout: 10 115053 22.16 MB/sec execute 181 sec latency 64.359 ms 2024-06-10T01:24:25.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:25 smithi007 ceph-mon[32398]: pgmap v985: 129 pgs: 129 active+clean; 593 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.5 MiB/s wr, 611 op/s 2024-06-10T01:24:25.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:25 smithi078 ceph-mon[36683]: pgmap v985: 129 pgs: 129 active+clean; 593 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.5 MiB/s wr, 611 op/s 2024-06-10T01:24:25.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:25 smithi175 ceph-mon[36636]: pgmap v985: 129 pgs: 129 active+clean; 593 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.5 MiB/s wr, 611 op/s 2024-06-10T01:24:25.536 INFO:tasks.workunit.client.0.smithi007.stdout: 10 115704 22.25 MB/sec execute 182 sec latency 56.275 ms 2024-06-10T01:24:26.536 INFO:tasks.workunit.client.0.smithi007.stdout: 10 116368 22.31 MB/sec execute 183 sec latency 71.217 ms 2024-06-10T01:24:27.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:27 smithi007 ceph-mon[32398]: pgmap v986: 129 pgs: 129 active+clean; 610 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.0 MiB/s wr, 769 op/s 2024-06-10T01:24:27.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:27 smithi078 ceph-mon[36683]: pgmap v986: 129 pgs: 129 active+clean; 610 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.0 MiB/s wr, 769 op/s 2024-06-10T01:24:27.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:27 smithi175 ceph-mon[36636]: pgmap v986: 129 pgs: 129 active+clean; 610 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.0 MiB/s wr, 769 op/s 2024-06-10T01:24:27.536 INFO:tasks.workunit.client.0.smithi007.stdout: 10 117206 22.42 MB/sec execute 184 sec latency 88.847 ms 2024-06-10T01:24:28.536 INFO:tasks.workunit.client.0.smithi007.stdout: 10 117826 22.49 MB/sec execute 185 sec latency 59.353 ms 2024-06-10T01:24:29.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:29 smithi007 ceph-mon[32398]: pgmap v987: 129 pgs: 129 active+clean; 611 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 3.9 MiB/s wr, 608 op/s 2024-06-10T01:24:29.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:29 smithi078 ceph-mon[36683]: pgmap v987: 129 pgs: 129 active+clean; 611 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 3.9 MiB/s wr, 608 op/s 2024-06-10T01:24:29.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:29 smithi175 ceph-mon[36636]: pgmap v987: 129 pgs: 129 active+clean; 611 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 3.9 MiB/s wr, 608 op/s 2024-06-10T01:24:29.536 INFO:tasks.workunit.client.0.smithi007.stdout: 10 118686 22.58 MB/sec execute 186 sec latency 48.013 ms 2024-06-10T01:24:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.261+0000 7f60f00ba640 1 -- 172.21.15.7:0/1225987343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60e80759e0 msgr2=0x7f60e8073e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.261+0000 7f60f00ba640 1 --2- 172.21.15.7:0/1225987343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60e80759e0 0x7f60e8073e30 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f60e4009900 tx=0x7f60e402a410 comp rx=0 tx=0).stop 2024-06-10T01:24:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.261+0000 7f60f00ba640 1 -- 172.21.15.7:0/1225987343 shutdown_connections 2024-06-10T01:24:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.261+0000 7f60f00ba640 1 --2- 172.21.15.7:0/1225987343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60e81098d0 0x7f60e810bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.261+0000 7f60f00ba640 1 --2- 172.21.15.7:0/1225987343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60e8074400 0x7f60e8074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.261+0000 7f60f00ba640 1 --2- 172.21.15.7:0/1225987343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60e80759e0 0x7f60e8073e30 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.261+0000 7f60f00ba640 1 -- 172.21.15.7:0/1225987343 >> 172.21.15.7:0/1225987343 conn(0x7f60e80fcf80 msgr2=0x7f60e80ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.261+0000 7f60f00ba640 1 -- 172.21.15.7:0/1225987343 shutdown_connections 2024-06-10T01:24:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.261+0000 7f60f00ba640 1 -- 172.21.15.7:0/1225987343 wait complete. 2024-06-10T01:24:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.262+0000 7f60f00ba640 1 Processor -- start 2024-06-10T01:24:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.262+0000 7f60f00ba640 1 -- start start 2024-06-10T01:24:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.262+0000 7f60f00ba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60e8074400 0x7f60e8199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.262+0000 7f60f00ba640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60e80759e0 0x7f60e819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.262+0000 7f60f00ba640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60e81098d0 0x7f60e81a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.262+0000 7f60f00ba640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f60e810f870 con 0x7f60e81098d0 2024-06-10T01:24:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.262+0000 7f60f00ba640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f60e810f6f0 con 0x7f60e8074400 2024-06-10T01:24:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.262+0000 7f60f00ba640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f60e810f9f0 con 0x7f60e80759e0 2024-06-10T01:24:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.262+0000 7f60ee657640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60e81098d0 0x7f60e81a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.262+0000 7f60ede56640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60e8074400 0x7f60e8199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.262+0000 7f60ee657640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60e81098d0 0x7f60e81a1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39110/0 (socket says 172.21.15.7:39110) 2024-06-10T01:24:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.262+0000 7f60ee657640 1 -- 172.21.15.7:0/3728188825 learned_addr learned my addr 172.21.15.7:0/3728188825 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.262+0000 7f60ed655640 1 --2- 172.21.15.7:0/3728188825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60e80759e0 0x7f60e819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.263+0000 7f60ede56640 1 -- 172.21.15.7:0/3728188825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60e80759e0 msgr2=0x7f60e819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.263+0000 7f60ede56640 1 --2- 172.21.15.7:0/3728188825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60e80759e0 0x7f60e819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.263+0000 7f60ede56640 1 -- 172.21.15.7:0/3728188825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60e81098d0 msgr2=0x7f60e81a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.263+0000 7f60ede56640 1 --2- 172.21.15.7:0/3728188825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60e81098d0 0x7f60e81a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.263+0000 7f60ede56640 1 -- 172.21.15.7:0/3728188825 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e81a1b70 con 0x7f60e8074400 2024-06-10T01:24:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.263+0000 7f60ee657640 1 --2- 172.21.15.7:0/3728188825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60e81098d0 0x7f60e81a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:24:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.263+0000 7f60ede56640 1 --2- 172.21.15.7:0/3728188825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60e8074400 0x7f60e8199fa0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f60e4009230 tx=0x7f60e40042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.263+0000 7f60deffd640 1 -- 172.21.15.7:0/3728188825 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e403f030 con 0x7f60e8074400 2024-06-10T01:24:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.263+0000 7f60deffd640 1 -- 172.21.15.7:0/3728188825 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60e4031a20 con 0x7f60e8074400 2024-06-10T01:24:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.263+0000 7f60f00ba640 1 -- 172.21.15.7:0/3728188825 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e81a1e00 con 0x7f60e8074400 2024-06-10T01:24:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.263+0000 7f60deffd640 1 -- 172.21.15.7:0/3728188825 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e4035070 con 0x7f60e8074400 2024-06-10T01:24:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.263+0000 7f60f00ba640 1 -- 172.21.15.7:0/3728188825 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e81a2260 con 0x7f60e8074400 2024-06-10T01:24:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.266+0000 7f60deffd640 1 -- 172.21.15.7:0/3728188825 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f60e4042030 con 0x7f60e8074400 2024-06-10T01:24:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.267+0000 7f60f2dc7740 1 Processor -- start 2024-06-10T01:24:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.267+0000 7f60f2dc7740 1 -- start start 2024-06-10T01:24:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.267+0000 7f60f2dc7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0d9855010 0x55a0d9855430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.267+0000 7f60f2dc7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0d9855a00 0x55a0d985a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.267+0000 7f60f2dc7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d985ab70 0x55a0d985d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.267+0000 7f60f2dc7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a0d97b84f0 con 0x55a0d9855010 2024-06-10T01:24:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.267+0000 7f60f2dc7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a0d9756870 con 0x55a0d985ab70 2024-06-10T01:24:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.267+0000 7f60f2dc7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a0d97b44b0 con 0x55a0d9855a00 2024-06-10T01:24:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.268+0000 7f60ede56640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0d9855a00 0x55a0d985a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.268+0000 7f60ee657640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0d9855010 0x55a0d9855430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.268+0000 7f60ee657640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0d9855010 0x55a0d9855430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39126/0 (socket says 172.21.15.7:39126) 2024-06-10T01:24:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.268+0000 7f60ee657640 1 -- 172.21.15.7:0/3202470012 learned_addr learned my addr 172.21.15.7:0/3202470012 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.268+0000 7f60ed655640 1 --2- 172.21.15.7:0/3202470012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d985ab70 0x55a0d985d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.268+0000 7f60deffd640 1 --2- 172.21.15.7:0/3728188825 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f60d40a2c00 0x7f60d40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.268+0000 7f60ee657640 1 -- 172.21.15.7:0/3202470012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0d9855a00 msgr2=0x55a0d985a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.268+0000 7f60ee657640 1 --2- 172.21.15.7:0/3202470012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0d9855a00 0x55a0d985a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.268+0000 7f60ee657640 1 -- 172.21.15.7:0/3202470012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d985ab70 msgr2=0x55a0d985d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.268+0000 7f60ee657640 1 --2- 172.21.15.7:0/3202470012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d985ab70 0x55a0d985d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.268+0000 7f60ee657640 1 -- 172.21.15.7:0/3202470012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0d97729b0 con 0x55a0d9855010 2024-06-10T01:24:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.268+0000 7f60ed655640 1 --2- 172.21.15.7:0/3202470012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d985ab70 0x55a0d985d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:24:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.268+0000 7f60ee657640 1 --2- 172.21.15.7:0/3728188825 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f60d40a2c00 0x7f60d40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.268+0000 7f60deffd640 1 -- 172.21.15.7:0/3728188825 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f60e40ef930 con 0x7f60e8074400 2024-06-10T01:24:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.268+0000 7f60ee657640 1 --2- 172.21.15.7:0/3202470012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0d9855010 0x55a0d9855430 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f60e0011a30 tx=0x7f60e0011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4347a1cd1dde2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.269+0000 7f60d3fff640 1 -- 172.21.15.7:0/3202470012 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e0017070 con 0x55a0d9855010 2024-06-10T01:24:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.269+0000 7f60d3fff640 1 -- 172.21.15.7:0/3202470012 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60e0010910 con 0x55a0d9855010 2024-06-10T01:24:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.269+0000 7f60d3fff640 1 -- 172.21.15.7:0/3202470012 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e000e6d0 con 0x55a0d9855010 2024-06-10T01:24:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.269+0000 7f60ee657640 1 --2- 172.21.15.7:0/3728188825 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f60d40a2c00 0x7f60d40a50c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f60e0023000 tx=0x7f60e0024000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.269+0000 7f60f2dc7740 1 -- 172.21.15.7:0/3202470012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0d9855010 msgr2=0x55a0d9855430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.269+0000 7f60f2dc7740 1 --2- 172.21.15.7:0/3202470012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0d9855010 0x55a0d9855430 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f60e0011a30 tx=0x7f60e0011ef0 comp rx=0 tx=0).stop 2024-06-10T01:24:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.270+0000 7f60f2dc7740 1 -- 172.21.15.7:0/3202470012 shutdown_connections 2024-06-10T01:24:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.270+0000 7f60f2dc7740 1 --2- 172.21.15.7:0/3202470012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d985ab70 0x55a0d985d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.270+0000 7f60f2dc7740 1 --2- 172.21.15.7:0/3202470012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0d9855a00 0x55a0d985a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.270+0000 7f60f2dc7740 1 --2- 172.21.15.7:0/3202470012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0d9855010 0x55a0d9855430 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.270+0000 7f60f2dc7740 1 -- 172.21.15.7:0/3202470012 >> 172.21.15.7:0/3202470012 conn(0x55a0d9850680 msgr2=0x55a0d9850a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.270+0000 7f60f2dc7740 1 -- 172.21.15.7:0/3202470012 shutdown_connections 2024-06-10T01:24:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.270+0000 7f60f2dc7740 1 -- 172.21.15.7:0/3202470012 wait complete. 2024-06-10T01:24:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.270+0000 7f60f2dc7740 1 -- start start 2024-06-10T01:24:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.270+0000 7f60f2dc7740 1 Processor -- start 2024-06-10T01:24:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.271+0000 7f60f2dc7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d9855010 0x55a0d98679e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.271+0000 7f60f2dc7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0d9855a00 0x55a0d9867f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.271+0000 7f60ede56640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d9855010 0x55a0d98679e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.271+0000 7f60ede56640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d9855010 0x55a0d98679e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:59994/0 (socket says 172.21.15.7:59994) 2024-06-10T01:24:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.271+0000 7f60ede56640 1 -- 172.21.15.7:0/3562068531 learned_addr learned my addr 172.21.15.7:0/3562068531 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.271+0000 7f60f2dc7740 1 --2- 172.21.15.7:0/3562068531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0d985ab70 0x55a0d986ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.271+0000 7f60f2dc7740 1 -- 172.21.15.7:0/3562068531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a0d985db20 con 0x55a0d985ab70 2024-06-10T01:24:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.271+0000 7f60f2dc7740 1 -- 172.21.15.7:0/3562068531 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a0d9741ae0 con 0x55a0d9855010 2024-06-10T01:24:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.271+0000 7f60f2dc7740 1 -- 172.21.15.7:0/3562068531 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a0d985dca0 con 0x55a0d9855a00 2024-06-10T01:24:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.271+0000 7f60ed655640 1 --2- 172.21.15.7:0/3562068531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0d9855a00 0x55a0d9867f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.271+0000 7f60f2dc7740 10 client.0 _ll_register_callbacks cb 0x55a0d977a9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:24:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.272+0000 7f60ee657640 1 --2- 172.21.15.7:0/3562068531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0d985ab70 0x55a0d986ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.272+0000 7f60ee657640 1 -- 172.21.15.7:0/3562068531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0d9855a00 msgr2=0x55a0d9867f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.272+0000 7f60ee657640 1 --2- 172.21.15.7:0/3562068531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0d9855a00 0x55a0d9867f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.272+0000 7f60ee657640 1 -- 172.21.15.7:0/3562068531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d9855010 msgr2=0x55a0d98679e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.272+0000 7f60ee657640 1 --2- 172.21.15.7:0/3562068531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d9855010 0x55a0d98679e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.272+0000 7f60ee657640 1 -- 172.21.15.7:0/3562068531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0d94cc210 con 0x55a0d985ab70 2024-06-10T01:24:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.273+0000 7f60ee657640 1 --2- 172.21.15.7:0/3562068531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0d985ab70 0x55a0d986ef60 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f60e0014040 tx=0x7f60e0009a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fecf5fc545f79719 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.273+0000 7f60d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:24:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.273+0000 7f60d17fa640 1 -- 172.21.15.7:0/3562068531 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e0017070 con 0x55a0d985ab70 2024-06-10T01:24:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.274+0000 7f60d17fa640 1 -- 172.21.15.7:0/3562068531 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f60e0010640 con 0x55a0d985ab70 2024-06-10T01:24:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.274+0000 7f60f2dc7740 1 -- 172.21.15.7:0/3562068531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60e0010910 con 0x55a0d985ab70 2024-06-10T01:24:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.275+0000 7f60d17fa640 1 -- 172.21.15.7:0/3562068531 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f60e0037050 con 0x55a0d985ab70 2024-06-10T01:24:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.276+0000 7f60f2dc7740 10 client.19977 fetch_fsmap learned FSMap version 50 2024-06-10T01:24:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.276+0000 7f60f2dc7740 1 -- 172.21.15.7:0/3562068531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55a0d9598620 con 0x55a0d985ab70 2024-06-10T01:24:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.277+0000 7f60d17fa640 1 -- 172.21.15.7:0/3562068531 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60e000e9f0 con 0x55a0d985ab70 2024-06-10T01:24:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.277+0000 7f60d17fa640 1 -- 172.21.15.7:0/3562068531 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7f60e003a020 con 0x55a0d985ab70 2024-06-10T01:24:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.277+0000 7f60f2dc7740 10 client.19977 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:24:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.277+0000 7f60f2dc7740 10 client.19977 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:24:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.277+0000 7f60f2dc7740 20 client.19977 populate_metadata read hostname 'smithi007' 2024-06-10T01:24:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.278+0000 7f60f2dc7740 1 --2- 172.21.15.7:0/3562068531 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a0d98736b0 0x55a0d9875aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.278+0000 7f60f2dc7740 4 client.19977 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:24:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.278+0000 7f60f2dc7740 1 -- 172.21.15.7:0/3562068531 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f60e0010910 con 0x55a0d98736b0 2024-06-10T01:24:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.278+0000 7f60ede56640 1 --2- 172.21.15.7:0/3562068531 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a0d98736b0 0x55a0d9875aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.279+0000 7f60ede56640 1 --2- 172.21.15.7:0/3562068531 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a0d98736b0 0x55a0d9875aa0 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18b15545e1e5d337 server_cookie=9a32d7e5172f00e8 in_seq=0 out_seq=0 2024-06-10T01:24:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.279+0000 7f60d17fa640 10 client.19977 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.281+0000 7f60d17fa640 1 -- 172.21.15.7:0/3562068531 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f60e0010910 con 0x55a0d98736b0 2024-06-10T01:24:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.281+0000 7f60d17fa640 10 client.19977 handle_command_reply: tid=0 2024-06-10T01:24:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.281+0000 7f60f2dc7740 1 client.19977 shutdown 2024-06-10T01:24:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.282+0000 7f60f2dc7740 1 -- 172.21.15.7:0/3562068531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0d985ab70 msgr2=0x55a0d986ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.282+0000 7f60f2dc7740 1 --2- 172.21.15.7:0/3562068531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0d985ab70 0x55a0d986ef60 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f60e0014040 tx=0x7f60e0009a30 comp rx=0 tx=0).stop 2024-06-10T01:24:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.282+0000 7f60f2dc7740 1 -- 172.21.15.7:0/3562068531 shutdown_connections 2024-06-10T01:24:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.283+0000 7f60f2dc7740 1 --2- 172.21.15.7:0/3562068531 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a0d98736b0 0x55a0d9875aa0 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.283+0000 7f60f2dc7740 1 --2- 172.21.15.7:0/3562068531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0d985ab70 0x55a0d986ef60 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.283+0000 7f60f2dc7740 1 --2- 172.21.15.7:0/3562068531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0d9855a00 0x55a0d9867f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.283+0000 7f60f2dc7740 1 --2- 172.21.15.7:0/3562068531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d9855010 0x55a0d98679e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.283+0000 7f60f2dc7740 1 -- 172.21.15.7:0/3562068531 >> 172.21.15.7:0/3562068531 conn(0x55a0d9850680 msgr2=0x55a0d984f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.283+0000 7f60d17fa640 0 client.19977 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.283+0000 7f60f2dc7740 1 -- 172.21.15.7:0/3562068531 shutdown_connections 2024-06-10T01:24:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.284+0000 7f60f2dc7740 1 -- 172.21.15.7:0/3562068531 wait complete. 2024-06-10T01:24:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.284+0000 7f60f2dc7740 20 client.19977 trim_cache size 0 max 16384 2024-06-10T01:24:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.314+0000 7f60f2dc7740 1 Processor -- start 2024-06-10T01:24:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.314+0000 7f60f2dc7740 1 -- start start 2024-06-10T01:24:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.315+0000 7f60f2dc7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0d9855010 0x55a0d98679e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.315+0000 7f60f2dc7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0d9855a00 0x55a0d9867f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.315+0000 7f60ede56640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0d9855010 0x55a0d98679e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.315+0000 7f60ede56640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0d9855010 0x55a0d98679e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:39238/0 (socket says 172.21.15.7:39238) 2024-06-10T01:24:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.315+0000 7f60ed655640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0d9855a00 0x55a0d9867f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.315+0000 7f60ed655640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0d9855a00 0x55a0d9867f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39154/0 (socket says 172.21.15.7:39154) 2024-06-10T01:24:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.315+0000 7f60ede56640 1 -- 172.21.15.7:0/2152210530 learned_addr learned my addr 172.21.15.7:0/2152210530 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.315+0000 7f60f2dc7740 1 --2- 172.21.15.7:0/2152210530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d985ab70 0x55a0d986ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.315+0000 7f60f2dc7740 1 -- 172.21.15.7:0/2152210530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f60e81aad20 con 0x55a0d9855a00 2024-06-10T01:24:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.315+0000 7f60f2dc7740 1 -- 172.21.15.7:0/2152210530 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f60e81a85f0 con 0x55a0d985ab70 2024-06-10T01:24:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.315+0000 7f60f2dc7740 1 -- 172.21.15.7:0/2152210530 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f60e81a6440 con 0x55a0d9855010 2024-06-10T01:24:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.316+0000 7f60ee657640 1 --2- 172.21.15.7:0/2152210530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d985ab70 0x55a0d986ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.316+0000 7f60ee657640 1 -- 172.21.15.7:0/2152210530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0d9855010 msgr2=0x55a0d98679e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.316+0000 7f60ee657640 1 --2- 172.21.15.7:0/2152210530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0d9855010 0x55a0d98679e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.316+0000 7f60ee657640 1 -- 172.21.15.7:0/2152210530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0d9855a00 msgr2=0x55a0d9867f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.316+0000 7f60ee657640 1 --2- 172.21.15.7:0/2152210530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0d9855a00 0x55a0d9867f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.316+0000 7f60ee657640 1 -- 172.21.15.7:0/2152210530 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0d9851350 con 0x55a0d985ab70 2024-06-10T01:24:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.316+0000 7f60ee657640 1 --2- 172.21.15.7:0/2152210530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d985ab70 0x55a0d986ef20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f60e00096f0 tx=0x7f60e0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d422b6c615939152 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:30.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.317+0000 7f60d37fe640 1 -- 172.21.15.7:0/2152210530 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e0017040 con 0x55a0d985ab70 2024-06-10T01:24:30.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.317+0000 7f60d37fe640 1 -- 172.21.15.7:0/2152210530 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60e001ca00 con 0x55a0d985ab70 2024-06-10T01:24:30.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.317+0000 7f60d37fe640 1 -- 172.21.15.7:0/2152210530 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e0010640 con 0x55a0d985ab70 2024-06-10T01:24:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.318+0000 7f60f2dc7740 1 -- 172.21.15.7:0/2152210530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d985ab70 msgr2=0x55a0d986ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.318+0000 7f60f2dc7740 1 --2- 172.21.15.7:0/2152210530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d985ab70 0x55a0d986ef20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f60e00096f0 tx=0x7f60e0011810 comp rx=0 tx=0).stop 2024-06-10T01:24:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.318+0000 7f60f2dc7740 1 -- 172.21.15.7:0/2152210530 shutdown_connections 2024-06-10T01:24:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.318+0000 7f60f2dc7740 1 --2- 172.21.15.7:0/2152210530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d985ab70 0x55a0d986ef20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.318+0000 7f60f2dc7740 1 --2- 172.21.15.7:0/2152210530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0d9855a00 0x55a0d9867f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.318+0000 7f60f2dc7740 1 --2- 172.21.15.7:0/2152210530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0d9855010 0x55a0d98679e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.318+0000 7f60f2dc7740 1 -- 172.21.15.7:0/2152210530 >> 172.21.15.7:0/2152210530 conn(0x55a0d9850680 msgr2=0x55a0d984b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.318+0000 7f60f2dc7740 1 -- 172.21.15.7:0/2152210530 shutdown_connections 2024-06-10T01:24:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.318+0000 7f60f2dc7740 1 -- 172.21.15.7:0/2152210530 wait complete. 2024-06-10T01:24:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.319+0000 7f60f2dc7740 1 -- start start 2024-06-10T01:24:30.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.319+0000 7f60f2dc7740 1 Processor -- start 2024-06-10T01:24:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.320+0000 7f60f2dc7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0d9855010 0x55a0d986ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.320+0000 7f60f2dc7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d9855a00 0x55a0d986b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.320+0000 7f60f2dc7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0d985ab70 0x55a0d986e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.320+0000 7f60f2dc7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f60e81a6ab0 con 0x55a0d985ab70 2024-06-10T01:24:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.320+0000 7f60f2dc7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f60e81a7f20 con 0x55a0d9855a00 2024-06-10T01:24:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.320+0000 7f60f2dc7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f60e81a7180 con 0x55a0d9855010 2024-06-10T01:24:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.320+0000 7f60f2dc7740 10 client.0 _ll_register_callbacks cb 0x7f60e00118e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:24:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.320+0000 7f60ed655640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d9855a00 0x55a0d986b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.320+0000 7f60ed655640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d9855a00 0x55a0d986b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:60010/0 (socket says 172.21.15.7:60010) 2024-06-10T01:24:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.320+0000 7f60ed655640 1 -- 172.21.15.7:0/947002886 learned_addr learned my addr 172.21.15.7:0/947002886 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.320+0000 7f60ede56640 1 --2- 172.21.15.7:0/947002886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0d9855010 0x55a0d986ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.320+0000 7f60ee657640 1 --2- 172.21.15.7:0/947002886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0d985ab70 0x55a0d986e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.320+0000 7f60ed655640 1 -- 172.21.15.7:0/947002886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0d9855010 msgr2=0x55a0d986ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.320+0000 7f60ed655640 1 --2- 172.21.15.7:0/947002886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0d9855010 0x55a0d986ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.320+0000 7f60ed655640 1 -- 172.21.15.7:0/947002886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0d985ab70 msgr2=0x55a0d986e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.320+0000 7f60ed655640 1 --2- 172.21.15.7:0/947002886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0d985ab70 0x55a0d986e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.320+0000 7f60ed655640 1 -- 172.21.15.7:0/947002886 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0d986e970 con 0x55a0d9855a00 2024-06-10T01:24:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.321+0000 7f60ee657640 1 --2- 172.21.15.7:0/947002886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0d985ab70 0x55a0d986e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:24:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.321+0000 7f60ed655640 1 --2- 172.21.15.7:0/947002886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d9855a00 0x55a0d986b3d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f60d8012910 tx=0x7f60d8012dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7414a45e1e41366 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.321+0000 7f60d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:24:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.322+0000 7f60d17fa640 1 -- 172.21.15.7:0/947002886 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d8022020 con 0x55a0d9855a00 2024-06-10T01:24:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.322+0000 7f60d17fa640 1 -- 172.21.15.7:0/947002886 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f60d800fbe0 con 0x55a0d9855a00 2024-06-10T01:24:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.322+0000 7f60f2dc7740 1 -- 172.21.15.7:0/947002886 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60e001ca00 con 0x55a0d9855a00 2024-06-10T01:24:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.323+0000 7f60d17fa640 1 -- 172.21.15.7:0/947002886 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f60d800fde0 con 0x55a0d9855a00 2024-06-10T01:24:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.323+0000 7f60f2dc7740 10 client.29189 fetch_fsmap learned FSMap version 50 2024-06-10T01:24:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.323+0000 7f60f2dc7740 1 -- 172.21.15.7:0/947002886 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55a0d98dcb80 con 0x55a0d9855a00 2024-06-10T01:24:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.323+0000 7f60d17fa640 1 -- 172.21.15.7:0/947002886 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60e001ca00 con 0x55a0d9855a00 2024-06-10T01:24:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.323+0000 7f60d17fa640 1 -- 172.21.15.7:0/947002886 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 50) ==== 8354+0+0 (secure 0 0 0) 0x7f60d80105f0 con 0x55a0d9855a00 2024-06-10T01:24:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.323+0000 7f60f2dc7740 10 client.29189 fetch_fsmap finished waiting for FSMap version 50 2024-06-10T01:24:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.323+0000 7f60f2dc7740 10 client.29189 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:24:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.323+0000 7f60f2dc7740 20 client.29189 populate_metadata read hostname 'smithi007' 2024-06-10T01:24:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.323+0000 7f60f2dc7740 1 --2- 172.21.15.7:0/947002886 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a0d98dd580 0x55a0d98df9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.323+0000 7f60f2dc7740 4 client.29189 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:24:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.323+0000 7f60f2dc7740 1 -- 172.21.15.7:0/947002886 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a0d97cbf10 con 0x55a0d98dd580 2024-06-10T01:24:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.325+0000 7f60ede56640 1 --2- 172.21.15.7:0/947002886 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a0d98dd580 0x55a0d98df9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:30.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.326+0000 7f60ede56640 1 --2- 172.21.15.7:0/947002886 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a0d98dd580 0x55a0d98df9a0 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36c2d20385c93afb server_cookie=19e020ed2ba36959 in_seq=0 out_seq=0 2024-06-10T01:24:30.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:30.326+0000 7f60d17fa640 10 client.29189 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:30.536 INFO:tasks.workunit.client.0.smithi007.stdout: 10 119429 22.67 MB/sec execute 187 sec latency 114.408 ms 2024-06-10T01:24:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:31.089+0000 7f60d17fa640 1 -- 172.21.15.7:0/947002886 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a0d97cbf10 con 0x55a0d98dd580 2024-06-10T01:24:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:31.089+0000 7f60d17fa640 10 client.29189 handle_command_reply: tid=0 2024-06-10T01:24:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:31.089+0000 7f60f2dc7740 1 client.29189 shutdown 2024-06-10T01:24:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:31.089+0000 7f60f2dc7740 1 -- 172.21.15.7:0/947002886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d9855a00 msgr2=0x55a0d986b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:31.089+0000 7f60f2dc7740 1 --2- 172.21.15.7:0/947002886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d9855a00 0x55a0d986b3d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f60d8012910 tx=0x7f60d8012dd0 comp rx=0 tx=0).stop 2024-06-10T01:24:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:31.090+0000 7f60f2dc7740 1 -- 172.21.15.7:0/947002886 shutdown_connections 2024-06-10T01:24:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:31.090+0000 7f60f2dc7740 1 --2- 172.21.15.7:0/947002886 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a0d98dd580 0x55a0d98df9a0 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:31.090+0000 7f60f2dc7740 1 --2- 172.21.15.7:0/947002886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0d985ab70 0x55a0d986e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:31.090+0000 7f60f2dc7740 1 --2- 172.21.15.7:0/947002886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d9855a00 0x55a0d986b3d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:31.090+0000 7f60f2dc7740 1 --2- 172.21.15.7:0/947002886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0d9855010 0x55a0d986ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:31.090+0000 7f60d17fa640 0 client.29189 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:31.090+0000 7f60f2dc7740 1 -- 172.21.15.7:0/947002886 >> 172.21.15.7:0/947002886 conn(0x55a0d9850680 msgr2=0x55a0d98679e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:31.090+0000 7f60f2dc7740 1 -- 172.21.15.7:0/947002886 shutdown_connections 2024-06-10T01:24:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:31.090+0000 7f60f2dc7740 1 -- 172.21.15.7:0/947002886 wait complete. 2024-06-10T01:24:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:31.090+0000 7f60f2dc7740 20 client.29189 trim_cache size 0 max 16384 2024-06-10T01:24:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:24:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:24:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:24:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:31.091+0000 7f60f00ba640 1 -- 172.21.15.7:0/3728188825 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f60d40a2c00 msgr2=0x7f60d40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:31.091+0000 7f60f00ba640 1 --2- 172.21.15.7:0/3728188825 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f60d40a2c00 0x7f60d40a50c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f60e0023000 tx=0x7f60e0024000 comp rx=0 tx=0).stop 2024-06-10T01:24:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:31.091+0000 7f60f00ba640 1 -- 172.21.15.7:0/3728188825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60e8074400 msgr2=0x7f60e8199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:31.091+0000 7f60f00ba640 1 --2- 172.21.15.7:0/3728188825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60e8074400 0x7f60e8199fa0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f60e4009230 tx=0x7f60e40042f0 comp rx=0 tx=0).stop 2024-06-10T01:24:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:31.091+0000 7f60f00ba640 1 -- 172.21.15.7:0/3728188825 shutdown_connections 2024-06-10T01:24:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:31.091+0000 7f60f00ba640 1 --2- 172.21.15.7:0/3728188825 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f60d40a2c00 0x7f60d40a50c0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:31.091+0000 7f60f00ba640 1 --2- 172.21.15.7:0/3728188825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60e81098d0 0x7f60e81a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:31.091+0000 7f60f00ba640 1 --2- 172.21.15.7:0/3728188825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60e80759e0 0x7f60e819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:31.091+0000 7f60f00ba640 1 --2- 172.21.15.7:0/3728188825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60e8074400 0x7f60e8199fa0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:31.091+0000 7f60f00ba640 1 -- 172.21.15.7:0/3728188825 >> 172.21.15.7:0/3728188825 conn(0x7f60e80fcf80 msgr2=0x7f60e81057d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:31.091+0000 7f60f00ba640 1 -- 172.21.15.7:0/3728188825 shutdown_connections 2024-06-10T01:24:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:31.091+0000 7f60f00ba640 1 -- 172.21.15.7:0/3728188825 wait complete. 2024-06-10T01:24:31.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:31 smithi007 ceph-mon[32398]: pgmap v988: 129 pgs: 129 active+clean; 624 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.8 MiB/s wr, 701 op/s 2024-06-10T01:24:31.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:31 smithi078 ceph-mon[36683]: pgmap v988: 129 pgs: 129 active+clean; 624 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.8 MiB/s wr, 701 op/s 2024-06-10T01:24:31.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:31 smithi175 ceph-mon[36636]: pgmap v988: 129 pgs: 129 active+clean; 624 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.8 MiB/s wr, 701 op/s 2024-06-10T01:24:31.536 INFO:tasks.workunit.client.0.smithi007.stdout: 10 119877 22.72 MB/sec execute 188 sec latency 451.782 ms 2024-06-10T01:24:32.536 INFO:tasks.workunit.client.0.smithi007.stdout: 10 119929 22.63 MB/sec execute 189 sec latency 849.986 ms 2024-06-10T01:24:33.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:33 smithi078 ceph-mon[36683]: pgmap v989: 129 pgs: 129 active+clean; 632 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.5 MiB/s wr, 571 op/s 2024-06-10T01:24:33.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:33 smithi175 ceph-mon[36636]: pgmap v989: 129 pgs: 129 active+clean; 632 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.5 MiB/s wr, 571 op/s 2024-06-10T01:24:33.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:33 smithi007 ceph-mon[32398]: pgmap v989: 129 pgs: 129 active+clean; 632 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.5 MiB/s wr, 571 op/s 2024-06-10T01:24:33.536 INFO:tasks.workunit.client.0.smithi007.stdout: 10 120169 22.61 MB/sec execute 190 sec latency 1230.004 ms 2024-06-10T01:24:34.536 INFO:tasks.workunit.client.0.smithi007.stdout: 10 120520 22.58 MB/sec execute 191 sec latency 236.373 ms 2024-06-10T01:24:35.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:35 smithi078 ceph-mon[36683]: pgmap v990: 129 pgs: 129 active+clean; 634 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 4.3 MiB/s wr, 497 op/s 2024-06-10T01:24:35.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:35 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:24:35.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:35 smithi175 ceph-mon[36636]: pgmap v990: 129 pgs: 129 active+clean; 634 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 4.3 MiB/s wr, 497 op/s 2024-06-10T01:24:35.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:35 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:24:35.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:35 smithi007 ceph-mon[32398]: pgmap v990: 129 pgs: 129 active+clean; 634 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 4.3 MiB/s wr, 497 op/s 2024-06-10T01:24:35.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:35 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:24:35.536 INFO:tasks.workunit.client.0.smithi007.stdout: 10 120800 22.52 MB/sec execute 192 sec latency 160.999 ms 2024-06-10T01:24:36.536 INFO:tasks.workunit.client.0.smithi007.stdout: 10 121422 22.56 MB/sec execute 193 sec latency 55.625 ms 2024-06-10T01:24:37.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:37 smithi078 ceph-mon[36683]: pgmap v991: 129 pgs: 129 active+clean; 638 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 876 op/s 2024-06-10T01:24:37.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:37 smithi175 ceph-mon[36636]: pgmap v991: 129 pgs: 129 active+clean; 638 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 876 op/s 2024-06-10T01:24:37.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:37 smithi007 ceph-mon[32398]: pgmap v991: 129 pgs: 129 active+clean; 638 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 876 op/s 2024-06-10T01:24:37.536 INFO:tasks.workunit.client.0.smithi007.stdout: 10 121450 22.45 MB/sec execute 194 sec latency 996.110 ms 2024-06-10T01:24:38.536 INFO:tasks.workunit.client.0.smithi007.stdout: 10 121563 22.36 MB/sec execute 195 sec latency 1284.746 ms 2024-06-10T01:24:39.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:39 smithi078 ceph-mon[36683]: pgmap v992: 129 pgs: 129 active+clean; 633 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 11 MiB/s wr, 775 op/s 2024-06-10T01:24:39.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:39 smithi175 ceph-mon[36636]: pgmap v992: 129 pgs: 129 active+clean; 633 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 11 MiB/s wr, 775 op/s 2024-06-10T01:24:39.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:39 smithi007 ceph-mon[32398]: pgmap v992: 129 pgs: 129 active+clean; 633 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 11 MiB/s wr, 775 op/s 2024-06-10T01:24:39.536 INFO:tasks.workunit.client.0.smithi007.stdout: 10 121874 22.33 MB/sec execute 196 sec latency 217.365 ms 2024-06-10T01:24:40.536 INFO:tasks.workunit.client.0.smithi007.stdout: 10 122224 22.32 MB/sec execute 197 sec latency 130.350 ms 2024-06-10T01:24:41.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:41 smithi078 ceph-mon[36683]: pgmap v993: 129 pgs: 129 active+clean; 630 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-06-10T01:24:41.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:41 smithi175 ceph-mon[36636]: pgmap v993: 129 pgs: 129 active+clean; 630 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-06-10T01:24:41.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:41 smithi007 ceph-mon[32398]: pgmap v993: 129 pgs: 129 active+clean; 630 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-06-10T01:24:41.536 INFO:tasks.workunit.client.0.smithi007.stdout: 10 123062 22.42 MB/sec execute 198 sec latency 93.888 ms 2024-06-10T01:24:42.536 INFO:tasks.workunit.client.0.smithi007.stdout: 10 123982 22.56 MB/sec execute 199 sec latency 47.692 ms 2024-06-10T01:24:43.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:43 smithi078 ceph-mon[36683]: pgmap v994: 129 pgs: 129 active+clean; 629 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 18 MiB/s wr, 971 op/s 2024-06-10T01:24:43.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:43 smithi175 ceph-mon[36636]: pgmap v994: 129 pgs: 129 active+clean; 629 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 18 MiB/s wr, 971 op/s 2024-06-10T01:24:43.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:43 smithi007 ceph-mon[32398]: pgmap v994: 129 pgs: 129 active+clean; 629 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 18 MiB/s wr, 971 op/s 2024-06-10T01:24:43.536 INFO:tasks.workunit.client.0.smithi007.stdout: 10 124738 22.67 MB/sec execute 200 sec latency 44.529 ms 2024-06-10T01:24:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.194+0000 7fa925109640 1 -- 172.21.15.7:0/2625524840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9201024b0 msgr2=0x7fa920108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.194+0000 7fa925109640 1 --2- 172.21.15.7:0/2625524840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9201024b0 0x7fa920108d40 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fa918009ca0 tx=0x7fa91802a8c0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.194+0000 7fa925109640 1 -- 172.21.15.7:0/2625524840 shutdown_connections 2024-06-10T01:24:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.194+0000 7fa925109640 1 --2- 172.21.15.7:0/2625524840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9201024b0 0x7fa920108d40 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.194+0000 7fa925109640 1 --2- 172.21.15.7:0/2625524840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa920101af0 0x7fa920101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.194+0000 7fa925109640 1 --2- 172.21.15.7:0/2625524840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa920100900 0x7fa920100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.194+0000 7fa925109640 1 -- 172.21.15.7:0/2625524840 >> 172.21.15.7:0/2625524840 conn(0x7fa9200fbe70 msgr2=0x7fa9200fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.194+0000 7fa925109640 1 -- 172.21.15.7:0/2625524840 shutdown_connections 2024-06-10T01:24:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.194+0000 7fa925109640 1 -- 172.21.15.7:0/2625524840 wait complete. 2024-06-10T01:24:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.195+0000 7fa925109640 1 Processor -- start 2024-06-10T01:24:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.195+0000 7fa925109640 1 -- start start 2024-06-10T01:24:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.196+0000 7fa925109640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa920100900 0x7fa920199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.196+0000 7fa925109640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa920101af0 0x7fa92019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.196+0000 7fa925109640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9201024b0 0x7fa9201a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.196+0000 7fa925109640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa92010c5d0 con 0x7fa920100900 2024-06-10T01:24:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.196+0000 7fa925109640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa92010c450 con 0x7fa9201024b0 2024-06-10T01:24:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.196+0000 7fa925109640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa92010c750 con 0x7fa920101af0 2024-06-10T01:24:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.196+0000 7fa91ed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa920100900 0x7fa920199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.196+0000 7fa91e59c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa920101af0 0x7fa92019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.196+0000 7fa91f59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9201024b0 0x7fa9201a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.196+0000 7fa91e59c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa920101af0 0x7fa92019a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:44820/0 (socket says 172.21.15.7:44820) 2024-06-10T01:24:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.196+0000 7fa91e59c640 1 -- 172.21.15.7:0/2465423796 learned_addr learned my addr 172.21.15.7:0/2465423796 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.196+0000 7fa91e59c640 1 -- 172.21.15.7:0/2465423796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9201024b0 msgr2=0x7fa9201a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.196+0000 7fa91e59c640 1 --2- 172.21.15.7:0/2465423796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9201024b0 0x7fa9201a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.196+0000 7fa91e59c640 1 -- 172.21.15.7:0/2465423796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa920100900 msgr2=0x7fa920199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:44.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.196+0000 7fa91e59c640 1 --2- 172.21.15.7:0/2465423796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa920100900 0x7fa920199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.196+0000 7fa91e59c640 1 -- 172.21.15.7:0/2465423796 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9201a1b30 con 0x7fa920101af0 2024-06-10T01:24:44.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.197+0000 7fa91e59c640 1 --2- 172.21.15.7:0/2465423796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa920101af0 0x7fa92019a450 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa91400e9f0 tx=0x7fa91400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:44.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.197+0000 7fa907fff640 1 -- 172.21.15.7:0/2465423796 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91400cd80 con 0x7fa920101af0 2024-06-10T01:24:44.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.197+0000 7fa907fff640 1 -- 172.21.15.7:0/2465423796 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa914005500 con 0x7fa920101af0 2024-06-10T01:24:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.197+0000 7fa907fff640 1 -- 172.21.15.7:0/2465423796 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa914010690 con 0x7fa920101af0 2024-06-10T01:24:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.197+0000 7fa925109640 1 -- 172.21.15.7:0/2465423796 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9201a1e20 con 0x7fa920101af0 2024-06-10T01:24:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.197+0000 7fa925109640 1 -- 172.21.15.7:0/2465423796 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9201a2290 con 0x7fa920101af0 2024-06-10T01:24:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.200+0000 7fa907fff640 1 -- 172.21.15.7:0/2465423796 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa9140046c0 con 0x7fa920101af0 2024-06-10T01:24:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.200+0000 7fa927e10740 1 Processor -- start 2024-06-10T01:24:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.200+0000 7fa927e10740 1 -- start start 2024-06-10T01:24:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.200+0000 7fa927e10740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562422807010 0x562422807430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.200+0000 7fa927e10740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562422807a00 0x56242280c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.200+0000 7fa927e10740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242280cb70 0x56242280f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.200+0000 7fa927e10740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56242276a4f0 con 0x56242280cb70 2024-06-10T01:24:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.200+0000 7fa927e10740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562422708870 con 0x562422807010 2024-06-10T01:24:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.200+0000 7fa927e10740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5624227664b0 con 0x562422807a00 2024-06-10T01:24:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.201+0000 7fa91f59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562422807010 0x562422807430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.201+0000 7fa91ed9d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562422807a00 0x56242280c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.201+0000 7fa91f59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562422807010 0x562422807430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:57900/0 (socket says 172.21.15.7:57900) 2024-06-10T01:24:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.201+0000 7fa91f59e640 1 -- 172.21.15.7:0/223045561 learned_addr learned my addr 172.21.15.7:0/223045561 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.201+0000 7fa91e59c640 1 --2- 172.21.15.7:0/223045561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242280cb70 0x56242280f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.201+0000 7fa91f59e640 1 -- 172.21.15.7:0/223045561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562422807a00 msgr2=0x56242280c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.201+0000 7fa907fff640 1 --2- 172.21.15.7:0/2465423796 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa9080a2c00 0x7fa9080a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.201+0000 7fa91f59e640 1 --2- 172.21.15.7:0/223045561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562422807a00 0x56242280c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.201+0000 7fa91f59e640 1 -- 172.21.15.7:0/223045561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242280cb70 msgr2=0x56242280f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.201+0000 7fa91f59e640 1 --2- 172.21.15.7:0/223045561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242280cb70 0x56242280f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.201+0000 7fa91f59e640 1 -- 172.21.15.7:0/223045561 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624227249b0 con 0x562422807010 2024-06-10T01:24:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.201+0000 7fa907fff640 1 -- 172.21.15.7:0/2465423796 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fa9140cf710 con 0x7fa920101af0 2024-06-10T01:24:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.201+0000 7fa91f59e640 1 --2- 172.21.15.7:0/2465423796 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa9080a2c00 0x7fa9080a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.202+0000 7fa91f59e640 1 --2- 172.21.15.7:0/223045561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562422807010 0x562422807430 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa918009ca0 tx=0x7fa9180093c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe3f0703bed2ea16 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.202+0000 7fa9057fa640 1 -- 172.21.15.7:0/223045561 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa918031730 con 0x562422807010 2024-06-10T01:24:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.202+0000 7fa9057fa640 1 -- 172.21.15.7:0/223045561 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa918036030 con 0x562422807010 2024-06-10T01:24:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.202+0000 7fa9057fa640 1 -- 172.21.15.7:0/223045561 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91802f8a0 con 0x562422807010 2024-06-10T01:24:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.203+0000 7fa927e10740 1 -- 172.21.15.7:0/223045561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562422807010 msgr2=0x562422807430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.203+0000 7fa91f59e640 1 --2- 172.21.15.7:0/2465423796 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa9080a2c00 0x7fa9080a50c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa918009430 tx=0x7fa91804b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.203+0000 7fa927e10740 1 --2- 172.21.15.7:0/223045561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562422807010 0x562422807430 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa918009ca0 tx=0x7fa9180093c0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.203+0000 7fa927e10740 1 -- 172.21.15.7:0/223045561 shutdown_connections 2024-06-10T01:24:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.203+0000 7fa927e10740 1 --2- 172.21.15.7:0/223045561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242280cb70 0x56242280f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.203+0000 7fa927e10740 1 --2- 172.21.15.7:0/223045561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562422807a00 0x56242280c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.203+0000 7fa927e10740 1 --2- 172.21.15.7:0/223045561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562422807010 0x562422807430 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.203+0000 7fa927e10740 1 -- 172.21.15.7:0/223045561 >> 172.21.15.7:0/223045561 conn(0x562422802680 msgr2=0x562422802a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.203+0000 7fa927e10740 1 -- 172.21.15.7:0/223045561 shutdown_connections 2024-06-10T01:24:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.203+0000 7fa927e10740 1 -- 172.21.15.7:0/223045561 wait complete. 2024-06-10T01:24:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.203+0000 7fa927e10740 1 -- start start 2024-06-10T01:24:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.203+0000 7fa927e10740 1 Processor -- start 2024-06-10T01:24:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.203+0000 7fa927e10740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562422807010 0x562422811170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.203+0000 7fa927e10740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562422807a00 0x5624228116b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.203+0000 7fa927e10740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56242280cb70 0x562422818680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.203+0000 7fa927e10740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56242280f9a0 con 0x562422807a00 2024-06-10T01:24:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.203+0000 7fa927e10740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5624226f3ae0 con 0x56242280cb70 2024-06-10T01:24:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.203+0000 7fa927e10740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56242280fb20 con 0x562422807010 2024-06-10T01:24:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.204+0000 7fa927e10740 10 client.0 _ll_register_callbacks cb 0x56242272c9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:24:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.204+0000 7fa91e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562422807a00 0x5624228116b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.204+0000 7fa91e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562422807a00 0x5624228116b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33326/0 (socket says 172.21.15.7:33326) 2024-06-10T01:24:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.204+0000 7fa91ed9d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562422807010 0x562422811170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.204+0000 7fa91e59c640 1 -- 172.21.15.7:0/1246327953 learned_addr learned my addr 172.21.15.7:0/1246327953 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:44.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.204+0000 7fa91f59e640 1 --2- 172.21.15.7:0/1246327953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56242280cb70 0x562422818680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:44.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.204+0000 7fa91e59c640 1 -- 172.21.15.7:0/1246327953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562422807010 msgr2=0x562422811170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:44.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.204+0000 7fa91e59c640 1 --2- 172.21.15.7:0/1246327953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562422807010 0x562422811170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.204+0000 7fa91e59c640 1 -- 172.21.15.7:0/1246327953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56242280cb70 msgr2=0x562422818680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:44.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.204+0000 7fa91e59c640 1 --2- 172.21.15.7:0/1246327953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56242280cb70 0x562422818680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.204+0000 7fa91e59c640 1 -- 172.21.15.7:0/1246327953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56242247e210 con 0x562422807a00 2024-06-10T01:24:44.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.204+0000 7fa91e59c640 1 --2- 172.21.15.7:0/1246327953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562422807a00 0x5624228116b0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fa91400c6a0 tx=0x7fa914004ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53a23ad03128dae server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:44.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.204+0000 7fa902ff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:24:44.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.204+0000 7fa902ff5640 1 -- 172.21.15.7:0/1246327953 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91409b730 con 0x562422807a00 2024-06-10T01:24:44.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.204+0000 7fa902ff5640 1 -- 172.21.15.7:0/1246327953 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fa9140a45d0 con 0x562422807a00 2024-06-10T01:24:44.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.204+0000 7fa927e10740 1 -- 172.21.15.7:0/1246327953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa918036030 con 0x562422807a00 2024-06-10T01:24:44.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.204+0000 7fa902ff5640 1 -- 172.21.15.7:0/1246327953 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fa91409b550 con 0x562422807a00 2024-06-10T01:24:44.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.204+0000 7fa927e10740 10 client.19995 fetch_fsmap learned FSMap version 51 2024-06-10T01:24:44.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.204+0000 7fa927e10740 1 -- 172.21.15.7:0/1246327953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x56242254a620 con 0x562422807a00 2024-06-10T01:24:44.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.205+0000 7fa902ff5640 1 -- 172.21.15.7:0/1246327953 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa918036030 con 0x562422807a00 2024-06-10T01:24:44.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.205+0000 7fa902ff5640 1 -- 172.21.15.7:0/1246327953 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 51) ==== 8350+0+0 (secure 0 0 0) 0x7fa91409b1d0 con 0x562422807a00 2024-06-10T01:24:44.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.205+0000 7fa927e10740 10 client.19995 fetch_fsmap finished waiting for FSMap version 51 2024-06-10T01:24:44.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.205+0000 7fa927e10740 10 client.19995 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:24:44.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.205+0000 7fa927e10740 20 client.19995 populate_metadata read hostname 'smithi007' 2024-06-10T01:24:44.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.205+0000 7fa927e10740 1 --2- 172.21.15.7:0/1246327953 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56242281d080 0x56242281f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:44.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.205+0000 7fa927e10740 4 client.19995 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:24:44.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.205+0000 7fa927e10740 1 -- 172.21.15.7:0/1246327953 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56242277df10 con 0x56242281d080 2024-06-10T01:24:44.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.205+0000 7fa91ed9d640 1 --2- 172.21.15.7:0/1246327953 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56242281d080 0x56242281f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:44.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.207+0000 7fa91ed9d640 1 --2- 172.21.15.7:0/1246327953 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56242281d080 0x56242281f470 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60d5ebd77586dcb8 server_cookie=62cb0bcd75bd3f7c in_seq=0 out_seq=0 2024-06-10T01:24:44.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.207+0000 7fa902ff5640 10 client.19995 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:44.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.211+0000 7fa902ff5640 1 -- 172.21.15.7:0/1246327953 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56242277df10 con 0x56242281d080 2024-06-10T01:24:44.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.211+0000 7fa902ff5640 10 client.19995 handle_command_reply: tid=0 2024-06-10T01:24:44.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.211+0000 7fa927e10740 1 client.19995 shutdown 2024-06-10T01:24:44.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.212+0000 7fa927e10740 1 -- 172.21.15.7:0/1246327953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562422807a00 msgr2=0x5624228116b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:44.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.212+0000 7fa927e10740 1 --2- 172.21.15.7:0/1246327953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562422807a00 0x5624228116b0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fa91400c6a0 tx=0x7fa914004ae0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.212+0000 7fa927e10740 1 -- 172.21.15.7:0/1246327953 shutdown_connections 2024-06-10T01:24:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.212+0000 7fa927e10740 1 --2- 172.21.15.7:0/1246327953 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56242281d080 0x56242281f470 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.212+0000 7fa927e10740 1 --2- 172.21.15.7:0/1246327953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56242280cb70 0x562422818680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.212+0000 7fa902ff5640 0 client.19995 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.212+0000 7fa927e10740 1 --2- 172.21.15.7:0/1246327953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562422807a00 0x5624228116b0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.212+0000 7fa927e10740 1 --2- 172.21.15.7:0/1246327953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562422807010 0x562422811170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.212+0000 7fa927e10740 1 -- 172.21.15.7:0/1246327953 >> 172.21.15.7:0/1246327953 conn(0x562422802680 msgr2=0x5624228017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.212+0000 7fa927e10740 1 -- 172.21.15.7:0/1246327953 shutdown_connections 2024-06-10T01:24:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.212+0000 7fa927e10740 1 -- 172.21.15.7:0/1246327953 wait complete. 2024-06-10T01:24:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.212+0000 7fa927e10740 20 client.19995 trim_cache size 0 max 16384 2024-06-10T01:24:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.231+0000 7fa927e10740 1 Processor -- start 2024-06-10T01:24:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.231+0000 7fa927e10740 1 -- start start 2024-06-10T01:24:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.231+0000 7fa927e10740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562422807010 0x562422808c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.231+0000 7fa927e10740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562422807a00 0x5624228091c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.231+0000 7fa927e10740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56242280cb70 0x562422818640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.231+0000 7fa927e10740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa9201af0b0 con 0x562422807010 2024-06-10T01:24:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.231+0000 7fa927e10740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa9201ac980 con 0x562422807a00 2024-06-10T01:24:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.231+0000 7fa927e10740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa9201aa7d0 con 0x56242280cb70 2024-06-10T01:24:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.231+0000 7fa91ed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562422807010 0x562422808c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.231+0000 7fa91ed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562422807010 0x562422808c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33340/0 (socket says 172.21.15.7:33340) 2024-06-10T01:24:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.231+0000 7fa91ed9d640 1 -- 172.21.15.7:0/1426728103 learned_addr learned my addr 172.21.15.7:0/1426728103 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.231+0000 7fa91f59e640 1 --2- 172.21.15.7:0/1426728103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56242280cb70 0x562422818640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.231+0000 7fa91e59c640 1 --2- 172.21.15.7:0/1426728103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562422807a00 0x5624228091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.231+0000 7fa91ed9d640 1 -- 172.21.15.7:0/1426728103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56242280cb70 msgr2=0x562422818640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.231+0000 7fa91ed9d640 1 --2- 172.21.15.7:0/1426728103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56242280cb70 0x562422818640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.231+0000 7fa91ed9d640 1 -- 172.21.15.7:0/1426728103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562422807a00 msgr2=0x5624228091c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.231+0000 7fa91ed9d640 1 --2- 172.21.15.7:0/1426728103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562422807a00 0x5624228091c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.231+0000 7fa91ed9d640 1 -- 172.21.15.7:0/1426728103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56242281fad0 con 0x562422807010 2024-06-10T01:24:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.231+0000 7fa91e59c640 1 --2- 172.21.15.7:0/1426728103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562422807a00 0x5624228091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:24:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.232+0000 7fa91ed9d640 1 --2- 172.21.15.7:0/1426728103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562422807010 0x562422808c80 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fa910000c00 tx=0x7fa910011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d7280789af1960 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.232+0000 7fa904ff9640 1 -- 172.21.15.7:0/1426728103 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910005220 con 0x562422807010 2024-06-10T01:24:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.232+0000 7fa904ff9640 1 -- 172.21.15.7:0/1426728103 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa910004c80 con 0x562422807010 2024-06-10T01:24:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.232+0000 7fa904ff9640 1 -- 172.21.15.7:0/1426728103 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91001e9b0 con 0x562422807010 2024-06-10T01:24:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.233+0000 7fa927e10740 1 -- 172.21.15.7:0/1426728103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562422807010 msgr2=0x562422808c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.233+0000 7fa927e10740 1 --2- 172.21.15.7:0/1426728103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562422807010 0x562422808c80 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fa910000c00 tx=0x7fa910011810 comp rx=0 tx=0).stop 2024-06-10T01:24:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.233+0000 7fa927e10740 1 -- 172.21.15.7:0/1426728103 shutdown_connections 2024-06-10T01:24:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.233+0000 7fa927e10740 1 --2- 172.21.15.7:0/1426728103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56242280cb70 0x562422818640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.233+0000 7fa927e10740 1 --2- 172.21.15.7:0/1426728103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562422807a00 0x5624228091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.233+0000 7fa927e10740 1 --2- 172.21.15.7:0/1426728103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562422807010 0x562422808c80 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.233+0000 7fa927e10740 1 -- 172.21.15.7:0/1426728103 >> 172.21.15.7:0/1426728103 conn(0x562422802680 msgr2=0x562422802a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.233+0000 7fa927e10740 1 -- 172.21.15.7:0/1426728103 shutdown_connections 2024-06-10T01:24:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.233+0000 7fa927e10740 1 -- 172.21.15.7:0/1426728103 wait complete. 2024-06-10T01:24:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.233+0000 7fa927e10740 1 -- start start 2024-06-10T01:24:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.233+0000 7fa927e10740 1 Processor -- start 2024-06-10T01:24:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.233+0000 7fa927e10740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562422807010 0x5624228157e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.233+0000 7fa927e10740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562422807a00 0x562422815d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.233+0000 7fa927e10740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242280cb70 0x562422816ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.233+0000 7fa927e10740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa9201aae40 con 0x56242280cb70 2024-06-10T01:24:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.233+0000 7fa927e10740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa9201ac2b0 con 0x562422807a00 2024-06-10T01:24:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.233+0000 7fa927e10740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa9201ab510 con 0x562422807010 2024-06-10T01:24:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.233+0000 7fa927e10740 10 client.0 _ll_register_callbacks cb 0x7fa9140d5240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:24:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.234+0000 7fa91f59e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242280cb70 0x562422816ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.234+0000 7fa91f59e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242280cb70 0x562422816ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33356/0 (socket says 172.21.15.7:33356) 2024-06-10T01:24:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.234+0000 7fa91f59e640 1 -- 172.21.15.7:0/1375173390 learned_addr learned my addr 172.21.15.7:0/1375173390 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.234+0000 7fa91e59c640 1 --2- 172.21.15.7:0/1375173390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562422807a00 0x562422815d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.234+0000 7fa91ed9d640 1 --2- 172.21.15.7:0/1375173390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562422807010 0x5624228157e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.234+0000 7fa91f59e640 1 -- 172.21.15.7:0/1375173390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562422807010 msgr2=0x5624228157e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.234+0000 7fa91f59e640 1 --2- 172.21.15.7:0/1375173390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562422807010 0x5624228157e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.234+0000 7fa91f59e640 1 -- 172.21.15.7:0/1375173390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562422807a00 msgr2=0x562422815d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.234+0000 7fa91f59e640 1 --2- 172.21.15.7:0/1375173390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562422807a00 0x562422815d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.234+0000 7fa91f59e640 1 -- 172.21.15.7:0/1375173390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624228172d0 con 0x56242280cb70 2024-06-10T01:24:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.234+0000 7fa91f59e640 1 --2- 172.21.15.7:0/1375173390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242280cb70 0x562422816ce0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fa91802ade0 tx=0x7fa91802f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5247ee7297d0df98 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.234+0000 7fa902ff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:24:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.234+0000 7fa902ff5640 1 -- 172.21.15.7:0/1375173390 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91802af40 con 0x56242280cb70 2024-06-10T01:24:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.234+0000 7fa902ff5640 1 -- 172.21.15.7:0/1375173390 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fa91804c040 con 0x56242280cb70 2024-06-10T01:24:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.234+0000 7fa927e10740 1 -- 172.21.15.7:0/1375173390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa910004c80 con 0x56242280cb70 2024-06-10T01:24:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.235+0000 7fa902ff5640 1 -- 172.21.15.7:0/1375173390 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fa91806c050 con 0x56242280cb70 2024-06-10T01:24:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.235+0000 7fa927e10740 10 client.20001 fetch_fsmap learned FSMap version 51 2024-06-10T01:24:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.235+0000 7fa927e10740 1 -- 172.21.15.7:0/1375173390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5624228185e0 con 0x56242280cb70 2024-06-10T01:24:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.235+0000 7fa902ff5640 1 -- 172.21.15.7:0/1375173390 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa910004c80 con 0x56242280cb70 2024-06-10T01:24:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.235+0000 7fa902ff5640 1 -- 172.21.15.7:0/1375173390 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 51) ==== 8350+0+0 (secure 0 0 0) 0x7fa918003f00 con 0x56242280cb70 2024-06-10T01:24:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.235+0000 7fa927e10740 10 client.20001 fetch_fsmap finished waiting for FSMap version 51 2024-06-10T01:24:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.235+0000 7fa927e10740 10 client.20001 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:24:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.235+0000 7fa927e10740 20 client.20001 populate_metadata read hostname 'smithi007' 2024-06-10T01:24:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.235+0000 7fa927e10740 1 --2- 172.21.15.7:0/1375173390 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562422887e40 0x56242288a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.235+0000 7fa927e10740 4 client.20001 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:24:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.235+0000 7fa927e10740 1 -- 172.21.15.7:0/1375173390 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562422777c20 con 0x562422887e40 2024-06-10T01:24:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.236+0000 7fa91ed9d640 1 --2- 172.21.15.7:0/1375173390 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562422887e40 0x56242288a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.238+0000 7fa91ed9d640 1 --2- 172.21.15.7:0/1375173390 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562422887e40 0x56242288a230 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6129a2606480e0 server_cookie=e662cfd8a8ef9c89 in_seq=0 out_seq=0 2024-06-10T01:24:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.238+0000 7fa902ff5640 10 client.20001 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:44.536 INFO:tasks.workunit.client.0.smithi007.stdout: 10 125371 22.76 MB/sec execute 201 sec latency 232.495 ms 2024-06-10T01:24:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.779+0000 7fa902ff5640 1 -- 172.21.15.7:0/1375173390 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562422777c20 con 0x562422887e40 2024-06-10T01:24:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.779+0000 7fa902ff5640 10 client.20001 handle_command_reply: tid=0 2024-06-10T01:24:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.779+0000 7fa927e10740 1 client.20001 shutdown 2024-06-10T01:24:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.779+0000 7fa927e10740 1 -- 172.21.15.7:0/1375173390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242280cb70 msgr2=0x562422816ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.779+0000 7fa927e10740 1 --2- 172.21.15.7:0/1375173390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242280cb70 0x562422816ce0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fa91802ade0 tx=0x7fa91802f8a0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.779+0000 7fa927e10740 1 -- 172.21.15.7:0/1375173390 shutdown_connections 2024-06-10T01:24:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.780+0000 7fa927e10740 1 --2- 172.21.15.7:0/1375173390 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562422887e40 0x56242288a230 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.780+0000 7fa927e10740 1 --2- 172.21.15.7:0/1375173390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242280cb70 0x562422816ce0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.780+0000 7fa902ff5640 0 client.20001 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.780+0000 7fa927e10740 1 --2- 172.21.15.7:0/1375173390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562422807a00 0x562422815d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.780+0000 7fa927e10740 1 --2- 172.21.15.7:0/1375173390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562422807010 0x5624228157e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.780+0000 7fa927e10740 1 -- 172.21.15.7:0/1375173390 >> 172.21.15.7:0/1375173390 conn(0x562422802680 msgr2=0x5624228110a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.780+0000 7fa927e10740 1 -- 172.21.15.7:0/1375173390 shutdown_connections 2024-06-10T01:24:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.780+0000 7fa927e10740 1 -- 172.21.15.7:0/1375173390 wait complete. 2024-06-10T01:24:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.780+0000 7fa927e10740 20 client.20001 trim_cache size 0 max 16384 2024-06-10T01:24:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:24:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:24:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:24:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.780+0000 7fa925109640 1 -- 172.21.15.7:0/2465423796 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa9080a2c00 msgr2=0x7fa9080a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.780+0000 7fa925109640 1 --2- 172.21.15.7:0/2465423796 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa9080a2c00 0x7fa9080a50c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa918009430 tx=0x7fa91804b000 comp rx=0 tx=0).stop 2024-06-10T01:24:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.780+0000 7fa925109640 1 -- 172.21.15.7:0/2465423796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa920101af0 msgr2=0x7fa92019a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.780+0000 7fa925109640 1 --2- 172.21.15.7:0/2465423796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa920101af0 0x7fa92019a450 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa91400e9f0 tx=0x7fa91400edf0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.780+0000 7fa925109640 1 -- 172.21.15.7:0/2465423796 shutdown_connections 2024-06-10T01:24:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.780+0000 7fa925109640 1 --2- 172.21.15.7:0/2465423796 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa9080a2c00 0x7fa9080a50c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.780+0000 7fa925109640 1 --2- 172.21.15.7:0/2465423796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9201024b0 0x7fa9201a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.780+0000 7fa925109640 1 --2- 172.21.15.7:0/2465423796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa920101af0 0x7fa92019a450 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.780+0000 7fa925109640 1 --2- 172.21.15.7:0/2465423796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa920100900 0x7fa920199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.780+0000 7fa925109640 1 -- 172.21.15.7:0/2465423796 >> 172.21.15.7:0/2465423796 conn(0x7fa9200fbe70 msgr2=0x7fa9200fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.781+0000 7fa925109640 1 -- 172.21.15.7:0/2465423796 shutdown_connections 2024-06-10T01:24:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:44.781+0000 7fa925109640 1 -- 172.21.15.7:0/2465423796 wait complete. 2024-06-10T01:24:45.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:45 smithi078 ceph-mon[36683]: pgmap v995: 129 pgs: 129 active+clean; 623 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 18 MiB/s wr, 906 op/s 2024-06-10T01:24:45.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:45 smithi175 ceph-mon[36636]: pgmap v995: 129 pgs: 129 active+clean; 623 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 18 MiB/s wr, 906 op/s 2024-06-10T01:24:45.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:45 smithi007 ceph-mon[32398]: pgmap v995: 129 pgs: 129 active+clean; 623 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 18 MiB/s wr, 906 op/s 2024-06-10T01:24:45.536 INFO:tasks.workunit.client.0.smithi007.stdout: 10 125833 22.75 MB/sec execute 202 sec latency 285.835 ms 2024-06-10T01:24:46.189 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:24:46.537 INFO:tasks.workunit.client.0.smithi007.stdout: 10 126322 22.76 MB/sec execute 203 sec latency 68.967 ms 2024-06-10T01:24:46.566 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:24:47.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:47 smithi007 ceph-mon[32398]: pgmap v996: 129 pgs: 129 active+clean; 605 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 19 MiB/s wr, 1.12k op/s 2024-06-10T01:24:47.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:47 smithi078 ceph-mon[36683]: pgmap v996: 129 pgs: 129 active+clean; 605 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 19 MiB/s wr, 1.12k op/s 2024-06-10T01:24:47.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:47 smithi175 ceph-mon[36636]: pgmap v996: 129 pgs: 129 active+clean; 605 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 19 MiB/s wr, 1.12k op/s 2024-06-10T01:24:47.536 INFO:tasks.workunit.client.0.smithi007.stdout: 10 127149 22.82 MB/sec execute 204 sec latency 61.834 ms 2024-06-10T01:24:48.537 INFO:tasks.workunit.client.0.smithi007.stdout: 10 128004 22.96 MB/sec execute 205 sec latency 82.280 ms 2024-06-10T01:24:49.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:49 smithi007 ceph-mon[32398]: pgmap v997: 129 pgs: 129 active+clean; 605 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 11 MiB/s wr, 780 op/s 2024-06-10T01:24:49.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:49 smithi078 ceph-mon[36683]: pgmap v997: 129 pgs: 129 active+clean; 605 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 11 MiB/s wr, 780 op/s 2024-06-10T01:24:49.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:49 smithi175 ceph-mon[36636]: pgmap v997: 129 pgs: 129 active+clean; 605 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 11 MiB/s wr, 780 op/s 2024-06-10T01:24:49.537 INFO:tasks.workunit.client.0.smithi007.stdout: 10 128725 23.10 MB/sec execute 206 sec latency 172.041 ms 2024-06-10T01:24:49.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.796+0000 7f72ff6f0700 1 -- 172.21.15.7:0/3377885425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72f811bc70 msgr2=0x7f72f811e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:49.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.796+0000 7f72ff6f0700 1 --2- 172.21.15.7:0/3377885425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72f811bc70 0x7f72f811e060 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f72f4009990 tx=0x7f72f4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.796+0000 7f72ff6f0700 1 -- 172.21.15.7:0/3377885425 shutdown_connections 2024-06-10T01:24:49.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.796+0000 7f72ff6f0700 1 --2- 172.21.15.7:0/3377885425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72f811bc70 0x7f72f811e060 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.796+0000 7f72ff6f0700 1 --2- 172.21.15.7:0/3377885425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72f8110c70 0x7f72f8109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.796+0000 7f72ff6f0700 1 --2- 172.21.15.7:0/3377885425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72f81099d0 0x7f72f8109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.796+0000 7f72ff6f0700 1 -- 172.21.15.7:0/3377885425 >> 172.21.15.7:0/3377885425 conn(0x7f72f806d600 msgr2=0x7f72f806da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:49.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.796+0000 7f72ff6f0700 1 -- 172.21.15.7:0/3377885425 shutdown_connections 2024-06-10T01:24:49.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.797+0000 7f72ff6f0700 1 -- 172.21.15.7:0/3377885425 wait complete. 2024-06-10T01:24:49.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.798+0000 7f72ff6f0700 1 Processor -- start 2024-06-10T01:24:49.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.798+0000 7f72ff6f0700 1 -- start start 2024-06-10T01:24:49.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.798+0000 7f72ff6f0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72f81099d0 0x7f72f81ad770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:49.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.798+0000 7f72ff6f0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72f8110c70 0x7f72f81adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:49.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.798+0000 7f72ff6f0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72f811bc70 0x7f72f81a79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:49.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.798+0000 7f72ff6f0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f72f81216f0 con 0x7f72f811bc70 2024-06-10T01:24:49.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.798+0000 7f72ff6f0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f72f8121570 con 0x7f72f81099d0 2024-06-10T01:24:49.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.798+0000 7f72ff6f0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f72f8121870 con 0x7f72f8110c70 2024-06-10T01:24:49.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.798+0000 7f72fd48c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72f81099d0 0x7f72f81ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:49.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.799+0000 7f72fd48c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72f81099d0 0x7f72f81ad770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:57944/0 (socket says 172.21.15.7:57944) 2024-06-10T01:24:49.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.799+0000 7f72fd48c700 1 -- 172.21.15.7:0/776100193 learned_addr learned my addr 172.21.15.7:0/776100193 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:49.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.799+0000 7f72fcc8b700 1 --2- 172.21.15.7:0/776100193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72f8110c70 0x7f72f81adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:49.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.799+0000 7f72fd48c700 1 -- 172.21.15.7:0/776100193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72f8110c70 msgr2=0x7f72f81adcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:49.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.799+0000 7f72fd48c700 1 --2- 172.21.15.7:0/776100193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72f8110c70 0x7f72f81adcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.799+0000 7f72fdc8d700 1 --2- 172.21.15.7:0/776100193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72f811bc70 0x7f72f81a79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:49.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.799+0000 7f72fd48c700 1 -- 172.21.15.7:0/776100193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72f811bc70 msgr2=0x7f72f81a79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:49.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.799+0000 7f72fd48c700 1 --2- 172.21.15.7:0/776100193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72f811bc70 0x7f72f81a79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.799+0000 7f72fd48c700 1 -- 172.21.15.7:0/776100193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72f81a8130 con 0x7f72f81099d0 2024-06-10T01:24:49.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.800+0000 7f72fd48c700 1 --2- 172.21.15.7:0/776100193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72f81099d0 0x7f72f81ad770 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f72e800d950 tx=0x7f72e800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:49.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.802+0000 7f72ee7fc700 1 -- 172.21.15.7:0/776100193 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e8016070 con 0x7f72f81099d0 2024-06-10T01:24:49.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.802+0000 7f72ee7fc700 1 -- 172.21.15.7:0/776100193 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72e8009e20 con 0x7f72f81099d0 2024-06-10T01:24:49.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.802+0000 7f72ee7fc700 1 -- 172.21.15.7:0/776100193 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e800f620 con 0x7f72f81099d0 2024-06-10T01:24:49.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.802+0000 7f72ff6f0700 1 -- 172.21.15.7:0/776100193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72f81a8420 con 0x7f72f81099d0 2024-06-10T01:24:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.802+0000 7f72ff6f0700 1 -- 172.21.15.7:0/776100193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72f81b4640 con 0x7f72f81099d0 2024-06-10T01:24:49.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.807+0000 7f72ee7fc700 1 -- 172.21.15.7:0/776100193 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f72e8019070 con 0x7f72f81099d0 2024-06-10T01:24:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.809+0000 7f7309f73b80 1 Processor -- start 2024-06-10T01:24:49.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.810+0000 7f72ee7fc700 1 --2- 172.21.15.7:0/776100193 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f72e40a2e20 0x7f72e40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:49.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.810+0000 7f72ee7fc700 1 -- 172.21.15.7:0/776100193 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f72e80cee70 con 0x7f72f81099d0 2024-06-10T01:24:49.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.811+0000 7f7309f73b80 1 -- start start 2024-06-10T01:24:49.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.811+0000 7f7309f73b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615b8a3df60 0x5615b8a3e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:49.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.811+0000 7f7309f73b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615b8a3e880 0x5615b8a38260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:49.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.811+0000 7f7309f73b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615b8a387a0 0x5615b8a38bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:49.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.811+0000 7f7309f73b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5615b892af60 con 0x5615b8a3e880 2024-06-10T01:24:49.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.811+0000 7f7309f73b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5615b8989850 con 0x5615b8a387a0 2024-06-10T01:24:49.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.811+0000 7f7309f73b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5615b88f1460 con 0x5615b8a3df60 2024-06-10T01:24:49.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.812+0000 7f72fd48c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615b8a3df60 0x5615b8a3e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:49.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.812+0000 7f72fd48c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615b8a3df60 0x5615b8a3e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:44884/0 (socket says 172.21.15.7:44884) 2024-06-10T01:24:49.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.812+0000 7f72fd48c700 1 -- 172.21.15.7:0/3434403804 learned_addr learned my addr 172.21.15.7:0/3434403804 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:49.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.812+0000 7f72fcc8b700 1 --2- 172.21.15.7:0/3434403804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615b8a3e880 0x5615b8a38260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:49.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.812+0000 7f72fdc8d700 1 --2- 172.21.15.7:0/3434403804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615b8a387a0 0x5615b8a38bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:49.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.812+0000 7f72fdc8d700 1 --2- 172.21.15.7:0/776100193 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f72e40a2e20 0x7f72e40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:49.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.812+0000 7f72fd48c700 1 -- 172.21.15.7:0/3434403804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615b8a387a0 msgr2=0x5615b8a38bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:49.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.812+0000 7f72fd48c700 1 --2- 172.21.15.7:0/3434403804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615b8a387a0 0x5615b8a38bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.812+0000 7f72fd48c700 1 -- 172.21.15.7:0/3434403804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615b8a3e880 msgr2=0x5615b8a38260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:49.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.812+0000 7f72fd48c700 1 --2- 172.21.15.7:0/3434403804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615b8a3e880 0x5615b8a38260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.812+0000 7f72fd48c700 1 -- 172.21.15.7:0/3434403804 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615b88d88d0 con 0x5615b8a3df60 2024-06-10T01:24:49.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.812+0000 7f72fcc8b700 1 --2- 172.21.15.7:0/3434403804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615b8a3e880 0x5615b8a38260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:24:49.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.812+0000 7f72fdc8d700 1 --2- 172.21.15.7:0/3434403804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615b8a387a0 0x5615b8a38bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:24:49.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.813+0000 7f72fd48c700 1 --2- 172.21.15.7:0/3434403804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615b8a3df60 0x5615b8a3e340 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f72e809aff0 tx=0x7f72e809af40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1ea51811573fa19 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:49.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.813+0000 7f72fdc8d700 1 --2- 172.21.15.7:0/776100193 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f72e40a2e20 0x7f72e40a52e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f72f4005e50 tx=0x7f72f4009ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:49.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.813+0000 7f72e37fe700 1 -- 172.21.15.7:0/3434403804 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e8020040 con 0x5615b8a3df60 2024-06-10T01:24:49.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.813+0000 7f72e37fe700 1 -- 172.21.15.7:0/3434403804 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72e80a3f60 con 0x5615b8a3df60 2024-06-10T01:24:49.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.814+0000 7f72e37fe700 1 -- 172.21.15.7:0/3434403804 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e8095e60 con 0x5615b8a3df60 2024-06-10T01:24:49.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.814+0000 7f7309f73b80 1 -- 172.21.15.7:0/3434403804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615b8a3df60 msgr2=0x5615b8a3e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:49.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.814+0000 7f7309f73b80 1 --2- 172.21.15.7:0/3434403804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615b8a3df60 0x5615b8a3e340 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f72e809aff0 tx=0x7f72e809af40 comp rx=0 tx=0).stop 2024-06-10T01:24:49.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.814+0000 7f7309f73b80 1 -- 172.21.15.7:0/3434403804 shutdown_connections 2024-06-10T01:24:49.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.814+0000 7f7309f73b80 1 --2- 172.21.15.7:0/3434403804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615b8a387a0 0x5615b8a38bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.814+0000 7f7309f73b80 1 --2- 172.21.15.7:0/3434403804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615b8a3df60 0x5615b8a3e340 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.814+0000 7f7309f73b80 1 --2- 172.21.15.7:0/3434403804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615b8a3e880 0x5615b8a38260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.814+0000 7f7309f73b80 1 -- 172.21.15.7:0/3434403804 >> 172.21.15.7:0/3434403804 conn(0x5615b8965080 msgr2=0x5615b8a33220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:49.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.815+0000 7f7309f73b80 1 -- 172.21.15.7:0/3434403804 shutdown_connections 2024-06-10T01:24:49.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.815+0000 7f7309f73b80 1 -- 172.21.15.7:0/3434403804 wait complete. 2024-06-10T01:24:49.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.815+0000 7f7309f73b80 1 -- start start 2024-06-10T01:24:49.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.815+0000 7f7309f73b80 1 Processor -- start 2024-06-10T01:24:49.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.816+0000 7f7309f73b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615b8a387a0 0x5615b8a445f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:49.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.816+0000 7f7309f73b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615b8a3df60 0x5615b8a418f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:49.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.816+0000 7f7309f73b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615b8a3e880 0x5615b8a41e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:49.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.816+0000 7f7309f73b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5615b8929fc0 con 0x5615b8a387a0 2024-06-10T01:24:49.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.816+0000 7f7309f73b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5615b8985810 con 0x5615b8a3e880 2024-06-10T01:24:49.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.816+0000 7f7309f73b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5615b8a45250 con 0x5615b8a3df60 2024-06-10T01:24:49.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.816+0000 7f7309f73b80 10 client.0 _ll_register_callbacks cb 0x5615b891f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:24:49.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.816+0000 7f72fd48c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615b8a387a0 0x5615b8a445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.816+0000 7f72fd48c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615b8a387a0 0x5615b8a445f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33400/0 (socket says 172.21.15.7:33400) 2024-06-10T01:24:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.816+0000 7f72fd48c700 1 -- 172.21.15.7:0/1197456644 learned_addr learned my addr 172.21.15.7:0/1197456644 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.816+0000 7f72fcc8b700 1 --2- 172.21.15.7:0/1197456644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615b8a3df60 0x5615b8a418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.817+0000 7f72fdc8d700 1 --2- 172.21.15.7:0/1197456644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615b8a3e880 0x5615b8a41e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.817+0000 7f72fd48c700 1 -- 172.21.15.7:0/1197456644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615b8a3df60 msgr2=0x5615b8a418f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.817+0000 7f72fd48c700 1 --2- 172.21.15.7:0/1197456644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615b8a3df60 0x5615b8a418f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.817+0000 7f72fd48c700 1 -- 172.21.15.7:0/1197456644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615b8a3e880 msgr2=0x5615b8a41e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.817+0000 7f72fd48c700 1 --2- 172.21.15.7:0/1197456644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615b8a3e880 0x5615b8a41e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.817+0000 7f72fd48c700 1 -- 172.21.15.7:0/1197456644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615b894f750 con 0x5615b8a387a0 2024-06-10T01:24:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.817+0000 7f72fcc8b700 1 --2- 172.21.15.7:0/1197456644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615b8a3df60 0x5615b8a418f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:24:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.817+0000 7f72fdc8d700 1 --2- 172.21.15.7:0/1197456644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615b8a3e880 0x5615b8a41e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:24:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.817+0000 7f72fd48c700 1 --2- 172.21.15.7:0/1197456644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615b8a387a0 0x5615b8a445f0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f72e80cc5e0 tx=0x7f72e8095e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b9a54fc943c91d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.817+0000 7f72e0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:24:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.817+0000 7f72e0ff9700 1 -- 172.21.15.7:0/1197456644 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e80b3020 con 0x5615b8a387a0 2024-06-10T01:24:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.817+0000 7f7309f73b80 1 -- 172.21.15.7:0/1197456644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72e80a3f60 con 0x5615b8a387a0 2024-06-10T01:24:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.817+0000 7f72e0ff9700 1 -- 172.21.15.7:0/1197456644 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f72e809b550 con 0x5615b8a387a0 2024-06-10T01:24:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.818+0000 7f72e0ff9700 1 -- 172.21.15.7:0/1197456644 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f72e8020070 con 0x5615b8a387a0 2024-06-10T01:24:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.818+0000 7f7309f73b80 10 client.20010 fetch_fsmap learned FSMap version 51 2024-06-10T01:24:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.818+0000 7f7309f73b80 1 -- 172.21.15.7:0/1197456644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5615b8a4f9f0 con 0x5615b8a387a0 2024-06-10T01:24:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.818+0000 7f72e0ff9700 1 -- 172.21.15.7:0/1197456644 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72e80a3f60 con 0x5615b8a387a0 2024-06-10T01:24:49.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.818+0000 7f72e0ff9700 1 -- 172.21.15.7:0/1197456644 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 51) ==== 8350+0+0 (secure 0 0 0) 0x7f72e809a740 con 0x5615b8a387a0 2024-06-10T01:24:49.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.818+0000 7f7309f73b80 10 client.20010 fetch_fsmap finished waiting for FSMap version 51 2024-06-10T01:24:49.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.818+0000 7f7309f73b80 10 client.20010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:24:49.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.819+0000 7f7309f73b80 20 client.20010 populate_metadata read hostname 'smithi007' 2024-06-10T01:24:49.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.819+0000 7f7309f73b80 1 --2- 172.21.15.7:0/1197456644 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5615b8a51ec0 0x5615b8a542b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:49.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.819+0000 7f7309f73b80 4 client.20010 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:24:49.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.819+0000 7f7309f73b80 1 -- 172.21.15.7:0/1197456644 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615b8996d50 con 0x5615b8a51ec0 2024-06-10T01:24:49.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.821+0000 7f72fcc8b700 1 --2- 172.21.15.7:0/1197456644 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5615b8a51ec0 0x5615b8a542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:49.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.822+0000 7f72fcc8b700 1 --2- 172.21.15.7:0/1197456644 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5615b8a51ec0 0x5615b8a542b0 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecddc450f393f784 server_cookie=33320f74ea620ea2 in_seq=0 out_seq=0 2024-06-10T01:24:49.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.822+0000 7f72e0ff9700 10 client.20010 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:49.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.823+0000 7f72e0ff9700 1 -- 172.21.15.7:0/1197456644 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5615b8996d50 con 0x5615b8a51ec0 2024-06-10T01:24:49.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.823+0000 7f72e0ff9700 10 client.20010 handle_command_reply: tid=0 2024-06-10T01:24:49.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.823+0000 7f7309f73b80 1 client.20010 shutdown 2024-06-10T01:24:49.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.824+0000 7f7309f73b80 1 -- 172.21.15.7:0/1197456644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615b8a387a0 msgr2=0x5615b8a445f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:49.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.824+0000 7f7309f73b80 1 --2- 172.21.15.7:0/1197456644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615b8a387a0 0x5615b8a445f0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f72e80cc5e0 tx=0x7f72e8095e60 comp rx=0 tx=0).stop 2024-06-10T01:24:49.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.828+0000 7f7309f73b80 1 -- 172.21.15.7:0/1197456644 shutdown_connections 2024-06-10T01:24:49.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.828+0000 7f7309f73b80 1 --2- 172.21.15.7:0/1197456644 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5615b8a51ec0 0x5615b8a542b0 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.828+0000 7f7309f73b80 1 --2- 172.21.15.7:0/1197456644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615b8a3e880 0x5615b8a41e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.828+0000 7f7309f73b80 1 --2- 172.21.15.7:0/1197456644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615b8a387a0 0x5615b8a445f0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.828+0000 7f7309f73b80 1 --2- 172.21.15.7:0/1197456644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615b8a3df60 0x5615b8a418f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.828+0000 7f7309f73b80 1 -- 172.21.15.7:0/1197456644 >> 172.21.15.7:0/1197456644 conn(0x5615b8965080 msgr2=0x5615b8a32700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:49.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.828+0000 7f72e0ff9700 0 client.20010 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:49.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.830+0000 7f7309f73b80 1 -- 172.21.15.7:0/1197456644 shutdown_connections 2024-06-10T01:24:49.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.831+0000 7f7309f73b80 1 -- 172.21.15.7:0/1197456644 wait complete. 2024-06-10T01:24:49.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.831+0000 7f7309f73b80 20 client.20010 trim_cache size 0 max 16384 2024-06-10T01:24:49.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.874+0000 7f7309f73b80 1 Processor -- start 2024-06-10T01:24:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.874+0000 7f7309f73b80 1 -- start start 2024-06-10T01:24:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.874+0000 7f7309f73b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615b8965080 0x5615b8a44130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.874+0000 7f7309f73b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615b8a387a0 0x5615b8a32e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.874+0000 7f7309f73b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615b8a3df60 0x5615b8a418b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.874+0000 7f7309f73b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f72f81bfa30 con 0x5615b8965080 2024-06-10T01:24:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.874+0000 7f7309f73b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f72f81bd2c0 con 0x5615b8a387a0 2024-06-10T01:24:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.874+0000 7f7309f73b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f72f81bb110 con 0x5615b8a3df60 2024-06-10T01:24:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.875+0000 7f72fcc8b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615b8a387a0 0x5615b8a32e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.875+0000 7f72fd48c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615b8965080 0x5615b8a44130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.875+0000 7f72fcc8b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615b8a387a0 0x5615b8a32e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:57974/0 (socket says 172.21.15.7:57974) 2024-06-10T01:24:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.875+0000 7f72fd48c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615b8965080 0x5615b8a44130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33412/0 (socket says 172.21.15.7:33412) 2024-06-10T01:24:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.875+0000 7f72fcc8b700 1 -- 172.21.15.7:0/1173440230 learned_addr learned my addr 172.21.15.7:0/1173440230 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.875+0000 7f72fdc8d700 1 --2- 172.21.15.7:0/1173440230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615b8a3df60 0x5615b8a418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:49.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.875+0000 7f72fcc8b700 1 -- 172.21.15.7:0/1173440230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615b8a3df60 msgr2=0x5615b8a418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:49.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.875+0000 7f72fcc8b700 1 --2- 172.21.15.7:0/1173440230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615b8a3df60 0x5615b8a418b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.875+0000 7f72fcc8b700 1 -- 172.21.15.7:0/1173440230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615b8965080 msgr2=0x5615b8a44130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:49.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.875+0000 7f72fcc8b700 1 --2- 172.21.15.7:0/1173440230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615b8965080 0x5615b8a44130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.875+0000 7f72fcc8b700 1 -- 172.21.15.7:0/1173440230 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615b8a3c360 con 0x5615b8a387a0 2024-06-10T01:24:49.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.875+0000 7f72fd48c700 1 --2- 172.21.15.7:0/1173440230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615b8965080 0x5615b8a44130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:24:49.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.875+0000 7f72fcc8b700 1 --2- 172.21.15.7:0/1173440230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615b8a387a0 0x5615b8a32e80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x5615b8a42ec0 tx=0x7f72f00091b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa573ec2e258a949 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:49.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.876+0000 7f72e2ffd700 1 -- 172.21.15.7:0/1173440230 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f0007920 con 0x5615b8a387a0 2024-06-10T01:24:49.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.876+0000 7f72e2ffd700 1 -- 172.21.15.7:0/1173440230 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72f001e2d0 con 0x5615b8a387a0 2024-06-10T01:24:49.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.876+0000 7f72e2ffd700 1 -- 172.21.15.7:0/1173440230 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f0017670 con 0x5615b8a387a0 2024-06-10T01:24:49.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.877+0000 7f7309f73b80 1 -- 172.21.15.7:0/1173440230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615b8a387a0 msgr2=0x5615b8a32e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:49.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.877+0000 7f7309f73b80 1 --2- 172.21.15.7:0/1173440230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615b8a387a0 0x5615b8a32e80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x5615b8a42ec0 tx=0x7f72f00091b0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.877+0000 7f7309f73b80 1 -- 172.21.15.7:0/1173440230 shutdown_connections 2024-06-10T01:24:49.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.877+0000 7f7309f73b80 1 --2- 172.21.15.7:0/1173440230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615b8a3df60 0x5615b8a418b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.877+0000 7f7309f73b80 1 --2- 172.21.15.7:0/1173440230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615b8965080 0x5615b8a44130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.877+0000 7f7309f73b80 1 --2- 172.21.15.7:0/1173440230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615b8a387a0 0x5615b8a32e80 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.877+0000 7f7309f73b80 1 -- 172.21.15.7:0/1173440230 >> 172.21.15.7:0/1173440230 conn(0x5615b8a3f6e0 msgr2=0x5615b8a411a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:49.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.878+0000 7f7309f73b80 1 -- 172.21.15.7:0/1173440230 shutdown_connections 2024-06-10T01:24:49.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.878+0000 7f7309f73b80 1 -- 172.21.15.7:0/1173440230 wait complete. 2024-06-10T01:24:49.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.878+0000 7f7309f73b80 1 -- start start 2024-06-10T01:24:49.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.878+0000 7f7309f73b80 1 Processor -- start 2024-06-10T01:24:49.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.879+0000 7f7309f73b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615b8965080 0x5615b8a55470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:49.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.879+0000 7f72fd48c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615b8965080 0x5615b8a55470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:49.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.879+0000 7f72fd48c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615b8965080 0x5615b8a55470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33422/0 (socket says 172.21.15.7:33422) 2024-06-10T01:24:49.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.879+0000 7f72fd48c700 1 -- 172.21.15.7:0/1190894046 learned_addr learned my addr 172.21.15.7:0/1190894046 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:49.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.879+0000 7f7309f73b80 1 --2- 172.21.15.7:0/1190894046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615b8a387a0 0x5615b8a559b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:49.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.879+0000 7f7309f73b80 1 --2- 172.21.15.7:0/1190894046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615b8a3df60 0x5615b8abb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:49.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.880+0000 7f7309f73b80 1 -- 172.21.15.7:0/1190894046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f72f81bb780 con 0x5615b8965080 2024-06-10T01:24:49.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.880+0000 7f7309f73b80 1 -- 172.21.15.7:0/1190894046 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f72f81bcbf0 con 0x5615b8a3df60 2024-06-10T01:24:49.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.880+0000 7f7309f73b80 1 -- 172.21.15.7:0/1190894046 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f72f81bbe50 con 0x5615b8a387a0 2024-06-10T01:24:49.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.880+0000 7f7309f73b80 10 client.0 _ll_register_callbacks cb 0x5615b891f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:24:49.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.880+0000 7f72fd48c700 1 -- 172.21.15.7:0/1190894046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615b8a387a0 msgr2=0x5615b8a559b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:24:49.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.880+0000 7f72fd48c700 1 --2- 172.21.15.7:0/1190894046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615b8a387a0 0x5615b8a559b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.880+0000 7f72fd48c700 1 -- 172.21.15.7:0/1190894046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615b8a3df60 msgr2=0x5615b8abb800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:24:49.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.880+0000 7f72fd48c700 1 --2- 172.21.15.7:0/1190894046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615b8a3df60 0x5615b8abb800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.880+0000 7f72fd48c700 1 -- 172.21.15.7:0/1190894046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615b8abbd40 con 0x5615b8965080 2024-06-10T01:24:49.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.881+0000 7f72fd48c700 1 --2- 172.21.15.7:0/1190894046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615b8965080 0x5615b8a55470 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f72e80995a0 tx=0x7f72e8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7486997e8f75d9d6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:49.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.881+0000 7f72e0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:24:49.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.881+0000 7f72e0ff9700 1 -- 172.21.15.7:0/1190894046 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e80b3040 con 0x5615b8965080 2024-06-10T01:24:49.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.881+0000 7f72e0ff9700 1 -- 172.21.15.7:0/1190894046 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f72e80af520 con 0x5615b8965080 2024-06-10T01:24:49.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.881+0000 7f7309f73b80 1 -- 172.21.15.7:0/1190894046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72f001e2d0 con 0x5615b8965080 2024-06-10T01:24:49.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.882+0000 7f72e0ff9700 1 -- 172.21.15.7:0/1190894046 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f72e80a3f60 con 0x5615b8965080 2024-06-10T01:24:49.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.882+0000 7f7309f73b80 10 client.20016 fetch_fsmap learned FSMap version 51 2024-06-10T01:24:49.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.882+0000 7f7309f73b80 1 -- 172.21.15.7:0/1190894046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5615b8abce90 con 0x5615b8965080 2024-06-10T01:24:49.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.882+0000 7f72e0ff9700 1 -- 172.21.15.7:0/1190894046 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72e80af210 con 0x5615b8965080 2024-06-10T01:24:49.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.882+0000 7f72e0ff9700 1 -- 172.21.15.7:0/1190894046 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 51) ==== 8350+0+0 (secure 0 0 0) 0x7f72e80a3ad0 con 0x5615b8965080 2024-06-10T01:24:49.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.883+0000 7f7309f73b80 10 client.20016 fetch_fsmap finished waiting for FSMap version 51 2024-06-10T01:24:49.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.883+0000 7f7309f73b80 10 client.20016 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:24:49.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.883+0000 7f7309f73b80 20 client.20016 populate_metadata read hostname 'smithi007' 2024-06-10T01:24:49.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.883+0000 7f7309f73b80 1 --2- 172.21.15.7:0/1190894046 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5615b8a3e880 0x5615b8abf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:49.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.883+0000 7f7309f73b80 4 client.20016 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:24:49.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.883+0000 7f7309f73b80 1 -- 172.21.15.7:0/1190894046 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f72f001e2d0 con 0x5615b8a3e880 2024-06-10T01:24:49.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.884+0000 7f72fcc8b700 1 --2- 172.21.15.7:0/1190894046 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5615b8a3e880 0x5615b8abf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:49.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.885+0000 7f72fcc8b700 1 --2- 172.21.15.7:0/1190894046 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5615b8a3e880 0x5615b8abf330 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8318c88293c70f3d server_cookie=bbdb654d1244beca in_seq=0 out_seq=0 2024-06-10T01:24:49.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.885+0000 7f72e0ff9700 10 client.20016 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:49.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.886+0000 7f72e0ff9700 1 -- 172.21.15.7:0/1190894046 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f72f001e2d0 con 0x5615b8a3e880 2024-06-10T01:24:49.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.886+0000 7f72e0ff9700 10 client.20016 handle_command_reply: tid=0 2024-06-10T01:24:49.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.886+0000 7f7309f73b80 1 client.20016 shutdown 2024-06-10T01:24:49.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.886+0000 7f7309f73b80 1 -- 172.21.15.7:0/1190894046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615b8965080 msgr2=0x5615b8a55470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:49.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.886+0000 7f7309f73b80 1 --2- 172.21.15.7:0/1190894046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615b8965080 0x5615b8a55470 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f72e80995a0 tx=0x7f72e8000bc0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.887+0000 7f7309f73b80 1 -- 172.21.15.7:0/1190894046 shutdown_connections 2024-06-10T01:24:49.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.887+0000 7f7309f73b80 1 --2- 172.21.15.7:0/1190894046 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5615b8a3e880 0x5615b8abf330 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.887+0000 7f7309f73b80 1 --2- 172.21.15.7:0/1190894046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615b8a3df60 0x5615b8abb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.887+0000 7f7309f73b80 1 --2- 172.21.15.7:0/1190894046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615b8965080 0x5615b8a55470 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.887+0000 7f7309f73b80 1 --2- 172.21.15.7:0/1190894046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615b8a387a0 0x5615b8a559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.887+0000 7f7309f73b80 1 -- 172.21.15.7:0/1190894046 >> 172.21.15.7:0/1190894046 conn(0x5615b8a3f6e0 msgr2=0x5615b8a41a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:49.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.887+0000 7f72e0ff9700 0 client.20016 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:49.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.887+0000 7f7309f73b80 1 -- 172.21.15.7:0/1190894046 shutdown_connections 2024-06-10T01:24:49.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.887+0000 7f7309f73b80 1 -- 172.21.15.7:0/1190894046 wait complete. 2024-06-10T01:24:49.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.887+0000 7f7309f73b80 20 client.20016 trim_cache size 0 max 16384 2024-06-10T01:24:49.883 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:24:49.883 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-10T01:24:49.883 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-10T01:24:49.883 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:24:49.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.888+0000 7f72ff6f0700 1 -- 172.21.15.7:0/776100193 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f72e40a2e20 msgr2=0x7f72e40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:49.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.888+0000 7f72ff6f0700 1 --2- 172.21.15.7:0/776100193 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f72e40a2e20 0x7f72e40a52e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f72f4005e50 tx=0x7f72f4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.888+0000 7f72ff6f0700 1 -- 172.21.15.7:0/776100193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72f81099d0 msgr2=0x7f72f81ad770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:49.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.888+0000 7f72ff6f0700 1 --2- 172.21.15.7:0/776100193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72f81099d0 0x7f72f81ad770 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f72e800d950 tx=0x7f72e800dc60 comp rx=0 tx=0).stop 2024-06-10T01:24:49.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.888+0000 7f72ff6f0700 1 -- 172.21.15.7:0/776100193 shutdown_connections 2024-06-10T01:24:49.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.888+0000 7f72ff6f0700 1 --2- 172.21.15.7:0/776100193 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f72e40a2e20 0x7f72e40a52e0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.889+0000 7f72ff6f0700 1 --2- 172.21.15.7:0/776100193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72f811bc70 0x7f72f81a79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.889+0000 7f72ff6f0700 1 --2- 172.21.15.7:0/776100193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72f81099d0 0x7f72f81ad770 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.889+0000 7f72ff6f0700 1 --2- 172.21.15.7:0/776100193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72f8110c70 0x7f72f81adcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:49.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.889+0000 7f72ff6f0700 1 -- 172.21.15.7:0/776100193 >> 172.21.15.7:0/776100193 conn(0x7f72f806d600 msgr2=0x7f72f8119c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:49.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.889+0000 7f72ff6f0700 1 -- 172.21.15.7:0/776100193 shutdown_connections 2024-06-10T01:24:49.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:49.889+0000 7f72ff6f0700 1 -- 172.21.15.7:0/776100193 wait complete. 2024-06-10T01:24:50.537 INFO:tasks.workunit.client.0.smithi007.stdout: 10 129252 23.08 MB/sec execute 207 sec latency 69.929 ms 2024-06-10T01:24:50.626 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:24:50.626 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:24:50.980 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:24:51.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:51 smithi078 ceph-mon[36683]: pgmap v998: 129 pgs: 129 active+clean; 612 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 12 MiB/s wr, 907 op/s 2024-06-10T01:24:51.394 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:51 smithi007 ceph-mon[32398]: pgmap v998: 129 pgs: 129 active+clean; 612 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 12 MiB/s wr, 907 op/s 2024-06-10T01:24:51.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:51 smithi175 ceph-mon[36636]: pgmap v998: 129 pgs: 129 active+clean; 612 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 12 MiB/s wr, 907 op/s 2024-06-10T01:24:51.537 INFO:tasks.workunit.client.0.smithi007.stdout: 10 129811 23.09 MB/sec execute 208 sec latency 67.005 ms 2024-06-10T01:24:52.537 INFO:tasks.workunit.client.0.smithi007.stdout: 10 130583 23.18 MB/sec execute 209 sec latency 162.055 ms 2024-06-10T01:24:52.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.973+0000 7f2c40977700 1 -- 172.21.15.7:0/4044122133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c3c10ef30 msgr2=0x7f2c3c111320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:52.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.973+0000 7f2c40977700 1 --2- 172.21.15.7:0/4044122133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c3c10ef30 0x7f2c3c111320 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400b0a0 tx=0x7f2c3400b3b0 comp rx=0 tx=0).stop 2024-06-10T01:24:52.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.973+0000 7f2c40977700 1 -- 172.21.15.7:0/4044122133 shutdown_connections 2024-06-10T01:24:52.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.973+0000 7f2c40977700 1 --2- 172.21.15.7:0/4044122133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c3c10ef30 0x7f2c3c111320 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:52.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.973+0000 7f2c40977700 1 --2- 172.21.15.7:0/4044122133 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c3c06aa30 0x7f2c3c06ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:52.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.973+0000 7f2c40977700 1 --2- 172.21.15.7:0/4044122133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c3c06b350 0x7f2c3c10e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:52.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.973+0000 7f2c40977700 1 -- 172.21.15.7:0/4044122133 >> 172.21.15.7:0/4044122133 conn(0x7f2c3c078300 msgr2=0x7f2c3c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:52.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.973+0000 7f2c40977700 1 -- 172.21.15.7:0/4044122133 shutdown_connections 2024-06-10T01:24:52.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.973+0000 7f2c40977700 1 -- 172.21.15.7:0/4044122133 wait complete. 2024-06-10T01:24:52.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.974+0000 7f2c40977700 1 Processor -- start 2024-06-10T01:24:52.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.974+0000 7f2c40977700 1 -- start start 2024-06-10T01:24:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.975+0000 7f2c40977700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c3c06aa30 0x7f2c3c1a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.975+0000 7f2c40977700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c3c06b350 0x7f2c3c1a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.975+0000 7f2c40977700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c3c10ef30 0x7f2c3c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.975+0000 7f2c40977700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c1149f0 con 0x7f2c3c06aa30 2024-06-10T01:24:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.975+0000 7f2c40977700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c114870 con 0x7f2c3c06b350 2024-06-10T01:24:52.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.975+0000 7f2c40977700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c114b70 con 0x7f2c3c10ef30 2024-06-10T01:24:52.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.975+0000 7f2c3a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c3c06aa30 0x7f2c3c1a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:52.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.975+0000 7f2c3a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c3c06aa30 0x7f2c3c1a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38838/0 (socket says 172.21.15.7:38838) 2024-06-10T01:24:52.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.975+0000 7f2c3a59c700 1 -- 172.21.15.7:0/2173882161 learned_addr learned my addr 172.21.15.7:0/2173882161 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:52.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.975+0000 7f2c39d9b700 1 --2- 172.21.15.7:0/2173882161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c3c06b350 0x7f2c3c1a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:52.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.975+0000 7f2c3ad9d700 1 --2- 172.21.15.7:0/2173882161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c3c10ef30 0x7f2c3c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:52.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.975+0000 7f2c3a59c700 1 -- 172.21.15.7:0/2173882161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c3c10ef30 msgr2=0x7f2c3c19aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:52.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.975+0000 7f2c3a59c700 1 --2- 172.21.15.7:0/2173882161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c3c10ef30 0x7f2c3c19aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:52.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.975+0000 7f2c3a59c700 1 -- 172.21.15.7:0/2173882161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c3c06b350 msgr2=0x7f2c3c1a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:52.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.975+0000 7f2c3a59c700 1 --2- 172.21.15.7:0/2173882161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c3c06b350 0x7f2c3c1a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:52.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.975+0000 7f2c3a59c700 1 -- 172.21.15.7:0/2173882161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3c19b1d0 con 0x7f2c3c06aa30 2024-06-10T01:24:52.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.975+0000 7f2c3a59c700 1 --2- 172.21.15.7:0/2173882161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c3c06aa30 0x7f2c3c1a08b0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c00e8e0 tx=0x7f2c2c00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:52.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.976+0000 7f2c2b7fe700 1 -- 172.21.15.7:0/2173882161 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c004a30 con 0x7f2c3c06aa30 2024-06-10T01:24:52.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.976+0000 7f2c2b7fe700 1 -- 172.21.15.7:0/2173882161 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c2c00cc90 con 0x7f2c3c06aa30 2024-06-10T01:24:52.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.976+0000 7f2c2b7fe700 1 -- 172.21.15.7:0/2173882161 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c004a30 con 0x7f2c3c06aa30 2024-06-10T01:24:52.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.976+0000 7f2c40977700 1 -- 172.21.15.7:0/2173882161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c3c19b4c0 con 0x7f2c3c06aa30 2024-06-10T01:24:52.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.976+0000 7f2c40977700 1 -- 172.21.15.7:0/2173882161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c3c1a7780 con 0x7f2c3c06aa30 2024-06-10T01:24:52.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.978+0000 7f2c40977700 1 -- 172.21.15.7:0/2173882161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c1c005360 con 0x7f2c3c06aa30 2024-06-10T01:24:52.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.979+0000 7f2c2b7fe700 1 -- 172.21.15.7:0/2173882161 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f2c2c004bd0 con 0x7f2c3c06aa30 2024-06-10T01:24:52.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.980+0000 7f2c2b7fe700 1 --2- 172.21.15.7:0/2173882161 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2c240a3060 0x7f2c240a5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:52.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.981+0000 7f2c2b7fe700 1 -- 172.21.15.7:0/2173882161 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f2c2c004830 con 0x7f2c3c06aa30 2024-06-10T01:24:52.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.982+0000 7f2c39d9b700 1 --2- 172.21.15.7:0/2173882161 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2c240a3060 0x7f2c240a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:52.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.983+0000 7f2c39d9b700 1 --2- 172.21.15.7:0/2173882161 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2c240a3060 0x7f2c240a5520 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c101230 tx=0x7f2c30009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:52.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:52.988+0000 7f2c2b7fe700 1 -- 172.21.15.7:0/2173882161 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2c2c016030 con 0x7f2c3c06aa30 2024-06-10T01:24:53.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:53 smithi007 ceph-mon[32398]: pgmap v999: 129 pgs: 129 active+clean; 613 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.3 MiB/s wr, 643 op/s 2024-06-10T01:24:53.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:53.298+0000 7f2c40977700 1 -- 172.21.15.7:0/2173882161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c1c005650 con 0x7f2c3c06aa30 2024-06-10T01:24:53.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:53.299+0000 7f2c2b7fe700 1 -- 172.21.15.7:0/2173882161 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+11856 (secure 0 0 0) 0x7f2c2c09b750 con 0x7f2c3c06aa30 2024-06-10T01:24:53.293 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:24:53.296 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":51,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:24:33.839012+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28825":{"gid":28825,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/1972426673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1972426673},{"type":"v1","addr":"172.21.15.175:6835","nonce":1972426673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24434,24436,24451]},"id":1}]} 2024-06-10T01:24:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:53.301+0000 7f2c40977700 1 -- 172.21.15.7:0/2173882161 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2c240a3060 msgr2=0x7f2c240a5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:53.301+0000 7f2c40977700 1 --2- 172.21.15.7:0/2173882161 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2c240a3060 0x7f2c240a5520 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c101230 tx=0x7f2c30009260 comp rx=0 tx=0).stop 2024-06-10T01:24:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:53.301+0000 7f2c40977700 1 -- 172.21.15.7:0/2173882161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c3c06aa30 msgr2=0x7f2c3c1a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:53.301+0000 7f2c40977700 1 --2- 172.21.15.7:0/2173882161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c3c06aa30 0x7f2c3c1a08b0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c00e8e0 tx=0x7f2c2c00ebf0 comp rx=0 tx=0).stop 2024-06-10T01:24:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:53.301+0000 7f2c40977700 1 -- 172.21.15.7:0/2173882161 shutdown_connections 2024-06-10T01:24:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:53.301+0000 7f2c40977700 1 --2- 172.21.15.7:0/2173882161 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2c240a3060 0x7f2c240a5520 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:53.301+0000 7f2c40977700 1 --2- 172.21.15.7:0/2173882161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c3c10ef30 0x7f2c3c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:53.301+0000 7f2c40977700 1 --2- 172.21.15.7:0/2173882161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c3c06aa30 0x7f2c3c1a08b0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:53.301+0000 7f2c40977700 1 --2- 172.21.15.7:0/2173882161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c3c06b350 0x7f2c3c1a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:53.301+0000 7f2c40977700 1 -- 172.21.15.7:0/2173882161 >> 172.21.15.7:0/2173882161 conn(0x7f2c3c078300 msgr2=0x7f2c3c1101a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:53.301+0000 7f2c40977700 1 -- 172.21.15.7:0/2173882161 shutdown_connections 2024-06-10T01:24:53.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:53.302+0000 7f2c40977700 1 -- 172.21.15.7:0/2173882161 wait complete. 2024-06-10T01:24:53.298 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 51 2024-06-10T01:24:53.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:53 smithi078 ceph-mon[36683]: pgmap v999: 129 pgs: 129 active+clean; 613 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.3 MiB/s wr, 643 op/s 2024-06-10T01:24:53.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:53 smithi175 ceph-mon[36636]: pgmap v999: 129 pgs: 129 active+clean; 613 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.3 MiB/s wr, 643 op/s 2024-06-10T01:24:53.537 INFO:tasks.workunit.client.0.smithi007.stdout: 10 131472 23.31 MB/sec execute 210 sec latency 62.744 ms 2024-06-10T01:24:54.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:54 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2173882161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:24:54.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:54 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2173882161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:24:54.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:54 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2173882161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:24:54.537 INFO:tasks.workunit.client.0.smithi007.stdout: 10 132175 23.38 MB/sec execute 211 sec latency 173.947 ms 2024-06-10T01:24:54.705 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:24:55.101 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:24:55.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:55 smithi078 ceph-mon[36683]: pgmap v1000: 129 pgs: 129 active+clean; 614 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.8 MiB/s wr, 558 op/s 2024-06-10T01:24:55.387 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:55 smithi007 ceph-mon[32398]: pgmap v1000: 129 pgs: 129 active+clean; 614 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.8 MiB/s wr, 558 op/s 2024-06-10T01:24:55.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:55 smithi175 ceph-mon[36636]: pgmap v1000: 129 pgs: 129 active+clean; 614 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.8 MiB/s wr, 558 op/s 2024-06-10T01:24:55.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 132765 23.42 MB/sec execute 212 sec latency 159.833 ms 2024-06-10T01:24:56.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:56 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:24:56.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:56 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:24:56.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:56 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:24:56.537 INFO:tasks.workunit.client.0.smithi007.stdout: 10 133410 23.47 MB/sec execute 213 sec latency 127.726 ms 2024-06-10T01:24:57.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.083+0000 7f6316348700 1 -- 172.21.15.7:0/121714949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f631010ae30 msgr2=0x7f631010b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:57.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.083+0000 7f6316348700 1 --2- 172.21.15.7:0/121714949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f631010ae30 0x7f631010b210 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f630800d210 tx=0x7f630800d520 comp rx=0 tx=0).stop 2024-06-10T01:24:57.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.083+0000 7f6316348700 1 -- 172.21.15.7:0/121714949 shutdown_connections 2024-06-10T01:24:57.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.083+0000 7f6316348700 1 --2- 172.21.15.7:0/121714949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63101143e0 0x7f63101167d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.083+0000 7f6316348700 1 --2- 172.21.15.7:0/121714949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f631010ae30 0x7f631010b210 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.083+0000 7f6316348700 1 --2- 172.21.15.7:0/121714949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f631010b750 0x7f6310113ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.083+0000 7f6316348700 1 -- 172.21.15.7:0/121714949 >> 172.21.15.7:0/121714949 conn(0x7f631006d600 msgr2=0x7f631006da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:57.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.083+0000 7f6316348700 1 -- 172.21.15.7:0/121714949 shutdown_connections 2024-06-10T01:24:57.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.083+0000 7f6316348700 1 -- 172.21.15.7:0/121714949 wait complete. 2024-06-10T01:24:57.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.085+0000 7f6316348700 1 Processor -- start 2024-06-10T01:24:57.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.085+0000 7f6316348700 1 -- start start 2024-06-10T01:24:57.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.085+0000 7f6316348700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f631010ae30 0x7f631019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.085+0000 7f6316348700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f631010b750 0x7f631019f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.085+0000 7f6316348700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63101143e0 0x7f631019fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.085+0000 7f6316348700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6310118e00 con 0x7f63101143e0 2024-06-10T01:24:57.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.085+0000 7f6316348700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6310118c80 con 0x7f631010b750 2024-06-10T01:24:57.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.085+0000 7f6316348700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6310118f80 con 0x7f631010ae30 2024-06-10T01:24:57.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.086+0000 7f630f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f631010b750 0x7f631019f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:57.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.086+0000 7f630f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f631010b750 0x7f631019f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:36212/0 (socket says 172.21.15.7:36212) 2024-06-10T01:24:57.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.086+0000 7f630f7fe700 1 -- 172.21.15.7:0/3600519466 learned_addr learned my addr 172.21.15.7:0/3600519466 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:57.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.086+0000 7f630ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f631010ae30 0x7f631019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:57.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.086+0000 7f63148e5700 1 --2- 172.21.15.7:0/3600519466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63101143e0 0x7f631019fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:57.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.086+0000 7f630f7fe700 1 -- 172.21.15.7:0/3600519466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f631010ae30 msgr2=0x7f631019f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:57.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.086+0000 7f630f7fe700 1 --2- 172.21.15.7:0/3600519466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f631010ae30 0x7f631019f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.086+0000 7f630f7fe700 1 -- 172.21.15.7:0/3600519466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63101143e0 msgr2=0x7f631019fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:57.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.086+0000 7f630f7fe700 1 --2- 172.21.15.7:0/3600519466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63101143e0 0x7f631019fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.086+0000 7f630f7fe700 1 -- 172.21.15.7:0/3600519466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63101b3d50 con 0x7f631010b750 2024-06-10T01:24:57.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.086+0000 7f63148e5700 1 --2- 172.21.15.7:0/3600519466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63101143e0 0x7f631019fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:24:57.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.086+0000 7f630f7fe700 1 --2- 172.21.15.7:0/3600519466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f631010b750 0x7f631019f620 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f630400e9f0 tx=0x7f630400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:57.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.086+0000 7f630d7fa700 1 -- 172.21.15.7:0/3600519466 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630400c9c0 con 0x7f631010b750 2024-06-10T01:24:57.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.086+0000 7f630d7fa700 1 -- 172.21.15.7:0/3600519466 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f630400cb60 con 0x7f631010b750 2024-06-10T01:24:57.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.086+0000 7f6316348700 1 -- 172.21.15.7:0/3600519466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63101b3fe0 con 0x7f631010b750 2024-06-10T01:24:57.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.086+0000 7f630d7fa700 1 -- 172.21.15.7:0/3600519466 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6304018680 con 0x7f631010b750 2024-06-10T01:24:57.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.086+0000 7f6316348700 1 -- 172.21.15.7:0/3600519466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63101b4540 con 0x7f631010b750 2024-06-10T01:24:57.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.089+0000 7f630d7fa700 1 -- 172.21.15.7:0/3600519466 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f6304018a30 con 0x7f631010b750 2024-06-10T01:24:57.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.090+0000 7f630d7fa700 1 --2- 172.21.15.7:0/3600519466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f62f80a2cd0 0x7f62f80a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.090+0000 7f630d7fa700 1 -- 172.21.15.7:0/3600519466 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f63040cf7e0 con 0x7f631010b750 2024-06-10T01:24:57.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.090+0000 7f630ffff700 1 --2- 172.21.15.7:0/3600519466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f62f80a2cd0 0x7f62f80a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:57.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.090+0000 7f630ffff700 1 --2- 172.21.15.7:0/3600519466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f62f80a2cd0 0x7f62f80a5190 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6308000f80 tx=0x7f630800d870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:57.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.091+0000 7f6320bcbb80 1 Processor -- start 2024-06-10T01:24:57.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.091+0000 7f6320bcbb80 1 -- start start 2024-06-10T01:24:57.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.091+0000 7f6320bcbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f832344640 0x55f832344a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.091+0000 7f6320bcbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f832344f60 0x55f83233e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.091+0000 7f6320bcbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f83233ef20 0x55f83233f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.091+0000 7f6320bcbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f832290810 con 0x55f832344640 2024-06-10T01:24:57.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.091+0000 7f6320bcbb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f832238720 con 0x55f83233ef20 2024-06-10T01:24:57.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.091+0000 7f6320bcbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f83228c7d0 con 0x55f832344f60 2024-06-10T01:24:57.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.092+0000 7f630ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f832344640 0x55f832344a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:57.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.092+0000 7f630ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f832344640 0x55f832344a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38870/0 (socket says 172.21.15.7:38870) 2024-06-10T01:24:57.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.092+0000 7f63148e5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f83233ef20 0x55f83233f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:57.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.092+0000 7f630ffff700 1 -- 172.21.15.7:0/3857392086 learned_addr learned my addr 172.21.15.7:0/3857392086 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:57.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.092+0000 7f630f7fe700 1 --2- 172.21.15.7:0/3857392086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f832344f60 0x55f83233e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:57.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.092+0000 7f630ffff700 1 -- 172.21.15.7:0/3857392086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f832344f60 msgr2=0x55f83233e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:57.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.092+0000 7f630ffff700 1 --2- 172.21.15.7:0/3857392086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f832344f60 0x55f83233e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.092+0000 7f630ffff700 1 -- 172.21.15.7:0/3857392086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f83233ef20 msgr2=0x55f83233f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:57.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.092+0000 7f630ffff700 1 --2- 172.21.15.7:0/3857392086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f83233ef20 0x55f83233f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.092+0000 7f630ffff700 1 -- 172.21.15.7:0/3857392086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f83222ef50 con 0x55f832344640 2024-06-10T01:24:57.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.093+0000 7f630ffff700 1 --2- 172.21.15.7:0/3857392086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f832344640 0x55f832344a20 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f6308003b30 tx=0x7f630801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9b8d3d5e40bf0e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:57.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.093+0000 7f62f67fc700 1 -- 172.21.15.7:0/3857392086 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6308022040 con 0x55f832344640 2024-06-10T01:24:57.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.093+0000 7f62f67fc700 1 -- 172.21.15.7:0/3857392086 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f630802e070 con 0x55f832344640 2024-06-10T01:24:57.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.093+0000 7f62f67fc700 1 -- 172.21.15.7:0/3857392086 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6308005620 con 0x55f832344640 2024-06-10T01:24:57.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.094+0000 7f6320bcbb80 1 -- 172.21.15.7:0/3857392086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f832344640 msgr2=0x55f832344a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:57.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.094+0000 7f6320bcbb80 1 --2- 172.21.15.7:0/3857392086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f832344640 0x55f832344a20 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f6308003b30 tx=0x7f630801e000 comp rx=0 tx=0).stop 2024-06-10T01:24:57.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.095+0000 7f6320bcbb80 1 -- 172.21.15.7:0/3857392086 shutdown_connections 2024-06-10T01:24:57.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.095+0000 7f6320bcbb80 1 --2- 172.21.15.7:0/3857392086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f83233ef20 0x55f83233f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.095+0000 7f6320bcbb80 1 --2- 172.21.15.7:0/3857392086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f832344640 0x55f832344a20 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.095+0000 7f6320bcbb80 1 --2- 172.21.15.7:0/3857392086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f832344f60 0x55f83233e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.095+0000 7f6320bcbb80 1 -- 172.21.15.7:0/3857392086 >> 172.21.15.7:0/3857392086 conn(0x55f832269cc0 msgr2=0x55f832339850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:57.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.097+0000 7f6320bcbb80 1 -- 172.21.15.7:0/3857392086 shutdown_connections 2024-06-10T01:24:57.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.097+0000 7f6320bcbb80 1 -- 172.21.15.7:0/3857392086 wait complete. 2024-06-10T01:24:57.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.098+0000 7f6320bcbb80 1 -- start start 2024-06-10T01:24:57.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.099+0000 7f6320bcbb80 1 Processor -- start 2024-06-10T01:24:57.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.099+0000 7f6320bcbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f83233ef20 0x55f83234acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.099+0000 7f6320bcbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f832344640 0x55f832347fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.099+0000 7f6320bcbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f832344f60 0x55f832348510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.100+0000 7f6320bcbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f83234b970 con 0x55f83233ef20 2024-06-10T01:24:57.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.100+0000 7f6320bcbb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f832230bf0 con 0x55f832344f60 2024-06-10T01:24:57.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.100+0000 7f6320bcbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f83234baf0 con 0x55f832344640 2024-06-10T01:24:57.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.100+0000 7f63148e5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f832344f60 0x55f832348510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:57.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.100+0000 7f63148e5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f832344f60 0x55f832348510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:36226/0 (socket says 172.21.15.7:36226) 2024-06-10T01:24:57.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.100+0000 7f6320bcbb80 10 client.0 _ll_register_callbacks cb 0x55f83222a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:24:57.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.100+0000 7f63148e5700 1 -- 172.21.15.7:0/3571142426 learned_addr learned my addr 172.21.15.7:0/3571142426 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:57.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.100+0000 7f630f7fe700 1 --2- 172.21.15.7:0/3571142426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f832344640 0x55f832347fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:57.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.101+0000 7f63148e5700 1 -- 172.21.15.7:0/3571142426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f832344640 msgr2=0x55f832347fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:57.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.101+0000 7f63148e5700 1 --2- 172.21.15.7:0/3571142426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f832344640 0x55f832347fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.101+0000 7f63148e5700 1 -- 172.21.15.7:0/3571142426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f83233ef20 msgr2=0x55f83234acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:57.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.101+0000 7f63148e5700 1 --2- 172.21.15.7:0/3571142426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f83233ef20 0x55f83234acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.101+0000 7f63148e5700 1 -- 172.21.15.7:0/3571142426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8321df8d0 con 0x55f832344f60 2024-06-10T01:24:57.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.101+0000 7f63148e5700 1 --2- 172.21.15.7:0/3571142426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f832344f60 0x55f832348510 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6300012700 tx=0x7f6300012a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c52a0810804a2100 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:57.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.101+0000 7f62ef7fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:24:57.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.101+0000 7f62ef7fe700 1 -- 172.21.15.7:0/3571142426 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6300004830 con 0x55f832344f60 2024-06-10T01:24:57.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.102+0000 7f6320bcbb80 1 -- 172.21.15.7:0/3571142426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f630802e070 con 0x55f832344f60 2024-06-10T01:24:57.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.102+0000 7f62ef7fe700 1 -- 172.21.15.7:0/3571142426 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f63000049d0 con 0x55f832344f60 2024-06-10T01:24:57.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.102+0000 7f62ef7fe700 1 -- 172.21.15.7:0/3571142426 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f6300004c10 con 0x55f832344f60 2024-06-10T01:24:57.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.102+0000 7f6320bcbb80 10 client.29240 fetch_fsmap learned FSMap version 51 2024-06-10T01:24:57.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.102+0000 7f6320bcbb80 1 -- 172.21.15.7:0/3571142426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55f832356330 con 0x55f832344f60 2024-06-10T01:24:57.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.102+0000 7f62ef7fe700 1 -- 172.21.15.7:0/3571142426 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f630802e070 con 0x55f832344f60 2024-06-10T01:24:57.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.103+0000 7f62ef7fe700 1 -- 172.21.15.7:0/3571142426 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 51) ==== 8350+0+0 (secure 0 0 0) 0x7f6300005380 con 0x55f832344f60 2024-06-10T01:24:57.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.103+0000 7f6320bcbb80 10 client.29240 fetch_fsmap finished waiting for FSMap version 51 2024-06-10T01:24:57.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.103+0000 7f6320bcbb80 10 client.29240 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:24:57.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.103+0000 7f6320bcbb80 20 client.29240 populate_metadata read hostname 'smithi007' 2024-06-10T01:24:57.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.103+0000 7f6320bcbb80 1 --2- 172.21.15.7:0/3571142426 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55f832358970 0x55f83235ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.103+0000 7f6320bcbb80 4 client.29240 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:24:57.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.103+0000 7f6320bcbb80 1 -- 172.21.15.7:0/3571142426 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f83229dd10 con 0x55f832358970 2024-06-10T01:24:57.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.103+0000 7f630ffff700 1 --2- 172.21.15.7:0/3571142426 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55f832358970 0x55f83235ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:57.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.104+0000 7f62ef7fe700 10 client.29240 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:24:57.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.106+0000 7f630ffff700 1 --2- 172.21.15.7:0/3571142426 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55f832358970 0x55f83235ad60 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=13fe259bd4b60941 server_cookie=81f69eca448d953e in_seq=0 out_seq=0 2024-06-10T01:24:57.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.107+0000 7f62ef7fe700 1 -- 172.21.15.7:0/3571142426 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f83229dd10 con 0x55f832358970 2024-06-10T01:24:57.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.107+0000 7f62ef7fe700 10 client.29240 handle_command_reply: tid=0 2024-06-10T01:24:57.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.107+0000 7f6320bcbb80 1 client.29240 shutdown 2024-06-10T01:24:57.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.107+0000 7f6320bcbb80 1 -- 172.21.15.7:0/3571142426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f832344f60 msgr2=0x55f832348510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:57.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.107+0000 7f6320bcbb80 1 --2- 172.21.15.7:0/3571142426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f832344f60 0x55f832348510 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6300012700 tx=0x7f6300012a10 comp rx=0 tx=0).stop 2024-06-10T01:24:57.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.107+0000 7f6320bcbb80 1 -- 172.21.15.7:0/3571142426 shutdown_connections 2024-06-10T01:24:57.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.107+0000 7f6320bcbb80 1 --2- 172.21.15.7:0/3571142426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f832344f60 0x55f832348510 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.107+0000 7f6320bcbb80 1 --2- 172.21.15.7:0/3571142426 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55f832358970 0x55f83235ad60 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.107+0000 7f6320bcbb80 1 --2- 172.21.15.7:0/3571142426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f83233ef20 0x55f83234acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.107+0000 7f6320bcbb80 1 --2- 172.21.15.7:0/3571142426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f832344640 0x55f832347fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.108+0000 7f6320bcbb80 1 -- 172.21.15.7:0/3571142426 >> 172.21.15.7:0/3571142426 conn(0x55f832269cc0 msgr2=0x55f8323395f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:57.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.108+0000 7f62ef7fe700 0 client.29240 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:24:57.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.108+0000 7f6320bcbb80 1 -- 172.21.15.7:0/3571142426 shutdown_connections 2024-06-10T01:24:57.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.109+0000 7f6320bcbb80 1 -- 172.21.15.7:0/3571142426 wait complete. 2024-06-10T01:24:57.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.109+0000 7f6320bcbb80 20 client.29240 trim_cache size 0 max 16384 2024-06-10T01:24:57.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.141+0000 7f6320bcbb80 1 Processor -- start 2024-06-10T01:24:57.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.142+0000 7f6320bcbb80 1 -- start start 2024-06-10T01:24:57.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.142+0000 7f6320bcbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f83233ef20 0x55f83233e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.142+0000 7f6320bcbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f832344640 0x55f832347f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.142+0000 7f6320bcbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f832344f60 0x55f8323484d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.142+0000 7f6320bcbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f63101bfa70 con 0x55f83233ef20 2024-06-10T01:24:57.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.142+0000 7f6320bcbb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f63101bd300 con 0x55f832344640 2024-06-10T01:24:57.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.142+0000 7f6320bcbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f63101bb150 con 0x55f832344f60 2024-06-10T01:24:57.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.142+0000 7f630ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f83233ef20 0x55f83233e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:57.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.142+0000 7f630ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f83233ef20 0x55f83233e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38888/0 (socket says 172.21.15.7:38888) 2024-06-10T01:24:57.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.142+0000 7f630ffff700 1 -- 172.21.15.7:0/2341667840 learned_addr learned my addr 172.21.15.7:0/2341667840 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:57.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.142+0000 7f63148e5700 1 --2- 172.21.15.7:0/2341667840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f832344f60 0x55f8323484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:57.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.142+0000 7f630f7fe700 1 --2- 172.21.15.7:0/2341667840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f832344640 0x55f832347f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:57.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.143+0000 7f630ffff700 1 -- 172.21.15.7:0/2341667840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f832344f60 msgr2=0x55f8323484d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:57.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.143+0000 7f630ffff700 1 --2- 172.21.15.7:0/2341667840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f832344f60 0x55f8323484d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.143+0000 7f630ffff700 1 -- 172.21.15.7:0/2341667840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f832344640 msgr2=0x55f832347f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:57.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.143+0000 7f630ffff700 1 --2- 172.21.15.7:0/2341667840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f832344640 0x55f832347f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.143+0000 7f630ffff700 1 -- 172.21.15.7:0/2341667840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f83233a470 con 0x55f83233ef20 2024-06-10T01:24:57.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.143+0000 7f630ffff700 1 --2- 172.21.15.7:0/2341667840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f83233ef20 0x55f83233e9e0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f630800b3b0 tx=0x7f6308005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea0518fc4165fd2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:57.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.143+0000 7f62f5ffb700 1 -- 172.21.15.7:0/2341667840 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6308022040 con 0x55f83233ef20 2024-06-10T01:24:57.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.144+0000 7f62f5ffb700 1 -- 172.21.15.7:0/2341667840 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6308037070 con 0x55f83233ef20 2024-06-10T01:24:57.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.144+0000 7f62f5ffb700 1 -- 172.21.15.7:0/2341667840 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630800add0 con 0x55f83233ef20 2024-06-10T01:24:57.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.145+0000 7f6320bcbb80 1 -- 172.21.15.7:0/2341667840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f83233ef20 msgr2=0x55f83233e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:57.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.145+0000 7f6320bcbb80 1 --2- 172.21.15.7:0/2341667840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f83233ef20 0x55f83233e9e0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f630800b3b0 tx=0x7f6308005040 comp rx=0 tx=0).stop 2024-06-10T01:24:57.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.145+0000 7f6320bcbb80 1 -- 172.21.15.7:0/2341667840 shutdown_connections 2024-06-10T01:24:57.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.145+0000 7f6320bcbb80 1 --2- 172.21.15.7:0/2341667840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f832344f60 0x55f8323484d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.145+0000 7f6320bcbb80 1 --2- 172.21.15.7:0/2341667840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f83233ef20 0x55f83233e9e0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.145+0000 7f6320bcbb80 1 --2- 172.21.15.7:0/2341667840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f832344640 0x55f832347f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.145+0000 7f6320bcbb80 1 -- 172.21.15.7:0/2341667840 >> 172.21.15.7:0/2341667840 conn(0x55f832269cc0 msgr2=0x55f832342190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:57.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.145+0000 7f6320bcbb80 1 -- 172.21.15.7:0/2341667840 shutdown_connections 2024-06-10T01:24:57.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.145+0000 7f6320bcbb80 1 -- 172.21.15.7:0/2341667840 wait complete. 2024-06-10T01:24:57.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.146+0000 7f6320bcbb80 1 -- start start 2024-06-10T01:24:57.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.146+0000 7f6320bcbb80 1 Processor -- start 2024-06-10T01:24:57.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.146+0000 7f6320bcbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f83233ef20 0x55f832350de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.146+0000 7f6320bcbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f832344640 0x55f832351320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.146+0000 7f6320bcbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f832344f60 0x55f83235f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.146+0000 7f6320bcbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f63101bb7c0 con 0x55f832344f60 2024-06-10T01:24:57.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.146+0000 7f6320bcbb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f63101bcc30 con 0x55f83233ef20 2024-06-10T01:24:57.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.146+0000 7f6320bcbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f63101bbe90 con 0x55f832344640 2024-06-10T01:24:57.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.146+0000 7f6320bcbb80 10 client.0 _ll_register_callbacks cb 0x55f83222a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:24:57.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.147+0000 7f63148e5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f832344f60 0x55f83235f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:57.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.147+0000 7f63148e5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f832344f60 0x55f83235f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38890/0 (socket says 172.21.15.7:38890) 2024-06-10T01:24:57.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.147+0000 7f63148e5700 1 -- 172.21.15.7:0/3125490180 learned_addr learned my addr 172.21.15.7:0/3125490180 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:57.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.147+0000 7f630ffff700 1 --2- 172.21.15.7:0/3125490180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f83233ef20 0x55f832350de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:57.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.147+0000 7f630f7fe700 1 --2- 172.21.15.7:0/3125490180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f832344640 0x55f832351320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:57.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.147+0000 7f630ffff700 1 -- 172.21.15.7:0/3125490180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f832344640 msgr2=0x55f832351320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:57.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.147+0000 7f630ffff700 1 --2- 172.21.15.7:0/3125490180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f832344640 0x55f832351320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.147+0000 7f630ffff700 1 -- 172.21.15.7:0/3125490180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f832344f60 msgr2=0x55f83235f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:57.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.147+0000 7f630ffff700 1 --2- 172.21.15.7:0/3125490180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f832344f60 0x55f83235f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.147+0000 7f630ffff700 1 -- 172.21.15.7:0/3125490180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f832344140 con 0x55f83233ef20 2024-06-10T01:24:57.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.147+0000 7f630ffff700 1 --2- 172.21.15.7:0/3125490180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f83233ef20 0x55f832350de0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f630800f010 tx=0x7f630800b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6141079ae3b1ee29 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:57.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.147+0000 7f62ef7fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:24:57.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.148+0000 7f62ef7fe700 1 -- 172.21.15.7:0/3125490180 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6308048060 con 0x55f83233ef20 2024-06-10T01:24:57.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.148+0000 7f62ef7fe700 1 -- 172.21.15.7:0/3125490180 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f6308005470 con 0x55f83233ef20 2024-06-10T01:24:57.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.148+0000 7f6320bcbb80 1 -- 172.21.15.7:0/3125490180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6308037070 con 0x55f83233ef20 2024-06-10T01:24:57.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.148+0000 7f62ef7fe700 1 -- 172.21.15.7:0/3125490180 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f6308022050 con 0x55f83233ef20 2024-06-10T01:24:57.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.148+0000 7f6320bcbb80 10 client.29246 fetch_fsmap learned FSMap version 51 2024-06-10T01:24:57.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.148+0000 7f6320bcbb80 1 -- 172.21.15.7:0/3125490180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55f8323c2a60 con 0x55f83233ef20 2024-06-10T01:24:57.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.148+0000 7f62ef7fe700 1 -- 172.21.15.7:0/3125490180 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6308037070 con 0x55f83233ef20 2024-06-10T01:24:57.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.149+0000 7f62ef7fe700 1 -- 172.21.15.7:0/3125490180 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 51) ==== 8350+0+0 (secure 0 0 0) 0x7f6308028040 con 0x55f83233ef20 2024-06-10T01:24:57.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.149+0000 7f6320bcbb80 10 client.29246 fetch_fsmap finished waiting for FSMap version 51 2024-06-10T01:24:57.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.149+0000 7f6320bcbb80 10 client.29246 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:24:57.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.149+0000 7f6320bcbb80 20 client.29246 populate_metadata read hostname 'smithi007' 2024-06-10T01:24:57.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.149+0000 7f6320bcbb80 1 --2- 172.21.15.7:0/3125490180 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55f8323c2c80 0x55f8323c5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.149+0000 7f6320bcbb80 4 client.29246 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:24:57.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.149+0000 7f6320bcbb80 1 -- 172.21.15.7:0/3125490180 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f832298290 con 0x55f8323c2c80 2024-06-10T01:24:57.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.149+0000 7f630f7fe700 1 --2- 172.21.15.7:0/3125490180 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55f8323c2c80 0x55f8323c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:57.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.149+0000 7f62ef7fe700 10 client.29246 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:24:57.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.149+0000 7f630f7fe700 1 --2- 172.21.15.7:0/3125490180 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55f8323c2c80 0x55f8323c5070 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=85a82407ad0c954f server_cookie=93d8a35e983af803 in_seq=0 out_seq=0 2024-06-10T01:24:57.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.150+0000 7f62ef7fe700 1 -- 172.21.15.7:0/3125490180 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f832298290 con 0x55f8323c2c80 2024-06-10T01:24:57.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.150+0000 7f62ef7fe700 10 client.29246 handle_command_reply: tid=0 2024-06-10T01:24:57.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.150+0000 7f6320bcbb80 1 client.29246 shutdown 2024-06-10T01:24:57.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.150+0000 7f6320bcbb80 1 -- 172.21.15.7:0/3125490180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f83233ef20 msgr2=0x55f832350de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:57.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.150+0000 7f6320bcbb80 1 --2- 172.21.15.7:0/3125490180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f83233ef20 0x55f832350de0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f630800f010 tx=0x7f630800b710 comp rx=0 tx=0).stop 2024-06-10T01:24:57.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.151+0000 7f6320bcbb80 1 -- 172.21.15.7:0/3125490180 shutdown_connections 2024-06-10T01:24:57.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.151+0000 7f6320bcbb80 1 --2- 172.21.15.7:0/3125490180 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55f8323c2c80 0x55f8323c5070 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.151+0000 7f6320bcbb80 1 --2- 172.21.15.7:0/3125490180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f832344f60 0x55f83235f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.151+0000 7f62ef7fe700 0 client.29246 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:24:57.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.151+0000 7f6320bcbb80 1 --2- 172.21.15.7:0/3125490180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f83233ef20 0x55f832350de0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.151+0000 7f6320bcbb80 1 --2- 172.21.15.7:0/3125490180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f832344640 0x55f832351320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.151+0000 7f6320bcbb80 1 -- 172.21.15.7:0/3125490180 >> 172.21.15.7:0/3125490180 conn(0x55f832269cc0 msgr2=0x55f832345400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:57.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.151+0000 7f6320bcbb80 1 -- 172.21.15.7:0/3125490180 shutdown_connections 2024-06-10T01:24:57.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.151+0000 7f6320bcbb80 1 -- 172.21.15.7:0/3125490180 wait complete. 2024-06-10T01:24:57.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.151+0000 7f6320bcbb80 20 client.29246 trim_cache size 0 max 16384 2024-06-10T01:24:57.146 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:24:57.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.152+0000 7f6316348700 1 -- 172.21.15.7:0/3600519466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f62f80a2cd0 msgr2=0x7f62f80a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:57.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.152+0000 7f6316348700 1 --2- 172.21.15.7:0/3600519466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f62f80a2cd0 0x7f62f80a5190 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6308000f80 tx=0x7f630800d870 comp rx=0 tx=0).stop 2024-06-10T01:24:57.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.152+0000 7f6316348700 1 -- 172.21.15.7:0/3600519466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f631010b750 msgr2=0x7f631019f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:57.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.152+0000 7f6316348700 1 --2- 172.21.15.7:0/3600519466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f631010b750 0x7f631019f620 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f630400e9f0 tx=0x7f630400ed00 comp rx=0 tx=0).stop 2024-06-10T01:24:57.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.152+0000 7f6316348700 1 -- 172.21.15.7:0/3600519466 shutdown_connections 2024-06-10T01:24:57.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.152+0000 7f6316348700 1 --2- 172.21.15.7:0/3600519466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f62f80a2cd0 0x7f62f80a5190 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.152+0000 7f6316348700 1 --2- 172.21.15.7:0/3600519466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63101143e0 0x7f631019fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.152+0000 7f6316348700 1 --2- 172.21.15.7:0/3600519466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f631010ae30 0x7f631019f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.152+0000 7f6316348700 1 --2- 172.21.15.7:0/3600519466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f631010b750 0x7f631019f620 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.152+0000 7f6316348700 1 -- 172.21.15.7:0/3600519466 >> 172.21.15.7:0/3600519466 conn(0x7f631006d600 msgr2=0x7f63100712b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:57.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.152+0000 7f6316348700 1 -- 172.21.15.7:0/3600519466 shutdown_connections 2024-06-10T01:24:57.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:24:57.152+0000 7f6316348700 1 -- 172.21.15.7:0/3600519466 wait complete. 2024-06-10T01:24:57.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:57 smithi007 ceph-mon[32398]: pgmap v1001: 129 pgs: 129 active+clean; 629 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 688 op/s 2024-06-10T01:24:57.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:57 smithi078 ceph-mon[36683]: pgmap v1001: 129 pgs: 129 active+clean; 629 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 688 op/s 2024-06-10T01:24:57.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:57 smithi175 ceph-mon[36636]: pgmap v1001: 129 pgs: 129 active+clean; 629 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 688 op/s 2024-06-10T01:24:57.537 INFO:tasks.workunit.client.0.smithi007.stdout: 10 134221 23.58 MB/sec execute 214 sec latency 79.530 ms 2024-06-10T01:24:57.894 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:24:57.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.931+0000 7fa8334a7640 1 -- 172.21.15.7:0/828596197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa824096830 msgr2=0x7fa82409d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:57.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.931+0000 7fa8334a7640 1 --2- 172.21.15.7:0/828596197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa824096830 0x7fa82409d0c0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fa82c067110 tx=0x7fa82c0963b0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.932+0000 7fa8334a7640 1 -- 172.21.15.7:0/828596197 shutdown_connections 2024-06-10T01:24:57.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.932+0000 7fa8334a7640 1 --2- 172.21.15.7:0/828596197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa824096830 0x7fa82409d0c0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.932+0000 7fa8334a7640 1 --2- 172.21.15.7:0/828596197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa824095e70 0x7fa8240962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.932+0000 7fa8334a7640 1 --2- 172.21.15.7:0/828596197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa824094c80 0x7fa8240950a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.932+0000 7fa8334a7640 1 -- 172.21.15.7:0/828596197 >> 172.21.15.7:0/828596197 conn(0x7fa8240901d0 msgr2=0x7fa824092630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:57.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.932+0000 7fa8334a7640 1 -- 172.21.15.7:0/828596197 shutdown_connections 2024-06-10T01:24:57.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.932+0000 7fa8334a7640 1 -- 172.21.15.7:0/828596197 wait complete. 2024-06-10T01:24:57.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.932+0000 7fa8334a7640 1 Processor -- start 2024-06-10T01:24:57.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.933+0000 7fa8334a7640 1 -- start start 2024-06-10T01:24:57.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.933+0000 7fa8334a7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa824094c80 0x7fa824098c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.933+0000 7fa8334a7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa824095e70 0x7fa824097250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.933+0000 7fa8334a7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa824096830 0x7fa824097790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.933+0000 7fa8334a7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa8240a0830 con 0x7fa824095e70 2024-06-10T01:24:57.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.933+0000 7fa8334a7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa8240a06b0 con 0x7fa824094c80 2024-06-10T01:24:57.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.933+0000 7fa8334a7640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa8240a09b0 con 0x7fa824096830 2024-06-10T01:24:57.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.933+0000 7fa831ca4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa824095e70 0x7fa824097250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:57.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.933+0000 7fa831ca4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa824095e70 0x7fa824097250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38906/0 (socket says 172.21.15.7:38906) 2024-06-10T01:24:57.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.933+0000 7fa831ca4640 1 -- 172.21.15.7:0/3443095603 learned_addr learned my addr 172.21.15.7:0/3443095603 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:57.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.934+0000 7fa831ca4640 1 -- 172.21.15.7:0/3443095603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa824096830 msgr2=0x7fa824097790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:57.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.934+0000 7fa831ca4640 1 --2- 172.21.15.7:0/3443095603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa824096830 0x7fa824097790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.934+0000 7fa831ca4640 1 -- 172.21.15.7:0/3443095603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa824094c80 msgr2=0x7fa824098c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:57.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.934+0000 7fa831ca4640 1 --2- 172.21.15.7:0/3443095603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa824094c80 0x7fa824098c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.934+0000 7fa831ca4640 1 -- 172.21.15.7:0/3443095603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa824097cd0 con 0x7fa824095e70 2024-06-10T01:24:57.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.934+0000 7fa831ca4640 1 --2- 172.21.15.7:0/3443095603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa824095e70 0x7fa824097250 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00ea60 tx=0x7fa81c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:57.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.934+0000 7fa8237fe640 1 -- 172.21.15.7:0/3443095603 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c00cde0 con 0x7fa824095e70 2024-06-10T01:24:57.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.935+0000 7fa8237fe640 1 -- 172.21.15.7:0/3443095603 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa81c0035f0 con 0x7fa824095e70 2024-06-10T01:24:57.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.935+0000 7fa8237fe640 1 -- 172.21.15.7:0/3443095603 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c010720 con 0x7fa824095e70 2024-06-10T01:24:57.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.936+0000 7fa8334a7640 1 -- 172.21.15.7:0/3443095603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa824097fc0 con 0x7fa824095e70 2024-06-10T01:24:57.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.936+0000 7fa8334a7640 1 -- 172.21.15.7:0/3443095603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa82413a9c0 con 0x7fa824095e70 2024-06-10T01:24:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.941+0000 7fa83740d740 1 Processor -- start 2024-06-10T01:24:57.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.942+0000 7fa8237fe640 1 -- 172.21.15.7:0/3443095603 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa81c003b30 con 0x7fa824095e70 2024-06-10T01:24:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.943+0000 7fa83740d740 1 -- start start 2024-06-10T01:24:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.943+0000 7fa83740d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56312e6ae200 0x56312e6ae620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.943+0000 7fa83740d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56312e6aebf0 0x56312e6b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.943+0000 7fa83740d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56312e6b3d60 0x56312e6b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.943+0000 7fa83740d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56312e6114f0 con 0x56312e6b3d60 2024-06-10T01:24:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.943+0000 7fa83740d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56312e5af870 con 0x56312e6ae200 2024-06-10T01:24:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.943+0000 7fa83740d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56312e60d4b0 con 0x56312e6aebf0 2024-06-10T01:24:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.944+0000 7fa832ca6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56312e6ae200 0x56312e6ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.944+0000 7fa832ca6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56312e6ae200 0x56312e6ae620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:36272/0 (socket says 172.21.15.7:36272) 2024-06-10T01:24:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.944+0000 7fa832ca6640 1 -- 172.21.15.7:0/2725143715 learned_addr learned my addr 172.21.15.7:0/2725143715 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.944+0000 7fa8237fe640 1 --2- 172.21.15.7:0/3443095603 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa8180a2d60 0x7fa8180a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.944+0000 7fa831ca4640 1 --2- 172.21.15.7:0/2725143715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56312e6b3d60 0x56312e6b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.945+0000 7fa8237fe640 1 -- 172.21.15.7:0/3443095603 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fa81c0cf660 con 0x7fa824095e70 2024-06-10T01:24:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.945+0000 7fa8324a5640 1 --2- 172.21.15.7:0/2725143715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56312e6aebf0 0x56312e6b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.945+0000 7fa832ca6640 1 -- 172.21.15.7:0/2725143715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56312e6aebf0 msgr2=0x56312e6b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.945+0000 7fa832ca6640 1 --2- 172.21.15.7:0/2725143715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56312e6aebf0 0x56312e6b36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.945+0000 7fa832ca6640 1 -- 172.21.15.7:0/2725143715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56312e6b3d60 msgr2=0x56312e6b61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.945+0000 7fa832ca6640 1 --2- 172.21.15.7:0/2725143715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56312e6b3d60 0x56312e6b61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.945+0000 7fa832ca6640 1 -- 172.21.15.7:0/2725143715 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56312e5cb9b0 con 0x56312e6ae200 2024-06-10T01:24:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.946+0000 7fa832ca6640 1 --2- 172.21.15.7:0/2725143715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56312e6ae200 0x56312e6ae620 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa82c050f40 tx=0x7fa82c068f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25ddfb3809b8f054 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.946+0000 7fa832ca6640 1 --2- 172.21.15.7:0/3443095603 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa8180a2d60 0x7fa8180a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.946+0000 7fa820ff9640 1 -- 172.21.15.7:0/2725143715 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c06a110 con 0x56312e6ae200 2024-06-10T01:24:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.946+0000 7fa820ff9640 1 -- 172.21.15.7:0/2725143715 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa82c09a040 con 0x56312e6ae200 2024-06-10T01:24:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.947+0000 7fa820ff9640 1 -- 172.21.15.7:0/2725143715 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c09fa80 con 0x56312e6ae200 2024-06-10T01:24:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.947+0000 7fa83740d740 1 -- 172.21.15.7:0/2725143715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56312e6ae200 msgr2=0x56312e6ae620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.947+0000 7fa83740d740 1 --2- 172.21.15.7:0/2725143715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56312e6ae200 0x56312e6ae620 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa82c050f40 tx=0x7fa82c068f10 comp rx=0 tx=0).stop 2024-06-10T01:24:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.950+0000 7fa83740d740 1 -- 172.21.15.7:0/2725143715 shutdown_connections 2024-06-10T01:24:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.950+0000 7fa83740d740 1 --2- 172.21.15.7:0/2725143715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56312e6b3d60 0x56312e6b61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.950+0000 7fa83740d740 1 --2- 172.21.15.7:0/2725143715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56312e6aebf0 0x56312e6b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.950+0000 7fa83740d740 1 --2- 172.21.15.7:0/2725143715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56312e6ae200 0x56312e6ae620 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.950+0000 7fa83740d740 1 -- 172.21.15.7:0/2725143715 >> 172.21.15.7:0/2725143715 conn(0x56312e6a9870 msgr2=0x56312e6a9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.950+0000 7fa83740d740 1 -- 172.21.15.7:0/2725143715 shutdown_connections 2024-06-10T01:24:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.950+0000 7fa832ca6640 1 --2- 172.21.15.7:0/3443095603 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa8180a2d60 0x7fa8180a5220 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x56312e6b6960 tx=0x7fa82c09fe10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.950+0000 7fa83740d740 1 -- 172.21.15.7:0/2725143715 wait complete. 2024-06-10T01:24:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.950+0000 7fa83740d740 1 -- start start 2024-06-10T01:24:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.951+0000 7fa83740d740 1 Processor -- start 2024-06-10T01:24:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.952+0000 7fa83740d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56312e6ae200 0x56312e6b83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.952+0000 7fa83740d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56312e6aebf0 0x56312e6b8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.952+0000 7fa83740d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56312e6b3d60 0x56312e6bf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.952+0000 7fa83740d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56312e6b6ca0 con 0x56312e6ae200 2024-06-10T01:24:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.952+0000 7fa83740d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56312e59aae0 con 0x56312e6aebf0 2024-06-10T01:24:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.952+0000 7fa83740d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56312e6b6e20 con 0x56312e6b3d60 2024-06-10T01:24:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.952+0000 7fa83740d740 10 client.0 _ll_register_callbacks cb 0x56312e5d39b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:24:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.952+0000 7fa831ca4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56312e6aebf0 0x56312e6b8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.952+0000 7fa831ca4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56312e6aebf0 0x56312e6b8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:36286/0 (socket says 172.21.15.7:36286) 2024-06-10T01:24:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.952+0000 7fa831ca4640 1 -- 172.21.15.7:0/1049445764 learned_addr learned my addr 172.21.15.7:0/1049445764 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.952+0000 7fa832ca6640 1 --2- 172.21.15.7:0/1049445764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56312e6b3d60 0x56312e6bf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.952+0000 7fa8324a5640 1 --2- 172.21.15.7:0/1049445764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56312e6ae200 0x56312e6b83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.953+0000 7fa831ca4640 1 -- 172.21.15.7:0/1049445764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56312e6b3d60 msgr2=0x56312e6bf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.953+0000 7fa831ca4640 1 --2- 172.21.15.7:0/1049445764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56312e6b3d60 0x56312e6bf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.953+0000 7fa831ca4640 1 -- 172.21.15.7:0/1049445764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56312e6ae200 msgr2=0x56312e6b83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.953+0000 7fa831ca4640 1 --2- 172.21.15.7:0/1049445764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56312e6ae200 0x56312e6b83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.953+0000 7fa831ca4640 1 -- 172.21.15.7:0/1049445764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56312e325210 con 0x56312e6aebf0 2024-06-10T01:24:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.954+0000 7fa831ca4640 1 --2- 172.21.15.7:0/1049445764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56312e6aebf0 0x56312e6b8930 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa81c09b7e0 tx=0x7fa81c06aae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d345ac63b96307d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.954+0000 7fa815ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:24:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.954+0000 7fa815ffb640 1 -- 172.21.15.7:0/1049445764 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c09fe90 con 0x56312e6aebf0 2024-06-10T01:24:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.954+0000 7fa815ffb640 1 -- 172.21.15.7:0/1049445764 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fa81c0037a0 con 0x56312e6aebf0 2024-06-10T01:24:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.955+0000 7fa83740d740 1 -- 172.21.15.7:0/1049445764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa82c09a040 con 0x56312e6aebf0 2024-06-10T01:24:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.960+0000 7fa815ffb640 1 -- 172.21.15.7:0/1049445764 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fa81c0a4750 con 0x56312e6aebf0 2024-06-10T01:24:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.962+0000 7fa83740d740 10 client.29255 fetch_fsmap learned FSMap version 51 2024-06-10T01:24:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.962+0000 7fa83740d740 1 -- 172.21.15.7:0/1049445764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x56312e3f1620 con 0x56312e6aebf0 2024-06-10T01:24:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.962+0000 7fa815ffb640 1 -- 172.21.15.7:0/1049445764 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa82c09a040 con 0x56312e6aebf0 2024-06-10T01:24:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.962+0000 7fa815ffb640 1 -- 172.21.15.7:0/1049445764 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 51) ==== 8350+0+0 (secure 0 0 0) 0x7fa81c0b4ce0 con 0x56312e6aebf0 2024-06-10T01:24:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.964+0000 7fa83740d740 10 client.29255 fetch_fsmap finished waiting for FSMap version 51 2024-06-10T01:24:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.964+0000 7fa83740d740 10 client.29255 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:24:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.965+0000 7fa83740d740 20 client.29255 populate_metadata read hostname 'smithi007' 2024-06-10T01:24:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.965+0000 7fa83740d740 1 --2- 172.21.15.7:0/1049445764 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56312e6c4220 0x56312e6c6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.965+0000 7fa83740d740 4 client.29255 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:24:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.965+0000 7fa83740d740 1 -- 172.21.15.7:0/1049445764 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56312e624f10 con 0x56312e6c4220 2024-06-10T01:24:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.966+0000 7fa8324a5640 1 --2- 172.21.15.7:0/1049445764 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56312e6c4220 0x56312e6c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.972+0000 7fa815ffb640 10 client.29255 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.972+0000 7fa8324a5640 1 --2- 172.21.15.7:0/1049445764 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56312e6c4220 0x56312e6c6690 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2405370d14ae3901 server_cookie=f0404164540d1d20 in_seq=0 out_seq=0 2024-06-10T01:24:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.978+0000 7fa815ffb640 1 -- 172.21.15.7:0/1049445764 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56312e624f10 con 0x56312e6c4220 2024-06-10T01:24:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.978+0000 7fa815ffb640 10 client.29255 handle_command_reply: tid=0 2024-06-10T01:24:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.978+0000 7fa83740d740 1 client.29255 shutdown 2024-06-10T01:24:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.978+0000 7fa83740d740 1 -- 172.21.15.7:0/1049445764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56312e6aebf0 msgr2=0x56312e6b8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.978+0000 7fa83740d740 1 --2- 172.21.15.7:0/1049445764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56312e6aebf0 0x56312e6b8930 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa81c09b7e0 tx=0x7fa81c06aae0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.978+0000 7fa83740d740 1 -- 172.21.15.7:0/1049445764 shutdown_connections 2024-06-10T01:24:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.978+0000 7fa83740d740 1 --2- 172.21.15.7:0/1049445764 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56312e6c4220 0x56312e6c6690 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.978+0000 7fa83740d740 1 --2- 172.21.15.7:0/1049445764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56312e6b3d60 0x56312e6bf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.978+0000 7fa83740d740 1 --2- 172.21.15.7:0/1049445764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56312e6aebf0 0x56312e6b8930 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.978+0000 7fa83740d740 1 --2- 172.21.15.7:0/1049445764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56312e6ae200 0x56312e6b83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.978+0000 7fa815ffb640 0 client.29255 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.978+0000 7fa83740d740 1 -- 172.21.15.7:0/1049445764 >> 172.21.15.7:0/1049445764 conn(0x56312e6a9870 msgr2=0x56312e6a89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.978+0000 7fa83740d740 1 -- 172.21.15.7:0/1049445764 shutdown_connections 2024-06-10T01:24:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.978+0000 7fa83740d740 1 -- 172.21.15.7:0/1049445764 wait complete. 2024-06-10T01:24:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.978+0000 7fa83740d740 20 client.29255 trim_cache size 0 max 16384 2024-06-10T01:24:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.999+0000 7fa83740d740 1 Processor -- start 2024-06-10T01:24:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.999+0000 7fa83740d740 1 -- start start 2024-06-10T01:24:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.999+0000 7fa83740d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56312e6ae200 0x56312e6afd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.999+0000 7fa83740d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56312e6aebf0 0x56312e6bf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.999+0000 7fa83740d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56312e6b3d60 0x56312e6b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.999+0000 7fa83740d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241477c0 con 0x56312e6aebf0 2024-06-10T01:24:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.999+0000 7fa83740d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa824145090 con 0x56312e6ae200 2024-06-10T01:24:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.999+0000 7fa83740d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa824142ee0 con 0x56312e6b3d60 2024-06-10T01:24:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.999+0000 7fa831ca4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56312e6aebf0 0x56312e6bf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.999+0000 7fa831ca4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56312e6aebf0 0x56312e6bf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38926/0 (socket says 172.21.15.7:38926) 2024-06-10T01:24:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.999+0000 7fa831ca4640 1 -- 172.21.15.7:0/118022033 learned_addr learned my addr 172.21.15.7:0/118022033 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:57.999+0000 7fa8324a5640 1 --2- 172.21.15.7:0/118022033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56312e6ae200 0x56312e6afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.000+0000 7fa832ca6640 1 --2- 172.21.15.7:0/118022033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56312e6b3d60 0x56312e6b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.000+0000 7fa8324a5640 1 -- 172.21.15.7:0/118022033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56312e6b3d60 msgr2=0x56312e6b02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.000+0000 7fa8324a5640 1 --2- 172.21.15.7:0/118022033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56312e6b3d60 0x56312e6b02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.000+0000 7fa8324a5640 1 -- 172.21.15.7:0/118022033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56312e6aebf0 msgr2=0x56312e6bf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.000+0000 7fa8324a5640 1 --2- 172.21.15.7:0/118022033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56312e6aebf0 0x56312e6bf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.000+0000 7fa8324a5640 1 -- 172.21.15.7:0/118022033 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56312e6c6d10 con 0x56312e6ae200 2024-06-10T01:24:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.000+0000 7fa831ca4640 1 --2- 172.21.15.7:0/118022033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56312e6aebf0 0x56312e6bf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:24:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.000+0000 7fa8324a5640 1 --2- 172.21.15.7:0/118022033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56312e6ae200 0x56312e6afd70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x56312e6c6f10 tx=0x7fa828009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af2c00c5a7200a76 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.000+0000 7fa817fff640 1 -- 172.21.15.7:0/118022033 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8280142d0 con 0x56312e6ae200 2024-06-10T01:24:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.000+0000 7fa817fff640 1 -- 172.21.15.7:0/118022033 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa82801b810 con 0x56312e6ae200 2024-06-10T01:24:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.000+0000 7fa817fff640 1 -- 172.21.15.7:0/118022033 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa828012aa0 con 0x56312e6ae200 2024-06-10T01:24:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.001+0000 7fa83740d740 1 -- 172.21.15.7:0/118022033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56312e6ae200 msgr2=0x56312e6afd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.001+0000 7fa83740d740 1 --2- 172.21.15.7:0/118022033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56312e6ae200 0x56312e6afd70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x56312e6c6f10 tx=0x7fa828009530 comp rx=0 tx=0).stop 2024-06-10T01:24:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.001+0000 7fa83740d740 1 -- 172.21.15.7:0/118022033 shutdown_connections 2024-06-10T01:24:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.001+0000 7fa83740d740 1 --2- 172.21.15.7:0/118022033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56312e6b3d60 0x56312e6b02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.001+0000 7fa83740d740 1 --2- 172.21.15.7:0/118022033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56312e6aebf0 0x56312e6bf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.001+0000 7fa83740d740 1 --2- 172.21.15.7:0/118022033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56312e6ae200 0x56312e6afd70 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.001+0000 7fa83740d740 1 -- 172.21.15.7:0/118022033 >> 172.21.15.7:0/118022033 conn(0x56312e6a9870 msgr2=0x56312e6b83f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.001+0000 7fa83740d740 1 -- 172.21.15.7:0/118022033 shutdown_connections 2024-06-10T01:24:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.001+0000 7fa83740d740 1 -- 172.21.15.7:0/118022033 wait complete. 2024-06-10T01:24:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.002+0000 7fa83740d740 1 -- start start 2024-06-10T01:24:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.002+0000 7fa83740d740 1 Processor -- start 2024-06-10T01:24:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.002+0000 7fa83740d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56312e6ae200 0x56312e6bc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.002+0000 7fa83740d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56312e6aebf0 0x56312e6bcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.002+0000 7fa83740d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56312e6b3d60 0x56312e72cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.002+0000 7fa83740d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa824143550 con 0x56312e6aebf0 2024-06-10T01:24:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.002+0000 7fa83740d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241449c0 con 0x56312e6ae200 2024-06-10T01:24:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.002+0000 7fa83740d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa824143c20 con 0x56312e6b3d60 2024-06-10T01:24:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.002+0000 7fa83740d740 10 client.0 _ll_register_callbacks cb 0x7fa81c09b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:24:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.002+0000 7fa831ca4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56312e6aebf0 0x56312e6bcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.002+0000 7fa8324a5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56312e6ae200 0x56312e6bc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.002+0000 7fa831ca4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56312e6aebf0 0x56312e6bcc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38930/0 (socket says 172.21.15.7:38930) 2024-06-10T01:24:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.002+0000 7fa831ca4640 1 -- 172.21.15.7:0/4171295074 learned_addr learned my addr 172.21.15.7:0/4171295074 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:24:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.002+0000 7fa832ca6640 1 --2- 172.21.15.7:0/4171295074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56312e6b3d60 0x56312e72cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.003+0000 7fa831ca4640 1 -- 172.21.15.7:0/4171295074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56312e6b3d60 msgr2=0x56312e72cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.003+0000 7fa831ca4640 1 --2- 172.21.15.7:0/4171295074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56312e6b3d60 0x56312e72cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.003+0000 7fa831ca4640 1 -- 172.21.15.7:0/4171295074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56312e6ae200 msgr2=0x56312e6bc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:24:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.003+0000 7fa831ca4640 1 --2- 172.21.15.7:0/4171295074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56312e6ae200 0x56312e6bc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.003+0000 7fa831ca4640 1 -- 172.21.15.7:0/4171295074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56312e6bd280 con 0x56312e6aebf0 2024-06-10T01:24:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.003+0000 7fa8324a5640 1 --2- 172.21.15.7:0/4171295074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56312e6ae200 0x56312e6bc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:24:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.003+0000 7fa831ca4640 1 --2- 172.21.15.7:0/4171295074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56312e6aebf0 0x56312e6bcc10 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fa81c098460 tx=0x7fa81c099ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da19f438d37d33af server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:24:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.003+0000 7fa815ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:24:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.003+0000 7fa815ffb640 1 -- 172.21.15.7:0/4171295074 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c09b8b0 con 0x56312e6aebf0 2024-06-10T01:24:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.003+0000 7fa815ffb640 1 -- 172.21.15.7:0/4171295074 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fa81c09af30 con 0x56312e6aebf0 2024-06-10T01:24:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.003+0000 7fa83740d740 1 -- 172.21.15.7:0/4171295074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa82801b810 con 0x56312e6aebf0 2024-06-10T01:24:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.003+0000 7fa815ffb640 1 -- 172.21.15.7:0/4171295074 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fa81c0ac050 con 0x56312e6aebf0 2024-06-10T01:24:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.003+0000 7fa83740d740 10 client.20055 fetch_fsmap learned FSMap version 51 2024-06-10T01:24:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.003+0000 7fa83740d740 1 -- 172.21.15.7:0/4171295074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x56312e72e620 con 0x56312e6aebf0 2024-06-10T01:24:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.004+0000 7fa815ffb640 1 -- 172.21.15.7:0/4171295074 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa82801b810 con 0x56312e6aebf0 2024-06-10T01:24:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.004+0000 7fa815ffb640 1 -- 172.21.15.7:0/4171295074 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 51) ==== 8350+0+0 (secure 0 0 0) 0x7fa81c0b4ce0 con 0x56312e6aebf0 2024-06-10T01:24:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.004+0000 7fa83740d740 10 client.20055 fetch_fsmap finished waiting for FSMap version 51 2024-06-10T01:24:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.004+0000 7fa83740d740 10 client.20055 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:24:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.004+0000 7fa83740d740 20 client.20055 populate_metadata read hostname 'smithi007' 2024-06-10T01:24:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.004+0000 7fa83740d740 1 --2- 172.21.15.7:0/4171295074 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56312e72ee70 0x56312e731290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:24:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.004+0000 7fa83740d740 4 client.20055 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:24:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.004+0000 7fa83740d740 1 -- 172.21.15.7:0/4171295074 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56312e61ec20 con 0x56312e72ee70 2024-06-10T01:24:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.004+0000 7fa8324a5640 1 --2- 172.21.15.7:0/4171295074 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56312e72ee70 0x56312e731290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:24:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.004+0000 7fa815ffb640 10 client.20055 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.004+0000 7fa8324a5640 1 --2- 172.21.15.7:0/4171295074 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56312e72ee70 0x56312e731290 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f90c19852eaba44 server_cookie=cab99c2f9496d4aa in_seq=0 out_seq=0 2024-06-10T01:24:58.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 134435 23.52 MB/sec execute 215 sec latency 386.312 ms 2024-06-10T01:24:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.790+0000 7fa815ffb640 1 -- 172.21.15.7:0/4171295074 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56312e61ec20 con 0x56312e72ee70 2024-06-10T01:24:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.790+0000 7fa815ffb640 10 client.20055 handle_command_reply: tid=0 2024-06-10T01:24:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.790+0000 7fa83740d740 1 client.20055 shutdown 2024-06-10T01:24:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.790+0000 7fa83740d740 1 -- 172.21.15.7:0/4171295074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56312e6aebf0 msgr2=0x56312e6bcc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.790+0000 7fa83740d740 1 --2- 172.21.15.7:0/4171295074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56312e6aebf0 0x56312e6bcc10 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fa81c098460 tx=0x7fa81c099ff0 comp rx=0 tx=0).stop 2024-06-10T01:24:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.790+0000 7fa83740d740 1 -- 172.21.15.7:0/4171295074 shutdown_connections 2024-06-10T01:24:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.790+0000 7fa83740d740 1 --2- 172.21.15.7:0/4171295074 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56312e72ee70 0x56312e731290 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.790+0000 7fa83740d740 1 --2- 172.21.15.7:0/4171295074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56312e6b3d60 0x56312e72cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.790+0000 7fa83740d740 1 --2- 172.21.15.7:0/4171295074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56312e6aebf0 0x56312e6bcc10 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.790+0000 7fa83740d740 1 --2- 172.21.15.7:0/4171295074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56312e6ae200 0x56312e6bc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.790+0000 7fa83740d740 1 -- 172.21.15.7:0/4171295074 >> 172.21.15.7:0/4171295074 conn(0x56312e6a9870 msgr2=0x56312e6b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.790+0000 7fa815ffb640 0 client.20055 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:24:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.790+0000 7fa83740d740 1 -- 172.21.15.7:0/4171295074 shutdown_connections 2024-06-10T01:24:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.790+0000 7fa83740d740 1 -- 172.21.15.7:0/4171295074 wait complete. 2024-06-10T01:24:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.790+0000 7fa83740d740 20 client.20055 trim_cache size 0 max 16384 2024-06-10T01:24:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:24:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:24:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:24:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.791+0000 7fa8334a7640 1 -- 172.21.15.7:0/3443095603 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa8180a2d60 msgr2=0x7fa8180a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.791+0000 7fa8334a7640 1 --2- 172.21.15.7:0/3443095603 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa8180a2d60 0x7fa8180a5220 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x56312e6b6960 tx=0x7fa82c09fe10 comp rx=0 tx=0).stop 2024-06-10T01:24:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.791+0000 7fa8334a7640 1 -- 172.21.15.7:0/3443095603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa824095e70 msgr2=0x7fa824097250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:24:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.792+0000 7fa8334a7640 1 --2- 172.21.15.7:0/3443095603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa824095e70 0x7fa824097250 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00ea60 tx=0x7fa81c00ef20 comp rx=0 tx=0).stop 2024-06-10T01:24:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.796+0000 7fa8334a7640 1 -- 172.21.15.7:0/3443095603 shutdown_connections 2024-06-10T01:24:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.796+0000 7fa8334a7640 1 --2- 172.21.15.7:0/3443095603 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa8180a2d60 0x7fa8180a5220 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.796+0000 7fa8334a7640 1 --2- 172.21.15.7:0/3443095603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa824096830 0x7fa824097790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.797+0000 7fa8334a7640 1 --2- 172.21.15.7:0/3443095603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa824095e70 0x7fa824097250 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.797+0000 7fa8334a7640 1 --2- 172.21.15.7:0/3443095603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa824094c80 0x7fa824098c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:24:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.797+0000 7fa8334a7640 1 -- 172.21.15.7:0/3443095603 >> 172.21.15.7:0/3443095603 conn(0x7fa8240901d0 msgr2=0x7fa824091cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:24:58.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.800+0000 7fa8334a7640 1 -- 172.21.15.7:0/3443095603 shutdown_connections 2024-06-10T01:24:58.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:24:58.800+0000 7fa8334a7640 1 -- 172.21.15.7:0/3443095603 wait complete. 2024-06-10T01:24:58.805 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:24:59.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:59 smithi007 ceph-mon[32398]: pgmap v1002: 129 pgs: 129 active+clean; 631 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.6 MiB/s wr, 475 op/s 2024-06-10T01:24:59.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:59 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:24:59.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:24:59 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:24:59.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:59 smithi078 ceph-mon[36683]: pgmap v1002: 129 pgs: 129 active+clean; 631 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.6 MiB/s wr, 475 op/s 2024-06-10T01:24:59.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:59 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:24:59.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:24:59 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:24:59.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:59 smithi175 ceph-mon[36636]: pgmap v1002: 129 pgs: 129 active+clean; 631 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.6 MiB/s wr, 475 op/s 2024-06-10T01:24:59.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:59 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:24:59.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:24:59 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:24:59.537 INFO:tasks.workunit.client.0.smithi007.stdout: 10 134846 23.51 MB/sec execute 216 sec latency 480.084 ms 2024-06-10T01:25:00.537 INFO:tasks.workunit.client.0.smithi007.stdout: 10 135339 23.53 MB/sec execute 217 sec latency 210.802 ms 2024-06-10T01:25:01.537 INFO:tasks.workunit.client.0.smithi007.stdout: 10 136161 23.57 MB/sec execute 218 sec latency 90.092 ms 2024-06-10T01:25:01.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:01 smithi007 ceph-mon[32398]: pgmap v1003: 129 pgs: 129 active+clean; 637 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.6 MiB/s wr, 648 op/s 2024-06-10T01:25:01.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:01 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:25:01.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:01 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:25:01.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:01 smithi078 ceph-mon[36683]: pgmap v1003: 129 pgs: 129 active+clean; 637 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.6 MiB/s wr, 648 op/s 2024-06-10T01:25:01.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:01 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:25:01.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:01 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:25:01.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:01 smithi175 ceph-mon[36636]: pgmap v1003: 129 pgs: 129 active+clean; 637 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.6 MiB/s wr, 648 op/s 2024-06-10T01:25:01.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:01 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:25:01.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:01 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:25:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.356+0000 7ff9a1f42700 1 -- 172.21.15.7:0/2238620903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff99c105330 msgr2=0x7ff99c105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.356+0000 7ff9a1f42700 1 --2- 172.21.15.7:0/2238620903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff99c105330 0x7ff99c105710 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff98c0098f0 tx=0x7ff98c009c00 comp rx=0 tx=0).stop 2024-06-10T01:25:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.356+0000 7ff9a1f42700 1 -- 172.21.15.7:0/2238620903 shutdown_connections 2024-06-10T01:25:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.356+0000 7ff9a1f42700 1 --2- 172.21.15.7:0/2238620903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff99c107b60 0x7ff99c10e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.356+0000 7ff9a1f42700 1 --2- 172.21.15.7:0/2238620903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff99c105330 0x7ff99c105710 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.356+0000 7ff9a1f42700 1 --2- 172.21.15.7:0/2238620903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff99c1071a0 0x7ff99c107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.356+0000 7ff9a1f42700 1 -- 172.21.15.7:0/2238620903 >> 172.21.15.7:0/2238620903 conn(0x7ff99c0766b0 msgr2=0x7ff99c076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:02.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.356+0000 7ff9a1f42700 1 -- 172.21.15.7:0/2238620903 shutdown_connections 2024-06-10T01:25:02.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.356+0000 7ff9a1f42700 1 -- 172.21.15.7:0/2238620903 wait complete. 2024-06-10T01:25:02.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.357+0000 7ff9a1f42700 1 Processor -- start 2024-06-10T01:25:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.358+0000 7ff9a1f42700 1 -- start start 2024-06-10T01:25:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.358+0000 7ff9a1f42700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff99c105330 0x7ff99c102030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.358+0000 7ff9a1f42700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff99c1071a0 0x7ff99c102570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.358+0000 7ff9a1f42700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff99c107b60 0x7ff99c102ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.358+0000 7ff9a1f42700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c111750 con 0x7ff99c107b60 2024-06-10T01:25:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.358+0000 7ff9a1f42700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c1115d0 con 0x7ff99c105330 2024-06-10T01:25:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.358+0000 7ff99b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff99c105330 0x7ff99c102030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.358+0000 7ff9a1f42700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c1118d0 con 0x7ff99c1071a0 2024-06-10T01:25:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.358+0000 7ff99b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff99c105330 0x7ff99c102030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:34026/0 (socket says 172.21.15.7:34026) 2024-06-10T01:25:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.358+0000 7ff99b7fe700 1 -- 172.21.15.7:0/370742825 learned_addr learned my addr 172.21.15.7:0/370742825 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.358+0000 7ff99bfff700 1 --2- 172.21.15.7:0/370742825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff99c107b60 0x7ff99c102ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.358+0000 7ff99affd700 1 --2- 172.21.15.7:0/370742825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff99c1071a0 0x7ff99c102570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.359+0000 7ff99bfff700 1 -- 172.21.15.7:0/370742825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff99c1071a0 msgr2=0x7ff99c102570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.359+0000 7ff99bfff700 1 --2- 172.21.15.7:0/370742825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff99c1071a0 0x7ff99c102570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.359+0000 7ff99bfff700 1 -- 172.21.15.7:0/370742825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff99c105330 msgr2=0x7ff99c102030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.359+0000 7ff99bfff700 1 --2- 172.21.15.7:0/370742825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff99c105330 0x7ff99c102030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.359+0000 7ff99bfff700 1 -- 172.21.15.7:0/370742825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99c1a7230 con 0x7ff99c107b60 2024-06-10T01:25:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.359+0000 7ff99b7fe700 1 --2- 172.21.15.7:0/370742825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff99c105330 0x7ff99c102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:25:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.359+0000 7ff99bfff700 1 --2- 172.21.15.7:0/370742825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff99c107b60 0x7ff99c102ab0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7ff994009cd0 tx=0x7ff99400c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.360+0000 7ff998ff9700 1 -- 172.21.15.7:0/370742825 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99400cd10 con 0x7ff99c107b60 2024-06-10T01:25:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.360+0000 7ff998ff9700 1 -- 172.21.15.7:0/370742825 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff994003a00 con 0x7ff99c107b60 2024-06-10T01:25:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.360+0000 7ff998ff9700 1 -- 172.21.15.7:0/370742825 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9940045a0 con 0x7ff99c107b60 2024-06-10T01:25:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.360+0000 7ff9a1f42700 1 -- 172.21.15.7:0/370742825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff99c1a74c0 con 0x7ff99c107b60 2024-06-10T01:25:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.360+0000 7ff9a1f42700 1 -- 172.21.15.7:0/370742825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff99c1a79b0 con 0x7ff99c107b60 2024-06-10T01:25:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.363+0000 7ff998ff9700 1 -- 172.21.15.7:0/370742825 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7ff99401a020 con 0x7ff99c107b60 2024-06-10T01:25:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.367+0000 7ff998ff9700 1 --2- 172.21.15.7:0/370742825 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff9840a2d20 0x7ff9840a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.367+0000 7ff998ff9700 1 -- 172.21.15.7:0/370742825 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7ff9940cec20 con 0x7ff99c107b60 2024-06-10T01:25:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.367+0000 7ff99b7fe700 1 --2- 172.21.15.7:0/370742825 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff9840a2d20 0x7ff9840a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.368+0000 7ff99b7fe700 1 --2- 172.21.15.7:0/370742825 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff9840a2d20 0x7ff9840a51e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff98c009680 tx=0x7ff98c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.369+0000 7ff9ac7c5b80 1 Processor -- start 2024-06-10T01:25:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.370+0000 7ff9ac7c5b80 1 -- start start 2024-06-10T01:25:02.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.370+0000 7ff9ac7c5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c61be3640 0x563c61be3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.371+0000 7ff9ac7c5b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c61be3f60 0x563c61bdd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.371+0000 7ff9ac7c5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c61bddf20 0x563c61bde340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.371+0000 7ff9ac7c5b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563c61b2f810 con 0x563c61be3f60 2024-06-10T01:25:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.371+0000 7ff9ac7c5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563c61ad7720 con 0x563c61bddf20 2024-06-10T01:25:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.371+0000 7ff9ac7c5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563c61b2b7d0 con 0x563c61be3640 2024-06-10T01:25:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.371+0000 7ff99bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c61bddf20 0x563c61bde340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.371+0000 7ff99affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c61be3f60 0x563c61bdd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.371+0000 7ff99bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c61bddf20 0x563c61bde340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:34042/0 (socket says 172.21.15.7:34042) 2024-06-10T01:25:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.371+0000 7ff99bfff700 1 -- 172.21.15.7:0/1154661235 learned_addr learned my addr 172.21.15.7:0/1154661235 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:02.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.372+0000 7ff99b7fe700 1 --2- 172.21.15.7:0/1154661235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c61be3640 0x563c61be3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:02.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.372+0000 7ff99bfff700 1 -- 172.21.15.7:0/1154661235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c61be3640 msgr2=0x563c61be3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:02.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.372+0000 7ff99bfff700 1 --2- 172.21.15.7:0/1154661235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c61be3640 0x563c61be3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:02.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.372+0000 7ff99bfff700 1 -- 172.21.15.7:0/1154661235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c61be3f60 msgr2=0x563c61bdd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:02.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.372+0000 7ff99bfff700 1 --2- 172.21.15.7:0/1154661235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c61be3f60 0x563c61bdd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:02.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.372+0000 7ff99bfff700 1 -- 172.21.15.7:0/1154661235 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c61acdf50 con 0x563c61bddf20 2024-06-10T01:25:02.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.372+0000 7ff99bfff700 1 --2- 172.21.15.7:0/1154661235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c61bddf20 0x563c61bde340 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff994097500 tx=0x7ff994009f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26a86ef8dd79e848 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:02.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.373+0000 7ff981ffb700 1 -- 172.21.15.7:0/1154661235 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99409b7a0 con 0x563c61bddf20 2024-06-10T01:25:02.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.373+0000 7ff981ffb700 1 -- 172.21.15.7:0/1154661235 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff99409b0a0 con 0x563c61bddf20 2024-06-10T01:25:02.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.373+0000 7ff981ffb700 1 -- 172.21.15.7:0/1154661235 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99409f080 con 0x563c61bddf20 2024-06-10T01:25:02.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.374+0000 7ff9ac7c5b80 1 -- 172.21.15.7:0/1154661235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c61bddf20 msgr2=0x563c61bde340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:02.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.374+0000 7ff9ac7c5b80 1 --2- 172.21.15.7:0/1154661235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c61bddf20 0x563c61bde340 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff994097500 tx=0x7ff994009f30 comp rx=0 tx=0).stop 2024-06-10T01:25:02.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.374+0000 7ff9ac7c5b80 1 -- 172.21.15.7:0/1154661235 shutdown_connections 2024-06-10T01:25:02.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.374+0000 7ff9ac7c5b80 1 --2- 172.21.15.7:0/1154661235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c61bddf20 0x563c61bde340 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:02.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.374+0000 7ff9ac7c5b80 1 --2- 172.21.15.7:0/1154661235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c61be3640 0x563c61be3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:02.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.374+0000 7ff9ac7c5b80 1 --2- 172.21.15.7:0/1154661235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c61be3f60 0x563c61bdd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:02.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.374+0000 7ff9ac7c5b80 1 -- 172.21.15.7:0/1154661235 >> 172.21.15.7:0/1154661235 conn(0x563c61b08cc0 msgr2=0x563c61bd8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:02.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.375+0000 7ff9ac7c5b80 1 -- 172.21.15.7:0/1154661235 shutdown_connections 2024-06-10T01:25:02.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.375+0000 7ff9ac7c5b80 1 -- 172.21.15.7:0/1154661235 wait complete. 2024-06-10T01:25:02.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.375+0000 7ff9ac7c5b80 1 -- start start 2024-06-10T01:25:02.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.375+0000 7ff9ac7c5b80 1 Processor -- start 2024-06-10T01:25:02.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.376+0000 7ff9ac7c5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c61bddf20 0x563c61be9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:02.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.376+0000 7ff9ac7c5b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c61be3640 0x563c61be6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:02.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.376+0000 7ff9ac7c5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c61be3f60 0x563c61be7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:02.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.376+0000 7ff9ac7c5b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563c61beaa20 con 0x563c61be3640 2024-06-10T01:25:02.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.376+0000 7ff9ac7c5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563c61acfbf0 con 0x563c61bddf20 2024-06-10T01:25:02.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.376+0000 7ff9ac7c5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563c61beaba0 con 0x563c61be3f60 2024-06-10T01:25:02.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.376+0000 7ff9ac7c5b80 10 client.0 _ll_register_callbacks cb 0x563c61ac9920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:25:02.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.376+0000 7ff99affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c61be3640 0x563c61be6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:02.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.376+0000 7ff99affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c61be3640 0x563c61be6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40662/0 (socket says 172.21.15.7:40662) 2024-06-10T01:25:02.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.376+0000 7ff99affd700 1 -- 172.21.15.7:0/4219071024 learned_addr learned my addr 172.21.15.7:0/4219071024 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:02.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.376+0000 7ff99b7fe700 1 --2- 172.21.15.7:0/4219071024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c61bddf20 0x563c61be9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:02.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.377+0000 7ff99bfff700 1 --2- 172.21.15.7:0/4219071024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c61be3f60 0x563c61be7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:02.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.377+0000 7ff99affd700 1 -- 172.21.15.7:0/4219071024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c61be3f60 msgr2=0x563c61be7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:02.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.377+0000 7ff99affd700 1 --2- 172.21.15.7:0/4219071024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c61be3f60 0x563c61be7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:02.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.377+0000 7ff99affd700 1 -- 172.21.15.7:0/4219071024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c61bddf20 msgr2=0x563c61be9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:02.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.377+0000 7ff99affd700 1 --2- 172.21.15.7:0/4219071024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c61bddf20 0x563c61be9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:02.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.377+0000 7ff99affd700 1 -- 172.21.15.7:0/4219071024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c61a7e8d0 con 0x563c61be3640 2024-06-10T01:25:02.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.377+0000 7ff99affd700 1 --2- 172.21.15.7:0/4219071024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c61be3640 0x563c61be6ff0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7ff990012840 tx=0x7ff990012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3111317bfb4b7779 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:02.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.377+0000 7ff97affd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:25:02.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.378+0000 7ff97affd700 1 -- 172.21.15.7:0/4219071024 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99000f5e0 con 0x563c61be3640 2024-06-10T01:25:02.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.378+0000 7ff97affd700 1 -- 172.21.15.7:0/4219071024 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7ff99000f780 con 0x563c61be3640 2024-06-10T01:25:02.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.378+0000 7ff9ac7c5b80 1 -- 172.21.15.7:0/4219071024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff99409b0a0 con 0x563c61be3640 2024-06-10T01:25:02.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.378+0000 7ff97affd700 1 -- 172.21.15.7:0/4219071024 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7ff99000f980 con 0x563c61be3640 2024-06-10T01:25:02.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.379+0000 7ff9ac7c5b80 10 client.20067 fetch_fsmap learned FSMap version 51 2024-06-10T01:25:02.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.379+0000 7ff9ac7c5b80 1 -- 172.21.15.7:0/4219071024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x563c61bf5400 con 0x563c61be3640 2024-06-10T01:25:02.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.381+0000 7ff97affd700 1 -- 172.21.15.7:0/4219071024 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff99409b0a0 con 0x563c61be3640 2024-06-10T01:25:02.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.381+0000 7ff97affd700 1 -- 172.21.15.7:0/4219071024 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 51) ==== 8350+0+0 (secure 0 0 0) 0x7ff990010430 con 0x563c61be3640 2024-06-10T01:25:02.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.381+0000 7ff9ac7c5b80 10 client.20067 fetch_fsmap finished waiting for FSMap version 51 2024-06-10T01:25:02.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.381+0000 7ff9ac7c5b80 10 client.20067 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:25:02.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.381+0000 7ff9ac7c5b80 20 client.20067 populate_metadata read hostname 'smithi007' 2024-06-10T01:25:02.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.381+0000 7ff9ac7c5b80 1 --2- 172.21.15.7:0/4219071024 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x563c61bf7990 0x563c61bf9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:02.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.381+0000 7ff9ac7c5b80 4 client.20067 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:25:02.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.381+0000 7ff9ac7c5b80 1 -- 172.21.15.7:0/4219071024 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c61b3cd10 con 0x563c61bf7990 2024-06-10T01:25:02.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.382+0000 7ff99b7fe700 1 --2- 172.21.15.7:0/4219071024 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x563c61bf7990 0x563c61bf9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:02.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.388+0000 7ff99b7fe700 1 --2- 172.21.15.7:0/4219071024 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x563c61bf7990 0x563c61bf9da0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a420841214fe18bf server_cookie=c89d5223fe8588ad in_seq=0 out_seq=0 2024-06-10T01:25:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.388+0000 7ff97affd700 10 client.20067 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:25:02.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.393+0000 7ff97affd700 1 -- 172.21.15.7:0/4219071024 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563c61b3cd10 con 0x563c61bf7990 2024-06-10T01:25:02.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.393+0000 7ff97affd700 10 client.20067 handle_command_reply: tid=0 2024-06-10T01:25:02.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.393+0000 7ff9ac7c5b80 1 client.20067 shutdown 2024-06-10T01:25:02.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.393+0000 7ff9ac7c5b80 1 -- 172.21.15.7:0/4219071024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c61be3640 msgr2=0x563c61be6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:02.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.393+0000 7ff9ac7c5b80 1 --2- 172.21.15.7:0/4219071024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c61be3640 0x563c61be6ff0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7ff990012840 tx=0x7ff990012c00 comp rx=0 tx=0).stop 2024-06-10T01:25:02.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.393+0000 7ff9ac7c5b80 1 -- 172.21.15.7:0/4219071024 shutdown_connections 2024-06-10T01:25:02.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.393+0000 7ff9ac7c5b80 1 --2- 172.21.15.7:0/4219071024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c61be3f60 0x563c61be7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:02.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.393+0000 7ff9ac7c5b80 1 --2- 172.21.15.7:0/4219071024 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x563c61bf7990 0x563c61bf9da0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:02.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.393+0000 7ff9ac7c5b80 1 --2- 172.21.15.7:0/4219071024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c61bddf20 0x563c61be9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:02.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.393+0000 7ff9ac7c5b80 1 --2- 172.21.15.7:0/4219071024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c61be3640 0x563c61be6ff0 secure :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7ff990012840 tx=0x7ff990012c00 comp rx=0 tx=0).stop 2024-06-10T01:25:02.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.394+0000 7ff9ac7c5b80 1 -- 172.21.15.7:0/4219071024 >> 172.21.15.7:0/4219071024 conn(0x563c61b08cc0 msgr2=0x563c61bdc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:02.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.394+0000 7ff97affd700 0 client.20067 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:25:02.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.395+0000 7ff9ac7c5b80 1 -- 172.21.15.7:0/4219071024 shutdown_connections 2024-06-10T01:25:02.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.395+0000 7ff9ac7c5b80 1 -- 172.21.15.7:0/4219071024 wait complete. 2024-06-10T01:25:02.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.395+0000 7ff9ac7c5b80 20 client.20067 trim_cache size 0 max 16384 2024-06-10T01:25:02.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.427+0000 7ff9ac7c5b80 1 Processor -- start 2024-06-10T01:25:02.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.428+0000 7ff9ac7c5b80 1 -- start start 2024-06-10T01:25:02.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.428+0000 7ff9ac7c5b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c61bddf20 0x563c61bdd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:02.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.428+0000 7ff9ac7c5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c61be3f60 0x563c61be58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:02.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.428+0000 7ff9ac7c5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c61bf3d40 0x563c61be5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.428+0000 7ff9ac7c5b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c1b2f30 con 0x563c61bddf20 2024-06-10T01:25:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.428+0000 7ff9ac7c5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c1b07c0 con 0x563c61bf3d40 2024-06-10T01:25:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.428+0000 7ff9ac7c5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c1ae610 con 0x563c61be3f60 2024-06-10T01:25:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.428+0000 7ff99bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c61bf3d40 0x563c61be5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.428+0000 7ff99b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c61bddf20 0x563c61bdd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.428+0000 7ff99bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c61bf3d40 0x563c61be5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:34050/0 (socket says 172.21.15.7:34050) 2024-06-10T01:25:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.428+0000 7ff99bfff700 1 -- 172.21.15.7:0/400361401 learned_addr learned my addr 172.21.15.7:0/400361401 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:02.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.429+0000 7ff99affd700 1 --2- 172.21.15.7:0/400361401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c61be3f60 0x563c61be58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:02.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.429+0000 7ff99affd700 1 -- 172.21.15.7:0/400361401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c61bf3d40 msgr2=0x563c61be5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:02.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.429+0000 7ff99affd700 1 --2- 172.21.15.7:0/400361401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c61bf3d40 0x563c61be5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:02.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.429+0000 7ff99affd700 1 -- 172.21.15.7:0/400361401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c61bddf20 msgr2=0x563c61bdd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:02.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.429+0000 7ff99affd700 1 --2- 172.21.15.7:0/400361401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c61bddf20 0x563c61bdd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:02.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.429+0000 7ff99affd700 1 -- 172.21.15.7:0/400361401 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c61bda2f0 con 0x563c61be3f60 2024-06-10T01:25:02.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.429+0000 7ff99affd700 1 --2- 172.21.15.7:0/400361401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c61be3f60 0x563c61be58e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x563c61be8080 tx=0x7ff9900123a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6decd8d9f96ff604 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:02.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.430+0000 7ff9817fa700 1 -- 172.21.15.7:0/400361401 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99002b070 con 0x563c61be3f60 2024-06-10T01:25:02.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.430+0000 7ff9817fa700 1 -- 172.21.15.7:0/400361401 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff990010530 con 0x563c61be3f60 2024-06-10T01:25:02.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.430+0000 7ff9817fa700 1 -- 172.21.15.7:0/400361401 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990004ac0 con 0x563c61be3f60 2024-06-10T01:25:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.434+0000 7ff9ac7c5b80 1 -- 172.21.15.7:0/400361401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c61be3f60 msgr2=0x563c61be58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.434+0000 7ff9ac7c5b80 1 --2- 172.21.15.7:0/400361401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c61be3f60 0x563c61be58e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x563c61be8080 tx=0x7ff9900123a0 comp rx=0 tx=0).stop 2024-06-10T01:25:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.434+0000 7ff9ac7c5b80 1 -- 172.21.15.7:0/400361401 shutdown_connections 2024-06-10T01:25:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.434+0000 7ff9ac7c5b80 1 --2- 172.21.15.7:0/400361401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c61bf3d40 0x563c61be5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.434+0000 7ff9ac7c5b80 1 --2- 172.21.15.7:0/400361401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c61bddf20 0x563c61bdd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.434+0000 7ff9ac7c5b80 1 --2- 172.21.15.7:0/400361401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c61be3f60 0x563c61be58e0 secure :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0x563c61be8080 tx=0x7ff9900123a0 comp rx=0 tx=0).stop 2024-06-10T01:25:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.434+0000 7ff9ac7c5b80 1 -- 172.21.15.7:0/400361401 >> 172.21.15.7:0/400361401 conn(0x563c61b08cc0 msgr2=0x563c61be6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:02.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.438+0000 7ff9ac7c5b80 1 -- 172.21.15.7:0/400361401 shutdown_connections 2024-06-10T01:25:02.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.438+0000 7ff9ac7c5b80 1 -- 172.21.15.7:0/400361401 wait complete. 2024-06-10T01:25:02.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.438+0000 7ff9ac7c5b80 1 -- start start 2024-06-10T01:25:02.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.439+0000 7ff9ac7c5b80 1 Processor -- start 2024-06-10T01:25:02.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.440+0000 7ff9ac7c5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c61bddf20 0x563c61bf0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.440+0000 7ff9ac7c5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c61bf3d40 0x563c61bfedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.440+0000 7ff9ac7c5b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c61bff330 0x563c61bff7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.440+0000 7ff9ac7c5b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c1aec80 con 0x563c61bff330 2024-06-10T01:25:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.440+0000 7ff9ac7c5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c1b00f0 con 0x563c61bddf20 2024-06-10T01:25:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.440+0000 7ff9ac7c5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c1af350 con 0x563c61bf3d40 2024-06-10T01:25:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.440+0000 7ff9ac7c5b80 10 client.0 _ll_register_callbacks cb 0x563c61ac9920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:25:02.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.440+0000 7ff99b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c61bddf20 0x563c61bf0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:02.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.440+0000 7ff99b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c61bddf20 0x563c61bf0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:34062/0 (socket says 172.21.15.7:34062) 2024-06-10T01:25:02.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.440+0000 7ff99b7fe700 1 -- 172.21.15.7:0/3905382388 learned_addr learned my addr 172.21.15.7:0/3905382388 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:02.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.442+0000 7ff99b7fe700 1 -- 172.21.15.7:0/3905382388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c61bf3d40 msgr2=0x563c61bfedf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:25:02.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.442+0000 7ff99b7fe700 1 --2- 172.21.15.7:0/3905382388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c61bf3d40 0x563c61bfedf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:02.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.442+0000 7ff99b7fe700 1 -- 172.21.15.7:0/3905382388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c61bff330 msgr2=0x563c61bff7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:02.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.442+0000 7ff99b7fe700 1 --2- 172.21.15.7:0/3905382388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c61bff330 0x563c61bff7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:02.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.442+0000 7ff99b7fe700 1 -- 172.21.15.7:0/3905382388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c61c5a690 con 0x563c61bddf20 2024-06-10T01:25:02.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.442+0000 7ff99b7fe700 1 --2- 172.21.15.7:0/3905382388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c61bddf20 0x563c61bf0440 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00fc10 tx=0x7ff98c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e55d420905d49d8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:02.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.442+0000 7ff97affd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:25:02.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.443+0000 7ff97affd700 1 -- 172.21.15.7:0/3905382388 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c003a90 con 0x563c61bddf20 2024-06-10T01:25:02.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.443+0000 7ff9ac7c5b80 1 -- 172.21.15.7:0/3905382388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff990010530 con 0x563c61bddf20 2024-06-10T01:25:02.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.443+0000 7ff97affd700 1 -- 172.21.15.7:0/3905382388 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7ff98c003c30 con 0x563c61bddf20 2024-06-10T01:25:02.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.443+0000 7ff97affd700 1 -- 172.21.15.7:0/3905382388 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7ff98c03e050 con 0x563c61bddf20 2024-06-10T01:25:02.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.444+0000 7ff9ac7c5b80 10 client.29276 fetch_fsmap learned FSMap version 51 2024-06-10T01:25:02.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.444+0000 7ff9ac7c5b80 1 -- 172.21.15.7:0/3905382388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x563c61c5b9d0 con 0x563c61bddf20 2024-06-10T01:25:02.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.444+0000 7ff97affd700 1 -- 172.21.15.7:0/3905382388 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff990010530 con 0x563c61bddf20 2024-06-10T01:25:02.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.445+0000 7ff97affd700 1 -- 172.21.15.7:0/3905382388 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 51) ==== 8350+0+0 (secure 0 0 0) 0x7ff98c00b560 con 0x563c61bddf20 2024-06-10T01:25:02.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.445+0000 7ff9ac7c5b80 10 client.29276 fetch_fsmap finished waiting for FSMap version 51 2024-06-10T01:25:02.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.445+0000 7ff9ac7c5b80 10 client.29276 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:25:02.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.445+0000 7ff9ac7c5b80 20 client.29276 populate_metadata read hostname 'smithi007' 2024-06-10T01:25:02.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.445+0000 7ff9ac7c5b80 1 --2- 172.21.15.7:0/3905382388 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x563c61c5c0e0 0x563c61c5e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:02.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.445+0000 7ff9ac7c5b80 4 client.29276 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:25:02.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.445+0000 7ff9ac7c5b80 1 -- 172.21.15.7:0/3905382388 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563c61b37290 con 0x563c61c5c0e0 2024-06-10T01:25:02.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.447+0000 7ff99affd700 1 --2- 172.21.15.7:0/3905382388 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x563c61c5c0e0 0x563c61c5e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:02.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.452+0000 7ff99affd700 1 --2- 172.21.15.7:0/3905382388 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x563c61c5c0e0 0x563c61c5e540 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1826d89c5c130d2a server_cookie=73730e303f86aa30 in_seq=0 out_seq=0 2024-06-10T01:25:02.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.452+0000 7ff97affd700 10 client.29276 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:25:02.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.453+0000 7ff97affd700 1 -- 172.21.15.7:0/3905382388 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563c61b37290 con 0x563c61c5c0e0 2024-06-10T01:25:02.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.453+0000 7ff97affd700 10 client.29276 handle_command_reply: tid=0 2024-06-10T01:25:02.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.453+0000 7ff9ac7c5b80 1 client.29276 shutdown 2024-06-10T01:25:02.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.454+0000 7ff9ac7c5b80 1 -- 172.21.15.7:0/3905382388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c61bddf20 msgr2=0x563c61bf0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:02.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.454+0000 7ff9ac7c5b80 1 --2- 172.21.15.7:0/3905382388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c61bddf20 0x563c61bf0440 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00fc10 tx=0x7ff98c009260 comp rx=0 tx=0).stop 2024-06-10T01:25:02.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.454+0000 7ff9ac7c5b80 1 -- 172.21.15.7:0/3905382388 shutdown_connections 2024-06-10T01:25:02.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.454+0000 7ff9ac7c5b80 1 --2- 172.21.15.7:0/3905382388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c61bff330 0x563c61bff7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:02.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.454+0000 7ff9ac7c5b80 1 --2- 172.21.15.7:0/3905382388 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x563c61c5c0e0 0x563c61c5e540 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:02.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.454+0000 7ff9ac7c5b80 1 --2- 172.21.15.7:0/3905382388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c61bddf20 0x563c61bf0440 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:02.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.454+0000 7ff9ac7c5b80 1 --2- 172.21.15.7:0/3905382388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c61bf3d40 0x563c61bfedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:02.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.454+0000 7ff9ac7c5b80 1 -- 172.21.15.7:0/3905382388 >> 172.21.15.7:0/3905382388 conn(0x563c61b08cc0 msgr2=0x563c61bd1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:02.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.455+0000 7ff97affd700 0 client.29276 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:25:02.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.455+0000 7ff9ac7c5b80 1 -- 172.21.15.7:0/3905382388 shutdown_connections 2024-06-10T01:25:02.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.455+0000 7ff9ac7c5b80 1 -- 172.21.15.7:0/3905382388 wait complete. 2024-06-10T01:25:02.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.455+0000 7ff9ac7c5b80 20 client.29276 trim_cache size 0 max 16384 2024-06-10T01:25:02.451 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:25:02.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.460+0000 7ff9a1f42700 1 -- 172.21.15.7:0/370742825 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff9840a2d20 msgr2=0x7ff9840a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.460+0000 7ff9a1f42700 1 --2- 172.21.15.7:0/370742825 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff9840a2d20 0x7ff9840a51e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff98c009680 tx=0x7ff98c01a040 comp rx=0 tx=0).stop 2024-06-10T01:25:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.460+0000 7ff9a1f42700 1 -- 172.21.15.7:0/370742825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff99c107b60 msgr2=0x7ff99c102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.460+0000 7ff9a1f42700 1 --2- 172.21.15.7:0/370742825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff99c107b60 0x7ff99c102ab0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7ff994009cd0 tx=0x7ff99400c620 comp rx=0 tx=0).stop 2024-06-10T01:25:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.461+0000 7ff9a1f42700 1 -- 172.21.15.7:0/370742825 shutdown_connections 2024-06-10T01:25:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.461+0000 7ff9a1f42700 1 --2- 172.21.15.7:0/370742825 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff9840a2d20 0x7ff9840a51e0 secure :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff98c009680 tx=0x7ff98c01a040 comp rx=0 tx=0).stop 2024-06-10T01:25:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.461+0000 7ff9a1f42700 1 --2- 172.21.15.7:0/370742825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff99c107b60 0x7ff99c102ab0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.461+0000 7ff9a1f42700 1 --2- 172.21.15.7:0/370742825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff99c105330 0x7ff99c102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.461+0000 7ff9a1f42700 1 --2- 172.21.15.7:0/370742825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff99c1071a0 0x7ff99c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.461+0000 7ff9a1f42700 1 -- 172.21.15.7:0/370742825 >> 172.21.15.7:0/370742825 conn(0x7ff99c0766b0 msgr2=0x7ff99c100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:02.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.462+0000 7ff9a1f42700 1 -- 172.21.15.7:0/370742825 shutdown_connections 2024-06-10T01:25:02.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:02.462+0000 7ff9a1f42700 1 -- 172.21.15.7:0/370742825 wait complete. 2024-06-10T01:25:02.537 INFO:tasks.workunit.client.0.smithi007.stdout: 10 137009 23.74 MB/sec execute 219 sec latency 67.358 ms 2024-06-10T01:25:03.354 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:25:03.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 137098 23.65 MB/sec execute 220 sec latency 864.978 ms 2024-06-10T01:25:03.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:03 smithi007 ceph-mon[32398]: pgmap v1004: 129 pgs: 129 active+clean; 640 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.4 MiB/s wr, 502 op/s 2024-06-10T01:25:03.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:03 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:25:03.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:03 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:25:03.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:03 smithi078 ceph-mon[36683]: pgmap v1004: 129 pgs: 129 active+clean; 640 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.4 MiB/s wr, 502 op/s 2024-06-10T01:25:03.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:03 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:25:03.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:03 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:25:03.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:03 smithi175 ceph-mon[36636]: pgmap v1004: 129 pgs: 129 active+clean; 640 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.4 MiB/s wr, 502 op/s 2024-06-10T01:25:03.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:03 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:25:03.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:03 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:25:04.002 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:25:04.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 137234 23.60 MB/sec execute 221 sec latency 1348.227 ms 2024-06-10T01:25:05.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:05 smithi007 ceph-mon[32398]: pgmap v1005: 129 pgs: 129 active+clean; 637 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.4 MiB/s wr, 501 op/s 2024-06-10T01:25:05.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:05 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:25:05.537 INFO:tasks.workunit.client.0.smithi007.stdout: 10 137469 23.55 MB/sec execute 222 sec latency 230.498 ms 2024-06-10T01:25:05.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:05 smithi078 ceph-mon[36683]: pgmap v1005: 129 pgs: 129 active+clean; 637 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.4 MiB/s wr, 501 op/s 2024-06-10T01:25:05.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:05 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:25:05.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:05 smithi175 ceph-mon[36636]: pgmap v1005: 129 pgs: 129 active+clean; 637 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.4 MiB/s wr, 501 op/s 2024-06-10T01:25:05.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:05 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:25:06.537 INFO:tasks.workunit.client.0.smithi007.stdout: 10 137649 23.48 MB/sec execute 223 sec latency 229.405 ms 2024-06-10T01:25:06.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.611+0000 7efe796ff700 1 -- 172.21.15.7:0/3872291762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe74111060 msgr2=0x7efe741099d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:06.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.611+0000 7efe796ff700 1 --2- 172.21.15.7:0/3872291762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe74111060 0x7efe741099d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7efe64009990 tx=0x7efe64009ca0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.614+0000 7efe796ff700 1 -- 172.21.15.7:0/3872291762 shutdown_connections 2024-06-10T01:25:06.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.614+0000 7efe796ff700 1 --2- 172.21.15.7:0/3872291762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe7411bc70 0x7efe7411e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.614+0000 7efe796ff700 1 --2- 172.21.15.7:0/3872291762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe74110bf0 0x7efe74109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.614+0000 7efe796ff700 1 --2- 172.21.15.7:0/3872291762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe74111060 0x7efe741099d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.615+0000 7efe796ff700 1 -- 172.21.15.7:0/3872291762 >> 172.21.15.7:0/3872291762 conn(0x7efe7406d600 msgr2=0x7efe7406da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:06.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.618+0000 7efe796ff700 1 -- 172.21.15.7:0/3872291762 shutdown_connections 2024-06-10T01:25:06.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.620+0000 7efe796ff700 1 -- 172.21.15.7:0/3872291762 wait complete. 2024-06-10T01:25:06.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.621+0000 7efe796ff700 1 Processor -- start 2024-06-10T01:25:06.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.621+0000 7efe796ff700 1 -- start start 2024-06-10T01:25:06.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.622+0000 7efe796ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe74110bf0 0x7efe741ad780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:06.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.622+0000 7efe796ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe7411bc70 0x7efe741adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:06.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.622+0000 7efe796ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe741a7940 0x7efe741a7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:06.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.622+0000 7efe796ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efe741216c0 con 0x7efe74110bf0 2024-06-10T01:25:06.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.622+0000 7efe796ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efe74121540 con 0x7efe7411bc70 2024-06-10T01:25:06.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.622+0000 7efe796ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efe74121840 con 0x7efe741a7940 2024-06-10T01:25:06.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.622+0000 7efe737fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe741a7940 0x7efe741a7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:06.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.622+0000 7efe737fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe741a7940 0x7efe741a7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:60784/0 (socket says 172.21.15.7:60784) 2024-06-10T01:25:06.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.622+0000 7efe737fe700 1 -- 172.21.15.7:0/889789118 learned_addr learned my addr 172.21.15.7:0/889789118 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:06.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.623+0000 7efe737fe700 1 -- 172.21.15.7:0/889789118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe7411bc70 msgr2=0x7efe741adcc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-10T01:25:06.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.623+0000 7efe737fe700 1 --2- 172.21.15.7:0/889789118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe7411bc70 0x7efe741adcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.623+0000 7efe737fe700 1 -- 172.21.15.7:0/889789118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe74110bf0 msgr2=0x7efe741ad780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:06.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.623+0000 7efe737fe700 1 --2- 172.21.15.7:0/889789118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe74110bf0 0x7efe741ad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.623+0000 7efe737fe700 1 -- 172.21.15.7:0/889789118 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe741a84c0 con 0x7efe741a7940 2024-06-10T01:25:06.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.623+0000 7efe737fe700 1 --2- 172.21.15.7:0/889789118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe741a7940 0x7efe741a7dc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efe6800e940 tx=0x7efe6800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:06.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.625+0000 7efe5bfff700 1 -- 172.21.15.7:0/889789118 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6800cbd0 con 0x7efe741a7940 2024-06-10T01:25:06.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.625+0000 7efe5bfff700 1 -- 172.21.15.7:0/889789118 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe6800cd70 con 0x7efe741a7940 2024-06-10T01:25:06.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.625+0000 7efe5bfff700 1 -- 172.21.15.7:0/889789118 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe68010430 con 0x7efe741a7940 2024-06-10T01:25:06.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.625+0000 7efe796ff700 1 -- 172.21.15.7:0/889789118 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe741b4650 con 0x7efe741a7940 2024-06-10T01:25:06.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.625+0000 7efe796ff700 1 -- 172.21.15.7:0/889789118 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe741b49f0 con 0x7efe741a7940 2024-06-10T01:25:06.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.629+0000 7efe5bfff700 1 -- 172.21.15.7:0/889789118 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7efe68004750 con 0x7efe741a7940 2024-06-10T01:25:06.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.633+0000 7efe83f82b80 1 Processor -- start 2024-06-10T01:25:06.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.633+0000 7efe83f82b80 1 -- start start 2024-06-10T01:25:06.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.633+0000 7efe83f82b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1babd8f60 0x55c1babd9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:06.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.633+0000 7efe83f82b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1babd9880 0x55c1babd3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:06.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.633+0000 7efe83f82b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1babd37a0 0x55c1babd3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:06.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.633+0000 7efe83f82b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c1baac5f60 con 0x55c1babd37a0 2024-06-10T01:25:06.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.633+0000 7efe83f82b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c1bab24850 con 0x55c1babd9880 2024-06-10T01:25:06.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.633+0000 7efe83f82b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c1baa8c460 con 0x55c1babd8f60 2024-06-10T01:25:06.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.633+0000 7efe72ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1babd9880 0x55c1babd3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:06.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.634+0000 7efe72ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1babd9880 0x55c1babd3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:34086/0 (socket says 172.21.15.7:34086) 2024-06-10T01:25:06.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.634+0000 7efe72ffd700 1 -- 172.21.15.7:0/1906956850 learned_addr learned my addr 172.21.15.7:0/1906956850 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:06.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.634+0000 7efe72ffd700 1 -- 172.21.15.7:0/1906956850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1babd8f60 msgr2=0x55c1babd9340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T01:25:06.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.634+0000 7efe72ffd700 1 --2- 172.21.15.7:0/1906956850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1babd8f60 0x55c1babd9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.634+0000 7efe72ffd700 1 -- 172.21.15.7:0/1906956850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1babd37a0 msgr2=0x55c1babd3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:06.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.634+0000 7efe72ffd700 1 --2- 172.21.15.7:0/1906956850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1babd37a0 0x55c1babd3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.634+0000 7efe72ffd700 1 -- 172.21.15.7:0/1906956850 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1baa738d0 con 0x55c1babd9880 2024-06-10T01:25:06.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.634+0000 7efe72ffd700 1 --2- 172.21.15.7:0/1906956850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1babd9880 0x55c1babd3260 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7efe6c009d70 tx=0x7efe6c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60881d4b0e5783f1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:06.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.635+0000 7efe597fa700 1 -- 172.21.15.7:0/1906956850 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c006e60 con 0x55c1babd9880 2024-06-10T01:25:06.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.635+0000 7efe597fa700 1 -- 172.21.15.7:0/1906956850 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe6c017070 con 0x55c1babd9880 2024-06-10T01:25:06.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.635+0000 7efe597fa700 1 -- 172.21.15.7:0/1906956850 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c011390 con 0x55c1babd9880 2024-06-10T01:25:06.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.635+0000 7efe5bfff700 1 --2- 172.21.15.7:0/889789118 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efe5c0a2da0 0x7efe5c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:06.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.635+0000 7efe5bfff700 1 -- 172.21.15.7:0/889789118 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7efe680cf5b0 con 0x7efe741a7940 2024-06-10T01:25:06.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.636+0000 7efe83f82b80 1 -- 172.21.15.7:0/1906956850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1babd9880 msgr2=0x55c1babd3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:06.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.636+0000 7efe83f82b80 1 --2- 172.21.15.7:0/1906956850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1babd9880 0x55c1babd3260 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7efe6c009d70 tx=0x7efe6c00b330 comp rx=0 tx=0).stop 2024-06-10T01:25:06.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.638+0000 7efe727fc700 1 --2- 172.21.15.7:0/889789118 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efe5c0a2da0 0x7efe5c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:06.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.639+0000 7efe83f82b80 1 -- 172.21.15.7:0/1906956850 shutdown_connections 2024-06-10T01:25:06.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.639+0000 7efe83f82b80 1 --2- 172.21.15.7:0/1906956850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1babd37a0 0x55c1babd3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.639+0000 7efe83f82b80 1 --2- 172.21.15.7:0/1906956850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1babd8f60 0x55c1babd9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.639+0000 7efe83f82b80 1 --2- 172.21.15.7:0/1906956850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1babd9880 0x55c1babd3260 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.639+0000 7efe727fc700 1 --2- 172.21.15.7:0/889789118 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efe5c0a2da0 0x7efe5c0a5260 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7efe640098e0 tx=0x7efe640095b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:06.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.639+0000 7efe83f82b80 1 -- 172.21.15.7:0/1906956850 >> 172.21.15.7:0/1906956850 conn(0x55c1bab00080 msgr2=0x55c1babce220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:06.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.639+0000 7efe83f82b80 1 -- 172.21.15.7:0/1906956850 shutdown_connections 2024-06-10T01:25:06.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.642+0000 7efe83f82b80 1 -- 172.21.15.7:0/1906956850 wait complete. 2024-06-10T01:25:06.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.642+0000 7efe83f82b80 1 -- start start 2024-06-10T01:25:06.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.643+0000 7efe83f82b80 1 Processor -- start 2024-06-10T01:25:06.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.643+0000 7efe83f82b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1babd37a0 0x55c1babdf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:06.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.643+0000 7efe83f82b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1babd8f60 0x55c1babdc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:06.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.643+0000 7efe83f82b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1babd9880 0x55c1babdceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:06.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.643+0000 7efe83f82b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c1baac4fc0 con 0x55c1babd37a0 2024-06-10T01:25:06.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.643+0000 7efe83f82b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c1bab20810 con 0x55c1babd9880 2024-06-10T01:25:06.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.643+0000 7efe83f82b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c1babe0260 con 0x55c1babd8f60 2024-06-10T01:25:06.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.643+0000 7efe83f82b80 10 client.0 _ll_register_callbacks cb 0x55c1baaba5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:25:06.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.643+0000 7efe72ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1babd37a0 0x55c1babdf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:06.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.643+0000 7efe737fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1babd9880 0x55c1babdceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:06.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.643+0000 7efe72ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1babd37a0 0x55c1babdf670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40698/0 (socket says 172.21.15.7:40698) 2024-06-10T01:25:06.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.643+0000 7efe72ffd700 1 -- 172.21.15.7:0/3746536704 learned_addr learned my addr 172.21.15.7:0/3746536704 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:06.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.644+0000 7efe727fc700 1 --2- 172.21.15.7:0/3746536704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1babd8f60 0x55c1babdc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:06.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.644+0000 7efe737fe700 1 -- 172.21.15.7:0/3746536704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1babd8f60 msgr2=0x55c1babdc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:06.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.644+0000 7efe737fe700 1 --2- 172.21.15.7:0/3746536704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1babd8f60 0x55c1babdc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.644+0000 7efe737fe700 1 -- 172.21.15.7:0/3746536704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1babd37a0 msgr2=0x55c1babdf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:06.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.644+0000 7efe737fe700 1 --2- 172.21.15.7:0/3746536704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1babd37a0 0x55c1babdf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.644+0000 7efe737fe700 1 -- 172.21.15.7:0/3746536704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1baaea750 con 0x55c1babd9880 2024-06-10T01:25:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.644+0000 7efe737fe700 1 --2- 172.21.15.7:0/3746536704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1babd9880 0x55c1babdceb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7efe680cd0a0 tx=0x7efe6809b520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=593abb8158eeda0f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.645+0000 7efe56ff5700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:25:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.645+0000 7efe56ff5700 1 -- 172.21.15.7:0/3746536704 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe680b3020 con 0x55c1babd9880 2024-06-10T01:25:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.645+0000 7efe56ff5700 1 -- 172.21.15.7:0/3746536704 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7efe680a4270 con 0x55c1babd9880 2024-06-10T01:25:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.645+0000 7efe83f82b80 1 -- 172.21.15.7:0/3746536704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe6c017070 con 0x55c1babd9880 2024-06-10T01:25:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.645+0000 7efe56ff5700 1 -- 172.21.15.7:0/3746536704 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7efe6809fe40 con 0x55c1babd9880 2024-06-10T01:25:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.645+0000 7efe83f82b80 10 client.29285 fetch_fsmap learned FSMap version 52 2024-06-10T01:25:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.645+0000 7efe83f82b80 1 -- 172.21.15.7:0/3746536704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55c1bac0cb30 con 0x55c1babd9880 2024-06-10T01:25:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.645+0000 7efe56ff5700 1 -- 172.21.15.7:0/3746536704 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe680a4700 con 0x55c1babd9880 2024-06-10T01:25:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.646+0000 7efe56ff5700 1 -- 172.21.15.7:0/3746536704 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7efe680977e0 con 0x55c1babd9880 2024-06-10T01:25:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.646+0000 7efe83f82b80 10 client.29285 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:25:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.646+0000 7efe83f82b80 10 client.29285 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:25:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.646+0000 7efe83f82b80 20 client.29285 populate_metadata read hostname 'smithi007' 2024-06-10T01:25:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.646+0000 7efe83f82b80 1 --2- 172.21.15.7:0/3746536704 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c1bac0f170 0x55c1bac11560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.646+0000 7efe83f82b80 4 client.29285 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:25:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.646+0000 7efe83f82b80 1 -- 172.21.15.7:0/3746536704 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efe6c017070 con 0x55c1bac0f170 2024-06-10T01:25:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.646+0000 7efe72ffd700 1 --2- 172.21.15.7:0/3746536704 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c1bac0f170 0x55c1bac11560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.647+0000 7efe72ffd700 1 --2- 172.21.15.7:0/3746536704 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c1bac0f170 0x55c1bac11560 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9268039c0bec40d0 server_cookie=befe4ac1d5c5b32e in_seq=0 out_seq=0 2024-06-10T01:25:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.647+0000 7efe56ff5700 10 client.29285 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.648+0000 7efe56ff5700 1 -- 172.21.15.7:0/3746536704 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7efe6c017070 con 0x55c1bac0f170 2024-06-10T01:25:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.648+0000 7efe56ff5700 10 client.29285 handle_command_reply: tid=0 2024-06-10T01:25:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.648+0000 7efe83f82b80 1 client.29285 shutdown 2024-06-10T01:25:06.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.649+0000 7efe83f82b80 1 -- 172.21.15.7:0/3746536704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1babd9880 msgr2=0x55c1babdceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:06.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.649+0000 7efe83f82b80 1 --2- 172.21.15.7:0/3746536704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1babd9880 0x55c1babdceb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7efe680cd0a0 tx=0x7efe6809b520 comp rx=0 tx=0).stop 2024-06-10T01:25:06.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.649+0000 7efe83f82b80 1 -- 172.21.15.7:0/3746536704 shutdown_connections 2024-06-10T01:25:06.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.649+0000 7efe83f82b80 1 --2- 172.21.15.7:0/3746536704 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c1bac0f170 0x55c1bac11560 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.649+0000 7efe83f82b80 1 --2- 172.21.15.7:0/3746536704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1babd9880 0x55c1babdceb0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.649+0000 7efe83f82b80 1 --2- 172.21.15.7:0/3746536704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1babd37a0 0x55c1babdf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.649+0000 7efe83f82b80 1 --2- 172.21.15.7:0/3746536704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1babd8f60 0x55c1babdc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.649+0000 7efe83f82b80 1 -- 172.21.15.7:0/3746536704 >> 172.21.15.7:0/3746536704 conn(0x55c1bab00080 msgr2=0x55c1babcd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.649+0000 7efe56ff5700 0 client.29285 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:06.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.660+0000 7efe83f82b80 1 -- 172.21.15.7:0/3746536704 shutdown_connections 2024-06-10T01:25:06.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.660+0000 7efe83f82b80 1 -- 172.21.15.7:0/3746536704 wait complete. 2024-06-10T01:25:06.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.660+0000 7efe83f82b80 20 client.29285 trim_cache size 0 max 16384 2024-06-10T01:25:06.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.695+0000 7efe83f82b80 1 Processor -- start 2024-06-10T01:25:06.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.695+0000 7efe83f82b80 1 -- start start 2024-06-10T01:25:06.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.695+0000 7efe83f82b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1babd37a0 0x55c1babd3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:06.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.695+0000 7efe83f82b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1babd8f60 0x55c1babe9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:06.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.695+0000 7efe83f82b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1babd9880 0x55c1babcde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:06.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.695+0000 7efe83f82b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efe60011340 con 0x55c1babd8f60 2024-06-10T01:25:06.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.695+0000 7efe83f82b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efe6000ebd0 con 0x55c1babd9880 2024-06-10T01:25:06.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.695+0000 7efe83f82b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efe6000ca20 con 0x55c1babd37a0 2024-06-10T01:25:06.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.695+0000 7efe727fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1babd8f60 0x55c1babe9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:06.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.695+0000 7efe727fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1babd8f60 0x55c1babe9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40706/0 (socket says 172.21.15.7:40706) 2024-06-10T01:25:06.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.695+0000 7efe727fc700 1 -- 172.21.15.7:0/1782899320 learned_addr learned my addr 172.21.15.7:0/1782899320 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:06.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.696+0000 7efe737fe700 1 --2- 172.21.15.7:0/1782899320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1babd9880 0x55c1babcde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:06.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.695+0000 7efe72ffd700 1 --2- 172.21.15.7:0/1782899320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1babd37a0 0x55c1babd3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:06.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.696+0000 7efe72ffd700 1 -- 172.21.15.7:0/1782899320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1babd9880 msgr2=0x55c1babcde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:06.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.696+0000 7efe72ffd700 1 --2- 172.21.15.7:0/1782899320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1babd9880 0x55c1babcde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.696+0000 7efe72ffd700 1 -- 172.21.15.7:0/1782899320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1babd8f60 msgr2=0x55c1babe9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:06.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.696+0000 7efe72ffd700 1 --2- 172.21.15.7:0/1782899320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1babd8f60 0x55c1babe9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.696+0000 7efe72ffd700 1 -- 172.21.15.7:0/1782899320 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1bac118a0 con 0x55c1babd37a0 2024-06-10T01:25:06.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.696+0000 7efe72ffd700 1 --2- 172.21.15.7:0/1782899320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1babd37a0 0x55c1babd3260 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55c1bac11d90 tx=0x7efe6c009ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61658fb21fe54b9a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:06.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.697+0000 7efe58ff9700 1 -- 172.21.15.7:0/1782899320 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c012c20 con 0x55c1babd37a0 2024-06-10T01:25:06.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.697+0000 7efe58ff9700 1 -- 172.21.15.7:0/1782899320 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe6c017040 con 0x55c1babd37a0 2024-06-10T01:25:06.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.698+0000 7efe83f82b80 1 -- 172.21.15.7:0/1782899320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1babd37a0 msgr2=0x55c1babd3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:06.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.698+0000 7efe83f82b80 1 --2- 172.21.15.7:0/1782899320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1babd37a0 0x55c1babd3260 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55c1bac11d90 tx=0x7efe6c009ec0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.702+0000 7efe83f82b80 1 -- 172.21.15.7:0/1782899320 shutdown_connections 2024-06-10T01:25:06.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.702+0000 7efe83f82b80 1 --2- 172.21.15.7:0/1782899320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1babd9880 0x55c1babcde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.702+0000 7efe83f82b80 1 --2- 172.21.15.7:0/1782899320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1babd37a0 0x55c1babd3260 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.702+0000 7efe83f82b80 1 --2- 172.21.15.7:0/1782899320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1babd8f60 0x55c1babe9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.702+0000 7efe83f82b80 1 -- 172.21.15.7:0/1782899320 >> 172.21.15.7:0/1782899320 conn(0x55c1bab00080 msgr2=0x55c1babdb260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:06.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.702+0000 7efe83f82b80 1 -- 172.21.15.7:0/1782899320 shutdown_connections 2024-06-10T01:25:06.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.702+0000 7efe83f82b80 1 -- 172.21.15.7:0/1782899320 wait complete. 2024-06-10T01:25:06.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.703+0000 7efe83f82b80 1 -- start start 2024-06-10T01:25:06.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.703+0000 7efe83f82b80 1 Processor -- start 2024-06-10T01:25:06.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.703+0000 7efe83f82b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1babd37a0 0x55c1babe57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:06.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.703+0000 7efe83f82b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1babd8f60 0x55c1babe5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:06.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.703+0000 7efe83f82b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1babd9880 0x55c1bac16430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:06.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.703+0000 7efe83f82b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efe6000d090 con 0x55c1babd37a0 2024-06-10T01:25:06.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.703+0000 7efe83f82b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efe6000e500 con 0x55c1babd9880 2024-06-10T01:25:06.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.703+0000 7efe83f82b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efe6000d760 con 0x55c1babd8f60 2024-06-10T01:25:06.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.703+0000 7efe83f82b80 10 client.0 _ll_register_callbacks cb 0x55c1baaba5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:25:06.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.706+0000 7efe727fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1babd8f60 0x55c1babe5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:06.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.706+0000 7efe737fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1babd9880 0x55c1bac16430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:06.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.706+0000 7efe737fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1babd9880 0x55c1bac16430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:34110/0 (socket says 172.21.15.7:34110) 2024-06-10T01:25:06.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.706+0000 7efe737fe700 1 -- 172.21.15.7:0/1757172290 learned_addr learned my addr 172.21.15.7:0/1757172290 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:06.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.706+0000 7efe737fe700 1 -- 172.21.15.7:0/1757172290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1babd8f60 msgr2=0x55c1babe5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:06.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.706+0000 7efe737fe700 1 --2- 172.21.15.7:0/1757172290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1babd8f60 0x55c1babe5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.706+0000 7efe737fe700 1 -- 172.21.15.7:0/1757172290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1babd37a0 msgr2=0x55c1babe57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:06.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.706+0000 7efe737fe700 1 --2- 172.21.15.7:0/1757172290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1babd37a0 0x55c1babe57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.706+0000 7efe737fe700 1 -- 172.21.15.7:0/1757172290 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1bac16ca0 con 0x55c1babd9880 2024-06-10T01:25:06.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.707+0000 7efe737fe700 1 --2- 172.21.15.7:0/1757172290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1babd9880 0x55c1bac16430 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efe68097ef0 tx=0x7efe6809b110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=542f1a026fee292c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:06.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.707+0000 7efe56ff5700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:25:06.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.707+0000 7efe56ff5700 1 -- 172.21.15.7:0/1757172290 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe680ac030 con 0x55c1babd9880 2024-06-10T01:25:06.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.707+0000 7efe83f82b80 1 -- 172.21.15.7:0/1757172290 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe6c017040 con 0x55c1babd9880 2024-06-10T01:25:06.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.707+0000 7efe56ff5700 1 -- 172.21.15.7:0/1757172290 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7efe680a4270 con 0x55c1babd9880 2024-06-10T01:25:06.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.710+0000 7efe56ff5700 1 -- 172.21.15.7:0/1757172290 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7efe6807f050 con 0x55c1babd9880 2024-06-10T01:25:06.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.711+0000 7efe83f82b80 10 client.29288 fetch_fsmap learned FSMap version 52 2024-06-10T01:25:06.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.711+0000 7efe83f82b80 1 -- 172.21.15.7:0/1757172290 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55c1bac79910 con 0x55c1babd9880 2024-06-10T01:25:06.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.714+0000 7efe56ff5700 1 -- 172.21.15.7:0/1757172290 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe6c017040 con 0x55c1babd9880 2024-06-10T01:25:06.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.714+0000 7efe56ff5700 1 -- 172.21.15.7:0/1757172290 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7efe6809aa20 con 0x55c1babd9880 2024-06-10T01:25:06.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.714+0000 7efe83f82b80 10 client.29288 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:25:06.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.715+0000 7efe83f82b80 10 client.29288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:25:06.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.715+0000 7efe83f82b80 20 client.29288 populate_metadata read hostname 'smithi007' 2024-06-10T01:25:06.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.715+0000 7efe83f82b80 1 --2- 172.21.15.7:0/1757172290 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c1bac79ec0 0x55c1bac7c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:06.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.715+0000 7efe83f82b80 4 client.29288 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:25:06.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.715+0000 7efe83f82b80 1 -- 172.21.15.7:0/1757172290 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c1bab31d50 con 0x55c1bac79ec0 2024-06-10T01:25:06.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.787+0000 7efe72ffd700 1 --2- 172.21.15.7:0/1757172290 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c1bac79ec0 0x55c1bac7c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:06.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.788+0000 7efe72ffd700 1 --2- 172.21.15.7:0/1757172290 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c1bac79ec0 0x55c1bac7c2e0 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff438479d652913d server_cookie=83a5335f970d0aaf in_seq=0 out_seq=0 2024-06-10T01:25:06.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.788+0000 7efe56ff5700 10 client.29288 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:06.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.797+0000 7efe56ff5700 1 -- 172.21.15.7:0/1757172290 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c1bab31d50 con 0x55c1bac79ec0 2024-06-10T01:25:06.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.797+0000 7efe56ff5700 10 client.29288 handle_command_reply: tid=0 2024-06-10T01:25:06.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.797+0000 7efe83f82b80 1 client.29288 shutdown 2024-06-10T01:25:06.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.798+0000 7efe83f82b80 1 -- 172.21.15.7:0/1757172290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1babd9880 msgr2=0x55c1bac16430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:06.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.798+0000 7efe83f82b80 1 --2- 172.21.15.7:0/1757172290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1babd9880 0x55c1bac16430 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efe68097ef0 tx=0x7efe6809b110 comp rx=0 tx=0).stop 2024-06-10T01:25:06.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.798+0000 7efe83f82b80 1 -- 172.21.15.7:0/1757172290 shutdown_connections 2024-06-10T01:25:06.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.798+0000 7efe83f82b80 1 --2- 172.21.15.7:0/1757172290 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c1bac79ec0 0x55c1bac7c2e0 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.798+0000 7efe83f82b80 1 --2- 172.21.15.7:0/1757172290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1babd9880 0x55c1bac16430 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.798+0000 7efe83f82b80 1 --2- 172.21.15.7:0/1757172290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1babd37a0 0x55c1babe57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.798+0000 7efe83f82b80 1 --2- 172.21.15.7:0/1757172290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c1babd8f60 0x55c1babe5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.798+0000 7efe83f82b80 1 -- 172.21.15.7:0/1757172290 >> 172.21.15.7:0/1757172290 conn(0x55c1bab00080 msgr2=0x55c1babd9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:06.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.798+0000 7efe56ff5700 0 client.29288 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:06.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.798+0000 7efe83f82b80 1 -- 172.21.15.7:0/1757172290 shutdown_connections 2024-06-10T01:25:06.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.798+0000 7efe83f82b80 1 -- 172.21.15.7:0/1757172290 wait complete. 2024-06-10T01:25:06.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.798+0000 7efe83f82b80 20 client.29288 trim_cache size 0 max 16384 2024-06-10T01:25:06.792 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:25:06.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.799+0000 7efe796ff700 1 -- 172.21.15.7:0/889789118 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efe5c0a2da0 msgr2=0x7efe5c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:06.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.799+0000 7efe796ff700 1 --2- 172.21.15.7:0/889789118 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efe5c0a2da0 0x7efe5c0a5260 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7efe640098e0 tx=0x7efe640095b0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.799+0000 7efe796ff700 1 -- 172.21.15.7:0/889789118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe741a7940 msgr2=0x7efe741a7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:06.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.799+0000 7efe796ff700 1 --2- 172.21.15.7:0/889789118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe741a7940 0x7efe741a7dc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efe6800e940 tx=0x7efe6800ed00 comp rx=0 tx=0).stop 2024-06-10T01:25:06.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.799+0000 7efe796ff700 1 -- 172.21.15.7:0/889789118 shutdown_connections 2024-06-10T01:25:06.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.799+0000 7efe796ff700 1 --2- 172.21.15.7:0/889789118 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efe5c0a2da0 0x7efe5c0a5260 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.799+0000 7efe796ff700 1 --2- 172.21.15.7:0/889789118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe741a7940 0x7efe741a7dc0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.799+0000 7efe796ff700 1 --2- 172.21.15.7:0/889789118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe74110bf0 0x7efe741ad780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.799+0000 7efe796ff700 1 --2- 172.21.15.7:0/889789118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe7411bc70 0x7efe741adcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:06.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.799+0000 7efe796ff700 1 -- 172.21.15.7:0/889789118 >> 172.21.15.7:0/889789118 conn(0x7efe7406d600 msgr2=0x7efe74119c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:06.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.799+0000 7efe796ff700 1 -- 172.21.15.7:0/889789118 shutdown_connections 2024-06-10T01:25:06.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:06.799+0000 7efe796ff700 1 -- 172.21.15.7:0/889789118 wait complete. 2024-06-10T01:25:07.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:07 smithi007 ceph-mon[32398]: pgmap v1006: 129 pgs: 129 active+clean; 699 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 716 op/s 2024-06-10T01:25:07.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 137985 23.45 MB/sec execute 224 sec latency 261.780 ms 2024-06-10T01:25:07.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:07 smithi078 ceph-mon[36683]: pgmap v1006: 129 pgs: 129 active+clean; 699 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 716 op/s 2024-06-10T01:25:07.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:07 smithi175 ceph-mon[36636]: pgmap v1006: 129 pgs: 129 active+clean; 699 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 716 op/s 2024-06-10T01:25:08.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 138189 23.37 MB/sec execute 225 sec latency 756.197 ms 2024-06-10T01:25:08.585 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:25:08.586 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:25:08.914 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:25:08.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:08 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:25:08.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:08 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:25:08.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:08 smithi175 ceph-mon[36636]: pgmap v1007: 129 pgs: 129 active+clean; 699 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 9.8 MiB/s wr, 596 op/s 2024-06-10T01:25:08.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:08 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:25:08.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:08 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:25:09.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:08 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:25:09.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:08 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:25:09.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:08 smithi007 ceph-mon[32398]: pgmap v1007: 129 pgs: 129 active+clean; 699 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 9.8 MiB/s wr, 596 op/s 2024-06-10T01:25:09.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:08 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:25:09.012 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:08 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:25:09.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:08 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:25:09.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:08 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:25:09.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:08 smithi078 ceph-mon[36683]: pgmap v1007: 129 pgs: 129 active+clean; 699 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 9.8 MiB/s wr, 596 op/s 2024-06-10T01:25:09.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:08 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:25:09.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:08 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:25:09.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 138371 23.30 MB/sec execute 226 sec latency 1026.638 ms 2024-06-10T01:25:09.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:09 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:25:09.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:09 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:25:09.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:09 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:25:10.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:09 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:25:10.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:09 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:25:10.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:09 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:25:10.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:09 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:25:10.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:09 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:25:10.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:09 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:25:10.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 138647 23.28 MB/sec execute 227 sec latency 143.763 ms 2024-06-10T01:25:10.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:10 smithi007 ceph-mon[32398]: pgmap v1008: 129 pgs: 129 active+clean; 669 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 14 MiB/s wr, 811 op/s 2024-06-10T01:25:10.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:10 smithi175 ceph-mon[36636]: pgmap v1008: 129 pgs: 129 active+clean; 669 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 14 MiB/s wr, 811 op/s 2024-06-10T01:25:11.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:10 smithi078 ceph-mon[36683]: pgmap v1008: 129 pgs: 129 active+clean; 669 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 14 MiB/s wr, 811 op/s 2024-06-10T01:25:11.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 138957 23.25 MB/sec execute 228 sec latency 260.905 ms 2024-06-10T01:25:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.908+0000 7fb4ffcd6640 1 -- 172.21.15.7:0/2486735992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f8101af0 msgr2=0x7fb4f8101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.908+0000 7fb4ffcd6640 1 --2- 172.21.15.7:0/2486735992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f8101af0 0x7fb4f8101f70 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8009900 tx=0x7fb4e802a410 comp rx=0 tx=0).stop 2024-06-10T01:25:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.909+0000 7fb4ffcd6640 1 -- 172.21.15.7:0/2486735992 shutdown_connections 2024-06-10T01:25:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.909+0000 7fb4ffcd6640 1 --2- 172.21.15.7:0/2486735992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4f81024b0 0x7fb4f8108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.909+0000 7fb4ffcd6640 1 --2- 172.21.15.7:0/2486735992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f8101af0 0x7fb4f8101f70 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.909+0000 7fb4ffcd6640 1 --2- 172.21.15.7:0/2486735992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4f8100900 0x7fb4f8100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.909+0000 7fb4ffcd6640 1 -- 172.21.15.7:0/2486735992 >> 172.21.15.7:0/2486735992 conn(0x7fb4f80fbe70 msgr2=0x7fb4f80fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.909+0000 7fb4ffcd6640 1 -- 172.21.15.7:0/2486735992 shutdown_connections 2024-06-10T01:25:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.909+0000 7fb4ffcd6640 1 -- 172.21.15.7:0/2486735992 wait complete. 2024-06-10T01:25:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.909+0000 7fb4ffcd6640 1 Processor -- start 2024-06-10T01:25:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.909+0000 7fb4ffcd6640 1 -- start start 2024-06-10T01:25:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.910+0000 7fb4ffcd6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4f8100900 0x7fb4f8195b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.910+0000 7fb4ffcd6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4f8101af0 0x7fb4f8196060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.910+0000 7fb4ffcd6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f81024b0 0x7fb4f819d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.910+0000 7fb4ffcd6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f810b480 con 0x7fb4f8101af0 2024-06-10T01:25:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.910+0000 7fb4ffcd6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f810b300 con 0x7fb4f81024b0 2024-06-10T01:25:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.910+0000 7fb4ffcd6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f810b600 con 0x7fb4f8100900 2024-06-10T01:25:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.910+0000 7fb4fd271640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4f8101af0 0x7fb4f8196060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.910+0000 7fb4fd271640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4f8101af0 0x7fb4f8196060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59356/0 (socket says 172.21.15.7:59356) 2024-06-10T01:25:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.910+0000 7fb4fd271640 1 -- 172.21.15.7:0/1655303010 learned_addr learned my addr 172.21.15.7:0/1655303010 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.910+0000 7fb4fda72640 1 --2- 172.21.15.7:0/1655303010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4f8100900 0x7fb4f8195b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.910+0000 7fb4fe273640 1 --2- 172.21.15.7:0/1655303010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f81024b0 0x7fb4f819d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.910+0000 7fb4fd271640 1 -- 172.21.15.7:0/1655303010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4f8100900 msgr2=0x7fb4f8195b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.910+0000 7fb4fd271640 1 --2- 172.21.15.7:0/1655303010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4f8100900 0x7fb4f8195b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.910+0000 7fb4fd271640 1 -- 172.21.15.7:0/1655303010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f81024b0 msgr2=0x7fb4f819d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.910+0000 7fb4fd271640 1 --2- 172.21.15.7:0/1655303010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f81024b0 0x7fb4f819d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.910+0000 7fb4fd271640 1 -- 172.21.15.7:0/1655303010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f819d740 con 0x7fb4f8101af0 2024-06-10T01:25:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.910+0000 7fb4fe273640 1 --2- 172.21.15.7:0/1655303010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f81024b0 0x7fb4f819d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:25:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.910+0000 7fb4fd271640 1 --2- 172.21.15.7:0/1655303010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4f8101af0 0x7fb4f8196060 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8009230 tx=0x7fb4e80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.911+0000 7fb4eeffd640 1 -- 172.21.15.7:0/1655303010 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e803f030 con 0x7fb4f8101af0 2024-06-10T01:25:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.911+0000 7fb4ffcd6640 1 -- 172.21.15.7:0/1655303010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4f819d9d0 con 0x7fb4f8101af0 2024-06-10T01:25:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.911+0000 7fb4eeffd640 1 -- 172.21.15.7:0/1655303010 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4e8003ce0 con 0x7fb4f8101af0 2024-06-10T01:25:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.911+0000 7fb4eeffd640 1 -- 172.21.15.7:0/1655303010 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8035070 con 0x7fb4f8101af0 2024-06-10T01:25:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.911+0000 7fb4ffcd6640 1 -- 172.21.15.7:0/1655303010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4f819dec0 con 0x7fb4f8101af0 2024-06-10T01:25:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.913+0000 7fb4eeffd640 1 -- 172.21.15.7:0/1655303010 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb4e8042030 con 0x7fb4f8101af0 2024-06-10T01:25:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.914+0000 7fb5029e2740 1 Processor -- start 2024-06-10T01:25:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.914+0000 7fb5029e2740 1 -- start start 2024-06-10T01:25:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.915+0000 7fb5029e2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef6fedd010 0x55ef6fedd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.915+0000 7fb5029e2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6fedda00 0x55ef6fee2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.915+0000 7fb5029e2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef6fee2b70 0x55ef6fee5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.915+0000 7fb5029e2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ef6fe404f0 con 0x55ef6fedd010 2024-06-10T01:25:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.915+0000 7fb5029e2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ef6fdde870 con 0x55ef6fedda00 2024-06-10T01:25:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.915+0000 7fb5029e2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ef6fe3c4b0 con 0x55ef6fee2b70 2024-06-10T01:25:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.915+0000 7fb4eeffd640 1 --2- 172.21.15.7:0/1655303010 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb4e40a2b80 0x7fb4e40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.915+0000 7fb4fda72640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6fedda00 0x55ef6fee2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.915+0000 7fb4fda72640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6fedda00 0x55ef6fee2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:45010/0 (socket says 172.21.15.7:45010) 2024-06-10T01:25:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.915+0000 7fb4fda72640 1 -- 172.21.15.7:0/3927837309 learned_addr learned my addr 172.21.15.7:0/3927837309 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.915+0000 7fb4fe273640 1 --2- 172.21.15.7:0/3927837309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef6fedd010 0x55ef6fedd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.915+0000 7fb4fe273640 1 --2- 172.21.15.7:0/1655303010 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb4e40a2b80 0x7fb4e40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.915+0000 7fb4eeffd640 1 -- 172.21.15.7:0/1655303010 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fb4e80ef800 con 0x7fb4f8101af0 2024-06-10T01:25:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.915+0000 7fb4fd271640 1 --2- 172.21.15.7:0/3927837309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef6fee2b70 0x55ef6fee5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.915+0000 7fb4fda72640 1 -- 172.21.15.7:0/3927837309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef6fee2b70 msgr2=0x55ef6fee5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.915+0000 7fb4fda72640 1 --2- 172.21.15.7:0/3927837309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef6fee2b70 0x55ef6fee5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.915+0000 7fb4fda72640 1 -- 172.21.15.7:0/3927837309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef6fedd010 msgr2=0x55ef6fedd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.915+0000 7fb4fda72640 1 --2- 172.21.15.7:0/3927837309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef6fedd010 0x55ef6fedd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.915+0000 7fb4fda72640 1 -- 172.21.15.7:0/3927837309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef6fdfa9b0 con 0x55ef6fedda00 2024-06-10T01:25:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.915+0000 7fb4fd271640 1 --2- 172.21.15.7:0/3927837309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef6fee2b70 0x55ef6fee5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:25:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.915+0000 7fb4fe273640 1 --2- 172.21.15.7:0/1655303010 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb4e40a2b80 0x7fb4e40a5040 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009fd0 tx=0x7fb4f0009f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.915+0000 7fb4fda72640 1 --2- 172.21.15.7:0/3927837309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6fedda00 0x55ef6fee2500 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009ca0 tx=0x7fb4f4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8350b0483f8c6df server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.916+0000 7fb4e3fff640 1 -- 172.21.15.7:0/3927837309 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4004220 con 0x55ef6fedda00 2024-06-10T01:25:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.916+0000 7fb4e3fff640 1 -- 172.21.15.7:0/3927837309 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4f40043c0 con 0x55ef6fedda00 2024-06-10T01:25:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.916+0000 7fb4e3fff640 1 -- 172.21.15.7:0/3927837309 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4019860 con 0x55ef6fedda00 2024-06-10T01:25:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.916+0000 7fb5029e2740 1 -- 172.21.15.7:0/3927837309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6fedda00 msgr2=0x55ef6fee2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.916+0000 7fb5029e2740 1 --2- 172.21.15.7:0/3927837309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6fedda00 0x55ef6fee2500 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009ca0 tx=0x7fb4f4011810 comp rx=0 tx=0).stop 2024-06-10T01:25:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.916+0000 7fb5029e2740 1 -- 172.21.15.7:0/3927837309 shutdown_connections 2024-06-10T01:25:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.916+0000 7fb5029e2740 1 --2- 172.21.15.7:0/3927837309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef6fee2b70 0x55ef6fee5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.916+0000 7fb5029e2740 1 --2- 172.21.15.7:0/3927837309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6fedda00 0x55ef6fee2500 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.916+0000 7fb5029e2740 1 --2- 172.21.15.7:0/3927837309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef6fedd010 0x55ef6fedd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.916+0000 7fb5029e2740 1 -- 172.21.15.7:0/3927837309 >> 172.21.15.7:0/3927837309 conn(0x55ef6fed8680 msgr2=0x55ef6fed8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.917+0000 7fb5029e2740 1 -- 172.21.15.7:0/3927837309 shutdown_connections 2024-06-10T01:25:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.917+0000 7fb5029e2740 1 -- 172.21.15.7:0/3927837309 wait complete. 2024-06-10T01:25:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.917+0000 7fb5029e2740 1 -- start start 2024-06-10T01:25:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.917+0000 7fb5029e2740 1 Processor -- start 2024-06-10T01:25:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.917+0000 7fb5029e2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6fedd010 0x55ef6fee7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.917+0000 7fb5029e2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef6fedda00 0x55ef6fee7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.917+0000 7fb5029e2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef6fee2b70 0x55ef6feee7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.917+0000 7fb5029e2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ef6fee5b20 con 0x55ef6fedda00 2024-06-10T01:25:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.917+0000 7fb5029e2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ef6fdc9ae0 con 0x55ef6fedd010 2024-06-10T01:25:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.917+0000 7fb5029e2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ef6fee5ca0 con 0x55ef6fee2b70 2024-06-10T01:25:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.917+0000 7fb5029e2740 10 client.0 _ll_register_callbacks cb 0x55ef6fe029b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:25:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.917+0000 7fb4fd271640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef6fedda00 0x55ef6fee7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.918+0000 7fb4fda72640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6fedd010 0x55ef6fee7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.918+0000 7fb4fd271640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef6fedda00 0x55ef6fee7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59382/0 (socket says 172.21.15.7:59382) 2024-06-10T01:25:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.918+0000 7fb4fd271640 1 -- 172.21.15.7:0/707281089 learned_addr learned my addr 172.21.15.7:0/707281089 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.918+0000 7fb4fe273640 1 --2- 172.21.15.7:0/707281089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef6fee2b70 0x55ef6feee7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.918+0000 7fb4fd271640 1 -- 172.21.15.7:0/707281089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef6fee2b70 msgr2=0x55ef6feee7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.918+0000 7fb4fd271640 1 --2- 172.21.15.7:0/707281089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef6fee2b70 0x55ef6feee7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.918+0000 7fb4fd271640 1 -- 172.21.15.7:0/707281089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6fedd010 msgr2=0x55ef6fee7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.918+0000 7fb4fd271640 1 --2- 172.21.15.7:0/707281089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6fedd010 0x55ef6fee7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.918+0000 7fb4fd271640 1 -- 172.21.15.7:0/707281089 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef6fb54210 con 0x55ef6fedda00 2024-06-10T01:25:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.918+0000 7fb4fe273640 1 --2- 172.21.15.7:0/707281089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef6fee2b70 0x55ef6feee7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:25:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.918+0000 7fb4fda72640 1 --2- 172.21.15.7:0/707281089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6fedd010 0x55ef6fee7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:25:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.919+0000 7fb4fd271640 1 --2- 172.21.15.7:0/707281089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef6fedda00 0x55ef6fee7760 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80314e0 tx=0x7fb4e80bb8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=328d731c7ecb92e4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.919+0000 7fb4e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:25:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.919+0000 7fb4e17fa640 1 -- 172.21.15.7:0/707281089 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8049040 con 0x55ef6fedda00 2024-06-10T01:25:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.919+0000 7fb4e17fa640 1 -- 172.21.15.7:0/707281089 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fb4e803b460 con 0x55ef6fedda00 2024-06-10T01:25:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.919+0000 7fb5029e2740 1 -- 172.21.15.7:0/707281089 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4f40043c0 con 0x55ef6fedda00 2024-06-10T01:25:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.919+0000 7fb4e17fa640 1 -- 172.21.15.7:0/707281089 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fb4e80c0090 con 0x55ef6fedda00 2024-06-10T01:25:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.919+0000 7fb5029e2740 10 client.20091 fetch_fsmap learned FSMap version 52 2024-06-10T01:25:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.919+0000 7fb5029e2740 1 -- 172.21.15.7:0/707281089 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55ef6fc20620 con 0x55ef6fedda00 2024-06-10T01:25:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.919+0000 7fb4e17fa640 1 -- 172.21.15.7:0/707281089 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4f40043c0 con 0x55ef6fedda00 2024-06-10T01:25:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.920+0000 7fb4e17fa640 1 -- 172.21.15.7:0/707281089 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7fb4e80d9030 con 0x55ef6fedda00 2024-06-10T01:25:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.920+0000 7fb5029e2740 10 client.20091 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:25:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.920+0000 7fb5029e2740 10 client.20091 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:25:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.920+0000 7fb5029e2740 20 client.20091 populate_metadata read hostname 'smithi007' 2024-06-10T01:25:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.920+0000 7fb5029e2740 1 --2- 172.21.15.7:0/707281089 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ef6fef3010 0x55ef6fef54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.920+0000 7fb5029e2740 4 client.20091 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:25:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.920+0000 7fb5029e2740 1 -- 172.21.15.7:0/707281089 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef6fe53f10 con 0x55ef6fef3010 2024-06-10T01:25:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.920+0000 7fb4fda72640 1 --2- 172.21.15.7:0/707281089 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ef6fef3010 0x55ef6fef54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.921+0000 7fb4fda72640 1 --2- 172.21.15.7:0/707281089 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ef6fef3010 0x55ef6fef54d0 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24bce8d3c9c0309e server_cookie=54cce167da92bda5 in_seq=0 out_seq=0 2024-06-10T01:25:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.921+0000 7fb4e17fa640 10 client.20091 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.922+0000 7fb4e17fa640 1 -- 172.21.15.7:0/707281089 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ef6fe53f10 con 0x55ef6fef3010 2024-06-10T01:25:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.922+0000 7fb4e17fa640 10 client.20091 handle_command_reply: tid=0 2024-06-10T01:25:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.923+0000 7fb5029e2740 1 client.20091 shutdown 2024-06-10T01:25:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.923+0000 7fb5029e2740 1 -- 172.21.15.7:0/707281089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef6fedda00 msgr2=0x55ef6fee7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.923+0000 7fb5029e2740 1 --2- 172.21.15.7:0/707281089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef6fedda00 0x55ef6fee7760 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80314e0 tx=0x7fb4e80bb8d0 comp rx=0 tx=0).stop 2024-06-10T01:25:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.923+0000 7fb5029e2740 1 -- 172.21.15.7:0/707281089 shutdown_connections 2024-06-10T01:25:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.923+0000 7fb5029e2740 1 --2- 172.21.15.7:0/707281089 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ef6fef3010 0x55ef6fef54d0 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.923+0000 7fb5029e2740 1 --2- 172.21.15.7:0/707281089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef6fee2b70 0x55ef6feee7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.923+0000 7fb5029e2740 1 --2- 172.21.15.7:0/707281089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef6fedda00 0x55ef6fee7760 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.923+0000 7fb4e17fa640 0 client.20091 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.923+0000 7fb5029e2740 1 --2- 172.21.15.7:0/707281089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6fedd010 0x55ef6fee7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.923+0000 7fb5029e2740 1 -- 172.21.15.7:0/707281089 >> 172.21.15.7:0/707281089 conn(0x55ef6fed8680 msgr2=0x55ef6fed77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.923+0000 7fb5029e2740 1 -- 172.21.15.7:0/707281089 shutdown_connections 2024-06-10T01:25:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.923+0000 7fb5029e2740 1 -- 172.21.15.7:0/707281089 wait complete. 2024-06-10T01:25:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.923+0000 7fb5029e2740 20 client.20091 trim_cache size 0 max 16384 2024-06-10T01:25:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.940+0000 7fb5029e2740 1 Processor -- start 2024-06-10T01:25:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.940+0000 7fb5029e2740 1 -- start start 2024-06-10T01:25:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.940+0000 7fb5029e2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef6fedd010 0x55ef6fee2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.940+0000 7fb5029e2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef6fedda00 0x55ef6feee760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.940+0000 7fb5029e2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6fee2b70 0x55ef6fedeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.940+0000 7fb5029e2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f81aac30 con 0x55ef6fedda00 2024-06-10T01:25:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.940+0000 7fb5029e2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f81a8500 con 0x55ef6fee2b70 2024-06-10T01:25:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.940+0000 7fb5029e2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f81a6350 con 0x55ef6fedd010 2024-06-10T01:25:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.940+0000 7fb4fd271640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef6fedda00 0x55ef6feee760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.940+0000 7fb4fd271640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef6fedda00 0x55ef6feee760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59394/0 (socket says 172.21.15.7:59394) 2024-06-10T01:25:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.940+0000 7fb4fd271640 1 -- 172.21.15.7:0/3393551917 learned_addr learned my addr 172.21.15.7:0/3393551917 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.940+0000 7fb4fda72640 1 --2- 172.21.15.7:0/3393551917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef6fedd010 0x55ef6fee2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.940+0000 7fb4fe273640 1 --2- 172.21.15.7:0/3393551917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6fee2b70 0x55ef6fedeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.940+0000 7fb4fd271640 1 -- 172.21.15.7:0/3393551917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef6fedd010 msgr2=0x55ef6fee2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.940+0000 7fb4fd271640 1 --2- 172.21.15.7:0/3393551917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef6fedd010 0x55ef6fee2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.940+0000 7fb4fd271640 1 -- 172.21.15.7:0/3393551917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6fee2b70 msgr2=0x55ef6fedeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.940+0000 7fb4fd271640 1 --2- 172.21.15.7:0/3393551917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6fee2b70 0x55ef6fedeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.940+0000 7fb4fd271640 1 -- 172.21.15.7:0/3393551917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef6fedd7a0 con 0x55ef6fedda00 2024-06-10T01:25:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.941+0000 7fb4fd271640 1 --2- 172.21.15.7:0/3393551917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef6fedda00 0x55ef6feee760 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80b8760 tx=0x7fb4e80ba2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66193f1bee994311 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.941+0000 7fb4e37fe640 1 -- 172.21.15.7:0/3393551917 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8049030 con 0x55ef6fedda00 2024-06-10T01:25:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.941+0000 7fb4e37fe640 1 -- 172.21.15.7:0/3393551917 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4e80c4950 con 0x55ef6fedda00 2024-06-10T01:25:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.941+0000 7fb4e37fe640 1 -- 172.21.15.7:0/3393551917 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e80c44c0 con 0x55ef6fedda00 2024-06-10T01:25:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.942+0000 7fb5029e2740 1 -- 172.21.15.7:0/3393551917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef6fedda00 msgr2=0x55ef6feee760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.942+0000 7fb5029e2740 1 --2- 172.21.15.7:0/3393551917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef6fedda00 0x55ef6feee760 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80b8760 tx=0x7fb4e80ba2a0 comp rx=0 tx=0).stop 2024-06-10T01:25:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.942+0000 7fb5029e2740 1 -- 172.21.15.7:0/3393551917 shutdown_connections 2024-06-10T01:25:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.942+0000 7fb5029e2740 1 --2- 172.21.15.7:0/3393551917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6fee2b70 0x55ef6fedeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.942+0000 7fb5029e2740 1 --2- 172.21.15.7:0/3393551917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef6fedda00 0x55ef6feee760 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.942+0000 7fb5029e2740 1 --2- 172.21.15.7:0/3393551917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef6fedd010 0x55ef6fee2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.942+0000 7fb5029e2740 1 -- 172.21.15.7:0/3393551917 >> 172.21.15.7:0/3393551917 conn(0x55ef6fed8680 msgr2=0x55ef6fed8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.942+0000 7fb5029e2740 1 -- 172.21.15.7:0/3393551917 shutdown_connections 2024-06-10T01:25:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.942+0000 7fb5029e2740 1 -- 172.21.15.7:0/3393551917 wait complete. 2024-06-10T01:25:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.942+0000 7fb5029e2740 1 -- start start 2024-06-10T01:25:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.942+0000 7fb5029e2740 1 Processor -- start 2024-06-10T01:25:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.943+0000 7fb5029e2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef6fedd010 0x55ef6feee620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.943+0000 7fb5029e2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6fedda00 0x55ef6feeb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.943+0000 7fb5029e2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef6fee2b70 0x55ef6feeb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.943+0000 7fb5029e2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f81a69c0 con 0x55ef6fee2b70 2024-06-10T01:25:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.943+0000 7fb5029e2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f81a7e30 con 0x55ef6fedda00 2024-06-10T01:25:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.943+0000 7fb5029e2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f81a7090 con 0x55ef6fedd010 2024-06-10T01:25:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.943+0000 7fb5029e2740 10 client.0 _ll_register_callbacks cb 0x7fb4e8031a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:25:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.943+0000 7fb4fd271640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6fedda00 0x55ef6feeb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.943+0000 7fb4fd271640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6fedda00 0x55ef6feeb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:45040/0 (socket says 172.21.15.7:45040) 2024-06-10T01:25:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.943+0000 7fb4fd271640 1 -- 172.21.15.7:0/3356718363 learned_addr learned my addr 172.21.15.7:0/3356718363 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.943+0000 7fb4fda72640 1 --2- 172.21.15.7:0/3356718363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef6fedd010 0x55ef6feee620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.943+0000 7fb4fe273640 1 --2- 172.21.15.7:0/3356718363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef6fee2b70 0x55ef6feeb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.943+0000 7fb4fd271640 1 -- 172.21.15.7:0/3356718363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef6fedd010 msgr2=0x55ef6feee620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.943+0000 7fb4fd271640 1 --2- 172.21.15.7:0/3356718363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef6fedd010 0x55ef6feee620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.943+0000 7fb4fd271640 1 -- 172.21.15.7:0/3356718363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef6fee2b70 msgr2=0x55ef6feeb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.943+0000 7fb4fd271640 1 --2- 172.21.15.7:0/3356718363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef6fee2b70 0x55ef6feeb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.943+0000 7fb4fd271640 1 -- 172.21.15.7:0/3356718363 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef6ff5bda0 con 0x55ef6fedda00 2024-06-10T01:25:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.944+0000 7fb4fd271640 1 --2- 172.21.15.7:0/3356718363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6fedda00 0x55ef6feeb0c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80bba90 tx=0x7fb4e80bba00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a8b4653113f7bee server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.944+0000 7fb4e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:25:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.944+0000 7fb4e17fa640 1 -- 172.21.15.7:0/3356718363 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8049070 con 0x55ef6fedda00 2024-06-10T01:25:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.944+0000 7fb4e17fa640 1 -- 172.21.15.7:0/3356718363 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fb4e80c46d0 con 0x55ef6fedda00 2024-06-10T01:25:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.944+0000 7fb5029e2740 1 -- 172.21.15.7:0/3356718363 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4e80c4950 con 0x55ef6fedda00 2024-06-10T01:25:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.944+0000 7fb4e17fa640 1 -- 172.21.15.7:0/3356718363 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fb4e80b8030 con 0x55ef6fedda00 2024-06-10T01:25:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.944+0000 7fb5029e2740 10 client.29306 fetch_fsmap learned FSMap version 52 2024-06-10T01:25:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.944+0000 7fb5029e2740 1 -- 172.21.15.7:0/3356718363 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55ef6ff5bda0 con 0x55ef6fedda00 2024-06-10T01:25:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.944+0000 7fb4e17fa640 1 -- 172.21.15.7:0/3356718363 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4e80c4950 con 0x55ef6fedda00 2024-06-10T01:25:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.944+0000 7fb4e17fa640 1 -- 172.21.15.7:0/3356718363 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7fb4e80d9070 con 0x55ef6fedda00 2024-06-10T01:25:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.944+0000 7fb5029e2740 10 client.29306 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:25:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.944+0000 7fb5029e2740 10 client.29306 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:25:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.944+0000 7fb5029e2740 20 client.29306 populate_metadata read hostname 'smithi007' 2024-06-10T01:25:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.944+0000 7fb5029e2740 1 --2- 172.21.15.7:0/3356718363 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ef6ff619c0 0x55ef6ff63de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.944+0000 7fb5029e2740 4 client.29306 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:25:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.944+0000 7fb5029e2740 1 -- 172.21.15.7:0/3356718363 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ef6fe4dc20 con 0x55ef6ff619c0 2024-06-10T01:25:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.945+0000 7fb4fda72640 1 --2- 172.21.15.7:0/3356718363 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ef6ff619c0 0x55ef6ff63de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.946+0000 7fb4fda72640 1 --2- 172.21.15.7:0/3356718363 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ef6ff619c0 0x55ef6ff63de0 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89da2bba7d736c5b server_cookie=abf7a351ee2ae94a in_seq=0 out_seq=0 2024-06-10T01:25:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:11.946+0000 7fb4e17fa640 10 client.29306 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:12.256+0000 7fb4e17fa640 1 -- 172.21.15.7:0/3356718363 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ef6fe4dc20 con 0x55ef6ff619c0 2024-06-10T01:25:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:12.256+0000 7fb4e17fa640 10 client.29306 handle_command_reply: tid=0 2024-06-10T01:25:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:12.256+0000 7fb5029e2740 1 client.29306 shutdown 2024-06-10T01:25:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:12.256+0000 7fb5029e2740 1 -- 172.21.15.7:0/3356718363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6fedda00 msgr2=0x55ef6feeb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:12.256+0000 7fb5029e2740 1 --2- 172.21.15.7:0/3356718363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6fedda00 0x55ef6feeb0c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80bba90 tx=0x7fb4e80bba00 comp rx=0 tx=0).stop 2024-06-10T01:25:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:12.256+0000 7fb5029e2740 1 -- 172.21.15.7:0/3356718363 shutdown_connections 2024-06-10T01:25:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:12.256+0000 7fb5029e2740 1 --2- 172.21.15.7:0/3356718363 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ef6ff619c0 0x55ef6ff63de0 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:12.256+0000 7fb5029e2740 1 --2- 172.21.15.7:0/3356718363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef6fee2b70 0x55ef6feeb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:12.256+0000 7fb5029e2740 1 --2- 172.21.15.7:0/3356718363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6fedda00 0x55ef6feeb0c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:12.256+0000 7fb5029e2740 1 --2- 172.21.15.7:0/3356718363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef6fedd010 0x55ef6feee620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:12.256+0000 7fb5029e2740 1 -- 172.21.15.7:0/3356718363 >> 172.21.15.7:0/3356718363 conn(0x55ef6fed8680 msgr2=0x55ef6fee7200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:12.256+0000 7fb4e17fa640 0 client.29306 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:12.256+0000 7fb5029e2740 1 -- 172.21.15.7:0/3356718363 shutdown_connections 2024-06-10T01:25:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:12.256+0000 7fb5029e2740 1 -- 172.21.15.7:0/3356718363 wait complete. 2024-06-10T01:25:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:12.256+0000 7fb5029e2740 20 client.29306 trim_cache size 0 max 16384 2024-06-10T01:25:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:25:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:25:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:25:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:12.257+0000 7fb4ffcd6640 1 -- 172.21.15.7:0/1655303010 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb4e40a2b80 msgr2=0x7fb4e40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:12.257+0000 7fb4ffcd6640 1 --2- 172.21.15.7:0/1655303010 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb4e40a2b80 0x7fb4e40a5040 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009fd0 tx=0x7fb4f0009f00 comp rx=0 tx=0).stop 2024-06-10T01:25:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:12.257+0000 7fb4ffcd6640 1 -- 172.21.15.7:0/1655303010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4f8101af0 msgr2=0x7fb4f8196060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:12.257+0000 7fb4ffcd6640 1 --2- 172.21.15.7:0/1655303010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4f8101af0 0x7fb4f8196060 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8009230 tx=0x7fb4e80042f0 comp rx=0 tx=0).stop 2024-06-10T01:25:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:12.257+0000 7fb4ffcd6640 1 -- 172.21.15.7:0/1655303010 shutdown_connections 2024-06-10T01:25:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:12.257+0000 7fb4ffcd6640 1 --2- 172.21.15.7:0/1655303010 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb4e40a2b80 0x7fb4e40a5040 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:12.257+0000 7fb4ffcd6640 1 --2- 172.21.15.7:0/1655303010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f81024b0 0x7fb4f819d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:12.257+0000 7fb4ffcd6640 1 --2- 172.21.15.7:0/1655303010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4f8101af0 0x7fb4f8196060 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:12.257+0000 7fb4ffcd6640 1 --2- 172.21.15.7:0/1655303010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4f8100900 0x7fb4f8195b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:12.257+0000 7fb4ffcd6640 1 -- 172.21.15.7:0/1655303010 >> 172.21.15.7:0/1655303010 conn(0x7fb4f80fbe70 msgr2=0x7fb4f8104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:12.257+0000 7fb4ffcd6640 1 -- 172.21.15.7:0/1655303010 shutdown_connections 2024-06-10T01:25:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:12.257+0000 7fb4ffcd6640 1 -- 172.21.15.7:0/1655303010 wait complete. 2024-06-10T01:25:12.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 139410 23.26 MB/sec execute 229 sec latency 199.827 ms 2024-06-10T01:25:13.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:12 smithi007 ceph-mon[32398]: pgmap v1009: 129 pgs: 129 active+clean; 666 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 14 MiB/s wr, 681 op/s 2024-06-10T01:25:13.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:13 smithi078 ceph-mon[36683]: pgmap v1009: 129 pgs: 129 active+clean; 666 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 14 MiB/s wr, 681 op/s 2024-06-10T01:25:13.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:13 smithi175 ceph-mon[36636]: pgmap v1009: 129 pgs: 129 active+clean; 666 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 14 MiB/s wr, 681 op/s 2024-06-10T01:25:13.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 140010 23.24 MB/sec execute 230 sec latency 140.172 ms 2024-06-10T01:25:13.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.557+0000 7fdcea171700 1 -- 172.21.15.7:0/2320312659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdce4107400 msgr2=0x7fdce41077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.557+0000 7fdcea171700 1 --2- 172.21.15.7:0/2320312659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdce4107400 0x7fdce41077e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009990 tx=0x7fdcd4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.558+0000 7fdcea171700 1 -- 172.21.15.7:0/2320312659 shutdown_connections 2024-06-10T01:25:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.558+0000 7fdcea171700 1 --2- 172.21.15.7:0/2320312659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdce4101d50 0x7fdce410e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.558+0000 7fdcea171700 1 --2- 172.21.15.7:0/2320312659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdce4107400 0x7fdce41077e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.558+0000 7fdcea171700 1 --2- 172.21.15.7:0/2320312659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdce4101390 0x7fdce4101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.558+0000 7fdcea171700 1 -- 172.21.15.7:0/2320312659 >> 172.21.15.7:0/2320312659 conn(0x7fdce4078340 msgr2=0x7fdce4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.558+0000 7fdcea171700 1 -- 172.21.15.7:0/2320312659 shutdown_connections 2024-06-10T01:25:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.558+0000 7fdcea171700 1 -- 172.21.15.7:0/2320312659 wait complete. 2024-06-10T01:25:13.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.558+0000 7fdcea171700 1 Processor -- start 2024-06-10T01:25:13.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.559+0000 7fdcea171700 1 -- start start 2024-06-10T01:25:13.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.559+0000 7fdcea171700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdce4101390 0x7fdce419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:13.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.559+0000 7fdcea171700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdce4101d50 0x7fdce419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:13.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.559+0000 7fdcea171700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdce4107400 0x7fdce4198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:13.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.559+0000 7fdcea171700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdce4111670 con 0x7fdce4101390 2024-06-10T01:25:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.559+0000 7fdcea171700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41114f0 con 0x7fdce4101d50 2024-06-10T01:25:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.559+0000 7fdcea171700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41117f0 con 0x7fdce4107400 2024-06-10T01:25:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.559+0000 7fdce37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdce4101390 0x7fdce419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.559+0000 7fdce37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdce4101390 0x7fdce419e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59434/0 (socket says 172.21.15.7:59434) 2024-06-10T01:25:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.559+0000 7fdce2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdce4101d50 0x7fdce419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.559+0000 7fdce37fe700 1 -- 172.21.15.7:0/875045973 learned_addr learned my addr 172.21.15.7:0/875045973 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.559+0000 7fdce3fff700 1 --2- 172.21.15.7:0/875045973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdce4107400 0x7fdce4198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.559+0000 7fdce37fe700 1 -- 172.21.15.7:0/875045973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdce4107400 msgr2=0x7fdce4198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.559+0000 7fdce37fe700 1 --2- 172.21.15.7:0/875045973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdce4107400 0x7fdce4198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.559+0000 7fdce37fe700 1 -- 172.21.15.7:0/875045973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdce4101d50 msgr2=0x7fdce419edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.559+0000 7fdce37fe700 1 --2- 172.21.15.7:0/875045973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdce4101d50 0x7fdce419edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.559+0000 7fdce37fe700 1 -- 172.21.15.7:0/875045973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce41991b0 con 0x7fdce4101390 2024-06-10T01:25:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.559+0000 7fdce2ffd700 1 --2- 172.21.15.7:0/875045973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdce4101d50 0x7fdce419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:25:13.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.560+0000 7fdce37fe700 1 --2- 172.21.15.7:0/875045973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdce4101390 0x7fdce419e890 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fdcd400bae0 tx=0x7fdcd400bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:13.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.560+0000 7fdce0ff9700 1 -- 172.21.15.7:0/875045973 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4026070 con 0x7fdce4101390 2024-06-10T01:25:13.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.560+0000 7fdcea171700 1 -- 172.21.15.7:0/875045973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdce4199440 con 0x7fdce4101390 2024-06-10T01:25:13.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.560+0000 7fdce0ff9700 1 -- 172.21.15.7:0/875045973 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcd4003d70 con 0x7fdce4101390 2024-06-10T01:25:13.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.560+0000 7fdce0ff9700 1 -- 172.21.15.7:0/875045973 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4021410 con 0x7fdce4101390 2024-06-10T01:25:13.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.560+0000 7fdcea171700 1 -- 172.21.15.7:0/875045973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdce41a5760 con 0x7fdce4101390 2024-06-10T01:25:13.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.562+0000 7fdce0ff9700 1 -- 172.21.15.7:0/875045973 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fdcd4029070 con 0x7fdce4101390 2024-06-10T01:25:13.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.563+0000 7fdce0ff9700 1 --2- 172.21.15.7:0/875045973 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdccc0a2c20 0x7fdccc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:13.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.564+0000 7fdce2ffd700 1 --2- 172.21.15.7:0/875045973 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdccc0a2c20 0x7fdccc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:13.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.564+0000 7fdce0ff9700 1 -- 172.21.15.7:0/875045973 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fdcd40d00d0 con 0x7fdce4101390 2024-06-10T01:25:13.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.564+0000 7fdce2ffd700 1 --2- 172.21.15.7:0/875045973 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdccc0a2c20 0x7fdccc0a50e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009680 tx=0x7fdcd8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:13.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.565+0000 7fdcf49f4b80 1 Processor -- start 2024-06-10T01:25:13.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.565+0000 7fdcf49f4b80 1 -- start start 2024-06-10T01:25:13.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.565+0000 7fdcf49f4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f43f936c0 0x559f43f93aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:13.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.565+0000 7fdcf49f4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f43f93fe0 0x559f43f8da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:13.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.565+0000 7fdcf49f4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f43f8dfa0 0x559f43f8e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:13.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.565+0000 7fdcf49f4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559f43edf890 con 0x559f43f93fe0 2024-06-10T01:25:13.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.565+0000 7fdcf49f4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559f43e877a0 con 0x559f43f936c0 2024-06-10T01:25:13.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.565+0000 7fdcf49f4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559f43edb850 con 0x559f43f8dfa0 2024-06-10T01:25:13.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.565+0000 7fdce2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f43f93fe0 0x559f43f8da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:13.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.565+0000 7fdce2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f43f93fe0 0x559f43f8da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59442/0 (socket says 172.21.15.7:59442) 2024-06-10T01:25:13.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.565+0000 7fdce2ffd700 1 -- 172.21.15.7:0/3388403800 learned_addr learned my addr 172.21.15.7:0/3388403800 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:13.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.565+0000 7fdce37fe700 1 --2- 172.21.15.7:0/3388403800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f43f936c0 0x559f43f93aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:13.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.565+0000 7fdce3fff700 1 --2- 172.21.15.7:0/3388403800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f43f8dfa0 0x559f43f8e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:13.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.566+0000 7fdce37fe700 1 -- 172.21.15.7:0/3388403800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f43f8dfa0 msgr2=0x559f43f8e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:13.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.566+0000 7fdce37fe700 1 --2- 172.21.15.7:0/3388403800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f43f8dfa0 0x559f43f8e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.566+0000 7fdce37fe700 1 -- 172.21.15.7:0/3388403800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f43f93fe0 msgr2=0x559f43f8da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:13.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.566+0000 7fdce37fe700 1 --2- 172.21.15.7:0/3388403800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f43f93fe0 0x559f43f8da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.566+0000 7fdce37fe700 1 -- 172.21.15.7:0/3388403800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f43e7dfd0 con 0x559f43f936c0 2024-06-10T01:25:13.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.566+0000 7fdce37fe700 1 --2- 172.21.15.7:0/3388403800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f43f936c0 0x559f43f93aa0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40cd850 tx=0x7fdcd4005a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ff7012ac3011286 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:13.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.566+0000 7fdcc9ffb700 1 -- 172.21.15.7:0/3388403800 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd409c180 con 0x559f43f936c0 2024-06-10T01:25:13.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.566+0000 7fdcc9ffb700 1 -- 172.21.15.7:0/3388403800 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcd409cd40 con 0x559f43f936c0 2024-06-10T01:25:13.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.566+0000 7fdcc9ffb700 1 -- 172.21.15.7:0/3388403800 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd409bfa0 con 0x559f43f936c0 2024-06-10T01:25:13.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.567+0000 7fdcf49f4b80 1 -- 172.21.15.7:0/3388403800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f43f936c0 msgr2=0x559f43f93aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:13.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.567+0000 7fdcf49f4b80 1 --2- 172.21.15.7:0/3388403800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f43f936c0 0x559f43f93aa0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40cd850 tx=0x7fdcd4005a00 comp rx=0 tx=0).stop 2024-06-10T01:25:13.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.567+0000 7fdcf49f4b80 1 -- 172.21.15.7:0/3388403800 shutdown_connections 2024-06-10T01:25:13.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.567+0000 7fdcf49f4b80 1 --2- 172.21.15.7:0/3388403800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f43f8dfa0 0x559f43f8e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.567+0000 7fdcf49f4b80 1 --2- 172.21.15.7:0/3388403800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f43f936c0 0x559f43f93aa0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.567+0000 7fdcf49f4b80 1 --2- 172.21.15.7:0/3388403800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f43f93fe0 0x559f43f8da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.567+0000 7fdcf49f4b80 1 -- 172.21.15.7:0/3388403800 >> 172.21.15.7:0/3388403800 conn(0x559f43eb8d40 msgr2=0x559f43f888d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:13.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.567+0000 7fdcf49f4b80 1 -- 172.21.15.7:0/3388403800 shutdown_connections 2024-06-10T01:25:13.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.567+0000 7fdcf49f4b80 1 -- 172.21.15.7:0/3388403800 wait complete. 2024-06-10T01:25:13.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.568+0000 7fdcf49f4b80 1 -- start start 2024-06-10T01:25:13.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.568+0000 7fdcf49f4b80 1 Processor -- start 2024-06-10T01:25:13.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.568+0000 7fdcf49f4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f43f8dfa0 0x559f43f99e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:13.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.568+0000 7fdcf49f4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f43f936c0 0x559f43f97120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:13.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.568+0000 7fdcf49f4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f43f93fe0 0x559f43f97660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:13.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.568+0000 7fdcf49f4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559f43f9a770 con 0x559f43f8dfa0 2024-06-10T01:25:13.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.568+0000 7fdcf49f4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559f43e7fc70 con 0x559f43f936c0 2024-06-10T01:25:13.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.568+0000 7fdcf49f4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559f43f9a8f0 con 0x559f43f93fe0 2024-06-10T01:25:13.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.568+0000 7fdcf49f4b80 10 client.0 _ll_register_callbacks cb 0x559f43e799a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:25:13.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.568+0000 7fdce2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f43f936c0 0x559f43f97120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:13.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.568+0000 7fdce37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f43f8dfa0 0x559f43f99e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:13.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.568+0000 7fdce3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f43f93fe0 0x559f43f97660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:13.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.568+0000 7fdce2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f43f936c0 0x559f43f97120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:45070/0 (socket says 172.21.15.7:45070) 2024-06-10T01:25:13.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.568+0000 7fdce2ffd700 1 -- 172.21.15.7:0/2106750932 learned_addr learned my addr 172.21.15.7:0/2106750932 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:13.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.569+0000 7fdce37fe700 1 -- 172.21.15.7:0/2106750932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f43f93fe0 msgr2=0x559f43f97660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:13.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.569+0000 7fdce37fe700 1 --2- 172.21.15.7:0/2106750932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f43f93fe0 0x559f43f97660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.569+0000 7fdce37fe700 1 -- 172.21.15.7:0/2106750932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f43f936c0 msgr2=0x559f43f97120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:13.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.569+0000 7fdce37fe700 1 --2- 172.21.15.7:0/2106750932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f43f936c0 0x559f43f97120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.569+0000 7fdce37fe700 1 -- 172.21.15.7:0/2106750932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f43e2e950 con 0x559f43f8dfa0 2024-06-10T01:25:13.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.569+0000 7fdce2ffd700 1 --2- 172.21.15.7:0/2106750932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f43f936c0 0x559f43f97120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:25:13.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.569+0000 7fdce37fe700 1 --2- 172.21.15.7:0/2106750932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f43f8dfa0 0x559f43f99e20 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4023000 tx=0x7fdcd400b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d3921290e9649e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:13.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.569+0000 7fdcc2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:25:13.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.569+0000 7fdcc2ffd700 1 -- 172.21.15.7:0/2106750932 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd409bfa0 con 0x559f43f8dfa0 2024-06-10T01:25:13.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.569+0000 7fdcc2ffd700 1 -- 172.21.15.7:0/2106750932 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fdcd4085dd0 con 0x559f43f8dfa0 2024-06-10T01:25:13.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.569+0000 7fdcf49f4b80 1 -- 172.21.15.7:0/2106750932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcd409cd40 con 0x559f43f8dfa0 2024-06-10T01:25:13.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.569+0000 7fdcc2ffd700 1 -- 172.21.15.7:0/2106750932 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fdcd409c180 con 0x559f43f8dfa0 2024-06-10T01:25:13.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.569+0000 7fdcf49f4b80 10 client.20109 fetch_fsmap learned FSMap version 52 2024-06-10T01:25:13.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.569+0000 7fdcf49f4b80 1 -- 172.21.15.7:0/2106750932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x559f43f9a770 con 0x559f43f8dfa0 2024-06-10T01:25:13.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.569+0000 7fdcc2ffd700 1 -- 172.21.15.7:0/2106750932 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcd409cd40 con 0x559f43f8dfa0 2024-06-10T01:25:13.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.570+0000 7fdcc2ffd700 1 -- 172.21.15.7:0/2106750932 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7fdcd4061b90 con 0x559f43f8dfa0 2024-06-10T01:25:13.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.570+0000 7fdcf49f4b80 10 client.20109 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:25:13.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.570+0000 7fdcf49f4b80 10 client.20109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:25:13.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.570+0000 7fdcf49f4b80 20 client.20109 populate_metadata read hostname 'smithi007' 2024-06-10T01:25:13.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.570+0000 7fdcf49f4b80 1 --2- 172.21.15.7:0/2106750932 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559f43fa7420 0x559f43fa9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:13.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.570+0000 7fdcf49f4b80 4 client.20109 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:25:13.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.570+0000 7fdcf49f4b80 1 -- 172.21.15.7:0/2106750932 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f43eecd90 con 0x559f43fa7420 2024-06-10T01:25:13.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.570+0000 7fdce2ffd700 1 --2- 172.21.15.7:0/2106750932 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559f43fa7420 0x559f43fa9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:13.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.571+0000 7fdce2ffd700 1 --2- 172.21.15.7:0/2106750932 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559f43fa7420 0x559f43fa9830 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cd4a0e343cb5be6 server_cookie=5097c5c6f61559ca in_seq=0 out_seq=0 2024-06-10T01:25:13.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.571+0000 7fdcc2ffd700 10 client.20109 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:13.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.573+0000 7fdcc2ffd700 1 -- 172.21.15.7:0/2106750932 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559f43eecd90 con 0x559f43fa7420 2024-06-10T01:25:13.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.573+0000 7fdcc2ffd700 10 client.20109 handle_command_reply: tid=0 2024-06-10T01:25:13.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.573+0000 7fdcf49f4b80 1 client.20109 shutdown 2024-06-10T01:25:13.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.574+0000 7fdcf49f4b80 1 -- 172.21.15.7:0/2106750932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f43f8dfa0 msgr2=0x559f43f99e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:13.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.574+0000 7fdcf49f4b80 1 --2- 172.21.15.7:0/2106750932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f43f8dfa0 0x559f43f99e20 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4023000 tx=0x7fdcd400b490 comp rx=0 tx=0).stop 2024-06-10T01:25:13.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.574+0000 7fdcf49f4b80 1 -- 172.21.15.7:0/2106750932 shutdown_connections 2024-06-10T01:25:13.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.574+0000 7fdcf49f4b80 1 --2- 172.21.15.7:0/2106750932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f43f93fe0 0x559f43f97660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.574+0000 7fdcf49f4b80 1 --2- 172.21.15.7:0/2106750932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f43f8dfa0 0x559f43f99e20 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.574+0000 7fdcf49f4b80 1 --2- 172.21.15.7:0/2106750932 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559f43fa7420 0x559f43fa9830 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.574+0000 7fdcf49f4b80 1 --2- 172.21.15.7:0/2106750932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f43f936c0 0x559f43f97120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.574+0000 7fdcf49f4b80 1 -- 172.21.15.7:0/2106750932 >> 172.21.15.7:0/2106750932 conn(0x559f43eb8d40 msgr2=0x559f43f88670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:13.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.574+0000 7fdcc2ffd700 0 client.20109 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:13.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.574+0000 7fdcf49f4b80 1 -- 172.21.15.7:0/2106750932 shutdown_connections 2024-06-10T01:25:13.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.574+0000 7fdcf49f4b80 1 -- 172.21.15.7:0/2106750932 wait complete. 2024-06-10T01:25:13.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.574+0000 7fdcf49f4b80 20 client.20109 trim_cache size 0 max 16384 2024-06-10T01:25:13.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.599+0000 7fdcf49f4b80 1 Processor -- start 2024-06-10T01:25:13.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.599+0000 7fdcf49f4b80 1 -- start start 2024-06-10T01:25:13.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.600+0000 7fdcf49f4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f43f8dfa0 0x559f43fa3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:13.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.600+0000 7fdcf49f4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f43f936c0 0x559f43f8da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:13.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.600+0000 7fdcf49f4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f43f93fe0 0x559f43f94ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:13.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.600+0000 7fdcf49f4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41b0a70 con 0x559f43f93fe0 2024-06-10T01:25:13.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.600+0000 7fdcf49f4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41ae300 con 0x559f43f936c0 2024-06-10T01:25:13.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.600+0000 7fdcf49f4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41ac150 con 0x559f43f8dfa0 2024-06-10T01:25:13.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.600+0000 7fdce3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f43f93fe0 0x559f43f94ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:13.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.600+0000 7fdce2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f43f936c0 0x559f43f8da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:13.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.600+0000 7fdce2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f43f936c0 0x559f43f8da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:45080/0 (socket says 172.21.15.7:45080) 2024-06-10T01:25:13.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.600+0000 7fdce3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f43f93fe0 0x559f43f94ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59460/0 (socket says 172.21.15.7:59460) 2024-06-10T01:25:13.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.600+0000 7fdce2ffd700 1 -- 172.21.15.7:0/4211992247 learned_addr learned my addr 172.21.15.7:0/4211992247 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:13.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.600+0000 7fdce37fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f43f8dfa0 0x559f43fa3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:13.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.600+0000 7fdce2ffd700 1 -- 172.21.15.7:0/4211992247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f43f8dfa0 msgr2=0x559f43fa3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:13.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.600+0000 7fdce2ffd700 1 --2- 172.21.15.7:0/4211992247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f43f8dfa0 0x559f43fa3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.600+0000 7fdce2ffd700 1 -- 172.21.15.7:0/4211992247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f43f93fe0 msgr2=0x559f43f94ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:13.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.600+0000 7fdce2ffd700 1 --2- 172.21.15.7:0/4211992247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f43f93fe0 0x559f43f94ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.600+0000 7fdce2ffd700 1 -- 172.21.15.7:0/4211992247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f43fa9e10 con 0x559f43f936c0 2024-06-10T01:25:13.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.600+0000 7fdce3fff700 1 --2- 172.21.15.7:0/4211992247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f43f93fe0 0x559f43f94ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:25:13.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.600+0000 7fdce2ffd700 1 --2- 172.21.15.7:0/4211992247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f43f936c0 0x559f43f8da60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x559f43f986f0 tx=0x7fdcd8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1702ba388c151716 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:13.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.601+0000 7fdcc97fa700 1 -- 172.21.15.7:0/4211992247 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8036070 con 0x559f43f936c0 2024-06-10T01:25:13.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.601+0000 7fdcc97fa700 1 -- 172.21.15.7:0/4211992247 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcd800e9f0 con 0x559f43f936c0 2024-06-10T01:25:13.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.601+0000 7fdcc97fa700 1 -- 172.21.15.7:0/4211992247 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8031c30 con 0x559f43f936c0 2024-06-10T01:25:13.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.601+0000 7fdcf49f4b80 1 -- 172.21.15.7:0/4211992247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f43f936c0 msgr2=0x559f43f8da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:13.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.601+0000 7fdcf49f4b80 1 --2- 172.21.15.7:0/4211992247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f43f936c0 0x559f43f8da60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x559f43f986f0 tx=0x7fdcd8014000 comp rx=0 tx=0).stop 2024-06-10T01:25:13.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.601+0000 7fdcf49f4b80 1 -- 172.21.15.7:0/4211992247 shutdown_connections 2024-06-10T01:25:13.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.601+0000 7fdcf49f4b80 1 --2- 172.21.15.7:0/4211992247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f43f93fe0 0x559f43f94ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.601+0000 7fdcf49f4b80 1 --2- 172.21.15.7:0/4211992247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f43f8dfa0 0x559f43fa3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.601+0000 7fdcf49f4b80 1 --2- 172.21.15.7:0/4211992247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f43f936c0 0x559f43f8da60 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.601+0000 7fdcf49f4b80 1 -- 172.21.15.7:0/4211992247 >> 172.21.15.7:0/4211992247 conn(0x559f43eb8d40 msgr2=0x559f43f943c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:13.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.602+0000 7fdcf49f4b80 1 -- 172.21.15.7:0/4211992247 shutdown_connections 2024-06-10T01:25:13.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.602+0000 7fdcf49f4b80 1 -- 172.21.15.7:0/4211992247 wait complete. 2024-06-10T01:25:13.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.602+0000 7fdcf49f4b80 1 -- start start 2024-06-10T01:25:13.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.602+0000 7fdcf49f4b80 1 Processor -- start 2024-06-10T01:25:13.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.603+0000 7fdcf49f4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f43f8dfa0 0x559f43f9fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:13.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.603+0000 7fdce37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f43f8dfa0 0x559f43f9fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:13.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.603+0000 7fdce37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f43f8dfa0 0x559f43f9fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59472/0 (socket says 172.21.15.7:59472) 2024-06-10T01:25:13.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.603+0000 7fdcf49f4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f43f936c0 0x559f43fa00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:13.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.603+0000 7fdce37fe700 1 -- 172.21.15.7:0/4104677273 learned_addr learned my addr 172.21.15.7:0/4104677273 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:13.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.603+0000 7fdcf49f4b80 1 --2- 172.21.15.7:0/4104677273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f43f93fe0 0x559f4400c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:13.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.603+0000 7fdcf49f4b80 1 -- 172.21.15.7:0/4104677273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41ac7c0 con 0x559f43f8dfa0 2024-06-10T01:25:13.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.603+0000 7fdcf49f4b80 1 -- 172.21.15.7:0/4104677273 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41adc30 con 0x559f43f936c0 2024-06-10T01:25:13.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.603+0000 7fdcf49f4b80 1 -- 172.21.15.7:0/4104677273 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41ace90 con 0x559f43f93fe0 2024-06-10T01:25:13.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.603+0000 7fdce2ffd700 1 --2- 172.21.15.7:0/4104677273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f43f936c0 0x559f43fa00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:13.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.603+0000 7fdce3fff700 1 --2- 172.21.15.7:0/4104677273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f43f93fe0 0x559f4400c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:13.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.603+0000 7fdcf49f4b80 10 client.0 _ll_register_callbacks cb 0x559f43e799a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:25:13.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.603+0000 7fdce37fe700 1 -- 172.21.15.7:0/4104677273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f43f93fe0 msgr2=0x559f4400c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:13.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.603+0000 7fdce37fe700 1 --2- 172.21.15.7:0/4104677273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f43f93fe0 0x559f4400c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.603+0000 7fdce37fe700 1 -- 172.21.15.7:0/4104677273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f43f936c0 msgr2=0x559f43fa00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:13.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.603+0000 7fdce37fe700 1 --2- 172.21.15.7:0/4104677273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f43f936c0 0x559f43fa00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.603+0000 7fdce37fe700 1 -- 172.21.15.7:0/4104677273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f43fafba0 con 0x559f43f8dfa0 2024-06-10T01:25:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.603+0000 7fdce2ffd700 1 --2- 172.21.15.7:0/4104677273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f43f936c0 0x559f43fa00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:25:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.603+0000 7fdce3fff700 1 --2- 172.21.15.7:0/4104677273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f43f93fe0 0x559f4400c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:25:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.603+0000 7fdcf49f4b80 1 -- 172.21.15.7:0/4104677273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcd800e9f0 con 0x559f43f8dfa0 2024-06-10T01:25:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.604+0000 7fdce37fe700 1 --2- 172.21.15.7:0/4104677273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f43f8dfa0 0x559f43f9fb80 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40cdbc0 tx=0x7fdcd4085f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575cb3b630c3b677 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.604+0000 7fdcc2ffd700 10 client.20115 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:25:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.604+0000 7fdcc2ffd700 1 -- 172.21.15.7:0/4104677273 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd409bfa0 con 0x559f43f8dfa0 2024-06-10T01:25:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.604+0000 7fdcc2ffd700 1 -- 172.21.15.7:0/4104677273 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fdcd409ba90 con 0x559f43f8dfa0 2024-06-10T01:25:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.604+0000 7fdcc2ffd700 1 -- 172.21.15.7:0/4104677273 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fdcd40988e0 con 0x559f43f8dfa0 2024-06-10T01:25:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.604+0000 7fdcf49f4b80 10 client.20115 fetch_fsmap learned FSMap version 52 2024-06-10T01:25:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.604+0000 7fdcf49f4b80 1 -- 172.21.15.7:0/4104677273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=52}) -- 0x559f43fafba0 con 0x559f43f8dfa0 2024-06-10T01:25:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.604+0000 7fdcc2ffd700 1 -- 172.21.15.7:0/4104677273 <== mon.0 v2:172.21.15.7:3300/0 4 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7fdcd4003890 con 0x559f43f8dfa0 2024-06-10T01:25:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.604+0000 7fdcf49f4b80 10 client.20115 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:25:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.604+0000 7fdcf49f4b80 10 client.20115 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:25:13.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.604+0000 7fdcf49f4b80 20 client.20115 populate_metadata read hostname 'smithi007' 2024-06-10T01:25:13.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.604+0000 7fdcf49f4b80 1 --2- 172.21.15.7:0/4104677273 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559f4401b150 0x559f4401d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:13.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.605+0000 7fdcf49f4b80 4 client.20115 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:25:13.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.605+0000 7fdcf49f4b80 1 -- 172.21.15.7:0/4104677273 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559f43ee7310 con 0x559f4401b150 2024-06-10T01:25:13.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.605+0000 7fdce2ffd700 1 --2- 172.21.15.7:0/4104677273 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559f4401b150 0x559f4401d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:13.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.605+0000 7fdcc2ffd700 10 client.20115 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:13.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.605+0000 7fdce2ffd700 1 --2- 172.21.15.7:0/4104677273 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559f4401b150 0x559f4401d540 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=269036f371cf27be server_cookie=c91e2d8314068fa7 in_seq=0 out_seq=0 2024-06-10T01:25:13.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.611+0000 7fdcc2ffd700 1 -- 172.21.15.7:0/4104677273 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559f43ee7310 con 0x559f4401b150 2024-06-10T01:25:13.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.611+0000 7fdcc2ffd700 10 client.20115 handle_command_reply: tid=0 2024-06-10T01:25:13.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.611+0000 7fdcf49f4b80 1 client.20115 shutdown 2024-06-10T01:25:13.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.611+0000 7fdcf49f4b80 1 -- 172.21.15.7:0/4104677273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f43f8dfa0 msgr2=0x559f43f9fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:13.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.611+0000 7fdcf49f4b80 1 --2- 172.21.15.7:0/4104677273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f43f8dfa0 0x559f43f9fb80 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40cdbc0 tx=0x7fdcd4085f30 comp rx=0 tx=0).stop 2024-06-10T01:25:13.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.611+0000 7fdcf49f4b80 1 -- 172.21.15.7:0/4104677273 shutdown_connections 2024-06-10T01:25:13.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.611+0000 7fdcf49f4b80 1 --2- 172.21.15.7:0/4104677273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f43f93fe0 0x559f4400c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.611+0000 7fdcf49f4b80 1 --2- 172.21.15.7:0/4104677273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f43f8dfa0 0x559f43f9fb80 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.611+0000 7fdcf49f4b80 1 --2- 172.21.15.7:0/4104677273 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559f4401b150 0x559f4401d540 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.611+0000 7fdcf49f4b80 1 --2- 172.21.15.7:0/4104677273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f43f936c0 0x559f43fa00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.611+0000 7fdcf49f4b80 1 -- 172.21.15.7:0/4104677273 >> 172.21.15.7:0/4104677273 conn(0x559f43eb8d40 msgr2=0x559f43f94c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:13.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.611+0000 7fdcc2ffd700 0 client.20115 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:13.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.612+0000 7fdcf49f4b80 1 -- 172.21.15.7:0/4104677273 shutdown_connections 2024-06-10T01:25:13.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.612+0000 7fdcf49f4b80 1 -- 172.21.15.7:0/4104677273 wait complete. 2024-06-10T01:25:13.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.612+0000 7fdcf49f4b80 20 client.20115 trim_cache size 0 max 16384 2024-06-10T01:25:13.704 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:25:13.704 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-10T01:25:13.704 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "20f68d12-c662-4600-8ecb-964e0a6882c0", 2024-06-10T01:25:13.705 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-10T01:25:13.705 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:25:13.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.612+0000 7fdcea171700 1 -- 172.21.15.7:0/875045973 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdccc0a2c20 msgr2=0x7fdccc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:13.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.612+0000 7fdcea171700 1 --2- 172.21.15.7:0/875045973 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdccc0a2c20 0x7fdccc0a50e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009680 tx=0x7fdcd8009260 comp rx=0 tx=0).stop 2024-06-10T01:25:13.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.612+0000 7fdcea171700 1 -- 172.21.15.7:0/875045973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdce4101390 msgr2=0x7fdce419e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:13.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.612+0000 7fdcea171700 1 --2- 172.21.15.7:0/875045973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdce4101390 0x7fdce419e890 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fdcd400bae0 tx=0x7fdcd400bb10 comp rx=0 tx=0).stop 2024-06-10T01:25:13.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.612+0000 7fdcea171700 1 -- 172.21.15.7:0/875045973 shutdown_connections 2024-06-10T01:25:13.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.612+0000 7fdcea171700 1 --2- 172.21.15.7:0/875045973 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdccc0a2c20 0x7fdccc0a50e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.612+0000 7fdcea171700 1 --2- 172.21.15.7:0/875045973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdce4107400 0x7fdce4198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.612+0000 7fdcea171700 1 --2- 172.21.15.7:0/875045973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdce4101390 0x7fdce419e890 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.612+0000 7fdcea171700 1 --2- 172.21.15.7:0/875045973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdce4101d50 0x7fdce419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:13.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.612+0000 7fdcea171700 1 -- 172.21.15.7:0/875045973 >> 172.21.15.7:0/875045973 conn(0x7fdce4078340 msgr2=0x7fdce41006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:13.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.613+0000 7fdcea171700 1 -- 172.21.15.7:0/875045973 shutdown_connections 2024-06-10T01:25:13.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:13.613+0000 7fdcea171700 1 -- 172.21.15.7:0/875045973 wait complete. 2024-06-10T01:25:14.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:14 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:25:14.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:14 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:25:14.438 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:25:14.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:14 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:25:14.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 140936 23.36 MB/sec execute 231 sec latency 42.540 ms 2024-06-10T01:25:14.810 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:25:15.194 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:15 smithi007 ceph-mon[32398]: scrub queued for path: / 2024-06-10T01:25:15.194 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:15 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:25:15.194 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:15 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:25:15.194 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:15 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:25:15.194 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:15 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:25:15.194 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:15 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:25:15.195 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:15 smithi007 ceph-mon[32398]: pgmap v1010: 129 pgs: 129 active+clean; 665 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 14 MiB/s wr, 638 op/s 2024-06-10T01:25:15.195 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:15 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:25:15.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:15 smithi078 ceph-mon[36683]: scrub queued for path: / 2024-06-10T01:25:15.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:15 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:25:15.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:15 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:25:15.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:15 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:25:15.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:15 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:25:15.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:15 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:25:15.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:15 smithi078 ceph-mon[36683]: pgmap v1010: 129 pgs: 129 active+clean; 665 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 14 MiB/s wr, 638 op/s 2024-06-10T01:25:15.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:15 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:25:15.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:15 smithi175 ceph-mon[36636]: scrub queued for path: / 2024-06-10T01:25:15.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:15 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:25:15.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:15 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:25:15.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:15 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:25:15.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:15 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:25:15.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:15 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:25:15.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:15 smithi175 ceph-mon[36636]: pgmap v1010: 129 pgs: 129 active+clean; 665 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 14 MiB/s wr, 638 op/s 2024-06-10T01:25:15.452 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:15 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:25:15.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 141744 23.52 MB/sec execute 232 sec latency 48.027 ms 2024-06-10T01:25:16.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 142311 23.55 MB/sec execute 233 sec latency 91.748 ms 2024-06-10T01:25:16.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.969+0000 7f2e1de90700 1 -- 172.21.15.7:0/4097041529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e18107400 msgr2=0x7f2e181077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:16.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.969+0000 7f2e1de90700 1 --2- 172.21.15.7:0/4097041529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e18107400 0x7f2e181077e0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f2e08009990 tx=0x7f2e08009ca0 comp rx=0 tx=0).stop 2024-06-10T01:25:16.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.970+0000 7f2e1de90700 1 -- 172.21.15.7:0/4097041529 shutdown_connections 2024-06-10T01:25:16.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.970+0000 7f2e1de90700 1 --2- 172.21.15.7:0/4097041529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e18101d50 0x7f2e1810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:16.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.970+0000 7f2e1de90700 1 --2- 172.21.15.7:0/4097041529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e18107400 0x7f2e181077e0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:16.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.970+0000 7f2e1de90700 1 --2- 172.21.15.7:0/4097041529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e18101390 0x7f2e18101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:16.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.970+0000 7f2e1de90700 1 -- 172.21.15.7:0/4097041529 >> 172.21.15.7:0/4097041529 conn(0x7f2e18078340 msgr2=0x7f2e18078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:16.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.970+0000 7f2e1de90700 1 -- 172.21.15.7:0/4097041529 shutdown_connections 2024-06-10T01:25:16.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.970+0000 7f2e1de90700 1 -- 172.21.15.7:0/4097041529 wait complete. 2024-06-10T01:25:16.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.971+0000 7f2e1de90700 1 Processor -- start 2024-06-10T01:25:16.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.971+0000 7f2e1de90700 1 -- start start 2024-06-10T01:25:16.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.971+0000 7f2e1de90700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e18101390 0x7f2e181978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:16.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.971+0000 7f2e1de90700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e18101d50 0x7f2e18197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:16.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.971+0000 7f2e1de90700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e18107400 0x7f2e1819c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:16.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.971+0000 7f2e1de90700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2e181116f0 con 0x7f2e18101d50 2024-06-10T01:25:16.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.971+0000 7f2e1de90700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2e18111570 con 0x7f2e18107400 2024-06-10T01:25:16.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.971+0000 7f2e1de90700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2e18111870 con 0x7f2e18101390 2024-06-10T01:25:16.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.971+0000 7f2e16ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e18101d50 0x7f2e18197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:16.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.971+0000 7f2e177fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e18101390 0x7f2e181978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:16.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.971+0000 7f2e17fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e18107400 0x7f2e1819c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:16.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.971+0000 7f2e17fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e18107400 0x7f2e1819c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:45104/0 (socket says 172.21.15.7:45104) 2024-06-10T01:25:16.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.971+0000 7f2e17fff700 1 -- 172.21.15.7:0/3219534775 learned_addr learned my addr 172.21.15.7:0/3219534775 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:16.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.971+0000 7f2e177fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e18101390 0x7f2e181978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:53162/0 (socket says 172.21.15.7:53162) 2024-06-10T01:25:16.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.972+0000 7f2e17fff700 1 -- 172.21.15.7:0/3219534775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e18101390 msgr2=0x7f2e181978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:16.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.972+0000 7f2e17fff700 1 --2- 172.21.15.7:0/3219534775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e18101390 0x7f2e181978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:16.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.972+0000 7f2e17fff700 1 -- 172.21.15.7:0/3219534775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e18101d50 msgr2=0x7f2e18197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:16.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.972+0000 7f2e17fff700 1 --2- 172.21.15.7:0/3219534775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e18101d50 0x7f2e18197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:16.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.972+0000 7f2e17fff700 1 -- 172.21.15.7:0/3219534775 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e1819ce70 con 0x7f2e18107400 2024-06-10T01:25:16.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.972+0000 7f2e16ffd700 1 --2- 172.21.15.7:0/3219534775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e18101d50 0x7f2e18197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:25:16.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.972+0000 7f2e17fff700 1 --2- 172.21.15.7:0/3219534775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e18107400 0x7f2e1819c930 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000ef30 tx=0x7f2e1000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:16.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.972+0000 7f2e14ff9700 1 -- 172.21.15.7:0/3219534775 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e100092e0 con 0x7f2e18107400 2024-06-10T01:25:16.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.972+0000 7f2e14ff9700 1 -- 172.21.15.7:0/3219534775 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e10004bb0 con 0x7f2e18107400 2024-06-10T01:25:16.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.972+0000 7f2e14ff9700 1 -- 172.21.15.7:0/3219534775 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e10004280 con 0x7f2e18107400 2024-06-10T01:25:16.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.972+0000 7f2e1de90700 1 -- 172.21.15.7:0/3219534775 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e1819d100 con 0x7f2e18107400 2024-06-10T01:25:16.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.972+0000 7f2e1de90700 1 -- 172.21.15.7:0/3219534775 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e1819d690 con 0x7f2e18107400 2024-06-10T01:25:16.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.976+0000 7f2e14ff9700 1 -- 172.21.15.7:0/3219534775 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f2e1001b030 con 0x7f2e18107400 2024-06-10T01:25:16.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.976+0000 7f2e14ff9700 1 --2- 172.21.15.7:0/3219534775 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2e000a2ca0 0x7f2e000a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:16.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.977+0000 7f2e14ff9700 1 -- 172.21.15.7:0/3219534775 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f2e100ced90 con 0x7f2e18107400 2024-06-10T01:25:16.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.977+0000 7f2e177fe700 1 --2- 172.21.15.7:0/3219534775 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2e000a2ca0 0x7f2e000a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:16.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.978+0000 7f2e177fe700 1 --2- 172.21.15.7:0/3219534775 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2e000a2ca0 0x7f2e000a5160 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2e08005b00 tx=0x7f2e08005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:16.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.978+0000 7f2e28713b80 1 Processor -- start 2024-06-10T01:25:16.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.978+0000 7f2e28713b80 1 -- start start 2024-06-10T01:25:16.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.978+0000 7f2e28713b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b198ed6f60 0x55b198ed7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:16.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.978+0000 7f2e28713b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b198ed7880 0x55b198ed1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:16.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.978+0000 7f2e28713b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b198ed17a0 0x55b198ed1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:16.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.978+0000 7f2e28713b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b198dc3f60 con 0x55b198ed6f60 2024-06-10T01:25:16.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.978+0000 7f2e28713b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b198e22850 con 0x55b198ed7880 2024-06-10T01:25:16.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.978+0000 7f2e28713b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b198d8a460 con 0x55b198ed17a0 2024-06-10T01:25:16.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.979+0000 7f2e16ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b198ed7880 0x55b198ed1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:16.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.979+0000 7f2e177fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b198ed6f60 0x55b198ed7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:16.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.979+0000 7f2e16ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b198ed7880 0x55b198ed1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:45116/0 (socket says 172.21.15.7:45116) 2024-06-10T01:25:16.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.979+0000 7f2e16ffd700 1 -- 172.21.15.7:0/1169592579 learned_addr learned my addr 172.21.15.7:0/1169592579 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:16.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.979+0000 7f2e17fff700 1 --2- 172.21.15.7:0/1169592579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b198ed17a0 0x55b198ed1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:16.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.979+0000 7f2e16ffd700 1 -- 172.21.15.7:0/1169592579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b198ed17a0 msgr2=0x55b198ed1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:16.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.979+0000 7f2e16ffd700 1 --2- 172.21.15.7:0/1169592579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b198ed17a0 0x55b198ed1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:16.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.979+0000 7f2e16ffd700 1 -- 172.21.15.7:0/1169592579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b198ed6f60 msgr2=0x55b198ed7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:16.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.979+0000 7f2e16ffd700 1 --2- 172.21.15.7:0/1169592579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b198ed6f60 0x55b198ed7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:16.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.979+0000 7f2e16ffd700 1 -- 172.21.15.7:0/1169592579 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b198d718d0 con 0x55b198ed7880 2024-06-10T01:25:16.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.979+0000 7f2e16ffd700 1 --2- 172.21.15.7:0/1169592579 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b198ed7880 0x55b198ed1260 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0104d0 tx=0x7f2e0c010890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c9d9b796c1f8338 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:16.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.979+0000 7f2dfdffb700 1 -- 172.21.15.7:0/1169592579 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c01c680 con 0x55b198ed7880 2024-06-10T01:25:16.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.980+0000 7f2dfdffb700 1 -- 172.21.15.7:0/1169592579 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e0c01cd00 con 0x55b198ed7880 2024-06-10T01:25:16.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.980+0000 7f2dfdffb700 1 -- 172.21.15.7:0/1169592579 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c011df0 con 0x55b198ed7880 2024-06-10T01:25:16.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.981+0000 7f2e28713b80 1 -- 172.21.15.7:0/1169592579 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b198ed7880 msgr2=0x55b198ed1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:16.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.981+0000 7f2e28713b80 1 --2- 172.21.15.7:0/1169592579 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b198ed7880 0x55b198ed1260 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0104d0 tx=0x7f2e0c010890 comp rx=0 tx=0).stop 2024-06-10T01:25:16.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.981+0000 7f2e28713b80 1 -- 172.21.15.7:0/1169592579 shutdown_connections 2024-06-10T01:25:16.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.981+0000 7f2e28713b80 1 --2- 172.21.15.7:0/1169592579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b198ed17a0 0x55b198ed1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:16.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.981+0000 7f2e28713b80 1 --2- 172.21.15.7:0/1169592579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b198ed6f60 0x55b198ed7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:16.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.981+0000 7f2e28713b80 1 --2- 172.21.15.7:0/1169592579 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b198ed7880 0x55b198ed1260 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:16.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.981+0000 7f2e28713b80 1 -- 172.21.15.7:0/1169592579 >> 172.21.15.7:0/1169592579 conn(0x55b198dfe080 msgr2=0x55b198ecc220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:16.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.981+0000 7f2e28713b80 1 -- 172.21.15.7:0/1169592579 shutdown_connections 2024-06-10T01:25:16.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.981+0000 7f2e28713b80 1 -- 172.21.15.7:0/1169592579 wait complete. 2024-06-10T01:25:16.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.982+0000 7f2e28713b80 1 -- start start 2024-06-10T01:25:16.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.982+0000 7f2e28713b80 1 Processor -- start 2024-06-10T01:25:16.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.982+0000 7f2e28713b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b198ed17a0 0x55b198edd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:16.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.982+0000 7f2e28713b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b198ed6f60 0x55b198eda870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:16.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.982+0000 7f2e28713b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b198ed7880 0x55b198edadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:16.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.983+0000 7f2e28713b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b198dc2fc0 con 0x55b198ed17a0 2024-06-10T01:25:16.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.983+0000 7f2e28713b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b198e1e810 con 0x55b198ed7880 2024-06-10T01:25:16.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.983+0000 7f2e28713b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b198ede300 con 0x55b198ed6f60 2024-06-10T01:25:16.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.983+0000 7f2e28713b80 10 client.0 _ll_register_callbacks cb 0x55b198db85b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:25:16.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.983+0000 7f2e17fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b198ed7880 0x55b198edadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:16.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.983+0000 7f2e177fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b198ed17a0 0x55b198edd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:16.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.983+0000 7f2e17fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b198ed7880 0x55b198edadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:45124/0 (socket says 172.21.15.7:45124) 2024-06-10T01:25:16.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.983+0000 7f2e17fff700 1 -- 172.21.15.7:0/3399390392 learned_addr learned my addr 172.21.15.7:0/3399390392 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:16.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.983+0000 7f2e16ffd700 1 --2- 172.21.15.7:0/3399390392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b198ed6f60 0x55b198eda870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:16.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.983+0000 7f2e17fff700 1 -- 172.21.15.7:0/3399390392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b198ed6f60 msgr2=0x55b198eda870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:16.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.983+0000 7f2e17fff700 1 --2- 172.21.15.7:0/3399390392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b198ed6f60 0x55b198eda870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:16.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.983+0000 7f2e17fff700 1 -- 172.21.15.7:0/3399390392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b198ed17a0 msgr2=0x55b198edd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:16.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.983+0000 7f2e17fff700 1 --2- 172.21.15.7:0/3399390392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b198ed17a0 0x55b198edd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:16.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.983+0000 7f2e17fff700 1 -- 172.21.15.7:0/3399390392 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b198de8750 con 0x55b198ed7880 2024-06-10T01:25:16.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.983+0000 7f2e17fff700 1 --2- 172.21.15.7:0/3399390392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b198ed7880 0x55b198edadb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000ef00 tx=0x7f2e1009b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77efe29b6c4ed6b8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:16.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.983+0000 7f2df6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:25:16.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.984+0000 7f2df6ffd700 1 -- 172.21.15.7:0/3399390392 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1009b300 con 0x55b198ed7880 2024-06-10T01:25:16.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.984+0000 7f2df6ffd700 1 -- 172.21.15.7:0/3399390392 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f2e10004d60 con 0x55b198ed7880 2024-06-10T01:25:16.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.984+0000 7f2e28713b80 1 -- 172.21.15.7:0/3399390392 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e0c01cd00 con 0x55b198ed7880 2024-06-10T01:25:16.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.984+0000 7f2df6ffd700 1 -- 172.21.15.7:0/3399390392 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f2e1009f6b0 con 0x55b198ed7880 2024-06-10T01:25:16.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.984+0000 7f2e28713b80 10 client.29336 fetch_fsmap learned FSMap version 52 2024-06-10T01:25:16.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.984+0000 7f2e28713b80 1 -- 172.21.15.7:0/3399390392 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55b198ee8a70 con 0x55b198ed7880 2024-06-10T01:25:16.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.984+0000 7f2df6ffd700 1 -- 172.21.15.7:0/3399390392 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e100a3ee0 con 0x55b198ed7880 2024-06-10T01:25:16.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.984+0000 7f2df6ffd700 1 -- 172.21.15.7:0/3399390392 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f2e1009a750 con 0x55b198ed7880 2024-06-10T01:25:16.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.984+0000 7f2e28713b80 10 client.29336 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:25:16.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.984+0000 7f2e28713b80 10 client.29336 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:25:16.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.984+0000 7f2e28713b80 20 client.29336 populate_metadata read hostname 'smithi007' 2024-06-10T01:25:16.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.984+0000 7f2e28713b80 1 --2- 172.21.15.7:0/3399390392 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b198eeb0b0 0x55b198eed4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:16.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.984+0000 7f2e28713b80 4 client.29336 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:25:16.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.984+0000 7f2e28713b80 1 -- 172.21.15.7:0/3399390392 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2e0c01cd00 con 0x55b198eeb0b0 2024-06-10T01:25:16.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.985+0000 7f2e177fe700 1 --2- 172.21.15.7:0/3399390392 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b198eeb0b0 0x55b198eed4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:16.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.985+0000 7f2e177fe700 1 --2- 172.21.15.7:0/3399390392 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b198eeb0b0 0x55b198eed4a0 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b52d1f42762e6fc server_cookie=655b3326778b2cd7 in_seq=0 out_seq=0 2024-06-10T01:25:16.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.985+0000 7f2df6ffd700 10 client.29336 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:16.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.990+0000 7f2df6ffd700 1 -- 172.21.15.7:0/3399390392 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f2e0c01cd00 con 0x55b198eeb0b0 2024-06-10T01:25:16.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.990+0000 7f2df6ffd700 10 client.29336 handle_command_reply: tid=0 2024-06-10T01:25:16.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.990+0000 7f2e28713b80 1 client.29336 shutdown 2024-06-10T01:25:16.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.990+0000 7f2e28713b80 1 -- 172.21.15.7:0/3399390392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b198ed7880 msgr2=0x55b198edadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:16.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.990+0000 7f2e28713b80 1 --2- 172.21.15.7:0/3399390392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b198ed7880 0x55b198edadb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000ef00 tx=0x7f2e1009b840 comp rx=0 tx=0).stop 2024-06-10T01:25:16.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.990+0000 7f2e28713b80 1 -- 172.21.15.7:0/3399390392 shutdown_connections 2024-06-10T01:25:16.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.990+0000 7f2e28713b80 1 --2- 172.21.15.7:0/3399390392 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b198eeb0b0 0x55b198eed4a0 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:16.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.990+0000 7f2e28713b80 1 --2- 172.21.15.7:0/3399390392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b198ed7880 0x55b198edadb0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:16.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.990+0000 7f2e28713b80 1 --2- 172.21.15.7:0/3399390392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b198ed17a0 0x55b198edd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:16.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.990+0000 7f2df6ffd700 0 client.29336 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:16.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.990+0000 7f2e28713b80 1 --2- 172.21.15.7:0/3399390392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b198ed6f60 0x55b198eda870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:16.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.990+0000 7f2e28713b80 1 -- 172.21.15.7:0/3399390392 >> 172.21.15.7:0/3399390392 conn(0x55b198dfe080 msgr2=0x55b198ecb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:16.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.990+0000 7f2e28713b80 1 -- 172.21.15.7:0/3399390392 shutdown_connections 2024-06-10T01:25:16.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.991+0000 7f2e28713b80 1 -- 172.21.15.7:0/3399390392 wait complete. 2024-06-10T01:25:16.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:16.991+0000 7f2e28713b80 20 client.29336 trim_cache size 0 max 16384 2024-06-10T01:25:17.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.018+0000 7f2e28713b80 1 Processor -- start 2024-06-10T01:25:17.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.018+0000 7f2e28713b80 1 -- start start 2024-06-10T01:25:17.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.019+0000 7f2e28713b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b198ed17a0 0x55b198ecbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:17.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.019+0000 7f2e28713b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b198ed6f60 0x55b198ecc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:17.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.019+0000 7f2e28713b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b198ed7880 0x55b198ed7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:17.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.019+0000 7f2e28713b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2e181b7290 con 0x55b198ed7880 2024-06-10T01:25:17.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.019+0000 7f2e28713b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2e181b4b20 con 0x55b198ed17a0 2024-06-10T01:25:17.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.019+0000 7f2e28713b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2e181b2970 con 0x55b198ed6f60 2024-06-10T01:25:17.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.019+0000 7f2e17fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b198ed7880 0x55b198ed7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:17.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.019+0000 7f2e177fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b198ed17a0 0x55b198ecbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:17.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.019+0000 7f2e17fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b198ed7880 0x55b198ed7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59514/0 (socket says 172.21.15.7:59514) 2024-06-10T01:25:17.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.019+0000 7f2e17fff700 1 -- 172.21.15.7:0/696160835 learned_addr learned my addr 172.21.15.7:0/696160835 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:17.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.019+0000 7f2e16ffd700 1 --2- 172.21.15.7:0/696160835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b198ed6f60 0x55b198ecc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:17.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.019+0000 7f2e16ffd700 1 -- 172.21.15.7:0/696160835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b198ed17a0 msgr2=0x55b198ecbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:17.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.019+0000 7f2e16ffd700 1 --2- 172.21.15.7:0/696160835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b198ed17a0 0x55b198ecbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:17.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.019+0000 7f2e16ffd700 1 -- 172.21.15.7:0/696160835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b198ed7880 msgr2=0x55b198ed7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:17.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.019+0000 7f2e16ffd700 1 --2- 172.21.15.7:0/696160835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b198ed7880 0x55b198ed7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:17.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.019+0000 7f2e16ffd700 1 -- 172.21.15.7:0/696160835 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b198eda640 con 0x55b198ed6f60 2024-06-10T01:25:17.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.020+0000 7f2e16ffd700 1 --2- 172.21.15.7:0/696160835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b198ed6f60 0x55b198ecc3c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c011600 tx=0x7f2e0c00bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5875451b94823b1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:17.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.020+0000 7f2dfd7fa700 1 -- 172.21.15.7:0/696160835 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c0052b0 con 0x55b198ed6f60 2024-06-10T01:25:17.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.020+0000 7f2dfd7fa700 1 -- 172.21.15.7:0/696160835 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e0c005450 con 0x55b198ed6f60 2024-06-10T01:25:17.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.020+0000 7f2dfd7fa700 1 -- 172.21.15.7:0/696160835 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c0038e0 con 0x55b198ed6f60 2024-06-10T01:25:17.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.021+0000 7f2e28713b80 1 -- 172.21.15.7:0/696160835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b198ed6f60 msgr2=0x55b198ecc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:17.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.021+0000 7f2e28713b80 1 --2- 172.21.15.7:0/696160835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b198ed6f60 0x55b198ecc3c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c011600 tx=0x7f2e0c00bab0 comp rx=0 tx=0).stop 2024-06-10T01:25:17.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.021+0000 7f2e28713b80 1 -- 172.21.15.7:0/696160835 shutdown_connections 2024-06-10T01:25:17.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.021+0000 7f2e28713b80 1 --2- 172.21.15.7:0/696160835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b198ed7880 0x55b198ed7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:17.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.021+0000 7f2e28713b80 1 --2- 172.21.15.7:0/696160835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b198ed17a0 0x55b198ecbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:17.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.021+0000 7f2e28713b80 1 --2- 172.21.15.7:0/696160835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b198ed6f60 0x55b198ecc3c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:17.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.021+0000 7f2e28713b80 1 -- 172.21.15.7:0/696160835 >> 172.21.15.7:0/696160835 conn(0x55b198dfe080 msgr2=0x55b198eda830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:17.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.022+0000 7f2e28713b80 1 -- 172.21.15.7:0/696160835 shutdown_connections 2024-06-10T01:25:17.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.022+0000 7f2e28713b80 1 -- 172.21.15.7:0/696160835 wait complete. 2024-06-10T01:25:17.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.022+0000 7f2e28713b80 1 -- start start 2024-06-10T01:25:17.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.022+0000 7f2e28713b80 1 Processor -- start 2024-06-10T01:25:17.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.023+0000 7f2e28713b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b198ed17a0 0x55b198ee3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:17.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.023+0000 7f2e28713b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b198ed6f60 0x55b198ee3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:17.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.023+0000 7f2e28713b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b198ed7880 0x55b198ee0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:17.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.023+0000 7f2e28713b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2e181b2fe0 con 0x55b198ed7880 2024-06-10T01:25:17.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.023+0000 7f2e28713b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2e181b4450 con 0x55b198ed17a0 2024-06-10T01:25:17.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.023+0000 7f2e28713b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2e181b36b0 con 0x55b198ed6f60 2024-06-10T01:25:17.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.023+0000 7f2e28713b80 10 client.0 _ll_register_callbacks cb 0x55b198db85b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:25:17.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.023+0000 7f2e177fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b198ed17a0 0x55b198ee3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:17.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.023+0000 7f2e17fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b198ed7880 0x55b198ee0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:17.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.023+0000 7f2e177fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b198ed17a0 0x55b198ee3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:45146/0 (socket says 172.21.15.7:45146) 2024-06-10T01:25:17.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.023+0000 7f2e177fe700 1 -- 172.21.15.7:0/3737697685 learned_addr learned my addr 172.21.15.7:0/3737697685 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:17.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.023+0000 7f2e16ffd700 1 --2- 172.21.15.7:0/3737697685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b198ed6f60 0x55b198ee3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:17.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.023+0000 7f2e177fe700 1 -- 172.21.15.7:0/3737697685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b198ed6f60 msgr2=0x55b198ee3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:17.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.023+0000 7f2e177fe700 1 --2- 172.21.15.7:0/3737697685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b198ed6f60 0x55b198ee3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:17.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.023+0000 7f2e177fe700 1 -- 172.21.15.7:0/3737697685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b198ed7880 msgr2=0x55b198ee0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:17.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.023+0000 7f2e177fe700 1 --2- 172.21.15.7:0/3737697685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b198ed7880 0x55b198ee0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:17.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.023+0000 7f2e177fe700 1 -- 172.21.15.7:0/3737697685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b198ee0b50 con 0x55b198ed17a0 2024-06-10T01:25:17.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.024+0000 7f2e177fe700 1 --2- 172.21.15.7:0/3737697685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b198ed17a0 0x55b198ee3640 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2e0800c010 tx=0x7f2e0801f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60aeba8d5bfa00e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:17.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.024+0000 7f2df6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:25:17.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.024+0000 7f2df6ffd700 1 -- 172.21.15.7:0/3737697685 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e08003a90 con 0x55b198ed17a0 2024-06-10T01:25:17.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.024+0000 7f2df6ffd700 1 -- 172.21.15.7:0/3737697685 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f2e08003c30 con 0x55b198ed17a0 2024-06-10T01:25:17.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.024+0000 7f2e28713b80 1 -- 172.21.15.7:0/3737697685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e0c005450 con 0x55b198ed17a0 2024-06-10T01:25:17.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.024+0000 7f2df6ffd700 1 -- 172.21.15.7:0/3737697685 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f2e08042050 con 0x55b198ed17a0 2024-06-10T01:25:17.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.024+0000 7f2e28713b80 10 client.29342 fetch_fsmap learned FSMap version 52 2024-06-10T01:25:17.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.024+0000 7f2e28713b80 1 -- 172.21.15.7:0/3737697685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55b198f56550 con 0x55b198ed17a0 2024-06-10T01:25:17.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.024+0000 7f2df6ffd700 1 -- 172.21.15.7:0/3737697685 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e0c005450 con 0x55b198ed17a0 2024-06-10T01:25:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.024+0000 7f2df6ffd700 1 -- 172.21.15.7:0/3737697685 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f2e0803a430 con 0x55b198ed17a0 2024-06-10T01:25:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.024+0000 7f2e28713b80 10 client.29342 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:25:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.024+0000 7f2e28713b80 10 client.29342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:25:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.024+0000 7f2e28713b80 20 client.29342 populate_metadata read hostname 'smithi007' 2024-06-10T01:25:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.024+0000 7f2e28713b80 1 --2- 172.21.15.7:0/3737697685 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b198f56c60 0x55b198f59080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.025+0000 7f2e28713b80 4 client.29342 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:25:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.025+0000 7f2e28713b80 1 -- 172.21.15.7:0/3737697685 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b198e2fd50 con 0x55b198f56c60 2024-06-10T01:25:17.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.025+0000 7f2e16ffd700 1 --2- 172.21.15.7:0/3737697685 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b198f56c60 0x55b198f59080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:17.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.025+0000 7f2df6ffd700 10 client.29342 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:17.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.025+0000 7f2e16ffd700 1 --2- 172.21.15.7:0/3737697685 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b198f56c60 0x55b198f59080 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=515c27cf2d03c8e3 server_cookie=ebb1b3881041898 in_seq=0 out_seq=0 2024-06-10T01:25:17.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.027+0000 7f2df6ffd700 1 -- 172.21.15.7:0/3737697685 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b198e2fd50 con 0x55b198f56c60 2024-06-10T01:25:17.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.027+0000 7f2df6ffd700 10 client.29342 handle_command_reply: tid=0 2024-06-10T01:25:17.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.027+0000 7f2e28713b80 1 client.29342 shutdown 2024-06-10T01:25:17.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.027+0000 7f2e28713b80 1 -- 172.21.15.7:0/3737697685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b198ed17a0 msgr2=0x55b198ee3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:17.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.027+0000 7f2e28713b80 1 --2- 172.21.15.7:0/3737697685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b198ed17a0 0x55b198ee3640 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2e0800c010 tx=0x7f2e0801f000 comp rx=0 tx=0).stop 2024-06-10T01:25:17.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.027+0000 7f2e28713b80 1 -- 172.21.15.7:0/3737697685 shutdown_connections 2024-06-10T01:25:17.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.027+0000 7f2e28713b80 1 --2- 172.21.15.7:0/3737697685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b198ed7880 0x55b198ee0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:17.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.027+0000 7f2e28713b80 1 --2- 172.21.15.7:0/3737697685 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b198f56c60 0x55b198f59080 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:17.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.027+0000 7f2e28713b80 1 --2- 172.21.15.7:0/3737697685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b198ed17a0 0x55b198ee3640 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:17.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.027+0000 7f2e28713b80 1 --2- 172.21.15.7:0/3737697685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b198ed6f60 0x55b198ee3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:17.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.027+0000 7f2df6ffd700 0 client.29342 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:17.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.027+0000 7f2e28713b80 1 -- 172.21.15.7:0/3737697685 >> 172.21.15.7:0/3737697685 conn(0x55b198dfe080 msgr2=0x55b198ed7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:17.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.027+0000 7f2e28713b80 1 -- 172.21.15.7:0/3737697685 shutdown_connections 2024-06-10T01:25:17.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.027+0000 7f2e28713b80 1 -- 172.21.15.7:0/3737697685 wait complete. 2024-06-10T01:25:17.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.027+0000 7f2e28713b80 20 client.29342 trim_cache size 0 max 16384 2024-06-10T01:25:17.025 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:25:17.025 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T01:25:17.025 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-10T01:25:17.026 INFO:teuthology.orchestra.run.smithi007.stdout: "20f68d12-c662-4600-8ecb-964e0a6882c0": { 2024-06-10T01:25:17.026 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-10T01:25:17.026 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "20f68d12-c662-4600-8ecb-964e0a6882c0", 2024-06-10T01:25:17.026 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-10T01:25:17.026 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:25:17.026 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:25:17.026 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:25:17.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.028+0000 7f2e1de90700 1 -- 172.21.15.7:0/3219534775 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2e000a2ca0 msgr2=0x7f2e000a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:17.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.028+0000 7f2e1de90700 1 --2- 172.21.15.7:0/3219534775 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2e000a2ca0 0x7f2e000a5160 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2e08005b00 tx=0x7f2e08005a90 comp rx=0 tx=0).stop 2024-06-10T01:25:17.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.028+0000 7f2e1de90700 1 -- 172.21.15.7:0/3219534775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e18107400 msgr2=0x7f2e1819c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:17.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.028+0000 7f2e1de90700 1 --2- 172.21.15.7:0/3219534775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e18107400 0x7f2e1819c930 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000ef30 tx=0x7f2e1000c3a0 comp rx=0 tx=0).stop 2024-06-10T01:25:17.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.028+0000 7f2e1de90700 1 -- 172.21.15.7:0/3219534775 shutdown_connections 2024-06-10T01:25:17.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.029+0000 7f2e1de90700 1 --2- 172.21.15.7:0/3219534775 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2e000a2ca0 0x7f2e000a5160 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:17.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.029+0000 7f2e1de90700 1 --2- 172.21.15.7:0/3219534775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e18107400 0x7f2e1819c930 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:17.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.029+0000 7f2e1de90700 1 --2- 172.21.15.7:0/3219534775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e18101390 0x7f2e181978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:17.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.029+0000 7f2e1de90700 1 --2- 172.21.15.7:0/3219534775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e18101d50 0x7f2e18197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:17.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.029+0000 7f2e1de90700 1 -- 172.21.15.7:0/3219534775 >> 172.21.15.7:0/3219534775 conn(0x7f2e18078340 msgr2=0x7f2e180fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:17.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.029+0000 7f2e1de90700 1 -- 172.21.15.7:0/3219534775 shutdown_connections 2024-06-10T01:25:17.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:17.029+0000 7f2e1de90700 1 -- 172.21.15.7:0/3219534775 wait complete. 2024-06-10T01:25:17.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:17 smithi007 ceph-mon[32398]: pgmap v1011: 129 pgs: 129 active+clean; 626 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 15 MiB/s wr, 905 op/s 2024-06-10T01:25:17.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:17 smithi078 ceph-mon[36683]: pgmap v1011: 129 pgs: 129 active+clean; 626 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 15 MiB/s wr, 905 op/s 2024-06-10T01:25:17.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:17 smithi175 ceph-mon[36636]: pgmap v1011: 129 pgs: 129 active+clean; 626 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 15 MiB/s wr, 905 op/s 2024-06-10T01:25:17.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 142835 23.56 MB/sec execute 234 sec latency 150.347 ms 2024-06-10T01:25:17.947 INFO:tasks.cephfs.filesystem:scrub status for tag:20f68d12-c662-4600-8ecb-964e0a6882c0 - {'path': '/', 'tag': '20f68d12-c662-4600-8ecb-964e0a6882c0', 'options': 'recursive,force'} 2024-06-10T01:25:18.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 143525 23.60 MB/sec execute 235 sec latency 62.448 ms 2024-06-10T01:25:19.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:19 smithi007 ceph-mon[32398]: pgmap v1012: 129 pgs: 129 active+clean; 624 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.6 MiB/s wr, 741 op/s 2024-06-10T01:25:19.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:19 smithi078 ceph-mon[36683]: pgmap v1012: 129 pgs: 129 active+clean; 624 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.6 MiB/s wr, 741 op/s 2024-06-10T01:25:19.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:19 smithi175 ceph-mon[36636]: pgmap v1012: 129 pgs: 129 active+clean; 624 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.6 MiB/s wr, 741 op/s 2024-06-10T01:25:19.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 144291 23.65 MB/sec execute 236 sec latency 74.770 ms 2024-06-10T01:25:20.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 145084 23.74 MB/sec execute 237 sec latency 76.364 ms 2024-06-10T01:25:21.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:21 smithi007 ceph-mon[32398]: pgmap v1013: 129 pgs: 129 active+clean; 623 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.6 MiB/s wr, 808 op/s 2024-06-10T01:25:21.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:21 smithi078 ceph-mon[36683]: pgmap v1013: 129 pgs: 129 active+clean; 623 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.6 MiB/s wr, 808 op/s 2024-06-10T01:25:21.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:21 smithi175 ceph-mon[36636]: pgmap v1013: 129 pgs: 129 active+clean; 623 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.6 MiB/s wr, 808 op/s 2024-06-10T01:25:21.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 145786 23.84 MB/sec execute 238 sec latency 78.352 ms 2024-06-10T01:25:22.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 146459 23.87 MB/sec execute 239 sec latency 86.656 ms 2024-06-10T01:25:23.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:23 smithi007 ceph-mon[32398]: pgmap v1014: 129 pgs: 129 active+clean; 627 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 4.7 MiB/s wr, 630 op/s 2024-06-10T01:25:23.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:23 smithi078 ceph-mon[36683]: pgmap v1014: 129 pgs: 129 active+clean; 627 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 4.7 MiB/s wr, 630 op/s 2024-06-10T01:25:23.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:23 smithi175 ceph-mon[36636]: pgmap v1014: 129 pgs: 129 active+clean; 627 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 4.7 MiB/s wr, 630 op/s 2024-06-10T01:25:23.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 147043 23.89 MB/sec execute 240 sec latency 94.356 ms 2024-06-10T01:25:24.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 148079 24.04 MB/sec execute 241 sec latency 39.948 ms 2024-06-10T01:25:25.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:25 smithi007 ceph-mon[32398]: pgmap v1015: 129 pgs: 129 active+clean; 631 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 3.7 MiB/s wr, 596 op/s 2024-06-10T01:25:25.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:25 smithi078 ceph-mon[36683]: pgmap v1015: 129 pgs: 129 active+clean; 631 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 3.7 MiB/s wr, 596 op/s 2024-06-10T01:25:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.365+0000 7f5e4559a640 1 -- 172.21.15.7:0/2613171539 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e380319b0 con 0x7f5e481024c0 2024-06-10T01:25:25.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.365+0000 7f5e4cdc8640 1 -- 172.21.15.7:0/2613171539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e481024c0 msgr2=0x7f5e481048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:25.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.365+0000 7f5e4cdc8640 1 --2- 172.21.15.7:0/2613171539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e481024c0 0x7f5e481048b0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009900 tx=0x7f5e3802a410 comp rx=0 tx=0).stop 2024-06-10T01:25:25.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.365+0000 7f5e4cdc8640 1 -- 172.21.15.7:0/2613171539 shutdown_connections 2024-06-10T01:25:25.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.365+0000 7f5e4cdc8640 1 --2- 172.21.15.7:0/2613171539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e48107720 0x7f5e48109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:25.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.365+0000 7f5e4cdc8640 1 --2- 172.21.15.7:0/2613171539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e48104df0 0x7f5e481071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:25.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.365+0000 7f5e4cdc8640 1 --2- 172.21.15.7:0/2613171539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e481024c0 0x7f5e481048b0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:25.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.365+0000 7f5e4cdc8640 1 -- 172.21.15.7:0/2613171539 >> 172.21.15.7:0/2613171539 conn(0x7f5e480fbed0 msgr2=0x7f5e480fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:25.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.366+0000 7f5e4cdc8640 1 -- 172.21.15.7:0/2613171539 shutdown_connections 2024-06-10T01:25:25.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.366+0000 7f5e4cdc8640 1 -- 172.21.15.7:0/2613171539 wait complete. 2024-06-10T01:25:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.366+0000 7f5e4cdc8640 1 Processor -- start 2024-06-10T01:25:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.367+0000 7f5e4cdc8640 1 -- start start 2024-06-10T01:25:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.367+0000 7f5e4cdc8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e481024c0 0x7f5e48197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.367+0000 7f5e4cdc8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e48104df0 0x7f5e48198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.367+0000 7f5e4cdc8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e48107720 0x7f5e4819f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.367+0000 7f5e4cdc8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4810d710 con 0x7f5e48104df0 2024-06-10T01:25:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.367+0000 7f5e4cdc8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4810d590 con 0x7f5e481024c0 2024-06-10T01:25:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.367+0000 7f5e4cdc8640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4810d890 con 0x7f5e48107720 2024-06-10T01:25:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.367+0000 7f5e4659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e481024c0 0x7f5e48197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.367+0000 7f5e46d9d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e48107720 0x7f5e4819f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.367+0000 7f5e45d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e48104df0 0x7f5e48198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.367+0000 7f5e4659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e481024c0 0x7f5e48197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:47888/0 (socket says 172.21.15.7:47888) 2024-06-10T01:25:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.367+0000 7f5e4659c640 1 -- 172.21.15.7:0/2359035111 learned_addr learned my addr 172.21.15.7:0/2359035111 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.367+0000 7f5e46d9d640 1 -- 172.21.15.7:0/2359035111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e481024c0 msgr2=0x7f5e48197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.367+0000 7f5e46d9d640 1 --2- 172.21.15.7:0/2359035111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e481024c0 0x7f5e48197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.367+0000 7f5e46d9d640 1 -- 172.21.15.7:0/2359035111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e48104df0 msgr2=0x7f5e48198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.367+0000 7f5e46d9d640 1 --2- 172.21.15.7:0/2359035111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e48104df0 0x7f5e48198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.367+0000 7f5e46d9d640 1 -- 172.21.15.7:0/2359035111 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4819f970 con 0x7f5e48107720 2024-06-10T01:25:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.367+0000 7f5e4659c640 1 --2- 172.21.15.7:0/2359035111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e481024c0 0x7f5e48197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:25:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.368+0000 7f5e46d9d640 1 --2- 172.21.15.7:0/2359035111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e48107720 0x7f5e4819f300 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5e40009fd0 tx=0x7f5e4000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:25.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.368+0000 7f5e377fe640 1 -- 172.21.15.7:0/2359035111 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4000ce20 con 0x7f5e48107720 2024-06-10T01:25:25.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.368+0000 7f5e4cdc8640 1 -- 172.21.15.7:0/2359035111 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e48100310 con 0x7f5e48107720 2024-06-10T01:25:25.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.368+0000 7f5e377fe640 1 -- 172.21.15.7:0/2359035111 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e400137d0 con 0x7f5e48107720 2024-06-10T01:25:25.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.368+0000 7f5e377fe640 1 -- 172.21.15.7:0/2359035111 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e400055e0 con 0x7f5e48107720 2024-06-10T01:25:25.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.368+0000 7f5e4cdc8640 1 -- 172.21.15.7:0/2359035111 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e48100860 con 0x7f5e48107720 2024-06-10T01:25:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.371+0000 7f5e4fbb3740 1 Processor -- start 2024-06-10T01:25:25.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.371+0000 7f5e377fe640 1 -- 172.21.15.7:0/2359035111 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f5e40005800 con 0x7f5e48107720 2024-06-10T01:25:25.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.372+0000 7f5e4fbb3740 1 -- start start 2024-06-10T01:25:25.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.372+0000 7f5e4fbb3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617d8f38820 0x5617d8f38c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:25.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.372+0000 7f5e4fbb3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617d8f39210 0x5617d8f3dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:25.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.372+0000 7f5e4fbb3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d8f3e380 0x5617d8f40810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:25.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.372+0000 7f5e4fbb3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5617d8e934f0 con 0x5617d8f38820 2024-06-10T01:25:25.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.372+0000 7f5e4fbb3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5617d8e31870 con 0x5617d8f39210 2024-06-10T01:25:25.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.372+0000 7f5e4fbb3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5617d8e8f4b0 con 0x5617d8f3e380 2024-06-10T01:25:25.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.372+0000 7f5e4659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617d8f39210 0x5617d8f3dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:25.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.372+0000 7f5e4659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617d8f39210 0x5617d8f3dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:47896/0 (socket says 172.21.15.7:47896) 2024-06-10T01:25:25.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.372+0000 7f5e4659c640 1 -- 172.21.15.7:0/2219727370 learned_addr learned my addr 172.21.15.7:0/2219727370 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:25.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.372+0000 7f5e45d9b640 1 --2- 172.21.15.7:0/2219727370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617d8f38820 0x5617d8f38c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:25.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.372+0000 7f5e45d9b640 1 --2- 172.21.15.7:0/2219727370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d8f3e380 0x5617d8f40810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:25.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.373+0000 7f5e4659c640 1 -- 172.21.15.7:0/2219727370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d8f3e380 msgr2=0x5617d8f40810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.373+0000 7f5e4659c640 1 --2- 172.21.15.7:0/2219727370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d8f3e380 0x5617d8f40810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.373+0000 7f5e4659c640 1 -- 172.21.15.7:0/2219727370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617d8f38820 msgr2=0x5617d8f38c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.373+0000 7f5e4659c640 1 --2- 172.21.15.7:0/2219727370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617d8f38820 0x5617d8f38c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.373+0000 7f5e4659c640 1 -- 172.21.15.7:0/2219727370 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617d8e4d9b0 con 0x5617d8f39210 2024-06-10T01:25:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.373+0000 7f5e377fe640 1 --2- 172.21.15.7:0/2359035111 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5e300a2c00 0x7f5e300a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.373+0000 7f5e4659c640 1 --2- 172.21.15.7:0/2219727370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617d8f39210 0x5617d8f3dd10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5e3802a930 tx=0x7f5e380043f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec09b679b332afdf server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.373+0000 7f5e377fe640 1 -- 172.21.15.7:0/2359035111 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f5e400cf6f0 con 0x7f5e48107720 2024-06-10T01:25:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.373+0000 7f5e45d9b640 1 --2- 172.21.15.7:0/2359035111 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5e300a2c00 0x7f5e300a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.373+0000 7f5e34ff9640 1 -- 172.21.15.7:0/2219727370 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e38040070 con 0x5617d8f39210 2024-06-10T01:25:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.373+0000 7f5e34ff9640 1 -- 172.21.15.7:0/2219727370 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e380047a0 con 0x5617d8f39210 2024-06-10T01:25:25.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.373+0000 7f5e34ff9640 1 -- 172.21.15.7:0/2219727370 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e38035040 con 0x5617d8f39210 2024-06-10T01:25:25.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.374+0000 7f5e45d9b640 1 --2- 172.21.15.7:0/2359035111 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5e300a2c00 0x7f5e300a50c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x5617d8f39a70 tx=0x7f5e3c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:25.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.374+0000 7f5e4fbb3740 1 -- 172.21.15.7:0/2219727370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617d8f39210 msgr2=0x5617d8f3dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.374+0000 7f5e4fbb3740 1 --2- 172.21.15.7:0/2219727370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617d8f39210 0x5617d8f3dd10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5e3802a930 tx=0x7f5e380043f0 comp rx=0 tx=0).stop 2024-06-10T01:25:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.374+0000 7f5e4fbb3740 1 -- 172.21.15.7:0/2219727370 shutdown_connections 2024-06-10T01:25:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.374+0000 7f5e4fbb3740 1 --2- 172.21.15.7:0/2219727370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d8f3e380 0x5617d8f40810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.374+0000 7f5e4fbb3740 1 --2- 172.21.15.7:0/2219727370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617d8f39210 0x5617d8f3dd10 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.374+0000 7f5e4fbb3740 1 --2- 172.21.15.7:0/2219727370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617d8f38820 0x5617d8f38c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.374+0000 7f5e4fbb3740 1 -- 172.21.15.7:0/2219727370 >> 172.21.15.7:0/2219727370 conn(0x5617d8f2b680 msgr2=0x5617d8f2ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.374+0000 7f5e4fbb3740 1 -- 172.21.15.7:0/2219727370 shutdown_connections 2024-06-10T01:25:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.374+0000 7f5e4fbb3740 1 -- 172.21.15.7:0/2219727370 wait complete. 2024-06-10T01:25:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.375+0000 7f5e4fbb3740 1 -- start start 2024-06-10T01:25:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.375+0000 7f5e4fbb3740 1 Processor -- start 2024-06-10T01:25:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.375+0000 7f5e4fbb3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617d8f38820 0x5617d8f42ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.375+0000 7f5e4fbb3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d8f39210 0x5617d8f43000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.375+0000 7f5e4fbb3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617d8f3e380 0x5617d8f4a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.375+0000 7f5e4fbb3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5617d8f412d0 con 0x5617d8f38820 2024-06-10T01:25:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.375+0000 7f5e4fbb3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5617d8e1cae0 con 0x5617d8f3e380 2024-06-10T01:25:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.375+0000 7f5e4fbb3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5617d8f41450 con 0x5617d8f39210 2024-06-10T01:25:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.375+0000 7f5e4fbb3740 10 client.0 _ll_register_callbacks cb 0x5617d8e559b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:25:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.375+0000 7f5e4659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617d8f38820 0x5617d8f42ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.375+0000 7f5e4659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617d8f38820 0x5617d8f42ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55218/0 (socket says 172.21.15.7:55218) 2024-06-10T01:25:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.375+0000 7f5e4659c640 1 -- 172.21.15.7:0/2753772387 learned_addr learned my addr 172.21.15.7:0/2753772387 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.375+0000 7f5e46d9d640 1 --2- 172.21.15.7:0/2753772387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617d8f3e380 0x5617d8f4a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.375+0000 7f5e45d9b640 1 --2- 172.21.15.7:0/2753772387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d8f39210 0x5617d8f43000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.375+0000 7f5e4659c640 1 -- 172.21.15.7:0/2753772387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d8f39210 msgr2=0x5617d8f43000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.376+0000 7f5e4659c640 1 --2- 172.21.15.7:0/2753772387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d8f39210 0x5617d8f43000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.376+0000 7f5e4659c640 1 -- 172.21.15.7:0/2753772387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617d8f3e380 msgr2=0x5617d8f4a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.376+0000 7f5e4659c640 1 --2- 172.21.15.7:0/2753772387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617d8f3e380 0x5617d8f4a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.376+0000 7f5e4659c640 1 -- 172.21.15.7:0/2753772387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617d8ba7210 con 0x5617d8f38820 2024-06-10T01:25:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.376+0000 7f5e46d9d640 1 --2- 172.21.15.7:0/2753772387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617d8f3e380 0x5617d8f4a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:25:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.376+0000 7f5e4659c640 1 --2- 172.21.15.7:0/2753772387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617d8f38820 0x5617d8f42ac0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f5e38031440 tx=0x7f5e38004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e56b695c4a6b38 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.376+0000 7f5e2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:25:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.376+0000 7f5e2dffb640 1 -- 172.21.15.7:0/2753772387 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e38035070 con 0x5617d8f38820 2024-06-10T01:25:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.376+0000 7f5e2dffb640 1 -- 172.21.15.7:0/2753772387 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f5e380053b0 con 0x5617d8f38820 2024-06-10T01:25:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.376+0000 7f5e4fbb3740 1 -- 172.21.15.7:0/2753772387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e380047a0 con 0x5617d8f38820 2024-06-10T01:25:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.376+0000 7f5e2dffb640 1 -- 172.21.15.7:0/2753772387 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f5e38040040 con 0x5617d8f38820 2024-06-10T01:25:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.376+0000 7f5e4fbb3740 10 client.20145 fetch_fsmap learned FSMap version 52 2024-06-10T01:25:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.376+0000 7f5e4fbb3740 1 -- 172.21.15.7:0/2753772387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5617d8c73620 con 0x5617d8f38820 2024-06-10T01:25:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.377+0000 7f5e2dffb640 1 -- 172.21.15.7:0/2753772387 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e380047a0 con 0x5617d8f38820 2024-06-10T01:25:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.377+0000 7f5e2dffb640 1 -- 172.21.15.7:0/2753772387 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f5e38003850 con 0x5617d8f38820 2024-06-10T01:25:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.377+0000 7f5e4fbb3740 10 client.20145 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:25:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.377+0000 7f5e4fbb3740 10 client.20145 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:25:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.377+0000 7f5e4fbb3740 20 client.20145 populate_metadata read hostname 'smithi007' 2024-06-10T01:25:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.377+0000 7f5e4fbb3740 1 --2- 172.21.15.7:0/2753772387 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5617d8f4e530 0x5617d8f50920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.377+0000 7f5e4fbb3740 4 client.20145 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:25:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.377+0000 7f5e4fbb3740 1 -- 172.21.15.7:0/2753772387 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617d8ea6f10 con 0x5617d8f4e530 2024-06-10T01:25:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.377+0000 7f5e45d9b640 1 --2- 172.21.15.7:0/2753772387 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5617d8f4e530 0x5617d8f50920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.378+0000 7f5e45d9b640 1 --2- 172.21.15.7:0/2753772387 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5617d8f4e530 0x5617d8f50920 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fffedc546a6ec6a6 server_cookie=53c4d71e70fc26d7 in_seq=0 out_seq=0 2024-06-10T01:25:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.378+0000 7f5e2dffb640 10 client.20145 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:25.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.385+0000 7f5e2dffb640 1 -- 172.21.15.7:0/2753772387 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5617d8ea6f10 con 0x5617d8f4e530 2024-06-10T01:25:25.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.385+0000 7f5e2dffb640 10 client.20145 handle_command_reply: tid=0 2024-06-10T01:25:25.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.385+0000 7f5e4fbb3740 1 client.20145 shutdown 2024-06-10T01:25:25.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.386+0000 7f5e4fbb3740 1 -- 172.21.15.7:0/2753772387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617d8f38820 msgr2=0x5617d8f42ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:25.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.386+0000 7f5e4fbb3740 1 --2- 172.21.15.7:0/2753772387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617d8f38820 0x5617d8f42ac0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f5e38031440 tx=0x7f5e38004b80 comp rx=0 tx=0).stop 2024-06-10T01:25:25.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.386+0000 7f5e4fbb3740 1 -- 172.21.15.7:0/2753772387 shutdown_connections 2024-06-10T01:25:25.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.386+0000 7f5e4fbb3740 1 --2- 172.21.15.7:0/2753772387 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5617d8f4e530 0x5617d8f50920 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:25.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.386+0000 7f5e4fbb3740 1 --2- 172.21.15.7:0/2753772387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617d8f3e380 0x5617d8f4a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:25.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.386+0000 7f5e4fbb3740 1 --2- 172.21.15.7:0/2753772387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d8f39210 0x5617d8f43000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:25.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.386+0000 7f5e4fbb3740 1 --2- 172.21.15.7:0/2753772387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617d8f38820 0x5617d8f42ac0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:25.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.386+0000 7f5e2dffb640 0 client.20145 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:25.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.386+0000 7f5e4fbb3740 1 -- 172.21.15.7:0/2753772387 >> 172.21.15.7:0/2753772387 conn(0x5617d8f2b680 msgr2=0x5617d8f2a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:25.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.386+0000 7f5e4fbb3740 1 -- 172.21.15.7:0/2753772387 shutdown_connections 2024-06-10T01:25:25.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.386+0000 7f5e4fbb3740 1 -- 172.21.15.7:0/2753772387 wait complete. 2024-06-10T01:25:25.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.386+0000 7f5e4fbb3740 20 client.20145 trim_cache size 0 max 16384 2024-06-10T01:25:25.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.407+0000 7f5e4fbb3740 1 Processor -- start 2024-06-10T01:25:25.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.407+0000 7f5e4fbb3740 1 -- start start 2024-06-10T01:25:25.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.407+0000 7f5e4fbb3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617d8f38820 0x5617d8f269c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:25.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.407+0000 7f5e4fbb3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617d8f39210 0x5617d8f4a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:25.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.407+0000 7f5e4fbb3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d8f3e380 0x5617d8f420e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:25.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.407+0000 7f5e4fbb3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5e481aace0 con 0x5617d8f39210 2024-06-10T01:25:25.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.407+0000 7f5e4fbb3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5e481a85b0 con 0x5617d8f38820 2024-06-10T01:25:25.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.407+0000 7f5e4fbb3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5e481a6400 con 0x5617d8f3e380 2024-06-10T01:25:25.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.407+0000 7f5e4659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617d8f38820 0x5617d8f269c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:25.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.408+0000 7f5e4659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617d8f38820 0x5617d8f269c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:47928/0 (socket says 172.21.15.7:47928) 2024-06-10T01:25:25.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.408+0000 7f5e4659c640 1 -- 172.21.15.7:0/2999692987 learned_addr learned my addr 172.21.15.7:0/2999692987 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:25.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.408+0000 7f5e46d9d640 1 --2- 172.21.15.7:0/2999692987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d8f3e380 0x5617d8f420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:25.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.408+0000 7f5e45d9b640 1 --2- 172.21.15.7:0/2999692987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617d8f39210 0x5617d8f4a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.408+0000 7f5e4659c640 1 -- 172.21.15.7:0/2999692987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d8f3e380 msgr2=0x5617d8f420e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.408+0000 7f5e4659c640 1 --2- 172.21.15.7:0/2999692987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d8f3e380 0x5617d8f420e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.408+0000 7f5e4659c640 1 -- 172.21.15.7:0/2999692987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617d8f39210 msgr2=0x5617d8f4a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.408+0000 7f5e4659c640 1 --2- 172.21.15.7:0/2999692987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617d8f39210 0x5617d8f4a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.408+0000 7f5e4659c640 1 -- 172.21.15.7:0/2999692987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617d8f2b4b0 con 0x5617d8f38820 2024-06-10T01:25:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.408+0000 7f5e4659c640 1 --2- 172.21.15.7:0/2999692987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617d8f38820 0x5617d8f269c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5e3802a930 tx=0x7f5e38005700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaa169386b83100c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:25.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.409+0000 7f5e2ffff640 1 -- 172.21.15.7:0/2999692987 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e38035070 con 0x5617d8f38820 2024-06-10T01:25:25.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.409+0000 7f5e2ffff640 1 -- 172.21.15.7:0/2999692987 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e3805b020 con 0x5617d8f38820 2024-06-10T01:25:25.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.409+0000 7f5e2ffff640 1 -- 172.21.15.7:0/2999692987 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e380316b0 con 0x5617d8f38820 2024-06-10T01:25:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.409+0000 7f5e4fbb3740 1 -- 172.21.15.7:0/2999692987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617d8f38820 msgr2=0x5617d8f269c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.409+0000 7f5e4fbb3740 1 --2- 172.21.15.7:0/2999692987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617d8f38820 0x5617d8f269c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5e3802a930 tx=0x7f5e38005700 comp rx=0 tx=0).stop 2024-06-10T01:25:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.409+0000 7f5e4fbb3740 1 -- 172.21.15.7:0/2999692987 shutdown_connections 2024-06-10T01:25:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.409+0000 7f5e4fbb3740 1 --2- 172.21.15.7:0/2999692987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d8f3e380 0x5617d8f420e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.409+0000 7f5e4fbb3740 1 --2- 172.21.15.7:0/2999692987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617d8f39210 0x5617d8f4a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.409+0000 7f5e4fbb3740 1 --2- 172.21.15.7:0/2999692987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617d8f38820 0x5617d8f269c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.409+0000 7f5e4fbb3740 1 -- 172.21.15.7:0/2999692987 >> 172.21.15.7:0/2999692987 conn(0x5617d8f2b680 msgr2=0x5617d8f36f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.409+0000 7f5e4fbb3740 1 -- 172.21.15.7:0/2999692987 shutdown_connections 2024-06-10T01:25:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.409+0000 7f5e4fbb3740 1 -- 172.21.15.7:0/2999692987 wait complete. 2024-06-10T01:25:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.410+0000 7f5e4fbb3740 1 -- start start 2024-06-10T01:25:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.410+0000 7f5e4fbb3740 1 Processor -- start 2024-06-10T01:25:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.410+0000 7f5e4fbb3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617d8f38820 0x5617d8f49ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.410+0000 7f5e4fbb3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d8f39210 0x5617d8f46660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.410+0000 7f5e4fbb3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617d8f3e380 0x5617d8f46ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.410+0000 7f5e4fbb3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5e481a6a70 con 0x5617d8f3e380 2024-06-10T01:25:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.410+0000 7f5e4fbb3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5e481a7ee0 con 0x5617d8f38820 2024-06-10T01:25:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.410+0000 7f5e4fbb3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5e481a7140 con 0x5617d8f39210 2024-06-10T01:25:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.410+0000 7f5e4fbb3740 10 client.0 _ll_register_callbacks cb 0x7f5e3c009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:25:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.410+0000 7f5e46d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617d8f3e380 0x5617d8f46ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.410+0000 7f5e45d9b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d8f39210 0x5617d8f46660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.410+0000 7f5e4659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617d8f38820 0x5617d8f49ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.410+0000 7f5e4659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617d8f38820 0x5617d8f49ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:47934/0 (socket says 172.21.15.7:47934) 2024-06-10T01:25:25.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.410+0000 7f5e46d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617d8f3e380 0x5617d8f46ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55242/0 (socket says 172.21.15.7:55242) 2024-06-10T01:25:25.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.410+0000 7f5e4659c640 1 -- 172.21.15.7:0/1945395412 learned_addr learned my addr 172.21.15.7:0/1945395412 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:25.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.410+0000 7f5e45d9b640 1 --2- 172.21.15.7:0/1945395412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d8f39210 0x5617d8f46660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:37876/0 (socket says 172.21.15.7:37876) 2024-06-10T01:25:25.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.411+0000 7f5e46d9d640 1 -- 172.21.15.7:0/1945395412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d8f39210 msgr2=0x5617d8f46660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:25.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.411+0000 7f5e46d9d640 1 --2- 172.21.15.7:0/1945395412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d8f39210 0x5617d8f46660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:25.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.411+0000 7f5e46d9d640 1 -- 172.21.15.7:0/1945395412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617d8f38820 msgr2=0x5617d8f49ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:25.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.411+0000 7f5e46d9d640 1 --2- 172.21.15.7:0/1945395412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617d8f38820 0x5617d8f49ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:25.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.411+0000 7f5e46d9d640 1 -- 172.21.15.7:0/1945395412 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617d8f38650 con 0x5617d8f3e380 2024-06-10T01:25:25.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.411+0000 7f5e4659c640 1 --2- 172.21.15.7:0/1945395412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617d8f38820 0x5617d8f49ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:25:25.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.411+0000 7f5e46d9d640 1 --2- 172.21.15.7:0/1945395412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617d8f3e380 0x5617d8f46ba0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f5e400ccec0 tx=0x7f5e40013e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598fb1f96b9d8116 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:25.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.411+0000 7f5e2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:25:25.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.411+0000 7f5e2dffb640 1 -- 172.21.15.7:0/1945395412 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e400b9060 con 0x5617d8f3e380 2024-06-10T01:25:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.411+0000 7f5e2dffb640 1 -- 172.21.15.7:0/1945395412 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f5e40099ed0 con 0x5617d8f3e380 2024-06-10T01:25:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.411+0000 7f5e4fbb3740 1 -- 172.21.15.7:0/1945395412 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e3805b020 con 0x5617d8f3e380 2024-06-10T01:25:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.411+0000 7f5e2dffb640 1 -- 172.21.15.7:0/1945395412 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f5e400a4840 con 0x5617d8f3e380 2024-06-10T01:25:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.411+0000 7f5e4fbb3740 10 client.20151 fetch_fsmap learned FSMap version 52 2024-06-10T01:25:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.411+0000 7f5e4fbb3740 1 -- 172.21.15.7:0/1945395412 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5617d8f2c350 con 0x5617d8f3e380 2024-06-10T01:25:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.411+0000 7f5e2dffb640 1 -- 172.21.15.7:0/1945395412 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e4007ee50 con 0x5617d8f3e380 2024-06-10T01:25:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.412+0000 7f5e2dffb640 1 -- 172.21.15.7:0/1945395412 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f5e40089140 con 0x5617d8f3e380 2024-06-10T01:25:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.412+0000 7f5e4fbb3740 10 client.20151 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:25:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.412+0000 7f5e4fbb3740 10 client.20151 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:25:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.412+0000 7f5e4fbb3740 20 client.20151 populate_metadata read hostname 'smithi007' 2024-06-10T01:25:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.412+0000 7f5e4fbb3740 1 --2- 172.21.15.7:0/1945395412 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5617d8fb85e0 0x5617d8fbaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.412+0000 7f5e4fbb3740 4 client.20151 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:25:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.412+0000 7f5e4fbb3740 1 -- 172.21.15.7:0/1945395412 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5617d8ea0c20 con 0x5617d8fb85e0 2024-06-10T01:25:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.412+0000 7f5e4659c640 1 --2- 172.21.15.7:0/1945395412 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5617d8fb85e0 0x5617d8fbaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.413+0000 7f5e2dffb640 10 client.20151 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:25.413+0000 7f5e4659c640 1 --2- 172.21.15.7:0/1945395412 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5617d8fb85e0 0x5617d8fbaa00 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc7f2e7a118eed74 server_cookie=b9eb8669c3dfe48a in_seq=0 out_seq=0 2024-06-10T01:25:25.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:25 smithi175 ceph-mon[36636]: pgmap v1015: 129 pgs: 129 active+clean; 631 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 3.7 MiB/s wr, 596 op/s 2024-06-10T01:25:25.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 148788 24.10 MB/sec execute 242 sec latency 110.128 ms 2024-06-10T01:25:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:26.147+0000 7f5e2dffb640 1 -- 172.21.15.7:0/1945395412 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5617d8ea0c20 con 0x5617d8fb85e0 2024-06-10T01:25:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:26.147+0000 7f5e2dffb640 10 client.20151 handle_command_reply: tid=0 2024-06-10T01:25:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:26.147+0000 7f5e4fbb3740 1 client.20151 shutdown 2024-06-10T01:25:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:26.147+0000 7f5e4fbb3740 1 -- 172.21.15.7:0/1945395412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617d8f3e380 msgr2=0x5617d8f46ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:26.147+0000 7f5e4fbb3740 1 --2- 172.21.15.7:0/1945395412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617d8f3e380 0x5617d8f46ba0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f5e400ccec0 tx=0x7f5e40013e30 comp rx=0 tx=0).stop 2024-06-10T01:25:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:26.147+0000 7f5e4fbb3740 1 -- 172.21.15.7:0/1945395412 shutdown_connections 2024-06-10T01:25:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:26.147+0000 7f5e4fbb3740 1 --2- 172.21.15.7:0/1945395412 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5617d8fb85e0 0x5617d8fbaa00 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:26.148+0000 7f5e2dffb640 0 client.20151 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:26.148+0000 7f5e4fbb3740 1 --2- 172.21.15.7:0/1945395412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617d8f3e380 0x5617d8f46ba0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:26.148+0000 7f5e4fbb3740 1 --2- 172.21.15.7:0/1945395412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617d8f39210 0x5617d8f46660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:26.148+0000 7f5e4fbb3740 1 --2- 172.21.15.7:0/1945395412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617d8f38820 0x5617d8f49ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:26.148+0000 7f5e4fbb3740 1 -- 172.21.15.7:0/1945395412 >> 172.21.15.7:0/1945395412 conn(0x5617d8f2b680 msgr2=0x5617d8f29f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:26.148+0000 7f5e4fbb3740 1 -- 172.21.15.7:0/1945395412 shutdown_connections 2024-06-10T01:25:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:26.148+0000 7f5e4fbb3740 1 -- 172.21.15.7:0/1945395412 wait complete. 2024-06-10T01:25:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:26.148+0000 7f5e4fbb3740 20 client.20151 trim_cache size 0 max 16384 2024-06-10T01:25:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:25:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:25:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:25:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:26.149+0000 7f5e4cdc8640 1 -- 172.21.15.7:0/2359035111 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5e300a2c00 msgr2=0x7f5e300a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:26.149+0000 7f5e4cdc8640 1 --2- 172.21.15.7:0/2359035111 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5e300a2c00 0x7f5e300a50c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x5617d8f39a70 tx=0x7f5e3c005850 comp rx=0 tx=0).stop 2024-06-10T01:25:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:26.149+0000 7f5e4cdc8640 1 -- 172.21.15.7:0/2359035111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e48107720 msgr2=0x7f5e4819f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:26.149+0000 7f5e4cdc8640 1 --2- 172.21.15.7:0/2359035111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e48107720 0x7f5e4819f300 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5e40009fd0 tx=0x7f5e4000ef10 comp rx=0 tx=0).stop 2024-06-10T01:25:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:26.149+0000 7f5e4cdc8640 1 -- 172.21.15.7:0/2359035111 shutdown_connections 2024-06-10T01:25:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:26.149+0000 7f5e4cdc8640 1 --2- 172.21.15.7:0/2359035111 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5e300a2c00 0x7f5e300a50c0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:26.149+0000 7f5e4cdc8640 1 --2- 172.21.15.7:0/2359035111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e48107720 0x7f5e4819f300 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:26.149+0000 7f5e4cdc8640 1 --2- 172.21.15.7:0/2359035111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e48104df0 0x7f5e48198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:26.149+0000 7f5e4cdc8640 1 --2- 172.21.15.7:0/2359035111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e481024c0 0x7f5e48197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:26.149+0000 7f5e4cdc8640 1 -- 172.21.15.7:0/2359035111 >> 172.21.15.7:0/2359035111 conn(0x7f5e480fbed0 msgr2=0x7f5e48102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:26.149+0000 7f5e4cdc8640 1 -- 172.21.15.7:0/2359035111 shutdown_connections 2024-06-10T01:25:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:26.149+0000 7f5e4cdc8640 1 -- 172.21.15.7:0/2359035111 wait complete. 2024-06-10T01:25:26.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 149156 24.10 MB/sec execute 243 sec latency 626.437 ms 2024-06-10T01:25:27.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:27 smithi078 ceph-mon[36683]: pgmap v1016: 129 pgs: 129 active+clean; 637 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.1 MiB/s wr, 639 op/s 2024-06-10T01:25:27.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:27 smithi175 ceph-mon[36636]: pgmap v1016: 129 pgs: 129 active+clean; 637 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.1 MiB/s wr, 639 op/s 2024-06-10T01:25:27.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:27 smithi007 ceph-mon[32398]: pgmap v1016: 129 pgs: 129 active+clean; 637 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.1 MiB/s wr, 639 op/s 2024-06-10T01:25:27.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 149658 24.11 MB/sec execute 244 sec latency 115.820 ms 2024-06-10T01:25:28.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:28 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:25:28.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:28 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:25:28.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:28 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:25:28.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:28 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:25:28.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:28 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:25:28.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:28 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:25:28.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 150240 24.11 MB/sec execute 245 sec latency 104.564 ms 2024-06-10T01:25:29.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:29 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:25:29.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:29 smithi175 ceph-mon[36636]: pgmap v1017: 129 pgs: 129 active+clean; 644 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.7 MiB/s wr, 483 op/s 2024-06-10T01:25:29.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:29 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:25:29.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:29 smithi078 ceph-mon[36683]: pgmap v1017: 129 pgs: 129 active+clean; 644 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.7 MiB/s wr, 483 op/s 2024-06-10T01:25:29.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:29 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:25:29.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:29 smithi007 ceph-mon[32398]: pgmap v1017: 129 pgs: 129 active+clean; 644 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.7 MiB/s wr, 483 op/s 2024-06-10T01:25:29.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 150961 24.18 MB/sec execute 246 sec latency 109.855 ms 2024-06-10T01:25:30.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 151863 24.25 MB/sec execute 247 sec latency 40.330 ms 2024-06-10T01:25:31.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:31 smithi078 ceph-mon[36683]: pgmap v1018: 129 pgs: 129 active+clean; 644 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.3 MiB/s wr, 662 op/s 2024-06-10T01:25:31.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:31 smithi175 ceph-mon[36636]: pgmap v1018: 129 pgs: 129 active+clean; 644 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.3 MiB/s wr, 662 op/s 2024-06-10T01:25:31.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:31 smithi007 ceph-mon[32398]: pgmap v1018: 129 pgs: 129 active+clean; 644 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.3 MiB/s wr, 662 op/s 2024-06-10T01:25:31.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 152542 24.29 MB/sec execute 248 sec latency 91.463 ms 2024-06-10T01:25:32.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 153307 24.42 MB/sec execute 249 sec latency 64.510 ms 2024-06-10T01:25:33.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:33 smithi078 ceph-mon[36683]: pgmap v1019: 129 pgs: 129 active+clean; 650 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.8 MiB/s wr, 619 op/s 2024-06-10T01:25:33.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:33 smithi175 ceph-mon[36636]: pgmap v1019: 129 pgs: 129 active+clean; 650 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.8 MiB/s wr, 619 op/s 2024-06-10T01:25:33.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:33 smithi007 ceph-mon[32398]: pgmap v1019: 129 pgs: 129 active+clean; 650 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.8 MiB/s wr, 619 op/s 2024-06-10T01:25:33.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 153787 24.44 MB/sec execute 250 sec latency 69.525 ms 2024-06-10T01:25:34.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 154289 24.42 MB/sec execute 251 sec latency 72.447 ms 2024-06-10T01:25:35.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:35 smithi078 ceph-mon[36683]: pgmap v1020: 129 pgs: 129 active+clean; 651 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.4 MiB/s wr, 564 op/s 2024-06-10T01:25:35.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:35 smithi175 ceph-mon[36636]: pgmap v1020: 129 pgs: 129 active+clean; 651 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.4 MiB/s wr, 564 op/s 2024-06-10T01:25:35.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:35 smithi007 ceph-mon[32398]: pgmap v1020: 129 pgs: 129 active+clean; 651 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.4 MiB/s wr, 564 op/s 2024-06-10T01:25:35.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 154868 24.44 MB/sec execute 252 sec latency 107.652 ms 2024-06-10T01:25:36.538 INFO:tasks.workunit.client.0.smithi007.stdout: 10 155538 24.47 MB/sec execute 253 sec latency 94.726 ms 2024-06-10T01:25:37.539 INFO:tasks.workunit.client.0.smithi007.stdout: 10 156201 24.51 MB/sec execute 254 sec latency 111.836 ms 2024-06-10T01:25:37.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:37 smithi007 ceph-mon[32398]: pgmap v1021: 129 pgs: 129 active+clean; 660 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 4.2 MiB/s wr, 631 op/s 2024-06-10T01:25:37.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:37 smithi078 ceph-mon[36683]: pgmap v1021: 129 pgs: 129 active+clean; 660 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 4.2 MiB/s wr, 631 op/s 2024-06-10T01:25:37.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:37 smithi175 ceph-mon[36636]: pgmap v1021: 129 pgs: 129 active+clean; 660 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 4.2 MiB/s wr, 631 op/s 2024-06-10T01:25:38.539 INFO:tasks.workunit.client.0.smithi007.stdout: 10 156962 24.58 MB/sec execute 255 sec latency 57.575 ms 2024-06-10T01:25:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.310+0000 7f235dc38640 1 -- 172.21.15.7:0/3871887768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2358100f10 msgr2=0x7f2358108d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.310+0000 7f235dc38640 1 --2- 172.21.15.7:0/3871887768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2358100f10 0x7f2358108d80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2348009860 tx=0x7f234802a380 comp rx=0 tx=0).stop 2024-06-10T01:25:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.315+0000 7f235dc38640 1 -- 172.21.15.7:0/3871887768 shutdown_connections 2024-06-10T01:25:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.315+0000 7f235dc38640 1 --2- 172.21.15.7:0/3871887768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2358100f10 0x7f2358108d80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.315+0000 7f235dc38640 1 --2- 172.21.15.7:0/3871887768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2358100550 0x7f23581009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.315+0000 7f235dc38640 1 --2- 172.21.15.7:0/3871887768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2358102020 0x7f2358102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.315+0000 7f235dc38640 1 -- 172.21.15.7:0/3871887768 >> 172.21.15.7:0/3871887768 conn(0x7f23580fbeb0 msgr2=0x7f23580fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.321+0000 7f235dc38640 1 -- 172.21.15.7:0/3871887768 shutdown_connections 2024-06-10T01:25:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.324+0000 7f235dc38640 1 -- 172.21.15.7:0/3871887768 wait complete. 2024-06-10T01:25:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.324+0000 7f235dc38640 1 Processor -- start 2024-06-10T01:25:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.328+0000 7f235dc38640 1 -- start start 2024-06-10T01:25:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.328+0000 7f235dc38640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2358100550 0x7f2358108020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.328+0000 7f235dc38640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2358100f10 0x7f2358108560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.328+0000 7f235dc38640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2358102020 0x7f23581949e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.328+0000 7f235dc38640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f235806cf50 con 0x7f2358102020 2024-06-10T01:25:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.328+0000 7f235dc38640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f235806cdd0 con 0x7f2358100550 2024-06-10T01:25:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.328+0000 7f235dc38640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f235806d0d0 con 0x7f2358100f10 2024-06-10T01:25:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.329+0000 7f2357fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2358102020 0x7f23581949e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.329+0000 7f2357fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2358102020 0x7f23581949e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39264/0 (socket says 172.21.15.7:39264) 2024-06-10T01:25:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.329+0000 7f2357fff640 1 -- 172.21.15.7:0/1861632900 learned_addr learned my addr 172.21.15.7:0/1861632900 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.330+0000 7f23577fe640 1 --2- 172.21.15.7:0/1861632900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2358100550 0x7f2358108020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.331+0000 7f23577fe640 1 -- 172.21.15.7:0/1861632900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2358100f10 msgr2=0x7f2358108560 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-10T01:25:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.331+0000 7f23577fe640 1 --2- 172.21.15.7:0/1861632900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2358100f10 0x7f2358108560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.331+0000 7f23577fe640 1 -- 172.21.15.7:0/1861632900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2358102020 msgr2=0x7f23581949e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.331+0000 7f23577fe640 1 --2- 172.21.15.7:0/1861632900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2358102020 0x7f23581949e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.331+0000 7f23577fe640 1 -- 172.21.15.7:0/1861632900 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2358195080 con 0x7f2358100550 2024-06-10T01:25:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.332+0000 7f23577fe640 1 --2- 172.21.15.7:0/1861632900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2358100550 0x7f2358108020 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f234c009fd0 tx=0x7f234c00b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.333+0000 7f2354ff9640 1 -- 172.21.15.7:0/1861632900 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c015020 con 0x7f2358100550 2024-06-10T01:25:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.333+0000 7f2354ff9640 1 -- 172.21.15.7:0/1861632900 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f234c011d10 con 0x7f2358100550 2024-06-10T01:25:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.333+0000 7f2354ff9640 1 -- 172.21.15.7:0/1861632900 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c00f2e0 con 0x7f2358100550 2024-06-10T01:25:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.334+0000 7f235dc38640 1 -- 172.21.15.7:0/1861632900 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2358195370 con 0x7f2358100550 2024-06-10T01:25:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.334+0000 7f235dc38640 1 -- 172.21.15.7:0/1861632900 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23581958f0 con 0x7f2358100550 2024-06-10T01:25:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.338+0000 7f2360a25740 1 Processor -- start 2024-06-10T01:25:39.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.355+0000 7f2354ff9640 1 -- 172.21.15.7:0/1861632900 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f234c018020 con 0x7f2358100550 2024-06-10T01:25:39.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.356+0000 7f2354ff9640 1 --2- 172.21.15.7:0/1861632900 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f23400c4da0 0x7f23400c7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:39.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.356+0000 7f2357fff640 1 --2- 172.21.15.7:0/1861632900 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f23400c4da0 0x7f23400c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:39.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.357+0000 7f2354ff9640 1 -- 172.21.15.7:0/1861632900 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f234c0ceb40 con 0x7f2358100550 2024-06-10T01:25:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.362+0000 7f2357fff640 1 --2- 172.21.15.7:0/1861632900 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f23400c4da0 0x7f23400c7260 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f23480305f0 tx=0x7f2348009390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.362+0000 7f2360a25740 1 -- start start 2024-06-10T01:25:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.362+0000 7f2360a25740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fb2fb7010 0x564fb2fb7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.362+0000 7f2360a25740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fb2fb7a00 0x564fb2fbc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.362+0000 7f2360a25740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fb2fbcb70 0x564fb2fbf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.363+0000 7f2360a25740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564fb2f1a4f0 con 0x564fb2fb7a00 2024-06-10T01:25:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.363+0000 7f2360a25740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564fb2eb8870 con 0x564fb2fbcb70 2024-06-10T01:25:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.363+0000 7f2360a25740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564fb2f164b0 con 0x564fb2fb7010 2024-06-10T01:25:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.363+0000 7f2357fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fb2fbcb70 0x564fb2fbf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.363+0000 7f2357fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fb2fbcb70 0x564fb2fbf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:50764/0 (socket says 172.21.15.7:50764) 2024-06-10T01:25:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.363+0000 7f2357fff640 1 -- 172.21.15.7:0/1556376673 learned_addr learned my addr 172.21.15.7:0/1556376673 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.363+0000 7f23577fe640 1 --2- 172.21.15.7:0/1556376673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fb2fb7010 0x564fb2fb7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.363+0000 7f2357fff640 1 -- 172.21.15.7:0/1556376673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fb2fb7010 msgr2=0x564fb2fb7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.363+0000 7f2357fff640 1 --2- 172.21.15.7:0/1556376673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fb2fb7010 0x564fb2fb7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.363+0000 7f2357fff640 1 -- 172.21.15.7:0/1556376673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fb2fb7a00 msgr2=0x564fb2fbc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.363+0000 7f2357fff640 1 --2- 172.21.15.7:0/1556376673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fb2fb7a00 0x564fb2fbc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.363+0000 7f2357fff640 1 -- 172.21.15.7:0/1556376673 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fb2ed49b0 con 0x564fb2fbcb70 2024-06-10T01:25:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.373+0000 7f2357fff640 1 --2- 172.21.15.7:0/1556376673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fb2fbcb70 0x564fb2fbf000 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f23480095f0 tx=0x7f234803b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a659a86bf7333c2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.376+0000 7f233dffb640 1 -- 172.21.15.7:0/1556376673 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2348044040 con 0x564fb2fbcb70 2024-06-10T01:25:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.376+0000 7f233dffb640 1 -- 172.21.15.7:0/1556376673 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f234804d070 con 0x564fb2fbcb70 2024-06-10T01:25:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.376+0000 7f233dffb640 1 -- 172.21.15.7:0/1556376673 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2348037040 con 0x564fb2fbcb70 2024-06-10T01:25:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.377+0000 7f2360a25740 1 -- 172.21.15.7:0/1556376673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fb2fbcb70 msgr2=0x564fb2fbf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.377+0000 7f2360a25740 1 --2- 172.21.15.7:0/1556376673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fb2fbcb70 0x564fb2fbf000 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f23480095f0 tx=0x7f234803b000 comp rx=0 tx=0).stop 2024-06-10T01:25:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.381+0000 7f2360a25740 1 -- 172.21.15.7:0/1556376673 shutdown_connections 2024-06-10T01:25:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.381+0000 7f2360a25740 1 --2- 172.21.15.7:0/1556376673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fb2fbcb70 0x564fb2fbf000 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.381+0000 7f2360a25740 1 --2- 172.21.15.7:0/1556376673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fb2fb7a00 0x564fb2fbc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.381+0000 7f2360a25740 1 --2- 172.21.15.7:0/1556376673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fb2fb7010 0x564fb2fb7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.381+0000 7f2360a25740 1 -- 172.21.15.7:0/1556376673 >> 172.21.15.7:0/1556376673 conn(0x564fb2fb2680 msgr2=0x564fb2fb2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.388+0000 7f2360a25740 1 -- 172.21.15.7:0/1556376673 shutdown_connections 2024-06-10T01:25:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.391+0000 7f2360a25740 1 -- 172.21.15.7:0/1556376673 wait complete. 2024-06-10T01:25:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.391+0000 7f2360a25740 1 -- start start 2024-06-10T01:25:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.392+0000 7f2360a25740 1 Processor -- start 2024-06-10T01:25:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.392+0000 7f2360a25740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fb2fb7010 0x564fb2fc1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.392+0000 7f2360a25740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fb2fb7a00 0x564fb2fc17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.392+0000 7f2360a25740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fb2fbcb70 0x564fb2fc8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.392+0000 7f2360a25740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564fb2fbfac0 con 0x564fb2fb7a00 2024-06-10T01:25:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.392+0000 7f2360a25740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564fb2ea3ae0 con 0x564fb2fbcb70 2024-06-10T01:25:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.392+0000 7f2360a25740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564fb2fbfc40 con 0x564fb2fb7010 2024-06-10T01:25:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.392+0000 7f2360a25740 10 client.0 _ll_register_callbacks cb 0x564fb2edc9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:25:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.393+0000 7f2357fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fb2fbcb70 0x564fb2fc8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.394+0000 7f23577fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fb2fb7010 0x564fb2fc1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.394+0000 7f2357fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fb2fbcb70 0x564fb2fc8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:50766/0 (socket says 172.21.15.7:50766) 2024-06-10T01:25:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.394+0000 7f2357fff640 1 -- 172.21.15.7:0/1414227451 learned_addr learned my addr 172.21.15.7:0/1414227451 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.394+0000 7f23577fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fb2fb7010 0x564fb2fc1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52004/0 (socket says 172.21.15.7:52004) 2024-06-10T01:25:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.395+0000 7f2357fff640 1 -- 172.21.15.7:0/1414227451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fb2fb7010 msgr2=0x564fb2fc1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.395+0000 7f2357fff640 1 --2- 172.21.15.7:0/1414227451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fb2fb7010 0x564fb2fc1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.395+0000 7f2357fff640 1 -- 172.21.15.7:0/1414227451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fb2fb7a00 msgr2=0x564fb2fc17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.395+0000 7f2357fff640 1 --2- 172.21.15.7:0/1414227451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fb2fb7a00 0x564fb2fc17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.395+0000 7f2357fff640 1 -- 172.21.15.7:0/1414227451 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fb2c2e210 con 0x564fb2fbcb70 2024-06-10T01:25:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.396+0000 7f2357fff640 1 --2- 172.21.15.7:0/1414227451 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fb2fbcb70 0x564fb2fc8810 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f23480095c0 tx=0x7f2348005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c79413b976e8fd54 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.396+0000 7f2336ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:25:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.398+0000 7f2336ffd640 1 -- 172.21.15.7:0/1414227451 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2348044040 con 0x564fb2fbcb70 2024-06-10T01:25:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.398+0000 7f2336ffd640 1 -- 172.21.15.7:0/1414227451 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f23480035f0 con 0x564fb2fbcb70 2024-06-10T01:25:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.400+0000 7f2360a25740 1 -- 172.21.15.7:0/1414227451 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f234804d070 con 0x564fb2fbcb70 2024-06-10T01:25:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.407+0000 7f2336ffd640 1 -- 172.21.15.7:0/1414227451 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f2348037040 con 0x564fb2fbcb70 2024-06-10T01:25:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.409+0000 7f2360a25740 10 client.29369 fetch_fsmap learned FSMap version 52 2024-06-10T01:25:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.409+0000 7f2360a25740 1 -- 172.21.15.7:0/1414227451 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x564fb2cfa620 con 0x564fb2fbcb70 2024-06-10T01:25:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.416+0000 7f2336ffd640 1 -- 172.21.15.7:0/1414227451 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f234804d070 con 0x564fb2fbcb70 2024-06-10T01:25:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.416+0000 7f2336ffd640 1 -- 172.21.15.7:0/1414227451 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f2348051450 con 0x564fb2fbcb70 2024-06-10T01:25:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.416+0000 7f2360a25740 10 client.29369 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:25:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.416+0000 7f2360a25740 10 client.29369 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:25:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.416+0000 7f2360a25740 20 client.29369 populate_metadata read hostname 'smithi007' 2024-06-10T01:25:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.416+0000 7f2360a25740 1 --2- 172.21.15.7:0/1414227451 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564fb2fccd00 0x564fb2fcf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.416+0000 7f2360a25740 4 client.29369 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:25:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.416+0000 7f2360a25740 1 -- 172.21.15.7:0/1414227451 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fb2f2df10 con 0x564fb2fccd00 2024-06-10T01:25:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.420+0000 7f23577fe640 1 --2- 172.21.15.7:0/1414227451 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564fb2fccd00 0x564fb2fcf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.429+0000 7f23577fe640 1 --2- 172.21.15.7:0/1414227451 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564fb2fccd00 0x564fb2fcf0f0 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d86d3ddff640f13d server_cookie=5cce3bf95c4f686a in_seq=0 out_seq=0 2024-06-10T01:25:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.433+0000 7f2336ffd640 10 client.29369 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.435+0000 7f2336ffd640 1 -- 172.21.15.7:0/1414227451 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564fb2f2df10 con 0x564fb2fccd00 2024-06-10T01:25:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.436+0000 7f2336ffd640 10 client.29369 handle_command_reply: tid=0 2024-06-10T01:25:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.436+0000 7f2360a25740 1 client.29369 shutdown 2024-06-10T01:25:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.444+0000 7f2360a25740 1 -- 172.21.15.7:0/1414227451 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fb2fbcb70 msgr2=0x564fb2fc8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.444+0000 7f2360a25740 1 --2- 172.21.15.7:0/1414227451 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fb2fbcb70 0x564fb2fc8810 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f23480095c0 tx=0x7f2348005fb0 comp rx=0 tx=0).stop 2024-06-10T01:25:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.450+0000 7f2360a25740 1 -- 172.21.15.7:0/1414227451 shutdown_connections 2024-06-10T01:25:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.450+0000 7f2360a25740 1 --2- 172.21.15.7:0/1414227451 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564fb2fccd00 0x564fb2fcf0f0 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.450+0000 7f2360a25740 1 --2- 172.21.15.7:0/1414227451 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fb2fbcb70 0x564fb2fc8810 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.450+0000 7f2360a25740 1 --2- 172.21.15.7:0/1414227451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fb2fb7a00 0x564fb2fc17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.450+0000 7f2360a25740 1 --2- 172.21.15.7:0/1414227451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fb2fb7010 0x564fb2fc1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.450+0000 7f2360a25740 1 -- 172.21.15.7:0/1414227451 >> 172.21.15.7:0/1414227451 conn(0x564fb2fb2680 msgr2=0x564fb2fb17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.450+0000 7f2336ffd640 0 client.29369 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.456+0000 7f2360a25740 1 -- 172.21.15.7:0/1414227451 shutdown_connections 2024-06-10T01:25:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.460+0000 7f2360a25740 1 -- 172.21.15.7:0/1414227451 wait complete. 2024-06-10T01:25:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.461+0000 7f2360a25740 20 client.29369 trim_cache size 0 max 16384 2024-06-10T01:25:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.497+0000 7f2360a25740 1 Processor -- start 2024-06-10T01:25:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.497+0000 7f2360a25740 1 -- start start 2024-06-10T01:25:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.497+0000 7f2360a25740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fb2fb7010 0x564fb2fc1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.498+0000 7f2360a25740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fb2fb7a00 0x564fb2fc17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.498+0000 7f2360a25740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fb2fbcb70 0x564fb2fc87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.498+0000 7f2360a25740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f23581b3680 con 0x564fb2fb7a00 2024-06-10T01:25:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.498+0000 7f2360a25740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f23581b0f50 con 0x564fb2fb7010 2024-06-10T01:25:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.498+0000 7f2360a25740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f23581aeda0 con 0x564fb2fbcb70 2024-06-10T01:25:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.499+0000 7f2356ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fb2fb7a00 0x564fb2fc17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.499+0000 7f2356ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fb2fb7a00 0x564fb2fc17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39278/0 (socket says 172.21.15.7:39278) 2024-06-10T01:25:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.499+0000 7f2356ffd640 1 -- 172.21.15.7:0/2892271302 learned_addr learned my addr 172.21.15.7:0/2892271302 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.499+0000 7f2357fff640 1 --2- 172.21.15.7:0/2892271302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fb2fbcb70 0x564fb2fc87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.499+0000 7f23577fe640 1 --2- 172.21.15.7:0/2892271302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fb2fb7010 0x564fb2fc1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.501+0000 7f2357fff640 1 -- 172.21.15.7:0/2892271302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fb2fb7010 msgr2=0x564fb2fc1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.501+0000 7f2357fff640 1 --2- 172.21.15.7:0/2892271302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fb2fb7010 0x564fb2fc1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.501+0000 7f2357fff640 1 -- 172.21.15.7:0/2892271302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fb2fb7a00 msgr2=0x564fb2fc17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.501+0000 7f2357fff640 1 --2- 172.21.15.7:0/2892271302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fb2fb7a00 0x564fb2fc17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.501+0000 7f2357fff640 1 -- 172.21.15.7:0/2892271302 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fb2fb24b0 con 0x564fb2fbcb70 2024-06-10T01:25:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.503+0000 7f2357fff640 1 --2- 172.21.15.7:0/2892271302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fb2fbcb70 0x564fb2fc87d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f234803d040 tx=0x7f2348030340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9826540b98507cd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.514+0000 7f233d7fa640 1 -- 172.21.15.7:0/2892271302 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2348044040 con 0x564fb2fbcb70 2024-06-10T01:25:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.514+0000 7f233d7fa640 1 -- 172.21.15.7:0/2892271302 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f234804d040 con 0x564fb2fbcb70 2024-06-10T01:25:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.514+0000 7f233d7fa640 1 -- 172.21.15.7:0/2892271302 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234805b450 con 0x564fb2fbcb70 2024-06-10T01:25:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.515+0000 7f2360a25740 1 -- 172.21.15.7:0/2892271302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fb2fbcb70 msgr2=0x564fb2fc87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.515+0000 7f2360a25740 1 --2- 172.21.15.7:0/2892271302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fb2fbcb70 0x564fb2fc87d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f234803d040 tx=0x7f2348030340 comp rx=0 tx=0).stop 2024-06-10T01:25:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.519+0000 7f2360a25740 1 -- 172.21.15.7:0/2892271302 shutdown_connections 2024-06-10T01:25:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.519+0000 7f2360a25740 1 --2- 172.21.15.7:0/2892271302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fb2fbcb70 0x564fb2fc87d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.519+0000 7f2360a25740 1 --2- 172.21.15.7:0/2892271302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fb2fb7a00 0x564fb2fc17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.519+0000 7f2360a25740 1 --2- 172.21.15.7:0/2892271302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fb2fb7010 0x564fb2fc1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.519+0000 7f2360a25740 1 -- 172.21.15.7:0/2892271302 >> 172.21.15.7:0/2892271302 conn(0x564fb2fb2680 msgr2=0x564fb2fbc320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.522+0000 7f2360a25740 1 -- 172.21.15.7:0/2892271302 shutdown_connections 2024-06-10T01:25:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.522+0000 7f2360a25740 1 -- 172.21.15.7:0/2892271302 wait complete. 2024-06-10T01:25:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.522+0000 7f2360a25740 1 -- start start 2024-06-10T01:25:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.523+0000 7f2360a25740 1 Processor -- start 2024-06-10T01:25:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.523+0000 7f2360a25740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fb2fb7010 0x564fb2fc54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.523+0000 7f2360a25740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fb2fb7a00 0x564fb2fc5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.523+0000 7f2360a25740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fb2fbcb70 0x564fb30357f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.523+0000 7f2360a25740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f23581af410 con 0x564fb2fbcb70 2024-06-10T01:25:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.523+0000 7f2360a25740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f23581b0880 con 0x564fb2fb7a00 2024-06-10T01:25:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.523+0000 7f2360a25740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f23581afae0 con 0x564fb2fb7010 2024-06-10T01:25:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.523+0000 7f2360a25740 10 client.0 _ll_register_callbacks cb 0x7f234c009290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:25:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.525+0000 7f23577fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fb2fb7010 0x564fb2fc54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.529+0000 7f2357fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fb2fbcb70 0x564fb30357f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.530+0000 7f2357fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fb2fbcb70 0x564fb30357f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39290/0 (socket says 172.21.15.7:39290) 2024-06-10T01:25:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.530+0000 7f2357fff640 1 -- 172.21.15.7:0/2149066269 learned_addr learned my addr 172.21.15.7:0/2149066269 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.530+0000 7f2356ffd640 1 --2- 172.21.15.7:0/2149066269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fb2fb7a00 0x564fb2fc5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.532+0000 7f2357fff640 1 -- 172.21.15.7:0/2149066269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fb2fb7010 msgr2=0x564fb2fc54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.532+0000 7f2357fff640 1 --2- 172.21.15.7:0/2149066269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fb2fb7010 0x564fb2fc54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.532+0000 7f2357fff640 1 -- 172.21.15.7:0/2149066269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fb2fb7a00 msgr2=0x564fb2fc5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.532+0000 7f2357fff640 1 --2- 172.21.15.7:0/2149066269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fb2fb7a00 0x564fb2fc5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.532+0000 7f2357fff640 1 -- 172.21.15.7:0/2149066269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fb2fb65c0 con 0x564fb2fbcb70 2024-06-10T01:25:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.532+0000 7f2357fff640 1 --2- 172.21.15.7:0/2149066269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fb2fbcb70 0x564fb30357f0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f234802a970 tx=0x7f2348003cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e1c211bb9206b2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.532+0000 7f2336ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:25:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.536+0000 7f2336ffd640 1 -- 172.21.15.7:0/2149066269 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234806a060 con 0x564fb2fbcb70 2024-06-10T01:25:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.536+0000 7f2336ffd640 1 -- 172.21.15.7:0/2149066269 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f2348051b70 con 0x564fb2fbcb70 2024-06-10T01:25:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.539+0000 7f2360a25740 1 -- 172.21.15.7:0/2149066269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f234804d040 con 0x564fb2fbcb70 2024-06-10T01:25:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.541+0000 7f2336ffd640 1 -- 172.21.15.7:0/2149066269 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f2348044040 con 0x564fb2fbcb70 2024-06-10T01:25:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.543+0000 7f2360a25740 10 client.20163 fetch_fsmap learned FSMap version 52 2024-06-10T01:25:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.543+0000 7f2360a25740 1 -- 172.21.15.7:0/2149066269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x564fb3036b10 con 0x564fb2fbcb70 2024-06-10T01:25:39.540 INFO:tasks.workunit.client.0.smithi007.stdout: 10 157030 24.51 MB/sec execute 256 sec latency 1025.684 ms 2024-06-10T01:25:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.553+0000 7f2336ffd640 1 -- 172.21.15.7:0/2149066269 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f234804d040 con 0x564fb2fbcb70 2024-06-10T01:25:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.553+0000 7f2336ffd640 1 -- 172.21.15.7:0/2149066269 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f234805b520 con 0x564fb2fbcb70 2024-06-10T01:25:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.553+0000 7f2360a25740 10 client.20163 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:25:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.553+0000 7f2360a25740 10 client.20163 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:25:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.553+0000 7f2360a25740 20 client.20163 populate_metadata read hostname 'smithi007' 2024-06-10T01:25:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.553+0000 7f2360a25740 1 --2- 172.21.15.7:0/2149066269 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564fb3037250 0x564fb3039670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.553+0000 7f2360a25740 4 client.20163 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:25:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.553+0000 7f2360a25740 1 -- 172.21.15.7:0/2149066269 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564fb2f27c20 con 0x564fb3037250 2024-06-10T01:25:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.557+0000 7f23577fe640 1 --2- 172.21.15.7:0/2149066269 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564fb3037250 0x564fb3039670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.559+0000 7f23577fe640 1 --2- 172.21.15.7:0/2149066269 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564fb3037250 0x564fb3039670 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aced978f91621e7 server_cookie=32062b579df70467 in_seq=0 out_seq=0 2024-06-10T01:25:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:39.559+0000 7f2336ffd640 10 client.20163 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:39.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:39 smithi007 ceph-mon[32398]: pgmap v1022: 129 pgs: 129 active+clean; 666 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.1 MiB/s wr, 650 op/s 2024-06-10T01:25:39.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:39 smithi078 ceph-mon[36683]: pgmap v1022: 129 pgs: 129 active+clean; 666 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.1 MiB/s wr, 650 op/s 2024-06-10T01:25:39.955 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:39 smithi175 ceph-mon[36636]: pgmap v1022: 129 pgs: 129 active+clean; 666 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.1 MiB/s wr, 650 op/s 2024-06-10T01:25:40.540 INFO:tasks.workunit.client.0.smithi007.stdout: 10 157054 24.41 MB/sec execute 257 sec latency 2010.118 ms 2024-06-10T01:25:41.540 INFO:tasks.workunit.client.0.smithi007.stdout: 10 157235 24.34 MB/sec execute 258 sec latency 2149.657 ms 2024-06-10T01:25:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:41.596+0000 7f2336ffd640 1 -- 172.21.15.7:0/2149066269 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564fb2f27c20 con 0x564fb3037250 2024-06-10T01:25:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:41.596+0000 7f2336ffd640 10 client.20163 handle_command_reply: tid=0 2024-06-10T01:25:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:41.597+0000 7f2360a25740 1 client.20163 shutdown 2024-06-10T01:25:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:41.597+0000 7f2360a25740 1 -- 172.21.15.7:0/2149066269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fb2fbcb70 msgr2=0x564fb30357f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:41.597+0000 7f2360a25740 1 --2- 172.21.15.7:0/2149066269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fb2fbcb70 0x564fb30357f0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f234802a970 tx=0x7f2348003cc0 comp rx=0 tx=0).stop 2024-06-10T01:25:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:41.597+0000 7f2360a25740 1 -- 172.21.15.7:0/2149066269 shutdown_connections 2024-06-10T01:25:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:41.597+0000 7f2360a25740 1 --2- 172.21.15.7:0/2149066269 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564fb3037250 0x564fb3039670 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:41.597+0000 7f2360a25740 1 --2- 172.21.15.7:0/2149066269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564fb2fbcb70 0x564fb30357f0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:41.597+0000 7f2360a25740 1 --2- 172.21.15.7:0/2149066269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fb2fb7a00 0x564fb2fc5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:41.597+0000 7f2360a25740 1 --2- 172.21.15.7:0/2149066269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fb2fb7010 0x564fb2fc54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:41.597+0000 7f2360a25740 1 -- 172.21.15.7:0/2149066269 >> 172.21.15.7:0/2149066269 conn(0x564fb2fb2680 msgr2=0x564fb2fced10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:41.598+0000 7f2336ffd640 0 client.20163 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:41.598+0000 7f2360a25740 1 -- 172.21.15.7:0/2149066269 shutdown_connections 2024-06-10T01:25:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:41.598+0000 7f2360a25740 1 -- 172.21.15.7:0/2149066269 wait complete. 2024-06-10T01:25:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:41.598+0000 7f2360a25740 20 client.20163 trim_cache size 0 max 16384 2024-06-10T01:25:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:25:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:25:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:25:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:41.599+0000 7f235dc38640 1 -- 172.21.15.7:0/1861632900 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f23400c4da0 msgr2=0x7f23400c7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:41.599+0000 7f235dc38640 1 --2- 172.21.15.7:0/1861632900 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f23400c4da0 0x7f23400c7260 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f23480305f0 tx=0x7f2348009390 comp rx=0 tx=0).stop 2024-06-10T01:25:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:41.599+0000 7f235dc38640 1 -- 172.21.15.7:0/1861632900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2358100550 msgr2=0x7f2358108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:41.599+0000 7f235dc38640 1 --2- 172.21.15.7:0/1861632900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2358100550 0x7f2358108020 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f234c009fd0 tx=0x7f234c00b870 comp rx=0 tx=0).stop 2024-06-10T01:25:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:41.599+0000 7f235dc38640 1 -- 172.21.15.7:0/1861632900 shutdown_connections 2024-06-10T01:25:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:41.599+0000 7f235dc38640 1 --2- 172.21.15.7:0/1861632900 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f23400c4da0 0x7f23400c7260 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:41.599+0000 7f235dc38640 1 --2- 172.21.15.7:0/1861632900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2358102020 0x7f23581949e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:41.599+0000 7f235dc38640 1 --2- 172.21.15.7:0/1861632900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2358100f10 0x7f2358108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:41.599+0000 7f235dc38640 1 --2- 172.21.15.7:0/1861632900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2358100550 0x7f2358108020 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:41.599+0000 7f235dc38640 1 -- 172.21.15.7:0/1861632900 >> 172.21.15.7:0/1861632900 conn(0x7f23580fbeb0 msgr2=0x7f235806fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:41.599+0000 7f235dc38640 1 -- 172.21.15.7:0/1861632900 shutdown_connections 2024-06-10T01:25:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:41.599+0000 7f235dc38640 1 -- 172.21.15.7:0/1861632900 wait complete. 2024-06-10T01:25:41.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:41 smithi007 ceph-mon[32398]: pgmap v1023: 129 pgs: 129 active+clean; 716 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 8.2 MiB/s wr, 670 op/s 2024-06-10T01:25:41.817 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:41 smithi175 ceph-mon[36636]: pgmap v1023: 129 pgs: 129 active+clean; 716 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 8.2 MiB/s wr, 670 op/s 2024-06-10T01:25:41.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:41 smithi078 ceph-mon[36683]: pgmap v1023: 129 pgs: 129 active+clean; 716 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 8.2 MiB/s wr, 670 op/s 2024-06-10T01:25:42.540 INFO:tasks.workunit.client.0.smithi007.stdout: 10 157522 24.32 MB/sec execute 259 sec latency 251.570 ms 2024-06-10T01:25:43.540 INFO:tasks.workunit.client.0.smithi007.stdout: 10 157823 24.30 MB/sec execute 260 sec latency 119.093 ms 2024-06-10T01:25:43.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:43 smithi007 ceph-mon[32398]: pgmap v1024: 129 pgs: 129 active+clean; 744 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 10 MiB/s wr, 569 op/s 2024-06-10T01:25:43.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:43 smithi078 ceph-mon[36683]: pgmap v1024: 129 pgs: 129 active+clean; 744 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 10 MiB/s wr, 569 op/s 2024-06-10T01:25:43.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:43 smithi175 ceph-mon[36636]: pgmap v1024: 129 pgs: 129 active+clean; 744 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 10 MiB/s wr, 569 op/s 2024-06-10T01:25:44.540 INFO:tasks.workunit.client.0.smithi007.stdout: 10 157946 24.24 MB/sec execute 261 sec latency 855.790 ms 2024-06-10T01:25:45.540 INFO:tasks.workunit.client.0.smithi007.stdout: 10 158213 24.20 MB/sec execute 262 sec latency 902.449 ms 2024-06-10T01:25:45.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:45 smithi007 ceph-mon[32398]: pgmap v1025: 129 pgs: 129 active+clean; 751 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 10 MiB/s wr, 533 op/s 2024-06-10T01:25:45.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:45 smithi078 ceph-mon[36683]: pgmap v1025: 129 pgs: 129 active+clean; 751 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 10 MiB/s wr, 533 op/s 2024-06-10T01:25:45.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:45 smithi175 ceph-mon[36636]: pgmap v1025: 129 pgs: 129 active+clean; 751 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 10 MiB/s wr, 533 op/s 2024-06-10T01:25:46.540 INFO:tasks.workunit.client.0.smithi007.stdout: 10 158504 24.17 MB/sec execute 263 sec latency 155.171 ms 2024-06-10T01:25:47.540 INFO:tasks.workunit.client.0.smithi007.stdout: 10 158866 24.16 MB/sec execute 264 sec latency 118.314 ms 2024-06-10T01:25:47.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:47 smithi078 ceph-mon[36683]: pgmap v1026: 129 pgs: 129 active+clean; 745 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 16 MiB/s wr, 873 op/s 2024-06-10T01:25:47.949 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:25:47.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:47 smithi175 ceph-mon[36636]: pgmap v1026: 129 pgs: 129 active+clean; 745 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 16 MiB/s wr, 873 op/s 2024-06-10T01:25:47.981 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:47 smithi007 ceph-mon[32398]: pgmap v1026: 129 pgs: 129 active+clean; 745 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 16 MiB/s wr, 873 op/s 2024-06-10T01:25:48.265 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:25:48.540 INFO:tasks.workunit.client.0.smithi007.stdout: 10 159422 24.18 MB/sec execute 265 sec latency 91.759 ms 2024-06-10T01:25:49.540 INFO:tasks.workunit.client.0.smithi007.stdout: 10 160043 24.19 MB/sec execute 266 sec latency 103.993 ms 2024-06-10T01:25:49.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:49 smithi078 ceph-mon[36683]: pgmap v1027: 129 pgs: 129 active+clean; 719 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 16 MiB/s wr, 862 op/s 2024-06-10T01:25:49.857 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:49 smithi007 ceph-mon[32398]: pgmap v1027: 129 pgs: 129 active+clean; 719 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 16 MiB/s wr, 862 op/s 2024-06-10T01:25:49.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:49 smithi175 ceph-mon[36636]: pgmap v1027: 129 pgs: 129 active+clean; 719 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 16 MiB/s wr, 862 op/s 2024-06-10T01:25:50.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.525+0000 7f99a0ca2700 1 -- 172.21.15.7:0/2359460914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f999c103120 msgr2=0x7f999c107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:50.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.525+0000 7f99a0ca2700 1 --2- 172.21.15.7:0/2359460914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f999c103120 0x7f999c107130 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f99900099c0 tx=0x7f9990009cd0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.526+0000 7f99a0ca2700 1 -- 172.21.15.7:0/2359460914 shutdown_connections 2024-06-10T01:25:50.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.526+0000 7f99a0ca2700 1 --2- 172.21.15.7:0/2359460914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f999c107860 0x7f999c10f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.526+0000 7f99a0ca2700 1 --2- 172.21.15.7:0/2359460914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f999c102770 0x7f999c102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.526+0000 7f99a0ca2700 1 --2- 172.21.15.7:0/2359460914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f999c103120 0x7f999c107130 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.526+0000 7f99a0ca2700 1 -- 172.21.15.7:0/2359460914 >> 172.21.15.7:0/2359460914 conn(0x7f999c079600 msgr2=0x7f999c079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:50.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.526+0000 7f99a0ca2700 1 -- 172.21.15.7:0/2359460914 shutdown_connections 2024-06-10T01:25:50.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.526+0000 7f99a0ca2700 1 -- 172.21.15.7:0/2359460914 wait complete. 2024-06-10T01:25:50.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.527+0000 7f99a0ca2700 1 Processor -- start 2024-06-10T01:25:50.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.527+0000 7f99a0ca2700 1 -- start start 2024-06-10T01:25:50.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.527+0000 7f99a0ca2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f999c102770 0x7f999c19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:50.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.527+0000 7f99a0ca2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f999c103120 0x7f999c19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:50.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.527+0000 7f99a0ca2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f999c107860 0x7f999c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:50.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.528+0000 7f99a0ca2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f999c112770 con 0x7f999c102770 2024-06-10T01:25:50.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.528+0000 7f99a0ca2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f999c1125f0 con 0x7f999c107860 2024-06-10T01:25:50.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.528+0000 7f99a0ca2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f999c1128f0 con 0x7f999c103120 2024-06-10T01:25:50.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.528+0000 7f999a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f999c102770 0x7f999c19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:50.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.528+0000 7f999a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f999c102770 0x7f999c19e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45450/0 (socket says 172.21.15.7:45450) 2024-06-10T01:25:50.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.528+0000 7f999a59c700 1 -- 172.21.15.7:0/720416499 learned_addr learned my addr 172.21.15.7:0/720416499 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:50.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.528+0000 7f999ad9d700 1 --2- 172.21.15.7:0/720416499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f999c107860 0x7f999c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:50.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.528+0000 7f999ad9d700 1 -- 172.21.15.7:0/720416499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f999c103120 msgr2=0x7f999c19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:50.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.528+0000 7f999ad9d700 1 --2- 172.21.15.7:0/720416499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f999c103120 0x7f999c19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.528+0000 7f999ad9d700 1 -- 172.21.15.7:0/720416499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f999c102770 msgr2=0x7f999c19e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:50.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.528+0000 7f999ad9d700 1 --2- 172.21.15.7:0/720416499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f999c102770 0x7f999c19e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.528+0000 7f999ad9d700 1 -- 172.21.15.7:0/720416499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f999c1991b0 con 0x7f999c107860 2024-06-10T01:25:50.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.528+0000 7f999a59c700 1 --2- 172.21.15.7:0/720416499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f999c102770 0x7f999c19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:25:50.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.529+0000 7f999ad9d700 1 --2- 172.21.15.7:0/720416499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f999c107860 0x7f999c198a50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f999400efc0 tx=0x7f999400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:50.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.529+0000 7f998b7fe700 1 -- 172.21.15.7:0/720416499 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99940092e0 con 0x7f999c107860 2024-06-10T01:25:50.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.529+0000 7f99a0ca2700 1 -- 172.21.15.7:0/720416499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f999c199440 con 0x7f999c107860 2024-06-10T01:25:50.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.529+0000 7f99a0ca2700 1 -- 172.21.15.7:0/720416499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f999c1a5710 con 0x7f999c107860 2024-06-10T01:25:50.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.529+0000 7f998b7fe700 1 -- 172.21.15.7:0/720416499 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9994004bb0 con 0x7f999c107860 2024-06-10T01:25:50.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.529+0000 7f998b7fe700 1 -- 172.21.15.7:0/720416499 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9994004280 con 0x7f999c107860 2024-06-10T01:25:50.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.532+0000 7f998b7fe700 1 -- 172.21.15.7:0/720416499 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f999401b030 con 0x7f999c107860 2024-06-10T01:25:50.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.534+0000 7f998b7fe700 1 --2- 172.21.15.7:0/720416499 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f99840a2ca0 0x7f99840a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:50.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.534+0000 7f999a59c700 1 --2- 172.21.15.7:0/720416499 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f99840a2ca0 0x7f99840a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:50.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.534+0000 7f998b7fe700 1 -- 172.21.15.7:0/720416499 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f99940cede0 con 0x7f999c107860 2024-06-10T01:25:50.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.535+0000 7f999a59c700 1 --2- 172.21.15.7:0/720416499 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f99840a2ca0 0x7f99840a5160 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f999c107420 tx=0x7f998c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:50.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.535+0000 7f99ab525b80 1 Processor -- start 2024-06-10T01:25:50.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.535+0000 7f99ab525b80 1 -- start start 2024-06-10T01:25:50.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.535+0000 7f99ab525b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56344a17b760 0x56344a1779e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:50.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.535+0000 7f99ab525b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56344a177f20 0x56344a1783a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:50.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.535+0000 7f99ab525b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56344a18a640 0x56344a18ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:50.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.536+0000 7f999ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56344a18a640 0x56344a18ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:50.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.536+0000 7f9999d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56344a177f20 0x56344a1783a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:50.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.536+0000 7f999ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56344a18a640 0x56344a18ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45460/0 (socket says 172.21.15.7:45460) 2024-06-10T01:25:50.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.536+0000 7f999ad9d700 1 -- 172.21.15.7:0/3606672359 learned_addr learned my addr 172.21.15.7:0/3606672359 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:50.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.536+0000 7f99ab525b80 1 -- 172.21.15.7:0/3606672359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56344a0c9810 con 0x56344a18a640 2024-06-10T01:25:50.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.536+0000 7f99ab525b80 1 -- 172.21.15.7:0/3606672359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56344a071720 con 0x56344a17b760 2024-06-10T01:25:50.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.536+0000 7f99ab525b80 1 -- 172.21.15.7:0/3606672359 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56344a0c57d0 con 0x56344a177f20 2024-06-10T01:25:50.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.536+0000 7f999a59c700 1 --2- 172.21.15.7:0/3606672359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56344a17b760 0x56344a1779e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:50.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.536+0000 7f999ad9d700 1 -- 172.21.15.7:0/3606672359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56344a177f20 msgr2=0x56344a1783a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:50.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.536+0000 7f999ad9d700 1 --2- 172.21.15.7:0/3606672359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56344a177f20 0x56344a1783a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.536+0000 7f999ad9d700 1 -- 172.21.15.7:0/3606672359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56344a17b760 msgr2=0x56344a1779e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:50.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.536+0000 7f999ad9d700 1 --2- 172.21.15.7:0/3606672359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56344a17b760 0x56344a1779e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.536+0000 7f999ad9d700 1 -- 172.21.15.7:0/3606672359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56344a067f50 con 0x56344a18a640 2024-06-10T01:25:50.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.537+0000 7f999ad9d700 1 --2- 172.21.15.7:0/3606672359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56344a18a640 0x56344a18ca30 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f999400ef90 tx=0x7f999409ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2b2b55a5a8ff70 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:50.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.538+0000 7f9988ff9700 1 -- 172.21.15.7:0/3606672359 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999409b350 con 0x56344a18a640 2024-06-10T01:25:50.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.538+0000 7f9988ff9700 1 -- 172.21.15.7:0/3606672359 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f999409f700 con 0x56344a18a640 2024-06-10T01:25:50.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.538+0000 7f9988ff9700 1 -- 172.21.15.7:0/3606672359 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99940bb0c0 con 0x56344a18a640 2024-06-10T01:25:50.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.540+0000 7f99ab525b80 1 -- 172.21.15.7:0/3606672359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56344a18a640 msgr2=0x56344a18ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:50.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.540+0000 7f99ab525b80 1 --2- 172.21.15.7:0/3606672359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56344a18a640 0x56344a18ca30 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f999400ef90 tx=0x7f999409ad30 comp rx=0 tx=0).stop 2024-06-10T01:25:50.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.540+0000 7f99ab525b80 1 -- 172.21.15.7:0/3606672359 shutdown_connections 2024-06-10T01:25:50.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.540+0000 7f99ab525b80 1 --2- 172.21.15.7:0/3606672359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56344a18a640 0x56344a18ca30 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.540+0000 7f99ab525b80 1 --2- 172.21.15.7:0/3606672359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56344a17b760 0x56344a1779e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.540+0000 7f99ab525b80 1 --2- 172.21.15.7:0/3606672359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56344a177f20 0x56344a1783a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.540+0000 7f99ab525b80 1 -- 172.21.15.7:0/3606672359 >> 172.21.15.7:0/3606672359 conn(0x56344a0a2cc0 msgr2=0x56344a172850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:50.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.540+0000 7f99ab525b80 1 -- 172.21.15.7:0/3606672359 shutdown_connections 2024-06-10T01:25:50.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.540+0000 7f99ab525b80 1 -- 172.21.15.7:0/3606672359 wait complete. 2024-06-10T01:25:50.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.541+0000 7f99ab525b80 1 -- start start 2024-06-10T01:25:50.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.541+0000 7f99ab525b80 1 Processor -- start 2024-06-10T01:25:50.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.541+0000 7f99ab525b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56344a177f20 0x56344a189dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:50.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.541+0000 7f99ab525b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56344a17b760 0x56344a1870d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:50.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.541+0000 7f99ab525b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56344a18a640 0x56344a187610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:50.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.541+0000 7f99ab525b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56344a18d1f0 con 0x56344a17b760 2024-06-10T01:25:50.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.541+0000 7f99ab525b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56344a069bf0 con 0x56344a177f20 2024-06-10T01:25:50.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.541+0000 7f99ab525b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56344a18d370 con 0x56344a18a640 2024-06-10T01:25:50.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.541+0000 7f99ab525b80 10 client.0 _ll_register_callbacks cb 0x56344a063920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:25:50.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.542+0000 7f999ad9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56344a18a640 0x56344a187610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:50.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.542+0000 7f999ad9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56344a18a640 0x56344a187610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36434/0 (socket says 172.21.15.7:36434) 2024-06-10T01:25:50.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.542+0000 7f999ad9d700 1 -- 172.21.15.7:0/1304165924 learned_addr learned my addr 172.21.15.7:0/1304165924 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:50.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.542+0000 7f999a59c700 1 --2- 172.21.15.7:0/1304165924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56344a177f20 0x56344a189dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:50.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.542+0000 7f999ad9d700 1 -- 172.21.15.7:0/1304165924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56344a177f20 msgr2=0x56344a189dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:50.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.542+0000 7f999ad9d700 1 --2- 172.21.15.7:0/1304165924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56344a177f20 0x56344a189dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.542+0000 7f999ad9d700 1 -- 172.21.15.7:0/1304165924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56344a17b760 msgr2=0x56344a1870d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T01:25:50.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.542+0000 7f999ad9d700 1 --2- 172.21.15.7:0/1304165924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56344a17b760 0x56344a1870d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.542+0000 7f999ad9d700 1 -- 172.21.15.7:0/1304165924 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56344a0188d0 con 0x56344a18a640 2024-06-10T01:25:50.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.543+0000 7f999ad9d700 1 --2- 172.21.15.7:0/1304165924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56344a18a640 0x56344a187610 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9994097d20 tx=0x7f9994099860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c89a16d3563bf03f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:50.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.543+0000 7f9981ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:25:50.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.543+0000 7f9981ffb700 1 -- 172.21.15.7:0/1304165924 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999409a9a0 con 0x56344a18a640 2024-06-10T01:25:50.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.543+0000 7f9981ffb700 1 -- 172.21.15.7:0/1304165924 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f99940bb050 con 0x56344a18a640 2024-06-10T01:25:50.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.543+0000 7f99ab525b80 1 -- 172.21.15.7:0/1304165924 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f999409f700 con 0x56344a18a640 2024-06-10T01:25:50.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.544+0000 7f9981ffb700 1 -- 172.21.15.7:0/1304165924 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f99940b42a0 con 0x56344a18a640 2024-06-10T01:25:50.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.544+0000 7f99ab525b80 10 client.29335 fetch_fsmap learned FSMap version 52 2024-06-10T01:25:50.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.544+0000 7f99ab525b80 1 -- 172.21.15.7:0/1304165924 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x56344a18d1f0 con 0x56344a18a640 2024-06-10T01:25:50.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.544+0000 7f9981ffb700 1 -- 172.21.15.7:0/1304165924 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99940a5020 con 0x56344a18a640 2024-06-10T01:25:50.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.544+0000 7f9981ffb700 1 -- 172.21.15.7:0/1304165924 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f999407e030 con 0x56344a18a640 2024-06-10T01:25:50.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.545+0000 7f99ab525b80 10 client.29335 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:25:50.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.545+0000 7f99ab525b80 10 client.29335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:25:50.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.545+0000 7f99ab525b80 20 client.29335 populate_metadata read hostname 'smithi007' 2024-06-10T01:25:50.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.545+0000 7f99ab525b80 1 --2- 172.21.15.7:0/1304165924 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56344a199c10 0x56344a19c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:50.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.545+0000 7f99ab525b80 4 client.29335 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:25:50.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.545+0000 7f99ab525b80 1 -- 172.21.15.7:0/1304165924 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f999409f700 con 0x56344a199c10 2024-06-10T01:25:50.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.545+0000 7f999a59c700 1 --2- 172.21.15.7:0/1304165924 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56344a199c10 0x56344a19c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:50.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.546+0000 7f9981ffb700 10 client.29335 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:50.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.546+0000 7f999a59c700 1 --2- 172.21.15.7:0/1304165924 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56344a199c10 0x56344a19c000 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d00b31b1ddae5cb server_cookie=1e6a1ce517762bb1 in_seq=0 out_seq=0 2024-06-10T01:25:50.541 INFO:tasks.workunit.client.0.smithi007.stdout: 10 160752 24.23 MB/sec execute 267 sec latency 103.144 ms 2024-06-10T01:25:50.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.548+0000 7f9981ffb700 1 -- 172.21.15.7:0/1304165924 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f999409f700 con 0x56344a199c10 2024-06-10T01:25:50.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.548+0000 7f9981ffb700 10 client.29335 handle_command_reply: tid=0 2024-06-10T01:25:50.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.548+0000 7f99ab525b80 1 client.29335 shutdown 2024-06-10T01:25:50.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.548+0000 7f99ab525b80 1 -- 172.21.15.7:0/1304165924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56344a18a640 msgr2=0x56344a187610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:50.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.548+0000 7f99ab525b80 1 --2- 172.21.15.7:0/1304165924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56344a18a640 0x56344a187610 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9994097d20 tx=0x7f9994099860 comp rx=0 tx=0).stop 2024-06-10T01:25:50.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.548+0000 7f99ab525b80 1 -- 172.21.15.7:0/1304165924 shutdown_connections 2024-06-10T01:25:50.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.548+0000 7f99ab525b80 1 --2- 172.21.15.7:0/1304165924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56344a18a640 0x56344a187610 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.548+0000 7f99ab525b80 1 --2- 172.21.15.7:0/1304165924 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56344a199c10 0x56344a19c000 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.548+0000 7f9981ffb700 0 client.29335 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:50.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.548+0000 7f99ab525b80 1 --2- 172.21.15.7:0/1304165924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56344a177f20 0x56344a189dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.548+0000 7f99ab525b80 1 --2- 172.21.15.7:0/1304165924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56344a17b760 0x56344a1870d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.548+0000 7f99ab525b80 1 -- 172.21.15.7:0/1304165924 >> 172.21.15.7:0/1304165924 conn(0x56344a0a2cc0 msgr2=0x56344a18aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:50.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.548+0000 7f99ab525b80 1 -- 172.21.15.7:0/1304165924 shutdown_connections 2024-06-10T01:25:50.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.548+0000 7f99ab525b80 1 -- 172.21.15.7:0/1304165924 wait complete. 2024-06-10T01:25:50.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.548+0000 7f99ab525b80 20 client.29335 trim_cache size 0 max 16384 2024-06-10T01:25:50.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.585+0000 7f99ab525b80 1 Processor -- start 2024-06-10T01:25:50.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.585+0000 7f99ab525b80 1 -- start start 2024-06-10T01:25:50.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.585+0000 7f99ab525b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56344a177f20 0x56344a184d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:50.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.585+0000 7f99ab525b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56344a17b760 0x56344a18ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:50.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.585+0000 7f99ab525b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56344a18a640 0x56344a18bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:50.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.585+0000 7f99ab525b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f997c011300 con 0x56344a177f20 2024-06-10T01:25:50.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.586+0000 7f999a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56344a177f20 0x56344a184d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:50.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.586+0000 7f999a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56344a177f20 0x56344a184d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45474/0 (socket says 172.21.15.7:45474) 2024-06-10T01:25:50.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.586+0000 7f9999d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56344a17b760 0x56344a18ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:50.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.586+0000 7f999a59c700 1 -- 172.21.15.7:0/905758371 learned_addr learned my addr 172.21.15.7:0/905758371 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:50.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.586+0000 7f99ab525b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f997c00eb90 con 0x56344a18a640 2024-06-10T01:25:50.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.586+0000 7f99ab525b80 1 -- 172.21.15.7:0/905758371 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f997c00c9e0 con 0x56344a17b760 2024-06-10T01:25:50.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.586+0000 7f999ad9d700 1 --2- 172.21.15.7:0/905758371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56344a18a640 0x56344a18bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:50.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.586+0000 7f999a59c700 1 -- 172.21.15.7:0/905758371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56344a17b760 msgr2=0x56344a18ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:50.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.586+0000 7f999a59c700 1 --2- 172.21.15.7:0/905758371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56344a17b760 0x56344a18ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.586+0000 7f999a59c700 1 -- 172.21.15.7:0/905758371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56344a18a640 msgr2=0x56344a18bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:50.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.586+0000 7f999a59c700 1 --2- 172.21.15.7:0/905758371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56344a18a640 0x56344a18bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.586+0000 7f999a59c700 1 -- 172.21.15.7:0/905758371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56344a17b320 con 0x56344a177f20 2024-06-10T01:25:50.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.586+0000 7f999ad9d700 1 --2- 172.21.15.7:0/905758371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56344a18a640 0x56344a18bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:25:50.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.586+0000 7f999a59c700 1 --2- 172.21.15.7:0/905758371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56344a177f20 0x56344a184d10 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f998c013010 tx=0x7f998c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6ae96278558cc8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:50.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.587+0000 7f9983fff700 1 -- 172.21.15.7:0/905758371 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c003bd0 con 0x56344a177f20 2024-06-10T01:25:50.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.587+0000 7f9983fff700 1 -- 172.21.15.7:0/905758371 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f999409f700 con 0x56344a177f20 2024-06-10T01:25:50.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.587+0000 7f9983fff700 1 -- 172.21.15.7:0/905758371 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c029660 con 0x56344a177f20 2024-06-10T01:25:50.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.588+0000 7f99ab525b80 1 -- 172.21.15.7:0/905758371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56344a177f20 msgr2=0x56344a184d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:50.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.588+0000 7f99ab525b80 1 --2- 172.21.15.7:0/905758371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56344a177f20 0x56344a184d10 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f998c013010 tx=0x7f998c014000 comp rx=0 tx=0).stop 2024-06-10T01:25:50.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.588+0000 7f99ab525b80 1 -- 172.21.15.7:0/905758371 shutdown_connections 2024-06-10T01:25:50.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.589+0000 7f99ab525b80 1 --2- 172.21.15.7:0/905758371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56344a18a640 0x56344a18bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.589+0000 7f99ab525b80 1 --2- 172.21.15.7:0/905758371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56344a177f20 0x56344a184d10 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.589+0000 7f99ab525b80 1 --2- 172.21.15.7:0/905758371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56344a17b760 0x56344a18ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.589+0000 7f99ab525b80 1 -- 172.21.15.7:0/905758371 >> 172.21.15.7:0/905758371 conn(0x56344a0a2cc0 msgr2=0x56344a184600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:50.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.589+0000 7f99ab525b80 1 -- 172.21.15.7:0/905758371 shutdown_connections 2024-06-10T01:25:50.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.589+0000 7f99ab525b80 1 -- 172.21.15.7:0/905758371 wait complete. 2024-06-10T01:25:50.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.590+0000 7f99ab525b80 1 -- start start 2024-06-10T01:25:50.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.590+0000 7f99ab525b80 1 Processor -- start 2024-06-10T01:25:50.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.591+0000 7f99ab525b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56344a177f20 0x56344a1a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:50.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.591+0000 7f99ab525b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56344a17b760 0x56344a192660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:50.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.591+0000 7f999a59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56344a177f20 0x56344a1a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:50.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.591+0000 7f999a59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56344a177f20 0x56344a1a1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36446/0 (socket says 172.21.15.7:36446) 2024-06-10T01:25:50.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.592+0000 7f999a59c700 1 -- 172.21.15.7:0/475989526 learned_addr learned my addr 172.21.15.7:0/475989526 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:50.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.592+0000 7f99ab525b80 1 --2- 172.21.15.7:0/475989526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56344a18a640 0x56344a192ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:50.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.592+0000 7f9999d9b700 1 --2- 172.21.15.7:0/475989526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56344a17b760 0x56344a192660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:50.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.592+0000 7f99ab525b80 1 -- 172.21.15.7:0/475989526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f997c00d050 con 0x56344a17b760 2024-06-10T01:25:50.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.592+0000 7f99ab525b80 1 -- 172.21.15.7:0/475989526 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f997c00e4c0 con 0x56344a18a640 2024-06-10T01:25:50.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.592+0000 7f99ab525b80 1 -- 172.21.15.7:0/475989526 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f997c00d720 con 0x56344a177f20 2024-06-10T01:25:50.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.592+0000 7f999ad9d700 1 --2- 172.21.15.7:0/475989526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56344a18a640 0x56344a192ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:50.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.592+0000 7f99ab525b80 10 client.0 _ll_register_callbacks cb 0x56344a063920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:25:50.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.593+0000 7f9999d9b700 1 -- 172.21.15.7:0/475989526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56344a177f20 msgr2=0x56344a1a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:50.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.593+0000 7f9999d9b700 1 --2- 172.21.15.7:0/475989526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56344a177f20 0x56344a1a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.593+0000 7f9999d9b700 1 -- 172.21.15.7:0/475989526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56344a18a640 msgr2=0x56344a192ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:50.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.593+0000 7f9999d9b700 1 --2- 172.21.15.7:0/475989526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56344a18a640 0x56344a192ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.593+0000 7f9999d9b700 1 -- 172.21.15.7:0/475989526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56344a175290 con 0x56344a17b760 2024-06-10T01:25:50.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.593+0000 7f9999d9b700 1 --2- 172.21.15.7:0/475989526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56344a17b760 0x56344a192660 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f9990009990 tx=0x7f9990005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a856fb012bb0e62 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:50.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.593+0000 7f9981ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:25:50.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.594+0000 7f9981ffb700 1 -- 172.21.15.7:0/475989526 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9990029060 con 0x56344a17b760 2024-06-10T01:25:50.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.594+0000 7f9981ffb700 1 -- 172.21.15.7:0/475989526 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f99900052c0 con 0x56344a17b760 2024-06-10T01:25:50.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.594+0000 7f99ab525b80 1 -- 172.21.15.7:0/475989526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f999409f700 con 0x56344a17b760 2024-06-10T01:25:50.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.594+0000 7f9981ffb700 1 -- 172.21.15.7:0/475989526 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f9990030050 con 0x56344a17b760 2024-06-10T01:25:50.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.594+0000 7f99ab525b80 10 client.20178 fetch_fsmap learned FSMap version 52 2024-06-10T01:25:50.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.594+0000 7f99ab525b80 1 -- 172.21.15.7:0/475989526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x56344a204810 con 0x56344a17b760 2024-06-10T01:25:50.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.595+0000 7f9981ffb700 1 -- 172.21.15.7:0/475989526 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9990009e60 con 0x56344a17b760 2024-06-10T01:25:50.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.595+0000 7f9981ffb700 1 -- 172.21.15.7:0/475989526 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f999000f600 con 0x56344a17b760 2024-06-10T01:25:50.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.595+0000 7f99ab525b80 10 client.20178 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:25:50.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.595+0000 7f99ab525b80 10 client.20178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:25:50.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.595+0000 7f99ab525b80 20 client.20178 populate_metadata read hostname 'smithi007' 2024-06-10T01:25:50.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.595+0000 7f99ab525b80 1 --2- 172.21.15.7:0/475989526 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56344a199c10 0x56344a206f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:50.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.595+0000 7f99ab525b80 4 client.20178 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:25:50.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.595+0000 7f99ab525b80 1 -- 172.21.15.7:0/475989526 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f999409f700 con 0x56344a199c10 2024-06-10T01:25:50.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.596+0000 7f999a59c700 1 --2- 172.21.15.7:0/475989526 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56344a199c10 0x56344a206f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:50.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.596+0000 7f999a59c700 1 --2- 172.21.15.7:0/475989526 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56344a199c10 0x56344a206f60 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59b1804d41f82181 server_cookie=7cc5465c84097477 in_seq=0 out_seq=0 2024-06-10T01:25:50.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.596+0000 7f9981ffb700 10 client.20178 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:50.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.601+0000 7f9981ffb700 1 -- 172.21.15.7:0/475989526 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f999409f700 con 0x56344a199c10 2024-06-10T01:25:50.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.601+0000 7f9981ffb700 10 client.20178 handle_command_reply: tid=0 2024-06-10T01:25:50.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.601+0000 7f99ab525b80 1 client.20178 shutdown 2024-06-10T01:25:50.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.601+0000 7f99ab525b80 1 -- 172.21.15.7:0/475989526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56344a17b760 msgr2=0x56344a192660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:50.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.601+0000 7f99ab525b80 1 --2- 172.21.15.7:0/475989526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56344a17b760 0x56344a192660 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f9990009990 tx=0x7f9990005be0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.601+0000 7f99ab525b80 1 -- 172.21.15.7:0/475989526 shutdown_connections 2024-06-10T01:25:50.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.601+0000 7f99ab525b80 1 --2- 172.21.15.7:0/475989526 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56344a199c10 0x56344a206f60 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.601+0000 7f9981ffb700 0 client.20178 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:50.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.601+0000 7f99ab525b80 1 --2- 172.21.15.7:0/475989526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56344a18a640 0x56344a192ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.601+0000 7f99ab525b80 1 --2- 172.21.15.7:0/475989526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56344a177f20 0x56344a1a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.601+0000 7f99ab525b80 1 --2- 172.21.15.7:0/475989526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56344a17b760 0x56344a192660 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.601+0000 7f99ab525b80 1 -- 172.21.15.7:0/475989526 >> 172.21.15.7:0/475989526 conn(0x56344a0a2cc0 msgr2=0x56344a16be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:50.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.601+0000 7f99ab525b80 1 -- 172.21.15.7:0/475989526 shutdown_connections 2024-06-10T01:25:50.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.602+0000 7f99ab525b80 1 -- 172.21.15.7:0/475989526 wait complete. 2024-06-10T01:25:50.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.602+0000 7f99ab525b80 20 client.20178 trim_cache size 0 max 16384 2024-06-10T01:25:50.599 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:25:50.599 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-10T01:25:50.599 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-10T01:25:50.600 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:25:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.602+0000 7f99a0ca2700 1 -- 172.21.15.7:0/720416499 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f99840a2ca0 msgr2=0x7f99840a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.603+0000 7f99a0ca2700 1 --2- 172.21.15.7:0/720416499 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f99840a2ca0 0x7f99840a5160 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f999c107420 tx=0x7f998c009260 comp rx=0 tx=0).stop 2024-06-10T01:25:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.603+0000 7f99a0ca2700 1 -- 172.21.15.7:0/720416499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f999c107860 msgr2=0x7f999c198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.603+0000 7f99a0ca2700 1 --2- 172.21.15.7:0/720416499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f999c107860 0x7f999c198a50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f999400efc0 tx=0x7f999400c3a0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.603+0000 7f99a0ca2700 1 -- 172.21.15.7:0/720416499 shutdown_connections 2024-06-10T01:25:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.603+0000 7f99a0ca2700 1 --2- 172.21.15.7:0/720416499 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f99840a2ca0 0x7f99840a5160 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.603+0000 7f99a0ca2700 1 --2- 172.21.15.7:0/720416499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f999c107860 0x7f999c198a50 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.603+0000 7f99a0ca2700 1 --2- 172.21.15.7:0/720416499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f999c102770 0x7f999c19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.603+0000 7f99a0ca2700 1 --2- 172.21.15.7:0/720416499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f999c103120 0x7f999c19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.603+0000 7f99a0ca2700 1 -- 172.21.15.7:0/720416499 >> 172.21.15.7:0/720416499 conn(0x7f999c079600 msgr2=0x7f999c101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.605+0000 7f99a0ca2700 1 -- 172.21.15.7:0/720416499 shutdown_connections 2024-06-10T01:25:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:50.605+0000 7f99a0ca2700 1 -- 172.21.15.7:0/720416499 wait complete. 2024-06-10T01:25:51.540 INFO:tasks.workunit.client.0.smithi007.stdout: 10 161423 24.31 MB/sec execute 268 sec latency 94.678 ms 2024-06-10T01:25:51.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:51 smithi078 ceph-mon[36683]: pgmap v1028: 129 pgs: 129 active+clean; 660 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 16 MiB/s wr, 942 op/s 2024-06-10T01:25:51.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:51 smithi175 ceph-mon[36636]: pgmap v1028: 129 pgs: 129 active+clean; 660 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 16 MiB/s wr, 942 op/s 2024-06-10T01:25:52.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:51 smithi007 ceph-mon[32398]: pgmap v1028: 129 pgs: 129 active+clean; 660 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 16 MiB/s wr, 942 op/s 2024-06-10T01:25:52.540 INFO:tasks.workunit.client.0.smithi007.stdout: 10 161984 24.33 MB/sec execute 269 sec latency 64.149 ms 2024-06-10T01:25:52.624 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:25:52.624 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:25:53.074 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:25:53.540 INFO:tasks.workunit.client.0.smithi007.stdout: 10 162528 24.32 MB/sec execute 270 sec latency 68.440 ms 2024-06-10T01:25:53.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:53 smithi007 ceph-mon[32398]: pgmap v1029: 129 pgs: 129 active+clean; 645 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 12 MiB/s wr, 906 op/s 2024-06-10T01:25:53.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:53 smithi078 ceph-mon[36683]: pgmap v1029: 129 pgs: 129 active+clean; 645 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 12 MiB/s wr, 906 op/s 2024-06-10T01:25:53.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:53 smithi175 ceph-mon[36636]: pgmap v1029: 129 pgs: 129 active+clean; 645 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 12 MiB/s wr, 906 op/s 2024-06-10T01:25:54.059 INFO:tasks.quiescer.fs.[cephfs]:Total duration (44.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-06-10T01:25:54.059 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.0 2024-06-10T01:25:54.059 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:25:54.540 INFO:tasks.workunit.client.0.smithi007.stdout: 10 163203 24.36 MB/sec execute 271 sec latency 114.380 ms 2024-06-10T01:25:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.718+0000 7f9552929640 1 -- 172.21.15.7:0/3709012353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f954c104e10 msgr2=0x7f954c107200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.718+0000 7f9552929640 1 --2- 172.21.15.7:0/3709012353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f954c104e10 0x7f954c107200 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f9540009900 tx=0x7f954002a410 comp rx=0 tx=0).stop 2024-06-10T01:25:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.718+0000 7f9552929640 1 -- 172.21.15.7:0/3709012353 shutdown_connections 2024-06-10T01:25:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.718+0000 7f9552929640 1 --2- 172.21.15.7:0/3709012353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f954c107740 0x7f954c109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.718+0000 7f9552929640 1 --2- 172.21.15.7:0/3709012353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f954c104e10 0x7f954c107200 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.718+0000 7f9552929640 1 --2- 172.21.15.7:0/3709012353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f954c1024e0 0x7f954c1048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.718+0000 7f9552929640 1 -- 172.21.15.7:0/3709012353 >> 172.21.15.7:0/3709012353 conn(0x7f954c0fbe70 msgr2=0x7f954c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.718+0000 7f9552929640 1 -- 172.21.15.7:0/3709012353 shutdown_connections 2024-06-10T01:25:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.718+0000 7f9552929640 1 -- 172.21.15.7:0/3709012353 wait complete. 2024-06-10T01:25:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.719+0000 7f9552929640 1 Processor -- start 2024-06-10T01:25:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.719+0000 7f9552929640 1 -- start start 2024-06-10T01:25:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.719+0000 7f9552929640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f954c1024e0 0x7f954c101e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.719+0000 7f9552929640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f954c104e10 0x7f954c100490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.719+0000 7f9552929640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f954c107740 0x7f954c1009d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.719+0000 7f9552929640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f954c10d680 con 0x7f954c104e10 2024-06-10T01:25:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.719+0000 7f9552929640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f954c10d500 con 0x7f954c107740 2024-06-10T01:25:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.719+0000 7f9552929640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f954c10d800 con 0x7f954c1024e0 2024-06-10T01:25:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.719+0000 7f954b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f954c104e10 0x7f954c100490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.719+0000 7f954b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f954c104e10 0x7f954c100490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36558/0 (socket says 172.21.15.7:36558) 2024-06-10T01:25:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.719+0000 7f954b7fe640 1 -- 172.21.15.7:0/3703735120 learned_addr learned my addr 172.21.15.7:0/3703735120 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.719+0000 7f9550ec6640 1 --2- 172.21.15.7:0/3703735120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f954c107740 0x7f954c1009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.719+0000 7f954bfff640 1 --2- 172.21.15.7:0/3703735120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f954c1024e0 0x7f954c101e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.720+0000 7f954b7fe640 1 -- 172.21.15.7:0/3703735120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f954c1024e0 msgr2=0x7f954c101e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.720+0000 7f954b7fe640 1 --2- 172.21.15.7:0/3703735120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f954c1024e0 0x7f954c101e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.720+0000 7f954b7fe640 1 -- 172.21.15.7:0/3703735120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f954c107740 msgr2=0x7f954c1009d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.720+0000 7f954b7fe640 1 --2- 172.21.15.7:0/3703735120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f954c107740 0x7f954c1009d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.720+0000 7f954b7fe640 1 -- 172.21.15.7:0/3703735120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f954c101260 con 0x7f954c104e10 2024-06-10T01:25:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.720+0000 7f9550ec6640 1 --2- 172.21.15.7:0/3703735120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f954c107740 0x7f954c1009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:25:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.720+0000 7f954b7fe640 1 --2- 172.21.15.7:0/3703735120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f954c104e10 0x7f954c100490 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f9540031610 tx=0x7f9540004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.720+0000 7f95497fa640 1 -- 172.21.15.7:0/3703735120 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954003f030 con 0x7f954c104e10 2024-06-10T01:25:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.720+0000 7f9552929640 1 -- 172.21.15.7:0/3703735120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f954c1a1e10 con 0x7f954c104e10 2024-06-10T01:25:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.720+0000 7f95497fa640 1 -- 172.21.15.7:0/3703735120 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9540003ad0 con 0x7f954c104e10 2024-06-10T01:25:54.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.720+0000 7f95497fa640 1 -- 172.21.15.7:0/3703735120 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9540035070 con 0x7f954c104e10 2024-06-10T01:25:54.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.720+0000 7f9552929640 1 -- 172.21.15.7:0/3703735120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f954c1a2280 con 0x7f954c104e10 2024-06-10T01:25:54.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.723+0000 7f9555639740 1 Processor -- start 2024-06-10T01:25:54.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.723+0000 7f9555639740 1 -- start start 2024-06-10T01:25:54.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.723+0000 7f9555639740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607e7481010 0x5607e7481430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.723+0000 7f9555639740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607e7481a00 0x5607e7486500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.723+0000 7f9555639740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607e7486b70 0x5607e7489000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.723+0000 7f9555639740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5607e73e44f0 con 0x5607e7486b70 2024-06-10T01:25:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.723+0000 7f9555639740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5607e7382870 con 0x5607e7481a00 2024-06-10T01:25:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.723+0000 7f9555639740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5607e73e04b0 con 0x5607e7481010 2024-06-10T01:25:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.723+0000 7f95497fa640 1 -- 172.21.15.7:0/3703735120 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9540042030 con 0x7f954c104e10 2024-06-10T01:25:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.723+0000 7f9550ec6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607e7481010 0x5607e7481430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.723+0000 7f954bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607e7481a00 0x5607e7486500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.723+0000 7f9550ec6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607e7481010 0x5607e7481430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:33746/0 (socket says 172.21.15.7:33746) 2024-06-10T01:25:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.723+0000 7f9550ec6640 1 -- 172.21.15.7:0/3741208998 learned_addr learned my addr 172.21.15.7:0/3741208998 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.724+0000 7f954b7fe640 1 --2- 172.21.15.7:0/3741208998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607e7486b70 0x5607e7489000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.724+0000 7f954bfff640 1 -- 172.21.15.7:0/3741208998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607e7481010 msgr2=0x5607e7481430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.724+0000 7f954bfff640 1 --2- 172.21.15.7:0/3741208998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607e7481010 0x5607e7481430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.724+0000 7f954bfff640 1 -- 172.21.15.7:0/3741208998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607e7486b70 msgr2=0x5607e7489000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.724+0000 7f954bfff640 1 --2- 172.21.15.7:0/3741208998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607e7486b70 0x5607e7489000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.724+0000 7f954bfff640 1 -- 172.21.15.7:0/3741208998 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607e739e9b0 con 0x5607e7481a00 2024-06-10T01:25:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.724+0000 7f9550ec6640 1 --2- 172.21.15.7:0/3741208998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607e7481010 0x5607e7481430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:25:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.724+0000 7f954b7fe640 1 --2- 172.21.15.7:0/3741208998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607e7486b70 0x5607e7489000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:25:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.724+0000 7f954bfff640 1 --2- 172.21.15.7:0/3741208998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607e7481a00 0x5607e7486500 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f953c009c20 tx=0x7f953c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2328afcbfbe160d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.724+0000 7f95327fc640 1 -- 172.21.15.7:0/3741208998 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c004050 con 0x5607e7481a00 2024-06-10T01:25:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.724+0000 7f95497fa640 1 --2- 172.21.15.7:0/3703735120 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f95340a2b80 0x7f95340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.724+0000 7f95327fc640 1 -- 172.21.15.7:0/3741208998 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f953c005500 con 0x5607e7481a00 2024-06-10T01:25:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.724+0000 7f95327fc640 1 -- 172.21.15.7:0/3741208998 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c004dd0 con 0x5607e7481a00 2024-06-10T01:25:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.724+0000 7f9550ec6640 1 --2- 172.21.15.7:0/3703735120 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f95340a2b80 0x7f95340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.725+0000 7f95497fa640 1 -- 172.21.15.7:0/3703735120 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f95400ef590 con 0x7f954c104e10 2024-06-10T01:25:54.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.725+0000 7f9555639740 1 -- 172.21.15.7:0/3741208998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607e7481a00 msgr2=0x5607e7486500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:54.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.725+0000 7f9555639740 1 --2- 172.21.15.7:0/3741208998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607e7481a00 0x5607e7486500 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f953c009c20 tx=0x7f953c011810 comp rx=0 tx=0).stop 2024-06-10T01:25:54.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.725+0000 7f9550ec6640 1 --2- 172.21.15.7:0/3703735120 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f95340a2b80 0x7f95340a5040 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x5607e7482260 tx=0x7f9544009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:54.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.725+0000 7f9555639740 1 -- 172.21.15.7:0/3741208998 shutdown_connections 2024-06-10T01:25:54.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.725+0000 7f9555639740 1 --2- 172.21.15.7:0/3741208998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607e7486b70 0x5607e7489000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:54.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.725+0000 7f9555639740 1 --2- 172.21.15.7:0/3741208998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607e7481a00 0x5607e7486500 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:54.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.725+0000 7f9555639740 1 --2- 172.21.15.7:0/3741208998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607e7481010 0x5607e7481430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:54.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.725+0000 7f9555639740 1 -- 172.21.15.7:0/3741208998 >> 172.21.15.7:0/3741208998 conn(0x5607e747c680 msgr2=0x5607e747ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:54.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.725+0000 7f9555639740 1 -- 172.21.15.7:0/3741208998 shutdown_connections 2024-06-10T01:25:54.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.725+0000 7f9555639740 1 -- 172.21.15.7:0/3741208998 wait complete. 2024-06-10T01:25:54.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.725+0000 7f9555639740 1 -- start start 2024-06-10T01:25:54.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.726+0000 7f9555639740 1 Processor -- start 2024-06-10T01:25:54.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.726+0000 7f9555639740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607e7481010 0x5607e748b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:54.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.726+0000 7f9555639740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607e7481a00 0x5607e748b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:54.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.726+0000 7f9555639740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607e7486b70 0x5607e7492800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:54.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.726+0000 7f9555639740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5607e7489b20 con 0x5607e7481010 2024-06-10T01:25:54.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.726+0000 7f9555639740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5607e736dae0 con 0x5607e7486b70 2024-06-10T01:25:54.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.726+0000 7f9555639740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5607e7489ca0 con 0x5607e7481a00 2024-06-10T01:25:54.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.726+0000 7f9555639740 10 client.0 _ll_register_callbacks cb 0x5607e73a69b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:25:54.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.726+0000 7f954bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607e7481010 0x5607e748b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:54.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.726+0000 7f954bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607e7481010 0x5607e748b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36562/0 (socket says 172.21.15.7:36562) 2024-06-10T01:25:54.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.726+0000 7f954bfff640 1 -- 172.21.15.7:0/4292105972 learned_addr learned my addr 172.21.15.7:0/4292105972 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:54.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.726+0000 7f9550ec6640 1 --2- 172.21.15.7:0/4292105972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607e7486b70 0x5607e7492800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:54.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.726+0000 7f954b7fe640 1 --2- 172.21.15.7:0/4292105972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607e7481a00 0x5607e748b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:54.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.726+0000 7f9550ec6640 1 -- 172.21.15.7:0/4292105972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607e7481a00 msgr2=0x5607e748b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:54.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.726+0000 7f9550ec6640 1 --2- 172.21.15.7:0/4292105972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607e7481a00 0x5607e748b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:54.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.726+0000 7f9550ec6640 1 -- 172.21.15.7:0/4292105972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607e7481010 msgr2=0x5607e748b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:54.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.726+0000 7f9550ec6640 1 --2- 172.21.15.7:0/4292105972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607e7481010 0x5607e748b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:54.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.726+0000 7f9550ec6640 1 -- 172.21.15.7:0/4292105972 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607e70f8210 con 0x5607e7486b70 2024-06-10T01:25:54.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.726+0000 7f954bfff640 1 --2- 172.21.15.7:0/4292105972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607e7481010 0x5607e748b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:25:54.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.727+0000 7f9550ec6640 1 --2- 172.21.15.7:0/4292105972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607e7486b70 0x5607e7492800 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9544009fd0 tx=0x7f9544004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d370f745ed4dddc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:54.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.727+0000 7f952b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:25:54.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.727+0000 7f952b7fe640 1 -- 172.21.15.7:0/4292105972 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9544004770 con 0x5607e7486b70 2024-06-10T01:25:54.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.727+0000 7f952b7fe640 1 -- 172.21.15.7:0/4292105972 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f9544017030 con 0x5607e7486b70 2024-06-10T01:25:54.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.727+0000 7f9555639740 1 -- 172.21.15.7:0/4292105972 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f953c005500 con 0x5607e7486b70 2024-06-10T01:25:54.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.727+0000 7f952b7fe640 1 -- 172.21.15.7:0/4292105972 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f9544033050 con 0x5607e7486b70 2024-06-10T01:25:54.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.727+0000 7f9555639740 10 client.29402 fetch_fsmap learned FSMap version 52 2024-06-10T01:25:54.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.727+0000 7f9555639740 1 -- 172.21.15.7:0/4292105972 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5607e71c4620 con 0x5607e7486b70 2024-06-10T01:25:54.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.727+0000 7f952b7fe640 1 -- 172.21.15.7:0/4292105972 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f953c005500 con 0x5607e7486b70 2024-06-10T01:25:54.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.727+0000 7f952b7fe640 1 -- 172.21.15.7:0/4292105972 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f9544010550 con 0x5607e7486b70 2024-06-10T01:25:54.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.728+0000 7f9555639740 10 client.29402 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:25:54.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.728+0000 7f9555639740 10 client.29402 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:25:54.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.728+0000 7f9555639740 20 client.29402 populate_metadata read hostname 'smithi007' 2024-06-10T01:25:54.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.728+0000 7f9555639740 1 --2- 172.21.15.7:0/4292105972 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5607e7497140 0x5607e74995d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:54.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.728+0000 7f9555639740 4 client.29402 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:25:54.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.728+0000 7f9555639740 1 -- 172.21.15.7:0/4292105972 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607e73f7f10 con 0x5607e7497140 2024-06-10T01:25:54.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.728+0000 7f954bfff640 1 --2- 172.21.15.7:0/4292105972 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5607e7497140 0x5607e74995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:54.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.728+0000 7f954bfff640 1 --2- 172.21.15.7:0/4292105972 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5607e7497140 0x5607e74995d0 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5979f74f46fadde server_cookie=9335bc620cf0fcdf in_seq=0 out_seq=0 2024-06-10T01:25:54.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.728+0000 7f952b7fe640 10 client.29402 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:54.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.732+0000 7f952b7fe640 1 -- 172.21.15.7:0/4292105972 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5607e73f7f10 con 0x5607e7497140 2024-06-10T01:25:54.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.732+0000 7f952b7fe640 10 client.29402 handle_command_reply: tid=0 2024-06-10T01:25:54.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.732+0000 7f9555639740 1 client.29402 shutdown 2024-06-10T01:25:54.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.732+0000 7f9555639740 1 -- 172.21.15.7:0/4292105972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607e7486b70 msgr2=0x5607e7492800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:54.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.732+0000 7f9555639740 1 --2- 172.21.15.7:0/4292105972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607e7486b70 0x5607e7492800 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9544009fd0 tx=0x7f9544004ab0 comp rx=0 tx=0).stop 2024-06-10T01:25:54.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.732+0000 7f9555639740 1 -- 172.21.15.7:0/4292105972 shutdown_connections 2024-06-10T01:25:54.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.732+0000 7f9555639740 1 --2- 172.21.15.7:0/4292105972 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5607e7497140 0x5607e74995d0 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:54.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.732+0000 7f9555639740 1 --2- 172.21.15.7:0/4292105972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607e7486b70 0x5607e7492800 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:54.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.732+0000 7f9555639740 1 --2- 172.21.15.7:0/4292105972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607e7481a00 0x5607e748b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:54.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.732+0000 7f952b7fe640 0 client.29402 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:54.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.732+0000 7f9555639740 1 --2- 172.21.15.7:0/4292105972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607e7481010 0x5607e748b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:54.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.732+0000 7f9555639740 1 -- 172.21.15.7:0/4292105972 >> 172.21.15.7:0/4292105972 conn(0x5607e747c680 msgr2=0x5607e747b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:54.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.732+0000 7f9555639740 1 -- 172.21.15.7:0/4292105972 shutdown_connections 2024-06-10T01:25:54.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.732+0000 7f9555639740 1 -- 172.21.15.7:0/4292105972 wait complete. 2024-06-10T01:25:54.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.732+0000 7f9555639740 20 client.29402 trim_cache size 0 max 16384 2024-06-10T01:25:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.751+0000 7f9555639740 1 Processor -- start 2024-06-10T01:25:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.752+0000 7f9555639740 1 -- start start 2024-06-10T01:25:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.752+0000 7f9555639740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607e7481010 0x5607e7482be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.752+0000 7f9555639740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607e7481a00 0x5607e7483120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.752+0000 7f9555639740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607e7486b70 0x5607e74927c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.752+0000 7f9555639740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f954c1af0b0 con 0x5607e7481a00 2024-06-10T01:25:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.752+0000 7f9555639740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f954c1ac980 con 0x5607e7481010 2024-06-10T01:25:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.752+0000 7f9555639740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f954c1aa7d0 con 0x5607e7486b70 2024-06-10T01:25:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.752+0000 7f954b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607e7481a00 0x5607e7483120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.752+0000 7f954b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607e7481a00 0x5607e7483120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36576/0 (socket says 172.21.15.7:36576) 2024-06-10T01:25:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.752+0000 7f954b7fe640 1 -- 172.21.15.7:0/3867666149 learned_addr learned my addr 172.21.15.7:0/3867666149 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.752+0000 7f954bfff640 1 --2- 172.21.15.7:0/3867666149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607e7481010 0x5607e7482be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.752+0000 7f9550ec6640 1 --2- 172.21.15.7:0/3867666149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607e7486b70 0x5607e74927c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.752+0000 7f954b7fe640 1 -- 172.21.15.7:0/3867666149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607e7486b70 msgr2=0x5607e74927c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.752+0000 7f954b7fe640 1 --2- 172.21.15.7:0/3867666149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607e7486b70 0x5607e74927c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.752+0000 7f954b7fe640 1 -- 172.21.15.7:0/3867666149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607e7481010 msgr2=0x5607e7482be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.752+0000 7f954b7fe640 1 --2- 172.21.15.7:0/3867666149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607e7481010 0x5607e7482be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.752+0000 7f954b7fe640 1 -- 172.21.15.7:0/3867666149 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607e747bf30 con 0x5607e7481a00 2024-06-10T01:25:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.752+0000 7f9550ec6640 1 --2- 172.21.15.7:0/3867666149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607e7486b70 0x5607e74927c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:25:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.752+0000 7f954b7fe640 1 --2- 172.21.15.7:0/3867666149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607e7481a00 0x5607e7483120 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f95400b7f80 tx=0x7f95400b7e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67cd254ffba0a21 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.753+0000 7f9531ffb640 1 -- 172.21.15.7:0/3867666149 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9540049070 con 0x5607e7481a00 2024-06-10T01:25:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.753+0000 7f9531ffb640 1 -- 172.21.15.7:0/3867666149 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f954003be40 con 0x5607e7481a00 2024-06-10T01:25:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.753+0000 7f9531ffb640 1 -- 172.21.15.7:0/3867666149 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95400ef2d0 con 0x5607e7481a00 2024-06-10T01:25:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.753+0000 7f9555639740 1 -- 172.21.15.7:0/3867666149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607e7481a00 msgr2=0x5607e7483120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.753+0000 7f9555639740 1 --2- 172.21.15.7:0/3867666149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607e7481a00 0x5607e7483120 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f95400b7f80 tx=0x7f95400b7e80 comp rx=0 tx=0).stop 2024-06-10T01:25:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.753+0000 7f9555639740 1 -- 172.21.15.7:0/3867666149 shutdown_connections 2024-06-10T01:25:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.753+0000 7f9555639740 1 --2- 172.21.15.7:0/3867666149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607e7486b70 0x5607e74927c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.753+0000 7f9555639740 1 --2- 172.21.15.7:0/3867666149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607e7481a00 0x5607e7483120 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.754+0000 7f9555639740 1 --2- 172.21.15.7:0/3867666149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607e7481010 0x5607e7482be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.754+0000 7f9555639740 1 -- 172.21.15.7:0/3867666149 >> 172.21.15.7:0/3867666149 conn(0x5607e747c680 msgr2=0x5607e747ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.754+0000 7f9555639740 1 -- 172.21.15.7:0/3867666149 shutdown_connections 2024-06-10T01:25:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.754+0000 7f9555639740 1 -- 172.21.15.7:0/3867666149 wait complete. 2024-06-10T01:25:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.754+0000 7f9555639740 1 -- start start 2024-06-10T01:25:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.754+0000 7f9555639740 1 Processor -- start 2024-06-10T01:25:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.754+0000 7f9555639740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607e7481010 0x5607e748f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.754+0000 7f9555639740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607e7481a00 0x5607e748fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.754+0000 7f9555639740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607e7486b70 0x5607e74ffea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.754+0000 7f9555639740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f954c1aae40 con 0x5607e7481a00 2024-06-10T01:25:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.754+0000 7f9555639740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f954c1ac2b0 con 0x5607e7486b70 2024-06-10T01:25:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.754+0000 7f9555639740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f954c1ab510 con 0x5607e7481010 2024-06-10T01:25:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.754+0000 7f9555639740 10 client.0 _ll_register_callbacks cb 0x7f9544003c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:25:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.755+0000 7f954b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607e7481a00 0x5607e748fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.755+0000 7f954b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607e7481a00 0x5607e748fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36592/0 (socket says 172.21.15.7:36592) 2024-06-10T01:25:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.755+0000 7f954b7fe640 1 -- 172.21.15.7:0/1364827015 learned_addr learned my addr 172.21.15.7:0/1364827015 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.755+0000 7f9550ec6640 1 --2- 172.21.15.7:0/1364827015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607e7486b70 0x5607e74ffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.755+0000 7f954bfff640 1 --2- 172.21.15.7:0/1364827015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607e7481010 0x5607e748f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.755+0000 7f954b7fe640 1 -- 172.21.15.7:0/1364827015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607e7481010 msgr2=0x5607e748f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.755+0000 7f954b7fe640 1 --2- 172.21.15.7:0/1364827015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607e7481010 0x5607e748f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.755+0000 7f954b7fe640 1 -- 172.21.15.7:0/1364827015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607e7486b70 msgr2=0x5607e74ffea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:25:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.755+0000 7f954b7fe640 1 --2- 172.21.15.7:0/1364827015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607e7486b70 0x5607e74ffea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.755+0000 7f954b7fe640 1 -- 172.21.15.7:0/1364827015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607e75003e0 con 0x5607e7481a00 2024-06-10T01:25:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.755+0000 7f954bfff640 1 --2- 172.21.15.7:0/1364827015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607e7481010 0x5607e748f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:25:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.755+0000 7f954b7fe640 1 --2- 172.21.15.7:0/1364827015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607e7481a00 0x5607e748fcd0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f95400ed080 tx=0x7f95400b7ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633c0d83ea47e08b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.755+0000 7f952b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:25:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.755+0000 7f952b7fe640 1 -- 172.21.15.7:0/1364827015 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9540049070 con 0x5607e7481a00 2024-06-10T01:25:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.755+0000 7f952b7fe640 1 -- 172.21.15.7:0/1364827015 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f9540003c80 con 0x5607e7481a00 2024-06-10T01:25:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.755+0000 7f9555639740 1 -- 172.21.15.7:0/1364827015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5607e73f1c20 con 0x5607e7481a00 2024-06-10T01:25:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.756+0000 7f952b7fe640 1 -- 172.21.15.7:0/1364827015 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f95400f2b80 con 0x5607e7481a00 2024-06-10T01:25:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.756+0000 7f9555639740 10 client.20196 fetch_fsmap learned FSMap version 52 2024-06-10T01:25:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.756+0000 7f9555639740 1 -- 172.21.15.7:0/1364827015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5607e75015b0 con 0x5607e7481a00 2024-06-10T01:25:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.756+0000 7f952b7fe640 1 -- 172.21.15.7:0/1364827015 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5607e73f1c20 con 0x5607e7481a00 2024-06-10T01:25:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.756+0000 7f952b7fe640 1 -- 172.21.15.7:0/1364827015 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f954004c060 con 0x5607e7481a00 2024-06-10T01:25:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.756+0000 7f9555639740 10 client.20196 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:25:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.756+0000 7f9555639740 10 client.20196 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:25:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.756+0000 7f9555639740 20 client.20196 populate_metadata read hostname 'smithi007' 2024-06-10T01:25:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.756+0000 7f9555639740 1 --2- 172.21.15.7:0/1364827015 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5607e7501f10 0x5607e7504330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.756+0000 7f9555639740 4 client.20196 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:25:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.756+0000 7f9555639740 1 -- 172.21.15.7:0/1364827015 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5607e73ebf70 con 0x5607e7501f10 2024-06-10T01:25:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.756+0000 7f954bfff640 1 --2- 172.21.15.7:0/1364827015 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5607e7501f10 0x5607e7504330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.758+0000 7f952b7fe640 10 client.20196 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:54.758+0000 7f954bfff640 1 --2- 172.21.15.7:0/1364827015 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5607e7501f10 0x5607e7504330 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5eb2a861cbd03a10 server_cookie=ceeaeeed082fbc3e in_seq=0 out_seq=0 2024-06-10T01:25:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:55.319+0000 7f952b7fe640 1 -- 172.21.15.7:0/1364827015 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5607e73ebf70 con 0x5607e7501f10 2024-06-10T01:25:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:55.319+0000 7f952b7fe640 10 client.20196 handle_command_reply: tid=0 2024-06-10T01:25:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:55.319+0000 7f9555639740 1 client.20196 shutdown 2024-06-10T01:25:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:55.319+0000 7f9555639740 1 -- 172.21.15.7:0/1364827015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607e7481a00 msgr2=0x5607e748fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:55.319+0000 7f9555639740 1 --2- 172.21.15.7:0/1364827015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607e7481a00 0x5607e748fcd0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f95400ed080 tx=0x7f95400b7ef0 comp rx=0 tx=0).stop 2024-06-10T01:25:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:55.320+0000 7f9555639740 1 -- 172.21.15.7:0/1364827015 shutdown_connections 2024-06-10T01:25:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:55.320+0000 7f9555639740 1 --2- 172.21.15.7:0/1364827015 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5607e7501f10 0x5607e7504330 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:55.320+0000 7f9555639740 1 --2- 172.21.15.7:0/1364827015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607e7486b70 0x5607e74ffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:55.320+0000 7f9555639740 1 --2- 172.21.15.7:0/1364827015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607e7481a00 0x5607e748fcd0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:55.320+0000 7f9555639740 1 --2- 172.21.15.7:0/1364827015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607e7481010 0x5607e748f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:55.320+0000 7f9555639740 1 -- 172.21.15.7:0/1364827015 >> 172.21.15.7:0/1364827015 conn(0x5607e747c680 msgr2=0x5607e748b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:55.320+0000 7f952b7fe640 0 client.20196 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:25:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:25:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:25:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:25:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:55.320+0000 7f9555639740 1 -- 172.21.15.7:0/1364827015 shutdown_connections 2024-06-10T01:25:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:55.320+0000 7f9555639740 1 -- 172.21.15.7:0/1364827015 wait complete. 2024-06-10T01:25:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:55.320+0000 7f9555639740 20 client.20196 trim_cache size 0 max 16384 2024-06-10T01:25:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:55.321+0000 7f9552929640 1 -- 172.21.15.7:0/3703735120 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f95340a2b80 msgr2=0x7f95340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:55.321+0000 7f9552929640 1 --2- 172.21.15.7:0/3703735120 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f95340a2b80 0x7f95340a5040 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x5607e7482260 tx=0x7f9544009500 comp rx=0 tx=0).stop 2024-06-10T01:25:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:55.321+0000 7f9552929640 1 -- 172.21.15.7:0/3703735120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f954c104e10 msgr2=0x7f954c100490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:55.321+0000 7f9552929640 1 --2- 172.21.15.7:0/3703735120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f954c104e10 0x7f954c100490 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f9540031610 tx=0x7f9540004270 comp rx=0 tx=0).stop 2024-06-10T01:25:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:55.321+0000 7f9552929640 1 -- 172.21.15.7:0/3703735120 shutdown_connections 2024-06-10T01:25:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:55.321+0000 7f9552929640 1 --2- 172.21.15.7:0/3703735120 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f95340a2b80 0x7f95340a5040 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:55.321+0000 7f9552929640 1 --2- 172.21.15.7:0/3703735120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f954c107740 0x7f954c1009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:55.321+0000 7f9552929640 1 --2- 172.21.15.7:0/3703735120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f954c104e10 0x7f954c100490 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:55.321+0000 7f9552929640 1 --2- 172.21.15.7:0/3703735120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f954c1024e0 0x7f954c101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:55.321+0000 7f9552929640 1 -- 172.21.15.7:0/3703735120 >> 172.21.15.7:0/3703735120 conn(0x7f954c0fbe70 msgr2=0x7f954c102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:55.322+0000 7f9552929640 1 -- 172.21.15.7:0/3703735120 shutdown_connections 2024-06-10T01:25:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:25:55.322+0000 7f9552929640 1 -- 172.21.15.7:0/3703735120 wait complete. 2024-06-10T01:25:55.541 INFO:tasks.workunit.client.0.smithi007.stdout: 10 163724 24.40 MB/sec execute 272 sec latency 418.707 ms 2024-06-10T01:25:55.569 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:25:55.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.598+0000 7f417c8d2700 1 -- 172.21.15.7:0/2594804321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4178108ac0 msgr2=0x7f4178108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:55.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.598+0000 7f417c8d2700 1 --2- 172.21.15.7:0/2594804321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4178108ac0 0x7f4178108f40 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f416c0098f0 tx=0x7f416c009c00 comp rx=0 tx=0).stop 2024-06-10T01:25:55.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.599+0000 7f417c8d2700 1 -- 172.21.15.7:0/2594804321 shutdown_connections 2024-06-10T01:25:55.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.599+0000 7f417c8d2700 1 --2- 172.21.15.7:0/2594804321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4178109480 0x7f4178115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:55.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.599+0000 7f417c8d2700 1 --2- 172.21.15.7:0/2594804321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f417810eac0 0x7f417810eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:55.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.599+0000 7f417c8d2700 1 --2- 172.21.15.7:0/2594804321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4178108ac0 0x7f4178108f40 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:55.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.599+0000 7f417c8d2700 1 -- 172.21.15.7:0/2594804321 >> 172.21.15.7:0/2594804321 conn(0x7f4178075c60 msgr2=0x7f41781072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:55.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.599+0000 7f417c8d2700 1 -- 172.21.15.7:0/2594804321 shutdown_connections 2024-06-10T01:25:55.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.599+0000 7f417c8d2700 1 -- 172.21.15.7:0/2594804321 wait complete. 2024-06-10T01:25:55.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.600+0000 7f417c8d2700 1 Processor -- start 2024-06-10T01:25:55.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.600+0000 7f417c8d2700 1 -- start start 2024-06-10T01:25:55.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.600+0000 7f417c8d2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4178108ac0 0x7f417819f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:55.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.600+0000 7f417c8d2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4178109480 0x7f417819f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:55.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.600+0000 7f417c8d2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f417810eac0 0x7f41781a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:55.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.601+0000 7f417c8d2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4178118e40 con 0x7f417810eac0 2024-06-10T01:25:55.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.601+0000 7f417c8d2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4178118cc0 con 0x7f4178109480 2024-06-10T01:25:55.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.601+0000 7f417c8d2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4178118fc0 con 0x7f4178108ac0 2024-06-10T01:25:55.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.601+0000 7f4176d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f417810eac0 0x7f41781a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:55.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.601+0000 7f417659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4178108ac0 0x7f417819f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:55.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.601+0000 7f417659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4178108ac0 0x7f417819f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:33788/0 (socket says 172.21.15.7:33788) 2024-06-10T01:25:55.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.601+0000 7f4176d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f417810eac0 0x7f41781a3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36616/0 (socket says 172.21.15.7:36616) 2024-06-10T01:25:55.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.601+0000 7f417659c700 1 -- 172.21.15.7:0/3721204743 learned_addr learned my addr 172.21.15.7:0/3721204743 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:55.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.601+0000 7f4175d9b700 1 --2- 172.21.15.7:0/3721204743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4178109480 0x7f417819f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:55.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.601+0000 7f417659c700 1 -- 172.21.15.7:0/3721204743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4178109480 msgr2=0x7f417819f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:55.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.601+0000 7f417659c700 1 --2- 172.21.15.7:0/3721204743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4178109480 0x7f417819f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:55.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.601+0000 7f417659c700 1 -- 172.21.15.7:0/3721204743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f417810eac0 msgr2=0x7f41781a3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:55.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.601+0000 7f417659c700 1 --2- 172.21.15.7:0/3721204743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f417810eac0 0x7f41781a3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:55.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.601+0000 7f417659c700 1 -- 172.21.15.7:0/3721204743 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41781a42b0 con 0x7f4178108ac0 2024-06-10T01:25:55.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.602+0000 7f417659c700 1 --2- 172.21.15.7:0/3721204743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4178108ac0 0x7f417819f2f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f416800e9a0 tx=0x7f416800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:55.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.602+0000 7f41677fe700 1 -- 172.21.15.7:0/3721204743 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416800cbd0 con 0x7f4178108ac0 2024-06-10T01:25:55.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.602+0000 7f41677fe700 1 -- 172.21.15.7:0/3721204743 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f416800cd70 con 0x7f4178108ac0 2024-06-10T01:25:55.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.602+0000 7f41677fe700 1 -- 172.21.15.7:0/3721204743 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4168018750 con 0x7f4178108ac0 2024-06-10T01:25:55.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.602+0000 7f417c8d2700 1 -- 172.21.15.7:0/3721204743 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41781a45a0 con 0x7f4178108ac0 2024-06-10T01:25:55.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.602+0000 7f417c8d2700 1 -- 172.21.15.7:0/3721204743 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41781abe20 con 0x7f4178108ac0 2024-06-10T01:25:55.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.603+0000 7f417c8d2700 1 -- 172.21.15.7:0/3721204743 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4178069180 con 0x7f4178108ac0 2024-06-10T01:25:55.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.605+0000 7f41677fe700 1 -- 172.21.15.7:0/3721204743 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f4168010430 con 0x7f4178108ac0 2024-06-10T01:25:55.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.605+0000 7f41677fe700 1 --2- 172.21.15.7:0/3721204743 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f41600a2ca0 0x7f41600a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:55.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.606+0000 7f41677fe700 1 -- 172.21.15.7:0/3721204743 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f41680cf920 con 0x7f4178108ac0 2024-06-10T01:25:55.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.606+0000 7f4175d9b700 1 --2- 172.21.15.7:0/3721204743 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f41600a2ca0 0x7f41600a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:55.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.606+0000 7f4175d9b700 1 --2- 172.21.15.7:0/3721204743 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f41600a2ca0 0x7f41600a5160 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f416c009fd0 tx=0x7f416c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:55.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.613+0000 7f41677fe700 1 -- 172.21.15.7:0/3721204743 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f416809c500 con 0x7f4178108ac0 2024-06-10T01:25:55.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:55 smithi007 ceph-mon[32398]: pgmap v1030: 129 pgs: 129 active+clean; 646 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 9.4 MiB/s wr, 776 op/s 2024-06-10T01:25:55.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:55 smithi078 ceph-mon[36683]: pgmap v1030: 129 pgs: 129 active+clean; 646 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 9.4 MiB/s wr, 776 op/s 2024-06-10T01:25:55.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.952+0000 7f417c8d2700 1 -- 172.21.15.7:0/3721204743 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f417802dca0 con 0x7f4178108ac0 2024-06-10T01:25:55.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.954+0000 7f41677fe700 1 -- 172.21.15.7:0/3721204743 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11858 (secure 0 0 0) 0x7f416809be00 con 0x7f4178108ac0 2024-06-10T01:25:55.947 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:25:55.947 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":52,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:25:03.837724+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28825":{"gid":28825,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/1972426673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1972426673},{"type":"v1","addr":"172.21.15.175:6835","nonce":1972426673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24451,24434,24436]},"id":1}]} 2024-06-10T01:25:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.956+0000 7f417c8d2700 1 -- 172.21.15.7:0/3721204743 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f41600a2ca0 msgr2=0x7f41600a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.956+0000 7f417c8d2700 1 --2- 172.21.15.7:0/3721204743 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f41600a2ca0 0x7f41600a5160 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f416c009fd0 tx=0x7f416c019040 comp rx=0 tx=0).stop 2024-06-10T01:25:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.956+0000 7f417c8d2700 1 -- 172.21.15.7:0/3721204743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4178108ac0 msgr2=0x7f417819f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.956+0000 7f417c8d2700 1 --2- 172.21.15.7:0/3721204743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4178108ac0 0x7f417819f2f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f416800e9a0 tx=0x7f416800ecb0 comp rx=0 tx=0).stop 2024-06-10T01:25:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.956+0000 7f417c8d2700 1 -- 172.21.15.7:0/3721204743 shutdown_connections 2024-06-10T01:25:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.956+0000 7f417c8d2700 1 --2- 172.21.15.7:0/3721204743 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f41600a2ca0 0x7f41600a5160 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.956+0000 7f417c8d2700 1 --2- 172.21.15.7:0/3721204743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f417810eac0 0x7f41781a3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.956+0000 7f417c8d2700 1 --2- 172.21.15.7:0/3721204743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4178108ac0 0x7f417819f2f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.956+0000 7f417c8d2700 1 --2- 172.21.15.7:0/3721204743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4178109480 0x7f417819f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.956+0000 7f417c8d2700 1 -- 172.21.15.7:0/3721204743 >> 172.21.15.7:0/3721204743 conn(0x7f4178075c60 msgr2=0x7f41780762c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.956+0000 7f417c8d2700 1 -- 172.21.15.7:0/3721204743 shutdown_connections 2024-06-10T01:25:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:55.956+0000 7f417c8d2700 1 -- 172.21.15.7:0/3721204743 wait complete. 2024-06-10T01:25:55.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:55 smithi175 ceph-mon[36636]: pgmap v1030: 129 pgs: 129 active+clean; 646 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 9.4 MiB/s wr, 776 op/s 2024-06-10T01:25:55.951 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 52 2024-06-10T01:25:56.540 INFO:tasks.workunit.client.0.smithi007.stdout: 10 164461 24.46 MB/sec execute 273 sec latency 78.709 ms 2024-06-10T01:25:56.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:56 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3721204743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:25:56.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:56 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3721204743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:25:56.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:56 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3721204743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:25:57.540 INFO:tasks.workunit.client.0.smithi007.stdout: 10 165042 24.48 MB/sec execute 274 sec latency 102.750 ms 2024-06-10T01:25:57.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:57 smithi078 ceph-mon[36683]: pgmap v1031: 129 pgs: 129 active+clean; 656 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 9.6 MiB/s wr, 873 op/s 2024-06-10T01:25:57.936 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:57 smithi007 ceph-mon[32398]: pgmap v1031: 129 pgs: 129 active+clean; 656 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 9.6 MiB/s wr, 873 op/s 2024-06-10T01:25:57.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:57 smithi175 ceph-mon[36636]: pgmap v1031: 129 pgs: 129 active+clean; 656 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 9.6 MiB/s wr, 873 op/s 2024-06-10T01:25:58.338 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:25:58.540 INFO:tasks.workunit.client.0.smithi007.stdout: 10 165649 24.51 MB/sec execute 275 sec latency 100.445 ms 2024-06-10T01:25:58.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.889+0000 7f71bf972700 1 -- 172.21.15.7:0/2882613082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b8101440 msgr2=0x7f71b810f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:58.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.889+0000 7f71bf972700 1 --2- 172.21.15.7:0/2882613082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b8101440 0x7f71b810f330 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009f90 tx=0x7f71b000b330 comp rx=0 tx=0).stop 2024-06-10T01:25:58.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.889+0000 7f71bf972700 1 -- 172.21.15.7:0/2882613082 shutdown_connections 2024-06-10T01:25:58.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.889+0000 7f71bf972700 1 --2- 172.21.15.7:0/2882613082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b8101440 0x7f71b810f330 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:58.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.889+0000 7f71bf972700 1 --2- 172.21.15.7:0/2882613082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71b8108510 0x7f71b81088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:58.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.889+0000 7f71bf972700 1 --2- 172.21.15.7:0/2882613082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71b8100a80 0x7f71b8100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:58.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.889+0000 7f71bf972700 1 -- 172.21.15.7:0/2882613082 >> 172.21.15.7:0/2882613082 conn(0x7f71b8077a00 msgr2=0x7f71b80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:58.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.890+0000 7f71bf972700 1 -- 172.21.15.7:0/2882613082 shutdown_connections 2024-06-10T01:25:58.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.890+0000 7f71bf972700 1 -- 172.21.15.7:0/2882613082 wait complete. 2024-06-10T01:25:58.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.891+0000 7f71bf972700 1 Processor -- start 2024-06-10T01:25:58.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.891+0000 7f71bf972700 1 -- start start 2024-06-10T01:25:58.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.891+0000 7f71bf972700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71b8100a80 0x7f71b819adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:58.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.892+0000 7f71bf972700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b8101440 0x7f71b819b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:58.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.892+0000 7f71bf972700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71b8108510 0x7f71b819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:58.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.892+0000 7f71bd70e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71b8100a80 0x7f71b819adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:58.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.892+0000 7f71bd70e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71b8100a80 0x7f71b819adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:45384/0 (socket says 172.21.15.7:45384) 2024-06-10T01:25:58.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.892+0000 7f71bd70e700 1 -- 172.21.15.7:0/2837710172 learned_addr learned my addr 172.21.15.7:0/2837710172 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:25:58.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.892+0000 7f71bf972700 1 -- 172.21.15.7:0/2837710172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f71b8112770 con 0x7f71b8101440 2024-06-10T01:25:58.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.892+0000 7f71bcf0d700 1 --2- 172.21.15.7:0/2837710172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b8101440 0x7f71b819b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:58.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.892+0000 7f71bdf0f700 1 --2- 172.21.15.7:0/2837710172 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71b8108510 0x7f71b819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:58.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.892+0000 7f71bf972700 1 -- 172.21.15.7:0/2837710172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71b81125f0 con 0x7f71b8100a80 2024-06-10T01:25:58.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.893+0000 7f71bf972700 1 -- 172.21.15.7:0/2837710172 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f71b81128f0 con 0x7f71b8108510 2024-06-10T01:25:58.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.893+0000 7f71bd70e700 1 -- 172.21.15.7:0/2837710172 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71b8108510 msgr2=0x7f71b819f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:58.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.893+0000 7f71bd70e700 1 --2- 172.21.15.7:0/2837710172 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71b8108510 0x7f71b819f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:58.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.893+0000 7f71bd70e700 1 -- 172.21.15.7:0/2837710172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b8101440 msgr2=0x7f71b819b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:58.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.893+0000 7f71bd70e700 1 --2- 172.21.15.7:0/2837710172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b8101440 0x7f71b819b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:58.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.893+0000 7f71bd70e700 1 -- 172.21.15.7:0/2837710172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b819fd70 con 0x7f71b8100a80 2024-06-10T01:25:58.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.893+0000 7f71bcf0d700 1 --2- 172.21.15.7:0/2837710172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b8101440 0x7f71b819b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:25:58.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.894+0000 7f71bd70e700 1 --2- 172.21.15.7:0/2837710172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71b8100a80 0x7f71b819adb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f71b400e9a0 tx=0x7f71b400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:58.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.894+0000 7f71ae7fc700 1 -- 172.21.15.7:0/2837710172 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b400c9c0 con 0x7f71b8100a80 2024-06-10T01:25:58.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.894+0000 7f71ae7fc700 1 -- 172.21.15.7:0/2837710172 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71b400cb60 con 0x7f71b8100a80 2024-06-10T01:25:58.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.894+0000 7f71ae7fc700 1 -- 172.21.15.7:0/2837710172 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b4018660 con 0x7f71b8100a80 2024-06-10T01:25:58.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.895+0000 7f71bf972700 1 -- 172.21.15.7:0/2837710172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71b81a0060 con 0x7f71b8100a80 2024-06-10T01:25:58.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.895+0000 7f71bf972700 1 -- 172.21.15.7:0/2837710172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71b81a78e0 con 0x7f71b8100a80 2024-06-10T01:25:58.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.896+0000 7f71bf972700 1 -- 172.21.15.7:0/2837710172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71b8069180 con 0x7f71b8100a80 2024-06-10T01:25:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.898+0000 7f71ae7fc700 1 -- 172.21.15.7:0/2837710172 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f71b400cd10 con 0x7f71b8100a80 2024-06-10T01:25:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.899+0000 7f71ae7fc700 1 --2- 172.21.15.7:0/2837710172 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f71a40a2df0 0x7f71a40a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:25:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.899+0000 7f71ae7fc700 1 -- 172.21.15.7:0/2837710172 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f71b40cf430 con 0x7f71b8100a80 2024-06-10T01:25:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.900+0000 7f71bcf0d700 1 --2- 172.21.15.7:0/2837710172 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f71a40a2df0 0x7f71a40a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:25:58.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.900+0000 7f71bcf0d700 1 --2- 172.21.15.7:0/2837710172 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f71a40a2df0 0x7f71a40a52b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f71b8103c80 tx=0x7f71a8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:25:58.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:58.906+0000 7f71ae7fc700 1 -- 172.21.15.7:0/2837710172 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f71b409bfb0 con 0x7f71b8100a80 2024-06-10T01:25:58.922 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:25:59.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:59.224+0000 7f71bf972700 1 -- 172.21.15.7:0/2837710172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71b81a7ab0 con 0x7f71b8100a80 2024-06-10T01:25:59.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:59.225+0000 7f71ae7fc700 1 -- 172.21.15.7:0/2837710172 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11858 (secure 0 0 0) 0x7f71b409b8b0 con 0x7f71b8100a80 2024-06-10T01:25:59.218 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:25:59.218 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":52,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:25:03.837724+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28825":{"gid":28825,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/1972426673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1972426673},{"type":"v1","addr":"172.21.15.175:6835","nonce":1972426673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24451,24434,24436]},"id":1}]} 2024-06-10T01:25:59.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:59.227+0000 7f71bf972700 1 -- 172.21.15.7:0/2837710172 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f71a40a2df0 msgr2=0x7f71a40a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:59.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:59.227+0000 7f71bf972700 1 --2- 172.21.15.7:0/2837710172 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f71a40a2df0 0x7f71a40a52b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f71b8103c80 tx=0x7f71a8009310 comp rx=0 tx=0).stop 2024-06-10T01:25:59.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:59.227+0000 7f71bf972700 1 -- 172.21.15.7:0/2837710172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71b8100a80 msgr2=0x7f71b819adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:25:59.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:59.227+0000 7f71bf972700 1 --2- 172.21.15.7:0/2837710172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71b8100a80 0x7f71b819adb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f71b400e9a0 tx=0x7f71b400ecb0 comp rx=0 tx=0).stop 2024-06-10T01:25:59.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:59.227+0000 7f71bf972700 1 -- 172.21.15.7:0/2837710172 shutdown_connections 2024-06-10T01:25:59.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:59.227+0000 7f71bf972700 1 --2- 172.21.15.7:0/2837710172 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f71a40a2df0 0x7f71a40a52b0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:59.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:59.227+0000 7f71bf972700 1 --2- 172.21.15.7:0/2837710172 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71b8108510 0x7f71b819f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:59.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:59.227+0000 7f71bf972700 1 --2- 172.21.15.7:0/2837710172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71b8100a80 0x7f71b819adb0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:59.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:59.227+0000 7f71bf972700 1 --2- 172.21.15.7:0/2837710172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b8101440 0x7f71b819b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:25:59.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:59.227+0000 7f71bf972700 1 -- 172.21.15.7:0/2837710172 >> 172.21.15.7:0/2837710172 conn(0x7f71b8077a00 msgr2=0x7f71b80feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:25:59.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:59.227+0000 7f71bf972700 1 -- 172.21.15.7:0/2837710172 shutdown_connections 2024-06-10T01:25:59.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:25:59.227+0000 7f71bf972700 1 -- 172.21.15.7:0/2837710172 wait complete. 2024-06-10T01:25:59.223 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 52 2024-06-10T01:25:59.540 INFO:tasks.workunit.client.0.smithi007.stdout: 10 166194 24.53 MB/sec execute 276 sec latency 133.668 ms 2024-06-10T01:25:59.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:59 smithi007 ceph-mon[32398]: pgmap v1032: 129 pgs: 129 active+clean; 650 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.1 MiB/s wr, 588 op/s 2024-06-10T01:25:59.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:25:59 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2837710172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:25:59.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:59 smithi078 ceph-mon[36683]: pgmap v1032: 129 pgs: 129 active+clean; 650 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.1 MiB/s wr, 588 op/s 2024-06-10T01:25:59.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:25:59 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2837710172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:25:59.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:59 smithi175 ceph-mon[36636]: pgmap v1032: 129 pgs: 129 active+clean; 650 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.1 MiB/s wr, 588 op/s 2024-06-10T01:25:59.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:25:59 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2837710172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:26:00.540 INFO:tasks.workunit.client.0.smithi007.stdout: 10 166721 24.54 MB/sec execute 277 sec latency 74.466 ms 2024-06-10T01:26:00.892 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db / --timeout 90 --expiration 142.0 --await-for 45.0' 2024-06-10T01:26:00.893 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db / --timeout 90 --expiration 142.0 --await-for 45.0 2024-06-10T01:26:01.541 INFO:tasks.workunit.client.0.smithi007.stdout: 10 167442 24.57 MB/sec execute 278 sec latency 73.094 ms 2024-06-10T01:26:01.626 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:26:01.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.650+0000 7f91cffc0700 1 -- 172.21.15.7:0/977892184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91c8102770 msgr2=0x7f91c8102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:01.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.650+0000 7f91cffc0700 1 --2- 172.21.15.7:0/977892184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91c8102770 0x7f91c8102b50 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f91c40099c0 tx=0x7f91c4009cd0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.644 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:01 smithi007 ceph-mon[32398]: pgmap v1033: 129 pgs: 129 active+clean; 654 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.5 MiB/s wr, 640 op/s 2024-06-10T01:26:01.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.650+0000 7f91cffc0700 1 -- 172.21.15.7:0/977892184 shutdown_connections 2024-06-10T01:26:01.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.650+0000 7f91cffc0700 1 --2- 172.21.15.7:0/977892184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91c8107860 0x7f91c810f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.650+0000 7f91cffc0700 1 --2- 172.21.15.7:0/977892184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91c8102770 0x7f91c8102b50 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.650+0000 7f91cffc0700 1 --2- 172.21.15.7:0/977892184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91c8103120 0x7f91c8107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.650+0000 7f91cffc0700 1 -- 172.21.15.7:0/977892184 >> 172.21.15.7:0/977892184 conn(0x7f91c8079600 msgr2=0x7f91c8079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:01.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.651+0000 7f91cffc0700 1 -- 172.21.15.7:0/977892184 shutdown_connections 2024-06-10T01:26:01.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.651+0000 7f91cffc0700 1 -- 172.21.15.7:0/977892184 wait complete. 2024-06-10T01:26:01.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.652+0000 7f91cffc0700 1 Processor -- start 2024-06-10T01:26:01.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.652+0000 7f91cffc0700 1 -- start start 2024-06-10T01:26:01.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.653+0000 7f91cffc0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91c8102770 0x7f91c8076160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:01.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.653+0000 7f91cffc0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91c8103120 0x7f91c8072200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:01.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.653+0000 7f91cffc0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91c8107860 0x7f91c8072740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:01.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.653+0000 7f91cffc0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f91c806c900 con 0x7f91c8102770 2024-06-10T01:26:01.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.653+0000 7f91cffc0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f91c806c780 con 0x7f91c8107860 2024-06-10T01:26:01.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.653+0000 7f91cffc0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f91c806ca80 con 0x7f91c8103120 2024-06-10T01:26:01.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.653+0000 7f91ce55d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91c8107860 0x7f91c8072740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:01.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.653+0000 7f91ce55d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91c8107860 0x7f91c8072740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:45408/0 (socket says 172.21.15.7:45408) 2024-06-10T01:26:01.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.653+0000 7f91ce55d700 1 -- 172.21.15.7:0/445112155 learned_addr learned my addr 172.21.15.7:0/445112155 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:01.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.654+0000 7f91cd55b700 1 --2- 172.21.15.7:0/445112155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91c8103120 0x7f91c8072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:01.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.654+0000 7f91ce55d700 1 -- 172.21.15.7:0/445112155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91c8103120 msgr2=0x7f91c8072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:01.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.654+0000 7f91ce55d700 1 --2- 172.21.15.7:0/445112155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91c8103120 0x7f91c8072200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.654+0000 7f91ce55d700 1 -- 172.21.15.7:0/445112155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91c8102770 msgr2=0x7f91c8076160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:01.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.654+0000 7f91ce55d700 1 --2- 172.21.15.7:0/445112155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91c8102770 0x7f91c8076160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.654+0000 7f91ce55d700 1 -- 172.21.15.7:0/445112155 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c8072ff0 con 0x7f91c8107860 2024-06-10T01:26:01.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.654+0000 7f91cd55b700 1 --2- 172.21.15.7:0/445112155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91c8103120 0x7f91c8072200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:26:01.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.655+0000 7f91ce55d700 1 --2- 172.21.15.7:0/445112155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91c8107860 0x7f91c8072740 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f91c000ef30 tx=0x7f91c000c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:01.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.655+0000 7f91beffd700 1 -- 172.21.15.7:0/445112155 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c00092e0 con 0x7f91c8107860 2024-06-10T01:26:01.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.655+0000 7f91beffd700 1 -- 172.21.15.7:0/445112155 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91c0004bb0 con 0x7f91c8107860 2024-06-10T01:26:01.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.655+0000 7f91cffc0700 1 -- 172.21.15.7:0/445112155 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91c81afb60 con 0x7f91c8107860 2024-06-10T01:26:01.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.655+0000 7f91beffd700 1 -- 172.21.15.7:0/445112155 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0004390 con 0x7f91c8107860 2024-06-10T01:26:01.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.655+0000 7f91cffc0700 1 -- 172.21.15.7:0/445112155 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91c81b0070 con 0x7f91c8107860 2024-06-10T01:26:01.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.659+0000 7f91beffd700 1 -- 172.21.15.7:0/445112155 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f91c001b030 con 0x7f91c8107860 2024-06-10T01:26:01.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.661+0000 7f91beffd700 1 --2- 172.21.15.7:0/445112155 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f91b40a2d20 0x7f91b40a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:01.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.661+0000 7f91beffd700 1 -- 172.21.15.7:0/445112155 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f91c00cef10 con 0x7f91c8107860 2024-06-10T01:26:01.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.661+0000 7f91cdd5c700 1 --2- 172.21.15.7:0/445112155 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f91b40a2d20 0x7f91b40a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:01.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.661+0000 7f91da843b80 1 Processor -- start 2024-06-10T01:26:01.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.661+0000 7f91cdd5c700 1 --2- 172.21.15.7:0/445112155 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f91b40a2d20 0x7f91b40a51e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f91c40098e0 tx=0x7f91c4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:01.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.662+0000 7f91da843b80 1 -- start start 2024-06-10T01:26:01.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.662+0000 7f91da843b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb5d384640 0x55bb5d384a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:01.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.662+0000 7f91da843b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb5d384f60 0x55bb5d37e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:01.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.662+0000 7f91da843b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb5d37ef20 0x55bb5d37f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:01.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.662+0000 7f91da843b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bb5d2d0810 con 0x55bb5d384f60 2024-06-10T01:26:01.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.662+0000 7f91da843b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bb5d278720 con 0x55bb5d37ef20 2024-06-10T01:26:01.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.662+0000 7f91da843b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bb5d2cc7d0 con 0x55bb5d384640 2024-06-10T01:26:01.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.662+0000 7f91ce55d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb5d37ef20 0x55bb5d37f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:01.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.662+0000 7f91cdd5c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb5d384640 0x55bb5d384a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:01.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.662+0000 7f91ce55d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb5d37ef20 0x55bb5d37f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:45418/0 (socket says 172.21.15.7:45418) 2024-06-10T01:26:01.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.662+0000 7f91ce55d700 1 -- 172.21.15.7:0/731718779 learned_addr learned my addr 172.21.15.7:0/731718779 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.663+0000 7f91cd55b700 1 --2- 172.21.15.7:0/731718779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb5d384f60 0x55bb5d37e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.663+0000 7f91ce55d700 1 -- 172.21.15.7:0/731718779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb5d384640 msgr2=0x55bb5d384a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.663+0000 7f91ce55d700 1 --2- 172.21.15.7:0/731718779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb5d384640 0x55bb5d384a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.663+0000 7f91ce55d700 1 -- 172.21.15.7:0/731718779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb5d384f60 msgr2=0x55bb5d37e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.663+0000 7f91ce55d700 1 --2- 172.21.15.7:0/731718779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb5d384f60 0x55bb5d37e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.663+0000 7f91ce55d700 1 -- 172.21.15.7:0/731718779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb5d26ef50 con 0x55bb5d37ef20 2024-06-10T01:26:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.663+0000 7f91ce55d700 1 --2- 172.21.15.7:0/731718779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb5d37ef20 0x55bb5d37f340 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f91c00a3ad0 tx=0x7f91c009b270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcad140cdb4de59b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.663+0000 7f91b3fff700 1 -- 172.21.15.7:0/731718779 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c009ad80 con 0x55bb5d37ef20 2024-06-10T01:26:01.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.663+0000 7f91b3fff700 1 -- 172.21.15.7:0/731718779 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91c009bb20 con 0x55bb5d37ef20 2024-06-10T01:26:01.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.663+0000 7f91b3fff700 1 -- 172.21.15.7:0/731718779 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0012340 con 0x55bb5d37ef20 2024-06-10T01:26:01.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.665+0000 7f91da843b80 1 -- 172.21.15.7:0/731718779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb5d37ef20 msgr2=0x55bb5d37f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:01.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.665+0000 7f91da843b80 1 --2- 172.21.15.7:0/731718779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb5d37ef20 0x55bb5d37f340 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f91c00a3ad0 tx=0x7f91c009b270 comp rx=0 tx=0).stop 2024-06-10T01:26:01.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.665+0000 7f91da843b80 1 -- 172.21.15.7:0/731718779 shutdown_connections 2024-06-10T01:26:01.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.665+0000 7f91da843b80 1 --2- 172.21.15.7:0/731718779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb5d37ef20 0x55bb5d37f340 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.665+0000 7f91da843b80 1 --2- 172.21.15.7:0/731718779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb5d384640 0x55bb5d384a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.665+0000 7f91da843b80 1 --2- 172.21.15.7:0/731718779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb5d384f60 0x55bb5d37e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.665+0000 7f91da843b80 1 -- 172.21.15.7:0/731718779 >> 172.21.15.7:0/731718779 conn(0x55bb5d2a9cc0 msgr2=0x55bb5d379850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:01.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.666+0000 7f91da843b80 1 -- 172.21.15.7:0/731718779 shutdown_connections 2024-06-10T01:26:01.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.666+0000 7f91da843b80 1 -- 172.21.15.7:0/731718779 wait complete. 2024-06-10T01:26:01.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.666+0000 7f91da843b80 1 -- start start 2024-06-10T01:26:01.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.667+0000 7f91da843b80 1 Processor -- start 2024-06-10T01:26:01.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.667+0000 7f91da843b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb5d37ef20 0x55bb5d38acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:01.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.667+0000 7f91da843b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb5d384640 0x55bb5d387ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:01.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.668+0000 7f91da843b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb5d384f60 0x55bb5d388530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:01.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.668+0000 7f91da843b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bb5d38b9d0 con 0x55bb5d37ef20 2024-06-10T01:26:01.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.668+0000 7f91da843b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bb5d270bf0 con 0x55bb5d384f60 2024-06-10T01:26:01.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.668+0000 7f91da843b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bb5d38bb50 con 0x55bb5d384640 2024-06-10T01:26:01.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.668+0000 7f91da843b80 10 client.0 _ll_register_callbacks cb 0x55bb5d26a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:26:01.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.668+0000 7f91ce55d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb5d384f60 0x55bb5d388530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:01.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.668+0000 7f91ce55d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb5d384f60 0x55bb5d388530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:45420/0 (socket says 172.21.15.7:45420) 2024-06-10T01:26:01.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.668+0000 7f91ce55d700 1 -- 172.21.15.7:0/2590720175 learned_addr learned my addr 172.21.15.7:0/2590720175 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:01.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.668+0000 7f91cd55b700 1 --2- 172.21.15.7:0/2590720175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb5d384640 0x55bb5d387ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:01.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.669+0000 7f91cdd5c700 1 --2- 172.21.15.7:0/2590720175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb5d37ef20 0x55bb5d38acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:01.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.669+0000 7f91cdd5c700 1 -- 172.21.15.7:0/2590720175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb5d384640 msgr2=0x55bb5d387ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:01.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.669+0000 7f91cdd5c700 1 --2- 172.21.15.7:0/2590720175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb5d384640 0x55bb5d387ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.669+0000 7f91cdd5c700 1 -- 172.21.15.7:0/2590720175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb5d384f60 msgr2=0x55bb5d388530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:01.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.670+0000 7f91cdd5c700 1 --2- 172.21.15.7:0/2590720175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb5d384f60 0x55bb5d388530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.670+0000 7f91cdd5c700 1 -- 172.21.15.7:0/2590720175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb5d21f8d0 con 0x55bb5d37ef20 2024-06-10T01:26:01.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.671+0000 7f91cdd5c700 1 --2- 172.21.15.7:0/2590720175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb5d37ef20 0x55bb5d38acf0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f91c4003800 tx=0x7f91c400b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce49a8dbd2b6c7af server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:01.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.671+0000 7f91b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:26:01.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.672+0000 7f91b17fa700 1 -- 172.21.15.7:0/2590720175 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c4034060 con 0x55bb5d37ef20 2024-06-10T01:26:01.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.672+0000 7f91b17fa700 1 -- 172.21.15.7:0/2590720175 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f91c400b9a0 con 0x55bb5d37ef20 2024-06-10T01:26:01.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.672+0000 7f91da843b80 1 -- 172.21.15.7:0/2590720175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91c009bb20 con 0x55bb5d37ef20 2024-06-10T01:26:01.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.672+0000 7f91b17fa700 1 -- 172.21.15.7:0/2590720175 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f91c40095b0 con 0x55bb5d37ef20 2024-06-10T01:26:01.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.672+0000 7f91da843b80 10 client.20217 fetch_fsmap learned FSMap version 52 2024-06-10T01:26:01.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.672+0000 7f91da843b80 1 -- 172.21.15.7:0/2590720175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55bb5d3962d0 con 0x55bb5d37ef20 2024-06-10T01:26:01.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.672+0000 7f91b17fa700 1 -- 172.21.15.7:0/2590720175 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91c009bb20 con 0x55bb5d37ef20 2024-06-10T01:26:01.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.673+0000 7f91b17fa700 1 -- 172.21.15.7:0/2590720175 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f91c40213f0 con 0x55bb5d37ef20 2024-06-10T01:26:01.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.673+0000 7f91da843b80 10 client.20217 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:26:01.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.673+0000 7f91da843b80 10 client.20217 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:26:01.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.673+0000 7f91da843b80 20 client.20217 populate_metadata read hostname 'smithi007' 2024-06-10T01:26:01.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.673+0000 7f91da843b80 1 --2- 172.21.15.7:0/2590720175 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55bb5d398910 0x55bb5d39ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:01.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.673+0000 7f91da843b80 4 client.20217 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:26:01.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.673+0000 7f91da843b80 1 -- 172.21.15.7:0/2590720175 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb5d2ddd10 con 0x55bb5d398910 2024-06-10T01:26:01.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.674+0000 7f91cd55b700 1 --2- 172.21.15.7:0/2590720175 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55bb5d398910 0x55bb5d39ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:01.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.674+0000 7f91b17fa700 10 client.20217 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:26:01.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.674+0000 7f91cd55b700 1 --2- 172.21.15.7:0/2590720175 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55bb5d398910 0x55bb5d39ad90 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=faea7dc006256d03 server_cookie=1b2cf7fadde57d15 in_seq=0 out_seq=0 2024-06-10T01:26:01.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.676+0000 7f91b17fa700 1 -- 172.21.15.7:0/2590720175 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bb5d2ddd10 con 0x55bb5d398910 2024-06-10T01:26:01.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.676+0000 7f91b17fa700 10 client.20217 handle_command_reply: tid=0 2024-06-10T01:26:01.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.676+0000 7f91da843b80 1 client.20217 shutdown 2024-06-10T01:26:01.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.677+0000 7f91da843b80 1 -- 172.21.15.7:0/2590720175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb5d37ef20 msgr2=0x55bb5d38acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:01.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.677+0000 7f91da843b80 1 --2- 172.21.15.7:0/2590720175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb5d37ef20 0x55bb5d38acf0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f91c4003800 tx=0x7f91c400b540 comp rx=0 tx=0).stop 2024-06-10T01:26:01.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.677+0000 7f91da843b80 1 -- 172.21.15.7:0/2590720175 shutdown_connections 2024-06-10T01:26:01.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.677+0000 7f91da843b80 1 --2- 172.21.15.7:0/2590720175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb5d384f60 0x55bb5d388530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.677+0000 7f91da843b80 1 --2- 172.21.15.7:0/2590720175 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55bb5d398910 0x55bb5d39ad90 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.677+0000 7f91da843b80 1 --2- 172.21.15.7:0/2590720175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb5d37ef20 0x55bb5d38acf0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.677+0000 7f91da843b80 1 --2- 172.21.15.7:0/2590720175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb5d384640 0x55bb5d387ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.677+0000 7f91da843b80 1 -- 172.21.15.7:0/2590720175 >> 172.21.15.7:0/2590720175 conn(0x55bb5d2a9cc0 msgr2=0x55bb5d37d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:01.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.677+0000 7f91b17fa700 0 client.20217 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:26:01.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.677+0000 7f91da843b80 1 -- 172.21.15.7:0/2590720175 shutdown_connections 2024-06-10T01:26:01.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.677+0000 7f91da843b80 1 -- 172.21.15.7:0/2590720175 wait complete. 2024-06-10T01:26:01.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.677+0000 7f91da843b80 20 client.20217 trim_cache size 0 max 16384 2024-06-10T01:26:01.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.701+0000 7f91da843b80 1 Processor -- start 2024-06-10T01:26:01.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.702+0000 7f91da843b80 1 -- start start 2024-06-10T01:26:01.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.702+0000 7f91da843b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb5d37ef20 0x55bb5d37e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:01.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.702+0000 7f91da843b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb5d384640 0x55bb5d394c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:01.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.702+0000 7f91da843b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb5d384f60 0x55bb5d373510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:01.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.702+0000 7f91da843b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f91c81bb530 con 0x55bb5d384640 2024-06-10T01:26:01.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.702+0000 7f91da843b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f91c81b8dc0 con 0x55bb5d384f60 2024-06-10T01:26:01.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.702+0000 7f91da843b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f91c81b6c10 con 0x55bb5d37ef20 2024-06-10T01:26:01.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.702+0000 7f91cdd5c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb5d37ef20 0x55bb5d37e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:01.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.702+0000 7f91cdd5c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb5d37ef20 0x55bb5d37e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:33846/0 (socket says 172.21.15.7:33846) 2024-06-10T01:26:01.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.702+0000 7f91cdd5c700 1 -- 172.21.15.7:0/2751424560 learned_addr learned my addr 172.21.15.7:0/2751424560 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:01.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.702+0000 7f91cd55b700 1 --2- 172.21.15.7:0/2751424560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb5d384640 0x55bb5d394c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:01.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.702+0000 7f91ce55d700 1 --2- 172.21.15.7:0/2751424560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb5d384f60 0x55bb5d373510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:01.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.703+0000 7f91cdd5c700 1 -- 172.21.15.7:0/2751424560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb5d384f60 msgr2=0x55bb5d373510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:01.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.703+0000 7f91cdd5c700 1 --2- 172.21.15.7:0/2751424560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb5d384f60 0x55bb5d373510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.703+0000 7f91cdd5c700 1 -- 172.21.15.7:0/2751424560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb5d384640 msgr2=0x55bb5d394c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:01.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.703+0000 7f91cdd5c700 1 --2- 172.21.15.7:0/2751424560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb5d384640 0x55bb5d394c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.703+0000 7f91cdd5c700 1 -- 172.21.15.7:0/2751424560 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb5d38a930 con 0x55bb5d37ef20 2024-06-10T01:26:01.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.703+0000 7f91cdd5c700 1 --2- 172.21.15.7:0/2751424560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb5d37ef20 0x55bb5d37e9e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f91c4009580 tx=0x7f91c4021570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3755159788a58059 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:01.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.703+0000 7f91b37fe700 1 -- 172.21.15.7:0/2751424560 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c4034070 con 0x55bb5d37ef20 2024-06-10T01:26:01.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.703+0000 7f91b37fe700 1 -- 172.21.15.7:0/2751424560 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91c400be80 con 0x55bb5d37ef20 2024-06-10T01:26:01.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.703+0000 7f91b37fe700 1 -- 172.21.15.7:0/2751424560 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c402f960 con 0x55bb5d37ef20 2024-06-10T01:26:01.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.704+0000 7f91da843b80 1 -- 172.21.15.7:0/2751424560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb5d37ef20 msgr2=0x55bb5d37e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:01.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.704+0000 7f91da843b80 1 --2- 172.21.15.7:0/2751424560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb5d37ef20 0x55bb5d37e9e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f91c4009580 tx=0x7f91c4021570 comp rx=0 tx=0).stop 2024-06-10T01:26:01.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.704+0000 7f91da843b80 1 -- 172.21.15.7:0/2751424560 shutdown_connections 2024-06-10T01:26:01.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.704+0000 7f91da843b80 1 --2- 172.21.15.7:0/2751424560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb5d384f60 0x55bb5d373510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.704+0000 7f91da843b80 1 --2- 172.21.15.7:0/2751424560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb5d37ef20 0x55bb5d37e9e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.704+0000 7f91da843b80 1 --2- 172.21.15.7:0/2751424560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb5d384640 0x55bb5d394c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.704+0000 7f91da843b80 1 -- 172.21.15.7:0/2751424560 >> 172.21.15.7:0/2751424560 conn(0x55bb5d2a9cc0 msgr2=0x55bb5d372e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:01.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.705+0000 7f91da843b80 1 -- 172.21.15.7:0/2751424560 shutdown_connections 2024-06-10T01:26:01.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.705+0000 7f91da843b80 1 -- 172.21.15.7:0/2751424560 wait complete. 2024-06-10T01:26:01.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.705+0000 7f91da843b80 1 -- start start 2024-06-10T01:26:01.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.705+0000 7f91da843b80 1 Processor -- start 2024-06-10T01:26:01.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.706+0000 7f91da843b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb5d37ef20 0x55bb5d391320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:01.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.706+0000 7f91da843b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb5d384640 0x55bb5d39fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:01.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.706+0000 7f91da843b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb5d384f60 0x55bb5d3a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:01.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.706+0000 7f91cdd5c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb5d37ef20 0x55bb5d391320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:01.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.706+0000 7f91cdd5c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb5d37ef20 0x55bb5d391320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:45442/0 (socket says 172.21.15.7:45442) 2024-06-10T01:26:01.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.706+0000 7f91cd55b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb5d384640 0x55bb5d39fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:01.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.706+0000 7f91cd55b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb5d384640 0x55bb5d39fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36678/0 (socket says 172.21.15.7:36678) 2024-06-10T01:26:01.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.706+0000 7f91cdd5c700 1 -- 172.21.15.7:0/106993173 learned_addr learned my addr 172.21.15.7:0/106993173 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:01.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.706+0000 7f91da843b80 1 -- 172.21.15.7:0/106993173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f91c81b7280 con 0x55bb5d384640 2024-06-10T01:26:01.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.706+0000 7f91da843b80 1 -- 172.21.15.7:0/106993173 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f91c81b86f0 con 0x55bb5d37ef20 2024-06-10T01:26:01.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.706+0000 7f91da843b80 1 -- 172.21.15.7:0/106993173 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f91c81b7950 con 0x55bb5d384f60 2024-06-10T01:26:01.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.706+0000 7f91ce55d700 1 --2- 172.21.15.7:0/106993173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb5d384f60 0x55bb5d3a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:01.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.706+0000 7f91da843b80 10 client.0 _ll_register_callbacks cb 0x55bb5d26a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:26:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.706+0000 7f91cd55b700 1 -- 172.21.15.7:0/106993173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb5d384f60 msgr2=0x55bb5d3a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.706+0000 7f91cd55b700 1 --2- 172.21.15.7:0/106993173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb5d384f60 0x55bb5d3a02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.706+0000 7f91cd55b700 1 -- 172.21.15.7:0/106993173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb5d37ef20 msgr2=0x55bb5d391320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.706+0000 7f91cd55b700 1 --2- 172.21.15.7:0/106993173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb5d37ef20 0x55bb5d391320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.706+0000 7f91cd55b700 1 -- 172.21.15.7:0/106993173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb5d3a0880 con 0x55bb5d384640 2024-06-10T01:26:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.707+0000 7f91cd55b700 1 --2- 172.21.15.7:0/106993173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb5d384640 0x55bb5d39fd80 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x55bb5d39b5d0 tx=0x7f91b8014750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29b00d0f23cfa67 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.707+0000 7f91b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:26:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.707+0000 7f91b17fa700 1 -- 172.21.15.7:0/106993173 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b80142d0 con 0x55bb5d384640 2024-06-10T01:26:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.707+0000 7f91b17fa700 1 -- 172.21.15.7:0/106993173 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f91b8028940 con 0x55bb5d384640 2024-06-10T01:26:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.707+0000 7f91da843b80 1 -- 172.21.15.7:0/106993173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bb5d2d8290 con 0x55bb5d384640 2024-06-10T01:26:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.707+0000 7f91b17fa700 1 -- 172.21.15.7:0/106993173 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f91b8028b40 con 0x55bb5d384640 2024-06-10T01:26:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.707+0000 7f91da843b80 10 client.20223 fetch_fsmap learned FSMap version 52 2024-06-10T01:26:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.707+0000 7f91da843b80 1 -- 172.21.15.7:0/106993173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55bb5d403430 con 0x55bb5d384640 2024-06-10T01:26:01.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.708+0000 7f91b17fa700 1 -- 172.21.15.7:0/106993173 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55bb5d2d8290 con 0x55bb5d384640 2024-06-10T01:26:01.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.708+0000 7f91b17fa700 1 -- 172.21.15.7:0/106993173 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f91b8003c80 con 0x55bb5d384640 2024-06-10T01:26:01.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.708+0000 7f91da843b80 10 client.20223 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:26:01.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.708+0000 7f91da843b80 10 client.20223 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:26:01.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.708+0000 7f91da843b80 20 client.20223 populate_metadata read hostname 'smithi007' 2024-06-10T01:26:01.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.708+0000 7f91da843b80 1 --2- 172.21.15.7:0/106993173 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55bb5d403b50 0x55bb5d405f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:01.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.708+0000 7f91da843b80 4 client.20223 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:26:01.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.708+0000 7f91da843b80 1 -- 172.21.15.7:0/106993173 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bb5d152250 con 0x55bb5d403b50 2024-06-10T01:26:01.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.709+0000 7f91cdd5c700 1 --2- 172.21.15.7:0/106993173 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55bb5d403b50 0x55bb5d405f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:01.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.709+0000 7f91cdd5c700 1 --2- 172.21.15.7:0/106993173 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55bb5d403b50 0x55bb5d405f70 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=33eb7d45832d7f25 server_cookie=690c6726365ff6fb in_seq=0 out_seq=0 2024-06-10T01:26:01.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.709+0000 7f91b17fa700 10 client.20223 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:26:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.713+0000 7f91b17fa700 1 -- 172.21.15.7:0/106993173 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bb5d152250 con 0x55bb5d403b50 2024-06-10T01:26:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.713+0000 7f91b17fa700 10 client.20223 handle_command_reply: tid=0 2024-06-10T01:26:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.713+0000 7f91da843b80 1 client.20223 shutdown 2024-06-10T01:26:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.714+0000 7f91da843b80 1 -- 172.21.15.7:0/106993173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb5d384640 msgr2=0x55bb5d39fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.714+0000 7f91da843b80 1 --2- 172.21.15.7:0/106993173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb5d384640 0x55bb5d39fd80 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x55bb5d39b5d0 tx=0x7f91b8014750 comp rx=0 tx=0).stop 2024-06-10T01:26:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.714+0000 7f91da843b80 1 -- 172.21.15.7:0/106993173 shutdown_connections 2024-06-10T01:26:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.714+0000 7f91da843b80 1 --2- 172.21.15.7:0/106993173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb5d384f60 0x55bb5d3a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.714+0000 7f91da843b80 1 --2- 172.21.15.7:0/106993173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb5d37ef20 0x55bb5d391320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.714+0000 7f91da843b80 1 --2- 172.21.15.7:0/106993173 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55bb5d403b50 0x55bb5d405f70 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.714+0000 7f91da843b80 1 --2- 172.21.15.7:0/106993173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb5d384640 0x55bb5d39fd80 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.714+0000 7f91b17fa700 0 client.20223 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:26:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.714+0000 7f91da843b80 1 -- 172.21.15.7:0/106993173 >> 172.21.15.7:0/106993173 conn(0x55bb5d2a9cc0 msgr2=0x55bb5d385340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.714+0000 7f91da843b80 1 -- 172.21.15.7:0/106993173 shutdown_connections 2024-06-10T01:26:01.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.714+0000 7f91da843b80 1 -- 172.21.15.7:0/106993173 wait complete. 2024-06-10T01:26:01.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.714+0000 7f91da843b80 20 client.20223 trim_cache size 0 max 16384 2024-06-10T01:26:01.708 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:26:01.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.715+0000 7f91cffc0700 1 -- 172.21.15.7:0/445112155 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f91b40a2d20 msgr2=0x7f91b40a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:01.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.715+0000 7f91cffc0700 1 --2- 172.21.15.7:0/445112155 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f91b40a2d20 0x7f91b40a51e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f91c40098e0 tx=0x7f91c4000bc0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.715+0000 7f91cffc0700 1 -- 172.21.15.7:0/445112155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91c8107860 msgr2=0x7f91c8072740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:01.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.715+0000 7f91cffc0700 1 --2- 172.21.15.7:0/445112155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91c8107860 0x7f91c8072740 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f91c000ef30 tx=0x7f91c000c450 comp rx=0 tx=0).stop 2024-06-10T01:26:01.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.715+0000 7f91cffc0700 1 -- 172.21.15.7:0/445112155 shutdown_connections 2024-06-10T01:26:01.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.715+0000 7f91cffc0700 1 --2- 172.21.15.7:0/445112155 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f91b40a2d20 0x7f91b40a51e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.715+0000 7f91cffc0700 1 --2- 172.21.15.7:0/445112155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91c8107860 0x7f91c8072740 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.715+0000 7f91cffc0700 1 --2- 172.21.15.7:0/445112155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91c8102770 0x7f91c8076160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.715+0000 7f91cffc0700 1 --2- 172.21.15.7:0/445112155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91c8103120 0x7f91c8072200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:01.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.715+0000 7f91cffc0700 1 -- 172.21.15.7:0/445112155 >> 172.21.15.7:0/445112155 conn(0x7f91c8079600 msgr2=0x7f91c8100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:01.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.715+0000 7f91cffc0700 1 -- 172.21.15.7:0/445112155 shutdown_connections 2024-06-10T01:26:01.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:01.715+0000 7f91cffc0700 1 -- 172.21.15.7:0/445112155 wait complete. 2024-06-10T01:26:01.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:01 smithi078 ceph-mon[36683]: pgmap v1033: 129 pgs: 129 active+clean; 654 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.5 MiB/s wr, 640 op/s 2024-06-10T01:26:01.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:01 smithi175 ceph-mon[36636]: pgmap v1033: 129 pgs: 129 active+clean; 654 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.5 MiB/s wr, 640 op/s 2024-06-10T01:26:02.541 INFO:tasks.workunit.client.0.smithi007.stdout: 10 168430 24.64 MB/sec execute 279 sec latency 50.310 ms 2024-06-10T01:26:02.720 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:26:03.541 INFO:tasks.workunit.client.0.smithi007.stdout: 10 169264 24.80 MB/sec execute 280 sec latency 53.024 ms 2024-06-10T01:26:03.729 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:03 smithi007 ceph-mon[32398]: pgmap v1034: 129 pgs: 129 active+clean; 661 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.6 MiB/s wr, 564 op/s 2024-06-10T01:26:03.791 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:26:03.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:03 smithi078 ceph-mon[36683]: pgmap v1034: 129 pgs: 129 active+clean; 661 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.6 MiB/s wr, 564 op/s 2024-06-10T01:26:03.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:03 smithi175 ceph-mon[36636]: pgmap v1034: 129 pgs: 129 active+clean; 661 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.6 MiB/s wr, 564 op/s 2024-06-10T01:26:04.541 INFO:tasks.workunit.client.0.smithi007.stdout: 10 169809 24.81 MB/sec execute 281 sec latency 85.512 ms 2024-06-10T01:26:05.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.528+0000 7fb7e43ad700 1 -- 172.21.15.7:0/1572391982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7dc109480 msgr2=0x7fb7dc115990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:05.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.528+0000 7fb7e43ad700 1 --2- 172.21.15.7:0/1572391982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7dc109480 0x7fb7dc115990 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb7d400b0a0 tx=0x7fb7d400b3b0 comp rx=0 tx=0).stop 2024-06-10T01:26:05.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.528+0000 7fb7e43ad700 1 -- 172.21.15.7:0/1572391982 shutdown_connections 2024-06-10T01:26:05.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.528+0000 7fb7e43ad700 1 --2- 172.21.15.7:0/1572391982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7dc109480 0x7fb7dc115990 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:05.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.528+0000 7fb7e43ad700 1 --2- 172.21.15.7:0/1572391982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7dc10eac0 0x7fb7dc10eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:05.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.528+0000 7fb7e43ad700 1 --2- 172.21.15.7:0/1572391982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7dc108ac0 0x7fb7dc108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:05.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.528+0000 7fb7e43ad700 1 -- 172.21.15.7:0/1572391982 >> 172.21.15.7:0/1572391982 conn(0x7fb7dc075c60 msgr2=0x7fb7dc078080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:05.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.529+0000 7fb7e43ad700 1 -- 172.21.15.7:0/1572391982 shutdown_connections 2024-06-10T01:26:05.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.529+0000 7fb7e43ad700 1 -- 172.21.15.7:0/1572391982 wait complete. 2024-06-10T01:26:05.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.530+0000 7fb7e43ad700 1 Processor -- start 2024-06-10T01:26:05.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.530+0000 7fb7e43ad700 1 -- start start 2024-06-10T01:26:05.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.530+0000 7fb7e43ad700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7dc108ac0 0x7fb7dc073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:05.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.530+0000 7fb7e43ad700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7dc109480 0x7fb7dc073ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:05.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.530+0000 7fb7e43ad700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7dc10eac0 0x7fb7dc06cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:05.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.531+0000 7fb7e43ad700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb7dc118ea0 con 0x7fb7dc109480 2024-06-10T01:26:05.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.531+0000 7fb7e43ad700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb7dc118d20 con 0x7fb7dc108ac0 2024-06-10T01:26:05.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.531+0000 7fb7e43ad700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb7dc119020 con 0x7fb7dc10eac0 2024-06-10T01:26:05.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.531+0000 7fb7e1948700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7dc109480 0x7fb7dc073ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:05.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.531+0000 7fb7e2149700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7dc108ac0 0x7fb7dc073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:05.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.531+0000 7fb7e2149700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7dc108ac0 0x7fb7dc073ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:33828/0 (socket says 172.21.15.7:33828) 2024-06-10T01:26:05.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.531+0000 7fb7e2149700 1 -- 172.21.15.7:0/1856152466 learned_addr learned my addr 172.21.15.7:0/1856152466 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:05.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.531+0000 7fb7e294a700 1 --2- 172.21.15.7:0/1856152466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7dc10eac0 0x7fb7dc06cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:05.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.531+0000 7fb7e2149700 1 -- 172.21.15.7:0/1856152466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7dc10eac0 msgr2=0x7fb7dc06cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:05.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.531+0000 7fb7e2149700 1 --2- 172.21.15.7:0/1856152466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7dc10eac0 0x7fb7dc06cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:05.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.531+0000 7fb7e2149700 1 -- 172.21.15.7:0/1856152466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7dc109480 msgr2=0x7fb7dc073ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:05.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.531+0000 7fb7e2149700 1 --2- 172.21.15.7:0/1856152466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7dc109480 0x7fb7dc073ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:05.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.531+0000 7fb7e2149700 1 -- 172.21.15.7:0/1856152466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7dc06d4c0 con 0x7fb7dc108ac0 2024-06-10T01:26:05.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.532+0000 7fb7e2149700 1 --2- 172.21.15.7:0/1856152466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7dc108ac0 0x7fb7dc073ab0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb7d800b530 tx=0x7fb7d800b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:05.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.532+0000 7fb7d37fe700 1 -- 172.21.15.7:0/1856152466 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8015020 con 0x7fb7dc108ac0 2024-06-10T01:26:05.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.532+0000 7fb7e43ad700 1 -- 172.21.15.7:0/1856152466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7dc06d7b0 con 0x7fb7dc108ac0 2024-06-10T01:26:05.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.532+0000 7fb7e43ad700 1 -- 172.21.15.7:0/1856152466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7dc0717b0 con 0x7fb7dc108ac0 2024-06-10T01:26:05.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.532+0000 7fb7d37fe700 1 -- 172.21.15.7:0/1856152466 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7d8011d10 con 0x7fb7dc108ac0 2024-06-10T01:26:05.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.532+0000 7fb7d37fe700 1 -- 172.21.15.7:0/1856152466 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d800f2e0 con 0x7fb7dc108ac0 2024-06-10T01:26:05.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.536+0000 7fb7d37fe700 1 -- 172.21.15.7:0/1856152466 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb7d8018020 con 0x7fb7dc108ac0 2024-06-10T01:26:05.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.537+0000 7fb7d37fe700 1 --2- 172.21.15.7:0/1856152466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb7c80a2f00 0x7fb7c80a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:05.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.538+0000 7fb7d37fe700 1 -- 172.21.15.7:0/1856152466 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fb7d80ceab0 con 0x7fb7dc108ac0 2024-06-10T01:26:05.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.538+0000 7fb7e1948700 1 --2- 172.21.15.7:0/1856152466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb7c80a2f00 0x7fb7c80a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:05.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.539+0000 7fb7e1948700 1 --2- 172.21.15.7:0/1856152466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb7c80a2f00 0x7fb7c80a53c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc06e300 tx=0x7fb7cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:05.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.540+0000 7fb7eec30b80 1 Processor -- start 2024-06-10T01:26:05.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.540+0000 7fb7eec30b80 1 -- start start 2024-06-10T01:26:05.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.541+0000 7fb7eec30b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6a0a92cd0 0x55c6a0a930b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:05.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.541+0000 7fb7eec30b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6a0a935f0 0x55c6a0a8cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:05.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.541+0000 7fb7eec30b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6a0a8d460 0x55c6a0a8d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:05.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.541+0000 7fb7e2149700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6a0a92cd0 0x55c6a0a930b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:05.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.541+0000 7fb7e2149700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6a0a92cd0 0x55c6a0a930b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:37956/0 (socket says 172.21.15.7:37956) 2024-06-10T01:26:05.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.541+0000 7fb7e2149700 1 -- 172.21.15.7:0/3165136083 learned_addr learned my addr 172.21.15.7:0/3165136083 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:05.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.541+0000 7fb7e1948700 1 --2- 172.21.15.7:0/3165136083 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6a0a935f0 0x55c6a0a8cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:05.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.541+0000 7fb7eec30b80 1 -- 172.21.15.7:0/3165136083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c6a09de9a0 con 0x55c6a0a8d460 2024-06-10T01:26:05.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.541+0000 7fb7eec30b80 1 -- 172.21.15.7:0/3165136083 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c6a09868b0 con 0x55c6a0a935f0 2024-06-10T01:26:05.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.541+0000 7fb7eec30b80 1 -- 172.21.15.7:0/3165136083 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c6a09da960 con 0x55c6a0a92cd0 2024-06-10T01:26:05.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.542+0000 7fb7e294a700 1 --2- 172.21.15.7:0/3165136083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6a0a8d460 0x55c6a0a8d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:05.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.542+0000 7fb7e1948700 1 -- 172.21.15.7:0/3165136083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6a0a92cd0 msgr2=0x55c6a0a930b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:05.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.542+0000 7fb7e1948700 1 --2- 172.21.15.7:0/3165136083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6a0a92cd0 0x55c6a0a930b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:05.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.542+0000 7fb7e1948700 1 -- 172.21.15.7:0/3165136083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6a0a8d460 msgr2=0x55c6a0a8d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:05.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.542+0000 7fb7e1948700 1 --2- 172.21.15.7:0/3165136083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6a0a8d460 0x55c6a0a8d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:05.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.542+0000 7fb7e1948700 1 -- 172.21.15.7:0/3165136083 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6a097d0e0 con 0x55c6a0a935f0 2024-06-10T01:26:05.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.542+0000 7fb7e294a700 1 --2- 172.21.15.7:0/3165136083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6a0a8d460 0x55c6a0a8d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:26:05.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.542+0000 7fb7e1948700 1 --2- 172.21.15.7:0/3165136083 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6a0a935f0 0x55c6a0a8cf20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc06e390 tx=0x7fb7cc014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49569a9984268eb2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:05.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.542+0000 7fb7d0d70700 1 -- 172.21.15.7:0/3165136083 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc012b60 con 0x55c6a0a935f0 2024-06-10T01:26:05.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.542+0000 7fb7d0d70700 1 -- 172.21.15.7:0/3165136083 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7cc004500 con 0x55c6a0a935f0 2024-06-10T01:26:05.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.542+0000 7fb7d0d70700 1 -- 172.21.15.7:0/3165136083 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc01f420 con 0x55c6a0a935f0 2024-06-10T01:26:05.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.543+0000 7fb7eec30b80 1 -- 172.21.15.7:0/3165136083 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6a0a935f0 msgr2=0x55c6a0a8cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:05.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.543+0000 7fb7eec30b80 1 --2- 172.21.15.7:0/3165136083 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6a0a935f0 0x55c6a0a8cf20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc06e390 tx=0x7fb7cc014000 comp rx=0 tx=0).stop 2024-06-10T01:26:05.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.544+0000 7fb7eec30b80 1 -- 172.21.15.7:0/3165136083 shutdown_connections 2024-06-10T01:26:05.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.544+0000 7fb7eec30b80 1 --2- 172.21.15.7:0/3165136083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6a0a8d460 0x55c6a0a8d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:05.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.544+0000 7fb7eec30b80 1 --2- 172.21.15.7:0/3165136083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6a0a92cd0 0x55c6a0a930b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:05.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.544+0000 7fb7eec30b80 1 --2- 172.21.15.7:0/3165136083 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6a0a935f0 0x55c6a0a8cf20 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:05.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.544+0000 7fb7eec30b80 1 -- 172.21.15.7:0/3165136083 >> 172.21.15.7:0/3165136083 conn(0x55c6a09b7e50 msgr2=0x55c6a0a881b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:05.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.544+0000 7fb7eec30b80 1 -- 172.21.15.7:0/3165136083 shutdown_connections 2024-06-10T01:26:05.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.544+0000 7fb7eec30b80 1 -- 172.21.15.7:0/3165136083 wait complete. 2024-06-10T01:26:05.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.545+0000 7fb7eec30b80 1 -- start start 2024-06-10T01:26:05.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.545+0000 7fb7eec30b80 1 Processor -- start 2024-06-10T01:26:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.546+0000 7fb7eec30b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6a0a8d460 0x55c6a0a993e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.546+0000 7fb7eec30b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6a0a92cd0 0x55c6a0a966e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.546+0000 7fb7e2149700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6a0a8d460 0x55c6a0a993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.546+0000 7fb7e2149700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6a0a8d460 0x55c6a0a993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:33858/0 (socket says 172.21.15.7:33858) 2024-06-10T01:26:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.546+0000 7fb7e2149700 1 -- 172.21.15.7:0/3643539463 learned_addr learned my addr 172.21.15.7:0/3643539463 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.547+0000 7fb7eec30b80 1 --2- 172.21.15.7:0/3643539463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6a0a935f0 0x55c6a0a96c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.547+0000 7fb7eec30b80 1 -- 172.21.15.7:0/3643539463 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c6a0a9a060 con 0x55c6a0a92cd0 2024-06-10T01:26:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.547+0000 7fb7eec30b80 1 -- 172.21.15.7:0/3643539463 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c6a097ed80 con 0x55c6a0a8d460 2024-06-10T01:26:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.547+0000 7fb7eec30b80 1 -- 172.21.15.7:0/3643539463 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c6a0a9a1e0 con 0x55c6a0a935f0 2024-06-10T01:26:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.547+0000 7fb7e294a700 1 --2- 172.21.15.7:0/3643539463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6a0a935f0 0x55c6a0a96c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:05.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.547+0000 7fb7eec30b80 10 client.0 _ll_register_callbacks cb 0x55c6a0978ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:26:05.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.547+0000 7fb7e2149700 1 -- 172.21.15.7:0/3643539463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6a0a935f0 msgr2=0x55c6a0a96c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:05.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.547+0000 7fb7e2149700 1 --2- 172.21.15.7:0/3643539463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6a0a935f0 0x55c6a0a96c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:05.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.547+0000 7fb7e2149700 1 -- 172.21.15.7:0/3643539463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6a0a92cd0 msgr2=0x55c6a0a966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:05.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.547+0000 7fb7e2149700 1 --2- 172.21.15.7:0/3643539463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6a0a92cd0 0x55c6a0a966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:05.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.547+0000 7fb7e2149700 1 -- 172.21.15.7:0/3643539463 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6a092da60 con 0x55c6a0a8d460 2024-06-10T01:26:05.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.547+0000 7fb7e294a700 1 --2- 172.21.15.7:0/3643539463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6a0a935f0 0x55c6a0a96c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:26:05.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.548+0000 7fb7e2149700 1 --2- 172.21.15.7:0/3643539463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6a0a8d460 0x55c6a0a993e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8097390 tx=0x7fb7d809aaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2aafdbc1d44a3984 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:05.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.548+0000 7fb7c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:26:05.542 INFO:tasks.workunit.client.0.smithi007.stdout: 10 170303 24.80 MB/sec execute 282 sec latency 93.900 ms 2024-06-10T01:26:05.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.552+0000 7fb7c5ffb700 1 -- 172.21.15.7:0/3643539463 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d80b3070 con 0x55c6a0a8d460 2024-06-10T01:26:05.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.552+0000 7fb7c5ffb700 1 -- 172.21.15.7:0/3643539463 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fb7d809b390 con 0x55c6a0a8d460 2024-06-10T01:26:05.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.552+0000 7fb7eec30b80 1 -- 172.21.15.7:0/3643539463 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7cc004500 con 0x55c6a0a8d460 2024-06-10T01:26:05.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.553+0000 7fb7c5ffb700 1 -- 172.21.15.7:0/3643539463 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fb7d809af30 con 0x55c6a0a8d460 2024-06-10T01:26:05.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.553+0000 7fb7eec30b80 10 client.29450 fetch_fsmap learned FSMap version 52 2024-06-10T01:26:05.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.553+0000 7fb7eec30b80 1 -- 172.21.15.7:0/3643539463 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55c6a0aa4790 con 0x55c6a0a8d460 2024-06-10T01:26:05.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.557+0000 7fb7c5ffb700 1 -- 172.21.15.7:0/3643539463 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7d80b6030 con 0x55c6a0a8d460 2024-06-10T01:26:05.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.557+0000 7fb7c5ffb700 1 -- 172.21.15.7:0/3643539463 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7fb7d80b9c80 con 0x55c6a0a8d460 2024-06-10T01:26:05.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.557+0000 7fb7eec30b80 10 client.29450 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:26:05.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.557+0000 7fb7eec30b80 10 client.29450 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:26:05.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.557+0000 7fb7eec30b80 20 client.29450 populate_metadata read hostname 'smithi007' 2024-06-10T01:26:05.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.557+0000 7fb7eec30b80 1 --2- 172.21.15.7:0/3643539463 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c6a0aa6dc0 0x55c6a0aa91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:05.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.557+0000 7fb7eec30b80 4 client.29450 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:26:05.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.557+0000 7fb7eec30b80 1 -- 172.21.15.7:0/3643539463 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb7cc004500 con 0x55c6a0aa6dc0 2024-06-10T01:26:05.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.558+0000 7fb7e1948700 1 --2- 172.21.15.7:0/3643539463 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c6a0aa6dc0 0x55c6a0aa91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:05.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.559+0000 7fb7e1948700 1 --2- 172.21.15.7:0/3643539463 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c6a0aa6dc0 0x55c6a0aa91b0 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6dc3896eca15f97 server_cookie=b6ab826591506a67 in_seq=0 out_seq=0 2024-06-10T01:26:05.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.559+0000 7fb7c5ffb700 10 client.29450 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:05.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.561+0000 7fb7c5ffb700 1 -- 172.21.15.7:0/3643539463 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fb7cc004500 con 0x55c6a0aa6dc0 2024-06-10T01:26:05.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.561+0000 7fb7c5ffb700 10 client.29450 handle_command_reply: tid=0 2024-06-10T01:26:05.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.561+0000 7fb7eec30b80 1 client.29450 shutdown 2024-06-10T01:26:05.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.563+0000 7fb7eec30b80 1 -- 172.21.15.7:0/3643539463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6a0a8d460 msgr2=0x55c6a0a993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:05.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.563+0000 7fb7eec30b80 1 --2- 172.21.15.7:0/3643539463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6a0a8d460 0x55c6a0a993e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8097390 tx=0x7fb7d809aaf0 comp rx=0 tx=0).stop 2024-06-10T01:26:05.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.563+0000 7fb7eec30b80 1 -- 172.21.15.7:0/3643539463 shutdown_connections 2024-06-10T01:26:05.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.563+0000 7fb7eec30b80 1 --2- 172.21.15.7:0/3643539463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6a0a935f0 0x55c6a0a96c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:05.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.563+0000 7fb7eec30b80 1 --2- 172.21.15.7:0/3643539463 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c6a0aa6dc0 0x55c6a0aa91b0 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:05.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.563+0000 7fb7eec30b80 1 --2- 172.21.15.7:0/3643539463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6a0a8d460 0x55c6a0a993e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:05.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.563+0000 7fb7c5ffb700 0 client.29450 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:05.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.563+0000 7fb7eec30b80 1 --2- 172.21.15.7:0/3643539463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6a0a92cd0 0x55c6a0a966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:05.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.563+0000 7fb7eec30b80 1 -- 172.21.15.7:0/3643539463 >> 172.21.15.7:0/3643539463 conn(0x55c6a09b7e50 msgr2=0x55c6a0a87da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:05.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.565+0000 7fb7eec30b80 1 -- 172.21.15.7:0/3643539463 shutdown_connections 2024-06-10T01:26:05.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.565+0000 7fb7eec30b80 1 -- 172.21.15.7:0/3643539463 wait complete. 2024-06-10T01:26:05.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.565+0000 7fb7eec30b80 20 client.29450 trim_cache size 0 max 16384 2024-06-10T01:26:05.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.603+0000 7fb7eec30b80 1 Processor -- start 2024-06-10T01:26:05.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.603+0000 7fb7eec30b80 1 -- start start 2024-06-10T01:26:05.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.603+0000 7fb7eec30b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6a0a8d460 0x55c6a0a8cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:05.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.603+0000 7fb7eec30b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6a0a92cd0 0x55c6a0a942c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:05.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.603+0000 7fb7eec30b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6a0a935f0 0x55c6a0a96c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:05.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.603+0000 7fb7eec30b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c0011330 con 0x55c6a0a935f0 2024-06-10T01:26:05.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.603+0000 7fb7eec30b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c000ebc0 con 0x55c6a0a92cd0 2024-06-10T01:26:05.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.603+0000 7fb7eec30b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c000ca10 con 0x55c6a0a8d460 2024-06-10T01:26:05.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.604+0000 7fb7e1948700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6a0a92cd0 0x55c6a0a942c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:05.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.604+0000 7fb7e1948700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6a0a92cd0 0x55c6a0a942c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:33872/0 (socket says 172.21.15.7:33872) 2024-06-10T01:26:05.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.604+0000 7fb7e1948700 1 -- 172.21.15.7:0/1821092027 learned_addr learned my addr 172.21.15.7:0/1821092027 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:05.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.604+0000 7fb7e2149700 1 --2- 172.21.15.7:0/1821092027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6a0a8d460 0x55c6a0a8cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:05.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.604+0000 7fb7e294a700 1 --2- 172.21.15.7:0/1821092027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6a0a935f0 0x55c6a0a96c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:05.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.604+0000 7fb7e1948700 1 -- 172.21.15.7:0/1821092027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6a0a8d460 msgr2=0x55c6a0a8cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:05.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.604+0000 7fb7e1948700 1 --2- 172.21.15.7:0/1821092027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6a0a8d460 0x55c6a0a8cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:05.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.604+0000 7fb7e1948700 1 -- 172.21.15.7:0/1821092027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6a0a935f0 msgr2=0x55c6a0a96c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:05.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.604+0000 7fb7e1948700 1 --2- 172.21.15.7:0/1821092027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6a0a935f0 0x55c6a0a96c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:05.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.604+0000 7fb7e1948700 1 -- 172.21.15.7:0/1821092027 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6a0a956a0 con 0x55c6a0a92cd0 2024-06-10T01:26:05.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.604+0000 7fb7e2149700 1 --2- 172.21.15.7:0/1821092027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6a0a8d460 0x55c6a0a8cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:26:05.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.604+0000 7fb7e1948700 1 --2- 172.21.15.7:0/1821092027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6a0a92cd0 0x55c6a0a942c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0097d0 tx=0x7fb7cc003ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=203539d30bfa5454 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:05.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.605+0000 7fb7c7fff700 1 -- 172.21.15.7:0/1821092027 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc003880 con 0x55c6a0a92cd0 2024-06-10T01:26:05.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.605+0000 7fb7c7fff700 1 -- 172.21.15.7:0/1821092027 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7cc037070 con 0x55c6a0a92cd0 2024-06-10T01:26:05.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.606+0000 7fb7eec30b80 1 -- 172.21.15.7:0/1821092027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6a0a92cd0 msgr2=0x55c6a0a942c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:05.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.606+0000 7fb7eec30b80 1 --2- 172.21.15.7:0/1821092027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6a0a92cd0 0x55c6a0a942c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0097d0 tx=0x7fb7cc003ac0 comp rx=0 tx=0).stop 2024-06-10T01:26:05.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.606+0000 7fb7eec30b80 1 -- 172.21.15.7:0/1821092027 shutdown_connections 2024-06-10T01:26:05.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.606+0000 7fb7eec30b80 1 --2- 172.21.15.7:0/1821092027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6a0a935f0 0x55c6a0a96c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:05.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.606+0000 7fb7eec30b80 1 --2- 172.21.15.7:0/1821092027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6a0a8d460 0x55c6a0a8cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:05.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.606+0000 7fb7eec30b80 1 --2- 172.21.15.7:0/1821092027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6a0a92cd0 0x55c6a0a942c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:05.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.606+0000 7fb7eec30b80 1 -- 172.21.15.7:0/1821092027 >> 172.21.15.7:0/1821092027 conn(0x55c6a09b7e50 msgr2=0x55c6a0a91860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:05.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.607+0000 7fb7eec30b80 1 -- 172.21.15.7:0/1821092027 shutdown_connections 2024-06-10T01:26:05.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.607+0000 7fb7eec30b80 1 -- 172.21.15.7:0/1821092027 wait complete. 2024-06-10T01:26:05.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.607+0000 7fb7eec30b80 1 -- start start 2024-06-10T01:26:05.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.608+0000 7fb7eec30b80 1 Processor -- start 2024-06-10T01:26:05.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.608+0000 7fb7eec30b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6a0a8d460 0x55c6a0a9f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:05.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.608+0000 7fb7eec30b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6a0a92cd0 0x55c6a0a9fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:05.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.608+0000 7fb7eec30b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6a0a935f0 0x55c6a0a9c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:05.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.608+0000 7fb7eec30b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c000d080 con 0x55c6a0a935f0 2024-06-10T01:26:05.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.608+0000 7fb7eec30b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c000e4f0 con 0x55c6a0a92cd0 2024-06-10T01:26:05.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.608+0000 7fb7eec30b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c000d750 con 0x55c6a0a8d460 2024-06-10T01:26:05.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.608+0000 7fb7eec30b80 10 client.0 _ll_register_callbacks cb 0x55c6a0a82a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:26:05.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.608+0000 7fb7e1948700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6a0a92cd0 0x55c6a0a9fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:05.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.609+0000 7fb7e2149700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6a0a8d460 0x55c6a0a9f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:05.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.609+0000 7fb7e294a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6a0a935f0 0x55c6a0a9c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:05.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.609+0000 7fb7e2149700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6a0a8d460 0x55c6a0a9f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:37980/0 (socket says 172.21.15.7:37980) 2024-06-10T01:26:05.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.609+0000 7fb7e294a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6a0a935f0 0x55c6a0a9c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58654/0 (socket says 172.21.15.7:58654) 2024-06-10T01:26:05.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.609+0000 7fb7e2149700 1 -- 172.21.15.7:0/637653413 learned_addr learned my addr 172.21.15.7:0/637653413 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:05.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.609+0000 7fb7e294a700 1 -- 172.21.15.7:0/637653413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6a0a8d460 msgr2=0x55c6a0a9f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:05.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.609+0000 7fb7e294a700 1 --2- 172.21.15.7:0/637653413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6a0a8d460 0x55c6a0a9f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:05.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.609+0000 7fb7e294a700 1 -- 172.21.15.7:0/637653413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6a0a92cd0 msgr2=0x55c6a0a9fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:05.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.609+0000 7fb7e294a700 1 --2- 172.21.15.7:0/637653413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6a0a92cd0 0x55c6a0a9fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:05.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.609+0000 7fb7e294a700 1 -- 172.21.15.7:0/637653413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6a0a9cb80 con 0x55c6a0a935f0 2024-06-10T01:26:05.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.610+0000 7fb7e294a700 1 --2- 172.21.15.7:0/637653413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6a0a935f0 0x55c6a0a9c4d0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc074690 tx=0x7fb7d4004e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884c748b05bf2888 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:05.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.610+0000 7fb7c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:26:05.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.610+0000 7fb7c5ffb700 1 -- 172.21.15.7:0/637653413 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d40076f0 con 0x55c6a0a935f0 2024-06-10T01:26:05.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.610+0000 7fb7eec30b80 1 -- 172.21.15.7:0/637653413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7cc037070 con 0x55c6a0a935f0 2024-06-10T01:26:05.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.611+0000 7fb7c5ffb700 1 -- 172.21.15.7:0/637653413 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fb7d4007890 con 0x55c6a0a935f0 2024-06-10T01:26:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.611+0000 7fb7c5ffb700 1 -- 172.21.15.7:0/637653413 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fb7d4007a90 con 0x55c6a0a935f0 2024-06-10T01:26:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.613+0000 7fb7eec30b80 10 client.20235 fetch_fsmap learned FSMap version 52 2024-06-10T01:26:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.613+0000 7fb7eec30b80 1 -- 172.21.15.7:0/637653413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55c6a0b12400 con 0x55c6a0a935f0 2024-06-10T01:26:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.614+0000 7fb7c5ffb700 1 -- 172.21.15.7:0/637653413 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7cc037070 con 0x55c6a0a935f0 2024-06-10T01:26:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.614+0000 7fb7c5ffb700 1 -- 172.21.15.7:0/637653413 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7fb7d4011070 con 0x55c6a0a935f0 2024-06-10T01:26:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.614+0000 7fb7eec30b80 10 client.20235 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:26:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.614+0000 7fb7eec30b80 10 client.20235 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:26:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.614+0000 7fb7eec30b80 20 client.20235 populate_metadata read hostname 'smithi007' 2024-06-10T01:26:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.614+0000 7fb7eec30b80 1 --2- 172.21.15.7:0/637653413 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c6a0aa6dc0 0x55c6a0b14ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.614+0000 7fb7eec30b80 4 client.20235 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.0, "await_for": 45.0}] 2024-06-10T01:26:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.614+0000 7fb7eec30b80 1 -- 172.21.15.7:0/637653413 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.0, "await_for": 45.0}) -- 0x55c6a09ebea0 con 0x55c6a0aa6dc0 2024-06-10T01:26:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.615+0000 7fb7e2149700 1 --2- 172.21.15.7:0/637653413 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c6a0aa6dc0 0x55c6a0b14ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.616+0000 7fb7c5ffb700 10 client.20235 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:05.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:05.616+0000 7fb7e2149700 1 --2- 172.21.15.7:0/637653413 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c6a0aa6dc0 0x55c6a0b14ba0 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d22ceb0f3d48490f server_cookie=2671af1d4979201d in_seq=0 out_seq=0 2024-06-10T01:26:05.725 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:05 smithi007 ceph-mon[32398]: pgmap v1035: 129 pgs: 129 active+clean; 660 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.3 MiB/s wr, 492 op/s 2024-06-10T01:26:05.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:05 smithi078 ceph-mon[36683]: pgmap v1035: 129 pgs: 129 active+clean; 660 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.3 MiB/s wr, 492 op/s 2024-06-10T01:26:05.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:05 smithi175 ceph-mon[36636]: pgmap v1035: 129 pgs: 129 active+clean; 660 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.3 MiB/s wr, 492 op/s 2024-06-10T01:26:06.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.047+0000 7fc97bfff700 1 -- 172.21.15.7:0/1644649368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc97409be40 msgr2=0x7fc97409c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:06.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.047+0000 7fc97bfff700 1 --2- 172.21.15.7:0/1644649368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc97409be40 0x7fc97409c220 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0098f0 tx=0x7fc96c009c00 comp rx=0 tx=0).stop 2024-06-10T01:26:06.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.047+0000 7fc97bfff700 1 -- 172.21.15.7:0/1644649368 shutdown_connections 2024-06-10T01:26:06.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.047+0000 7fc97bfff700 1 --2- 172.21.15.7:0/1644649368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc974005700 0x7fc9740a2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.047+0000 7fc97bfff700 1 --2- 172.21.15.7:0/1644649368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc97409be40 0x7fc97409c220 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.047+0000 7fc97bfff700 1 --2- 172.21.15.7:0/1644649368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc974004d40 0x7fc9740051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.047+0000 7fc97bfff700 1 -- 172.21.15.7:0/1644649368 >> 172.21.15.7:0/1644649368 conn(0x7fc97400a7a0 msgr2=0x7fc97400abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:06.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.048+0000 7fc97bfff700 1 -- 172.21.15.7:0/1644649368 shutdown_connections 2024-06-10T01:26:06.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.048+0000 7fc97bfff700 1 -- 172.21.15.7:0/1644649368 wait complete. 2024-06-10T01:26:06.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.049+0000 7fc97bfff700 1 Processor -- start 2024-06-10T01:26:06.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.049+0000 7fc97bfff700 1 -- start start 2024-06-10T01:26:06.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.049+0000 7fc97bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc974004d40 0x7fc9741387c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:06.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.049+0000 7fc97bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc974005700 0x7fc974138d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:06.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.049+0000 7fc97affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc974004d40 0x7fc9741387c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:06.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.049+0000 7fc97bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc97409be40 0x7fc9740a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:06.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.049+0000 7fc97bfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc9740a5fb0 con 0x7fc974004d40 2024-06-10T01:26:06.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.049+0000 7fc97affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc974004d40 0x7fc9741387c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58672/0 (socket says 172.21.15.7:58672) 2024-06-10T01:26:06.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.049+0000 7fc97bfff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc9740a5e30 con 0x7fc97409be40 2024-06-10T01:26:06.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.049+0000 7fc97affd700 1 -- 172.21.15.7:0/3876315090 learned_addr learned my addr 172.21.15.7:0/3876315090 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:06.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.049+0000 7fc97bfff700 1 -- 172.21.15.7:0/3876315090 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc9740a6130 con 0x7fc974005700 2024-06-10T01:26:06.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.049+0000 7fc97a7fc700 1 --2- 172.21.15.7:0/3876315090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc974005700 0x7fc974138d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:06.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.050+0000 7fc97b7fe700 1 --2- 172.21.15.7:0/3876315090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc97409be40 0x7fc9740a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:06.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.050+0000 7fc97affd700 1 -- 172.21.15.7:0/3876315090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc974005700 msgr2=0x7fc974138d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:06.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.050+0000 7fc97affd700 1 --2- 172.21.15.7:0/3876315090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc974005700 0x7fc974138d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.050+0000 7fc97affd700 1 -- 172.21.15.7:0/3876315090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc97409be40 msgr2=0x7fc9740a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:06.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.050+0000 7fc97affd700 1 --2- 172.21.15.7:0/3876315090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc97409be40 0x7fc9740a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.050+0000 7fc97affd700 1 -- 172.21.15.7:0/3876315090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9740a25b0 con 0x7fc974004d40 2024-06-10T01:26:06.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.050+0000 7fc97a7fc700 1 --2- 172.21.15.7:0/3876315090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc974005700 0x7fc974138d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:26:06.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.050+0000 7fc97affd700 1 --2- 172.21.15.7:0/3876315090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc974004d40 0x7fc9741387c0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fc96c009230 tx=0x7fc96c00fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:06.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.050+0000 7fc963fff700 1 -- 172.21.15.7:0/3876315090 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c005140 con 0x7fc974004d40 2024-06-10T01:26:06.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.050+0000 7fc97bfff700 1 -- 172.21.15.7:0/3876315090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9740a2840 con 0x7fc974004d40 2024-06-10T01:26:06.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.050+0000 7fc963fff700 1 -- 172.21.15.7:0/3876315090 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc96c00b560 con 0x7fc974004d40 2024-06-10T01:26:06.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.050+0000 7fc963fff700 1 -- 172.21.15.7:0/3876315090 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c003a40 con 0x7fc974004d40 2024-06-10T01:26:06.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.050+0000 7fc97bfff700 1 -- 172.21.15.7:0/3876315090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc97413f690 con 0x7fc974004d40 2024-06-10T01:26:06.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.055+0000 7fc963fff700 1 -- 172.21.15.7:0/3876315090 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fc96c029060 con 0x7fc974004d40 2024-06-10T01:26:06.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.055+0000 7fc98bf11b80 1 Processor -- start 2024-06-10T01:26:06.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.056+0000 7fc98bf11b80 1 -- start start 2024-06-10T01:26:06.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.056+0000 7fc98bf11b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622e7a5d0a0 0x5622e7a5d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:06.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.056+0000 7fc98bf11b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622e7a5d9c0 0x5622e7a584b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.056+0000 7fc98bf11b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622e7a589f0 0x5622e7a58e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.056+0000 7fc963fff700 1 --2- 172.21.15.7:0/3876315090 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc9640a2ca0 0x7fc9640a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.056+0000 7fc98bf11b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5622e794af60 con 0x5622e7a5d9c0 2024-06-10T01:26:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.056+0000 7fc98bf11b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5622e79a9850 con 0x5622e7a589f0 2024-06-10T01:26:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.056+0000 7fc98bf11b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5622e7911460 con 0x5622e7a5d0a0 2024-06-10T01:26:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.056+0000 7fc97a7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622e7a589f0 0x5622e7a58e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.056+0000 7fc97a7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622e7a589f0 0x5622e7a58e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:33902/0 (socket says 172.21.15.7:33902) 2024-06-10T01:26:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.056+0000 7fc97a7fc700 1 -- 172.21.15.7:0/409354231 learned_addr learned my addr 172.21.15.7:0/409354231 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.056+0000 7fc97b7fe700 1 --2- 172.21.15.7:0/3876315090 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc9640a2ca0 0x7fc9640a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.056+0000 7fc963fff700 1 -- 172.21.15.7:0/3876315090 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fc96c0cfc60 con 0x7fc974004d40 2024-06-10T01:26:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.056+0000 7fc97b7fe700 1 --2- 172.21.15.7:0/409354231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622e7a5d0a0 0x5622e7a5d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.056+0000 7fc97affd700 1 --2- 172.21.15.7:0/409354231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622e7a5d9c0 0x5622e7a584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:06.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.056+0000 7fc97a7fc700 1 -- 172.21.15.7:0/409354231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622e7a5d0a0 msgr2=0x5622e7a5d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:06.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.056+0000 7fc97a7fc700 1 --2- 172.21.15.7:0/409354231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622e7a5d0a0 0x5622e7a5d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.057+0000 7fc97a7fc700 1 -- 172.21.15.7:0/409354231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622e7a5d9c0 msgr2=0x5622e7a584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:06.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.057+0000 7fc97a7fc700 1 --2- 172.21.15.7:0/409354231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622e7a5d9c0 0x5622e7a584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.057+0000 7fc97a7fc700 1 -- 172.21.15.7:0/409354231 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622e78f88d0 con 0x5622e7a589f0 2024-06-10T01:26:06.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.057+0000 7fc97affd700 1 --2- 172.21.15.7:0/409354231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622e7a5d9c0 0x5622e7a584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:26:06.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.057+0000 7fc97b7fe700 1 --2- 172.21.15.7:0/409354231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622e7a5d0a0 0x5622e7a5d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:26:06.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.057+0000 7fc97a7fc700 1 --2- 172.21.15.7:0/409354231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622e7a589f0 0x5622e7a58e50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc9700115f0 tx=0x7fc970011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39551f5221a30375 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:06.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.057+0000 7fc97b7fe700 1 --2- 172.21.15.7:0/3876315090 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc9640a2ca0 0x7fc9640a5160 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc97c065ce0 tx=0x7fc97c050b00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:06.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.057+0000 7fc9617fa700 1 -- 172.21.15.7:0/409354231 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97001e070 con 0x5622e7a589f0 2024-06-10T01:26:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.057+0000 7fc9617fa700 1 -- 172.21.15.7:0/409354231 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc970004510 con 0x5622e7a589f0 2024-06-10T01:26:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.057+0000 7fc9617fa700 1 -- 172.21.15.7:0/409354231 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970019690 con 0x5622e7a589f0 2024-06-10T01:26:06.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.058+0000 7fc98bf11b80 1 -- 172.21.15.7:0/409354231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622e7a589f0 msgr2=0x5622e7a58e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:06.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.058+0000 7fc98bf11b80 1 --2- 172.21.15.7:0/409354231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622e7a589f0 0x5622e7a58e50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc9700115f0 tx=0x7fc970011900 comp rx=0 tx=0).stop 2024-06-10T01:26:06.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.058+0000 7fc98bf11b80 1 -- 172.21.15.7:0/409354231 shutdown_connections 2024-06-10T01:26:06.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.058+0000 7fc98bf11b80 1 --2- 172.21.15.7:0/409354231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622e7a589f0 0x5622e7a58e50 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.058+0000 7fc98bf11b80 1 --2- 172.21.15.7:0/409354231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622e7a5d0a0 0x5622e7a5d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.058+0000 7fc98bf11b80 1 --2- 172.21.15.7:0/409354231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622e7a5d9c0 0x5622e7a584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.058+0000 7fc98bf11b80 1 -- 172.21.15.7:0/409354231 >> 172.21.15.7:0/409354231 conn(0x5622e7985080 msgr2=0x5622e7a53390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:06.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.059+0000 7fc98bf11b80 1 -- 172.21.15.7:0/409354231 shutdown_connections 2024-06-10T01:26:06.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.059+0000 7fc98bf11b80 1 -- 172.21.15.7:0/409354231 wait complete. 2024-06-10T01:26:06.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.059+0000 7fc98bf11b80 1 -- start start 2024-06-10T01:26:06.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.059+0000 7fc98bf11b80 1 Processor -- start 2024-06-10T01:26:06.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.060+0000 7fc98bf11b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622e7a589f0 0x5622e7a64700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:06.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.060+0000 7fc98bf11b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622e7a5d0a0 0x5622e7a61a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:06.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.060+0000 7fc98bf11b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622e7a5d9c0 0x5622e7a61f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:06.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.060+0000 7fc98bf11b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5622e7949fc0 con 0x5622e7a5d9c0 2024-06-10T01:26:06.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.060+0000 7fc98bf11b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5622e79a5810 con 0x5622e7a5d0a0 2024-06-10T01:26:06.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.060+0000 7fc98bf11b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5622e7a65450 con 0x5622e7a589f0 2024-06-10T01:26:06.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.060+0000 7fc98bf11b80 10 client.0 _ll_register_callbacks cb 0x5622e793f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:26:06.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.060+0000 7fc97affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622e7a589f0 0x5622e7a64700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:06.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.060+0000 7fc97affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622e7a589f0 0x5622e7a64700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:38018/0 (socket says 172.21.15.7:38018) 2024-06-10T01:26:06.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.060+0000 7fc97affd700 1 -- 172.21.15.7:0/3825407972 learned_addr learned my addr 172.21.15.7:0/3825407972 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:06.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.060+0000 7fc97a7fc700 1 --2- 172.21.15.7:0/3825407972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622e7a5d0a0 0x5622e7a61a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:06.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.060+0000 7fc97b7fe700 1 --2- 172.21.15.7:0/3825407972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622e7a5d9c0 0x5622e7a61f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:06.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.060+0000 7fc97affd700 1 -- 172.21.15.7:0/3825407972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622e7a5d0a0 msgr2=0x5622e7a61a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:06.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.060+0000 7fc97affd700 1 --2- 172.21.15.7:0/3825407972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622e7a5d0a0 0x5622e7a61a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.060+0000 7fc97affd700 1 -- 172.21.15.7:0/3825407972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622e7a5d9c0 msgr2=0x5622e7a61f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:06.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.060+0000 7fc97affd700 1 --2- 172.21.15.7:0/3825407972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622e7a5d9c0 0x5622e7a61f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.060+0000 7fc97affd700 1 -- 172.21.15.7:0/3825407972 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622e796f750 con 0x5622e7a589f0 2024-06-10T01:26:06.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.060+0000 7fc97affd700 1 --2- 172.21.15.7:0/3825407972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622e7a589f0 0x5622e7a64700 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0a4820 tx=0x7fc96c09bb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53ab7e61f1400ac0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:06.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.061+0000 7fc95eff5700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:26:06.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.061+0000 7fc95eff5700 1 -- 172.21.15.7:0/3825407972 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c0a0520 con 0x5622e7a589f0 2024-06-10T01:26:06.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.061+0000 7fc95eff5700 1 -- 172.21.15.7:0/3825407972 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fc96c0987e0 con 0x5622e7a589f0 2024-06-10T01:26:06.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.061+0000 7fc98bf11b80 1 -- 172.21.15.7:0/3825407972 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc970004510 con 0x5622e7a589f0 2024-06-10T01:26:06.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.061+0000 7fc95eff5700 1 -- 172.21.15.7:0/3825407972 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fc96c0a4d50 con 0x5622e7a589f0 2024-06-10T01:26:06.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.061+0000 7fc98bf11b80 10 client.29404 fetch_fsmap learned FSMap version 52 2024-06-10T01:26:06.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.061+0000 7fc98bf11b80 1 -- 172.21.15.7:0/3825407972 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5622e7a6f890 con 0x5622e7a589f0 2024-06-10T01:26:06.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.061+0000 7fc95eff5700 1 -- 172.21.15.7:0/3825407972 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc970004510 con 0x5622e7a589f0 2024-06-10T01:26:06.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.061+0000 7fc95eff5700 1 -- 172.21.15.7:0/3825407972 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7fc96c096ce0 con 0x5622e7a589f0 2024-06-10T01:26:06.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.061+0000 7fc98bf11b80 10 client.29404 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:26:06.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.061+0000 7fc98bf11b80 10 client.29404 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:26:06.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.061+0000 7fc98bf11b80 20 client.29404 populate_metadata read hostname 'smithi007' 2024-06-10T01:26:06.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.062+0000 7fc98bf11b80 1 --2- 172.21.15.7:0/3825407972 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5622e7a71fd0 0x5622e7a743f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:06.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.062+0000 7fc98bf11b80 4 client.29404 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:26:06.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.062+0000 7fc98bf11b80 1 -- 172.21.15.7:0/3825407972 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622e79b6d50 con 0x5622e7a71fd0 2024-06-10T01:26:06.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.062+0000 7fc97a7fc700 1 --2- 172.21.15.7:0/3825407972 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5622e7a71fd0 0x5622e7a743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:06.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.062+0000 7fc95eff5700 10 client.29404 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:26:06.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.062+0000 7fc97a7fc700 1 --2- 172.21.15.7:0/3825407972 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5622e7a71fd0 0x5622e7a743f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=31b04511104c602e server_cookie=87c24242261143e0 in_seq=0 out_seq=0 2024-06-10T01:26:06.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.063+0000 7fc95eff5700 1 -- 172.21.15.7:0/3825407972 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5622e79b6d50 con 0x5622e7a71fd0 2024-06-10T01:26:06.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.063+0000 7fc95eff5700 10 client.29404 handle_command_reply: tid=0 2024-06-10T01:26:06.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.063+0000 7fc98bf11b80 1 client.29404 shutdown 2024-06-10T01:26:06.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.063+0000 7fc98bf11b80 1 -- 172.21.15.7:0/3825407972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622e7a589f0 msgr2=0x5622e7a64700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:06.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.063+0000 7fc98bf11b80 1 --2- 172.21.15.7:0/3825407972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622e7a589f0 0x5622e7a64700 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0a4820 tx=0x7fc96c09bb50 comp rx=0 tx=0).stop 2024-06-10T01:26:06.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.063+0000 7fc98bf11b80 1 -- 172.21.15.7:0/3825407972 shutdown_connections 2024-06-10T01:26:06.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.063+0000 7fc98bf11b80 1 --2- 172.21.15.7:0/3825407972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622e7a5d9c0 0x5622e7a61f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.063+0000 7fc98bf11b80 1 --2- 172.21.15.7:0/3825407972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622e7a589f0 0x5622e7a64700 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.063+0000 7fc98bf11b80 1 --2- 172.21.15.7:0/3825407972 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5622e7a71fd0 0x5622e7a743f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.063+0000 7fc98bf11b80 1 --2- 172.21.15.7:0/3825407972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622e7a5d0a0 0x5622e7a61a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.063+0000 7fc98bf11b80 1 -- 172.21.15.7:0/3825407972 >> 172.21.15.7:0/3825407972 conn(0x5622e7985080 msgr2=0x5622e7a520c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:06.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.063+0000 7fc95eff5700 0 client.29404 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:26:06.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.064+0000 7fc98bf11b80 1 -- 172.21.15.7:0/3825407972 shutdown_connections 2024-06-10T01:26:06.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.064+0000 7fc98bf11b80 1 -- 172.21.15.7:0/3825407972 wait complete. 2024-06-10T01:26:06.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.064+0000 7fc98bf11b80 20 client.29404 trim_cache size 0 max 16384 2024-06-10T01:26:06.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.085+0000 7fc98bf11b80 1 Processor -- start 2024-06-10T01:26:06.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.085+0000 7fc98bf11b80 1 -- start start 2024-06-10T01:26:06.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.085+0000 7fc98bf11b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622e7a589f0 0x5622e7a63940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:06.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.085+0000 7fc98bf11b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622e7a5d0a0 0x5622e7a530d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:06.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.085+0000 7fc98bf11b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622e7a5d9c0 0x5622e7a564a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:06.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.085+0000 7fc98bf11b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc97414a850 con 0x5622e7a5d9c0 2024-06-10T01:26:06.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.085+0000 7fc98bf11b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc9741480e0 con 0x5622e7a5d0a0 2024-06-10T01:26:06.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.085+0000 7fc98bf11b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc974145f30 con 0x5622e7a589f0 2024-06-10T01:26:06.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.086+0000 7fc97affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622e7a589f0 0x5622e7a63940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:06.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.086+0000 7fc97b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622e7a5d9c0 0x5622e7a564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:06.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.086+0000 7fc97affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622e7a589f0 0x5622e7a63940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:38022/0 (socket says 172.21.15.7:38022) 2024-06-10T01:26:06.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.086+0000 7fc97affd700 1 -- 172.21.15.7:0/2026564599 learned_addr learned my addr 172.21.15.7:0/2026564599 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:06.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.086+0000 7fc97a7fc700 1 --2- 172.21.15.7:0/2026564599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622e7a5d0a0 0x5622e7a530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:06.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.086+0000 7fc97b7fe700 1 -- 172.21.15.7:0/2026564599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622e7a589f0 msgr2=0x5622e7a63940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:06.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.086+0000 7fc97b7fe700 1 --2- 172.21.15.7:0/2026564599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622e7a589f0 0x5622e7a63940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.086+0000 7fc97b7fe700 1 -- 172.21.15.7:0/2026564599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622e7a5d0a0 msgr2=0x5622e7a530d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:06.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.086+0000 7fc97b7fe700 1 --2- 172.21.15.7:0/2026564599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622e7a5d0a0 0x5622e7a530d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.086+0000 7fc97b7fe700 1 -- 172.21.15.7:0/2026564599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622e7a5ff70 con 0x5622e7a5d9c0 2024-06-10T01:26:06.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.086+0000 7fc97a7fc700 1 --2- 172.21.15.7:0/2026564599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622e7a5d0a0 0x5622e7a530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:26:06.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.086+0000 7fc97b7fe700 1 --2- 172.21.15.7:0/2026564599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622e7a5d9c0 0x5622e7a564a0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fc97c067f10 tx=0x7fc97c084070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b638b191b1eced server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:06.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.087+0000 7fc960ff9700 1 -- 172.21.15.7:0/2026564599 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c0698f0 con 0x5622e7a5d9c0 2024-06-10T01:26:06.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.087+0000 7fc960ff9700 1 -- 172.21.15.7:0/2026564599 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc97c086070 con 0x5622e7a5d9c0 2024-06-10T01:26:06.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.087+0000 7fc960ff9700 1 -- 172.21.15.7:0/2026564599 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c082df0 con 0x5622e7a5d9c0 2024-06-10T01:26:06.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.088+0000 7fc98bf11b80 1 -- 172.21.15.7:0/2026564599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622e7a5d9c0 msgr2=0x5622e7a564a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:06.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.088+0000 7fc98bf11b80 1 --2- 172.21.15.7:0/2026564599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622e7a5d9c0 0x5622e7a564a0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fc97c067f10 tx=0x7fc97c084070 comp rx=0 tx=0).stop 2024-06-10T01:26:06.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.088+0000 7fc98bf11b80 1 -- 172.21.15.7:0/2026564599 shutdown_connections 2024-06-10T01:26:06.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.088+0000 7fc98bf11b80 1 --2- 172.21.15.7:0/2026564599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622e7a5d9c0 0x5622e7a564a0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.088+0000 7fc98bf11b80 1 --2- 172.21.15.7:0/2026564599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622e7a589f0 0x5622e7a63940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.088+0000 7fc98bf11b80 1 --2- 172.21.15.7:0/2026564599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622e7a5d0a0 0x5622e7a530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.088+0000 7fc98bf11b80 1 -- 172.21.15.7:0/2026564599 >> 172.21.15.7:0/2026564599 conn(0x5622e7985080 msgr2=0x5622e7a63230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:06.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.090+0000 7fc98bf11b80 1 -- 172.21.15.7:0/2026564599 shutdown_connections 2024-06-10T01:26:06.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.090+0000 7fc98bf11b80 1 -- 172.21.15.7:0/2026564599 wait complete. 2024-06-10T01:26:06.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.090+0000 7fc98bf11b80 1 -- start start 2024-06-10T01:26:06.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.090+0000 7fc98bf11b80 1 Processor -- start 2024-06-10T01:26:06.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.091+0000 7fc98bf11b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622e7a589f0 0x5622e7a6ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:06.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.091+0000 7fc98bf11b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622e7a5d0a0 0x5622e7a5e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:06.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.091+0000 7fc98bf11b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622e7a5d9c0 0x5622e7a5ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:06.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.091+0000 7fc98bf11b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc9741465a0 con 0x5622e7a5d0a0 2024-06-10T01:26:06.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.091+0000 7fc98bf11b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc974147a10 con 0x5622e7a5d9c0 2024-06-10T01:26:06.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.091+0000 7fc98bf11b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc974146c70 con 0x5622e7a589f0 2024-06-10T01:26:06.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.091+0000 7fc98bf11b80 10 client.0 _ll_register_callbacks cb 0x5622e793f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:26:06.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.091+0000 7fc97b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622e7a5d9c0 0x5622e7a5ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:06.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.091+0000 7fc97affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622e7a589f0 0x5622e7a6ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:06.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.091+0000 7fc97b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622e7a5d9c0 0x5622e7a5ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:33934/0 (socket says 172.21.15.7:33934) 2024-06-10T01:26:06.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.091+0000 7fc97b7fe700 1 -- 172.21.15.7:0/2717066469 learned_addr learned my addr 172.21.15.7:0/2717066469 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:06.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.091+0000 7fc97affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622e7a589f0 0x5622e7a6ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:38024/0 (socket says 172.21.15.7:38024) 2024-06-10T01:26:06.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.091+0000 7fc97a7fc700 1 --2- 172.21.15.7:0/2717066469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622e7a5d0a0 0x5622e7a5e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:06.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.091+0000 7fc97b7fe700 1 -- 172.21.15.7:0/2717066469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622e7a589f0 msgr2=0x5622e7a6ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:06.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.091+0000 7fc97b7fe700 1 --2- 172.21.15.7:0/2717066469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622e7a589f0 0x5622e7a6ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.091+0000 7fc97b7fe700 1 -- 172.21.15.7:0/2717066469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622e7a5d0a0 msgr2=0x5622e7a5e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:06.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.091+0000 7fc97b7fe700 1 --2- 172.21.15.7:0/2717066469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622e7a5d0a0 0x5622e7a5e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.091+0000 7fc97b7fe700 1 -- 172.21.15.7:0/2717066469 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622e7a5f000 con 0x5622e7a5d9c0 2024-06-10T01:26:06.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.092+0000 7fc97a7fc700 1 --2- 172.21.15.7:0/2717066469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622e7a5d0a0 0x5622e7a5e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:26:06.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.092+0000 7fc97b7fe700 1 --2- 172.21.15.7:0/2717066469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622e7a5d9c0 0x5622e7a5ea70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc97c065600 tx=0x7fc97c06acb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=794984752e0a2f1f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:06.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.092+0000 7fc95eff5700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:26:06.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.092+0000 7fc95eff5700 1 -- 172.21.15.7:0/2717066469 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c082460 con 0x5622e7a5d9c0 2024-06-10T01:26:06.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.092+0000 7fc95eff5700 1 -- 172.21.15.7:0/2717066469 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fc97c082600 con 0x5622e7a5d9c0 2024-06-10T01:26:06.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.092+0000 7fc98bf11b80 1 -- 172.21.15.7:0/2717066469 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc97c086070 con 0x5622e7a5d9c0 2024-06-10T01:26:06.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.092+0000 7fc95eff5700 1 -- 172.21.15.7:0/2717066469 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fc97c082840 con 0x5622e7a5d9c0 2024-06-10T01:26:06.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.092+0000 7fc98bf11b80 10 client.29474 fetch_fsmap learned FSMap version 52 2024-06-10T01:26:06.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.092+0000 7fc98bf11b80 1 -- 172.21.15.7:0/2717066469 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5622e7a69560 con 0x5622e7a5d9c0 2024-06-10T01:26:06.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.093+0000 7fc95eff5700 1 -- 172.21.15.7:0/2717066469 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc97c099430 con 0x5622e7a5d9c0 2024-06-10T01:26:06.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.093+0000 7fc95eff5700 1 -- 172.21.15.7:0/2717066469 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7fc97c08f430 con 0x5622e7a5d9c0 2024-06-10T01:26:06.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.093+0000 7fc98bf11b80 10 client.29474 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:26:06.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.093+0000 7fc98bf11b80 10 client.29474 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:26:06.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.093+0000 7fc98bf11b80 20 client.29474 populate_metadata read hostname 'smithi007' 2024-06-10T01:26:06.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.093+0000 7fc98bf11b80 1 --2- 172.21.15.7:0/2717066469 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5622e7add030 0x5622e7adf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:06.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.093+0000 7fc98bf11b80 4 client.29474 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:26:06.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.093+0000 7fc98bf11b80 1 -- 172.21.15.7:0/2717066469 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fc97c086070 con 0x5622e7add030 2024-06-10T01:26:06.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.094+0000 7fc97affd700 1 --2- 172.21.15.7:0/2717066469 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5622e7add030 0x5622e7adf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:06.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.094+0000 7fc95eff5700 10 client.29474 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:26:06.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.094+0000 7fc97affd700 1 --2- 172.21.15.7:0/2717066469 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5622e7add030 0x5622e7adf450 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ebd89f870643f614 server_cookie=ae73dfa738f3db68 in_seq=0 out_seq=0 2024-06-10T01:26:06.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.094+0000 7fc95eff5700 1 -- 172.21.15.7:0/2717066469 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fc97c086070 con 0x5622e7add030 2024-06-10T01:26:06.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.094+0000 7fc95eff5700 10 client.29474 handle_command_reply: tid=0 2024-06-10T01:26:06.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.094+0000 7fc98bf11b80 1 client.29474 shutdown 2024-06-10T01:26:06.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.094+0000 7fc98bf11b80 1 -- 172.21.15.7:0/2717066469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622e7a5d9c0 msgr2=0x5622e7a5ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:06.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.094+0000 7fc98bf11b80 1 --2- 172.21.15.7:0/2717066469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622e7a5d9c0 0x5622e7a5ea70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc97c065600 tx=0x7fc97c06acb0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.095+0000 7fc98bf11b80 1 -- 172.21.15.7:0/2717066469 shutdown_connections 2024-06-10T01:26:06.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.095+0000 7fc98bf11b80 1 --2- 172.21.15.7:0/2717066469 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5622e7add030 0x5622e7adf450 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.095+0000 7fc98bf11b80 1 --2- 172.21.15.7:0/2717066469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622e7a5d9c0 0x5622e7a5ea70 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.095+0000 7fc98bf11b80 1 --2- 172.21.15.7:0/2717066469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622e7a589f0 0x5622e7a6ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.095+0000 7fc98bf11b80 1 --2- 172.21.15.7:0/2717066469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622e7a5d0a0 0x5622e7a5e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.095+0000 7fc95eff5700 0 client.29474 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:26:06.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.095+0000 7fc98bf11b80 1 -- 172.21.15.7:0/2717066469 >> 172.21.15.7:0/2717066469 conn(0x5622e7985080 msgr2=0x5622e7a61b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:06.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.095+0000 7fc98bf11b80 1 -- 172.21.15.7:0/2717066469 shutdown_connections 2024-06-10T01:26:06.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.095+0000 7fc98bf11b80 1 -- 172.21.15.7:0/2717066469 wait complete. 2024-06-10T01:26:06.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.095+0000 7fc98bf11b80 20 client.29474 trim_cache size 0 max 16384 2024-06-10T01:26:06.092 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:26:06.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.096+0000 7fc97bfff700 1 -- 172.21.15.7:0/3876315090 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc9640a2ca0 msgr2=0x7fc9640a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:06.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.096+0000 7fc97bfff700 1 --2- 172.21.15.7:0/3876315090 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc9640a2ca0 0x7fc9640a5160 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc97c065ce0 tx=0x7fc97c050b00 comp rx=0 tx=0).stop 2024-06-10T01:26:06.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.096+0000 7fc97bfff700 1 -- 172.21.15.7:0/3876315090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc974004d40 msgr2=0x7fc9741387c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:06.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.096+0000 7fc97bfff700 1 --2- 172.21.15.7:0/3876315090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc974004d40 0x7fc9741387c0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fc96c009230 tx=0x7fc96c00fa70 comp rx=0 tx=0).stop 2024-06-10T01:26:06.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.097+0000 7fc97bfff700 1 -- 172.21.15.7:0/3876315090 shutdown_connections 2024-06-10T01:26:06.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.097+0000 7fc97bfff700 1 --2- 172.21.15.7:0/3876315090 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc9640a2ca0 0x7fc9640a5160 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.097+0000 7fc97bfff700 1 --2- 172.21.15.7:0/3876315090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc97409be40 0x7fc9740a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.097+0000 7fc97bfff700 1 --2- 172.21.15.7:0/3876315090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc974004d40 0x7fc9741387c0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.097+0000 7fc97bfff700 1 --2- 172.21.15.7:0/3876315090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc974005700 0x7fc974138d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.097+0000 7fc97bfff700 1 -- 172.21.15.7:0/3876315090 >> 172.21.15.7:0/3876315090 conn(0x7fc97400a7a0 msgr2=0x7fc974093aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:06.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.097+0000 7fc97bfff700 1 -- 172.21.15.7:0/3876315090 shutdown_connections 2024-06-10T01:26:06.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.097+0000 7fc97bfff700 1 -- 172.21.15.7:0/3876315090 wait complete. 2024-06-10T01:26:06.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.203+0000 7fb7c5ffb700 1 -- 172.21.15.7:0/637653413 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c6a09ebea0 con 0x55c6a0aa6dc0 2024-06-10T01:26:06.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.203+0000 7fb7c5ffb700 10 client.20235 handle_command_reply: tid=0 2024-06-10T01:26:06.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.203+0000 7fb7eec30b80 1 client.20235 shutdown 2024-06-10T01:26:06.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.203+0000 7fb7eec30b80 1 -- 172.21.15.7:0/637653413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6a0a935f0 msgr2=0x55c6a0a9c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:06.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.203+0000 7fb7eec30b80 1 --2- 172.21.15.7:0/637653413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6a0a935f0 0x55c6a0a9c4d0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc074690 tx=0x7fb7d4004e10 comp rx=0 tx=0).stop 2024-06-10T01:26:06.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.203+0000 7fb7eec30b80 1 -- 172.21.15.7:0/637653413 shutdown_connections 2024-06-10T01:26:06.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.203+0000 7fb7eec30b80 1 --2- 172.21.15.7:0/637653413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6a0a935f0 0x55c6a0a9c4d0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.203+0000 7fb7eec30b80 1 --2- 172.21.15.7:0/637653413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6a0a8d460 0x55c6a0a9f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.203+0000 7fb7eec30b80 1 --2- 172.21.15.7:0/637653413 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c6a0aa6dc0 0x55c6a0b14ba0 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.203+0000 7fb7eec30b80 1 --2- 172.21.15.7:0/637653413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6a0a92cd0 0x55c6a0a9fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.203+0000 7fb7c5ffb700 0 client.20235 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:06.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.203+0000 7fb7eec30b80 1 -- 172.21.15.7:0/637653413 >> 172.21.15.7:0/637653413 conn(0x55c6a09b7e50 msgr2=0x55c6a0a94260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:06.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.203+0000 7fb7eec30b80 1 -- 172.21.15.7:0/637653413 shutdown_connections 2024-06-10T01:26:06.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.203+0000 7fb7eec30b80 1 -- 172.21.15.7:0/637653413 wait complete. 2024-06-10T01:26:06.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.203+0000 7fb7eec30b80 20 client.20235 trim_cache size 0 max 16384 2024-06-10T01:26:06.197 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:26:06.197 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 52, 2024-06-10T01:26:06.198 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:26:06.198 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 62, 2024-06-10T01:26:06.198 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:26:06.198 INFO:teuthology.orchestra.run.smithi007.stdout: "d3f8e8a1": { 2024-06-10T01:26:06.198 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 62, 2024-06-10T01:26:06.198 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:26:06.198 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:26:06.198 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:26:06.198 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:26:06.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:26:06.198 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:26:06.198 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 142.0, 2024-06-10T01:26:06.198 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:26:06.199 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:26:06.199 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:26:06.199 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:26:06.199 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:26:06.199 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:26:06.199 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:26:06.199 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:26:06.199 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:26:06.199 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:26:06.199 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:26:06.199 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:26:06.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.204+0000 7fb7e43ad700 1 -- 172.21.15.7:0/1856152466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb7c80a2f00 msgr2=0x7fb7c80a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:06.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.204+0000 7fb7e43ad700 1 --2- 172.21.15.7:0/1856152466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb7c80a2f00 0x7fb7c80a53c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc06e300 tx=0x7fb7cc009260 comp rx=0 tx=0).stop 2024-06-10T01:26:06.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.204+0000 7fb7e43ad700 1 -- 172.21.15.7:0/1856152466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7dc108ac0 msgr2=0x7fb7dc073ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:06.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.204+0000 7fb7e43ad700 1 --2- 172.21.15.7:0/1856152466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7dc108ac0 0x7fb7dc073ab0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb7d800b530 tx=0x7fb7d800b840 comp rx=0 tx=0).stop 2024-06-10T01:26:06.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.204+0000 7fb7e43ad700 1 -- 172.21.15.7:0/1856152466 shutdown_connections 2024-06-10T01:26:06.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.204+0000 7fb7e43ad700 1 --2- 172.21.15.7:0/1856152466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb7c80a2f00 0x7fb7c80a53c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.204+0000 7fb7e43ad700 1 --2- 172.21.15.7:0/1856152466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7dc10eac0 0x7fb7dc06cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.204+0000 7fb7e43ad700 1 --2- 172.21.15.7:0/1856152466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7dc108ac0 0x7fb7dc073ab0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.204+0000 7fb7e43ad700 1 --2- 172.21.15.7:0/1856152466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7dc109480 0x7fb7dc073ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:06.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.204+0000 7fb7e43ad700 1 -- 172.21.15.7:0/1856152466 >> 172.21.15.7:0/1856152466 conn(0x7fb7dc075c60 msgr2=0x7fb7dc076140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:06.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.204+0000 7fb7e43ad700 1 -- 172.21.15.7:0/1856152466 shutdown_connections 2024-06-10T01:26:06.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:06.204+0000 7fb7e43ad700 1 -- 172.21.15.7:0/1856152466 wait complete. 2024-06-10T01:26:06.541 INFO:tasks.workunit.client.0.smithi007.stdout: 10 170372 24.73 MB/sec execute 283 sec latency 882.347 ms 2024-06-10T01:26:07.143 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd3f8e8a1', quiesce took 13.1 seconds. Will release after: 8.9 2024-06-10T01:26:07.541 INFO:tasks.workunit.client.0.smithi007.stdout: 10 170372 24.64 MB/sec execute 284 sec latency 1882.382 ms 2024-06-10T01:26:07.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:07 smithi078 ceph-mon[36683]: pgmap v1036: 129 pgs: 129 active+clean; 655 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.0 MiB/s wr, 580 op/s 2024-06-10T01:26:07.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:07 smithi175 ceph-mon[36636]: pgmap v1036: 129 pgs: 129 active+clean; 655 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.0 MiB/s wr, 580 op/s 2024-06-10T01:26:08.004 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:07 smithi007 ceph-mon[32398]: pgmap v1036: 129 pgs: 129 active+clean; 655 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.0 MiB/s wr, 580 op/s 2024-06-10T01:26:08.039 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:26:08.308 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:26:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.421+0000 7f7bac1ce640 1 -- 172.21.15.7:0/739123101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ba4100900 msgr2=0x7f7ba4100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.421+0000 7f7bac1ce640 1 --2- 172.21.15.7:0/739123101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ba4100900 0x7f7ba4100d20 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f7b98009900 tx=0x7f7b9802a410 comp rx=0 tx=0).stop 2024-06-10T01:26:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.422+0000 7f7bac1ce640 1 -- 172.21.15.7:0/739123101 shutdown_connections 2024-06-10T01:26:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.422+0000 7f7bac1ce640 1 --2- 172.21.15.7:0/739123101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ba41024b0 0x7f7ba4108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.422+0000 7f7bac1ce640 1 --2- 172.21.15.7:0/739123101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ba4101af0 0x7f7ba4101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.422+0000 7f7bac1ce640 1 --2- 172.21.15.7:0/739123101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ba4100900 0x7f7ba4100d20 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.422+0000 7f7bac1ce640 1 -- 172.21.15.7:0/739123101 >> 172.21.15.7:0/739123101 conn(0x7f7ba40fbe70 msgr2=0x7f7ba40fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.422+0000 7f7bac1ce640 1 -- 172.21.15.7:0/739123101 shutdown_connections 2024-06-10T01:26:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.422+0000 7f7bac1ce640 1 -- 172.21.15.7:0/739123101 wait complete. 2024-06-10T01:26:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.422+0000 7f7bac1ce640 1 Processor -- start 2024-06-10T01:26:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.422+0000 7f7bac1ce640 1 -- start start 2024-06-10T01:26:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.422+0000 7f7bac1ce640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ba4100900 0x7f7ba4197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.422+0000 7f7bac1ce640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ba4101af0 0x7f7ba41982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.422+0000 7f7bac1ce640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ba41024b0 0x7f7ba419f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.422+0000 7f7bac1ce640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba410c580 con 0x7f7ba4100900 2024-06-10T01:26:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.422+0000 7f7bac1ce640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba410c400 con 0x7f7ba4101af0 2024-06-10T01:26:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.422+0000 7f7bac1ce640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba410c700 con 0x7f7ba41024b0 2024-06-10T01:26:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.423+0000 7f7ba9f6a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ba4100900 0x7f7ba4197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.423+0000 7f7ba9f6a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ba4100900 0x7f7ba4197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58742/0 (socket says 172.21.15.7:58742) 2024-06-10T01:26:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.423+0000 7f7ba9f6a640 1 -- 172.21.15.7:0/2827900669 learned_addr learned my addr 172.21.15.7:0/2827900669 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.423+0000 7f7ba9769640 1 --2- 172.21.15.7:0/2827900669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ba4101af0 0x7f7ba41982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.423+0000 7f7baa76b640 1 --2- 172.21.15.7:0/2827900669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ba41024b0 0x7f7ba419f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.423+0000 7f7ba9f6a640 1 -- 172.21.15.7:0/2827900669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ba41024b0 msgr2=0x7f7ba419f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.423+0000 7f7ba9f6a640 1 --2- 172.21.15.7:0/2827900669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ba41024b0 0x7f7ba419f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.423+0000 7f7ba9f6a640 1 -- 172.21.15.7:0/2827900669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ba4101af0 msgr2=0x7f7ba41982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.423+0000 7f7ba9f6a640 1 --2- 172.21.15.7:0/2827900669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ba4101af0 0x7f7ba41982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.423+0000 7f7ba9f6a640 1 -- 172.21.15.7:0/2827900669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba419f970 con 0x7f7ba4100900 2024-06-10T01:26:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.423+0000 7f7baa76b640 1 --2- 172.21.15.7:0/2827900669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ba41024b0 0x7f7ba419f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:26:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.423+0000 7f7ba9f6a640 1 --2- 172.21.15.7:0/2827900669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ba4100900 0x7f7ba4197da0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f7b98031610 tx=0x7f7b980043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.423+0000 7f7b96ffd640 1 -- 172.21.15.7:0/2827900669 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9803f030 con 0x7f7ba4100900 2024-06-10T01:26:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.423+0000 7f7b96ffd640 1 -- 172.21.15.7:0/2827900669 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b98003ce0 con 0x7f7ba4100900 2024-06-10T01:26:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.423+0000 7f7bac1ce640 1 -- 172.21.15.7:0/2827900669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ba419fc00 con 0x7f7ba4100900 2024-06-10T01:26:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.423+0000 7f7b96ffd640 1 -- 172.21.15.7:0/2827900669 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b98035070 con 0x7f7ba4100900 2024-06-10T01:26:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.423+0000 7f7bac1ce640 1 -- 172.21.15.7:0/2827900669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ba41a0140 con 0x7f7ba4100900 2024-06-10T01:26:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.426+0000 7f7b96ffd640 1 -- 172.21.15.7:0/2827900669 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f7b98042030 con 0x7f7ba4100900 2024-06-10T01:26:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.426+0000 7f7baefba740 1 Processor -- start 2024-06-10T01:26:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.426+0000 7f7baefba740 1 -- start start 2024-06-10T01:26:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.426+0000 7f7baefba740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588ac902420 0x5588ac902840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.426+0000 7f7baefba740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588ac902e10 0x5588ac907910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.426+0000 7f7baefba740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588ac907f80 0x5588ac90a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.426+0000 7f7baefba740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5588ac8614f0 con 0x5588ac907f80 2024-06-10T01:26:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.426+0000 7f7baefba740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5588ac7ff870 con 0x5588ac902e10 2024-06-10T01:26:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.426+0000 7f7baefba740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5588ac85d4b0 con 0x5588ac902420 2024-06-10T01:26:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.427+0000 7f7ba9769640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588ac907f80 0x5588ac90a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.427+0000 7f7ba9769640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588ac907f80 0x5588ac90a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58754/0 (socket says 172.21.15.7:58754) 2024-06-10T01:26:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.427+0000 7f7ba9769640 1 -- 172.21.15.7:0/3696188931 learned_addr learned my addr 172.21.15.7:0/3696188931 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.427+0000 7f7baa76b640 1 --2- 172.21.15.7:0/3696188931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588ac902420 0x5588ac902840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.427+0000 7f7ba9f6a640 1 --2- 172.21.15.7:0/3696188931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588ac902e10 0x5588ac907910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.427+0000 7f7b96ffd640 1 --2- 172.21.15.7:0/2827900669 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7b900a2c00 0x7f7b900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.427+0000 7f7ba9769640 1 -- 172.21.15.7:0/3696188931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588ac902420 msgr2=0x5588ac902840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.427+0000 7f7ba9769640 1 --2- 172.21.15.7:0/3696188931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588ac902420 0x5588ac902840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.427+0000 7f7ba9769640 1 -- 172.21.15.7:0/3696188931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588ac902e10 msgr2=0x5588ac907910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.427+0000 7f7ba9769640 1 --2- 172.21.15.7:0/3696188931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588ac902e10 0x5588ac907910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.427+0000 7f7ba9769640 1 -- 172.21.15.7:0/3696188931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588ac81b9b0 con 0x5588ac907f80 2024-06-10T01:26:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.427+0000 7f7ba9f6a640 1 --2- 172.21.15.7:0/3696188931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588ac902e10 0x5588ac907910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:26:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.427+0000 7f7baa76b640 1 --2- 172.21.15.7:0/2827900669 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7b900a2c00 0x7f7b900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.427+0000 7f7b96ffd640 1 -- 172.21.15.7:0/2827900669 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f7b980ef5b0 con 0x7f7ba4100900 2024-06-10T01:26:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.427+0000 7f7ba9769640 1 --2- 172.21.15.7:0/3696188931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588ac907f80 0x5588ac90a410 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000bcb0 tx=0x7f7ba00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c1eb28a563ee738 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.427+0000 7f7b8ffff640 1 -- 172.21.15.7:0/3696188931 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0004030 con 0x5588ac907f80 2024-06-10T01:26:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.427+0000 7f7b8ffff640 1 -- 172.21.15.7:0/3696188931 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ba00041d0 con 0x5588ac907f80 2024-06-10T01:26:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.427+0000 7f7b8ffff640 1 -- 172.21.15.7:0/3696188931 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba00117b0 con 0x5588ac907f80 2024-06-10T01:26:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.428+0000 7f7baefba740 1 -- 172.21.15.7:0/3696188931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588ac907f80 msgr2=0x5588ac90a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.428+0000 7f7baefba740 1 --2- 172.21.15.7:0/3696188931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588ac907f80 0x5588ac90a410 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000bcb0 tx=0x7f7ba00107a0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.428+0000 7f7baa76b640 1 --2- 172.21.15.7:0/2827900669 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7b900a2c00 0x7f7b900a50c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x5588ac903670 tx=0x7f7b9c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.428+0000 7f7baefba740 1 -- 172.21.15.7:0/3696188931 shutdown_connections 2024-06-10T01:26:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.428+0000 7f7baefba740 1 --2- 172.21.15.7:0/3696188931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588ac907f80 0x5588ac90a410 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.428+0000 7f7baefba740 1 --2- 172.21.15.7:0/3696188931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588ac902e10 0x5588ac907910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.428+0000 7f7baefba740 1 --2- 172.21.15.7:0/3696188931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588ac902420 0x5588ac902840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.428+0000 7f7baefba740 1 -- 172.21.15.7:0/3696188931 >> 172.21.15.7:0/3696188931 conn(0x5588ac8f9680 msgr2=0x5588ac8f9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.428+0000 7f7baefba740 1 -- 172.21.15.7:0/3696188931 shutdown_connections 2024-06-10T01:26:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.428+0000 7f7baefba740 1 -- 172.21.15.7:0/3696188931 wait complete. 2024-06-10T01:26:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.429+0000 7f7baefba740 1 -- start start 2024-06-10T01:26:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.429+0000 7f7baefba740 1 Processor -- start 2024-06-10T01:26:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.429+0000 7f7baefba740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588ac902420 0x5588ac90c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.429+0000 7f7baefba740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588ac902e10 0x5588ac90cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.429+0000 7f7baefba740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588ac907f80 0x5588ac913be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.429+0000 7f7baefba740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5588ac90af30 con 0x5588ac902e10 2024-06-10T01:26:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.429+0000 7f7baefba740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5588ac7eaae0 con 0x5588ac902420 2024-06-10T01:26:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.429+0000 7f7baefba740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5588ac90b0b0 con 0x5588ac907f80 2024-06-10T01:26:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.429+0000 7f7baefba740 10 client.0 _ll_register_callbacks cb 0x5588ac8239b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:26:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.429+0000 7f7ba9769640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588ac902e10 0x5588ac90cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.429+0000 7f7ba9769640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588ac902e10 0x5588ac90cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58758/0 (socket says 172.21.15.7:58758) 2024-06-10T01:26:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.429+0000 7f7ba9769640 1 -- 172.21.15.7:0/2413787185 learned_addr learned my addr 172.21.15.7:0/2413787185 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.429+0000 7f7baa76b640 1 --2- 172.21.15.7:0/2413787185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588ac907f80 0x5588ac913be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.429+0000 7f7ba9f6a640 1 --2- 172.21.15.7:0/2413787185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588ac902420 0x5588ac90c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.430+0000 7f7ba9769640 1 -- 172.21.15.7:0/2413787185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588ac907f80 msgr2=0x5588ac913be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.430+0000 7f7ba9769640 1 --2- 172.21.15.7:0/2413787185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588ac907f80 0x5588ac913be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.430+0000 7f7ba9769640 1 -- 172.21.15.7:0/2413787185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588ac902420 msgr2=0x5588ac90c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.430+0000 7f7ba9769640 1 --2- 172.21.15.7:0/2413787185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588ac902420 0x5588ac90c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.430+0000 7f7ba9769640 1 -- 172.21.15.7:0/2413787185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588ac575210 con 0x5588ac902e10 2024-06-10T01:26:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.430+0000 7f7ba9769640 1 --2- 172.21.15.7:0/2413787185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588ac902e10 0x5588ac90cba0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000b620 tx=0x7f7ba000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f58b7e405bf14e26 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.430+0000 7f7b8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:26:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.430+0000 7f7b8d7fa640 1 -- 172.21.15.7:0/2413787185 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba002a060 con 0x5588ac902e10 2024-06-10T01:26:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.430+0000 7f7baefba740 1 -- 172.21.15.7:0/2413787185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ba00041d0 con 0x5588ac902e10 2024-06-10T01:26:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.430+0000 7f7b8d7fa640 1 -- 172.21.15.7:0/2413787185 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f7ba00035f0 con 0x5588ac902e10 2024-06-10T01:26:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.430+0000 7f7b8d7fa640 1 -- 172.21.15.7:0/2413787185 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f7ba0031050 con 0x5588ac902e10 2024-06-10T01:26:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.430+0000 7f7baefba740 10 client.20259 fetch_fsmap learned FSMap version 52 2024-06-10T01:26:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.430+0000 7f7baefba740 1 -- 172.21.15.7:0/2413787185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5588ac641620 con 0x5588ac902e10 2024-06-10T01:26:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.430+0000 7f7b8d7fa640 1 -- 172.21.15.7:0/2413787185 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ba00041d0 con 0x5588ac902e10 2024-06-10T01:26:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.431+0000 7f7b8d7fa640 1 -- 172.21.15.7:0/2413787185 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f7ba0004a70 con 0x5588ac902e10 2024-06-10T01:26:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.431+0000 7f7baefba740 10 client.20259 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:26:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.431+0000 7f7baefba740 10 client.20259 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:26:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.431+0000 7f7baefba740 20 client.20259 populate_metadata read hostname 'smithi007' 2024-06-10T01:26:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.431+0000 7f7baefba740 1 --2- 172.21.15.7:0/2413787185 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5588ac918180 0x5588ac91a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.431+0000 7f7baefba740 4 client.20259 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:26:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.431+0000 7f7baefba740 1 -- 172.21.15.7:0/2413787185 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588ac874f10 con 0x5588ac918180 2024-06-10T01:26:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.431+0000 7f7ba9f6a640 1 --2- 172.21.15.7:0/2413787185 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5588ac918180 0x5588ac91a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.431+0000 7f7ba9f6a640 1 --2- 172.21.15.7:0/2413787185 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5588ac918180 0x5588ac91a570 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59e310802d15e972 server_cookie=5b22b2e39993c5fd in_seq=0 out_seq=0 2024-06-10T01:26:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.431+0000 7f7b8d7fa640 10 client.20259 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.432+0000 7f7b8d7fa640 1 -- 172.21.15.7:0/2413787185 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5588ac874f10 con 0x5588ac918180 2024-06-10T01:26:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.432+0000 7f7b8d7fa640 10 client.20259 handle_command_reply: tid=0 2024-06-10T01:26:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.432+0000 7f7baefba740 1 client.20259 shutdown 2024-06-10T01:26:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.432+0000 7f7baefba740 1 -- 172.21.15.7:0/2413787185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588ac902e10 msgr2=0x5588ac90cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.432+0000 7f7baefba740 1 --2- 172.21.15.7:0/2413787185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588ac902e10 0x5588ac90cba0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000b620 tx=0x7f7ba000bfd0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.432+0000 7f7baefba740 1 -- 172.21.15.7:0/2413787185 shutdown_connections 2024-06-10T01:26:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.432+0000 7f7baefba740 1 --2- 172.21.15.7:0/2413787185 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5588ac918180 0x5588ac91a570 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.432+0000 7f7baefba740 1 --2- 172.21.15.7:0/2413787185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588ac907f80 0x5588ac913be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.432+0000 7f7baefba740 1 --2- 172.21.15.7:0/2413787185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588ac902e10 0x5588ac90cba0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.432+0000 7f7b8d7fa640 0 client.20259 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.432+0000 7f7baefba740 1 --2- 172.21.15.7:0/2413787185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588ac902420 0x5588ac90c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.432+0000 7f7baefba740 1 -- 172.21.15.7:0/2413787185 >> 172.21.15.7:0/2413787185 conn(0x5588ac8f9680 msgr2=0x5588ac8f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.432+0000 7f7baefba740 1 -- 172.21.15.7:0/2413787185 shutdown_connections 2024-06-10T01:26:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.432+0000 7f7baefba740 1 -- 172.21.15.7:0/2413787185 wait complete. 2024-06-10T01:26:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.433+0000 7f7baefba740 20 client.20259 trim_cache size 0 max 16384 2024-06-10T01:26:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.448+0000 7f7baefba740 1 Processor -- start 2024-06-10T01:26:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.448+0000 7f7baefba740 1 -- start start 2024-06-10T01:26:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.448+0000 7f7baefba740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588ac902420 0x5588ac90bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.448+0000 7f7baefba740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588ac902e10 0x5588ac913ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.448+0000 7f7baefba740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588ac907f80 0x5588ac903ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.448+0000 7f7baefba740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba41aaca0 con 0x5588ac907f80 2024-06-10T01:26:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.448+0000 7f7baefba740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba41a8570 con 0x5588ac902e10 2024-06-10T01:26:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.448+0000 7f7baefba740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba41a63c0 con 0x5588ac902420 2024-06-10T01:26:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.449+0000 7f7baa76b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588ac907f80 0x5588ac903ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.449+0000 7f7ba9769640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588ac902e10 0x5588ac913ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.449+0000 7f7baa76b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588ac907f80 0x5588ac903ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58768/0 (socket says 172.21.15.7:58768) 2024-06-10T01:26:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.449+0000 7f7baa76b640 1 -- 172.21.15.7:0/2359086687 learned_addr learned my addr 172.21.15.7:0/2359086687 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.449+0000 7f7ba9f6a640 1 --2- 172.21.15.7:0/2359086687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588ac902420 0x5588ac90bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.449+0000 7f7ba9769640 1 -- 172.21.15.7:0/2359086687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588ac902420 msgr2=0x5588ac90bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.449+0000 7f7ba9769640 1 --2- 172.21.15.7:0/2359086687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588ac902420 0x5588ac90bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.449+0000 7f7ba9769640 1 -- 172.21.15.7:0/2359086687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588ac907f80 msgr2=0x5588ac903ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.449+0000 7f7ba9769640 1 --2- 172.21.15.7:0/2359086687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588ac907f80 0x5588ac903ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.449+0000 7f7ba9769640 1 -- 172.21.15.7:0/2359086687 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588ac8f94b0 con 0x5588ac902e10 2024-06-10T01:26:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.449+0000 7f7baa76b640 1 --2- 172.21.15.7:0/2359086687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588ac907f80 0x5588ac903ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:26:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.449+0000 7f7ba9769640 1 --2- 172.21.15.7:0/2359086687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588ac902e10 0x5588ac913ba0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0004ec0 tx=0x7f7ba0010ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1e9ec83a1a972ca server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.449+0000 7f7b8f7fe640 1 -- 172.21.15.7:0/2359086687 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba002a070 con 0x5588ac902e10 2024-06-10T01:26:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.450+0000 7f7b8f7fe640 1 -- 172.21.15.7:0/2359086687 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ba00035f0 con 0x5588ac902e10 2024-06-10T01:26:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.450+0000 7f7b8f7fe640 1 -- 172.21.15.7:0/2359086687 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0011cb0 con 0x5588ac902e10 2024-06-10T01:26:08.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.450+0000 7f7baefba740 1 -- 172.21.15.7:0/2359086687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588ac902e10 msgr2=0x5588ac913ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:08.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.450+0000 7f7baefba740 1 --2- 172.21.15.7:0/2359086687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588ac902e10 0x5588ac913ba0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0004ec0 tx=0x7f7ba0010ca0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.450+0000 7f7baefba740 1 -- 172.21.15.7:0/2359086687 shutdown_connections 2024-06-10T01:26:08.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.450+0000 7f7baefba740 1 --2- 172.21.15.7:0/2359086687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588ac907f80 0x5588ac903ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.450+0000 7f7baefba740 1 --2- 172.21.15.7:0/2359086687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588ac902e10 0x5588ac913ba0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.450+0000 7f7baefba740 1 --2- 172.21.15.7:0/2359086687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588ac902420 0x5588ac90bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.450+0000 7f7baefba740 1 -- 172.21.15.7:0/2359086687 >> 172.21.15.7:0/2359086687 conn(0x5588ac8f9680 msgr2=0x5588ac907660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:08.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.450+0000 7f7baefba740 1 -- 172.21.15.7:0/2359086687 shutdown_connections 2024-06-10T01:26:08.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.450+0000 7f7baefba740 1 -- 172.21.15.7:0/2359086687 wait complete. 2024-06-10T01:26:08.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.451+0000 7f7baefba740 1 -- start start 2024-06-10T01:26:08.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.451+0000 7f7baefba740 1 Processor -- start 2024-06-10T01:26:08.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.451+0000 7f7baefba740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588ac902420 0x5588ac910a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:08.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.451+0000 7f7baefba740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588ac902e10 0x5588ac910fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:08.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.451+0000 7f7baefba740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588ac907f80 0x5588ac980db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:08.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.451+0000 7f7baefba740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba41a6a30 con 0x5588ac907f80 2024-06-10T01:26:08.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.451+0000 7f7baefba740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba41a7ea0 con 0x5588ac902e10 2024-06-10T01:26:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.451+0000 7f7baefba740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba41a7100 con 0x5588ac902420 2024-06-10T01:26:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.451+0000 7f7baa76b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588ac907f80 0x5588ac980db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.451+0000 7f7baefba740 10 client.0 _ll_register_callbacks cb 0x7f7b9c0096a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:26:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.452+0000 7f7baa76b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588ac907f80 0x5588ac980db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58778/0 (socket says 172.21.15.7:58778) 2024-06-10T01:26:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.452+0000 7f7baa76b640 1 -- 172.21.15.7:0/1466766930 learned_addr learned my addr 172.21.15.7:0/1466766930 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.452+0000 7f7ba9f6a640 1 --2- 172.21.15.7:0/1466766930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588ac902420 0x5588ac910a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.452+0000 7f7ba9769640 1 --2- 172.21.15.7:0/1466766930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588ac902e10 0x5588ac910fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.452+0000 7f7baa76b640 1 -- 172.21.15.7:0/1466766930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588ac902420 msgr2=0x5588ac910a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.452+0000 7f7baa76b640 1 --2- 172.21.15.7:0/1466766930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588ac902420 0x5588ac910a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.452+0000 7f7baa76b640 1 -- 172.21.15.7:0/1466766930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588ac902e10 msgr2=0x5588ac910fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.452+0000 7f7baa76b640 1 --2- 172.21.15.7:0/1466766930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588ac902e10 0x5588ac910fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.452+0000 7f7baa76b640 1 -- 172.21.15.7:0/1466766930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588ac903ae0 con 0x5588ac907f80 2024-06-10T01:26:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.452+0000 7f7ba9769640 1 --2- 172.21.15.7:0/1466766930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588ac902e10 0x5588ac910fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:26:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.452+0000 7f7baa76b640 1 --2- 172.21.15.7:0/1466766930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588ac907f80 0x5588ac980db0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c010ab0 tx=0x7f7b9c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7381fba8eb5dfc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.452+0000 7f7b8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:26:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.452+0000 7f7b8d7fa640 1 -- 172.21.15.7:0/1466766930 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c02d020 con 0x5588ac907f80 2024-06-10T01:26:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.452+0000 7f7b8d7fa640 1 -- 172.21.15.7:0/1466766930 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f7b9c01da00 con 0x5588ac907f80 2024-06-10T01:26:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.452+0000 7f7baefba740 1 -- 172.21.15.7:0/1466766930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ba00035f0 con 0x5588ac907f80 2024-06-10T01:26:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.452+0000 7f7b8d7fa640 1 -- 172.21.15.7:0/1466766930 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f7b9c038050 con 0x5588ac907f80 2024-06-10T01:26:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.452+0000 7f7baefba740 10 client.20265 fetch_fsmap learned FSMap version 52 2024-06-10T01:26:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.452+0000 7f7baefba740 1 -- 172.21.15.7:0/1466766930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5588ac9824b0 con 0x5588ac907f80 2024-06-10T01:26:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.453+0000 7f7b8d7fa640 1 -- 172.21.15.7:0/1466766930 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ba00035f0 con 0x5588ac907f80 2024-06-10T01:26:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.453+0000 7f7b8d7fa640 1 -- 172.21.15.7:0/1466766930 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f7b9c00f040 con 0x5588ac907f80 2024-06-10T01:26:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.453+0000 7f7baefba740 10 client.20265 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:26:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.453+0000 7f7baefba740 10 client.20265 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:26:08.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.453+0000 7f7baefba740 20 client.20265 populate_metadata read hostname 'smithi007' 2024-06-10T01:26:08.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.453+0000 7f7baefba740 1 --2- 172.21.15.7:0/1466766930 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5588ac982e80 0x5588ac9852a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:08.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.453+0000 7f7baefba740 4 client.20265 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:26:08.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.453+0000 7f7baefba740 1 -- 172.21.15.7:0/1466766930 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5588ac86ec20 con 0x5588ac982e80 2024-06-10T01:26:08.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.453+0000 7f7ba9f6a640 1 --2- 172.21.15.7:0/1466766930 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5588ac982e80 0x5588ac9852a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:08.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.454+0000 7f7ba9f6a640 1 --2- 172.21.15.7:0/1466766930 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5588ac982e80 0x5588ac9852a0 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8cf13f5e3167876 server_cookie=b7e2f9ca4d719dd9 in_seq=0 out_seq=0 2024-06-10T01:26:08.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:08.454+0000 7f7b8d7fa640 10 client.20265 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:08.542 INFO:tasks.workunit.client.0.smithi007.stdout: 10 170372 24.56 MB/sec execute 285 sec latency 2882.428 ms 2024-06-10T01:26:08.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:09.000+0000 7f7b8d7fa640 1 -- 172.21.15.7:0/1466766930 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5588ac86ec20 con 0x5588ac982e80 2024-06-10T01:26:08.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:09.000+0000 7f7b8d7fa640 10 client.20265 handle_command_reply: tid=0 2024-06-10T01:26:08.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:09.000+0000 7f7baefba740 1 client.20265 shutdown 2024-06-10T01:26:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:09.000+0000 7f7baefba740 1 -- 172.21.15.7:0/1466766930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588ac907f80 msgr2=0x5588ac980db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:09.000+0000 7f7baefba740 1 --2- 172.21.15.7:0/1466766930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588ac907f80 0x5588ac980db0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c010ab0 tx=0x7f7b9c004ab0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:09.001+0000 7f7baefba740 1 -- 172.21.15.7:0/1466766930 shutdown_connections 2024-06-10T01:26:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:09.001+0000 7f7baefba740 1 --2- 172.21.15.7:0/1466766930 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5588ac982e80 0x5588ac9852a0 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:09.001+0000 7f7b8d7fa640 0 client.20265 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:09.001+0000 7f7baefba740 1 --2- 172.21.15.7:0/1466766930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588ac907f80 0x5588ac980db0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:09.001+0000 7f7baefba740 1 --2- 172.21.15.7:0/1466766930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588ac902e10 0x5588ac910fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:09.001+0000 7f7baefba740 1 --2- 172.21.15.7:0/1466766930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588ac902420 0x5588ac910a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:09.001+0000 7f7baefba740 1 -- 172.21.15.7:0/1466766930 >> 172.21.15.7:0/1466766930 conn(0x5588ac8f9680 msgr2=0x5588ac90c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:09.001+0000 7f7baefba740 1 -- 172.21.15.7:0/1466766930 shutdown_connections 2024-06-10T01:26:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:09.001+0000 7f7baefba740 1 -- 172.21.15.7:0/1466766930 wait complete. 2024-06-10T01:26:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:09.001+0000 7f7baefba740 20 client.20265 trim_cache size 0 max 16384 2024-06-10T01:26:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:26:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:26:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:26:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:09.002+0000 7f7bac1ce640 1 -- 172.21.15.7:0/2827900669 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7b900a2c00 msgr2=0x7f7b900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:09.002+0000 7f7bac1ce640 1 --2- 172.21.15.7:0/2827900669 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7b900a2c00 0x7f7b900a50c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x5588ac903670 tx=0x7f7b9c004a40 comp rx=0 tx=0).stop 2024-06-10T01:26:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:09.002+0000 7f7bac1ce640 1 -- 172.21.15.7:0/2827900669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ba4100900 msgr2=0x7f7ba4197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:09.002+0000 7f7bac1ce640 1 --2- 172.21.15.7:0/2827900669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ba4100900 0x7f7ba4197da0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f7b98031610 tx=0x7f7b980043b0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:09.002+0000 7f7bac1ce640 1 -- 172.21.15.7:0/2827900669 shutdown_connections 2024-06-10T01:26:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:09.002+0000 7f7bac1ce640 1 --2- 172.21.15.7:0/2827900669 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7b900a2c00 0x7f7b900a50c0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:09.002+0000 7f7bac1ce640 1 --2- 172.21.15.7:0/2827900669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ba41024b0 0x7f7ba419f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:09.002+0000 7f7bac1ce640 1 --2- 172.21.15.7:0/2827900669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ba4101af0 0x7f7ba41982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:09.002+0000 7f7bac1ce640 1 --2- 172.21.15.7:0/2827900669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ba4100900 0x7f7ba4197da0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:09.002+0000 7f7bac1ce640 1 -- 172.21.15.7:0/2827900669 >> 172.21.15.7:0/2827900669 conn(0x7f7ba40fbe70 msgr2=0x7f7ba40fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:09.002+0000 7f7bac1ce640 1 -- 172.21.15.7:0/2827900669 shutdown_connections 2024-06-10T01:26:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:09.002+0000 7f7bac1ce640 1 -- 172.21.15.7:0/2827900669 wait complete. 2024-06-10T01:26:09.541 INFO:tasks.workunit.client.0.smithi007.stdout: 10 170372 24.47 MB/sec execute 286 sec latency 3882.471 ms 2024-06-10T01:26:09.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:09 smithi078 ceph-mon[36683]: pgmap v1037: 129 pgs: 129 active+clean; 660 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.4 MiB/s wr, 498 op/s 2024-06-10T01:26:09.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:09 smithi175 ceph-mon[36636]: pgmap v1037: 129 pgs: 129 active+clean; 660 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.4 MiB/s wr, 498 op/s 2024-06-10T01:26:10.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:09 smithi007 ceph-mon[32398]: pgmap v1037: 129 pgs: 129 active+clean; 660 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.4 MiB/s wr, 498 op/s 2024-06-10T01:26:10.545 INFO:tasks.workunit.client.0.smithi007.stdout: 10 170372 24.39 MB/sec execute 287 sec latency 4885.431 ms 2024-06-10T01:26:10.709 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:10 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:26:10.709 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:10 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:26:10.709 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:10 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:26:10.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:10 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:26:10.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:10 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:26:10.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:10 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:26:10.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:10 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:26:10.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:10 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:26:10.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:10 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:26:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.171+0000 7fa5f09e8700 1 -- 172.21.15.7:0/1911618621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5ec102ee0 msgr2=0x7fa5ec10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.171+0000 7fa5f09e8700 1 --2- 172.21.15.7:0/1911618621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5ec102ee0 0x7fa5ec10f3c0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400b0a0 tx=0x7fa5e400b3b0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.171+0000 7fa5f09e8700 1 -- 172.21.15.7:0/1911618621 shutdown_connections 2024-06-10T01:26:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.171+0000 7fa5f09e8700 1 --2- 172.21.15.7:0/1911618621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5ec102ee0 0x7fa5ec10f3c0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.171+0000 7fa5f09e8700 1 --2- 172.21.15.7:0/1911618621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5ec107450 0x7fa5ec107830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.171+0000 7fa5f09e8700 1 --2- 172.21.15.7:0/1911618621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5ec102540 0x7fa5ec1029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.171+0000 7fa5f09e8700 1 -- 172.21.15.7:0/1911618621 >> 172.21.15.7:0/1911618621 conn(0x7fa5ec0795f0 msgr2=0x7fa5ec079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.171+0000 7fa5f09e8700 1 -- 172.21.15.7:0/1911618621 shutdown_connections 2024-06-10T01:26:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.171+0000 7fa5f09e8700 1 -- 172.21.15.7:0/1911618621 wait complete. 2024-06-10T01:26:11.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.172+0000 7fa5f09e8700 1 Processor -- start 2024-06-10T01:26:11.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.172+0000 7fa5f09e8700 1 -- start start 2024-06-10T01:26:11.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.173+0000 7fa5f09e8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5ec102540 0x7fa5ec19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:11.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.173+0000 7fa5f09e8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5ec102ee0 0x7fa5ec19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:11.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.173+0000 7fa5f09e8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5ec107450 0x7fa5ec198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:11.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.173+0000 7fa5ea59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5ec102540 0x7fa5ec19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:11.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.173+0000 7fa5ea59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5ec102540 0x7fa5ec19e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:34008/0 (socket says 172.21.15.7:34008) 2024-06-10T01:26:11.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.173+0000 7fa5e9d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5ec102ee0 0x7fa5ec19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:11.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.173+0000 7fa5e9d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5ec102ee0 0x7fa5ec19eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:38106/0 (socket says 172.21.15.7:38106) 2024-06-10T01:26:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.173+0000 7fa5ea59c700 1 -- 172.21.15.7:0/3700268024 learned_addr learned my addr 172.21.15.7:0/3700268024 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.173+0000 7fa5f09e8700 1 -- 172.21.15.7:0/3700268024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ec112830 con 0x7fa5ec107450 2024-06-10T01:26:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.173+0000 7fa5f09e8700 1 -- 172.21.15.7:0/3700268024 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ec1126b0 con 0x7fa5ec102540 2024-06-10T01:26:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.173+0000 7fa5ead9d700 1 --2- 172.21.15.7:0/3700268024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5ec107450 0x7fa5ec198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.173+0000 7fa5f09e8700 1 -- 172.21.15.7:0/3700268024 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ec1129b0 con 0x7fa5ec102ee0 2024-06-10T01:26:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.173+0000 7fa5ea59c700 1 -- 172.21.15.7:0/3700268024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5ec102ee0 msgr2=0x7fa5ec19eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.173+0000 7fa5ea59c700 1 --2- 172.21.15.7:0/3700268024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5ec102ee0 0x7fa5ec19eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.173+0000 7fa5ea59c700 1 -- 172.21.15.7:0/3700268024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5ec107450 msgr2=0x7fa5ec198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.173+0000 7fa5ea59c700 1 --2- 172.21.15.7:0/3700268024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5ec107450 0x7fa5ec198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.173+0000 7fa5ea59c700 1 -- 172.21.15.7:0/3700268024 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ec1991d0 con 0x7fa5ec102540 2024-06-10T01:26:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.174+0000 7fa5ea59c700 1 --2- 172.21.15.7:0/3700268024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5ec102540 0x7fa5ec19e860 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc00e970 tx=0x7fa5dc00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.174+0000 7fa5db7fe700 1 -- 172.21.15.7:0/3700268024 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc004220 con 0x7fa5ec102540 2024-06-10T01:26:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.174+0000 7fa5db7fe700 1 -- 172.21.15.7:0/3700268024 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5dc0043c0 con 0x7fa5ec102540 2024-06-10T01:26:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.174+0000 7fa5db7fe700 1 -- 172.21.15.7:0/3700268024 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc003dc0 con 0x7fa5ec102540 2024-06-10T01:26:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.174+0000 7fa5f09e8700 1 -- 172.21.15.7:0/3700268024 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5ec1994c0 con 0x7fa5ec102540 2024-06-10T01:26:11.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.174+0000 7fa5f09e8700 1 -- 172.21.15.7:0/3700268024 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5ec0746c0 con 0x7fa5ec102540 2024-06-10T01:26:11.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.178+0000 7fa5db7fe700 1 -- 172.21.15.7:0/3700268024 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa5dc00c7e0 con 0x7fa5ec102540 2024-06-10T01:26:11.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.179+0000 7fa5db7fe700 1 --2- 172.21.15.7:0/3700268024 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa5d40a2c20 0x7fa5d40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:11.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.180+0000 7fa5e9d9b700 1 --2- 172.21.15.7:0/3700268024 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa5d40a2c20 0x7fa5d40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:11.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.180+0000 7fa5db7fe700 1 -- 172.21.15.7:0/3700268024 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fa5dc0cf520 con 0x7fa5ec102540 2024-06-10T01:26:11.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.180+0000 7fa5e9d9b700 1 --2- 172.21.15.7:0/3700268024 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa5d40a2c20 0x7fa5d40a50e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0009680 tx=0x7fa5e0012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:11.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.181+0000 7fa5fb26bb80 1 Processor -- start 2024-06-10T01:26:11.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.181+0000 7fa5fb26bb80 1 -- start start 2024-06-10T01:26:11.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.182+0000 7fa5fb26bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564431a8df60 0x564431a8e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:11.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.182+0000 7fa5fb26bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564431a8e880 0x564431a88260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:11.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.182+0000 7fa5fb26bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564431a887a0 0x564431a88bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:11.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.182+0000 7fa5fb26bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56443197af60 con 0x564431a887a0 2024-06-10T01:26:11.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.182+0000 7fa5fb26bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5644319d9850 con 0x564431a8df60 2024-06-10T01:26:11.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.182+0000 7fa5fb26bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564431941460 con 0x564431a8e880 2024-06-10T01:26:11.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.182+0000 7fa5ea59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564431a8df60 0x564431a8e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:11.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.182+0000 7fa5ea59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564431a8df60 0x564431a8e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:34022/0 (socket says 172.21.15.7:34022) 2024-06-10T01:26:11.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.182+0000 7fa5ea59c700 1 -- 172.21.15.7:0/2245761030 learned_addr learned my addr 172.21.15.7:0/2245761030 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:11.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.182+0000 7fa5e9d9b700 1 --2- 172.21.15.7:0/2245761030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564431a8e880 0x564431a88260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:11.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.182+0000 7fa5ead9d700 1 --2- 172.21.15.7:0/2245761030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564431a887a0 0x564431a88bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:11.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.182+0000 7fa5ea59c700 1 -- 172.21.15.7:0/2245761030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564431a8e880 msgr2=0x564431a88260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:11.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.182+0000 7fa5ea59c700 1 --2- 172.21.15.7:0/2245761030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564431a8e880 0x564431a88260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.182+0000 7fa5ea59c700 1 -- 172.21.15.7:0/2245761030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564431a887a0 msgr2=0x564431a88bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:11.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.182+0000 7fa5ea59c700 1 --2- 172.21.15.7:0/2245761030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564431a887a0 0x564431a88bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.182+0000 7fa5ea59c700 1 -- 172.21.15.7:0/2245761030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644319288d0 con 0x564431a8df60 2024-06-10T01:26:11.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.183+0000 7fa5ea59c700 1 --2- 172.21.15.7:0/2245761030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564431a8df60 0x564431a8e340 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc0983d0 tx=0x7fa5dc09b540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a58373659c0f02ce server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:11.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.183+0000 7fa5d8ff9700 1 -- 172.21.15.7:0/2245761030 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc0b4070 con 0x564431a8df60 2024-06-10T01:26:11.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.183+0000 7fa5d8ff9700 1 -- 172.21.15.7:0/2245761030 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5dc003680 con 0x564431a8df60 2024-06-10T01:26:11.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.183+0000 7fa5d8ff9700 1 -- 172.21.15.7:0/2245761030 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc099d90 con 0x564431a8df60 2024-06-10T01:26:11.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.184+0000 7fa5fb26bb80 1 -- 172.21.15.7:0/2245761030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564431a8df60 msgr2=0x564431a8e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:11.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.184+0000 7fa5fb26bb80 1 --2- 172.21.15.7:0/2245761030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564431a8df60 0x564431a8e340 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc0983d0 tx=0x7fa5dc09b540 comp rx=0 tx=0).stop 2024-06-10T01:26:11.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.184+0000 7fa5fb26bb80 1 -- 172.21.15.7:0/2245761030 shutdown_connections 2024-06-10T01:26:11.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.184+0000 7fa5fb26bb80 1 --2- 172.21.15.7:0/2245761030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564431a887a0 0x564431a88bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.184+0000 7fa5fb26bb80 1 --2- 172.21.15.7:0/2245761030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564431a8df60 0x564431a8e340 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.184+0000 7fa5fb26bb80 1 --2- 172.21.15.7:0/2245761030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564431a8e880 0x564431a88260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.184+0000 7fa5fb26bb80 1 -- 172.21.15.7:0/2245761030 >> 172.21.15.7:0/2245761030 conn(0x5644319b5080 msgr2=0x564431a83220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:11.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.184+0000 7fa5fb26bb80 1 -- 172.21.15.7:0/2245761030 shutdown_connections 2024-06-10T01:26:11.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.184+0000 7fa5fb26bb80 1 -- 172.21.15.7:0/2245761030 wait complete. 2024-06-10T01:26:11.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.185+0000 7fa5fb26bb80 1 -- start start 2024-06-10T01:26:11.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.185+0000 7fa5fb26bb80 1 Processor -- start 2024-06-10T01:26:11.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.185+0000 7fa5fb26bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564431a887a0 0x564431a94720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:11.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.185+0000 7fa5fb26bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564431a8df60 0x564431a91a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:11.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.185+0000 7fa5fb26bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564431a8e880 0x564431a91f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:11.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.185+0000 7fa5fb26bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564431979fc0 con 0x564431a8df60 2024-06-10T01:26:11.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.185+0000 7fa5fb26bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5644319d5810 con 0x564431a8e880 2024-06-10T01:26:11.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.185+0000 7fa5fb26bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564431a95260 con 0x564431a887a0 2024-06-10T01:26:11.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.185+0000 7fa5fb26bb80 10 client.0 _ll_register_callbacks cb 0x56443196f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:26:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.185+0000 7fa5ead9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564431a8e880 0x564431a91f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.186+0000 7fa5e9d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564431a8df60 0x564431a91a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.186+0000 7fa5ead9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564431a8e880 0x564431a91f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:34034/0 (socket says 172.21.15.7:34034) 2024-06-10T01:26:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.186+0000 7fa5ead9d700 1 -- 172.21.15.7:0/3249862831 learned_addr learned my addr 172.21.15.7:0/3249862831 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.186+0000 7fa5ea59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564431a887a0 0x564431a94720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.186+0000 7fa5ead9d700 1 -- 172.21.15.7:0/3249862831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564431a887a0 msgr2=0x564431a94720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.186+0000 7fa5ead9d700 1 --2- 172.21.15.7:0/3249862831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564431a887a0 0x564431a94720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.186+0000 7fa5ead9d700 1 -- 172.21.15.7:0/3249862831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564431a8df60 msgr2=0x564431a91a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.186+0000 7fa5ead9d700 1 --2- 172.21.15.7:0/3249862831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564431a8df60 0x564431a91a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.186+0000 7fa5ead9d700 1 -- 172.21.15.7:0/3249862831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56443199f750 con 0x564431a8e880 2024-06-10T01:26:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.186+0000 7fa5e9d9b700 1 --2- 172.21.15.7:0/3249862831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564431a8df60 0x564431a91a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:26:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.186+0000 7fa5ead9d700 1 --2- 172.21.15.7:0/3249862831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564431a8e880 0x564431a91f60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400b0a0 tx=0x7fa5e4020cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93d3ca091b9d64c7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:11.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.186+0000 7fa5d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:26:11.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.186+0000 7fa5d1ffb700 1 -- 172.21.15.7:0/3249862831 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e402f060 con 0x564431a8e880 2024-06-10T01:26:11.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.186+0000 7fa5fb26bb80 1 -- 172.21.15.7:0/3249862831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5644319e6d50 con 0x564431a8e880 2024-06-10T01:26:11.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.186+0000 7fa5d1ffb700 1 -- 172.21.15.7:0/3249862831 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fa5e4003c00 con 0x564431a8e880 2024-06-10T01:26:11.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.187+0000 7fa5d1ffb700 1 -- 172.21.15.7:0/3249862831 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fa5e4036050 con 0x564431a8e880 2024-06-10T01:26:11.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.187+0000 7fa5fb26bb80 10 client.29504 fetch_fsmap learned FSMap version 52 2024-06-10T01:26:11.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.187+0000 7fa5fb26bb80 1 -- 172.21.15.7:0/3249862831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x564431a9fb40 con 0x564431a8e880 2024-06-10T01:26:11.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.187+0000 7fa5d1ffb700 1 -- 172.21.15.7:0/3249862831 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5644319e6d50 con 0x564431a8e880 2024-06-10T01:26:11.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.187+0000 7fa5d1ffb700 1 -- 172.21.15.7:0/3249862831 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7fa5e4011070 con 0x564431a8e880 2024-06-10T01:26:11.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.187+0000 7fa5fb26bb80 10 client.29504 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:26:11.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.187+0000 7fa5fb26bb80 10 client.29504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:26:11.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.187+0000 7fa5fb26bb80 20 client.29504 populate_metadata read hostname 'smithi007' 2024-06-10T01:26:11.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.187+0000 7fa5fb26bb80 1 --2- 172.21.15.7:0/3249862831 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564431aa20d0 0x564431aa44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:11.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.187+0000 7fa5fb26bb80 4 client.29504 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:26:11.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.187+0000 7fa5fb26bb80 1 -- 172.21.15.7:0/3249862831 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644319e12d0 con 0x564431aa20d0 2024-06-10T01:26:11.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.187+0000 7fa5ea59c700 1 --2- 172.21.15.7:0/3249862831 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564431aa20d0 0x564431aa44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:11.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.188+0000 7fa5d1ffb700 10 client.29504 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:11.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.188+0000 7fa5ea59c700 1 --2- 172.21.15.7:0/3249862831 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564431aa20d0 0x564431aa44f0 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=763733cd6b78e368 server_cookie=accc1832e363d62e in_seq=0 out_seq=0 2024-06-10T01:26:11.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.189+0000 7fa5d1ffb700 1 -- 172.21.15.7:0/3249862831 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5644319e12d0 con 0x564431aa20d0 2024-06-10T01:26:11.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.189+0000 7fa5d1ffb700 10 client.29504 handle_command_reply: tid=0 2024-06-10T01:26:11.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.189+0000 7fa5fb26bb80 1 client.29504 shutdown 2024-06-10T01:26:11.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.189+0000 7fa5fb26bb80 1 -- 172.21.15.7:0/3249862831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564431a8e880 msgr2=0x564431a91f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:11.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.189+0000 7fa5fb26bb80 1 --2- 172.21.15.7:0/3249862831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564431a8e880 0x564431a91f60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400b0a0 tx=0x7fa5e4020cd0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.189+0000 7fa5fb26bb80 1 -- 172.21.15.7:0/3249862831 shutdown_connections 2024-06-10T01:26:11.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.189+0000 7fa5fb26bb80 1 --2- 172.21.15.7:0/3249862831 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564431aa20d0 0x564431aa44f0 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.189+0000 7fa5d1ffb700 0 client.29504 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:11.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.189+0000 7fa5fb26bb80 1 --2- 172.21.15.7:0/3249862831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564431a8e880 0x564431a91f60 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.189+0000 7fa5fb26bb80 1 --2- 172.21.15.7:0/3249862831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564431a887a0 0x564431a94720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.189+0000 7fa5fb26bb80 1 --2- 172.21.15.7:0/3249862831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564431a8df60 0x564431a91a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.189+0000 7fa5fb26bb80 1 -- 172.21.15.7:0/3249862831 >> 172.21.15.7:0/3249862831 conn(0x5644319b5080 msgr2=0x564431a82700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:11.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.189+0000 7fa5fb26bb80 1 -- 172.21.15.7:0/3249862831 shutdown_connections 2024-06-10T01:26:11.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.189+0000 7fa5fb26bb80 1 -- 172.21.15.7:0/3249862831 wait complete. 2024-06-10T01:26:11.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.189+0000 7fa5fb26bb80 20 client.29504 trim_cache size 0 max 16384 2024-06-10T01:26:11.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.221+0000 7fa5fb26bb80 1 Processor -- start 2024-06-10T01:26:11.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.221+0000 7fa5fb26bb80 1 -- start start 2024-06-10T01:26:11.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.221+0000 7fa5fb26bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564431a887a0 0x564431a88260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:11.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.221+0000 7fa5fb26bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564431a8df60 0x564431a9e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:11.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.221+0000 7fa5fb26bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564431a8e880 0x564431a90980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:11.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.221+0000 7fa5fb26bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ec1ae980 con 0x564431a887a0 2024-06-10T01:26:11.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.221+0000 7fa5fb26bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ec1ac210 con 0x564431a8df60 2024-06-10T01:26:11.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.221+0000 7fa5fb26bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ec1aa060 con 0x564431a8e880 2024-06-10T01:26:11.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.221+0000 7fa5ea59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564431a887a0 0x564431a88260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:11.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.221+0000 7fa5ea59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564431a887a0 0x564431a88260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58826/0 (socket says 172.21.15.7:58826) 2024-06-10T01:26:11.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.221+0000 7fa5ea59c700 1 -- 172.21.15.7:0/2139338621 learned_addr learned my addr 172.21.15.7:0/2139338621 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:11.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.221+0000 7fa5ead9d700 1 --2- 172.21.15.7:0/2139338621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564431a8e880 0x564431a90980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:11.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.221+0000 7fa5e9d9b700 1 --2- 172.21.15.7:0/2139338621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564431a8df60 0x564431a9e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:11.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.222+0000 7fa5ea59c700 1 -- 172.21.15.7:0/2139338621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564431a8e880 msgr2=0x564431a90980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:11.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.222+0000 7fa5ea59c700 1 --2- 172.21.15.7:0/2139338621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564431a8e880 0x564431a90980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.222+0000 7fa5ea59c700 1 -- 172.21.15.7:0/2139338621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564431a8df60 msgr2=0x564431a9e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:11.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.222+0000 7fa5ea59c700 1 --2- 172.21.15.7:0/2139338621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564431a8df60 0x564431a9e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.222+0000 7fa5ea59c700 1 -- 172.21.15.7:0/2139338621 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564431a92540 con 0x564431a887a0 2024-06-10T01:26:11.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.222+0000 7fa5ead9d700 1 --2- 172.21.15.7:0/2139338621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564431a8e880 0x564431a90980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:26:11.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.222+0000 7fa5ea59c700 1 --2- 172.21.15.7:0/2139338621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564431a887a0 0x564431a88260 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc005950 tx=0x7fa5dc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee02027353d99629 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:11.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.222+0000 7fa5d3fff700 1 -- 172.21.15.7:0/2139338621 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc065070 con 0x564431a887a0 2024-06-10T01:26:11.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.222+0000 7fa5d3fff700 1 -- 172.21.15.7:0/2139338621 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5dc09c100 con 0x564431a887a0 2024-06-10T01:26:11.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.222+0000 7fa5d3fff700 1 -- 172.21.15.7:0/2139338621 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc0b4040 con 0x564431a887a0 2024-06-10T01:26:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.223+0000 7fa5fb26bb80 1 -- 172.21.15.7:0/2139338621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564431a887a0 msgr2=0x564431a88260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.223+0000 7fa5fb26bb80 1 --2- 172.21.15.7:0/2139338621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564431a887a0 0x564431a88260 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc005950 tx=0x7fa5dc009840 comp rx=0 tx=0).stop 2024-06-10T01:26:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.223+0000 7fa5fb26bb80 1 -- 172.21.15.7:0/2139338621 shutdown_connections 2024-06-10T01:26:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.223+0000 7fa5fb26bb80 1 --2- 172.21.15.7:0/2139338621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564431a8e880 0x564431a90980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.223+0000 7fa5fb26bb80 1 --2- 172.21.15.7:0/2139338621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564431a887a0 0x564431a88260 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.223+0000 7fa5fb26bb80 1 --2- 172.21.15.7:0/2139338621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564431a8df60 0x564431a9e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.223+0000 7fa5fb26bb80 1 -- 172.21.15.7:0/2139338621 >> 172.21.15.7:0/2139338621 conn(0x5644319b5080 msgr2=0x564431a8f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.223+0000 7fa5fb26bb80 1 -- 172.21.15.7:0/2139338621 shutdown_connections 2024-06-10T01:26:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.223+0000 7fa5fb26bb80 1 -- 172.21.15.7:0/2139338621 wait complete. 2024-06-10T01:26:11.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.223+0000 7fa5fb26bb80 1 -- start start 2024-06-10T01:26:11.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.224+0000 7fa5fb26bb80 1 Processor -- start 2024-06-10T01:26:11.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.224+0000 7fa5fb26bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564431a887a0 0x564431a9a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:11.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.224+0000 7fa5fb26bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564431a8df60 0x564431a9aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:11.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.224+0000 7fa5fb26bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564431a8e880 0x564431aa8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:11.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.224+0000 7fa5fb26bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ec1aa6d0 con 0x564431a8e880 2024-06-10T01:26:11.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.224+0000 7fa5fb26bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ec1abb40 con 0x564431a887a0 2024-06-10T01:26:11.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.224+0000 7fa5fb26bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ec1aada0 con 0x564431a8df60 2024-06-10T01:26:11.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.224+0000 7fa5fb26bb80 10 client.0 _ll_register_callbacks cb 0x56443196f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:26:11.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.224+0000 7fa5ead9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564431a8e880 0x564431aa8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:11.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.224+0000 7fa5e9d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564431a8df60 0x564431a9aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:11.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.224+0000 7fa5ead9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564431a8e880 0x564431aa8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58834/0 (socket says 172.21.15.7:58834) 2024-06-10T01:26:11.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.224+0000 7fa5ea59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564431a887a0 0x564431a9a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:11.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.224+0000 7fa5ead9d700 1 -- 172.21.15.7:0/3684536098 learned_addr learned my addr 172.21.15.7:0/3684536098 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:11.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.224+0000 7fa5ea59c700 1 -- 172.21.15.7:0/3684536098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564431a8df60 msgr2=0x564431a9aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:11.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.224+0000 7fa5ea59c700 1 --2- 172.21.15.7:0/3684536098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564431a8df60 0x564431a9aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.224+0000 7fa5ea59c700 1 -- 172.21.15.7:0/3684536098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564431a8e880 msgr2=0x564431aa8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:11.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.224+0000 7fa5ea59c700 1 --2- 172.21.15.7:0/3684536098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564431a8e880 0x564431aa8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.224+0000 7fa5ea59c700 1 -- 172.21.15.7:0/3684536098 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564431a9b0d0 con 0x564431a887a0 2024-06-10T01:26:11.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.224+0000 7fa5ead9d700 1 --2- 172.21.15.7:0/3684536098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564431a8e880 0x564431aa8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:26:11.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.225+0000 7fa5ea59c700 1 --2- 172.21.15.7:0/3684536098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564431a887a0 0x564431a9a4f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x564431a94c30 tx=0x7fa5dc003860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6a027f31ec0a3f1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:11.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.225+0000 7fa5d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:26:11.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.225+0000 7fa5d1ffb700 1 -- 172.21.15.7:0/3684536098 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc065020 con 0x564431a887a0 2024-06-10T01:26:11.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.225+0000 7fa5d1ffb700 1 -- 172.21.15.7:0/3684536098 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fa5dc0ba330 con 0x564431a887a0 2024-06-10T01:26:11.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.225+0000 7fa5fb26bb80 1 -- 172.21.15.7:0/3684536098 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5dc09c100 con 0x564431a887a0 2024-06-10T01:26:11.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.225+0000 7fa5d1ffb700 1 -- 172.21.15.7:0/3684536098 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fa5dc0b4040 con 0x564431a887a0 2024-06-10T01:26:11.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.225+0000 7fa5fb26bb80 10 client.29510 fetch_fsmap learned FSMap version 52 2024-06-10T01:26:11.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.225+0000 7fa5fb26bb80 1 -- 172.21.15.7:0/3684536098 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x564431b0c350 con 0x564431a887a0 2024-06-10T01:26:11.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.225+0000 7fa5d1ffb700 1 -- 172.21.15.7:0/3684536098 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5dc09c100 con 0x564431a887a0 2024-06-10T01:26:11.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.226+0000 7fa5d1ffb700 1 -- 172.21.15.7:0/3684536098 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7fa5dc0a41e0 con 0x564431a887a0 2024-06-10T01:26:11.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.226+0000 7fa5fb26bb80 10 client.29510 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:26:11.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.226+0000 7fa5fb26bb80 10 client.29510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:26:11.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.226+0000 7fa5fb26bb80 20 client.29510 populate_metadata read hostname 'smithi007' 2024-06-10T01:26:11.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.226+0000 7fa5fb26bb80 1 --2- 172.21.15.7:0/3684536098 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564431b0c520 0x564431b0e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:11.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.226+0000 7fa5fb26bb80 4 client.29510 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:26:11.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.226+0000 7fa5fb26bb80 1 -- 172.21.15.7:0/3684536098 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564431994f50 con 0x564431b0c520 2024-06-10T01:26:11.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.226+0000 7fa5e9d9b700 1 --2- 172.21.15.7:0/3684536098 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564431b0c520 0x564431b0e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:11.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.226+0000 7fa5e9d9b700 1 --2- 172.21.15.7:0/3684536098 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564431b0c520 0x564431b0e910 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccd19286729290f6 server_cookie=cb2781e470e6da6d in_seq=0 out_seq=0 2024-06-10T01:26:11.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.226+0000 7fa5d1ffb700 10 client.29510 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:11.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.226+0000 7fa5d1ffb700 1 -- 172.21.15.7:0/3684536098 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564431994f50 con 0x564431b0c520 2024-06-10T01:26:11.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.226+0000 7fa5d1ffb700 10 client.29510 handle_command_reply: tid=0 2024-06-10T01:26:11.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.226+0000 7fa5fb26bb80 1 client.29510 shutdown 2024-06-10T01:26:11.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.227+0000 7fa5fb26bb80 1 -- 172.21.15.7:0/3684536098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564431a887a0 msgr2=0x564431a9a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:11.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.227+0000 7fa5fb26bb80 1 --2- 172.21.15.7:0/3684536098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564431a887a0 0x564431a9a4f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x564431a94c30 tx=0x7fa5dc003860 comp rx=0 tx=0).stop 2024-06-10T01:26:11.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.227+0000 7fa5fb26bb80 1 -- 172.21.15.7:0/3684536098 shutdown_connections 2024-06-10T01:26:11.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.227+0000 7fa5fb26bb80 1 --2- 172.21.15.7:0/3684536098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564431a8e880 0x564431aa8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.227+0000 7fa5fb26bb80 1 --2- 172.21.15.7:0/3684536098 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x564431b0c520 0x564431b0e910 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.227+0000 7fa5fb26bb80 1 --2- 172.21.15.7:0/3684536098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564431a887a0 0x564431a9a4f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.227+0000 7fa5fb26bb80 1 --2- 172.21.15.7:0/3684536098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564431a8df60 0x564431a9aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.227+0000 7fa5d1ffb700 0 client.29510 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:11.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.227+0000 7fa5fb26bb80 1 -- 172.21.15.7:0/3684536098 >> 172.21.15.7:0/3684536098 conn(0x5644319b5080 msgr2=0x564431a8cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:11.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.227+0000 7fa5fb26bb80 1 -- 172.21.15.7:0/3684536098 shutdown_connections 2024-06-10T01:26:11.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.227+0000 7fa5fb26bb80 1 -- 172.21.15.7:0/3684536098 wait complete. 2024-06-10T01:26:11.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.227+0000 7fa5fb26bb80 20 client.29510 trim_cache size 0 max 16384 2024-06-10T01:26:11.226 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:26:11.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.227+0000 7fa5f09e8700 1 -- 172.21.15.7:0/3700268024 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa5d40a2c20 msgr2=0x7fa5d40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:11.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.227+0000 7fa5f09e8700 1 --2- 172.21.15.7:0/3700268024 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa5d40a2c20 0x7fa5d40a50e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0009680 tx=0x7fa5e0012040 comp rx=0 tx=0).stop 2024-06-10T01:26:11.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.227+0000 7fa5f09e8700 1 -- 172.21.15.7:0/3700268024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5ec102540 msgr2=0x7fa5ec19e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:11.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.227+0000 7fa5f09e8700 1 --2- 172.21.15.7:0/3700268024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5ec102540 0x7fa5ec19e860 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc00e970 tx=0x7fa5dc00ec80 comp rx=0 tx=0).stop 2024-06-10T01:26:11.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.228+0000 7fa5f09e8700 1 -- 172.21.15.7:0/3700268024 shutdown_connections 2024-06-10T01:26:11.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.228+0000 7fa5f09e8700 1 --2- 172.21.15.7:0/3700268024 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa5d40a2c20 0x7fa5d40a50e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.228+0000 7fa5f09e8700 1 --2- 172.21.15.7:0/3700268024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5ec107450 0x7fa5ec198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.228+0000 7fa5f09e8700 1 --2- 172.21.15.7:0/3700268024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5ec102540 0x7fa5ec19e860 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.228+0000 7fa5f09e8700 1 --2- 172.21.15.7:0/3700268024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5ec102ee0 0x7fa5ec19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:11.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.228+0000 7fa5f09e8700 1 -- 172.21.15.7:0/3700268024 >> 172.21.15.7:0/3700268024 conn(0x7fa5ec0795f0 msgr2=0x7fa5ec100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:11.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.228+0000 7fa5f09e8700 1 -- 172.21.15.7:0/3700268024 shutdown_connections 2024-06-10T01:26:11.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:11.228+0000 7fa5f09e8700 1 -- 172.21.15.7:0/3700268024 wait complete. 2024-06-10T01:26:11.544 INFO:tasks.workunit.client.0.smithi007.stdout: 10 170372 24.30 MB/sec execute 288 sec latency 5885.539 ms 2024-06-10T01:26:11.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:11 smithi175 ceph-mon[36636]: pgmap v1038: 129 pgs: 129 active+clean; 680 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 5.2 MiB/s wr, 537 op/s 2024-06-10T01:26:11.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:11 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:26:11.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:11 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:26:11.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:11 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:26:11.952 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:11 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:26:12.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:11 smithi007 ceph-mon[32398]: pgmap v1038: 129 pgs: 129 active+clean; 680 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 5.2 MiB/s wr, 537 op/s 2024-06-10T01:26:12.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:11 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:26:12.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:11 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:26:12.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:11 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:26:12.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:11 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:26:12.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:11 smithi078 ceph-mon[36683]: pgmap v1038: 129 pgs: 129 active+clean; 680 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 5.2 MiB/s wr, 537 op/s 2024-06-10T01:26:12.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:11 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:26:12.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:11 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:26:12.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:11 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:26:12.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:11 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:26:12.545 INFO:tasks.workunit.client.0.smithi007.stdout: 10 170372 24.22 MB/sec execute 289 sec latency 6885.590 ms 2024-06-10T01:26:13.166 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:26:13.166 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:26:13.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:13 smithi007 ceph-mon[32398]: pgmap v1039: 129 pgs: 129 active+clean; 712 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 8.0 MiB/s wr, 533 op/s 2024-06-10T01:26:13.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:13 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:26:13.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:13 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:26:13.512 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:13 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:26:13.512 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:13 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:26:13.512 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:13 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:26:13.545 INFO:tasks.workunit.client.0.smithi007.stdout: 10 170372 24.13 MB/sec execute 290 sec latency 7886.428 ms 2024-06-10T01:26:13.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:13 smithi078 ceph-mon[36683]: pgmap v1039: 129 pgs: 129 active+clean; 712 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 8.0 MiB/s wr, 533 op/s 2024-06-10T01:26:13.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:13 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:26:13.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:13 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:26:13.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:13 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:26:13.578 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:13 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:26:13.578 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:13 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:26:13.664 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:26:13.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:13 smithi175 ceph-mon[36636]: pgmap v1039: 129 pgs: 129 active+clean; 712 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 8.0 MiB/s wr, 533 op/s 2024-06-10T01:26:13.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:13 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:26:13.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:13 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:26:13.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:13 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:26:13.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:13 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:26:13.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:13 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:26:14.545 INFO:tasks.workunit.client.0.smithi007.stdout: 10 170372 24.05 MB/sec execute 291 sec latency 8886.473 ms 2024-06-10T01:26:15.483 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:15 smithi007 ceph-mon[32398]: pgmap v1040: 129 pgs: 129 active+clean; 713 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 8.1 MiB/s wr, 470 op/s 2024-06-10T01:26:15.545 INFO:tasks.workunit.client.0.smithi007.stdout: 10 170372 23.97 MB/sec execute 292 sec latency 9886.509 ms 2024-06-10T01:26:15.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:15 smithi078 ceph-mon[36683]: pgmap v1040: 129 pgs: 129 active+clean; 713 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 8.1 MiB/s wr, 470 op/s 2024-06-10T01:26:15.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:15 smithi175 ceph-mon[36636]: pgmap v1040: 129 pgs: 129 active+clean; 713 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 8.1 MiB/s wr, 470 op/s 2024-06-10T01:26:15.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.890+0000 7fdef6506700 1 -- 172.21.15.7:0/174272960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdef0101440 msgr2=0x7fdef010f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:15.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.890+0000 7fdef6506700 1 --2- 172.21.15.7:0/174272960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdef0101440 0x7fdef010f330 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fdee800b0a0 tx=0x7fdee800b3b0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.890+0000 7fdef6506700 1 -- 172.21.15.7:0/174272960 shutdown_connections 2024-06-10T01:26:15.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.890+0000 7fdef6506700 1 --2- 172.21.15.7:0/174272960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdef0101440 0x7fdef010f330 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.890+0000 7fdef6506700 1 --2- 172.21.15.7:0/174272960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdef0108510 0x7fdef01088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.890+0000 7fdef6506700 1 --2- 172.21.15.7:0/174272960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdef0100a80 0x7fdef0100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.890+0000 7fdef6506700 1 -- 172.21.15.7:0/174272960 >> 172.21.15.7:0/174272960 conn(0x7fdef0077a00 msgr2=0x7fdef00fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.890+0000 7fdef6506700 1 -- 172.21.15.7:0/174272960 shutdown_connections 2024-06-10T01:26:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.890+0000 7fdef6506700 1 -- 172.21.15.7:0/174272960 wait complete. 2024-06-10T01:26:15.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.891+0000 7fdef6506700 1 Processor -- start 2024-06-10T01:26:15.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.891+0000 7fdef6506700 1 -- start start 2024-06-10T01:26:15.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.891+0000 7fdef6506700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdef0100a80 0x7fdef019af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:15.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.891+0000 7fdef6506700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdef0101440 0x7fdef019b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:15.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.891+0000 7fdef6506700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdef0108510 0x7fdef019f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:15.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.891+0000 7fdef6506700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdef01127f0 con 0x7fdef0101440 2024-06-10T01:26:15.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.891+0000 7fdef6506700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdef0112670 con 0x7fdef0100a80 2024-06-10T01:26:15.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.891+0000 7fdef6506700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdef0112970 con 0x7fdef0108510 2024-06-10T01:26:15.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.891+0000 7fdeef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdef0101440 0x7fdef019b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:15.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.891+0000 7fdeef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdef0101440 0x7fdef019b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43270/0 (socket says 172.21.15.7:43270) 2024-06-10T01:26:15.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.891+0000 7fdeef7fe700 1 -- 172.21.15.7:0/3247441165 learned_addr learned my addr 172.21.15.7:0/3247441165 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:15.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.891+0000 7fdeeffff700 1 --2- 172.21.15.7:0/3247441165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdef0100a80 0x7fdef019af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:15.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.891+0000 7fdef4aa3700 1 --2- 172.21.15.7:0/3247441165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdef0108510 0x7fdef019f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:15.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.892+0000 7fdeef7fe700 1 -- 172.21.15.7:0/3247441165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdef0108510 msgr2=0x7fdef019f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:15.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.892+0000 7fdeef7fe700 1 --2- 172.21.15.7:0/3247441165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdef0108510 0x7fdef019f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.892+0000 7fdeef7fe700 1 -- 172.21.15.7:0/3247441165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdef0100a80 msgr2=0x7fdef019af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:15.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.892+0000 7fdeef7fe700 1 --2- 172.21.15.7:0/3247441165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdef0100a80 0x7fdef019af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.892+0000 7fdeef7fe700 1 -- 172.21.15.7:0/3247441165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef019ff10 con 0x7fdef0101440 2024-06-10T01:26:15.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.892+0000 7fdef4aa3700 1 --2- 172.21.15.7:0/3247441165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdef0108510 0x7fdef019f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:26:15.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.892+0000 7fdeef7fe700 1 --2- 172.21.15.7:0/3247441165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdef0101440 0x7fdef019b440 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fdee400b530 tx=0x7fdee400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:15.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.892+0000 7fdeed7fa700 1 -- 172.21.15.7:0/3247441165 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee4015020 con 0x7fdef0101440 2024-06-10T01:26:15.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.892+0000 7fdeed7fa700 1 -- 172.21.15.7:0/3247441165 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdee4011d10 con 0x7fdef0101440 2024-06-10T01:26:15.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.892+0000 7fdef6506700 1 -- 172.21.15.7:0/3247441165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdef01a0200 con 0x7fdef0101440 2024-06-10T01:26:15.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.892+0000 7fdeed7fa700 1 -- 172.21.15.7:0/3247441165 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee400f2e0 con 0x7fdef0101440 2024-06-10T01:26:15.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.892+0000 7fdef6506700 1 -- 172.21.15.7:0/3247441165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdef01a7a80 con 0x7fdef0101440 2024-06-10T01:26:15.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.894+0000 7fdeed7fa700 1 -- 172.21.15.7:0/3247441165 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fdee4018020 con 0x7fdef0101440 2024-06-10T01:26:15.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.895+0000 7fdeed7fa700 1 --2- 172.21.15.7:0/3247441165 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fded80a2c20 0x7fded80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:15.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.895+0000 7fdeeffff700 1 --2- 172.21.15.7:0/3247441165 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fded80a2c20 0x7fded80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:15.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.895+0000 7fdeed7fa700 1 -- 172.21.15.7:0/3247441165 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fdee40cec80 con 0x7fdef0101440 2024-06-10T01:26:15.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.895+0000 7fdeeffff700 1 --2- 172.21.15.7:0/3247441165 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fded80a2c20 0x7fded80a50e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fdee0005950 tx=0x7fdee000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:15.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.896+0000 7fdf00d89b80 1 Processor -- start 2024-06-10T01:26:15.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.896+0000 7fdf00d89b80 1 -- start start 2024-06-10T01:26:15.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.896+0000 7fdf00d89b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560684791fe0 0x5606847923c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:15.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.896+0000 7fdf00d89b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560684792900 0x56068478c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:15.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.896+0000 7fdf00d89b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56068478c820 0x56068478cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:15.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.896+0000 7fdf00d89b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56068467efe0 con 0x560684792900 2024-06-10T01:26:15.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.896+0000 7fdf00d89b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5606846dd8d0 con 0x56068478c820 2024-06-10T01:26:15.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.896+0000 7fdf00d89b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5606846454e0 con 0x560684791fe0 2024-06-10T01:26:15.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.896+0000 7fdeef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560684792900 0x56068478c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:15.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.896+0000 7fdeef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560684792900 0x56068478c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43272/0 (socket says 172.21.15.7:43272) 2024-06-10T01:26:15.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.896+0000 7fdeef7fe700 1 -- 172.21.15.7:0/1489472614 learned_addr learned my addr 172.21.15.7:0/1489472614 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:15.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.896+0000 7fdef4aa3700 1 --2- 172.21.15.7:0/1489472614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56068478c820 0x56068478cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:15.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.896+0000 7fdeeffff700 1 --2- 172.21.15.7:0/1489472614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560684791fe0 0x5606847923c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:15.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.897+0000 7fdeef7fe700 1 -- 172.21.15.7:0/1489472614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560684791fe0 msgr2=0x5606847923c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:15.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.897+0000 7fdeef7fe700 1 --2- 172.21.15.7:0/1489472614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560684791fe0 0x5606847923c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.897+0000 7fdeef7fe700 1 -- 172.21.15.7:0/1489472614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56068478c820 msgr2=0x56068478cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:15.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.897+0000 7fdeef7fe700 1 --2- 172.21.15.7:0/1489472614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56068478c820 0x56068478cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.897+0000 7fdeef7fe700 1 -- 172.21.15.7:0/1489472614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56068462c950 con 0x560684792900 2024-06-10T01:26:15.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.897+0000 7fdef4aa3700 1 --2- 172.21.15.7:0/1489472614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56068478c820 0x56068478cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:26:15.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.897+0000 7fdeef7fe700 1 --2- 172.21.15.7:0/1489472614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560684792900 0x56068478c2e0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fdee4097560 tx=0x7fdee4010a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ed069c54e66958 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:15.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.897+0000 7fded67fc700 1 -- 172.21.15.7:0/1489472614 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee409b100 con 0x560684792900 2024-06-10T01:26:15.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.897+0000 7fded67fc700 1 -- 172.21.15.7:0/1489472614 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdee409f4b0 con 0x560684792900 2024-06-10T01:26:15.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.897+0000 7fded67fc700 1 -- 172.21.15.7:0/1489472614 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee409f160 con 0x560684792900 2024-06-10T01:26:15.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.898+0000 7fdf00d89b80 1 -- 172.21.15.7:0/1489472614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560684792900 msgr2=0x56068478c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:15.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.898+0000 7fdf00d89b80 1 --2- 172.21.15.7:0/1489472614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560684792900 0x56068478c2e0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fdee4097560 tx=0x7fdee4010a10 comp rx=0 tx=0).stop 2024-06-10T01:26:15.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.898+0000 7fdf00d89b80 1 -- 172.21.15.7:0/1489472614 shutdown_connections 2024-06-10T01:26:15.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.898+0000 7fdf00d89b80 1 --2- 172.21.15.7:0/1489472614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56068478c820 0x56068478cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.898+0000 7fdf00d89b80 1 --2- 172.21.15.7:0/1489472614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560684791fe0 0x5606847923c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.898+0000 7fdf00d89b80 1 --2- 172.21.15.7:0/1489472614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560684792900 0x56068478c2e0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.898+0000 7fdf00d89b80 1 -- 172.21.15.7:0/1489472614 >> 172.21.15.7:0/1489472614 conn(0x5606846b9100 msgr2=0x5606847872a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:15.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.898+0000 7fdf00d89b80 1 -- 172.21.15.7:0/1489472614 shutdown_connections 2024-06-10T01:26:15.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.898+0000 7fdf00d89b80 1 -- 172.21.15.7:0/1489472614 wait complete. 2024-06-10T01:26:15.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.898+0000 7fdf00d89b80 1 -- start start 2024-06-10T01:26:15.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.899+0000 7fdf00d89b80 1 Processor -- start 2024-06-10T01:26:15.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.899+0000 7fdf00d89b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068478c820 0x560684798690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:15.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.899+0000 7fdf00d89b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560684791fe0 0x560684795990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:15.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.899+0000 7fdf00d89b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560684792900 0x560684795ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:15.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.899+0000 7fdf00d89b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56068467e040 con 0x560684791fe0 2024-06-10T01:26:15.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.899+0000 7fdf00d89b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5606846d9890 con 0x560684792900 2024-06-10T01:26:15.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.899+0000 7fdf00d89b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5606847992e0 con 0x56068478c820 2024-06-10T01:26:15.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.899+0000 7fdf00d89b80 10 client.0 _ll_register_callbacks cb 0x560684673630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:26:15.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.899+0000 7fdeef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560684791fe0 0x560684795990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:15.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.899+0000 7fdeef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560684791fe0 0x560684795990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43284/0 (socket says 172.21.15.7:43284) 2024-06-10T01:26:15.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.899+0000 7fdeef7fe700 1 -- 172.21.15.7:0/2639461525 learned_addr learned my addr 172.21.15.7:0/2639461525 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:15.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.899+0000 7fdef4aa3700 1 --2- 172.21.15.7:0/2639461525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560684792900 0x560684795ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:15.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.899+0000 7fdeeffff700 1 --2- 172.21.15.7:0/2639461525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068478c820 0x560684798690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:15.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.899+0000 7fdeef7fe700 1 -- 172.21.15.7:0/2639461525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068478c820 msgr2=0x560684798690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:15.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.899+0000 7fdeef7fe700 1 --2- 172.21.15.7:0/2639461525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068478c820 0x560684798690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.899+0000 7fdeef7fe700 1 -- 172.21.15.7:0/2639461525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560684792900 msgr2=0x560684795ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.899+0000 7fdeef7fe700 1 --2- 172.21.15.7:0/2639461525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560684792900 0x560684795ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.899+0000 7fdeef7fe700 1 -- 172.21.15.7:0/2639461525 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606846a37d0 con 0x560684791fe0 2024-06-10T01:26:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.899+0000 7fdeef7fe700 1 --2- 172.21.15.7:0/2639461525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560684791fe0 0x560684795990 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fdee40a3840 tx=0x7fdee409b170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=253ad973e0499f5f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.900+0000 7fdecf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:26:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.900+0000 7fdecf7fe700 1 -- 172.21.15.7:0/2639461525 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee409f160 con 0x560684791fe0 2024-06-10T01:26:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.900+0000 7fdecf7fe700 1 -- 172.21.15.7:0/2639461525 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fdee40bb070 con 0x560684791fe0 2024-06-10T01:26:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.900+0000 7fdf00d89b80 1 -- 172.21.15.7:0/2639461525 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdee409f4b0 con 0x560684791fe0 2024-06-10T01:26:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.900+0000 7fdecf7fe700 1 -- 172.21.15.7:0/2639461525 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fdee409b360 con 0x560684791fe0 2024-06-10T01:26:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.900+0000 7fdf00d89b80 10 client.20295 fetch_fsmap learned FSMap version 52 2024-06-10T01:26:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.900+0000 7fdf00d89b80 1 -- 172.21.15.7:0/2639461525 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5606847a3a90 con 0x560684791fe0 2024-06-10T01:26:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.900+0000 7fdecf7fe700 1 -- 172.21.15.7:0/2639461525 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdee409f4b0 con 0x560684791fe0 2024-06-10T01:26:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.900+0000 7fdecf7fe700 1 -- 172.21.15.7:0/2639461525 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7fdee40a3970 con 0x560684791fe0 2024-06-10T01:26:15.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.900+0000 7fdf00d89b80 10 client.20295 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:26:15.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.900+0000 7fdf00d89b80 10 client.20295 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:26:15.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.900+0000 7fdf00d89b80 20 client.20295 populate_metadata read hostname 'smithi007' 2024-06-10T01:26:15.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.900+0000 7fdf00d89b80 1 --2- 172.21.15.7:0/2639461525 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5606847a5f60 0x5606847a8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:15.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.900+0000 7fdf00d89b80 4 client.20295 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:26:15.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.900+0000 7fdf00d89b80 1 -- 172.21.15.7:0/2639461525 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606846eadd0 con 0x5606847a5f60 2024-06-10T01:26:15.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.901+0000 7fdeeffff700 1 --2- 172.21.15.7:0/2639461525 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5606847a5f60 0x5606847a8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:15.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.901+0000 7fdecf7fe700 10 client.20295 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:15.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.901+0000 7fdeeffff700 1 --2- 172.21.15.7:0/2639461525 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5606847a5f60 0x5606847a8350 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6890c5824eec5114 server_cookie=77db4874db789402 in_seq=0 out_seq=0 2024-06-10T01:26:15.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.902+0000 7fdecf7fe700 1 -- 172.21.15.7:0/2639461525 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5606846eadd0 con 0x5606847a5f60 2024-06-10T01:26:15.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.902+0000 7fdecf7fe700 10 client.20295 handle_command_reply: tid=0 2024-06-10T01:26:15.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.902+0000 7fdf00d89b80 1 client.20295 shutdown 2024-06-10T01:26:15.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.902+0000 7fdf00d89b80 1 -- 172.21.15.7:0/2639461525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560684791fe0 msgr2=0x560684795990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:15.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.902+0000 7fdf00d89b80 1 --2- 172.21.15.7:0/2639461525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560684791fe0 0x560684795990 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fdee40a3840 tx=0x7fdee409b170 comp rx=0 tx=0).stop 2024-06-10T01:26:15.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.902+0000 7fdf00d89b80 1 -- 172.21.15.7:0/2639461525 shutdown_connections 2024-06-10T01:26:15.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.902+0000 7fdf00d89b80 1 --2- 172.21.15.7:0/2639461525 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5606847a5f60 0x5606847a8350 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.902+0000 7fdf00d89b80 1 --2- 172.21.15.7:0/2639461525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560684792900 0x560684795ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.902+0000 7fdf00d89b80 1 --2- 172.21.15.7:0/2639461525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068478c820 0x560684798690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.902+0000 7fdecf7fe700 0 client.20295 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:15.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.902+0000 7fdf00d89b80 1 --2- 172.21.15.7:0/2639461525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560684791fe0 0x560684795990 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.902+0000 7fdf00d89b80 1 -- 172.21.15.7:0/2639461525 >> 172.21.15.7:0/2639461525 conn(0x5606846b9100 msgr2=0x560684786780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:15.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.902+0000 7fdf00d89b80 1 -- 172.21.15.7:0/2639461525 shutdown_connections 2024-06-10T01:26:15.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.902+0000 7fdf00d89b80 1 -- 172.21.15.7:0/2639461525 wait complete. 2024-06-10T01:26:15.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.902+0000 7fdf00d89b80 20 client.20295 trim_cache size 0 max 16384 2024-06-10T01:26:15.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.923+0000 7fdf00d89b80 1 Processor -- start 2024-06-10T01:26:15.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.923+0000 7fdf00d89b80 1 -- start start 2024-06-10T01:26:15.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.923+0000 7fdf00d89b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606846b9100 0x560684786f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:15.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.923+0000 7fdf00d89b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068478c820 0x560684787440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:15.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.923+0000 7fdf00d89b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560684791fe0 0x5606847ab370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:15.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.923+0000 7fdf00d89b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdef01b2ea0 con 0x560684791fe0 2024-06-10T01:26:15.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.923+0000 7fdf00d89b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdef01b0730 con 0x5606846b9100 2024-06-10T01:26:15.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.923+0000 7fdf00d89b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdef01ae580 con 0x56068478c820 2024-06-10T01:26:15.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.924+0000 7fdef4aa3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560684791fe0 0x5606847ab370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:15.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.924+0000 7fdef4aa3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560684791fe0 0x5606847ab370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43292/0 (socket says 172.21.15.7:43292) 2024-06-10T01:26:15.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.924+0000 7fdef4aa3700 1 -- 172.21.15.7:0/1519044480 learned_addr learned my addr 172.21.15.7:0/1519044480 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:15.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.924+0000 7fdeef7fe700 1 --2- 172.21.15.7:0/1519044480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068478c820 0x560684787440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:15.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.924+0000 7fdeeffff700 1 --2- 172.21.15.7:0/1519044480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606846b9100 0x560684786f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:15.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.924+0000 7fdef4aa3700 1 -- 172.21.15.7:0/1519044480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068478c820 msgr2=0x560684787440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:15.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.924+0000 7fdef4aa3700 1 --2- 172.21.15.7:0/1519044480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068478c820 0x560684787440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.924+0000 7fdef4aa3700 1 -- 172.21.15.7:0/1519044480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606846b9100 msgr2=0x560684786f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:15.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.924+0000 7fdef4aa3700 1 --2- 172.21.15.7:0/1519044480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606846b9100 0x560684786f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.924+0000 7fdef4aa3700 1 -- 172.21.15.7:0/1519044480 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606847968d0 con 0x560684791fe0 2024-06-10T01:26:15.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.924+0000 7fdeef7fe700 1 --2- 172.21.15.7:0/1519044480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068478c820 0x560684787440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:26:15.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.924+0000 7fdeeffff700 1 --2- 172.21.15.7:0/1519044480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606846b9100 0x560684786f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:26:15.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.924+0000 7fdef4aa3700 1 --2- 172.21.15.7:0/1519044480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560684791fe0 0x5606847ab370 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fdee8015000 tx=0x7fdee8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7844aeaae93dc20a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:15.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.924+0000 7fded5ffb700 1 -- 172.21.15.7:0/1519044480 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee8004a50 con 0x560684791fe0 2024-06-10T01:26:15.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.924+0000 7fded5ffb700 1 -- 172.21.15.7:0/1519044480 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdee8004bf0 con 0x560684791fe0 2024-06-10T01:26:15.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.924+0000 7fded5ffb700 1 -- 172.21.15.7:0/1519044480 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee8005540 con 0x560684791fe0 2024-06-10T01:26:15.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.925+0000 7fdf00d89b80 1 -- 172.21.15.7:0/1519044480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560684791fe0 msgr2=0x5606847ab370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:15.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.925+0000 7fdf00d89b80 1 --2- 172.21.15.7:0/1519044480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560684791fe0 0x5606847ab370 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fdee8015000 tx=0x7fdee8009310 comp rx=0 tx=0).stop 2024-06-10T01:26:15.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.925+0000 7fdf00d89b80 1 -- 172.21.15.7:0/1519044480 shutdown_connections 2024-06-10T01:26:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.925+0000 7fdf00d89b80 1 --2- 172.21.15.7:0/1519044480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560684791fe0 0x5606847ab370 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.925+0000 7fdf00d89b80 1 --2- 172.21.15.7:0/1519044480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606846b9100 0x560684786f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.925+0000 7fdf00d89b80 1 --2- 172.21.15.7:0/1519044480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068478c820 0x560684787440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.925+0000 7fdf00d89b80 1 -- 172.21.15.7:0/1519044480 >> 172.21.15.7:0/1519044480 conn(0x5606847902f0 msgr2=0x560684797a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.925+0000 7fdf00d89b80 1 -- 172.21.15.7:0/1519044480 shutdown_connections 2024-06-10T01:26:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.925+0000 7fdf00d89b80 1 -- 172.21.15.7:0/1519044480 wait complete. 2024-06-10T01:26:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.925+0000 7fdf00d89b80 1 -- start start 2024-06-10T01:26:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.925+0000 7fdf00d89b80 1 Processor -- start 2024-06-10T01:26:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.926+0000 7fdf00d89b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606846b9100 0x5606847abca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.926+0000 7fdf00d89b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068478c820 0x5606847aa050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.926+0000 7fdf00d89b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560684791fe0 0x5606847aa790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.926+0000 7fdf00d89b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdef01aebf0 con 0x5606846b9100 2024-06-10T01:26:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.926+0000 7fdf00d89b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdef01b0060 con 0x560684791fe0 2024-06-10T01:26:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.926+0000 7fdf00d89b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdef01af2c0 con 0x56068478c820 2024-06-10T01:26:15.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.926+0000 7fdf00d89b80 10 client.0 _ll_register_callbacks cb 0x560684673630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:26:15.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.926+0000 7fdeeffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606846b9100 0x5606847abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:15.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.926+0000 7fdeeffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606846b9100 0x5606847abca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43300/0 (socket says 172.21.15.7:43300) 2024-06-10T01:26:15.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.926+0000 7fdeeffff700 1 -- 172.21.15.7:0/965829607 learned_addr learned my addr 172.21.15.7:0/965829607 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:15.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.926+0000 7fdef4aa3700 1 --2- 172.21.15.7:0/965829607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560684791fe0 0x5606847aa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:15.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.926+0000 7fdeef7fe700 1 --2- 172.21.15.7:0/965829607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068478c820 0x5606847aa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:15.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.926+0000 7fdeeffff700 1 -- 172.21.15.7:0/965829607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068478c820 msgr2=0x5606847aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:15.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.926+0000 7fdeeffff700 1 --2- 172.21.15.7:0/965829607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068478c820 0x5606847aa050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.926+0000 7fdeeffff700 1 -- 172.21.15.7:0/965829607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560684791fe0 msgr2=0x5606847aa790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:15.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.926+0000 7fdeeffff700 1 --2- 172.21.15.7:0/965829607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560684791fe0 0x5606847aa790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.926+0000 7fdeeffff700 1 -- 172.21.15.7:0/965829607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606848108b0 con 0x5606846b9100 2024-06-10T01:26:15.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.926+0000 7fdeeffff700 1 --2- 172.21.15.7:0/965829607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606846b9100 0x5606847abca0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x56068467f040 tx=0x7fdee00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4d6938c93a4fe4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:15.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.926+0000 7fdecf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:26:15.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.927+0000 7fdecf7fe700 1 -- 172.21.15.7:0/965829607 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee0016070 con 0x5606846b9100 2024-06-10T01:26:15.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.927+0000 7fdecf7fe700 1 -- 172.21.15.7:0/965829607 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fdee0004830 con 0x5606846b9100 2024-06-10T01:26:15.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.927+0000 7fdf00d89b80 1 -- 172.21.15.7:0/965829607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdee8004bf0 con 0x5606846b9100 2024-06-10T01:26:15.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.927+0000 7fdecf7fe700 1 -- 172.21.15.7:0/965829607 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fdee0035050 con 0x5606846b9100 2024-06-10T01:26:15.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.927+0000 7fdf00d89b80 10 client.20301 fetch_fsmap learned FSMap version 52 2024-06-10T01:26:15.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.927+0000 7fdf00d89b80 1 -- 172.21.15.7:0/965829607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x560684811ab0 con 0x5606846b9100 2024-06-10T01:26:15.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.927+0000 7fdecf7fe700 1 -- 172.21.15.7:0/965829607 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdee0005020 con 0x5606846b9100 2024-06-10T01:26:15.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.927+0000 7fdecf7fe700 1 -- 172.21.15.7:0/965829607 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7fdee00263c0 con 0x5606846b9100 2024-06-10T01:26:15.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.927+0000 7fdf00d89b80 10 client.20301 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:26:15.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.927+0000 7fdf00d89b80 10 client.20301 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:26:15.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.927+0000 7fdf00d89b80 20 client.20301 populate_metadata read hostname 'smithi007' 2024-06-10T01:26:15.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.927+0000 7fdf00d89b80 1 --2- 172.21.15.7:0/965829607 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560684792900 0x560684814200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:15.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.927+0000 7fdf00d89b80 4 client.20301 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:26:15.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.927+0000 7fdf00d89b80 1 -- 172.21.15.7:0/965829607 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fdee8004bf0 con 0x560684792900 2024-06-10T01:26:15.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.928+0000 7fdeef7fe700 1 --2- 172.21.15.7:0/965829607 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560684792900 0x560684814200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:15.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.928+0000 7fdeef7fe700 1 --2- 172.21.15.7:0/965829607 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560684792900 0x560684814200 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e76ee132a3633a2 server_cookie=a58daafc597517d9 in_seq=0 out_seq=0 2024-06-10T01:26:15.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.928+0000 7fdecf7fe700 10 client.20301 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:15.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.929+0000 7fdecf7fe700 1 -- 172.21.15.7:0/965829607 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fdee8004bf0 con 0x560684792900 2024-06-10T01:26:15.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.929+0000 7fdecf7fe700 10 client.20301 handle_command_reply: tid=0 2024-06-10T01:26:15.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.929+0000 7fdf00d89b80 1 client.20301 shutdown 2024-06-10T01:26:15.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.929+0000 7fdf00d89b80 1 -- 172.21.15.7:0/965829607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606846b9100 msgr2=0x5606847abca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:15.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.929+0000 7fdf00d89b80 1 --2- 172.21.15.7:0/965829607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606846b9100 0x5606847abca0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x56068467f040 tx=0x7fdee00047c0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.930+0000 7fdf00d89b80 1 -- 172.21.15.7:0/965829607 shutdown_connections 2024-06-10T01:26:15.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.930+0000 7fdf00d89b80 1 --2- 172.21.15.7:0/965829607 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560684792900 0x560684814200 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.930+0000 7fdf00d89b80 1 --2- 172.21.15.7:0/965829607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560684791fe0 0x5606847aa790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.930+0000 7fdf00d89b80 1 --2- 172.21.15.7:0/965829607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606846b9100 0x5606847abca0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.930+0000 7fdf00d89b80 1 --2- 172.21.15.7:0/965829607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068478c820 0x5606847aa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.930+0000 7fdf00d89b80 1 -- 172.21.15.7:0/965829607 >> 172.21.15.7:0/965829607 conn(0x5606847902f0 msgr2=0x5606847971c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:15.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.930+0000 7fdecf7fe700 0 client.20301 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:15.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.930+0000 7fdf00d89b80 1 -- 172.21.15.7:0/965829607 shutdown_connections 2024-06-10T01:26:15.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.930+0000 7fdf00d89b80 1 -- 172.21.15.7:0/965829607 wait complete. 2024-06-10T01:26:15.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.930+0000 7fdf00d89b80 20 client.20301 trim_cache size 0 max 16384 2024-06-10T01:26:15.931 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:26:15.931 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-10T01:26:15.931 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "2b655214-48e0-4649-adf2-0be7b37161e8", 2024-06-10T01:26:15.931 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-10T01:26:15.931 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:26:15.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.930+0000 7fdef6506700 1 -- 172.21.15.7:0/3247441165 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fded80a2c20 msgr2=0x7fded80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:15.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.930+0000 7fdef6506700 1 --2- 172.21.15.7:0/3247441165 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fded80a2c20 0x7fded80a50e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fdee0005950 tx=0x7fdee000a300 comp rx=0 tx=0).stop 2024-06-10T01:26:15.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.930+0000 7fdef6506700 1 -- 172.21.15.7:0/3247441165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdef0101440 msgr2=0x7fdef019b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:15.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.930+0000 7fdef6506700 1 --2- 172.21.15.7:0/3247441165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdef0101440 0x7fdef019b440 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fdee400b530 tx=0x7fdee400b840 comp rx=0 tx=0).stop 2024-06-10T01:26:15.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.931+0000 7fdef6506700 1 -- 172.21.15.7:0/3247441165 shutdown_connections 2024-06-10T01:26:15.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.931+0000 7fdef6506700 1 --2- 172.21.15.7:0/3247441165 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fded80a2c20 0x7fded80a50e0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.931+0000 7fdef6506700 1 --2- 172.21.15.7:0/3247441165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdef0108510 0x7fdef019f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.931+0000 7fdef6506700 1 --2- 172.21.15.7:0/3247441165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdef0100a80 0x7fdef019af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.931+0000 7fdef6506700 1 --2- 172.21.15.7:0/3247441165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdef0101440 0x7fdef019b440 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:15.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.931+0000 7fdef6506700 1 -- 172.21.15.7:0/3247441165 >> 172.21.15.7:0/3247441165 conn(0x7fdef0077a00 msgr2=0x7fdef0101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:15.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.931+0000 7fdef6506700 1 -- 172.21.15.7:0/3247441165 shutdown_connections 2024-06-10T01:26:15.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:15.931+0000 7fdef6506700 1 -- 172.21.15.7:0/3247441165 wait complete. 2024-06-10T01:26:16.044 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:26:16.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:16 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:26:16.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:16 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:26:16.545 INFO:tasks.workunit.client.0.smithi007.stdout: 10 170372 23.89 MB/sec execute 293 sec latency 10886.537 ms 2024-06-10T01:26:16.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:16 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:26:16.578 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:16 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:26:16.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:16 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:26:16.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:16 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:26:16.711 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:26:16.884 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:26:17.340 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:17 smithi007 ceph-mon[32398]: scrub queued for path: / 2024-06-10T01:26:17.340 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:17 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:26:17.340 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:17 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:26:17.340 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:17 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:26:17.340 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:17 smithi007 ceph-mon[32398]: pgmap v1041: 129 pgs: 129 active+clean; 682 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 11 MiB/s wr, 617 op/s 2024-06-10T01:26:17.340 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:17 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:26:17.340 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:17 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:26:17.545 INFO:tasks.workunit.client.0.smithi007.stdout: 10 170372 23.80 MB/sec execute 294 sec latency 11886.610 ms 2024-06-10T01:26:17.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:17 smithi078 ceph-mon[36683]: scrub queued for path: / 2024-06-10T01:26:17.578 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:17 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:26:17.578 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:17 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:26:17.578 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:17 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:26:17.578 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:17 smithi078 ceph-mon[36683]: pgmap v1041: 129 pgs: 129 active+clean; 682 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 11 MiB/s wr, 617 op/s 2024-06-10T01:26:17.578 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:17 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:26:17.578 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:17 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:26:17.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:17 smithi175 ceph-mon[36636]: scrub queued for path: / 2024-06-10T01:26:17.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:17 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:26:17.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:17 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:26:17.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:17 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:26:17.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:17 smithi175 ceph-mon[36636]: pgmap v1041: 129 pgs: 129 active+clean; 682 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 11 MiB/s wr, 617 op/s 2024-06-10T01:26:17.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:17 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:26:17.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:17 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:26:18.546 INFO:tasks.workunit.client.0.smithi007.stdout: 10 170372 23.72 MB/sec execute 295 sec latency 12886.669 ms 2024-06-10T01:26:19.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:19 smithi007 ceph-mon[32398]: pgmap v1042: 129 pgs: 129 active+clean; 663 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 11 MiB/s wr, 599 op/s 2024-06-10T01:26:19.546 INFO:tasks.workunit.client.0.smithi007.stdout: 10 170372 23.64 MB/sec execute 296 sec latency 13886.739 ms 2024-06-10T01:26:19.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:19 smithi078 ceph-mon[36683]: pgmap v1042: 129 pgs: 129 active+clean; 663 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 11 MiB/s wr, 599 op/s 2024-06-10T01:26:19.651 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:26:19.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:19 smithi175 ceph-mon[36636]: pgmap v1042: 129 pgs: 129 active+clean; 663 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 11 MiB/s wr, 599 op/s 2024-06-10T01:26:20.547 INFO:tasks.workunit.client.0.smithi007.stdout: 10 170372 23.56 MB/sec execute 297 sec latency 14886.783 ms 2024-06-10T01:26:20.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.798+0000 7f1267b10700 1 -- 172.21.15.7:0/2051713375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1260101fd0 msgr2=0x7f1260105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:20.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.798+0000 7f1267b10700 1 --2- 172.21.15.7:0/2051713375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1260101fd0 0x7f1260105fd0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f12500099c0 tx=0x7f1250009cd0 comp rx=0 tx=0).stop 2024-06-10T01:26:20.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.798+0000 7f1267b10700 1 -- 172.21.15.7:0/2051713375 shutdown_connections 2024-06-10T01:26:20.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.798+0000 7f1267b10700 1 --2- 172.21.15.7:0/2051713375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12601066b0 0x7f126010e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:20.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.798+0000 7f1267b10700 1 --2- 172.21.15.7:0/2051713375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1260101620 0x7f1260101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:20.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.798+0000 7f1267b10700 1 --2- 172.21.15.7:0/2051713375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1260101fd0 0x7f1260105fd0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:20.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.798+0000 7f1267b10700 1 -- 172.21.15.7:0/2051713375 >> 172.21.15.7:0/2051713375 conn(0x7f1260078340 msgr2=0x7f1260078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:20.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.798+0000 7f1267b10700 1 -- 172.21.15.7:0/2051713375 shutdown_connections 2024-06-10T01:26:20.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.798+0000 7f1267b10700 1 -- 172.21.15.7:0/2051713375 wait complete. 2024-06-10T01:26:20.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.799+0000 7f1267b10700 1 Processor -- start 2024-06-10T01:26:20.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.799+0000 7f1267b10700 1 -- start start 2024-06-10T01:26:20.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.800+0000 7f1267b10700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1260101620 0x7f126019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:20.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.800+0000 7f1267b10700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1260101fd0 0x7f126019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:20.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.800+0000 7f1267b10700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12601066b0 0x7f1260198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:20.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.800+0000 7f1267b10700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f12601116f0 con 0x7f1260101fd0 2024-06-10T01:26:20.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.800+0000 7f1267b10700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1260111570 con 0x7f12601066b0 2024-06-10T01:26:20.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.800+0000 7f1267b10700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1260111870 con 0x7f1260101620 2024-06-10T01:26:20.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.800+0000 7f12650ab700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1260101fd0 0x7f126019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:20.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.800+0000 7f12650ab700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1260101fd0 0x7f126019ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43304/0 (socket says 172.21.15.7:43304) 2024-06-10T01:26:20.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.800+0000 7f12650ab700 1 -- 172.21.15.7:0/3225344496 learned_addr learned my addr 172.21.15.7:0/3225344496 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:20.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.800+0000 7f12658ac700 1 --2- 172.21.15.7:0/3225344496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1260101620 0x7f126019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:20.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.800+0000 7f12660ad700 1 --2- 172.21.15.7:0/3225344496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12601066b0 0x7f1260198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:20.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.800+0000 7f12650ab700 1 -- 172.21.15.7:0/3225344496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1260101620 msgr2=0x7f126019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:20.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.800+0000 7f12650ab700 1 --2- 172.21.15.7:0/3225344496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1260101620 0x7f126019e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:20.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.800+0000 7f12650ab700 1 -- 172.21.15.7:0/3225344496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12601066b0 msgr2=0x7f1260198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:20.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.800+0000 7f12650ab700 1 --2- 172.21.15.7:0/3225344496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12601066b0 0x7f1260198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:20.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.800+0000 7f12650ab700 1 -- 172.21.15.7:0/3225344496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1260199260 con 0x7f1260101fd0 2024-06-10T01:26:20.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.800+0000 7f12650ab700 1 --2- 172.21.15.7:0/3225344496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1260101fd0 0x7f126019ee80 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f125000c010 tx=0x7f12500047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:20.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.800+0000 7f1256ffd700 1 -- 172.21.15.7:0/3225344496 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1250004830 con 0x7f1260101fd0 2024-06-10T01:26:20.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.801+0000 7f1267b10700 1 -- 172.21.15.7:0/3225344496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12601994f0 con 0x7f1260101fd0 2024-06-10T01:26:20.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.801+0000 7f1256ffd700 1 -- 172.21.15.7:0/3225344496 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1250005020 con 0x7f1260101fd0 2024-06-10T01:26:20.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.801+0000 7f1256ffd700 1 -- 172.21.15.7:0/3225344496 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1250003cb0 con 0x7f1260101fd0 2024-06-10T01:26:20.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.801+0000 7f1267b10700 1 -- 172.21.15.7:0/3225344496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12601a5810 con 0x7f1260101fd0 2024-06-10T01:26:20.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.802+0000 7f1267b10700 1 -- 172.21.15.7:0/3225344496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1260069180 con 0x7f1260101fd0 2024-06-10T01:26:20.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.803+0000 7f1256ffd700 1 -- 172.21.15.7:0/3225344496 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f12500049d0 con 0x7f1260101fd0 2024-06-10T01:26:20.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.804+0000 7f1256ffd700 1 --2- 172.21.15.7:0/3225344496 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f124c0a2c20 0x7f124c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:20.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.804+0000 7f12658ac700 1 --2- 172.21.15.7:0/3225344496 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f124c0a2c20 0x7f124c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:20.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.804+0000 7f1256ffd700 1 -- 172.21.15.7:0/3225344496 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f12500d0770 con 0x7f1260101fd0 2024-06-10T01:26:20.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.805+0000 7f12658ac700 1 --2- 172.21.15.7:0/3225344496 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f124c0a2c20 0x7f124c0a50e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f125c005950 tx=0x7f125c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:20.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:20.808+0000 7f1256ffd700 1 -- 172.21.15.7:0/3225344496 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1250013070 con 0x7f1260101fd0 2024-06-10T01:26:21.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:21.064+0000 7f1267b10700 1 -- 172.21.15.7:0/3225344496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f126002dca0 con 0x7f1260101fd0 2024-06-10T01:26:21.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:21.064+0000 7f1256ffd700 1 -- 172.21.15.7:0/3225344496 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11858 (secure 0 0 0) 0x7f12500095a0 con 0x7f1260101fd0 2024-06-10T01:26:21.057 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:26:21.058 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":52,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:25:03.837724+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28825":{"gid":28825,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/1972426673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1972426673},{"type":"v1","addr":"172.21.15.175:6835","nonce":1972426673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24451,24434,24436]},"id":1}]} 2024-06-10T01:26:21.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:21.066+0000 7f1267b10700 1 -- 172.21.15.7:0/3225344496 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f124c0a2c20 msgr2=0x7f124c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:21.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:21.066+0000 7f1267b10700 1 --2- 172.21.15.7:0/3225344496 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f124c0a2c20 0x7f124c0a50e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f125c005950 tx=0x7f125c00b410 comp rx=0 tx=0).stop 2024-06-10T01:26:21.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:21.066+0000 7f1267b10700 1 -- 172.21.15.7:0/3225344496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1260101fd0 msgr2=0x7f126019ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:21.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:21.066+0000 7f1267b10700 1 --2- 172.21.15.7:0/3225344496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1260101fd0 0x7f126019ee80 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f125000c010 tx=0x7f12500047b0 comp rx=0 tx=0).stop 2024-06-10T01:26:21.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:21.066+0000 7f1267b10700 1 -- 172.21.15.7:0/3225344496 shutdown_connections 2024-06-10T01:26:21.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:21.066+0000 7f1267b10700 1 --2- 172.21.15.7:0/3225344496 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f124c0a2c20 0x7f124c0a50e0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:21.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:21.066+0000 7f1267b10700 1 --2- 172.21.15.7:0/3225344496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12601066b0 0x7f1260198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:21.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:21.066+0000 7f1267b10700 1 --2- 172.21.15.7:0/3225344496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1260101620 0x7f126019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:21.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:21.066+0000 7f1267b10700 1 --2- 172.21.15.7:0/3225344496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1260101fd0 0x7f126019ee80 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:21.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:21.066+0000 7f1267b10700 1 -- 172.21.15.7:0/3225344496 >> 172.21.15.7:0/3225344496 conn(0x7f1260078340 msgr2=0x7f126006aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:21.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:21.067+0000 7f1267b10700 1 -- 172.21.15.7:0/3225344496 shutdown_connections 2024-06-10T01:26:21.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:21.067+0000 7f1267b10700 1 -- 172.21.15.7:0/3225344496 wait complete. 2024-06-10T01:26:21.061 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 52 2024-06-10T01:26:21.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:21 smithi007 ceph-mon[32398]: pgmap v1043: 129 pgs: 129 active+clean; 657 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 10 MiB/s wr, 573 op/s 2024-06-10T01:26:21.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:21 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3225344496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:26:21.546 INFO:tasks.workunit.client.0.smithi007.stdout: 10 170372 23.49 MB/sec execute 298 sec latency 15886.855 ms 2024-06-10T01:26:21.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:21 smithi078 ceph-mon[36683]: pgmap v1043: 129 pgs: 129 active+clean; 657 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 10 MiB/s wr, 573 op/s 2024-06-10T01:26:21.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:21 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3225344496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:26:21.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:21 smithi175 ceph-mon[36636]: pgmap v1043: 129 pgs: 129 active+clean; 657 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 10 MiB/s wr, 573 op/s 2024-06-10T01:26:21.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:21 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3225344496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:26:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.097+0000 7efe1c322640 1 -- 172.21.15.7:0/3368336138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe14102c20 msgr2=0x7efe141030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.097+0000 7efe1c322640 1 --2- 172.21.15.7:0/3368336138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe14102c20 0x7efe141030a0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7efe04009900 tx=0x7efe0402a410 comp rx=0 tx=0).stop 2024-06-10T01:26:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.097+0000 7efe1c322640 1 -- 172.21.15.7:0/3368336138 shutdown_connections 2024-06-10T01:26:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.097+0000 7efe1c322640 1 --2- 172.21.15.7:0/3368336138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe141035e0 0x7efe14109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.097+0000 7efe1c322640 1 --2- 172.21.15.7:0/3368336138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe14102c20 0x7efe141030a0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.097+0000 7efe1c322640 1 --2- 172.21.15.7:0/3368336138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe14101a30 0x7efe14101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.097+0000 7efe1c322640 1 -- 172.21.15.7:0/3368336138 >> 172.21.15.7:0/3368336138 conn(0x7efe140fcfc0 msgr2=0x7efe140ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.098+0000 7efe1c322640 1 -- 172.21.15.7:0/3368336138 shutdown_connections 2024-06-10T01:26:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.098+0000 7efe1c322640 1 -- 172.21.15.7:0/3368336138 wait complete. 2024-06-10T01:26:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.098+0000 7efe1c322640 1 Processor -- start 2024-06-10T01:26:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.098+0000 7efe1c322640 1 -- start start 2024-06-10T01:26:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.098+0000 7efe1c322640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe14101a30 0x7efe14197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.098+0000 7efe1c322640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe14102c20 0x7efe14198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.098+0000 7efe1c322640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe141035e0 0x7efe1419f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.098+0000 7efe1c322640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efe1410d700 con 0x7efe14102c20 2024-06-10T01:26:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.098+0000 7efe1c322640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efe1410d580 con 0x7efe141035e0 2024-06-10T01:26:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.098+0000 7efe1c322640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efe1410d880 con 0x7efe14101a30 2024-06-10T01:26:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.098+0000 7efe198bd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe14102c20 0x7efe14198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.098+0000 7efe198bd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe14102c20 0x7efe14198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51140/0 (socket says 172.21.15.7:51140) 2024-06-10T01:26:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.098+0000 7efe198bd640 1 -- 172.21.15.7:0/1128345144 learned_addr learned my addr 172.21.15.7:0/1128345144 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.099+0000 7efe1a0be640 1 --2- 172.21.15.7:0/1128345144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe14101a30 0x7efe14197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.099+0000 7efe1a8bf640 1 --2- 172.21.15.7:0/1128345144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe141035e0 0x7efe1419f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.099+0000 7efe198bd640 1 -- 172.21.15.7:0/1128345144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe14101a30 msgr2=0x7efe14197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.099+0000 7efe198bd640 1 --2- 172.21.15.7:0/1128345144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe14101a30 0x7efe14197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.099+0000 7efe198bd640 1 -- 172.21.15.7:0/1128345144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe141035e0 msgr2=0x7efe1419f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.099+0000 7efe198bd640 1 --2- 172.21.15.7:0/1128345144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe141035e0 0x7efe1419f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.099+0000 7efe198bd640 1 -- 172.21.15.7:0/1128345144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1419f940 con 0x7efe14102c20 2024-06-10T01:26:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.099+0000 7efe198bd640 1 --2- 172.21.15.7:0/1128345144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe14102c20 0x7efe14198270 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7efe04031440 tx=0x7efe04032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.099+0000 7efe0b7fe640 1 -- 172.21.15.7:0/1128345144 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0403f030 con 0x7efe14102c20 2024-06-10T01:26:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.099+0000 7efe0b7fe640 1 -- 172.21.15.7:0/1128345144 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe04003ce0 con 0x7efe14102c20 2024-06-10T01:26:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.099+0000 7efe1c322640 1 -- 172.21.15.7:0/1128345144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe1419fbd0 con 0x7efe14102c20 2024-06-10T01:26:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.099+0000 7efe0b7fe640 1 -- 172.21.15.7:0/1128345144 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04035070 con 0x7efe14102c20 2024-06-10T01:26:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.099+0000 7efe1c322640 1 -- 172.21.15.7:0/1128345144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe141a00c0 con 0x7efe14102c20 2024-06-10T01:26:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.101+0000 7efe0b7fe640 1 -- 172.21.15.7:0/1128345144 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7efe04042030 con 0x7efe14102c20 2024-06-10T01:26:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.102+0000 7efe1f034740 1 Processor -- start 2024-06-10T01:26:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.102+0000 7efe1f034740 1 -- start start 2024-06-10T01:26:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.102+0000 7efe1f034740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebbd72a420 0x55ebbd72a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.102+0000 7efe1f034740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbd72ae10 0x55ebbd72f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.102+0000 7efe1f034740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbd72ff80 0x55ebbd732410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.102+0000 7efe1f034740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ebbd6894f0 con 0x55ebbd72ae10 2024-06-10T01:26:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.102+0000 7efe1f034740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ebbd627870 con 0x55ebbd72a420 2024-06-10T01:26:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.102+0000 7efe1f034740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ebbd6854b0 con 0x55ebbd72ff80 2024-06-10T01:26:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.102+0000 7efe1a0be640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbd72ae10 0x55ebbd72f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.102+0000 7efe1a0be640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbd72ae10 0x55ebbd72f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51150/0 (socket says 172.21.15.7:51150) 2024-06-10T01:26:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.102+0000 7efe1a0be640 1 -- 172.21.15.7:0/1744200944 learned_addr learned my addr 172.21.15.7:0/1744200944 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.103+0000 7efe0b7fe640 1 --2- 172.21.15.7:0/1128345144 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efe000a2b80 0x7efe000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.103+0000 7efe1a8bf640 1 --2- 172.21.15.7:0/1744200944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebbd72a420 0x55ebbd72a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.103+0000 7efe198bd640 1 --2- 172.21.15.7:0/1744200944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbd72ff80 0x55ebbd732410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.103+0000 7efe1a8bf640 1 --2- 172.21.15.7:0/1128345144 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efe000a2b80 0x7efe000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.103+0000 7efe1a0be640 1 -- 172.21.15.7:0/1744200944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbd72ff80 msgr2=0x55ebbd732410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.103+0000 7efe1a0be640 1 --2- 172.21.15.7:0/1744200944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbd72ff80 0x55ebbd732410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.103+0000 7efe1a0be640 1 -- 172.21.15.7:0/1744200944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebbd72a420 msgr2=0x55ebbd72a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.103+0000 7efe1a0be640 1 --2- 172.21.15.7:0/1744200944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebbd72a420 0x55ebbd72a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.103+0000 7efe1a0be640 1 -- 172.21.15.7:0/1744200944 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebbd6439b0 con 0x55ebbd72ae10 2024-06-10T01:26:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.103+0000 7efe198bd640 1 --2- 172.21.15.7:0/1744200944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbd72ff80 0x55ebbd732410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:26:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.103+0000 7efe0b7fe640 1 -- 172.21.15.7:0/1128345144 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7efe040ef770 con 0x7efe14102c20 2024-06-10T01:26:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.103+0000 7efe1a0be640 1 --2- 172.21.15.7:0/1744200944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbd72ae10 0x55ebbd72f910 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7efe1000bc70 tx=0x7efe100107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=609e0d4d4e7fea9b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.103+0000 7efe1a8bf640 1 --2- 172.21.15.7:0/1128345144 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efe000a2b80 0x7efe000a5040 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7efe0c009fd0 tx=0x7efe0c005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.103+0000 7efe08bfe640 1 -- 172.21.15.7:0/1744200944 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1001c480 con 0x55ebbd72ae10 2024-06-10T01:26:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.103+0000 7efe08bfe640 1 -- 172.21.15.7:0/1744200944 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe1001cb00 con 0x55ebbd72ae10 2024-06-10T01:26:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.103+0000 7efe08bfe640 1 -- 172.21.15.7:0/1744200944 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe10011c10 con 0x55ebbd72ae10 2024-06-10T01:26:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.104+0000 7efe1f034740 1 -- 172.21.15.7:0/1744200944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbd72ae10 msgr2=0x55ebbd72f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.104+0000 7efe1f034740 1 --2- 172.21.15.7:0/1744200944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbd72ae10 0x55ebbd72f910 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7efe1000bc70 tx=0x7efe100107a0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.104+0000 7efe1f034740 1 -- 172.21.15.7:0/1744200944 shutdown_connections 2024-06-10T01:26:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.104+0000 7efe1f034740 1 --2- 172.21.15.7:0/1744200944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbd72ff80 0x55ebbd732410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.104+0000 7efe1f034740 1 --2- 172.21.15.7:0/1744200944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbd72ae10 0x55ebbd72f910 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.104+0000 7efe1f034740 1 --2- 172.21.15.7:0/1744200944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebbd72a420 0x55ebbd72a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.104+0000 7efe1f034740 1 -- 172.21.15.7:0/1744200944 >> 172.21.15.7:0/1744200944 conn(0x55ebbd721680 msgr2=0x55ebbd721a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.104+0000 7efe1f034740 1 -- 172.21.15.7:0/1744200944 shutdown_connections 2024-06-10T01:26:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.104+0000 7efe1f034740 1 -- 172.21.15.7:0/1744200944 wait complete. 2024-06-10T01:26:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.104+0000 7efe1f034740 1 -- start start 2024-06-10T01:26:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.104+0000 7efe1f034740 1 Processor -- start 2024-06-10T01:26:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.105+0000 7efe1f034740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbd72a420 0x55ebbd734660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.105+0000 7efe1f034740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbd72ae10 0x55ebbd734ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.105+0000 7efe1f034740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebbd72ff80 0x55ebbd73bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.105+0000 7efe1f034740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ebbd732f30 con 0x55ebbd72ae10 2024-06-10T01:26:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.105+0000 7efe1f034740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ebbd612ae0 con 0x55ebbd72ff80 2024-06-10T01:26:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.105+0000 7efe1f034740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ebbd7330b0 con 0x55ebbd72a420 2024-06-10T01:26:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.105+0000 7efe1f034740 10 client.0 _ll_register_callbacks cb 0x55ebbd64b9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:26:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.105+0000 7efe198bd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbd72ae10 0x55ebbd734ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.105+0000 7efe198bd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbd72ae10 0x55ebbd734ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51160/0 (socket says 172.21.15.7:51160) 2024-06-10T01:26:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.105+0000 7efe198bd640 1 -- 172.21.15.7:0/1472157157 learned_addr learned my addr 172.21.15.7:0/1472157157 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.105+0000 7efe1a8bf640 1 --2- 172.21.15.7:0/1472157157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebbd72ff80 0x55ebbd73bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.105+0000 7efe1a0be640 1 --2- 172.21.15.7:0/1472157157 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbd72a420 0x55ebbd734660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.105+0000 7efe198bd640 1 -- 172.21.15.7:0/1472157157 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbd72a420 msgr2=0x55ebbd734660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.105+0000 7efe198bd640 1 --2- 172.21.15.7:0/1472157157 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbd72a420 0x55ebbd734660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.105+0000 7efe198bd640 1 -- 172.21.15.7:0/1472157157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebbd72ff80 msgr2=0x55ebbd73bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.105+0000 7efe198bd640 1 --2- 172.21.15.7:0/1472157157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebbd72ff80 0x55ebbd73bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.105+0000 7efe198bd640 1 -- 172.21.15.7:0/1472157157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebbd39d210 con 0x55ebbd72ae10 2024-06-10T01:26:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.105+0000 7efe198bd640 1 --2- 172.21.15.7:0/1472157157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbd72ae10 0x55ebbd734ba0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7efe04000c00 tx=0x7efe0408aae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df7047eb0b5c11b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.105+0000 7efdfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:26:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.105+0000 7efdfdffb640 1 -- 172.21.15.7:0/1472157157 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04049070 con 0x55ebbd72ae10 2024-06-10T01:26:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.105+0000 7efdfdffb640 1 -- 172.21.15.7:0/1472157157 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7efe040a6330 con 0x55ebbd72ae10 2024-06-10T01:26:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.105+0000 7efe1f034740 1 -- 172.21.15.7:0/1472157157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe1001cb00 con 0x55ebbd72ae10 2024-06-10T01:26:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.106+0000 7efdfdffb640 1 -- 172.21.15.7:0/1472157157 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7efe040b9d30 con 0x55ebbd72ae10 2024-06-10T01:26:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.106+0000 7efe1f034740 10 client.20319 fetch_fsmap learned FSMap version 52 2024-06-10T01:26:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.106+0000 7efe1f034740 1 -- 172.21.15.7:0/1472157157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55ebbd469620 con 0x55ebbd72ae10 2024-06-10T01:26:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.106+0000 7efdfdffb640 1 -- 172.21.15.7:0/1472157157 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe1001cb00 con 0x55ebbd72ae10 2024-06-10T01:26:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.106+0000 7efdfdffb640 1 -- 172.21.15.7:0/1472157157 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7efe040bbe50 con 0x55ebbd72ae10 2024-06-10T01:26:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.106+0000 7efe1f034740 10 client.20319 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:26:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.106+0000 7efe1f034740 10 client.20319 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:26:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.106+0000 7efe1f034740 20 client.20319 populate_metadata read hostname 'smithi007' 2024-06-10T01:26:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.106+0000 7efe1f034740 1 --2- 172.21.15.7:0/1472157157 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ebbd740420 0x55ebbd742850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.106+0000 7efe1f034740 4 client.20319 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:26:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.106+0000 7efe1f034740 1 -- 172.21.15.7:0/1472157157 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebbd69cf10 con 0x55ebbd740420 2024-06-10T01:26:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.106+0000 7efe1a0be640 1 --2- 172.21.15.7:0/1472157157 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ebbd740420 0x55ebbd742850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.107+0000 7efe1a0be640 1 --2- 172.21.15.7:0/1472157157 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ebbd740420 0x55ebbd742850 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9d4d88b02597b2b server_cookie=2a61666b97914c58 in_seq=0 out_seq=0 2024-06-10T01:26:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.107+0000 7efdfdffb640 10 client.20319 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.108+0000 7efdfdffb640 1 -- 172.21.15.7:0/1472157157 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ebbd69cf10 con 0x55ebbd740420 2024-06-10T01:26:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.108+0000 7efdfdffb640 10 client.20319 handle_command_reply: tid=0 2024-06-10T01:26:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.108+0000 7efe1f034740 1 client.20319 shutdown 2024-06-10T01:26:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.108+0000 7efe1f034740 1 -- 172.21.15.7:0/1472157157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbd72ae10 msgr2=0x55ebbd734ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.108+0000 7efe1f034740 1 --2- 172.21.15.7:0/1472157157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbd72ae10 0x55ebbd734ba0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7efe04000c00 tx=0x7efe0408aae0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.108+0000 7efe1f034740 1 -- 172.21.15.7:0/1472157157 shutdown_connections 2024-06-10T01:26:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.108+0000 7efe1f034740 1 --2- 172.21.15.7:0/1472157157 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ebbd740420 0x55ebbd742850 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.108+0000 7efe1f034740 1 --2- 172.21.15.7:0/1472157157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebbd72ff80 0x55ebbd73bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.108+0000 7efdfdffb640 0 client.20319 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.108+0000 7efe1f034740 1 --2- 172.21.15.7:0/1472157157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbd72ae10 0x55ebbd734ba0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.108+0000 7efe1f034740 1 --2- 172.21.15.7:0/1472157157 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbd72a420 0x55ebbd734660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.108+0000 7efe1f034740 1 -- 172.21.15.7:0/1472157157 >> 172.21.15.7:0/1472157157 conn(0x55ebbd721680 msgr2=0x55ebbd7207e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.108+0000 7efe1f034740 1 -- 172.21.15.7:0/1472157157 shutdown_connections 2024-06-10T01:26:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.108+0000 7efe1f034740 1 -- 172.21.15.7:0/1472157157 wait complete. 2024-06-10T01:26:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.108+0000 7efe1f034740 20 client.20319 trim_cache size 0 max 16384 2024-06-10T01:26:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.123+0000 7efe1f034740 1 Processor -- start 2024-06-10T01:26:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.123+0000 7efe1f034740 1 -- start start 2024-06-10T01:26:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.123+0000 7efe1f034740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebbd72a420 0x55ebbd72bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.123+0000 7efe1f034740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbd72ae10 0x55ebbd72c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.124+0000 7efe1f034740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbd72ff80 0x55ebbd73bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.124+0000 7efe1f034740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efe141aab60 con 0x55ebbd72ff80 2024-06-10T01:26:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.124+0000 7efe1f034740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efe141a8430 con 0x55ebbd72a420 2024-06-10T01:26:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.124+0000 7efe1f034740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efe141a6280 con 0x55ebbd72ae10 2024-06-10T01:26:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.124+0000 7efe1a8bf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbd72ff80 0x55ebbd73bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.124+0000 7efe1a8bf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbd72ff80 0x55ebbd73bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51170/0 (socket says 172.21.15.7:51170) 2024-06-10T01:26:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.124+0000 7efe1a8bf640 1 -- 172.21.15.7:0/3061234658 learned_addr learned my addr 172.21.15.7:0/3061234658 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.124+0000 7efe1a0be640 1 --2- 172.21.15.7:0/3061234658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebbd72a420 0x55ebbd72bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.124+0000 7efe198bd640 1 --2- 172.21.15.7:0/3061234658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbd72ae10 0x55ebbd72c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.124+0000 7efe1a8bf640 1 -- 172.21.15.7:0/3061234658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbd72ae10 msgr2=0x55ebbd72c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.124+0000 7efe1a8bf640 1 --2- 172.21.15.7:0/3061234658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbd72ae10 0x55ebbd72c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.124+0000 7efe1a8bf640 1 -- 172.21.15.7:0/3061234658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebbd72a420 msgr2=0x55ebbd72bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.124+0000 7efe1a8bf640 1 --2- 172.21.15.7:0/3061234658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebbd72a420 0x55ebbd72bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.124+0000 7efe1a8bf640 1 -- 172.21.15.7:0/3061234658 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebbd7214b0 con 0x55ebbd72ff80 2024-06-10T01:26:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.124+0000 7efe1a8bf640 1 --2- 172.21.15.7:0/3061234658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbd72ff80 0x55ebbd73bba0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7efe0c010890 tx=0x7efe0c009720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a92cd780dafbad server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.124+0000 7efdfffff640 1 -- 172.21.15.7:0/3061234658 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c010e10 con 0x55ebbd72ff80 2024-06-10T01:26:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.124+0000 7efdfffff640 1 -- 172.21.15.7:0/3061234658 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe0c020980 con 0x55ebbd72ff80 2024-06-10T01:26:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.124+0000 7efdfffff640 1 -- 172.21.15.7:0/3061234658 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c0057d0 con 0x55ebbd72ff80 2024-06-10T01:26:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.125+0000 7efe1f034740 1 -- 172.21.15.7:0/3061234658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbd72ff80 msgr2=0x55ebbd73bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.125+0000 7efe1f034740 1 --2- 172.21.15.7:0/3061234658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbd72ff80 0x55ebbd73bba0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7efe0c010890 tx=0x7efe0c009720 comp rx=0 tx=0).stop 2024-06-10T01:26:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.125+0000 7efe1f034740 1 -- 172.21.15.7:0/3061234658 shutdown_connections 2024-06-10T01:26:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.125+0000 7efe1f034740 1 --2- 172.21.15.7:0/3061234658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbd72ff80 0x55ebbd73bba0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.125+0000 7efe1f034740 1 --2- 172.21.15.7:0/3061234658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbd72ae10 0x55ebbd72c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.125+0000 7efe1f034740 1 --2- 172.21.15.7:0/3061234658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebbd72a420 0x55ebbd72bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.125+0000 7efe1f034740 1 -- 172.21.15.7:0/3061234658 >> 172.21.15.7:0/3061234658 conn(0x55ebbd721680 msgr2=0x55ebbd734660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.125+0000 7efe1f034740 1 -- 172.21.15.7:0/3061234658 shutdown_connections 2024-06-10T01:26:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.125+0000 7efe1f034740 1 -- 172.21.15.7:0/3061234658 wait complete. 2024-06-10T01:26:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.125+0000 7efe1f034740 1 -- start start 2024-06-10T01:26:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.125+0000 7efe1f034740 1 Processor -- start 2024-06-10T01:26:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.126+0000 7efe1f034740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebbd72a420 0x55ebbd738df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.126+0000 7efe1f034740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbd72ae10 0x55ebbd739330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.126+0000 7efe1f034740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbd72ff80 0x55ebbd73a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.126+0000 7efe1f034740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efe141a68f0 con 0x55ebbd72ae10 2024-06-10T01:26:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.126+0000 7efe1f034740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efe141a7d60 con 0x55ebbd72a420 2024-06-10T01:26:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.126+0000 7efe1f034740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efe141a6fc0 con 0x55ebbd72ff80 2024-06-10T01:26:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.126+0000 7efe1f034740 10 client.0 _ll_register_callbacks cb 0x7efe04009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:26:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.126+0000 7efe198bd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbd72ae10 0x55ebbd739330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.126+0000 7efe198bd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbd72ae10 0x55ebbd739330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51184/0 (socket says 172.21.15.7:51184) 2024-06-10T01:26:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.126+0000 7efe198bd640 1 -- 172.21.15.7:0/1730606371 learned_addr learned my addr 172.21.15.7:0/1730606371 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.126+0000 7efe1a0be640 1 --2- 172.21.15.7:0/1730606371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebbd72a420 0x55ebbd738df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.126+0000 7efe1a8bf640 1 --2- 172.21.15.7:0/1730606371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbd72ff80 0x55ebbd73a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.126+0000 7efe198bd640 1 -- 172.21.15.7:0/1730606371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbd72ff80 msgr2=0x55ebbd73a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.126+0000 7efe198bd640 1 --2- 172.21.15.7:0/1730606371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbd72ff80 0x55ebbd73a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.126+0000 7efe198bd640 1 -- 172.21.15.7:0/1730606371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebbd72a420 msgr2=0x55ebbd738df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.126+0000 7efe198bd640 1 --2- 172.21.15.7:0/1730606371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebbd72a420 0x55ebbd738df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.126+0000 7efe198bd640 1 -- 172.21.15.7:0/1730606371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebbd72bae0 con 0x55ebbd72ae10 2024-06-10T01:26:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.126+0000 7efe198bd640 1 --2- 172.21.15.7:0/1730606371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbd72ae10 0x55ebbd739330 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7efe040bb8f0 tx=0x7efe040a6640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11eb4b4c958c3da5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.126+0000 7efdfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:26:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.126+0000 7efdfdffb640 1 -- 172.21.15.7:0/1730606371 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe040d2070 con 0x55ebbd72ae10 2024-06-10T01:26:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.126+0000 7efdfdffb640 1 -- 172.21.15.7:0/1730606371 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7efe040bbe50 con 0x55ebbd72ae10 2024-06-10T01:26:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.126+0000 7efe1f034740 1 -- 172.21.15.7:0/1730606371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe0c020980 con 0x55ebbd72ae10 2024-06-10T01:26:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.126+0000 7efdfdffb640 1 -- 172.21.15.7:0/1730606371 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7efe04049040 con 0x55ebbd72ae10 2024-06-10T01:26:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.127+0000 7efe1f034740 10 client.20325 fetch_fsmap learned FSMap version 52 2024-06-10T01:26:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.127+0000 7efe1f034740 1 -- 172.21.15.7:0/1730606371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55ebbd73b9e0 con 0x55ebbd72ae10 2024-06-10T01:26:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.127+0000 7efdfdffb640 1 -- 172.21.15.7:0/1730606371 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe0c020980 con 0x55ebbd72ae10 2024-06-10T01:26:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.127+0000 7efdfdffb640 1 -- 172.21.15.7:0/1730606371 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7efe0403b460 con 0x55ebbd72ae10 2024-06-10T01:26:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.127+0000 7efe1f034740 10 client.20325 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:26:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.127+0000 7efe1f034740 10 client.20325 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:26:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.127+0000 7efe1f034740 20 client.20325 populate_metadata read hostname 'smithi007' 2024-06-10T01:26:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.127+0000 7efe1f034740 1 --2- 172.21.15.7:0/1730606371 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ebbd7ab130 0x55ebbd7ad520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.127+0000 7efe1f034740 4 client.20325 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:26:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.127+0000 7efe1f034740 1 -- 172.21.15.7:0/1730606371 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ebbd696c20 con 0x55ebbd7ab130 2024-06-10T01:26:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.127+0000 7efe1a0be640 1 --2- 172.21.15.7:0/1730606371 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ebbd7ab130 0x55ebbd7ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.127+0000 7efdfdffb640 10 client.20325 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.127+0000 7efe1a0be640 1 --2- 172.21.15.7:0/1730606371 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ebbd7ab130 0x55ebbd7ad520 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d58e2404f78863fd server_cookie=a4f615401c049ae6 in_seq=0 out_seq=0 2024-06-10T01:26:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.201+0000 7efdfdffb640 1 -- 172.21.15.7:0/1730606371 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ebbd696c20 con 0x55ebbd7ab130 2024-06-10T01:26:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.202+0000 7efdfdffb640 10 client.20325 handle_command_reply: tid=0 2024-06-10T01:26:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.202+0000 7efe1f034740 1 client.20325 shutdown 2024-06-10T01:26:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.202+0000 7efe1f034740 1 -- 172.21.15.7:0/1730606371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbd72ae10 msgr2=0x55ebbd739330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.202+0000 7efe1f034740 1 --2- 172.21.15.7:0/1730606371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbd72ae10 0x55ebbd739330 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7efe040bb8f0 tx=0x7efe040a6640 comp rx=0 tx=0).stop 2024-06-10T01:26:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.202+0000 7efe1f034740 1 -- 172.21.15.7:0/1730606371 shutdown_connections 2024-06-10T01:26:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.202+0000 7efe1f034740 1 --2- 172.21.15.7:0/1730606371 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ebbd7ab130 0x55ebbd7ad520 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.202+0000 7efe1f034740 1 --2- 172.21.15.7:0/1730606371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbd72ff80 0x55ebbd73a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.202+0000 7efdfdffb640 0 client.20325 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.202+0000 7efe1f034740 1 --2- 172.21.15.7:0/1730606371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbd72ae10 0x55ebbd739330 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.202+0000 7efe1f034740 1 --2- 172.21.15.7:0/1730606371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ebbd72a420 0x55ebbd738df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.202+0000 7efe1f034740 1 -- 172.21.15.7:0/1730606371 >> 172.21.15.7:0/1730606371 conn(0x55ebbd721680 msgr2=0x55ebbd734660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.202+0000 7efe1f034740 1 -- 172.21.15.7:0/1730606371 shutdown_connections 2024-06-10T01:26:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.202+0000 7efe1f034740 1 -- 172.21.15.7:0/1730606371 wait complete. 2024-06-10T01:26:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.202+0000 7efe1f034740 20 client.20325 trim_cache size 0 max 16384 2024-06-10T01:26:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:26:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:26:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:26:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.202+0000 7efe1c322640 1 -- 172.21.15.7:0/1128345144 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efe000a2b80 msgr2=0x7efe000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.202+0000 7efe1c322640 1 --2- 172.21.15.7:0/1128345144 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efe000a2b80 0x7efe000a5040 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7efe0c009fd0 tx=0x7efe0c005bd0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.202+0000 7efe1c322640 1 -- 172.21.15.7:0/1128345144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe14102c20 msgr2=0x7efe14198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.202+0000 7efe1c322640 1 --2- 172.21.15.7:0/1128345144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe14102c20 0x7efe14198270 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7efe04031440 tx=0x7efe04032e20 comp rx=0 tx=0).stop 2024-06-10T01:26:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.203+0000 7efe1c322640 1 -- 172.21.15.7:0/1128345144 shutdown_connections 2024-06-10T01:26:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.203+0000 7efe1c322640 1 --2- 172.21.15.7:0/1128345144 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7efe000a2b80 0x7efe000a5040 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.203+0000 7efe1c322640 1 --2- 172.21.15.7:0/1128345144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe141035e0 0x7efe1419f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.203+0000 7efe1c322640 1 --2- 172.21.15.7:0/1128345144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe14102c20 0x7efe14198270 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.203+0000 7efe1c322640 1 --2- 172.21.15.7:0/1128345144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe14101a30 0x7efe14197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.203+0000 7efe1c322640 1 -- 172.21.15.7:0/1128345144 >> 172.21.15.7:0/1128345144 conn(0x7efe140fcfc0 msgr2=0x7efe14105ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.203+0000 7efe1c322640 1 -- 172.21.15.7:0/1128345144 shutdown_connections 2024-06-10T01:26:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:22.203+0000 7efe1c322640 1 -- 172.21.15.7:0/1128345144 wait complete. 2024-06-10T01:26:22.257 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db --set-id d3f8e8a1 --release --await' 2024-06-10T01:26:22.257 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db --set-id d3f8e8a1 --release --await 2024-06-10T01:26:22.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.486+0000 7fc7916a9700 1 -- 172.21.15.7:0/873849020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc78c113660 msgr2=0x7fc78c115a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:22.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.486+0000 7fc7916a9700 1 --2- 172.21.15.7:0/873849020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc78c113660 0x7fc78c115a50 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fc78400b0a0 tx=0x7fc78400b3b0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.486+0000 7fc7916a9700 1 -- 172.21.15.7:0/873849020 shutdown_connections 2024-06-10T01:26:22.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.486+0000 7fc7916a9700 1 --2- 172.21.15.7:0/873849020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc78c113660 0x7fc78c115a50 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.486+0000 7fc7916a9700 1 --2- 172.21.15.7:0/873849020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc78c074700 0x7fc78c074ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.486+0000 7fc7916a9700 1 --2- 172.21.15.7:0/873849020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc78c075020 0x7fc78c113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.486+0000 7fc7916a9700 1 -- 172.21.15.7:0/873849020 >> 172.21.15.7:0/873849020 conn(0x7fc78c079680 msgr2=0x7fc78c079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:22.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.486+0000 7fc7916a9700 1 -- 172.21.15.7:0/873849020 shutdown_connections 2024-06-10T01:26:22.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.486+0000 7fc7916a9700 1 -- 172.21.15.7:0/873849020 wait complete. 2024-06-10T01:26:22.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.487+0000 7fc7916a9700 1 Processor -- start 2024-06-10T01:26:22.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.487+0000 7fc7916a9700 1 -- start start 2024-06-10T01:26:22.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.487+0000 7fc7916a9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc78c074700 0x7fc78c19f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.488+0000 7fc7916a9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc78c075020 0x7fc78c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.488+0000 7fc7916a9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc78c113660 0x7fc78c1a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.488+0000 7fc7916a9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c118e10 con 0x7fc78c075020 2024-06-10T01:26:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.488+0000 7fc7916a9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c118c90 con 0x7fc78c074700 2024-06-10T01:26:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.488+0000 7fc7916a9700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c118f90 con 0x7fc78c113660 2024-06-10T01:26:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.488+0000 7fc78affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc78c074700 0x7fc78c19f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.488+0000 7fc78a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc78c075020 0x7fc78c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.488+0000 7fc78affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc78c074700 0x7fc78c19f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:39722/0 (socket says 172.21.15.7:39722) 2024-06-10T01:26:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.488+0000 7fc78affd700 1 -- 172.21.15.7:0/1808159946 learned_addr learned my addr 172.21.15.7:0/1808159946 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.488+0000 7fc78b7fe700 1 --2- 172.21.15.7:0/1808159946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc78c113660 0x7fc78c1a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.488+0000 7fc78a7fc700 1 -- 172.21.15.7:0/1808159946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc78c113660 msgr2=0x7fc78c1a3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.488+0000 7fc78a7fc700 1 --2- 172.21.15.7:0/1808159946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc78c113660 0x7fc78c1a3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.488+0000 7fc78a7fc700 1 -- 172.21.15.7:0/1808159946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc78c074700 msgr2=0x7fc78c19f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.488+0000 7fc78a7fc700 1 --2- 172.21.15.7:0/1808159946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc78c074700 0x7fc78c19f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.488+0000 7fc78a7fc700 1 -- 172.21.15.7:0/1808159946 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78c1a4210 con 0x7fc78c075020 2024-06-10T01:26:22.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.488+0000 7fc78b7fe700 1 --2- 172.21.15.7:0/1808159946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc78c113660 0x7fc78c1a3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:26:22.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.488+0000 7fc78affd700 1 --2- 172.21.15.7:0/1808159946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc78c074700 0x7fc78c19f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:26:22.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.488+0000 7fc78a7fc700 1 --2- 172.21.15.7:0/1808159946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc78c075020 0x7fc78c19f740 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fc78000d750 tx=0x7fc78000da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:22.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.489+0000 7fc773fff700 1 -- 172.21.15.7:0/1808159946 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc780014030 con 0x7fc78c075020 2024-06-10T01:26:22.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.489+0000 7fc773fff700 1 -- 172.21.15.7:0/1808159946 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc780004500 con 0x7fc78c075020 2024-06-10T01:26:22.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.489+0000 7fc7916a9700 1 -- 172.21.15.7:0/1808159946 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc78c1a4500 con 0x7fc78c075020 2024-06-10T01:26:22.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.489+0000 7fc773fff700 1 -- 172.21.15.7:0/1808159946 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7800054e0 con 0x7fc78c075020 2024-06-10T01:26:22.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.489+0000 7fc7916a9700 1 -- 172.21.15.7:0/1808159946 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc78c1abd80 con 0x7fc78c075020 2024-06-10T01:26:22.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.492+0000 7fc773fff700 1 -- 172.21.15.7:0/1808159946 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fc780004750 con 0x7fc78c075020 2024-06-10T01:26:22.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.493+0000 7fc773fff700 1 --2- 172.21.15.7:0/1808159946 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc7740a2ca0 0x7fc7740a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.493+0000 7fc78affd700 1 --2- 172.21.15.7:0/1808159946 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc7740a2ca0 0x7fc7740a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.493+0000 7fc773fff700 1 -- 172.21.15.7:0/1808159946 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fc7800ceb40 con 0x7fc78c075020 2024-06-10T01:26:22.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.494+0000 7fc78affd700 1 --2- 172.21.15.7:0/1808159946 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc7740a2ca0 0x7fc7740a5160 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc77c005950 tx=0x7fc77c00b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:22.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.494+0000 7fc79bf2cb80 1 Processor -- start 2024-06-10T01:26:22.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.494+0000 7fc79bf2cb80 1 -- start start 2024-06-10T01:26:22.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.494+0000 7fc79bf2cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f67e9cf60 0x556f67e9d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.494+0000 7fc79bf2cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f67e9d880 0x556f67e97260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.494+0000 7fc79bf2cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f67e977a0 0x556f67e97bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.494+0000 7fc79bf2cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556f67d89f60 con 0x556f67e977a0 2024-06-10T01:26:22.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.494+0000 7fc79bf2cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556f67de8850 con 0x556f67e9d880 2024-06-10T01:26:22.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.494+0000 7fc79bf2cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556f67d50460 con 0x556f67e9cf60 2024-06-10T01:26:22.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.494+0000 7fc78b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f67e977a0 0x556f67e97bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.494+0000 7fc78b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f67e977a0 0x556f67e97bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51206/0 (socket says 172.21.15.7:51206) 2024-06-10T01:26:22.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.494+0000 7fc78b7fe700 1 -- 172.21.15.7:0/1413503651 learned_addr learned my addr 172.21.15.7:0/1413503651 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:22.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.495+0000 7fc78a7fc700 1 --2- 172.21.15.7:0/1413503651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f67e9d880 0x556f67e97260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.495+0000 7fc78affd700 1 --2- 172.21.15.7:0/1413503651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f67e9cf60 0x556f67e9d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.495+0000 7fc78b7fe700 1 -- 172.21.15.7:0/1413503651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f67e9cf60 msgr2=0x556f67e9d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:22.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.495+0000 7fc78b7fe700 1 --2- 172.21.15.7:0/1413503651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f67e9cf60 0x556f67e9d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.495+0000 7fc78b7fe700 1 -- 172.21.15.7:0/1413503651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f67e9d880 msgr2=0x556f67e97260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:22.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.495+0000 7fc78b7fe700 1 --2- 172.21.15.7:0/1413503651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f67e9d880 0x556f67e97260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.495+0000 7fc78b7fe700 1 -- 172.21.15.7:0/1413503651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f67d378d0 con 0x556f67e977a0 2024-06-10T01:26:22.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.495+0000 7fc78b7fe700 1 --2- 172.21.15.7:0/1413503651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f67e977a0 0x556f67e97bc0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fc784015000 tx=0x7fc784009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9efff3f76fd69aae server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:22.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.495+0000 7fc7717fa700 1 -- 172.21.15.7:0/1413503651 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc784004d30 con 0x556f67e977a0 2024-06-10T01:26:22.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.495+0000 7fc7717fa700 1 -- 172.21.15.7:0/1413503651 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7840079c0 con 0x556f67e977a0 2024-06-10T01:26:22.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.496+0000 7fc7717fa700 1 -- 172.21.15.7:0/1413503651 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc784005680 con 0x556f67e977a0 2024-06-10T01:26:22.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.496+0000 7fc79bf2cb80 1 -- 172.21.15.7:0/1413503651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f67e977a0 msgr2=0x556f67e97bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:22.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.496+0000 7fc79bf2cb80 1 --2- 172.21.15.7:0/1413503651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f67e977a0 0x556f67e97bc0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fc784015000 tx=0x7fc784009500 comp rx=0 tx=0).stop 2024-06-10T01:26:22.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.496+0000 7fc79bf2cb80 1 -- 172.21.15.7:0/1413503651 shutdown_connections 2024-06-10T01:26:22.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.496+0000 7fc79bf2cb80 1 --2- 172.21.15.7:0/1413503651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f67e977a0 0x556f67e97bc0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.496+0000 7fc79bf2cb80 1 --2- 172.21.15.7:0/1413503651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f67e9cf60 0x556f67e9d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.496+0000 7fc79bf2cb80 1 --2- 172.21.15.7:0/1413503651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f67e9d880 0x556f67e97260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.496+0000 7fc79bf2cb80 1 -- 172.21.15.7:0/1413503651 >> 172.21.15.7:0/1413503651 conn(0x556f67dc4080 msgr2=0x556f67e92220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:22.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.496+0000 7fc79bf2cb80 1 -- 172.21.15.7:0/1413503651 shutdown_connections 2024-06-10T01:26:22.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.496+0000 7fc79bf2cb80 1 -- 172.21.15.7:0/1413503651 wait complete. 2024-06-10T01:26:22.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.497+0000 7fc79bf2cb80 1 -- start start 2024-06-10T01:26:22.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.497+0000 7fc79bf2cb80 1 Processor -- start 2024-06-10T01:26:22.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.497+0000 7fc79bf2cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f67e977a0 0x556f67ea34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.497+0000 7fc79bf2cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f67e9cf60 0x556f67ea07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.497+0000 7fc79bf2cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f67e9d880 0x556f67ea0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.497+0000 7fc79bf2cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556f67d88fc0 con 0x556f67e9d880 2024-06-10T01:26:22.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.497+0000 7fc79bf2cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556f67de4810 con 0x556f67e9cf60 2024-06-10T01:26:22.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.497+0000 7fc79bf2cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556f67ea4250 con 0x556f67e977a0 2024-06-10T01:26:22.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.497+0000 7fc79bf2cb80 10 client.0 _ll_register_callbacks cb 0x556f67d7e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:26:22.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.497+0000 7fc78b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f67e9d880 0x556f67ea0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.497+0000 7fc78b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f67e9d880 0x556f67ea0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51220/0 (socket says 172.21.15.7:51220) 2024-06-10T01:26:22.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.497+0000 7fc78b7fe700 1 -- 172.21.15.7:0/3897342692 learned_addr learned my addr 172.21.15.7:0/3897342692 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:22.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.498+0000 7fc78a7fc700 1 --2- 172.21.15.7:0/3897342692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f67e9cf60 0x556f67ea07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.498+0000 7fc78affd700 1 --2- 172.21.15.7:0/3897342692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f67e977a0 0x556f67ea34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.498+0000 7fc78b7fe700 1 -- 172.21.15.7:0/3897342692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f67e977a0 msgr2=0x556f67ea34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:22.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.498+0000 7fc78b7fe700 1 --2- 172.21.15.7:0/3897342692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f67e977a0 0x556f67ea34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.498+0000 7fc78b7fe700 1 -- 172.21.15.7:0/3897342692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f67e9cf60 msgr2=0x556f67ea07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:22.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.498+0000 7fc78b7fe700 1 --2- 172.21.15.7:0/3897342692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f67e9cf60 0x556f67ea07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.498+0000 7fc78b7fe700 1 -- 172.21.15.7:0/3897342692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f67dae750 con 0x556f67e9d880 2024-06-10T01:26:22.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.498+0000 7fc78a7fc700 1 --2- 172.21.15.7:0/3897342692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f67e9cf60 0x556f67ea07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:26:22.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.498+0000 7fc78b7fe700 1 --2- 172.21.15.7:0/3897342692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f67e9d880 0x556f67ea0ce0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fc78400b3f0 tx=0x7fc784004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6cdb332ba68c73 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:22.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.498+0000 7fc76eff5700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:26:22.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.498+0000 7fc76eff5700 1 -- 172.21.15.7:0/3897342692 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc784005260 con 0x556f67e9d880 2024-06-10T01:26:22.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.498+0000 7fc76eff5700 1 -- 172.21.15.7:0/3897342692 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fc784005400 con 0x556f67e9d880 2024-06-10T01:26:22.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.498+0000 7fc79bf2cb80 1 -- 172.21.15.7:0/3897342692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7840079c0 con 0x556f67e9d880 2024-06-10T01:26:22.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.498+0000 7fc76eff5700 1 -- 172.21.15.7:0/3897342692 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fc784039050 con 0x556f67e9d880 2024-06-10T01:26:22.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.498+0000 7fc79bf2cb80 10 client.20337 fetch_fsmap learned FSMap version 52 2024-06-10T01:26:22.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.498+0000 7fc79bf2cb80 1 -- 172.21.15.7:0/3897342692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x556f67eae900 con 0x556f67e9d880 2024-06-10T01:26:22.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.499+0000 7fc76eff5700 1 -- 172.21.15.7:0/3897342692 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7840079c0 con 0x556f67e9d880 2024-06-10T01:26:22.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.499+0000 7fc76eff5700 1 -- 172.21.15.7:0/3897342692 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7fc784011070 con 0x556f67e9d880 2024-06-10T01:26:22.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.499+0000 7fc79bf2cb80 10 client.20337 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:26:22.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.499+0000 7fc79bf2cb80 10 client.20337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:26:22.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.499+0000 7fc79bf2cb80 20 client.20337 populate_metadata read hostname 'smithi007' 2024-06-10T01:26:22.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.499+0000 7fc79bf2cb80 1 --2- 172.21.15.7:0/3897342692 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556f67eb0dd0 0x556f67eb31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.499+0000 7fc79bf2cb80 4 client.20337 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:26:22.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.499+0000 7fc79bf2cb80 1 -- 172.21.15.7:0/3897342692 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f67df5d50 con 0x556f67eb0dd0 2024-06-10T01:26:22.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.499+0000 7fc78affd700 1 --2- 172.21.15.7:0/3897342692 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556f67eb0dd0 0x556f67eb31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.500+0000 7fc78affd700 1 --2- 172.21.15.7:0/3897342692 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556f67eb0dd0 0x556f67eb31c0 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16c6a1bfba305a84 server_cookie=db22fd25bfab1041 in_seq=0 out_seq=0 2024-06-10T01:26:22.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.500+0000 7fc76eff5700 10 client.20337 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:22.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.500+0000 7fc76eff5700 1 -- 172.21.15.7:0/3897342692 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556f67df5d50 con 0x556f67eb0dd0 2024-06-10T01:26:22.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.500+0000 7fc76eff5700 10 client.20337 handle_command_reply: tid=0 2024-06-10T01:26:22.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.501+0000 7fc79bf2cb80 1 client.20337 shutdown 2024-06-10T01:26:22.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.501+0000 7fc79bf2cb80 1 -- 172.21.15.7:0/3897342692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f67e9d880 msgr2=0x556f67ea0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:22.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.501+0000 7fc79bf2cb80 1 --2- 172.21.15.7:0/3897342692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f67e9d880 0x556f67ea0ce0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fc78400b3f0 tx=0x7fc784004e50 comp rx=0 tx=0).stop 2024-06-10T01:26:22.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.501+0000 7fc79bf2cb80 1 -- 172.21.15.7:0/3897342692 shutdown_connections 2024-06-10T01:26:22.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.501+0000 7fc79bf2cb80 1 --2- 172.21.15.7:0/3897342692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f67e9d880 0x556f67ea0ce0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.501+0000 7fc79bf2cb80 1 --2- 172.21.15.7:0/3897342692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f67e977a0 0x556f67ea34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.501+0000 7fc79bf2cb80 1 --2- 172.21.15.7:0/3897342692 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556f67eb0dd0 0x556f67eb31c0 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.501+0000 7fc79bf2cb80 1 --2- 172.21.15.7:0/3897342692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f67e9cf60 0x556f67ea07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.501+0000 7fc76eff5700 0 client.20337 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:22.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.501+0000 7fc79bf2cb80 1 -- 172.21.15.7:0/3897342692 >> 172.21.15.7:0/3897342692 conn(0x556f67dc4080 msgr2=0x556f67e91700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:22.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.501+0000 7fc79bf2cb80 1 -- 172.21.15.7:0/3897342692 shutdown_connections 2024-06-10T01:26:22.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.501+0000 7fc79bf2cb80 1 -- 172.21.15.7:0/3897342692 wait complete. 2024-06-10T01:26:22.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.501+0000 7fc79bf2cb80 20 client.20337 trim_cache size 0 max 16384 2024-06-10T01:26:22.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.523+0000 7fc79bf2cb80 1 Processor -- start 2024-06-10T01:26:22.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.523+0000 7fc79bf2cb80 1 -- start start 2024-06-10T01:26:22.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.523+0000 7fc79bf2cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f67e977a0 0x556f67e97260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.523+0000 7fc79bf2cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f67e9cf60 0x556f67e91e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.523+0000 7fc79bf2cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f67e9d880 0x556f67e923c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.523+0000 7fc79bf2cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c1b7010 con 0x556f67e9d880 2024-06-10T01:26:22.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.523+0000 7fc79bf2cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c1b48a0 con 0x556f67e9cf60 2024-06-10T01:26:22.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.523+0000 7fc79bf2cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c1b26f0 con 0x556f67e977a0 2024-06-10T01:26:22.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.524+0000 7fc78b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f67e9d880 0x556f67e923c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.524+0000 7fc78b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f67e9d880 0x556f67e923c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51226/0 (socket says 172.21.15.7:51226) 2024-06-10T01:26:22.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.524+0000 7fc78b7fe700 1 -- 172.21.15.7:0/3110148116 learned_addr learned my addr 172.21.15.7:0/3110148116 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:22.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.524+0000 7fc78affd700 1 --2- 172.21.15.7:0/3110148116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f67e977a0 0x556f67e97260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.524+0000 7fc78a7fc700 1 --2- 172.21.15.7:0/3110148116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f67e9cf60 0x556f67e91e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.524+0000 7fc78b7fe700 1 -- 172.21.15.7:0/3110148116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f67e977a0 msgr2=0x556f67e97260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:22.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.524+0000 7fc78b7fe700 1 --2- 172.21.15.7:0/3110148116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f67e977a0 0x556f67e97260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.524+0000 7fc78b7fe700 1 -- 172.21.15.7:0/3110148116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f67e9cf60 msgr2=0x556f67e91e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:22.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.524+0000 7fc78b7fe700 1 --2- 172.21.15.7:0/3110148116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f67e9cf60 0x556f67e91e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.524+0000 7fc78b7fe700 1 -- 172.21.15.7:0/3110148116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f67e9f760 con 0x556f67e9d880 2024-06-10T01:26:22.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.524+0000 7fc78b7fe700 1 --2- 172.21.15.7:0/3110148116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f67e9d880 0x556f67e923c0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fc784002230 tx=0x7fc784000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3310554d1e5fc6ef server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:22.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.524+0000 7fc770ff9700 1 -- 172.21.15.7:0/3110148116 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7840079b0 con 0x556f67e9d880 2024-06-10T01:26:22.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.524+0000 7fc770ff9700 1 -- 172.21.15.7:0/3110148116 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc78401c410 con 0x556f67e9d880 2024-06-10T01:26:22.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.524+0000 7fc770ff9700 1 -- 172.21.15.7:0/3110148116 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7840054a0 con 0x556f67e9d880 2024-06-10T01:26:22.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.525+0000 7fc79bf2cb80 1 -- 172.21.15.7:0/3110148116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f67e9d880 msgr2=0x556f67e923c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:22.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.525+0000 7fc79bf2cb80 1 --2- 172.21.15.7:0/3110148116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f67e9d880 0x556f67e923c0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fc784002230 tx=0x7fc784000f40 comp rx=0 tx=0).stop 2024-06-10T01:26:22.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.526+0000 7fc79bf2cb80 1 -- 172.21.15.7:0/3110148116 shutdown_connections 2024-06-10T01:26:22.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.526+0000 7fc79bf2cb80 1 --2- 172.21.15.7:0/3110148116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f67e9d880 0x556f67e923c0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.526+0000 7fc79bf2cb80 1 --2- 172.21.15.7:0/3110148116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f67e977a0 0x556f67e97260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.526+0000 7fc79bf2cb80 1 --2- 172.21.15.7:0/3110148116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f67e9cf60 0x556f67e91e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.526+0000 7fc79bf2cb80 1 -- 172.21.15.7:0/3110148116 >> 172.21.15.7:0/3110148116 conn(0x556f67dc4080 msgr2=0x556f67ea2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:22.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.526+0000 7fc79bf2cb80 1 -- 172.21.15.7:0/3110148116 shutdown_connections 2024-06-10T01:26:22.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.526+0000 7fc79bf2cb80 1 -- 172.21.15.7:0/3110148116 wait complete. 2024-06-10T01:26:22.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.526+0000 7fc79bf2cb80 1 -- start start 2024-06-10T01:26:22.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.526+0000 7fc79bf2cb80 1 Processor -- start 2024-06-10T01:26:22.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.527+0000 7fc79bf2cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f67e977a0 0x556f67eb3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.527+0000 7fc79bf2cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f67e9cf60 0x556f67eb4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.527+0000 7fc79bf2cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f67e9d880 0x556f67eb4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.527+0000 7fc78b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f67e9d880 0x556f67eb4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.527+0000 7fc78b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f67e9d880 0x556f67eb4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51232/0 (socket says 172.21.15.7:51232) 2024-06-10T01:26:22.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.527+0000 7fc78b7fe700 1 -- 172.21.15.7:0/1997966704 learned_addr learned my addr 172.21.15.7:0/1997966704 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:22.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.527+0000 7fc79bf2cb80 1 -- 172.21.15.7:0/1997966704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c1b2d60 con 0x556f67e9d880 2024-06-10T01:26:22.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.527+0000 7fc78affd700 1 --2- 172.21.15.7:0/1997966704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f67e977a0 0x556f67eb3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.527+0000 7fc79bf2cb80 1 -- 172.21.15.7:0/1997966704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c1b41d0 con 0x556f67e9cf60 2024-06-10T01:26:22.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.527+0000 7fc79bf2cb80 1 -- 172.21.15.7:0/1997966704 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c1b3430 con 0x556f67e977a0 2024-06-10T01:26:22.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.527+0000 7fc78a7fc700 1 --2- 172.21.15.7:0/1997966704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f67e9cf60 0x556f67eb4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.527+0000 7fc79bf2cb80 10 client.0 _ll_register_callbacks cb 0x556f67d7e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:26:22.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.527+0000 7fc78affd700 1 -- 172.21.15.7:0/1997966704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f67e9cf60 msgr2=0x556f67eb4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:22.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.527+0000 7fc78affd700 1 --2- 172.21.15.7:0/1997966704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f67e9cf60 0x556f67eb4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.527+0000 7fc78affd700 1 -- 172.21.15.7:0/1997966704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f67e9d880 msgr2=0x556f67eb4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:22.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.527+0000 7fc78affd700 1 --2- 172.21.15.7:0/1997966704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f67e9d880 0x556f67eb4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.527+0000 7fc78affd700 1 -- 172.21.15.7:0/1997966704 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f67f1a710 con 0x556f67e977a0 2024-06-10T01:26:22.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.527+0000 7fc78a7fc700 1 --2- 172.21.15.7:0/1997966704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f67e9cf60 0x556f67eb4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:26:22.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.527+0000 7fc78b7fe700 1 --2- 172.21.15.7:0/1997966704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f67e9d880 0x556f67eb4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:26:22.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.528+0000 7fc78affd700 1 --2- 172.21.15.7:0/1997966704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f67e977a0 0x556f67eb3f00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00bfd0 tx=0x7fc77c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdc1657cd0d91cf7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:22.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.528+0000 7fc76eff5700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:26:22.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.528+0000 7fc76eff5700 1 -- 172.21.15.7:0/1997966704 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c004220 con 0x556f67e977a0 2024-06-10T01:26:22.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.528+0000 7fc79bf2cb80 1 -- 172.21.15.7:0/1997966704 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc78401c410 con 0x556f67e977a0 2024-06-10T01:26:22.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.528+0000 7fc76eff5700 1 -- 172.21.15.7:0/1997966704 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fc77c0043c0 con 0x556f67e977a0 2024-06-10T01:26:22.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.528+0000 7fc76eff5700 1 -- 172.21.15.7:0/1997966704 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fc77c036050 con 0x556f67e977a0 2024-06-10T01:26:22.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.528+0000 7fc79bf2cb80 10 client.29494 fetch_fsmap learned FSMap version 52 2024-06-10T01:26:22.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.528+0000 7fc79bf2cb80 1 -- 172.21.15.7:0/1997966704 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x556f67f1b9a0 con 0x556f67e977a0 2024-06-10T01:26:22.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.529+0000 7fc76eff5700 1 -- 172.21.15.7:0/1997966704 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc78401c410 con 0x556f67e977a0 2024-06-10T01:26:22.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.529+0000 7fc76eff5700 1 -- 172.21.15.7:0/1997966704 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7fc77c01f840 con 0x556f67e977a0 2024-06-10T01:26:22.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.529+0000 7fc79bf2cb80 10 client.29494 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:26:22.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.529+0000 7fc79bf2cb80 10 client.29494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:26:22.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.529+0000 7fc79bf2cb80 20 client.29494 populate_metadata read hostname 'smithi007' 2024-06-10T01:26:22.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.529+0000 7fc79bf2cb80 1 --2- 172.21.15.7:0/1997966704 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556f67f1c080 0x556f67f1e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:22.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.529+0000 7fc79bf2cb80 4 client.29494 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:26:22.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.529+0000 7fc79bf2cb80 1 -- 172.21.15.7:0/1997966704 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556f67df02d0 con 0x556f67f1c080 2024-06-10T01:26:22.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.529+0000 7fc78a7fc700 1 --2- 172.21.15.7:0/1997966704 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556f67f1c080 0x556f67f1e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:22.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.530+0000 7fc78a7fc700 1 --2- 172.21.15.7:0/1997966704 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556f67f1c080 0x556f67f1e4a0 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b276ccc2614522 server_cookie=c2132f3eeabbfb20 in_seq=0 out_seq=0 2024-06-10T01:26:22.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.530+0000 7fc76eff5700 10 client.29494 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:22.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.530+0000 7fc76eff5700 1 -- 172.21.15.7:0/1997966704 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556f67df02d0 con 0x556f67f1c080 2024-06-10T01:26:22.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.530+0000 7fc76eff5700 10 client.29494 handle_command_reply: tid=0 2024-06-10T01:26:22.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.530+0000 7fc79bf2cb80 1 client.29494 shutdown 2024-06-10T01:26:22.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.530+0000 7fc79bf2cb80 1 -- 172.21.15.7:0/1997966704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f67e977a0 msgr2=0x556f67eb3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:22.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.530+0000 7fc79bf2cb80 1 --2- 172.21.15.7:0/1997966704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f67e977a0 0x556f67eb3f00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00bfd0 tx=0x7fc77c0047c0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.530+0000 7fc79bf2cb80 1 -- 172.21.15.7:0/1997966704 shutdown_connections 2024-06-10T01:26:22.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.530+0000 7fc79bf2cb80 1 --2- 172.21.15.7:0/1997966704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f67e9d880 0x556f67eb4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.530+0000 7fc79bf2cb80 1 --2- 172.21.15.7:0/1997966704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f67e977a0 0x556f67eb3f00 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.530+0000 7fc79bf2cb80 1 --2- 172.21.15.7:0/1997966704 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556f67f1c080 0x556f67f1e4a0 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.530+0000 7fc79bf2cb80 1 --2- 172.21.15.7:0/1997966704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f67e9cf60 0x556f67eb4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.530+0000 7fc76eff5700 0 client.29494 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:22.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.530+0000 7fc79bf2cb80 1 -- 172.21.15.7:0/1997966704 >> 172.21.15.7:0/1997966704 conn(0x556f67dc4080 msgr2=0x556f67ea0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:22.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.530+0000 7fc79bf2cb80 1 -- 172.21.15.7:0/1997966704 shutdown_connections 2024-06-10T01:26:22.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.530+0000 7fc79bf2cb80 1 -- 172.21.15.7:0/1997966704 wait complete. 2024-06-10T01:26:22.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.530+0000 7fc79bf2cb80 20 client.29494 trim_cache size 0 max 16384 2024-06-10T01:26:22.528 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:26:22.528 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T01:26:22.528 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-10T01:26:22.528 INFO:teuthology.orchestra.run.smithi007.stdout: "2b655214-48e0-4649-adf2-0be7b37161e8": { 2024-06-10T01:26:22.528 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-10T01:26:22.529 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "2b655214-48e0-4649-adf2-0be7b37161e8", 2024-06-10T01:26:22.529 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-10T01:26:22.529 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:26:22.529 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:26:22.529 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:26:22.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.531+0000 7fc7916a9700 1 -- 172.21.15.7:0/1808159946 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc7740a2ca0 msgr2=0x7fc7740a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:22.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.531+0000 7fc7916a9700 1 --2- 172.21.15.7:0/1808159946 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc7740a2ca0 0x7fc7740a5160 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc77c005950 tx=0x7fc77c00b500 comp rx=0 tx=0).stop 2024-06-10T01:26:22.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.531+0000 7fc7916a9700 1 -- 172.21.15.7:0/1808159946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc78c075020 msgr2=0x7fc78c19f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:22.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.531+0000 7fc7916a9700 1 --2- 172.21.15.7:0/1808159946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc78c075020 0x7fc78c19f740 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fc78000d750 tx=0x7fc78000da60 comp rx=0 tx=0).stop 2024-06-10T01:26:22.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.531+0000 7fc7916a9700 1 -- 172.21.15.7:0/1808159946 shutdown_connections 2024-06-10T01:26:22.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.531+0000 7fc7916a9700 1 --2- 172.21.15.7:0/1808159946 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc7740a2ca0 0x7fc7740a5160 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.531+0000 7fc7916a9700 1 --2- 172.21.15.7:0/1808159946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc78c113660 0x7fc78c1a3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.531+0000 7fc7916a9700 1 --2- 172.21.15.7:0/1808159946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc78c074700 0x7fc78c19f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.531+0000 7fc7916a9700 1 --2- 172.21.15.7:0/1808159946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc78c075020 0x7fc78c19f740 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:22.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.531+0000 7fc7916a9700 1 -- 172.21.15.7:0/1808159946 >> 172.21.15.7:0/1808159946 conn(0x7fc78c079680 msgr2=0x7fc78c10f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:22.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.531+0000 7fc7916a9700 1 -- 172.21.15.7:0/1808159946 shutdown_connections 2024-06-10T01:26:22.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:22.531+0000 7fc7916a9700 1 -- 172.21.15.7:0/1808159946 wait complete. 2024-06-10T01:26:22.545 INFO:tasks.workunit.client.0.smithi007.stdout: 10 170372 23.41 MB/sec execute 299 sec latency 16886.889 ms 2024-06-10T01:26:22.740 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:26:23.209 INFO:tasks.cephfs.filesystem:scrub status for tag:2b655214-48e0-4649-adf2-0be7b37161e8 - {'path': '/', 'tag': '2b655214-48e0-4649-adf2-0be7b37161e8', 'options': 'recursive,force'} 2024-06-10T01:26:23.503 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:23 smithi175 ceph-mon[36636]: pgmap v1044: 129 pgs: 129 active+clean; 649 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 8.3 MiB/s wr, 515 op/s 2024-06-10T01:26:23.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:23 smithi007 ceph-mon[32398]: pgmap v1044: 129 pgs: 129 active+clean; 649 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 8.3 MiB/s wr, 515 op/s 2024-06-10T01:26:23.546 INFO:tasks.workunit.client.0.smithi007.stdout: 10 170372 23.33 MB/sec execute 300 sec latency 17886.928 ms 2024-06-10T01:26:23.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:23 smithi078 ceph-mon[36683]: pgmap v1044: 129 pgs: 129 active+clean; 649 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 8.3 MiB/s wr, 515 op/s 2024-06-10T01:26:24.546 INFO:tasks.workunit.client.0.smithi007.stdout: 10 170372 23.25 MB/sec execute 301 sec latency 18887.033 ms 2024-06-10T01:26:24.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.840+0000 7f40769b5700 1 -- 172.21.15.7:0/3926493469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4070107400 msgr2=0x7f40701077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:24.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.840+0000 7f40769b5700 1 --2- 172.21.15.7:0/3926493469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4070107400 0x7f40701077e0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f4060009990 tx=0x7f4060009ca0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.840+0000 7f40769b5700 1 -- 172.21.15.7:0/3926493469 shutdown_connections 2024-06-10T01:26:24.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.840+0000 7f40769b5700 1 --2- 172.21.15.7:0/3926493469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4070101d50 0x7f407010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.840+0000 7f40769b5700 1 --2- 172.21.15.7:0/3926493469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4070107400 0x7f40701077e0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.840+0000 7f40769b5700 1 --2- 172.21.15.7:0/3926493469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4070101390 0x7f4070101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.841+0000 7f40769b5700 1 -- 172.21.15.7:0/3926493469 >> 172.21.15.7:0/3926493469 conn(0x7f4070078340 msgr2=0x7f4070078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:24.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.841+0000 7f40769b5700 1 -- 172.21.15.7:0/3926493469 shutdown_connections 2024-06-10T01:26:24.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.841+0000 7f40769b5700 1 -- 172.21.15.7:0/3926493469 wait complete. 2024-06-10T01:26:24.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.841+0000 7f40769b5700 1 Processor -- start 2024-06-10T01:26:24.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.841+0000 7f40769b5700 1 -- start start 2024-06-10T01:26:24.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.842+0000 7f40769b5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4070101390 0x7f4070103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:24.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.842+0000 7f40769b5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4070101d50 0x7f40701035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:24.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.842+0000 7f40769b5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4070107400 0x7f4070103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:24.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.842+0000 7f40769b5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f40701116a0 con 0x7f4070107400 2024-06-10T01:26:24.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.842+0000 7f40769b5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4070111520 con 0x7f4070101390 2024-06-10T01:26:24.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.842+0000 7f40769b5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4070111820 con 0x7f4070101d50 2024-06-10T01:26:24.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.842+0000 7f4074f52700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4070107400 0x7f4070103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:24.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.842+0000 7f4074f52700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4070107400 0x7f4070103af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51246/0 (socket says 172.21.15.7:51246) 2024-06-10T01:26:24.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.842+0000 7f4074f52700 1 -- 172.21.15.7:0/395964825 learned_addr learned my addr 172.21.15.7:0/395964825 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:24.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.842+0000 7f4067fff700 1 --2- 172.21.15.7:0/395964825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4070101d50 0x7f40701035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:24.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.842+0000 7f406ffff700 1 --2- 172.21.15.7:0/395964825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4070101390 0x7f4070103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:24.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.842+0000 7f4074f52700 1 -- 172.21.15.7:0/395964825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4070101d50 msgr2=0x7f40701035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:24.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.842+0000 7f4074f52700 1 --2- 172.21.15.7:0/395964825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4070101d50 0x7f40701035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.842+0000 7f4074f52700 1 -- 172.21.15.7:0/395964825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4070101390 msgr2=0x7f4070103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:24.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.842+0000 7f4074f52700 1 --2- 172.21.15.7:0/395964825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4070101390 0x7f4070103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.842+0000 7f4074f52700 1 -- 172.21.15.7:0/395964825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40701a7100 con 0x7f4070107400 2024-06-10T01:26:24.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.842+0000 7f4074f52700 1 --2- 172.21.15.7:0/395964825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4070107400 0x7f4070103af0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f4068007ee0 tx=0x7f406800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:24.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.842+0000 7f406dffb700 1 -- 172.21.15.7:0/395964825 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406800dbd0 con 0x7f4070107400 2024-06-10T01:26:24.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.842+0000 7f406dffb700 1 -- 172.21.15.7:0/395964825 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f406800dd70 con 0x7f4070107400 2024-06-10T01:26:24.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.842+0000 7f406dffb700 1 -- 172.21.15.7:0/395964825 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4068005a60 con 0x7f4070107400 2024-06-10T01:26:24.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.842+0000 7f40769b5700 1 -- 172.21.15.7:0/395964825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40701a73f0 con 0x7f4070107400 2024-06-10T01:26:24.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.843+0000 7f40769b5700 1 -- 172.21.15.7:0/395964825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40701a7980 con 0x7f4070107400 2024-06-10T01:26:24.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.845+0000 7f406dffb700 1 -- 172.21.15.7:0/395964825 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f406801a020 con 0x7f4070107400 2024-06-10T01:26:24.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.845+0000 7f406dffb700 1 --2- 172.21.15.7:0/395964825 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f405c0a2ca0 0x7f405c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:24.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.846+0000 7f406dffb700 1 -- 172.21.15.7:0/395964825 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f40680cea10 con 0x7f4070107400 2024-06-10T01:26:24.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.846+0000 7f406ffff700 1 --2- 172.21.15.7:0/395964825 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f405c0a2ca0 0x7f405c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:24.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.846+0000 7f406ffff700 1 --2- 172.21.15.7:0/395964825 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f405c0a2ca0 0x7f405c0a5160 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4060006010 tx=0x7f406000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:24.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.846+0000 7f4081238b80 1 Processor -- start 2024-06-10T01:26:24.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.846+0000 7f4081238b80 1 -- start start 2024-06-10T01:26:24.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.846+0000 7f4081238b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56266cc63c40 0x56266cc64020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:24.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.847+0000 7f4081238b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56266cc64560 0x56266cc5de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:24.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.847+0000 7f4081238b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56266cc5e3d0 0x56266cc5e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:24.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.847+0000 7f4081238b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56266cbaf910 con 0x56266cc63c40 2024-06-10T01:26:24.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.847+0000 7f4081238b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56266cb57820 con 0x56266cc5e3d0 2024-06-10T01:26:24.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.847+0000 7f4081238b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56266cbab8d0 con 0x56266cc64560 2024-06-10T01:26:24.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.847+0000 7f406ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56266cc63c40 0x56266cc64020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:24.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.847+0000 7f406ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56266cc63c40 0x56266cc64020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51248/0 (socket says 172.21.15.7:51248) 2024-06-10T01:26:24.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.847+0000 7f406ffff700 1 -- 172.21.15.7:0/2121182069 learned_addr learned my addr 172.21.15.7:0/2121182069 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:24.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.847+0000 7f4074f52700 1 --2- 172.21.15.7:0/2121182069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56266cc5e3d0 0x56266cc5e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:24.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.847+0000 7f4067fff700 1 --2- 172.21.15.7:0/2121182069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56266cc64560 0x56266cc5de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:24.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.847+0000 7f406ffff700 1 -- 172.21.15.7:0/2121182069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56266cc64560 msgr2=0x56266cc5de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:24.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.847+0000 7f406ffff700 1 --2- 172.21.15.7:0/2121182069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56266cc64560 0x56266cc5de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.847+0000 7f406ffff700 1 -- 172.21.15.7:0/2121182069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56266cc5e3d0 msgr2=0x56266cc5e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:24.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.847+0000 7f406ffff700 1 --2- 172.21.15.7:0/2121182069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56266cc5e3d0 0x56266cc5e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.847+0000 7f406ffff700 1 -- 172.21.15.7:0/2121182069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56266cb4e050 con 0x56266cc63c40 2024-06-10T01:26:24.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.847+0000 7f4067fff700 1 --2- 172.21.15.7:0/2121182069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56266cc64560 0x56266cc5de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:26:24.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.847+0000 7f406ffff700 1 --2- 172.21.15.7:0/2121182069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56266cc63c40 0x56266cc64020 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f406000ba60 tx=0x7f406001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7bcefb3cb4a3649 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:24.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.848+0000 7f40667fc700 1 -- 172.21.15.7:0/2121182069 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406000bea0 con 0x56266cc63c40 2024-06-10T01:26:24.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.848+0000 7f40667fc700 1 -- 172.21.15.7:0/2121182069 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4060003780 con 0x56266cc63c40 2024-06-10T01:26:24.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.848+0000 7f40667fc700 1 -- 172.21.15.7:0/2121182069 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4060024870 con 0x56266cc63c40 2024-06-10T01:26:24.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.848+0000 7f4081238b80 1 -- 172.21.15.7:0/2121182069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56266cc63c40 msgr2=0x56266cc64020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:24.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.848+0000 7f4081238b80 1 --2- 172.21.15.7:0/2121182069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56266cc63c40 0x56266cc64020 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f406000ba60 tx=0x7f406001f000 comp rx=0 tx=0).stop 2024-06-10T01:26:24.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.848+0000 7f4081238b80 1 -- 172.21.15.7:0/2121182069 shutdown_connections 2024-06-10T01:26:24.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.848+0000 7f4081238b80 1 --2- 172.21.15.7:0/2121182069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56266cc5e3d0 0x56266cc5e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.848+0000 7f4081238b80 1 --2- 172.21.15.7:0/2121182069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56266cc63c40 0x56266cc64020 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.848+0000 7f4081238b80 1 --2- 172.21.15.7:0/2121182069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56266cc64560 0x56266cc5de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.849+0000 7f4081238b80 1 -- 172.21.15.7:0/2121182069 >> 172.21.15.7:0/2121182069 conn(0x56266cb88dc0 msgr2=0x56266cc59120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:24.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.849+0000 7f4081238b80 1 -- 172.21.15.7:0/2121182069 shutdown_connections 2024-06-10T01:26:24.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.849+0000 7f4081238b80 1 -- 172.21.15.7:0/2121182069 wait complete. 2024-06-10T01:26:24.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.849+0000 7f4081238b80 1 -- start start 2024-06-10T01:26:24.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.849+0000 7f4081238b80 1 Processor -- start 2024-06-10T01:26:24.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.850+0000 7f4081238b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56266cc5e3d0 0x56266cc6a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:24.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.850+0000 7f4081238b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56266cc64560 0x56266cc676c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:24.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.850+0000 7f4081238b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56266cc67c00 0x56266cc68080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:24.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.850+0000 7f4081238b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56266cc6af10 con 0x56266cc67c00 2024-06-10T01:26:24.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.850+0000 7f4081238b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56266cb4fcf0 con 0x56266cc64560 2024-06-10T01:26:24.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.850+0000 7f4081238b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56266cc6b090 con 0x56266cc5e3d0 2024-06-10T01:26:24.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.850+0000 7f4074f52700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56266cc67c00 0x56266cc68080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:24.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.850+0000 7f4074f52700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56266cc67c00 0x56266cc68080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51262/0 (socket says 172.21.15.7:51262) 2024-06-10T01:26:24.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.850+0000 7f406ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56266cc5e3d0 0x56266cc6a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:24.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.850+0000 7f4074f52700 1 -- 172.21.15.7:0/2415959870 learned_addr learned my addr 172.21.15.7:0/2415959870 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:24.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.850+0000 7f4067fff700 1 --2- 172.21.15.7:0/2415959870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56266cc64560 0x56266cc676c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:24.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.850+0000 7f4081238b80 10 client.0 _ll_register_callbacks cb 0x56266cb49a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:26:24.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.850+0000 7f4074f52700 1 -- 172.21.15.7:0/2415959870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56266cc5e3d0 msgr2=0x56266cc6a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:24.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.850+0000 7f4074f52700 1 --2- 172.21.15.7:0/2415959870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56266cc5e3d0 0x56266cc6a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.850+0000 7f4074f52700 1 -- 172.21.15.7:0/2415959870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56266cc64560 msgr2=0x56266cc676c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:24.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.850+0000 7f4074f52700 1 --2- 172.21.15.7:0/2415959870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56266cc64560 0x56266cc676c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.850+0000 7f4074f52700 1 -- 172.21.15.7:0/2415959870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56266cafe9d0 con 0x56266cc67c00 2024-06-10T01:26:24.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.850+0000 7f406ffff700 1 --2- 172.21.15.7:0/2415959870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56266cc5e3d0 0x56266cc6a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:26:24.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.850+0000 7f4074f52700 1 --2- 172.21.15.7:0/2415959870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56266cc67c00 0x56266cc68080 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f40680cc190 tx=0x7f406809b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb4fdbdc029af583 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:24.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.850+0000 7f404f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:26:24.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.850+0000 7f404f7fe700 1 -- 172.21.15.7:0/2415959870 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40680a3b60 con 0x56266cc67c00 2024-06-10T01:26:24.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.851+0000 7f404f7fe700 1 -- 172.21.15.7:0/2415959870 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f4068022250 con 0x56266cc67c00 2024-06-10T01:26:24.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.851+0000 7f4081238b80 1 -- 172.21.15.7:0/2415959870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4060003780 con 0x56266cc67c00 2024-06-10T01:26:24.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.851+0000 7f404f7fe700 1 -- 172.21.15.7:0/2415959870 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f406807e050 con 0x56266cc67c00 2024-06-10T01:26:24.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.851+0000 7f4081238b80 10 client.20355 fetch_fsmap learned FSMap version 52 2024-06-10T01:26:24.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.851+0000 7f4081238b80 1 -- 172.21.15.7:0/2415959870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x56266cc75d50 con 0x56266cc67c00 2024-06-10T01:26:24.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.851+0000 7f404f7fe700 1 -- 172.21.15.7:0/2415959870 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4060003780 con 0x56266cc67c00 2024-06-10T01:26:24.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.851+0000 7f404f7fe700 1 -- 172.21.15.7:0/2415959870 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f40680a37d0 con 0x56266cc67c00 2024-06-10T01:26:24.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.851+0000 7f4081238b80 10 client.20355 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:26:24.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.851+0000 7f4081238b80 10 client.20355 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:26:24.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.851+0000 7f4081238b80 20 client.20355 populate_metadata read hostname 'smithi007' 2024-06-10T01:26:24.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.851+0000 7f4081238b80 1 --2- 172.21.15.7:0/2415959870 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56266cc78390 0x56266cc7a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:24.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.851+0000 7f4081238b80 4 client.20355 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:26:24.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.851+0000 7f4081238b80 1 -- 172.21.15.7:0/2415959870 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56266cbbce10 con 0x56266cc78390 2024-06-10T01:26:24.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.852+0000 7f406ffff700 1 --2- 172.21.15.7:0/2415959870 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56266cc78390 0x56266cc7a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:24.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.852+0000 7f406ffff700 1 --2- 172.21.15.7:0/2415959870 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56266cc78390 0x56266cc7a7f0 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e56d8a895f92a91 server_cookie=91e9cb26d08c0f7d in_seq=0 out_seq=0 2024-06-10T01:26:24.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.852+0000 7f404f7fe700 10 client.20355 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:24.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.853+0000 7f404f7fe700 1 -- 172.21.15.7:0/2415959870 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56266cbbce10 con 0x56266cc78390 2024-06-10T01:26:24.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.853+0000 7f404f7fe700 10 client.20355 handle_command_reply: tid=0 2024-06-10T01:26:24.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.853+0000 7f4081238b80 1 client.20355 shutdown 2024-06-10T01:26:24.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.853+0000 7f4081238b80 1 -- 172.21.15.7:0/2415959870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56266cc67c00 msgr2=0x56266cc68080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:24.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.853+0000 7f4081238b80 1 --2- 172.21.15.7:0/2415959870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56266cc67c00 0x56266cc68080 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f40680cc190 tx=0x7f406809b6e0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.853+0000 7f4081238b80 1 -- 172.21.15.7:0/2415959870 shutdown_connections 2024-06-10T01:26:24.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.853+0000 7f4081238b80 1 --2- 172.21.15.7:0/2415959870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56266cc67c00 0x56266cc68080 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.853+0000 7f4081238b80 1 --2- 172.21.15.7:0/2415959870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56266cc5e3d0 0x56266cc6a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.853+0000 7f4081238b80 1 --2- 172.21.15.7:0/2415959870 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56266cc78390 0x56266cc7a7f0 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.853+0000 7f4081238b80 1 --2- 172.21.15.7:0/2415959870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56266cc64560 0x56266cc676c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.853+0000 7f404f7fe700 0 client.20355 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:24.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.853+0000 7f4081238b80 1 -- 172.21.15.7:0/2415959870 >> 172.21.15.7:0/2415959870 conn(0x56266cb88dc0 msgr2=0x56266cc5ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:24.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.853+0000 7f4081238b80 1 -- 172.21.15.7:0/2415959870 shutdown_connections 2024-06-10T01:26:24.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.853+0000 7f4081238b80 1 -- 172.21.15.7:0/2415959870 wait complete. 2024-06-10T01:26:24.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.853+0000 7f4081238b80 20 client.20355 trim_cache size 0 max 16384 2024-06-10T01:26:24.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.874+0000 7f4081238b80 1 Processor -- start 2024-06-10T01:26:24.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.874+0000 7f4081238b80 1 -- start start 2024-06-10T01:26:24.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.874+0000 7f4081238b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56266cc5e3d0 0x56266cc61dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:24.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.874+0000 7f4081238b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56266cc64560 0x56266cc800c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:24.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.874+0000 7f4081238b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56266cc67c00 0x56266cc80600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:24.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.874+0000 7f4081238b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f40701b2e00 con 0x56266cc64560 2024-06-10T01:26:24.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.875+0000 7f4081238b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f40701b0690 con 0x56266cc67c00 2024-06-10T01:26:24.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.875+0000 7f4081238b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f40701ae4e0 con 0x56266cc5e3d0 2024-06-10T01:26:24.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.875+0000 7f4067fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56266cc64560 0x56266cc800c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:24.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.875+0000 7f4067fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56266cc64560 0x56266cc800c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51266/0 (socket says 172.21.15.7:51266) 2024-06-10T01:26:24.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.875+0000 7f4067fff700 1 -- 172.21.15.7:0/1486881895 learned_addr learned my addr 172.21.15.7:0/1486881895 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:24.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.875+0000 7f406ffff700 1 --2- 172.21.15.7:0/1486881895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56266cc5e3d0 0x56266cc61dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:24.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.875+0000 7f4074f52700 1 --2- 172.21.15.7:0/1486881895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56266cc67c00 0x56266cc80600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:24.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.875+0000 7f4067fff700 1 -- 172.21.15.7:0/1486881895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56266cc5e3d0 msgr2=0x56266cc61dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:24.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.875+0000 7f4067fff700 1 --2- 172.21.15.7:0/1486881895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56266cc5e3d0 0x56266cc61dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.875+0000 7f4067fff700 1 -- 172.21.15.7:0/1486881895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56266cc67c00 msgr2=0x56266cc80600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:24.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.875+0000 7f4067fff700 1 --2- 172.21.15.7:0/1486881895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56266cc67c00 0x56266cc80600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.875+0000 7f4067fff700 1 -- 172.21.15.7:0/1486881895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56266cc65c30 con 0x56266cc64560 2024-06-10T01:26:24.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.875+0000 7f406ffff700 1 --2- 172.21.15.7:0/1486881895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56266cc5e3d0 0x56266cc61dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:26:24.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.875+0000 7f4074f52700 1 --2- 172.21.15.7:0/1486881895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56266cc67c00 0x56266cc80600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:26:24.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.875+0000 7f4067fff700 1 --2- 172.21.15.7:0/1486881895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56266cc64560 0x56266cc800c0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f405800ad20 tx=0x7f4058012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8a8849cdca9987 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:24.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.875+0000 7f4065ffb700 1 -- 172.21.15.7:0/1486881895 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4058004020 con 0x56266cc64560 2024-06-10T01:26:24.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.875+0000 7f4065ffb700 1 -- 172.21.15.7:0/1486881895 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4058004830 con 0x56266cc64560 2024-06-10T01:26:24.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.875+0000 7f4065ffb700 1 -- 172.21.15.7:0/1486881895 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4058005470 con 0x56266cc64560 2024-06-10T01:26:24.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.876+0000 7f4081238b80 1 -- 172.21.15.7:0/1486881895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56266cc64560 msgr2=0x56266cc800c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:24.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.876+0000 7f4081238b80 1 --2- 172.21.15.7:0/1486881895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56266cc64560 0x56266cc800c0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f405800ad20 tx=0x7f4058012730 comp rx=0 tx=0).stop 2024-06-10T01:26:24.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.876+0000 7f4081238b80 1 -- 172.21.15.7:0/1486881895 shutdown_connections 2024-06-10T01:26:24.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.876+0000 7f4081238b80 1 --2- 172.21.15.7:0/1486881895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56266cc67c00 0x56266cc80600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.876+0000 7f4081238b80 1 --2- 172.21.15.7:0/1486881895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56266cc5e3d0 0x56266cc61dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.876+0000 7f4081238b80 1 --2- 172.21.15.7:0/1486881895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56266cc64560 0x56266cc800c0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.876+0000 7f4081238b80 1 -- 172.21.15.7:0/1486881895 >> 172.21.15.7:0/1486881895 conn(0x56266cb88dc0 msgr2=0x56266cc616c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:24.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.876+0000 7f4081238b80 1 -- 172.21.15.7:0/1486881895 shutdown_connections 2024-06-10T01:26:24.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.876+0000 7f4081238b80 1 -- 172.21.15.7:0/1486881895 wait complete. 2024-06-10T01:26:24.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.876+0000 7f4081238b80 1 -- start start 2024-06-10T01:26:24.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.877+0000 7f4081238b80 1 Processor -- start 2024-06-10T01:26:24.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.877+0000 7f4081238b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56266cc5e3d0 0x56266cc70c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:24.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.877+0000 7f4081238b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56266cc64560 0x56266cc7fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:24.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.877+0000 7f4081238b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56266cc67c00 0x56266cc80170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:24.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.877+0000 7f4081238b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f40701aeb50 con 0x56266cc5e3d0 2024-06-10T01:26:24.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.877+0000 7f4081238b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f40701affc0 con 0x56266cc67c00 2024-06-10T01:26:24.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.877+0000 7f4081238b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f40701af220 con 0x56266cc64560 2024-06-10T01:26:24.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.877+0000 7f4081238b80 10 client.0 _ll_register_callbacks cb 0x56266cc74ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:26:24.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.877+0000 7f406ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56266cc5e3d0 0x56266cc70c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:24.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.877+0000 7f406ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56266cc5e3d0 0x56266cc70c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51278/0 (socket says 172.21.15.7:51278) 2024-06-10T01:26:24.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.877+0000 7f406ffff700 1 -- 172.21.15.7:0/1726157303 learned_addr learned my addr 172.21.15.7:0/1726157303 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:24.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.877+0000 7f4067fff700 1 --2- 172.21.15.7:0/1726157303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56266cc64560 0x56266cc7fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:24.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.877+0000 7f4074f52700 1 --2- 172.21.15.7:0/1726157303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56266cc67c00 0x56266cc80170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:24.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.877+0000 7f406ffff700 1 -- 172.21.15.7:0/1726157303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56266cc64560 msgr2=0x56266cc7fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:24.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.877+0000 7f406ffff700 1 --2- 172.21.15.7:0/1726157303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56266cc64560 0x56266cc7fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.877+0000 7f406ffff700 1 -- 172.21.15.7:0/1726157303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56266cc67c00 msgr2=0x56266cc80170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:24.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.877+0000 7f406ffff700 1 --2- 172.21.15.7:0/1726157303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56266cc67c00 0x56266cc80170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.877+0000 7f406ffff700 1 -- 172.21.15.7:0/1726157303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56266cce2090 con 0x56266cc5e3d0 2024-06-10T01:26:24.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.877+0000 7f4074f52700 1 --2- 172.21.15.7:0/1726157303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56266cc67c00 0x56266cc80170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:26:24.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.878+0000 7f406ffff700 1 --2- 172.21.15.7:0/1726157303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56266cc5e3d0 0x56266cc70c00 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f4060031040 tx=0x7f4060005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b72f2000110b447 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:24.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.878+0000 7f404f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:26:24.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.878+0000 7f404f7fe700 1 -- 172.21.15.7:0/1726157303 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406003d070 con 0x56266cc5e3d0 2024-06-10T01:26:24.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.878+0000 7f404f7fe700 1 -- 172.21.15.7:0/1726157303 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f4060035040 con 0x56266cc5e3d0 2024-06-10T01:26:24.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.878+0000 7f4081238b80 1 -- 172.21.15.7:0/1726157303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4058004830 con 0x56266cc5e3d0 2024-06-10T01:26:24.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.878+0000 7f404f7fe700 1 -- 172.21.15.7:0/1726157303 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f406004a050 con 0x56266cc5e3d0 2024-06-10T01:26:24.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.878+0000 7f4081238b80 10 client.20361 fetch_fsmap learned FSMap version 52 2024-06-10T01:26:24.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.878+0000 7f4081238b80 1 -- 172.21.15.7:0/1726157303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x56266cce31a0 con 0x56266cc5e3d0 2024-06-10T01:26:24.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.878+0000 7f404f7fe700 1 -- 172.21.15.7:0/1726157303 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4058004830 con 0x56266cc5e3d0 2024-06-10T01:26:24.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.878+0000 7f404f7fe700 1 -- 172.21.15.7:0/1726157303 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f4060024b50 con 0x56266cc5e3d0 2024-06-10T01:26:24.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.878+0000 7f4081238b80 10 client.20361 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:26:24.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.878+0000 7f4081238b80 10 client.20361 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:26:24.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.878+0000 7f4081238b80 20 client.20361 populate_metadata read hostname 'smithi007' 2024-06-10T01:26:24.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.878+0000 7f4081238b80 1 --2- 172.21.15.7:0/1726157303 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56266cce3430 0x56266cce5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:24.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.878+0000 7f4081238b80 4 client.20361 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d3f8e8a1", "release": true, "await": true}] 2024-06-10T01:26:24.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.878+0000 7f4081238b80 1 -- 172.21.15.7:0/1726157303 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d3f8e8a1", "release": true, "await": true}) -- 0x56266cbb7390 con 0x56266cce3430 2024-06-10T01:26:24.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.879+0000 7f4067fff700 1 --2- 172.21.15.7:0/1726157303 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56266cce3430 0x56266cce5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:24.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.879+0000 7f4067fff700 1 --2- 172.21.15.7:0/1726157303 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56266cce3430 0x56266cce5820 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f0da83577076810 server_cookie=9d4ffeac68745045 in_seq=0 out_seq=0 2024-06-10T01:26:24.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.879+0000 7f404f7fe700 10 client.20361 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.932+0000 7f404f7fe700 1 -- 172.21.15.7:0/1726157303 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56266cbb7390 con 0x56266cce3430 2024-06-10T01:26:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.932+0000 7f404f7fe700 10 client.20361 handle_command_reply: tid=0 2024-06-10T01:26:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.932+0000 7f4081238b80 1 client.20361 shutdown 2024-06-10T01:26:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.932+0000 7f4081238b80 1 -- 172.21.15.7:0/1726157303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56266cc5e3d0 msgr2=0x56266cc70c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.932+0000 7f4081238b80 1 --2- 172.21.15.7:0/1726157303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56266cc5e3d0 0x56266cc70c00 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f4060031040 tx=0x7f4060005a50 comp rx=0 tx=0).stop 2024-06-10T01:26:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.932+0000 7f4081238b80 1 -- 172.21.15.7:0/1726157303 shutdown_connections 2024-06-10T01:26:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.932+0000 7f4081238b80 1 --2- 172.21.15.7:0/1726157303 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56266cce3430 0x56266cce5820 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.932+0000 7f4081238b80 1 --2- 172.21.15.7:0/1726157303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56266cc67c00 0x56266cc80170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.932+0000 7f4081238b80 1 --2- 172.21.15.7:0/1726157303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56266cc5e3d0 0x56266cc70c00 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.932+0000 7f4081238b80 1 --2- 172.21.15.7:0/1726157303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56266cc64560 0x56266cc7fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.932+0000 7f404f7fe700 0 client.20361 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.932+0000 7f4081238b80 1 -- 172.21.15.7:0/1726157303 >> 172.21.15.7:0/1726157303 conn(0x56266cb88dc0 msgr2=0x56266cc65fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.932+0000 7f4081238b80 1 -- 172.21.15.7:0/1726157303 shutdown_connections 2024-06-10T01:26:24.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.932+0000 7f4081238b80 1 -- 172.21.15.7:0/1726157303 wait complete. 2024-06-10T01:26:24.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.933+0000 7f4081238b80 20 client.20361 trim_cache size 0 max 16384 2024-06-10T01:26:24.927 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:26:24.927 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 52, 2024-06-10T01:26:24.927 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:26:24.927 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 64, 2024-06-10T01:26:24.927 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:26:24.927 INFO:teuthology.orchestra.run.smithi007.stdout: "d3f8e8a1": { 2024-06-10T01:26:24.927 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 64, 2024-06-10T01:26:24.928 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:26:24.928 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:26:24.928 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:26:24.928 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:26:24.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:26:24.928 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:26:24.928 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 142.0, 2024-06-10T01:26:24.928 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:26:24.928 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:26:24.928 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:26:24.928 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:26:24.928 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:26:24.928 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:26:24.928 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:26:24.929 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:26:24.929 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:26:24.929 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:26:24.929 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:26:24.929 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:26:24.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.933+0000 7f40769b5700 1 -- 172.21.15.7:0/395964825 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f405c0a2ca0 msgr2=0x7f405c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:24.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.933+0000 7f40769b5700 1 --2- 172.21.15.7:0/395964825 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f405c0a2ca0 0x7f405c0a5160 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4060006010 tx=0x7f406000b540 comp rx=0 tx=0).stop 2024-06-10T01:26:24.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.933+0000 7f40769b5700 1 -- 172.21.15.7:0/395964825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4070107400 msgr2=0x7f4070103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:24.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.933+0000 7f40769b5700 1 --2- 172.21.15.7:0/395964825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4070107400 0x7f4070103af0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f4068007ee0 tx=0x7f406800d3b0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.933+0000 7f40769b5700 1 -- 172.21.15.7:0/395964825 shutdown_connections 2024-06-10T01:26:24.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.933+0000 7f40769b5700 1 --2- 172.21.15.7:0/395964825 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f405c0a2ca0 0x7f405c0a5160 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.933+0000 7f40769b5700 1 --2- 172.21.15.7:0/395964825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4070107400 0x7f4070103af0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.933+0000 7f40769b5700 1 --2- 172.21.15.7:0/395964825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4070101390 0x7f4070103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.933+0000 7f40769b5700 1 --2- 172.21.15.7:0/395964825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4070101d50 0x7f40701035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:24.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.933+0000 7f40769b5700 1 -- 172.21.15.7:0/395964825 >> 172.21.15.7:0/395964825 conn(0x7f4070078340 msgr2=0x7f40701005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:24.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.933+0000 7f40769b5700 1 -- 172.21.15.7:0/395964825 shutdown_connections 2024-06-10T01:26:24.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:24.934+0000 7f40769b5700 1 -- 172.21.15.7:0/395964825 wait complete. 2024-06-10T01:26:25.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:25 smithi007 ceph-mon[32398]: pgmap v1045: 129 pgs: 129 active+clean; 649 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 4.8 MiB/s wr, 368 op/s 2024-06-10T01:26:25.546 INFO:tasks.workunit.client.0.smithi007.stdout: 10 170821 23.23 MB/sec execute 302 sec latency 19266.302 ms 2024-06-10T01:26:25.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:25 smithi078 ceph-mon[36683]: pgmap v1045: 129 pgs: 129 active+clean; 649 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 4.8 MiB/s wr, 368 op/s 2024-06-10T01:26:25.627 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd3f8e8a1', total seconds elapsed: 31.6 2024-06-10T01:26:25.627 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.0 2024-06-10T01:26:25.627 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:26:25.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:25 smithi175 ceph-mon[36636]: pgmap v1045: 129 pgs: 129 active+clean; 649 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 4.8 MiB/s wr, 368 op/s 2024-06-10T01:26:25.943 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:26:26.546 INFO:tasks.workunit.client.0.smithi007.stdout: 10 171469 23.26 MB/sec execute 303 sec latency 87.360 ms 2024-06-10T01:26:27.430 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:27 smithi007 ceph-mon[32398]: pgmap v1046: 129 pgs: 129 active+clean; 649 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 3.9 MiB/s wr, 353 op/s 2024-06-10T01:26:27.546 INFO:tasks.workunit.client.0.smithi007.stdout: 10 172090 23.31 MB/sec execute 304 sec latency 85.324 ms 2024-06-10T01:26:27.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:27 smithi078 ceph-mon[36683]: pgmap v1046: 129 pgs: 129 active+clean; 649 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 3.9 MiB/s wr, 353 op/s 2024-06-10T01:26:27.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:27 smithi175 ceph-mon[36636]: pgmap v1046: 129 pgs: 129 active+clean; 649 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 3.9 MiB/s wr, 353 op/s 2024-06-10T01:26:27.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.771+0000 7fdf13557700 1 -- 172.21.15.7:0/1525521370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf0c101d40 msgr2=0x7fdf0c10e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:27.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.771+0000 7fdf13557700 1 --2- 172.21.15.7:0/1525521370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf0c101d40 0x7fdf0c10e230 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400b0a0 tx=0x7fdf0400b3b0 comp rx=0 tx=0).stop 2024-06-10T01:26:27.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.771+0000 7fdf13557700 1 -- 172.21.15.7:0/1525521370 shutdown_connections 2024-06-10T01:26:27.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.771+0000 7fdf13557700 1 --2- 172.21.15.7:0/1525521370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf0c101d40 0x7fdf0c10e230 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:27.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.771+0000 7fdf13557700 1 --2- 172.21.15.7:0/1525521370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf0c1073f0 0x7fdf0c1077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:27.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.771+0000 7fdf13557700 1 --2- 172.21.15.7:0/1525521370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf0c101380 0x7fdf0c101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:27.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.771+0000 7fdf13557700 1 -- 172.21.15.7:0/1525521370 >> 172.21.15.7:0/1525521370 conn(0x7fdf0c078300 msgr2=0x7fdf0c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:27.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.771+0000 7fdf13557700 1 -- 172.21.15.7:0/1525521370 shutdown_connections 2024-06-10T01:26:27.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.771+0000 7fdf13557700 1 -- 172.21.15.7:0/1525521370 wait complete. 2024-06-10T01:26:27.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.772+0000 7fdf13557700 1 Processor -- start 2024-06-10T01:26:27.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.772+0000 7fdf13557700 1 -- start start 2024-06-10T01:26:27.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.772+0000 7fdf13557700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf0c101380 0x7fdf0c197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:27.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.772+0000 7fdf13557700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf0c101d40 0x7fdf0c197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:27.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.772+0000 7fdf13557700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf0c1073f0 0x7fdf0c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:27.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.772+0000 7fdf13557700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c1116b0 con 0x7fdf0c101d40 2024-06-10T01:26:27.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.772+0000 7fdf13557700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c111530 con 0x7fdf0c1073f0 2024-06-10T01:26:27.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.772+0000 7fdf13557700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c111830 con 0x7fdf0c101380 2024-06-10T01:26:27.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.773+0000 7fdf11af4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf0c1073f0 0x7fdf0c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:27.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.773+0000 7fdf11af4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf0c1073f0 0x7fdf0c1983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:39840/0 (socket says 172.21.15.7:39840) 2024-06-10T01:26:27.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.773+0000 7fdf11af4700 1 -- 172.21.15.7:0/1585283761 learned_addr learned my addr 172.21.15.7:0/1585283761 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:27.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.773+0000 7fdf10af2700 1 --2- 172.21.15.7:0/1585283761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf0c101d40 0x7fdf0c197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:27.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.773+0000 7fdf112f3700 1 --2- 172.21.15.7:0/1585283761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf0c101380 0x7fdf0c197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:27.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.773+0000 7fdf11af4700 1 -- 172.21.15.7:0/1585283761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf0c101380 msgr2=0x7fdf0c197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:27.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.773+0000 7fdf11af4700 1 --2- 172.21.15.7:0/1585283761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf0c101380 0x7fdf0c197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:27.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.773+0000 7fdf11af4700 1 -- 172.21.15.7:0/1585283761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf0c101d40 msgr2=0x7fdf0c197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:27.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.773+0000 7fdf11af4700 1 --2- 172.21.15.7:0/1585283761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf0c101d40 0x7fdf0c197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:27.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.773+0000 7fdf11af4700 1 -- 172.21.15.7:0/1585283761 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf0c1a71b0 con 0x7fdf0c1073f0 2024-06-10T01:26:27.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.773+0000 7fdf11af4700 1 --2- 172.21.15.7:0/1585283761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf0c1073f0 0x7fdf0c1983c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400b070 tx=0x7fdf04004c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:27.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.773+0000 7fdf027fc700 1 -- 172.21.15.7:0/1585283761 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04021070 con 0x7fdf0c1073f0 2024-06-10T01:26:27.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.774+0000 7fdf027fc700 1 -- 172.21.15.7:0/1585283761 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf04003c00 con 0x7fdf0c1073f0 2024-06-10T01:26:27.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.774+0000 7fdf027fc700 1 -- 172.21.15.7:0/1585283761 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04004740 con 0x7fdf0c1073f0 2024-06-10T01:26:27.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.774+0000 7fdf13557700 1 -- 172.21.15.7:0/1585283761 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf0c1a7440 con 0x7fdf0c1073f0 2024-06-10T01:26:27.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.774+0000 7fdf13557700 1 -- 172.21.15.7:0/1585283761 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf0c1a78a0 con 0x7fdf0c1073f0 2024-06-10T01:26:27.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.775+0000 7fdf13557700 1 -- 172.21.15.7:0/1585283761 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf0c069180 con 0x7fdf0c1073f0 2024-06-10T01:26:27.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.776+0000 7fdf027fc700 1 -- 172.21.15.7:0/1585283761 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fdf04007980 con 0x7fdf0c1073f0 2024-06-10T01:26:27.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.777+0000 7fdf027fc700 1 --2- 172.21.15.7:0/1585283761 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdef80a2ca0 0x7fdef80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:27.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.777+0000 7fdf112f3700 1 --2- 172.21.15.7:0/1585283761 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdef80a2ca0 0x7fdef80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:27.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.778+0000 7fdf027fc700 1 -- 172.21.15.7:0/1585283761 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fdf0400f020 con 0x7fdf0c1073f0 2024-06-10T01:26:27.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.778+0000 7fdf112f3700 1 --2- 172.21.15.7:0/1585283761 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdef80a2ca0 0x7fdef80a5160 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fdf08009680 tx=0x7fdf08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:27.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:27.782+0000 7fdf027fc700 1 -- 172.21.15.7:0/1585283761 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdf0402a020 con 0x7fdf0c1073f0 2024-06-10T01:26:28.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:28.198+0000 7fdf13557700 1 -- 172.21.15.7:0/1585283761 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf0c02dca0 con 0x7fdf0c1073f0 2024-06-10T01:26:28.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:28.201+0000 7fdf027fc700 1 -- 172.21.15.7:0/1585283761 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11858 (secure 0 0 0) 0x7fdf0409d2a0 con 0x7fdf0c1073f0 2024-06-10T01:26:28.194 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:26:28.194 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":52,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:25:03.837724+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28825":{"gid":28825,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/1972426673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1972426673},{"type":"v1","addr":"172.21.15.175:6835","nonce":1972426673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24451,24434,24436]},"id":1}]} 2024-06-10T01:26:28.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:28.204+0000 7fdf13557700 1 -- 172.21.15.7:0/1585283761 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdef80a2ca0 msgr2=0x7fdef80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:28.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:28.204+0000 7fdf13557700 1 --2- 172.21.15.7:0/1585283761 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdef80a2ca0 0x7fdef80a5160 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fdf08009680 tx=0x7fdf08009260 comp rx=0 tx=0).stop 2024-06-10T01:26:28.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:28.205+0000 7fdf13557700 1 -- 172.21.15.7:0/1585283761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf0c1073f0 msgr2=0x7fdf0c1983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:28.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:28.205+0000 7fdf13557700 1 --2- 172.21.15.7:0/1585283761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf0c1073f0 0x7fdf0c1983c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400b070 tx=0x7fdf04004c60 comp rx=0 tx=0).stop 2024-06-10T01:26:28.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:28.205+0000 7fdf13557700 1 -- 172.21.15.7:0/1585283761 shutdown_connections 2024-06-10T01:26:28.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:28.205+0000 7fdf13557700 1 --2- 172.21.15.7:0/1585283761 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdef80a2ca0 0x7fdef80a5160 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:28.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:28.205+0000 7fdf13557700 1 --2- 172.21.15.7:0/1585283761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf0c1073f0 0x7fdf0c1983c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:28.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:28.205+0000 7fdf13557700 1 --2- 172.21.15.7:0/1585283761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf0c101380 0x7fdf0c197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:28.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:28.205+0000 7fdf13557700 1 --2- 172.21.15.7:0/1585283761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf0c101d40 0x7fdf0c197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:28.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:28.205+0000 7fdf13557700 1 -- 172.21.15.7:0/1585283761 >> 172.21.15.7:0/1585283761 conn(0x7fdf0c078300 msgr2=0x7fdf0c0ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:28.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:28.206+0000 7fdf13557700 1 -- 172.21.15.7:0/1585283761 shutdown_connections 2024-06-10T01:26:28.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:28.206+0000 7fdf13557700 1 -- 172.21.15.7:0/1585283761 wait complete. 2024-06-10T01:26:28.201 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 52 2024-06-10T01:26:28.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:28 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:26:28.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:28 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:26:28.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:28 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:26:28.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:28 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1585283761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:26:28.546 INFO:tasks.workunit.client.0.smithi007.stdout: 10 172657 23.34 MB/sec execute 305 sec latency 79.155 ms 2024-06-10T01:26:28.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:28 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:26:28.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:28 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:26:28.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:28 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:26:28.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:28 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1585283761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:26:28.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:28 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:26:28.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:28 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:26:28.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:28 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:26:28.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:28 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1585283761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:26:28.925 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db / --timeout 90 --expiration 142.0 --await-for 45.0' 2024-06-10T01:26:28.925 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db / --timeout 90 --expiration 142.0 --await-for 45.0 2024-06-10T01:26:29.277 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:26:29.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:29 smithi007 ceph-mon[32398]: pgmap v1047: 129 pgs: 129 active+clean; 652 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 218 op/s 2024-06-10T01:26:29.546 INFO:tasks.workunit.client.0.smithi007.stdout: 10 173312 23.37 MB/sec execute 306 sec latency 52.597 ms 2024-06-10T01:26:29.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:29 smithi078 ceph-mon[36683]: pgmap v1047: 129 pgs: 129 active+clean; 652 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 218 op/s 2024-06-10T01:26:29.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:29 smithi175 ceph-mon[36636]: pgmap v1047: 129 pgs: 129 active+clean; 652 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 218 op/s 2024-06-10T01:26:30.546 INFO:tasks.workunit.client.0.smithi007.stdout: 10 174048 23.44 MB/sec execute 307 sec latency 60.478 ms 2024-06-10T01:26:31.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.237+0000 7f212479f700 1 -- 172.21.15.7:0/3195307965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f211c1066c0 msgr2=0x7f211c10e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:31.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.237+0000 7f212479f700 1 --2- 172.21.15.7:0/3195307965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f211c1066c0 0x7f211c10e290 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f21140099c0 tx=0x7f2114009cd0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.237+0000 7f212479f700 1 -- 172.21.15.7:0/3195307965 shutdown_connections 2024-06-10T01:26:31.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.237+0000 7f212479f700 1 --2- 172.21.15.7:0/3195307965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f211c1066c0 0x7f211c10e290 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.237+0000 7f212479f700 1 --2- 172.21.15.7:0/3195307965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f211c1015f0 0x7f211c1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.237+0000 7f212479f700 1 --2- 172.21.15.7:0/3195307965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f211c101fa0 0x7f211c105fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.237+0000 7f212479f700 1 -- 172.21.15.7:0/3195307965 >> 172.21.15.7:0/3195307965 conn(0x7f211c078330 msgr2=0x7f211c078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:31.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.237+0000 7f212479f700 1 -- 172.21.15.7:0/3195307965 shutdown_connections 2024-06-10T01:26:31.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.237+0000 7f212479f700 1 -- 172.21.15.7:0/3195307965 wait complete. 2024-06-10T01:26:31.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.238+0000 7f212479f700 1 Processor -- start 2024-06-10T01:26:31.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.238+0000 7f212479f700 1 -- start start 2024-06-10T01:26:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.238+0000 7f212479f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f211c1015f0 0x7f211c197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.238+0000 7f212479f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f211c101fa0 0x7f211c197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.238+0000 7f212479f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f211c1066c0 0x7f211c19c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.238+0000 7f212479f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f211c1116d0 con 0x7f211c1015f0 2024-06-10T01:26:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.238+0000 7f212479f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f211c111550 con 0x7f211c101fa0 2024-06-10T01:26:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.238+0000 7f212479f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f211c111850 con 0x7f211c1066c0 2024-06-10T01:26:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.239+0000 7f212253b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f211c1015f0 0x7f211c197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.239+0000 7f2121d3a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f211c101fa0 0x7f211c197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.239+0000 7f212253b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f211c1015f0 0x7f211c197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51322/0 (socket says 172.21.15.7:51322) 2024-06-10T01:26:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.239+0000 7f2121d3a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f211c101fa0 0x7f211c197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:39856/0 (socket says 172.21.15.7:39856) 2024-06-10T01:26:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.239+0000 7f212253b700 1 -- 172.21.15.7:0/1226649862 learned_addr learned my addr 172.21.15.7:0/1226649862 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.239+0000 7f2122d3c700 1 --2- 172.21.15.7:0/1226649862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f211c1066c0 0x7f211c19c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.239+0000 7f2121d3a700 1 -- 172.21.15.7:0/1226649862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f211c1066c0 msgr2=0x7f211c19c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.239+0000 7f2121d3a700 1 --2- 172.21.15.7:0/1226649862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f211c1066c0 0x7f211c19c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.239+0000 7f2121d3a700 1 -- 172.21.15.7:0/1226649862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f211c1015f0 msgr2=0x7f211c197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:31.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.239+0000 7f2121d3a700 1 --2- 172.21.15.7:0/1226649862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f211c1015f0 0x7f211c197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.239+0000 7f2121d3a700 1 -- 172.21.15.7:0/1226649862 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f211c19cc90 con 0x7f211c101fa0 2024-06-10T01:26:31.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.239+0000 7f212253b700 1 --2- 172.21.15.7:0/1226649862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f211c1015f0 0x7f211c197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:26:31.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.239+0000 7f2121d3a700 1 --2- 172.21.15.7:0/1226649862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f211c101fa0 0x7f211c197c40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f210c00e9c0 tx=0x7f210c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:31.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.239+0000 7f21137fe700 1 -- 172.21.15.7:0/1226649862 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c00cbd0 con 0x7f211c101fa0 2024-06-10T01:26:31.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.239+0000 7f212479f700 1 -- 172.21.15.7:0/1226649862 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f211c19cf80 con 0x7f211c101fa0 2024-06-10T01:26:31.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.239+0000 7f21137fe700 1 -- 172.21.15.7:0/1226649862 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f210c004500 con 0x7f211c101fa0 2024-06-10T01:26:31.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.239+0000 7f21137fe700 1 -- 172.21.15.7:0/1226649862 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c005040 con 0x7f211c101fa0 2024-06-10T01:26:31.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.239+0000 7f212479f700 1 -- 172.21.15.7:0/1226649862 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f211c19d470 con 0x7f211c101fa0 2024-06-10T01:26:31.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.242+0000 7f21137fe700 1 -- 172.21.15.7:0/1226649862 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f210c005300 con 0x7f211c101fa0 2024-06-10T01:26:31.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.243+0000 7f21137fe700 1 --2- 172.21.15.7:0/1226649862 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21080a2cc0 0x7f21080a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:31.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.243+0000 7f212253b700 1 --2- 172.21.15.7:0/1226649862 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21080a2cc0 0x7f21080a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:31.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.243+0000 7f21137fe700 1 -- 172.21.15.7:0/1226649862 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f210c0cf2c0 con 0x7f211c101fa0 2024-06-10T01:26:31.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.244+0000 7f212253b700 1 --2- 172.21.15.7:0/1226649862 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21080a2cc0 0x7f21080a5180 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2118009c10 tx=0x7f211800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:31.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.245+0000 7f212f022b80 1 Processor -- start 2024-06-10T01:26:31.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.245+0000 7f212f022b80 1 -- start start 2024-06-10T01:26:31.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.245+0000 7f212f022b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c8533590 0x5593c8533970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:31.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.245+0000 7f212f022b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593c8533eb0 0x5593c852d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:31.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.245+0000 7f212f022b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5593c852dce0 0x5593c852e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:31.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.245+0000 7f212f022b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5593c84200f0 con 0x5593c8533590 2024-06-10T01:26:31.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.245+0000 7f212253b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c8533590 0x5593c8533970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:31.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.245+0000 7f212f022b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5593c847e9e0 con 0x5593c8533eb0 2024-06-10T01:26:31.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.245+0000 7f212f022b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5593c83e65f0 con 0x5593c852dce0 2024-06-10T01:26:31.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.245+0000 7f2121d3a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593c8533eb0 0x5593c852d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:31.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.245+0000 7f2121d3a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593c8533eb0 0x5593c852d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:39870/0 (socket says 172.21.15.7:39870) 2024-06-10T01:26:31.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.245+0000 7f2122d3c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5593c852dce0 0x5593c852e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:31.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.245+0000 7f2121d3a700 1 -- 172.21.15.7:0/3259137876 learned_addr learned my addr 172.21.15.7:0/3259137876 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:31.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.245+0000 7f212253b700 1 --2- 172.21.15.7:0/3259137876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c8533590 0x5593c8533970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51324/0 (socket says 172.21.15.7:51324) 2024-06-10T01:26:31.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.246+0000 7f2122d3c700 1 -- 172.21.15.7:0/3259137876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593c8533eb0 msgr2=0x5593c852d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:31.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.246+0000 7f2122d3c700 1 --2- 172.21.15.7:0/3259137876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593c8533eb0 0x5593c852d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.246+0000 7f2122d3c700 1 -- 172.21.15.7:0/3259137876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c8533590 msgr2=0x5593c8533970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:31.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.246+0000 7f2122d3c700 1 --2- 172.21.15.7:0/3259137876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c8533590 0x5593c8533970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.246+0000 7f2122d3c700 1 -- 172.21.15.7:0/3259137876 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593c83cda60 con 0x5593c852dce0 2024-06-10T01:26:31.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.246+0000 7f2121d3a700 1 --2- 172.21.15.7:0/3259137876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593c8533eb0 0x5593c852d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:26:31.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.246+0000 7f2122d3c700 1 --2- 172.21.15.7:0/3259137876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5593c852dce0 0x5593c852e100 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f211400b580 tx=0x7f2114005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa3f889ed94aadca server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:31.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.246+0000 7f2110ff9700 1 -- 172.21.15.7:0/3259137876 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2114027070 con 0x5593c852dce0 2024-06-10T01:26:31.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.246+0000 7f2110ff9700 1 -- 172.21.15.7:0/3259137876 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21140058e0 con 0x5593c852dce0 2024-06-10T01:26:31.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.246+0000 7f2110ff9700 1 -- 172.21.15.7:0/3259137876 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211400b790 con 0x5593c852dce0 2024-06-10T01:26:31.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.247+0000 7f212f022b80 1 -- 172.21.15.7:0/3259137876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5593c852dce0 msgr2=0x5593c852e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:31.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.247+0000 7f212f022b80 1 --2- 172.21.15.7:0/3259137876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5593c852dce0 0x5593c852e100 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f211400b580 tx=0x7f2114005c00 comp rx=0 tx=0).stop 2024-06-10T01:26:31.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.247+0000 7f212f022b80 1 -- 172.21.15.7:0/3259137876 shutdown_connections 2024-06-10T01:26:31.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.247+0000 7f212f022b80 1 --2- 172.21.15.7:0/3259137876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5593c852dce0 0x5593c852e100 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.247+0000 7f212f022b80 1 --2- 172.21.15.7:0/3259137876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c8533590 0x5593c8533970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.247+0000 7f212f022b80 1 --2- 172.21.15.7:0/3259137876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593c8533eb0 0x5593c852d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.247+0000 7f212f022b80 1 -- 172.21.15.7:0/3259137876 >> 172.21.15.7:0/3259137876 conn(0x5593c845a210 msgr2=0x5593c8528a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:31.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.247+0000 7f212f022b80 1 -- 172.21.15.7:0/3259137876 shutdown_connections 2024-06-10T01:26:31.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.247+0000 7f212f022b80 1 -- 172.21.15.7:0/3259137876 wait complete. 2024-06-10T01:26:31.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.248+0000 7f212f022b80 1 -- start start 2024-06-10T01:26:31.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.248+0000 7f212f022b80 1 Processor -- start 2024-06-10T01:26:31.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.248+0000 7f212f022b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5593c852dce0 0x5593c8539bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:31.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.248+0000 7f212f022b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c8533590 0x5593c8536f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:31.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.248+0000 7f212f022b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593c8533eb0 0x5593c8537450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:31.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.248+0000 7f212f022b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5593c841f150 con 0x5593c8533590 2024-06-10T01:26:31.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.248+0000 7f212253b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5593c852dce0 0x5593c8539bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:31.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.249+0000 7f212253b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5593c852dce0 0x5593c8539bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:56496/0 (socket says 172.21.15.7:56496) 2024-06-10T01:26:31.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.249+0000 7f212253b700 1 -- 172.21.15.7:0/1463392684 learned_addr learned my addr 172.21.15.7:0/1463392684 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:31.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.248+0000 7f2121d3a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c8533590 0x5593c8536f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:31.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.249+0000 7f212f022b80 1 -- 172.21.15.7:0/1463392684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5593c847a9a0 con 0x5593c8533eb0 2024-06-10T01:26:31.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.249+0000 7f212f022b80 1 -- 172.21.15.7:0/1463392684 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5593c853a830 con 0x5593c852dce0 2024-06-10T01:26:31.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.249+0000 7f2122d3c700 1 --2- 172.21.15.7:0/1463392684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593c8533eb0 0x5593c8537450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:31.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.249+0000 7f212f022b80 10 client.0 _ll_register_callbacks cb 0x5593c8414740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:26:31.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.249+0000 7f2122d3c700 1 -- 172.21.15.7:0/1463392684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5593c852dce0 msgr2=0x5593c8539bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:31.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.249+0000 7f2122d3c700 1 --2- 172.21.15.7:0/1463392684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5593c852dce0 0x5593c8539bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.249+0000 7f2122d3c700 1 -- 172.21.15.7:0/1463392684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c8533590 msgr2=0x5593c8536f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:31.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.249+0000 7f2122d3c700 1 --2- 172.21.15.7:0/1463392684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c8533590 0x5593c8536f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.249+0000 7f2122d3c700 1 -- 172.21.15.7:0/1463392684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593c84448e0 con 0x5593c8533eb0 2024-06-10T01:26:31.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.249+0000 7f2122d3c700 1 --2- 172.21.15.7:0/1463392684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593c8533eb0 0x5593c8537450 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f21140037c0 tx=0x7f21140056c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84a52219d8313b79 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:31.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.249+0000 7f2105ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:26:31.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.249+0000 7f2105ffb700 1 -- 172.21.15.7:0/1463392684 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2114031060 con 0x5593c8533eb0 2024-06-10T01:26:31.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.250+0000 7f2105ffb700 1 -- 172.21.15.7:0/1463392684 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f2114003a60 con 0x5593c8533eb0 2024-06-10T01:26:31.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.250+0000 7f212f022b80 1 -- 172.21.15.7:0/1463392684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21140058e0 con 0x5593c8533eb0 2024-06-10T01:26:31.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.250+0000 7f2105ffb700 1 -- 172.21.15.7:0/1463392684 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f2114038050 con 0x5593c8533eb0 2024-06-10T01:26:31.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.250+0000 7f212f022b80 10 client.29594 fetch_fsmap learned FSMap version 52 2024-06-10T01:26:31.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.250+0000 7f212f022b80 1 -- 172.21.15.7:0/1463392684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5593c8544f50 con 0x5593c8533eb0 2024-06-10T01:26:31.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.250+0000 7f2105ffb700 1 -- 172.21.15.7:0/1463392684 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21140058e0 con 0x5593c8533eb0 2024-06-10T01:26:31.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.250+0000 7f2105ffb700 1 -- 172.21.15.7:0/1463392684 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f2114022480 con 0x5593c8533eb0 2024-06-10T01:26:31.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.251+0000 7f212f022b80 10 client.29594 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:26:31.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.251+0000 7f212f022b80 10 client.29594 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:26:31.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.251+0000 7f212f022b80 20 client.29594 populate_metadata read hostname 'smithi007' 2024-06-10T01:26:31.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.251+0000 7f212f022b80 1 --2- 172.21.15.7:0/1463392684 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5593c8547420 0x5593c8549810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:31.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.251+0000 7f212f022b80 4 client.29594 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:26:31.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.251+0000 7f212f022b80 1 -- 172.21.15.7:0/1463392684 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593c848bee0 con 0x5593c8547420 2024-06-10T01:26:31.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.251+0000 7f212253b700 1 --2- 172.21.15.7:0/1463392684 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5593c8547420 0x5593c8549810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:31.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.251+0000 7f212253b700 1 --2- 172.21.15.7:0/1463392684 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5593c8547420 0x5593c8549810 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b58abd7b7913bf9b server_cookie=252ebed4ab2a905d in_seq=0 out_seq=0 2024-06-10T01:26:31.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.251+0000 7f2105ffb700 10 client.29594 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:31.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.254+0000 7f2105ffb700 1 -- 172.21.15.7:0/1463392684 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5593c848bee0 con 0x5593c8547420 2024-06-10T01:26:31.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.254+0000 7f2105ffb700 10 client.29594 handle_command_reply: tid=0 2024-06-10T01:26:31.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.254+0000 7f212f022b80 1 client.29594 shutdown 2024-06-10T01:26:31.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.255+0000 7f212f022b80 1 -- 172.21.15.7:0/1463392684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593c8533eb0 msgr2=0x5593c8537450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:31.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.255+0000 7f212f022b80 1 --2- 172.21.15.7:0/1463392684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593c8533eb0 0x5593c8537450 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f21140037c0 tx=0x7f21140056c0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.255+0000 7f212f022b80 1 -- 172.21.15.7:0/1463392684 shutdown_connections 2024-06-10T01:26:31.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.255+0000 7f212f022b80 1 --2- 172.21.15.7:0/1463392684 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5593c8547420 0x5593c8549810 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.255+0000 7f212f022b80 1 --2- 172.21.15.7:0/1463392684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593c8533eb0 0x5593c8537450 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.255+0000 7f212f022b80 1 --2- 172.21.15.7:0/1463392684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5593c852dce0 0x5593c8539bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.255+0000 7f212f022b80 1 --2- 172.21.15.7:0/1463392684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c8533590 0x5593c8536f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.255+0000 7f212f022b80 1 -- 172.21.15.7:0/1463392684 >> 172.21.15.7:0/1463392684 conn(0x5593c845a210 msgr2=0x5593c8527c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:31.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.255+0000 7f2105ffb700 0 client.29594 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:31.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.255+0000 7f212f022b80 1 -- 172.21.15.7:0/1463392684 shutdown_connections 2024-06-10T01:26:31.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.255+0000 7f212f022b80 1 -- 172.21.15.7:0/1463392684 wait complete. 2024-06-10T01:26:31.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.255+0000 7f212f022b80 20 client.29594 trim_cache size 0 max 16384 2024-06-10T01:26:31.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.279+0000 7f212f022b80 1 Processor -- start 2024-06-10T01:26:31.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.279+0000 7f212f022b80 1 -- start start 2024-06-10T01:26:31.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.279+0000 7f212f022b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5593c852dce0 0x5593c852d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:31.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.279+0000 7f212f022b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c8533590 0x5593c85283c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:31.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.280+0000 7f212f022b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593c8533eb0 0x5593c8528900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:31.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.280+0000 7f212f022b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f211c1b6f50 con 0x5593c8533590 2024-06-10T01:26:31.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.280+0000 7f212f022b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f211c1b47e0 con 0x5593c8533eb0 2024-06-10T01:26:31.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.280+0000 7f212f022b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f211c1b2630 con 0x5593c852dce0 2024-06-10T01:26:31.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.280+0000 7f2121d3a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c8533590 0x5593c85283c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:31.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.280+0000 7f2121d3a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c8533590 0x5593c85283c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51338/0 (socket says 172.21.15.7:51338) 2024-06-10T01:26:31.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.280+0000 7f2121d3a700 1 -- 172.21.15.7:0/4204864284 learned_addr learned my addr 172.21.15.7:0/4204864284 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:31.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.280+0000 7f212253b700 1 --2- 172.21.15.7:0/4204864284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5593c852dce0 0x5593c852d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:31.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.280+0000 7f2122d3c700 1 --2- 172.21.15.7:0/4204864284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593c8533eb0 0x5593c8528900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:31.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.280+0000 7f2121d3a700 1 -- 172.21.15.7:0/4204864284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5593c852dce0 msgr2=0x5593c852d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:31.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.280+0000 7f2121d3a700 1 --2- 172.21.15.7:0/4204864284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5593c852dce0 0x5593c852d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.280+0000 7f2121d3a700 1 -- 172.21.15.7:0/4204864284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593c8533eb0 msgr2=0x5593c8528900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:31.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.280+0000 7f2121d3a700 1 --2- 172.21.15.7:0/4204864284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593c8533eb0 0x5593c8528900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.280+0000 7f2121d3a700 1 -- 172.21.15.7:0/4204864284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593c8533970 con 0x5593c8533590 2024-06-10T01:26:31.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.280+0000 7f2122d3c700 1 --2- 172.21.15.7:0/4204864284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593c8533eb0 0x5593c8528900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:26:31.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.280+0000 7f2121d3a700 1 --2- 172.21.15.7:0/4204864284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c8533590 0x5593c85283c0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f210c0ccdb0 tx=0x7f210c086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7538f09605d573 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:31.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.280+0000 7f2107fff700 1 -- 172.21.15.7:0/4204864284 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c01c030 con 0x5593c8533590 2024-06-10T01:26:31.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.280+0000 7f2107fff700 1 -- 172.21.15.7:0/4204864284 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f210c09bea0 con 0x5593c8533590 2024-06-10T01:26:31.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.280+0000 7f2107fff700 1 -- 172.21.15.7:0/4204864284 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c099aa0 con 0x5593c8533590 2024-06-10T01:26:31.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.281+0000 7f212f022b80 1 -- 172.21.15.7:0/4204864284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c8533590 msgr2=0x5593c85283c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:31.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.281+0000 7f212f022b80 1 --2- 172.21.15.7:0/4204864284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c8533590 0x5593c85283c0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f210c0ccdb0 tx=0x7f210c086040 comp rx=0 tx=0).stop 2024-06-10T01:26:31.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.281+0000 7f212f022b80 1 -- 172.21.15.7:0/4204864284 shutdown_connections 2024-06-10T01:26:31.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.281+0000 7f212f022b80 1 --2- 172.21.15.7:0/4204864284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593c8533eb0 0x5593c8528900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.281+0000 7f212f022b80 1 --2- 172.21.15.7:0/4204864284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5593c852dce0 0x5593c852d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.281+0000 7f212f022b80 1 --2- 172.21.15.7:0/4204864284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c8533590 0x5593c85283c0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.281+0000 7f212f022b80 1 -- 172.21.15.7:0/4204864284 >> 172.21.15.7:0/4204864284 conn(0x5593c845a210 msgr2=0x5593c8538f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:31.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.281+0000 7f212f022b80 1 -- 172.21.15.7:0/4204864284 shutdown_connections 2024-06-10T01:26:31.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.281+0000 7f212f022b80 1 -- 172.21.15.7:0/4204864284 wait complete. 2024-06-10T01:26:31.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.282+0000 7f212f022b80 1 -- start start 2024-06-10T01:26:31.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.282+0000 7f212f022b80 1 Processor -- start 2024-06-10T01:26:31.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.282+0000 7f212f022b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5593c852dce0 0x5593c854b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:31.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.282+0000 7f212f022b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593c8533590 0x5593c854bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:31.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.282+0000 7f212f022b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c8533eb0 0x5593c85b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:31.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.282+0000 7f212f022b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f211c1b2ca0 con 0x5593c8533eb0 2024-06-10T01:26:31.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.282+0000 7f212f022b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f211c1b4110 con 0x5593c8533590 2024-06-10T01:26:31.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.282+0000 7f212f022b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f211c1b3370 con 0x5593c852dce0 2024-06-10T01:26:31.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.282+0000 7f212f022b80 10 client.0 _ll_register_callbacks cb 0x5593c8414740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:26:31.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.282+0000 7f2122d3c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c8533eb0 0x5593c85b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:31.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.282+0000 7f2121d3a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593c8533590 0x5593c854bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:31.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.282+0000 7f212253b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5593c852dce0 0x5593c854b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:31.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.282+0000 7f2122d3c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c8533eb0 0x5593c85b1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51352/0 (socket says 172.21.15.7:51352) 2024-06-10T01:26:31.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.282+0000 7f2122d3c700 1 -- 172.21.15.7:0/2262084388 learned_addr learned my addr 172.21.15.7:0/2262084388 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:31.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.282+0000 7f2122d3c700 1 -- 172.21.15.7:0/2262084388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5593c852dce0 msgr2=0x5593c854b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:31.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.282+0000 7f2122d3c700 1 --2- 172.21.15.7:0/2262084388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5593c852dce0 0x5593c854b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.282+0000 7f2122d3c700 1 -- 172.21.15.7:0/2262084388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593c8533590 msgr2=0x5593c854bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:31.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.282+0000 7f2122d3c700 1 --2- 172.21.15.7:0/2262084388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593c8533590 0x5593c854bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.282+0000 7f2122d3c700 1 -- 172.21.15.7:0/2262084388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593c85b22b0 con 0x5593c8533eb0 2024-06-10T01:26:31.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.283+0000 7f212253b700 1 --2- 172.21.15.7:0/2262084388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5593c852dce0 0x5593c854b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:26:31.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.283+0000 7f2122d3c700 1 --2- 172.21.15.7:0/2262084388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c8533eb0 0x5593c85b1d70 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f21140229f0 tx=0x7f2114003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d35fb8f119105 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:31.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.283+0000 7f2105ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:26:31.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.283+0000 7f2105ffb700 1 -- 172.21.15.7:0/2262084388 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2114031040 con 0x5593c8533eb0 2024-06-10T01:26:31.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.283+0000 7f2105ffb700 1 -- 172.21.15.7:0/2262084388 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f2114003860 con 0x5593c8533eb0 2024-06-10T01:26:31.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.283+0000 7f212f022b80 1 -- 172.21.15.7:0/2262084388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f210c09bea0 con 0x5593c8533eb0 2024-06-10T01:26:31.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.283+0000 7f2105ffb700 1 -- 172.21.15.7:0/2262084388 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f2114027070 con 0x5593c8533eb0 2024-06-10T01:26:31.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.283+0000 7f212f022b80 10 client.20385 fetch_fsmap learned FSMap version 52 2024-06-10T01:26:31.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.283+0000 7f212f022b80 1 -- 172.21.15.7:0/2262084388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5593c85b33f0 con 0x5593c8533eb0 2024-06-10T01:26:31.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.284+0000 7f2105ffb700 1 -- 172.21.15.7:0/2262084388 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f210c09bea0 con 0x5593c8533eb0 2024-06-10T01:26:31.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.284+0000 7f2105ffb700 1 -- 172.21.15.7:0/2262084388 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f211400ba30 con 0x5593c8533eb0 2024-06-10T01:26:31.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.284+0000 7f212f022b80 10 client.20385 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:26:31.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.284+0000 7f212f022b80 10 client.20385 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:26:31.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.284+0000 7f212f022b80 20 client.20385 populate_metadata read hostname 'smithi007' 2024-06-10T01:26:31.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.284+0000 7f212f022b80 1 --2- 172.21.15.7:0/2262084388 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5593c85b38c0 0x5593c85b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:31.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.284+0000 7f212f022b80 4 client.20385 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.0, "await_for": 45.0}] 2024-06-10T01:26:31.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.284+0000 7f212f022b80 1 -- 172.21.15.7:0/2262084388 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.0, "await_for": 45.0}) -- 0x5593c8486460 con 0x5593c85b38c0 2024-06-10T01:26:31.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.284+0000 7f212253b700 1 --2- 172.21.15.7:0/2262084388 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5593c85b38c0 0x5593c85b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:31.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.285+0000 7f2105ffb700 10 client.20385 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:31.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.285+0000 7f212253b700 1 --2- 172.21.15.7:0/2262084388 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5593c85b38c0 0x5593c85b5cb0 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c72a0998152b05b server_cookie=dcd44d79790704a0 in_seq=0 out_seq=0 2024-06-10T01:26:31.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:31 smithi007 ceph-mon[32398]: pgmap v1048: 129 pgs: 129 active+clean; 653 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 995 KiB/s wr, 225 op/s 2024-06-10T01:26:31.546 INFO:tasks.workunit.client.0.smithi007.stdout: 10 174552 23.45 MB/sec execute 308 sec latency 275.487 ms 2024-06-10T01:26:31.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:31 smithi078 ceph-mon[36683]: pgmap v1048: 129 pgs: 129 active+clean; 653 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 995 KiB/s wr, 225 op/s 2024-06-10T01:26:31.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:31 smithi175 ceph-mon[36636]: pgmap v1048: 129 pgs: 129 active+clean; 653 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 995 KiB/s wr, 225 op/s 2024-06-10T01:26:31.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.976+0000 7f2105ffb700 1 -- 172.21.15.7:0/2262084388 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5593c8486460 con 0x5593c85b38c0 2024-06-10T01:26:31.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.976+0000 7f2105ffb700 10 client.20385 handle_command_reply: tid=0 2024-06-10T01:26:31.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.976+0000 7f212f022b80 1 client.20385 shutdown 2024-06-10T01:26:31.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.976+0000 7f212f022b80 1 -- 172.21.15.7:0/2262084388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c8533eb0 msgr2=0x5593c85b1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:31.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.976+0000 7f212f022b80 1 --2- 172.21.15.7:0/2262084388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c8533eb0 0x5593c85b1d70 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f21140229f0 tx=0x7f2114003680 comp rx=0 tx=0).stop 2024-06-10T01:26:31.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.976+0000 7f212f022b80 1 -- 172.21.15.7:0/2262084388 shutdown_connections 2024-06-10T01:26:31.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.976+0000 7f212f022b80 1 --2- 172.21.15.7:0/2262084388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c8533eb0 0x5593c85b1d70 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.976+0000 7f212f022b80 1 --2- 172.21.15.7:0/2262084388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5593c852dce0 0x5593c854b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.976+0000 7f212f022b80 1 --2- 172.21.15.7:0/2262084388 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5593c85b38c0 0x5593c85b5cb0 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.976+0000 7f212f022b80 1 --2- 172.21.15.7:0/2262084388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593c8533590 0x5593c854bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.976+0000 7f2105ffb700 0 client.20385 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:31.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.976+0000 7f212f022b80 1 -- 172.21.15.7:0/2262084388 >> 172.21.15.7:0/2262084388 conn(0x5593c845a210 msgr2=0x5593c8531f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:31.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.976+0000 7f212f022b80 1 -- 172.21.15.7:0/2262084388 shutdown_connections 2024-06-10T01:26:31.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.976+0000 7f212f022b80 1 -- 172.21.15.7:0/2262084388 wait complete. 2024-06-10T01:26:31.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.976+0000 7f212f022b80 20 client.20385 trim_cache size 0 max 16384 2024-06-10T01:26:31.971 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:26:31.971 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 52, 2024-06-10T01:26:31.971 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:26:31.971 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 66, 2024-06-10T01:26:31.971 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:26:31.971 INFO:teuthology.orchestra.run.smithi007.stdout: "2b2d6bd9": { 2024-06-10T01:26:31.971 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 66, 2024-06-10T01:26:31.971 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:26:31.971 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:26:31.971 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:26:31.972 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:26:31.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:26:31.972 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:26:31.972 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 142.0, 2024-06-10T01:26:31.972 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:26:31.972 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:26:31.972 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:26:31.972 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:26:31.972 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:26:31.972 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:26:31.973 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:26:31.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:26:31.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:26:31.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:26:31.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:26:31.974 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:26:31.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.977+0000 7f212479f700 1 -- 172.21.15.7:0/1226649862 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21080a2cc0 msgr2=0x7f21080a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:31.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.977+0000 7f212479f700 1 --2- 172.21.15.7:0/1226649862 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21080a2cc0 0x7f21080a5180 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2118009c10 tx=0x7f211800b040 comp rx=0 tx=0).stop 2024-06-10T01:26:31.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.977+0000 7f212479f700 1 -- 172.21.15.7:0/1226649862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f211c101fa0 msgr2=0x7f211c197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:31.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.977+0000 7f212479f700 1 --2- 172.21.15.7:0/1226649862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f211c101fa0 0x7f211c197c40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f210c00e9c0 tx=0x7f210c00ecd0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.977+0000 7f212479f700 1 -- 172.21.15.7:0/1226649862 shutdown_connections 2024-06-10T01:26:31.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.977+0000 7f212479f700 1 --2- 172.21.15.7:0/1226649862 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21080a2cc0 0x7f21080a5180 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.977+0000 7f212479f700 1 --2- 172.21.15.7:0/1226649862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f211c1066c0 0x7f211c19c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.977+0000 7f212479f700 1 --2- 172.21.15.7:0/1226649862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f211c1015f0 0x7f211c197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.977+0000 7f212479f700 1 --2- 172.21.15.7:0/1226649862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f211c101fa0 0x7f211c197c40 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:31.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.977+0000 7f212479f700 1 -- 172.21.15.7:0/1226649862 >> 172.21.15.7:0/1226649862 conn(0x7f211c078330 msgr2=0x7f211c0ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:31.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.977+0000 7f212479f700 1 -- 172.21.15.7:0/1226649862 shutdown_connections 2024-06-10T01:26:31.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:31.977+0000 7f212479f700 1 -- 172.21.15.7:0/1226649862 wait complete. 2024-06-10T01:26:32.546 INFO:tasks.workunit.client.0.smithi007.stdout: 10 174555 23.38 MB/sec execute 309 sec latency 1275.572 ms 2024-06-10T01:26:32.615 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2b2d6bd9', quiesce took 7.0 seconds. Will release after: 15.0 2024-06-10T01:26:33.546 INFO:tasks.workunit.client.0.smithi007.stdout: 10 174555 23.30 MB/sec execute 310 sec latency 2275.678 ms 2024-06-10T01:26:33.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:33 smithi078 ceph-mon[36683]: pgmap v1049: 129 pgs: 129 active+clean; 655 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1.7 MiB/s wr, 283 op/s 2024-06-10T01:26:33.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:33 smithi175 ceph-mon[36636]: pgmap v1049: 129 pgs: 129 active+clean; 655 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1.7 MiB/s wr, 283 op/s 2024-06-10T01:26:33.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:33 smithi007 ceph-mon[32398]: pgmap v1049: 129 pgs: 129 active+clean; 655 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1.7 MiB/s wr, 283 op/s 2024-06-10T01:26:34.546 INFO:tasks.workunit.client.0.smithi007.stdout: 10 174555 23.23 MB/sec execute 311 sec latency 3275.781 ms 2024-06-10T01:26:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.305+0000 7f049f85d640 1 -- 172.21.15.7:0/2483006518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0498101a50 msgr2=0x7f0498101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.305+0000 7f049f85d640 1 --2- 172.21.15.7:0/2483006518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0498101a50 0x7f0498101e70 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f0494009900 tx=0x7f049402a410 comp rx=0 tx=0).stop 2024-06-10T01:26:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.305+0000 7f049f85d640 1 -- 172.21.15.7:0/2483006518 shutdown_connections 2024-06-10T01:26:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.305+0000 7f049f85d640 1 --2- 172.21.15.7:0/2483006518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0498103600 0x7f0498109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.305+0000 7f049f85d640 1 --2- 172.21.15.7:0/2483006518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0498102c40 0x7f04981030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.305+0000 7f049f85d640 1 --2- 172.21.15.7:0/2483006518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0498101a50 0x7f0498101e70 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.305+0000 7f049f85d640 1 -- 172.21.15.7:0/2483006518 >> 172.21.15.7:0/2483006518 conn(0x7f04980fcfc0 msgr2=0x7f04980ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.305+0000 7f049f85d640 1 -- 172.21.15.7:0/2483006518 shutdown_connections 2024-06-10T01:26:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.305+0000 7f049f85d640 1 -- 172.21.15.7:0/2483006518 wait complete. 2024-06-10T01:26:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.306+0000 7f049f85d640 1 Processor -- start 2024-06-10T01:26:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.306+0000 7f049f85d640 1 -- start start 2024-06-10T01:26:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.306+0000 7f049f85d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0498101a50 0x7f0498197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.306+0000 7f049f85d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0498102c40 0x7f0498198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.306+0000 7f049f85d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0498103600 0x7f049819f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.306+0000 7f049f85d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f049810d6d0 con 0x7f0498101a50 2024-06-10T01:26:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.306+0000 7f049f85d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f049810d550 con 0x7f0498103600 2024-06-10T01:26:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.306+0000 7f049f85d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f049810d850 con 0x7f0498102c40 2024-06-10T01:26:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.306+0000 7f049d5f9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0498101a50 0x7f0498197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.306+0000 7f049d5f9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0498101a50 0x7f0498197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34756/0 (socket says 172.21.15.7:34756) 2024-06-10T01:26:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.306+0000 7f049d5f9640 1 -- 172.21.15.7:0/1624615008 learned_addr learned my addr 172.21.15.7:0/1624615008 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.306+0000 7f049ddfa640 1 --2- 172.21.15.7:0/1624615008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0498103600 0x7f049819f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.306+0000 7f049cdf8640 1 --2- 172.21.15.7:0/1624615008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0498102c40 0x7f0498198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.307+0000 7f049d5f9640 1 -- 172.21.15.7:0/1624615008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0498102c40 msgr2=0x7f0498198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.307+0000 7f049d5f9640 1 --2- 172.21.15.7:0/1624615008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0498102c40 0x7f0498198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.307+0000 7f049d5f9640 1 -- 172.21.15.7:0/1624615008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0498103600 msgr2=0x7f049819f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.307+0000 7f049d5f9640 1 --2- 172.21.15.7:0/1624615008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0498103600 0x7f049819f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.307+0000 7f049d5f9640 1 -- 172.21.15.7:0/1624615008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f049819f960 con 0x7f0498101a50 2024-06-10T01:26:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.307+0000 7f049d5f9640 1 --2- 172.21.15.7:0/1624615008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0498101a50 0x7f0498197d40 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f0494031610 tx=0x7f0494004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.307+0000 7f048e7fc640 1 -- 172.21.15.7:0/1624615008 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049403f030 con 0x7f0498101a50 2024-06-10T01:26:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.307+0000 7f048e7fc640 1 -- 172.21.15.7:0/1624615008 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0494003ad0 con 0x7f0498101a50 2024-06-10T01:26:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.307+0000 7f049f85d640 1 -- 172.21.15.7:0/1624615008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f049819fbf0 con 0x7f0498101a50 2024-06-10T01:26:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.307+0000 7f048e7fc640 1 -- 172.21.15.7:0/1624615008 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0494035070 con 0x7f0498101a50 2024-06-10T01:26:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.307+0000 7f049f85d640 1 -- 172.21.15.7:0/1624615008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04981a0130 con 0x7f0498101a50 2024-06-10T01:26:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.310+0000 7f048e7fc640 1 -- 172.21.15.7:0/1624615008 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0494042030 con 0x7f0498101a50 2024-06-10T01:26:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.310+0000 7f04a256e740 1 Processor -- start 2024-06-10T01:26:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.310+0000 7f04a256e740 1 -- start start 2024-06-10T01:26:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.311+0000 7f04a256e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608db812010 0x5608db812430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.311+0000 7f04a256e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608db812a00 0x5608db817500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.311+0000 7f04a256e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608db817b70 0x5608db81a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.311+0000 7f04a256e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5608db7754f0 con 0x5608db812a00 2024-06-10T01:26:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.311+0000 7f04a256e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5608db713870 con 0x5608db812010 2024-06-10T01:26:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.311+0000 7f04a256e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5608db7714b0 con 0x5608db817b70 2024-06-10T01:26:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.311+0000 7f049d5f9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608db812a00 0x5608db817500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.311+0000 7f049d5f9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608db812a00 0x5608db817500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34764/0 (socket says 172.21.15.7:34764) 2024-06-10T01:26:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.311+0000 7f049d5f9640 1 -- 172.21.15.7:0/4044998917 learned_addr learned my addr 172.21.15.7:0/4044998917 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.311+0000 7f049ddfa640 1 --2- 172.21.15.7:0/4044998917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608db812010 0x5608db812430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.311+0000 7f049cdf8640 1 --2- 172.21.15.7:0/4044998917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608db817b70 0x5608db81a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.311+0000 7f048e7fc640 1 --2- 172.21.15.7:0/1624615008 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f04840a2c00 0x7f04840a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.311+0000 7f049d5f9640 1 -- 172.21.15.7:0/4044998917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608db817b70 msgr2=0x5608db81a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.311+0000 7f049d5f9640 1 --2- 172.21.15.7:0/4044998917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608db817b70 0x5608db81a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.311+0000 7f049d5f9640 1 -- 172.21.15.7:0/4044998917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608db812010 msgr2=0x5608db812430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.311+0000 7f049d5f9640 1 --2- 172.21.15.7:0/4044998917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608db812010 0x5608db812430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.311+0000 7f049d5f9640 1 -- 172.21.15.7:0/4044998917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608db72f9b0 con 0x5608db812a00 2024-06-10T01:26:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.311+0000 7f049ddfa640 1 --2- 172.21.15.7:0/1624615008 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f04840a2c00 0x7f04840a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.311+0000 7f048e7fc640 1 -- 172.21.15.7:0/1624615008 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f04940ef950 con 0x7f0498101a50 2024-06-10T01:26:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.312+0000 7f049d5f9640 1 --2- 172.21.15.7:0/4044998917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608db812a00 0x5608db817500 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f0494099dd0 tx=0x7f0494086b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d246d5c787e24a59 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.312+0000 7f04837fe640 1 -- 172.21.15.7:0/4044998917 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0494049070 con 0x5608db812a00 2024-06-10T01:26:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.312+0000 7f04837fe640 1 -- 172.21.15.7:0/4044998917 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f049403be40 con 0x5608db812a00 2024-06-10T01:26:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.312+0000 7f04837fe640 1 -- 172.21.15.7:0/4044998917 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04940bbe80 con 0x5608db812a00 2024-06-10T01:26:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.313+0000 7f04a256e740 1 -- 172.21.15.7:0/4044998917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608db812a00 msgr2=0x5608db817500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.313+0000 7f04a256e740 1 --2- 172.21.15.7:0/4044998917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608db812a00 0x5608db817500 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f0494099dd0 tx=0x7f0494086b00 comp rx=0 tx=0).stop 2024-06-10T01:26:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.313+0000 7f049ddfa640 1 --2- 172.21.15.7:0/1624615008 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f04840a2c00 0x7f04840a50c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x5608db813260 tx=0x7f0490014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.313+0000 7f04a256e740 1 -- 172.21.15.7:0/4044998917 shutdown_connections 2024-06-10T01:26:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.313+0000 7f04a256e740 1 --2- 172.21.15.7:0/4044998917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608db817b70 0x5608db81a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.313+0000 7f04a256e740 1 --2- 172.21.15.7:0/4044998917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608db812a00 0x5608db817500 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.313+0000 7f04a256e740 1 --2- 172.21.15.7:0/4044998917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608db812010 0x5608db812430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.313+0000 7f04a256e740 1 -- 172.21.15.7:0/4044998917 >> 172.21.15.7:0/4044998917 conn(0x5608db80d680 msgr2=0x5608db80da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.313+0000 7f04a256e740 1 -- 172.21.15.7:0/4044998917 shutdown_connections 2024-06-10T01:26:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.313+0000 7f04a256e740 1 -- 172.21.15.7:0/4044998917 wait complete. 2024-06-10T01:26:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.313+0000 7f04a256e740 1 -- start start 2024-06-10T01:26:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.313+0000 7f04a256e740 1 Processor -- start 2024-06-10T01:26:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.314+0000 7f04a256e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608db812010 0x5608db81bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.314+0000 7f04a256e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608db812a00 0x5608db81c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.314+0000 7f04a256e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608db817b70 0x5608db8234d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.314+0000 7f04a256e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5608db81a8e0 con 0x5608db817b70 2024-06-10T01:26:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.314+0000 7f04a256e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5608db6feae0 con 0x5608db812a00 2024-06-10T01:26:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.314+0000 7f04a256e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5608db81aa60 con 0x5608db812010 2024-06-10T01:26:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.314+0000 7f04a256e740 10 client.0 _ll_register_callbacks cb 0x5608db7379b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:26:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.314+0000 7f049ddfa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608db817b70 0x5608db8234d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.314+0000 7f049ddfa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608db817b70 0x5608db8234d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34778/0 (socket says 172.21.15.7:34778) 2024-06-10T01:26:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.314+0000 7f049ddfa640 1 -- 172.21.15.7:0/1388749598 learned_addr learned my addr 172.21.15.7:0/1388749598 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.314+0000 7f049d5f9640 1 --2- 172.21.15.7:0/1388749598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608db812010 0x5608db81bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.314+0000 7f049cdf8640 1 --2- 172.21.15.7:0/1388749598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608db812a00 0x5608db81c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.314+0000 7f049ddfa640 1 -- 172.21.15.7:0/1388749598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608db812010 msgr2=0x5608db81bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.314+0000 7f049ddfa640 1 --2- 172.21.15.7:0/1388749598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608db812010 0x5608db81bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.314+0000 7f049ddfa640 1 -- 172.21.15.7:0/1388749598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608db812a00 msgr2=0x5608db81c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.314+0000 7f049ddfa640 1 --2- 172.21.15.7:0/1388749598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608db812a00 0x5608db81c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.314+0000 7f049ddfa640 1 -- 172.21.15.7:0/1388749598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608db489210 con 0x5608db817b70 2024-06-10T01:26:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.314+0000 7f049cdf8640 1 --2- 172.21.15.7:0/1388749598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608db812a00 0x5608db81c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:26:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.314+0000 7f049ddfa640 1 --2- 172.21.15.7:0/1388749598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608db817b70 0x5608db8234d0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f0490009fd0 tx=0x7f0490009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6ac5a4ba01a408 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.314+0000 7f0480ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:26:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.315+0000 7f0480ff9640 1 -- 172.21.15.7:0/1388749598 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049001c950 con 0x5608db817b70 2024-06-10T01:26:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.315+0000 7f0480ff9640 1 -- 172.21.15.7:0/1388749598 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f049001caf0 con 0x5608db817b70 2024-06-10T01:26:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.315+0000 7f04a256e740 1 -- 172.21.15.7:0/1388749598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5608db788f10 con 0x5608db817b70 2024-06-10T01:26:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.315+0000 7f0480ff9640 1 -- 172.21.15.7:0/1388749598 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f049001ccf0 con 0x5608db817b70 2024-06-10T01:26:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.315+0000 7f04a256e740 10 client.20397 fetch_fsmap learned FSMap version 52 2024-06-10T01:26:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.315+0000 7f04a256e740 1 -- 172.21.15.7:0/1388749598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5608db555620 con 0x5608db817b70 2024-06-10T01:26:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.315+0000 7f0480ff9640 1 -- 172.21.15.7:0/1388749598 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5608db788f10 con 0x5608db817b70 2024-06-10T01:26:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.315+0000 7f0480ff9640 1 -- 172.21.15.7:0/1388749598 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f0490025020 con 0x5608db817b70 2024-06-10T01:26:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.315+0000 7f04a256e740 10 client.20397 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:26:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.315+0000 7f04a256e740 10 client.20397 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:26:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.315+0000 7f04a256e740 20 client.20397 populate_metadata read hostname 'smithi007' 2024-06-10T01:26:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.315+0000 7f04a256e740 1 --2- 172.21.15.7:0/1388749598 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5608db827e90 0x5608db82a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.315+0000 7f04a256e740 4 client.20397 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:26:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.315+0000 7f04a256e740 1 -- 172.21.15.7:0/1388749598 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608db782c20 con 0x5608db827e90 2024-06-10T01:26:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.316+0000 7f049d5f9640 1 --2- 172.21.15.7:0/1388749598 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5608db827e90 0x5608db82a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.316+0000 7f0480ff9640 10 client.20397 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.316+0000 7f049d5f9640 1 --2- 172.21.15.7:0/1388749598 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5608db827e90 0x5608db82a280 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=683cee414f89d4f4 server_cookie=10504a144d1e39d0 in_seq=0 out_seq=0 2024-06-10T01:26:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.317+0000 7f0480ff9640 1 -- 172.21.15.7:0/1388749598 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5608db782c20 con 0x5608db827e90 2024-06-10T01:26:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.317+0000 7f0480ff9640 10 client.20397 handle_command_reply: tid=0 2024-06-10T01:26:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.317+0000 7f04a256e740 1 client.20397 shutdown 2024-06-10T01:26:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.317+0000 7f04a256e740 1 -- 172.21.15.7:0/1388749598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608db817b70 msgr2=0x5608db8234d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.317+0000 7f04a256e740 1 --2- 172.21.15.7:0/1388749598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608db817b70 0x5608db8234d0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f0490009fd0 tx=0x7f0490009500 comp rx=0 tx=0).stop 2024-06-10T01:26:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.317+0000 7f04a256e740 1 -- 172.21.15.7:0/1388749598 shutdown_connections 2024-06-10T01:26:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.317+0000 7f04a256e740 1 --2- 172.21.15.7:0/1388749598 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5608db827e90 0x5608db82a280 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.317+0000 7f04a256e740 1 --2- 172.21.15.7:0/1388749598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608db817b70 0x5608db8234d0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.317+0000 7f04a256e740 1 --2- 172.21.15.7:0/1388749598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608db812a00 0x5608db81c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.317+0000 7f04a256e740 1 --2- 172.21.15.7:0/1388749598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608db812010 0x5608db81bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.317+0000 7f04a256e740 1 -- 172.21.15.7:0/1388749598 >> 172.21.15.7:0/1388749598 conn(0x5608db80d680 msgr2=0x5608db80c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.317+0000 7f0480ff9640 0 client.20397 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.317+0000 7f04a256e740 1 -- 172.21.15.7:0/1388749598 shutdown_connections 2024-06-10T01:26:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.317+0000 7f04a256e740 1 -- 172.21.15.7:0/1388749598 wait complete. 2024-06-10T01:26:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.317+0000 7f04a256e740 20 client.20397 trim_cache size 0 max 16384 2024-06-10T01:26:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.334+0000 7f04a256e740 1 Processor -- start 2024-06-10T01:26:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.334+0000 7f04a256e740 1 -- start start 2024-06-10T01:26:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.334+0000 7f04a256e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608db812010 0x5608db81bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.334+0000 7f04a256e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608db812a00 0x5608db81c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.334+0000 7f04a256e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608db817b70 0x5608db823490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.334+0000 7f04a256e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f04981ace80 con 0x5608db812a00 2024-06-10T01:26:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.334+0000 7f04a256e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f04981aa750 con 0x5608db812010 2024-06-10T01:26:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.334+0000 7f04a256e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f04981a85a0 con 0x5608db817b70 2024-06-10T01:26:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.334+0000 7f049cdf8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608db812a00 0x5608db81c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.334+0000 7f049cdf8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608db812a00 0x5608db81c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34790/0 (socket says 172.21.15.7:34790) 2024-06-10T01:26:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.334+0000 7f049cdf8640 1 -- 172.21.15.7:0/2264199183 learned_addr learned my addr 172.21.15.7:0/2264199183 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.334+0000 7f049d5f9640 1 --2- 172.21.15.7:0/2264199183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608db812010 0x5608db81bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.334+0000 7f049ddfa640 1 --2- 172.21.15.7:0/2264199183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608db817b70 0x5608db823490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.334+0000 7f049cdf8640 1 -- 172.21.15.7:0/2264199183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608db817b70 msgr2=0x5608db823490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.334+0000 7f049cdf8640 1 --2- 172.21.15.7:0/2264199183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608db817b70 0x5608db823490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.334+0000 7f049cdf8640 1 -- 172.21.15.7:0/2264199183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608db812010 msgr2=0x5608db81bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.334+0000 7f049cdf8640 1 --2- 172.21.15.7:0/2264199183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608db812010 0x5608db81bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.334+0000 7f049cdf8640 1 -- 172.21.15.7:0/2264199183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608db80d4b0 con 0x5608db812a00 2024-06-10T01:26:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.335+0000 7f049cdf8640 1 --2- 172.21.15.7:0/2264199183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608db812a00 0x5608db81c490 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f0488009e10 tx=0x7f0488010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51869c8af305bb67 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.335+0000 7f0482ffd640 1 -- 172.21.15.7:0/2264199183 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0488005220 con 0x5608db812a00 2024-06-10T01:26:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.335+0000 7f0482ffd640 1 -- 172.21.15.7:0/2264199183 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04880047a0 con 0x5608db812a00 2024-06-10T01:26:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.335+0000 7f0482ffd640 1 -- 172.21.15.7:0/2264199183 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048800f950 con 0x5608db812a00 2024-06-10T01:26:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.336+0000 7f04a256e740 1 -- 172.21.15.7:0/2264199183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608db812a00 msgr2=0x5608db81c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.336+0000 7f04a256e740 1 --2- 172.21.15.7:0/2264199183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608db812a00 0x5608db81c490 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f0488009e10 tx=0x7f0488010800 comp rx=0 tx=0).stop 2024-06-10T01:26:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.336+0000 7f04a256e740 1 -- 172.21.15.7:0/2264199183 shutdown_connections 2024-06-10T01:26:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.336+0000 7f04a256e740 1 --2- 172.21.15.7:0/2264199183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608db817b70 0x5608db823490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.336+0000 7f04a256e740 1 --2- 172.21.15.7:0/2264199183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608db812a00 0x5608db81c490 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.336+0000 7f04a256e740 1 --2- 172.21.15.7:0/2264199183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608db812010 0x5608db81bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.336+0000 7f04a256e740 1 -- 172.21.15.7:0/2264199183 >> 172.21.15.7:0/2264199183 conn(0x5608db80d680 msgr2=0x5608db808560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.336+0000 7f04a256e740 1 -- 172.21.15.7:0/2264199183 shutdown_connections 2024-06-10T01:26:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.336+0000 7f04a256e740 1 -- 172.21.15.7:0/2264199183 wait complete. 2024-06-10T01:26:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.336+0000 7f04a256e740 1 -- start start 2024-06-10T01:26:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.336+0000 7f04a256e740 1 Processor -- start 2024-06-10T01:26:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.336+0000 7f04a256e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608db812010 0x5608db820240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.336+0000 7f04a256e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608db812a00 0x5608db820780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.337+0000 7f04a256e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608db817b70 0x5608db890b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.337+0000 7f04a256e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f04981a8c10 con 0x5608db817b70 2024-06-10T01:26:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.337+0000 7f04a256e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f04981aa080 con 0x5608db812010 2024-06-10T01:26:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.337+0000 7f04a256e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f04981a92e0 con 0x5608db812a00 2024-06-10T01:26:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.337+0000 7f04a256e740 10 client.0 _ll_register_callbacks cb 0x7f0490009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:26:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.337+0000 7f049ddfa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608db817b70 0x5608db890b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.337+0000 7f049d5f9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608db812010 0x5608db820240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.337+0000 7f049cdf8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608db812a00 0x5608db820780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.337+0000 7f049ddfa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608db817b70 0x5608db890b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34798/0 (socket says 172.21.15.7:34798) 2024-06-10T01:26:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.337+0000 7f049d5f9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608db812010 0x5608db820240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:37216/0 (socket says 172.21.15.7:37216) 2024-06-10T01:26:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.337+0000 7f049ddfa640 1 -- 172.21.15.7:0/1922446142 learned_addr learned my addr 172.21.15.7:0/1922446142 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.337+0000 7f049ddfa640 1 -- 172.21.15.7:0/1922446142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608db812a00 msgr2=0x5608db820780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.337+0000 7f049ddfa640 1 --2- 172.21.15.7:0/1922446142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608db812a00 0x5608db820780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.337+0000 7f049ddfa640 1 -- 172.21.15.7:0/1922446142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608db812010 msgr2=0x5608db820240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.337+0000 7f049ddfa640 1 --2- 172.21.15.7:0/1922446142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608db812010 0x5608db820240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.337+0000 7f049ddfa640 1 -- 172.21.15.7:0/1922446142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608db820df0 con 0x5608db817b70 2024-06-10T01:26:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.337+0000 7f049ddfa640 1 --2- 172.21.15.7:0/1922446142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608db817b70 0x5608db890b50 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f049001d580 tx=0x7f0490005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2fb7c97cd79942 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.337+0000 7f0480ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:26:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.337+0000 7f0480ff9640 1 -- 172.21.15.7:0/1922446142 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049001ce90 con 0x5608db817b70 2024-06-10T01:26:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.337+0000 7f0480ff9640 1 -- 172.21.15.7:0/1922446142 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f049001c750 con 0x5608db817b70 2024-06-10T01:26:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.337+0000 7f04a256e740 1 -- 172.21.15.7:0/1922446142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04880047a0 con 0x5608db817b70 2024-06-10T01:26:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.337+0000 7f0480ff9640 1 -- 172.21.15.7:0/1922446142 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f0490041050 con 0x5608db817b70 2024-06-10T01:26:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.338+0000 7f04a256e740 10 client.20403 fetch_fsmap learned FSMap version 52 2024-06-10T01:26:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.338+0000 7f04a256e740 1 -- 172.21.15.7:0/1922446142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5608db8921f0 con 0x5608db817b70 2024-06-10T01:26:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.338+0000 7f0480ff9640 1 -- 172.21.15.7:0/1922446142 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04880047a0 con 0x5608db817b70 2024-06-10T01:26:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.338+0000 7f0480ff9640 1 -- 172.21.15.7:0/1922446142 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f049003c030 con 0x5608db817b70 2024-06-10T01:26:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.338+0000 7f04a256e740 10 client.20403 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:26:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.338+0000 7f04a256e740 10 client.20403 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:26:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.338+0000 7f04a256e740 20 client.20403 populate_metadata read hostname 'smithi007' 2024-06-10T01:26:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.338+0000 7f04a256e740 1 --2- 172.21.15.7:0/1922446142 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5608db892b60 0x5608db894f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.338+0000 7f04a256e740 4 client.20403 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:26:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.338+0000 7f04a256e740 1 -- 172.21.15.7:0/1922446142 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5608db77cf70 con 0x5608db892b60 2024-06-10T01:26:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.338+0000 7f049d5f9640 1 --2- 172.21.15.7:0/1922446142 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5608db892b60 0x5608db894f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.339+0000 7f049d5f9640 1 --2- 172.21.15.7:0/1922446142 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5608db892b60 0x5608db894f80 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e236cc936e8a15aa server_cookie=272b882d19450f7c in_seq=0 out_seq=0 2024-06-10T01:26:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.339+0000 7f0480ff9640 10 client.20403 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:35.546 INFO:tasks.workunit.client.0.smithi007.stdout: 10 174555 23.15 MB/sec execute 312 sec latency 4275.823 ms 2024-06-10T01:26:35.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:35 smithi078 ceph-mon[36683]: pgmap v1050: 129 pgs: 129 active+clean; 655 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.7 MiB/s wr, 246 op/s 2024-06-10T01:26:35.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.642+0000 7f0480ff9640 1 -- 172.21.15.7:0/1922446142 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5608db77cf70 con 0x5608db892b60 2024-06-10T01:26:35.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.642+0000 7f0480ff9640 10 client.20403 handle_command_reply: tid=0 2024-06-10T01:26:35.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.642+0000 7f04a256e740 1 client.20403 shutdown 2024-06-10T01:26:35.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.642+0000 7f04a256e740 1 -- 172.21.15.7:0/1922446142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608db817b70 msgr2=0x5608db890b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:35.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.642+0000 7f04a256e740 1 --2- 172.21.15.7:0/1922446142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608db817b70 0x5608db890b50 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f049001d580 tx=0x7f0490005c40 comp rx=0 tx=0).stop 2024-06-10T01:26:35.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.642+0000 7f04a256e740 1 -- 172.21.15.7:0/1922446142 shutdown_connections 2024-06-10T01:26:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.642+0000 7f04a256e740 1 --2- 172.21.15.7:0/1922446142 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5608db892b60 0x5608db894f80 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.642+0000 7f04a256e740 1 --2- 172.21.15.7:0/1922446142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608db817b70 0x5608db890b50 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.642+0000 7f0480ff9640 0 client.20403 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.642+0000 7f04a256e740 1 --2- 172.21.15.7:0/1922446142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608db812a00 0x5608db820780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.642+0000 7f04a256e740 1 --2- 172.21.15.7:0/1922446142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608db812010 0x5608db820240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.642+0000 7f04a256e740 1 -- 172.21.15.7:0/1922446142 >> 172.21.15.7:0/1922446142 conn(0x5608db80d680 msgr2=0x5608db81bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.642+0000 7f04a256e740 1 -- 172.21.15.7:0/1922446142 shutdown_connections 2024-06-10T01:26:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.642+0000 7f04a256e740 1 -- 172.21.15.7:0/1922446142 wait complete. 2024-06-10T01:26:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.642+0000 7f04a256e740 20 client.20403 trim_cache size 0 max 16384 2024-06-10T01:26:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:26:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:26:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:26:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.643+0000 7f049f85d640 1 -- 172.21.15.7:0/1624615008 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f04840a2c00 msgr2=0x7f04840a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.643+0000 7f049f85d640 1 --2- 172.21.15.7:0/1624615008 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f04840a2c00 0x7f04840a50c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x5608db813260 tx=0x7f0490014040 comp rx=0 tx=0).stop 2024-06-10T01:26:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.643+0000 7f049f85d640 1 -- 172.21.15.7:0/1624615008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0498101a50 msgr2=0x7f0498197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.643+0000 7f049f85d640 1 --2- 172.21.15.7:0/1624615008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0498101a50 0x7f0498197d40 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f0494031610 tx=0x7f0494004270 comp rx=0 tx=0).stop 2024-06-10T01:26:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.643+0000 7f049f85d640 1 -- 172.21.15.7:0/1624615008 shutdown_connections 2024-06-10T01:26:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.643+0000 7f049f85d640 1 --2- 172.21.15.7:0/1624615008 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f04840a2c00 0x7f04840a50c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.643+0000 7f049f85d640 1 --2- 172.21.15.7:0/1624615008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0498103600 0x7f049819f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.643+0000 7f049f85d640 1 --2- 172.21.15.7:0/1624615008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0498102c40 0x7f0498198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.643+0000 7f049f85d640 1 --2- 172.21.15.7:0/1624615008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0498101a50 0x7f0498197d40 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.643+0000 7f049f85d640 1 -- 172.21.15.7:0/1624615008 >> 172.21.15.7:0/1624615008 conn(0x7f04980fcfc0 msgr2=0x7f0498105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.643+0000 7f049f85d640 1 -- 172.21.15.7:0/1624615008 shutdown_connections 2024-06-10T01:26:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:35.643+0000 7f049f85d640 1 -- 172.21.15.7:0/1624615008 wait complete. 2024-06-10T01:26:35.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:35 smithi175 ceph-mon[36636]: pgmap v1050: 129 pgs: 129 active+clean; 655 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.7 MiB/s wr, 246 op/s 2024-06-10T01:26:35.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:35 smithi007 ceph-mon[32398]: pgmap v1050: 129 pgs: 129 active+clean; 655 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.7 MiB/s wr, 246 op/s 2024-06-10T01:26:36.546 INFO:tasks.workunit.client.0.smithi007.stdout: 10 174555 23.08 MB/sec execute 313 sec latency 5275.942 ms 2024-06-10T01:26:37.546 INFO:tasks.workunit.client.0.smithi007.stdout: 10 174555 23.00 MB/sec execute 314 sec latency 6275.994 ms 2024-06-10T01:26:37.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:37 smithi007 ceph-mon[32398]: pgmap v1051: 129 pgs: 129 active+clean; 659 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.2 MiB/s wr, 331 op/s 2024-06-10T01:26:37.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:37 smithi078 ceph-mon[36683]: pgmap v1051: 129 pgs: 129 active+clean; 659 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.2 MiB/s wr, 331 op/s 2024-06-10T01:26:37.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:37 smithi175 ceph-mon[36636]: pgmap v1051: 129 pgs: 129 active+clean; 659 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.2 MiB/s wr, 331 op/s 2024-06-10T01:26:38.547 INFO:tasks.workunit.client.0.smithi007.stdout: 10 174555 22.93 MB/sec execute 315 sec latency 7276.081 ms 2024-06-10T01:26:39.547 INFO:tasks.workunit.client.0.smithi007.stdout: 10 174555 22.86 MB/sec execute 316 sec latency 8276.217 ms 2024-06-10T01:26:39.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:39 smithi007 ceph-mon[32398]: pgmap v1052: 129 pgs: 129 active+clean; 655 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.1 MiB/s wr, 339 op/s 2024-06-10T01:26:39.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:39 smithi078 ceph-mon[36683]: pgmap v1052: 129 pgs: 129 active+clean; 655 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.1 MiB/s wr, 339 op/s 2024-06-10T01:26:39.954 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:39 smithi175 ceph-mon[36636]: pgmap v1052: 129 pgs: 129 active+clean; 655 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.1 MiB/s wr, 339 op/s 2024-06-10T01:26:40.547 INFO:tasks.workunit.client.0.smithi007.stdout: 10 174555 22.79 MB/sec execute 317 sec latency 9276.272 ms 2024-06-10T01:26:41.547 INFO:tasks.workunit.client.0.smithi007.stdout: 10 174555 22.71 MB/sec execute 318 sec latency 10276.332 ms 2024-06-10T01:26:41.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:41 smithi007 ceph-mon[32398]: pgmap v1053: 129 pgs: 129 active+clean; 655 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.9 MiB/s wr, 323 op/s 2024-06-10T01:26:41.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:41 smithi078 ceph-mon[36683]: pgmap v1053: 129 pgs: 129 active+clean; 655 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.9 MiB/s wr, 323 op/s 2024-06-10T01:26:41.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:41 smithi175 ceph-mon[36636]: pgmap v1053: 129 pgs: 129 active+clean; 655 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.9 MiB/s wr, 323 op/s 2024-06-10T01:26:42.547 INFO:tasks.workunit.client.0.smithi007.stdout: 10 174555 22.64 MB/sec execute 319 sec latency 11276.576 ms 2024-06-10T01:26:43.547 INFO:tasks.workunit.client.0.smithi007.stdout: 10 174555 22.57 MB/sec execute 320 sec latency 12276.690 ms 2024-06-10T01:26:43.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:43 smithi007 ceph-mon[32398]: pgmap v1054: 129 pgs: 129 active+clean; 664 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.3 MiB/s wr, 276 op/s 2024-06-10T01:26:43.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:43 smithi078 ceph-mon[36683]: pgmap v1054: 129 pgs: 129 active+clean; 664 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.3 MiB/s wr, 276 op/s 2024-06-10T01:26:43.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:43 smithi175 ceph-mon[36636]: pgmap v1054: 129 pgs: 129 active+clean; 664 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.3 MiB/s wr, 276 op/s 2024-06-10T01:26:44.547 INFO:tasks.workunit.client.0.smithi007.stdout: 10 174555 22.50 MB/sec execute 321 sec latency 13276.751 ms 2024-06-10T01:26:45.547 INFO:tasks.workunit.client.0.smithi007.stdout: 10 174555 22.43 MB/sec execute 322 sec latency 14276.881 ms 2024-06-10T01:26:45.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:45 smithi078 ceph-mon[36683]: pgmap v1055: 129 pgs: 129 active+clean; 665 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 192 op/s 2024-06-10T01:26:45.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:45 smithi175 ceph-mon[36636]: pgmap v1055: 129 pgs: 129 active+clean; 665 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 192 op/s 2024-06-10T01:26:46.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:45 smithi007 ceph-mon[32398]: pgmap v1055: 129 pgs: 129 active+clean; 665 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 192 op/s 2024-06-10T01:26:46.548 INFO:tasks.workunit.client.0.smithi007.stdout: 10 174555 22.36 MB/sec execute 323 sec latency 15276.984 ms 2024-06-10T01:26:47.548 INFO:tasks.workunit.client.0.smithi007.stdout: 10 174555 22.29 MB/sec execute 324 sec latency 16277.107 ms 2024-06-10T01:26:47.616 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:26:47.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:47 smithi078 ceph-mon[36683]: pgmap v1056: 129 pgs: 129 active+clean; 674 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 268 op/s 2024-06-10T01:26:47.883 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:26:47.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:47 smithi175 ceph-mon[36636]: pgmap v1056: 129 pgs: 129 active+clean; 674 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 268 op/s 2024-06-10T01:26:48.006 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:47 smithi007 ceph-mon[32398]: pgmap v1056: 129 pgs: 129 active+clean; 674 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 268 op/s 2024-06-10T01:26:48.548 INFO:tasks.workunit.client.0.smithi007.stdout: 10 174555 22.22 MB/sec execute 325 sec latency 17277.203 ms 2024-06-10T01:26:48.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.738+0000 7f22bad53640 1 -- 172.21.15.7:0/413001024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22b41035e0 msgr2=0x7f22b4109e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:48.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.738+0000 7f22bad53640 1 --2- 172.21.15.7:0/413001024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22b41035e0 0x7f22b4109e70 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f22ac009900 tx=0x7f22ac02a410 comp rx=0 tx=0).stop 2024-06-10T01:26:48.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.738+0000 7f22bad53640 1 -- 172.21.15.7:0/413001024 shutdown_connections 2024-06-10T01:26:48.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.738+0000 7f22bad53640 1 --2- 172.21.15.7:0/413001024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22b41035e0 0x7f22b4109e70 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.738+0000 7f22bad53640 1 --2- 172.21.15.7:0/413001024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22b4102c20 0x7f22b41030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.738+0000 7f22bad53640 1 --2- 172.21.15.7:0/413001024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22b4101a30 0x7f22b4101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.738+0000 7f22bad53640 1 -- 172.21.15.7:0/413001024 >> 172.21.15.7:0/413001024 conn(0x7f22b40fcfc0 msgr2=0x7f22b40ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.738+0000 7f22bad53640 1 -- 172.21.15.7:0/413001024 shutdown_connections 2024-06-10T01:26:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.738+0000 7f22bad53640 1 -- 172.21.15.7:0/413001024 wait complete. 2024-06-10T01:26:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.739+0000 7f22bad53640 1 Processor -- start 2024-06-10T01:26:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.739+0000 7f22bad53640 1 -- start start 2024-06-10T01:26:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.739+0000 7f22bad53640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22b4101a30 0x7f22b4197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.739+0000 7f22bad53640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22b4102c20 0x7f22b4198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.739+0000 7f22bad53640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22b41035e0 0x7f22b419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.739+0000 7f22bad53640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f22b410d690 con 0x7f22b4101a30 2024-06-10T01:26:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.739+0000 7f22bad53640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f22b410d510 con 0x7f22b41035e0 2024-06-10T01:26:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.739+0000 7f22bad53640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f22b410d810 con 0x7f22b4102c20 2024-06-10T01:26:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.739+0000 7f22b8aef640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22b4101a30 0x7f22b4197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.739+0000 7f22b8aef640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22b4101a30 0x7f22b4197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52940/0 (socket says 172.21.15.7:52940) 2024-06-10T01:26:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.739+0000 7f22b8aef640 1 -- 172.21.15.7:0/2256561491 learned_addr learned my addr 172.21.15.7:0/2256561491 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.739+0000 7f22b92f0640 1 --2- 172.21.15.7:0/2256561491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22b41035e0 0x7f22b419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.739+0000 7f22b3fff640 1 --2- 172.21.15.7:0/2256561491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22b4102c20 0x7f22b4198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.739+0000 7f22b8aef640 1 -- 172.21.15.7:0/2256561491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22b4102c20 msgr2=0x7f22b4198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.739+0000 7f22b8aef640 1 --2- 172.21.15.7:0/2256561491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22b4102c20 0x7f22b4198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.739+0000 7f22b8aef640 1 -- 172.21.15.7:0/2256561491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22b41035e0 msgr2=0x7f22b419f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.739+0000 7f22b8aef640 1 --2- 172.21.15.7:0/2256561491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22b41035e0 0x7f22b419f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.739+0000 7f22b8aef640 1 -- 172.21.15.7:0/2256561491 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b419f840 con 0x7f22b4101a30 2024-06-10T01:26:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.740+0000 7f22b8aef640 1 --2- 172.21.15.7:0/2256561491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22b4101a30 0x7f22b4197c30 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f22a400ccb0 tx=0x7f22a400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.740+0000 7f22b1ffb640 1 -- 172.21.15.7:0/2256561491 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4003b70 con 0x7f22b4101a30 2024-06-10T01:26:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.740+0000 7f22b1ffb640 1 -- 172.21.15.7:0/2256561491 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22a400ae50 con 0x7f22b4101a30 2024-06-10T01:26:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.740+0000 7f22bad53640 1 -- 172.21.15.7:0/2256561491 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22b419fb30 con 0x7f22b4101a30 2024-06-10T01:26:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.740+0000 7f22b1ffb640 1 -- 172.21.15.7:0/2256561491 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a400f810 con 0x7f22b4101a30 2024-06-10T01:26:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.740+0000 7f22bad53640 1 -- 172.21.15.7:0/2256561491 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22b41a0080 con 0x7f22b4101a30 2024-06-10T01:26:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.742+0000 7f22b1ffb640 1 -- 172.21.15.7:0/2256561491 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f22a400f9b0 con 0x7f22b4101a30 2024-06-10T01:26:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.742+0000 7f22bda5e740 1 Processor -- start 2024-06-10T01:26:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.742+0000 7f22bda5e740 1 -- start start 2024-06-10T01:26:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.743+0000 7f22bda5e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55874a260010 0x55874a260430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.743+0000 7f22bda5e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874a260a00 0x55874a265500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.743+0000 7f22bda5e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874a265b70 0x55874a268000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.743+0000 7f22bda5e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55874a1c34f0 con 0x55874a265b70 2024-06-10T01:26:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.743+0000 7f22bda5e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55874a161870 con 0x55874a260a00 2024-06-10T01:26:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.743+0000 7f22b1ffb640 1 --2- 172.21.15.7:0/2256561491 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f22a00a2c00 0x7f22a00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.743+0000 7f22bda5e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55874a1bf4b0 con 0x55874a260010 2024-06-10T01:26:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.743+0000 7f22b3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874a265b70 0x55874a268000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.743+0000 7f22b3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874a265b70 0x55874a268000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52946/0 (socket says 172.21.15.7:52946) 2024-06-10T01:26:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.743+0000 7f22b3fff640 1 -- 172.21.15.7:0/2699533297 learned_addr learned my addr 172.21.15.7:0/2699533297 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.743+0000 7f22b92f0640 1 --2- 172.21.15.7:0/2256561491 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f22a00a2c00 0x7f22a00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.743+0000 7f22b1ffb640 1 -- 172.21.15.7:0/2256561491 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f22a40cf650 con 0x7f22b4101a30 2024-06-10T01:26:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.743+0000 7f22b8aef640 1 --2- 172.21.15.7:0/2699533297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874a260a00 0x55874a265500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.743+0000 7f22b92f0640 1 --2- 172.21.15.7:0/2699533297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55874a260010 0x55874a260430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.743+0000 7f22b3fff640 1 -- 172.21.15.7:0/2699533297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55874a260010 msgr2=0x55874a260430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.743+0000 7f22b3fff640 1 --2- 172.21.15.7:0/2699533297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55874a260010 0x55874a260430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.743+0000 7f22b3fff640 1 -- 172.21.15.7:0/2699533297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874a260a00 msgr2=0x55874a265500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.743+0000 7f22b3fff640 1 --2- 172.21.15.7:0/2699533297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874a260a00 0x55874a265500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.743+0000 7f22b3fff640 1 -- 172.21.15.7:0/2699533297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55874a17d9b0 con 0x55874a265b70 2024-06-10T01:26:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.743+0000 7f22b92f0640 1 --2- 172.21.15.7:0/2256561491 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f22a00a2c00 0x7f22a00a50c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f22ac006010 tx=0x7f22ac031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.743+0000 7f22b3fff640 1 --2- 172.21.15.7:0/2699533297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874a265b70 0x55874a268000 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f229c00bcb0 tx=0x7f229c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee252325d7f6145 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.744+0000 7f22aaffd640 1 -- 172.21.15.7:0/2699533297 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c015070 con 0x55874a265b70 2024-06-10T01:26:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.744+0000 7f22aaffd640 1 -- 172.21.15.7:0/2699533297 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f229c01a940 con 0x55874a265b70 2024-06-10T01:26:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.744+0000 7f22aaffd640 1 -- 172.21.15.7:0/2699533297 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c011a80 con 0x55874a265b70 2024-06-10T01:26:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.744+0000 7f22bda5e740 1 -- 172.21.15.7:0/2699533297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874a265b70 msgr2=0x55874a268000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.744+0000 7f22bda5e740 1 --2- 172.21.15.7:0/2699533297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874a265b70 0x55874a268000 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f229c00bcb0 tx=0x7f229c00f730 comp rx=0 tx=0).stop 2024-06-10T01:26:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.744+0000 7f22bda5e740 1 -- 172.21.15.7:0/2699533297 shutdown_connections 2024-06-10T01:26:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.744+0000 7f22bda5e740 1 --2- 172.21.15.7:0/2699533297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874a265b70 0x55874a268000 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.744+0000 7f22bda5e740 1 --2- 172.21.15.7:0/2699533297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874a260a00 0x55874a265500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.744+0000 7f22bda5e740 1 --2- 172.21.15.7:0/2699533297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55874a260010 0x55874a260430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.744+0000 7f22bda5e740 1 -- 172.21.15.7:0/2699533297 >> 172.21.15.7:0/2699533297 conn(0x55874a25b680 msgr2=0x55874a25ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.744+0000 7f22bda5e740 1 -- 172.21.15.7:0/2699533297 shutdown_connections 2024-06-10T01:26:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.744+0000 7f22bda5e740 1 -- 172.21.15.7:0/2699533297 wait complete. 2024-06-10T01:26:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.745+0000 7f22bda5e740 1 -- start start 2024-06-10T01:26:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.745+0000 7f22bda5e740 1 Processor -- start 2024-06-10T01:26:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.745+0000 7f22bda5e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874a260010 0x55874a26a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.745+0000 7f22bda5e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55874a260a00 0x55874a26a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.745+0000 7f22bda5e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874a265b70 0x55874a271770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.745+0000 7f22bda5e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55874a268b20 con 0x55874a265b70 2024-06-10T01:26:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.745+0000 7f22bda5e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55874a14cae0 con 0x55874a260010 2024-06-10T01:26:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.745+0000 7f22bda5e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55874a268ca0 con 0x55874a260a00 2024-06-10T01:26:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.745+0000 7f22bda5e740 10 client.0 _ll_register_callbacks cb 0x55874a1859b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:26:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.745+0000 7f22b92f0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874a265b70 0x55874a271770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.745+0000 7f22b92f0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874a265b70 0x55874a271770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52960/0 (socket says 172.21.15.7:52960) 2024-06-10T01:26:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.745+0000 7f22b92f0640 1 -- 172.21.15.7:0/3895075723 learned_addr learned my addr 172.21.15.7:0/3895075723 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.745+0000 7f22b3fff640 1 --2- 172.21.15.7:0/3895075723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55874a260a00 0x55874a26a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.746+0000 7f22b92f0640 1 -- 172.21.15.7:0/3895075723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55874a260a00 msgr2=0x55874a26a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.746+0000 7f22b92f0640 1 --2- 172.21.15.7:0/3895075723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55874a260a00 0x55874a26a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.746+0000 7f22b92f0640 1 -- 172.21.15.7:0/3895075723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874a260010 msgr2=0x55874a26a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.746+0000 7f22b92f0640 1 --2- 172.21.15.7:0/3895075723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874a260010 0x55874a26a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.746+0000 7f22b92f0640 1 -- 172.21.15.7:0/3895075723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558749ed7210 con 0x55874a265b70 2024-06-10T01:26:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.746+0000 7f22b92f0640 1 --2- 172.21.15.7:0/3895075723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874a265b70 0x55874a271770 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f22ac009900 tx=0x7f22ac02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28fddd19c186a337 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.746+0000 7f2293fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:26:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.746+0000 7f2293fff640 1 -- 172.21.15.7:0/3895075723 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac003ac0 con 0x55874a265b70 2024-06-10T01:26:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.746+0000 7f2293fff640 1 -- 172.21.15.7:0/3895075723 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f22ac003c60 con 0x55874a265b70 2024-06-10T01:26:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.746+0000 7f22bda5e740 1 -- 172.21.15.7:0/3895075723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f229c01a940 con 0x55874a265b70 2024-06-10T01:26:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.746+0000 7f2293fff640 1 -- 172.21.15.7:0/3895075723 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f22ac059050 con 0x55874a265b70 2024-06-10T01:26:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.746+0000 7f22bda5e740 10 client.20415 fetch_fsmap learned FSMap version 52 2024-06-10T01:26:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.746+0000 7f22bda5e740 1 -- 172.21.15.7:0/3895075723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x558749fa3620 con 0x55874a265b70 2024-06-10T01:26:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.746+0000 7f2293fff640 1 -- 172.21.15.7:0/3895075723 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f229c01a940 con 0x55874a265b70 2024-06-10T01:26:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.746+0000 7f2293fff640 1 -- 172.21.15.7:0/3895075723 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f22ac03c470 con 0x55874a265b70 2024-06-10T01:26:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.746+0000 7f22bda5e740 10 client.20415 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:26:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.746+0000 7f22bda5e740 10 client.20415 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:26:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.746+0000 7f22bda5e740 20 client.20415 populate_metadata read hostname 'smithi007' 2024-06-10T01:26:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.746+0000 7f22bda5e740 1 --2- 172.21.15.7:0/3895075723 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55874a276130 0x55874a2785b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.746+0000 7f22bda5e740 4 client.20415 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:26:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.746+0000 7f22bda5e740 1 -- 172.21.15.7:0/3895075723 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55874a1d6f10 con 0x55874a276130 2024-06-10T01:26:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.747+0000 7f22b8aef640 1 --2- 172.21.15.7:0/3895075723 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55874a276130 0x55874a2785b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.747+0000 7f2293fff640 10 client.20415 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.747+0000 7f22b8aef640 1 --2- 172.21.15.7:0/3895075723 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55874a276130 0x55874a2785b0 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=859e847f3da19e50 server_cookie=f1022894456781c7 in_seq=0 out_seq=0 2024-06-10T01:26:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.748+0000 7f2293fff640 1 -- 172.21.15.7:0/3895075723 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55874a1d6f10 con 0x55874a276130 2024-06-10T01:26:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.748+0000 7f2293fff640 10 client.20415 handle_command_reply: tid=0 2024-06-10T01:26:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.748+0000 7f22bda5e740 1 client.20415 shutdown 2024-06-10T01:26:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.748+0000 7f22bda5e740 1 -- 172.21.15.7:0/3895075723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874a265b70 msgr2=0x55874a271770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.748+0000 7f22bda5e740 1 --2- 172.21.15.7:0/3895075723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874a265b70 0x55874a271770 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f22ac009900 tx=0x7f22ac02a410 comp rx=0 tx=0).stop 2024-06-10T01:26:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.748+0000 7f22bda5e740 1 -- 172.21.15.7:0/3895075723 shutdown_connections 2024-06-10T01:26:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.748+0000 7f22bda5e740 1 --2- 172.21.15.7:0/3895075723 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55874a276130 0x55874a2785b0 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.748+0000 7f22bda5e740 1 --2- 172.21.15.7:0/3895075723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874a265b70 0x55874a271770 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.748+0000 7f22bda5e740 1 --2- 172.21.15.7:0/3895075723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55874a260a00 0x55874a26a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.748+0000 7f2293fff640 0 client.20415 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.748+0000 7f22bda5e740 1 --2- 172.21.15.7:0/3895075723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874a260010 0x55874a26a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.748+0000 7f22bda5e740 1 -- 172.21.15.7:0/3895075723 >> 172.21.15.7:0/3895075723 conn(0x55874a25b680 msgr2=0x55874a25a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.748+0000 7f22bda5e740 1 -- 172.21.15.7:0/3895075723 shutdown_connections 2024-06-10T01:26:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.748+0000 7f22bda5e740 1 -- 172.21.15.7:0/3895075723 wait complete. 2024-06-10T01:26:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.748+0000 7f22bda5e740 20 client.20415 trim_cache size 0 max 16384 2024-06-10T01:26:48.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.763+0000 7f22bda5e740 1 Processor -- start 2024-06-10T01:26:48.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.764+0000 7f22bda5e740 1 -- start start 2024-06-10T01:26:48.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.764+0000 7f22bda5e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55874a260010 0x55874a26a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:48.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.764+0000 7f22bda5e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874a260a00 0x55874a26a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.764+0000 7f22bda5e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874a265b70 0x55874a271730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.764+0000 7f22bda5e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f22b41acd60 con 0x55874a260a00 2024-06-10T01:26:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.764+0000 7f22bda5e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f22b41aa630 con 0x55874a265b70 2024-06-10T01:26:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.764+0000 7f22bda5e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f22b41a8480 con 0x55874a260010 2024-06-10T01:26:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.764+0000 7f22b3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874a260a00 0x55874a26a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.764+0000 7f22b3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874a260a00 0x55874a26a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52966/0 (socket says 172.21.15.7:52966) 2024-06-10T01:26:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.764+0000 7f22b3fff640 1 -- 172.21.15.7:0/693312375 learned_addr learned my addr 172.21.15.7:0/693312375 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.764+0000 7f22b92f0640 1 --2- 172.21.15.7:0/693312375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874a265b70 0x55874a271730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.764+0000 7f22b8aef640 1 --2- 172.21.15.7:0/693312375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55874a260010 0x55874a26a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.764+0000 7f22b3fff640 1 -- 172.21.15.7:0/693312375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55874a260010 msgr2=0x55874a26a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.764+0000 7f22b3fff640 1 --2- 172.21.15.7:0/693312375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55874a260010 0x55874a26a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.764+0000 7f22b3fff640 1 -- 172.21.15.7:0/693312375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874a265b70 msgr2=0x55874a271730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.764+0000 7f22b3fff640 1 --2- 172.21.15.7:0/693312375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874a265b70 0x55874a271730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.764+0000 7f22b3fff640 1 -- 172.21.15.7:0/693312375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55874a25b4b0 con 0x55874a260a00 2024-06-10T01:26:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.764+0000 7f22b8aef640 1 --2- 172.21.15.7:0/693312375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55874a260010 0x55874a26a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:26:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.764+0000 7f22b92f0640 1 --2- 172.21.15.7:0/693312375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874a265b70 0x55874a271730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:26:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.764+0000 7f22b3fff640 1 --2- 172.21.15.7:0/693312375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874a260a00 0x55874a26a730 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x55874a25c550 tx=0x7f229c0115d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26605bfa6715ba0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:48.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.764+0000 7f22aa7fc640 1 -- 172.21.15.7:0/693312375 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c029070 con 0x55874a260a00 2024-06-10T01:26:48.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.764+0000 7f22aa7fc640 1 -- 172.21.15.7:0/693312375 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f229c00b760 con 0x55874a260a00 2024-06-10T01:26:48.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.764+0000 7f22aa7fc640 1 -- 172.21.15.7:0/693312375 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c015040 con 0x55874a260a00 2024-06-10T01:26:48.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.765+0000 7f22bda5e740 1 -- 172.21.15.7:0/693312375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874a260a00 msgr2=0x55874a26a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:48.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.765+0000 7f22bda5e740 1 --2- 172.21.15.7:0/693312375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874a260a00 0x55874a26a730 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x55874a25c550 tx=0x7f229c0115d0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.765+0000 7f22bda5e740 1 -- 172.21.15.7:0/693312375 shutdown_connections 2024-06-10T01:26:48.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.765+0000 7f22bda5e740 1 --2- 172.21.15.7:0/693312375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874a265b70 0x55874a271730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.765+0000 7f22bda5e740 1 --2- 172.21.15.7:0/693312375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874a260a00 0x55874a26a730 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.765+0000 7f22bda5e740 1 --2- 172.21.15.7:0/693312375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55874a260010 0x55874a26a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.765+0000 7f22bda5e740 1 -- 172.21.15.7:0/693312375 >> 172.21.15.7:0/693312375 conn(0x55874a25b680 msgr2=0x55874a265210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:48.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.765+0000 7f22bda5e740 1 -- 172.21.15.7:0/693312375 shutdown_connections 2024-06-10T01:26:48.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.765+0000 7f22bda5e740 1 -- 172.21.15.7:0/693312375 wait complete. 2024-06-10T01:26:48.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.765+0000 7f22bda5e740 1 -- start start 2024-06-10T01:26:48.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.765+0000 7f22bda5e740 1 Processor -- start 2024-06-10T01:26:48.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.766+0000 7f22bda5e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55874a260010 0x55874a26e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:48.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.766+0000 7f22bda5e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874a260a00 0x55874a26ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:48.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.766+0000 7f22bda5e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874a265b70 0x55874a2dee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:48.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.766+0000 7f22bda5e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f22b41a8af0 con 0x55874a265b70 2024-06-10T01:26:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.766+0000 7f22bda5e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f22b41a9f60 con 0x55874a260a00 2024-06-10T01:26:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.766+0000 7f22bda5e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f22b41a91c0 con 0x55874a260010 2024-06-10T01:26:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.766+0000 7f22bda5e740 10 client.0 _ll_register_callbacks cb 0x7f22ac032e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:26:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.766+0000 7f22b92f0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874a265b70 0x55874a2dee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.766+0000 7f22b92f0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874a265b70 0x55874a2dee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52978/0 (socket says 172.21.15.7:52978) 2024-06-10T01:26:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.766+0000 7f22b92f0640 1 -- 172.21.15.7:0/2490219612 learned_addr learned my addr 172.21.15.7:0/2490219612 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.766+0000 7f22b8aef640 1 --2- 172.21.15.7:0/2490219612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55874a260010 0x55874a26e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.766+0000 7f22b3fff640 1 --2- 172.21.15.7:0/2490219612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874a260a00 0x55874a26ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.766+0000 7f22b92f0640 1 -- 172.21.15.7:0/2490219612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55874a260010 msgr2=0x55874a26e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.766+0000 7f22b92f0640 1 --2- 172.21.15.7:0/2490219612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55874a260010 0x55874a26e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.766+0000 7f22b92f0640 1 -- 172.21.15.7:0/2490219612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874a260a00 msgr2=0x55874a26ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.766+0000 7f22b92f0640 1 --2- 172.21.15.7:0/2490219612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874a260a00 0x55874a26ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.766+0000 7f22b92f0640 1 -- 172.21.15.7:0/2490219612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55874a26f0a0 con 0x55874a265b70 2024-06-10T01:26:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.766+0000 7f22b92f0640 1 --2- 172.21.15.7:0/2490219612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874a265b70 0x55874a2dee80 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f22ac004650 tx=0x7f22ac03ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=642fb8b6e1527743 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.766+0000 7f2293fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:26:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.767+0000 7f2293fff640 1 -- 172.21.15.7:0/2490219612 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac061020 con 0x55874a265b70 2024-06-10T01:26:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.767+0000 7f22bda5e740 1 -- 172.21.15.7:0/2490219612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f229c00b760 con 0x55874a265b70 2024-06-10T01:26:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.767+0000 7f2293fff640 1 -- 172.21.15.7:0/2490219612 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f22ac0038c0 con 0x55874a265b70 2024-06-10T01:26:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.767+0000 7f2293fff640 1 -- 172.21.15.7:0/2490219612 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f22ac009520 con 0x55874a265b70 2024-06-10T01:26:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.767+0000 7f22bda5e740 10 client.20421 fetch_fsmap learned FSMap version 52 2024-06-10T01:26:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.767+0000 7f22bda5e740 1 -- 172.21.15.7:0/2490219612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55874a2e0480 con 0x55874a265b70 2024-06-10T01:26:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.767+0000 7f2293fff640 1 -- 172.21.15.7:0/2490219612 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f229c00b760 con 0x55874a265b70 2024-06-10T01:26:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.767+0000 7f2293fff640 1 -- 172.21.15.7:0/2490219612 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f22ac0054f0 con 0x55874a265b70 2024-06-10T01:26:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.767+0000 7f22bda5e740 10 client.20421 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:26:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.767+0000 7f22bda5e740 10 client.20421 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:26:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.767+0000 7f22bda5e740 20 client.20421 populate_metadata read hostname 'smithi007' 2024-06-10T01:26:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.767+0000 7f22bda5e740 1 --2- 172.21.15.7:0/2490219612 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55874a2e0eb0 0x55874a2e32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.767+0000 7f22bda5e740 4 client.20421 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:26:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.767+0000 7f22bda5e740 1 -- 172.21.15.7:0/2490219612 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55874a1d0c20 con 0x55874a2e0eb0 2024-06-10T01:26:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.768+0000 7f22b8aef640 1 --2- 172.21.15.7:0/2490219612 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55874a2e0eb0 0x55874a2e32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.768+0000 7f22b8aef640 1 --2- 172.21.15.7:0/2490219612 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55874a2e0eb0 0x55874a2e32d0 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6062290797605dc7 server_cookie=7a4d5784ee262124 in_seq=0 out_seq=0 2024-06-10T01:26:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.768+0000 7f2293fff640 10 client.20421 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.829+0000 7f2293fff640 1 -- 172.21.15.7:0/2490219612 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55874a1d0c20 con 0x55874a2e0eb0 2024-06-10T01:26:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.829+0000 7f2293fff640 10 client.20421 handle_command_reply: tid=0 2024-06-10T01:26:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.829+0000 7f22bda5e740 1 client.20421 shutdown 2024-06-10T01:26:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.829+0000 7f22bda5e740 1 -- 172.21.15.7:0/2490219612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874a265b70 msgr2=0x55874a2dee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.829+0000 7f22bda5e740 1 --2- 172.21.15.7:0/2490219612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874a265b70 0x55874a2dee80 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f22ac004650 tx=0x7f22ac03ce00 comp rx=0 tx=0).stop 2024-06-10T01:26:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.829+0000 7f22bda5e740 1 -- 172.21.15.7:0/2490219612 shutdown_connections 2024-06-10T01:26:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.829+0000 7f22bda5e740 1 --2- 172.21.15.7:0/2490219612 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55874a2e0eb0 0x55874a2e32d0 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.829+0000 7f22bda5e740 1 --2- 172.21.15.7:0/2490219612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55874a265b70 0x55874a2dee80 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.829+0000 7f22bda5e740 1 --2- 172.21.15.7:0/2490219612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874a260a00 0x55874a26ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.829+0000 7f22bda5e740 1 --2- 172.21.15.7:0/2490219612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55874a260010 0x55874a26e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.829+0000 7f2293fff640 0 client.20421 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.829+0000 7f22bda5e740 1 -- 172.21.15.7:0/2490219612 >> 172.21.15.7:0/2490219612 conn(0x55874a25b680 msgr2=0x55874a26a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.829+0000 7f22bda5e740 1 -- 172.21.15.7:0/2490219612 shutdown_connections 2024-06-10T01:26:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.830+0000 7f22bda5e740 1 -- 172.21.15.7:0/2490219612 wait complete. 2024-06-10T01:26:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.830+0000 7f22bda5e740 20 client.20421 trim_cache size 0 max 16384 2024-06-10T01:26:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:26:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:26:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:26:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.830+0000 7f22bad53640 1 -- 172.21.15.7:0/2256561491 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f22a00a2c00 msgr2=0x7f22a00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.830+0000 7f22bad53640 1 --2- 172.21.15.7:0/2256561491 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f22a00a2c00 0x7f22a00a50c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f22ac006010 tx=0x7f22ac031400 comp rx=0 tx=0).stop 2024-06-10T01:26:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.830+0000 7f22bad53640 1 -- 172.21.15.7:0/2256561491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22b4101a30 msgr2=0x7f22b4197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.830+0000 7f22bad53640 1 --2- 172.21.15.7:0/2256561491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22b4101a30 0x7f22b4197c30 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f22a400ccb0 tx=0x7f22a400a620 comp rx=0 tx=0).stop 2024-06-10T01:26:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.830+0000 7f22bad53640 1 -- 172.21.15.7:0/2256561491 shutdown_connections 2024-06-10T01:26:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.830+0000 7f22bad53640 1 --2- 172.21.15.7:0/2256561491 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f22a00a2c00 0x7f22a00a50c0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.830+0000 7f22bad53640 1 --2- 172.21.15.7:0/2256561491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22b41035e0 0x7f22b419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.830+0000 7f22bad53640 1 --2- 172.21.15.7:0/2256561491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22b4102c20 0x7f22b4198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.830+0000 7f22bad53640 1 --2- 172.21.15.7:0/2256561491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22b4101a30 0x7f22b4197c30 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.830+0000 7f22bad53640 1 -- 172.21.15.7:0/2256561491 >> 172.21.15.7:0/2256561491 conn(0x7f22b40fcfc0 msgr2=0x7f22b4105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.830+0000 7f22bad53640 1 -- 172.21.15.7:0/2256561491 shutdown_connections 2024-06-10T01:26:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:26:48.830+0000 7f22bad53640 1 -- 172.21.15.7:0/2256561491 wait complete. 2024-06-10T01:26:49.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.504+0000 7febdd97d700 1 -- 172.21.15.7:0/4213719726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd8100b50 msgr2=0x7febd8100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:49.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.504+0000 7febdd97d700 1 --2- 172.21.15.7:0/4213719726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd8100b50 0x7febd8100fd0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7febcc0099c0 tx=0x7febcc009cd0 comp rx=0 tx=0).stop 2024-06-10T01:26:49.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.504+0000 7febdd97d700 1 -- 172.21.15.7:0/4213719726 shutdown_connections 2024-06-10T01:26:49.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.504+0000 7febdd97d700 1 --2- 172.21.15.7:0/4213719726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febd8101510 0x7febd810d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:49.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.504+0000 7febdd97d700 1 --2- 172.21.15.7:0/4213719726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febd8106c00 0x7febd8106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:49.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.504+0000 7febdd97d700 1 --2- 172.21.15.7:0/4213719726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd8100b50 0x7febd8100fd0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:49.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.504+0000 7febdd97d700 1 -- 172.21.15.7:0/4213719726 >> 172.21.15.7:0/4213719726 conn(0x7febd8077990 msgr2=0x7febd80fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:49.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.504+0000 7febdd97d700 1 -- 172.21.15.7:0/4213719726 shutdown_connections 2024-06-10T01:26:49.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.504+0000 7febdd97d700 1 -- 172.21.15.7:0/4213719726 wait complete. 2024-06-10T01:26:49.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.504+0000 7febdd97d700 1 Processor -- start 2024-06-10T01:26:49.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.505+0000 7febdd97d700 1 -- start start 2024-06-10T01:26:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.505+0000 7febdd97d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febd8100b50 0x7febd81027c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.505+0000 7febdd97d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febd8101510 0x7febd8102d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.505+0000 7febdd97d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd8106c00 0x7febd8103240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:49.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.505+0000 7febdd97d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7febd8110eb0 con 0x7febd8106c00 2024-06-10T01:26:49.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.505+0000 7febdd97d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7febd8110d30 con 0x7febd8101510 2024-06-10T01:26:49.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.505+0000 7febdd97d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7febd8111030 con 0x7febd8100b50 2024-06-10T01:26:49.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.505+0000 7febd77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd8106c00 0x7febd8103240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:49.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.505+0000 7febd77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd8106c00 0x7febd8103240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53010/0 (socket says 172.21.15.7:53010) 2024-06-10T01:26:49.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.505+0000 7febd77fe700 1 -- 172.21.15.7:0/1964002386 learned_addr learned my addr 172.21.15.7:0/1964002386 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:49.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.505+0000 7febd67fc700 1 --2- 172.21.15.7:0/1964002386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febd8101510 0x7febd8102d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:49.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.505+0000 7febd6ffd700 1 --2- 172.21.15.7:0/1964002386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febd8100b50 0x7febd81027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:49.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.505+0000 7febd77fe700 1 -- 172.21.15.7:0/1964002386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febd8100b50 msgr2=0x7febd81027c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:49.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.505+0000 7febd77fe700 1 --2- 172.21.15.7:0/1964002386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febd8100b50 0x7febd81027c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:49.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.505+0000 7febd77fe700 1 -- 172.21.15.7:0/1964002386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febd8101510 msgr2=0x7febd8102d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:49.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.505+0000 7febd77fe700 1 --2- 172.21.15.7:0/1964002386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febd8101510 0x7febd8102d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:49.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.505+0000 7febd77fe700 1 -- 172.21.15.7:0/1964002386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd81a71c0 con 0x7febd8106c00 2024-06-10T01:26:49.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.506+0000 7febd77fe700 1 --2- 172.21.15.7:0/1964002386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd8106c00 0x7febd8103240 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7febd000b9e0 tx=0x7febd000bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:49.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.506+0000 7febdc97b700 1 -- 172.21.15.7:0/1964002386 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd000d820 con 0x7febd8106c00 2024-06-10T01:26:49.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.506+0000 7febdc97b700 1 -- 172.21.15.7:0/1964002386 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febd0015420 con 0x7febd8106c00 2024-06-10T01:26:49.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.506+0000 7febdd97d700 1 -- 172.21.15.7:0/1964002386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd81a74b0 con 0x7febd8106c00 2024-06-10T01:26:49.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.506+0000 7febdc97b700 1 -- 172.21.15.7:0/1964002386 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd0014630 con 0x7febd8106c00 2024-06-10T01:26:49.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.506+0000 7febdd97d700 1 -- 172.21.15.7:0/1964002386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd81a7970 con 0x7febd8106c00 2024-06-10T01:26:49.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.507+0000 7febdd97d700 1 -- 172.21.15.7:0/1964002386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febd8069180 con 0x7febd8106c00 2024-06-10T01:26:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.508+0000 7febdc97b700 1 -- 172.21.15.7:0/1964002386 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7febd00155d0 con 0x7febd8106c00 2024-06-10T01:26:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.509+0000 7febdc97b700 1 --2- 172.21.15.7:0/1964002386 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7febc00a2c20 0x7febc00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.509+0000 7febd6ffd700 1 --2- 172.21.15.7:0/1964002386 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7febc00a2c20 0x7febc00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.509+0000 7febdc97b700 1 -- 172.21.15.7:0/1964002386 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7febd00cf170 con 0x7febd8106c00 2024-06-10T01:26:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.509+0000 7febd6ffd700 1 --2- 172.21.15.7:0/1964002386 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7febc00a2c20 0x7febc00a50e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7febc8005950 tx=0x7febc800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:49.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.512+0000 7febdc97b700 1 -- 172.21.15.7:0/1964002386 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7febd009bcf0 con 0x7febd8106c00 2024-06-10T01:26:49.548 INFO:tasks.workunit.client.0.smithi007.stdout: 10 174555 22.16 MB/sec execute 326 sec latency 18277.266 ms 2024-06-10T01:26:49.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.766+0000 7febdd97d700 1 -- 172.21.15.7:0/1964002386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febd81a7c30 con 0x7febd8106c00 2024-06-10T01:26:49.759 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:49 smithi007 ceph-mon[32398]: pgmap v1057: 129 pgs: 129 active+clean; 665 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 3.6 MiB/s wr, 195 op/s 2024-06-10T01:26:49.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.767+0000 7febdc97b700 1 -- 172.21.15.7:0/1964002386 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11858 (secure 0 0 0) 0x7febd009b5f0 con 0x7febd8106c00 2024-06-10T01:26:49.760 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:26:49.760 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":52,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:25:03.837724+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28825":{"gid":28825,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/1972426673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1972426673},{"type":"v1","addr":"172.21.15.175:6835","nonce":1972426673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24451,24434,24436]},"id":1}]} 2024-06-10T01:26:49.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.769+0000 7febdd97d700 1 -- 172.21.15.7:0/1964002386 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7febc00a2c20 msgr2=0x7febc00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:49.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.769+0000 7febdd97d700 1 --2- 172.21.15.7:0/1964002386 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7febc00a2c20 0x7febc00a50e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7febc8005950 tx=0x7febc800b410 comp rx=0 tx=0).stop 2024-06-10T01:26:49.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.769+0000 7febdd97d700 1 -- 172.21.15.7:0/1964002386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd8106c00 msgr2=0x7febd8103240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:49.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.769+0000 7febdd97d700 1 --2- 172.21.15.7:0/1964002386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd8106c00 0x7febd8103240 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7febd000b9e0 tx=0x7febd000bcf0 comp rx=0 tx=0).stop 2024-06-10T01:26:49.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.769+0000 7febdd97d700 1 -- 172.21.15.7:0/1964002386 shutdown_connections 2024-06-10T01:26:49.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.769+0000 7febdd97d700 1 --2- 172.21.15.7:0/1964002386 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7febc00a2c20 0x7febc00a50e0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:49.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.769+0000 7febdd97d700 1 --2- 172.21.15.7:0/1964002386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd8106c00 0x7febd8103240 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:49.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.769+0000 7febdd97d700 1 --2- 172.21.15.7:0/1964002386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febd8100b50 0x7febd81027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:49.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.769+0000 7febdd97d700 1 --2- 172.21.15.7:0/1964002386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febd8101510 0x7febd8102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:49.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.769+0000 7febdd97d700 1 -- 172.21.15.7:0/1964002386 >> 172.21.15.7:0/1964002386 conn(0x7febd8077990 msgr2=0x7febd80fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:49.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.769+0000 7febdd97d700 1 -- 172.21.15.7:0/1964002386 shutdown_connections 2024-06-10T01:26:49.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:49.769+0000 7febdd97d700 1 -- 172.21.15.7:0/1964002386 wait complete. 2024-06-10T01:26:49.763 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 52 2024-06-10T01:26:49.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:49 smithi078 ceph-mon[36683]: pgmap v1057: 129 pgs: 129 active+clean; 665 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 3.6 MiB/s wr, 195 op/s 2024-06-10T01:26:49.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:49 smithi175 ceph-mon[36636]: pgmap v1057: 129 pgs: 129 active+clean; 665 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 3.6 MiB/s wr, 195 op/s 2024-06-10T01:26:50.358 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db --set-id 2b2d6bd9 --release --await' 2024-06-10T01:26:50.359 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db --set-id 2b2d6bd9 --release --await 2024-06-10T01:26:50.548 INFO:tasks.workunit.client.0.smithi007.stdout: 10 174555 22.09 MB/sec execute 327 sec latency 19277.295 ms 2024-06-10T01:26:50.609 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:26:50.732 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:50 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1964002386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:26:50.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:50 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1964002386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:26:50.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:50 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1964002386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:26:51.548 INFO:tasks.workunit.client.0.smithi007.stdout: 10 174555 22.02 MB/sec execute 328 sec latency 20277.408 ms 2024-06-10T01:26:51.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:51 smithi007 ceph-mon[32398]: pgmap v1058: 129 pgs: 129 active+clean; 665 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 3.6 MiB/s wr, 187 op/s 2024-06-10T01:26:51.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:51 smithi078 ceph-mon[36683]: pgmap v1058: 129 pgs: 129 active+clean; 665 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 3.6 MiB/s wr, 187 op/s 2024-06-10T01:26:51.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:51 smithi175 ceph-mon[36636]: pgmap v1058: 129 pgs: 129 active+clean; 665 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 3.6 MiB/s wr, 187 op/s 2024-06-10T01:26:52.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.366+0000 7fe7d6062700 1 -- 172.21.15.7:0/2311685067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7d0102740 msgr2=0x7fe7d0102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:52.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.366+0000 7fe7d6062700 1 --2- 172.21.15.7:0/2311685067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7d0102740 0x7fe7d0102b20 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fe7c000b0d0 tx=0x7fe7c000b3e0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.366+0000 7fe7d6062700 1 -- 172.21.15.7:0/2311685067 shutdown_connections 2024-06-10T01:26:52.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.366+0000 7fe7d6062700 1 --2- 172.21.15.7:0/2311685067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d0107840 0x7fe7d010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.366+0000 7fe7d6062700 1 --2- 172.21.15.7:0/2311685067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7d0102740 0x7fe7d0102b20 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.366+0000 7fe7d6062700 1 --2- 172.21.15.7:0/2311685067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7d01030f0 0x7fe7d0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.366+0000 7fe7d6062700 1 -- 172.21.15.7:0/2311685067 >> 172.21.15.7:0/2311685067 conn(0x7fe7d00795f0 msgr2=0x7fe7d0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:52.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.366+0000 7fe7d6062700 1 -- 172.21.15.7:0/2311685067 shutdown_connections 2024-06-10T01:26:52.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.366+0000 7fe7d6062700 1 -- 172.21.15.7:0/2311685067 wait complete. 2024-06-10T01:26:52.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.367+0000 7fe7d6062700 1 Processor -- start 2024-06-10T01:26:52.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.367+0000 7fe7d6062700 1 -- start start 2024-06-10T01:26:52.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.367+0000 7fe7d6062700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7d0102740 0x7fe7d019ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:52.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.367+0000 7fe7d6062700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d01030f0 0x7fe7d019b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:52.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.367+0000 7fe7d6062700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7d0107840 0x7fe7d019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:52.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.367+0000 7fe7d6062700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d01127e0 con 0x7fe7d0107840 2024-06-10T01:26:52.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.367+0000 7fe7d6062700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d0112660 con 0x7fe7d0102740 2024-06-10T01:26:52.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.367+0000 7fe7d6062700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d0112960 con 0x7fe7d01030f0 2024-06-10T01:26:52.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.367+0000 7fe7cffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7d0107840 0x7fe7d019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:52.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.367+0000 7fe7cffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7d0107840 0x7fe7d019f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50326/0 (socket says 172.21.15.7:50326) 2024-06-10T01:26:52.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.367+0000 7fe7cffff700 1 -- 172.21.15.7:0/1282404239 learned_addr learned my addr 172.21.15.7:0/1282404239 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:52.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.367+0000 7fe7cf7fe700 1 --2- 172.21.15.7:0/1282404239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7d0102740 0x7fe7d019ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:52.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.367+0000 7fe7c6ffd700 1 --2- 172.21.15.7:0/1282404239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d01030f0 0x7fe7d019b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:52.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.368+0000 7fe7cffff700 1 -- 172.21.15.7:0/1282404239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d01030f0 msgr2=0x7fe7d019b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:52.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.368+0000 7fe7cffff700 1 --2- 172.21.15.7:0/1282404239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d01030f0 0x7fe7d019b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.368+0000 7fe7cffff700 1 -- 172.21.15.7:0/1282404239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7d0102740 msgr2=0x7fe7d019ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:52.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.368+0000 7fe7cffff700 1 --2- 172.21.15.7:0/1282404239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7d0102740 0x7fe7d019ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.368+0000 7fe7cffff700 1 -- 172.21.15.7:0/1282404239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d019fe40 con 0x7fe7d0107840 2024-06-10T01:26:52.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.368+0000 7fe7cffff700 1 --2- 172.21.15.7:0/1282404239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7d0107840 0x7fe7d019f700 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fe7c800b550 tx=0x7fe7c800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:52.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.368+0000 7fe7cd7fa700 1 -- 172.21.15.7:0/1282404239 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8015020 con 0x7fe7d0107840 2024-06-10T01:26:52.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.368+0000 7fe7cd7fa700 1 -- 172.21.15.7:0/1282404239 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7c8011d10 con 0x7fe7d0107840 2024-06-10T01:26:52.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.368+0000 7fe7d6062700 1 -- 172.21.15.7:0/1282404239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d01a0130 con 0x7fe7d0107840 2024-06-10T01:26:52.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.368+0000 7fe7cd7fa700 1 -- 172.21.15.7:0/1282404239 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c800f2e0 con 0x7fe7d0107840 2024-06-10T01:26:52.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.368+0000 7fe7d6062700 1 -- 172.21.15.7:0/1282404239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d01a79b0 con 0x7fe7d0107840 2024-06-10T01:26:52.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.371+0000 7fe7cd7fa700 1 -- 172.21.15.7:0/1282404239 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fe7c8018020 con 0x7fe7d0107840 2024-06-10T01:26:52.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.372+0000 7fe7cd7fa700 1 --2- 172.21.15.7:0/1282404239 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe7bc0a2c20 0x7fe7bc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:52.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.372+0000 7fe7cf7fe700 1 --2- 172.21.15.7:0/1282404239 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe7bc0a2c20 0x7fe7bc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:52.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.372+0000 7fe7cd7fa700 1 -- 172.21.15.7:0/1282404239 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fe7c80ceb20 con 0x7fe7d0107840 2024-06-10T01:26:52.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.372+0000 7fe7e08e5b80 1 Processor -- start 2024-06-10T01:26:52.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.372+0000 7fe7e08e5b80 1 -- start start 2024-06-10T01:26:52.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.372+0000 7fe7cf7fe700 1 --2- 172.21.15.7:0/1282404239 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe7bc0a2c20 0x7fe7bc0a50e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0009fd0 tx=0x7fe7c000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:52.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.373+0000 7fe7e08e5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2775c5500 0x55c2775c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:52.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.373+0000 7fe7e08e5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2775c5e20 0x55c2775bf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:52.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.373+0000 7fe7e08e5b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2775bfc50 0x55c2775c0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:52.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.373+0000 7fe7e08e5b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c2774b2060 con 0x55c2775bfc50 2024-06-10T01:26:52.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.373+0000 7fe7e08e5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c277510950 con 0x55c2775c5500 2024-06-10T01:26:52.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.373+0000 7fe7e08e5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c277478560 con 0x55c2775c5e20 2024-06-10T01:26:52.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.373+0000 7fe7cffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2775bfc50 0x55c2775c0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:52.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.373+0000 7fe7cffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2775bfc50 0x55c2775c0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50330/0 (socket says 172.21.15.7:50330) 2024-06-10T01:26:52.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.373+0000 7fe7cffff700 1 -- 172.21.15.7:0/3555775394 learned_addr learned my addr 172.21.15.7:0/3555775394 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:52.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.373+0000 7fe7c6ffd700 1 --2- 172.21.15.7:0/3555775394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2775c5e20 0x55c2775bf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:52.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.373+0000 7fe7cf7fe700 1 --2- 172.21.15.7:0/3555775394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2775c5500 0x55c2775c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:52.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.373+0000 7fe7cffff700 1 -- 172.21.15.7:0/3555775394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2775c5e20 msgr2=0x55c2775bf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:52.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.373+0000 7fe7cffff700 1 --2- 172.21.15.7:0/3555775394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2775c5e20 0x55c2775bf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.373+0000 7fe7cffff700 1 -- 172.21.15.7:0/3555775394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2775c5500 msgr2=0x55c2775c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:52.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.373+0000 7fe7cffff700 1 --2- 172.21.15.7:0/3555775394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2775c5500 0x55c2775c58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.373+0000 7fe7cffff700 1 -- 172.21.15.7:0/3555775394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c27745f9d0 con 0x55c2775bfc50 2024-06-10T01:26:52.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.373+0000 7fe7cffff700 1 --2- 172.21.15.7:0/3555775394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2775bfc50 0x55c2775c0070 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8097460 tx=0x7fe7c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4736f91bb802c5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:52.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.373+0000 7fe7c5ffb700 1 -- 172.21.15.7:0/3555775394 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c809b000 con 0x55c2775bfc50 2024-06-10T01:26:52.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.373+0000 7fe7c5ffb700 1 -- 172.21.15.7:0/3555775394 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7c809f3b0 con 0x55c2775bfc50 2024-06-10T01:26:52.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.373+0000 7fe7c5ffb700 1 -- 172.21.15.7:0/3555775394 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c80a37e0 con 0x55c2775bfc50 2024-06-10T01:26:52.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.374+0000 7fe7e08e5b80 1 -- 172.21.15.7:0/3555775394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2775bfc50 msgr2=0x55c2775c0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:52.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.374+0000 7fe7e08e5b80 1 --2- 172.21.15.7:0/3555775394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2775bfc50 0x55c2775c0070 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8097460 tx=0x7fe7c8004750 comp rx=0 tx=0).stop 2024-06-10T01:26:52.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.374+0000 7fe7e08e5b80 1 -- 172.21.15.7:0/3555775394 shutdown_connections 2024-06-10T01:26:52.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.374+0000 7fe7e08e5b80 1 --2- 172.21.15.7:0/3555775394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2775bfc50 0x55c2775c0070 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.374+0000 7fe7e08e5b80 1 --2- 172.21.15.7:0/3555775394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2775c5500 0x55c2775c58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.374+0000 7fe7e08e5b80 1 --2- 172.21.15.7:0/3555775394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2775c5e20 0x55c2775bf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.374+0000 7fe7e08e5b80 1 -- 172.21.15.7:0/3555775394 >> 172.21.15.7:0/3555775394 conn(0x55c2774ec180 msgr2=0x55c2775baa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:52.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.374+0000 7fe7e08e5b80 1 -- 172.21.15.7:0/3555775394 shutdown_connections 2024-06-10T01:26:52.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.374+0000 7fe7e08e5b80 1 -- 172.21.15.7:0/3555775394 wait complete. 2024-06-10T01:26:52.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.374+0000 7fe7e08e5b80 1 -- start start 2024-06-10T01:26:52.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.375+0000 7fe7e08e5b80 1 Processor -- start 2024-06-10T01:26:52.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.375+0000 7fe7e08e5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2775bfc50 0x55c2775cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:52.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.375+0000 7fe7e08e5b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2775c5500 0x55c2775c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:52.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.375+0000 7fe7e08e5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2775c5e20 0x55c2775c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:52.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.375+0000 7fe7e08e5b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c2774b10c0 con 0x55c2775c5500 2024-06-10T01:26:52.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.375+0000 7fe7e08e5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c27750c910 con 0x55c2775bfc50 2024-06-10T01:26:52.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.375+0000 7fe7e08e5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c2775cc800 con 0x55c2775c5e20 2024-06-10T01:26:52.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.375+0000 7fe7e08e5b80 10 client.0 _ll_register_callbacks cb 0x55c2774a66b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:26:52.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.375+0000 7fe7c6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2775c5500 0x55c2775c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:52.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.375+0000 7fe7c6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2775c5500 0x55c2775c8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50336/0 (socket says 172.21.15.7:50336) 2024-06-10T01:26:52.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.375+0000 7fe7c6ffd700 1 -- 172.21.15.7:0/4085504717 learned_addr learned my addr 172.21.15.7:0/4085504717 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:52.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.375+0000 7fe7cffff700 1 --2- 172.21.15.7:0/4085504717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2775c5e20 0x55c2775c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:52.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.375+0000 7fe7cf7fe700 1 --2- 172.21.15.7:0/4085504717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2775bfc50 0x55c2775cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:52.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.375+0000 7fe7c6ffd700 1 -- 172.21.15.7:0/4085504717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2775c5e20 msgr2=0x55c2775c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:52.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.375+0000 7fe7c6ffd700 1 --2- 172.21.15.7:0/4085504717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2775c5e20 0x55c2775c93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.375+0000 7fe7c6ffd700 1 -- 172.21.15.7:0/4085504717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2775bfc50 msgr2=0x55c2775cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:52.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.376+0000 7fe7c6ffd700 1 --2- 172.21.15.7:0/4085504717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2775bfc50 0x55c2775cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.376+0000 7fe7c6ffd700 1 -- 172.21.15.7:0/4085504717 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2774d6850 con 0x55c2775c5500 2024-06-10T01:26:52.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.376+0000 7fe7c6ffd700 1 --2- 172.21.15.7:0/4085504717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2775c5500 0x55c2775c8eb0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800bf20 tx=0x7fe7b8012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c28a0cdb607f2057 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:52.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.376+0000 7fe7aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:26:52.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.376+0000 7fe7aeffd700 1 -- 172.21.15.7:0/4085504717 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b80107e0 con 0x55c2775c5500 2024-06-10T01:26:52.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.376+0000 7fe7aeffd700 1 -- 172.21.15.7:0/4085504717 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fe7b8010980 con 0x55c2775c5500 2024-06-10T01:26:52.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.376+0000 7fe7e08e5b80 1 -- 172.21.15.7:0/4085504717 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7c809f3b0 con 0x55c2775c5500 2024-06-10T01:26:52.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.376+0000 7fe7aeffd700 1 -- 172.21.15.7:0/4085504717 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fe7b8010b80 con 0x55c2775c5500 2024-06-10T01:26:52.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.376+0000 7fe7e08e5b80 10 client.20439 fetch_fsmap learned FSMap version 52 2024-06-10T01:26:52.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.376+0000 7fe7e08e5b80 1 -- 172.21.15.7:0/4085504717 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55c2775d7040 con 0x55c2775c5500 2024-06-10T01:26:52.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.376+0000 7fe7aeffd700 1 -- 172.21.15.7:0/4085504717 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7c809f3b0 con 0x55c2775c5500 2024-06-10T01:26:52.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.377+0000 7fe7aeffd700 1 -- 172.21.15.7:0/4085504717 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7fe7b8019070 con 0x55c2775c5500 2024-06-10T01:26:52.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.377+0000 7fe7e08e5b80 10 client.20439 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:26:52.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.377+0000 7fe7e08e5b80 10 client.20439 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:26:52.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.377+0000 7fe7e08e5b80 20 client.20439 populate_metadata read hostname 'smithi007' 2024-06-10T01:26:52.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.377+0000 7fe7e08e5b80 1 --2- 172.21.15.7:0/4085504717 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c2775d9680 0x55c2775dba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:52.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.377+0000 7fe7e08e5b80 4 client.20439 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:26:52.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.377+0000 7fe7e08e5b80 1 -- 172.21.15.7:0/4085504717 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c27751de50 con 0x55c2775d9680 2024-06-10T01:26:52.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.377+0000 7fe7cf7fe700 1 --2- 172.21.15.7:0/4085504717 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c2775d9680 0x55c2775dba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:52.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.377+0000 7fe7cf7fe700 1 --2- 172.21.15.7:0/4085504717 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c2775d9680 0x55c2775dba70 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec3151b18313b99 server_cookie=a0a43d377acf88dd in_seq=0 out_seq=0 2024-06-10T01:26:52.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.377+0000 7fe7aeffd700 10 client.20439 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:52.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.378+0000 7fe7aeffd700 1 -- 172.21.15.7:0/4085504717 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c27751de50 con 0x55c2775d9680 2024-06-10T01:26:52.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.378+0000 7fe7aeffd700 10 client.20439 handle_command_reply: tid=0 2024-06-10T01:26:52.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.378+0000 7fe7e08e5b80 1 client.20439 shutdown 2024-06-10T01:26:52.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.378+0000 7fe7e08e5b80 1 -- 172.21.15.7:0/4085504717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2775c5500 msgr2=0x55c2775c8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:52.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.378+0000 7fe7e08e5b80 1 --2- 172.21.15.7:0/4085504717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2775c5500 0x55c2775c8eb0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800bf20 tx=0x7fe7b8012730 comp rx=0 tx=0).stop 2024-06-10T01:26:52.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.378+0000 7fe7e08e5b80 1 -- 172.21.15.7:0/4085504717 shutdown_connections 2024-06-10T01:26:52.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.378+0000 7fe7e08e5b80 1 --2- 172.21.15.7:0/4085504717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2775c5e20 0x55c2775c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.378+0000 7fe7e08e5b80 1 --2- 172.21.15.7:0/4085504717 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c2775d9680 0x55c2775dba70 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.378+0000 7fe7e08e5b80 1 --2- 172.21.15.7:0/4085504717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2775bfc50 0x55c2775cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.378+0000 7fe7e08e5b80 1 --2- 172.21.15.7:0/4085504717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2775c5500 0x55c2775c8eb0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.378+0000 7fe7aeffd700 0 client.20439 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:52.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.378+0000 7fe7e08e5b80 1 -- 172.21.15.7:0/4085504717 >> 172.21.15.7:0/4085504717 conn(0x55c2774ec180 msgr2=0x55c2775b9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:52.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.379+0000 7fe7e08e5b80 1 -- 172.21.15.7:0/4085504717 shutdown_connections 2024-06-10T01:26:52.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.379+0000 7fe7e08e5b80 1 -- 172.21.15.7:0/4085504717 wait complete. 2024-06-10T01:26:52.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.379+0000 7fe7e08e5b80 20 client.20439 trim_cache size 0 max 16384 2024-06-10T01:26:52.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.399+0000 7fe7e08e5b80 1 Processor -- start 2024-06-10T01:26:52.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.399+0000 7fe7e08e5b80 1 -- start start 2024-06-10T01:26:52.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.399+0000 7fe7e08e5b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2775bfc50 0x55c2775d5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:52.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.399+0000 7fe7e08e5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2775c5500 0x55c2775ba330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:52.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.399+0000 7fe7e08e5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2775c5e20 0x55c2775ba870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:52.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.399+0000 7fe7e08e5b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d01b2e70 con 0x55c2775bfc50 2024-06-10T01:26:52.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.399+0000 7fe7e08e5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d01b0700 con 0x55c2775c5500 2024-06-10T01:26:52.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.399+0000 7fe7e08e5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d01ae550 con 0x55c2775c5e20 2024-06-10T01:26:52.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.399+0000 7fe7cf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2775bfc50 0x55c2775d5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:52.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.399+0000 7fe7cf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2775bfc50 0x55c2775d5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50352/0 (socket says 172.21.15.7:50352) 2024-06-10T01:26:52.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.399+0000 7fe7cf7fe700 1 -- 172.21.15.7:0/3760881846 learned_addr learned my addr 172.21.15.7:0/3760881846 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:52.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.400+0000 7fe7cffff700 1 --2- 172.21.15.7:0/3760881846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2775c5e20 0x55c2775ba870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:52.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.400+0000 7fe7c6ffd700 1 --2- 172.21.15.7:0/3760881846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2775c5500 0x55c2775ba330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:52.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.400+0000 7fe7cf7fe700 1 -- 172.21.15.7:0/3760881846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2775c5e20 msgr2=0x55c2775ba870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:52.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.400+0000 7fe7cf7fe700 1 --2- 172.21.15.7:0/3760881846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2775c5e20 0x55c2775ba870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.400+0000 7fe7cf7fe700 1 -- 172.21.15.7:0/3760881846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2775c5500 msgr2=0x55c2775ba330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:52.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.400+0000 7fe7cf7fe700 1 --2- 172.21.15.7:0/3760881846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2775c5500 0x55c2775ba330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.400+0000 7fe7cf7fe700 1 -- 172.21.15.7:0/3760881846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2775c8c80 con 0x55c2775bfc50 2024-06-10T01:26:52.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.400+0000 7fe7cf7fe700 1 --2- 172.21.15.7:0/3760881846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2775bfc50 0x55c2775d5950 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fe7c000b010 tx=0x7fe7c001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b55eedea050b05 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:52.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.400+0000 7fe7c57fa700 1 -- 172.21.15.7:0/3760881846 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0007ad0 con 0x55c2775bfc50 2024-06-10T01:26:52.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.400+0000 7fe7c57fa700 1 -- 172.21.15.7:0/3760881846 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7c0039070 con 0x55c2775bfc50 2024-06-10T01:26:52.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.400+0000 7fe7c57fa700 1 -- 172.21.15.7:0/3760881846 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c00052a0 con 0x55c2775bfc50 2024-06-10T01:26:52.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.401+0000 7fe7e08e5b80 1 -- 172.21.15.7:0/3760881846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2775bfc50 msgr2=0x55c2775d5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:52.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.401+0000 7fe7e08e5b80 1 --2- 172.21.15.7:0/3760881846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2775bfc50 0x55c2775d5950 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fe7c000b010 tx=0x7fe7c001d000 comp rx=0 tx=0).stop 2024-06-10T01:26:52.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.401+0000 7fe7e08e5b80 1 -- 172.21.15.7:0/3760881846 shutdown_connections 2024-06-10T01:26:52.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.401+0000 7fe7e08e5b80 1 --2- 172.21.15.7:0/3760881846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2775c5e20 0x55c2775ba870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.401+0000 7fe7e08e5b80 1 --2- 172.21.15.7:0/3760881846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2775bfc50 0x55c2775d5950 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.401+0000 7fe7e08e5b80 1 --2- 172.21.15.7:0/3760881846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2775c5500 0x55c2775ba330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.401+0000 7fe7e08e5b80 1 -- 172.21.15.7:0/3760881846 >> 172.21.15.7:0/3760881846 conn(0x55c2774ec180 msgr2=0x55c2775be700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:52.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.401+0000 7fe7e08e5b80 1 -- 172.21.15.7:0/3760881846 shutdown_connections 2024-06-10T01:26:52.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.401+0000 7fe7e08e5b80 1 -- 172.21.15.7:0/3760881846 wait complete. 2024-06-10T01:26:52.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.401+0000 7fe7e08e5b80 1 -- start start 2024-06-10T01:26:52.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.401+0000 7fe7e08e5b80 1 Processor -- start 2024-06-10T01:26:52.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.402+0000 7fe7e08e5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2775bfc50 0x55c2775d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:52.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.402+0000 7fe7e08e5b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2775c5500 0x55c2775cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:52.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.402+0000 7fe7e08e5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2775c5e20 0x55c2775cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:52.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.402+0000 7fe7e08e5b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d01aebc0 con 0x55c2775c5500 2024-06-10T01:26:52.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.402+0000 7fe7e08e5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d01b0030 con 0x55c2775bfc50 2024-06-10T01:26:52.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.402+0000 7fe7e08e5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d01af290 con 0x55c2775c5e20 2024-06-10T01:26:52.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.402+0000 7fe7e08e5b80 10 client.0 _ll_register_callbacks cb 0x55c2775c8320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:26:52.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.402+0000 7fe7c6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2775c5500 0x55c2775cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:52.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.402+0000 7fe7c6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2775c5500 0x55c2775cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50358/0 (socket says 172.21.15.7:50358) 2024-06-10T01:26:52.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.402+0000 7fe7c6ffd700 1 -- 172.21.15.7:0/1969280021 learned_addr learned my addr 172.21.15.7:0/1969280021 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:52.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.402+0000 7fe7cf7fe700 1 --2- 172.21.15.7:0/1969280021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2775bfc50 0x55c2775d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:52.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.402+0000 7fe7cffff700 1 --2- 172.21.15.7:0/1969280021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2775c5e20 0x55c2775cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:52.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.402+0000 7fe7c6ffd700 1 -- 172.21.15.7:0/1969280021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2775c5e20 msgr2=0x55c2775cefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:52.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.402+0000 7fe7c6ffd700 1 --2- 172.21.15.7:0/1969280021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2775c5e20 0x55c2775cefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.402+0000 7fe7c6ffd700 1 -- 172.21.15.7:0/1969280021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2775bfc50 msgr2=0x55c2775d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:52.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.402+0000 7fe7c6ffd700 1 --2- 172.21.15.7:0/1969280021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2775bfc50 0x55c2775d2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.402+0000 7fe7c6ffd700 1 -- 172.21.15.7:0/1969280021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2775cf710 con 0x55c2775c5500 2024-06-10T01:26:52.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.402+0000 7fe7cffff700 1 --2- 172.21.15.7:0/1969280021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2775c5e20 0x55c2775cefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:26:52.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.402+0000 7fe7c6ffd700 1 --2- 172.21.15.7:0/1969280021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2775c5500 0x55c2775cea90 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fe7b801a720 tx=0x7fe7b8010890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a96cca77a65d03 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:52.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.402+0000 7fe7aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:26:52.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.402+0000 7fe7aeffd700 1 -- 172.21.15.7:0/1969280021 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8012a80 con 0x55c2775c5500 2024-06-10T01:26:52.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.402+0000 7fe7aeffd700 1 -- 172.21.15.7:0/1969280021 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fe7b8005250 con 0x55c2775c5500 2024-06-10T01:26:52.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.402+0000 7fe7e08e5b80 1 -- 172.21.15.7:0/1969280021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7c0039070 con 0x55c2775c5500 2024-06-10T01:26:52.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.403+0000 7fe7aeffd700 1 -- 172.21.15.7:0/1969280021 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fe7b8036050 con 0x55c2775c5500 2024-06-10T01:26:52.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.403+0000 7fe7e08e5b80 10 client.20445 fetch_fsmap learned FSMap version 52 2024-06-10T01:26:52.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.403+0000 7fe7e08e5b80 1 -- 172.21.15.7:0/1969280021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55c277645320 con 0x55c2775c5500 2024-06-10T01:26:52.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.403+0000 7fe7aeffd700 1 -- 172.21.15.7:0/1969280021 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7c0039070 con 0x55c2775c5500 2024-06-10T01:26:52.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.403+0000 7fe7aeffd700 1 -- 172.21.15.7:0/1969280021 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7fe7b8019050 con 0x55c2775c5500 2024-06-10T01:26:52.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.403+0000 7fe7e08e5b80 10 client.20445 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:26:52.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.403+0000 7fe7e08e5b80 10 client.20445 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:26:52.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.403+0000 7fe7e08e5b80 20 client.20445 populate_metadata read hostname 'smithi007' 2024-06-10T01:26:52.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.403+0000 7fe7e08e5b80 1 --2- 172.21.15.7:0/1969280021 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c277645ab0 0x55c277647ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:52.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.403+0000 7fe7e08e5b80 4 client.20445 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2b2d6bd9", "release": true, "await": true}] 2024-06-10T01:26:52.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.403+0000 7fe7e08e5b80 1 -- 172.21.15.7:0/1969280021 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2b2d6bd9", "release": true, "await": true}) -- 0x55c2775183d0 con 0x55c277645ab0 2024-06-10T01:26:52.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.404+0000 7fe7cf7fe700 1 --2- 172.21.15.7:0/1969280021 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c277645ab0 0x55c277647ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:52.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.404+0000 7fe7aeffd700 10 client.20445 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:52.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.404+0000 7fe7cf7fe700 1 --2- 172.21.15.7:0/1969280021 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c277645ab0 0x55c277647ed0 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a7f79e3c439f946 server_cookie=a2ea3ca774c1f338 in_seq=0 out_seq=0 2024-06-10T01:26:52.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.461+0000 7fe7aeffd700 1 -- 172.21.15.7:0/1969280021 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c2775183d0 con 0x55c277645ab0 2024-06-10T01:26:52.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.461+0000 7fe7aeffd700 10 client.20445 handle_command_reply: tid=0 2024-06-10T01:26:52.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.461+0000 7fe7e08e5b80 1 client.20445 shutdown 2024-06-10T01:26:52.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.461+0000 7fe7e08e5b80 1 -- 172.21.15.7:0/1969280021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2775c5500 msgr2=0x55c2775cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:52.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.461+0000 7fe7e08e5b80 1 --2- 172.21.15.7:0/1969280021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2775c5500 0x55c2775cea90 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fe7b801a720 tx=0x7fe7b8010890 comp rx=0 tx=0).stop 2024-06-10T01:26:52.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.461+0000 7fe7e08e5b80 1 -- 172.21.15.7:0/1969280021 shutdown_connections 2024-06-10T01:26:52.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.461+0000 7fe7e08e5b80 1 --2- 172.21.15.7:0/1969280021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c2775c5e20 0x55c2775cefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.461+0000 7fe7e08e5b80 1 --2- 172.21.15.7:0/1969280021 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c277645ab0 0x55c277647ed0 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.461+0000 7fe7e08e5b80 1 --2- 172.21.15.7:0/1969280021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2775bfc50 0x55c2775d2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.461+0000 7fe7e08e5b80 1 --2- 172.21.15.7:0/1969280021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2775c5500 0x55c2775cea90 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.461+0000 7fe7e08e5b80 1 -- 172.21.15.7:0/1969280021 >> 172.21.15.7:0/1969280021 conn(0x55c2774ec180 msgr2=0x55c2775bef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:52.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.461+0000 7fe7aeffd700 0 client.20445 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:52.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.462+0000 7fe7e08e5b80 1 -- 172.21.15.7:0/1969280021 shutdown_connections 2024-06-10T01:26:52.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.462+0000 7fe7e08e5b80 1 -- 172.21.15.7:0/1969280021 wait complete. 2024-06-10T01:26:52.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.462+0000 7fe7e08e5b80 20 client.20445 trim_cache size 0 max 16384 2024-06-10T01:26:52.457 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:26:52.457 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 52, 2024-06-10T01:26:52.457 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:26:52.457 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 68, 2024-06-10T01:26:52.457 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:26:52.457 INFO:teuthology.orchestra.run.smithi007.stdout: "2b2d6bd9": { 2024-06-10T01:26:52.457 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 68, 2024-06-10T01:26:52.457 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:26:52.457 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:26:52.457 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:26:52.458 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:26:52.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:26:52.458 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:26:52.458 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 142.0, 2024-06-10T01:26:52.458 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:26:52.458 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:26:52.458 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:26:52.458 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:26:52.458 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:26:52.458 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:26:52.458 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:26:52.458 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:26:52.459 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:26:52.459 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:26:52.459 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:26:52.459 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:26:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.463+0000 7fe7d6062700 1 -- 172.21.15.7:0/1282404239 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe7bc0a2c20 msgr2=0x7fe7bc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:52.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.463+0000 7fe7d6062700 1 --2- 172.21.15.7:0/1282404239 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe7bc0a2c20 0x7fe7bc0a50e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0009fd0 tx=0x7fe7c000b040 comp rx=0 tx=0).stop 2024-06-10T01:26:52.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.463+0000 7fe7d6062700 1 -- 172.21.15.7:0/1282404239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7d0107840 msgr2=0x7fe7d019f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:52.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.463+0000 7fe7d6062700 1 --2- 172.21.15.7:0/1282404239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7d0107840 0x7fe7d019f700 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fe7c800b550 tx=0x7fe7c800b860 comp rx=0 tx=0).stop 2024-06-10T01:26:52.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.463+0000 7fe7d6062700 1 -- 172.21.15.7:0/1282404239 shutdown_connections 2024-06-10T01:26:52.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.463+0000 7fe7d6062700 1 --2- 172.21.15.7:0/1282404239 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe7bc0a2c20 0x7fe7bc0a50e0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.463+0000 7fe7d6062700 1 --2- 172.21.15.7:0/1282404239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7d0107840 0x7fe7d019f700 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.463+0000 7fe7d6062700 1 --2- 172.21.15.7:0/1282404239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7d0102740 0x7fe7d019ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.463+0000 7fe7d6062700 1 --2- 172.21.15.7:0/1282404239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d01030f0 0x7fe7d019b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:52.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.463+0000 7fe7d6062700 1 -- 172.21.15.7:0/1282404239 >> 172.21.15.7:0/1282404239 conn(0x7fe7d00795f0 msgr2=0x7fe7d0100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:52.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.463+0000 7fe7d6062700 1 -- 172.21.15.7:0/1282404239 shutdown_connections 2024-06-10T01:26:52.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:52.463+0000 7fe7d6062700 1 -- 172.21.15.7:0/1282404239 wait complete. 2024-06-10T01:26:52.548 INFO:tasks.workunit.client.0.smithi007.stdout: 10 174629 21.97 MB/sec execute 329 sec latency 21161.515 ms 2024-06-10T01:26:53.211 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:26:53.453 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2b2d6bd9', total seconds elapsed: 27.8 2024-06-10T01:26:53.548 INFO:tasks.workunit.client.0.smithi007.stdout: 10 175347 22.01 MB/sec execute 330 sec latency 60.728 ms 2024-06-10T01:26:53.549 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:26:53.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:53 smithi078 ceph-mon[36683]: pgmap v1059: 129 pgs: 129 active+clean; 665 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 3.6 MiB/s wr, 162 op/s 2024-06-10T01:26:53.845 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:53 smithi007 ceph-mon[32398]: pgmap v1059: 129 pgs: 129 active+clean; 665 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 3.6 MiB/s wr, 162 op/s 2024-06-10T01:26:53.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:53 smithi175 ceph-mon[36636]: pgmap v1059: 129 pgs: 129 active+clean; 665 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 3.6 MiB/s wr, 162 op/s 2024-06-10T01:26:54.548 INFO:tasks.workunit.client.0.smithi007.stdout: 10 175974 22.04 MB/sec execute 331 sec latency 65.355 ms 2024-06-10T01:26:55.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.415+0000 7fe5f3018700 1 -- 172.21.15.7:0/1349453953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5ec102530 msgr2=0x7fe5ec1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:55.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.415+0000 7fe5f3018700 1 --2- 172.21.15.7:0/1349453953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5ec102530 0x7fe5ec1029b0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80099c0 tx=0x7fe5d8009cd0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.415+0000 7fe5f3018700 1 -- 172.21.15.7:0/1349453953 shutdown_connections 2024-06-10T01:26:55.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.415+0000 7fe5f3018700 1 --2- 172.21.15.7:0/1349453953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5ec102ef0 0x7fe5ec10f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.415+0000 7fe5f3018700 1 --2- 172.21.15.7:0/1349453953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5ec1085d0 0x7fe5ec1089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.415+0000 7fe5f3018700 1 --2- 172.21.15.7:0/1349453953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5ec102530 0x7fe5ec1029b0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.415+0000 7fe5f3018700 1 -- 172.21.15.7:0/1349453953 >> 172.21.15.7:0/1349453953 conn(0x7fe5ec079640 msgr2=0x7fe5ec079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:55.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.415+0000 7fe5f3018700 1 -- 172.21.15.7:0/1349453953 shutdown_connections 2024-06-10T01:26:55.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.415+0000 7fe5f3018700 1 -- 172.21.15.7:0/1349453953 wait complete. 2024-06-10T01:26:55.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.416+0000 7fe5f3018700 1 Processor -- start 2024-06-10T01:26:55.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.416+0000 7fe5f3018700 1 -- start start 2024-06-10T01:26:55.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.416+0000 7fe5f3018700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5ec102530 0x7fe5ec19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:55.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.416+0000 7fe5f3018700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5ec102ef0 0x7fe5ec19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:55.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.416+0000 7fe5f3018700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5ec1085d0 0x7fe5ec19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:55.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.416+0000 7fe5f3018700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ec075e80 con 0x7fe5ec102530 2024-06-10T01:26:55.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.416+0000 7fe5f3018700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ec075d00 con 0x7fe5ec102ef0 2024-06-10T01:26:55.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.416+0000 7fe5f3018700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ec076000 con 0x7fe5ec1085d0 2024-06-10T01:26:55.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.416+0000 7fe5f0db4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5ec102530 0x7fe5ec19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:55.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.416+0000 7fe5f0db4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5ec102530 0x7fe5ec19adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50366/0 (socket says 172.21.15.7:50366) 2024-06-10T01:26:55.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.416+0000 7fe5f0db4700 1 -- 172.21.15.7:0/2551899709 learned_addr learned my addr 172.21.15.7:0/2551899709 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:55.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.416+0000 7fe5ebfff700 1 --2- 172.21.15.7:0/2551899709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5ec102ef0 0x7fe5ec19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:55.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.416+0000 7fe5f15b5700 1 --2- 172.21.15.7:0/2551899709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5ec1085d0 0x7fe5ec19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:55.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.416+0000 7fe5ebfff700 1 -- 172.21.15.7:0/2551899709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5ec1085d0 msgr2=0x7fe5ec19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:55.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.416+0000 7fe5ebfff700 1 --2- 172.21.15.7:0/2551899709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5ec1085d0 0x7fe5ec19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.417+0000 7fe5ebfff700 1 -- 172.21.15.7:0/2551899709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5ec102530 msgr2=0x7fe5ec19adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:55.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.417+0000 7fe5ebfff700 1 --2- 172.21.15.7:0/2551899709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5ec102530 0x7fe5ec19adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.417+0000 7fe5ebfff700 1 -- 172.21.15.7:0/2551899709 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5ec19fdc0 con 0x7fe5ec102ef0 2024-06-10T01:26:55.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.417+0000 7fe5f0db4700 1 --2- 172.21.15.7:0/2551899709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5ec102530 0x7fe5ec19adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:26:55.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.417+0000 7fe5ebfff700 1 --2- 172.21.15.7:0/2551899709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5ec102ef0 0x7fe5ec19b2f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80098e0 tx=0x7fe5d8005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:55.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.417+0000 7fe5e9ffb700 1 -- 172.21.15.7:0/2551899709 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8027070 con 0x7fe5ec102ef0 2024-06-10T01:26:55.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.417+0000 7fe5f3018700 1 -- 172.21.15.7:0/2551899709 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5ec1a0050 con 0x7fe5ec102ef0 2024-06-10T01:26:55.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.417+0000 7fe5e9ffb700 1 -- 172.21.15.7:0/2551899709 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d8003680 con 0x7fe5ec102ef0 2024-06-10T01:26:55.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.417+0000 7fe5e9ffb700 1 -- 172.21.15.7:0/2551899709 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8022e10 con 0x7fe5ec102ef0 2024-06-10T01:26:55.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.417+0000 7fe5f3018700 1 -- 172.21.15.7:0/2551899709 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5ec1a7930 con 0x7fe5ec102ef0 2024-06-10T01:26:55.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.421+0000 7fe5e9ffb700 1 -- 172.21.15.7:0/2551899709 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fe5d802a070 con 0x7fe5ec102ef0 2024-06-10T01:26:55.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.422+0000 7fe5e9ffb700 1 --2- 172.21.15.7:0/2551899709 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe5dc0a2c20 0x7fe5dc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:55.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.422+0000 7fe5f0db4700 1 --2- 172.21.15.7:0/2551899709 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe5dc0a2c20 0x7fe5dc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:55.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.422+0000 7fe5e9ffb700 1 -- 172.21.15.7:0/2551899709 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fe5d80cfd90 con 0x7fe5ec102ef0 2024-06-10T01:26:55.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.422+0000 7fe5f0db4700 1 --2- 172.21.15.7:0/2551899709 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe5dc0a2c20 0x7fe5dc0a50e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0011550 tx=0x7fe5e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:55.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.423+0000 7fe5fd89bb80 1 Processor -- start 2024-06-10T01:26:55.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.423+0000 7fe5fd89bb80 1 -- start start 2024-06-10T01:26:55.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.423+0000 7fe5fd89bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a69795640 0x560a69795a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:55.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.423+0000 7fe5fd89bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a69795f60 0x560a6978f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:55.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.423+0000 7fe5fd89bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6978ff20 0x560a69790340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:55.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.423+0000 7fe5fd89bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560a696e1810 con 0x560a69795640 2024-06-10T01:26:55.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.423+0000 7fe5fd89bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560a69689720 con 0x560a69795f60 2024-06-10T01:26:55.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.423+0000 7fe5fd89bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560a696dd7d0 con 0x560a6978ff20 2024-06-10T01:26:55.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.423+0000 7fe5f0db4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a69795640 0x560a69795a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:55.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.423+0000 7fe5f0db4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a69795640 0x560a69795a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50376/0 (socket says 172.21.15.7:50376) 2024-06-10T01:26:55.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.423+0000 7fe5f0db4700 1 -- 172.21.15.7:0/1249510320 learned_addr learned my addr 172.21.15.7:0/1249510320 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:55.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.423+0000 7fe5f15b5700 1 --2- 172.21.15.7:0/1249510320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6978ff20 0x560a69790340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:55.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.423+0000 7fe5ebfff700 1 --2- 172.21.15.7:0/1249510320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a69795f60 0x560a6978f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:55.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.424+0000 7fe5f15b5700 1 -- 172.21.15.7:0/1249510320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a69795f60 msgr2=0x560a6978f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:55.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.424+0000 7fe5f15b5700 1 --2- 172.21.15.7:0/1249510320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a69795f60 0x560a6978f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.424+0000 7fe5f15b5700 1 -- 172.21.15.7:0/1249510320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a69795640 msgr2=0x560a69795a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:55.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.424+0000 7fe5f15b5700 1 --2- 172.21.15.7:0/1249510320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a69795640 0x560a69795a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.424+0000 7fe5f15b5700 1 -- 172.21.15.7:0/1249510320 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a6967ff50 con 0x560a6978ff20 2024-06-10T01:26:55.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.424+0000 7fe5ebfff700 1 --2- 172.21.15.7:0/1249510320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a69795f60 0x560a6978f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:26:55.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.424+0000 7fe5f0db4700 1 --2- 172.21.15.7:0/1249510320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a69795640 0x560a69795a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:26:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.424+0000 7fe5f15b5700 1 --2- 172.21.15.7:0/1249510320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6978ff20 0x560a69790340 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4011580 tx=0x7fe5e4009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9d8edc73a649dfe server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.424+0000 7fe5d6ffd700 1 -- 172.21.15.7:0/1249510320 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e4019070 con 0x560a6978ff20 2024-06-10T01:26:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.424+0000 7fe5d6ffd700 1 -- 172.21.15.7:0/1249510320 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5e4011e80 con 0x560a6978ff20 2024-06-10T01:26:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.424+0000 7fe5d6ffd700 1 -- 172.21.15.7:0/1249510320 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e4007500 con 0x560a6978ff20 2024-06-10T01:26:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.425+0000 7fe5fd89bb80 1 -- 172.21.15.7:0/1249510320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6978ff20 msgr2=0x560a69790340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.425+0000 7fe5fd89bb80 1 --2- 172.21.15.7:0/1249510320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6978ff20 0x560a69790340 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4011580 tx=0x7fe5e4009920 comp rx=0 tx=0).stop 2024-06-10T01:26:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.425+0000 7fe5fd89bb80 1 -- 172.21.15.7:0/1249510320 shutdown_connections 2024-06-10T01:26:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.425+0000 7fe5fd89bb80 1 --2- 172.21.15.7:0/1249510320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6978ff20 0x560a69790340 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.425+0000 7fe5fd89bb80 1 --2- 172.21.15.7:0/1249510320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a69795640 0x560a69795a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.425+0000 7fe5fd89bb80 1 --2- 172.21.15.7:0/1249510320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a69795f60 0x560a6978f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.425+0000 7fe5fd89bb80 1 -- 172.21.15.7:0/1249510320 >> 172.21.15.7:0/1249510320 conn(0x560a696bacc0 msgr2=0x560a6978a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:55.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.425+0000 7fe5fd89bb80 1 -- 172.21.15.7:0/1249510320 shutdown_connections 2024-06-10T01:26:55.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.425+0000 7fe5fd89bb80 1 -- 172.21.15.7:0/1249510320 wait complete. 2024-06-10T01:26:55.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.426+0000 7fe5fd89bb80 1 -- start start 2024-06-10T01:26:55.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.426+0000 7fe5fd89bb80 1 Processor -- start 2024-06-10T01:26:55.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.427+0000 7fe5fd89bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6978ff20 0x560a6979bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:55.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.427+0000 7fe5fd89bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a69795640 0x560a69798fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:55.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.427+0000 7fe5fd89bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a69795f60 0x560a69799520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:55.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.427+0000 7fe5fd89bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560a6979c9d0 con 0x560a69795f60 2024-06-10T01:26:55.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.427+0000 7fe5fd89bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560a69681bf0 con 0x560a69795640 2024-06-10T01:26:55.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.427+0000 7fe5fd89bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560a6979cb50 con 0x560a6978ff20 2024-06-10T01:26:55.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.427+0000 7fe5fd89bb80 10 client.0 _ll_register_callbacks cb 0x560a6967b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:26:55.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.427+0000 7fe5f15b5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a69795f60 0x560a69799520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:55.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.427+0000 7fe5f0db4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6978ff20 0x560a6979bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:55.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.427+0000 7fe5ebfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a69795640 0x560a69798fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:55.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.427+0000 7fe5f0db4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6978ff20 0x560a6979bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:60124/0 (socket says 172.21.15.7:60124) 2024-06-10T01:26:55.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.427+0000 7fe5f0db4700 1 -- 172.21.15.7:0/3570842481 learned_addr learned my addr 172.21.15.7:0/3570842481 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:55.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.427+0000 7fe5f0db4700 1 -- 172.21.15.7:0/3570842481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a69795640 msgr2=0x560a69798fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:55.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.427+0000 7fe5f0db4700 1 --2- 172.21.15.7:0/3570842481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a69795640 0x560a69798fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.427+0000 7fe5f0db4700 1 -- 172.21.15.7:0/3570842481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a69795f60 msgr2=0x560a69799520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:55.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.427+0000 7fe5f0db4700 1 --2- 172.21.15.7:0/3570842481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a69795f60 0x560a69799520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.427+0000 7fe5f0db4700 1 -- 172.21.15.7:0/3570842481 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a696308d0 con 0x560a6978ff20 2024-06-10T01:26:55.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.427+0000 7fe5ebfff700 1 --2- 172.21.15.7:0/3570842481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a69795640 0x560a69798fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:26:55.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.427+0000 7fe5f15b5700 1 --2- 172.21.15.7:0/3570842481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a69795f60 0x560a69799520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:26:55.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.428+0000 7fe5f0db4700 1 --2- 172.21.15.7:0/3570842481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6978ff20 0x560a6979bce0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0011ca0 tx=0x7fe5e00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=630fec0cc8664ef8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:55.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.428+0000 7fe5cbfff700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:26:55.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.428+0000 7fe5cbfff700 1 -- 172.21.15.7:0/3570842481 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e002b060 con 0x560a6978ff20 2024-06-10T01:26:55.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.428+0000 7fe5cbfff700 1 -- 172.21.15.7:0/3570842481 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fe5e001dd10 con 0x560a6978ff20 2024-06-10T01:26:55.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.428+0000 7fe5fd89bb80 1 -- 172.21.15.7:0/3570842481 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560a696eed10 con 0x560a6978ff20 2024-06-10T01:26:55.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.428+0000 7fe5cbfff700 1 -- 172.21.15.7:0/3570842481 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fe5e0032050 con 0x560a6978ff20 2024-06-10T01:26:55.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.428+0000 7fe5fd89bb80 10 client.29596 fetch_fsmap learned FSMap version 52 2024-06-10T01:26:55.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.428+0000 7fe5fd89bb80 1 -- 172.21.15.7:0/3570842481 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x560a697afb50 con 0x560a6978ff20 2024-06-10T01:26:55.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.428+0000 7fe5cbfff700 1 -- 172.21.15.7:0/3570842481 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560a696eed10 con 0x560a6978ff20 2024-06-10T01:26:55.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.429+0000 7fe5cbfff700 1 -- 172.21.15.7:0/3570842481 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7fe5e001c500 con 0x560a6978ff20 2024-06-10T01:26:55.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.429+0000 7fe5fd89bb80 10 client.29596 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:26:55.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.429+0000 7fe5fd89bb80 10 client.29596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:26:55.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.429+0000 7fe5fd89bb80 20 client.29596 populate_metadata read hostname 'smithi007' 2024-06-10T01:26:55.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.429+0000 7fe5fd89bb80 1 --2- 172.21.15.7:0/3570842481 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560a697b2190 0x560a697b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:55.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.429+0000 7fe5fd89bb80 4 client.29596 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:26:55.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.429+0000 7fe5fd89bb80 1 -- 172.21.15.7:0/3570842481 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a696e9290 con 0x560a697b2190 2024-06-10T01:26:55.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.429+0000 7fe5ebfff700 1 --2- 172.21.15.7:0/3570842481 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560a697b2190 0x560a697b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:55.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.430+0000 7fe5ebfff700 1 --2- 172.21.15.7:0/3570842481 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560a697b2190 0x560a697b4580 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bec88547c22f3cf server_cookie=2df09f0ce8f2532e in_seq=0 out_seq=0 2024-06-10T01:26:55.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.430+0000 7fe5cbfff700 10 client.29596 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:55.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.432+0000 7fe5cbfff700 1 -- 172.21.15.7:0/3570842481 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560a696e9290 con 0x560a697b2190 2024-06-10T01:26:55.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.432+0000 7fe5cbfff700 10 client.29596 handle_command_reply: tid=0 2024-06-10T01:26:55.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.433+0000 7fe5fd89bb80 1 client.29596 shutdown 2024-06-10T01:26:55.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.433+0000 7fe5fd89bb80 1 -- 172.21.15.7:0/3570842481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6978ff20 msgr2=0x560a6979bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:55.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.433+0000 7fe5fd89bb80 1 --2- 172.21.15.7:0/3570842481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6978ff20 0x560a6979bce0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0011ca0 tx=0x7fe5e00047c0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.433+0000 7fe5fd89bb80 1 -- 172.21.15.7:0/3570842481 shutdown_connections 2024-06-10T01:26:55.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.433+0000 7fe5fd89bb80 1 --2- 172.21.15.7:0/3570842481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a69795f60 0x560a69799520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.433+0000 7fe5fd89bb80 1 --2- 172.21.15.7:0/3570842481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6978ff20 0x560a6979bce0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.433+0000 7fe5fd89bb80 1 --2- 172.21.15.7:0/3570842481 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560a697b2190 0x560a697b4580 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.433+0000 7fe5fd89bb80 1 --2- 172.21.15.7:0/3570842481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a69795640 0x560a69798fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.433+0000 7fe5fd89bb80 1 -- 172.21.15.7:0/3570842481 >> 172.21.15.7:0/3570842481 conn(0x560a696bacc0 msgr2=0x560a6978e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:55.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.433+0000 7fe5cbfff700 0 client.29596 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:55.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.433+0000 7fe5fd89bb80 1 -- 172.21.15.7:0/3570842481 shutdown_connections 2024-06-10T01:26:55.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.433+0000 7fe5fd89bb80 1 -- 172.21.15.7:0/3570842481 wait complete. 2024-06-10T01:26:55.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.434+0000 7fe5fd89bb80 20 client.29596 trim_cache size 0 max 16384 2024-06-10T01:26:55.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.470+0000 7fe5fd89bb80 1 Processor -- start 2024-06-10T01:26:55.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.470+0000 7fe5fd89bb80 1 -- start start 2024-06-10T01:26:55.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.470+0000 7fe5fd89bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6978ff20 0x560a6978f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:55.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.470+0000 7fe5fd89bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a69795640 0x560a69784690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:55.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.470+0000 7fe5fd89bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a69795f60 0x560a6979af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:55.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.470+0000 7fe5fd89bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ec1ae7b0 con 0x560a69795640 2024-06-10T01:26:55.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.470+0000 7fe5fd89bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ec1ac040 con 0x560a69795f60 2024-06-10T01:26:55.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.470+0000 7fe5fd89bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ec1a9f30 con 0x560a6978ff20 2024-06-10T01:26:55.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.471+0000 7fe5f0db4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6978ff20 0x560a6978f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:55.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.471+0000 7fe5f0db4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6978ff20 0x560a6978f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:60138/0 (socket says 172.21.15.7:60138) 2024-06-10T01:26:55.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.471+0000 7fe5f0db4700 1 -- 172.21.15.7:0/2832260859 learned_addr learned my addr 172.21.15.7:0/2832260859 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:55.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.471+0000 7fe5f15b5700 1 --2- 172.21.15.7:0/2832260859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a69795f60 0x560a6979af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:55.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.471+0000 7fe5ebfff700 1 --2- 172.21.15.7:0/2832260859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a69795640 0x560a69784690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:55.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.471+0000 7fe5f0db4700 1 -- 172.21.15.7:0/2832260859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a69795f60 msgr2=0x560a6979af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:55.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.471+0000 7fe5f0db4700 1 --2- 172.21.15.7:0/2832260859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a69795f60 0x560a6979af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.471+0000 7fe5f0db4700 1 -- 172.21.15.7:0/2832260859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a69795640 msgr2=0x560a69784690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:55.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.471+0000 7fe5f0db4700 1 --2- 172.21.15.7:0/2832260859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a69795640 0x560a69784690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.471+0000 7fe5f0db4700 1 -- 172.21.15.7:0/2832260859 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a6978b500 con 0x560a6978ff20 2024-06-10T01:26:55.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.471+0000 7fe5f0db4700 1 --2- 172.21.15.7:0/2832260859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6978ff20 0x560a6978f9e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0016040 tx=0x7fe5e001cf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=757ffb68f4a7281f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:55.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.472+0000 7fe5d67fc700 1 -- 172.21.15.7:0/2832260859 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e001cb40 con 0x560a6978ff20 2024-06-10T01:26:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.472+0000 7fe5d67fc700 1 -- 172.21.15.7:0/2832260859 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5e001cce0 con 0x560a6978ff20 2024-06-10T01:26:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.472+0000 7fe5d67fc700 1 -- 172.21.15.7:0/2832260859 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0005290 con 0x560a6978ff20 2024-06-10T01:26:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.473+0000 7fe5fd89bb80 1 -- 172.21.15.7:0/2832260859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6978ff20 msgr2=0x560a6978f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.473+0000 7fe5fd89bb80 1 --2- 172.21.15.7:0/2832260859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6978ff20 0x560a6978f9e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0016040 tx=0x7fe5e001cf10 comp rx=0 tx=0).stop 2024-06-10T01:26:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.473+0000 7fe5fd89bb80 1 -- 172.21.15.7:0/2832260859 shutdown_connections 2024-06-10T01:26:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.473+0000 7fe5fd89bb80 1 --2- 172.21.15.7:0/2832260859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a69795f60 0x560a6979af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.473+0000 7fe5fd89bb80 1 --2- 172.21.15.7:0/2832260859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6978ff20 0x560a6978f9e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.473+0000 7fe5fd89bb80 1 --2- 172.21.15.7:0/2832260859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a69795640 0x560a69784690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.473+0000 7fe5fd89bb80 1 -- 172.21.15.7:0/2832260859 >> 172.21.15.7:0/2832260859 conn(0x560a696bacc0 msgr2=0x560a6979a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.473+0000 7fe5fd89bb80 1 -- 172.21.15.7:0/2832260859 shutdown_connections 2024-06-10T01:26:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.473+0000 7fe5fd89bb80 1 -- 172.21.15.7:0/2832260859 wait complete. 2024-06-10T01:26:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.474+0000 7fe5fd89bb80 1 -- start start 2024-06-10T01:26:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.474+0000 7fe5fd89bb80 1 Processor -- start 2024-06-10T01:26:55.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.475+0000 7fe5fd89bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a6978ff20 0x560a697a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:55.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.475+0000 7fe5fd89bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a69795640 0x560a6979f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:55.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.475+0000 7fe5fd89bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a69795f60 0x560a6979fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:55.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.475+0000 7fe5fd89bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ec1aa500 con 0x560a69795640 2024-06-10T01:26:55.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.475+0000 7fe5fd89bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ec1ab970 con 0x560a6978ff20 2024-06-10T01:26:55.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.475+0000 7fe5f0db4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a6978ff20 0x560a697a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:55.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.475+0000 7fe5fd89bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ec1aabd0 con 0x560a69795f60 2024-06-10T01:26:55.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.475+0000 7fe5f0db4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a6978ff20 0x560a697a24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:54674/0 (socket says 172.21.15.7:54674) 2024-06-10T01:26:55.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.475+0000 7fe5f0db4700 1 -- 172.21.15.7:0/2993304292 learned_addr learned my addr 172.21.15.7:0/2993304292 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:55.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.475+0000 7fe5fd89bb80 10 client.0 _ll_register_callbacks cb 0x560a6967b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:26:55.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.475+0000 7fe5ebfff700 1 --2- 172.21.15.7:0/2993304292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a69795640 0x560a6979f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:55.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.476+0000 7fe5f15b5700 1 --2- 172.21.15.7:0/2993304292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a69795f60 0x560a6979fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:55.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.476+0000 7fe5f0db4700 1 -- 172.21.15.7:0/2993304292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a69795f60 msgr2=0x560a6979fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:55.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.476+0000 7fe5f0db4700 1 --2- 172.21.15.7:0/2993304292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a69795f60 0x560a6979fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.476+0000 7fe5f0db4700 1 -- 172.21.15.7:0/2993304292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a69795640 msgr2=0x560a6979f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:26:55.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.476+0000 7fe5f0db4700 1 --2- 172.21.15.7:0/2993304292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a69795640 0x560a6979f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.476+0000 7fe5f0db4700 1 -- 172.21.15.7:0/2993304292 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a697a0600 con 0x560a6978ff20 2024-06-10T01:26:55.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.476+0000 7fe5ebfff700 1 --2- 172.21.15.7:0/2993304292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a69795640 0x560a6979f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:26:55.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.476+0000 7fe5f0db4700 1 --2- 172.21.15.7:0/2993304292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a6978ff20 0x560a697a24e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0000c00 tx=0x7fe5e0009f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=485540ce88cfb7fc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:55.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.476+0000 7fe5cbfff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:26:55.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.476+0000 7fe5cbfff700 1 -- 172.21.15.7:0/2993304292 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e003d070 con 0x560a6978ff20 2024-06-10T01:26:55.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.477+0000 7fe5cbfff700 1 -- 172.21.15.7:0/2993304292 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fe5e00049c0 con 0x560a6978ff20 2024-06-10T01:26:55.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.477+0000 7fe5fd89bb80 1 -- 172.21.15.7:0/2993304292 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5e001cce0 con 0x560a6978ff20 2024-06-10T01:26:55.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.477+0000 7fe5cbfff700 1 -- 172.21.15.7:0/2993304292 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fe5e004b050 con 0x560a6978ff20 2024-06-10T01:26:55.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.477+0000 7fe5fd89bb80 10 client.29666 fetch_fsmap learned FSMap version 52 2024-06-10T01:26:55.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.477+0000 7fe5fd89bb80 1 -- 172.21.15.7:0/2993304292 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x560a6981cdc0 con 0x560a6978ff20 2024-06-10T01:26:55.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.477+0000 7fe5cbfff700 1 -- 172.21.15.7:0/2993304292 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5e001cce0 con 0x560a6978ff20 2024-06-10T01:26:55.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.477+0000 7fe5cbfff700 1 -- 172.21.15.7:0/2993304292 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7fe5e0042480 con 0x560a6978ff20 2024-06-10T01:26:55.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.478+0000 7fe5fd89bb80 10 client.29666 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:26:55.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.478+0000 7fe5fd89bb80 10 client.29666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:26:55.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.478+0000 7fe5fd89bb80 20 client.29666 populate_metadata read hostname 'smithi007' 2024-06-10T01:26:55.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.478+0000 7fe5fd89bb80 1 --2- 172.21.15.7:0/2993304292 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560a6981d340 0x560a6981f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:55.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.478+0000 7fe5fd89bb80 4 client.29666 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:26:55.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.478+0000 7fe5fd89bb80 1 -- 172.21.15.7:0/2993304292 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560a69563250 con 0x560a6981d340 2024-06-10T01:26:55.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.478+0000 7fe5ebfff700 1 --2- 172.21.15.7:0/2993304292 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560a6981d340 0x560a6981f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:55.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.479+0000 7fe5ebfff700 1 --2- 172.21.15.7:0/2993304292 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560a6981d340 0x560a6981f760 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d7d45aee0e07a8d server_cookie=b18c9e2fc79b0d30 in_seq=0 out_seq=0 2024-06-10T01:26:55.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.479+0000 7fe5cbfff700 10 client.29666 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.481+0000 7fe5cbfff700 1 -- 172.21.15.7:0/2993304292 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560a69563250 con 0x560a6981d340 2024-06-10T01:26:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.481+0000 7fe5cbfff700 10 client.29666 handle_command_reply: tid=0 2024-06-10T01:26:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.481+0000 7fe5fd89bb80 1 client.29666 shutdown 2024-06-10T01:26:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.482+0000 7fe5fd89bb80 1 -- 172.21.15.7:0/2993304292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a6978ff20 msgr2=0x560a697a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.482+0000 7fe5fd89bb80 1 --2- 172.21.15.7:0/2993304292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a6978ff20 0x560a697a24e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0000c00 tx=0x7fe5e0009f40 comp rx=0 tx=0).stop 2024-06-10T01:26:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.482+0000 7fe5fd89bb80 1 -- 172.21.15.7:0/2993304292 shutdown_connections 2024-06-10T01:26:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.482+0000 7fe5fd89bb80 1 --2- 172.21.15.7:0/2993304292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a69795f60 0x560a6979fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.482+0000 7fe5fd89bb80 1 --2- 172.21.15.7:0/2993304292 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560a6981d340 0x560a6981f760 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.482+0000 7fe5fd89bb80 1 --2- 172.21.15.7:0/2993304292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a6978ff20 0x560a697a24e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.482+0000 7fe5fd89bb80 1 --2- 172.21.15.7:0/2993304292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a69795640 0x560a6979f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.482+0000 7fe5fd89bb80 1 -- 172.21.15.7:0/2993304292 >> 172.21.15.7:0/2993304292 conn(0x560a696bacc0 msgr2=0x560a697949a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:55.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.482+0000 7fe5cbfff700 0 client.29666 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:26:55.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.482+0000 7fe5fd89bb80 1 -- 172.21.15.7:0/2993304292 shutdown_connections 2024-06-10T01:26:55.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.483+0000 7fe5fd89bb80 1 -- 172.21.15.7:0/2993304292 wait complete. 2024-06-10T01:26:55.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.483+0000 7fe5fd89bb80 20 client.29666 trim_cache size 0 max 16384 2024-06-10T01:26:55.477 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:26:55.477 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-10T01:26:55.477 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-10T01:26:55.478 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:26:55.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.484+0000 7fe5f3018700 1 -- 172.21.15.7:0/2551899709 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe5dc0a2c20 msgr2=0x7fe5dc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:55.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.484+0000 7fe5f3018700 1 --2- 172.21.15.7:0/2551899709 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe5dc0a2c20 0x7fe5dc0a50e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0011550 tx=0x7fe5e0009260 comp rx=0 tx=0).stop 2024-06-10T01:26:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.484+0000 7fe5f3018700 1 -- 172.21.15.7:0/2551899709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5ec102ef0 msgr2=0x7fe5ec19b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.484+0000 7fe5f3018700 1 --2- 172.21.15.7:0/2551899709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5ec102ef0 0x7fe5ec19b2f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80098e0 tx=0x7fe5d8005f10 comp rx=0 tx=0).stop 2024-06-10T01:26:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.484+0000 7fe5f3018700 1 -- 172.21.15.7:0/2551899709 shutdown_connections 2024-06-10T01:26:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.485+0000 7fe5f3018700 1 --2- 172.21.15.7:0/2551899709 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe5dc0a2c20 0x7fe5dc0a50e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.485+0000 7fe5f3018700 1 --2- 172.21.15.7:0/2551899709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5ec1085d0 0x7fe5ec19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.485+0000 7fe5f3018700 1 --2- 172.21.15.7:0/2551899709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5ec102530 0x7fe5ec19adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.485+0000 7fe5f3018700 1 --2- 172.21.15.7:0/2551899709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5ec102ef0 0x7fe5ec19b2f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.485+0000 7fe5f3018700 1 -- 172.21.15.7:0/2551899709 >> 172.21.15.7:0/2551899709 conn(0x7fe5ec079640 msgr2=0x7fe5ec1060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.485+0000 7fe5f3018700 1 -- 172.21.15.7:0/2551899709 shutdown_connections 2024-06-10T01:26:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:55.485+0000 7fe5f3018700 1 -- 172.21.15.7:0/2551899709 wait complete. 2024-06-10T01:26:55.548 INFO:tasks.workunit.client.0.smithi007.stdout: 10 176716 22.10 MB/sec execute 332 sec latency 83.185 ms 2024-06-10T01:26:55.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:55 smithi007 ceph-mon[32398]: pgmap v1060: 129 pgs: 129 active+clean; 666 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 2.7 MiB/s wr, 137 op/s 2024-06-10T01:26:55.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:55 smithi078 ceph-mon[36683]: pgmap v1060: 129 pgs: 129 active+clean; 666 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 2.7 MiB/s wr, 137 op/s 2024-06-10T01:26:55.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:55 smithi175 ceph-mon[36636]: pgmap v1060: 129 pgs: 129 active+clean; 666 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 2.7 MiB/s wr, 137 op/s 2024-06-10T01:26:56.340 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:26:56.340 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:26:56.548 INFO:tasks.workunit.client.0.smithi007.stdout: 10 177241 22.13 MB/sec execute 333 sec latency 107.559 ms 2024-06-10T01:26:56.693 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:26:57.548 INFO:tasks.workunit.client.0.smithi007.stdout: 10 177819 22.14 MB/sec execute 334 sec latency 84.078 ms 2024-06-10T01:26:57.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:57 smithi078 ceph-mon[36683]: pgmap v1061: 129 pgs: 129 active+clean; 663 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 214 op/s 2024-06-10T01:26:57.835 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:57 smithi007 ceph-mon[32398]: pgmap v1061: 129 pgs: 129 active+clean; 663 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 214 op/s 2024-06-10T01:26:57.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:57 smithi175 ceph-mon[36636]: pgmap v1061: 129 pgs: 129 active+clean; 663 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 214 op/s 2024-06-10T01:26:58.548 INFO:tasks.workunit.client.0.smithi007.stdout: 10 178689 22.19 MB/sec execute 335 sec latency 80.668 ms 2024-06-10T01:26:58.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.679+0000 7f0a979f5700 1 -- 172.21.15.7:0/1008898687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a90101390 msgr2=0x7f0a90101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:58.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.679+0000 7f0a979f5700 1 --2- 172.21.15.7:0/1008898687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a90101390 0x7f0a90101810 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0a800098f0 tx=0x7f0a80009c00 comp rx=0 tx=0).stop 2024-06-10T01:26:58.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.679+0000 7f0a979f5700 1 -- 172.21.15.7:0/1008898687 shutdown_connections 2024-06-10T01:26:58.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.679+0000 7f0a979f5700 1 --2- 172.21.15.7:0/1008898687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a90101d50 0x7f0a9010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:58.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.679+0000 7f0a979f5700 1 --2- 172.21.15.7:0/1008898687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a90107400 0x7f0a901077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:58.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.679+0000 7f0a979f5700 1 --2- 172.21.15.7:0/1008898687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a90101390 0x7f0a90101810 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:58.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.679+0000 7f0a979f5700 1 -- 172.21.15.7:0/1008898687 >> 172.21.15.7:0/1008898687 conn(0x7f0a90078340 msgr2=0x7f0a90078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:58.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.679+0000 7f0a979f5700 1 -- 172.21.15.7:0/1008898687 shutdown_connections 2024-06-10T01:26:58.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.679+0000 7f0a979f5700 1 -- 172.21.15.7:0/1008898687 wait complete. 2024-06-10T01:26:58.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.680+0000 7f0a979f5700 1 Processor -- start 2024-06-10T01:26:58.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.680+0000 7f0a979f5700 1 -- start start 2024-06-10T01:26:58.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.680+0000 7f0a979f5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a90101390 0x7f0a9010c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:58.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.680+0000 7f0a979f5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a90101d50 0x7f0a9010c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:58.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.680+0000 7f0a979f5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a90107400 0x7f0a9010ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:58.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.680+0000 7f0a979f5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9010f8c0 con 0x7f0a90101390 2024-06-10T01:26:58.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.681+0000 7f0a979f5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9010f740 con 0x7f0a90107400 2024-06-10T01:26:58.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.681+0000 7f0a979f5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9010fa40 con 0x7f0a90101d50 2024-06-10T01:26:58.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.681+0000 7f0a94f90700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a90101d50 0x7f0a9010c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:58.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.681+0000 7f0a94f90700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a90101d50 0x7f0a9010c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:60168/0 (socket says 172.21.15.7:60168) 2024-06-10T01:26:58.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.681+0000 7f0a94f90700 1 -- 172.21.15.7:0/3596507625 learned_addr learned my addr 172.21.15.7:0/3596507625 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:26:58.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.681+0000 7f0a95791700 1 --2- 172.21.15.7:0/3596507625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a90101390 0x7f0a9010c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:58.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.681+0000 7f0a95f92700 1 --2- 172.21.15.7:0/3596507625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a90107400 0x7f0a9010ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:58.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.681+0000 7f0a94f90700 1 -- 172.21.15.7:0/3596507625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a90107400 msgr2=0x7f0a9010ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.681+0000 7f0a94f90700 1 --2- 172.21.15.7:0/3596507625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a90107400 0x7f0a9010ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.681+0000 7f0a94f90700 1 -- 172.21.15.7:0/3596507625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a90101390 msgr2=0x7f0a9010c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.681+0000 7f0a94f90700 1 --2- 172.21.15.7:0/3596507625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a90101390 0x7f0a9010c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.681+0000 7f0a94f90700 1 -- 172.21.15.7:0/3596507625 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a901ab5d0 con 0x7f0a90101d50 2024-06-10T01:26:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.681+0000 7f0a94f90700 1 --2- 172.21.15.7:0/3596507625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a90101d50 0x7f0a9010c900 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0a80005950 tx=0x7f0a8000fb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.682+0000 7f0a867fc700 1 -- 172.21.15.7:0/3596507625 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a800051f0 con 0x7f0a90101d50 2024-06-10T01:26:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.682+0000 7f0a979f5700 1 -- 172.21.15.7:0/3596507625 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a901ab860 con 0x7f0a90101d50 2024-06-10T01:26:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.682+0000 7f0a867fc700 1 -- 172.21.15.7:0/3596507625 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a8000b560 con 0x7f0a90101d50 2024-06-10T01:26:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.682+0000 7f0a867fc700 1 -- 172.21.15.7:0/3596507625 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80003ba0 con 0x7f0a90101d50 2024-06-10T01:26:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.682+0000 7f0a979f5700 1 -- 172.21.15.7:0/3596507625 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a901abd10 con 0x7f0a90101d50 2024-06-10T01:26:58.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.683+0000 7f0a979f5700 1 -- 172.21.15.7:0/3596507625 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a90069180 con 0x7f0a90101d50 2024-06-10T01:26:58.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.684+0000 7f0a867fc700 1 -- 172.21.15.7:0/3596507625 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0a8000ba80 con 0x7f0a90101d50 2024-06-10T01:26:58.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.685+0000 7f0a867fc700 1 --2- 172.21.15.7:0/3596507625 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0a7c0a2c20 0x7f0a7c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:26:58.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.685+0000 7f0a95791700 1 --2- 172.21.15.7:0/3596507625 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0a7c0a2c20 0x7f0a7c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:26:58.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.685+0000 7f0a867fc700 1 -- 172.21.15.7:0/3596507625 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f0a800d03c0 con 0x7f0a90101d50 2024-06-10T01:26:58.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.685+0000 7f0a95791700 1 --2- 172.21.15.7:0/3596507625 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0a7c0a2c20 0x7f0a7c0a50e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c005fd0 tx=0x7f0a8c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:26:58.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:58.689+0000 7f0a867fc700 1 -- 172.21.15.7:0/3596507625 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0a80013030 con 0x7f0a90101d50 2024-06-10T01:26:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:59.036+0000 7f0a979f5700 1 -- 172.21.15.7:0/3596507625 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a901abf70 con 0x7f0a90101d50 2024-06-10T01:26:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:59.037+0000 7f0a867fc700 1 -- 172.21.15.7:0/3596507625 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11858 (secure 0 0 0) 0x7f0a8009cfd0 con 0x7f0a90101d50 2024-06-10T01:26:59.031 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:26:59.031 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":52,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:25:03.837724+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28825":{"gid":28825,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/1972426673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1972426673},{"type":"v1","addr":"172.21.15.175:6835","nonce":1972426673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24451,24434,24436]},"id":1}]} 2024-06-10T01:26:59.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:59.039+0000 7f0a979f5700 1 -- 172.21.15.7:0/3596507625 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0a7c0a2c20 msgr2=0x7f0a7c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:59.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:59.039+0000 7f0a979f5700 1 --2- 172.21.15.7:0/3596507625 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0a7c0a2c20 0x7f0a7c0a50e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c005fd0 tx=0x7f0a8c009310 comp rx=0 tx=0).stop 2024-06-10T01:26:59.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:59.039+0000 7f0a979f5700 1 -- 172.21.15.7:0/3596507625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a90101d50 msgr2=0x7f0a9010c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:26:59.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:59.039+0000 7f0a979f5700 1 --2- 172.21.15.7:0/3596507625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a90101d50 0x7f0a9010c900 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0a80005950 tx=0x7f0a8000fb20 comp rx=0 tx=0).stop 2024-06-10T01:26:59.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:59.039+0000 7f0a979f5700 1 -- 172.21.15.7:0/3596507625 shutdown_connections 2024-06-10T01:26:59.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:59.039+0000 7f0a979f5700 1 --2- 172.21.15.7:0/3596507625 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0a7c0a2c20 0x7f0a7c0a50e0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:59.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:59.039+0000 7f0a979f5700 1 --2- 172.21.15.7:0/3596507625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a90107400 0x7f0a9010ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:59.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:59.039+0000 7f0a979f5700 1 --2- 172.21.15.7:0/3596507625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a90101390 0x7f0a9010c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:59.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:59.039+0000 7f0a979f5700 1 --2- 172.21.15.7:0/3596507625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a90101d50 0x7f0a9010c900 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:26:59.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:59.039+0000 7f0a979f5700 1 -- 172.21.15.7:0/3596507625 >> 172.21.15.7:0/3596507625 conn(0x7f0a90078340 msgr2=0x7f0a900ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:26:59.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:59.039+0000 7f0a979f5700 1 -- 172.21.15.7:0/3596507625 shutdown_connections 2024-06-10T01:26:59.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:26:59.039+0000 7f0a979f5700 1 -- 172.21.15.7:0/3596507625 wait complete. 2024-06-10T01:26:59.034 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 52 2024-06-10T01:26:59.548 INFO:tasks.workunit.client.0.smithi007.stdout: 10 179445 22.27 MB/sec execute 336 sec latency 74.134 ms 2024-06-10T01:26:59.752 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:59 smithi007 ceph-mon[32398]: pgmap v1062: 129 pgs: 129 active+clean; 663 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 903 KiB/s wr, 151 op/s 2024-06-10T01:26:59.752 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:26:59 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3596507625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:26:59.784 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:26:59.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:59 smithi078 ceph-mon[36683]: pgmap v1062: 129 pgs: 129 active+clean; 663 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 903 KiB/s wr, 151 op/s 2024-06-10T01:26:59.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:26:59 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3596507625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:26:59.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:59 smithi175 ceph-mon[36636]: pgmap v1062: 129 pgs: 129 active+clean; 663 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 903 KiB/s wr, 151 op/s 2024-06-10T01:26:59.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:26:59 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3596507625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:27:00.111 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:27:00.548 INFO:tasks.workunit.client.0.smithi007.stdout: 10 180186 22.35 MB/sec execute 337 sec latency 64.557 ms 2024-06-10T01:27:01.548 INFO:tasks.workunit.client.0.smithi007.stdout: 10 180720 22.35 MB/sec execute 338 sec latency 78.563 ms 2024-06-10T01:27:01.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:01 smithi078 ceph-mon[36683]: pgmap v1063: 129 pgs: 129 active+clean; 672 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.0 MiB/s wr, 290 op/s 2024-06-10T01:27:01.856 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:01 smithi007 ceph-mon[32398]: pgmap v1063: 129 pgs: 129 active+clean; 672 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.0 MiB/s wr, 290 op/s 2024-06-10T01:27:01.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:01 smithi175 ceph-mon[36636]: pgmap v1063: 129 pgs: 129 active+clean; 672 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.0 MiB/s wr, 290 op/s 2024-06-10T01:27:01.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.966+0000 7fb38ce88640 1 -- 172.21.15.7:0/2037955184 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3800307c0 con 0x7fb3881035d0 2024-06-10T01:27:01.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.966+0000 7fb3900ee640 1 -- 172.21.15.7:0/2037955184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3881035d0 msgr2=0x7fb388109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:01.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.966+0000 7fb3900ee640 1 --2- 172.21.15.7:0/2037955184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3881035d0 0x7fb388109e60 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fb380009ca0 tx=0x7fb38002a8c0 comp rx=0 tx=0).stop 2024-06-10T01:27:01.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.966+0000 7fb3900ee640 1 -- 172.21.15.7:0/2037955184 shutdown_connections 2024-06-10T01:27:01.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.966+0000 7fb3900ee640 1 --2- 172.21.15.7:0/2037955184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3881035d0 0x7fb388109e60 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:01.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.966+0000 7fb3900ee640 1 --2- 172.21.15.7:0/2037955184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb388102c10 0x7fb388103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:01.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.966+0000 7fb3900ee640 1 --2- 172.21.15.7:0/2037955184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb388101a20 0x7fb388101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:01.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.966+0000 7fb3900ee640 1 -- 172.21.15.7:0/2037955184 >> 172.21.15.7:0/2037955184 conn(0x7fb3880fcf90 msgr2=0x7fb3880ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:01.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.966+0000 7fb3900ee640 1 -- 172.21.15.7:0/2037955184 shutdown_connections 2024-06-10T01:27:01.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.967+0000 7fb3900ee640 1 -- 172.21.15.7:0/2037955184 wait complete. 2024-06-10T01:27:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.967+0000 7fb3900ee640 1 Processor -- start 2024-06-10T01:27:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.967+0000 7fb3900ee640 1 -- start start 2024-06-10T01:27:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.967+0000 7fb3900ee640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb388101a20 0x7fb38806b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.967+0000 7fb3900ee640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb388102c10 0x7fb38806bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.968+0000 7fb3900ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3881035d0 0x7fb388072d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.968+0000 7fb3900ee640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb38810b510 con 0x7fb388101a20 2024-06-10T01:27:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.968+0000 7fb3900ee640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb38810b390 con 0x7fb3881035d0 2024-06-10T01:27:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.968+0000 7fb3900ee640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb38810b690 con 0x7fb388102c10 2024-06-10T01:27:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.968+0000 7fb38d689640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb388102c10 0x7fb38806bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.968+0000 7fb38d689640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb388102c10 0x7fb38806bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43066/0 (socket says 172.21.15.7:43066) 2024-06-10T01:27:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.968+0000 7fb38d689640 1 -- 172.21.15.7:0/2475802944 learned_addr learned my addr 172.21.15.7:0/2475802944 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.968+0000 7fb38e68b640 1 --2- 172.21.15.7:0/2475802944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3881035d0 0x7fb388072d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.968+0000 7fb38de8a640 1 --2- 172.21.15.7:0/2475802944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb388101a20 0x7fb38806b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.968+0000 7fb38e68b640 1 -- 172.21.15.7:0/2475802944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb388102c10 msgr2=0x7fb38806bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.968+0000 7fb38e68b640 1 --2- 172.21.15.7:0/2475802944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb388102c10 0x7fb38806bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.968+0000 7fb38e68b640 1 -- 172.21.15.7:0/2475802944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb388101a20 msgr2=0x7fb38806b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.968+0000 7fb38e68b640 1 --2- 172.21.15.7:0/2475802944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb388101a20 0x7fb38806b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.968+0000 7fb38e68b640 1 -- 172.21.15.7:0/2475802944 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3880733e0 con 0x7fb3881035d0 2024-06-10T01:27:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.969+0000 7fb38e68b640 1 --2- 172.21.15.7:0/2475802944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3881035d0 0x7fb388072d70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb380031b80 tx=0x7fb380003ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.969+0000 7fb37effd640 1 -- 172.21.15.7:0/2475802944 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb380004450 con 0x7fb3881035d0 2024-06-10T01:27:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.969+0000 7fb3900ee640 1 -- 172.21.15.7:0/2475802944 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb388073670 con 0x7fb3881035d0 2024-06-10T01:27:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.969+0000 7fb37effd640 1 -- 172.21.15.7:0/2475802944 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb380040070 con 0x7fb3881035d0 2024-06-10T01:27:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.969+0000 7fb3900ee640 1 -- 172.21.15.7:0/2475802944 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3881a6890 con 0x7fb3881035d0 2024-06-10T01:27:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.971+0000 7fb37effd640 1 -- 172.21.15.7:0/2475802944 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb380045670 con 0x7fb3881035d0 2024-06-10T01:27:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.973+0000 7fb392df6740 1 Processor -- start 2024-06-10T01:27:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.973+0000 7fb37effd640 1 -- 172.21.15.7:0/2475802944 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb3800045f0 con 0x7fb3881035d0 2024-06-10T01:27:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.974+0000 7fb392df6740 1 -- start start 2024-06-10T01:27:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.974+0000 7fb392df6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d768c90010 0x55d768c90430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.974+0000 7fb392df6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d768c90a00 0x55d768c95500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.974+0000 7fb392df6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d768c95b70 0x55d768c98000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.974+0000 7fb392df6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d768bf34f0 con 0x55d768c90010 2024-06-10T01:27:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.974+0000 7fb392df6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d768b91870 con 0x55d768c95b70 2024-06-10T01:27:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.974+0000 7fb392df6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d768bef4b0 con 0x55d768c90a00 2024-06-10T01:27:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.975+0000 7fb38d689640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d768c90010 0x55d768c90430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.975+0000 7fb38d689640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d768c90010 0x55d768c90430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56758/0 (socket says 172.21.15.7:56758) 2024-06-10T01:27:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.975+0000 7fb38d689640 1 -- 172.21.15.7:0/1175279218 learned_addr learned my addr 172.21.15.7:0/1175279218 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.975+0000 7fb38de8a640 1 --2- 172.21.15.7:0/1175279218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d768c90a00 0x55d768c95500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.975+0000 7fb37effd640 1 --2- 172.21.15.7:0/2475802944 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb3740a2b80 0x7fb3740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.975+0000 7fb38d689640 1 -- 172.21.15.7:0/1175279218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d768c90a00 msgr2=0x55d768c95500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.975+0000 7fb38e68b640 1 --2- 172.21.15.7:0/2475802944 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb3740a2b80 0x7fb3740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.975+0000 7fb38d689640 1 --2- 172.21.15.7:0/1175279218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d768c90a00 0x55d768c95500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.975+0000 7fb38d689640 1 -- 172.21.15.7:0/1175279218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d768c95b70 msgr2=0x55d768c98000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.975+0000 7fb37effd640 1 -- 172.21.15.7:0/2475802944 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fb3800f0920 con 0x7fb3881035d0 2024-06-10T01:27:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.976+0000 7fb38d689640 1 --2- 172.21.15.7:0/1175279218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d768c95b70 0x55d768c98000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.976+0000 7fb38d689640 1 -- 172.21.15.7:0/1175279218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d768bad9b0 con 0x55d768c90010 2024-06-10T01:27:01.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.976+0000 7fb38d689640 1 --2- 172.21.15.7:0/1175279218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d768c90010 0x55d768c90430 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fb38400bcf0 tx=0x7fb3840107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a240d83b9567a752 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:01.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.976+0000 7fb38e68b640 1 --2- 172.21.15.7:0/2475802944 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb3740a2b80 0x7fb3740a5040 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb3800b97d0 tx=0x7fb3800879d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:01.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.976+0000 7fb373fff640 1 -- 172.21.15.7:0/1175279218 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb384004220 con 0x55d768c90010 2024-06-10T01:27:01.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.976+0000 7fb373fff640 1 -- 172.21.15.7:0/1175279218 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb384015030 con 0x55d768c90010 2024-06-10T01:27:01.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.976+0000 7fb373fff640 1 -- 172.21.15.7:0/1175279218 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb384004220 con 0x55d768c90010 2024-06-10T01:27:01.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.977+0000 7fb392df6740 1 -- 172.21.15.7:0/1175279218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d768c90010 msgr2=0x55d768c90430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:01.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.977+0000 7fb392df6740 1 --2- 172.21.15.7:0/1175279218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d768c90010 0x55d768c90430 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fb38400bcf0 tx=0x7fb3840107a0 comp rx=0 tx=0).stop 2024-06-10T01:27:01.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.977+0000 7fb392df6740 1 -- 172.21.15.7:0/1175279218 shutdown_connections 2024-06-10T01:27:01.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.977+0000 7fb392df6740 1 --2- 172.21.15.7:0/1175279218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d768c95b70 0x55d768c98000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:01.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.977+0000 7fb392df6740 1 --2- 172.21.15.7:0/1175279218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d768c90a00 0x55d768c95500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:01.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.977+0000 7fb392df6740 1 --2- 172.21.15.7:0/1175279218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d768c90010 0x55d768c90430 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:01.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.977+0000 7fb392df6740 1 -- 172.21.15.7:0/1175279218 >> 172.21.15.7:0/1175279218 conn(0x55d768c8b680 msgr2=0x55d768c8ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:01.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.978+0000 7fb392df6740 1 -- 172.21.15.7:0/1175279218 shutdown_connections 2024-06-10T01:27:01.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.978+0000 7fb392df6740 1 -- 172.21.15.7:0/1175279218 wait complete. 2024-06-10T01:27:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.978+0000 7fb392df6740 1 -- start start 2024-06-10T01:27:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.978+0000 7fb392df6740 1 Processor -- start 2024-06-10T01:27:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.979+0000 7fb392df6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d768c90010 0x55d768c9a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.979+0000 7fb392df6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d768c90a00 0x55d768c9a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.979+0000 7fb392df6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d768c95b70 0x55d768ca1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.980+0000 7fb392df6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d768c98b20 con 0x55d768c90a00 2024-06-10T01:27:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.980+0000 7fb392df6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d768b7cae0 con 0x55d768c90010 2024-06-10T01:27:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.980+0000 7fb392df6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d768c98ca0 con 0x55d768c95b70 2024-06-10T01:27:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.980+0000 7fb392df6740 10 client.0 _ll_register_callbacks cb 0x55d768bb59b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:27:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.981+0000 7fb38d689640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d768c90010 0x55d768c9a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.981+0000 7fb38d689640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d768c90010 0x55d768c9a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:34830/0 (socket says 172.21.15.7:34830) 2024-06-10T01:27:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.981+0000 7fb38d689640 1 -- 172.21.15.7:0/876152044 learned_addr learned my addr 172.21.15.7:0/876152044 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.981+0000 7fb38d689640 1 --2- 172.21.15.7:0/876152044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d768c95b70 0x55d768ca1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:01.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.984+0000 7fb38de8a640 1 --2- 172.21.15.7:0/876152044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d768c90a00 0x55d768c9a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:01.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.985+0000 7fb38de8a640 1 -- 172.21.15.7:0/876152044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d768c95b70 msgr2=0x55d768ca1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:01.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.985+0000 7fb38de8a640 1 --2- 172.21.15.7:0/876152044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d768c95b70 0x55d768ca1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:01.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.985+0000 7fb38de8a640 1 -- 172.21.15.7:0/876152044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d768c90010 msgr2=0x55d768c9a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:01.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.985+0000 7fb38de8a640 1 --2- 172.21.15.7:0/876152044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d768c90010 0x55d768c9a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:01.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.985+0000 7fb38de8a640 1 -- 172.21.15.7:0/876152044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d768907210 con 0x55d768c90a00 2024-06-10T01:27:01.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.986+0000 7fb38de8a640 1 --2- 172.21.15.7:0/876152044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d768c90a00 0x55d768c9a820 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x55d768c90e00 tx=0x7fb378008b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2281f3a8796ba58 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:01.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.986+0000 7fb3717fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:27:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.987+0000 7fb3717fa640 1 -- 172.21.15.7:0/876152044 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb378003ac0 con 0x55d768c90a00 2024-06-10T01:27:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.988+0000 7fb3717fa640 1 -- 172.21.15.7:0/876152044 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fb378003c60 con 0x55d768c90a00 2024-06-10T01:27:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.988+0000 7fb392df6740 1 -- 172.21.15.7:0/876152044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb384015030 con 0x55d768c90a00 2024-06-10T01:27:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.990+0000 7fb3717fa640 1 -- 172.21.15.7:0/876152044 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fb378003e60 con 0x55d768c90a00 2024-06-10T01:27:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.991+0000 7fb392df6740 10 client.20481 fetch_fsmap learned FSMap version 52 2024-06-10T01:27:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.991+0000 7fb392df6740 1 -- 172.21.15.7:0/876152044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55d7689d3620 con 0x55d768c90a00 2024-06-10T01:27:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.991+0000 7fb3717fa640 1 -- 172.21.15.7:0/876152044 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb384015030 con 0x55d768c90a00 2024-06-10T01:27:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.991+0000 7fb3717fa640 1 -- 172.21.15.7:0/876152044 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7fb378020640 con 0x55d768c90a00 2024-06-10T01:27:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.991+0000 7fb392df6740 10 client.20481 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:27:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.992+0000 7fb392df6740 10 client.20481 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:27:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.992+0000 7fb392df6740 20 client.20481 populate_metadata read hostname 'smithi007' 2024-06-10T01:27:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.992+0000 7fb392df6740 1 --2- 172.21.15.7:0/876152044 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d768ca65e0 0x55d768ca8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.992+0000 7fb392df6740 4 client.20481 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:27:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.992+0000 7fb392df6740 1 -- 172.21.15.7:0/876152044 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d768c06f10 con 0x55d768ca65e0 2024-06-10T01:27:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.995+0000 7fb38d689640 1 --2- 172.21.15.7:0/876152044 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d768ca65e0 0x55d768ca8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:01.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.999+0000 7fb38d689640 1 --2- 172.21.15.7:0/876152044 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d768ca65e0 0x55d768ca8a60 crc :-1 s=READY pgs=477 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61d2e44531c6f788 server_cookie=57d147857a195dc4 in_seq=0 out_seq=0 2024-06-10T01:27:01.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:01.999+0000 7fb3717fa640 10 client.20481 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.006+0000 7fb3717fa640 1 -- 172.21.15.7:0/876152044 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d768c06f10 con 0x55d768ca65e0 2024-06-10T01:27:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.006+0000 7fb3717fa640 10 client.20481 handle_command_reply: tid=0 2024-06-10T01:27:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.006+0000 7fb392df6740 1 client.20481 shutdown 2024-06-10T01:27:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.006+0000 7fb392df6740 1 -- 172.21.15.7:0/876152044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d768c90a00 msgr2=0x55d768c9a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.006+0000 7fb392df6740 1 --2- 172.21.15.7:0/876152044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d768c90a00 0x55d768c9a820 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x55d768c90e00 tx=0x7fb378008b70 comp rx=0 tx=0).stop 2024-06-10T01:27:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.007+0000 7fb392df6740 1 -- 172.21.15.7:0/876152044 shutdown_connections 2024-06-10T01:27:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.007+0000 7fb392df6740 1 --2- 172.21.15.7:0/876152044 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d768ca65e0 0x55d768ca8a60 crc :-1 s=READY pgs=477 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.007+0000 7fb392df6740 1 --2- 172.21.15.7:0/876152044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d768c95b70 0x55d768ca1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.007+0000 7fb392df6740 1 --2- 172.21.15.7:0/876152044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d768c90a00 0x55d768c9a820 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.007+0000 7fb392df6740 1 --2- 172.21.15.7:0/876152044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d768c90010 0x55d768c9a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.007+0000 7fb392df6740 1 -- 172.21.15.7:0/876152044 >> 172.21.15.7:0/876152044 conn(0x55d768c8b680 msgr2=0x55d768c8a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.007+0000 7fb3717fa640 0 client.20481 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:02.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.007+0000 7fb392df6740 1 -- 172.21.15.7:0/876152044 shutdown_connections 2024-06-10T01:27:02.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.007+0000 7fb392df6740 1 -- 172.21.15.7:0/876152044 wait complete. 2024-06-10T01:27:02.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.007+0000 7fb392df6740 20 client.20481 trim_cache size 0 max 16384 2024-06-10T01:27:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.036+0000 7fb392df6740 1 Processor -- start 2024-06-10T01:27:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.036+0000 7fb392df6740 1 -- start start 2024-06-10T01:27:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.037+0000 7fb392df6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d768c90010 0x55d768c97f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.037+0000 7fb392df6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d768c90a00 0x55d768ca1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.037+0000 7fb392df6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d768c95b70 0x55d768c91c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.037+0000 7fb392df6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb3881b3560 con 0x55d768c90a00 2024-06-10T01:27:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.037+0000 7fb392df6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb3881b0e30 con 0x55d768c95b70 2024-06-10T01:27:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.037+0000 7fb392df6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb3881aec80 con 0x55d768c90010 2024-06-10T01:27:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.037+0000 7fb38d689640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d768c90010 0x55d768c97f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.037+0000 7fb38de8a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d768c90a00 0x55d768ca1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.037+0000 7fb38de8a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d768c90a00 0x55d768ca1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56780/0 (socket says 172.21.15.7:56780) 2024-06-10T01:27:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.037+0000 7fb38de8a640 1 -- 172.21.15.7:0/374662297 learned_addr learned my addr 172.21.15.7:0/374662297 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.037+0000 7fb38d689640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d768c90010 0x55d768c97f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43088/0 (socket says 172.21.15.7:43088) 2024-06-10T01:27:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.037+0000 7fb38d689640 1 --2- 172.21.15.7:0/374662297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d768c95b70 0x55d768c91c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.037+0000 7fb38de8a640 1 -- 172.21.15.7:0/374662297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d768c90010 msgr2=0x55d768c97f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.037+0000 7fb38de8a640 1 --2- 172.21.15.7:0/374662297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d768c90010 0x55d768c97f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.037+0000 7fb38de8a640 1 -- 172.21.15.7:0/374662297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d768c95b70 msgr2=0x55d768c91c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.037+0000 7fb38de8a640 1 --2- 172.21.15.7:0/374662297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d768c95b70 0x55d768c91c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.037+0000 7fb38de8a640 1 -- 172.21.15.7:0/374662297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d768c9b180 con 0x55d768c90a00 2024-06-10T01:27:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.037+0000 7fb38de8a640 1 --2- 172.21.15.7:0/374662297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d768c90a00 0x55d768ca1820 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fb378016c40 tx=0x7fb378021f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926cf30993de7efc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.038+0000 7fb3737fe640 1 -- 172.21.15.7:0/374662297 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb378003bc0 con 0x55d768c90a00 2024-06-10T01:27:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.038+0000 7fb3737fe640 1 -- 172.21.15.7:0/374662297 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb378025070 con 0x55d768c90a00 2024-06-10T01:27:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.038+0000 7fb3737fe640 1 -- 172.21.15.7:0/374662297 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3780047f0 con 0x55d768c90a00 2024-06-10T01:27:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.039+0000 7fb392df6740 1 -- 172.21.15.7:0/374662297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d768c90a00 msgr2=0x55d768ca1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.039+0000 7fb392df6740 1 --2- 172.21.15.7:0/374662297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d768c90a00 0x55d768ca1820 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fb378016c40 tx=0x7fb378021f70 comp rx=0 tx=0).stop 2024-06-10T01:27:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.039+0000 7fb392df6740 1 -- 172.21.15.7:0/374662297 shutdown_connections 2024-06-10T01:27:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.039+0000 7fb392df6740 1 --2- 172.21.15.7:0/374662297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d768c95b70 0x55d768c91c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.039+0000 7fb392df6740 1 --2- 172.21.15.7:0/374662297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d768c90a00 0x55d768ca1820 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.039+0000 7fb392df6740 1 --2- 172.21.15.7:0/374662297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d768c90010 0x55d768c97f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.039+0000 7fb392df6740 1 -- 172.21.15.7:0/374662297 >> 172.21.15.7:0/374662297 conn(0x55d768c8b680 msgr2=0x55d768c9a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.039+0000 7fb392df6740 1 -- 172.21.15.7:0/374662297 shutdown_connections 2024-06-10T01:27:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.039+0000 7fb392df6740 1 -- 172.21.15.7:0/374662297 wait complete. 2024-06-10T01:27:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.040+0000 7fb392df6740 1 -- start start 2024-06-10T01:27:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.040+0000 7fb392df6740 1 Processor -- start 2024-06-10T01:27:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.040+0000 7fb392df6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d768c90010 0x55d768ca16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.041+0000 7fb392df6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d768c90a00 0x55d768c9e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.041+0000 7fb392df6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d768c95b70 0x55d768c9e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.041+0000 7fb38d689640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d768c90010 0x55d768ca16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.041+0000 7fb38d689640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d768c90010 0x55d768ca16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56788/0 (socket says 172.21.15.7:56788) 2024-06-10T01:27:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.041+0000 7fb38d689640 1 -- 172.21.15.7:0/1752662442 learned_addr learned my addr 172.21.15.7:0/1752662442 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.041+0000 7fb392df6740 1 -- 172.21.15.7:0/1752662442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb3881af2f0 con 0x55d768c90010 2024-06-10T01:27:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.041+0000 7fb392df6740 1 -- 172.21.15.7:0/1752662442 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb3881b0760 con 0x55d768c95b70 2024-06-10T01:27:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.041+0000 7fb392df6740 1 -- 172.21.15.7:0/1752662442 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb3881af9c0 con 0x55d768c90a00 2024-06-10T01:27:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.041+0000 7fb392df6740 10 client.0 _ll_register_callbacks cb 0x7fb378004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:27:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.041+0000 7fb38d689640 1 --2- 172.21.15.7:0/1752662442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d768c95b70 0x55d768c9e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.041+0000 7fb38de8a640 1 --2- 172.21.15.7:0/1752662442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d768c90a00 0x55d768c9e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.041+0000 7fb38d689640 1 -- 172.21.15.7:0/1752662442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d768c90a00 msgr2=0x55d768c9e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.041+0000 7fb38d689640 1 --2- 172.21.15.7:0/1752662442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d768c90a00 0x55d768c9e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.041+0000 7fb38d689640 1 -- 172.21.15.7:0/1752662442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d768c95b70 msgr2=0x55d768c9e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.041+0000 7fb38d689640 1 --2- 172.21.15.7:0/1752662442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d768c95b70 0x55d768c9e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.041+0000 7fb38d689640 1 -- 172.21.15.7:0/1752662442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d768d0f330 con 0x55d768c90010 2024-06-10T01:27:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.042+0000 7fb38d689640 1 --2- 172.21.15.7:0/1752662442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d768c90010 0x55d768ca16a0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x55d768ca92e0 tx=0x7fb378016900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada5248733706354 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.042+0000 7fb3717fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:27:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.042+0000 7fb3717fa640 1 -- 172.21.15.7:0/1752662442 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb384004720 con 0x55d768c90010 2024-06-10T01:27:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.042+0000 7fb3717fa640 1 -- 172.21.15.7:0/1752662442 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fb3840048c0 con 0x55d768c90010 2024-06-10T01:27:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.042+0000 7fb392df6740 1 -- 172.21.15.7:0/1752662442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb378025070 con 0x55d768c90010 2024-06-10T01:27:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.042+0000 7fb3717fa640 1 -- 172.21.15.7:0/1752662442 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fb384004ac0 con 0x55d768c90010 2024-06-10T01:27:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.042+0000 7fb392df6740 10 client.20487 fetch_fsmap learned FSMap version 52 2024-06-10T01:27:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.042+0000 7fb392df6740 1 -- 172.21.15.7:0/1752662442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55d768d105b0 con 0x55d768c90010 2024-06-10T01:27:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.042+0000 7fb3717fa640 1 -- 172.21.15.7:0/1752662442 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb378025070 con 0x55d768c90010 2024-06-10T01:27:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.043+0000 7fb3717fa640 1 -- 172.21.15.7:0/1752662442 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7fb384032070 con 0x55d768c90010 2024-06-10T01:27:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.043+0000 7fb392df6740 10 client.20487 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:27:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.043+0000 7fb392df6740 10 client.20487 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:27:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.043+0000 7fb392df6740 20 client.20487 populate_metadata read hostname 'smithi007' 2024-06-10T01:27:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.043+0000 7fb392df6740 1 --2- 172.21.15.7:0/1752662442 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d768d10e00 0x55d768d13220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.043+0000 7fb392df6740 4 client.20487 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:27:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.043+0000 7fb392df6740 1 -- 172.21.15.7:0/1752662442 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d768c00c20 con 0x55d768d10e00 2024-06-10T01:27:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.043+0000 7fb38de8a640 1 --2- 172.21.15.7:0/1752662442 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d768d10e00 0x55d768d13220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.043+0000 7fb38de8a640 1 --2- 172.21.15.7:0/1752662442 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d768d10e00 0x55d768d13220 crc :-1 s=READY pgs=478 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e9595cae2e5954c server_cookie=80ac2547c8d8acb2 in_seq=0 out_seq=0 2024-06-10T01:27:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.043+0000 7fb3717fa640 10 client.20487 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:02.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.415+0000 7f7753fff700 1 -- 172.21.15.7:0/1072156141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7754110b90 msgr2=0x7f7754109490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:02.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.415+0000 7f7753fff700 1 --2- 172.21.15.7:0/1072156141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7754110b90 0x7f7754109490 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7744009990 tx=0x7f7744009ca0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.421+0000 7f7753fff700 1 -- 172.21.15.7:0/1072156141 shutdown_connections 2024-06-10T01:27:02.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.421+0000 7f7753fff700 1 --2- 172.21.15.7:0/1072156141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7754109ff0 0x7f775411dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.421+0000 7f7753fff700 1 --2- 172.21.15.7:0/1072156141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7754110b90 0x7f7754109490 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.421+0000 7f7753fff700 1 --2- 172.21.15.7:0/1072156141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7754110f70 0x7f77541099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.421+0000 7f7753fff700 1 -- 172.21.15.7:0/1072156141 >> 172.21.15.7:0/1072156141 conn(0x7f775406d4c0 msgr2=0x7f775406d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:02.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.427+0000 7f7753fff700 1 -- 172.21.15.7:0/1072156141 shutdown_connections 2024-06-10T01:27:02.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.430+0000 7f7753fff700 1 -- 172.21.15.7:0/1072156141 wait complete. 2024-06-10T01:27:02.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.431+0000 7f7753fff700 1 Processor -- start 2024-06-10T01:27:02.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.431+0000 7f7753fff700 1 -- start start 2024-06-10T01:27:02.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.431+0000 7f7753fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7754109ff0 0x7f775411d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:02.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.431+0000 7f7753fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7754110b90 0x7f7754116830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:02.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.431+0000 7f7753fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7754110f70 0x7f7754116d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:02.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.431+0000 7f7753fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7754121670 con 0x7f7754110f70 2024-06-10T01:27:02.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.431+0000 7f7753fff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f77541214f0 con 0x7f7754109ff0 2024-06-10T01:27:02.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.431+0000 7f7753fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f77541217f0 con 0x7f7754110b90 2024-06-10T01:27:02.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.433+0000 7f77537fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7754110f70 0x7f7754116d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:02.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.433+0000 7f77537fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7754110f70 0x7f7754116d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56796/0 (socket says 172.21.15.7:56796) 2024-06-10T01:27:02.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.433+0000 7f77537fe700 1 -- 172.21.15.7:0/3833717804 learned_addr learned my addr 172.21.15.7:0/3833717804 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:02.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.435+0000 7f7752ffd700 1 --2- 172.21.15.7:0/3833717804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7754109ff0 0x7f775411d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:02.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.435+0000 7f77527fc700 1 --2- 172.21.15.7:0/3833717804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7754110b90 0x7f7754116830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:02.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.436+0000 7f77537fe700 1 -- 172.21.15.7:0/3833717804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7754110b90 msgr2=0x7f7754116830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:02.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.436+0000 7f77537fe700 1 --2- 172.21.15.7:0/3833717804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7754110b90 0x7f7754116830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.436+0000 7f77537fe700 1 -- 172.21.15.7:0/3833717804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7754109ff0 msgr2=0x7f775411d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.436+0000 7f77537fe700 1 --2- 172.21.15.7:0/3833717804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7754109ff0 0x7f775411d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.436+0000 7f77537fe700 1 -- 172.21.15.7:0/3833717804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77541173b0 con 0x7f7754110f70 2024-06-10T01:27:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.438+0000 7f77537fe700 1 --2- 172.21.15.7:0/3833717804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7754110f70 0x7f7754116d70 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f774c00b970 tx=0x7f774c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:02.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.442+0000 7f773bfff700 1 -- 172.21.15.7:0/3833717804 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c019070 con 0x7f7754110f70 2024-06-10T01:27:02.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.442+0000 7f773bfff700 1 -- 172.21.15.7:0/3833717804 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f774c014440 con 0x7f7754110f70 2024-06-10T01:27:02.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.443+0000 7f773bfff700 1 -- 172.21.15.7:0/3833717804 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c007590 con 0x7f7754110f70 2024-06-10T01:27:02.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.443+0000 7f7753fff700 1 -- 172.21.15.7:0/3833717804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f775411b2d0 con 0x7f7754110f70 2024-06-10T01:27:02.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.443+0000 7f7753fff700 1 -- 172.21.15.7:0/3833717804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f775411b790 con 0x7f7754110f70 2024-06-10T01:27:02.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.449+0000 7f7763be6b80 1 Processor -- start 2024-06-10T01:27:02.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.450+0000 7f773bfff700 1 -- 172.21.15.7:0/3833717804 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f774c0145f0 con 0x7f7754110f70 2024-06-10T01:27:02.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.451+0000 7f773bfff700 1 --2- 172.21.15.7:0/3833717804 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f773c0a2c90 0x7f773c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:02.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.452+0000 7f7763be6b80 1 -- start start 2024-06-10T01:27:02.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.452+0000 7f773bfff700 1 -- 172.21.15.7:0/3833717804 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f774c0ce700 con 0x7f7754110f70 2024-06-10T01:27:02.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.452+0000 7f7763be6b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630432658f0 0x563043265cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:02.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.452+0000 7f7763be6b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563043266210 0x56304325fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:02.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.452+0000 7f7763be6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563043260080 0x5630432604c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:02.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.452+0000 7f7763be6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5630431b1810 con 0x563043260080 2024-06-10T01:27:02.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.452+0000 7f7763be6b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563043159720 con 0x563043266210 2024-06-10T01:27:02.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.452+0000 7f7763be6b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5630431ad7d0 con 0x5630432658f0 2024-06-10T01:27:02.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.453+0000 7f77527fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563043266210 0x56304325fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:02.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.453+0000 7f77527fc700 1 --2- 172.21.15.7:0/3833717804 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f773c0a2c90 0x7f773c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:02.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.453+0000 7f77527fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563043266210 0x56304325fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:34874/0 (socket says 172.21.15.7:34874) 2024-06-10T01:27:02.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.453+0000 7f77527fc700 1 -- 172.21.15.7:0/532439569 learned_addr learned my addr 172.21.15.7:0/532439569 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:02.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.454+0000 7f77537fe700 1 --2- 172.21.15.7:0/532439569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563043260080 0x5630432604c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:02.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.455+0000 7f77527fc700 1 -- 172.21.15.7:0/532439569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630432658f0 msgr2=0x563043265cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:27:02.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.455+0000 7f77527fc700 1 --2- 172.21.15.7:0/532439569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630432658f0 0x563043265cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.455+0000 7f77527fc700 1 -- 172.21.15.7:0/532439569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563043260080 msgr2=0x5630432604c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:02.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.455+0000 7f77527fc700 1 --2- 172.21.15.7:0/532439569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563043260080 0x5630432604c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.455+0000 7f77527fc700 1 -- 172.21.15.7:0/532439569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56304314ff50 con 0x563043266210 2024-06-10T01:27:02.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.455+0000 7f77527fc700 1 --2- 172.21.15.7:0/3833717804 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f773c0a2c90 0x7f773c0a5150 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7748013da0 tx=0x7f7748009000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:02.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.455+0000 7f77527fc700 1 --2- 172.21.15.7:0/532439569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563043266210 0x56304325fb40 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7748013740 tx=0x7f7748013a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bbf9b2f11cc8859 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:02.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.459+0000 7f77397fa700 1 -- 172.21.15.7:0/532439569 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774801c070 con 0x563043266210 2024-06-10T01:27:02.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.459+0000 7f77397fa700 1 -- 172.21.15.7:0/532439569 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7748017dc0 con 0x563043266210 2024-06-10T01:27:02.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.459+0000 7f77397fa700 1 -- 172.21.15.7:0/532439569 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774800f040 con 0x563043266210 2024-06-10T01:27:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.460+0000 7f7763be6b80 1 -- 172.21.15.7:0/532439569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563043266210 msgr2=0x56304325fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.460+0000 7f7763be6b80 1 --2- 172.21.15.7:0/532439569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563043266210 0x56304325fb40 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7748013740 tx=0x7f7748013a50 comp rx=0 tx=0).stop 2024-06-10T01:27:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.463+0000 7f7763be6b80 1 -- 172.21.15.7:0/532439569 shutdown_connections 2024-06-10T01:27:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.463+0000 7f7763be6b80 1 --2- 172.21.15.7:0/532439569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563043260080 0x5630432604c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.463+0000 7f7763be6b80 1 --2- 172.21.15.7:0/532439569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630432658f0 0x563043265cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.463+0000 7f7763be6b80 1 --2- 172.21.15.7:0/532439569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563043266210 0x56304325fb40 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.463+0000 7f7763be6b80 1 -- 172.21.15.7:0/532439569 >> 172.21.15.7:0/532439569 conn(0x56304318acc0 msgr2=0x56304325aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.468+0000 7f7763be6b80 1 -- 172.21.15.7:0/532439569 shutdown_connections 2024-06-10T01:27:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.468+0000 7f7763be6b80 1 -- 172.21.15.7:0/532439569 wait complete. 2024-06-10T01:27:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.468+0000 7f7763be6b80 1 -- start start 2024-06-10T01:27:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.468+0000 7f7763be6b80 1 Processor -- start 2024-06-10T01:27:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.469+0000 7f7763be6b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563043260080 0x56304326bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.469+0000 7f7763be6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630432658f0 0x563043269280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.469+0000 7f7763be6b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563043266210 0x5630432697c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.469+0000 7f7763be6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56304326cb10 con 0x5630432658f0 2024-06-10T01:27:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.469+0000 7f7763be6b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563043151bf0 con 0x563043266210 2024-06-10T01:27:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.469+0000 7f7763be6b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56304326cc90 con 0x563043260080 2024-06-10T01:27:02.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.469+0000 7f7763be6b80 10 client.0 _ll_register_callbacks cb 0x56304314b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:27:02.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.471+0000 7f77537fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563043266210 0x5630432697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:02.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.471+0000 7f77537fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563043266210 0x5630432697c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:34890/0 (socket says 172.21.15.7:34890) 2024-06-10T01:27:02.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.471+0000 7f77537fe700 1 -- 172.21.15.7:0/3376744050 learned_addr learned my addr 172.21.15.7:0/3376744050 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:02.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.473+0000 7f77527fc700 1 --2- 172.21.15.7:0/3376744050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630432658f0 0x563043269280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:02.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.475+0000 7f7752ffd700 1 --2- 172.21.15.7:0/3376744050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563043260080 0x56304326bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.475+0000 7f77537fe700 1 -- 172.21.15.7:0/3376744050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563043260080 msgr2=0x56304326bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.475+0000 7f77537fe700 1 --2- 172.21.15.7:0/3376744050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563043260080 0x56304326bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.475+0000 7f77537fe700 1 -- 172.21.15.7:0/3376744050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630432658f0 msgr2=0x563043269280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.475+0000 7f77537fe700 1 --2- 172.21.15.7:0/3376744050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630432658f0 0x563043269280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.475+0000 7f77537fe700 1 -- 172.21.15.7:0/3376744050 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630431008d0 con 0x563043266210 2024-06-10T01:27:02.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.477+0000 7f77527fc700 1 --2- 172.21.15.7:0/3376744050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630432658f0 0x563043269280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:27:02.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.477+0000 7f7752ffd700 1 --2- 172.21.15.7:0/3376744050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563043260080 0x56304326bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:27:02.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.480+0000 7f77537fe700 1 --2- 172.21.15.7:0/3376744050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563043266210 0x5630432697c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f774c0094e0 tx=0x7f774c09b3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0fc6a16d3433262 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.482+0000 7f7736ff5700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:27:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.482+0000 7f7736ff5700 1 -- 172.21.15.7:0/3376744050 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c0b8060 con 0x563043266210 2024-06-10T01:27:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.482+0000 7f7736ff5700 1 -- 172.21.15.7:0/3376744050 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f774c014980 con 0x563043266210 2024-06-10T01:27:02.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.484+0000 7f7763be6b80 1 -- 172.21.15.7:0/3376744050 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7748017dc0 con 0x563043266210 2024-06-10T01:27:02.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.488+0000 7f7736ff5700 1 -- 172.21.15.7:0/3376744050 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f774c09ade0 con 0x563043266210 2024-06-10T01:27:02.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.489+0000 7f7763be6b80 10 client.29690 fetch_fsmap learned FSMap version 52 2024-06-10T01:27:02.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.489+0000 7f7763be6b80 1 -- 172.21.15.7:0/3376744050 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5630432775a0 con 0x563043266210 2024-06-10T01:27:02.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.491+0000 7f7736ff5700 1 -- 172.21.15.7:0/3376744050 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7748017dc0 con 0x563043266210 2024-06-10T01:27:02.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.491+0000 7f7736ff5700 1 -- 172.21.15.7:0/3376744050 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f774c099fd0 con 0x563043266210 2024-06-10T01:27:02.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.491+0000 7f7763be6b80 10 client.29690 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:27:02.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.491+0000 7f7763be6b80 10 client.29690 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:27:02.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.491+0000 7f7763be6b80 20 client.29690 populate_metadata read hostname 'smithi007' 2024-06-10T01:27:02.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.491+0000 7f7763be6b80 1 --2- 172.21.15.7:0/3376744050 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x563043279b30 0x56304327bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:02.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.491+0000 7f7763be6b80 4 client.29690 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:27:02.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.491+0000 7f7763be6b80 1 -- 172.21.15.7:0/3376744050 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630431bed10 con 0x563043279b30 2024-06-10T01:27:02.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.498+0000 7f7752ffd700 1 --2- 172.21.15.7:0/3376744050 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x563043279b30 0x56304327bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:02.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.511+0000 7f7752ffd700 1 --2- 172.21.15.7:0/3376744050 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x563043279b30 0x56304327bf20 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8d28e4533aaad554 server_cookie=1e263248c2c644ae in_seq=0 out_seq=0 2024-06-10T01:27:02.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.512+0000 7f7736ff5700 10 client.29690 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:27:02.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.516+0000 7f7736ff5700 1 -- 172.21.15.7:0/3376744050 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5630431bed10 con 0x563043279b30 2024-06-10T01:27:02.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.516+0000 7f7736ff5700 10 client.29690 handle_command_reply: tid=0 2024-06-10T01:27:02.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.516+0000 7f7763be6b80 1 client.29690 shutdown 2024-06-10T01:27:02.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.518+0000 7f7763be6b80 1 -- 172.21.15.7:0/3376744050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563043266210 msgr2=0x5630432697c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:02.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.518+0000 7f7763be6b80 1 --2- 172.21.15.7:0/3376744050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563043266210 0x5630432697c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f774c0094e0 tx=0x7f774c09b3d0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.518+0000 7f7763be6b80 1 -- 172.21.15.7:0/3376744050 shutdown_connections 2024-06-10T01:27:02.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.518+0000 7f7763be6b80 1 --2- 172.21.15.7:0/3376744050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563043266210 0x5630432697c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.518+0000 7f7763be6b80 1 --2- 172.21.15.7:0/3376744050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563043260080 0x56304326bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.518+0000 7f7763be6b80 1 --2- 172.21.15.7:0/3376744050 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x563043279b30 0x56304327bf20 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.518+0000 7f7763be6b80 1 --2- 172.21.15.7:0/3376744050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630432658f0 0x563043269280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.518+0000 7f7736ff5700 0 client.29690 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:27:02.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.518+0000 7f7763be6b80 1 -- 172.21.15.7:0/3376744050 >> 172.21.15.7:0/3376744050 conn(0x56304318acc0 msgr2=0x56304325eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:02.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.518+0000 7f7763be6b80 1 -- 172.21.15.7:0/3376744050 shutdown_connections 2024-06-10T01:27:02.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.518+0000 7f7763be6b80 1 -- 172.21.15.7:0/3376744050 wait complete. 2024-06-10T01:27:02.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.519+0000 7f7763be6b80 20 client.29690 trim_cache size 0 max 16384 2024-06-10T01:27:02.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.549+0000 7f7763be6b80 1 Processor -- start 2024-06-10T01:27:02.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.549+0000 7f7763be6b80 1 -- start start 2024-06-10T01:27:02.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.549+0000 7f7763be6b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56304318acc0 0x56304325fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:02.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.549+0000 7f7763be6b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563043260080 0x563043275de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:02.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.549+0000 7f7763be6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630432658f0 0x563043266620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:02.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.549+0000 7f7763be6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f77541c7fd0 con 0x5630432658f0 2024-06-10T01:27:02.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.549+0000 7f7763be6b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f77541c5860 con 0x563043260080 2024-06-10T01:27:02.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.549+0000 7f7763be6b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f77541c36b0 con 0x56304318acc0 2024-06-10T01:27:02.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.549+0000 7f77537fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630432658f0 0x563043266620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:02.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.549+0000 7f77537fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630432658f0 0x563043266620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56830/0 (socket says 172.21.15.7:56830) 2024-06-10T01:27:02.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.549+0000 7f77537fe700 1 -- 172.21.15.7:0/2918861380 learned_addr learned my addr 172.21.15.7:0/2918861380 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:02.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.550+0000 7f77537fe700 1 -- 172.21.15.7:0/2918861380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56304318acc0 msgr2=0x56304325fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:02.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.550+0000 7f77537fe700 1 --2- 172.21.15.7:0/2918861380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56304318acc0 0x56304325fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.550+0000 7f77537fe700 1 -- 172.21.15.7:0/2918861380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563043260080 msgr2=0x563043275de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:02.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.550+0000 7f77537fe700 1 --2- 172.21.15.7:0/2918861380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563043260080 0x563043275de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.550+0000 7f77537fe700 1 -- 172.21.15.7:0/2918861380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630432677f0 con 0x5630432658f0 2024-06-10T01:27:02.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.550+0000 7f77537fe700 1 --2- 172.21.15.7:0/2918861380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630432658f0 0x563043266620 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f774c098e30 tx=0x7f774c099090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef0949eed9bb7b8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:02.548 INFO:tasks.workunit.client.0.smithi007.stdout: 10 181043 22.34 MB/sec execute 339 sec latency 381.777 ms 2024-06-10T01:27:02.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.557+0000 7f7738ff9700 1 -- 172.21.15.7:0/2918861380 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c09a950 con 0x5630432658f0 2024-06-10T01:27:02.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.557+0000 7f7738ff9700 1 -- 172.21.15.7:0/2918861380 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f774c0cbd60 con 0x5630432658f0 2024-06-10T01:27:02.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.557+0000 7f7738ff9700 1 -- 172.21.15.7:0/2918861380 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c014980 con 0x5630432658f0 2024-06-10T01:27:02.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.560+0000 7f7763be6b80 1 -- 172.21.15.7:0/2918861380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630432658f0 msgr2=0x563043266620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:02.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.560+0000 7f7763be6b80 1 --2- 172.21.15.7:0/2918861380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630432658f0 0x563043266620 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f774c098e30 tx=0x7f774c099090 comp rx=0 tx=0).stop 2024-06-10T01:27:02.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.560+0000 7f7763be6b80 1 -- 172.21.15.7:0/2918861380 shutdown_connections 2024-06-10T01:27:02.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.560+0000 7f7763be6b80 1 --2- 172.21.15.7:0/2918861380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630432658f0 0x563043266620 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.560+0000 7f7763be6b80 1 --2- 172.21.15.7:0/2918861380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56304318acc0 0x56304325fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.560+0000 7f7763be6b80 1 --2- 172.21.15.7:0/2918861380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563043260080 0x563043275de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.560+0000 7f7763be6b80 1 -- 172.21.15.7:0/2918861380 >> 172.21.15.7:0/2918861380 conn(0x563043263a80 msgr2=0x563043267b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:02.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.563+0000 7f7763be6b80 1 -- 172.21.15.7:0/2918861380 shutdown_connections 2024-06-10T01:27:02.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.563+0000 7f7763be6b80 1 -- 172.21.15.7:0/2918861380 wait complete. 2024-06-10T01:27:02.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.564+0000 7f7763be6b80 1 -- start start 2024-06-10T01:27:02.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.564+0000 7f7763be6b80 1 Processor -- start 2024-06-10T01:27:02.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.565+0000 7f7763be6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56304318acc0 0x563043281450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:02.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.565+0000 7f7763be6b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563043260080 0x563043281990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:02.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.565+0000 7f7752ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56304318acc0 0x563043281450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:02.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.565+0000 7f7752ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56304318acc0 0x563043281450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56842/0 (socket says 172.21.15.7:56842) 2024-06-10T01:27:02.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.565+0000 7f7752ffd700 1 -- 172.21.15.7:0/176740530 learned_addr learned my addr 172.21.15.7:0/176740530 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:02.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.566+0000 7f7763be6b80 1 --2- 172.21.15.7:0/176740530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630432658f0 0x563043281ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:02.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.566+0000 7f77527fc700 1 --2- 172.21.15.7:0/176740530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563043260080 0x563043281990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:02.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.566+0000 7f7763be6b80 1 -- 172.21.15.7:0/176740530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f77541c3d20 con 0x56304318acc0 2024-06-10T01:27:02.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.566+0000 7f7763be6b80 1 -- 172.21.15.7:0/176740530 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f77541c5190 con 0x563043260080 2024-06-10T01:27:02.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.566+0000 7f77537fe700 1 --2- 172.21.15.7:0/176740530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630432658f0 0x563043281ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:02.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.566+0000 7f7763be6b80 1 -- 172.21.15.7:0/176740530 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f77541c43f0 con 0x5630432658f0 2024-06-10T01:27:02.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.567+0000 7f7752ffd700 1 -- 172.21.15.7:0/176740530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630432658f0 msgr2=0x563043281ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:02.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.567+0000 7f7752ffd700 1 --2- 172.21.15.7:0/176740530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630432658f0 0x563043281ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.567+0000 7f7752ffd700 1 -- 172.21.15.7:0/176740530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563043260080 msgr2=0x563043281990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:02.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.567+0000 7f7752ffd700 1 --2- 172.21.15.7:0/176740530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563043260080 0x563043281990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.567+0000 7f7752ffd700 1 -- 172.21.15.7:0/176740530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630432e3810 con 0x56304318acc0 2024-06-10T01:27:02.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.567+0000 7f7763be6b80 10 client.0 _ll_register_callbacks cb 0x56304314b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:27:02.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.567+0000 7f7763be6b80 1 -- 172.21.15.7:0/176740530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f774c0cbd60 con 0x56304318acc0 2024-06-10T01:27:02.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.568+0000 7f7752ffd700 1 --2- 172.21.15.7:0/176740530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56304318acc0 0x563043281450 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f7744015000 tx=0x7f7744005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89538db373ac28cb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:02.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.568+0000 7f7736ff5700 10 client.20502 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:27:02.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.569+0000 7f7736ff5700 1 -- 172.21.15.7:0/176740530 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7744003e10 con 0x56304318acc0 2024-06-10T01:27:02.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.569+0000 7f7736ff5700 1 -- 172.21.15.7:0/176740530 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f77440050b0 con 0x56304318acc0 2024-06-10T01:27:02.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.569+0000 7f7736ff5700 1 -- 172.21.15.7:0/176740530 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f7744005310 con 0x56304318acc0 2024-06-10T01:27:02.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.570+0000 7f7763be6b80 10 client.20502 fetch_fsmap learned FSMap version 52 2024-06-10T01:27:02.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.570+0000 7f7763be6b80 1 -- 172.21.15.7:0/176740530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=52}) -- 0x5630432e4980 con 0x56304318acc0 2024-06-10T01:27:02.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.574+0000 7f7736ff5700 1 -- 172.21.15.7:0/176740530 <== mon.0 v2:172.21.15.7:3300/0 4 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f774402c060 con 0x56304318acc0 2024-06-10T01:27:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.575+0000 7f7763be6b80 10 client.20502 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:27:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.575+0000 7f7763be6b80 10 client.20502 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:27:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.575+0000 7f7763be6b80 20 client.20502 populate_metadata read hostname 'smithi007' 2024-06-10T01:27:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.575+0000 7f7763be6b80 1 --2- 172.21.15.7:0/176740530 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x563043266210 0x5630432e7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.575+0000 7f7763be6b80 4 client.20502 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:27:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.575+0000 7f7763be6b80 1 -- 172.21.15.7:0/176740530 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5630431b9290 con 0x563043266210 2024-06-10T01:27:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.576+0000 7f77527fc700 1 --2- 172.21.15.7:0/176740530 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x563043266210 0x5630432e7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:02.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.576+0000 7f77527fc700 1 --2- 172.21.15.7:0/176740530 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x563043266210 0x5630432e7020 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=320ce1d28f48b4fc server_cookie=d17c6b50b8e49e23 in_seq=0 out_seq=0 2024-06-10T01:27:02.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.577+0000 7f7736ff5700 10 client.20502 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:27:02.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.580+0000 7f7736ff5700 1 -- 172.21.15.7:0/176740530 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5630431b9290 con 0x563043266210 2024-06-10T01:27:02.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.580+0000 7f7736ff5700 10 client.20502 handle_command_reply: tid=0 2024-06-10T01:27:02.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.581+0000 7f7763be6b80 1 client.20502 shutdown 2024-06-10T01:27:02.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.586+0000 7f7763be6b80 1 -- 172.21.15.7:0/176740530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56304318acc0 msgr2=0x563043281450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:02.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.586+0000 7f7763be6b80 1 --2- 172.21.15.7:0/176740530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56304318acc0 0x563043281450 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f7744015000 tx=0x7f7744005e00 comp rx=0 tx=0).stop 2024-06-10T01:27:02.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.590+0000 7f7763be6b80 1 -- 172.21.15.7:0/176740530 shutdown_connections 2024-06-10T01:27:02.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.590+0000 7f7763be6b80 1 --2- 172.21.15.7:0/176740530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630432658f0 0x563043281ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.590+0000 7f7763be6b80 1 --2- 172.21.15.7:0/176740530 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x563043266210 0x5630432e7020 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.590+0000 7f7763be6b80 1 --2- 172.21.15.7:0/176740530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56304318acc0 0x563043281450 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.590+0000 7f7763be6b80 1 --2- 172.21.15.7:0/176740530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563043260080 0x563043281990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.590+0000 7f7763be6b80 1 -- 172.21.15.7:0/176740530 >> 172.21.15.7:0/176740530 conn(0x563043263a80 msgr2=0x563043266620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:02.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.590+0000 7f7736ff5700 0 client.20502 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:27:02.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.592+0000 7f7763be6b80 1 -- 172.21.15.7:0/176740530 shutdown_connections 2024-06-10T01:27:02.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.592+0000 7f7763be6b80 1 -- 172.21.15.7:0/176740530 wait complete. 2024-06-10T01:27:02.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.593+0000 7f7763be6b80 20 client.20502 trim_cache size 0 max 16384 2024-06-10T01:27:02.587 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:27:02.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.594+0000 7f7753fff700 1 -- 172.21.15.7:0/3833717804 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f773c0a2c90 msgr2=0x7f773c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:02.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.594+0000 7f7753fff700 1 --2- 172.21.15.7:0/3833717804 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f773c0a2c90 0x7f773c0a5150 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7748013da0 tx=0x7f7748009000 comp rx=0 tx=0).stop 2024-06-10T01:27:02.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.594+0000 7f7753fff700 1 -- 172.21.15.7:0/3833717804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7754110f70 msgr2=0x7f7754116d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:02.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.594+0000 7f7753fff700 1 --2- 172.21.15.7:0/3833717804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7754110f70 0x7f7754116d70 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f774c00b970 tx=0x7f774c00bd30 comp rx=0 tx=0).stop 2024-06-10T01:27:02.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.595+0000 7f7753fff700 1 -- 172.21.15.7:0/3833717804 shutdown_connections 2024-06-10T01:27:02.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.595+0000 7f7753fff700 1 --2- 172.21.15.7:0/3833717804 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f773c0a2c90 0x7f773c0a5150 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.595+0000 7f7753fff700 1 --2- 172.21.15.7:0/3833717804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7754110f70 0x7f7754116d70 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.595+0000 7f7753fff700 1 --2- 172.21.15.7:0/3833717804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7754109ff0 0x7f775411d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.595+0000 7f7753fff700 1 --2- 172.21.15.7:0/3833717804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7754110b90 0x7f7754116830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.595+0000 7f7753fff700 1 -- 172.21.15.7:0/3833717804 >> 172.21.15.7:0/3833717804 conn(0x7f775406d4c0 msgr2=0x7f7754072340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:02.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.596+0000 7f7753fff700 1 -- 172.21.15.7:0/3833717804 shutdown_connections 2024-06-10T01:27:02.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:02.596+0000 7f7753fff700 1 -- 172.21.15.7:0/3833717804 wait complete. 2024-06-10T01:27:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.675+0000 7fb3717fa640 1 -- 172.21.15.7:0/1752662442 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d768c00c20 con 0x55d768d10e00 2024-06-10T01:27:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.675+0000 7fb3717fa640 10 client.20487 handle_command_reply: tid=0 2024-06-10T01:27:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.675+0000 7fb392df6740 1 client.20487 shutdown 2024-06-10T01:27:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.676+0000 7fb392df6740 1 -- 172.21.15.7:0/1752662442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d768c90010 msgr2=0x55d768ca16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.676+0000 7fb392df6740 1 --2- 172.21.15.7:0/1752662442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d768c90010 0x55d768ca16a0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x55d768ca92e0 tx=0x7fb378016900 comp rx=0 tx=0).stop 2024-06-10T01:27:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.676+0000 7fb392df6740 1 -- 172.21.15.7:0/1752662442 shutdown_connections 2024-06-10T01:27:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.676+0000 7fb392df6740 1 --2- 172.21.15.7:0/1752662442 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d768d10e00 0x55d768d13220 crc :-1 s=READY pgs=478 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.676+0000 7fb3717fa640 0 client.20487 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.676+0000 7fb392df6740 1 --2- 172.21.15.7:0/1752662442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d768c95b70 0x55d768c9e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.676+0000 7fb392df6740 1 --2- 172.21.15.7:0/1752662442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d768c90a00 0x55d768c9e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.676+0000 7fb392df6740 1 --2- 172.21.15.7:0/1752662442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d768c90010 0x55d768ca16a0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.676+0000 7fb392df6740 1 -- 172.21.15.7:0/1752662442 >> 172.21.15.7:0/1752662442 conn(0x55d768c8b680 msgr2=0x55d768c91c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.676+0000 7fb392df6740 1 -- 172.21.15.7:0/1752662442 shutdown_connections 2024-06-10T01:27:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.677+0000 7fb392df6740 1 -- 172.21.15.7:0/1752662442 wait complete. 2024-06-10T01:27:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.677+0000 7fb392df6740 20 client.20487 trim_cache size 0 max 16384 2024-06-10T01:27:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:27:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:27:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:27:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.678+0000 7fb3900ee640 1 -- 172.21.15.7:0/2475802944 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb3740a2b80 msgr2=0x7fb3740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.678+0000 7fb3900ee640 1 --2- 172.21.15.7:0/2475802944 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb3740a2b80 0x7fb3740a5040 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb3800b97d0 tx=0x7fb3800879d0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.678+0000 7fb3900ee640 1 -- 172.21.15.7:0/2475802944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3881035d0 msgr2=0x7fb388072d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.678+0000 7fb3900ee640 1 --2- 172.21.15.7:0/2475802944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3881035d0 0x7fb388072d70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb380031b80 tx=0x7fb380003ef0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.678+0000 7fb3900ee640 1 -- 172.21.15.7:0/2475802944 shutdown_connections 2024-06-10T01:27:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.678+0000 7fb3900ee640 1 --2- 172.21.15.7:0/2475802944 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb3740a2b80 0x7fb3740a5040 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.678+0000 7fb3900ee640 1 --2- 172.21.15.7:0/2475802944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3881035d0 0x7fb388072d70 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.678+0000 7fb3900ee640 1 --2- 172.21.15.7:0/2475802944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb388102c10 0x7fb38806bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.678+0000 7fb3900ee640 1 --2- 172.21.15.7:0/2475802944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb388101a20 0x7fb38806b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.678+0000 7fb3900ee640 1 -- 172.21.15.7:0/2475802944 >> 172.21.15.7:0/2475802944 conn(0x7fb3880fcf90 msgr2=0x7fb388105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.678+0000 7fb3900ee640 1 -- 172.21.15.7:0/2475802944 shutdown_connections 2024-06-10T01:27:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:02.678+0000 7fb3900ee640 1 -- 172.21.15.7:0/2475802944 wait complete. 2024-06-10T01:27:03.548 INFO:tasks.workunit.client.0.smithi007.stdout: 10 181651 22.34 MB/sec execute 340 sec latency 283.073 ms 2024-06-10T01:27:03.719 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:27:03.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:03 smithi078 ceph-mon[36683]: pgmap v1064: 129 pgs: 129 active+clean; 675 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.5 MiB/s wr, 340 op/s 2024-06-10T01:27:03.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:03 smithi175 ceph-mon[36636]: pgmap v1064: 129 pgs: 129 active+clean; 675 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.5 MiB/s wr, 340 op/s 2024-06-10T01:27:04.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:03 smithi007 ceph-mon[32398]: pgmap v1064: 129 pgs: 129 active+clean; 675 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.5 MiB/s wr, 340 op/s 2024-06-10T01:27:04.090 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:27:04.549 INFO:tasks.workunit.client.0.smithi007.stdout: 10 182181 22.38 MB/sec execute 341 sec latency 131.266 ms 2024-06-10T01:27:05.549 INFO:tasks.workunit.client.0.smithi007.stdout: 10 182776 22.41 MB/sec execute 342 sec latency 99.100 ms 2024-06-10T01:27:05.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:05 smithi078 ceph-mon[36683]: pgmap v1065: 129 pgs: 129 active+clean; 672 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 2.7 MiB/s wr, 363 op/s 2024-06-10T01:27:05.888 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:05 smithi007 ceph-mon[32398]: pgmap v1065: 129 pgs: 129 active+clean; 672 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 2.7 MiB/s wr, 363 op/s 2024-06-10T01:27:05.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:05 smithi175 ceph-mon[36636]: pgmap v1065: 129 pgs: 129 active+clean; 672 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 2.7 MiB/s wr, 363 op/s 2024-06-10T01:27:06.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.054+0000 7f4ae6a4b700 1 -- 172.21.15.7:0/2715004158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ae01030f0 msgr2=0x7f4ae0107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:06.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.054+0000 7f4ae6a4b700 1 --2- 172.21.15.7:0/2715004158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ae01030f0 0x7f4ae0107110 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f4ac80099c0 tx=0x7f4ac8009cd0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.054+0000 7f4ae6a4b700 1 -- 172.21.15.7:0/2715004158 shutdown_connections 2024-06-10T01:27:06.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.054+0000 7f4ae6a4b700 1 --2- 172.21.15.7:0/2715004158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ae0107840 0x7f4ae010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.054+0000 7f4ae6a4b700 1 --2- 172.21.15.7:0/2715004158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ae0102740 0x7f4ae0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.054+0000 7f4ae6a4b700 1 --2- 172.21.15.7:0/2715004158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ae01030f0 0x7f4ae0107110 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.054+0000 7f4ae6a4b700 1 -- 172.21.15.7:0/2715004158 >> 172.21.15.7:0/2715004158 conn(0x7f4ae00795f0 msgr2=0x7f4ae0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:06.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.055+0000 7f4ae6a4b700 1 -- 172.21.15.7:0/2715004158 shutdown_connections 2024-06-10T01:27:06.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.055+0000 7f4ae6a4b700 1 -- 172.21.15.7:0/2715004158 wait complete. 2024-06-10T01:27:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.056+0000 7f4ae6a4b700 1 Processor -- start 2024-06-10T01:27:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.056+0000 7f4ae6a4b700 1 -- start start 2024-06-10T01:27:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.057+0000 7f4ae6a4b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ae0102740 0x7f4ae01093d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.057+0000 7f4ae6a4b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ae01030f0 0x7f4ae0109910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.057+0000 7f4ae6a4b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ae0107840 0x7f4ae0109e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.057+0000 7f4ae6a4b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae0112750 con 0x7f4ae0107840 2024-06-10T01:27:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.057+0000 7f4ae6a4b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae01125d0 con 0x7f4ae01030f0 2024-06-10T01:27:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.057+0000 7f4ae6a4b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae01128d0 con 0x7f4ae0102740 2024-06-10T01:27:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.057+0000 7f4ae4fe8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ae0107840 0x7f4ae0109e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.057+0000 7f4ad7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ae01030f0 0x7f4ae0109910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:06.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.057+0000 7f4ae4fe8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ae0107840 0x7f4ae0109e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56854/0 (socket says 172.21.15.7:56854) 2024-06-10T01:27:06.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.057+0000 7f4ae4fe8700 1 -- 172.21.15.7:0/643448241 learned_addr learned my addr 172.21.15.7:0/643448241 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:06.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.057+0000 7f4adffff700 1 --2- 172.21.15.7:0/643448241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ae0102740 0x7f4ae01093d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.058+0000 7f4ae4fe8700 1 -- 172.21.15.7:0/643448241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ae0102740 msgr2=0x7f4ae01093d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.058+0000 7f4ae4fe8700 1 --2- 172.21.15.7:0/643448241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ae0102740 0x7f4ae01093d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.058+0000 7f4ae4fe8700 1 -- 172.21.15.7:0/643448241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ae01030f0 msgr2=0x7f4ae0109910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.058+0000 7f4ae4fe8700 1 --2- 172.21.15.7:0/643448241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ae01030f0 0x7f4ae0109910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.058+0000 7f4ae4fe8700 1 -- 172.21.15.7:0/643448241 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae01ab440 con 0x7f4ae0107840 2024-06-10T01:27:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.058+0000 7f4ad7fff700 1 --2- 172.21.15.7:0/643448241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ae01030f0 0x7f4ae0109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:27:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.058+0000 7f4ae4fe8700 1 --2- 172.21.15.7:0/643448241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ae0107840 0x7f4ae0109e50 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8007f10 tx=0x7f4ad800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.059+0000 7f4addffb700 1 -- 172.21.15.7:0/643448241 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8004bb0 con 0x7f4ae0107840 2024-06-10T01:27:06.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.059+0000 7f4addffb700 1 -- 172.21.15.7:0/643448241 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ad8004d50 con 0x7f4ae0107840 2024-06-10T01:27:06.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.059+0000 7f4ae6a4b700 1 -- 172.21.15.7:0/643448241 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ae01ab730 con 0x7f4ae0107840 2024-06-10T01:27:06.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.059+0000 7f4addffb700 1 -- 172.21.15.7:0/643448241 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad80059f0 con 0x7f4ae0107840 2024-06-10T01:27:06.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.059+0000 7f4ae6a4b700 1 -- 172.21.15.7:0/643448241 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ae01abbf0 con 0x7f4ae0107840 2024-06-10T01:27:06.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.062+0000 7f4addffb700 1 -- 172.21.15.7:0/643448241 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f4ad801a020 con 0x7f4ae0107840 2024-06-10T01:27:06.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.063+0000 7f4addffb700 1 --2- 172.21.15.7:0/643448241 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4acc0a2c50 0x7f4acc0a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:06.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.063+0000 7f4addffb700 1 -- 172.21.15.7:0/643448241 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f4ad80ceb20 con 0x7f4ae0107840 2024-06-10T01:27:06.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.063+0000 7f4adffff700 1 --2- 172.21.15.7:0/643448241 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4acc0a2c50 0x7f4acc0a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:06.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.064+0000 7f4adffff700 1 --2- 172.21.15.7:0/643448241 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4acc0a2c50 0x7f4acc0a5110 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0009c60 tx=0x7f4ad0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:06.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.065+0000 7f4af12ceb80 1 Processor -- start 2024-06-10T01:27:06.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.065+0000 7f4af12ceb80 1 -- start start 2024-06-10T01:27:06.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.066+0000 7f4af12ceb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c710369640 0x55c710369a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:06.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.066+0000 7f4af12ceb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c710369f60 0x55c7103639e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:06.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.066+0000 7f4af12ceb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c710363f20 0x55c710364340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:06.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.066+0000 7f4af12ceb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c7102b5810 con 0x55c710369640 2024-06-10T01:27:06.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.066+0000 7f4af12ceb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c71025d720 con 0x55c710363f20 2024-06-10T01:27:06.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.066+0000 7f4af12ceb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c7102b17d0 con 0x55c710369f60 2024-06-10T01:27:06.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.066+0000 7f4adffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c710369640 0x55c710369a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:06.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.066+0000 7f4adffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c710369640 0x55c710369a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56870/0 (socket says 172.21.15.7:56870) 2024-06-10T01:27:06.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.066+0000 7f4adffff700 1 -- 172.21.15.7:0/1178456292 learned_addr learned my addr 172.21.15.7:0/1178456292 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:06.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.066+0000 7f4ae4fe8700 1 --2- 172.21.15.7:0/1178456292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c710363f20 0x55c710364340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:06.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.066+0000 7f4adffff700 1 -- 172.21.15.7:0/1178456292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c710369f60 msgr2=0x55c7103639e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:06.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.066+0000 7f4adffff700 1 --2- 172.21.15.7:0/1178456292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c710369f60 0x55c7103639e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.066+0000 7f4adffff700 1 -- 172.21.15.7:0/1178456292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c710363f20 msgr2=0x55c710364340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:06.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.066+0000 7f4adffff700 1 --2- 172.21.15.7:0/1178456292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c710363f20 0x55c710364340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.066+0000 7f4adffff700 1 -- 172.21.15.7:0/1178456292 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c710253f50 con 0x55c710369640 2024-06-10T01:27:06.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.067+0000 7f4adffff700 1 --2- 172.21.15.7:0/1178456292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c710369640 0x55c710369a20 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f4ad000e4e0 tx=0x7f4ad0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2780a8c241ac04 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:06.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.067+0000 7f4ad67fc700 1 -- 172.21.15.7:0/1178456292 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0026430 con 0x55c710369640 2024-06-10T01:27:06.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.067+0000 7f4ad67fc700 1 -- 172.21.15.7:0/1178456292 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ad001c8f0 con 0x55c710369640 2024-06-10T01:27:06.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.067+0000 7f4ad67fc700 1 -- 172.21.15.7:0/1178456292 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad001d9b0 con 0x55c710369640 2024-06-10T01:27:06.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.068+0000 7f4af12ceb80 1 -- 172.21.15.7:0/1178456292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c710369640 msgr2=0x55c710369a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:06.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.068+0000 7f4af12ceb80 1 --2- 172.21.15.7:0/1178456292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c710369640 0x55c710369a20 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f4ad000e4e0 tx=0x7f4ad0014000 comp rx=0 tx=0).stop 2024-06-10T01:27:06.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.068+0000 7f4af12ceb80 1 -- 172.21.15.7:0/1178456292 shutdown_connections 2024-06-10T01:27:06.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.068+0000 7f4af12ceb80 1 --2- 172.21.15.7:0/1178456292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c710363f20 0x55c710364340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.068+0000 7f4af12ceb80 1 --2- 172.21.15.7:0/1178456292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c710369640 0x55c710369a20 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.068+0000 7f4af12ceb80 1 --2- 172.21.15.7:0/1178456292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c710369f60 0x55c7103639e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.068+0000 7f4af12ceb80 1 -- 172.21.15.7:0/1178456292 >> 172.21.15.7:0/1178456292 conn(0x55c71028ecc0 msgr2=0x55c71035e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:06.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.068+0000 7f4af12ceb80 1 -- 172.21.15.7:0/1178456292 shutdown_connections 2024-06-10T01:27:06.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.069+0000 7f4af12ceb80 1 -- 172.21.15.7:0/1178456292 wait complete. 2024-06-10T01:27:06.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.069+0000 7f4af12ceb80 1 -- start start 2024-06-10T01:27:06.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.069+0000 7f4af12ceb80 1 Processor -- start 2024-06-10T01:27:06.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.070+0000 7f4af12ceb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c710363f20 0x55c71036fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:06.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.070+0000 7f4af12ceb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c710369640 0x55c71036d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:06.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.070+0000 7f4adffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c710363f20 0x55c71036fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:06.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.070+0000 7f4adffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c710363f20 0x55c71036fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43192/0 (socket says 172.21.15.7:43192) 2024-06-10T01:27:06.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.070+0000 7f4adffff700 1 -- 172.21.15.7:0/3550587061 learned_addr learned my addr 172.21.15.7:0/3550587061 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:06.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.070+0000 7f4ad7fff700 1 --2- 172.21.15.7:0/3550587061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c710369640 0x55c71036d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:06.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.071+0000 7f4af12ceb80 1 --2- 172.21.15.7:0/3550587061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c710369f60 0x55c71036d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:06.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.071+0000 7f4af12ceb80 1 -- 172.21.15.7:0/3550587061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c7103709d0 con 0x55c710369f60 2024-06-10T01:27:06.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.071+0000 7f4ae4fe8700 1 --2- 172.21.15.7:0/3550587061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c710369f60 0x55c71036d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:06.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.071+0000 7f4af12ceb80 1 -- 172.21.15.7:0/3550587061 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c710255bf0 con 0x55c710369640 2024-06-10T01:27:06.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.071+0000 7f4af12ceb80 1 -- 172.21.15.7:0/3550587061 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c710370b50 con 0x55c710363f20 2024-06-10T01:27:06.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.071+0000 7f4af12ceb80 10 client.0 _ll_register_callbacks cb 0x55c71024f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:27:06.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.071+0000 7f4ad7fff700 1 -- 172.21.15.7:0/3550587061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c710363f20 msgr2=0x55c71036fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:06.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.071+0000 7f4ad7fff700 1 --2- 172.21.15.7:0/3550587061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c710363f20 0x55c71036fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.071+0000 7f4ad7fff700 1 -- 172.21.15.7:0/3550587061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c710369f60 msgr2=0x55c71036d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:06.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.071+0000 7f4ad7fff700 1 --2- 172.21.15.7:0/3550587061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c710369f60 0x55c71036d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.071+0000 7f4ad7fff700 1 -- 172.21.15.7:0/3550587061 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7102048d0 con 0x55c710369640 2024-06-10T01:27:06.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.072+0000 7f4adffff700 1 --2- 172.21.15.7:0/3550587061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c710363f20 0x55c71036fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:27:06.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.072+0000 7f4ae4fe8700 1 --2- 172.21.15.7:0/3550587061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c710369f60 0x55c71036d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:27:06.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.072+0000 7f4ad7fff700 1 --2- 172.21.15.7:0/3550587061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c710369640 0x55c71036d050 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8009990 tx=0x7f4ac8003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=224029c63bed0baa server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:06.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.072+0000 7f4abf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:27:06.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.072+0000 7f4abf7fe700 1 -- 172.21.15.7:0/3550587061 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8028060 con 0x55c710369640 2024-06-10T01:27:06.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.073+0000 7f4abf7fe700 1 -- 172.21.15.7:0/3550587061 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f4ac8003c70 con 0x55c710369640 2024-06-10T01:27:06.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.073+0000 7f4af12ceb80 1 -- 172.21.15.7:0/3550587061 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ad001c8f0 con 0x55c710369640 2024-06-10T01:27:06.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.073+0000 7f4abf7fe700 1 -- 172.21.15.7:0/3550587061 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f4ac802f050 con 0x55c710369640 2024-06-10T01:27:06.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.073+0000 7f4af12ceb80 10 client.29705 fetch_fsmap learned FSMap version 52 2024-06-10T01:27:06.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.073+0000 7f4af12ceb80 1 -- 172.21.15.7:0/3550587061 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55c71037b1b0 con 0x55c710369640 2024-06-10T01:27:06.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.073+0000 7f4abf7fe700 1 -- 172.21.15.7:0/3550587061 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ad001c8f0 con 0x55c710369640 2024-06-10T01:27:06.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.073+0000 7f4abf7fe700 1 -- 172.21.15.7:0/3550587061 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f4ac800f760 con 0x55c710369640 2024-06-10T01:27:06.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.073+0000 7f4af12ceb80 10 client.29705 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:27:06.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.073+0000 7f4af12ceb80 10 client.29705 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:27:06.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.073+0000 7f4af12ceb80 20 client.29705 populate_metadata read hostname 'smithi007' 2024-06-10T01:27:06.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.073+0000 7f4af12ceb80 1 --2- 172.21.15.7:0/3550587061 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55c71037d7f0 0x55c71037fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:06.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.074+0000 7f4af12ceb80 4 client.29705 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:27:06.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.074+0000 7f4af12ceb80 1 -- 172.21.15.7:0/3550587061 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7102c2d10 con 0x55c71037d7f0 2024-06-10T01:27:06.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.074+0000 7f4adffff700 1 --2- 172.21.15.7:0/3550587061 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55c71037d7f0 0x55c71037fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:06.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.076+0000 7f4adffff700 1 --2- 172.21.15.7:0/3550587061 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55c71037d7f0 0x55c71037fbe0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=27ff69cdc2f96a3a server_cookie=606193f6caed93dc in_seq=0 out_seq=0 2024-06-10T01:27:06.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.077+0000 7f4abf7fe700 10 client.29705 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:27:06.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.082+0000 7f4abf7fe700 1 -- 172.21.15.7:0/3550587061 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c7102c2d10 con 0x55c71037d7f0 2024-06-10T01:27:06.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.082+0000 7f4abf7fe700 10 client.29705 handle_command_reply: tid=0 2024-06-10T01:27:06.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.082+0000 7f4af12ceb80 1 client.29705 shutdown 2024-06-10T01:27:06.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.082+0000 7f4af12ceb80 1 -- 172.21.15.7:0/3550587061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c710369640 msgr2=0x55c71036d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.082+0000 7f4af12ceb80 1 --2- 172.21.15.7:0/3550587061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c710369640 0x55c71036d050 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8009990 tx=0x7f4ac8003680 comp rx=0 tx=0).stop 2024-06-10T01:27:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.083+0000 7f4af12ceb80 1 -- 172.21.15.7:0/3550587061 shutdown_connections 2024-06-10T01:27:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.083+0000 7f4af12ceb80 1 --2- 172.21.15.7:0/3550587061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c710369f60 0x55c71036d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.083+0000 7f4af12ceb80 1 --2- 172.21.15.7:0/3550587061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c710363f20 0x55c71036fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.083+0000 7f4af12ceb80 1 --2- 172.21.15.7:0/3550587061 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55c71037d7f0 0x55c71037fbe0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.083+0000 7f4af12ceb80 1 --2- 172.21.15.7:0/3550587061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c710369640 0x55c71036d050 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.083+0000 7f4af12ceb80 1 -- 172.21.15.7:0/3550587061 >> 172.21.15.7:0/3550587061 conn(0x55c71028ecc0 msgr2=0x55c7103629d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.083+0000 7f4abf7fe700 0 client.29705 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:27:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.083+0000 7f4af12ceb80 1 -- 172.21.15.7:0/3550587061 shutdown_connections 2024-06-10T01:27:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.083+0000 7f4af12ceb80 1 -- 172.21.15.7:0/3550587061 wait complete. 2024-06-10T01:27:06.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.083+0000 7f4af12ceb80 20 client.29705 trim_cache size 0 max 16384 2024-06-10T01:27:06.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.109+0000 7f4af12ceb80 1 Processor -- start 2024-06-10T01:27:06.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.109+0000 7f4af12ceb80 1 -- start start 2024-06-10T01:27:06.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.110+0000 7f4af12ceb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c710363f20 0x55c7103639e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:06.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.110+0000 7f4af12ceb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c710369640 0x55c71036ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:06.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.110+0000 7f4af12ceb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c710369f60 0x55c710385440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:06.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.110+0000 7f4af12ceb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae01b71d0 con 0x55c710369640 2024-06-10T01:27:06.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.110+0000 7f4af12ceb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae01b4a60 con 0x55c710363f20 2024-06-10T01:27:06.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.110+0000 7f4adffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c710363f20 0x55c7103639e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:06.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.110+0000 7f4adffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c710363f20 0x55c7103639e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:34962/0 (socket says 172.21.15.7:34962) 2024-06-10T01:27:06.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.110+0000 7f4adffff700 1 -- 172.21.15.7:0/3500283647 learned_addr learned my addr 172.21.15.7:0/3500283647 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:06.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.110+0000 7f4ae4fe8700 1 --2- 172.21.15.7:0/3500283647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c710369f60 0x55c710385440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:06.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.110+0000 7f4af12ceb80 1 -- 172.21.15.7:0/3500283647 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae01b28b0 con 0x55c710369f60 2024-06-10T01:27:06.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.111+0000 7f4ad7fff700 1 --2- 172.21.15.7:0/3500283647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c710369640 0x55c71036ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:06.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.111+0000 7f4ae4fe8700 1 -- 172.21.15.7:0/3500283647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c710363f20 msgr2=0x55c7103639e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:06.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.111+0000 7f4ae4fe8700 1 --2- 172.21.15.7:0/3500283647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c710363f20 0x55c7103639e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.111+0000 7f4ae4fe8700 1 -- 172.21.15.7:0/3500283647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c710369640 msgr2=0x55c71036ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:06.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.111+0000 7f4ae4fe8700 1 --2- 172.21.15.7:0/3500283647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c710369640 0x55c71036ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.111+0000 7f4ae4fe8700 1 -- 172.21.15.7:0/3500283647 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c71036b050 con 0x55c710369f60 2024-06-10T01:27:06.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.112+0000 7f4ae4fe8700 1 --2- 172.21.15.7:0/3500283647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c710369f60 0x55c710385440 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800d700 tx=0x7f4ad800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25146ef22cae9149 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:06.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.112+0000 7f4ad5ffb700 1 -- 172.21.15.7:0/3500283647 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad809a900 con 0x55c710369f60 2024-06-10T01:27:06.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.113+0000 7f4ad5ffb700 1 -- 172.21.15.7:0/3500283647 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ad809b6a0 con 0x55c710369f60 2024-06-10T01:27:06.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.113+0000 7f4ad5ffb700 1 -- 172.21.15.7:0/3500283647 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad80b9e40 con 0x55c710369f60 2024-06-10T01:27:06.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.114+0000 7f4af12ceb80 1 -- 172.21.15.7:0/3500283647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c710369f60 msgr2=0x55c710385440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:06.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.114+0000 7f4af12ceb80 1 --2- 172.21.15.7:0/3500283647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c710369f60 0x55c710385440 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800d700 tx=0x7f4ad800b040 comp rx=0 tx=0).stop 2024-06-10T01:27:06.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.114+0000 7f4af12ceb80 1 -- 172.21.15.7:0/3500283647 shutdown_connections 2024-06-10T01:27:06.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.114+0000 7f4af12ceb80 1 --2- 172.21.15.7:0/3500283647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c710369f60 0x55c710385440 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.114+0000 7f4af12ceb80 1 --2- 172.21.15.7:0/3500283647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c710363f20 0x55c7103639e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.115+0000 7f4af12ceb80 1 --2- 172.21.15.7:0/3500283647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c710369640 0x55c71036ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.115+0000 7f4af12ceb80 1 -- 172.21.15.7:0/3500283647 >> 172.21.15.7:0/3500283647 conn(0x55c71028ecc0 msgr2=0x55c71036a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:06.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.116+0000 7f4af12ceb80 1 -- 172.21.15.7:0/3500283647 shutdown_connections 2024-06-10T01:27:06.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.116+0000 7f4af12ceb80 1 -- 172.21.15.7:0/3500283647 wait complete. 2024-06-10T01:27:06.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.116+0000 7f4af12ceb80 1 -- start start 2024-06-10T01:27:06.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.116+0000 7f4af12ceb80 1 Processor -- start 2024-06-10T01:27:06.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.117+0000 7f4af12ceb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c710363f20 0x55c7103762c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:06.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.117+0000 7f4af12ceb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c710369640 0x55c7103849f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:06.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.117+0000 7f4af12ceb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c710369f60 0x55c710384f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:06.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.117+0000 7f4af12ceb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae01b2f20 con 0x55c710363f20 2024-06-10T01:27:06.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.118+0000 7f4af12ceb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae01b4390 con 0x55c710369640 2024-06-10T01:27:06.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.118+0000 7f4af12ceb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae01b35f0 con 0x55c710369f60 2024-06-10T01:27:06.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.118+0000 7f4af12ceb80 10 client.0 _ll_register_callbacks cb 0x55c71024f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:27:06.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.118+0000 7f4adffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c710363f20 0x55c7103762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:06.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.118+0000 7f4adffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c710363f20 0x55c7103762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56898/0 (socket says 172.21.15.7:56898) 2024-06-10T01:27:06.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.118+0000 7f4adffff700 1 -- 172.21.15.7:0/3861555889 learned_addr learned my addr 172.21.15.7:0/3861555889 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:06.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.119+0000 7f4ad7fff700 1 --2- 172.21.15.7:0/3861555889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c710369640 0x55c7103849f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:06.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.119+0000 7f4adffff700 1 -- 172.21.15.7:0/3861555889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c710369f60 msgr2=0x55c710384f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:06.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.119+0000 7f4adffff700 1 --2- 172.21.15.7:0/3861555889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c710369f60 0x55c710384f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.119+0000 7f4adffff700 1 -- 172.21.15.7:0/3861555889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c710369640 msgr2=0x55c7103849f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:06.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.119+0000 7f4adffff700 1 --2- 172.21.15.7:0/3861555889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c710369640 0x55c7103849f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.119+0000 7f4adffff700 1 -- 172.21.15.7:0/3861555889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c710385470 con 0x55c710363f20 2024-06-10T01:27:06.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.119+0000 7f4adffff700 1 --2- 172.21.15.7:0/3861555889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c710363f20 0x55c7103762c0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0009b20 tx=0x7f4ad001d4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593d85ccc741f665 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:06.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.119+0000 7f4abf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:27:06.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.120+0000 7f4abf7fe700 1 -- 172.21.15.7:0/3861555889 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0035040 con 0x55c710363f20 2024-06-10T01:27:06.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.120+0000 7f4abf7fe700 1 -- 172.21.15.7:0/3861555889 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f4ad0000c30 con 0x55c710363f20 2024-06-10T01:27:06.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.120+0000 7f4af12ceb80 1 -- 172.21.15.7:0/3861555889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ad809b6a0 con 0x55c710363f20 2024-06-10T01:27:06.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.121+0000 7f4abf7fe700 1 -- 172.21.15.7:0/3861555889 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f4ad0040050 con 0x55c710363f20 2024-06-10T01:27:06.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.121+0000 7f4af12ceb80 10 client.20520 fetch_fsmap learned FSMap version 52 2024-06-10T01:27:06.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.121+0000 7f4af12ceb80 1 -- 172.21.15.7:0/3861555889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55c7103e8180 con 0x55c710363f20 2024-06-10T01:27:06.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.121+0000 7f4abf7fe700 1 -- 172.21.15.7:0/3861555889 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ad000ecb0 con 0x55c710363f20 2024-06-10T01:27:06.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.121+0000 7f4abf7fe700 1 -- 172.21.15.7:0/3861555889 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f4ad001c5b0 con 0x55c710363f20 2024-06-10T01:27:06.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.122+0000 7f4af12ceb80 10 client.20520 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:27:06.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.122+0000 7f4af12ceb80 10 client.20520 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:27:06.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.122+0000 7f4af12ceb80 20 client.20520 populate_metadata read hostname 'smithi007' 2024-06-10T01:27:06.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.122+0000 7f4af12ceb80 1 --2- 172.21.15.7:0/3861555889 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55c7103e85b0 0x55c7103ea9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:06.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.122+0000 7f4af12ceb80 4 client.20520 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:27:06.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.122+0000 7f4af12ceb80 1 -- 172.21.15.7:0/3861555889 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f4ad809b6a0 con 0x55c7103e85b0 2024-06-10T01:27:06.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.125+0000 7f4ad7fff700 1 --2- 172.21.15.7:0/3861555889 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55c7103e85b0 0x55c7103ea9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:06.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.126+0000 7f4ad7fff700 1 --2- 172.21.15.7:0/3861555889 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55c7103e85b0 0x55c7103ea9a0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ebddf8214426609d server_cookie=b1f436aee8961f50 in_seq=0 out_seq=0 2024-06-10T01:27:06.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.126+0000 7f4abf7fe700 10 client.20520 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:27:06.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.131+0000 7f4abf7fe700 1 -- 172.21.15.7:0/3861555889 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f4ad809b6a0 con 0x55c7103e85b0 2024-06-10T01:27:06.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.131+0000 7f4abf7fe700 10 client.20520 handle_command_reply: tid=0 2024-06-10T01:27:06.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.131+0000 7f4af12ceb80 1 client.20520 shutdown 2024-06-10T01:27:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.134+0000 7f4af12ceb80 1 -- 172.21.15.7:0/3861555889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c710363f20 msgr2=0x55c7103762c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.134+0000 7f4af12ceb80 1 --2- 172.21.15.7:0/3861555889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c710363f20 0x55c7103762c0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0009b20 tx=0x7f4ad001d4b0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.135+0000 7f4af12ceb80 1 -- 172.21.15.7:0/3861555889 shutdown_connections 2024-06-10T01:27:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.135+0000 7f4af12ceb80 1 --2- 172.21.15.7:0/3861555889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c710369f60 0x55c710384f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.135+0000 7f4af12ceb80 1 --2- 172.21.15.7:0/3861555889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c710363f20 0x55c7103762c0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.135+0000 7f4af12ceb80 1 --2- 172.21.15.7:0/3861555889 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x55c7103e85b0 0x55c7103ea9a0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.135+0000 7f4abf7fe700 0 client.20520 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:27:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.135+0000 7f4af12ceb80 1 --2- 172.21.15.7:0/3861555889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c710369640 0x55c7103849f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.135+0000 7f4af12ceb80 1 -- 172.21.15.7:0/3861555889 >> 172.21.15.7:0/3861555889 conn(0x55c71028ecc0 msgr2=0x55c7103681d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:06.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.135+0000 7f4af12ceb80 1 -- 172.21.15.7:0/3861555889 shutdown_connections 2024-06-10T01:27:06.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.136+0000 7f4af12ceb80 1 -- 172.21.15.7:0/3861555889 wait complete. 2024-06-10T01:27:06.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.136+0000 7f4af12ceb80 20 client.20520 trim_cache size 0 max 16384 2024-06-10T01:27:06.130 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:27:06.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.138+0000 7f4ae6a4b700 1 -- 172.21.15.7:0/643448241 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4acc0a2c50 msgr2=0x7f4acc0a5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:06.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.138+0000 7f4ae6a4b700 1 --2- 172.21.15.7:0/643448241 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4acc0a2c50 0x7f4acc0a5110 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0009c60 tx=0x7f4ad0009260 comp rx=0 tx=0).stop 2024-06-10T01:27:06.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.138+0000 7f4ae6a4b700 1 -- 172.21.15.7:0/643448241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ae0107840 msgr2=0x7f4ae0109e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:06.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.138+0000 7f4ae6a4b700 1 --2- 172.21.15.7:0/643448241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ae0107840 0x7f4ae0109e50 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8007f10 tx=0x7f4ad800d3b0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.138+0000 7f4ae6a4b700 1 -- 172.21.15.7:0/643448241 shutdown_connections 2024-06-10T01:27:06.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.138+0000 7f4ae6a4b700 1 --2- 172.21.15.7:0/643448241 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4acc0a2c50 0x7f4acc0a5110 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.138+0000 7f4ae6a4b700 1 --2- 172.21.15.7:0/643448241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ae0107840 0x7f4ae0109e50 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.138+0000 7f4ae6a4b700 1 --2- 172.21.15.7:0/643448241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ae0102740 0x7f4ae01093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.138+0000 7f4ae6a4b700 1 --2- 172.21.15.7:0/643448241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ae01030f0 0x7f4ae0109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:06.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.138+0000 7f4ae6a4b700 1 -- 172.21.15.7:0/643448241 >> 172.21.15.7:0/643448241 conn(0x7f4ae00795f0 msgr2=0x7f4ae0101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:06.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.138+0000 7f4ae6a4b700 1 -- 172.21.15.7:0/643448241 shutdown_connections 2024-06-10T01:27:06.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:06.138+0000 7f4ae6a4b700 1 -- 172.21.15.7:0/643448241 wait complete. 2024-06-10T01:27:06.549 INFO:tasks.workunit.client.0.smithi007.stdout: 10 183313 22.41 MB/sec execute 343 sec latency 192.643 ms 2024-06-10T01:27:07.549 INFO:tasks.workunit.client.0.smithi007.stdout: 10 183917 22.45 MB/sec execute 344 sec latency 96.083 ms 2024-06-10T01:27:07.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:07 smithi078 ceph-mon[36683]: pgmap v1066: 129 pgs: 129 active+clean; 669 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.6 MiB/s wr, 546 op/s 2024-06-10T01:27:07.909 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:07 smithi007 ceph-mon[32398]: pgmap v1066: 129 pgs: 129 active+clean; 669 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.6 MiB/s wr, 546 op/s 2024-06-10T01:27:07.947 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:27:07.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:07 smithi175 ceph-mon[36636]: pgmap v1066: 129 pgs: 129 active+clean; 669 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.6 MiB/s wr, 546 op/s 2024-06-10T01:27:08.281 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:27:08.549 INFO:tasks.workunit.client.0.smithi007.stdout: 10 184691 22.49 MB/sec execute 345 sec latency 80.193 ms 2024-06-10T01:27:09.551 INFO:tasks.workunit.client.0.smithi007.stdout: 10 185459 22.58 MB/sec execute 346 sec latency 49.172 ms 2024-06-10T01:27:09.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:09 smithi007 ceph-mon[32398]: pgmap v1067: 129 pgs: 129 active+clean; 665 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.5 MiB/s wr, 562 op/s 2024-06-10T01:27:09.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:09 smithi078 ceph-mon[36683]: pgmap v1067: 129 pgs: 129 active+clean; 665 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.5 MiB/s wr, 562 op/s 2024-06-10T01:27:09.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:09 smithi175 ceph-mon[36636]: pgmap v1067: 129 pgs: 129 active+clean; 665 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.5 MiB/s wr, 562 op/s 2024-06-10T01:27:10.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.244+0000 7f7a70bbc700 1 -- 172.21.15.7:0/3957091368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a6c1084f0 msgr2=0x7f7a6c1088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:10.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.245+0000 7f7a70bbc700 1 --2- 172.21.15.7:0/3957091368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a6c1084f0 0x7f7a6c1088d0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c009990 tx=0x7f7a5c009ca0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.245+0000 7f7a70bbc700 1 -- 172.21.15.7:0/3957091368 shutdown_connections 2024-06-10T01:27:10.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.245+0000 7f7a70bbc700 1 --2- 172.21.15.7:0/3957091368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a6c06b3c0 0x7f7a6c10f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.245+0000 7f7a70bbc700 1 --2- 172.21.15.7:0/3957091368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a6c1084f0 0x7f7a6c1088d0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.245+0000 7f7a70bbc700 1 --2- 172.21.15.7:0/3957091368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a6c06aa00 0x7f7a6c06ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.245+0000 7f7a70bbc700 1 -- 172.21.15.7:0/3957091368 >> 172.21.15.7:0/3957091368 conn(0x7f7a6c078300 msgr2=0x7f7a6c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:10.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.245+0000 7f7a70bbc700 1 -- 172.21.15.7:0/3957091368 shutdown_connections 2024-06-10T01:27:10.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.245+0000 7f7a70bbc700 1 -- 172.21.15.7:0/3957091368 wait complete. 2024-06-10T01:27:10.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.246+0000 7f7a70bbc700 1 Processor -- start 2024-06-10T01:27:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.246+0000 7f7a70bbc700 1 -- start start 2024-06-10T01:27:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.246+0000 7f7a70bbc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a6c06aa00 0x7f7a6c101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.246+0000 7f7a70bbc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a6c06b3c0 0x7f7a6c101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.246+0000 7f7a70bbc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a6c1084f0 0x7f7a6c101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.246+0000 7f7a70bbc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c1127c0 con 0x7f7a6c06b3c0 2024-06-10T01:27:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.246+0000 7f7a70bbc700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c112640 con 0x7f7a6c06aa00 2024-06-10T01:27:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.246+0000 7f7a70bbc700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c112940 con 0x7f7a6c1084f0 2024-06-10T01:27:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.246+0000 7f7a6a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a6c06aa00 0x7f7a6c101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.246+0000 7f7a69d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a6c06b3c0 0x7f7a6c101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.246+0000 7f7a6a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a6c06aa00 0x7f7a6c101450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:34986/0 (socket says 172.21.15.7:34986) 2024-06-10T01:27:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.246+0000 7f7a6a59c700 1 -- 172.21.15.7:0/1634592088 learned_addr learned my addr 172.21.15.7:0/1634592088 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.246+0000 7f7a6ad9d700 1 --2- 172.21.15.7:0/1634592088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a6c1084f0 0x7f7a6c101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:10.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.247+0000 7f7a6a59c700 1 -- 172.21.15.7:0/1634592088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a6c1084f0 msgr2=0x7f7a6c101ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:10.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.247+0000 7f7a6a59c700 1 --2- 172.21.15.7:0/1634592088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a6c1084f0 0x7f7a6c101ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.247+0000 7f7a6a59c700 1 -- 172.21.15.7:0/1634592088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a6c06b3c0 msgr2=0x7f7a6c101990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:10.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.247+0000 7f7a6a59c700 1 --2- 172.21.15.7:0/1634592088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a6c06b3c0 0x7f7a6c101990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.247+0000 7f7a6a59c700 1 -- 172.21.15.7:0/1634592088 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a6c1a7160 con 0x7f7a6c06aa00 2024-06-10T01:27:10.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.247+0000 7f7a69d9b700 1 --2- 172.21.15.7:0/1634592088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a6c06b3c0 0x7f7a6c101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:27:10.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.247+0000 7f7a6a59c700 1 --2- 172.21.15.7:0/1634592088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a6c06aa00 0x7f7a6c101450 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c000c00 tx=0x7f7a5c00bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:10.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.247+0000 7f7a5b7fe700 1 -- 172.21.15.7:0/1634592088 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c027070 con 0x7f7a6c06aa00 2024-06-10T01:27:10.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.247+0000 7f7a5b7fe700 1 -- 172.21.15.7:0/1634592088 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a5c0056d0 con 0x7f7a6c06aa00 2024-06-10T01:27:10.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.247+0000 7f7a70bbc700 1 -- 172.21.15.7:0/1634592088 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a6c1a73f0 con 0x7f7a6c06aa00 2024-06-10T01:27:10.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.247+0000 7f7a70bbc700 1 -- 172.21.15.7:0/1634592088 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a6c1a7840 con 0x7f7a6c06aa00 2024-06-10T01:27:10.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.247+0000 7f7a5b7fe700 1 -- 172.21.15.7:0/1634592088 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c022e10 con 0x7f7a6c06aa00 2024-06-10T01:27:10.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.251+0000 7f7a5b7fe700 1 -- 172.21.15.7:0/1634592088 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f7a5c02a070 con 0x7f7a6c06aa00 2024-06-10T01:27:10.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.252+0000 7f7a7b43fb80 1 Processor -- start 2024-06-10T01:27:10.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.252+0000 7f7a5b7fe700 1 --2- 172.21.15.7:0/1634592088 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7a540a2ca0 0x7f7a540a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:10.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.252+0000 7f7a7b43fb80 1 -- start start 2024-06-10T01:27:10.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.252+0000 7f7a5b7fe700 1 -- 172.21.15.7:0/1634592088 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f7a5c0cff70 con 0x7f7a6c06aa00 2024-06-10T01:27:10.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.252+0000 7f7a6ad9d700 1 --2- 172.21.15.7:0/1634592088 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7a540a2ca0 0x7f7a540a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:10.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.252+0000 7f7a7b43fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654b419ee50 0x5654b419f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:10.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.252+0000 7f7a7b43fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654b419f770 0x5654b41991f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:10.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.252+0000 7f7a7b43fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5654b4199730 0x5654b4199b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:10.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.252+0000 7f7a7b43fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5654b40e2810 con 0x5654b419ee50 2024-06-10T01:27:10.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.252+0000 7f7a7b43fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5654b408a720 con 0x5654b419f770 2024-06-10T01:27:10.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.252+0000 7f7a7b43fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5654b40de7d0 con 0x5654b4199730 2024-06-10T01:27:10.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.252+0000 7f7a6a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654b419ee50 0x5654b419f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:10.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.252+0000 7f7a6a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654b419ee50 0x5654b419f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56922/0 (socket says 172.21.15.7:56922) 2024-06-10T01:27:10.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.253+0000 7f7a6a59c700 1 -- 172.21.15.7:0/2799059571 learned_addr learned my addr 172.21.15.7:0/2799059571 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:10.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.253+0000 7f7a69d9b700 1 --2- 172.21.15.7:0/2799059571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654b419f770 0x5654b41991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:10.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.253+0000 7f7a6ad9d700 1 --2- 172.21.15.7:0/2799059571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5654b4199730 0x5654b4199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:10.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.253+0000 7f7a6a59c700 1 -- 172.21.15.7:0/2799059571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5654b4199730 msgr2=0x5654b4199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:10.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.253+0000 7f7a6a59c700 1 --2- 172.21.15.7:0/2799059571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5654b4199730 0x5654b4199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.253+0000 7f7a6a59c700 1 -- 172.21.15.7:0/2799059571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654b419f770 msgr2=0x5654b41991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:10.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.253+0000 7f7a6a59c700 1 --2- 172.21.15.7:0/2799059571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654b419f770 0x5654b41991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.253+0000 7f7a6a59c700 1 -- 172.21.15.7:0/2799059571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654b4080f50 con 0x5654b419ee50 2024-06-10T01:27:10.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.253+0000 7f7a6a59c700 1 --2- 172.21.15.7:0/2799059571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654b419ee50 0x5654b419f230 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c09c180 tx=0x7f7a5c09c250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962c0a2e71f32286 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:10.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.254+0000 7f7a58ff9700 1 -- 172.21.15.7:0/2799059571 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c09bde0 con 0x5654b419ee50 2024-06-10T01:27:10.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.254+0000 7f7a58ff9700 1 -- 172.21.15.7:0/2799059571 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a5c09cb80 con 0x5654b419ee50 2024-06-10T01:27:10.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.254+0000 7f7a58ff9700 1 -- 172.21.15.7:0/2799059571 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c0a4e30 con 0x5654b419ee50 2024-06-10T01:27:10.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.254+0000 7f7a7b43fb80 1 -- 172.21.15.7:0/2799059571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654b419ee50 msgr2=0x5654b419f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:10.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.254+0000 7f7a7b43fb80 1 --2- 172.21.15.7:0/2799059571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654b419ee50 0x5654b419f230 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c09c180 tx=0x7f7a5c09c250 comp rx=0 tx=0).stop 2024-06-10T01:27:10.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.254+0000 7f7a6ad9d700 1 --2- 172.21.15.7:0/1634592088 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7a540a2ca0 0x7f7a540a5160 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f7a6400f5b0 tx=0x7f7a6400f970 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:10.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.255+0000 7f7a7b43fb80 1 -- 172.21.15.7:0/2799059571 shutdown_connections 2024-06-10T01:27:10.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.255+0000 7f7a7b43fb80 1 --2- 172.21.15.7:0/2799059571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5654b4199730 0x5654b4199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.255+0000 7f7a7b43fb80 1 --2- 172.21.15.7:0/2799059571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654b419ee50 0x5654b419f230 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.255+0000 7f7a7b43fb80 1 --2- 172.21.15.7:0/2799059571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654b419f770 0x5654b41991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.255+0000 7f7a7b43fb80 1 -- 172.21.15.7:0/2799059571 >> 172.21.15.7:0/2799059571 conn(0x5654b40bbcc0 msgr2=0x5654b418b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:10.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.255+0000 7f7a7b43fb80 1 -- 172.21.15.7:0/2799059571 shutdown_connections 2024-06-10T01:27:10.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.255+0000 7f7a7b43fb80 1 -- 172.21.15.7:0/2799059571 wait complete. 2024-06-10T01:27:10.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.255+0000 7f7a7b43fb80 1 -- start start 2024-06-10T01:27:10.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.255+0000 7f7a7b43fb80 1 Processor -- start 2024-06-10T01:27:10.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.255+0000 7f7a7b43fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654b4199730 0x5654b41a5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:10.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.255+0000 7f7a7b43fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5654b419ee50 0x5654b41a2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:10.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.255+0000 7f7a7b43fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654b419f770 0x5654b41a2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:10.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.255+0000 7f7a7b43fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5654b41b6f10 con 0x5654b419f770 2024-06-10T01:27:10.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.255+0000 7f7a7b43fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5654b4082bf0 con 0x5654b4199730 2024-06-10T01:27:10.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.255+0000 7f7a7b43fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5654b41b7090 con 0x5654b419ee50 2024-06-10T01:27:10.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.256+0000 7f7a7b43fb80 10 client.0 _ll_register_callbacks cb 0x5654b407c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:27:10.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.256+0000 7f7a69d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5654b419ee50 0x5654b41a2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:10.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.256+0000 7f7a69d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5654b419ee50 0x5654b41a2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43240/0 (socket says 172.21.15.7:43240) 2024-06-10T01:27:10.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.256+0000 7f7a69d9b700 1 -- 172.21.15.7:0/4077773662 learned_addr learned my addr 172.21.15.7:0/4077773662 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:10.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.256+0000 7f7a6ad9d700 1 --2- 172.21.15.7:0/4077773662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654b419f770 0x5654b41a2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:10.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.256+0000 7f7a6a59c700 1 --2- 172.21.15.7:0/4077773662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654b4199730 0x5654b41a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:10.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.256+0000 7f7a6a59c700 1 -- 172.21.15.7:0/4077773662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5654b419ee50 msgr2=0x5654b41a2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:10.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.256+0000 7f7a6a59c700 1 --2- 172.21.15.7:0/4077773662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5654b419ee50 0x5654b41a2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.256+0000 7f7a6a59c700 1 -- 172.21.15.7:0/4077773662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654b419f770 msgr2=0x5654b41a2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:10.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.256+0000 7f7a6a59c700 1 --2- 172.21.15.7:0/4077773662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654b419f770 0x5654b41a2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.256+0000 7f7a6a59c700 1 -- 172.21.15.7:0/4077773662 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654b40318d0 con 0x5654b4199730 2024-06-10T01:27:10.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.256+0000 7f7a6a59c700 1 --2- 172.21.15.7:0/4077773662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654b4199730 0x5654b41a5520 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c09c640 tx=0x7f7a5c0a4e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29243807ed24f7d0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:10.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.256+0000 7f7a51ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:27:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.256+0000 7f7a51ffb700 1 -- 172.21.15.7:0/4077773662 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c09c480 con 0x5654b4199730 2024-06-10T01:27:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.256+0000 7f7a7b43fb80 1 -- 172.21.15.7:0/4077773662 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a5c09cb80 con 0x5654b4199730 2024-06-10T01:27:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.256+0000 7f7a51ffb700 1 -- 172.21.15.7:0/4077773662 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f7a5c022680 con 0x5654b4199730 2024-06-10T01:27:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.257+0000 7f7a51ffb700 1 -- 172.21.15.7:0/4077773662 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f7a5c0a0830 con 0x5654b4199730 2024-06-10T01:27:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.257+0000 7f7a7b43fb80 10 client.29720 fetch_fsmap learned FSMap version 52 2024-06-10T01:27:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.257+0000 7f7a7b43fb80 1 -- 172.21.15.7:0/4077773662 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5654b41b6f10 con 0x5654b4199730 2024-06-10T01:27:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.257+0000 7f7a51ffb700 1 -- 172.21.15.7:0/4077773662 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a5c09cb80 con 0x5654b4199730 2024-06-10T01:27:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.257+0000 7f7a51ffb700 1 -- 172.21.15.7:0/4077773662 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f7a5c07c060 con 0x5654b4199730 2024-06-10T01:27:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.257+0000 7f7a7b43fb80 10 client.29720 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:27:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.257+0000 7f7a7b43fb80 10 client.29720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:27:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.257+0000 7f7a7b43fb80 20 client.29720 populate_metadata read hostname 'smithi007' 2024-06-10T01:27:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.257+0000 7f7a7b43fb80 1 --2- 172.21.15.7:0/4077773662 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5654b4190f50 0x5654b4193360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.257+0000 7f7a7b43fb80 4 client.29720 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:27:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.257+0000 7f7a7b43fb80 1 -- 172.21.15.7:0/4077773662 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654b40efd10 con 0x5654b4190f50 2024-06-10T01:27:10.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.257+0000 7f7a69d9b700 1 --2- 172.21.15.7:0/4077773662 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5654b4190f50 0x5654b4193360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:10.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.258+0000 7f7a51ffb700 10 client.29720 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:10.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.258+0000 7f7a69d9b700 1 --2- 172.21.15.7:0/4077773662 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5654b4190f50 0x5654b4193360 crc :-1 s=READY pgs=479 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f47c3c483ed2a28 server_cookie=64d0c06776ce4413 in_seq=0 out_seq=0 2024-06-10T01:27:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.263+0000 7f7a51ffb700 1 -- 172.21.15.7:0/4077773662 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5654b40efd10 con 0x5654b4190f50 2024-06-10T01:27:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.263+0000 7f7a51ffb700 10 client.29720 handle_command_reply: tid=0 2024-06-10T01:27:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.263+0000 7f7a7b43fb80 1 client.29720 shutdown 2024-06-10T01:27:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.264+0000 7f7a7b43fb80 1 -- 172.21.15.7:0/4077773662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654b4199730 msgr2=0x5654b41a5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.264+0000 7f7a7b43fb80 1 --2- 172.21.15.7:0/4077773662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654b4199730 0x5654b41a5520 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c09c640 tx=0x7f7a5c0a4e00 comp rx=0 tx=0).stop 2024-06-10T01:27:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.264+0000 7f7a7b43fb80 1 -- 172.21.15.7:0/4077773662 shutdown_connections 2024-06-10T01:27:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.264+0000 7f7a7b43fb80 1 --2- 172.21.15.7:0/4077773662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654b419f770 0x5654b41a2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.264+0000 7f7a7b43fb80 1 --2- 172.21.15.7:0/4077773662 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5654b4190f50 0x5654b4193360 crc :-1 s=READY pgs=479 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.264+0000 7f7a7b43fb80 1 --2- 172.21.15.7:0/4077773662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654b4199730 0x5654b41a5520 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.264+0000 7f7a51ffb700 0 client.29720 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:10.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.264+0000 7f7a7b43fb80 1 --2- 172.21.15.7:0/4077773662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5654b419ee50 0x5654b41a2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.264+0000 7f7a7b43fb80 1 -- 172.21.15.7:0/4077773662 >> 172.21.15.7:0/4077773662 conn(0x5654b40bbcc0 msgr2=0x5654b418b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:10.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.264+0000 7f7a7b43fb80 1 -- 172.21.15.7:0/4077773662 shutdown_connections 2024-06-10T01:27:10.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.264+0000 7f7a7b43fb80 1 -- 172.21.15.7:0/4077773662 wait complete. 2024-06-10T01:27:10.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.264+0000 7f7a7b43fb80 20 client.29720 trim_cache size 0 max 16384 2024-06-10T01:27:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.285+0000 7f7a7b43fb80 1 Processor -- start 2024-06-10T01:27:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.285+0000 7f7a7b43fb80 1 -- start start 2024-06-10T01:27:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.285+0000 7f7a7b43fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654b4190f50 0x5654b41a1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.285+0000 7f7a7b43fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5654b4199730 0x5654b41a1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.285+0000 7f7a7b43fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654b419ee50 0x5654b4198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.285+0000 7f7a7b43fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c011220 con 0x5654b419ee50 2024-06-10T01:27:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.285+0000 7f7a7b43fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c00eab0 con 0x5654b4190f50 2024-06-10T01:27:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.285+0000 7f7a7b43fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c00c900 con 0x5654b4199730 2024-06-10T01:27:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.285+0000 7f7a6a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654b4190f50 0x5654b41a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:10.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.285+0000 7f7a6a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654b4190f50 0x5654b41a1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:35012/0 (socket says 172.21.15.7:35012) 2024-06-10T01:27:10.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.285+0000 7f7a6a59c700 1 -- 172.21.15.7:0/1388706081 learned_addr learned my addr 172.21.15.7:0/1388706081 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:10.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.285+0000 7f7a6ad9d700 1 --2- 172.21.15.7:0/1388706081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654b419ee50 0x5654b4198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:10.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.285+0000 7f7a69d9b700 1 --2- 172.21.15.7:0/1388706081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5654b4199730 0x5654b41a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:10.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.286+0000 7f7a6a59c700 1 -- 172.21.15.7:0/1388706081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5654b4199730 msgr2=0x5654b41a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:10.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.286+0000 7f7a6a59c700 1 --2- 172.21.15.7:0/1388706081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5654b4199730 0x5654b41a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.286+0000 7f7a6a59c700 1 -- 172.21.15.7:0/1388706081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654b419ee50 msgr2=0x5654b4198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:10.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.286+0000 7f7a6a59c700 1 --2- 172.21.15.7:0/1388706081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654b419ee50 0x5654b4198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.286+0000 7f7a6a59c700 1 -- 172.21.15.7:0/1388706081 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654b41a5160 con 0x5654b4190f50 2024-06-10T01:27:10.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.286+0000 7f7a6a59c700 1 --2- 172.21.15.7:0/1388706081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654b4190f50 0x5654b41a1110 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c098e50 tx=0x7f7a5c09bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99e2dd9212f82b98 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:10.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.286+0000 7f7a53fff700 1 -- 172.21.15.7:0/1388706081 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c09c250 con 0x5654b4190f50 2024-06-10T01:27:10.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.286+0000 7f7a53fff700 1 -- 172.21.15.7:0/1388706081 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a5c09c480 con 0x5654b4190f50 2024-06-10T01:27:10.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.286+0000 7f7a53fff700 1 -- 172.21.15.7:0/1388706081 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c0cddb0 con 0x5654b4190f50 2024-06-10T01:27:10.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.287+0000 7f7a7b43fb80 1 -- 172.21.15.7:0/1388706081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654b4190f50 msgr2=0x5654b41a1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:10.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.287+0000 7f7a7b43fb80 1 --2- 172.21.15.7:0/1388706081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654b4190f50 0x5654b41a1110 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c098e50 tx=0x7f7a5c09bea0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.287+0000 7f7a7b43fb80 1 -- 172.21.15.7:0/1388706081 shutdown_connections 2024-06-10T01:27:10.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.287+0000 7f7a7b43fb80 1 --2- 172.21.15.7:0/1388706081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654b419ee50 0x5654b4198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.287+0000 7f7a7b43fb80 1 --2- 172.21.15.7:0/1388706081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654b4190f50 0x5654b41a1110 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.287+0000 7f7a7b43fb80 1 --2- 172.21.15.7:0/1388706081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5654b4199730 0x5654b41a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.287+0000 7f7a7b43fb80 1 -- 172.21.15.7:0/1388706081 >> 172.21.15.7:0/1388706081 conn(0x5654b40bbcc0 msgr2=0x5654b418b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:10.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.287+0000 7f7a7b43fb80 1 -- 172.21.15.7:0/1388706081 shutdown_connections 2024-06-10T01:27:10.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.287+0000 7f7a7b43fb80 1 -- 172.21.15.7:0/1388706081 wait complete. 2024-06-10T01:27:10.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.288+0000 7f7a7b43fb80 1 -- start start 2024-06-10T01:27:10.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.288+0000 7f7a7b43fb80 1 Processor -- start 2024-06-10T01:27:10.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.288+0000 7f7a7b43fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654b4190f50 0x5654b41d6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:10.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.288+0000 7f7a7b43fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654b4199730 0x5654b41bc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:10.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.289+0000 7f7a6a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654b4190f50 0x5654b41d6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:10.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.289+0000 7f7a6a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654b4190f50 0x5654b41d6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:35016/0 (socket says 172.21.15.7:35016) 2024-06-10T01:27:10.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.289+0000 7f7a69d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654b4199730 0x5654b41bc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.289+0000 7f7a69d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654b4199730 0x5654b41bc2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56946/0 (socket says 172.21.15.7:56946) 2024-06-10T01:27:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.289+0000 7f7a6a59c700 1 -- 172.21.15.7:0/3079764462 learned_addr learned my addr 172.21.15.7:0/3079764462 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.289+0000 7f7a7b43fb80 1 --2- 172.21.15.7:0/3079764462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5654b419ee50 0x5654b41bc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.289+0000 7f7a7b43fb80 1 -- 172.21.15.7:0/3079764462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c00cf70 con 0x5654b4199730 2024-06-10T01:27:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.289+0000 7f7a7b43fb80 1 -- 172.21.15.7:0/3079764462 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c00e3e0 con 0x5654b4190f50 2024-06-10T01:27:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.289+0000 7f7a7b43fb80 1 -- 172.21.15.7:0/3079764462 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c00d640 con 0x5654b419ee50 2024-06-10T01:27:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.289+0000 7f7a6a59c700 1 -- 172.21.15.7:0/3079764462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5654b419ee50 msgr2=0x5654b41bc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.289+0000 7f7a6ad9d700 1 --2- 172.21.15.7:0/3079764462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5654b419ee50 0x5654b41bc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.289+0000 7f7a6a59c700 1 --2- 172.21.15.7:0/3079764462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5654b419ee50 0x5654b41bc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.289+0000 7f7a6a59c700 1 -- 172.21.15.7:0/3079764462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654b4199730 msgr2=0x5654b41bc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.289+0000 7f7a6a59c700 1 --2- 172.21.15.7:0/3079764462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654b4199730 0x5654b41bc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.289+0000 7f7a6a59c700 1 -- 172.21.15.7:0/3079764462 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654b41d53a0 con 0x5654b4190f50 2024-06-10T01:27:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.289+0000 7f7a69d9b700 1 --2- 172.21.15.7:0/3079764462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654b4199730 0x5654b41bc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:27:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.289+0000 7f7a7b43fb80 10 client.0 _ll_register_callbacks cb 0x5654b407c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:27:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.289+0000 7f7a7b43fb80 1 -- 172.21.15.7:0/3079764462 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5654b40ea290 con 0x5654b4190f50 2024-06-10T01:27:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.290+0000 7f7a6a59c700 1 --2- 172.21.15.7:0/3079764462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654b4190f50 0x5654b41d6c40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c0988e0 tx=0x7f7a5c00bfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=172149ed38744088 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.290+0000 7f7a51ffb700 10 client.29726 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:27:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.290+0000 7f7a51ffb700 1 -- 172.21.15.7:0/3079764462 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c09c250 con 0x5654b4190f50 2024-06-10T01:27:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.290+0000 7f7a51ffb700 1 -- 172.21.15.7:0/3079764462 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f7a5c0983d0 con 0x5654b4190f50 2024-06-10T01:27:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.290+0000 7f7a51ffb700 1 -- 172.21.15.7:0/3079764462 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f7a5c09cb80 con 0x5654b4190f50 2024-06-10T01:27:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.290+0000 7f7a7b43fb80 10 client.29726 fetch_fsmap learned FSMap version 52 2024-06-10T01:27:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.290+0000 7f7a7b43fb80 1 -- 172.21.15.7:0/3079764462 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=52}) -- 0x5654b41d53a0 con 0x5654b4190f50 2024-06-10T01:27:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.290+0000 7f7a51ffb700 1 -- 172.21.15.7:0/3079764462 <== mon.2 v2:172.21.15.78:3300/0 4 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f7a5c07c070 con 0x5654b4190f50 2024-06-10T01:27:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.290+0000 7f7a7b43fb80 10 client.29726 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:27:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.290+0000 7f7a7b43fb80 10 client.29726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:27:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.290+0000 7f7a7b43fb80 20 client.29726 populate_metadata read hostname 'smithi007' 2024-06-10T01:27:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.290+0000 7f7a7b43fb80 1 --2- 172.21.15.7:0/3079764462 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5654b419f770 0x5654b4238670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:10.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.290+0000 7f7a7b43fb80 4 client.29726 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:27:10.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.290+0000 7f7a7b43fb80 1 -- 172.21.15.7:0/3079764462 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5654b3f64250 con 0x5654b419f770 2024-06-10T01:27:10.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.291+0000 7f7a69d9b700 1 --2- 172.21.15.7:0/3079764462 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5654b419f770 0x5654b4238670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:10.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.291+0000 7f7a69d9b700 1 --2- 172.21.15.7:0/3079764462 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5654b419f770 0x5654b4238670 crc :-1 s=READY pgs=480 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80a807f22ca57177 server_cookie=5d8d8a7c78c9240c in_seq=0 out_seq=0 2024-06-10T01:27:10.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.291+0000 7f7a51ffb700 10 client.29726 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:10.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.295+0000 7f7a51ffb700 1 -- 172.21.15.7:0/3079764462 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5654b3f64250 con 0x5654b419f770 2024-06-10T01:27:10.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.295+0000 7f7a51ffb700 10 client.29726 handle_command_reply: tid=0 2024-06-10T01:27:10.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.295+0000 7f7a7b43fb80 1 client.29726 shutdown 2024-06-10T01:27:10.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.295+0000 7f7a7b43fb80 1 -- 172.21.15.7:0/3079764462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654b4190f50 msgr2=0x5654b41d6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:10.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.295+0000 7f7a7b43fb80 1 --2- 172.21.15.7:0/3079764462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654b4190f50 0x5654b41d6c40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c0988e0 tx=0x7f7a5c00bfb0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.295+0000 7f7a7b43fb80 1 -- 172.21.15.7:0/3079764462 shutdown_connections 2024-06-10T01:27:10.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.295+0000 7f7a7b43fb80 1 --2- 172.21.15.7:0/3079764462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5654b419ee50 0x5654b41bc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.295+0000 7f7a7b43fb80 1 --2- 172.21.15.7:0/3079764462 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5654b419f770 0x5654b4238670 crc :-1 s=READY pgs=480 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.295+0000 7f7a7b43fb80 1 --2- 172.21.15.7:0/3079764462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654b4190f50 0x5654b41d6c40 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.295+0000 7f7a7b43fb80 1 --2- 172.21.15.7:0/3079764462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654b4199730 0x5654b41bc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.295+0000 7f7a7b43fb80 1 -- 172.21.15.7:0/3079764462 >> 172.21.15.7:0/3079764462 conn(0x5654b40bbcc0 msgr2=0x5654b4184e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:10.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.295+0000 7f7a51ffb700 0 client.29726 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:10.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.295+0000 7f7a7b43fb80 1 -- 172.21.15.7:0/3079764462 shutdown_connections 2024-06-10T01:27:10.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.295+0000 7f7a7b43fb80 1 -- 172.21.15.7:0/3079764462 wait complete. 2024-06-10T01:27:10.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.295+0000 7f7a7b43fb80 20 client.29726 trim_cache size 0 max 16384 2024-06-10T01:27:10.290 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:27:10.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.296+0000 7f7a70bbc700 1 -- 172.21.15.7:0/1634592088 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7a540a2ca0 msgr2=0x7f7a540a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.296+0000 7f7a70bbc700 1 --2- 172.21.15.7:0/1634592088 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7a540a2ca0 0x7f7a540a5160 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f7a6400f5b0 tx=0x7f7a6400f970 comp rx=0 tx=0).stop 2024-06-10T01:27:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.296+0000 7f7a70bbc700 1 -- 172.21.15.7:0/1634592088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a6c06aa00 msgr2=0x7f7a6c101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.296+0000 7f7a70bbc700 1 --2- 172.21.15.7:0/1634592088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a6c06aa00 0x7f7a6c101450 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c000c00 tx=0x7f7a5c00bab0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.296+0000 7f7a70bbc700 1 -- 172.21.15.7:0/1634592088 shutdown_connections 2024-06-10T01:27:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.296+0000 7f7a70bbc700 1 --2- 172.21.15.7:0/1634592088 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7a540a2ca0 0x7f7a540a5160 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.296+0000 7f7a70bbc700 1 --2- 172.21.15.7:0/1634592088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a6c1084f0 0x7f7a6c101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.296+0000 7f7a70bbc700 1 --2- 172.21.15.7:0/1634592088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a6c06aa00 0x7f7a6c101450 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.296+0000 7f7a70bbc700 1 --2- 172.21.15.7:0/1634592088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a6c06b3c0 0x7f7a6c101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.296+0000 7f7a70bbc700 1 -- 172.21.15.7:0/1634592088 >> 172.21.15.7:0/1634592088 conn(0x7f7a6c078300 msgr2=0x7f7a6c0fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.296+0000 7f7a70bbc700 1 -- 172.21.15.7:0/1634592088 shutdown_connections 2024-06-10T01:27:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:10.296+0000 7f7a70bbc700 1 -- 172.21.15.7:0/1634592088 wait complete. 2024-06-10T01:27:10.550 INFO:tasks.workunit.client.0.smithi007.stdout: 10 185917 22.59 MB/sec execute 347 sec latency 79.105 ms 2024-06-10T01:27:11.550 INFO:tasks.workunit.client.0.smithi007.stdout: 10 186394 22.59 MB/sec execute 348 sec latency 97.082 ms 2024-06-10T01:27:11.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:11 smithi078 ceph-mon[36683]: pgmap v1068: 129 pgs: 129 active+clean; 671 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 3.9 MiB/s wr, 612 op/s 2024-06-10T01:27:11.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:11 smithi175 ceph-mon[36636]: pgmap v1068: 129 pgs: 129 active+clean; 671 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 3.9 MiB/s wr, 612 op/s 2024-06-10T01:27:12.001 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:11 smithi007 ceph-mon[32398]: pgmap v1068: 129 pgs: 129 active+clean; 671 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 3.9 MiB/s wr, 612 op/s 2024-06-10T01:27:12.013 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:27:12.013 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:27:12.358 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:27:12.551 INFO:tasks.workunit.client.0.smithi007.stdout: 10 187048 22.59 MB/sec execute 349 sec latency 102.999 ms 2024-06-10T01:27:12.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:12 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:27:12.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:12 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:27:13.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:12 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:27:13.551 INFO:tasks.workunit.client.0.smithi007.stdout: 10 187440 22.58 MB/sec execute 350 sec latency 329.370 ms 2024-06-10T01:27:13.854 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:13 smithi007 ceph-mon[32398]: pgmap v1069: 129 pgs: 129 active+clean; 678 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 3.5 MiB/s wr, 552 op/s 2024-06-10T01:27:13.854 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:13 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:27:13.854 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:13 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:27:13.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:13 smithi175 ceph-mon[36636]: pgmap v1069: 129 pgs: 129 active+clean; 678 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 3.5 MiB/s wr, 552 op/s 2024-06-10T01:27:13.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:13 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:27:13.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:13 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:27:14.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:13 smithi078 ceph-mon[36683]: pgmap v1069: 129 pgs: 129 active+clean; 678 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 3.5 MiB/s wr, 552 op/s 2024-06-10T01:27:14.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:13 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:27:14.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:13 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:27:14.551 INFO:tasks.workunit.client.0.smithi007.stdout: 10 187486 22.53 MB/sec execute 351 sec latency 1296.172 ms 2024-06-10T01:27:14.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.766+0000 7f39e5caf700 1 -- 172.21.15.7:0/571158360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e0110b80 msgr2=0x7f39e0109490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:14.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.766+0000 7f39e5caf700 1 --2- 172.21.15.7:0/571158360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e0110b80 0x7f39e0109490 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009990 tx=0x7f39d4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.768+0000 7f39e5caf700 1 -- 172.21.15.7:0/571158360 shutdown_connections 2024-06-10T01:27:14.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.768+0000 7f39e5caf700 1 --2- 172.21.15.7:0/571158360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39e011bc70 0x7f39e011e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.768+0000 7f39e5caf700 1 --2- 172.21.15.7:0/571158360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e0110b80 0x7f39e0109490 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.768+0000 7f39e5caf700 1 --2- 172.21.15.7:0/571158360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39e0110ff0 0x7f39e01099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.768+0000 7f39e5caf700 1 -- 172.21.15.7:0/571158360 >> 172.21.15.7:0/571158360 conn(0x7f39e006d470 msgr2=0x7f39e006d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:14.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.773+0000 7f39e5caf700 1 -- 172.21.15.7:0/571158360 shutdown_connections 2024-06-10T01:27:14.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.775+0000 7f39e5caf700 1 -- 172.21.15.7:0/571158360 wait complete. 2024-06-10T01:27:14.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.776+0000 7f39e5caf700 1 Processor -- start 2024-06-10T01:27:14.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.778+0000 7f39e5caf700 1 -- start start 2024-06-10T01:27:14.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.778+0000 7f39e5caf700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39e0110b80 0x7f39e01ad700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:14.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.778+0000 7f39e5caf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e0110ff0 0x7f39e01adc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:14.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.778+0000 7f39e5caf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39e011bc70 0x7f39e01a78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:14.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.778+0000 7f39e5caf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f39e0121640 con 0x7f39e0110ff0 2024-06-10T01:27:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.778+0000 7f39e5caf700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f39e01214c0 con 0x7f39e011bc70 2024-06-10T01:27:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.778+0000 7f39e5caf700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f39e01217c0 con 0x7f39e0110b80 2024-06-10T01:27:14.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.781+0000 7f39e4cad700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39e0110b80 0x7f39e01ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:14.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.781+0000 7f39e4cad700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39e0110b80 0x7f39e01ad700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:56080/0 (socket says 172.21.15.7:56080) 2024-06-10T01:27:14.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.781+0000 7f39e4cad700 1 -- 172.21.15.7:0/784627849 learned_addr learned my addr 172.21.15.7:0/784627849 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:14.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.785+0000 7f39e4cad700 1 -- 172.21.15.7:0/784627849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39e011bc70 msgr2=0x7f39e01a78c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-10T01:27:14.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.785+0000 7f39e4cad700 1 --2- 172.21.15.7:0/784627849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39e011bc70 0x7f39e01a78c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.785+0000 7f39e4cad700 1 -- 172.21.15.7:0/784627849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e0110ff0 msgr2=0x7f39e01adc40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T01:27:14.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.785+0000 7f39e4cad700 1 --2- 172.21.15.7:0/784627849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e0110ff0 0x7f39e01adc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.785+0000 7f39e4cad700 1 -- 172.21.15.7:0/784627849 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e01a8020 con 0x7f39e0110b80 2024-06-10T01:27:14.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.789+0000 7f39e4cad700 1 --2- 172.21.15.7:0/784627849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39e0110b80 0x7f39e01ad700 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f39d40057c0 tx=0x7f39d40057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:14.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.790+0000 7f39ddffb700 1 -- 172.21.15.7:0/784627849 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d4026070 con 0x7f39e0110b80 2024-06-10T01:27:14.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.790+0000 7f39ddffb700 1 -- 172.21.15.7:0/784627849 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d40040f0 con 0x7f39e0110b80 2024-06-10T01:27:14.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.790+0000 7f39ddffb700 1 -- 172.21.15.7:0/784627849 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d400fe10 con 0x7f39e0110b80 2024-06-10T01:27:14.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.790+0000 7f39e5caf700 1 -- 172.21.15.7:0/784627849 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e01a82b0 con 0x7f39e0110b80 2024-06-10T01:27:14.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.791+0000 7f39e5caf700 1 -- 172.21.15.7:0/784627849 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e01a8680 con 0x7f39e0110b80 2024-06-10T01:27:14.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.803+0000 7f39f1794b80 1 Processor -- start 2024-06-10T01:27:14.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.803+0000 7f39ddffb700 1 -- 172.21.15.7:0/784627849 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f39d4029070 con 0x7f39e0110b80 2024-06-10T01:27:14.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.805+0000 7f39f1794b80 1 -- start start 2024-06-10T01:27:14.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.805+0000 7f39f1794b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b920c4c970 0x55b920c4cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:14.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.805+0000 7f39f1794b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b920c4d290 0x55b920c46bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:14.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.805+0000 7f39f1794b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b920c47100 0x55b920c47540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:14.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.805+0000 7f39f1794b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b920b98890 con 0x55b920c47100 2024-06-10T01:27:14.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.805+0000 7f39f1794b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b920b407a0 con 0x55b920c4d290 2024-06-10T01:27:14.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.805+0000 7f39f1794b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b920b94850 con 0x55b920c4c970 2024-06-10T01:27:14.800 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:14 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:27:14.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.808+0000 7f39e54ae700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b920c4c970 0x55b920c4cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:14.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.808+0000 7f39e54ae700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b920c4c970 0x55b920c4cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:56084/0 (socket says 172.21.15.7:56084) 2024-06-10T01:27:14.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.808+0000 7f39e54ae700 1 -- 172.21.15.7:0/584951035 learned_addr learned my addr 172.21.15.7:0/584951035 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:14.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.808+0000 7f39e4cad700 1 --2- 172.21.15.7:0/584951035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b920c4d290 0x55b920c46bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:14.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.809+0000 7f39e4cad700 1 -- 172.21.15.7:0/584951035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b920c4c970 msgr2=0x55b920c4cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:14.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.809+0000 7f39e4cad700 1 --2- 172.21.15.7:0/584951035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b920c4c970 0x55b920c4cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.809+0000 7f39e4cad700 1 -- 172.21.15.7:0/584951035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b920c47100 msgr2=0x55b920c47540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:14.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.809+0000 7f39e4cad700 1 --2- 172.21.15.7:0/584951035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b920c47100 0x55b920c47540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.809+0000 7f39e4cad700 1 -- 172.21.15.7:0/584951035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b920b36fd0 con 0x55b920c4d290 2024-06-10T01:27:14.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.809+0000 7f39e4cad700 1 --2- 172.21.15.7:0/584951035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b920c4d290 0x55b920c46bc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f39d40cd3d0 tx=0x7f39d40d85f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=616862be92b5b2ba server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:14.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.810+0000 7f39ddffb700 1 --2- 172.21.15.7:0/784627849 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f39c80a2e20 0x7f39c80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:14.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.810+0000 7f39ddffb700 1 -- 172.21.15.7:0/784627849 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f39d40cfd10 con 0x7f39e0110b80 2024-06-10T01:27:14.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.813+0000 7f39c6ffd700 1 -- 172.21.15.7:0/584951035 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d40e2400 con 0x55b920c4d290 2024-06-10T01:27:14.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.813+0000 7f39c6ffd700 1 -- 172.21.15.7:0/584951035 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d40da8d0 con 0x55b920c4d290 2024-06-10T01:27:14.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.814+0000 7f39f1794b80 1 -- 172.21.15.7:0/584951035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b920c4d290 msgr2=0x55b920c46bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:14.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.814+0000 7f39f1794b80 1 --2- 172.21.15.7:0/584951035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b920c4d290 0x55b920c46bc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f39d40cd3d0 tx=0x7f39d40d85f0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.815+0000 7f39e54ae700 1 --2- 172.21.15.7:0/784627849 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f39c80a2e20 0x7f39c80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:14.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.815+0000 7f39f1794b80 1 -- 172.21.15.7:0/584951035 shutdown_connections 2024-06-10T01:27:14.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.815+0000 7f39f1794b80 1 --2- 172.21.15.7:0/584951035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b920c47100 0x55b920c47540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.815+0000 7f39f1794b80 1 --2- 172.21.15.7:0/584951035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b920c4c970 0x55b920c4cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.815+0000 7f39f1794b80 1 --2- 172.21.15.7:0/584951035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b920c4d290 0x55b920c46bc0 secure :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f39d40cd3d0 tx=0x7f39d40d85f0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.815+0000 7f39f1794b80 1 -- 172.21.15.7:0/584951035 >> 172.21.15.7:0/584951035 conn(0x55b920b71d40 msgr2=0x55b920c41b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:14.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.820+0000 7f39f1794b80 1 -- 172.21.15.7:0/584951035 shutdown_connections 2024-06-10T01:27:14.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.826+0000 7f39f1794b80 1 -- 172.21.15.7:0/584951035 wait complete. 2024-06-10T01:27:14.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.826+0000 7f39f1794b80 1 -- start start 2024-06-10T01:27:14.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.826+0000 7f39f1794b80 1 Processor -- start 2024-06-10T01:27:14.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.828+0000 7f39e54ae700 1 --2- 172.21.15.7:0/784627849 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f39c80a2e20 0x7f39c80a52e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f39d8009690 tx=0x7f39d8009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:14.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.830+0000 7f39f1794b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b920c47100 0x55b920c53240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:14.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.830+0000 7f39f1794b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b920c4c970 0x55b920c50590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:14.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.830+0000 7f39f1794b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b920c50c30 0x55b920c5cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:14.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.830+0000 7f39f1794b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b920c53970 con 0x55b920c47100 2024-06-10T01:27:14.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.830+0000 7f39f1794b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b920b38c70 con 0x55b920c50c30 2024-06-10T01:27:14.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.830+0000 7f39f1794b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b920c53af0 con 0x55b920c4c970 2024-06-10T01:27:14.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.830+0000 7f39f1794b80 10 client.0 _ll_register_callbacks cb 0x55b920b329a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:27:14.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.833+0000 7f39e4cad700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b920c47100 0x55b920c53240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:14.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.833+0000 7f39e4cad700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b920c47100 0x55b920c53240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55400/0 (socket says 172.21.15.7:55400) 2024-06-10T01:27:14.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.833+0000 7f39e4cad700 1 -- 172.21.15.7:0/4021817333 learned_addr learned my addr 172.21.15.7:0/4021817333 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:14.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.833+0000 7f39e54ae700 1 --2- 172.21.15.7:0/4021817333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b920c50c30 0x55b920c5cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:14.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.833+0000 7f39dffff700 1 --2- 172.21.15.7:0/4021817333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b920c4c970 0x55b920c50590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:14.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.835+0000 7f39e54ae700 1 -- 172.21.15.7:0/4021817333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b920c4c970 msgr2=0x55b920c50590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:14.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.835+0000 7f39e54ae700 1 --2- 172.21.15.7:0/4021817333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b920c4c970 0x55b920c50590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.835+0000 7f39e54ae700 1 -- 172.21.15.7:0/4021817333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b920c47100 msgr2=0x55b920c53240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:14.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.835+0000 7f39e54ae700 1 --2- 172.21.15.7:0/4021817333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b920c47100 0x55b920c53240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.836+0000 7f39e54ae700 1 -- 172.21.15.7:0/4021817333 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b920ae7950 con 0x55b920c50c30 2024-06-10T01:27:14.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.836+0000 7f39e54ae700 1 --2- 172.21.15.7:0/4021817333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b920c50c30 0x55b920c5cec0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f39d8007b70 tx=0x7f39d8004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=627f07de79c0cf18 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:14.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.838+0000 7f39bffff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:27:14.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.838+0000 7f39bffff700 1 -- 172.21.15.7:0/4021817333 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d8016b60 con 0x55b920c50c30 2024-06-10T01:27:14.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.838+0000 7f39bffff700 1 -- 172.21.15.7:0/4021817333 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f39d8016d00 con 0x55b920c50c30 2024-06-10T01:27:14.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.839+0000 7f39f1794b80 1 -- 172.21.15.7:0/4021817333 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b920ba5d90 con 0x55b920c50c30 2024-06-10T01:27:14.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.842+0000 7f39bffff700 1 -- 172.21.15.7:0/4021817333 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f39d8003a00 con 0x55b920c50c30 2024-06-10T01:27:14.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.844+0000 7f39f1794b80 10 client.29735 fetch_fsmap learned FSMap version 52 2024-06-10T01:27:14.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.844+0000 7f39f1794b80 1 -- 172.21.15.7:0/4021817333 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55b920c6fab0 con 0x55b920c50c30 2024-06-10T01:27:14.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.849+0000 7f39bffff700 1 -- 172.21.15.7:0/4021817333 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b920ba5d90 con 0x55b920c50c30 2024-06-10T01:27:14.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.850+0000 7f39bffff700 1 -- 172.21.15.7:0/4021817333 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f39d8021400 con 0x55b920c50c30 2024-06-10T01:27:14.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.851+0000 7f39f1794b80 10 client.29735 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:27:14.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.851+0000 7f39f1794b80 10 client.29735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:27:14.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.851+0000 7f39f1794b80 20 client.29735 populate_metadata read hostname 'smithi007' 2024-06-10T01:27:14.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.851+0000 7f39f1794b80 1 --2- 172.21.15.7:0/4021817333 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b920c724a0 0x55b920c748e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:14.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.851+0000 7f39f1794b80 4 client.29735 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:27:14.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.851+0000 7f39f1794b80 1 -- 172.21.15.7:0/4021817333 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b920ba0310 con 0x55b920c724a0 2024-06-10T01:27:14.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.856+0000 7f39e4cad700 1 --2- 172.21.15.7:0/4021817333 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b920c724a0 0x55b920c748e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:14.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.867+0000 7f39bffff700 10 client.29735 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:14.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.867+0000 7f39e4cad700 1 --2- 172.21.15.7:0/4021817333 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b920c724a0 0x55b920c748e0 crc :-1 s=READY pgs=481 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fd0332ffc622739 server_cookie=9d2549c6018a11f8 in_seq=0 out_seq=0 2024-06-10T01:27:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.875+0000 7f39bffff700 1 -- 172.21.15.7:0/4021817333 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b920ba0310 con 0x55b920c724a0 2024-06-10T01:27:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.876+0000 7f39bffff700 10 client.29735 handle_command_reply: tid=0 2024-06-10T01:27:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.876+0000 7f39f1794b80 1 client.29735 shutdown 2024-06-10T01:27:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.876+0000 7f39f1794b80 1 -- 172.21.15.7:0/4021817333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b920c50c30 msgr2=0x55b920c5cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.876+0000 7f39f1794b80 1 --2- 172.21.15.7:0/4021817333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b920c50c30 0x55b920c5cec0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f39d8007b70 tx=0x7f39d8004ad0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.876+0000 7f39f1794b80 1 -- 172.21.15.7:0/4021817333 shutdown_connections 2024-06-10T01:27:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.876+0000 7f39f1794b80 1 --2- 172.21.15.7:0/4021817333 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b920c724a0 0x55b920c748e0 crc :-1 s=READY pgs=481 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.876+0000 7f39f1794b80 1 --2- 172.21.15.7:0/4021817333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b920c50c30 0x55b920c5cec0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.876+0000 7f39f1794b80 1 --2- 172.21.15.7:0/4021817333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b920c47100 0x55b920c53240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.876+0000 7f39bffff700 0 client.29735 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.876+0000 7f39f1794b80 1 --2- 172.21.15.7:0/4021817333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b920c4c970 0x55b920c50590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.876+0000 7f39f1794b80 1 -- 172.21.15.7:0/4021817333 >> 172.21.15.7:0/4021817333 conn(0x55b920b71d40 msgr2=0x55b920c45bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:14.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.876+0000 7f39f1794b80 1 -- 172.21.15.7:0/4021817333 shutdown_connections 2024-06-10T01:27:14.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.876+0000 7f39f1794b80 1 -- 172.21.15.7:0/4021817333 wait complete. 2024-06-10T01:27:14.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.876+0000 7f39f1794b80 20 client.29735 trim_cache size 0 max 16384 2024-06-10T01:27:14.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.921+0000 7f39f1794b80 1 Processor -- start 2024-06-10T01:27:14.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.921+0000 7f39f1794b80 1 -- start start 2024-06-10T01:27:14.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.921+0000 7f39f1794b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b920c47100 0x55b920c41c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:14.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.921+0000 7f39f1794b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b920c4c970 0x55b920c5ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:14.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.921+0000 7f39f1794b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b920c50c30 0x55b920c7a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:14.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.921+0000 7f39f1794b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f39e01b6e70 con 0x55b920c47100 2024-06-10T01:27:14.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.921+0000 7f39f1794b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f39e01b4700 con 0x55b920c4c970 2024-06-10T01:27:14.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.921+0000 7f39f1794b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f39e0117c20 con 0x55b920c50c30 2024-06-10T01:27:14.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.922+0000 7f39e4cad700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b920c47100 0x55b920c41c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:14.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.922+0000 7f39e54ae700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b920c50c30 0x55b920c7a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:14.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.924+0000 7f39e54ae700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b920c50c30 0x55b920c7a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:56100/0 (socket says 172.21.15.7:56100) 2024-06-10T01:27:14.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.924+0000 7f39e54ae700 1 -- 172.21.15.7:0/3005136436 learned_addr learned my addr 172.21.15.7:0/3005136436 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:14.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.924+0000 7f39e54ae700 1 -- 172.21.15.7:0/3005136436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b920c4c970 msgr2=0x55b920c5ccb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T01:27:14.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.924+0000 7f39e54ae700 1 --2- 172.21.15.7:0/3005136436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b920c4c970 0x55b920c5ccb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.924+0000 7f39e54ae700 1 -- 172.21.15.7:0/3005136436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b920c47100 msgr2=0x55b920c41c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:14.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.924+0000 7f39e54ae700 1 --2- 172.21.15.7:0/3005136436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b920c47100 0x55b920c41c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.924+0000 7f39e54ae700 1 -- 172.21.15.7:0/3005136436 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b920c432f0 con 0x55b920c50c30 2024-06-10T01:27:14.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.925+0000 7f39e54ae700 1 --2- 172.21.15.7:0/3005136436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b920c50c30 0x55b920c7a1f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f39d8000f80 tx=0x7f39d8016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1673527d808c6975 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:14.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.925+0000 7f39c67fc700 1 -- 172.21.15.7:0/3005136436 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d8021400 con 0x55b920c50c30 2024-06-10T01:27:14.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.925+0000 7f39c67fc700 1 -- 172.21.15.7:0/3005136436 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d80215a0 con 0x55b920c50c30 2024-06-10T01:27:14.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.925+0000 7f39c67fc700 1 -- 172.21.15.7:0/3005136436 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d80366d0 con 0x55b920c50c30 2024-06-10T01:27:14.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.927+0000 7f39f1794b80 1 -- 172.21.15.7:0/3005136436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b920c50c30 msgr2=0x55b920c7a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:14.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.927+0000 7f39f1794b80 1 --2- 172.21.15.7:0/3005136436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b920c50c30 0x55b920c7a1f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f39d8000f80 tx=0x7f39d8016000 comp rx=0 tx=0).stop 2024-06-10T01:27:14.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.930+0000 7f39f1794b80 1 -- 172.21.15.7:0/3005136436 shutdown_connections 2024-06-10T01:27:14.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.930+0000 7f39f1794b80 1 --2- 172.21.15.7:0/3005136436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b920c50c30 0x55b920c7a1f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.930+0000 7f39f1794b80 1 --2- 172.21.15.7:0/3005136436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b920c47100 0x55b920c41c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.930+0000 7f39f1794b80 1 --2- 172.21.15.7:0/3005136436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b920c4c970 0x55b920c5ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.930+0000 7f39f1794b80 1 -- 172.21.15.7:0/3005136436 >> 172.21.15.7:0/3005136436 conn(0x55b920b71d40 msgr2=0x55b920c41560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:14.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.932+0000 7f39f1794b80 1 -- 172.21.15.7:0/3005136436 shutdown_connections 2024-06-10T01:27:14.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.932+0000 7f39f1794b80 1 -- 172.21.15.7:0/3005136436 wait complete. 2024-06-10T01:27:14.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.933+0000 7f39f1794b80 1 -- start start 2024-06-10T01:27:14.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.933+0000 7f39f1794b80 1 Processor -- start 2024-06-10T01:27:14.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.934+0000 7f39f1794b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b920c47100 0x55b920c58e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:14.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.934+0000 7f39f1794b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b920c4c970 0x55b920c59360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:14.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.934+0000 7f39e4cad700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b920c47100 0x55b920c58e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:14.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.934+0000 7f39e4cad700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b920c47100 0x55b920c58e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:39122/0 (socket says 172.21.15.7:39122) 2024-06-10T01:27:14.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.934+0000 7f39e4cad700 1 -- 172.21.15.7:0/2974993577 learned_addr learned my addr 172.21.15.7:0/2974993577 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:14.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.934+0000 7f39dffff700 1 --2- 172.21.15.7:0/2974993577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b920c4c970 0x55b920c59360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:14.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.934+0000 7f39f1794b80 1 --2- 172.21.15.7:0/2974993577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b920c50c30 0x55b920c55df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:14.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.935+0000 7f39f1794b80 1 -- 172.21.15.7:0/2974993577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f39e01181e0 con 0x55b920c50c30 2024-06-10T01:27:14.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.935+0000 7f39f1794b80 1 -- 172.21.15.7:0/2974993577 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f39e0119650 con 0x55b920c47100 2024-06-10T01:27:14.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.935+0000 7f39f1794b80 1 -- 172.21.15.7:0/2974993577 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f39e01188b0 con 0x55b920c4c970 2024-06-10T01:27:14.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.935+0000 7f39e4cad700 1 -- 172.21.15.7:0/2974993577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b920c4c970 msgr2=0x55b920c59360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:14.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.935+0000 7f39e4cad700 1 --2- 172.21.15.7:0/2974993577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b920c4c970 0x55b920c59360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.935+0000 7f39e4cad700 1 -- 172.21.15.7:0/2974993577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b920c50c30 msgr2=0x55b920c55df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:14.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.935+0000 7f39e4cad700 1 --2- 172.21.15.7:0/2974993577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b920c50c30 0x55b920c55df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.935+0000 7f39e4cad700 1 -- 172.21.15.7:0/2974993577 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b920c56330 con 0x55b920c47100 2024-06-10T01:27:14.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.935+0000 7f39dffff700 1 --2- 172.21.15.7:0/2974993577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b920c4c970 0x55b920c59360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:27:14.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.936+0000 7f39e4cad700 1 --2- 172.21.15.7:0/2974993577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b920c47100 0x55b920c58e20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f39d4098bf0 tx=0x7f39d4005ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3dfa5b71075e54e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:14.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.936+0000 7f39f1794b80 10 client.0 _ll_register_callbacks cb 0x55b920b329a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:27:14.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.936+0000 7f39f1794b80 1 -- 172.21.15.7:0/2974993577 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39d80215a0 con 0x55b920c47100 2024-06-10T01:27:14.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.936+0000 7f39bffff700 10 client.29738 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:27:14.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.936+0000 7f39bffff700 1 -- 172.21.15.7:0/2974993577 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d409bff0 con 0x55b920c47100 2024-06-10T01:27:14.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.936+0000 7f39bffff700 1 -- 172.21.15.7:0/2974993577 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f39d40d3200 con 0x55b920c47100 2024-06-10T01:27:14.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.937+0000 7f39bffff700 1 -- 172.21.15.7:0/2974993577 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f39d409bb80 con 0x55b920c47100 2024-06-10T01:27:14.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.937+0000 7f39f1794b80 10 client.29738 fetch_fsmap learned FSMap version 52 2024-06-10T01:27:14.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.937+0000 7f39f1794b80 1 -- 172.21.15.7:0/2974993577 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=52}) -- 0x55b920cdd810 con 0x55b920c47100 2024-06-10T01:27:14.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.941+0000 7f39bffff700 1 -- 172.21.15.7:0/2974993577 <== mon.2 v2:172.21.15.78:3300/0 4 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f39d40d6070 con 0x55b920c47100 2024-06-10T01:27:14.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.941+0000 7f39f1794b80 10 client.29738 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:27:14.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.941+0000 7f39f1794b80 10 client.29738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:27:14.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.941+0000 7f39f1794b80 20 client.29738 populate_metadata read hostname 'smithi007' 2024-06-10T01:27:14.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.941+0000 7f39f1794b80 1 --2- 172.21.15.7:0/2974993577 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b920cddc80 0x55b920ce0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:14.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.941+0000 7f39f1794b80 4 client.29738 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:27:14.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.941+0000 7f39f1794b80 1 -- 172.21.15.7:0/2974993577 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f39d80215a0 con 0x55b920cddc80 2024-06-10T01:27:14.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.943+0000 7f39dffff700 1 --2- 172.21.15.7:0/2974993577 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b920cddc80 0x55b920ce0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:14.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.955+0000 7f39dffff700 1 --2- 172.21.15.7:0/2974993577 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b920cddc80 0x55b920ce0120 crc :-1 s=READY pgs=482 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5515047dd961244b server_cookie=adf8db92a24e2f7d in_seq=0 out_seq=0 2024-06-10T01:27:14.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.958+0000 7f39bffff700 10 client.29738 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:14.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:14 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:27:14.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.969+0000 7f39bffff700 1 -- 172.21.15.7:0/2974993577 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f39d80215a0 con 0x55b920cddc80 2024-06-10T01:27:14.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.969+0000 7f39bffff700 10 client.29738 handle_command_reply: tid=0 2024-06-10T01:27:14.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.969+0000 7f39f1794b80 1 client.29738 shutdown 2024-06-10T01:27:14.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.971+0000 7f39f1794b80 1 -- 172.21.15.7:0/2974993577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b920c47100 msgr2=0x55b920c58e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:14.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.971+0000 7f39f1794b80 1 --2- 172.21.15.7:0/2974993577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b920c47100 0x55b920c58e20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f39d4098bf0 tx=0x7f39d4005ad0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.971+0000 7f39f1794b80 1 -- 172.21.15.7:0/2974993577 shutdown_connections 2024-06-10T01:27:14.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.971+0000 7f39f1794b80 1 --2- 172.21.15.7:0/2974993577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b920c50c30 0x55b920c55df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.971+0000 7f39f1794b80 1 --2- 172.21.15.7:0/2974993577 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55b920cddc80 0x55b920ce0120 crc :-1 s=READY pgs=482 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.971+0000 7f39f1794b80 1 --2- 172.21.15.7:0/2974993577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b920c47100 0x55b920c58e20 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.971+0000 7f39f1794b80 1 --2- 172.21.15.7:0/2974993577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b920c4c970 0x55b920c59360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.971+0000 7f39bffff700 0 client.29738 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:14.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.971+0000 7f39f1794b80 1 -- 172.21.15.7:0/2974993577 >> 172.21.15.7:0/2974993577 conn(0x55b920b71d40 msgr2=0x55b920c3afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:14.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.972+0000 7f39f1794b80 1 -- 172.21.15.7:0/2974993577 shutdown_connections 2024-06-10T01:27:14.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.972+0000 7f39f1794b80 1 -- 172.21.15.7:0/2974993577 wait complete. 2024-06-10T01:27:14.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.972+0000 7f39f1794b80 20 client.29738 trim_cache size 0 max 16384 2024-06-10T01:27:14.965 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:27:14.965 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-10T01:27:14.965 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "cb984063-c0ed-419c-ac2d-d464b9612156", 2024-06-10T01:27:14.965 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-10T01:27:14.965 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:27:14.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.973+0000 7f39e5caf700 1 -- 172.21.15.7:0/784627849 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f39c80a2e20 msgr2=0x7f39c80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:14.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.973+0000 7f39e5caf700 1 --2- 172.21.15.7:0/784627849 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f39c80a2e20 0x7f39c80a52e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f39d8009690 tx=0x7f39d8009040 comp rx=0 tx=0).stop 2024-06-10T01:27:14.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.973+0000 7f39e5caf700 1 -- 172.21.15.7:0/784627849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39e0110b80 msgr2=0x7f39e01ad700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:14.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.973+0000 7f39e5caf700 1 --2- 172.21.15.7:0/784627849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39e0110b80 0x7f39e01ad700 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f39d40057c0 tx=0x7f39d40057f0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.973+0000 7f39e5caf700 1 -- 172.21.15.7:0/784627849 shutdown_connections 2024-06-10T01:27:14.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.973+0000 7f39e5caf700 1 --2- 172.21.15.7:0/784627849 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f39c80a2e20 0x7f39c80a52e0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.973+0000 7f39e5caf700 1 --2- 172.21.15.7:0/784627849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39e011bc70 0x7f39e01a78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.973+0000 7f39e5caf700 1 --2- 172.21.15.7:0/784627849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39e0110b80 0x7f39e01ad700 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.973+0000 7f39e5caf700 1 --2- 172.21.15.7:0/784627849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e0110ff0 0x7f39e01adc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:14.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.973+0000 7f39e5caf700 1 -- 172.21.15.7:0/784627849 >> 172.21.15.7:0/784627849 conn(0x7f39e006d470 msgr2=0x7f39e010e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:14.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.974+0000 7f39e5caf700 1 -- 172.21.15.7:0/784627849 shutdown_connections 2024-06-10T01:27:14.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:14.974+0000 7f39e5caf700 1 -- 172.21.15.7:0/784627849 wait complete. 2024-06-10T01:27:15.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:14 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:27:15.551 INFO:tasks.workunit.client.0.smithi007.stdout: 10 187621 22.51 MB/sec execute 352 sec latency 1327.962 ms 2024-06-10T01:27:15.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:15 smithi007 ceph-mon[32398]: pgmap v1070: 129 pgs: 129 active+clean; 680 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.2 MiB/s wr, 499 op/s 2024-06-10T01:27:15.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:15 smithi007 ceph-mon[32398]: scrub queued for path: / 2024-06-10T01:27:15.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:15 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:27:15.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:15 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:27:15.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:15 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:27:15.822 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:27:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.852+0000 7fb666d1b640 1 -- 172.21.15.7:0/3438793823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb660109c00 msgr2=0x7fb660110490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.852+0000 7fb666d1b640 1 --2- 172.21.15.7:0/3438793823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb660109c00 0x7fb660110490 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb658009ca0 tx=0x7fb65802a8b0 comp rx=0 tx=0).stop 2024-06-10T01:27:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.855+0000 7fb666d1b640 1 -- 172.21.15.7:0/3438793823 shutdown_connections 2024-06-10T01:27:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.855+0000 7fb666d1b640 1 --2- 172.21.15.7:0/3438793823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb660109c00 0x7fb660110490 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.855+0000 7fb666d1b640 1 --2- 172.21.15.7:0/3438793823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb660109240 0x7fb6601096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.855+0000 7fb666d1b640 1 --2- 172.21.15.7:0/3438793823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6601080a0 0x7fb6601084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.856+0000 7fb666d1b640 1 -- 172.21.15.7:0/3438793823 >> 172.21.15.7:0/3438793823 conn(0x7fb660076ff0 msgr2=0x7fb660079430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.860+0000 7fb666d1b640 1 -- 172.21.15.7:0/3438793823 shutdown_connections 2024-06-10T01:27:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.862+0000 7fb666d1b640 1 -- 172.21.15.7:0/3438793823 wait complete. 2024-06-10T01:27:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.863+0000 7fb666d1b640 1 Processor -- start 2024-06-10T01:27:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.863+0000 7fb666d1b640 1 -- start start 2024-06-10T01:27:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.864+0000 7fb666d1b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6601080a0 0x7fb66007f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.864+0000 7fb666d1b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb660109240 0x7fb66007f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.864+0000 7fb666d1b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb660109c00 0x7fb66019d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.864+0000 7fb666d1b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb660113ce0 con 0x7fb6601080a0 2024-06-10T01:27:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.864+0000 7fb666d1b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb660113b60 con 0x7fb660109240 2024-06-10T01:27:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.864+0000 7fb666d1b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb660113e60 con 0x7fb660109c00 2024-06-10T01:27:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.865+0000 7fb6652b8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb660109c00 0x7fb66019d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.866+0000 7fb6652b8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb660109c00 0x7fb66019d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:56122/0 (socket says 172.21.15.7:56122) 2024-06-10T01:27:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.866+0000 7fb6652b8640 1 -- 172.21.15.7:0/25191171 learned_addr learned my addr 172.21.15.7:0/25191171 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.867+0000 7fb65ffff640 1 --2- 172.21.15.7:0/25191171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb660109240 0x7fb66007f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.867+0000 7fb664ab7640 1 --2- 172.21.15.7:0/25191171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6601080a0 0x7fb66007f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.867+0000 7fb65ffff640 1 -- 172.21.15.7:0/25191171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb660109c00 msgr2=0x7fb66019d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.867+0000 7fb65ffff640 1 --2- 172.21.15.7:0/25191171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb660109c00 0x7fb66019d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.867+0000 7fb65ffff640 1 -- 172.21.15.7:0/25191171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6601080a0 msgr2=0x7fb66007f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.867+0000 7fb65ffff640 1 --2- 172.21.15.7:0/25191171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6601080a0 0x7fb66007f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.867+0000 7fb65ffff640 1 -- 172.21.15.7:0/25191171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb66019d8e0 con 0x7fb660109240 2024-06-10T01:27:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.868+0000 7fb65ffff640 1 --2- 172.21.15.7:0/25191171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb660109240 0x7fb66007f630 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb65000e9f0 tx=0x7fb65000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.868+0000 7fb6652b8640 1 --2- 172.21.15.7:0/25191171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb660109c00 0x7fb66019d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:27:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.871+0000 7fb65dffb640 1 -- 172.21.15.7:0/25191171 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65000ccc0 con 0x7fb660109240 2024-06-10T01:27:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.871+0000 7fb666d1b640 1 -- 172.21.15.7:0/25191171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb66019dbd0 con 0x7fb660109240 2024-06-10T01:27:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.871+0000 7fb666d1b640 1 -- 172.21.15.7:0/25191171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb66019e120 con 0x7fb660109240 2024-06-10T01:27:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.871+0000 7fb65dffb640 1 -- 172.21.15.7:0/25191171 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb650010910 con 0x7fb660109240 2024-06-10T01:27:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.871+0000 7fb65dffb640 1 -- 172.21.15.7:0/25191171 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6500189c0 con 0x7fb660109240 2024-06-10T01:27:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.876+0000 7fb669a2e740 1 Processor -- start 2024-06-10T01:27:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.877+0000 7fb669a2e740 1 -- start start 2024-06-10T01:27:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.877+0000 7fb669a2e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fca88a4010 0x55fca88a4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.877+0000 7fb669a2e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca88a4a00 0x55fca88a9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.877+0000 7fb669a2e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fca88a9b70 0x55fca88ac000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.877+0000 7fb669a2e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fca88074f0 con 0x55fca88a4010 2024-06-10T01:27:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.877+0000 7fb669a2e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fca87a5870 con 0x55fca88a9b70 2024-06-10T01:27:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.877+0000 7fb669a2e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fca88034b0 con 0x55fca88a4a00 2024-06-10T01:27:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.878+0000 7fb664ab7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca88a4a00 0x55fca88a9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.879+0000 7fb664ab7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca88a4a00 0x55fca88a9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:56128/0 (socket says 172.21.15.7:56128) 2024-06-10T01:27:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.879+0000 7fb664ab7640 1 -- 172.21.15.7:0/2084031032 learned_addr learned my addr 172.21.15.7:0/2084031032 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.879+0000 7fb65ffff640 1 --2- 172.21.15.7:0/2084031032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fca88a9b70 0x55fca88ac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.879+0000 7fb65dffb640 1 -- 172.21.15.7:0/25191171 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb65001f020 con 0x7fb660109240 2024-06-10T01:27:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.879+0000 7fb664ab7640 1 -- 172.21.15.7:0/2084031032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fca88a9b70 msgr2=0x55fca88ac000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.879+0000 7fb664ab7640 1 --2- 172.21.15.7:0/2084031032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fca88a9b70 0x55fca88ac000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.879+0000 7fb664ab7640 1 -- 172.21.15.7:0/2084031032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fca88a4010 msgr2=0x55fca88a4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.879+0000 7fb664ab7640 1 --2- 172.21.15.7:0/2084031032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fca88a4010 0x55fca88a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.879+0000 7fb664ab7640 1 -- 172.21.15.7:0/2084031032 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fca87c19b0 con 0x55fca88a4a00 2024-06-10T01:27:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.879+0000 7fb664ab7640 1 --2- 172.21.15.7:0/2084031032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca88a4a00 0x55fca88a9500 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb65400bc50 tx=0x7fb65400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c34d0dfde2721c98 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.881+0000 7fb65dffb640 1 --2- 172.21.15.7:0/25191171 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb6480a2d90 0x7fb6480a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.881+0000 7fb646ffd640 1 -- 172.21.15.7:0/2084031032 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb654015070 con 0x55fca88a4a00 2024-06-10T01:27:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.881+0000 7fb646ffd640 1 -- 172.21.15.7:0/2084031032 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6540035f0 con 0x55fca88a4a00 2024-06-10T01:27:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.881+0000 7fb646ffd640 1 -- 172.21.15.7:0/2084031032 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6540116a0 con 0x55fca88a4a00 2024-06-10T01:27:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.881+0000 7fb65dffb640 1 -- 172.21.15.7:0/25191171 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fb6500cf520 con 0x7fb660109240 2024-06-10T01:27:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.884+0000 7fb669a2e740 1 -- 172.21.15.7:0/2084031032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca88a4a00 msgr2=0x55fca88a9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.884+0000 7fb669a2e740 1 --2- 172.21.15.7:0/2084031032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca88a4a00 0x55fca88a9500 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb65400bc50 tx=0x7fb65400f730 comp rx=0 tx=0).stop 2024-06-10T01:27:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.885+0000 7fb6652b8640 1 --2- 172.21.15.7:0/25191171 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb6480a2d90 0x7fb6480a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.897+0000 7fb669a2e740 1 -- 172.21.15.7:0/2084031032 shutdown_connections 2024-06-10T01:27:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.897+0000 7fb669a2e740 1 --2- 172.21.15.7:0/2084031032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fca88a9b70 0x55fca88ac000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.897+0000 7fb669a2e740 1 --2- 172.21.15.7:0/2084031032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca88a4a00 0x55fca88a9500 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.897+0000 7fb669a2e740 1 --2- 172.21.15.7:0/2084031032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fca88a4010 0x55fca88a4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.897+0000 7fb669a2e740 1 -- 172.21.15.7:0/2084031032 >> 172.21.15.7:0/2084031032 conn(0x55fca889f680 msgr2=0x55fca889fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.899+0000 7fb6652b8640 1 --2- 172.21.15.7:0/25191171 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb6480a2d90 0x7fb6480a5250 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x55fca88a5260 tx=0x7fb658030930 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.904+0000 7fb669a2e740 1 -- 172.21.15.7:0/2084031032 shutdown_connections 2024-06-10T01:27:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.907+0000 7fb669a2e740 1 -- 172.21.15.7:0/2084031032 wait complete. 2024-06-10T01:27:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.907+0000 7fb669a2e740 1 -- start start 2024-06-10T01:27:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.907+0000 7fb669a2e740 1 Processor -- start 2024-06-10T01:27:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.908+0000 7fb669a2e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fca88a4010 0x55fca88ae380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.908+0000 7fb669a2e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca88a4a00 0x55fca88ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.908+0000 7fb669a2e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fca88a9b70 0x55fca88b5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.908+0000 7fb669a2e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fca88acb20 con 0x55fca88a9b70 2024-06-10T01:27:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.908+0000 7fb669a2e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fca8790ae0 con 0x55fca88a4010 2024-06-10T01:27:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.908+0000 7fb669a2e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fca88acca0 con 0x55fca88a4a00 2024-06-10T01:27:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.909+0000 7fb669a2e740 10 client.0 _ll_register_callbacks cb 0x55fca87c99b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:27:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.910+0000 7fb664ab7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fca88a4010 0x55fca88ae380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.910+0000 7fb664ab7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fca88a4010 0x55fca88ae380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:39158/0 (socket says 172.21.15.7:39158) 2024-06-10T01:27:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.910+0000 7fb664ab7640 1 -- 172.21.15.7:0/283449879 learned_addr learned my addr 172.21.15.7:0/283449879 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.910+0000 7fb65ffff640 1 --2- 172.21.15.7:0/283449879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca88a4a00 0x55fca88ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.911+0000 7fb65ffff640 1 -- 172.21.15.7:0/283449879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fca88a4010 msgr2=0x55fca88ae380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.911+0000 7fb65ffff640 1 --2- 172.21.15.7:0/283449879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fca88a4010 0x55fca88ae380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.911+0000 7fb65ffff640 1 -- 172.21.15.7:0/283449879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fca88a9b70 msgr2=0x55fca88b5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.911+0000 7fb65ffff640 1 --2- 172.21.15.7:0/283449879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fca88a9b70 0x55fca88b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.911+0000 7fb65ffff640 1 -- 172.21.15.7:0/283449879 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fca851b210 con 0x55fca88a4a00 2024-06-10T01:27:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.911+0000 7fb65ffff640 1 --2- 172.21.15.7:0/283449879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca88a4a00 0x55fca88ae8c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb65009b930 tx=0x7fb65009b5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8b52343956a86aa server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.911+0000 7fb63ffff640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:27:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.915+0000 7fb63ffff640 1 -- 172.21.15.7:0/283449879 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65009b9a0 con 0x55fca88a4a00 2024-06-10T01:27:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.915+0000 7fb669a2e740 1 -- 172.21.15.7:0/283449879 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6540035f0 con 0x55fca88a4a00 2024-06-10T01:27:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.915+0000 7fb63ffff640 1 -- 172.21.15.7:0/283449879 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fb6500a4180 con 0x55fca88a4a00 2024-06-10T01:27:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.920+0000 7fb63ffff640 1 -- 172.21.15.7:0/283449879 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fb65009fd50 con 0x55fca88a4a00 2024-06-10T01:27:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.921+0000 7fb669a2e740 10 client.29680 fetch_fsmap learned FSMap version 52 2024-06-10T01:27:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.921+0000 7fb669a2e740 1 -- 172.21.15.7:0/283449879 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55fca85e7620 con 0x55fca88a4a00 2024-06-10T01:27:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.926+0000 7fb63ffff640 1 -- 172.21.15.7:0/283449879 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6540035f0 con 0x55fca88a4a00 2024-06-10T01:27:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.927+0000 7fb63ffff640 1 -- 172.21.15.7:0/283449879 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7fb65009adf0 con 0x55fca88a4a00 2024-06-10T01:27:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.927+0000 7fb669a2e740 10 client.29680 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:27:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.927+0000 7fb669a2e740 10 client.29680 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:27:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.927+0000 7fb669a2e740 20 client.29680 populate_metadata read hostname 'smithi007' 2024-06-10T01:27:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.927+0000 7fb669a2e740 1 --2- 172.21.15.7:0/283449879 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fca88ba200 0x55fca88bc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.927+0000 7fb669a2e740 4 client.29680 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:27:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.927+0000 7fb669a2e740 1 -- 172.21.15.7:0/283449879 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fca881af10 con 0x55fca88ba200 2024-06-10T01:27:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.935+0000 7fb664ab7640 1 --2- 172.21.15.7:0/283449879 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fca88ba200 0x55fca88bc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.948+0000 7fb664ab7640 1 --2- 172.21.15.7:0/283449879 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fca88ba200 0x55fca88bc640 crc :-1 s=READY pgs=483 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36f363706e3e0baf server_cookie=9bd65f3f3eb77240 in_seq=0 out_seq=0 2024-06-10T01:27:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.949+0000 7fb63ffff640 10 client.29680 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.954+0000 7fb63ffff640 1 -- 172.21.15.7:0/283449879 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fca881af10 con 0x55fca88ba200 2024-06-10T01:27:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.954+0000 7fb63ffff640 10 client.29680 handle_command_reply: tid=0 2024-06-10T01:27:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.954+0000 7fb669a2e740 1 client.29680 shutdown 2024-06-10T01:27:15.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:15 smithi175 ceph-mon[36636]: pgmap v1070: 129 pgs: 129 active+clean; 680 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.2 MiB/s wr, 499 op/s 2024-06-10T01:27:15.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:15 smithi175 ceph-mon[36636]: scrub queued for path: / 2024-06-10T01:27:15.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:15 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:27:15.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:15 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:27:15.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:15 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:27:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.959+0000 7fb669a2e740 1 -- 172.21.15.7:0/283449879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca88a4a00 msgr2=0x55fca88ae8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.959+0000 7fb669a2e740 1 --2- 172.21.15.7:0/283449879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca88a4a00 0x55fca88ae8c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb65009b930 tx=0x7fb65009b5a0 comp rx=0 tx=0).stop 2024-06-10T01:27:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.960+0000 7fb669a2e740 1 -- 172.21.15.7:0/283449879 shutdown_connections 2024-06-10T01:27:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.960+0000 7fb669a2e740 1 --2- 172.21.15.7:0/283449879 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fca88ba200 0x55fca88bc640 crc :-1 s=READY pgs=483 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.960+0000 7fb669a2e740 1 --2- 172.21.15.7:0/283449879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fca88a9b70 0x55fca88b5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.960+0000 7fb669a2e740 1 --2- 172.21.15.7:0/283449879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca88a4a00 0x55fca88ae8c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.960+0000 7fb669a2e740 1 --2- 172.21.15.7:0/283449879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fca88a4010 0x55fca88ae380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.960+0000 7fb669a2e740 1 -- 172.21.15.7:0/283449879 >> 172.21.15.7:0/283449879 conn(0x55fca889f680 msgr2=0x55fca889e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.960+0000 7fb63ffff640 0 client.29680 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.966+0000 7fb669a2e740 1 -- 172.21.15.7:0/283449879 shutdown_connections 2024-06-10T01:27:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.967+0000 7fb669a2e740 1 -- 172.21.15.7:0/283449879 wait complete. 2024-06-10T01:27:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:15.968+0000 7fb669a2e740 20 client.29680 trim_cache size 0 max 16384 2024-06-10T01:27:15.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.004+0000 7fb669a2e740 1 Processor -- start 2024-06-10T01:27:15.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.004+0000 7fb669a2e740 1 -- start start 2024-06-10T01:27:15.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.004+0000 7fb669a2e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fca88a4010 0x55fca88a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:15.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.005+0000 7fb669a2e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca88a4a00 0x55fca88b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:15.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.005+0000 7fb669a2e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fca88a9b70 0x55fca88a6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:15.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.005+0000 7fb65ffff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca88a4a00 0x55fca88b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:15.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.005+0000 7fb65ffff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca88a4a00 0x55fca88b58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:56156/0 (socket says 172.21.15.7:56156) 2024-06-10T01:27:15.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.005+0000 7fb664ab7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fca88a4010 0x55fca88a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:15.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.005+0000 7fb664ab7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fca88a4010 0x55fca88a5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:39164/0 (socket says 172.21.15.7:39164) 2024-06-10T01:27:15.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.005+0000 7fb6652b8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fca88a9b70 0x55fca88a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:15.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.005+0000 7fb6652b8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fca88a9b70 0x55fca88a6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55454/0 (socket says 172.21.15.7:55454) 2024-06-10T01:27:15.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.005+0000 7fb65ffff640 1 -- 172.21.15.7:0/2990012064 learned_addr learned my addr 172.21.15.7:0/2990012064 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:15.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.006+0000 7fb669a2e740 1 -- 172.21.15.7:0/2990012064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c012410 con 0x55fca88a9b70 2024-06-10T01:27:15.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.006+0000 7fb669a2e740 1 -- 172.21.15.7:0/2990012064 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c00fce0 con 0x55fca88a4010 2024-06-10T01:27:15.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.006+0000 7fb669a2e740 1 -- 172.21.15.7:0/2990012064 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c00db30 con 0x55fca88a4a00 2024-06-10T01:27:15.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.006+0000 7fb6652b8640 1 -- 172.21.15.7:0/2990012064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca88a4a00 msgr2=0x55fca88b58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:15.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.006+0000 7fb6652b8640 1 --2- 172.21.15.7:0/2990012064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca88a4a00 0x55fca88b58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:15.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.006+0000 7fb6652b8640 1 -- 172.21.15.7:0/2990012064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fca88a4010 msgr2=0x55fca88a5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:16.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.006+0000 7fb6652b8640 1 --2- 172.21.15.7:0/2990012064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fca88a4010 0x55fca88a5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:16.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.006+0000 7fb6652b8640 1 -- 172.21.15.7:0/2990012064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fca88af220 con 0x55fca88a9b70 2024-06-10T01:27:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.016+0000 7fb6652b8640 1 --2- 172.21.15.7:0/2990012064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fca88a9b70 0x55fca88a6200 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fb658006010 tx=0x7fb65803e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=813d68aac851d0e5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.021+0000 7fb6467fc640 1 -- 172.21.15.7:0/2990012064 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb658030d70 con 0x55fca88a9b70 2024-06-10T01:27:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.021+0000 7fb6467fc640 1 -- 172.21.15.7:0/2990012064 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb658036040 con 0x55fca88a9b70 2024-06-10T01:27:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.021+0000 7fb6467fc640 1 -- 172.21.15.7:0/2990012064 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb658049420 con 0x55fca88a9b70 2024-06-10T01:27:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.023+0000 7fb669a2e740 1 -- 172.21.15.7:0/2990012064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fca88a9b70 msgr2=0x55fca88a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.023+0000 7fb669a2e740 1 --2- 172.21.15.7:0/2990012064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fca88a9b70 0x55fca88a6200 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fb658006010 tx=0x7fb65803e070 comp rx=0 tx=0).stop 2024-06-10T01:27:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.024+0000 7fb669a2e740 1 -- 172.21.15.7:0/2990012064 shutdown_connections 2024-06-10T01:27:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.024+0000 7fb669a2e740 1 --2- 172.21.15.7:0/2990012064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fca88a9b70 0x55fca88a6200 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.024+0000 7fb669a2e740 1 --2- 172.21.15.7:0/2990012064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca88a4a00 0x55fca88b58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.024+0000 7fb669a2e740 1 --2- 172.21.15.7:0/2990012064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fca88a4010 0x55fca88a5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.024+0000 7fb669a2e740 1 -- 172.21.15.7:0/2990012064 >> 172.21.15.7:0/2990012064 conn(0x55fca889f680 msgr2=0x55fca889fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.029+0000 7fb669a2e740 1 -- 172.21.15.7:0/2990012064 shutdown_connections 2024-06-10T01:27:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.031+0000 7fb669a2e740 1 -- 172.21.15.7:0/2990012064 wait complete. 2024-06-10T01:27:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.031+0000 7fb669a2e740 1 -- start start 2024-06-10T01:27:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.031+0000 7fb669a2e740 1 Processor -- start 2024-06-10T01:27:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.032+0000 7fb669a2e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fca88a4010 0x55fca88b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.032+0000 7fb669a2e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fca88a4a00 0x55fca88b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.033+0000 7fb664ab7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fca88a4010 0x55fca88b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.033+0000 7fb664ab7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fca88a4010 0x55fca88b25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:39174/0 (socket says 172.21.15.7:39174) 2024-06-10T01:27:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.033+0000 7fb664ab7640 1 -- 172.21.15.7:0/1399464593 learned_addr learned my addr 172.21.15.7:0/1399464593 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.033+0000 7fb669a2e740 1 --2- 172.21.15.7:0/1399464593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca88a9b70 0x55fca8922f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.033+0000 7fb669a2e740 1 -- 172.21.15.7:0/1399464593 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c00e1a0 con 0x55fca88a4a00 2024-06-10T01:27:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.033+0000 7fb669a2e740 1 -- 172.21.15.7:0/1399464593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c00f610 con 0x55fca88a4010 2024-06-10T01:27:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.033+0000 7fb669a2e740 1 -- 172.21.15.7:0/1399464593 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c00e870 con 0x55fca88a9b70 2024-06-10T01:27:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.033+0000 7fb669a2e740 10 client.0 _ll_register_callbacks cb 0x7fb650097c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:27:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.035+0000 7fb65ffff640 1 --2- 172.21.15.7:0/1399464593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fca88a4a00 0x55fca88b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.035+0000 7fb6652b8640 1 --2- 172.21.15.7:0/1399464593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca88a9b70 0x55fca8922f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.035+0000 7fb664ab7640 1 -- 172.21.15.7:0/1399464593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca88a9b70 msgr2=0x55fca8922f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.035+0000 7fb664ab7640 1 --2- 172.21.15.7:0/1399464593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca88a9b70 0x55fca8922f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.035+0000 7fb664ab7640 1 -- 172.21.15.7:0/1399464593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fca88a4a00 msgr2=0x55fca88b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.035+0000 7fb664ab7640 1 --2- 172.21.15.7:0/1399464593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fca88a4a00 0x55fca88b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.035+0000 7fb664ab7640 1 -- 172.21.15.7:0/1399464593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fca889f4b0 con 0x55fca88a4010 2024-06-10T01:27:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.035+0000 7fb664ab7640 1 --2- 172.21.15.7:0/1399464593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fca88a4010 0x55fca88b25a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb65400bc50 tx=0x7fb654005a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c054a5a8e95951e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.035+0000 7fb63ffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:27:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.036+0000 7fb63ffff640 1 -- 172.21.15.7:0/1399464593 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb654015040 con 0x55fca88a4010 2024-06-10T01:27:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.036+0000 7fb669a2e740 1 -- 172.21.15.7:0/1399464593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb658036040 con 0x55fca88a4010 2024-06-10T01:27:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.036+0000 7fb63ffff640 1 -- 172.21.15.7:0/1399464593 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fb654004180 con 0x55fca88a4010 2024-06-10T01:27:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.037+0000 7fb63ffff640 1 -- 172.21.15.7:0/1399464593 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fb6540043c0 con 0x55fca88a4010 2024-06-10T01:27:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.038+0000 7fb669a2e740 10 client.29756 fetch_fsmap learned FSMap version 52 2024-06-10T01:27:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.038+0000 7fb669a2e740 1 -- 172.21.15.7:0/1399464593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55fca89242b0 con 0x55fca88a4010 2024-06-10T01:27:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.040+0000 7fb63ffff640 1 -- 172.21.15.7:0/1399464593 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb654003ad0 con 0x55fca88a4010 2024-06-10T01:27:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.040+0000 7fb63ffff640 1 -- 172.21.15.7:0/1399464593 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7fb654011a60 con 0x55fca88a4010 2024-06-10T01:27:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.040+0000 7fb669a2e740 10 client.29756 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:27:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.040+0000 7fb669a2e740 10 client.29756 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:27:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.040+0000 7fb669a2e740 20 client.29756 populate_metadata read hostname 'smithi007' 2024-06-10T01:27:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.040+0000 7fb669a2e740 1 --2- 172.21.15.7:0/1399464593 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fca8924b80 0x55fca8926fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.040+0000 7fb669a2e740 4 client.29756 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:27:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.040+0000 7fb669a2e740 1 -- 172.21.15.7:0/1399464593 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fb658036040 con 0x55fca8924b80 2024-06-10T01:27:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.042+0000 7fb65ffff640 1 --2- 172.21.15.7:0/1399464593 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fca8924b80 0x55fca8926fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.062+0000 7fb65ffff640 1 --2- 172.21.15.7:0/1399464593 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fca8924b80 0x55fca8926fa0 crc :-1 s=READY pgs=484 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65fbcf9656057436 server_cookie=cf5d2a241d81ff99 in_seq=0 out_seq=0 2024-06-10T01:27:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.063+0000 7fb63ffff640 10 client.29756 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:16.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:15 smithi078 ceph-mon[36683]: pgmap v1070: 129 pgs: 129 active+clean; 680 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.2 MiB/s wr, 499 op/s 2024-06-10T01:27:16.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:15 smithi078 ceph-mon[36683]: scrub queued for path: / 2024-06-10T01:27:16.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:15 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:27:16.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:15 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:27:16.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:15 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:27:16.415 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:27:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.483+0000 7fb63ffff640 1 -- 172.21.15.7:0/1399464593 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fb658036040 con 0x55fca8924b80 2024-06-10T01:27:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.483+0000 7fb63ffff640 10 client.29756 handle_command_reply: tid=0 2024-06-10T01:27:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.483+0000 7fb669a2e740 1 client.29756 shutdown 2024-06-10T01:27:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.491+0000 7fb669a2e740 1 -- 172.21.15.7:0/1399464593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fca88a4010 msgr2=0x55fca88b25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.491+0000 7fb669a2e740 1 --2- 172.21.15.7:0/1399464593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fca88a4010 0x55fca88b25a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb65400bc50 tx=0x7fb654005a70 comp rx=0 tx=0).stop 2024-06-10T01:27:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.491+0000 7fb669a2e740 1 -- 172.21.15.7:0/1399464593 shutdown_connections 2024-06-10T01:27:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.491+0000 7fb669a2e740 1 --2- 172.21.15.7:0/1399464593 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55fca8924b80 0x55fca8926fa0 crc :-1 s=READY pgs=484 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.491+0000 7fb669a2e740 1 --2- 172.21.15.7:0/1399464593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca88a9b70 0x55fca8922f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.491+0000 7fb669a2e740 1 --2- 172.21.15.7:0/1399464593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fca88a4a00 0x55fca88b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.491+0000 7fb669a2e740 1 --2- 172.21.15.7:0/1399464593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fca88a4010 0x55fca88b25a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.491+0000 7fb669a2e740 1 -- 172.21.15.7:0/1399464593 >> 172.21.15.7:0/1399464593 conn(0x55fca889f680 msgr2=0x55fca88ae380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.491+0000 7fb63ffff640 0 client.29756 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.491+0000 7fb669a2e740 1 -- 172.21.15.7:0/1399464593 shutdown_connections 2024-06-10T01:27:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.491+0000 7fb669a2e740 1 -- 172.21.15.7:0/1399464593 wait complete. 2024-06-10T01:27:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.492+0000 7fb669a2e740 20 client.29756 trim_cache size 0 max 16384 2024-06-10T01:27:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:27:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:27:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:27:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.498+0000 7fb666d1b640 1 -- 172.21.15.7:0/25191171 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb6480a2d90 msgr2=0x7fb6480a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.498+0000 7fb666d1b640 1 --2- 172.21.15.7:0/25191171 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb6480a2d90 0x7fb6480a5250 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x55fca88a5260 tx=0x7fb658030930 comp rx=0 tx=0).stop 2024-06-10T01:27:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.498+0000 7fb666d1b640 1 -- 172.21.15.7:0/25191171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb660109240 msgr2=0x7fb66007f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.498+0000 7fb666d1b640 1 --2- 172.21.15.7:0/25191171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb660109240 0x7fb66007f630 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb65000e9f0 tx=0x7fb65000eeb0 comp rx=0 tx=0).stop 2024-06-10T01:27:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.498+0000 7fb666d1b640 1 -- 172.21.15.7:0/25191171 shutdown_connections 2024-06-10T01:27:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.498+0000 7fb666d1b640 1 --2- 172.21.15.7:0/25191171 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb6480a2d90 0x7fb6480a5250 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.498+0000 7fb666d1b640 1 --2- 172.21.15.7:0/25191171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb660109c00 0x7fb66019d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.498+0000 7fb666d1b640 1 --2- 172.21.15.7:0/25191171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb660109240 0x7fb66007f630 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.498+0000 7fb666d1b640 1 --2- 172.21.15.7:0/25191171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6601080a0 0x7fb66007f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.498+0000 7fb666d1b640 1 -- 172.21.15.7:0/25191171 >> 172.21.15.7:0/25191171 conn(0x7fb660076ff0 msgr2=0x7fb66010c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.500+0000 7fb666d1b640 1 -- 172.21.15.7:0/25191171 shutdown_connections 2024-06-10T01:27:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:16.500+0000 7fb666d1b640 1 -- 172.21.15.7:0/25191171 wait complete. 2024-06-10T01:27:16.553 INFO:tasks.workunit.client.0.smithi007.stdout: 10 187881 22.49 MB/sec execute 353 sec latency 259.274 ms 2024-06-10T01:27:16.834 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:16 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:27:16.834 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:16 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:27:16.835 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:16 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:27:16.835 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:16 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:27:16.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:16 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:27:16.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:16 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:27:16.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:16 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:27:16.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:16 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:27:17.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:16 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:27:17.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:16 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:27:17.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:16 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:27:17.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:16 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:27:17.552 INFO:tasks.workunit.client.0.smithi007.stdout: 10 188239 22.47 MB/sec execute 354 sec latency 126.256 ms 2024-06-10T01:27:17.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:17 smithi175 ceph-mon[36636]: pgmap v1071: 129 pgs: 129 active+clean; 731 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 9.1 MiB/s wr, 646 op/s 2024-06-10T01:27:18.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:17 smithi007 ceph-mon[32398]: pgmap v1071: 129 pgs: 129 active+clean; 731 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 9.1 MiB/s wr, 646 op/s 2024-06-10T01:27:18.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:17 smithi078 ceph-mon[36683]: pgmap v1071: 129 pgs: 129 active+clean; 731 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 9.1 MiB/s wr, 646 op/s 2024-06-10T01:27:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.512+0000 7f16f00ea700 1 -- 172.21.15.7:0/3307776790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16e81027a0 msgr2=0x7f16e8102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.512+0000 7f16f00ea700 1 --2- 172.21.15.7:0/3307776790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16e81027a0 0x7f16e8102b80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f16e4009920 tx=0x7f16e4009c30 comp rx=0 tx=0).stop 2024-06-10T01:27:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.512+0000 7f16f00ea700 1 -- 172.21.15.7:0/3307776790 shutdown_connections 2024-06-10T01:27:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.513+0000 7f16f00ea700 1 --2- 172.21.15.7:0/3307776790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16e8107870 0x7f16e810f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.513+0000 7f16f00ea700 1 --2- 172.21.15.7:0/3307776790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16e81027a0 0x7f16e8102b80 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.513+0000 7f16f00ea700 1 --2- 172.21.15.7:0/3307776790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16e8103150 0x7f16e8107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.513+0000 7f16f00ea700 1 -- 172.21.15.7:0/3307776790 >> 172.21.15.7:0/3307776790 conn(0x7f16e8079570 msgr2=0x7f16e8079980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:18.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.514+0000 7f16f00ea700 1 -- 172.21.15.7:0/3307776790 shutdown_connections 2024-06-10T01:27:18.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.514+0000 7f16f00ea700 1 -- 172.21.15.7:0/3307776790 wait complete. 2024-06-10T01:27:18.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.515+0000 7f16f00ea700 1 Processor -- start 2024-06-10T01:27:18.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.515+0000 7f16f00ea700 1 -- start start 2024-06-10T01:27:18.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.516+0000 7f16f00ea700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16e81027a0 0x7f16e819e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:18.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.516+0000 7f16f00ea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16e8103150 0x7f16e819edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.516+0000 7f16f00ea700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16e8107870 0x7f16e8198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.516+0000 7f16f00ea700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f16e8112830 con 0x7f16e8103150 2024-06-10T01:27:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.516+0000 7f16f00ea700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81126b0 con 0x7f16e8107870 2024-06-10T01:27:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.516+0000 7f16f00ea700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81129b0 con 0x7f16e81027a0 2024-06-10T01:27:18.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.516+0000 7f16ee687700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16e8107870 0x7f16e8198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:18.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.516+0000 7f16ed685700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16e8103150 0x7f16e819edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:18.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.516+0000 7f16ee687700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16e8107870 0x7f16e8198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:39190/0 (socket says 172.21.15.7:39190) 2024-06-10T01:27:18.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.516+0000 7f16ee687700 1 -- 172.21.15.7:0/3534741171 learned_addr learned my addr 172.21.15.7:0/3534741171 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:18.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.517+0000 7f16ede86700 1 --2- 172.21.15.7:0/3534741171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16e81027a0 0x7f16e819e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:18.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.518+0000 7f16ee687700 1 -- 172.21.15.7:0/3534741171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16e81027a0 msgr2=0x7f16e819e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:18.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.518+0000 7f16ee687700 1 --2- 172.21.15.7:0/3534741171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16e81027a0 0x7f16e819e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.518+0000 7f16ee687700 1 -- 172.21.15.7:0/3534741171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16e8103150 msgr2=0x7f16e819edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:18.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.518+0000 7f16ee687700 1 --2- 172.21.15.7:0/3534741171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16e8103150 0x7f16e819edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.518+0000 7f16ee687700 1 -- 172.21.15.7:0/3534741171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e81991d0 con 0x7f16e8107870 2024-06-10T01:27:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.520+0000 7f16ee687700 1 --2- 172.21.15.7:0/3534741171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16e8107870 0x7f16e8198a70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f16e000efc0 tx=0x7f16e000c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:18.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.520+0000 7f16deffd700 1 -- 172.21.15.7:0/3534741171 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0004bb0 con 0x7f16e8107870 2024-06-10T01:27:18.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.520+0000 7f16f00ea700 1 -- 172.21.15.7:0/3534741171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e81994c0 con 0x7f16e8107870 2024-06-10T01:27:18.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.520+0000 7f16deffd700 1 -- 172.21.15.7:0/3534741171 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16e0017070 con 0x7f16e8107870 2024-06-10T01:27:18.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.520+0000 7f16deffd700 1 -- 172.21.15.7:0/3534741171 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0007d40 con 0x7f16e8107870 2024-06-10T01:27:18.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.520+0000 7f16f00ea700 1 -- 172.21.15.7:0/3534741171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e80746c0 con 0x7f16e8107870 2024-06-10T01:27:18.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.527+0000 7f16deffd700 1 -- 172.21.15.7:0/3534741171 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f16e0013390 con 0x7f16e8107870 2024-06-10T01:27:18.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.528+0000 7f16fa96db80 1 Processor -- start 2024-06-10T01:27:18.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.528+0000 7f16fa96db80 1 -- start start 2024-06-10T01:27:18.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.528+0000 7f16fa96db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647cd426640 0x5647cd426a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:18.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.528+0000 7f16fa96db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647cd426f60 0x5647cd4209e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:18.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.528+0000 7f16fa96db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647cd420f20 0x5647cd421340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:18.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.528+0000 7f16fa96db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5647cd372810 con 0x5647cd426f60 2024-06-10T01:27:18.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.528+0000 7f16fa96db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5647cd31a720 con 0x5647cd426640 2024-06-10T01:27:18.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.528+0000 7f16fa96db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5647cd36e7d0 con 0x5647cd420f20 2024-06-10T01:27:18.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.529+0000 7f16ed685700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647cd426640 0x5647cd426a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:18.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.529+0000 7f16ed685700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647cd426640 0x5647cd426a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:39198/0 (socket says 172.21.15.7:39198) 2024-06-10T01:27:18.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.529+0000 7f16ed685700 1 -- 172.21.15.7:0/2610184574 learned_addr learned my addr 172.21.15.7:0/2610184574 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:18.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.529+0000 7f16ede86700 1 --2- 172.21.15.7:0/2610184574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647cd426f60 0x5647cd4209e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:18.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.530+0000 7f16deffd700 1 --2- 172.21.15.7:0/3534741171 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f16d40a2d70 0x7f16d40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:18.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.530+0000 7f16deffd700 1 -- 172.21.15.7:0/3534741171 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f16e00cfc80 con 0x7f16e8107870 2024-06-10T01:27:18.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.531+0000 7f16ed685700 1 --2- 172.21.15.7:0/2610184574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647cd420f20 0x5647cd421340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:18.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.531+0000 7f16ed685700 1 -- 172.21.15.7:0/2610184574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647cd420f20 msgr2=0x5647cd421340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:18.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.531+0000 7f16ed685700 1 --2- 172.21.15.7:0/2610184574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647cd420f20 0x5647cd421340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.531+0000 7f16ed685700 1 -- 172.21.15.7:0/2610184574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647cd426f60 msgr2=0x5647cd4209e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:18.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.531+0000 7f16ed685700 1 --2- 172.21.15.7:0/2610184574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647cd426f60 0x5647cd4209e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.531+0000 7f16ed685700 1 -- 172.21.15.7:0/2610184574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647cd310f50 con 0x5647cd426640 2024-06-10T01:27:18.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.531+0000 7f16ede86700 1 --2- 172.21.15.7:0/2610184574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647cd426f60 0x5647cd4209e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:27:18.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.531+0000 7f16ed685700 1 --2- 172.21.15.7:0/2610184574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647cd426640 0x5647cd426a20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f16d8011540 tx=0x7f16d800a7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efa436484d429268 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:18.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.532+0000 7f16d3fff700 1 -- 172.21.15.7:0/2610184574 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d8004a30 con 0x5647cd426640 2024-06-10T01:27:18.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.532+0000 7f16d3fff700 1 -- 172.21.15.7:0/2610184574 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16d8011e50 con 0x5647cd426640 2024-06-10T01:27:18.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.532+0000 7f16d3fff700 1 -- 172.21.15.7:0/2610184574 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d800fa20 con 0x5647cd426640 2024-06-10T01:27:18.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.533+0000 7f16ee687700 1 --2- 172.21.15.7:0/3534741171 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f16d40a2d70 0x7f16d40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:18.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.533+0000 7f16fa96db80 1 -- 172.21.15.7:0/2610184574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647cd426640 msgr2=0x5647cd426a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:18.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.533+0000 7f16fa96db80 1 --2- 172.21.15.7:0/2610184574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647cd426640 0x5647cd426a20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f16d8011540 tx=0x7f16d800a7e0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.533+0000 7f16fa96db80 1 -- 172.21.15.7:0/2610184574 shutdown_connections 2024-06-10T01:27:18.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.533+0000 7f16fa96db80 1 --2- 172.21.15.7:0/2610184574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647cd420f20 0x5647cd421340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.533+0000 7f16ee687700 1 --2- 172.21.15.7:0/3534741171 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f16d40a2d70 0x7f16d40a5230 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f16e0098ad0 tx=0x7f16e0015040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:18.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.534+0000 7f16fa96db80 1 --2- 172.21.15.7:0/2610184574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647cd426640 0x5647cd426a20 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.534+0000 7f16fa96db80 1 --2- 172.21.15.7:0/2610184574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647cd426f60 0x5647cd4209e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.534+0000 7f16fa96db80 1 -- 172.21.15.7:0/2610184574 >> 172.21.15.7:0/2610184574 conn(0x5647cd34bcc0 msgr2=0x5647cd41b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:18.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.534+0000 7f16fa96db80 1 -- 172.21.15.7:0/2610184574 shutdown_connections 2024-06-10T01:27:18.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.534+0000 7f16fa96db80 1 -- 172.21.15.7:0/2610184574 wait complete. 2024-06-10T01:27:18.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.535+0000 7f16fa96db80 1 -- start start 2024-06-10T01:27:18.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.535+0000 7f16fa96db80 1 Processor -- start 2024-06-10T01:27:18.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.536+0000 7f16fa96db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647cd420f20 0x5647cd42cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:18.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.536+0000 7f16fa96db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647cd426640 0x5647cd42a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:18.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.536+0000 7f16fa96db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647cd426f60 0x5647cd42a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:18.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.536+0000 7f16fa96db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5647cd43e9e0 con 0x5647cd420f20 2024-06-10T01:27:18.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.536+0000 7f16fa96db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5647cd312bf0 con 0x5647cd426640 2024-06-10T01:27:18.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.536+0000 7f16fa96db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5647cd43eb60 con 0x5647cd426f60 2024-06-10T01:27:18.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.536+0000 7f16fa96db80 10 client.0 _ll_register_callbacks cb 0x5647cd30c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:27:18.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.536+0000 7f16ede86700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647cd426640 0x5647cd42a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:18.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.537+0000 7f16ed685700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647cd426f60 0x5647cd42a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:18.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.537+0000 7f16ede86700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647cd426640 0x5647cd42a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:39206/0 (socket says 172.21.15.7:39206) 2024-06-10T01:27:18.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.537+0000 7f16ede86700 1 -- 172.21.15.7:0/3632888890 learned_addr learned my addr 172.21.15.7:0/3632888890 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:18.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.537+0000 7f16ed685700 1 --2- 172.21.15.7:0/3632888890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647cd420f20 0x5647cd42cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:18.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.537+0000 7f16ede86700 1 -- 172.21.15.7:0/3632888890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647cd426f60 msgr2=0x5647cd42a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:18.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.537+0000 7f16ede86700 1 --2- 172.21.15.7:0/3632888890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647cd426f60 0x5647cd42a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.537+0000 7f16ede86700 1 -- 172.21.15.7:0/3632888890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647cd420f20 msgr2=0x5647cd42cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:18.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.537+0000 7f16ede86700 1 --2- 172.21.15.7:0/3632888890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647cd420f20 0x5647cd42cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.537+0000 7f16ede86700 1 -- 172.21.15.7:0/3632888890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647cd2c18d0 con 0x5647cd426640 2024-06-10T01:27:18.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.537+0000 7f16ede86700 1 --2- 172.21.15.7:0/3632888890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647cd426640 0x5647cd42a0a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f16e4016000 tx=0x7f16e400fc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd1c1596492db44c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:18.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.538+0000 7f16d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:27:18.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.538+0000 7f16d17fa700 1 -- 172.21.15.7:0/3632888890 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e4005310 con 0x5647cd426640 2024-06-10T01:27:18.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.538+0000 7f16fa96db80 1 -- 172.21.15.7:0/3632888890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16d8011e50 con 0x5647cd426640 2024-06-10T01:27:18.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.538+0000 7f16d17fa700 1 -- 172.21.15.7:0/3632888890 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f16e40054b0 con 0x5647cd426640 2024-06-10T01:27:18.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.538+0000 7f16d17fa700 1 -- 172.21.15.7:0/3632888890 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f16e402b050 con 0x5647cd426640 2024-06-10T01:27:18.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.538+0000 7f16fa96db80 10 client.29768 fetch_fsmap learned FSMap version 52 2024-06-10T01:27:18.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.538+0000 7f16fa96db80 1 -- 172.21.15.7:0/3632888890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5647cd449330 con 0x5647cd426640 2024-06-10T01:27:18.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.539+0000 7f16d17fa700 1 -- 172.21.15.7:0/3632888890 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16d8011e50 con 0x5647cd426640 2024-06-10T01:27:18.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.539+0000 7f16d17fa700 1 -- 172.21.15.7:0/3632888890 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f16e400b860 con 0x5647cd426640 2024-06-10T01:27:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.539+0000 7f16fa96db80 10 client.29768 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:27:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.539+0000 7f16fa96db80 10 client.29768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:27:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.539+0000 7f16fa96db80 20 client.29768 populate_metadata read hostname 'smithi007' 2024-06-10T01:27:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.539+0000 7f16fa96db80 1 --2- 172.21.15.7:0/3632888890 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5647cd44bca0 0x5647cd44e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.539+0000 7f16fa96db80 4 client.29768 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:27:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.539+0000 7f16fa96db80 1 -- 172.21.15.7:0/3632888890 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647cd37fd10 con 0x5647cd44bca0 2024-06-10T01:27:18.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.543+0000 7f16ed685700 1 --2- 172.21.15.7:0/3632888890 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5647cd44bca0 0x5647cd44e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:18.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.544+0000 7f16ed685700 1 --2- 172.21.15.7:0/3632888890 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5647cd44bca0 0x5647cd44e120 crc :-1 s=READY pgs=485 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0d59a17ab9bf7fe server_cookie=8ac5c1284f03a773 in_seq=0 out_seq=0 2024-06-10T01:27:18.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.544+0000 7f16d17fa700 10 client.29768 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:18.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.549+0000 7f16d17fa700 1 -- 172.21.15.7:0/3632888890 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5647cd37fd10 con 0x5647cd44bca0 2024-06-10T01:27:18.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.549+0000 7f16d17fa700 10 client.29768 handle_command_reply: tid=0 2024-06-10T01:27:18.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.549+0000 7f16fa96db80 1 client.29768 shutdown 2024-06-10T01:27:18.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.549+0000 7f16fa96db80 1 -- 172.21.15.7:0/3632888890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647cd426640 msgr2=0x5647cd42a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:18.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.549+0000 7f16fa96db80 1 --2- 172.21.15.7:0/3632888890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647cd426640 0x5647cd42a0a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f16e4016000 tx=0x7f16e400fc00 comp rx=0 tx=0).stop 2024-06-10T01:27:18.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.549+0000 7f16fa96db80 1 -- 172.21.15.7:0/3632888890 shutdown_connections 2024-06-10T01:27:18.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.549+0000 7f16fa96db80 1 --2- 172.21.15.7:0/3632888890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647cd426f60 0x5647cd42a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.549+0000 7f16fa96db80 1 --2- 172.21.15.7:0/3632888890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647cd420f20 0x5647cd42cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.549+0000 7f16fa96db80 1 --2- 172.21.15.7:0/3632888890 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5647cd44bca0 0x5647cd44e120 crc :-1 s=READY pgs=485 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.549+0000 7f16fa96db80 1 --2- 172.21.15.7:0/3632888890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647cd426640 0x5647cd42a0a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.549+0000 7f16fa96db80 1 -- 172.21.15.7:0/3632888890 >> 172.21.15.7:0/3632888890 conn(0x5647cd34bcc0 msgr2=0x5647cd41f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:18.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.549+0000 7f16d17fa700 0 client.29768 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:18.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.549+0000 7f16fa96db80 1 -- 172.21.15.7:0/3632888890 shutdown_connections 2024-06-10T01:27:18.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.550+0000 7f16fa96db80 1 -- 172.21.15.7:0/3632888890 wait complete. 2024-06-10T01:27:18.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.550+0000 7f16fa96db80 20 client.29768 trim_cache size 0 max 16384 2024-06-10T01:27:18.552 INFO:tasks.workunit.client.0.smithi007.stdout: 10 188597 22.46 MB/sec execute 355 sec latency 170.875 ms 2024-06-10T01:27:18.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.579+0000 7f16fa96db80 1 Processor -- start 2024-06-10T01:27:18.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.579+0000 7f16fa96db80 1 -- start start 2024-06-10T01:27:18.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.579+0000 7f16fa96db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647cd420f20 0x5647cd447cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:18.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.580+0000 7f16fa96db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647cd426640 0x5647cd4209e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:18.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.580+0000 7f16fa96db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647cd426f60 0x5647cd415510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:18.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.580+0000 7f16fa96db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81a6100 con 0x5647cd426f60 2024-06-10T01:27:18.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.580+0000 7f16fa96db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16e80728d0 con 0x5647cd420f20 2024-06-10T01:27:18.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.580+0000 7f16fa96db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f16e8070720 con 0x5647cd426640 2024-06-10T01:27:18.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.580+0000 7f16ed685700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647cd420f20 0x5647cd447cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:18.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.580+0000 7f16ed685700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647cd420f20 0x5647cd447cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:39214/0 (socket says 172.21.15.7:39214) 2024-06-10T01:27:18.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.580+0000 7f16ed685700 1 -- 172.21.15.7:0/2587256821 learned_addr learned my addr 172.21.15.7:0/2587256821 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:18.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.580+0000 7f16ed685700 1 --2- 172.21.15.7:0/2587256821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647cd426f60 0x5647cd415510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:18.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.580+0000 7f16ede86700 1 --2- 172.21.15.7:0/2587256821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647cd426640 0x5647cd4209e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:18.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.580+0000 7f16ed685700 1 -- 172.21.15.7:0/2587256821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647cd426640 msgr2=0x5647cd4209e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:18.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.580+0000 7f16ed685700 1 --2- 172.21.15.7:0/2587256821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647cd426640 0x5647cd4209e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.580+0000 7f16ed685700 1 -- 172.21.15.7:0/2587256821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647cd426f60 msgr2=0x5647cd415510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:18.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.580+0000 7f16ed685700 1 --2- 172.21.15.7:0/2587256821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647cd426f60 0x5647cd415510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.580+0000 7f16ed685700 1 -- 172.21.15.7:0/2587256821 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647cd42abc0 con 0x5647cd420f20 2024-06-10T01:27:18.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.580+0000 7f16ed685700 1 --2- 172.21.15.7:0/2587256821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647cd420f20 0x5647cd447cb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f16d8011be0 tx=0x7f16d800a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=955e0f92cf2b1e40 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:18.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.581+0000 7f16d37fe700 1 -- 172.21.15.7:0/2587256821 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d8027040 con 0x5647cd420f20 2024-06-10T01:27:18.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.581+0000 7f16d37fe700 1 -- 172.21.15.7:0/2587256821 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16d8011e50 con 0x5647cd420f20 2024-06-10T01:27:18.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.581+0000 7f16d37fe700 1 -- 172.21.15.7:0/2587256821 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d800fb40 con 0x5647cd420f20 2024-06-10T01:27:18.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.582+0000 7f16fa96db80 1 -- 172.21.15.7:0/2587256821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647cd420f20 msgr2=0x5647cd447cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:18.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.582+0000 7f16fa96db80 1 --2- 172.21.15.7:0/2587256821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647cd420f20 0x5647cd447cb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f16d8011be0 tx=0x7f16d800a300 comp rx=0 tx=0).stop 2024-06-10T01:27:18.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.582+0000 7f16fa96db80 1 -- 172.21.15.7:0/2587256821 shutdown_connections 2024-06-10T01:27:18.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.582+0000 7f16fa96db80 1 --2- 172.21.15.7:0/2587256821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647cd426f60 0x5647cd415510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.582+0000 7f16fa96db80 1 --2- 172.21.15.7:0/2587256821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647cd420f20 0x5647cd447cb0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.582+0000 7f16fa96db80 1 --2- 172.21.15.7:0/2587256821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647cd426640 0x5647cd4209e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.582+0000 7f16fa96db80 1 -- 172.21.15.7:0/2587256821 >> 172.21.15.7:0/2587256821 conn(0x5647cd34bcc0 msgr2=0x5647cd414e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:18.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.582+0000 7f16fa96db80 1 -- 172.21.15.7:0/2587256821 shutdown_connections 2024-06-10T01:27:18.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.582+0000 7f16fa96db80 1 -- 172.21.15.7:0/2587256821 wait complete. 2024-06-10T01:27:18.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.582+0000 7f16fa96db80 1 -- start start 2024-06-10T01:27:18.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.582+0000 7f16fa96db80 1 Processor -- start 2024-06-10T01:27:18.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.583+0000 7f16fa96db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647cd420f20 0x5647cd4444d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:18.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.583+0000 7f16fa96db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647cd426640 0x5647cd4532e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:18.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.583+0000 7f16fa96db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647cd426f60 0x5647cd453820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:18.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.583+0000 7f16fa96db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f16e8070d90 con 0x5647cd426f60 2024-06-10T01:27:18.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.583+0000 7f16fa96db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16e8072200 con 0x5647cd426640 2024-06-10T01:27:18.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.583+0000 7f16fa96db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f16e8071460 con 0x5647cd420f20 2024-06-10T01:27:18.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.583+0000 7f16fa96db80 10 client.0 _ll_register_callbacks cb 0x5647cd30c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:27:18.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.583+0000 7f16ed685700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647cd420f20 0x5647cd4444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:18.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.583+0000 7f16ed685700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647cd420f20 0x5647cd4444d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:56206/0 (socket says 172.21.15.7:56206) 2024-06-10T01:27:18.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.583+0000 7f16ed685700 1 -- 172.21.15.7:0/3218911737 learned_addr learned my addr 172.21.15.7:0/3218911737 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:18.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.583+0000 7f16ede86700 1 --2- 172.21.15.7:0/3218911737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647cd426640 0x5647cd4532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:18.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.583+0000 7f16ed685700 1 --2- 172.21.15.7:0/3218911737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647cd426f60 0x5647cd453820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:18.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.584+0000 7f16ede86700 1 -- 172.21.15.7:0/3218911737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647cd420f20 msgr2=0x5647cd4444d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:18.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.584+0000 7f16ede86700 1 --2- 172.21.15.7:0/3218911737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647cd420f20 0x5647cd4444d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.584+0000 7f16ede86700 1 -- 172.21.15.7:0/3218911737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647cd426f60 msgr2=0x5647cd453820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:18.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.584+0000 7f16ede86700 1 --2- 172.21.15.7:0/3218911737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647cd426f60 0x5647cd453820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.584+0000 7f16ede86700 1 -- 172.21.15.7:0/3218911737 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647cd453fd0 con 0x5647cd426640 2024-06-10T01:27:18.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.584+0000 7f16ede86700 1 --2- 172.21.15.7:0/3218911737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647cd426640 0x5647cd4532e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f16e4017420 tx=0x7f16e400fb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c6648740ca4ac50 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:18.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.584+0000 7f16d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:27:18.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.584+0000 7f16d17fa700 1 -- 172.21.15.7:0/3218911737 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e401b040 con 0x5647cd426640 2024-06-10T01:27:18.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.584+0000 7f16d17fa700 1 -- 172.21.15.7:0/3218911737 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f16e400b770 con 0x5647cd426640 2024-06-10T01:27:18.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.584+0000 7f16fa96db80 1 -- 172.21.15.7:0/3218911737 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16d8011e50 con 0x5647cd426640 2024-06-10T01:27:18.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.584+0000 7f16d17fa700 1 -- 172.21.15.7:0/3218911737 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f16e4039050 con 0x5647cd426640 2024-06-10T01:27:18.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.584+0000 7f16fa96db80 10 client.29774 fetch_fsmap learned FSMap version 52 2024-06-10T01:27:18.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.584+0000 7f16fa96db80 1 -- 172.21.15.7:0/3218911737 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5647cd4b6990 con 0x5647cd426640 2024-06-10T01:27:18.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.585+0000 7f16d17fa700 1 -- 172.21.15.7:0/3218911737 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16d8011e50 con 0x5647cd426640 2024-06-10T01:27:18.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.585+0000 7f16d17fa700 1 -- 172.21.15.7:0/3218911737 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 52) ==== 8354+0+0 (secure 0 0 0) 0x7f16e402e020 con 0x5647cd426640 2024-06-10T01:27:18.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.585+0000 7f16fa96db80 10 client.29774 fetch_fsmap finished waiting for FSMap version 52 2024-06-10T01:27:18.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.585+0000 7f16fa96db80 10 client.29774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:27:18.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.585+0000 7f16fa96db80 20 client.29774 populate_metadata read hostname 'smithi007' 2024-06-10T01:27:18.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.585+0000 7f16fa96db80 1 --2- 172.21.15.7:0/3218911737 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5647cd4b7040 0x5647cd4b9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:18.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.585+0000 7f16fa96db80 4 client.29774 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:27:18.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.585+0000 7f16fa96db80 1 -- 172.21.15.7:0/3218911737 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5647cd37a290 con 0x5647cd4b7040 2024-06-10T01:27:18.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.585+0000 7f16ed685700 1 --2- 172.21.15.7:0/3218911737 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5647cd4b7040 0x5647cd4b9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:18.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.586+0000 7f16ed685700 1 --2- 172.21.15.7:0/3218911737 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5647cd4b7040 0x5647cd4b9460 crc :-1 s=READY pgs=486 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dccd300ed2eb0450 server_cookie=f7c78f4ebe08aa22 in_seq=0 out_seq=0 2024-06-10T01:27:18.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.586+0000 7f16d17fa700 10 client.29774 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:18.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.588+0000 7f16d17fa700 1 -- 172.21.15.7:0/3218911737 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5647cd37a290 con 0x5647cd4b7040 2024-06-10T01:27:18.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.588+0000 7f16d17fa700 10 client.29774 handle_command_reply: tid=0 2024-06-10T01:27:18.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.588+0000 7f16fa96db80 1 client.29774 shutdown 2024-06-10T01:27:18.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.589+0000 7f16fa96db80 1 -- 172.21.15.7:0/3218911737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647cd426640 msgr2=0x5647cd4532e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:18.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.589+0000 7f16fa96db80 1 --2- 172.21.15.7:0/3218911737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647cd426640 0x5647cd4532e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f16e4017420 tx=0x7f16e400fb70 comp rx=0 tx=0).stop 2024-06-10T01:27:18.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.589+0000 7f16fa96db80 1 -- 172.21.15.7:0/3218911737 shutdown_connections 2024-06-10T01:27:18.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.589+0000 7f16fa96db80 1 --2- 172.21.15.7:0/3218911737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647cd426f60 0x5647cd453820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.589+0000 7f16fa96db80 1 --2- 172.21.15.7:0/3218911737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647cd420f20 0x5647cd4444d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.589+0000 7f16fa96db80 1 --2- 172.21.15.7:0/3218911737 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5647cd4b7040 0x5647cd4b9460 crc :-1 s=READY pgs=486 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.589+0000 7f16fa96db80 1 --2- 172.21.15.7:0/3218911737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647cd426640 0x5647cd4532e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.589+0000 7f16fa96db80 1 -- 172.21.15.7:0/3218911737 >> 172.21.15.7:0/3218911737 conn(0x5647cd34bcc0 msgr2=0x5647cd42b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:18.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.589+0000 7f16d17fa700 0 client.29774 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:18.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.589+0000 7f16fa96db80 1 -- 172.21.15.7:0/3218911737 shutdown_connections 2024-06-10T01:27:18.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.589+0000 7f16fa96db80 1 -- 172.21.15.7:0/3218911737 wait complete. 2024-06-10T01:27:18.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.589+0000 7f16fa96db80 20 client.29774 trim_cache size 0 max 16384 2024-06-10T01:27:18.584 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:27:18.584 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T01:27:18.584 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-10T01:27:18.584 INFO:teuthology.orchestra.run.smithi007.stdout: "cb984063-c0ed-419c-ac2d-d464b9612156": { 2024-06-10T01:27:18.584 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-10T01:27:18.584 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "cb984063-c0ed-419c-ac2d-d464b9612156", 2024-06-10T01:27:18.584 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-10T01:27:18.584 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:27:18.584 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:27:18.585 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:27:18.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.590+0000 7f16f00ea700 1 -- 172.21.15.7:0/3534741171 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f16d40a2d70 msgr2=0x7f16d40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:18.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.590+0000 7f16f00ea700 1 --2- 172.21.15.7:0/3534741171 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f16d40a2d70 0x7f16d40a5230 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f16e0098ad0 tx=0x7f16e0015040 comp rx=0 tx=0).stop 2024-06-10T01:27:18.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.590+0000 7f16f00ea700 1 -- 172.21.15.7:0/3534741171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16e8107870 msgr2=0x7f16e8198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:18.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.590+0000 7f16f00ea700 1 --2- 172.21.15.7:0/3534741171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16e8107870 0x7f16e8198a70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f16e000efc0 tx=0x7f16e000c3f0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.590+0000 7f16f00ea700 1 -- 172.21.15.7:0/3534741171 shutdown_connections 2024-06-10T01:27:18.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.590+0000 7f16f00ea700 1 --2- 172.21.15.7:0/3534741171 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f16d40a2d70 0x7f16d40a5230 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.590+0000 7f16f00ea700 1 --2- 172.21.15.7:0/3534741171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16e8107870 0x7f16e8198a70 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.590+0000 7f16f00ea700 1 --2- 172.21.15.7:0/3534741171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16e81027a0 0x7f16e819e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.590+0000 7f16f00ea700 1 --2- 172.21.15.7:0/3534741171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16e8103150 0x7f16e819edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:18.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.590+0000 7f16f00ea700 1 -- 172.21.15.7:0/3534741171 >> 172.21.15.7:0/3534741171 conn(0x7f16e8079570 msgr2=0x7f16e81089c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:18.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.591+0000 7f16f00ea700 1 -- 172.21.15.7:0/3534741171 shutdown_connections 2024-06-10T01:27:18.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:18.591+0000 7f16f00ea700 1 -- 172.21.15.7:0/3534741171 wait complete. 2024-06-10T01:27:18.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:18 smithi175 ceph-mon[36636]: pgmap v1072: 129 pgs: 129 active+clean; 738 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 10 MiB/s wr, 567 op/s 2024-06-10T01:27:19.012 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:18 smithi007 ceph-mon[32398]: pgmap v1072: 129 pgs: 129 active+clean; 738 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 10 MiB/s wr, 567 op/s 2024-06-10T01:27:19.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:18 smithi078 ceph-mon[36683]: pgmap v1072: 129 pgs: 129 active+clean; 738 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 10 MiB/s wr, 567 op/s 2024-06-10T01:27:19.552 INFO:tasks.workunit.client.0.smithi007.stdout: 10 189121 22.49 MB/sec execute 356 sec latency 174.016 ms 2024-06-10T01:27:19.656 INFO:tasks.cephfs.filesystem:scrub status for tag:cb984063-c0ed-419c-ac2d-d464b9612156 - {'path': '/', 'tag': 'cb984063-c0ed-419c-ac2d-d464b9612156', 'options': 'recursive,force'} 2024-06-10T01:27:20.552 INFO:tasks.workunit.client.0.smithi007.stdout: 10 189670 22.51 MB/sec execute 357 sec latency 74.086 ms 2024-06-10T01:27:21.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:21 smithi078 ceph-mon[36683]: pgmap v1073: 129 pgs: 129 active+clean; 723 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 11 MiB/s wr, 554 op/s 2024-06-10T01:27:21.352 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:21 smithi007 ceph-mon[32398]: pgmap v1073: 129 pgs: 129 active+clean; 723 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 11 MiB/s wr, 554 op/s 2024-06-10T01:27:21.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:21 smithi175 ceph-mon[36636]: pgmap v1073: 129 pgs: 129 active+clean; 723 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 11 MiB/s wr, 554 op/s 2024-06-10T01:27:21.552 INFO:tasks.workunit.client.0.smithi007.stdout: 10 190323 22.54 MB/sec execute 358 sec latency 85.968 ms 2024-06-10T01:27:22.552 INFO:tasks.workunit.client.0.smithi007.stdout: 10 191021 22.57 MB/sec execute 359 sec latency 62.134 ms 2024-06-10T01:27:23.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:23 smithi078 ceph-mon[36683]: pgmap v1074: 129 pgs: 129 active+clean; 656 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 12 MiB/s wr, 778 op/s 2024-06-10T01:27:23.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:23 smithi175 ceph-mon[36636]: pgmap v1074: 129 pgs: 129 active+clean; 656 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 12 MiB/s wr, 778 op/s 2024-06-10T01:27:23.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:23 smithi007 ceph-mon[32398]: pgmap v1074: 129 pgs: 129 active+clean; 656 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 12 MiB/s wr, 778 op/s 2024-06-10T01:27:23.552 INFO:tasks.workunit.client.0.smithi007.stdout: 10 191787 22.65 MB/sec execute 360 sec latency 62.775 ms 2024-06-10T01:27:24.552 INFO:tasks.workunit.client.0.smithi007.stdout: 10 192338 22.66 MB/sec execute 361 sec latency 83.560 ms 2024-06-10T01:27:25.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:25 smithi078 ceph-mon[36683]: pgmap v1075: 129 pgs: 129 active+clean; 656 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 11 MiB/s wr, 708 op/s 2024-06-10T01:27:25.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:25 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:27:25.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:25 smithi175 ceph-mon[36636]: pgmap v1075: 129 pgs: 129 active+clean; 656 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 11 MiB/s wr, 708 op/s 2024-06-10T01:27:25.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:25 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:27:25.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:25 smithi007 ceph-mon[32398]: pgmap v1075: 129 pgs: 129 active+clean; 656 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 11 MiB/s wr, 708 op/s 2024-06-10T01:27:25.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:25 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:27:25.553 INFO:tasks.workunit.client.0.smithi007.stdout: 10 192979 22.68 MB/sec execute 362 sec latency 84.349 ms 2024-06-10T01:27:26.552 INFO:tasks.workunit.client.0.smithi007.stdout: 10 193541 22.71 MB/sec execute 363 sec latency 55.569 ms 2024-06-10T01:27:27.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:27 smithi078 ceph-mon[36683]: pgmap v1076: 129 pgs: 129 active+clean; 663 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 12 MiB/s wr, 821 op/s 2024-06-10T01:27:27.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:27 smithi175 ceph-mon[36636]: pgmap v1076: 129 pgs: 129 active+clean; 663 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 12 MiB/s wr, 821 op/s 2024-06-10T01:27:27.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:27 smithi007 ceph-mon[32398]: pgmap v1076: 129 pgs: 129 active+clean; 663 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 12 MiB/s wr, 821 op/s 2024-06-10T01:27:27.552 INFO:tasks.workunit.client.0.smithi007.stdout: 10 194217 22.73 MB/sec execute 364 sec latency 70.114 ms 2024-06-10T01:27:28.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:28 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:27:28.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:28 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:27:28.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:28 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:27:28.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:28 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:27:28.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:28 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:27:28.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:28 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:27:28.552 INFO:tasks.workunit.client.0.smithi007.stdout: 10 195083 22.78 MB/sec execute 365 sec latency 102.312 ms 2024-06-10T01:27:29.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:29 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:27:29.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:29 smithi078 ceph-mon[36683]: pgmap v1077: 129 pgs: 129 active+clean; 666 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.5 MiB/s wr, 694 op/s 2024-06-10T01:27:29.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:29 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:27:29.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:29 smithi175 ceph-mon[36636]: pgmap v1077: 129 pgs: 129 active+clean; 666 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.5 MiB/s wr, 694 op/s 2024-06-10T01:27:29.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:29 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:27:29.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:29 smithi007 ceph-mon[32398]: pgmap v1077: 129 pgs: 129 active+clean; 666 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.5 MiB/s wr, 694 op/s 2024-06-10T01:27:29.553 INFO:tasks.workunit.client.0.smithi007.stdout: 10 195921 22.90 MB/sec execute 366 sec latency 74.210 ms 2024-06-10T01:27:29.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.617+0000 7f35a1914640 1 -- 172.21.15.7:0/1058826525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f359c1035c0 msgr2=0x7f359c109e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:29.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.617+0000 7f35a1914640 1 --2- 172.21.15.7:0/1058826525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f359c1035c0 0x7f359c109e50 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f3594009ca0 tx=0x7f359402a8c0 comp rx=0 tx=0).stop 2024-06-10T01:27:29.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.617+0000 7f35a1914640 1 -- 172.21.15.7:0/1058826525 shutdown_connections 2024-06-10T01:27:29.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.617+0000 7f35a1914640 1 --2- 172.21.15.7:0/1058826525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f359c1035c0 0x7f359c109e50 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:29.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.617+0000 7f35a1914640 1 --2- 172.21.15.7:0/1058826525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f359c102c00 0x7f359c103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:29.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.617+0000 7f35a1914640 1 --2- 172.21.15.7:0/1058826525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f359c101a10 0x7f359c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:29.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.617+0000 7f35a1914640 1 -- 172.21.15.7:0/1058826525 >> 172.21.15.7:0/1058826525 conn(0x7f359c0fcf80 msgr2=0x7f359c0ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:29.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.617+0000 7f35a1914640 1 -- 172.21.15.7:0/1058826525 shutdown_connections 2024-06-10T01:27:29.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.617+0000 7f35a1914640 1 -- 172.21.15.7:0/1058826525 wait complete. 2024-06-10T01:27:29.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.618+0000 7f35a1914640 1 Processor -- start 2024-06-10T01:27:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.618+0000 7f35a1914640 1 -- start start 2024-06-10T01:27:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.618+0000 7f35a1914640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f359c101a10 0x7f359c197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.618+0000 7f35a1914640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f359c102c00 0x7f359c198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.618+0000 7f35a1914640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f359c1035c0 0x7f359c19f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.618+0000 7f35a1914640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f359c10d690 con 0x7f359c101a10 2024-06-10T01:27:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.618+0000 7f35a1914640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f359c10d510 con 0x7f359c1035c0 2024-06-10T01:27:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.618+0000 7f35a1914640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f359c10d810 con 0x7f359c102c00 2024-06-10T01:27:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.618+0000 7f359affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f359c101a10 0x7f359c197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.618+0000 7f359affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f359c101a10 0x7f359c197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43954/0 (socket says 172.21.15.7:43954) 2024-06-10T01:27:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.618+0000 7f359affd640 1 -- 172.21.15.7:0/2801298872 learned_addr learned my addr 172.21.15.7:0/2801298872 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.618+0000 7f359a7fc640 1 --2- 172.21.15.7:0/2801298872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f359c102c00 0x7f359c198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.618+0000 7f359b7fe640 1 --2- 172.21.15.7:0/2801298872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f359c1035c0 0x7f359c19f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.619+0000 7f359b7fe640 1 -- 172.21.15.7:0/2801298872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f359c102c00 msgr2=0x7f359c198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.619+0000 7f359b7fe640 1 --2- 172.21.15.7:0/2801298872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f359c102c00 0x7f359c198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.619+0000 7f359b7fe640 1 -- 172.21.15.7:0/2801298872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f359c101a10 msgr2=0x7f359c197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.619+0000 7f359b7fe640 1 --2- 172.21.15.7:0/2801298872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f359c101a10 0x7f359c197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.619+0000 7f359b7fe640 1 -- 172.21.15.7:0/2801298872 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359c073c50 con 0x7f359c1035c0 2024-06-10T01:27:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.619+0000 7f359affd640 1 --2- 172.21.15.7:0/2801298872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f359c101a10 0x7f359c197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:27:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.619+0000 7f359b7fe640 1 --2- 172.21.15.7:0/2801298872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f359c1035c0 0x7f359c19f290 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3594031ae0 tx=0x7f3594004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.619+0000 7f35a0912640 1 -- 172.21.15.7:0/2801298872 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594003b70 con 0x7f359c1035c0 2024-06-10T01:27:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.619+0000 7f35a1914640 1 -- 172.21.15.7:0/2801298872 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f359c073ee0 con 0x7f359c1035c0 2024-06-10T01:27:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.619+0000 7f35a0912640 1 -- 172.21.15.7:0/2801298872 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3594003d10 con 0x7f359c1035c0 2024-06-10T01:27:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.619+0000 7f35a0912640 1 -- 172.21.15.7:0/2801298872 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359402fc40 con 0x7f359c1035c0 2024-06-10T01:27:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.619+0000 7f35a1914640 1 -- 172.21.15.7:0/2801298872 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f359c074450 con 0x7f359c1035c0 2024-06-10T01:27:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.622+0000 7f35a0912640 1 -- 172.21.15.7:0/2801298872 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f3594004050 con 0x7f359c1035c0 2024-06-10T01:27:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.623+0000 7f35a4624740 1 Processor -- start 2024-06-10T01:27:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.623+0000 7f35a0912640 1 --2- 172.21.15.7:0/2801298872 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35880a2b80 0x7f35880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.623+0000 7f35a4624740 1 -- start start 2024-06-10T01:27:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.623+0000 7f359a7fc640 1 --2- 172.21.15.7:0/2801298872 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35880a2b80 0x7f35880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.623+0000 7f35a4624740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573734ac010 0x5573734ac430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.623+0000 7f35a4624740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573734aca00 0x5573734b1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.623+0000 7f35a4624740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5573734b1b70 0x5573734b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.624+0000 7f35a4624740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55737340f4f0 con 0x5573734aca00 2024-06-10T01:27:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.624+0000 7f35a4624740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5573733ad870 con 0x5573734b1b70 2024-06-10T01:27:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.624+0000 7f35a4624740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55737340b4b0 con 0x5573734ac010 2024-06-10T01:27:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.624+0000 7f359b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5573734b1b70 0x5573734b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.624+0000 7f359a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573734aca00 0x5573734b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.624+0000 7f359b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5573734b1b70 0x5573734b4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:50108/0 (socket says 172.21.15.7:50108) 2024-06-10T01:27:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.624+0000 7f359b7fe640 1 -- 172.21.15.7:0/1145979929 learned_addr learned my addr 172.21.15.7:0/1145979929 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.624+0000 7f35a0912640 1 -- 172.21.15.7:0/2801298872 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f35940f0440 con 0x7f359c1035c0 2024-06-10T01:27:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.624+0000 7f359a7fc640 1 --2- 172.21.15.7:0/2801298872 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35880a2b80 0x7f35880a5040 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3584011550 tx=0x7f3584011950 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.624+0000 7f359affd640 1 --2- 172.21.15.7:0/1145979929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573734ac010 0x5573734ac430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.625+0000 7f359a7fc640 1 -- 172.21.15.7:0/1145979929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573734ac010 msgr2=0x5573734ac430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.625+0000 7f359a7fc640 1 --2- 172.21.15.7:0/1145979929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573734ac010 0x5573734ac430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.625+0000 7f359a7fc640 1 -- 172.21.15.7:0/1145979929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5573734b1b70 msgr2=0x5573734b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.625+0000 7f359a7fc640 1 --2- 172.21.15.7:0/1145979929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5573734b1b70 0x5573734b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.625+0000 7f359a7fc640 1 -- 172.21.15.7:0/1145979929 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573733c99b0 con 0x5573734aca00 2024-06-10T01:27:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.625+0000 7f359b7fe640 1 --2- 172.21.15.7:0/1145979929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5573734b1b70 0x5573734b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:27:29.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.625+0000 7f359a7fc640 1 --2- 172.21.15.7:0/1145979929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573734aca00 0x5573734b1500 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f3584004330 tx=0x7f358401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd7200e69997140 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:29.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.625+0000 7f3591ffb640 1 -- 172.21.15.7:0/1145979929 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358401a040 con 0x5573734aca00 2024-06-10T01:27:29.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.626+0000 7f3591ffb640 1 -- 172.21.15.7:0/1145979929 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3584024020 con 0x5573734aca00 2024-06-10T01:27:29.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.626+0000 7f3591ffb640 1 -- 172.21.15.7:0/1145979929 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3584005660 con 0x5573734aca00 2024-06-10T01:27:29.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.626+0000 7f35a4624740 1 -- 172.21.15.7:0/1145979929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573734aca00 msgr2=0x5573734b1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:29.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.626+0000 7f35a4624740 1 --2- 172.21.15.7:0/1145979929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573734aca00 0x5573734b1500 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f3584004330 tx=0x7f358401c000 comp rx=0 tx=0).stop 2024-06-10T01:27:29.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.626+0000 7f35a4624740 1 -- 172.21.15.7:0/1145979929 shutdown_connections 2024-06-10T01:27:29.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.626+0000 7f35a4624740 1 --2- 172.21.15.7:0/1145979929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5573734b1b70 0x5573734b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:29.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.626+0000 7f35a4624740 1 --2- 172.21.15.7:0/1145979929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573734aca00 0x5573734b1500 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:29.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.626+0000 7f35a4624740 1 --2- 172.21.15.7:0/1145979929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573734ac010 0x5573734ac430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:29.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.626+0000 7f35a4624740 1 -- 172.21.15.7:0/1145979929 >> 172.21.15.7:0/1145979929 conn(0x5573734a7680 msgr2=0x5573734a7a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:29.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.627+0000 7f35a4624740 1 -- 172.21.15.7:0/1145979929 shutdown_connections 2024-06-10T01:27:29.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.627+0000 7f35a4624740 1 -- 172.21.15.7:0/1145979929 wait complete. 2024-06-10T01:27:29.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.627+0000 7f35a4624740 1 -- start start 2024-06-10T01:27:29.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.627+0000 7f35a4624740 1 Processor -- start 2024-06-10T01:27:29.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.627+0000 7f35a4624740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5573734ac010 0x5573734b6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:29.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.627+0000 7f35a4624740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573734aca00 0x5573734b6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:29.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.627+0000 7f35a4624740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573734b1b70 0x5573734bd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:29.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.627+0000 7f35a4624740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5573734b4ac0 con 0x5573734b1b70 2024-06-10T01:27:29.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.627+0000 7f35a4624740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557373398ae0 con 0x5573734ac010 2024-06-10T01:27:29.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.627+0000 7f35a4624740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5573734b4c40 con 0x5573734aca00 2024-06-10T01:27:29.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.628+0000 7f35a4624740 10 client.0 _ll_register_callbacks cb 0x5573733d19b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:27:29.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.628+0000 7f359b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573734b1b70 0x5573734bd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:29.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.628+0000 7f359affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5573734ac010 0x5573734b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.628+0000 7f359affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5573734ac010 0x5573734b6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:50116/0 (socket says 172.21.15.7:50116) 2024-06-10T01:27:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.628+0000 7f359affd640 1 -- 172.21.15.7:0/4230275773 learned_addr learned my addr 172.21.15.7:0/4230275773 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.628+0000 7f359a7fc640 1 --2- 172.21.15.7:0/4230275773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573734aca00 0x5573734b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.628+0000 7f359affd640 1 -- 172.21.15.7:0/4230275773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573734aca00 msgr2=0x5573734b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.628+0000 7f359affd640 1 --2- 172.21.15.7:0/4230275773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573734aca00 0x5573734b6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.628+0000 7f359affd640 1 -- 172.21.15.7:0/4230275773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573734b1b70 msgr2=0x5573734bd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.628+0000 7f359affd640 1 --2- 172.21.15.7:0/4230275773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573734b1b70 0x5573734bd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.628+0000 7f359affd640 1 -- 172.21.15.7:0/4230275773 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557373123210 con 0x5573734ac010 2024-06-10T01:27:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.628+0000 7f359affd640 1 --2- 172.21.15.7:0/4230275773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5573734ac010 0x5573734b6220 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f358c012d20 tx=0x7f358c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=105048c24c70e21c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.628+0000 7f357affd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:27:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.629+0000 7f357affd640 1 -- 172.21.15.7:0/4230275773 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c012110 con 0x5573734ac010 2024-06-10T01:27:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.629+0000 7f357affd640 1 -- 172.21.15.7:0/4230275773 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f358c004050 con 0x5573734ac010 2024-06-10T01:27:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.629+0000 7f35a4624740 1 -- 172.21.15.7:0/4230275773 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3584024020 con 0x5573734ac010 2024-06-10T01:27:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.629+0000 7f357affd640 1 -- 172.21.15.7:0/4230275773 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f358c0122b0 con 0x5573734ac010 2024-06-10T01:27:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.629+0000 7f35a4624740 10 client.29786 fetch_fsmap learned FSMap version 53 2024-06-10T01:27:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.629+0000 7f35a4624740 1 -- 172.21.15.7:0/4230275773 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x5573731ef620 con 0x5573734ac010 2024-06-10T01:27:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.629+0000 7f357affd640 1 -- 172.21.15.7:0/4230275773 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3584024020 con 0x5573734ac010 2024-06-10T01:27:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.630+0000 7f357affd640 1 -- 172.21.15.7:0/4230275773 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7f358c010430 con 0x5573734ac010 2024-06-10T01:27:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.630+0000 7f35a4624740 10 client.29786 fetch_fsmap finished waiting for FSMap version 53 2024-06-10T01:27:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.630+0000 7f35a4624740 10 client.29786 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:27:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.630+0000 7f35a4624740 20 client.29786 populate_metadata read hostname 'smithi007' 2024-06-10T01:27:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.630+0000 7f35a4624740 1 --2- 172.21.15.7:0/4230275773 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5573734c2080 0x5573734c44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.630+0000 7f35a4624740 4 client.29786 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:27:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.630+0000 7f35a4624740 1 -- 172.21.15.7:0/4230275773 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557373422f10 con 0x5573734c2080 2024-06-10T01:27:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.630+0000 7f359a7fc640 1 --2- 172.21.15.7:0/4230275773 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5573734c2080 0x5573734c44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.631+0000 7f357affd640 10 client.29786 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.631+0000 7f359a7fc640 1 --2- 172.21.15.7:0/4230275773 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5573734c2080 0x5573734c44d0 crc :-1 s=READY pgs=487 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=267e2700ae6d326d server_cookie=fd15fe00e959c507 in_seq=0 out_seq=0 2024-06-10T01:27:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.633+0000 7f357affd640 1 -- 172.21.15.7:0/4230275773 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557373422f10 con 0x5573734c2080 2024-06-10T01:27:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.633+0000 7f357affd640 10 client.29786 handle_command_reply: tid=0 2024-06-10T01:27:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.634+0000 7f35a4624740 1 client.29786 shutdown 2024-06-10T01:27:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.634+0000 7f35a4624740 1 -- 172.21.15.7:0/4230275773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5573734ac010 msgr2=0x5573734b6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.634+0000 7f35a4624740 1 --2- 172.21.15.7:0/4230275773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5573734ac010 0x5573734b6220 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f358c012d20 tx=0x7f358c00f790 comp rx=0 tx=0).stop 2024-06-10T01:27:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.634+0000 7f35a4624740 1 -- 172.21.15.7:0/4230275773 shutdown_connections 2024-06-10T01:27:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.634+0000 7f35a4624740 1 --2- 172.21.15.7:0/4230275773 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5573734c2080 0x5573734c44d0 crc :-1 s=READY pgs=487 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.634+0000 7f35a4624740 1 --2- 172.21.15.7:0/4230275773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573734b1b70 0x5573734bd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.634+0000 7f35a4624740 1 --2- 172.21.15.7:0/4230275773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573734aca00 0x5573734b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.634+0000 7f357affd640 0 client.29786 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.634+0000 7f35a4624740 1 --2- 172.21.15.7:0/4230275773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5573734ac010 0x5573734b6220 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.634+0000 7f35a4624740 1 -- 172.21.15.7:0/4230275773 >> 172.21.15.7:0/4230275773 conn(0x5573734a7680 msgr2=0x5573734a67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.634+0000 7f35a4624740 1 -- 172.21.15.7:0/4230275773 shutdown_connections 2024-06-10T01:27:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.634+0000 7f35a4624740 1 -- 172.21.15.7:0/4230275773 wait complete. 2024-06-10T01:27:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.634+0000 7f35a4624740 20 client.29786 trim_cache size 0 max 16384 2024-06-10T01:27:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.657+0000 7f35a4624740 1 Processor -- start 2024-06-10T01:27:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.657+0000 7f35a4624740 1 -- start start 2024-06-10T01:27:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.657+0000 7f35a4624740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573734ac010 0x5573734aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.657+0000 7f35a4624740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573734aca00 0x5573734aafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.657+0000 7f35a4624740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5573734b1b70 0x5573734bd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.657+0000 7f35a4624740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1aac90 con 0x5573734ac010 2024-06-10T01:27:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.657+0000 7f35a4624740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1a8560 con 0x5573734b1b70 2024-06-10T01:27:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.657+0000 7f35a4624740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1a63b0 con 0x5573734aca00 2024-06-10T01:27:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.658+0000 7f359affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573734ac010 0x5573734aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.658+0000 7f359b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5573734b1b70 0x5573734bd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.658+0000 7f359affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573734ac010 0x5573734aaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43988/0 (socket says 172.21.15.7:43988) 2024-06-10T01:27:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.658+0000 7f359affd640 1 -- 172.21.15.7:0/782208850 learned_addr learned my addr 172.21.15.7:0/782208850 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.658+0000 7f359a7fc640 1 --2- 172.21.15.7:0/782208850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573734aca00 0x5573734aafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.658+0000 7f359affd640 1 -- 172.21.15.7:0/782208850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573734aca00 msgr2=0x5573734aafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.658+0000 7f359affd640 1 --2- 172.21.15.7:0/782208850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573734aca00 0x5573734aafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.658+0000 7f359affd640 1 -- 172.21.15.7:0/782208850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5573734b1b70 msgr2=0x5573734bd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.658+0000 7f359affd640 1 --2- 172.21.15.7:0/782208850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5573734b1b70 0x5573734bd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.658+0000 7f359affd640 1 -- 172.21.15.7:0/782208850 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573734c0a50 con 0x5573734ac010 2024-06-10T01:27:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.658+0000 7f359b7fe640 1 --2- 172.21.15.7:0/782208850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5573734b1b70 0x5573734bd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:27:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.658+0000 7f359affd640 1 --2- 172.21.15.7:0/782208850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573734ac010 0x5573734aaa70 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f358c010800 tx=0x7f358c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d355e0f20cb6ff server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.658+0000 7f35917fa640 1 -- 172.21.15.7:0/782208850 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c025030 con 0x5573734ac010 2024-06-10T01:27:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.658+0000 7f35917fa640 1 -- 172.21.15.7:0/782208850 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f358c02e070 con 0x5573734ac010 2024-06-10T01:27:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.658+0000 7f35917fa640 1 -- 172.21.15.7:0/782208850 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c0035f0 con 0x5573734ac010 2024-06-10T01:27:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.659+0000 7f35a4624740 1 -- 172.21.15.7:0/782208850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573734ac010 msgr2=0x5573734aaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.659+0000 7f35a4624740 1 --2- 172.21.15.7:0/782208850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573734ac010 0x5573734aaa70 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f358c010800 tx=0x7f358c009bf0 comp rx=0 tx=0).stop 2024-06-10T01:27:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.659+0000 7f35a4624740 1 -- 172.21.15.7:0/782208850 shutdown_connections 2024-06-10T01:27:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.659+0000 7f35a4624740 1 --2- 172.21.15.7:0/782208850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5573734b1b70 0x5573734bd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.659+0000 7f35a4624740 1 --2- 172.21.15.7:0/782208850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573734aca00 0x5573734aafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.659+0000 7f35a4624740 1 --2- 172.21.15.7:0/782208850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573734ac010 0x5573734aaa70 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.659+0000 7f35a4624740 1 -- 172.21.15.7:0/782208850 >> 172.21.15.7:0/782208850 conn(0x5573734a7680 msgr2=0x5573734aa360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.659+0000 7f35a4624740 1 -- 172.21.15.7:0/782208850 shutdown_connections 2024-06-10T01:27:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.660+0000 7f35a4624740 1 -- 172.21.15.7:0/782208850 wait complete. 2024-06-10T01:27:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.660+0000 7f35a4624740 1 -- start start 2024-06-10T01:27:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.660+0000 7f35a4624740 1 Processor -- start 2024-06-10T01:27:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.660+0000 7f35a4624740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573734ac010 0x5573734ba6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.660+0000 7f35a4624740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5573734aca00 0x5573734bac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.660+0000 7f35a4624740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573734b1b70 0x55737352adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.660+0000 7f35a4624740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1a6a20 con 0x5573734b1b70 2024-06-10T01:27:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.660+0000 7f35a4624740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1a7e90 con 0x5573734aca00 2024-06-10T01:27:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.660+0000 7f35a4624740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1a70f0 con 0x5573734ac010 2024-06-10T01:27:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.661+0000 7f35a4624740 10 client.0 _ll_register_callbacks cb 0x7f358c012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:27:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.661+0000 7f359a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5573734aca00 0x5573734bac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.661+0000 7f359b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573734b1b70 0x55737352adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.661+0000 7f359affd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573734ac010 0x5573734ba6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.661+0000 7f359a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5573734aca00 0x5573734bac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:50134/0 (socket says 172.21.15.7:50134) 2024-06-10T01:27:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.661+0000 7f359a7fc640 1 -- 172.21.15.7:0/3375851167 learned_addr learned my addr 172.21.15.7:0/3375851167 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.661+0000 7f359a7fc640 1 -- 172.21.15.7:0/3375851167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573734ac010 msgr2=0x5573734ba6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.661+0000 7f359a7fc640 1 --2- 172.21.15.7:0/3375851167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573734ac010 0x5573734ba6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.661+0000 7f359a7fc640 1 -- 172.21.15.7:0/3375851167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573734b1b70 msgr2=0x55737352adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.661+0000 7f359a7fc640 1 --2- 172.21.15.7:0/3375851167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573734b1b70 0x55737352adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.661+0000 7f359a7fc640 1 -- 172.21.15.7:0/3375851167 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55737352b330 con 0x5573734aca00 2024-06-10T01:27:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.661+0000 7f359b7fe640 1 --2- 172.21.15.7:0/3375851167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573734b1b70 0x55737352adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:27:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.661+0000 7f359a7fc640 1 --2- 172.21.15.7:0/3375851167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5573734aca00 0x5573734bac00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3584003fd0 tx=0x7f35840099b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dbfdfc948fd3a1a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.661+0000 7f357affd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:27:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.661+0000 7f357affd640 1 -- 172.21.15.7:0/3375851167 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3584038020 con 0x5573734aca00 2024-06-10T01:27:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.661+0000 7f35a4624740 1 -- 172.21.15.7:0/3375851167 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f358c02e070 con 0x5573734aca00 2024-06-10T01:27:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.661+0000 7f357affd640 1 -- 172.21.15.7:0/3375851167 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7f3584005100 con 0x5573734aca00 2024-06-10T01:27:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.662+0000 7f357affd640 1 -- 172.21.15.7:0/3375851167 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f3584043050 con 0x5573734aca00 2024-06-10T01:27:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.662+0000 7f35a4624740 10 client.29792 fetch_fsmap learned FSMap version 53 2024-06-10T01:27:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.662+0000 7f35a4624740 1 -- 172.21.15.7:0/3375851167 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55737352c600 con 0x5573734aca00 2024-06-10T01:27:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.662+0000 7f357affd640 1 -- 172.21.15.7:0/3375851167 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f358c02e070 con 0x5573734aca00 2024-06-10T01:27:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.662+0000 7f357affd640 1 -- 172.21.15.7:0/3375851167 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7f358400fbd0 con 0x5573734aca00 2024-06-10T01:27:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.662+0000 7f35a4624740 10 client.29792 fetch_fsmap finished waiting for FSMap version 53 2024-06-10T01:27:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.662+0000 7f35a4624740 10 client.29792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:27:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.662+0000 7f35a4624740 20 client.29792 populate_metadata read hostname 'smithi007' 2024-06-10T01:27:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.662+0000 7f35a4624740 1 --2- 172.21.15.7:0/3375851167 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55737352cea0 0x55737352f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.662+0000 7f35a4624740 4 client.29792 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:27:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.662+0000 7f35a4624740 1 -- 172.21.15.7:0/3375851167 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55737341cc20 con 0x55737352cea0 2024-06-10T01:27:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.662+0000 7f359affd640 1 --2- 172.21.15.7:0/3375851167 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55737352cea0 0x55737352f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.663+0000 7f359affd640 1 --2- 172.21.15.7:0/3375851167 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55737352cea0 0x55737352f2c0 crc :-1 s=READY pgs=488 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13f36f2ed64386b4 server_cookie=9431db37d99969be in_seq=0 out_seq=0 2024-06-10T01:27:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:29.663+0000 7f357affd640 10 client.29792 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:30.406+0000 7f357affd640 1 -- 172.21.15.7:0/3375851167 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55737341cc20 con 0x55737352cea0 2024-06-10T01:27:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:30.406+0000 7f357affd640 10 client.29792 handle_command_reply: tid=0 2024-06-10T01:27:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:30.406+0000 7f35a4624740 1 client.29792 shutdown 2024-06-10T01:27:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:30.406+0000 7f35a4624740 1 -- 172.21.15.7:0/3375851167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5573734aca00 msgr2=0x5573734bac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:30.406+0000 7f35a4624740 1 --2- 172.21.15.7:0/3375851167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5573734aca00 0x5573734bac00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3584003fd0 tx=0x7f35840099b0 comp rx=0 tx=0).stop 2024-06-10T01:27:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:30.406+0000 7f35a4624740 1 -- 172.21.15.7:0/3375851167 shutdown_connections 2024-06-10T01:27:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:30.406+0000 7f35a4624740 1 --2- 172.21.15.7:0/3375851167 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55737352cea0 0x55737352f2c0 crc :-1 s=READY pgs=488 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:30.406+0000 7f35a4624740 1 --2- 172.21.15.7:0/3375851167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573734b1b70 0x55737352adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:30.406+0000 7f35a4624740 1 --2- 172.21.15.7:0/3375851167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5573734aca00 0x5573734bac00 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:30.406+0000 7f35a4624740 1 --2- 172.21.15.7:0/3375851167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5573734ac010 0x5573734ba6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:30.406+0000 7f35a4624740 1 -- 172.21.15.7:0/3375851167 >> 172.21.15.7:0/3375851167 conn(0x5573734a7680 msgr2=0x5573734a67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:30.406+0000 7f357affd640 0 client.29792 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:30.407+0000 7f35a4624740 1 -- 172.21.15.7:0/3375851167 shutdown_connections 2024-06-10T01:27:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:30.407+0000 7f35a4624740 1 -- 172.21.15.7:0/3375851167 wait complete. 2024-06-10T01:27:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:30.407+0000 7f35a4624740 20 client.29792 trim_cache size 0 max 16384 2024-06-10T01:27:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:27:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:27:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:27:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:30.408+0000 7f35a1914640 1 -- 172.21.15.7:0/2801298872 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35880a2b80 msgr2=0x7f35880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:30.408+0000 7f35a1914640 1 --2- 172.21.15.7:0/2801298872 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35880a2b80 0x7f35880a5040 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3584011550 tx=0x7f3584011950 comp rx=0 tx=0).stop 2024-06-10T01:27:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:30.408+0000 7f35a1914640 1 -- 172.21.15.7:0/2801298872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f359c1035c0 msgr2=0x7f359c19f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:30.408+0000 7f35a1914640 1 --2- 172.21.15.7:0/2801298872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f359c1035c0 0x7f359c19f290 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3594031ae0 tx=0x7f3594004ab0 comp rx=0 tx=0).stop 2024-06-10T01:27:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:30.408+0000 7f35a1914640 1 -- 172.21.15.7:0/2801298872 shutdown_connections 2024-06-10T01:27:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:30.408+0000 7f35a1914640 1 --2- 172.21.15.7:0/2801298872 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f35880a2b80 0x7f35880a5040 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:30.408+0000 7f35a1914640 1 --2- 172.21.15.7:0/2801298872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f359c1035c0 0x7f359c19f290 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:30.408+0000 7f35a1914640 1 --2- 172.21.15.7:0/2801298872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f359c102c00 0x7f359c198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:30.408+0000 7f35a1914640 1 --2- 172.21.15.7:0/2801298872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f359c101a10 0x7f359c197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:30.408+0000 7f35a1914640 1 -- 172.21.15.7:0/2801298872 >> 172.21.15.7:0/2801298872 conn(0x7f359c0fcf80 msgr2=0x7f359c105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:30.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:30.409+0000 7f35a1914640 1 -- 172.21.15.7:0/2801298872 shutdown_connections 2024-06-10T01:27:30.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:30.409+0000 7f35a1914640 1 -- 172.21.15.7:0/2801298872 wait complete. 2024-06-10T01:27:30.553 INFO:tasks.workunit.client.0.smithi007.stdout: 10 196225 22.89 MB/sec execute 367 sec latency 366.268 ms 2024-06-10T01:27:31.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:31 smithi078 ceph-mon[36683]: pgmap v1078: 129 pgs: 129 active+clean; 671 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 4.8 MiB/s wr, 641 op/s 2024-06-10T01:27:31.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:31 smithi175 ceph-mon[36636]: pgmap v1078: 129 pgs: 129 active+clean; 671 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 4.8 MiB/s wr, 641 op/s 2024-06-10T01:27:31.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:31 smithi007 ceph-mon[32398]: pgmap v1078: 129 pgs: 129 active+clean; 671 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 4.8 MiB/s wr, 641 op/s 2024-06-10T01:27:31.552 INFO:tasks.workunit.client.0.smithi007.stdout: 10 196652 22.86 MB/sec execute 368 sec latency 93.094 ms 2024-06-10T01:27:32.553 INFO:tasks.workunit.client.0.smithi007.stdout: 10 197124 22.85 MB/sec execute 369 sec latency 95.045 ms 2024-06-10T01:27:33.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:33 smithi078 ceph-mon[36683]: pgmap v1079: 129 pgs: 129 active+clean; 681 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.6 MiB/s wr, 760 op/s 2024-06-10T01:27:33.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:33 smithi175 ceph-mon[36636]: pgmap v1079: 129 pgs: 129 active+clean; 681 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.6 MiB/s wr, 760 op/s 2024-06-10T01:27:33.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:33 smithi007 ceph-mon[32398]: pgmap v1079: 129 pgs: 129 active+clean; 681 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.6 MiB/s wr, 760 op/s 2024-06-10T01:27:33.552 INFO:tasks.workunit.client.0.smithi007.stdout: 10 197994 22.91 MB/sec execute 370 sec latency 100.186 ms 2024-06-10T01:27:34.553 INFO:tasks.workunit.client.0.smithi007.stdout: 10 198551 22.93 MB/sec execute 371 sec latency 352.959 ms 2024-06-10T01:27:35.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:35 smithi078 ceph-mon[36683]: pgmap v1080: 129 pgs: 129 active+clean; 683 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 3.3 MiB/s wr, 515 op/s 2024-06-10T01:27:35.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:35 smithi175 ceph-mon[36636]: pgmap v1080: 129 pgs: 129 active+clean; 683 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 3.3 MiB/s wr, 515 op/s 2024-06-10T01:27:35.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:35 smithi007 ceph-mon[32398]: pgmap v1080: 129 pgs: 129 active+clean; 683 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 3.3 MiB/s wr, 515 op/s 2024-06-10T01:27:35.553 INFO:tasks.workunit.client.0.smithi007.stdout: 10 198577 22.87 MB/sec execute 372 sec latency 1330.797 ms 2024-06-10T01:27:36.553 INFO:tasks.workunit.client.0.smithi007.stdout: 10 198744 22.84 MB/sec execute 373 sec latency 1648.391 ms 2024-06-10T01:27:37.553 INFO:tasks.workunit.client.0.smithi007.stdout: 10 198967 22.81 MB/sec execute 374 sec latency 218.023 ms 2024-06-10T01:27:37.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:37 smithi078 ceph-mon[36683]: pgmap v1081: 129 pgs: 129 active+clean; 712 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 7.1 MiB/s wr, 669 op/s 2024-06-10T01:27:37.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:37 smithi175 ceph-mon[36636]: pgmap v1081: 129 pgs: 129 active+clean; 712 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 7.1 MiB/s wr, 669 op/s 2024-06-10T01:27:38.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:37 smithi007 ceph-mon[32398]: pgmap v1081: 129 pgs: 129 active+clean; 712 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 7.1 MiB/s wr, 669 op/s 2024-06-10T01:27:38.553 INFO:tasks.workunit.client.0.smithi007.stdout: 10 199221 22.80 MB/sec execute 375 sec latency 117.986 ms 2024-06-10T01:27:39.553 INFO:tasks.workunit.client.0.smithi007.stdout: 10 199669 22.81 MB/sec execute 376 sec latency 107.173 ms 2024-06-10T01:27:39.812 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:39 smithi007 ceph-mon[32398]: pgmap v1082: 129 pgs: 129 active+clean; 733 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.4 MiB/s wr, 632 op/s 2024-06-10T01:27:39.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:39 smithi078 ceph-mon[36683]: pgmap v1082: 129 pgs: 129 active+clean; 733 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.4 MiB/s wr, 632 op/s 2024-06-10T01:27:39.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:39 smithi175 ceph-mon[36636]: pgmap v1082: 129 pgs: 129 active+clean; 733 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.4 MiB/s wr, 632 op/s 2024-06-10T01:27:40.553 INFO:tasks.workunit.client.0.smithi007.stdout: 10 200352 22.83 MB/sec execute 377 sec latency 65.754 ms 2024-06-10T01:27:41.553 INFO:tasks.workunit.client.0.smithi007.stdout: 10 200817 22.85 MB/sec execute 378 sec latency 117.903 ms 2024-06-10T01:27:41.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:41 smithi007 ceph-mon[32398]: pgmap v1083: 129 pgs: 129 active+clean; 732 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.6 MiB/s wr, 647 op/s 2024-06-10T01:27:41.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:41 smithi078 ceph-mon[36683]: pgmap v1083: 129 pgs: 129 active+clean; 732 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.6 MiB/s wr, 647 op/s 2024-06-10T01:27:41.954 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:41 smithi175 ceph-mon[36636]: pgmap v1083: 129 pgs: 129 active+clean; 732 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.6 MiB/s wr, 647 op/s 2024-06-10T01:27:42.553 INFO:tasks.workunit.client.0.smithi007.stdout: 10 201464 22.88 MB/sec execute 379 sec latency 178.031 ms 2024-06-10T01:27:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.536+0000 7fdbb6eee640 1 -- 172.21.15.7:0/1691729843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbb01024b0 msgr2=0x7fdbb0108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.536+0000 7fdbb6eee640 1 --2- 172.21.15.7:0/1691729843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbb01024b0 0x7fdbb0108d40 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009ca0 tx=0x7fdba802a8c0 comp rx=0 tx=0).stop 2024-06-10T01:27:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.536+0000 7fdbb6eee640 1 -- 172.21.15.7:0/1691729843 shutdown_connections 2024-06-10T01:27:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.536+0000 7fdbb6eee640 1 --2- 172.21.15.7:0/1691729843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbb01024b0 0x7fdbb0108d40 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.536+0000 7fdbb6eee640 1 --2- 172.21.15.7:0/1691729843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbb0101af0 0x7fdbb0101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.536+0000 7fdbb6eee640 1 --2- 172.21.15.7:0/1691729843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbb0100900 0x7fdbb0100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.536+0000 7fdbb6eee640 1 -- 172.21.15.7:0/1691729843 >> 172.21.15.7:0/1691729843 conn(0x7fdbb00fbe70 msgr2=0x7fdbb00fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.537+0000 7fdbb6eee640 1 -- 172.21.15.7:0/1691729843 shutdown_connections 2024-06-10T01:27:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.537+0000 7fdbb6eee640 1 -- 172.21.15.7:0/1691729843 wait complete. 2024-06-10T01:27:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.537+0000 7fdbb6eee640 1 Processor -- start 2024-06-10T01:27:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.538+0000 7fdbb6eee640 1 -- start start 2024-06-10T01:27:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.538+0000 7fdbb6eee640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbb0100900 0x7fdbb0199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.538+0000 7fdbb6eee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbb0101af0 0x7fdbb019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.538+0000 7fdbb6eee640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbb01024b0 0x7fdbb01a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.538+0000 7fdbb6eee640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb010b4d0 con 0x7fdbb0100900 2024-06-10T01:27:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.538+0000 7fdbb6eee640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb010b350 con 0x7fdbb0101af0 2024-06-10T01:27:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.538+0000 7fdbb6eee640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb010b650 con 0x7fdbb01024b0 2024-06-10T01:27:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.538+0000 7fdbb4c8a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbb0100900 0x7fdbb0199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.538+0000 7fdbb4c8a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbb0100900 0x7fdbb0199eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33946/0 (socket says 172.21.15.7:33946) 2024-06-10T01:27:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.538+0000 7fdbb4c8a640 1 -- 172.21.15.7:0/1761263040 learned_addr learned my addr 172.21.15.7:0/1761263040 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.538+0000 7fdbaffff640 1 --2- 172.21.15.7:0/1761263040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbb0101af0 0x7fdbb019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.538+0000 7fdbb548b640 1 --2- 172.21.15.7:0/1761263040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbb01024b0 0x7fdbb01a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.538+0000 7fdbb4c8a640 1 -- 172.21.15.7:0/1761263040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbb01024b0 msgr2=0x7fdbb01a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.538+0000 7fdbb4c8a640 1 --2- 172.21.15.7:0/1761263040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbb01024b0 0x7fdbb01a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.538+0000 7fdbb4c8a640 1 -- 172.21.15.7:0/1761263040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbb0101af0 msgr2=0x7fdbb019a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.538+0000 7fdbb4c8a640 1 --2- 172.21.15.7:0/1761263040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbb0101af0 0x7fdbb019a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.538+0000 7fdbb4c8a640 1 -- 172.21.15.7:0/1761263040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb01a1ad0 con 0x7fdbb0100900 2024-06-10T01:27:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.538+0000 7fdbaffff640 1 --2- 172.21.15.7:0/1761263040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbb0101af0 0x7fdbb019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:27:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.538+0000 7fdbb4c8a640 1 --2- 172.21.15.7:0/1761263040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbb0100900 0x7fdbb0199eb0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fdba4009960 tx=0x7fdba4009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.539+0000 7fdbadffb640 1 -- 172.21.15.7:0/1761263040 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba400ca60 con 0x7fdbb0100900 2024-06-10T01:27:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.539+0000 7fdbb6eee640 1 -- 172.21.15.7:0/1761263040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb01a1dc0 con 0x7fdbb0100900 2024-06-10T01:27:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.539+0000 7fdbadffb640 1 -- 172.21.15.7:0/1761263040 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdba400cc00 con 0x7fdbb0100900 2024-06-10T01:27:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.539+0000 7fdbadffb640 1 -- 172.21.15.7:0/1761263040 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba4003e00 con 0x7fdbb0100900 2024-06-10T01:27:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.539+0000 7fdbb6eee640 1 -- 172.21.15.7:0/1761263040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb01a2230 con 0x7fdbb0100900 2024-06-10T01:27:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.542+0000 7fdbadffb640 1 -- 172.21.15.7:0/1761263040 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fdba401d020 con 0x7fdbb0100900 2024-06-10T01:27:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.542+0000 7fdbb9bfc740 1 Processor -- start 2024-06-10T01:27:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.542+0000 7fdbb9bfc740 1 -- start start 2024-06-10T01:27:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.543+0000 7fdbb9bfc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55621d29a010 0x55621d29a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.543+0000 7fdbb9bfc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55621d29aa00 0x55621d29f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.543+0000 7fdbb9bfc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55621d29fb70 0x55621d2a2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.543+0000 7fdbb9bfc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55621d1fd4f0 con 0x55621d29aa00 2024-06-10T01:27:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.543+0000 7fdbb9bfc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55621d19b870 con 0x55621d29fb70 2024-06-10T01:27:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.543+0000 7fdbb548b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55621d29a010 0x55621d29a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.543+0000 7fdbb548b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55621d29a010 0x55621d29a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:40212/0 (socket says 172.21.15.7:40212) 2024-06-10T01:27:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.543+0000 7fdbb548b640 1 -- 172.21.15.7:0/1951802786 learned_addr learned my addr 172.21.15.7:0/1951802786 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.543+0000 7fdbb4c8a640 1 --2- 172.21.15.7:0/1951802786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55621d29aa00 0x55621d29f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.543+0000 7fdbb9bfc740 1 -- 172.21.15.7:0/1951802786 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55621d1f94b0 con 0x55621d29a010 2024-06-10T01:27:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.543+0000 7fdbaffff640 1 --2- 172.21.15.7:0/1951802786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55621d29fb70 0x55621d2a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.543+0000 7fdbb548b640 1 -- 172.21.15.7:0/1951802786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55621d29fb70 msgr2=0x55621d2a2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.543+0000 7fdbb548b640 1 --2- 172.21.15.7:0/1951802786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55621d29fb70 0x55621d2a2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.543+0000 7fdbadffb640 1 --2- 172.21.15.7:0/1761263040 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdba00a2b80 0x7fdba00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.543+0000 7fdbb548b640 1 -- 172.21.15.7:0/1951802786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55621d29aa00 msgr2=0x55621d29f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.543+0000 7fdbb548b640 1 --2- 172.21.15.7:0/1951802786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55621d29aa00 0x55621d29f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.543+0000 7fdbb548b640 1 -- 172.21.15.7:0/1951802786 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55621d1b79b0 con 0x55621d29a010 2024-06-10T01:27:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.544+0000 7fdbadffb640 1 -- 172.21.15.7:0/1761263040 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fdba40ce8f0 con 0x7fdbb0100900 2024-06-10T01:27:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.544+0000 7fdbb548b640 1 --2- 172.21.15.7:0/1761263040 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdba00a2b80 0x7fdba00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.544+0000 7fdbb548b640 1 --2- 172.21.15.7:0/1951802786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55621d29a010 0x55621d29a430 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009ca0 tx=0x7fdba80093c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfbfff15cd0b63f3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.544+0000 7fdb9affd640 1 -- 172.21.15.7:0/1951802786 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba804a070 con 0x55621d29a010 2024-06-10T01:27:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.544+0000 7fdb9affd640 1 -- 172.21.15.7:0/1951802786 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdba8036030 con 0x55621d29a010 2024-06-10T01:27:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.544+0000 7fdb9affd640 1 -- 172.21.15.7:0/1951802786 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba802f870 con 0x55621d29a010 2024-06-10T01:27:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.545+0000 7fdbb9bfc740 1 -- 172.21.15.7:0/1951802786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55621d29a010 msgr2=0x55621d29a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.545+0000 7fdbb548b640 1 --2- 172.21.15.7:0/1761263040 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdba00a2b80 0x7fdba00a5040 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009430 tx=0x7fdba802faf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.545+0000 7fdbb9bfc740 1 --2- 172.21.15.7:0/1951802786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55621d29a010 0x55621d29a430 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009ca0 tx=0x7fdba80093c0 comp rx=0 tx=0).stop 2024-06-10T01:27:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.545+0000 7fdbb9bfc740 1 -- 172.21.15.7:0/1951802786 shutdown_connections 2024-06-10T01:27:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.545+0000 7fdbb9bfc740 1 --2- 172.21.15.7:0/1951802786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55621d29fb70 0x55621d2a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.545+0000 7fdbb9bfc740 1 --2- 172.21.15.7:0/1951802786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55621d29aa00 0x55621d29f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.545+0000 7fdbb9bfc740 1 --2- 172.21.15.7:0/1951802786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55621d29a010 0x55621d29a430 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.545+0000 7fdbb9bfc740 1 -- 172.21.15.7:0/1951802786 >> 172.21.15.7:0/1951802786 conn(0x55621d295680 msgr2=0x55621d295a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.545+0000 7fdbb9bfc740 1 -- 172.21.15.7:0/1951802786 shutdown_connections 2024-06-10T01:27:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.545+0000 7fdbb9bfc740 1 -- 172.21.15.7:0/1951802786 wait complete. 2024-06-10T01:27:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.545+0000 7fdbb9bfc740 1 -- start start 2024-06-10T01:27:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.546+0000 7fdbb9bfc740 1 Processor -- start 2024-06-10T01:27:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.546+0000 7fdbb9bfc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55621d29a010 0x55621d2a4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.546+0000 7fdbb9bfc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55621d29aa00 0x55621d2a4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.546+0000 7fdbb9bfc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55621d29fb70 0x55621d2ab6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.546+0000 7fdbb9bfc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55621d2a29a0 con 0x55621d29a010 2024-06-10T01:27:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.546+0000 7fdbb9bfc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55621d186ae0 con 0x55621d29aa00 2024-06-10T01:27:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.546+0000 7fdbb9bfc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55621d2a2b20 con 0x55621d29fb70 2024-06-10T01:27:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.546+0000 7fdbb4c8a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55621d29a010 0x55621d2a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.546+0000 7fdbaffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55621d29aa00 0x55621d2a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.546+0000 7fdbb9bfc740 10 client.0 _ll_register_callbacks cb 0x55621d1bf9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:27:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.546+0000 7fdbb4c8a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55621d29a010 0x55621d2a4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33974/0 (socket says 172.21.15.7:33974) 2024-06-10T01:27:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.546+0000 7fdbb4c8a640 1 -- 172.21.15.7:0/3575635324 learned_addr learned my addr 172.21.15.7:0/3575635324 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.547+0000 7fdbb4c8a640 1 -- 172.21.15.7:0/3575635324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55621d29fb70 msgr2=0x55621d2ab6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:27:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.547+0000 7fdbb4c8a640 1 --2- 172.21.15.7:0/3575635324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55621d29fb70 0x55621d2ab6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.547+0000 7fdbb4c8a640 1 -- 172.21.15.7:0/3575635324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55621d29aa00 msgr2=0x55621d2a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.547+0000 7fdbb4c8a640 1 --2- 172.21.15.7:0/3575635324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55621d29aa00 0x55621d2a4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.547+0000 7fdbb4c8a640 1 -- 172.21.15.7:0/3575635324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55621cf11210 con 0x55621d29a010 2024-06-10T01:27:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.547+0000 7fdbb4c8a640 1 --2- 172.21.15.7:0/3575635324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55621d29a010 0x55621d2a4140 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fdba400c640 tx=0x7fdba40036b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5dfc500948201fc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.547+0000 7fdb8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:27:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.547+0000 7fdb8ffff640 1 -- 172.21.15.7:0/3575635324 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba409aba0 con 0x55621d29a010 2024-06-10T01:27:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.547+0000 7fdb8ffff640 1 -- 172.21.15.7:0/3575635324 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fdba40a35b0 con 0x55621d29a010 2024-06-10T01:27:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.547+0000 7fdbb9bfc740 1 -- 172.21.15.7:0/3575635324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdba8036030 con 0x55621d29a010 2024-06-10T01:27:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.547+0000 7fdb8ffff640 1 -- 172.21.15.7:0/3575635324 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fdba409a730 con 0x55621d29a010 2024-06-10T01:27:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.547+0000 7fdbb9bfc740 10 client.20604 fetch_fsmap learned FSMap version 53 2024-06-10T01:27:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.547+0000 7fdbb9bfc740 1 -- 172.21.15.7:0/3575635324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55621cfdd620 con 0x55621d29a010 2024-06-10T01:27:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.548+0000 7fdb8ffff640 1 -- 172.21.15.7:0/3575635324 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdba8036030 con 0x55621d29a010 2024-06-10T01:27:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.548+0000 7fdb8ffff640 1 -- 172.21.15.7:0/3575635324 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7fdba40b0060 con 0x55621d29a010 2024-06-10T01:27:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.548+0000 7fdbb9bfc740 10 client.20604 fetch_fsmap finished waiting for FSMap version 53 2024-06-10T01:27:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.548+0000 7fdbb9bfc740 10 client.20604 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:27:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.548+0000 7fdbb9bfc740 20 client.20604 populate_metadata read hostname 'smithi007' 2024-06-10T01:27:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.548+0000 7fdbb9bfc740 1 --2- 172.21.15.7:0/3575635324 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55621d2b00a0 0x55621d2b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.548+0000 7fdbb9bfc740 4 client.20604 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:27:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.548+0000 7fdbb9bfc740 1 -- 172.21.15.7:0/3575635324 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55621d210f10 con 0x55621d2b00a0 2024-06-10T01:27:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.548+0000 7fdbaffff640 1 --2- 172.21.15.7:0/3575635324 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55621d2b00a0 0x55621d2b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.549+0000 7fdbaffff640 1 --2- 172.21.15.7:0/3575635324 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55621d2b00a0 0x55621d2b24b0 crc :-1 s=READY pgs=489 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cb774170d70a8f0 server_cookie=95115b3df8b77e5f in_seq=0 out_seq=0 2024-06-10T01:27:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.549+0000 7fdb8ffff640 10 client.20604 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.551+0000 7fdb8ffff640 1 -- 172.21.15.7:0/3575635324 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55621d210f10 con 0x55621d2b00a0 2024-06-10T01:27:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.551+0000 7fdb8ffff640 10 client.20604 handle_command_reply: tid=0 2024-06-10T01:27:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.551+0000 7fdbb9bfc740 1 client.20604 shutdown 2024-06-10T01:27:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.552+0000 7fdbb9bfc740 1 -- 172.21.15.7:0/3575635324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55621d29a010 msgr2=0x55621d2a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.552+0000 7fdbb9bfc740 1 --2- 172.21.15.7:0/3575635324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55621d29a010 0x55621d2a4140 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fdba400c640 tx=0x7fdba40036b0 comp rx=0 tx=0).stop 2024-06-10T01:27:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.552+0000 7fdbb9bfc740 1 -- 172.21.15.7:0/3575635324 shutdown_connections 2024-06-10T01:27:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.552+0000 7fdbb9bfc740 1 --2- 172.21.15.7:0/3575635324 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55621d2b00a0 0x55621d2b24b0 crc :-1 s=READY pgs=489 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.552+0000 7fdbb9bfc740 1 --2- 172.21.15.7:0/3575635324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55621d29fb70 0x55621d2ab6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.552+0000 7fdbb9bfc740 1 --2- 172.21.15.7:0/3575635324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55621d29aa00 0x55621d2a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.552+0000 7fdbb9bfc740 1 --2- 172.21.15.7:0/3575635324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55621d29a010 0x55621d2a4140 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.552+0000 7fdbb9bfc740 1 -- 172.21.15.7:0/3575635324 >> 172.21.15.7:0/3575635324 conn(0x55621d295680 msgr2=0x55621d2947e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.552+0000 7fdb8ffff640 0 client.20604 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.552+0000 7fdbb9bfc740 1 -- 172.21.15.7:0/3575635324 shutdown_connections 2024-06-10T01:27:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.552+0000 7fdbb9bfc740 1 -- 172.21.15.7:0/3575635324 wait complete. 2024-06-10T01:27:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.552+0000 7fdbb9bfc740 20 client.20604 trim_cache size 0 max 16384 2024-06-10T01:27:43.553 INFO:tasks.workunit.client.0.smithi007.stdout: 10 202047 22.89 MB/sec execute 380 sec latency 170.076 ms 2024-06-10T01:27:43.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.572+0000 7fdbb9bfc740 1 Processor -- start 2024-06-10T01:27:43.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.572+0000 7fdbb9bfc740 1 -- start start 2024-06-10T01:27:43.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.572+0000 7fdbb9bfc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55621d29a010 0x55621d298ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:43.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.572+0000 7fdbb9bfc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55621d29aa00 0x55621d2ab650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:43.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.572+0000 7fdbb9bfc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55621d29fb70 0x55621d2a3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:43.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.572+0000 7fdbb9bfc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb01af020 con 0x55621d29aa00 2024-06-10T01:27:43.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.572+0000 7fdbb9bfc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb01ac8f0 con 0x55621d29fb70 2024-06-10T01:27:43.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.572+0000 7fdbb9bfc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb01aa740 con 0x55621d29a010 2024-06-10T01:27:43.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.572+0000 7fdbaffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55621d29aa00 0x55621d2ab650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:43.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.572+0000 7fdbaffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55621d29aa00 0x55621d2ab650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33990/0 (socket says 172.21.15.7:33990) 2024-06-10T01:27:43.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.572+0000 7fdbaffff640 1 -- 172.21.15.7:0/1045628706 learned_addr learned my addr 172.21.15.7:0/1045628706 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:43.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.572+0000 7fdbb4c8a640 1 --2- 172.21.15.7:0/1045628706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55621d29a010 0x55621d298ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:43.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.572+0000 7fdbb548b640 1 --2- 172.21.15.7:0/1045628706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55621d29fb70 0x55621d2a3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:43.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.573+0000 7fdbaffff640 1 -- 172.21.15.7:0/1045628706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55621d29a010 msgr2=0x55621d298ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:43.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.573+0000 7fdbaffff640 1 --2- 172.21.15.7:0/1045628706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55621d29a010 0x55621d298ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:43.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.573+0000 7fdbaffff640 1 -- 172.21.15.7:0/1045628706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55621d29fb70 msgr2=0x55621d2a3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:43.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.573+0000 7fdbaffff640 1 --2- 172.21.15.7:0/1045628706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55621d29fb70 0x55621d2a3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:43.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.573+0000 7fdbaffff640 1 -- 172.21.15.7:0/1045628706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55621d2954b0 con 0x55621d29aa00 2024-06-10T01:27:43.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.573+0000 7fdbb4c8a640 1 --2- 172.21.15.7:0/1045628706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55621d29a010 0x55621d298ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:27:43.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.573+0000 7fdbaffff640 1 --2- 172.21.15.7:0/1045628706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55621d29aa00 0x55621d2ab650 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c000c00 tx=0x7fdb9c0045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad923aa20048980f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:43.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.573+0000 7fdb9a7fc640 1 -- 172.21.15.7:0/1045628706 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c022030 con 0x55621d29aa00 2024-06-10T01:27:43.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.573+0000 7fdb9a7fc640 1 -- 172.21.15.7:0/1045628706 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb9c005500 con 0x55621d29aa00 2024-06-10T01:27:43.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.573+0000 7fdb9a7fc640 1 -- 172.21.15.7:0/1045628706 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c0035f0 con 0x55621d29aa00 2024-06-10T01:27:43.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.575+0000 7fdbb9bfc740 1 -- 172.21.15.7:0/1045628706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55621d29aa00 msgr2=0x55621d2ab650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:43.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.575+0000 7fdbb9bfc740 1 --2- 172.21.15.7:0/1045628706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55621d29aa00 0x55621d2ab650 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c000c00 tx=0x7fdb9c0045d0 comp rx=0 tx=0).stop 2024-06-10T01:27:43.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.575+0000 7fdbb9bfc740 1 -- 172.21.15.7:0/1045628706 shutdown_connections 2024-06-10T01:27:43.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.575+0000 7fdbb9bfc740 1 --2- 172.21.15.7:0/1045628706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55621d29fb70 0x55621d2a3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:43.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.575+0000 7fdbb9bfc740 1 --2- 172.21.15.7:0/1045628706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55621d29aa00 0x55621d2ab650 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:43.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.575+0000 7fdbb9bfc740 1 --2- 172.21.15.7:0/1045628706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55621d29a010 0x55621d298ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:43.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.575+0000 7fdbb9bfc740 1 -- 172.21.15.7:0/1045628706 >> 172.21.15.7:0/1045628706 conn(0x55621d295680 msgr2=0x55621d298590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:43.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.575+0000 7fdbb9bfc740 1 -- 172.21.15.7:0/1045628706 shutdown_connections 2024-06-10T01:27:43.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.575+0000 7fdbb9bfc740 1 -- 172.21.15.7:0/1045628706 wait complete. 2024-06-10T01:27:43.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.576+0000 7fdbb9bfc740 1 -- start start 2024-06-10T01:27:43.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.576+0000 7fdbb9bfc740 1 Processor -- start 2024-06-10T01:27:43.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.577+0000 7fdbb9bfc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55621d29a010 0x55621d2a8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:43.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.577+0000 7fdbb9bfc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55621d29aa00 0x55621d2a87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:43.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.577+0000 7fdbb9bfc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55621d29fb70 0x55621d318d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:43.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.577+0000 7fdbb9bfc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb01aadb0 con 0x55621d29a010 2024-06-10T01:27:43.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.577+0000 7fdbb9bfc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb01ac220 con 0x55621d29fb70 2024-06-10T01:27:43.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.577+0000 7fdbb9bfc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb01ab480 con 0x55621d29aa00 2024-06-10T01:27:43.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.577+0000 7fdbb9bfc740 10 client.0 _ll_register_callbacks cb 0x7fdba4009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:27:43.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.577+0000 7fdbb4c8a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55621d29a010 0x55621d2a8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:43.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.577+0000 7fdbaffff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55621d29aa00 0x55621d2a87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:43.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.577+0000 7fdbb4c8a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55621d29a010 0x55621d2a8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33994/0 (socket says 172.21.15.7:33994) 2024-06-10T01:27:43.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.577+0000 7fdbb4c8a640 1 -- 172.21.15.7:0/4207834960 learned_addr learned my addr 172.21.15.7:0/4207834960 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:43.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.578+0000 7fdbb548b640 1 --2- 172.21.15.7:0/4207834960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55621d29fb70 0x55621d318d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.578+0000 7fdbaffff640 1 -- 172.21.15.7:0/4207834960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55621d29fb70 msgr2=0x55621d318d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.578+0000 7fdbaffff640 1 --2- 172.21.15.7:0/4207834960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55621d29fb70 0x55621d318d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.578+0000 7fdbaffff640 1 -- 172.21.15.7:0/4207834960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55621d29a010 msgr2=0x55621d2a8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.578+0000 7fdbaffff640 1 --2- 172.21.15.7:0/4207834960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55621d29a010 0x55621d2a8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.578+0000 7fdbaffff640 1 -- 172.21.15.7:0/4207834960 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55621d2ab850 con 0x55621d29aa00 2024-06-10T01:27:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.578+0000 7fdbaffff640 1 --2- 172.21.15.7:0/4207834960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55621d29aa00 0x55621d2a87d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55621d29f280 tx=0x7fdb9c0046e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e1476070a52104e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.578+0000 7fdb8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:27:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.578+0000 7fdb8ffff640 1 -- 172.21.15.7:0/4207834960 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c005220 con 0x55621d29aa00 2024-06-10T01:27:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.579+0000 7fdb8ffff640 1 -- 172.21.15.7:0/4207834960 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9129+0+0 (secure 0 0 0) 0x7fdb9c0035f0 con 0x55621d29aa00 2024-06-10T01:27:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.579+0000 7fdbb9bfc740 1 -- 172.21.15.7:0/4207834960 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb9c005500 con 0x55621d29aa00 2024-06-10T01:27:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.579+0000 7fdb8ffff640 1 -- 172.21.15.7:0/4207834960 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fdb9c036050 con 0x55621d29aa00 2024-06-10T01:27:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.579+0000 7fdbb9bfc740 10 client.29731 fetch_fsmap learned FSMap version 53 2024-06-10T01:27:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.579+0000 7fdbb9bfc740 1 -- 172.21.15.7:0/4207834960 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55621d31a2b0 con 0x55621d29aa00 2024-06-10T01:27:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.580+0000 7fdb8ffff640 1 -- 172.21.15.7:0/4207834960 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb9c004e40 con 0x55621d29aa00 2024-06-10T01:27:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.580+0000 7fdb8ffff640 1 -- 172.21.15.7:0/4207834960 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7fdb9c028860 con 0x55621d29aa00 2024-06-10T01:27:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.580+0000 7fdbb9bfc740 10 client.29731 fetch_fsmap finished waiting for FSMap version 53 2024-06-10T01:27:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.580+0000 7fdbb9bfc740 10 client.29731 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:27:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.580+0000 7fdbb9bfc740 20 client.29731 populate_metadata read hostname 'smithi007' 2024-06-10T01:27:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.580+0000 7fdbb9bfc740 1 --2- 172.21.15.7:0/4207834960 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55621d31ac80 0x55621d31d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.580+0000 7fdbb9bfc740 4 client.29731 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:27:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.580+0000 7fdbb9bfc740 1 -- 172.21.15.7:0/4207834960 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fdb9c005500 con 0x55621d31ac80 2024-06-10T01:27:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.580+0000 7fdbb4c8a640 1 --2- 172.21.15.7:0/4207834960 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55621d31ac80 0x55621d31d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.581+0000 7fdbb4c8a640 1 --2- 172.21.15.7:0/4207834960 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55621d31ac80 0x55621d31d0a0 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a09326833740dcc4 server_cookie=8800c89536e0911e in_seq=0 out_seq=0 2024-06-10T01:27:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:43.581+0000 7fdb8ffff640 10 client.29731 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:43.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:43 smithi078 ceph-mon[36683]: pgmap v1084: 129 pgs: 129 active+clean; 684 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 11 MiB/s wr, 819 op/s 2024-06-10T01:27:43.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:43 smithi175 ceph-mon[36636]: pgmap v1084: 129 pgs: 129 active+clean; 684 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 11 MiB/s wr, 819 op/s 2024-06-10T01:27:44.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:43 smithi007 ceph-mon[32398]: pgmap v1084: 129 pgs: 129 active+clean; 684 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 11 MiB/s wr, 819 op/s 2024-06-10T01:27:44.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:44.027+0000 7fdb8ffff640 1 -- 172.21.15.7:0/4207834960 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fdb9c005500 con 0x55621d31ac80 2024-06-10T01:27:44.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:44.027+0000 7fdb8ffff640 10 client.29731 handle_command_reply: tid=0 2024-06-10T01:27:44.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:44.027+0000 7fdbb9bfc740 1 client.29731 shutdown 2024-06-10T01:27:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:44.027+0000 7fdbb9bfc740 1 -- 172.21.15.7:0/4207834960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55621d29aa00 msgr2=0x55621d2a87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:44.027+0000 7fdbb9bfc740 1 --2- 172.21.15.7:0/4207834960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55621d29aa00 0x55621d2a87d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55621d29f280 tx=0x7fdb9c0046e0 comp rx=0 tx=0).stop 2024-06-10T01:27:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:44.027+0000 7fdbb9bfc740 1 -- 172.21.15.7:0/4207834960 shutdown_connections 2024-06-10T01:27:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:44.027+0000 7fdbb9bfc740 1 --2- 172.21.15.7:0/4207834960 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55621d31ac80 0x55621d31d0a0 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:44.027+0000 7fdbb9bfc740 1 --2- 172.21.15.7:0/4207834960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55621d29fb70 0x55621d318d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:44.027+0000 7fdb8ffff640 0 client.29731 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:44.027+0000 7fdbb9bfc740 1 --2- 172.21.15.7:0/4207834960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55621d29aa00 0x55621d2a87d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:44.027+0000 7fdbb9bfc740 1 --2- 172.21.15.7:0/4207834960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55621d29a010 0x55621d2a8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:44.027+0000 7fdbb9bfc740 1 -- 172.21.15.7:0/4207834960 >> 172.21.15.7:0/4207834960 conn(0x55621d295680 msgr2=0x55621d2a3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:44.028+0000 7fdbb9bfc740 1 -- 172.21.15.7:0/4207834960 shutdown_connections 2024-06-10T01:27:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:44.028+0000 7fdbb9bfc740 1 -- 172.21.15.7:0/4207834960 wait complete. 2024-06-10T01:27:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:44.028+0000 7fdbb9bfc740 20 client.29731 trim_cache size 0 max 16384 2024-06-10T01:27:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:27:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:27:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:27:44.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:44.029+0000 7fdbb6eee640 1 -- 172.21.15.7:0/1761263040 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdba00a2b80 msgr2=0x7fdba00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:44.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:44.029+0000 7fdbb6eee640 1 --2- 172.21.15.7:0/1761263040 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdba00a2b80 0x7fdba00a5040 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009430 tx=0x7fdba802faf0 comp rx=0 tx=0).stop 2024-06-10T01:27:44.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:44.029+0000 7fdbb6eee640 1 -- 172.21.15.7:0/1761263040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbb0100900 msgr2=0x7fdbb0199eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:44.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:44.029+0000 7fdbb6eee640 1 --2- 172.21.15.7:0/1761263040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbb0100900 0x7fdbb0199eb0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fdba4009960 tx=0x7fdba4009e20 comp rx=0 tx=0).stop 2024-06-10T01:27:44.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:44.029+0000 7fdbb6eee640 1 -- 172.21.15.7:0/1761263040 shutdown_connections 2024-06-10T01:27:44.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:44.029+0000 7fdbb6eee640 1 --2- 172.21.15.7:0/1761263040 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdba00a2b80 0x7fdba00a5040 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:44.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:44.029+0000 7fdbb6eee640 1 --2- 172.21.15.7:0/1761263040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbb01024b0 0x7fdbb01a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:44.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:44.029+0000 7fdbb6eee640 1 --2- 172.21.15.7:0/1761263040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbb0101af0 0x7fdbb019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:44.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:44.029+0000 7fdbb6eee640 1 --2- 172.21.15.7:0/1761263040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbb0100900 0x7fdbb0199eb0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:44.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:44.029+0000 7fdbb6eee640 1 -- 172.21.15.7:0/1761263040 >> 172.21.15.7:0/1761263040 conn(0x7fdbb00fbe70 msgr2=0x7fdbb0104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:44.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:44.029+0000 7fdbb6eee640 1 -- 172.21.15.7:0/1761263040 shutdown_connections 2024-06-10T01:27:44.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:44.029+0000 7fdbb6eee640 1 -- 172.21.15.7:0/1761263040 wait complete. 2024-06-10T01:27:44.553 INFO:tasks.workunit.client.0.smithi007.stdout: 10 202585 22.93 MB/sec execute 381 sec latency 340.273 ms 2024-06-10T01:27:44.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:44 smithi175 ceph-mon[36636]: daemon mds.f restarted 2024-06-10T01:27:44.701 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:27:44 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-f[66185]: ignoring --setuser ceph since I am not root 2024-06-10T01:27:44.701 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:27:44 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-f[66185]: ignoring --setgroup ceph since I am not root 2024-06-10T01:27:44.701 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:27:44 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-f[66185]: starting mds.f at 2024-06-10T01:27:44.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:44 smithi078 ceph-mon[36683]: daemon mds.f restarted 2024-06-10T01:27:45.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:44 smithi007 ceph-mon[32398]: daemon mds.f restarted 2024-06-10T01:27:45.553 INFO:tasks.workunit.client.0.smithi007.stdout: 10 203153 22.95 MB/sec execute 382 sec latency 98.388 ms 2024-06-10T01:27:45.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:45 smithi078 ceph-mon[36683]: pgmap v1085: 129 pgs: 129 active+clean; 682 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 11 MiB/s wr, 684 op/s 2024-06-10T01:27:45.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:45 smithi078 ceph-mon[36683]: osdmap e102: 12 total, 12 up, 12 in 2024-06-10T01:27:45.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:45 smithi078 ceph-mon[36683]: mds.? [v2:172.21.15.175:6834/138896429,v1:172.21.15.175:6835/138896429] up:boot 2024-06-10T01:27:45.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:45 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 2 up:standby-replay 7 up:standby 2024-06-10T01:27:45.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:45 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T01:27:45.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:45 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:27:45.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:45 smithi175 ceph-mon[36636]: pgmap v1085: 129 pgs: 129 active+clean; 682 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 11 MiB/s wr, 684 op/s 2024-06-10T01:27:45.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:45 smithi175 ceph-mon[36636]: osdmap e102: 12 total, 12 up, 12 in 2024-06-10T01:27:45.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:45 smithi175 ceph-mon[36636]: mds.? [v2:172.21.15.175:6834/138896429,v1:172.21.15.175:6835/138896429] up:boot 2024-06-10T01:27:45.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:45 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 2 up:standby-replay 7 up:standby 2024-06-10T01:27:45.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:45 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T01:27:45.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:45 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:27:46.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:45 smithi007 ceph-mon[32398]: pgmap v1085: 129 pgs: 129 active+clean; 682 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 11 MiB/s wr, 684 op/s 2024-06-10T01:27:46.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:45 smithi007 ceph-mon[32398]: osdmap e102: 12 total, 12 up, 12 in 2024-06-10T01:27:46.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:45 smithi007 ceph-mon[32398]: mds.? [v2:172.21.15.175:6834/138896429,v1:172.21.15.175:6835/138896429] up:boot 2024-06-10T01:27:46.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:45 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 2 up:standby-replay 7 up:standby 2024-06-10T01:27:46.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:45 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T01:27:46.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:45 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:27:46.553 INFO:tasks.workunit.client.0.smithi007.stdout: 10 203602 22.94 MB/sec execute 383 sec latency 107.398 ms 2024-06-10T01:27:47.554 INFO:tasks.workunit.client.0.smithi007.stdout: 10 204131 22.95 MB/sec execute 384 sec latency 82.543 ms 2024-06-10T01:27:47.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:47 smithi078 ceph-mon[36683]: pgmap v1087: 129 pgs: 129 active+clean; 678 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.8 MiB/s wr, 706 op/s 2024-06-10T01:27:47.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:47 smithi175 ceph-mon[36636]: pgmap v1087: 129 pgs: 129 active+clean; 678 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.8 MiB/s wr, 706 op/s 2024-06-10T01:27:48.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:47 smithi007 ceph-mon[32398]: pgmap v1087: 129 pgs: 129 active+clean; 678 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.8 MiB/s wr, 706 op/s 2024-06-10T01:27:48.553 INFO:tasks.workunit.client.0.smithi007.stdout: 10 204926 22.97 MB/sec execute 385 sec latency 77.862 ms 2024-06-10T01:27:49.554 INFO:tasks.workunit.client.0.smithi007.stdout: 10 205652 23.04 MB/sec execute 386 sec latency 72.754 ms 2024-06-10T01:27:49.657 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:27:49.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:49 smithi078 ceph-mon[36683]: pgmap v1088: 129 pgs: 129 active+clean; 679 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 6.2 MiB/s wr, 648 op/s 2024-06-10T01:27:49.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:49 smithi175 ceph-mon[36636]: pgmap v1088: 129 pgs: 129 active+clean; 679 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 6.2 MiB/s wr, 648 op/s 2024-06-10T01:27:50.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:49 smithi007 ceph-mon[32398]: pgmap v1088: 129 pgs: 129 active+clean; 679 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 6.2 MiB/s wr, 648 op/s 2024-06-10T01:27:50.027 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:27:50.554 INFO:tasks.workunit.client.0.smithi007.stdout: 10 206253 23.09 MB/sec execute 387 sec latency 87.029 ms 2024-06-10T01:27:51.554 INFO:tasks.workunit.client.0.smithi007.stdout: 10 206669 23.08 MB/sec execute 388 sec latency 69.137 ms 2024-06-10T01:27:51.800 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:51 smithi007 ceph-mon[32398]: pgmap v1089: 129 pgs: 129 active+clean; 679 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 5.2 MiB/s wr, 690 op/s 2024-06-10T01:27:51.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:51 smithi078 ceph-mon[36683]: pgmap v1089: 129 pgs: 129 active+clean; 679 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 5.2 MiB/s wr, 690 op/s 2024-06-10T01:27:51.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:51 smithi175 ceph-mon[36636]: pgmap v1089: 129 pgs: 129 active+clean; 679 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 5.2 MiB/s wr, 690 op/s 2024-06-10T01:27:52.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.474+0000 7fba0f7fe700 1 -- 172.21.15.7:0/3039424079 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0800d220 con 0x7fba10101d50 2024-06-10T01:27:52.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.474+0000 7fba163cf700 1 -- 172.21.15.7:0/3039424079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba10101d50 msgr2=0x7fba1010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:52.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.474+0000 7fba163cf700 1 --2- 172.21.15.7:0/3039424079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba10101d50 0x7fba1010e240 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fba08009990 tx=0x7fba08009ca0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.474+0000 7fba163cf700 1 -- 172.21.15.7:0/3039424079 shutdown_connections 2024-06-10T01:27:52.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.474+0000 7fba163cf700 1 --2- 172.21.15.7:0/3039424079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba10101d50 0x7fba1010e240 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.474+0000 7fba163cf700 1 --2- 172.21.15.7:0/3039424079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba10107400 0x7fba101077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.474+0000 7fba163cf700 1 --2- 172.21.15.7:0/3039424079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba10101390 0x7fba10101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.474+0000 7fba163cf700 1 -- 172.21.15.7:0/3039424079 >> 172.21.15.7:0/3039424079 conn(0x7fba10078340 msgr2=0x7fba10078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.474+0000 7fba163cf700 1 -- 172.21.15.7:0/3039424079 shutdown_connections 2024-06-10T01:27:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.476+0000 7fba163cf700 1 -- 172.21.15.7:0/3039424079 wait complete. 2024-06-10T01:27:52.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.477+0000 7fba163cf700 1 Processor -- start 2024-06-10T01:27:52.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.477+0000 7fba163cf700 1 -- start start 2024-06-10T01:27:52.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.477+0000 7fba163cf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba10101390 0x7fba10073930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:52.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.477+0000 7fba163cf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba10101d50 0x7fba10073e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:52.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.477+0000 7fba163cf700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba10107400 0x7fba1006d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:52.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.477+0000 7fba163cf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fba1006c4a0 con 0x7fba10101390 2024-06-10T01:27:52.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.477+0000 7fba163cf700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fba1006c320 con 0x7fba10101d50 2024-06-10T01:27:52.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.477+0000 7fba163cf700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fba1006c620 con 0x7fba10107400 2024-06-10T01:27:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.478+0000 7fba1496c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba10107400 0x7fba1006d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.478+0000 7fba077fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba10101d50 0x7fba10073e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.478+0000 7fba077fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba10101d50 0x7fba10073e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:35314/0 (socket says 172.21.15.7:35314) 2024-06-10T01:27:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.478+0000 7fba077fe700 1 -- 172.21.15.7:0/2075554108 learned_addr learned my addr 172.21.15.7:0/2075554108 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.478+0000 7fba1496c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba10107400 0x7fba1006d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:58542/0 (socket says 172.21.15.7:58542) 2024-06-10T01:27:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.479+0000 7fba1496c700 1 -- 172.21.15.7:0/2075554108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba10101d50 msgr2=0x7fba10073e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.479+0000 7fba1496c700 1 --2- 172.21.15.7:0/2075554108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba10101d50 0x7fba10073e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.479+0000 7fba1496c700 1 -- 172.21.15.7:0/2075554108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba10101390 msgr2=0x7fba10073930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.479+0000 7fba1496c700 1 --2- 172.21.15.7:0/2075554108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba10101390 0x7fba10073930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.479+0000 7fba1496c700 1 -- 172.21.15.7:0/2075554108 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba1006e180 con 0x7fba10107400 2024-06-10T01:27:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.480+0000 7fba1496c700 1 --2- 172.21.15.7:0/2075554108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba10107400 0x7fba1006d9b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fba08006010 tx=0x7fba0800d990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.480+0000 7fba0dffb700 1 -- 172.21.15.7:0/2075554108 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba08024070 con 0x7fba10107400 2024-06-10T01:27:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.480+0000 7fba0dffb700 1 -- 172.21.15.7:0/2075554108 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba080046c0 con 0x7fba10107400 2024-06-10T01:27:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.480+0000 7fba163cf700 1 -- 172.21.15.7:0/2075554108 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba1006e410 con 0x7fba10107400 2024-06-10T01:27:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.481+0000 7fba163cf700 1 -- 172.21.15.7:0/2075554108 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba101b4610 con 0x7fba10107400 2024-06-10T01:27:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.481+0000 7fba0dffb700 1 -- 172.21.15.7:0/2075554108 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba080038e0 con 0x7fba10107400 2024-06-10T01:27:52.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.488+0000 7fba0dffb700 1 -- 172.21.15.7:0/2075554108 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fba08027070 con 0x7fba10107400 2024-06-10T01:27:52.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.489+0000 7fba20c52b80 1 Processor -- start 2024-06-10T01:27:52.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.489+0000 7fba20c52b80 1 -- start start 2024-06-10T01:27:52.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.489+0000 7fba20c52b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556419a8a640 0x556419a8aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:52.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.489+0000 7fba20c52b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556419a8af60 0x556419a849e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:52.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.489+0000 7fba20c52b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556419a84f20 0x556419a85340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:52.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.489+0000 7fba20c52b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5564199d6810 con 0x556419a8af60 2024-06-10T01:27:52.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.489+0000 7fba077fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556419a8a640 0x556419a8aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:52.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.489+0000 7fba077fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556419a8a640 0x556419a8aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:58544/0 (socket says 172.21.15.7:58544) 2024-06-10T01:27:52.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.489+0000 7fba077fe700 1 -- 172.21.15.7:0/240782312 learned_addr learned my addr 172.21.15.7:0/240782312 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:52.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.490+0000 7fba0ffff700 1 --2- 172.21.15.7:0/240782312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556419a8af60 0x556419a849e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:52.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.490+0000 7fba0dffb700 1 --2- 172.21.15.7:0/2075554108 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb9fc0a2c50 0x7fb9fc0a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:52.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.490+0000 7fba20c52b80 1 -- 172.21.15.7:0/240782312 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55641997e720 con 0x556419a84f20 2024-06-10T01:27:52.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.490+0000 7fba20c52b80 1 -- 172.21.15.7:0/240782312 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5564199d27d0 con 0x556419a8a640 2024-06-10T01:27:52.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.490+0000 7fba1496c700 1 --2- 172.21.15.7:0/2075554108 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb9fc0a2c50 0x7fb9fc0a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:52.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.490+0000 7fba0dffb700 1 -- 172.21.15.7:0/2075554108 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fba080cf460 con 0x7fba10107400 2024-06-10T01:27:52.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.490+0000 7fba077fe700 1 --2- 172.21.15.7:0/240782312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556419a84f20 0x556419a85340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:52.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.491+0000 7fba077fe700 1 -- 172.21.15.7:0/240782312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556419a84f20 msgr2=0x556419a85340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:52.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.491+0000 7fba077fe700 1 --2- 172.21.15.7:0/240782312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556419a84f20 0x556419a85340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.491+0000 7fba077fe700 1 -- 172.21.15.7:0/240782312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556419a8af60 msgr2=0x556419a849e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:52.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.491+0000 7fba077fe700 1 --2- 172.21.15.7:0/240782312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556419a8af60 0x556419a849e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.491+0000 7fba077fe700 1 -- 172.21.15.7:0/240782312 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556419974f50 con 0x556419a8a640 2024-06-10T01:27:52.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.492+0000 7fba077fe700 1 --2- 172.21.15.7:0/240782312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556419a8a640 0x556419a8aa20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8009450 tx=0x7fb9f80095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4a9c7c47a500265 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:52.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.492+0000 7fba067fc700 1 -- 172.21.15.7:0/240782312 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f80185f0 con 0x556419a8a640 2024-06-10T01:27:52.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.493+0000 7fba067fc700 1 -- 172.21.15.7:0/240782312 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9f801d070 con 0x556419a8a640 2024-06-10T01:27:52.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.493+0000 7fba067fc700 1 -- 172.21.15.7:0/240782312 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f80185f0 con 0x556419a8a640 2024-06-10T01:27:52.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.494+0000 7fba1496c700 1 --2- 172.21.15.7:0/2075554108 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb9fc0a2c50 0x7fb9fc0a5110 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fba08097dd0 tx=0x7fba08090e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:52.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.494+0000 7fba20c52b80 1 -- 172.21.15.7:0/240782312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556419a8a640 msgr2=0x556419a8aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:52.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.494+0000 7fba20c52b80 1 --2- 172.21.15.7:0/240782312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556419a8a640 0x556419a8aa20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8009450 tx=0x7fb9f80095e0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.494+0000 7fba20c52b80 1 -- 172.21.15.7:0/240782312 shutdown_connections 2024-06-10T01:27:52.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.494+0000 7fba20c52b80 1 --2- 172.21.15.7:0/240782312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556419a84f20 0x556419a85340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.494+0000 7fba20c52b80 1 --2- 172.21.15.7:0/240782312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556419a8a640 0x556419a8aa20 secure :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8009450 tx=0x7fb9f80095e0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.494+0000 7fba20c52b80 1 --2- 172.21.15.7:0/240782312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556419a8af60 0x556419a849e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.494+0000 7fba20c52b80 1 -- 172.21.15.7:0/240782312 >> 172.21.15.7:0/240782312 conn(0x5564199afcc0 msgr2=0x556419a7f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:52.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.495+0000 7fba20c52b80 1 -- 172.21.15.7:0/240782312 shutdown_connections 2024-06-10T01:27:52.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.495+0000 7fba20c52b80 1 -- 172.21.15.7:0/240782312 wait complete. 2024-06-10T01:27:52.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.495+0000 7fba20c52b80 1 -- start start 2024-06-10T01:27:52.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.496+0000 7fba20c52b80 1 Processor -- start 2024-06-10T01:27:52.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.496+0000 7fba20c52b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556419a84f20 0x556419a90f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:52.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.496+0000 7fba20c52b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556419a8af60 0x556419a8e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:52.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.496+0000 7fba20c52b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556419a8e7a0 0x556419a8ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:52.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.496+0000 7fba077fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556419a84f20 0x556419a90f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:52.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.496+0000 7fba077fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556419a84f20 0x556419a90f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41438/0 (socket says 172.21.15.7:41438) 2024-06-10T01:27:52.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.496+0000 7fba077fe700 1 -- 172.21.15.7:0/549714831 learned_addr learned my addr 172.21.15.7:0/549714831 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:52.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.496+0000 7fba20c52b80 1 -- 172.21.15.7:0/549714831 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556419a91970 con 0x556419a84f20 2024-06-10T01:27:52.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.496+0000 7fba20c52b80 1 -- 172.21.15.7:0/549714831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556419976bf0 con 0x556419a8e7a0 2024-06-10T01:27:52.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.496+0000 7fba20c52b80 1 -- 172.21.15.7:0/549714831 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556419a91af0 con 0x556419a8af60 2024-06-10T01:27:52.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.497+0000 7fba077fe700 1 --2- 172.21.15.7:0/549714831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556419a8e7a0 0x556419a8ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:52.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.497+0000 7fba20c52b80 10 client.0 _ll_register_callbacks cb 0x556419970920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:27:52.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.497+0000 7fba077fe700 1 -- 172.21.15.7:0/549714831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556419a8af60 msgr2=0x556419a8e210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T01:27:52.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.497+0000 7fba077fe700 1 --2- 172.21.15.7:0/549714831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556419a8af60 0x556419a8e210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.497+0000 7fba077fe700 1 -- 172.21.15.7:0/549714831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556419a8e7a0 msgr2=0x556419a8ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:52.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.497+0000 7fba077fe700 1 --2- 172.21.15.7:0/549714831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556419a8e7a0 0x556419a8ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.497+0000 7fba077fe700 1 -- 172.21.15.7:0/549714831 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564199258d0 con 0x556419a84f20 2024-06-10T01:27:52.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.497+0000 7fba20c52b80 1 -- 172.21.15.7:0/549714831 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9f801d070 con 0x556419a84f20 2024-06-10T01:27:52.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.497+0000 7fba077fe700 1 --2- 172.21.15.7:0/549714831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556419a84f20 0x556419a90f10 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8009420 tx=0x7fb9f8009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7985c6bc955face2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:52.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.497+0000 7fb9ef7fe700 10 client.20625 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:27:52.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.498+0000 7fb9ef7fe700 1 -- 172.21.15.7:0/549714831 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8036050 con 0x556419a84f20 2024-06-10T01:27:52.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.498+0000 7fb9ef7fe700 1 -- 172.21.15.7:0/549714831 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fb9f80183f0 con 0x556419a84f20 2024-06-10T01:27:52.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.499+0000 7fb9ef7fe700 1 -- 172.21.15.7:0/549714831 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fb9f802b030 con 0x556419a84f20 2024-06-10T01:27:52.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.499+0000 7fba20c52b80 10 client.20625 fetch_fsmap learned FSMap version 55 2024-06-10T01:27:52.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.499+0000 7fba20c52b80 1 -- 172.21.15.7:0/549714831 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=55}) -- 0x556419a9c870 con 0x556419a84f20 2024-06-10T01:27:52.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.502+0000 7fb9ef7fe700 1 -- 172.21.15.7:0/549714831 <== mon.0 v2:172.21.15.7:3300/0 4 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fb9f80179d0 con 0x556419a84f20 2024-06-10T01:27:52.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.502+0000 7fba20c52b80 10 client.20625 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:27:52.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.502+0000 7fba20c52b80 10 client.20625 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:27:52.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.502+0000 7fba20c52b80 20 client.20625 populate_metadata read hostname 'smithi007' 2024-06-10T01:27:52.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.502+0000 7fba20c52b80 1 --2- 172.21.15.7:0/549714831 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556419a9eff0 0x556419aa1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:52.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.502+0000 7fba20c52b80 4 client.20625 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:27:52.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.502+0000 7fba20c52b80 1 -- 172.21.15.7:0/549714831 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564199e3d10 con 0x556419a9eff0 2024-06-10T01:27:52.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.508+0000 7fba0ffff700 1 --2- 172.21.15.7:0/549714831 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556419a9eff0 0x556419aa1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:52.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.511+0000 7fb9ef7fe700 10 client.20625 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:52.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.511+0000 7fba0ffff700 1 --2- 172.21.15.7:0/549714831 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556419a9eff0 0x556419aa1410 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad80a237fa344b59 server_cookie=f44e48da2dec69bf in_seq=0 out_seq=0 2024-06-10T01:27:52.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.517+0000 7fb9ef7fe700 1 -- 172.21.15.7:0/549714831 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5564199e3d10 con 0x556419a9eff0 2024-06-10T01:27:52.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.517+0000 7fb9ef7fe700 10 client.20625 handle_command_reply: tid=0 2024-06-10T01:27:52.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.517+0000 7fba20c52b80 1 client.20625 shutdown 2024-06-10T01:27:52.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.517+0000 7fba20c52b80 1 -- 172.21.15.7:0/549714831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556419a84f20 msgr2=0x556419a90f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:52.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.517+0000 7fba20c52b80 1 --2- 172.21.15.7:0/549714831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556419a84f20 0x556419a90f10 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8009420 tx=0x7fb9f8009eb0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.517+0000 7fba20c52b80 1 -- 172.21.15.7:0/549714831 shutdown_connections 2024-06-10T01:27:52.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.517+0000 7fba20c52b80 1 --2- 172.21.15.7:0/549714831 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556419a9eff0 0x556419aa1410 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.517+0000 7fba20c52b80 1 --2- 172.21.15.7:0/549714831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556419a8e7a0 0x556419a8ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.517+0000 7fba20c52b80 1 --2- 172.21.15.7:0/549714831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556419a84f20 0x556419a90f10 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.517+0000 7fba20c52b80 1 --2- 172.21.15.7:0/549714831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556419a8af60 0x556419a8e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.517+0000 7fba20c52b80 1 -- 172.21.15.7:0/549714831 >> 172.21.15.7:0/549714831 conn(0x5564199afcc0 msgr2=0x556419a7f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:52.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.517+0000 7fb9ef7fe700 0 client.20625 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:52.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.517+0000 7fba20c52b80 1 -- 172.21.15.7:0/549714831 shutdown_connections 2024-06-10T01:27:52.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.517+0000 7fba20c52b80 1 -- 172.21.15.7:0/549714831 wait complete. 2024-06-10T01:27:52.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.518+0000 7fba20c52b80 20 client.20625 trim_cache size 0 max 16384 2024-06-10T01:27:52.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.546+0000 7fba20c52b80 1 Processor -- start 2024-06-10T01:27:52.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.546+0000 7fba20c52b80 1 -- start start 2024-06-10T01:27:52.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.546+0000 7fba20c52b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556419a84f20 0x556419a849e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:52.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.546+0000 7fba20c52b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556419a8af60 0x556419a888a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:52.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.546+0000 7fba20c52b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556419a8e7a0 0x556419a8dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:52.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.546+0000 7fba20c52b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fba101bf610 con 0x556419a8e7a0 2024-06-10T01:27:52.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.546+0000 7fba20c52b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fba101bcea0 con 0x556419a84f20 2024-06-10T01:27:52.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.546+0000 7fba20c52b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fba101bacf0 con 0x556419a8af60 2024-06-10T01:27:52.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.547+0000 7fba0ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556419a8af60 0x556419a888a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:52.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.547+0000 7fba077fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556419a8e7a0 0x556419a8dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:52.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.547+0000 7fba0ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556419a8af60 0x556419a888a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:58560/0 (socket says 172.21.15.7:58560) 2024-06-10T01:27:52.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.547+0000 7fba0ffff700 1 -- 172.21.15.7:0/2657592454 learned_addr learned my addr 172.21.15.7:0/2657592454 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:52.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.547+0000 7fba077fe700 1 --2- 172.21.15.7:0/2657592454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556419a84f20 0x556419a849e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:52.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.547+0000 7fba0ffff700 1 -- 172.21.15.7:0/2657592454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556419a84f20 msgr2=0x556419a849e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:52.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.547+0000 7fba0ffff700 1 --2- 172.21.15.7:0/2657592454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556419a84f20 0x556419a849e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.547+0000 7fba0ffff700 1 -- 172.21.15.7:0/2657592454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556419a8e7a0 msgr2=0x556419a8dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:52.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.547+0000 7fba0ffff700 1 --2- 172.21.15.7:0/2657592454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556419a8e7a0 0x556419a8dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.547+0000 7fba0ffff700 1 -- 172.21.15.7:0/2657592454 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556419a914f0 con 0x556419a8af60 2024-06-10T01:27:52.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.547+0000 7fba077fe700 1 --2- 172.21.15.7:0/2657592454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556419a84f20 0x556419a849e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:27:52.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.548+0000 7fba0ffff700 1 --2- 172.21.15.7:0/2657592454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556419a8af60 0x556419a888a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fba00009410 tx=0x7fba00009040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da0b44ec4b8a6203 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:52.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.548+0000 7fba05ffb700 1 -- 172.21.15.7:0/2657592454 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba00007db0 con 0x556419a8af60 2024-06-10T01:27:52.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.548+0000 7fba05ffb700 1 -- 172.21.15.7:0/2657592454 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba00004030 con 0x556419a8af60 2024-06-10T01:27:52.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.548+0000 7fba05ffb700 1 -- 172.21.15.7:0/2657592454 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0001a6a0 con 0x556419a8af60 2024-06-10T01:27:52.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.549+0000 7fba20c52b80 1 -- 172.21.15.7:0/2657592454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556419a8af60 msgr2=0x556419a888a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:52.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.549+0000 7fba20c52b80 1 --2- 172.21.15.7:0/2657592454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556419a8af60 0x556419a888a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fba00009410 tx=0x7fba00009040 comp rx=0 tx=0).stop 2024-06-10T01:27:52.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.549+0000 7fba20c52b80 1 -- 172.21.15.7:0/2657592454 shutdown_connections 2024-06-10T01:27:52.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.549+0000 7fba20c52b80 1 --2- 172.21.15.7:0/2657592454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556419a8e7a0 0x556419a8dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.549+0000 7fba20c52b80 1 --2- 172.21.15.7:0/2657592454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556419a84f20 0x556419a849e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.549+0000 7fba20c52b80 1 --2- 172.21.15.7:0/2657592454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556419a8af60 0x556419a888a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.549+0000 7fba20c52b80 1 -- 172.21.15.7:0/2657592454 >> 172.21.15.7:0/2657592454 conn(0x5564199afcc0 msgr2=0x556419a88190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:52.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.549+0000 7fba20c52b80 1 -- 172.21.15.7:0/2657592454 shutdown_connections 2024-06-10T01:27:52.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.549+0000 7fba20c52b80 1 -- 172.21.15.7:0/2657592454 wait complete. 2024-06-10T01:27:52.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.550+0000 7fba20c52b80 1 -- start start 2024-06-10T01:27:52.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.550+0000 7fba20c52b80 1 Processor -- start 2024-06-10T01:27:52.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.550+0000 7fba20c52b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556419a84f20 0x556419a96de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:52.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.550+0000 7fba20c52b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556419a8af60 0x556419a97320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:52.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.550+0000 7fba20c52b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556419a8e7a0 0x556419aa5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:52.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.550+0000 7fba20c52b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fba101bb360 con 0x556419a8af60 2024-06-10T01:27:52.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.550+0000 7fba20c52b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fba101bc7d0 con 0x556419a8e7a0 2024-06-10T01:27:52.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.550+0000 7fba20c52b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fba101bba30 con 0x556419a84f20 2024-06-10T01:27:52.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.550+0000 7fba20c52b80 10 client.0 _ll_register_callbacks cb 0x556419970920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:27:52.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.551+0000 7fba0ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556419a8af60 0x556419a97320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:52.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.551+0000 7fba0ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556419a8af60 0x556419a97320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41460/0 (socket says 172.21.15.7:41460) 2024-06-10T01:27:52.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.551+0000 7fba0ffff700 1 -- 172.21.15.7:0/1909723946 learned_addr learned my addr 172.21.15.7:0/1909723946 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:52.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.551+0000 7fba077fe700 1 --2- 172.21.15.7:0/1909723946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556419a84f20 0x556419a96de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:52.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.551+0000 7fba077fe700 1 --2- 172.21.15.7:0/1909723946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556419a8e7a0 0x556419aa5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:52.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.551+0000 7fba0ffff700 1 -- 172.21.15.7:0/1909723946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556419a84f20 msgr2=0x556419a96de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:52.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.551+0000 7fba0ffff700 1 --2- 172.21.15.7:0/1909723946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556419a84f20 0x556419a96de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.551+0000 7fba0ffff700 1 -- 172.21.15.7:0/1909723946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556419a8e7a0 msgr2=0x556419aa5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:52.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.551+0000 7fba0ffff700 1 --2- 172.21.15.7:0/1909723946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556419a8e7a0 0x556419aa5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.551+0000 7fba0ffff700 1 -- 172.21.15.7:0/1909723946 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556419a97a20 con 0x556419a8af60 2024-06-10T01:27:52.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.551+0000 7fba0ffff700 1 --2- 172.21.15.7:0/1909723946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556419a8af60 0x556419a97320 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fba00009690 tx=0x7fba00006070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68eaa7f4e7c89faf server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:52.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.551+0000 7fb9ef7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:27:52.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.552+0000 7fb9ef7fe700 1 -- 172.21.15.7:0/1909723946 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba00003e60 con 0x556419a8af60 2024-06-10T01:27:52.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.552+0000 7fb9ef7fe700 1 -- 172.21.15.7:0/1909723946 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fba00004bb0 con 0x556419a8af60 2024-06-10T01:27:52.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.552+0000 7fba20c52b80 1 -- 172.21.15.7:0/1909723946 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba00004030 con 0x556419a8af60 2024-06-10T01:27:52.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.552+0000 7fb9ef7fe700 1 -- 172.21.15.7:0/1909723946 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fba00034050 con 0x556419a8af60 2024-06-10T01:27:52.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.552+0000 7fba20c52b80 10 client.20631 fetch_fsmap learned FSMap version 55 2024-06-10T01:27:52.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.552+0000 7fba20c52b80 1 -- 172.21.15.7:0/1909723946 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x556419b092e0 con 0x556419a8af60 2024-06-10T01:27:52.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.552+0000 7fb9ef7fe700 1 -- 172.21.15.7:0/1909723946 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba00004030 con 0x556419a8af60 2024-06-10T01:27:52.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.553+0000 7fb9ef7fe700 1 -- 172.21.15.7:0/1909723946 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fba0002b070 con 0x556419a8af60 2024-06-10T01:27:52.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.553+0000 7fba20c52b80 10 client.20631 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:27:52.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.553+0000 7fba20c52b80 10 client.20631 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:27:52.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.553+0000 7fba20c52b80 20 client.20631 populate_metadata read hostname 'smithi007' 2024-06-10T01:27:52.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.553+0000 7fba20c52b80 1 --2- 172.21.15.7:0/1909723946 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556419b09830 0x556419b0bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:52.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.553+0000 7fba20c52b80 4 client.20631 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:27:52.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.553+0000 7fba20c52b80 1 -- 172.21.15.7:0/1909723946 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5564199de290 con 0x556419b09830 2024-06-10T01:27:52.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.553+0000 7fba077fe700 1 --2- 172.21.15.7:0/1909723946 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556419b09830 0x556419b0bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:52.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.554+0000 7fb9ef7fe700 10 client.20631 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:52.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.554+0000 7fba077fe700 1 --2- 172.21.15.7:0/1909723946 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556419b09830 0x556419b0bc50 crc :-1 s=READY pgs=492 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c08517dfc1cbf7b server_cookie=58c8c0ebe4c1e9e3 in_seq=0 out_seq=0 2024-06-10T01:27:52.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.556+0000 7fb9ef7fe700 1 -- 172.21.15.7:0/1909723946 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5564199de290 con 0x556419b09830 2024-06-10T01:27:52.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.556+0000 7fb9ef7fe700 10 client.20631 handle_command_reply: tid=0 2024-06-10T01:27:52.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.556+0000 7fba20c52b80 1 client.20631 shutdown 2024-06-10T01:27:52.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.556+0000 7fba20c52b80 1 -- 172.21.15.7:0/1909723946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556419a8af60 msgr2=0x556419a97320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:52.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.556+0000 7fba20c52b80 1 --2- 172.21.15.7:0/1909723946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556419a8af60 0x556419a97320 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fba00009690 tx=0x7fba00006070 comp rx=0 tx=0).stop 2024-06-10T01:27:52.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.556+0000 7fba20c52b80 1 -- 172.21.15.7:0/1909723946 shutdown_connections 2024-06-10T01:27:52.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.556+0000 7fba20c52b80 1 --2- 172.21.15.7:0/1909723946 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556419b09830 0x556419b0bc50 crc :-1 s=READY pgs=492 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.557+0000 7fba20c52b80 1 --2- 172.21.15.7:0/1909723946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556419a8e7a0 0x556419aa5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.557+0000 7fba20c52b80 1 --2- 172.21.15.7:0/1909723946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556419a84f20 0x556419a96de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.557+0000 7fba20c52b80 1 --2- 172.21.15.7:0/1909723946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556419a8af60 0x556419a97320 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.557+0000 7fba20c52b80 1 -- 172.21.15.7:0/1909723946 >> 172.21.15.7:0/1909723946 conn(0x5564199afcc0 msgr2=0x556419a8d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:52.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.557+0000 7fb9ef7fe700 0 client.20631 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:52.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.557+0000 7fba20c52b80 1 -- 172.21.15.7:0/1909723946 shutdown_connections 2024-06-10T01:27:52.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.557+0000 7fba20c52b80 1 -- 172.21.15.7:0/1909723946 wait complete. 2024-06-10T01:27:52.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.557+0000 7fba20c52b80 20 client.20631 trim_cache size 0 max 16384 2024-06-10T01:27:52.552 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:27:52.552 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-10T01:27:52.552 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-10T01:27:52.552 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:27:52.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.558+0000 7fba163cf700 1 -- 172.21.15.7:0/2075554108 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb9fc0a2c50 msgr2=0x7fb9fc0a5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:52.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.558+0000 7fba163cf700 1 --2- 172.21.15.7:0/2075554108 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb9fc0a2c50 0x7fb9fc0a5110 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fba08097dd0 tx=0x7fba08090e50 comp rx=0 tx=0).stop 2024-06-10T01:27:52.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.558+0000 7fba163cf700 1 -- 172.21.15.7:0/2075554108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba10107400 msgr2=0x7fba1006d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:52.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.558+0000 7fba163cf700 1 --2- 172.21.15.7:0/2075554108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba10107400 0x7fba1006d9b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fba08006010 tx=0x7fba0800d990 comp rx=0 tx=0).stop 2024-06-10T01:27:52.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.558+0000 7fba163cf700 1 -- 172.21.15.7:0/2075554108 shutdown_connections 2024-06-10T01:27:52.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.558+0000 7fba163cf700 1 --2- 172.21.15.7:0/2075554108 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fb9fc0a2c50 0x7fb9fc0a5110 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.558+0000 7fba163cf700 1 --2- 172.21.15.7:0/2075554108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba10107400 0x7fba1006d9b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.558+0000 7fba163cf700 1 --2- 172.21.15.7:0/2075554108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba10101390 0x7fba10073930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.558+0000 7fba163cf700 1 --2- 172.21.15.7:0/2075554108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba10101d50 0x7fba10073e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:52.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.558+0000 7fba163cf700 1 -- 172.21.15.7:0/2075554108 >> 172.21.15.7:0/2075554108 conn(0x7fba10078340 msgr2=0x7fba100ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:52.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.558+0000 7fba163cf700 1 -- 172.21.15.7:0/2075554108 shutdown_connections 2024-06-10T01:27:52.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:52.558+0000 7fba163cf700 1 -- 172.21.15.7:0/2075554108 wait complete. 2024-06-10T01:27:52.554 INFO:tasks.workunit.client.0.smithi007.stdout: 10 207140 23.07 MB/sec execute 389 sec latency 121.612 ms 2024-06-10T01:27:53.379 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:27:53.379 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:27:53.554 INFO:tasks.workunit.client.0.smithi007.stdout: 10 207658 23.08 MB/sec execute 390 sec latency 123.710 ms 2024-06-10T01:27:53.704 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:27:53.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:53 smithi007 ceph-mon[32398]: pgmap v1090: 129 pgs: 129 active+clean; 673 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.2 MiB/s wr, 498 op/s 2024-06-10T01:27:53.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:53 smithi078 ceph-mon[36683]: pgmap v1090: 129 pgs: 129 active+clean; 673 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.2 MiB/s wr, 498 op/s 2024-06-10T01:27:53.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:53 smithi175 ceph-mon[36636]: pgmap v1090: 129 pgs: 129 active+clean; 673 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.2 MiB/s wr, 498 op/s 2024-06-10T01:27:54.554 INFO:tasks.workunit.client.0.smithi007.stdout: 10 208482 23.14 MB/sec execute 391 sec latency 60.556 ms 2024-06-10T01:27:55.554 INFO:tasks.workunit.client.0.smithi007.stdout: 10 209053 23.15 MB/sec execute 392 sec latency 89.538 ms 2024-06-10T01:27:55.730 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:55 smithi007 ceph-mon[32398]: pgmap v1091: 129 pgs: 129 active+clean; 677 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.3 MiB/s wr, 470 op/s 2024-06-10T01:27:55.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.737+0000 7f69abdcb700 1 -- 172.21.15.7:0/1594835989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69a4101440 msgr2=0x7f69a410f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:55.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.737+0000 7f69abdcb700 1 --2- 172.21.15.7:0/1594835989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69a4101440 0x7f69a410f330 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f699c00b0a0 tx=0x7f699c00b3b0 comp rx=0 tx=0).stop 2024-06-10T01:27:55.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.737+0000 7f69abdcb700 1 -- 172.21.15.7:0/1594835989 shutdown_connections 2024-06-10T01:27:55.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.737+0000 7f69abdcb700 1 --2- 172.21.15.7:0/1594835989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69a4101440 0x7f69a410f330 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:55.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.737+0000 7f69abdcb700 1 --2- 172.21.15.7:0/1594835989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69a4108510 0x7f69a41088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:55.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.737+0000 7f69abdcb700 1 --2- 172.21.15.7:0/1594835989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69a4100a80 0x7f69a4100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:55.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.737+0000 7f69abdcb700 1 -- 172.21.15.7:0/1594835989 >> 172.21.15.7:0/1594835989 conn(0x7f69a4077a00 msgr2=0x7f69a40fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:55.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.738+0000 7f69abdcb700 1 -- 172.21.15.7:0/1594835989 shutdown_connections 2024-06-10T01:27:55.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.738+0000 7f69abdcb700 1 -- 172.21.15.7:0/1594835989 wait complete. 2024-06-10T01:27:55.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.739+0000 7f69abdcb700 1 Processor -- start 2024-06-10T01:27:55.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.739+0000 7f69abdcb700 1 -- start start 2024-06-10T01:27:55.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.739+0000 7f69abdcb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69a4100a80 0x7f69a41a3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:55.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.739+0000 7f69abdcb700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69a4101440 0x7f69a41a41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:55.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.739+0000 7f69abdcb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69a4108510 0x7f69a410bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:55.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.740+0000 7f69abdcb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f69a4103390 con 0x7f69a4108510 2024-06-10T01:27:55.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.740+0000 7f69abdcb700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f69a4103210 con 0x7f69a4100a80 2024-06-10T01:27:55.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.740+0000 7f69abdcb700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f69a4103510 con 0x7f69a4101440 2024-06-10T01:27:55.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.740+0000 7f69aa368700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69a4108510 0x7f69a410bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:55.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.740+0000 7f69aa368700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69a4108510 0x7f69a410bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41470/0 (socket says 172.21.15.7:41470) 2024-06-10T01:27:55.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.740+0000 7f69aa368700 1 -- 172.21.15.7:0/267565927 learned_addr learned my addr 172.21.15.7:0/267565927 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:55.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.740+0000 7f69aa368700 1 -- 172.21.15.7:0/267565927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69a4101440 msgr2=0x7f69a41a41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:55.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.740+0000 7f69aa368700 1 --2- 172.21.15.7:0/267565927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69a4101440 0x7f69a41a41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:55.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.740+0000 7f69aa368700 1 -- 172.21.15.7:0/267565927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69a4100a80 msgr2=0x7f69a41a3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:55.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.740+0000 7f69aa368700 1 --2- 172.21.15.7:0/267565927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69a4100a80 0x7f69a41a3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:55.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.740+0000 7f69aa368700 1 -- 172.21.15.7:0/267565927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a410c6b0 con 0x7f69a4108510 2024-06-10T01:27:55.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.741+0000 7f69aa368700 1 --2- 172.21.15.7:0/267565927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69a4108510 0x7f69a410bde0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f699c0062a0 tx=0x7f699c00cfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:55.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.741+0000 7f699affd700 1 -- 172.21.15.7:0/267565927 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699c021070 con 0x7f69a4108510 2024-06-10T01:27:55.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.741+0000 7f699affd700 1 -- 172.21.15.7:0/267565927 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f699c007df0 con 0x7f69a4108510 2024-06-10T01:27:55.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.741+0000 7f699affd700 1 -- 172.21.15.7:0/267565927 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699c01d6b0 con 0x7f69a4108510 2024-06-10T01:27:55.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.741+0000 7f69abdcb700 1 -- 172.21.15.7:0/267565927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a410c940 con 0x7f69a4108510 2024-06-10T01:27:55.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.741+0000 7f69abdcb700 1 -- 172.21.15.7:0/267565927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a41abb60 con 0x7f69a4108510 2024-06-10T01:27:55.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.743+0000 7f69abdcb700 1 -- 172.21.15.7:0/267565927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69a4103da0 con 0x7f69a4108510 2024-06-10T01:27:55.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.745+0000 7f699affd700 1 -- 172.21.15.7:0/267565927 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f699c004220 con 0x7f69a4108510 2024-06-10T01:27:55.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.746+0000 7f699affd700 1 --2- 172.21.15.7:0/267565927 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f69900a2d20 0x7f69900a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:55.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.746+0000 7f699affd700 1 -- 172.21.15.7:0/267565927 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f699c0d0a40 con 0x7f69a4108510 2024-06-10T01:27:55.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.747+0000 7f69a9b67700 1 --2- 172.21.15.7:0/267565927 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f69900a2d20 0x7f69900a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:55.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.748+0000 7f69a9b67700 1 --2- 172.21.15.7:0/267565927 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f69900a2d20 0x7f69900a51e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f69a0005e90 tx=0x7f69a0005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:55.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:55.753+0000 7f699affd700 1 -- 172.21.15.7:0/267565927 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f699c09d650 con 0x7f69a4108510 2024-06-10T01:27:55.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:55 smithi078 ceph-mon[36683]: pgmap v1091: 129 pgs: 129 active+clean; 677 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.3 MiB/s wr, 470 op/s 2024-06-10T01:27:55.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:55 smithi175 ceph-mon[36636]: pgmap v1091: 129 pgs: 129 active+clean; 677 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.3 MiB/s wr, 470 op/s 2024-06-10T01:27:56.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:56.108+0000 7f69abdcb700 1 -- 172.21.15.7:0/267565927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69a4069180 con 0x7f69a4108510 2024-06-10T01:27:56.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:56.109+0000 7f699affd700 1 -- 172.21.15.7:0/267565927 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11852 (secure 0 0 0) 0x7f69a4069180 con 0x7f69a4108510 2024-06-10T01:27:56.102 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:27:56.103 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:27:44.542390+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29813":{"gid":29813,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/138896429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":138896429},{"type":"v1","addr":"172.21.15.175:6835","nonce":138896429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24451,24434,24436]},"id":1}]} 2024-06-10T01:27:56.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:56.113+0000 7f69abdcb700 1 -- 172.21.15.7:0/267565927 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f69900a2d20 msgr2=0x7f69900a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:56.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:56.113+0000 7f69abdcb700 1 --2- 172.21.15.7:0/267565927 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f69900a2d20 0x7f69900a51e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f69a0005e90 tx=0x7f69a0005d00 comp rx=0 tx=0).stop 2024-06-10T01:27:56.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:56.113+0000 7f69abdcb700 1 -- 172.21.15.7:0/267565927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69a4108510 msgr2=0x7f69a410bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:56.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:56.113+0000 7f69abdcb700 1 --2- 172.21.15.7:0/267565927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69a4108510 0x7f69a410bde0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f699c0062a0 tx=0x7f699c00cfb0 comp rx=0 tx=0).stop 2024-06-10T01:27:56.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:56.113+0000 7f69abdcb700 1 -- 172.21.15.7:0/267565927 shutdown_connections 2024-06-10T01:27:56.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:56.113+0000 7f69abdcb700 1 --2- 172.21.15.7:0/267565927 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f69900a2d20 0x7f69900a51e0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:56.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:56.113+0000 7f69abdcb700 1 --2- 172.21.15.7:0/267565927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69a4108510 0x7f69a410bde0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:56.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:56.113+0000 7f69abdcb700 1 --2- 172.21.15.7:0/267565927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69a4100a80 0x7f69a41a3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:56.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:56.113+0000 7f69abdcb700 1 --2- 172.21.15.7:0/267565927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69a4101440 0x7f69a41a41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:56.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:56.113+0000 7f69abdcb700 1 -- 172.21.15.7:0/267565927 >> 172.21.15.7:0/267565927 conn(0x7f69a4077a00 msgr2=0x7f69a4106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:56.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:56.114+0000 7f69abdcb700 1 -- 172.21.15.7:0/267565927 shutdown_connections 2024-06-10T01:27:56.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:27:56.114+0000 7f69abdcb700 1 -- 172.21.15.7:0/267565927 wait complete. 2024-06-10T01:27:56.108 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 55 2024-06-10T01:27:56.554 INFO:tasks.workunit.client.0.smithi007.stdout: 10 209671 23.20 MB/sec execute 393 sec latency 86.895 ms 2024-06-10T01:27:56.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:56 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/267565927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:27:56.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:56 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/267565927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:27:56.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:56 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/267565927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:27:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.143+0000 7f230d429640 1 -- 172.21.15.7:0/4057614422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23081024b0 msgr2=0x7f2308108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.143+0000 7f230d429640 1 --2- 172.21.15.7:0/4057614422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23081024b0 0x7f2308108d40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2300009be0 tx=0x7f230002a6c0 comp rx=0 tx=0).stop 2024-06-10T01:27:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.143+0000 7f230d429640 1 -- 172.21.15.7:0/4057614422 shutdown_connections 2024-06-10T01:27:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.143+0000 7f230d429640 1 --2- 172.21.15.7:0/4057614422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23081024b0 0x7f2308108d40 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.143+0000 7f230d429640 1 --2- 172.21.15.7:0/4057614422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2308101af0 0x7f2308101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.143+0000 7f230d429640 1 --2- 172.21.15.7:0/4057614422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2308100900 0x7f2308100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.143+0000 7f230d429640 1 -- 172.21.15.7:0/4057614422 >> 172.21.15.7:0/4057614422 conn(0x7f23080fbe70 msgr2=0x7f23080fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.143+0000 7f230d429640 1 -- 172.21.15.7:0/4057614422 shutdown_connections 2024-06-10T01:27:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.143+0000 7f230d429640 1 -- 172.21.15.7:0/4057614422 wait complete. 2024-06-10T01:27:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.144+0000 7f230d429640 1 Processor -- start 2024-06-10T01:27:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.145+0000 7f230d429640 1 -- start start 2024-06-10T01:27:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.145+0000 7f230d429640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2308100900 0x7f2308197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.145+0000 7f230d429640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2308101af0 0x7f2308198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.145+0000 7f230d429640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23081024b0 0x7f230819f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.145+0000 7f230d429640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f230810d6a0 con 0x7f2308100900 2024-06-10T01:27:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.145+0000 7f230d429640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f230810d520 con 0x7f23081024b0 2024-06-10T01:27:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.145+0000 7f230d429640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f230810d820 con 0x7f2308101af0 2024-06-10T01:27:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.145+0000 7f2306ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2308100900 0x7f2308197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.145+0000 7f23067fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2308101af0 0x7f2308198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.145+0000 7f23077fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23081024b0 0x7f230819f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.145+0000 7f2306ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2308100900 0x7f2308197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41486/0 (socket says 172.21.15.7:41486) 2024-06-10T01:27:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.145+0000 7f23077fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23081024b0 0x7f230819f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:35368/0 (socket says 172.21.15.7:35368) 2024-06-10T01:27:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.145+0000 7f23067fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2308101af0 0x7f2308198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:58608/0 (socket says 172.21.15.7:58608) 2024-06-10T01:27:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.145+0000 7f2306ffd640 1 -- 172.21.15.7:0/739687198 learned_addr learned my addr 172.21.15.7:0/739687198 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.145+0000 7f2306ffd640 1 -- 172.21.15.7:0/739687198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2308101af0 msgr2=0x7f2308198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.145+0000 7f2306ffd640 1 --2- 172.21.15.7:0/739687198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2308101af0 0x7f2308198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.145+0000 7f2306ffd640 1 -- 172.21.15.7:0/739687198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23081024b0 msgr2=0x7f230819f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.145+0000 7f2306ffd640 1 --2- 172.21.15.7:0/739687198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23081024b0 0x7f230819f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.146+0000 7f2306ffd640 1 -- 172.21.15.7:0/739687198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f230819f930 con 0x7f2308100900 2024-06-10T01:27:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.146+0000 7f23077fe640 1 --2- 172.21.15.7:0/739687198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23081024b0 0x7f230819f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:27:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.146+0000 7f2306ffd640 1 --2- 172.21.15.7:0/739687198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2308100900 0x7f2308197d20 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f22f800e9c0 tx=0x7f22f800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.146+0000 7f22effff640 1 -- 172.21.15.7:0/739687198 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f800ccc0 con 0x7f2308100900 2024-06-10T01:27:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.146+0000 7f22effff640 1 -- 172.21.15.7:0/739687198 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22f80035f0 con 0x7f2308100900 2024-06-10T01:27:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.146+0000 7f22effff640 1 -- 172.21.15.7:0/739687198 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f800ce60 con 0x7f2308100900 2024-06-10T01:27:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.146+0000 7f230d429640 1 -- 172.21.15.7:0/739687198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2308109760 con 0x7f2308100900 2024-06-10T01:27:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.146+0000 7f230d429640 1 -- 172.21.15.7:0/739687198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2308109c60 con 0x7f2308100900 2024-06-10T01:27:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.149+0000 7f22effff640 1 -- 172.21.15.7:0/739687198 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f22f80037a0 con 0x7f2308100900 2024-06-10T01:27:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.149+0000 7f231020b740 1 Processor -- start 2024-06-10T01:27:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.149+0000 7f231020b740 1 -- start start 2024-06-10T01:27:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.150+0000 7f231020b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576aa3c0010 0x5576aa3c0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.150+0000 7f231020b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576aa3c0a00 0x5576aa3c5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.150+0000 7f22effff640 1 --2- 172.21.15.7:0/739687198 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f22f00a2c40 0x7f22f00a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.150+0000 7f231020b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576aa3c5b70 0x5576aa3c8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.150+0000 7f231020b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5576aa3234f0 con 0x5576aa3c0010 2024-06-10T01:27:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.150+0000 7f231020b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5576aa2c1870 con 0x5576aa3c5b70 2024-06-10T01:27:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.150+0000 7f231020b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5576aa31f4b0 con 0x5576aa3c0a00 2024-06-10T01:27:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.150+0000 7f23077fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576aa3c0010 0x5576aa3c0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.150+0000 7f2306ffd640 1 --2- 172.21.15.7:0/739687198 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f22f00a2c40 0x7f22f00a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.150+0000 7f23067fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576aa3c0a00 0x5576aa3c5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.150+0000 7f23077fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576aa3c0010 0x5576aa3c0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41496/0 (socket says 172.21.15.7:41496) 2024-06-10T01:27:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.150+0000 7f23067fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576aa3c0a00 0x5576aa3c5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:58618/0 (socket says 172.21.15.7:58618) 2024-06-10T01:27:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.150+0000 7f23067fc640 1 -- 172.21.15.7:0/1796308548 learned_addr learned my addr 172.21.15.7:0/1796308548 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.150+0000 7f22effff640 1 -- 172.21.15.7:0/739687198 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f22f80cf6d0 con 0x7f2308100900 2024-06-10T01:27:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.150+0000 7f23077fe640 1 --2- 172.21.15.7:0/1796308548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576aa3c5b70 0x5576aa3c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.150+0000 7f23067fc640 1 -- 172.21.15.7:0/1796308548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576aa3c5b70 msgr2=0x5576aa3c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.150+0000 7f23067fc640 1 --2- 172.21.15.7:0/1796308548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576aa3c5b70 0x5576aa3c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.150+0000 7f23067fc640 1 -- 172.21.15.7:0/1796308548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576aa3c0010 msgr2=0x5576aa3c0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.150+0000 7f23067fc640 1 --2- 172.21.15.7:0/1796308548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576aa3c0010 0x5576aa3c0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.150+0000 7f23067fc640 1 -- 172.21.15.7:0/1796308548 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576aa2dd9b0 con 0x5576aa3c0a00 2024-06-10T01:27:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.150+0000 7f2306ffd640 1 --2- 172.21.15.7:0/739687198 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f22f00a2c40 0x7f22f00a5100 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f22f809bb40 tx=0x7f22f80667a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.150+0000 7f23067fc640 1 --2- 172.21.15.7:0/1796308548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576aa3c0a00 0x5576aa3c5500 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009cc0 tx=0x7f22fc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b312e724efc18492 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.151+0000 7f22ed7fa640 1 -- 172.21.15.7:0/1796308548 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc009db0 con 0x5576aa3c0a00 2024-06-10T01:27:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.151+0000 7f22ed7fa640 1 -- 172.21.15.7:0/1796308548 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22fc005500 con 0x5576aa3c0a00 2024-06-10T01:27:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.151+0000 7f22ed7fa640 1 -- 172.21.15.7:0/1796308548 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc019670 con 0x5576aa3c0a00 2024-06-10T01:27:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.152+0000 7f231020b740 1 -- 172.21.15.7:0/1796308548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576aa3c0a00 msgr2=0x5576aa3c5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.152+0000 7f231020b740 1 --2- 172.21.15.7:0/1796308548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576aa3c0a00 0x5576aa3c5500 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009cc0 tx=0x7f22fc011810 comp rx=0 tx=0).stop 2024-06-10T01:27:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.152+0000 7f231020b740 1 -- 172.21.15.7:0/1796308548 shutdown_connections 2024-06-10T01:27:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.152+0000 7f231020b740 1 --2- 172.21.15.7:0/1796308548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576aa3c5b70 0x5576aa3c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.152+0000 7f231020b740 1 --2- 172.21.15.7:0/1796308548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576aa3c0a00 0x5576aa3c5500 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.152+0000 7f231020b740 1 --2- 172.21.15.7:0/1796308548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576aa3c0010 0x5576aa3c0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.152+0000 7f231020b740 1 -- 172.21.15.7:0/1796308548 >> 172.21.15.7:0/1796308548 conn(0x5576aa3bb680 msgr2=0x5576aa3bba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.152+0000 7f231020b740 1 -- 172.21.15.7:0/1796308548 shutdown_connections 2024-06-10T01:27:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.152+0000 7f231020b740 1 -- 172.21.15.7:0/1796308548 wait complete. 2024-06-10T01:27:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.152+0000 7f231020b740 1 -- start start 2024-06-10T01:27:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.152+0000 7f231020b740 1 Processor -- start 2024-06-10T01:27:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.152+0000 7f231020b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576aa3c0010 0x5576aa3ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.152+0000 7f231020b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576aa3c0a00 0x5576aa3ca730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.152+0000 7f231020b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576aa3c5b70 0x5576aa3d1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.152+0000 7f231020b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5576aa3c8b20 con 0x5576aa3c0a00 2024-06-10T01:27:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.152+0000 7f231020b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5576aa2acae0 con 0x5576aa3c5b70 2024-06-10T01:27:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.152+0000 7f231020b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5576aa3c8ca0 con 0x5576aa3c0010 2024-06-10T01:27:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.152+0000 7f231020b740 10 client.0 _ll_register_callbacks cb 0x5576aa2e59b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:27:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.153+0000 7f23067fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576aa3c0a00 0x5576aa3ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.153+0000 7f23077fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576aa3c0010 0x5576aa3ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.153+0000 7f23067fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576aa3c0a00 0x5576aa3ca730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41500/0 (socket says 172.21.15.7:41500) 2024-06-10T01:27:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.153+0000 7f23067fc640 1 -- 172.21.15.7:0/3472865656 learned_addr learned my addr 172.21.15.7:0/3472865656 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.153+0000 7f23077fe640 1 --2- 172.21.15.7:0/3472865656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576aa3c5b70 0x5576aa3d1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.153+0000 7f23067fc640 1 -- 172.21.15.7:0/3472865656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576aa3c0010 msgr2=0x5576aa3ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.153+0000 7f23067fc640 1 --2- 172.21.15.7:0/3472865656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576aa3c0010 0x5576aa3ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.153+0000 7f23067fc640 1 -- 172.21.15.7:0/3472865656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576aa3c5b70 msgr2=0x5576aa3d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.153+0000 7f23067fc640 1 --2- 172.21.15.7:0/3472865656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576aa3c5b70 0x5576aa3d1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.153+0000 7f23067fc640 1 -- 172.21.15.7:0/3472865656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576aa037210 con 0x5576aa3c0a00 2024-06-10T01:27:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.153+0000 7f23067fc640 1 --2- 172.21.15.7:0/3472865656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576aa3c0a00 0x5576aa3ca730 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f22fc011d10 tx=0x7f22fc005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1fecc78b6630f6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.153+0000 7f22eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:27:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.153+0000 7f22eaff5640 1 -- 172.21.15.7:0/3472865656 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc028070 con 0x5576aa3c0a00 2024-06-10T01:27:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.153+0000 7f22eaff5640 1 -- 172.21.15.7:0/3472865656 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f22fc005190 con 0x5576aa3c0a00 2024-06-10T01:27:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.153+0000 7f231020b740 1 -- 172.21.15.7:0/3472865656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22fc005500 con 0x5576aa3c0a00 2024-06-10T01:27:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.153+0000 7f22eaff5640 1 -- 172.21.15.7:0/3472865656 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f22fc034050 con 0x5576aa3c0a00 2024-06-10T01:27:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.153+0000 7f231020b740 10 client.20649 fetch_fsmap learned FSMap version 55 2024-06-10T01:27:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.154+0000 7f231020b740 1 -- 172.21.15.7:0/3472865656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5576aa103620 con 0x5576aa3c0a00 2024-06-10T01:27:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.154+0000 7f22eaff5640 1 -- 172.21.15.7:0/3472865656 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22fc005500 con 0x5576aa3c0a00 2024-06-10T01:27:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.154+0000 7f22eaff5640 1 -- 172.21.15.7:0/3472865656 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f22fc022410 con 0x5576aa3c0a00 2024-06-10T01:27:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.154+0000 7f231020b740 10 client.20649 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:27:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.154+0000 7f231020b740 10 client.20649 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:27:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.154+0000 7f231020b740 20 client.20649 populate_metadata read hostname 'smithi007' 2024-06-10T01:27:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.154+0000 7f231020b740 1 --2- 172.21.15.7:0/3472865656 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5576aa3d5ec0 0x5576aa3d82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.154+0000 7f231020b740 4 client.20649 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:27:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.154+0000 7f231020b740 1 -- 172.21.15.7:0/3472865656 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576aa336f10 con 0x5576aa3d5ec0 2024-06-10T01:27:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.154+0000 7f23077fe640 1 --2- 172.21.15.7:0/3472865656 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5576aa3d5ec0 0x5576aa3d82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.155+0000 7f22eaff5640 10 client.20649 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.155+0000 7f23077fe640 1 --2- 172.21.15.7:0/3472865656 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5576aa3d5ec0 0x5576aa3d82b0 crc :-1 s=READY pgs=493 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4219ec00df43c40b server_cookie=11036958a9e5102f in_seq=0 out_seq=0 2024-06-10T01:27:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.156+0000 7f22eaff5640 1 -- 172.21.15.7:0/3472865656 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5576aa336f10 con 0x5576aa3d5ec0 2024-06-10T01:27:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.156+0000 7f22eaff5640 10 client.20649 handle_command_reply: tid=0 2024-06-10T01:27:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.156+0000 7f231020b740 1 client.20649 shutdown 2024-06-10T01:27:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.157+0000 7f231020b740 1 -- 172.21.15.7:0/3472865656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576aa3c0a00 msgr2=0x5576aa3ca730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.157+0000 7f231020b740 1 --2- 172.21.15.7:0/3472865656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576aa3c0a00 0x5576aa3ca730 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f22fc011d10 tx=0x7f22fc005e80 comp rx=0 tx=0).stop 2024-06-10T01:27:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.157+0000 7f231020b740 1 -- 172.21.15.7:0/3472865656 shutdown_connections 2024-06-10T01:27:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.157+0000 7f231020b740 1 --2- 172.21.15.7:0/3472865656 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5576aa3d5ec0 0x5576aa3d82b0 crc :-1 s=READY pgs=493 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.157+0000 7f231020b740 1 --2- 172.21.15.7:0/3472865656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576aa3c5b70 0x5576aa3d1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.157+0000 7f231020b740 1 --2- 172.21.15.7:0/3472865656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576aa3c0a00 0x5576aa3ca730 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.157+0000 7f231020b740 1 --2- 172.21.15.7:0/3472865656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576aa3c0010 0x5576aa3ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.157+0000 7f231020b740 1 -- 172.21.15.7:0/3472865656 >> 172.21.15.7:0/3472865656 conn(0x5576aa3bb680 msgr2=0x5576aa3ba7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.157+0000 7f22eaff5640 0 client.20649 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.157+0000 7f231020b740 1 -- 172.21.15.7:0/3472865656 shutdown_connections 2024-06-10T01:27:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.157+0000 7f231020b740 1 -- 172.21.15.7:0/3472865656 wait complete. 2024-06-10T01:27:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.157+0000 7f231020b740 20 client.20649 trim_cache size 0 max 16384 2024-06-10T01:27:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.177+0000 7f231020b740 1 Processor -- start 2024-06-10T01:27:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.177+0000 7f231020b740 1 -- start start 2024-06-10T01:27:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.177+0000 7f231020b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576aa3c0010 0x5576aa3ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.177+0000 7f231020b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576aa3c0a00 0x5576aa3ca730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.177+0000 7f231020b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576aa3c5b70 0x5576aa3d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.177+0000 7f231020b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f23081aab60 con 0x5576aa3c0a00 2024-06-10T01:27:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.177+0000 7f231020b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f23081a8430 con 0x5576aa3c0010 2024-06-10T01:27:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.177+0000 7f231020b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f23081a6280 con 0x5576aa3c5b70 2024-06-10T01:27:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.177+0000 7f23067fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576aa3c0a00 0x5576aa3ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.177+0000 7f23067fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576aa3c0a00 0x5576aa3ca730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41514/0 (socket says 172.21.15.7:41514) 2024-06-10T01:27:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.177+0000 7f23067fc640 1 -- 172.21.15.7:0/4025848001 learned_addr learned my addr 172.21.15.7:0/4025848001 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.177+0000 7f23077fe640 1 --2- 172.21.15.7:0/4025848001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576aa3c0010 0x5576aa3ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.177+0000 7f23077fe640 1 --2- 172.21.15.7:0/4025848001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576aa3c5b70 0x5576aa3d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.177+0000 7f23067fc640 1 -- 172.21.15.7:0/4025848001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576aa3c5b70 msgr2=0x5576aa3d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.177+0000 7f23067fc640 1 --2- 172.21.15.7:0/4025848001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576aa3c5b70 0x5576aa3d1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.177+0000 7f23067fc640 1 -- 172.21.15.7:0/4025848001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576aa3c0010 msgr2=0x5576aa3ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.177+0000 7f23067fc640 1 --2- 172.21.15.7:0/4025848001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576aa3c0010 0x5576aa3ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.177+0000 7f23067fc640 1 -- 172.21.15.7:0/4025848001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576aa3c9610 con 0x5576aa3c0a00 2024-06-10T01:27:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.177+0000 7f23077fe640 1 --2- 172.21.15.7:0/4025848001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576aa3c0010 0x5576aa3ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:27:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.177+0000 7f23067fc640 1 --2- 172.21.15.7:0/4025848001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576aa3c0a00 0x5576aa3ca730 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009530 tx=0x7f22fc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f97e9f97ea502bd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.178+0000 7f22ecff9640 1 -- 172.21.15.7:0/4025848001 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc028040 con 0x5576aa3c0a00 2024-06-10T01:27:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.178+0000 7f22ecff9640 1 -- 172.21.15.7:0/4025848001 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22fc016070 con 0x5576aa3c0a00 2024-06-10T01:27:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.178+0000 7f22ecff9640 1 -- 172.21.15.7:0/4025848001 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc02d440 con 0x5576aa3c0a00 2024-06-10T01:27:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.179+0000 7f231020b740 1 -- 172.21.15.7:0/4025848001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576aa3c0a00 msgr2=0x5576aa3ca730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.179+0000 7f231020b740 1 --2- 172.21.15.7:0/4025848001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576aa3c0a00 0x5576aa3ca730 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009530 tx=0x7f22fc005fd0 comp rx=0 tx=0).stop 2024-06-10T01:27:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.179+0000 7f231020b740 1 -- 172.21.15.7:0/4025848001 shutdown_connections 2024-06-10T01:27:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.179+0000 7f231020b740 1 --2- 172.21.15.7:0/4025848001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576aa3c5b70 0x5576aa3d1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.179+0000 7f231020b740 1 --2- 172.21.15.7:0/4025848001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576aa3c0a00 0x5576aa3ca730 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.179+0000 7f231020b740 1 --2- 172.21.15.7:0/4025848001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576aa3c0010 0x5576aa3ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.179+0000 7f231020b740 1 -- 172.21.15.7:0/4025848001 >> 172.21.15.7:0/4025848001 conn(0x5576aa3bb680 msgr2=0x5576aa3c51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.179+0000 7f231020b740 1 -- 172.21.15.7:0/4025848001 shutdown_connections 2024-06-10T01:27:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.179+0000 7f231020b740 1 -- 172.21.15.7:0/4025848001 wait complete. 2024-06-10T01:27:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.179+0000 7f231020b740 1 -- start start 2024-06-10T01:27:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.179+0000 7f231020b740 1 Processor -- start 2024-06-10T01:27:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.180+0000 7f231020b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576aa3c0010 0x5576aa3d1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.180+0000 7f231020b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576aa3c0a00 0x5576aa3ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.180+0000 7f231020b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576aa3c5b70 0x5576aa3ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.180+0000 7f23067fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576aa3c0a00 0x5576aa3ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.180+0000 7f23067fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576aa3c0a00 0x5576aa3ce0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:35404/0 (socket says 172.21.15.7:35404) 2024-06-10T01:27:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.180+0000 7f23067fc640 1 -- 172.21.15.7:0/1367988937 learned_addr learned my addr 172.21.15.7:0/1367988937 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:27:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.180+0000 7f231020b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f23081a68f0 con 0x5576aa3c5b70 2024-06-10T01:27:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.180+0000 7f231020b740 1 -- 172.21.15.7:0/1367988937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f23081a7d60 con 0x5576aa3c0a00 2024-06-10T01:27:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.180+0000 7f231020b740 1 -- 172.21.15.7:0/1367988937 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f23081a6fc0 con 0x5576aa3c0010 2024-06-10T01:27:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.180+0000 7f23077fe640 1 --2- 172.21.15.7:0/1367988937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576aa3c5b70 0x5576aa3ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.180+0000 7f231020b740 10 client.0 _ll_register_callbacks cb 0x7f22fc005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:27:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.180+0000 7f23067fc640 1 -- 172.21.15.7:0/1367988937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576aa3c0010 msgr2=0x5576aa3d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.180+0000 7f23067fc640 1 --2- 172.21.15.7:0/1367988937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576aa3c0010 0x5576aa3d1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.180+0000 7f23067fc640 1 -- 172.21.15.7:0/1367988937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576aa3c5b70 msgr2=0x5576aa3ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:27:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.180+0000 7f23067fc640 1 --2- 172.21.15.7:0/1367988937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576aa3c5b70 0x5576aa3ce5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.180+0000 7f23067fc640 1 -- 172.21.15.7:0/1367988937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576aa43eb30 con 0x5576aa3c0a00 2024-06-10T01:27:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.180+0000 7f23077fe640 1 --2- 172.21.15.7:0/1367988937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576aa3c5b70 0x5576aa3ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:27:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.181+0000 7f23067fc640 1 --2- 172.21.15.7:0/1367988937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576aa3c0a00 0x5576aa3ce0b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009530 tx=0x7f22fc005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a457afba6047af87 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:27:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.181+0000 7f22eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:27:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.181+0000 7f22eaff5640 1 -- 172.21.15.7:0/1367988937 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc03c020 con 0x5576aa3c0a00 2024-06-10T01:27:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.181+0000 7f22eaff5640 1 -- 172.21.15.7:0/1367988937 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f22fc005190 con 0x5576aa3c0a00 2024-06-10T01:27:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.181+0000 7f231020b740 1 -- 172.21.15.7:0/1367988937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22fc016070 con 0x5576aa3c0a00 2024-06-10T01:27:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.181+0000 7f22eaff5640 1 -- 172.21.15.7:0/1367988937 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f22fc0225a0 con 0x5576aa3c0a00 2024-06-10T01:27:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.181+0000 7f231020b740 10 client.29849 fetch_fsmap learned FSMap version 55 2024-06-10T01:27:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.181+0000 7f231020b740 1 -- 172.21.15.7:0/1367988937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5576aa43fca0 con 0x5576aa3c0a00 2024-06-10T01:27:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.182+0000 7f22eaff5640 1 -- 172.21.15.7:0/1367988937 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22fc024440 con 0x5576aa3c0a00 2024-06-10T01:27:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.182+0000 7f22eaff5640 1 -- 172.21.15.7:0/1367988937 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f22fc02d440 con 0x5576aa3c0a00 2024-06-10T01:27:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.182+0000 7f231020b740 10 client.29849 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:27:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.182+0000 7f231020b740 10 client.29849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:27:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.182+0000 7f231020b740 20 client.29849 populate_metadata read hostname 'smithi007' 2024-06-10T01:27:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.182+0000 7f231020b740 1 --2- 172.21.15.7:0/1367988937 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5576aa440670 0x5576aa442a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:27:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.182+0000 7f231020b740 4 client.29849 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:27:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.182+0000 7f231020b740 1 -- 172.21.15.7:0/1367988937 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f22fc016070 con 0x5576aa440670 2024-06-10T01:27:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.183+0000 7f23077fe640 1 --2- 172.21.15.7:0/1367988937 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5576aa440670 0x5576aa442a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:27:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.184+0000 7f23077fe640 1 --2- 172.21.15.7:0/1367988937 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5576aa440670 0x5576aa442a90 crc :-1 s=READY pgs=494 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53c2b8c45ad752d1 server_cookie=f226114679dbdc66 in_seq=0 out_seq=0 2024-06-10T01:27:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:57.184+0000 7f22eaff5640 10 client.29849 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:57.518 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:27:57.556 INFO:tasks.workunit.client.0.smithi007.stdout: 10 210169 23.18 MB/sec execute 394 sec latency 291.558 ms 2024-06-10T01:27:57.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:57 smithi007 ceph-mon[32398]: pgmap v1092: 129 pgs: 129 active+clean; 686 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 3.8 MiB/s wr, 532 op/s 2024-06-10T01:27:57.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:57 smithi078 ceph-mon[36683]: pgmap v1092: 129 pgs: 129 active+clean; 686 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 3.8 MiB/s wr, 532 op/s 2024-06-10T01:27:57.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:57 smithi175 ceph-mon[36636]: pgmap v1092: 129 pgs: 129 active+clean; 686 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 3.8 MiB/s wr, 532 op/s 2024-06-10T01:27:58.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:58.016+0000 7f22eaff5640 1 -- 172.21.15.7:0/1367988937 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f22fc016070 con 0x5576aa440670 2024-06-10T01:27:58.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:58.016+0000 7f22eaff5640 10 client.29849 handle_command_reply: tid=0 2024-06-10T01:27:58.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:58.016+0000 7f231020b740 1 client.29849 shutdown 2024-06-10T01:27:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:58.019+0000 7f231020b740 1 -- 172.21.15.7:0/1367988937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576aa3c0a00 msgr2=0x5576aa3ce0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:58.019+0000 7f231020b740 1 --2- 172.21.15.7:0/1367988937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576aa3c0a00 0x5576aa3ce0b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009530 tx=0x7f22fc005c50 comp rx=0 tx=0).stop 2024-06-10T01:27:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:58.019+0000 7f231020b740 1 -- 172.21.15.7:0/1367988937 shutdown_connections 2024-06-10T01:27:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:58.019+0000 7f231020b740 1 --2- 172.21.15.7:0/1367988937 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5576aa440670 0x5576aa442a90 crc :-1 s=READY pgs=494 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:58.019+0000 7f231020b740 1 --2- 172.21.15.7:0/1367988937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576aa3c5b70 0x5576aa3ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:58.019+0000 7f231020b740 1 --2- 172.21.15.7:0/1367988937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576aa3c0a00 0x5576aa3ce0b0 secure :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009530 tx=0x7f22fc005c50 comp rx=0 tx=0).stop 2024-06-10T01:27:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:58.019+0000 7f231020b740 1 --2- 172.21.15.7:0/1367988937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5576aa3c0010 0x5576aa3d1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:58.019+0000 7f231020b740 1 -- 172.21.15.7:0/1367988937 >> 172.21.15.7:0/1367988937 conn(0x5576aa3bb680 msgr2=0x5576aa3be5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:58.019+0000 7f22eaff5640 0 client.29849 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:27:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:58.020+0000 7f231020b740 1 -- 172.21.15.7:0/1367988937 shutdown_connections 2024-06-10T01:27:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:27:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:27:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:27:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:58.020+0000 7f231020b740 1 -- 172.21.15.7:0/1367988937 wait complete. 2024-06-10T01:27:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:58.020+0000 7f231020b740 20 client.29849 trim_cache size 0 max 16384 2024-06-10T01:27:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:58.027+0000 7f230d429640 1 -- 172.21.15.7:0/739687198 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f22f00a2c40 msgr2=0x7f22f00a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:58.027+0000 7f230d429640 1 --2- 172.21.15.7:0/739687198 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f22f00a2c40 0x7f22f00a5100 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f22f809bb40 tx=0x7f22f80667a0 comp rx=0 tx=0).stop 2024-06-10T01:27:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:58.027+0000 7f230d429640 1 -- 172.21.15.7:0/739687198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2308100900 msgr2=0x7f2308197d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:58.027+0000 7f230d429640 1 --2- 172.21.15.7:0/739687198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2308100900 0x7f2308197d20 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f22f800e9c0 tx=0x7f22f800ee80 comp rx=0 tx=0).stop 2024-06-10T01:27:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:58.027+0000 7f230d429640 1 -- 172.21.15.7:0/739687198 shutdown_connections 2024-06-10T01:27:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:58.027+0000 7f230d429640 1 --2- 172.21.15.7:0/739687198 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f22f00a2c40 0x7f22f00a5100 secure :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f22f809bb40 tx=0x7f22f80667a0 comp rx=0 tx=0).stop 2024-06-10T01:27:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:58.027+0000 7f230d429640 1 --2- 172.21.15.7:0/739687198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23081024b0 0x7f230819f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:58.027+0000 7f230d429640 1 --2- 172.21.15.7:0/739687198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2308101af0 0x7f2308198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:27:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:58.027+0000 7f230d429640 1 --2- 172.21.15.7:0/739687198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2308100900 0x7f2308197d20 secure :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f22f800e9c0 tx=0x7f22f800ee80 comp rx=0 tx=0).stop 2024-06-10T01:27:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:58.027+0000 7f230d429640 1 -- 172.21.15.7:0/739687198 >> 172.21.15.7:0/739687198 conn(0x7f23080fbe70 msgr2=0x7f2308104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:27:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:58.027+0000 7f230d429640 1 -- 172.21.15.7:0/739687198 shutdown_connections 2024-06-10T01:27:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:27:58.027+0000 7f230d429640 1 -- 172.21.15.7:0/739687198 wait complete. 2024-06-10T01:27:58.021 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:27:58.554 INFO:tasks.workunit.client.0.smithi007.stdout: 10 210632 23.21 MB/sec execute 395 sec latency 582.111 ms 2024-06-10T01:27:59.554 INFO:tasks.workunit.client.0.smithi007.stdout: 10 211328 23.24 MB/sec execute 396 sec latency 54.534 ms 2024-06-10T01:27:59.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:27:59 smithi078 ceph-mon[36683]: pgmap v1093: 129 pgs: 129 active+clean; 689 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.3 MiB/s wr, 426 op/s 2024-06-10T01:27:59.829 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:27:59 smithi007 ceph-mon[32398]: pgmap v1093: 129 pgs: 129 active+clean; 689 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.3 MiB/s wr, 426 op/s 2024-06-10T01:27:59.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:27:59 smithi175 ceph-mon[36636]: pgmap v1093: 129 pgs: 129 active+clean; 689 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.3 MiB/s wr, 426 op/s 2024-06-10T01:27:59.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.005+0000 7f54c2ffd700 1 -- 172.21.15.7:0/3807592751 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b4003ce0 con 0x7f54c4110c70 2024-06-10T01:27:59.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.006+0000 7f54ca20e700 1 -- 172.21.15.7:0/3807592751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54c4110c70 msgr2=0x7f54c4109490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:27:59.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.006+0000 7f54ca20e700 1 --2- 172.21.15.7:0/3807592751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54c4110c70 0x7f54c4109490 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f54b401a9a0 tx=0x7f54b401acb0 comp rx=0 tx=0).stop 2024-06-10T01:27:59.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.006+0000 7f54ca20e700 1 -- 172.21.15.7:0/3807592751 shutdown_connections 2024-06-10T01:28:00.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.006+0000 7f54ca20e700 1 --2- 172.21.15.7:0/3807592751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54c411bc70 0x7f54c411e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.006+0000 7f54ca20e700 1 --2- 172.21.15.7:0/3807592751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54c4110c70 0x7f54c4109490 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.006+0000 7f54ca20e700 1 --2- 172.21.15.7:0/3807592751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54c41099d0 0x7f54c4109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.006+0000 7f54ca20e700 1 -- 172.21.15.7:0/3807592751 >> 172.21.15.7:0/3807592751 conn(0x7f54c406d600 msgr2=0x7f54c406da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:00.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.007+0000 7f54ca20e700 1 -- 172.21.15.7:0/3807592751 shutdown_connections 2024-06-10T01:28:00.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.007+0000 7f54ca20e700 1 -- 172.21.15.7:0/3807592751 wait complete. 2024-06-10T01:28:00.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.007+0000 7f54ca20e700 1 Processor -- start 2024-06-10T01:28:00.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.008+0000 7f54ca20e700 1 -- start start 2024-06-10T01:28:00.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.008+0000 7f54ca20e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54c41099d0 0x7f54c411b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:00.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.008+0000 7f54ca20e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54c4110c70 0x7f54c4117330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:00.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.008+0000 7f54ca20e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54c411bc70 0x7f54c4117870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:00.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.008+0000 7f54ca20e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f54c4113de0 con 0x7f54c411bc70 2024-06-10T01:28:00.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.008+0000 7f54ca20e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f54c4113c60 con 0x7f54c4110c70 2024-06-10T01:28:00.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.008+0000 7f54ca20e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f54c4113f60 con 0x7f54c41099d0 2024-06-10T01:28:00.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.008+0000 7f54bbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54c4110c70 0x7f54c4117330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:00.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.008+0000 7f54bbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54c4110c70 0x7f54c4117330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:35422/0 (socket says 172.21.15.7:35422) 2024-06-10T01:28:00.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.008+0000 7f54bbfff700 1 -- 172.21.15.7:0/2351858341 learned_addr learned my addr 172.21.15.7:0/2351858341 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:00.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.008+0000 7f54bbfff700 1 -- 172.21.15.7:0/2351858341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54c41099d0 msgr2=0x7f54c411b290 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-10T01:28:00.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.008+0000 7f54bbfff700 1 --2- 172.21.15.7:0/2351858341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54c41099d0 0x7f54c411b290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.008+0000 7f54bbfff700 1 -- 172.21.15.7:0/2351858341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54c411bc70 msgr2=0x7f54c4117870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T01:28:00.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.008+0000 7f54bbfff700 1 --2- 172.21.15.7:0/2351858341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54c411bc70 0x7f54c4117870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.009+0000 7f54bbfff700 1 -- 172.21.15.7:0/2351858341 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c4117e60 con 0x7f54c4110c70 2024-06-10T01:28:00.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.009+0000 7f54bbfff700 1 --2- 172.21.15.7:0/2351858341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54c4110c70 0x7f54c4117330 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f54ac00e9a0 tx=0x7f54ac00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:00.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.010+0000 7f54c17fa700 1 -- 172.21.15.7:0/2351858341 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac00cbd0 con 0x7f54c4110c70 2024-06-10T01:28:00.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.010+0000 7f54ca20e700 1 -- 172.21.15.7:0/2351858341 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c4118150 con 0x7f54c4110c70 2024-06-10T01:28:00.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.010+0000 7f54ca20e700 1 -- 172.21.15.7:0/2351858341 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c41abe80 con 0x7f54c4110c70 2024-06-10T01:28:00.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.010+0000 7f54c17fa700 1 -- 172.21.15.7:0/2351858341 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54ac00cd70 con 0x7f54c4110c70 2024-06-10T01:28:00.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.011+0000 7f54c17fa700 1 -- 172.21.15.7:0/2351858341 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac018840 con 0x7f54c4110c70 2024-06-10T01:28:00.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.017+0000 7f54d4a91b80 1 Processor -- start 2024-06-10T01:28:00.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.018+0000 7f54d4a91b80 1 -- start start 2024-06-10T01:28:00.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.018+0000 7f54d4a91b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a05232a640 0x55a05232aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:00.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.018+0000 7f54d4a91b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a05232af60 0x55a0523249e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:00.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.018+0000 7f54d4a91b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a052324f20 0x55a052325340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:00.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.018+0000 7f54d4a91b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a052276810 con 0x55a052324f20 2024-06-10T01:28:00.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.018+0000 7f54d4a91b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a05221e720 con 0x55a05232af60 2024-06-10T01:28:00.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.018+0000 7f54d4a91b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a0522727d0 con 0x55a05232a640 2024-06-10T01:28:00.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.019+0000 7f54c37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a05232af60 0x55a0523249e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:00.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.019+0000 7f54c37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a05232af60 0x55a0523249e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:35428/0 (socket says 172.21.15.7:35428) 2024-06-10T01:28:00.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.019+0000 7f54c37fe700 1 -- 172.21.15.7:0/1433626147 learned_addr learned my addr 172.21.15.7:0/1433626147 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:00.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.019+0000 7f54c37fe700 1 -- 172.21.15.7:0/1433626147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a05232a640 msgr2=0x55a05232aa20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:28:00.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.019+0000 7f54c37fe700 1 --2- 172.21.15.7:0/1433626147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a05232a640 0x55a05232aa20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.019+0000 7f54c37fe700 1 -- 172.21.15.7:0/1433626147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a052324f20 msgr2=0x55a052325340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T01:28:00.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.019+0000 7f54c37fe700 1 --2- 172.21.15.7:0/1433626147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a052324f20 0x55a052325340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.019+0000 7f54c37fe700 1 -- 172.21.15.7:0/1433626147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a052214f50 con 0x55a05232af60 2024-06-10T01:28:00.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.020+0000 7f54c17fa700 1 -- 172.21.15.7:0/2351858341 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f54ac0189e0 con 0x7f54c4110c70 2024-06-10T01:28:00.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.020+0000 7f54c37fe700 1 --2- 172.21.15.7:0/1433626147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a05232af60 0x55a0523249e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f54b4006010 tx=0x7f54b401a6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d10c6757c69eb1e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:00.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.021+0000 7f54b9ffb700 1 -- 172.21.15.7:0/1433626147 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b4003960 con 0x55a05232af60 2024-06-10T01:28:00.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.021+0000 7f54b9ffb700 1 -- 172.21.15.7:0/1433626147 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54b40050d0 con 0x55a05232af60 2024-06-10T01:28:00.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.021+0000 7f54c17fa700 1 --2- 172.21.15.7:0/2351858341 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f54b00a2cd0 0x7f54b00a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:00.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.021+0000 7f54c17fa700 1 -- 172.21.15.7:0/2351858341 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f54ac0cf5e0 con 0x7f54c4110c70 2024-06-10T01:28:00.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.022+0000 7f54b9ffb700 1 -- 172.21.15.7:0/1433626147 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b401ca50 con 0x55a05232af60 2024-06-10T01:28:00.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.022+0000 7f54d4a91b80 1 -- 172.21.15.7:0/1433626147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a05232af60 msgr2=0x55a0523249e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:00.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.022+0000 7f54d4a91b80 1 --2- 172.21.15.7:0/1433626147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a05232af60 0x55a0523249e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f54b4006010 tx=0x7f54b401a6b0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.023+0000 7f54c3fff700 1 --2- 172.21.15.7:0/2351858341 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f54b00a2cd0 0x7f54b00a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:00.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.023+0000 7f54d4a91b80 1 -- 172.21.15.7:0/1433626147 shutdown_connections 2024-06-10T01:28:00.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.023+0000 7f54d4a91b80 1 --2- 172.21.15.7:0/1433626147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a052324f20 0x55a052325340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.023+0000 7f54d4a91b80 1 --2- 172.21.15.7:0/1433626147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a05232a640 0x55a05232aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.023+0000 7f54d4a91b80 1 --2- 172.21.15.7:0/1433626147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a05232af60 0x55a0523249e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.023+0000 7f54d4a91b80 1 -- 172.21.15.7:0/1433626147 >> 172.21.15.7:0/1433626147 conn(0x55a05224fcc0 msgr2=0x55a05231f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:00.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.023+0000 7f54c3fff700 1 --2- 172.21.15.7:0/2351858341 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f54b00a2cd0 0x7f54b00a5190 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f54c406bc00 tx=0x7f54bc009250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:00.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.029+0000 7f54d4a91b80 1 -- 172.21.15.7:0/1433626147 shutdown_connections 2024-06-10T01:28:00.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.030+0000 7f54d4a91b80 1 -- 172.21.15.7:0/1433626147 wait complete. 2024-06-10T01:28:00.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.030+0000 7f54d4a91b80 1 -- start start 2024-06-10T01:28:00.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.031+0000 7f54d4a91b80 1 Processor -- start 2024-06-10T01:28:00.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.031+0000 7f54d4a91b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052324f20 0x55a052330cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:00.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.031+0000 7f54d4a91b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a05232a640 0x55a05232dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:00.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.031+0000 7f54d4a91b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a05232af60 0x55a05232e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:00.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.031+0000 7f54d4a91b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a0523319c0 con 0x55a05232af60 2024-06-10T01:28:00.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.031+0000 7f54d4a91b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a052216bf0 con 0x55a052324f20 2024-06-10T01:28:00.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.031+0000 7f54d4a91b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a052331b40 con 0x55a05232a640 2024-06-10T01:28:00.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.031+0000 7f54d4a91b80 10 client.0 _ll_register_callbacks cb 0x55a052210920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:28:00.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.032+0000 7f54c37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052324f20 0x55a052330cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:00.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.032+0000 7f54c3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a05232af60 0x55a05232e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:00.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.032+0000 7f54c37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052324f20 0x55a052330cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:35436/0 (socket says 172.21.15.7:35436) 2024-06-10T01:28:00.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.032+0000 7f54c37fe700 1 -- 172.21.15.7:0/773740706 learned_addr learned my addr 172.21.15.7:0/773740706 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:00.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.032+0000 7f54c3fff700 1 -- 172.21.15.7:0/773740706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a05232a640 msgr2=0x55a05232dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:00.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.032+0000 7f54c3fff700 1 --2- 172.21.15.7:0/773740706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a05232a640 0x55a05232dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.032+0000 7f54c3fff700 1 -- 172.21.15.7:0/773740706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052324f20 msgr2=0x55a052330cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:00.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.032+0000 7f54c3fff700 1 --2- 172.21.15.7:0/773740706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052324f20 0x55a052330cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.032+0000 7f54c3fff700 1 -- 172.21.15.7:0/773740706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0521c58d0 con 0x55a05232af60 2024-06-10T01:28:00.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.033+0000 7f54c3fff700 1 --2- 172.21.15.7:0/773740706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a05232af60 0x55a05232e500 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f54bc00d6a0 tx=0x7f54bc004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1546510aa1cc2402 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:00.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.033+0000 7f54a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:28:00.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.033+0000 7f54a2ffd700 1 -- 172.21.15.7:0/773740706 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54bc01a830 con 0x55a05232af60 2024-06-10T01:28:00.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.033+0000 7f54a2ffd700 1 -- 172.21.15.7:0/773740706 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f54bc01a9d0 con 0x55a05232af60 2024-06-10T01:28:00.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.034+0000 7f54d4a91b80 1 -- 172.21.15.7:0/773740706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54b40050d0 con 0x55a05232af60 2024-06-10T01:28:00.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.034+0000 7f54a2ffd700 1 -- 172.21.15.7:0/773740706 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f54bc01abd0 con 0x55a05232af60 2024-06-10T01:28:00.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.034+0000 7f54d4a91b80 10 client.20661 fetch_fsmap learned FSMap version 55 2024-06-10T01:28:00.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.034+0000 7f54d4a91b80 1 -- 172.21.15.7:0/773740706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55a05235e430 con 0x55a05232af60 2024-06-10T01:28:00.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.035+0000 7f54a2ffd700 1 -- 172.21.15.7:0/773740706 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54b40050d0 con 0x55a05232af60 2024-06-10T01:28:00.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.035+0000 7f54a2ffd700 1 -- 172.21.15.7:0/773740706 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f54bc022490 con 0x55a05232af60 2024-06-10T01:28:00.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.035+0000 7f54d4a91b80 10 client.20661 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:28:00.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.035+0000 7f54d4a91b80 10 client.20661 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:28:00.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.035+0000 7f54d4a91b80 20 client.20661 populate_metadata read hostname 'smithi007' 2024-06-10T01:28:00.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.035+0000 7f54d4a91b80 1 --2- 172.21.15.7:0/773740706 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55a0523609c0 0x55a052362db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:00.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.035+0000 7f54d4a91b80 4 client.20661 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:28:00.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.035+0000 7f54d4a91b80 1 -- 172.21.15.7:0/773740706 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a052283d10 con 0x55a0523609c0 2024-06-10T01:28:00.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.035+0000 7f54c37fe700 1 --2- 172.21.15.7:0/773740706 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55a0523609c0 0x55a052362db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:00.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.036+0000 7f54c37fe700 1 --2- 172.21.15.7:0/773740706 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55a0523609c0 0x55a052362db0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=753c3378b1bf8c7a server_cookie=52fcd8bbe0003611 in_seq=0 out_seq=0 2024-06-10T01:28:00.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.036+0000 7f54a2ffd700 10 client.20661 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:28:00.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.037+0000 7f54a2ffd700 1 -- 172.21.15.7:0/773740706 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a052283d10 con 0x55a0523609c0 2024-06-10T01:28:00.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.037+0000 7f54a2ffd700 10 client.20661 handle_command_reply: tid=0 2024-06-10T01:28:00.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.037+0000 7f54d4a91b80 1 client.20661 shutdown 2024-06-10T01:28:00.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.037+0000 7f54d4a91b80 1 -- 172.21.15.7:0/773740706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a05232af60 msgr2=0x55a05232e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:00.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.037+0000 7f54d4a91b80 1 --2- 172.21.15.7:0/773740706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a05232af60 0x55a05232e500 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f54bc00d6a0 tx=0x7f54bc004b60 comp rx=0 tx=0).stop 2024-06-10T01:28:00.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.037+0000 7f54d4a91b80 1 -- 172.21.15.7:0/773740706 shutdown_connections 2024-06-10T01:28:00.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.037+0000 7f54d4a91b80 1 --2- 172.21.15.7:0/773740706 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55a0523609c0 0x55a052362db0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.037+0000 7f54d4a91b80 1 --2- 172.21.15.7:0/773740706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a05232af60 0x55a05232e500 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.037+0000 7f54d4a91b80 1 --2- 172.21.15.7:0/773740706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052324f20 0x55a052330cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.037+0000 7f54d4a91b80 1 --2- 172.21.15.7:0/773740706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a05232a640 0x55a05232dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.037+0000 7f54d4a91b80 1 -- 172.21.15.7:0/773740706 >> 172.21.15.7:0/773740706 conn(0x55a05224fcc0 msgr2=0x55a0523239d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:00.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.038+0000 7f54a2ffd700 0 client.20661 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:28:00.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.038+0000 7f54d4a91b80 1 -- 172.21.15.7:0/773740706 shutdown_connections 2024-06-10T01:28:00.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.038+0000 7f54d4a91b80 1 -- 172.21.15.7:0/773740706 wait complete. 2024-06-10T01:28:00.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.039+0000 7f54d4a91b80 20 client.20661 trim_cache size 0 max 16384 2024-06-10T01:28:00.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.069+0000 7f54d4a91b80 1 Processor -- start 2024-06-10T01:28:00.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.069+0000 7f54d4a91b80 1 -- start start 2024-06-10T01:28:00.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.069+0000 7f54d4a91b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a052324f20 0x55a0523249e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:00.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.069+0000 7f54d4a91b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a05232a640 0x55a05231f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:00.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.070+0000 7f54d4a91b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a05232af60 0x55a05231f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:00.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.070+0000 7f54d4a91b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f54a4011430 con 0x55a052324f20 2024-06-10T01:28:00.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.070+0000 7f54d4a91b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f54a400ecc0 con 0x55a05232a640 2024-06-10T01:28:00.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.070+0000 7f54d4a91b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f54a400cb10 con 0x55a05232af60 2024-06-10T01:28:00.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.070+0000 7f54c37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a052324f20 0x55a0523249e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:00.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.070+0000 7f54c37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a052324f20 0x55a0523249e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41548/0 (socket says 172.21.15.7:41548) 2024-06-10T01:28:00.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.070+0000 7f54c37fe700 1 -- 172.21.15.7:0/2040582141 learned_addr learned my addr 172.21.15.7:0/2040582141 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:00.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.070+0000 7f54c3fff700 1 --2- 172.21.15.7:0/2040582141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a05232af60 0x55a05231f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:00.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.071+0000 7f54c37fe700 1 -- 172.21.15.7:0/2040582141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a05232af60 msgr2=0x55a05231f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:00.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.071+0000 7f54c37fe700 1 --2- 172.21.15.7:0/2040582141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a05232af60 0x55a05231f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.071+0000 7f54c37fe700 1 -- 172.21.15.7:0/2040582141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a05232a640 msgr2=0x55a05231f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:00.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.071+0000 7f54c37fe700 1 --2- 172.21.15.7:0/2040582141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a05232a640 0x55a05231f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.071+0000 7f54c37fe700 1 -- 172.21.15.7:0/2040582141 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a05232dc30 con 0x55a052324f20 2024-06-10T01:28:00.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.071+0000 7f54c37fe700 1 --2- 172.21.15.7:0/2040582141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a052324f20 0x55a0523249e0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f54b401acf0 tx=0x7f54b4005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51b9b0f1a56da02 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:00.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.072+0000 7f54b97fa700 1 -- 172.21.15.7:0/2040582141 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b40050c0 con 0x55a052324f20 2024-06-10T01:28:00.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.072+0000 7f54b97fa700 1 -- 172.21.15.7:0/2040582141 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54b4003e40 con 0x55a052324f20 2024-06-10T01:28:00.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.072+0000 7f54b97fa700 1 -- 172.21.15.7:0/2040582141 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b400d430 con 0x55a052324f20 2024-06-10T01:28:00.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.073+0000 7f54d4a91b80 1 -- 172.21.15.7:0/2040582141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a052324f20 msgr2=0x55a0523249e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:00.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.073+0000 7f54d4a91b80 1 --2- 172.21.15.7:0/2040582141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a052324f20 0x55a0523249e0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f54b401acf0 tx=0x7f54b4005a90 comp rx=0 tx=0).stop 2024-06-10T01:28:00.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.073+0000 7f54d4a91b80 1 -- 172.21.15.7:0/2040582141 shutdown_connections 2024-06-10T01:28:00.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.073+0000 7f54d4a91b80 1 --2- 172.21.15.7:0/2040582141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a05232af60 0x55a05231f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.073+0000 7f54d4a91b80 1 --2- 172.21.15.7:0/2040582141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a052324f20 0x55a0523249e0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.073+0000 7f54d4a91b80 1 --2- 172.21.15.7:0/2040582141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a05232a640 0x55a05231f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.073+0000 7f54d4a91b80 1 -- 172.21.15.7:0/2040582141 >> 172.21.15.7:0/2040582141 conn(0x55a05224fcc0 msgr2=0x55a052328190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:00.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.073+0000 7f54d4a91b80 1 -- 172.21.15.7:0/2040582141 shutdown_connections 2024-06-10T01:28:00.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.073+0000 7f54d4a91b80 1 -- 172.21.15.7:0/2040582141 wait complete. 2024-06-10T01:28:00.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.074+0000 7f54d4a91b80 1 -- start start 2024-06-10T01:28:00.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.074+0000 7f54d4a91b80 1 Processor -- start 2024-06-10T01:28:00.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.075+0000 7f54d4a91b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052324f20 0x55a052337090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:00.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.075+0000 7f54d4a91b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a05232a640 0x55a0523375d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:00.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.075+0000 7f54d4a91b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a05232af60 0x55a052367a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:00.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.075+0000 7f54d4a91b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f54a400d180 con 0x55a05232a640 2024-06-10T01:28:00.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.075+0000 7f54d4a91b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f54a400e5f0 con 0x55a052324f20 2024-06-10T01:28:00.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.075+0000 7f54d4a91b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f54a400d850 con 0x55a05232af60 2024-06-10T01:28:00.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.076+0000 7f54d4a91b80 10 client.0 _ll_register_callbacks cb 0x55a052210920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:28:00.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.076+0000 7f54c37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052324f20 0x55a052337090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:00.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.076+0000 7f54c37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052324f20 0x55a052337090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:35456/0 (socket says 172.21.15.7:35456) 2024-06-10T01:28:00.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.076+0000 7f54c37fe700 1 -- 172.21.15.7:0/3874226250 learned_addr learned my addr 172.21.15.7:0/3874226250 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:00.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.076+0000 7f54bbfff700 1 --2- 172.21.15.7:0/3874226250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a05232a640 0x55a0523375d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:00.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.076+0000 7f54c3fff700 1 --2- 172.21.15.7:0/3874226250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a05232af60 0x55a052367a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:00.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.076+0000 7f54bbfff700 1 -- 172.21.15.7:0/3874226250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a05232af60 msgr2=0x55a052367a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:00.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.077+0000 7f54bbfff700 1 --2- 172.21.15.7:0/3874226250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a05232af60 0x55a052367a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.077+0000 7f54bbfff700 1 -- 172.21.15.7:0/3874226250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052324f20 msgr2=0x55a052337090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:00.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.077+0000 7f54bbfff700 1 --2- 172.21.15.7:0/3874226250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052324f20 0x55a052337090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.077+0000 7f54bbfff700 1 -- 172.21.15.7:0/3874226250 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0523682b0 con 0x55a05232a640 2024-06-10T01:28:00.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.077+0000 7f54bbfff700 1 --2- 172.21.15.7:0/3874226250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a05232a640 0x55a0523375d0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x55a05232f590 tx=0x7f54ac09bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab1319dc39d433d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:00.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.077+0000 7f54a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:28:00.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.077+0000 7f54a2ffd700 1 -- 172.21.15.7:0/3874226250 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac09ba60 con 0x55a05232a640 2024-06-10T01:28:00.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.077+0000 7f54a2ffd700 1 -- 172.21.15.7:0/3874226250 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f54ac09aeb0 con 0x55a05232a640 2024-06-10T01:28:00.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.078+0000 7f54d4a91b80 1 -- 172.21.15.7:0/3874226250 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a05227e290 con 0x55a05232a640 2024-06-10T01:28:00.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.078+0000 7f54a2ffd700 1 -- 172.21.15.7:0/3874226250 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f54ac09fe10 con 0x55a05232a640 2024-06-10T01:28:00.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.078+0000 7f54d4a91b80 10 client.20667 fetch_fsmap learned FSMap version 55 2024-06-10T01:28:00.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.078+0000 7f54d4a91b80 1 -- 172.21.15.7:0/3874226250 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55a0523cb0f0 con 0x55a05232a640 2024-06-10T01:28:00.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.079+0000 7f54a2ffd700 1 -- 172.21.15.7:0/3874226250 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54ac0a46d0 con 0x55a05232a640 2024-06-10T01:28:00.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.079+0000 7f54a2ffd700 1 -- 172.21.15.7:0/3874226250 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f54ac09ab20 con 0x55a05232a640 2024-06-10T01:28:00.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.079+0000 7f54d4a91b80 10 client.20667 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:28:00.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.079+0000 7f54d4a91b80 10 client.20667 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:28:00.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.079+0000 7f54d4a91b80 20 client.20667 populate_metadata read hostname 'smithi007' 2024-06-10T01:28:00.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.079+0000 7f54d4a91b80 1 --2- 172.21.15.7:0/3874226250 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55a0523cb700 0x55a0523cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:00.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.080+0000 7f54d4a91b80 4 client.20667 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:28:00.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.080+0000 7f54d4a91b80 1 -- 172.21.15.7:0/3874226250 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a05227e290 con 0x55a0523cb700 2024-06-10T01:28:00.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.086+0000 7f54c37fe700 1 --2- 172.21.15.7:0/3874226250 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55a0523cb700 0x55a0523cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:00.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.089+0000 7f54c37fe700 1 --2- 172.21.15.7:0/3874226250 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55a0523cb700 0x55a0523cdb20 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c08ad0cab0fb5006 server_cookie=287a192a2e2c70f5 in_seq=0 out_seq=0 2024-06-10T01:28:00.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.089+0000 7f54a2ffd700 10 client.20667 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:28:00.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.090+0000 7f54a2ffd700 1 -- 172.21.15.7:0/3874226250 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a05227e290 con 0x55a0523cb700 2024-06-10T01:28:00.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.090+0000 7f54a2ffd700 10 client.20667 handle_command_reply: tid=0 2024-06-10T01:28:00.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.090+0000 7f54d4a91b80 1 client.20667 shutdown 2024-06-10T01:28:00.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.092+0000 7f54d4a91b80 1 -- 172.21.15.7:0/3874226250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a05232a640 msgr2=0x55a0523375d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:00.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.092+0000 7f54d4a91b80 1 --2- 172.21.15.7:0/3874226250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a05232a640 0x55a0523375d0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x55a05232f590 tx=0x7f54ac09bfa0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.093+0000 7f54d4a91b80 1 -- 172.21.15.7:0/3874226250 shutdown_connections 2024-06-10T01:28:00.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.093+0000 7f54d4a91b80 1 --2- 172.21.15.7:0/3874226250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a05232af60 0x55a052367a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.093+0000 7f54d4a91b80 1 --2- 172.21.15.7:0/3874226250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052324f20 0x55a052337090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.093+0000 7f54d4a91b80 1 --2- 172.21.15.7:0/3874226250 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55a0523cb700 0x55a0523cdb20 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.093+0000 7f54a2ffd700 0 client.20667 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:28:00.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.093+0000 7f54d4a91b80 1 --2- 172.21.15.7:0/3874226250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a05232a640 0x55a0523375d0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.093+0000 7f54d4a91b80 1 -- 172.21.15.7:0/3874226250 >> 172.21.15.7:0/3874226250 conn(0x55a05224fcc0 msgr2=0x55a05231f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:00.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.094+0000 7f54d4a91b80 1 -- 172.21.15.7:0/3874226250 shutdown_connections 2024-06-10T01:28:00.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.094+0000 7f54d4a91b80 1 -- 172.21.15.7:0/3874226250 wait complete. 2024-06-10T01:28:00.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.096+0000 7f54d4a91b80 20 client.20667 trim_cache size 0 max 16384 2024-06-10T01:28:00.092 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:28:00.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.100+0000 7f54ca20e700 1 -- 172.21.15.7:0/2351858341 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f54b00a2cd0 msgr2=0x7f54b00a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:00.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.100+0000 7f54ca20e700 1 --2- 172.21.15.7:0/2351858341 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f54b00a2cd0 0x7f54b00a5190 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f54c406bc00 tx=0x7f54bc009250 comp rx=0 tx=0).stop 2024-06-10T01:28:00.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.100+0000 7f54ca20e700 1 -- 172.21.15.7:0/2351858341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54c4110c70 msgr2=0x7f54c4117330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:00.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.100+0000 7f54ca20e700 1 --2- 172.21.15.7:0/2351858341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54c4110c70 0x7f54c4117330 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f54ac00e9a0 tx=0x7f54ac00ed60 comp rx=0 tx=0).stop 2024-06-10T01:28:00.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.101+0000 7f54ca20e700 1 -- 172.21.15.7:0/2351858341 shutdown_connections 2024-06-10T01:28:00.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.101+0000 7f54ca20e700 1 --2- 172.21.15.7:0/2351858341 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f54b00a2cd0 0x7f54b00a5190 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.101+0000 7f54ca20e700 1 --2- 172.21.15.7:0/2351858341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54c411bc70 0x7f54c4117870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.101+0000 7f54ca20e700 1 --2- 172.21.15.7:0/2351858341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54c41099d0 0x7f54c411b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.101+0000 7f54ca20e700 1 --2- 172.21.15.7:0/2351858341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54c4110c70 0x7f54c4117330 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:00.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.101+0000 7f54ca20e700 1 -- 172.21.15.7:0/2351858341 >> 172.21.15.7:0/2351858341 conn(0x7f54c406d600 msgr2=0x7f54c411d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:00.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.101+0000 7f54ca20e700 1 -- 172.21.15.7:0/2351858341 shutdown_connections 2024-06-10T01:28:00.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:00.101+0000 7f54ca20e700 1 -- 172.21.15.7:0/2351858341 wait complete. 2024-06-10T01:28:00.554 INFO:tasks.workunit.client.0.smithi007.stdout: 10 211989 23.27 MB/sec execute 397 sec latency 86.149 ms 2024-06-10T01:28:00.986 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:28:01.391 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:28:01.555 INFO:tasks.workunit.client.0.smithi007.stdout: 10 212619 23.30 MB/sec execute 398 sec latency 112.042 ms 2024-06-10T01:28:01.788 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:01 smithi007 ceph-mon[32398]: pgmap v1094: 129 pgs: 129 active+clean; 682 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.3 MiB/s wr, 438 op/s 2024-06-10T01:28:01.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:01 smithi078 ceph-mon[36683]: pgmap v1094: 129 pgs: 129 active+clean; 682 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.3 MiB/s wr, 438 op/s 2024-06-10T01:28:01.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:01 smithi175 ceph-mon[36636]: pgmap v1094: 129 pgs: 129 active+clean; 682 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.3 MiB/s wr, 438 op/s 2024-06-10T01:28:02.555 INFO:tasks.workunit.client.0.smithi007.stdout: 10 213065 23.30 MB/sec execute 399 sec latency 127.757 ms 2024-06-10T01:28:03.555 INFO:tasks.workunit.client.0.smithi007.stdout: 10 213574 23.31 MB/sec execute 400 sec latency 112.712 ms 2024-06-10T01:28:03.728 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:03 smithi007 ceph-mon[32398]: pgmap v1095: 129 pgs: 129 active+clean; 674 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.7 MiB/s wr, 568 op/s 2024-06-10T01:28:03.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.745+0000 7f18b30cb700 1 -- 172.21.15.7:0/538901292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18ac1084f0 msgr2=0x7f18ac1088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:03.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.745+0000 7f18b30cb700 1 --2- 172.21.15.7:0/538901292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18ac1084f0 0x7f18ac1088d0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f18a8009990 tx=0x7f18a8009ca0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.745+0000 7f18b30cb700 1 -- 172.21.15.7:0/538901292 shutdown_connections 2024-06-10T01:28:03.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.745+0000 7f18b30cb700 1 --2- 172.21.15.7:0/538901292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18ac06b3c0 0x7f18ac10f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.745+0000 7f18b30cb700 1 --2- 172.21.15.7:0/538901292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18ac1084f0 0x7f18ac1088d0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.745+0000 7f18b30cb700 1 --2- 172.21.15.7:0/538901292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18ac06aa00 0x7f18ac06ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.745+0000 7f18b30cb700 1 -- 172.21.15.7:0/538901292 >> 172.21.15.7:0/538901292 conn(0x7f18ac078300 msgr2=0x7f18ac078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:03.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.745+0000 7f18b30cb700 1 -- 172.21.15.7:0/538901292 shutdown_connections 2024-06-10T01:28:03.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.745+0000 7f18b30cb700 1 -- 172.21.15.7:0/538901292 wait complete. 2024-06-10T01:28:03.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.746+0000 7f18b30cb700 1 Processor -- start 2024-06-10T01:28:03.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.746+0000 7f18b30cb700 1 -- start start 2024-06-10T01:28:03.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.746+0000 7f18b30cb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18ac06aa00 0x7f18ac19ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:03.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.747+0000 7f18b30cb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18ac06b3c0 0x7f18ac19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:03.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.747+0000 7f18b30cb700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18ac1084f0 0x7f18ac19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:03.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.747+0000 7f18b30cb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac102c00 con 0x7f18ac06aa00 2024-06-10T01:28:03.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.747+0000 7f18b30cb700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac102a80 con 0x7f18ac06b3c0 2024-06-10T01:28:03.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.747+0000 7f18b30cb700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac102d80 con 0x7f18ac1084f0 2024-06-10T01:28:03.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.747+0000 7f18b0e67700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18ac06aa00 0x7f18ac19ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:03.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.747+0000 7f18b0e67700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18ac06aa00 0x7f18ac19ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47396/0 (socket says 172.21.15.7:47396) 2024-06-10T01:28:03.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.747+0000 7f18b0e67700 1 -- 172.21.15.7:0/1325801356 learned_addr learned my addr 172.21.15.7:0/1325801356 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:03.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.747+0000 7f18b1668700 1 --2- 172.21.15.7:0/1325801356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18ac1084f0 0x7f18ac19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:03.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.747+0000 7f18a3fff700 1 --2- 172.21.15.7:0/1325801356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18ac06b3c0 0x7f18ac19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:03.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.747+0000 7f18b0e67700 1 -- 172.21.15.7:0/1325801356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18ac1084f0 msgr2=0x7f18ac19f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:03.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.747+0000 7f18b0e67700 1 --2- 172.21.15.7:0/1325801356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18ac1084f0 0x7f18ac19f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.747+0000 7f18b0e67700 1 -- 172.21.15.7:0/1325801356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18ac06b3c0 msgr2=0x7f18ac19b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.747+0000 7f18b0e67700 1 --2- 172.21.15.7:0/1325801356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18ac06b3c0 0x7f18ac19b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.747+0000 7f18b0e67700 1 -- 172.21.15.7:0/1325801356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18ac19fe80 con 0x7f18ac06aa00 2024-06-10T01:28:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.747+0000 7f18a3fff700 1 --2- 172.21.15.7:0/1325801356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18ac06b3c0 0x7f18ac19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:28:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.747+0000 7f18b0e67700 1 --2- 172.21.15.7:0/1325801356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18ac06aa00 0x7f18ac19ae70 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f18a8000c00 tx=0x7f18a800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.747+0000 7f18a1ffb700 1 -- 172.21.15.7:0/1325801356 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8027070 con 0x7f18ac06aa00 2024-06-10T01:28:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.747+0000 7f18b30cb700 1 -- 172.21.15.7:0/1325801356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18ac1a0110 con 0x7f18ac06aa00 2024-06-10T01:28:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.747+0000 7f18a1ffb700 1 -- 172.21.15.7:0/1325801356 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18a8003d70 con 0x7f18ac06aa00 2024-06-10T01:28:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.747+0000 7f18a1ffb700 1 -- 172.21.15.7:0/1325801356 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a800f460 con 0x7f18ac06aa00 2024-06-10T01:28:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.748+0000 7f18b30cb700 1 -- 172.21.15.7:0/1325801356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18ac1a79f0 con 0x7f18ac06aa00 2024-06-10T01:28:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.750+0000 7f18a1ffb700 1 -- 172.21.15.7:0/1325801356 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f18a802a070 con 0x7f18ac06aa00 2024-06-10T01:28:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.751+0000 7f18a1ffb700 1 --2- 172.21.15.7:0/1325801356 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f18940a2c20 0x7f18940a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.751+0000 7f18a1ffb700 1 -- 172.21.15.7:0/1325801356 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f18a80d00c0 con 0x7f18ac06aa00 2024-06-10T01:28:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.751+0000 7f18a3fff700 1 --2- 172.21.15.7:0/1325801356 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f18940a2c20 0x7f18940a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:03.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.752+0000 7f18a3fff700 1 --2- 172.21.15.7:0/1325801356 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f18940a2c20 0x7f18940a50e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f18ac078720 tx=0x7f189c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:03.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.752+0000 7f18bd94eb80 1 Processor -- start 2024-06-10T01:28:03.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.752+0000 7f18bd94eb80 1 -- start start 2024-06-10T01:28:03.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.752+0000 7f18bd94eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555908bf2f60 0x555908bf3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:03.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.752+0000 7f18bd94eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555908bf3880 0x555908bed260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:03.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.752+0000 7f18bd94eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555908bed7a0 0x555908bedbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:03.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.752+0000 7f18bd94eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555908adff60 con 0x555908bf3880 2024-06-10T01:28:03.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.752+0000 7f18bd94eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555908b3e850 con 0x555908bed7a0 2024-06-10T01:28:03.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.752+0000 7f18bd94eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555908aa6460 con 0x555908bf2f60 2024-06-10T01:28:03.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.752+0000 7f18a3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555908bf3880 0x555908bed260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:03.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.752+0000 7f18b1668700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555908bed7a0 0x555908bedbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:03.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.752+0000 7f18b0e67700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555908bf2f60 0x555908bf3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.752+0000 7f18a3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555908bf3880 0x555908bed260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47410/0 (socket says 172.21.15.7:47410) 2024-06-10T01:28:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.752+0000 7f18b1668700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555908bed7a0 0x555908bedbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:54206/0 (socket says 172.21.15.7:54206) 2024-06-10T01:28:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.752+0000 7f18a3fff700 1 -- 172.21.15.7:0/1176969905 learned_addr learned my addr 172.21.15.7:0/1176969905 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.753+0000 7f18b0e67700 1 -- 172.21.15.7:0/1176969905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555908bed7a0 msgr2=0x555908bedbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.753+0000 7f18b0e67700 1 --2- 172.21.15.7:0/1176969905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555908bed7a0 0x555908bedbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.753+0000 7f18b0e67700 1 -- 172.21.15.7:0/1176969905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555908bf3880 msgr2=0x555908bed260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.753+0000 7f18b0e67700 1 --2- 172.21.15.7:0/1176969905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555908bf3880 0x555908bed260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.753+0000 7f18b0e67700 1 -- 172.21.15.7:0/1176969905 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555908a8d8d0 con 0x555908bf2f60 2024-06-10T01:28:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.753+0000 7f18b1668700 1 --2- 172.21.15.7:0/1176969905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555908bed7a0 0x555908bedbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:28:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.753+0000 7f18a3fff700 1 --2- 172.21.15.7:0/1176969905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555908bf3880 0x555908bed260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:28:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.753+0000 7f18b0e67700 1 --2- 172.21.15.7:0/1176969905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555908bf2f60 0x555908bf3340 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f18a80cd890 tx=0x7f18a800bfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=121f6e42d2827c9c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.753+0000 7f1892ffd700 1 -- 172.21.15.7:0/1176969905 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a809c630 con 0x555908bf2f60 2024-06-10T01:28:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.753+0000 7f1892ffd700 1 -- 172.21.15.7:0/1176969905 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18a80a09e0 con 0x555908bf2f60 2024-06-10T01:28:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.753+0000 7f1892ffd700 1 -- 172.21.15.7:0/1176969905 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a809caf0 con 0x555908bf2f60 2024-06-10T01:28:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.754+0000 7f18bd94eb80 1 -- 172.21.15.7:0/1176969905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555908bf2f60 msgr2=0x555908bf3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:03.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.754+0000 7f18bd94eb80 1 --2- 172.21.15.7:0/1176969905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555908bf2f60 0x555908bf3340 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f18a80cd890 tx=0x7f18a800bfa0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.754+0000 7f18bd94eb80 1 -- 172.21.15.7:0/1176969905 shutdown_connections 2024-06-10T01:28:03.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.754+0000 7f18bd94eb80 1 --2- 172.21.15.7:0/1176969905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555908bed7a0 0x555908bedbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.754+0000 7f18bd94eb80 1 --2- 172.21.15.7:0/1176969905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555908bf2f60 0x555908bf3340 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.754+0000 7f18bd94eb80 1 --2- 172.21.15.7:0/1176969905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555908bf3880 0x555908bed260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.754+0000 7f18bd94eb80 1 -- 172.21.15.7:0/1176969905 >> 172.21.15.7:0/1176969905 conn(0x555908b1a080 msgr2=0x555908be8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:03.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.754+0000 7f18bd94eb80 1 -- 172.21.15.7:0/1176969905 shutdown_connections 2024-06-10T01:28:03.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.754+0000 7f18bd94eb80 1 -- 172.21.15.7:0/1176969905 wait complete. 2024-06-10T01:28:03.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.755+0000 7f18bd94eb80 1 -- start start 2024-06-10T01:28:03.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.755+0000 7f18bd94eb80 1 Processor -- start 2024-06-10T01:28:03.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.755+0000 7f18bd94eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555908bed7a0 0x555908bf9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:03.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.755+0000 7f18bd94eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555908bf2f60 0x555908bf6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.755+0000 7f18bd94eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555908bf3880 0x555908bf6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.755+0000 7f18bd94eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555908adefc0 con 0x555908bed7a0 2024-06-10T01:28:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.755+0000 7f18bd94eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555908b3a810 con 0x555908bf2f60 2024-06-10T01:28:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.755+0000 7f18bd94eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555908bfa010 con 0x555908bf3880 2024-06-10T01:28:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.756+0000 7f18b0e67700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555908bed7a0 0x555908bf9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.756+0000 7f18a3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555908bf2f60 0x555908bf6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.756+0000 7f18b0e67700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555908bed7a0 0x555908bf9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47422/0 (socket says 172.21.15.7:47422) 2024-06-10T01:28:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.756+0000 7f18b0e67700 1 -- 172.21.15.7:0/4033273159 learned_addr learned my addr 172.21.15.7:0/4033273159 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.756+0000 7f18b1668700 1 --2- 172.21.15.7:0/4033273159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555908bf3880 0x555908bf6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.756+0000 7f18bd94eb80 10 client.0 _ll_register_callbacks cb 0x555908ad45b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:28:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.756+0000 7f18a3fff700 1 -- 172.21.15.7:0/4033273159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555908bf3880 msgr2=0x555908bf6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.756+0000 7f18a3fff700 1 --2- 172.21.15.7:0/4033273159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555908bf3880 0x555908bf6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.756+0000 7f18a3fff700 1 -- 172.21.15.7:0/4033273159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555908bed7a0 msgr2=0x555908bf9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.756+0000 7f18a3fff700 1 --2- 172.21.15.7:0/4033273159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555908bed7a0 0x555908bf9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.756+0000 7f18a3fff700 1 -- 172.21.15.7:0/4033273159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555908b04750 con 0x555908bf2f60 2024-06-10T01:28:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.756+0000 7f18b0e67700 1 --2- 172.21.15.7:0/4033273159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555908bed7a0 0x555908bf9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:28:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.756+0000 7f18a3fff700 1 --2- 172.21.15.7:0/4033273159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555908bf2f60 0x555908bf6910 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f189c009b70 tx=0x7f189c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6090073c1d5776a3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.756+0000 7f188bfff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:28:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.757+0000 7f188bfff700 1 -- 172.21.15.7:0/4033273159 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c012650 con 0x555908bf2f60 2024-06-10T01:28:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.757+0000 7f18bd94eb80 1 -- 172.21.15.7:0/4033273159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18a80a09e0 con 0x555908bf2f60 2024-06-10T01:28:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.757+0000 7f188bfff700 1 -- 172.21.15.7:0/4033273159 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f189c0127f0 con 0x555908bf2f60 2024-06-10T01:28:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.757+0000 7f188bfff700 1 -- 172.21.15.7:0/4033273159 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f189c011e90 con 0x555908bf2f60 2024-06-10T01:28:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.757+0000 7f18bd94eb80 10 client.29876 fetch_fsmap learned FSMap version 55 2024-06-10T01:28:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.757+0000 7f18bd94eb80 1 -- 172.21.15.7:0/4033273159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x555908c04740 con 0x555908bf2f60 2024-06-10T01:28:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.757+0000 7f188bfff700 1 -- 172.21.15.7:0/4033273159 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18a80a09e0 con 0x555908bf2f60 2024-06-10T01:28:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.758+0000 7f188bfff700 1 -- 172.21.15.7:0/4033273159 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f189c028ba0 con 0x555908bf2f60 2024-06-10T01:28:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.758+0000 7f18bd94eb80 10 client.29876 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:28:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.758+0000 7f18bd94eb80 10 client.29876 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:28:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.758+0000 7f18bd94eb80 20 client.29876 populate_metadata read hostname 'smithi007' 2024-06-10T01:28:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.758+0000 7f18bd94eb80 1 --2- 172.21.15.7:0/4033273159 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x555908c06d80 0x555908c09170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.758+0000 7f18bd94eb80 4 client.29876 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:28:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.758+0000 7f18bd94eb80 1 -- 172.21.15.7:0/4033273159 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555908b4bd50 con 0x555908c06d80 2024-06-10T01:28:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.758+0000 7f18b0e67700 1 --2- 172.21.15.7:0/4033273159 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x555908c06d80 0x555908c09170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.758+0000 7f18b0e67700 1 --2- 172.21.15.7:0/4033273159 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x555908c06d80 0x555908c09170 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c5dba9ab1954e7df server_cookie=ad2e26312ce4dbfe in_seq=0 out_seq=0 2024-06-10T01:28:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.758+0000 7f188bfff700 10 client.29876 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:28:03.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.763+0000 7f188bfff700 1 -- 172.21.15.7:0/4033273159 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555908b4bd50 con 0x555908c06d80 2024-06-10T01:28:03.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.763+0000 7f188bfff700 10 client.29876 handle_command_reply: tid=0 2024-06-10T01:28:03.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.763+0000 7f18bd94eb80 1 client.29876 shutdown 2024-06-10T01:28:03.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.764+0000 7f18bd94eb80 1 -- 172.21.15.7:0/4033273159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555908bf2f60 msgr2=0x555908bf6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:03.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.764+0000 7f18bd94eb80 1 --2- 172.21.15.7:0/4033273159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555908bf2f60 0x555908bf6910 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f189c009b70 tx=0x7f189c0047c0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.764+0000 7f18bd94eb80 1 -- 172.21.15.7:0/4033273159 shutdown_connections 2024-06-10T01:28:03.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.764+0000 7f18bd94eb80 1 --2- 172.21.15.7:0/4033273159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555908bf3880 0x555908bf6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.764+0000 7f18bd94eb80 1 --2- 172.21.15.7:0/4033273159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555908bed7a0 0x555908bf9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.764+0000 7f18bd94eb80 1 --2- 172.21.15.7:0/4033273159 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x555908c06d80 0x555908c09170 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.764+0000 7f18bd94eb80 1 --2- 172.21.15.7:0/4033273159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555908bf2f60 0x555908bf6910 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.764+0000 7f188bfff700 0 client.29876 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:28:03.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.764+0000 7f18bd94eb80 1 -- 172.21.15.7:0/4033273159 >> 172.21.15.7:0/4033273159 conn(0x555908b1a080 msgr2=0x555908be7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:03.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.764+0000 7f18bd94eb80 1 -- 172.21.15.7:0/4033273159 shutdown_connections 2024-06-10T01:28:03.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.764+0000 7f18bd94eb80 1 -- 172.21.15.7:0/4033273159 wait complete. 2024-06-10T01:28:03.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.764+0000 7f18bd94eb80 20 client.29876 trim_cache size 0 max 16384 2024-06-10T01:28:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.781+0000 7f18bd94eb80 1 Processor -- start 2024-06-10T01:28:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.781+0000 7f18bd94eb80 1 -- start start 2024-06-10T01:28:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.781+0000 7f18bd94eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555908bed7a0 0x555908be7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:03.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.781+0000 7f18bd94eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555908bf2f60 0x555908be83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.781+0000 7f18bd94eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555908bf3880 0x555908c0ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.781+0000 7f18bd94eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac1b2c60 con 0x555908bf2f60 2024-06-10T01:28:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.781+0000 7f18bd94eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac1b04f0 con 0x555908bf3880 2024-06-10T01:28:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.781+0000 7f18bd94eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac1ae340 con 0x555908bed7a0 2024-06-10T01:28:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.781+0000 7f18a3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555908bf2f60 0x555908be83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.781+0000 7f18a3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555908bf2f60 0x555908be83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47434/0 (socket says 172.21.15.7:47434) 2024-06-10T01:28:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.781+0000 7f18a3fff700 1 -- 172.21.15.7:0/3210737026 learned_addr learned my addr 172.21.15.7:0/3210737026 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.781+0000 7f18b0e67700 1 --2- 172.21.15.7:0/3210737026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555908bed7a0 0x555908be7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.781+0000 7f18b1668700 1 --2- 172.21.15.7:0/3210737026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555908bf3880 0x555908c0ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.782+0000 7f18a3fff700 1 -- 172.21.15.7:0/3210737026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555908bed7a0 msgr2=0x555908be7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.782+0000 7f18a3fff700 1 --2- 172.21.15.7:0/3210737026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555908bed7a0 0x555908be7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.782+0000 7f18a3fff700 1 -- 172.21.15.7:0/3210737026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555908bf3880 msgr2=0x555908c0ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.782+0000 7f18a3fff700 1 --2- 172.21.15.7:0/3210737026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555908bf3880 0x555908c0ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.782+0000 7f18a3fff700 1 -- 172.21.15.7:0/3210737026 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555908bed5a0 con 0x555908bf2f60 2024-06-10T01:28:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.782+0000 7f18b1668700 1 --2- 172.21.15.7:0/3210737026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555908bf3880 0x555908c0ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:28:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.782+0000 7f18a3fff700 1 --2- 172.21.15.7:0/3210737026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555908bf2f60 0x555908be83c0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f189c020450 tx=0x7f189c020830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4c8f2f456a3621 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.782+0000 7f18927fc700 1 -- 172.21.15.7:0/3210737026 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c009e70 con 0x555908bf2f60 2024-06-10T01:28:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.782+0000 7f18927fc700 1 -- 172.21.15.7:0/3210737026 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f189c016070 con 0x555908bf2f60 2024-06-10T01:28:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.782+0000 7f18927fc700 1 -- 172.21.15.7:0/3210737026 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c012a50 con 0x555908bf2f60 2024-06-10T01:28:03.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.783+0000 7f18bd94eb80 1 -- 172.21.15.7:0/3210737026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555908bf2f60 msgr2=0x555908be83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:03.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.783+0000 7f18bd94eb80 1 --2- 172.21.15.7:0/3210737026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555908bf2f60 0x555908be83c0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f189c020450 tx=0x7f189c020830 comp rx=0 tx=0).stop 2024-06-10T01:28:03.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.783+0000 7f18bd94eb80 1 -- 172.21.15.7:0/3210737026 shutdown_connections 2024-06-10T01:28:03.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.783+0000 7f18bd94eb80 1 --2- 172.21.15.7:0/3210737026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555908bf3880 0x555908c0ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.783+0000 7f18bd94eb80 1 --2- 172.21.15.7:0/3210737026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555908bed7a0 0x555908be7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.783+0000 7f18bd94eb80 1 --2- 172.21.15.7:0/3210737026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555908bf2f60 0x555908be83c0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.783+0000 7f18bd94eb80 1 -- 172.21.15.7:0/3210737026 >> 172.21.15.7:0/3210737026 conn(0x555908b1a080 msgr2=0x555908bf5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.783+0000 7f18bd94eb80 1 -- 172.21.15.7:0/3210737026 shutdown_connections 2024-06-10T01:28:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.783+0000 7f18bd94eb80 1 -- 172.21.15.7:0/3210737026 wait complete. 2024-06-10T01:28:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.784+0000 7f18bd94eb80 1 -- start start 2024-06-10T01:28:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.784+0000 7f18bd94eb80 1 Processor -- start 2024-06-10T01:28:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.784+0000 7f18bd94eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555908bed7a0 0x555908bff260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.784+0000 7f18bd94eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555908bf2f60 0x555908bff7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.784+0000 7f18bd94eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555908bf3880 0x555908c0da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.784+0000 7f18bd94eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac1ae9b0 con 0x555908bf2f60 2024-06-10T01:28:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.784+0000 7f18bd94eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac1afe20 con 0x555908bf3880 2024-06-10T01:28:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.784+0000 7f18bd94eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac1af080 con 0x555908bed7a0 2024-06-10T01:28:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.784+0000 7f18bd94eb80 10 client.0 _ll_register_callbacks cb 0x555908ad45b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:28:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.785+0000 7f18a3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555908bf2f60 0x555908bff7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.785+0000 7f18a3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555908bf2f60 0x555908bff7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47450/0 (socket says 172.21.15.7:47450) 2024-06-10T01:28:03.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.785+0000 7f18a3fff700 1 -- 172.21.15.7:0/3704442635 learned_addr learned my addr 172.21.15.7:0/3704442635 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:03.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.785+0000 7f18b0e67700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555908bed7a0 0x555908bff260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:03.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.785+0000 7f18b1668700 1 --2- 172.21.15.7:0/3704442635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555908bf3880 0x555908c0da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:03.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.785+0000 7f18a3fff700 1 -- 172.21.15.7:0/3704442635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555908bed7a0 msgr2=0x555908bff260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:03.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.785+0000 7f18a3fff700 1 --2- 172.21.15.7:0/3704442635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555908bed7a0 0x555908bff260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.785+0000 7f18a3fff700 1 -- 172.21.15.7:0/3704442635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555908bf3880 msgr2=0x555908c0da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:03.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.785+0000 7f18a3fff700 1 --2- 172.21.15.7:0/3704442635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555908bf3880 0x555908c0da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.785+0000 7f18a3fff700 1 -- 172.21.15.7:0/3704442635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555908c0df60 con 0x555908bf2f60 2024-06-10T01:28:03.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.785+0000 7f18b1668700 1 --2- 172.21.15.7:0/3704442635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555908bf3880 0x555908c0da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:28:03.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.785+0000 7f18a3fff700 1 --2- 172.21.15.7:0/3704442635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555908bf2f60 0x555908bff7a0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f189c029fd0 tx=0x7f189c020ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f30a8478a4858b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:03.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.785+0000 7f188bfff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:28:03.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.786+0000 7f188bfff700 1 -- 172.21.15.7:0/3704442635 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c009e70 con 0x555908bf2f60 2024-06-10T01:28:03.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.786+0000 7f188bfff700 1 -- 172.21.15.7:0/3704442635 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f189c012a00 con 0x555908bf2f60 2024-06-10T01:28:03.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.786+0000 7f18bd94eb80 1 -- 172.21.15.7:0/3704442635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f189c016070 con 0x555908bf2f60 2024-06-10T01:28:03.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.786+0000 7f188bfff700 1 -- 172.21.15.7:0/3704442635 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f189c012c00 con 0x555908bf2f60 2024-06-10T01:28:03.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.786+0000 7f18bd94eb80 10 client.20685 fetch_fsmap learned FSMap version 55 2024-06-10T01:28:03.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.786+0000 7f18bd94eb80 1 -- 172.21.15.7:0/3704442635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x555908c71090 con 0x555908bf2f60 2024-06-10T01:28:03.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.786+0000 7f188bfff700 1 -- 172.21.15.7:0/3704442635 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f189c016070 con 0x555908bf2f60 2024-06-10T01:28:03.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.786+0000 7f188bfff700 1 -- 172.21.15.7:0/3704442635 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f189c004730 con 0x555908bf2f60 2024-06-10T01:28:03.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.786+0000 7f18bd94eb80 10 client.20685 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:28:03.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.786+0000 7f18bd94eb80 10 client.20685 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:28:03.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.786+0000 7f18bd94eb80 20 client.20685 populate_metadata read hostname 'smithi007' 2024-06-10T01:28:03.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.786+0000 7f18bd94eb80 1 --2- 172.21.15.7:0/3704442635 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x555908c71520 0x555908c73940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:03.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.786+0000 7f18bd94eb80 4 client.20685 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:28:03.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.786+0000 7f18bd94eb80 1 -- 172.21.15.7:0/3704442635 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555908b462d0 con 0x555908c71520 2024-06-10T01:28:03.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.787+0000 7f18b0e67700 1 --2- 172.21.15.7:0/3704442635 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x555908c71520 0x555908c73940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.788+0000 7f188bfff700 10 client.20685 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:28:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.788+0000 7f18b0e67700 1 --2- 172.21.15.7:0/3704442635 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x555908c71520 0x555908c73940 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8f68f30728b533ab server_cookie=40881d69ce66c5c9 in_seq=0 out_seq=0 2024-06-10T01:28:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.790+0000 7f188bfff700 1 -- 172.21.15.7:0/3704442635 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555908b462d0 con 0x555908c71520 2024-06-10T01:28:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.790+0000 7f188bfff700 10 client.20685 handle_command_reply: tid=0 2024-06-10T01:28:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.790+0000 7f18bd94eb80 1 client.20685 shutdown 2024-06-10T01:28:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.790+0000 7f18bd94eb80 1 -- 172.21.15.7:0/3704442635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555908bf2f60 msgr2=0x555908bff7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.790+0000 7f18bd94eb80 1 --2- 172.21.15.7:0/3704442635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555908bf2f60 0x555908bff7a0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f189c029fd0 tx=0x7f189c020ed0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.790+0000 7f18bd94eb80 1 -- 172.21.15.7:0/3704442635 shutdown_connections 2024-06-10T01:28:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.790+0000 7f18bd94eb80 1 --2- 172.21.15.7:0/3704442635 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x555908c71520 0x555908c73940 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.790+0000 7f188bfff700 0 client.20685 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:28:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.790+0000 7f18bd94eb80 1 --2- 172.21.15.7:0/3704442635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555908bf3880 0x555908c0da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.790+0000 7f18bd94eb80 1 --2- 172.21.15.7:0/3704442635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555908bed7a0 0x555908bff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.790+0000 7f18bd94eb80 1 --2- 172.21.15.7:0/3704442635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555908bf2f60 0x555908bff7a0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.790+0000 7f18bd94eb80 1 -- 172.21.15.7:0/3704442635 >> 172.21.15.7:0/3704442635 conn(0x555908b1a080 msgr2=0x555908bf3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.790+0000 7f18bd94eb80 1 -- 172.21.15.7:0/3704442635 shutdown_connections 2024-06-10T01:28:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.790+0000 7f18bd94eb80 1 -- 172.21.15.7:0/3704442635 wait complete. 2024-06-10T01:28:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.790+0000 7f18bd94eb80 20 client.20685 trim_cache size 0 max 16384 2024-06-10T01:28:03.785 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:28:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.791+0000 7f18b30cb700 1 -- 172.21.15.7:0/1325801356 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f18940a2c20 msgr2=0x7f18940a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.791+0000 7f18b30cb700 1 --2- 172.21.15.7:0/1325801356 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f18940a2c20 0x7f18940a50e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f18ac078720 tx=0x7f189c009260 comp rx=0 tx=0).stop 2024-06-10T01:28:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.791+0000 7f18b30cb700 1 -- 172.21.15.7:0/1325801356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18ac06aa00 msgr2=0x7f18ac19ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.791+0000 7f18b30cb700 1 --2- 172.21.15.7:0/1325801356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18ac06aa00 0x7f18ac19ae70 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f18a8000c00 tx=0x7f18a800bab0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.791+0000 7f18b30cb700 1 -- 172.21.15.7:0/1325801356 shutdown_connections 2024-06-10T01:28:03.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.791+0000 7f18b30cb700 1 --2- 172.21.15.7:0/1325801356 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f18940a2c20 0x7f18940a50e0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.791+0000 7f18b30cb700 1 --2- 172.21.15.7:0/1325801356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18ac1084f0 0x7f18ac19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.791+0000 7f18b30cb700 1 --2- 172.21.15.7:0/1325801356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18ac06aa00 0x7f18ac19ae70 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.791+0000 7f18b30cb700 1 --2- 172.21.15.7:0/1325801356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18ac06b3c0 0x7f18ac19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:03.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.791+0000 7f18b30cb700 1 -- 172.21.15.7:0/1325801356 >> 172.21.15.7:0/1325801356 conn(0x7f18ac078300 msgr2=0x7f18ac0ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:03.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.792+0000 7f18b30cb700 1 -- 172.21.15.7:0/1325801356 shutdown_connections 2024-06-10T01:28:03.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:03.792+0000 7f18b30cb700 1 -- 172.21.15.7:0/1325801356 wait complete. 2024-06-10T01:28:03.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:03 smithi078 ceph-mon[36683]: pgmap v1095: 129 pgs: 129 active+clean; 674 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.7 MiB/s wr, 568 op/s 2024-06-10T01:28:03.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:03 smithi175 ceph-mon[36636]: pgmap v1095: 129 pgs: 129 active+clean; 674 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.7 MiB/s wr, 568 op/s 2024-06-10T01:28:04.555 INFO:tasks.workunit.client.0.smithi007.stdout: 10 214207 23.32 MB/sec execute 401 sec latency 89.464 ms 2024-06-10T01:28:04.650 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:28:04.947 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:28:05.555 INFO:tasks.workunit.client.0.smithi007.stdout: 10 214881 23.35 MB/sec execute 402 sec latency 69.353 ms 2024-06-10T01:28:05.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:05 smithi007 ceph-mon[32398]: pgmap v1096: 129 pgs: 129 active+clean; 678 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.5 MiB/s wr, 549 op/s 2024-06-10T01:28:05.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:05 smithi078 ceph-mon[36683]: pgmap v1096: 129 pgs: 129 active+clean; 678 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.5 MiB/s wr, 549 op/s 2024-06-10T01:28:05.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:05 smithi175 ceph-mon[36636]: pgmap v1096: 129 pgs: 129 active+clean; 678 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.5 MiB/s wr, 549 op/s 2024-06-10T01:28:06.555 INFO:tasks.workunit.client.0.smithi007.stdout: 10 215679 23.41 MB/sec execute 403 sec latency 54.372 ms 2024-06-10T01:28:07.555 INFO:tasks.workunit.client.0.smithi007.stdout: 10 216329 23.45 MB/sec execute 404 sec latency 83.951 ms 2024-06-10T01:28:07.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:07 smithi078 ceph-mon[36683]: pgmap v1097: 129 pgs: 129 active+clean; 683 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.6 MiB/s wr, 554 op/s 2024-06-10T01:28:07.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:07 smithi175 ceph-mon[36636]: pgmap v1097: 129 pgs: 129 active+clean; 683 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.6 MiB/s wr, 554 op/s 2024-06-10T01:28:08.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:07 smithi007 ceph-mon[32398]: pgmap v1097: 129 pgs: 129 active+clean; 683 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.6 MiB/s wr, 554 op/s 2024-06-10T01:28:08.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.180+0000 7f9b4a05f700 1 -- 172.21.15.7:0/1222452485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b44102540 msgr2=0x7f9b441029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:08.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.180+0000 7f9b4a05f700 1 --2- 172.21.15.7:0/1222452485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b44102540 0x7f9b441029a0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f9b38009990 tx=0x7f9b38009ca0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.180+0000 7f9b4a05f700 1 -- 172.21.15.7:0/1222452485 shutdown_connections 2024-06-10T01:28:08.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.180+0000 7f9b4a05f700 1 --2- 172.21.15.7:0/1222452485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b44102ee0 0x7f9b4410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.180+0000 7f9b4a05f700 1 --2- 172.21.15.7:0/1222452485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b44107450 0x7f9b44107830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.180+0000 7f9b4a05f700 1 --2- 172.21.15.7:0/1222452485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b44102540 0x7f9b441029a0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.180+0000 7f9b4a05f700 1 -- 172.21.15.7:0/1222452485 >> 172.21.15.7:0/1222452485 conn(0x7f9b440795f0 msgr2=0x7f9b44079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:08.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.180+0000 7f9b4a05f700 1 -- 172.21.15.7:0/1222452485 shutdown_connections 2024-06-10T01:28:08.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.180+0000 7f9b4a05f700 1 -- 172.21.15.7:0/1222452485 wait complete. 2024-06-10T01:28:08.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.181+0000 7f9b4a05f700 1 Processor -- start 2024-06-10T01:28:08.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.181+0000 7f9b4a05f700 1 -- start start 2024-06-10T01:28:08.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.181+0000 7f9b4a05f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b44102540 0x7f9b4419ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:08.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.181+0000 7f9b4a05f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b44102ee0 0x7f9b4419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:08.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.181+0000 7f9b4a05f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b44107450 0x7f9b4419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:08.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.181+0000 7f9b4a05f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9b441127e0 con 0x7f9b44102540 2024-06-10T01:28:08.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.181+0000 7f9b4a05f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9b44112660 con 0x7f9b44107450 2024-06-10T01:28:08.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.181+0000 7f9b4a05f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9b44112960 con 0x7f9b44102ee0 2024-06-10T01:28:08.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.181+0000 7f9b437fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b44102540 0x7f9b4419ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:08.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.181+0000 7f9b437fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b44102540 0x7f9b4419ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47474/0 (socket says 172.21.15.7:47474) 2024-06-10T01:28:08.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.181+0000 7f9b437fe700 1 -- 172.21.15.7:0/993134292 learned_addr learned my addr 172.21.15.7:0/993134292 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:08.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.181+0000 7f9b42ffd700 1 --2- 172.21.15.7:0/993134292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b44102ee0 0x7f9b4419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:08.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.181+0000 7f9b43fff700 1 --2- 172.21.15.7:0/993134292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b44107450 0x7f9b4419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:08.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.182+0000 7f9b437fe700 1 -- 172.21.15.7:0/993134292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b44102ee0 msgr2=0x7f9b4419b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:08.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.182+0000 7f9b437fe700 1 --2- 172.21.15.7:0/993134292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b44102ee0 0x7f9b4419b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.182+0000 7f9b437fe700 1 -- 172.21.15.7:0/993134292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b44107450 msgr2=0x7f9b4419f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:08.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.182+0000 7f9b437fe700 1 --2- 172.21.15.7:0/993134292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b44107450 0x7f9b4419f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.182+0000 7f9b437fe700 1 -- 172.21.15.7:0/993134292 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4419fe70 con 0x7f9b44102540 2024-06-10T01:28:08.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.182+0000 7f9b437fe700 1 --2- 172.21.15.7:0/993134292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b44102540 0x7f9b4419ae60 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400e950 tx=0x7f9b3400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:08.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.182+0000 7f9b40ff9700 1 -- 172.21.15.7:0/993134292 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b34004230 con 0x7f9b44102540 2024-06-10T01:28:08.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.182+0000 7f9b40ff9700 1 -- 172.21.15.7:0/993134292 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b340043d0 con 0x7f9b44102540 2024-06-10T01:28:08.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.182+0000 7f9b4a05f700 1 -- 172.21.15.7:0/993134292 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b441a0160 con 0x7f9b44102540 2024-06-10T01:28:08.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.182+0000 7f9b40ff9700 1 -- 172.21.15.7:0/993134292 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b34003dd0 con 0x7f9b44102540 2024-06-10T01:28:08.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.182+0000 7f9b4a05f700 1 -- 172.21.15.7:0/993134292 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b441a79e0 con 0x7f9b44102540 2024-06-10T01:28:08.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.184+0000 7f9b40ff9700 1 -- 172.21.15.7:0/993134292 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9b3400c7e0 con 0x7f9b44102540 2024-06-10T01:28:08.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.185+0000 7f9b40ff9700 1 --2- 172.21.15.7:0/993134292 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9b2c0a2ca0 0x7f9b2c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:08.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.185+0000 7f9b40ff9700 1 -- 172.21.15.7:0/993134292 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f9b340cf550 con 0x7f9b44102540 2024-06-10T01:28:08.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.185+0000 7f9b42ffd700 1 --2- 172.21.15.7:0/993134292 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9b2c0a2ca0 0x7f9b2c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:08.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.186+0000 7f9b42ffd700 1 --2- 172.21.15.7:0/993134292 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9b2c0a2ca0 0x7f9b2c0a5160 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800b2e0 tx=0x7f9b38005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:08.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.186+0000 7f9b548e2b80 1 Processor -- start 2024-06-10T01:28:08.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.186+0000 7f9b548e2b80 1 -- start start 2024-06-10T01:28:08.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.186+0000 7f9b548e2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55816e04d640 0x55816e04da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:08.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.186+0000 7f9b548e2b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55816e04df60 0x55816e0479e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:08.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.186+0000 7f9b548e2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55816e047f20 0x55816e048340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:08.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.186+0000 7f9b548e2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55816df99810 con 0x55816e04d640 2024-06-10T01:28:08.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.186+0000 7f9b548e2b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55816df41720 con 0x55816e04df60 2024-06-10T01:28:08.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.186+0000 7f9b548e2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55816df957d0 con 0x55816e047f20 2024-06-10T01:28:08.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.186+0000 7f9b437fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55816e04d640 0x55816e04da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:08.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.186+0000 7f9b437fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55816e04d640 0x55816e04da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47488/0 (socket says 172.21.15.7:47488) 2024-06-10T01:28:08.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.186+0000 7f9b437fe700 1 -- 172.21.15.7:0/2056581559 learned_addr learned my addr 172.21.15.7:0/2056581559 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:08.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.186+0000 7f9b42ffd700 1 --2- 172.21.15.7:0/2056581559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55816e04df60 0x55816e0479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:08.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.186+0000 7f9b43fff700 1 --2- 172.21.15.7:0/2056581559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55816e047f20 0x55816e048340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:08.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.186+0000 7f9b437fe700 1 -- 172.21.15.7:0/2056581559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55816e047f20 msgr2=0x55816e048340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:08.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.186+0000 7f9b437fe700 1 --2- 172.21.15.7:0/2056581559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55816e047f20 0x55816e048340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.186+0000 7f9b437fe700 1 -- 172.21.15.7:0/2056581559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55816e04df60 msgr2=0x55816e0479e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:08.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.186+0000 7f9b437fe700 1 --2- 172.21.15.7:0/2056581559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55816e04df60 0x55816e0479e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.186+0000 7f9b437fe700 1 -- 172.21.15.7:0/2056581559 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55816df37f50 con 0x55816e04d640 2024-06-10T01:28:08.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.186+0000 7f9b43fff700 1 --2- 172.21.15.7:0/2056581559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55816e047f20 0x55816e048340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:28:08.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.187+0000 7f9b437fe700 1 --2- 172.21.15.7:0/2056581559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55816e04d640 0x55816e04da20 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f9b34098400 tx=0x7f9b3409b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f7025e9da0bb81 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:08.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.187+0000 7f9b29ffb700 1 -- 172.21.15.7:0/2056581559 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3409ba30 con 0x55816e04d640 2024-06-10T01:28:08.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.187+0000 7f9b29ffb700 1 -- 172.21.15.7:0/2056581559 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b3409fde0 con 0x55816e04d640 2024-06-10T01:28:08.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.187+0000 7f9b29ffb700 1 -- 172.21.15.7:0/2056581559 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3409bef0 con 0x55816e04d640 2024-06-10T01:28:08.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.188+0000 7f9b548e2b80 1 -- 172.21.15.7:0/2056581559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55816e04d640 msgr2=0x55816e04da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:08.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.188+0000 7f9b548e2b80 1 --2- 172.21.15.7:0/2056581559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55816e04d640 0x55816e04da20 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f9b34098400 tx=0x7f9b3409b880 comp rx=0 tx=0).stop 2024-06-10T01:28:08.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.188+0000 7f9b548e2b80 1 -- 172.21.15.7:0/2056581559 shutdown_connections 2024-06-10T01:28:08.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.188+0000 7f9b548e2b80 1 --2- 172.21.15.7:0/2056581559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55816e047f20 0x55816e048340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.188+0000 7f9b548e2b80 1 --2- 172.21.15.7:0/2056581559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55816e04d640 0x55816e04da20 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.188+0000 7f9b548e2b80 1 --2- 172.21.15.7:0/2056581559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55816e04df60 0x55816e0479e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.188+0000 7f9b548e2b80 1 -- 172.21.15.7:0/2056581559 >> 172.21.15.7:0/2056581559 conn(0x55816df72cc0 msgr2=0x55816e042850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:08.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.188+0000 7f9b548e2b80 1 -- 172.21.15.7:0/2056581559 shutdown_connections 2024-06-10T01:28:08.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.188+0000 7f9b548e2b80 1 -- 172.21.15.7:0/2056581559 wait complete. 2024-06-10T01:28:08.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.188+0000 7f9b548e2b80 1 -- start start 2024-06-10T01:28:08.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.188+0000 7f9b548e2b80 1 Processor -- start 2024-06-10T01:28:08.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.189+0000 7f9b548e2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55816e047f20 0x55816e053d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:08.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.189+0000 7f9b548e2b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55816e04d640 0x55816e051010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:08.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.189+0000 7f9b548e2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55816e04df60 0x55816e051550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:08.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.189+0000 7f9b548e2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55816e0549d0 con 0x55816e047f20 2024-06-10T01:28:08.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.189+0000 7f9b548e2b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55816df39bf0 con 0x55816e04d640 2024-06-10T01:28:08.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.189+0000 7f9b548e2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55816e054b50 con 0x55816e04df60 2024-06-10T01:28:08.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.189+0000 7f9b548e2b80 10 client.0 _ll_register_callbacks cb 0x55816df33920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:28:08.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.189+0000 7f9b42ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55816e04d640 0x55816e051010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:08.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.189+0000 7f9b437fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55816e047f20 0x55816e053d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:08.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.189+0000 7f9b42ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55816e04d640 0x55816e051010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:54270/0 (socket says 172.21.15.7:54270) 2024-06-10T01:28:08.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.189+0000 7f9b42ffd700 1 -- 172.21.15.7:0/2666622306 learned_addr learned my addr 172.21.15.7:0/2666622306 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:08.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.189+0000 7f9b43fff700 1 --2- 172.21.15.7:0/2666622306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55816e04df60 0x55816e051550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:08.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.189+0000 7f9b437fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55816e047f20 0x55816e053d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47498/0 (socket says 172.21.15.7:47498) 2024-06-10T01:28:08.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.189+0000 7f9b437fe700 1 -- 172.21.15.7:0/2666622306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55816e04df60 msgr2=0x55816e051550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:08.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.189+0000 7f9b437fe700 1 --2- 172.21.15.7:0/2666622306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55816e04df60 0x55816e051550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.189+0000 7f9b437fe700 1 -- 172.21.15.7:0/2666622306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55816e04d640 msgr2=0x55816e051010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:08.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.189+0000 7f9b437fe700 1 --2- 172.21.15.7:0/2666622306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55816e04d640 0x55816e051010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.189+0000 7f9b437fe700 1 -- 172.21.15.7:0/2666622306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55816dee88d0 con 0x55816e047f20 2024-06-10T01:28:08.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.190+0000 7f9b437fe700 1 --2- 172.21.15.7:0/2666622306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55816e047f20 0x55816e053d10 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f9b340ccd20 tx=0x7f9b3409c1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e813ae1927feee server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:08.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.190+0000 7f9b22ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:28:08.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.190+0000 7f9b22ffd700 1 -- 172.21.15.7:0/2666622306 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b34080060 con 0x55816e047f20 2024-06-10T01:28:08.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.190+0000 7f9b22ffd700 1 -- 172.21.15.7:0/2666622306 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f9b340a4210 con 0x55816e047f20 2024-06-10T01:28:08.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.190+0000 7f9b548e2b80 1 -- 172.21.15.7:0/2666622306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b3409fde0 con 0x55816e047f20 2024-06-10T01:28:08.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.190+0000 7f9b22ffd700 1 -- 172.21.15.7:0/2666622306 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f9b3409ba30 con 0x55816e047f20 2024-06-10T01:28:08.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.190+0000 7f9b548e2b80 10 client.20697 fetch_fsmap learned FSMap version 55 2024-06-10T01:28:08.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.190+0000 7f9b548e2b80 1 -- 172.21.15.7:0/2666622306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55816e05f2e0 con 0x55816e047f20 2024-06-10T01:28:08.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.190+0000 7f9b22ffd700 1 -- 172.21.15.7:0/2666622306 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b3409fde0 con 0x55816e047f20 2024-06-10T01:28:08.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.190+0000 7f9b22ffd700 1 -- 172.21.15.7:0/2666622306 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f9b3409ae80 con 0x55816e047f20 2024-06-10T01:28:08.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.190+0000 7f9b548e2b80 10 client.20697 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:28:08.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.190+0000 7f9b548e2b80 10 client.20697 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:28:08.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.191+0000 7f9b548e2b80 20 client.20697 populate_metadata read hostname 'smithi007' 2024-06-10T01:28:08.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.191+0000 7f9b548e2b80 1 --2- 172.21.15.7:0/2666622306 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55816e0617b0 0x55816e063ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:08.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.191+0000 7f9b548e2b80 4 client.20697 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:28:08.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.191+0000 7f9b548e2b80 1 -- 172.21.15.7:0/2666622306 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55816dfa6d10 con 0x55816e0617b0 2024-06-10T01:28:08.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.191+0000 7f9b42ffd700 1 --2- 172.21.15.7:0/2666622306 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55816e0617b0 0x55816e063ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:08.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.191+0000 7f9b42ffd700 1 --2- 172.21.15.7:0/2666622306 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55816e0617b0 0x55816e063ba0 crc :-1 s=READY pgs=495 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19c85ffd7d0ad3e0 server_cookie=ea2fbfc9a0f54f19 in_seq=0 out_seq=0 2024-06-10T01:28:08.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.191+0000 7f9b22ffd700 10 client.20697 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:08.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.195+0000 7f9b22ffd700 1 -- 172.21.15.7:0/2666622306 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55816dfa6d10 con 0x55816e0617b0 2024-06-10T01:28:08.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.195+0000 7f9b22ffd700 10 client.20697 handle_command_reply: tid=0 2024-06-10T01:28:08.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.195+0000 7f9b548e2b80 1 client.20697 shutdown 2024-06-10T01:28:08.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.196+0000 7f9b548e2b80 1 -- 172.21.15.7:0/2666622306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55816e047f20 msgr2=0x55816e053d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:08.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.196+0000 7f9b548e2b80 1 --2- 172.21.15.7:0/2666622306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55816e047f20 0x55816e053d10 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f9b340ccd20 tx=0x7f9b3409c1f0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.196+0000 7f9b548e2b80 1 -- 172.21.15.7:0/2666622306 shutdown_connections 2024-06-10T01:28:08.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.196+0000 7f9b548e2b80 1 --2- 172.21.15.7:0/2666622306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55816e04df60 0x55816e051550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.196+0000 7f9b548e2b80 1 --2- 172.21.15.7:0/2666622306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55816e047f20 0x55816e053d10 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.196+0000 7f9b548e2b80 1 --2- 172.21.15.7:0/2666622306 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55816e0617b0 0x55816e063ba0 crc :-1 s=READY pgs=495 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.196+0000 7f9b548e2b80 1 --2- 172.21.15.7:0/2666622306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55816e04d640 0x55816e051010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.196+0000 7f9b548e2b80 1 -- 172.21.15.7:0/2666622306 >> 172.21.15.7:0/2666622306 conn(0x55816df72cc0 msgr2=0x55816e0469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:08.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.196+0000 7f9b22ffd700 0 client.20697 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:08.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.196+0000 7f9b548e2b80 1 -- 172.21.15.7:0/2666622306 shutdown_connections 2024-06-10T01:28:08.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.196+0000 7f9b548e2b80 1 -- 172.21.15.7:0/2666622306 wait complete. 2024-06-10T01:28:08.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.196+0000 7f9b548e2b80 20 client.20697 trim_cache size 0 max 16384 2024-06-10T01:28:08.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.215+0000 7f9b548e2b80 1 Processor -- start 2024-06-10T01:28:08.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.215+0000 7f9b548e2b80 1 -- start start 2024-06-10T01:28:08.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.215+0000 7f9b548e2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55816e04d640 0x55816e0479e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:08.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.215+0000 7f9b548e2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55816e04df60 0x55816e04ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:08.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.215+0000 7f9b548e2b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55816e0667e0 0x55816e066c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:08.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.215+0000 7f9b548e2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9b441b2e10 con 0x55816e04d640 2024-06-10T01:28:08.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.215+0000 7f9b548e2b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9b441b06a0 con 0x55816e0667e0 2024-06-10T01:28:08.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.215+0000 7f9b548e2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9b441ae4f0 con 0x55816e04df60 2024-06-10T01:28:08.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.215+0000 7f9b437fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55816e04d640 0x55816e0479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:08.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.216+0000 7f9b437fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55816e04d640 0x55816e0479e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47500/0 (socket says 172.21.15.7:47500) 2024-06-10T01:28:08.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.216+0000 7f9b437fe700 1 -- 172.21.15.7:0/255803910 learned_addr learned my addr 172.21.15.7:0/255803910 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:08.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.216+0000 7f9b42ffd700 1 --2- 172.21.15.7:0/255803910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55816e04df60 0x55816e04ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:08.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.216+0000 7f9b43fff700 1 --2- 172.21.15.7:0/255803910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55816e0667e0 0x55816e066c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:08.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.216+0000 7f9b437fe700 1 -- 172.21.15.7:0/255803910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55816e04df60 msgr2=0x55816e04ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:08.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.216+0000 7f9b437fe700 1 --2- 172.21.15.7:0/255803910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55816e04df60 0x55816e04ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.216+0000 7f9b437fe700 1 -- 172.21.15.7:0/255803910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55816e0667e0 msgr2=0x55816e066c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:08.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.216+0000 7f9b437fe700 1 --2- 172.21.15.7:0/255803910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55816e0667e0 0x55816e066c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.216+0000 7f9b437fe700 1 -- 172.21.15.7:0/255803910 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55816e051f50 con 0x55816e04d640 2024-06-10T01:28:08.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.216+0000 7f9b43fff700 1 --2- 172.21.15.7:0/255803910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55816e0667e0 0x55816e066c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:28:08.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.216+0000 7f9b437fe700 1 --2- 172.21.15.7:0/255803910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55816e04d640 0x55816e0479e0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f9b3409b730 tx=0x7f9b340ccd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7064a433fd51d01 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:08.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.216+0000 7f9b297fa700 1 -- 172.21.15.7:0/255803910 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b34080070 con 0x55816e04d640 2024-06-10T01:28:08.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.216+0000 7f9b297fa700 1 -- 172.21.15.7:0/255803910 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b340a46a0 con 0x55816e04d640 2024-06-10T01:28:08.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.216+0000 7f9b297fa700 1 -- 172.21.15.7:0/255803910 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3409ae80 con 0x55816e04d640 2024-06-10T01:28:08.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.217+0000 7f9b548e2b80 1 -- 172.21.15.7:0/255803910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55816e04d640 msgr2=0x55816e0479e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:08.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.217+0000 7f9b548e2b80 1 --2- 172.21.15.7:0/255803910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55816e04d640 0x55816e0479e0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f9b3409b730 tx=0x7f9b340ccd50 comp rx=0 tx=0).stop 2024-06-10T01:28:08.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.217+0000 7f9b548e2b80 1 -- 172.21.15.7:0/255803910 shutdown_connections 2024-06-10T01:28:08.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.217+0000 7f9b548e2b80 1 --2- 172.21.15.7:0/255803910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55816e0667e0 0x55816e066c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.217+0000 7f9b548e2b80 1 --2- 172.21.15.7:0/255803910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55816e04d640 0x55816e0479e0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.217+0000 7f9b548e2b80 1 --2- 172.21.15.7:0/255803910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55816e04df60 0x55816e04ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.217+0000 7f9b548e2b80 1 -- 172.21.15.7:0/255803910 >> 172.21.15.7:0/255803910 conn(0x55816df72cc0 msgr2=0x55816e04e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:08.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.217+0000 7f9b548e2b80 1 -- 172.21.15.7:0/255803910 shutdown_connections 2024-06-10T01:28:08.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.217+0000 7f9b548e2b80 1 -- 172.21.15.7:0/255803910 wait complete. 2024-06-10T01:28:08.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.218+0000 7f9b548e2b80 1 -- start start 2024-06-10T01:28:08.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.218+0000 7f9b548e2b80 1 Processor -- start 2024-06-10T01:28:08.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.218+0000 7f9b548e2b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55816e04d640 0x55816e0637c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:08.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.218+0000 7f9b548e2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55816e04df60 0x55816e063d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:08.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.218+0000 7f9b548e2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55816e0667e0 0x55816e064240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.218+0000 7f9b548e2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9b441aeb60 con 0x55816e04df60 2024-06-10T01:28:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.218+0000 7f9b548e2b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9b441affd0 con 0x55816e04d640 2024-06-10T01:28:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.218+0000 7f9b548e2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9b441af230 con 0x55816e0667e0 2024-06-10T01:28:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.218+0000 7f9b548e2b80 10 client.0 _ll_register_callbacks cb 0x55816df33920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:28:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.219+0000 7f9b42ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55816e04df60 0x55816e063d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.219+0000 7f9b42ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55816e04df60 0x55816e063d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47510/0 (socket says 172.21.15.7:47510) 2024-06-10T01:28:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.219+0000 7f9b42ffd700 1 -- 172.21.15.7:0/544017189 learned_addr learned my addr 172.21.15.7:0/544017189 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.219+0000 7f9b43fff700 1 --2- 172.21.15.7:0/544017189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55816e0667e0 0x55816e064240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.219+0000 7f9b42ffd700 1 -- 172.21.15.7:0/544017189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55816e0667e0 msgr2=0x55816e064240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.219+0000 7f9b42ffd700 1 --2- 172.21.15.7:0/544017189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55816e0667e0 0x55816e064240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.219+0000 7f9b42ffd700 1 -- 172.21.15.7:0/544017189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55816e04d640 msgr2=0x55816e0637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.219+0000 7f9b42ffd700 1 --2- 172.21.15.7:0/544017189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55816e04d640 0x55816e0637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.219+0000 7f9b42ffd700 1 -- 172.21.15.7:0/544017189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55816e0d3900 con 0x55816e04df60 2024-06-10T01:28:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.219+0000 7f9b42ffd700 1 --2- 172.21.15.7:0/544017189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55816e04df60 0x55816e063d00 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f9b38000c00 tx=0x7f9b38009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510709b606024a52 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:08.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.219+0000 7f9b22ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:28:08.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.219+0000 7f9b22ffd700 1 -- 172.21.15.7:0/544017189 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b38003b50 con 0x55816e04df60 2024-06-10T01:28:08.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.219+0000 7f9b22ffd700 1 -- 172.21.15.7:0/544017189 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f9b38003cf0 con 0x55816e04df60 2024-06-10T01:28:08.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.219+0000 7f9b548e2b80 1 -- 172.21.15.7:0/544017189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b340a46a0 con 0x55816e04df60 2024-06-10T01:28:08.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.219+0000 7f9b22ffd700 1 -- 172.21.15.7:0/544017189 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f9b38042050 con 0x55816e04df60 2024-06-10T01:28:08.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.220+0000 7f9b548e2b80 10 client.20703 fetch_fsmap learned FSMap version 55 2024-06-10T01:28:08.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.220+0000 7f9b548e2b80 1 -- 172.21.15.7:0/544017189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55816e0d4b80 con 0x55816e04df60 2024-06-10T01:28:08.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.220+0000 7f9b22ffd700 1 -- 172.21.15.7:0/544017189 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b340a46a0 con 0x55816e04df60 2024-06-10T01:28:08.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.220+0000 7f9b22ffd700 1 -- 172.21.15.7:0/544017189 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f9b380304b0 con 0x55816e04df60 2024-06-10T01:28:08.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.220+0000 7f9b548e2b80 10 client.20703 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:28:08.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.220+0000 7f9b548e2b80 10 client.20703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:28:08.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.220+0000 7f9b548e2b80 20 client.20703 populate_metadata read hostname 'smithi007' 2024-06-10T01:28:08.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.220+0000 7f9b548e2b80 1 --2- 172.21.15.7:0/544017189 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55816e0d5270 0x55816e0d7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:08.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.220+0000 7f9b548e2b80 4 client.20703 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:28:08.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.220+0000 7f9b548e2b80 1 -- 172.21.15.7:0/544017189 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55816dfa1290 con 0x55816e0d5270 2024-06-10T01:28:08.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.220+0000 7f9b437fe700 1 --2- 172.21.15.7:0/544017189 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55816e0d5270 0x55816e0d7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:08.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.221+0000 7f9b437fe700 1 --2- 172.21.15.7:0/544017189 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55816e0d5270 0x55816e0d7690 crc :-1 s=READY pgs=496 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=921bd5bffa6f6aea server_cookie=c54586ce9a17fe08 in_seq=0 out_seq=0 2024-06-10T01:28:08.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.221+0000 7f9b22ffd700 10 client.20703 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:08.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.226+0000 7f9b22ffd700 1 -- 172.21.15.7:0/544017189 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55816dfa1290 con 0x55816e0d5270 2024-06-10T01:28:08.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.226+0000 7f9b22ffd700 10 client.20703 handle_command_reply: tid=0 2024-06-10T01:28:08.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.226+0000 7f9b548e2b80 1 client.20703 shutdown 2024-06-10T01:28:08.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.227+0000 7f9b548e2b80 1 -- 172.21.15.7:0/544017189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55816e04df60 msgr2=0x55816e063d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:08.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.227+0000 7f9b548e2b80 1 --2- 172.21.15.7:0/544017189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55816e04df60 0x55816e063d00 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f9b38000c00 tx=0x7f9b38009ca0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.227+0000 7f9b548e2b80 1 -- 172.21.15.7:0/544017189 shutdown_connections 2024-06-10T01:28:08.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.227+0000 7f9b548e2b80 1 --2- 172.21.15.7:0/544017189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55816e0667e0 0x55816e064240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.227+0000 7f9b548e2b80 1 --2- 172.21.15.7:0/544017189 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55816e0d5270 0x55816e0d7690 crc :-1 s=READY pgs=496 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.227+0000 7f9b548e2b80 1 --2- 172.21.15.7:0/544017189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55816e04d640 0x55816e0637c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.227+0000 7f9b548e2b80 1 --2- 172.21.15.7:0/544017189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55816e04df60 0x55816e063d00 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.227+0000 7f9b22ffd700 0 client.20703 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:08.335 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:28:08.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.227+0000 7f9b548e2b80 1 -- 172.21.15.7:0/544017189 >> 172.21.15.7:0/544017189 conn(0x55816df72cc0 msgr2=0x55816e03be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:08.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.227+0000 7f9b548e2b80 1 -- 172.21.15.7:0/544017189 shutdown_connections 2024-06-10T01:28:08.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.227+0000 7f9b548e2b80 1 -- 172.21.15.7:0/544017189 wait complete. 2024-06-10T01:28:08.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.227+0000 7f9b548e2b80 20 client.20703 trim_cache size 0 max 16384 2024-06-10T01:28:08.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.228+0000 7f9b4a05f700 1 -- 172.21.15.7:0/993134292 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9b2c0a2ca0 msgr2=0x7f9b2c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:08.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.228+0000 7f9b4a05f700 1 --2- 172.21.15.7:0/993134292 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9b2c0a2ca0 0x7f9b2c0a5160 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800b2e0 tx=0x7f9b38005960 comp rx=0 tx=0).stop 2024-06-10T01:28:08.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.228+0000 7f9b4a05f700 1 -- 172.21.15.7:0/993134292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b44102540 msgr2=0x7f9b4419ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:08.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.228+0000 7f9b4a05f700 1 --2- 172.21.15.7:0/993134292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b44102540 0x7f9b4419ae60 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400e950 tx=0x7f9b3400ec60 comp rx=0 tx=0).stop 2024-06-10T01:28:08.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.228+0000 7f9b4a05f700 1 -- 172.21.15.7:0/993134292 shutdown_connections 2024-06-10T01:28:08.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.228+0000 7f9b4a05f700 1 --2- 172.21.15.7:0/993134292 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9b2c0a2ca0 0x7f9b2c0a5160 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.228+0000 7f9b4a05f700 1 --2- 172.21.15.7:0/993134292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b44107450 0x7f9b4419f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.228+0000 7f9b4a05f700 1 --2- 172.21.15.7:0/993134292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b44102540 0x7f9b4419ae60 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.228+0000 7f9b4a05f700 1 --2- 172.21.15.7:0/993134292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b44102ee0 0x7f9b4419b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:08.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.228+0000 7f9b4a05f700 1 -- 172.21.15.7:0/993134292 >> 172.21.15.7:0/993134292 conn(0x7f9b440795f0 msgr2=0x7f9b441003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:08.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.228+0000 7f9b4a05f700 1 -- 172.21.15.7:0/993134292 shutdown_connections 2024-06-10T01:28:08.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:07.228+0000 7f9b4a05f700 1 -- 172.21.15.7:0/993134292 wait complete. 2024-06-10T01:28:08.555 INFO:tasks.workunit.client.0.smithi007.stdout: 10 216801 23.46 MB/sec execute 405 sec latency 203.430 ms 2024-06-10T01:28:09.555 INFO:tasks.workunit.client.0.smithi007.stdout: 10 217278 23.46 MB/sec execute 406 sec latency 106.563 ms 2024-06-10T01:28:09.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:09 smithi007 ceph-mon[32398]: pgmap v1098: 129 pgs: 129 active+clean; 689 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.3 MiB/s wr, 523 op/s 2024-06-10T01:28:09.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:09 smithi078 ceph-mon[36683]: pgmap v1098: 129 pgs: 129 active+clean; 689 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.3 MiB/s wr, 523 op/s 2024-06-10T01:28:09.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:09 smithi175 ceph-mon[36636]: pgmap v1098: 129 pgs: 129 active+clean; 689 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.3 MiB/s wr, 523 op/s 2024-06-10T01:28:10.555 INFO:tasks.workunit.client.0.smithi007.stdout: 10 217673 23.45 MB/sec execute 407 sec latency 503.310 ms 2024-06-10T01:28:10.956 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:28:10.957 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:28:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.240+0000 7f9cc2702640 1 -- 172.21.15.7:0/1308460874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cbc10dcc0 msgr2=0x7f9cbc10e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.240+0000 7f9cc2702640 1 --2- 172.21.15.7:0/1308460874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cbc10dcc0 0x7f9cbc10e140 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0009900 tx=0x7f9cb002a440 comp rx=0 tx=0).stop 2024-06-10T01:28:11.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.244+0000 7f9cc2702640 1 -- 172.21.15.7:0/1308460874 shutdown_connections 2024-06-10T01:28:11.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.244+0000 7f9cc2702640 1 --2- 172.21.15.7:0/1308460874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cbc10dcc0 0x7f9cbc10e140 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:11.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.244+0000 7f9cc2702640 1 --2- 172.21.15.7:0/1308460874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cbc10b310 0x7f9cbc10d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:11.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.244+0000 7f9cc2702640 1 --2- 172.21.15.7:0/1308460874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cbc1089a0 0x7f9cbc10add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:11.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.244+0000 7f9cc2702640 1 -- 172.21.15.7:0/1308460874 >> 172.21.15.7:0/1308460874 conn(0x7f9cbc06e9e0 msgr2=0x7f9cbc070e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:11.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.250+0000 7f9cc2702640 1 -- 172.21.15.7:0/1308460874 shutdown_connections 2024-06-10T01:28:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.254+0000 7f9cc2702640 1 -- 172.21.15.7:0/1308460874 wait complete. 2024-06-10T01:28:11.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.254+0000 7f9cc2702640 1 Processor -- start 2024-06-10T01:28:11.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.255+0000 7f9cc2702640 1 -- start start 2024-06-10T01:28:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.259+0000 7f9cc2702640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cbc1089a0 0x7f9cbc115070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.259+0000 7f9cc2702640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cbc10b310 0x7f9cbc1155b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.259+0000 7f9cc2702640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cbc10dcc0 0x7f9cbc1b6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.259+0000 7f9cc2702640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc11c4e0 con 0x7f9cbc10b310 2024-06-10T01:28:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.259+0000 7f9cc2702640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc11c360 con 0x7f9cbc10dcc0 2024-06-10T01:28:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.259+0000 7f9cc2702640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc11c660 con 0x7f9cbc1089a0 2024-06-10T01:28:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.259+0000 7f9cc0c9f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cbc10dcc0 0x7f9cbc1b6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.259+0000 7f9cc0c9f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cbc10dcc0 0x7f9cbc1b6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:54316/0 (socket says 172.21.15.7:54316) 2024-06-10T01:28:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.259+0000 7f9cc0c9f640 1 -- 172.21.15.7:0/2027836486 learned_addr learned my addr 172.21.15.7:0/2027836486 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.260+0000 7f9cbbfff640 1 --2- 172.21.15.7:0/2027836486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cbc1089a0 0x7f9cbc115070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.260+0000 7f9cc0c9f640 1 -- 172.21.15.7:0/2027836486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cbc1089a0 msgr2=0x7f9cbc115070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.260+0000 7f9cc0c9f640 1 --2- 172.21.15.7:0/2027836486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cbc1089a0 0x7f9cbc115070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.260+0000 7f9cc0c9f640 1 -- 172.21.15.7:0/2027836486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cbc10b310 msgr2=0x7f9cbc1155b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T01:28:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.260+0000 7f9cc0c9f640 1 --2- 172.21.15.7:0/2027836486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cbc10b310 0x7f9cbc1155b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.260+0000 7f9cc0c9f640 1 -- 172.21.15.7:0/2027836486 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cbc1b7130 con 0x7f9cbc10dcc0 2024-06-10T01:28:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.260+0000 7f9cc0c9f640 1 --2- 172.21.15.7:0/2027836486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cbc10dcc0 0x7f9cbc1b6bf0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0009850 tx=0x7f9cb0032d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:11.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.264+0000 7f9cb97fa640 1 -- 172.21.15.7:0/2027836486 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb003f030 con 0x7f9cbc10dcc0 2024-06-10T01:28:11.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.264+0000 7f9cc2702640 1 -- 172.21.15.7:0/2027836486 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cbc1b7300 con 0x7f9cbc10dcc0 2024-06-10T01:28:11.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.264+0000 7f9cc2702640 1 -- 172.21.15.7:0/2027836486 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cbc1b7700 con 0x7f9cbc10dcc0 2024-06-10T01:28:11.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.264+0000 7f9cb97fa640 1 -- 172.21.15.7:0/2027836486 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cb0003ce0 con 0x7f9cbc10dcc0 2024-06-10T01:28:11.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.266+0000 7f9cb97fa640 1 -- 172.21.15.7:0/2027836486 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0035070 con 0x7f9cbc10dcc0 2024-06-10T01:28:11.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.268+0000 7f9cc5405740 1 Processor -- start 2024-06-10T01:28:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.270+0000 7f9cb97fa640 1 -- 172.21.15.7:0/2027836486 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9cb0042030 con 0x7f9cbc10dcc0 2024-06-10T01:28:11.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.274+0000 7f9cc5405740 1 -- start start 2024-06-10T01:28:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.274+0000 7f9cc5405740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea92500010 0x55ea92500430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.274+0000 7f9cc5405740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea92500a00 0x55ea92505500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.274+0000 7f9cc5405740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea92505b70 0x55ea92508000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.274+0000 7f9cc5405740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ea924634f0 con 0x55ea92500010 2024-06-10T01:28:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.274+0000 7f9cc5405740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ea92401870 con 0x55ea92500a00 2024-06-10T01:28:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.274+0000 7f9cc5405740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ea9245f4b0 con 0x55ea92505b70 2024-06-10T01:28:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.274+0000 7f9cbbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea92500a00 0x55ea92505500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.274+0000 7f9cbbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea92500a00 0x55ea92505500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:54318/0 (socket says 172.21.15.7:54318) 2024-06-10T01:28:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.274+0000 7f9cbbfff640 1 -- 172.21.15.7:0/429606914 learned_addr learned my addr 172.21.15.7:0/429606914 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.275+0000 7f9cbbfff640 1 -- 172.21.15.7:0/429606914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea92505b70 msgr2=0x55ea92508000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T01:28:11.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.275+0000 7f9cbbfff640 1 --2- 172.21.15.7:0/429606914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea92505b70 0x55ea92508000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:11.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.275+0000 7f9cbbfff640 1 -- 172.21.15.7:0/429606914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea92500010 msgr2=0x55ea92500430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T01:28:11.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.275+0000 7f9cbbfff640 1 --2- 172.21.15.7:0/429606914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea92500010 0x55ea92500430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:11.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.275+0000 7f9cbbfff640 1 -- 172.21.15.7:0/429606914 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea9241d9b0 con 0x55ea92500a00 2024-06-10T01:28:11.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.275+0000 7f9cbbfff640 1 --2- 172.21.15.7:0/429606914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea92500a00 0x55ea92505500 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4007e10 tx=0x7f9cb400c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8be0d4cefbe5cbdc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:11.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.276+0000 7f9cb97fa640 1 --2- 172.21.15.7:0/2027836486 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9ca40a2d90 0x7f9ca40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:11.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.276+0000 7f9ca27fc640 1 -- 172.21.15.7:0/429606914 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4015020 con 0x55ea92500a00 2024-06-10T01:28:11.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.276+0000 7f9ca27fc640 1 -- 172.21.15.7:0/429606914 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cb4011b10 con 0x55ea92500a00 2024-06-10T01:28:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.276+0000 7f9cb97fa640 1 -- 172.21.15.7:0/2027836486 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f9cb00f0010 con 0x7f9cbc10dcc0 2024-06-10T01:28:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.277+0000 7f9cc5405740 1 -- 172.21.15.7:0/429606914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea92500a00 msgr2=0x55ea92505500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.277+0000 7f9cc5405740 1 --2- 172.21.15.7:0/429606914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea92500a00 0x55ea92505500 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4007e10 tx=0x7f9cb400c560 comp rx=0 tx=0).stop 2024-06-10T01:28:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.277+0000 7f9cc5405740 1 -- 172.21.15.7:0/429606914 shutdown_connections 2024-06-10T01:28:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.277+0000 7f9cc5405740 1 --2- 172.21.15.7:0/429606914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea92505b70 0x55ea92508000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.277+0000 7f9cc5405740 1 --2- 172.21.15.7:0/429606914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea92500a00 0x55ea92505500 secure :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4007e10 tx=0x7f9cb400c560 comp rx=0 tx=0).stop 2024-06-10T01:28:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.277+0000 7f9cc5405740 1 --2- 172.21.15.7:0/429606914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea92500010 0x55ea92500430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.277+0000 7f9cc5405740 1 -- 172.21.15.7:0/429606914 >> 172.21.15.7:0/429606914 conn(0x55ea924fb680 msgr2=0x55ea924fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.281+0000 7f9cc5405740 1 -- 172.21.15.7:0/429606914 shutdown_connections 2024-06-10T01:28:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.282+0000 7f9cbb7fe640 1 --2- 172.21.15.7:0/2027836486 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9ca40a2d90 0x7f9ca40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.282+0000 7f9cc5405740 1 -- 172.21.15.7:0/429606914 wait complete. 2024-06-10T01:28:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.282+0000 7f9cc5405740 1 -- start start 2024-06-10T01:28:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.282+0000 7f9cc5405740 1 Processor -- start 2024-06-10T01:28:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.284+0000 7f9cc5405740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea92500010 0x55ea9250a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.284+0000 7f9cc5405740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea92505b70 0x55ea9250a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.284+0000 7f9cc5405740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9250f9c0 0x55ea92511db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.284+0000 7f9cc5405740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ea92508b70 con 0x55ea9250f9c0 2024-06-10T01:28:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.284+0000 7f9cc5405740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ea923ecae0 con 0x55ea92500010 2024-06-10T01:28:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.284+0000 7f9cc5405740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ea92508cf0 con 0x55ea92505b70 2024-06-10T01:28:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.284+0000 7f9cc5405740 10 client.0 _ll_register_callbacks cb 0x55ea924259b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:28:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.285+0000 7f9cbbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea92500010 0x55ea9250a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.286+0000 7f9cbbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea92500010 0x55ea9250a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:54334/0 (socket says 172.21.15.7:54334) 2024-06-10T01:28:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.286+0000 7f9cbbfff640 1 -- 172.21.15.7:0/3967078204 learned_addr learned my addr 172.21.15.7:0/3967078204 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.291+0000 7f9cc0c9f640 1 --2- 172.21.15.7:0/3967078204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9250f9c0 0x55ea92511db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.291+0000 7f9cbb7fe640 1 --2- 172.21.15.7:0/3967078204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea92505b70 0x55ea9250a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.292+0000 7f9cc0c9f640 1 -- 172.21.15.7:0/3967078204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea92505b70 msgr2=0x55ea9250a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.292+0000 7f9cc0c9f640 1 --2- 172.21.15.7:0/3967078204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea92505b70 0x55ea9250a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.292+0000 7f9cc0c9f640 1 -- 172.21.15.7:0/3967078204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea92500010 msgr2=0x55ea9250a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.292+0000 7f9cc0c9f640 1 --2- 172.21.15.7:0/3967078204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea92500010 0x55ea9250a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.292+0000 7f9cc0c9f640 1 -- 172.21.15.7:0/3967078204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea92177210 con 0x55ea9250f9c0 2024-06-10T01:28:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.295+0000 7f9cbbfff640 1 --2- 172.21.15.7:0/3967078204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea92500010 0x55ea9250a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:28:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.296+0000 7f9cbb7fe640 1 --2- 172.21.15.7:0/2027836486 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9ca40a2d90 0x7f9ca40a5250 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x55ea92501260 tx=0x7f9cac005b60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.301+0000 7f9cc0c9f640 1 --2- 172.21.15.7:0/3967078204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9250f9c0 0x55ea92511db0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0031470 tx=0x7f9cb0009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e1d7b0fcaaafd1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.303+0000 7f9c9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:28:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.304+0000 7f9c9b7fe640 1 -- 172.21.15.7:0/3967078204 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb00d5020 con 0x55ea9250f9c0 2024-06-10T01:28:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.304+0000 7f9c9b7fe640 1 -- 172.21.15.7:0/3967078204 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f9cb00b95f0 con 0x55ea9250f9c0 2024-06-10T01:28:11.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.304+0000 7f9cc5405740 1 -- 172.21.15.7:0/3967078204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cb4011b10 con 0x55ea9250f9c0 2024-06-10T01:28:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.313+0000 7f9c9b7fe640 1 -- 172.21.15.7:0/3967078204 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f9cb0049040 con 0x55ea9250f9c0 2024-06-10T01:28:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.313+0000 7f9cc5405740 10 client.20709 fetch_fsmap learned FSMap version 55 2024-06-10T01:28:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.313+0000 7f9cc5405740 1 -- 172.21.15.7:0/3967078204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55ea92243620 con 0x55ea9250f9c0 2024-06-10T01:28:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.320+0000 7f9c9b7fe640 1 -- 172.21.15.7:0/3967078204 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cb4011b10 con 0x55ea9250f9c0 2024-06-10T01:28:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.320+0000 7f9c9b7fe640 1 -- 172.21.15.7:0/3967078204 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f9cb00c5870 con 0x55ea9250f9c0 2024-06-10T01:28:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.320+0000 7f9cc5405740 10 client.20709 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:28:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.320+0000 7f9cc5405740 10 client.20709 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:28:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.320+0000 7f9cc5405740 20 client.20709 populate_metadata read hostname 'smithi007' 2024-06-10T01:28:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.320+0000 7f9cc5405740 1 --2- 172.21.15.7:0/3967078204 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ea925169c0 0x55ea92518de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.320+0000 7f9cc5405740 4 client.20709 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:28:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.320+0000 7f9cc5405740 1 -- 172.21.15.7:0/3967078204 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea92476f10 con 0x55ea925169c0 2024-06-10T01:28:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.326+0000 7f9cbbfff640 1 --2- 172.21.15.7:0/3967078204 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ea925169c0 0x55ea92518de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.341+0000 7f9cbbfff640 1 --2- 172.21.15.7:0/3967078204 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ea925169c0 0x55ea92518de0 crc :-1 s=READY pgs=497 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de71319ab91e1c77 server_cookie=23dc66a2263b38e9 in_seq=0 out_seq=0 2024-06-10T01:28:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.342+0000 7f9c9b7fe640 10 client.20709 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.355+0000 7f9c9b7fe640 1 -- 172.21.15.7:0/3967078204 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ea92476f10 con 0x55ea925169c0 2024-06-10T01:28:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.355+0000 7f9c9b7fe640 10 client.20709 handle_command_reply: tid=0 2024-06-10T01:28:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.355+0000 7f9cc5405740 1 client.20709 shutdown 2024-06-10T01:28:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.370+0000 7f9cc5405740 1 -- 172.21.15.7:0/3967078204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9250f9c0 msgr2=0x55ea92511db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.370+0000 7f9cc5405740 1 --2- 172.21.15.7:0/3967078204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9250f9c0 0x55ea92511db0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0031470 tx=0x7f9cb0009660 comp rx=0 tx=0).stop 2024-06-10T01:28:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.370+0000 7f9cc5405740 1 -- 172.21.15.7:0/3967078204 shutdown_connections 2024-06-10T01:28:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.370+0000 7f9cc5405740 1 --2- 172.21.15.7:0/3967078204 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ea925169c0 0x55ea92518de0 crc :-1 s=READY pgs=497 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.370+0000 7f9cc5405740 1 --2- 172.21.15.7:0/3967078204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9250f9c0 0x55ea92511db0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.370+0000 7f9cc5405740 1 --2- 172.21.15.7:0/3967078204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea92505b70 0x55ea9250a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.370+0000 7f9cc5405740 1 --2- 172.21.15.7:0/3967078204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea92500010 0x55ea9250a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.370+0000 7f9cc5405740 1 -- 172.21.15.7:0/3967078204 >> 172.21.15.7:0/3967078204 conn(0x55ea924fb680 msgr2=0x55ea924fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.370+0000 7f9c9b7fe640 0 client.20709 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.370+0000 7f9cc5405740 1 -- 172.21.15.7:0/3967078204 shutdown_connections 2024-06-10T01:28:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.370+0000 7f9cc5405740 1 -- 172.21.15.7:0/3967078204 wait complete. 2024-06-10T01:28:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.373+0000 7f9cc5405740 20 client.20709 trim_cache size 0 max 16384 2024-06-10T01:28:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.424+0000 7f9cc5405740 1 Processor -- start 2024-06-10T01:28:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.425+0000 7f9cc5405740 1 -- start start 2024-06-10T01:28:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.425+0000 7f9cc5405740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea92500010 0x55ea92507f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.425+0000 7f9cc5405740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea92505b70 0x55ea92501c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.425+0000 7f9cc5405740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9250f9c0 0x55ea925021c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.426+0000 7f9cc5405740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca8012570 con 0x55ea92500010 2024-06-10T01:28:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.426+0000 7f9cc0c9f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9250f9c0 0x55ea925021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.426+0000 7f9cc0c9f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9250f9c0 0x55ea925021c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:41534/0 (socket says 172.21.15.7:41534) 2024-06-10T01:28:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.426+0000 7f9cc0c9f640 1 -- 172.21.15.7:0/3186634802 learned_addr learned my addr 172.21.15.7:0/3186634802 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.426+0000 7f9cc5405740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca800fe40 con 0x55ea92505b70 2024-06-10T01:28:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.426+0000 7f9cc5405740 1 -- 172.21.15.7:0/3186634802 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca800dc90 con 0x55ea9250f9c0 2024-06-10T01:28:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.430+0000 7f9cbbfff640 1 --2- 172.21.15.7:0/3186634802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea92500010 0x55ea92507f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.430+0000 7f9cc0c9f640 1 -- 172.21.15.7:0/3186634802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea92505b70 msgr2=0x55ea92501c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:28:11.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.430+0000 7f9cc0c9f640 1 --2- 172.21.15.7:0/3186634802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea92505b70 0x55ea92501c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:11.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.430+0000 7f9cc0c9f640 1 -- 172.21.15.7:0/3186634802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea92500010 msgr2=0x55ea92507f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:11.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.430+0000 7f9cc0c9f640 1 --2- 172.21.15.7:0/3186634802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea92500010 0x55ea92507f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:11.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.430+0000 7f9cc0c9f640 1 -- 172.21.15.7:0/3186634802 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea92508a00 con 0x55ea9250f9c0 2024-06-10T01:28:11.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.431+0000 7f9cc0c9f640 1 --2- 172.21.15.7:0/3186634802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9250f9c0 0x55ea925021c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9cb00bcc80 tx=0x7f9cb0005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b51953804f11773 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:11.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.435+0000 7f9ca1ffb640 1 -- 172.21.15.7:0/3186634802 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb00d5070 con 0x55ea9250f9c0 2024-06-10T01:28:11.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.436+0000 7f9ca1ffb640 1 -- 172.21.15.7:0/3186634802 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cb00d7030 con 0x55ea9250f9c0 2024-06-10T01:28:11.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.436+0000 7f9ca1ffb640 1 -- 172.21.15.7:0/3186634802 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb00b9750 con 0x55ea9250f9c0 2024-06-10T01:28:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.437+0000 7f9cc5405740 1 -- 172.21.15.7:0/3186634802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9250f9c0 msgr2=0x55ea925021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.437+0000 7f9cc5405740 1 --2- 172.21.15.7:0/3186634802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9250f9c0 0x55ea925021c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9cb00bcc80 tx=0x7f9cb0005c90 comp rx=0 tx=0).stop 2024-06-10T01:28:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.438+0000 7f9cc5405740 1 -- 172.21.15.7:0/3186634802 shutdown_connections 2024-06-10T01:28:11.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.438+0000 7f9cc5405740 1 --2- 172.21.15.7:0/3186634802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9250f9c0 0x55ea925021c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:11.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.438+0000 7f9cc5405740 1 --2- 172.21.15.7:0/3186634802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea92505b70 0x55ea92501c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:11.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.438+0000 7f9cc5405740 1 --2- 172.21.15.7:0/3186634802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea92500010 0x55ea92507f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:11.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.438+0000 7f9cc5405740 1 -- 172.21.15.7:0/3186634802 >> 172.21.15.7:0/3186634802 conn(0x55ea924fb680 msgr2=0x55ea9250a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:11.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.440+0000 7f9cc5405740 1 -- 172.21.15.7:0/3186634802 shutdown_connections 2024-06-10T01:28:11.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.440+0000 7f9cc5405740 1 -- 172.21.15.7:0/3186634802 wait complete. 2024-06-10T01:28:11.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.441+0000 7f9cc5405740 1 -- start start 2024-06-10T01:28:11.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.441+0000 7f9cc5405740 1 Processor -- start 2024-06-10T01:28:11.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.441+0000 7f9cc5405740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea92500010 0x55ea924fe820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:11.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.441+0000 7f9cc5405740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea92505b70 0x55ea924fed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.441+0000 7f9cc5405740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9250f9c0 0x55ea9252a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.441+0000 7f9cc5405740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca800e300 con 0x55ea9250f9c0 2024-06-10T01:28:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.441+0000 7f9cc5405740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca800f770 con 0x55ea92500010 2024-06-10T01:28:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.441+0000 7f9cc5405740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca800e9d0 con 0x55ea92505b70 2024-06-10T01:28:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.441+0000 7f9cc5405740 10 client.0 _ll_register_callbacks cb 0x7f9cb0005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:28:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.442+0000 7f9cbb7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea92505b70 0x55ea924fed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.442+0000 7f9cbb7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea92505b70 0x55ea924fed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:41542/0 (socket says 172.21.15.7:41542) 2024-06-10T01:28:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.442+0000 7f9cbb7fe640 1 -- 172.21.15.7:0/1657653949 learned_addr learned my addr 172.21.15.7:0/1657653949 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.443+0000 7f9cbbfff640 1 --2- 172.21.15.7:0/1657653949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea92500010 0x55ea924fe820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.444+0000 7f9cc0c9f640 1 --2- 172.21.15.7:0/1657653949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9250f9c0 0x55ea9252a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.444+0000 7f9cbbfff640 1 -- 172.21.15.7:0/1657653949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea92505b70 msgr2=0x55ea924fed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.444+0000 7f9cbbfff640 1 --2- 172.21.15.7:0/1657653949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea92505b70 0x55ea924fed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.444+0000 7f9cbbfff640 1 -- 172.21.15.7:0/1657653949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9250f9c0 msgr2=0x55ea9252a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.444+0000 7f9cbbfff640 1 --2- 172.21.15.7:0/1657653949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9250f9c0 0x55ea9252a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.444+0000 7f9cbbfff640 1 -- 172.21.15.7:0/1657653949 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea9252ace0 con 0x55ea92500010 2024-06-10T01:28:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.444+0000 7f9cbbfff640 1 --2- 172.21.15.7:0/1657653949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea92500010 0x55ea924fe820 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9cb400ca60 tx=0x7f9cb40011e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4725b15bdaa86686 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.444+0000 7f9c9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:28:11.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.445+0000 7f9c9b7fe640 1 -- 172.21.15.7:0/1657653949 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4015070 con 0x55ea92500010 2024-06-10T01:28:11.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.445+0000 7f9cc5405740 1 -- 172.21.15.7:0/1657653949 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cb00d7030 con 0x55ea92500010 2024-06-10T01:28:11.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.445+0000 7f9c9b7fe640 1 -- 172.21.15.7:0/1657653949 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f9cb4003a10 con 0x55ea92500010 2024-06-10T01:28:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.450+0000 7f9c9b7fe640 1 -- 172.21.15.7:0/1657653949 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f9cb4039050 con 0x55ea92500010 2024-06-10T01:28:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.450+0000 7f9cc5405740 10 client.29906 fetch_fsmap learned FSMap version 55 2024-06-10T01:28:11.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.450+0000 7f9cc5405740 1 -- 172.21.15.7:0/1657653949 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55ea925802f0 con 0x55ea92500010 2024-06-10T01:28:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.455+0000 7f9c9b7fe640 1 -- 172.21.15.7:0/1657653949 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cb00d7030 con 0x55ea92500010 2024-06-10T01:28:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.455+0000 7f9c9b7fe640 1 -- 172.21.15.7:0/1657653949 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f9cb402f030 con 0x55ea92500010 2024-06-10T01:28:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.455+0000 7f9cc5405740 10 client.29906 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:28:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.455+0000 7f9cc5405740 10 client.29906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:28:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.455+0000 7f9cc5405740 20 client.29906 populate_metadata read hostname 'smithi007' 2024-06-10T01:28:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.455+0000 7f9cc5405740 1 --2- 172.21.15.7:0/1657653949 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ea925809f0 0x55ea92582e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.455+0000 7f9cc5405740 4 client.29906 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:28:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.455+0000 7f9cc5405740 1 -- 172.21.15.7:0/1657653949 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ea92470c20 con 0x55ea925809f0 2024-06-10T01:28:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.461+0000 7f9cbb7fe640 1 --2- 172.21.15.7:0/1657653949 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ea925809f0 0x55ea92582e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.481+0000 7f9cbb7fe640 1 --2- 172.21.15.7:0/1657653949 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ea925809f0 0x55ea92582e10 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0051ba1fea1ab2a server_cookie=d46646a63bbb09fd in_seq=0 out_seq=0 2024-06-10T01:28:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:11.485+0000 7f9c9b7fe640 10 client.29906 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:11.556 INFO:tasks.workunit.client.0.smithi007.stdout: 10 217693 23.39 MB/sec execute 408 sec latency 1242.497 ms 2024-06-10T01:28:11.597 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:28:11.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:11 smithi078 ceph-mon[36683]: pgmap v1099: 129 pgs: 129 active+clean; 691 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.2 MiB/s wr, 497 op/s 2024-06-10T01:28:11.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:11 smithi175 ceph-mon[36636]: pgmap v1099: 129 pgs: 129 active+clean; 691 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.2 MiB/s wr, 497 op/s 2024-06-10T01:28:12.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:11 smithi007 ceph-mon[32398]: pgmap v1099: 129 pgs: 129 active+clean; 691 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.2 MiB/s wr, 497 op/s 2024-06-10T01:28:12.556 INFO:tasks.workunit.client.0.smithi007.stdout: 10 217740 23.34 MB/sec execute 409 sec latency 2242.540 ms 2024-06-10T01:28:13.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:13.450+0000 7f9c9b7fe640 1 -- 172.21.15.7:0/1657653949 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ea92470c20 con 0x55ea925809f0 2024-06-10T01:28:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:13.450+0000 7f9c9b7fe640 10 client.29906 handle_command_reply: tid=0 2024-06-10T01:28:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:13.450+0000 7f9cc5405740 1 client.29906 shutdown 2024-06-10T01:28:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:13.451+0000 7f9cc5405740 1 -- 172.21.15.7:0/1657653949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea92500010 msgr2=0x55ea924fe820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:13.451+0000 7f9cc5405740 1 --2- 172.21.15.7:0/1657653949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea92500010 0x55ea924fe820 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9cb400ca60 tx=0x7f9cb40011e0 comp rx=0 tx=0).stop 2024-06-10T01:28:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:13.451+0000 7f9cc5405740 1 -- 172.21.15.7:0/1657653949 shutdown_connections 2024-06-10T01:28:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:13.451+0000 7f9cc5405740 1 --2- 172.21.15.7:0/1657653949 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ea925809f0 0x55ea92582e10 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:13.451+0000 7f9cc5405740 1 --2- 172.21.15.7:0/1657653949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9250f9c0 0x55ea9252a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:13.451+0000 7f9cc5405740 1 --2- 172.21.15.7:0/1657653949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea92505b70 0x55ea924fed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:13.451+0000 7f9cc5405740 1 --2- 172.21.15.7:0/1657653949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea92500010 0x55ea924fe820 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:13.451+0000 7f9cc5405740 1 -- 172.21.15.7:0/1657653949 >> 172.21.15.7:0/1657653949 conn(0x55ea924fb680 msgr2=0x55ea92501c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:13.451+0000 7f9c9b7fe640 0 client.29906 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:13.451+0000 7f9cc5405740 1 -- 172.21.15.7:0/1657653949 shutdown_connections 2024-06-10T01:28:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:13.451+0000 7f9cc5405740 1 -- 172.21.15.7:0/1657653949 wait complete. 2024-06-10T01:28:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:13.451+0000 7f9cc5405740 20 client.29906 trim_cache size 0 max 16384 2024-06-10T01:28:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:28:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:28:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:28:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:13.452+0000 7f9cc2702640 1 -- 172.21.15.7:0/2027836486 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9ca40a2d90 msgr2=0x7f9ca40a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:13.452+0000 7f9cc2702640 1 --2- 172.21.15.7:0/2027836486 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9ca40a2d90 0x7f9ca40a5250 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x55ea92501260 tx=0x7f9cac005b60 comp rx=0 tx=0).stop 2024-06-10T01:28:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:13.452+0000 7f9cc2702640 1 -- 172.21.15.7:0/2027836486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cbc10dcc0 msgr2=0x7f9cbc1b6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:13.452+0000 7f9cc2702640 1 --2- 172.21.15.7:0/2027836486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cbc10dcc0 0x7f9cbc1b6bf0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0009850 tx=0x7f9cb0032d30 comp rx=0 tx=0).stop 2024-06-10T01:28:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:13.452+0000 7f9cc2702640 1 -- 172.21.15.7:0/2027836486 shutdown_connections 2024-06-10T01:28:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:13.452+0000 7f9cc2702640 1 --2- 172.21.15.7:0/2027836486 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9ca40a2d90 0x7f9ca40a5250 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:13.452+0000 7f9cc2702640 1 --2- 172.21.15.7:0/2027836486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cbc10dcc0 0x7f9cbc1b6bf0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:13.452+0000 7f9cc2702640 1 --2- 172.21.15.7:0/2027836486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cbc10b310 0x7f9cbc1155b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:13.452+0000 7f9cc2702640 1 --2- 172.21.15.7:0/2027836486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cbc1089a0 0x7f9cbc115070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:13.452+0000 7f9cc2702640 1 -- 172.21.15.7:0/2027836486 >> 172.21.15.7:0/2027836486 conn(0x7f9cbc06e9e0 msgr2=0x7f9cbc10ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:13.452+0000 7f9cc2702640 1 -- 172.21.15.7:0/2027836486 shutdown_connections 2024-06-10T01:28:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:13.452+0000 7f9cc2702640 1 -- 172.21.15.7:0/2027836486 wait complete. 2024-06-10T01:28:13.555 INFO:tasks.workunit.client.0.smithi007.stdout: 10 217866 23.29 MB/sec execute 410 sec latency 2606.971 ms 2024-06-10T01:28:13.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:13 smithi007 ceph-mon[32398]: pgmap v1100: 129 pgs: 129 active+clean; 727 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.8 MiB/s wr, 626 op/s 2024-06-10T01:28:13.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:13 smithi175 ceph-mon[36636]: pgmap v1100: 129 pgs: 129 active+clean; 727 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.8 MiB/s wr, 626 op/s 2024-06-10T01:28:14.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:13 smithi078 ceph-mon[36683]: pgmap v1100: 129 pgs: 129 active+clean; 727 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.8 MiB/s wr, 626 op/s 2024-06-10T01:28:14.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.178+0000 7f3847086700 1 -- 172.21.15.7:0/2150767557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3840109a60 msgr2=0x7f3840109ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:14.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.178+0000 7f3847086700 1 --2- 172.21.15.7:0/2150767557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3840109a60 0x7f3840109ee0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f38280098f0 tx=0x7f3828009c00 comp rx=0 tx=0).stop 2024-06-10T01:28:14.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.179+0000 7f3847086700 1 -- 172.21.15.7:0/2150767557 shutdown_connections 2024-06-10T01:28:14.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.179+0000 7f3847086700 1 --2- 172.21.15.7:0/2150767557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38401175c0 0x7f38401199b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.179+0000 7f3847086700 1 --2- 172.21.15.7:0/2150767557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f384010cc10 0x7f3840109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.179+0000 7f3847086700 1 --2- 172.21.15.7:0/2150767557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3840109a60 0x7f3840109ee0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.179+0000 7f3847086700 1 -- 172.21.15.7:0/2150767557 >> 172.21.15.7:0/2150767557 conn(0x7f384006d600 msgr2=0x7f384006da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:14.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.179+0000 7f3847086700 1 -- 172.21.15.7:0/2150767557 shutdown_connections 2024-06-10T01:28:14.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.179+0000 7f3847086700 1 -- 172.21.15.7:0/2150767557 wait complete. 2024-06-10T01:28:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.180+0000 7f3847086700 1 Processor -- start 2024-06-10T01:28:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.180+0000 7f3847086700 1 -- start start 2024-06-10T01:28:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.180+0000 7f3847086700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3840109a60 0x7f3840122dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.180+0000 7f3847086700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f384010cc10 0x7f3840123300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.180+0000 7f3847086700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38401175c0 0x7f3840123840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.180+0000 7f3847086700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f384010e340 con 0x7f384010cc10 2024-06-10T01:28:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.180+0000 7f3847086700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f384010e1c0 con 0x7f3840109a60 2024-06-10T01:28:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.180+0000 7f3847086700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f384010e4c0 con 0x7f38401175c0 2024-06-10T01:28:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.181+0000 7f3844e22700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3840109a60 0x7f3840122dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:14.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.181+0000 7f3844e22700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3840109a60 0x7f3840122dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:40814/0 (socket says 172.21.15.7:40814) 2024-06-10T01:28:14.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.181+0000 7f3844e22700 1 -- 172.21.15.7:0/1303079173 learned_addr learned my addr 172.21.15.7:0/1303079173 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:14.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.181+0000 7f3845623700 1 --2- 172.21.15.7:0/1303079173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38401175c0 0x7f3840123840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:14.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.182+0000 7f383ffff700 1 --2- 172.21.15.7:0/1303079173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f384010cc10 0x7f3840123300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:14.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.182+0000 7f3845623700 1 -- 172.21.15.7:0/1303079173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3840109a60 msgr2=0x7f3840122dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:14.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.182+0000 7f3845623700 1 --2- 172.21.15.7:0/1303079173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3840109a60 0x7f3840122dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.182+0000 7f3845623700 1 -- 172.21.15.7:0/1303079173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f384010cc10 msgr2=0x7f3840123300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:14.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.182+0000 7f3845623700 1 --2- 172.21.15.7:0/1303079173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f384010cc10 0x7f3840123300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.182+0000 7f3845623700 1 -- 172.21.15.7:0/1303079173 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38401bc5b0 con 0x7f38401175c0 2024-06-10T01:28:14.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.182+0000 7f383ffff700 1 --2- 172.21.15.7:0/1303079173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f384010cc10 0x7f3840123300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:28:14.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.182+0000 7f3844e22700 1 --2- 172.21.15.7:0/1303079173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3840109a60 0x7f3840122dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:28:14.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.183+0000 7f3845623700 1 --2- 172.21.15.7:0/1303079173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38401175c0 0x7f3840123840 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f383800e940 tx=0x7f383800ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.184+0000 7f383dffb700 1 -- 172.21.15.7:0/1303079173 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838004220 con 0x7f38401175c0 2024-06-10T01:28:14.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.184+0000 7f383dffb700 1 -- 172.21.15.7:0/1303079173 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3838004d10 con 0x7f38401175c0 2024-06-10T01:28:14.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.184+0000 7f3847086700 1 -- 172.21.15.7:0/1303079173 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38401bc8a0 con 0x7f38401175c0 2024-06-10T01:28:14.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.184+0000 7f3847086700 1 -- 172.21.15.7:0/1303079173 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38401bcd90 con 0x7f38401175c0 2024-06-10T01:28:14.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.185+0000 7f383dffb700 1 -- 172.21.15.7:0/1303079173 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838004220 con 0x7f38401175c0 2024-06-10T01:28:14.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.192+0000 7f3851909b80 1 Processor -- start 2024-06-10T01:28:14.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.192+0000 7f383dffb700 1 -- 172.21.15.7:0/1303079173 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f383800cc20 con 0x7f38401175c0 2024-06-10T01:28:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.193+0000 7f383dffb700 1 --2- 172.21.15.7:0/1303079173 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f382c0a3060 0x7f382c0a5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.194+0000 7f383dffb700 1 -- 172.21.15.7:0/1303079173 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f3838004020 con 0x7f38401175c0 2024-06-10T01:28:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.194+0000 7f3851909b80 1 -- start start 2024-06-10T01:28:14.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.194+0000 7f383ffff700 1 --2- 172.21.15.7:0/1303079173 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f382c0a3060 0x7f382c0a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.195+0000 7f3851909b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556659e44fe0 0x556659e453c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.195+0000 7f3851909b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556659e45900 0x556659e3f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.195+0000 7f3851909b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556659e3f820 0x556659e3fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.195+0000 7f3851909b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556659d31fe0 con 0x556659e3f820 2024-06-10T01:28:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.195+0000 7f3844e22700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556659e44fe0 0x556659e453c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.195+0000 7f3844e22700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556659e44fe0 0x556659e453c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:40816/0 (socket says 172.21.15.7:40816) 2024-06-10T01:28:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.195+0000 7f3844e22700 1 -- 172.21.15.7:0/2302573999 learned_addr learned my addr 172.21.15.7:0/2302573999 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.195+0000 7f3851909b80 1 -- 172.21.15.7:0/2302573999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556659d908d0 con 0x556659e44fe0 2024-06-10T01:28:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.195+0000 7f383ffff700 1 --2- 172.21.15.7:0/2302573999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556659e45900 0x556659e3f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.195+0000 7f3845623700 1 --2- 172.21.15.7:0/2302573999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556659e3f820 0x556659e3fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.196+0000 7f3851909b80 1 -- 172.21.15.7:0/2302573999 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556659cf84e0 con 0x556659e45900 2024-06-10T01:28:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.196+0000 7f383ffff700 1 --2- 172.21.15.7:0/1303079173 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f382c0a3060 0x7f382c0a5520 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3840124000 tx=0x7f3828019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.196+0000 7f3844e22700 1 -- 172.21.15.7:0/2302573999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556659e45900 msgr2=0x556659e3f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:14.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.196+0000 7f3844e22700 1 --2- 172.21.15.7:0/2302573999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556659e45900 0x556659e3f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.196+0000 7f3844e22700 1 -- 172.21.15.7:0/2302573999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556659e3f820 msgr2=0x556659e3fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:14.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.196+0000 7f3844e22700 1 --2- 172.21.15.7:0/2302573999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556659e3f820 0x556659e3fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.196+0000 7f3844e22700 1 -- 172.21.15.7:0/2302573999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556659cdf950 con 0x556659e44fe0 2024-06-10T01:28:14.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.199+0000 7f3844e22700 1 --2- 172.21.15.7:0/2302573999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556659e44fe0 0x556659e453c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3830007580 tx=0x7f38300092b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=772b02a9f4cd859f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:14.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.201+0000 7f3836ffd700 1 -- 172.21.15.7:0/2302573999 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3830003a00 con 0x556659e44fe0 2024-06-10T01:28:14.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.201+0000 7f3836ffd700 1 -- 172.21.15.7:0/2302573999 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3830003ba0 con 0x556659e44fe0 2024-06-10T01:28:14.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.201+0000 7f3836ffd700 1 -- 172.21.15.7:0/2302573999 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38300042a0 con 0x556659e44fe0 2024-06-10T01:28:14.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.202+0000 7f3851909b80 1 -- 172.21.15.7:0/2302573999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556659e44fe0 msgr2=0x556659e453c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:14.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.202+0000 7f3851909b80 1 --2- 172.21.15.7:0/2302573999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556659e44fe0 0x556659e453c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3830007580 tx=0x7f38300092b0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.204+0000 7f3851909b80 1 -- 172.21.15.7:0/2302573999 shutdown_connections 2024-06-10T01:28:14.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.204+0000 7f3851909b80 1 --2- 172.21.15.7:0/2302573999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556659e3f820 0x556659e3fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.204+0000 7f3851909b80 1 --2- 172.21.15.7:0/2302573999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556659e44fe0 0x556659e453c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.204+0000 7f3851909b80 1 --2- 172.21.15.7:0/2302573999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556659e45900 0x556659e3f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.204+0000 7f3851909b80 1 -- 172.21.15.7:0/2302573999 >> 172.21.15.7:0/2302573999 conn(0x556659d6c100 msgr2=0x556659e3a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.207+0000 7f3851909b80 1 -- 172.21.15.7:0/2302573999 shutdown_connections 2024-06-10T01:28:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.207+0000 7f3851909b80 1 -- 172.21.15.7:0/2302573999 wait complete. 2024-06-10T01:28:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.208+0000 7f3851909b80 1 -- start start 2024-06-10T01:28:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.208+0000 7f3851909b80 1 Processor -- start 2024-06-10T01:28:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.209+0000 7f3851909b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556659e3f820 0x556659e4b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.209+0000 7f3851909b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556659e44fe0 0x556659e488e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.209+0000 7f3844e22700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556659e3f820 0x556659e4b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:14.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.209+0000 7f3844e22700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556659e3f820 0x556659e4b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37808/0 (socket says 172.21.15.7:37808) 2024-06-10T01:28:14.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.209+0000 7f3851909b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556659e45900 0x556659e48e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:14.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.210+0000 7f3844e22700 1 -- 172.21.15.7:0/3895162988 learned_addr learned my addr 172.21.15.7:0/3895162988 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:14.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.210+0000 7f3851909b80 1 -- 172.21.15.7:0/3895162988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556659d31040 con 0x556659e3f820 2024-06-10T01:28:14.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.210+0000 7f3851909b80 1 -- 172.21.15.7:0/3895162988 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556659d8c890 con 0x556659e44fe0 2024-06-10T01:28:14.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.210+0000 7f3845623700 1 --2- 172.21.15.7:0/3895162988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556659e45900 0x556659e48e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:14.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.210+0000 7f3851909b80 1 -- 172.21.15.7:0/3895162988 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556659e4c330 con 0x556659e45900 2024-06-10T01:28:14.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.210+0000 7f383ffff700 1 --2- 172.21.15.7:0/3895162988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556659e44fe0 0x556659e488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:14.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.210+0000 7f3851909b80 10 client.0 _ll_register_callbacks cb 0x556659d26630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:28:14.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.212+0000 7f3845623700 1 -- 172.21.15.7:0/3895162988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556659e44fe0 msgr2=0x556659e488e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:14.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.212+0000 7f3845623700 1 --2- 172.21.15.7:0/3895162988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556659e44fe0 0x556659e488e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.212+0000 7f3845623700 1 -- 172.21.15.7:0/3895162988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556659e3f820 msgr2=0x556659e4b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:14.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.212+0000 7f3845623700 1 --2- 172.21.15.7:0/3895162988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556659e3f820 0x556659e4b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.212+0000 7f3845623700 1 -- 172.21.15.7:0/3895162988 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556659d567d0 con 0x556659e45900 2024-06-10T01:28:14.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.212+0000 7f3844e22700 1 --2- 172.21.15.7:0/3895162988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556659e3f820 0x556659e4b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:28:14.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.213+0000 7f3845623700 1 --2- 172.21.15.7:0/3895162988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556659e45900 0x556659e48e20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f38380957e0 tx=0x7f3838095810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=940fdf2bebcc6a23 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.218+0000 7f381ffff700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:28:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.218+0000 7f381ffff700 1 -- 172.21.15.7:0/3895162988 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838097740 con 0x556659e45900 2024-06-10T01:28:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.218+0000 7f381ffff700 1 -- 172.21.15.7:0/3895162988 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f38380b9570 con 0x556659e45900 2024-06-10T01:28:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.218+0000 7f3851909b80 1 -- 172.21.15.7:0/3895162988 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3830003ba0 con 0x556659e45900 2024-06-10T01:28:14.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.219+0000 7f381ffff700 1 -- 172.21.15.7:0/3895162988 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f38380978e0 con 0x556659e45900 2024-06-10T01:28:14.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.219+0000 7f3851909b80 10 client.29830 fetch_fsmap learned FSMap version 55 2024-06-10T01:28:14.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.219+0000 7f3851909b80 1 -- 172.21.15.7:0/3895162988 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x556659e56a30 con 0x556659e45900 2024-06-10T01:28:14.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.224+0000 7f381ffff700 1 -- 172.21.15.7:0/3895162988 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3830003ba0 con 0x556659e45900 2024-06-10T01:28:14.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.224+0000 7f381ffff700 1 -- 172.21.15.7:0/3895162988 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f383809a670 con 0x556659e45900 2024-06-10T01:28:14.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.224+0000 7f3851909b80 10 client.29830 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:28:14.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.224+0000 7f3851909b80 10 client.29830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:28:14.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.224+0000 7f3851909b80 20 client.29830 populate_metadata read hostname 'smithi007' 2024-06-10T01:28:14.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.224+0000 7f3851909b80 1 --2- 172.21.15.7:0/3895162988 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556659e59010 0x556659e5b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:14.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.224+0000 7f3851909b80 4 client.29830 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:28:14.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.224+0000 7f3851909b80 1 -- 172.21.15.7:0/3895162988 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556659d9ddd0 con 0x556659e59010 2024-06-10T01:28:14.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.227+0000 7f3844e22700 1 --2- 172.21.15.7:0/3895162988 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556659e59010 0x556659e5b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.228+0000 7f381ffff700 10 client.29830 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.229+0000 7f3844e22700 1 --2- 172.21.15.7:0/3895162988 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556659e59010 0x556659e5b460 crc :-1 s=READY pgs=499 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c9beca30f2a70c3 server_cookie=37431f6a527b5045 in_seq=0 out_seq=0 2024-06-10T01:28:14.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.237+0000 7f381ffff700 1 -- 172.21.15.7:0/3895162988 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556659d9ddd0 con 0x556659e59010 2024-06-10T01:28:14.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.238+0000 7f381ffff700 10 client.29830 handle_command_reply: tid=0 2024-06-10T01:28:14.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.238+0000 7f3851909b80 1 client.29830 shutdown 2024-06-10T01:28:14.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.238+0000 7f3851909b80 1 -- 172.21.15.7:0/3895162988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556659e45900 msgr2=0x556659e48e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:14.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.238+0000 7f3851909b80 1 --2- 172.21.15.7:0/3895162988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556659e45900 0x556659e48e20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f38380957e0 tx=0x7f3838095810 comp rx=0 tx=0).stop 2024-06-10T01:28:14.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.238+0000 7f3851909b80 1 -- 172.21.15.7:0/3895162988 shutdown_connections 2024-06-10T01:28:14.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.238+0000 7f3851909b80 1 --2- 172.21.15.7:0/3895162988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556659e45900 0x556659e48e20 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.238+0000 7f3851909b80 1 --2- 172.21.15.7:0/3895162988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556659e3f820 0x556659e4b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.238+0000 7f3851909b80 1 --2- 172.21.15.7:0/3895162988 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556659e59010 0x556659e5b460 crc :-1 s=READY pgs=499 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.239+0000 7f3851909b80 1 --2- 172.21.15.7:0/3895162988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556659e44fe0 0x556659e488e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.239+0000 7f381ffff700 0 client.29830 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:14.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.239+0000 7f3851909b80 1 -- 172.21.15.7:0/3895162988 >> 172.21.15.7:0/3895162988 conn(0x556659d6c100 msgr2=0x556659e39780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:14.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.239+0000 7f3851909b80 1 -- 172.21.15.7:0/3895162988 shutdown_connections 2024-06-10T01:28:14.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.240+0000 7f3851909b80 1 -- 172.21.15.7:0/3895162988 wait complete. 2024-06-10T01:28:14.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.240+0000 7f3851909b80 20 client.29830 trim_cache size 0 max 16384 2024-06-10T01:28:14.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.290+0000 7f3851909b80 1 Processor -- start 2024-06-10T01:28:14.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.290+0000 7f3851909b80 1 -- start start 2024-06-10T01:28:14.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.291+0000 7f3851909b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556659e3f820 0x556659e47a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:14.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.291+0000 7f3851909b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556659e44fe0 0x556659e39f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:14.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.291+0000 7f3844e22700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556659e3f820 0x556659e47a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:14.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.291+0000 7f3844e22700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556659e3f820 0x556659e47a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:40834/0 (socket says 172.21.15.7:40834) 2024-06-10T01:28:14.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.291+0000 7f3844e22700 1 -- 172.21.15.7:0/2610971849 learned_addr learned my addr 172.21.15.7:0/2610971849 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:14.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.292+0000 7f383ffff700 1 --2- 172.21.15.7:0/2610971849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556659e44fe0 0x556659e39f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.292+0000 7f3851909b80 1 --2- 172.21.15.7:0/2610971849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556659e45900 0x556659e3a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.292+0000 7f3851909b80 1 -- 172.21.15.7:0/2610971849 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f38401c8260 con 0x556659e45900 2024-06-10T01:28:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.292+0000 7f3851909b80 1 -- 172.21.15.7:0/2610971849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f38401c5af0 con 0x556659e3f820 2024-06-10T01:28:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.292+0000 7f3851909b80 1 -- 172.21.15.7:0/2610971849 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f38401c3940 con 0x556659e44fe0 2024-06-10T01:28:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.293+0000 7f3845623700 1 --2- 172.21.15.7:0/2610971849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556659e45900 0x556659e3a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.293+0000 7f3844e22700 1 -- 172.21.15.7:0/2610971849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556659e44fe0 msgr2=0x556659e39f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:14.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.293+0000 7f3844e22700 1 --2- 172.21.15.7:0/2610971849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556659e44fe0 0x556659e39f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.293+0000 7f3844e22700 1 -- 172.21.15.7:0/2610971849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556659e45900 msgr2=0x556659e3a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:14.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.293+0000 7f3844e22700 1 --2- 172.21.15.7:0/2610971849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556659e45900 0x556659e3a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.293+0000 7f3844e22700 1 -- 172.21.15.7:0/2610971849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556659e4c090 con 0x556659e3f820 2024-06-10T01:28:14.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.293+0000 7f3844e22700 1 --2- 172.21.15.7:0/2610971849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556659e3f820 0x556659e47a60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3830028fd0 tx=0x7f383000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3702c1224fdaea2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:14.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.294+0000 7f38367fc700 1 -- 172.21.15.7:0/2610971849 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383002e070 con 0x556659e3f820 2024-06-10T01:28:14.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.294+0000 7f38367fc700 1 -- 172.21.15.7:0/2610971849 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556659d9ddd0 con 0x556659e3f820 2024-06-10T01:28:14.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.294+0000 7f38367fc700 1 -- 172.21.15.7:0/2610971849 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3830028610 con 0x556659e3f820 2024-06-10T01:28:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.295+0000 7f3851909b80 1 -- 172.21.15.7:0/2610971849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556659e3f820 msgr2=0x556659e47a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.295+0000 7f3851909b80 1 --2- 172.21.15.7:0/2610971849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556659e3f820 0x556659e47a60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3830028fd0 tx=0x7f383000b040 comp rx=0 tx=0).stop 2024-06-10T01:28:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.296+0000 7f3851909b80 1 -- 172.21.15.7:0/2610971849 shutdown_connections 2024-06-10T01:28:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.296+0000 7f3851909b80 1 --2- 172.21.15.7:0/2610971849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556659e45900 0x556659e3a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.296+0000 7f3851909b80 1 --2- 172.21.15.7:0/2610971849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556659e3f820 0x556659e47a60 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.296+0000 7f3851909b80 1 --2- 172.21.15.7:0/2610971849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556659e44fe0 0x556659e39f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.296+0000 7f3851909b80 1 -- 172.21.15.7:0/2610971849 >> 172.21.15.7:0/2610971849 conn(0x556659d6c100 msgr2=0x556659e46350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:14.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.296+0000 7f3851909b80 1 -- 172.21.15.7:0/2610971849 shutdown_connections 2024-06-10T01:28:14.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.296+0000 7f3851909b80 1 -- 172.21.15.7:0/2610971849 wait complete. 2024-06-10T01:28:14.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.297+0000 7f3851909b80 1 -- start start 2024-06-10T01:28:14.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.297+0000 7f3851909b80 1 Processor -- start 2024-06-10T01:28:14.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.298+0000 7f3851909b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556659e3f820 0x556659e4e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:14.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.298+0000 7f3851909b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556659e44fe0 0x556659e4eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:14.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.298+0000 7f3851909b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556659e45900 0x556659e4f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:14.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.299+0000 7f383ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556659e44fe0 0x556659e4eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:14.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.299+0000 7f383ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556659e44fe0 0x556659e4eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:40842/0 (socket says 172.21.15.7:40842) 2024-06-10T01:28:14.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.299+0000 7f383ffff700 1 -- 172.21.15.7:0/2541300713 learned_addr learned my addr 172.21.15.7:0/2541300713 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:14.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.300+0000 7f3844e22700 1 --2- 172.21.15.7:0/2541300713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556659e3f820 0x556659e4e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:14.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.300+0000 7f3845623700 1 --2- 172.21.15.7:0/2541300713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556659e45900 0x556659e4f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:14.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.300+0000 7f3851909b80 1 -- 172.21.15.7:0/2541300713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f38401c3fb0 con 0x556659e3f820 2024-06-10T01:28:14.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.300+0000 7f3851909b80 1 -- 172.21.15.7:0/2541300713 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f38401c5420 con 0x556659e44fe0 2024-06-10T01:28:14.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.300+0000 7f3851909b80 1 -- 172.21.15.7:0/2541300713 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f38401c4680 con 0x556659e45900 2024-06-10T01:28:14.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.301+0000 7f3851909b80 10 client.0 _ll_register_callbacks cb 0x556659d26630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:28:14.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.301+0000 7f3845623700 1 -- 172.21.15.7:0/2541300713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556659e44fe0 msgr2=0x556659e4eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:14.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.301+0000 7f3845623700 1 --2- 172.21.15.7:0/2541300713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556659e44fe0 0x556659e4eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.301+0000 7f3845623700 1 -- 172.21.15.7:0/2541300713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556659e3f820 msgr2=0x556659e4e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:14.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.301+0000 7f3845623700 1 --2- 172.21.15.7:0/2541300713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556659e3f820 0x556659e4e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.301+0000 7f3845623700 1 -- 172.21.15.7:0/2541300713 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556659ec3ea0 con 0x556659e45900 2024-06-10T01:28:14.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.302+0000 7f3845623700 1 --2- 172.21.15.7:0/2541300713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556659e45900 0x556659e4f090 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3838005330 tx=0x7f3838009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdfd36b7fe35cfda server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:14.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.302+0000 7f381ffff700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:28:14.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.302+0000 7f381ffff700 1 -- 172.21.15.7:0/2541300713 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838097a80 con 0x556659e45900 2024-06-10T01:28:14.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.303+0000 7f381ffff700 1 -- 172.21.15.7:0/2541300713 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f383809a160 con 0x556659e45900 2024-06-10T01:28:14.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.303+0000 7f3851909b80 1 -- 172.21.15.7:0/2541300713 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556659d9ddd0 con 0x556659e45900 2024-06-10T01:28:14.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.303+0000 7f381ffff700 1 -- 172.21.15.7:0/2541300713 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f3838097c50 con 0x556659e45900 2024-06-10T01:28:14.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.306+0000 7f3851909b80 10 client.29836 fetch_fsmap learned FSMap version 55 2024-06-10T01:28:14.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.306+0000 7f3851909b80 1 -- 172.21.15.7:0/2541300713 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({fsmap=55}) -- 0x556659ec3ea0 con 0x556659e45900 2024-06-10T01:28:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.310+0000 7f381ffff700 1 -- 172.21.15.7:0/2541300713 <== mon.1 v2:172.21.15.175:3300/0 4 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f383809a670 con 0x556659e45900 2024-06-10T01:28:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.310+0000 7f3851909b80 10 client.29836 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:28:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.310+0000 7f3851909b80 10 client.29836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:28:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.310+0000 7f3851909b80 20 client.29836 populate_metadata read hostname 'smithi007' 2024-06-10T01:28:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.311+0000 7f3851909b80 1 --2- 172.21.15.7:0/2541300713 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556659ec52f0 0x556659ec7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.311+0000 7f3851909b80 4 client.29836 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:28:14.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.311+0000 7f3851909b80 1 -- 172.21.15.7:0/2541300713 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556659d98350 con 0x556659ec52f0 2024-06-10T01:28:14.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.319+0000 7f3844e22700 1 --2- 172.21.15.7:0/2541300713 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556659ec52f0 0x556659ec7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:14.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.323+0000 7f3844e22700 1 --2- 172.21.15.7:0/2541300713 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556659ec52f0 0x556659ec7710 crc :-1 s=READY pgs=500 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1342df16497d556a server_cookie=ba906caed55fc425 in_seq=0 out_seq=0 2024-06-10T01:28:14.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.324+0000 7f381ffff700 10 client.29836 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:14.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.332+0000 7f381ffff700 1 -- 172.21.15.7:0/2541300713 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556659d98350 con 0x556659ec52f0 2024-06-10T01:28:14.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.333+0000 7f381ffff700 10 client.29836 handle_command_reply: tid=0 2024-06-10T01:28:14.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.333+0000 7f3851909b80 1 client.29836 shutdown 2024-06-10T01:28:14.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.333+0000 7f3851909b80 1 -- 172.21.15.7:0/2541300713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556659e45900 msgr2=0x556659e4f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:14.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.333+0000 7f3851909b80 1 --2- 172.21.15.7:0/2541300713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556659e45900 0x556659e4f090 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3838005330 tx=0x7f3838009700 comp rx=0 tx=0).stop 2024-06-10T01:28:14.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.333+0000 7f3851909b80 1 -- 172.21.15.7:0/2541300713 shutdown_connections 2024-06-10T01:28:14.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.333+0000 7f3851909b80 1 --2- 172.21.15.7:0/2541300713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556659e45900 0x556659e4f090 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.333+0000 7f3851909b80 1 --2- 172.21.15.7:0/2541300713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556659e3f820 0x556659e4e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.333+0000 7f3851909b80 1 --2- 172.21.15.7:0/2541300713 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556659ec52f0 0x556659ec7710 crc :-1 s=READY pgs=500 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.333+0000 7f3851909b80 1 --2- 172.21.15.7:0/2541300713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556659e44fe0 0x556659e4eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.333+0000 7f381ffff700 0 client.29836 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:14.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.333+0000 7f3851909b80 1 -- 172.21.15.7:0/2541300713 >> 172.21.15.7:0/2541300713 conn(0x556659d6c100 msgr2=0x556659e4a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:14.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.333+0000 7f3851909b80 1 -- 172.21.15.7:0/2541300713 shutdown_connections 2024-06-10T01:28:14.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.333+0000 7f3851909b80 1 -- 172.21.15.7:0/2541300713 wait complete. 2024-06-10T01:28:14.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.333+0000 7f3851909b80 20 client.29836 trim_cache size 0 max 16384 2024-06-10T01:28:14.328 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:28:14.329 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-10T01:28:14.329 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "e840de91-d4f2-4612-8c22-6780c0eb41a8", 2024-06-10T01:28:14.329 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-10T01:28:14.329 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:28:14.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.334+0000 7f3847086700 1 -- 172.21.15.7:0/1303079173 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f382c0a3060 msgr2=0x7f382c0a5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:14.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.334+0000 7f3847086700 1 --2- 172.21.15.7:0/1303079173 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f382c0a3060 0x7f382c0a5520 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3840124000 tx=0x7f3828019040 comp rx=0 tx=0).stop 2024-06-10T01:28:14.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.335+0000 7f3847086700 1 -- 172.21.15.7:0/1303079173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38401175c0 msgr2=0x7f3840123840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:14.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.335+0000 7f3847086700 1 --2- 172.21.15.7:0/1303079173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38401175c0 0x7f3840123840 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f383800e940 tx=0x7f383800ec50 comp rx=0 tx=0).stop 2024-06-10T01:28:14.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.335+0000 7f3847086700 1 -- 172.21.15.7:0/1303079173 shutdown_connections 2024-06-10T01:28:14.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.335+0000 7f3847086700 1 --2- 172.21.15.7:0/1303079173 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f382c0a3060 0x7f382c0a5520 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.335+0000 7f3847086700 1 --2- 172.21.15.7:0/1303079173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38401175c0 0x7f3840123840 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.335+0000 7f3847086700 1 --2- 172.21.15.7:0/1303079173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3840109a60 0x7f3840122dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.335+0000 7f3847086700 1 --2- 172.21.15.7:0/1303079173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f384010cc10 0x7f3840123300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:14.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.335+0000 7f3847086700 1 -- 172.21.15.7:0/1303079173 >> 172.21.15.7:0/1303079173 conn(0x7f384006d600 msgr2=0x7f3840118d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:14.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.335+0000 7f3847086700 1 -- 172.21.15.7:0/1303079173 shutdown_connections 2024-06-10T01:28:14.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:14.335+0000 7f3847086700 1 -- 172.21.15.7:0/1303079173 wait complete. 2024-06-10T01:28:14.555 INFO:tasks.workunit.client.0.smithi007.stdout: 10 218208 23.27 MB/sec execute 411 sec latency 147.697 ms 2024-06-10T01:28:14.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:14 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:28:14.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:14 smithi175 ceph-mon[36636]: scrub queued for path: / 2024-06-10T01:28:14.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:14 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:28:14.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:14 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:28:15.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:14 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:28:15.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:14 smithi007 ceph-mon[32398]: scrub queued for path: / 2024-06-10T01:28:15.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:14 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:28:15.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:14 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:28:15.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:14 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:28:15.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:14 smithi078 ceph-mon[36683]: scrub queued for path: / 2024-06-10T01:28:15.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:14 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:28:15.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:14 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:28:15.168 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.1 2024-06-10T01:28:15.169 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:28:15.242 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:28:15.557 INFO:tasks.workunit.client.0.smithi007.stdout: 10 218415 23.24 MB/sec execute 412 sec latency 339.833 ms 2024-06-10T01:28:15.908 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:28:15.924 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:28:15.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:15 smithi175 ceph-mon[36636]: pgmap v1101: 129 pgs: 129 active+clean; 745 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.8 MiB/s wr, 458 op/s 2024-06-10T01:28:15.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:15 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:28:15.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:15 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:28:15.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:15 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:28:15.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:15 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:28:16.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:15 smithi007 ceph-mon[32398]: pgmap v1101: 129 pgs: 129 active+clean; 745 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.8 MiB/s wr, 458 op/s 2024-06-10T01:28:16.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:15 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:28:16.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:15 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:28:16.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:15 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:28:16.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:15 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:28:16.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:15 smithi078 ceph-mon[36683]: pgmap v1101: 129 pgs: 129 active+clean; 745 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.8 MiB/s wr, 458 op/s 2024-06-10T01:28:16.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:15 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:28:16.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:15 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:28:16.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:15 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:28:16.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:15 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:28:16.558 INFO:tasks.workunit.client.0.smithi007.stdout: 10 218600 23.21 MB/sec execute 413 sec latency 813.326 ms 2024-06-10T01:28:16.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:16 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:28:16.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:16 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:28:16.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:16 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:28:16.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:16 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:28:16.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:16 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:28:16.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:16 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:28:16.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:16 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:28:16.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:16 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:28:17.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:16 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:28:17.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:16 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:28:17.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:16 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:28:17.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:16 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:28:17.558 INFO:tasks.workunit.client.0.smithi007.stdout: 10 218904 23.19 MB/sec execute 414 sec latency 147.448 ms 2024-06-10T01:28:17.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:17 smithi007 ceph-mon[32398]: pgmap v1102: 129 pgs: 129 active+clean; 772 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 13 MiB/s wr, 649 op/s 2024-06-10T01:28:17.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:17 smithi175 ceph-mon[36636]: pgmap v1102: 129 pgs: 129 active+clean; 772 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 13 MiB/s wr, 649 op/s 2024-06-10T01:28:18.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:17 smithi078 ceph-mon[36683]: pgmap v1102: 129 pgs: 129 active+clean; 772 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 13 MiB/s wr, 649 op/s 2024-06-10T01:28:18.558 INFO:tasks.workunit.client.0.smithi007.stdout: 10 219157 23.19 MB/sec execute 415 sec latency 155.113 ms 2024-06-10T01:28:19.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.494+0000 7f1bc2b56700 1 -- 172.21.15.7:0/3781848775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bbc10ef90 msgr2=0x7f1bbc10f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:19.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.497+0000 7f1bc2b56700 1 --2- 172.21.15.7:0/3781848775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bbc10ef90 0x7f1bbc10f370 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f1bac007510 tx=0x7f1bac00af90 comp rx=0 tx=0).stop 2024-06-10T01:28:19.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.502+0000 7f1bc2b56700 1 -- 172.21.15.7:0/3781848775 shutdown_connections 2024-06-10T01:28:19.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.502+0000 7f1bc2b56700 1 --2- 172.21.15.7:0/3781848775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bbc109e50 0x7f1bbc115d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.502+0000 7f1bc2b56700 1 --2- 172.21.15.7:0/3781848775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bbc10ef90 0x7f1bbc10f370 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.502+0000 7f1bc2b56700 1 --2- 172.21.15.7:0/3781848775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bbc109490 0x7f1bbc109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.502+0000 7f1bc2b56700 1 -- 172.21.15.7:0/3781848775 >> 172.21.15.7:0/3781848775 conn(0x7f1bbc06d470 msgr2=0x7f1bbc06d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:19.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.509+0000 7f1bc2b56700 1 -- 172.21.15.7:0/3781848775 shutdown_connections 2024-06-10T01:28:19.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.511+0000 7f1bc2b56700 1 -- 172.21.15.7:0/3781848775 wait complete. 2024-06-10T01:28:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.512+0000 7f1bc2b56700 1 Processor -- start 2024-06-10T01:28:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.513+0000 7f1bc2b56700 1 -- start start 2024-06-10T01:28:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.513+0000 7f1bc2b56700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bbc109490 0x7f1bbc19f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.514+0000 7f1bc2b56700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bbc109e50 0x7f1bbc19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.514+0000 7f1bc2b56700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bbc10ef90 0x7f1bbc1a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.514+0000 7f1bc2b56700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc118de0 con 0x7f1bbc10ef90 2024-06-10T01:28:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.514+0000 7f1bc2b56700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc118c60 con 0x7f1bbc109490 2024-06-10T01:28:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.514+0000 7f1bc2b56700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc118f60 con 0x7f1bbc109e50 2024-06-10T01:28:19.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.518+0000 7f1bc2355700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bbc10ef90 0x7f1bbc1a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:19.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.518+0000 7f1bc2355700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bbc10ef90 0x7f1bbc1a3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37830/0 (socket says 172.21.15.7:37830) 2024-06-10T01:28:19.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.518+0000 7f1bc2355700 1 -- 172.21.15.7:0/79632070 learned_addr learned my addr 172.21.15.7:0/79632070 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:19.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.519+0000 7f1bc1b54700 1 --2- 172.21.15.7:0/79632070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bbc109490 0x7f1bbc19f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:19.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.519+0000 7f1bc1353700 1 --2- 172.21.15.7:0/79632070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bbc109e50 0x7f1bbc19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:19.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.519+0000 7f1bc1b54700 1 -- 172.21.15.7:0/79632070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bbc109e50 msgr2=0x7f1bbc19f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:19.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.519+0000 7f1bc1b54700 1 --2- 172.21.15.7:0/79632070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bbc109e50 0x7f1bbc19f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.519+0000 7f1bc1b54700 1 -- 172.21.15.7:0/79632070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bbc10ef90 msgr2=0x7f1bbc1a3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:19.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.519+0000 7f1bc1b54700 1 --2- 172.21.15.7:0/79632070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bbc10ef90 0x7f1bbc1a3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.519+0000 7f1bc1b54700 1 -- 172.21.15.7:0/79632070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bbc1a4240 con 0x7f1bbc109490 2024-06-10T01:28:19.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.529+0000 7f1bc1b54700 1 --2- 172.21.15.7:0/79632070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bbc109490 0x7f1bbc19f230 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00c370 tx=0x7f1bac004210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:19.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.530+0000 7f1bb2ffd700 1 -- 172.21.15.7:0/79632070 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac018070 con 0x7f1bbc109490 2024-06-10T01:28:19.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.530+0000 7f1bb2ffd700 1 -- 172.21.15.7:0/79632070 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bac005510 con 0x7f1bbc109490 2024-06-10T01:28:19.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.531+0000 7f1bb2ffd700 1 -- 172.21.15.7:0/79632070 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac004920 con 0x7f1bbc109490 2024-06-10T01:28:19.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.533+0000 7f1bc2b56700 1 -- 172.21.15.7:0/79632070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bbc1a44d0 con 0x7f1bbc109490 2024-06-10T01:28:19.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.533+0000 7f1bc2b56700 1 -- 172.21.15.7:0/79632070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bbc1abdb0 con 0x7f1bbc109490 2024-06-10T01:28:19.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.540+0000 7f1bce63bb80 1 Processor -- start 2024-06-10T01:28:19.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.540+0000 7f1bce63bb80 1 -- start start 2024-06-10T01:28:19.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.541+0000 7f1bce63bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560186e10530 0x560186e10910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:19.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.541+0000 7f1bce63bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560186e10e50 0x560186e14ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:19.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.541+0000 7f1bce63bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560186e151e0 0x560186e15640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:19.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.541+0000 7f1bce63bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560186d59810 con 0x560186e10530 2024-06-10T01:28:19.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.541+0000 7f1bce63bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560186d01720 con 0x560186e10e50 2024-06-10T01:28:19.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.541+0000 7f1bce63bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560186d557d0 con 0x560186e151e0 2024-06-10T01:28:19.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.541+0000 7f1bc1353700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560186e151e0 0x560186e15640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:19.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.541+0000 7f1bc1353700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560186e151e0 0x560186e15640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:50014/0 (socket says 172.21.15.7:50014) 2024-06-10T01:28:19.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.541+0000 7f1bc1353700 1 -- 172.21.15.7:0/2464217345 learned_addr learned my addr 172.21.15.7:0/2464217345 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:19.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.541+0000 7f1bc2355700 1 --2- 172.21.15.7:0/2464217345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560186e10530 0x560186e10910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:19.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.542+0000 7f1bc1353700 1 -- 172.21.15.7:0/2464217345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560186e10e50 msgr2=0x560186e14ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:19.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.542+0000 7f1bc1353700 1 --2- 172.21.15.7:0/2464217345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560186e10e50 0x560186e14ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.542+0000 7f1bc1353700 1 -- 172.21.15.7:0/2464217345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560186e10530 msgr2=0x560186e10910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:19.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.542+0000 7f1bc1353700 1 --2- 172.21.15.7:0/2464217345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560186e10530 0x560186e10910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.542+0000 7f1bc1353700 1 -- 172.21.15.7:0/2464217345 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560186cf7f50 con 0x560186e151e0 2024-06-10T01:28:19.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.543+0000 7f1bc1353700 1 --2- 172.21.15.7:0/2464217345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560186e151e0 0x560186e15640 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8009bf0 tx=0x7f1bb8009f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f73ceb75a43100d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:19.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.544+0000 7f1bb2ffd700 1 -- 172.21.15.7:0/79632070 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f1bac005030 con 0x7f1bbc109490 2024-06-10T01:28:19.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.545+0000 7f1bb2ffd700 1 --2- 172.21.15.7:0/79632070 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1ba80a2c50 0x7f1ba80a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:19.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.545+0000 7f1bb2ffd700 1 -- 172.21.15.7:0/79632070 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f1bac0044b0 con 0x7f1bbc109490 2024-06-10T01:28:19.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.546+0000 7f1ba7fff700 1 -- 172.21.15.7:0/2464217345 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8016070 con 0x560186e151e0 2024-06-10T01:28:19.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.546+0000 7f1bc2355700 1 --2- 172.21.15.7:0/79632070 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1ba80a2c50 0x7f1ba80a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:19.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.546+0000 7f1ba7fff700 1 -- 172.21.15.7:0/2464217345 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bb8004830 con 0x560186e151e0 2024-06-10T01:28:19.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.547+0000 7f1bce63bb80 1 -- 172.21.15.7:0/2464217345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560186e151e0 msgr2=0x560186e15640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:19.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.547+0000 7f1bce63bb80 1 --2- 172.21.15.7:0/2464217345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560186e151e0 0x560186e15640 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8009bf0 tx=0x7f1bb8009f00 comp rx=0 tx=0).stop 2024-06-10T01:28:19.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.547+0000 7f1bc2355700 1 --2- 172.21.15.7:0/79632070 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1ba80a2c50 0x7f1ba80a5110 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4009fd0 tx=0x7f1bb400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:19.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.548+0000 7f1bce63bb80 1 -- 172.21.15.7:0/2464217345 shutdown_connections 2024-06-10T01:28:19.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.548+0000 7f1bce63bb80 1 --2- 172.21.15.7:0/2464217345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560186e151e0 0x560186e15640 secure :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8009bf0 tx=0x7f1bb8009f00 comp rx=0 tx=0).stop 2024-06-10T01:28:19.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.548+0000 7f1bce63bb80 1 --2- 172.21.15.7:0/2464217345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560186e10530 0x560186e10910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.548+0000 7f1bce63bb80 1 --2- 172.21.15.7:0/2464217345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560186e10e50 0x560186e14ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.549+0000 7f1bce63bb80 1 -- 172.21.15.7:0/2464217345 >> 172.21.15.7:0/2464217345 conn(0x560186d32cc0 msgr2=0x560186e02af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:19.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.549+0000 7f1bce63bb80 1 -- 172.21.15.7:0/2464217345 shutdown_connections 2024-06-10T01:28:19.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.549+0000 7f1bce63bb80 1 -- 172.21.15.7:0/2464217345 wait complete. 2024-06-10T01:28:19.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.550+0000 7f1bce63bb80 1 -- start start 2024-06-10T01:28:19.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.551+0000 7f1bce63bb80 1 Processor -- start 2024-06-10T01:28:19.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.552+0000 7f1bce63bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560186e10530 0x560186e1c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:19.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.552+0000 7f1bce63bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560186e10e50 0x560186e1c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:19.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.552+0000 7f1bce63bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560186e0f650 0x560186e0fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:19.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.552+0000 7f1bce63bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560186e08000 con 0x560186e0f650 2024-06-10T01:28:19.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.552+0000 7f1bce63bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560186cf9bf0 con 0x560186e10e50 2024-06-10T01:28:19.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.552+0000 7f1bce63bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560186e08180 con 0x560186e10530 2024-06-10T01:28:19.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.552+0000 7f1bc1b54700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560186e10530 0x560186e1c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:19.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.552+0000 7f1bc1b54700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560186e10530 0x560186e1c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:50022/0 (socket says 172.21.15.7:50022) 2024-06-10T01:28:19.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.552+0000 7f1bc1b54700 1 -- 172.21.15.7:0/4190015342 learned_addr learned my addr 172.21.15.7:0/4190015342 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:19.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.553+0000 7f1bc1b54700 1 -- 172.21.15.7:0/4190015342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560186e10e50 msgr2=0x560186e1c9e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T01:28:19.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.553+0000 7f1bc1b54700 1 --2- 172.21.15.7:0/4190015342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560186e10e50 0x560186e1c9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.553+0000 7f1bc1b54700 1 -- 172.21.15.7:0/4190015342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560186e0f650 msgr2=0x560186e0fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:19.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.553+0000 7f1bc1b54700 1 --2- 172.21.15.7:0/4190015342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560186e0f650 0x560186e0fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.553+0000 7f1bce63bb80 10 client.0 _ll_register_callbacks cb 0x560186cf3920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:28:19.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.553+0000 7f1bc1b54700 1 -- 172.21.15.7:0/4190015342 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560186ca88d0 con 0x560186e10530 2024-06-10T01:28:19.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.554+0000 7f1bc1b54700 1 --2- 172.21.15.7:0/4190015342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560186e10530 0x560186e1c4a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1bac006010 tx=0x7f1bac098240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92574932f7b2a365 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:19.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.554+0000 7f1bce63bb80 1 -- 172.21.15.7:0/4190015342 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560186d66d10 con 0x560186e10530 2024-06-10T01:28:19.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.554+0000 7f1ba57fa700 10 client.29845 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:28:19.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.554+0000 7f1ba57fa700 1 -- 172.21.15.7:0/4190015342 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac09a170 con 0x560186e10530 2024-06-10T01:28:19.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.554+0000 7f1ba57fa700 1 -- 172.21.15.7:0/4190015342 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f1bac099810 con 0x560186e10530 2024-06-10T01:28:19.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.555+0000 7f1ba57fa700 1 -- 172.21.15.7:0/4190015342 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f1bac09a310 con 0x560186e10530 2024-06-10T01:28:19.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.555+0000 7f1bce63bb80 10 client.29845 fetch_fsmap learned FSMap version 55 2024-06-10T01:28:19.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.556+0000 7f1bce63bb80 1 -- 172.21.15.7:0/4190015342 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({fsmap=55}) -- 0x560186e08000 con 0x560186e10530 2024-06-10T01:28:19.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.557+0000 7f1ba57fa700 1 -- 172.21.15.7:0/4190015342 <== mon.1 v2:172.21.15.175:3300/0 4 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f1bac01b070 con 0x560186e10530 2024-06-10T01:28:19.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.557+0000 7f1bce63bb80 10 client.29845 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:28:19.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.557+0000 7f1bce63bb80 10 client.29845 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:28:19.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.557+0000 7f1bce63bb80 20 client.29845 populate_metadata read hostname 'smithi007' 2024-06-10T01:28:19.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.557+0000 7f1bce63bb80 1 --2- 172.21.15.7:0/4190015342 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560186e32b40 0x560186e34fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:19.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.557+0000 7f1bce63bb80 4 client.29845 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:28:19.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.557+0000 7f1bce63bb80 1 -- 172.21.15.7:0/4190015342 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560186d61290 con 0x560186e32b40 2024-06-10T01:28:19.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.558+0000 7f1bc1353700 1 --2- 172.21.15.7:0/4190015342 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560186e32b40 0x560186e34fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:19.557 INFO:tasks.workunit.client.0.smithi007.stdout: 10 219457 23.19 MB/sec execute 416 sec latency 135.183 ms 2024-06-10T01:28:19.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.565+0000 7f1bc1353700 1 --2- 172.21.15.7:0/4190015342 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560186e32b40 0x560186e34fd0 crc :-1 s=READY pgs=501 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16c930bf8a1a6ee1 server_cookie=e616fab844f5a7a in_seq=0 out_seq=0 2024-06-10T01:28:19.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.565+0000 7f1ba57fa700 10 client.29845 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:19.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.566+0000 7f1ba57fa700 1 -- 172.21.15.7:0/4190015342 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560186d61290 con 0x560186e32b40 2024-06-10T01:28:19.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.566+0000 7f1ba57fa700 10 client.29845 handle_command_reply: tid=0 2024-06-10T01:28:19.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.566+0000 7f1bce63bb80 1 client.29845 shutdown 2024-06-10T01:28:19.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.567+0000 7f1bce63bb80 1 -- 172.21.15.7:0/4190015342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560186e10530 msgr2=0x560186e1c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:19.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.567+0000 7f1bce63bb80 1 --2- 172.21.15.7:0/4190015342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560186e10530 0x560186e1c4a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1bac006010 tx=0x7f1bac098240 comp rx=0 tx=0).stop 2024-06-10T01:28:19.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.568+0000 7f1bce63bb80 1 -- 172.21.15.7:0/4190015342 shutdown_connections 2024-06-10T01:28:19.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.568+0000 7f1bce63bb80 1 --2- 172.21.15.7:0/4190015342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560186e0f650 0x560186e0fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.568+0000 7f1bce63bb80 1 --2- 172.21.15.7:0/4190015342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560186e10530 0x560186e1c4a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.568+0000 7f1bce63bb80 1 --2- 172.21.15.7:0/4190015342 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560186e32b40 0x560186e34fd0 crc :-1 s=READY pgs=501 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.568+0000 7f1bce63bb80 1 --2- 172.21.15.7:0/4190015342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560186e10e50 0x560186e1c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.568+0000 7f1bce63bb80 1 -- 172.21.15.7:0/4190015342 >> 172.21.15.7:0/4190015342 conn(0x560186d32cc0 msgr2=0x560186e06b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:19.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.568+0000 7f1ba57fa700 0 client.29845 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:19.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.568+0000 7f1bce63bb80 1 -- 172.21.15.7:0/4190015342 shutdown_connections 2024-06-10T01:28:19.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.568+0000 7f1bce63bb80 1 -- 172.21.15.7:0/4190015342 wait complete. 2024-06-10T01:28:19.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.568+0000 7f1bce63bb80 20 client.29845 trim_cache size 0 max 16384 2024-06-10T01:28:19.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.604+0000 7f1bce63bb80 1 Processor -- start 2024-06-10T01:28:19.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.604+0000 7f1bce63bb80 1 -- start start 2024-06-10T01:28:19.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.604+0000 7f1bce63bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560186d32cc0 0x560186e02bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:19.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.604+0000 7f1bce63bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560186e0f650 0x560186e3a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:19.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.604+0000 7f1bce63bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560186e10530 0x560186e3ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:19.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.604+0000 7f1bce63bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc1b6c50 con 0x560186d32cc0 2024-06-10T01:28:19.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.604+0000 7f1bce63bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc1b44e0 con 0x560186e10530 2024-06-10T01:28:19.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.604+0000 7f1bce63bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc1b2330 con 0x560186e0f650 2024-06-10T01:28:19.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.605+0000 7f1bc1b54700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560186d32cc0 0x560186e02bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:19.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.605+0000 7f1bc1b54700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560186d32cc0 0x560186e02bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37864/0 (socket says 172.21.15.7:37864) 2024-06-10T01:28:19.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.605+0000 7f1bc1b54700 1 -- 172.21.15.7:0/169089786 learned_addr learned my addr 172.21.15.7:0/169089786 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:19.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.605+0000 7f1bc1353700 1 --2- 172.21.15.7:0/169089786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560186e0f650 0x560186e3a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:19.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.605+0000 7f1bc2355700 1 --2- 172.21.15.7:0/169089786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560186e10530 0x560186e3ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:19.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.606+0000 7f1bc1b54700 1 -- 172.21.15.7:0/169089786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560186e0f650 msgr2=0x560186e3a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:19.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.606+0000 7f1bc1b54700 1 --2- 172.21.15.7:0/169089786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560186e0f650 0x560186e3a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.606+0000 7f1bc1b54700 1 -- 172.21.15.7:0/169089786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560186e10530 msgr2=0x560186e3ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:19.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.606+0000 7f1bc1b54700 1 --2- 172.21.15.7:0/169089786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560186e10530 0x560186e3ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.606+0000 7f1bc1b54700 1 -- 172.21.15.7:0/169089786 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560186e11250 con 0x560186d32cc0 2024-06-10T01:28:19.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.606+0000 7f1bc1b54700 1 --2- 172.21.15.7:0/169089786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560186d32cc0 0x560186e02bf0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f1bac09a9c0 tx=0x7f1bac028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ebd70ed538d94ee server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:19.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.607+0000 7f1ba77fe700 1 -- 172.21.15.7:0/169089786 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac0b5070 con 0x560186d32cc0 2024-06-10T01:28:19.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.607+0000 7f1ba77fe700 1 -- 172.21.15.7:0/169089786 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560186d66d10 con 0x560186d32cc0 2024-06-10T01:28:19.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.607+0000 7f1ba77fe700 1 -- 172.21.15.7:0/169089786 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac09a4b0 con 0x560186d32cc0 2024-06-10T01:28:19.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.608+0000 7f1bce63bb80 1 -- 172.21.15.7:0/169089786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560186d32cc0 msgr2=0x560186e02bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:19.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.608+0000 7f1bce63bb80 1 --2- 172.21.15.7:0/169089786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560186d32cc0 0x560186e02bf0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f1bac09a9c0 tx=0x7f1bac028000 comp rx=0 tx=0).stop 2024-06-10T01:28:19.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.609+0000 7f1bce63bb80 1 -- 172.21.15.7:0/169089786 shutdown_connections 2024-06-10T01:28:19.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.609+0000 7f1bce63bb80 1 --2- 172.21.15.7:0/169089786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560186e10530 0x560186e3ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.609+0000 7f1bce63bb80 1 --2- 172.21.15.7:0/169089786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560186d32cc0 0x560186e02bf0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.609+0000 7f1bce63bb80 1 --2- 172.21.15.7:0/169089786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560186e0f650 0x560186e3a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.609+0000 7f1bce63bb80 1 -- 172.21.15.7:0/169089786 >> 172.21.15.7:0/169089786 conn(0x560186e193c0 msgr2=0x560186e024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:19.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.609+0000 7f1bce63bb80 1 -- 172.21.15.7:0/169089786 shutdown_connections 2024-06-10T01:28:19.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.609+0000 7f1bce63bb80 1 -- 172.21.15.7:0/169089786 wait complete. 2024-06-10T01:28:19.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.609+0000 7f1bce63bb80 1 -- start start 2024-06-10T01:28:19.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.610+0000 7f1bce63bb80 1 Processor -- start 2024-06-10T01:28:19.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.610+0000 7f1bce63bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560186d32cc0 0x560186e0e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:19.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.610+0000 7f1bce63bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560186e0f650 0x560186e0ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:19.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.610+0000 7f1bce63bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560186e10530 0x560186e0bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:19.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.610+0000 7f1bce63bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc1b29a0 con 0x560186d32cc0 2024-06-10T01:28:19.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.610+0000 7f1bce63bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc1b3e10 con 0x560186e0f650 2024-06-10T01:28:19.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.610+0000 7f1bce63bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc1b3070 con 0x560186e10530 2024-06-10T01:28:19.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.610+0000 7f1bce63bb80 10 client.0 _ll_register_callbacks cb 0x560186cf3920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:28:19.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.611+0000 7f1bc1353700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560186e0f650 0x560186e0ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:19.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.611+0000 7f1bc1353700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560186e0f650 0x560186e0ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:40862/0 (socket says 172.21.15.7:40862) 2024-06-10T01:28:19.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.611+0000 7f1bc1353700 1 -- 172.21.15.7:0/3595673600 learned_addr learned my addr 172.21.15.7:0/3595673600 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:19.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.611+0000 7f1bc1b54700 1 --2- 172.21.15.7:0/3595673600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560186d32cc0 0x560186e0e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:19.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.611+0000 7f1bc2355700 1 --2- 172.21.15.7:0/3595673600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560186e10530 0x560186e0bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:19.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.611+0000 7f1bc1353700 1 -- 172.21.15.7:0/3595673600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560186e10530 msgr2=0x560186e0bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:19.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.611+0000 7f1bc1353700 1 --2- 172.21.15.7:0/3595673600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560186e10530 0x560186e0bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.611+0000 7f1bc1353700 1 -- 172.21.15.7:0/3595673600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560186d32cc0 msgr2=0x560186e0e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:19.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.611+0000 7f1bc1353700 1 --2- 172.21.15.7:0/3595673600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560186d32cc0 0x560186e0e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.611+0000 7f1bc1353700 1 -- 172.21.15.7:0/3595673600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560186e0c3b0 con 0x560186e0f650 2024-06-10T01:28:19.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.612+0000 7f1bc1353700 1 --2- 172.21.15.7:0/3595673600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560186e0f650 0x560186e0ec80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800e680 tx=0x7f1bb800e950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c557959adcfe9cc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:19.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.612+0000 7f1ba57fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:28:19.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.612+0000 7f1ba57fa700 1 -- 172.21.15.7:0/3595673600 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8016040 con 0x560186e0f650 2024-06-10T01:28:19.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.612+0000 7f1ba57fa700 1 -- 172.21.15.7:0/3595673600 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f1bb8026430 con 0x560186e0f650 2024-06-10T01:28:19.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.612+0000 7f1bce63bb80 1 -- 172.21.15.7:0/3595673600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560186d66d10 con 0x560186e0f650 2024-06-10T01:28:19.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.612+0000 7f1bc2355700 1 --2- 172.21.15.7:0/3595673600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560186e10530 0x560186e0bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:28:19.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.613+0000 7f1ba57fa700 1 -- 172.21.15.7:0/3595673600 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f1bb8034050 con 0x560186e0f650 2024-06-10T01:28:19.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.613+0000 7f1bce63bb80 10 client.29936 fetch_fsmap learned FSMap version 55 2024-06-10T01:28:19.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.613+0000 7f1bce63bb80 1 -- 172.21.15.7:0/3595673600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x560186e9d3d0 con 0x560186e0f650 2024-06-10T01:28:19.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.613+0000 7f1ba57fa700 1 -- 172.21.15.7:0/3595673600 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560186d66d10 con 0x560186e0f650 2024-06-10T01:28:19.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.613+0000 7f1ba57fa700 1 -- 172.21.15.7:0/3595673600 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f1bb800f040 con 0x560186e0f650 2024-06-10T01:28:19.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.613+0000 7f1bce63bb80 10 client.29936 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:28:19.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.613+0000 7f1bce63bb80 10 client.29936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:28:19.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.613+0000 7f1bce63bb80 20 client.29936 populate_metadata read hostname 'smithi007' 2024-06-10T01:28:19.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.613+0000 7f1bce63bb80 1 --2- 172.21.15.7:0/3595673600 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560186e10e50 0x560186e9f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:19.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.613+0000 7f1bce63bb80 4 client.29936 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:28:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.613+0000 7f1bce63bb80 1 -- 172.21.15.7:0/3595673600 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560186bdb250 con 0x560186e10e50 2024-06-10T01:28:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.614+0000 7f1bc1b54700 1 --2- 172.21.15.7:0/3595673600 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560186e10e50 0x560186e9f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.614+0000 7f1bc1b54700 1 --2- 172.21.15.7:0/3595673600 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560186e10e50 0x560186e9f760 crc :-1 s=READY pgs=502 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29402c7d435180be server_cookie=8be402919d3b8d98 in_seq=0 out_seq=0 2024-06-10T01:28:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.614+0000 7f1ba57fa700 10 client.29936 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:19.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.621+0000 7f1ba57fa700 1 -- 172.21.15.7:0/3595673600 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560186bdb250 con 0x560186e10e50 2024-06-10T01:28:19.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.621+0000 7f1ba57fa700 10 client.29936 handle_command_reply: tid=0 2024-06-10T01:28:19.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.621+0000 7f1bce63bb80 1 client.29936 shutdown 2024-06-10T01:28:19.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.622+0000 7f1bce63bb80 1 -- 172.21.15.7:0/3595673600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560186e0f650 msgr2=0x560186e0ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:19.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.622+0000 7f1bce63bb80 1 --2- 172.21.15.7:0/3595673600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560186e0f650 0x560186e0ec80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800e680 tx=0x7f1bb800e950 comp rx=0 tx=0).stop 2024-06-10T01:28:19.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.622+0000 7f1bce63bb80 1 -- 172.21.15.7:0/3595673600 shutdown_connections 2024-06-10T01:28:19.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.622+0000 7f1bce63bb80 1 --2- 172.21.15.7:0/3595673600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560186e10530 0x560186e0bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.622+0000 7f1bce63bb80 1 --2- 172.21.15.7:0/3595673600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560186d32cc0 0x560186e0e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.622+0000 7f1bce63bb80 1 --2- 172.21.15.7:0/3595673600 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560186e10e50 0x560186e9f760 crc :-1 s=READY pgs=502 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.622+0000 7f1bce63bb80 1 --2- 172.21.15.7:0/3595673600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560186e0f650 0x560186e0ec80 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.622+0000 7f1bce63bb80 1 -- 172.21.15.7:0/3595673600 >> 172.21.15.7:0/3595673600 conn(0x560186e193c0 msgr2=0x560186dfbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:19.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.622+0000 7f1ba57fa700 0 client.29936 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:19.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.622+0000 7f1bce63bb80 1 -- 172.21.15.7:0/3595673600 shutdown_connections 2024-06-10T01:28:19.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.622+0000 7f1bce63bb80 1 -- 172.21.15.7:0/3595673600 wait complete. 2024-06-10T01:28:19.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.623+0000 7f1bce63bb80 20 client.29936 trim_cache size 0 max 16384 2024-06-10T01:28:19.617 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:28:19.618 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T01:28:19.618 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-10T01:28:19.618 INFO:teuthology.orchestra.run.smithi007.stdout: "e840de91-d4f2-4612-8c22-6780c0eb41a8": { 2024-06-10T01:28:19.618 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-10T01:28:19.618 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "e840de91-d4f2-4612-8c22-6780c0eb41a8", 2024-06-10T01:28:19.618 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-10T01:28:19.618 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:28:19.618 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:28:19.618 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:28:19.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.624+0000 7f1bc2b56700 1 -- 172.21.15.7:0/79632070 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1ba80a2c50 msgr2=0x7f1ba80a5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:19.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.624+0000 7f1bc2b56700 1 --2- 172.21.15.7:0/79632070 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1ba80a2c50 0x7f1ba80a5110 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4009fd0 tx=0x7f1bb400b040 comp rx=0 tx=0).stop 2024-06-10T01:28:19.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.624+0000 7f1bc2b56700 1 -- 172.21.15.7:0/79632070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bbc109490 msgr2=0x7f1bbc19f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:19.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.624+0000 7f1bc2b56700 1 --2- 172.21.15.7:0/79632070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bbc109490 0x7f1bbc19f230 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00c370 tx=0x7f1bac004210 comp rx=0 tx=0).stop 2024-06-10T01:28:19.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.624+0000 7f1bc2b56700 1 -- 172.21.15.7:0/79632070 shutdown_connections 2024-06-10T01:28:19.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.624+0000 7f1bc2b56700 1 --2- 172.21.15.7:0/79632070 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1ba80a2c50 0x7f1ba80a5110 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.624+0000 7f1bc2b56700 1 --2- 172.21.15.7:0/79632070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bbc10ef90 0x7f1bbc1a3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.624+0000 7f1bc2b56700 1 --2- 172.21.15.7:0/79632070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bbc109490 0x7f1bbc19f230 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.624+0000 7f1bc2b56700 1 --2- 172.21.15.7:0/79632070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bbc109e50 0x7f1bbc19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.624+0000 7f1bc2b56700 1 -- 172.21.15.7:0/79632070 >> 172.21.15.7:0/79632070 conn(0x7f1bbc06d470 msgr2=0x7f1bbc10ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:19.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.624+0000 7f1bc2b56700 1 -- 172.21.15.7:0/79632070 shutdown_connections 2024-06-10T01:28:19.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.624+0000 7f1bc2b56700 1 -- 172.21.15.7:0/79632070 wait complete. 2024-06-10T01:28:19.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.657+0000 7ffaf8764700 1 -- 172.21.15.7:0/2554376570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffaf010a6d0 msgr2=0x7ffaf010aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:19.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.657+0000 7ffaf8764700 1 --2- 172.21.15.7:0/2554376570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffaf010a6d0 0x7ffaf010aab0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7ffaec009990 tx=0x7ffaec009ca0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.657+0000 7ffaf8764700 1 -- 172.21.15.7:0/2554376570 shutdown_connections 2024-06-10T01:28:19.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.657+0000 7ffaf8764700 1 --2- 172.21.15.7:0/2554376570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffaf0105090 0x7ffaf01115a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.657+0000 7ffaf8764700 1 --2- 172.21.15.7:0/2554376570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffaf010a6d0 0x7ffaf010aab0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.657+0000 7ffaf8764700 1 --2- 172.21.15.7:0/2554376570 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffaf01046d0 0x7ffaf0104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.657+0000 7ffaf8764700 1 -- 172.21.15.7:0/2554376570 >> 172.21.15.7:0/2554376570 conn(0x7ffaf01001c0 msgr2=0x7ffaf01025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:19.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.658+0000 7ffaf8764700 1 -- 172.21.15.7:0/2554376570 shutdown_connections 2024-06-10T01:28:19.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.658+0000 7ffaf8764700 1 -- 172.21.15.7:0/2554376570 wait complete. 2024-06-10T01:28:19.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.659+0000 7ffaf8764700 1 Processor -- start 2024-06-10T01:28:19.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.659+0000 7ffaf8764700 1 -- start start 2024-06-10T01:28:19.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.659+0000 7ffaf8764700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffaf01046d0 0x7ffaf00769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:19.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.659+0000 7ffaf8764700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffaf0105090 0x7ffaf0076ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:19.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.659+0000 7ffaf8764700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffaf010a6d0 0x7ffaf0077420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:19.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.659+0000 7ffaf8764700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf0114a60 con 0x7ffaf010a6d0 2024-06-10T01:28:19.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.659+0000 7ffaf8764700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf01148e0 con 0x7ffaf01046d0 2024-06-10T01:28:19.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.659+0000 7ffaf8764700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf0114be0 con 0x7ffaf0105090 2024-06-10T01:28:19.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.659+0000 7ffaf6500700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffaf01046d0 0x7ffaf00769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:19.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.660+0000 7ffaf6500700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffaf01046d0 0x7ffaf00769a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:40882/0 (socket says 172.21.15.7:40882) 2024-06-10T01:28:19.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.660+0000 7ffaf6500700 1 -- 172.21.15.7:0/343323712 learned_addr learned my addr 172.21.15.7:0/343323712 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:19.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.660+0000 7ffaf6d01700 1 --2- 172.21.15.7:0/343323712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffaf010a6d0 0x7ffaf0077420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:19.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.660+0000 7ffaf5cff700 1 --2- 172.21.15.7:0/343323712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffaf0105090 0x7ffaf0076ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:19.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.660+0000 7ffaf6500700 1 -- 172.21.15.7:0/343323712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffaf0105090 msgr2=0x7ffaf0076ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:19.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.660+0000 7ffaf6500700 1 --2- 172.21.15.7:0/343323712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffaf0105090 0x7ffaf0076ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.660+0000 7ffaf6500700 1 -- 172.21.15.7:0/343323712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffaf010a6d0 msgr2=0x7ffaf0077420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:19.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.660+0000 7ffaf6500700 1 --2- 172.21.15.7:0/343323712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffaf010a6d0 0x7ffaf0077420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.660+0000 7ffaf6500700 1 -- 172.21.15.7:0/343323712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaf01ab5a0 con 0x7ffaf01046d0 2024-06-10T01:28:19.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.660+0000 7ffaf6500700 1 --2- 172.21.15.7:0/343323712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffaf01046d0 0x7ffaf00769a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ffaec005f50 tx=0x7ffaec00bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:19.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.660+0000 7ffae77fe700 1 -- 172.21.15.7:0/343323712 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaec027070 con 0x7ffaf01046d0 2024-06-10T01:28:19.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.660+0000 7ffae77fe700 1 -- 172.21.15.7:0/343323712 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaec003d70 con 0x7ffaf01046d0 2024-06-10T01:28:19.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.660+0000 7ffaf8764700 1 -- 172.21.15.7:0/343323712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaf01ab830 con 0x7ffaf01046d0 2024-06-10T01:28:19.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.660+0000 7ffae77fe700 1 -- 172.21.15.7:0/343323712 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaec00f460 con 0x7ffaf01046d0 2024-06-10T01:28:19.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.660+0000 7ffaf8764700 1 -- 172.21.15.7:0/343323712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaf01abc80 con 0x7ffaf01046d0 2024-06-10T01:28:19.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.661+0000 7ffaf8764700 1 -- 172.21.15.7:0/343323712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaf0105e10 con 0x7ffaf01046d0 2024-06-10T01:28:19.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.663+0000 7ffae77fe700 1 -- 172.21.15.7:0/343323712 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7ffaec02a070 con 0x7ffaf01046d0 2024-06-10T01:28:19.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.664+0000 7ffae77fe700 1 --2- 172.21.15.7:0/343323712 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ffadc0a2c20 0x7ffadc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:19.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.664+0000 7ffae77fe700 1 -- 172.21.15.7:0/343323712 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7ffaec0d0680 con 0x7ffaf01046d0 2024-06-10T01:28:19.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.664+0000 7ffaf5cff700 1 --2- 172.21.15.7:0/343323712 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ffadc0a2c20 0x7ffadc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:19.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.665+0000 7ffaf5cff700 1 --2- 172.21.15.7:0/343323712 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ffadc0a2c20 0x7ffadc0a50e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0078160 tx=0x7ffae000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:19.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.668+0000 7ffae77fe700 1 -- 172.21.15.7:0/343323712 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ffaec031070 con 0x7ffaf01046d0 2024-06-10T01:28:19.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:19 smithi007 ceph-mon[32398]: pgmap v1103: 129 pgs: 129 active+clean; 772 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 15 MiB/s wr, 797 op/s 2024-06-10T01:28:19.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:19 smithi175 ceph-mon[36636]: pgmap v1103: 129 pgs: 129 active+clean; 772 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 15 MiB/s wr, 797 op/s 2024-06-10T01:28:19.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.995+0000 7ffaf8764700 1 -- 172.21.15.7:0/343323712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffaf002dca0 con 0x7ffaf01046d0 2024-06-10T01:28:19.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.996+0000 7ffae77fe700 1 -- 172.21.15.7:0/343323712 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11852 (secure 0 0 0) 0x7ffaec013070 con 0x7ffaf01046d0 2024-06-10T01:28:19.990 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:28:19.990 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:27:44.542390+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29813":{"gid":29813,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/138896429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":138896429},{"type":"v1","addr":"172.21.15.175:6835","nonce":138896429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24451,24434,24436]},"id":1}]} 2024-06-10T01:28:19.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.999+0000 7ffaf8764700 1 -- 172.21.15.7:0/343323712 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ffadc0a2c20 msgr2=0x7ffadc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:19.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.999+0000 7ffaf8764700 1 --2- 172.21.15.7:0/343323712 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ffadc0a2c20 0x7ffadc0a50e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0078160 tx=0x7ffae000b410 comp rx=0 tx=0).stop 2024-06-10T01:28:19.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.999+0000 7ffaf8764700 1 -- 172.21.15.7:0/343323712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffaf01046d0 msgr2=0x7ffaf00769a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:19.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.999+0000 7ffaf8764700 1 --2- 172.21.15.7:0/343323712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffaf01046d0 0x7ffaf00769a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ffaec005f50 tx=0x7ffaec00bab0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.999+0000 7ffaf8764700 1 -- 172.21.15.7:0/343323712 shutdown_connections 2024-06-10T01:28:19.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.999+0000 7ffaf8764700 1 --2- 172.21.15.7:0/343323712 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ffadc0a2c20 0x7ffadc0a50e0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.999+0000 7ffaf8764700 1 --2- 172.21.15.7:0/343323712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffaf010a6d0 0x7ffaf0077420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.999+0000 7ffaf8764700 1 --2- 172.21.15.7:0/343323712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffaf01046d0 0x7ffaf00769a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.999+0000 7ffaf8764700 1 --2- 172.21.15.7:0/343323712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffaf0105090 0x7ffaf0076ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:19.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.999+0000 7ffaf8764700 1 -- 172.21.15.7:0/343323712 >> 172.21.15.7:0/343323712 conn(0x7ffaf01001c0 msgr2=0x7ffaf01007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:19.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.999+0000 7ffaf8764700 1 -- 172.21.15.7:0/343323712 shutdown_connections 2024-06-10T01:28:19.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:19.999+0000 7ffaf8764700 1 -- 172.21.15.7:0/343323712 wait complete. 2024-06-10T01:28:19.994 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 55 2024-06-10T01:28:20.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:19 smithi078 ceph-mon[36683]: pgmap v1103: 129 pgs: 129 active+clean; 772 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 15 MiB/s wr, 797 op/s 2024-06-10T01:28:20.558 INFO:tasks.workunit.client.0.smithi007.stdout: 10 220004 23.19 MB/sec execute 417 sec latency 109.023 ms 2024-06-10T01:28:20.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:20 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/343323712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:28:20.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:20 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/343323712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:28:21.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:20 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/343323712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:28:21.538 INFO:tasks.cephfs.filesystem:scrub status for tag:e840de91-d4f2-4612-8c22-6780c0eb41a8 - {'path': '/', 'tag': 'e840de91-d4f2-4612-8c22-6780c0eb41a8', 'options': 'recursive,force'} 2024-06-10T01:28:21.558 INFO:tasks.workunit.client.0.smithi007.stdout: 10 220531 23.20 MB/sec execute 418 sec latency 78.567 ms 2024-06-10T01:28:21.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:21 smithi007 ceph-mon[32398]: pgmap v1104: 129 pgs: 129 active+clean; 749 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 15 MiB/s wr, 802 op/s 2024-06-10T01:28:21.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:21 smithi175 ceph-mon[36636]: pgmap v1104: 129 pgs: 129 active+clean; 749 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 15 MiB/s wr, 802 op/s 2024-06-10T01:28:22.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:21 smithi078 ceph-mon[36683]: pgmap v1104: 129 pgs: 129 active+clean; 749 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 15 MiB/s wr, 802 op/s 2024-06-10T01:28:22.210 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db / --timeout 90 --expiration 142.1 --await-for 45.0' 2024-06-10T01:28:22.211 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db / --timeout 90 --expiration 142.1 --await-for 45.0 2024-06-10T01:28:22.558 INFO:tasks.workunit.client.0.smithi007.stdout: 10 221183 23.22 MB/sec execute 419 sec latency 108.593 ms 2024-06-10T01:28:22.584 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:28:22.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:22 smithi175 ceph-mon[36636]: pgmap v1105: 129 pgs: 129 active+clean; 687 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 16 MiB/s wr, 1.01k op/s 2024-06-10T01:28:23.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:22 smithi007 ceph-mon[32398]: pgmap v1105: 129 pgs: 129 active+clean; 687 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 16 MiB/s wr, 1.01k op/s 2024-06-10T01:28:23.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:22 smithi078 ceph-mon[36683]: pgmap v1105: 129 pgs: 129 active+clean; 687 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 16 MiB/s wr, 1.01k op/s 2024-06-10T01:28:23.558 INFO:tasks.workunit.client.0.smithi007.stdout: 10 222029 23.29 MB/sec execute 420 sec latency 43.500 ms 2024-06-10T01:28:24.558 INFO:tasks.workunit.client.0.smithi007.stdout: 10 222575 23.29 MB/sec execute 421 sec latency 104.466 ms 2024-06-10T01:28:25.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.332+0000 7f94c0f7b700 1 -- 172.21.15.7:0/502348287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94bc101fa0 msgr2=0x7f94bc105fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:25.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.332+0000 7f94c0f7b700 1 --2- 172.21.15.7:0/502348287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94bc101fa0 0x7f94bc105fe0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f94b00099c0 tx=0x7f94b0009cd0 comp rx=0 tx=0).stop 2024-06-10T01:28:25.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.332+0000 7f94c0f7b700 1 -- 172.21.15.7:0/502348287 shutdown_connections 2024-06-10T01:28:25.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.332+0000 7f94c0f7b700 1 --2- 172.21.15.7:0/502348287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94bc1066c0 0x7f94bc10e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:25.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.332+0000 7f94c0f7b700 1 --2- 172.21.15.7:0/502348287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94bc1015f0 0x7f94bc1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:25.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.332+0000 7f94c0f7b700 1 --2- 172.21.15.7:0/502348287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94bc101fa0 0x7f94bc105fe0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:25.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.332+0000 7f94c0f7b700 1 -- 172.21.15.7:0/502348287 >> 172.21.15.7:0/502348287 conn(0x7f94bc078330 msgr2=0x7f94bc078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:25.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.332+0000 7f94c0f7b700 1 -- 172.21.15.7:0/502348287 shutdown_connections 2024-06-10T01:28:25.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.333+0000 7f94c0f7b700 1 -- 172.21.15.7:0/502348287 wait complete. 2024-06-10T01:28:25.327 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:25 smithi007 ceph-mon[32398]: pgmap v1106: 129 pgs: 129 active+clean; 682 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 12 MiB/s wr, 855 op/s 2024-06-10T01:28:25.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.333+0000 7f94c0f7b700 1 Processor -- start 2024-06-10T01:28:25.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.334+0000 7f94c0f7b700 1 -- start start 2024-06-10T01:28:25.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.334+0000 7f94c0f7b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94bc1015f0 0x7f94bc197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:25.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.334+0000 7f94c0f7b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94bc101fa0 0x7f94bc197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:25.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.334+0000 7f94c0f7b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94bc1066c0 0x7f94bc198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:25.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.334+0000 7f94c0f7b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc1116b0 con 0x7f94bc1015f0 2024-06-10T01:28:25.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.334+0000 7f94c0f7b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc111530 con 0x7f94bc101fa0 2024-06-10T01:28:25.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.334+0000 7f94c0f7b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc111830 con 0x7f94bc1066c0 2024-06-10T01:28:25.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.334+0000 7f94ba59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94bc1015f0 0x7f94bc197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:25.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.334+0000 7f94ba59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94bc1015f0 0x7f94bc197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38384/0 (socket says 172.21.15.7:38384) 2024-06-10T01:28:25.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.334+0000 7f94ba59c700 1 -- 172.21.15.7:0/2908454923 learned_addr learned my addr 172.21.15.7:0/2908454923 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:25.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.334+0000 7f94b9d9b700 1 --2- 172.21.15.7:0/2908454923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94bc101fa0 0x7f94bc197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:25.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.334+0000 7f94bad9d700 1 --2- 172.21.15.7:0/2908454923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94bc1066c0 0x7f94bc198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:25.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.334+0000 7f94ba59c700 1 -- 172.21.15.7:0/2908454923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94bc1066c0 msgr2=0x7f94bc198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:25.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.334+0000 7f94ba59c700 1 --2- 172.21.15.7:0/2908454923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94bc1066c0 0x7f94bc198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:25.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.334+0000 7f94ba59c700 1 -- 172.21.15.7:0/2908454923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94bc101fa0 msgr2=0x7f94bc197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:25.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.334+0000 7f94ba59c700 1 --2- 172.21.15.7:0/2908454923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94bc101fa0 0x7f94bc197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:25.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.334+0000 7f94ba59c700 1 -- 172.21.15.7:0/2908454923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94bc1a7070 con 0x7f94bc1015f0 2024-06-10T01:28:25.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.335+0000 7f94ba59c700 1 --2- 172.21.15.7:0/2908454923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94bc1015f0 0x7f94bc197800 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00e9a0 tx=0x7f94ac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:25.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.335+0000 7f94ab7fe700 1 -- 172.21.15.7:0/2908454923 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac00ca10 con 0x7f94bc1015f0 2024-06-10T01:28:25.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.335+0000 7f94ab7fe700 1 -- 172.21.15.7:0/2908454923 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94ac00cbb0 con 0x7f94bc1015f0 2024-06-10T01:28:25.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.335+0000 7f94c0f7b700 1 -- 172.21.15.7:0/2908454923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94bc1a7360 con 0x7f94bc1015f0 2024-06-10T01:28:25.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.335+0000 7f94ab7fe700 1 -- 172.21.15.7:0/2908454923 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac018680 con 0x7f94bc1015f0 2024-06-10T01:28:25.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.335+0000 7f94c0f7b700 1 -- 172.21.15.7:0/2908454923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94bc1a7800 con 0x7f94bc1015f0 2024-06-10T01:28:25.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:25 smithi078 ceph-mon[36683]: pgmap v1106: 129 pgs: 129 active+clean; 682 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 12 MiB/s wr, 855 op/s 2024-06-10T01:28:25.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.337+0000 7f94ab7fe700 1 -- 172.21.15.7:0/2908454923 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f94ac010430 con 0x7f94bc1015f0 2024-06-10T01:28:25.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.338+0000 7f94ab7fe700 1 --2- 172.21.15.7:0/2908454923 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f94a40a2dc0 0x7f94a40a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:25.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.338+0000 7f94ab7fe700 1 -- 172.21.15.7:0/2908454923 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f94ac0cf5e0 con 0x7f94bc1015f0 2024-06-10T01:28:25.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.338+0000 7f94b9d9b700 1 --2- 172.21.15.7:0/2908454923 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f94a40a2dc0 0x7f94a40a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:25.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.339+0000 7f94b9d9b700 1 --2- 172.21.15.7:0/2908454923 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f94a40a2dc0 0x7f94a40a5280 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f94b0005b40 tx=0x7f94b0005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:25.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.339+0000 7f94cb7feb80 1 Processor -- start 2024-06-10T01:28:25.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.339+0000 7f94cb7feb80 1 -- start start 2024-06-10T01:28:25.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.340+0000 7f94cb7feb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625a02a4590 0x5625a02a4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:25.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.340+0000 7f94cb7feb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625a02a4eb0 0x5625a029e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:25.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.340+0000 7f94cb7feb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625a029ece0 0x5625a029f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:25.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.340+0000 7f94cb7feb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5625a01910f0 con 0x5625a02a4eb0 2024-06-10T01:28:25.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.340+0000 7f94cb7feb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5625a01ef9e0 con 0x5625a029ece0 2024-06-10T01:28:25.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.340+0000 7f94cb7feb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5625a01575f0 con 0x5625a02a4590 2024-06-10T01:28:25.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.340+0000 7f94b9d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625a02a4eb0 0x5625a029e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:25.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.340+0000 7f94b9d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625a02a4eb0 0x5625a029e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38390/0 (socket says 172.21.15.7:38390) 2024-06-10T01:28:25.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.340+0000 7f94b9d9b700 1 -- 172.21.15.7:0/3104434757 learned_addr learned my addr 172.21.15.7:0/3104434757 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:25.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.340+0000 7f94bad9d700 1 --2- 172.21.15.7:0/3104434757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625a029ece0 0x5625a029f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:25.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.340+0000 7f94b9d9b700 1 -- 172.21.15.7:0/3104434757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625a02a4590 msgr2=0x5625a02a4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:25.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.340+0000 7f94b9d9b700 1 --2- 172.21.15.7:0/3104434757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625a02a4590 0x5625a02a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:25.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.340+0000 7f94b9d9b700 1 -- 172.21.15.7:0/3104434757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625a029ece0 msgr2=0x5625a029f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:25.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.340+0000 7f94b9d9b700 1 --2- 172.21.15.7:0/3104434757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625a029ece0 0x5625a029f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:25.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.340+0000 7f94b9d9b700 1 -- 172.21.15.7:0/3104434757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625a013ea60 con 0x5625a02a4eb0 2024-06-10T01:28:25.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.340+0000 7f94b9d9b700 1 --2- 172.21.15.7:0/3104434757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625a02a4eb0 0x5625a029e7a0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f94b0015000 tx=0x7f94b001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7660d3b2c4538c6a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:25.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.341+0000 7f94a8ff9700 1 -- 172.21.15.7:0/3104434757 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b0034070 con 0x5625a02a4eb0 2024-06-10T01:28:25.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.341+0000 7f94a8ff9700 1 -- 172.21.15.7:0/3104434757 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94b000bd60 con 0x5625a02a4eb0 2024-06-10T01:28:25.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.341+0000 7f94a8ff9700 1 -- 172.21.15.7:0/3104434757 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b002f8d0 con 0x5625a02a4eb0 2024-06-10T01:28:25.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.341+0000 7f94cb7feb80 1 -- 172.21.15.7:0/3104434757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625a02a4eb0 msgr2=0x5625a029e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:25.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.341+0000 7f94cb7feb80 1 --2- 172.21.15.7:0/3104434757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625a02a4eb0 0x5625a029e7a0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f94b0015000 tx=0x7f94b001f000 comp rx=0 tx=0).stop 2024-06-10T01:28:25.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.341+0000 7f94cb7feb80 1 -- 172.21.15.7:0/3104434757 shutdown_connections 2024-06-10T01:28:25.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.341+0000 7f94cb7feb80 1 --2- 172.21.15.7:0/3104434757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625a029ece0 0x5625a029f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:25.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.341+0000 7f94cb7feb80 1 --2- 172.21.15.7:0/3104434757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625a02a4590 0x5625a02a4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:25.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.341+0000 7f94cb7feb80 1 --2- 172.21.15.7:0/3104434757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625a02a4eb0 0x5625a029e7a0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:25.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.341+0000 7f94cb7feb80 1 -- 172.21.15.7:0/3104434757 >> 172.21.15.7:0/3104434757 conn(0x5625a01cb210 msgr2=0x5625a0299a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:25.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.342+0000 7f94cb7feb80 1 -- 172.21.15.7:0/3104434757 shutdown_connections 2024-06-10T01:28:25.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.342+0000 7f94cb7feb80 1 -- 172.21.15.7:0/3104434757 wait complete. 2024-06-10T01:28:25.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.342+0000 7f94cb7feb80 1 -- start start 2024-06-10T01:28:25.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.342+0000 7f94cb7feb80 1 Processor -- start 2024-06-10T01:28:25.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.342+0000 7f94cb7feb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625a029ece0 0x5625a02aac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:25.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.342+0000 7f94cb7feb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625a02a4590 0x5625a02a7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:25.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.342+0000 7f94cb7feb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625a02a4eb0 0x5625a02a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:25.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.342+0000 7f94cb7feb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5625a0190150 con 0x5625a02a4eb0 2024-06-10T01:28:25.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.342+0000 7f94cb7feb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5625a01eb9a0 con 0x5625a02a4590 2024-06-10T01:28:25.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.342+0000 7f94cb7feb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5625a02ab830 con 0x5625a029ece0 2024-06-10T01:28:25.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.342+0000 7f94cb7feb80 10 client.0 _ll_register_callbacks cb 0x5625a0185740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:28:25.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.342+0000 7f94bad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625a02a4eb0 0x5625a02a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:25.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.342+0000 7f94bad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625a02a4eb0 0x5625a02a84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38396/0 (socket says 172.21.15.7:38396) 2024-06-10T01:28:25.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.342+0000 7f94bad9d700 1 -- 172.21.15.7:0/1320503058 learned_addr learned my addr 172.21.15.7:0/1320503058 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:25.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.342+0000 7f94ba59c700 1 --2- 172.21.15.7:0/1320503058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625a029ece0 0x5625a02aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:25.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.342+0000 7f94b9d9b700 1 --2- 172.21.15.7:0/1320503058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625a02a4590 0x5625a02a7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:25.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.343+0000 7f94bad9d700 1 -- 172.21.15.7:0/1320503058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625a029ece0 msgr2=0x5625a02aac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:25.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.343+0000 7f94bad9d700 1 --2- 172.21.15.7:0/1320503058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625a029ece0 0x5625a02aac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:25.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.343+0000 7f94bad9d700 1 -- 172.21.15.7:0/1320503058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625a02a4590 msgr2=0x5625a02a7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:25.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.343+0000 7f94bad9d700 1 --2- 172.21.15.7:0/1320503058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625a02a4590 0x5625a02a7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:25.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.343+0000 7f94bad9d700 1 -- 172.21.15.7:0/1320503058 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625a01b58e0 con 0x5625a02a4eb0 2024-06-10T01:28:25.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.343+0000 7f94b9d9b700 1 --2- 172.21.15.7:0/1320503058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625a02a4590 0x5625a02a7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:28:25.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.343+0000 7f94bad9d700 1 --2- 172.21.15.7:0/1320503058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625a02a4eb0 0x5625a02a84c0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f94b4011e80 tx=0x7f94b4014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df295fa7adec13f6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:25.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.343+0000 7f94a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:28:25.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.344+0000 7f94a1ffb700 1 -- 172.21.15.7:0/1320503058 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b401e620 con 0x5625a02a4eb0 2024-06-10T01:28:25.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.344+0000 7f94a1ffb700 1 -- 172.21.15.7:0/1320503058 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f94b401e7c0 con 0x5625a02a4eb0 2024-06-10T01:28:25.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.344+0000 7f94cb7feb80 1 -- 172.21.15.7:0/1320503058 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94b000bd60 con 0x5625a02a4eb0 2024-06-10T01:28:25.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.344+0000 7f94a1ffb700 1 -- 172.21.15.7:0/1320503058 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f94b4029050 con 0x5625a02a4eb0 2024-06-10T01:28:25.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.344+0000 7f94cb7feb80 10 client.20766 fetch_fsmap learned FSMap version 55 2024-06-10T01:28:25.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.345+0000 7f94cb7feb80 1 -- 172.21.15.7:0/1320503058 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5625a02b60a0 con 0x5625a02a4eb0 2024-06-10T01:28:25.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.345+0000 7f94a1ffb700 1 -- 172.21.15.7:0/1320503058 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94b4014e60 con 0x5625a02a4eb0 2024-06-10T01:28:25.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.345+0000 7f94a1ffb700 1 -- 172.21.15.7:0/1320503058 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f94b4016bc0 con 0x5625a02a4eb0 2024-06-10T01:28:25.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.345+0000 7f94cb7feb80 10 client.20766 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:28:25.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.345+0000 7f94cb7feb80 10 client.20766 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:28:25.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.345+0000 7f94cb7feb80 20 client.20766 populate_metadata read hostname 'smithi007' 2024-06-10T01:28:25.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.345+0000 7f94cb7feb80 1 --2- 172.21.15.7:0/1320503058 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5625a02b8730 0x5625a02bab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:25.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.346+0000 7f94cb7feb80 4 client.20766 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:28:25.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.346+0000 7f94cb7feb80 1 -- 172.21.15.7:0/1320503058 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f94b000bd60 con 0x5625a02b8730 2024-06-10T01:28:25.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.346+0000 7f94ba59c700 1 --2- 172.21.15.7:0/1320503058 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5625a02b8730 0x5625a02bab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:25.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.346+0000 7f94a1ffb700 10 client.20766 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:25.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.346+0000 7f94ba59c700 1 --2- 172.21.15.7:0/1320503058 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5625a02b8730 0x5625a02bab20 crc :-1 s=READY pgs=503 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f614234e6fe72ebe server_cookie=a8390b1bd12f0231 in_seq=0 out_seq=0 2024-06-10T01:28:25.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.347+0000 7f94a1ffb700 1 -- 172.21.15.7:0/1320503058 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f94b000bd60 con 0x5625a02b8730 2024-06-10T01:28:25.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.347+0000 7f94a1ffb700 10 client.20766 handle_command_reply: tid=0 2024-06-10T01:28:25.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.347+0000 7f94cb7feb80 1 client.20766 shutdown 2024-06-10T01:28:25.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.347+0000 7f94cb7feb80 1 -- 172.21.15.7:0/1320503058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625a02a4eb0 msgr2=0x5625a02a84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:25.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.347+0000 7f94cb7feb80 1 --2- 172.21.15.7:0/1320503058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625a02a4eb0 0x5625a02a84c0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f94b4011e80 tx=0x7f94b4014600 comp rx=0 tx=0).stop 2024-06-10T01:28:25.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.348+0000 7f94cb7feb80 1 -- 172.21.15.7:0/1320503058 shutdown_connections 2024-06-10T01:28:25.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.348+0000 7f94cb7feb80 1 --2- 172.21.15.7:0/1320503058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625a02a4eb0 0x5625a02a84c0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:25.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.348+0000 7f94cb7feb80 1 --2- 172.21.15.7:0/1320503058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625a029ece0 0x5625a02aac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:25.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.348+0000 7f94cb7feb80 1 --2- 172.21.15.7:0/1320503058 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5625a02b8730 0x5625a02bab20 crc :-1 s=READY pgs=503 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:25.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.348+0000 7f94a1ffb700 0 client.20766 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:25.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.348+0000 7f94cb7feb80 1 --2- 172.21.15.7:0/1320503058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625a02a4590 0x5625a02a7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:25.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.348+0000 7f94cb7feb80 1 -- 172.21.15.7:0/1320503058 >> 172.21.15.7:0/1320503058 conn(0x5625a01cb210 msgr2=0x5625a02983b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:25.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.348+0000 7f94cb7feb80 1 -- 172.21.15.7:0/1320503058 shutdown_connections 2024-06-10T01:28:25.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.348+0000 7f94cb7feb80 1 -- 172.21.15.7:0/1320503058 wait complete. 2024-06-10T01:28:25.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.348+0000 7f94cb7feb80 20 client.20766 trim_cache size 0 max 16384 2024-06-10T01:28:25.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.369+0000 7f94cb7feb80 1 Processor -- start 2024-06-10T01:28:25.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.369+0000 7f94cb7feb80 1 -- start start 2024-06-10T01:28:25.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.369+0000 7f94cb7feb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625a029ece0 0x5625a02aa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:25.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.369+0000 7f94cb7feb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625a02a4590 0x5625a02a3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:25.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.369+0000 7f94cb7feb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625a02a4eb0 0x5625a02a3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:25.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.369+0000 7f94cb7feb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc1b2bc0 con 0x5625a029ece0 2024-06-10T01:28:25.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.369+0000 7f94cb7feb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc1b0450 con 0x5625a02a4590 2024-06-10T01:28:25.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.369+0000 7f94cb7feb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc1ae2a0 con 0x5625a02a4eb0 2024-06-10T01:28:25.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.369+0000 7f94ba59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625a029ece0 0x5625a02aa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:25.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.369+0000 7f94ba59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625a029ece0 0x5625a02aa7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38408/0 (socket says 172.21.15.7:38408) 2024-06-10T01:28:25.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.369+0000 7f94ba59c700 1 -- 172.21.15.7:0/2814018335 learned_addr learned my addr 172.21.15.7:0/2814018335 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:25.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.369+0000 7f94b9d9b700 1 --2- 172.21.15.7:0/2814018335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625a02a4590 0x5625a02a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:25.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.369+0000 7f94bad9d700 1 --2- 172.21.15.7:0/2814018335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625a02a4eb0 0x5625a02a3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:25.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.370+0000 7f94ba59c700 1 -- 172.21.15.7:0/2814018335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625a02a4eb0 msgr2=0x5625a02a3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:25.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.370+0000 7f94ba59c700 1 --2- 172.21.15.7:0/2814018335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625a02a4eb0 0x5625a02a3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:25.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.370+0000 7f94ba59c700 1 -- 172.21.15.7:0/2814018335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625a02a4590 msgr2=0x5625a02a3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:25.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.370+0000 7f94ba59c700 1 --2- 172.21.15.7:0/2814018335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625a02a4590 0x5625a02a3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:25.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.370+0000 7f94ba59c700 1 -- 172.21.15.7:0/2814018335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625a02a41e0 con 0x5625a029ece0 2024-06-10T01:28:25.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.370+0000 7f94ba59c700 1 --2- 172.21.15.7:0/2814018335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625a029ece0 0x5625a02aa7c0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f94ac09b760 tx=0x7f94ac09b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfaab6e002f13450 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:25.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.370+0000 7f94a3fff700 1 -- 172.21.15.7:0/2814018335 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac0b3070 con 0x5625a029ece0 2024-06-10T01:28:25.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.370+0000 7f94a3fff700 1 -- 172.21.15.7:0/2814018335 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94ac00cd60 con 0x5625a029ece0 2024-06-10T01:28:25.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.370+0000 7f94a3fff700 1 -- 172.21.15.7:0/2814018335 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac0a4240 con 0x5625a029ece0 2024-06-10T01:28:25.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.371+0000 7f94cb7feb80 1 -- 172.21.15.7:0/2814018335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625a029ece0 msgr2=0x5625a02aa7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:25.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.371+0000 7f94cb7feb80 1 --2- 172.21.15.7:0/2814018335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625a029ece0 0x5625a02aa7c0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f94ac09b760 tx=0x7f94ac09b960 comp rx=0 tx=0).stop 2024-06-10T01:28:25.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.371+0000 7f94cb7feb80 1 -- 172.21.15.7:0/2814018335 shutdown_connections 2024-06-10T01:28:25.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.371+0000 7f94cb7feb80 1 --2- 172.21.15.7:0/2814018335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625a02a4eb0 0x5625a02a3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:25.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.371+0000 7f94cb7feb80 1 --2- 172.21.15.7:0/2814018335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625a029ece0 0x5625a02aa7c0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:25.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.371+0000 7f94cb7feb80 1 --2- 172.21.15.7:0/2814018335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625a02a4590 0x5625a02a3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:25.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.371+0000 7f94cb7feb80 1 -- 172.21.15.7:0/2814018335 >> 172.21.15.7:0/2814018335 conn(0x5625a01cb210 msgr2=0x5625a02a3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:25.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.371+0000 7f94cb7feb80 1 -- 172.21.15.7:0/2814018335 shutdown_connections 2024-06-10T01:28:25.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.371+0000 7f94cb7feb80 1 -- 172.21.15.7:0/2814018335 wait complete. 2024-06-10T01:28:25.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.371+0000 7f94cb7feb80 1 -- start start 2024-06-10T01:28:25.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.371+0000 7f94cb7feb80 1 Processor -- start 2024-06-10T01:28:25.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.372+0000 7f94cb7feb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625a029ece0 0x5625a02b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:25.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.372+0000 7f94cb7feb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625a02a4590 0x5625a02b1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:25.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.372+0000 7f94cb7feb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625a02a4eb0 0x5625a02adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:25.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.372+0000 7f94cb7feb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc1ae910 con 0x5625a02a4eb0 2024-06-10T01:28:25.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.372+0000 7f94cb7feb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc1afd80 con 0x5625a029ece0 2024-06-10T01:28:25.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.372+0000 7f94cb7feb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc1aefe0 con 0x5625a02a4590 2024-06-10T01:28:25.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.372+0000 7f94cb7feb80 10 client.0 _ll_register_callbacks cb 0x5625a0185740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:28:25.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.372+0000 7f94bad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625a02a4eb0 0x5625a02adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:25.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.372+0000 7f94ba59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625a029ece0 0x5625a02b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:25.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.372+0000 7f94bad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625a02a4eb0 0x5625a02adaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38420/0 (socket says 172.21.15.7:38420) 2024-06-10T01:28:25.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.372+0000 7f94bad9d700 1 -- 172.21.15.7:0/1805746725 learned_addr learned my addr 172.21.15.7:0/1805746725 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:25.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.372+0000 7f94b9d9b700 1 --2- 172.21.15.7:0/1805746725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625a02a4590 0x5625a02b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:25.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.372+0000 7f94ba59c700 1 -- 172.21.15.7:0/1805746725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625a02a4590 msgr2=0x5625a02b1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:25.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.372+0000 7f94ba59c700 1 --2- 172.21.15.7:0/1805746725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625a02a4590 0x5625a02b1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:25.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.372+0000 7f94ba59c700 1 -- 172.21.15.7:0/1805746725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625a02a4eb0 msgr2=0x5625a02adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:25.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.372+0000 7f94ba59c700 1 --2- 172.21.15.7:0/1805746725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625a02a4eb0 0x5625a02adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:25.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.372+0000 7f94ba59c700 1 -- 172.21.15.7:0/1805746725 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625a02ae030 con 0x5625a029ece0 2024-06-10T01:28:25.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.372+0000 7f94bad9d700 1 --2- 172.21.15.7:0/1805746725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625a02a4eb0 0x5625a02adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:28:25.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.372+0000 7f94b9d9b700 1 --2- 172.21.15.7:0/1805746725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625a02a4590 0x5625a02b1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:28:25.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.372+0000 7f94ba59c700 1 --2- 172.21.15.7:0/1805746725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625a029ece0 0x5625a02b0b20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00c2d0 tx=0x7f94ac0cce10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3178a49a091b188f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:25.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.372+0000 7f94a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:28:25.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.373+0000 7f94a1ffb700 1 -- 172.21.15.7:0/1805746725 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac0b3070 con 0x5625a029ece0 2024-06-10T01:28:25.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.373+0000 7f94cb7feb80 1 -- 172.21.15.7:0/1805746725 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94ac00cd60 con 0x5625a029ece0 2024-06-10T01:28:25.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.373+0000 7f94a1ffb700 1 -- 172.21.15.7:0/1805746725 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f94ac09bd60 con 0x5625a029ece0 2024-06-10T01:28:25.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.373+0000 7f94a1ffb700 1 -- 172.21.15.7:0/1805746725 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f94ac0a4240 con 0x5625a029ece0 2024-06-10T01:28:25.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.373+0000 7f94cb7feb80 10 client.29960 fetch_fsmap learned FSMap version 55 2024-06-10T01:28:25.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.373+0000 7f94cb7feb80 1 -- 172.21.15.7:0/1805746725 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5625a0323ad0 con 0x5625a029ece0 2024-06-10T01:28:25.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.373+0000 7f94a1ffb700 1 -- 172.21.15.7:0/1805746725 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94ac00cd60 con 0x5625a029ece0 2024-06-10T01:28:25.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.373+0000 7f94a1ffb700 1 -- 172.21.15.7:0/1805746725 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f94ac09aeb0 con 0x5625a029ece0 2024-06-10T01:28:25.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.373+0000 7f94cb7feb80 10 client.29960 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:28:25.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.373+0000 7f94cb7feb80 10 client.29960 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:28:25.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.373+0000 7f94cb7feb80 20 client.29960 populate_metadata read hostname 'smithi007' 2024-06-10T01:28:25.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.373+0000 7f94cb7feb80 1 --2- 172.21.15.7:0/1805746725 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5625a0324220 0x5625a0326640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:25.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.373+0000 7f94cb7feb80 4 client.29960 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.1, "await_for": 45.0}] 2024-06-10T01:28:25.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.373+0000 7f94cb7feb80 1 -- 172.21.15.7:0/1805746725 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.1, "await_for": 45.0}) -- 0x5625a01fcee0 con 0x5625a0324220 2024-06-10T01:28:25.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.374+0000 7f94b9d9b700 1 --2- 172.21.15.7:0/1805746725 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5625a0324220 0x5625a0326640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:25.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.374+0000 7f94b9d9b700 1 --2- 172.21.15.7:0/1805746725 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5625a0324220 0x5625a0326640 crc :-1 s=READY pgs=504 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=268263eadad28c15 server_cookie=ae7444be6ccc1ab4 in_seq=0 out_seq=0 2024-06-10T01:28:25.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:25.374+0000 7f94a1ffb700 10 client.29960 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:25.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:25 smithi175 ceph-mon[36636]: pgmap v1106: 129 pgs: 129 active+clean; 682 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 12 MiB/s wr, 855 op/s 2024-06-10T01:28:25.558 INFO:tasks.workunit.client.0.smithi007.stdout: 10 223018 23.31 MB/sec execute 422 sec latency 346.796 ms 2024-06-10T01:28:26.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:26.335+0000 7f94a1ffb700 1 -- 172.21.15.7:0/1805746725 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5625a01fcee0 con 0x5625a0324220 2024-06-10T01:28:26.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:26.335+0000 7f94a1ffb700 10 client.29960 handle_command_reply: tid=0 2024-06-10T01:28:26.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:26.335+0000 7f94cb7feb80 1 client.29960 shutdown 2024-06-10T01:28:26.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:26.335+0000 7f94cb7feb80 1 -- 172.21.15.7:0/1805746725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625a029ece0 msgr2=0x5625a02b0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:26.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:26.335+0000 7f94cb7feb80 1 --2- 172.21.15.7:0/1805746725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625a029ece0 0x5625a02b0b20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00c2d0 tx=0x7f94ac0cce10 comp rx=0 tx=0).stop 2024-06-10T01:28:26.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:26.335+0000 7f94cb7feb80 1 -- 172.21.15.7:0/1805746725 shutdown_connections 2024-06-10T01:28:26.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:26.335+0000 7f94cb7feb80 1 --2- 172.21.15.7:0/1805746725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625a02a4eb0 0x5625a02adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:26.335+0000 7f94cb7feb80 1 --2- 172.21.15.7:0/1805746725 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5625a0324220 0x5625a0326640 crc :-1 s=READY pgs=504 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:26.335+0000 7f94cb7feb80 1 --2- 172.21.15.7:0/1805746725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625a029ece0 0x5625a02b0b20 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:26.335+0000 7f94a1ffb700 0 client.29960 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:26.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:26.335+0000 7f94cb7feb80 1 --2- 172.21.15.7:0/1805746725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625a02a4590 0x5625a02b1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:26.335+0000 7f94cb7feb80 1 -- 172.21.15.7:0/1805746725 >> 172.21.15.7:0/1805746725 conn(0x5625a01cb210 msgr2=0x5625a02a38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:26.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:26.335+0000 7f94cb7feb80 1 -- 172.21.15.7:0/1805746725 shutdown_connections 2024-06-10T01:28:26.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:26.336+0000 7f94cb7feb80 1 -- 172.21.15.7:0/1805746725 wait complete. 2024-06-10T01:28:26.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:26.336+0000 7f94cb7feb80 20 client.29960 trim_cache size 0 max 16384 2024-06-10T01:28:26.330 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:28:26.330 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 55, 2024-06-10T01:28:26.330 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:28:26.330 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 70, 2024-06-10T01:28:26.330 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:28:26.330 INFO:teuthology.orchestra.run.smithi007.stdout: "8d5f0376": { 2024-06-10T01:28:26.330 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 70, 2024-06-10T01:28:26.331 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:28:26.331 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:28:26.331 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:28:26.331 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:28:26.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:28:26.331 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:28:26.331 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 142.1, 2024-06-10T01:28:26.331 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:28:26.331 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:28:26.331 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:28:26.331 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:28:26.331 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:28:26.331 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:28:26.331 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:28:26.331 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:28:26.332 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:28:26.332 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:28:26.332 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:28:26.332 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:28:26.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:26.336+0000 7f94c0f7b700 1 -- 172.21.15.7:0/2908454923 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f94a40a2dc0 msgr2=0x7f94a40a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:26.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:26.336+0000 7f94c0f7b700 1 --2- 172.21.15.7:0/2908454923 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f94a40a2dc0 0x7f94a40a5280 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f94b0005b40 tx=0x7f94b0005a90 comp rx=0 tx=0).stop 2024-06-10T01:28:26.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:26.336+0000 7f94c0f7b700 1 -- 172.21.15.7:0/2908454923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94bc1015f0 msgr2=0x7f94bc197800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:26.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:26.336+0000 7f94c0f7b700 1 --2- 172.21.15.7:0/2908454923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94bc1015f0 0x7f94bc197800 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00e9a0 tx=0x7f94ac00ecb0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:26.336+0000 7f94c0f7b700 1 -- 172.21.15.7:0/2908454923 shutdown_connections 2024-06-10T01:28:26.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:26.336+0000 7f94c0f7b700 1 --2- 172.21.15.7:0/2908454923 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f94a40a2dc0 0x7f94a40a5280 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:26.336+0000 7f94c0f7b700 1 --2- 172.21.15.7:0/2908454923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94bc1066c0 0x7f94bc198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:26.336+0000 7f94c0f7b700 1 --2- 172.21.15.7:0/2908454923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94bc1015f0 0x7f94bc197800 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:26.336+0000 7f94c0f7b700 1 --2- 172.21.15.7:0/2908454923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94bc101fa0 0x7f94bc197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:26.336+0000 7f94c0f7b700 1 -- 172.21.15.7:0/2908454923 >> 172.21.15.7:0/2908454923 conn(0x7f94bc078330 msgr2=0x7f94bc0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:26.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:26.337+0000 7f94c0f7b700 1 -- 172.21.15.7:0/2908454923 shutdown_connections 2024-06-10T01:28:26.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:26.337+0000 7f94c0f7b700 1 -- 172.21.15.7:0/2908454923 wait complete. 2024-06-10T01:28:26.558 INFO:tasks.workunit.client.0.smithi007.stdout: 10 223032 23.25 MB/sec execute 423 sec latency 1318.675 ms 2024-06-10T01:28:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.570+0000 7f7fe16a5640 1 -- 172.21.15.7:0/1275241872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fdc107720 msgr2=0x7f7fdc109b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.570+0000 7f7fe16a5640 1 --2- 172.21.15.7:0/1275241872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fdc107720 0x7f7fdc109b70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009900 tx=0x7f7fd402a3f0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.570+0000 7f7fe16a5640 1 -- 172.21.15.7:0/1275241872 shutdown_connections 2024-06-10T01:28:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.570+0000 7f7fe16a5640 1 --2- 172.21.15.7:0/1275241872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fdc107720 0x7f7fdc109b70 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.570+0000 7f7fe16a5640 1 --2- 172.21.15.7:0/1275241872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7fdc104df0 0x7f7fdc1071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.570+0000 7f7fe16a5640 1 --2- 172.21.15.7:0/1275241872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fdc1024c0 0x7f7fdc1048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.570+0000 7f7fe16a5640 1 -- 172.21.15.7:0/1275241872 >> 172.21.15.7:0/1275241872 conn(0x7f7fdc0fbed0 msgr2=0x7f7fdc0fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.570+0000 7f7fe16a5640 1 -- 172.21.15.7:0/1275241872 shutdown_connections 2024-06-10T01:28:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.570+0000 7f7fe16a5640 1 -- 172.21.15.7:0/1275241872 wait complete. 2024-06-10T01:28:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.571+0000 7f7fe16a5640 1 Processor -- start 2024-06-10T01:28:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.571+0000 7f7fe16a5640 1 -- start start 2024-06-10T01:28:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.571+0000 7f7fe16a5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7fdc1024c0 0x7f7fdc197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.572+0000 7f7fe16a5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fdc104df0 0x7f7fdc198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.572+0000 7f7fe16a5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fdc107720 0x7f7fdc19f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.572+0000 7f7fe16a5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc10d6e0 con 0x7f7fdc107720 2024-06-10T01:28:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.572+0000 7f7fe16a5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc10d560 con 0x7f7fdc104df0 2024-06-10T01:28:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.572+0000 7f7fe16a5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc10d860 con 0x7f7fdc1024c0 2024-06-10T01:28:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.572+0000 7f7fdb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fdc107720 0x7f7fdc19f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.572+0000 7f7fdb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fdc107720 0x7f7fdc19f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38428/0 (socket says 172.21.15.7:38428) 2024-06-10T01:28:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.572+0000 7f7fdb7fe640 1 -- 172.21.15.7:0/1698940270 learned_addr learned my addr 172.21.15.7:0/1698940270 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.572+0000 7f7fdaffd640 1 --2- 172.21.15.7:0/1698940270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7fdc1024c0 0x7f7fdc197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.572+0000 7f7fda7fc640 1 --2- 172.21.15.7:0/1698940270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fdc104df0 0x7f7fdc198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.572+0000 7f7fdb7fe640 1 -- 172.21.15.7:0/1698940270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7fdc1024c0 msgr2=0x7f7fdc197cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.572+0000 7f7fdb7fe640 1 --2- 172.21.15.7:0/1698940270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7fdc1024c0 0x7f7fdc197cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.572+0000 7f7fdb7fe640 1 -- 172.21.15.7:0/1698940270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fdc104df0 msgr2=0x7f7fdc198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.572+0000 7f7fdb7fe640 1 --2- 172.21.15.7:0/1698940270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fdc104df0 0x7f7fdc198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.572+0000 7f7fdb7fe640 1 -- 172.21.15.7:0/1698940270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fdc19f900 con 0x7f7fdc107720 2024-06-10T01:28:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.572+0000 7f7fdb7fe640 1 --2- 172.21.15.7:0/1698940270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fdc107720 0x7f7fdc19f260 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40314e0 tx=0x7f7fd4031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.572+0000 7f7fc3fff640 1 -- 172.21.15.7:0/1698940270 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd403f030 con 0x7f7fdc107720 2024-06-10T01:28:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.572+0000 7f7fe16a5640 1 -- 172.21.15.7:0/1698940270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fdc19fb90 con 0x7f7fdc107720 2024-06-10T01:28:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.573+0000 7f7fc3fff640 1 -- 172.21.15.7:0/1698940270 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fd4003ad0 con 0x7f7fdc107720 2024-06-10T01:28:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.573+0000 7f7fc3fff640 1 -- 172.21.15.7:0/1698940270 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4035070 con 0x7f7fdc107720 2024-06-10T01:28:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.573+0000 7f7fe16a5640 1 -- 172.21.15.7:0/1698940270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fdc1a00d0 con 0x7f7fdc107720 2024-06-10T01:28:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.575+0000 7f7fc3fff640 1 -- 172.21.15.7:0/1698940270 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f7fd4042030 con 0x7f7fdc107720 2024-06-10T01:28:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.575+0000 7f7fc3fff640 1 --2- 172.21.15.7:0/1698940270 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7fc40a2b80 0x7f7fc40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.576+0000 7f7fdaffd640 1 --2- 172.21.15.7:0/1698940270 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7fc40a2b80 0x7f7fc40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.576+0000 7f7fc3fff640 1 -- 172.21.15.7:0/1698940270 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f7fd40efa20 con 0x7f7fdc107720 2024-06-10T01:28:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.576+0000 7f7fdaffd640 1 --2- 172.21.15.7:0/1698940270 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7fc40a2b80 0x7f7fc40a5040 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc1991c0 tx=0x7f7fcc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.577+0000 7f7fe43b2740 1 Processor -- start 2024-06-10T01:28:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.577+0000 7f7fe43b2740 1 -- start start 2024-06-10T01:28:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.577+0000 7f7fe43b2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624d92ff010 0x5624d92ff430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.577+0000 7f7fe43b2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5624d92ffa00 0x5624d9304500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.577+0000 7f7fe43b2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5624d9304b70 0x5624d9307000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.577+0000 7f7fe43b2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5624d92624f0 con 0x5624d92ffa00 2024-06-10T01:28:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.577+0000 7f7fe43b2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5624d9200870 con 0x5624d92ff010 2024-06-10T01:28:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.577+0000 7f7fe43b2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5624d925e4b0 con 0x5624d9304b70 2024-06-10T01:28:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.577+0000 7f7fda7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5624d92ffa00 0x5624d9304500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.577+0000 7f7fda7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5624d92ffa00 0x5624d9304500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38444/0 (socket says 172.21.15.7:38444) 2024-06-10T01:28:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.577+0000 7f7fdb7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5624d9304b70 0x5624d9307000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.577+0000 7f7fda7fc640 1 -- 172.21.15.7:0/1813077698 learned_addr learned my addr 172.21.15.7:0/1813077698 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.577+0000 7f7fdaffd640 1 --2- 172.21.15.7:0/1813077698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624d92ff010 0x5624d92ff430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.577+0000 7f7fda7fc640 1 -- 172.21.15.7:0/1813077698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5624d9304b70 msgr2=0x5624d9307000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.577+0000 7f7fda7fc640 1 --2- 172.21.15.7:0/1813077698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5624d9304b70 0x5624d9307000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.577+0000 7f7fda7fc640 1 -- 172.21.15.7:0/1813077698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624d92ff010 msgr2=0x5624d92ff430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.577+0000 7f7fda7fc640 1 --2- 172.21.15.7:0/1813077698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624d92ff010 0x5624d92ff430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.577+0000 7f7fda7fc640 1 -- 172.21.15.7:0/1813077698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624d921c9b0 con 0x5624d92ffa00 2024-06-10T01:28:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.578+0000 7f7fda7fc640 1 --2- 172.21.15.7:0/1813077698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5624d92ffa00 0x5624d9304500 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0010420 tx=0x7f7fd0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994fd24729afc7f9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.578+0000 7f7fc17fa640 1 -- 172.21.15.7:0/1813077698 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0016070 con 0x5624d92ffa00 2024-06-10T01:28:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.578+0000 7f7fc17fa640 1 -- 172.21.15.7:0/1813077698 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fd0004d20 con 0x5624d92ffa00 2024-06-10T01:28:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.578+0000 7f7fc17fa640 1 -- 172.21.15.7:0/1813077698 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0005560 con 0x5624d92ffa00 2024-06-10T01:28:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.579+0000 7f7fe43b2740 1 -- 172.21.15.7:0/1813077698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5624d92ffa00 msgr2=0x5624d9304500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.579+0000 7f7fe43b2740 1 --2- 172.21.15.7:0/1813077698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5624d92ffa00 0x5624d9304500 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0010420 tx=0x7f7fd0009f90 comp rx=0 tx=0).stop 2024-06-10T01:28:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.579+0000 7f7fe43b2740 1 -- 172.21.15.7:0/1813077698 shutdown_connections 2024-06-10T01:28:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.579+0000 7f7fe43b2740 1 --2- 172.21.15.7:0/1813077698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5624d9304b70 0x5624d9307000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.579+0000 7f7fe43b2740 1 --2- 172.21.15.7:0/1813077698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5624d92ffa00 0x5624d9304500 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.579+0000 7f7fe43b2740 1 --2- 172.21.15.7:0/1813077698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624d92ff010 0x5624d92ff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.579+0000 7f7fe43b2740 1 -- 172.21.15.7:0/1813077698 >> 172.21.15.7:0/1813077698 conn(0x5624d92fa680 msgr2=0x5624d92faa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.579+0000 7f7fe43b2740 1 -- 172.21.15.7:0/1813077698 shutdown_connections 2024-06-10T01:28:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.579+0000 7f7fe43b2740 1 -- 172.21.15.7:0/1813077698 wait complete. 2024-06-10T01:28:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.579+0000 7f7fe43b2740 1 -- start start 2024-06-10T01:28:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.580+0000 7f7fe43b2740 1 Processor -- start 2024-06-10T01:28:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.580+0000 7f7fe43b2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5624d92ff010 0x5624d93092b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.580+0000 7f7fe43b2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5624d92ffa00 0x5624d93097f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.580+0000 7f7fe43b2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624d9304b70 0x5624d9310830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.580+0000 7f7fe43b2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5624d9307b20 con 0x5624d92ffa00 2024-06-10T01:28:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.580+0000 7f7fe43b2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5624d91ebae0 con 0x5624d9304b70 2024-06-10T01:28:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.580+0000 7f7fe43b2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5624d9307ca0 con 0x5624d92ff010 2024-06-10T01:28:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.580+0000 7f7fda7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5624d92ffa00 0x5624d93097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.580+0000 7f7fda7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5624d92ffa00 0x5624d93097f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38446/0 (socket says 172.21.15.7:38446) 2024-06-10T01:28:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.580+0000 7f7fda7fc640 1 -- 172.21.15.7:0/3544869535 learned_addr learned my addr 172.21.15.7:0/3544869535 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.580+0000 7f7fe43b2740 10 client.0 _ll_register_callbacks cb 0x5624d92249b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:28:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.580+0000 7f7fdaffd640 1 --2- 172.21.15.7:0/3544869535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5624d92ff010 0x5624d93092b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.580+0000 7f7fdb7fe640 1 --2- 172.21.15.7:0/3544869535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624d9304b70 0x5624d9310830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.580+0000 7f7fda7fc640 1 -- 172.21.15.7:0/3544869535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5624d92ff010 msgr2=0x5624d93092b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.580+0000 7f7fda7fc640 1 --2- 172.21.15.7:0/3544869535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5624d92ff010 0x5624d93092b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.580+0000 7f7fda7fc640 1 -- 172.21.15.7:0/3544869535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624d9304b70 msgr2=0x5624d9310830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.580+0000 7f7fda7fc640 1 --2- 172.21.15.7:0/3544869535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624d9304b70 0x5624d9310830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.580+0000 7f7fda7fc640 1 -- 172.21.15.7:0/3544869535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624d8f76210 con 0x5624d92ffa00 2024-06-10T01:28:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.581+0000 7f7fda7fc640 1 --2- 172.21.15.7:0/3544869535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5624d92ffa00 0x5624d93097f0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0019000 tx=0x7f7fd0004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c23f7a72a26e37 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.581+0000 7f7fbeff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:28:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.581+0000 7f7fbeff5640 1 -- 172.21.15.7:0/3544869535 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0016070 con 0x5624d92ffa00 2024-06-10T01:28:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.581+0000 7f7fbeff5640 1 -- 172.21.15.7:0/3544869535 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f7fd0003b80 con 0x5624d92ffa00 2024-06-10T01:28:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.581+0000 7f7fe43b2740 1 -- 172.21.15.7:0/3544869535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fd0004d20 con 0x5624d92ffa00 2024-06-10T01:28:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.581+0000 7f7fbeff5640 1 -- 172.21.15.7:0/3544869535 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f7fd0003d80 con 0x5624d92ffa00 2024-06-10T01:28:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.581+0000 7f7fe43b2740 10 client.20784 fetch_fsmap learned FSMap version 55 2024-06-10T01:28:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.581+0000 7f7fe43b2740 1 -- 172.21.15.7:0/3544869535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5624d9042620 con 0x5624d92ffa00 2024-06-10T01:28:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.581+0000 7f7fbeff5640 1 -- 172.21.15.7:0/3544869535 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fd0004d20 con 0x5624d92ffa00 2024-06-10T01:28:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.582+0000 7f7fbeff5640 1 -- 172.21.15.7:0/3544869535 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f7fd000f040 con 0x5624d92ffa00 2024-06-10T01:28:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.582+0000 7f7fe43b2740 10 client.20784 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:28:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.582+0000 7f7fe43b2740 10 client.20784 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:28:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.582+0000 7f7fe43b2740 20 client.20784 populate_metadata read hostname 'smithi007' 2024-06-10T01:28:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.582+0000 7f7fe43b2740 1 --2- 172.21.15.7:0/3544869535 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5624d9314fa0 0x5624d9317400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.582+0000 7f7fe43b2740 4 client.20784 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:28:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.582+0000 7f7fe43b2740 1 -- 172.21.15.7:0/3544869535 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624d9275f10 con 0x5624d9314fa0 2024-06-10T01:28:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.583+0000 7f7fdaffd640 1 --2- 172.21.15.7:0/3544869535 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5624d9314fa0 0x5624d9317400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.583+0000 7f7fdaffd640 1 --2- 172.21.15.7:0/3544869535 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5624d9314fa0 0x5624d9317400 crc :-1 s=READY pgs=505 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccf436b7b69b1a43 server_cookie=199c003ebb38ab22 in_seq=0 out_seq=0 2024-06-10T01:28:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.583+0000 7f7fbeff5640 10 client.20784 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.584+0000 7f7fbeff5640 1 -- 172.21.15.7:0/3544869535 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5624d9275f10 con 0x5624d9314fa0 2024-06-10T01:28:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.584+0000 7f7fbeff5640 10 client.20784 handle_command_reply: tid=0 2024-06-10T01:28:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.584+0000 7f7fe43b2740 1 client.20784 shutdown 2024-06-10T01:28:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.584+0000 7f7fe43b2740 1 -- 172.21.15.7:0/3544869535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5624d92ffa00 msgr2=0x5624d93097f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.584+0000 7f7fe43b2740 1 --2- 172.21.15.7:0/3544869535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5624d92ffa00 0x5624d93097f0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0019000 tx=0x7f7fd0004c70 comp rx=0 tx=0).stop 2024-06-10T01:28:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.584+0000 7f7fe43b2740 1 -- 172.21.15.7:0/3544869535 shutdown_connections 2024-06-10T01:28:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.584+0000 7f7fe43b2740 1 --2- 172.21.15.7:0/3544869535 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5624d9314fa0 0x5624d9317400 crc :-1 s=READY pgs=505 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.584+0000 7f7fe43b2740 1 --2- 172.21.15.7:0/3544869535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624d9304b70 0x5624d9310830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.584+0000 7f7fbeff5640 0 client.20784 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.584+0000 7f7fe43b2740 1 --2- 172.21.15.7:0/3544869535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5624d92ffa00 0x5624d93097f0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.584+0000 7f7fe43b2740 1 --2- 172.21.15.7:0/3544869535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5624d92ff010 0x5624d93092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.584+0000 7f7fe43b2740 1 -- 172.21.15.7:0/3544869535 >> 172.21.15.7:0/3544869535 conn(0x5624d92fa680 msgr2=0x5624d92f97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.584+0000 7f7fe43b2740 1 -- 172.21.15.7:0/3544869535 shutdown_connections 2024-06-10T01:28:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.584+0000 7f7fe43b2740 1 -- 172.21.15.7:0/3544869535 wait complete. 2024-06-10T01:28:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.584+0000 7f7fe43b2740 20 client.20784 trim_cache size 0 max 16384 2024-06-10T01:28:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.599+0000 7f7fe43b2740 1 Processor -- start 2024-06-10T01:28:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.599+0000 7f7fe43b2740 1 -- start start 2024-06-10T01:28:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.600+0000 7f7fe43b2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624d92ff010 0x5624d9311280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.600+0000 7f7fe43b2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5624d92ffa00 0x5624d93107f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.600+0000 7f7fe43b2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5624d9304b70 0x5624d93117c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.600+0000 7f7fe43b2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc1acdc0 con 0x5624d9304b70 2024-06-10T01:28:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.600+0000 7f7fe43b2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc1aa690 con 0x5624d92ff010 2024-06-10T01:28:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.600+0000 7f7fe43b2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc1a84e0 con 0x5624d92ffa00 2024-06-10T01:28:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.600+0000 7f7fdb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5624d9304b70 0x5624d93117c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.600+0000 7f7fdb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5624d9304b70 0x5624d93117c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38460/0 (socket says 172.21.15.7:38460) 2024-06-10T01:28:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.600+0000 7f7fdb7fe640 1 -- 172.21.15.7:0/2288720967 learned_addr learned my addr 172.21.15.7:0/2288720967 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:26.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.600+0000 7f7fdaffd640 1 --2- 172.21.15.7:0/2288720967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624d92ff010 0x5624d9311280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:26.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.600+0000 7f7fda7fc640 1 --2- 172.21.15.7:0/2288720967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5624d92ffa00 0x5624d93107f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:26.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.600+0000 7f7fdb7fe640 1 -- 172.21.15.7:0/2288720967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5624d92ffa00 msgr2=0x5624d93107f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:26.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.600+0000 7f7fdb7fe640 1 --2- 172.21.15.7:0/2288720967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5624d92ffa00 0x5624d93107f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.600+0000 7f7fdb7fe640 1 -- 172.21.15.7:0/2288720967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624d92ff010 msgr2=0x5624d9311280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:26.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.600+0000 7f7fdb7fe640 1 --2- 172.21.15.7:0/2288720967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624d92ff010 0x5624d9311280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.600+0000 7f7fdb7fe640 1 -- 172.21.15.7:0/2288720967 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624d92fb350 con 0x5624d9304b70 2024-06-10T01:28:26.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.600+0000 7f7fdb7fe640 1 --2- 172.21.15.7:0/2288720967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5624d9304b70 0x5624d93117c0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40b8890 tx=0x7f7fd403b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6799dcff2e921139 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:26.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.601+0000 7f7fc0ff9640 1 -- 172.21.15.7:0/2288720967 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4049040 con 0x5624d9304b70 2024-06-10T01:28:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.601+0000 7f7fc0ff9640 1 -- 172.21.15.7:0/2288720967 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fd40bbf50 con 0x5624d9304b70 2024-06-10T01:28:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.601+0000 7f7fc0ff9640 1 -- 172.21.15.7:0/2288720967 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd40b8530 con 0x5624d9304b70 2024-06-10T01:28:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.601+0000 7f7fe43b2740 1 -- 172.21.15.7:0/2288720967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5624d9304b70 msgr2=0x5624d93117c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.601+0000 7f7fe43b2740 1 --2- 172.21.15.7:0/2288720967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5624d9304b70 0x5624d93117c0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40b8890 tx=0x7f7fd403b670 comp rx=0 tx=0).stop 2024-06-10T01:28:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.601+0000 7f7fe43b2740 1 -- 172.21.15.7:0/2288720967 shutdown_connections 2024-06-10T01:28:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.601+0000 7f7fe43b2740 1 --2- 172.21.15.7:0/2288720967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5624d9304b70 0x5624d93117c0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.601+0000 7f7fe43b2740 1 --2- 172.21.15.7:0/2288720967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5624d92ffa00 0x5624d93107f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.601+0000 7f7fe43b2740 1 --2- 172.21.15.7:0/2288720967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624d92ff010 0x5624d9311280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.601+0000 7f7fe43b2740 1 -- 172.21.15.7:0/2288720967 >> 172.21.15.7:0/2288720967 conn(0x5624d92fa680 msgr2=0x5624d9300c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.601+0000 7f7fe43b2740 1 -- 172.21.15.7:0/2288720967 shutdown_connections 2024-06-10T01:28:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.601+0000 7f7fe43b2740 1 -- 172.21.15.7:0/2288720967 wait complete. 2024-06-10T01:28:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.602+0000 7f7fe43b2740 1 -- start start 2024-06-10T01:28:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.602+0000 7f7fe43b2740 1 Processor -- start 2024-06-10T01:28:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.602+0000 7f7fe43b2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5624d92ff010 0x5624d9310480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.602+0000 7f7fe43b2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5624d92ffa00 0x5624d930cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.602+0000 7f7fe43b2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624d9304b70 0x5624d930d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.602+0000 7f7fe43b2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc1a8b50 con 0x5624d92ff010 2024-06-10T01:28:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.602+0000 7f7fe43b2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc1a9fc0 con 0x5624d9304b70 2024-06-10T01:28:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.602+0000 7f7fe43b2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc1a9220 con 0x5624d92ffa00 2024-06-10T01:28:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.602+0000 7f7fe43b2740 10 client.0 _ll_register_callbacks cb 0x7f7fcc0099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:28:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.602+0000 7f7fdaffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5624d92ff010 0x5624d9310480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.602+0000 7f7fdaffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5624d92ff010 0x5624d9310480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38466/0 (socket says 172.21.15.7:38466) 2024-06-10T01:28:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.602+0000 7f7fdaffd640 1 -- 172.21.15.7:0/2996439245 learned_addr learned my addr 172.21.15.7:0/2996439245 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.602+0000 7f7fdb7fe640 1 --2- 172.21.15.7:0/2996439245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624d9304b70 0x5624d930d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.602+0000 7f7fda7fc640 1 --2- 172.21.15.7:0/2996439245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5624d92ffa00 0x5624d930cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.602+0000 7f7fdaffd640 1 -- 172.21.15.7:0/2996439245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5624d92ffa00 msgr2=0x5624d930cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.602+0000 7f7fdaffd640 1 --2- 172.21.15.7:0/2996439245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5624d92ffa00 0x5624d930cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.602+0000 7f7fdaffd640 1 -- 172.21.15.7:0/2996439245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624d9304b70 msgr2=0x5624d930d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.602+0000 7f7fdaffd640 1 --2- 172.21.15.7:0/2996439245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624d9304b70 0x5624d930d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.602+0000 7f7fdaffd640 1 -- 172.21.15.7:0/2996439245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624d930d9d0 con 0x5624d92ff010 2024-06-10T01:28:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.602+0000 7f7fda7fc640 1 --2- 172.21.15.7:0/2996439245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5624d92ffa00 0x5624d930cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:28:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.603+0000 7f7fdaffd640 1 --2- 172.21.15.7:0/2996439245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5624d92ff010 0x5624d9310480 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc009a90 tx=0x7f7fcc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1de6e997ce8b1a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.603+0000 7f7fbeff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:28:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.603+0000 7f7fbeff5640 1 -- 172.21.15.7:0/2996439245 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc037060 con 0x5624d92ff010 2024-06-10T01:28:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.603+0000 7f7fbeff5640 1 -- 172.21.15.7:0/2996439245 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f7fcc00e490 con 0x5624d92ff010 2024-06-10T01:28:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.603+0000 7f7fe43b2740 1 -- 172.21.15.7:0/2996439245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fd40bbf50 con 0x5624d92ff010 2024-06-10T01:28:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.603+0000 7f7fbeff5640 1 -- 172.21.15.7:0/2996439245 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f7fcc03e050 con 0x5624d92ff010 2024-06-10T01:28:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.603+0000 7f7fe43b2740 10 client.20790 fetch_fsmap learned FSMap version 55 2024-06-10T01:28:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.603+0000 7f7fe43b2740 1 -- 172.21.15.7:0/2996439245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5624d937ed60 con 0x5624d92ff010 2024-06-10T01:28:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.603+0000 7f7fbeff5640 1 -- 172.21.15.7:0/2996439245 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fd40bbf50 con 0x5624d92ff010 2024-06-10T01:28:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.603+0000 7f7fbeff5640 1 -- 172.21.15.7:0/2996439245 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f7fcc032870 con 0x5624d92ff010 2024-06-10T01:28:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.603+0000 7f7fe43b2740 10 client.20790 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:28:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.603+0000 7f7fe43b2740 10 client.20790 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:28:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.603+0000 7f7fe43b2740 20 client.20790 populate_metadata read hostname 'smithi007' 2024-06-10T01:28:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.603+0000 7f7fe43b2740 1 --2- 172.21.15.7:0/2996439245 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5624d937f740 0x5624d9381b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.603+0000 7f7fe43b2740 4 client.20790 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:28:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.603+0000 7f7fe43b2740 1 -- 172.21.15.7:0/2996439245 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5624d926fc20 con 0x5624d937f740 2024-06-10T01:28:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.604+0000 7f7fda7fc640 1 --2- 172.21.15.7:0/2996439245 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5624d937f740 0x5624d9381b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.604+0000 7f7fbeff5640 10 client.20790 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:26.604+0000 7f7fda7fc640 1 --2- 172.21.15.7:0/2996439245 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5624d937f740 0x5624d9381b60 crc :-1 s=READY pgs=506 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=147f03856b84abc4 server_cookie=96251f9be5ea5ca6 in_seq=0 out_seq=0 2024-06-10T01:28:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:27.155+0000 7f7fbeff5640 1 -- 172.21.15.7:0/2996439245 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5624d926fc20 con 0x5624d937f740 2024-06-10T01:28:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:27.155+0000 7f7fbeff5640 10 client.20790 handle_command_reply: tid=0 2024-06-10T01:28:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:27.155+0000 7f7fe43b2740 1 client.20790 shutdown 2024-06-10T01:28:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:27.155+0000 7f7fe43b2740 1 -- 172.21.15.7:0/2996439245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5624d92ff010 msgr2=0x5624d9310480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:27.155+0000 7f7fe43b2740 1 --2- 172.21.15.7:0/2996439245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5624d92ff010 0x5624d9310480 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc009a90 tx=0x7f7fcc004730 comp rx=0 tx=0).stop 2024-06-10T01:28:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:27.155+0000 7f7fe43b2740 1 -- 172.21.15.7:0/2996439245 shutdown_connections 2024-06-10T01:28:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:27.155+0000 7f7fe43b2740 1 --2- 172.21.15.7:0/2996439245 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5624d937f740 0x5624d9381b60 crc :-1 s=READY pgs=506 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:27.156+0000 7f7fe43b2740 1 --2- 172.21.15.7:0/2996439245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624d9304b70 0x5624d930d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:27.156+0000 7f7fbeff5640 0 client.20790 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:27.156+0000 7f7fe43b2740 1 --2- 172.21.15.7:0/2996439245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5624d92ffa00 0x5624d930cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:27.156+0000 7f7fe43b2740 1 --2- 172.21.15.7:0/2996439245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5624d92ff010 0x5624d9310480 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:27.156+0000 7f7fe43b2740 1 -- 172.21.15.7:0/2996439245 >> 172.21.15.7:0/2996439245 conn(0x5624d92fa680 msgr2=0x5624d9311280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:27.156+0000 7f7fe43b2740 1 -- 172.21.15.7:0/2996439245 shutdown_connections 2024-06-10T01:28:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:27.156+0000 7f7fe43b2740 1 -- 172.21.15.7:0/2996439245 wait complete. 2024-06-10T01:28:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:27.156+0000 7f7fe43b2740 20 client.20790 trim_cache size 0 max 16384 2024-06-10T01:28:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:28:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:28:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:28:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:27.156+0000 7f7fe16a5640 1 -- 172.21.15.7:0/1698940270 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7fc40a2b80 msgr2=0x7f7fc40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:27.156+0000 7f7fe16a5640 1 --2- 172.21.15.7:0/1698940270 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7fc40a2b80 0x7f7fc40a5040 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc1991c0 tx=0x7f7fcc005f90 comp rx=0 tx=0).stop 2024-06-10T01:28:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:27.156+0000 7f7fe16a5640 1 -- 172.21.15.7:0/1698940270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fdc107720 msgr2=0x7f7fdc19f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:27.156+0000 7f7fe16a5640 1 --2- 172.21.15.7:0/1698940270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fdc107720 0x7f7fdc19f260 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40314e0 tx=0x7f7fd4031b60 comp rx=0 tx=0).stop 2024-06-10T01:28:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:27.157+0000 7f7fe16a5640 1 -- 172.21.15.7:0/1698940270 shutdown_connections 2024-06-10T01:28:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:27.157+0000 7f7fe16a5640 1 --2- 172.21.15.7:0/1698940270 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7fc40a2b80 0x7f7fc40a5040 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:27.157+0000 7f7fe16a5640 1 --2- 172.21.15.7:0/1698940270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fdc107720 0x7f7fdc19f260 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:27.157+0000 7f7fe16a5640 1 --2- 172.21.15.7:0/1698940270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fdc104df0 0x7f7fdc198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:27.157+0000 7f7fe16a5640 1 --2- 172.21.15.7:0/1698940270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7fdc1024c0 0x7f7fdc197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:27.157+0000 7f7fe16a5640 1 -- 172.21.15.7:0/1698940270 >> 172.21.15.7:0/1698940270 conn(0x7f7fdc0fbed0 msgr2=0x7f7fdc102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:27.157+0000 7f7fe16a5640 1 -- 172.21.15.7:0/1698940270 shutdown_connections 2024-06-10T01:28:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:27.157+0000 7f7fe16a5640 1 -- 172.21.15.7:0/1698940270 wait complete. 2024-06-10T01:28:27.260 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:27 smithi007 ceph-mon[32398]: pgmap v1107: 129 pgs: 129 active+clean; 687 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 11 MiB/s wr, 868 op/s 2024-06-10T01:28:27.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:27 smithi078 ceph-mon[36683]: pgmap v1107: 129 pgs: 129 active+clean; 687 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 11 MiB/s wr, 868 op/s 2024-06-10T01:28:27.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:27 smithi175 ceph-mon[36636]: pgmap v1107: 129 pgs: 129 active+clean; 687 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 11 MiB/s wr, 868 op/s 2024-06-10T01:28:27.458 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8d5f0376', quiesce took 12.3 seconds. Will release after: 9.8 2024-06-10T01:28:27.559 INFO:tasks.workunit.client.0.smithi007.stdout: 10 223032 23.20 MB/sec execute 424 sec latency 2318.742 ms 2024-06-10T01:28:28.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:28 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:28:28.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:28 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:28:28.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:28 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:28:28.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:28 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:28:28.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:28 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:28:28.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:28 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:28:28.558 INFO:tasks.workunit.client.0.smithi007.stdout: 10 223032 23.14 MB/sec execute 425 sec latency 3318.786 ms 2024-06-10T01:28:29.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:29 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:28:29.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:29 smithi078 ceph-mon[36683]: pgmap v1108: 129 pgs: 129 active+clean; 694 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.3 MiB/s wr, 748 op/s 2024-06-10T01:28:29.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:29 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:28:29.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:29 smithi175 ceph-mon[36636]: pgmap v1108: 129 pgs: 129 active+clean; 694 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.3 MiB/s wr, 748 op/s 2024-06-10T01:28:29.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:29 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:28:29.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:29 smithi007 ceph-mon[32398]: pgmap v1108: 129 pgs: 129 active+clean; 694 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.3 MiB/s wr, 748 op/s 2024-06-10T01:28:29.558 INFO:tasks.workunit.client.0.smithi007.stdout: 10 223032 23.09 MB/sec execute 426 sec latency 4318.890 ms 2024-06-10T01:28:30.558 INFO:tasks.workunit.client.0.smithi007.stdout: 10 223032 23.03 MB/sec execute 427 sec latency 5318.950 ms 2024-06-10T01:28:31.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:31 smithi078 ceph-mon[36683]: pgmap v1109: 129 pgs: 129 active+clean; 694 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.4 MiB/s wr, 572 op/s 2024-06-10T01:28:31.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:31 smithi175 ceph-mon[36636]: pgmap v1109: 129 pgs: 129 active+clean; 694 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.4 MiB/s wr, 572 op/s 2024-06-10T01:28:31.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:31 smithi007 ceph-mon[32398]: pgmap v1109: 129 pgs: 129 active+clean; 694 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.4 MiB/s wr, 572 op/s 2024-06-10T01:28:31.558 INFO:tasks.workunit.client.0.smithi007.stdout: 10 223032 22.98 MB/sec execute 428 sec latency 6319.066 ms 2024-06-10T01:28:32.559 INFO:tasks.workunit.client.0.smithi007.stdout: 10 223032 22.93 MB/sec execute 429 sec latency 7319.118 ms 2024-06-10T01:28:33.558 INFO:tasks.workunit.client.0.smithi007.stdout: 10 223032 22.87 MB/sec execute 430 sec latency 8319.206 ms 2024-06-10T01:28:33.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:33 smithi078 ceph-mon[36683]: pgmap v1110: 129 pgs: 129 active+clean; 708 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.1 MiB/s wr, 587 op/s 2024-06-10T01:28:33.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:33 smithi175 ceph-mon[36636]: pgmap v1110: 129 pgs: 129 active+clean; 708 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.1 MiB/s wr, 587 op/s 2024-06-10T01:28:34.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:33 smithi007 ceph-mon[32398]: pgmap v1110: 129 pgs: 129 active+clean; 708 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.1 MiB/s wr, 587 op/s 2024-06-10T01:28:34.559 INFO:tasks.workunit.client.0.smithi007.stdout: 10 223032 22.82 MB/sec execute 431 sec latency 9319.310 ms 2024-06-10T01:28:35.559 INFO:tasks.workunit.client.0.smithi007.stdout: 10 223032 22.77 MB/sec execute 432 sec latency 10319.409 ms 2024-06-10T01:28:35.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:35 smithi078 ceph-mon[36683]: pgmap v1111: 129 pgs: 129 active+clean; 712 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.7 MiB/s wr, 383 op/s 2024-06-10T01:28:35.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:35 smithi175 ceph-mon[36636]: pgmap v1111: 129 pgs: 129 active+clean; 712 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.7 MiB/s wr, 383 op/s 2024-06-10T01:28:36.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:35 smithi007 ceph-mon[32398]: pgmap v1111: 129 pgs: 129 active+clean; 712 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.7 MiB/s wr, 383 op/s 2024-06-10T01:28:36.559 INFO:tasks.workunit.client.0.smithi007.stdout: 10 223032 22.72 MB/sec execute 433 sec latency 11319.487 ms 2024-06-10T01:28:37.259 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:28:37.559 INFO:tasks.workunit.client.0.smithi007.stdout: 10 223032 22.66 MB/sec execute 434 sec latency 12319.536 ms 2024-06-10T01:28:37.592 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:28:37.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:37 smithi078 ceph-mon[36683]: pgmap v1112: 129 pgs: 129 active+clean; 710 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.3 MiB/s wr, 392 op/s 2024-06-10T01:28:37.885 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:37 smithi007 ceph-mon[32398]: pgmap v1112: 129 pgs: 129 active+clean; 710 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.3 MiB/s wr, 392 op/s 2024-06-10T01:28:37.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:37 smithi175 ceph-mon[36636]: pgmap v1112: 129 pgs: 129 active+clean; 710 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.3 MiB/s wr, 392 op/s 2024-06-10T01:28:38.559 INFO:tasks.workunit.client.0.smithi007.stdout: 10 223032 22.61 MB/sec execute 435 sec latency 13319.568 ms 2024-06-10T01:28:39.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.438+0000 7f0a3e599700 1 -- 172.21.15.7:0/3103078966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a38101390 msgr2=0x7f0a38101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:39.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.438+0000 7f0a3e599700 1 --2- 172.21.15.7:0/3103078966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a38101390 0x7f0a38101810 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f0a20009990 tx=0x7f0a20009ca0 comp rx=0 tx=0).stop 2024-06-10T01:28:39.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.438+0000 7f0a3e599700 1 -- 172.21.15.7:0/3103078966 shutdown_connections 2024-06-10T01:28:39.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.438+0000 7f0a3e599700 1 --2- 172.21.15.7:0/3103078966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a38101d50 0x7f0a3810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:39.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.438+0000 7f0a3e599700 1 --2- 172.21.15.7:0/3103078966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a38107400 0x7f0a381077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:39.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.438+0000 7f0a3e599700 1 --2- 172.21.15.7:0/3103078966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a38101390 0x7f0a38101810 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:39.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.438+0000 7f0a3e599700 1 -- 172.21.15.7:0/3103078966 >> 172.21.15.7:0/3103078966 conn(0x7f0a38078340 msgr2=0x7f0a38078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:39.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.439+0000 7f0a3e599700 1 -- 172.21.15.7:0/3103078966 shutdown_connections 2024-06-10T01:28:39.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.439+0000 7f0a3e599700 1 -- 172.21.15.7:0/3103078966 wait complete. 2024-06-10T01:28:39.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.439+0000 7f0a3e599700 1 Processor -- start 2024-06-10T01:28:39.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.439+0000 7f0a3e599700 1 -- start start 2024-06-10T01:28:39.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.439+0000 7f0a3e599700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a38101390 0x7f0a3819e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:39.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.439+0000 7f0a3e599700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a38101d50 0x7f0a3819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:39.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.439+0000 7f0a3e599700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a38107400 0x7f0a381989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:39.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.439+0000 7f0a3e599700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0a38111670 con 0x7f0a38107400 2024-06-10T01:28:39.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.439+0000 7f0a3e599700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0a381114f0 con 0x7f0a38101d50 2024-06-10T01:28:39.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.439+0000 7f0a3e599700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0a381117f0 con 0x7f0a38101390 2024-06-10T01:28:39.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.440+0000 7f0a3cb36700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a38107400 0x7f0a381989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:39.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.440+0000 7f0a3cb36700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a38107400 0x7f0a381989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60138/0 (socket says 172.21.15.7:60138) 2024-06-10T01:28:39.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.440+0000 7f0a3cb36700 1 -- 172.21.15.7:0/3986197763 learned_addr learned my addr 172.21.15.7:0/3986197763 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:39.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.440+0000 7f0a37fff700 1 --2- 172.21.15.7:0/3986197763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a38101390 0x7f0a3819e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:39.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.440+0000 7f0a2ffff700 1 --2- 172.21.15.7:0/3986197763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a38101d50 0x7f0a3819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:39.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.440+0000 7f0a3cb36700 1 -- 172.21.15.7:0/3986197763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a38101390 msgr2=0x7f0a3819e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:39.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.440+0000 7f0a3cb36700 1 --2- 172.21.15.7:0/3986197763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a38101390 0x7f0a3819e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:39.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.440+0000 7f0a3cb36700 1 -- 172.21.15.7:0/3986197763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a38101d50 msgr2=0x7f0a3819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:39.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.440+0000 7f0a3cb36700 1 --2- 172.21.15.7:0/3986197763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a38101d50 0x7f0a3819ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:39.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.440+0000 7f0a3cb36700 1 -- 172.21.15.7:0/3986197763 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a38199120 con 0x7f0a38107400 2024-06-10T01:28:39.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.440+0000 7f0a3cb36700 1 --2- 172.21.15.7:0/3986197763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a38107400 0x7f0a381989c0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f0a30007ee0 tx=0x7f0a3000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:39.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.440+0000 7f0a35ffb700 1 -- 172.21.15.7:0/3986197763 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a30004bb0 con 0x7f0a38107400 2024-06-10T01:28:39.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.440+0000 7f0a35ffb700 1 -- 172.21.15.7:0/3986197763 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a30004d50 con 0x7f0a38107400 2024-06-10T01:28:39.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.440+0000 7f0a3e599700 1 -- 172.21.15.7:0/3986197763 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a38199410 con 0x7f0a38107400 2024-06-10T01:28:39.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.440+0000 7f0a35ffb700 1 -- 172.21.15.7:0/3986197763 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a30005960 con 0x7f0a38107400 2024-06-10T01:28:39.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.440+0000 7f0a3e599700 1 -- 172.21.15.7:0/3986197763 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a381a56d0 con 0x7f0a38107400 2024-06-10T01:28:39.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.441+0000 7f0a3e599700 1 -- 172.21.15.7:0/3986197763 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a38069180 con 0x7f0a38107400 2024-06-10T01:28:39.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.443+0000 7f0a35ffb700 1 -- 172.21.15.7:0/3986197763 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0a3001a020 con 0x7f0a38107400 2024-06-10T01:28:39.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.444+0000 7f0a35ffb700 1 --2- 172.21.15.7:0/3986197763 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0a240a2c20 0x7f0a240a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:39.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.444+0000 7f0a35ffb700 1 -- 172.21.15.7:0/3986197763 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f0a300cf1b0 con 0x7f0a38107400 2024-06-10T01:28:39.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.444+0000 7f0a37fff700 1 --2- 172.21.15.7:0/3986197763 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0a240a2c20 0x7f0a240a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:39.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.445+0000 7f0a37fff700 1 --2- 172.21.15.7:0/3986197763 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0a240a2c20 0x7f0a240a50e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0a38199b00 tx=0x7f0a2800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:39.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.448+0000 7f0a35ffb700 1 -- 172.21.15.7:0/3986197763 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0a30010070 con 0x7f0a38107400 2024-06-10T01:28:39.559 INFO:tasks.workunit.client.0.smithi007.stdout: 10 223032 22.56 MB/sec execute 436 sec latency 14319.617 ms 2024-06-10T01:28:39.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.701+0000 7f0a3e599700 1 -- 172.21.15.7:0/3986197763 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a381a58a0 con 0x7f0a38107400 2024-06-10T01:28:39.694 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:39 smithi007 ceph-mon[32398]: pgmap v1113: 129 pgs: 129 active+clean; 691 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.4 MiB/s wr, 379 op/s 2024-06-10T01:28:39.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.702+0000 7f0a35ffb700 1 -- 172.21.15.7:0/3986197763 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11852 (secure 0 0 0) 0x7f0a3009bd90 con 0x7f0a38107400 2024-06-10T01:28:39.695 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:28:39.695 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:27:44.542390+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29813":{"gid":29813,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/138896429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":138896429},{"type":"v1","addr":"172.21.15.175:6835","nonce":138896429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24451,24434,24436]},"id":1}]} 2024-06-10T01:28:39.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.704+0000 7f0a3e599700 1 -- 172.21.15.7:0/3986197763 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0a240a2c20 msgr2=0x7f0a240a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:39.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.704+0000 7f0a3e599700 1 --2- 172.21.15.7:0/3986197763 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0a240a2c20 0x7f0a240a50e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0a38199b00 tx=0x7f0a2800b410 comp rx=0 tx=0).stop 2024-06-10T01:28:39.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.704+0000 7f0a3e599700 1 -- 172.21.15.7:0/3986197763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a38107400 msgr2=0x7f0a381989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:39.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.704+0000 7f0a3e599700 1 --2- 172.21.15.7:0/3986197763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a38107400 0x7f0a381989c0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f0a30007ee0 tx=0x7f0a3000d3b0 comp rx=0 tx=0).stop 2024-06-10T01:28:39.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.704+0000 7f0a3e599700 1 -- 172.21.15.7:0/3986197763 shutdown_connections 2024-06-10T01:28:39.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.704+0000 7f0a3e599700 1 --2- 172.21.15.7:0/3986197763 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0a240a2c20 0x7f0a240a50e0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:39.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.704+0000 7f0a3e599700 1 --2- 172.21.15.7:0/3986197763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a38107400 0x7f0a381989c0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:39.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.704+0000 7f0a3e599700 1 --2- 172.21.15.7:0/3986197763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a38101390 0x7f0a3819e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:39.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.704+0000 7f0a3e599700 1 --2- 172.21.15.7:0/3986197763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a38101d50 0x7f0a3819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:39.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.704+0000 7f0a3e599700 1 -- 172.21.15.7:0/3986197763 >> 172.21.15.7:0/3986197763 conn(0x7f0a38078340 msgr2=0x7f0a381005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:39.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.704+0000 7f0a3e599700 1 -- 172.21.15.7:0/3986197763 shutdown_connections 2024-06-10T01:28:39.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:39.704+0000 7f0a3e599700 1 -- 172.21.15.7:0/3986197763 wait complete. 2024-06-10T01:28:39.698 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 55 2024-06-10T01:28:39.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:39 smithi078 ceph-mon[36683]: pgmap v1113: 129 pgs: 129 active+clean; 691 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.4 MiB/s wr, 379 op/s 2024-06-10T01:28:39.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:39 smithi175 ceph-mon[36636]: pgmap v1113: 129 pgs: 129 active+clean; 691 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.4 MiB/s wr, 379 op/s 2024-06-10T01:28:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.279+0000 7fd17f770640 1 -- 172.21.15.7:0/1632581926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd178100900 msgr2=0x7fd178100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.279+0000 7fd17f770640 1 --2- 172.21.15.7:0/1632581926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd178100900 0x7fd178100d20 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fd174009ca0 tx=0x7fd17402a8c0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.279+0000 7fd17f770640 1 -- 172.21.15.7:0/1632581926 shutdown_connections 2024-06-10T01:28:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.279+0000 7fd17f770640 1 --2- 172.21.15.7:0/1632581926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1781024b0 0x7fd178108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.279+0000 7fd17f770640 1 --2- 172.21.15.7:0/1632581926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd178101af0 0x7fd178101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.279+0000 7fd17f770640 1 --2- 172.21.15.7:0/1632581926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd178100900 0x7fd178100d20 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.279+0000 7fd17f770640 1 -- 172.21.15.7:0/1632581926 >> 172.21.15.7:0/1632581926 conn(0x7fd1780fbe70 msgr2=0x7fd1780fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.279+0000 7fd17f770640 1 -- 172.21.15.7:0/1632581926 shutdown_connections 2024-06-10T01:28:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.279+0000 7fd17f770640 1 -- 172.21.15.7:0/1632581926 wait complete. 2024-06-10T01:28:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.279+0000 7fd17f770640 1 Processor -- start 2024-06-10T01:28:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.279+0000 7fd17f770640 1 -- start start 2024-06-10T01:28:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.280+0000 7fd17f770640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd178100900 0x7fd178069d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.280+0000 7fd17f770640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd178101af0 0x7fd17819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.280+0000 7fd17f770640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1781024b0 0x7fd17819f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.280+0000 7fd17f770640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd17810c5a0 con 0x7fd178100900 2024-06-10T01:28:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.280+0000 7fd17f770640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd17810c420 con 0x7fd1781024b0 2024-06-10T01:28:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.280+0000 7fd17f770640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd17810c720 con 0x7fd178101af0 2024-06-10T01:28:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.280+0000 7fd17d50c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd178100900 0x7fd178069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.280+0000 7fd17d50c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd178100900 0x7fd178069d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60154/0 (socket says 172.21.15.7:60154) 2024-06-10T01:28:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.280+0000 7fd17d50c640 1 -- 172.21.15.7:0/1966623951 learned_addr learned my addr 172.21.15.7:0/1966623951 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.280+0000 7fd17cd0b640 1 --2- 172.21.15.7:0/1966623951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd178101af0 0x7fd17819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.280+0000 7fd17d50c640 1 -- 172.21.15.7:0/1966623951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd178101af0 msgr2=0x7fd17819ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.280+0000 7fd17d50c640 1 --2- 172.21.15.7:0/1966623951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd178101af0 0x7fd17819ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.280+0000 7fd17d50c640 1 -- 172.21.15.7:0/1966623951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1781024b0 msgr2=0x7fd17819f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.280+0000 7fd17dd0d640 1 --2- 172.21.15.7:0/1966623951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1781024b0 0x7fd17819f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.280+0000 7fd17d50c640 1 --2- 172.21.15.7:0/1966623951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1781024b0 0x7fd17819f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.280+0000 7fd17d50c640 1 -- 172.21.15.7:0/1966623951 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17819f960 con 0x7fd178100900 2024-06-10T01:28:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.280+0000 7fd17cd0b640 1 --2- 172.21.15.7:0/1966623951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd178101af0 0x7fd17819ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:28:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.280+0000 7fd17dd0d640 1 --2- 172.21.15.7:0/1966623951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1781024b0 0x7fd17819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:28:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.280+0000 7fd17d50c640 1 --2- 172.21.15.7:0/1966623951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd178100900 0x7fd178069d90 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fd174000f80 tx=0x7fd1740319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.280+0000 7fd16e7fc640 1 -- 172.21.15.7:0/1966623951 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd174003c70 con 0x7fd178100900 2024-06-10T01:28:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.280+0000 7fd17f770640 1 -- 172.21.15.7:0/1966623951 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd17819fbf0 con 0x7fd178100900 2024-06-10T01:28:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.280+0000 7fd16e7fc640 1 -- 172.21.15.7:0/1966623951 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd174003e10 con 0x7fd178100900 2024-06-10T01:28:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.280+0000 7fd16e7fc640 1 -- 172.21.15.7:0/1966623951 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17402fda0 con 0x7fd178100900 2024-06-10T01:28:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.280+0000 7fd17f770640 1 -- 172.21.15.7:0/1966623951 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1781a0190 con 0x7fd178100900 2024-06-10T01:28:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.284+0000 7fd182122740 1 Processor -- start 2024-06-10T01:28:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.284+0000 7fd16e7fc640 1 -- 172.21.15.7:0/1966623951 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fd174004040 con 0x7fd178100900 2024-06-10T01:28:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.284+0000 7fd182122740 1 -- start start 2024-06-10T01:28:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.284+0000 7fd182122740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558961b95010 0x558961b95430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.284+0000 7fd182122740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558961b95a00 0x558961b9a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.284+0000 7fd182122740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558961b9ab70 0x558961b9d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.284+0000 7fd182122740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558961af84f0 con 0x558961b9ab70 2024-06-10T01:28:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.284+0000 7fd182122740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558961a96870 con 0x558961b95010 2024-06-10T01:28:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.284+0000 7fd182122740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558961af44b0 con 0x558961b95a00 2024-06-10T01:28:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.284+0000 7fd17cd0b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558961b9ab70 0x558961b9d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.284+0000 7fd17d50c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558961b95a00 0x558961b9a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.284+0000 7fd17cd0b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558961b9ab70 0x558961b9d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60166/0 (socket says 172.21.15.7:60166) 2024-06-10T01:28:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.284+0000 7fd17cd0b640 1 -- 172.21.15.7:0/2705158818 learned_addr learned my addr 172.21.15.7:0/2705158818 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.284+0000 7fd17dd0d640 1 --2- 172.21.15.7:0/2705158818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558961b95010 0x558961b95430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.284+0000 7fd17cd0b640 1 -- 172.21.15.7:0/2705158818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558961b95a00 msgr2=0x558961b9a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.284+0000 7fd17cd0b640 1 --2- 172.21.15.7:0/2705158818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558961b95a00 0x558961b9a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.284+0000 7fd17cd0b640 1 -- 172.21.15.7:0/2705158818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558961b95010 msgr2=0x558961b95430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.284+0000 7fd17cd0b640 1 --2- 172.21.15.7:0/2705158818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558961b95010 0x558961b95430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.284+0000 7fd17cd0b640 1 -- 172.21.15.7:0/2705158818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558961ab29b0 con 0x558961b9ab70 2024-06-10T01:28:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.285+0000 7fd16e7fc640 1 --2- 172.21.15.7:0/1966623951 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd1640a2b80 0x7fd1640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.285+0000 7fd17dd0d640 1 --2- 172.21.15.7:0/1966623951 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd1640a2b80 0x7fd1640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.285+0000 7fd17cd0b640 1 --2- 172.21.15.7:0/2705158818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558961b9ab70 0x558961b9d000 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fd16800bd30 tx=0x7fd16800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2ddb16866096d5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.285+0000 7fd16e7fc640 1 -- 172.21.15.7:0/1966623951 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fd1740f0a40 con 0x7fd178100900 2024-06-10T01:28:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.285+0000 7fd1637fe640 1 -- 172.21.15.7:0/2705158818 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd168005220 con 0x558961b9ab70 2024-06-10T01:28:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.285+0000 7fd1637fe640 1 -- 172.21.15.7:0/2705158818 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd16800be20 con 0x558961b9ab70 2024-06-10T01:28:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.285+0000 7fd1637fe640 1 -- 172.21.15.7:0/2705158818 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1680117c0 con 0x558961b9ab70 2024-06-10T01:28:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.286+0000 7fd182122740 1 -- 172.21.15.7:0/2705158818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558961b9ab70 msgr2=0x558961b9d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.286+0000 7fd17dd0d640 1 --2- 172.21.15.7:0/1966623951 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd1640a2b80 0x7fd1640a5040 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x558961b96260 tx=0x7fd1700093d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.286+0000 7fd182122740 1 --2- 172.21.15.7:0/2705158818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558961b9ab70 0x558961b9d000 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fd16800bd30 tx=0x7fd16800f730 comp rx=0 tx=0).stop 2024-06-10T01:28:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.286+0000 7fd182122740 1 -- 172.21.15.7:0/2705158818 shutdown_connections 2024-06-10T01:28:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.286+0000 7fd182122740 1 --2- 172.21.15.7:0/2705158818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558961b9ab70 0x558961b9d000 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.286+0000 7fd182122740 1 --2- 172.21.15.7:0/2705158818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558961b95a00 0x558961b9a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.286+0000 7fd182122740 1 --2- 172.21.15.7:0/2705158818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558961b95010 0x558961b95430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.286+0000 7fd182122740 1 -- 172.21.15.7:0/2705158818 >> 172.21.15.7:0/2705158818 conn(0x558961b90680 msgr2=0x558961b90a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.286+0000 7fd182122740 1 -- 172.21.15.7:0/2705158818 shutdown_connections 2024-06-10T01:28:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.286+0000 7fd182122740 1 -- 172.21.15.7:0/2705158818 wait complete. 2024-06-10T01:28:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.286+0000 7fd182122740 1 -- start start 2024-06-10T01:28:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.286+0000 7fd182122740 1 Processor -- start 2024-06-10T01:28:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.287+0000 7fd182122740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558961b95010 0x558961b9f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.287+0000 7fd182122740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558961b95a00 0x558961b9f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.287+0000 7fd182122740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558961b9ab70 0x558961ba67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.287+0000 7fd182122740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558961b9db20 con 0x558961b95a00 2024-06-10T01:28:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.287+0000 7fd182122740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558961a81ae0 con 0x558961b9ab70 2024-06-10T01:28:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.287+0000 7fd182122740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558961b9dca0 con 0x558961b95010 2024-06-10T01:28:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.287+0000 7fd182122740 10 client.0 _ll_register_callbacks cb 0x558961aba9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:28:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.287+0000 7fd17cd0b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558961b95a00 0x558961b9f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.287+0000 7fd17cd0b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558961b95a00 0x558961b9f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60174/0 (socket says 172.21.15.7:60174) 2024-06-10T01:28:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.287+0000 7fd17cd0b640 1 -- 172.21.15.7:0/2521017160 learned_addr learned my addr 172.21.15.7:0/2521017160 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.287+0000 7fd17d50c640 1 --2- 172.21.15.7:0/2521017160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558961b95010 0x558961b9f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.287+0000 7fd17dd0d640 1 --2- 172.21.15.7:0/2521017160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558961b9ab70 0x558961ba67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.287+0000 7fd17cd0b640 1 -- 172.21.15.7:0/2521017160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558961b95010 msgr2=0x558961b9f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.287+0000 7fd17cd0b640 1 --2- 172.21.15.7:0/2521017160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558961b95010 0x558961b9f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.287+0000 7fd17cd0b640 1 -- 172.21.15.7:0/2521017160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558961b9ab70 msgr2=0x558961ba67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.287+0000 7fd17cd0b640 1 --2- 172.21.15.7:0/2521017160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558961b9ab70 0x558961ba67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.287+0000 7fd17cd0b640 1 -- 172.21.15.7:0/2521017160 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55896180c210 con 0x558961b95a00 2024-06-10T01:28:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.287+0000 7fd17dd0d640 1 --2- 172.21.15.7:0/2521017160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558961b9ab70 0x558961ba67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:28:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.287+0000 7fd17cd0b640 1 --2- 172.21.15.7:0/2521017160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558961b95a00 0x558961b9f760 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fd16800c010 tx=0x7fd168005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f20bc6ddff4a3f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.287+0000 7fd160ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:28:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.287+0000 7fd160ff9640 1 -- 172.21.15.7:0/2521017160 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16802b060 con 0x558961b95a00 2024-06-10T01:28:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.287+0000 7fd160ff9640 1 -- 172.21.15.7:0/2521017160 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fd1680048d0 con 0x558961b95a00 2024-06-10T01:28:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.287+0000 7fd182122740 1 -- 172.21.15.7:0/2521017160 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd16800be20 con 0x558961b95a00 2024-06-10T01:28:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.288+0000 7fd160ff9640 1 -- 172.21.15.7:0/2521017160 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fd168032050 con 0x558961b95a00 2024-06-10T01:28:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.288+0000 7fd182122740 10 client.20808 fetch_fsmap learned FSMap version 55 2024-06-10T01:28:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.288+0000 7fd182122740 1 -- 172.21.15.7:0/2521017160 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5589618d8620 con 0x558961b95a00 2024-06-10T01:28:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.288+0000 7fd160ff9640 1 -- 172.21.15.7:0/2521017160 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd16800be20 con 0x558961b95a00 2024-06-10T01:28:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.288+0000 7fd160ff9640 1 -- 172.21.15.7:0/2521017160 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fd16801b410 con 0x558961b95a00 2024-06-10T01:28:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.288+0000 7fd182122740 10 client.20808 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:28:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.288+0000 7fd182122740 10 client.20808 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:28:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.288+0000 7fd182122740 20 client.20808 populate_metadata read hostname 'smithi007' 2024-06-10T01:28:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.288+0000 7fd182122740 1 --2- 172.21.15.7:0/2521017160 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558961baad40 0x558961bad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.288+0000 7fd182122740 4 client.20808 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:28:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.288+0000 7fd182122740 1 -- 172.21.15.7:0/2521017160 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558961b0bf10 con 0x558961baad40 2024-06-10T01:28:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.288+0000 7fd17d50c640 1 --2- 172.21.15.7:0/2521017160 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558961baad40 0x558961bad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.289+0000 7fd17d50c640 1 --2- 172.21.15.7:0/2521017160 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558961baad40 0x558961bad130 crc :-1 s=READY pgs=507 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=243977aee0db8c65 server_cookie=83fa9dbfe52ceac6 in_seq=0 out_seq=0 2024-06-10T01:28:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.289+0000 7fd160ff9640 10 client.20808 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.289+0000 7fd160ff9640 1 -- 172.21.15.7:0/2521017160 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558961b0bf10 con 0x558961baad40 2024-06-10T01:28:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.289+0000 7fd160ff9640 10 client.20808 handle_command_reply: tid=0 2024-06-10T01:28:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.290+0000 7fd182122740 1 client.20808 shutdown 2024-06-10T01:28:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.290+0000 7fd182122740 1 -- 172.21.15.7:0/2521017160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558961b95a00 msgr2=0x558961b9f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.290+0000 7fd182122740 1 --2- 172.21.15.7:0/2521017160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558961b95a00 0x558961b9f760 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fd16800c010 tx=0x7fd168005230 comp rx=0 tx=0).stop 2024-06-10T01:28:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.290+0000 7fd182122740 1 -- 172.21.15.7:0/2521017160 shutdown_connections 2024-06-10T01:28:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.290+0000 7fd182122740 1 --2- 172.21.15.7:0/2521017160 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558961baad40 0x558961bad130 crc :-1 s=READY pgs=507 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.290+0000 7fd182122740 1 --2- 172.21.15.7:0/2521017160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558961b9ab70 0x558961ba67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.290+0000 7fd182122740 1 --2- 172.21.15.7:0/2521017160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558961b95a00 0x558961b9f760 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.290+0000 7fd160ff9640 0 client.20808 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.290+0000 7fd182122740 1 --2- 172.21.15.7:0/2521017160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558961b95010 0x558961b9f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.290+0000 7fd182122740 1 -- 172.21.15.7:0/2521017160 >> 172.21.15.7:0/2521017160 conn(0x558961b90680 msgr2=0x558961b8f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.290+0000 7fd182122740 1 -- 172.21.15.7:0/2521017160 shutdown_connections 2024-06-10T01:28:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.290+0000 7fd182122740 1 -- 172.21.15.7:0/2521017160 wait complete. 2024-06-10T01:28:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.290+0000 7fd182122740 20 client.20808 trim_cache size 0 max 16384 2024-06-10T01:28:40.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.306+0000 7fd182122740 1 Processor -- start 2024-06-10T01:28:40.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.306+0000 7fd182122740 1 -- start start 2024-06-10T01:28:40.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.306+0000 7fd182122740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558961b95010 0x558961b93360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:40.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.306+0000 7fd182122740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558961b95a00 0x558961ba6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:40.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.306+0000 7fd182122740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558961b9ab70 0x558961b938a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:40.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.306+0000 7fd182122740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd1781ace80 con 0x558961b95010 2024-06-10T01:28:40.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.306+0000 7fd182122740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd1781aa750 con 0x558961b95a00 2024-06-10T01:28:40.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.306+0000 7fd182122740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd1781a85a0 con 0x558961b9ab70 2024-06-10T01:28:40.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.306+0000 7fd17d50c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558961b95010 0x558961b93360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.307+0000 7fd17d50c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558961b95010 0x558961b93360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60178/0 (socket says 172.21.15.7:60178) 2024-06-10T01:28:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.307+0000 7fd17d50c640 1 -- 172.21.15.7:0/374073764 learned_addr learned my addr 172.21.15.7:0/374073764 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.307+0000 7fd17cd0b640 1 --2- 172.21.15.7:0/374073764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558961b95a00 0x558961ba6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.307+0000 7fd17dd0d640 1 --2- 172.21.15.7:0/374073764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558961b9ab70 0x558961b938a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.307+0000 7fd17d50c640 1 -- 172.21.15.7:0/374073764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558961b9ab70 msgr2=0x558961b938a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.307+0000 7fd17d50c640 1 --2- 172.21.15.7:0/374073764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558961b9ab70 0x558961b938a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.307+0000 7fd17d50c640 1 -- 172.21.15.7:0/374073764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558961b95a00 msgr2=0x558961ba6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.307+0000 7fd17d50c640 1 --2- 172.21.15.7:0/374073764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558961b95a00 0x558961ba6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.307+0000 7fd17d50c640 1 -- 172.21.15.7:0/374073764 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558961b904b0 con 0x558961b95010 2024-06-10T01:28:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.307+0000 7fd17dd0d640 1 --2- 172.21.15.7:0/374073764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558961b9ab70 0x558961b938a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:28:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.307+0000 7fd17cd0b640 1 --2- 172.21.15.7:0/374073764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558961b95a00 0x558961ba6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:28:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.307+0000 7fd17d50c640 1 --2- 172.21.15.7:0/374073764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558961b95010 0x558961b93360 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fd1740ee1c0 tx=0x7fd1740ee090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44dc76e438925723 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.307+0000 7fd162ffd640 1 -- 172.21.15.7:0/374073764 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1740bcd80 con 0x558961b95010 2024-06-10T01:28:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.307+0000 7fd162ffd640 1 -- 172.21.15.7:0/374073764 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd174036040 con 0x558961b95010 2024-06-10T01:28:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.307+0000 7fd162ffd640 1 -- 172.21.15.7:0/374073764 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1740bc910 con 0x558961b95010 2024-06-10T01:28:40.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.308+0000 7fd182122740 1 -- 172.21.15.7:0/374073764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558961b95010 msgr2=0x558961b93360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:40.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.308+0000 7fd182122740 1 --2- 172.21.15.7:0/374073764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558961b95010 0x558961b93360 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fd1740ee1c0 tx=0x7fd1740ee090 comp rx=0 tx=0).stop 2024-06-10T01:28:40.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.308+0000 7fd182122740 1 -- 172.21.15.7:0/374073764 shutdown_connections 2024-06-10T01:28:40.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.308+0000 7fd182122740 1 --2- 172.21.15.7:0/374073764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558961b9ab70 0x558961b938a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.308+0000 7fd182122740 1 --2- 172.21.15.7:0/374073764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558961b95a00 0x558961ba6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.308+0000 7fd182122740 1 --2- 172.21.15.7:0/374073764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558961b95010 0x558961b93360 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.308+0000 7fd182122740 1 -- 172.21.15.7:0/374073764 >> 172.21.15.7:0/374073764 conn(0x558961b90680 msgr2=0x558961b96ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:40.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.308+0000 7fd182122740 1 -- 172.21.15.7:0/374073764 shutdown_connections 2024-06-10T01:28:40.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.308+0000 7fd182122740 1 -- 172.21.15.7:0/374073764 wait complete. 2024-06-10T01:28:40.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.308+0000 7fd182122740 1 -- start start 2024-06-10T01:28:40.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.308+0000 7fd182122740 1 Processor -- start 2024-06-10T01:28:40.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.309+0000 7fd182122740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558961b95010 0x558961ba35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.309+0000 7fd182122740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558961b95a00 0x558961ba3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.309+0000 7fd182122740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558961b9ab70 0x558961c139c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.309+0000 7fd182122740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd1781a8c10 con 0x558961b95a00 2024-06-10T01:28:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.309+0000 7fd182122740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd1781aa080 con 0x558961b95010 2024-06-10T01:28:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.309+0000 7fd182122740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd1781a92e0 con 0x558961b9ab70 2024-06-10T01:28:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.309+0000 7fd182122740 10 client.0 _ll_register_callbacks cb 0x7fd168015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:28:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.309+0000 7fd17cd0b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558961b95a00 0x558961ba3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.309+0000 7fd17cd0b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558961b95a00 0x558961ba3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60194/0 (socket says 172.21.15.7:60194) 2024-06-10T01:28:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.309+0000 7fd17cd0b640 1 -- 172.21.15.7:0/3040569707 learned_addr learned my addr 172.21.15.7:0/3040569707 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.309+0000 7fd17dd0d640 1 --2- 172.21.15.7:0/3040569707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558961b9ab70 0x558961c139c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.309+0000 7fd17d50c640 1 --2- 172.21.15.7:0/3040569707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558961b95010 0x558961ba35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.309+0000 7fd17cd0b640 1 -- 172.21.15.7:0/3040569707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558961b9ab70 msgr2=0x558961c139c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.309+0000 7fd17cd0b640 1 --2- 172.21.15.7:0/3040569707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558961b9ab70 0x558961c139c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.309+0000 7fd17cd0b640 1 -- 172.21.15.7:0/3040569707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558961b95010 msgr2=0x558961ba35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.309+0000 7fd17cd0b640 1 --2- 172.21.15.7:0/3040569707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558961b95010 0x558961ba35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.309+0000 7fd17cd0b640 1 -- 172.21.15.7:0/3040569707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558961c13f00 con 0x558961b95a00 2024-06-10T01:28:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.310+0000 7fd17cd0b640 1 --2- 172.21.15.7:0/3040569707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558961b95a00 0x558961ba3ae0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fd168006010 tx=0x7fd168005310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c15cfe987b45d85 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.310+0000 7fd160ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:28:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.310+0000 7fd160ff9640 1 -- 172.21.15.7:0/3040569707 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16802b040 con 0x558961b95a00 2024-06-10T01:28:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.310+0000 7fd182122740 1 -- 172.21.15.7:0/3040569707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd174036040 con 0x558961b95a00 2024-06-10T01:28:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.310+0000 7fd160ff9640 1 -- 172.21.15.7:0/3040569707 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fd16800fc30 con 0x558961b95a00 2024-06-10T01:28:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.310+0000 7fd160ff9640 1 -- 172.21.15.7:0/3040569707 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fd16803d050 con 0x558961b95a00 2024-06-10T01:28:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.310+0000 7fd182122740 10 client.20814 fetch_fsmap learned FSMap version 55 2024-06-10T01:28:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.310+0000 7fd182122740 1 -- 172.21.15.7:0/3040569707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x558961c150f0 con 0x558961b95a00 2024-06-10T01:28:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.310+0000 7fd160ff9640 1 -- 172.21.15.7:0/3040569707 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd174036040 con 0x558961b95a00 2024-06-10T01:28:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.310+0000 7fd160ff9640 1 -- 172.21.15.7:0/3040569707 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fd1680048d0 con 0x558961b95a00 2024-06-10T01:28:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.310+0000 7fd182122740 10 client.20814 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:28:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.310+0000 7fd182122740 10 client.20814 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:28:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.310+0000 7fd182122740 20 client.20814 populate_metadata read hostname 'smithi007' 2024-06-10T01:28:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.310+0000 7fd182122740 1 --2- 172.21.15.7:0/3040569707 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558961c15a40 0x558961c17e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.310+0000 7fd182122740 4 client.20814 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:28:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.310+0000 7fd182122740 1 -- 172.21.15.7:0/3040569707 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558961b05c20 con 0x558961c15a40 2024-06-10T01:28:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.311+0000 7fd17d50c640 1 --2- 172.21.15.7:0/3040569707 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558961c15a40 0x558961c17e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.311+0000 7fd17d50c640 1 --2- 172.21.15.7:0/3040569707 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558961c15a40 0x558961c17e60 crc :-1 s=READY pgs=508 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49f4a9b89393cb4a server_cookie=cb8d171d89a0284f in_seq=0 out_seq=0 2024-06-10T01:28:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.311+0000 7fd160ff9640 10 client.20814 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:40.329 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db --set-id 8d5f0376 --release --await' 2024-06-10T01:28:40.330 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db --set-id 8d5f0376 --release --await 2024-06-10T01:28:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.398+0000 7fd160ff9640 1 -- 172.21.15.7:0/3040569707 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558961b05c20 con 0x558961c15a40 2024-06-10T01:28:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.398+0000 7fd160ff9640 10 client.20814 handle_command_reply: tid=0 2024-06-10T01:28:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.398+0000 7fd182122740 1 client.20814 shutdown 2024-06-10T01:28:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.399+0000 7fd182122740 1 -- 172.21.15.7:0/3040569707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558961b95a00 msgr2=0x558961ba3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.399+0000 7fd182122740 1 --2- 172.21.15.7:0/3040569707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558961b95a00 0x558961ba3ae0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fd168006010 tx=0x7fd168005310 comp rx=0 tx=0).stop 2024-06-10T01:28:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.399+0000 7fd182122740 1 -- 172.21.15.7:0/3040569707 shutdown_connections 2024-06-10T01:28:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.399+0000 7fd182122740 1 --2- 172.21.15.7:0/3040569707 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558961c15a40 0x558961c17e60 crc :-1 s=READY pgs=508 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.399+0000 7fd182122740 1 --2- 172.21.15.7:0/3040569707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558961b9ab70 0x558961c139c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.399+0000 7fd160ff9640 0 client.20814 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.399+0000 7fd182122740 1 --2- 172.21.15.7:0/3040569707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558961b95a00 0x558961ba3ae0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.399+0000 7fd182122740 1 --2- 172.21.15.7:0/3040569707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558961b95010 0x558961ba35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.399+0000 7fd182122740 1 -- 172.21.15.7:0/3040569707 >> 172.21.15.7:0/3040569707 conn(0x558961b90680 msgr2=0x558961b9f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.399+0000 7fd182122740 1 -- 172.21.15.7:0/3040569707 shutdown_connections 2024-06-10T01:28:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.399+0000 7fd182122740 1 -- 172.21.15.7:0/3040569707 wait complete. 2024-06-10T01:28:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.399+0000 7fd182122740 20 client.20814 trim_cache size 0 max 16384 2024-06-10T01:28:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:28:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:28:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:28:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.399+0000 7fd17f770640 1 -- 172.21.15.7:0/1966623951 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd1640a2b80 msgr2=0x7fd1640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.399+0000 7fd17f770640 1 --2- 172.21.15.7:0/1966623951 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd1640a2b80 0x7fd1640a5040 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x558961b96260 tx=0x7fd1700093d0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.399+0000 7fd17f770640 1 -- 172.21.15.7:0/1966623951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd178100900 msgr2=0x7fd178069d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.399+0000 7fd17f770640 1 --2- 172.21.15.7:0/1966623951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd178100900 0x7fd178069d90 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fd174000f80 tx=0x7fd1740319f0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.399+0000 7fd17f770640 1 -- 172.21.15.7:0/1966623951 shutdown_connections 2024-06-10T01:28:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.399+0000 7fd17f770640 1 --2- 172.21.15.7:0/1966623951 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd1640a2b80 0x7fd1640a5040 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.400+0000 7fd17f770640 1 --2- 172.21.15.7:0/1966623951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1781024b0 0x7fd17819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.400+0000 7fd17f770640 1 --2- 172.21.15.7:0/1966623951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd178101af0 0x7fd17819ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.400+0000 7fd17f770640 1 --2- 172.21.15.7:0/1966623951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd178100900 0x7fd178069d90 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.400+0000 7fd17f770640 1 -- 172.21.15.7:0/1966623951 >> 172.21.15.7:0/1966623951 conn(0x7fd1780fbe70 msgr2=0x7fd178104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.400+0000 7fd17f770640 1 -- 172.21.15.7:0/1966623951 shutdown_connections 2024-06-10T01:28:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:40.400+0000 7fd17f770640 1 -- 172.21.15.7:0/1966623951 wait complete. 2024-06-10T01:28:40.559 INFO:tasks.workunit.client.0.smithi007.stdout: 10 223032 22.51 MB/sec execute 437 sec latency 15319.646 ms 2024-06-10T01:28:40.645 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:28:40.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:40 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3986197763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:28:40.877 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:40 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3986197763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:28:40.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:40 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3986197763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:28:41.559 INFO:tasks.workunit.client.0.smithi007.stdout: 10 223032 22.46 MB/sec execute 438 sec latency 16319.705 ms 2024-06-10T01:28:41.679 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:41 smithi007 ceph-mon[32398]: pgmap v1114: 129 pgs: 129 active+clean; 691 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 4.7 MiB/s wr, 252 op/s 2024-06-10T01:28:41.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:41 smithi078 ceph-mon[36683]: pgmap v1114: 129 pgs: 129 active+clean; 691 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 4.7 MiB/s wr, 252 op/s 2024-06-10T01:28:41.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:41 smithi175 ceph-mon[36636]: pgmap v1114: 129 pgs: 129 active+clean; 691 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 4.7 MiB/s wr, 252 op/s 2024-06-10T01:28:42.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.229+0000 7ff6b248e700 1 -- 172.21.15.7:0/2444218568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6ac1013c0 msgr2=0x7ff6ac101820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:42.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.229+0000 7ff6b248e700 1 --2- 172.21.15.7:0/2444218568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6ac1013c0 0x7ff6ac101820 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7ff694009990 tx=0x7ff694009ca0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.229+0000 7ff6b248e700 1 -- 172.21.15.7:0/2444218568 shutdown_connections 2024-06-10T01:28:42.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.229+0000 7ff6b248e700 1 --2- 172.21.15.7:0/2444218568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6ac101d60 0x7ff6ac10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.229+0000 7ff6b248e700 1 --2- 172.21.15.7:0/2444218568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6ac107400 0x7ff6ac1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.229+0000 7ff6b248e700 1 --2- 172.21.15.7:0/2444218568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6ac1013c0 0x7ff6ac101820 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.229+0000 7ff6b248e700 1 -- 172.21.15.7:0/2444218568 >> 172.21.15.7:0/2444218568 conn(0x7ff6ac078330 msgr2=0x7ff6ac078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:42.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.229+0000 7ff6b248e700 1 -- 172.21.15.7:0/2444218568 shutdown_connections 2024-06-10T01:28:42.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.229+0000 7ff6b248e700 1 -- 172.21.15.7:0/2444218568 wait complete. 2024-06-10T01:28:42.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.230+0000 7ff6b248e700 1 Processor -- start 2024-06-10T01:28:42.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.230+0000 7ff6b248e700 1 -- start start 2024-06-10T01:28:42.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.230+0000 7ff6b248e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6ac1013c0 0x7ff6ac19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:42.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.230+0000 7ff6b248e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6ac101d60 0x7ff6ac19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:42.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.230+0000 7ff6b248e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6ac107400 0x7ff6ac198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:42.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.230+0000 7ff6b248e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac112800 con 0x7ff6ac107400 2024-06-10T01:28:42.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.230+0000 7ff6b248e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac112680 con 0x7ff6ac101d60 2024-06-10T01:28:42.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.230+0000 7ff6b248e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac112980 con 0x7ff6ac1013c0 2024-06-10T01:28:42.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.230+0000 7ff6b0a2b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6ac107400 0x7ff6ac198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:42.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.230+0000 7ff6b0a2b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6ac107400 0x7ff6ac198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43654/0 (socket says 172.21.15.7:43654) 2024-06-10T01:28:42.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.231+0000 7ff6b0a2b700 1 -- 172.21.15.7:0/2169491548 learned_addr learned my addr 172.21.15.7:0/2169491548 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:42.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.231+0000 7ff6a3fff700 1 --2- 172.21.15.7:0/2169491548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6ac101d60 0x7ff6ac19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:42.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.231+0000 7ff6abfff700 1 --2- 172.21.15.7:0/2169491548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6ac1013c0 0x7ff6ac19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:42.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.231+0000 7ff6b0a2b700 1 -- 172.21.15.7:0/2169491548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6ac1013c0 msgr2=0x7ff6ac19e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:42.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.231+0000 7ff6b0a2b700 1 --2- 172.21.15.7:0/2169491548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6ac1013c0 0x7ff6ac19e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.231+0000 7ff6b0a2b700 1 -- 172.21.15.7:0/2169491548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6ac101d60 msgr2=0x7ff6ac19edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:42.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.231+0000 7ff6b0a2b700 1 --2- 172.21.15.7:0/2169491548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6ac101d60 0x7ff6ac19edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.231+0000 7ff6b0a2b700 1 -- 172.21.15.7:0/2169491548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6ac1991a0 con 0x7ff6ac107400 2024-06-10T01:28:42.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.231+0000 7ff6b0a2b700 1 --2- 172.21.15.7:0/2169491548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6ac107400 0x7ff6ac198a40 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4007ee0 tx=0x7ff6a400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:42.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.231+0000 7ff6a9ffb700 1 -- 172.21.15.7:0/2169491548 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a400dd20 con 0x7ff6ac107400 2024-06-10T01:28:42.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.231+0000 7ff6b248e700 1 -- 172.21.15.7:0/2169491548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6ac199490 con 0x7ff6ac107400 2024-06-10T01:28:42.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.231+0000 7ff6a9ffb700 1 -- 172.21.15.7:0/2169491548 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6a4004020 con 0x7ff6ac107400 2024-06-10T01:28:42.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.231+0000 7ff6a9ffb700 1 -- 172.21.15.7:0/2169491548 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a4015660 con 0x7ff6ac107400 2024-06-10T01:28:42.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.231+0000 7ff6b248e700 1 -- 172.21.15.7:0/2169491548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6ac1a5750 con 0x7ff6ac107400 2024-06-10T01:28:42.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.234+0000 7ff6a9ffb700 1 -- 172.21.15.7:0/2169491548 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7ff6a40041d0 con 0x7ff6ac107400 2024-06-10T01:28:42.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.234+0000 7ff6a9ffb700 1 --2- 172.21.15.7:0/2169491548 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff6980a2c20 0x7ff6980a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.234+0000 7ff6a9ffb700 1 -- 172.21.15.7:0/2169491548 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7ff6a40cee10 con 0x7ff6ac107400 2024-06-10T01:28:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.234+0000 7ff6abfff700 1 --2- 172.21.15.7:0/2169491548 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff6980a2c20 0x7ff6980a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.235+0000 7ff6abfff700 1 --2- 172.21.15.7:0/2169491548 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff6980a2c20 0x7ff6980a50e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac199b80 tx=0x7ff69c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.235+0000 7ff6bcd11b80 1 Processor -- start 2024-06-10T01:28:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.235+0000 7ff6bcd11b80 1 -- start start 2024-06-10T01:28:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.235+0000 7ff6bcd11b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a258820500 0x55a2588208e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.235+0000 7ff6bcd11b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a258820e20 0x55a25881a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.235+0000 7ff6bcd11b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a25881ac50 0x55a25881b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.235+0000 7ff6bcd11b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a25870d060 con 0x55a258820500 2024-06-10T01:28:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.235+0000 7ff6bcd11b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a25876b950 con 0x55a258820e20 2024-06-10T01:28:42.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.235+0000 7ff6bcd11b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a2586d3560 con 0x55a25881ac50 2024-06-10T01:28:42.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.235+0000 7ff6abfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a258820500 0x55a2588208e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:42.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.235+0000 7ff6abfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a258820500 0x55a2588208e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43670/0 (socket says 172.21.15.7:43670) 2024-06-10T01:28:42.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.235+0000 7ff6abfff700 1 -- 172.21.15.7:0/418888643 learned_addr learned my addr 172.21.15.7:0/418888643 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:42.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.236+0000 7ff6a3fff700 1 --2- 172.21.15.7:0/418888643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a258820e20 0x55a25881a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:42.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.236+0000 7ff6b0a2b700 1 --2- 172.21.15.7:0/418888643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a25881ac50 0x55a25881b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:42.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.236+0000 7ff6abfff700 1 -- 172.21.15.7:0/418888643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a25881ac50 msgr2=0x55a25881b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:42.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.236+0000 7ff6abfff700 1 --2- 172.21.15.7:0/418888643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a25881ac50 0x55a25881b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.236+0000 7ff6abfff700 1 -- 172.21.15.7:0/418888643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a258820e20 msgr2=0x55a25881a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:42.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.236+0000 7ff6abfff700 1 --2- 172.21.15.7:0/418888643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a258820e20 0x55a25881a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.236+0000 7ff6abfff700 1 -- 172.21.15.7:0/418888643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2586ba9d0 con 0x55a258820500 2024-06-10T01:28:42.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.236+0000 7ff6a3fff700 1 --2- 172.21.15.7:0/418888643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a258820e20 0x55a25881a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:28:42.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.236+0000 7ff6abfff700 1 --2- 172.21.15.7:0/418888643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a258820500 0x55a2588208e0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7ff69c011d60 tx=0x7ff69c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6177ee3d8b960280 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:42.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.236+0000 7ff6a27fc700 1 -- 172.21.15.7:0/418888643 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c011d90 con 0x55a258820500 2024-06-10T01:28:42.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.236+0000 7ff6a27fc700 1 -- 172.21.15.7:0/418888643 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff69c004510 con 0x55a258820500 2024-06-10T01:28:42.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.236+0000 7ff6a27fc700 1 -- 172.21.15.7:0/418888643 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c003db0 con 0x55a258820500 2024-06-10T01:28:42.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.237+0000 7ff6bcd11b80 1 -- 172.21.15.7:0/418888643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a258820500 msgr2=0x55a2588208e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:42.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.237+0000 7ff6bcd11b80 1 --2- 172.21.15.7:0/418888643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a258820500 0x55a2588208e0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7ff69c011d60 tx=0x7ff69c014000 comp rx=0 tx=0).stop 2024-06-10T01:28:42.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.237+0000 7ff6bcd11b80 1 -- 172.21.15.7:0/418888643 shutdown_connections 2024-06-10T01:28:42.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.237+0000 7ff6bcd11b80 1 --2- 172.21.15.7:0/418888643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a25881ac50 0x55a25881b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.237+0000 7ff6bcd11b80 1 --2- 172.21.15.7:0/418888643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a258820500 0x55a2588208e0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.237+0000 7ff6bcd11b80 1 --2- 172.21.15.7:0/418888643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a258820e20 0x55a25881a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.237+0000 7ff6bcd11b80 1 -- 172.21.15.7:0/418888643 >> 172.21.15.7:0/418888643 conn(0x55a258747180 msgr2=0x55a258815a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:42.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.237+0000 7ff6bcd11b80 1 -- 172.21.15.7:0/418888643 shutdown_connections 2024-06-10T01:28:42.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.237+0000 7ff6bcd11b80 1 -- 172.21.15.7:0/418888643 wait complete. 2024-06-10T01:28:42.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.237+0000 7ff6bcd11b80 1 -- start start 2024-06-10T01:28:42.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.238+0000 7ff6bcd11b80 1 Processor -- start 2024-06-10T01:28:42.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.238+0000 7ff6bcd11b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a25881ac50 0x55a258826b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:42.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.238+0000 7ff6bcd11b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a258820500 0x55a258823e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:42.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.238+0000 7ff6bcd11b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a258820e20 0x55a258824380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:42.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.238+0000 7ff6bcd11b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a25870c0c0 con 0x55a258820500 2024-06-10T01:28:42.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.238+0000 7ff6bcd11b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a258767910 con 0x55a258820e20 2024-06-10T01:28:42.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.238+0000 7ff6bcd11b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a258827800 con 0x55a25881ac50 2024-06-10T01:28:42.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.238+0000 7ff6bcd11b80 10 client.0 _ll_register_callbacks cb 0x55a2587016b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:28:42.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.238+0000 7ff6a3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a258820500 0x55a258823e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:42.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.238+0000 7ff6a3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a258820500 0x55a258823e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43678/0 (socket says 172.21.15.7:43678) 2024-06-10T01:28:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.238+0000 7ff6a3fff700 1 -- 172.21.15.7:0/1077611656 learned_addr learned my addr 172.21.15.7:0/1077611656 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.238+0000 7ff6b0a2b700 1 --2- 172.21.15.7:0/1077611656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a258820e20 0x55a258824380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.238+0000 7ff6abfff700 1 --2- 172.21.15.7:0/1077611656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a25881ac50 0x55a258826b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.238+0000 7ff6a3fff700 1 -- 172.21.15.7:0/1077611656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a25881ac50 msgr2=0x55a258826b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.238+0000 7ff6a3fff700 1 --2- 172.21.15.7:0/1077611656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a25881ac50 0x55a258826b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.239+0000 7ff6a3fff700 1 -- 172.21.15.7:0/1077611656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a258820e20 msgr2=0x55a258824380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.239+0000 7ff6a3fff700 1 --2- 172.21.15.7:0/1077611656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a258820e20 0x55a258824380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.239+0000 7ff6a3fff700 1 -- 172.21.15.7:0/1077611656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a258731850 con 0x55a258820500 2024-06-10T01:28:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.239+0000 7ff6b0a2b700 1 --2- 172.21.15.7:0/1077611656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a258820e20 0x55a258824380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:28:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.239+0000 7ff6a3fff700 1 --2- 172.21.15.7:0/1077611656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a258820500 0x55a258823e40 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7ff694009990 tx=0x7ff69400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2873bdf76bab3cd6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:42.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.239+0000 7ff68b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:28:42.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.239+0000 7ff68b7fe700 1 -- 172.21.15.7:0/1077611656 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff694005270 con 0x55a258820500 2024-06-10T01:28:42.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.239+0000 7ff68b7fe700 1 -- 172.21.15.7:0/1077611656 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7ff694005410 con 0x55a258820500 2024-06-10T01:28:42.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.239+0000 7ff6bcd11b80 1 -- 172.21.15.7:0/1077611656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff69c004510 con 0x55a258820500 2024-06-10T01:28:42.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.239+0000 7ff68b7fe700 1 -- 172.21.15.7:0/1077611656 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7ff69402f050 con 0x55a258820500 2024-06-10T01:28:42.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.239+0000 7ff6bcd11b80 10 client.20826 fetch_fsmap learned FSMap version 55 2024-06-10T01:28:42.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.239+0000 7ff6bcd11b80 1 -- 172.21.15.7:0/1077611656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55a258831ea0 con 0x55a258820500 2024-06-10T01:28:42.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.239+0000 7ff68b7fe700 1 -- 172.21.15.7:0/1077611656 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff69c004510 con 0x55a258820500 2024-06-10T01:28:42.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.240+0000 7ff68b7fe700 1 -- 172.21.15.7:0/1077611656 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7ff69402a030 con 0x55a258820500 2024-06-10T01:28:42.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.240+0000 7ff6bcd11b80 10 client.20826 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:28:42.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.240+0000 7ff6bcd11b80 10 client.20826 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:28:42.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.240+0000 7ff6bcd11b80 20 client.20826 populate_metadata read hostname 'smithi007' 2024-06-10T01:28:42.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.240+0000 7ff6bcd11b80 1 --2- 172.21.15.7:0/1077611656 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a258834530 0x55a258836950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:42.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.240+0000 7ff6bcd11b80 4 client.20826 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:28:42.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.240+0000 7ff6bcd11b80 1 -- 172.21.15.7:0/1077611656 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a258778e50 con 0x55a258834530 2024-06-10T01:28:42.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.240+0000 7ff6abfff700 1 --2- 172.21.15.7:0/1077611656 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a258834530 0x55a258836950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:42.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.241+0000 7ff68b7fe700 10 client.20826 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:42.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.241+0000 7ff6abfff700 1 --2- 172.21.15.7:0/1077611656 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a258834530 0x55a258836950 crc :-1 s=READY pgs=509 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c21c40149118692 server_cookie=e99b3227f18ab952 in_seq=0 out_seq=0 2024-06-10T01:28:42.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.241+0000 7ff68b7fe700 1 -- 172.21.15.7:0/1077611656 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a258778e50 con 0x55a258834530 2024-06-10T01:28:42.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.241+0000 7ff68b7fe700 10 client.20826 handle_command_reply: tid=0 2024-06-10T01:28:42.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.241+0000 7ff6bcd11b80 1 client.20826 shutdown 2024-06-10T01:28:42.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.242+0000 7ff6bcd11b80 1 -- 172.21.15.7:0/1077611656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a258820500 msgr2=0x55a258823e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:42.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.242+0000 7ff6bcd11b80 1 --2- 172.21.15.7:0/1077611656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a258820500 0x55a258823e40 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7ff694009990 tx=0x7ff69400bfd0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.242+0000 7ff6bcd11b80 1 -- 172.21.15.7:0/1077611656 shutdown_connections 2024-06-10T01:28:42.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.242+0000 7ff6bcd11b80 1 --2- 172.21.15.7:0/1077611656 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a258834530 0x55a258836950 crc :-1 s=READY pgs=509 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.242+0000 7ff6bcd11b80 1 --2- 172.21.15.7:0/1077611656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a258820e20 0x55a258824380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.242+0000 7ff6bcd11b80 1 --2- 172.21.15.7:0/1077611656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a25881ac50 0x55a258826b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.242+0000 7ff6bcd11b80 1 --2- 172.21.15.7:0/1077611656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a258820500 0x55a258823e40 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.242+0000 7ff6bcd11b80 1 -- 172.21.15.7:0/1077611656 >> 172.21.15.7:0/1077611656 conn(0x55a258747180 msgr2=0x55a258814bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:42.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.242+0000 7ff68b7fe700 0 client.20826 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:42.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.242+0000 7ff6bcd11b80 1 -- 172.21.15.7:0/1077611656 shutdown_connections 2024-06-10T01:28:42.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.242+0000 7ff6bcd11b80 1 -- 172.21.15.7:0/1077611656 wait complete. 2024-06-10T01:28:42.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.242+0000 7ff6bcd11b80 20 client.20826 trim_cache size 0 max 16384 2024-06-10T01:28:42.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.262+0000 7ff6bcd11b80 1 Processor -- start 2024-06-10T01:28:42.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.263+0000 7ff6bcd11b80 1 -- start start 2024-06-10T01:28:42.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.263+0000 7ff6bcd11b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a25881ac50 0x55a258822e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:42.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.263+0000 7ff6bcd11b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a258820500 0x55a258819e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:42.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.263+0000 7ff6bcd11b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a258820e20 0x55a25881a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:42.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.263+0000 7ff6bcd11b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac1b0ba0 con 0x55a258820e20 2024-06-10T01:28:42.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.263+0000 7ff6bcd11b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac1ae430 con 0x55a25881ac50 2024-06-10T01:28:42.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.263+0000 7ff6bcd11b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac1ac280 con 0x55a258820500 2024-06-10T01:28:42.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.263+0000 7ff6b0a2b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a258820e20 0x55a25881a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:42.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.263+0000 7ff6b0a2b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a258820e20 0x55a25881a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43692/0 (socket says 172.21.15.7:43692) 2024-06-10T01:28:42.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.263+0000 7ff6b0a2b700 1 -- 172.21.15.7:0/2664817120 learned_addr learned my addr 172.21.15.7:0/2664817120 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:42.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.263+0000 7ff6abfff700 1 --2- 172.21.15.7:0/2664817120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a25881ac50 0x55a258822e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:42.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.263+0000 7ff6a3fff700 1 --2- 172.21.15.7:0/2664817120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a258820500 0x55a258819e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:42.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.263+0000 7ff6b0a2b700 1 -- 172.21.15.7:0/2664817120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a258820500 msgr2=0x55a258819e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:42.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.263+0000 7ff6b0a2b700 1 --2- 172.21.15.7:0/2664817120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a258820500 0x55a258819e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.263+0000 7ff6b0a2b700 1 -- 172.21.15.7:0/2664817120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a25881ac50 msgr2=0x55a258822e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:42.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.263+0000 7ff6b0a2b700 1 --2- 172.21.15.7:0/2664817120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a25881ac50 0x55a258822e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.263+0000 7ff6b0a2b700 1 -- 172.21.15.7:0/2664817120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2588223b0 con 0x55a258820e20 2024-06-10T01:28:42.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.263+0000 7ff6a3fff700 1 --2- 172.21.15.7:0/2664817120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a258820500 0x55a258819e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:28:42.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.263+0000 7ff6b0a2b700 1 --2- 172.21.15.7:0/2664817120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a258820e20 0x55a25881a350 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7ff6a400d700 tx=0x7ff6a409af20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73e390d4f312f8e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:42.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.264+0000 7ff6a1ffb700 1 -- 172.21.15.7:0/2664817120 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a4095e60 con 0x55a258820e20 2024-06-10T01:28:42.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.264+0000 7ff6a1ffb700 1 -- 172.21.15.7:0/2664817120 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6a409b2f0 con 0x55a258820e20 2024-06-10T01:28:42.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.264+0000 7ff6a1ffb700 1 -- 172.21.15.7:0/2664817120 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a409b7b0 con 0x55a258820e20 2024-06-10T01:28:42.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.264+0000 7ff6bcd11b80 1 -- 172.21.15.7:0/2664817120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a258820e20 msgr2=0x55a25881a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:42.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.264+0000 7ff6bcd11b80 1 --2- 172.21.15.7:0/2664817120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a258820e20 0x55a25881a350 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7ff6a400d700 tx=0x7ff6a409af20 comp rx=0 tx=0).stop 2024-06-10T01:28:42.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.264+0000 7ff6bcd11b80 1 -- 172.21.15.7:0/2664817120 shutdown_connections 2024-06-10T01:28:42.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.264+0000 7ff6bcd11b80 1 --2- 172.21.15.7:0/2664817120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a258820e20 0x55a25881a350 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.264+0000 7ff6bcd11b80 1 --2- 172.21.15.7:0/2664817120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a25881ac50 0x55a258822e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.264+0000 7ff6bcd11b80 1 --2- 172.21.15.7:0/2664817120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a258820500 0x55a258819e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.264+0000 7ff6bcd11b80 1 -- 172.21.15.7:0/2664817120 >> 172.21.15.7:0/2664817120 conn(0x55a258747180 msgr2=0x55a258819700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:42.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.264+0000 7ff6bcd11b80 1 -- 172.21.15.7:0/2664817120 shutdown_connections 2024-06-10T01:28:42.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.264+0000 7ff6bcd11b80 1 -- 172.21.15.7:0/2664817120 wait complete. 2024-06-10T01:28:42.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.265+0000 7ff6bcd11b80 1 -- start start 2024-06-10T01:28:42.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.265+0000 7ff6bcd11b80 1 Processor -- start 2024-06-10T01:28:42.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.265+0000 7ff6bcd11b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a25881ac50 0x55a258829a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:42.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.265+0000 7ff6bcd11b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a258820500 0x55a258829fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:42.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.265+0000 7ff6bcd11b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a258820e20 0x55a25882a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:42.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.265+0000 7ff6bcd11b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac1ac8f0 con 0x55a258820500 2024-06-10T01:28:42.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.265+0000 7ff6bcd11b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac1add60 con 0x55a25881ac50 2024-06-10T01:28:42.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.265+0000 7ff6bcd11b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac1acfc0 con 0x55a258820e20 2024-06-10T01:28:42.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.265+0000 7ff6bcd11b80 10 client.0 _ll_register_callbacks cb 0x55a2587016b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:28:42.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.266+0000 7ff6a3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a258820500 0x55a258829fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:42.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.266+0000 7ff6a3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a258820500 0x55a258829fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43700/0 (socket says 172.21.15.7:43700) 2024-06-10T01:28:42.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.266+0000 7ff6a3fff700 1 -- 172.21.15.7:0/4087478847 learned_addr learned my addr 172.21.15.7:0/4087478847 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:42.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.266+0000 7ff6abfff700 1 --2- 172.21.15.7:0/4087478847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a25881ac50 0x55a258829a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:42.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.266+0000 7ff6b0a2b700 1 --2- 172.21.15.7:0/4087478847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a258820e20 0x55a25882a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:42.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.266+0000 7ff6a3fff700 1 -- 172.21.15.7:0/4087478847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a258820e20 msgr2=0x55a25882a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:42.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.266+0000 7ff6a3fff700 1 --2- 172.21.15.7:0/4087478847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a258820e20 0x55a25882a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.266+0000 7ff6a3fff700 1 -- 172.21.15.7:0/4087478847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a25881ac50 msgr2=0x55a258829a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:42.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.266+0000 7ff6a3fff700 1 --2- 172.21.15.7:0/4087478847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a25881ac50 0x55a258829a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.266+0000 7ff6a3fff700 1 -- 172.21.15.7:0/4087478847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a25889f2e0 con 0x55a258820500 2024-06-10T01:28:42.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.266+0000 7ff6a3fff700 1 --2- 172.21.15.7:0/4087478847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a258820500 0x55a258829fd0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7ff694003a00 tx=0x7ff694003700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba44dee715fb7007 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:42.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.266+0000 7ff68b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:28:42.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.266+0000 7ff68b7fe700 1 -- 172.21.15.7:0/4087478847 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6940054f0 con 0x55a258820500 2024-06-10T01:28:42.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.266+0000 7ff68b7fe700 1 -- 172.21.15.7:0/4087478847 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7ff69400f740 con 0x55a258820500 2024-06-10T01:28:42.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.266+0000 7ff6bcd11b80 1 -- 172.21.15.7:0/4087478847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6a409b2f0 con 0x55a258820500 2024-06-10T01:28:42.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.266+0000 7ff68b7fe700 1 -- 172.21.15.7:0/4087478847 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7ff694042050 con 0x55a258820500 2024-06-10T01:28:42.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.266+0000 7ff6bcd11b80 10 client.20832 fetch_fsmap learned FSMap version 55 2024-06-10T01:28:42.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.266+0000 7ff6bcd11b80 1 -- 172.21.15.7:0/4087478847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55a2588a4a30 con 0x55a258820500 2024-06-10T01:28:42.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.267+0000 7ff68b7fe700 1 -- 172.21.15.7:0/4087478847 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6a409b2f0 con 0x55a258820500 2024-06-10T01:28:42.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.267+0000 7ff68b7fe700 1 -- 172.21.15.7:0/4087478847 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7ff69402a040 con 0x55a258820500 2024-06-10T01:28:42.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.267+0000 7ff6bcd11b80 10 client.20832 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:28:42.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.267+0000 7ff6bcd11b80 10 client.20832 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:28:42.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.267+0000 7ff6bcd11b80 20 client.20832 populate_metadata read hostname 'smithi007' 2024-06-10T01:28:42.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.267+0000 7ff6bcd11b80 1 --2- 172.21.15.7:0/4087478847 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a2588a5100 0x55a2588a7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:42.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.267+0000 7ff6bcd11b80 4 client.20832 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8d5f0376", "release": true, "await": true}] 2024-06-10T01:28:42.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.267+0000 7ff6bcd11b80 1 -- 172.21.15.7:0/4087478847 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8d5f0376", "release": true, "await": true}) -- 0x55a2587733d0 con 0x55a2588a5100 2024-06-10T01:28:42.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.267+0000 7ff6abfff700 1 --2- 172.21.15.7:0/4087478847 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a2588a5100 0x55a2588a7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:42.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.268+0000 7ff6abfff700 1 --2- 172.21.15.7:0/4087478847 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a2588a5100 0x55a2588a7520 crc :-1 s=READY pgs=510 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab0fb88cfdfe3c6a server_cookie=7ed8b0928d000dda in_seq=0 out_seq=0 2024-06-10T01:28:42.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.268+0000 7ff68b7fe700 10 client.20832 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:42.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.312+0000 7ff68b7fe700 1 -- 172.21.15.7:0/4087478847 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a2587733d0 con 0x55a2588a5100 2024-06-10T01:28:42.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.312+0000 7ff68b7fe700 10 client.20832 handle_command_reply: tid=0 2024-06-10T01:28:42.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.312+0000 7ff6bcd11b80 1 client.20832 shutdown 2024-06-10T01:28:42.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.312+0000 7ff6bcd11b80 1 -- 172.21.15.7:0/4087478847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a258820500 msgr2=0x55a258829fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:42.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.312+0000 7ff6bcd11b80 1 --2- 172.21.15.7:0/4087478847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a258820500 0x55a258829fd0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7ff694003a00 tx=0x7ff694003700 comp rx=0 tx=0).stop 2024-06-10T01:28:42.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.313+0000 7ff6bcd11b80 1 -- 172.21.15.7:0/4087478847 shutdown_connections 2024-06-10T01:28:42.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.313+0000 7ff6bcd11b80 1 --2- 172.21.15.7:0/4087478847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a258820e20 0x55a25882a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.313+0000 7ff6bcd11b80 1 --2- 172.21.15.7:0/4087478847 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a2588a5100 0x55a2588a7520 crc :-1 s=READY pgs=510 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.313+0000 7ff6bcd11b80 1 --2- 172.21.15.7:0/4087478847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a25881ac50 0x55a258829a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.313+0000 7ff6bcd11b80 1 --2- 172.21.15.7:0/4087478847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a258820500 0x55a258829fd0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.313+0000 7ff6bcd11b80 1 -- 172.21.15.7:0/4087478847 >> 172.21.15.7:0/4087478847 conn(0x55a258747180 msgr2=0x55a258819f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:42.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.313+0000 7ff68b7fe700 0 client.20832 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:42.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.313+0000 7ff6bcd11b80 1 -- 172.21.15.7:0/4087478847 shutdown_connections 2024-06-10T01:28:42.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.313+0000 7ff6bcd11b80 1 -- 172.21.15.7:0/4087478847 wait complete. 2024-06-10T01:28:42.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.313+0000 7ff6bcd11b80 20 client.20832 trim_cache size 0 max 16384 2024-06-10T01:28:42.308 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:28:42.308 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 55, 2024-06-10T01:28:42.308 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:28:42.308 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 72, 2024-06-10T01:28:42.308 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:28:42.309 INFO:teuthology.orchestra.run.smithi007.stdout: "8d5f0376": { 2024-06-10T01:28:42.309 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 72, 2024-06-10T01:28:42.309 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:28:42.309 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:28:42.309 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:28:42.309 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:28:42.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:28:42.309 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:28:42.309 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 142.1, 2024-06-10T01:28:42.310 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:28:42.310 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:28:42.310 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:28:42.310 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:28:42.310 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:28:42.310 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:28:42.310 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:28:42.310 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:28:42.310 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:28:42.311 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:28:42.311 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:28:42.311 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:28:42.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.314+0000 7ff6b248e700 1 -- 172.21.15.7:0/2169491548 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff6980a2c20 msgr2=0x7ff6980a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:42.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.314+0000 7ff6b248e700 1 --2- 172.21.15.7:0/2169491548 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff6980a2c20 0x7ff6980a50e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac199b80 tx=0x7ff69c009260 comp rx=0 tx=0).stop 2024-06-10T01:28:42.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.314+0000 7ff6b248e700 1 -- 172.21.15.7:0/2169491548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6ac107400 msgr2=0x7ff6ac198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:42.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.314+0000 7ff6b248e700 1 --2- 172.21.15.7:0/2169491548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6ac107400 0x7ff6ac198a40 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4007ee0 tx=0x7ff6a400d3b0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.314+0000 7ff6b248e700 1 -- 172.21.15.7:0/2169491548 shutdown_connections 2024-06-10T01:28:42.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.314+0000 7ff6b248e700 1 --2- 172.21.15.7:0/2169491548 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff6980a2c20 0x7ff6980a50e0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.314+0000 7ff6b248e700 1 --2- 172.21.15.7:0/2169491548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6ac107400 0x7ff6ac198a40 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.314+0000 7ff6b248e700 1 --2- 172.21.15.7:0/2169491548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6ac1013c0 0x7ff6ac19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.314+0000 7ff6b248e700 1 --2- 172.21.15.7:0/2169491548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6ac101d60 0x7ff6ac19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:42.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.314+0000 7ff6b248e700 1 -- 172.21.15.7:0/2169491548 >> 172.21.15.7:0/2169491548 conn(0x7ff6ac078330 msgr2=0x7ff6ac100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:42.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.314+0000 7ff6b248e700 1 -- 172.21.15.7:0/2169491548 shutdown_connections 2024-06-10T01:28:42.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:42.314+0000 7ff6b248e700 1 -- 172.21.15.7:0/2169491548 wait complete. 2024-06-10T01:28:42.559 INFO:tasks.workunit.client.0.smithi007.stdout: 10 223185 22.42 MB/sec execute 439 sec latency 17094.609 ms 2024-06-10T01:28:42.964 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8d5f0376', total seconds elapsed: 27.8 2024-06-10T01:28:43.559 INFO:tasks.workunit.client.0.smithi007.stdout: 10 223882 22.45 MB/sec execute 440 sec latency 67.469 ms 2024-06-10T01:28:43.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:43 smithi078 ceph-mon[36683]: pgmap v1115: 129 pgs: 129 active+clean; 691 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 4.7 MiB/s wr, 239 op/s 2024-06-10T01:28:43.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:43 smithi175 ceph-mon[36636]: pgmap v1115: 129 pgs: 129 active+clean; 691 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 4.7 MiB/s wr, 239 op/s 2024-06-10T01:28:44.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:43 smithi007 ceph-mon[32398]: pgmap v1115: 129 pgs: 129 active+clean; 691 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 4.7 MiB/s wr, 239 op/s 2024-06-10T01:28:44.559 INFO:tasks.workunit.client.0.smithi007.stdout: 10 224355 22.45 MB/sec execute 441 sec latency 82.963 ms 2024-06-10T01:28:45.559 INFO:tasks.workunit.client.0.smithi007.stdout: 10 225050 22.49 MB/sec execute 442 sec latency 90.593 ms 2024-06-10T01:28:45.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:45 smithi078 ceph-mon[36683]: pgmap v1116: 129 pgs: 129 active+clean; 690 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 2.3 MiB/s wr, 132 op/s 2024-06-10T01:28:45.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:45 smithi175 ceph-mon[36636]: pgmap v1116: 129 pgs: 129 active+clean; 690 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 2.3 MiB/s wr, 132 op/s 2024-06-10T01:28:46.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:45 smithi007 ceph-mon[32398]: pgmap v1116: 129 pgs: 129 active+clean; 690 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 2.3 MiB/s wr, 132 op/s 2024-06-10T01:28:46.559 INFO:tasks.workunit.client.0.smithi007.stdout: 10 225737 22.52 MB/sec execute 443 sec latency 78.137 ms 2024-06-10T01:28:47.559 INFO:tasks.workunit.client.0.smithi007.stdout: 10 226485 22.56 MB/sec execute 444 sec latency 68.887 ms 2024-06-10T01:28:47.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:47 smithi078 ceph-mon[36683]: pgmap v1117: 129 pgs: 129 active+clean; 691 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 134 op/s 2024-06-10T01:28:47.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:47 smithi175 ceph-mon[36636]: pgmap v1117: 129 pgs: 129 active+clean; 691 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 134 op/s 2024-06-10T01:28:48.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:47 smithi007 ceph-mon[32398]: pgmap v1117: 129 pgs: 129 active+clean; 691 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 134 op/s 2024-06-10T01:28:48.559 INFO:tasks.workunit.client.0.smithi007.stdout: 10 227007 22.60 MB/sec execute 445 sec latency 76.148 ms 2024-06-10T01:28:49.559 INFO:tasks.workunit.client.0.smithi007.stdout: 10 227458 22.59 MB/sec execute 446 sec latency 118.921 ms 2024-06-10T01:28:49.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:49 smithi078 ceph-mon[36683]: pgmap v1118: 129 pgs: 129 active+clean; 687 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.8 MiB/s wr, 217 op/s 2024-06-10T01:28:49.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:49 smithi175 ceph-mon[36636]: pgmap v1118: 129 pgs: 129 active+clean; 687 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.8 MiB/s wr, 217 op/s 2024-06-10T01:28:50.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:49 smithi007 ceph-mon[32398]: pgmap v1118: 129 pgs: 129 active+clean; 687 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.8 MiB/s wr, 217 op/s 2024-06-10T01:28:50.559 INFO:tasks.workunit.client.0.smithi007.stdout: 10 227986 22.59 MB/sec execute 447 sec latency 100.869 ms 2024-06-10T01:28:51.539 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:28:51.559 INFO:tasks.workunit.client.0.smithi007.stdout: 10 228734 22.63 MB/sec execute 448 sec latency 81.365 ms 2024-06-10T01:28:51.817 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:51 smithi007 ceph-mon[32398]: pgmap v1119: 129 pgs: 129 active+clean; 682 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.8 MiB/s wr, 266 op/s 2024-06-10T01:28:51.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:51 smithi078 ceph-mon[36683]: pgmap v1119: 129 pgs: 129 active+clean; 682 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.8 MiB/s wr, 266 op/s 2024-06-10T01:28:51.864 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:28:51.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:51 smithi175 ceph-mon[36636]: pgmap v1119: 129 pgs: 129 active+clean; 682 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.8 MiB/s wr, 266 op/s 2024-06-10T01:28:52.559 INFO:tasks.workunit.client.0.smithi007.stdout: 10 229446 22.66 MB/sec execute 449 sec latency 79.662 ms 2024-06-10T01:28:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.513+0000 7fd5a37b7640 1 -- 172.21.15.7:0/3693131620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd59c104db0 msgr2=0x7fd59c1071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.513+0000 7fd5a37b7640 1 --2- 172.21.15.7:0/3693131620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd59c104db0 0x7fd59c1071a0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009860 tx=0x7fd58c02a340 comp rx=0 tx=0).stop 2024-06-10T01:28:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.514+0000 7fd5a37b7640 1 -- 172.21.15.7:0/3693131620 shutdown_connections 2024-06-10T01:28:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.514+0000 7fd5a37b7640 1 --2- 172.21.15.7:0/3693131620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd59c1076e0 0x7fd59c109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.514+0000 7fd5a37b7640 1 --2- 172.21.15.7:0/3693131620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd59c104db0 0x7fd59c1071a0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.514+0000 7fd5a37b7640 1 --2- 172.21.15.7:0/3693131620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd59c102480 0x7fd59c104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.514+0000 7fd5a37b7640 1 -- 172.21.15.7:0/3693131620 >> 172.21.15.7:0/3693131620 conn(0x7fd59c0fbe70 msgr2=0x7fd59c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.514+0000 7fd5a37b7640 1 -- 172.21.15.7:0/3693131620 shutdown_connections 2024-06-10T01:28:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.514+0000 7fd5a37b7640 1 -- 172.21.15.7:0/3693131620 wait complete. 2024-06-10T01:28:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.514+0000 7fd5a37b7640 1 Processor -- start 2024-06-10T01:28:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.514+0000 7fd5a37b7640 1 -- start start 2024-06-10T01:28:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.515+0000 7fd5a37b7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd59c102480 0x7fd59c197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.515+0000 7fd5a37b7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd59c104db0 0x7fd59c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.515+0000 7fd5a37b7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd59c1076e0 0x7fd59c19f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.515+0000 7fd5a37b7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c10d670 con 0x7fd59c102480 2024-06-10T01:28:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.515+0000 7fd5a37b7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c10d4f0 con 0x7fd59c1076e0 2024-06-10T01:28:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.515+0000 7fd5a37b7640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c10d7f0 con 0x7fd59c104db0 2024-06-10T01:28:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.515+0000 7fd5a1553640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd59c102480 0x7fd59c197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.515+0000 7fd5a0d52640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd59c104db0 0x7fd59c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.515+0000 7fd5a1553640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd59c102480 0x7fd59c197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41634/0 (socket says 172.21.15.7:41634) 2024-06-10T01:28:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.515+0000 7fd5a0d52640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd59c104db0 0x7fd59c198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:51040/0 (socket says 172.21.15.7:51040) 2024-06-10T01:28:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.515+0000 7fd5a1553640 1 -- 172.21.15.7:0/702455810 learned_addr learned my addr 172.21.15.7:0/702455810 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.515+0000 7fd5a1d54640 1 --2- 172.21.15.7:0/702455810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd59c1076e0 0x7fd59c19f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.515+0000 7fd5a1553640 1 -- 172.21.15.7:0/702455810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd59c104db0 msgr2=0x7fd59c198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.515+0000 7fd5a1553640 1 --2- 172.21.15.7:0/702455810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd59c104db0 0x7fd59c198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.515+0000 7fd5a1553640 1 -- 172.21.15.7:0/702455810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd59c1076e0 msgr2=0x7fd59c19f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.515+0000 7fd5a1553640 1 --2- 172.21.15.7:0/702455810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd59c1076e0 0x7fd59c19f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.515+0000 7fd5a1553640 1 -- 172.21.15.7:0/702455810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd59c19f960 con 0x7fd59c102480 2024-06-10T01:28:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.515+0000 7fd5a1553640 1 --2- 172.21.15.7:0/702455810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd59c102480 0x7fd59c197d40 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fd59400c420 tx=0x7fd594009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.515+0000 7fd5927fc640 1 -- 172.21.15.7:0/702455810 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd594003970 con 0x7fd59c102480 2024-06-10T01:28:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.515+0000 7fd5927fc640 1 -- 172.21.15.7:0/702455810 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd594004020 con 0x7fd59c102480 2024-06-10T01:28:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.515+0000 7fd5a37b7640 1 -- 172.21.15.7:0/702455810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd59c19fc50 con 0x7fd59c102480 2024-06-10T01:28:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.515+0000 7fd5927fc640 1 -- 172.21.15.7:0/702455810 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59400bad0 con 0x7fd59c102480 2024-06-10T01:28:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.516+0000 7fd5a37b7640 1 -- 172.21.15.7:0/702455810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd59c1a0170 con 0x7fd59c102480 2024-06-10T01:28:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.518+0000 7fd5927fc640 1 -- 172.21.15.7:0/702455810 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fd59401d020 con 0x7fd59c102480 2024-06-10T01:28:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.519+0000 7fd5a65a5740 1 Processor -- start 2024-06-10T01:28:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.519+0000 7fd5a65a5740 1 -- start start 2024-06-10T01:28:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.519+0000 7fd5a65a5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abb1b40010 0x55abb1b40430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.519+0000 7fd5a65a5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abb1b40a00 0x55abb1b45500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.519+0000 7fd5a65a5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abb1b45b70 0x55abb1b48000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.519+0000 7fd5a65a5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55abb1aa34f0 con 0x55abb1b45b70 2024-06-10T01:28:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.519+0000 7fd5a65a5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55abb1a41870 con 0x55abb1b40010 2024-06-10T01:28:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.519+0000 7fd5a65a5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55abb1a9f4b0 con 0x55abb1b40a00 2024-06-10T01:28:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.519+0000 7fd5927fc640 1 --2- 172.21.15.7:0/702455810 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd5880a2b80 0x7fd5880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.519+0000 7fd5a0d52640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abb1b45b70 0x55abb1b48000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.519+0000 7fd5a1553640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abb1b40a00 0x55abb1b45500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.519+0000 7fd5a0d52640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abb1b45b70 0x55abb1b48000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41638/0 (socket says 172.21.15.7:41638) 2024-06-10T01:28:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.519+0000 7fd5a0d52640 1 -- 172.21.15.7:0/1881002991 learned_addr learned my addr 172.21.15.7:0/1881002991 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.519+0000 7fd5a1d54640 1 --2- 172.21.15.7:0/702455810 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd5880a2b80 0x7fd5880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.519+0000 7fd5927fc640 1 -- 172.21.15.7:0/702455810 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fd5940cf210 con 0x7fd59c102480 2024-06-10T01:28:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.520+0000 7fd5a0d52640 1 -- 172.21.15.7:0/1881002991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abb1b40a00 msgr2=0x55abb1b45500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.520+0000 7fd5a0d52640 1 --2- 172.21.15.7:0/1881002991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abb1b40a00 0x55abb1b45500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.520+0000 7fd5a0d52640 1 -- 172.21.15.7:0/1881002991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abb1b40010 msgr2=0x55abb1b40430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.520+0000 7fd5a0d52640 1 --2- 172.21.15.7:0/1881002991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abb1b40010 0x55abb1b40430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.520+0000 7fd5a0d52640 1 -- 172.21.15.7:0/1881002991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abb1a5d9b0 con 0x55abb1b45b70 2024-06-10T01:28:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.520+0000 7fd5a0d52640 1 --2- 172.21.15.7:0/1881002991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abb1b45b70 0x55abb1b48000 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fd58c02a930 tx=0x7fd58c0095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c388f32d9aed63 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.520+0000 7fd5a1d54640 1 --2- 172.21.15.7:0/702455810 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd5880a2b80 0x7fd5880a5040 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fd598009710 tx=0x7fd598009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.520+0000 7fd5877fe640 1 -- 172.21.15.7:0/1881002991 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c037040 con 0x55abb1b45b70 2024-06-10T01:28:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.520+0000 7fd5877fe640 1 -- 172.21.15.7:0/1881002991 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd58c030e50 con 0x55abb1b45b70 2024-06-10T01:28:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.520+0000 7fd5877fe640 1 -- 172.21.15.7:0/1881002991 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c005500 con 0x55abb1b45b70 2024-06-10T01:28:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.521+0000 7fd5a65a5740 1 -- 172.21.15.7:0/1881002991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abb1b45b70 msgr2=0x55abb1b48000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.521+0000 7fd5a65a5740 1 --2- 172.21.15.7:0/1881002991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abb1b45b70 0x55abb1b48000 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fd58c02a930 tx=0x7fd58c0095a0 comp rx=0 tx=0).stop 2024-06-10T01:28:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.521+0000 7fd5a65a5740 1 -- 172.21.15.7:0/1881002991 shutdown_connections 2024-06-10T01:28:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.521+0000 7fd5a65a5740 1 --2- 172.21.15.7:0/1881002991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abb1b45b70 0x55abb1b48000 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.521+0000 7fd5a65a5740 1 --2- 172.21.15.7:0/1881002991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abb1b40a00 0x55abb1b45500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.521+0000 7fd5a65a5740 1 --2- 172.21.15.7:0/1881002991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abb1b40010 0x55abb1b40430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.521+0000 7fd5a65a5740 1 -- 172.21.15.7:0/1881002991 >> 172.21.15.7:0/1881002991 conn(0x55abb1b3b680 msgr2=0x55abb1b3ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.521+0000 7fd5a65a5740 1 -- 172.21.15.7:0/1881002991 shutdown_connections 2024-06-10T01:28:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.521+0000 7fd5a65a5740 1 -- 172.21.15.7:0/1881002991 wait complete. 2024-06-10T01:28:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.521+0000 7fd5a65a5740 1 -- start start 2024-06-10T01:28:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.521+0000 7fd5a65a5740 1 Processor -- start 2024-06-10T01:28:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.522+0000 7fd5a65a5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abb1b40010 0x55abb1b4a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.522+0000 7fd5a65a5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abb1b40a00 0x55abb1b4a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.522+0000 7fd5a65a5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abb1b45b70 0x55abb1b517a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.522+0000 7fd5a65a5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55abb1b48ac0 con 0x55abb1b45b70 2024-06-10T01:28:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.522+0000 7fd5a65a5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55abb1a2cae0 con 0x55abb1b40a00 2024-06-10T01:28:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.522+0000 7fd5a65a5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55abb1b48c40 con 0x55abb1b40010 2024-06-10T01:28:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.522+0000 7fd5a65a5740 10 client.0 _ll_register_callbacks cb 0x55abb1a659b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:28:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.522+0000 7fd5a1d54640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abb1b45b70 0x55abb1b517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.522+0000 7fd5a0d52640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abb1b40a00 0x55abb1b4a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.522+0000 7fd5a1d54640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abb1b45b70 0x55abb1b517a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41646/0 (socket says 172.21.15.7:41646) 2024-06-10T01:28:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.522+0000 7fd5a1d54640 1 -- 172.21.15.7:0/2898516113 learned_addr learned my addr 172.21.15.7:0/2898516113 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.522+0000 7fd5a1553640 1 --2- 172.21.15.7:0/2898516113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abb1b40010 0x55abb1b4a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.522+0000 7fd5a1d54640 1 -- 172.21.15.7:0/2898516113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abb1b40010 msgr2=0x55abb1b4a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.522+0000 7fd5a1d54640 1 --2- 172.21.15.7:0/2898516113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abb1b40010 0x55abb1b4a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.522+0000 7fd5a1d54640 1 -- 172.21.15.7:0/2898516113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abb1b40a00 msgr2=0x55abb1b4a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.522+0000 7fd5a1d54640 1 --2- 172.21.15.7:0/2898516113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abb1b40a00 0x55abb1b4a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.522+0000 7fd5a1d54640 1 -- 172.21.15.7:0/2898516113 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abb17b7210 con 0x55abb1b45b70 2024-06-10T01:28:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.522+0000 7fd5a1553640 1 --2- 172.21.15.7:0/2898516113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abb1b40010 0x55abb1b4a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:28:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.523+0000 7fd5a1d54640 1 --2- 172.21.15.7:0/2898516113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abb1b45b70 0x55abb1b517a0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fd59801d970 tx=0x7fd598004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9499faadc20da7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.523+0000 7fd584ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:28:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.523+0000 7fd584ff9640 1 -- 172.21.15.7:0/2898516113 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59802c060 con 0x55abb1b45b70 2024-06-10T01:28:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.523+0000 7fd584ff9640 1 -- 172.21.15.7:0/2898516113 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fd598009d80 con 0x55abb1b45b70 2024-06-10T01:28:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.523+0000 7fd5a65a5740 1 -- 172.21.15.7:0/2898516113 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd58c030e50 con 0x55abb1b45b70 2024-06-10T01:28:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.523+0000 7fd584ff9640 1 -- 172.21.15.7:0/2898516113 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fd598033050 con 0x55abb1b45b70 2024-06-10T01:28:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.523+0000 7fd5a65a5740 10 client.20844 fetch_fsmap learned FSMap version 55 2024-06-10T01:28:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.523+0000 7fd5a65a5740 1 -- 172.21.15.7:0/2898516113 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55abb1883620 con 0x55abb1b45b70 2024-06-10T01:28:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.523+0000 7fd584ff9640 1 -- 172.21.15.7:0/2898516113 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd58c030e50 con 0x55abb1b45b70 2024-06-10T01:28:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.523+0000 7fd584ff9640 1 -- 172.21.15.7:0/2898516113 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fd598027a70 con 0x55abb1b45b70 2024-06-10T01:28:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.523+0000 7fd5a65a5740 10 client.20844 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:28:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.523+0000 7fd5a65a5740 10 client.20844 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:28:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.523+0000 7fd5a65a5740 20 client.20844 populate_metadata read hostname 'smithi007' 2024-06-10T01:28:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.523+0000 7fd5a65a5740 1 --2- 172.21.15.7:0/2898516113 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55abb1b56150 0x55abb1b58580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.523+0000 7fd5a65a5740 4 client.20844 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:28:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.523+0000 7fd5a65a5740 1 -- 172.21.15.7:0/2898516113 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abb1ab6f10 con 0x55abb1b56150 2024-06-10T01:28:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.524+0000 7fd5a1553640 1 --2- 172.21.15.7:0/2898516113 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55abb1b56150 0x55abb1b58580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.524+0000 7fd584ff9640 10 client.20844 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.524+0000 7fd5a1553640 1 --2- 172.21.15.7:0/2898516113 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55abb1b56150 0x55abb1b58580 crc :-1 s=READY pgs=511 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffba7221d4503c47 server_cookie=7092a40b9458103 in_seq=0 out_seq=0 2024-06-10T01:28:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.531+0000 7fd584ff9640 1 -- 172.21.15.7:0/2898516113 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55abb1ab6f10 con 0x55abb1b56150 2024-06-10T01:28:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.531+0000 7fd584ff9640 10 client.20844 handle_command_reply: tid=0 2024-06-10T01:28:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.531+0000 7fd5a65a5740 1 client.20844 shutdown 2024-06-10T01:28:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.531+0000 7fd5a65a5740 1 -- 172.21.15.7:0/2898516113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abb1b45b70 msgr2=0x55abb1b517a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.531+0000 7fd5a65a5740 1 --2- 172.21.15.7:0/2898516113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abb1b45b70 0x55abb1b517a0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fd59801d970 tx=0x7fd598004730 comp rx=0 tx=0).stop 2024-06-10T01:28:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.531+0000 7fd5a65a5740 1 -- 172.21.15.7:0/2898516113 shutdown_connections 2024-06-10T01:28:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.532+0000 7fd5a65a5740 1 --2- 172.21.15.7:0/2898516113 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55abb1b56150 0x55abb1b58580 crc :-1 s=READY pgs=511 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.532+0000 7fd5a65a5740 1 --2- 172.21.15.7:0/2898516113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abb1b45b70 0x55abb1b517a0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.532+0000 7fd5a65a5740 1 --2- 172.21.15.7:0/2898516113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abb1b40a00 0x55abb1b4a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.532+0000 7fd584ff9640 0 client.20844 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.532+0000 7fd5a65a5740 1 --2- 172.21.15.7:0/2898516113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abb1b40010 0x55abb1b4a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.532+0000 7fd5a65a5740 1 -- 172.21.15.7:0/2898516113 >> 172.21.15.7:0/2898516113 conn(0x55abb1b3b680 msgr2=0x55abb1b3a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.532+0000 7fd5a65a5740 1 -- 172.21.15.7:0/2898516113 shutdown_connections 2024-06-10T01:28:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.532+0000 7fd5a65a5740 1 -- 172.21.15.7:0/2898516113 wait complete. 2024-06-10T01:28:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.532+0000 7fd5a65a5740 20 client.20844 trim_cache size 0 max 16384 2024-06-10T01:28:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.554+0000 7fd5a65a5740 1 Processor -- start 2024-06-10T01:28:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.554+0000 7fd5a65a5740 1 -- start start 2024-06-10T01:28:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.554+0000 7fd5a65a5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abb1b40010 0x55abb1b47f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.554+0000 7fd5a65a5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abb1b40a00 0x55abb1b51760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.554+0000 7fd5a65a5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abb1b45b70 0x55abb1b4a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.554+0000 7fd5a65a5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c1ace60 con 0x55abb1b45b70 2024-06-10T01:28:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.554+0000 7fd5a65a5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c1aa730 con 0x55abb1b40010 2024-06-10T01:28:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.554+0000 7fd5a65a5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c1a8580 con 0x55abb1b40a00 2024-06-10T01:28:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.554+0000 7fd5a1d54640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abb1b45b70 0x55abb1b4a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.554+0000 7fd5a1d54640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abb1b45b70 0x55abb1b4a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41658/0 (socket says 172.21.15.7:41658) 2024-06-10T01:28:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.554+0000 7fd5a1553640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abb1b40010 0x55abb1b47f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.554+0000 7fd5a1d54640 1 -- 172.21.15.7:0/2494566579 learned_addr learned my addr 172.21.15.7:0/2494566579 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.554+0000 7fd5a0d52640 1 --2- 172.21.15.7:0/2494566579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abb1b40a00 0x55abb1b51760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.555+0000 7fd5a1553640 1 -- 172.21.15.7:0/2494566579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abb1b40a00 msgr2=0x55abb1b51760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.555+0000 7fd5a1553640 1 --2- 172.21.15.7:0/2494566579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abb1b40a00 0x55abb1b51760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.555+0000 7fd5a1553640 1 -- 172.21.15.7:0/2494566579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abb1b45b70 msgr2=0x55abb1b4a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.555+0000 7fd5a1553640 1 --2- 172.21.15.7:0/2494566579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abb1b45b70 0x55abb1b4a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.555+0000 7fd5a1553640 1 -- 172.21.15.7:0/2494566579 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abb1b3b4b0 con 0x55abb1b40010 2024-06-10T01:28:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.555+0000 7fd5a1d54640 1 --2- 172.21.15.7:0/2494566579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abb1b45b70 0x55abb1b4a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:28:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.555+0000 7fd5a1553640 1 --2- 172.21.15.7:0/2494566579 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abb1b40010 0x55abb1b47f60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd5940980c0 tx=0x7fd5940bb920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ca4cab31b5f205 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.555+0000 7fd586ffd640 1 -- 172.21.15.7:0/2494566579 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59409b6f0 con 0x55abb1b40010 2024-06-10T01:28:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.555+0000 7fd586ffd640 1 -- 172.21.15.7:0/2494566579 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5940cce50 con 0x55abb1b40010 2024-06-10T01:28:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.555+0000 7fd586ffd640 1 -- 172.21.15.7:0/2494566579 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59400bc70 con 0x55abb1b40010 2024-06-10T01:28:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.556+0000 7fd5a65a5740 1 -- 172.21.15.7:0/2494566579 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abb1b40010 msgr2=0x55abb1b47f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.556+0000 7fd5a65a5740 1 --2- 172.21.15.7:0/2494566579 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abb1b40010 0x55abb1b47f60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd5940980c0 tx=0x7fd5940bb920 comp rx=0 tx=0).stop 2024-06-10T01:28:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.556+0000 7fd5a65a5740 1 -- 172.21.15.7:0/2494566579 shutdown_connections 2024-06-10T01:28:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.556+0000 7fd5a65a5740 1 --2- 172.21.15.7:0/2494566579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abb1b45b70 0x55abb1b4a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.556+0000 7fd5a65a5740 1 --2- 172.21.15.7:0/2494566579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abb1b40a00 0x55abb1b51760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.556+0000 7fd5a65a5740 1 --2- 172.21.15.7:0/2494566579 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abb1b40010 0x55abb1b47f60 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.556+0000 7fd5a65a5740 1 -- 172.21.15.7:0/2494566579 >> 172.21.15.7:0/2494566579 conn(0x55abb1b3b680 msgr2=0x55abb1b452b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.556+0000 7fd5a65a5740 1 -- 172.21.15.7:0/2494566579 shutdown_connections 2024-06-10T01:28:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.556+0000 7fd5a65a5740 1 -- 172.21.15.7:0/2494566579 wait complete. 2024-06-10T01:28:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.556+0000 7fd5a65a5740 1 -- start start 2024-06-10T01:28:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.556+0000 7fd5a65a5740 1 Processor -- start 2024-06-10T01:28:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.557+0000 7fd5a65a5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abb1b40010 0x55abb1b514a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.557+0000 7fd5a65a5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abb1b40a00 0x55abb1b4df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.557+0000 7fd5a65a5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abb1b45b70 0x55abb1b4e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.557+0000 7fd5a65a5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c1a8bf0 con 0x55abb1b40a00 2024-06-10T01:28:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.557+0000 7fd5a65a5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c1aa060 con 0x55abb1b45b70 2024-06-10T01:28:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.557+0000 7fd5a65a5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c1a92c0 con 0x55abb1b40010 2024-06-10T01:28:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.557+0000 7fd5a65a5740 10 client.0 _ll_register_callbacks cb 0x7fd598009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:28:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.557+0000 7fd5a0d52640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abb1b40a00 0x55abb1b4df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.557+0000 7fd5a1d54640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abb1b45b70 0x55abb1b4e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.557+0000 7fd5a1553640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abb1b40010 0x55abb1b514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.557+0000 7fd5a0d52640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abb1b40a00 0x55abb1b4df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41660/0 (socket says 172.21.15.7:41660) 2024-06-10T01:28:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.557+0000 7fd5a0d52640 1 -- 172.21.15.7:0/276219818 learned_addr learned my addr 172.21.15.7:0/276219818 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.557+0000 7fd5a0d52640 1 -- 172.21.15.7:0/276219818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abb1b40010 msgr2=0x55abb1b514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.557+0000 7fd5a0d52640 1 --2- 172.21.15.7:0/276219818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abb1b40010 0x55abb1b514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.557+0000 7fd5a0d52640 1 -- 172.21.15.7:0/276219818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abb1b45b70 msgr2=0x55abb1b4e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.557+0000 7fd5a0d52640 1 --2- 172.21.15.7:0/276219818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abb1b45b70 0x55abb1b4e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.557+0000 7fd5a0d52640 1 -- 172.21.15.7:0/276219818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abb1b4ea70 con 0x55abb1b40a00 2024-06-10T01:28:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.557+0000 7fd5a1d54640 1 --2- 172.21.15.7:0/276219818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abb1b45b70 0x55abb1b4e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:28:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.557+0000 7fd5a1553640 1 --2- 172.21.15.7:0/276219818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abb1b40010 0x55abb1b514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:28:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.558+0000 7fd5a0d52640 1 --2- 172.21.15.7:0/276219818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abb1b40a00 0x55abb1b4df40 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fd58c02c010 tx=0x7fd58c030740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbdb9a606d7d7367 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.558+0000 7fd584ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:28:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.558+0000 7fd584ff9640 1 -- 172.21.15.7:0/276219818 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c037040 con 0x55abb1b40a00 2024-06-10T01:28:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.558+0000 7fd584ff9640 1 -- 172.21.15.7:0/276219818 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fd58c005230 con 0x55abb1b40a00 2024-06-10T01:28:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.558+0000 7fd5a65a5740 1 -- 172.21.15.7:0/276219818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5940cce50 con 0x55abb1b40a00 2024-06-10T01:28:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.558+0000 7fd584ff9640 1 -- 172.21.15.7:0/276219818 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fd58c05a050 con 0x55abb1b40a00 2024-06-10T01:28:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.558+0000 7fd5a65a5740 10 client.20850 fetch_fsmap learned FSMap version 55 2024-06-10T01:28:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.558+0000 7fd5a65a5740 1 -- 172.21.15.7:0/276219818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55abb1bbff00 con 0x55abb1b40a00 2024-06-10T01:28:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.558+0000 7fd584ff9640 1 -- 172.21.15.7:0/276219818 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5940cce50 con 0x55abb1b40a00 2024-06-10T01:28:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.558+0000 7fd584ff9640 1 -- 172.21.15.7:0/276219818 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fd58c03d5a0 con 0x55abb1b40a00 2024-06-10T01:28:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.558+0000 7fd5a65a5740 10 client.20850 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:28:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.558+0000 7fd5a65a5740 10 client.20850 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:28:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.558+0000 7fd5a65a5740 20 client.20850 populate_metadata read hostname 'smithi007' 2024-06-10T01:28:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.558+0000 7fd5a65a5740 1 --2- 172.21.15.7:0/276219818 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55abb1bc0470 0x55abb1bc2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.559+0000 7fd5a65a5740 4 client.20850 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:28:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.559+0000 7fd5a65a5740 1 -- 172.21.15.7:0/276219818 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55abb1ab0c20 con 0x55abb1bc0470 2024-06-10T01:28:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.559+0000 7fd5a1553640 1 --2- 172.21.15.7:0/276219818 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55abb1bc0470 0x55abb1bc2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.559+0000 7fd5a1553640 1 --2- 172.21.15.7:0/276219818 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55abb1bc0470 0x55abb1bc2860 crc :-1 s=READY pgs=512 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a9f8121d13016ca server_cookie=78f0756101945556 in_seq=0 out_seq=0 2024-06-10T01:28:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:53.559+0000 7fd584ff9640 10 client.20850 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:53.560 INFO:tasks.workunit.client.0.smithi007.stdout: 10 229989 22.68 MB/sec execute 450 sec latency 58.713 ms 2024-06-10T01:28:53.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:53 smithi078 ceph-mon[36683]: pgmap v1120: 129 pgs: 129 active+clean; 664 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 2.5 MiB/s wr, 348 op/s 2024-06-10T01:28:53.897 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:53 smithi007 ceph-mon[32398]: pgmap v1120: 129 pgs: 129 active+clean; 664 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 2.5 MiB/s wr, 348 op/s 2024-06-10T01:28:53.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:53 smithi175 ceph-mon[36636]: pgmap v1120: 129 pgs: 129 active+clean; 664 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 2.5 MiB/s wr, 348 op/s 2024-06-10T01:28:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.269+0000 7f6c4b59e700 1 -- 172.21.15.7:0/3183750336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c4c110ff0 msgr2=0x7f6c4c1099d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.269+0000 7f6c4b59e700 1 --2- 172.21.15.7:0/3183750336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c4c110ff0 0x7f6c4c1099d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400b0a0 tx=0x7f6c4400b3b0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.270+0000 7f6c4b59e700 1 -- 172.21.15.7:0/3183750336 shutdown_connections 2024-06-10T01:28:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.270+0000 7f6c4b59e700 1 --2- 172.21.15.7:0/3183750336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c4c11bc70 0x7f6c4c11e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.270+0000 7f6c4b59e700 1 --2- 172.21.15.7:0/3183750336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c4c110b80 0x7f6c4c109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.270+0000 7f6c4b59e700 1 --2- 172.21.15.7:0/3183750336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c4c110ff0 0x7f6c4c1099d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.270+0000 7f6c4b59e700 1 -- 172.21.15.7:0/3183750336 >> 172.21.15.7:0/3183750336 conn(0x7f6c4c06d470 msgr2=0x7f6c4c06d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:54.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.275+0000 7f6c4b59e700 1 -- 172.21.15.7:0/3183750336 shutdown_connections 2024-06-10T01:28:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.276+0000 7f6c4b59e700 1 -- 172.21.15.7:0/3183750336 wait complete. 2024-06-10T01:28:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.277+0000 7f6c4b59e700 1 Processor -- start 2024-06-10T01:28:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.277+0000 7f6c4b59e700 1 -- start start 2024-06-10T01:28:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.277+0000 7f6c4b59e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c4c110b80 0x7f6c4c1ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.277+0000 7f6c4b59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c4c110ff0 0x7f6c4c1adc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.277+0000 7f6c4b59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c4c11bc70 0x7f6c4c1a7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.277+0000 7f6c4b59e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1215c0 con 0x7f6c4c11bc70 2024-06-10T01:28:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.277+0000 7f6c4b59e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c121440 con 0x7f6c4c110ff0 2024-06-10T01:28:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.277+0000 7f6c4b59e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c121740 con 0x7f6c4c110b80 2024-06-10T01:28:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.282+0000 7f6c4ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c4c11bc70 0x7f6c4c1a7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.282+0000 7f6c4ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c4c11bc70 0x7f6c4c1a7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41688/0 (socket says 172.21.15.7:41688) 2024-06-10T01:28:54.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.282+0000 7f6c4ad9d700 1 -- 172.21.15.7:0/236581300 learned_addr learned my addr 172.21.15.7:0/236581300 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:54.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.282+0000 7f6c49d9b700 1 --2- 172.21.15.7:0/236581300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c4c110ff0 0x7f6c4c1adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:54.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.282+0000 7f6c4a59c700 1 --2- 172.21.15.7:0/236581300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c4c110b80 0x7f6c4c1ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:54.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.282+0000 7f6c49d9b700 1 -- 172.21.15.7:0/236581300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c4c110b80 msgr2=0x7f6c4c1ad6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:54.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.282+0000 7f6c49d9b700 1 --2- 172.21.15.7:0/236581300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c4c110b80 0x7f6c4c1ad6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.282+0000 7f6c49d9b700 1 -- 172.21.15.7:0/236581300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c4c11bc70 msgr2=0x7f6c4c1a7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:54.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.282+0000 7f6c49d9b700 1 --2- 172.21.15.7:0/236581300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c4c11bc70 0x7f6c4c1a7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.282+0000 7f6c49d9b700 1 -- 172.21.15.7:0/236581300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4c1a8020 con 0x7f6c4c110ff0 2024-06-10T01:28:54.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.283+0000 7f6c4a59c700 1 --2- 172.21.15.7:0/236581300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c4c110b80 0x7f6c4c1ad6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:28:54.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.283+0000 7f6c49d9b700 1 --2- 172.21.15.7:0/236581300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c4c110ff0 0x7f6c4c1adc00 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6c44000f80 tx=0x7f6c44004dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:54.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.283+0000 7f6c3b7fe700 1 -- 172.21.15.7:0/236581300 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44021070 con 0x7f6c4c110ff0 2024-06-10T01:28:54.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.283+0000 7f6c4b59e700 1 -- 172.21.15.7:0/236581300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c4c1a82b0 con 0x7f6c4c110ff0 2024-06-10T01:28:54.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.283+0000 7f6c4b59e700 1 -- 172.21.15.7:0/236581300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c4c1b4520 con 0x7f6c4c110ff0 2024-06-10T01:28:54.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.284+0000 7f6c3b7fe700 1 -- 172.21.15.7:0/236581300 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c44007cc0 con 0x7f6c4c110ff0 2024-06-10T01:28:54.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.284+0000 7f6c3b7fe700 1 -- 172.21.15.7:0/236581300 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44005370 con 0x7f6c4c110ff0 2024-06-10T01:28:54.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.295+0000 7f6c3b7fe700 1 -- 172.21.15.7:0/236581300 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f6c44024070 con 0x7f6c4c110ff0 2024-06-10T01:28:54.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.295+0000 7f6c5b1a4b80 1 Processor -- start 2024-06-10T01:28:54.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.296+0000 7f6c5b1a4b80 1 -- start start 2024-06-10T01:28:54.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.296+0000 7f6c5b1a4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574d5c538f0 0x5574d5c53cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:54.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.297+0000 7f6c5b1a4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574d5c54210 0x5574d5c4db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:54.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.297+0000 7f6c5b1a4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574d5c4e080 0x5574d5c4e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:54.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.297+0000 7f6c5b1a4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5574d5b9f810 con 0x5574d5c538f0 2024-06-10T01:28:54.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.297+0000 7f6c5b1a4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5574d5b47720 con 0x5574d5c4e080 2024-06-10T01:28:54.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.297+0000 7f6c5b1a4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5574d5b9b7d0 con 0x5574d5c54210 2024-06-10T01:28:54.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.298+0000 7f6c3b7fe700 1 --2- 172.21.15.7:0/236581300 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6c340a2e20 0x7f6c340a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:54.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.298+0000 7f6c3b7fe700 1 -- 172.21.15.7:0/236581300 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f6c440d1660 con 0x7f6c4c110ff0 2024-06-10T01:28:54.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.302+0000 7f6c49d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574d5c4e080 0x5574d5c4e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:54.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.303+0000 7f6c4a59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574d5c54210 0x5574d5c4db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:54.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.303+0000 7f6c4a59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574d5c54210 0x5574d5c4db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:51088/0 (socket says 172.21.15.7:51088) 2024-06-10T01:28:54.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.303+0000 7f6c4a59c700 1 -- 172.21.15.7:0/3888444349 learned_addr learned my addr 172.21.15.7:0/3888444349 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:54.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.303+0000 7f6c4ad9d700 1 --2- 172.21.15.7:0/236581300 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6c340a2e20 0x7f6c340a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:54.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.303+0000 7f6c4ad9d700 1 --2- 172.21.15.7:0/3888444349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574d5c538f0 0x5574d5c53cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:54.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.306+0000 7f6c4a59c700 1 -- 172.21.15.7:0/3888444349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574d5c4e080 msgr2=0x5574d5c4e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:54.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.306+0000 7f6c4a59c700 1 --2- 172.21.15.7:0/3888444349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574d5c4e080 0x5574d5c4e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.306+0000 7f6c4a59c700 1 -- 172.21.15.7:0/3888444349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574d5c538f0 msgr2=0x5574d5c53cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.306+0000 7f6c4a59c700 1 --2- 172.21.15.7:0/3888444349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574d5c538f0 0x5574d5c53cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.306+0000 7f6c4a59c700 1 -- 172.21.15.7:0/3888444349 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574d5b3df50 con 0x5574d5c54210 2024-06-10T01:28:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.306+0000 7f6c4a59c700 1 --2- 172.21.15.7:0/3888444349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574d5c54210 0x5574d5c4db40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009e20 tx=0x7f6c3c00e5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74b4af0fb1d18a95 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.307+0000 7f6c38ff9700 1 -- 172.21.15.7:0/3888444349 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c016070 con 0x5574d5c54210 2024-06-10T01:28:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.307+0000 7f6c38ff9700 1 -- 172.21.15.7:0/3888444349 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c3c004d10 con 0x5574d5c54210 2024-06-10T01:28:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.308+0000 7f6c5b1a4b80 1 -- 172.21.15.7:0/3888444349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574d5c54210 msgr2=0x5574d5c4db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.308+0000 7f6c5b1a4b80 1 --2- 172.21.15.7:0/3888444349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574d5c54210 0x5574d5c4db40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009e20 tx=0x7f6c3c00e5d0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.309+0000 7f6c4ad9d700 1 --2- 172.21.15.7:0/3888444349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574d5c538f0 0x5574d5c53cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:28:54.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.311+0000 7f6c5b1a4b80 1 -- 172.21.15.7:0/3888444349 shutdown_connections 2024-06-10T01:28:54.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.311+0000 7f6c5b1a4b80 1 --2- 172.21.15.7:0/3888444349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574d5c4e080 0x5574d5c4e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.311+0000 7f6c5b1a4b80 1 --2- 172.21.15.7:0/3888444349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574d5c538f0 0x5574d5c53cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.311+0000 7f6c5b1a4b80 1 --2- 172.21.15.7:0/3888444349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574d5c54210 0x5574d5c4db40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.311+0000 7f6c5b1a4b80 1 -- 172.21.15.7:0/3888444349 >> 172.21.15.7:0/3888444349 conn(0x5574d5b78cc0 msgr2=0x5574d5c48af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:54.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.315+0000 7f6c4ad9d700 1 --2- 172.21.15.7:0/236581300 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6c340a2e20 0x7f6c340a52e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000bfd0 tx=0x7f6c4000b8a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:54.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.315+0000 7f6c5b1a4b80 1 -- 172.21.15.7:0/3888444349 shutdown_connections 2024-06-10T01:28:54.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.316+0000 7f6c5b1a4b80 1 -- 172.21.15.7:0/3888444349 wait complete. 2024-06-10T01:28:54.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.317+0000 7f6c5b1a4b80 1 -- start start 2024-06-10T01:28:54.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.317+0000 7f6c5b1a4b80 1 Processor -- start 2024-06-10T01:28:54.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.319+0000 7f6c5b1a4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574d5c4e080 0x5574d5c59d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:54.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.319+0000 7f6c5b1a4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574d5c538f0 0x5574d5c57060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:54.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.319+0000 7f6c5b1a4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574d5c54210 0x5574d5c575a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:54.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.319+0000 7f6c5b1a4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5574d5c5ac70 con 0x5574d5c538f0 2024-06-10T01:28:54.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.319+0000 7f6c5b1a4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5574d5b3fbf0 con 0x5574d5c54210 2024-06-10T01:28:54.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.319+0000 7f6c5b1a4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5574d5c5adf0 con 0x5574d5c4e080 2024-06-10T01:28:54.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.319+0000 7f6c5b1a4b80 10 client.0 _ll_register_callbacks cb 0x5574d5b39920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:28:54.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.321+0000 7f6c4ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574d5c54210 0x5574d5c575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:54.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.321+0000 7f6c4ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574d5c54210 0x5574d5c575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:57490/0 (socket says 172.21.15.7:57490) 2024-06-10T01:28:54.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.321+0000 7f6c4ad9d700 1 -- 172.21.15.7:0/2402515921 learned_addr learned my addr 172.21.15.7:0/2402515921 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:54.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.321+0000 7f6c4a59c700 1 --2- 172.21.15.7:0/2402515921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574d5c4e080 0x5574d5c59d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:54.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.322+0000 7f6c49d9b700 1 --2- 172.21.15.7:0/2402515921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574d5c538f0 0x5574d5c57060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:54.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.324+0000 7f6c49d9b700 1 -- 172.21.15.7:0/2402515921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574d5c4e080 msgr2=0x5574d5c59d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:54.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.324+0000 7f6c49d9b700 1 --2- 172.21.15.7:0/2402515921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574d5c4e080 0x5574d5c59d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.324+0000 7f6c49d9b700 1 -- 172.21.15.7:0/2402515921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574d5c54210 msgr2=0x5574d5c575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:54.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.324+0000 7f6c49d9b700 1 --2- 172.21.15.7:0/2402515921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574d5c54210 0x5574d5c575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.324+0000 7f6c49d9b700 1 -- 172.21.15.7:0/2402515921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574d5aee8d0 con 0x5574d5c538f0 2024-06-10T01:28:54.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.331+0000 7f6c49d9b700 1 --2- 172.21.15.7:0/2402515921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574d5c538f0 0x5574d5c57060 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f6c440ced50 tx=0x7f6c4409d550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cbdb97004bda229 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:54.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.332+0000 7f6c31ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:28:54.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.336+0000 7f6c31ffb700 1 -- 172.21.15.7:0/2402515921 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4409dbf0 con 0x5574d5c538f0 2024-06-10T01:28:54.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.336+0000 7f6c5b1a4b80 1 -- 172.21.15.7:0/2402515921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c3c004d10 con 0x5574d5c538f0 2024-06-10T01:28:54.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.336+0000 7f6c31ffb700 1 -- 172.21.15.7:0/2402515921 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f6c4400f040 con 0x5574d5c538f0 2024-06-10T01:28:54.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.339+0000 7f6c31ffb700 1 -- 172.21.15.7:0/2402515921 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f6c440bb030 con 0x5574d5c538f0 2024-06-10T01:28:54.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.339+0000 7f6c5b1a4b80 10 client.20862 fetch_fsmap learned FSMap version 55 2024-06-10T01:28:54.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.339+0000 7f6c5b1a4b80 1 -- 172.21.15.7:0/2402515921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5574d5c654d0 con 0x5574d5c538f0 2024-06-10T01:28:54.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.339+0000 7f6c31ffb700 1 -- 172.21.15.7:0/2402515921 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c3c004d10 con 0x5574d5c538f0 2024-06-10T01:28:54.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.341+0000 7f6c31ffb700 1 -- 172.21.15.7:0/2402515921 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f6c44011070 con 0x5574d5c538f0 2024-06-10T01:28:54.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.341+0000 7f6c5b1a4b80 10 client.20862 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:28:54.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.341+0000 7f6c5b1a4b80 10 client.20862 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:28:54.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.341+0000 7f6c5b1a4b80 20 client.20862 populate_metadata read hostname 'smithi007' 2024-06-10T01:28:54.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.341+0000 7f6c5b1a4b80 1 --2- 172.21.15.7:0/2402515921 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5574d5c67dd0 0x5574d5c6a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:54.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.341+0000 7f6c5b1a4b80 4 client.20862 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:28:54.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.341+0000 7f6c5b1a4b80 1 -- 172.21.15.7:0/2402515921 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574d5bacd10 con 0x5574d5c67dd0 2024-06-10T01:28:54.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.343+0000 7f6c4a59c700 1 --2- 172.21.15.7:0/2402515921 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5574d5c67dd0 0x5574d5c6a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:54.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.347+0000 7f6c31ffb700 10 client.20862 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:54.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.347+0000 7f6c4a59c700 1 --2- 172.21.15.7:0/2402515921 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5574d5c67dd0 0x5574d5c6a1f0 crc :-1 s=READY pgs=513 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=443b4db011e6d176 server_cookie=875cf997349ac10a in_seq=0 out_seq=0 2024-06-10T01:28:54.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.352+0000 7f6c31ffb700 1 -- 172.21.15.7:0/2402515921 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5574d5bacd10 con 0x5574d5c67dd0 2024-06-10T01:28:54.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.352+0000 7f6c31ffb700 10 client.20862 handle_command_reply: tid=0 2024-06-10T01:28:54.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.352+0000 7f6c5b1a4b80 1 client.20862 shutdown 2024-06-10T01:28:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.357+0000 7f6c5b1a4b80 1 -- 172.21.15.7:0/2402515921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574d5c538f0 msgr2=0x5574d5c57060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.357+0000 7f6c5b1a4b80 1 --2- 172.21.15.7:0/2402515921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574d5c538f0 0x5574d5c57060 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f6c440ced50 tx=0x7f6c4409d550 comp rx=0 tx=0).stop 2024-06-10T01:28:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.357+0000 7f6c5b1a4b80 1 -- 172.21.15.7:0/2402515921 shutdown_connections 2024-06-10T01:28:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.357+0000 7f6c5b1a4b80 1 --2- 172.21.15.7:0/2402515921 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5574d5c67dd0 0x5574d5c6a1f0 crc :-1 s=READY pgs=513 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.357+0000 7f6c5b1a4b80 1 --2- 172.21.15.7:0/2402515921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574d5c54210 0x5574d5c575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.357+0000 7f6c5b1a4b80 1 --2- 172.21.15.7:0/2402515921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574d5c4e080 0x5574d5c59d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.357+0000 7f6c5b1a4b80 1 --2- 172.21.15.7:0/2402515921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574d5c538f0 0x5574d5c57060 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.357+0000 7f6c31ffb700 0 client.20862 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.357+0000 7f6c5b1a4b80 1 -- 172.21.15.7:0/2402515921 >> 172.21.15.7:0/2402515921 conn(0x5574d5b78cc0 msgr2=0x5574d5c4cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:54.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.358+0000 7f6c5b1a4b80 1 -- 172.21.15.7:0/2402515921 shutdown_connections 2024-06-10T01:28:54.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.358+0000 7f6c5b1a4b80 1 -- 172.21.15.7:0/2402515921 wait complete. 2024-06-10T01:28:54.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.359+0000 7f6c5b1a4b80 20 client.20862 trim_cache size 0 max 16384 2024-06-10T01:28:54.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.402+0000 7f6c5b1a4b80 1 Processor -- start 2024-06-10T01:28:54.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.402+0000 7f6c5b1a4b80 1 -- start start 2024-06-10T01:28:54.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.403+0000 7f6c5b1a4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574d5c4e080 0x5574d5c4db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:54.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.403+0000 7f6c5b1a4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574d5c538f0 0x5574d5c6a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:54.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.403+0000 7f6c5b1a4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574d5c54210 0x5574d5c54c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:54.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.403+0000 7f6c5b1a4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c0111f0 con 0x5574d5c538f0 2024-06-10T01:28:54.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.403+0000 7f6c5b1a4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c00ea80 con 0x5574d5c4e080 2024-06-10T01:28:54.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.403+0000 7f6c4a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574d5c4e080 0x5574d5c4db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:54.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.403+0000 7f6c4a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574d5c4e080 0x5574d5c4db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:57492/0 (socket says 172.21.15.7:57492) 2024-06-10T01:28:54.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.403+0000 7f6c4a59c700 1 -- 172.21.15.7:0/4267968039 learned_addr learned my addr 172.21.15.7:0/4267968039 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:54.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.404+0000 7f6c5b1a4b80 1 -- 172.21.15.7:0/4267968039 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c00c980 con 0x5574d5c54210 2024-06-10T01:28:54.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.404+0000 7f6c4ad9d700 1 --2- 172.21.15.7:0/4267968039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574d5c54210 0x5574d5c54c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:54.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.405+0000 7f6c4a59c700 1 -- 172.21.15.7:0/4267968039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574d5c54210 msgr2=0x5574d5c54c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:54.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.405+0000 7f6c4a59c700 1 --2- 172.21.15.7:0/4267968039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574d5c54210 0x5574d5c54c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.405+0000 7f6c4a59c700 1 -- 172.21.15.7:0/4267968039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574d5c538f0 msgr2=0x5574d5c6a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:54.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.405+0000 7f6c4a59c700 1 --2- 172.21.15.7:0/4267968039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574d5c538f0 0x5574d5c6a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.405+0000 7f6c4a59c700 1 -- 172.21.15.7:0/4267968039 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574d5c4e820 con 0x5574d5c4e080 2024-06-10T01:28:54.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.406+0000 7f6c4a59c700 1 --2- 172.21.15.7:0/4267968039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574d5c4e080 0x5574d5c4db40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009e20 tx=0x7f6c3c0053e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=679ae773373872c2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:54.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.408+0000 7f6c33fff700 1 -- 172.21.15.7:0/4267968039 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c016040 con 0x5574d5c4e080 2024-06-10T01:28:54.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.409+0000 7f6c33fff700 1 -- 172.21.15.7:0/4267968039 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c3c0094a0 con 0x5574d5c4e080 2024-06-10T01:28:54.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.410+0000 7f6c5b1a4b80 1 -- 172.21.15.7:0/4267968039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574d5c4e080 msgr2=0x5574d5c4db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:54.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.410+0000 7f6c5b1a4b80 1 --2- 172.21.15.7:0/4267968039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574d5c4e080 0x5574d5c4db40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009e20 tx=0x7f6c3c0053e0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.410+0000 7f6c5b1a4b80 1 -- 172.21.15.7:0/4267968039 shutdown_connections 2024-06-10T01:28:54.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.410+0000 7f6c5b1a4b80 1 --2- 172.21.15.7:0/4267968039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574d5c54210 0x5574d5c54c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.410+0000 7f6c5b1a4b80 1 --2- 172.21.15.7:0/4267968039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574d5c4e080 0x5574d5c4db40 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.410+0000 7f6c5b1a4b80 1 --2- 172.21.15.7:0/4267968039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574d5c538f0 0x5574d5c6a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.410+0000 7f6c5b1a4b80 1 -- 172.21.15.7:0/4267968039 >> 172.21.15.7:0/4267968039 conn(0x5574d5b78cc0 msgr2=0x5574d5c41f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:54.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.414+0000 7f6c5b1a4b80 1 -- 172.21.15.7:0/4267968039 shutdown_connections 2024-06-10T01:28:54.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.414+0000 7f6c5b1a4b80 1 -- 172.21.15.7:0/4267968039 wait complete. 2024-06-10T01:28:54.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.415+0000 7f6c5b1a4b80 1 -- start start 2024-06-10T01:28:54.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.415+0000 7f6c5b1a4b80 1 Processor -- start 2024-06-10T01:28:54.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.415+0000 7f6c5b1a4b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574d5c4e080 0x5574d5c60650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:54.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.415+0000 7f6c5b1a4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574d5c538f0 0x5574d5c6f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:54.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.415+0000 7f6c5b1a4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574d5c54210 0x5574d5c6f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:54.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.415+0000 7f6c5b1a4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c00cf40 con 0x5574d5c54210 2024-06-10T01:28:54.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.415+0000 7f6c5b1a4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c00e3b0 con 0x5574d5c538f0 2024-06-10T01:28:54.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.415+0000 7f6c5b1a4b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c00d610 con 0x5574d5c4e080 2024-06-10T01:28:54.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.416+0000 7f6c5b1a4b80 10 client.0 _ll_register_callbacks cb 0x5574d5b39920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:28:54.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.416+0000 7f6c4a59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574d5c4e080 0x5574d5c60650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:54.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.416+0000 7f6c4a59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574d5c4e080 0x5574d5c60650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:51124/0 (socket says 172.21.15.7:51124) 2024-06-10T01:28:54.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.417+0000 7f6c4a59c700 1 -- 172.21.15.7:0/2441718946 learned_addr learned my addr 172.21.15.7:0/2441718946 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:54.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.417+0000 7f6c49d9b700 1 --2- 172.21.15.7:0/2441718946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574d5c538f0 0x5574d5c6f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:54.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.418+0000 7f6c4a59c700 1 -- 172.21.15.7:0/2441718946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574d5c538f0 msgr2=0x5574d5c6f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:54.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.418+0000 7f6c4a59c700 1 --2- 172.21.15.7:0/2441718946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574d5c538f0 0x5574d5c6f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.418+0000 7f6c4a59c700 1 -- 172.21.15.7:0/2441718946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574d5c54210 msgr2=0x5574d5c6f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:28:54.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.419+0000 7f6c4a59c700 1 --2- 172.21.15.7:0/2441718946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574d5c54210 0x5574d5c6f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.419+0000 7f6c4a59c700 1 -- 172.21.15.7:0/2441718946 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574d5c6fc80 con 0x5574d5c4e080 2024-06-10T01:28:54.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.420+0000 7f6c4a59c700 1 --2- 172.21.15.7:0/2441718946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574d5c4e080 0x5574d5c60650 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009d40 tx=0x7f6c3c0092a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=778f51c2e6462e40 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:54.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.420+0000 7f6c31ffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:28:54.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.422+0000 7f6c31ffb700 1 -- 172.21.15.7:0/2441718946 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c016030 con 0x5574d5c4e080 2024-06-10T01:28:54.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.422+0000 7f6c5b1a4b80 1 -- 172.21.15.7:0/2441718946 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5574d5ba7290 con 0x5574d5c4e080 2024-06-10T01:28:54.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.422+0000 7f6c31ffb700 1 -- 172.21.15.7:0/2441718946 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f6c3c005140 con 0x5574d5c4e080 2024-06-10T01:28:54.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.422+0000 7f6c31ffb700 1 -- 172.21.15.7:0/2441718946 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f6c3c03f050 con 0x5574d5c4e080 2024-06-10T01:28:54.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.423+0000 7f6c5b1a4b80 10 client.29956 fetch_fsmap learned FSMap version 55 2024-06-10T01:28:54.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.423+0000 7f6c5b1a4b80 1 -- 172.21.15.7:0/2441718946 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5574d5ce3840 con 0x5574d5c4e080 2024-06-10T01:28:54.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.427+0000 7f6c31ffb700 1 -- 172.21.15.7:0/2441718946 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5574d5ba7290 con 0x5574d5c4e080 2024-06-10T01:28:54.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.427+0000 7f6c31ffb700 1 -- 172.21.15.7:0/2441718946 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f6c3c00f040 con 0x5574d5c4e080 2024-06-10T01:28:54.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.427+0000 7f6c5b1a4b80 10 client.29956 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:28:54.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.427+0000 7f6c5b1a4b80 10 client.29956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:28:54.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.427+0000 7f6c5b1a4b80 20 client.29956 populate_metadata read hostname 'smithi007' 2024-06-10T01:28:54.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.427+0000 7f6c5b1a4b80 1 --2- 172.21.15.7:0/2441718946 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5574d5ce3de0 0x5574d5ce6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:54.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.427+0000 7f6c5b1a4b80 4 client.29956 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:28:54.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.427+0000 7f6c5b1a4b80 1 -- 172.21.15.7:0/2441718946 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5574d5a21250 con 0x5574d5ce3de0 2024-06-10T01:28:54.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.428+0000 7f6c49d9b700 1 --2- 172.21.15.7:0/2441718946 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5574d5ce3de0 0x5574d5ce6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:54.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.428+0000 7f6c31ffb700 10 client.29956 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:54.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.428+0000 7f6c49d9b700 1 --2- 172.21.15.7:0/2441718946 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5574d5ce3de0 0x5574d5ce6200 crc :-1 s=READY pgs=514 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=786ef53817904df7 server_cookie=b93cdadcaf992615 in_seq=0 out_seq=0 2024-06-10T01:28:54.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.435+0000 7f6c31ffb700 1 -- 172.21.15.7:0/2441718946 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5574d5a21250 con 0x5574d5ce3de0 2024-06-10T01:28:54.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.435+0000 7f6c31ffb700 10 client.29956 handle_command_reply: tid=0 2024-06-10T01:28:54.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.435+0000 7f6c5b1a4b80 1 client.29956 shutdown 2024-06-10T01:28:54.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.435+0000 7f6c5b1a4b80 1 -- 172.21.15.7:0/2441718946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574d5c4e080 msgr2=0x5574d5c60650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:54.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.435+0000 7f6c5b1a4b80 1 --2- 172.21.15.7:0/2441718946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574d5c4e080 0x5574d5c60650 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009d40 tx=0x7f6c3c0092a0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.435+0000 7f6c5b1a4b80 1 -- 172.21.15.7:0/2441718946 shutdown_connections 2024-06-10T01:28:54.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.435+0000 7f6c5b1a4b80 1 --2- 172.21.15.7:0/2441718946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5574d5c54210 0x5574d5c6f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.436+0000 7f6c5b1a4b80 1 --2- 172.21.15.7:0/2441718946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574d5c4e080 0x5574d5c60650 secure :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009d40 tx=0x7f6c3c0092a0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.436+0000 7f6c5b1a4b80 1 --2- 172.21.15.7:0/2441718946 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5574d5ce3de0 0x5574d5ce6200 crc :-1 s=READY pgs=514 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.436+0000 7f6c5b1a4b80 1 --2- 172.21.15.7:0/2441718946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574d5c538f0 0x5574d5c6f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.436+0000 7f6c5b1a4b80 1 -- 172.21.15.7:0/2441718946 >> 172.21.15.7:0/2441718946 conn(0x5574d5b78cc0 msgr2=0x5574d5c58890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:54.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.436+0000 7f6c31ffb700 0 client.29956 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:54.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.436+0000 7f6c5b1a4b80 1 -- 172.21.15.7:0/2441718946 shutdown_connections 2024-06-10T01:28:54.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.436+0000 7f6c5b1a4b80 1 -- 172.21.15.7:0/2441718946 wait complete. 2024-06-10T01:28:54.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.437+0000 7f6c5b1a4b80 20 client.29956 trim_cache size 0 max 16384 2024-06-10T01:28:54.430 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:28:54.430 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-10T01:28:54.430 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-10T01:28:54.430 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:28:54.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.437+0000 7f6c4b59e700 1 -- 172.21.15.7:0/236581300 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6c340a2e20 msgr2=0x7f6c340a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:54.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.437+0000 7f6c4b59e700 1 --2- 172.21.15.7:0/236581300 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6c340a2e20 0x7f6c340a52e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000bfd0 tx=0x7f6c4000b8a0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.437+0000 7f6c4b59e700 1 -- 172.21.15.7:0/236581300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c4c110ff0 msgr2=0x7f6c4c1adc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:54.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.437+0000 7f6c4b59e700 1 --2- 172.21.15.7:0/236581300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c4c110ff0 0x7f6c4c1adc00 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6c44000f80 tx=0x7f6c44004dc0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.438+0000 7f6c4b59e700 1 -- 172.21.15.7:0/236581300 shutdown_connections 2024-06-10T01:28:54.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.438+0000 7f6c4b59e700 1 --2- 172.21.15.7:0/236581300 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f6c340a2e20 0x7f6c340a52e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.438+0000 7f6c4b59e700 1 --2- 172.21.15.7:0/236581300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c4c11bc70 0x7f6c4c1a7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.438+0000 7f6c4b59e700 1 --2- 172.21.15.7:0/236581300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c4c110b80 0x7f6c4c1ad6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.438+0000 7f6c4b59e700 1 --2- 172.21.15.7:0/236581300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c4c110ff0 0x7f6c4c1adc00 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.438+0000 7f6c4b59e700 1 -- 172.21.15.7:0/236581300 >> 172.21.15.7:0/236581300 conn(0x7f6c4c06d470 msgr2=0x7f6c4c10e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:54.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.438+0000 7f6c4b59e700 1 -- 172.21.15.7:0/236581300 shutdown_connections 2024-06-10T01:28:54.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:54.438+0000 7f6c4b59e700 1 -- 172.21.15.7:0/236581300 wait complete. 2024-06-10T01:28:54.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:54.446+0000 7fd584ff9640 1 -- 172.21.15.7:0/276219818 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55abb1ab0c20 con 0x55abb1bc0470 2024-06-10T01:28:54.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:54.446+0000 7fd584ff9640 10 client.20850 handle_command_reply: tid=0 2024-06-10T01:28:54.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:54.446+0000 7fd5a65a5740 1 client.20850 shutdown 2024-06-10T01:28:54.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:54.446+0000 7fd5a65a5740 1 -- 172.21.15.7:0/276219818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abb1b40a00 msgr2=0x55abb1b4df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:54.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:54.446+0000 7fd5a65a5740 1 --2- 172.21.15.7:0/276219818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abb1b40a00 0x55abb1b4df40 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fd58c02c010 tx=0x7fd58c030740 comp rx=0 tx=0).stop 2024-06-10T01:28:54.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:54.446+0000 7fd5a65a5740 1 -- 172.21.15.7:0/276219818 shutdown_connections 2024-06-10T01:28:54.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:54.446+0000 7fd5a65a5740 1 --2- 172.21.15.7:0/276219818 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55abb1bc0470 0x55abb1bc2860 crc :-1 s=READY pgs=512 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:54.446+0000 7fd5a65a5740 1 --2- 172.21.15.7:0/276219818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abb1b45b70 0x55abb1b4e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:54.446+0000 7fd5a65a5740 1 --2- 172.21.15.7:0/276219818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abb1b40a00 0x55abb1b4df40 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:54.446+0000 7fd5a65a5740 1 --2- 172.21.15.7:0/276219818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abb1b40010 0x55abb1b514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:54.446+0000 7fd5a65a5740 1 -- 172.21.15.7:0/276219818 >> 172.21.15.7:0/276219818 conn(0x55abb1b3b680 msgr2=0x55abb1b4a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:54.446+0000 7fd584ff9640 0 client.20850 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:28:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:54.447+0000 7fd5a65a5740 1 -- 172.21.15.7:0/276219818 shutdown_connections 2024-06-10T01:28:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:54.447+0000 7fd5a65a5740 1 -- 172.21.15.7:0/276219818 wait complete. 2024-06-10T01:28:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:54.447+0000 7fd5a65a5740 20 client.20850 trim_cache size 0 max 16384 2024-06-10T01:28:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:28:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:28:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:28:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:54.448+0000 7fd5a37b7640 1 -- 172.21.15.7:0/702455810 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd5880a2b80 msgr2=0x7fd5880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:54.448+0000 7fd5a37b7640 1 --2- 172.21.15.7:0/702455810 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd5880a2b80 0x7fd5880a5040 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fd598009710 tx=0x7fd598009920 comp rx=0 tx=0).stop 2024-06-10T01:28:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:54.448+0000 7fd5a37b7640 1 -- 172.21.15.7:0/702455810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd59c102480 msgr2=0x7fd59c197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:54.448+0000 7fd5a37b7640 1 --2- 172.21.15.7:0/702455810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd59c102480 0x7fd59c197d40 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fd59400c420 tx=0x7fd594009f90 comp rx=0 tx=0).stop 2024-06-10T01:28:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:54.448+0000 7fd5a37b7640 1 -- 172.21.15.7:0/702455810 shutdown_connections 2024-06-10T01:28:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:54.448+0000 7fd5a37b7640 1 --2- 172.21.15.7:0/702455810 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd5880a2b80 0x7fd5880a5040 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:54.448+0000 7fd5a37b7640 1 --2- 172.21.15.7:0/702455810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd59c1076e0 0x7fd59c19f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:54.448+0000 7fd5a37b7640 1 --2- 172.21.15.7:0/702455810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd59c104db0 0x7fd59c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:54.448+0000 7fd5a37b7640 1 --2- 172.21.15.7:0/702455810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd59c102480 0x7fd59c197d40 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:54.448+0000 7fd5a37b7640 1 -- 172.21.15.7:0/702455810 >> 172.21.15.7:0/702455810 conn(0x7fd59c0fbe70 msgr2=0x7fd59c102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:54.450+0000 7fd5a37b7640 1 -- 172.21.15.7:0/702455810 shutdown_connections 2024-06-10T01:28:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:28:54.450+0000 7fd5a37b7640 1 -- 172.21.15.7:0/702455810 wait complete. 2024-06-10T01:28:54.559 INFO:tasks.workunit.client.0.smithi007.stdout: 10 230184 22.66 MB/sec execute 451 sec latency 599.078 ms 2024-06-10T01:28:55.314 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:28:55.315 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:28:55.559 INFO:tasks.workunit.client.0.smithi007.stdout: 10 230747 22.67 MB/sec execute 452 sec latency 70.052 ms 2024-06-10T01:28:55.732 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:28:55.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:55 smithi078 ceph-mon[36683]: pgmap v1121: 129 pgs: 129 active+clean; 665 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 2.6 MiB/s wr, 351 op/s 2024-06-10T01:28:55.857 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:55 smithi007 ceph-mon[32398]: pgmap v1121: 129 pgs: 129 active+clean; 665 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 2.6 MiB/s wr, 351 op/s 2024-06-10T01:28:55.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:55 smithi175 ceph-mon[36636]: pgmap v1121: 129 pgs: 129 active+clean; 665 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 2.6 MiB/s wr, 351 op/s 2024-06-10T01:28:56.559 INFO:tasks.workunit.client.0.smithi007.stdout: 10 231421 22.70 MB/sec execute 453 sec latency 87.371 ms 2024-06-10T01:28:57.560 INFO:tasks.workunit.client.0.smithi007.stdout: 10 232138 22.74 MB/sec execute 454 sec latency 177.139 ms 2024-06-10T01:28:57.852 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:57 smithi007 ceph-mon[32398]: pgmap v1122: 129 pgs: 129 active+clean; 656 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.3 MiB/s wr, 496 op/s 2024-06-10T01:28:57.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:57 smithi175 ceph-mon[36636]: pgmap v1122: 129 pgs: 129 active+clean; 656 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.3 MiB/s wr, 496 op/s 2024-06-10T01:28:58.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:57 smithi078 ceph-mon[36683]: pgmap v1122: 129 pgs: 129 active+clean; 656 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.3 MiB/s wr, 496 op/s 2024-06-10T01:28:58.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.276+0000 7fe235aa3700 1 -- 172.21.15.7:0/3726215280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe230107400 msgr2=0x7fe2301077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:58.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.276+0000 7fe235aa3700 1 --2- 172.21.15.7:0/3726215280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe230107400 0x7fe2301077e0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fe220009990 tx=0x7fe220009ca0 comp rx=0 tx=0).stop 2024-06-10T01:28:58.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.276+0000 7fe235aa3700 1 -- 172.21.15.7:0/3726215280 shutdown_connections 2024-06-10T01:28:58.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.276+0000 7fe235aa3700 1 --2- 172.21.15.7:0/3726215280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe230101d50 0x7fe23010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:58.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.276+0000 7fe235aa3700 1 --2- 172.21.15.7:0/3726215280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe230107400 0x7fe2301077e0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:58.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.276+0000 7fe235aa3700 1 --2- 172.21.15.7:0/3726215280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe230101390 0x7fe230101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:58.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.276+0000 7fe235aa3700 1 -- 172.21.15.7:0/3726215280 >> 172.21.15.7:0/3726215280 conn(0x7fe230078340 msgr2=0x7fe230078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:58.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.276+0000 7fe235aa3700 1 -- 172.21.15.7:0/3726215280 shutdown_connections 2024-06-10T01:28:58.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.276+0000 7fe235aa3700 1 -- 172.21.15.7:0/3726215280 wait complete. 2024-06-10T01:28:58.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.277+0000 7fe235aa3700 1 Processor -- start 2024-06-10T01:28:58.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.277+0000 7fe235aa3700 1 -- start start 2024-06-10T01:28:58.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.278+0000 7fe235aa3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe230101390 0x7fe23019d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:58.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.278+0000 7fe235aa3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe230101d50 0x7fe23019dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:58.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.278+0000 7fe235aa3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe230107400 0x7fe2301979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:58.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.278+0000 7fe235aa3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe230111670 con 0x7fe230101d50 2024-06-10T01:28:58.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.278+0000 7fe235aa3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe2301114f0 con 0x7fe230101390 2024-06-10T01:28:58.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.278+0000 7fe235aa3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe2301117f0 con 0x7fe230107400 2024-06-10T01:28:58.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.278+0000 7fe22effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe230101d50 0x7fe23019dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:58.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.278+0000 7fe22f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe230101390 0x7fe23019d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:58.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.278+0000 7fe22f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe230101390 0x7fe23019d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:57524/0 (socket says 172.21.15.7:57524) 2024-06-10T01:28:58.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.278+0000 7fe22effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe230101d50 0x7fe23019dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41740/0 (socket says 172.21.15.7:41740) 2024-06-10T01:28:58.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.278+0000 7fe22f7fe700 1 -- 172.21.15.7:0/3508948431 learned_addr learned my addr 172.21.15.7:0/3508948431 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:28:58.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.278+0000 7fe22ffff700 1 --2- 172.21.15.7:0/3508948431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe230107400 0x7fe2301979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:58.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.278+0000 7fe22effd700 1 -- 172.21.15.7:0/3508948431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe230107400 msgr2=0x7fe2301979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:58.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.278+0000 7fe22effd700 1 --2- 172.21.15.7:0/3508948431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe230107400 0x7fe2301979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:58.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.278+0000 7fe22effd700 1 -- 172.21.15.7:0/3508948431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe230101390 msgr2=0x7fe23019d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:58.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.278+0000 7fe22effd700 1 --2- 172.21.15.7:0/3508948431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe230101390 0x7fe23019d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:58.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.278+0000 7fe22effd700 1 -- 172.21.15.7:0/3508948431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe230198150 con 0x7fe230101d50 2024-06-10T01:28:58.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.278+0000 7fe22f7fe700 1 --2- 172.21.15.7:0/3508948431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe230101390 0x7fe23019d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:28:58.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.279+0000 7fe22effd700 1 --2- 172.21.15.7:0/3508948431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe230101d50 0x7fe23019dd70 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fe22400e9f0 tx=0x7fe22400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:58.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.280+0000 7fe22cff9700 1 -- 172.21.15.7:0/3508948431 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22400c9c0 con 0x7fe230101d50 2024-06-10T01:28:58.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.280+0000 7fe22cff9700 1 -- 172.21.15.7:0/3508948431 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe22400cb60 con 0x7fe230101d50 2024-06-10T01:28:58.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.280+0000 7fe22cff9700 1 -- 172.21.15.7:0/3508948431 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe224018680 con 0x7fe230101d50 2024-06-10T01:28:58.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.280+0000 7fe235aa3700 1 -- 172.21.15.7:0/3508948431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe230198440 con 0x7fe230101d50 2024-06-10T01:28:58.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.280+0000 7fe235aa3700 1 -- 172.21.15.7:0/3508948431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe23006a170 con 0x7fe230101d50 2024-06-10T01:28:58.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.284+0000 7fe235aa3700 1 -- 172.21.15.7:0/3508948431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe230102ab0 con 0x7fe230101d50 2024-06-10T01:28:58.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.285+0000 7fe22cff9700 1 -- 172.21.15.7:0/3508948431 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fe224010430 con 0x7fe230101d50 2024-06-10T01:28:58.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.286+0000 7fe22cff9700 1 --2- 172.21.15.7:0/3508948431 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe2180a2d60 0x7fe2180a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:28:58.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.286+0000 7fe22cff9700 1 -- 172.21.15.7:0/3508948431 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fe2240cf550 con 0x7fe230101d50 2024-06-10T01:28:58.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.286+0000 7fe22f7fe700 1 --2- 172.21.15.7:0/3508948431 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe2180a2d60 0x7fe2180a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:28:58.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.287+0000 7fe22f7fe700 1 --2- 172.21.15.7:0/3508948431 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe2180a2d60 0x7fe2180a5220 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe220009750 tx=0x7fe220005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:28:58.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.295+0000 7fe22cff9700 1 -- 172.21.15.7:0/3508948431 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe22409c0d0 con 0x7fe230101d50 2024-06-10T01:28:58.560 INFO:tasks.workunit.client.0.smithi007.stdout: 10 232590 22.75 MB/sec execute 455 sec latency 277.179 ms 2024-06-10T01:28:58.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.687+0000 7fe235aa3700 1 -- 172.21.15.7:0/3508948431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe230198f20 con 0x7fe230101d50 2024-06-10T01:28:58.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.689+0000 7fe22cff9700 1 -- 172.21.15.7:0/3508948431 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11852 (secure 0 0 0) 0x7fe22409b9d0 con 0x7fe230101d50 2024-06-10T01:28:58.681 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:28:58.681 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:27:44.542390+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29813":{"gid":29813,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/138896429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":138896429},{"type":"v1","addr":"172.21.15.175:6835","nonce":138896429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24451,24434,24436]},"id":1}]} 2024-06-10T01:28:58.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.694+0000 7fe235aa3700 1 -- 172.21.15.7:0/3508948431 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe2180a2d60 msgr2=0x7fe2180a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:58.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.694+0000 7fe235aa3700 1 --2- 172.21.15.7:0/3508948431 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe2180a2d60 0x7fe2180a5220 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe220009750 tx=0x7fe220005e80 comp rx=0 tx=0).stop 2024-06-10T01:28:58.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.694+0000 7fe235aa3700 1 -- 172.21.15.7:0/3508948431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe230101d50 msgr2=0x7fe23019dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:28:58.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.694+0000 7fe235aa3700 1 --2- 172.21.15.7:0/3508948431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe230101d50 0x7fe23019dd70 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fe22400e9f0 tx=0x7fe22400ed00 comp rx=0 tx=0).stop 2024-06-10T01:28:58.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.694+0000 7fe235aa3700 1 -- 172.21.15.7:0/3508948431 shutdown_connections 2024-06-10T01:28:58.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.695+0000 7fe235aa3700 1 --2- 172.21.15.7:0/3508948431 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe2180a2d60 0x7fe2180a5220 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:58.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.695+0000 7fe235aa3700 1 --2- 172.21.15.7:0/3508948431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe230107400 0x7fe2301979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:58.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.695+0000 7fe235aa3700 1 --2- 172.21.15.7:0/3508948431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe230101390 0x7fe23019d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:58.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.695+0000 7fe235aa3700 1 --2- 172.21.15.7:0/3508948431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe230101d50 0x7fe23019dd70 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:28:58.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.695+0000 7fe235aa3700 1 -- 172.21.15.7:0/3508948431 >> 172.21.15.7:0/3508948431 conn(0x7fe230078340 msgr2=0x7fe2301000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:28:58.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.695+0000 7fe235aa3700 1 -- 172.21.15.7:0/3508948431 shutdown_connections 2024-06-10T01:28:58.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:28:58.695+0000 7fe235aa3700 1 -- 172.21.15.7:0/3508948431 wait complete. 2024-06-10T01:28:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 55 2024-06-10T01:28:59.560 INFO:tasks.workunit.client.0.smithi007.stdout: 10 233022 22.75 MB/sec execute 456 sec latency 140.976 ms 2024-06-10T01:28:59.668 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:28:59.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:59 smithi007 ceph-mon[32398]: pgmap v1123: 129 pgs: 129 active+clean; 656 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.6 MiB/s wr, 586 op/s 2024-06-10T01:28:59.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:28:59 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3508948431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:28:59.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:59 smithi175 ceph-mon[36636]: pgmap v1123: 129 pgs: 129 active+clean; 656 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.6 MiB/s wr, 586 op/s 2024-06-10T01:28:59.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:28:59 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3508948431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:29:00.039 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:29:00.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:59 smithi078 ceph-mon[36683]: pgmap v1123: 129 pgs: 129 active+clean; 656 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.6 MiB/s wr, 586 op/s 2024-06-10T01:29:00.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:28:59 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3508948431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:29:00.560 INFO:tasks.workunit.client.0.smithi007.stdout: 10 233663 22.77 MB/sec execute 457 sec latency 92.968 ms 2024-06-10T01:29:01.560 INFO:tasks.workunit.client.0.smithi007.stdout: 10 234247 22.77 MB/sec execute 458 sec latency 126.872 ms 2024-06-10T01:29:01.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:01 smithi007 ceph-mon[32398]: pgmap v1124: 129 pgs: 129 active+clean; 660 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 3.6 MiB/s wr, 592 op/s 2024-06-10T01:29:01.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:01 smithi175 ceph-mon[36636]: pgmap v1124: 129 pgs: 129 active+clean; 660 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 3.6 MiB/s wr, 592 op/s 2024-06-10T01:29:01.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.983+0000 7f7a5610b700 1 -- 172.21.15.7:0/4087446375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a50102530 msgr2=0x7f7a501029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:01.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.983+0000 7f7a5610b700 1 --2- 172.21.15.7:0/4087446375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a50102530 0x7f7a501029b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7a44009990 tx=0x7f7a44009ca0 comp rx=0 tx=0).stop 2024-06-10T01:29:01.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.984+0000 7f7a5610b700 1 -- 172.21.15.7:0/4087446375 shutdown_connections 2024-06-10T01:29:01.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.984+0000 7f7a5610b700 1 --2- 172.21.15.7:0/4087446375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a50102ef0 0x7f7a5010f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:01.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.984+0000 7f7a5610b700 1 --2- 172.21.15.7:0/4087446375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a501085d0 0x7f7a501089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:01.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.984+0000 7f7a5610b700 1 --2- 172.21.15.7:0/4087446375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a50102530 0x7f7a501029b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:01.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.984+0000 7f7a5610b700 1 -- 172.21.15.7:0/4087446375 >> 172.21.15.7:0/4087446375 conn(0x7f7a50079640 msgr2=0x7f7a50079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:01.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.985+0000 7f7a5610b700 1 -- 172.21.15.7:0/4087446375 shutdown_connections 2024-06-10T01:29:01.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.985+0000 7f7a5610b700 1 -- 172.21.15.7:0/4087446375 wait complete. 2024-06-10T01:29:01.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.986+0000 7f7a5610b700 1 Processor -- start 2024-06-10T01:29:01.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.986+0000 7f7a5610b700 1 -- start start 2024-06-10T01:29:01.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.987+0000 7f7a5610b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a50102530 0x7f7a501ad780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:01.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.987+0000 7f7a5610b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a50102ef0 0x7f7a501adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:01.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.987+0000 7f7a5610b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a501085d0 0x7f7a501a7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:01.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.987+0000 7f7a5610b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7a50110710 con 0x7f7a50102530 2024-06-10T01:29:01.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.987+0000 7f7a5610b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7a50110590 con 0x7f7a501085d0 2024-06-10T01:29:01.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.987+0000 7f7a5610b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7a50110890 con 0x7f7a50102ef0 2024-06-10T01:29:01.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.987+0000 7f7a4effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a50102ef0 0x7f7a501adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:01.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.987+0000 7f7a4effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a50102ef0 0x7f7a501adcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36294/0 (socket says 172.21.15.7:36294) 2024-06-10T01:29:01.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.987+0000 7f7a4effd700 1 -- 172.21.15.7:0/3919452012 learned_addr learned my addr 172.21.15.7:0/3919452012 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:01.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.987+0000 7f7a4ffff700 1 --2- 172.21.15.7:0/3919452012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a501085d0 0x7f7a501a7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:01.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.988+0000 7f7a4effd700 1 -- 172.21.15.7:0/3919452012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a501085d0 msgr2=0x7f7a501a7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:01.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.988+0000 7f7a4effd700 1 --2- 172.21.15.7:0/3919452012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a501085d0 0x7f7a501a7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:01.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.988+0000 7f7a4effd700 1 -- 172.21.15.7:0/3919452012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a50102530 msgr2=0x7f7a501ad780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:01.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.988+0000 7f7a4effd700 1 --2- 172.21.15.7:0/3919452012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a50102530 0x7f7a501ad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:01.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.988+0000 7f7a4effd700 1 -- 172.21.15.7:0/3919452012 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a501a80a0 con 0x7f7a50102ef0 2024-06-10T01:29:01.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.988+0000 7f7a4effd700 1 --2- 172.21.15.7:0/3919452012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a50102ef0 0x7f7a501adcc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7a44005f50 tx=0x7f7a440057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:01.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.989+0000 7f7a4cff9700 1 -- 172.21.15.7:0/3919452012 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a44026070 con 0x7f7a50102ef0 2024-06-10T01:29:01.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.989+0000 7f7a4cff9700 1 -- 172.21.15.7:0/3919452012 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a44021410 con 0x7f7a50102ef0 2024-06-10T01:29:01.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.989+0000 7f7a4cff9700 1 -- 172.21.15.7:0/3919452012 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a44017650 con 0x7f7a50102ef0 2024-06-10T01:29:01.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.989+0000 7f7a5610b700 1 -- 172.21.15.7:0/3919452012 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a501a8330 con 0x7f7a50102ef0 2024-06-10T01:29:01.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.989+0000 7f7a5610b700 1 -- 172.21.15.7:0/3919452012 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a501b4650 con 0x7f7a50102ef0 2024-06-10T01:29:01.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.997+0000 7f7a6098eb80 1 Processor -- start 2024-06-10T01:29:01.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.997+0000 7f7a6098eb80 1 -- start start 2024-06-10T01:29:01.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.997+0000 7f7a6098eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db66ae5f60 0x55db66ae6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:01.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.997+0000 7f7a6098eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db66ae6880 0x55db66ae0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:01.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.998+0000 7f7a4cff9700 1 -- 172.21.15.7:0/3919452012 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f7a44029070 con 0x7f7a50102ef0 2024-06-10T01:29:01.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.998+0000 7f7a4ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db66ae5f60 0x55db66ae6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:01.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.998+0000 7f7a4ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db66ae5f60 0x55db66ae6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36296/0 (socket says 172.21.15.7:36296) 2024-06-10T01:29:01.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.998+0000 7f7a4ffff700 1 -- 172.21.15.7:0/2237077481 learned_addr learned my addr 172.21.15.7:0/2237077481 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:01.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.998+0000 7f7a4f7fe700 1 --2- 172.21.15.7:0/2237077481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db66ae6880 0x55db66ae0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:01.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.999+0000 7f7a6098eb80 1 --2- 172.21.15.7:0/2237077481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db66ae07a0 0x55db66ae0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:01.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.999+0000 7f7a6098eb80 1 -- 172.21.15.7:0/2237077481 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55db669d2f60 con 0x55db66ae6880 2024-06-10T01:29:01.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.999+0000 7f7a6098eb80 1 -- 172.21.15.7:0/2237077481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55db66a31850 con 0x55db66ae07a0 2024-06-10T01:29:01.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.999+0000 7f7a6098eb80 1 -- 172.21.15.7:0/2237077481 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55db66999460 con 0x55db66ae5f60 2024-06-10T01:29:01.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.999+0000 7f7a4cff9700 1 --2- 172.21.15.7:0/3919452012 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7a380a2d70 0x7f7a380a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:01.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.999+0000 7f7a4cff9700 1 -- 172.21.15.7:0/3919452012 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f7a440cfff0 con 0x7f7a50102ef0 2024-06-10T01:29:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.999+0000 7f7a4ffff700 1 -- 172.21.15.7:0/2237077481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db66ae07a0 msgr2=0x55db66ae0bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T01:29:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.999+0000 7f7a4ffff700 1 --2- 172.21.15.7:0/2237077481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db66ae07a0 0x55db66ae0bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.999+0000 7f7a4ffff700 1 -- 172.21.15.7:0/2237077481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db66ae6880 msgr2=0x55db66ae0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.999+0000 7f7a4ffff700 1 --2- 172.21.15.7:0/2237077481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db66ae6880 0x55db66ae0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.999+0000 7f7a4ffff700 1 -- 172.21.15.7:0/2237077481 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db669808d0 con 0x55db66ae5f60 2024-06-10T01:29:01.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:01.999+0000 7f7a4ffff700 1 --2- 172.21.15.7:0/3919452012 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7a380a2d70 0x7f7a380a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:01.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.000+0000 7f7a4ffff700 1 --2- 172.21.15.7:0/2237077481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db66ae5f60 0x55db66ae6340 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7a48003b00 tx=0x7f7a48003e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91b3bb9f16f8f9f5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:01.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.000+0000 7f7a4f7fe700 1 --2- 172.21.15.7:0/2237077481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db66ae6880 0x55db66ae0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:29:01.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.000+0000 7f7a35ffb700 1 -- 172.21.15.7:0/2237077481 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a48007d20 con 0x55db66ae5f60 2024-06-10T01:29:01.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.001+0000 7f7a4ffff700 1 --2- 172.21.15.7:0/3919452012 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7a380a2d70 0x7f7a380a5230 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7a48021000 tx=0x7f7a4801b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:01.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.001+0000 7f7a35ffb700 1 -- 172.21.15.7:0/2237077481 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a48022070 con 0x55db66ae5f60 2024-06-10T01:29:01.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.002+0000 7f7a6098eb80 1 -- 172.21.15.7:0/2237077481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db66ae5f60 msgr2=0x55db66ae6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:01.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.002+0000 7f7a6098eb80 1 --2- 172.21.15.7:0/2237077481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db66ae5f60 0x55db66ae6340 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7a48003b00 tx=0x7f7a48003e10 comp rx=0 tx=0).stop 2024-06-10T01:29:01.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.003+0000 7f7a6098eb80 1 -- 172.21.15.7:0/2237077481 shutdown_connections 2024-06-10T01:29:01.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.003+0000 7f7a6098eb80 1 --2- 172.21.15.7:0/2237077481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db66ae07a0 0x55db66ae0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:01.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.003+0000 7f7a6098eb80 1 --2- 172.21.15.7:0/2237077481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db66ae5f60 0x55db66ae6340 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:01.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.003+0000 7f7a6098eb80 1 --2- 172.21.15.7:0/2237077481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db66ae6880 0x55db66ae0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:01.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.003+0000 7f7a6098eb80 1 -- 172.21.15.7:0/2237077481 >> 172.21.15.7:0/2237077481 conn(0x55db66a0d080 msgr2=0x55db66adb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:01.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.004+0000 7f7a6098eb80 1 -- 172.21.15.7:0/2237077481 shutdown_connections 2024-06-10T01:29:01.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.004+0000 7f7a6098eb80 1 -- 172.21.15.7:0/2237077481 wait complete. 2024-06-10T01:29:01.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.004+0000 7f7a6098eb80 1 -- start start 2024-06-10T01:29:01.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.004+0000 7f7a6098eb80 1 Processor -- start 2024-06-10T01:29:01.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.005+0000 7f7a6098eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db66ae07a0 0x55db66aec580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:01.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.005+0000 7f7a6098eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db66ae5f60 0x55db66ae9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:01.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.005+0000 7f7a6098eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db66ae6880 0x55db66ae9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:02.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.005+0000 7f7a6098eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55db669d1fc0 con 0x55db66ae07a0 2024-06-10T01:29:02.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.005+0000 7f7a6098eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55db66a2d810 con 0x55db66ae6880 2024-06-10T01:29:02.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.005+0000 7f7a6098eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55db66aed2b0 con 0x55db66ae5f60 2024-06-10T01:29:02.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.005+0000 7f7a6098eb80 10 client.0 _ll_register_callbacks cb 0x55db669c75b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:29:02.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.005+0000 7f7a4ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db66ae6880 0x55db66ae9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:02.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.005+0000 7f7a4f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db66ae07a0 0x55db66aec580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:02.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.005+0000 7f7a4ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db66ae6880 0x55db66ae9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:54254/0 (socket says 172.21.15.7:54254) 2024-06-10T01:29:02.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.005+0000 7f7a4ffff700 1 -- 172.21.15.7:0/3305927288 learned_addr learned my addr 172.21.15.7:0/3305927288 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:02.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.005+0000 7f7a4effd700 1 --2- 172.21.15.7:0/3305927288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db66ae5f60 0x55db66ae9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:02.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.005+0000 7f7a4ffff700 1 -- 172.21.15.7:0/3305927288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db66ae5f60 msgr2=0x55db66ae9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:02.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.005+0000 7f7a4ffff700 1 --2- 172.21.15.7:0/3305927288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db66ae5f60 0x55db66ae9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:02.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.005+0000 7f7a4ffff700 1 -- 172.21.15.7:0/3305927288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db66ae07a0 msgr2=0x55db66aec580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:02.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.005+0000 7f7a4ffff700 1 --2- 172.21.15.7:0/3305927288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db66ae07a0 0x55db66aec580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:02.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.006+0000 7f7a4ffff700 1 -- 172.21.15.7:0/3305927288 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db669f7750 con 0x55db66ae6880 2024-06-10T01:29:02.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.006+0000 7f7a4ffff700 1 --2- 172.21.15.7:0/3305927288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db66ae6880 0x55db66ae9dc0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009ce0 tx=0x7f7a4801d850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b432dab51ca09d59 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:02.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.006+0000 7f7a2effd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:29:02.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.006+0000 7f7a2effd700 1 -- 172.21.15.7:0/3305927288 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a48004a90 con 0x55db66ae6880 2024-06-10T01:29:02.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.006+0000 7f7a6098eb80 1 -- 172.21.15.7:0/3305927288 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a48022070 con 0x55db66ae6880 2024-06-10T01:29:02.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.006+0000 7f7a2effd700 1 -- 172.21.15.7:0/3305927288 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f7a48004c30 con 0x55db66ae6880 2024-06-10T01:29:02.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.006+0000 7f7a2effd700 1 -- 172.21.15.7:0/3305927288 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f7a48004e30 con 0x55db66ae6880 2024-06-10T01:29:02.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.006+0000 7f7a6098eb80 10 client.30059 fetch_fsmap learned FSMap version 55 2024-06-10T01:29:02.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.006+0000 7f7a6098eb80 1 -- 172.21.15.7:0/3305927288 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55db66af7980 con 0x55db66ae6880 2024-06-10T01:29:02.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.007+0000 7f7a2effd700 1 -- 172.21.15.7:0/3305927288 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a48022070 con 0x55db66ae6880 2024-06-10T01:29:02.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.007+0000 7f7a2effd700 1 -- 172.21.15.7:0/3305927288 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f7a4802d070 con 0x55db66ae6880 2024-06-10T01:29:02.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.007+0000 7f7a6098eb80 10 client.30059 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:29:02.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.007+0000 7f7a6098eb80 10 client.30059 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:29:02.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.007+0000 7f7a6098eb80 20 client.30059 populate_metadata read hostname 'smithi007' 2024-06-10T01:29:02.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.007+0000 7f7a6098eb80 1 --2- 172.21.15.7:0/3305927288 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55db66afa0d0 0x55db66afc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:02.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.007+0000 7f7a6098eb80 4 client.30059 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:29:02.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.007+0000 7f7a6098eb80 1 -- 172.21.15.7:0/3305927288 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db66a3ed50 con 0x55db66afa0d0 2024-06-10T01:29:02.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.007+0000 7f7a4f7fe700 1 --2- 172.21.15.7:0/3305927288 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55db66afa0d0 0x55db66afc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:02.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.008+0000 7f7a4f7fe700 1 --2- 172.21.15.7:0/3305927288 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55db66afa0d0 0x55db66afc510 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cf54add2c3709d27 server_cookie=14bae18b9f6febbd in_seq=0 out_seq=0 2024-06-10T01:29:02.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.008+0000 7f7a2effd700 10 client.30059 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:29:02.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.010+0000 7f7a2effd700 1 -- 172.21.15.7:0/3305927288 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55db66a3ed50 con 0x55db66afa0d0 2024-06-10T01:29:02.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.010+0000 7f7a2effd700 10 client.30059 handle_command_reply: tid=0 2024-06-10T01:29:02.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.010+0000 7f7a6098eb80 1 client.30059 shutdown 2024-06-10T01:29:02.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.011+0000 7f7a6098eb80 1 -- 172.21.15.7:0/3305927288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db66ae6880 msgr2=0x55db66ae9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:02.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.011+0000 7f7a6098eb80 1 --2- 172.21.15.7:0/3305927288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db66ae6880 0x55db66ae9dc0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009ce0 tx=0x7f7a4801d850 comp rx=0 tx=0).stop 2024-06-10T01:29:02.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.011+0000 7f7a6098eb80 1 -- 172.21.15.7:0/3305927288 shutdown_connections 2024-06-10T01:29:02.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.011+0000 7f7a6098eb80 1 --2- 172.21.15.7:0/3305927288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db66ae6880 0x55db66ae9dc0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:02.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.011+0000 7f7a6098eb80 1 --2- 172.21.15.7:0/3305927288 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55db66afa0d0 0x55db66afc510 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:02.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.011+0000 7f7a2effd700 0 client.30059 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:29:02.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.011+0000 7f7a6098eb80 1 --2- 172.21.15.7:0/3305927288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db66ae07a0 0x55db66aec580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:02.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.011+0000 7f7a6098eb80 1 --2- 172.21.15.7:0/3305927288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db66ae5f60 0x55db66ae9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:02.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.011+0000 7f7a6098eb80 1 -- 172.21.15.7:0/3305927288 >> 172.21.15.7:0/3305927288 conn(0x55db66a0d080 msgr2=0x55db66ada700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:02.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.011+0000 7f7a6098eb80 1 -- 172.21.15.7:0/3305927288 shutdown_connections 2024-06-10T01:29:02.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.011+0000 7f7a6098eb80 1 -- 172.21.15.7:0/3305927288 wait complete. 2024-06-10T01:29:02.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.011+0000 7f7a6098eb80 20 client.30059 trim_cache size 0 max 16384 2024-06-10T01:29:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.033+0000 7f7a6098eb80 1 Processor -- start 2024-06-10T01:29:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.033+0000 7f7a6098eb80 1 -- start start 2024-06-10T01:29:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.034+0000 7f7a6098eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db66ae07a0 0x55db66ae9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.034+0000 7f7a6098eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db66ae5f60 0x55db66adae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.034+0000 7f7a6098eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db66ae6880 0x55db66adb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.034+0000 7f7a6098eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7a501bf8c0 con 0x55db66ae6880 2024-06-10T01:29:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.034+0000 7f7a6098eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7a501bd150 con 0x55db66ae5f60 2024-06-10T01:29:02.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.034+0000 7f7a6098eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7a501bafa0 con 0x55db66ae07a0 2024-06-10T01:29:02.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.034+0000 7f7a4ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db66ae6880 0x55db66adb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:02.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.034+0000 7f7a4effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db66ae5f60 0x55db66adae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:02.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.034+0000 7f7a4ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db66ae6880 0x55db66adb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54308/0 (socket says 172.21.15.7:54308) 2024-06-10T01:29:02.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.034+0000 7f7a4ffff700 1 -- 172.21.15.7:0/2868558070 learned_addr learned my addr 172.21.15.7:0/2868558070 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:02.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.034+0000 7f7a4f7fe700 1 --2- 172.21.15.7:0/2868558070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db66ae07a0 0x55db66ae9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:02.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.034+0000 7f7a4ffff700 1 -- 172.21.15.7:0/2868558070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db66ae07a0 msgr2=0x55db66ae9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:02.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.035+0000 7f7a4ffff700 1 --2- 172.21.15.7:0/2868558070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db66ae07a0 0x55db66ae9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:02.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.035+0000 7f7a4ffff700 1 -- 172.21.15.7:0/2868558070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db66ae5f60 msgr2=0x55db66adae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:02.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.035+0000 7f7a4ffff700 1 --2- 172.21.15.7:0/2868558070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db66ae5f60 0x55db66adae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:02.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.035+0000 7f7a4ffff700 1 -- 172.21.15.7:0/2868558070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db66af9ea0 con 0x55db66ae6880 2024-06-10T01:29:02.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.035+0000 7f7a4ffff700 1 --2- 172.21.15.7:0/2868558070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db66ae6880 0x55db66adb3c0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009cb0 tx=0x7f7a48005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf7208b0b18f4b4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:02.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.035+0000 7f7a357fa700 1 -- 172.21.15.7:0/2868558070 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a48009090 con 0x55db66ae6880 2024-06-10T01:29:02.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.035+0000 7f7a357fa700 1 -- 172.21.15.7:0/2868558070 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a48022030 con 0x55db66ae6880 2024-06-10T01:29:02.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.035+0000 7f7a357fa700 1 -- 172.21.15.7:0/2868558070 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a480052d0 con 0x55db66ae6880 2024-06-10T01:29:02.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.036+0000 7f7a4effd700 1 --2- 172.21.15.7:0/2868558070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db66ae5f60 0x55db66adae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:29:02.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.036+0000 7f7a6098eb80 1 -- 172.21.15.7:0/2868558070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db66ae6880 msgr2=0x55db66adb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:02.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.036+0000 7f7a6098eb80 1 --2- 172.21.15.7:0/2868558070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db66ae6880 0x55db66adb3c0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009cb0 tx=0x7f7a48005050 comp rx=0 tx=0).stop 2024-06-10T01:29:02.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.036+0000 7f7a6098eb80 1 -- 172.21.15.7:0/2868558070 shutdown_connections 2024-06-10T01:29:02.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.037+0000 7f7a6098eb80 1 --2- 172.21.15.7:0/2868558070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db66ae6880 0x55db66adb3c0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:02.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.037+0000 7f7a6098eb80 1 --2- 172.21.15.7:0/2868558070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db66ae07a0 0x55db66ae9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:02.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.037+0000 7f7a6098eb80 1 --2- 172.21.15.7:0/2868558070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db66ae5f60 0x55db66adae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:02.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.037+0000 7f7a6098eb80 1 -- 172.21.15.7:0/2868558070 >> 172.21.15.7:0/2868558070 conn(0x55db66a0d080 msgr2=0x55db66ae8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:02.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.037+0000 7f7a6098eb80 1 -- 172.21.15.7:0/2868558070 shutdown_connections 2024-06-10T01:29:02.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.037+0000 7f7a6098eb80 1 -- 172.21.15.7:0/2868558070 wait complete. 2024-06-10T01:29:02.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.037+0000 7f7a6098eb80 1 -- start start 2024-06-10T01:29:02.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.038+0000 7f7a6098eb80 1 Processor -- start 2024-06-10T01:29:02.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.038+0000 7f7a6098eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db66ae07a0 0x55db66af25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:02.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.038+0000 7f7a6098eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db66ae5f60 0x55db66af2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:02.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.038+0000 7f7a6098eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db66ae6880 0x55db66b00e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:02.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.038+0000 7f7a6098eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7a501bb610 con 0x55db66ae07a0 2024-06-10T01:29:02.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.038+0000 7f7a6098eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7a501bca80 con 0x55db66ae5f60 2024-06-10T01:29:02.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.038+0000 7f7a6098eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7a501bbce0 con 0x55db66ae6880 2024-06-10T01:29:02.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.038+0000 7f7a6098eb80 10 client.0 _ll_register_callbacks cb 0x55db669c75b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:29:02.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.038+0000 7f7a4effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db66ae5f60 0x55db66af2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:02.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.039+0000 7f7a4effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db66ae5f60 0x55db66af2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:54268/0 (socket says 172.21.15.7:54268) 2024-06-10T01:29:02.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.039+0000 7f7a4effd700 1 -- 172.21.15.7:0/2970899422 learned_addr learned my addr 172.21.15.7:0/2970899422 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:02.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.039+0000 7f7a4f7fe700 1 --2- 172.21.15.7:0/2970899422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db66ae07a0 0x55db66af25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:02.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.039+0000 7f7a4effd700 1 -- 172.21.15.7:0/2970899422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db66ae6880 msgr2=0x55db66b00e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:02.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.039+0000 7f7a4effd700 1 --2- 172.21.15.7:0/2970899422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db66ae6880 0x55db66b00e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:02.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.039+0000 7f7a4effd700 1 -- 172.21.15.7:0/2970899422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db66ae07a0 msgr2=0x55db66af25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:02.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.039+0000 7f7a4effd700 1 --2- 172.21.15.7:0/2970899422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db66ae07a0 0x55db66af25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:02.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.040+0000 7f7a4effd700 1 -- 172.21.15.7:0/2970899422 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db66b01350 con 0x55db66ae5f60 2024-06-10T01:29:02.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.040+0000 7f7a4effd700 1 --2- 172.21.15.7:0/2970899422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db66ae5f60 0x55db66af2b00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7a44098960 tx=0x7f7a440218a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=beab35c73a0572f0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:02.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.040+0000 7f7a2effd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:29:02.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.040+0000 7f7a2effd700 1 -- 172.21.15.7:0/2970899422 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4409c960 con 0x55db66ae5f60 2024-06-10T01:29:02.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.040+0000 7f7a2effd700 1 -- 172.21.15.7:0/2970899422 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f7a440cdbd0 con 0x55db66ae5f60 2024-06-10T01:29:02.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.040+0000 7f7a6098eb80 1 -- 172.21.15.7:0/2970899422 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a48022030 con 0x55db66ae5f60 2024-06-10T01:29:02.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.041+0000 7f7a2effd700 1 -- 172.21.15.7:0/2970899422 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f7a4409c500 con 0x55db66ae5f60 2024-06-10T01:29:02.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.041+0000 7f7a6098eb80 10 client.30062 fetch_fsmap learned FSMap version 55 2024-06-10T01:29:02.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.041+0000 7f7a6098eb80 1 -- 172.21.15.7:0/2970899422 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55db66b64410 con 0x55db66ae5f60 2024-06-10T01:29:02.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.041+0000 7f7a2effd700 1 -- 172.21.15.7:0/2970899422 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a48022030 con 0x55db66ae5f60 2024-06-10T01:29:02.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.041+0000 7f7a2effd700 1 -- 172.21.15.7:0/2970899422 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f7a440a4c50 con 0x55db66ae5f60 2024-06-10T01:29:02.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.041+0000 7f7a6098eb80 10 client.30062 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:29:02.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.041+0000 7f7a6098eb80 10 client.30062 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:29:02.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.041+0000 7f7a6098eb80 20 client.30062 populate_metadata read hostname 'smithi007' 2024-06-10T01:29:02.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.041+0000 7f7a6098eb80 1 --2- 172.21.15.7:0/2970899422 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55db66afa0d0 0x55db66b66b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:02.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.041+0000 7f7a6098eb80 4 client.30062 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:29:02.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.041+0000 7f7a6098eb80 1 -- 172.21.15.7:0/2970899422 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55db66a392d0 con 0x55db66afa0d0 2024-06-10T01:29:02.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.042+0000 7f7a4f7fe700 1 --2- 172.21.15.7:0/2970899422 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55db66afa0d0 0x55db66b66b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.043+0000 7f7a4f7fe700 1 --2- 172.21.15.7:0/2970899422 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55db66afa0d0 0x55db66b66b40 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b6a7611a731569b5 server_cookie=ecb127fa92ae82d5 in_seq=0 out_seq=0 2024-06-10T01:29:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.043+0000 7f7a2effd700 10 client.30062 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:29:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.043+0000 7f7a2effd700 1 -- 172.21.15.7:0/2970899422 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55db66a392d0 con 0x55db66afa0d0 2024-06-10T01:29:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.043+0000 7f7a2effd700 10 client.30062 handle_command_reply: tid=0 2024-06-10T01:29:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.043+0000 7f7a6098eb80 1 client.30062 shutdown 2024-06-10T01:29:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.043+0000 7f7a6098eb80 1 -- 172.21.15.7:0/2970899422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db66ae5f60 msgr2=0x55db66af2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.043+0000 7f7a6098eb80 1 --2- 172.21.15.7:0/2970899422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db66ae5f60 0x55db66af2b00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7a44098960 tx=0x7f7a440218a0 comp rx=0 tx=0).stop 2024-06-10T01:29:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.044+0000 7f7a6098eb80 1 -- 172.21.15.7:0/2970899422 shutdown_connections 2024-06-10T01:29:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.044+0000 7f7a6098eb80 1 --2- 172.21.15.7:0/2970899422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db66ae6880 0x55db66b00e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.044+0000 7f7a6098eb80 1 --2- 172.21.15.7:0/2970899422 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55db66afa0d0 0x55db66b66b40 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.044+0000 7f7a2effd700 0 client.30062 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:29:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.044+0000 7f7a6098eb80 1 --2- 172.21.15.7:0/2970899422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db66ae07a0 0x55db66af25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.044+0000 7f7a6098eb80 1 --2- 172.21.15.7:0/2970899422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db66ae5f60 0x55db66af2b00 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.044+0000 7f7a6098eb80 1 -- 172.21.15.7:0/2970899422 >> 172.21.15.7:0/2970899422 conn(0x55db66a0d080 msgr2=0x55db66ae90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.045+0000 7f7a6098eb80 1 -- 172.21.15.7:0/2970899422 shutdown_connections 2024-06-10T01:29:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.045+0000 7f7a6098eb80 1 -- 172.21.15.7:0/2970899422 wait complete. 2024-06-10T01:29:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.045+0000 7f7a6098eb80 20 client.30062 trim_cache size 0 max 16384 2024-06-10T01:29:02.040 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:29:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.046+0000 7f7a5610b700 1 -- 172.21.15.7:0/3919452012 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7a380a2d70 msgr2=0x7f7a380a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.046+0000 7f7a5610b700 1 --2- 172.21.15.7:0/3919452012 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7a380a2d70 0x7f7a380a5230 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7a48021000 tx=0x7f7a4801b000 comp rx=0 tx=0).stop 2024-06-10T01:29:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.046+0000 7f7a5610b700 1 -- 172.21.15.7:0/3919452012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a50102ef0 msgr2=0x7f7a501adcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.046+0000 7f7a5610b700 1 --2- 172.21.15.7:0/3919452012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a50102ef0 0x7f7a501adcc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7a44005f50 tx=0x7f7a440057c0 comp rx=0 tx=0).stop 2024-06-10T01:29:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.047+0000 7f7a5610b700 1 -- 172.21.15.7:0/3919452012 shutdown_connections 2024-06-10T01:29:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.047+0000 7f7a5610b700 1 --2- 172.21.15.7:0/3919452012 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7a380a2d70 0x7f7a380a5230 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.047+0000 7f7a5610b700 1 --2- 172.21.15.7:0/3919452012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a501085d0 0x7f7a501a7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.047+0000 7f7a5610b700 1 --2- 172.21.15.7:0/3919452012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a50102530 0x7f7a501ad780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.047+0000 7f7a5610b700 1 --2- 172.21.15.7:0/3919452012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a50102ef0 0x7f7a501adcc0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.047+0000 7f7a5610b700 1 -- 172.21.15.7:0/3919452012 >> 172.21.15.7:0/3919452012 conn(0x7f7a50079640 msgr2=0x7f7a50100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.047+0000 7f7a5610b700 1 -- 172.21.15.7:0/3919452012 shutdown_connections 2024-06-10T01:29:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:02.047+0000 7f7a5610b700 1 -- 172.21.15.7:0/3919452012 wait complete. 2024-06-10T01:29:02.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:01 smithi078 ceph-mon[36683]: pgmap v1124: 129 pgs: 129 active+clean; 660 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 3.6 MiB/s wr, 592 op/s 2024-06-10T01:29:02.560 INFO:tasks.workunit.client.0.smithi007.stdout: 10 234858 22.81 MB/sec execute 459 sec latency 95.677 ms 2024-06-10T01:29:03.560 INFO:tasks.workunit.client.0.smithi007.stdout: 10 235027 22.78 MB/sec execute 460 sec latency 781.138 ms 2024-06-10T01:29:03.757 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:03 smithi007 ceph-mon[32398]: pgmap v1125: 129 pgs: 129 active+clean; 666 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.4 MiB/s wr, 555 op/s 2024-06-10T01:29:03.811 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:29:03.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:03 smithi175 ceph-mon[36636]: pgmap v1125: 129 pgs: 129 active+clean; 666 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.4 MiB/s wr, 555 op/s 2024-06-10T01:29:04.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:03 smithi078 ceph-mon[36683]: pgmap v1125: 129 pgs: 129 active+clean; 666 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.4 MiB/s wr, 555 op/s 2024-06-10T01:29:04.561 INFO:tasks.workunit.client.0.smithi007.stdout: 10 235084 22.74 MB/sec execute 461 sec latency 1386.321 ms 2024-06-10T01:29:04.612 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:29:05.561 INFO:tasks.workunit.client.0.smithi007.stdout: 10 235304 22.71 MB/sec execute 462 sec latency 155.789 ms 2024-06-10T01:29:05.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:05 smithi175 ceph-mon[36636]: pgmap v1126: 129 pgs: 129 active+clean; 669 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.1 MiB/s wr, 479 op/s 2024-06-10T01:29:05.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:05 smithi007 ceph-mon[32398]: pgmap v1126: 129 pgs: 129 active+clean; 669 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.1 MiB/s wr, 479 op/s 2024-06-10T01:29:06.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:05 smithi078 ceph-mon[36683]: pgmap v1126: 129 pgs: 129 active+clean; 669 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.1 MiB/s wr, 479 op/s 2024-06-10T01:29:06.561 INFO:tasks.workunit.client.0.smithi007.stdout: 10 235582 22.69 MB/sec execute 463 sec latency 132.907 ms 2024-06-10T01:29:07.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.279+0000 7f7c3c73b700 1 -- 172.21.15.7:0/836310532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c34101f80 msgr2=0x7f7c34105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:07.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.279+0000 7f7c3c73b700 1 --2- 172.21.15.7:0/836310532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c34101f80 0x7f7c34105fd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7c300099c0 tx=0x7f7c30009cd0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.279+0000 7f7c3c73b700 1 -- 172.21.15.7:0/836310532 shutdown_connections 2024-06-10T01:29:07.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.279+0000 7f7c3c73b700 1 --2- 172.21.15.7:0/836310532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c341066b0 0x7f7c3410e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.279+0000 7f7c3c73b700 1 --2- 172.21.15.7:0/836310532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c341015d0 0x7f7c341019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.279+0000 7f7c3c73b700 1 --2- 172.21.15.7:0/836310532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c34101f80 0x7f7c34105fd0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.279+0000 7f7c3c73b700 1 -- 172.21.15.7:0/836310532 >> 172.21.15.7:0/836310532 conn(0x7f7c34078430 msgr2=0x7f7c34078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:07.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.279+0000 7f7c3c73b700 1 -- 172.21.15.7:0/836310532 shutdown_connections 2024-06-10T01:29:07.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.279+0000 7f7c3c73b700 1 -- 172.21.15.7:0/836310532 wait complete. 2024-06-10T01:29:07.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.280+0000 7f7c3c73b700 1 Processor -- start 2024-06-10T01:29:07.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.280+0000 7f7c3c73b700 1 -- start start 2024-06-10T01:29:07.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.281+0000 7f7c3c73b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c341015d0 0x7f7c3419e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.281+0000 7f7c3c73b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c34101f80 0x7f7c3419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.281+0000 7f7c3c73b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c341066b0 0x7f7c34198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.281+0000 7f7c3c73b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341116e0 con 0x7f7c341015d0 2024-06-10T01:29:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.281+0000 7f7c3c73b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7c34111560 con 0x7f7c34101f80 2024-06-10T01:29:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.281+0000 7f7c3c73b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7c34111860 con 0x7f7c341066b0 2024-06-10T01:29:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.281+0000 7f7c3a4d7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c341015d0 0x7f7c3419e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.281+0000 7f7c3a4d7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c341015d0 0x7f7c3419e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54338/0 (socket says 172.21.15.7:54338) 2024-06-10T01:29:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.281+0000 7f7c39cd6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c34101f80 0x7f7c3419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.281+0000 7f7c3a4d7700 1 -- 172.21.15.7:0/3415940345 learned_addr learned my addr 172.21.15.7:0/3415940345 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.281+0000 7f7c3acd8700 1 --2- 172.21.15.7:0/3415940345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c341066b0 0x7f7c34198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.281+0000 7f7c3a4d7700 1 -- 172.21.15.7:0/3415940345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c341066b0 msgr2=0x7f7c34198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.282+0000 7f7c3a4d7700 1 --2- 172.21.15.7:0/3415940345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c341066b0 0x7f7c34198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.282+0000 7f7c3a4d7700 1 -- 172.21.15.7:0/3415940345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c34101f80 msgr2=0x7f7c3419ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.282+0000 7f7c3a4d7700 1 --2- 172.21.15.7:0/3415940345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c34101f80 0x7f7c3419ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.282+0000 7f7c3a4d7700 1 -- 172.21.15.7:0/3415940345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c341991f0 con 0x7f7c341015d0 2024-06-10T01:29:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.282+0000 7f7c3a4d7700 1 --2- 172.21.15.7:0/3415940345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c341015d0 0x7f7c3419e8d0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800d980 tx=0x7f7c2800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.282+0000 7f7c277fe700 1 -- 172.21.15.7:0/3415940345 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c28016070 con 0x7f7c341015d0 2024-06-10T01:29:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.283+0000 7f7c277fe700 1 -- 172.21.15.7:0/3415940345 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c28004d10 con 0x7f7c341015d0 2024-06-10T01:29:07.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.283+0000 7f7c277fe700 1 -- 172.21.15.7:0/3415940345 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c28005670 con 0x7f7c341015d0 2024-06-10T01:29:07.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.283+0000 7f7c3c73b700 1 -- 172.21.15.7:0/3415940345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c341994e0 con 0x7f7c341015d0 2024-06-10T01:29:07.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.283+0000 7f7c3c73b700 1 -- 172.21.15.7:0/3415940345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c34102fa0 con 0x7f7c341015d0 2024-06-10T01:29:07.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.287+0000 7f7c277fe700 1 -- 172.21.15.7:0/3415940345 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f7c28004750 con 0x7f7c341015d0 2024-06-10T01:29:07.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.289+0000 7f7c277fe700 1 --2- 172.21.15.7:0/3415940345 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7c200a2d60 0x7f7c200a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.290+0000 7f7c277fe700 1 -- 172.21.15.7:0/3415940345 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f7c280ceeb0 con 0x7f7c341015d0 2024-06-10T01:29:07.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.290+0000 7f7c46fbeb80 1 Processor -- start 2024-06-10T01:29:07.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.290+0000 7f7c46fbeb80 1 -- start start 2024-06-10T01:29:07.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.291+0000 7f7c39cd6700 1 --2- 172.21.15.7:0/3415940345 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7c200a2d60 0x7f7c200a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.291+0000 7f7c46fbeb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a7e982f60 0x559a7e983340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.291+0000 7f7c46fbeb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a7e983880 0x559a7e97d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.291+0000 7f7c46fbeb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a7e97d7a0 0x559a7e97dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.291+0000 7f7c46fbeb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559a7e86ff60 con 0x559a7e97d7a0 2024-06-10T01:29:07.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.291+0000 7f7c46fbeb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559a7e8ce850 con 0x559a7e983880 2024-06-10T01:29:07.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.291+0000 7f7c46fbeb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559a7e836460 con 0x559a7e982f60 2024-06-10T01:29:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.291+0000 7f7c3acd8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a7e97d7a0 0x559a7e97dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.291+0000 7f7c3acd8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a7e97d7a0 0x559a7e97dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54340/0 (socket says 172.21.15.7:54340) 2024-06-10T01:29:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.291+0000 7f7c3acd8700 1 -- 172.21.15.7:0/3141986446 learned_addr learned my addr 172.21.15.7:0/3141986446 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.291+0000 7f7c39cd6700 1 --2- 172.21.15.7:0/3141986446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a7e983880 0x559a7e97d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.292+0000 7f7c39cd6700 1 --2- 172.21.15.7:0/3415940345 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7c200a2d60 0x7f7c200a5220 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f7c30000c00 tx=0x7f7c30005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.292+0000 7f7c39cd6700 1 -- 172.21.15.7:0/3141986446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a7e982f60 msgr2=0x559a7e983340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T01:29:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.292+0000 7f7c39cd6700 1 --2- 172.21.15.7:0/3141986446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a7e982f60 0x559a7e983340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.292+0000 7f7c39cd6700 1 -- 172.21.15.7:0/3141986446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a7e97d7a0 msgr2=0x559a7e97dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.292+0000 7f7c39cd6700 1 --2- 172.21.15.7:0/3141986446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a7e97d7a0 0x559a7e97dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.292+0000 7f7c39cd6700 1 -- 172.21.15.7:0/3141986446 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a7e81d8d0 con 0x559a7e983880 2024-06-10T01:29:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.292+0000 7f7c3acd8700 1 --2- 172.21.15.7:0/3141986446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a7e97d7a0 0x559a7e97dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:29:07.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.293+0000 7f7c39cd6700 1 --2- 172.21.15.7:0/3141986446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a7e983880 0x559a7e97d260 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009990 tx=0x7f7c30027000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9507bb0a092d3fa server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:07.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.293+0000 7f7c24ff9700 1 -- 172.21.15.7:0/3141986446 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30029040 con 0x559a7e983880 2024-06-10T01:29:07.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.294+0000 7f7c24ff9700 1 -- 172.21.15.7:0/3141986446 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c30032070 con 0x559a7e983880 2024-06-10T01:29:07.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.294+0000 7f7c24ff9700 1 -- 172.21.15.7:0/3141986446 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30024a30 con 0x559a7e983880 2024-06-10T01:29:07.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.295+0000 7f7c46fbeb80 1 -- 172.21.15.7:0/3141986446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a7e983880 msgr2=0x559a7e97d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:07.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.295+0000 7f7c46fbeb80 1 --2- 172.21.15.7:0/3141986446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a7e983880 0x559a7e97d260 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009990 tx=0x7f7c30027000 comp rx=0 tx=0).stop 2024-06-10T01:29:07.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.295+0000 7f7c46fbeb80 1 -- 172.21.15.7:0/3141986446 shutdown_connections 2024-06-10T01:29:07.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.295+0000 7f7c46fbeb80 1 --2- 172.21.15.7:0/3141986446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a7e97d7a0 0x559a7e97dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.295+0000 7f7c46fbeb80 1 --2- 172.21.15.7:0/3141986446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a7e982f60 0x559a7e983340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.295+0000 7f7c46fbeb80 1 --2- 172.21.15.7:0/3141986446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a7e983880 0x559a7e97d260 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.295+0000 7f7c46fbeb80 1 -- 172.21.15.7:0/3141986446 >> 172.21.15.7:0/3141986446 conn(0x559a7e8aa080 msgr2=0x559a7e978220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:07.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.296+0000 7f7c46fbeb80 1 -- 172.21.15.7:0/3141986446 shutdown_connections 2024-06-10T01:29:07.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.296+0000 7f7c46fbeb80 1 -- 172.21.15.7:0/3141986446 wait complete. 2024-06-10T01:29:07.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.297+0000 7f7c46fbeb80 1 -- start start 2024-06-10T01:29:07.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.297+0000 7f7c46fbeb80 1 Processor -- start 2024-06-10T01:29:07.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.297+0000 7f7c46fbeb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a7e97d7a0 0x559a7e989600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.297+0000 7f7c46fbeb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a7e982f60 0x559a7e986900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.297+0000 7f7c46fbeb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a7e983880 0x559a7e986e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.297+0000 7f7c46fbeb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559a7e86efc0 con 0x559a7e983880 2024-06-10T01:29:07.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.297+0000 7f7c46fbeb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559a7e8ca810 con 0x559a7e982f60 2024-06-10T01:29:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.297+0000 7f7c46fbeb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559a7e98a2a0 con 0x559a7e97d7a0 2024-06-10T01:29:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.297+0000 7f7c46fbeb80 10 client.0 _ll_register_callbacks cb 0x559a7e8645b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:29:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.298+0000 7f7c3acd8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a7e983880 0x559a7e986e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.298+0000 7f7c3a4d7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a7e97d7a0 0x559a7e989600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.298+0000 7f7c3a4d7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a7e97d7a0 0x559a7e989600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36360/0 (socket says 172.21.15.7:36360) 2024-06-10T01:29:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.298+0000 7f7c3a4d7700 1 -- 172.21.15.7:0/3426184328 learned_addr learned my addr 172.21.15.7:0/3426184328 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.298+0000 7f7c3acd8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a7e983880 0x559a7e986e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54342/0 (socket says 172.21.15.7:54342) 2024-06-10T01:29:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.298+0000 7f7c39cd6700 1 --2- 172.21.15.7:0/3426184328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a7e982f60 0x559a7e986900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.298+0000 7f7c3a4d7700 1 -- 172.21.15.7:0/3426184328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a7e982f60 msgr2=0x559a7e986900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.298+0000 7f7c3a4d7700 1 --2- 172.21.15.7:0/3426184328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a7e982f60 0x559a7e986900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.298+0000 7f7c3a4d7700 1 -- 172.21.15.7:0/3426184328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a7e983880 msgr2=0x559a7e986e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.298+0000 7f7c3a4d7700 1 --2- 172.21.15.7:0/3426184328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a7e983880 0x559a7e986e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.298+0000 7f7c3a4d7700 1 -- 172.21.15.7:0/3426184328 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a7e894750 con 0x559a7e97d7a0 2024-06-10T01:29:07.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.298+0000 7f7c1dffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:29:07.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.298+0000 7f7c3a4d7700 1 --2- 172.21.15.7:0/3426184328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a7e97d7a0 0x559a7e989600 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7c28097790 tx=0x7f7c2809bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc6b0f70ffdcbc3f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:07.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.299+0000 7f7c1dffb700 1 -- 172.21.15.7:0/3426184328 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c280b4060 con 0x559a7e97d7a0 2024-06-10T01:29:07.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.299+0000 7f7c46fbeb80 1 -- 172.21.15.7:0/3426184328 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c30032070 con 0x559a7e97d7a0 2024-06-10T01:29:07.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.299+0000 7f7c1dffb700 1 -- 172.21.15.7:0/3426184328 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f7c2800b2e0 con 0x559a7e97d7a0 2024-06-10T01:29:07.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.299+0000 7f7c1dffb700 1 -- 172.21.15.7:0/3426184328 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f7c280b4060 con 0x559a7e97d7a0 2024-06-10T01:29:07.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.299+0000 7f7c46fbeb80 10 client.29986 fetch_fsmap learned FSMap version 55 2024-06-10T01:29:07.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.299+0000 7f7c46fbeb80 1 -- 172.21.15.7:0/3426184328 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x559a7e994b30 con 0x559a7e97d7a0 2024-06-10T01:29:07.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.300+0000 7f7c1dffb700 1 -- 172.21.15.7:0/3426184328 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c30032070 con 0x559a7e97d7a0 2024-06-10T01:29:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.300+0000 7f7c1dffb700 1 -- 172.21.15.7:0/3426184328 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f7c2809b590 con 0x559a7e97d7a0 2024-06-10T01:29:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.300+0000 7f7c46fbeb80 10 client.29986 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:29:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.300+0000 7f7c46fbeb80 10 client.29986 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:29:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.300+0000 7f7c46fbeb80 20 client.29986 populate_metadata read hostname 'smithi007' 2024-06-10T01:29:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.300+0000 7f7c46fbeb80 1 --2- 172.21.15.7:0/3426184328 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x559a7e9974d0 0x559a7e9998f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.300+0000 7f7c46fbeb80 4 client.29986 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:29:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.300+0000 7f7c46fbeb80 1 -- 172.21.15.7:0/3426184328 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a7e8dbd50 con 0x559a7e9974d0 2024-06-10T01:29:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.301+0000 7f7c39cd6700 1 --2- 172.21.15.7:0/3426184328 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x559a7e9974d0 0x559a7e9998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.302+0000 7f7c1dffb700 10 client.29986 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:29:07.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.302+0000 7f7c39cd6700 1 --2- 172.21.15.7:0/3426184328 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x559a7e9974d0 0x559a7e9998f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1193bbcf2dcf8a84 server_cookie=efc059e7bb9f3b7a in_seq=0 out_seq=0 2024-06-10T01:29:07.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.307+0000 7f7c1dffb700 1 -- 172.21.15.7:0/3426184328 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559a7e8dbd50 con 0x559a7e9974d0 2024-06-10T01:29:07.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.308+0000 7f7c1dffb700 10 client.29986 handle_command_reply: tid=0 2024-06-10T01:29:07.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.308+0000 7f7c46fbeb80 1 client.29986 shutdown 2024-06-10T01:29:07.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.308+0000 7f7c46fbeb80 1 -- 172.21.15.7:0/3426184328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a7e97d7a0 msgr2=0x559a7e989600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:07.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.308+0000 7f7c46fbeb80 1 --2- 172.21.15.7:0/3426184328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a7e97d7a0 0x559a7e989600 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7c28097790 tx=0x7f7c2809bab0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.308+0000 7f7c46fbeb80 1 -- 172.21.15.7:0/3426184328 shutdown_connections 2024-06-10T01:29:07.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.308+0000 7f7c46fbeb80 1 --2- 172.21.15.7:0/3426184328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a7e983880 0x559a7e986e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.308+0000 7f7c46fbeb80 1 --2- 172.21.15.7:0/3426184328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a7e97d7a0 0x559a7e989600 secure :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7c28097790 tx=0x7f7c2809bab0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.309+0000 7f7c46fbeb80 1 --2- 172.21.15.7:0/3426184328 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x559a7e9974d0 0x559a7e9998f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.309+0000 7f7c46fbeb80 1 --2- 172.21.15.7:0/3426184328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a7e982f60 0x559a7e986900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.309+0000 7f7c46fbeb80 1 -- 172.21.15.7:0/3426184328 >> 172.21.15.7:0/3426184328 conn(0x559a7e8aa080 msgr2=0x559a7e977700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:07.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.309+0000 7f7c1dffb700 0 client.29986 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:29:07.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.309+0000 7f7c46fbeb80 1 -- 172.21.15.7:0/3426184328 shutdown_connections 2024-06-10T01:29:07.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.309+0000 7f7c46fbeb80 1 -- 172.21.15.7:0/3426184328 wait complete. 2024-06-10T01:29:07.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.310+0000 7f7c46fbeb80 20 client.29986 trim_cache size 0 max 16384 2024-06-10T01:29:07.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.331+0000 7f7c46fbeb80 1 Processor -- start 2024-06-10T01:29:07.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.331+0000 7f7c46fbeb80 1 -- start start 2024-06-10T01:29:07.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.332+0000 7f7c46fbeb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a7e982f60 0x559a7e9857d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.332+0000 7f7c46fbeb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a7e983880 0x559a7e977e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.332+0000 7f7c46fbeb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a7e985d10 0x559a7e9783c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.332+0000 7f7c46fbeb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341b7350 con 0x559a7e985d10 2024-06-10T01:29:07.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.332+0000 7f7c46fbeb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341b4be0 con 0x559a7e983880 2024-06-10T01:29:07.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.332+0000 7f7c46fbeb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341b29d0 con 0x559a7e982f60 2024-06-10T01:29:07.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.332+0000 7f7c39cd6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a7e983880 0x559a7e977e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.332+0000 7f7c3a4d7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a7e982f60 0x559a7e9857d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.332+0000 7f7c39cd6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a7e983880 0x559a7e977e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:54300/0 (socket says 172.21.15.7:54300) 2024-06-10T01:29:07.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.332+0000 7f7c39cd6700 1 -- 172.21.15.7:0/28284132 learned_addr learned my addr 172.21.15.7:0/28284132 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:07.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.332+0000 7f7c3acd8700 1 --2- 172.21.15.7:0/28284132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a7e985d10 0x559a7e9783c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.332+0000 7f7c39cd6700 1 -- 172.21.15.7:0/28284132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a7e982f60 msgr2=0x559a7e9857d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:07.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.332+0000 7f7c39cd6700 1 --2- 172.21.15.7:0/28284132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a7e982f60 0x559a7e9857d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.332+0000 7f7c39cd6700 1 -- 172.21.15.7:0/28284132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a7e985d10 msgr2=0x559a7e9783c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:07.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.332+0000 7f7c39cd6700 1 --2- 172.21.15.7:0/28284132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a7e985d10 0x559a7e9783c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.332+0000 7f7c39cd6700 1 -- 172.21.15.7:0/28284132 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a7e987830 con 0x559a7e983880 2024-06-10T01:29:07.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.333+0000 7f7c39cd6700 1 --2- 172.21.15.7:0/28284132 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a7e983880 0x559a7e977e80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7c300099c0 tx=0x7f7c30028000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63bad93a712d68f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:07.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.333+0000 7f7c1ffff700 1 -- 172.21.15.7:0/28284132 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30029040 con 0x559a7e983880 2024-06-10T01:29:07.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.333+0000 7f7c1ffff700 1 -- 172.21.15.7:0/28284132 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c30032040 con 0x559a7e983880 2024-06-10T01:29:07.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.333+0000 7f7c1ffff700 1 -- 172.21.15.7:0/28284132 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30038410 con 0x559a7e983880 2024-06-10T01:29:07.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.334+0000 7f7c46fbeb80 1 -- 172.21.15.7:0/28284132 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a7e983880 msgr2=0x559a7e977e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:07.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.334+0000 7f7c46fbeb80 1 --2- 172.21.15.7:0/28284132 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a7e983880 0x559a7e977e80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7c300099c0 tx=0x7f7c30028000 comp rx=0 tx=0).stop 2024-06-10T01:29:07.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.334+0000 7f7c46fbeb80 1 -- 172.21.15.7:0/28284132 shutdown_connections 2024-06-10T01:29:07.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.334+0000 7f7c46fbeb80 1 --2- 172.21.15.7:0/28284132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a7e985d10 0x559a7e9783c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.334+0000 7f7c46fbeb80 1 --2- 172.21.15.7:0/28284132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a7e982f60 0x559a7e9857d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.334+0000 7f7c46fbeb80 1 --2- 172.21.15.7:0/28284132 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a7e983880 0x559a7e977e80 secure :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7c300099c0 tx=0x7f7c30028000 comp rx=0 tx=0).stop 2024-06-10T01:29:07.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.334+0000 7f7c46fbeb80 1 -- 172.21.15.7:0/28284132 >> 172.21.15.7:0/28284132 conn(0x559a7e8aa080 msgr2=0x559a7e9850c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:07.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.335+0000 7f7c46fbeb80 1 -- 172.21.15.7:0/28284132 shutdown_connections 2024-06-10T01:29:07.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.335+0000 7f7c46fbeb80 1 -- 172.21.15.7:0/28284132 wait complete. 2024-06-10T01:29:07.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.336+0000 7f7c46fbeb80 1 -- start start 2024-06-10T01:29:07.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.336+0000 7f7c46fbeb80 1 Processor -- start 2024-06-10T01:29:07.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.336+0000 7f7c46fbeb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a7e982f60 0x559a7e98fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.337+0000 7f7c46fbeb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a7e985d10 0x559a7e99e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.337+0000 7f7c46fbeb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a7e99ee10 0x559a7e99f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.337+0000 7f7c3a4d7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a7e982f60 0x559a7e98fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.337+0000 7f7c3a4d7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a7e982f60 0x559a7e98fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36376/0 (socket says 172.21.15.7:36376) 2024-06-10T01:29:07.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.337+0000 7f7c3a4d7700 1 -- 172.21.15.7:0/1969804137 learned_addr learned my addr 172.21.15.7:0/1969804137 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:07.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.337+0000 7f7c46fbeb80 1 -- 172.21.15.7:0/1969804137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341b30a0 con 0x559a7e99ee10 2024-06-10T01:29:07.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.337+0000 7f7c46fbeb80 1 -- 172.21.15.7:0/1969804137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341b4510 con 0x559a7e985d10 2024-06-10T01:29:07.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.337+0000 7f7c46fbeb80 1 -- 172.21.15.7:0/1969804137 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341b3770 con 0x559a7e982f60 2024-06-10T01:29:07.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.337+0000 7f7c39cd6700 1 --2- 172.21.15.7:0/1969804137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a7e985d10 0x559a7e99e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.337+0000 7f7c46fbeb80 10 client.0 _ll_register_callbacks cb 0x559a7e8645b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:29:07.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.338+0000 7f7c3acd8700 1 --2- 172.21.15.7:0/1969804137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a7e99ee10 0x559a7e99f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.338+0000 7f7c39cd6700 1 -- 172.21.15.7:0/1969804137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a7e982f60 msgr2=0x559a7e98fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:07.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.338+0000 7f7c39cd6700 1 --2- 172.21.15.7:0/1969804137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a7e982f60 0x559a7e98fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.338+0000 7f7c39cd6700 1 -- 172.21.15.7:0/1969804137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a7e99ee10 msgr2=0x559a7e99f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:07.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.338+0000 7f7c39cd6700 1 --2- 172.21.15.7:0/1969804137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a7e99ee10 0x559a7e99f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.338+0000 7f7c39cd6700 1 -- 172.21.15.7:0/1969804137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a7ea011b0 con 0x559a7e985d10 2024-06-10T01:29:07.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.339+0000 7f7c39cd6700 1 --2- 172.21.15.7:0/1969804137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a7e985d10 0x559a7e99e8d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x559a7e999790 tx=0x7f7c3000be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8de3cc0c15457c8e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:07.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.339+0000 7f7c1dffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:29:07.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.339+0000 7f7c1dffb700 1 -- 172.21.15.7:0/1969804137 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30029030 con 0x559a7e985d10 2024-06-10T01:29:07.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.339+0000 7f7c1dffb700 1 -- 172.21.15.7:0/1969804137 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f7c30024b50 con 0x559a7e985d10 2024-06-10T01:29:07.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.339+0000 7f7c46fbeb80 1 -- 172.21.15.7:0/1969804137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c30032040 con 0x559a7e985d10 2024-06-10T01:29:07.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.339+0000 7f7c1dffb700 1 -- 172.21.15.7:0/1969804137 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f7c30051050 con 0x559a7e985d10 2024-06-10T01:29:07.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.339+0000 7f7c46fbeb80 10 client.30080 fetch_fsmap learned FSMap version 55 2024-06-10T01:29:07.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.339+0000 7f7c46fbeb80 1 -- 172.21.15.7:0/1969804137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x559a7ea02450 con 0x559a7e985d10 2024-06-10T01:29:07.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.340+0000 7f7c1dffb700 1 -- 172.21.15.7:0/1969804137 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c30032040 con 0x559a7e985d10 2024-06-10T01:29:07.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.340+0000 7f7c1dffb700 1 -- 172.21.15.7:0/1969804137 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f7c3002c030 con 0x559a7e985d10 2024-06-10T01:29:07.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.340+0000 7f7c46fbeb80 10 client.30080 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:29:07.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.340+0000 7f7c46fbeb80 10 client.30080 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:29:07.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.340+0000 7f7c46fbeb80 20 client.30080 populate_metadata read hostname 'smithi007' 2024-06-10T01:29:07.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.340+0000 7f7c46fbeb80 1 --2- 172.21.15.7:0/1969804137 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x559a7ea02620 0x559a7ea04a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.340+0000 7f7c46fbeb80 4 client.30080 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:29:07.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.340+0000 7f7c46fbeb80 1 -- 172.21.15.7:0/1969804137 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a7e8d62d0 con 0x559a7ea02620 2024-06-10T01:29:07.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.342+0000 7f7c3a4d7700 1 --2- 172.21.15.7:0/1969804137 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x559a7ea02620 0x559a7ea04a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.344+0000 7f7c3a4d7700 1 --2- 172.21.15.7:0/1969804137 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x559a7ea02620 0x559a7ea04a10 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=538c3652665547a5 server_cookie=940ac214cc6f3c78 in_seq=0 out_seq=0 2024-06-10T01:29:07.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.344+0000 7f7c1dffb700 10 client.30080 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:29:07.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.345+0000 7f7c1dffb700 1 -- 172.21.15.7:0/1969804137 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a7e8d62d0 con 0x559a7ea02620 2024-06-10T01:29:07.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.345+0000 7f7c1dffb700 10 client.30080 handle_command_reply: tid=0 2024-06-10T01:29:07.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.345+0000 7f7c46fbeb80 1 client.30080 shutdown 2024-06-10T01:29:07.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.345+0000 7f7c46fbeb80 1 -- 172.21.15.7:0/1969804137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a7e985d10 msgr2=0x559a7e99e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:07.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.345+0000 7f7c46fbeb80 1 --2- 172.21.15.7:0/1969804137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a7e985d10 0x559a7e99e8d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x559a7e999790 tx=0x7f7c3000be30 comp rx=0 tx=0).stop 2024-06-10T01:29:07.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.346+0000 7f7c46fbeb80 1 -- 172.21.15.7:0/1969804137 shutdown_connections 2024-06-10T01:29:07.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.346+0000 7f7c46fbeb80 1 --2- 172.21.15.7:0/1969804137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a7e99ee10 0x559a7e99f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.340 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:29:07.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.346+0000 7f7c46fbeb80 1 --2- 172.21.15.7:0/1969804137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a7e982f60 0x559a7e98fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.346+0000 7f7c46fbeb80 1 --2- 172.21.15.7:0/1969804137 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x559a7ea02620 0x559a7ea04a10 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.346+0000 7f7c46fbeb80 1 --2- 172.21.15.7:0/1969804137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a7e985d10 0x559a7e99e8d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.346+0000 7f7c46fbeb80 1 -- 172.21.15.7:0/1969804137 >> 172.21.15.7:0/1969804137 conn(0x559a7e8aa080 msgr2=0x559a7e988130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:07.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.346+0000 7f7c1dffb700 0 client.30080 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:29:07.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.346+0000 7f7c46fbeb80 1 -- 172.21.15.7:0/1969804137 shutdown_connections 2024-06-10T01:29:07.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.346+0000 7f7c46fbeb80 1 -- 172.21.15.7:0/1969804137 wait complete. 2024-06-10T01:29:07.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.346+0000 7f7c46fbeb80 20 client.30080 trim_cache size 0 max 16384 2024-06-10T01:29:07.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.348+0000 7f7c3c73b700 1 -- 172.21.15.7:0/3415940345 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7c200a2d60 msgr2=0x7f7c200a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:07.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.348+0000 7f7c3c73b700 1 --2- 172.21.15.7:0/3415940345 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7c200a2d60 0x7f7c200a5220 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f7c30000c00 tx=0x7f7c30005fb0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.348+0000 7f7c3c73b700 1 -- 172.21.15.7:0/3415940345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c341015d0 msgr2=0x7f7c3419e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:07.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.348+0000 7f7c3c73b700 1 --2- 172.21.15.7:0/3415940345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c341015d0 0x7f7c3419e8d0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800d980 tx=0x7f7c2800dc90 comp rx=0 tx=0).stop 2024-06-10T01:29:07.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.348+0000 7f7c3c73b700 1 -- 172.21.15.7:0/3415940345 shutdown_connections 2024-06-10T01:29:07.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.348+0000 7f7c3c73b700 1 --2- 172.21.15.7:0/3415940345 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f7c200a2d60 0x7f7c200a5220 secure :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f7c30000c00 tx=0x7f7c30005fb0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.348+0000 7f7c3c73b700 1 --2- 172.21.15.7:0/3415940345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c341066b0 0x7f7c34198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.348+0000 7f7c3c73b700 1 --2- 172.21.15.7:0/3415940345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c341015d0 0x7f7c3419e8d0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.348+0000 7f7c3c73b700 1 --2- 172.21.15.7:0/3415940345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c34101f80 0x7f7c3419ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.348+0000 7f7c3c73b700 1 -- 172.21.15.7:0/3415940345 >> 172.21.15.7:0/3415940345 conn(0x7f7c34078430 msgr2=0x7f7c3406ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:07.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.348+0000 7f7c3c73b700 1 -- 172.21.15.7:0/3415940345 shutdown_connections 2024-06-10T01:29:07.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:07.348+0000 7f7c3c73b700 1 -- 172.21.15.7:0/3415940345 wait complete. 2024-06-10T01:29:07.561 INFO:tasks.workunit.client.0.smithi007.stdout: 10 236101 22.70 MB/sec execute 464 sec latency 122.459 ms 2024-06-10T01:29:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.673+0000 7fca5c15a640 1 -- 172.21.15.7:0/493318121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca541120e0 msgr2=0x7fca54114530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.673+0000 7fca5c15a640 1 --2- 172.21.15.7:0/493318121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca541120e0 0x7fca54114530 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fca4c009ca0 tx=0x7fca4c02a8d0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.674+0000 7fca5c15a640 1 -- 172.21.15.7:0/493318121 shutdown_connections 2024-06-10T01:29:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.675+0000 7fca5c15a640 1 --2- 172.21.15.7:0/493318121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca541120e0 0x7fca54114530 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.675+0000 7fca5c15a640 1 --2- 172.21.15.7:0/493318121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca5410b310 0x7fca5410b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.675+0000 7fca5c15a640 1 --2- 172.21.15.7:0/493318121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca541089a0 0x7fca5410add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.675+0000 7fca5c15a640 1 -- 172.21.15.7:0/493318121 >> 172.21.15.7:0/493318121 conn(0x7fca5406e9e0 msgr2=0x7fca54070e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.675+0000 7fca5c15a640 1 -- 172.21.15.7:0/493318121 shutdown_connections 2024-06-10T01:29:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.675+0000 7fca5c15a640 1 -- 172.21.15.7:0/493318121 wait complete. 2024-06-10T01:29:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.676+0000 7fca5c15a640 1 Processor -- start 2024-06-10T01:29:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.676+0000 7fca5c15a640 1 -- start start 2024-06-10T01:29:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.677+0000 7fca5c15a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca541089a0 0x7fca541a2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.677+0000 7fca5c15a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca5410b310 0x7fca541a2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.677+0000 7fca5c15a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca541120e0 0x7fca541a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.677+0000 7fca5c15a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fca54118140 con 0x7fca541089a0 2024-06-10T01:29:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.677+0000 7fca5c15a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fca54117fc0 con 0x7fca541120e0 2024-06-10T01:29:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.677+0000 7fca5c15a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fca541182c0 con 0x7fca5410b310 2024-06-10T01:29:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.677+0000 7fca59ef6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca541089a0 0x7fca541a2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.677+0000 7fca596f5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca5410b310 0x7fca541a2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.677+0000 7fca5a6f7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca541120e0 0x7fca541a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.677+0000 7fca59ef6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca541089a0 0x7fca541a2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54368/0 (socket says 172.21.15.7:54368) 2024-06-10T01:29:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.677+0000 7fca59ef6640 1 -- 172.21.15.7:0/492249329 learned_addr learned my addr 172.21.15.7:0/492249329 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.677+0000 7fca59ef6640 1 -- 172.21.15.7:0/492249329 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca5410b310 msgr2=0x7fca541a2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.677+0000 7fca59ef6640 1 --2- 172.21.15.7:0/492249329 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca5410b310 0x7fca541a2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.677+0000 7fca59ef6640 1 -- 172.21.15.7:0/492249329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca541120e0 msgr2=0x7fca541a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.677+0000 7fca59ef6640 1 --2- 172.21.15.7:0/492249329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca541120e0 0x7fca541a9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.677+0000 7fca59ef6640 1 -- 172.21.15.7:0/492249329 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca541aa2d0 con 0x7fca541089a0 2024-06-10T01:29:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.677+0000 7fca5a6f7640 1 --2- 172.21.15.7:0/492249329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca541120e0 0x7fca541a9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:29:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.678+0000 7fca59ef6640 1 --2- 172.21.15.7:0/492249329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca541089a0 0x7fca541a2700 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fca5000b450 tx=0x7fca5000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.678+0000 7fca4affd640 1 -- 172.21.15.7:0/492249329 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca50015020 con 0x7fca541089a0 2024-06-10T01:29:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.678+0000 7fca4affd640 1 -- 172.21.15.7:0/492249329 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca50011d70 con 0x7fca541089a0 2024-06-10T01:29:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.678+0000 7fca5c15a640 1 -- 172.21.15.7:0/492249329 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca541aa5c0 con 0x7fca541089a0 2024-06-10T01:29:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.678+0000 7fca4affd640 1 -- 172.21.15.7:0/492249329 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5000f340 con 0x7fca541089a0 2024-06-10T01:29:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.678+0000 7fca5c15a640 1 -- 172.21.15.7:0/492249329 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca541aab10 con 0x7fca541089a0 2024-06-10T01:29:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.681+0000 7fca4affd640 1 -- 172.21.15.7:0/492249329 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fca50018020 con 0x7fca541089a0 2024-06-10T01:29:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.682+0000 7fca4affd640 1 --2- 172.21.15.7:0/492249329 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fca400a2d10 0x7fca400a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.682+0000 7fca596f5640 1 --2- 172.21.15.7:0/492249329 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fca400a2d10 0x7fca400a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.682+0000 7fca4affd640 1 -- 172.21.15.7:0/492249329 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fca500ced20 con 0x7fca541089a0 2024-06-10T01:29:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.683+0000 7fca596f5640 1 --2- 172.21.15.7:0/492249329 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fca400a2d10 0x7fca400a51d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fca541a3bc0 tx=0x7fca44005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.721+0000 7fca5ee68740 1 Processor -- start 2024-06-10T01:29:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.721+0000 7fca5ee68740 1 -- start start 2024-06-10T01:29:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.721+0000 7fca5ee68740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651804c8820 0x5651804c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.721+0000 7fca5ee68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651804c9210 0x5651804cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.721+0000 7fca5ee68740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651804ce380 0x5651804d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.721+0000 7fca5ee68740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5651804234f0 con 0x5651804ce380 2024-06-10T01:29:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.721+0000 7fca5ee68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5651803c1870 con 0x5651804c9210 2024-06-10T01:29:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.721+0000 7fca5ee68740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56518041f4b0 con 0x5651804c8820 2024-06-10T01:29:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.722+0000 7fca5a6f7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651804ce380 0x5651804d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.722+0000 7fca596f5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651804c9210 0x5651804cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.722+0000 7fca5a6f7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651804ce380 0x5651804d0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54370/0 (socket says 172.21.15.7:54370) 2024-06-10T01:29:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.722+0000 7fca5a6f7640 1 -- 172.21.15.7:0/3470482201 learned_addr learned my addr 172.21.15.7:0/3470482201 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.722+0000 7fca59ef6640 1 --2- 172.21.15.7:0/3470482201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651804c8820 0x5651804c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.722+0000 7fca5a6f7640 1 -- 172.21.15.7:0/3470482201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651804c8820 msgr2=0x5651804c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.722+0000 7fca5a6f7640 1 --2- 172.21.15.7:0/3470482201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651804c8820 0x5651804c8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.722+0000 7fca5a6f7640 1 -- 172.21.15.7:0/3470482201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651804c9210 msgr2=0x5651804cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.722+0000 7fca5a6f7640 1 --2- 172.21.15.7:0/3470482201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651804c9210 0x5651804cdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.722+0000 7fca5a6f7640 1 -- 172.21.15.7:0/3470482201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651803dd9b0 con 0x5651804ce380 2024-06-10T01:29:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.722+0000 7fca5a6f7640 1 --2- 172.21.15.7:0/3470482201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651804ce380 0x5651804d0810 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fca4c035000 tx=0x7fca4c031810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf528bd18b6bb225 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.722+0000 7fca3ffff640 1 -- 172.21.15.7:0/3470482201 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c003970 con 0x5651804ce380 2024-06-10T01:29:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.722+0000 7fca3ffff640 1 -- 172.21.15.7:0/3470482201 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca4c036030 con 0x5651804ce380 2024-06-10T01:29:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.722+0000 7fca3ffff640 1 -- 172.21.15.7:0/3470482201 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c02f880 con 0x5651804ce380 2024-06-10T01:29:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.723+0000 7fca5ee68740 1 -- 172.21.15.7:0/3470482201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651804ce380 msgr2=0x5651804d0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.723+0000 7fca5ee68740 1 --2- 172.21.15.7:0/3470482201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651804ce380 0x5651804d0810 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fca4c035000 tx=0x7fca4c031810 comp rx=0 tx=0).stop 2024-06-10T01:29:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.723+0000 7fca5ee68740 1 -- 172.21.15.7:0/3470482201 shutdown_connections 2024-06-10T01:29:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.723+0000 7fca5ee68740 1 --2- 172.21.15.7:0/3470482201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651804ce380 0x5651804d0810 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.723+0000 7fca5ee68740 1 --2- 172.21.15.7:0/3470482201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651804c9210 0x5651804cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.723+0000 7fca5ee68740 1 --2- 172.21.15.7:0/3470482201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651804c8820 0x5651804c8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.723+0000 7fca5ee68740 1 -- 172.21.15.7:0/3470482201 >> 172.21.15.7:0/3470482201 conn(0x5651804bb680 msgr2=0x5651804bba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.724+0000 7fca5ee68740 1 -- 172.21.15.7:0/3470482201 shutdown_connections 2024-06-10T01:29:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.724+0000 7fca5ee68740 1 -- 172.21.15.7:0/3470482201 wait complete. 2024-06-10T01:29:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.724+0000 7fca5ee68740 1 -- start start 2024-06-10T01:29:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.724+0000 7fca5ee68740 1 Processor -- start 2024-06-10T01:29:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.724+0000 7fca5ee68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651804c8820 0x5651804d29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.724+0000 7fca5ee68740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651804c9210 0x5651804d2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.725+0000 7fca5ee68740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651804ce380 0x5651804d9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.725+0000 7fca5ee68740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5651804d12d0 con 0x5651804c9210 2024-06-10T01:29:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.725+0000 7fca5ee68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5651803acae0 con 0x5651804c8820 2024-06-10T01:29:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.725+0000 7fca5ee68740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5651804d1450 con 0x5651804ce380 2024-06-10T01:29:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.725+0000 7fca596f5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651804c9210 0x5651804d2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.725+0000 7fca596f5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651804c9210 0x5651804d2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54372/0 (socket says 172.21.15.7:54372) 2024-06-10T01:29:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.725+0000 7fca596f5640 1 -- 172.21.15.7:0/2116353552 learned_addr learned my addr 172.21.15.7:0/2116353552 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.725+0000 7fca5ee68740 10 client.0 _ll_register_callbacks cb 0x5651803e59b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:29:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.725+0000 7fca59ef6640 1 --2- 172.21.15.7:0/2116353552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651804c8820 0x5651804d29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.725+0000 7fca5a6f7640 1 --2- 172.21.15.7:0/2116353552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651804ce380 0x5651804d9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.725+0000 7fca596f5640 1 -- 172.21.15.7:0/2116353552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651804ce380 msgr2=0x5651804d9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.725+0000 7fca596f5640 1 --2- 172.21.15.7:0/2116353552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651804ce380 0x5651804d9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.725+0000 7fca596f5640 1 -- 172.21.15.7:0/2116353552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651804c8820 msgr2=0x5651804d29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.725+0000 7fca596f5640 1 --2- 172.21.15.7:0/2116353552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651804c8820 0x5651804d29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.725+0000 7fca596f5640 1 -- 172.21.15.7:0/2116353552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565180137210 con 0x5651804c9210 2024-06-10T01:29:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.725+0000 7fca59ef6640 1 --2- 172.21.15.7:0/2116353552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651804c8820 0x5651804d29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:29:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.725+0000 7fca596f5640 1 --2- 172.21.15.7:0/2116353552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651804c9210 0x5651804d2ee0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fca44003850 tx=0x7fca44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cab7b44c9a6b27a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.725+0000 7fca3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:29:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.726+0000 7fca3d7fa640 1 -- 172.21.15.7:0/2116353552 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4400e6a0 con 0x5651804c9210 2024-06-10T01:29:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.726+0000 7fca5ee68740 1 -- 172.21.15.7:0/2116353552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca4c036030 con 0x5651804c9210 2024-06-10T01:29:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.726+0000 7fca3d7fa640 1 -- 172.21.15.7:0/2116353552 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fca4400e840 con 0x5651804c9210 2024-06-10T01:29:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.726+0000 7fca3d7fa640 1 -- 172.21.15.7:0/2116353552 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fca4400ea40 con 0x5651804c9210 2024-06-10T01:29:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.726+0000 7fca5ee68740 10 client.20907 fetch_fsmap learned FSMap version 55 2024-06-10T01:29:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.726+0000 7fca5ee68740 1 -- 172.21.15.7:0/2116353552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x565180203620 con 0x5651804c9210 2024-06-10T01:29:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.726+0000 7fca3d7fa640 1 -- 172.21.15.7:0/2116353552 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca4c036030 con 0x5651804c9210 2024-06-10T01:29:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.727+0000 7fca3d7fa640 1 -- 172.21.15.7:0/2116353552 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fca4401d410 con 0x5651804c9210 2024-06-10T01:29:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.727+0000 7fca5ee68740 10 client.20907 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:29:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.727+0000 7fca5ee68740 10 client.20907 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:29:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.727+0000 7fca5ee68740 20 client.20907 populate_metadata read hostname 'smithi007' 2024-06-10T01:29:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.727+0000 7fca5ee68740 1 --2- 172.21.15.7:0/2116353552 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5651804de8c0 0x5651804e0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.727+0000 7fca5ee68740 4 client.20907 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:29:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.727+0000 7fca5ee68740 1 -- 172.21.15.7:0/2116353552 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565180436f10 con 0x5651804de8c0 2024-06-10T01:29:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.727+0000 7fca59ef6640 1 --2- 172.21.15.7:0/2116353552 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5651804de8c0 0x5651804e0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.728+0000 7fca59ef6640 1 --2- 172.21.15.7:0/2116353552 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5651804de8c0 0x5651804e0d50 crc :-1 s=READY pgs=515 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3567042b271d394 server_cookie=b47ff4cea7453587 in_seq=0 out_seq=0 2024-06-10T01:29:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.728+0000 7fca3d7fa640 10 client.20907 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.729+0000 7fca3d7fa640 1 -- 172.21.15.7:0/2116353552 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x565180436f10 con 0x5651804de8c0 2024-06-10T01:29:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.729+0000 7fca3d7fa640 10 client.20907 handle_command_reply: tid=0 2024-06-10T01:29:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.729+0000 7fca5ee68740 1 client.20907 shutdown 2024-06-10T01:29:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.730+0000 7fca5ee68740 1 -- 172.21.15.7:0/2116353552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651804c9210 msgr2=0x5651804d2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.730+0000 7fca5ee68740 1 --2- 172.21.15.7:0/2116353552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651804c9210 0x5651804d2ee0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fca44003850 tx=0x7fca44004730 comp rx=0 tx=0).stop 2024-06-10T01:29:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.730+0000 7fca5ee68740 1 -- 172.21.15.7:0/2116353552 shutdown_connections 2024-06-10T01:29:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.730+0000 7fca5ee68740 1 --2- 172.21.15.7:0/2116353552 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5651804de8c0 0x5651804e0d50 crc :-1 s=READY pgs=515 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.730+0000 7fca5ee68740 1 --2- 172.21.15.7:0/2116353552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651804ce380 0x5651804d9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.730+0000 7fca3d7fa640 0 client.20907 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.730+0000 7fca5ee68740 1 --2- 172.21.15.7:0/2116353552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651804c9210 0x5651804d2ee0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.730+0000 7fca5ee68740 1 --2- 172.21.15.7:0/2116353552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651804c8820 0x5651804d29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.730+0000 7fca5ee68740 1 -- 172.21.15.7:0/2116353552 >> 172.21.15.7:0/2116353552 conn(0x5651804bb680 msgr2=0x5651804ba7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.730+0000 7fca5ee68740 1 -- 172.21.15.7:0/2116353552 shutdown_connections 2024-06-10T01:29:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.730+0000 7fca5ee68740 1 -- 172.21.15.7:0/2116353552 wait complete. 2024-06-10T01:29:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.730+0000 7fca5ee68740 20 client.20907 trim_cache size 0 max 16384 2024-06-10T01:29:07.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.752+0000 7fca5ee68740 1 Processor -- start 2024-06-10T01:29:07.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.752+0000 7fca5ee68740 1 -- start start 2024-06-10T01:29:07.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.752+0000 7fca5ee68740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651804c8820 0x5651804cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.752+0000 7fca5ee68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651804c9210 0x5651804d9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.752+0000 7fca5ee68740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651804ce380 0x5651804ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.752+0000 7fca5ee68740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fca541b32c0 con 0x5651804ce380 2024-06-10T01:29:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.752+0000 7fca5ee68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fca541b0b90 con 0x5651804c9210 2024-06-10T01:29:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.752+0000 7fca5ee68740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fca541ae9e0 con 0x5651804c8820 2024-06-10T01:29:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.752+0000 7fca5a6f7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651804ce380 0x5651804ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.752+0000 7fca596f5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651804c9210 0x5651804d9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.752+0000 7fca59ef6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651804c8820 0x5651804cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.752+0000 7fca596f5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651804c9210 0x5651804d9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:54344/0 (socket says 172.21.15.7:54344) 2024-06-10T01:29:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.752+0000 7fca5a6f7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651804ce380 0x5651804ca3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54382/0 (socket says 172.21.15.7:54382) 2024-06-10T01:29:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.752+0000 7fca596f5640 1 -- 172.21.15.7:0/56591279 learned_addr learned my addr 172.21.15.7:0/56591279 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.753+0000 7fca5a6f7640 1 -- 172.21.15.7:0/56591279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651804c8820 msgr2=0x5651804cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.753+0000 7fca5a6f7640 1 --2- 172.21.15.7:0/56591279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651804c8820 0x5651804cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.753+0000 7fca5a6f7640 1 -- 172.21.15.7:0/56591279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651804c9210 msgr2=0x5651804d9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.753+0000 7fca5a6f7640 1 --2- 172.21.15.7:0/56591279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651804c9210 0x5651804d9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.753+0000 7fca5a6f7640 1 -- 172.21.15.7:0/56591279 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651804c9ea0 con 0x5651804ce380 2024-06-10T01:29:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.753+0000 7fca596f5640 1 --2- 172.21.15.7:0/56591279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651804c9210 0x5651804d9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:29:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.753+0000 7fca5a6f7640 1 --2- 172.21.15.7:0/56591279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651804ce380 0x5651804ca3b0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fca4c009bf0 tx=0x7fca4c02a8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc79c8a28ce2991d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.753+0000 7fca3f7fe640 1 -- 172.21.15.7:0/56591279 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c054070 con 0x5651804ce380 2024-06-10T01:29:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.753+0000 7fca3f7fe640 1 -- 172.21.15.7:0/56591279 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca4c036050 con 0x5651804ce380 2024-06-10T01:29:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.753+0000 7fca3f7fe640 1 -- 172.21.15.7:0/56591279 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c004800 con 0x5651804ce380 2024-06-10T01:29:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.754+0000 7fca5ee68740 1 -- 172.21.15.7:0/56591279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651804ce380 msgr2=0x5651804ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.754+0000 7fca5ee68740 1 --2- 172.21.15.7:0/56591279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651804ce380 0x5651804ca3b0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fca4c009bf0 tx=0x7fca4c02a8d0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.754+0000 7fca5ee68740 1 -- 172.21.15.7:0/56591279 shutdown_connections 2024-06-10T01:29:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.754+0000 7fca5ee68740 1 --2- 172.21.15.7:0/56591279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651804ce380 0x5651804ca3b0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.754+0000 7fca5ee68740 1 --2- 172.21.15.7:0/56591279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651804c9210 0x5651804d9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.754+0000 7fca5ee68740 1 --2- 172.21.15.7:0/56591279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651804c8820 0x5651804cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.754+0000 7fca5ee68740 1 -- 172.21.15.7:0/56591279 >> 172.21.15.7:0/56591279 conn(0x5651804bb680 msgr2=0x5651804bba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.754+0000 7fca5ee68740 1 -- 172.21.15.7:0/56591279 shutdown_connections 2024-06-10T01:29:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.754+0000 7fca5ee68740 1 -- 172.21.15.7:0/56591279 wait complete. 2024-06-10T01:29:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.754+0000 7fca5ee68740 1 -- start start 2024-06-10T01:29:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.754+0000 7fca5ee68740 1 Processor -- start 2024-06-10T01:29:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.755+0000 7fca5ee68740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651804c8820 0x5651804d6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.755+0000 7fca5ee68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651804c9210 0x5651804d6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.755+0000 7fca5ee68740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651804ce380 0x565180547620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.755+0000 7fca5ee68740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fca541af050 con 0x5651804c8820 2024-06-10T01:29:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.755+0000 7fca5ee68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fca541b04c0 con 0x5651804c9210 2024-06-10T01:29:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.755+0000 7fca5ee68740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fca541af720 con 0x5651804ce380 2024-06-10T01:29:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.755+0000 7fca5ee68740 10 client.0 _ll_register_callbacks cb 0x7fca44004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:29:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.755+0000 7fca59ef6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651804c8820 0x5651804d6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.755+0000 7fca596f5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651804c9210 0x5651804d6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.755+0000 7fca59ef6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651804c8820 0x5651804d6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54386/0 (socket says 172.21.15.7:54386) 2024-06-10T01:29:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.755+0000 7fca59ef6640 1 -- 172.21.15.7:0/3251315367 learned_addr learned my addr 172.21.15.7:0/3251315367 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.755+0000 7fca5a6f7640 1 --2- 172.21.15.7:0/3251315367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651804ce380 0x565180547620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.756+0000 7fca596f5640 1 -- 172.21.15.7:0/3251315367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651804ce380 msgr2=0x565180547620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.756+0000 7fca596f5640 1 --2- 172.21.15.7:0/3251315367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651804ce380 0x565180547620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.756+0000 7fca596f5640 1 -- 172.21.15.7:0/3251315367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651804c8820 msgr2=0x5651804d6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.756+0000 7fca596f5640 1 --2- 172.21.15.7:0/3251315367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651804c8820 0x5651804d6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.756+0000 7fca596f5640 1 -- 172.21.15.7:0/3251315367 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651804da080 con 0x5651804c9210 2024-06-10T01:29:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.756+0000 7fca5a6f7640 1 --2- 172.21.15.7:0/3251315367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651804ce380 0x565180547620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:29:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.756+0000 7fca596f5640 1 --2- 172.21.15.7:0/3251315367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651804c9210 0x5651804d6fb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fca440092e0 tx=0x7fca44014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec4b1c86bd52d6e4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.756+0000 7fca3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:29:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.756+0000 7fca3d7fa640 1 -- 172.21.15.7:0/3251315367 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca44039070 con 0x5651804c9210 2024-06-10T01:29:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.756+0000 7fca5ee68740 1 -- 172.21.15.7:0/3251315367 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca4c036050 con 0x5651804c9210 2024-06-10T01:29:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.756+0000 7fca3d7fa640 1 -- 172.21.15.7:0/3251315367 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fca44009040 con 0x5651804c9210 2024-06-10T01:29:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.756+0000 7fca3d7fa640 1 -- 172.21.15.7:0/3251315367 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fca44045050 con 0x5651804c9210 2024-06-10T01:29:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.756+0000 7fca5ee68740 10 client.30098 fetch_fsmap learned FSMap version 55 2024-06-10T01:29:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.756+0000 7fca5ee68740 1 -- 172.21.15.7:0/3251315367 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x565180548a20 con 0x5651804c9210 2024-06-10T01:29:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.756+0000 7fca3d7fa640 1 -- 172.21.15.7:0/3251315367 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca4c036050 con 0x5651804c9210 2024-06-10T01:29:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.757+0000 7fca3d7fa640 1 -- 172.21.15.7:0/3251315367 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fca440359c0 con 0x5651804c9210 2024-06-10T01:29:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.757+0000 7fca5ee68740 10 client.30098 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:29:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.757+0000 7fca5ee68740 10 client.30098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:29:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.757+0000 7fca5ee68740 20 client.30098 populate_metadata read hostname 'smithi007' 2024-06-10T01:29:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.757+0000 7fca5ee68740 1 --2- 172.21.15.7:0/3251315367 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5651804de8c0 0x56518054b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.757+0000 7fca5ee68740 4 client.30098 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:29:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.757+0000 7fca5ee68740 1 -- 172.21.15.7:0/3251315367 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565180430c20 con 0x5651804de8c0 2024-06-10T01:29:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.757+0000 7fca59ef6640 1 --2- 172.21.15.7:0/3251315367 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5651804de8c0 0x56518054b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.757+0000 7fca59ef6640 1 --2- 172.21.15.7:0/3251315367 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5651804de8c0 0x56518054b3b0 crc :-1 s=READY pgs=516 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bcc34de081646ee server_cookie=73d06eb822e9e83a in_seq=0 out_seq=0 2024-06-10T01:29:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:07.757+0000 7fca3d7fa640 10 client.30098 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:07.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:07 smithi175 ceph-mon[36636]: pgmap v1127: 129 pgs: 129 active+clean; 699 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 5.9 MiB/s wr, 591 op/s 2024-06-10T01:29:07.959 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:07 smithi007 ceph-mon[32398]: pgmap v1127: 129 pgs: 129 active+clean; 699 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 5.9 MiB/s wr, 591 op/s 2024-06-10T01:29:08.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:07 smithi078 ceph-mon[36683]: pgmap v1127: 129 pgs: 129 active+clean; 699 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 5.9 MiB/s wr, 591 op/s 2024-06-10T01:29:08.245 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:29:08.562 INFO:tasks.workunit.client.0.smithi007.stdout: 10 236302 22.67 MB/sec execute 465 sec latency 709.718 ms 2024-06-10T01:29:08.790 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:29:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:08.975+0000 7fca3d7fa640 1 -- 172.21.15.7:0/3251315367 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x565180430c20 con 0x5651804de8c0 2024-06-10T01:29:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:08.975+0000 7fca3d7fa640 10 client.30098 handle_command_reply: tid=0 2024-06-10T01:29:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:08.975+0000 7fca5ee68740 1 client.30098 shutdown 2024-06-10T01:29:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:08.975+0000 7fca5ee68740 1 -- 172.21.15.7:0/3251315367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651804c9210 msgr2=0x5651804d6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:08.975+0000 7fca5ee68740 1 --2- 172.21.15.7:0/3251315367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651804c9210 0x5651804d6fb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fca440092e0 tx=0x7fca44014000 comp rx=0 tx=0).stop 2024-06-10T01:29:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:08.975+0000 7fca5ee68740 1 -- 172.21.15.7:0/3251315367 shutdown_connections 2024-06-10T01:29:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:08.975+0000 7fca5ee68740 1 --2- 172.21.15.7:0/3251315367 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5651804de8c0 0x56518054b3b0 crc :-1 s=READY pgs=516 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:08.975+0000 7fca5ee68740 1 --2- 172.21.15.7:0/3251315367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651804ce380 0x565180547620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:08.975+0000 7fca5ee68740 1 --2- 172.21.15.7:0/3251315367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651804c9210 0x5651804d6fb0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:08.975+0000 7fca3d7fa640 0 client.30098 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:08.975+0000 7fca5ee68740 1 --2- 172.21.15.7:0/3251315367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651804c8820 0x5651804d6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:08.975+0000 7fca5ee68740 1 -- 172.21.15.7:0/3251315367 >> 172.21.15.7:0/3251315367 conn(0x5651804bb680 msgr2=0x5651804d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:08.975+0000 7fca5ee68740 1 -- 172.21.15.7:0/3251315367 shutdown_connections 2024-06-10T01:29:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:08.976+0000 7fca5ee68740 1 -- 172.21.15.7:0/3251315367 wait complete. 2024-06-10T01:29:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:08.976+0000 7fca5ee68740 20 client.30098 trim_cache size 0 max 16384 2024-06-10T01:29:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:29:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:29:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:29:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:08.976+0000 7fca5c15a640 1 -- 172.21.15.7:0/492249329 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fca400a2d10 msgr2=0x7fca400a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:08.976+0000 7fca5c15a640 1 --2- 172.21.15.7:0/492249329 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fca400a2d10 0x7fca400a51d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fca541a3bc0 tx=0x7fca44005f90 comp rx=0 tx=0).stop 2024-06-10T01:29:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:08.976+0000 7fca5c15a640 1 -- 172.21.15.7:0/492249329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca541089a0 msgr2=0x7fca541a2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:08.976+0000 7fca5c15a640 1 --2- 172.21.15.7:0/492249329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca541089a0 0x7fca541a2700 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fca5000b450 tx=0x7fca5000b910 comp rx=0 tx=0).stop 2024-06-10T01:29:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:08.976+0000 7fca5c15a640 1 -- 172.21.15.7:0/492249329 shutdown_connections 2024-06-10T01:29:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:08.976+0000 7fca5c15a640 1 --2- 172.21.15.7:0/492249329 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fca400a2d10 0x7fca400a51d0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:08.976+0000 7fca5c15a640 1 --2- 172.21.15.7:0/492249329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca541120e0 0x7fca541a9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:08.976+0000 7fca5c15a640 1 --2- 172.21.15.7:0/492249329 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca5410b310 0x7fca541a2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:08.976+0000 7fca5c15a640 1 --2- 172.21.15.7:0/492249329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca541089a0 0x7fca541a2700 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:08.976+0000 7fca5c15a640 1 -- 172.21.15.7:0/492249329 >> 172.21.15.7:0/492249329 conn(0x7fca5406e9e0 msgr2=0x7fca5410ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:08.976+0000 7fca5c15a640 1 -- 172.21.15.7:0/492249329 shutdown_connections 2024-06-10T01:29:08.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:08.976+0000 7fca5c15a640 1 -- 172.21.15.7:0/492249329 wait complete. 2024-06-10T01:29:09.562 INFO:tasks.workunit.client.0.smithi007.stdout: 10 236698 22.68 MB/sec execute 466 sec latency 724.858 ms 2024-06-10T01:29:09.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:09 smithi175 ceph-mon[36636]: pgmap v1128: 129 pgs: 129 active+clean; 698 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.9 MiB/s wr, 600 op/s 2024-06-10T01:29:09.976 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:09 smithi007 ceph-mon[32398]: pgmap v1128: 129 pgs: 129 active+clean; 698 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.9 MiB/s wr, 600 op/s 2024-06-10T01:29:10.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:09 smithi078 ceph-mon[36683]: pgmap v1128: 129 pgs: 129 active+clean; 698 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.9 MiB/s wr, 600 op/s 2024-06-10T01:29:10.562 INFO:tasks.workunit.client.0.smithi007.stdout: 10 237043 22.67 MB/sec execute 467 sec latency 213.480 ms 2024-06-10T01:29:10.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.894+0000 7fc30b192700 1 -- 172.21.15.7:0/1989400468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc304101390 msgr2=0x7fc304101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:10.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.894+0000 7fc30b192700 1 --2- 172.21.15.7:0/1989400468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc304101390 0x7fc304101810 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4009f90 tx=0x7fc2f400b330 comp rx=0 tx=0).stop 2024-06-10T01:29:10.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.894+0000 7fc30b192700 1 -- 172.21.15.7:0/1989400468 shutdown_connections 2024-06-10T01:29:10.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.894+0000 7fc30b192700 1 --2- 172.21.15.7:0/1989400468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc304101d50 0x7fc30410e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.894+0000 7fc30b192700 1 --2- 172.21.15.7:0/1989400468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc304107400 0x7fc3041077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.894+0000 7fc30b192700 1 --2- 172.21.15.7:0/1989400468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc304101390 0x7fc304101810 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.894+0000 7fc30b192700 1 -- 172.21.15.7:0/1989400468 >> 172.21.15.7:0/1989400468 conn(0x7fc304078340 msgr2=0x7fc304078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:10.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.895+0000 7fc30b192700 1 -- 172.21.15.7:0/1989400468 shutdown_connections 2024-06-10T01:29:10.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.895+0000 7fc30b192700 1 -- 172.21.15.7:0/1989400468 wait complete. 2024-06-10T01:29:10.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.896+0000 7fc30b192700 1 Processor -- start 2024-06-10T01:29:10.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.896+0000 7fc30b192700 1 -- start start 2024-06-10T01:29:10.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.896+0000 7fc30b192700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc304101390 0x7fc30419d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:10.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.896+0000 7fc30b192700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc304101d50 0x7fc30419dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:10.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.896+0000 7fc30b192700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc304107400 0x7fc304197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:10.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.897+0000 7fc308f2e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc304101390 0x7fc30419d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:10.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.897+0000 7fc303fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc304101d50 0x7fc30419dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:10.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.897+0000 7fc30b192700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc304111740 con 0x7fc304107400 2024-06-10T01:29:10.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.897+0000 7fc30b192700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc3041115c0 con 0x7fc304101d50 2024-06-10T01:29:10.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.897+0000 7fc30b192700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc3041118c0 con 0x7fc304101390 2024-06-10T01:29:10.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.897+0000 7fc303fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc304101d50 0x7fc30419dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:54358/0 (socket says 172.21.15.7:54358) 2024-06-10T01:29:10.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.897+0000 7fc303fff700 1 -- 172.21.15.7:0/2762590281 learned_addr learned my addr 172.21.15.7:0/2762590281 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:10.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.897+0000 7fc308f2e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc304101390 0x7fc30419d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36456/0 (socket says 172.21.15.7:36456) 2024-06-10T01:29:10.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.897+0000 7fc30972f700 1 --2- 172.21.15.7:0/2762590281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc304107400 0x7fc304197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:10.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.897+0000 7fc303fff700 1 -- 172.21.15.7:0/2762590281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc304101390 msgr2=0x7fc30419d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:10.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.898+0000 7fc303fff700 1 --2- 172.21.15.7:0/2762590281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc304101390 0x7fc30419d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.898+0000 7fc303fff700 1 -- 172.21.15.7:0/2762590281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc304107400 msgr2=0x7fc304197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:10.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.898+0000 7fc303fff700 1 --2- 172.21.15.7:0/2762590281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc304107400 0x7fc304197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.898+0000 7fc303fff700 1 -- 172.21.15.7:0/2762590281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc304198160 con 0x7fc304101d50 2024-06-10T01:29:10.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.898+0000 7fc303fff700 1 --2- 172.21.15.7:0/2762590281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc304101d50 0x7fc30419dd30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc2f400cfa0 tx=0x7fc2f400cfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:10.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.898+0000 7fc301ffb700 1 -- 172.21.15.7:0/2762590281 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4021070 con 0x7fc304101d50 2024-06-10T01:29:10.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.898+0000 7fc301ffb700 1 -- 172.21.15.7:0/2762590281 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f4007ac0 con 0x7fc304101d50 2024-06-10T01:29:10.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.898+0000 7fc301ffb700 1 -- 172.21.15.7:0/2762590281 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f40052a0 con 0x7fc304101d50 2024-06-10T01:29:10.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.898+0000 7fc30b192700 1 -- 172.21.15.7:0/2762590281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3041983f0 con 0x7fc304101d50 2024-06-10T01:29:10.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.899+0000 7fc30b192700 1 -- 172.21.15.7:0/2762590281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc30406a170 con 0x7fc304101d50 2024-06-10T01:29:10.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.903+0000 7fc301ffb700 1 -- 172.21.15.7:0/2762590281 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fc2f4024070 con 0x7fc304101d50 2024-06-10T01:29:10.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.905+0000 7fc315a15b80 1 Processor -- start 2024-06-10T01:29:10.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.905+0000 7fc301ffb700 1 --2- 172.21.15.7:0/2762590281 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc2f00a4f00 0x7fc2f00a73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:10.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.905+0000 7fc301ffb700 1 -- 172.21.15.7:0/2762590281 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fc2f400f020 con 0x7fc304101d50 2024-06-10T01:29:10.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.906+0000 7fc315a15b80 1 -- start start 2024-06-10T01:29:10.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.906+0000 7fc30972f700 1 --2- 172.21.15.7:0/2762590281 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc2f00a4f00 0x7fc2f00a73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:10.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.906+0000 7fc315a15b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56083071e000 0x56083071a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:10.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.906+0000 7fc315a15b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56083071a7a0 0x56083071ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:10.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.906+0000 7fc315a15b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56083072cec0 0x56083072f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:10.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.906+0000 7fc315a15b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56083060cf60 con 0x56083072cec0 2024-06-10T01:29:10.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.906+0000 7fc315a15b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56083066b850 con 0x56083071a7a0 2024-06-10T01:29:10.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.906+0000 7fc315a15b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5608305d3460 con 0x56083071e000 2024-06-10T01:29:10.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.907+0000 7fc303fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56083071a7a0 0x56083071ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:10.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.907+0000 7fc308f2e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56083071e000 0x56083071a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:10.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.907+0000 7fc303fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56083071a7a0 0x56083071ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:54362/0 (socket says 172.21.15.7:54362) 2024-06-10T01:29:10.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.907+0000 7fc303fff700 1 -- 172.21.15.7:0/26829711 learned_addr learned my addr 172.21.15.7:0/26829711 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.907+0000 7fc30972f700 1 --2- 172.21.15.7:0/2762590281 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc2f00a4f00 0x7fc2f00a73c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009c40 tx=0x7fc2ec0113e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.907+0000 7fc30972f700 1 --2- 172.21.15.7:0/26829711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56083072cec0 0x56083072f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.907+0000 7fc303fff700 1 -- 172.21.15.7:0/26829711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56083071e000 msgr2=0x56083071a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.907+0000 7fc303fff700 1 --2- 172.21.15.7:0/26829711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56083071e000 0x56083071a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.907+0000 7fc303fff700 1 -- 172.21.15.7:0/26829711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56083072cec0 msgr2=0x56083072f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.907+0000 7fc303fff700 1 --2- 172.21.15.7:0/26829711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56083072cec0 0x56083072f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.907+0000 7fc303fff700 1 -- 172.21.15.7:0/26829711 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608305ba8d0 con 0x56083071a7a0 2024-06-10T01:29:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.908+0000 7fc303fff700 1 --2- 172.21.15.7:0/26829711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56083071a7a0 0x56083071ac20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc2f40a5370 tx=0x7fc2f401dc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a2743e3e7180adb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:10.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.908+0000 7fc2faffd700 1 -- 172.21.15.7:0/26829711 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f409ccc0 con 0x56083071a7a0 2024-06-10T01:29:10.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.908+0000 7fc2faffd700 1 -- 172.21.15.7:0/26829711 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f40a1070 con 0x56083071a7a0 2024-06-10T01:29:10.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.909+0000 7fc2faffd700 1 -- 172.21.15.7:0/26829711 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f409d180 con 0x56083071a7a0 2024-06-10T01:29:10.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.909+0000 7fc315a15b80 1 -- 172.21.15.7:0/26829711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56083071a7a0 msgr2=0x56083071ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:10.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.909+0000 7fc315a15b80 1 --2- 172.21.15.7:0/26829711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56083071a7a0 0x56083071ac20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc2f40a5370 tx=0x7fc2f401dc20 comp rx=0 tx=0).stop 2024-06-10T01:29:10.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.909+0000 7fc315a15b80 1 -- 172.21.15.7:0/26829711 shutdown_connections 2024-06-10T01:29:10.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.909+0000 7fc315a15b80 1 --2- 172.21.15.7:0/26829711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56083072cec0 0x56083072f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.909+0000 7fc315a15b80 1 --2- 172.21.15.7:0/26829711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56083071e000 0x56083071a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.909+0000 7fc315a15b80 1 --2- 172.21.15.7:0/26829711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56083071a7a0 0x56083071ac20 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.909+0000 7fc315a15b80 1 -- 172.21.15.7:0/26829711 >> 172.21.15.7:0/26829711 conn(0x560830647080 msgr2=0x560830715220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:10.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.910+0000 7fc315a15b80 1 -- 172.21.15.7:0/26829711 shutdown_connections 2024-06-10T01:29:10.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.910+0000 7fc315a15b80 1 -- 172.21.15.7:0/26829711 wait complete. 2024-06-10T01:29:10.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.910+0000 7fc315a15b80 1 -- start start 2024-06-10T01:29:10.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.910+0000 7fc315a15b80 1 Processor -- start 2024-06-10T01:29:10.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.911+0000 7fc315a15b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56083071a7a0 0x56083072c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.911+0000 7fc315a15b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56083071e000 0x5608307299e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.911+0000 7fc315a15b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56083072cec0 0x560830729f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.911+0000 7fc315a15b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56083060bfc0 con 0x56083071e000 2024-06-10T01:29:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.911+0000 7fc315a15b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560830667810 con 0x56083072cec0 2024-06-10T01:29:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.911+0000 7fc315a15b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56083072f850 con 0x56083071a7a0 2024-06-10T01:29:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.911+0000 7fc315a15b80 10 client.0 _ll_register_callbacks cb 0x5608306015b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:29:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.911+0000 7fc303fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56083071e000 0x5608307299e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.911+0000 7fc303fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56083071e000 0x5608307299e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54396/0 (socket says 172.21.15.7:54396) 2024-06-10T01:29:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.911+0000 7fc303fff700 1 -- 172.21.15.7:0/3535892004 learned_addr learned my addr 172.21.15.7:0/3535892004 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.911+0000 7fc30972f700 1 --2- 172.21.15.7:0/3535892004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56083072cec0 0x560830729f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.911+0000 7fc308f2e700 1 --2- 172.21.15.7:0/3535892004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56083071a7a0 0x56083072c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.911+0000 7fc303fff700 1 -- 172.21.15.7:0/3535892004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56083071a7a0 msgr2=0x56083072c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.911+0000 7fc303fff700 1 --2- 172.21.15.7:0/3535892004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56083071a7a0 0x56083072c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.911+0000 7fc303fff700 1 -- 172.21.15.7:0/3535892004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56083072cec0 msgr2=0x560830729f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:10.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.911+0000 7fc303fff700 1 --2- 172.21.15.7:0/3535892004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56083072cec0 0x560830729f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.912+0000 7fc303fff700 1 -- 172.21.15.7:0/3535892004 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560830631750 con 0x56083071e000 2024-06-10T01:29:10.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.912+0000 7fc308f2e700 1 --2- 172.21.15.7:0/3535892004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56083071a7a0 0x56083072c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:29:10.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.912+0000 7fc303fff700 1 --2- 172.21.15.7:0/3535892004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56083071e000 0x5608307299e0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4099690 tx=0x7fc2f409d4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65abda9ad48bf0b1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:10.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.912+0000 7fc2e3fff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:29:10.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.912+0000 7fc2e3fff700 1 -- 172.21.15.7:0/3535892004 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4007c70 con 0x56083071e000 2024-06-10T01:29:10.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.912+0000 7fc315a15b80 1 -- 172.21.15.7:0/3535892004 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2f40a1070 con 0x56083071e000 2024-06-10T01:29:10.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.912+0000 7fc2e3fff700 1 -- 172.21.15.7:0/3535892004 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fc2f40a5410 con 0x56083071e000 2024-06-10T01:29:10.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.913+0000 7fc2e3fff700 1 -- 172.21.15.7:0/3535892004 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fc2f40a5610 con 0x56083071e000 2024-06-10T01:29:10.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.913+0000 7fc315a15b80 10 client.20925 fetch_fsmap learned FSMap version 55 2024-06-10T01:29:10.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.913+0000 7fc315a15b80 1 -- 172.21.15.7:0/3535892004 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x560830739fe0 con 0x56083071e000 2024-06-10T01:29:10.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.913+0000 7fc2e3fff700 1 -- 172.21.15.7:0/3535892004 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f40a1070 con 0x56083071e000 2024-06-10T01:29:10.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.914+0000 7fc2e3fff700 1 -- 172.21.15.7:0/3535892004 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fc2f4011070 con 0x56083071e000 2024-06-10T01:29:10.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.914+0000 7fc315a15b80 10 client.20925 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:29:10.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.914+0000 7fc315a15b80 10 client.20925 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:29:10.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.914+0000 7fc315a15b80 20 client.20925 populate_metadata read hostname 'smithi007' 2024-06-10T01:29:10.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.914+0000 7fc315a15b80 1 --2- 172.21.15.7:0/3535892004 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56083073c380 0x56083073e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:10.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.914+0000 7fc315a15b80 4 client.20925 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:29:10.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.914+0000 7fc315a15b80 1 -- 172.21.15.7:0/3535892004 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560830678d50 con 0x56083073c380 2024-06-10T01:29:10.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.915+0000 7fc308f2e700 1 --2- 172.21.15.7:0/3535892004 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56083073c380 0x56083073e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:10.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.916+0000 7fc2e3fff700 10 client.20925 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:10.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.916+0000 7fc308f2e700 1 --2- 172.21.15.7:0/3535892004 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56083073c380 0x56083073e790 crc :-1 s=READY pgs=517 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8f7ff32ff4a3c60 server_cookie=24e72d84771e9ef4 in_seq=0 out_seq=0 2024-06-10T01:29:10.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.918+0000 7fc2e3fff700 1 -- 172.21.15.7:0/3535892004 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560830678d50 con 0x56083073c380 2024-06-10T01:29:10.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.918+0000 7fc2e3fff700 10 client.20925 handle_command_reply: tid=0 2024-06-10T01:29:10.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.918+0000 7fc315a15b80 1 client.20925 shutdown 2024-06-10T01:29:10.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.919+0000 7fc315a15b80 1 -- 172.21.15.7:0/3535892004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56083071e000 msgr2=0x5608307299e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:10.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.919+0000 7fc315a15b80 1 --2- 172.21.15.7:0/3535892004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56083071e000 0x5608307299e0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4099690 tx=0x7fc2f409d4c0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.919+0000 7fc315a15b80 1 -- 172.21.15.7:0/3535892004 shutdown_connections 2024-06-10T01:29:10.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.919+0000 7fc315a15b80 1 --2- 172.21.15.7:0/3535892004 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56083073c380 0x56083073e790 crc :-1 s=READY pgs=517 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.919+0000 7fc315a15b80 1 --2- 172.21.15.7:0/3535892004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56083072cec0 0x560830729f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.919+0000 7fc315a15b80 1 --2- 172.21.15.7:0/3535892004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56083071a7a0 0x56083072c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.919+0000 7fc315a15b80 1 --2- 172.21.15.7:0/3535892004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56083071e000 0x5608307299e0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.919+0000 7fc315a15b80 1 -- 172.21.15.7:0/3535892004 >> 172.21.15.7:0/3535892004 conn(0x560830647080 msgr2=0x560830714700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:10.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.919+0000 7fc2e3fff700 0 client.20925 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:10.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.919+0000 7fc315a15b80 1 -- 172.21.15.7:0/3535892004 shutdown_connections 2024-06-10T01:29:10.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.919+0000 7fc315a15b80 1 -- 172.21.15.7:0/3535892004 wait complete. 2024-06-10T01:29:10.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.920+0000 7fc315a15b80 20 client.20925 trim_cache size 0 max 16384 2024-06-10T01:29:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.942+0000 7fc315a15b80 1 Processor -- start 2024-06-10T01:29:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.942+0000 7fc315a15b80 1 -- start start 2024-06-10T01:29:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.943+0000 7fc315a15b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56083071a7a0 0x56083072a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.943+0000 7fc315a15b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56083071e000 0x560830714e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.943+0000 7fc315a15b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56083072cec0 0x5608307153c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.943+0000 7fc315a15b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc3041ae760 con 0x56083071a7a0 2024-06-10T01:29:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.943+0000 7fc315a15b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc3041abff0 con 0x56083072cec0 2024-06-10T01:29:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.943+0000 7fc315a15b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc3041a9ef0 con 0x56083071e000 2024-06-10T01:29:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.943+0000 7fc308f2e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56083071a7a0 0x56083072a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.943+0000 7fc303fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56083071e000 0x560830714e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.943+0000 7fc303fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56083071e000 0x560830714e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36494/0 (socket says 172.21.15.7:36494) 2024-06-10T01:29:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.943+0000 7fc303fff700 1 -- 172.21.15.7:0/3425627020 learned_addr learned my addr 172.21.15.7:0/3425627020 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.943+0000 7fc30972f700 1 --2- 172.21.15.7:0/3425627020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56083072cec0 0x5608307153c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.944+0000 7fc303fff700 1 -- 172.21.15.7:0/3425627020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56083072cec0 msgr2=0x5608307153c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.944+0000 7fc303fff700 1 --2- 172.21.15.7:0/3425627020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56083072cec0 0x5608307153c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.944+0000 7fc303fff700 1 -- 172.21.15.7:0/3425627020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56083071a7a0 msgr2=0x56083072a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.944+0000 7fc303fff700 1 --2- 172.21.15.7:0/3425627020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56083071a7a0 0x56083072a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.944+0000 7fc303fff700 1 -- 172.21.15.7:0/3425627020 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56083073ee10 con 0x56083071e000 2024-06-10T01:29:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.944+0000 7fc30972f700 1 --2- 172.21.15.7:0/3425627020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56083072cec0 0x5608307153c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:29:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.944+0000 7fc303fff700 1 --2- 172.21.15.7:0/3425627020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56083071e000 0x560830714e80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc2f409af70 tx=0x7fc2f401dc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ca8d1096bc6115c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.944+0000 7fc2fa7fc700 1 -- 172.21.15.7:0/3425627020 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f40a58a0 con 0x56083071e000 2024-06-10T01:29:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.944+0000 7fc2fa7fc700 1 -- 172.21.15.7:0/3425627020 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f4007420 con 0x56083071e000 2024-06-10T01:29:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.944+0000 7fc2fa7fc700 1 -- 172.21.15.7:0/3425627020 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f40a5b70 con 0x56083071e000 2024-06-10T01:29:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.945+0000 7fc315a15b80 1 -- 172.21.15.7:0/3425627020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56083071e000 msgr2=0x560830714e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.945+0000 7fc315a15b80 1 --2- 172.21.15.7:0/3425627020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56083071e000 0x560830714e80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc2f409af70 tx=0x7fc2f401dc20 comp rx=0 tx=0).stop 2024-06-10T01:29:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.945+0000 7fc315a15b80 1 -- 172.21.15.7:0/3425627020 shutdown_connections 2024-06-10T01:29:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.945+0000 7fc315a15b80 1 --2- 172.21.15.7:0/3425627020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56083072cec0 0x5608307153c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.945+0000 7fc315a15b80 1 --2- 172.21.15.7:0/3425627020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56083071a7a0 0x56083072a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.945+0000 7fc315a15b80 1 --2- 172.21.15.7:0/3425627020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56083071e000 0x560830714e80 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.945+0000 7fc315a15b80 1 -- 172.21.15.7:0/3425627020 >> 172.21.15.7:0/3425627020 conn(0x560830647080 msgr2=0x5608307299a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.945+0000 7fc315a15b80 1 -- 172.21.15.7:0/3425627020 shutdown_connections 2024-06-10T01:29:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.945+0000 7fc315a15b80 1 -- 172.21.15.7:0/3425627020 wait complete. 2024-06-10T01:29:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.946+0000 7fc315a15b80 1 -- start start 2024-06-10T01:29:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.946+0000 7fc315a15b80 1 Processor -- start 2024-06-10T01:29:10.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.947+0000 7fc315a15b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56083071a7a0 0x560830734aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:10.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.947+0000 7fc315a15b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56083071e000 0x560830734fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:10.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.947+0000 7fc315a15b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56083072cec0 0x560830717380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:10.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.947+0000 7fc315a15b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc3041aa4b0 con 0x56083071a7a0 2024-06-10T01:29:10.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.947+0000 7fc315a15b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc3041ab920 con 0x56083072cec0 2024-06-10T01:29:10.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.947+0000 7fc315a15b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc3041aab80 con 0x56083071e000 2024-06-10T01:29:10.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.947+0000 7fc308f2e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56083071a7a0 0x560830734aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:10.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.947+0000 7fc308f2e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56083071a7a0 0x560830734aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54422/0 (socket says 172.21.15.7:54422) 2024-06-10T01:29:10.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.947+0000 7fc308f2e700 1 -- 172.21.15.7:0/3984554955 learned_addr learned my addr 172.21.15.7:0/3984554955 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:10.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.947+0000 7fc30972f700 1 --2- 172.21.15.7:0/3984554955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56083072cec0 0x560830717380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:10.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.947+0000 7fc303fff700 1 --2- 172.21.15.7:0/3984554955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56083071e000 0x560830734fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:10.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.947+0000 7fc315a15b80 10 client.0 _ll_register_callbacks cb 0x5608306015b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:29:10.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.947+0000 7fc30972f700 1 -- 172.21.15.7:0/3984554955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56083071e000 msgr2=0x560830734fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:10.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.947+0000 7fc30972f700 1 --2- 172.21.15.7:0/3984554955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56083071e000 0x560830734fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.947+0000 7fc30972f700 1 -- 172.21.15.7:0/3984554955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56083071a7a0 msgr2=0x560830734aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:10.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.947+0000 7fc30972f700 1 --2- 172.21.15.7:0/3984554955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56083071a7a0 0x560830734aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.947+0000 7fc30972f700 1 -- 172.21.15.7:0/3984554955 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608307178c0 con 0x56083072cec0 2024-06-10T01:29:10.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.948+0000 7fc30972f700 1 --2- 172.21.15.7:0/3984554955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56083072cec0 0x560830717380 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x560830715900 tx=0x7fc2ec0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8da2fc6ad84a14c2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:10.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.948+0000 7fc308f2e700 1 --2- 172.21.15.7:0/3984554955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56083071a7a0 0x560830734aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:29:10.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.948+0000 7fc2e3fff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:29:10.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.948+0000 7fc2e3fff700 1 -- 172.21.15.7:0/3984554955 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec003ae0 con 0x56083072cec0 2024-06-10T01:29:10.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.948+0000 7fc2e3fff700 1 -- 172.21.15.7:0/3984554955 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fc2ec003c80 con 0x56083072cec0 2024-06-10T01:29:10.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.948+0000 7fc315a15b80 1 -- 172.21.15.7:0/3984554955 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2f4007420 con 0x56083072cec0 2024-06-10T01:29:10.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.948+0000 7fc2e3fff700 1 -- 172.21.15.7:0/3984554955 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fc2ec034050 con 0x56083072cec0 2024-06-10T01:29:10.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.948+0000 7fc315a15b80 10 client.30116 fetch_fsmap learned FSMap version 55 2024-06-10T01:29:10.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.948+0000 7fc315a15b80 1 -- 172.21.15.7:0/3984554955 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5608307a6870 con 0x56083072cec0 2024-06-10T01:29:10.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.949+0000 7fc2e3fff700 1 -- 172.21.15.7:0/3984554955 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f4007420 con 0x56083072cec0 2024-06-10T01:29:10.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.949+0000 7fc2e3fff700 1 -- 172.21.15.7:0/3984554955 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fc2ec021540 con 0x56083072cec0 2024-06-10T01:29:10.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.949+0000 7fc315a15b80 10 client.30116 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:29:10.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.949+0000 7fc315a15b80 10 client.30116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:29:10.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.949+0000 7fc315a15b80 20 client.30116 populate_metadata read hostname 'smithi007' 2024-06-10T01:29:10.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.949+0000 7fc315a15b80 1 --2- 172.21.15.7:0/3984554955 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5608307a6f60 0x5608307a9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:10.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.949+0000 7fc315a15b80 4 client.30116 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:29:10.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.949+0000 7fc315a15b80 1 -- 172.21.15.7:0/3984554955 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5608306732d0 con 0x5608307a6f60 2024-06-10T01:29:10.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.949+0000 7fc308f2e700 1 --2- 172.21.15.7:0/3984554955 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5608307a6f60 0x5608307a9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:10.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.950+0000 7fc308f2e700 1 --2- 172.21.15.7:0/3984554955 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5608307a6f60 0x5608307a9380 crc :-1 s=READY pgs=518 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11abcc2a0522d5eb server_cookie=10dc1ea5834fdf49 in_seq=0 out_seq=0 2024-06-10T01:29:10.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.950+0000 7fc2e3fff700 10 client.30116 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.954+0000 7fc2e3fff700 1 -- 172.21.15.7:0/3984554955 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5608306732d0 con 0x5608307a6f60 2024-06-10T01:29:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.954+0000 7fc2e3fff700 10 client.30116 handle_command_reply: tid=0 2024-06-10T01:29:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.954+0000 7fc315a15b80 1 client.30116 shutdown 2024-06-10T01:29:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.954+0000 7fc315a15b80 1 -- 172.21.15.7:0/3984554955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56083072cec0 msgr2=0x560830717380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.954+0000 7fc315a15b80 1 --2- 172.21.15.7:0/3984554955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56083072cec0 0x560830717380 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x560830715900 tx=0x7fc2ec0047c0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.955+0000 7fc315a15b80 1 -- 172.21.15.7:0/3984554955 shutdown_connections 2024-06-10T01:29:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.955+0000 7fc315a15b80 1 --2- 172.21.15.7:0/3984554955 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5608307a6f60 0x5608307a9380 crc :-1 s=READY pgs=518 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.955+0000 7fc2e3fff700 0 client.30116 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.955+0000 7fc315a15b80 1 --2- 172.21.15.7:0/3984554955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56083072cec0 0x560830717380 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.955+0000 7fc315a15b80 1 --2- 172.21.15.7:0/3984554955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56083071a7a0 0x560830734aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.955+0000 7fc315a15b80 1 --2- 172.21.15.7:0/3984554955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56083071e000 0x560830734fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.955+0000 7fc315a15b80 1 -- 172.21.15.7:0/3984554955 >> 172.21.15.7:0/3984554955 conn(0x560830647080 msgr2=0x56083072b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.955+0000 7fc315a15b80 1 -- 172.21.15.7:0/3984554955 shutdown_connections 2024-06-10T01:29:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.955+0000 7fc315a15b80 1 -- 172.21.15.7:0/3984554955 wait complete. 2024-06-10T01:29:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.955+0000 7fc315a15b80 20 client.30116 trim_cache size 0 max 16384 2024-06-10T01:29:10.949 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:29:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.957+0000 7fc30b192700 1 -- 172.21.15.7:0/2762590281 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc2f00a4f00 msgr2=0x7fc2f00a73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.957+0000 7fc30b192700 1 --2- 172.21.15.7:0/2762590281 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc2f00a4f00 0x7fc2f00a73c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009c40 tx=0x7fc2ec0113e0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.957+0000 7fc30b192700 1 -- 172.21.15.7:0/2762590281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc304101d50 msgr2=0x7fc30419dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.957+0000 7fc30b192700 1 --2- 172.21.15.7:0/2762590281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc304101d50 0x7fc30419dd30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc2f400cfa0 tx=0x7fc2f400cfd0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.958+0000 7fc30b192700 1 -- 172.21.15.7:0/2762590281 shutdown_connections 2024-06-10T01:29:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.958+0000 7fc30b192700 1 --2- 172.21.15.7:0/2762590281 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fc2f00a4f00 0x7fc2f00a73c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.958+0000 7fc30b192700 1 --2- 172.21.15.7:0/2762590281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc304107400 0x7fc304197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.958+0000 7fc30b192700 1 --2- 172.21.15.7:0/2762590281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc304101390 0x7fc30419d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.958+0000 7fc30b192700 1 --2- 172.21.15.7:0/2762590281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc304101d50 0x7fc30419dd30 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.958+0000 7fc30b192700 1 -- 172.21.15.7:0/2762590281 >> 172.21.15.7:0/2762590281 conn(0x7fc304078340 msgr2=0x7fc3040ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.958+0000 7fc30b192700 1 -- 172.21.15.7:0/2762590281 shutdown_connections 2024-06-10T01:29:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:10.958+0000 7fc30b192700 1 -- 172.21.15.7:0/2762590281 wait complete. 2024-06-10T01:29:11.562 INFO:tasks.workunit.client.0.smithi007.stdout: 10 237629 22.70 MB/sec execute 468 sec latency 133.520 ms 2024-06-10T01:29:11.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:11 smithi007 ceph-mon[32398]: pgmap v1129: 129 pgs: 129 active+clean; 682 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 9.8 MiB/s wr, 632 op/s 2024-06-10T01:29:11.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:11 smithi175 ceph-mon[36636]: pgmap v1129: 129 pgs: 129 active+clean; 682 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 9.8 MiB/s wr, 632 op/s 2024-06-10T01:29:12.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:11 smithi078 ceph-mon[36683]: pgmap v1129: 129 pgs: 129 active+clean; 682 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 9.8 MiB/s wr, 632 op/s 2024-06-10T01:29:12.562 INFO:tasks.workunit.client.0.smithi007.stdout: 10 238069 22.70 MB/sec execute 469 sec latency 355.370 ms 2024-06-10T01:29:12.896 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:29:12.897 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:29:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:29:13.562 INFO:tasks.workunit.client.0.smithi007.stdout: 10 238548 22.70 MB/sec execute 470 sec latency 109.272 ms 2024-06-10T01:29:13.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:13 smithi007 ceph-mon[32398]: pgmap v1130: 129 pgs: 129 active+clean; 672 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 11 MiB/s wr, 698 op/s 2024-06-10T01:29:13.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:13 smithi175 ceph-mon[36636]: pgmap v1130: 129 pgs: 129 active+clean; 672 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 11 MiB/s wr, 698 op/s 2024-06-10T01:29:14.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:13 smithi078 ceph-mon[36683]: pgmap v1130: 129 pgs: 129 active+clean; 672 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 11 MiB/s wr, 698 op/s 2024-06-10T01:29:14.562 INFO:tasks.workunit.client.0.smithi007.stdout: 10 239285 22.74 MB/sec execute 471 sec latency 174.171 ms 2024-06-10T01:29:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.462+0000 7f473be2c700 1 -- 172.21.15.7:0/2536143689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4734101690 msgr2=0x7f4734101b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.462+0000 7f473be2c700 1 --2- 172.21.15.7:0/2536143689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4734101690 0x7f4734101b10 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f4724009990 tx=0x7f4724009ca0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.462+0000 7f473be2c700 1 -- 172.21.15.7:0/2536143689 shutdown_connections 2024-06-10T01:29:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.462+0000 7f473be2c700 1 --2- 172.21.15.7:0/2536143689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4734102050 0x7f473410e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.462+0000 7f473be2c700 1 --2- 172.21.15.7:0/2536143689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4734107590 0x7f4734107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.462+0000 7f473be2c700 1 --2- 172.21.15.7:0/2536143689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4734101690 0x7f4734101b10 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.462+0000 7f473be2c700 1 -- 172.21.15.7:0/2536143689 >> 172.21.15.7:0/2536143689 conn(0x7f4734078150 msgr2=0x7f4734078560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.462+0000 7f473be2c700 1 -- 172.21.15.7:0/2536143689 shutdown_connections 2024-06-10T01:29:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.462+0000 7f473be2c700 1 -- 172.21.15.7:0/2536143689 wait complete. 2024-06-10T01:29:15.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.463+0000 7f473be2c700 1 Processor -- start 2024-06-10T01:29:15.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.463+0000 7f473be2c700 1 -- start start 2024-06-10T01:29:15.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.463+0000 7f473be2c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4734101690 0x7f473419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:15.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.463+0000 7f473be2c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4734102050 0x7f473419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:15.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.463+0000 7f473be2c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4734107590 0x7f4734198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:15.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.463+0000 7f473be2c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4734112870 con 0x7f4734102050 2024-06-10T01:29:15.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.463+0000 7f473be2c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f47341126f0 con 0x7f4734101690 2024-06-10T01:29:15.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.463+0000 7f4739bc8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4734101690 0x7f473419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:15.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.463+0000 7f4739bc8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4734101690 0x7f473419e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:51770/0 (socket says 172.21.15.7:51770) 2024-06-10T01:29:15.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.464+0000 7f4739bc8700 1 -- 172.21.15.7:0/2132299190 learned_addr learned my addr 172.21.15.7:0/2132299190 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:15.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.464+0000 7f47393c7700 1 --2- 172.21.15.7:0/2132299190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4734102050 0x7f473419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:15.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.464+0000 7f473be2c700 1 -- 172.21.15.7:0/2132299190 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f47341129f0 con 0x7f4734107590 2024-06-10T01:29:15.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.464+0000 7f473a3c9700 1 --2- 172.21.15.7:0/2132299190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4734107590 0x7f4734198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:15.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.464+0000 7f4739bc8700 1 -- 172.21.15.7:0/2132299190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4734107590 msgr2=0x7f4734198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:15.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.464+0000 7f4739bc8700 1 --2- 172.21.15.7:0/2132299190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4734107590 0x7f4734198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.464+0000 7f4739bc8700 1 -- 172.21.15.7:0/2132299190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4734102050 msgr2=0x7f473419eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:15.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.464+0000 7f4739bc8700 1 --2- 172.21.15.7:0/2132299190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4734102050 0x7f473419eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.464+0000 7f4739bc8700 1 -- 172.21.15.7:0/2132299190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47341991d0 con 0x7f4734101690 2024-06-10T01:29:15.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.464+0000 7f47393c7700 1 --2- 172.21.15.7:0/2132299190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4734102050 0x7f473419eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:29:15.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.464+0000 7f4739bc8700 1 --2- 172.21.15.7:0/2132299190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4734101690 0x7f473419e860 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f473000e9a0 tx=0x7f473000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:15.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.465+0000 7f472affd700 1 -- 172.21.15.7:0/2132299190 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473000ca10 con 0x7f4734101690 2024-06-10T01:29:15.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.465+0000 7f473be2c700 1 -- 172.21.15.7:0/2132299190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47341994c0 con 0x7f4734101690 2024-06-10T01:29:15.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.465+0000 7f473be2c700 1 -- 172.21.15.7:0/2132299190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47341a5730 con 0x7f4734101690 2024-06-10T01:29:15.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.465+0000 7f472affd700 1 -- 172.21.15.7:0/2132299190 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f473000cbb0 con 0x7f4734101690 2024-06-10T01:29:15.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.465+0000 7f472affd700 1 -- 172.21.15.7:0/2132299190 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4730018680 con 0x7f4734101690 2024-06-10T01:29:15.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.468+0000 7f472affd700 1 -- 172.21.15.7:0/2132299190 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f4730018820 con 0x7f4734101690 2024-06-10T01:29:15.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.468+0000 7f472affd700 1 --2- 172.21.15.7:0/2132299190 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f47200a2ca0 0x7f47200a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:15.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.469+0000 7f472affd700 1 -- 172.21.15.7:0/2132299190 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f47300cf470 con 0x7f4734101690 2024-06-10T01:29:15.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.469+0000 7f47393c7700 1 --2- 172.21.15.7:0/2132299190 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f47200a2ca0 0x7f47200a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:15.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.469+0000 7f47466afb80 1 Processor -- start 2024-06-10T01:29:15.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.469+0000 7f47466afb80 1 -- start start 2024-06-10T01:29:15.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.470+0000 7f47393c7700 1 --2- 172.21.15.7:0/2132299190 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f47200a2ca0 0x7f47200a5160 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f472400b2e0 tx=0x7f4724005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:15.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.470+0000 7f47466afb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556489eaf050 0x556489eaf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:15.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.470+0000 7f47466afb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556489eaf970 0x556489eb3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:15.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.471+0000 7f4739bc8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556489eaf050 0x556489eaf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:15.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.471+0000 7f47466afb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556489eb3c80 0x556489eb40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:15.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.471+0000 7f4739bc8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556489eaf050 0x556489eaf430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:51780/0 (socket says 172.21.15.7:51780) 2024-06-10T01:29:15.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.471+0000 7f4739bc8700 1 -- 172.21.15.7:0/3970359107 learned_addr learned my addr 172.21.15.7:0/3970359107 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:15.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.471+0000 7f473a3c9700 1 --2- 172.21.15.7:0/3970359107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556489eb3c80 0x556489eb40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:15.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.471+0000 7f47393c7700 1 --2- 172.21.15.7:0/3970359107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556489eaf970 0x556489eb3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:15.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.471+0000 7f47466afb80 1 -- 172.21.15.7:0/3970359107 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556489dfc890 con 0x556489eb3c80 2024-06-10T01:29:15.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.471+0000 7f47466afb80 1 -- 172.21.15.7:0/3970359107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556489da47a0 con 0x556489eaf050 2024-06-10T01:29:15.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.471+0000 7f47466afb80 1 -- 172.21.15.7:0/3970359107 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556489df8850 con 0x556489eaf970 2024-06-10T01:29:15.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.472+0000 7f4739bc8700 1 -- 172.21.15.7:0/3970359107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556489eaf970 msgr2=0x556489eb3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:15.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.472+0000 7f4739bc8700 1 --2- 172.21.15.7:0/3970359107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556489eaf970 0x556489eb3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.472+0000 7f4739bc8700 1 -- 172.21.15.7:0/3970359107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556489eb3c80 msgr2=0x556489eb40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:15.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.472+0000 7f4739bc8700 1 --2- 172.21.15.7:0/3970359107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556489eb3c80 0x556489eb40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.472+0000 7f4739bc8700 1 -- 172.21.15.7:0/3970359107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556489d9afd0 con 0x556489eaf050 2024-06-10T01:29:15.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.472+0000 7f47393c7700 1 --2- 172.21.15.7:0/3970359107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556489eaf970 0x556489eb3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:29:15.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.472+0000 7f473a3c9700 1 --2- 172.21.15.7:0/3970359107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556489eb3c80 0x556489eb40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:29:15.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.472+0000 7f4739bc8700 1 --2- 172.21.15.7:0/3970359107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556489eaf050 0x556489eaf430 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f473000cfd0 tx=0x7f473009b950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abb52a8f5728794d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:15.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.472+0000 7f471ffff700 1 -- 172.21.15.7:0/3970359107 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47300a45c0 con 0x556489eaf050 2024-06-10T01:29:15.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.472+0000 7f471ffff700 1 -- 172.21.15.7:0/3970359107 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f473000cd60 con 0x556489eaf050 2024-06-10T01:29:15.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.472+0000 7f471ffff700 1 -- 172.21.15.7:0/3970359107 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4730086310 con 0x556489eaf050 2024-06-10T01:29:15.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.473+0000 7f47466afb80 1 -- 172.21.15.7:0/3970359107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556489eaf050 msgr2=0x556489eaf430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:15.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.473+0000 7f47466afb80 1 --2- 172.21.15.7:0/3970359107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556489eaf050 0x556489eaf430 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f473000cfd0 tx=0x7f473009b950 comp rx=0 tx=0).stop 2024-06-10T01:29:15.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.474+0000 7f47466afb80 1 -- 172.21.15.7:0/3970359107 shutdown_connections 2024-06-10T01:29:15.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.474+0000 7f47466afb80 1 --2- 172.21.15.7:0/3970359107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556489eb3c80 0x556489eb40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.474+0000 7f47466afb80 1 --2- 172.21.15.7:0/3970359107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556489eaf050 0x556489eaf430 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.474+0000 7f47466afb80 1 --2- 172.21.15.7:0/3970359107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556489eaf970 0x556489eb3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.474+0000 7f47466afb80 1 -- 172.21.15.7:0/3970359107 >> 172.21.15.7:0/3970359107 conn(0x556489dd5d40 msgr2=0x556489ea58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:15.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.474+0000 7f47466afb80 1 -- 172.21.15.7:0/3970359107 shutdown_connections 2024-06-10T01:29:15.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.474+0000 7f47466afb80 1 -- 172.21.15.7:0/3970359107 wait complete. 2024-06-10T01:29:15.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.475+0000 7f47466afb80 1 -- start start 2024-06-10T01:29:15.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.475+0000 7f47466afb80 1 Processor -- start 2024-06-10T01:29:15.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.475+0000 7f47466afb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556489eaf050 0x556489eadd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.475+0000 7f47466afb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556489eaf970 0x556489eae270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.476+0000 7f47466afb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556489eb3c80 0x556489eae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.476+0000 7f4739bc8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556489eaf050 0x556489eadd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.476+0000 7f47393c7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556489eaf970 0x556489eae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.476+0000 7f47466afb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556489eaad40 con 0x556489eaf970 2024-06-10T01:29:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.476+0000 7f47393c7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556489eaf970 0x556489eae270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39300/0 (socket says 172.21.15.7:39300) 2024-06-10T01:29:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.476+0000 7f47393c7700 1 -- 172.21.15.7:0/1591473231 learned_addr learned my addr 172.21.15.7:0/1591473231 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.476+0000 7f47466afb80 1 -- 172.21.15.7:0/1591473231 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556489d9cc70 con 0x556489eb3c80 2024-06-10T01:29:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.476+0000 7f47466afb80 1 -- 172.21.15.7:0/1591473231 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556489eaaec0 con 0x556489eaf050 2024-06-10T01:29:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.476+0000 7f473a3c9700 1 --2- 172.21.15.7:0/1591473231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556489eb3c80 0x556489eae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:15.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.476+0000 7f47466afb80 10 client.0 _ll_register_callbacks cb 0x556489d969a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:29:15.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.476+0000 7f473a3c9700 1 -- 172.21.15.7:0/1591473231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556489eaf050 msgr2=0x556489eadd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:15.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.476+0000 7f473a3c9700 1 --2- 172.21.15.7:0/1591473231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556489eaf050 0x556489eadd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.476+0000 7f473a3c9700 1 -- 172.21.15.7:0/1591473231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556489eaf970 msgr2=0x556489eae270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:15.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.476+0000 7f473a3c9700 1 --2- 172.21.15.7:0/1591473231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556489eaf970 0x556489eae270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.476+0000 7f473a3c9700 1 -- 172.21.15.7:0/1591473231 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556489d4b950 con 0x556489eb3c80 2024-06-10T01:29:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.477+0000 7f473a3c9700 1 --2- 172.21.15.7:0/1591473231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556489eb3c80 0x556489eae7b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f472c007cb0 tx=0x7f472c0094a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6a78bbb88f21f3f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.477+0000 7f471d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:29:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.477+0000 7f471d7fa700 1 -- 172.21.15.7:0/1591473231 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c011e40 con 0x556489eb3c80 2024-06-10T01:29:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.477+0000 7f47466afb80 1 -- 172.21.15.7:0/1591473231 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f473000cd60 con 0x556489eb3c80 2024-06-10T01:29:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.477+0000 7f471d7fa700 1 -- 172.21.15.7:0/1591473231 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f472c004bb0 con 0x556489eb3c80 2024-06-10T01:29:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.477+0000 7f471d7fa700 1 -- 172.21.15.7:0/1591473231 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f472c027050 con 0x556489eb3c80 2024-06-10T01:29:15.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.477+0000 7f47466afb80 10 client.30128 fetch_fsmap learned FSMap version 55 2024-06-10T01:29:15.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.477+0000 7f47466afb80 1 -- 172.21.15.7:0/1591473231 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x556489ec2460 con 0x556489eb3c80 2024-06-10T01:29:15.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.478+0000 7f471d7fa700 1 -- 172.21.15.7:0/1591473231 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f473000cd60 con 0x556489eb3c80 2024-06-10T01:29:15.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.478+0000 7f471d7fa700 1 -- 172.21.15.7:0/1591473231 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f472c016400 con 0x556489eb3c80 2024-06-10T01:29:15.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.478+0000 7f47466afb80 10 client.30128 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:29:15.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.478+0000 7f47466afb80 10 client.30128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:29:15.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.478+0000 7f47466afb80 20 client.30128 populate_metadata read hostname 'smithi007' 2024-06-10T01:29:15.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.478+0000 7f47466afb80 1 --2- 172.21.15.7:0/1591473231 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556489ed5a60 0x556489ed7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:15.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.478+0000 7f47466afb80 4 client.30128 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:29:15.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.478+0000 7f47466afb80 1 -- 172.21.15.7:0/1591473231 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556489e09d90 con 0x556489ed5a60 2024-06-10T01:29:15.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.478+0000 7f4739bc8700 1 --2- 172.21.15.7:0/1591473231 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556489ed5a60 0x556489ed7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:15.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.480+0000 7f4739bc8700 1 --2- 172.21.15.7:0/1591473231 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556489ed5a60 0x556489ed7e50 crc :-1 s=READY pgs=519 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=465af5d66218e09f server_cookie=a871b6fe6185db0a in_seq=0 out_seq=0 2024-06-10T01:29:15.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.480+0000 7f471d7fa700 10 client.30128 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:15.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.485+0000 7f471d7fa700 1 -- 172.21.15.7:0/1591473231 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556489e09d90 con 0x556489ed5a60 2024-06-10T01:29:15.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.485+0000 7f471d7fa700 10 client.30128 handle_command_reply: tid=0 2024-06-10T01:29:15.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.485+0000 7f47466afb80 1 client.30128 shutdown 2024-06-10T01:29:15.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.485+0000 7f47466afb80 1 -- 172.21.15.7:0/1591473231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556489eb3c80 msgr2=0x556489eae7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:15.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.485+0000 7f47466afb80 1 --2- 172.21.15.7:0/1591473231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556489eb3c80 0x556489eae7b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f472c007cb0 tx=0x7f472c0094a0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.485+0000 7f47466afb80 1 -- 172.21.15.7:0/1591473231 shutdown_connections 2024-06-10T01:29:15.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.485+0000 7f47466afb80 1 --2- 172.21.15.7:0/1591473231 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556489ed5a60 0x556489ed7e50 crc :-1 s=READY pgs=519 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.486+0000 7f47466afb80 1 --2- 172.21.15.7:0/1591473231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556489eb3c80 0x556489eae7b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.486+0000 7f47466afb80 1 --2- 172.21.15.7:0/1591473231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556489eaf050 0x556489eadd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.486+0000 7f47466afb80 1 --2- 172.21.15.7:0/1591473231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556489eaf970 0x556489eae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.486+0000 7f471d7fa700 0 client.30128 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:15.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.486+0000 7f47466afb80 1 -- 172.21.15.7:0/1591473231 >> 172.21.15.7:0/1591473231 conn(0x556489dd5d40 msgr2=0x556489ea9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:15.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.486+0000 7f47466afb80 1 -- 172.21.15.7:0/1591473231 shutdown_connections 2024-06-10T01:29:15.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.486+0000 7f47466afb80 1 -- 172.21.15.7:0/1591473231 wait complete. 2024-06-10T01:29:15.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.486+0000 7f47466afb80 20 client.30128 trim_cache size 0 max 16384 2024-06-10T01:29:15.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.514+0000 7f47466afb80 1 Processor -- start 2024-06-10T01:29:15.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.514+0000 7f47466afb80 1 -- start start 2024-06-10T01:29:15.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.514+0000 7f47466afb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556489eaf050 0x556489eab880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:15.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.514+0000 7f47466afb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556489eaf970 0x556489edd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:15.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.514+0000 7f47466afb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556489eb3c80 0x556489eddca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:15.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.514+0000 7f47466afb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4718011300 con 0x556489eaf050 2024-06-10T01:29:15.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.514+0000 7f47466afb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f471800eb90 con 0x556489eb3c80 2024-06-10T01:29:15.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.514+0000 7f47466afb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f471800c9e0 con 0x556489eaf970 2024-06-10T01:29:15.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.515+0000 7f4739bc8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556489eaf050 0x556489eab880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:15.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.515+0000 7f4739bc8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556489eaf050 0x556489eab880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39312/0 (socket says 172.21.15.7:39312) 2024-06-10T01:29:15.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.515+0000 7f4739bc8700 1 -- 172.21.15.7:0/4221777551 learned_addr learned my addr 172.21.15.7:0/4221777551 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:15.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.515+0000 7f47393c7700 1 --2- 172.21.15.7:0/4221777551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556489eaf970 0x556489edd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:15.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.515+0000 7f473a3c9700 1 --2- 172.21.15.7:0/4221777551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556489eb3c80 0x556489eddca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:15.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.515+0000 7f4739bc8700 1 -- 172.21.15.7:0/4221777551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556489eaf970 msgr2=0x556489edd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:15.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.515+0000 7f4739bc8700 1 --2- 172.21.15.7:0/4221777551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556489eaf970 0x556489edd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.515+0000 7f4739bc8700 1 -- 172.21.15.7:0/4221777551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556489eb3c80 msgr2=0x556489eddca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:15.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.515+0000 7f4739bc8700 1 --2- 172.21.15.7:0/4221777551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556489eb3c80 0x556489eddca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.515+0000 7f4739bc8700 1 -- 172.21.15.7:0/4221777551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556489ebb1c0 con 0x556489eaf050 2024-06-10T01:29:15.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.515+0000 7f4739bc8700 1 --2- 172.21.15.7:0/4221777551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556489eaf050 0x556489eab880 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f473009fec0 tx=0x7f473009b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee1ab08e7c27386 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:15.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.515+0000 7f471f7fe700 1 -- 172.21.15.7:0/4221777551 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47300b3070 con 0x556489eaf050 2024-06-10T01:29:15.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.516+0000 7f471f7fe700 1 -- 172.21.15.7:0/4221777551 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4730099c50 con 0x556489eaf050 2024-06-10T01:29:15.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.516+0000 7f471f7fe700 1 -- 172.21.15.7:0/4221777551 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47300a4390 con 0x556489eaf050 2024-06-10T01:29:15.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.516+0000 7f47466afb80 1 -- 172.21.15.7:0/4221777551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556489eaf050 msgr2=0x556489eab880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:15.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.516+0000 7f47466afb80 1 --2- 172.21.15.7:0/4221777551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556489eaf050 0x556489eab880 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f473009fec0 tx=0x7f473009b850 comp rx=0 tx=0).stop 2024-06-10T01:29:15.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.516+0000 7f47466afb80 1 -- 172.21.15.7:0/4221777551 shutdown_connections 2024-06-10T01:29:15.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.516+0000 7f47466afb80 1 --2- 172.21.15.7:0/4221777551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556489eb3c80 0x556489eddca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.516+0000 7f47466afb80 1 --2- 172.21.15.7:0/4221777551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556489eaf050 0x556489eab880 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.516+0000 7f47466afb80 1 --2- 172.21.15.7:0/4221777551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556489eaf970 0x556489edd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.516+0000 7f47466afb80 1 -- 172.21.15.7:0/4221777551 >> 172.21.15.7:0/4221777551 conn(0x556489dd5d40 msgr2=0x556489eab170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:15.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.516+0000 7f47466afb80 1 -- 172.21.15.7:0/4221777551 shutdown_connections 2024-06-10T01:29:15.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.517+0000 7f47466afb80 1 -- 172.21.15.7:0/4221777551 wait complete. 2024-06-10T01:29:15.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.517+0000 7f47466afb80 1 -- start start 2024-06-10T01:29:15.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.517+0000 7f47466afb80 1 Processor -- start 2024-06-10T01:29:15.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.518+0000 7f47466afb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556489eaf050 0x556489ec13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:15.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.518+0000 7f47466afb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556489eaf970 0x556489ebbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:15.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.518+0000 7f47466afb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556489eb3c80 0x556489ebc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:15.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.518+0000 7f47466afb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f471800d050 con 0x556489eaf050 2024-06-10T01:29:15.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.518+0000 7f4739bc8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556489eaf050 0x556489ec13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:15.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.518+0000 7f4739bc8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556489eaf050 0x556489ec13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39322/0 (socket says 172.21.15.7:39322) 2024-06-10T01:29:15.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.518+0000 7f47466afb80 1 -- 172.21.15.7:0/3466620036 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f471800e4c0 con 0x556489eb3c80 2024-06-10T01:29:15.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.518+0000 7f47393c7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556489eaf970 0x556489ebbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:15.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.518+0000 7f47466afb80 1 -- 172.21.15.7:0/3466620036 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f471800d720 con 0x556489eaf970 2024-06-10T01:29:15.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.518+0000 7f4739bc8700 1 -- 172.21.15.7:0/3466620036 learned_addr learned my addr 172.21.15.7:0/3466620036 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:15.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.518+0000 7f473a3c9700 1 --2- 172.21.15.7:0/3466620036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556489eb3c80 0x556489ebc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:15.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.518+0000 7f47466afb80 10 client.0 _ll_register_callbacks cb 0x556489d969a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:29:15.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.518+0000 7f47393c7700 1 -- 172.21.15.7:0/3466620036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556489eb3c80 msgr2=0x556489ebc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:15.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.518+0000 7f47393c7700 1 --2- 172.21.15.7:0/3466620036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556489eb3c80 0x556489ebc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.518+0000 7f47393c7700 1 -- 172.21.15.7:0/3466620036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556489eaf050 msgr2=0x556489ec13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:15.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.518+0000 7f47393c7700 1 --2- 172.21.15.7:0/3466620036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556489eaf050 0x556489ec13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.518+0000 7f47393c7700 1 -- 172.21.15.7:0/3466620036 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556489ebc8e0 con 0x556489eaf970 2024-06-10T01:29:15.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.518+0000 7f473a3c9700 1 --2- 172.21.15.7:0/3466620036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556489eb3c80 0x556489ebc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:29:15.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.518+0000 7f4739bc8700 1 --2- 172.21.15.7:0/3466620036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556489eaf050 0x556489ec13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:29:15.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.519+0000 7f47393c7700 1 --2- 172.21.15.7:0/3466620036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556489eaf970 0x556489ebbb60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x556489ea6930 tx=0x7f4724009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d0feb006bdc854a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:15.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.519+0000 7f471d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:29:15.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.519+0000 7f471d7fa700 1 -- 172.21.15.7:0/3466620036 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4724038060 con 0x556489eaf970 2024-06-10T01:29:15.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.519+0000 7f47466afb80 1 -- 172.21.15.7:0/3466620036 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4730099c50 con 0x556489eaf970 2024-06-10T01:29:15.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.519+0000 7f471d7fa700 1 -- 172.21.15.7:0/3466620036 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f4724005440 con 0x556489eaf970 2024-06-10T01:29:15.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.519+0000 7f471d7fa700 1 -- 172.21.15.7:0/3466620036 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f472403f050 con 0x556489eaf970 2024-06-10T01:29:15.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.519+0000 7f47466afb80 10 client.30043 fetch_fsmap learned FSMap version 55 2024-06-10T01:29:15.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.519+0000 7f47466afb80 1 -- 172.21.15.7:0/3466620036 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x556489f416c0 con 0x556489eaf970 2024-06-10T01:29:15.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.519+0000 7f471d7fa700 1 -- 172.21.15.7:0/3466620036 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4730099c50 con 0x556489eaf970 2024-06-10T01:29:15.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.520+0000 7f471d7fa700 1 -- 172.21.15.7:0/3466620036 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f4724033720 con 0x556489eaf970 2024-06-10T01:29:15.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.520+0000 7f47466afb80 10 client.30043 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:29:15.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.520+0000 7f47466afb80 10 client.30043 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:29:15.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.520+0000 7f47466afb80 20 client.30043 populate_metadata read hostname 'smithi007' 2024-06-10T01:29:15.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.520+0000 7f47466afb80 1 --2- 172.21.15.7:0/3466620036 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556489f41e00 0x556489f44220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:15.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.520+0000 7f47466afb80 4 client.30043 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:29:15.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.520+0000 7f47466afb80 1 -- 172.21.15.7:0/3466620036 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556489e04310 con 0x556489f41e00 2024-06-10T01:29:15.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.520+0000 7f4739bc8700 1 --2- 172.21.15.7:0/3466620036 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556489f41e00 0x556489f44220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:15.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.520+0000 7f471d7fa700 10 client.30043 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:15.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.520+0000 7f4739bc8700 1 --2- 172.21.15.7:0/3466620036 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556489f41e00 0x556489f44220 crc :-1 s=READY pgs=520 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96fd70be55f678fd server_cookie=716a4e682edb9ea4 in_seq=0 out_seq=0 2024-06-10T01:29:15.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.524+0000 7f471d7fa700 1 -- 172.21.15.7:0/3466620036 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556489e04310 con 0x556489f41e00 2024-06-10T01:29:15.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.524+0000 7f471d7fa700 10 client.30043 handle_command_reply: tid=0 2024-06-10T01:29:15.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.524+0000 7f47466afb80 1 client.30043 shutdown 2024-06-10T01:29:15.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.525+0000 7f47466afb80 1 -- 172.21.15.7:0/3466620036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556489eaf970 msgr2=0x556489ebbb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:15.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.525+0000 7f47466afb80 1 --2- 172.21.15.7:0/3466620036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556489eaf970 0x556489ebbb60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x556489ea6930 tx=0x7f4724009ca0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.525+0000 7f47466afb80 1 -- 172.21.15.7:0/3466620036 shutdown_connections 2024-06-10T01:29:15.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.525+0000 7f47466afb80 1 --2- 172.21.15.7:0/3466620036 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556489f41e00 0x556489f44220 crc :-1 s=READY pgs=520 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.525+0000 7f47466afb80 1 --2- 172.21.15.7:0/3466620036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556489eb3c80 0x556489ebc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.525+0000 7f47466afb80 1 --2- 172.21.15.7:0/3466620036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556489eaf050 0x556489ec13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.525+0000 7f471d7fa700 0 client.30043 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.525+0000 7f47466afb80 1 --2- 172.21.15.7:0/3466620036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556489eaf970 0x556489ebbb60 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.525+0000 7f47466afb80 1 -- 172.21.15.7:0/3466620036 >> 172.21.15.7:0/3466620036 conn(0x556489dd5d40 msgr2=0x556489eac210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.525+0000 7f47466afb80 1 -- 172.21.15.7:0/3466620036 shutdown_connections 2024-06-10T01:29:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.525+0000 7f47466afb80 1 -- 172.21.15.7:0/3466620036 wait complete. 2024-06-10T01:29:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.525+0000 7f47466afb80 20 client.30043 trim_cache size 0 max 16384 2024-06-10T01:29:15.519 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:29:15.519 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-10T01:29:15.519 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "74d7b72d-6bf0-487e-93ff-1e1128b419b8", 2024-06-10T01:29:15.519 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-10T01:29:15.520 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:29:15.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.526+0000 7f473be2c700 1 -- 172.21.15.7:0/2132299190 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f47200a2ca0 msgr2=0x7f47200a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:15.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.526+0000 7f473be2c700 1 --2- 172.21.15.7:0/2132299190 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f47200a2ca0 0x7f47200a5160 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f472400b2e0 tx=0x7f4724005960 comp rx=0 tx=0).stop 2024-06-10T01:29:15.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.526+0000 7f473be2c700 1 -- 172.21.15.7:0/2132299190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4734101690 msgr2=0x7f473419e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:15.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.526+0000 7f473be2c700 1 --2- 172.21.15.7:0/2132299190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4734101690 0x7f473419e860 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f473000e9a0 tx=0x7f473000ecb0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.526+0000 7f473be2c700 1 -- 172.21.15.7:0/2132299190 shutdown_connections 2024-06-10T01:29:15.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.526+0000 7f473be2c700 1 --2- 172.21.15.7:0/2132299190 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f47200a2ca0 0x7f47200a5160 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.526+0000 7f473be2c700 1 --2- 172.21.15.7:0/2132299190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4734107590 0x7f4734198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.526+0000 7f473be2c700 1 --2- 172.21.15.7:0/2132299190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4734101690 0x7f473419e860 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.526+0000 7f473be2c700 1 --2- 172.21.15.7:0/2132299190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4734102050 0x7f473419eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:15.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.526+0000 7f473be2c700 1 -- 172.21.15.7:0/2132299190 >> 172.21.15.7:0/2132299190 conn(0x7f4734078150 msgr2=0x7f4734100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:15.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.527+0000 7f473be2c700 1 -- 172.21.15.7:0/2132299190 shutdown_connections 2024-06-10T01:29:15.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:15.527+0000 7f473be2c700 1 -- 172.21.15.7:0/2132299190 wait complete. 2024-06-10T01:29:15.562 INFO:tasks.workunit.client.0.smithi007.stdout: 10 239952 22.76 MB/sec execute 472 sec latency 84.834 ms 2024-06-10T01:29:15.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:15 smithi007 ceph-mon[32398]: pgmap v1131: 129 pgs: 129 active+clean; 667 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 11 MiB/s wr, 641 op/s 2024-06-10T01:29:15.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:15 smithi175 ceph-mon[36636]: pgmap v1131: 129 pgs: 129 active+clean; 667 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 11 MiB/s wr, 641 op/s 2024-06-10T01:29:16.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:15 smithi078 ceph-mon[36683]: pgmap v1131: 129 pgs: 129 active+clean; 667 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 11 MiB/s wr, 641 op/s 2024-06-10T01:29:16.446 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:29:16.562 INFO:tasks.workunit.client.0.smithi007.stdout: 10 240458 22.78 MB/sec execute 473 sec latency 68.957 ms 2024-06-10T01:29:16.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:16 smithi007 ceph-mon[32398]: scrub queued for path: / 2024-06-10T01:29:16.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:16 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:29:16.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:16 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:29:16.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:16 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:29:16.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:16 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:29:16.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:16 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:29:16.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:16 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:29:16.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:16 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:29:16.846 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:29:16.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:16 smithi175 ceph-mon[36636]: scrub queued for path: / 2024-06-10T01:29:16.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:16 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:29:16.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:16 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:29:16.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:16 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:29:16.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:16 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:29:16.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:16 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:29:16.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:16 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:29:16.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:16 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:29:17.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:16 smithi078 ceph-mon[36683]: scrub queued for path: / 2024-06-10T01:29:17.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:16 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:29:17.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:16 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:29:17.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:16 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:29:17.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:16 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:29:17.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:16 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:29:17.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:16 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:29:17.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:16 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:29:17.562 INFO:tasks.workunit.client.0.smithi007.stdout: 10 241048 22.79 MB/sec execute 474 sec latency 116.679 ms 2024-06-10T01:29:17.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:17 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:29:17.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:17 smithi175 ceph-mon[36636]: pgmap v1132: 129 pgs: 129 active+clean; 667 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 11 MiB/s wr, 741 op/s 2024-06-10T01:29:17.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:17 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:29:17.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:17 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:29:17.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:17 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:29:18.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:17 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:29:18.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:17 smithi007 ceph-mon[32398]: pgmap v1132: 129 pgs: 129 active+clean; 667 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 11 MiB/s wr, 741 op/s 2024-06-10T01:29:18.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:17 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:29:18.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:17 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:29:18.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:17 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:29:18.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:17 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:29:18.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:17 smithi078 ceph-mon[36683]: pgmap v1132: 129 pgs: 129 active+clean; 667 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 11 MiB/s wr, 741 op/s 2024-06-10T01:29:18.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:17 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:29:18.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:17 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:29:18.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:17 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:29:18.563 INFO:tasks.workunit.client.0.smithi007.stdout: 10 241653 22.80 MB/sec execute 475 sec latency 73.969 ms 2024-06-10T01:29:18.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.711+0000 7f3ca5139700 1 -- 172.21.15.7:0/2063333055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ca011bc70 msgr2=0x7f3ca011e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:18.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.711+0000 7f3ca5139700 1 --2- 172.21.15.7:0/2063333055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ca011bc70 0x7f3ca011e060 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009990 tx=0x7f3c90009ca0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.711+0000 7f3ca5139700 1 -- 172.21.15.7:0/2063333055 shutdown_connections 2024-06-10T01:29:18.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.711+0000 7f3ca5139700 1 --2- 172.21.15.7:0/2063333055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ca011bc70 0x7f3ca011e060 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.711+0000 7f3ca5139700 1 --2- 172.21.15.7:0/2063333055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ca0110c70 0x7f3ca0109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.711+0000 7f3ca5139700 1 --2- 172.21.15.7:0/2063333055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ca01099d0 0x7f3ca0109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.711+0000 7f3ca5139700 1 -- 172.21.15.7:0/2063333055 >> 172.21.15.7:0/2063333055 conn(0x7f3ca006d600 msgr2=0x7f3ca006da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:18.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.711+0000 7f3ca5139700 1 -- 172.21.15.7:0/2063333055 shutdown_connections 2024-06-10T01:29:18.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.711+0000 7f3ca5139700 1 -- 172.21.15.7:0/2063333055 wait complete. 2024-06-10T01:29:18.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.712+0000 7f3ca5139700 1 Processor -- start 2024-06-10T01:29:18.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.713+0000 7f3ca5139700 1 -- start start 2024-06-10T01:29:18.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.713+0000 7f3ca5139700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ca01099d0 0x7f3ca011a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:18.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.713+0000 7f3ca5139700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ca0110c70 0x7f3ca011a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:18.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.713+0000 7f3ca5139700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ca011bc70 0x7f3ca01133a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:18.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.713+0000 7f3ca5139700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca0121650 con 0x7f3ca01099d0 2024-06-10T01:29:18.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.713+0000 7f3ca5139700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca01214d0 con 0x7f3ca011bc70 2024-06-10T01:29:18.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.713+0000 7f3ca5139700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca01217d0 con 0x7f3ca0110c70 2024-06-10T01:29:18.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.713+0000 7f3c9e59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ca0110c70 0x7f3ca011a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:18.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.713+0000 7f3c9e59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ca0110c70 0x7f3ca011a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:35604/0 (socket says 172.21.15.7:35604) 2024-06-10T01:29:18.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.713+0000 7f3c9e59c700 1 -- 172.21.15.7:0/660837788 learned_addr learned my addr 172.21.15.7:0/660837788 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:18.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.713+0000 7f3c9f59e700 1 --2- 172.21.15.7:0/660837788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ca011bc70 0x7f3ca01133a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:18.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.713+0000 7f3c9ed9d700 1 --2- 172.21.15.7:0/660837788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ca01099d0 0x7f3ca011a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:18.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.714+0000 7f3c9f59e700 1 -- 172.21.15.7:0/660837788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ca0110c70 msgr2=0x7f3ca011a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:18.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.714+0000 7f3c9f59e700 1 --2- 172.21.15.7:0/660837788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ca0110c70 0x7f3ca011a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.714+0000 7f3c9f59e700 1 -- 172.21.15.7:0/660837788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ca01099d0 msgr2=0x7f3ca011a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:18.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.714+0000 7f3c9f59e700 1 --2- 172.21.15.7:0/660837788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ca01099d0 0x7f3ca011a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.714+0000 7f3c9f59e700 1 -- 172.21.15.7:0/660837788 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca0113ca0 con 0x7f3ca011bc70 2024-06-10T01:29:18.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.714+0000 7f3c9f59e700 1 --2- 172.21.15.7:0/660837788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ca011bc70 0x7f3ca01133a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3c9000b2e0 tx=0x7f3c9000b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:18.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.714+0000 7f3c87fff700 1 -- 172.21.15.7:0/660837788 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9000f840 con 0x7f3ca011bc70 2024-06-10T01:29:18.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.715+0000 7f3c87fff700 1 -- 172.21.15.7:0/660837788 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c900173f0 con 0x7f3ca011bc70 2024-06-10T01:29:18.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.715+0000 7f3c87fff700 1 -- 172.21.15.7:0/660837788 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9000e640 con 0x7f3ca011bc70 2024-06-10T01:29:18.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.715+0000 7f3ca5139700 1 -- 172.21.15.7:0/660837788 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca0113f30 con 0x7f3ca011bc70 2024-06-10T01:29:18.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.715+0000 7f3ca5139700 1 -- 172.21.15.7:0/660837788 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca0117e50 con 0x7f3ca011bc70 2024-06-10T01:29:18.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.718+0000 7f3c87fff700 1 -- 172.21.15.7:0/660837788 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f3c90009d10 con 0x7f3ca011bc70 2024-06-10T01:29:18.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.719+0000 7f3c87fff700 1 --2- 172.21.15.7:0/660837788 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3c880a2d70 0x7f3c880a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:18.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.719+0000 7f3c87fff700 1 -- 172.21.15.7:0/660837788 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f3c900d0070 con 0x7f3ca011bc70 2024-06-10T01:29:18.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.720+0000 7f3c9ed9d700 1 --2- 172.21.15.7:0/660837788 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3c880a2d70 0x7f3c880a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:18.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.720+0000 7f3c9ed9d700 1 --2- 172.21.15.7:0/660837788 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3c880a2d70 0x7f3c880a5230 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800f540 tx=0x7f3c9800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:18.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.721+0000 7f3caf9bcb80 1 Processor -- start 2024-06-10T01:29:18.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.721+0000 7f3caf9bcb80 1 -- start start 2024-06-10T01:29:18.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.722+0000 7f3caf9bcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556abf5e5f60 0x556abf5e6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:18.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.722+0000 7f3caf9bcb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556abf5e6880 0x556abf5e0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:18.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.722+0000 7f3caf9bcb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556abf5e07a0 0x556abf5e0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:18.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.722+0000 7f3c9e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556abf5e6880 0x556abf5e0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:18.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.722+0000 7f3c9e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556abf5e6880 0x556abf5e0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39340/0 (socket says 172.21.15.7:39340) 2024-06-10T01:29:18.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.722+0000 7f3c9e59c700 1 -- 172.21.15.7:0/247787406 learned_addr learned my addr 172.21.15.7:0/247787406 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:18.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.722+0000 7f3c9f59e700 1 --2- 172.21.15.7:0/247787406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556abf5e07a0 0x556abf5e0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:18.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.722+0000 7f3caf9bcb80 1 -- 172.21.15.7:0/247787406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556abf4d2f60 con 0x556abf5e6880 2024-06-10T01:29:18.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.722+0000 7f3caf9bcb80 1 -- 172.21.15.7:0/247787406 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556abf531850 con 0x556abf5e5f60 2024-06-10T01:29:18.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.722+0000 7f3caf9bcb80 1 -- 172.21.15.7:0/247787406 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556abf499460 con 0x556abf5e07a0 2024-06-10T01:29:18.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.722+0000 7f3c9e59c700 1 -- 172.21.15.7:0/247787406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556abf5e07a0 msgr2=0x556abf5e0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:18.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.722+0000 7f3c9e59c700 1 --2- 172.21.15.7:0/247787406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556abf5e07a0 0x556abf5e0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.722+0000 7f3c9e59c700 1 -- 172.21.15.7:0/247787406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556abf5e5f60 msgr2=0x556abf5e6340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T01:29:18.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.722+0000 7f3c9e59c700 1 --2- 172.21.15.7:0/247787406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556abf5e5f60 0x556abf5e6340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.723+0000 7f3c9e59c700 1 -- 172.21.15.7:0/247787406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556abf4808d0 con 0x556abf5e6880 2024-06-10T01:29:18.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.723+0000 7f3c9e59c700 1 --2- 172.21.15.7:0/247787406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556abf5e6880 0x556abf5e0260 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f3c94009db0 tx=0x7f3c94011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d097c911f7c83f0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:18.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.723+0000 7f3c857fa700 1 -- 172.21.15.7:0/247787406 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94004220 con 0x556abf5e6880 2024-06-10T01:29:18.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.724+0000 7f3c857fa700 1 -- 172.21.15.7:0/247787406 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c940043c0 con 0x556abf5e6880 2024-06-10T01:29:18.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.724+0000 7f3c857fa700 1 -- 172.21.15.7:0/247787406 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94019990 con 0x556abf5e6880 2024-06-10T01:29:18.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.725+0000 7f3caf9bcb80 1 -- 172.21.15.7:0/247787406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556abf5e6880 msgr2=0x556abf5e0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:18.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.725+0000 7f3caf9bcb80 1 --2- 172.21.15.7:0/247787406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556abf5e6880 0x556abf5e0260 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f3c94009db0 tx=0x7f3c94011720 comp rx=0 tx=0).stop 2024-06-10T01:29:18.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.725+0000 7f3caf9bcb80 1 -- 172.21.15.7:0/247787406 shutdown_connections 2024-06-10T01:29:18.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.725+0000 7f3caf9bcb80 1 --2- 172.21.15.7:0/247787406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556abf5e07a0 0x556abf5e0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.725+0000 7f3caf9bcb80 1 --2- 172.21.15.7:0/247787406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556abf5e5f60 0x556abf5e6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.725+0000 7f3caf9bcb80 1 --2- 172.21.15.7:0/247787406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556abf5e6880 0x556abf5e0260 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.725+0000 7f3caf9bcb80 1 -- 172.21.15.7:0/247787406 >> 172.21.15.7:0/247787406 conn(0x556abf50d080 msgr2=0x556abf5db220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:18.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.725+0000 7f3caf9bcb80 1 -- 172.21.15.7:0/247787406 shutdown_connections 2024-06-10T01:29:18.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.725+0000 7f3caf9bcb80 1 -- 172.21.15.7:0/247787406 wait complete. 2024-06-10T01:29:18.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.726+0000 7f3caf9bcb80 1 -- start start 2024-06-10T01:29:18.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.726+0000 7f3caf9bcb80 1 Processor -- start 2024-06-10T01:29:18.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.726+0000 7f3caf9bcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556abf5e07a0 0x556abf5ec610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:18.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.726+0000 7f3caf9bcb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556abf5e5f60 0x556abf5e9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:18.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.726+0000 7f3caf9bcb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556abf5e6880 0x556abf5e9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:18.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.726+0000 7f3caf9bcb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556abf4d1fc0 con 0x556abf5e5f60 2024-06-10T01:29:18.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.727+0000 7f3c9ed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556abf5e07a0 0x556abf5ec610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:18.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.727+0000 7f3c9e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556abf5e5f60 0x556abf5e9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:18.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.727+0000 7f3c9ed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556abf5e07a0 0x556abf5ec610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:51832/0 (socket says 172.21.15.7:51832) 2024-06-10T01:29:18.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.727+0000 7f3c9ed9d700 1 -- 172.21.15.7:0/3081301811 learned_addr learned my addr 172.21.15.7:0/3081301811 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:18.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.727+0000 7f3caf9bcb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556abf52d810 con 0x556abf5e07a0 2024-06-10T01:29:18.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.727+0000 7f3caf9bcb80 1 -- 172.21.15.7:0/3081301811 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556abf5ed260 con 0x556abf5e6880 2024-06-10T01:29:18.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.727+0000 7f3caf9bcb80 10 client.0 _ll_register_callbacks cb 0x556abf4c75b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:29:18.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.727+0000 7f3c9f59e700 1 --2- 172.21.15.7:0/3081301811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556abf5e6880 0x556abf5e9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:18.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.727+0000 7f3c9f59e700 1 -- 172.21.15.7:0/3081301811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556abf5e07a0 msgr2=0x556abf5ec610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:18.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.727+0000 7f3c9f59e700 1 --2- 172.21.15.7:0/3081301811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556abf5e07a0 0x556abf5ec610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.727+0000 7f3c9f59e700 1 -- 172.21.15.7:0/3081301811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556abf5e5f60 msgr2=0x556abf5e9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:18.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.727+0000 7f3c9f59e700 1 --2- 172.21.15.7:0/3081301811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556abf5e5f60 0x556abf5e9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.727+0000 7f3c9f59e700 1 -- 172.21.15.7:0/3081301811 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556abf4f7750 con 0x556abf5e6880 2024-06-10T01:29:18.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.727+0000 7f3c9e59c700 1 --2- 172.21.15.7:0/3081301811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556abf5e5f60 0x556abf5e9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:29:18.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.728+0000 7f3c9f59e700 1 --2- 172.21.15.7:0/3081301811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556abf5e6880 0x556abf5e9e50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3c9009c2e0 tx=0x7f3c90017c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f812bc414bc3c008 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:18.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.728+0000 7f3c82ff5700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:29:18.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.728+0000 7f3c82ff5700 1 -- 172.21.15.7:0/3081301811 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9009c5e0 con 0x556abf5e6880 2024-06-10T01:29:18.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.728+0000 7f3c82ff5700 1 -- 172.21.15.7:0/3081301811 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f3c9009c840 con 0x556abf5e6880 2024-06-10T01:29:18.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.728+0000 7f3caf9bcb80 1 -- 172.21.15.7:0/3081301811 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c940043c0 con 0x556abf5e6880 2024-06-10T01:29:18.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.728+0000 7f3c82ff5700 1 -- 172.21.15.7:0/3081301811 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f3c900a0990 con 0x556abf5e6880 2024-06-10T01:29:18.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.728+0000 7f3caf9bcb80 10 client.30055 fetch_fsmap learned FSMap version 55 2024-06-10T01:29:18.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.729+0000 7f3caf9bcb80 1 -- 172.21.15.7:0/3081301811 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x556abf5f7800 con 0x556abf5e6880 2024-06-10T01:29:18.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.729+0000 7f3c82ff5700 1 -- 172.21.15.7:0/3081301811 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c940043c0 con 0x556abf5e6880 2024-06-10T01:29:18.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.730+0000 7f3c82ff5700 1 -- 172.21.15.7:0/3081301811 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f3c900b5220 con 0x556abf5e6880 2024-06-10T01:29:18.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.730+0000 7f3caf9bcb80 10 client.30055 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:29:18.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.730+0000 7f3caf9bcb80 10 client.30055 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:29:18.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.730+0000 7f3caf9bcb80 20 client.30055 populate_metadata read hostname 'smithi007' 2024-06-10T01:29:18.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.730+0000 7f3caf9bcb80 1 --2- 172.21.15.7:0/3081301811 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556abf5f9e10 0x556abf5fc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:18.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.730+0000 7f3caf9bcb80 4 client.30055 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:29:18.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.730+0000 7f3caf9bcb80 1 -- 172.21.15.7:0/3081301811 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556abf53ed50 con 0x556abf5f9e10 2024-06-10T01:29:18.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.730+0000 7f3c9ed9d700 1 --2- 172.21.15.7:0/3081301811 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556abf5f9e10 0x556abf5fc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:18.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.731+0000 7f3c9ed9d700 1 --2- 172.21.15.7:0/3081301811 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556abf5f9e10 0x556abf5fc200 crc :-1 s=READY pgs=521 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cc56f0278e2160a server_cookie=a3a000352c5de359 in_seq=0 out_seq=0 2024-06-10T01:29:18.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.731+0000 7f3c82ff5700 10 client.30055 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:18.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.734+0000 7f3c82ff5700 1 -- 172.21.15.7:0/3081301811 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556abf53ed50 con 0x556abf5f9e10 2024-06-10T01:29:18.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.734+0000 7f3c82ff5700 10 client.30055 handle_command_reply: tid=0 2024-06-10T01:29:18.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.734+0000 7f3caf9bcb80 1 client.30055 shutdown 2024-06-10T01:29:18.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.734+0000 7f3caf9bcb80 1 -- 172.21.15.7:0/3081301811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556abf5e6880 msgr2=0x556abf5e9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:18.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.734+0000 7f3caf9bcb80 1 --2- 172.21.15.7:0/3081301811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556abf5e6880 0x556abf5e9e50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3c9009c2e0 tx=0x7f3c90017c40 comp rx=0 tx=0).stop 2024-06-10T01:29:18.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.735+0000 7f3caf9bcb80 1 -- 172.21.15.7:0/3081301811 shutdown_connections 2024-06-10T01:29:18.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.735+0000 7f3caf9bcb80 1 --2- 172.21.15.7:0/3081301811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556abf5e6880 0x556abf5e9e50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.735+0000 7f3caf9bcb80 1 --2- 172.21.15.7:0/3081301811 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556abf5f9e10 0x556abf5fc200 crc :-1 s=READY pgs=521 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.735+0000 7f3caf9bcb80 1 --2- 172.21.15.7:0/3081301811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556abf5e07a0 0x556abf5ec610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.735+0000 7f3c82ff5700 0 client.30055 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:18.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.735+0000 7f3caf9bcb80 1 --2- 172.21.15.7:0/3081301811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556abf5e5f60 0x556abf5e9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.735+0000 7f3caf9bcb80 1 -- 172.21.15.7:0/3081301811 >> 172.21.15.7:0/3081301811 conn(0x556abf50d080 msgr2=0x556abf5da700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:18.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.735+0000 7f3caf9bcb80 1 -- 172.21.15.7:0/3081301811 shutdown_connections 2024-06-10T01:29:18.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.736+0000 7f3caf9bcb80 1 -- 172.21.15.7:0/3081301811 wait complete. 2024-06-10T01:29:18.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.736+0000 7f3caf9bcb80 20 client.30055 trim_cache size 0 max 16384 2024-06-10T01:29:18.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.765+0000 7f3caf9bcb80 1 Processor -- start 2024-06-10T01:29:18.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.765+0000 7f3caf9bcb80 1 -- start start 2024-06-10T01:29:18.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.765+0000 7f3caf9bcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556abf5e07a0 0x556abf5e8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:18.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.765+0000 7f3caf9bcb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556abf5e5f60 0x556abf5dae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:18.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.765+0000 7f3caf9bcb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556abf5e6880 0x556abf5db3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:18.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.765+0000 7f3caf9bcb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca01c8010 con 0x556abf5e6880 2024-06-10T01:29:18.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.765+0000 7f3caf9bcb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca01c58a0 con 0x556abf5e07a0 2024-06-10T01:29:18.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.765+0000 7f3caf9bcb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca01c36f0 con 0x556abf5e5f60 2024-06-10T01:29:18.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.765+0000 7f3c9f59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556abf5e6880 0x556abf5db3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:18.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.765+0000 7f3c9f59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556abf5e6880 0x556abf5db3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39360/0 (socket says 172.21.15.7:39360) 2024-06-10T01:29:18.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.765+0000 7f3c9f59e700 1 -- 172.21.15.7:0/840862524 learned_addr learned my addr 172.21.15.7:0/840862524 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:18.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.766+0000 7f3c9ed9d700 1 --2- 172.21.15.7:0/840862524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556abf5e07a0 0x556abf5e8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:18.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.766+0000 7f3c9f59e700 1 -- 172.21.15.7:0/840862524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556abf5e5f60 msgr2=0x556abf5dae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:18.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.766+0000 7f3c9f59e700 1 --2- 172.21.15.7:0/840862524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556abf5e5f60 0x556abf5dae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.766+0000 7f3c9f59e700 1 -- 172.21.15.7:0/840862524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556abf5e07a0 msgr2=0x556abf5e8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:18.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.766+0000 7f3c9f59e700 1 --2- 172.21.15.7:0/840862524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556abf5e07a0 0x556abf5e8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.766+0000 7f3c9f59e700 1 -- 172.21.15.7:0/840862524 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556abf5e7ec0 con 0x556abf5e6880 2024-06-10T01:29:18.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.766+0000 7f3c9f59e700 1 --2- 172.21.15.7:0/840862524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556abf5e6880 0x556abf5db3c0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f3c90098f60 tx=0x7f3c90005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eef08ba6291af72 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:18.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.767+0000 7f3c84ff9700 1 -- 172.21.15.7:0/840862524 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9009c5e0 con 0x556abf5e6880 2024-06-10T01:29:18.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.767+0000 7f3c84ff9700 1 -- 172.21.15.7:0/840862524 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c900b5070 con 0x556abf5e6880 2024-06-10T01:29:18.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.767+0000 7f3c84ff9700 1 -- 172.21.15.7:0/840862524 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9009bf40 con 0x556abf5e6880 2024-06-10T01:29:18.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.768+0000 7f3caf9bcb80 1 -- 172.21.15.7:0/840862524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556abf5e6880 msgr2=0x556abf5db3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:18.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.768+0000 7f3caf9bcb80 1 --2- 172.21.15.7:0/840862524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556abf5e6880 0x556abf5db3c0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f3c90098f60 tx=0x7f3c90005fd0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.768+0000 7f3caf9bcb80 1 -- 172.21.15.7:0/840862524 shutdown_connections 2024-06-10T01:29:18.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.768+0000 7f3caf9bcb80 1 --2- 172.21.15.7:0/840862524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556abf5e6880 0x556abf5db3c0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.768+0000 7f3caf9bcb80 1 --2- 172.21.15.7:0/840862524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556abf5e07a0 0x556abf5e8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.768+0000 7f3caf9bcb80 1 --2- 172.21.15.7:0/840862524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556abf5e5f60 0x556abf5dae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.768+0000 7f3caf9bcb80 1 -- 172.21.15.7:0/840862524 >> 172.21.15.7:0/840862524 conn(0x556abf50d080 msgr2=0x556abf5e8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:18.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.768+0000 7f3caf9bcb80 1 -- 172.21.15.7:0/840862524 shutdown_connections 2024-06-10T01:29:18.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.768+0000 7f3caf9bcb80 1 -- 172.21.15.7:0/840862524 wait complete. 2024-06-10T01:29:18.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.769+0000 7f3caf9bcb80 1 -- start start 2024-06-10T01:29:18.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.769+0000 7f3caf9bcb80 1 Processor -- start 2024-06-10T01:29:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.770+0000 7f3caf9bcb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556abf5e07a0 0x556abf5f2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.770+0000 7f3caf9bcb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556abf5e5f60 0x556abf5f2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.770+0000 7f3caf9bcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556abf5e6880 0x556abf5ef4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.770+0000 7f3caf9bcb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca01c3d60 con 0x556abf5e5f60 2024-06-10T01:29:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.770+0000 7f3caf9bcb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca01c51d0 con 0x556abf5e6880 2024-06-10T01:29:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.770+0000 7f3caf9bcb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca01c4430 con 0x556abf5e07a0 2024-06-10T01:29:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.770+0000 7f3caf9bcb80 10 client.0 _ll_register_callbacks cb 0x556abf4c75b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:29:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.771+0000 7f3c9ed9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556abf5e07a0 0x556abf5f2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.771+0000 7f3c9ed9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556abf5e07a0 0x556abf5f2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:35640/0 (socket says 172.21.15.7:35640) 2024-06-10T01:29:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.771+0000 7f3c9ed9d700 1 -- 172.21.15.7:0/3110536515 learned_addr learned my addr 172.21.15.7:0/3110536515 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.771+0000 7f3c9e59c700 1 --2- 172.21.15.7:0/3110536515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556abf5e5f60 0x556abf5f2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.771+0000 7f3c9f59e700 1 --2- 172.21.15.7:0/3110536515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556abf5e6880 0x556abf5ef4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.771+0000 7f3c9ed9d700 1 -- 172.21.15.7:0/3110536515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556abf5e6880 msgr2=0x556abf5ef4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.771+0000 7f3c9ed9d700 1 --2- 172.21.15.7:0/3110536515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556abf5e6880 0x556abf5ef4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.771+0000 7f3c9ed9d700 1 -- 172.21.15.7:0/3110536515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556abf5e5f60 msgr2=0x556abf5f2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.771+0000 7f3c9ed9d700 1 --2- 172.21.15.7:0/3110536515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556abf5e5f60 0x556abf5f2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.771+0000 7f3c9ed9d700 1 -- 172.21.15.7:0/3110536515 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556abf5f3110 con 0x556abf5e07a0 2024-06-10T01:29:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.771+0000 7f3c9f59e700 1 --2- 172.21.15.7:0/3110536515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556abf5e6880 0x556abf5ef4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:29:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.771+0000 7f3c82ff5700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:29:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.771+0000 7f3c9ed9d700 1 --2- 172.21.15.7:0/3110536515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556abf5e07a0 0x556abf5f2520 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x556abf5fca80 tx=0x7f3c98004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90b046dc8739b132 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:18.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.772+0000 7f3c82ff5700 1 -- 172.21.15.7:0/3110536515 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9801d040 con 0x556abf5e07a0 2024-06-10T01:29:18.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.772+0000 7f3caf9bcb80 1 -- 172.21.15.7:0/3110536515 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c900b5070 con 0x556abf5e07a0 2024-06-10T01:29:18.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.772+0000 7f3c82ff5700 1 -- 172.21.15.7:0/3110536515 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f3c98004870 con 0x556abf5e07a0 2024-06-10T01:29:18.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.772+0000 7f3c82ff5700 1 -- 172.21.15.7:0/3110536515 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f3c9803c050 con 0x556abf5e07a0 2024-06-10T01:29:18.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.772+0000 7f3caf9bcb80 10 client.30058 fetch_fsmap learned FSMap version 55 2024-06-10T01:29:18.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.772+0000 7f3caf9bcb80 1 -- 172.21.15.7:0/3110536515 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x556abf665120 con 0x556abf5e07a0 2024-06-10T01:29:18.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.773+0000 7f3c82ff5700 1 -- 172.21.15.7:0/3110536515 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c900b5070 con 0x556abf5e07a0 2024-06-10T01:29:18.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.773+0000 7f3c82ff5700 1 -- 172.21.15.7:0/3110536515 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f3c98033780 con 0x556abf5e07a0 2024-06-10T01:29:18.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.773+0000 7f3caf9bcb80 10 client.30058 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:29:18.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.773+0000 7f3caf9bcb80 10 client.30058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:29:18.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.773+0000 7f3caf9bcb80 20 client.30058 populate_metadata read hostname 'smithi007' 2024-06-10T01:29:18.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.773+0000 7f3caf9bcb80 1 --2- 172.21.15.7:0/3110536515 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556abf5f9e10 0x556abf667850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:18.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.773+0000 7f3caf9bcb80 4 client.30058 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:29:18.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.773+0000 7f3caf9bcb80 1 -- 172.21.15.7:0/3110536515 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556abf5392d0 con 0x556abf5f9e10 2024-06-10T01:29:18.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.773+0000 7f3c9e59c700 1 --2- 172.21.15.7:0/3110536515 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556abf5f9e10 0x556abf667850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:18.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.774+0000 7f3c9e59c700 1 --2- 172.21.15.7:0/3110536515 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556abf5f9e10 0x556abf667850 crc :-1 s=READY pgs=522 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f51f5145e1cb3fb server_cookie=b728482c0e9629c8 in_seq=0 out_seq=0 2024-06-10T01:29:18.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.774+0000 7f3c82ff5700 10 client.30058 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:18.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.775+0000 7f3c82ff5700 1 -- 172.21.15.7:0/3110536515 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556abf5392d0 con 0x556abf5f9e10 2024-06-10T01:29:18.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.775+0000 7f3c82ff5700 10 client.30058 handle_command_reply: tid=0 2024-06-10T01:29:18.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.775+0000 7f3caf9bcb80 1 client.30058 shutdown 2024-06-10T01:29:18.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.775+0000 7f3caf9bcb80 1 -- 172.21.15.7:0/3110536515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556abf5e07a0 msgr2=0x556abf5f2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:18.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.775+0000 7f3caf9bcb80 1 --2- 172.21.15.7:0/3110536515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556abf5e07a0 0x556abf5f2520 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x556abf5fca80 tx=0x7f3c98004b40 comp rx=0 tx=0).stop 2024-06-10T01:29:18.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.775+0000 7f3caf9bcb80 1 -- 172.21.15.7:0/3110536515 shutdown_connections 2024-06-10T01:29:18.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.775+0000 7f3caf9bcb80 1 --2- 172.21.15.7:0/3110536515 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x556abf5f9e10 0x556abf667850 crc :-1 s=READY pgs=522 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.775+0000 7f3caf9bcb80 1 --2- 172.21.15.7:0/3110536515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556abf5e6880 0x556abf5ef4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.775+0000 7f3caf9bcb80 1 --2- 172.21.15.7:0/3110536515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556abf5e07a0 0x556abf5f2520 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.775+0000 7f3caf9bcb80 1 --2- 172.21.15.7:0/3110536515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556abf5e5f60 0x556abf5f2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.775+0000 7f3caf9bcb80 1 -- 172.21.15.7:0/3110536515 >> 172.21.15.7:0/3110536515 conn(0x556abf50d080 msgr2=0x556abf5eb140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:18.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.775+0000 7f3c82ff5700 0 client.30058 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:18.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.776+0000 7f3caf9bcb80 1 -- 172.21.15.7:0/3110536515 shutdown_connections 2024-06-10T01:29:18.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.776+0000 7f3caf9bcb80 1 -- 172.21.15.7:0/3110536515 wait complete. 2024-06-10T01:29:18.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.776+0000 7f3caf9bcb80 20 client.30058 trim_cache size 0 max 16384 2024-06-10T01:29:18.770 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:29:18.770 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T01:29:18.770 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-10T01:29:18.770 INFO:teuthology.orchestra.run.smithi007.stdout: "74d7b72d-6bf0-487e-93ff-1e1128b419b8": { 2024-06-10T01:29:18.770 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-10T01:29:18.770 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "74d7b72d-6bf0-487e-93ff-1e1128b419b8", 2024-06-10T01:29:18.770 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-10T01:29:18.770 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:29:18.770 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:29:18.770 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:29:18.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.777+0000 7f3ca5139700 1 -- 172.21.15.7:0/660837788 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3c880a2d70 msgr2=0x7f3c880a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:18.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.777+0000 7f3ca5139700 1 --2- 172.21.15.7:0/660837788 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3c880a2d70 0x7f3c880a5230 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800f540 tx=0x7f3c9800b040 comp rx=0 tx=0).stop 2024-06-10T01:29:18.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.777+0000 7f3ca5139700 1 -- 172.21.15.7:0/660837788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ca011bc70 msgr2=0x7f3ca01133a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:18.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.777+0000 7f3ca5139700 1 --2- 172.21.15.7:0/660837788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ca011bc70 0x7f3ca01133a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3c9000b2e0 tx=0x7f3c9000b610 comp rx=0 tx=0).stop 2024-06-10T01:29:18.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.777+0000 7f3ca5139700 1 -- 172.21.15.7:0/660837788 shutdown_connections 2024-06-10T01:29:18.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.777+0000 7f3ca5139700 1 --2- 172.21.15.7:0/660837788 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3c880a2d70 0x7f3c880a5230 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.777+0000 7f3ca5139700 1 --2- 172.21.15.7:0/660837788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ca011bc70 0x7f3ca01133a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.777+0000 7f3ca5139700 1 --2- 172.21.15.7:0/660837788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ca01099d0 0x7f3ca011a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.777+0000 7f3ca5139700 1 --2- 172.21.15.7:0/660837788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ca0110c70 0x7f3ca011a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:18.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.777+0000 7f3ca5139700 1 -- 172.21.15.7:0/660837788 >> 172.21.15.7:0/660837788 conn(0x7f3ca006d600 msgr2=0x7f3ca0071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:18.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.777+0000 7f3ca5139700 1 -- 172.21.15.7:0/660837788 shutdown_connections 2024-06-10T01:29:18.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:18.777+0000 7f3ca5139700 1 -- 172.21.15.7:0/660837788 wait complete. 2024-06-10T01:29:19.563 INFO:tasks.workunit.client.0.smithi007.stdout: 10 242390 22.83 MB/sec execute 476 sec latency 86.566 ms 2024-06-10T01:29:19.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:19 smithi175 ceph-mon[36636]: pgmap v1133: 129 pgs: 129 active+clean; 673 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 8.9 MiB/s wr, 744 op/s 2024-06-10T01:29:19.970 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:19 smithi007 ceph-mon[32398]: pgmap v1133: 129 pgs: 129 active+clean; 673 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 8.9 MiB/s wr, 744 op/s 2024-06-10T01:29:20.003 INFO:tasks.cephfs.filesystem:scrub status for tag:74d7b72d-6bf0-487e-93ff-1e1128b419b8 - {'path': '/', 'tag': '74d7b72d-6bf0-487e-93ff-1e1128b419b8', 'options': 'recursive,force'} 2024-06-10T01:29:20.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:19 smithi078 ceph-mon[36683]: pgmap v1133: 129 pgs: 129 active+clean; 673 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 8.9 MiB/s wr, 744 op/s 2024-06-10T01:29:20.563 INFO:tasks.workunit.client.0.smithi007.stdout: 10 243163 22.90 MB/sec execute 477 sec latency 124.643 ms 2024-06-10T01:29:21.563 INFO:tasks.workunit.client.0.smithi007.stdout: 10 243608 22.90 MB/sec execute 478 sec latency 130.149 ms 2024-06-10T01:29:21.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:21 smithi175 ceph-mon[36636]: pgmap v1134: 129 pgs: 129 active+clean; 678 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 5.5 MiB/s wr, 666 op/s 2024-06-10T01:29:22.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:21 smithi007 ceph-mon[32398]: pgmap v1134: 129 pgs: 129 active+clean; 678 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 5.5 MiB/s wr, 666 op/s 2024-06-10T01:29:22.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:21 smithi078 ceph-mon[36683]: pgmap v1134: 129 pgs: 129 active+clean; 678 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 5.5 MiB/s wr, 666 op/s 2024-06-10T01:29:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.093+0000 7f8e4034f640 1 -- 172.21.15.7:0/2754011557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e38107760 msgr2=0x7f8e38109b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.093+0000 7f8e4034f640 1 --2- 172.21.15.7:0/2754011557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e38107760 0x7f8e38109b50 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f8e30009ca0 tx=0x7f8e3002a8c0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.093+0000 7f8e4034f640 1 -- 172.21.15.7:0/2754011557 shutdown_connections 2024-06-10T01:29:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.093+0000 7f8e4034f640 1 --2- 172.21.15.7:0/2754011557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e38107760 0x7f8e38109b50 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.093+0000 7f8e4034f640 1 --2- 172.21.15.7:0/2754011557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e38104e30 0x7f8e38107220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.093+0000 7f8e4034f640 1 --2- 172.21.15.7:0/2754011557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e38102500 0x7f8e381048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.093+0000 7f8e4034f640 1 -- 172.21.15.7:0/2754011557 >> 172.21.15.7:0/2754011557 conn(0x7f8e380fbeb0 msgr2=0x7f8e380fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.093+0000 7f8e4034f640 1 -- 172.21.15.7:0/2754011557 shutdown_connections 2024-06-10T01:29:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.093+0000 7f8e4034f640 1 -- 172.21.15.7:0/2754011557 wait complete. 2024-06-10T01:29:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.094+0000 7f8e4034f640 1 Processor -- start 2024-06-10T01:29:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.094+0000 7f8e4034f640 1 -- start start 2024-06-10T01:29:22.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.094+0000 7f8e4034f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e38102500 0x7f8e38101e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:22.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.094+0000 7f8e4034f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e38104e30 0x7f8e381004b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:22.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.094+0000 7f8e4034f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e38107760 0x7f8e381009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:22.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.094+0000 7f8e4034f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3810d6a0 con 0x7f8e38107760 2024-06-10T01:29:22.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.094+0000 7f8e4034f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3810d520 con 0x7f8e38104e30 2024-06-10T01:29:22.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.094+0000 7f8e4034f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3810d820 con 0x7f8e38102500 2024-06-10T01:29:22.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.094+0000 7f8e3d8ea640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e38104e30 0x7f8e381004b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:22.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.094+0000 7f8e3e8ec640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e38107760 0x7f8e381009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:22.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.094+0000 7f8e3e8ec640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e38107760 0x7f8e381009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56070/0 (socket says 172.21.15.7:56070) 2024-06-10T01:29:22.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.094+0000 7f8e3d8ea640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e38104e30 0x7f8e381004b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:46498/0 (socket says 172.21.15.7:46498) 2024-06-10T01:29:22.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.094+0000 7f8e3e8ec640 1 -- 172.21.15.7:0/2645342526 learned_addr learned my addr 172.21.15.7:0/2645342526 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:22.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.095+0000 7f8e3e8ec640 1 -- 172.21.15.7:0/2645342526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e38102500 msgr2=0x7f8e38101e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:22.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.095+0000 7f8e3e8ec640 1 --2- 172.21.15.7:0/2645342526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e38102500 0x7f8e38101e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.095+0000 7f8e3e8ec640 1 -- 172.21.15.7:0/2645342526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e38104e30 msgr2=0x7f8e381004b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:22.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.095+0000 7f8e3e8ec640 1 --2- 172.21.15.7:0/2645342526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e38104e30 0x7f8e381004b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.095+0000 7f8e3e8ec640 1 -- 172.21.15.7:0/2645342526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e38101280 con 0x7f8e38107760 2024-06-10T01:29:22.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.095+0000 7f8e3e8ec640 1 --2- 172.21.15.7:0/2645342526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e38107760 0x7f8e381009f0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f8e30031ae0 tx=0x7f8e30031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:22.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.095+0000 7f8e2f7fe640 1 -- 172.21.15.7:0/2645342526 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e30003b70 con 0x7f8e38107760 2024-06-10T01:29:22.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.095+0000 7f8e2f7fe640 1 -- 172.21.15.7:0/2645342526 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e30003d10 con 0x7f8e38107760 2024-06-10T01:29:22.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.095+0000 7f8e2f7fe640 1 -- 172.21.15.7:0/2645342526 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3002fc40 con 0x7f8e38107760 2024-06-10T01:29:22.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.095+0000 7f8e4034f640 1 -- 172.21.15.7:0/2645342526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e381a1de0 con 0x7f8e38107760 2024-06-10T01:29:22.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.096+0000 7f8e4034f640 1 -- 172.21.15.7:0/2645342526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e381a2190 con 0x7f8e38107760 2024-06-10T01:29:22.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.099+0000 7f8e2f7fe640 1 -- 172.21.15.7:0/2645342526 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f8e30004050 con 0x7f8e38107760 2024-06-10T01:29:22.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.099+0000 7f8e4305b740 1 Processor -- start 2024-06-10T01:29:22.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.100+0000 7f8e4305b740 1 -- start start 2024-06-10T01:29:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.100+0000 7f8e4305b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560810a96010 0x560810a96430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.100+0000 7f8e4305b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560810a96a00 0x560810a9b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.100+0000 7f8e3d8ea640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560810a96010 0x560810a96430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.100+0000 7f8e3d8ea640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560810a96010 0x560810a96430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56082/0 (socket says 172.21.15.7:56082) 2024-06-10T01:29:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.100+0000 7f8e3d8ea640 1 -- 172.21.15.7:0/2994213798 learned_addr learned my addr 172.21.15.7:0/2994213798 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.101+0000 7f8e4305b740 1 --2- 172.21.15.7:0/2994213798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560810a9bb70 0x560810a9e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.101+0000 7f8e4305b740 1 -- 172.21.15.7:0/2994213798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5608109f94f0 con 0x560810a96010 2024-06-10T01:29:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.101+0000 7f8e3e0eb640 1 --2- 172.21.15.7:0/2994213798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560810a96a00 0x560810a9b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.101+0000 7f8e4305b740 1 -- 172.21.15.7:0/2994213798 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560810997870 con 0x560810a9bb70 2024-06-10T01:29:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.101+0000 7f8e4305b740 1 -- 172.21.15.7:0/2994213798 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5608109f54b0 con 0x560810a96a00 2024-06-10T01:29:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.101+0000 7f8e2f7fe640 1 --2- 172.21.15.7:0/2645342526 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8e240a2c00 0x7f8e240a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.101+0000 7f8e3d8ea640 1 --2- 172.21.15.7:0/2994213798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560810a9bb70 0x560810a9e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.101+0000 7f8e3d8ea640 1 -- 172.21.15.7:0/2994213798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560810a96a00 msgr2=0x560810a9b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.101+0000 7f8e3d8ea640 1 --2- 172.21.15.7:0/2994213798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560810a96a00 0x560810a9b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.101+0000 7f8e3d8ea640 1 -- 172.21.15.7:0/2994213798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560810a9bb70 msgr2=0x560810a9e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.101+0000 7f8e3d8ea640 1 --2- 172.21.15.7:0/2994213798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560810a9bb70 0x560810a9e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.101+0000 7f8e3d8ea640 1 -- 172.21.15.7:0/2994213798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608109b39b0 con 0x560810a96010 2024-06-10T01:29:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.102+0000 7f8e3d8ea640 1 --2- 172.21.15.7:0/2994213798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560810a96010 0x560810a96430 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f8e28009fd0 tx=0x7f8e28009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83c026d4696c07f7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.102+0000 7f8e2f7fe640 1 -- 172.21.15.7:0/2645342526 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f8e300f0370 con 0x7f8e38107760 2024-06-10T01:29:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.102+0000 7f8e2cbfe640 1 -- 172.21.15.7:0/2994213798 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e28026070 con 0x560810a96010 2024-06-10T01:29:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.102+0000 7f8e2cbfe640 1 -- 172.21.15.7:0/2994213798 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e28004250 con 0x560810a96010 2024-06-10T01:29:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.102+0000 7f8e2cbfe640 1 -- 172.21.15.7:0/2994213798 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e28003dd0 con 0x560810a96010 2024-06-10T01:29:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.103+0000 7f8e4305b740 1 -- 172.21.15.7:0/2994213798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560810a96010 msgr2=0x560810a96430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.103+0000 7f8e4305b740 1 --2- 172.21.15.7:0/2994213798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560810a96010 0x560810a96430 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f8e28009fd0 tx=0x7f8e28009910 comp rx=0 tx=0).stop 2024-06-10T01:29:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.104+0000 7f8e3e8ec640 1 --2- 172.21.15.7:0/2645342526 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8e240a2c00 0x7f8e240a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.104+0000 7f8e4305b740 1 -- 172.21.15.7:0/2994213798 shutdown_connections 2024-06-10T01:29:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.104+0000 7f8e4305b740 1 --2- 172.21.15.7:0/2994213798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560810a9bb70 0x560810a9e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.104+0000 7f8e4305b740 1 --2- 172.21.15.7:0/2994213798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560810a96a00 0x560810a9b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.104+0000 7f8e4305b740 1 --2- 172.21.15.7:0/2994213798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560810a96010 0x560810a96430 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.104+0000 7f8e4305b740 1 -- 172.21.15.7:0/2994213798 >> 172.21.15.7:0/2994213798 conn(0x560810a91680 msgr2=0x560810a91a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.104+0000 7f8e4305b740 1 -- 172.21.15.7:0/2994213798 shutdown_connections 2024-06-10T01:29:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.104+0000 7f8e4305b740 1 -- 172.21.15.7:0/2994213798 wait complete. 2024-06-10T01:29:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.104+0000 7f8e3e8ec640 1 --2- 172.21.15.7:0/2645342526 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8e240a2c00 0x7f8e240a50c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f8e30009620 tx=0x7f8e3002a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.105+0000 7f8e4305b740 1 -- start start 2024-06-10T01:29:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.105+0000 7f8e4305b740 1 Processor -- start 2024-06-10T01:29:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.105+0000 7f8e4305b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560810a96010 0x560810aa0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.105+0000 7f8e4305b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560810a96a00 0x560810aa08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.105+0000 7f8e4305b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560810a9bb70 0x560810aa7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.105+0000 7f8e4305b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560810a9eb20 con 0x560810a9bb70 2024-06-10T01:29:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.105+0000 7f8e4305b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560810982ae0 con 0x560810a96a00 2024-06-10T01:29:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.105+0000 7f8e3e0eb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560810a96a00 0x560810aa08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.105+0000 7f8e4305b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560810a9eca0 con 0x560810a96010 2024-06-10T01:29:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.105+0000 7f8e3e0eb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560810a96a00 0x560810aa08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:46508/0 (socket says 172.21.15.7:46508) 2024-06-10T01:29:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.106+0000 7f8e3e0eb640 1 -- 172.21.15.7:0/2249260410 learned_addr learned my addr 172.21.15.7:0/2249260410 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.105+0000 7f8e3d8ea640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560810a96010 0x560810aa0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.106+0000 7f8e4305b740 10 client.0 _ll_register_callbacks cb 0x5608109bb9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:29:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.106+0000 7f8e3d8ea640 1 --2- 172.21.15.7:0/2249260410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560810a9bb70 0x560810aa7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.106+0000 7f8e3e0eb640 1 -- 172.21.15.7:0/2249260410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560810a96010 msgr2=0x560810aa0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.106+0000 7f8e3e0eb640 1 --2- 172.21.15.7:0/2249260410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560810a96010 0x560810aa0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.106+0000 7f8e3e0eb640 1 -- 172.21.15.7:0/2249260410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560810a9bb70 msgr2=0x560810aa7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.106+0000 7f8e3e0eb640 1 --2- 172.21.15.7:0/2249260410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560810a9bb70 0x560810aa7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.106+0000 7f8e3e0eb640 1 -- 172.21.15.7:0/2249260410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56081070d210 con 0x560810a96a00 2024-06-10T01:29:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.106+0000 7f8e3e0eb640 1 --2- 172.21.15.7:0/2249260410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560810a96a00 0x560810aa08c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8e340116f0 tx=0x7f8e34011bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa2152795ad9d654 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.106+0000 7f8e21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:29:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.106+0000 7f8e21ffb640 1 -- 172.21.15.7:0/2249260410 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e34004460 con 0x560810a96a00 2024-06-10T01:29:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.107+0000 7f8e21ffb640 1 -- 172.21.15.7:0/2249260410 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f8e3400f920 con 0x560810a96a00 2024-06-10T01:29:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.107+0000 7f8e4305b740 1 -- 172.21.15.7:0/2249260410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e28004250 con 0x560810a96a00 2024-06-10T01:29:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.107+0000 7f8e21ffb640 1 -- 172.21.15.7:0/2249260410 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f8e3400fb20 con 0x560810a96a00 2024-06-10T01:29:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.107+0000 7f8e4305b740 10 client.30161 fetch_fsmap learned FSMap version 55 2024-06-10T01:29:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.107+0000 7f8e4305b740 1 -- 172.21.15.7:0/2249260410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5608107d9620 con 0x560810a96a00 2024-06-10T01:29:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.108+0000 7f8e21ffb640 1 -- 172.21.15.7:0/2249260410 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e28004250 con 0x560810a96a00 2024-06-10T01:29:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.108+0000 7f8e21ffb640 1 -- 172.21.15.7:0/2249260410 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f8e34018070 con 0x560810a96a00 2024-06-10T01:29:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.108+0000 7f8e4305b740 10 client.30161 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:29:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.108+0000 7f8e4305b740 10 client.30161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:29:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.108+0000 7f8e4305b740 20 client.30161 populate_metadata read hostname 'smithi007' 2024-06-10T01:29:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.108+0000 7f8e4305b740 1 --2- 172.21.15.7:0/2249260410 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560810aac240 0x560810aae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.108+0000 7f8e4305b740 4 client.30161 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:29:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.108+0000 7f8e4305b740 1 -- 172.21.15.7:0/2249260410 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560810a0cf10 con 0x560810aac240 2024-06-10T01:29:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.108+0000 7f8e3d8ea640 1 --2- 172.21.15.7:0/2249260410 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560810aac240 0x560810aae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.109+0000 7f8e3d8ea640 1 --2- 172.21.15.7:0/2249260410 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560810aac240 0x560810aae6e0 crc :-1 s=READY pgs=523 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=773a777562096b02 server_cookie=c0d382b57909bd99 in_seq=0 out_seq=0 2024-06-10T01:29:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.109+0000 7f8e21ffb640 10 client.30161 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.112+0000 7f8e21ffb640 1 -- 172.21.15.7:0/2249260410 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560810a0cf10 con 0x560810aac240 2024-06-10T01:29:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.112+0000 7f8e21ffb640 10 client.30161 handle_command_reply: tid=0 2024-06-10T01:29:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.112+0000 7f8e4305b740 1 client.30161 shutdown 2024-06-10T01:29:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.112+0000 7f8e4305b740 1 -- 172.21.15.7:0/2249260410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560810a96a00 msgr2=0x560810aa08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.112+0000 7f8e4305b740 1 --2- 172.21.15.7:0/2249260410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560810a96a00 0x560810aa08c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8e340116f0 tx=0x7f8e34011bb0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.112+0000 7f8e4305b740 1 -- 172.21.15.7:0/2249260410 shutdown_connections 2024-06-10T01:29:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.112+0000 7f8e4305b740 1 --2- 172.21.15.7:0/2249260410 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560810aac240 0x560810aae6e0 crc :-1 s=READY pgs=523 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.112+0000 7f8e4305b740 1 --2- 172.21.15.7:0/2249260410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560810a9bb70 0x560810aa7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.112+0000 7f8e21ffb640 0 client.30161 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.112+0000 7f8e4305b740 1 --2- 172.21.15.7:0/2249260410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560810a96a00 0x560810aa08c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.112+0000 7f8e4305b740 1 --2- 172.21.15.7:0/2249260410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560810a96010 0x560810aa0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.112+0000 7f8e4305b740 1 -- 172.21.15.7:0/2249260410 >> 172.21.15.7:0/2249260410 conn(0x560810a91680 msgr2=0x560810a907e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.112+0000 7f8e4305b740 1 -- 172.21.15.7:0/2249260410 shutdown_connections 2024-06-10T01:29:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.112+0000 7f8e4305b740 1 -- 172.21.15.7:0/2249260410 wait complete. 2024-06-10T01:29:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.113+0000 7f8e4305b740 20 client.30161 trim_cache size 0 max 16384 2024-06-10T01:29:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.134+0000 7f8e4305b740 1 Processor -- start 2024-06-10T01:29:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.134+0000 7f8e4305b740 1 -- start start 2024-06-10T01:29:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.135+0000 7f8e4305b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560810a96010 0x560810a9df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.135+0000 7f8e4305b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560810a96a00 0x560810aa78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.135+0000 7f8e4305b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560810a9bb70 0x560810aa0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.135+0000 7f8e4305b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8e381aac70 con 0x560810a9bb70 2024-06-10T01:29:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.135+0000 7f8e4305b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8e381a8540 con 0x560810a96010 2024-06-10T01:29:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.135+0000 7f8e4305b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8e381a6390 con 0x560810a96a00 2024-06-10T01:29:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.135+0000 7f8e3d8ea640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560810a9bb70 0x560810aa0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.135+0000 7f8e3e0eb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560810a96a00 0x560810aa78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.135+0000 7f8e3d8ea640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560810a9bb70 0x560810aa0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56108/0 (socket says 172.21.15.7:56108) 2024-06-10T01:29:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.135+0000 7f8e3d8ea640 1 -- 172.21.15.7:0/1857046195 learned_addr learned my addr 172.21.15.7:0/1857046195 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.135+0000 7f8e3d8ea640 1 --2- 172.21.15.7:0/1857046195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560810a96010 0x560810a9df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.135+0000 7f8e3d8ea640 1 -- 172.21.15.7:0/1857046195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560810a96a00 msgr2=0x560810aa78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.135+0000 7f8e3d8ea640 1 --2- 172.21.15.7:0/1857046195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560810a96a00 0x560810aa78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.135+0000 7f8e3d8ea640 1 -- 172.21.15.7:0/1857046195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560810a96010 msgr2=0x560810a9df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.135+0000 7f8e3d8ea640 1 --2- 172.21.15.7:0/1857046195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560810a96010 0x560810a9df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.135+0000 7f8e3d8ea640 1 -- 172.21.15.7:0/1857046195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560810a92350 con 0x560810a9bb70 2024-06-10T01:29:22.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.135+0000 7f8e3d8ea640 1 --2- 172.21.15.7:0/1857046195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560810a9bb70 0x560810aa0380 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f8e28005ff0 tx=0x7f8e28009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=782219ae2d017b6a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:22.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.136+0000 7f8e23fff640 1 -- 172.21.15.7:0/1857046195 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e28026030 con 0x560810a9bb70 2024-06-10T01:29:22.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.136+0000 7f8e23fff640 1 -- 172.21.15.7:0/1857046195 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e28027040 con 0x560810a9bb70 2024-06-10T01:29:22.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.136+0000 7f8e23fff640 1 -- 172.21.15.7:0/1857046195 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e280038b0 con 0x560810a9bb70 2024-06-10T01:29:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.137+0000 7f8e4305b740 1 -- 172.21.15.7:0/1857046195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560810a9bb70 msgr2=0x560810aa0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.137+0000 7f8e4305b740 1 --2- 172.21.15.7:0/1857046195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560810a9bb70 0x560810aa0380 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f8e28005ff0 tx=0x7f8e28009850 comp rx=0 tx=0).stop 2024-06-10T01:29:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.137+0000 7f8e4305b740 1 -- 172.21.15.7:0/1857046195 shutdown_connections 2024-06-10T01:29:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.137+0000 7f8e4305b740 1 --2- 172.21.15.7:0/1857046195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560810a9bb70 0x560810aa0380 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.137+0000 7f8e4305b740 1 --2- 172.21.15.7:0/1857046195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560810a96a00 0x560810aa78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.137+0000 7f8e4305b740 1 --2- 172.21.15.7:0/1857046195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560810a96010 0x560810a9df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.137+0000 7f8e4305b740 1 -- 172.21.15.7:0/1857046195 >> 172.21.15.7:0/1857046195 conn(0x560810a91680 msgr2=0x560810a91a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.137+0000 7f8e4305b740 1 -- 172.21.15.7:0/1857046195 shutdown_connections 2024-06-10T01:29:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.137+0000 7f8e4305b740 1 -- 172.21.15.7:0/1857046195 wait complete. 2024-06-10T01:29:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.137+0000 7f8e4305b740 1 -- start start 2024-06-10T01:29:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.137+0000 7f8e4305b740 1 Processor -- start 2024-06-10T01:29:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.138+0000 7f8e4305b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560810a96010 0x560810aa77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.138+0000 7f8e4305b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560810a96a00 0x560810aa42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.138+0000 7f8e4305b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560810a9bb70 0x560810aa4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.138+0000 7f8e4305b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8e381a6a00 con 0x560810a96010 2024-06-10T01:29:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.138+0000 7f8e4305b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8e381a7e70 con 0x560810a96a00 2024-06-10T01:29:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.138+0000 7f8e4305b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8e381a70d0 con 0x560810a9bb70 2024-06-10T01:29:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.138+0000 7f8e4305b740 10 client.0 _ll_register_callbacks cb 0x7f8e34009710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:29:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.138+0000 7f8e3e0eb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560810a96a00 0x560810aa42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.138+0000 7f8e3d8ea640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560810a96010 0x560810aa77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.138+0000 7f8e3e0eb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560810a96a00 0x560810aa42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:46516/0 (socket says 172.21.15.7:46516) 2024-06-10T01:29:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.138+0000 7f8e3e0eb640 1 -- 172.21.15.7:0/2608759372 learned_addr learned my addr 172.21.15.7:0/2608759372 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.138+0000 7f8e3d8ea640 1 --2- 172.21.15.7:0/2608759372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560810a9bb70 0x560810aa4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.138+0000 7f8e3e0eb640 1 -- 172.21.15.7:0/2608759372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560810a9bb70 msgr2=0x560810aa4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.138+0000 7f8e3e0eb640 1 --2- 172.21.15.7:0/2608759372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560810a9bb70 0x560810aa4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.138+0000 7f8e3e0eb640 1 -- 172.21.15.7:0/2608759372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560810a96010 msgr2=0x560810aa77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.138+0000 7f8e3e0eb640 1 --2- 172.21.15.7:0/2608759372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560810a96010 0x560810aa77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.138+0000 7f8e3e0eb640 1 -- 172.21.15.7:0/2608759372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560810b14fb0 con 0x560810a96a00 2024-06-10T01:29:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.138+0000 7f8e3d8ea640 1 --2- 172.21.15.7:0/2608759372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560810a9bb70 0x560810aa4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:29:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.139+0000 7f8e3e0eb640 1 --2- 172.21.15.7:0/2608759372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560810a96a00 0x560810aa42c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x560810a92520 tx=0x7f8e340049d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=529b4ce5593f062c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.139+0000 7f8e21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:29:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.139+0000 7f8e21ffb640 1 -- 172.21.15.7:0/2608759372 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e34005c50 con 0x560810a96a00 2024-06-10T01:29:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.139+0000 7f8e4305b740 1 -- 172.21.15.7:0/2608759372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e28027040 con 0x560810a96a00 2024-06-10T01:29:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.139+0000 7f8e21ffb640 1 -- 172.21.15.7:0/2608759372 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f8e34004470 con 0x560810a96a00 2024-06-10T01:29:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.139+0000 7f8e21ffb640 1 -- 172.21.15.7:0/2608759372 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f8e34037050 con 0x560810a96a00 2024-06-10T01:29:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.139+0000 7f8e4305b740 10 client.30167 fetch_fsmap learned FSMap version 55 2024-06-10T01:29:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.139+0000 7f8e4305b740 1 -- 172.21.15.7:0/2608759372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x560810b16310 con 0x560810a96a00 2024-06-10T01:29:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.139+0000 7f8e21ffb640 1 -- 172.21.15.7:0/2608759372 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e28027040 con 0x560810a96a00 2024-06-10T01:29:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.140+0000 7f8e21ffb640 1 -- 172.21.15.7:0/2608759372 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f8e34018070 con 0x560810a96a00 2024-06-10T01:29:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.140+0000 7f8e4305b740 10 client.30167 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:29:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.140+0000 7f8e4305b740 10 client.30167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:29:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.140+0000 7f8e4305b740 20 client.30167 populate_metadata read hostname 'smithi007' 2024-06-10T01:29:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.140+0000 7f8e4305b740 1 --2- 172.21.15.7:0/2608759372 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560810aac240 0x560810b18d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.140+0000 7f8e4305b740 4 client.30167 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:29:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.140+0000 7f8e4305b740 1 -- 172.21.15.7:0/2608759372 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560810a06c20 con 0x560810aac240 2024-06-10T01:29:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.142+0000 7f8e3d8ea640 1 --2- 172.21.15.7:0/2608759372 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560810aac240 0x560810b18d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.143+0000 7f8e3d8ea640 1 --2- 172.21.15.7:0/2608759372 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560810aac240 0x560810b18d30 crc :-1 s=READY pgs=524 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74f8b0d292609493 server_cookie=77adc28082c3bda6 in_seq=0 out_seq=0 2024-06-10T01:29:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.143+0000 7f8e21ffb640 10 client.30167 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:22.563 INFO:tasks.workunit.client.0.smithi007.stdout: 10 243948 22.88 MB/sec execute 479 sec latency 334.770 ms 2024-06-10T01:29:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.978+0000 7f8e21ffb640 1 -- 172.21.15.7:0/2608759372 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560810a06c20 con 0x560810aac240 2024-06-10T01:29:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.978+0000 7f8e21ffb640 10 client.30167 handle_command_reply: tid=0 2024-06-10T01:29:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.978+0000 7f8e4305b740 1 client.30167 shutdown 2024-06-10T01:29:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.979+0000 7f8e4305b740 1 -- 172.21.15.7:0/2608759372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560810a96a00 msgr2=0x560810aa42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.979+0000 7f8e4305b740 1 --2- 172.21.15.7:0/2608759372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560810a96a00 0x560810aa42c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x560810a92520 tx=0x7f8e340049d0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.979+0000 7f8e4305b740 1 -- 172.21.15.7:0/2608759372 shutdown_connections 2024-06-10T01:29:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.979+0000 7f8e4305b740 1 --2- 172.21.15.7:0/2608759372 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x560810aac240 0x560810b18d30 crc :-1 s=READY pgs=524 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.979+0000 7f8e4305b740 1 --2- 172.21.15.7:0/2608759372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560810a9bb70 0x560810aa4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.979+0000 7f8e21ffb640 0 client.30167 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.979+0000 7f8e4305b740 1 --2- 172.21.15.7:0/2608759372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560810a96a00 0x560810aa42c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.979+0000 7f8e4305b740 1 --2- 172.21.15.7:0/2608759372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560810a96010 0x560810aa77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.979+0000 7f8e4305b740 1 -- 172.21.15.7:0/2608759372 >> 172.21.15.7:0/2608759372 conn(0x560810a91680 msgr2=0x560810aa0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.979+0000 7f8e4305b740 1 -- 172.21.15.7:0/2608759372 shutdown_connections 2024-06-10T01:29:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.979+0000 7f8e4305b740 1 -- 172.21.15.7:0/2608759372 wait complete. 2024-06-10T01:29:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.980+0000 7f8e4305b740 20 client.30167 trim_cache size 0 max 16384 2024-06-10T01:29:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:29:22.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:29:22.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:29:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.980+0000 7f8e4034f640 1 -- 172.21.15.7:0/2645342526 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8e240a2c00 msgr2=0x7f8e240a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.981+0000 7f8e4034f640 1 --2- 172.21.15.7:0/2645342526 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8e240a2c00 0x7f8e240a50c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f8e30009620 tx=0x7f8e3002a8c0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.981+0000 7f8e4034f640 1 -- 172.21.15.7:0/2645342526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e38107760 msgr2=0x7f8e381009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.981+0000 7f8e4034f640 1 --2- 172.21.15.7:0/2645342526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e38107760 0x7f8e381009f0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f8e30031ae0 tx=0x7f8e30031ee0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.981+0000 7f8e4034f640 1 -- 172.21.15.7:0/2645342526 shutdown_connections 2024-06-10T01:29:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.981+0000 7f8e4034f640 1 --2- 172.21.15.7:0/2645342526 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f8e240a2c00 0x7f8e240a50c0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.981+0000 7f8e4034f640 1 --2- 172.21.15.7:0/2645342526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e38107760 0x7f8e381009f0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.981+0000 7f8e4034f640 1 --2- 172.21.15.7:0/2645342526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e38104e30 0x7f8e381004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.981+0000 7f8e4034f640 1 --2- 172.21.15.7:0/2645342526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e38102500 0x7f8e38101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.981+0000 7f8e4034f640 1 -- 172.21.15.7:0/2645342526 >> 172.21.15.7:0/2645342526 conn(0x7f8e380fbeb0 msgr2=0x7f8e38102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.981+0000 7f8e4034f640 1 -- 172.21.15.7:0/2645342526 shutdown_connections 2024-06-10T01:29:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:22.982+0000 7f8e4034f640 1 -- 172.21.15.7:0/2645342526 wait complete. 2024-06-10T01:29:23.563 INFO:tasks.workunit.client.0.smithi007.stdout: 10 244283 22.87 MB/sec execute 480 sec latency 584.943 ms 2024-06-10T01:29:23.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:23 smithi175 ceph-mon[36636]: pgmap v1135: 129 pgs: 129 active+clean; 686 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.1 MiB/s wr, 643 op/s 2024-06-10T01:29:24.008 INFO:tasks.quiescer.fs.[cephfs]:Total duration (44.9) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-06-10T01:29:24.008 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.45 2024-06-10T01:29:24.009 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:29:24.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:23 smithi007 ceph-mon[32398]: pgmap v1135: 129 pgs: 129 active+clean; 686 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.1 MiB/s wr, 643 op/s 2024-06-10T01:29:24.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:23 smithi078 ceph-mon[36683]: pgmap v1135: 129 pgs: 129 active+clean; 686 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.1 MiB/s wr, 643 op/s 2024-06-10T01:29:24.376 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:29:24.563 INFO:tasks.workunit.client.0.smithi007.stdout: 10 245053 22.88 MB/sec execute 481 sec latency 69.584 ms 2024-06-10T01:29:25.563 INFO:tasks.workunit.client.0.smithi007.stdout: 10 245829 22.95 MB/sec execute 482 sec latency 63.606 ms 2024-06-10T01:29:25.878 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:25 smithi078 ceph-mon[36683]: pgmap v1136: 129 pgs: 129 active+clean; 689 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.3 MiB/s wr, 508 op/s 2024-06-10T01:29:25.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:25 smithi175 ceph-mon[36636]: pgmap v1136: 129 pgs: 129 active+clean; 689 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.3 MiB/s wr, 508 op/s 2024-06-10T01:29:26.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:25 smithi007 ceph-mon[32398]: pgmap v1136: 129 pgs: 129 active+clean; 689 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.3 MiB/s wr, 508 op/s 2024-06-10T01:29:26.563 INFO:tasks.workunit.client.0.smithi007.stdout: 10 246415 22.98 MB/sec execute 483 sec latency 102.284 ms 2024-06-10T01:29:27.563 INFO:tasks.workunit.client.0.smithi007.stdout: 10 246923 22.98 MB/sec execute 484 sec latency 115.002 ms 2024-06-10T01:29:27.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:27 smithi175 ceph-mon[36636]: pgmap v1137: 129 pgs: 129 active+clean; 687 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 3.4 MiB/s wr, 567 op/s 2024-06-10T01:29:28.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:27 smithi007 ceph-mon[32398]: pgmap v1137: 129 pgs: 129 active+clean; 687 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 3.4 MiB/s wr, 567 op/s 2024-06-10T01:29:28.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:27 smithi078 ceph-mon[36683]: pgmap v1137: 129 pgs: 129 active+clean; 687 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 3.4 MiB/s wr, 567 op/s 2024-06-10T01:29:28.563 INFO:tasks.workunit.client.0.smithi007.stdout: 10 247309 22.97 MB/sec execute 485 sec latency 163.515 ms 2024-06-10T01:29:28.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.575+0000 7f2161377700 1 -- 172.21.15.7:0/4259564660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f215c101690 msgr2=0x7f215c101b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:28.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.575+0000 7f2161377700 1 --2- 172.21.15.7:0/4259564660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f215c101690 0x7f215c101b10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f215400b0a0 tx=0x7f215400b3b0 comp rx=0 tx=0).stop 2024-06-10T01:29:28.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.576+0000 7f2161377700 1 -- 172.21.15.7:0/4259564660 shutdown_connections 2024-06-10T01:29:28.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.576+0000 7f2161377700 1 --2- 172.21.15.7:0/4259564660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f215c102050 0x7f215c071ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:28.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.576+0000 7f2161377700 1 --2- 172.21.15.7:0/4259564660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f215c103350 0x7f215c103730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:28.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.576+0000 7f2161377700 1 --2- 172.21.15.7:0/4259564660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f215c101690 0x7f215c101b10 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:28.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.576+0000 7f2161377700 1 -- 172.21.15.7:0/4259564660 >> 172.21.15.7:0/4259564660 conn(0x7f215c078190 msgr2=0x7f215c0785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:28.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.576+0000 7f2161377700 1 -- 172.21.15.7:0/4259564660 shutdown_connections 2024-06-10T01:29:28.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.576+0000 7f2161377700 1 -- 172.21.15.7:0/4259564660 wait complete. 2024-06-10T01:29:28.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.577+0000 7f2161377700 1 Processor -- start 2024-06-10T01:29:28.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.577+0000 7f2161377700 1 -- start start 2024-06-10T01:29:28.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.578+0000 7f2161377700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f215c102050 0x7f215c1a33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:28.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.578+0000 7f2161377700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f215c103350 0x7f215c1a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:28.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.578+0000 7f2161377700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f215c1a7c90 0x7f215c1a8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:28.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.578+0000 7f2161377700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f215c074320 con 0x7f215c102050 2024-06-10T01:29:28.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.578+0000 7f2161377700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f215c0741a0 con 0x7f215c103350 2024-06-10T01:29:28.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.578+0000 7f2161377700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f215c0744a0 con 0x7f215c1a7c90 2024-06-10T01:29:28.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.578+0000 7f215affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f215c102050 0x7f215c1a33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:28.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.578+0000 7f215affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f215c102050 0x7f215c1a33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56126/0 (socket says 172.21.15.7:56126) 2024-06-10T01:29:28.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.578+0000 7f215affd700 1 -- 172.21.15.7:0/2807740113 learned_addr learned my addr 172.21.15.7:0/2807740113 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:28.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.578+0000 7f215b7fe700 1 --2- 172.21.15.7:0/2807740113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f215c1a7c90 0x7f215c1a8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:28.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.579+0000 7f215b7fe700 1 -- 172.21.15.7:0/2807740113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f215c103350 msgr2=0x7f215c1a3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:28.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.579+0000 7f215b7fe700 1 --2- 172.21.15.7:0/2807740113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f215c103350 0x7f215c1a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:28.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.579+0000 7f215b7fe700 1 -- 172.21.15.7:0/2807740113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f215c102050 msgr2=0x7f215c1a33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:28.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.579+0000 7f215b7fe700 1 --2- 172.21.15.7:0/2807740113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f215c102050 0x7f215c1a33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:28.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.579+0000 7f215b7fe700 1 -- 172.21.15.7:0/2807740113 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215c1a8900 con 0x7f215c1a7c90 2024-06-10T01:29:28.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.580+0000 7f215b7fe700 1 --2- 172.21.15.7:0/2807740113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f215c1a7c90 0x7f215c1a8140 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f215000b410 tx=0x7f215000b720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:28.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.585+0000 7f2143fff700 1 -- 172.21.15.7:0/2807740113 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2150015020 con 0x7f215c1a7c90 2024-06-10T01:29:28.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.585+0000 7f2161377700 1 -- 172.21.15.7:0/2807740113 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f215c10e360 con 0x7f215c1a7c90 2024-06-10T01:29:28.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.585+0000 7f2161377700 1 -- 172.21.15.7:0/2807740113 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f215c10e8a0 con 0x7f215c1a7c90 2024-06-10T01:29:28.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.585+0000 7f2143fff700 1 -- 172.21.15.7:0/2807740113 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2150011d10 con 0x7f215c1a7c90 2024-06-10T01:29:28.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.585+0000 7f2143fff700 1 -- 172.21.15.7:0/2807740113 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2150010c50 con 0x7f215c1a7c90 2024-06-10T01:29:28.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.586+0000 7f2161377700 1 -- 172.21.15.7:0/2807740113 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2148005330 con 0x7f215c1a7c90 2024-06-10T01:29:28.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.589+0000 7f2143fff700 1 -- 172.21.15.7:0/2807740113 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f2150018020 con 0x7f215c1a7c90 2024-06-10T01:29:28.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.590+0000 7f2143fff700 1 --2- 172.21.15.7:0/2807740113 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21440a2e30 0x7f21440a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:28.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.590+0000 7f2143fff700 1 -- 172.21.15.7:0/2807740113 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f21500cedc0 con 0x7f215c1a7c90 2024-06-10T01:29:28.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.593+0000 7f215affd700 1 --2- 172.21.15.7:0/2807740113 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21440a2e30 0x7f21440a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:28.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.597+0000 7f2143fff700 1 -- 172.21.15.7:0/2807740113 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f215001f020 con 0x7f215c1a7c90 2024-06-10T01:29:28.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:28.609+0000 7f215affd700 1 --2- 172.21.15.7:0/2807740113 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21440a2e30 0x7f21440a52f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f214c009d20 tx=0x7f214c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:28.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:28 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:29:28.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:28 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:29:28.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:28 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:29:29.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:28 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:29:29.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:28 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:29:29.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:28 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:29:29.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:28 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:29:29.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:28 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:29:29.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:28 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:29:29.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:29.106+0000 7f2161377700 1 -- 172.21.15.7:0/2807740113 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2148005620 con 0x7f215c1a7c90 2024-06-10T01:29:29.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:29.107+0000 7f2143fff700 1 -- 172.21.15.7:0/2807740113 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11852 (secure 0 0 0) 0x7f215009b9c0 con 0x7f215c1a7c90 2024-06-10T01:29:29.101 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:29:29.101 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:27:44.542390+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29813":{"gid":29813,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/138896429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":138896429},{"type":"v1","addr":"172.21.15.175:6835","nonce":138896429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24451,24434,24436]},"id":1}]} 2024-06-10T01:29:29.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:29.112+0000 7f2161377700 1 -- 172.21.15.7:0/2807740113 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21440a2e30 msgr2=0x7f21440a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:29.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:29.112+0000 7f2161377700 1 --2- 172.21.15.7:0/2807740113 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21440a2e30 0x7f21440a52f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f214c009d20 tx=0x7f214c009260 comp rx=0 tx=0).stop 2024-06-10T01:29:29.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:29.112+0000 7f2161377700 1 -- 172.21.15.7:0/2807740113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f215c1a7c90 msgr2=0x7f215c1a8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:29.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:29.112+0000 7f2161377700 1 --2- 172.21.15.7:0/2807740113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f215c1a7c90 0x7f215c1a8140 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f215000b410 tx=0x7f215000b720 comp rx=0 tx=0).stop 2024-06-10T01:29:29.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:29.114+0000 7f2161377700 1 -- 172.21.15.7:0/2807740113 shutdown_connections 2024-06-10T01:29:29.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:29.114+0000 7f2161377700 1 --2- 172.21.15.7:0/2807740113 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f21440a2e30 0x7f21440a52f0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:29.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:29.114+0000 7f2161377700 1 --2- 172.21.15.7:0/2807740113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f215c1a7c90 0x7f215c1a8140 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:29.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:29.114+0000 7f2161377700 1 --2- 172.21.15.7:0/2807740113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f215c102050 0x7f215c1a33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:29.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:29.114+0000 7f2161377700 1 --2- 172.21.15.7:0/2807740113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f215c103350 0x7f215c1a3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:29.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:29.114+0000 7f2161377700 1 -- 172.21.15.7:0/2807740113 >> 172.21.15.7:0/2807740113 conn(0x7f215c078190 msgr2=0x7f215c106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:29.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:29.115+0000 7f2161377700 1 -- 172.21.15.7:0/2807740113 shutdown_connections 2024-06-10T01:29:29.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:29.115+0000 7f2161377700 1 -- 172.21.15.7:0/2807740113 wait complete. 2024-06-10T01:29:29.114 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 55 2024-06-10T01:29:29.563 INFO:tasks.workunit.client.0.smithi007.stdout: 10 247332 22.92 MB/sec execute 486 sec latency 1159.984 ms 2024-06-10T01:29:29.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:29 smithi007 ceph-mon[32398]: pgmap v1138: 129 pgs: 129 active+clean; 692 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 652 op/s 2024-06-10T01:29:29.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:29 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2807740113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:29:29.915 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db / --timeout 90 --expiration 142.45 --await-for 45.0' 2024-06-10T01:29:29.915 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db / --timeout 90 --expiration 142.45 --await-for 45.0 2024-06-10T01:29:29.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:29 smithi175 ceph-mon[36636]: pgmap v1138: 129 pgs: 129 active+clean; 692 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 652 op/s 2024-06-10T01:29:29.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:29 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2807740113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:29:30.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:29 smithi078 ceph-mon[36683]: pgmap v1138: 129 pgs: 129 active+clean; 692 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 652 op/s 2024-06-10T01:29:30.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:29 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2807740113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:29:30.489 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:29:30.563 INFO:tasks.workunit.client.0.smithi007.stdout: 10 247356 22.88 MB/sec execute 487 sec latency 2140.514 ms 2024-06-10T01:29:30.909 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:30 smithi007 ceph-mon[32398]: pgmap v1139: 129 pgs: 129 active+clean; 716 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.3 MiB/s wr, 610 op/s 2024-06-10T01:29:30.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:30 smithi175 ceph-mon[36636]: pgmap v1139: 129 pgs: 129 active+clean; 716 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.3 MiB/s wr, 610 op/s 2024-06-10T01:29:31.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:30 smithi078 ceph-mon[36683]: pgmap v1139: 129 pgs: 129 active+clean; 716 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.3 MiB/s wr, 610 op/s 2024-06-10T01:29:31.563 INFO:tasks.workunit.client.0.smithi007.stdout: 10 247645 22.85 MB/sec execute 488 sec latency 2408.809 ms 2024-06-10T01:29:32.564 INFO:tasks.workunit.client.0.smithi007.stdout: 10 247870 22.82 MB/sec execute 489 sec latency 187.418 ms 2024-06-10T01:29:32.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.687+0000 7fec1759e700 1 -- 172.21.15.7:0/2666117381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec10099c80 msgr2=0x7fec1009a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:32.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.687+0000 7fec1759e700 1 --2- 172.21.15.7:0/2666117381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec10099c80 0x7fec1009a060 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fec08009990 tx=0x7fec08009ca0 comp rx=0 tx=0).stop 2024-06-10T01:29:32.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.687+0000 7fec1759e700 1 -- 172.21.15.7:0/2666117381 shutdown_connections 2024-06-10T01:29:32.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.687+0000 7fec1759e700 1 --2- 172.21.15.7:0/2666117381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec10095930 0x7fec100a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:32.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.687+0000 7fec1759e700 1 --2- 172.21.15.7:0/2666117381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec10099c80 0x7fec1009a060 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:32.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.687+0000 7fec1759e700 1 --2- 172.21.15.7:0/2666117381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec10094f70 0x7fec100953f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:32.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.687+0000 7fec1759e700 1 -- 172.21.15.7:0/2666117381 >> 172.21.15.7:0/2666117381 conn(0x7fec1000a7a0 msgr2=0x7fec1000abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:32.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.689+0000 7fec1759e700 1 -- 172.21.15.7:0/2666117381 shutdown_connections 2024-06-10T01:29:32.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.693+0000 7fec1759e700 1 -- 172.21.15.7:0/2666117381 wait complete. 2024-06-10T01:29:32.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.694+0000 7fec1759e700 1 Processor -- start 2024-06-10T01:29:32.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.694+0000 7fec1759e700 1 -- start start 2024-06-10T01:29:32.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.695+0000 7fec1759e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec10095930 0x7fec1013b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:32.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.695+0000 7fec1759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec10099c80 0x7fec1013b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:32.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.695+0000 7fec1759e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec1013fc70 0x7fec101400f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:32.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.695+0000 7fec1759e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fec100b4f50 con 0x7fec10099c80 2024-06-10T01:29:32.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.695+0000 7fec1759e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fec100b4dd0 con 0x7fec10095930 2024-06-10T01:29:32.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.695+0000 7fec1759e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fec100b50d0 con 0x7fec1013fc70 2024-06-10T01:29:32.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.697+0000 7fec1659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec10095930 0x7fec1013b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:32.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.697+0000 7fec1659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec10095930 0x7fec1013b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:34678/0 (socket says 172.21.15.7:34678) 2024-06-10T01:29:32.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.697+0000 7fec1659c700 1 -- 172.21.15.7:0/363080466 learned_addr learned my addr 172.21.15.7:0/363080466 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:32.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.698+0000 7fec16d9d700 1 --2- 172.21.15.7:0/363080466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec1013fc70 0x7fec101400f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:32.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.698+0000 7fec15d9b700 1 --2- 172.21.15.7:0/363080466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec10099c80 0x7fec1013b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:32.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.699+0000 7fec16d9d700 1 -- 172.21.15.7:0/363080466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec10095930 msgr2=0x7fec1013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:32.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.699+0000 7fec16d9d700 1 --2- 172.21.15.7:0/363080466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec10095930 0x7fec1013b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:32.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.699+0000 7fec16d9d700 1 -- 172.21.15.7:0/363080466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec10099c80 msgr2=0x7fec1013b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:32.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.699+0000 7fec16d9d700 1 --2- 172.21.15.7:0/363080466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec10099c80 0x7fec1013b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:32.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.699+0000 7fec16d9d700 1 -- 172.21.15.7:0/363080466 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec10140790 con 0x7fec1013fc70 2024-06-10T01:29:32.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.699+0000 7fec16d9d700 1 --2- 172.21.15.7:0/363080466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec1013fc70 0x7fec101400f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fec18068f20 tx=0x7fec18069a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:32.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.700+0000 7fec077fe700 1 -- 172.21.15.7:0/363080466 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec18078710 con 0x7fec1013fc70 2024-06-10T01:29:32.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.700+0000 7fec077fe700 1 -- 172.21.15.7:0/363080466 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec18078d90 con 0x7fec1013fc70 2024-06-10T01:29:32.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.700+0000 7fec1759e700 1 -- 172.21.15.7:0/363080466 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec10140a80 con 0x7fec1013fc70 2024-06-10T01:29:32.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.700+0000 7fec077fe700 1 -- 172.21.15.7:0/363080466 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1807b300 con 0x7fec1013fc70 2024-06-10T01:29:32.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.702+0000 7fec1759e700 1 -- 172.21.15.7:0/363080466 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec10150aa0 con 0x7fec1013fc70 2024-06-10T01:29:32.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.703+0000 7fec077fe700 1 -- 172.21.15.7:0/363080466 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fec18073070 con 0x7fec1013fc70 2024-06-10T01:29:32.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.704+0000 7fec077fe700 1 --2- 172.21.15.7:0/363080466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fec000a2db0 0x7fec000a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:32.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.707+0000 7fec1659c700 1 --2- 172.21.15.7:0/363080466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fec000a2db0 0x7fec000a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:32.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.707+0000 7fec077fe700 1 -- 172.21.15.7:0/363080466 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fec180fc340 con 0x7fec1013fc70 2024-06-10T01:29:32.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.711+0000 7fec27806b80 1 Processor -- start 2024-06-10T01:29:32.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.712+0000 7fec27806b80 1 -- start start 2024-06-10T01:29:32.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.713+0000 7fec1659c700 1 --2- 172.21.15.7:0/363080466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fec000a2db0 0x7fec000a5270 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fec08009750 tx=0x7fec08005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:32.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.713+0000 7fec27806b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f592916880 0x55f592916c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:32.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.713+0000 7fec27806b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5929171a0 0x55f592921b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:32.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.713+0000 7fec27806b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f592922050 0x55f592924440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:32.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.713+0000 7fec27806b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f59280a100 con 0x55f592916880 2024-06-10T01:29:32.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.713+0000 7fec27806b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f5928689f0 con 0x55f592922050 2024-06-10T01:29:32.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.713+0000 7fec27806b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f5927d0600 con 0x55f5929171a0 2024-06-10T01:29:32.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.713+0000 7fec1659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f592916880 0x55f592916c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:32.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.713+0000 7fec1659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f592916880 0x55f592916c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47018/0 (socket says 172.21.15.7:47018) 2024-06-10T01:29:32.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.713+0000 7fec1659c700 1 -- 172.21.15.7:0/2901663759 learned_addr learned my addr 172.21.15.7:0/2901663759 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:32.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.714+0000 7fec16d9d700 1 --2- 172.21.15.7:0/2901663759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f592922050 0x55f592924440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:32.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.714+0000 7fec15d9b700 1 --2- 172.21.15.7:0/2901663759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5929171a0 0x55f592921b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:32.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.718+0000 7fec1659c700 1 -- 172.21.15.7:0/2901663759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5929171a0 msgr2=0x55f592921b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:32.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.718+0000 7fec1659c700 1 --2- 172.21.15.7:0/2901663759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5929171a0 0x55f592921b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:32.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.718+0000 7fec1659c700 1 -- 172.21.15.7:0/2901663759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f592922050 msgr2=0x55f592924440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:32.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.718+0000 7fec1659c700 1 --2- 172.21.15.7:0/2901663759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f592922050 0x55f592924440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:32.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.718+0000 7fec1659c700 1 -- 172.21.15.7:0/2901663759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5927b7a70 con 0x55f592916880 2024-06-10T01:29:32.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.719+0000 7fec1659c700 1 --2- 172.21.15.7:0/2901663759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f592916880 0x55f592916c60 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fec080092e0 tx=0x7fec0801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fada91d05be63c5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:32.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.720+0000 7fec04ff9700 1 -- 172.21.15.7:0/2901663759 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec08003880 con 0x55f592916880 2024-06-10T01:29:32.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.720+0000 7fec04ff9700 1 -- 172.21.15.7:0/2901663759 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec0800f460 con 0x55f592916880 2024-06-10T01:29:32.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.721+0000 7fec27806b80 1 -- 172.21.15.7:0/2901663759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f592916880 msgr2=0x55f592916c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:32.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.721+0000 7fec27806b80 1 --2- 172.21.15.7:0/2901663759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f592916880 0x55f592916c60 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fec080092e0 tx=0x7fec0801f000 comp rx=0 tx=0).stop 2024-06-10T01:29:32.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.723+0000 7fec27806b80 1 -- 172.21.15.7:0/2901663759 shutdown_connections 2024-06-10T01:29:32.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.723+0000 7fec27806b80 1 --2- 172.21.15.7:0/2901663759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f592922050 0x55f592924440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:32.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.723+0000 7fec27806b80 1 --2- 172.21.15.7:0/2901663759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f592916880 0x55f592916c60 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:32.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.723+0000 7fec27806b80 1 --2- 172.21.15.7:0/2901663759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5929171a0 0x55f592921b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:32.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.723+0000 7fec27806b80 1 -- 172.21.15.7:0/2901663759 >> 172.21.15.7:0/2901663759 conn(0x55f592844220 msgr2=0x55f592912d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:32.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.725+0000 7fec27806b80 1 -- 172.21.15.7:0/2901663759 shutdown_connections 2024-06-10T01:29:32.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.726+0000 7fec27806b80 1 -- 172.21.15.7:0/2901663759 wait complete. 2024-06-10T01:29:32.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.726+0000 7fec27806b80 1 -- start start 2024-06-10T01:29:32.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.727+0000 7fec27806b80 1 Processor -- start 2024-06-10T01:29:32.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.727+0000 7fec27806b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f592916880 0x55f5929217f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:32.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.727+0000 7fec27806b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5929171a0 0x55f59291eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:32.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.727+0000 7fec27806b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f592922050 0x55f59291f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:32.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.727+0000 7fec27806b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f592809160 con 0x55f5929171a0 2024-06-10T01:29:32.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.727+0000 7fec27806b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f5928649b0 con 0x55f592916880 2024-06-10T01:29:32.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.727+0000 7fec27806b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f592935b80 con 0x55f592922050 2024-06-10T01:29:32.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.727+0000 7fec27806b80 10 client.0 _ll_register_callbacks cb 0x55f5927fe750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:29:32.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.727+0000 7fec16d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f592922050 0x55f59291f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:32.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.728+0000 7fec16d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f592922050 0x55f59291f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:41624/0 (socket says 172.21.15.7:41624) 2024-06-10T01:29:32.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.728+0000 7fec16d9d700 1 -- 172.21.15.7:0/2120026601 learned_addr learned my addr 172.21.15.7:0/2120026601 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:32.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.728+0000 7fec16d9d700 1 -- 172.21.15.7:0/2120026601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f592916880 msgr2=0x55f5929217f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:32.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.728+0000 7fec16d9d700 1 --2- 172.21.15.7:0/2120026601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f592916880 0x55f5929217f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:32.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.728+0000 7fec16d9d700 1 -- 172.21.15.7:0/2120026601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5929171a0 msgr2=0x55f59291eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:32.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.728+0000 7fec16d9d700 1 --2- 172.21.15.7:0/2120026601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5929171a0 0x55f59291eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:32.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.728+0000 7fec16d9d700 1 -- 172.21.15.7:0/2120026601 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f59282e8f0 con 0x55f592922050 2024-06-10T01:29:32.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.729+0000 7fec16d9d700 1 --2- 172.21.15.7:0/2120026601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f592922050 0x55f59291f030 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fec180fac70 tx=0x7fec18068610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=223026f985dd96fd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:32.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.729+0000 7febfdffb700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:29:32.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.729+0000 7febfdffb700 1 -- 172.21.15.7:0/2120026601 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec18136060 con 0x55f592922050 2024-06-10T01:29:32.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.729+0000 7fec27806b80 1 -- 172.21.15.7:0/2120026601 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f592875ef0 con 0x55f592922050 2024-06-10T01:29:32.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.729+0000 7febfdffb700 1 -- 172.21.15.7:0/2120026601 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fec18102180 con 0x55f592922050 2024-06-10T01:29:32.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.729+0000 7febfdffb700 1 -- 172.21.15.7:0/2120026601 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fec1807c3a0 con 0x55f592922050 2024-06-10T01:29:32.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.734+0000 7fec27806b80 10 client.30088 fetch_fsmap learned FSMap version 55 2024-06-10T01:29:32.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.734+0000 7fec27806b80 1 -- 172.21.15.7:0/2120026601 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55f592940400 con 0x55f592922050 2024-06-10T01:29:32.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.735+0000 7febfdffb700 1 -- 172.21.15.7:0/2120026601 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f592875ef0 con 0x55f592922050 2024-06-10T01:29:32.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.735+0000 7febfdffb700 1 -- 172.21.15.7:0/2120026601 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fec1811e0c0 con 0x55f592922050 2024-06-10T01:29:32.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.735+0000 7fec27806b80 10 client.30088 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:29:32.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.735+0000 7fec27806b80 10 client.30088 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:29:32.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.735+0000 7fec27806b80 20 client.30088 populate_metadata read hostname 'smithi007' 2024-06-10T01:29:32.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.735+0000 7fec27806b80 1 --2- 172.21.15.7:0/2120026601 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f592942d60 0x55f592945220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:32.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.735+0000 7fec27806b80 4 client.30088 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:29:32.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.735+0000 7fec27806b80 1 -- 172.21.15.7:0/2120026601 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f592870470 con 0x55f592942d60 2024-06-10T01:29:32.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.737+0000 7fec1659c700 1 --2- 172.21.15.7:0/2120026601 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f592942d60 0x55f592945220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:32.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.748+0000 7fec1659c700 1 --2- 172.21.15.7:0/2120026601 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f592942d60 0x55f592945220 crc :-1 s=READY pgs=525 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45a59004da8f3cd6 server_cookie=b70cf2f10a12559e in_seq=0 out_seq=0 2024-06-10T01:29:32.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.748+0000 7febfdffb700 10 client.30088 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:32.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.753+0000 7febfdffb700 1 -- 172.21.15.7:0/2120026601 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f592870470 con 0x55f592942d60 2024-06-10T01:29:32.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.753+0000 7febfdffb700 10 client.30088 handle_command_reply: tid=0 2024-06-10T01:29:32.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.753+0000 7fec27806b80 1 client.30088 shutdown 2024-06-10T01:29:32.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.761+0000 7fec27806b80 1 -- 172.21.15.7:0/2120026601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f592922050 msgr2=0x55f59291f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:32.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.761+0000 7fec27806b80 1 --2- 172.21.15.7:0/2120026601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f592922050 0x55f59291f030 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fec180fac70 tx=0x7fec18068610 comp rx=0 tx=0).stop 2024-06-10T01:29:32.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.762+0000 7fec27806b80 1 -- 172.21.15.7:0/2120026601 shutdown_connections 2024-06-10T01:29:32.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.762+0000 7fec27806b80 1 --2- 172.21.15.7:0/2120026601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f592922050 0x55f59291f030 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:32.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.762+0000 7fec27806b80 1 --2- 172.21.15.7:0/2120026601 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f592942d60 0x55f592945220 crc :-1 s=READY pgs=525 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:32.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.762+0000 7fec27806b80 1 --2- 172.21.15.7:0/2120026601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f592916880 0x55f5929217f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:32.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.762+0000 7fec27806b80 1 --2- 172.21.15.7:0/2120026601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5929171a0 0x55f59291eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:32.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.762+0000 7fec27806b80 1 -- 172.21.15.7:0/2120026601 >> 172.21.15.7:0/2120026601 conn(0x55f592844220 msgr2=0x55f592912920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:32.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.762+0000 7febfdffb700 0 client.30088 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:32.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.766+0000 7fec27806b80 1 -- 172.21.15.7:0/2120026601 shutdown_connections 2024-06-10T01:29:32.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.766+0000 7fec27806b80 1 -- 172.21.15.7:0/2120026601 wait complete. 2024-06-10T01:29:32.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.767+0000 7fec27806b80 20 client.30088 trim_cache size 0 max 16384 2024-06-10T01:29:32.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.810+0000 7fec27806b80 1 Processor -- start 2024-06-10T01:29:32.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.810+0000 7fec27806b80 1 -- start start 2024-06-10T01:29:32.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.810+0000 7fec27806b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f592916880 0x55f59294ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:32.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.810+0000 7fec27806b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5929171a0 0x55f59294b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:32.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.811+0000 7fec27806b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f592922050 0x55f59291bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:32.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.811+0000 7fec27806b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fec101537d0 con 0x55f592916880 2024-06-10T01:29:32.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.811+0000 7fec27806b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fec10151060 con 0x55f592922050 2024-06-10T01:29:32.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.811+0000 7fec27806b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fec1014e8b0 con 0x55f5929171a0 2024-06-10T01:29:32.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.811+0000 7fec16d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f592922050 0x55f59291bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:32.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.811+0000 7fec16d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f592922050 0x55f59291bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:34708/0 (socket says 172.21.15.7:34708) 2024-06-10T01:29:32.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.811+0000 7fec16d9d700 1 -- 172.21.15.7:0/2908966171 learned_addr learned my addr 172.21.15.7:0/2908966171 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:32.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.811+0000 7fec1659c700 1 --2- 172.21.15.7:0/2908966171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f592916880 0x55f59294ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:32.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.811+0000 7fec16d9d700 1 -- 172.21.15.7:0/2908966171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5929171a0 msgr2=0x55f59294b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:32.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.811+0000 7fec16d9d700 1 --2- 172.21.15.7:0/2908966171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5929171a0 0x55f59294b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:32.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.811+0000 7fec16d9d700 1 -- 172.21.15.7:0/2908966171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f592916880 msgr2=0x55f59294ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:32.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.811+0000 7fec16d9d700 1 --2- 172.21.15.7:0/2908966171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f592916880 0x55f59294ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:32.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.812+0000 7fec16d9d700 1 -- 172.21.15.7:0/2908966171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f59291d060 con 0x55f592922050 2024-06-10T01:29:32.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.814+0000 7fec16d9d700 1 --2- 172.21.15.7:0/2908966171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f592922050 0x55f59291bd70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fec18000eb0 tx=0x7fec18068eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e2b21e7aad3a0ca server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:32.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.816+0000 7febfffff700 1 -- 172.21.15.7:0/2908966171 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec180fe2a0 con 0x55f592922050 2024-06-10T01:29:32.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.816+0000 7febfffff700 1 -- 172.21.15.7:0/2908966171 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec18102650 con 0x55f592922050 2024-06-10T01:29:32.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.816+0000 7febfffff700 1 -- 172.21.15.7:0/2908966171 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec180fd6f0 con 0x55f592922050 2024-06-10T01:29:32.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.817+0000 7fec27806b80 1 -- 172.21.15.7:0/2908966171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f592922050 msgr2=0x55f59291bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:32.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.817+0000 7fec27806b80 1 --2- 172.21.15.7:0/2908966171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f592922050 0x55f59291bd70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fec18000eb0 tx=0x7fec18068eb0 comp rx=0 tx=0).stop 2024-06-10T01:29:32.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.817+0000 7fec27806b80 1 -- 172.21.15.7:0/2908966171 shutdown_connections 2024-06-10T01:29:32.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.817+0000 7fec27806b80 1 --2- 172.21.15.7:0/2908966171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f592922050 0x55f59291bd70 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:32.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.817+0000 7fec27806b80 1 --2- 172.21.15.7:0/2908966171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f592916880 0x55f59294ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:32.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.817+0000 7fec27806b80 1 --2- 172.21.15.7:0/2908966171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5929171a0 0x55f59294b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:32.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.817+0000 7fec27806b80 1 -- 172.21.15.7:0/2908966171 >> 172.21.15.7:0/2908966171 conn(0x55f592844220 msgr2=0x55f592920bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:32.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.821+0000 7fec27806b80 1 -- 172.21.15.7:0/2908966171 shutdown_connections 2024-06-10T01:29:32.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.823+0000 7fec27806b80 1 -- 172.21.15.7:0/2908966171 wait complete. 2024-06-10T01:29:32.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.824+0000 7fec27806b80 1 -- start start 2024-06-10T01:29:32.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.824+0000 7fec27806b80 1 Processor -- start 2024-06-10T01:29:32.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.825+0000 7fec27806b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f592916880 0x55f59293b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:32.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.825+0000 7fec27806b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5929171a0 0x55f592938af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:32.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.826+0000 7fec27806b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f592922050 0x55f5929391e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:32.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.826+0000 7fec15d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5929171a0 0x55f592938af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:32.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.826+0000 7fec1659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f592916880 0x55f59293b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:32.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.826+0000 7fec27806b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fec1014ef20 con 0x55f5929171a0 2024-06-10T01:29:32.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.826+0000 7fec27806b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fec10150390 con 0x55f592922050 2024-06-10T01:29:32.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.826+0000 7fec27806b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fec1014f5f0 con 0x55f592916880 2024-06-10T01:29:32.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.826+0000 7fec27806b80 10 client.0 _ll_register_callbacks cb 0x55f5927fe750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:29:32.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.827+0000 7fec1659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f592916880 0x55f59293b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:41642/0 (socket says 172.21.15.7:41642) 2024-06-10T01:29:32.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.827+0000 7fec1659c700 1 -- 172.21.15.7:0/2919021255 learned_addr learned my addr 172.21.15.7:0/2919021255 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:32.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.827+0000 7fec16d9d700 1 --2- 172.21.15.7:0/2919021255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f592922050 0x55f5929391e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:32.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.830+0000 7fec15d9b700 1 -- 172.21.15.7:0/2919021255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f592916880 msgr2=0x55f59293b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:32.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.830+0000 7fec15d9b700 1 --2- 172.21.15.7:0/2919021255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f592916880 0x55f59293b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:32.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.830+0000 7fec15d9b700 1 -- 172.21.15.7:0/2919021255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f592922050 msgr2=0x55f5929391e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:32.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.830+0000 7fec15d9b700 1 --2- 172.21.15.7:0/2919021255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f592922050 0x55f5929391e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:32.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.830+0000 7fec15d9b700 1 -- 172.21.15.7:0/2919021255 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5929acb10 con 0x55f5929171a0 2024-06-10T01:29:32.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.835+0000 7fec15d9b700 1 --2- 172.21.15.7:0/2919021255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5929171a0 0x55f592938af0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x55f59290f4b0 tx=0x7fec0c004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5803e5476ffded server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:32.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.835+0000 7febfdffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:29:32.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.837+0000 7febfdffb700 1 -- 172.21.15.7:0/2919021255 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c01f060 con 0x55f5929171a0 2024-06-10T01:29:32.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.838+0000 7febfdffb700 1 -- 172.21.15.7:0/2919021255 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fec0c01a480 con 0x55f5929171a0 2024-06-10T01:29:32.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.838+0000 7fec27806b80 1 -- 172.21.15.7:0/2919021255 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec18102650 con 0x55f5929171a0 2024-06-10T01:29:32.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.846+0000 7febfdffb700 1 -- 172.21.15.7:0/2919021255 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fec0c00be30 con 0x55f5929171a0 2024-06-10T01:29:32.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.846+0000 7fec27806b80 10 client.21003 fetch_fsmap learned FSMap version 55 2024-06-10T01:29:32.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.846+0000 7fec27806b80 1 -- 172.21.15.7:0/2919021255 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55f5929add40 con 0x55f5929171a0 2024-06-10T01:29:32.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.856+0000 7febfdffb700 1 -- 172.21.15.7:0/2919021255 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec18102650 con 0x55f5929171a0 2024-06-10T01:29:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.856+0000 7febfdffb700 1 -- 172.21.15.7:0/2919021255 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fec0c023750 con 0x55f5929171a0 2024-06-10T01:29:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.856+0000 7fec27806b80 10 client.21003 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:29:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.856+0000 7fec27806b80 10 client.21003 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:29:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.856+0000 7fec27806b80 20 client.21003 populate_metadata read hostname 'smithi007' 2024-06-10T01:29:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.856+0000 7fec27806b80 1 --2- 172.21.15.7:0/2919021255 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f5929ae470 0x55f5929b0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.856+0000 7fec27806b80 4 client.21003 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.45, "await_for": 45.0}] 2024-06-10T01:29:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.856+0000 7fec27806b80 1 -- 172.21.15.7:0/2919021255 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.45, "await_for": 45.0}) -- 0x55f5928240f0 con 0x55f5929ae470 2024-06-10T01:29:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.857+0000 7fec1659c700 1 --2- 172.21.15.7:0/2919021255 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f5929ae470 0x55f5929b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:32.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.865+0000 7fec1659c700 1 --2- 172.21.15.7:0/2919021255 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f5929ae470 0x55f5929b0890 crc :-1 s=READY pgs=526 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df6110413d63ad0e server_cookie=34a9ef972eba819d in_seq=0 out_seq=0 2024-06-10T01:29:32.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:32.865+0000 7febfdffb700 10 client.21003 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:33.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:33 smithi175 ceph-mon[36636]: pgmap v1140: 129 pgs: 129 active+clean; 751 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.4 MiB/s wr, 666 op/s 2024-06-10T01:29:33.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:33 smithi007 ceph-mon[32398]: pgmap v1140: 129 pgs: 129 active+clean; 751 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.4 MiB/s wr, 666 op/s 2024-06-10T01:29:33.564 INFO:tasks.workunit.client.0.smithi007.stdout: 10 247947 22.79 MB/sec execute 490 sec latency 689.246 ms 2024-06-10T01:29:33.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:33 smithi078 ceph-mon[36683]: pgmap v1140: 129 pgs: 129 active+clean; 751 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.4 MiB/s wr, 666 op/s 2024-06-10T01:29:33.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:33.977+0000 7febfdffb700 1 -- 172.21.15.7:0/2919021255 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f5928240f0 con 0x55f5929ae470 2024-06-10T01:29:33.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:33.977+0000 7febfdffb700 10 client.21003 handle_command_reply: tid=0 2024-06-10T01:29:33.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:33.977+0000 7fec27806b80 1 client.21003 shutdown 2024-06-10T01:29:33.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:33.977+0000 7fec27806b80 1 -- 172.21.15.7:0/2919021255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5929171a0 msgr2=0x55f592938af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:33.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:33.977+0000 7fec27806b80 1 --2- 172.21.15.7:0/2919021255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5929171a0 0x55f592938af0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x55f59290f4b0 tx=0x7fec0c004dc0 comp rx=0 tx=0).stop 2024-06-10T01:29:33.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:33.977+0000 7fec27806b80 1 -- 172.21.15.7:0/2919021255 shutdown_connections 2024-06-10T01:29:33.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:33.977+0000 7fec27806b80 1 --2- 172.21.15.7:0/2919021255 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f5929ae470 0x55f5929b0890 crc :-1 s=READY pgs=526 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:33.972 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:29:33.972 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 55, 2024-06-10T01:29:33.972 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:29:33.972 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 74, 2024-06-10T01:29:33.972 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:29:33.972 INFO:teuthology.orchestra.run.smithi007.stdout: "6e35bda3": { 2024-06-10T01:29:33.973 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 74, 2024-06-10T01:29:33.973 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:29:33.973 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:29:33.973 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:29:33.973 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:29:33.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:29:33.973 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:29:33.973 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 142.4, 2024-06-10T01:29:33.973 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:29:33.973 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:29:33.973 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:29:33.973 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:29:33.973 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:29:33.973 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:29:33.973 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:29:33.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:29:33.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:29:33.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:29:33.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:29:33.974 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:29:33.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:33.977+0000 7fec27806b80 1 --2- 172.21.15.7:0/2919021255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f592922050 0x55f5929391e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:33.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:33.977+0000 7fec27806b80 1 --2- 172.21.15.7:0/2919021255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f592916880 0x55f59293b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:33.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:33.977+0000 7fec27806b80 1 --2- 172.21.15.7:0/2919021255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5929171a0 0x55f592938af0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:33.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:33.977+0000 7fec27806b80 1 -- 172.21.15.7:0/2919021255 >> 172.21.15.7:0/2919021255 conn(0x55f592844220 msgr2=0x55f592911e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:33.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:33.977+0000 7febfdffb700 0 client.21003 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:33.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:33.978+0000 7fec27806b80 1 -- 172.21.15.7:0/2919021255 shutdown_connections 2024-06-10T01:29:33.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:33.978+0000 7fec27806b80 1 -- 172.21.15.7:0/2919021255 wait complete. 2024-06-10T01:29:33.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:33.978+0000 7fec27806b80 20 client.21003 trim_cache size 0 max 16384 2024-06-10T01:29:33.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:33.980+0000 7fec1759e700 1 -- 172.21.15.7:0/363080466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fec000a2db0 msgr2=0x7fec000a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:33.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:33.980+0000 7fec1759e700 1 --2- 172.21.15.7:0/363080466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fec000a2db0 0x7fec000a5270 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fec08009750 tx=0x7fec08005a90 comp rx=0 tx=0).stop 2024-06-10T01:29:33.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:33.980+0000 7fec1759e700 1 -- 172.21.15.7:0/363080466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec1013fc70 msgr2=0x7fec101400f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:33.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:33.980+0000 7fec1759e700 1 --2- 172.21.15.7:0/363080466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec1013fc70 0x7fec101400f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fec18068f20 tx=0x7fec18069a20 comp rx=0 tx=0).stop 2024-06-10T01:29:33.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:33.980+0000 7fec1759e700 1 -- 172.21.15.7:0/363080466 shutdown_connections 2024-06-10T01:29:33.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:33.980+0000 7fec1759e700 1 --2- 172.21.15.7:0/363080466 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fec000a2db0 0x7fec000a5270 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:33.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:33.980+0000 7fec1759e700 1 --2- 172.21.15.7:0/363080466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec1013fc70 0x7fec101400f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:33.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:33.980+0000 7fec1759e700 1 --2- 172.21.15.7:0/363080466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec10095930 0x7fec1013b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:33.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:33.980+0000 7fec1759e700 1 --2- 172.21.15.7:0/363080466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec10099c80 0x7fec1013b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:33.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:33.980+0000 7fec1759e700 1 -- 172.21.15.7:0/363080466 >> 172.21.15.7:0/363080466 conn(0x7fec1000a7a0 msgr2=0x7fec10092980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:33.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:33.980+0000 7fec1759e700 1 -- 172.21.15.7:0/363080466 shutdown_connections 2024-06-10T01:29:33.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:33.980+0000 7fec1759e700 1 -- 172.21.15.7:0/363080466 wait complete. 2024-06-10T01:29:34.564 INFO:tasks.workunit.client.0.smithi007.stdout: 10 247957 22.75 MB/sec execute 491 sec latency 1433.613 ms 2024-06-10T01:29:34.696 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6e35bda3', quiesce took 10.7 seconds. Will release after: 11.75 2024-06-10T01:29:35.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:35 smithi175 ceph-mon[36636]: pgmap v1141: 129 pgs: 129 active+clean; 758 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 10 MiB/s wr, 590 op/s 2024-06-10T01:29:35.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:35 smithi007 ceph-mon[32398]: pgmap v1141: 129 pgs: 129 active+clean; 758 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 10 MiB/s wr, 590 op/s 2024-06-10T01:29:35.564 INFO:tasks.workunit.client.0.smithi007.stdout: 10 247957 22.70 MB/sec execute 492 sec latency 2433.739 ms 2024-06-10T01:29:35.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:35 smithi078 ceph-mon[36683]: pgmap v1141: 129 pgs: 129 active+clean; 758 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 10 MiB/s wr, 590 op/s 2024-06-10T01:29:36.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.089+0000 7f18d77ee640 1 -- 172.21.15.7:0/1495704968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18d01076e0 msgr2=0x7f18d0109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.089+0000 7f18d77ee640 1 --2- 172.21.15.7:0/1495704968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18d01076e0 0x7f18d0109b30 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f18c8009ca0 tx=0x7f18c802a8c0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.089+0000 7f18d77ee640 1 -- 172.21.15.7:0/1495704968 shutdown_connections 2024-06-10T01:29:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.089+0000 7f18d77ee640 1 --2- 172.21.15.7:0/1495704968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18d01076e0 0x7f18d0109b30 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.089+0000 7f18d77ee640 1 --2- 172.21.15.7:0/1495704968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18d0104db0 0x7f18d01071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.089+0000 7f18d77ee640 1 --2- 172.21.15.7:0/1495704968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18d0102480 0x7f18d0104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.089+0000 7f18d77ee640 1 -- 172.21.15.7:0/1495704968 >> 172.21.15.7:0/1495704968 conn(0x7f18d00fbe70 msgr2=0x7f18d00fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.090+0000 7f18d77ee640 1 -- 172.21.15.7:0/1495704968 shutdown_connections 2024-06-10T01:29:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.090+0000 7f18d77ee640 1 -- 172.21.15.7:0/1495704968 wait complete. 2024-06-10T01:29:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.090+0000 7f18d77ee640 1 Processor -- start 2024-06-10T01:29:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.090+0000 7f18d77ee640 1 -- start start 2024-06-10T01:29:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.090+0000 7f18d77ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18d0102480 0x7f18d0101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.090+0000 7f18d77ee640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18d0104db0 0x7f18d0100430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.091+0000 7f18d77ee640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18d01076e0 0x7f18d0100970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.091+0000 7f18d77ee640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f18d010d6a0 con 0x7f18d01076e0 2024-06-10T01:29:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.091+0000 7f18d77ee640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f18d010d520 con 0x7f18d0102480 2024-06-10T01:29:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.091+0000 7f18d77ee640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f18d010d820 con 0x7f18d0104db0 2024-06-10T01:29:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.091+0000 7f18d558a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18d0102480 0x7f18d0101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.091+0000 7f18d558a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18d0102480 0x7f18d0101df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:34740/0 (socket says 172.21.15.7:34740) 2024-06-10T01:29:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.091+0000 7f18d558a640 1 -- 172.21.15.7:0/4171902140 learned_addr learned my addr 172.21.15.7:0/4171902140 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.091+0000 7f18d4d89640 1 --2- 172.21.15.7:0/4171902140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18d0104db0 0x7f18d0100430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.091+0000 7f18d5d8b640 1 --2- 172.21.15.7:0/4171902140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18d01076e0 0x7f18d0100970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.091+0000 7f18d558a640 1 -- 172.21.15.7:0/4171902140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18d0104db0 msgr2=0x7f18d0100430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.091+0000 7f18d558a640 1 --2- 172.21.15.7:0/4171902140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18d0104db0 0x7f18d0100430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.091+0000 7f18d558a640 1 -- 172.21.15.7:0/4171902140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18d01076e0 msgr2=0x7f18d0100970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.091+0000 7f18d558a640 1 --2- 172.21.15.7:0/4171902140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18d01076e0 0x7f18d0100970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.091+0000 7f18d558a640 1 -- 172.21.15.7:0/4171902140 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d0101200 con 0x7f18d0102480 2024-06-10T01:29:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.091+0000 7f18d558a640 1 --2- 172.21.15.7:0/4171902140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18d0102480 0x7f18d0101df0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f18cc00d950 tx=0x7f18cc00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:36.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.094+0000 7f18c67fc640 1 -- 172.21.15.7:0/4171902140 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc017030 con 0x7f18d0102480 2024-06-10T01:29:36.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.094+0000 7f18c67fc640 1 -- 172.21.15.7:0/4171902140 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18cc004020 con 0x7f18d0102480 2024-06-10T01:29:36.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.094+0000 7f18d77ee640 1 -- 172.21.15.7:0/4171902140 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18d01a1e00 con 0x7f18d0102480 2024-06-10T01:29:36.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.094+0000 7f18c67fc640 1 -- 172.21.15.7:0/4171902140 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc0047a0 con 0x7f18d0102480 2024-06-10T01:29:36.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.094+0000 7f18d77ee640 1 -- 172.21.15.7:0/4171902140 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18d01a2260 con 0x7f18d0102480 2024-06-10T01:29:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.096+0000 7f18c67fc640 1 -- 172.21.15.7:0/4171902140 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f18cc0041d0 con 0x7f18d0102480 2024-06-10T01:29:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.097+0000 7f18da5da740 1 Processor -- start 2024-06-10T01:29:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.097+0000 7f18da5da740 1 -- start start 2024-06-10T01:29:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.097+0000 7f18da5da740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ce559010 0x5637ce559430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.097+0000 7f18da5da740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ce559a00 0x5637ce55e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.097+0000 7f18da5da740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ce55eb70 0x5637ce561000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.097+0000 7f18da5da740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5637ce4bc4f0 con 0x5637ce559010 2024-06-10T01:29:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.097+0000 7f18da5da740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5637ce45a870 con 0x5637ce559a00 2024-06-10T01:29:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.097+0000 7f18da5da740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5637ce4b84b0 con 0x5637ce55eb70 2024-06-10T01:29:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.097+0000 7f18d5d8b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ce559010 0x5637ce559430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.097+0000 7f18c67fc640 1 --2- 172.21.15.7:0/4171902140 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f18bc0a2c00 0x7f18bc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.097+0000 7f18d5d8b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ce559010 0x5637ce559430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47070/0 (socket says 172.21.15.7:47070) 2024-06-10T01:29:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.097+0000 7f18d5d8b640 1 -- 172.21.15.7:0/290999413 learned_addr learned my addr 172.21.15.7:0/290999413 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.097+0000 7f18d558a640 1 --2- 172.21.15.7:0/290999413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ce559a00 0x5637ce55e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.097+0000 7f18d5d8b640 1 --2- 172.21.15.7:0/4171902140 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f18bc0a2c00 0x7f18bc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.097+0000 7f18d4d89640 1 --2- 172.21.15.7:0/290999413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ce55eb70 0x5637ce561000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.098+0000 7f18d5d8b640 1 -- 172.21.15.7:0/290999413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ce55eb70 msgr2=0x5637ce561000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.098+0000 7f18d5d8b640 1 --2- 172.21.15.7:0/290999413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ce55eb70 0x5637ce561000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.098+0000 7f18d5d8b640 1 -- 172.21.15.7:0/290999413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ce559a00 msgr2=0x5637ce55e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.098+0000 7f18d5d8b640 1 --2- 172.21.15.7:0/290999413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ce559a00 0x5637ce55e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.098+0000 7f18d5d8b640 1 -- 172.21.15.7:0/290999413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637ce4769b0 con 0x5637ce559010 2024-06-10T01:29:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.098+0000 7f18c67fc640 1 -- 172.21.15.7:0/4171902140 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f18cc0ced30 con 0x7f18d0102480 2024-06-10T01:29:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.098+0000 7f18d558a640 1 --2- 172.21.15.7:0/290999413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ce559a00 0x5637ce55e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:29:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.098+0000 7f18d5d8b640 1 --2- 172.21.15.7:0/290999413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ce559010 0x5637ce559430 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f18c8004af0 tx=0x7f18c8031eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d3420215dae039 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.098+0000 7f18bb7fe640 1 -- 172.21.15.7:0/290999413 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c8031870 con 0x5637ce559010 2024-06-10T01:29:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.098+0000 7f18bb7fe640 1 -- 172.21.15.7:0/290999413 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18c804f070 con 0x5637ce559010 2024-06-10T01:29:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.098+0000 7f18bb7fe640 1 -- 172.21.15.7:0/290999413 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c802f990 con 0x5637ce559010 2024-06-10T01:29:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.099+0000 7f18da5da740 1 -- 172.21.15.7:0/290999413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ce559010 msgr2=0x5637ce559430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.099+0000 7f18d5d8b640 1 --2- 172.21.15.7:0/4171902140 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f18bc0a2c00 0x7f18bc0a50c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f18c804e000 tx=0x7f18c804c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.099+0000 7f18da5da740 1 --2- 172.21.15.7:0/290999413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ce559010 0x5637ce559430 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f18c8004af0 tx=0x7f18c8031eb0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.099+0000 7f18da5da740 1 -- 172.21.15.7:0/290999413 shutdown_connections 2024-06-10T01:29:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.099+0000 7f18da5da740 1 --2- 172.21.15.7:0/290999413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ce55eb70 0x5637ce561000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.099+0000 7f18da5da740 1 --2- 172.21.15.7:0/290999413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ce559a00 0x5637ce55e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.099+0000 7f18da5da740 1 --2- 172.21.15.7:0/290999413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ce559010 0x5637ce559430 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.099+0000 7f18da5da740 1 -- 172.21.15.7:0/290999413 >> 172.21.15.7:0/290999413 conn(0x5637ce554680 msgr2=0x5637ce554a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.099+0000 7f18da5da740 1 -- 172.21.15.7:0/290999413 shutdown_connections 2024-06-10T01:29:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.099+0000 7f18da5da740 1 -- 172.21.15.7:0/290999413 wait complete. 2024-06-10T01:29:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.099+0000 7f18da5da740 1 -- start start 2024-06-10T01:29:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.099+0000 7f18da5da740 1 Processor -- start 2024-06-10T01:29:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.100+0000 7f18da5da740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ce559010 0x5637ce5630a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.100+0000 7f18da5da740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ce559a00 0x5637ce5635e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.100+0000 7f18da5da740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ce55eb70 0x5637ce56a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.100+0000 7f18da5da740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5637ce5619a0 con 0x5637ce559a00 2024-06-10T01:29:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.100+0000 7f18da5da740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5637ce445ae0 con 0x5637ce559010 2024-06-10T01:29:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.100+0000 7f18da5da740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5637ce561b20 con 0x5637ce55eb70 2024-06-10T01:29:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.100+0000 7f18da5da740 10 client.0 _ll_register_callbacks cb 0x5637ce47e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:29:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.100+0000 7f18d4d89640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ce559a00 0x5637ce5635e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.100+0000 7f18d4d89640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ce559a00 0x5637ce5635e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47084/0 (socket says 172.21.15.7:47084) 2024-06-10T01:29:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.100+0000 7f18d4d89640 1 -- 172.21.15.7:0/1422444474 learned_addr learned my addr 172.21.15.7:0/1422444474 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.100+0000 7f18d558a640 1 --2- 172.21.15.7:0/1422444474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ce559010 0x5637ce5630a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.100+0000 7f18d5d8b640 1 --2- 172.21.15.7:0/1422444474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ce55eb70 0x5637ce56a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.100+0000 7f18d4d89640 1 -- 172.21.15.7:0/1422444474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ce55eb70 msgr2=0x5637ce56a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.100+0000 7f18d4d89640 1 --2- 172.21.15.7:0/1422444474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ce55eb70 0x5637ce56a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.100+0000 7f18d4d89640 1 -- 172.21.15.7:0/1422444474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ce559010 msgr2=0x5637ce5630a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.100+0000 7f18d4d89640 1 --2- 172.21.15.7:0/1422444474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ce559010 0x5637ce5630a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.100+0000 7f18d4d89640 1 -- 172.21.15.7:0/1422444474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637ce1d0210 con 0x5637ce559a00 2024-06-10T01:29:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.100+0000 7f18d5d8b640 1 --2- 172.21.15.7:0/1422444474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ce55eb70 0x5637ce56a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:29:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.101+0000 7f18d4d89640 1 --2- 172.21.15.7:0/1422444474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ce559a00 0x5637ce5635e0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f18c000b920 tx=0x7f18c000bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93d177a2eaff8a5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.101+0000 7f18b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:29:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.101+0000 7f18b8ff9640 1 -- 172.21.15.7:0/1422444474 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c0005020 con 0x5637ce559a00 2024-06-10T01:29:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.101+0000 7f18b8ff9640 1 -- 172.21.15.7:0/1422444474 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f18c00051c0 con 0x5637ce559a00 2024-06-10T01:29:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.101+0000 7f18da5da740 1 -- 172.21.15.7:0/1422444474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18c804f070 con 0x5637ce559a00 2024-06-10T01:29:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.101+0000 7f18b8ff9640 1 -- 172.21.15.7:0/1422444474 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f18c00053c0 con 0x5637ce559a00 2024-06-10T01:29:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.101+0000 7f18da5da740 10 client.21015 fetch_fsmap learned FSMap version 55 2024-06-10T01:29:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.101+0000 7f18da5da740 1 -- 172.21.15.7:0/1422444474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5637ce29c620 con 0x5637ce559a00 2024-06-10T01:29:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.101+0000 7f18b8ff9640 1 -- 172.21.15.7:0/1422444474 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18c804f070 con 0x5637ce559a00 2024-06-10T01:29:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.101+0000 7f18b8ff9640 1 -- 172.21.15.7:0/1422444474 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f18c0019030 con 0x5637ce559a00 2024-06-10T01:29:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.101+0000 7f18da5da740 10 client.21015 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:29:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.101+0000 7f18da5da740 10 client.21015 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:29:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.101+0000 7f18da5da740 20 client.21015 populate_metadata read hostname 'smithi007' 2024-06-10T01:29:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.101+0000 7f18da5da740 1 --2- 172.21.15.7:0/1422444474 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5637ce56efd0 0x5637ce571430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.101+0000 7f18da5da740 4 client.21015 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:29:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.101+0000 7f18da5da740 1 -- 172.21.15.7:0/1422444474 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637ce4cff10 con 0x5637ce56efd0 2024-06-10T01:29:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.102+0000 7f18d558a640 1 --2- 172.21.15.7:0/1422444474 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5637ce56efd0 0x5637ce571430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.102+0000 7f18d558a640 1 --2- 172.21.15.7:0/1422444474 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5637ce56efd0 0x5637ce571430 crc :-1 s=READY pgs=527 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2580913ba17de31 server_cookie=eed5140212d895b2 in_seq=0 out_seq=0 2024-06-10T01:29:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.102+0000 7f18b8ff9640 10 client.21015 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.103+0000 7f18b8ff9640 1 -- 172.21.15.7:0/1422444474 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5637ce4cff10 con 0x5637ce56efd0 2024-06-10T01:29:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.103+0000 7f18b8ff9640 10 client.21015 handle_command_reply: tid=0 2024-06-10T01:29:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.103+0000 7f18da5da740 1 client.21015 shutdown 2024-06-10T01:29:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.103+0000 7f18da5da740 1 -- 172.21.15.7:0/1422444474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ce559a00 msgr2=0x5637ce5635e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.103+0000 7f18da5da740 1 --2- 172.21.15.7:0/1422444474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ce559a00 0x5637ce5635e0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f18c000b920 tx=0x7f18c000bde0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.103+0000 7f18da5da740 1 -- 172.21.15.7:0/1422444474 shutdown_connections 2024-06-10T01:29:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.103+0000 7f18da5da740 1 --2- 172.21.15.7:0/1422444474 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5637ce56efd0 0x5637ce571430 crc :-1 s=READY pgs=527 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.103+0000 7f18da5da740 1 --2- 172.21.15.7:0/1422444474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ce55eb70 0x5637ce56a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.103+0000 7f18b8ff9640 0 client.21015 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.103+0000 7f18da5da740 1 --2- 172.21.15.7:0/1422444474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ce559a00 0x5637ce5635e0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.103+0000 7f18da5da740 1 --2- 172.21.15.7:0/1422444474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ce559010 0x5637ce5630a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.103+0000 7f18da5da740 1 -- 172.21.15.7:0/1422444474 >> 172.21.15.7:0/1422444474 conn(0x5637ce554680 msgr2=0x5637ce5537e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.103+0000 7f18da5da740 1 -- 172.21.15.7:0/1422444474 shutdown_connections 2024-06-10T01:29:36.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.103+0000 7f18da5da740 1 -- 172.21.15.7:0/1422444474 wait complete. 2024-06-10T01:29:36.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.103+0000 7f18da5da740 20 client.21015 trim_cache size 0 max 16384 2024-06-10T01:29:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.119+0000 7f18da5da740 1 Processor -- start 2024-06-10T01:29:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.119+0000 7f18da5da740 1 -- start start 2024-06-10T01:29:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.119+0000 7f18da5da740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ce559010 0x5637ce5630a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.119+0000 7f18da5da740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ce559a00 0x5637ce5635e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.119+0000 7f18da5da740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ce55eb70 0x5637ce56a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.119+0000 7f18da5da740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f18d01af0c0 con 0x5637ce559a00 2024-06-10T01:29:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.119+0000 7f18da5da740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f18d01ac990 con 0x5637ce55eb70 2024-06-10T01:29:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.119+0000 7f18da5da740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f18d01aa7e0 con 0x5637ce559010 2024-06-10T01:29:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.119+0000 7f18d4d89640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ce559a00 0x5637ce5635e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.119+0000 7f18d4d89640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ce559a00 0x5637ce5635e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47086/0 (socket says 172.21.15.7:47086) 2024-06-10T01:29:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.120+0000 7f18d4d89640 1 -- 172.21.15.7:0/3227324447 learned_addr learned my addr 172.21.15.7:0/3227324447 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.120+0000 7f18d5d8b640 1 --2- 172.21.15.7:0/3227324447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ce55eb70 0x5637ce56a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.120+0000 7f18d558a640 1 --2- 172.21.15.7:0/3227324447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ce559010 0x5637ce5630a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.120+0000 7f18d4d89640 1 -- 172.21.15.7:0/3227324447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ce559010 msgr2=0x5637ce5630a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.120+0000 7f18d4d89640 1 --2- 172.21.15.7:0/3227324447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ce559010 0x5637ce5630a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.120+0000 7f18d4d89640 1 -- 172.21.15.7:0/3227324447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ce55eb70 msgr2=0x5637ce56a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.120+0000 7f18d4d89640 1 --2- 172.21.15.7:0/3227324447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ce55eb70 0x5637ce56a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.120+0000 7f18d4d89640 1 -- 172.21.15.7:0/3227324447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637ce5544b0 con 0x5637ce559a00 2024-06-10T01:29:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.120+0000 7f18d4d89640 1 --2- 172.21.15.7:0/3227324447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ce559a00 0x5637ce5635e0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f18c0004340 tx=0x7f18c0004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d4b1466435b05f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.120+0000 7f18baffd640 1 -- 172.21.15.7:0/3227324447 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c002b020 con 0x5637ce559a00 2024-06-10T01:29:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.120+0000 7f18baffd640 1 -- 172.21.15.7:0/3227324447 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18c0005560 con 0x5637ce559a00 2024-06-10T01:29:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.120+0000 7f18baffd640 1 -- 172.21.15.7:0/3227324447 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c0027b70 con 0x5637ce559a00 2024-06-10T01:29:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.121+0000 7f18da5da740 1 -- 172.21.15.7:0/3227324447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ce559a00 msgr2=0x5637ce5635e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.121+0000 7f18da5da740 1 --2- 172.21.15.7:0/3227324447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ce559a00 0x5637ce5635e0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f18c0004340 tx=0x7f18c0004460 comp rx=0 tx=0).stop 2024-06-10T01:29:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.121+0000 7f18da5da740 1 -- 172.21.15.7:0/3227324447 shutdown_connections 2024-06-10T01:29:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.121+0000 7f18da5da740 1 --2- 172.21.15.7:0/3227324447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ce55eb70 0x5637ce56a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.121+0000 7f18da5da740 1 --2- 172.21.15.7:0/3227324447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ce559a00 0x5637ce5635e0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.121+0000 7f18da5da740 1 --2- 172.21.15.7:0/3227324447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ce559010 0x5637ce5630a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.121+0000 7f18da5da740 1 -- 172.21.15.7:0/3227324447 >> 172.21.15.7:0/3227324447 conn(0x5637ce554680 msgr2=0x5637ce55e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.121+0000 7f18da5da740 1 -- 172.21.15.7:0/3227324447 shutdown_connections 2024-06-10T01:29:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.121+0000 7f18da5da740 1 -- 172.21.15.7:0/3227324447 wait complete. 2024-06-10T01:29:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.121+0000 7f18da5da740 1 -- start start 2024-06-10T01:29:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.121+0000 7f18da5da740 1 Processor -- start 2024-06-10T01:29:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.122+0000 7f18da5da740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ce559010 0x5637ce5677f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.122+0000 7f18da5da740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ce559a00 0x5637ce567d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.122+0000 7f18da5da740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ce55eb70 0x5637ce568d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.122+0000 7f18da5da740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f18d01aae50 con 0x5637ce559010 2024-06-10T01:29:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.122+0000 7f18da5da740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f18d01ac2c0 con 0x5637ce55eb70 2024-06-10T01:29:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.122+0000 7f18da5da740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f18d01ab520 con 0x5637ce559a00 2024-06-10T01:29:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.122+0000 7f18da5da740 10 client.0 _ll_register_callbacks cb 0x7f18c0004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:29:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.122+0000 7f18d558a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ce559010 0x5637ce5677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.122+0000 7f18d558a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ce559010 0x5637ce5677f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47092/0 (socket says 172.21.15.7:47092) 2024-06-10T01:29:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.122+0000 7f18d558a640 1 -- 172.21.15.7:0/1199369320 learned_addr learned my addr 172.21.15.7:0/1199369320 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.122+0000 7f18d5d8b640 1 --2- 172.21.15.7:0/1199369320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ce55eb70 0x5637ce568d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.122+0000 7f18d4d89640 1 --2- 172.21.15.7:0/1199369320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ce559a00 0x5637ce567d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.122+0000 7f18d558a640 1 -- 172.21.15.7:0/1199369320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ce559a00 msgr2=0x5637ce567d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.122+0000 7f18d558a640 1 --2- 172.21.15.7:0/1199369320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ce559a00 0x5637ce567d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.122+0000 7f18d558a640 1 -- 172.21.15.7:0/1199369320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ce55eb70 msgr2=0x5637ce568d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.122+0000 7f18d558a640 1 --2- 172.21.15.7:0/1199369320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ce55eb70 0x5637ce568d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.122+0000 7f18d558a640 1 -- 172.21.15.7:0/1199369320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637ce569330 con 0x5637ce559010 2024-06-10T01:29:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.122+0000 7f18d558a640 1 --2- 172.21.15.7:0/1199369320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ce559010 0x5637ce5677f0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f18cc097610 tx=0x7f18cc0662d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38435eb43cf07ab0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.122+0000 7f18b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:29:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.122+0000 7f18b8ff9640 1 -- 172.21.15.7:0/1199369320 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc09f560 con 0x5637ce559010 2024-06-10T01:29:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.122+0000 7f18b8ff9640 1 -- 172.21.15.7:0/1199369320 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f18cc0cc560 con 0x5637ce559010 2024-06-10T01:29:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.122+0000 7f18da5da740 1 -- 172.21.15.7:0/1199369320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18c0005560 con 0x5637ce559010 2024-06-10T01:29:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.123+0000 7f18b8ff9640 1 -- 172.21.15.7:0/1199369320 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f18cc0b41b0 con 0x5637ce559010 2024-06-10T01:29:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.123+0000 7f18da5da740 10 client.21021 fetch_fsmap learned FSMap version 55 2024-06-10T01:29:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.123+0000 7f18da5da740 1 -- 172.21.15.7:0/1199369320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5637ce56a710 con 0x5637ce559010 2024-06-10T01:29:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.123+0000 7f18b8ff9640 1 -- 172.21.15.7:0/1199369320 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18c0005560 con 0x5637ce559010 2024-06-10T01:29:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.123+0000 7f18b8ff9640 1 -- 172.21.15.7:0/1199369320 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f18cc065ea0 con 0x5637ce559010 2024-06-10T01:29:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.123+0000 7f18da5da740 10 client.21021 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:29:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.123+0000 7f18da5da740 10 client.21021 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:29:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.123+0000 7f18da5da740 20 client.21021 populate_metadata read hostname 'smithi007' 2024-06-10T01:29:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.123+0000 7f18da5da740 1 --2- 172.21.15.7:0/1199369320 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5637ce5da290 0x5637ce5dc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.123+0000 7f18da5da740 4 client.21021 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:29:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.123+0000 7f18da5da740 1 -- 172.21.15.7:0/1199369320 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5637ce4c9c20 con 0x5637ce5da290 2024-06-10T01:29:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.123+0000 7f18d4d89640 1 --2- 172.21.15.7:0/1199369320 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5637ce5da290 0x5637ce5dc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.124+0000 7f18d4d89640 1 --2- 172.21.15.7:0/1199369320 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5637ce5da290 0x5637ce5dc680 crc :-1 s=READY pgs=528 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dae315fa79e6faec server_cookie=f1e60a8123e2d68d in_seq=0 out_seq=0 2024-06-10T01:29:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.124+0000 7f18b8ff9640 10 client.21021 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.281+0000 7f18b8ff9640 1 -- 172.21.15.7:0/1199369320 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5637ce4c9c20 con 0x5637ce5da290 2024-06-10T01:29:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.281+0000 7f18b8ff9640 10 client.21021 handle_command_reply: tid=0 2024-06-10T01:29:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.281+0000 7f18da5da740 1 client.21021 shutdown 2024-06-10T01:29:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.281+0000 7f18da5da740 1 -- 172.21.15.7:0/1199369320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ce559010 msgr2=0x5637ce5677f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.281+0000 7f18da5da740 1 --2- 172.21.15.7:0/1199369320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ce559010 0x5637ce5677f0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f18cc097610 tx=0x7f18cc0662d0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.281+0000 7f18da5da740 1 -- 172.21.15.7:0/1199369320 shutdown_connections 2024-06-10T01:29:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.281+0000 7f18da5da740 1 --2- 172.21.15.7:0/1199369320 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5637ce5da290 0x5637ce5dc680 crc :-1 s=READY pgs=528 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.281+0000 7f18da5da740 1 --2- 172.21.15.7:0/1199369320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ce55eb70 0x5637ce568d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.281+0000 7f18da5da740 1 --2- 172.21.15.7:0/1199369320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ce559a00 0x5637ce567d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.281+0000 7f18b8ff9640 0 client.21021 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.281+0000 7f18da5da740 1 --2- 172.21.15.7:0/1199369320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ce559010 0x5637ce5677f0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.281+0000 7f18da5da740 1 -- 172.21.15.7:0/1199369320 >> 172.21.15.7:0/1199369320 conn(0x5637ce554680 msgr2=0x5637ce55ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.281+0000 7f18da5da740 1 -- 172.21.15.7:0/1199369320 shutdown_connections 2024-06-10T01:29:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.281+0000 7f18da5da740 1 -- 172.21.15.7:0/1199369320 wait complete. 2024-06-10T01:29:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.282+0000 7f18da5da740 20 client.21021 trim_cache size 0 max 16384 2024-06-10T01:29:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:29:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:29:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:29:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.282+0000 7f18d77ee640 1 -- 172.21.15.7:0/4171902140 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f18bc0a2c00 msgr2=0x7f18bc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.282+0000 7f18d77ee640 1 --2- 172.21.15.7:0/4171902140 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f18bc0a2c00 0x7f18bc0a50c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f18c804e000 tx=0x7f18c804c000 comp rx=0 tx=0).stop 2024-06-10T01:29:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.282+0000 7f18d77ee640 1 -- 172.21.15.7:0/4171902140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18d0102480 msgr2=0x7f18d0101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.282+0000 7f18d77ee640 1 --2- 172.21.15.7:0/4171902140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18d0102480 0x7f18d0101df0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f18cc00d950 tx=0x7f18cc00de10 comp rx=0 tx=0).stop 2024-06-10T01:29:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.282+0000 7f18d77ee640 1 -- 172.21.15.7:0/4171902140 shutdown_connections 2024-06-10T01:29:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.282+0000 7f18d77ee640 1 --2- 172.21.15.7:0/4171902140 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f18bc0a2c00 0x7f18bc0a50c0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.282+0000 7f18d77ee640 1 --2- 172.21.15.7:0/4171902140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18d01076e0 0x7f18d0100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.282+0000 7f18d77ee640 1 --2- 172.21.15.7:0/4171902140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18d0104db0 0x7f18d0100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.282+0000 7f18d77ee640 1 --2- 172.21.15.7:0/4171902140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18d0102480 0x7f18d0101df0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.282+0000 7f18d77ee640 1 -- 172.21.15.7:0/4171902140 >> 172.21.15.7:0/4171902140 conn(0x7f18d00fbe70 msgr2=0x7f18d0102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.282+0000 7f18d77ee640 1 -- 172.21.15.7:0/4171902140 shutdown_connections 2024-06-10T01:29:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:36.282+0000 7f18d77ee640 1 -- 172.21.15.7:0/4171902140 wait complete. 2024-06-10T01:29:36.564 INFO:tasks.workunit.client.0.smithi007.stdout: 10 247957 22.65 MB/sec execute 493 sec latency 3433.840 ms 2024-06-10T01:29:37.564 INFO:tasks.workunit.client.0.smithi007.stdout: 10 247957 22.61 MB/sec execute 494 sec latency 4433.908 ms 2024-06-10T01:29:37.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:37 smithi078 ceph-mon[36683]: pgmap v1142: 129 pgs: 129 active+clean; 719 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 11 MiB/s wr, 645 op/s 2024-06-10T01:29:37.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:37 smithi175 ceph-mon[36636]: pgmap v1142: 129 pgs: 129 active+clean; 719 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 11 MiB/s wr, 645 op/s 2024-06-10T01:29:38.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:37 smithi007 ceph-mon[32398]: pgmap v1142: 129 pgs: 129 active+clean; 719 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 11 MiB/s wr, 645 op/s 2024-06-10T01:29:38.564 INFO:tasks.workunit.client.0.smithi007.stdout: 10 247957 22.56 MB/sec execute 495 sec latency 5433.974 ms 2024-06-10T01:29:39.564 INFO:tasks.workunit.client.0.smithi007.stdout: 10 247957 22.52 MB/sec execute 496 sec latency 6434.077 ms 2024-06-10T01:29:39.813 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:39 smithi007 ceph-mon[32398]: pgmap v1143: 129 pgs: 129 active+clean; 664 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 11 MiB/s wr, 723 op/s 2024-06-10T01:29:39.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:39 smithi078 ceph-mon[36683]: pgmap v1143: 129 pgs: 129 active+clean; 664 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 11 MiB/s wr, 723 op/s 2024-06-10T01:29:39.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:39 smithi175 ceph-mon[36636]: pgmap v1143: 129 pgs: 129 active+clean; 664 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 11 MiB/s wr, 723 op/s 2024-06-10T01:29:40.565 INFO:tasks.workunit.client.0.smithi007.stdout: 10 247957 22.47 MB/sec execute 497 sec latency 7434.168 ms 2024-06-10T01:29:41.564 INFO:tasks.workunit.client.0.smithi007.stdout: 10 247957 22.43 MB/sec execute 498 sec latency 8434.229 ms 2024-06-10T01:29:41.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:41 smithi078 ceph-mon[36683]: pgmap v1144: 129 pgs: 129 active+clean; 662 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 551 op/s 2024-06-10T01:29:41.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:41 smithi175 ceph-mon[36636]: pgmap v1144: 129 pgs: 129 active+clean; 662 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 551 op/s 2024-06-10T01:29:42.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:41 smithi007 ceph-mon[32398]: pgmap v1144: 129 pgs: 129 active+clean; 662 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 551 op/s 2024-06-10T01:29:42.565 INFO:tasks.workunit.client.0.smithi007.stdout: 10 247957 22.38 MB/sec execute 499 sec latency 9434.322 ms 2024-06-10T01:29:43.565 INFO:tasks.workunit.client.0.smithi007.stdout: 10 247957 22.34 MB/sec execute 500 sec latency 10434.424 ms 2024-06-10T01:29:43.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:43 smithi078 ceph-mon[36683]: pgmap v1145: 129 pgs: 129 active+clean; 658 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.9 MiB/s wr, 471 op/s 2024-06-10T01:29:43.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:43 smithi175 ceph-mon[36636]: pgmap v1145: 129 pgs: 129 active+clean; 658 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.9 MiB/s wr, 471 op/s 2024-06-10T01:29:44.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:43 smithi007 ceph-mon[32398]: pgmap v1145: 129 pgs: 129 active+clean; 658 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.9 MiB/s wr, 471 op/s 2024-06-10T01:29:44.565 INFO:tasks.workunit.client.0.smithi007.stdout: 10 247957 22.29 MB/sec execute 501 sec latency 11434.535 ms 2024-06-10T01:29:45.565 INFO:tasks.workunit.client.0.smithi007.stdout: 10 247957 22.25 MB/sec execute 502 sec latency 12434.651 ms 2024-06-10T01:29:45.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:45 smithi078 ceph-mon[36683]: pgmap v1146: 129 pgs: 129 active+clean; 658 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 336 op/s 2024-06-10T01:29:45.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:45 smithi175 ceph-mon[36636]: pgmap v1146: 129 pgs: 129 active+clean; 658 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 336 op/s 2024-06-10T01:29:46.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:45 smithi007 ceph-mon[32398]: pgmap v1146: 129 pgs: 129 active+clean; 658 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 336 op/s 2024-06-10T01:29:46.448 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:29:46.565 INFO:tasks.workunit.client.0.smithi007.stdout: 10 247957 22.20 MB/sec execute 503 sec latency 13434.723 ms 2024-06-10T01:29:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:29:47.565 INFO:tasks.workunit.client.0.smithi007.stdout: 10 247957 22.16 MB/sec execute 504 sec latency 14434.758 ms 2024-06-10T01:29:47.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:47 smithi007 ceph-mon[32398]: pgmap v1147: 129 pgs: 129 active+clean; 658 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 287 op/s 2024-06-10T01:29:47.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:47 smithi078 ceph-mon[36683]: pgmap v1147: 129 pgs: 129 active+clean; 658 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 287 op/s 2024-06-10T01:29:47.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:47 smithi175 ceph-mon[36636]: pgmap v1147: 129 pgs: 129 active+clean; 658 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 287 op/s 2024-06-10T01:29:48.565 INFO:tasks.workunit.client.0.smithi007.stdout: 10 247957 22.12 MB/sec execute 505 sec latency 15434.791 ms 2024-06-10T01:29:48.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.960+0000 7ff2cb364700 1 -- 172.21.15.7:0/578477401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c4107840 msgr2=0x7ff2c410f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:48.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.960+0000 7ff2cb364700 1 --2- 172.21.15.7:0/578477401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c4107840 0x7ff2c410f3c0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc00b0d0 tx=0x7ff2bc00b3e0 comp rx=0 tx=0).stop 2024-06-10T01:29:48.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.961+0000 7ff2cb364700 1 -- 172.21.15.7:0/578477401 shutdown_connections 2024-06-10T01:29:48.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.961+0000 7ff2cb364700 1 --2- 172.21.15.7:0/578477401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c4107840 0x7ff2c410f3c0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:48.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.961+0000 7ff2cb364700 1 --2- 172.21.15.7:0/578477401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2c4102740 0x7ff2c4102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:48.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.961+0000 7ff2cb364700 1 --2- 172.21.15.7:0/578477401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2c41030f0 0x7ff2c4107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:48.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.961+0000 7ff2cb364700 1 -- 172.21.15.7:0/578477401 >> 172.21.15.7:0/578477401 conn(0x7ff2c40795f0 msgr2=0x7ff2c4079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:48.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.961+0000 7ff2cb364700 1 -- 172.21.15.7:0/578477401 shutdown_connections 2024-06-10T01:29:48.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.961+0000 7ff2cb364700 1 -- 172.21.15.7:0/578477401 wait complete. 2024-06-10T01:29:48.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.961+0000 7ff2cb364700 1 Processor -- start 2024-06-10T01:29:48.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.961+0000 7ff2cb364700 1 -- start start 2024-06-10T01:29:48.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.961+0000 7ff2cb364700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2c4102740 0x7ff2c419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:48.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.961+0000 7ff2cb364700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c41030f0 0x7ff2c419ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:48.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.962+0000 7ff2cb364700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2c4107840 0x7ff2c41988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:48.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.962+0000 7ff2cb364700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c4112780 con 0x7ff2c41030f0 2024-06-10T01:29:48.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.962+0000 7ff2cb364700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c4112600 con 0x7ff2c4102740 2024-06-10T01:29:48.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.962+0000 7ff2cb364700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c4112900 con 0x7ff2c4107840 2024-06-10T01:29:48.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.962+0000 7ff2c88ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c41030f0 0x7ff2c419ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:48.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.962+0000 7ff2c88ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c41030f0 0x7ff2c419ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43626/0 (socket says 172.21.15.7:43626) 2024-06-10T01:29:48.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.962+0000 7ff2c88ff700 1 -- 172.21.15.7:0/3074162591 learned_addr learned my addr 172.21.15.7:0/3074162591 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:48.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.962+0000 7ff2c9100700 1 --2- 172.21.15.7:0/3074162591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2c4102740 0x7ff2c419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:48.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.962+0000 7ff2c9901700 1 --2- 172.21.15.7:0/3074162591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2c4107840 0x7ff2c41988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:48.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.962+0000 7ff2c88ff700 1 -- 172.21.15.7:0/3074162591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2c4107840 msgr2=0x7ff2c41988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:48.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.962+0000 7ff2c88ff700 1 --2- 172.21.15.7:0/3074162591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2c4107840 0x7ff2c41988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:48.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.962+0000 7ff2c88ff700 1 -- 172.21.15.7:0/3074162591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2c4102740 msgr2=0x7ff2c419e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:48.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.962+0000 7ff2c88ff700 1 --2- 172.21.15.7:0/3074162591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2c4102740 0x7ff2c419e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:48.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.962+0000 7ff2c88ff700 1 -- 172.21.15.7:0/3074162591 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c4199020 con 0x7ff2c41030f0 2024-06-10T01:29:48.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.962+0000 7ff2c88ff700 1 --2- 172.21.15.7:0/3074162591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c41030f0 0x7ff2c419ec40 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7ff2b400c9b0 tx=0x7ff2b400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:48.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.962+0000 7ff2ba7fc700 1 -- 172.21.15.7:0/3074162591 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4007b00 con 0x7ff2c41030f0 2024-06-10T01:29:48.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.962+0000 7ff2ba7fc700 1 -- 172.21.15.7:0/3074162591 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2b4007ca0 con 0x7ff2c41030f0 2024-06-10T01:29:48.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.962+0000 7ff2cb364700 1 -- 172.21.15.7:0/3074162591 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c4199310 con 0x7ff2c41030f0 2024-06-10T01:29:48.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.962+0000 7ff2ba7fc700 1 -- 172.21.15.7:0/3074162591 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4018730 con 0x7ff2c41030f0 2024-06-10T01:29:48.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.962+0000 7ff2cb364700 1 -- 172.21.15.7:0/3074162591 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c41a55d0 con 0x7ff2c41030f0 2024-06-10T01:29:48.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.963+0000 7ff2cb364700 1 -- 172.21.15.7:0/3074162591 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c410aa10 con 0x7ff2c41030f0 2024-06-10T01:29:48.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.965+0000 7ff2ba7fc700 1 -- 172.21.15.7:0/3074162591 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7ff2b40188d0 con 0x7ff2c41030f0 2024-06-10T01:29:48.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.965+0000 7ff2ba7fc700 1 --2- 172.21.15.7:0/3074162591 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff2b00a2c20 0x7ff2b00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:48.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.965+0000 7ff2c9100700 1 --2- 172.21.15.7:0/3074162591 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff2b00a2c20 0x7ff2b00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:48.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.965+0000 7ff2ba7fc700 1 -- 172.21.15.7:0/3074162591 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7ff2b40cf620 con 0x7ff2c41030f0 2024-06-10T01:29:48.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.966+0000 7ff2c9100700 1 --2- 172.21.15.7:0/3074162591 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff2b00a2c20 0x7ff2b00a50e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0005950 tx=0x7ff2c000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:48.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:48.969+0000 7ff2ba7fc700 1 -- 172.21.15.7:0/3074162591 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff2b4014030 con 0x7ff2c41030f0 2024-06-10T01:29:49.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:49.221+0000 7ff2cb364700 1 -- 172.21.15.7:0/3074162591 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2c4069180 con 0x7ff2c41030f0 2024-06-10T01:29:49.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:49.221+0000 7ff2ba7fc700 1 -- 172.21.15.7:0/3074162591 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11852 (secure 0 0 0) 0x7ff2b409c290 con 0x7ff2c41030f0 2024-06-10T01:29:49.214 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:29:49.214 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:27:44.542390+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29813":{"gid":29813,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/138896429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":138896429},{"type":"v1","addr":"172.21.15.175:6835","nonce":138896429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24451,24434,24436]},"id":1}]} 2024-06-10T01:29:49.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:49.224+0000 7ff2cb364700 1 -- 172.21.15.7:0/3074162591 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff2b00a2c20 msgr2=0x7ff2b00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:49.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:49.224+0000 7ff2cb364700 1 --2- 172.21.15.7:0/3074162591 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff2b00a2c20 0x7ff2b00a50e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0005950 tx=0x7ff2c000a300 comp rx=0 tx=0).stop 2024-06-10T01:29:49.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:49.224+0000 7ff2cb364700 1 -- 172.21.15.7:0/3074162591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c41030f0 msgr2=0x7ff2c419ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:49.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:49.224+0000 7ff2cb364700 1 --2- 172.21.15.7:0/3074162591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c41030f0 0x7ff2c419ec40 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7ff2b400c9b0 tx=0x7ff2b400ccc0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:49.224+0000 7ff2cb364700 1 -- 172.21.15.7:0/3074162591 shutdown_connections 2024-06-10T01:29:49.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:49.224+0000 7ff2cb364700 1 --2- 172.21.15.7:0/3074162591 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff2b00a2c20 0x7ff2b00a50e0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:49.224+0000 7ff2cb364700 1 --2- 172.21.15.7:0/3074162591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2c4107840 0x7ff2c41988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:49.224+0000 7ff2cb364700 1 --2- 172.21.15.7:0/3074162591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2c4102740 0x7ff2c419e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:49.224+0000 7ff2cb364700 1 --2- 172.21.15.7:0/3074162591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c41030f0 0x7ff2c419ec40 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:49.224+0000 7ff2cb364700 1 -- 172.21.15.7:0/3074162591 >> 172.21.15.7:0/3074162591 conn(0x7ff2c40795f0 msgr2=0x7ff2c41004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:49.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:49.224+0000 7ff2cb364700 1 -- 172.21.15.7:0/3074162591 shutdown_connections 2024-06-10T01:29:49.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:49.224+0000 7ff2cb364700 1 -- 172.21.15.7:0/3074162591 wait complete. 2024-06-10T01:29:49.218 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 55 2024-06-10T01:29:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.374+0000 7f81a1066640 1 -- 172.21.15.7:0/3527316604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f819c102510 msgr2=0x7f819c104900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.374+0000 7f81a1066640 1 --2- 172.21.15.7:0/3527316604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f819c102510 0x7f819c104900 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f818c009900 tx=0x7f818c02a410 comp rx=0 tx=0).stop 2024-06-10T01:29:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.374+0000 7f81a1066640 1 -- 172.21.15.7:0/3527316604 shutdown_connections 2024-06-10T01:29:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.374+0000 7f81a1066640 1 --2- 172.21.15.7:0/3527316604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f819c107770 0x7f819c109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.374+0000 7f81a1066640 1 --2- 172.21.15.7:0/3527316604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f819c104e40 0x7f819c107230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.374+0000 7f81a1066640 1 --2- 172.21.15.7:0/3527316604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f819c102510 0x7f819c104900 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.374+0000 7f81a1066640 1 -- 172.21.15.7:0/3527316604 >> 172.21.15.7:0/3527316604 conn(0x7f819c0fbec0 msgr2=0x7f819c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.374+0000 7f81a1066640 1 -- 172.21.15.7:0/3527316604 shutdown_connections 2024-06-10T01:29:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.374+0000 7f81a1066640 1 -- 172.21.15.7:0/3527316604 wait complete. 2024-06-10T01:29:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.375+0000 7f81a1066640 1 Processor -- start 2024-06-10T01:29:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.375+0000 7f81a1066640 1 -- start start 2024-06-10T01:29:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.375+0000 7f81a1066640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f819c102510 0x7f819c197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.376+0000 7f81a1066640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f819c104e40 0x7f819c198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.376+0000 7f81a1066640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f819c107770 0x7f819c19f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.376+0000 7f81a1066640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f819c10d6a0 con 0x7f819c102510 2024-06-10T01:29:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.376+0000 7f81a1066640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f819c10d520 con 0x7f819c107770 2024-06-10T01:29:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.376+0000 7f81a1066640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f819c10d820 con 0x7f819c104e40 2024-06-10T01:29:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.376+0000 7f819ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f819c102510 0x7f819c197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.376+0000 7f819ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f819c102510 0x7f819c197cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43642/0 (socket says 172.21.15.7:43642) 2024-06-10T01:29:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.376+0000 7f819ad9d640 1 -- 172.21.15.7:0/2157078295 learned_addr learned my addr 172.21.15.7:0/2157078295 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.376+0000 7f819b59e640 1 --2- 172.21.15.7:0/2157078295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f819c107770 0x7f819c19f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.376+0000 7f819a59c640 1 --2- 172.21.15.7:0/2157078295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f819c104e40 0x7f819c198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.376+0000 7f819ad9d640 1 -- 172.21.15.7:0/2157078295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f819c104e40 msgr2=0x7f819c198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.376+0000 7f819ad9d640 1 --2- 172.21.15.7:0/2157078295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f819c104e40 0x7f819c198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.376+0000 7f819ad9d640 1 -- 172.21.15.7:0/2157078295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f819c107770 msgr2=0x7f819c19f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.376+0000 7f819ad9d640 1 --2- 172.21.15.7:0/2157078295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f819c107770 0x7f819c19f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.376+0000 7f819ad9d640 1 -- 172.21.15.7:0/2157078295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819c19f900 con 0x7f819c102510 2024-06-10T01:29:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.376+0000 7f819ad9d640 1 --2- 172.21.15.7:0/2157078295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f819c102510 0x7f819c197cc0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f818c009230 tx=0x7f818c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.376+0000 7f8183fff640 1 -- 172.21.15.7:0/2157078295 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c03f030 con 0x7f819c102510 2024-06-10T01:29:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.376+0000 7f8183fff640 1 -- 172.21.15.7:0/2157078295 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f818c003ce0 con 0x7f819c102510 2024-06-10T01:29:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.376+0000 7f81a1066640 1 -- 172.21.15.7:0/2157078295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f819c19fb90 con 0x7f819c102510 2024-06-10T01:29:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.376+0000 7f8183fff640 1 -- 172.21.15.7:0/2157078295 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c035070 con 0x7f819c102510 2024-06-10T01:29:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.376+0000 7f81a1066640 1 -- 172.21.15.7:0/2157078295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f819c1a00d0 con 0x7f819c102510 2024-06-10T01:29:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.379+0000 7f8183fff640 1 -- 172.21.15.7:0/2157078295 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f818c042030 con 0x7f819c102510 2024-06-10T01:29:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.379+0000 7f81a3d70740 1 Processor -- start 2024-06-10T01:29:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.379+0000 7f81a3d70740 1 -- start start 2024-06-10T01:29:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.379+0000 7f81a3d70740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f08205b010 0x55f08205b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.379+0000 7f81a3d70740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f08205ba00 0x55f082060500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.379+0000 7f81a3d70740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f082060b70 0x55f082063000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.379+0000 7f81a3d70740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f081fbe4f0 con 0x55f082060b70 2024-06-10T01:29:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.379+0000 7f81a3d70740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f081f5c870 con 0x55f08205ba00 2024-06-10T01:29:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.379+0000 7f81a3d70740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f081fba4b0 con 0x55f08205b010 2024-06-10T01:29:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.379+0000 7f819a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f082060b70 0x55f082063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.379+0000 7f819a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f082060b70 0x55f082063000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43652/0 (socket says 172.21.15.7:43652) 2024-06-10T01:29:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.379+0000 7f819a59c640 1 -- 172.21.15.7:0/135039343 learned_addr learned my addr 172.21.15.7:0/135039343 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.380+0000 7f8183fff640 1 --2- 172.21.15.7:0/2157078295 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f81840a2c00 0x7f81840a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.380+0000 7f819ad9d640 1 --2- 172.21.15.7:0/135039343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f08205ba00 0x55f082060500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.380+0000 7f819b59e640 1 --2- 172.21.15.7:0/135039343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f08205b010 0x55f08205b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.380+0000 7f819a59c640 1 -- 172.21.15.7:0/135039343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f08205b010 msgr2=0x55f08205b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.380+0000 7f819a59c640 1 --2- 172.21.15.7:0/135039343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f08205b010 0x55f08205b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.380+0000 7f819a59c640 1 -- 172.21.15.7:0/135039343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f08205ba00 msgr2=0x55f082060500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.380+0000 7f819a59c640 1 --2- 172.21.15.7:0/135039343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f08205ba00 0x55f082060500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.380+0000 7f819a59c640 1 -- 172.21.15.7:0/135039343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f081f789b0 con 0x55f082060b70 2024-06-10T01:29:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.380+0000 7f819b59e640 1 --2- 172.21.15.7:0/2157078295 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f81840a2c00 0x7f81840a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.380+0000 7f8183fff640 1 -- 172.21.15.7:0/2157078295 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f818c0efd60 con 0x7f819c102510 2024-06-10T01:29:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.380+0000 7f819a59c640 1 --2- 172.21.15.7:0/135039343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f082060b70 0x55f082063000 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f819000dcb0 tx=0x7f819000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72317e4055edce1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.380+0000 7f81817fa640 1 -- 172.21.15.7:0/135039343 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8190004030 con 0x55f082060b70 2024-06-10T01:29:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.380+0000 7f81817fa640 1 -- 172.21.15.7:0/135039343 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81900041d0 con 0x55f082060b70 2024-06-10T01:29:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.380+0000 7f81817fa640 1 -- 172.21.15.7:0/135039343 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81900116a0 con 0x55f082060b70 2024-06-10T01:29:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.380+0000 7f819b59e640 1 --2- 172.21.15.7:0/2157078295 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f81840a2c00 0x7f81840a50c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f8194007310 tx=0x7f8194004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.381+0000 7f81a3d70740 1 -- 172.21.15.7:0/135039343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f082060b70 msgr2=0x55f082063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.381+0000 7f81a3d70740 1 --2- 172.21.15.7:0/135039343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f082060b70 0x55f082063000 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f819000dcb0 tx=0x7f819000f730 comp rx=0 tx=0).stop 2024-06-10T01:29:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.381+0000 7f81a3d70740 1 -- 172.21.15.7:0/135039343 shutdown_connections 2024-06-10T01:29:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.381+0000 7f81a3d70740 1 --2- 172.21.15.7:0/135039343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f082060b70 0x55f082063000 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.381+0000 7f81a3d70740 1 --2- 172.21.15.7:0/135039343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f08205ba00 0x55f082060500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.381+0000 7f81a3d70740 1 --2- 172.21.15.7:0/135039343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f08205b010 0x55f08205b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.381+0000 7f81a3d70740 1 -- 172.21.15.7:0/135039343 >> 172.21.15.7:0/135039343 conn(0x55f082056680 msgr2=0x55f082056a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.381+0000 7f81a3d70740 1 -- 172.21.15.7:0/135039343 shutdown_connections 2024-06-10T01:29:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.381+0000 7f81a3d70740 1 -- 172.21.15.7:0/135039343 wait complete. 2024-06-10T01:29:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.381+0000 7f81a3d70740 1 -- start start 2024-06-10T01:29:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.381+0000 7f81a3d70740 1 Processor -- start 2024-06-10T01:29:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.381+0000 7f81a3d70740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f08205b010 0x55f0820651a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.381+0000 7f81a3d70740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f08205ba00 0x55f0820656e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.382+0000 7f81a3d70740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f082060b70 0x55f08206c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.382+0000 7f81a3d70740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f082063b20 con 0x55f082060b70 2024-06-10T01:29:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.382+0000 7f81a3d70740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f081f47ae0 con 0x55f08205b010 2024-06-10T01:29:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.382+0000 7f81a3d70740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f082063ca0 con 0x55f08205ba00 2024-06-10T01:29:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.382+0000 7f81a3d70740 10 client.0 _ll_register_callbacks cb 0x55f081f809b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:29:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.382+0000 7f819b59e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f082060b70 0x55f08206c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.382+0000 7f819b59e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f082060b70 0x55f08206c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43664/0 (socket says 172.21.15.7:43664) 2024-06-10T01:29:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.382+0000 7f819b59e640 1 -- 172.21.15.7:0/3217991349 learned_addr learned my addr 172.21.15.7:0/3217991349 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.382+0000 7f819ad9d640 1 --2- 172.21.15.7:0/3217991349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f08205b010 0x55f0820651a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.382+0000 7f819a59c640 1 --2- 172.21.15.7:0/3217991349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f08205ba00 0x55f0820656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.382+0000 7f819b59e640 1 -- 172.21.15.7:0/3217991349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f08205ba00 msgr2=0x55f0820656e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.382+0000 7f819b59e640 1 --2- 172.21.15.7:0/3217991349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f08205ba00 0x55f0820656e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.382+0000 7f819b59e640 1 -- 172.21.15.7:0/3217991349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f08205b010 msgr2=0x55f0820651a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.382+0000 7f819b59e640 1 --2- 172.21.15.7:0/3217991349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f08205b010 0x55f0820651a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.382+0000 7f819b59e640 1 -- 172.21.15.7:0/3217991349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f081cd2210 con 0x55f082060b70 2024-06-10T01:29:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.382+0000 7f819b59e640 1 --2- 172.21.15.7:0/3217991349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f082060b70 0x55f08206c720 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f81940049b0 tx=0x7f8194004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494ffc2c9f2afc29 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.382+0000 7f817eff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:29:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.382+0000 7f817eff5640 1 -- 172.21.15.7:0/3217991349 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8194004d20 con 0x55f082060b70 2024-06-10T01:29:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.382+0000 7f81a3d70740 1 -- 172.21.15.7:0/3217991349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81900041d0 con 0x55f082060b70 2024-06-10T01:29:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.382+0000 7f817eff5640 1 -- 172.21.15.7:0/3217991349 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f8194005020 con 0x55f082060b70 2024-06-10T01:29:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.382+0000 7f817eff5640 1 -- 172.21.15.7:0/3217991349 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f8194005220 con 0x55f082060b70 2024-06-10T01:29:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.382+0000 7f81a3d70740 10 client.21039 fetch_fsmap learned FSMap version 55 2024-06-10T01:29:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.383+0000 7f81a3d70740 1 -- 172.21.15.7:0/3217991349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55f081d9e620 con 0x55f082060b70 2024-06-10T01:29:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.383+0000 7f817eff5640 1 -- 172.21.15.7:0/3217991349 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81900041d0 con 0x55f082060b70 2024-06-10T01:29:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.383+0000 7f817eff5640 1 -- 172.21.15.7:0/3217991349 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f819401ea90 con 0x55f082060b70 2024-06-10T01:29:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.383+0000 7f81a3d70740 10 client.21039 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:29:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.383+0000 7f81a3d70740 10 client.21039 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:29:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.383+0000 7f81a3d70740 20 client.21039 populate_metadata read hostname 'smithi007' 2024-06-10T01:29:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.383+0000 7f81a3d70740 1 --2- 172.21.15.7:0/3217991349 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f082071120 0x55f0820735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.383+0000 7f81a3d70740 4 client.21039 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:29:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.383+0000 7f81a3d70740 1 -- 172.21.15.7:0/3217991349 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f081fd1f10 con 0x55f082071120 2024-06-10T01:29:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.383+0000 7f819ad9d640 1 --2- 172.21.15.7:0/3217991349 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f082071120 0x55f0820735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.384+0000 7f817eff5640 10 client.21039 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.384+0000 7f819ad9d640 1 --2- 172.21.15.7:0/3217991349 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f082071120 0x55f0820735c0 crc :-1 s=READY pgs=529 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75241df957c91706 server_cookie=c7fe6f6c8294187f in_seq=0 out_seq=0 2024-06-10T01:29:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.384+0000 7f817eff5640 1 -- 172.21.15.7:0/3217991349 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f081fd1f10 con 0x55f082071120 2024-06-10T01:29:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.384+0000 7f817eff5640 10 client.21039 handle_command_reply: tid=0 2024-06-10T01:29:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.384+0000 7f81a3d70740 1 client.21039 shutdown 2024-06-10T01:29:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.384+0000 7f81a3d70740 1 -- 172.21.15.7:0/3217991349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f082060b70 msgr2=0x55f08206c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.385+0000 7f81a3d70740 1 --2- 172.21.15.7:0/3217991349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f082060b70 0x55f08206c720 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f81940049b0 tx=0x7f8194004ab0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.385+0000 7f81a3d70740 1 -- 172.21.15.7:0/3217991349 shutdown_connections 2024-06-10T01:29:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.385+0000 7f81a3d70740 1 --2- 172.21.15.7:0/3217991349 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f082071120 0x55f0820735c0 crc :-1 s=READY pgs=529 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.385+0000 7f81a3d70740 1 --2- 172.21.15.7:0/3217991349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f082060b70 0x55f08206c720 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.385+0000 7f81a3d70740 1 --2- 172.21.15.7:0/3217991349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f08205ba00 0x55f0820656e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.385+0000 7f817eff5640 0 client.21039 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.385+0000 7f81a3d70740 1 --2- 172.21.15.7:0/3217991349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f08205b010 0x55f0820651a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.385+0000 7f81a3d70740 1 -- 172.21.15.7:0/3217991349 >> 172.21.15.7:0/3217991349 conn(0x55f082056680 msgr2=0x55f0820557e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.385+0000 7f81a3d70740 1 -- 172.21.15.7:0/3217991349 shutdown_connections 2024-06-10T01:29:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.385+0000 7f81a3d70740 1 -- 172.21.15.7:0/3217991349 wait complete. 2024-06-10T01:29:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.385+0000 7f81a3d70740 20 client.21039 trim_cache size 0 max 16384 2024-06-10T01:29:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.400+0000 7f81a3d70740 1 Processor -- start 2024-06-10T01:29:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.401+0000 7f81a3d70740 1 -- start start 2024-06-10T01:29:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.401+0000 7f81a3d70740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f08205b010 0x55f0820651a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.401+0000 7f81a3d70740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f08205ba00 0x55f0820656e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.401+0000 7f81a3d70740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f082060b70 0x55f08206c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.401+0000 7f81a3d70740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f819c1acdf0 con 0x55f08205b010 2024-06-10T01:29:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.401+0000 7f81a3d70740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f819c1aa6c0 con 0x55f08205ba00 2024-06-10T01:29:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.401+0000 7f81a3d70740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f819c1a8510 con 0x55f082060b70 2024-06-10T01:29:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.401+0000 7f819ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f08205b010 0x55f0820651a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.401+0000 7f819ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f08205b010 0x55f0820651a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43676/0 (socket says 172.21.15.7:43676) 2024-06-10T01:29:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.401+0000 7f819ad9d640 1 -- 172.21.15.7:0/1225706309 learned_addr learned my addr 172.21.15.7:0/1225706309 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.401+0000 7f819a59c640 1 --2- 172.21.15.7:0/1225706309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f08205ba00 0x55f0820656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.401+0000 7f819ad9d640 1 -- 172.21.15.7:0/1225706309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f082060b70 msgr2=0x55f08206c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.401+0000 7f819b59e640 1 --2- 172.21.15.7:0/1225706309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f082060b70 0x55f08206c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.401+0000 7f819ad9d640 1 --2- 172.21.15.7:0/1225706309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f082060b70 0x55f08206c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.401+0000 7f819ad9d640 1 -- 172.21.15.7:0/1225706309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f08205ba00 msgr2=0x55f0820656e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.401+0000 7f819ad9d640 1 --2- 172.21.15.7:0/1225706309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f08205ba00 0x55f0820656e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.401+0000 7f819ad9d640 1 -- 172.21.15.7:0/1225706309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0820564b0 con 0x55f08205b010 2024-06-10T01:29:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.401+0000 7f819a59c640 1 --2- 172.21.15.7:0/1225706309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f08205ba00 0x55f0820656e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:29:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.401+0000 7f819b59e640 1 --2- 172.21.15.7:0/1225706309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f082060b70 0x55f08206c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:29:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.401+0000 7f819ad9d640 1 --2- 172.21.15.7:0/1225706309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f08205b010 0x55f0820651a0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f818c003fb0 tx=0x7f818c0dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d57bec4c21ee1e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.401+0000 7f8180ff9640 1 -- 172.21.15.7:0/1225706309 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c049040 con 0x55f08205b010 2024-06-10T01:29:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.402+0000 7f8180ff9640 1 -- 172.21.15.7:0/1225706309 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f818c0bc240 con 0x55f08205b010 2024-06-10T01:29:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.402+0000 7f8180ff9640 1 -- 172.21.15.7:0/1225706309 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c0c0170 con 0x55f08205b010 2024-06-10T01:29:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.402+0000 7f81a3d70740 1 -- 172.21.15.7:0/1225706309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f08205b010 msgr2=0x55f0820651a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.402+0000 7f81a3d70740 1 --2- 172.21.15.7:0/1225706309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f08205b010 0x55f0820651a0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f818c003fb0 tx=0x7f818c0dc040 comp rx=0 tx=0).stop 2024-06-10T01:29:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.402+0000 7f81a3d70740 1 -- 172.21.15.7:0/1225706309 shutdown_connections 2024-06-10T01:29:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.402+0000 7f81a3d70740 1 --2- 172.21.15.7:0/1225706309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f082060b70 0x55f08206c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.402+0000 7f81a3d70740 1 --2- 172.21.15.7:0/1225706309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f08205ba00 0x55f0820656e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.402+0000 7f81a3d70740 1 --2- 172.21.15.7:0/1225706309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f08205b010 0x55f0820651a0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.402+0000 7f81a3d70740 1 -- 172.21.15.7:0/1225706309 >> 172.21.15.7:0/1225706309 conn(0x55f082056680 msgr2=0x55f082056a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.402+0000 7f81a3d70740 1 -- 172.21.15.7:0/1225706309 shutdown_connections 2024-06-10T01:29:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.402+0000 7f81a3d70740 1 -- 172.21.15.7:0/1225706309 wait complete. 2024-06-10T01:29:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.403+0000 7f81a3d70740 1 -- start start 2024-06-10T01:29:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.403+0000 7f81a3d70740 1 Processor -- start 2024-06-10T01:29:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.403+0000 7f81a3d70740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f08205b010 0x55f0820697b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.403+0000 7f81a3d70740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f08205ba00 0x55f082069cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.403+0000 7f81a3d70740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f082060b70 0x55f0820d9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.403+0000 7f81a3d70740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f819c1a8b80 con 0x55f08205ba00 2024-06-10T01:29:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.403+0000 7f81a3d70740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f819c1a9ff0 con 0x55f082060b70 2024-06-10T01:29:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.403+0000 7f81a3d70740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f819c1a9250 con 0x55f08205b010 2024-06-10T01:29:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.403+0000 7f81a3d70740 10 client.0 _ll_register_callbacks cb 0x7f8194007860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:29:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.403+0000 7f819a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f08205ba00 0x55f082069cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.403+0000 7f819a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f08205ba00 0x55f082069cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43684/0 (socket says 172.21.15.7:43684) 2024-06-10T01:29:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.403+0000 7f819a59c640 1 -- 172.21.15.7:0/4187497441 learned_addr learned my addr 172.21.15.7:0/4187497441 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.403+0000 7f819ad9d640 1 --2- 172.21.15.7:0/4187497441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f08205b010 0x55f0820697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.403+0000 7f819b59e640 1 --2- 172.21.15.7:0/4187497441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f082060b70 0x55f0820d9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.403+0000 7f819a59c640 1 -- 172.21.15.7:0/4187497441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f08205b010 msgr2=0x55f0820697b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.403+0000 7f819a59c640 1 --2- 172.21.15.7:0/4187497441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f08205b010 0x55f0820697b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.403+0000 7f819a59c640 1 -- 172.21.15.7:0/4187497441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f082060b70 msgr2=0x55f0820d9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.403+0000 7f819a59c640 1 --2- 172.21.15.7:0/4187497441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f082060b70 0x55f0820d9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.403+0000 7f819a59c640 1 -- 172.21.15.7:0/4187497441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0820da480 con 0x55f08205ba00 2024-06-10T01:29:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.404+0000 7f819a59c640 1 --2- 172.21.15.7:0/4187497441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f08205ba00 0x55f082069cf0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f819000dcb0 tx=0x7f819000d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fcfe24432e1ee97 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.404+0000 7f817eff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:29:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.404+0000 7f817eff5640 1 -- 172.21.15.7:0/4187497441 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8190005040 con 0x55f08205ba00 2024-06-10T01:29:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.404+0000 7f817eff5640 1 -- 172.21.15.7:0/4187497441 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f81900051e0 con 0x55f08205ba00 2024-06-10T01:29:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.404+0000 7f81a3d70740 1 -- 172.21.15.7:0/4187497441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f818c0bc240 con 0x55f08205ba00 2024-06-10T01:29:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.404+0000 7f817eff5640 1 -- 172.21.15.7:0/4187497441 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f8190035050 con 0x55f08205ba00 2024-06-10T01:29:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.404+0000 7f81a3d70740 10 client.21045 fetch_fsmap learned FSMap version 55 2024-06-10T01:29:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.404+0000 7f81a3d70740 1 -- 172.21.15.7:0/4187497441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55f0820db730 con 0x55f08205ba00 2024-06-10T01:29:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.404+0000 7f817eff5640 1 -- 172.21.15.7:0/4187497441 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f818c0bc240 con 0x55f08205ba00 2024-06-10T01:29:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.404+0000 7f817eff5640 1 -- 172.21.15.7:0/4187497441 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f81900035f0 con 0x55f08205ba00 2024-06-10T01:29:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.404+0000 7f81a3d70740 10 client.21045 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:29:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.404+0000 7f81a3d70740 10 client.21045 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:29:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.404+0000 7f81a3d70740 20 client.21045 populate_metadata read hostname 'smithi007' 2024-06-10T01:29:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.404+0000 7f81a3d70740 1 --2- 172.21.15.7:0/4187497441 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f0820dbfc0 0x55f0820de3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.404+0000 7f81a3d70740 4 client.21045 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:29:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.404+0000 7f81a3d70740 1 -- 172.21.15.7:0/4187497441 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f081fcbc20 con 0x55f0820dbfc0 2024-06-10T01:29:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.405+0000 7f819ad9d640 1 --2- 172.21.15.7:0/4187497441 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f0820dbfc0 0x55f0820de3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.405+0000 7f819ad9d640 1 --2- 172.21.15.7:0/4187497441 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f0820dbfc0 0x55f0820de3e0 crc :-1 s=READY pgs=530 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f44e2afd9e9bf282 server_cookie=acd089f8aae6956 in_seq=0 out_seq=0 2024-06-10T01:29:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.405+0000 7f817eff5640 10 client.21045 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.450+0000 7f817eff5640 1 -- 172.21.15.7:0/4187497441 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f081fcbc20 con 0x55f0820dbfc0 2024-06-10T01:29:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.450+0000 7f817eff5640 10 client.21045 handle_command_reply: tid=0 2024-06-10T01:29:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.450+0000 7f81a3d70740 1 client.21045 shutdown 2024-06-10T01:29:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.451+0000 7f81a3d70740 1 -- 172.21.15.7:0/4187497441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f08205ba00 msgr2=0x55f082069cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.451+0000 7f81a3d70740 1 --2- 172.21.15.7:0/4187497441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f08205ba00 0x55f082069cf0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f819000dcb0 tx=0x7f819000d9f0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.451+0000 7f81a3d70740 1 -- 172.21.15.7:0/4187497441 shutdown_connections 2024-06-10T01:29:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.451+0000 7f81a3d70740 1 --2- 172.21.15.7:0/4187497441 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f0820dbfc0 0x55f0820de3e0 crc :-1 s=READY pgs=530 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.451+0000 7f81a3d70740 1 --2- 172.21.15.7:0/4187497441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f082060b70 0x55f0820d9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.451+0000 7f81a3d70740 1 --2- 172.21.15.7:0/4187497441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f08205ba00 0x55f082069cf0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.451+0000 7f817eff5640 0 client.21045 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.451+0000 7f81a3d70740 1 --2- 172.21.15.7:0/4187497441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f08205b010 0x55f0820697b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.451+0000 7f81a3d70740 1 -- 172.21.15.7:0/4187497441 >> 172.21.15.7:0/4187497441 conn(0x55f082056680 msgr2=0x55f08205cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.451+0000 7f81a3d70740 1 -- 172.21.15.7:0/4187497441 shutdown_connections 2024-06-10T01:29:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.451+0000 7f81a3d70740 1 -- 172.21.15.7:0/4187497441 wait complete. 2024-06-10T01:29:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.451+0000 7f81a3d70740 20 client.21045 trim_cache size 0 max 16384 2024-06-10T01:29:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:29:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:29:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:29:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.451+0000 7f81a1066640 1 -- 172.21.15.7:0/2157078295 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f81840a2c00 msgr2=0x7f81840a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.451+0000 7f81a1066640 1 --2- 172.21.15.7:0/2157078295 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f81840a2c00 0x7f81840a50c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f8194007310 tx=0x7f8194004a40 comp rx=0 tx=0).stop 2024-06-10T01:29:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.451+0000 7f81a1066640 1 -- 172.21.15.7:0/2157078295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f819c102510 msgr2=0x7f819c197cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.451+0000 7f81a1066640 1 --2- 172.21.15.7:0/2157078295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f819c102510 0x7f819c197cc0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f818c009230 tx=0x7f818c004360 comp rx=0 tx=0).stop 2024-06-10T01:29:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.451+0000 7f81a1066640 1 -- 172.21.15.7:0/2157078295 shutdown_connections 2024-06-10T01:29:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.451+0000 7f81a1066640 1 --2- 172.21.15.7:0/2157078295 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f81840a2c00 0x7f81840a50c0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.451+0000 7f81a1066640 1 --2- 172.21.15.7:0/2157078295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f819c107770 0x7f819c19f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.451+0000 7f81a1066640 1 --2- 172.21.15.7:0/2157078295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f819c104e40 0x7f819c198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.451+0000 7f81a1066640 1 --2- 172.21.15.7:0/2157078295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f819c102510 0x7f819c197cc0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.452+0000 7f81a1066640 1 -- 172.21.15.7:0/2157078295 >> 172.21.15.7:0/2157078295 conn(0x7f819c0fbec0 msgr2=0x7f819c102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.452+0000 7f81a1066640 1 -- 172.21.15.7:0/2157078295 shutdown_connections 2024-06-10T01:29:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:29:49.452+0000 7f81a1066640 1 -- 172.21.15.7:0/2157078295 wait complete. 2024-06-10T01:29:49.565 INFO:tasks.workunit.client.0.smithi007.stdout: 10 247957 22.07 MB/sec execute 506 sec latency 16434.821 ms 2024-06-10T01:29:49.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:49 smithi078 ceph-mon[36683]: pgmap v1148: 129 pgs: 129 active+clean; 658 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 317 KiB/s wr, 187 op/s 2024-06-10T01:29:49.828 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:49 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3074162591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:29:49.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:49 smithi175 ceph-mon[36636]: pgmap v1148: 129 pgs: 129 active+clean; 658 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 317 KiB/s wr, 187 op/s 2024-06-10T01:29:49.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:49 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3074162591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:29:49.962 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:49 smithi007 ceph-mon[32398]: pgmap v1148: 129 pgs: 129 active+clean; 658 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 317 KiB/s wr, 187 op/s 2024-06-10T01:29:49.962 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:49 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3074162591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:29:49.990 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db --set-id 6e35bda3 --release --await' 2024-06-10T01:29:49.990 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db --set-id 6e35bda3 --release --await 2024-06-10T01:29:50.004 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:29:50.253 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:29:50.288 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:29:50.565 INFO:tasks.workunit.client.0.smithi007.stdout: 10 247957 22.03 MB/sec execute 507 sec latency 17434.850 ms 2024-06-10T01:29:51.565 INFO:tasks.workunit.client.0.smithi007.stdout: 10 247957 21.99 MB/sec execute 508 sec latency 18434.885 ms 2024-06-10T01:29:51.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:51 smithi007 ceph-mon[32398]: pgmap v1149: 129 pgs: 129 active+clean; 658 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 33 op/s 2024-06-10T01:29:51.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:51 smithi078 ceph-mon[36683]: pgmap v1149: 129 pgs: 129 active+clean; 658 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 33 op/s 2024-06-10T01:29:51.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:51 smithi175 ceph-mon[36636]: pgmap v1149: 129 pgs: 129 active+clean; 658 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 33 op/s 2024-06-10T01:29:52.565 INFO:tasks.workunit.client.0.smithi007.stdout: 10 247957 21.94 MB/sec execute 509 sec latency 19434.949 ms 2024-06-10T01:29:53.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.268+0000 7f2cd45bb700 1 -- 172.21.15.7:0/2271780390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ccc101620 msgr2=0x7f2ccc101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:53.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.268+0000 7f2cd45bb700 1 --2- 172.21.15.7:0/2271780390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ccc101620 0x7f2ccc101a00 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f2cc80099c0 tx=0x7f2cc8009cd0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.268+0000 7f2cd45bb700 1 -- 172.21.15.7:0/2271780390 shutdown_connections 2024-06-10T01:29:53.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.268+0000 7f2cd45bb700 1 --2- 172.21.15.7:0/2271780390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ccc1066b0 0x7f2ccc10e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.268+0000 7f2cd45bb700 1 --2- 172.21.15.7:0/2271780390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ccc101620 0x7f2ccc101a00 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.268+0000 7f2cd45bb700 1 --2- 172.21.15.7:0/2271780390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ccc101fd0 0x7f2ccc105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.268+0000 7f2cd45bb700 1 -- 172.21.15.7:0/2271780390 >> 172.21.15.7:0/2271780390 conn(0x7f2ccc078340 msgr2=0x7f2ccc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:53.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.269+0000 7f2cd45bb700 1 -- 172.21.15.7:0/2271780390 shutdown_connections 2024-06-10T01:29:53.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.269+0000 7f2cd45bb700 1 -- 172.21.15.7:0/2271780390 wait complete. 2024-06-10T01:29:53.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.269+0000 7f2cd45bb700 1 Processor -- start 2024-06-10T01:29:53.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.269+0000 7f2cd45bb700 1 -- start start 2024-06-10T01:29:53.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.269+0000 7f2cd45bb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ccc101620 0x7f2ccc19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.269+0000 7f2cd45bb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ccc101fd0 0x7f2ccc19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.269+0000 7f2cd45bb700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ccc1066b0 0x7f2ccc198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.270+0000 7f2cd45bb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc1127b0 con 0x7f2ccc101fd0 2024-06-10T01:29:53.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.270+0000 7f2cd45bb700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc112630 con 0x7f2ccc101620 2024-06-10T01:29:53.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.270+0000 7f2cd45bb700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc112930 con 0x7f2ccc1066b0 2024-06-10T01:29:53.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.270+0000 7f2cd1b56700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ccc101fd0 0x7f2ccc19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.270+0000 7f2cd1b56700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ccc101fd0 0x7f2ccc19edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49358/0 (socket says 172.21.15.7:49358) 2024-06-10T01:29:53.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.270+0000 7f2cd1b56700 1 -- 172.21.15.7:0/4265450183 learned_addr learned my addr 172.21.15.7:0/4265450183 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:53.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.270+0000 7f2cd2357700 1 --2- 172.21.15.7:0/4265450183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ccc101620 0x7f2ccc19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.270+0000 7f2cd2b58700 1 --2- 172.21.15.7:0/4265450183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ccc1066b0 0x7f2ccc198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.270+0000 7f2cd1b56700 1 -- 172.21.15.7:0/4265450183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ccc1066b0 msgr2=0x7f2ccc198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:53.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.270+0000 7f2cd1b56700 1 --2- 172.21.15.7:0/4265450183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ccc1066b0 0x7f2ccc198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.270+0000 7f2cd1b56700 1 -- 172.21.15.7:0/4265450183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ccc101620 msgr2=0x7f2ccc19e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:53.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.270+0000 7f2cd1b56700 1 --2- 172.21.15.7:0/4265450183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ccc101620 0x7f2ccc19e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.270+0000 7f2cd1b56700 1 -- 172.21.15.7:0/4265450183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ccc1991b0 con 0x7f2ccc101fd0 2024-06-10T01:29:53.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.270+0000 7f2cd1b56700 1 --2- 172.21.15.7:0/4265450183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ccc101fd0 0x7f2ccc19edd0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00c8f0 tx=0x7f2cbc00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:53.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.270+0000 7f2cc37fe700 1 -- 172.21.15.7:0/4265450183 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc0078d0 con 0x7f2ccc101fd0 2024-06-10T01:29:53.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.270+0000 7f2cc37fe700 1 -- 172.21.15.7:0/4265450183 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cbc00f450 con 0x7f2ccc101fd0 2024-06-10T01:29:53.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.270+0000 7f2cd45bb700 1 -- 172.21.15.7:0/4265450183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ccc1994a0 con 0x7f2ccc101fd0 2024-06-10T01:29:53.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.270+0000 7f2cc37fe700 1 -- 172.21.15.7:0/4265450183 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc018640 con 0x7f2ccc101fd0 2024-06-10T01:29:53.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.270+0000 7f2cd45bb700 1 -- 172.21.15.7:0/4265450183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ccc1a5760 con 0x7f2ccc101fd0 2024-06-10T01:29:53.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.273+0000 7f2cc37fe700 1 -- 172.21.15.7:0/4265450183 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f2cbc00f600 con 0x7f2ccc101fd0 2024-06-10T01:29:53.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.273+0000 7f2cc37fe700 1 --2- 172.21.15.7:0/4265450183 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2cb80a2c20 0x7f2cb80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.274+0000 7f2cd2357700 1 --2- 172.21.15.7:0/4265450183 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2cb80a2c20 0x7f2cb80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.274+0000 7f2cc37fe700 1 -- 172.21.15.7:0/4265450183 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f2cbc0cf1f0 con 0x7f2ccc101fd0 2024-06-10T01:29:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.274+0000 7f2cdee3eb80 1 Processor -- start 2024-06-10T01:29:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.274+0000 7f2cd2357700 1 --2- 172.21.15.7:0/4265450183 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2cb80a2c20 0x7f2cb80a50e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8000c00 tx=0x7f2cc8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.274+0000 7f2cdee3eb80 1 -- start start 2024-06-10T01:29:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.274+0000 7f2cdee3eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3fe156050 0x55a3fe156430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.275+0000 7f2cdee3eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3fe156970 0x55a3fe1502a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.275+0000 7f2cdee3eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3fe1507e0 0x55a3fe150c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.275+0000 7f2cdee3eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a3fe09d910 con 0x55a3fe1507e0 2024-06-10T01:29:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.275+0000 7f2cdee3eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a3fe045820 con 0x55a3fe156970 2024-06-10T01:29:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.275+0000 7f2cdee3eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a3fe0998d0 con 0x55a3fe156050 2024-06-10T01:29:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.275+0000 7f2cd2b58700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3fe1507e0 0x55a3fe150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.275+0000 7f2cd2357700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3fe156050 0x55a3fe156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.275+0000 7f2cd2b58700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3fe1507e0 0x55a3fe150c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49362/0 (socket says 172.21.15.7:49362) 2024-06-10T01:29:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.275+0000 7f2cd2b58700 1 -- 172.21.15.7:0/2984919758 learned_addr learned my addr 172.21.15.7:0/2984919758 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.275+0000 7f2cd1b56700 1 --2- 172.21.15.7:0/2984919758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3fe156970 0x55a3fe1502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.275+0000 7f2cd2b58700 1 -- 172.21.15.7:0/2984919758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3fe156050 msgr2=0x55a3fe156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.275+0000 7f2cd2b58700 1 --2- 172.21.15.7:0/2984919758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3fe156050 0x55a3fe156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.275+0000 7f2cd2b58700 1 -- 172.21.15.7:0/2984919758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3fe156970 msgr2=0x55a3fe1502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.275+0000 7f2cd2b58700 1 --2- 172.21.15.7:0/2984919758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3fe156970 0x55a3fe1502a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.275+0000 7f2cd2b58700 1 -- 172.21.15.7:0/2984919758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3fe03c050 con 0x55a3fe1507e0 2024-06-10T01:29:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.275+0000 7f2cd1b56700 1 --2- 172.21.15.7:0/2984919758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3fe156970 0x55a3fe1502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:29:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.275+0000 7f2cd2b58700 1 --2- 172.21.15.7:0/2984919758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3fe1507e0 0x55a3fe150c20 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4003fa0 tx=0x7f2cc4009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3bd2a92618b4a5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.276+0000 7f2cc0ff9700 1 -- 172.21.15.7:0/2984919758 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4004bb0 con 0x55a3fe1507e0 2024-06-10T01:29:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.276+0000 7f2cc0ff9700 1 -- 172.21.15.7:0/2984919758 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc4004d50 con 0x55a3fe1507e0 2024-06-10T01:29:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.276+0000 7f2cc0ff9700 1 -- 172.21.15.7:0/2984919758 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4007dc0 con 0x55a3fe1507e0 2024-06-10T01:29:53.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.276+0000 7f2cdee3eb80 1 -- 172.21.15.7:0/2984919758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3fe1507e0 msgr2=0x55a3fe150c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:53.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.276+0000 7f2cdee3eb80 1 --2- 172.21.15.7:0/2984919758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3fe1507e0 0x55a3fe150c20 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4003fa0 tx=0x7f2cc4009ba0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.276+0000 7f2cdee3eb80 1 -- 172.21.15.7:0/2984919758 shutdown_connections 2024-06-10T01:29:53.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.276+0000 7f2cdee3eb80 1 --2- 172.21.15.7:0/2984919758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3fe1507e0 0x55a3fe150c20 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.276+0000 7f2cdee3eb80 1 --2- 172.21.15.7:0/2984919758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3fe156050 0x55a3fe156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.276+0000 7f2cdee3eb80 1 --2- 172.21.15.7:0/2984919758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3fe156970 0x55a3fe1502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.276+0000 7f2cdee3eb80 1 -- 172.21.15.7:0/2984919758 >> 172.21.15.7:0/2984919758 conn(0x55a3fe076dc0 msgr2=0x55a3fe147120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.277+0000 7f2cdee3eb80 1 -- 172.21.15.7:0/2984919758 shutdown_connections 2024-06-10T01:29:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.277+0000 7f2cdee3eb80 1 -- 172.21.15.7:0/2984919758 wait complete. 2024-06-10T01:29:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.277+0000 7f2cdee3eb80 1 -- start start 2024-06-10T01:29:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.277+0000 7f2cdee3eb80 1 Processor -- start 2024-06-10T01:29:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.277+0000 7f2cdee3eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3fe1507e0 0x55a3fe15c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.277+0000 7f2cdee3eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3fe156050 0x55a3fe159900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.277+0000 7f2cdee3eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3fe156970 0x55a3fe159e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.277+0000 7f2cdee3eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a3fe15d3e0 con 0x55a3fe156970 2024-06-10T01:29:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.277+0000 7f2cdee3eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a3fe03dcf0 con 0x55a3fe156050 2024-06-10T01:29:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.277+0000 7f2cdee3eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a3fe15d560 con 0x55a3fe1507e0 2024-06-10T01:29:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.277+0000 7f2cdee3eb80 10 client.0 _ll_register_callbacks cb 0x55a3fe037a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:29:53.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.277+0000 7f2cd2b58700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3fe156970 0x55a3fe159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.277+0000 7f2cd2b58700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3fe156970 0x55a3fe159e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49378/0 (socket says 172.21.15.7:49378) 2024-06-10T01:29:53.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.278+0000 7f2cd2b58700 1 -- 172.21.15.7:0/339843731 learned_addr learned my addr 172.21.15.7:0/339843731 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:53.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.278+0000 7f2cd1b56700 1 --2- 172.21.15.7:0/339843731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3fe156050 0x55a3fe159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.278+0000 7f2cd2357700 1 --2- 172.21.15.7:0/339843731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3fe1507e0 0x55a3fe15c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.278+0000 7f2cd2b58700 1 -- 172.21.15.7:0/339843731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3fe1507e0 msgr2=0x55a3fe15c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:53.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.278+0000 7f2cd2b58700 1 --2- 172.21.15.7:0/339843731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3fe1507e0 0x55a3fe15c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.278+0000 7f2cd2b58700 1 -- 172.21.15.7:0/339843731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3fe156050 msgr2=0x55a3fe159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:53.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.278+0000 7f2cd2b58700 1 --2- 172.21.15.7:0/339843731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3fe156050 0x55a3fe159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.278+0000 7f2cd2b58700 1 -- 172.21.15.7:0/339843731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3fdfec9d0 con 0x55a3fe156970 2024-06-10T01:29:53.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.278+0000 7f2cd2b58700 1 --2- 172.21.15.7:0/339843731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3fe156970 0x55a3fe159e40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40079e0 tx=0x7f2cc40094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d55ec2f52df40c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:53.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.278+0000 7f2cb5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:29:53.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.278+0000 7f2cb5ffb700 1 -- 172.21.15.7:0/339843731 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4022070 con 0x55a3fe156970 2024-06-10T01:29:53.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.278+0000 7f2cb5ffb700 1 -- 172.21.15.7:0/339843731 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f2cc4004b50 con 0x55a3fe156970 2024-06-10T01:29:53.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.278+0000 7f2cdee3eb80 1 -- 172.21.15.7:0/339843731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cc4004d50 con 0x55a3fe156970 2024-06-10T01:29:53.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.278+0000 7f2cb5ffb700 1 -- 172.21.15.7:0/339843731 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f2cc402e050 con 0x55a3fe156970 2024-06-10T01:29:53.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.278+0000 7f2cdee3eb80 10 client.21057 fetch_fsmap learned FSMap version 55 2024-06-10T01:29:53.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.278+0000 7f2cdee3eb80 1 -- 172.21.15.7:0/339843731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55a3fe167c40 con 0x55a3fe156970 2024-06-10T01:29:53.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.279+0000 7f2cb5ffb700 1 -- 172.21.15.7:0/339843731 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc4004d50 con 0x55a3fe156970 2024-06-10T01:29:53.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.279+0000 7f2cb5ffb700 1 -- 172.21.15.7:0/339843731 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f2cc401e410 con 0x55a3fe156970 2024-06-10T01:29:53.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.279+0000 7f2cdee3eb80 10 client.21057 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:29:53.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.279+0000 7f2cdee3eb80 10 client.21057 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:29:53.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.279+0000 7f2cdee3eb80 20 client.21057 populate_metadata read hostname 'smithi007' 2024-06-10T01:29:53.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.279+0000 7f2cdee3eb80 1 --2- 172.21.15.7:0/339843731 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a3fe16a110 0x55a3fe16c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.279+0000 7f2cdee3eb80 4 client.21057 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:29:53.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.279+0000 7f2cdee3eb80 1 -- 172.21.15.7:0/339843731 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3fe0aae10 con 0x55a3fe16a110 2024-06-10T01:29:53.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.279+0000 7f2cd2357700 1 --2- 172.21.15.7:0/339843731 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a3fe16a110 0x55a3fe16c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.280+0000 7f2cd2357700 1 --2- 172.21.15.7:0/339843731 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a3fe16a110 0x55a3fe16c500 crc :-1 s=READY pgs=531 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29530434ed270402 server_cookie=5c2115cd0fb9db01 in_seq=0 out_seq=0 2024-06-10T01:29:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.280+0000 7f2cb5ffb700 10 client.21057 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.280+0000 7f2cb5ffb700 1 -- 172.21.15.7:0/339843731 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a3fe0aae10 con 0x55a3fe16a110 2024-06-10T01:29:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.280+0000 7f2cb5ffb700 10 client.21057 handle_command_reply: tid=0 2024-06-10T01:29:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.280+0000 7f2cdee3eb80 1 client.21057 shutdown 2024-06-10T01:29:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.281+0000 7f2cdee3eb80 1 -- 172.21.15.7:0/339843731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3fe156970 msgr2=0x55a3fe159e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.281+0000 7f2cdee3eb80 1 --2- 172.21.15.7:0/339843731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3fe156970 0x55a3fe159e40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40079e0 tx=0x7f2cc40094e0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.281+0000 7f2cdee3eb80 1 -- 172.21.15.7:0/339843731 shutdown_connections 2024-06-10T01:29:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.281+0000 7f2cdee3eb80 1 --2- 172.21.15.7:0/339843731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3fe156970 0x55a3fe159e40 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.281+0000 7f2cdee3eb80 1 --2- 172.21.15.7:0/339843731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3fe1507e0 0x55a3fe15c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.281+0000 7f2cdee3eb80 1 --2- 172.21.15.7:0/339843731 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a3fe16a110 0x55a3fe16c500 crc :-1 s=READY pgs=531 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.281+0000 7f2cdee3eb80 1 --2- 172.21.15.7:0/339843731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3fe156050 0x55a3fe159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.281+0000 7f2cb5ffb700 0 client.21057 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.281+0000 7f2cdee3eb80 1 -- 172.21.15.7:0/339843731 >> 172.21.15.7:0/339843731 conn(0x55a3fe076dc0 msgr2=0x55a3fe146d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.281+0000 7f2cdee3eb80 1 -- 172.21.15.7:0/339843731 shutdown_connections 2024-06-10T01:29:53.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.281+0000 7f2cdee3eb80 1 -- 172.21.15.7:0/339843731 wait complete. 2024-06-10T01:29:53.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.281+0000 7f2cdee3eb80 20 client.21057 trim_cache size 0 max 16384 2024-06-10T01:29:53.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.301+0000 7f2cdee3eb80 1 Processor -- start 2024-06-10T01:29:53.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.301+0000 7f2cdee3eb80 1 -- start start 2024-06-10T01:29:53.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.301+0000 7f2cdee3eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3fe1507e0 0x55a3fe158920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.302+0000 7f2cdee3eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3fe156050 0x55a3fe171b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.302+0000 7f2cdee3eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3fe156970 0x55a3fe172060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.302+0000 7f2cdee3eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc1aea90 con 0x55a3fe156970 2024-06-10T01:29:53.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.302+0000 7f2cdee3eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc1ac320 con 0x55a3fe156050 2024-06-10T01:29:53.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.302+0000 7f2cdee3eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc1aa190 con 0x55a3fe1507e0 2024-06-10T01:29:53.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.302+0000 7f2cd2b58700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3fe156970 0x55a3fe172060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.302+0000 7f2cd2b58700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3fe156970 0x55a3fe172060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49392/0 (socket says 172.21.15.7:49392) 2024-06-10T01:29:53.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.302+0000 7f2cd2b58700 1 -- 172.21.15.7:0/2271542281 learned_addr learned my addr 172.21.15.7:0/2271542281 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:53.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.302+0000 7f2cd2357700 1 --2- 172.21.15.7:0/2271542281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3fe1507e0 0x55a3fe158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.302+0000 7f2cd2b58700 1 -- 172.21.15.7:0/2271542281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3fe1507e0 msgr2=0x55a3fe158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:53.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.302+0000 7f2cd1b56700 1 --2- 172.21.15.7:0/2271542281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3fe156050 0x55a3fe171b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.302+0000 7f2cd2b58700 1 --2- 172.21.15.7:0/2271542281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3fe1507e0 0x55a3fe158920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.302+0000 7f2cd2b58700 1 -- 172.21.15.7:0/2271542281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3fe156050 msgr2=0x55a3fe171b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:53.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.302+0000 7f2cd2b58700 1 --2- 172.21.15.7:0/2271542281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3fe156050 0x55a3fe171b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.302+0000 7f2cd2b58700 1 -- 172.21.15.7:0/2271542281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3fe141820 con 0x55a3fe156970 2024-06-10T01:29:53.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.302+0000 7f2cd2b58700 1 --2- 172.21.15.7:0/2271542281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3fe156970 0x55a3fe172060 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f2cc401e9d0 tx=0x7f2cc40095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b2ab36d342943ac server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:53.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.302+0000 7f2cb7fff700 1 -- 172.21.15.7:0/2271542281 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc402b020 con 0x55a3fe156970 2024-06-10T01:29:53.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.302+0000 7f2cb7fff700 1 -- 172.21.15.7:0/2271542281 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc401e410 con 0x55a3fe156970 2024-06-10T01:29:53.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.302+0000 7f2cb7fff700 1 -- 172.21.15.7:0/2271542281 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4022070 con 0x55a3fe156970 2024-06-10T01:29:53.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.303+0000 7f2cdee3eb80 1 -- 172.21.15.7:0/2271542281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3fe156970 msgr2=0x55a3fe172060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.303+0000 7f2cdee3eb80 1 --2- 172.21.15.7:0/2271542281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3fe156970 0x55a3fe172060 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f2cc401e9d0 tx=0x7f2cc40095d0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.303+0000 7f2cdee3eb80 1 -- 172.21.15.7:0/2271542281 shutdown_connections 2024-06-10T01:29:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.303+0000 7f2cdee3eb80 1 --2- 172.21.15.7:0/2271542281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3fe156970 0x55a3fe172060 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.303+0000 7f2cdee3eb80 1 --2- 172.21.15.7:0/2271542281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3fe1507e0 0x55a3fe158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.303+0000 7f2cdee3eb80 1 --2- 172.21.15.7:0/2271542281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3fe156050 0x55a3fe171b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.303+0000 7f2cdee3eb80 1 -- 172.21.15.7:0/2271542281 >> 172.21.15.7:0/2271542281 conn(0x55a3fe076dc0 msgr2=0x55a3fe158210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.303+0000 7f2cdee3eb80 1 -- 172.21.15.7:0/2271542281 shutdown_connections 2024-06-10T01:29:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.303+0000 7f2cdee3eb80 1 -- 172.21.15.7:0/2271542281 wait complete. 2024-06-10T01:29:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.303+0000 7f2cdee3eb80 1 -- start start 2024-06-10T01:29:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.303+0000 7f2cdee3eb80 1 Processor -- start 2024-06-10T01:29:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.304+0000 7f2cdee3eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3fe1507e0 0x55a3fe15f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.304+0000 7f2cdee3eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3fe156050 0x55a3fe15f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.304+0000 7f2cdee3eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3fe156970 0x55a3fe1ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.304+0000 7f2cdee3eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc1aa7e0 con 0x55a3fe1507e0 2024-06-10T01:29:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.304+0000 7f2cdee3eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc1abc50 con 0x55a3fe156050 2024-06-10T01:29:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.304+0000 7f2cdee3eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc1aaeb0 con 0x55a3fe156970 2024-06-10T01:29:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.304+0000 7f2cdee3eb80 10 client.0 _ll_register_callbacks cb 0x55a3fe037a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:29:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.304+0000 7f2cd2357700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3fe1507e0 0x55a3fe15f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.304+0000 7f2cd2357700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3fe1507e0 0x55a3fe15f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49394/0 (socket says 172.21.15.7:49394) 2024-06-10T01:29:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.304+0000 7f2cd2357700 1 -- 172.21.15.7:0/246972862 learned_addr learned my addr 172.21.15.7:0/246972862 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.304+0000 7f2cd1b56700 1 --2- 172.21.15.7:0/246972862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3fe156050 0x55a3fe15f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.304+0000 7f2cd2b58700 1 --2- 172.21.15.7:0/246972862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3fe156970 0x55a3fe1ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.304+0000 7f2cd2357700 1 -- 172.21.15.7:0/246972862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3fe156970 msgr2=0x55a3fe1ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.304+0000 7f2cd2357700 1 --2- 172.21.15.7:0/246972862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3fe156970 0x55a3fe1ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.304+0000 7f2cd2357700 1 -- 172.21.15.7:0/246972862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3fe156050 msgr2=0x55a3fe15f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.304+0000 7f2cd2357700 1 --2- 172.21.15.7:0/246972862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3fe156050 0x55a3fe15f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.304+0000 7f2cd2357700 1 -- 172.21.15.7:0/246972862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3fe1cd160 con 0x55a3fe1507e0 2024-06-10T01:29:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.304+0000 7f2cd2357700 1 --2- 172.21.15.7:0/246972862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3fe1507e0 0x55a3fe15f140 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f2cc80099c0 tx=0x7f2cc8009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c1a59f77bbcbbb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.304+0000 7f2cb5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:29:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.304+0000 7f2cb5ffb700 1 -- 172.21.15.7:0/246972862 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc800fc50 con 0x55a3fe1507e0 2024-06-10T01:29:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.304+0000 7f2cdee3eb80 1 -- 172.21.15.7:0/246972862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cc401e410 con 0x55a3fe1507e0 2024-06-10T01:29:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.304+0000 7f2cb5ffb700 1 -- 172.21.15.7:0/246972862 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f2cc800fdf0 con 0x55a3fe1507e0 2024-06-10T01:29:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.305+0000 7f2cb5ffb700 1 -- 172.21.15.7:0/246972862 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f2cc8045050 con 0x55a3fe1507e0 2024-06-10T01:29:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.305+0000 7f2cdee3eb80 10 client.21063 fetch_fsmap learned FSMap version 55 2024-06-10T01:29:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.305+0000 7f2cdee3eb80 1 -- 172.21.15.7:0/246972862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55a3fe1d5710 con 0x55a3fe1507e0 2024-06-10T01:29:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.305+0000 7f2cb5ffb700 1 -- 172.21.15.7:0/246972862 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc401e410 con 0x55a3fe1507e0 2024-06-10T01:29:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.305+0000 7f2cb5ffb700 1 -- 172.21.15.7:0/246972862 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f2cc80244a0 con 0x55a3fe1507e0 2024-06-10T01:29:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.305+0000 7f2cdee3eb80 10 client.21063 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:29:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.305+0000 7f2cdee3eb80 10 client.21063 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:29:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.305+0000 7f2cdee3eb80 20 client.21063 populate_metadata read hostname 'smithi007' 2024-06-10T01:29:53.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.305+0000 7f2cdee3eb80 1 --2- 172.21.15.7:0/246972862 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a3fe1d5e00 0x55a3fe1d8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.305+0000 7f2cdee3eb80 4 client.21063 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6e35bda3", "release": true, "await": true}] 2024-06-10T01:29:53.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.305+0000 7f2cdee3eb80 1 -- 172.21.15.7:0/246972862 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6e35bda3", "release": true, "await": true}) -- 0x55a3fe0a5390 con 0x55a3fe1d5e00 2024-06-10T01:29:53.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.305+0000 7f2cd1b56700 1 --2- 172.21.15.7:0/246972862 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a3fe1d5e00 0x55a3fe1d8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.306+0000 7f2cd1b56700 1 --2- 172.21.15.7:0/246972862 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a3fe1d5e00 0x55a3fe1d8220 crc :-1 s=READY pgs=532 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36a02e63420b0e98 server_cookie=2b314433b224c0fd in_seq=0 out_seq=0 2024-06-10T01:29:53.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.306+0000 7f2cb5ffb700 10 client.21063 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:53.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.347+0000 7f2cb5ffb700 1 -- 172.21.15.7:0/246972862 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a3fe0a5390 con 0x55a3fe1d5e00 2024-06-10T01:29:53.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.347+0000 7f2cb5ffb700 10 client.21063 handle_command_reply: tid=0 2024-06-10T01:29:53.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.347+0000 7f2cdee3eb80 1 client.21063 shutdown 2024-06-10T01:29:53.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.347+0000 7f2cdee3eb80 1 -- 172.21.15.7:0/246972862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3fe1507e0 msgr2=0x55a3fe15f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:53.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.347+0000 7f2cdee3eb80 1 --2- 172.21.15.7:0/246972862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3fe1507e0 0x55a3fe15f140 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f2cc80099c0 tx=0x7f2cc8009cd0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.347+0000 7f2cdee3eb80 1 -- 172.21.15.7:0/246972862 shutdown_connections 2024-06-10T01:29:53.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.347+0000 7f2cdee3eb80 1 --2- 172.21.15.7:0/246972862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a3fe156970 0x55a3fe1ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.347+0000 7f2cdee3eb80 1 --2- 172.21.15.7:0/246972862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a3fe1507e0 0x55a3fe15f140 secure :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f2cc80099c0 tx=0x7f2cc8009cd0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.347+0000 7f2cdee3eb80 1 --2- 172.21.15.7:0/246972862 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55a3fe1d5e00 0x55a3fe1d8220 crc :-1 s=READY pgs=532 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.347+0000 7f2cb5ffb700 0 client.21063 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:53.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.347+0000 7f2cdee3eb80 1 --2- 172.21.15.7:0/246972862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a3fe156050 0x55a3fe15f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.347+0000 7f2cdee3eb80 1 -- 172.21.15.7:0/246972862 >> 172.21.15.7:0/246972862 conn(0x55a3fe076dc0 msgr2=0x55a3fe1470c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:53.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.350+0000 7f2cdee3eb80 1 -- 172.21.15.7:0/246972862 shutdown_connections 2024-06-10T01:29:53.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.350+0000 7f2cdee3eb80 1 -- 172.21.15.7:0/246972862 wait complete. 2024-06-10T01:29:53.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.350+0000 7f2cdee3eb80 20 client.21063 trim_cache size 0 max 16384 2024-06-10T01:29:53.343 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:29:53.343 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 55, 2024-06-10T01:29:53.343 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:29:53.343 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 76, 2024-06-10T01:29:53.344 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:29:53.344 INFO:teuthology.orchestra.run.smithi007.stdout: "6e35bda3": { 2024-06-10T01:29:53.344 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 76, 2024-06-10T01:29:53.344 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:29:53.344 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:29:53.344 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:29:53.344 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:29:53.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:29:53.344 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:29:53.344 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 142.4, 2024-06-10T01:29:53.344 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:29:53.344 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:29:53.344 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:29:53.344 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:29:53.345 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:29:53.345 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:29:53.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:29:53.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:29:53.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:29:53.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:29:53.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:29:53.345 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:29:53.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.352+0000 7f2cd45bb700 1 -- 172.21.15.7:0/4265450183 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2cb80a2c20 msgr2=0x7f2cb80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:53.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.352+0000 7f2cd45bb700 1 --2- 172.21.15.7:0/4265450183 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2cb80a2c20 0x7f2cb80a50e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8000c00 tx=0x7f2cc8005fb0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.352+0000 7f2cd45bb700 1 -- 172.21.15.7:0/4265450183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ccc101fd0 msgr2=0x7f2ccc19edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:53.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.352+0000 7f2cd45bb700 1 --2- 172.21.15.7:0/4265450183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ccc101fd0 0x7f2ccc19edd0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00c8f0 tx=0x7f2cbc00cc00 comp rx=0 tx=0).stop 2024-06-10T01:29:53.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.352+0000 7f2cd45bb700 1 -- 172.21.15.7:0/4265450183 shutdown_connections 2024-06-10T01:29:53.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.352+0000 7f2cd45bb700 1 --2- 172.21.15.7:0/4265450183 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2cb80a2c20 0x7f2cb80a50e0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.352+0000 7f2cd45bb700 1 --2- 172.21.15.7:0/4265450183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ccc1066b0 0x7f2ccc198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.352+0000 7f2cd45bb700 1 --2- 172.21.15.7:0/4265450183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ccc101620 0x7f2ccc19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.352+0000 7f2cd45bb700 1 --2- 172.21.15.7:0/4265450183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ccc101fd0 0x7f2ccc19edd0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.352+0000 7f2cd45bb700 1 -- 172.21.15.7:0/4265450183 >> 172.21.15.7:0/4265450183 conn(0x7f2ccc078340 msgr2=0x7f2ccc0ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:53.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.352+0000 7f2cd45bb700 1 -- 172.21.15.7:0/4265450183 shutdown_connections 2024-06-10T01:29:53.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.352+0000 7f2cd45bb700 1 -- 172.21.15.7:0/4265450183 wait complete. 2024-06-10T01:29:53.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.525+0000 7f9df6d09700 1 -- 172.21.15.7:0/2043882766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9df006aa30 msgr2=0x7f9df006ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:53.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.525+0000 7f9df6d09700 1 --2- 172.21.15.7:0/2043882766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9df006aa30 0x7f9df006ae10 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f9de4009990 tx=0x7f9de4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.526+0000 7f9df6d09700 1 -- 172.21.15.7:0/2043882766 shutdown_connections 2024-06-10T01:29:53.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.526+0000 7f9df6d09700 1 --2- 172.21.15.7:0/2043882766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9df010ef30 0x7f9df0111320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.526+0000 7f9df6d09700 1 --2- 172.21.15.7:0/2043882766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9df006aa30 0x7f9df006ae10 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.526+0000 7f9df6d09700 1 --2- 172.21.15.7:0/2043882766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9df006b350 0x7f9df010e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.526+0000 7f9df6d09700 1 -- 172.21.15.7:0/2043882766 >> 172.21.15.7:0/2043882766 conn(0x7f9df0078300 msgr2=0x7f9df0078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:53.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.526+0000 7f9df6d09700 1 -- 172.21.15.7:0/2043882766 shutdown_connections 2024-06-10T01:29:53.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.526+0000 7f9df6d09700 1 -- 172.21.15.7:0/2043882766 wait complete. 2024-06-10T01:29:53.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.528+0000 7f9df6d09700 1 Processor -- start 2024-06-10T01:29:53.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.528+0000 7f9df6d09700 1 -- start start 2024-06-10T01:29:53.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.528+0000 7f9df6d09700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9df006aa30 0x7f9df019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.528+0000 7f9df6d09700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9df006b350 0x7f9df019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.528+0000 7f9df6d09700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9df010ef30 0x7f9df019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.528+0000 7f9df6d09700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9df0114a20 con 0x7f9df006b350 2024-06-10T01:29:53.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.528+0000 7f9df6d09700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9df01148a0 con 0x7f9df006aa30 2024-06-10T01:29:53.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.528+0000 7f9df6d09700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9df0114ba0 con 0x7f9df010ef30 2024-06-10T01:29:53.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.528+0000 7f9df4aa5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9df006aa30 0x7f9df019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.528+0000 7f9df4aa5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9df006aa30 0x7f9df019ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:33334/0 (socket says 172.21.15.7:33334) 2024-06-10T01:29:53.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.528+0000 7f9df4aa5700 1 -- 172.21.15.7:0/3528462635 learned_addr learned my addr 172.21.15.7:0/3528462635 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:53.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.529+0000 7f9df52a6700 1 --2- 172.21.15.7:0/3528462635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9df010ef30 0x7f9df019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.529+0000 7f9deffff700 1 --2- 172.21.15.7:0/3528462635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9df006b350 0x7f9df019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.529+0000 7f9df4aa5700 1 -- 172.21.15.7:0/3528462635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9df010ef30 msgr2=0x7f9df019f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:53.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.529+0000 7f9df4aa5700 1 --2- 172.21.15.7:0/3528462635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9df010ef30 0x7f9df019f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.529+0000 7f9df4aa5700 1 -- 172.21.15.7:0/3528462635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9df006b350 msgr2=0x7f9df019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:53.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.529+0000 7f9df4aa5700 1 --2- 172.21.15.7:0/3528462635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9df006b350 0x7f9df019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.529+0000 7f9df4aa5700 1 -- 172.21.15.7:0/3528462635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df019fe80 con 0x7f9df006aa30 2024-06-10T01:29:53.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.529+0000 7f9deffff700 1 --2- 172.21.15.7:0/3528462635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9df006b350 0x7f9df019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:29:53.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.529+0000 7f9df4aa5700 1 --2- 172.21.15.7:0/3528462635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9df006aa30 0x7f9df019ae70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9de4005910 tx=0x7f9de400bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:53.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.529+0000 7f9dedffb700 1 -- 172.21.15.7:0/3528462635 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de4026070 con 0x7f9df006aa30 2024-06-10T01:29:53.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.529+0000 7f9df6d09700 1 -- 172.21.15.7:0/3528462635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9df01a0110 con 0x7f9df006aa30 2024-06-10T01:29:53.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.529+0000 7f9dedffb700 1 -- 172.21.15.7:0/3528462635 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9de4003d70 con 0x7f9df006aa30 2024-06-10T01:29:53.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.530+0000 7f9dedffb700 1 -- 172.21.15.7:0/3528462635 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de4021410 con 0x7f9df006aa30 2024-06-10T01:29:53.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.530+0000 7f9df6d09700 1 -- 172.21.15.7:0/3528462635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9df01a79f0 con 0x7f9df006aa30 2024-06-10T01:29:53.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.533+0000 7f9dedffb700 1 -- 172.21.15.7:0/3528462635 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9de4029070 con 0x7f9df006aa30 2024-06-10T01:29:53.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.534+0000 7f9dedffb700 1 --2- 172.21.15.7:0/3528462635 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9de00a2cf0 0x7f9de00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.534+0000 7f9dedffb700 1 -- 172.21.15.7:0/3528462635 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f9de40d0170 con 0x7f9df006aa30 2024-06-10T01:29:53.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.535+0000 7f9deffff700 1 --2- 172.21.15.7:0/3528462635 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9de00a2cf0 0x7f9de00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.535+0000 7f9deffff700 1 --2- 172.21.15.7:0/3528462635 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9de00a2cf0 0x7f9de00a51b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9df0102800 tx=0x7f9ddc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:53.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.536+0000 7f9e0158cb80 1 Processor -- start 2024-06-10T01:29:53.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.536+0000 7f9e0158cb80 1 -- start start 2024-06-10T01:29:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.536+0000 7f9e0158cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888409f640 0x55888409fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.536+0000 7f9e0158cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55888409ff60 0x5588840999e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.536+0000 7f9df4aa5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888409f640 0x55888409fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.537+0000 7f9df4aa5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888409f640 0x55888409fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:33344/0 (socket says 172.21.15.7:33344) 2024-06-10T01:29:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.537+0000 7f9df4aa5700 1 -- 172.21.15.7:0/3631611248 learned_addr learned my addr 172.21.15.7:0/3631611248 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.537+0000 7f9e0158cb80 1 --2- 172.21.15.7:0/3631611248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558884099f20 0x55888409a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.537+0000 7f9e0158cb80 1 -- 172.21.15.7:0/3631611248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558883feb810 con 0x55888409ff60 2024-06-10T01:29:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.537+0000 7f9e0158cb80 1 -- 172.21.15.7:0/3631611248 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558883f93720 con 0x55888409f640 2024-06-10T01:29:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.537+0000 7f9e0158cb80 1 -- 172.21.15.7:0/3631611248 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558883fe77d0 con 0x558884099f20 2024-06-10T01:29:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.537+0000 7f9deffff700 1 --2- 172.21.15.7:0/3631611248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55888409ff60 0x5588840999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.537+0000 7f9df4aa5700 1 -- 172.21.15.7:0/3631611248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558884099f20 msgr2=0x55888409a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.537+0000 7f9df4aa5700 1 --2- 172.21.15.7:0/3631611248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558884099f20 0x55888409a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.537+0000 7f9df4aa5700 1 -- 172.21.15.7:0/3631611248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55888409ff60 msgr2=0x5588840999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:53.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.537+0000 7f9df4aa5700 1 --2- 172.21.15.7:0/3631611248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55888409ff60 0x5588840999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.537+0000 7f9df4aa5700 1 -- 172.21.15.7:0/3631611248 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558883f89f50 con 0x55888409f640 2024-06-10T01:29:53.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.537+0000 7f9deffff700 1 --2- 172.21.15.7:0/3631611248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55888409ff60 0x5588840999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:29:53.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.537+0000 7f9df4aa5700 1 --2- 172.21.15.7:0/3631611248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888409f640 0x55888409fa20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9de409c610 tx=0x7f9de4009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4702906956a4c8c9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:53.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.538+0000 7f9ddaffd700 1 -- 172.21.15.7:0/3631611248 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de409bfe0 con 0x55888409f640 2024-06-10T01:29:53.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.538+0000 7f9ddaffd700 1 -- 172.21.15.7:0/3631611248 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9de40043d0 con 0x55888409f640 2024-06-10T01:29:53.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.538+0000 7f9ddaffd700 1 -- 172.21.15.7:0/3631611248 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de40a4dd0 con 0x55888409f640 2024-06-10T01:29:53.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.539+0000 7f9e0158cb80 1 -- 172.21.15.7:0/3631611248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888409f640 msgr2=0x55888409fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:53.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.539+0000 7f9e0158cb80 1 --2- 172.21.15.7:0/3631611248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888409f640 0x55888409fa20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9de409c610 tx=0x7f9de4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.539+0000 7f9e0158cb80 1 -- 172.21.15.7:0/3631611248 shutdown_connections 2024-06-10T01:29:53.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.539+0000 7f9e0158cb80 1 --2- 172.21.15.7:0/3631611248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558884099f20 0x55888409a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.539+0000 7f9e0158cb80 1 --2- 172.21.15.7:0/3631611248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888409f640 0x55888409fa20 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.539+0000 7f9e0158cb80 1 --2- 172.21.15.7:0/3631611248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55888409ff60 0x5588840999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.539+0000 7f9e0158cb80 1 -- 172.21.15.7:0/3631611248 >> 172.21.15.7:0/3631611248 conn(0x558883fc4cc0 msgr2=0x558884094850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:53.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.539+0000 7f9e0158cb80 1 -- 172.21.15.7:0/3631611248 shutdown_connections 2024-06-10T01:29:53.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.539+0000 7f9e0158cb80 1 -- 172.21.15.7:0/3631611248 wait complete. 2024-06-10T01:29:53.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.540+0000 7f9e0158cb80 1 -- start start 2024-06-10T01:29:53.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.540+0000 7f9e0158cb80 1 Processor -- start 2024-06-10T01:29:53.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.541+0000 7f9e0158cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558884099f20 0x5588840a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.541+0000 7f9e0158cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55888409f640 0x5588840a3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.541+0000 7f9df4aa5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558884099f20 0x5588840a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.541+0000 7f9df4aa5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558884099f20 0x5588840a5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:60148/0 (socket says 172.21.15.7:60148) 2024-06-10T01:29:53.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.541+0000 7f9df4aa5700 1 -- 172.21.15.7:0/91185513 learned_addr learned my addr 172.21.15.7:0/91185513 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:53.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.541+0000 7f9e0158cb80 1 --2- 172.21.15.7:0/91185513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888409ff60 0x5588840a35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.542+0000 7f9e0158cb80 1 -- 172.21.15.7:0/91185513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5588840a66f0 con 0x55888409f640 2024-06-10T01:29:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.542+0000 7f9deffff700 1 --2- 172.21.15.7:0/91185513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55888409f640 0x5588840a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.542+0000 7f9df52a6700 1 --2- 172.21.15.7:0/91185513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888409ff60 0x5588840a35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.542+0000 7f9e0158cb80 1 -- 172.21.15.7:0/91185513 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558883f8bbf0 con 0x55888409ff60 2024-06-10T01:29:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.542+0000 7f9e0158cb80 1 -- 172.21.15.7:0/91185513 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5588840a6870 con 0x558884099f20 2024-06-10T01:29:53.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.542+0000 7f9df4aa5700 1 -- 172.21.15.7:0/91185513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888409ff60 msgr2=0x5588840a35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.542+0000 7f9df4aa5700 1 --2- 172.21.15.7:0/91185513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888409ff60 0x5588840a35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.542+0000 7f9df4aa5700 1 -- 172.21.15.7:0/91185513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55888409f640 msgr2=0x5588840a3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.542+0000 7f9df4aa5700 1 --2- 172.21.15.7:0/91185513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55888409f640 0x5588840a3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.542+0000 7f9df4aa5700 1 -- 172.21.15.7:0/91185513 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558883f3a8d0 con 0x558884099f20 2024-06-10T01:29:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.542+0000 7f9e0158cb80 10 client.0 _ll_register_callbacks cb 0x558883f85920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:29:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.542+0000 7f9df4aa5700 1 --2- 172.21.15.7:0/91185513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558884099f20 0x5588840a5d60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9de40d00e0 tx=0x7f9de409c580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f0a4ea91f3d961e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.542+0000 7f9dcffff700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:29:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.542+0000 7f9e0158cb80 1 -- 172.21.15.7:0/91185513 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9de40043d0 con 0x558884099f20 2024-06-10T01:29:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.543+0000 7f9dcffff700 1 -- 172.21.15.7:0/91185513 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de40ba060 con 0x558884099f20 2024-06-10T01:29:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.543+0000 7f9dcffff700 1 -- 172.21.15.7:0/91185513 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f9de40a1dc0 con 0x558884099f20 2024-06-10T01:29:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.543+0000 7f9dcffff700 1 -- 172.21.15.7:0/91185513 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f9de409bfe0 con 0x558884099f20 2024-06-10T01:29:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.543+0000 7f9e0158cb80 10 client.30151 fetch_fsmap learned FSMap version 55 2024-06-10T01:29:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.543+0000 7f9e0158cb80 1 -- 172.21.15.7:0/91185513 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({fsmap=55}) -- 0x5588840a66f0 con 0x558884099f20 2024-06-10T01:29:53.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.543+0000 7f9dcffff700 1 -- 172.21.15.7:0/91185513 <== mon.1 v2:172.21.15.175:3300/0 4 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f9de409bad0 con 0x558884099f20 2024-06-10T01:29:53.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.543+0000 7f9e0158cb80 10 client.30151 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:29:53.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.543+0000 7f9e0158cb80 10 client.30151 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:29:53.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.543+0000 7f9e0158cb80 20 client.30151 populate_metadata read hostname 'smithi007' 2024-06-10T01:29:53.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.543+0000 7f9e0158cb80 1 --2- 172.21.15.7:0/91185513 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5588840b3100 0x5588840b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.543+0000 7f9e0158cb80 4 client.30151 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:29:53.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.543+0000 7f9e0158cb80 1 -- 172.21.15.7:0/91185513 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9de40043d0 con 0x5588840b3100 2024-06-10T01:29:53.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.544+0000 7f9deffff700 1 --2- 172.21.15.7:0/91185513 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5588840b3100 0x5588840b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.544+0000 7f9deffff700 1 --2- 172.21.15.7:0/91185513 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5588840b3100 0x5588840b54f0 crc :-1 s=READY pgs=533 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f4e4d471e305c1f server_cookie=88c084668b6910f2 in_seq=0 out_seq=0 2024-06-10T01:29:53.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.544+0000 7f9dcffff700 10 client.30151 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:53.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.547+0000 7f9dcffff700 1 -- 172.21.15.7:0/91185513 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f9de40043d0 con 0x5588840b3100 2024-06-10T01:29:53.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.547+0000 7f9dcffff700 10 client.30151 handle_command_reply: tid=0 2024-06-10T01:29:53.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.547+0000 7f9e0158cb80 1 client.30151 shutdown 2024-06-10T01:29:53.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.547+0000 7f9e0158cb80 1 -- 172.21.15.7:0/91185513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558884099f20 msgr2=0x5588840a5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:53.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.547+0000 7f9e0158cb80 1 --2- 172.21.15.7:0/91185513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558884099f20 0x5588840a5d60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9de40d00e0 tx=0x7f9de409c580 comp rx=0 tx=0).stop 2024-06-10T01:29:53.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.547+0000 7f9e0158cb80 1 -- 172.21.15.7:0/91185513 shutdown_connections 2024-06-10T01:29:53.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.547+0000 7f9e0158cb80 1 --2- 172.21.15.7:0/91185513 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5588840b3100 0x5588840b54f0 crc :-1 s=READY pgs=533 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.547+0000 7f9e0158cb80 1 --2- 172.21.15.7:0/91185513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888409ff60 0x5588840a35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.547+0000 7f9dcffff700 0 client.30151 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:53.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.547+0000 7f9e0158cb80 1 --2- 172.21.15.7:0/91185513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558884099f20 0x5588840a5d60 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.547+0000 7f9e0158cb80 1 --2- 172.21.15.7:0/91185513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55888409f640 0x5588840a3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.547+0000 7f9e0158cb80 1 -- 172.21.15.7:0/91185513 >> 172.21.15.7:0/91185513 conn(0x558883fc4cc0 msgr2=0x5588840989d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:53.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.547+0000 7f9e0158cb80 1 -- 172.21.15.7:0/91185513 shutdown_connections 2024-06-10T01:29:53.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.547+0000 7f9e0158cb80 1 -- 172.21.15.7:0/91185513 wait complete. 2024-06-10T01:29:53.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.547+0000 7f9e0158cb80 20 client.30151 trim_cache size 0 max 16384 2024-06-10T01:29:53.565 INFO:tasks.workunit.client.0.smithi007.stdout: 10 248129 21.92 MB/sec execute 510 sec latency 20200.423 ms 2024-06-10T01:29:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.581+0000 7f9e0158cb80 1 Processor -- start 2024-06-10T01:29:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.581+0000 7f9e0158cb80 1 -- start start 2024-06-10T01:29:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.581+0000 7f9e0158cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558884099f20 0x5588840999e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.581+0000 7f9e0158cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55888409f640 0x5588840af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.581+0000 7f9e0158cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888409ff60 0x5588840a2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.581+0000 7f9e0158cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9df01b2ca0 con 0x55888409f640 2024-06-10T01:29:53.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.581+0000 7f9e0158cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9df01b0530 con 0x55888409ff60 2024-06-10T01:29:53.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.581+0000 7f9e0158cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9df01ae380 con 0x558884099f20 2024-06-10T01:29:53.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.581+0000 7f9df4aa5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558884099f20 0x5588840999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.581+0000 7f9df52a6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888409ff60 0x5588840a2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.581+0000 7f9df4aa5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558884099f20 0x5588840999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:60150/0 (socket says 172.21.15.7:60150) 2024-06-10T01:29:53.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.581+0000 7f9deffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55888409f640 0x5588840af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.581+0000 7f9df4aa5700 1 -- 172.21.15.7:0/899095031 learned_addr learned my addr 172.21.15.7:0/899095031 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:53.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.582+0000 7f9df52a6700 1 -- 172.21.15.7:0/899095031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558884099f20 msgr2=0x5588840999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:53.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.582+0000 7f9df52a6700 1 --2- 172.21.15.7:0/899095031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558884099f20 0x5588840999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.582+0000 7f9df52a6700 1 -- 172.21.15.7:0/899095031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55888409f640 msgr2=0x5588840af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:53.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.582+0000 7f9df52a6700 1 --2- 172.21.15.7:0/899095031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55888409f640 0x5588840af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.582+0000 7f9df52a6700 1 -- 172.21.15.7:0/899095031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558884095520 con 0x55888409ff60 2024-06-10T01:29:53.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.582+0000 7f9df52a6700 1 --2- 172.21.15.7:0/899095031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888409ff60 0x5588840a2100 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9de8014660 tx=0x7f9de8014970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db0dff9dfbdd2edb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:53.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.582+0000 7f9dda7fc700 1 -- 172.21.15.7:0/899095031 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de8003a00 con 0x55888409ff60 2024-06-10T01:29:53.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.582+0000 7f9dda7fc700 1 -- 172.21.15.7:0/899095031 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9de8003ba0 con 0x55888409ff60 2024-06-10T01:29:53.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.582+0000 7f9dda7fc700 1 -- 172.21.15.7:0/899095031 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de8007820 con 0x55888409ff60 2024-06-10T01:29:53.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.583+0000 7f9e0158cb80 1 -- 172.21.15.7:0/899095031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888409ff60 msgr2=0x5588840a2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:53.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.583+0000 7f9e0158cb80 1 --2- 172.21.15.7:0/899095031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888409ff60 0x5588840a2100 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9de8014660 tx=0x7f9de8014970 comp rx=0 tx=0).stop 2024-06-10T01:29:53.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.583+0000 7f9e0158cb80 1 -- 172.21.15.7:0/899095031 shutdown_connections 2024-06-10T01:29:53.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.583+0000 7f9e0158cb80 1 --2- 172.21.15.7:0/899095031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888409ff60 0x5588840a2100 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.583+0000 7f9e0158cb80 1 --2- 172.21.15.7:0/899095031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558884099f20 0x5588840999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.583+0000 7f9e0158cb80 1 --2- 172.21.15.7:0/899095031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55888409f640 0x5588840af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.583+0000 7f9e0158cb80 1 -- 172.21.15.7:0/899095031 >> 172.21.15.7:0/899095031 conn(0x558883fc4cc0 msgr2=0x5588840a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:53.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.584+0000 7f9e0158cb80 1 -- 172.21.15.7:0/899095031 shutdown_connections 2024-06-10T01:29:53.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.584+0000 7f9e0158cb80 1 -- 172.21.15.7:0/899095031 wait complete. 2024-06-10T01:29:53.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.584+0000 7f9e0158cb80 1 -- start start 2024-06-10T01:29:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.584+0000 7f9e0158cb80 1 Processor -- start 2024-06-10T01:29:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.585+0000 7f9e0158cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558884099f20 0x5588840a80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.585+0000 7f9e0158cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55888409f640 0x5588840a85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.585+0000 7f9e0158cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888409ff60 0x55888411ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.585+0000 7f9e0158cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9df01ae9f0 con 0x55888409f640 2024-06-10T01:29:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.585+0000 7f9e0158cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9df01afe60 con 0x55888409ff60 2024-06-10T01:29:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.585+0000 7f9e0158cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9df01af0c0 con 0x558884099f20 2024-06-10T01:29:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.585+0000 7f9df52a6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888409ff60 0x55888411ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.585+0000 7f9df52a6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888409ff60 0x55888411ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:33378/0 (socket says 172.21.15.7:33378) 2024-06-10T01:29:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.585+0000 7f9df52a6700 1 -- 172.21.15.7:0/2777708111 learned_addr learned my addr 172.21.15.7:0/2777708111 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.585+0000 7f9deffff700 1 --2- 172.21.15.7:0/2777708111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55888409f640 0x5588840a85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.586+0000 7f9e0158cb80 10 client.0 _ll_register_callbacks cb 0x558883f85920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:29:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.586+0000 7f9df52a6700 1 -- 172.21.15.7:0/2777708111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558884099f20 msgr2=0x5588840a80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.586+0000 7f9df52a6700 1 --2- 172.21.15.7:0/2777708111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558884099f20 0x5588840a80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.586+0000 7f9df52a6700 1 -- 172.21.15.7:0/2777708111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55888409f640 msgr2=0x5588840a85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:29:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.586+0000 7f9df52a6700 1 --2- 172.21.15.7:0/2777708111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55888409f640 0x5588840a85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.586+0000 7f9df52a6700 1 -- 172.21.15.7:0/2777708111 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55888411d2b0 con 0x55888409ff60 2024-06-10T01:29:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.586+0000 7f9df52a6700 1 --2- 172.21.15.7:0/2777708111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888409ff60 0x55888411ccf0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f9de8009790 tx=0x7f9de8003a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d069504fa28d4bbf server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.586+0000 7f9dcffff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:29:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.586+0000 7f9dcffff700 1 -- 172.21.15.7:0/2777708111 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de802b060 con 0x55888409ff60 2024-06-10T01:29:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.586+0000 7f9dcffff700 1 -- 172.21.15.7:0/2777708111 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f9de8007700 con 0x55888409ff60 2024-06-10T01:29:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.586+0000 7f9e0158cb80 1 -- 172.21.15.7:0/2777708111 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9de8003ba0 con 0x55888409ff60 2024-06-10T01:29:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.587+0000 7f9dcffff700 1 -- 172.21.15.7:0/2777708111 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f9de8032050 con 0x55888409ff60 2024-06-10T01:29:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.587+0000 7f9e0158cb80 10 client.30254 fetch_fsmap learned FSMap version 55 2024-06-10T01:29:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.587+0000 7f9e0158cb80 1 -- 172.21.15.7:0/2777708111 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55888411e560 con 0x55888409ff60 2024-06-10T01:29:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.587+0000 7f9dcffff700 1 -- 172.21.15.7:0/2777708111 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9de8007e20 con 0x55888409ff60 2024-06-10T01:29:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.587+0000 7f9dcffff700 1 -- 172.21.15.7:0/2777708111 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f9de8025030 con 0x55888409ff60 2024-06-10T01:29:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.587+0000 7f9e0158cb80 10 client.30254 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:29:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.587+0000 7f9e0158cb80 10 client.30254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:29:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.587+0000 7f9e0158cb80 20 client.30254 populate_metadata read hostname 'smithi007' 2024-06-10T01:29:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.587+0000 7f9e0158cb80 1 --2- 172.21.15.7:0/2777708111 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5588840b3100 0x558884120b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.587+0000 7f9e0158cb80 4 client.30254 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:29:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.587+0000 7f9e0158cb80 1 -- 172.21.15.7:0/2777708111 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f9de8003ba0 con 0x5588840b3100 2024-06-10T01:29:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.587+0000 7f9df4aa5700 1 --2- 172.21.15.7:0/2777708111 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5588840b3100 0x558884120b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.588+0000 7f9dcffff700 10 client.30254 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:53.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.588+0000 7f9df4aa5700 1 --2- 172.21.15.7:0/2777708111 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5588840b3100 0x558884120b20 crc :-1 s=READY pgs=534 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc6945eb95396ccc server_cookie=599067eac8ecad66 in_seq=0 out_seq=0 2024-06-10T01:29:53.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.588+0000 7f9dcffff700 1 -- 172.21.15.7:0/2777708111 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f9de8003ba0 con 0x5588840b3100 2024-06-10T01:29:53.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.588+0000 7f9dcffff700 10 client.30254 handle_command_reply: tid=0 2024-06-10T01:29:53.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.588+0000 7f9e0158cb80 1 client.30254 shutdown 2024-06-10T01:29:53.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.589+0000 7f9e0158cb80 1 -- 172.21.15.7:0/2777708111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888409ff60 msgr2=0x55888411ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:53.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.589+0000 7f9e0158cb80 1 --2- 172.21.15.7:0/2777708111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888409ff60 0x55888411ccf0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f9de8009790 tx=0x7f9de8003a00 comp rx=0 tx=0).stop 2024-06-10T01:29:53.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.589+0000 7f9e0158cb80 1 -- 172.21.15.7:0/2777708111 shutdown_connections 2024-06-10T01:29:53.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.589+0000 7f9e0158cb80 1 --2- 172.21.15.7:0/2777708111 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5588840b3100 0x558884120b20 crc :-1 s=READY pgs=534 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.589+0000 7f9dcffff700 0 client.30254 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:29:53.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.589+0000 7f9e0158cb80 1 --2- 172.21.15.7:0/2777708111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888409ff60 0x55888411ccf0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.589+0000 7f9e0158cb80 1 --2- 172.21.15.7:0/2777708111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558884099f20 0x5588840a80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.589+0000 7f9e0158cb80 1 --2- 172.21.15.7:0/2777708111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55888409f640 0x5588840a85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.589+0000 7f9e0158cb80 1 -- 172.21.15.7:0/2777708111 >> 172.21.15.7:0/2777708111 conn(0x558883fc4cc0 msgr2=0x55888409ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:53.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.589+0000 7f9e0158cb80 1 -- 172.21.15.7:0/2777708111 shutdown_connections 2024-06-10T01:29:53.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.589+0000 7f9e0158cb80 1 -- 172.21.15.7:0/2777708111 wait complete. 2024-06-10T01:29:53.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.589+0000 7f9e0158cb80 20 client.30254 trim_cache size 0 max 16384 2024-06-10T01:29:53.587 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:29:53.588 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-10T01:29:53.588 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-10T01:29:53.588 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:29:53.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.590+0000 7f9df6d09700 1 -- 172.21.15.7:0/3528462635 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9de00a2cf0 msgr2=0x7f9de00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:53.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.590+0000 7f9df6d09700 1 --2- 172.21.15.7:0/3528462635 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9de00a2cf0 0x7f9de00a51b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9df0102800 tx=0x7f9ddc009260 comp rx=0 tx=0).stop 2024-06-10T01:29:53.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.590+0000 7f9df6d09700 1 -- 172.21.15.7:0/3528462635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9df006aa30 msgr2=0x7f9df019ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:53.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.590+0000 7f9df6d09700 1 --2- 172.21.15.7:0/3528462635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9df006aa30 0x7f9df019ae70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9de4005910 tx=0x7f9de400bab0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.590+0000 7f9df6d09700 1 -- 172.21.15.7:0/3528462635 shutdown_connections 2024-06-10T01:29:53.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.590+0000 7f9df6d09700 1 --2- 172.21.15.7:0/3528462635 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9de00a2cf0 0x7f9de00a51b0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.590+0000 7f9df6d09700 1 --2- 172.21.15.7:0/3528462635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9df010ef30 0x7f9df019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.590+0000 7f9df6d09700 1 --2- 172.21.15.7:0/3528462635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9df006aa30 0x7f9df019ae70 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.590+0000 7f9df6d09700 1 --2- 172.21.15.7:0/3528462635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9df006b350 0x7f9df019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:53.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.591+0000 7f9df6d09700 1 -- 172.21.15.7:0/3528462635 >> 172.21.15.7:0/3528462635 conn(0x7f9df0078300 msgr2=0x7f9df0110340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:53.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.591+0000 7f9df6d09700 1 -- 172.21.15.7:0/3528462635 shutdown_connections 2024-06-10T01:29:53.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:53.591+0000 7f9df6d09700 1 -- 172.21.15.7:0/3528462635 wait complete. 2024-06-10T01:29:53.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:53 smithi007 ceph-mon[32398]: pgmap v1150: 129 pgs: 129 active+clean; 658 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-06-10T01:29:53.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:53 smithi078 ceph-mon[36683]: pgmap v1150: 129 pgs: 129 active+clean; 658 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-06-10T01:29:53.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:53 smithi175 ceph-mon[36636]: pgmap v1150: 129 pgs: 129 active+clean; 658 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-06-10T01:29:54.565 INFO:tasks.workunit.client.0.smithi007.stdout: 10 248898 21.98 MB/sec execute 511 sec latency 86.106 ms 2024-06-10T01:29:54.640 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:29:54.641 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:29:55.294 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6e35bda3', total seconds elapsed: 31.3 2024-06-10T01:29:55.294 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.45 2024-06-10T01:29:55.294 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:29:55.410 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:29:55.565 INFO:tasks.workunit.client.0.smithi007.stdout: 10 249577 22.00 MB/sec execute 512 sec latency 157.590 ms 2024-06-10T01:29:55.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:55 smithi078 ceph-mon[36683]: pgmap v1151: 129 pgs: 129 active+clean; 658 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-06-10T01:29:55.885 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:55 smithi007 ceph-mon[32398]: pgmap v1151: 129 pgs: 129 active+clean; 658 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-06-10T01:29:55.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:55 smithi175 ceph-mon[36636]: pgmap v1151: 129 pgs: 129 active+clean; 658 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-06-10T01:29:56.565 INFO:tasks.workunit.client.0.smithi007.stdout: 10 250111 22.03 MB/sec execute 513 sec latency 79.550 ms 2024-06-10T01:29:56.814 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:29:57.565 INFO:tasks.workunit.client.0.smithi007.stdout: 10 250634 22.02 MB/sec execute 514 sec latency 101.582 ms 2024-06-10T01:29:57.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:57 smithi007 ceph-mon[32398]: pgmap v1152: 129 pgs: 129 active+clean; 656 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 207 KiB/s wr, 44 op/s 2024-06-10T01:29:57.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:57 smithi078 ceph-mon[36683]: pgmap v1152: 129 pgs: 129 active+clean; 656 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 207 KiB/s wr, 44 op/s 2024-06-10T01:29:57.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:57 smithi175 ceph-mon[36636]: pgmap v1152: 129 pgs: 129 active+clean; 656 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 207 KiB/s wr, 44 op/s 2024-06-10T01:29:58.565 INFO:tasks.workunit.client.0.smithi007.stdout: 10 251120 22.03 MB/sec execute 515 sec latency 88.753 ms 2024-06-10T01:29:58.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.636+0000 7fde563b7700 1 -- 172.21.15.7:0/3264460931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde50101d80 msgr2=0x7fde5010e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:58.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.636+0000 7fde563b7700 1 --2- 172.21.15.7:0/3264460931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde50101d80 0x7fde5010e270 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fde4800b0a0 tx=0x7fde4800b3b0 comp rx=0 tx=0).stop 2024-06-10T01:29:58.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.636+0000 7fde563b7700 1 -- 172.21.15.7:0/3264460931 shutdown_connections 2024-06-10T01:29:58.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.636+0000 7fde563b7700 1 --2- 172.21.15.7:0/3264460931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde50101d80 0x7fde5010e270 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:58.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.636+0000 7fde563b7700 1 --2- 172.21.15.7:0/3264460931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde50107430 0x7fde50107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:58.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.636+0000 7fde563b7700 1 --2- 172.21.15.7:0/3264460931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde501013c0 0x7fde50101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:58.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.636+0000 7fde563b7700 1 -- 172.21.15.7:0/3264460931 >> 172.21.15.7:0/3264460931 conn(0x7fde500783d0 msgr2=0x7fde500787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:58.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.636+0000 7fde563b7700 1 -- 172.21.15.7:0/3264460931 shutdown_connections 2024-06-10T01:29:58.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.636+0000 7fde563b7700 1 -- 172.21.15.7:0/3264460931 wait complete. 2024-06-10T01:29:58.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.642+0000 7fde563b7700 1 Processor -- start 2024-06-10T01:29:58.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.642+0000 7fde563b7700 1 -- start start 2024-06-10T01:29:58.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.643+0000 7fde563b7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde501013c0 0x7fde5019d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:58.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.643+0000 7fde563b7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde50101d80 0x7fde5019dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:58.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.643+0000 7fde563b7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde50107430 0x7fde50197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:58.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.643+0000 7fde47fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde50101d80 0x7fde5019dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:58.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.643+0000 7fde47fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde50101d80 0x7fde5019dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:33402/0 (socket says 172.21.15.7:33402) 2024-06-10T01:29:58.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.643+0000 7fde47fff700 1 -- 172.21.15.7:0/2683698703 learned_addr learned my addr 172.21.15.7:0/2683698703 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:58.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.644+0000 7fde563b7700 1 -- 172.21.15.7:0/2683698703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fde501116c0 con 0x7fde50107430 2024-06-10T01:29:58.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.644+0000 7fde563b7700 1 -- 172.21.15.7:0/2683698703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fde50111540 con 0x7fde50101d80 2024-06-10T01:29:58.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.644+0000 7fde563b7700 1 -- 172.21.15.7:0/2683698703 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fde50111840 con 0x7fde501013c0 2024-06-10T01:29:58.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.644+0000 7fde54954700 1 --2- 172.21.15.7:0/2683698703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde50107430 0x7fde50197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:58.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.644+0000 7fde47fff700 1 -- 172.21.15.7:0/2683698703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde501013c0 msgr2=0x7fde5019d760 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-10T01:29:58.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.644+0000 7fde47fff700 1 --2- 172.21.15.7:0/2683698703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde501013c0 0x7fde5019d760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:58.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.644+0000 7fde47fff700 1 -- 172.21.15.7:0/2683698703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde50107430 msgr2=0x7fde50197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:58.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.644+0000 7fde47fff700 1 --2- 172.21.15.7:0/2683698703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde50107430 0x7fde50197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:58.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.644+0000 7fde47fff700 1 -- 172.21.15.7:0/2683698703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde50198080 con 0x7fde50101d80 2024-06-10T01:29:58.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.645+0000 7fde47fff700 1 --2- 172.21.15.7:0/2683698703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde50101d80 0x7fde5019dca0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fde38009cb0 tx=0x7fde3800b2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:58.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.645+0000 7fde4dffb700 1 -- 172.21.15.7:0/2683698703 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3801e070 con 0x7fde50101d80 2024-06-10T01:29:58.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.645+0000 7fde4dffb700 1 -- 172.21.15.7:0/2683698703 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde3800bc00 con 0x7fde50101d80 2024-06-10T01:29:58.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.645+0000 7fde4dffb700 1 -- 172.21.15.7:0/2683698703 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde380186a0 con 0x7fde50101d80 2024-06-10T01:29:58.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.645+0000 7fde563b7700 1 -- 172.21.15.7:0/2683698703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde50198370 con 0x7fde50101d80 2024-06-10T01:29:58.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.646+0000 7fde563b7700 1 -- 172.21.15.7:0/2683698703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde5006ab60 con 0x7fde50101d80 2024-06-10T01:29:58.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.650+0000 7fde4dffb700 1 -- 172.21.15.7:0/2683698703 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fde38021070 con 0x7fde50101d80 2024-06-10T01:29:58.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.650+0000 7fde46ffd700 1 -- 172.21.15.7:0/2683698703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde50102ae0 con 0x7fde50101d80 2024-06-10T01:29:58.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.651+0000 7fde4dffb700 1 --2- 172.21.15.7:0/2683698703 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fde300a2df0 0x7fde300a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:58.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.651+0000 7fde4dffb700 1 -- 172.21.15.7:0/2683698703 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fde380d2a00 con 0x7fde50101d80 2024-06-10T01:29:58.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.652+0000 7fde4ffff700 1 --2- 172.21.15.7:0/2683698703 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fde300a2df0 0x7fde300a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:58.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.653+0000 7fde4ffff700 1 --2- 172.21.15.7:0/2683698703 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fde300a2df0 0x7fde300a52b0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fde50198a60 tx=0x7fde40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:58.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:58.659+0000 7fde4dffb700 1 -- 172.21.15.7:0/2683698703 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fde3809f580 con 0x7fde50101d80 2024-06-10T01:29:59.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.099+0000 7fde46ffd700 1 -- 172.21.15.7:0/2683698703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde50198e50 con 0x7fde50101d80 2024-06-10T01:29:59.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.100+0000 7fde4dffb700 1 -- 172.21.15.7:0/2683698703 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11852 (secure 0 0 0) 0x7fde3809ee80 con 0x7fde50101d80 2024-06-10T01:29:59.098 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:29:59.098 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:27:44.542390+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29813":{"gid":29813,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/138896429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":138896429},{"type":"v1","addr":"172.21.15.175:6835","nonce":138896429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24451,24434,24436]},"id":1}]} 2024-06-10T01:29:59.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.110+0000 7fde46ffd700 1 -- 172.21.15.7:0/2683698703 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fde300a2df0 msgr2=0x7fde300a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:59.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.110+0000 7fde46ffd700 1 --2- 172.21.15.7:0/2683698703 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fde300a2df0 0x7fde300a52b0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fde50198a60 tx=0x7fde40009260 comp rx=0 tx=0).stop 2024-06-10T01:29:59.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.110+0000 7fde46ffd700 1 -- 172.21.15.7:0/2683698703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde50101d80 msgr2=0x7fde5019dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:59.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.110+0000 7fde46ffd700 1 --2- 172.21.15.7:0/2683698703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde50101d80 0x7fde5019dca0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fde38009cb0 tx=0x7fde3800b2c0 comp rx=0 tx=0).stop 2024-06-10T01:29:59.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.111+0000 7fde46ffd700 1 -- 172.21.15.7:0/2683698703 shutdown_connections 2024-06-10T01:29:59.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.111+0000 7fde46ffd700 1 --2- 172.21.15.7:0/2683698703 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fde300a2df0 0x7fde300a52b0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:59.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.111+0000 7fde46ffd700 1 --2- 172.21.15.7:0/2683698703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde50107430 0x7fde50197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:59.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.111+0000 7fde46ffd700 1 --2- 172.21.15.7:0/2683698703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde501013c0 0x7fde5019d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:59.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.111+0000 7fde46ffd700 1 --2- 172.21.15.7:0/2683698703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde50101d80 0x7fde5019dca0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:59.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.111+0000 7fde46ffd700 1 -- 172.21.15.7:0/2683698703 >> 172.21.15.7:0/2683698703 conn(0x7fde500783d0 msgr2=0x7fde50100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:59.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.113+0000 7fde46ffd700 1 -- 172.21.15.7:0/2683698703 shutdown_connections 2024-06-10T01:29:59.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.114+0000 7fde46ffd700 1 -- 172.21.15.7:0/2683698703 wait complete. 2024-06-10T01:29:59.118 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 55 2024-06-10T01:29:59.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.177+0000 7fd94bf83700 1 -- 172.21.15.7:0/1698441462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd944109480 msgr2=0x7fd944115990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:59.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.177+0000 7fd94bf83700 1 --2- 172.21.15.7:0/1698441462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd944109480 0x7fd944115990 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd9340098f0 tx=0x7fd934009c00 comp rx=0 tx=0).stop 2024-06-10T01:29:59.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.177+0000 7fd94bf83700 1 -- 172.21.15.7:0/1698441462 shutdown_connections 2024-06-10T01:29:59.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.177+0000 7fd94bf83700 1 --2- 172.21.15.7:0/1698441462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd944109480 0x7fd944115990 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:59.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.177+0000 7fd94bf83700 1 --2- 172.21.15.7:0/1698441462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd94410eac0 0x7fd94410eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:59.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.177+0000 7fd94bf83700 1 --2- 172.21.15.7:0/1698441462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd944108ac0 0x7fd944108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:59.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.177+0000 7fd94bf83700 1 -- 172.21.15.7:0/1698441462 >> 172.21.15.7:0/1698441462 conn(0x7fd944075c60 msgr2=0x7fd9441072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:59.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.177+0000 7fd94bf83700 1 -- 172.21.15.7:0/1698441462 shutdown_connections 2024-06-10T01:29:59.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.177+0000 7fd94bf83700 1 -- 172.21.15.7:0/1698441462 wait complete. 2024-06-10T01:29:59.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.178+0000 7fd94bf83700 1 Processor -- start 2024-06-10T01:29:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.178+0000 7fd94bf83700 1 -- start start 2024-06-10T01:29:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.179+0000 7fd94bf83700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd944108ac0 0x7fd94419f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.179+0000 7fd94bf83700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd944109480 0x7fd94419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.179+0000 7fd94bf83700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd94410eac0 0x7fd9441a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.179+0000 7fd94bf83700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd944118ea0 con 0x7fd944108ac0 2024-06-10T01:29:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.179+0000 7fd94bf83700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd944118d20 con 0x7fd94410eac0 2024-06-10T01:29:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.179+0000 7fd94bf83700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd944119020 con 0x7fd944109480 2024-06-10T01:29:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.179+0000 7fd94a520700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd94410eac0 0x7fd9441a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.179+0000 7fd94a520700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd94410eac0 0x7fd9441a3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:33434/0 (socket says 172.21.15.7:33434) 2024-06-10T01:29:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.179+0000 7fd94a520700 1 -- 172.21.15.7:0/3843638081 learned_addr learned my addr 172.21.15.7:0/3843638081 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:29:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.179+0000 7fd94951e700 1 --2- 172.21.15.7:0/3843638081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd944109480 0x7fd94419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.180+0000 7fd949d1f700 1 --2- 172.21.15.7:0/3843638081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd944108ac0 0x7fd94419f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.180+0000 7fd94a520700 1 -- 172.21.15.7:0/3843638081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd944109480 msgr2=0x7fd94419f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.180+0000 7fd94a520700 1 --2- 172.21.15.7:0/3843638081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd944109480 0x7fd94419f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:59.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.180+0000 7fd94a520700 1 -- 172.21.15.7:0/3843638081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd944108ac0 msgr2=0x7fd94419f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:59.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.180+0000 7fd94a520700 1 --2- 172.21.15.7:0/3843638081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd944108ac0 0x7fd94419f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:59.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.180+0000 7fd94a520700 1 -- 172.21.15.7:0/3843638081 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9441a42a0 con 0x7fd94410eac0 2024-06-10T01:29:59.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.180+0000 7fd94a520700 1 --2- 172.21.15.7:0/3843638081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd94410eac0 0x7fd9441a3b60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd93400b870 tx=0x7fd93400b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:59.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.180+0000 7fd93affd700 1 -- 172.21.15.7:0/3843638081 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd934005320 con 0x7fd94410eac0 2024-06-10T01:29:59.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.180+0000 7fd93affd700 1 -- 172.21.15.7:0/3843638081 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9340054c0 con 0x7fd94410eac0 2024-06-10T01:29:59.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.180+0000 7fd93affd700 1 -- 172.21.15.7:0/3843638081 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd934017660 con 0x7fd94410eac0 2024-06-10T01:29:59.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.181+0000 7fd94bf83700 1 -- 172.21.15.7:0/3843638081 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9441a4530 con 0x7fd94410eac0 2024-06-10T01:29:59.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.181+0000 7fd94bf83700 1 -- 172.21.15.7:0/3843638081 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9441abe10 con 0x7fd94410eac0 2024-06-10T01:29:59.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.183+0000 7fd94bf83700 1 -- 172.21.15.7:0/3843638081 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd944069180 con 0x7fd94410eac0 2024-06-10T01:29:59.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.185+0000 7fd93affd700 1 -- 172.21.15.7:0/3843638081 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fd9340037a0 con 0x7fd94410eac0 2024-06-10T01:29:59.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.186+0000 7fd93affd700 1 --2- 172.21.15.7:0/3843638081 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd9300a2e20 0x7fd9300a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:29:59.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.187+0000 7fd93affd700 1 -- 172.21.15.7:0/3843638081 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fd9340cfbb0 con 0x7fd94410eac0 2024-06-10T01:29:59.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.187+0000 7fd949d1f700 1 --2- 172.21.15.7:0/3843638081 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd9300a2e20 0x7fd9300a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:29:59.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.188+0000 7fd949d1f700 1 --2- 172.21.15.7:0/3843638081 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd9300a2e20 0x7fd9300a52e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd94000dad0 tx=0x7fd94000d4b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:29:59.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.192+0000 7fd93affd700 1 -- 172.21.15.7:0/3843638081 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd934026020 con 0x7fd94410eac0 2024-06-10T01:29:59.565 INFO:tasks.workunit.client.0.smithi007.stdout: 10 251694 22.05 MB/sec execute 516 sec latency 147.676 ms 2024-06-10T01:29:59.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.629+0000 7fd94bf83700 1 -- 172.21.15.7:0/3843638081 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd944111070 con 0x7fd94410eac0 2024-06-10T01:29:59.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.636+0000 7fd93affd700 1 -- 172.21.15.7:0/3843638081 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11852 (secure 0 0 0) 0x7fd934013030 con 0x7fd94410eac0 2024-06-10T01:29:59.628 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:29:59.628 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:27:44.542390+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29813":{"gid":29813,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/138896429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":138896429},{"type":"v1","addr":"172.21.15.175:6835","nonce":138896429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24451,24434,24436]},"id":1}]} 2024-06-10T01:29:59.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.641+0000 7fd94bf83700 1 -- 172.21.15.7:0/3843638081 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd9300a2e20 msgr2=0x7fd9300a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:59.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.641+0000 7fd94bf83700 1 --2- 172.21.15.7:0/3843638081 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd9300a2e20 0x7fd9300a52e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd94000dad0 tx=0x7fd94000d4b0 comp rx=0 tx=0).stop 2024-06-10T01:29:59.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.641+0000 7fd94bf83700 1 -- 172.21.15.7:0/3843638081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd94410eac0 msgr2=0x7fd9441a3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:29:59.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.641+0000 7fd94bf83700 1 --2- 172.21.15.7:0/3843638081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd94410eac0 0x7fd9441a3b60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd93400b870 tx=0x7fd93400b8a0 comp rx=0 tx=0).stop 2024-06-10T01:29:59.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.642+0000 7fd94bf83700 1 -- 172.21.15.7:0/3843638081 shutdown_connections 2024-06-10T01:29:59.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.642+0000 7fd94bf83700 1 --2- 172.21.15.7:0/3843638081 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd9300a2e20 0x7fd9300a52e0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:59.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.642+0000 7fd94bf83700 1 --2- 172.21.15.7:0/3843638081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd94410eac0 0x7fd9441a3b60 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:59.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.642+0000 7fd94bf83700 1 --2- 172.21.15.7:0/3843638081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd944108ac0 0x7fd94419f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:59.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.642+0000 7fd94bf83700 1 --2- 172.21.15.7:0/3843638081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd944109480 0x7fd94419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:29:59.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.642+0000 7fd94bf83700 1 -- 172.21.15.7:0/3843638081 >> 172.21.15.7:0/3843638081 conn(0x7fd944075c60 msgr2=0x7fd9440761e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:29:59.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.644+0000 7fd94bf83700 1 -- 172.21.15.7:0/3843638081 shutdown_connections 2024-06-10T01:29:59.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:29:59.645+0000 7fd94bf83700 1 -- 172.21.15.7:0/3843638081 wait complete. 2024-06-10T01:29:59.641 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 55 2024-06-10T01:29:59.643 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:59 smithi007 ceph-mon[32398]: pgmap v1153: 129 pgs: 129 active+clean; 653 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 720 KiB/s wr, 111 op/s 2024-06-10T01:29:59.643 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:29:59 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2683698703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:29:59.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:59 smithi078 ceph-mon[36683]: pgmap v1153: 129 pgs: 129 active+clean; 653 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 720 KiB/s wr, 111 op/s 2024-06-10T01:29:59.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:29:59 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2683698703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:29:59.933 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:29:59.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:59 smithi175 ceph-mon[36636]: pgmap v1153: 129 pgs: 129 active+clean; 653 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 720 KiB/s wr, 111 op/s 2024-06-10T01:29:59.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:29:59 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2683698703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:30:00.565 INFO:tasks.workunit.client.0.smithi007.stdout: 10 252314 22.05 MB/sec execute 517 sec latency 118.216 ms 2024-06-10T01:30:00.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:00 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3843638081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:30:00.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:00 smithi078 ceph-mon[36683]: overall HEALTH_OK 2024-06-10T01:30:00.864 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:00 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3843638081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:30:00.865 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:00 smithi007 ceph-mon[32398]: overall HEALTH_OK 2024-06-10T01:30:00.866 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db / --timeout 90 --expiration 142.45 --await-for 45.0' 2024-06-10T01:30:00.866 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db / --timeout 90 --expiration 142.45 --await-for 45.0 2024-06-10T01:30:00.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:00 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3843638081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:30:00.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:00 smithi175 ceph-mon[36636]: overall HEALTH_OK 2024-06-10T01:30:01.001 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:30:01.566 INFO:tasks.workunit.client.0.smithi007.stdout: 10 252826 22.08 MB/sec execute 518 sec latency 114.821 ms 2024-06-10T01:30:01.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:01 smithi007 ceph-mon[32398]: pgmap v1154: 129 pgs: 129 active+clean; 646 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.3 MiB/s wr, 182 op/s 2024-06-10T01:30:01.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:01 smithi175 ceph-mon[36636]: pgmap v1154: 129 pgs: 129 active+clean; 646 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.3 MiB/s wr, 182 op/s 2024-06-10T01:30:02.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:01 smithi078 ceph-mon[36683]: pgmap v1154: 129 pgs: 129 active+clean; 646 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.3 MiB/s wr, 182 op/s 2024-06-10T01:30:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.571+0000 7f2a38088640 1 -- 172.21.15.7:0/1603744465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a301035d0 msgr2=0x7f2a30109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.571+0000 7f2a38088640 1 --2- 172.21.15.7:0/1603744465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a301035d0 0x7f2a30109e60 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2a20009900 tx=0x7f2a2002a410 comp rx=0 tx=0).stop 2024-06-10T01:30:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.571+0000 7f2a38088640 1 -- 172.21.15.7:0/1603744465 shutdown_connections 2024-06-10T01:30:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.571+0000 7f2a38088640 1 --2- 172.21.15.7:0/1603744465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a301035d0 0x7f2a30109e60 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.571+0000 7f2a38088640 1 --2- 172.21.15.7:0/1603744465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a30102c10 0x7f2a30103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.571+0000 7f2a38088640 1 --2- 172.21.15.7:0/1603744465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a30101a20 0x7f2a30101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.571+0000 7f2a38088640 1 -- 172.21.15.7:0/1603744465 >> 172.21.15.7:0/1603744465 conn(0x7f2a300fcf90 msgr2=0x7f2a300ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:02.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.571+0000 7f2a38088640 1 -- 172.21.15.7:0/1603744465 shutdown_connections 2024-06-10T01:30:02.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.571+0000 7f2a38088640 1 -- 172.21.15.7:0/1603744465 wait complete. 2024-06-10T01:30:02.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.572+0000 7f2a38088640 1 Processor -- start 2024-06-10T01:30:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.572+0000 7f2a38088640 1 -- start start 2024-06-10T01:30:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.572+0000 7f2a38088640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a30101a20 0x7f2a3019e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.572+0000 7f2a38088640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a30102c10 0x7f2a3019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.573+0000 7f2a38088640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a301035d0 0x7f2a301a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.573+0000 7f2a38088640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3010d6a0 con 0x7f2a301035d0 2024-06-10T01:30:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.573+0000 7f2a38088640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3010d520 con 0x7f2a30102c10 2024-06-10T01:30:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.573+0000 7f2a38088640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3010d820 con 0x7f2a30101a20 2024-06-10T01:30:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.573+0000 7f2a35e24640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a30101a20 0x7f2a3019e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.573+0000 7f2a35e24640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a30101a20 0x7f2a3019e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:59466/0 (socket says 172.21.15.7:59466) 2024-06-10T01:30:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.573+0000 7f2a35e24640 1 -- 172.21.15.7:0/1371754503 learned_addr learned my addr 172.21.15.7:0/1371754503 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.573+0000 7f2a36625640 1 --2- 172.21.15.7:0/1371754503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a301035d0 0x7f2a301a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.573+0000 7f2a35623640 1 --2- 172.21.15.7:0/1371754503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a30102c10 0x7f2a3019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.573+0000 7f2a35623640 1 -- 172.21.15.7:0/1371754503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a30101a20 msgr2=0x7f2a3019e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:02.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.573+0000 7f2a35623640 1 --2- 172.21.15.7:0/1371754503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a30101a20 0x7f2a3019e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:02.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.573+0000 7f2a35623640 1 -- 172.21.15.7:0/1371754503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a301035d0 msgr2=0x7f2a301a58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:02.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.573+0000 7f2a35623640 1 --2- 172.21.15.7:0/1371754503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a301035d0 0x7f2a301a58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:02.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.573+0000 7f2a35623640 1 -- 172.21.15.7:0/1371754503 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a301a5f50 con 0x7f2a30102c10 2024-06-10T01:30:02.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.573+0000 7f2a35e24640 1 --2- 172.21.15.7:0/1371754503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a30101a20 0x7f2a3019e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:30:02.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.573+0000 7f2a35623640 1 --2- 172.21.15.7:0/1371754503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a30102c10 0x7f2a3019e870 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2a2400e9f0 tx=0x7f2a2400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:02.568 INFO:tasks.workunit.client.0.smithi007.stdout: 10 253380 22.10 MB/sec execute 519 sec latency 123.397 ms 2024-06-10T01:30:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.574+0000 7f2a2effd640 1 -- 172.21.15.7:0/1371754503 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2400ccc0 con 0x7f2a30102c10 2024-06-10T01:30:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.574+0000 7f2a2effd640 1 -- 172.21.15.7:0/1371754503 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a24005020 con 0x7f2a30102c10 2024-06-10T01:30:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.574+0000 7f2a2effd640 1 -- 172.21.15.7:0/1371754503 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a240049e0 con 0x7f2a30102c10 2024-06-10T01:30:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.574+0000 7f2a38088640 1 -- 172.21.15.7:0/1371754503 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a301a6240 con 0x7f2a30102c10 2024-06-10T01:30:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.574+0000 7f2a38088640 1 -- 172.21.15.7:0/1371754503 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a301a6740 con 0x7f2a30102c10 2024-06-10T01:30:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.578+0000 7f2a2effd640 1 -- 172.21.15.7:0/1371754503 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f2a24005560 con 0x7f2a30102c10 2024-06-10T01:30:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.578+0000 7f2a3ad94740 1 Processor -- start 2024-06-10T01:30:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.579+0000 7f2a2effd640 1 --2- 172.21.15.7:0/1371754503 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2a180a2c00 0x7f2a180a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.580+0000 7f2a2effd640 1 -- 172.21.15.7:0/1371754503 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f2a240cf380 con 0x7f2a30102c10 2024-06-10T01:30:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.580+0000 7f2a3ad94740 1 -- start start 2024-06-10T01:30:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.580+0000 7f2a36625640 1 --2- 172.21.15.7:0/1371754503 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2a180a2c00 0x7f2a180a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.581+0000 7f2a3ad94740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56503ce41010 0x56503ce41430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.581+0000 7f2a3ad94740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56503ce41a00 0x56503ce46500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.581+0000 7f2a3ad94740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56503ce46b70 0x56503ce49000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.581+0000 7f2a3ad94740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56503cda44f0 con 0x56503ce41a00 2024-06-10T01:30:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.581+0000 7f2a3ad94740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56503cd42870 con 0x56503ce46b70 2024-06-10T01:30:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.581+0000 7f2a3ad94740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56503cda04b0 con 0x56503ce41010 2024-06-10T01:30:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.581+0000 7f2a35e24640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56503ce41010 0x56503ce41430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.581+0000 7f2a35e24640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56503ce41010 0x56503ce41430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:59482/0 (socket says 172.21.15.7:59482) 2024-06-10T01:30:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.581+0000 7f2a35e24640 1 -- 172.21.15.7:0/1708787708 learned_addr learned my addr 172.21.15.7:0/1708787708 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.581+0000 7f2a36625640 1 --2- 172.21.15.7:0/1708787708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56503ce46b70 0x56503ce49000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.582+0000 7f2a35e24640 1 -- 172.21.15.7:0/1708787708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56503ce46b70 msgr2=0x56503ce49000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.582+0000 7f2a35e24640 1 --2- 172.21.15.7:0/1708787708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56503ce46b70 0x56503ce49000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.582+0000 7f2a35e24640 1 -- 172.21.15.7:0/1708787708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56503ce41a00 msgr2=0x56503ce46500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.582+0000 7f2a35e24640 1 --2- 172.21.15.7:0/1708787708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56503ce41a00 0x56503ce46500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.582+0000 7f2a35e24640 1 -- 172.21.15.7:0/1708787708 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56503cd5e9b0 con 0x56503ce41010 2024-06-10T01:30:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.582+0000 7f2a36625640 1 --2- 172.21.15.7:0/1371754503 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2a180a2c00 0x7f2a180a50c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2a20031530 tx=0x7f2a20005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.582+0000 7f2a35e24640 1 --2- 172.21.15.7:0/1708787708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56503ce41010 0x56503ce41430 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2a28011900 tx=0x7f2a28011dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d27839a85c6767e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.583+0000 7f2a17fff640 1 -- 172.21.15.7:0/1708787708 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a280112f0 con 0x56503ce41010 2024-06-10T01:30:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.583+0000 7f2a17fff640 1 -- 172.21.15.7:0/1708787708 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a28010430 con 0x56503ce41010 2024-06-10T01:30:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.583+0000 7f2a17fff640 1 -- 172.21.15.7:0/1708787708 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a28019640 con 0x56503ce41010 2024-06-10T01:30:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.584+0000 7f2a3ad94740 1 -- 172.21.15.7:0/1708787708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56503ce41010 msgr2=0x56503ce41430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.584+0000 7f2a3ad94740 1 --2- 172.21.15.7:0/1708787708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56503ce41010 0x56503ce41430 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2a28011900 tx=0x7f2a28011dc0 comp rx=0 tx=0).stop 2024-06-10T01:30:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.584+0000 7f2a3ad94740 1 -- 172.21.15.7:0/1708787708 shutdown_connections 2024-06-10T01:30:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.584+0000 7f2a3ad94740 1 --2- 172.21.15.7:0/1708787708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56503ce46b70 0x56503ce49000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.584+0000 7f2a3ad94740 1 --2- 172.21.15.7:0/1708787708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56503ce41a00 0x56503ce46500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.584+0000 7f2a3ad94740 1 --2- 172.21.15.7:0/1708787708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56503ce41010 0x56503ce41430 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.584+0000 7f2a3ad94740 1 -- 172.21.15.7:0/1708787708 >> 172.21.15.7:0/1708787708 conn(0x56503ce3c680 msgr2=0x56503ce3ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.585+0000 7f2a3ad94740 1 -- 172.21.15.7:0/1708787708 shutdown_connections 2024-06-10T01:30:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.586+0000 7f2a3ad94740 1 -- 172.21.15.7:0/1708787708 wait complete. 2024-06-10T01:30:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.587+0000 7f2a3ad94740 1 -- start start 2024-06-10T01:30:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.587+0000 7f2a3ad94740 1 Processor -- start 2024-06-10T01:30:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.587+0000 7f2a3ad94740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56503ce41010 0x56503ce5c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.588+0000 7f2a3ad94740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56503ce41a00 0x56503ce5c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.588+0000 7f2a3ad94740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56503ce46b70 0x56503ce637e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.588+0000 7f2a35e24640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56503ce41010 0x56503ce5c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.588+0000 7f2a35e24640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56503ce41010 0x56503ce5c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52984/0 (socket says 172.21.15.7:52984) 2024-06-10T01:30:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.588+0000 7f2a3ad94740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56503ce5ab30 con 0x56503ce41010 2024-06-10T01:30:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.588+0000 7f2a35e24640 1 -- 172.21.15.7:0/2831304873 learned_addr learned my addr 172.21.15.7:0/2831304873 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.588+0000 7f2a35623640 1 --2- 172.21.15.7:0/2831304873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56503ce41a00 0x56503ce5c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.588+0000 7f2a3ad94740 1 -- 172.21.15.7:0/2831304873 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56503cd2dae0 con 0x56503ce41a00 2024-06-10T01:30:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.588+0000 7f2a3ad94740 1 -- 172.21.15.7:0/2831304873 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56503ce5acb0 con 0x56503ce46b70 2024-06-10T01:30:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.588+0000 7f2a36625640 1 --2- 172.21.15.7:0/2831304873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56503ce46b70 0x56503ce637e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.588+0000 7f2a3ad94740 10 client.0 _ll_register_callbacks cb 0x56503cd669b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:30:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.589+0000 7f2a36625640 1 -- 172.21.15.7:0/2831304873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56503ce41a00 msgr2=0x56503ce5c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.589+0000 7f2a36625640 1 --2- 172.21.15.7:0/2831304873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56503ce41a00 0x56503ce5c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.589+0000 7f2a36625640 1 -- 172.21.15.7:0/2831304873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56503ce41010 msgr2=0x56503ce5c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.589+0000 7f2a36625640 1 --2- 172.21.15.7:0/2831304873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56503ce41010 0x56503ce5c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.589+0000 7f2a36625640 1 -- 172.21.15.7:0/2831304873 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56503cab8210 con 0x56503ce46b70 2024-06-10T01:30:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.589+0000 7f2a35623640 1 --2- 172.21.15.7:0/2831304873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56503ce41a00 0x56503ce5c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:30:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.589+0000 7f2a36625640 1 --2- 172.21.15.7:0/2831304873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56503ce46b70 0x56503ce637e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2a20031400 tx=0x7f2a2002a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cff6fd3300f9af8d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.589+0000 7f2a157fa640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:30:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.589+0000 7f2a157fa640 1 -- 172.21.15.7:0/2831304873 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a20035050 con 0x56503ce46b70 2024-06-10T01:30:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.589+0000 7f2a157fa640 1 -- 172.21.15.7:0/2831304873 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f2a200037f0 con 0x56503ce46b70 2024-06-10T01:30:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.590+0000 7f2a3ad94740 1 -- 172.21.15.7:0/2831304873 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a28010430 con 0x56503ce46b70 2024-06-10T01:30:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.590+0000 7f2a157fa640 1 -- 172.21.15.7:0/2831304873 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f2a20031e60 con 0x56503ce46b70 2024-06-10T01:30:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.590+0000 7f2a3ad94740 10 client.30175 fetch_fsmap learned FSMap version 55 2024-06-10T01:30:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.590+0000 7f2a3ad94740 1 -- 172.21.15.7:0/2831304873 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x56503cb84620 con 0x56503ce46b70 2024-06-10T01:30:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.590+0000 7f2a157fa640 1 -- 172.21.15.7:0/2831304873 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a28010430 con 0x56503ce46b70 2024-06-10T01:30:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.590+0000 7f2a157fa640 1 -- 172.21.15.7:0/2831304873 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f2a2003dc00 con 0x56503ce46b70 2024-06-10T01:30:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.591+0000 7f2a3ad94740 10 client.30175 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:30:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.591+0000 7f2a3ad94740 10 client.30175 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:30:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.591+0000 7f2a3ad94740 20 client.30175 populate_metadata read hostname 'smithi007' 2024-06-10T01:30:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.591+0000 7f2a3ad94740 1 --2- 172.21.15.7:0/2831304873 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56503ce67e10 0x56503ce6a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.591+0000 7f2a3ad94740 4 client.30175 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:30:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.591+0000 7f2a3ad94740 1 -- 172.21.15.7:0/2831304873 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56503cdb7f10 con 0x56503ce67e10 2024-06-10T01:30:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.591+0000 7f2a35e24640 1 --2- 172.21.15.7:0/2831304873 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56503ce67e10 0x56503ce6a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.592+0000 7f2a35e24640 1 --2- 172.21.15.7:0/2831304873 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56503ce67e10 0x56503ce6a280 crc :-1 s=READY pgs=535 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7386e8bc6e197abc server_cookie=1f53273c9b1df06b in_seq=0 out_seq=0 2024-06-10T01:30:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.592+0000 7f2a157fa640 10 client.30175 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.593+0000 7f2a157fa640 1 -- 172.21.15.7:0/2831304873 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56503cdb7f10 con 0x56503ce67e10 2024-06-10T01:30:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.593+0000 7f2a157fa640 10 client.30175 handle_command_reply: tid=0 2024-06-10T01:30:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.593+0000 7f2a3ad94740 1 client.30175 shutdown 2024-06-10T01:30:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.593+0000 7f2a3ad94740 1 -- 172.21.15.7:0/2831304873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56503ce46b70 msgr2=0x56503ce637e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.593+0000 7f2a3ad94740 1 --2- 172.21.15.7:0/2831304873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56503ce46b70 0x56503ce637e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2a20031400 tx=0x7f2a2002a410 comp rx=0 tx=0).stop 2024-06-10T01:30:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.593+0000 7f2a3ad94740 1 -- 172.21.15.7:0/2831304873 shutdown_connections 2024-06-10T01:30:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.593+0000 7f2a3ad94740 1 --2- 172.21.15.7:0/2831304873 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56503ce67e10 0x56503ce6a280 crc :-1 s=READY pgs=535 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.593+0000 7f2a157fa640 0 client.30175 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.593+0000 7f2a3ad94740 1 --2- 172.21.15.7:0/2831304873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56503ce46b70 0x56503ce637e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.593+0000 7f2a3ad94740 1 --2- 172.21.15.7:0/2831304873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56503ce41a00 0x56503ce5c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.593+0000 7f2a3ad94740 1 --2- 172.21.15.7:0/2831304873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56503ce41010 0x56503ce5c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.593+0000 7f2a3ad94740 1 -- 172.21.15.7:0/2831304873 >> 172.21.15.7:0/2831304873 conn(0x56503ce3c680 msgr2=0x56503ce3b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.594+0000 7f2a3ad94740 1 -- 172.21.15.7:0/2831304873 shutdown_connections 2024-06-10T01:30:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.594+0000 7f2a3ad94740 1 -- 172.21.15.7:0/2831304873 wait complete. 2024-06-10T01:30:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.594+0000 7f2a3ad94740 20 client.30175 trim_cache size 0 max 16384 2024-06-10T01:30:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.618+0000 7f2a3ad94740 1 Processor -- start 2024-06-10T01:30:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.618+0000 7f2a3ad94740 1 -- start start 2024-06-10T01:30:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.618+0000 7f2a3ad94740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56503ce41010 0x56503ce42be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.619+0000 7f2a3ad94740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56503ce41a00 0x56503ce637a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.619+0000 7f2a3ad94740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56503ce46b70 0x56503ce43120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.619+0000 7f2a3ad94740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2a301aae00 con 0x56503ce46b70 2024-06-10T01:30:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.619+0000 7f2a35e24640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56503ce41010 0x56503ce42be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.619+0000 7f2a35623640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56503ce41a00 0x56503ce637a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.619+0000 7f2a35e24640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56503ce41010 0x56503ce42be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:37052/0 (socket says 172.21.15.7:37052) 2024-06-10T01:30:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.619+0000 7f2a35e24640 1 -- 172.21.15.7:0/1087261049 learned_addr learned my addr 172.21.15.7:0/1087261049 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.619+0000 7f2a36625640 1 --2- 172.21.15.7:0/1087261049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56503ce46b70 0x56503ce43120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.619+0000 7f2a3ad94740 1 -- 172.21.15.7:0/1087261049 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2a301a86d0 con 0x56503ce41010 2024-06-10T01:30:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.619+0000 7f2a3ad94740 1 -- 172.21.15.7:0/1087261049 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2a300736e0 con 0x56503ce41a00 2024-06-10T01:30:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.619+0000 7f2a35623640 1 -- 172.21.15.7:0/1087261049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56503ce41010 msgr2=0x56503ce42be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.619+0000 7f2a35623640 1 --2- 172.21.15.7:0/1087261049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56503ce41010 0x56503ce42be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.619+0000 7f2a35623640 1 -- 172.21.15.7:0/1087261049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56503ce46b70 msgr2=0x56503ce43120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.619+0000 7f2a35623640 1 --2- 172.21.15.7:0/1087261049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56503ce46b70 0x56503ce43120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.619+0000 7f2a35623640 1 -- 172.21.15.7:0/1087261049 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56503ce6a900 con 0x56503ce41a00 2024-06-10T01:30:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.620+0000 7f2a35623640 1 --2- 172.21.15.7:0/1087261049 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56503ce41a00 0x56503ce637a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x56503ce42470 tx=0x7f2a2409b280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97354684787e9672 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.620+0000 7f2a177fe640 1 -- 172.21.15.7:0/1087261049 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2401c070 con 0x56503ce41a00 2024-06-10T01:30:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.620+0000 7f2a177fe640 1 -- 172.21.15.7:0/1087261049 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a2409b860 con 0x56503ce41a00 2024-06-10T01:30:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.620+0000 7f2a177fe640 1 -- 172.21.15.7:0/1087261049 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a24086070 con 0x56503ce41a00 2024-06-10T01:30:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.621+0000 7f2a3ad94740 1 -- 172.21.15.7:0/1087261049 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56503ce41a00 msgr2=0x56503ce637a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.622+0000 7f2a3ad94740 1 --2- 172.21.15.7:0/1087261049 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56503ce41a00 0x56503ce637a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x56503ce42470 tx=0x7f2a2409b280 comp rx=0 tx=0).stop 2024-06-10T01:30:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.623+0000 7f2a3ad94740 1 -- 172.21.15.7:0/1087261049 shutdown_connections 2024-06-10T01:30:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.623+0000 7f2a3ad94740 1 --2- 172.21.15.7:0/1087261049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56503ce46b70 0x56503ce43120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.623+0000 7f2a3ad94740 1 --2- 172.21.15.7:0/1087261049 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56503ce41a00 0x56503ce637a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.623+0000 7f2a3ad94740 1 --2- 172.21.15.7:0/1087261049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56503ce41010 0x56503ce42be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.623+0000 7f2a3ad94740 1 -- 172.21.15.7:0/1087261049 >> 172.21.15.7:0/1087261049 conn(0x56503ce3c680 msgr2=0x56503ce46220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.623+0000 7f2a3ad94740 1 -- 172.21.15.7:0/1087261049 shutdown_connections 2024-06-10T01:30:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.623+0000 7f2a3ad94740 1 -- 172.21.15.7:0/1087261049 wait complete. 2024-06-10T01:30:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.624+0000 7f2a3ad94740 1 -- start start 2024-06-10T01:30:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.624+0000 7f2a3ad94740 1 Processor -- start 2024-06-10T01:30:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.625+0000 7f2a3ad94740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56503ce41010 0x56503ce603c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.625+0000 7f2a3ad94740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56503ce41a00 0x56503ce60900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.625+0000 7f2a3ad94740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56503ce46b70 0x56503ced0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.625+0000 7f2a3ad94740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2a301a6b90 con 0x56503ce46b70 2024-06-10T01:30:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.625+0000 7f2a3ad94740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2a301a8000 con 0x56503ce41a00 2024-06-10T01:30:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.625+0000 7f2a3ad94740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2a301a7260 con 0x56503ce41010 2024-06-10T01:30:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.625+0000 7f2a3ad94740 10 client.0 _ll_register_callbacks cb 0x7f2a24005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:30:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.625+0000 7f2a35623640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56503ce41a00 0x56503ce60900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.625+0000 7f2a35623640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56503ce41a00 0x56503ce60900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:37068/0 (socket says 172.21.15.7:37068) 2024-06-10T01:30:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.625+0000 7f2a35623640 1 -- 172.21.15.7:0/1516387332 learned_addr learned my addr 172.21.15.7:0/1516387332 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.625+0000 7f2a36625640 1 --2- 172.21.15.7:0/1516387332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56503ce46b70 0x56503ced0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.626+0000 7f2a35e24640 1 --2- 172.21.15.7:0/1516387332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56503ce41010 0x56503ce603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.626+0000 7f2a36625640 1 -- 172.21.15.7:0/1516387332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56503ce41010 msgr2=0x56503ce603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.626+0000 7f2a36625640 1 --2- 172.21.15.7:0/1516387332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56503ce41010 0x56503ce603c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.626+0000 7f2a36625640 1 -- 172.21.15.7:0/1516387332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56503ce41a00 msgr2=0x56503ce60900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.626+0000 7f2a36625640 1 --2- 172.21.15.7:0/1516387332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56503ce41a00 0x56503ce60900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.626+0000 7f2a36625640 1 -- 172.21.15.7:0/1516387332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56503ce426d0 con 0x56503ce46b70 2024-06-10T01:30:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.626+0000 7f2a36625640 1 --2- 172.21.15.7:0/1516387332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56503ce46b70 0x56503ced0ba0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f2a2002a930 tx=0x7f2a200038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8e9b7358c7db91 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.626+0000 7f2a157fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:30:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.627+0000 7f2a157fa640 1 -- 172.21.15.7:0/1516387332 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a20035040 con 0x56503ce46b70 2024-06-10T01:30:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.627+0000 7f2a157fa640 1 -- 172.21.15.7:0/1516387332 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f2a2003d450 con 0x56503ce46b70 2024-06-10T01:30:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.627+0000 7f2a3ad94740 1 -- 172.21.15.7:0/1516387332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a2409b860 con 0x56503ce46b70 2024-06-10T01:30:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.627+0000 7f2a157fa640 1 -- 172.21.15.7:0/1516387332 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f2a2006d050 con 0x56503ce46b70 2024-06-10T01:30:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.627+0000 7f2a3ad94740 10 client.21105 fetch_fsmap learned FSMap version 55 2024-06-10T01:30:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.627+0000 7f2a3ad94740 1 -- 172.21.15.7:0/1516387332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x56503ced1f80 con 0x56503ce46b70 2024-06-10T01:30:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.627+0000 7f2a157fa640 1 -- 172.21.15.7:0/1516387332 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a2409b860 con 0x56503ce46b70 2024-06-10T01:30:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.628+0000 7f2a157fa640 1 -- 172.21.15.7:0/1516387332 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f2a2003db90 con 0x56503ce46b70 2024-06-10T01:30:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.628+0000 7f2a3ad94740 10 client.21105 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:30:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.628+0000 7f2a3ad94740 10 client.21105 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:30:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.628+0000 7f2a3ad94740 20 client.21105 populate_metadata read hostname 'smithi007' 2024-06-10T01:30:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.628+0000 7f2a3ad94740 1 --2- 172.21.15.7:0/1516387332 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56503ced28d0 0x56503ced4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.628+0000 7f2a3ad94740 4 client.21105 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:30:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.628+0000 7f2a3ad94740 1 -- 172.21.15.7:0/1516387332 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56503cdb1c20 con 0x56503ced28d0 2024-06-10T01:30:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.628+0000 7f2a35e24640 1 --2- 172.21.15.7:0/1516387332 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56503ced28d0 0x56503ced4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.629+0000 7f2a157fa640 10 client.21105 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:02.629+0000 7f2a35e24640 1 --2- 172.21.15.7:0/1516387332 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56503ced28d0 0x56503ced4cf0 crc :-1 s=READY pgs=536 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26d11c22b9d87496 server_cookie=2bcbb515e68dcdf in_seq=0 out_seq=0 2024-06-10T01:30:02.781 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:30:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:03.435+0000 7f2a157fa640 1 -- 172.21.15.7:0/1516387332 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56503cdb1c20 con 0x56503ced28d0 2024-06-10T01:30:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:03.435+0000 7f2a157fa640 10 client.21105 handle_command_reply: tid=0 2024-06-10T01:30:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:03.436+0000 7f2a3ad94740 1 client.21105 shutdown 2024-06-10T01:30:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:03.436+0000 7f2a3ad94740 1 -- 172.21.15.7:0/1516387332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56503ce46b70 msgr2=0x56503ced0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:03.436+0000 7f2a3ad94740 1 --2- 172.21.15.7:0/1516387332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56503ce46b70 0x56503ced0ba0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f2a2002a930 tx=0x7f2a200038a0 comp rx=0 tx=0).stop 2024-06-10T01:30:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:03.436+0000 7f2a3ad94740 1 -- 172.21.15.7:0/1516387332 shutdown_connections 2024-06-10T01:30:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:03.436+0000 7f2a3ad94740 1 --2- 172.21.15.7:0/1516387332 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x56503ced28d0 0x56503ced4cf0 crc :-1 s=READY pgs=536 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:03.436+0000 7f2a3ad94740 1 --2- 172.21.15.7:0/1516387332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56503ce46b70 0x56503ced0ba0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:03.436+0000 7f2a3ad94740 1 --2- 172.21.15.7:0/1516387332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56503ce41a00 0x56503ce60900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:03.436+0000 7f2a3ad94740 1 --2- 172.21.15.7:0/1516387332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56503ce41010 0x56503ce603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:03.436+0000 7f2a3ad94740 1 -- 172.21.15.7:0/1516387332 >> 172.21.15.7:0/1516387332 conn(0x56503ce3c680 msgr2=0x56503ce5c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:03.436+0000 7f2a157fa640 0 client.21105 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:03.436+0000 7f2a3ad94740 1 -- 172.21.15.7:0/1516387332 shutdown_connections 2024-06-10T01:30:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:03.436+0000 7f2a3ad94740 1 -- 172.21.15.7:0/1516387332 wait complete. 2024-06-10T01:30:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:03.436+0000 7f2a3ad94740 20 client.21105 trim_cache size 0 max 16384 2024-06-10T01:30:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:30:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:30:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:30:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:03.437+0000 7f2a38088640 1 -- 172.21.15.7:0/1371754503 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2a180a2c00 msgr2=0x7f2a180a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:03.437+0000 7f2a38088640 1 --2- 172.21.15.7:0/1371754503 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2a180a2c00 0x7f2a180a50c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2a20031530 tx=0x7f2a20005db0 comp rx=0 tx=0).stop 2024-06-10T01:30:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:03.437+0000 7f2a38088640 1 -- 172.21.15.7:0/1371754503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a30102c10 msgr2=0x7f2a3019e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:03.437+0000 7f2a38088640 1 --2- 172.21.15.7:0/1371754503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a30102c10 0x7f2a3019e870 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2a2400e9f0 tx=0x7f2a2400eeb0 comp rx=0 tx=0).stop 2024-06-10T01:30:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:03.438+0000 7f2a38088640 1 -- 172.21.15.7:0/1371754503 shutdown_connections 2024-06-10T01:30:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:03.438+0000 7f2a38088640 1 --2- 172.21.15.7:0/1371754503 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2a180a2c00 0x7f2a180a50c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:03.438+0000 7f2a38088640 1 --2- 172.21.15.7:0/1371754503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a301035d0 0x7f2a301a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:03.438+0000 7f2a38088640 1 --2- 172.21.15.7:0/1371754503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a30102c10 0x7f2a3019e870 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:03.438+0000 7f2a38088640 1 --2- 172.21.15.7:0/1371754503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a30101a20 0x7f2a3019e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:03.438+0000 7f2a38088640 1 -- 172.21.15.7:0/1371754503 >> 172.21.15.7:0/1371754503 conn(0x7f2a300fcf90 msgr2=0x7f2a30103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:03.438+0000 7f2a38088640 1 -- 172.21.15.7:0/1371754503 shutdown_connections 2024-06-10T01:30:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:03.438+0000 7f2a38088640 1 -- 172.21.15.7:0/1371754503 wait complete. 2024-06-10T01:30:03.566 INFO:tasks.workunit.client.0.smithi007.stdout: 10 253629 22.09 MB/sec execute 520 sec latency 584.014 ms 2024-06-10T01:30:03.758 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:03 smithi007 ceph-mon[32398]: pgmap v1155: 129 pgs: 129 active+clean; 652 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 2.0 MiB/s wr, 279 op/s 2024-06-10T01:30:03.867 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:03 smithi078 ceph-mon[36683]: pgmap v1155: 129 pgs: 129 active+clean; 652 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 2.0 MiB/s wr, 279 op/s 2024-06-10T01:30:03.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:03 smithi175 ceph-mon[36636]: pgmap v1155: 129 pgs: 129 active+clean; 652 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 2.0 MiB/s wr, 279 op/s 2024-06-10T01:30:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.158+0000 7fe7e0867700 1 -- 172.21.15.7:0/1702419321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7dc100870 msgr2=0x7fe7dc100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.158+0000 7fe7e0867700 1 --2- 172.21.15.7:0/1702419321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7dc100870 0x7fe7dc100c50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400b0a0 tx=0x7fe7d400b3b0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.158+0000 7fe7e0867700 1 -- 172.21.15.7:0/1702419321 shutdown_connections 2024-06-10T01:30:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.158+0000 7fe7e0867700 1 --2- 172.21.15.7:0/1702419321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7dc108950 0x7fe7dc10f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.158+0000 7fe7e0867700 1 --2- 172.21.15.7:0/1702419321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7dc100870 0x7fe7dc100c50 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.158+0000 7fe7e0867700 1 --2- 172.21.15.7:0/1702419321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7dc101220 0x7fe7dc108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.158+0000 7fe7e0867700 1 -- 172.21.15.7:0/1702419321 >> 172.21.15.7:0/1702419321 conn(0x7fe7dc077590 msgr2=0x7fe7dc0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.158+0000 7fe7e0867700 1 -- 172.21.15.7:0/1702419321 shutdown_connections 2024-06-10T01:30:04.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.158+0000 7fe7e0867700 1 -- 172.21.15.7:0/1702419321 wait complete. 2024-06-10T01:30:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.159+0000 7fe7e0867700 1 Processor -- start 2024-06-10T01:30:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.159+0000 7fe7e0867700 1 -- start start 2024-06-10T01:30:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.159+0000 7fe7e0867700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7dc100870 0x7fe7dc19c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.159+0000 7fe7e0867700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7dc101220 0x7fe7dc19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.160+0000 7fe7e0867700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7dc108950 0x7fe7dc196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.160+0000 7fe7e0867700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc110690 con 0x7fe7dc101220 2024-06-10T01:30:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.160+0000 7fe7e0867700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc110510 con 0x7fe7dc100870 2024-06-10T01:30:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.160+0000 7fe7e0867700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc110810 con 0x7fe7dc108950 2024-06-10T01:30:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.160+0000 7fe7da59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7dc100870 0x7fe7dc19c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.160+0000 7fe7da59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7dc100870 0x7fe7dc19c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:37080/0 (socket says 172.21.15.7:37080) 2024-06-10T01:30:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.160+0000 7fe7da59c700 1 -- 172.21.15.7:0/2193918192 learned_addr learned my addr 172.21.15.7:0/2193918192 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.160+0000 7fe7d9d9b700 1 --2- 172.21.15.7:0/2193918192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7dc101220 0x7fe7dc19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:04.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.160+0000 7fe7dad9d700 1 --2- 172.21.15.7:0/2193918192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7dc108950 0x7fe7dc196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:04.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.160+0000 7fe7da59c700 1 -- 172.21.15.7:0/2193918192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7dc108950 msgr2=0x7fe7dc196910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:04.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.160+0000 7fe7da59c700 1 --2- 172.21.15.7:0/2193918192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7dc108950 0x7fe7dc196910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.160+0000 7fe7da59c700 1 -- 172.21.15.7:0/2193918192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7dc101220 msgr2=0x7fe7dc19cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:04.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.160+0000 7fe7da59c700 1 --2- 172.21.15.7:0/2193918192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7dc101220 0x7fe7dc19cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.160+0000 7fe7da59c700 1 -- 172.21.15.7:0/2193918192 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7dc197070 con 0x7fe7dc100870 2024-06-10T01:30:04.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.160+0000 7fe7d9d9b700 1 --2- 172.21.15.7:0/2193918192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7dc101220 0x7fe7dc19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:30:04.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.161+0000 7fe7da59c700 1 --2- 172.21.15.7:0/2193918192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7dc100870 0x7fe7dc19c750 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe7d40062a0 tx=0x7fe7d4004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:04.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.161+0000 7fe7cb7fe700 1 -- 172.21.15.7:0/2193918192 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4021070 con 0x7fe7dc100870 2024-06-10T01:30:04.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.161+0000 7fe7e0867700 1 -- 172.21.15.7:0/2193918192 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7dc197300 con 0x7fe7dc100870 2024-06-10T01:30:04.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.161+0000 7fe7e0867700 1 -- 172.21.15.7:0/2193918192 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7dc1a3620 con 0x7fe7dc100870 2024-06-10T01:30:04.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.161+0000 7fe7cb7fe700 1 -- 172.21.15.7:0/2193918192 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7d4007b80 con 0x7fe7dc100870 2024-06-10T01:30:04.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.161+0000 7fe7cb7fe700 1 -- 172.21.15.7:0/2193918192 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d40052a0 con 0x7fe7dc100870 2024-06-10T01:30:04.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.164+0000 7fe7cb7fe700 1 -- 172.21.15.7:0/2193918192 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fe7d4024070 con 0x7fe7dc100870 2024-06-10T01:30:04.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.164+0000 7fe7cb7fe700 1 --2- 172.21.15.7:0/2193918192 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe7c40a2d20 0x7fe7c40a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:04.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.165+0000 7fe7d9d9b700 1 --2- 172.21.15.7:0/2193918192 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe7c40a2d20 0x7fe7c40a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:04.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.165+0000 7fe7cb7fe700 1 -- 172.21.15.7:0/2193918192 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fe7d400f020 con 0x7fe7dc100870 2024-06-10T01:30:04.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.165+0000 7fe7d9d9b700 1 --2- 172.21.15.7:0/2193918192 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe7c40a2d20 0x7fe7c40a51e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0ffc80 tx=0x7fe7d0013040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:04.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.165+0000 7fe7eb0eab80 1 Processor -- start 2024-06-10T01:30:04.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.165+0000 7fe7eb0eab80 1 -- start start 2024-06-10T01:30:04.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.166+0000 7fe7eb0eab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562953f26f60 0x562953f27340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:04.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.166+0000 7fe7eb0eab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562953f27880 0x562953f21260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:04.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.166+0000 7fe7eb0eab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562953f217a0 0x562953f21bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:04.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.166+0000 7fe7eb0eab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562953e13f60 con 0x562953f26f60 2024-06-10T01:30:04.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.166+0000 7fe7eb0eab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562953e72850 con 0x562953f27880 2024-06-10T01:30:04.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.166+0000 7fe7eb0eab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562953dda460 con 0x562953f217a0 2024-06-10T01:30:04.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.166+0000 7fe7d9d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562953f27880 0x562953f21260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:04.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.166+0000 7fe7dad9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562953f217a0 0x562953f21bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:04.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.166+0000 7fe7dad9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562953f217a0 0x562953f21bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:59528/0 (socket says 172.21.15.7:59528) 2024-06-10T01:30:04.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.166+0000 7fe7d9d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562953f27880 0x562953f21260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:37090/0 (socket says 172.21.15.7:37090) 2024-06-10T01:30:04.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.166+0000 7fe7dad9d700 1 -- 172.21.15.7:0/2604817495 learned_addr learned my addr 172.21.15.7:0/2604817495 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:04.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.166+0000 7fe7da59c700 1 --2- 172.21.15.7:0/2604817495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562953f26f60 0x562953f27340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:04.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.166+0000 7fe7da59c700 1 -- 172.21.15.7:0/2604817495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562953f217a0 msgr2=0x562953f21bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:04.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.166+0000 7fe7da59c700 1 --2- 172.21.15.7:0/2604817495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562953f217a0 0x562953f21bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.166+0000 7fe7da59c700 1 -- 172.21.15.7:0/2604817495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562953f27880 msgr2=0x562953f21260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:04.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.166+0000 7fe7da59c700 1 --2- 172.21.15.7:0/2604817495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562953f27880 0x562953f21260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.166+0000 7fe7da59c700 1 -- 172.21.15.7:0/2604817495 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562953dc18d0 con 0x562953f26f60 2024-06-10T01:30:04.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.167+0000 7fe7da59c700 1 --2- 172.21.15.7:0/2604817495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562953f26f60 0x562953f27340 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fe7d401ddd0 tx=0x7fe7d400b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8515a82e8d903b7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:04.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.167+0000 7fe7c8ff9700 1 -- 172.21.15.7:0/2604817495 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d40a6a30 con 0x562953f26f60 2024-06-10T01:30:04.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.167+0000 7fe7c8ff9700 1 -- 172.21.15.7:0/2604817495 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7d40a0030 con 0x562953f26f60 2024-06-10T01:30:04.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.167+0000 7fe7c8ff9700 1 -- 172.21.15.7:0/2604817495 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4007d30 con 0x562953f26f60 2024-06-10T01:30:04.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.168+0000 7fe7eb0eab80 1 -- 172.21.15.7:0/2604817495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562953f26f60 msgr2=0x562953f27340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:04.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.168+0000 7fe7eb0eab80 1 --2- 172.21.15.7:0/2604817495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562953f26f60 0x562953f27340 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fe7d401ddd0 tx=0x7fe7d400b3b0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.168+0000 7fe7eb0eab80 1 -- 172.21.15.7:0/2604817495 shutdown_connections 2024-06-10T01:30:04.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.168+0000 7fe7eb0eab80 1 --2- 172.21.15.7:0/2604817495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562953f217a0 0x562953f21bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.168+0000 7fe7eb0eab80 1 --2- 172.21.15.7:0/2604817495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562953f26f60 0x562953f27340 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.168+0000 7fe7eb0eab80 1 --2- 172.21.15.7:0/2604817495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562953f27880 0x562953f21260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.168+0000 7fe7eb0eab80 1 -- 172.21.15.7:0/2604817495 >> 172.21.15.7:0/2604817495 conn(0x562953e4e080 msgr2=0x562953f1c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:04.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.168+0000 7fe7eb0eab80 1 -- 172.21.15.7:0/2604817495 shutdown_connections 2024-06-10T01:30:04.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.168+0000 7fe7eb0eab80 1 -- 172.21.15.7:0/2604817495 wait complete. 2024-06-10T01:30:04.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.169+0000 7fe7eb0eab80 1 -- start start 2024-06-10T01:30:04.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.169+0000 7fe7eb0eab80 1 Processor -- start 2024-06-10T01:30:04.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.169+0000 7fe7eb0eab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562953f217a0 0x562953f2d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:04.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.169+0000 7fe7eb0eab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562953f26f60 0x562953f2a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:04.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.169+0000 7fe7da59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562953f217a0 0x562953f2d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:04.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.170+0000 7fe7da59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562953f217a0 0x562953f2d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:37102/0 (socket says 172.21.15.7:37102) 2024-06-10T01:30:04.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.170+0000 7fe7da59c700 1 -- 172.21.15.7:0/503916756 learned_addr learned my addr 172.21.15.7:0/503916756 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:04.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.170+0000 7fe7eb0eab80 1 --2- 172.21.15.7:0/503916756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562953f27880 0x562953f2ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:04.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.170+0000 7fe7eb0eab80 1 -- 172.21.15.7:0/503916756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562953e12fc0 con 0x562953f26f60 2024-06-10T01:30:04.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.170+0000 7fe7eb0eab80 1 -- 172.21.15.7:0/503916756 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562953e6e810 con 0x562953f217a0 2024-06-10T01:30:04.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.170+0000 7fe7eb0eab80 1 -- 172.21.15.7:0/503916756 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562953f2e010 con 0x562953f27880 2024-06-10T01:30:04.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.170+0000 7fe7dad9d700 1 --2- 172.21.15.7:0/503916756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562953f27880 0x562953f2ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:04.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.170+0000 7fe7eb0eab80 10 client.0 _ll_register_callbacks cb 0x562953e085b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:30:04.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.170+0000 7fe7da59c700 1 -- 172.21.15.7:0/503916756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562953f27880 msgr2=0x562953f2ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:04.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.170+0000 7fe7da59c700 1 --2- 172.21.15.7:0/503916756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562953f27880 0x562953f2ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.170+0000 7fe7da59c700 1 -- 172.21.15.7:0/503916756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562953f26f60 msgr2=0x562953f2a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:04.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.170+0000 7fe7da59c700 1 --2- 172.21.15.7:0/503916756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562953f26f60 0x562953f2a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.170+0000 7fe7da59c700 1 -- 172.21.15.7:0/503916756 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562953e38750 con 0x562953f217a0 2024-06-10T01:30:04.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.170+0000 7fe7da59c700 1 --2- 172.21.15.7:0/503916756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562953f217a0 0x562953f2d4e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe7d409a2b0 tx=0x7fe7d40a2370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a390faca78626b0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:04.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.170+0000 7fe7c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:30:04.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.170+0000 7fe7c1ffb700 1 -- 172.21.15.7:0/503916756 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d40aa020 con 0x562953f217a0 2024-06-10T01:30:04.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.171+0000 7fe7c1ffb700 1 -- 172.21.15.7:0/503916756 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fe7d40ce130 con 0x562953f217a0 2024-06-10T01:30:04.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.171+0000 7fe7eb0eab80 1 -- 172.21.15.7:0/503916756 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7d40a0030 con 0x562953f217a0 2024-06-10T01:30:04.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.171+0000 7fe7c1ffb700 1 -- 172.21.15.7:0/503916756 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fe7d40074e0 con 0x562953f217a0 2024-06-10T01:30:04.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.171+0000 7fe7eb0eab80 10 client.30293 fetch_fsmap learned FSMap version 55 2024-06-10T01:30:04.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.171+0000 7fe7eb0eab80 1 -- 172.21.15.7:0/503916756 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x562953f386e0 con 0x562953f217a0 2024-06-10T01:30:04.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.171+0000 7fe7c1ffb700 1 -- 172.21.15.7:0/503916756 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7d401d930 con 0x562953f217a0 2024-06-10T01:30:04.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.171+0000 7fe7c1ffb700 1 -- 172.21.15.7:0/503916756 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fe7d4011070 con 0x562953f217a0 2024-06-10T01:30:04.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.172+0000 7fe7eb0eab80 10 client.30293 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:30:04.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.172+0000 7fe7eb0eab80 10 client.30293 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:30:04.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.172+0000 7fe7eb0eab80 20 client.30293 populate_metadata read hostname 'smithi007' 2024-06-10T01:30:04.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.172+0000 7fe7eb0eab80 1 --2- 172.21.15.7:0/503916756 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x562953f3abb0 0x562953f3cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:04.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.172+0000 7fe7eb0eab80 4 client.30293 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:30:04.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.172+0000 7fe7eb0eab80 1 -- 172.21.15.7:0/503916756 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe7d40a0030 con 0x562953f3abb0 2024-06-10T01:30:04.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.172+0000 7fe7d9d9b700 1 --2- 172.21.15.7:0/503916756 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x562953f3abb0 0x562953f3cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:04.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.172+0000 7fe7d9d9b700 1 --2- 172.21.15.7:0/503916756 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x562953f3abb0 0x562953f3cfa0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=63ba1c8b10690fb8 server_cookie=33e36ae65f7192a7 in_seq=0 out_seq=0 2024-06-10T01:30:04.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.172+0000 7fe7c1ffb700 10 client.30293 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:30:04.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.173+0000 7fe7c1ffb700 1 -- 172.21.15.7:0/503916756 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fe7d40a0030 con 0x562953f3abb0 2024-06-10T01:30:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.173+0000 7fe7c1ffb700 10 client.30293 handle_command_reply: tid=0 2024-06-10T01:30:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.173+0000 7fe7eb0eab80 1 client.30293 shutdown 2024-06-10T01:30:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.174+0000 7fe7eb0eab80 1 -- 172.21.15.7:0/503916756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562953f217a0 msgr2=0x562953f2d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.174+0000 7fe7eb0eab80 1 --2- 172.21.15.7:0/503916756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562953f217a0 0x562953f2d4e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe7d409a2b0 tx=0x7fe7d40a2370 comp rx=0 tx=0).stop 2024-06-10T01:30:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.174+0000 7fe7eb0eab80 1 -- 172.21.15.7:0/503916756 shutdown_connections 2024-06-10T01:30:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.174+0000 7fe7eb0eab80 1 --2- 172.21.15.7:0/503916756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562953f27880 0x562953f2ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.174+0000 7fe7eb0eab80 1 --2- 172.21.15.7:0/503916756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562953f217a0 0x562953f2d4e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.174+0000 7fe7eb0eab80 1 --2- 172.21.15.7:0/503916756 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x562953f3abb0 0x562953f3cfa0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.174+0000 7fe7eb0eab80 1 --2- 172.21.15.7:0/503916756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562953f26f60 0x562953f2a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.174+0000 7fe7eb0eab80 1 -- 172.21.15.7:0/503916756 >> 172.21.15.7:0/503916756 conn(0x562953e4e080 msgr2=0x562953f1b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.174+0000 7fe7c1ffb700 0 client.30293 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:30:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.175+0000 7fe7eb0eab80 1 -- 172.21.15.7:0/503916756 shutdown_connections 2024-06-10T01:30:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.175+0000 7fe7eb0eab80 1 -- 172.21.15.7:0/503916756 wait complete. 2024-06-10T01:30:04.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.175+0000 7fe7eb0eab80 20 client.30293 trim_cache size 0 max 16384 2024-06-10T01:30:04.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.192+0000 7fe7eb0eab80 1 Processor -- start 2024-06-10T01:30:04.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.192+0000 7fe7eb0eab80 1 -- start start 2024-06-10T01:30:04.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.192+0000 7fe7eb0eab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562953f217a0 0x562953f21260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:04.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.192+0000 7fe7eb0eab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562953f26f60 0x562953f2ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:04.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.192+0000 7fe7eb0eab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562953f27880 0x562953f24ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:04.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.192+0000 7fe7eb0eab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc011310 con 0x562953f27880 2024-06-10T01:30:04.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.192+0000 7fe7eb0eab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc00eba0 con 0x562953f26f60 2024-06-10T01:30:04.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.192+0000 7fe7eb0eab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc00c9f0 con 0x562953f217a0 2024-06-10T01:30:04.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.192+0000 7fe7dad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562953f27880 0x562953f24ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:04.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.192+0000 7fe7da59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562953f217a0 0x562953f21260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:04.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.192+0000 7fe7d9d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562953f26f60 0x562953f2ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:04.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.192+0000 7fe7da59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562953f217a0 0x562953f21260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:59536/0 (socket says 172.21.15.7:59536) 2024-06-10T01:30:04.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.192+0000 7fe7d9d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562953f26f60 0x562953f2ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:37112/0 (socket says 172.21.15.7:37112) 2024-06-10T01:30:04.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.192+0000 7fe7da59c700 1 -- 172.21.15.7:0/4196154605 learned_addr learned my addr 172.21.15.7:0/4196154605 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:04.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.193+0000 7fe7d9d9b700 1 -- 172.21.15.7:0/4196154605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562953f217a0 msgr2=0x562953f21260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:04.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.193+0000 7fe7d9d9b700 1 --2- 172.21.15.7:0/4196154605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562953f217a0 0x562953f21260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.193+0000 7fe7d9d9b700 1 -- 172.21.15.7:0/4196154605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562953f27880 msgr2=0x562953f24ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:04.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.193+0000 7fe7d9d9b700 1 --2- 172.21.15.7:0/4196154605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562953f27880 0x562953f24ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.193+0000 7fe7d9d9b700 1 -- 172.21.15.7:0/4196154605 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562953f28de0 con 0x562953f26f60 2024-06-10T01:30:04.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.193+0000 7fe7da59c700 1 --2- 172.21.15.7:0/4196154605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562953f217a0 0x562953f21260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:30:04.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.193+0000 7fe7d9d9b700 1 --2- 172.21.15.7:0/4196154605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562953f26f60 0x562953f2ae80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x562953f3d6e0 tx=0x7fe7d00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0dddaf2072ed8bf server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:04.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.193+0000 7fe7c3fff700 1 -- 172.21.15.7:0/4196154605 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d002b650 con 0x562953f26f60 2024-06-10T01:30:04.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.193+0000 7fe7c3fff700 1 -- 172.21.15.7:0/4196154605 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7d002bcd0 con 0x562953f26f60 2024-06-10T01:30:04.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.193+0000 7fe7c3fff700 1 -- 172.21.15.7:0/4196154605 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d001fdd0 con 0x562953f26f60 2024-06-10T01:30:04.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.194+0000 7fe7eb0eab80 1 -- 172.21.15.7:0/4196154605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562953f26f60 msgr2=0x562953f2ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:04.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.194+0000 7fe7eb0eab80 1 --2- 172.21.15.7:0/4196154605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562953f26f60 0x562953f2ae80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x562953f3d6e0 tx=0x7fe7d00047c0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.194+0000 7fe7eb0eab80 1 -- 172.21.15.7:0/4196154605 shutdown_connections 2024-06-10T01:30:04.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.194+0000 7fe7eb0eab80 1 --2- 172.21.15.7:0/4196154605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562953f27880 0x562953f24ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.194+0000 7fe7eb0eab80 1 --2- 172.21.15.7:0/4196154605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562953f217a0 0x562953f21260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.194+0000 7fe7eb0eab80 1 --2- 172.21.15.7:0/4196154605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562953f26f60 0x562953f2ae80 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.194+0000 7fe7eb0eab80 1 -- 172.21.15.7:0/4196154605 >> 172.21.15.7:0/4196154605 conn(0x562953e4e080 msgr2=0x562953f247d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:04.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.194+0000 7fe7eb0eab80 1 -- 172.21.15.7:0/4196154605 shutdown_connections 2024-06-10T01:30:04.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.194+0000 7fe7eb0eab80 1 -- 172.21.15.7:0/4196154605 wait complete. 2024-06-10T01:30:04.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.195+0000 7fe7eb0eab80 1 -- start start 2024-06-10T01:30:04.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.195+0000 7fe7eb0eab80 1 Processor -- start 2024-06-10T01:30:04.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.195+0000 7fe7eb0eab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562953f217a0 0x562953f335a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:04.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.195+0000 7fe7eb0eab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562953f26f60 0x562953f33ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:04.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.195+0000 7fe7eb0eab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562953f27880 0x562953f41ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:04.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.195+0000 7fe7eb0eab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc00d060 con 0x562953f26f60 2024-06-10T01:30:04.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.195+0000 7fe7da59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562953f217a0 0x562953f335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:04.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.195+0000 7fe7da59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562953f217a0 0x562953f335a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:59538/0 (socket says 172.21.15.7:59538) 2024-06-10T01:30:04.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.195+0000 7fe7da59c700 1 -- 172.21.15.7:0/52358701 learned_addr learned my addr 172.21.15.7:0/52358701 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:04.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.195+0000 7fe7eb0eab80 1 -- 172.21.15.7:0/52358701 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc00e4d0 con 0x562953f27880 2024-06-10T01:30:04.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.195+0000 7fe7eb0eab80 1 -- 172.21.15.7:0/52358701 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc00d730 con 0x562953f217a0 2024-06-10T01:30:04.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.195+0000 7fe7d9d9b700 1 --2- 172.21.15.7:0/52358701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562953f26f60 0x562953f33ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:04.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.196+0000 7fe7dad9d700 1 --2- 172.21.15.7:0/52358701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562953f27880 0x562953f41ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:04.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.196+0000 7fe7eb0eab80 10 client.0 _ll_register_callbacks cb 0x562953e085b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:30:04.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.196+0000 7fe7d9d9b700 1 -- 172.21.15.7:0/52358701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562953f217a0 msgr2=0x562953f335a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:04.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.196+0000 7fe7d9d9b700 1 --2- 172.21.15.7:0/52358701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562953f217a0 0x562953f335a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.196+0000 7fe7d9d9b700 1 -- 172.21.15.7:0/52358701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562953f27880 msgr2=0x562953f41ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:04.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.196+0000 7fe7d9d9b700 1 --2- 172.21.15.7:0/52358701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562953f27880 0x562953f41ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.196+0000 7fe7d9d9b700 1 -- 172.21.15.7:0/52358701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562953f42710 con 0x562953f26f60 2024-06-10T01:30:04.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.196+0000 7fe7d9d9b700 1 --2- 172.21.15.7:0/52358701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562953f26f60 0x562953f33ae0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x562953f1d110 tx=0x7fe7d001f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed366fa56d1cb6f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:04.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.196+0000 7fe7c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:30:04.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.196+0000 7fe7c1ffb700 1 -- 172.21.15.7:0/52358701 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d00118e0 con 0x562953f26f60 2024-06-10T01:30:04.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.196+0000 7fe7c1ffb700 1 -- 172.21.15.7:0/52358701 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fe7d001fc60 con 0x562953f26f60 2024-06-10T01:30:04.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.196+0000 7fe7eb0eab80 1 -- 172.21.15.7:0/52358701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7d002bcd0 con 0x562953f26f60 2024-06-10T01:30:04.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.197+0000 7fe7c1ffb700 1 -- 172.21.15.7:0/52358701 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fe7d001fe60 con 0x562953f26f60 2024-06-10T01:30:04.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.197+0000 7fe7eb0eab80 10 client.21117 fetch_fsmap learned FSMap version 55 2024-06-10T01:30:04.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.197+0000 7fe7eb0eab80 1 -- 172.21.15.7:0/52358701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x562953fa53d0 con 0x562953f26f60 2024-06-10T01:30:04.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.197+0000 7fe7c1ffb700 1 -- 172.21.15.7:0/52358701 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7d002bcd0 con 0x562953f26f60 2024-06-10T01:30:04.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.197+0000 7fe7c1ffb700 1 -- 172.21.15.7:0/52358701 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fe7d003b070 con 0x562953f26f60 2024-06-10T01:30:04.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.197+0000 7fe7eb0eab80 10 client.21117 fetch_fsmap finished waiting for FSMap version 55 2024-06-10T01:30:04.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.197+0000 7fe7eb0eab80 10 client.21117 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:30:04.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.197+0000 7fe7eb0eab80 20 client.21117 populate_metadata read hostname 'smithi007' 2024-06-10T01:30:04.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.197+0000 7fe7eb0eab80 1 --2- 172.21.15.7:0/52358701 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x562953fa5a40 0x562953fa7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:04.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.197+0000 7fe7eb0eab80 4 client.21117 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:30:04.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.197+0000 7fe7eb0eab80 1 -- 172.21.15.7:0/52358701 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562953e7fd50 con 0x562953fa5a40 2024-06-10T01:30:04.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.198+0000 7fe7da59c700 1 --2- 172.21.15.7:0/52358701 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x562953fa5a40 0x562953fa7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:04.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.198+0000 7fe7c1ffb700 10 client.21117 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:30:04.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.198+0000 7fe7da59c700 1 --2- 172.21.15.7:0/52358701 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x562953fa5a40 0x562953fa7e60 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=201ab6ba733aa3fe server_cookie=3adc3e4b378b6400 in_seq=0 out_seq=0 2024-06-10T01:30:04.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.200+0000 7fe7c1ffb700 1 -- 172.21.15.7:0/52358701 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562953e7fd50 con 0x562953fa5a40 2024-06-10T01:30:04.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.200+0000 7fe7c1ffb700 10 client.21117 handle_command_reply: tid=0 2024-06-10T01:30:04.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.200+0000 7fe7eb0eab80 1 client.21117 shutdown 2024-06-10T01:30:04.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.200+0000 7fe7eb0eab80 1 -- 172.21.15.7:0/52358701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562953f26f60 msgr2=0x562953f33ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:04.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.200+0000 7fe7eb0eab80 1 --2- 172.21.15.7:0/52358701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562953f26f60 0x562953f33ae0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x562953f1d110 tx=0x7fe7d001f670 comp rx=0 tx=0).stop 2024-06-10T01:30:04.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.200+0000 7fe7eb0eab80 1 -- 172.21.15.7:0/52358701 shutdown_connections 2024-06-10T01:30:04.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.200+0000 7fe7eb0eab80 1 --2- 172.21.15.7:0/52358701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562953f27880 0x562953f41ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.200+0000 7fe7eb0eab80 1 --2- 172.21.15.7:0/52358701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562953f217a0 0x562953f335a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.200+0000 7fe7eb0eab80 1 --2- 172.21.15.7:0/52358701 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x562953fa5a40 0x562953fa7e60 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.200+0000 7fe7eb0eab80 1 --2- 172.21.15.7:0/52358701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562953f26f60 0x562953f33ae0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.200+0000 7fe7c1ffb700 0 client.21117 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:30:04.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.200+0000 7fe7eb0eab80 1 -- 172.21.15.7:0/52358701 >> 172.21.15.7:0/52358701 conn(0x562953e4e080 msgr2=0x562953f27c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:04.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.200+0000 7fe7eb0eab80 1 -- 172.21.15.7:0/52358701 shutdown_connections 2024-06-10T01:30:04.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.200+0000 7fe7eb0eab80 1 -- 172.21.15.7:0/52358701 wait complete. 2024-06-10T01:30:04.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.200+0000 7fe7eb0eab80 20 client.21117 trim_cache size 0 max 16384 2024-06-10T01:30:04.197 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:30:04.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.201+0000 7fe7e0867700 1 -- 172.21.15.7:0/2193918192 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe7c40a2d20 msgr2=0x7fe7c40a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:04.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.201+0000 7fe7e0867700 1 --2- 172.21.15.7:0/2193918192 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe7c40a2d20 0x7fe7c40a51e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0ffc80 tx=0x7fe7d0013040 comp rx=0 tx=0).stop 2024-06-10T01:30:04.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.201+0000 7fe7e0867700 1 -- 172.21.15.7:0/2193918192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7dc100870 msgr2=0x7fe7dc19c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:04.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.201+0000 7fe7e0867700 1 --2- 172.21.15.7:0/2193918192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7dc100870 0x7fe7dc19c750 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe7d40062a0 tx=0x7fe7d4004d00 comp rx=0 tx=0).stop 2024-06-10T01:30:04.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.201+0000 7fe7e0867700 1 -- 172.21.15.7:0/2193918192 shutdown_connections 2024-06-10T01:30:04.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.201+0000 7fe7e0867700 1 --2- 172.21.15.7:0/2193918192 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe7c40a2d20 0x7fe7c40a51e0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.201+0000 7fe7e0867700 1 --2- 172.21.15.7:0/2193918192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7dc108950 0x7fe7dc196910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.201+0000 7fe7e0867700 1 --2- 172.21.15.7:0/2193918192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7dc100870 0x7fe7dc19c750 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.201+0000 7fe7e0867700 1 --2- 172.21.15.7:0/2193918192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7dc101220 0x7fe7dc19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:04.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.201+0000 7fe7e0867700 1 -- 172.21.15.7:0/2193918192 >> 172.21.15.7:0/2193918192 conn(0x7fe7dc077590 msgr2=0x7fe7dc106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:04.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.201+0000 7fe7e0867700 1 -- 172.21.15.7:0/2193918192 shutdown_connections 2024-06-10T01:30:04.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:04.201+0000 7fe7e0867700 1 -- 172.21.15.7:0/2193918192 wait complete. 2024-06-10T01:30:04.566 INFO:tasks.workunit.client.0.smithi007.stdout: 10 253997 22.07 MB/sec execute 521 sec latency 242.359 ms 2024-06-10T01:30:05.367 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:30:05.566 INFO:tasks.workunit.client.0.smithi007.stdout: 10 254540 22.09 MB/sec execute 522 sec latency 119.731 ms 2024-06-10T01:30:05.843 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:05 smithi007 ceph-mon[32398]: pgmap v1156: 129 pgs: 129 active+clean; 653 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 2.1 MiB/s wr, 283 op/s 2024-06-10T01:30:05.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:05 smithi175 ceph-mon[36636]: pgmap v1156: 129 pgs: 129 active+clean; 653 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 2.1 MiB/s wr, 283 op/s 2024-06-10T01:30:06.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:05 smithi078 ceph-mon[36683]: pgmap v1156: 129 pgs: 129 active+clean; 653 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 2.1 MiB/s wr, 283 op/s 2024-06-10T01:30:06.180 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:30:06.566 INFO:tasks.workunit.client.0.smithi007.stdout: 10 255136 22.11 MB/sec execute 523 sec latency 131.127 ms 2024-06-10T01:30:07.566 INFO:tasks.workunit.client.0.smithi007.stdout: 10 255756 22.14 MB/sec execute 524 sec latency 136.418 ms 2024-06-10T01:30:07.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:07 smithi175 ceph-mon[36636]: pgmap v1157: 129 pgs: 129 active+clean; 661 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 3.0 MiB/s wr, 424 op/s 2024-06-10T01:30:08.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:07 smithi007 ceph-mon[32398]: pgmap v1157: 129 pgs: 129 active+clean; 661 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 3.0 MiB/s wr, 424 op/s 2024-06-10T01:30:08.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:07 smithi078 ceph-mon[36683]: pgmap v1157: 129 pgs: 129 active+clean; 661 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 3.0 MiB/s wr, 424 op/s 2024-06-10T01:30:08.566 INFO:tasks.workunit.client.0.smithi007.stdout: 10 256359 22.15 MB/sec execute 525 sec latency 135.137 ms 2024-06-10T01:30:08.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:08 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:30:08.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:08 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:30:09.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:08 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:30:09.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.457+0000 7fad1bb9b700 1 -- 172.21.15.7:0/337874314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad14107430 msgr2=0x7fad14107810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:09.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.457+0000 7fad1bb9b700 1 --2- 172.21.15.7:0/337874314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad14107430 0x7fad14107810 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fad10009990 tx=0x7fad10009ca0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.457+0000 7fad1bb9b700 1 -- 172.21.15.7:0/337874314 shutdown_connections 2024-06-10T01:30:09.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.457+0000 7fad1bb9b700 1 --2- 172.21.15.7:0/337874314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad14101d40 0x7fad1410e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.457+0000 7fad1bb9b700 1 --2- 172.21.15.7:0/337874314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad14107430 0x7fad14107810 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.457+0000 7fad1bb9b700 1 --2- 172.21.15.7:0/337874314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad14101380 0x7fad14101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.457+0000 7fad1bb9b700 1 -- 172.21.15.7:0/337874314 >> 172.21.15.7:0/337874314 conn(0x7fad14078390 msgr2=0x7fad140787a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:09.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.458+0000 7fad1bb9b700 1 -- 172.21.15.7:0/337874314 shutdown_connections 2024-06-10T01:30:09.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.458+0000 7fad1bb9b700 1 -- 172.21.15.7:0/337874314 wait complete. 2024-06-10T01:30:09.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.459+0000 7fad1bb9b700 1 Processor -- start 2024-06-10T01:30:09.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.459+0000 7fad1bb9b700 1 -- start start 2024-06-10T01:30:09.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.459+0000 7fad1bb9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad14101380 0x7fad14199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:09.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.459+0000 7fad1bb9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad14101d40 0x7fad14199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:09.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.459+0000 7fad1bb9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad14107430 0x7fad141a4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:09.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.459+0000 7fad1bb9b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fad141116f0 con 0x7fad14107430 2024-06-10T01:30:09.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.459+0000 7fad1bb9b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fad14111570 con 0x7fad14101d40 2024-06-10T01:30:09.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.459+0000 7fad1bb9b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fad14111870 con 0x7fad14101380 2024-06-10T01:30:09.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.459+0000 7fad1a138700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad14107430 0x7fad141a4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:09.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.459+0000 7fad19136700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad14101d40 0x7fad14199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:09.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.459+0000 7fad19937700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad14101380 0x7fad14199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:09.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.459+0000 7fad19136700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad14101d40 0x7fad14199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:37142/0 (socket says 172.21.15.7:37142) 2024-06-10T01:30:09.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.459+0000 7fad19937700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad14101380 0x7fad14199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:59552/0 (socket says 172.21.15.7:59552) 2024-06-10T01:30:09.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.459+0000 7fad19136700 1 -- 172.21.15.7:0/671993413 learned_addr learned my addr 172.21.15.7:0/671993413 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:09.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.459+0000 7fad1a138700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad14107430 0x7fad141a4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53072/0 (socket says 172.21.15.7:53072) 2024-06-10T01:30:09.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.459+0000 7fad19136700 1 -- 172.21.15.7:0/671993413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad14101380 msgr2=0x7fad14199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:09.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.459+0000 7fad19136700 1 --2- 172.21.15.7:0/671993413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad14101380 0x7fad14199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.460+0000 7fad19136700 1 -- 172.21.15.7:0/671993413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad14107430 msgr2=0x7fad141a4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:09.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.460+0000 7fad19136700 1 --2- 172.21.15.7:0/671993413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad14107430 0x7fad141a4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.460+0000 7fad19136700 1 -- 172.21.15.7:0/671993413 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad141a4f50 con 0x7fad14101d40 2024-06-10T01:30:09.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.460+0000 7fad1a138700 1 --2- 172.21.15.7:0/671993413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad14107430 0x7fad141a4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:30:09.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.460+0000 7fad19136700 1 --2- 172.21.15.7:0/671993413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad14101d40 0x7fad14199ed0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fad0400e9c0 tx=0x7fad0400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:09.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.460+0000 7fad0affd700 1 -- 172.21.15.7:0/671993413 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0400cbd0 con 0x7fad14101d40 2024-06-10T01:30:09.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.460+0000 7fad1bb9b700 1 -- 172.21.15.7:0/671993413 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad141a5180 con 0x7fad14101d40 2024-06-10T01:30:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.460+0000 7fad1bb9b700 1 -- 172.21.15.7:0/671993413 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad141a5690 con 0x7fad14101d40 2024-06-10T01:30:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.460+0000 7fad0affd700 1 -- 172.21.15.7:0/671993413 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad0400ee60 con 0x7fad14101d40 2024-06-10T01:30:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.460+0000 7fad0affd700 1 -- 172.21.15.7:0/671993413 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad040055f0 con 0x7fad14101d40 2024-06-10T01:30:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.463+0000 7fad0affd700 1 -- 172.21.15.7:0/671993413 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fad04004750 con 0x7fad14101d40 2024-06-10T01:30:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.464+0000 7fad0affd700 1 --2- 172.21.15.7:0/671993413 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fad000a2cc0 0x7fad000a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.464+0000 7fad0affd700 1 -- 172.21.15.7:0/671993413 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fad040cf6a0 con 0x7fad14101d40 2024-06-10T01:30:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.464+0000 7fad19937700 1 --2- 172.21.15.7:0/671993413 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fad000a2cc0 0x7fad000a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.465+0000 7fad19937700 1 --2- 172.21.15.7:0/671993413 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fad000a2cc0 0x7fad000a5180 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fad1000b2e0 tx=0x7fad10005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.465+0000 7fad2641eb80 1 Processor -- start 2024-06-10T01:30:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.465+0000 7fad2641eb80 1 -- start start 2024-06-10T01:30:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.466+0000 7fad2641eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555aae900ce0 0x555aae9010c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.466+0000 7fad2641eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555aae901600 0x555aae8faf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.466+0000 7fad2641eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aae8fb470 0x555aae8fb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.466+0000 7fad2641eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555aae84c9b0 con 0x555aae900ce0 2024-06-10T01:30:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.466+0000 7fad2641eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555aae7f48c0 con 0x555aae901600 2024-06-10T01:30:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.466+0000 7fad2641eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555aae848970 con 0x555aae8fb470 2024-06-10T01:30:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.466+0000 7fad19136700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555aae901600 0x555aae8faf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.466+0000 7fad19136700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555aae901600 0x555aae8faf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:37146/0 (socket says 172.21.15.7:37146) 2024-06-10T01:30:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.466+0000 7fad19136700 1 -- 172.21.15.7:0/3347313655 learned_addr learned my addr 172.21.15.7:0/3347313655 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.466+0000 7fad1a138700 1 --2- 172.21.15.7:0/3347313655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aae8fb470 0x555aae8fb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.466+0000 7fad19136700 1 -- 172.21.15.7:0/3347313655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aae8fb470 msgr2=0x555aae8fb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.466+0000 7fad19136700 1 --2- 172.21.15.7:0/3347313655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aae8fb470 0x555aae8fb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.466+0000 7fad19136700 1 -- 172.21.15.7:0/3347313655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555aae900ce0 msgr2=0x555aae9010c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.466+0000 7fad19136700 1 --2- 172.21.15.7:0/3347313655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555aae900ce0 0x555aae9010c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.466+0000 7fad19136700 1 -- 172.21.15.7:0/3347313655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aae7eb0f0 con 0x555aae901600 2024-06-10T01:30:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.467+0000 7fad19136700 1 --2- 172.21.15.7:0/3347313655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555aae901600 0x555aae8faf30 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fad0409bab0 tx=0x7fad0409b550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b77bf3d177bd93c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.467+0000 7facfffff700 1 -- 172.21.15.7:0/3347313655 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0409fed0 con 0x555aae901600 2024-06-10T01:30:09.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.467+0000 7facfffff700 1 -- 172.21.15.7:0/3347313655 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad040b9070 con 0x555aae901600 2024-06-10T01:30:09.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.467+0000 7facfffff700 1 -- 172.21.15.7:0/3347313655 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0409bd80 con 0x555aae901600 2024-06-10T01:30:09.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.469+0000 7fad2641eb80 1 -- 172.21.15.7:0/3347313655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555aae901600 msgr2=0x555aae8faf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:09.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.469+0000 7fad2641eb80 1 --2- 172.21.15.7:0/3347313655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555aae901600 0x555aae8faf30 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fad0409bab0 tx=0x7fad0409b550 comp rx=0 tx=0).stop 2024-06-10T01:30:09.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.469+0000 7fad2641eb80 1 -- 172.21.15.7:0/3347313655 shutdown_connections 2024-06-10T01:30:09.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.469+0000 7fad2641eb80 1 --2- 172.21.15.7:0/3347313655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aae8fb470 0x555aae8fb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.469+0000 7fad2641eb80 1 --2- 172.21.15.7:0/3347313655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555aae900ce0 0x555aae9010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.469+0000 7fad2641eb80 1 --2- 172.21.15.7:0/3347313655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555aae901600 0x555aae8faf30 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.469+0000 7fad2641eb80 1 -- 172.21.15.7:0/3347313655 >> 172.21.15.7:0/3347313655 conn(0x555aae825e60 msgr2=0x555aae8f61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:09.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.469+0000 7fad2641eb80 1 -- 172.21.15.7:0/3347313655 shutdown_connections 2024-06-10T01:30:09.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.469+0000 7fad2641eb80 1 -- 172.21.15.7:0/3347313655 wait complete. 2024-06-10T01:30:09.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.469+0000 7fad2641eb80 1 -- start start 2024-06-10T01:30:09.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.470+0000 7fad2641eb80 1 Processor -- start 2024-06-10T01:30:09.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.470+0000 7fad2641eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555aae8fb470 0x555aae907320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:09.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.470+0000 7fad2641eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aae900ce0 0x555aae904690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:09.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.470+0000 7fad2641eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555aae901600 0x555aae904bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:09.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.470+0000 7fad2641eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555aae908010 con 0x555aae901600 2024-06-10T01:30:09.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.470+0000 7fad2641eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555aae7ecd90 con 0x555aae8fb470 2024-06-10T01:30:09.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.470+0000 7fad2641eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555aae908190 con 0x555aae900ce0 2024-06-10T01:30:09.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.470+0000 7fad19937700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555aae8fb470 0x555aae907320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:09.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.470+0000 7fad19937700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555aae8fb470 0x555aae907320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:37152/0 (socket says 172.21.15.7:37152) 2024-06-10T01:30:09.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.470+0000 7fad19937700 1 -- 172.21.15.7:0/1041254785 learned_addr learned my addr 172.21.15.7:0/1041254785 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:09.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.471+0000 7fad1a138700 1 --2- 172.21.15.7:0/1041254785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555aae901600 0x555aae904bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:09.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.471+0000 7fad2641eb80 10 client.0 _ll_register_callbacks cb 0x555aae7e6ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:30:09.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.471+0000 7fad19937700 1 -- 172.21.15.7:0/1041254785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aae900ce0 msgr2=0x555aae904690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:09.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.471+0000 7fad19937700 1 --2- 172.21.15.7:0/1041254785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aae900ce0 0x555aae904690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.471+0000 7fad19937700 1 -- 172.21.15.7:0/1041254785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555aae901600 msgr2=0x555aae904bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:09.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.471+0000 7fad19937700 1 --2- 172.21.15.7:0/1041254785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555aae901600 0x555aae904bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.471+0000 7fad19937700 1 -- 172.21.15.7:0/1041254785 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aae79ba70 con 0x555aae8fb470 2024-06-10T01:30:09.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.471+0000 7fad1a138700 1 --2- 172.21.15.7:0/1041254785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555aae901600 0x555aae904bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:30:09.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.472+0000 7fad19937700 1 --2- 172.21.15.7:0/1041254785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555aae8fb470 0x555aae907320 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fad10015000 tx=0x7fad10005680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cff69cbccf816b71 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:09.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.472+0000 7facfd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:30:09.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.472+0000 7facfd7fa700 1 -- 172.21.15.7:0/1041254785 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad10003c20 con 0x555aae8fb470 2024-06-10T01:30:09.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.472+0000 7facfd7fa700 1 -- 172.21.15.7:0/1041254785 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fad10003dc0 con 0x555aae8fb470 2024-06-10T01:30:09.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.472+0000 7fad2641eb80 1 -- 172.21.15.7:0/1041254785 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad040b9070 con 0x555aae8fb470 2024-06-10T01:30:09.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.472+0000 7facfd7fa700 1 -- 172.21.15.7:0/1041254785 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fad1000f460 con 0x555aae8fb470 2024-06-10T01:30:09.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.473+0000 7fad2641eb80 10 client.30311 fetch_fsmap learned FSMap version 56 2024-06-10T01:30:09.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.473+0000 7fad2641eb80 1 -- 172.21.15.7:0/1041254785 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x555aae91b160 con 0x555aae8fb470 2024-06-10T01:30:09.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.473+0000 7facfd7fa700 1 -- 172.21.15.7:0/1041254785 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad1000fae0 con 0x555aae8fb470 2024-06-10T01:30:09.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.473+0000 7facfd7fa700 1 -- 172.21.15.7:0/1041254785 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 56) ==== 8350+0+0 (secure 0 0 0) 0x7fad10022940 con 0x555aae8fb470 2024-06-10T01:30:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.473+0000 7fad2641eb80 10 client.30311 fetch_fsmap finished waiting for FSMap version 56 2024-06-10T01:30:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.473+0000 7fad2641eb80 10 client.30311 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:30:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.473+0000 7fad2641eb80 20 client.30311 populate_metadata read hostname 'smithi007' 2024-06-10T01:30:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.473+0000 7fad2641eb80 1 --2- 172.21.15.7:0/1041254785 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555aae91d7a0 0x555aae91fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.473+0000 7fad2641eb80 4 client.30311 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:30:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.473+0000 7fad2641eb80 1 -- 172.21.15.7:0/1041254785 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fad040b9070 con 0x555aae91d7a0 2024-06-10T01:30:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.474+0000 7fad19136700 1 --2- 172.21.15.7:0/1041254785 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555aae91d7a0 0x555aae91fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.476+0000 7fad19136700 1 --2- 172.21.15.7:0/1041254785 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555aae91d7a0 0x555aae91fbb0 crc :-1 s=READY pgs=537 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd7de49eb265491e server_cookie=192984180498d266 in_seq=0 out_seq=0 2024-06-10T01:30:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.476+0000 7facfd7fa700 10 client.30311 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.478+0000 7facfd7fa700 1 -- 172.21.15.7:0/1041254785 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fad040b9070 con 0x555aae91d7a0 2024-06-10T01:30:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.478+0000 7facfd7fa700 10 client.30311 handle_command_reply: tid=0 2024-06-10T01:30:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.478+0000 7fad2641eb80 1 client.30311 shutdown 2024-06-10T01:30:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.478+0000 7fad2641eb80 1 -- 172.21.15.7:0/1041254785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555aae8fb470 msgr2=0x555aae907320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.478+0000 7fad2641eb80 1 --2- 172.21.15.7:0/1041254785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555aae8fb470 0x555aae907320 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fad10015000 tx=0x7fad10005680 comp rx=0 tx=0).stop 2024-06-10T01:30:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.479+0000 7fad2641eb80 1 -- 172.21.15.7:0/1041254785 shutdown_connections 2024-06-10T01:30:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.479+0000 7fad2641eb80 1 --2- 172.21.15.7:0/1041254785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555aae901600 0x555aae904bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.479+0000 7fad2641eb80 1 --2- 172.21.15.7:0/1041254785 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555aae91d7a0 0x555aae91fbb0 crc :-1 s=READY pgs=537 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.479+0000 7fad2641eb80 1 --2- 172.21.15.7:0/1041254785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555aae8fb470 0x555aae907320 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.479+0000 7fad2641eb80 1 --2- 172.21.15.7:0/1041254785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aae900ce0 0x555aae904690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.479+0000 7facfd7fa700 0 client.30311 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.479+0000 7fad2641eb80 1 -- 172.21.15.7:0/1041254785 >> 172.21.15.7:0/1041254785 conn(0x555aae825e60 msgr2=0x555aae8f5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.479+0000 7fad2641eb80 1 -- 172.21.15.7:0/1041254785 shutdown_connections 2024-06-10T01:30:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.479+0000 7fad2641eb80 1 -- 172.21.15.7:0/1041254785 wait complete. 2024-06-10T01:30:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.479+0000 7fad2641eb80 20 client.30311 trim_cache size 0 max 16384 2024-06-10T01:30:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.502+0000 7fad2641eb80 1 Processor -- start 2024-06-10T01:30:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.502+0000 7fad2641eb80 1 -- start start 2024-06-10T01:30:09.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.502+0000 7fad2641eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aae8fb470 0x555aae8faf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:09.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.503+0000 7fad2641eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555aae900ce0 0x555aae903670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:09.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.503+0000 7fad2641eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555aae901600 0x555aae9022b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:09.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.503+0000 7fad2641eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fad141aea30 con 0x555aae900ce0 2024-06-10T01:30:09.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.503+0000 7fad2641eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fad141ac290 con 0x555aae901600 2024-06-10T01:30:09.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.503+0000 7fad2641eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fad141aa0e0 con 0x555aae8fb470 2024-06-10T01:30:09.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.503+0000 7fad19136700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555aae900ce0 0x555aae903670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:09.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.503+0000 7fad19937700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aae8fb470 0x555aae8faf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:09.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.503+0000 7fad19937700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aae8fb470 0x555aae8faf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:59576/0 (socket says 172.21.15.7:59576) 2024-06-10T01:30:09.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.503+0000 7fad19136700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555aae900ce0 0x555aae903670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53092/0 (socket says 172.21.15.7:53092) 2024-06-10T01:30:09.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.503+0000 7fad19937700 1 -- 172.21.15.7:0/1612742686 learned_addr learned my addr 172.21.15.7:0/1612742686 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:09.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.503+0000 7fad19136700 1 -- 172.21.15.7:0/1612742686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aae8fb470 msgr2=0x555aae8faf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:09.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.503+0000 7fad19136700 1 --2- 172.21.15.7:0/1612742686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aae8fb470 0x555aae8faf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.503+0000 7fad19136700 1 -- 172.21.15.7:0/1612742686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555aae901600 msgr2=0x555aae9022b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:30:09.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.503+0000 7fad19136700 1 --2- 172.21.15.7:0/1612742686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555aae901600 0x555aae9022b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.503+0000 7fad19136700 1 -- 172.21.15.7:0/1612742686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aae911310 con 0x555aae900ce0 2024-06-10T01:30:09.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.503+0000 7fad19937700 1 --2- 172.21.15.7:0/1612742686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aae8fb470 0x555aae8faf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:30:09.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.503+0000 7fad19136700 1 --2- 172.21.15.7:0/1612742686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555aae900ce0 0x555aae903670 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fad0400c5b0 tx=0x7fad0409ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662f5058bfcc3424 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:09.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.504+0000 7facff7fe700 1 -- 172.21.15.7:0/1612742686 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04099bc0 con 0x555aae900ce0 2024-06-10T01:30:09.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.504+0000 7facff7fe700 1 -- 172.21.15.7:0/1612742686 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad040b9040 con 0x555aae900ce0 2024-06-10T01:30:09.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.504+0000 7facff7fe700 1 -- 172.21.15.7:0/1612742686 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad040a4300 con 0x555aae900ce0 2024-06-10T01:30:09.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.505+0000 7fad2641eb80 1 -- 172.21.15.7:0/1612742686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555aae900ce0 msgr2=0x555aae903670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:09.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.505+0000 7fad2641eb80 1 --2- 172.21.15.7:0/1612742686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555aae900ce0 0x555aae903670 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fad0400c5b0 tx=0x7fad0409ba20 comp rx=0 tx=0).stop 2024-06-10T01:30:09.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.505+0000 7fad2641eb80 1 -- 172.21.15.7:0/1612742686 shutdown_connections 2024-06-10T01:30:09.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.505+0000 7fad2641eb80 1 --2- 172.21.15.7:0/1612742686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555aae901600 0x555aae9022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.505+0000 7fad2641eb80 1 --2- 172.21.15.7:0/1612742686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aae8fb470 0x555aae8faf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.505+0000 7fad2641eb80 1 --2- 172.21.15.7:0/1612742686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555aae900ce0 0x555aae903670 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.505+0000 7fad2641eb80 1 -- 172.21.15.7:0/1612742686 >> 172.21.15.7:0/1612742686 conn(0x555aae825e60 msgr2=0x555aae8f5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.505+0000 7fad2641eb80 1 -- 172.21.15.7:0/1612742686 shutdown_connections 2024-06-10T01:30:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.505+0000 7fad2641eb80 1 -- 172.21.15.7:0/1612742686 wait complete. 2024-06-10T01:30:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.505+0000 7fad2641eb80 1 -- start start 2024-06-10T01:30:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.505+0000 7fad2641eb80 1 Processor -- start 2024-06-10T01:30:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.506+0000 7fad2641eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aae8fb470 0x555aae90d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.506+0000 7fad2641eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555aae900ce0 0x555aae90dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.506+0000 7fad2641eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555aae901600 0x555aae90a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.506+0000 7fad2641eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fad141aa750 con 0x555aae901600 2024-06-10T01:30:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.506+0000 7fad2641eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fad141abbc0 con 0x555aae900ce0 2024-06-10T01:30:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.506+0000 7fad2641eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fad141aae20 con 0x555aae8fb470 2024-06-10T01:30:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.506+0000 7fad2641eb80 10 client.0 _ll_register_callbacks cb 0x555aae7e6ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:30:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.506+0000 7fad1a138700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555aae901600 0x555aae90a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.506+0000 7fad19937700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aae8fb470 0x555aae90d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:09.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.506+0000 7fad1a138700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555aae901600 0x555aae90a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53100/0 (socket says 172.21.15.7:53100) 2024-06-10T01:30:09.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.506+0000 7fad1a138700 1 -- 172.21.15.7:0/2779356656 learned_addr learned my addr 172.21.15.7:0/2779356656 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:09.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.506+0000 7fad19937700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aae8fb470 0x555aae90d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:59578/0 (socket says 172.21.15.7:59578) 2024-06-10T01:30:09.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.506+0000 7fad19136700 1 --2- 172.21.15.7:0/2779356656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555aae900ce0 0x555aae90dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:09.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.506+0000 7fad1a138700 1 -- 172.21.15.7:0/2779356656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aae8fb470 msgr2=0x555aae90d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:09.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.506+0000 7fad1a138700 1 --2- 172.21.15.7:0/2779356656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aae8fb470 0x555aae90d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.506+0000 7fad1a138700 1 -- 172.21.15.7:0/2779356656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555aae900ce0 msgr2=0x555aae90dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:09.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.506+0000 7fad1a138700 1 --2- 172.21.15.7:0/2779356656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555aae900ce0 0x555aae90dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.506+0000 7fad1a138700 1 -- 172.21.15.7:0/2779356656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aae90aa40 con 0x555aae901600 2024-06-10T01:30:09.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.506+0000 7fad19937700 1 --2- 172.21.15.7:0/2779356656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aae8fb470 0x555aae90d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:30:09.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.506+0000 7fad1a138700 1 --2- 172.21.15.7:0/2779356656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555aae901600 0x555aae90a450 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fad0c014520 tx=0x7fad0c014830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213f7d53197c5865 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:09.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.506+0000 7facfd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:30:09.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.507+0000 7facfd7fa700 1 -- 172.21.15.7:0/2779356656 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c017830 con 0x555aae901600 2024-06-10T01:30:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.507+0000 7fad2641eb80 1 -- 172.21.15.7:0/2779356656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad040b9040 con 0x555aae901600 2024-06-10T01:30:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.507+0000 7facfd7fa700 1 -- 172.21.15.7:0/2779356656 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fad0c0179d0 con 0x555aae901600 2024-06-10T01:30:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.507+0000 7facfd7fa700 1 -- 172.21.15.7:0/2779356656 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fad0c014e90 con 0x555aae901600 2024-06-10T01:30:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.507+0000 7fad2641eb80 10 client.21132 fetch_fsmap learned FSMap version 56 2024-06-10T01:30:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.507+0000 7fad2641eb80 1 -- 172.21.15.7:0/2779356656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x555aae988e10 con 0x555aae901600 2024-06-10T01:30:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.507+0000 7facfd7fa700 1 -- 172.21.15.7:0/2779356656 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad040b9040 con 0x555aae901600 2024-06-10T01:30:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.507+0000 7facfd7fa700 1 -- 172.21.15.7:0/2779356656 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 56) ==== 8350+0+0 (secure 0 0 0) 0x7fad0c016380 con 0x555aae901600 2024-06-10T01:30:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.507+0000 7fad2641eb80 10 client.21132 fetch_fsmap finished waiting for FSMap version 56 2024-06-10T01:30:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.507+0000 7fad2641eb80 10 client.21132 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:30:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.507+0000 7fad2641eb80 20 client.21132 populate_metadata read hostname 'smithi007' 2024-06-10T01:30:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.507+0000 7fad2641eb80 1 --2- 172.21.15.7:0/2779356656 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555aae9895a0 0x555aae98b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:09.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.507+0000 7fad2641eb80 4 client.21132 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.45, "await_for": 45.0}] 2024-06-10T01:30:09.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.507+0000 7fad2641eb80 1 -- 172.21.15.7:0/2779356656 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.45, "await_for": 45.0}) -- 0x555aae859eb0 con 0x555aae9895a0 2024-06-10T01:30:09.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.508+0000 7fad19937700 1 --2- 172.21.15.7:0/2779356656 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555aae9895a0 0x555aae98b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:09.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.508+0000 7fad19937700 1 --2- 172.21.15.7:0/2779356656 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555aae9895a0 0x555aae98b9c0 crc :-1 s=READY pgs=538 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11ed618127838654 server_cookie=69782d28fbdabd48 in_seq=0 out_seq=0 2024-06-10T01:30:09.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:08.508+0000 7facfd7fa700 10 client.21132 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:09.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:09.328+0000 7facfd7fa700 1 -- 172.21.15.7:0/2779356656 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555aae859eb0 con 0x555aae9895a0 2024-06-10T01:30:09.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:09.328+0000 7facfd7fa700 10 client.21132 handle_command_reply: tid=0 2024-06-10T01:30:09.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:09.328+0000 7fad2641eb80 1 client.21132 shutdown 2024-06-10T01:30:09.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:09.329+0000 7fad2641eb80 1 -- 172.21.15.7:0/2779356656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555aae901600 msgr2=0x555aae90a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:09.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:09.329+0000 7fad2641eb80 1 --2- 172.21.15.7:0/2779356656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555aae901600 0x555aae90a450 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fad0c014520 tx=0x7fad0c014830 comp rx=0 tx=0).stop 2024-06-10T01:30:09.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:09.329+0000 7fad2641eb80 1 -- 172.21.15.7:0/2779356656 shutdown_connections 2024-06-10T01:30:09.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:09.329+0000 7fad2641eb80 1 --2- 172.21.15.7:0/2779356656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555aae901600 0x555aae90a450 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:09.329+0000 7fad2641eb80 1 --2- 172.21.15.7:0/2779356656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aae8fb470 0x555aae90d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:09.329+0000 7fad2641eb80 1 --2- 172.21.15.7:0/2779356656 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x555aae9895a0 0x555aae98b9c0 crc :-1 s=READY pgs=538 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:09.329+0000 7fad2641eb80 1 --2- 172.21.15.7:0/2779356656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555aae900ce0 0x555aae90dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:09.329+0000 7facfd7fa700 0 client.21132 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:09.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:09.329+0000 7fad2641eb80 1 -- 172.21.15.7:0/2779356656 >> 172.21.15.7:0/2779356656 conn(0x555aae825e60 msgr2=0x555aae910300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:09.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:09.329+0000 7fad2641eb80 1 -- 172.21.15.7:0/2779356656 shutdown_connections 2024-06-10T01:30:09.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:09.329+0000 7fad2641eb80 1 -- 172.21.15.7:0/2779356656 wait complete. 2024-06-10T01:30:09.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:09.329+0000 7fad2641eb80 20 client.21132 trim_cache size 0 max 16384 2024-06-10T01:30:09.323 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:30:09.323 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 56, 2024-06-10T01:30:09.323 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:30:09.323 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 78, 2024-06-10T01:30:09.323 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:30:09.323 INFO:teuthology.orchestra.run.smithi007.stdout: "11d6bcd3": { 2024-06-10T01:30:09.323 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 78, 2024-06-10T01:30:09.324 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:30:09.324 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:30:09.324 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:30:09.324 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:30:09.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:30:09.324 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:30:09.324 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 142.4, 2024-06-10T01:30:09.324 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:30:09.324 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:30:09.324 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:30:09.324 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:30:09.324 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "QUIESCED", 2024-06-10T01:30:09.324 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:30:09.324 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:30:09.325 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:30:09.325 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:30:09.325 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:30:09.325 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:30:09.325 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:30:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:09.330+0000 7fad1bb9b700 1 -- 172.21.15.7:0/671993413 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fad000a2cc0 msgr2=0x7fad000a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:09.330+0000 7fad1bb9b700 1 --2- 172.21.15.7:0/671993413 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fad000a2cc0 0x7fad000a5180 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fad1000b2e0 tx=0x7fad10005960 comp rx=0 tx=0).stop 2024-06-10T01:30:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:09.330+0000 7fad1bb9b700 1 -- 172.21.15.7:0/671993413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad14101d40 msgr2=0x7fad14199ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:09.330+0000 7fad1bb9b700 1 --2- 172.21.15.7:0/671993413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad14101d40 0x7fad14199ed0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fad0400e9c0 tx=0x7fad0400ecd0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:09.330+0000 7fad1bb9b700 1 -- 172.21.15.7:0/671993413 shutdown_connections 2024-06-10T01:30:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:09.330+0000 7fad1bb9b700 1 --2- 172.21.15.7:0/671993413 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fad000a2cc0 0x7fad000a5180 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:09.330+0000 7fad1bb9b700 1 --2- 172.21.15.7:0/671993413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad14107430 0x7fad141a4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:09.330+0000 7fad1bb9b700 1 --2- 172.21.15.7:0/671993413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad14101380 0x7fad14199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:09.330+0000 7fad1bb9b700 1 --2- 172.21.15.7:0/671993413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad14101d40 0x7fad14199ed0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:09.330+0000 7fad1bb9b700 1 -- 172.21.15.7:0/671993413 >> 172.21.15.7:0/671993413 conn(0x7fad14078390 msgr2=0x7fad140ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:09.330+0000 7fad1bb9b700 1 -- 172.21.15.7:0/671993413 shutdown_connections 2024-06-10T01:30:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:09.330+0000 7fad1bb9b700 1 -- 172.21.15.7:0/671993413 wait complete. 2024-06-10T01:30:09.566 INFO:tasks.workunit.client.0.smithi007.stdout: 10 256380 22.11 MB/sec execute 526 sec latency 984.702 ms 2024-06-10T01:30:09.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:09 smithi175 ceph-mon[36636]: pgmap v1158: 129 pgs: 129 active+clean; 662 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 3.9 MiB/s wr, 530 op/s 2024-06-10T01:30:09.961 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:09 smithi007 ceph-mon[32398]: pgmap v1158: 129 pgs: 129 active+clean; 662 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 3.9 MiB/s wr, 530 op/s 2024-06-10T01:30:10.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:09 smithi078 ceph-mon[36683]: pgmap v1158: 129 pgs: 129 active+clean; 662 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 3.9 MiB/s wr, 530 op/s 2024-06-10T01:30:10.566 INFO:tasks.workunit.client.0.smithi007.stdout: 10 256380 22.07 MB/sec execute 527 sec latency 1984.755 ms 2024-06-10T01:30:11.316 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '11d6bcd3', quiesce took 16.0 seconds. Will release after: 6.449999999999999 2024-06-10T01:30:11.566 INFO:tasks.workunit.client.0.smithi007.stdout: 10 256380 22.02 MB/sec execute 528 sec latency 2984.801 ms 2024-06-10T01:30:11.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.697+0000 7f262f71d700 1 -- 172.21.15.7:0/4233697386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2628103120 msgr2=0x7f2628107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:11.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.697+0000 7f262f71d700 1 --2- 172.21.15.7:0/4233697386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2628103120 0x7f2628107130 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f261c0099c0 tx=0x7f261c009cd0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.697+0000 7f262f71d700 1 -- 172.21.15.7:0/4233697386 shutdown_connections 2024-06-10T01:30:11.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.697+0000 7f262f71d700 1 --2- 172.21.15.7:0/4233697386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2628107860 0x7f262810f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.697+0000 7f262f71d700 1 --2- 172.21.15.7:0/4233697386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2628102770 0x7f2628102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.697+0000 7f262f71d700 1 --2- 172.21.15.7:0/4233697386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2628103120 0x7f2628107130 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.697+0000 7f262f71d700 1 -- 172.21.15.7:0/4233697386 >> 172.21.15.7:0/4233697386 conn(0x7f2628079600 msgr2=0x7f2628079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:11.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.697+0000 7f262f71d700 1 -- 172.21.15.7:0/4233697386 shutdown_connections 2024-06-10T01:30:11.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.697+0000 7f262f71d700 1 -- 172.21.15.7:0/4233697386 wait complete. 2024-06-10T01:30:11.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.698+0000 7f262f71d700 1 Processor -- start 2024-06-10T01:30:11.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.698+0000 7f262f71d700 1 -- start start 2024-06-10T01:30:11.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.698+0000 7f262f71d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2628102770 0x7f262819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:11.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.698+0000 7f262f71d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2628103120 0x7f262819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:11.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.698+0000 7f262f71d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2628107860 0x7f2628198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:11.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.698+0000 7f262f71d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2628112790 con 0x7f2628102770 2024-06-10T01:30:11.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.698+0000 7f262f71d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2628112610 con 0x7f2628107860 2024-06-10T01:30:11.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.698+0000 7f262f71d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2628112910 con 0x7f2628103120 2024-06-10T01:30:11.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.698+0000 7f262d4b9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2628102770 0x7f262819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:11.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.698+0000 7f262d4b9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2628102770 0x7f262819e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53120/0 (socket says 172.21.15.7:53120) 2024-06-10T01:30:11.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.698+0000 7f262d4b9700 1 -- 172.21.15.7:0/2390452985 learned_addr learned my addr 172.21.15.7:0/2390452985 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:11.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.698+0000 7f262dcba700 1 --2- 172.21.15.7:0/2390452985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2628107860 0x7f2628198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:11.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.698+0000 7f262ccb8700 1 --2- 172.21.15.7:0/2390452985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2628103120 0x7f262819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:11.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.698+0000 7f262d4b9700 1 -- 172.21.15.7:0/2390452985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2628103120 msgr2=0x7f262819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:11.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.698+0000 7f262d4b9700 1 --2- 172.21.15.7:0/2390452985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2628103120 0x7f262819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.698+0000 7f262d4b9700 1 -- 172.21.15.7:0/2390452985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2628107860 msgr2=0x7f2628198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:11.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.698+0000 7f262d4b9700 1 --2- 172.21.15.7:0/2390452985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2628107860 0x7f2628198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.698+0000 7f262d4b9700 1 -- 172.21.15.7:0/2390452985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2628199190 con 0x7f2628102770 2024-06-10T01:30:11.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.699+0000 7f262ccb8700 1 --2- 172.21.15.7:0/2390452985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2628103120 0x7f262819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:30:11.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.699+0000 7f262d4b9700 1 --2- 172.21.15.7:0/2390452985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2628102770 0x7f262819e820 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f261800ef30 tx=0x7f261800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:11.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.699+0000 7f26267fc700 1 -- 172.21.15.7:0/2390452985 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26180092e0 con 0x7f2628102770 2024-06-10T01:30:11.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.699+0000 7f26267fc700 1 -- 172.21.15.7:0/2390452985 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2618004bb0 con 0x7f2628102770 2024-06-10T01:30:11.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.699+0000 7f262f71d700 1 -- 172.21.15.7:0/2390452985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2628199420 con 0x7f2628102770 2024-06-10T01:30:11.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.699+0000 7f26267fc700 1 -- 172.21.15.7:0/2390452985 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2618004280 con 0x7f2628102770 2024-06-10T01:30:11.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.699+0000 7f262f71d700 1 -- 172.21.15.7:0/2390452985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26281a56f0 con 0x7f2628102770 2024-06-10T01:30:11.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.701+0000 7f26267fc700 1 -- 172.21.15.7:0/2390452985 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f261801b030 con 0x7f2628102770 2024-06-10T01:30:11.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.702+0000 7f26267fc700 1 --2- 172.21.15.7:0/2390452985 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f26100a2c20 0x7f26100a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:11.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.702+0000 7f262ccb8700 1 --2- 172.21.15.7:0/2390452985 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f26100a2c20 0x7f26100a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:11.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.702+0000 7f26267fc700 1 -- 172.21.15.7:0/2390452985 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f26180cf060 con 0x7f2628102770 2024-06-10T01:30:11.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.702+0000 7f262ccb8700 1 --2- 172.21.15.7:0/2390452985 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f26100a2c20 0x7f26100a50e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f26281020e0 tx=0x7f261c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:11.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.703+0000 7f2639fa0b80 1 Processor -- start 2024-06-10T01:30:11.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.703+0000 7f2639fa0b80 1 -- start start 2024-06-10T01:30:11.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.703+0000 7f2639fa0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620b9ba6f60 0x5620b9ba7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:11.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.703+0000 7f2639fa0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620b9ba7880 0x5620b9ba1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:11.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.703+0000 7f2639fa0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620b9ba17a0 0x5620b9ba1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:11.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.703+0000 7f2639fa0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5620b9a93f60 con 0x5620b9ba17a0 2024-06-10T01:30:11.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.703+0000 7f2639fa0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5620b9af2850 con 0x5620b9ba6f60 2024-06-10T01:30:11.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.703+0000 7f2639fa0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5620b9a5a460 con 0x5620b9ba7880 2024-06-10T01:30:11.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.703+0000 7f262dcba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620b9ba17a0 0x5620b9ba1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:11.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.703+0000 7f262dcba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620b9ba17a0 0x5620b9ba1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53134/0 (socket says 172.21.15.7:53134) 2024-06-10T01:30:11.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.703+0000 7f262dcba700 1 -- 172.21.15.7:0/3079699152 learned_addr learned my addr 172.21.15.7:0/3079699152 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.703+0000 7f262ccb8700 1 --2- 172.21.15.7:0/3079699152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620b9ba7880 0x5620b9ba1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.703+0000 7f262d4b9700 1 --2- 172.21.15.7:0/3079699152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620b9ba6f60 0x5620b9ba7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.704+0000 7f262dcba700 1 -- 172.21.15.7:0/3079699152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620b9ba7880 msgr2=0x5620b9ba1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.704+0000 7f262dcba700 1 --2- 172.21.15.7:0/3079699152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620b9ba7880 0x5620b9ba1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.704+0000 7f262dcba700 1 -- 172.21.15.7:0/3079699152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620b9ba6f60 msgr2=0x5620b9ba7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.704+0000 7f262dcba700 1 --2- 172.21.15.7:0/3079699152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620b9ba6f60 0x5620b9ba7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.704+0000 7f262dcba700 1 -- 172.21.15.7:0/3079699152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620b9a418d0 con 0x5620b9ba17a0 2024-06-10T01:30:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.704+0000 7f262ccb8700 1 --2- 172.21.15.7:0/3079699152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620b9ba7880 0x5620b9ba1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:30:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.704+0000 7f262d4b9700 1 --2- 172.21.15.7:0/3079699152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620b9ba6f60 0x5620b9ba7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:30:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.704+0000 7f262dcba700 1 --2- 172.21.15.7:0/3079699152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620b9ba17a0 0x5620b9ba1bc0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f262000b6c0 tx=0x7f262000b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f69c13a5272f2d70 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.704+0000 7f260f7fe700 1 -- 172.21.15.7:0/3079699152 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620004a30 con 0x5620b9ba17a0 2024-06-10T01:30:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.704+0000 7f260f7fe700 1 -- 172.21.15.7:0/3079699152 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2620015070 con 0x5620b9ba17a0 2024-06-10T01:30:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.704+0000 7f260f7fe700 1 -- 172.21.15.7:0/3079699152 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620003ac0 con 0x5620b9ba17a0 2024-06-10T01:30:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.705+0000 7f2639fa0b80 1 -- 172.21.15.7:0/3079699152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620b9ba17a0 msgr2=0x5620b9ba1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.705+0000 7f2639fa0b80 1 --2- 172.21.15.7:0/3079699152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620b9ba17a0 0x5620b9ba1bc0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f262000b6c0 tx=0x7f262000b9d0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.705+0000 7f2639fa0b80 1 -- 172.21.15.7:0/3079699152 shutdown_connections 2024-06-10T01:30:11.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.705+0000 7f2639fa0b80 1 --2- 172.21.15.7:0/3079699152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620b9ba17a0 0x5620b9ba1bc0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.705+0000 7f2639fa0b80 1 --2- 172.21.15.7:0/3079699152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620b9ba6f60 0x5620b9ba7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.705+0000 7f2639fa0b80 1 --2- 172.21.15.7:0/3079699152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620b9ba7880 0x5620b9ba1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.705+0000 7f2639fa0b80 1 -- 172.21.15.7:0/3079699152 >> 172.21.15.7:0/3079699152 conn(0x5620b9ace080 msgr2=0x5620b9b9c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:11.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.705+0000 7f2639fa0b80 1 -- 172.21.15.7:0/3079699152 shutdown_connections 2024-06-10T01:30:11.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.705+0000 7f2639fa0b80 1 -- 172.21.15.7:0/3079699152 wait complete. 2024-06-10T01:30:11.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.705+0000 7f2639fa0b80 1 -- start start 2024-06-10T01:30:11.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.705+0000 7f2639fa0b80 1 Processor -- start 2024-06-10T01:30:11.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.706+0000 7f2639fa0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620b9ba17a0 0x5620b9bad680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:11.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.706+0000 7f2639fa0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620b9ba6f60 0x5620b9baa980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:11.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.706+0000 7f2639fa0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620b9ba7880 0x5620b9baaec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:11.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.706+0000 7f2639fa0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5620b9a92fc0 con 0x5620b9ba17a0 2024-06-10T01:30:11.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.706+0000 7f2639fa0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5620b9aee810 con 0x5620b9ba6f60 2024-06-10T01:30:11.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.706+0000 7f2639fa0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5620b9bae260 con 0x5620b9ba7880 2024-06-10T01:30:11.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.706+0000 7f262d4b9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620b9ba17a0 0x5620b9bad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:11.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.706+0000 7f2639fa0b80 10 client.0 _ll_register_callbacks cb 0x5620b9a885b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:30:11.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.706+0000 7f262d4b9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620b9ba17a0 0x5620b9bad680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53146/0 (socket says 172.21.15.7:53146) 2024-06-10T01:30:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.706+0000 7f262d4b9700 1 -- 172.21.15.7:0/3745028191 learned_addr learned my addr 172.21.15.7:0/3745028191 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.706+0000 7f262ccb8700 1 --2- 172.21.15.7:0/3745028191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620b9ba6f60 0x5620b9baa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.706+0000 7f262dcba700 1 --2- 172.21.15.7:0/3745028191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620b9ba7880 0x5620b9baaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.706+0000 7f262d4b9700 1 -- 172.21.15.7:0/3745028191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620b9ba7880 msgr2=0x5620b9baaec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.706+0000 7f262d4b9700 1 --2- 172.21.15.7:0/3745028191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620b9ba7880 0x5620b9baaec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.706+0000 7f262d4b9700 1 -- 172.21.15.7:0/3745028191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620b9ba6f60 msgr2=0x5620b9baa980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.706+0000 7f262d4b9700 1 --2- 172.21.15.7:0/3745028191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620b9ba6f60 0x5620b9baa980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.706+0000 7f262d4b9700 1 -- 172.21.15.7:0/3745028191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620b9ab8750 con 0x5620b9ba17a0 2024-06-10T01:30:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.706+0000 7f262dcba700 1 --2- 172.21.15.7:0/3745028191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620b9ba7880 0x5620b9baaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:30:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.706+0000 7f262d4b9700 1 --2- 172.21.15.7:0/3745028191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620b9ba17a0 0x5620b9bad680 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f261809b560 tx=0x7f261801b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a92baf8a42ccab server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.707+0000 7f260cff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:30:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.707+0000 7f260cff9700 1 -- 172.21.15.7:0/3745028191 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26180b3070 con 0x5620b9ba17a0 2024-06-10T01:30:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.707+0000 7f260cff9700 1 -- 172.21.15.7:0/3745028191 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f26180a3d20 con 0x5620b9ba17a0 2024-06-10T01:30:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.707+0000 7f2639fa0b80 1 -- 172.21.15.7:0/3745028191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2620015070 con 0x5620b9ba17a0 2024-06-10T01:30:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.707+0000 7f260cff9700 1 -- 172.21.15.7:0/3745028191 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f261809b5d0 con 0x5620b9ba17a0 2024-06-10T01:30:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.707+0000 7f2639fa0b80 10 client.21144 fetch_fsmap learned FSMap version 56 2024-06-10T01:30:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.707+0000 7f2639fa0b80 1 -- 172.21.15.7:0/3745028191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x5620b9bb8af0 con 0x5620b9ba17a0 2024-06-10T01:30:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.707+0000 7f260cff9700 1 -- 172.21.15.7:0/3745028191 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2620015070 con 0x5620b9ba17a0 2024-06-10T01:30:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.707+0000 7f260cff9700 1 -- 172.21.15.7:0/3745028191 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 56) ==== 8350+0+0 (secure 0 0 0) 0x7f2618004420 con 0x5620b9ba17a0 2024-06-10T01:30:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.707+0000 7f2639fa0b80 10 client.21144 fetch_fsmap finished waiting for FSMap version 56 2024-06-10T01:30:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.707+0000 7f2639fa0b80 10 client.21144 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:30:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.707+0000 7f2639fa0b80 20 client.21144 populate_metadata read hostname 'smithi007' 2024-06-10T01:30:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.707+0000 7f2639fa0b80 1 --2- 172.21.15.7:0/3745028191 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5620b9bbb590 0x5620b9bbd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.707+0000 7f2639fa0b80 4 client.21144 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:30:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.707+0000 7f2639fa0b80 1 -- 172.21.15.7:0/3745028191 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620b9affd50 con 0x5620b9bbb590 2024-06-10T01:30:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.708+0000 7f262ccb8700 1 --2- 172.21.15.7:0/3745028191 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5620b9bbb590 0x5620b9bbd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.708+0000 7f260cff9700 10 client.21144 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:30:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.708+0000 7f262ccb8700 1 --2- 172.21.15.7:0/3745028191 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5620b9bbb590 0x5620b9bbd9e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=21e53c769a34c3d2 server_cookie=d6e6fd9bd28f9005 in_seq=0 out_seq=0 2024-06-10T01:30:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.709+0000 7f260cff9700 1 -- 172.21.15.7:0/3745028191 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5620b9affd50 con 0x5620b9bbb590 2024-06-10T01:30:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.709+0000 7f260cff9700 10 client.21144 handle_command_reply: tid=0 2024-06-10T01:30:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.709+0000 7f2639fa0b80 1 client.21144 shutdown 2024-06-10T01:30:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.709+0000 7f2639fa0b80 1 -- 172.21.15.7:0/3745028191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620b9ba17a0 msgr2=0x5620b9bad680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.709+0000 7f2639fa0b80 1 --2- 172.21.15.7:0/3745028191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620b9ba17a0 0x5620b9bad680 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f261809b560 tx=0x7f261801b600 comp rx=0 tx=0).stop 2024-06-10T01:30:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.709+0000 7f2639fa0b80 1 -- 172.21.15.7:0/3745028191 shutdown_connections 2024-06-10T01:30:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.709+0000 7f2639fa0b80 1 --2- 172.21.15.7:0/3745028191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620b9ba7880 0x5620b9baaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.709+0000 7f2639fa0b80 1 --2- 172.21.15.7:0/3745028191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620b9ba17a0 0x5620b9bad680 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.709+0000 7f2639fa0b80 1 --2- 172.21.15.7:0/3745028191 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5620b9bbb590 0x5620b9bbd9e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.709+0000 7f2639fa0b80 1 --2- 172.21.15.7:0/3745028191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620b9ba6f60 0x5620b9baa980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.709+0000 7f2639fa0b80 1 -- 172.21.15.7:0/3745028191 >> 172.21.15.7:0/3745028191 conn(0x5620b9ace080 msgr2=0x5620b9b9b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.709+0000 7f260cff9700 0 client.21144 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:30:11.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.709+0000 7f2639fa0b80 1 -- 172.21.15.7:0/3745028191 shutdown_connections 2024-06-10T01:30:11.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.709+0000 7f2639fa0b80 1 -- 172.21.15.7:0/3745028191 wait complete. 2024-06-10T01:30:11.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.709+0000 7f2639fa0b80 20 client.21144 trim_cache size 0 max 16384 2024-06-10T01:30:11.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.725+0000 7f2639fa0b80 1 Processor -- start 2024-06-10T01:30:11.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.725+0000 7f2639fa0b80 1 -- start start 2024-06-10T01:30:11.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.725+0000 7f2639fa0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620b9ba17a0 0x5620b9ba1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:11.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.725+0000 7f2639fa0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620b9ba6f60 0x5620b9ba6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:11.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.725+0000 7f2639fa0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620b9ba7880 0x5620b9bad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.725+0000 7f2639fa0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f26281b0bb0 con 0x5620b9ba7880 2024-06-10T01:30:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.725+0000 7f2639fa0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f26281ae410 con 0x5620b9ba17a0 2024-06-10T01:30:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.725+0000 7f2639fa0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f26281ac260 con 0x5620b9ba6f60 2024-06-10T01:30:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.725+0000 7f262dcba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620b9ba7880 0x5620b9bad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.725+0000 7f262dcba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620b9ba7880 0x5620b9bad1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53158/0 (socket says 172.21.15.7:53158) 2024-06-10T01:30:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.725+0000 7f262dcba700 1 -- 172.21.15.7:0/1718743441 learned_addr learned my addr 172.21.15.7:0/1718743441 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.725+0000 7f262d4b9700 1 --2- 172.21.15.7:0/1718743441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620b9ba17a0 0x5620b9ba1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.725+0000 7f262dcba700 1 -- 172.21.15.7:0/1718743441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620b9ba6f60 msgr2=0x5620b9ba6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.725+0000 7f262ccb8700 1 --2- 172.21.15.7:0/1718743441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620b9ba6f60 0x5620b9ba6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.725+0000 7f262dcba700 1 --2- 172.21.15.7:0/1718743441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620b9ba6f60 0x5620b9ba6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.725+0000 7f262dcba700 1 -- 172.21.15.7:0/1718743441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620b9ba17a0 msgr2=0x5620b9ba1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.725+0000 7f262dcba700 1 --2- 172.21.15.7:0/1718743441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620b9ba17a0 0x5620b9ba1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.726+0000 7f262dcba700 1 -- 172.21.15.7:0/1718743441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620b9b98810 con 0x5620b9ba7880 2024-06-10T01:30:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.726+0000 7f262ccb8700 1 --2- 172.21.15.7:0/1718743441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620b9ba6f60 0x5620b9ba6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:30:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.726+0000 7f262dcba700 1 --2- 172.21.15.7:0/1718743441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620b9ba7880 0x5620b9bad1c0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f2620005260 tx=0x7f2620005290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b054b862539e20 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.726+0000 7f260effd700 1 -- 172.21.15.7:0/1718743441 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620027070 con 0x5620b9ba7880 2024-06-10T01:30:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.726+0000 7f260effd700 1 -- 172.21.15.7:0/1718743441 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2620015070 con 0x5620b9ba7880 2024-06-10T01:30:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.726+0000 7f260effd700 1 -- 172.21.15.7:0/1718743441 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620003740 con 0x5620b9ba7880 2024-06-10T01:30:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.726+0000 7f2639fa0b80 1 -- 172.21.15.7:0/1718743441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620b9ba7880 msgr2=0x5620b9bad1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.726+0000 7f2639fa0b80 1 --2- 172.21.15.7:0/1718743441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620b9ba7880 0x5620b9bad1c0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f2620005260 tx=0x7f2620005290 comp rx=0 tx=0).stop 2024-06-10T01:30:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.727+0000 7f2639fa0b80 1 -- 172.21.15.7:0/1718743441 shutdown_connections 2024-06-10T01:30:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.727+0000 7f2639fa0b80 1 --2- 172.21.15.7:0/1718743441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620b9ba7880 0x5620b9bad1c0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.727+0000 7f2639fa0b80 1 --2- 172.21.15.7:0/1718743441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620b9ba17a0 0x5620b9ba1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.727+0000 7f2639fa0b80 1 --2- 172.21.15.7:0/1718743441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620b9ba6f60 0x5620b9ba6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.727+0000 7f2639fa0b80 1 -- 172.21.15.7:0/1718743441 >> 172.21.15.7:0/1718743441 conn(0x5620b9ace080 msgr2=0x5620b9ba82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.727+0000 7f2639fa0b80 1 -- 172.21.15.7:0/1718743441 shutdown_connections 2024-06-10T01:30:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.727+0000 7f2639fa0b80 1 -- 172.21.15.7:0/1718743441 wait complete. 2024-06-10T01:30:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.727+0000 7f2639fa0b80 1 -- start start 2024-06-10T01:30:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.727+0000 7f2639fa0b80 1 Processor -- start 2024-06-10T01:30:11.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.727+0000 7f2639fa0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620b9ba17a0 0x5620b9bb3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.727+0000 7f2639fa0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620b9ba6f60 0x5620b9bb3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.727+0000 7f2639fa0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620b9ba7880 0x5620b9bc22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.728+0000 7f2639fa0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f26281ac8d0 con 0x5620b9ba17a0 2024-06-10T01:30:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.728+0000 7f2639fa0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f26281add40 con 0x5620b9ba7880 2024-06-10T01:30:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.728+0000 7f2639fa0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f26281acfa0 con 0x5620b9ba6f60 2024-06-10T01:30:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.728+0000 7f2639fa0b80 10 client.0 _ll_register_callbacks cb 0x5620b9a885b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:30:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.728+0000 7f262d4b9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620b9ba17a0 0x5620b9bb3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.728+0000 7f262d4b9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620b9ba17a0 0x5620b9bb3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53164/0 (socket says 172.21.15.7:53164) 2024-06-10T01:30:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.728+0000 7f262d4b9700 1 -- 172.21.15.7:0/4238645564 learned_addr learned my addr 172.21.15.7:0/4238645564 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.728+0000 7f262dcba700 1 --2- 172.21.15.7:0/4238645564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620b9ba7880 0x5620b9bc22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.728+0000 7f262ccb8700 1 --2- 172.21.15.7:0/4238645564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620b9ba6f60 0x5620b9bb3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.728+0000 7f262d4b9700 1 -- 172.21.15.7:0/4238645564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620b9ba6f60 msgr2=0x5620b9bb3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.728+0000 7f262d4b9700 1 --2- 172.21.15.7:0/4238645564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620b9ba6f60 0x5620b9bb3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.728+0000 7f262d4b9700 1 -- 172.21.15.7:0/4238645564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620b9ba7880 msgr2=0x5620b9bc22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.728+0000 7f262d4b9700 1 --2- 172.21.15.7:0/4238645564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620b9ba7880 0x5620b9bc22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.728+0000 7f262d4b9700 1 -- 172.21.15.7:0/4238645564 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620b9bb4100 con 0x5620b9ba17a0 2024-06-10T01:30:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.728+0000 7f262dcba700 1 --2- 172.21.15.7:0/4238645564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620b9ba7880 0x5620b9bc22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:30:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.728+0000 7f262d4b9700 1 --2- 172.21.15.7:0/4238645564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620b9ba17a0 0x5620b9bb3520 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f2618097a30 tx=0x7f2618004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4b07b728a53a52 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.728+0000 7f260cff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:30:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.728+0000 7f260cff9700 1 -- 172.21.15.7:0/4238645564 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26180b3040 con 0x5620b9ba17a0 2024-06-10T01:30:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.728+0000 7f260cff9700 1 -- 172.21.15.7:0/4238645564 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f261809aa20 con 0x5620b9ba17a0 2024-06-10T01:30:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.728+0000 7f2639fa0b80 1 -- 172.21.15.7:0/4238645564 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2620015070 con 0x5620b9ba17a0 2024-06-10T01:30:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.728+0000 7f260cff9700 1 -- 172.21.15.7:0/4238645564 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f2618083050 con 0x5620b9ba17a0 2024-06-10T01:30:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.728+0000 7f2639fa0b80 10 client.21150 fetch_fsmap learned FSMap version 56 2024-06-10T01:30:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.729+0000 7f2639fa0b80 1 -- 172.21.15.7:0/4238645564 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x5620b9c257c0 con 0x5620b9ba17a0 2024-06-10T01:30:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.729+0000 7f260cff9700 1 -- 172.21.15.7:0/4238645564 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2620015070 con 0x5620b9ba17a0 2024-06-10T01:30:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.729+0000 7f260cff9700 1 -- 172.21.15.7:0/4238645564 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 56) ==== 8350+0+0 (secure 0 0 0) 0x7f26180a0b10 con 0x5620b9ba17a0 2024-06-10T01:30:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.729+0000 7f2639fa0b80 10 client.21150 fetch_fsmap finished waiting for FSMap version 56 2024-06-10T01:30:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.729+0000 7f2639fa0b80 10 client.21150 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:30:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.729+0000 7f2639fa0b80 20 client.21150 populate_metadata read hostname 'smithi007' 2024-06-10T01:30:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.729+0000 7f2639fa0b80 1 --2- 172.21.15.7:0/4238645564 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5620b9c25ed0 0x5620b9c282f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.729+0000 7f2639fa0b80 4 client.21150 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:30:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.729+0000 7f2639fa0b80 1 -- 172.21.15.7:0/4238645564 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5620b9afa2d0 con 0x5620b9c25ed0 2024-06-10T01:30:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.729+0000 7f262ccb8700 1 --2- 172.21.15.7:0/4238645564 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5620b9c25ed0 0x5620b9c282f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.730+0000 7f262ccb8700 1 --2- 172.21.15.7:0/4238645564 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5620b9c25ed0 0x5620b9c282f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6fa19afe7708c3d1 server_cookie=6cadc820d7a67443 in_seq=0 out_seq=0 2024-06-10T01:30:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.730+0000 7f260cff9700 10 client.21150 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:30:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.730+0000 7f260cff9700 1 -- 172.21.15.7:0/4238645564 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5620b9afa2d0 con 0x5620b9c25ed0 2024-06-10T01:30:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.730+0000 7f260cff9700 10 client.21150 handle_command_reply: tid=0 2024-06-10T01:30:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.730+0000 7f2639fa0b80 1 client.21150 shutdown 2024-06-10T01:30:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.730+0000 7f2639fa0b80 1 -- 172.21.15.7:0/4238645564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620b9ba17a0 msgr2=0x5620b9bb3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.730+0000 7f2639fa0b80 1 --2- 172.21.15.7:0/4238645564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620b9ba17a0 0x5620b9bb3520 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f2618097a30 tx=0x7f2618004f90 comp rx=0 tx=0).stop 2024-06-10T01:30:11.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.730+0000 7f2639fa0b80 1 -- 172.21.15.7:0/4238645564 shutdown_connections 2024-06-10T01:30:11.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.730+0000 7f2639fa0b80 1 --2- 172.21.15.7:0/4238645564 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x5620b9c25ed0 0x5620b9c282f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.730+0000 7f2639fa0b80 1 --2- 172.21.15.7:0/4238645564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5620b9ba7880 0x5620b9bc22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.730+0000 7f2639fa0b80 1 --2- 172.21.15.7:0/4238645564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620b9ba17a0 0x5620b9bb3520 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.730+0000 7f2639fa0b80 1 --2- 172.21.15.7:0/4238645564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5620b9ba6f60 0x5620b9bb3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.730+0000 7f2639fa0b80 1 -- 172.21.15.7:0/4238645564 >> 172.21.15.7:0/4238645564 conn(0x5620b9ace080 msgr2=0x5620b9ba6320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:11.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.730+0000 7f260cff9700 0 client.21150 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:30:11.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.730+0000 7f2639fa0b80 1 -- 172.21.15.7:0/4238645564 shutdown_connections 2024-06-10T01:30:11.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.730+0000 7f2639fa0b80 1 -- 172.21.15.7:0/4238645564 wait complete. 2024-06-10T01:30:11.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.730+0000 7f2639fa0b80 20 client.21150 trim_cache size 0 max 16384 2024-06-10T01:30:11.730 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:30:11.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.731+0000 7f262f71d700 1 -- 172.21.15.7:0/2390452985 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f26100a2c20 msgr2=0x7f26100a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:11.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.731+0000 7f262f71d700 1 --2- 172.21.15.7:0/2390452985 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f26100a2c20 0x7f26100a50e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f26281020e0 tx=0x7f261c005fd0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.731+0000 7f262f71d700 1 -- 172.21.15.7:0/2390452985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2628102770 msgr2=0x7f262819e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:11.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.731+0000 7f262f71d700 1 --2- 172.21.15.7:0/2390452985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2628102770 0x7f262819e820 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f261800ef30 tx=0x7f261800c3a0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.731+0000 7f262f71d700 1 -- 172.21.15.7:0/2390452985 shutdown_connections 2024-06-10T01:30:11.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.731+0000 7f262f71d700 1 --2- 172.21.15.7:0/2390452985 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f26100a2c20 0x7f26100a50e0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.731+0000 7f262f71d700 1 --2- 172.21.15.7:0/2390452985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2628107860 0x7f2628198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.731+0000 7f262f71d700 1 --2- 172.21.15.7:0/2390452985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2628102770 0x7f262819e820 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.731+0000 7f262f71d700 1 --2- 172.21.15.7:0/2390452985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2628103120 0x7f262819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:11.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.731+0000 7f262f71d700 1 -- 172.21.15.7:0/2390452985 >> 172.21.15.7:0/2390452985 conn(0x7f2628079600 msgr2=0x7f26281007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:11.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.731+0000 7f262f71d700 1 -- 172.21.15.7:0/2390452985 shutdown_connections 2024-06-10T01:30:11.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:11.731+0000 7f262f71d700 1 -- 172.21.15.7:0/2390452985 wait complete. 2024-06-10T01:30:11.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:11 smithi175 ceph-mon[36636]: pgmap v1159: 129 pgs: 129 active+clean; 666 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.2 MiB/s wr, 495 op/s 2024-06-10T01:30:12.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:11 smithi007 ceph-mon[32398]: pgmap v1159: 129 pgs: 129 active+clean; 666 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.2 MiB/s wr, 495 op/s 2024-06-10T01:30:12.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:11 smithi078 ceph-mon[36683]: pgmap v1159: 129 pgs: 129 active+clean; 666 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.2 MiB/s wr, 495 op/s 2024-06-10T01:30:12.436 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:30:12.566 INFO:tasks.workunit.client.0.smithi007.stdout: 10 256380 21.98 MB/sec execute 529 sec latency 3984.844 ms 2024-06-10T01:30:13.028 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:30:13.566 INFO:tasks.workunit.client.0.smithi007.stdout: 10 256380 21.94 MB/sec execute 530 sec latency 4984.888 ms 2024-06-10T01:30:13.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:13 smithi007 ceph-mon[32398]: pgmap v1160: 129 pgs: 129 active+clean; 708 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.9 MiB/s wr, 604 op/s 2024-06-10T01:30:13.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:13 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:30:13.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:13 smithi175 ceph-mon[36636]: pgmap v1160: 129 pgs: 129 active+clean; 708 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.9 MiB/s wr, 604 op/s 2024-06-10T01:30:13.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:13 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:30:14.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:13 smithi078 ceph-mon[36683]: pgmap v1160: 129 pgs: 129 active+clean; 708 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.9 MiB/s wr, 604 op/s 2024-06-10T01:30:14.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:13 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:30:14.566 INFO:tasks.workunit.client.0.smithi007.stdout: 10 256380 21.90 MB/sec execute 531 sec latency 5984.991 ms 2024-06-10T01:30:15.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.201+0000 7f80f4da8700 1 -- 172.21.15.7:0/3239880312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80f0106ba0 msgr2=0x7f80f0100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:15.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.201+0000 7f80f4da8700 1 --2- 172.21.15.7:0/3239880312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80f0106ba0 0x7f80f0100e50 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009990 tx=0x7f80e4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.201+0000 7f80f4da8700 1 -- 172.21.15.7:0/3239880312 shutdown_connections 2024-06-10T01:30:15.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.201+0000 7f80f4da8700 1 --2- 172.21.15.7:0/3239880312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80f0101420 0x7f80f01018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.201+0000 7f80f4da8700 1 --2- 172.21.15.7:0/3239880312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80f01061f0 0x7f80f01065d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.201+0000 7f80f4da8700 1 --2- 172.21.15.7:0/3239880312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80f0106ba0 0x7f80f0100e50 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.201+0000 7f80f4da8700 1 -- 172.21.15.7:0/3239880312 >> 172.21.15.7:0/3239880312 conn(0x7f80f00766b0 msgr2=0x7f80f0076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:15.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.201+0000 7f80f4da8700 1 -- 172.21.15.7:0/3239880312 shutdown_connections 2024-06-10T01:30:15.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.201+0000 7f80f4da8700 1 -- 172.21.15.7:0/3239880312 wait complete. 2024-06-10T01:30:15.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.202+0000 7f80f4da8700 1 Processor -- start 2024-06-10T01:30:15.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.202+0000 7f80f4da8700 1 -- start start 2024-06-10T01:30:15.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.202+0000 7f80f4da8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80f0101420 0x7f80f019f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:15.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.202+0000 7f80f4da8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80f01061f0 0x7f80f019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:15.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.202+0000 7f80f4da8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80f0106ba0 0x7f80f01a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:15.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.202+0000 7f80f4da8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f80f0118ef0 con 0x7f80f0106ba0 2024-06-10T01:30:15.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.202+0000 7f80f4da8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f80f0118d70 con 0x7f80f01061f0 2024-06-10T01:30:15.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.202+0000 7f80f4da8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f80f0119070 con 0x7f80f0101420 2024-06-10T01:30:15.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.202+0000 7f80edd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80f01061f0 0x7f80f019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:15.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.202+0000 7f80edd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80f01061f0 0x7f80f019f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:59982/0 (socket says 172.21.15.7:59982) 2024-06-10T01:30:15.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.202+0000 7f80edd9b700 1 -- 172.21.15.7:0/1573821033 learned_addr learned my addr 172.21.15.7:0/1573821033 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:15.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.203+0000 7f80eed9d700 1 --2- 172.21.15.7:0/1573821033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80f0106ba0 0x7f80f01a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:15.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.203+0000 7f80ee59c700 1 --2- 172.21.15.7:0/1573821033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80f0101420 0x7f80f019f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:15.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.203+0000 7f80edd9b700 1 -- 172.21.15.7:0/1573821033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80f0101420 msgr2=0x7f80f019f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:15.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.203+0000 7f80edd9b700 1 --2- 172.21.15.7:0/1573821033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80f0101420 0x7f80f019f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.203+0000 7f80edd9b700 1 -- 172.21.15.7:0/1573821033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80f0106ba0 msgr2=0x7f80f01a3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:15.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.203+0000 7f80edd9b700 1 --2- 172.21.15.7:0/1573821033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80f0106ba0 0x7f80f01a3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.203+0000 7f80edd9b700 1 -- 172.21.15.7:0/1573821033 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f01a42b0 con 0x7f80f01061f0 2024-06-10T01:30:15.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.203+0000 7f80eed9d700 1 --2- 172.21.15.7:0/1573821033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80f0106ba0 0x7f80f01a3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:30:15.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.203+0000 7f80edd9b700 1 --2- 172.21.15.7:0/1573821033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80f01061f0 0x7f80f019f7e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f80e4000c00 tx=0x7f80e40047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:15.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.203+0000 7f80df7fe700 1 -- 172.21.15.7:0/1573821033 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e4027070 con 0x7f80f01061f0 2024-06-10T01:30:15.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.203+0000 7f80f4da8700 1 -- 172.21.15.7:0/1573821033 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f01a4540 con 0x7f80f01061f0 2024-06-10T01:30:15.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.203+0000 7f80df7fe700 1 -- 172.21.15.7:0/1573821033 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80e4003e40 con 0x7f80f01061f0 2024-06-10T01:30:15.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.203+0000 7f80df7fe700 1 -- 172.21.15.7:0/1573821033 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e4022d10 con 0x7f80f01061f0 2024-06-10T01:30:15.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.203+0000 7f80f4da8700 1 -- 172.21.15.7:0/1573821033 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f01abe20 con 0x7f80f01061f0 2024-06-10T01:30:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.206+0000 7f80df7fe700 1 -- 172.21.15.7:0/1573821033 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f80e402a070 con 0x7f80f01061f0 2024-06-10T01:30:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.207+0000 7f80ff62bb80 1 Processor -- start 2024-06-10T01:30:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.207+0000 7f80ff62bb80 1 -- start start 2024-06-10T01:30:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.207+0000 7f80ff62bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d537a40f60 0x55d537a41340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.207+0000 7f80ff62bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d537a41880 0x55d537a3b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.207+0000 7f80ff62bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d537a3b7a0 0x55d537a3bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.207+0000 7f80ff62bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d53792df60 con 0x55d537a41880 2024-06-10T01:30:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.207+0000 7f80ff62bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d53798c850 con 0x55d537a40f60 2024-06-10T01:30:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.207+0000 7f80ff62bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d5378f4460 con 0x55d537a3b7a0 2024-06-10T01:30:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.207+0000 7f80df7fe700 1 --2- 172.21.15.7:0/1573821033 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f80d80a2c20 0x7f80d80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.208+0000 7f80ee59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d537a41880 0x55d537a3b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.208+0000 7f80ee59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d537a41880 0x55d537a3b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43134/0 (socket says 172.21.15.7:43134) 2024-06-10T01:30:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.208+0000 7f80ee59c700 1 -- 172.21.15.7:0/3504658456 learned_addr learned my addr 172.21.15.7:0/3504658456 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.208+0000 7f80eed9d700 1 --2- 172.21.15.7:0/3504658456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d537a40f60 0x55d537a41340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.208+0000 7f80df7fe700 1 -- 172.21.15.7:0/1573821033 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f80e40d01f0 con 0x7f80f01061f0 2024-06-10T01:30:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.208+0000 7f80edd9b700 1 --2- 172.21.15.7:0/3504658456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d537a3b7a0 0x55d537a3bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.208+0000 7f80eed9d700 1 --2- 172.21.15.7:0/1573821033 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f80d80a2c20 0x7f80d80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.208+0000 7f80ee59c700 1 -- 172.21.15.7:0/3504658456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d537a3b7a0 msgr2=0x55d537a3bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.208+0000 7f80ee59c700 1 --2- 172.21.15.7:0/3504658456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d537a3b7a0 0x55d537a3bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.208+0000 7f80ee59c700 1 -- 172.21.15.7:0/3504658456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d537a40f60 msgr2=0x55d537a41340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.208+0000 7f80ee59c700 1 --2- 172.21.15.7:0/3504658456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d537a40f60 0x55d537a41340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.208+0000 7f80ee59c700 1 -- 172.21.15.7:0/3504658456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5378db8d0 con 0x55d537a41880 2024-06-10T01:30:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.208+0000 7f80eed9d700 1 --2- 172.21.15.7:0/3504658456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d537a40f60 0x55d537a41340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:30:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.208+0000 7f80ee59c700 1 --2- 172.21.15.7:0/3504658456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d537a41880 0x55d537a3b260 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f80e000bd40 tx=0x7f80e00106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7f5b168670f1dd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.208+0000 7f80eed9d700 1 --2- 172.21.15.7:0/1573821033 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f80d80a2c20 0x7f80d80a50e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f80e8007fd0 tx=0x7f80e8009100 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.208+0000 7f80dcff9700 1 -- 172.21.15.7:0/3504658456 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e0010e10 con 0x55d537a41880 2024-06-10T01:30:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.209+0000 7f80dcff9700 1 -- 172.21.15.7:0/3504658456 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80e0004510 con 0x55d537a41880 2024-06-10T01:30:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.209+0000 7f80dcff9700 1 -- 172.21.15.7:0/3504658456 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e0005240 con 0x55d537a41880 2024-06-10T01:30:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.209+0000 7f80ff62bb80 1 -- 172.21.15.7:0/3504658456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d537a41880 msgr2=0x55d537a3b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.209+0000 7f80ff62bb80 1 --2- 172.21.15.7:0/3504658456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d537a41880 0x55d537a3b260 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f80e000bd40 tx=0x7f80e00106b0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.209+0000 7f80ff62bb80 1 -- 172.21.15.7:0/3504658456 shutdown_connections 2024-06-10T01:30:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.209+0000 7f80ff62bb80 1 --2- 172.21.15.7:0/3504658456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d537a3b7a0 0x55d537a3bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.209+0000 7f80ff62bb80 1 --2- 172.21.15.7:0/3504658456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d537a40f60 0x55d537a41340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.209+0000 7f80ff62bb80 1 --2- 172.21.15.7:0/3504658456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d537a41880 0x55d537a3b260 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.209+0000 7f80ff62bb80 1 -- 172.21.15.7:0/3504658456 >> 172.21.15.7:0/3504658456 conn(0x55d537968080 msgr2=0x55d537a36220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.209+0000 7f80ff62bb80 1 -- 172.21.15.7:0/3504658456 shutdown_connections 2024-06-10T01:30:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.210+0000 7f80ff62bb80 1 -- 172.21.15.7:0/3504658456 wait complete. 2024-06-10T01:30:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.210+0000 7f80ff62bb80 1 -- start start 2024-06-10T01:30:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.210+0000 7f80ff62bb80 1 Processor -- start 2024-06-10T01:30:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.210+0000 7f80ff62bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d537a3b7a0 0x55d537a47580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.210+0000 7f80ff62bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d537a40f60 0x55d537a44880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.210+0000 7f80ff62bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d537a41880 0x55d537a44dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.210+0000 7f80ff62bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d53792cfc0 con 0x55d537a3b7a0 2024-06-10T01:30:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.210+0000 7f80ff62bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d537988810 con 0x55d537a40f60 2024-06-10T01:30:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.210+0000 7f80ff62bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d537a48260 con 0x55d537a41880 2024-06-10T01:30:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.210+0000 7f80ff62bb80 10 client.0 _ll_register_callbacks cb 0x55d5379225b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:30:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.210+0000 7f80ee59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d537a3b7a0 0x55d537a47580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.210+0000 7f80ee59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d537a3b7a0 0x55d537a47580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43150/0 (socket says 172.21.15.7:43150) 2024-06-10T01:30:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.210+0000 7f80ee59c700 1 -- 172.21.15.7:0/2473992685 learned_addr learned my addr 172.21.15.7:0/2473992685 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.211+0000 7f80edd9b700 1 --2- 172.21.15.7:0/2473992685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d537a40f60 0x55d537a44880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.211+0000 7f80eed9d700 1 --2- 172.21.15.7:0/2473992685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d537a41880 0x55d537a44dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.211+0000 7f80ee59c700 1 -- 172.21.15.7:0/2473992685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d537a41880 msgr2=0x55d537a44dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.211+0000 7f80ee59c700 1 --2- 172.21.15.7:0/2473992685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d537a41880 0x55d537a44dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.211+0000 7f80ee59c700 1 -- 172.21.15.7:0/2473992685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d537a40f60 msgr2=0x55d537a44880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.211+0000 7f80ee59c700 1 --2- 172.21.15.7:0/2473992685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d537a40f60 0x55d537a44880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.211+0000 7f80ee59c700 1 -- 172.21.15.7:0/2473992685 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d537952750 con 0x55d537a3b7a0 2024-06-10T01:30:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.211+0000 7f80eed9d700 1 --2- 172.21.15.7:0/2473992685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d537a41880 0x55d537a44dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:30:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.211+0000 7f80edd9b700 1 --2- 172.21.15.7:0/2473992685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d537a40f60 0x55d537a44880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:30:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.211+0000 7f80ee59c700 1 --2- 172.21.15.7:0/2473992685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d537a3b7a0 0x55d537a47580 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f80e0005910 tx=0x7f80e0004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3b15d68544d7f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.211+0000 7f80d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:30:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.211+0000 7f80d5ffb700 1 -- 172.21.15.7:0/2473992685 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e0004ea0 con 0x55d537a3b7a0 2024-06-10T01:30:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.211+0000 7f80d5ffb700 1 -- 172.21.15.7:0/2473992685 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f80e0005040 con 0x55d537a3b7a0 2024-06-10T01:30:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.211+0000 7f80ff62bb80 1 -- 172.21.15.7:0/2473992685 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80e0004510 con 0x55d537a3b7a0 2024-06-10T01:30:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.211+0000 7f80d5ffb700 1 -- 172.21.15.7:0/2473992685 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f80e0005240 con 0x55d537a3b7a0 2024-06-10T01:30:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.211+0000 7f80ff62bb80 10 client.21162 fetch_fsmap learned FSMap version 57 2024-06-10T01:30:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.211+0000 7f80ff62bb80 1 -- 172.21.15.7:0/2473992685 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55d537a52970 con 0x55d537a3b7a0 2024-06-10T01:30:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.211+0000 7f80d5ffb700 1 -- 172.21.15.7:0/2473992685 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80e0004510 con 0x55d537a3b7a0 2024-06-10T01:30:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.212+0000 7f80d5ffb700 1 -- 172.21.15.7:0/2473992685 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7f80e0011c20 con 0x55d537a3b7a0 2024-06-10T01:30:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.212+0000 7f80ff62bb80 10 client.21162 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:30:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.212+0000 7f80ff62bb80 10 client.21162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:30:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.212+0000 7f80ff62bb80 20 client.21162 populate_metadata read hostname 'smithi007' 2024-06-10T01:30:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.212+0000 7f80ff62bb80 1 --2- 172.21.15.7:0/2473992685 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d537a54e40 0x55d537a57230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.212+0000 7f80ff62bb80 4 client.21162 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:30:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.212+0000 7f80ff62bb80 1 -- 172.21.15.7:0/2473992685 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d537999d50 con 0x55d537a54e40 2024-06-10T01:30:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.212+0000 7f80edd9b700 1 --2- 172.21.15.7:0/2473992685 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d537a54e40 0x55d537a57230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.212+0000 7f80d5ffb700 10 client.21162 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.212+0000 7f80edd9b700 1 --2- 172.21.15.7:0/2473992685 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d537a54e40 0x55d537a57230 crc :-1 s=READY pgs=539 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47e771ba381a5e1e server_cookie=f017dfd19e7332d in_seq=0 out_seq=0 2024-06-10T01:30:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.213+0000 7f80d5ffb700 1 -- 172.21.15.7:0/2473992685 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d537999d50 con 0x55d537a54e40 2024-06-10T01:30:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.213+0000 7f80d5ffb700 10 client.21162 handle_command_reply: tid=0 2024-06-10T01:30:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.213+0000 7f80ff62bb80 1 client.21162 shutdown 2024-06-10T01:30:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.214+0000 7f80ff62bb80 1 -- 172.21.15.7:0/2473992685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d537a3b7a0 msgr2=0x55d537a47580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.214+0000 7f80ff62bb80 1 --2- 172.21.15.7:0/2473992685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d537a3b7a0 0x55d537a47580 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f80e0005910 tx=0x7f80e0004c30 comp rx=0 tx=0).stop 2024-06-10T01:30:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.214+0000 7f80ff62bb80 1 -- 172.21.15.7:0/2473992685 shutdown_connections 2024-06-10T01:30:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.214+0000 7f80ff62bb80 1 --2- 172.21.15.7:0/2473992685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d537a41880 0x55d537a44dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.214+0000 7f80ff62bb80 1 --2- 172.21.15.7:0/2473992685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d537a3b7a0 0x55d537a47580 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.214+0000 7f80ff62bb80 1 --2- 172.21.15.7:0/2473992685 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d537a54e40 0x55d537a57230 crc :-1 s=READY pgs=539 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.214+0000 7f80ff62bb80 1 --2- 172.21.15.7:0/2473992685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d537a40f60 0x55d537a44880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.214+0000 7f80d5ffb700 0 client.21162 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.214+0000 7f80ff62bb80 1 -- 172.21.15.7:0/2473992685 >> 172.21.15.7:0/2473992685 conn(0x55d537968080 msgr2=0x55d537a35700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.214+0000 7f80ff62bb80 1 -- 172.21.15.7:0/2473992685 shutdown_connections 2024-06-10T01:30:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.214+0000 7f80ff62bb80 1 -- 172.21.15.7:0/2473992685 wait complete. 2024-06-10T01:30:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.214+0000 7f80ff62bb80 20 client.21162 trim_cache size 0 max 16384 2024-06-10T01:30:15.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.229+0000 7f80ff62bb80 1 Processor -- start 2024-06-10T01:30:15.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.229+0000 7f80ff62bb80 1 -- start start 2024-06-10T01:30:15.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.229+0000 7f80ff62bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d537a3b7a0 0x55d537a3b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:15.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.229+0000 7f80ff62bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d537a40f60 0x55d537a42430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:15.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.229+0000 7f80ff62bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d537a41880 0x55d537a35e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:15.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.229+0000 7f80ff62bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01b7080 con 0x55d537a41880 2024-06-10T01:30:15.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.229+0000 7f80ff62bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01b48b0 con 0x55d537a40f60 2024-06-10T01:30:15.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.229+0000 7f80ff62bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01b2700 con 0x55d537a3b7a0 2024-06-10T01:30:15.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.229+0000 7f80eed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d537a41880 0x55d537a35e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:15.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.229+0000 7f80eed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d537a41880 0x55d537a35e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43152/0 (socket says 172.21.15.7:43152) 2024-06-10T01:30:15.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.229+0000 7f80eed9d700 1 -- 172.21.15.7:0/3416140500 learned_addr learned my addr 172.21.15.7:0/3416140500 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:15.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.230+0000 7f80ee59c700 1 --2- 172.21.15.7:0/3416140500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d537a3b7a0 0x55d537a3b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:15.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.230+0000 7f80eed9d700 1 -- 172.21.15.7:0/3416140500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d537a3b7a0 msgr2=0x55d537a3b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:15.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.230+0000 7f80eed9d700 1 --2- 172.21.15.7:0/3416140500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d537a3b7a0 0x55d537a3b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.230+0000 7f80eed9d700 1 -- 172.21.15.7:0/3416140500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d537a40f60 msgr2=0x55d537a42430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:15.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.230+0000 7f80eed9d700 1 --2- 172.21.15.7:0/3416140500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d537a40f60 0x55d537a42430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.230+0000 7f80eed9d700 1 -- 172.21.15.7:0/3416140500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d537a41680 con 0x55d537a41880 2024-06-10T01:30:15.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.230+0000 7f80eed9d700 1 --2- 172.21.15.7:0/3416140500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d537a41880 0x55d537a35e80 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f80e8019ac0 tx=0x7f80e800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2dda0bc7d53a03d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:15.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.230+0000 7f80d7fff700 1 -- 172.21.15.7:0/3416140500 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e8004030 con 0x55d537a41880 2024-06-10T01:30:15.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.230+0000 7f80d7fff700 1 -- 172.21.15.7:0/3416140500 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80e80046b0 con 0x55d537a41880 2024-06-10T01:30:15.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.230+0000 7f80d7fff700 1 -- 172.21.15.7:0/3416140500 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e801baa0 con 0x55d537a41880 2024-06-10T01:30:15.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.231+0000 7f80ff62bb80 1 -- 172.21.15.7:0/3416140500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d537a41880 msgr2=0x55d537a35e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:15.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.231+0000 7f80ff62bb80 1 --2- 172.21.15.7:0/3416140500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d537a41880 0x55d537a35e80 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f80e8019ac0 tx=0x7f80e800b040 comp rx=0 tx=0).stop 2024-06-10T01:30:15.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.231+0000 7f80ff62bb80 1 -- 172.21.15.7:0/3416140500 shutdown_connections 2024-06-10T01:30:15.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.231+0000 7f80ff62bb80 1 --2- 172.21.15.7:0/3416140500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d537a41880 0x55d537a35e80 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.231+0000 7f80ff62bb80 1 --2- 172.21.15.7:0/3416140500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d537a3b7a0 0x55d537a3b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.231+0000 7f80ff62bb80 1 --2- 172.21.15.7:0/3416140500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d537a40f60 0x55d537a42430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.231+0000 7f80ff62bb80 1 -- 172.21.15.7:0/3416140500 >> 172.21.15.7:0/3416140500 conn(0x55d537968080 msgr2=0x55d537a41d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:15.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.231+0000 7f80ff62bb80 1 -- 172.21.15.7:0/3416140500 shutdown_connections 2024-06-10T01:30:15.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.231+0000 7f80ff62bb80 1 -- 172.21.15.7:0/3416140500 wait complete. 2024-06-10T01:30:15.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.231+0000 7f80ff62bb80 1 -- start start 2024-06-10T01:30:15.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.231+0000 7f80ff62bb80 1 Processor -- start 2024-06-10T01:30:15.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.231+0000 7f80ff62bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d537a3b7a0 0x55d537a4d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:15.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.231+0000 7f80ff62bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d537a40f60 0x55d537a4da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:15.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.231+0000 7f80ff62bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d537a41880 0x55d537a5b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:15.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.231+0000 7f80ff62bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01b2d70 con 0x55d537a41880 2024-06-10T01:30:15.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.231+0000 7f80ff62bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01b41e0 con 0x55d537a40f60 2024-06-10T01:30:15.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.231+0000 7f80ff62bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01b3440 con 0x55d537a3b7a0 2024-06-10T01:30:15.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.232+0000 7f80ff62bb80 10 client.0 _ll_register_callbacks cb 0x55d5379225b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:30:15.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.232+0000 7f80eed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d537a41880 0x55d537a5b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:15.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.232+0000 7f80eed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d537a41880 0x55d537a5b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43154/0 (socket says 172.21.15.7:43154) 2024-06-10T01:30:15.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.232+0000 7f80eed9d700 1 -- 172.21.15.7:0/3505487831 learned_addr learned my addr 172.21.15.7:0/3505487831 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:15.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.232+0000 7f80edd9b700 1 --2- 172.21.15.7:0/3505487831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d537a40f60 0x55d537a4da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:15.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.232+0000 7f80ee59c700 1 --2- 172.21.15.7:0/3505487831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d537a3b7a0 0x55d537a4d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:15.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.232+0000 7f80eed9d700 1 -- 172.21.15.7:0/3505487831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d537a3b7a0 msgr2=0x55d537a4d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:15.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.232+0000 7f80eed9d700 1 --2- 172.21.15.7:0/3505487831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d537a3b7a0 0x55d537a4d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.232+0000 7f80eed9d700 1 -- 172.21.15.7:0/3505487831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d537a40f60 msgr2=0x55d537a4da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:15.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.232+0000 7f80eed9d700 1 --2- 172.21.15.7:0/3505487831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d537a40f60 0x55d537a4da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.232+0000 7f80eed9d700 1 -- 172.21.15.7:0/3505487831 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d537a5bf30 con 0x55d537a41880 2024-06-10T01:30:15.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.232+0000 7f80eed9d700 1 --2- 172.21.15.7:0/3505487831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d537a41880 0x55d537a5b9f0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f80e8009d50 tx=0x7f80e8007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae35b74c5855e180 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:15.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.232+0000 7f80d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:30:15.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.232+0000 7f80d5ffb700 1 -- 172.21.15.7:0/3505487831 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e802b040 con 0x55d537a41880 2024-06-10T01:30:15.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.232+0000 7f80d5ffb700 1 -- 172.21.15.7:0/3505487831 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f80e8004ad0 con 0x55d537a41880 2024-06-10T01:30:15.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.232+0000 7f80ff62bb80 1 -- 172.21.15.7:0/3505487831 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80e80046b0 con 0x55d537a41880 2024-06-10T01:30:15.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.233+0000 7f80d5ffb700 1 -- 172.21.15.7:0/3505487831 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f80e803c050 con 0x55d537a41880 2024-06-10T01:30:15.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.233+0000 7f80ff62bb80 10 client.21168 fetch_fsmap learned FSMap version 57 2024-06-10T01:30:15.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.233+0000 7f80ff62bb80 1 -- 172.21.15.7:0/3505487831 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55d537abf0f0 con 0x55d537a41880 2024-06-10T01:30:15.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.233+0000 7f80d5ffb700 1 -- 172.21.15.7:0/3505487831 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80e80046b0 con 0x55d537a41880 2024-06-10T01:30:15.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.233+0000 7f80d5ffb700 1 -- 172.21.15.7:0/3505487831 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7f80e8037030 con 0x55d537a41880 2024-06-10T01:30:15.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.233+0000 7f80ff62bb80 10 client.21168 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:30:15.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.233+0000 7f80ff62bb80 10 client.21168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:30:15.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.233+0000 7f80ff62bb80 20 client.21168 populate_metadata read hostname 'smithi007' 2024-06-10T01:30:15.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.233+0000 7f80ff62bb80 1 --2- 172.21.15.7:0/3505487831 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d537abf7f0 0x55d537ac1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:15.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.233+0000 7f80ff62bb80 4 client.21168 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:30:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.233+0000 7f80ff62bb80 1 -- 172.21.15.7:0/3505487831 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d5379942d0 con 0x55d537abf7f0 2024-06-10T01:30:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.233+0000 7f80ee59c700 1 --2- 172.21.15.7:0/3505487831 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d537abf7f0 0x55d537ac1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.234+0000 7f80ee59c700 1 --2- 172.21.15.7:0/3505487831 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d537abf7f0 0x55d537ac1c10 crc :-1 s=READY pgs=540 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65a7ff305a270464 server_cookie=a47744c012aad564 in_seq=0 out_seq=0 2024-06-10T01:30:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.234+0000 7f80d5ffb700 10 client.21168 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.234+0000 7f80d5ffb700 1 -- 172.21.15.7:0/3505487831 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d5379942d0 con 0x55d537abf7f0 2024-06-10T01:30:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.234+0000 7f80d5ffb700 10 client.21168 handle_command_reply: tid=0 2024-06-10T01:30:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.234+0000 7f80ff62bb80 1 client.21168 shutdown 2024-06-10T01:30:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.234+0000 7f80ff62bb80 1 -- 172.21.15.7:0/3505487831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d537a41880 msgr2=0x55d537a5b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.234+0000 7f80ff62bb80 1 --2- 172.21.15.7:0/3505487831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d537a41880 0x55d537a5b9f0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f80e8009d50 tx=0x7f80e8007500 comp rx=0 tx=0).stop 2024-06-10T01:30:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.234+0000 7f80ff62bb80 1 -- 172.21.15.7:0/3505487831 shutdown_connections 2024-06-10T01:30:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.234+0000 7f80ff62bb80 1 --2- 172.21.15.7:0/3505487831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d537a41880 0x55d537a5b9f0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.234+0000 7f80ff62bb80 1 --2- 172.21.15.7:0/3505487831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d537a3b7a0 0x55d537a4d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.234+0000 7f80ff62bb80 1 --2- 172.21.15.7:0/3505487831 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55d537abf7f0 0x55d537ac1c10 crc :-1 s=READY pgs=540 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.234+0000 7f80ff62bb80 1 --2- 172.21.15.7:0/3505487831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d537a40f60 0x55d537a4da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.234+0000 7f80ff62bb80 1 -- 172.21.15.7:0/3505487831 >> 172.21.15.7:0/3505487831 conn(0x55d537968080 msgr2=0x55d537a460b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:15.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.234+0000 7f80d5ffb700 0 client.21168 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:15.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.234+0000 7f80ff62bb80 1 -- 172.21.15.7:0/3505487831 shutdown_connections 2024-06-10T01:30:15.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.234+0000 7f80ff62bb80 1 -- 172.21.15.7:0/3505487831 wait complete. 2024-06-10T01:30:15.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.235+0000 7f80ff62bb80 20 client.21168 trim_cache size 0 max 16384 2024-06-10T01:30:15.232 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:30:15.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.235+0000 7f80f4da8700 1 -- 172.21.15.7:0/1573821033 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f80d80a2c20 msgr2=0x7f80d80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:15.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.235+0000 7f80f4da8700 1 --2- 172.21.15.7:0/1573821033 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f80d80a2c20 0x7f80d80a50e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f80e8007fd0 tx=0x7f80e8009100 comp rx=0 tx=0).stop 2024-06-10T01:30:15.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.235+0000 7f80f4da8700 1 -- 172.21.15.7:0/1573821033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80f01061f0 msgr2=0x7f80f019f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:15.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.235+0000 7f80f4da8700 1 --2- 172.21.15.7:0/1573821033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80f01061f0 0x7f80f019f7e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f80e4000c00 tx=0x7f80e40047b0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.235+0000 7f80f4da8700 1 -- 172.21.15.7:0/1573821033 shutdown_connections 2024-06-10T01:30:15.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.235+0000 7f80f4da8700 1 --2- 172.21.15.7:0/1573821033 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f80d80a2c20 0x7f80d80a50e0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.235+0000 7f80f4da8700 1 --2- 172.21.15.7:0/1573821033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80f0106ba0 0x7f80f01a3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.235+0000 7f80f4da8700 1 --2- 172.21.15.7:0/1573821033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80f0101420 0x7f80f019f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.235+0000 7f80f4da8700 1 --2- 172.21.15.7:0/1573821033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80f01061f0 0x7f80f019f7e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:15.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.235+0000 7f80f4da8700 1 -- 172.21.15.7:0/1573821033 >> 172.21.15.7:0/1573821033 conn(0x7f80f00766b0 msgr2=0x7f80f0101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:15.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.235+0000 7f80f4da8700 1 -- 172.21.15.7:0/1573821033 shutdown_connections 2024-06-10T01:30:15.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:15.235+0000 7f80f4da8700 1 -- 172.21.15.7:0/1573821033 wait complete. 2024-06-10T01:30:15.566 INFO:tasks.workunit.client.0.smithi007.stdout: 10 256380 21.86 MB/sec execute 532 sec latency 6985.029 ms 2024-06-10T01:30:15.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:15 smithi175 ceph-mon[36636]: pgmap v1161: 129 pgs: 129 active+clean; 711 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.7 MiB/s wr, 526 op/s 2024-06-10T01:30:15.983 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:15 smithi007 ceph-mon[32398]: pgmap v1161: 129 pgs: 129 active+clean; 711 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.7 MiB/s wr, 526 op/s 2024-06-10T01:30:16.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:15 smithi078 ceph-mon[36683]: pgmap v1161: 129 pgs: 129 active+clean; 711 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.7 MiB/s wr, 526 op/s 2024-06-10T01:30:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.548+0000 7f7313af5640 1 -- 172.21.15.7:0/39827733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f730c101b00 msgr2=0x7f730c101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.548+0000 7f7313af5640 1 --2- 172.21.15.7:0/39827733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f730c101b00 0x7f730c101f80 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f72fc009900 tx=0x7f72fc02a410 comp rx=0 tx=0).stop 2024-06-10T01:30:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.548+0000 7f7313af5640 1 -- 172.21.15.7:0/39827733 shutdown_connections 2024-06-10T01:30:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.548+0000 7f7313af5640 1 --2- 172.21.15.7:0/39827733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f730c1024c0 0x7f730c108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.548+0000 7f7313af5640 1 --2- 172.21.15.7:0/39827733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f730c101b00 0x7f730c101f80 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.548+0000 7f7313af5640 1 --2- 172.21.15.7:0/39827733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f730c100910 0x7f730c100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.548+0000 7f7313af5640 1 -- 172.21.15.7:0/39827733 >> 172.21.15.7:0/39827733 conn(0x7f730c0fbe80 msgr2=0x7f730c0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.548+0000 7f7313af5640 1 -- 172.21.15.7:0/39827733 shutdown_connections 2024-06-10T01:30:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.548+0000 7f7313af5640 1 -- 172.21.15.7:0/39827733 wait complete. 2024-06-10T01:30:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.548+0000 7f7313af5640 1 Processor -- start 2024-06-10T01:30:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.548+0000 7f7313af5640 1 -- start start 2024-06-10T01:30:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.549+0000 7f7313af5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f730c100910 0x7f730c197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.549+0000 7f7313af5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f730c101b00 0x7f730c198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.549+0000 7f7313af5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f730c1024c0 0x7f730c19f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.549+0000 7f7313af5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f730c10c590 con 0x7f730c100910 2024-06-10T01:30:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.549+0000 7f7313af5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f730c10c410 con 0x7f730c1024c0 2024-06-10T01:30:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.549+0000 7f7313af5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f730c10c710 con 0x7f730c101b00 2024-06-10T01:30:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.549+0000 7f7311891640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f730c100910 0x7f730c197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.549+0000 7f7311891640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f730c100910 0x7f730c197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43178/0 (socket says 172.21.15.7:43178) 2024-06-10T01:30:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.549+0000 7f7311891640 1 -- 172.21.15.7:0/960544209 learned_addr learned my addr 172.21.15.7:0/960544209 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.549+0000 7f7311090640 1 --2- 172.21.15.7:0/960544209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f730c101b00 0x7f730c198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.549+0000 7f7312092640 1 --2- 172.21.15.7:0/960544209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f730c1024c0 0x7f730c19f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.549+0000 7f7311891640 1 -- 172.21.15.7:0/960544209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f730c101b00 msgr2=0x7f730c198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.549+0000 7f7311891640 1 --2- 172.21.15.7:0/960544209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f730c101b00 0x7f730c198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.549+0000 7f7311891640 1 -- 172.21.15.7:0/960544209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f730c1024c0 msgr2=0x7f730c19f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.549+0000 7f7311891640 1 --2- 172.21.15.7:0/960544209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f730c1024c0 0x7f730c19f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.549+0000 7f7311891640 1 -- 172.21.15.7:0/960544209 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f730c19f960 con 0x7f730c100910 2024-06-10T01:30:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.549+0000 7f7311891640 1 --2- 172.21.15.7:0/960544209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f730c100910 0x7f730c197d50 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f730800e9c0 tx=0x7f730800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.550+0000 7f7302ffd640 1 -- 172.21.15.7:0/960544209 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730800cd80 con 0x7f730c100910 2024-06-10T01:30:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.550+0000 7f7302ffd640 1 -- 172.21.15.7:0/960544209 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73080035f0 con 0x7f730c100910 2024-06-10T01:30:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.550+0000 7f7313af5640 1 -- 172.21.15.7:0/960544209 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f730c19fc50 con 0x7f730c100910 2024-06-10T01:30:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.550+0000 7f7302ffd640 1 -- 172.21.15.7:0/960544209 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308010690 con 0x7f730c100910 2024-06-10T01:30:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.550+0000 7f7313af5640 1 -- 172.21.15.7:0/960544209 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f730c1a01a0 con 0x7f730c100910 2024-06-10T01:30:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.552+0000 7f7302ffd640 1 -- 172.21.15.7:0/960544209 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f73080037a0 con 0x7f730c100910 2024-06-10T01:30:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.552+0000 7f7316800740 1 Processor -- start 2024-06-10T01:30:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.553+0000 7f7316800740 1 -- start start 2024-06-10T01:30:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.553+0000 7f7316800740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7acd26010 0x55c7acd26430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.553+0000 7f7316800740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7acd26a00 0x55c7acd2b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.553+0000 7f7316800740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7acd2bb70 0x55c7acd2e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.553+0000 7f7316800740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c7acc894f0 con 0x55c7acd26010 2024-06-10T01:30:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.553+0000 7f7316800740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c7acc27870 con 0x55c7acd26a00 2024-06-10T01:30:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.553+0000 7f7316800740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c7acc854b0 con 0x55c7acd2bb70 2024-06-10T01:30:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.553+0000 7f7312092640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7acd26010 0x55c7acd26430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.553+0000 7f7312092640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7acd26010 0x55c7acd26430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43190/0 (socket says 172.21.15.7:43190) 2024-06-10T01:30:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.553+0000 7f7312092640 1 -- 172.21.15.7:0/955441028 learned_addr learned my addr 172.21.15.7:0/955441028 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.553+0000 7f7302ffd640 1 --2- 172.21.15.7:0/960544209 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f72f80a2c00 0x7f72f80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.553+0000 7f7311891640 1 --2- 172.21.15.7:0/955441028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7acd26a00 0x55c7acd2b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.553+0000 7f7311090640 1 --2- 172.21.15.7:0/955441028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7acd2bb70 0x55c7acd2e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.554+0000 7f7312092640 1 -- 172.21.15.7:0/955441028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7acd2bb70 msgr2=0x55c7acd2e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.554+0000 7f7312092640 1 --2- 172.21.15.7:0/955441028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7acd2bb70 0x55c7acd2e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.554+0000 7f7312092640 1 -- 172.21.15.7:0/955441028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7acd26a00 msgr2=0x55c7acd2b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.554+0000 7f7312092640 1 --2- 172.21.15.7:0/955441028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7acd26a00 0x55c7acd2b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.554+0000 7f7312092640 1 -- 172.21.15.7:0/955441028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7acc439b0 con 0x55c7acd26010 2024-06-10T01:30:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.554+0000 7f7311090640 1 --2- 172.21.15.7:0/955441028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7acd2bb70 0x55c7acd2e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:30:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.554+0000 7f7302ffd640 1 -- 172.21.15.7:0/960544209 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f73080cf750 con 0x7f730c100910 2024-06-10T01:30:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.554+0000 7f7312092640 1 --2- 172.21.15.7:0/960544209 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f72f80a2c00 0x7f72f80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.554+0000 7f7312092640 1 --2- 172.21.15.7:0/955441028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7acd26010 0x55c7acd26430 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f7304010510 tx=0x7f7304009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab399c40f99f2a9d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.554+0000 7f72f7fff640 1 -- 172.21.15.7:0/955441028 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730401d400 con 0x55c7acd26010 2024-06-10T01:30:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.554+0000 7f72f7fff640 1 -- 172.21.15.7:0/955441028 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73040134c0 con 0x55c7acd26010 2024-06-10T01:30:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.554+0000 7f72f7fff640 1 -- 172.21.15.7:0/955441028 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7304011920 con 0x55c7acd26010 2024-06-10T01:30:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.554+0000 7f7312092640 1 --2- 172.21.15.7:0/960544209 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f72f80a2c00 0x7f72f80a50c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7304021000 tx=0x7f730401e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.555+0000 7f7316800740 1 -- 172.21.15.7:0/955441028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7acd26010 msgr2=0x55c7acd26430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.555+0000 7f7316800740 1 --2- 172.21.15.7:0/955441028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7acd26010 0x55c7acd26430 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f7304010510 tx=0x7f7304009f90 comp rx=0 tx=0).stop 2024-06-10T01:30:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.555+0000 7f7316800740 1 -- 172.21.15.7:0/955441028 shutdown_connections 2024-06-10T01:30:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.555+0000 7f7316800740 1 --2- 172.21.15.7:0/955441028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7acd2bb70 0x55c7acd2e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.555+0000 7f7316800740 1 --2- 172.21.15.7:0/955441028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7acd26a00 0x55c7acd2b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.555+0000 7f7316800740 1 --2- 172.21.15.7:0/955441028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7acd26010 0x55c7acd26430 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.555+0000 7f7316800740 1 -- 172.21.15.7:0/955441028 >> 172.21.15.7:0/955441028 conn(0x55c7acd21680 msgr2=0x55c7acd21a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.555+0000 7f7316800740 1 -- 172.21.15.7:0/955441028 shutdown_connections 2024-06-10T01:30:16.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.555+0000 7f7316800740 1 -- 172.21.15.7:0/955441028 wait complete. 2024-06-10T01:30:16.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.555+0000 7f7316800740 1 -- start start 2024-06-10T01:30:16.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.556+0000 7f7316800740 1 Processor -- start 2024-06-10T01:30:16.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.556+0000 7f7316800740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7acd26010 0x55c7acd30490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:16.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.556+0000 7f7316800740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7acd26a00 0x55c7acd309d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:16.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.556+0000 7f7316800740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7acd2bb70 0x55c7acd37a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:16.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.556+0000 7f7316800740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c7acd2eb20 con 0x55c7acd26a00 2024-06-10T01:30:16.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.556+0000 7f7316800740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c7acc12ae0 con 0x55c7acd2bb70 2024-06-10T01:30:16.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.556+0000 7f7316800740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c7acd2eca0 con 0x55c7acd26010 2024-06-10T01:30:16.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.556+0000 7f7316800740 10 client.0 _ll_register_callbacks cb 0x55c7acc4b9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:30:16.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.556+0000 7f7311090640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7acd26a00 0x55c7acd309d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:16.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.556+0000 7f7311090640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7acd26a00 0x55c7acd309d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43200/0 (socket says 172.21.15.7:43200) 2024-06-10T01:30:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.556+0000 7f7311090640 1 -- 172.21.15.7:0/4149515106 learned_addr learned my addr 172.21.15.7:0/4149515106 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.556+0000 7f7312092640 1 --2- 172.21.15.7:0/4149515106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7acd2bb70 0x55c7acd37a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.556+0000 7f7311891640 1 --2- 172.21.15.7:0/4149515106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7acd26010 0x55c7acd30490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.556+0000 7f7311090640 1 -- 172.21.15.7:0/4149515106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7acd26010 msgr2=0x55c7acd30490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.556+0000 7f7311090640 1 --2- 172.21.15.7:0/4149515106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7acd26010 0x55c7acd30490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.556+0000 7f7311090640 1 -- 172.21.15.7:0/4149515106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7acd2bb70 msgr2=0x55c7acd37a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.556+0000 7f7311090640 1 --2- 172.21.15.7:0/4149515106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7acd2bb70 0x55c7acd37a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.556+0000 7f7311090640 1 -- 172.21.15.7:0/4149515106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7ac99d210 con 0x55c7acd26a00 2024-06-10T01:30:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.556+0000 7f7311891640 1 --2- 172.21.15.7:0/4149515106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7acd26010 0x55c7acd30490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:30:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.556+0000 7f7312092640 1 --2- 172.21.15.7:0/4149515106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7acd2bb70 0x55c7acd37a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:30:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.557+0000 7f7311090640 1 --2- 172.21.15.7:0/4149515106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7acd26a00 0x55c7acd309d0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f72fc031610 tx=0x7f72fc009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=effd338b70951121 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.557+0000 7f72f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:30:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.557+0000 7f72f57fa640 1 -- 172.21.15.7:0/4149515106 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc040070 con 0x55c7acd26a00 2024-06-10T01:30:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.557+0000 7f72f57fa640 1 -- 172.21.15.7:0/4149515106 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f72fc004240 con 0x55c7acd26a00 2024-06-10T01:30:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.557+0000 7f7316800740 1 -- 172.21.15.7:0/4149515106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73040134c0 con 0x55c7acd26a00 2024-06-10T01:30:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.557+0000 7f72f57fa640 1 -- 172.21.15.7:0/4149515106 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f72fc04d050 con 0x55c7acd26a00 2024-06-10T01:30:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.557+0000 7f7316800740 10 client.21180 fetch_fsmap learned FSMap version 57 2024-06-10T01:30:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.557+0000 7f7316800740 1 -- 172.21.15.7:0/4149515106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55c7aca69620 con 0x55c7acd26a00 2024-06-10T01:30:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.557+0000 7f72f57fa640 1 -- 172.21.15.7:0/4149515106 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73040134c0 con 0x55c7acd26a00 2024-06-10T01:30:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.557+0000 7f72f57fa640 1 -- 172.21.15.7:0/4149515106 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7f72fc004af0 con 0x55c7acd26a00 2024-06-10T01:30:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.558+0000 7f7316800740 10 client.21180 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:30:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.558+0000 7f7316800740 10 client.21180 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:30:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.558+0000 7f7316800740 20 client.21180 populate_metadata read hostname 'smithi007' 2024-06-10T01:30:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.558+0000 7f7316800740 1 --2- 172.21.15.7:0/4149515106 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c7acd3c2d0 0x55c7acd3e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.558+0000 7f7316800740 4 client.21180 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:30:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.558+0000 7f7316800740 1 -- 172.21.15.7:0/4149515106 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7acc9cf10 con 0x55c7acd3c2d0 2024-06-10T01:30:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.558+0000 7f7311891640 1 --2- 172.21.15.7:0/4149515106 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c7acd3c2d0 0x55c7acd3e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.558+0000 7f7311891640 1 --2- 172.21.15.7:0/4149515106 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c7acd3c2d0 0x55c7acd3e710 crc :-1 s=READY pgs=541 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ade39529391fd868 server_cookie=efae6e4c0ae08df5 in_seq=0 out_seq=0 2024-06-10T01:30:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.558+0000 7f72f57fa640 10 client.21180 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.559+0000 7f72f57fa640 1 -- 172.21.15.7:0/4149515106 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c7acc9cf10 con 0x55c7acd3c2d0 2024-06-10T01:30:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.559+0000 7f72f57fa640 10 client.21180 handle_command_reply: tid=0 2024-06-10T01:30:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.559+0000 7f7316800740 1 client.21180 shutdown 2024-06-10T01:30:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.559+0000 7f7316800740 1 -- 172.21.15.7:0/4149515106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7acd26a00 msgr2=0x55c7acd309d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.559+0000 7f7316800740 1 --2- 172.21.15.7:0/4149515106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7acd26a00 0x55c7acd309d0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f72fc031610 tx=0x7f72fc009650 comp rx=0 tx=0).stop 2024-06-10T01:30:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.559+0000 7f7316800740 1 -- 172.21.15.7:0/4149515106 shutdown_connections 2024-06-10T01:30:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.559+0000 7f7316800740 1 --2- 172.21.15.7:0/4149515106 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c7acd3c2d0 0x55c7acd3e710 crc :-1 s=READY pgs=541 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.559+0000 7f7316800740 1 --2- 172.21.15.7:0/4149515106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7acd2bb70 0x55c7acd37a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.559+0000 7f7316800740 1 --2- 172.21.15.7:0/4149515106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7acd26a00 0x55c7acd309d0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.559+0000 7f72f57fa640 0 client.21180 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.559+0000 7f7316800740 1 --2- 172.21.15.7:0/4149515106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7acd26010 0x55c7acd30490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.559+0000 7f7316800740 1 -- 172.21.15.7:0/4149515106 >> 172.21.15.7:0/4149515106 conn(0x55c7acd21680 msgr2=0x55c7acd207e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.559+0000 7f7316800740 1 -- 172.21.15.7:0/4149515106 shutdown_connections 2024-06-10T01:30:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.559+0000 7f7316800740 1 -- 172.21.15.7:0/4149515106 wait complete. 2024-06-10T01:30:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.560+0000 7f7316800740 20 client.21180 trim_cache size 0 max 16384 2024-06-10T01:30:16.566 INFO:tasks.workunit.client.0.smithi007.stdout: 10 256380 21.82 MB/sec execute 533 sec latency 7985.070 ms 2024-06-10T01:30:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.575+0000 7f7316800740 1 Processor -- start 2024-06-10T01:30:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.575+0000 7f7316800740 1 -- start start 2024-06-10T01:30:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.576+0000 7f7316800740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7acd26010 0x55c7acd1c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.576+0000 7f7316800740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7acd26a00 0x55c7acd379d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.576+0000 7f7316800740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7acd2bb70 0x55c7acd27e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.576+0000 7f7316800740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f730c1ace90 con 0x55c7acd26010 2024-06-10T01:30:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.576+0000 7f7316800740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f730c1aa700 con 0x55c7acd26a00 2024-06-10T01:30:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.576+0000 7f7316800740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f730c1a8550 con 0x55c7acd2bb70 2024-06-10T01:30:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.576+0000 7f7311891640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7acd26010 0x55c7acd1c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.576+0000 7f7311891640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7acd26010 0x55c7acd1c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43216/0 (socket says 172.21.15.7:43216) 2024-06-10T01:30:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.576+0000 7f7311891640 1 -- 172.21.15.7:0/3435686439 learned_addr learned my addr 172.21.15.7:0/3435686439 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.576+0000 7f7312092640 1 --2- 172.21.15.7:0/3435686439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7acd2bb70 0x55c7acd27e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.576+0000 7f7311090640 1 --2- 172.21.15.7:0/3435686439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7acd26a00 0x55c7acd379d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.576+0000 7f7311891640 1 -- 172.21.15.7:0/3435686439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7acd2bb70 msgr2=0x55c7acd27e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.576+0000 7f7311891640 1 --2- 172.21.15.7:0/3435686439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7acd2bb70 0x55c7acd27e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.576+0000 7f7311891640 1 -- 172.21.15.7:0/3435686439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7acd26a00 msgr2=0x55c7acd379d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.576+0000 7f7311891640 1 --2- 172.21.15.7:0/3435686439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7acd26a00 0x55c7acd379d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.576+0000 7f7311891640 1 -- 172.21.15.7:0/3435686439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7acd27290 con 0x55c7acd26010 2024-06-10T01:30:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.576+0000 7f7311090640 1 --2- 172.21.15.7:0/3435686439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7acd26a00 0x55c7acd379d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:30:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.576+0000 7f7312092640 1 --2- 172.21.15.7:0/3435686439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7acd2bb70 0x55c7acd27e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:30:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.576+0000 7f7311891640 1 --2- 172.21.15.7:0/3435686439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7acd26010 0x55c7acd1c840 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f730809bba0 tx=0x7f73080c67a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f5f5a709d04129 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.576+0000 7f72f77fe640 1 -- 172.21.15.7:0/3435686439 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730809c310 con 0x55c7acd26010 2024-06-10T01:30:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.576+0000 7f72f77fe640 1 -- 172.21.15.7:0/3435686439 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f730809bc10 con 0x55c7acd26010 2024-06-10T01:30:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.576+0000 7f72f77fe640 1 -- 172.21.15.7:0/3435686439 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73080981f0 con 0x55c7acd26010 2024-06-10T01:30:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.577+0000 7f7316800740 1 -- 172.21.15.7:0/3435686439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7acd26010 msgr2=0x55c7acd1c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.577+0000 7f7316800740 1 --2- 172.21.15.7:0/3435686439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7acd26010 0x55c7acd1c840 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f730809bba0 tx=0x7f73080c67a0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.577+0000 7f7316800740 1 -- 172.21.15.7:0/3435686439 shutdown_connections 2024-06-10T01:30:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.577+0000 7f7316800740 1 --2- 172.21.15.7:0/3435686439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7acd2bb70 0x55c7acd27e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.577+0000 7f7316800740 1 --2- 172.21.15.7:0/3435686439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7acd26a00 0x55c7acd379d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.577+0000 7f7316800740 1 --2- 172.21.15.7:0/3435686439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7acd26010 0x55c7acd1c840 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.577+0000 7f7316800740 1 -- 172.21.15.7:0/3435686439 >> 172.21.15.7:0/3435686439 conn(0x55c7acd21680 msgr2=0x55c7acd21a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.577+0000 7f7316800740 1 -- 172.21.15.7:0/3435686439 shutdown_connections 2024-06-10T01:30:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.577+0000 7f7316800740 1 -- 172.21.15.7:0/3435686439 wait complete. 2024-06-10T01:30:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.577+0000 7f7316800740 1 -- start start 2024-06-10T01:30:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.577+0000 7f7316800740 1 Processor -- start 2024-06-10T01:30:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.578+0000 7f7316800740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7acd26010 0x55c7acd37580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.578+0000 7f7316800740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7acd26a00 0x55c7acd34090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.578+0000 7f7316800740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7acd2bb70 0x55c7acd345d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.578+0000 7f7316800740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f730c1a8bc0 con 0x55c7acd2bb70 2024-06-10T01:30:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.578+0000 7f7316800740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f730c1aa030 con 0x55c7acd26a00 2024-06-10T01:30:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.578+0000 7f7316800740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f730c1a9290 con 0x55c7acd26010 2024-06-10T01:30:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.578+0000 7f7316800740 10 client.0 _ll_register_callbacks cb 0x7f72fc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:30:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.578+0000 7f7312092640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7acd2bb70 0x55c7acd345d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.578+0000 7f7312092640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7acd2bb70 0x55c7acd345d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43222/0 (socket says 172.21.15.7:43222) 2024-06-10T01:30:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.578+0000 7f7312092640 1 -- 172.21.15.7:0/2985094247 learned_addr learned my addr 172.21.15.7:0/2985094247 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.578+0000 7f7311891640 1 --2- 172.21.15.7:0/2985094247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7acd26010 0x55c7acd37580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.578+0000 7f7311090640 1 --2- 172.21.15.7:0/2985094247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7acd26a00 0x55c7acd34090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.578+0000 7f7312092640 1 -- 172.21.15.7:0/2985094247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7acd26010 msgr2=0x55c7acd37580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.578+0000 7f7312092640 1 --2- 172.21.15.7:0/2985094247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7acd26010 0x55c7acd37580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.578+0000 7f7312092640 1 -- 172.21.15.7:0/2985094247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7acd26a00 msgr2=0x55c7acd34090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.578+0000 7f7312092640 1 --2- 172.21.15.7:0/2985094247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7acd26a00 0x55c7acd34090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.578+0000 7f7312092640 1 -- 172.21.15.7:0/2985094247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7acd37bb0 con 0x55c7acd2bb70 2024-06-10T01:30:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.579+0000 7f7312092640 1 --2- 172.21.15.7:0/2985094247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7acd2bb70 0x55c7acd345d0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f7304009d20 tx=0x7f7304009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959d90dfe0b0e5c3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.579+0000 7f72f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:30:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.579+0000 7f72f57fa640 1 -- 172.21.15.7:0/2985094247 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7304038060 con 0x55c7acd2bb70 2024-06-10T01:30:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.579+0000 7f72f57fa640 1 -- 172.21.15.7:0/2985094247 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f7304003b80 con 0x55c7acd2bb70 2024-06-10T01:30:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.579+0000 7f7316800740 1 -- 172.21.15.7:0/2985094247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f730809bc10 con 0x55c7acd2bb70 2024-06-10T01:30:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.579+0000 7f72f57fa640 1 -- 172.21.15.7:0/2985094247 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f730403f050 con 0x55c7acd2bb70 2024-06-10T01:30:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.579+0000 7f7316800740 10 client.21186 fetch_fsmap learned FSMap version 57 2024-06-10T01:30:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.579+0000 7f7316800740 1 -- 172.21.15.7:0/2985094247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55c7acda5ed0 con 0x55c7acd2bb70 2024-06-10T01:30:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.579+0000 7f72f57fa640 1 -- 172.21.15.7:0/2985094247 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f730809bc10 con 0x55c7acd2bb70 2024-06-10T01:30:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.579+0000 7f72f57fa640 1 -- 172.21.15.7:0/2985094247 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7f730400f040 con 0x55c7acd2bb70 2024-06-10T01:30:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.579+0000 7f7316800740 10 client.21186 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:30:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.579+0000 7f7316800740 10 client.21186 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:30:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.579+0000 7f7316800740 20 client.21186 populate_metadata read hostname 'smithi007' 2024-06-10T01:30:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.579+0000 7f7316800740 1 --2- 172.21.15.7:0/2985094247 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c7acda6790 0x55c7acda8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.579+0000 7f7316800740 4 client.21186 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:30:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.579+0000 7f7316800740 1 -- 172.21.15.7:0/2985094247 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c7acc96c20 con 0x55c7acda6790 2024-06-10T01:30:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.580+0000 7f7311891640 1 --2- 172.21.15.7:0/2985094247 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c7acda6790 0x55c7acda8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.580+0000 7f7311891640 1 --2- 172.21.15.7:0/2985094247 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c7acda6790 0x55c7acda8bb0 crc :-1 s=READY pgs=542 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49a533495300803d server_cookie=7e5786dacf50d625 in_seq=0 out_seq=0 2024-06-10T01:30:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.580+0000 7f72f57fa640 10 client.21186 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.750+0000 7f72f57fa640 1 -- 172.21.15.7:0/2985094247 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c7acc96c20 con 0x55c7acda6790 2024-06-10T01:30:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.750+0000 7f72f57fa640 10 client.21186 handle_command_reply: tid=0 2024-06-10T01:30:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.750+0000 7f7316800740 1 client.21186 shutdown 2024-06-10T01:30:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.750+0000 7f7316800740 1 -- 172.21.15.7:0/2985094247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7acd2bb70 msgr2=0x55c7acd345d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.750+0000 7f7316800740 1 --2- 172.21.15.7:0/2985094247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7acd2bb70 0x55c7acd345d0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f7304009d20 tx=0x7f7304009d50 comp rx=0 tx=0).stop 2024-06-10T01:30:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.750+0000 7f7316800740 1 -- 172.21.15.7:0/2985094247 shutdown_connections 2024-06-10T01:30:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.750+0000 7f7316800740 1 --2- 172.21.15.7:0/2985094247 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55c7acda6790 0x55c7acda8bb0 crc :-1 s=READY pgs=542 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.750+0000 7f7316800740 1 --2- 172.21.15.7:0/2985094247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7acd2bb70 0x55c7acd345d0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.750+0000 7f7316800740 1 --2- 172.21.15.7:0/2985094247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7acd26a00 0x55c7acd34090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.750+0000 7f72f57fa640 0 client.21186 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.750+0000 7f7316800740 1 --2- 172.21.15.7:0/2985094247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7acd26010 0x55c7acd37580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.750+0000 7f7316800740 1 -- 172.21.15.7:0/2985094247 >> 172.21.15.7:0/2985094247 conn(0x55c7acd21680 msgr2=0x55c7acd27e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.751+0000 7f7316800740 1 -- 172.21.15.7:0/2985094247 shutdown_connections 2024-06-10T01:30:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.751+0000 7f7316800740 1 -- 172.21.15.7:0/2985094247 wait complete. 2024-06-10T01:30:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.751+0000 7f7316800740 20 client.21186 trim_cache size 0 max 16384 2024-06-10T01:30:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:30:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:30:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:30:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.751+0000 7f7313af5640 1 -- 172.21.15.7:0/960544209 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f72f80a2c00 msgr2=0x7f72f80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.751+0000 7f7313af5640 1 --2- 172.21.15.7:0/960544209 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f72f80a2c00 0x7f72f80a50c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7304021000 tx=0x7f730401e000 comp rx=0 tx=0).stop 2024-06-10T01:30:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.751+0000 7f7313af5640 1 -- 172.21.15.7:0/960544209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f730c100910 msgr2=0x7f730c197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.751+0000 7f7313af5640 1 --2- 172.21.15.7:0/960544209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f730c100910 0x7f730c197d50 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f730800e9c0 tx=0x7f730800ee80 comp rx=0 tx=0).stop 2024-06-10T01:30:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.751+0000 7f7313af5640 1 -- 172.21.15.7:0/960544209 shutdown_connections 2024-06-10T01:30:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.751+0000 7f7313af5640 1 --2- 172.21.15.7:0/960544209 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f72f80a2c00 0x7f72f80a50c0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.751+0000 7f7313af5640 1 --2- 172.21.15.7:0/960544209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f730c1024c0 0x7f730c19f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.751+0000 7f7313af5640 1 --2- 172.21.15.7:0/960544209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f730c101b00 0x7f730c198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.751+0000 7f7313af5640 1 --2- 172.21.15.7:0/960544209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f730c100910 0x7f730c197d50 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.751+0000 7f7313af5640 1 -- 172.21.15.7:0/960544209 >> 172.21.15.7:0/960544209 conn(0x7f730c0fbe80 msgr2=0x7f730c0fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.751+0000 7f7313af5640 1 -- 172.21.15.7:0/960544209 shutdown_connections 2024-06-10T01:30:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:16.751+0000 7f7313af5640 1 -- 172.21.15.7:0/960544209 wait complete. 2024-06-10T01:30:16.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:16 smithi175 ceph-mon[36636]: pgmap v1162: 129 pgs: 129 active+clean; 697 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 9.3 MiB/s wr, 605 op/s 2024-06-10T01:30:17.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:16 smithi007 ceph-mon[32398]: pgmap v1162: 129 pgs: 129 active+clean; 697 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 9.3 MiB/s wr, 605 op/s 2024-06-10T01:30:17.016 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:30:17.016 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:30:17.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:16 smithi078 ceph-mon[36683]: pgmap v1162: 129 pgs: 129 active+clean; 697 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 9.3 MiB/s wr, 605 op/s 2024-06-10T01:30:17.266 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:30:17.567 INFO:tasks.workunit.client.0.smithi007.stdout: 10 256380 21.78 MB/sec execute 534 sec latency 8985.110 ms 2024-06-10T01:30:17.768 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:30:17.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:17 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:30:18.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:17 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:30:18.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:17 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:30:18.558 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:30:18.566 INFO:tasks.workunit.client.0.smithi007.stdout: 10 256380 21.74 MB/sec execute 535 sec latency 9985.155 ms 2024-06-10T01:30:18.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:18 smithi007 ceph-mon[32398]: pgmap v1163: 129 pgs: 129 active+clean; 669 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.5 MiB/s wr, 516 op/s 2024-06-10T01:30:18.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:18 smithi175 ceph-mon[36636]: pgmap v1163: 129 pgs: 129 active+clean; 669 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.5 MiB/s wr, 516 op/s 2024-06-10T01:30:19.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.035+0000 7f990a597700 1 -- 172.21.15.7:0/199129723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9904107450 msgr2=0x7f9904107830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:19.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.035+0000 7f990a597700 1 --2- 172.21.15.7:0/199129723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9904107450 0x7f9904107830 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009990 tx=0x7f98f4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.035+0000 7f990a597700 1 -- 172.21.15.7:0/199129723 shutdown_connections 2024-06-10T01:30:19.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.035+0000 7f990a597700 1 --2- 172.21.15.7:0/199129723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9904102ee0 0x7f990410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.035+0000 7f990a597700 1 --2- 172.21.15.7:0/199129723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9904107450 0x7f9904107830 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.035+0000 7f990a597700 1 --2- 172.21.15.7:0/199129723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9904102540 0x7f99041029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.035+0000 7f990a597700 1 -- 172.21.15.7:0/199129723 >> 172.21.15.7:0/199129723 conn(0x7f99040795f0 msgr2=0x7f9904079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:19.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.035+0000 7f990a597700 1 -- 172.21.15.7:0/199129723 shutdown_connections 2024-06-10T01:30:19.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.035+0000 7f990a597700 1 -- 172.21.15.7:0/199129723 wait complete. 2024-06-10T01:30:19.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.036+0000 7f990a597700 1 Processor -- start 2024-06-10T01:30:19.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.036+0000 7f990a597700 1 -- start start 2024-06-10T01:30:19.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.036+0000 7f990a597700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9904102540 0x7f990419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:19.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.036+0000 7f990a597700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9904102ee0 0x7f990419ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:19.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.036+0000 7f990a597700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9904107450 0x7f9904198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:19.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.036+0000 7f990a597700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9904112840 con 0x7f9904102ee0 2024-06-10T01:30:19.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.036+0000 7f990a597700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f99041126c0 con 0x7f9904107450 2024-06-10T01:30:19.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.036+0000 7f990a597700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f99041129c0 con 0x7f9904102540 2024-06-10T01:30:19.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.036+0000 7f99037fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9904102ee0 0x7f990419ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:19.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.037+0000 7f99037fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9904102ee0 0x7f990419ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43232/0 (socket says 172.21.15.7:43232) 2024-06-10T01:30:19.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.037+0000 7f99037fe700 1 -- 172.21.15.7:0/3758992737 learned_addr learned my addr 172.21.15.7:0/3758992737 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:19.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.037+0000 7f9908b34700 1 --2- 172.21.15.7:0/3758992737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9904107450 0x7f9904198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:19.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.037+0000 7f9903fff700 1 --2- 172.21.15.7:0/3758992737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9904102540 0x7f990419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:19.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.037+0000 7f99037fe700 1 -- 172.21.15.7:0/3758992737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9904102540 msgr2=0x7f990419e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:19.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.037+0000 7f99037fe700 1 --2- 172.21.15.7:0/3758992737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9904102540 0x7f990419e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.037+0000 7f99037fe700 1 -- 172.21.15.7:0/3758992737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9904107450 msgr2=0x7f9904198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:19.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.037+0000 7f99037fe700 1 --2- 172.21.15.7:0/3758992737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9904107450 0x7f9904198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.037+0000 7f99037fe700 1 -- 172.21.15.7:0/3758992737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9904199260 con 0x7f9904102ee0 2024-06-10T01:30:19.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.037+0000 7f99037fe700 1 --2- 172.21.15.7:0/3758992737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9904102ee0 0x7f990419ee80 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f98f800b530 tx=0x7f98f800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:19.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.037+0000 7f99017fa700 1 -- 172.21.15.7:0/3758992737 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f8015020 con 0x7f9904102ee0 2024-06-10T01:30:19.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.037+0000 7f99017fa700 1 -- 172.21.15.7:0/3758992737 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98f8011d10 con 0x7f9904102ee0 2024-06-10T01:30:19.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.037+0000 7f990a597700 1 -- 172.21.15.7:0/3758992737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9904199550 con 0x7f9904102ee0 2024-06-10T01:30:19.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.037+0000 7f99017fa700 1 -- 172.21.15.7:0/3758992737 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f800f2e0 con 0x7f9904102ee0 2024-06-10T01:30:19.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.038+0000 7f990a597700 1 -- 172.21.15.7:0/3758992737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99041a5810 con 0x7f9904102ee0 2024-06-10T01:30:19.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.040+0000 7f99017fa700 1 -- 172.21.15.7:0/3758992737 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f98f8018020 con 0x7f9904102ee0 2024-06-10T01:30:19.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.041+0000 7f99017fa700 1 --2- 172.21.15.7:0/3758992737 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f98ec0a2ca0 0x7f98ec0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:19.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.041+0000 7f9903fff700 1 --2- 172.21.15.7:0/3758992737 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f98ec0a2ca0 0x7f98ec0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:19.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.041+0000 7f99017fa700 1 -- 172.21.15.7:0/3758992737 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f98f80cea90 con 0x7f9904102ee0 2024-06-10T01:30:19.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.041+0000 7f9903fff700 1 --2- 172.21.15.7:0/3758992737 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f98ec0a2ca0 0x7f98ec0a5160 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f98f4006010 tx=0x7f98f4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:19.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.042+0000 7f9914e1ab80 1 Processor -- start 2024-06-10T01:30:19.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.042+0000 7f9914e1ab80 1 -- start start 2024-06-10T01:30:19.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.042+0000 7f9914e1ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612a19d93f0 0x5612a19d97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:19.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.042+0000 7f9914e1ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612a19d9d10 0x5612a19d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:19.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.042+0000 7f9914e1ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612a19d3c30 0x5612a19d4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:19.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.042+0000 7f9914e1ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5612a18c1fe0 con 0x5612a19d9d10 2024-06-10T01:30:19.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.042+0000 7f9914e1ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5612a19208d0 con 0x5612a19d93f0 2024-06-10T01:30:19.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.042+0000 7f9914e1ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5612a18884e0 con 0x5612a19d3c30 2024-06-10T01:30:19.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.042+0000 7f99037fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612a19d9d10 0x5612a19d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:19.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.042+0000 7f99037fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612a19d9d10 0x5612a19d36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43244/0 (socket says 172.21.15.7:43244) 2024-06-10T01:30:19.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.042+0000 7f99037fe700 1 -- 172.21.15.7:0/1486996074 learned_addr learned my addr 172.21.15.7:0/1486996074 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:19.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.042+0000 7f9903fff700 1 --2- 172.21.15.7:0/1486996074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612a19d93f0 0x5612a19d97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:19.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.042+0000 7f9908b34700 1 --2- 172.21.15.7:0/1486996074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612a19d3c30 0x5612a19d4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:19.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.042+0000 7f99037fe700 1 -- 172.21.15.7:0/1486996074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612a19d3c30 msgr2=0x5612a19d4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:19.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.042+0000 7f99037fe700 1 --2- 172.21.15.7:0/1486996074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612a19d3c30 0x5612a19d4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.042+0000 7f99037fe700 1 -- 172.21.15.7:0/1486996074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612a19d93f0 msgr2=0x5612a19d97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:19.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.042+0000 7f99037fe700 1 --2- 172.21.15.7:0/1486996074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612a19d93f0 0x5612a19d97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.042+0000 7f99037fe700 1 -- 172.21.15.7:0/1486996074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612a186f950 con 0x5612a19d9d10 2024-06-10T01:30:19.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.043+0000 7f99037fe700 1 --2- 172.21.15.7:0/1486996074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612a19d9d10 0x5612a19d36f0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f98f8097370 tx=0x7f98f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e38cdebcdd39e0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:19.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.043+0000 7f98ea7fc700 1 -- 172.21.15.7:0/1486996074 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f809af10 con 0x5612a19d9d10 2024-06-10T01:30:19.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.043+0000 7f98ea7fc700 1 -- 172.21.15.7:0/1486996074 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98f809f2c0 con 0x5612a19d9d10 2024-06-10T01:30:19.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.043+0000 7f98ea7fc700 1 -- 172.21.15.7:0/1486996074 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f809b3d0 con 0x5612a19d9d10 2024-06-10T01:30:19.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.044+0000 7f9914e1ab80 1 -- 172.21.15.7:0/1486996074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612a19d9d10 msgr2=0x5612a19d36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:19.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.044+0000 7f9914e1ab80 1 --2- 172.21.15.7:0/1486996074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612a19d9d10 0x5612a19d36f0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f98f8097370 tx=0x7f98f8004750 comp rx=0 tx=0).stop 2024-06-10T01:30:19.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.044+0000 7f9914e1ab80 1 -- 172.21.15.7:0/1486996074 shutdown_connections 2024-06-10T01:30:19.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.044+0000 7f9914e1ab80 1 --2- 172.21.15.7:0/1486996074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612a19d3c30 0x5612a19d4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.044+0000 7f9914e1ab80 1 --2- 172.21.15.7:0/1486996074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612a19d93f0 0x5612a19d97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.044+0000 7f9914e1ab80 1 --2- 172.21.15.7:0/1486996074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612a19d9d10 0x5612a19d36f0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.044+0000 7f9914e1ab80 1 -- 172.21.15.7:0/1486996074 >> 172.21.15.7:0/1486996074 conn(0x5612a18fc100 msgr2=0x5612a19ca2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:19.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.044+0000 7f9914e1ab80 1 -- 172.21.15.7:0/1486996074 shutdown_connections 2024-06-10T01:30:19.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.044+0000 7f9914e1ab80 1 -- 172.21.15.7:0/1486996074 wait complete. 2024-06-10T01:30:19.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.044+0000 7f9914e1ab80 1 -- start start 2024-06-10T01:30:19.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.044+0000 7f9914e1ab80 1 Processor -- start 2024-06-10T01:30:19.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.045+0000 7f9914e1ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612a19d3c30 0x5612a19dfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:19.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.045+0000 7f9914e1ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612a19d93f0 0x5612a19dcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:19.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.045+0000 7f9914e1ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612a19d9d10 0x5612a19dd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:19.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.045+0000 7f9914e1ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5612a18c1040 con 0x5612a19d3c30 2024-06-10T01:30:19.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.045+0000 7f9914e1ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5612a191c890 con 0x5612a19d93f0 2024-06-10T01:30:19.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.045+0000 7f9914e1ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5612a19e06f0 con 0x5612a19d9d10 2024-06-10T01:30:19.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.045+0000 7f9914e1ab80 10 client.0 _ll_register_callbacks cb 0x5612a18b6630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:30:19.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.045+0000 7f9903fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612a19d3c30 0x5612a19dfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:19.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.045+0000 7f9903fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612a19d3c30 0x5612a19dfa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43256/0 (socket says 172.21.15.7:43256) 2024-06-10T01:30:19.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.045+0000 7f9903fff700 1 -- 172.21.15.7:0/2236531746 learned_addr learned my addr 172.21.15.7:0/2236531746 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:19.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.045+0000 7f99037fe700 1 --2- 172.21.15.7:0/2236531746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612a19d93f0 0x5612a19dcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:19.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.045+0000 7f9908b34700 1 --2- 172.21.15.7:0/2236531746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612a19d9d10 0x5612a19dd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:19.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.045+0000 7f9903fff700 1 -- 172.21.15.7:0/2236531746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612a19d9d10 msgr2=0x5612a19dd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:19.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.045+0000 7f9903fff700 1 --2- 172.21.15.7:0/2236531746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612a19d9d10 0x5612a19dd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.045+0000 7f9903fff700 1 -- 172.21.15.7:0/2236531746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612a19d93f0 msgr2=0x5612a19dcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:19.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.045+0000 7f9903fff700 1 --2- 172.21.15.7:0/2236531746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612a19d93f0 0x5612a19dcd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.045+0000 7f9903fff700 1 -- 172.21.15.7:0/2236531746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612a18e67d0 con 0x5612a19d3c30 2024-06-10T01:30:19.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.045+0000 7f99037fe700 1 --2- 172.21.15.7:0/2236531746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612a19d93f0 0x5612a19dcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:30:19.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.045+0000 7f9903fff700 1 --2- 172.21.15.7:0/2236531746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612a19d3c30 0x5612a19dfa80 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009750 tx=0x7f98f4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321da2a226ca9f88 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:19.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.045+0000 7f98e37fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:30:19.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.046+0000 7f98e37fe700 1 -- 172.21.15.7:0/2236531746 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f4035060 con 0x5612a19d3c30 2024-06-10T01:30:19.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.046+0000 7f9914e1ab80 1 -- 172.21.15.7:0/2236531746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98f809f2c0 con 0x5612a19d3c30 2024-06-10T01:30:19.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.046+0000 7f98e37fe700 1 -- 172.21.15.7:0/2236531746 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f98f4030630 con 0x5612a19d3c30 2024-06-10T01:30:19.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.046+0000 7f98e37fe700 1 -- 172.21.15.7:0/2236531746 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f98f403c050 con 0x5612a19d3c30 2024-06-10T01:30:19.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.046+0000 7f9914e1ab80 10 client.21198 fetch_fsmap learned FSMap version 57 2024-06-10T01:30:19.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.046+0000 7f9914e1ab80 1 -- 172.21.15.7:0/2236531746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5612a19eaef0 con 0x5612a19d3c30 2024-06-10T01:30:19.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.046+0000 7f98e37fe700 1 -- 172.21.15.7:0/2236531746 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98f809f2c0 con 0x5612a19d3c30 2024-06-10T01:30:19.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.046+0000 7f98e37fe700 1 -- 172.21.15.7:0/2236531746 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7f98f402f070 con 0x5612a19d3c30 2024-06-10T01:30:19.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.046+0000 7f9914e1ab80 10 client.21198 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:30:19.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.046+0000 7f9914e1ab80 10 client.21198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:30:19.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.046+0000 7f9914e1ab80 20 client.21198 populate_metadata read hostname 'smithi007' 2024-06-10T01:30:19.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.046+0000 7f9914e1ab80 1 --2- 172.21.15.7:0/2236531746 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5612a19ed480 0x5612a19ef870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:19.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.046+0000 7f9914e1ab80 4 client.21198 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:30:19.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.046+0000 7f9914e1ab80 1 -- 172.21.15.7:0/2236531746 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612a192ddd0 con 0x5612a19ed480 2024-06-10T01:30:19.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.047+0000 7f99037fe700 1 --2- 172.21.15.7:0/2236531746 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5612a19ed480 0x5612a19ef870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:19.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.047+0000 7f99037fe700 1 --2- 172.21.15.7:0/2236531746 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5612a19ed480 0x5612a19ef870 crc :-1 s=READY pgs=543 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fa8a7a3d48ab9b7 server_cookie=fe9b949f360a6be8 in_seq=0 out_seq=0 2024-06-10T01:30:19.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.047+0000 7f98e37fe700 10 client.21198 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:19.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.048+0000 7f98e37fe700 1 -- 172.21.15.7:0/2236531746 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5612a192ddd0 con 0x5612a19ed480 2024-06-10T01:30:19.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.048+0000 7f98e37fe700 10 client.21198 handle_command_reply: tid=0 2024-06-10T01:30:19.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.048+0000 7f9914e1ab80 1 client.21198 shutdown 2024-06-10T01:30:19.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.048+0000 7f9914e1ab80 1 -- 172.21.15.7:0/2236531746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612a19d3c30 msgr2=0x5612a19dfa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:19.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.048+0000 7f9914e1ab80 1 --2- 172.21.15.7:0/2236531746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612a19d3c30 0x5612a19dfa80 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009750 tx=0x7f98f4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.048+0000 7f9914e1ab80 1 -- 172.21.15.7:0/2236531746 shutdown_connections 2024-06-10T01:30:19.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.048+0000 7f9914e1ab80 1 --2- 172.21.15.7:0/2236531746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612a19d9d10 0x5612a19dd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.048+0000 7f9914e1ab80 1 --2- 172.21.15.7:0/2236531746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612a19d3c30 0x5612a19dfa80 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.048+0000 7f9914e1ab80 1 --2- 172.21.15.7:0/2236531746 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5612a19ed480 0x5612a19ef870 crc :-1 s=READY pgs=543 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.048+0000 7f9914e1ab80 1 --2- 172.21.15.7:0/2236531746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612a19d93f0 0x5612a19dcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.048+0000 7f9914e1ab80 1 -- 172.21.15.7:0/2236531746 >> 172.21.15.7:0/2236531746 conn(0x5612a18fc100 msgr2=0x5612a19c9780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:19.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.048+0000 7f98e37fe700 0 client.21198 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:19.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.048+0000 7f9914e1ab80 1 -- 172.21.15.7:0/2236531746 shutdown_connections 2024-06-10T01:30:19.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.048+0000 7f9914e1ab80 1 -- 172.21.15.7:0/2236531746 wait complete. 2024-06-10T01:30:19.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.048+0000 7f9914e1ab80 20 client.21198 trim_cache size 0 max 16384 2024-06-10T01:30:19.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.069+0000 7f9914e1ab80 1 Processor -- start 2024-06-10T01:30:19.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.070+0000 7f9914e1ab80 1 -- start start 2024-06-10T01:30:19.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.070+0000 7f9914e1ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612a19d93f0 0x5612a19d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:19.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.070+0000 7f9914e1ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612a19d9d10 0x5612a19da7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:19.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.070+0000 7f9914e1ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612a19e9840 0x5612a19c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:19.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.070+0000 7f9914e1ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f99041ae9c0 con 0x5612a19d9d10 2024-06-10T01:30:19.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.070+0000 7f9914e1ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f99041ac1f0 con 0x5612a19e9840 2024-06-10T01:30:19.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.070+0000 7f9914e1ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f99041aa040 con 0x5612a19d93f0 2024-06-10T01:30:19.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.070+0000 7f99037fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612a19d9d10 0x5612a19da7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:19.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.070+0000 7f99037fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612a19d9d10 0x5612a19da7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43270/0 (socket says 172.21.15.7:43270) 2024-06-10T01:30:19.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.070+0000 7f99037fe700 1 -- 172.21.15.7:0/529382553 learned_addr learned my addr 172.21.15.7:0/529382553 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:19.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.070+0000 7f9908b34700 1 --2- 172.21.15.7:0/529382553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612a19e9840 0x5612a19c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.070+0000 7f9903fff700 1 --2- 172.21.15.7:0/529382553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612a19d93f0 0x5612a19d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.070+0000 7f99037fe700 1 -- 172.21.15.7:0/529382553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612a19d93f0 msgr2=0x5612a19d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.070+0000 7f99037fe700 1 --2- 172.21.15.7:0/529382553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612a19d93f0 0x5612a19d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.070+0000 7f99037fe700 1 -- 172.21.15.7:0/529382553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612a19e9840 msgr2=0x5612a19c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.070+0000 7f99037fe700 1 --2- 172.21.15.7:0/529382553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612a19e9840 0x5612a19c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.070+0000 7f99037fe700 1 -- 172.21.15.7:0/529382553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612a19dbd40 con 0x5612a19d9d10 2024-06-10T01:30:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.070+0000 7f9908b34700 1 --2- 172.21.15.7:0/529382553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612a19e9840 0x5612a19c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:30:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.070+0000 7f99037fe700 1 --2- 172.21.15.7:0/529382553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612a19d9d10 0x5612a19da7a0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f98f809b0d0 tx=0x7f98f80bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81db7e67db23d129 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.071+0000 7f98e9ffb700 1 -- 172.21.15.7:0/529382553 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f8010610 con 0x5612a19d9d10 2024-06-10T01:30:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.071+0000 7f98e9ffb700 1 -- 172.21.15.7:0/529382553 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98f8099210 con 0x5612a19d9d10 2024-06-10T01:30:19.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.071+0000 7f98e9ffb700 1 -- 172.21.15.7:0/529382553 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f80a3950 con 0x5612a19d9d10 2024-06-10T01:30:19.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.071+0000 7f9914e1ab80 1 -- 172.21.15.7:0/529382553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612a19d9d10 msgr2=0x5612a19da7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:19.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.071+0000 7f9914e1ab80 1 --2- 172.21.15.7:0/529382553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612a19d9d10 0x5612a19da7a0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f98f809b0d0 tx=0x7f98f80bb040 comp rx=0 tx=0).stop 2024-06-10T01:30:19.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.071+0000 7f9914e1ab80 1 -- 172.21.15.7:0/529382553 shutdown_connections 2024-06-10T01:30:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.071+0000 7f9914e1ab80 1 --2- 172.21.15.7:0/529382553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612a19e9840 0x5612a19c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.071+0000 7f9914e1ab80 1 --2- 172.21.15.7:0/529382553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612a19d93f0 0x5612a19d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.071+0000 7f9914e1ab80 1 --2- 172.21.15.7:0/529382553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612a19d9d10 0x5612a19da7a0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.071+0000 7f9914e1ab80 1 -- 172.21.15.7:0/529382553 >> 172.21.15.7:0/529382553 conn(0x5612a18fc100 msgr2=0x5612a19dee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.071+0000 7f9914e1ab80 1 -- 172.21.15.7:0/529382553 shutdown_connections 2024-06-10T01:30:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.071+0000 7f9914e1ab80 1 -- 172.21.15.7:0/529382553 wait complete. 2024-06-10T01:30:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.072+0000 7f9914e1ab80 1 -- start start 2024-06-10T01:30:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.072+0000 7f9914e1ab80 1 Processor -- start 2024-06-10T01:30:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.072+0000 7f9914e1ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612a19d93f0 0x5612a19e5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.072+0000 7f9914e1ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612a19d9d10 0x5612a19e60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.072+0000 7f9914e1ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612a19e9840 0x5612a19e2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.072+0000 7f9914e1ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f99041aa6b0 con 0x5612a19e9840 2024-06-10T01:30:19.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.072+0000 7f9914e1ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f99041abb20 con 0x5612a19d93f0 2024-06-10T01:30:19.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.072+0000 7f9914e1ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f99041aad80 con 0x5612a19d9d10 2024-06-10T01:30:19.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.072+0000 7f9914e1ab80 10 client.0 _ll_register_callbacks cb 0x5612a18b6630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:30:19.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.072+0000 7f9908b34700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612a19e9840 0x5612a19e2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:19.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.072+0000 7f9908b34700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612a19e9840 0x5612a19e2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43282/0 (socket says 172.21.15.7:43282) 2024-06-10T01:30:19.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.072+0000 7f9908b34700 1 -- 172.21.15.7:0/3797586327 learned_addr learned my addr 172.21.15.7:0/3797586327 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:19.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.073+0000 7f9903fff700 1 --2- 172.21.15.7:0/3797586327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612a19d93f0 0x5612a19e5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:19.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.073+0000 7f99037fe700 1 --2- 172.21.15.7:0/3797586327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612a19d9d10 0x5612a19e60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:19.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.073+0000 7f9908b34700 1 -- 172.21.15.7:0/3797586327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612a19d9d10 msgr2=0x5612a19e60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:19.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.073+0000 7f9908b34700 1 --2- 172.21.15.7:0/3797586327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612a19d9d10 0x5612a19e60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.073+0000 7f9908b34700 1 -- 172.21.15.7:0/3797586327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612a19d93f0 msgr2=0x5612a19e5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:19.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.073+0000 7f9908b34700 1 --2- 172.21.15.7:0/3797586327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612a19d93f0 0x5612a19e5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.073+0000 7f9908b34700 1 -- 172.21.15.7:0/3797586327 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612a19e3030 con 0x5612a19e9840 2024-06-10T01:30:19.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.073+0000 7f9903fff700 1 --2- 172.21.15.7:0/3797586327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612a19d93f0 0x5612a19e5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:30:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.073+0000 7f9908b34700 1 --2- 172.21.15.7:0/3797586327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612a19e9840 0x5612a19e2980 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f98fc007bd0 tx=0x7f98fc007cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d3a5c51b03f11 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.073+0000 7f98e37fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:30:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.073+0000 7f98e37fe700 1 -- 172.21.15.7:0/3797586327 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc017a20 con 0x5612a19e9840 2024-06-10T01:30:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.073+0000 7f98e37fe700 1 -- 172.21.15.7:0/3797586327 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f98fc017bc0 con 0x5612a19e9840 2024-06-10T01:30:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.073+0000 7f9914e1ab80 1 -- 172.21.15.7:0/3797586327 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98f8099210 con 0x5612a19e9840 2024-06-10T01:30:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.073+0000 7f98e37fe700 1 -- 172.21.15.7:0/3797586327 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f98fc017dc0 con 0x5612a19e9840 2024-06-10T01:30:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.073+0000 7f9914e1ab80 10 client.21204 fetch_fsmap learned FSMap version 57 2024-06-10T01:30:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.073+0000 7f9914e1ab80 1 -- 172.21.15.7:0/3797586327 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5612a1a58b10 con 0x5612a19e9840 2024-06-10T01:30:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.073+0000 7f98e37fe700 1 -- 172.21.15.7:0/3797586327 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98f8099210 con 0x5612a19e9840 2024-06-10T01:30:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.074+0000 7f98e37fe700 1 -- 172.21.15.7:0/3797586327 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7f98fc018560 con 0x5612a19e9840 2024-06-10T01:30:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.074+0000 7f9914e1ab80 10 client.21204 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:30:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.074+0000 7f9914e1ab80 10 client.21204 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:30:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.074+0000 7f9914e1ab80 20 client.21204 populate_metadata read hostname 'smithi007' 2024-06-10T01:30:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.074+0000 7f9914e1ab80 1 --2- 172.21.15.7:0/3797586327 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5612a1a59300 0x5612a1a5b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:19.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.074+0000 7f9914e1ab80 4 client.21204 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:30:19.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.074+0000 7f9914e1ab80 1 -- 172.21.15.7:0/3797586327 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5612a1928350 con 0x5612a1a59300 2024-06-10T01:30:19.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.074+0000 7f9903fff700 1 --2- 172.21.15.7:0/3797586327 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5612a1a59300 0x5612a1a5b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:19.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.074+0000 7f9903fff700 1 --2- 172.21.15.7:0/3797586327 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5612a1a59300 0x5612a1a5b720 crc :-1 s=READY pgs=544 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cc7842d8a51357a server_cookie=bc14e5940f87e6e5 in_seq=0 out_seq=0 2024-06-10T01:30:19.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.074+0000 7f98e37fe700 10 client.21204 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:19.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.075+0000 7f98e37fe700 1 -- 172.21.15.7:0/3797586327 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5612a1928350 con 0x5612a1a59300 2024-06-10T01:30:19.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.075+0000 7f98e37fe700 10 client.21204 handle_command_reply: tid=0 2024-06-10T01:30:19.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.075+0000 7f9914e1ab80 1 client.21204 shutdown 2024-06-10T01:30:19.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.076+0000 7f9914e1ab80 1 -- 172.21.15.7:0/3797586327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612a19e9840 msgr2=0x5612a19e2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:19.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.076+0000 7f9914e1ab80 1 --2- 172.21.15.7:0/3797586327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612a19e9840 0x5612a19e2980 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f98fc007bd0 tx=0x7f98fc007cb0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.076+0000 7f9914e1ab80 1 -- 172.21.15.7:0/3797586327 shutdown_connections 2024-06-10T01:30:19.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.076+0000 7f9914e1ab80 1 --2- 172.21.15.7:0/3797586327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612a19e9840 0x5612a19e2980 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.076+0000 7f9914e1ab80 1 --2- 172.21.15.7:0/3797586327 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5612a1a59300 0x5612a1a5b720 crc :-1 s=READY pgs=544 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.076+0000 7f9914e1ab80 1 --2- 172.21.15.7:0/3797586327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612a19d93f0 0x5612a19e5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.076+0000 7f9914e1ab80 1 --2- 172.21.15.7:0/3797586327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612a19d9d10 0x5612a19e60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.076+0000 7f98e37fe700 0 client.21204 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:19.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.076+0000 7f9914e1ab80 1 -- 172.21.15.7:0/3797586327 >> 172.21.15.7:0/3797586327 conn(0x5612a18fc100 msgr2=0x5612a19e90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:19.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.076+0000 7f9914e1ab80 1 -- 172.21.15.7:0/3797586327 shutdown_connections 2024-06-10T01:30:19.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.076+0000 7f9914e1ab80 1 -- 172.21.15.7:0/3797586327 wait complete. 2024-06-10T01:30:19.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.076+0000 7f9914e1ab80 20 client.21204 trim_cache size 0 max 16384 2024-06-10T01:30:19.074 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:30:19.074 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-10T01:30:19.074 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "5b2dce0c-b7c0-4ac0-ab7c-96a0b18216bc", 2024-06-10T01:30:19.075 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-10T01:30:19.075 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:30:19.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.077+0000 7f990a597700 1 -- 172.21.15.7:0/3758992737 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f98ec0a2ca0 msgr2=0x7f98ec0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:19.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.077+0000 7f990a597700 1 --2- 172.21.15.7:0/3758992737 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f98ec0a2ca0 0x7f98ec0a5160 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f98f4006010 tx=0x7f98f4005960 comp rx=0 tx=0).stop 2024-06-10T01:30:19.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.077+0000 7f990a597700 1 -- 172.21.15.7:0/3758992737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9904102ee0 msgr2=0x7f990419ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:19.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.077+0000 7f990a597700 1 --2- 172.21.15.7:0/3758992737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9904102ee0 0x7f990419ee80 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f98f800b530 tx=0x7f98f800b840 comp rx=0 tx=0).stop 2024-06-10T01:30:19.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.077+0000 7f990a597700 1 -- 172.21.15.7:0/3758992737 shutdown_connections 2024-06-10T01:30:19.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.077+0000 7f990a597700 1 --2- 172.21.15.7:0/3758992737 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f98ec0a2ca0 0x7f98ec0a5160 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.077+0000 7f990a597700 1 --2- 172.21.15.7:0/3758992737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9904107450 0x7f9904198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.077+0000 7f990a597700 1 --2- 172.21.15.7:0/3758992737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9904102540 0x7f990419e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.077+0000 7f990a597700 1 --2- 172.21.15.7:0/3758992737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9904102ee0 0x7f990419ee80 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:19.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.077+0000 7f990a597700 1 -- 172.21.15.7:0/3758992737 >> 172.21.15.7:0/3758992737 conn(0x7f99040795f0 msgr2=0x7f99041005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:19.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.077+0000 7f990a597700 1 -- 172.21.15.7:0/3758992737 shutdown_connections 2024-06-10T01:30:19.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:19.077+0000 7f990a597700 1 -- 172.21.15.7:0/3758992737 wait complete. 2024-06-10T01:30:19.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:18 smithi078 ceph-mon[36683]: pgmap v1163: 129 pgs: 129 active+clean; 669 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.5 MiB/s wr, 516 op/s 2024-06-10T01:30:19.450 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jun 10 01:30:19 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-i[66802]: 2024-06-10T01:30:19.130+0000 7fe512307700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000044c3 [...2,head] /volumes/qa/sv_1/56b71f3b-2893-4c82-ac3a-2d6fa4194c93/client.0/tmp/clients/client5/~dmtmp/PARADOX/ auth{1=1,2=1} v53570 ap=3 f(v11 m2024-06-10T01:30:08.491471+0000 25=25+0) n(v1 rc2024-06-10T01:30:08.503471+0000 b5234335 26=25+1) (inest sync->mix g=1,2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:3820 nref=3)) caps={24973=pAsLsXsFs/pAsLsXsFsxcral@9520} mcw={2=AsLsXsFs} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x55e19c341700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000044c3:[<0x10000004375/PARADOX v376>,<0x10000004374/~dmtmp v40>,<0x10000000007/client5 v1700>,<0x10000000006/clients v1366>,<0x10000000005/tmp v1327>,<0x100000001fb/client.0 v1322>,<0x100000001fa/56b71f3b-2893-4c82-ac3a-2d6fa4194c93 v1314>,<0x10000000001/sv_1 v1318>,<0x10000000000/qa v1298>,<0x1/volumes v1255>]//[]","memoryvalue":"(2)0x100000044c3:[<0x10000004375/PARADOX v53570>,<0x10000004374/~dmtmp v1193>,<0x10000000007/client5 v6883>,<0x10000000006/clients v3387>,<0x10000000005/tmp v2436>,<0x100000001fb/client.0 v2412>,<0x100000001fa/56b71f3b-2893-4c82-ac3a-2d6fa4194c93 v2386>,<0x10000000001/sv_1 v2389>,<0x10000000000/qa v2359>,<0x1/volumes v2301>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-10T01:30:08.434472+0000 24=24+0)","ondisk_value.rstat":"n(v0 rc2024-06-10T01:30:08.503471+0000 b5234335 26=25+1)","memory_value.dirstat":"f(v11 m2024-06-10T01:30:08.491471+0000 25=25+0)","memory_value.rstat":"n(v1 rc2024-06-10T01:30:08.503471+0000 b5234335 26=25+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-10T01:30:19.567 INFO:tasks.workunit.client.0.smithi007.stdout: 10 256380 21.69 MB/sec execute 536 sec latency 10985.241 ms 2024-06-10T01:30:20.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:19 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:30:20.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:19 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:30:20.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:19 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:30:20.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:19 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:30:20.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:19 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:30:20.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:19 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:30:20.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:19 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:30:20.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:19 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:30:20.200 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:19 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:30:20.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:19 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:30:20.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:19 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:30:20.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:19 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:30:20.567 INFO:tasks.workunit.client.0.smithi007.stdout: 10 256380 21.65 MB/sec execute 537 sec latency 11985.365 ms 2024-06-10T01:30:21.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:20 smithi007 ceph-mon[32398]: scrub queued for path: / 2024-06-10T01:30:21.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:20 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:30:21.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:20 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:30:21.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:20 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:30:21.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:20 smithi007 ceph-mon[32398]: Scrub error on inode 0x100000044c3 (/volumes/qa/sv_1/56b71f3b-2893-4c82-ac3a-2d6fa4194c93/client.0/tmp/clients/client5/~dmtmp/PARADOX) see mds.i log and `damage ls` output for details 2024-06-10T01:30:21.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:20 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:30:21.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:20 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:30:21.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:20 smithi007 ceph-mon[32398]: pgmap v1164: 129 pgs: 129 active+clean; 665 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.5 MiB/s wr, 408 op/s 2024-06-10T01:30:21.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:20 smithi078 ceph-mon[36683]: scrub queued for path: / 2024-06-10T01:30:21.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:20 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:30:21.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:20 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:30:21.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:20 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:30:21.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:20 smithi078 ceph-mon[36683]: Scrub error on inode 0x100000044c3 (/volumes/qa/sv_1/56b71f3b-2893-4c82-ac3a-2d6fa4194c93/client.0/tmp/clients/client5/~dmtmp/PARADOX) see mds.i log and `damage ls` output for details 2024-06-10T01:30:21.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:20 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:30:21.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:20 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:30:21.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:20 smithi078 ceph-mon[36683]: pgmap v1164: 129 pgs: 129 active+clean; 665 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.5 MiB/s wr, 408 op/s 2024-06-10T01:30:21.200 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:20 smithi175 ceph-mon[36636]: scrub queued for path: / 2024-06-10T01:30:21.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:20 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:30:21.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:20 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:30:21.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:20 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:30:21.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:20 smithi175 ceph-mon[36636]: Scrub error on inode 0x100000044c3 (/volumes/qa/sv_1/56b71f3b-2893-4c82-ac3a-2d6fa4194c93/client.0/tmp/clients/client5/~dmtmp/PARADOX) see mds.i log and `damage ls` output for details 2024-06-10T01:30:21.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:20 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:30:21.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:20 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:30:21.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:20 smithi175 ceph-mon[36636]: pgmap v1164: 129 pgs: 129 active+clean; 665 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.5 MiB/s wr, 408 op/s 2024-06-10T01:30:21.567 INFO:tasks.workunit.client.0.smithi007.stdout: 10 256380 21.61 MB/sec execute 538 sec latency 12985.466 ms 2024-06-10T01:30:22.041 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:30:22.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.205+0000 7f701b01e700 1 -- 172.21.15.7:0/444169031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7014101d30 msgr2=0x7f701410e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:22.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.205+0000 7f701b01e700 1 --2- 172.21.15.7:0/444169031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7014101d30 0x7f701410e210 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f700c009990 tx=0x7f700c009ca0 comp rx=0 tx=0).stop 2024-06-10T01:30:22.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.205+0000 7f701b01e700 1 -- 172.21.15.7:0/444169031 shutdown_connections 2024-06-10T01:30:22.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.205+0000 7f701b01e700 1 --2- 172.21.15.7:0/444169031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7014101d30 0x7f701410e210 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:22.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.206+0000 7f701b01e700 1 --2- 172.21.15.7:0/444169031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70141073d0 0x7f70141077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:22.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.206+0000 7f701b01e700 1 --2- 172.21.15.7:0/444169031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7014101390 0x7f70141017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:22.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.206+0000 7f701b01e700 1 -- 172.21.15.7:0/444169031 >> 172.21.15.7:0/444169031 conn(0x7f70140782c0 msgr2=0x7f70140786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:22.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.206+0000 7f701b01e700 1 -- 172.21.15.7:0/444169031 shutdown_connections 2024-06-10T01:30:22.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.206+0000 7f701b01e700 1 -- 172.21.15.7:0/444169031 wait complete. 2024-06-10T01:30:22.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.207+0000 7f701b01e700 1 Processor -- start 2024-06-10T01:30:22.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.207+0000 7f701b01e700 1 -- start start 2024-06-10T01:30:22.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.207+0000 7f701b01e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7014101390 0x7f701410eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:22.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.207+0000 7f701b01e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7014101d30 0x7f701410f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:22.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.207+0000 7f701b01e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70141073d0 0x7f701410f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:22.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.207+0000 7f701b01e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f70141127d0 con 0x7f70141073d0 2024-06-10T01:30:22.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.207+0000 7f701b01e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7014112650 con 0x7f7014101d30 2024-06-10T01:30:22.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.207+0000 7f701b01e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7014112950 con 0x7f7014101390 2024-06-10T01:30:22.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.207+0000 7f70195bb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70141073d0 0x7f701410f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:22.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.207+0000 7f70195bb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70141073d0 0x7f701410f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44012/0 (socket says 172.21.15.7:44012) 2024-06-10T01:30:22.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.207+0000 7f70195bb700 1 -- 172.21.15.7:0/47685699 learned_addr learned my addr 172.21.15.7:0/47685699 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:22.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.207+0000 7f7013fff700 1 --2- 172.21.15.7:0/47685699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7014101d30 0x7f701410f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:22.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.207+0000 7f7018dba700 1 --2- 172.21.15.7:0/47685699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7014101390 0x7f701410eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:22.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.208+0000 7f70195bb700 1 -- 172.21.15.7:0/47685699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7014101390 msgr2=0x7f701410eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:22.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.208+0000 7f70195bb700 1 --2- 172.21.15.7:0/47685699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7014101390 0x7f701410eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:22.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.208+0000 7f70195bb700 1 -- 172.21.15.7:0/47685699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7014101d30 msgr2=0x7f701410f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:22.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.208+0000 7f70195bb700 1 --2- 172.21.15.7:0/47685699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7014101d30 0x7f701410f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:22.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.208+0000 7f70195bb700 1 -- 172.21.15.7:0/47685699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70141a7090 con 0x7f70141073d0 2024-06-10T01:30:22.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.208+0000 7f70195bb700 1 --2- 172.21.15.7:0/47685699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70141073d0 0x7f701410f720 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f700c0038b0 tx=0x7f700c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:22.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.208+0000 7f7011ffb700 1 -- 172.21.15.7:0/47685699 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c027070 con 0x7f70141073d0 2024-06-10T01:30:22.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.208+0000 7f701b01e700 1 -- 172.21.15.7:0/47685699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70141a7320 con 0x7f70141073d0 2024-06-10T01:30:22.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.208+0000 7f7011ffb700 1 -- 172.21.15.7:0/47685699 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f700c003e40 con 0x7f70141073d0 2024-06-10T01:30:22.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.208+0000 7f7011ffb700 1 -- 172.21.15.7:0/47685699 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c022cc0 con 0x7f70141073d0 2024-06-10T01:30:22.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.208+0000 7f701b01e700 1 -- 172.21.15.7:0/47685699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70141a77d0 con 0x7f70141073d0 2024-06-10T01:30:22.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.209+0000 7f701b01e700 1 -- 172.21.15.7:0/47685699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70141097b0 con 0x7f70141073d0 2024-06-10T01:30:22.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.211+0000 7f7011ffb700 1 -- 172.21.15.7:0/47685699 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f700c02a070 con 0x7f70141073d0 2024-06-10T01:30:22.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.211+0000 7f7011ffb700 1 --2- 172.21.15.7:0/47685699 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f70040a2c20 0x7f70040a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:22.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.212+0000 7f7011ffb700 1 -- 172.21.15.7:0/47685699 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f700c0d07c0 con 0x7f70141073d0 2024-06-10T01:30:22.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.212+0000 7f7018dba700 1 --2- 172.21.15.7:0/47685699 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f70040a2c20 0x7f70040a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:22.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.212+0000 7f7018dba700 1 --2- 172.21.15.7:0/47685699 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f70040a2c20 0x7f70040a50e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f7008009550 tx=0x7f70080091f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:22.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.219+0000 7f7011ffb700 1 -- 172.21.15.7:0/47685699 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f700c031020 con 0x7f70141073d0 2024-06-10T01:30:22.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.486+0000 7f701b01e700 1 -- 172.21.15.7:0/47685699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70141101d0 con 0x7f70141073d0 2024-06-10T01:30:22.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.487+0000 7f7011ffb700 1 -- 172.21.15.7:0/47685699 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+11855 (secure 0 0 0) 0x7f700c013070 con 0x7f70141073d0 2024-06-10T01:30:22.479 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:30:22.479 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":57,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:30:12.372548+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29813":{"gid":29813,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/138896429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":138896429},{"type":"v1","addr":"172.21.15.175:6835","nonce":138896429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24434,24436,24451]},"id":1}]} 2024-06-10T01:30:22.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.489+0000 7f701b01e700 1 -- 172.21.15.7:0/47685699 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f70040a2c20 msgr2=0x7f70040a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.489+0000 7f701b01e700 1 --2- 172.21.15.7:0/47685699 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f70040a2c20 0x7f70040a50e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f7008009550 tx=0x7f70080091f0 comp rx=0 tx=0).stop 2024-06-10T01:30:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.489+0000 7f701b01e700 1 -- 172.21.15.7:0/47685699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70141073d0 msgr2=0x7f701410f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.489+0000 7f701b01e700 1 --2- 172.21.15.7:0/47685699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70141073d0 0x7f701410f720 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f700c0038b0 tx=0x7f700c005e80 comp rx=0 tx=0).stop 2024-06-10T01:30:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.489+0000 7f701b01e700 1 -- 172.21.15.7:0/47685699 shutdown_connections 2024-06-10T01:30:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.489+0000 7f701b01e700 1 --2- 172.21.15.7:0/47685699 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f70040a2c20 0x7f70040a50e0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.489+0000 7f701b01e700 1 --2- 172.21.15.7:0/47685699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70141073d0 0x7f701410f720 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.489+0000 7f701b01e700 1 --2- 172.21.15.7:0/47685699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7014101390 0x7f701410eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.489+0000 7f701b01e700 1 --2- 172.21.15.7:0/47685699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7014101d30 0x7f701410f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.489+0000 7f701b01e700 1 -- 172.21.15.7:0/47685699 >> 172.21.15.7:0/47685699 conn(0x7f70140782c0 msgr2=0x7f7014100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.489+0000 7f701b01e700 1 -- 172.21.15.7:0/47685699 shutdown_connections 2024-06-10T01:30:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:22.489+0000 7f701b01e700 1 -- 172.21.15.7:0/47685699 wait complete. 2024-06-10T01:30:22.483 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 57 2024-06-10T01:30:22.567 INFO:tasks.workunit.client.0.smithi007.stdout: 10 256380 21.57 MB/sec execute 539 sec latency 13985.510 ms 2024-06-10T01:30:23.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:23 smithi175 ceph-mon[36636]: pgmap v1165: 129 pgs: 129 active+clean; 666 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.7 MiB/s wr, 439 op/s 2024-06-10T01:30:23.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:23 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/47685699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:30:23.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:23 smithi007 ceph-mon[32398]: pgmap v1165: 129 pgs: 129 active+clean; 666 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.7 MiB/s wr, 439 op/s 2024-06-10T01:30:23.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:23 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/47685699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:30:23.567 INFO:tasks.workunit.client.0.smithi007.stdout: 10 256380 21.53 MB/sec execute 540 sec latency 14985.793 ms 2024-06-10T01:30:23.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:23 smithi078 ceph-mon[36683]: pgmap v1165: 129 pgs: 129 active+clean; 666 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.7 MiB/s wr, 439 op/s 2024-06-10T01:30:23.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:23 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/47685699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:30:23.662 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db --set-id 11d6bcd3 --release --await' 2024-06-10T01:30:23.662 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db --set-id 11d6bcd3 --release --await 2024-06-10T01:30:24.242 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:30:24.567 INFO:tasks.workunit.client.0.smithi007.stdout: 10 256380 21.49 MB/sec execute 541 sec latency 15985.825 ms 2024-06-10T01:30:25.260 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:25 smithi007 ceph-mon[32398]: pgmap v1166: 129 pgs: 129 active+clean; 666 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 2.4 MiB/s wr, 260 op/s 2024-06-10T01:30:25.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:25 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:30:25.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:25 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:30:25.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:25 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:30:25.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:25 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:30:25.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:25 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:30:25.379 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:25 smithi078 ceph-mon[36683]: pgmap v1166: 129 pgs: 129 active+clean; 666 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 2.4 MiB/s wr, 260 op/s 2024-06-10T01:30:25.379 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:25 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:30:25.379 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:25 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:30:25.379 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:25 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:30:25.379 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:25 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:30:25.379 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:25 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:30:25.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:25 smithi175 ceph-mon[36636]: pgmap v1166: 129 pgs: 129 active+clean; 666 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 2.4 MiB/s wr, 260 op/s 2024-06-10T01:30:25.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:25 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:30:25.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:25 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:30:25.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:25 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:30:25.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:25 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:30:25.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:25 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:30:25.567 INFO:tasks.workunit.client.0.smithi007.stdout: 10 256380 21.45 MB/sec execute 542 sec latency 16985.905 ms 2024-06-10T01:30:25.846 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:30:26.568 INFO:tasks.workunit.client.0.smithi007.stdout: 10 256380 21.42 MB/sec execute 543 sec latency 17985.977 ms 2024-06-10T01:30:26.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.993+0000 7f50e0665700 1 -- 172.21.15.7:0/609911739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d8107840 msgr2=0x7f50d810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:26.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.993+0000 7f50e0665700 1 --2- 172.21.15.7:0/609911739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d8107840 0x7f50d810f3c0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f50d000b0d0 tx=0x7f50d000b3e0 comp rx=0 tx=0).stop 2024-06-10T01:30:26.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.993+0000 7f50e0665700 1 -- 172.21.15.7:0/609911739 shutdown_connections 2024-06-10T01:30:26.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.993+0000 7f50e0665700 1 --2- 172.21.15.7:0/609911739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d8107840 0x7f50d810f3c0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:26.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.993+0000 7f50e0665700 1 --2- 172.21.15.7:0/609911739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50d8102740 0x7f50d8102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:26.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.993+0000 7f50e0665700 1 --2- 172.21.15.7:0/609911739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50d81030f0 0x7f50d8107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:26.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.993+0000 7f50e0665700 1 -- 172.21.15.7:0/609911739 >> 172.21.15.7:0/609911739 conn(0x7f50d80795f0 msgr2=0x7f50d8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:26.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.993+0000 7f50e0665700 1 -- 172.21.15.7:0/609911739 shutdown_connections 2024-06-10T01:30:26.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.993+0000 7f50e0665700 1 -- 172.21.15.7:0/609911739 wait complete. 2024-06-10T01:30:26.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.994+0000 7f50e0665700 1 Processor -- start 2024-06-10T01:30:26.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.994+0000 7f50e0665700 1 -- start start 2024-06-10T01:30:26.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.994+0000 7f50e0665700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d8102740 0x7f50d8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:26.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.994+0000 7f50e0665700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50d81030f0 0x7f50d8074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:26.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.994+0000 7f50e0665700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50d8107840 0x7f50d8075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:26.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.994+0000 7f50e0665700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f50d8112780 con 0x7f50d8102740 2024-06-10T01:30:26.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.994+0000 7f50e0665700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50d8112600 con 0x7f50d81030f0 2024-06-10T01:30:26.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.994+0000 7f50e0665700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f50d8112900 con 0x7f50d8107840 2024-06-10T01:30:26.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.994+0000 7f50de401700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d8102740 0x7f50d8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:26.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.994+0000 7f50de401700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d8102740 0x7f50d8074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44028/0 (socket says 172.21.15.7:44028) 2024-06-10T01:30:26.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.994+0000 7f50de401700 1 -- 172.21.15.7:0/1601957245 learned_addr learned my addr 172.21.15.7:0/1601957245 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:26.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.994+0000 7f50dec02700 1 --2- 172.21.15.7:0/1601957245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50d8107840 0x7f50d8075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:26.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.994+0000 7f50ddc00700 1 --2- 172.21.15.7:0/1601957245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50d81030f0 0x7f50d8074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:26.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.995+0000 7f50de401700 1 -- 172.21.15.7:0/1601957245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50d8107840 msgr2=0x7f50d8075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:26.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.995+0000 7f50de401700 1 --2- 172.21.15.7:0/1601957245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50d8107840 0x7f50d8075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:26.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.995+0000 7f50de401700 1 -- 172.21.15.7:0/1601957245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50d81030f0 msgr2=0x7f50d8074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:26.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.995+0000 7f50de401700 1 --2- 172.21.15.7:0/1601957245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50d81030f0 0x7f50d8074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:26.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.995+0000 7f50de401700 1 -- 172.21.15.7:0/1601957245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d81a7060 con 0x7f50d8102740 2024-06-10T01:30:26.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.995+0000 7f50de401700 1 --2- 172.21.15.7:0/1601957245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d8102740 0x7f50d8074790 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f50d400e9c0 tx=0x7f50d400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:26.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.995+0000 7f50cf7fe700 1 -- 172.21.15.7:0/1601957245 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d400cbd0 con 0x7f50d8102740 2024-06-10T01:30:26.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.995+0000 7f50e0665700 1 -- 172.21.15.7:0/1601957245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50d81a7350 con 0x7f50d8102740 2024-06-10T01:30:26.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.995+0000 7f50cf7fe700 1 -- 172.21.15.7:0/1601957245 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50d400cd70 con 0x7f50d8102740 2024-06-10T01:30:26.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.995+0000 7f50cf7fe700 1 -- 172.21.15.7:0/1601957245 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d4010430 con 0x7f50d8102740 2024-06-10T01:30:26.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.995+0000 7f50e0665700 1 -- 172.21.15.7:0/1601957245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50d81a7840 con 0x7f50d8102740 2024-06-10T01:30:26.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.997+0000 7f50cf7fe700 1 -- 172.21.15.7:0/1601957245 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f50d4004750 con 0x7f50d8102740 2024-06-10T01:30:26.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.998+0000 7f50cf7fe700 1 --2- 172.21.15.7:0/1601957245 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f50c40a2c20 0x7f50c40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:26.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.998+0000 7f50cf7fe700 1 -- 172.21.15.7:0/1601957245 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f50d40cf660 con 0x7f50d8102740 2024-06-10T01:30:26.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.998+0000 7f50ddc00700 1 --2- 172.21.15.7:0/1601957245 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f50c40a2c20 0x7f50c40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:26.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.998+0000 7f50ddc00700 1 --2- 172.21.15.7:0/1601957245 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f50c40a2c20 0x7f50c40a50e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f50d8101f70 tx=0x7f50c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:26.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.999+0000 7f50eaee8b80 1 Processor -- start 2024-06-10T01:30:26.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.999+0000 7f50eaee8b80 1 -- start start 2024-06-10T01:30:26.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.999+0000 7f50eaee8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f6579e640 0x559f6579ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:26.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.999+0000 7f50eaee8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f6579ef60 0x559f657989e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:26.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.999+0000 7f50eaee8b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f65798f20 0x559f65799340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:26.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.999+0000 7f50eaee8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559f656ea810 con 0x559f6579e640 2024-06-10T01:30:26.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.999+0000 7f50eaee8b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559f65692720 con 0x559f65798f20 2024-06-10T01:30:26.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.999+0000 7f50eaee8b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559f656e67d0 con 0x559f6579ef60 2024-06-10T01:30:26.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.999+0000 7f50de401700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f6579e640 0x559f6579ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:26.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.999+0000 7f50de401700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f6579e640 0x559f6579ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44040/0 (socket says 172.21.15.7:44040) 2024-06-10T01:30:26.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:26.999+0000 7f50de401700 1 -- 172.21.15.7:0/3479271592 learned_addr learned my addr 172.21.15.7:0/3479271592 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:26.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.000+0000 7f50ddc00700 1 --2- 172.21.15.7:0/3479271592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f6579ef60 0x559f657989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:26.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.000+0000 7f50dec02700 1 --2- 172.21.15.7:0/3479271592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f65798f20 0x559f65799340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:26.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.000+0000 7f50de401700 1 -- 172.21.15.7:0/3479271592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f6579ef60 msgr2=0x559f657989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:26.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.000+0000 7f50de401700 1 --2- 172.21.15.7:0/3479271592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f6579ef60 0x559f657989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:26.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.000+0000 7f50de401700 1 -- 172.21.15.7:0/3479271592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f65798f20 msgr2=0x559f65799340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:26.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.000+0000 7f50de401700 1 --2- 172.21.15.7:0/3479271592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f65798f20 0x559f65799340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:26.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.000+0000 7f50de401700 1 -- 172.21.15.7:0/3479271592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f65688f50 con 0x559f6579e640 2024-06-10T01:30:26.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.000+0000 7f50dec02700 1 --2- 172.21.15.7:0/3479271592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f65798f20 0x559f65799340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:30:26.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.000+0000 7f50de401700 1 --2- 172.21.15.7:0/3479271592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f6579e640 0x559f6579ea20 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f50d4097fa0 tx=0x7f50d409b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=879d19a8978cfadb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:26.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.000+0000 7f50ccff9700 1 -- 172.21.15.7:0/3479271592 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d409bb40 con 0x559f6579e640 2024-06-10T01:30:26.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.000+0000 7f50ccff9700 1 -- 172.21.15.7:0/3479271592 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50d409fef0 con 0x559f6579e640 2024-06-10T01:30:26.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.000+0000 7f50ccff9700 1 -- 172.21.15.7:0/3479271592 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d4004a40 con 0x559f6579e640 2024-06-10T01:30:26.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.001+0000 7f50eaee8b80 1 -- 172.21.15.7:0/3479271592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f6579e640 msgr2=0x559f6579ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:26.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.001+0000 7f50eaee8b80 1 --2- 172.21.15.7:0/3479271592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f6579e640 0x559f6579ea20 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f50d4097fa0 tx=0x7f50d409b9e0 comp rx=0 tx=0).stop 2024-06-10T01:30:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.001+0000 7f50eaee8b80 1 -- 172.21.15.7:0/3479271592 shutdown_connections 2024-06-10T01:30:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.001+0000 7f50eaee8b80 1 --2- 172.21.15.7:0/3479271592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f65798f20 0x559f65799340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.001+0000 7f50eaee8b80 1 --2- 172.21.15.7:0/3479271592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f6579e640 0x559f6579ea20 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.001+0000 7f50eaee8b80 1 --2- 172.21.15.7:0/3479271592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f6579ef60 0x559f657989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.001+0000 7f50eaee8b80 1 -- 172.21.15.7:0/3479271592 >> 172.21.15.7:0/3479271592 conn(0x559f656c3cc0 msgr2=0x559f65793850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.001+0000 7f50eaee8b80 1 -- 172.21.15.7:0/3479271592 shutdown_connections 2024-06-10T01:30:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.001+0000 7f50eaee8b80 1 -- 172.21.15.7:0/3479271592 wait complete. 2024-06-10T01:30:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.002+0000 7f50eaee8b80 1 -- start start 2024-06-10T01:30:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.002+0000 7f50eaee8b80 1 Processor -- start 2024-06-10T01:30:26.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.002+0000 7f50eaee8b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f65798f20 0x559f657a4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:26.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.002+0000 7f50eaee8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f6579e640 0x559f657a1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:26.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.002+0000 7f50eaee8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f6579ef60 0x559f657a2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:26.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.002+0000 7f50eaee8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559f657a59d0 con 0x559f6579e640 2024-06-10T01:30:26.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.002+0000 7f50eaee8b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559f6568abf0 con 0x559f65798f20 2024-06-10T01:30:26.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.002+0000 7f50eaee8b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559f657a5b50 con 0x559f6579ef60 2024-06-10T01:30:26.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.002+0000 7f50eaee8b80 10 client.0 _ll_register_callbacks cb 0x559f65684920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:30:26.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.002+0000 7f50ddc00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f6579e640 0x559f657a1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:26.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.002+0000 7f50ddc00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f6579e640 0x559f657a1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44048/0 (socket says 172.21.15.7:44048) 2024-06-10T01:30:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.002+0000 7f50ddc00700 1 -- 172.21.15.7:0/22269893 learned_addr learned my addr 172.21.15.7:0/22269893 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.002+0000 7f50dec02700 1 --2- 172.21.15.7:0/22269893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f6579ef60 0x559f657a2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.002+0000 7f50de401700 1 --2- 172.21.15.7:0/22269893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f65798f20 0x559f657a4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.002+0000 7f50ddc00700 1 -- 172.21.15.7:0/22269893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f6579ef60 msgr2=0x559f657a2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.002+0000 7f50ddc00700 1 --2- 172.21.15.7:0/22269893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f6579ef60 0x559f657a2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.002+0000 7f50ddc00700 1 -- 172.21.15.7:0/22269893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f65798f20 msgr2=0x559f657a4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.002+0000 7f50ddc00700 1 --2- 172.21.15.7:0/22269893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f65798f20 0x559f657a4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.002+0000 7f50ddc00700 1 -- 172.21.15.7:0/22269893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f656398d0 con 0x559f6579e640 2024-06-10T01:30:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.003+0000 7f50ddc00700 1 --2- 172.21.15.7:0/22269893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f6579e640 0x559f657a1fd0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f50c80117c0 tx=0x7f50c80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df38ba4a8dce19b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.003+0000 7f50c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:30:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.003+0000 7f50c1ffb700 1 -- 172.21.15.7:0/22269893 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c801f840 con 0x559f6579e640 2024-06-10T01:30:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.003+0000 7f50c1ffb700 1 -- 172.21.15.7:0/22269893 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f50c801f9e0 con 0x559f6579e640 2024-06-10T01:30:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.003+0000 7f50eaee8b80 1 -- 172.21.15.7:0/22269893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50d409fef0 con 0x559f6579e640 2024-06-10T01:30:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.003+0000 7f50c1ffb700 1 -- 172.21.15.7:0/22269893 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f50c8031050 con 0x559f6579e640 2024-06-10T01:30:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.003+0000 7f50eaee8b80 10 client.21222 fetch_fsmap learned FSMap version 57 2024-06-10T01:30:27.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.003+0000 7f50eaee8b80 1 -- 172.21.15.7:0/22269893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x559f657b0270 con 0x559f6579e640 2024-06-10T01:30:27.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.003+0000 7f50c1ffb700 1 -- 172.21.15.7:0/22269893 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50d409fef0 con 0x559f6579e640 2024-06-10T01:30:27.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.003+0000 7f50c1ffb700 1 -- 172.21.15.7:0/22269893 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7f50c801a3b0 con 0x559f6579e640 2024-06-10T01:30:27.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.004+0000 7f50eaee8b80 10 client.21222 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:30:27.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.004+0000 7f50eaee8b80 10 client.21222 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:30:27.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.004+0000 7f50eaee8b80 20 client.21222 populate_metadata read hostname 'smithi007' 2024-06-10T01:30:27.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.004+0000 7f50eaee8b80 1 --2- 172.21.15.7:0/22269893 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559f657b28b0 0x559f657b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:27.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.004+0000 7f50eaee8b80 4 client.21222 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:30:27.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.004+0000 7f50eaee8b80 1 -- 172.21.15.7:0/22269893 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f656f7d10 con 0x559f657b28b0 2024-06-10T01:30:27.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.004+0000 7f50de401700 1 --2- 172.21.15.7:0/22269893 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559f657b28b0 0x559f657b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:27.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.004+0000 7f50de401700 1 --2- 172.21.15.7:0/22269893 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559f657b28b0 0x559f657b4ca0 crc :-1 s=READY pgs=545 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4252272435c80708 server_cookie=65be4de34adf5d39 in_seq=0 out_seq=0 2024-06-10T01:30:27.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.004+0000 7f50c1ffb700 10 client.21222 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:27.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.005+0000 7f50c1ffb700 1 -- 172.21.15.7:0/22269893 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559f656f7d10 con 0x559f657b28b0 2024-06-10T01:30:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.005+0000 7f50c1ffb700 10 client.21222 handle_command_reply: tid=0 2024-06-10T01:30:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.005+0000 7f50eaee8b80 1 client.21222 shutdown 2024-06-10T01:30:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.005+0000 7f50eaee8b80 1 -- 172.21.15.7:0/22269893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f6579e640 msgr2=0x559f657a1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.005+0000 7f50eaee8b80 1 --2- 172.21.15.7:0/22269893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f6579e640 0x559f657a1fd0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f50c80117c0 tx=0x7f50c80047c0 comp rx=0 tx=0).stop 2024-06-10T01:30:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.005+0000 7f50eaee8b80 1 -- 172.21.15.7:0/22269893 shutdown_connections 2024-06-10T01:30:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.005+0000 7f50eaee8b80 1 --2- 172.21.15.7:0/22269893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f6579ef60 0x559f657a2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.005+0000 7f50eaee8b80 1 --2- 172.21.15.7:0/22269893 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559f657b28b0 0x559f657b4ca0 crc :-1 s=READY pgs=545 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.005+0000 7f50eaee8b80 1 --2- 172.21.15.7:0/22269893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f65798f20 0x559f657a4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.005+0000 7f50c1ffb700 0 client.21222 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.005+0000 7f50eaee8b80 1 --2- 172.21.15.7:0/22269893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f6579e640 0x559f657a1fd0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.005+0000 7f50eaee8b80 1 -- 172.21.15.7:0/22269893 >> 172.21.15.7:0/22269893 conn(0x559f656c3cc0 msgr2=0x559f657979d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.005+0000 7f50eaee8b80 1 -- 172.21.15.7:0/22269893 shutdown_connections 2024-06-10T01:30:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.005+0000 7f50eaee8b80 1 -- 172.21.15.7:0/22269893 wait complete. 2024-06-10T01:30:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.006+0000 7f50eaee8b80 20 client.21222 trim_cache size 0 max 16384 2024-06-10T01:30:27.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.026+0000 7f50eaee8b80 1 Processor -- start 2024-06-10T01:30:27.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.026+0000 7f50eaee8b80 1 -- start start 2024-06-10T01:30:27.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.026+0000 7f50eaee8b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f65798f20 0x559f657989e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:27.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.026+0000 7f50eaee8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f6579e640 0x559f6578ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:27.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.026+0000 7f50eaee8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f6579ef60 0x559f6578d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:27.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.026+0000 7f50eaee8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f50d81b2e50 con 0x559f6579ef60 2024-06-10T01:30:27.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.026+0000 7f50eaee8b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50d81b0680 con 0x559f65798f20 2024-06-10T01:30:27.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.026+0000 7f50eaee8b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f50d81ae4d0 con 0x559f6579e640 2024-06-10T01:30:27.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.027+0000 7f50dec02700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f6579ef60 0x559f6578d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:27.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.027+0000 7f50dec02700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f6579ef60 0x559f6578d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44062/0 (socket says 172.21.15.7:44062) 2024-06-10T01:30:27.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.027+0000 7f50dec02700 1 -- 172.21.15.7:0/2452172897 learned_addr learned my addr 172.21.15.7:0/2452172897 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:27.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.027+0000 7f50de401700 1 --2- 172.21.15.7:0/2452172897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f65798f20 0x559f657989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:27.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.027+0000 7f50ddc00700 1 --2- 172.21.15.7:0/2452172897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f6579e640 0x559f6578ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:27.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.027+0000 7f50dec02700 1 -- 172.21.15.7:0/2452172897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f6579e640 msgr2=0x559f6578ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:27.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.027+0000 7f50dec02700 1 --2- 172.21.15.7:0/2452172897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f6579e640 0x559f6578ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:27.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.027+0000 7f50dec02700 1 -- 172.21.15.7:0/2452172897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f65798f20 msgr2=0x559f657989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:27.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.027+0000 7f50dec02700 1 --2- 172.21.15.7:0/2452172897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f65798f20 0x559f657989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:27.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.027+0000 7f50dec02700 1 -- 172.21.15.7:0/2452172897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f657a2ed0 con 0x559f6579ef60 2024-06-10T01:30:27.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.027+0000 7f50dec02700 1 --2- 172.21.15.7:0/2452172897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f6579ef60 0x559f6578d340 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f50d0009390 tx=0x7f50d00093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a898c758ea31756 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:27.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.028+0000 7f50c3fff700 1 -- 172.21.15.7:0/2452172897 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d000bca0 con 0x559f6579ef60 2024-06-10T01:30:27.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.028+0000 7f50c3fff700 1 -- 172.21.15.7:0/2452172897 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50d0022070 con 0x559f6579ef60 2024-06-10T01:30:27.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.028+0000 7f50c3fff700 1 -- 172.21.15.7:0/2452172897 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d0007e10 con 0x559f6579ef60 2024-06-10T01:30:27.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.028+0000 7f50eaee8b80 1 -- 172.21.15.7:0/2452172897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f6579ef60 msgr2=0x559f6578d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:27.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.028+0000 7f50eaee8b80 1 --2- 172.21.15.7:0/2452172897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f6579ef60 0x559f6578d340 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f50d0009390 tx=0x7f50d00093c0 comp rx=0 tx=0).stop 2024-06-10T01:30:27.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.028+0000 7f50eaee8b80 1 -- 172.21.15.7:0/2452172897 shutdown_connections 2024-06-10T01:30:27.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.028+0000 7f50eaee8b80 1 --2- 172.21.15.7:0/2452172897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f6579ef60 0x559f6578d340 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:27.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.028+0000 7f50eaee8b80 1 --2- 172.21.15.7:0/2452172897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f65798f20 0x559f657989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:27.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.028+0000 7f50eaee8b80 1 --2- 172.21.15.7:0/2452172897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f6579e640 0x559f6578ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:27.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.028+0000 7f50eaee8b80 1 -- 172.21.15.7:0/2452172897 >> 172.21.15.7:0/2452172897 conn(0x559f656c3cc0 msgr2=0x559f65793380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:27.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.029+0000 7f50eaee8b80 1 -- 172.21.15.7:0/2452172897 shutdown_connections 2024-06-10T01:30:27.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.029+0000 7f50eaee8b80 1 -- 172.21.15.7:0/2452172897 wait complete. 2024-06-10T01:30:27.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.029+0000 7f50eaee8b80 1 -- start start 2024-06-10T01:30:27.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.029+0000 7f50eaee8b80 1 Processor -- start 2024-06-10T01:30:27.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.029+0000 7f50eaee8b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f65798f20 0x559f657aade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:27.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.029+0000 7f50eaee8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f6579e640 0x559f657ab320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:27.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.029+0000 7f50eaee8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f6579ef60 0x559f657a7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:27.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.029+0000 7f50eaee8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f50d81aeb40 con 0x559f6579e640 2024-06-10T01:30:27.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.029+0000 7f50eaee8b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50d81affb0 con 0x559f65798f20 2024-06-10T01:30:27.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.029+0000 7f50eaee8b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f50d81af210 con 0x559f6579ef60 2024-06-10T01:30:27.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.029+0000 7f50eaee8b80 10 client.0 _ll_register_callbacks cb 0x559f65684920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:30:27.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.029+0000 7f50ddc00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f6579e640 0x559f657ab320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:27.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.029+0000 7f50ddc00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f6579e640 0x559f657ab320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44078/0 (socket says 172.21.15.7:44078) 2024-06-10T01:30:27.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.029+0000 7f50ddc00700 1 -- 172.21.15.7:0/2871483114 learned_addr learned my addr 172.21.15.7:0/2871483114 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:27.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.030+0000 7f50de401700 1 --2- 172.21.15.7:0/2871483114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f65798f20 0x559f657aade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:27.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.030+0000 7f50dec02700 1 --2- 172.21.15.7:0/2871483114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f6579ef60 0x559f657a7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:27.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.030+0000 7f50ddc00700 1 -- 172.21.15.7:0/2871483114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f6579ef60 msgr2=0x559f657a7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:27.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.030+0000 7f50ddc00700 1 --2- 172.21.15.7:0/2871483114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f6579ef60 0x559f657a7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.030+0000 7f50ddc00700 1 -- 172.21.15.7:0/2871483114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f65798f20 msgr2=0x559f657aade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.030+0000 7f50ddc00700 1 --2- 172.21.15.7:0/2871483114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f65798f20 0x559f657aade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.030+0000 7f50ddc00700 1 -- 172.21.15.7:0/2871483114 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f657a83a0 con 0x559f6579e640 2024-06-10T01:30:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.030+0000 7f50dec02700 1 --2- 172.21.15.7:0/2871483114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f6579ef60 0x559f657a7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:30:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.030+0000 7f50de401700 1 --2- 172.21.15.7:0/2871483114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f65798f20 0x559f657aade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:30:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.030+0000 7f50ddc00700 1 --2- 172.21.15.7:0/2871483114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f6579e640 0x559f657ab320 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f50c8010b40 tx=0x7f50c8011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37690753431c94c6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.030+0000 7f50c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:30:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.030+0000 7f50c1ffb700 1 -- 172.21.15.7:0/2871483114 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c803a030 con 0x559f6579e640 2024-06-10T01:30:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.030+0000 7f50c1ffb700 1 -- 172.21.15.7:0/2871483114 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f50c801ae40 con 0x559f6579e640 2024-06-10T01:30:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.030+0000 7f50eaee8b80 1 -- 172.21.15.7:0/2871483114 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50d0022070 con 0x559f6579e640 2024-06-10T01:30:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.030+0000 7f50c1ffb700 1 -- 172.21.15.7:0/2871483114 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f50c8047050 con 0x559f6579e640 2024-06-10T01:30:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.030+0000 7f50eaee8b80 10 client.21228 fetch_fsmap learned FSMap version 57 2024-06-10T01:30:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.030+0000 7f50eaee8b80 1 -- 172.21.15.7:0/2871483114 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x559f6581ddb0 con 0x559f6579e640 2024-06-10T01:30:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.031+0000 7f50c1ffb700 1 -- 172.21.15.7:0/2871483114 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50d0022070 con 0x559f6579e640 2024-06-10T01:30:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.031+0000 7f50c1ffb700 1 -- 172.21.15.7:0/2871483114 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7f50c8034040 con 0x559f6579e640 2024-06-10T01:30:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.031+0000 7f50eaee8b80 10 client.21228 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:30:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.031+0000 7f50eaee8b80 10 client.21228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:30:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.031+0000 7f50eaee8b80 20 client.21228 populate_metadata read hostname 'smithi007' 2024-06-10T01:30:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.031+0000 7f50eaee8b80 1 --2- 172.21.15.7:0/2871483114 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559f657b28b0 0x559f65820590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.031+0000 7f50eaee8b80 4 client.21228 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:30:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.031+0000 7f50eaee8b80 1 -- 172.21.15.7:0/2871483114 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559f656f2290 con 0x559f657b28b0 2024-06-10T01:30:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.031+0000 7f50de401700 1 --2- 172.21.15.7:0/2871483114 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559f657b28b0 0x559f65820590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.032+0000 7f50c1ffb700 10 client.21228 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.032+0000 7f50de401700 1 --2- 172.21.15.7:0/2871483114 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559f657b28b0 0x559f65820590 crc :-1 s=READY pgs=546 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6ee7770c80fd070 server_cookie=e6957366a1448dc9 in_seq=0 out_seq=0 2024-06-10T01:30:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.032+0000 7f50c1ffb700 1 -- 172.21.15.7:0/2871483114 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559f656f2290 con 0x559f657b28b0 2024-06-10T01:30:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.032+0000 7f50c1ffb700 10 client.21228 handle_command_reply: tid=0 2024-06-10T01:30:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.032+0000 7f50eaee8b80 1 client.21228 shutdown 2024-06-10T01:30:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.032+0000 7f50eaee8b80 1 -- 172.21.15.7:0/2871483114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f6579e640 msgr2=0x559f657ab320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.032+0000 7f50eaee8b80 1 --2- 172.21.15.7:0/2871483114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f6579e640 0x559f657ab320 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f50c8010b40 tx=0x7f50c8011db0 comp rx=0 tx=0).stop 2024-06-10T01:30:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.032+0000 7f50eaee8b80 1 -- 172.21.15.7:0/2871483114 shutdown_connections 2024-06-10T01:30:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.032+0000 7f50eaee8b80 1 --2- 172.21.15.7:0/2871483114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f6579ef60 0x559f657a7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.032+0000 7f50eaee8b80 1 --2- 172.21.15.7:0/2871483114 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x559f657b28b0 0x559f65820590 crc :-1 s=READY pgs=546 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.032+0000 7f50eaee8b80 1 --2- 172.21.15.7:0/2871483114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f65798f20 0x559f657aade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.032+0000 7f50eaee8b80 1 --2- 172.21.15.7:0/2871483114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f6579e640 0x559f657ab320 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.032+0000 7f50eaee8b80 1 -- 172.21.15.7:0/2871483114 >> 172.21.15.7:0/2871483114 conn(0x559f656c3cc0 msgr2=0x559f6578ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.032+0000 7f50c1ffb700 0 client.21228 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.032+0000 7f50eaee8b80 1 -- 172.21.15.7:0/2871483114 shutdown_connections 2024-06-10T01:30:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.032+0000 7f50eaee8b80 1 -- 172.21.15.7:0/2871483114 wait complete. 2024-06-10T01:30:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.032+0000 7f50eaee8b80 20 client.21228 trim_cache size 0 max 16384 2024-06-10T01:30:27.029 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:30:27.029 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T01:30:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-10T01:30:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: "5b2dce0c-b7c0-4ac0-ab7c-96a0b18216bc": { 2024-06-10T01:30:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-10T01:30:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "5b2dce0c-b7c0-4ac0-ab7c-96a0b18216bc", 2024-06-10T01:30:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-10T01:30:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:30:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:30:27.030 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:30:27.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.033+0000 7f50e0665700 1 -- 172.21.15.7:0/1601957245 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f50c40a2c20 msgr2=0x7f50c40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:27.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.033+0000 7f50e0665700 1 --2- 172.21.15.7:0/1601957245 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f50c40a2c20 0x7f50c40a50e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f50d8101f70 tx=0x7f50c8009260 comp rx=0 tx=0).stop 2024-06-10T01:30:27.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.033+0000 7f50e0665700 1 -- 172.21.15.7:0/1601957245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d8102740 msgr2=0x7f50d8074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:27.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.033+0000 7f50e0665700 1 --2- 172.21.15.7:0/1601957245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d8102740 0x7f50d8074790 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f50d400e9c0 tx=0x7f50d400ecd0 comp rx=0 tx=0).stop 2024-06-10T01:30:27.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.033+0000 7f50e0665700 1 -- 172.21.15.7:0/1601957245 shutdown_connections 2024-06-10T01:30:27.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.033+0000 7f50e0665700 1 --2- 172.21.15.7:0/1601957245 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f50c40a2c20 0x7f50c40a50e0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:27.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.033+0000 7f50e0665700 1 --2- 172.21.15.7:0/1601957245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50d8107840 0x7f50d8075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:27.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.033+0000 7f50e0665700 1 --2- 172.21.15.7:0/1601957245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d8102740 0x7f50d8074790 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:27.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.033+0000 7f50e0665700 1 --2- 172.21.15.7:0/1601957245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50d81030f0 0x7f50d8074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:27.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.033+0000 7f50e0665700 1 -- 172.21.15.7:0/1601957245 >> 172.21.15.7:0/1601957245 conn(0x7f50d80795f0 msgr2=0x7f50d8100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:27.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.033+0000 7f50e0665700 1 -- 172.21.15.7:0/1601957245 shutdown_connections 2024-06-10T01:30:27.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:27.033+0000 7f50e0665700 1 -- 172.21.15.7:0/1601957245 wait complete. 2024-06-10T01:30:27.242 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:27 smithi007 ceph-mon[32398]: pgmap v1167: 129 pgs: 129 active+clean; 666 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 1.8 MiB/s wr, 242 op/s 2024-06-10T01:30:27.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:27 smithi175 ceph-mon[36636]: pgmap v1167: 129 pgs: 129 active+clean; 666 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 1.8 MiB/s wr, 242 op/s 2024-06-10T01:30:27.567 INFO:tasks.workunit.client.0.smithi007.stdout: 10 256380 21.38 MB/sec execute 544 sec latency 18986.029 ms 2024-06-10T01:30:27.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:27 smithi078 ceph-mon[36683]: pgmap v1167: 129 pgs: 129 active+clean; 666 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 1.8 MiB/s wr, 242 op/s 2024-06-10T01:30:28.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.400+0000 7fcb6fe93700 1 -- 172.21.15.7:0/1218691637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb68101620 msgr2=0x7fcb68101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:28.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.400+0000 7fcb6fe93700 1 --2- 172.21.15.7:0/1218691637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb68101620 0x7fcb68101a00 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fcb64009990 tx=0x7fcb64009ca0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.400+0000 7fcb6fe93700 1 -- 172.21.15.7:0/1218691637 shutdown_connections 2024-06-10T01:30:28.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.400+0000 7fcb6fe93700 1 --2- 172.21.15.7:0/1218691637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb681066e0 0x7fcb6810e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.400+0000 7fcb6fe93700 1 --2- 172.21.15.7:0/1218691637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb68101620 0x7fcb68101a00 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.400+0000 7fcb6fe93700 1 --2- 172.21.15.7:0/1218691637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb68101fd0 0x7fcb68106000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.400+0000 7fcb6fe93700 1 -- 172.21.15.7:0/1218691637 >> 172.21.15.7:0/1218691637 conn(0x7fcb68078330 msgr2=0x7fcb68078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:28.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.400+0000 7fcb6fe93700 1 -- 172.21.15.7:0/1218691637 shutdown_connections 2024-06-10T01:30:28.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.400+0000 7fcb6fe93700 1 -- 172.21.15.7:0/1218691637 wait complete. 2024-06-10T01:30:28.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.401+0000 7fcb6fe93700 1 Processor -- start 2024-06-10T01:30:28.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.401+0000 7fcb6fe93700 1 -- start start 2024-06-10T01:30:28.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.401+0000 7fcb6fe93700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb68101620 0x7fcb681030a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:28.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.401+0000 7fcb6fe93700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb68101fd0 0x7fcb681035e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:28.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.401+0000 7fcb6fe93700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb681066e0 0x7fcb68103b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:28.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.401+0000 7fcb6fe93700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcb681116d0 con 0x7fcb68101fd0 2024-06-10T01:30:28.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.401+0000 7fcb6fe93700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcb68111550 con 0x7fcb681066e0 2024-06-10T01:30:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.401+0000 7fcb6fe93700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcb68111850 con 0x7fcb68101620 2024-06-10T01:30:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.401+0000 7fcb6d42e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb68101fd0 0x7fcb681035e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.401+0000 7fcb6d42e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb68101fd0 0x7fcb681035e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44096/0 (socket says 172.21.15.7:44096) 2024-06-10T01:30:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.401+0000 7fcb6d42e700 1 -- 172.21.15.7:0/1069759424 learned_addr learned my addr 172.21.15.7:0/1069759424 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.401+0000 7fcb6dc2f700 1 --2- 172.21.15.7:0/1069759424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb68101620 0x7fcb681030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.401+0000 7fcb6e430700 1 --2- 172.21.15.7:0/1069759424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb681066e0 0x7fcb68103b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.401+0000 7fcb6d42e700 1 -- 172.21.15.7:0/1069759424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb68101620 msgr2=0x7fcb681030a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.401+0000 7fcb6d42e700 1 --2- 172.21.15.7:0/1069759424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb68101620 0x7fcb681030a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.401+0000 7fcb6d42e700 1 -- 172.21.15.7:0/1069759424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb681066e0 msgr2=0x7fcb68103b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.401+0000 7fcb6d42e700 1 --2- 172.21.15.7:0/1069759424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb681066e0 0x7fcb68103b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.401+0000 7fcb6d42e700 1 -- 172.21.15.7:0/1069759424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb681a71c0 con 0x7fcb68101fd0 2024-06-10T01:30:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.402+0000 7fcb6d42e700 1 --2- 172.21.15.7:0/1069759424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb68101fd0 0x7fcb681035e0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800d980 tx=0x7fcb5800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.402+0000 7fcb5effd700 1 -- 172.21.15.7:0/1069759424 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb58017070 con 0x7fcb68101fd0 2024-06-10T01:30:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.402+0000 7fcb5effd700 1 -- 172.21.15.7:0/1069759424 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb58009e10 con 0x7fcb68101fd0 2024-06-10T01:30:28.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.402+0000 7fcb6fe93700 1 -- 172.21.15.7:0/1069759424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb681a74b0 con 0x7fcb68101fd0 2024-06-10T01:30:28.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.402+0000 7fcb5effd700 1 -- 172.21.15.7:0/1069759424 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5800f620 con 0x7fcb68101fd0 2024-06-10T01:30:28.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.402+0000 7fcb6fe93700 1 -- 172.21.15.7:0/1069759424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb681a7a40 con 0x7fcb68101fd0 2024-06-10T01:30:28.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.404+0000 7fcb5effd700 1 -- 172.21.15.7:0/1069759424 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fcb58010460 con 0x7fcb68101fd0 2024-06-10T01:30:28.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.405+0000 7fcb5effd700 1 --2- 172.21.15.7:0/1069759424 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcb540a2ca0 0x7fcb540a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:28.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.405+0000 7fcb6dc2f700 1 --2- 172.21.15.7:0/1069759424 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcb540a2ca0 0x7fcb540a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:28.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.405+0000 7fcb5effd700 1 -- 172.21.15.7:0/1069759424 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fcb580ceee0 con 0x7fcb68101fd0 2024-06-10T01:30:28.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.405+0000 7fcb6dc2f700 1 --2- 172.21.15.7:0/1069759424 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcb540a2ca0 0x7fcb540a5160 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fcb640098e0 tx=0x7fcb64005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:28.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.406+0000 7fcb7a716b80 1 Processor -- start 2024-06-10T01:30:28.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.406+0000 7fcb7a716b80 1 -- start start 2024-06-10T01:30:28.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.406+0000 7fcb7a716b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584d93b0500 0x5584d93b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:28.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.406+0000 7fcb7a716b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d93b0e20 0x5584d93aa710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:28.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.406+0000 7fcb7a716b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584d93aac50 0x5584d93ab070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:28.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.406+0000 7fcb7a716b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5584d929d060 con 0x5584d93b0500 2024-06-10T01:30:28.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.406+0000 7fcb7a716b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5584d92fb950 con 0x5584d93b0e20 2024-06-10T01:30:28.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.406+0000 7fcb7a716b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5584d9263560 con 0x5584d93aac50 2024-06-10T01:30:28.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.406+0000 7fcb6dc2f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584d93b0500 0x5584d93b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:28.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.406+0000 7fcb6dc2f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584d93b0500 0x5584d93b08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44100/0 (socket says 172.21.15.7:44100) 2024-06-10T01:30:28.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.406+0000 7fcb6dc2f700 1 -- 172.21.15.7:0/2442640409 learned_addr learned my addr 172.21.15.7:0/2442640409 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:28.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.406+0000 7fcb6e430700 1 --2- 172.21.15.7:0/2442640409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584d93aac50 0x5584d93ab070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:28.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.406+0000 7fcb6d42e700 1 --2- 172.21.15.7:0/2442640409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d93b0e20 0x5584d93aa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:28.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.407+0000 7fcb6dc2f700 1 -- 172.21.15.7:0/2442640409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584d93aac50 msgr2=0x5584d93ab070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:28.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.407+0000 7fcb6dc2f700 1 --2- 172.21.15.7:0/2442640409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584d93aac50 0x5584d93ab070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.407+0000 7fcb6dc2f700 1 -- 172.21.15.7:0/2442640409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d93b0e20 msgr2=0x5584d93aa710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:28.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.407+0000 7fcb6dc2f700 1 --2- 172.21.15.7:0/2442640409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d93b0e20 0x5584d93aa710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.407+0000 7fcb6dc2f700 1 -- 172.21.15.7:0/2442640409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584d924a9d0 con 0x5584d93b0500 2024-06-10T01:30:28.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.407+0000 7fcb6d42e700 1 --2- 172.21.15.7:0/2442640409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d93b0e20 0x5584d93aa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:30:28.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.407+0000 7fcb6e430700 1 --2- 172.21.15.7:0/2442640409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584d93aac50 0x5584d93ab070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:30:28.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.407+0000 7fcb6dc2f700 1 --2- 172.21.15.7:0/2442640409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584d93b0500 0x5584d93b08e0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fcb64015000 tx=0x7fcb64020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a143ddf4584520b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:28.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.407+0000 7fcb53fff700 1 -- 172.21.15.7:0/2442640409 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb64003780 con 0x5584d93b0500 2024-06-10T01:30:28.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.407+0000 7fcb53fff700 1 -- 172.21.15.7:0/2442640409 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb6400bdf0 con 0x5584d93b0500 2024-06-10T01:30:28.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.407+0000 7fcb53fff700 1 -- 172.21.15.7:0/2442640409 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb64024d50 con 0x5584d93b0500 2024-06-10T01:30:28.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.408+0000 7fcb7a716b80 1 -- 172.21.15.7:0/2442640409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584d93b0500 msgr2=0x5584d93b08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:28.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.408+0000 7fcb7a716b80 1 --2- 172.21.15.7:0/2442640409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584d93b0500 0x5584d93b08e0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fcb64015000 tx=0x7fcb64020000 comp rx=0 tx=0).stop 2024-06-10T01:30:28.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.408+0000 7fcb7a716b80 1 -- 172.21.15.7:0/2442640409 shutdown_connections 2024-06-10T01:30:28.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.408+0000 7fcb7a716b80 1 --2- 172.21.15.7:0/2442640409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584d93aac50 0x5584d93ab070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.408+0000 7fcb7a716b80 1 --2- 172.21.15.7:0/2442640409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584d93b0500 0x5584d93b08e0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.408+0000 7fcb7a716b80 1 --2- 172.21.15.7:0/2442640409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d93b0e20 0x5584d93aa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.408+0000 7fcb7a716b80 1 -- 172.21.15.7:0/2442640409 >> 172.21.15.7:0/2442640409 conn(0x5584d92d7180 msgr2=0x5584d93a5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:28.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.408+0000 7fcb7a716b80 1 -- 172.21.15.7:0/2442640409 shutdown_connections 2024-06-10T01:30:28.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.408+0000 7fcb7a716b80 1 -- 172.21.15.7:0/2442640409 wait complete. 2024-06-10T01:30:28.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.408+0000 7fcb7a716b80 1 -- start start 2024-06-10T01:30:28.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.408+0000 7fcb7a716b80 1 Processor -- start 2024-06-10T01:30:28.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.409+0000 7fcb7a716b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d93aac50 0x5584d93b6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:28.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.409+0000 7fcb7a716b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584d93b0500 0x5584d93b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:28.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.409+0000 7fcb7a716b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584d93b0e20 0x5584d93b4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:28.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.409+0000 7fcb7a716b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5584d929c0c0 con 0x5584d93b0500 2024-06-10T01:30:28.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.409+0000 7fcb7a716b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5584d92f7910 con 0x5584d93aac50 2024-06-10T01:30:28.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.409+0000 7fcb7a716b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5584d93b7740 con 0x5584d93b0e20 2024-06-10T01:30:28.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.409+0000 7fcb7a716b80 10 client.0 _ll_register_callbacks cb 0x5584d92916b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:30:28.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.409+0000 7fcb6d42e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584d93b0500 0x5584d93b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:28.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.409+0000 7fcb6d42e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584d93b0500 0x5584d93b3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44106/0 (socket says 172.21.15.7:44106) 2024-06-10T01:30:28.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.409+0000 7fcb6d42e700 1 -- 172.21.15.7:0/1536354497 learned_addr learned my addr 172.21.15.7:0/1536354497 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:28.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.409+0000 7fcb6e430700 1 --2- 172.21.15.7:0/1536354497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584d93b0e20 0x5584d93b4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:28.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.409+0000 7fcb6d42e700 1 -- 172.21.15.7:0/1536354497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584d93b0e20 msgr2=0x5584d93b4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:28.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.409+0000 7fcb6d42e700 1 --2- 172.21.15.7:0/1536354497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584d93b0e20 0x5584d93b4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.409+0000 7fcb6d42e700 1 -- 172.21.15.7:0/1536354497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d93aac50 msgr2=0x5584d93b6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:28.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.409+0000 7fcb6d42e700 1 --2- 172.21.15.7:0/1536354497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d93aac50 0x5584d93b6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.409+0000 7fcb6d42e700 1 -- 172.21.15.7:0/1536354497 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584d92c1850 con 0x5584d93b0500 2024-06-10T01:30:28.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.410+0000 7fcb6d42e700 1 --2- 172.21.15.7:0/1536354497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584d93b0500 0x5584d93b3dd0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800d950 tx=0x7fcb5809bbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c88c818bdeeb74ac server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:28.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.410+0000 7fcb517fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:30:28.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.410+0000 7fcb517fa700 1 -- 172.21.15.7:0/1536354497 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb58020040 con 0x5584d93b0500 2024-06-10T01:30:28.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.410+0000 7fcb517fa700 1 -- 172.21.15.7:0/1536354497 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fcb580a3ba0 con 0x5584d93b0500 2024-06-10T01:30:28.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.410+0000 7fcb7a716b80 1 -- 172.21.15.7:0/1536354497 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb6400bdf0 con 0x5584d93b0500 2024-06-10T01:30:28.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.410+0000 7fcb517fa700 1 -- 172.21.15.7:0/1536354497 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fcb580a4030 con 0x5584d93b0500 2024-06-10T01:30:28.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.410+0000 7fcb7a716b80 10 client.21240 fetch_fsmap learned FSMap version 57 2024-06-10T01:30:28.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.410+0000 7fcb7a716b80 1 -- 172.21.15.7:0/1536354497 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5584d93c1ed0 con 0x5584d93b0500 2024-06-10T01:30:28.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.410+0000 7fcb517fa700 1 -- 172.21.15.7:0/1536354497 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb6400bdf0 con 0x5584d93b0500 2024-06-10T01:30:28.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.410+0000 7fcb517fa700 1 -- 172.21.15.7:0/1536354497 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7fcb58009930 con 0x5584d93b0500 2024-06-10T01:30:28.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.410+0000 7fcb7a716b80 10 client.21240 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:30:28.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.411+0000 7fcb7a716b80 10 client.21240 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:30:28.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.411+0000 7fcb7a716b80 20 client.21240 populate_metadata read hostname 'smithi007' 2024-06-10T01:30:28.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.411+0000 7fcb7a716b80 1 --2- 172.21.15.7:0/1536354497 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5584d93c4500 0x5584d93c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:28.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.411+0000 7fcb7a716b80 4 client.21240 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:30:28.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.411+0000 7fcb7a716b80 1 -- 172.21.15.7:0/1536354497 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584d9308e50 con 0x5584d93c4500 2024-06-10T01:30:28.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.411+0000 7fcb6dc2f700 1 --2- 172.21.15.7:0/1536354497 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5584d93c4500 0x5584d93c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:28.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.411+0000 7fcb517fa700 10 client.21240 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:28.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.411+0000 7fcb6dc2f700 1 --2- 172.21.15.7:0/1536354497 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5584d93c4500 0x5584d93c68f0 crc :-1 s=READY pgs=547 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f88c1b140a5ee0dd server_cookie=eaa5e286b6e41e7 in_seq=0 out_seq=0 2024-06-10T01:30:28.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.412+0000 7fcb517fa700 1 -- 172.21.15.7:0/1536354497 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5584d9308e50 con 0x5584d93c4500 2024-06-10T01:30:28.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.412+0000 7fcb517fa700 10 client.21240 handle_command_reply: tid=0 2024-06-10T01:30:28.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.412+0000 7fcb7a716b80 1 client.21240 shutdown 2024-06-10T01:30:28.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.412+0000 7fcb7a716b80 1 -- 172.21.15.7:0/1536354497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584d93b0500 msgr2=0x5584d93b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:28.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.412+0000 7fcb7a716b80 1 --2- 172.21.15.7:0/1536354497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584d93b0500 0x5584d93b3dd0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800d950 tx=0x7fcb5809bbf0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.412+0000 7fcb7a716b80 1 -- 172.21.15.7:0/1536354497 shutdown_connections 2024-06-10T01:30:28.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.412+0000 7fcb7a716b80 1 --2- 172.21.15.7:0/1536354497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584d93b0e20 0x5584d93b4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.412+0000 7fcb7a716b80 1 --2- 172.21.15.7:0/1536354497 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5584d93c4500 0x5584d93c68f0 crc :-1 s=READY pgs=547 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.412+0000 7fcb7a716b80 1 --2- 172.21.15.7:0/1536354497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d93aac50 0x5584d93b6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.412+0000 7fcb517fa700 0 client.21240 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:28.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.412+0000 7fcb7a716b80 1 --2- 172.21.15.7:0/1536354497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584d93b0500 0x5584d93b3dd0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.412+0000 7fcb7a716b80 1 -- 172.21.15.7:0/1536354497 >> 172.21.15.7:0/1536354497 conn(0x5584d92d7180 msgr2=0x5584d93a4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:28.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.412+0000 7fcb7a716b80 1 -- 172.21.15.7:0/1536354497 shutdown_connections 2024-06-10T01:30:28.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.413+0000 7fcb7a716b80 1 -- 172.21.15.7:0/1536354497 wait complete. 2024-06-10T01:30:28.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.413+0000 7fcb7a716b80 20 client.21240 trim_cache size 0 max 16384 2024-06-10T01:30:28.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.433+0000 7fcb7a716b80 1 Processor -- start 2024-06-10T01:30:28.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.433+0000 7fcb7a716b80 1 -- start start 2024-06-10T01:30:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.433+0000 7fcb7a716b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584d93aac50 0x5584d93b6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.433+0000 7fcb7a716b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584d93b0500 0x5584d93a9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.433+0000 7fcb7a716b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d93b0e20 0x5584d93aa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.433+0000 7fcb7a716b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcb681b2e80 con 0x5584d93aac50 2024-06-10T01:30:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.433+0000 7fcb7a716b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcb681b06b0 con 0x5584d93b0e20 2024-06-10T01:30:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.433+0000 7fcb7a716b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcb681ae500 con 0x5584d93b0500 2024-06-10T01:30:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.433+0000 7fcb6dc2f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584d93aac50 0x5584d93b6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.433+0000 7fcb6dc2f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584d93aac50 0x5584d93b6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44108/0 (socket says 172.21.15.7:44108) 2024-06-10T01:30:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.433+0000 7fcb6dc2f700 1 -- 172.21.15.7:0/2433096181 learned_addr learned my addr 172.21.15.7:0/2433096181 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.434+0000 7fcb6e430700 1 --2- 172.21.15.7:0/2433096181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d93b0e20 0x5584d93aa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.434+0000 7fcb6d42e700 1 --2- 172.21.15.7:0/2433096181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584d93b0500 0x5584d93a9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.434+0000 7fcb6dc2f700 1 -- 172.21.15.7:0/2433096181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584d93b0500 msgr2=0x5584d93a9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.434+0000 7fcb6dc2f700 1 --2- 172.21.15.7:0/2433096181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584d93b0500 0x5584d93a9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.434+0000 7fcb6dc2f700 1 -- 172.21.15.7:0/2433096181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d93b0e20 msgr2=0x5584d93aa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.434+0000 7fcb6dc2f700 1 --2- 172.21.15.7:0/2433096181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d93b0e20 0x5584d93aa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.434+0000 7fcb6dc2f700 1 -- 172.21.15.7:0/2433096181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584d93b2360 con 0x5584d93aac50 2024-06-10T01:30:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.434+0000 7fcb6e430700 1 --2- 172.21.15.7:0/2433096181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d93b0e20 0x5584d93aa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:30:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.434+0000 7fcb6dc2f700 1 --2- 172.21.15.7:0/2433096181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584d93aac50 0x5584d93b6610 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fcb640059d0 tx=0x7fcb6401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f8dd83b34a85c50 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.434+0000 7fcb537fe700 1 -- 172.21.15.7:0/2433096181 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb640245d0 con 0x5584d93aac50 2024-06-10T01:30:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.434+0000 7fcb537fe700 1 -- 172.21.15.7:0/2433096181 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb64024e40 con 0x5584d93aac50 2024-06-10T01:30:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.434+0000 7fcb537fe700 1 -- 172.21.15.7:0/2433096181 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb64039630 con 0x5584d93aac50 2024-06-10T01:30:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.435+0000 7fcb7a716b80 1 -- 172.21.15.7:0/2433096181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584d93aac50 msgr2=0x5584d93b6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.435+0000 7fcb7a716b80 1 --2- 172.21.15.7:0/2433096181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584d93aac50 0x5584d93b6610 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fcb640059d0 tx=0x7fcb6401a040 comp rx=0 tx=0).stop 2024-06-10T01:30:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.435+0000 7fcb7a716b80 1 -- 172.21.15.7:0/2433096181 shutdown_connections 2024-06-10T01:30:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.435+0000 7fcb7a716b80 1 --2- 172.21.15.7:0/2433096181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d93b0e20 0x5584d93aa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.435+0000 7fcb7a716b80 1 --2- 172.21.15.7:0/2433096181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584d93aac50 0x5584d93b6610 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.435+0000 7fcb7a716b80 1 --2- 172.21.15.7:0/2433096181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584d93b0500 0x5584d93a9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.435+0000 7fcb7a716b80 1 -- 172.21.15.7:0/2433096181 >> 172.21.15.7:0/2433096181 conn(0x5584d92d7180 msgr2=0x5584d93a9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.435+0000 7fcb7a716b80 1 -- 172.21.15.7:0/2433096181 shutdown_connections 2024-06-10T01:30:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.435+0000 7fcb7a716b80 1 -- 172.21.15.7:0/2433096181 wait complete. 2024-06-10T01:30:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.435+0000 7fcb7a716b80 1 -- start start 2024-06-10T01:30:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.435+0000 7fcb7a716b80 1 Processor -- start 2024-06-10T01:30:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.436+0000 7fcb7a716b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584d93aac50 0x5584d93bd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.436+0000 7fcb7a716b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584d93b0500 0x5584d93b9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.436+0000 7fcb7a716b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d93b0e20 0x5584d93ba040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.436+0000 7fcb7a716b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcb681aeb70 con 0x5584d93b0500 2024-06-10T01:30:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.436+0000 7fcb7a716b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcb681affe0 con 0x5584d93b0e20 2024-06-10T01:30:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.436+0000 7fcb7a716b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcb681af240 con 0x5584d93aac50 2024-06-10T01:30:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.436+0000 7fcb7a716b80 10 client.0 _ll_register_callbacks cb 0x5584d92916b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:30:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.436+0000 7fcb6d42e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584d93b0500 0x5584d93b9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.436+0000 7fcb6d42e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584d93b0500 0x5584d93b9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44110/0 (socket says 172.21.15.7:44110) 2024-06-10T01:30:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.436+0000 7fcb6d42e700 1 -- 172.21.15.7:0/1093485617 learned_addr learned my addr 172.21.15.7:0/1093485617 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.436+0000 7fcb6dc2f700 1 --2- 172.21.15.7:0/1093485617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584d93aac50 0x5584d93bd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.436+0000 7fcb6e430700 1 --2- 172.21.15.7:0/1093485617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d93b0e20 0x5584d93ba040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.436+0000 7fcb6d42e700 1 -- 172.21.15.7:0/1093485617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584d93aac50 msgr2=0x5584d93bd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:28.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.436+0000 7fcb6d42e700 1 --2- 172.21.15.7:0/1093485617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584d93aac50 0x5584d93bd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.436+0000 7fcb6d42e700 1 -- 172.21.15.7:0/1093485617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d93b0e20 msgr2=0x5584d93ba040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:28.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.436+0000 7fcb6d42e700 1 --2- 172.21.15.7:0/1093485617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d93b0e20 0x5584d93ba040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.436+0000 7fcb6d42e700 1 -- 172.21.15.7:0/1093485617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584d942f1e0 con 0x5584d93b0500 2024-06-10T01:30:28.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.436+0000 7fcb6e430700 1 --2- 172.21.15.7:0/1093485617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d93b0e20 0x5584d93ba040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:30:28.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.437+0000 7fcb6d42e700 1 --2- 172.21.15.7:0/1093485617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584d93b0500 0x5584d93b9a30 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009520 tx=0x7fcb580b44b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256a2cfda5b76374 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:28.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.437+0000 7fcb517fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:30:28.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.437+0000 7fcb517fa700 1 -- 172.21.15.7:0/1093485617 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb58020070 con 0x5584d93b0500 2024-06-10T01:30:28.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.437+0000 7fcb517fa700 1 -- 172.21.15.7:0/1093485617 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fcb580a3da0 con 0x5584d93b0500 2024-06-10T01:30:28.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.437+0000 7fcb7a716b80 1 -- 172.21.15.7:0/1093485617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5584d93033d0 con 0x5584d93b0500 2024-06-10T01:30:28.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.437+0000 7fcb517fa700 1 -- 172.21.15.7:0/1093485617 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fcb5809a810 con 0x5584d93b0500 2024-06-10T01:30:28.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.437+0000 7fcb7a716b80 10 client.21246 fetch_fsmap learned FSMap version 57 2024-06-10T01:30:28.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.437+0000 7fcb7a716b80 1 -- 172.21.15.7:0/1093485617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5584d9430360 con 0x5584d93b0500 2024-06-10T01:30:28.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.437+0000 7fcb517fa700 1 -- 172.21.15.7:0/1093485617 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5584d93033d0 con 0x5584d93b0500 2024-06-10T01:30:28.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.437+0000 7fcb517fa700 1 -- 172.21.15.7:0/1093485617 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7fcb580a0900 con 0x5584d93b0500 2024-06-10T01:30:28.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.437+0000 7fcb7a716b80 10 client.21246 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:30:28.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.437+0000 7fcb7a716b80 10 client.21246 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:30:28.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.437+0000 7fcb7a716b80 20 client.21246 populate_metadata read hostname 'smithi007' 2024-06-10T01:30:28.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.437+0000 7fcb7a716b80 1 --2- 172.21.15.7:0/1093485617 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5584d9430900 0x5584d9432d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:28.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.437+0000 7fcb7a716b80 4 client.21246 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "11d6bcd3", "release": true, "await": true}] 2024-06-10T01:30:28.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.437+0000 7fcb7a716b80 1 -- 172.21.15.7:0/1093485617 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "set_id": "11d6bcd3", "release": true, "await": true}) -- 0x5584d92b7050 con 0x5584d9430900 2024-06-10T01:30:28.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.438+0000 7fcb6dc2f700 1 --2- 172.21.15.7:0/1093485617 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5584d9430900 0x5584d9432d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:28.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.438+0000 7fcb517fa700 10 client.21246 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:28.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.438+0000 7fcb6dc2f700 1 --2- 172.21.15.7:0/1093485617 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5584d9430900 0x5584d9432d20 crc :-1 s=READY pgs=548 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6675355d9831f6de server_cookie=e0112eb90fe3e473 in_seq=0 out_seq=0 2024-06-10T01:30:28.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.504+0000 7fcb517fa700 1 -- 172.21.15.7:0/1093485617 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5584d92b7050 con 0x5584d9430900 2024-06-10T01:30:28.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.504+0000 7fcb517fa700 10 client.21246 handle_command_reply: tid=0 2024-06-10T01:30:28.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.504+0000 7fcb7a716b80 1 client.21246 shutdown 2024-06-10T01:30:28.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.504+0000 7fcb7a716b80 1 -- 172.21.15.7:0/1093485617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584d93b0500 msgr2=0x5584d93b9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:28.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.504+0000 7fcb7a716b80 1 --2- 172.21.15.7:0/1093485617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584d93b0500 0x5584d93b9a30 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009520 tx=0x7fcb580b44b0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.505+0000 7fcb7a716b80 1 -- 172.21.15.7:0/1093485617 shutdown_connections 2024-06-10T01:30:28.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.505+0000 7fcb7a716b80 1 --2- 172.21.15.7:0/1093485617 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5584d9430900 0x5584d9432d20 crc :-1 s=READY pgs=548 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.505+0000 7fcb7a716b80 1 --2- 172.21.15.7:0/1093485617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d93b0e20 0x5584d93ba040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.505+0000 7fcb7a716b80 1 --2- 172.21.15.7:0/1093485617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5584d93aac50 0x5584d93bd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.505+0000 7fcb7a716b80 1 --2- 172.21.15.7:0/1093485617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584d93b0500 0x5584d93b9a30 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.505+0000 7fcb517fa700 0 client.21246 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:28.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.505+0000 7fcb7a716b80 1 -- 172.21.15.7:0/1093485617 >> 172.21.15.7:0/1093485617 conn(0x5584d92d7180 msgr2=0x5584d93a9f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:28.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.505+0000 7fcb7a716b80 1 -- 172.21.15.7:0/1093485617 shutdown_connections 2024-06-10T01:30:28.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.505+0000 7fcb7a716b80 1 -- 172.21.15.7:0/1093485617 wait complete. 2024-06-10T01:30:28.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.505+0000 7fcb7a716b80 20 client.21246 trim_cache size 0 max 16384 2024-06-10T01:30:28.499 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:30:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 57, 2024-06-10T01:30:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:30:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 80, 2024-06-10T01:30:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": { 2024-06-10T01:30:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: "11d6bcd3": { 2024-06-10T01:30:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 80, 2024-06-10T01:30:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: "age_ref": 0.0, 2024-06-10T01:30:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:30:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:30:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:30:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:30:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: "timeout": 90.0, 2024-06-10T01:30:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: "expiration": 142.4, 2024-06-10T01:30:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: "members": { 2024-06-10T01:30:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: "file:/": { 2024-06-10T01:30:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: "excluded": false, 2024-06-10T01:30:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: "state": { 2024-06-10T01:30:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RELEASED", 2024-06-10T01:30:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0 2024-06-10T01:30:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:30:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:30:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:30:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:30:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:30:28.501 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:30:28.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.505+0000 7fcb6fe93700 1 -- 172.21.15.7:0/1069759424 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcb540a2ca0 msgr2=0x7fcb540a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:28.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.505+0000 7fcb6fe93700 1 --2- 172.21.15.7:0/1069759424 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcb540a2ca0 0x7fcb540a5160 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fcb640098e0 tx=0x7fcb64005960 comp rx=0 tx=0).stop 2024-06-10T01:30:28.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.505+0000 7fcb6fe93700 1 -- 172.21.15.7:0/1069759424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb68101fd0 msgr2=0x7fcb681035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:28.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.505+0000 7fcb6fe93700 1 --2- 172.21.15.7:0/1069759424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb68101fd0 0x7fcb681035e0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800d980 tx=0x7fcb5800dc90 comp rx=0 tx=0).stop 2024-06-10T01:30:28.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.505+0000 7fcb6fe93700 1 -- 172.21.15.7:0/1069759424 shutdown_connections 2024-06-10T01:30:28.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.506+0000 7fcb6fe93700 1 --2- 172.21.15.7:0/1069759424 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fcb540a2ca0 0x7fcb540a5160 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.506+0000 7fcb6fe93700 1 --2- 172.21.15.7:0/1069759424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb681066e0 0x7fcb68103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.506+0000 7fcb6fe93700 1 --2- 172.21.15.7:0/1069759424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb68101620 0x7fcb681030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.506+0000 7fcb6fe93700 1 --2- 172.21.15.7:0/1069759424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb68101fd0 0x7fcb681035e0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:28.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.506+0000 7fcb6fe93700 1 -- 172.21.15.7:0/1069759424 >> 172.21.15.7:0/1069759424 conn(0x7fcb68078330 msgr2=0x7fcb6806afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:28.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.506+0000 7fcb6fe93700 1 -- 172.21.15.7:0/1069759424 shutdown_connections 2024-06-10T01:30:28.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:30:28.506+0000 7fcb6fe93700 1 -- 172.21.15.7:0/1069759424 wait complete. 2024-06-10T01:30:28.567 INFO:tasks.workunit.client.0.smithi007.stdout: 10 256430 21.34 MB/sec execute 545 sec latency 19934.843 ms 2024-06-10T01:30:28.823 INFO:tasks.cephfs.filesystem:scrub status for tag:5b2dce0c-b7c0-4ac0-ab7c-96a0b18216bc - {'path': '/', 'tag': '5b2dce0c-b7c0-4ac0-ab7c-96a0b18216bc', 'options': 'recursive,force'} 2024-06-10T01:30:29.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:29 smithi175 ceph-mon[36636]: pgmap v1168: 129 pgs: 129 active+clean; 666 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 184 KiB/s wr, 154 op/s 2024-06-10T01:30:29.464 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:29 smithi007 ceph-mon[32398]: pgmap v1168: 129 pgs: 129 active+clean; 666 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 184 KiB/s wr, 154 op/s 2024-06-10T01:30:29.512 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '11d6bcd3', total seconds elapsed: 34.2 2024-06-10T01:30:29.567 INFO:tasks.workunit.client.0.smithi007.stdout: 10 257050 21.37 MB/sec execute 546 sec latency 52.633 ms 2024-06-10T01:30:29.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:29 smithi078 ceph-mon[36683]: pgmap v1168: 129 pgs: 129 active+clean; 666 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 184 KiB/s wr, 154 op/s 2024-06-10T01:30:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.851+0000 7f68bfca5640 1 -- 172.21.15.7:0/2563673382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68b8104e30 msgr2=0x7f68b8107220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.851+0000 7f68bfca5640 1 --2- 172.21.15.7:0/2563673382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68b8104e30 0x7f68b8107220 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009900 tx=0x7f68a802a410 comp rx=0 tx=0).stop 2024-06-10T01:30:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.851+0000 7f68bfca5640 1 -- 172.21.15.7:0/2563673382 shutdown_connections 2024-06-10T01:30:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.851+0000 7f68bfca5640 1 --2- 172.21.15.7:0/2563673382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68b8107760 0x7f68b8109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.851+0000 7f68bfca5640 1 --2- 172.21.15.7:0/2563673382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68b8104e30 0x7f68b8107220 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.851+0000 7f68bfca5640 1 --2- 172.21.15.7:0/2563673382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68b8102500 0x7f68b81048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.851+0000 7f68bfca5640 1 -- 172.21.15.7:0/2563673382 >> 172.21.15.7:0/2563673382 conn(0x7f68b80fbe70 msgr2=0x7f68b80fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.851+0000 7f68bfca5640 1 -- 172.21.15.7:0/2563673382 shutdown_connections 2024-06-10T01:30:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.851+0000 7f68bfca5640 1 -- 172.21.15.7:0/2563673382 wait complete. 2024-06-10T01:30:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.852+0000 7f68bfca5640 1 Processor -- start 2024-06-10T01:30:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.852+0000 7f68bfca5640 1 -- start start 2024-06-10T01:30:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.852+0000 7f68bfca5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68b8102500 0x7f68b8197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.852+0000 7f68bfca5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68b8104e30 0x7f68b81982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.852+0000 7f68bfca5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68b8107760 0x7f68b819f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.852+0000 7f68bfca5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f68b810d650 con 0x7f68b8107760 2024-06-10T01:30:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.852+0000 7f68bfca5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f68b810d4d0 con 0x7f68b8102500 2024-06-10T01:30:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.852+0000 7f68bfca5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f68b810d7d0 con 0x7f68b8104e30 2024-06-10T01:30:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.852+0000 7f68be242640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68b8107760 0x7f68b819f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.852+0000 7f68be242640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68b8107760 0x7f68b819f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44130/0 (socket says 172.21.15.7:44130) 2024-06-10T01:30:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.852+0000 7f68be242640 1 -- 172.21.15.7:0/3947059711 learned_addr learned my addr 172.21.15.7:0/3947059711 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.852+0000 7f68bd240640 1 --2- 172.21.15.7:0/3947059711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68b8104e30 0x7f68b81982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.852+0000 7f68bda41640 1 --2- 172.21.15.7:0/3947059711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68b8102500 0x7f68b8197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.853+0000 7f68be242640 1 -- 172.21.15.7:0/3947059711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68b8104e30 msgr2=0x7f68b81982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.853+0000 7f68be242640 1 --2- 172.21.15.7:0/3947059711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68b8104e30 0x7f68b81982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.853+0000 7f68be242640 1 -- 172.21.15.7:0/3947059711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68b8102500 msgr2=0x7f68b8197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.853+0000 7f68be242640 1 --2- 172.21.15.7:0/3947059711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68b8102500 0x7f68b8197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.853+0000 7f68be242640 1 -- 172.21.15.7:0/3947059711 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b819f980 con 0x7f68b8107760 2024-06-10T01:30:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.853+0000 7f68be242640 1 --2- 172.21.15.7:0/3947059711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68b8107760 0x7f68b819f310 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f68b000eb10 tx=0x7f68b000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.853+0000 7f68aeffd640 1 -- 172.21.15.7:0/3947059711 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b0004b20 con 0x7f68b8107760 2024-06-10T01:30:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.853+0000 7f68aeffd640 1 -- 172.21.15.7:0/3947059711 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68b0004cc0 con 0x7f68b8107760 2024-06-10T01:30:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.853+0000 7f68aeffd640 1 -- 172.21.15.7:0/3947059711 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b0010760 con 0x7f68b8107760 2024-06-10T01:30:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.854+0000 7f68bfca5640 1 -- 172.21.15.7:0/3947059711 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b819fc70 con 0x7f68b8107760 2024-06-10T01:30:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.854+0000 7f68bfca5640 1 -- 172.21.15.7:0/3947059711 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b81a0140 con 0x7f68b8107760 2024-06-10T01:30:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.856+0000 7f68aeffd640 1 -- 172.21.15.7:0/3947059711 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f68b0005020 con 0x7f68b8107760 2024-06-10T01:30:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.856+0000 7f68c29bb740 1 Processor -- start 2024-06-10T01:30:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.857+0000 7f68c29bb740 1 -- start start 2024-06-10T01:30:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.857+0000 7f68c29bb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cf1b99010 0x562cf1b99430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.857+0000 7f68c29bb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562cf1b99a00 0x562cf1b9e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.857+0000 7f68c29bb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562cf1b9eb70 0x562cf1ba1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.857+0000 7f68c29bb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562cf1afc4f0 con 0x562cf1b99010 2024-06-10T01:30:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.857+0000 7f68c29bb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562cf1a9a870 con 0x562cf1b99a00 2024-06-10T01:30:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.857+0000 7f68c29bb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562cf1af84b0 con 0x562cf1b9eb70 2024-06-10T01:30:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.857+0000 7f68bd240640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cf1b99010 0x562cf1b99430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.857+0000 7f68bd240640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cf1b99010 0x562cf1b99430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44134/0 (socket says 172.21.15.7:44134) 2024-06-10T01:30:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.857+0000 7f68bd240640 1 -- 172.21.15.7:0/2842679482 learned_addr learned my addr 172.21.15.7:0/2842679482 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.857+0000 7f68bda41640 1 --2- 172.21.15.7:0/2842679482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562cf1b99a00 0x562cf1b9e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.857+0000 7f68aeffd640 1 --2- 172.21.15.7:0/3947059711 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f68a40a2c00 0x7f68a40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.857+0000 7f68bd240640 1 --2- 172.21.15.7:0/2842679482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562cf1b9eb70 0x562cf1ba1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.857+0000 7f68be242640 1 --2- 172.21.15.7:0/3947059711 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f68a40a2c00 0x7f68a40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.857+0000 7f68bd240640 1 -- 172.21.15.7:0/2842679482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562cf1b9eb70 msgr2=0x562cf1ba1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.857+0000 7f68bd240640 1 --2- 172.21.15.7:0/2842679482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562cf1b9eb70 0x562cf1ba1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.857+0000 7f68bd240640 1 -- 172.21.15.7:0/2842679482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562cf1b99a00 msgr2=0x562cf1b9e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.857+0000 7f68bd240640 1 --2- 172.21.15.7:0/2842679482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562cf1b99a00 0x562cf1b9e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.857+0000 7f68bd240640 1 -- 172.21.15.7:0/2842679482 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cf1ab69b0 con 0x562cf1b99010 2024-06-10T01:30:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.858+0000 7f68aeffd640 1 -- 172.21.15.7:0/3947059711 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f68b00cf8f0 con 0x7f68b8107760 2024-06-10T01:30:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.858+0000 7f68bd240640 1 --2- 172.21.15.7:0/2842679482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cf1b99010 0x562cf1b99430 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f68a8031610 tx=0x7f68a8005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c73a54b47428ca server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.858+0000 7f68a3fff640 1 -- 172.21.15.7:0/2842679482 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8044020 con 0x562cf1b99010 2024-06-10T01:30:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.858+0000 7f68be242640 1 --2- 172.21.15.7:0/3947059711 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f68a40a2c00 0x7f68a40a50c0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f68b00cd3e0 tx=0x7f68b0004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.858+0000 7f68a3fff640 1 -- 172.21.15.7:0/2842679482 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68a8031930 con 0x562cf1b99010 2024-06-10T01:30:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.858+0000 7f68a3fff640 1 -- 172.21.15.7:0/2842679482 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8035040 con 0x562cf1b99010 2024-06-10T01:30:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.859+0000 7f68c29bb740 1 -- 172.21.15.7:0/2842679482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cf1b99010 msgr2=0x562cf1b99430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.859+0000 7f68c29bb740 1 --2- 172.21.15.7:0/2842679482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cf1b99010 0x562cf1b99430 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f68a8031610 tx=0x7f68a8005b50 comp rx=0 tx=0).stop 2024-06-10T01:30:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.859+0000 7f68c29bb740 1 -- 172.21.15.7:0/2842679482 shutdown_connections 2024-06-10T01:30:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.859+0000 7f68c29bb740 1 --2- 172.21.15.7:0/2842679482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562cf1b9eb70 0x562cf1ba1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.859+0000 7f68c29bb740 1 --2- 172.21.15.7:0/2842679482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562cf1b99a00 0x562cf1b9e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.859+0000 7f68c29bb740 1 --2- 172.21.15.7:0/2842679482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cf1b99010 0x562cf1b99430 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.859+0000 7f68c29bb740 1 -- 172.21.15.7:0/2842679482 >> 172.21.15.7:0/2842679482 conn(0x562cf1b94680 msgr2=0x562cf1b94a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.859+0000 7f68c29bb740 1 -- 172.21.15.7:0/2842679482 shutdown_connections 2024-06-10T01:30:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.859+0000 7f68c29bb740 1 -- 172.21.15.7:0/2842679482 wait complete. 2024-06-10T01:30:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.859+0000 7f68c29bb740 1 -- start start 2024-06-10T01:30:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.859+0000 7f68c29bb740 1 Processor -- start 2024-06-10T01:30:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.860+0000 7f68c29bb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cf1b99010 0x562cf1ba3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.860+0000 7f68c29bb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562cf1b99a00 0x562cf1ba37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.860+0000 7f68c29bb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562cf1b9eb70 0x562cf1baa800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.860+0000 7f68c29bb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562cf1ba1ac0 con 0x562cf1b99010 2024-06-10T01:30:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.860+0000 7f68c29bb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562cf1a85ae0 con 0x562cf1b99a00 2024-06-10T01:30:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.860+0000 7f68c29bb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562cf1ba1c40 con 0x562cf1b9eb70 2024-06-10T01:30:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.860+0000 7f68c29bb740 10 client.0 _ll_register_callbacks cb 0x562cf1abe9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:30:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.860+0000 7f68bd240640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cf1b99010 0x562cf1ba3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.860+0000 7f68bda41640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562cf1b99a00 0x562cf1ba37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.860+0000 7f68bd240640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cf1b99010 0x562cf1ba3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44136/0 (socket says 172.21.15.7:44136) 2024-06-10T01:30:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.860+0000 7f68bd240640 1 -- 172.21.15.7:0/1250182545 learned_addr learned my addr 172.21.15.7:0/1250182545 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:29.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.860+0000 7f68bd240640 1 --2- 172.21.15.7:0/1250182545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562cf1b9eb70 0x562cf1baa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:29.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.860+0000 7f68bda41640 1 -- 172.21.15.7:0/1250182545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562cf1b9eb70 msgr2=0x562cf1baa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:29.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.860+0000 7f68bda41640 1 --2- 172.21.15.7:0/1250182545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562cf1b9eb70 0x562cf1baa800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:29.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.860+0000 7f68bda41640 1 -- 172.21.15.7:0/1250182545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cf1b99010 msgr2=0x562cf1ba3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:29.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.860+0000 7f68bda41640 1 --2- 172.21.15.7:0/1250182545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cf1b99010 0x562cf1ba3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:29.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.860+0000 7f68bda41640 1 -- 172.21.15.7:0/1250182545 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cf1810210 con 0x562cf1b99a00 2024-06-10T01:30:29.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.860+0000 7f68bd240640 1 --2- 172.21.15.7:0/1250182545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cf1b99010 0x562cf1ba3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:30:29.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.861+0000 7f68bda41640 1 --2- 172.21.15.7:0/1250182545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562cf1b99a00 0x562cf1ba37c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x562cf1b99e00 tx=0x7f68b4012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb4409e472be9d6c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:29.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.861+0000 7f68a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:30:29.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.861+0000 7f68a17fa640 1 -- 172.21.15.7:0/1250182545 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b400f890 con 0x562cf1b99a00 2024-06-10T01:30:29.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.861+0000 7f68c29bb740 1 -- 172.21.15.7:0/1250182545 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68a8031930 con 0x562cf1b99a00 2024-06-10T01:30:29.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.861+0000 7f68a17fa640 1 -- 172.21.15.7:0/1250182545 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f68b400fa30 con 0x562cf1b99a00 2024-06-10T01:30:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.861+0000 7f68a17fa640 1 -- 172.21.15.7:0/1250182545 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f68b400fc30 con 0x562cf1b99a00 2024-06-10T01:30:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.861+0000 7f68c29bb740 10 client.30422 fetch_fsmap learned FSMap version 57 2024-06-10T01:30:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.861+0000 7f68c29bb740 1 -- 172.21.15.7:0/1250182545 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x562cf18dc620 con 0x562cf1b99a00 2024-06-10T01:30:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.862+0000 7f68a17fa640 1 -- 172.21.15.7:0/1250182545 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68a8031930 con 0x562cf1b99a00 2024-06-10T01:30:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.862+0000 7f68a17fa640 1 -- 172.21.15.7:0/1250182545 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7f68b401c420 con 0x562cf1b99a00 2024-06-10T01:30:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.862+0000 7f68c29bb740 10 client.30422 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:30:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.862+0000 7f68c29bb740 10 client.30422 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:30:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.862+0000 7f68c29bb740 20 client.30422 populate_metadata read hostname 'smithi007' 2024-06-10T01:30:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.862+0000 7f68c29bb740 1 --2- 172.21.15.7:0/1250182545 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562cf1baf1b0 0x562cf1bb1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.862+0000 7f68c29bb740 4 client.30422 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:30:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.862+0000 7f68c29bb740 1 -- 172.21.15.7:0/1250182545 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cf1b0ff10 con 0x562cf1baf1b0 2024-06-10T01:30:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.862+0000 7f68bd240640 1 --2- 172.21.15.7:0/1250182545 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562cf1baf1b0 0x562cf1bb1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.864+0000 7f68bd240640 1 --2- 172.21.15.7:0/1250182545 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562cf1baf1b0 0x562cf1bb1610 crc :-1 s=READY pgs=549 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd0e5829bd466d61 server_cookie=fd7487a6e51d8164 in_seq=0 out_seq=0 2024-06-10T01:30:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.864+0000 7f68a17fa640 10 client.30422 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.870+0000 7f68a17fa640 1 -- 172.21.15.7:0/1250182545 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562cf1b0ff10 con 0x562cf1baf1b0 2024-06-10T01:30:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.870+0000 7f68a17fa640 10 client.30422 handle_command_reply: tid=0 2024-06-10T01:30:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.870+0000 7f68c29bb740 1 client.30422 shutdown 2024-06-10T01:30:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.871+0000 7f68c29bb740 1 -- 172.21.15.7:0/1250182545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562cf1b99a00 msgr2=0x562cf1ba37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.871+0000 7f68c29bb740 1 --2- 172.21.15.7:0/1250182545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562cf1b99a00 0x562cf1ba37c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x562cf1b99e00 tx=0x7f68b4012b60 comp rx=0 tx=0).stop 2024-06-10T01:30:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.871+0000 7f68c29bb740 1 -- 172.21.15.7:0/1250182545 shutdown_connections 2024-06-10T01:30:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.871+0000 7f68c29bb740 1 --2- 172.21.15.7:0/1250182545 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562cf1baf1b0 0x562cf1bb1610 crc :-1 s=READY pgs=549 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.871+0000 7f68c29bb740 1 --2- 172.21.15.7:0/1250182545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562cf1b9eb70 0x562cf1baa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.871+0000 7f68c29bb740 1 --2- 172.21.15.7:0/1250182545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562cf1b99a00 0x562cf1ba37c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.871+0000 7f68c29bb740 1 --2- 172.21.15.7:0/1250182545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cf1b99010 0x562cf1ba3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.871+0000 7f68a17fa640 0 client.30422 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.871+0000 7f68c29bb740 1 -- 172.21.15.7:0/1250182545 >> 172.21.15.7:0/1250182545 conn(0x562cf1b94680 msgr2=0x562cf1b937e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.871+0000 7f68c29bb740 1 -- 172.21.15.7:0/1250182545 shutdown_connections 2024-06-10T01:30:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.871+0000 7f68c29bb740 1 -- 172.21.15.7:0/1250182545 wait complete. 2024-06-10T01:30:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.871+0000 7f68c29bb740 20 client.30422 trim_cache size 0 max 16384 2024-06-10T01:30:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.888+0000 7f68c29bb740 1 Processor -- start 2024-06-10T01:30:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.888+0000 7f68c29bb740 1 -- start start 2024-06-10T01:30:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.889+0000 7f68c29bb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cf1b99010 0x562cf1ba3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.889+0000 7f68c29bb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562cf1b99a00 0x562cf1ba37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.889+0000 7f68c29bb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562cf1b9eb70 0x562cf1baa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.889+0000 7f68c29bb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f68b81ace70 con 0x562cf1b99010 2024-06-10T01:30:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.889+0000 7f68c29bb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f68b81aa6e0 con 0x562cf1b99a00 2024-06-10T01:30:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.889+0000 7f68c29bb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f68b81a8530 con 0x562cf1b9eb70 2024-06-10T01:30:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.889+0000 7f68bd240640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cf1b99010 0x562cf1ba3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.889+0000 7f68bd240640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cf1b99010 0x562cf1ba3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44138/0 (socket says 172.21.15.7:44138) 2024-06-10T01:30:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.889+0000 7f68bd240640 1 -- 172.21.15.7:0/4212775162 learned_addr learned my addr 172.21.15.7:0/4212775162 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.889+0000 7f68bda41640 1 --2- 172.21.15.7:0/4212775162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562cf1b99a00 0x562cf1ba37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.889+0000 7f68bd240640 1 -- 172.21.15.7:0/4212775162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562cf1b9eb70 msgr2=0x562cf1baa7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.889+0000 7f68bd240640 1 --2- 172.21.15.7:0/4212775162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562cf1b9eb70 0x562cf1baa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.889+0000 7f68bd240640 1 -- 172.21.15.7:0/4212775162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562cf1b99a00 msgr2=0x562cf1ba37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.889+0000 7f68bd240640 1 --2- 172.21.15.7:0/4212775162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562cf1b99a00 0x562cf1ba37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.889+0000 7f68bd240640 1 -- 172.21.15.7:0/4212775162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cf1bb1c90 con 0x562cf1b99010 2024-06-10T01:30:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.889+0000 7f68bd240640 1 --2- 172.21.15.7:0/4212775162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cf1b99010 0x562cf1ba3280 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f68a80314e0 tx=0x7f68a8031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fcde481ed73ec09 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.889+0000 7f68a37fe640 1 -- 172.21.15.7:0/4212775162 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8044040 con 0x562cf1b99010 2024-06-10T01:30:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.889+0000 7f68a37fe640 1 -- 172.21.15.7:0/4212775162 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68a80558a0 con 0x562cf1b99010 2024-06-10T01:30:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.889+0000 7f68a37fe640 1 -- 172.21.15.7:0/4212775162 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8035040 con 0x562cf1b99010 2024-06-10T01:30:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.890+0000 7f68c29bb740 1 -- 172.21.15.7:0/4212775162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cf1b99010 msgr2=0x562cf1ba3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.890+0000 7f68c29bb740 1 --2- 172.21.15.7:0/4212775162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cf1b99010 0x562cf1ba3280 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f68a80314e0 tx=0x7f68a8031400 comp rx=0 tx=0).stop 2024-06-10T01:30:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.890+0000 7f68c29bb740 1 -- 172.21.15.7:0/4212775162 shutdown_connections 2024-06-10T01:30:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.890+0000 7f68c29bb740 1 --2- 172.21.15.7:0/4212775162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562cf1b9eb70 0x562cf1baa7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.890+0000 7f68c29bb740 1 --2- 172.21.15.7:0/4212775162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562cf1b99a00 0x562cf1ba37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.890+0000 7f68c29bb740 1 --2- 172.21.15.7:0/4212775162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cf1b99010 0x562cf1ba3280 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.890+0000 7f68c29bb740 1 -- 172.21.15.7:0/4212775162 >> 172.21.15.7:0/4212775162 conn(0x562cf1b94680 msgr2=0x562cf1b94a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.890+0000 7f68c29bb740 1 -- 172.21.15.7:0/4212775162 shutdown_connections 2024-06-10T01:30:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.890+0000 7f68c29bb740 1 -- 172.21.15.7:0/4212775162 wait complete. 2024-06-10T01:30:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.891+0000 7f68c29bb740 1 -- start start 2024-06-10T01:30:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.891+0000 7f68c29bb740 1 Processor -- start 2024-06-10T01:30:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.891+0000 7f68c29bb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cf1b99a00 0x562cf1ba7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.891+0000 7f68c29bb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562cf1b9eb70 0x562cf1ba7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.891+0000 7f68c29bb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562cf1c15ed0 0x562cf1c182c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.891+0000 7f68c29bb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f68b81a8ba0 con 0x562cf1b99a00 2024-06-10T01:30:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.891+0000 7f68c29bb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f68b81aa010 con 0x562cf1c15ed0 2024-06-10T01:30:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.891+0000 7f68c29bb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f68b81a9270 con 0x562cf1b9eb70 2024-06-10T01:30:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.891+0000 7f68c29bb740 10 client.0 _ll_register_callbacks cb 0x7f68b4009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:30:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.891+0000 7f68bd240640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cf1b99a00 0x562cf1ba7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.891+0000 7f68bd240640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cf1b99a00 0x562cf1ba7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44144/0 (socket says 172.21.15.7:44144) 2024-06-10T01:30:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.891+0000 7f68bd240640 1 -- 172.21.15.7:0/3427001682 learned_addr learned my addr 172.21.15.7:0/3427001682 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.891+0000 7f68bda41640 1 --2- 172.21.15.7:0/3427001682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562cf1b9eb70 0x562cf1ba7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.892+0000 7f68bd240640 1 --2- 172.21.15.7:0/3427001682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562cf1c15ed0 0x562cf1c182c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.892+0000 7f68bd240640 1 -- 172.21.15.7:0/3427001682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562cf1b9eb70 msgr2=0x562cf1ba7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.892+0000 7f68bd240640 1 --2- 172.21.15.7:0/3427001682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562cf1b9eb70 0x562cf1ba7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.892+0000 7f68bd240640 1 -- 172.21.15.7:0/3427001682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562cf1c15ed0 msgr2=0x562cf1c182c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.892+0000 7f68bd240640 1 --2- 172.21.15.7:0/3427001682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562cf1c15ed0 0x562cf1c182c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.892+0000 7f68bd240640 1 -- 172.21.15.7:0/3427001682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cf1b8ffe0 con 0x562cf1b99a00 2024-06-10T01:30:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.892+0000 7f68bd240640 1 --2- 172.21.15.7:0/3427001682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cf1b99a00 0x562cf1ba7700 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f68a80057c0 tx=0x7f68a8009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a758fe66e9eaf79f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.892+0000 7f68a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:30:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.892+0000 7f68a17fa640 1 -- 172.21.15.7:0/3427001682 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8035070 con 0x562cf1b99a00 2024-06-10T01:30:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.892+0000 7f68a17fa640 1 -- 172.21.15.7:0/3427001682 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f68a803a3f0 con 0x562cf1b99a00 2024-06-10T01:30:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.892+0000 7f68c29bb740 1 -- 172.21.15.7:0/3427001682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68a80558a0 con 0x562cf1b99a00 2024-06-10T01:30:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.892+0000 7f68a17fa640 1 -- 172.21.15.7:0/3427001682 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f68a8044070 con 0x562cf1b99a00 2024-06-10T01:30:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.892+0000 7f68c29bb740 10 client.21264 fetch_fsmap learned FSMap version 57 2024-06-10T01:30:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.892+0000 7f68c29bb740 1 -- 172.21.15.7:0/3427001682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x562cf1b98b70 con 0x562cf1b99a00 2024-06-10T01:30:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.893+0000 7f68a17fa640 1 -- 172.21.15.7:0/3427001682 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68a80558a0 con 0x562cf1b99a00 2024-06-10T01:30:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.893+0000 7f68a17fa640 1 -- 172.21.15.7:0/3427001682 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7f68a805f5c0 con 0x562cf1b99a00 2024-06-10T01:30:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.893+0000 7f68c29bb740 10 client.21264 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:30:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.893+0000 7f68c29bb740 10 client.21264 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:30:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.893+0000 7f68c29bb740 20 client.21264 populate_metadata read hostname 'smithi007' 2024-06-10T01:30:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.893+0000 7f68c29bb740 1 --2- 172.21.15.7:0/3427001682 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562cf1c19b60 0x562cf1c1bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.893+0000 7f68c29bb740 4 client.21264 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:30:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.893+0000 7f68c29bb740 1 -- 172.21.15.7:0/3427001682 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562cf1b09c20 con 0x562cf1c19b60 2024-06-10T01:30:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.893+0000 7f68bda41640 1 --2- 172.21.15.7:0/3427001682 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562cf1c19b60 0x562cf1c1bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.895+0000 7f68a17fa640 10 client.21264 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:29.895+0000 7f68bda41640 1 --2- 172.21.15.7:0/3427001682 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562cf1c19b60 0x562cf1c1bf80 crc :-1 s=READY pgs=550 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4ec06c913cfc1 server_cookie=4ec17fd4c85a49f4 in_seq=0 out_seq=0 2024-06-10T01:30:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:30.148+0000 7f68a17fa640 1 -- 172.21.15.7:0/3427001682 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562cf1b09c20 con 0x562cf1c19b60 2024-06-10T01:30:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:30.148+0000 7f68a17fa640 10 client.21264 handle_command_reply: tid=0 2024-06-10T01:30:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:30.148+0000 7f68c29bb740 1 client.21264 shutdown 2024-06-10T01:30:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:30.148+0000 7f68c29bb740 1 -- 172.21.15.7:0/3427001682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cf1b99a00 msgr2=0x562cf1ba7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:30.148+0000 7f68c29bb740 1 --2- 172.21.15.7:0/3427001682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cf1b99a00 0x562cf1ba7700 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f68a80057c0 tx=0x7f68a8009520 comp rx=0 tx=0).stop 2024-06-10T01:30:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:30.148+0000 7f68c29bb740 1 -- 172.21.15.7:0/3427001682 shutdown_connections 2024-06-10T01:30:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:30.148+0000 7f68c29bb740 1 --2- 172.21.15.7:0/3427001682 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x562cf1c19b60 0x562cf1c1bf80 crc :-1 s=READY pgs=550 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:30.148+0000 7f68c29bb740 1 --2- 172.21.15.7:0/3427001682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562cf1c15ed0 0x562cf1c182c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:30.148+0000 7f68a17fa640 0 client.21264 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:30.148+0000 7f68c29bb740 1 --2- 172.21.15.7:0/3427001682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562cf1b9eb70 0x562cf1ba7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:30.148+0000 7f68c29bb740 1 --2- 172.21.15.7:0/3427001682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cf1b99a00 0x562cf1ba7700 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:30.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:30.148+0000 7f68c29bb740 1 -- 172.21.15.7:0/3427001682 >> 172.21.15.7:0/3427001682 conn(0x562cf1b94680 msgr2=0x562cf1ba3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:30.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:30.148+0000 7f68c29bb740 1 -- 172.21.15.7:0/3427001682 shutdown_connections 2024-06-10T01:30:30.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:30.148+0000 7f68c29bb740 1 -- 172.21.15.7:0/3427001682 wait complete. 2024-06-10T01:30:30.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:30.148+0000 7f68c29bb740 20 client.21264 trim_cache size 0 max 16384 2024-06-10T01:30:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:30:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:30:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:30:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:30.149+0000 7f68bfca5640 1 -- 172.21.15.7:0/3947059711 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f68a40a2c00 msgr2=0x7f68a40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:30.149+0000 7f68bfca5640 1 --2- 172.21.15.7:0/3947059711 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f68a40a2c00 0x7f68a40a50c0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f68b00cd3e0 tx=0x7f68b0004a40 comp rx=0 tx=0).stop 2024-06-10T01:30:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:30.149+0000 7f68bfca5640 1 -- 172.21.15.7:0/3947059711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68b8107760 msgr2=0x7f68b819f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:30.149+0000 7f68bfca5640 1 --2- 172.21.15.7:0/3947059711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68b8107760 0x7f68b819f310 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f68b000eb10 tx=0x7f68b000c2d0 comp rx=0 tx=0).stop 2024-06-10T01:30:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:30.149+0000 7f68bfca5640 1 -- 172.21.15.7:0/3947059711 shutdown_connections 2024-06-10T01:30:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:30.149+0000 7f68bfca5640 1 --2- 172.21.15.7:0/3947059711 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f68a40a2c00 0x7f68a40a50c0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:30.149+0000 7f68bfca5640 1 --2- 172.21.15.7:0/3947059711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68b8107760 0x7f68b819f310 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:30.149+0000 7f68bfca5640 1 --2- 172.21.15.7:0/3947059711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68b8104e30 0x7f68b81982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:30.149+0000 7f68bfca5640 1 --2- 172.21.15.7:0/3947059711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68b8102500 0x7f68b8197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:30.149+0000 7f68bfca5640 1 -- 172.21.15.7:0/3947059711 >> 172.21.15.7:0/3947059711 conn(0x7f68b80fbe70 msgr2=0x7f68b8102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:30.149+0000 7f68bfca5640 1 -- 172.21.15.7:0/3947059711 shutdown_connections 2024-06-10T01:30:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:30.149+0000 7f68bfca5640 1 -- 172.21.15.7:0/3947059711 wait complete. 2024-06-10T01:30:30.568 INFO:tasks.workunit.client.0.smithi007.stdout: 10 257442 21.37 MB/sec execute 547 sec latency 196.586 ms 2024-06-10T01:30:31.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:31 smithi175 ceph-mon[36636]: pgmap v1169: 129 pgs: 129 active+clean; 666 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 104 KiB/s wr, 108 op/s 2024-06-10T01:30:31.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:31 smithi007 ceph-mon[32398]: pgmap v1169: 129 pgs: 129 active+clean; 666 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 104 KiB/s wr, 108 op/s 2024-06-10T01:30:31.568 INFO:tasks.workunit.client.0.smithi007.stdout: 10 257970 21.37 MB/sec execute 548 sec latency 141.460 ms 2024-06-10T01:30:31.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:31 smithi078 ceph-mon[36683]: pgmap v1169: 129 pgs: 129 active+clean; 666 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 104 KiB/s wr, 108 op/s 2024-06-10T01:30:32.568 INFO:tasks.workunit.client.0.smithi007.stdout: 10 258528 21.38 MB/sec execute 549 sec latency 133.213 ms 2024-06-10T01:30:33.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:33 smithi175 ceph-mon[36636]: pgmap v1170: 129 pgs: 129 active+clean; 662 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 662 KiB/s wr, 179 op/s 2024-06-10T01:30:33.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:33 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:30:33.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:33 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:30:33.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:33 smithi007 ceph-mon[32398]: pgmap v1170: 129 pgs: 129 active+clean; 662 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 662 KiB/s wr, 179 op/s 2024-06-10T01:30:33.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:33 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:30:33.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:33 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:30:33.568 INFO:tasks.workunit.client.0.smithi007.stdout: 10 259346 21.43 MB/sec execute 550 sec latency 59.253 ms 2024-06-10T01:30:33.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:33 smithi078 ceph-mon[36683]: pgmap v1170: 129 pgs: 129 active+clean; 662 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 662 KiB/s wr, 179 op/s 2024-06-10T01:30:33.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:33 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:30:33.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:33 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:30:34.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:34 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:30:34.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:34 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:30:34.568 INFO:tasks.workunit.client.0.smithi007.stdout: 10 259927 21.47 MB/sec execute 551 sec latency 73.235 ms 2024-06-10T01:30:34.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:34 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:30:35.118 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:35 smithi175 ceph-mon[36636]: pgmap v1171: 129 pgs: 129 active+clean; 661 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 683 KiB/s wr, 127 op/s 2024-06-10T01:30:35.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:35 smithi007 ceph-mon[32398]: pgmap v1171: 129 pgs: 129 active+clean; 661 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 683 KiB/s wr, 127 op/s 2024-06-10T01:30:35.568 INFO:tasks.workunit.client.0.smithi007.stdout: 10 260423 21.46 MB/sec execute 552 sec latency 100.472 ms 2024-06-10T01:30:35.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:35 smithi078 ceph-mon[36683]: pgmap v1171: 129 pgs: 129 active+clean; 661 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 683 KiB/s wr, 127 op/s 2024-06-10T01:30:36.568 INFO:tasks.workunit.client.0.smithi007.stdout: 10 260911 21.47 MB/sec execute 553 sec latency 122.028 ms 2024-06-10T01:30:37.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:37 smithi175 ceph-mon[36636]: pgmap v1172: 129 pgs: 129 active+clean; 666 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.6 MiB/s wr, 254 op/s 2024-06-10T01:30:37.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:37 smithi007 ceph-mon[32398]: pgmap v1172: 129 pgs: 129 active+clean; 666 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.6 MiB/s wr, 254 op/s 2024-06-10T01:30:37.568 INFO:tasks.workunit.client.0.smithi007.stdout: 10 261657 21.49 MB/sec execute 554 sec latency 58.662 ms 2024-06-10T01:30:37.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:37 smithi078 ceph-mon[36683]: pgmap v1172: 129 pgs: 129 active+clean; 666 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.6 MiB/s wr, 254 op/s 2024-06-10T01:30:38.568 INFO:tasks.workunit.client.0.smithi007.stdout: 10 262417 21.54 MB/sec execute 555 sec latency 68.270 ms 2024-06-10T01:30:39.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:39 smithi175 ceph-mon[36636]: pgmap v1173: 129 pgs: 129 active+clean; 670 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.1 MiB/s wr, 315 op/s 2024-06-10T01:30:39.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:39 smithi007 ceph-mon[32398]: pgmap v1173: 129 pgs: 129 active+clean; 670 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.1 MiB/s wr, 315 op/s 2024-06-10T01:30:39.568 INFO:tasks.workunit.client.0.smithi007.stdout: 10 262954 21.56 MB/sec execute 556 sec latency 95.764 ms 2024-06-10T01:30:39.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:39 smithi078 ceph-mon[36683]: pgmap v1173: 129 pgs: 129 active+clean; 670 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.1 MiB/s wr, 315 op/s 2024-06-10T01:30:40.568 INFO:tasks.workunit.client.0.smithi007.stdout: 10 263445 21.57 MB/sec execute 557 sec latency 159.469 ms 2024-06-10T01:30:41.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:41 smithi175 ceph-mon[36636]: pgmap v1174: 129 pgs: 129 active+clean; 674 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 2.5 MiB/s wr, 371 op/s 2024-06-10T01:30:41.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:41 smithi007 ceph-mon[32398]: pgmap v1174: 129 pgs: 129 active+clean; 674 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 2.5 MiB/s wr, 371 op/s 2024-06-10T01:30:41.568 INFO:tasks.workunit.client.0.smithi007.stdout: 10 263992 21.58 MB/sec execute 558 sec latency 77.159 ms 2024-06-10T01:30:41.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:41 smithi078 ceph-mon[36683]: pgmap v1174: 129 pgs: 129 active+clean; 674 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 2.5 MiB/s wr, 371 op/s 2024-06-10T01:30:42.568 INFO:tasks.workunit.client.0.smithi007.stdout: 10 264665 21.59 MB/sec execute 559 sec latency 85.220 ms 2024-06-10T01:30:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.286+0000 7f179819b640 1 -- 172.21.15.7:0/2050892743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1790104e70 msgr2=0x7f1790107260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.286+0000 7f179819b640 1 --2- 172.21.15.7:0/2050892743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1790104e70 0x7f1790107260 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f178000fb60 tx=0x7f178002e4b0 comp rx=0 tx=0).stop 2024-06-10T01:30:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.286+0000 7f179819b640 1 -- 172.21.15.7:0/2050892743 shutdown_connections 2024-06-10T01:30:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.286+0000 7f179819b640 1 --2- 172.21.15.7:0/2050892743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17901077a0 0x7f1790109b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.286+0000 7f179819b640 1 --2- 172.21.15.7:0/2050892743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1790104e70 0x7f1790107260 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.286+0000 7f179819b640 1 --2- 172.21.15.7:0/2050892743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1790102540 0x7f1790104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.286+0000 7f179819b640 1 -- 172.21.15.7:0/2050892743 >> 172.21.15.7:0/2050892743 conn(0x7f17900fbe70 msgr2=0x7f17900fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.287+0000 7f179819b640 1 -- 172.21.15.7:0/2050892743 shutdown_connections 2024-06-10T01:30:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.287+0000 7f179819b640 1 -- 172.21.15.7:0/2050892743 wait complete. 2024-06-10T01:30:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.287+0000 7f179819b640 1 Processor -- start 2024-06-10T01:30:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.288+0000 7f179819b640 1 -- start start 2024-06-10T01:30:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.288+0000 7f179819b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1790102540 0x7f1790195af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.288+0000 7f179819b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1790104e70 0x7f1790196030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.288+0000 7f179819b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17901077a0 0x7f179019d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.288+0000 7f179819b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f179010b4d0 con 0x7f1790104e70 2024-06-10T01:30:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.288+0000 7f179819b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f179010b350 con 0x7f1790102540 2024-06-10T01:30:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.288+0000 7f179819b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f179010b650 con 0x7f17901077a0 2024-06-10T01:30:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.288+0000 7f1795736640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1790104e70 0x7f1790196030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.288+0000 7f1795736640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1790104e70 0x7f1790196030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44580/0 (socket says 172.21.15.7:44580) 2024-06-10T01:30:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.288+0000 7f1795736640 1 -- 172.21.15.7:0/608802959 learned_addr learned my addr 172.21.15.7:0/608802959 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.288+0000 7f1795f37640 1 --2- 172.21.15.7:0/608802959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1790102540 0x7f1790195af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.288+0000 7f1796738640 1 --2- 172.21.15.7:0/608802959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17901077a0 0x7f179019d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.288+0000 7f1795f37640 1 -- 172.21.15.7:0/608802959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17901077a0 msgr2=0x7f179019d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.289+0000 7f1795f37640 1 --2- 172.21.15.7:0/608802959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17901077a0 0x7f179019d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.289+0000 7f1795f37640 1 -- 172.21.15.7:0/608802959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1790104e70 msgr2=0x7f1790196030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.289+0000 7f1795f37640 1 --2- 172.21.15.7:0/608802959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1790104e70 0x7f1790196030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.289+0000 7f1795f37640 1 -- 172.21.15.7:0/608802959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179019d710 con 0x7f1790102540 2024-06-10T01:30:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.289+0000 7f1795736640 1 --2- 172.21.15.7:0/608802959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1790104e70 0x7f1790196030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:30:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.289+0000 7f1795f37640 1 --2- 172.21.15.7:0/608802959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1790102540 0x7f1790195af0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f178c00e9f0 tx=0x7f178c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.289+0000 7f1786ffd640 1 -- 172.21.15.7:0/608802959 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c00cd80 con 0x7f1790102540 2024-06-10T01:30:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.289+0000 7f1786ffd640 1 -- 172.21.15.7:0/608802959 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f178c0035f0 con 0x7f1790102540 2024-06-10T01:30:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.289+0000 7f179819b640 1 -- 172.21.15.7:0/608802959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f179019da00 con 0x7f1790102540 2024-06-10T01:30:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.289+0000 7f1786ffd640 1 -- 172.21.15.7:0/608802959 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c010690 con 0x7f1790102540 2024-06-10T01:30:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.289+0000 7f179819b640 1 -- 172.21.15.7:0/608802959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f179019df50 con 0x7f1790102540 2024-06-10T01:30:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.292+0000 7f1786ffd640 1 -- 172.21.15.7:0/608802959 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f178c010830 con 0x7f1790102540 2024-06-10T01:30:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.292+0000 7f179af83740 1 Processor -- start 2024-06-10T01:30:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.293+0000 7f1786ffd640 1 --2- 172.21.15.7:0/608802959 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f177c0a2c00 0x7f177c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.293+0000 7f1786ffd640 1 -- 172.21.15.7:0/608802959 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f178c0cf6e0 con 0x7f1790102540 2024-06-10T01:30:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.293+0000 7f179af83740 1 -- start start 2024-06-10T01:30:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.294+0000 7f1796738640 1 --2- 172.21.15.7:0/608802959 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f177c0a2c00 0x7f177c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.294+0000 7f179af83740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f6ebd820 0x55f9f6ebdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.294+0000 7f179af83740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f9f6ebe210 0x55f9f6ec2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.294+0000 7f179af83740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9f6ec3380 0x55f9f6ec5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.294+0000 7f179af83740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f9f6e184f0 con 0x55f9f6ec3380 2024-06-10T01:30:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.294+0000 7f179af83740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f9f6db6870 con 0x55f9f6ebd820 2024-06-10T01:30:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.294+0000 7f179af83740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f9f6e144b0 con 0x55f9f6ebe210 2024-06-10T01:30:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.294+0000 7f1795f37640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f6ebd820 0x55f9f6ebdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.294+0000 7f1795736640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f9f6ebe210 0x55f9f6ec2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:43.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.294+0000 7f1795736640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f9f6ebe210 0x55f9f6ec2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:50858/0 (socket says 172.21.15.7:50858) 2024-06-10T01:30:43.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.294+0000 7f1795736640 1 -- 172.21.15.7:0/4035067830 learned_addr learned my addr 172.21.15.7:0/4035067830 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:43.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.294+0000 7f1795f37640 1 -- 172.21.15.7:0/4035067830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f9f6ebe210 msgr2=0x55f9f6ec2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:43.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.294+0000 7f1795f37640 1 --2- 172.21.15.7:0/4035067830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f9f6ebe210 0x55f9f6ec2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:43.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.294+0000 7f1795f37640 1 -- 172.21.15.7:0/4035067830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9f6ec3380 msgr2=0x55f9f6ec5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:43.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.294+0000 7f1795f37640 1 --2- 172.21.15.7:0/4035067830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9f6ec3380 0x55f9f6ec5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:43.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.294+0000 7f1795f37640 1 -- 172.21.15.7:0/4035067830 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9f6dd29b0 con 0x55f9f6ebd820 2024-06-10T01:30:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.295+0000 7f1795f37640 1 --2- 172.21.15.7:0/4035067830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f6ebd820 0x55f9f6ebdc40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f178c0cce40 tx=0x7f178c009590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae8a185f1e6668af server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.295+0000 7f1796738640 1 --2- 172.21.15.7:0/608802959 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f177c0a2c00 0x7f177c0a50c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f1788005fd0 tx=0x7f178800f8e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.295+0000 7f177bfff640 1 -- 172.21.15.7:0/4035067830 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c09ff50 con 0x55f9f6ebd820 2024-06-10T01:30:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.295+0000 7f177bfff640 1 -- 172.21.15.7:0/4035067830 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f178c0a4810 con 0x55f9f6ebd820 2024-06-10T01:30:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.295+0000 7f177bfff640 1 -- 172.21.15.7:0/4035067830 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c0a45e0 con 0x55f9f6ebd820 2024-06-10T01:30:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.296+0000 7f179af83740 1 -- 172.21.15.7:0/4035067830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f6ebd820 msgr2=0x55f9f6ebdc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.296+0000 7f179af83740 1 --2- 172.21.15.7:0/4035067830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f6ebd820 0x55f9f6ebdc40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f178c0cce40 tx=0x7f178c009590 comp rx=0 tx=0).stop 2024-06-10T01:30:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.296+0000 7f179af83740 1 -- 172.21.15.7:0/4035067830 shutdown_connections 2024-06-10T01:30:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.296+0000 7f179af83740 1 --2- 172.21.15.7:0/4035067830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9f6ec3380 0x55f9f6ec5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.296+0000 7f179af83740 1 --2- 172.21.15.7:0/4035067830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f9f6ebe210 0x55f9f6ec2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.296+0000 7f179af83740 1 --2- 172.21.15.7:0/4035067830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f6ebd820 0x55f9f6ebdc40 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.296+0000 7f179af83740 1 -- 172.21.15.7:0/4035067830 >> 172.21.15.7:0/4035067830 conn(0x55f9f6eb0680 msgr2=0x55f9f6eb0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.296+0000 7f179af83740 1 -- 172.21.15.7:0/4035067830 shutdown_connections 2024-06-10T01:30:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.297+0000 7f179af83740 1 -- 172.21.15.7:0/4035067830 wait complete. 2024-06-10T01:30:43.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.297+0000 7f179af83740 1 -- start start 2024-06-10T01:30:43.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.297+0000 7f179af83740 1 Processor -- start 2024-06-10T01:30:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.298+0000 7f179af83740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9f6ebd820 0x55f9f6ec7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.298+0000 7f179af83740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f6ebe210 0x55f9f6ec80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.298+0000 7f179af83740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f9f6ec3380 0x55f9f6ecf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.298+0000 7f1795f37640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9f6ebd820 0x55f9f6ec7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.298+0000 7f1795f37640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9f6ebd820 0x55f9f6ec7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44598/0 (socket says 172.21.15.7:44598) 2024-06-10T01:30:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.299+0000 7f1796738640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f9f6ec3380 0x55f9f6ecf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.299+0000 7f1795736640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f6ebe210 0x55f9f6ec80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.299+0000 7f1796738640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f9f6ec3380 0x55f9f6ecf100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:50872/0 (socket says 172.21.15.7:50872) 2024-06-10T01:30:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.299+0000 7f1795736640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f6ebe210 0x55f9f6ec80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:58226/0 (socket says 172.21.15.7:58226) 2024-06-10T01:30:43.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.299+0000 7f179af83740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f9f6ec6330 con 0x55f9f6ebd820 2024-06-10T01:30:43.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.299+0000 7f1795736640 1 -- 172.21.15.7:0/2646412032 learned_addr learned my addr 172.21.15.7:0/2646412032 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:43.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.299+0000 7f179af83740 1 -- 172.21.15.7:0/2646412032 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f9f6da1ae0 con 0x55f9f6ebe210 2024-06-10T01:30:43.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.299+0000 7f179af83740 1 -- 172.21.15.7:0/2646412032 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f9f6ec64b0 con 0x55f9f6ec3380 2024-06-10T01:30:43.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.299+0000 7f179af83740 10 client.0 _ll_register_callbacks cb 0x55f9f6dda9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:30:43.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.299+0000 7f1795736640 1 -- 172.21.15.7:0/2646412032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f9f6ec3380 msgr2=0x55f9f6ecf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:43.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.299+0000 7f1795736640 1 --2- 172.21.15.7:0/2646412032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f9f6ec3380 0x55f9f6ecf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:43.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.299+0000 7f1795736640 1 -- 172.21.15.7:0/2646412032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9f6ebd820 msgr2=0x55f9f6ec7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:43.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.299+0000 7f1795736640 1 --2- 172.21.15.7:0/2646412032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9f6ebd820 0x55f9f6ec7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:43.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.299+0000 7f1795736640 1 -- 172.21.15.7:0/2646412032 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9f6b2c210 con 0x55f9f6ebe210 2024-06-10T01:30:43.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.299+0000 7f1795f37640 1 --2- 172.21.15.7:0/2646412032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9f6ebd820 0x55f9f6ec7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:30:43.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.299+0000 7f1796738640 1 --2- 172.21.15.7:0/2646412032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f9f6ec3380 0x55f9f6ecf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:30:43.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.299+0000 7f1795736640 1 --2- 172.21.15.7:0/2646412032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f6ebe210 0x55f9f6ec80c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f178000fb30 tx=0x7f178000f5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7f54ce41344a5f9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:43.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.299+0000 7f17797fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:30:43.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.300+0000 7f17797fa640 1 -- 172.21.15.7:0/2646412032 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780035600 con 0x55f9f6ebe210 2024-06-10T01:30:43.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.300+0000 7f17797fa640 1 -- 172.21.15.7:0/2646412032 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f1780035d60 con 0x55f9f6ebe210 2024-06-10T01:30:43.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.300+0000 7f179af83740 1 -- 172.21.15.7:0/2646412032 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f178c0a4810 con 0x55f9f6ebe210 2024-06-10T01:30:43.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.300+0000 7f17797fa640 1 -- 172.21.15.7:0/2646412032 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f1780051050 con 0x55f9f6ebe210 2024-06-10T01:30:43.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.300+0000 7f179af83740 10 client.30440 fetch_fsmap learned FSMap version 57 2024-06-10T01:30:43.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.300+0000 7f179af83740 1 -- 172.21.15.7:0/2646412032 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55f9f6bf8620 con 0x55f9f6ebe210 2024-06-10T01:30:43.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.300+0000 7f17797fa640 1 -- 172.21.15.7:0/2646412032 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f178c0a4810 con 0x55f9f6ebe210 2024-06-10T01:30:43.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.300+0000 7f17797fa640 1 -- 172.21.15.7:0/2646412032 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7f1780003710 con 0x55f9f6ebe210 2024-06-10T01:30:43.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.300+0000 7f179af83740 10 client.30440 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:30:43.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.300+0000 7f179af83740 10 client.30440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:30:43.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.300+0000 7f179af83740 20 client.30440 populate_metadata read hostname 'smithi007' 2024-06-10T01:30:43.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.300+0000 7f179af83740 1 --2- 172.21.15.7:0/2646412032 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f9f6ed39a0 0x55f9f6ed5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:43.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.300+0000 7f179af83740 4 client.30440 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:30:43.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.300+0000 7f179af83740 1 -- 172.21.15.7:0/2646412032 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9f6e2bf10 con 0x55f9f6ed39a0 2024-06-10T01:30:43.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.301+0000 7f1795f37640 1 --2- 172.21.15.7:0/2646412032 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f9f6ed39a0 0x55f9f6ed5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:43.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.301+0000 7f1795f37640 1 --2- 172.21.15.7:0/2646412032 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f9f6ed39a0 0x55f9f6ed5e20 crc :-1 s=READY pgs=551 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c130e974df9d6fd4 server_cookie=421cf41ced4a7026 in_seq=0 out_seq=0 2024-06-10T01:30:43.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.301+0000 7f17797fa640 10 client.30440 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:43.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.305+0000 7f17797fa640 1 -- 172.21.15.7:0/2646412032 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f9f6e2bf10 con 0x55f9f6ed39a0 2024-06-10T01:30:43.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.305+0000 7f17797fa640 10 client.30440 handle_command_reply: tid=0 2024-06-10T01:30:43.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.305+0000 7f179af83740 1 client.30440 shutdown 2024-06-10T01:30:43.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.306+0000 7f179af83740 1 -- 172.21.15.7:0/2646412032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f6ebe210 msgr2=0x55f9f6ec80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.306+0000 7f179af83740 1 --2- 172.21.15.7:0/2646412032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f6ebe210 0x55f9f6ec80c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f178000fb30 tx=0x7f178000f5f0 comp rx=0 tx=0).stop 2024-06-10T01:30:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.306+0000 7f179af83740 1 -- 172.21.15.7:0/2646412032 shutdown_connections 2024-06-10T01:30:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.306+0000 7f179af83740 1 --2- 172.21.15.7:0/2646412032 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f9f6ed39a0 0x55f9f6ed5e20 crc :-1 s=READY pgs=551 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.306+0000 7f17797fa640 0 client.30440 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.306+0000 7f179af83740 1 --2- 172.21.15.7:0/2646412032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f9f6ec3380 0x55f9f6ecf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.306+0000 7f179af83740 1 --2- 172.21.15.7:0/2646412032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f6ebe210 0x55f9f6ec80c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.306+0000 7f179af83740 1 --2- 172.21.15.7:0/2646412032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9f6ebd820 0x55f9f6ec7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.306+0000 7f179af83740 1 -- 172.21.15.7:0/2646412032 >> 172.21.15.7:0/2646412032 conn(0x55f9f6eb0680 msgr2=0x55f9f6eaf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.306+0000 7f179af83740 1 -- 172.21.15.7:0/2646412032 shutdown_connections 2024-06-10T01:30:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.306+0000 7f179af83740 1 -- 172.21.15.7:0/2646412032 wait complete. 2024-06-10T01:30:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.306+0000 7f179af83740 20 client.30440 trim_cache size 0 max 16384 2024-06-10T01:30:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.326+0000 7f179af83740 1 Processor -- start 2024-06-10T01:30:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.326+0000 7f179af83740 1 -- start start 2024-06-10T01:30:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.327+0000 7f179af83740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f9f6ebd820 0x55f9f6ec5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.327+0000 7f179af83740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f6ebe210 0x55f9f6ecf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.327+0000 7f179af83740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9f6ec3380 0x55f9f6ec7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.327+0000 7f179af83740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f17901aad00 con 0x55f9f6ec3380 2024-06-10T01:30:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.327+0000 7f179af83740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f17901a8570 con 0x55f9f6ebe210 2024-06-10T01:30:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.327+0000 7f179af83740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f17901a63c0 con 0x55f9f6ebd820 2024-06-10T01:30:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.327+0000 7f1795f37640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f9f6ebd820 0x55f9f6ec5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.327+0000 7f1795f37640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f9f6ebd820 0x55f9f6ec5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:50878/0 (socket says 172.21.15.7:50878) 2024-06-10T01:30:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.327+0000 7f1795f37640 1 -- 172.21.15.7:0/3267821742 learned_addr learned my addr 172.21.15.7:0/3267821742 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.327+0000 7f1796738640 1 --2- 172.21.15.7:0/3267821742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9f6ec3380 0x55f9f6ec7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.327+0000 7f1795736640 1 --2- 172.21.15.7:0/3267821742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f6ebe210 0x55f9f6ecf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.328+0000 7f1795f37640 1 -- 172.21.15.7:0/3267821742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f6ebe210 msgr2=0x55f9f6ecf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.328+0000 7f1795f37640 1 --2- 172.21.15.7:0/3267821742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f6ebe210 0x55f9f6ecf0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.328+0000 7f1795f37640 1 -- 172.21.15.7:0/3267821742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9f6ec3380 msgr2=0x55f9f6ec7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.328+0000 7f1795f37640 1 --2- 172.21.15.7:0/3267821742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9f6ec3380 0x55f9f6ec7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.328+0000 7f1795f37640 1 -- 172.21.15.7:0/3267821742 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9f6eb04b0 con 0x55f9f6ebd820 2024-06-10T01:30:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.328+0000 7f1796738640 1 --2- 172.21.15.7:0/3267821742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9f6ec3380 0x55f9f6ec7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:30:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.331+0000 7f1795f37640 1 --2- 172.21.15.7:0/3267821742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f9f6ebd820 0x55f9f6ec5770 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f178c098520 tx=0x7f178c0b6040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c22de9789fc17e00 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.332+0000 7f177b7fe640 1 -- 172.21.15.7:0/3267821742 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c0a4810 con 0x55f9f6ebd820 2024-06-10T01:30:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.332+0000 7f177b7fe640 1 -- 172.21.15.7:0/3267821742 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f9f6e2bf10 con 0x55f9f6ebd820 2024-06-10T01:30:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.332+0000 7f177b7fe640 1 -- 172.21.15.7:0/3267821742 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c09b250 con 0x55f9f6ebd820 2024-06-10T01:30:43.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.333+0000 7f179af83740 1 -- 172.21.15.7:0/3267821742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f9f6ebd820 msgr2=0x55f9f6ec5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:43.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.333+0000 7f179af83740 1 --2- 172.21.15.7:0/3267821742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f9f6ebd820 0x55f9f6ec5770 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f178c098520 tx=0x7f178c0b6040 comp rx=0 tx=0).stop 2024-06-10T01:30:43.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.333+0000 7f179af83740 1 -- 172.21.15.7:0/3267821742 shutdown_connections 2024-06-10T01:30:43.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.333+0000 7f179af83740 1 --2- 172.21.15.7:0/3267821742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9f6ec3380 0x55f9f6ec7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:43.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.333+0000 7f179af83740 1 --2- 172.21.15.7:0/3267821742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f6ebe210 0x55f9f6ecf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:43.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.333+0000 7f179af83740 1 --2- 172.21.15.7:0/3267821742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f9f6ebd820 0x55f9f6ec5770 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:43.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.333+0000 7f179af83740 1 -- 172.21.15.7:0/3267821742 >> 172.21.15.7:0/3267821742 conn(0x55f9f6eb0680 msgr2=0x55f9f6eb0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:43.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.333+0000 7f179af83740 1 -- 172.21.15.7:0/3267821742 shutdown_connections 2024-06-10T01:30:43.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.333+0000 7f179af83740 1 -- 172.21.15.7:0/3267821742 wait complete. 2024-06-10T01:30:43.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.333+0000 7f179af83740 1 -- start start 2024-06-10T01:30:43.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.333+0000 7f179af83740 1 Processor -- start 2024-06-10T01:30:43.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.334+0000 7f179af83740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f9f6ebd820 0x55f9f6ecf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:43.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.334+0000 7f179af83740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f6ebe210 0x55f9f6ecbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:43.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.334+0000 7f179af83740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9f6ec3380 0x55f9f6ecc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:43.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.334+0000 7f179af83740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f17901a6a30 con 0x55f9f6ec3380 2024-06-10T01:30:43.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.334+0000 7f179af83740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f17901a7ea0 con 0x55f9f6ebe210 2024-06-10T01:30:43.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.334+0000 7f179af83740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f17901a7100 con 0x55f9f6ebd820 2024-06-10T01:30:43.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.334+0000 7f179af83740 10 client.0 _ll_register_callbacks cb 0x7f178000f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:30:43.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.334+0000 7f1795f37640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f9f6ebd820 0x55f9f6ecf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.334+0000 7f1795f37640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f9f6ebd820 0x55f9f6ecf190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:50884/0 (socket says 172.21.15.7:50884) 2024-06-10T01:30:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.334+0000 7f1795f37640 1 -- 172.21.15.7:0/322465272 learned_addr learned my addr 172.21.15.7:0/322465272 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.334+0000 7f1795736640 1 --2- 172.21.15.7:0/322465272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f6ebe210 0x55f9f6ecbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.334+0000 7f1796738640 1 --2- 172.21.15.7:0/322465272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9f6ec3380 0x55f9f6ecc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.334+0000 7f1795f37640 1 -- 172.21.15.7:0/322465272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f6ebe210 msgr2=0x55f9f6ecbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.334+0000 7f1795f37640 1 --2- 172.21.15.7:0/322465272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f6ebe210 0x55f9f6ecbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.334+0000 7f1795f37640 1 -- 172.21.15.7:0/322465272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9f6ec3380 msgr2=0x55f9f6ecc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.335+0000 7f1795f37640 1 --2- 172.21.15.7:0/322465272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9f6ec3380 0x55f9f6ecc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.335+0000 7f1795f37640 1 -- 172.21.15.7:0/322465272 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9f6f3c770 con 0x55f9f6ebd820 2024-06-10T01:30:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.335+0000 7f1795f37640 1 --2- 172.21.15.7:0/322465272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f9f6ebd820 0x55f9f6ecf190 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f178c09bd60 tx=0x7f178c09ffe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa28ba27070a8001 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.335+0000 7f17797fa640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:30:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.335+0000 7f17797fa640 1 -- 172.21.15.7:0/322465272 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c098210 con 0x55f9f6ebd820 2024-06-10T01:30:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.335+0000 7f17797fa640 1 -- 172.21.15.7:0/322465272 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f178c09aff0 con 0x55f9f6ebd820 2024-06-10T01:30:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.335+0000 7f179af83740 1 -- 172.21.15.7:0/322465272 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f9f6e2bf10 con 0x55f9f6ebd820 2024-06-10T01:30:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.335+0000 7f17797fa640 1 -- 172.21.15.7:0/322465272 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f178c0a4810 con 0x55f9f6ebd820 2024-06-10T01:30:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.335+0000 7f179af83740 10 client.30334 fetch_fsmap learned FSMap version 57 2024-06-10T01:30:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.335+0000 7f179af83740 1 -- 172.21.15.7:0/322465272 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55f9f6f3da80 con 0x55f9f6ebd820 2024-06-10T01:30:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.335+0000 7f17797fa640 1 -- 172.21.15.7:0/322465272 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f9f6e2bf10 con 0x55f9f6ebd820 2024-06-10T01:30:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.336+0000 7f17797fa640 1 -- 172.21.15.7:0/322465272 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7f178c0ba410 con 0x55f9f6ebd820 2024-06-10T01:30:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.336+0000 7f179af83740 10 client.30334 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:30:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.336+0000 7f179af83740 10 client.30334 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:30:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.336+0000 7f179af83740 20 client.30334 populate_metadata read hostname 'smithi007' 2024-06-10T01:30:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.336+0000 7f179af83740 1 --2- 172.21.15.7:0/322465272 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f9f6f3def0 0x55f9f6f402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.336+0000 7f179af83740 4 client.30334 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:30:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.336+0000 7f179af83740 1 -- 172.21.15.7:0/322465272 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f9f6e25c20 con 0x55f9f6f3def0 2024-06-10T01:30:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.336+0000 7f1795736640 1 --2- 172.21.15.7:0/322465272 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f9f6f3def0 0x55f9f6f402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.337+0000 7f1795736640 1 --2- 172.21.15.7:0/322465272 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f9f6f3def0 0x55f9f6f402e0 crc :-1 s=READY pgs=552 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e88009b4ba21d32 server_cookie=6251819aa0e33341 in_seq=0 out_seq=0 2024-06-10T01:30:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:43.337+0000 7f17797fa640 10 client.30334 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:43.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:43 smithi175 ceph-mon[36636]: pgmap v1175: 129 pgs: 129 active+clean; 683 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.5 MiB/s wr, 483 op/s 2024-06-10T01:30:43.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:43 smithi007 ceph-mon[32398]: pgmap v1175: 129 pgs: 129 active+clean; 683 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.5 MiB/s wr, 483 op/s 2024-06-10T01:30:43.568 INFO:tasks.workunit.client.0.smithi007.stdout: 10 265279 21.62 MB/sec execute 560 sec latency 161.720 ms 2024-06-10T01:30:43.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:43 smithi078 ceph-mon[36683]: pgmap v1175: 129 pgs: 129 active+clean; 683 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.5 MiB/s wr, 483 op/s 2024-06-10T01:30:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:44.151+0000 7f17797fa640 1 -- 172.21.15.7:0/322465272 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f9f6e25c20 con 0x55f9f6f3def0 2024-06-10T01:30:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:44.151+0000 7f17797fa640 10 client.30334 handle_command_reply: tid=0 2024-06-10T01:30:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:44.151+0000 7f179af83740 1 client.30334 shutdown 2024-06-10T01:30:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:44.151+0000 7f179af83740 1 -- 172.21.15.7:0/322465272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f9f6ebd820 msgr2=0x55f9f6ecf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:44.151+0000 7f179af83740 1 --2- 172.21.15.7:0/322465272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f9f6ebd820 0x55f9f6ecf190 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f178c09bd60 tx=0x7f178c09ffe0 comp rx=0 tx=0).stop 2024-06-10T01:30:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:44.152+0000 7f179af83740 1 -- 172.21.15.7:0/322465272 shutdown_connections 2024-06-10T01:30:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:44.152+0000 7f179af83740 1 --2- 172.21.15.7:0/322465272 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f9f6f3def0 0x55f9f6f402e0 crc :-1 s=READY pgs=552 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:44.152+0000 7f17797fa640 0 client.30334 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:44.152+0000 7f179af83740 1 --2- 172.21.15.7:0/322465272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9f6ec3380 0x55f9f6ecc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:44.152+0000 7f179af83740 1 --2- 172.21.15.7:0/322465272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f6ebe210 0x55f9f6ecbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:44.152+0000 7f179af83740 1 --2- 172.21.15.7:0/322465272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f9f6ebd820 0x55f9f6ecf190 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:44.152+0000 7f179af83740 1 -- 172.21.15.7:0/322465272 >> 172.21.15.7:0/322465272 conn(0x55f9f6eb0680 msgr2=0x55f9f6ec5770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:44.152+0000 7f179af83740 1 -- 172.21.15.7:0/322465272 shutdown_connections 2024-06-10T01:30:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:44.152+0000 7f179af83740 1 -- 172.21.15.7:0/322465272 wait complete. 2024-06-10T01:30:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:44.152+0000 7f179af83740 20 client.30334 trim_cache size 0 max 16384 2024-06-10T01:30:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:30:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:30:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:30:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:44.153+0000 7f179819b640 1 -- 172.21.15.7:0/608802959 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f177c0a2c00 msgr2=0x7f177c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:44.153+0000 7f179819b640 1 --2- 172.21.15.7:0/608802959 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f177c0a2c00 0x7f177c0a50c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f1788005fd0 tx=0x7f178800f8e0 comp rx=0 tx=0).stop 2024-06-10T01:30:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:44.153+0000 7f179819b640 1 -- 172.21.15.7:0/608802959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1790102540 msgr2=0x7f1790195af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:44.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:44.153+0000 7f179819b640 1 --2- 172.21.15.7:0/608802959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1790102540 0x7f1790195af0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f178c00e9f0 tx=0x7f178c00eeb0 comp rx=0 tx=0).stop 2024-06-10T01:30:44.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:44.153+0000 7f179819b640 1 -- 172.21.15.7:0/608802959 shutdown_connections 2024-06-10T01:30:44.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:44.154+0000 7f179819b640 1 --2- 172.21.15.7:0/608802959 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f177c0a2c00 0x7f177c0a50c0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:44.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:44.154+0000 7f179819b640 1 --2- 172.21.15.7:0/608802959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17901077a0 0x7f179019d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:44.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:44.154+0000 7f179819b640 1 --2- 172.21.15.7:0/608802959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1790104e70 0x7f1790196030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:44.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:44.154+0000 7f179819b640 1 --2- 172.21.15.7:0/608802959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1790102540 0x7f1790195af0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:44.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:44.154+0000 7f179819b640 1 -- 172.21.15.7:0/608802959 >> 172.21.15.7:0/608802959 conn(0x7f17900fbe70 msgr2=0x7f1790102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:44.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:44.154+0000 7f179819b640 1 -- 172.21.15.7:0/608802959 shutdown_connections 2024-06-10T01:30:44.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:44.154+0000 7f179819b640 1 -- 172.21.15.7:0/608802959 wait complete. 2024-06-10T01:30:44.568 INFO:tasks.workunit.client.0.smithi007.stdout: 10 265750 21.64 MB/sec execute 561 sec latency 441.754 ms 2024-06-10T01:30:45.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:45 smithi175 ceph-mon[36636]: pgmap v1176: 129 pgs: 129 active+clean; 684 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.0 MiB/s wr, 393 op/s 2024-06-10T01:30:45.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:45 smithi007 ceph-mon[32398]: pgmap v1176: 129 pgs: 129 active+clean; 684 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.0 MiB/s wr, 393 op/s 2024-06-10T01:30:45.568 INFO:tasks.workunit.client.0.smithi007.stdout: 10 266162 21.64 MB/sec execute 562 sec latency 478.003 ms 2024-06-10T01:30:45.576 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:45 smithi078 ceph-mon[36683]: pgmap v1176: 129 pgs: 129 active+clean; 684 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.0 MiB/s wr, 393 op/s 2024-06-10T01:30:46.568 INFO:tasks.workunit.client.0.smithi007.stdout: 10 266596 21.63 MB/sec execute 563 sec latency 623.134 ms 2024-06-10T01:30:47.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:47 smithi175 ceph-mon[36636]: pgmap v1177: 129 pgs: 129 active+clean; 684 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.9 MiB/s wr, 586 op/s 2024-06-10T01:30:47.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:47 smithi007 ceph-mon[32398]: pgmap v1177: 129 pgs: 129 active+clean; 684 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.9 MiB/s wr, 586 op/s 2024-06-10T01:30:47.568 INFO:tasks.workunit.client.0.smithi007.stdout: 10 267322 21.66 MB/sec execute 564 sec latency 248.598 ms 2024-06-10T01:30:47.576 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:47 smithi078 ceph-mon[36683]: pgmap v1177: 129 pgs: 129 active+clean; 684 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.9 MiB/s wr, 586 op/s 2024-06-10T01:30:48.568 INFO:tasks.workunit.client.0.smithi007.stdout: 10 267975 21.69 MB/sec execute 565 sec latency 79.943 ms 2024-06-10T01:30:49.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:49 smithi175 ceph-mon[36636]: pgmap v1178: 129 pgs: 129 active+clean; 688 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 3.4 MiB/s wr, 550 op/s 2024-06-10T01:30:49.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:49 smithi007 ceph-mon[32398]: pgmap v1178: 129 pgs: 129 active+clean; 688 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 3.4 MiB/s wr, 550 op/s 2024-06-10T01:30:49.568 INFO:tasks.workunit.client.0.smithi007.stdout: 10 268629 21.72 MB/sec execute 566 sec latency 67.576 ms 2024-06-10T01:30:49.576 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:49 smithi078 ceph-mon[36683]: pgmap v1178: 129 pgs: 129 active+clean; 688 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 3.4 MiB/s wr, 550 op/s 2024-06-10T01:30:50.568 INFO:tasks.workunit.client.0.smithi007.stdout: 10 269161 21.72 MB/sec execute 567 sec latency 90.942 ms 2024-06-10T01:30:51.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:51 smithi175 ceph-mon[36636]: pgmap v1179: 129 pgs: 129 active+clean; 692 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.2 MiB/s wr, 538 op/s 2024-06-10T01:30:51.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:51 smithi007 ceph-mon[32398]: pgmap v1179: 129 pgs: 129 active+clean; 692 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.2 MiB/s wr, 538 op/s 2024-06-10T01:30:51.568 INFO:tasks.workunit.client.0.smithi007.stdout: 10 269693 21.74 MB/sec execute 568 sec latency 118.043 ms 2024-06-10T01:30:51.576 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:51 smithi078 ceph-mon[36683]: pgmap v1179: 129 pgs: 129 active+clean; 692 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.2 MiB/s wr, 538 op/s 2024-06-10T01:30:52.569 INFO:tasks.workunit.client.0.smithi007.stdout: 10 270468 21.77 MB/sec execute 569 sec latency 62.348 ms 2024-06-10T01:30:53.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:53 smithi175 ceph-mon[36636]: pgmap v1180: 129 pgs: 129 active+clean; 674 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.8 MiB/s wr, 610 op/s 2024-06-10T01:30:53.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:53 smithi007 ceph-mon[32398]: pgmap v1180: 129 pgs: 129 active+clean; 674 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.8 MiB/s wr, 610 op/s 2024-06-10T01:30:53.569 INFO:tasks.workunit.client.0.smithi007.stdout: 10 271105 21.80 MB/sec execute 570 sec latency 66.556 ms 2024-06-10T01:30:53.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:53 smithi078 ceph-mon[36683]: pgmap v1180: 129 pgs: 129 active+clean; 674 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.8 MiB/s wr, 610 op/s 2024-06-10T01:30:54.569 INFO:tasks.workunit.client.0.smithi007.stdout: 10 271651 21.81 MB/sec execute 571 sec latency 80.228 ms 2024-06-10T01:30:55.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:55 smithi175 ceph-mon[36636]: pgmap v1181: 129 pgs: 129 active+clean; 673 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.1 MiB/s wr, 531 op/s 2024-06-10T01:30:55.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:55 smithi007 ceph-mon[32398]: pgmap v1181: 129 pgs: 129 active+clean; 673 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.1 MiB/s wr, 531 op/s 2024-06-10T01:30:55.569 INFO:tasks.workunit.client.0.smithi007.stdout: 10 272256 21.83 MB/sec execute 572 sec latency 65.425 ms 2024-06-10T01:30:55.576 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:55 smithi078 ceph-mon[36683]: pgmap v1181: 129 pgs: 129 active+clean; 673 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.1 MiB/s wr, 531 op/s 2024-06-10T01:30:56.569 INFO:tasks.workunit.client.0.smithi007.stdout: 10 272814 21.84 MB/sec execute 573 sec latency 185.901 ms 2024-06-10T01:30:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.290+0000 7f9b0c29e640 1 -- 172.21.15.7:0/3988643143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b04100940 msgr2=0x7f9b04100d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.290+0000 7f9b0c29e640 1 --2- 172.21.15.7:0/3988643143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b04100940 0x7f9b04100d60 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009900 tx=0x7f9b0002a410 comp rx=0 tx=0).stop 2024-06-10T01:30:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.291+0000 7f9b0c29e640 1 -- 172.21.15.7:0/3988643143 shutdown_connections 2024-06-10T01:30:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.291+0000 7f9b0c29e640 1 --2- 172.21.15.7:0/3988643143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b041024f0 0x7f9b04108d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.291+0000 7f9b0c29e640 1 --2- 172.21.15.7:0/3988643143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b04101b30 0x7f9b04101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.291+0000 7f9b0c29e640 1 --2- 172.21.15.7:0/3988643143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b04100940 0x7f9b04100d60 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.291+0000 7f9b0c29e640 1 -- 172.21.15.7:0/3988643143 >> 172.21.15.7:0/3988643143 conn(0x7f9b040fbed0 msgr2=0x7f9b040fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.291+0000 7f9b0c29e640 1 -- 172.21.15.7:0/3988643143 shutdown_connections 2024-06-10T01:30:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.291+0000 7f9b0c29e640 1 -- 172.21.15.7:0/3988643143 wait complete. 2024-06-10T01:30:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.292+0000 7f9b0c29e640 1 Processor -- start 2024-06-10T01:30:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.292+0000 7f9b0c29e640 1 -- start start 2024-06-10T01:30:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.292+0000 7f9b0c29e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b04100940 0x7f9b04197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.292+0000 7f9b0c29e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b04101b30 0x7f9b04198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.292+0000 7f9b0c29e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b041024f0 0x7f9b0419f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.292+0000 7f9b0c29e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0410c5b0 con 0x7f9b04101b30 2024-06-10T01:30:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.292+0000 7f9b0c29e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0410c430 con 0x7f9b04100940 2024-06-10T01:30:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.292+0000 7f9b0c29e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0410c730 con 0x7f9b041024f0 2024-06-10T01:30:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.292+0000 7f9b09839640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b04101b30 0x7f9b04198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.292+0000 7f9b09839640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b04101b30 0x7f9b04198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33386/0 (socket says 172.21.15.7:33386) 2024-06-10T01:30:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.292+0000 7f9b09839640 1 -- 172.21.15.7:0/3132453257 learned_addr learned my addr 172.21.15.7:0/3132453257 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.292+0000 7f9b0a83b640 1 --2- 172.21.15.7:0/3132453257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b041024f0 0x7f9b0419f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.292+0000 7f9b0a03a640 1 --2- 172.21.15.7:0/3132453257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b04100940 0x7f9b04197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.293+0000 7f9b09839640 1 -- 172.21.15.7:0/3132453257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b041024f0 msgr2=0x7f9b0419f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.293+0000 7f9b09839640 1 --2- 172.21.15.7:0/3132453257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b041024f0 0x7f9b0419f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.293+0000 7f9b09839640 1 -- 172.21.15.7:0/3132453257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b04100940 msgr2=0x7f9b04197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.293+0000 7f9b09839640 1 --2- 172.21.15.7:0/3132453257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b04100940 0x7f9b04197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.293+0000 7f9b09839640 1 -- 172.21.15.7:0/3132453257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b0419f960 con 0x7f9b04101b30 2024-06-10T01:30:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.293+0000 7f9b09839640 1 --2- 172.21.15.7:0/3132453257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b04101b30 0x7f9b04198290 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f9af400e9f0 tx=0x7f9af400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.293+0000 7f9afb7fe640 1 -- 172.21.15.7:0/3132453257 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af400cd80 con 0x7f9b04101b30 2024-06-10T01:30:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.293+0000 7f9afb7fe640 1 -- 172.21.15.7:0/3132453257 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9af4005500 con 0x7f9b04101b30 2024-06-10T01:30:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.293+0000 7f9afb7fe640 1 -- 172.21.15.7:0/3132453257 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af4010690 con 0x7f9b04101b30 2024-06-10T01:30:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.293+0000 7f9b0c29e640 1 -- 172.21.15.7:0/3132453257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b0419fc50 con 0x7f9b04101b30 2024-06-10T01:30:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.293+0000 7f9b0c29e640 1 -- 172.21.15.7:0/3132453257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b041a0120 con 0x7f9b04101b30 2024-06-10T01:30:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.296+0000 7f9afb7fe640 1 -- 172.21.15.7:0/3132453257 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9af40046c0 con 0x7f9b04101b30 2024-06-10T01:30:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.297+0000 7f9b0efad740 1 Processor -- start 2024-06-10T01:30:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.297+0000 7f9afb7fe640 1 --2- 172.21.15.7:0/3132453257 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9af00a2c00 0x7f9af00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.297+0000 7f9b0efad740 1 -- start start 2024-06-10T01:30:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.297+0000 7f9b0efad740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580597c7420 0x5580597c7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.297+0000 7f9b0a83b640 1 --2- 172.21.15.7:0/3132453257 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9af00a2c00 0x7f9af00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.297+0000 7f9b0efad740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580597c7e10 0x5580597cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.297+0000 7f9afb7fe640 1 -- 172.21.15.7:0/3132453257 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f9af40cf7d0 con 0x7f9b04101b30 2024-06-10T01:30:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.297+0000 7f9b0efad740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580597ccf80 0x5580597cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.297+0000 7f9b0efad740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5580597264f0 con 0x5580597c7e10 2024-06-10T01:30:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.297+0000 7f9b0efad740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5580596c4870 con 0x5580597ccf80 2024-06-10T01:30:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.297+0000 7f9b0efad740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5580597224b0 con 0x5580597c7420 2024-06-10T01:30:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.297+0000 7f9b09839640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580597c7e10 0x5580597cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.298+0000 7f9b0a83b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580597ccf80 0x5580597cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.298+0000 7f9b09839640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580597c7e10 0x5580597cc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33400/0 (socket says 172.21.15.7:33400) 2024-06-10T01:30:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.298+0000 7f9b0a83b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580597ccf80 0x5580597cf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:52300/0 (socket says 172.21.15.7:52300) 2024-06-10T01:30:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.298+0000 7f9b09839640 1 -- 172.21.15.7:0/1114075996 learned_addr learned my addr 172.21.15.7:0/1114075996 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.298+0000 7f9b0a03a640 1 --2- 172.21.15.7:0/1114075996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580597c7420 0x5580597c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.298+0000 7f9b0a83b640 1 --2- 172.21.15.7:0/3132453257 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9af00a2c00 0x7f9af00a50c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f9afc005fd0 tx=0x7f9afc011bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.298+0000 7f9b0a83b640 1 -- 172.21.15.7:0/1114075996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580597c7420 msgr2=0x5580597c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.298+0000 7f9b0a83b640 1 --2- 172.21.15.7:0/1114075996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580597c7420 0x5580597c7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.298+0000 7f9b0a83b640 1 -- 172.21.15.7:0/1114075996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580597c7e10 msgr2=0x5580597cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.298+0000 7f9b0a83b640 1 --2- 172.21.15.7:0/1114075996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580597c7e10 0x5580597cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.298+0000 7f9b0a83b640 1 -- 172.21.15.7:0/1114075996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580596e09b0 con 0x5580597ccf80 2024-06-10T01:30:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.298+0000 7f9b0a83b640 1 --2- 172.21.15.7:0/1114075996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580597ccf80 0x5580597cf410 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00fce0 tx=0x7f9afc020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58047adebc2d0258 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.298+0000 7f9af8bfe640 1 -- 172.21.15.7:0/1114075996 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc01a040 con 0x5580597ccf80 2024-06-10T01:30:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.298+0000 7f9af8bfe640 1 -- 172.21.15.7:0/1114075996 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9afc024030 con 0x5580597ccf80 2024-06-10T01:30:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.298+0000 7f9af8bfe640 1 -- 172.21.15.7:0/1114075996 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc00e3b0 con 0x5580597ccf80 2024-06-10T01:30:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.299+0000 7f9b0efad740 1 -- 172.21.15.7:0/1114075996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580597ccf80 msgr2=0x5580597cf410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.299+0000 7f9b0efad740 1 --2- 172.21.15.7:0/1114075996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580597ccf80 0x5580597cf410 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00fce0 tx=0x7f9afc020000 comp rx=0 tx=0).stop 2024-06-10T01:30:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.299+0000 7f9b0efad740 1 -- 172.21.15.7:0/1114075996 shutdown_connections 2024-06-10T01:30:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.299+0000 7f9b0efad740 1 --2- 172.21.15.7:0/1114075996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580597ccf80 0x5580597cf410 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.299+0000 7f9b0efad740 1 --2- 172.21.15.7:0/1114075996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580597c7e10 0x5580597cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.299+0000 7f9b0efad740 1 --2- 172.21.15.7:0/1114075996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580597c7420 0x5580597c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.299+0000 7f9b0efad740 1 -- 172.21.15.7:0/1114075996 >> 172.21.15.7:0/1114075996 conn(0x5580597be680 msgr2=0x5580597bea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.299+0000 7f9b0efad740 1 -- 172.21.15.7:0/1114075996 shutdown_connections 2024-06-10T01:30:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.299+0000 7f9b0efad740 1 -- 172.21.15.7:0/1114075996 wait complete. 2024-06-10T01:30:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.300+0000 7f9b0efad740 1 -- start start 2024-06-10T01:30:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.300+0000 7f9b0efad740 1 Processor -- start 2024-06-10T01:30:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.300+0000 7f9b0efad740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580597c7420 0x5580597ca460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.300+0000 7f9b0efad740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580597c7e10 0x5580597ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.300+0000 7f9b0efad740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580597ccf80 0x5580597cb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.300+0000 7f9b0efad740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5580597c2ed0 con 0x5580597c7e10 2024-06-10T01:30:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.300+0000 7f9b0efad740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5580596afae0 con 0x5580597c7420 2024-06-10T01:30:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.300+0000 7f9b0efad740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5580597c3050 con 0x5580597ccf80 2024-06-10T01:30:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.300+0000 7f9b0efad740 10 client.0 _ll_register_callbacks cb 0x5580596e89b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:30:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.300+0000 7f9b0a03a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580597c7420 0x5580597ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.300+0000 7f9b0a03a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580597c7420 0x5580597ca460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:52314/0 (socket says 172.21.15.7:52314) 2024-06-10T01:30:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.301+0000 7f9b0a03a640 1 -- 172.21.15.7:0/325831752 learned_addr learned my addr 172.21.15.7:0/325831752 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.301+0000 7f9b0a83b640 1 --2- 172.21.15.7:0/325831752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580597ccf80 0x5580597cb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.301+0000 7f9b09839640 1 --2- 172.21.15.7:0/325831752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580597c7e10 0x5580597ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.301+0000 7f9b0a03a640 1 -- 172.21.15.7:0/325831752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580597ccf80 msgr2=0x5580597cb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.301+0000 7f9b0a03a640 1 --2- 172.21.15.7:0/325831752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580597ccf80 0x5580597cb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.301+0000 7f9b0a03a640 1 -- 172.21.15.7:0/325831752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580597c7e10 msgr2=0x5580597ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.301+0000 7f9b0a03a640 1 --2- 172.21.15.7:0/325831752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580597c7e10 0x5580597ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.301+0000 7f9b0a03a640 1 -- 172.21.15.7:0/325831752 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55805943a210 con 0x5580597c7420 2024-06-10T01:30:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.301+0000 7f9b09839640 1 --2- 172.21.15.7:0/325831752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580597c7e10 0x5580597ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:30:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.301+0000 7f9b0a03a640 1 --2- 172.21.15.7:0/325831752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580597c7420 0x5580597ca460 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009900 tx=0x7f9b00032ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7578f5fadb008f2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.301+0000 7f9aedffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:30:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.301+0000 7f9aedffb640 1 -- 172.21.15.7:0/325831752 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00035040 con 0x5580597c7420 2024-06-10T01:30:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.302+0000 7f9b0efad740 1 -- 172.21.15.7:0/325831752 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9afc024030 con 0x5580597c7420 2024-06-10T01:30:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.302+0000 7f9aedffb640 1 -- 172.21.15.7:0/325831752 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f9b00041d70 con 0x5580597c7420 2024-06-10T01:30:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.302+0000 7f9aedffb640 1 -- 172.21.15.7:0/325831752 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f9b00055050 con 0x5580597c7420 2024-06-10T01:30:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.302+0000 7f9b0efad740 10 client.30458 fetch_fsmap learned FSMap version 57 2024-06-10T01:30:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.302+0000 7f9b0efad740 1 -- 172.21.15.7:0/325831752 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x558059506620 con 0x5580597c7420 2024-06-10T01:30:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.302+0000 7f9aedffb640 1 -- 172.21.15.7:0/325831752 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9afc024030 con 0x5580597c7420 2024-06-10T01:30:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.302+0000 7f9aedffb640 1 -- 172.21.15.7:0/325831752 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7f9b000051f0 con 0x5580597c7420 2024-06-10T01:30:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.302+0000 7f9b0efad740 10 client.30458 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:30:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.302+0000 7f9b0efad740 10 client.30458 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:30:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.302+0000 7f9b0efad740 20 client.30458 populate_metadata read hostname 'smithi007' 2024-06-10T01:30:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.302+0000 7f9b0efad740 1 --2- 172.21.15.7:0/325831752 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5580597d94b0 0x5580597db950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.302+0000 7f9b0efad740 4 client.30458 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:30:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.302+0000 7f9b0efad740 1 -- 172.21.15.7:0/325831752 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558059739f10 con 0x5580597d94b0 2024-06-10T01:30:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.303+0000 7f9b09839640 1 --2- 172.21.15.7:0/325831752 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5580597d94b0 0x5580597db950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.303+0000 7f9aedffb640 10 client.30458 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.303+0000 7f9b09839640 1 --2- 172.21.15.7:0/325831752 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5580597d94b0 0x5580597db950 crc :-1 s=READY pgs=553 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c082a3767f74b02 server_cookie=687989ba649df2d2 in_seq=0 out_seq=0 2024-06-10T01:30:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.304+0000 7f9aedffb640 1 -- 172.21.15.7:0/325831752 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558059739f10 con 0x5580597d94b0 2024-06-10T01:30:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.304+0000 7f9aedffb640 10 client.30458 handle_command_reply: tid=0 2024-06-10T01:30:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.304+0000 7f9b0efad740 1 client.30458 shutdown 2024-06-10T01:30:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.304+0000 7f9b0efad740 1 -- 172.21.15.7:0/325831752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580597c7420 msgr2=0x5580597ca460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.304+0000 7f9b0efad740 1 --2- 172.21.15.7:0/325831752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580597c7420 0x5580597ca460 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009900 tx=0x7f9b00032ed0 comp rx=0 tx=0).stop 2024-06-10T01:30:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.304+0000 7f9b0efad740 1 -- 172.21.15.7:0/325831752 shutdown_connections 2024-06-10T01:30:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.304+0000 7f9b0efad740 1 --2- 172.21.15.7:0/325831752 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5580597d94b0 0x5580597db950 crc :-1 s=READY pgs=553 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.304+0000 7f9b0efad740 1 --2- 172.21.15.7:0/325831752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580597ccf80 0x5580597cb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.304+0000 7f9b0efad740 1 --2- 172.21.15.7:0/325831752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580597c7e10 0x5580597ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.304+0000 7f9b0efad740 1 --2- 172.21.15.7:0/325831752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580597c7420 0x5580597ca460 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.304+0000 7f9b0efad740 1 -- 172.21.15.7:0/325831752 >> 172.21.15.7:0/325831752 conn(0x5580597be680 msgr2=0x5580597bd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.304+0000 7f9aedffb640 0 client.30458 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.304+0000 7f9b0efad740 1 -- 172.21.15.7:0/325831752 shutdown_connections 2024-06-10T01:30:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.304+0000 7f9b0efad740 1 -- 172.21.15.7:0/325831752 wait complete. 2024-06-10T01:30:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.304+0000 7f9b0efad740 20 client.30458 trim_cache size 0 max 16384 2024-06-10T01:30:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.323+0000 7f9b0efad740 1 Processor -- start 2024-06-10T01:30:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.323+0000 7f9b0efad740 1 -- start start 2024-06-10T01:30:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.323+0000 7f9b0efad740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580597c7e10 0x5580597c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.323+0000 7f9b0efad740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580597ccf80 0x5580597c9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.323+0000 7f9b0efad740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580597c43a0 0x5580597c1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.323+0000 7f9b0efad740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9b041aac30 con 0x5580597ccf80 2024-06-10T01:30:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.323+0000 7f9b0efad740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9b041a84a0 con 0x5580597c7e10 2024-06-10T01:30:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.323+0000 7f9b0efad740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9b041a62f0 con 0x5580597c43a0 2024-06-10T01:30:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.323+0000 7f9b09839640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580597ccf80 0x5580597c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.323+0000 7f9b0a03a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580597c7e10 0x5580597c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.323+0000 7f9b0a03a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580597c7e10 0x5580597c8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:52326/0 (socket says 172.21.15.7:52326) 2024-06-10T01:30:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.323+0000 7f9b09839640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580597ccf80 0x5580597c9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33414/0 (socket says 172.21.15.7:33414) 2024-06-10T01:30:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.323+0000 7f9b0a03a640 1 -- 172.21.15.7:0/800705711 learned_addr learned my addr 172.21.15.7:0/800705711 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.323+0000 7f9b0a83b640 1 --2- 172.21.15.7:0/800705711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580597c43a0 0x5580597c1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.323+0000 7f9b0a03a640 1 -- 172.21.15.7:0/800705711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580597c43a0 msgr2=0x5580597c1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.323+0000 7f9b0a03a640 1 --2- 172.21.15.7:0/800705711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580597c43a0 0x5580597c1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.323+0000 7f9b0a03a640 1 -- 172.21.15.7:0/800705711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580597ccf80 msgr2=0x5580597c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.323+0000 7f9b0a03a640 1 --2- 172.21.15.7:0/800705711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580597ccf80 0x5580597c9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.323+0000 7f9b0a03a640 1 -- 172.21.15.7:0/800705711 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580597be4b0 con 0x5580597c7e10 2024-06-10T01:30:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.324+0000 7f9b0a03a640 1 --2- 172.21.15.7:0/800705711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580597c7e10 0x5580597c8ac0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9b00000c00 tx=0x7f9b00009830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f7bde3a1c869ec5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.324+0000 7f9aeffff640 1 -- 172.21.15.7:0/800705711 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0005e070 con 0x5580597c7e10 2024-06-10T01:30:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.324+0000 7f9aeffff640 1 -- 172.21.15.7:0/800705711 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b00041d70 con 0x5580597c7e10 2024-06-10T01:30:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.324+0000 7f9aeffff640 1 -- 172.21.15.7:0/800705711 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00035030 con 0x5580597c7e10 2024-06-10T01:30:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.325+0000 7f9b0efad740 1 -- 172.21.15.7:0/800705711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580597c7e10 msgr2=0x5580597c8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.325+0000 7f9b0efad740 1 --2- 172.21.15.7:0/800705711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580597c7e10 0x5580597c8ac0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9b00000c00 tx=0x7f9b00009830 comp rx=0 tx=0).stop 2024-06-10T01:30:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.325+0000 7f9b0efad740 1 -- 172.21.15.7:0/800705711 shutdown_connections 2024-06-10T01:30:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.325+0000 7f9b0efad740 1 --2- 172.21.15.7:0/800705711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580597c43a0 0x5580597c1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.325+0000 7f9b0efad740 1 --2- 172.21.15.7:0/800705711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580597ccf80 0x5580597c9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.325+0000 7f9b0efad740 1 --2- 172.21.15.7:0/800705711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580597c7e10 0x5580597c8ac0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.325+0000 7f9b0efad740 1 -- 172.21.15.7:0/800705711 >> 172.21.15.7:0/800705711 conn(0x5580597be680 msgr2=0x5580597c3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.325+0000 7f9b0efad740 1 -- 172.21.15.7:0/800705711 shutdown_connections 2024-06-10T01:30:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.325+0000 7f9b0efad740 1 -- 172.21.15.7:0/800705711 wait complete. 2024-06-10T01:30:57.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.325+0000 7f9b0efad740 1 -- start start 2024-06-10T01:30:57.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.325+0000 7f9b0efad740 1 Processor -- start 2024-06-10T01:30:57.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.326+0000 7f9b0efad740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580597c43a0 0x5580597d51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:57.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.326+0000 7f9b0efad740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580597c7e10 0x5580597d5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:57.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.326+0000 7f9b0efad740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580597ccf80 0x5580597d1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:57.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.326+0000 7f9b0efad740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9b041a6960 con 0x5580597ccf80 2024-06-10T01:30:57.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.326+0000 7f9b0efad740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9b041a7dd0 con 0x5580597c7e10 2024-06-10T01:30:57.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.326+0000 7f9b0efad740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9b041a7030 con 0x5580597c43a0 2024-06-10T01:30:57.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.326+0000 7f9b0a03a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580597c43a0 0x5580597d51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:57.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.326+0000 7f9b09839640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580597c7e10 0x5580597d5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:57.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.326+0000 7f9b0a83b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580597ccf80 0x5580597d1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:57.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.326+0000 7f9b0a03a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580597c43a0 0x5580597d51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:39124/0 (socket says 172.21.15.7:39124) 2024-06-10T01:30:57.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.326+0000 7f9b0a03a640 1 -- 172.21.15.7:0/1211357488 learned_addr learned my addr 172.21.15.7:0/1211357488 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:30:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.326+0000 7f9b0efad740 10 client.0 _ll_register_callbacks cb 0x7f9b00009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:30:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.326+0000 7f9b0a03a640 1 -- 172.21.15.7:0/1211357488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580597c7e10 msgr2=0x5580597d5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.326+0000 7f9b0a03a640 1 --2- 172.21.15.7:0/1211357488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580597c7e10 0x5580597d5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.326+0000 7f9b0a03a640 1 -- 172.21.15.7:0/1211357488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580597ccf80 msgr2=0x5580597d1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:30:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.326+0000 7f9b0a03a640 1 --2- 172.21.15.7:0/1211357488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580597ccf80 0x5580597d1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.326+0000 7f9b0a03a640 1 -- 172.21.15.7:0/1211357488 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580597c6fa0 con 0x5580597c43a0 2024-06-10T01:30:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.326+0000 7f9b09839640 1 --2- 172.21.15.7:0/1211357488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580597c7e10 0x5580597d5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:30:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.326+0000 7f9b0a03a640 1 --2- 172.21.15.7:0/1211357488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580597c43a0 0x5580597d51f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9b0002a930 tx=0x7f9b000049b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34b9c228cc0dd43c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:30:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.326+0000 7f9aedffb640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:30:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.327+0000 7f9aedffb640 1 -- 172.21.15.7:0/1211357488 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00035070 con 0x5580597c43a0 2024-06-10T01:30:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.327+0000 7f9b0efad740 1 -- 172.21.15.7:0/1211357488 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b00041d70 con 0x5580597c43a0 2024-06-10T01:30:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.327+0000 7f9aedffb640 1 -- 172.21.15.7:0/1211357488 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f9b000041c0 con 0x5580597c43a0 2024-06-10T01:30:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.327+0000 7f9aedffb640 1 -- 172.21.15.7:0/1211357488 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f9b0005e040 con 0x5580597c43a0 2024-06-10T01:30:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.327+0000 7f9b0efad740 10 client.30352 fetch_fsmap learned FSMap version 57 2024-06-10T01:30:57.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.327+0000 7f9b0efad740 1 -- 172.21.15.7:0/1211357488 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x558059842d90 con 0x5580597c43a0 2024-06-10T01:30:57.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.327+0000 7f9aedffb640 1 -- 172.21.15.7:0/1211357488 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b00041d70 con 0x5580597c43a0 2024-06-10T01:30:57.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.327+0000 7f9aedffb640 1 -- 172.21.15.7:0/1211357488 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7f9b00004f30 con 0x5580597c43a0 2024-06-10T01:30:57.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.327+0000 7f9b0efad740 10 client.30352 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:30:57.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.327+0000 7f9b0efad740 10 client.30352 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:30:57.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.327+0000 7f9b0efad740 20 client.30352 populate_metadata read hostname 'smithi007' 2024-06-10T01:30:57.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.327+0000 7f9b0efad740 1 --2- 172.21.15.7:0/1211357488 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558059843790 0x558059845bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:30:57.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.327+0000 7f9b0efad740 4 client.30352 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:30:57.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.327+0000 7f9b0efad740 1 -- 172.21.15.7:0/1211357488 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558059733c20 con 0x558059843790 2024-06-10T01:30:57.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.328+0000 7f9b09839640 1 --2- 172.21.15.7:0/1211357488 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558059843790 0x558059845bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:30:57.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.328+0000 7f9b09839640 1 --2- 172.21.15.7:0/1211357488 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558059843790 0x558059845bb0 crc :-1 s=READY pgs=554 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=726e10666636eec0 server_cookie=9c9ccf56036f5d0c in_seq=0 out_seq=0 2024-06-10T01:30:57.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:57.328+0000 7f9aedffb640 10 client.30352 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:57.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:57 smithi175 ceph-mon[36636]: pgmap v1182: 129 pgs: 129 active+clean; 662 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 599 op/s 2024-06-10T01:30:57.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:57 smithi007 ceph-mon[32398]: pgmap v1182: 129 pgs: 129 active+clean; 662 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 599 op/s 2024-06-10T01:30:57.569 INFO:tasks.workunit.client.0.smithi007.stdout: 10 273273 21.85 MB/sec execute 574 sec latency 130.972 ms 2024-06-10T01:30:57.578 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:57 smithi078 ceph-mon[36683]: pgmap v1182: 129 pgs: 129 active+clean; 662 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 599 op/s 2024-06-10T01:30:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:58.163+0000 7f9aedffb640 1 -- 172.21.15.7:0/1211357488 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558059733c20 con 0x558059843790 2024-06-10T01:30:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:58.163+0000 7f9aedffb640 10 client.30352 handle_command_reply: tid=0 2024-06-10T01:30:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:58.163+0000 7f9b0efad740 1 client.30352 shutdown 2024-06-10T01:30:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:58.163+0000 7f9b0efad740 1 -- 172.21.15.7:0/1211357488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580597c43a0 msgr2=0x5580597d51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:58.163+0000 7f9b0efad740 1 --2- 172.21.15.7:0/1211357488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580597c43a0 0x5580597d51f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9b0002a930 tx=0x7f9b000049b0 comp rx=0 tx=0).stop 2024-06-10T01:30:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:58.163+0000 7f9b0efad740 1 -- 172.21.15.7:0/1211357488 shutdown_connections 2024-06-10T01:30:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:58.163+0000 7f9b0efad740 1 --2- 172.21.15.7:0/1211357488 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x558059843790 0x558059845bb0 crc :-1 s=READY pgs=554 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:58.163+0000 7f9b0efad740 1 --2- 172.21.15.7:0/1211357488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580597ccf80 0x5580597d1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:58.163+0000 7f9aedffb640 0 client.30352 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:30:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:58.163+0000 7f9b0efad740 1 --2- 172.21.15.7:0/1211357488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580597c7e10 0x5580597d5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:58.163+0000 7f9b0efad740 1 --2- 172.21.15.7:0/1211357488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580597c43a0 0x5580597d51f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:58.163+0000 7f9b0efad740 1 -- 172.21.15.7:0/1211357488 >> 172.21.15.7:0/1211357488 conn(0x5580597be680 msgr2=0x5580597c8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:58.163+0000 7f9b0efad740 1 -- 172.21.15.7:0/1211357488 shutdown_connections 2024-06-10T01:30:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:58.163+0000 7f9b0efad740 1 -- 172.21.15.7:0/1211357488 wait complete. 2024-06-10T01:30:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:58.163+0000 7f9b0efad740 20 client.30352 trim_cache size 0 max 16384 2024-06-10T01:30:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:30:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:30:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:30:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:58.164+0000 7f9b0c29e640 1 -- 172.21.15.7:0/3132453257 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9af00a2c00 msgr2=0x7f9af00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:58.164+0000 7f9b0c29e640 1 --2- 172.21.15.7:0/3132453257 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9af00a2c00 0x7f9af00a50c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f9afc005fd0 tx=0x7f9afc011bc0 comp rx=0 tx=0).stop 2024-06-10T01:30:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:58.164+0000 7f9b0c29e640 1 -- 172.21.15.7:0/3132453257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b04101b30 msgr2=0x7f9b04198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:30:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:58.164+0000 7f9b0c29e640 1 --2- 172.21.15.7:0/3132453257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b04101b30 0x7f9b04198290 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f9af400e9f0 tx=0x7f9af400edf0 comp rx=0 tx=0).stop 2024-06-10T01:30:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:58.164+0000 7f9b0c29e640 1 -- 172.21.15.7:0/3132453257 shutdown_connections 2024-06-10T01:30:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:58.164+0000 7f9b0c29e640 1 --2- 172.21.15.7:0/3132453257 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f9af00a2c00 0x7f9af00a50c0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:58.164+0000 7f9b0c29e640 1 --2- 172.21.15.7:0/3132453257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b041024f0 0x7f9b0419f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:58.164+0000 7f9b0c29e640 1 --2- 172.21.15.7:0/3132453257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b04101b30 0x7f9b04198290 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:58.164+0000 7f9b0c29e640 1 --2- 172.21.15.7:0/3132453257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b04100940 0x7f9b04197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:30:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:58.164+0000 7f9b0c29e640 1 -- 172.21.15.7:0/3132453257 >> 172.21.15.7:0/3132453257 conn(0x7f9b040fbed0 msgr2=0x7f9b040fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:30:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:58.164+0000 7f9b0c29e640 1 -- 172.21.15.7:0/3132453257 shutdown_connections 2024-06-10T01:30:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:30:58.164+0000 7f9b0c29e640 1 -- 172.21.15.7:0/3132453257 wait complete. 2024-06-10T01:30:58.569 INFO:tasks.workunit.client.0.smithi007.stdout: 10 273672 21.85 MB/sec execute 575 sec latency 564.571 ms 2024-06-10T01:30:58.824 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:30:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:30:59.433 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:30:59 smithi007 ceph-mon[32398]: pgmap v1183: 129 pgs: 129 active+clean; 669 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 3.4 MiB/s wr, 461 op/s 2024-06-10T01:30:59.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:30:59 smithi175 ceph-mon[36636]: pgmap v1183: 129 pgs: 129 active+clean; 669 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 3.4 MiB/s wr, 461 op/s 2024-06-10T01:30:59.569 INFO:tasks.workunit.client.0.smithi007.stdout: 10 274172 21.85 MB/sec execute 576 sec latency 97.348 ms 2024-06-10T01:30:59.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:30:59 smithi078 ceph-mon[36683]: pgmap v1183: 129 pgs: 129 active+clean; 669 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 3.4 MiB/s wr, 461 op/s 2024-06-10T01:31:00.569 INFO:tasks.workunit.client.0.smithi007.stdout: 10 274596 21.85 MB/sec execute 577 sec latency 183.252 ms 2024-06-10T01:31:00.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:00.996+0000 7f1ecd560700 1 -- 172.21.15.7:0/2206454151 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ec8101a60 msgr2=0x7f1ec8105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:00.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:00.996+0000 7f1ecd560700 1 --2- 172.21.15.7:0/2206454151 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ec8101a60 0x7f1ec8105820 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0098f0 tx=0x7f1ebc009c00 comp rx=0 tx=0).stop 2024-06-10T01:31:00.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:00.996+0000 7f1ecd560700 1 -- 172.21.15.7:0/2206454151 shutdown_connections 2024-06-10T01:31:00.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:00.996+0000 7f1ecd560700 1 --2- 172.21.15.7:0/2206454151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ec8105f50 0x7f1ec810da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:00.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:00.996+0000 7f1ecd560700 1 --2- 172.21.15.7:0/2206454151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ec81010b0 0x7f1ec8101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:00.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:00.996+0000 7f1ecd560700 1 --2- 172.21.15.7:0/2206454151 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ec8101a60 0x7f1ec8105820 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:00.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:00.996+0000 7f1ecd560700 1 -- 172.21.15.7:0/2206454151 >> 172.21.15.7:0/2206454151 conn(0x7f1ec80766b0 msgr2=0x7f1ec8076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:00.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:00.996+0000 7f1ecd560700 1 -- 172.21.15.7:0/2206454151 shutdown_connections 2024-06-10T01:31:00.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:00.996+0000 7f1ecd560700 1 -- 172.21.15.7:0/2206454151 wait complete. 2024-06-10T01:31:00.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:00.998+0000 7f1ecd560700 1 Processor -- start 2024-06-10T01:31:00.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:00.998+0000 7f1ecd560700 1 -- start start 2024-06-10T01:31:00.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:00.998+0000 7f1ecd560700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ec81010b0 0x7f1ec819e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:00.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:00.999+0000 7f1ecd560700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ec8101a60 0x7f1ec819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:00.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:00.999+0000 7f1ecd560700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ec8105f50 0x7f1ec8198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:00.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:00.999+0000 7f1ecd560700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec8110660 con 0x7f1ec8101a60 2024-06-10T01:31:00.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:00.999+0000 7f1ecd560700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec81104e0 con 0x7f1ec8105f50 2024-06-10T01:31:00.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:00.999+0000 7f1ec77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ec8105f50 0x7f1ec8198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:00.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:00.999+0000 7f1ec77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ec8105f50 0x7f1ec8198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:52342/0 (socket says 172.21.15.7:52342) 2024-06-10T01:31:00.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:00.999+0000 7f1ec77fe700 1 -- 172.21.15.7:0/2450904971 learned_addr learned my addr 172.21.15.7:0/2450904971 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:00.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:00.999+0000 7f1ecd560700 1 -- 172.21.15.7:0/2450904971 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec81107e0 con 0x7f1ec81010b0 2024-06-10T01:31:00.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.000+0000 7f1ec67fc700 1 --2- 172.21.15.7:0/2450904971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ec8101a60 0x7f1ec819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:00.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.001+0000 7f1ec67fc700 1 -- 172.21.15.7:0/2450904971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ec81010b0 msgr2=0x7f1ec819e8c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-10T01:31:00.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.001+0000 7f1ec67fc700 1 --2- 172.21.15.7:0/2450904971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ec81010b0 0x7f1ec819e8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:00.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.001+0000 7f1ec67fc700 1 -- 172.21.15.7:0/2450904971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ec8105f50 msgr2=0x7f1ec8198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:00.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.001+0000 7f1ec67fc700 1 --2- 172.21.15.7:0/2450904971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ec8105f50 0x7f1ec8198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:00.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.001+0000 7f1ec67fc700 1 -- 172.21.15.7:0/2450904971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec81991e0 con 0x7f1ec8101a60 2024-06-10T01:31:00.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.002+0000 7f1ec67fc700 1 --2- 172.21.15.7:0/2450904971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ec8101a60 0x7f1ec819ee00 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009230 tx=0x7f1ebc00fb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:00.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.003+0000 7f1eaffff700 1 -- 172.21.15.7:0/2450904971 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc02a070 con 0x7f1ec8101a60 2024-06-10T01:31:00.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.003+0000 7f1eaffff700 1 -- 172.21.15.7:0/2450904971 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ebc01b070 con 0x7f1ec8101a60 2024-06-10T01:31:00.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.003+0000 7f1eaffff700 1 -- 172.21.15.7:0/2450904971 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc003680 con 0x7f1ec8101a60 2024-06-10T01:31:00.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.004+0000 7f1ecd560700 1 -- 172.21.15.7:0/2450904971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec8199470 con 0x7f1ec8101a60 2024-06-10T01:31:00.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.004+0000 7f1ecd560700 1 -- 172.21.15.7:0/2450904971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec8199810 con 0x7f1ec8101a60 2024-06-10T01:31:00.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.007+0000 7f1eaffff700 1 -- 172.21.15.7:0/2450904971 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f1ebc0268d0 con 0x7f1ec8101a60 2024-06-10T01:31:01.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.008+0000 7f1eaffff700 1 --2- 172.21.15.7:0/2450904971 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1eb00a2d70 0x7f1eb00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:01.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.008+0000 7f1ec6ffd700 1 --2- 172.21.15.7:0/2450904971 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1eb00a2d70 0x7f1eb00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:01.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.008+0000 7f1eaffff700 1 -- 172.21.15.7:0/2450904971 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f1ebc0d7970 con 0x7f1ec8101a60 2024-06-10T01:31:01.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.009+0000 7f1ed7de3b80 1 Processor -- start 2024-06-10T01:31:01.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.009+0000 7f1ec6ffd700 1 --2- 172.21.15.7:0/2450904971 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1eb00a2d70 0x7f1eb00a5230 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8199bc0 tx=0x7f1eb800a420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:01.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.009+0000 7f1ed7de3b80 1 -- start start 2024-06-10T01:31:01.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.009+0000 7f1ed7de3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce252d7f60 0x55ce252d8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:01.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.009+0000 7f1ed7de3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce252d8880 0x55ce252d2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:01.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.009+0000 7f1ed7de3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce252d27a0 0x55ce252d2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:01.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.009+0000 7f1ed7de3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ce251c4f60 con 0x55ce252d27a0 2024-06-10T01:31:01.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.009+0000 7f1ed7de3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ce25223850 con 0x55ce252d7f60 2024-06-10T01:31:01.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.009+0000 7f1ed7de3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ce2518b460 con 0x55ce252d8880 2024-06-10T01:31:01.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.009+0000 7f1ec67fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce252d8880 0x55ce252d2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:01.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.009+0000 7f1ec6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce252d7f60 0x55ce252d8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:01.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.009+0000 7f1ec67fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce252d8880 0x55ce252d2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:39146/0 (socket says 172.21.15.7:39146) 2024-06-10T01:31:01.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.009+0000 7f1ec67fc700 1 -- 172.21.15.7:0/429503909 learned_addr learned my addr 172.21.15.7:0/429503909 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:01.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.010+0000 7f1ec77fe700 1 --2- 172.21.15.7:0/429503909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce252d27a0 0x55ce252d2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:01.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.010+0000 7f1ec67fc700 1 -- 172.21.15.7:0/429503909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce252d7f60 msgr2=0x55ce252d8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:01.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.010+0000 7f1ec67fc700 1 --2- 172.21.15.7:0/429503909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce252d7f60 0x55ce252d8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:01.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.010+0000 7f1ec67fc700 1 -- 172.21.15.7:0/429503909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce252d27a0 msgr2=0x55ce252d2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:01.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.010+0000 7f1ec67fc700 1 --2- 172.21.15.7:0/429503909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce252d27a0 0x55ce252d2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:01.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.010+0000 7f1ec67fc700 1 -- 172.21.15.7:0/429503909 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce251728d0 con 0x55ce252d8880 2024-06-10T01:31:01.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.010+0000 7f1ec77fe700 1 --2- 172.21.15.7:0/429503909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce252d27a0 0x55ce252d2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:31:01.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.010+0000 7f1ec67fc700 1 --2- 172.21.15.7:0/429503909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce252d8880 0x55ce252d2260 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00bdb0 tx=0x7f1ebc000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52908ff35774922d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:01.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.010+0000 7f1ead7fa700 1 -- 172.21.15.7:0/429503909 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc0a8290 con 0x55ce252d8880 2024-06-10T01:31:01.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.010+0000 7f1ead7fa700 1 -- 172.21.15.7:0/429503909 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ebc0acac0 con 0x55ce252d8880 2024-06-10T01:31:01.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.010+0000 7f1ead7fa700 1 -- 172.21.15.7:0/429503909 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc0a4140 con 0x55ce252d8880 2024-06-10T01:31:01.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.011+0000 7f1ed7de3b80 1 -- 172.21.15.7:0/429503909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce252d8880 msgr2=0x55ce252d2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:01.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.011+0000 7f1ed7de3b80 1 --2- 172.21.15.7:0/429503909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce252d8880 0x55ce252d2260 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00bdb0 tx=0x7f1ebc000bc0 comp rx=0 tx=0).stop 2024-06-10T01:31:01.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.011+0000 7f1ed7de3b80 1 -- 172.21.15.7:0/429503909 shutdown_connections 2024-06-10T01:31:01.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.011+0000 7f1ed7de3b80 1 --2- 172.21.15.7:0/429503909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce252d27a0 0x55ce252d2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:01.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.011+0000 7f1ed7de3b80 1 --2- 172.21.15.7:0/429503909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce252d7f60 0x55ce252d8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:01.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.011+0000 7f1ed7de3b80 1 --2- 172.21.15.7:0/429503909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce252d8880 0x55ce252d2260 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:01.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.011+0000 7f1ed7de3b80 1 -- 172.21.15.7:0/429503909 >> 172.21.15.7:0/429503909 conn(0x55ce251ff080 msgr2=0x55ce252cd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:01.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.011+0000 7f1ed7de3b80 1 -- 172.21.15.7:0/429503909 shutdown_connections 2024-06-10T01:31:01.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.011+0000 7f1ed7de3b80 1 -- 172.21.15.7:0/429503909 wait complete. 2024-06-10T01:31:01.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.012+0000 7f1ed7de3b80 1 -- start start 2024-06-10T01:31:01.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.012+0000 7f1ed7de3b80 1 Processor -- start 2024-06-10T01:31:01.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.012+0000 7f1ed7de3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce252d27a0 0x55ce252de620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:01.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.012+0000 7f1ed7de3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce252d7f60 0x55ce252db920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:01.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.012+0000 7f1ed7de3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce252d8880 0x55ce252dbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:01.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.012+0000 7f1ed7de3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ce251c3fc0 con 0x55ce252d8880 2024-06-10T01:31:01.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.012+0000 7f1ed7de3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ce2521f810 con 0x55ce252d7f60 2024-06-10T01:31:01.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.012+0000 7f1ed7de3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ce252df010 con 0x55ce252d27a0 2024-06-10T01:31:01.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.012+0000 7f1ed7de3b80 10 client.0 _ll_register_callbacks cb 0x55ce251b95b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:31:01.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.012+0000 7f1ec6ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce252d27a0 0x55ce252de620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:01.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.013+0000 7f1ec67fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce252d7f60 0x55ce252db920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:01.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.013+0000 7f1ec6ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce252d27a0 0x55ce252de620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:39154/0 (socket says 172.21.15.7:39154) 2024-06-10T01:31:01.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.013+0000 7f1ec6ffd700 1 -- 172.21.15.7:0/4220872082 learned_addr learned my addr 172.21.15.7:0/4220872082 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:01.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.013+0000 7f1ec77fe700 1 --2- 172.21.15.7:0/4220872082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce252d8880 0x55ce252dbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:01.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.013+0000 7f1ec67fc700 1 -- 172.21.15.7:0/4220872082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce252d27a0 msgr2=0x55ce252de620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:01.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.013+0000 7f1ec67fc700 1 --2- 172.21.15.7:0/4220872082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce252d27a0 0x55ce252de620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:01.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.013+0000 7f1ec67fc700 1 -- 172.21.15.7:0/4220872082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce252d8880 msgr2=0x55ce252dbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:01.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.013+0000 7f1ec67fc700 1 --2- 172.21.15.7:0/4220872082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce252d8880 0x55ce252dbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:01.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.013+0000 7f1ec67fc700 1 -- 172.21.15.7:0/4220872082 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce251e9750 con 0x55ce252d7f60 2024-06-10T01:31:01.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.013+0000 7f1ec67fc700 1 --2- 172.21.15.7:0/4220872082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce252d7f60 0x55ce252db920 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0d5140 tx=0x7f1ebc005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f826d4da9a292756 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:01.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.013+0000 7f1eaaff5700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:31:01.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.013+0000 7f1eaaff5700 1 -- 172.21.15.7:0/4220872082 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc0ac830 con 0x55ce252d7f60 2024-06-10T01:31:01.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.013+0000 7f1eaaff5700 1 -- 172.21.15.7:0/4220872082 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f1ebc0a3330 con 0x55ce252d7f60 2024-06-10T01:31:01.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.013+0000 7f1ed7de3b80 1 -- 172.21.15.7:0/4220872082 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ebc0acac0 con 0x55ce252d7f60 2024-06-10T01:31:01.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.014+0000 7f1eaaff5700 1 -- 172.21.15.7:0/4220872082 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f1ebc0a8290 con 0x55ce252d7f60 2024-06-10T01:31:01.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.014+0000 7f1ed7de3b80 10 client.30476 fetch_fsmap learned FSMap version 57 2024-06-10T01:31:01.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.014+0000 7f1ed7de3b80 1 -- 172.21.15.7:0/4220872082 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55ce252e9710 con 0x55ce252d7f60 2024-06-10T01:31:01.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.014+0000 7f1eaaff5700 1 -- 172.21.15.7:0/4220872082 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ebc0c0030 con 0x55ce252d7f60 2024-06-10T01:31:01.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.014+0000 7f1eaaff5700 1 -- 172.21.15.7:0/4220872082 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7f1ebc0c2a90 con 0x55ce252d7f60 2024-06-10T01:31:01.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.014+0000 7f1ed7de3b80 10 client.30476 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:31:01.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.014+0000 7f1ed7de3b80 10 client.30476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:31:01.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.014+0000 7f1ed7de3b80 20 client.30476 populate_metadata read hostname 'smithi007' 2024-06-10T01:31:01.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.014+0000 7f1ed7de3b80 1 --2- 172.21.15.7:0/4220872082 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ce252ebbe0 0x55ce252edfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:01.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.014+0000 7f1ed7de3b80 4 client.30476 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:31:01.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.015+0000 7f1ed7de3b80 1 -- 172.21.15.7:0/4220872082 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce25230d50 con 0x55ce252ebbe0 2024-06-10T01:31:01.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.017+0000 7f1ec6ffd700 1 --2- 172.21.15.7:0/4220872082 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ce252ebbe0 0x55ce252edfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:01.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.018+0000 7f1ec6ffd700 1 --2- 172.21.15.7:0/4220872082 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ce252ebbe0 0x55ce252edfd0 crc :-1 s=READY pgs=555 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9a9f519e3909746 server_cookie=cc703f1c08c0f449 in_seq=0 out_seq=0 2024-06-10T01:31:01.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.018+0000 7f1eaaff5700 10 client.30476 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:01.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.025+0000 7f1eaaff5700 1 -- 172.21.15.7:0/4220872082 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ce25230d50 con 0x55ce252ebbe0 2024-06-10T01:31:01.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.025+0000 7f1eaaff5700 10 client.30476 handle_command_reply: tid=0 2024-06-10T01:31:01.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.025+0000 7f1ed7de3b80 1 client.30476 shutdown 2024-06-10T01:31:01.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.025+0000 7f1ed7de3b80 1 -- 172.21.15.7:0/4220872082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce252d7f60 msgr2=0x55ce252db920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:01.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.025+0000 7f1ed7de3b80 1 --2- 172.21.15.7:0/4220872082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce252d7f60 0x55ce252db920 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0d5140 tx=0x7f1ebc005e50 comp rx=0 tx=0).stop 2024-06-10T01:31:01.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.025+0000 7f1ed7de3b80 1 -- 172.21.15.7:0/4220872082 shutdown_connections 2024-06-10T01:31:01.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.025+0000 7f1ed7de3b80 1 --2- 172.21.15.7:0/4220872082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce252d8880 0x55ce252dbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:01.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.025+0000 7f1ed7de3b80 1 --2- 172.21.15.7:0/4220872082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce252d27a0 0x55ce252de620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:01.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.025+0000 7f1ed7de3b80 1 --2- 172.21.15.7:0/4220872082 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ce252ebbe0 0x55ce252edfd0 crc :-1 s=READY pgs=555 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:01.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.025+0000 7f1ed7de3b80 1 --2- 172.21.15.7:0/4220872082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce252d7f60 0x55ce252db920 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:01.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.025+0000 7f1eaaff5700 0 client.30476 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:01.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.025+0000 7f1ed7de3b80 1 -- 172.21.15.7:0/4220872082 >> 172.21.15.7:0/4220872082 conn(0x55ce251ff080 msgr2=0x55ce252cc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:01.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.026+0000 7f1ed7de3b80 1 -- 172.21.15.7:0/4220872082 shutdown_connections 2024-06-10T01:31:01.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.026+0000 7f1ed7de3b80 1 -- 172.21.15.7:0/4220872082 wait complete. 2024-06-10T01:31:01.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.026+0000 7f1ed7de3b80 20 client.30476 trim_cache size 0 max 16384 2024-06-10T01:31:01.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.053+0000 7f1ed7de3b80 1 Processor -- start 2024-06-10T01:31:01.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.053+0000 7f1ed7de3b80 1 -- start start 2024-06-10T01:31:01.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.053+0000 7f1ed7de3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce252d27a0 0x55ce252cce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:01.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.053+0000 7f1ed7de3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce252d7f60 0x55ce252cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:01.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.053+0000 7f1ed7de3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce252d8880 0x55ce252d8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:01.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.053+0000 7f1ed7de3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec81ae880 con 0x55ce252d7f60 2024-06-10T01:31:01.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.053+0000 7f1ed7de3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec81ac0b0 con 0x55ce252d27a0 2024-06-10T01:31:01.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.053+0000 7f1ed7de3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec81a9f00 con 0x55ce252d8880 2024-06-10T01:31:01.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.053+0000 7f1ec6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce252d27a0 0x55ce252cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:01.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.053+0000 7f1ec6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce252d27a0 0x55ce252cce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:52352/0 (socket says 172.21.15.7:52352) 2024-06-10T01:31:01.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.053+0000 7f1ec6ffd700 1 -- 172.21.15.7:0/803041969 learned_addr learned my addr 172.21.15.7:0/803041969 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:01.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.053+0000 7f1ec77fe700 1 --2- 172.21.15.7:0/803041969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce252d8880 0x55ce252d8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:01.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.053+0000 7f1ec67fc700 1 --2- 172.21.15.7:0/803041969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce252d7f60 0x55ce252cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:01.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.054+0000 7f1ec6ffd700 1 -- 172.21.15.7:0/803041969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce252d8880 msgr2=0x55ce252d8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:01.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.054+0000 7f1ec6ffd700 1 --2- 172.21.15.7:0/803041969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce252d8880 0x55ce252d8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:01.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.054+0000 7f1ec6ffd700 1 -- 172.21.15.7:0/803041969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce252d7f60 msgr2=0x55ce252cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:01.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.054+0000 7f1ec6ffd700 1 --2- 172.21.15.7:0/803041969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce252d7f60 0x55ce252cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:01.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.054+0000 7f1ec6ffd700 1 -- 172.21.15.7:0/803041969 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce252d2260 con 0x55ce252d27a0 2024-06-10T01:31:01.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.054+0000 7f1ec6ffd700 1 --2- 172.21.15.7:0/803041969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce252d27a0 0x55ce252cce80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1eb800ecf0 tx=0x7f1eb8011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99e6bbd38ed1ef0b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:01.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.054+0000 7f1eacff9700 1 -- 172.21.15.7:0/803041969 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb801a190 con 0x55ce252d27a0 2024-06-10T01:31:01.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.054+0000 7f1eacff9700 1 -- 172.21.15.7:0/803041969 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eb8004830 con 0x55ce252d27a0 2024-06-10T01:31:01.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.054+0000 7f1eacff9700 1 -- 172.21.15.7:0/803041969 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8005440 con 0x55ce252d27a0 2024-06-10T01:31:01.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.055+0000 7f1ed7de3b80 1 -- 172.21.15.7:0/803041969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce252d27a0 msgr2=0x55ce252cce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:01.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.055+0000 7f1ed7de3b80 1 --2- 172.21.15.7:0/803041969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce252d27a0 0x55ce252cce80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1eb800ecf0 tx=0x7f1eb8011000 comp rx=0 tx=0).stop 2024-06-10T01:31:01.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.055+0000 7f1ed7de3b80 1 -- 172.21.15.7:0/803041969 shutdown_connections 2024-06-10T01:31:01.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.055+0000 7f1ed7de3b80 1 --2- 172.21.15.7:0/803041969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce252d8880 0x55ce252d8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:01.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.055+0000 7f1ed7de3b80 1 --2- 172.21.15.7:0/803041969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce252d27a0 0x55ce252cce80 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:01.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.055+0000 7f1ed7de3b80 1 --2- 172.21.15.7:0/803041969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce252d7f60 0x55ce252cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:01.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.055+0000 7f1ed7de3b80 1 -- 172.21.15.7:0/803041969 >> 172.21.15.7:0/803041969 conn(0x55ce251ff080 msgr2=0x55ce252d59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:01.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.055+0000 7f1ed7de3b80 1 -- 172.21.15.7:0/803041969 shutdown_connections 2024-06-10T01:31:01.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.056+0000 7f1ed7de3b80 1 -- 172.21.15.7:0/803041969 wait complete. 2024-06-10T01:31:01.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.056+0000 7f1ed7de3b80 1 -- start start 2024-06-10T01:31:01.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.056+0000 7f1ed7de3b80 1 Processor -- start 2024-06-10T01:31:01.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.057+0000 7f1ed7de3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce252d27a0 0x55ce252e4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:01.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.057+0000 7f1ed7de3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce252d7f60 0x55ce252e4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:01.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.057+0000 7f1ec67fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce252d7f60 0x55ce252e4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:01.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.057+0000 7f1ec67fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce252d7f60 0x55ce252e4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33490/0 (socket says 172.21.15.7:33490) 2024-06-10T01:31:01.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.057+0000 7f1ec6ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce252d27a0 0x55ce252e4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:01.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.057+0000 7f1ec6ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce252d27a0 0x55ce252e4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:39170/0 (socket says 172.21.15.7:39170) 2024-06-10T01:31:01.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.057+0000 7f1ed7de3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce252d8880 0x55ce252f2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:01.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.057+0000 7f1ec67fc700 1 -- 172.21.15.7:0/2504665303 learned_addr learned my addr 172.21.15.7:0/2504665303 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:01.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.057+0000 7f1ed7de3b80 1 -- 172.21.15.7:0/2504665303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec81aa570 con 0x55ce252d7f60 2024-06-10T01:31:01.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.057+0000 7f1ed7de3b80 1 -- 172.21.15.7:0/2504665303 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec81ab9e0 con 0x55ce252d8880 2024-06-10T01:31:01.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.057+0000 7f1ec77fe700 1 --2- 172.21.15.7:0/2504665303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce252d8880 0x55ce252f2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:01.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.057+0000 7f1ed7de3b80 1 -- 172.21.15.7:0/2504665303 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec81aac40 con 0x55ce252d27a0 2024-06-10T01:31:01.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.058+0000 7f1ed7de3b80 10 client.0 _ll_register_callbacks cb 0x55ce251b95b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:31:01.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.058+0000 7f1ec67fc700 1 -- 172.21.15.7:0/2504665303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce252d27a0 msgr2=0x55ce252e4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:01.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.058+0000 7f1ec67fc700 1 --2- 172.21.15.7:0/2504665303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce252d27a0 0x55ce252e4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:01.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.058+0000 7f1ec67fc700 1 -- 172.21.15.7:0/2504665303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce252d8880 msgr2=0x55ce252f2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:01.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.058+0000 7f1ec67fc700 1 --2- 172.21.15.7:0/2504665303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce252d8880 0x55ce252f2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:01.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.058+0000 7f1ec67fc700 1 -- 172.21.15.7:0/2504665303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce252f37f0 con 0x55ce252d7f60 2024-06-10T01:31:01.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.058+0000 7f1ec77fe700 1 --2- 172.21.15.7:0/2504665303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce252d8880 0x55ce252f2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:31:01.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.058+0000 7f1ec67fc700 1 --2- 172.21.15.7:0/2504665303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce252d7f60 0x55ce252e4b80 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0a3be0 tx=0x7f1ebc00b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c8c2f1f3cc9316 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:01.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.058+0000 7f1eaaff5700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:31:01.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.058+0000 7f1eaaff5700 1 -- 172.21.15.7:0/2504665303 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc0a3330 con 0x55ce252d7f60 2024-06-10T01:31:01.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.058+0000 7f1eaaff5700 1 -- 172.21.15.7:0/2504665303 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f1ebc0a34d0 con 0x55ce252d7f60 2024-06-10T01:31:01.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.058+0000 7f1ed7de3b80 1 -- 172.21.15.7:0/2504665303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1eb8004830 con 0x55ce252d7f60 2024-06-10T01:31:01.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.059+0000 7f1eaaff5700 1 -- 172.21.15.7:0/2504665303 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f1ebc09fc30 con 0x55ce252d7f60 2024-06-10T01:31:01.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.059+0000 7f1ed7de3b80 10 client.21315 fetch_fsmap learned FSMap version 57 2024-06-10T01:31:01.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.059+0000 7f1ed7de3b80 1 -- 172.21.15.7:0/2504665303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55ce252f37f0 con 0x55ce252d7f60 2024-06-10T01:31:01.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.059+0000 7f1eaaff5700 1 -- 172.21.15.7:0/2504665303 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eb8004830 con 0x55ce252d7f60 2024-06-10T01:31:01.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.059+0000 7f1eaaff5700 1 -- 172.21.15.7:0/2504665303 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7f1ebc09ea50 con 0x55ce252d7f60 2024-06-10T01:31:01.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.059+0000 7f1ed7de3b80 10 client.21315 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:31:01.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.059+0000 7f1ed7de3b80 10 client.21315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:31:01.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.059+0000 7f1ed7de3b80 20 client.21315 populate_metadata read hostname 'smithi007' 2024-06-10T01:31:01.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.059+0000 7f1ed7de3b80 1 --2- 172.21.15.7:0/2504665303 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ce252ebbe0 0x55ce253586e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:01.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.059+0000 7f1ed7de3b80 4 client.21315 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:31:01.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.059+0000 7f1ed7de3b80 1 -- 172.21.15.7:0/2504665303 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ce2522b2d0 con 0x55ce252ebbe0 2024-06-10T01:31:01.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.060+0000 7f1ec6ffd700 1 --2- 172.21.15.7:0/2504665303 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ce252ebbe0 0x55ce253586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:01.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.060+0000 7f1eaaff5700 10 client.21315 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:01.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.061+0000 7f1ec6ffd700 1 --2- 172.21.15.7:0/2504665303 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ce252ebbe0 0x55ce253586e0 crc :-1 s=READY pgs=556 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7956575341a6bd58 server_cookie=50a03adbcc70f0e5 in_seq=0 out_seq=0 2024-06-10T01:31:01.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.063+0000 7f1eaaff5700 1 -- 172.21.15.7:0/2504665303 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ce2522b2d0 con 0x55ce252ebbe0 2024-06-10T01:31:01.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.063+0000 7f1eaaff5700 10 client.21315 handle_command_reply: tid=0 2024-06-10T01:31:01.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.063+0000 7f1ed7de3b80 1 client.21315 shutdown 2024-06-10T01:31:01.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.064+0000 7f1ed7de3b80 1 -- 172.21.15.7:0/2504665303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce252d7f60 msgr2=0x55ce252e4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:01.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.064+0000 7f1ed7de3b80 1 --2- 172.21.15.7:0/2504665303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce252d7f60 0x55ce252e4b80 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0a3be0 tx=0x7f1ebc00b820 comp rx=0 tx=0).stop 2024-06-10T01:31:01.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.064+0000 7f1ed7de3b80 1 -- 172.21.15.7:0/2504665303 shutdown_connections 2024-06-10T01:31:01.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.064+0000 7f1ed7de3b80 1 --2- 172.21.15.7:0/2504665303 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55ce252ebbe0 0x55ce253586e0 crc :-1 s=READY pgs=556 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:01.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.064+0000 7f1ed7de3b80 1 --2- 172.21.15.7:0/2504665303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce252d8880 0x55ce252f2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:01.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.064+0000 7f1ed7de3b80 1 --2- 172.21.15.7:0/2504665303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce252d27a0 0x55ce252e4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:01.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.064+0000 7f1ed7de3b80 1 --2- 172.21.15.7:0/2504665303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce252d7f60 0x55ce252e4b80 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:01.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.064+0000 7f1ed7de3b80 1 -- 172.21.15.7:0/2504665303 >> 172.21.15.7:0/2504665303 conn(0x55ce251ff080 msgr2=0x55ce252d8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:01.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.064+0000 7f1eaaff5700 0 client.21315 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:01.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.064+0000 7f1ed7de3b80 1 -- 172.21.15.7:0/2504665303 shutdown_connections 2024-06-10T01:31:01.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.064+0000 7f1ed7de3b80 1 -- 172.21.15.7:0/2504665303 wait complete. 2024-06-10T01:31:01.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.064+0000 7f1ed7de3b80 20 client.21315 trim_cache size 0 max 16384 2024-06-10T01:31:01.058 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:31:01.058 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-10T01:31:01.058 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-10T01:31:01.058 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:31:01.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.065+0000 7f1ecd560700 1 -- 172.21.15.7:0/2450904971 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1eb00a2d70 msgr2=0x7f1eb00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:01.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.065+0000 7f1ecd560700 1 --2- 172.21.15.7:0/2450904971 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1eb00a2d70 0x7f1eb00a5230 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8199bc0 tx=0x7f1eb800a420 comp rx=0 tx=0).stop 2024-06-10T01:31:01.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.065+0000 7f1ecd560700 1 -- 172.21.15.7:0/2450904971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ec8101a60 msgr2=0x7f1ec819ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:01.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.065+0000 7f1ecd560700 1 --2- 172.21.15.7:0/2450904971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ec8101a60 0x7f1ec819ee00 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009230 tx=0x7f1ebc00fb40 comp rx=0 tx=0).stop 2024-06-10T01:31:01.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.065+0000 7f1ecd560700 1 -- 172.21.15.7:0/2450904971 shutdown_connections 2024-06-10T01:31:01.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.065+0000 7f1ecd560700 1 --2- 172.21.15.7:0/2450904971 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1eb00a2d70 0x7f1eb00a5230 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:01.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.065+0000 7f1ecd560700 1 --2- 172.21.15.7:0/2450904971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ec8105f50 0x7f1ec8198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:01.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.065+0000 7f1ecd560700 1 --2- 172.21.15.7:0/2450904971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ec81010b0 0x7f1ec819e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:01.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.065+0000 7f1ecd560700 1 --2- 172.21.15.7:0/2450904971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ec8101a60 0x7f1ec819ee00 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:01.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.065+0000 7f1ecd560700 1 -- 172.21.15.7:0/2450904971 >> 172.21.15.7:0/2450904971 conn(0x7f1ec80766b0 msgr2=0x7f1ec8100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:01.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.066+0000 7f1ecd560700 1 -- 172.21.15.7:0/2450904971 shutdown_connections 2024-06-10T01:31:01.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:01.066+0000 7f1ecd560700 1 -- 172.21.15.7:0/2450904971 wait complete. 2024-06-10T01:31:01.260 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:01 smithi007 ceph-mon[32398]: pgmap v1184: 129 pgs: 129 active+clean; 667 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 3.8 MiB/s wr, 475 op/s 2024-06-10T01:31:01.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:01 smithi175 ceph-mon[36636]: pgmap v1184: 129 pgs: 129 active+clean; 667 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 3.8 MiB/s wr, 475 op/s 2024-06-10T01:31:01.569 INFO:tasks.workunit.client.0.smithi007.stdout: 10 275169 21.87 MB/sec execute 578 sec latency 102.945 ms 2024-06-10T01:31:01.576 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:01 smithi078 ceph-mon[36683]: pgmap v1184: 129 pgs: 129 active+clean; 667 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 3.8 MiB/s wr, 475 op/s 2024-06-10T01:31:01.859 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:31:01.860 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:31:02.256 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:31:02.569 INFO:tasks.workunit.client.0.smithi007.stdout: 10 275857 21.90 MB/sec execute 579 sec latency 69.561 ms 2024-06-10T01:31:03.260 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:03 smithi007 ceph-mon[32398]: pgmap v1185: 129 pgs: 129 active+clean; 674 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.9 MiB/s wr, 609 op/s 2024-06-10T01:31:03.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:03 smithi175 ceph-mon[36636]: pgmap v1185: 129 pgs: 129 active+clean; 674 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.9 MiB/s wr, 609 op/s 2024-06-10T01:31:03.569 INFO:tasks.workunit.client.0.smithi007.stdout: 10 276463 21.93 MB/sec execute 580 sec latency 45.272 ms 2024-06-10T01:31:03.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:03 smithi078 ceph-mon[36683]: pgmap v1185: 129 pgs: 129 active+clean; 674 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.9 MiB/s wr, 609 op/s 2024-06-10T01:31:04.569 INFO:tasks.workunit.client.0.smithi007.stdout: 10 276824 21.92 MB/sec execute 581 sec latency 82.899 ms 2024-06-10T01:31:05.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.054+0000 7fa1f1c9d700 1 -- 172.21.15.7:0/3755007070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1ec11bc70 msgr2=0x7fa1ec11e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:05.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.054+0000 7fa1f1c9d700 1 --2- 172.21.15.7:0/3755007070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1ec11bc70 0x7fa1ec11e060 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009f90 tx=0x7fa1e400b330 comp rx=0 tx=0).stop 2024-06-10T01:31:05.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.055+0000 7fa1f1c9d700 1 -- 172.21.15.7:0/3755007070 shutdown_connections 2024-06-10T01:31:05.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.055+0000 7fa1f1c9d700 1 --2- 172.21.15.7:0/3755007070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1ec11bc70 0x7fa1ec11e060 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:05.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.055+0000 7fa1f1c9d700 1 --2- 172.21.15.7:0/3755007070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1ec110c70 0x7fa1ec109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:05.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.055+0000 7fa1f1c9d700 1 --2- 172.21.15.7:0/3755007070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1ec1099d0 0x7fa1ec109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:05.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.055+0000 7fa1f1c9d700 1 -- 172.21.15.7:0/3755007070 >> 172.21.15.7:0/3755007070 conn(0x7fa1ec06d600 msgr2=0x7fa1ec06da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:05.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.060+0000 7fa1f1c9d700 1 -- 172.21.15.7:0/3755007070 shutdown_connections 2024-06-10T01:31:05.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.060+0000 7fa1f1c9d700 1 -- 172.21.15.7:0/3755007070 wait complete. 2024-06-10T01:31:05.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.061+0000 7fa1f1c9d700 1 Processor -- start 2024-06-10T01:31:05.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.062+0000 7fa1f1c9d700 1 -- start start 2024-06-10T01:31:05.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.062+0000 7fa1f1c9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1ec1099d0 0x7fa1ec11b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:05.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.063+0000 7fa1f1c9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1ec110c70 0x7fa1ec11b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:05.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.063+0000 7fa1f1c9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1ec11bc70 0x7fa1ec1174b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:05.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.063+0000 7fa1f1c9d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec113d70 con 0x7fa1ec11bc70 2024-06-10T01:31:05.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.063+0000 7fa1f1c9d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec113bf0 con 0x7fa1ec110c70 2024-06-10T01:31:05.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.063+0000 7fa1f1c9d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec113ef0 con 0x7fa1ec1099d0 2024-06-10T01:31:05.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.067+0000 7fa1ebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1ec11bc70 0x7fa1ec1174b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:05.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.067+0000 7fa1ebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1ec11bc70 0x7fa1ec1174b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56644/0 (socket says 172.21.15.7:56644) 2024-06-10T01:31:05.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.067+0000 7fa1ebfff700 1 -- 172.21.15.7:0/3517358860 learned_addr learned my addr 172.21.15.7:0/3517358860 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:05.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.068+0000 7fa1eb7fe700 1 --2- 172.21.15.7:0/3517358860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1ec1099d0 0x7fa1ec11b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:05.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.068+0000 7fa1eaffd700 1 --2- 172.21.15.7:0/3517358860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1ec110c70 0x7fa1ec11b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:05.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.071+0000 7fa1ebfff700 1 -- 172.21.15.7:0/3517358860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1ec1099d0 msgr2=0x7fa1ec11b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:05.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.071+0000 7fa1ebfff700 1 --2- 172.21.15.7:0/3517358860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1ec1099d0 0x7fa1ec11b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:05.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.071+0000 7fa1ebfff700 1 -- 172.21.15.7:0/3517358860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1ec110c70 msgr2=0x7fa1ec11b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:05.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.071+0000 7fa1ebfff700 1 --2- 172.21.15.7:0/3517358860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1ec110c70 0x7fa1ec11b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:05.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.071+0000 7fa1ebfff700 1 -- 172.21.15.7:0/3517358860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ec117db0 con 0x7fa1ec11bc70 2024-06-10T01:31:05.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.074+0000 7fa1ebfff700 1 --2- 172.21.15.7:0/3517358860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1ec11bc70 0x7fa1ec1174b0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009f60 tx=0x7fa1e4009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:05.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.076+0000 7fa1e8ff9700 1 -- 172.21.15.7:0/3517358860 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4021070 con 0x7fa1ec11bc70 2024-06-10T01:31:05.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.076+0000 7fa1f1c9d700 1 -- 172.21.15.7:0/3517358860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1ec118040 con 0x7fa1ec11bc70 2024-06-10T01:31:05.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.076+0000 7fa1f1c9d700 1 -- 172.21.15.7:0/3517358860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1ec1abd30 con 0x7fa1ec11bc70 2024-06-10T01:31:05.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.077+0000 7fa1e8ff9700 1 -- 172.21.15.7:0/3517358860 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1e4007bf0 con 0x7fa1ec11bc70 2024-06-10T01:31:05.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.077+0000 7fa1e8ff9700 1 -- 172.21.15.7:0/3517358860 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e40052a0 con 0x7fa1ec11bc70 2024-06-10T01:31:05.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.079+0000 7fa1f1c9d700 1 -- 172.21.15.7:0/3517358860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1d8005330 con 0x7fa1ec11bc70 2024-06-10T01:31:05.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.081+0000 7fa1e8ff9700 1 -- 172.21.15.7:0/3517358860 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa1e4024070 con 0x7fa1ec11bc70 2024-06-10T01:31:05.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.082+0000 7fa1e8ff9700 1 --2- 172.21.15.7:0/3517358860 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa1d40a2e20 0x7fa1d40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:05.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.082+0000 7fa1e8ff9700 1 -- 172.21.15.7:0/3517358860 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fa1e40d01e0 con 0x7fa1ec11bc70 2024-06-10T01:31:05.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.091+0000 7fa1e8ff9700 1 -- 172.21.15.7:0/3517358860 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa1e409cdf0 con 0x7fa1ec11bc70 2024-06-10T01:31:05.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.092+0000 7fa1eb7fe700 1 --2- 172.21.15.7:0/3517358860 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa1d40a2e20 0x7fa1d40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:05.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.105+0000 7fa1eb7fe700 1 --2- 172.21.15.7:0/3517358860 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa1d40a2e20 0x7fa1d40a52e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc005fd0 tx=0x7fa1dc01a320 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:05.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:05 smithi175 ceph-mon[36636]: pgmap v1186: 129 pgs: 129 active+clean; 674 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.1 MiB/s wr, 493 op/s 2024-06-10T01:31:05.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:05 smithi007 ceph-mon[32398]: pgmap v1186: 129 pgs: 129 active+clean; 674 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.1 MiB/s wr, 493 op/s 2024-06-10T01:31:05.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.573+0000 7fa1f1c9d700 1 -- 172.21.15.7:0/3517358860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1d8005620 con 0x7fa1ec11bc70 2024-06-10T01:31:05.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.574+0000 7fa1e8ff9700 1 -- 172.21.15.7:0/3517358860 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+11855 (secure 0 0 0) 0x7fa1e409c6f0 con 0x7fa1ec11bc70 2024-06-10T01:31:05.566 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:31:05.567 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":57,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:30:12.372548+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29813":{"gid":29813,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/138896429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":138896429},{"type":"v1","addr":"172.21.15.175:6835","nonce":138896429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24434,24436,24451]},"id":1}]} 2024-06-10T01:31:05.569 INFO:tasks.workunit.client.0.smithi007.stdout: 10 276927 21.89 MB/sec execute 582 sec latency 678.947 ms 2024-06-10T01:31:05.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.577+0000 7fa1f1c9d700 1 -- 172.21.15.7:0/3517358860 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa1d40a2e20 msgr2=0x7fa1d40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:05.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.577+0000 7fa1f1c9d700 1 --2- 172.21.15.7:0/3517358860 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa1d40a2e20 0x7fa1d40a52e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc005fd0 tx=0x7fa1dc01a320 comp rx=0 tx=0).stop 2024-06-10T01:31:05.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.577+0000 7fa1f1c9d700 1 -- 172.21.15.7:0/3517358860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1ec11bc70 msgr2=0x7fa1ec1174b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:05.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.577+0000 7fa1f1c9d700 1 --2- 172.21.15.7:0/3517358860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1ec11bc70 0x7fa1ec1174b0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009f60 tx=0x7fa1e4009390 comp rx=0 tx=0).stop 2024-06-10T01:31:05.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.578+0000 7fa1f1c9d700 1 -- 172.21.15.7:0/3517358860 shutdown_connections 2024-06-10T01:31:05.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.578+0000 7fa1f1c9d700 1 --2- 172.21.15.7:0/3517358860 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa1d40a2e20 0x7fa1d40a52e0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:05.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.578+0000 7fa1f1c9d700 1 --2- 172.21.15.7:0/3517358860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1ec11bc70 0x7fa1ec1174b0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:05.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.578+0000 7fa1f1c9d700 1 --2- 172.21.15.7:0/3517358860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1ec1099d0 0x7fa1ec11b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.578+0000 7fa1f1c9d700 1 --2- 172.21.15.7:0/3517358860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1ec110c70 0x7fa1ec11b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.578+0000 7fa1f1c9d700 1 -- 172.21.15.7:0/3517358860 >> 172.21.15.7:0/3517358860 conn(0x7fa1ec06d600 msgr2=0x7fa1ec11d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.578+0000 7fa1f1c9d700 1 -- 172.21.15.7:0/3517358860 shutdown_connections 2024-06-10T01:31:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:05.578+0000 7fa1f1c9d700 1 -- 172.21.15.7:0/3517358860 wait complete. 2024-06-10T01:31:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 57 2024-06-10T01:31:05.576 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:05 smithi078 ceph-mon[36683]: pgmap v1186: 129 pgs: 129 active+clean; 674 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.1 MiB/s wr, 493 op/s 2024-06-10T01:31:06.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:06 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3517358860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:31:06.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:06 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3517358860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:31:06.569 INFO:tasks.workunit.client.0.smithi007.stdout: 10 277191 21.88 MB/sec execute 583 sec latency 190.022 ms 2024-06-10T01:31:06.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:06 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3517358860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:31:06.729 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:31:07.072 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:31:07.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:07 smithi007 ceph-mon[32398]: pgmap v1187: 129 pgs: 129 active+clean; 685 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 5.3 MiB/s wr, 571 op/s 2024-06-10T01:31:07.569 INFO:tasks.workunit.client.0.smithi007.stdout: 10 277796 21.88 MB/sec execute 584 sec latency 89.859 ms 2024-06-10T01:31:07.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:07 smithi175 ceph-mon[36636]: pgmap v1187: 129 pgs: 129 active+clean; 685 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 5.3 MiB/s wr, 571 op/s 2024-06-10T01:31:07.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:07 smithi078 ceph-mon[36683]: pgmap v1187: 129 pgs: 129 active+clean; 685 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 5.3 MiB/s wr, 571 op/s 2024-06-10T01:31:08.569 INFO:tasks.workunit.client.0.smithi007.stdout: 10 278551 21.91 MB/sec execute 585 sec latency 113.400 ms 2024-06-10T01:31:09.569 INFO:tasks.workunit.client.0.smithi007.stdout: 10 279280 21.95 MB/sec execute 586 sec latency 62.312 ms 2024-06-10T01:31:09.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:09 smithi175 ceph-mon[36636]: pgmap v1188: 129 pgs: 129 active+clean; 693 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.2 MiB/s wr, 584 op/s 2024-06-10T01:31:09.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:09 smithi007 ceph-mon[32398]: pgmap v1188: 129 pgs: 129 active+clean; 693 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.2 MiB/s wr, 584 op/s 2024-06-10T01:31:09.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:09 smithi078 ceph-mon[36683]: pgmap v1188: 129 pgs: 129 active+clean; 693 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.2 MiB/s wr, 584 op/s 2024-06-10T01:31:10.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.267+0000 7f81f2fcb700 1 -- 172.21.15.7:0/551787295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81ec109480 msgr2=0x7f81ec115990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:10.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.267+0000 7f81f2fcb700 1 --2- 172.21.15.7:0/551787295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81ec109480 0x7f81ec115990 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009990 tx=0x7f81e4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.271+0000 7f81f2fcb700 1 -- 172.21.15.7:0/551787295 shutdown_connections 2024-06-10T01:31:10.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.271+0000 7f81f2fcb700 1 --2- 172.21.15.7:0/551787295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81ec109480 0x7f81ec115990 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.271+0000 7f81f2fcb700 1 --2- 172.21.15.7:0/551787295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81ec10eac0 0x7f81ec10eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.271+0000 7f81f2fcb700 1 --2- 172.21.15.7:0/551787295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81ec108ac0 0x7f81ec108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.271+0000 7f81f2fcb700 1 -- 172.21.15.7:0/551787295 >> 172.21.15.7:0/551787295 conn(0x7f81ec075c60 msgr2=0x7f81ec078080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:10.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.272+0000 7f81f2fcb700 1 -- 172.21.15.7:0/551787295 shutdown_connections 2024-06-10T01:31:10.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.272+0000 7f81f2fcb700 1 -- 172.21.15.7:0/551787295 wait complete. 2024-06-10T01:31:10.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.274+0000 7f81f2fcb700 1 Processor -- start 2024-06-10T01:31:10.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.275+0000 7f81f2fcb700 1 -- start start 2024-06-10T01:31:10.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.275+0000 7f81f2fcb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81ec108ac0 0x7f81ec082970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:10.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.275+0000 7f81f2fcb700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81ec109480 0x7f81ec082eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:10.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.275+0000 7f81f2fcb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81ec10eac0 0x7f81ec07dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:10.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.275+0000 7f81f2fcb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec07c5e0 con 0x7f81ec108ac0 2024-06-10T01:31:10.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.275+0000 7f81f2fcb700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec07c460 con 0x7f81ec10eac0 2024-06-10T01:31:10.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.275+0000 7f81f2fcb700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec07c760 con 0x7f81ec109480 2024-06-10T01:31:10.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.276+0000 7f81f1568700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81ec10eac0 0x7f81ec07dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:10.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.276+0000 7f81f1568700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81ec10eac0 0x7f81ec07dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:34966/0 (socket says 172.21.15.7:34966) 2024-06-10T01:31:10.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.276+0000 7f81f1568700 1 -- 172.21.15.7:0/2814041541 learned_addr learned my addr 172.21.15.7:0/2814041541 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:10.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.276+0000 7f81e3fff700 1 --2- 172.21.15.7:0/2814041541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81ec109480 0x7f81ec082eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:10.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.276+0000 7f81f0d67700 1 --2- 172.21.15.7:0/2814041541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81ec108ac0 0x7f81ec082970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:10.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.277+0000 7f81e3fff700 1 -- 172.21.15.7:0/2814041541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81ec10eac0 msgr2=0x7f81ec07dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:10.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.277+0000 7f81e3fff700 1 --2- 172.21.15.7:0/2814041541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81ec10eac0 0x7f81ec07dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.277+0000 7f81e3fff700 1 -- 172.21.15.7:0/2814041541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81ec108ac0 msgr2=0x7f81ec082970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:10.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.277+0000 7f81e3fff700 1 --2- 172.21.15.7:0/2814041541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81ec108ac0 0x7f81ec082970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.277+0000 7f81e3fff700 1 -- 172.21.15.7:0/2814041541 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81ec07e3d0 con 0x7f81ec109480 2024-06-10T01:31:10.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.277+0000 7f81e3fff700 1 --2- 172.21.15.7:0/2814041541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81ec109480 0x7f81ec082eb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f81d400d950 tx=0x7f81d400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:10.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.279+0000 7f81ea7fc700 1 -- 172.21.15.7:0/2814041541 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4017070 con 0x7f81ec109480 2024-06-10T01:31:10.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.279+0000 7f81f2fcb700 1 -- 172.21.15.7:0/2814041541 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81ec07e6c0 con 0x7f81ec109480 2024-06-10T01:31:10.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.280+0000 7f81f2fcb700 1 -- 172.21.15.7:0/2814041541 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81ec0730c0 con 0x7f81ec109480 2024-06-10T01:31:10.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.290+0000 7f81fd84eb80 1 Processor -- start 2024-06-10T01:31:10.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.290+0000 7f81ea7fc700 1 -- 172.21.15.7:0/2814041541 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81d4009e10 con 0x7f81ec109480 2024-06-10T01:31:10.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.291+0000 7f81ea7fc700 1 -- 172.21.15.7:0/2814041541 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d400f3c0 con 0x7f81ec109480 2024-06-10T01:31:10.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.291+0000 7f81fd84eb80 1 -- start start 2024-06-10T01:31:10.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.292+0000 7f81fd84eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556284c0f60 0x5556284c1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:10.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.292+0000 7f81fd84eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556284c1880 0x5556284bb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:10.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.292+0000 7f81fd84eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556284bb7a0 0x5556284bbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.292+0000 7f81fd84eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5556283adf60 con 0x5556284c1880 2024-06-10T01:31:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.292+0000 7f81fd84eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55562840c850 con 0x5556284bb7a0 2024-06-10T01:31:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.292+0000 7f81fd84eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555628374460 con 0x5556284c0f60 2024-06-10T01:31:10.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.298+0000 7f81f1568700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556284c0f60 0x5556284c1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:10.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.298+0000 7f81f1568700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556284c0f60 0x5556284c1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36304/0 (socket says 172.21.15.7:36304) 2024-06-10T01:31:10.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.298+0000 7f81f1568700 1 -- 172.21.15.7:0/2483780653 learned_addr learned my addr 172.21.15.7:0/2483780653 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:10.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.299+0000 7f81ea7fc700 1 -- 172.21.15.7:0/2814041541 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f81d401a070 con 0x7f81ec109480 2024-06-10T01:31:10.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.299+0000 7f81e3fff700 1 --2- 172.21.15.7:0/2483780653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556284bb7a0 0x5556284bbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:10.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.299+0000 7f81e3fff700 1 -- 172.21.15.7:0/2483780653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556284c0f60 msgr2=0x5556284c1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:10.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.299+0000 7f81e3fff700 1 --2- 172.21.15.7:0/2483780653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556284c0f60 0x5556284c1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.299+0000 7f81e3fff700 1 -- 172.21.15.7:0/2483780653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556284c1880 msgr2=0x5556284bb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:10.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.299+0000 7f81e3fff700 1 --2- 172.21.15.7:0/2483780653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556284c1880 0x5556284bb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.299+0000 7f81e3fff700 1 -- 172.21.15.7:0/2483780653 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55562835b8d0 con 0x5556284bb7a0 2024-06-10T01:31:10.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.300+0000 7f81e3fff700 1 --2- 172.21.15.7:0/2483780653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556284bb7a0 0x5556284bbbc0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f81d40cc350 tx=0x7f81d40d2e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1e682d0e1968dae server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:10.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.301+0000 7f81e2ffd700 1 -- 172.21.15.7:0/2483780653 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d40d7920 con 0x5556284bb7a0 2024-06-10T01:31:10.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.301+0000 7f81e2ffd700 1 -- 172.21.15.7:0/2483780653 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81d40e0070 con 0x5556284bb7a0 2024-06-10T01:31:10.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.301+0000 7f81e2ffd700 1 -- 172.21.15.7:0/2483780653 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d40da360 con 0x5556284bb7a0 2024-06-10T01:31:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.302+0000 7f81fd84eb80 1 -- 172.21.15.7:0/2483780653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556284bb7a0 msgr2=0x5556284bbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.302+0000 7f81fd84eb80 1 --2- 172.21.15.7:0/2483780653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556284bb7a0 0x5556284bbbc0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f81d40cc350 tx=0x7f81d40d2e20 comp rx=0 tx=0).stop 2024-06-10T01:31:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.303+0000 7f81ea7fc700 1 --2- 172.21.15.7:0/2814041541 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f81d80a2db0 0x7f81d80a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.303+0000 7f81ea7fc700 1 -- 172.21.15.7:0/2814041541 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f81d40cec00 con 0x7f81ec109480 2024-06-10T01:31:10.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.305+0000 7f81f1568700 1 --2- 172.21.15.7:0/2814041541 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f81d80a2db0 0x7f81d80a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:10.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.306+0000 7f81f1568700 1 --2- 172.21.15.7:0/2814041541 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f81d80a2db0 0x7f81d80a5270 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009750 tx=0x7f81e401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:10.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.306+0000 7f81fd84eb80 1 -- 172.21.15.7:0/2483780653 shutdown_connections 2024-06-10T01:31:10.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.306+0000 7f81fd84eb80 1 --2- 172.21.15.7:0/2483780653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556284bb7a0 0x5556284bbbc0 secure :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f81d40cc350 tx=0x7f81d40d2e20 comp rx=0 tx=0).stop 2024-06-10T01:31:10.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.306+0000 7f81fd84eb80 1 --2- 172.21.15.7:0/2483780653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556284c0f60 0x5556284c1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.306+0000 7f81fd84eb80 1 --2- 172.21.15.7:0/2483780653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556284c1880 0x5556284bb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.306+0000 7f81fd84eb80 1 -- 172.21.15.7:0/2483780653 >> 172.21.15.7:0/2483780653 conn(0x5556283e8080 msgr2=0x5556284b6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:10.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.310+0000 7f81fd84eb80 1 -- 172.21.15.7:0/2483780653 shutdown_connections 2024-06-10T01:31:10.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.310+0000 7f81fd84eb80 1 -- 172.21.15.7:0/2483780653 wait complete. 2024-06-10T01:31:10.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.311+0000 7f81fd84eb80 1 -- start start 2024-06-10T01:31:10.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.311+0000 7f81fd84eb80 1 Processor -- start 2024-06-10T01:31:10.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.312+0000 7f81fd84eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556284c0f60 0x5556284c7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:10.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.312+0000 7f81fd84eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556284c1880 0x5556284c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:10.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.312+0000 7f81fd84eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556284c50d0 0x5556284c5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:10.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.312+0000 7f81fd84eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5556283acfc0 con 0x5556284c1880 2024-06-10T01:31:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.312+0000 7f81fd84eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555628408810 con 0x5556284c0f60 2024-06-10T01:31:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.313+0000 7f81fd84eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5556284c8250 con 0x5556284c50d0 2024-06-10T01:31:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.313+0000 7f81fd84eb80 10 client.0 _ll_register_callbacks cb 0x5556283a25b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:31:10.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.313+0000 7f81e3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556284c1880 0x5556284c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:10.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.313+0000 7f81e3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556284c1880 0x5556284c4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56670/0 (socket says 172.21.15.7:56670) 2024-06-10T01:31:10.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.313+0000 7f81e3fff700 1 -- 172.21.15.7:0/2802550045 learned_addr learned my addr 172.21.15.7:0/2802550045 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:10.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.314+0000 7f81f1568700 1 --2- 172.21.15.7:0/2802550045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556284c50d0 0x5556284c5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:10.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.314+0000 7f81f0d67700 1 --2- 172.21.15.7:0/2802550045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556284c0f60 0x5556284c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:10.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.314+0000 7f81e3fff700 1 -- 172.21.15.7:0/2802550045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556284c50d0 msgr2=0x5556284c5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:10.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.314+0000 7f81e3fff700 1 --2- 172.21.15.7:0/2802550045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556284c50d0 0x5556284c5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.314+0000 7f81e3fff700 1 -- 172.21.15.7:0/2802550045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556284c0f60 msgr2=0x5556284c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:10.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.314+0000 7f81e3fff700 1 --2- 172.21.15.7:0/2802550045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556284c0f60 0x5556284c7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.314+0000 7f81e3fff700 1 -- 172.21.15.7:0/2802550045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556283d2750 con 0x5556284c1880 2024-06-10T01:31:10.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.315+0000 7f81e3fff700 1 --2- 172.21.15.7:0/2802550045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556284c1880 0x5556284c4b40 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f81d4005950 tx=0x7f81d40d74c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=837f4173a1495850 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:10.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.316+0000 7f81cbfff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:31:10.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.316+0000 7f81cbfff700 1 -- 172.21.15.7:0/2802550045 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4021030 con 0x5556284c1880 2024-06-10T01:31:10.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.316+0000 7f81cbfff700 1 -- 172.21.15.7:0/2802550045 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f81d40d2110 con 0x5556284c1880 2024-06-10T01:31:10.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.316+0000 7f81fd84eb80 1 -- 172.21.15.7:0/2802550045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81d40e0070 con 0x5556284c1880 2024-06-10T01:31:10.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.317+0000 7f81cbfff700 1 -- 172.21.15.7:0/2802550045 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f81d4097380 con 0x5556284c1880 2024-06-10T01:31:10.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.318+0000 7f81fd84eb80 10 client.21330 fetch_fsmap learned FSMap version 57 2024-06-10T01:31:10.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.318+0000 7f81fd84eb80 1 -- 172.21.15.7:0/2802550045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5556284d30c0 con 0x5556284c1880 2024-06-10T01:31:10.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.321+0000 7f81cbfff700 1 -- 172.21.15.7:0/2802550045 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81d409b7e0 con 0x5556284c1880 2024-06-10T01:31:10.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.321+0000 7f81cbfff700 1 -- 172.21.15.7:0/2802550045 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7f81d409f100 con 0x5556284c1880 2024-06-10T01:31:10.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.323+0000 7f81fd84eb80 10 client.21330 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:31:10.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.323+0000 7f81fd84eb80 10 client.21330 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:31:10.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.323+0000 7f81fd84eb80 20 client.21330 populate_metadata read hostname 'smithi007' 2024-06-10T01:31:10.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.323+0000 7f81fd84eb80 1 --2- 172.21.15.7:0/2802550045 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5556284d5590 0x5556284d7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:10.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.323+0000 7f81fd84eb80 4 client.21330 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:31:10.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.323+0000 7f81fd84eb80 1 -- 172.21.15.7:0/2802550045 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f81d40e0070 con 0x5556284d5590 2024-06-10T01:31:10.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.329+0000 7f81f0d67700 1 --2- 172.21.15.7:0/2802550045 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5556284d5590 0x5556284d7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:10.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.341+0000 7f81f0d67700 1 --2- 172.21.15.7:0/2802550045 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5556284d5590 0x5556284d7980 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c1d71de09c7bdbb7 server_cookie=c3b559e8df714ff1 in_seq=0 out_seq=0 2024-06-10T01:31:10.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.342+0000 7f81cbfff700 10 client.21330 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:31:10.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.348+0000 7f81cbfff700 1 -- 172.21.15.7:0/2802550045 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f81d40e0070 con 0x5556284d5590 2024-06-10T01:31:10.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.348+0000 7f81cbfff700 10 client.21330 handle_command_reply: tid=0 2024-06-10T01:31:10.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.348+0000 7f81fd84eb80 1 client.21330 shutdown 2024-06-10T01:31:10.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.348+0000 7f81fd84eb80 1 -- 172.21.15.7:0/2802550045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556284c1880 msgr2=0x5556284c4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:10.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.348+0000 7f81fd84eb80 1 --2- 172.21.15.7:0/2802550045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556284c1880 0x5556284c4b40 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f81d4005950 tx=0x7f81d40d74c0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.349+0000 7f81fd84eb80 1 -- 172.21.15.7:0/2802550045 shutdown_connections 2024-06-10T01:31:10.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.349+0000 7f81fd84eb80 1 --2- 172.21.15.7:0/2802550045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556284c50d0 0x5556284c5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.349+0000 7f81fd84eb80 1 --2- 172.21.15.7:0/2802550045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556284c0f60 0x5556284c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.349+0000 7f81fd84eb80 1 --2- 172.21.15.7:0/2802550045 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5556284d5590 0x5556284d7980 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.349+0000 7f81fd84eb80 1 --2- 172.21.15.7:0/2802550045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556284c1880 0x5556284c4b40 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.349+0000 7f81cbfff700 0 client.21330 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:31:10.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.349+0000 7f81fd84eb80 1 -- 172.21.15.7:0/2802550045 >> 172.21.15.7:0/2802550045 conn(0x5556283e8080 msgr2=0x5556284b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:10.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.349+0000 7f81fd84eb80 1 -- 172.21.15.7:0/2802550045 shutdown_connections 2024-06-10T01:31:10.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.350+0000 7f81fd84eb80 1 -- 172.21.15.7:0/2802550045 wait complete. 2024-06-10T01:31:10.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.353+0000 7f81fd84eb80 20 client.21330 trim_cache size 0 max 16384 2024-06-10T01:31:10.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.383+0000 7f81fd84eb80 1 Processor -- start 2024-06-10T01:31:10.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.383+0000 7f81fd84eb80 1 -- start start 2024-06-10T01:31:10.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.383+0000 7f81fd84eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556284c0f60 0x5556284bb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:10.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.383+0000 7f81fd84eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556284c1880 0x5556284c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:10.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.383+0000 7f81fd84eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556284c50d0 0x5556284c03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:10.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.383+0000 7f81fd84eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc0114c0 con 0x5556284c50d0 2024-06-10T01:31:10.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.383+0000 7f81fd84eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc00ecf0 con 0x5556284c1880 2024-06-10T01:31:10.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.383+0000 7f81fd84eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc00cb40 con 0x5556284c0f60 2024-06-10T01:31:10.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.383+0000 7f81e3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556284c1880 0x5556284c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:10.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.384+0000 7f81e3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556284c1880 0x5556284c3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:34996/0 (socket says 172.21.15.7:34996) 2024-06-10T01:31:10.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.384+0000 7f81e3fff700 1 -- 172.21.15.7:0/2451256727 learned_addr learned my addr 172.21.15.7:0/2451256727 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:10.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.384+0000 7f81e3fff700 1 -- 172.21.15.7:0/2451256727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556284c0f60 msgr2=0x5556284bb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:10.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.384+0000 7f81e3fff700 1 --2- 172.21.15.7:0/2451256727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556284c0f60 0x5556284bb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.384+0000 7f81e3fff700 1 -- 172.21.15.7:0/2451256727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556284c50d0 msgr2=0x5556284c03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:10.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.384+0000 7f81e3fff700 1 --2- 172.21.15.7:0/2451256727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556284c50d0 0x5556284c03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.384+0000 7f81e3fff700 1 -- 172.21.15.7:0/2451256727 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556284b7d30 con 0x5556284c1880 2024-06-10T01:31:10.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.384+0000 7f81e3fff700 1 --2- 172.21.15.7:0/2451256727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556284c1880 0x5556284c3aa0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f81d4010b30 tx=0x7f81d40d2fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2bf5bfd37666662 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:10.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.384+0000 7f81e27fc700 1 -- 172.21.15.7:0/2451256727 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d40d2030 con 0x5556284c1880 2024-06-10T01:31:10.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.385+0000 7f81e27fc700 1 -- 172.21.15.7:0/2451256727 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81d40e0040 con 0x5556284c1880 2024-06-10T01:31:10.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.385+0000 7f81e27fc700 1 -- 172.21.15.7:0/2451256727 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4021070 con 0x5556284c1880 2024-06-10T01:31:10.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.386+0000 7f81fd84eb80 1 -- 172.21.15.7:0/2451256727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556284c1880 msgr2=0x5556284c3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:10.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.386+0000 7f81fd84eb80 1 --2- 172.21.15.7:0/2451256727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556284c1880 0x5556284c3aa0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f81d4010b30 tx=0x7f81d40d2fb0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.386+0000 7f81fd84eb80 1 -- 172.21.15.7:0/2451256727 shutdown_connections 2024-06-10T01:31:10.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.386+0000 7f81fd84eb80 1 --2- 172.21.15.7:0/2451256727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556284c50d0 0x5556284c03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.386+0000 7f81fd84eb80 1 --2- 172.21.15.7:0/2451256727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556284c0f60 0x5556284bb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.386+0000 7f81fd84eb80 1 --2- 172.21.15.7:0/2451256727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556284c1880 0x5556284c3aa0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.386+0000 7f81fd84eb80 1 -- 172.21.15.7:0/2451256727 >> 172.21.15.7:0/2451256727 conn(0x5556283e8080 msgr2=0x5556284bfc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:10.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.386+0000 7f81fd84eb80 1 -- 172.21.15.7:0/2451256727 shutdown_connections 2024-06-10T01:31:10.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.386+0000 7f81fd84eb80 1 -- 172.21.15.7:0/2451256727 wait complete. 2024-06-10T01:31:10.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.387+0000 7f81fd84eb80 1 -- start start 2024-06-10T01:31:10.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.387+0000 7f81fd84eb80 1 Processor -- start 2024-06-10T01:31:10.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.388+0000 7f81fd84eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556284c0f60 0x5556284d7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:10.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.388+0000 7f81fd84eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556284c1880 0x5556284d7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:10.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.388+0000 7f81fd84eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556284c50d0 0x5556284d8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:10.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.388+0000 7f81fd84eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc00d1b0 con 0x5556284c0f60 2024-06-10T01:31:10.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.388+0000 7f81fd84eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc00e620 con 0x5556284c1880 2024-06-10T01:31:10.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.388+0000 7f81fd84eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc00d880 con 0x5556284c50d0 2024-06-10T01:31:10.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.388+0000 7f81fd84eb80 10 client.0 _ll_register_callbacks cb 0x5556283a25b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:31:10.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.389+0000 7f81e3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556284c1880 0x5556284d7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:10.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.389+0000 7f81e3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556284c1880 0x5556284d7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:35004/0 (socket says 172.21.15.7:35004) 2024-06-10T01:31:10.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.389+0000 7f81e3fff700 1 -- 172.21.15.7:0/926691557 learned_addr learned my addr 172.21.15.7:0/926691557 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:10.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.389+0000 7f81f0d67700 1 --2- 172.21.15.7:0/926691557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556284c0f60 0x5556284d7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:10.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.390+0000 7f81f1568700 1 --2- 172.21.15.7:0/926691557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556284c50d0 0x5556284d8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:10.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.390+0000 7f81e3fff700 1 -- 172.21.15.7:0/926691557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556284c50d0 msgr2=0x5556284d8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:10.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.390+0000 7f81e3fff700 1 --2- 172.21.15.7:0/926691557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556284c50d0 0x5556284d8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.390+0000 7f81e3fff700 1 -- 172.21.15.7:0/926691557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556284c0f60 msgr2=0x5556284d7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:10.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.390+0000 7f81e3fff700 1 --2- 172.21.15.7:0/926691557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556284c0f60 0x5556284d7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.390+0000 7f81e3fff700 1 -- 172.21.15.7:0/926691557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55562853df10 con 0x5556284c1880 2024-06-10T01:31:10.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.392+0000 7f81e3fff700 1 --2- 172.21.15.7:0/926691557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556284c1880 0x5556284d7ec0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f81d4097540 tx=0x7f81d4099570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7218675977f269a6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:10.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.392+0000 7f81cbfff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:31:10.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.392+0000 7f81cbfff700 1 -- 172.21.15.7:0/926691557 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4021050 con 0x5556284c1880 2024-06-10T01:31:10.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.392+0000 7f81cbfff700 1 -- 172.21.15.7:0/926691557 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f81d409b340 con 0x5556284c1880 2024-06-10T01:31:10.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.392+0000 7f81fd84eb80 1 -- 172.21.15.7:0/926691557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81d40e0040 con 0x5556284c1880 2024-06-10T01:31:10.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.393+0000 7f81cbfff700 1 -- 172.21.15.7:0/926691557 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f81d40d2040 con 0x5556284c1880 2024-06-10T01:31:10.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.393+0000 7f81fd84eb80 10 client.30500 fetch_fsmap learned FSMap version 57 2024-06-10T01:31:10.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.393+0000 7f81fd84eb80 1 -- 172.21.15.7:0/926691557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55562853f210 con 0x5556284c1880 2024-06-10T01:31:10.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.395+0000 7f81cbfff700 1 -- 172.21.15.7:0/926691557 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81d40e0040 con 0x5556284c1880 2024-06-10T01:31:10.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.395+0000 7f81cbfff700 1 -- 172.21.15.7:0/926691557 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7f81d40d21e0 con 0x5556284c1880 2024-06-10T01:31:10.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.395+0000 7f81fd84eb80 10 client.30500 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:31:10.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.395+0000 7f81fd84eb80 10 client.30500 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:31:10.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.395+0000 7f81fd84eb80 20 client.30500 populate_metadata read hostname 'smithi007' 2024-06-10T01:31:10.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.395+0000 7f81fd84eb80 1 --2- 172.21.15.7:0/926691557 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5556284d5590 0x5556285415d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:10.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.395+0000 7f81fd84eb80 4 client.30500 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:31:10.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.395+0000 7f81fd84eb80 1 -- 172.21.15.7:0/926691557 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555628419d50 con 0x5556284d5590 2024-06-10T01:31:10.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.399+0000 7f81f0d67700 1 --2- 172.21.15.7:0/926691557 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5556284d5590 0x5556285415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:10.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.403+0000 7f81cbfff700 10 client.30500 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:31:10.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.403+0000 7f81f0d67700 1 --2- 172.21.15.7:0/926691557 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5556284d5590 0x5556285415d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=17573a3c62042495 server_cookie=e9cc254b8c1c2c47 in_seq=0 out_seq=0 2024-06-10T01:31:10.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.412+0000 7f81cbfff700 1 -- 172.21.15.7:0/926691557 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555628419d50 con 0x5556284d5590 2024-06-10T01:31:10.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.412+0000 7f81cbfff700 10 client.30500 handle_command_reply: tid=0 2024-06-10T01:31:10.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.412+0000 7f81fd84eb80 1 client.30500 shutdown 2024-06-10T01:31:10.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.412+0000 7f81fd84eb80 1 -- 172.21.15.7:0/926691557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556284c1880 msgr2=0x5556284d7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:10.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.412+0000 7f81fd84eb80 1 --2- 172.21.15.7:0/926691557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556284c1880 0x5556284d7ec0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f81d4097540 tx=0x7f81d4099570 comp rx=0 tx=0).stop 2024-06-10T01:31:10.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.413+0000 7f81fd84eb80 1 -- 172.21.15.7:0/926691557 shutdown_connections 2024-06-10T01:31:10.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.413+0000 7f81fd84eb80 1 --2- 172.21.15.7:0/926691557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5556284c50d0 0x5556284d8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.413+0000 7f81fd84eb80 1 --2- 172.21.15.7:0/926691557 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x5556284d5590 0x5556285415d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.413+0000 7f81fd84eb80 1 --2- 172.21.15.7:0/926691557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556284c0f60 0x5556284d7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.413+0000 7f81fd84eb80 1 --2- 172.21.15.7:0/926691557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556284c1880 0x5556284d7ec0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.413+0000 7f81cbfff700 0 client.30500 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:31:10.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.413+0000 7f81fd84eb80 1 -- 172.21.15.7:0/926691557 >> 172.21.15.7:0/926691557 conn(0x5556283e8080 msgr2=0x5556284d0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:10.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.413+0000 7f81fd84eb80 1 -- 172.21.15.7:0/926691557 shutdown_connections 2024-06-10T01:31:10.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.413+0000 7f81fd84eb80 1 -- 172.21.15.7:0/926691557 wait complete. 2024-06-10T01:31:10.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.414+0000 7f81fd84eb80 20 client.30500 trim_cache size 0 max 16384 2024-06-10T01:31:10.407 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:31:10.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.416+0000 7f81f2fcb700 1 -- 172.21.15.7:0/2814041541 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f81d80a2db0 msgr2=0x7f81d80a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:10.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.416+0000 7f81f2fcb700 1 --2- 172.21.15.7:0/2814041541 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f81d80a2db0 0x7f81d80a5270 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009750 tx=0x7f81e401a040 comp rx=0 tx=0).stop 2024-06-10T01:31:10.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.416+0000 7f81f2fcb700 1 -- 172.21.15.7:0/2814041541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81ec109480 msgr2=0x7f81ec082eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:10.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.416+0000 7f81f2fcb700 1 --2- 172.21.15.7:0/2814041541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81ec109480 0x7f81ec082eb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f81d400d950 tx=0x7f81d400dc60 comp rx=0 tx=0).stop 2024-06-10T01:31:10.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.416+0000 7f81f2fcb700 1 -- 172.21.15.7:0/2814041541 shutdown_connections 2024-06-10T01:31:10.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.416+0000 7f81f2fcb700 1 --2- 172.21.15.7:0/2814041541 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f81d80a2db0 0x7f81d80a5270 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.416+0000 7f81f2fcb700 1 --2- 172.21.15.7:0/2814041541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81ec10eac0 0x7f81ec07dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.416+0000 7f81f2fcb700 1 --2- 172.21.15.7:0/2814041541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81ec108ac0 0x7f81ec082970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.416+0000 7f81f2fcb700 1 --2- 172.21.15.7:0/2814041541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81ec109480 0x7f81ec082eb0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:10.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.416+0000 7f81f2fcb700 1 -- 172.21.15.7:0/2814041541 >> 172.21.15.7:0/2814041541 conn(0x7f81ec075c60 msgr2=0x7f81ec0760e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:10.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.416+0000 7f81f2fcb700 1 -- 172.21.15.7:0/2814041541 shutdown_connections 2024-06-10T01:31:10.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:10.417+0000 7f81f2fcb700 1 -- 172.21.15.7:0/2814041541 wait complete. 2024-06-10T01:31:10.569 INFO:tasks.workunit.client.0.smithi007.stdout: 10 279521 21.94 MB/sec execute 587 sec latency 630.631 ms 2024-06-10T01:31:11.244 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:31:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.345+0000 7f3455641640 1 -- 172.21.15.7:0/1821747474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3450101c30 msgr2=0x7f34501084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.345+0000 7f3455641640 1 --2- 172.21.15.7:0/1821747474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3450101c30 0x7f34501084c0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f344000aa10 tx=0x7f344002b760 comp rx=0 tx=0).stop 2024-06-10T01:31:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.345+0000 7f3455641640 1 -- 172.21.15.7:0/1821747474 shutdown_connections 2024-06-10T01:31:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.345+0000 7f3455641640 1 --2- 172.21.15.7:0/1821747474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3450101c30 0x7f34501084c0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.345+0000 7f3455641640 1 --2- 172.21.15.7:0/1821747474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3450101270 0x7f34501016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.345+0000 7f3455641640 1 --2- 172.21.15.7:0/1821747474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3450100080 0x7f34501004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.345+0000 7f3455641640 1 -- 172.21.15.7:0/1821747474 >> 172.21.15.7:0/1821747474 conn(0x7f34500fb610 msgr2=0x7f34500fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:11.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.350+0000 7f3455641640 1 -- 172.21.15.7:0/1821747474 shutdown_connections 2024-06-10T01:31:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.355+0000 7f3455641640 1 -- 172.21.15.7:0/1821747474 wait complete. 2024-06-10T01:31:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.355+0000 7f3455641640 1 Processor -- start 2024-06-10T01:31:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.356+0000 7f3455641640 1 -- start start 2024-06-10T01:31:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.356+0000 7f3455641640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3450100080 0x7f345006e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.356+0000 7f3455641640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3450101270 0x7f345006e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.356+0000 7f3455641640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34500736e0 0x7f345010fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.356+0000 7f3455641640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f345010abf0 con 0x7f34500736e0 2024-06-10T01:31:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.356+0000 7f3455641640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f345010aa70 con 0x7f3450100080 2024-06-10T01:31:11.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.356+0000 7f3455641640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f345010ad70 con 0x7f3450101270 2024-06-10T01:31:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.357+0000 7f344f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34500736e0 0x7f345010fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.357+0000 7f344f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34500736e0 0x7f345010fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56698/0 (socket says 172.21.15.7:56698) 2024-06-10T01:31:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.357+0000 7f344f7fe640 1 -- 172.21.15.7:0/583538546 learned_addr learned my addr 172.21.15.7:0/583538546 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.360+0000 7f344e7fc640 1 --2- 172.21.15.7:0/583538546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3450101270 0x7f345006e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.361+0000 7f344effd640 1 --2- 172.21.15.7:0/583538546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3450100080 0x7f345006e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.365+0000 7f344f7fe640 1 -- 172.21.15.7:0/583538546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3450101270 msgr2=0x7f345006e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.365+0000 7f344f7fe640 1 --2- 172.21.15.7:0/583538546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3450101270 0x7f345006e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.365+0000 7f344f7fe640 1 -- 172.21.15.7:0/583538546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3450100080 msgr2=0x7f345006e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.365+0000 7f344f7fe640 1 --2- 172.21.15.7:0/583538546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3450100080 0x7f345006e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.365+0000 7f344f7fe640 1 -- 172.21.15.7:0/583538546 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34501103a0 con 0x7f34500736e0 2024-06-10T01:31:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.365+0000 7f344e7fc640 1 --2- 172.21.15.7:0/583538546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3450101270 0x7f345006e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:31:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.365+0000 7f344f7fe640 1 --2- 172.21.15.7:0/583538546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34500736e0 0x7f345010fe30 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f344002bd50 tx=0x7f3440030c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.365+0000 7f344effd640 1 --2- 172.21.15.7:0/583538546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3450100080 0x7f345006e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:31:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.366+0000 7f3437fff640 1 -- 172.21.15.7:0/583538546 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344003f070 con 0x7f34500736e0 2024-06-10T01:31:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.366+0000 7f3455641640 1 -- 172.21.15.7:0/583538546 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3450110630 con 0x7f34500736e0 2024-06-10T01:31:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.366+0000 7f3437fff640 1 -- 172.21.15.7:0/583538546 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3440003cd0 con 0x7f34500736e0 2024-06-10T01:31:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.366+0000 7f3437fff640 1 -- 172.21.15.7:0/583538546 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3440036070 con 0x7f34500736e0 2024-06-10T01:31:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.366+0000 7f3455641640 1 -- 172.21.15.7:0/583538546 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3450110b70 con 0x7f34500736e0 2024-06-10T01:31:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.371+0000 7f3458420740 1 Processor -- start 2024-06-10T01:31:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.372+0000 7f3437fff640 1 -- 172.21.15.7:0/583538546 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f3440042070 con 0x7f34500736e0 2024-06-10T01:31:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.373+0000 7f3437fff640 1 --2- 172.21.15.7:0/583538546 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f34380a2cc0 0x7f34380a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.374+0000 7f3437fff640 1 -- 172.21.15.7:0/583538546 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f34400efe70 con 0x7f34500736e0 2024-06-10T01:31:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.374+0000 7f344e7fc640 1 --2- 172.21.15.7:0/583538546 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f34380a2cc0 0x7f34380a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.374+0000 7f3458420740 1 -- start start 2024-06-10T01:31:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.375+0000 7f3458420740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6007bf010 0x55f6007bf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.375+0000 7f3458420740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6007bfa00 0x55f6007c4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.375+0000 7f3458420740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6007c4b70 0x55f6007c7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.375+0000 7f3458420740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f6007224f0 con 0x55f6007c4b70 2024-06-10T01:31:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.375+0000 7f3458420740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f6006c0870 con 0x55f6007bf010 2024-06-10T01:31:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.375+0000 7f3458420740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f60071e4b0 con 0x55f6007bfa00 2024-06-10T01:31:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.375+0000 7f344f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6007c4b70 0x55f6007c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.375+0000 7f344f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6007c4b70 0x55f6007c7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56702/0 (socket says 172.21.15.7:56702) 2024-06-10T01:31:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.375+0000 7f344f7fe640 1 -- 172.21.15.7:0/9799474 learned_addr learned my addr 172.21.15.7:0/9799474 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.376+0000 7f344f7fe640 1 -- 172.21.15.7:0/9799474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6007bfa00 msgr2=0x55f6007c4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.376+0000 7f344f7fe640 1 --2- 172.21.15.7:0/9799474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6007bfa00 0x55f6007c4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.376+0000 7f344f7fe640 1 -- 172.21.15.7:0/9799474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6007bf010 msgr2=0x55f6007bf430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:31:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.376+0000 7f344f7fe640 1 --2- 172.21.15.7:0/9799474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6007bf010 0x55f6007bf430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.376+0000 7f344f7fe640 1 -- 172.21.15.7:0/9799474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6006dc9b0 con 0x55f6007c4b70 2024-06-10T01:31:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.376+0000 7f344f7fe640 1 --2- 172.21.15.7:0/9799474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6007c4b70 0x55f6007c7000 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f34400c4a30 tx=0x7f34400bbf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8076eacdfd6457 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.379+0000 7f344e7fc640 1 --2- 172.21.15.7:0/583538546 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f34380a2cc0 0x7f34380a5180 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3444006fd0 tx=0x7f3444009d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.383+0000 7f34357fa640 1 -- 172.21.15.7:0/9799474 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3440049040 con 0x55f6007c4b70 2024-06-10T01:31:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.383+0000 7f34357fa640 1 -- 172.21.15.7:0/9799474 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34400c06a0 con 0x55f6007c4b70 2024-06-10T01:31:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.383+0000 7f34357fa640 1 -- 172.21.15.7:0/9799474 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34400b6e60 con 0x55f6007c4b70 2024-06-10T01:31:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.384+0000 7f3458420740 1 -- 172.21.15.7:0/9799474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6007c4b70 msgr2=0x55f6007c7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.384+0000 7f3458420740 1 --2- 172.21.15.7:0/9799474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6007c4b70 0x55f6007c7000 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f34400c4a30 tx=0x7f34400bbf60 comp rx=0 tx=0).stop 2024-06-10T01:31:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.385+0000 7f3458420740 1 -- 172.21.15.7:0/9799474 shutdown_connections 2024-06-10T01:31:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.385+0000 7f3458420740 1 --2- 172.21.15.7:0/9799474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6007c4b70 0x55f6007c7000 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.385+0000 7f3458420740 1 --2- 172.21.15.7:0/9799474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6007bfa00 0x55f6007c4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.385+0000 7f3458420740 1 --2- 172.21.15.7:0/9799474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6007bf010 0x55f6007bf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.385+0000 7f3458420740 1 -- 172.21.15.7:0/9799474 >> 172.21.15.7:0/9799474 conn(0x55f6007ba680 msgr2=0x55f6007baa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.385+0000 7f3458420740 1 -- 172.21.15.7:0/9799474 shutdown_connections 2024-06-10T01:31:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.385+0000 7f3458420740 1 -- 172.21.15.7:0/9799474 wait complete. 2024-06-10T01:31:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.386+0000 7f3458420740 1 -- start start 2024-06-10T01:31:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.386+0000 7f3458420740 1 Processor -- start 2024-06-10T01:31:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.389+0000 7f3458420740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6007bf010 0x55f6007c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.389+0000 7f3458420740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6007bfa00 0x55f6007c9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.389+0000 7f3458420740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6007c4b70 0x55f6007d0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.389+0000 7f3458420740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f6007c78e0 con 0x55f6007c4b70 2024-06-10T01:31:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.389+0000 7f3458420740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f6006abae0 con 0x55f6007bfa00 2024-06-10T01:31:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.389+0000 7f3458420740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f6007c7a60 con 0x55f6007bf010 2024-06-10T01:31:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.389+0000 7f3458420740 10 client.0 _ll_register_callbacks cb 0x55f6006e49b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:31:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.390+0000 7f344effd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6007bf010 0x55f6007c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.390+0000 7f344effd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6007bf010 0x55f6007c8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36362/0 (socket says 172.21.15.7:36362) 2024-06-10T01:31:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.390+0000 7f344effd640 1 -- 172.21.15.7:0/2524323916 learned_addr learned my addr 172.21.15.7:0/2524323916 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.390+0000 7f344e7fc640 1 --2- 172.21.15.7:0/2524323916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6007bfa00 0x55f6007c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.390+0000 7f344effd640 1 -- 172.21.15.7:0/2524323916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6007bfa00 msgr2=0x55f6007c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.390+0000 7f344effd640 1 --2- 172.21.15.7:0/2524323916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6007bfa00 0x55f6007c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.390+0000 7f344effd640 1 -- 172.21.15.7:0/2524323916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6007c4b70 msgr2=0x55f6007d0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.390+0000 7f344effd640 1 --2- 172.21.15.7:0/2524323916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6007c4b70 0x55f6007d0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.390+0000 7f344effd640 1 -- 172.21.15.7:0/2524323916 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f600436210 con 0x55f6007bf010 2024-06-10T01:31:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.391+0000 7f344effd640 1 --2- 172.21.15.7:0/2524323916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6007bf010 0x55f6007c8ff0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f344800e6b0 tx=0x7f344800eb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d8cf1706a4c250c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.391+0000 7f3432ff5640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:31:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.391+0000 7f3432ff5640 1 -- 172.21.15.7:0/2524323916 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3448004d20 con 0x55f6007bf010 2024-06-10T01:31:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.391+0000 7f3458420740 1 -- 172.21.15.7:0/2524323916 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34400c06a0 con 0x55f6007bf010 2024-06-10T01:31:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.391+0000 7f3432ff5640 1 -- 172.21.15.7:0/2524323916 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f344800c7f0 con 0x55f6007bf010 2024-06-10T01:31:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.395+0000 7f3432ff5640 1 -- 172.21.15.7:0/2524323916 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f3448022050 con 0x55f6007bf010 2024-06-10T01:31:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.398+0000 7f3458420740 10 client.30388 fetch_fsmap learned FSMap version 57 2024-06-10T01:31:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.398+0000 7f3458420740 1 -- 172.21.15.7:0/2524323916 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55f600502620 con 0x55f6007bf010 2024-06-10T01:31:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.399+0000 7f3432ff5640 1 -- 172.21.15.7:0/2524323916 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34400c06a0 con 0x55f6007bf010 2024-06-10T01:31:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.400+0000 7f3432ff5640 1 -- 172.21.15.7:0/2524323916 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7f344800f040 con 0x55f6007bf010 2024-06-10T01:31:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.400+0000 7f3458420740 10 client.30388 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:31:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.400+0000 7f3458420740 10 client.30388 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:31:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.400+0000 7f3458420740 20 client.30388 populate_metadata read hostname 'smithi007' 2024-06-10T01:31:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.400+0000 7f3458420740 1 --2- 172.21.15.7:0/2524323916 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f6007d53f0 0x55f6007d7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.400+0000 7f3458420740 4 client.30388 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:31:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.400+0000 7f3458420740 1 -- 172.21.15.7:0/2524323916 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f600735f10 con 0x55f6007d53f0 2024-06-10T01:31:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.402+0000 7f344e7fc640 1 --2- 172.21.15.7:0/2524323916 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f6007d53f0 0x55f6007d7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.407+0000 7f344e7fc640 1 --2- 172.21.15.7:0/2524323916 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f6007d53f0 0x55f6007d7870 crc :-1 s=READY pgs=557 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b88707086fbd8fa server_cookie=50fcec465605b98d in_seq=0 out_seq=0 2024-06-10T01:31:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.407+0000 7f3432ff5640 10 client.30388 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.414+0000 7f3432ff5640 1 -- 172.21.15.7:0/2524323916 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f600735f10 con 0x55f6007d53f0 2024-06-10T01:31:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.414+0000 7f3432ff5640 10 client.30388 handle_command_reply: tid=0 2024-06-10T01:31:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.415+0000 7f3458420740 1 client.30388 shutdown 2024-06-10T01:31:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.423+0000 7f3458420740 1 -- 172.21.15.7:0/2524323916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6007bf010 msgr2=0x55f6007c8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.423+0000 7f3458420740 1 --2- 172.21.15.7:0/2524323916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6007bf010 0x55f6007c8ff0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f344800e6b0 tx=0x7f344800eb70 comp rx=0 tx=0).stop 2024-06-10T01:31:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.424+0000 7f3458420740 1 -- 172.21.15.7:0/2524323916 shutdown_connections 2024-06-10T01:31:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.424+0000 7f3458420740 1 --2- 172.21.15.7:0/2524323916 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f6007d53f0 0x55f6007d7870 crc :-1 s=READY pgs=557 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.424+0000 7f3458420740 1 --2- 172.21.15.7:0/2524323916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6007c4b70 0x55f6007d0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.424+0000 7f3458420740 1 --2- 172.21.15.7:0/2524323916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6007bfa00 0x55f6007c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.424+0000 7f3458420740 1 --2- 172.21.15.7:0/2524323916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6007bf010 0x55f6007c8ff0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.424+0000 7f3458420740 1 -- 172.21.15.7:0/2524323916 >> 172.21.15.7:0/2524323916 conn(0x55f6007ba680 msgr2=0x55f6007b97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.424+0000 7f3432ff5640 0 client.30388 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.426+0000 7f3458420740 1 -- 172.21.15.7:0/2524323916 shutdown_connections 2024-06-10T01:31:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.426+0000 7f3458420740 1 -- 172.21.15.7:0/2524323916 wait complete. 2024-06-10T01:31:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.426+0000 7f3458420740 20 client.30388 trim_cache size 0 max 16384 2024-06-10T01:31:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.463+0000 7f3458420740 1 Processor -- start 2024-06-10T01:31:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.463+0000 7f3458420740 1 -- start start 2024-06-10T01:31:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.463+0000 7f3458420740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6007bf010 0x55f6007bdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.463+0000 7f3458420740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6007bfa00 0x55f6007d0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.463+0000 7f3458420740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6007c4b70 0x55f6007c8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.463+0000 7f3458420740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f34501c4560 con 0x55f6007bf010 2024-06-10T01:31:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.463+0000 7f3458420740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f34501c1dd0 con 0x55f6007c4b70 2024-06-10T01:31:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.463+0000 7f3458420740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f34501bfcd0 con 0x55f6007bfa00 2024-06-10T01:31:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.464+0000 7f344effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6007bf010 0x55f6007bdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.464+0000 7f344effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6007bf010 0x55f6007bdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56718/0 (socket says 172.21.15.7:56718) 2024-06-10T01:31:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.464+0000 7f344effd640 1 -- 172.21.15.7:0/250731638 learned_addr learned my addr 172.21.15.7:0/250731638 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.464+0000 7f344f7fe640 1 --2- 172.21.15.7:0/250731638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6007c4b70 0x55f6007c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.464+0000 7f344e7fc640 1 --2- 172.21.15.7:0/250731638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6007bfa00 0x55f6007d0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.464+0000 7f344e7fc640 1 -- 172.21.15.7:0/250731638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6007c4b70 msgr2=0x55f6007c8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.464+0000 7f344e7fc640 1 --2- 172.21.15.7:0/250731638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6007c4b70 0x55f6007c8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.464+0000 7f344e7fc640 1 -- 172.21.15.7:0/250731638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6007bf010 msgr2=0x55f6007bdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.464+0000 7f344e7fc640 1 --2- 172.21.15.7:0/250731638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6007bf010 0x55f6007bdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.464+0000 7f344e7fc640 1 -- 172.21.15.7:0/250731638 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6007d7ef0 con 0x55f6007bfa00 2024-06-10T01:31:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.465+0000 7f344e7fc640 1 --2- 172.21.15.7:0/250731638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6007bfa00 0x55f6007d0500 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x55f6007d80f0 tx=0x7f3444015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a0f5462809e8102 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.465+0000 7f3434ff9640 1 -- 172.21.15.7:0/250731638 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344400dc50 con 0x55f6007bfa00 2024-06-10T01:31:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.467+0000 7f3434ff9640 1 -- 172.21.15.7:0/250731638 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f344400ddf0 con 0x55f6007bfa00 2024-06-10T01:31:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.468+0000 7f3458420740 1 -- 172.21.15.7:0/250731638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6007bfa00 msgr2=0x55f6007d0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.468+0000 7f3458420740 1 --2- 172.21.15.7:0/250731638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6007bfa00 0x55f6007d0500 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x55f6007d80f0 tx=0x7f3444015000 comp rx=0 tx=0).stop 2024-06-10T01:31:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.475+0000 7f3458420740 1 -- 172.21.15.7:0/250731638 shutdown_connections 2024-06-10T01:31:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.475+0000 7f3458420740 1 --2- 172.21.15.7:0/250731638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6007c4b70 0x55f6007c8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.475+0000 7f3458420740 1 --2- 172.21.15.7:0/250731638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6007bfa00 0x55f6007d0500 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.475+0000 7f3458420740 1 --2- 172.21.15.7:0/250731638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6007bf010 0x55f6007bdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.475+0000 7f3458420740 1 -- 172.21.15.7:0/250731638 >> 172.21.15.7:0/250731638 conn(0x55f6007ba680 msgr2=0x55f6007bd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.478+0000 7f3458420740 1 -- 172.21.15.7:0/250731638 shutdown_connections 2024-06-10T01:31:11.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.478+0000 7f3458420740 1 -- 172.21.15.7:0/250731638 wait complete. 2024-06-10T01:31:11.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.479+0000 7f3458420740 1 -- start start 2024-06-10T01:31:11.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.479+0000 7f3458420740 1 Processor -- start 2024-06-10T01:31:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.479+0000 7f3458420740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6007bf010 0x55f6007cd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.479+0000 7f3458420740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6007bfa00 0x55f6007cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.479+0000 7f3458420740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6007c4b70 0x55f6007ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.479+0000 7f3458420740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f34501c0290 con 0x55f6007bfa00 2024-06-10T01:31:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.479+0000 7f3458420740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f34501c1700 con 0x55f6007bf010 2024-06-10T01:31:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.479+0000 7f3458420740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f34501c0960 con 0x55f6007c4b70 2024-06-10T01:31:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.480+0000 7f3458420740 10 client.0 _ll_register_callbacks cb 0x7f3448009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:31:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.480+0000 7f344effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6007bf010 0x55f6007cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.480+0000 7f344effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6007bf010 0x55f6007cd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:35068/0 (socket says 172.21.15.7:35068) 2024-06-10T01:31:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.480+0000 7f344effd640 1 -- 172.21.15.7:0/3396989506 learned_addr learned my addr 172.21.15.7:0/3396989506 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.480+0000 7f344e7fc640 1 --2- 172.21.15.7:0/3396989506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6007bfa00 0x55f6007cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.481+0000 7f344f7fe640 1 --2- 172.21.15.7:0/3396989506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6007c4b70 0x55f6007ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.481+0000 7f344e7fc640 1 -- 172.21.15.7:0/3396989506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6007c4b70 msgr2=0x55f6007ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.481+0000 7f344e7fc640 1 --2- 172.21.15.7:0/3396989506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6007c4b70 0x55f6007ced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.481+0000 7f344e7fc640 1 -- 172.21.15.7:0/3396989506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6007bf010 msgr2=0x55f6007cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.481+0000 7f344e7fc640 1 --2- 172.21.15.7:0/3396989506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6007bf010 0x55f6007cd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.481+0000 7f344e7fc640 1 -- 172.21.15.7:0/3396989506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6007ba4b0 con 0x55f6007bfa00 2024-06-10T01:31:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.482+0000 7f344e7fc640 1 --2- 172.21.15.7:0/3396989506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6007bfa00 0x55f6007cdd40 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f3444008010 tx=0x7f3444017c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39865d9d62bf6655 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.482+0000 7f3432ff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:31:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.482+0000 7f3432ff5640 1 -- 172.21.15.7:0/3396989506 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3444031070 con 0x55f6007bfa00 2024-06-10T01:31:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.482+0000 7f3458420740 1 -- 172.21.15.7:0/3396989506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f344400ddf0 con 0x55f6007bfa00 2024-06-10T01:31:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.483+0000 7f3432ff5640 1 -- 172.21.15.7:0/3396989506 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f3444004250 con 0x55f6007bfa00 2024-06-10T01:31:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.483+0000 7f3432ff5640 1 -- 172.21.15.7:0/3396989506 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f344403f050 con 0x55f6007bfa00 2024-06-10T01:31:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.483+0000 7f3458420740 10 client.21348 fetch_fsmap learned FSMap version 57 2024-06-10T01:31:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.483+0000 7f3458420740 1 -- 172.21.15.7:0/3396989506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55f6007d0480 con 0x55f6007bfa00 2024-06-10T01:31:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.483+0000 7f3432ff5640 1 -- 172.21.15.7:0/3396989506 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f344400ddf0 con 0x55f6007bfa00 2024-06-10T01:31:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.484+0000 7f3432ff5640 1 -- 172.21.15.7:0/3396989506 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7f3444024630 con 0x55f6007bfa00 2024-06-10T01:31:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.484+0000 7f3458420740 10 client.21348 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:31:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.484+0000 7f3458420740 10 client.21348 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:31:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.484+0000 7f3458420740 20 client.21348 populate_metadata read hostname 'smithi007' 2024-06-10T01:31:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.484+0000 7f3458420740 1 --2- 172.21.15.7:0/3396989506 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f6008404c0 0x55f6008428e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.484+0000 7f3458420740 4 client.21348 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:31:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.484+0000 7f3458420740 1 -- 172.21.15.7:0/3396989506 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f60072fc20 con 0x55f6008404c0 2024-06-10T01:31:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.484+0000 7f344effd640 1 --2- 172.21.15.7:0/3396989506 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f6008404c0 0x55f6008428e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.485+0000 7f344effd640 1 --2- 172.21.15.7:0/3396989506 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f6008404c0 0x55f6008428e0 crc :-1 s=READY pgs=558 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=396ef54761973b61 server_cookie=aa2a679ce34a061f in_seq=0 out_seq=0 2024-06-10T01:31:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:11.485+0000 7f3432ff5640 10 client.21348 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:11.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:11 smithi007 ceph-mon[32398]: pgmap v1189: 129 pgs: 129 active+clean; 694 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.0 MiB/s wr, 532 op/s 2024-06-10T01:31:11.572 INFO:tasks.workunit.client.0.smithi007.stdout: 10 279859 21.94 MB/sec execute 588 sec latency 89.713 ms 2024-06-10T01:31:11.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:11 smithi175 ceph-mon[36636]: pgmap v1189: 129 pgs: 129 active+clean; 694 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.0 MiB/s wr, 532 op/s 2024-06-10T01:31:11.822 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:31:11.826 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:11 smithi078 ceph-mon[36683]: pgmap v1189: 129 pgs: 129 active+clean; 694 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.0 MiB/s wr, 532 op/s 2024-06-10T01:31:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:12.459+0000 7f3432ff5640 1 -- 172.21.15.7:0/3396989506 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f60072fc20 con 0x55f6008404c0 2024-06-10T01:31:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:12.459+0000 7f3432ff5640 10 client.21348 handle_command_reply: tid=0 2024-06-10T01:31:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:12.459+0000 7f3458420740 1 client.21348 shutdown 2024-06-10T01:31:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:12.460+0000 7f3458420740 1 -- 172.21.15.7:0/3396989506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6007bfa00 msgr2=0x55f6007cdd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:12.460+0000 7f3458420740 1 --2- 172.21.15.7:0/3396989506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6007bfa00 0x55f6007cdd40 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f3444008010 tx=0x7f3444017c00 comp rx=0 tx=0).stop 2024-06-10T01:31:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:12.460+0000 7f3458420740 1 -- 172.21.15.7:0/3396989506 shutdown_connections 2024-06-10T01:31:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:12.460+0000 7f3458420740 1 --2- 172.21.15.7:0/3396989506 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55f6008404c0 0x55f6008428e0 crc :-1 s=READY pgs=558 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:12.460+0000 7f3458420740 1 --2- 172.21.15.7:0/3396989506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6007c4b70 0x55f6007ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:12.460+0000 7f3458420740 1 --2- 172.21.15.7:0/3396989506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6007bfa00 0x55f6007cdd40 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:12.460+0000 7f3458420740 1 --2- 172.21.15.7:0/3396989506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6007bf010 0x55f6007cd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:12.460+0000 7f3432ff5640 0 client.21348 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:12.460+0000 7f3458420740 1 -- 172.21.15.7:0/3396989506 >> 172.21.15.7:0/3396989506 conn(0x55f6007ba680 msgr2=0x55f6007b8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:12.460+0000 7f3458420740 1 -- 172.21.15.7:0/3396989506 shutdown_connections 2024-06-10T01:31:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:12.460+0000 7f3458420740 1 -- 172.21.15.7:0/3396989506 wait complete. 2024-06-10T01:31:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:12.460+0000 7f3458420740 20 client.21348 trim_cache size 0 max 16384 2024-06-10T01:31:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:31:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:31:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:31:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:12.461+0000 7f3455641640 1 -- 172.21.15.7:0/583538546 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f34380a2cc0 msgr2=0x7f34380a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:12.461+0000 7f3455641640 1 --2- 172.21.15.7:0/583538546 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f34380a2cc0 0x7f34380a5180 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3444006fd0 tx=0x7f3444009d90 comp rx=0 tx=0).stop 2024-06-10T01:31:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:12.461+0000 7f3455641640 1 -- 172.21.15.7:0/583538546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34500736e0 msgr2=0x7f345010fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:12.461+0000 7f3455641640 1 --2- 172.21.15.7:0/583538546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34500736e0 0x7f345010fe30 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f344002bd50 tx=0x7f3440030c80 comp rx=0 tx=0).stop 2024-06-10T01:31:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:12.461+0000 7f3455641640 1 -- 172.21.15.7:0/583538546 shutdown_connections 2024-06-10T01:31:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:12.461+0000 7f3455641640 1 --2- 172.21.15.7:0/583538546 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f34380a2cc0 0x7f34380a5180 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:12.461+0000 7f3455641640 1 --2- 172.21.15.7:0/583538546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34500736e0 0x7f345010fe30 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:12.461+0000 7f3455641640 1 --2- 172.21.15.7:0/583538546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3450101270 0x7f345006e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:12.461+0000 7f3455641640 1 --2- 172.21.15.7:0/583538546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3450100080 0x7f345006e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:12.461+0000 7f3455641640 1 -- 172.21.15.7:0/583538546 >> 172.21.15.7:0/583538546 conn(0x7f34500fb610 msgr2=0x7f34500fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:12.461+0000 7f3455641640 1 -- 172.21.15.7:0/583538546 shutdown_connections 2024-06-10T01:31:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:12.461+0000 7f3455641640 1 -- 172.21.15.7:0/583538546 wait complete. 2024-06-10T01:31:12.572 INFO:tasks.workunit.client.0.smithi007.stdout: 10 280109 21.92 MB/sec execute 589 sec latency 564.642 ms 2024-06-10T01:31:13.572 INFO:tasks.workunit.client.0.smithi007.stdout: 10 280636 21.94 MB/sec execute 590 sec latency 112.255 ms 2024-06-10T01:31:13.597 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:13 smithi007 ceph-mon[32398]: pgmap v1190: 129 pgs: 129 active+clean; 702 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 7.9 MiB/s wr, 640 op/s 2024-06-10T01:31:13.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:13 smithi175 ceph-mon[36636]: pgmap v1190: 129 pgs: 129 active+clean; 702 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 7.9 MiB/s wr, 640 op/s 2024-06-10T01:31:13.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:13 smithi078 ceph-mon[36683]: pgmap v1190: 129 pgs: 129 active+clean; 702 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 7.9 MiB/s wr, 640 op/s 2024-06-10T01:31:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.026+0000 7f4effd24700 1 -- 172.21.15.7:0/510212831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ef81046d0 msgr2=0x7f4ef8104b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.026+0000 7f4effd24700 1 --2- 172.21.15.7:0/510212831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ef81046d0 0x7f4ef8104b50 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f4ee80098f0 tx=0x7f4ee8009c00 comp rx=0 tx=0).stop 2024-06-10T01:31:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.026+0000 7f4effd24700 1 -- 172.21.15.7:0/510212831 shutdown_connections 2024-06-10T01:31:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.026+0000 7f4effd24700 1 --2- 172.21.15.7:0/510212831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ef8105090 0x7f4ef81115a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.026+0000 7f4effd24700 1 --2- 172.21.15.7:0/510212831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ef810a6d0 0x7f4ef810aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.026+0000 7f4effd24700 1 --2- 172.21.15.7:0/510212831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ef81046d0 0x7f4ef8104b50 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.026+0000 7f4effd24700 1 -- 172.21.15.7:0/510212831 >> 172.21.15.7:0/510212831 conn(0x7f4ef81001c0 msgr2=0x7f4ef81025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.026+0000 7f4effd24700 1 -- 172.21.15.7:0/510212831 shutdown_connections 2024-06-10T01:31:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.026+0000 7f4effd24700 1 -- 172.21.15.7:0/510212831 wait complete. 2024-06-10T01:31:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.027+0000 7f4effd24700 1 Processor -- start 2024-06-10T01:31:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.027+0000 7f4effd24700 1 -- start start 2024-06-10T01:31:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.027+0000 7f4effd24700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ef81046d0 0x7f4ef819aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.027+0000 7f4effd24700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ef8105090 0x7f4ef819b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.027+0000 7f4effd24700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ef810a6d0 0x7f4ef819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.027+0000 7f4effd24700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef8114a60 con 0x7f4ef810a6d0 2024-06-10T01:31:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.027+0000 7f4effd24700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef81148e0 con 0x7f4ef8105090 2024-06-10T01:31:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.027+0000 7f4effd24700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef8114be0 con 0x7f4ef81046d0 2024-06-10T01:31:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.027+0000 7f4efe2c1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ef810a6d0 0x7f4ef819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.027+0000 7f4efd2bf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ef8105090 0x7f4ef819b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.027+0000 7f4efd2bf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ef8105090 0x7f4ef819b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:37082/0 (socket says 172.21.15.7:37082) 2024-06-10T01:31:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.027+0000 7f4efd2bf700 1 -- 172.21.15.7:0/2243227916 learned_addr learned my addr 172.21.15.7:0/2243227916 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.028+0000 7f4efd2bf700 1 -- 172.21.15.7:0/2243227916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ef81046d0 msgr2=0x7f4ef819aed0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-10T01:31:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.028+0000 7f4efd2bf700 1 --2- 172.21.15.7:0/2243227916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ef81046d0 0x7f4ef819aed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.028+0000 7f4efd2bf700 1 -- 172.21.15.7:0/2243227916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ef810a6d0 msgr2=0x7f4ef819f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.028+0000 7f4efd2bf700 1 --2- 172.21.15.7:0/2243227916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ef810a6d0 0x7f4ef819f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.028+0000 7f4efd2bf700 1 -- 172.21.15.7:0/2243227916 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef819fee0 con 0x7f4ef8105090 2024-06-10T01:31:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.028+0000 7f4efd2bf700 1 --2- 172.21.15.7:0/2243227916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ef8105090 0x7f4ef819b410 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8009230 tx=0x7f4ee800fb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.028+0000 7f4eeeffd700 1 -- 172.21.15.7:0/2243227916 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee8005360 con 0x7f4ef8105090 2024-06-10T01:31:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.028+0000 7f4eeeffd700 1 -- 172.21.15.7:0/2243227916 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ee8005500 con 0x7f4ef8105090 2024-06-10T01:31:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.028+0000 7f4effd24700 1 -- 172.21.15.7:0/2243227916 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ef81a0170 con 0x7f4ef8105090 2024-06-10T01:31:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.028+0000 7f4eeeffd700 1 -- 172.21.15.7:0/2243227916 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee8003cc0 con 0x7f4ef8105090 2024-06-10T01:31:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.028+0000 7f4effd24700 1 -- 172.21.15.7:0/2243227916 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ef81a7a50 con 0x7f4ef8105090 2024-06-10T01:31:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.031+0000 7f4eeeffd700 1 -- 172.21.15.7:0/2243227916 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f4ee8029060 con 0x7f4ef8105090 2024-06-10T01:31:14.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.031+0000 7f4eeeffd700 1 --2- 172.21.15.7:0/2243227916 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4ee40a2ca0 0x7f4ee40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:14.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.032+0000 7f4efdac0700 1 --2- 172.21.15.7:0/2243227916 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4ee40a2ca0 0x7f4ee40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:14.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.032+0000 7f4eeeffd700 1 -- 172.21.15.7:0/2243227916 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f4ee80cfd00 con 0x7f4ef8105090 2024-06-10T01:31:14.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.032+0000 7f4efdac0700 1 --2- 172.21.15.7:0/2243227916 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4ee40a2ca0 0x7f4ee40a5160 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400b990 tx=0x7f4ef400b370 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:14.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.032+0000 7f4f0a5a7b80 1 Processor -- start 2024-06-10T01:31:14.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.032+0000 7f4f0a5a7b80 1 -- start start 2024-06-10T01:31:14.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.033+0000 7f4f0a5a7b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f5454cf60 0x558f5454d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:14.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.033+0000 7f4f0a5a7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f5454d880 0x558f54547260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.033+0000 7f4f0a5a7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f545477a0 0x558f54547bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.033+0000 7f4f0a5a7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558f54439f60 con 0x558f545477a0 2024-06-10T01:31:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.033+0000 7f4f0a5a7b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558f54498850 con 0x558f5454cf60 2024-06-10T01:31:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.033+0000 7f4f0a5a7b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558f54400460 con 0x558f5454d880 2024-06-10T01:31:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.033+0000 7f4efdac0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f5454cf60 0x558f5454d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.033+0000 7f4efe2c1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f545477a0 0x558f54547bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.033+0000 7f4efdac0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f5454cf60 0x558f5454d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:37098/0 (socket says 172.21.15.7:37098) 2024-06-10T01:31:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.033+0000 7f4efdac0700 1 -- 172.21.15.7:0/2388971910 learned_addr learned my addr 172.21.15.7:0/2388971910 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.033+0000 7f4efd2bf700 1 --2- 172.21.15.7:0/2388971910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f5454d880 0x558f54547260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.033+0000 7f4efdac0700 1 -- 172.21.15.7:0/2388971910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f5454d880 msgr2=0x558f54547260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.033+0000 7f4efdac0700 1 --2- 172.21.15.7:0/2388971910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f5454d880 0x558f54547260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.033+0000 7f4efdac0700 1 -- 172.21.15.7:0/2388971910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f545477a0 msgr2=0x558f54547bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.033+0000 7f4efdac0700 1 --2- 172.21.15.7:0/2388971910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f545477a0 0x558f54547bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.033+0000 7f4efdac0700 1 -- 172.21.15.7:0/2388971910 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f543e78d0 con 0x558f5454cf60 2024-06-10T01:31:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.033+0000 7f4efe2c1700 1 --2- 172.21.15.7:0/2388971910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f545477a0 0x558f54547bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:31:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.033+0000 7f4efd2bf700 1 --2- 172.21.15.7:0/2388971910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f5454d880 0x558f54547260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:31:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.033+0000 7f4efdac0700 1 --2- 172.21.15.7:0/2388971910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f5454cf60 0x558f5454d340 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400db60 tx=0x7f4ef4010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c173fff94787bfe6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.034+0000 7f4ee3fff700 1 -- 172.21.15.7:0/2388971910 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4010af0 con 0x558f5454cf60 2024-06-10T01:31:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.034+0000 7f4ee3fff700 1 -- 172.21.15.7:0/2388971910 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ef4022070 con 0x558f5454cf60 2024-06-10T01:31:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.034+0000 7f4ee3fff700 1 -- 172.21.15.7:0/2388971910 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef400f3c0 con 0x558f5454cf60 2024-06-10T01:31:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.034+0000 7f4f0a5a7b80 1 -- 172.21.15.7:0/2388971910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f5454cf60 msgr2=0x558f5454d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.034+0000 7f4f0a5a7b80 1 --2- 172.21.15.7:0/2388971910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f5454cf60 0x558f5454d340 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400db60 tx=0x7f4ef4010000 comp rx=0 tx=0).stop 2024-06-10T01:31:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.035+0000 7f4f0a5a7b80 1 -- 172.21.15.7:0/2388971910 shutdown_connections 2024-06-10T01:31:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.035+0000 7f4f0a5a7b80 1 --2- 172.21.15.7:0/2388971910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f545477a0 0x558f54547bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.035+0000 7f4f0a5a7b80 1 --2- 172.21.15.7:0/2388971910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f5454cf60 0x558f5454d340 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.035+0000 7f4f0a5a7b80 1 --2- 172.21.15.7:0/2388971910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f5454d880 0x558f54547260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.035+0000 7f4f0a5a7b80 1 -- 172.21.15.7:0/2388971910 >> 172.21.15.7:0/2388971910 conn(0x558f54474080 msgr2=0x558f54542220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.035+0000 7f4f0a5a7b80 1 -- 172.21.15.7:0/2388971910 shutdown_connections 2024-06-10T01:31:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.035+0000 7f4f0a5a7b80 1 -- 172.21.15.7:0/2388971910 wait complete. 2024-06-10T01:31:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.035+0000 7f4f0a5a7b80 1 -- start start 2024-06-10T01:31:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.035+0000 7f4f0a5a7b80 1 Processor -- start 2024-06-10T01:31:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.036+0000 7f4f0a5a7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f545477a0 0x558f54553630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.036+0000 7f4f0a5a7b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f5454cf60 0x558f54550930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.036+0000 7f4f0a5a7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f5454d880 0x558f54550e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.036+0000 7f4f0a5a7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558f54438fc0 con 0x558f5454d880 2024-06-10T01:31:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.036+0000 7f4f0a5a7b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558f54494810 con 0x558f5454cf60 2024-06-10T01:31:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.036+0000 7f4f0a5a7b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558f54554200 con 0x558f545477a0 2024-06-10T01:31:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.036+0000 7f4f0a5a7b80 10 client.0 _ll_register_callbacks cb 0x558f5442e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:31:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.036+0000 7f4efe2c1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f5454d880 0x558f54550e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.036+0000 7f4efe2c1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f5454d880 0x558f54550e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33798/0 (socket says 172.21.15.7:33798) 2024-06-10T01:31:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.036+0000 7f4efe2c1700 1 -- 172.21.15.7:0/3132696275 learned_addr learned my addr 172.21.15.7:0/3132696275 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.036+0000 7f4efd2bf700 1 --2- 172.21.15.7:0/3132696275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f5454cf60 0x558f54550930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.036+0000 7f4efdac0700 1 --2- 172.21.15.7:0/3132696275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f545477a0 0x558f54553630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.036+0000 7f4efe2c1700 1 -- 172.21.15.7:0/3132696275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f545477a0 msgr2=0x558f54553630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.036+0000 7f4efe2c1700 1 --2- 172.21.15.7:0/3132696275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f545477a0 0x558f54553630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.036+0000 7f4efe2c1700 1 -- 172.21.15.7:0/3132696275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f5454cf60 msgr2=0x558f54550930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.036+0000 7f4efe2c1700 1 --2- 172.21.15.7:0/3132696275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f5454cf60 0x558f54550930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.036+0000 7f4efe2c1700 1 -- 172.21.15.7:0/3132696275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f5445e750 con 0x558f5454d880 2024-06-10T01:31:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.036+0000 7f4efd2bf700 1 --2- 172.21.15.7:0/3132696275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f5454cf60 0x558f54550930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:31:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.037+0000 7f4efe2c1700 1 --2- 172.21.15.7:0/3132696275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f5454d880 0x558f54550e70 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0007c90 tx=0x7f4ef00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df4540d0fa50e5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.037+0000 7f4ee17fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:31:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.037+0000 7f4ee17fa700 1 -- 172.21.15.7:0/3132696275 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0004db0 con 0x558f5454d880 2024-06-10T01:31:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.037+0000 7f4ee17fa700 1 -- 172.21.15.7:0/3132696275 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f4ef0014a60 con 0x558f5454d880 2024-06-10T01:31:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.037+0000 7f4f0a5a7b80 1 -- 172.21.15.7:0/3132696275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ef4022070 con 0x558f5454d880 2024-06-10T01:31:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.037+0000 7f4ee17fa700 1 -- 172.21.15.7:0/3132696275 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f4ef0027050 con 0x558f5454d880 2024-06-10T01:31:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.037+0000 7f4f0a5a7b80 10 client.21360 fetch_fsmap learned FSMap version 57 2024-06-10T01:31:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.037+0000 7f4f0a5a7b80 1 -- 172.21.15.7:0/3132696275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x558f54567200 con 0x558f5454d880 2024-06-10T01:31:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.037+0000 7f4ee17fa700 1 -- 172.21.15.7:0/3132696275 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ef4022070 con 0x558f5454d880 2024-06-10T01:31:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.037+0000 7f4ee17fa700 1 -- 172.21.15.7:0/3132696275 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7f4ef0016400 con 0x558f5454d880 2024-06-10T01:31:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.038+0000 7f4f0a5a7b80 10 client.21360 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:31:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.038+0000 7f4f0a5a7b80 10 client.21360 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:31:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.038+0000 7f4f0a5a7b80 20 client.21360 populate_metadata read hostname 'smithi007' 2024-06-10T01:31:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.038+0000 7f4f0a5a7b80 1 --2- 172.21.15.7:0/3132696275 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x558f545697e0 0x558f5456bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.038+0000 7f4f0a5a7b80 4 client.21360 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:31:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.038+0000 7f4f0a5a7b80 1 -- 172.21.15.7:0/3132696275 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f544a5d50 con 0x558f545697e0 2024-06-10T01:31:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.038+0000 7f4efdac0700 1 --2- 172.21.15.7:0/3132696275 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x558f545697e0 0x558f5456bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.038+0000 7f4ee17fa700 10 client.21360 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:31:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.038+0000 7f4efdac0700 1 --2- 172.21.15.7:0/3132696275 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x558f545697e0 0x558f5456bbd0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1cee0686aa25575a server_cookie=154549681a97c788 in_seq=0 out_seq=0 2024-06-10T01:31:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.041+0000 7f4ee17fa700 1 -- 172.21.15.7:0/3132696275 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558f544a5d50 con 0x558f545697e0 2024-06-10T01:31:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.041+0000 7f4ee17fa700 10 client.21360 handle_command_reply: tid=0 2024-06-10T01:31:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.041+0000 7f4f0a5a7b80 1 client.21360 shutdown 2024-06-10T01:31:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.041+0000 7f4f0a5a7b80 1 -- 172.21.15.7:0/3132696275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f5454d880 msgr2=0x558f54550e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.041+0000 7f4f0a5a7b80 1 --2- 172.21.15.7:0/3132696275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f5454d880 0x558f54550e70 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0007c90 tx=0x7f4ef00095c0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.041+0000 7f4f0a5a7b80 1 -- 172.21.15.7:0/3132696275 shutdown_connections 2024-06-10T01:31:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.041+0000 7f4f0a5a7b80 1 --2- 172.21.15.7:0/3132696275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f5454d880 0x558f54550e70 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.041+0000 7f4f0a5a7b80 1 --2- 172.21.15.7:0/3132696275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f545477a0 0x558f54553630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.041+0000 7f4f0a5a7b80 1 --2- 172.21.15.7:0/3132696275 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x558f545697e0 0x558f5456bbd0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.041+0000 7f4ee17fa700 0 client.21360 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:31:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.041+0000 7f4f0a5a7b80 1 --2- 172.21.15.7:0/3132696275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f5454cf60 0x558f54550930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.041+0000 7f4f0a5a7b80 1 -- 172.21.15.7:0/3132696275 >> 172.21.15.7:0/3132696275 conn(0x558f54474080 msgr2=0x558f54541700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.041+0000 7f4f0a5a7b80 1 -- 172.21.15.7:0/3132696275 shutdown_connections 2024-06-10T01:31:14.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.041+0000 7f4f0a5a7b80 1 -- 172.21.15.7:0/3132696275 wait complete. 2024-06-10T01:31:14.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.041+0000 7f4f0a5a7b80 20 client.21360 trim_cache size 0 max 16384 2024-06-10T01:31:14.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.058+0000 7f4f0a5a7b80 1 Processor -- start 2024-06-10T01:31:14.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.058+0000 7f4f0a5a7b80 1 -- start start 2024-06-10T01:31:14.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.058+0000 7f4f0a5a7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f545477a0 0x558f54552870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:14.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.058+0000 7f4f0a5a7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f5454cf60 0x558f54541e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:14.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.058+0000 7f4f0a5a7b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f5454d880 0x558f545423c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:14.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.058+0000 7f4f0a5a7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef81ae950 con 0x558f5454cf60 2024-06-10T01:31:14.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.058+0000 7f4f0a5a7b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef81ac180 con 0x558f5454d880 2024-06-10T01:31:14.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.058+0000 7f4f0a5a7b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef81aa050 con 0x558f545477a0 2024-06-10T01:31:14.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.058+0000 7f4efd2bf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f5454cf60 0x558f54541e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:14.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.058+0000 7f4efd2bf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f5454cf60 0x558f54541e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33802/0 (socket says 172.21.15.7:33802) 2024-06-10T01:31:14.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.058+0000 7f4efdac0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f545477a0 0x558f54552870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:14.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.058+0000 7f4efe2c1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f5454d880 0x558f545423c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:14.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.058+0000 7f4efd2bf700 1 -- 172.21.15.7:0/355865618 learned_addr learned my addr 172.21.15.7:0/355865618 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.059+0000 7f4efdac0700 1 -- 172.21.15.7:0/355865618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f5454d880 msgr2=0x558f545423c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.059+0000 7f4efdac0700 1 --2- 172.21.15.7:0/355865618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f5454d880 0x558f545423c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.059+0000 7f4efdac0700 1 -- 172.21.15.7:0/355865618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f5454cf60 msgr2=0x558f54541e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.059+0000 7f4efdac0700 1 --2- 172.21.15.7:0/355865618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f5454cf60 0x558f54541e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.059+0000 7f4efdac0700 1 -- 172.21.15.7:0/355865618 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f545439f0 con 0x558f545477a0 2024-06-10T01:31:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.059+0000 7f4efe2c1700 1 --2- 172.21.15.7:0/355865618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f5454d880 0x558f545423c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:31:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.059+0000 7f4efdac0700 1 --2- 172.21.15.7:0/355865618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f545477a0 0x558f54552870 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400d5c0 tx=0x7f4ef4005fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b071ae052d0bee6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.059+0000 7f4ee37fe700 1 -- 172.21.15.7:0/355865618 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef401ddb0 con 0x558f545477a0 2024-06-10T01:31:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.059+0000 7f4ee37fe700 1 -- 172.21.15.7:0/355865618 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ef4022050 con 0x558f545477a0 2024-06-10T01:31:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.059+0000 7f4ee37fe700 1 -- 172.21.15.7:0/355865618 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4005040 con 0x558f545477a0 2024-06-10T01:31:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.060+0000 7f4f0a5a7b80 1 -- 172.21.15.7:0/355865618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f545477a0 msgr2=0x558f54552870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.060+0000 7f4f0a5a7b80 1 --2- 172.21.15.7:0/355865618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f545477a0 0x558f54552870 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400d5c0 tx=0x7f4ef4005fa0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.060+0000 7f4f0a5a7b80 1 -- 172.21.15.7:0/355865618 shutdown_connections 2024-06-10T01:31:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.060+0000 7f4f0a5a7b80 1 --2- 172.21.15.7:0/355865618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f5454d880 0x558f545423c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.060+0000 7f4f0a5a7b80 1 --2- 172.21.15.7:0/355865618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f545477a0 0x558f54552870 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.060+0000 7f4f0a5a7b80 1 --2- 172.21.15.7:0/355865618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f5454cf60 0x558f54541e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.060+0000 7f4f0a5a7b80 1 -- 172.21.15.7:0/355865618 >> 172.21.15.7:0/355865618 conn(0x558f54474080 msgr2=0x558f54552160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.060+0000 7f4f0a5a7b80 1 -- 172.21.15.7:0/355865618 shutdown_connections 2024-06-10T01:31:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.060+0000 7f4f0a5a7b80 1 -- 172.21.15.7:0/355865618 wait complete. 2024-06-10T01:31:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.061+0000 7f4f0a5a7b80 1 -- start start 2024-06-10T01:31:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.061+0000 7f4f0a5a7b80 1 Processor -- start 2024-06-10T01:31:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.061+0000 7f4f0a5a7b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f545477a0 0x558f5455f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.061+0000 7f4f0a5a7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f5454cf60 0x558f5455f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.061+0000 7f4f0a5a7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f5454d880 0x558f545d2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.061+0000 7f4f0a5a7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef81aa640 con 0x558f5454d880 2024-06-10T01:31:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.061+0000 7f4f0a5a7b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef81abab0 con 0x558f545477a0 2024-06-10T01:31:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.061+0000 7f4f0a5a7b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef81aad10 con 0x558f5454cf60 2024-06-10T01:31:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.061+0000 7f4f0a5a7b80 10 client.0 _ll_register_callbacks cb 0x558f5442e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:31:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.061+0000 7f4efdac0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f545477a0 0x558f5455f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.061+0000 7f4efdac0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f545477a0 0x558f5455f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:37124/0 (socket says 172.21.15.7:37124) 2024-06-10T01:31:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.061+0000 7f4efdac0700 1 -- 172.21.15.7:0/1364438255 learned_addr learned my addr 172.21.15.7:0/1364438255 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.062+0000 7f4efd2bf700 1 --2- 172.21.15.7:0/1364438255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f5454cf60 0x558f5455f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.062+0000 7f4efe2c1700 1 --2- 172.21.15.7:0/1364438255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f5454d880 0x558f545d2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.062+0000 7f4efdac0700 1 -- 172.21.15.7:0/1364438255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f5454cf60 msgr2=0x558f5455f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.062+0000 7f4efdac0700 1 --2- 172.21.15.7:0/1364438255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f5454cf60 0x558f5455f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.062+0000 7f4efdac0700 1 -- 172.21.15.7:0/1364438255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f5454d880 msgr2=0x558f545d2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.062+0000 7f4efdac0700 1 --2- 172.21.15.7:0/1364438255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f5454d880 0x558f545d2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.062+0000 7f4efdac0700 1 -- 172.21.15.7:0/1364438255 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f545d2990 con 0x558f545477a0 2024-06-10T01:31:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.062+0000 7f4efe2c1700 1 --2- 172.21.15.7:0/1364438255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f5454d880 0x558f545d2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:31:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.062+0000 7f4efdac0700 1 --2- 172.21.15.7:0/1364438255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f545477a0 0x558f5455f300 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4005d50 tx=0x7f4ef401dee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c38bc9410e68cf8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.062+0000 7f4ee17fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:31:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.062+0000 7f4ee17fa700 1 -- 172.21.15.7:0/1364438255 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef403f060 con 0x558f545477a0 2024-06-10T01:31:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.062+0000 7f4f0a5a7b80 1 -- 172.21.15.7:0/1364438255 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ef4022050 con 0x558f545477a0 2024-06-10T01:31:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.062+0000 7f4ee17fa700 1 -- 172.21.15.7:0/1364438255 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f4ef401d760 con 0x558f545477a0 2024-06-10T01:31:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.062+0000 7f4ee17fa700 1 -- 172.21.15.7:0/1364438255 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f4ef4046050 con 0x558f545477a0 2024-06-10T01:31:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.062+0000 7f4f0a5a7b80 10 client.30530 fetch_fsmap learned FSMap version 57 2024-06-10T01:31:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.062+0000 7f4f0a5a7b80 1 -- 172.21.15.7:0/1364438255 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x558f545d3b90 con 0x558f545477a0 2024-06-10T01:31:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.063+0000 7f4ee17fa700 1 -- 172.21.15.7:0/1364438255 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ef4022050 con 0x558f545477a0 2024-06-10T01:31:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.063+0000 7f4ee17fa700 1 -- 172.21.15.7:0/1364438255 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7f4ef40054b0 con 0x558f545477a0 2024-06-10T01:31:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.063+0000 7f4f0a5a7b80 10 client.30530 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:31:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.063+0000 7f4f0a5a7b80 10 client.30530 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:31:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.063+0000 7f4f0a5a7b80 20 client.30530 populate_metadata read hostname 'smithi007' 2024-06-10T01:31:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.063+0000 7f4f0a5a7b80 1 --2- 172.21.15.7:0/1364438255 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x558f545d4050 0x558f545d6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.063+0000 7f4f0a5a7b80 4 client.30530 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:31:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.063+0000 7f4f0a5a7b80 1 -- 172.21.15.7:0/1364438255 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f544a02d0 con 0x558f545d4050 2024-06-10T01:31:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.063+0000 7f4efd2bf700 1 --2- 172.21.15.7:0/1364438255 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x558f545d4050 0x558f545d6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.064+0000 7f4ee17fa700 10 client.30530 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:31:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.064+0000 7f4efd2bf700 1 --2- 172.21.15.7:0/1364438255 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x558f545d4050 0x558f545d6440 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e2112dfa2381f24a server_cookie=8ed36c667998272d in_seq=0 out_seq=0 2024-06-10T01:31:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.067+0000 7f4ee17fa700 1 -- 172.21.15.7:0/1364438255 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f544a02d0 con 0x558f545d4050 2024-06-10T01:31:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.067+0000 7f4ee17fa700 10 client.30530 handle_command_reply: tid=0 2024-06-10T01:31:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.067+0000 7f4f0a5a7b80 1 client.30530 shutdown 2024-06-10T01:31:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.067+0000 7f4f0a5a7b80 1 -- 172.21.15.7:0/1364438255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f545477a0 msgr2=0x558f5455f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.067+0000 7f4f0a5a7b80 1 --2- 172.21.15.7:0/1364438255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f545477a0 0x558f5455f300 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4005d50 tx=0x7f4ef401dee0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.067+0000 7f4f0a5a7b80 1 -- 172.21.15.7:0/1364438255 shutdown_connections 2024-06-10T01:31:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.067+0000 7f4f0a5a7b80 1 --2- 172.21.15.7:0/1364438255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f5454d880 0x558f545d2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.067+0000 7f4f0a5a7b80 1 --2- 172.21.15.7:0/1364438255 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x558f545d4050 0x558f545d6440 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.067+0000 7f4f0a5a7b80 1 --2- 172.21.15.7:0/1364438255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f545477a0 0x558f5455f300 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.067+0000 7f4f0a5a7b80 1 --2- 172.21.15.7:0/1364438255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f5454cf60 0x558f5455f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.067+0000 7f4ee17fa700 0 client.30530 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:31:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.067+0000 7f4f0a5a7b80 1 -- 172.21.15.7:0/1364438255 >> 172.21.15.7:0/1364438255 conn(0x558f54474080 msgr2=0x558f5454e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.067+0000 7f4f0a5a7b80 1 -- 172.21.15.7:0/1364438255 shutdown_connections 2024-06-10T01:31:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.067+0000 7f4f0a5a7b80 1 -- 172.21.15.7:0/1364438255 wait complete. 2024-06-10T01:31:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.067+0000 7f4f0a5a7b80 20 client.30530 trim_cache size 0 max 16384 2024-06-10T01:31:14.062 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:31:14.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.068+0000 7f4effd24700 1 -- 172.21.15.7:0/2243227916 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4ee40a2ca0 msgr2=0x7f4ee40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:14.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.068+0000 7f4effd24700 1 --2- 172.21.15.7:0/2243227916 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4ee40a2ca0 0x7f4ee40a5160 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400b990 tx=0x7f4ef400b370 comp rx=0 tx=0).stop 2024-06-10T01:31:14.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.068+0000 7f4effd24700 1 -- 172.21.15.7:0/2243227916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ef8105090 msgr2=0x7f4ef819b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:14.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.068+0000 7f4effd24700 1 --2- 172.21.15.7:0/2243227916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ef8105090 0x7f4ef819b410 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8009230 tx=0x7f4ee800fb40 comp rx=0 tx=0).stop 2024-06-10T01:31:14.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.068+0000 7f4effd24700 1 -- 172.21.15.7:0/2243227916 shutdown_connections 2024-06-10T01:31:14.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.068+0000 7f4effd24700 1 --2- 172.21.15.7:0/2243227916 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4ee40a2ca0 0x7f4ee40a5160 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.068+0000 7f4effd24700 1 --2- 172.21.15.7:0/2243227916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ef810a6d0 0x7f4ef819f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.068+0000 7f4effd24700 1 --2- 172.21.15.7:0/2243227916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ef81046d0 0x7f4ef819aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.068+0000 7f4effd24700 1 --2- 172.21.15.7:0/2243227916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ef8105090 0x7f4ef819b410 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:14.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.068+0000 7f4effd24700 1 -- 172.21.15.7:0/2243227916 >> 172.21.15.7:0/2243227916 conn(0x7f4ef81001c0 msgr2=0x7f4ef8100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:14.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.068+0000 7f4effd24700 1 -- 172.21.15.7:0/2243227916 shutdown_connections 2024-06-10T01:31:14.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:14.068+0000 7f4effd24700 1 -- 172.21.15.7:0/2243227916 wait complete. 2024-06-10T01:31:14.572 INFO:tasks.workunit.client.0.smithi007.stdout: 10 281201 21.94 MB/sec execute 591 sec latency 75.954 ms 2024-06-10T01:31:14.870 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:31:15.457 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:31:15.572 INFO:tasks.workunit.client.0.smithi007.stdout: 10 281434 21.93 MB/sec execute 592 sec latency 513.302 ms 2024-06-10T01:31:15.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:15 smithi175 ceph-mon[36636]: pgmap v1191: 129 pgs: 129 active+clean; 702 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.6 MiB/s wr, 483 op/s 2024-06-10T01:31:15.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:15 smithi007 ceph-mon[32398]: pgmap v1191: 129 pgs: 129 active+clean; 702 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.6 MiB/s wr, 483 op/s 2024-06-10T01:31:15.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:15 smithi078 ceph-mon[36683]: pgmap v1191: 129 pgs: 129 active+clean; 702 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.6 MiB/s wr, 483 op/s 2024-06-10T01:31:16.572 INFO:tasks.workunit.client.0.smithi007.stdout: 10 281682 21.91 MB/sec execute 593 sec latency 174.704 ms 2024-06-10T01:31:17.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.472+0000 7f3f0409f700 1 -- 172.21.15.7:0/3229236934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3efc101d50 msgr2=0x7f3efc10e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:17.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.472+0000 7f3f0409f700 1 --2- 172.21.15.7:0/3229236934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3efc101d50 0x7f3efc10e240 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009f90 tx=0x7f3ef400b330 comp rx=0 tx=0).stop 2024-06-10T01:31:17.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.472+0000 7f3f0409f700 1 -- 172.21.15.7:0/3229236934 shutdown_connections 2024-06-10T01:31:17.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.472+0000 7f3f0409f700 1 --2- 172.21.15.7:0/3229236934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3efc101d50 0x7f3efc10e240 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.472+0000 7f3f0409f700 1 --2- 172.21.15.7:0/3229236934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3efc107400 0x7f3efc1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.472+0000 7f3f0409f700 1 --2- 172.21.15.7:0/3229236934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3efc101390 0x7f3efc101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.472+0000 7f3f0409f700 1 -- 172.21.15.7:0/3229236934 >> 172.21.15.7:0/3229236934 conn(0x7f3efc078340 msgr2=0x7f3efc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:17.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.473+0000 7f3f0409f700 1 -- 172.21.15.7:0/3229236934 shutdown_connections 2024-06-10T01:31:17.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.473+0000 7f3f0409f700 1 -- 172.21.15.7:0/3229236934 wait complete. 2024-06-10T01:31:17.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.473+0000 7f3f0409f700 1 Processor -- start 2024-06-10T01:31:17.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.473+0000 7f3f0409f700 1 -- start start 2024-06-10T01:31:17.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.473+0000 7f3f0409f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3efc101390 0x7f3efc19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:17.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.473+0000 7f3f0409f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3efc101d50 0x7f3efc19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:17.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.473+0000 7f3f0409f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3efc107400 0x7f3efc19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:17.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.474+0000 7f3f0409f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc1105a0 con 0x7f3efc107400 2024-06-10T01:31:17.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.474+0000 7f3f0409f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc110420 con 0x7f3efc101d50 2024-06-10T01:31:17.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.474+0000 7f3f0409f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc110720 con 0x7f3efc101390 2024-06-10T01:31:17.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.474+0000 7f3f01e3b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3efc101390 0x7f3efc19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:17.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.474+0000 7f3f0163a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3efc101d50 0x7f3efc19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:17.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.474+0000 7f3f0163a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3efc101d50 0x7f3efc19b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:37138/0 (socket says 172.21.15.7:37138) 2024-06-10T01:31:17.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.474+0000 7f3f01e3b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3efc101390 0x7f3efc19ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:60526/0 (socket says 172.21.15.7:60526) 2024-06-10T01:31:17.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.474+0000 7f3f0263c700 1 --2- 172.21.15.7:0/1520811602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3efc107400 0x7f3efc19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:17.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.474+0000 7f3f0163a700 1 -- 172.21.15.7:0/1520811602 learned_addr learned my addr 172.21.15.7:0/1520811602 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:17.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.474+0000 7f3f01e3b700 1 -- 172.21.15.7:0/1520811602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3efc101d50 msgr2=0x7f3efc19b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:17.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.474+0000 7f3f01e3b700 1 --2- 172.21.15.7:0/1520811602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3efc101d50 0x7f3efc19b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.474+0000 7f3f01e3b700 1 -- 172.21.15.7:0/1520811602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3efc107400 msgr2=0x7f3efc19f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:17.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.474+0000 7f3f01e3b700 1 --2- 172.21.15.7:0/1520811602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3efc107400 0x7f3efc19f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.474+0000 7f3f01e3b700 1 -- 172.21.15.7:0/1520811602 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3efc19fe20 con 0x7f3efc101390 2024-06-10T01:31:17.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.474+0000 7f3f0263c700 1 --2- 172.21.15.7:0/1520811602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3efc107400 0x7f3efc19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:31:17.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.474+0000 7f3f01e3b700 1 --2- 172.21.15.7:0/1520811602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3efc101390 0x7f3efc19ae10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3ef800eb90 tx=0x7f3ef800eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:17.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.475+0000 7f3ef2ffd700 1 -- 172.21.15.7:0/1520811602 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef800cc90 con 0x7f3efc101390 2024-06-10T01:31:17.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.475+0000 7f3f0409f700 1 -- 172.21.15.7:0/1520811602 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3efc1a0110 con 0x7f3efc101390 2024-06-10T01:31:17.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.475+0000 7f3ef2ffd700 1 -- 172.21.15.7:0/1520811602 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ef800ce30 con 0x7f3efc101390 2024-06-10T01:31:17.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.475+0000 7f3ef2ffd700 1 -- 172.21.15.7:0/1520811602 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8010430 con 0x7f3efc101390 2024-06-10T01:31:17.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.475+0000 7f3f0409f700 1 -- 172.21.15.7:0/1520811602 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3efc1a7990 con 0x7f3efc101390 2024-06-10T01:31:17.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.478+0000 7f3ef2ffd700 1 -- 172.21.15.7:0/1520811602 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f3ef8003680 con 0x7f3efc101390 2024-06-10T01:31:17.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.478+0000 7f3ef2ffd700 1 --2- 172.21.15.7:0/1520811602 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3ee80a2ca0 0x7f3ee80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:17.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.479+0000 7f3ef2ffd700 1 -- 172.21.15.7:0/1520811602 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f3ef80cf600 con 0x7f3efc101390 2024-06-10T01:31:17.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.479+0000 7f3f0163a700 1 --2- 172.21.15.7:0/1520811602 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3ee80a2ca0 0x7f3ee80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:17.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.479+0000 7f3f0163a700 1 --2- 172.21.15.7:0/1520811602 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3ee80a2ca0 0x7f3ee80a5160 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00b3c0 tx=0x7f3eec00d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:17.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.480+0000 7f3f0e922b80 1 Processor -- start 2024-06-10T01:31:17.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.480+0000 7f3f0e922b80 1 -- start start 2024-06-10T01:31:17.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.480+0000 7f3f0e922b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ca6d1f60 0x5637ca6d2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:17.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.480+0000 7f3f0e922b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ca6d2880 0x5637ca6cc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:17.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.480+0000 7f3f0e922b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ca6cc7a0 0x5637ca6ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:17.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.481+0000 7f3f0e922b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5637ca5bef60 con 0x5637ca6d2880 2024-06-10T01:31:17.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.481+0000 7f3f0e922b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5637ca61d850 con 0x5637ca6d1f60 2024-06-10T01:31:17.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.481+0000 7f3f0e922b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5637ca585460 con 0x5637ca6cc7a0 2024-06-10T01:31:17.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.481+0000 7f3f0163a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ca6d2880 0x5637ca6cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:17.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.481+0000 7f3f01e3b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ca6d1f60 0x5637ca6d2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:17.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.481+0000 7f3f0163a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ca6d2880 0x5637ca6cc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33828/0 (socket says 172.21.15.7:33828) 2024-06-10T01:31:17.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.481+0000 7f3f0163a700 1 -- 172.21.15.7:0/2259934767 learned_addr learned my addr 172.21.15.7:0/2259934767 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:17.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.481+0000 7f3f01e3b700 1 -- 172.21.15.7:0/2259934767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ca6cc7a0 msgr2=0x5637ca6ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:17.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.481+0000 7f3f01e3b700 1 --2- 172.21.15.7:0/2259934767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ca6cc7a0 0x5637ca6ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.481+0000 7f3f01e3b700 1 -- 172.21.15.7:0/2259934767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ca6d2880 msgr2=0x5637ca6cc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:17.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.481+0000 7f3f01e3b700 1 --2- 172.21.15.7:0/2259934767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ca6d2880 0x5637ca6cc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.481+0000 7f3f01e3b700 1 -- 172.21.15.7:0/2259934767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637ca56c8d0 con 0x5637ca6d1f60 2024-06-10T01:31:17.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.481+0000 7f3f0163a700 1 --2- 172.21.15.7:0/2259934767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ca6d2880 0x5637ca6cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:31:17.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.481+0000 7f3f01e3b700 1 --2- 172.21.15.7:0/2259934767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ca6d1f60 0x5637ca6d2340 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3ef80ccd20 tx=0x7f3ef809b640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5ce46f57bf2dabb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:17.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.481+0000 7f3ee7fff700 1 -- 172.21.15.7:0/2259934767 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef809ba80 con 0x5637ca6d1f60 2024-06-10T01:31:17.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.481+0000 7f3ee7fff700 1 -- 172.21.15.7:0/2259934767 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ef809fe30 con 0x5637ca6d1f60 2024-06-10T01:31:17.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.481+0000 7f3ee7fff700 1 -- 172.21.15.7:0/2259934767 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef809bce0 con 0x5637ca6d1f60 2024-06-10T01:31:17.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.482+0000 7f3f0e922b80 1 -- 172.21.15.7:0/2259934767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ca6d1f60 msgr2=0x5637ca6d2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:17.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.482+0000 7f3f0e922b80 1 --2- 172.21.15.7:0/2259934767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ca6d1f60 0x5637ca6d2340 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3ef80ccd20 tx=0x7f3ef809b640 comp rx=0 tx=0).stop 2024-06-10T01:31:17.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.482+0000 7f3f0e922b80 1 -- 172.21.15.7:0/2259934767 shutdown_connections 2024-06-10T01:31:17.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.482+0000 7f3f0e922b80 1 --2- 172.21.15.7:0/2259934767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ca6cc7a0 0x5637ca6ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.482+0000 7f3f0e922b80 1 --2- 172.21.15.7:0/2259934767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ca6d1f60 0x5637ca6d2340 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.482+0000 7f3f0e922b80 1 --2- 172.21.15.7:0/2259934767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ca6d2880 0x5637ca6cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.482+0000 7f3f0e922b80 1 -- 172.21.15.7:0/2259934767 >> 172.21.15.7:0/2259934767 conn(0x5637ca5f9080 msgr2=0x5637ca6c7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:17.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.482+0000 7f3f0e922b80 1 -- 172.21.15.7:0/2259934767 shutdown_connections 2024-06-10T01:31:17.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.482+0000 7f3f0e922b80 1 -- 172.21.15.7:0/2259934767 wait complete. 2024-06-10T01:31:17.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.483+0000 7f3f0e922b80 1 -- start start 2024-06-10T01:31:17.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.483+0000 7f3f0e922b80 1 Processor -- start 2024-06-10T01:31:17.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.483+0000 7f3f0e922b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ca6cc7a0 0x5637ca6d8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:17.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.483+0000 7f3f0e922b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ca6d1f60 0x5637ca6d5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:17.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.483+0000 7f3f0e922b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ca6d2880 0x5637ca6d5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:17.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.483+0000 7f3f0e922b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5637ca5bdfc0 con 0x5637ca6d1f60 2024-06-10T01:31:17.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.483+0000 7f3f0e922b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5637ca619810 con 0x5637ca6cc7a0 2024-06-10T01:31:17.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.483+0000 7f3f0e922b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5637ca6d9200 con 0x5637ca6d2880 2024-06-10T01:31:17.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.483+0000 7f3f0e922b80 10 client.0 _ll_register_callbacks cb 0x5637ca5b35b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:31:17.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.484+0000 7f3f0163a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ca6d1f60 0x5637ca6d5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:17.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.484+0000 7f3f0163a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ca6d1f60 0x5637ca6d5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33844/0 (socket says 172.21.15.7:33844) 2024-06-10T01:31:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.484+0000 7f3f0163a700 1 -- 172.21.15.7:0/912063179 learned_addr learned my addr 172.21.15.7:0/912063179 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.484+0000 7f3f01e3b700 1 --2- 172.21.15.7:0/912063179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ca6cc7a0 0x5637ca6d8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.484+0000 7f3f0263c700 1 --2- 172.21.15.7:0/912063179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ca6d2880 0x5637ca6d5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.484+0000 7f3f01e3b700 1 -- 172.21.15.7:0/912063179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ca6d2880 msgr2=0x5637ca6d5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.484+0000 7f3f01e3b700 1 --2- 172.21.15.7:0/912063179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ca6d2880 0x5637ca6d5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.484+0000 7f3f01e3b700 1 -- 172.21.15.7:0/912063179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ca6d1f60 msgr2=0x5637ca6d5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.484+0000 7f3f01e3b700 1 --2- 172.21.15.7:0/912063179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ca6d1f60 0x5637ca6d5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.484+0000 7f3f01e3b700 1 -- 172.21.15.7:0/912063179 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637ca5e3750 con 0x5637ca6cc7a0 2024-06-10T01:31:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.484+0000 7f3f01e3b700 1 --2- 172.21.15.7:0/912063179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ca6cc7a0 0x5637ca6d8700 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3ef809b730 tx=0x7f3ef80a4760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89beea800331a1d6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.484+0000 7f3ee57fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:31:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.484+0000 7f3ee57fa700 1 -- 172.21.15.7:0/912063179 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef80ba070 con 0x5637ca6cc7a0 2024-06-10T01:31:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.485+0000 7f3f0e922b80 1 -- 172.21.15.7:0/912063179 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ef809fe30 con 0x5637ca6cc7a0 2024-06-10T01:31:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.485+0000 7f3ee57fa700 1 -- 172.21.15.7:0/912063179 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f3ef80cf2f0 con 0x5637ca6cc7a0 2024-06-10T01:31:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.485+0000 7f3ee57fa700 1 -- 172.21.15.7:0/912063179 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f3ef809bce0 con 0x5637ca6cc7a0 2024-06-10T01:31:17.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.485+0000 7f3f0e922b80 10 client.30542 fetch_fsmap learned FSMap version 57 2024-06-10T01:31:17.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.485+0000 7f3f0e922b80 1 -- 172.21.15.7:0/912063179 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5637ca6e3a60 con 0x5637ca6cc7a0 2024-06-10T01:31:17.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.485+0000 7f3ee57fa700 1 -- 172.21.15.7:0/912063179 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ef809fe30 con 0x5637ca6cc7a0 2024-06-10T01:31:17.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.486+0000 7f3ee57fa700 1 -- 172.21.15.7:0/912063179 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7f3ef80a42f0 con 0x5637ca6cc7a0 2024-06-10T01:31:17.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.486+0000 7f3f0e922b80 10 client.30542 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:31:17.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.486+0000 7f3f0e922b80 10 client.30542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:31:17.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.486+0000 7f3f0e922b80 20 client.30542 populate_metadata read hostname 'smithi007' 2024-06-10T01:31:17.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.486+0000 7f3f0e922b80 1 --2- 172.21.15.7:0/912063179 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5637ca6e6170 0x5637ca6e8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:17.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.486+0000 7f3f0e922b80 4 client.30542 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:31:17.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.486+0000 7f3f0e922b80 1 -- 172.21.15.7:0/912063179 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637ca62ad50 con 0x5637ca6e6170 2024-06-10T01:31:17.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.486+0000 7f3f0163a700 1 --2- 172.21.15.7:0/912063179 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5637ca6e6170 0x5637ca6e8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:17.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.487+0000 7f3ee57fa700 10 client.30542 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:17.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.487+0000 7f3f0163a700 1 --2- 172.21.15.7:0/912063179 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5637ca6e6170 0x5637ca6e8590 crc :-1 s=READY pgs=559 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea4cb5067ad269d9 server_cookie=703cff99286ee899 in_seq=0 out_seq=0 2024-06-10T01:31:17.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.488+0000 7f3ee57fa700 1 -- 172.21.15.7:0/912063179 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5637ca62ad50 con 0x5637ca6e6170 2024-06-10T01:31:17.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.488+0000 7f3ee57fa700 10 client.30542 handle_command_reply: tid=0 2024-06-10T01:31:17.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.488+0000 7f3f0e922b80 1 client.30542 shutdown 2024-06-10T01:31:17.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.488+0000 7f3f0e922b80 1 -- 172.21.15.7:0/912063179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ca6cc7a0 msgr2=0x5637ca6d8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:17.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.488+0000 7f3f0e922b80 1 --2- 172.21.15.7:0/912063179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ca6cc7a0 0x5637ca6d8700 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3ef809b730 tx=0x7f3ef80a4760 comp rx=0 tx=0).stop 2024-06-10T01:31:17.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.488+0000 7f3f0e922b80 1 -- 172.21.15.7:0/912063179 shutdown_connections 2024-06-10T01:31:17.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.488+0000 7f3f0e922b80 1 --2- 172.21.15.7:0/912063179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ca6d2880 0x5637ca6d5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.488+0000 7f3f0e922b80 1 --2- 172.21.15.7:0/912063179 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5637ca6e6170 0x5637ca6e8590 crc :-1 s=READY pgs=559 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.488+0000 7f3f0e922b80 1 --2- 172.21.15.7:0/912063179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ca6cc7a0 0x5637ca6d8700 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.488+0000 7f3f0e922b80 1 --2- 172.21.15.7:0/912063179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ca6d1f60 0x5637ca6d5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.488+0000 7f3f0e922b80 1 -- 172.21.15.7:0/912063179 >> 172.21.15.7:0/912063179 conn(0x5637ca5f9080 msgr2=0x5637ca6c6700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:17.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.488+0000 7f3ee57fa700 0 client.30542 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:17.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.488+0000 7f3f0e922b80 1 -- 172.21.15.7:0/912063179 shutdown_connections 2024-06-10T01:31:17.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.488+0000 7f3f0e922b80 1 -- 172.21.15.7:0/912063179 wait complete. 2024-06-10T01:31:17.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.489+0000 7f3f0e922b80 20 client.30542 trim_cache size 0 max 16384 2024-06-10T01:31:17.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.508+0000 7f3f0e922b80 1 Processor -- start 2024-06-10T01:31:17.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.508+0000 7f3f0e922b80 1 -- start start 2024-06-10T01:31:17.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.509+0000 7f3f0e922b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ca6cc7a0 0x5637ca6d3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:17.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.509+0000 7f3f0e922b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ca6d1f60 0x5637ca6d8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:17.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.509+0000 7f3f0e922b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ca6d2880 0x5637ca6c6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:17.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.509+0000 7f3f0e922b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc1b2e20 con 0x5637ca6d1f60 2024-06-10T01:31:17.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.509+0000 7f3f0e922b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc1b0650 con 0x5637ca6d2880 2024-06-10T01:31:17.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.509+0000 7f3f0e922b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc1ae4a0 con 0x5637ca6cc7a0 2024-06-10T01:31:17.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.509+0000 7f3f0163a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ca6d1f60 0x5637ca6d8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:17.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.509+0000 7f3f0263c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ca6d2880 0x5637ca6c6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:17.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.509+0000 7f3f0263c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ca6d2880 0x5637ca6c6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:37166/0 (socket says 172.21.15.7:37166) 2024-06-10T01:31:17.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.509+0000 7f3f0263c700 1 -- 172.21.15.7:0/971724868 learned_addr learned my addr 172.21.15.7:0/971724868 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:17.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.509+0000 7f3f0263c700 1 -- 172.21.15.7:0/971724868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ca6cc7a0 msgr2=0x5637ca6d3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:17.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.509+0000 7f3f0263c700 1 --2- 172.21.15.7:0/971724868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ca6cc7a0 0x5637ca6d3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.509+0000 7f3f0263c700 1 -- 172.21.15.7:0/971724868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ca6d1f60 msgr2=0x5637ca6d8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:17.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.509+0000 7f3f0263c700 1 --2- 172.21.15.7:0/971724868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ca6d1f60 0x5637ca6d8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.509+0000 7f3f0263c700 1 -- 172.21.15.7:0/971724868 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637ca6d3f70 con 0x5637ca6d2880 2024-06-10T01:31:17.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.510+0000 7f3f0263c700 1 --2- 172.21.15.7:0/971724868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ca6d2880 0x5637ca6c6e80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009b70 tx=0x7f3ef4009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76da7f162a33022a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:17.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.510+0000 7f3ee77fe700 1 -- 172.21.15.7:0/971724868 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef400bbf0 con 0x5637ca6d2880 2024-06-10T01:31:17.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.510+0000 7f3ee77fe700 1 -- 172.21.15.7:0/971724868 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ef400bd90 con 0x5637ca6d2880 2024-06-10T01:31:17.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.510+0000 7f3ee77fe700 1 -- 172.21.15.7:0/971724868 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef401d850 con 0x5637ca6d2880 2024-06-10T01:31:17.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.511+0000 7f3f0e922b80 1 -- 172.21.15.7:0/971724868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ca6d2880 msgr2=0x5637ca6c6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:17.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.511+0000 7f3f0e922b80 1 --2- 172.21.15.7:0/971724868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ca6d2880 0x5637ca6c6e80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009b70 tx=0x7f3ef4009ba0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.511+0000 7f3f0e922b80 1 -- 172.21.15.7:0/971724868 shutdown_connections 2024-06-10T01:31:17.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.511+0000 7f3f0e922b80 1 --2- 172.21.15.7:0/971724868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ca6d2880 0x5637ca6c6e80 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.511+0000 7f3f0e922b80 1 --2- 172.21.15.7:0/971724868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ca6cc7a0 0x5637ca6d3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.511+0000 7f3f0e922b80 1 --2- 172.21.15.7:0/971724868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ca6d1f60 0x5637ca6d8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.511+0000 7f3f0e922b80 1 -- 172.21.15.7:0/971724868 >> 172.21.15.7:0/971724868 conn(0x5637ca5f9080 msgr2=0x5637ca6d13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:17.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.511+0000 7f3f0e922b80 1 -- 172.21.15.7:0/971724868 shutdown_connections 2024-06-10T01:31:17.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.511+0000 7f3f0e922b80 1 -- 172.21.15.7:0/971724868 wait complete. 2024-06-10T01:31:17.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.512+0000 7f3f0e922b80 1 -- start start 2024-06-10T01:31:17.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.512+0000 7f3f0e922b80 1 Processor -- start 2024-06-10T01:31:17.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.513+0000 7f3f0e922b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ca6cc7a0 0x5637ca6ee200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:17.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.513+0000 7f3f0e922b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ca6d1f60 0x5637ca6eb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:17.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.513+0000 7f3f0e922b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ca6d2880 0x5637ca6ebae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:17.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.513+0000 7f3f0e922b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc1aeb10 con 0x5637ca6d2880 2024-06-10T01:31:17.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.513+0000 7f3f0e922b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc1aff80 con 0x5637ca6cc7a0 2024-06-10T01:31:17.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.513+0000 7f3f0e922b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc1af1e0 con 0x5637ca6d1f60 2024-06-10T01:31:17.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.513+0000 7f3f0e922b80 10 client.0 _ll_register_callbacks cb 0x5637ca5b35b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:31:17.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.513+0000 7f3f01e3b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ca6cc7a0 0x5637ca6ee200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:17.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.513+0000 7f3f0163a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ca6d1f60 0x5637ca6eb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:17.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.513+0000 7f3f0263c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ca6d2880 0x5637ca6ebae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:17.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.513+0000 7f3f01e3b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ca6cc7a0 0x5637ca6ee200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:37182/0 (socket says 172.21.15.7:37182) 2024-06-10T01:31:17.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.513+0000 7f3f0263c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ca6d2880 0x5637ca6ebae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33864/0 (socket says 172.21.15.7:33864) 2024-06-10T01:31:17.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.513+0000 7f3f01e3b700 1 -- 172.21.15.7:0/3132868311 learned_addr learned my addr 172.21.15.7:0/3132868311 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:17.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.514+0000 7f3f0263c700 1 -- 172.21.15.7:0/3132868311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ca6d1f60 msgr2=0x5637ca6eb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:17.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.514+0000 7f3f0263c700 1 --2- 172.21.15.7:0/3132868311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ca6d1f60 0x5637ca6eb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.514+0000 7f3f0263c700 1 -- 172.21.15.7:0/3132868311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ca6cc7a0 msgr2=0x5637ca6ee200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:17.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.514+0000 7f3f0263c700 1 --2- 172.21.15.7:0/3132868311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ca6cc7a0 0x5637ca6ee200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.514+0000 7f3f0263c700 1 -- 172.21.15.7:0/3132868311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637ca6ec120 con 0x5637ca6d2880 2024-06-10T01:31:17.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.514+0000 7f3f0263c700 1 --2- 172.21.15.7:0/3132868311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ca6d2880 0x5637ca6ebae0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009450 tx=0x7f3ef4005090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8abeb7b5b40990 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:17.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.514+0000 7f3ee57fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:31:17.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.514+0000 7f3ee57fa700 1 -- 172.21.15.7:0/3132868311 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4032480 con 0x5637ca6d2880 2024-06-10T01:31:17.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.515+0000 7f3ee57fa700 1 -- 172.21.15.7:0/3132868311 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f3ef4032620 con 0x5637ca6d2880 2024-06-10T01:31:17.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.515+0000 7f3f0e922b80 1 -- 172.21.15.7:0/3132868311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ef400bd90 con 0x5637ca6d2880 2024-06-10T01:31:17.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.515+0000 7f3ee57fa700 1 -- 172.21.15.7:0/3132868311 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f3ef403b050 con 0x5637ca6d2880 2024-06-10T01:31:17.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.515+0000 7f3f0e922b80 10 client.21381 fetch_fsmap learned FSMap version 57 2024-06-10T01:31:17.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.515+0000 7f3f0e922b80 1 -- 172.21.15.7:0/3132868311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5637ca750ae0 con 0x5637ca6d2880 2024-06-10T01:31:17.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.515+0000 7f3ee57fa700 1 -- 172.21.15.7:0/3132868311 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ef400bd90 con 0x5637ca6d2880 2024-06-10T01:31:17.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.515+0000 7f3ee57fa700 1 -- 172.21.15.7:0/3132868311 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7f3ef4011070 con 0x5637ca6d2880 2024-06-10T01:31:17.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.515+0000 7f3f0e922b80 10 client.21381 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:31:17.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.515+0000 7f3f0e922b80 10 client.21381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:31:17.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.515+0000 7f3f0e922b80 20 client.21381 populate_metadata read hostname 'smithi007' 2024-06-10T01:31:17.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.515+0000 7f3f0e922b80 1 --2- 172.21.15.7:0/3132868311 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5637ca7510d0 0x5637ca7534f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:17.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.515+0000 7f3f0e922b80 4 client.21381 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:31:17.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.515+0000 7f3f0e922b80 1 -- 172.21.15.7:0/3132868311 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5637ca6252d0 con 0x5637ca7510d0 2024-06-10T01:31:17.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.516+0000 7f3f01e3b700 1 --2- 172.21.15.7:0/3132868311 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5637ca7510d0 0x5637ca7534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:17.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.516+0000 7f3f01e3b700 1 --2- 172.21.15.7:0/3132868311 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5637ca7510d0 0x5637ca7534f0 crc :-1 s=READY pgs=560 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3019556ade092691 server_cookie=fc687342e9306f69 in_seq=0 out_seq=0 2024-06-10T01:31:17.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.516+0000 7f3ee57fa700 10 client.21381 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:17.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.517+0000 7f3ee57fa700 1 -- 172.21.15.7:0/3132868311 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5637ca6252d0 con 0x5637ca7510d0 2024-06-10T01:31:17.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.517+0000 7f3ee57fa700 10 client.21381 handle_command_reply: tid=0 2024-06-10T01:31:17.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.517+0000 7f3f0e922b80 1 client.21381 shutdown 2024-06-10T01:31:17.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.517+0000 7f3f0e922b80 1 -- 172.21.15.7:0/3132868311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ca6d2880 msgr2=0x5637ca6ebae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:17.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.517+0000 7f3f0e922b80 1 --2- 172.21.15.7:0/3132868311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ca6d2880 0x5637ca6ebae0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009450 tx=0x7f3ef4005090 comp rx=0 tx=0).stop 2024-06-10T01:31:17.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.517+0000 7f3f0e922b80 1 -- 172.21.15.7:0/3132868311 shutdown_connections 2024-06-10T01:31:17.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.517+0000 7f3f0e922b80 1 --2- 172.21.15.7:0/3132868311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637ca6d2880 0x5637ca6ebae0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.517+0000 7f3f0e922b80 1 --2- 172.21.15.7:0/3132868311 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5637ca7510d0 0x5637ca7534f0 crc :-1 s=READY pgs=560 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.517+0000 7f3f0e922b80 1 --2- 172.21.15.7:0/3132868311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637ca6cc7a0 0x5637ca6ee200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.517+0000 7f3f0e922b80 1 --2- 172.21.15.7:0/3132868311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637ca6d1f60 0x5637ca6eb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.517+0000 7f3f0e922b80 1 -- 172.21.15.7:0/3132868311 >> 172.21.15.7:0/3132868311 conn(0x5637ca5f9080 msgr2=0x5637ca6d0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:17.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.517+0000 7f3ee57fa700 0 client.21381 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:17.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.517+0000 7f3f0e922b80 1 -- 172.21.15.7:0/3132868311 shutdown_connections 2024-06-10T01:31:17.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.517+0000 7f3f0e922b80 1 -- 172.21.15.7:0/3132868311 wait complete. 2024-06-10T01:31:17.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.517+0000 7f3f0e922b80 20 client.21381 trim_cache size 0 max 16384 2024-06-10T01:31:17.514 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:31:17.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.518+0000 7f3f0409f700 1 -- 172.21.15.7:0/1520811602 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3ee80a2ca0 msgr2=0x7f3ee80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:17.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.518+0000 7f3f0409f700 1 --2- 172.21.15.7:0/1520811602 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3ee80a2ca0 0x7f3ee80a5160 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00b3c0 tx=0x7f3eec00d040 comp rx=0 tx=0).stop 2024-06-10T01:31:17.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.518+0000 7f3f0409f700 1 -- 172.21.15.7:0/1520811602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3efc101390 msgr2=0x7f3efc19ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:17.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.518+0000 7f3f0409f700 1 --2- 172.21.15.7:0/1520811602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3efc101390 0x7f3efc19ae10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3ef800eb90 tx=0x7f3ef800eea0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.518+0000 7f3f0409f700 1 -- 172.21.15.7:0/1520811602 shutdown_connections 2024-06-10T01:31:17.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.518+0000 7f3f0409f700 1 --2- 172.21.15.7:0/1520811602 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f3ee80a2ca0 0x7f3ee80a5160 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.518+0000 7f3f0409f700 1 --2- 172.21.15.7:0/1520811602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3efc107400 0x7f3efc19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.518+0000 7f3f0409f700 1 --2- 172.21.15.7:0/1520811602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3efc101390 0x7f3efc19ae10 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.518+0000 7f3f0409f700 1 --2- 172.21.15.7:0/1520811602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3efc101d50 0x7f3efc19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:17.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.518+0000 7f3f0409f700 1 -- 172.21.15.7:0/1520811602 >> 172.21.15.7:0/1520811602 conn(0x7f3efc078340 msgr2=0x7f3efc0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:17.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.519+0000 7f3f0409f700 1 -- 172.21.15.7:0/1520811602 shutdown_connections 2024-06-10T01:31:17.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:17.519+0000 7f3f0409f700 1 -- 172.21.15.7:0/1520811602 wait complete. 2024-06-10T01:31:17.572 INFO:tasks.workunit.client.0.smithi007.stdout: 10 282277 21.93 MB/sec execute 594 sec latency 114.321 ms 2024-06-10T01:31:17.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:17 smithi175 ceph-mon[36636]: pgmap v1192: 129 pgs: 129 active+clean; 702 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.4 MiB/s wr, 700 op/s 2024-06-10T01:31:17.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:17 smithi007 ceph-mon[32398]: pgmap v1192: 129 pgs: 129 active+clean; 702 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.4 MiB/s wr, 700 op/s 2024-06-10T01:31:17.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:17 smithi078 ceph-mon[36683]: pgmap v1192: 129 pgs: 129 active+clean; 702 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.4 MiB/s wr, 700 op/s 2024-06-10T01:31:18.572 INFO:tasks.workunit.client.0.smithi007.stdout: 10 282967 21.95 MB/sec execute 595 sec latency 78.856 ms 2024-06-10T01:31:19.322 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:31:19.322 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:31:19.572 INFO:tasks.workunit.client.0.smithi007.stdout: 10 283474 21.96 MB/sec execute 596 sec latency 94.391 ms 2024-06-10T01:31:19.657 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:31:19.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:19 smithi175 ceph-mon[36636]: pgmap v1193: 129 pgs: 129 active+clean; 700 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.9 MiB/s wr, 653 op/s 2024-06-10T01:31:19.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:19 smithi007 ceph-mon[32398]: pgmap v1193: 129 pgs: 129 active+clean; 700 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.9 MiB/s wr, 653 op/s 2024-06-10T01:31:19.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:19 smithi078 ceph-mon[36683]: pgmap v1193: 129 pgs: 129 active+clean; 700 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.9 MiB/s wr, 653 op/s 2024-06-10T01:31:20.572 INFO:tasks.workunit.client.0.smithi007.stdout: 10 283777 21.94 MB/sec execute 597 sec latency 575.620 ms 2024-06-10T01:31:21.572 INFO:tasks.workunit.client.0.smithi007.stdout: 10 284110 21.94 MB/sec execute 598 sec latency 588.420 ms 2024-06-10T01:31:21.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:21 smithi175 ceph-mon[36636]: pgmap v1194: 129 pgs: 129 active+clean; 700 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.4 MiB/s wr, 566 op/s 2024-06-10T01:31:21.750 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:21 smithi007 ceph-mon[32398]: pgmap v1194: 129 pgs: 129 active+clean; 700 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.4 MiB/s wr, 566 op/s 2024-06-10T01:31:21.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:21 smithi078 ceph-mon[36683]: pgmap v1194: 129 pgs: 129 active+clean; 700 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.4 MiB/s wr, 566 op/s 2024-06-10T01:31:21.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.952+0000 7f34cb22e700 1 -- 172.21.15.7:0/1136497197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34c406e910 msgr2=0x7f34c411b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:21.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.952+0000 7f34cb22e700 1 --2- 172.21.15.7:0/1136497197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34c406e910 0x7f34c411b5e0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f34b80098f0 tx=0x7f34b8009c00 comp rx=0 tx=0).stop 2024-06-10T01:31:21.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.953+0000 7f34cb22e700 1 -- 172.21.15.7:0/1136497197 shutdown_connections 2024-06-10T01:31:21.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.953+0000 7f34cb22e700 1 --2- 172.21.15.7:0/1136497197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34c411bb20 0x7f34c411df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:21.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.953+0000 7f34cb22e700 1 --2- 172.21.15.7:0/1136497197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34c406dff0 0x7f34c406e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:21.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.953+0000 7f34cb22e700 1 --2- 172.21.15.7:0/1136497197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34c406e910 0x7f34c411b5e0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:21.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.953+0000 7f34cb22e700 1 -- 172.21.15.7:0/1136497197 >> 172.21.15.7:0/1136497197 conn(0x7f34c406d680 msgr2=0x7f34c410b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:21.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.954+0000 7f34cb22e700 1 -- 172.21.15.7:0/1136497197 shutdown_connections 2024-06-10T01:31:21.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.954+0000 7f34cb22e700 1 -- 172.21.15.7:0/1136497197 wait complete. 2024-06-10T01:31:21.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.955+0000 7f34cb22e700 1 Processor -- start 2024-06-10T01:31:21.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.957+0000 7f34cb22e700 1 -- start start 2024-06-10T01:31:21.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.957+0000 7f34cb22e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34c406dff0 0x7f34c411b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:21.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.957+0000 7f34cb22e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34c406e910 0x7f34c411b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:21.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.957+0000 7f34cb22e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34c411bb20 0x7f34c4114340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:21.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.957+0000 7f34cb22e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f34c41216e0 con 0x7f34c406dff0 2024-06-10T01:31:21.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.957+0000 7f34cb22e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f34c4121560 con 0x7f34c406e910 2024-06-10T01:31:21.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.957+0000 7f34cb22e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f34c4121860 con 0x7f34c411bb20 2024-06-10T01:31:21.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.958+0000 7f34c3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34c406e910 0x7f34c411b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:21.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.958+0000 7f34c3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34c406e910 0x7f34c411b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:38696/0 (socket says 172.21.15.7:38696) 2024-06-10T01:31:21.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.958+0000 7f34c3fff700 1 -- 172.21.15.7:0/288756324 learned_addr learned my addr 172.21.15.7:0/288756324 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:21.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.959+0000 7f34c3fff700 1 -- 172.21.15.7:0/288756324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34c411bb20 msgr2=0x7f34c4114340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T01:31:21.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.959+0000 7f34c3fff700 1 --2- 172.21.15.7:0/288756324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34c411bb20 0x7f34c4114340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:21.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.959+0000 7f34c3fff700 1 -- 172.21.15.7:0/288756324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34c406dff0 msgr2=0x7f34c411b080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T01:31:21.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.959+0000 7f34c3fff700 1 --2- 172.21.15.7:0/288756324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34c406dff0 0x7f34c411b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:21.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.959+0000 7f34c3fff700 1 -- 172.21.15.7:0/288756324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c4114c40 con 0x7f34c406e910 2024-06-10T01:31:21.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.962+0000 7f34c3fff700 1 --2- 172.21.15.7:0/288756324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34c406e910 0x7f34c411b5c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009810 tx=0x7f34b800fc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:21.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.964+0000 7f34c1ffb700 1 -- 172.21.15.7:0/288756324 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b8003a90 con 0x7f34c406e910 2024-06-10T01:31:21.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.964+0000 7f34c1ffb700 1 -- 172.21.15.7:0/288756324 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34b8003c30 con 0x7f34c406e910 2024-06-10T01:31:21.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.964+0000 7f34c1ffb700 1 -- 172.21.15.7:0/288756324 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b8017770 con 0x7f34c406e910 2024-06-10T01:31:21.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.964+0000 7f34cb22e700 1 -- 172.21.15.7:0/288756324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34c4114ed0 con 0x7f34c406e910 2024-06-10T01:31:21.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.965+0000 7f34cb22e700 1 -- 172.21.15.7:0/288756324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34c4118d80 con 0x7f34c406e910 2024-06-10T01:31:21.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.975+0000 7f34d5ab1b80 1 Processor -- start 2024-06-10T01:31:21.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.977+0000 7f34d5ab1b80 1 -- start start 2024-06-10T01:31:21.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.977+0000 7f34d5ab1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651a34e36c0 0x5651a34e3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:21.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.978+0000 7f34c1ffb700 1 -- 172.21.15.7:0/288756324 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f34b800b610 con 0x7f34c406e910 2024-06-10T01:31:21.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.978+0000 7f34d5ab1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651a34e3fe0 0x5651a34dda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:21.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.978+0000 7f34d5ab1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651a34ddfa0 0x5651a34de3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:21.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.978+0000 7f34d5ab1b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5651a342f890 con 0x5651a34e3fe0 2024-06-10T01:31:21.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.978+0000 7f34d5ab1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5651a33d77a0 con 0x5651a34e36c0 2024-06-10T01:31:21.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.978+0000 7f34d5ab1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5651a342b850 con 0x5651a34ddfa0 2024-06-10T01:31:21.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.978+0000 7f34c8fca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651a34e3fe0 0x5651a34dda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:21.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.978+0000 7f34c8fca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651a34e3fe0 0x5651a34dda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44582/0 (socket says 172.21.15.7:44582) 2024-06-10T01:31:21.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.978+0000 7f34c8fca700 1 -- 172.21.15.7:0/1997876224 learned_addr learned my addr 172.21.15.7:0/1997876224 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:21.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.979+0000 7f34c97cb700 1 --2- 172.21.15.7:0/1997876224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651a34e36c0 0x5651a34e3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.980+0000 7f34c8fca700 1 -- 172.21.15.7:0/1997876224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651a34ddfa0 msgr2=0x5651a34de3c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T01:31:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.980+0000 7f34c8fca700 1 --2- 172.21.15.7:0/1997876224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651a34ddfa0 0x5651a34de3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.980+0000 7f34c8fca700 1 -- 172.21.15.7:0/1997876224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651a34e36c0 msgr2=0x5651a34e3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.980+0000 7f34c8fca700 1 --2- 172.21.15.7:0/1997876224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651a34e36c0 0x5651a34e3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.980+0000 7f34c8fca700 1 -- 172.21.15.7:0/1997876224 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651a33cdfd0 con 0x5651a34e3fe0 2024-06-10T01:31:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.980+0000 7f34c1ffb700 1 --2- 172.21.15.7:0/288756324 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f34ac0a2e20 0x7f34ac0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.980+0000 7f34c1ffb700 1 -- 172.21.15.7:0/288756324 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f34b80d02a0 con 0x7f34c406e910 2024-06-10T01:31:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.981+0000 7f34c97cb700 1 --2- 172.21.15.7:0/288756324 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f34ac0a2e20 0x7f34ac0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.982+0000 7f34c8fca700 1 --2- 172.21.15.7:0/1997876224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651a34e3fe0 0x5651a34dda60 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f34b400bb50 tx=0x7f34b400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b8dc6c563cfcd6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.983+0000 7f34aaffd700 1 -- 172.21.15.7:0/1997876224 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b4014030 con 0x5651a34e3fe0 2024-06-10T01:31:21.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.983+0000 7f34aaffd700 1 -- 172.21.15.7:0/1997876224 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34b4004d10 con 0x5651a34e3fe0 2024-06-10T01:31:21.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.983+0000 7f34aaffd700 1 -- 172.21.15.7:0/1997876224 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b4005440 con 0x5651a34e3fe0 2024-06-10T01:31:21.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.986+0000 7f34d5ab1b80 1 -- 172.21.15.7:0/1997876224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651a34e3fe0 msgr2=0x5651a34dda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:21.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.986+0000 7f34d5ab1b80 1 --2- 172.21.15.7:0/1997876224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651a34e3fe0 0x5651a34dda60 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f34b400bb50 tx=0x7f34b400be60 comp rx=0 tx=0).stop 2024-06-10T01:31:21.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.987+0000 7f34c97cb700 1 --2- 172.21.15.7:0/288756324 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f34ac0a2e20 0x7f34ac0a52e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f34c4073e30 tx=0x7f34bc009410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:21.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.987+0000 7f34d5ab1b80 1 -- 172.21.15.7:0/1997876224 shutdown_connections 2024-06-10T01:31:21.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.987+0000 7f34d5ab1b80 1 --2- 172.21.15.7:0/1997876224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651a34ddfa0 0x5651a34de3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:21.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.987+0000 7f34d5ab1b80 1 --2- 172.21.15.7:0/1997876224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651a34e36c0 0x5651a34e3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:21.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.987+0000 7f34d5ab1b80 1 --2- 172.21.15.7:0/1997876224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651a34e3fe0 0x5651a34dda60 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:21.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.987+0000 7f34d5ab1b80 1 -- 172.21.15.7:0/1997876224 >> 172.21.15.7:0/1997876224 conn(0x5651a3408d40 msgr2=0x5651a34d88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:21.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.988+0000 7f34d5ab1b80 1 -- 172.21.15.7:0/1997876224 shutdown_connections 2024-06-10T01:31:21.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.988+0000 7f34d5ab1b80 1 -- 172.21.15.7:0/1997876224 wait complete. 2024-06-10T01:31:21.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.989+0000 7f34d5ab1b80 1 -- start start 2024-06-10T01:31:21.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.989+0000 7f34d5ab1b80 1 Processor -- start 2024-06-10T01:31:21.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.989+0000 7f34d5ab1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651a34ddfa0 0x5651a34e9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:21.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.989+0000 7f34d5ab1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651a34e36c0 0x5651a34e7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:21.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.989+0000 7f34d5ab1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651a34e3fe0 0x5651a34e7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:21.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.989+0000 7f34d5ab1b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5651a34eaa50 con 0x5651a34e36c0 2024-06-10T01:31:21.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.989+0000 7f34d5ab1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5651a33cfc70 con 0x5651a34ddfa0 2024-06-10T01:31:21.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.989+0000 7f34d5ab1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5651a34eabd0 con 0x5651a34e3fe0 2024-06-10T01:31:21.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.989+0000 7f34d5ab1b80 10 client.0 _ll_register_callbacks cb 0x5651a33c99a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:31:21.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.990+0000 7f34c97cb700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651a34e3fe0 0x5651a34e7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:21.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.990+0000 7f34c97cb700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651a34e3fe0 0x5651a34e7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:42306/0 (socket says 172.21.15.7:42306) 2024-06-10T01:31:21.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.990+0000 7f34c97cb700 1 -- 172.21.15.7:0/3547812900 learned_addr learned my addr 172.21.15.7:0/3547812900 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:21.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.993+0000 7f34c97cb700 1 -- 172.21.15.7:0/3547812900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651a34ddfa0 msgr2=0x5651a34e9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:21.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.993+0000 7f34c97cb700 1 --2- 172.21.15.7:0/3547812900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651a34ddfa0 0x5651a34e9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:21.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.993+0000 7f34c97cb700 1 -- 172.21.15.7:0/3547812900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651a34e36c0 msgr2=0x5651a34e7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:21.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.993+0000 7f34c97cb700 1 --2- 172.21.15.7:0/3547812900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651a34e36c0 0x5651a34e7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:21.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.993+0000 7f34c97cb700 1 -- 172.21.15.7:0/3547812900 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651a337e950 con 0x5651a34e3fe0 2024-06-10T01:31:21.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.996+0000 7f34c97cb700 1 --2- 172.21.15.7:0/3547812900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651a34e3fe0 0x5651a34e7640 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f34bc0188d0 tx=0x7f34bc004b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81ba11ea36f63c40 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:21.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.996+0000 7f34a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:31:21.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.996+0000 7f34a3fff700 1 -- 172.21.15.7:0/3547812900 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34bc028060 con 0x5651a34e3fe0 2024-06-10T01:31:21.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.996+0000 7f34d5ab1b80 1 -- 172.21.15.7:0/3547812900 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34b4004d10 con 0x5651a34e3fe0 2024-06-10T01:31:21.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.996+0000 7f34a3fff700 1 -- 172.21.15.7:0/3547812900 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f34bc012030 con 0x5651a34e3fe0 2024-06-10T01:31:21.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.996+0000 7f34a3fff700 1 -- 172.21.15.7:0/3547812900 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f34bc02f050 con 0x5651a34e3fe0 2024-06-10T01:31:21.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.997+0000 7f34d5ab1b80 10 client.30424 fetch_fsmap learned FSMap version 57 2024-06-10T01:31:21.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.997+0000 7f34d5ab1b80 1 -- 172.21.15.7:0/3547812900 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5651a34f5490 con 0x5651a34e3fe0 2024-06-10T01:31:21.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.998+0000 7f34a3fff700 1 -- 172.21.15.7:0/3547812900 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34b4004d10 con 0x5651a34e3fe0 2024-06-10T01:31:21.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:21.998+0000 7f34a3fff700 1 -- 172.21.15.7:0/3547812900 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7f34bc021680 con 0x5651a34e3fe0 2024-06-10T01:31:21.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.000+0000 7f34d5ab1b80 10 client.30424 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:31:21.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.000+0000 7f34d5ab1b80 10 client.30424 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:31:21.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.000+0000 7f34d5ab1b80 20 client.30424 populate_metadata read hostname 'smithi007' 2024-06-10T01:31:21.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.000+0000 7f34d5ab1b80 1 --2- 172.21.15.7:0/3547812900 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5651a34f7ad0 0x5651a34f9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:21.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.000+0000 7f34d5ab1b80 4 client.30424 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:31:21.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.000+0000 7f34d5ab1b80 1 -- 172.21.15.7:0/3547812900 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651a343cd90 con 0x5651a34f7ad0 2024-06-10T01:31:21.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.000+0000 7f34c8fca700 1 --2- 172.21.15.7:0/3547812900 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5651a34f7ad0 0x5651a34f9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:21.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.001+0000 7f34c8fca700 1 --2- 172.21.15.7:0/3547812900 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5651a34f7ad0 0x5651a34f9f60 crc :-1 s=READY pgs=561 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30589152cbf82fe2 server_cookie=798a7715fcf87346 in_seq=0 out_seq=0 2024-06-10T01:31:21.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.001+0000 7f34a3fff700 10 client.30424 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:21.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.002+0000 7f34a3fff700 1 -- 172.21.15.7:0/3547812900 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5651a343cd90 con 0x5651a34f7ad0 2024-06-10T01:31:21.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.002+0000 7f34a3fff700 10 client.30424 handle_command_reply: tid=0 2024-06-10T01:31:21.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.003+0000 7f34d5ab1b80 1 client.30424 shutdown 2024-06-10T01:31:21.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.004+0000 7f34d5ab1b80 1 -- 172.21.15.7:0/3547812900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651a34e3fe0 msgr2=0x5651a34e7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:21.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.004+0000 7f34d5ab1b80 1 --2- 172.21.15.7:0/3547812900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651a34e3fe0 0x5651a34e7640 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f34bc0188d0 tx=0x7f34bc004b80 comp rx=0 tx=0).stop 2024-06-10T01:31:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.004+0000 7f34d5ab1b80 1 -- 172.21.15.7:0/3547812900 shutdown_connections 2024-06-10T01:31:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.004+0000 7f34d5ab1b80 1 --2- 172.21.15.7:0/3547812900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651a34e3fe0 0x5651a34e7640 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.004+0000 7f34d5ab1b80 1 --2- 172.21.15.7:0/3547812900 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5651a34f7ad0 0x5651a34f9f60 crc :-1 s=READY pgs=561 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.004+0000 7f34d5ab1b80 1 --2- 172.21.15.7:0/3547812900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651a34ddfa0 0x5651a34e9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.004+0000 7f34d5ab1b80 1 --2- 172.21.15.7:0/3547812900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651a34e36c0 0x5651a34e7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.004+0000 7f34d5ab1b80 1 -- 172.21.15.7:0/3547812900 >> 172.21.15.7:0/3547812900 conn(0x5651a3408d40 msgr2=0x5651a34dca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.004+0000 7f34a3fff700 0 client.30424 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:21.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.005+0000 7f34d5ab1b80 1 -- 172.21.15.7:0/3547812900 shutdown_connections 2024-06-10T01:31:21.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.005+0000 7f34d5ab1b80 1 -- 172.21.15.7:0/3547812900 wait complete. 2024-06-10T01:31:21.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.005+0000 7f34d5ab1b80 20 client.30424 trim_cache size 0 max 16384 2024-06-10T01:31:22.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.031+0000 7f34d5ab1b80 1 Processor -- start 2024-06-10T01:31:22.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.031+0000 7f34d5ab1b80 1 -- start start 2024-06-10T01:31:22.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.032+0000 7f34d5ab1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651a34ddfa0 0x5651a34f3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:22.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.032+0000 7f34d5ab1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651a34e36c0 0x5651a34dda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:22.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.032+0000 7f34c8fca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651a34ddfa0 0x5651a34f3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:22.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.032+0000 7f34c8fca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651a34ddfa0 0x5651a34f3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44604/0 (socket says 172.21.15.7:44604) 2024-06-10T01:31:22.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.032+0000 7f34c8fca700 1 -- 172.21.15.7:0/841068752 learned_addr learned my addr 172.21.15.7:0/841068752 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:22.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.032+0000 7f34d5ab1b80 1 --2- 172.21.15.7:0/841068752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651a34e3fe0 0x5651a34e6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:22.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.032+0000 7f34d5ab1b80 1 -- 172.21.15.7:0/841068752 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f34c41c8280 con 0x5651a34ddfa0 2024-06-10T01:31:22.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.032+0000 7f34c3fff700 1 --2- 172.21.15.7:0/841068752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651a34e36c0 0x5651a34dda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:22.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.032+0000 7f34d5ab1b80 1 -- 172.21.15.7:0/841068752 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f34c41c5ab0 con 0x5651a34e36c0 2024-06-10T01:31:22.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.032+0000 7f34d5ab1b80 1 -- 172.21.15.7:0/841068752 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f34c41c3900 con 0x5651a34e3fe0 2024-06-10T01:31:22.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.032+0000 7f34c97cb700 1 --2- 172.21.15.7:0/841068752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651a34e3fe0 0x5651a34e6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:22.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.032+0000 7f34c3fff700 1 -- 172.21.15.7:0/841068752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651a34e3fe0 msgr2=0x5651a34e6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:22.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.032+0000 7f34c3fff700 1 --2- 172.21.15.7:0/841068752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651a34e3fe0 0x5651a34e6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:22.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.032+0000 7f34c3fff700 1 -- 172.21.15.7:0/841068752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651a34ddfa0 msgr2=0x5651a34f3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:22.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.032+0000 7f34c3fff700 1 --2- 172.21.15.7:0/841068752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651a34ddfa0 0x5651a34f3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:22.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.032+0000 7f34c3fff700 1 -- 172.21.15.7:0/841068752 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651a34e7c40 con 0x5651a34e36c0 2024-06-10T01:31:22.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.033+0000 7f34c3fff700 1 --2- 172.21.15.7:0/841068752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651a34e36c0 0x5651a34dda60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x5651a34e86d0 tx=0x7f34b809c580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e174424b298b4413 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:22.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.033+0000 7f34aa7fc700 1 -- 172.21.15.7:0/841068752 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b8026040 con 0x5651a34e36c0 2024-06-10T01:31:22.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.033+0000 7f34aa7fc700 1 -- 172.21.15.7:0/841068752 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34b800f460 con 0x5651a34e36c0 2024-06-10T01:31:22.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.033+0000 7f34aa7fc700 1 -- 172.21.15.7:0/841068752 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b809c7b0 con 0x5651a34e36c0 2024-06-10T01:31:22.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.034+0000 7f34d5ab1b80 1 -- 172.21.15.7:0/841068752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651a34e36c0 msgr2=0x5651a34dda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:22.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.034+0000 7f34d5ab1b80 1 --2- 172.21.15.7:0/841068752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651a34e36c0 0x5651a34dda60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x5651a34e86d0 tx=0x7f34b809c580 comp rx=0 tx=0).stop 2024-06-10T01:31:22.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.034+0000 7f34d5ab1b80 1 -- 172.21.15.7:0/841068752 shutdown_connections 2024-06-10T01:31:22.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.034+0000 7f34d5ab1b80 1 --2- 172.21.15.7:0/841068752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651a34e3fe0 0x5651a34e6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:22.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.034+0000 7f34d5ab1b80 1 --2- 172.21.15.7:0/841068752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651a34ddfa0 0x5651a34f3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:22.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.034+0000 7f34d5ab1b80 1 --2- 172.21.15.7:0/841068752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651a34e36c0 0x5651a34dda60 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:22.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.034+0000 7f34d5ab1b80 1 -- 172.21.15.7:0/841068752 >> 172.21.15.7:0/841068752 conn(0x5651a3408d40 msgr2=0x5651a34d1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:22.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.034+0000 7f34d5ab1b80 1 -- 172.21.15.7:0/841068752 shutdown_connections 2024-06-10T01:31:22.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.034+0000 7f34d5ab1b80 1 -- 172.21.15.7:0/841068752 wait complete. 2024-06-10T01:31:22.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.034+0000 7f34d5ab1b80 1 -- start start 2024-06-10T01:31:22.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.034+0000 7f34d5ab1b80 1 Processor -- start 2024-06-10T01:31:22.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.035+0000 7f34d5ab1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651a34ddfa0 0x5651a34f0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:22.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.035+0000 7f34d5ab1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651a34e36c0 0x5651a34f05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:22.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.035+0000 7f34d5ab1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651a34e3fe0 0x5651a34ece60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:22.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.035+0000 7f34d5ab1b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f34c41c3f70 con 0x5651a34e3fe0 2024-06-10T01:31:22.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.035+0000 7f34d5ab1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f34c41c53e0 con 0x5651a34ddfa0 2024-06-10T01:31:22.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.035+0000 7f34d5ab1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f34c41c4640 con 0x5651a34e36c0 2024-06-10T01:31:22.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.035+0000 7f34c8fca700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651a34ddfa0 0x5651a34f0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:22.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.035+0000 7f34d5ab1b80 10 client.0 _ll_register_callbacks cb 0x5651a33c99a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:31:22.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.035+0000 7f34c8fca700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651a34ddfa0 0x5651a34f0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:38726/0 (socket says 172.21.15.7:38726) 2024-06-10T01:31:22.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.035+0000 7f34c8fca700 1 -- 172.21.15.7:0/3837727826 learned_addr learned my addr 172.21.15.7:0/3837727826 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:22.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.035+0000 7f34c97cb700 1 --2- 172.21.15.7:0/3837727826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651a34e3fe0 0x5651a34ece60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:22.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.035+0000 7f34c3fff700 1 --2- 172.21.15.7:0/3837727826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651a34e36c0 0x5651a34f05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:22.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.035+0000 7f34c8fca700 1 -- 172.21.15.7:0/3837727826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651a34e36c0 msgr2=0x5651a34f05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:22.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.035+0000 7f34c8fca700 1 --2- 172.21.15.7:0/3837727826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651a34e36c0 0x5651a34f05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:22.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.036+0000 7f34c8fca700 1 -- 172.21.15.7:0/3837727826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651a34e3fe0 msgr2=0x5651a34ece60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:22.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.036+0000 7f34c8fca700 1 --2- 172.21.15.7:0/3837727826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651a34e3fe0 0x5651a34ece60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:22.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.036+0000 7f34c8fca700 1 -- 172.21.15.7:0/3837727826 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651a34ed6a0 con 0x5651a34ddfa0 2024-06-10T01:31:22.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.036+0000 7f34c8fca700 1 --2- 172.21.15.7:0/3837727826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651a34ddfa0 0x5651a34f0070 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f34b400bac0 tx=0x7f34b4005a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f56fd0972a135e8a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:22.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.036+0000 7f34a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:31:22.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.036+0000 7f34a3fff700 1 -- 172.21.15.7:0/3837727826 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b4014040 con 0x5651a34ddfa0 2024-06-10T01:31:22.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.036+0000 7f34d5ab1b80 1 -- 172.21.15.7:0/3837727826 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34b800f460 con 0x5651a34ddfa0 2024-06-10T01:31:22.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.036+0000 7f34a3fff700 1 -- 172.21.15.7:0/3837727826 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f34b4004fe0 con 0x5651a34ddfa0 2024-06-10T01:31:22.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.036+0000 7f34a3fff700 1 -- 172.21.15.7:0/3837727826 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f34b4005240 con 0x5651a34ddfa0 2024-06-10T01:31:22.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.036+0000 7f34d5ab1b80 10 client.30563 fetch_fsmap learned FSMap version 57 2024-06-10T01:31:22.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.036+0000 7f34d5ab1b80 1 -- 172.21.15.7:0/3837727826 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5651a3563330 con 0x5651a34ddfa0 2024-06-10T01:31:22.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.037+0000 7f34a3fff700 1 -- 172.21.15.7:0/3837727826 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34b800f460 con 0x5651a34ddfa0 2024-06-10T01:31:22.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.037+0000 7f34a3fff700 1 -- 172.21.15.7:0/3837727826 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7f34b4029020 con 0x5651a34ddfa0 2024-06-10T01:31:22.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.037+0000 7f34d5ab1b80 10 client.30563 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:31:22.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.037+0000 7f34d5ab1b80 10 client.30563 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:31:22.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.037+0000 7f34d5ab1b80 20 client.30563 populate_metadata read hostname 'smithi007' 2024-06-10T01:31:22.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.037+0000 7f34d5ab1b80 1 --2- 172.21.15.7:0/3837727826 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5651a35638d0 0x5651a3565cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:22.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.037+0000 7f34d5ab1b80 4 client.30563 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:31:22.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.037+0000 7f34d5ab1b80 1 -- 172.21.15.7:0/3837727826 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5651a3437310 con 0x5651a35638d0 2024-06-10T01:31:22.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.037+0000 7f34c3fff700 1 --2- 172.21.15.7:0/3837727826 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5651a35638d0 0x5651a3565cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:22.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.039+0000 7f34c3fff700 1 --2- 172.21.15.7:0/3837727826 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5651a35638d0 0x5651a3565cf0 crc :-1 s=READY pgs=562 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa00829c06ff67ab server_cookie=fcbd2b265bc70203 in_seq=0 out_seq=0 2024-06-10T01:31:22.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.039+0000 7f34a3fff700 10 client.30563 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:22.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.042+0000 7f34a3fff700 1 -- 172.21.15.7:0/3837727826 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5651a3437310 con 0x5651a35638d0 2024-06-10T01:31:22.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.042+0000 7f34a3fff700 10 client.30563 handle_command_reply: tid=0 2024-06-10T01:31:22.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.042+0000 7f34d5ab1b80 1 client.30563 shutdown 2024-06-10T01:31:22.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.043+0000 7f34d5ab1b80 1 -- 172.21.15.7:0/3837727826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651a34ddfa0 msgr2=0x5651a34f0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:22.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.043+0000 7f34d5ab1b80 1 --2- 172.21.15.7:0/3837727826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651a34ddfa0 0x5651a34f0070 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f34b400bac0 tx=0x7f34b4005a90 comp rx=0 tx=0).stop 2024-06-10T01:31:22.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.043+0000 7f34d5ab1b80 1 -- 172.21.15.7:0/3837727826 shutdown_connections 2024-06-10T01:31:22.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.043+0000 7f34d5ab1b80 1 --2- 172.21.15.7:0/3837727826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651a34e3fe0 0x5651a34ece60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:22.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.043+0000 7f34d5ab1b80 1 --2- 172.21.15.7:0/3837727826 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x5651a35638d0 0x5651a3565cf0 crc :-1 s=READY pgs=562 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:22.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.043+0000 7f34d5ab1b80 1 --2- 172.21.15.7:0/3837727826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651a34ddfa0 0x5651a34f0070 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:22.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.043+0000 7f34d5ab1b80 1 --2- 172.21.15.7:0/3837727826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5651a34e36c0 0x5651a34f05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:22.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.043+0000 7f34a3fff700 0 client.30563 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:22.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.043+0000 7f34d5ab1b80 1 -- 172.21.15.7:0/3837727826 >> 172.21.15.7:0/3837727826 conn(0x5651a3408d40 msgr2=0x5651a34e43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:22.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.043+0000 7f34d5ab1b80 1 -- 172.21.15.7:0/3837727826 shutdown_connections 2024-06-10T01:31:22.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.043+0000 7f34d5ab1b80 1 -- 172.21.15.7:0/3837727826 wait complete. 2024-06-10T01:31:22.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.043+0000 7f34d5ab1b80 20 client.30563 trim_cache size 0 max 16384 2024-06-10T01:31:22.037 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:31:22.037 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-10T01:31:22.037 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "e91dc0a6-8ffa-4cf1-b5df-9d13bace9eaa", 2024-06-10T01:31:22.037 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-10T01:31:22.037 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:31:22.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.044+0000 7f34cb22e700 1 -- 172.21.15.7:0/288756324 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f34ac0a2e20 msgr2=0x7f34ac0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:22.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.044+0000 7f34cb22e700 1 --2- 172.21.15.7:0/288756324 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f34ac0a2e20 0x7f34ac0a52e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f34c4073e30 tx=0x7f34bc009410 comp rx=0 tx=0).stop 2024-06-10T01:31:22.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.044+0000 7f34cb22e700 1 -- 172.21.15.7:0/288756324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34c406e910 msgr2=0x7f34c411b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:22.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.044+0000 7f34cb22e700 1 --2- 172.21.15.7:0/288756324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34c406e910 0x7f34c411b5c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009810 tx=0x7f34b800fc60 comp rx=0 tx=0).stop 2024-06-10T01:31:22.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.044+0000 7f34cb22e700 1 -- 172.21.15.7:0/288756324 shutdown_connections 2024-06-10T01:31:22.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.044+0000 7f34cb22e700 1 --2- 172.21.15.7:0/288756324 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f34ac0a2e20 0x7f34ac0a52e0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:22.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.044+0000 7f34cb22e700 1 --2- 172.21.15.7:0/288756324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34c411bb20 0x7f34c4114340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:22.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.044+0000 7f34cb22e700 1 --2- 172.21.15.7:0/288756324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34c406dff0 0x7f34c411b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:22.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.044+0000 7f34cb22e700 1 --2- 172.21.15.7:0/288756324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34c406e910 0x7f34c411b5c0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:22.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.044+0000 7f34cb22e700 1 -- 172.21.15.7:0/288756324 >> 172.21.15.7:0/288756324 conn(0x7f34c406d680 msgr2=0x7f34c40714a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:22.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.044+0000 7f34cb22e700 1 -- 172.21.15.7:0/288756324 shutdown_connections 2024-06-10T01:31:22.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:22.044+0000 7f34cb22e700 1 -- 172.21.15.7:0/288756324 wait complete. 2024-06-10T01:31:22.572 INFO:tasks.workunit.client.0.smithi007.stdout: 10 284618 21.96 MB/sec execute 599 sec latency 121.743 ms 2024-06-10T01:31:22.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:22 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:31:22.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:22 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:31:22.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:22 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:31:22.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:22 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:31:22.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:22 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:31:22.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:22 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:31:22.826 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:22 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:31:22.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:22 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:31:22.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:22 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:31:22.869 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:31:23.162 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:31:23.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:23 smithi007 ceph-mon[32398]: scrub queued for path: / 2024-06-10T01:31:23.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:23 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:31:23.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:23 smithi007 ceph-mon[32398]: scrub summary: active paths [/] 2024-06-10T01:31:23.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:23 smithi007 ceph-mon[32398]: pgmap v1195: 129 pgs: 129 active+clean; 707 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 8.8 MiB/s wr, 765 op/s 2024-06-10T01:31:23.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:23 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:31:23.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:23 smithi007 ceph-mon[32398]: scrub summary: idle+waiting paths [/] 2024-06-10T01:31:23.572 INFO:tasks.workunit.client.0.smithi007.stdout: 10 cleanup 600 sec 2024-06-10T01:31:23.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:23 smithi175 ceph-mon[36636]: scrub queued for path: / 2024-06-10T01:31:23.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:23 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:31:23.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:23 smithi175 ceph-mon[36636]: scrub summary: active paths [/] 2024-06-10T01:31:23.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:23 smithi175 ceph-mon[36636]: pgmap v1195: 129 pgs: 129 active+clean; 707 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 8.8 MiB/s wr, 765 op/s 2024-06-10T01:31:23.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:23 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:31:23.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:23 smithi175 ceph-mon[36636]: scrub summary: idle+waiting paths [/] 2024-06-10T01:31:23.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:23 smithi078 ceph-mon[36683]: scrub queued for path: / 2024-06-10T01:31:23.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:23 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:31:23.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:23 smithi078 ceph-mon[36683]: scrub summary: active paths [/] 2024-06-10T01:31:23.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:23 smithi078 ceph-mon[36683]: pgmap v1195: 129 pgs: 129 active+clean; 707 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 8.8 MiB/s wr, 765 op/s 2024-06-10T01:31:23.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:23 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:31:23.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:23 smithi078 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-06-10T01:31:24.407 INFO:tasks.workunit.client.0.smithi007.stdout: 0 cleanup 601 sec 2024-06-10T01:31:24.407 INFO:tasks.workunit.client.0.smithi007.stdout: 2024-06-10T01:31:24.407 INFO:tasks.workunit.client.0.smithi007.stdout: Operation Count AvgLat MaxLat 2024-06-10T01:31:24.407 INFO:tasks.workunit.client.0.smithi007.stdout: ---------------------------------------- 2024-06-10T01:31:24.407 INFO:tasks.workunit.client.0.smithi007.stdout: NTCreateX 423224 6.586 33415.783 2024-06-10T01:31:24.407 INFO:tasks.workunit.client.0.smithi007.stdout: Close 310708 0.003 0.175 2024-06-10T01:31:24.407 INFO:tasks.workunit.client.0.smithi007.stdout: Rename 17942 21.144 21161.509 2024-06-10T01:31:24.407 INFO:tasks.workunit.client.0.smithi007.stdout: Unlink 85597 7.729 33327.556 2024-06-10T01:31:24.407 INFO:tasks.workunit.client.0.smithi007.stdout: Qpathinfo 383840 3.122 1995.094 2024-06-10T01:31:24.407 INFO:tasks.workunit.client.0.smithi007.stdout: Qfileinfo 66896 0.003 20.389 2024-06-10T01:31:24.407 INFO:tasks.workunit.client.0.smithi007.stdout: Qfsinfo 70361 0.454 377.546 2024-06-10T01:31:24.408 INFO:tasks.workunit.client.0.smithi007.stdout: Sfileinfo 34494 4.170 20200.411 2024-06-10T01:31:24.408 INFO:tasks.workunit.client.0.smithi007.stdout: Find 148274 0.295 661.475 2024-06-10T01:31:24.408 INFO:tasks.workunit.client.0.smithi007.stdout: WriteX 209071 0.457 32873.356 2024-06-10T01:31:24.408 INFO:tasks.workunit.client.0.smithi007.stdout: ReadX 663806 0.103 1348.220 2024-06-10T01:31:24.408 INFO:tasks.workunit.client.0.smithi007.stdout: LockX 1380 3.334 33.777 2024-06-10T01:31:24.408 INFO:tasks.workunit.client.0.smithi007.stdout: UnlockX 1380 2.881 50.341 2024-06-10T01:31:24.408 INFO:tasks.workunit.client.0.smithi007.stdout: Flush 29655 19.407 2606.965 2024-06-10T01:31:24.408 INFO:tasks.workunit.client.0.smithi007.stdout: 2024-06-10T01:31:24.408 INFO:tasks.workunit.client.0.smithi007.stdout:Throughput 21.9555 MB/sec 10 clients 10 procs max_latency=33415.786 ms 2024-06-10T01:31:24.408 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-10T01:31:24.408 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-10T01:31:25.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.661+0000 7fe432415640 1 -- 172.21.15.7:0/3901011658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe42c10b310 msgr2=0x7fe42c10d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:25.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.661+0000 7fe432415640 1 --2- 172.21.15.7:0/3901011658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe42c10b310 0x7fe42c10d780 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fe424009ca0 tx=0x7fe42402a8f0 comp rx=0 tx=0).stop 2024-06-10T01:31:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.668+0000 7fe432415640 1 -- 172.21.15.7:0/3901011658 shutdown_connections 2024-06-10T01:31:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.668+0000 7fe432415640 1 --2- 172.21.15.7:0/3901011658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe42c10dcc0 0x7fe42c10e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.668+0000 7fe432415640 1 --2- 172.21.15.7:0/3901011658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe42c10b310 0x7fe42c10d780 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.668+0000 7fe432415640 1 --2- 172.21.15.7:0/3901011658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe42c1089a0 0x7fe42c10add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.668+0000 7fe432415640 1 -- 172.21.15.7:0/3901011658 >> 172.21.15.7:0/3901011658 conn(0x7fe42c06e9e0 msgr2=0x7fe42c070e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.672+0000 7fe432415640 1 -- 172.21.15.7:0/3901011658 shutdown_connections 2024-06-10T01:31:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.683+0000 7fe432415640 1 -- 172.21.15.7:0/3901011658 wait complete. 2024-06-10T01:31:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.683+0000 7fe432415640 1 Processor -- start 2024-06-10T01:31:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.685+0000 7fe432415640 1 -- start start 2024-06-10T01:31:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.686+0000 7fe432415640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe42c1089a0 0x7fe42c117030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.686+0000 7fe432415640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe42c10b310 0x7fe42c117570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.686+0000 7fe432415640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe42c10dcc0 0x7fe42c1b6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.686+0000 7fe432415640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe42c11c4b0 con 0x7fe42c10dcc0 2024-06-10T01:31:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.686+0000 7fe432415640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe42c11c330 con 0x7fe42c1089a0 2024-06-10T01:31:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.686+0000 7fe432415640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe42c11c630 con 0x7fe42c10b310 2024-06-10T01:31:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.686+0000 7fe42bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe42c1089a0 0x7fe42c117030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.686+0000 7fe42bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe42c1089a0 0x7fe42c117030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:38742/0 (socket says 172.21.15.7:38742) 2024-06-10T01:31:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.686+0000 7fe42bfff640 1 -- 172.21.15.7:0/3322209114 learned_addr learned my addr 172.21.15.7:0/3322209114 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.686+0000 7fe42bfff640 1 -- 172.21.15.7:0/3322209114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe42c10b310 msgr2=0x7fe42c117570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T01:31:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.686+0000 7fe42bfff640 1 --2- 172.21.15.7:0/3322209114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe42c10b310 0x7fe42c117570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.686+0000 7fe42bfff640 1 -- 172.21.15.7:0/3322209114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe42c10dcc0 msgr2=0x7fe42c1b6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.686+0000 7fe42bfff640 1 --2- 172.21.15.7:0/3322209114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe42c10dcc0 0x7fe42c1b6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.686+0000 7fe42bfff640 1 -- 172.21.15.7:0/3322209114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe42c1b70e0 con 0x7fe42c1089a0 2024-06-10T01:31:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.687+0000 7fe42bfff640 1 --2- 172.21.15.7:0/3322209114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe42c1089a0 0x7fe42c117030 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe42000d950 tx=0x7fe42000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.691+0000 7fe4297fa640 1 -- 172.21.15.7:0/3322209114 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe420017070 con 0x7fe42c1089a0 2024-06-10T01:31:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.691+0000 7fe432415640 1 -- 172.21.15.7:0/3322209114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe42c1b72b0 con 0x7fe42c1089a0 2024-06-10T01:31:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.691+0000 7fe432415640 1 -- 172.21.15.7:0/3322209114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe42c1b7770 con 0x7fe42c1089a0 2024-06-10T01:31:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.691+0000 7fe4297fa640 1 -- 172.21.15.7:0/3322209114 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe420009c30 con 0x7fe42c1089a0 2024-06-10T01:31:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.691+0000 7fe4297fa640 1 -- 172.21.15.7:0/3322209114 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42000f6b0 con 0x7fe42c1089a0 2024-06-10T01:31:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.695+0000 7fe435202740 1 Processor -- start 2024-06-10T01:31:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.696+0000 7fe435202740 1 -- start start 2024-06-10T01:31:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.696+0000 7fe435202740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8da05010 0x563f8da05430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.696+0000 7fe435202740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f8da05a00 0x563f8da0a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.696+0000 7fe435202740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f8da0ab70 0x563f8da0d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.696+0000 7fe435202740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563f8d9684f0 con 0x563f8da0ab70 2024-06-10T01:31:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.696+0000 7fe435202740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563f8d906870 con 0x563f8da05010 2024-06-10T01:31:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.696+0000 7fe435202740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563f8d9644b0 con 0x563f8da05a00 2024-06-10T01:31:25.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.697+0000 7fe4309b2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8da05010 0x563f8da05430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:25.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.697+0000 7fe4309b2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8da05010 0x563f8da05430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:38746/0 (socket says 172.21.15.7:38746) 2024-06-10T01:31:25.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.697+0000 7fe4309b2640 1 -- 172.21.15.7:0/951156935 learned_addr learned my addr 172.21.15.7:0/951156935 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:25.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.699+0000 7fe42b7fe640 1 --2- 172.21.15.7:0/951156935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f8da0ab70 0x563f8da0d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:25.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.700+0000 7fe4309b2640 1 -- 172.21.15.7:0/951156935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f8da05a00 msgr2=0x563f8da0a500 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:31:25.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.700+0000 7fe4309b2640 1 --2- 172.21.15.7:0/951156935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f8da05a00 0x563f8da0a500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:25.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.700+0000 7fe4309b2640 1 -- 172.21.15.7:0/951156935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f8da0ab70 msgr2=0x563f8da0d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:25.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.700+0000 7fe4309b2640 1 --2- 172.21.15.7:0/951156935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f8da0ab70 0x563f8da0d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:25.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.700+0000 7fe4309b2640 1 -- 172.21.15.7:0/951156935 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f8d9229b0 con 0x563f8da05010 2024-06-10T01:31:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.703+0000 7fe4297fa640 1 -- 172.21.15.7:0/3322209114 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fe4200109d0 con 0x7fe42c1089a0 2024-06-10T01:31:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.704+0000 7fe4309b2640 1 --2- 172.21.15.7:0/951156935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8da05010 0x563f8da05430 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe41c009c40 tx=0x7fe41c00e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fa77dce5bcc305 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.704+0000 7fe4297fa640 1 --2- 172.21.15.7:0/3322209114 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe4140a2d90 0x7fe4140a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.704+0000 7fe4297fa640 1 -- 172.21.15.7:0/3322209114 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fe4200cf000 con 0x7fe42c1089a0 2024-06-10T01:31:25.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:25 smithi175 ceph-mon[36636]: pgmap v1196: 129 pgs: 129 active+clean; 708 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.2 MiB/s wr, 565 op/s 2024-06-10T01:31:25.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:25 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:31:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.706+0000 7fe42bfff640 1 --2- 172.21.15.7:0/3322209114 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe4140a2d90 0x7fe4140a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.707+0000 7fe4127fc640 1 -- 172.21.15.7:0/951156935 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c005020 con 0x563f8da05010 2024-06-10T01:31:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.711+0000 7fe4127fc640 1 -- 172.21.15.7:0/951156935 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe41c0047a0 con 0x563f8da05010 2024-06-10T01:31:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.712+0000 7fe435202740 1 -- 172.21.15.7:0/951156935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8da05010 msgr2=0x563f8da05430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.712+0000 7fe435202740 1 --2- 172.21.15.7:0/951156935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8da05010 0x563f8da05430 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe41c009c40 tx=0x7fe41c00e720 comp rx=0 tx=0).stop 2024-06-10T01:31:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.716+0000 7fe435202740 1 -- 172.21.15.7:0/951156935 shutdown_connections 2024-06-10T01:31:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.716+0000 7fe435202740 1 --2- 172.21.15.7:0/951156935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f8da0ab70 0x563f8da0d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.716+0000 7fe435202740 1 --2- 172.21.15.7:0/951156935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f8da05a00 0x563f8da0a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.716+0000 7fe435202740 1 --2- 172.21.15.7:0/951156935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8da05010 0x563f8da05430 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.716+0000 7fe435202740 1 -- 172.21.15.7:0/951156935 >> 172.21.15.7:0/951156935 conn(0x563f8da00680 msgr2=0x563f8da00a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.717+0000 7fe435202740 1 -- 172.21.15.7:0/951156935 shutdown_connections 2024-06-10T01:31:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.717+0000 7fe435202740 1 -- 172.21.15.7:0/951156935 wait complete. 2024-06-10T01:31:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.718+0000 7fe435202740 1 -- start start 2024-06-10T01:31:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.718+0000 7fe435202740 1 Processor -- start 2024-06-10T01:31:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.718+0000 7fe435202740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f8da05010 0x563f8da0f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.718+0000 7fe435202740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8da05a00 0x563f8da0f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.718+0000 7fe435202740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f8da0ab70 0x563f8da167e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.718+0000 7fe435202740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563f8da0db70 con 0x563f8da0ab70 2024-06-10T01:31:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.718+0000 7fe435202740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563f8d8f1ae0 con 0x563f8da05a00 2024-06-10T01:31:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.718+0000 7fe435202740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563f8da0dcf0 con 0x563f8da05010 2024-06-10T01:31:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.719+0000 7fe435202740 10 client.0 _ll_register_callbacks cb 0x563f8d92a9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:31:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.719+0000 7fe42bfff640 1 --2- 172.21.15.7:0/3322209114 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe4140a2d90 0x7fe4140a5250 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe420097e50 tx=0x7fe420005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.719+0000 7fe4309b2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f8da0ab70 0x563f8da167e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.719+0000 7fe4309b2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f8da0ab70 0x563f8da167e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44652/0 (socket says 172.21.15.7:44652) 2024-06-10T01:31:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.719+0000 7fe4309b2640 1 -- 172.21.15.7:0/3378227918 learned_addr learned my addr 172.21.15.7:0/3378227918 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.720+0000 7fe42b7fe640 1 --2- 172.21.15.7:0/3378227918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8da05a00 0x563f8da0f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.720+0000 7fe4309b2640 1 -- 172.21.15.7:0/3378227918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f8da05010 msgr2=0x563f8da0f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.720+0000 7fe4309b2640 1 --2- 172.21.15.7:0/3378227918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f8da05010 0x563f8da0f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.720+0000 7fe4309b2640 1 -- 172.21.15.7:0/3378227918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8da05a00 msgr2=0x563f8da0f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.720+0000 7fe4309b2640 1 --2- 172.21.15.7:0/3378227918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8da05a00 0x563f8da0f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.720+0000 7fe4309b2640 1 -- 172.21.15.7:0/3378227918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f8d67c210 con 0x563f8da0ab70 2024-06-10T01:31:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.720+0000 7fe4309b2640 1 --2- 172.21.15.7:0/3378227918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f8da0ab70 0x563f8da167e0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fe41c009560 tx=0x7fe41c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406006033ed68e7f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.720+0000 7fe40b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:31:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.721+0000 7fe40b7fe640 1 -- 172.21.15.7:0/3378227918 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c005370 con 0x563f8da0ab70 2024-06-10T01:31:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.721+0000 7fe40b7fe640 1 -- 172.21.15.7:0/3378227918 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fe41c005510 con 0x563f8da0ab70 2024-06-10T01:31:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.721+0000 7fe435202740 1 -- 172.21.15.7:0/3378227918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe41c0047a0 con 0x563f8da0ab70 2024-06-10T01:31:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.721+0000 7fe40b7fe640 1 -- 172.21.15.7:0/3378227918 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fe41c0108f0 con 0x563f8da0ab70 2024-06-10T01:31:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.722+0000 7fe435202740 10 client.21402 fetch_fsmap learned FSMap version 57 2024-06-10T01:31:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.722+0000 7fe435202740 1 -- 172.21.15.7:0/3378227918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x563f8d748620 con 0x563f8da0ab70 2024-06-10T01:31:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.723+0000 7fe40b7fe640 1 -- 172.21.15.7:0/3378227918 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe41c0047a0 con 0x563f8da0ab70 2024-06-10T01:31:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.723+0000 7fe40b7fe640 1 -- 172.21.15.7:0/3378227918 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7fe41c018c90 con 0x563f8da0ab70 2024-06-10T01:31:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.723+0000 7fe435202740 10 client.21402 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:31:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.723+0000 7fe435202740 10 client.21402 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:31:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.723+0000 7fe435202740 20 client.21402 populate_metadata read hostname 'smithi007' 2024-06-10T01:31:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.723+0000 7fe435202740 1 --2- 172.21.15.7:0/3378227918 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563f8da3cdc0 0x563f8da3f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.723+0000 7fe435202740 4 client.21402 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:31:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.723+0000 7fe435202740 1 -- 172.21.15.7:0/3378227918 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f8d97bf10 con 0x563f8da3cdc0 2024-06-10T01:31:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.724+0000 7fe42b7fe640 1 --2- 172.21.15.7:0/3378227918 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563f8da3cdc0 0x563f8da3f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.729+0000 7fe42b7fe640 1 --2- 172.21.15.7:0/3378227918 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563f8da3cdc0 0x563f8da3f1b0 crc :-1 s=READY pgs=564 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbe75355508638dc server_cookie=30c7b46b4b0ddee9 in_seq=0 out_seq=0 2024-06-10T01:31:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.729+0000 7fe40b7fe640 10 client.21402 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.739+0000 7fe40b7fe640 1 -- 172.21.15.7:0/3378227918 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563f8d97bf10 con 0x563f8da3cdc0 2024-06-10T01:31:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.739+0000 7fe40b7fe640 10 client.21402 handle_command_reply: tid=0 2024-06-10T01:31:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.739+0000 7fe435202740 1 client.21402 shutdown 2024-06-10T01:31:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.745+0000 7fe435202740 1 -- 172.21.15.7:0/3378227918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f8da0ab70 msgr2=0x563f8da167e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.745+0000 7fe435202740 1 --2- 172.21.15.7:0/3378227918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f8da0ab70 0x563f8da167e0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fe41c009560 tx=0x7fe41c009f60 comp rx=0 tx=0).stop 2024-06-10T01:31:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.745+0000 7fe435202740 1 -- 172.21.15.7:0/3378227918 shutdown_connections 2024-06-10T01:31:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.745+0000 7fe435202740 1 --2- 172.21.15.7:0/3378227918 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563f8da3cdc0 0x563f8da3f1b0 crc :-1 s=READY pgs=564 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.745+0000 7fe435202740 1 --2- 172.21.15.7:0/3378227918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f8da0ab70 0x563f8da167e0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.745+0000 7fe435202740 1 --2- 172.21.15.7:0/3378227918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8da05a00 0x563f8da0f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.745+0000 7fe435202740 1 --2- 172.21.15.7:0/3378227918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f8da05010 0x563f8da0f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.745+0000 7fe435202740 1 -- 172.21.15.7:0/3378227918 >> 172.21.15.7:0/3378227918 conn(0x563f8da00680 msgr2=0x563f8d9ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.746+0000 7fe40b7fe640 0 client.21402 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.746+0000 7fe435202740 1 -- 172.21.15.7:0/3378227918 shutdown_connections 2024-06-10T01:31:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.747+0000 7fe435202740 1 -- 172.21.15.7:0/3378227918 wait complete. 2024-06-10T01:31:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.747+0000 7fe435202740 20 client.21402 trim_cache size 0 max 16384 2024-06-10T01:31:25.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:25 smithi007 ceph-mon[32398]: pgmap v1196: 129 pgs: 129 active+clean; 708 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.2 MiB/s wr, 565 op/s 2024-06-10T01:31:25.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:25 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:31:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.797+0000 7fe435202740 1 Processor -- start 2024-06-10T01:31:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.797+0000 7fe435202740 1 -- start start 2024-06-10T01:31:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.798+0000 7fe435202740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f8da05010 0x563f8da0e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.798+0000 7fe435202740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8da05a00 0x563f8da167a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.798+0000 7fe4309b2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f8da05010 0x563f8da0e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.798+0000 7fe4309b2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f8da05010 0x563f8da0e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:42368/0 (socket says 172.21.15.7:42368) 2024-06-10T01:31:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.798+0000 7fe4309b2640 1 -- 172.21.15.7:0/727913501 learned_addr learned my addr 172.21.15.7:0/727913501 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.798+0000 7fe42b7fe640 1 --2- 172.21.15.7:0/727913501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8da05a00 0x563f8da167a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.798+0000 7fe435202740 1 --2- 172.21.15.7:0/727913501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f8da0ab70 0x563f8d9fff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.798+0000 7fe435202740 1 -- 172.21.15.7:0/727913501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe4180125d0 con 0x563f8da0ab70 2024-06-10T01:31:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.798+0000 7fe435202740 1 -- 172.21.15.7:0/727913501 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe41800fe40 con 0x563f8da05a00 2024-06-10T01:31:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.799+0000 7fe435202740 1 -- 172.21.15.7:0/727913501 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe41800dc90 con 0x563f8da05010 2024-06-10T01:31:25.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.803+0000 7fe4309b2640 1 -- 172.21.15.7:0/727913501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8da05a00 msgr2=0x563f8da167a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:25.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.803+0000 7fe4309b2640 1 --2- 172.21.15.7:0/727913501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8da05a00 0x563f8da167a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:25.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.803+0000 7fe4309b2640 1 -- 172.21.15.7:0/727913501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f8da0ab70 msgr2=0x563f8d9fff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:25.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.803+0000 7fe4309b2640 1 --2- 172.21.15.7:0/727913501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f8da0ab70 0x563f8d9fff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:25.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.803+0000 7fe4309b2640 1 -- 172.21.15.7:0/727913501 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f8da004a0 con 0x563f8da05010 2024-06-10T01:31:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.805+0000 7fe4309b2640 1 --2- 172.21.15.7:0/727913501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f8da05010 0x563f8da0e880 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe41c010850 tx=0x7fe41c010880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ba8dacf26c93592 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.810+0000 7fe411ffb640 1 -- 172.21.15.7:0/727913501 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c0047a0 con 0x563f8da05010 2024-06-10T01:31:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.810+0000 7fe411ffb640 1 -- 172.21.15.7:0/727913501 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe41c028070 con 0x563f8da05010 2024-06-10T01:31:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.811+0000 7fe435202740 1 -- 172.21.15.7:0/727913501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f8da05010 msgr2=0x563f8da0e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.811+0000 7fe435202740 1 --2- 172.21.15.7:0/727913501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f8da05010 0x563f8da0e880 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe41c010850 tx=0x7fe41c010880 comp rx=0 tx=0).stop 2024-06-10T01:31:25.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.813+0000 7fe435202740 1 -- 172.21.15.7:0/727913501 shutdown_connections 2024-06-10T01:31:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.813+0000 7fe435202740 1 --2- 172.21.15.7:0/727913501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f8da0ab70 0x563f8d9fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.813+0000 7fe435202740 1 --2- 172.21.15.7:0/727913501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8da05a00 0x563f8da167a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.813+0000 7fe435202740 1 --2- 172.21.15.7:0/727913501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f8da05010 0x563f8da0e880 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.813+0000 7fe435202740 1 -- 172.21.15.7:0/727913501 >> 172.21.15.7:0/727913501 conn(0x563f8da00680 msgr2=0x563f8da0e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.816+0000 7fe435202740 1 -- 172.21.15.7:0/727913501 shutdown_connections 2024-06-10T01:31:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.817+0000 7fe435202740 1 -- 172.21.15.7:0/727913501 wait complete. 2024-06-10T01:31:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.818+0000 7fe435202740 1 -- start start 2024-06-10T01:31:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.818+0000 7fe435202740 1 Processor -- start 2024-06-10T01:31:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.819+0000 7fe435202740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f8da05010 0x563f8da13700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.819+0000 7fe435202740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8da05a00 0x563f8da13c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.819+0000 7fe4309b2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f8da05010 0x563f8da13700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.819+0000 7fe4309b2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f8da05010 0x563f8da13700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:42384/0 (socket says 172.21.15.7:42384) 2024-06-10T01:31:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.819+0000 7fe4309b2640 1 -- 172.21.15.7:0/1401816655 learned_addr learned my addr 172.21.15.7:0/1401816655 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.819+0000 7fe42b7fe640 1 --2- 172.21.15.7:0/1401816655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8da05a00 0x563f8da13c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.819+0000 7fe435202740 1 --2- 172.21.15.7:0/1401816655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f8da0ab70 0x563f8daa59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.819+0000 7fe435202740 1 -- 172.21.15.7:0/1401816655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe41800e300 con 0x563f8da0ab70 2024-06-10T01:31:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.820+0000 7fe435202740 1 -- 172.21.15.7:0/1401816655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe41800f770 con 0x563f8da05a00 2024-06-10T01:31:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.820+0000 7fe435202740 1 -- 172.21.15.7:0/1401816655 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe41800e9d0 con 0x563f8da05010 2024-06-10T01:31:25.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.820+0000 7fe4309b2640 1 -- 172.21.15.7:0/1401816655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8da05a00 msgr2=0x563f8da13c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:25.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.820+0000 7fe4309b2640 1 --2- 172.21.15.7:0/1401816655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8da05a00 0x563f8da13c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:25.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.820+0000 7fe4309b2640 1 -- 172.21.15.7:0/1401816655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f8da0ab70 msgr2=0x563f8daa59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.820+0000 7fe4309b2640 1 --2- 172.21.15.7:0/1401816655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f8da0ab70 0x563f8daa59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.820+0000 7fe4309b2640 1 -- 172.21.15.7:0/1401816655 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f8daa5f20 con 0x563f8da05010 2024-06-10T01:31:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.820+0000 7fe4309b2640 1 --2- 172.21.15.7:0/1401816655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f8da05010 0x563f8da13700 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe41c00e760 tx=0x7fe41c009790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da41572e86836f4e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.821+0000 7fe435202740 10 client.0 _ll_register_callbacks cb 0x7fe41c009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:31:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.821+0000 7fe435202740 1 -- 172.21.15.7:0/1401816655 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe41c028070 con 0x563f8da05010 2024-06-10T01:31:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.821+0000 7fe40b7fe640 10 client.30439 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:31:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.821+0000 7fe40b7fe640 1 -- 172.21.15.7:0/1401816655 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c01d040 con 0x563f8da05010 2024-06-10T01:31:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.821+0000 7fe40b7fe640 1 -- 172.21.15.7:0/1401816655 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fe41c004030 con 0x563f8da05010 2024-06-10T01:31:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.821+0000 7fe40b7fe640 1 -- 172.21.15.7:0/1401816655 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fe41c0047a0 con 0x563f8da05010 2024-06-10T01:31:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.827+0000 7fe435202740 10 client.30439 fetch_fsmap learned FSMap version 57 2024-06-10T01:31:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.827+0000 7fe435202740 1 -- 172.21.15.7:0/1401816655 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({fsmap=57}) -- 0x563f8daa70a0 con 0x563f8da05010 2024-06-10T01:31:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.828+0000 7fe40b7fe640 1 -- 172.21.15.7:0/1401816655 <== mon.1 v2:172.21.15.175:3300/0 4 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7fe41c004250 con 0x563f8da05010 2024-06-10T01:31:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.828+0000 7fe435202740 10 client.30439 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:31:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.828+0000 7fe435202740 10 client.30439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:31:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.828+0000 7fe435202740 20 client.30439 populate_metadata read hostname 'smithi007' 2024-06-10T01:31:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.828+0000 7fe435202740 1 --2- 172.21.15.7:0/1401816655 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563f8daa79f0 0x563f8daa9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.828+0000 7fe435202740 4 client.30439 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:31:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.828+0000 7fe435202740 1 -- 172.21.15.7:0/1401816655 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fe41c028070 con 0x563f8daa79f0 2024-06-10T01:31:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.829+0000 7fe42b7fe640 1 --2- 172.21.15.7:0/1401816655 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563f8daa79f0 0x563f8daa9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:25.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:25 smithi078 ceph-mon[36683]: pgmap v1196: 129 pgs: 129 active+clean; 708 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.2 MiB/s wr, 565 op/s 2024-06-10T01:31:25.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:25 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:31:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.838+0000 7fe42b7fe640 1 --2- 172.21.15.7:0/1401816655 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563f8daa79f0 0x563f8daa9e10 crc :-1 s=READY pgs=565 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e642be9c36d8ab6b server_cookie=14ec1cac5eece67a in_seq=0 out_seq=0 2024-06-10T01:31:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:25.839+0000 7fe40b7fe640 10 client.30439 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:26.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.341+0000 7ff5ca67a700 1 -- 172.21.15.7:0/1723011031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5c4110c70 msgr2=0x7ff5c4109490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:26.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.341+0000 7ff5ca67a700 1 --2- 172.21.15.7:0/1723011031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5c4110c70 0x7ff5c4109490 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4009990 tx=0x7ff5b4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.343+0000 7ff5ca67a700 1 -- 172.21.15.7:0/1723011031 shutdown_connections 2024-06-10T01:31:26.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.343+0000 7ff5ca67a700 1 --2- 172.21.15.7:0/1723011031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5c411bc70 0x7ff5c411e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.343+0000 7ff5ca67a700 1 --2- 172.21.15.7:0/1723011031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5c4110c70 0x7ff5c4109490 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.343+0000 7ff5ca67a700 1 --2- 172.21.15.7:0/1723011031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5c41099d0 0x7ff5c4109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.343+0000 7ff5ca67a700 1 -- 172.21.15.7:0/1723011031 >> 172.21.15.7:0/1723011031 conn(0x7ff5c406d600 msgr2=0x7ff5c406da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:26.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.343+0000 7ff5ca67a700 1 -- 172.21.15.7:0/1723011031 shutdown_connections 2024-06-10T01:31:26.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.343+0000 7ff5ca67a700 1 -- 172.21.15.7:0/1723011031 wait complete. 2024-06-10T01:31:26.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.345+0000 7ff5ca67a700 1 Processor -- start 2024-06-10T01:31:26.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.345+0000 7ff5ca67a700 1 -- start start 2024-06-10T01:31:26.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.345+0000 7ff5ca67a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5c41099d0 0x7ff5c41ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:26.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.345+0000 7ff5ca67a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5c4110c70 0x7ff5c41add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:26.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.345+0000 7ff5ca67a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5c411bc70 0x7ff5c41a7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:26.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.345+0000 7ff5ca67a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c41217c0 con 0x7ff5c4110c70 2024-06-10T01:31:26.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.345+0000 7ff5ca67a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c4121640 con 0x7ff5c41099d0 2024-06-10T01:31:26.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.345+0000 7ff5ca67a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c4121940 con 0x7ff5c411bc70 2024-06-10T01:31:26.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.350+0000 7ff5c8c17700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5c411bc70 0x7ff5c41a7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:26.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.350+0000 7ff5c8c17700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5c411bc70 0x7ff5c41a7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:42404/0 (socket says 172.21.15.7:42404) 2024-06-10T01:31:26.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.350+0000 7ff5c8c17700 1 -- 172.21.15.7:0/3908334424 learned_addr learned my addr 172.21.15.7:0/3908334424 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:26.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.350+0000 7ff5c3fff700 1 --2- 172.21.15.7:0/3908334424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5c41099d0 0x7ff5c41ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:26.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.350+0000 7ff5c8c17700 1 -- 172.21.15.7:0/3908334424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5c41099d0 msgr2=0x7ff5c41ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.350+0000 7ff5c8c17700 1 --2- 172.21.15.7:0/3908334424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5c41099d0 0x7ff5c41ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.350+0000 7ff5c8c17700 1 -- 172.21.15.7:0/3908334424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5c4110c70 msgr2=0x7ff5c41add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.350+0000 7ff5c8c17700 1 --2- 172.21.15.7:0/3908334424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5c4110c70 0x7ff5c41add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.350+0000 7ff5c8c17700 1 -- 172.21.15.7:0/3908334424 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c41a8160 con 0x7ff5c411bc70 2024-06-10T01:31:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.351+0000 7ff5c8c17700 1 --2- 172.21.15.7:0/3908334424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5c411bc70 0x7ff5c41a7a00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00b9e0 tx=0x7ff5bc00bcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.351+0000 7ff5c17fa700 1 -- 172.21.15.7:0/3908334424 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc019070 con 0x7ff5c411bc70 2024-06-10T01:31:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.351+0000 7ff5ca67a700 1 -- 172.21.15.7:0/3908334424 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c41a8450 con 0x7ff5c411bc70 2024-06-10T01:31:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.351+0000 7ff5ca67a700 1 -- 172.21.15.7:0/3908334424 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c41b4710 con 0x7ff5c411bc70 2024-06-10T01:31:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.351+0000 7ff5c17fa700 1 -- 172.21.15.7:0/3908334424 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5bc00ce10 con 0x7ff5c411bc70 2024-06-10T01:31:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.351+0000 7ff5c17fa700 1 -- 172.21.15.7:0/3908334424 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc012340 con 0x7ff5c411bc70 2024-06-10T01:31:26.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.355+0000 7ff5c17fa700 1 -- 172.21.15.7:0/3908334424 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7ff5bc01c070 con 0x7ff5c411bc70 2024-06-10T01:31:26.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.357+0000 7ff5c17fa700 1 --2- 172.21.15.7:0/3908334424 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff5ac0a2cf0 0x7ff5ac0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:26.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.358+0000 7ff5c17fa700 1 -- 172.21.15.7:0/3908334424 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7ff5bc0cf720 con 0x7ff5c411bc70 2024-06-10T01:31:26.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.361+0000 7ff5c3fff700 1 --2- 172.21.15.7:0/3908334424 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff5ac0a2cf0 0x7ff5ac0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:26.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.364+0000 7ff5d4efdb80 1 Processor -- start 2024-06-10T01:31:26.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.367+0000 7ff5d4efdb80 1 -- start start 2024-06-10T01:31:26.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.367+0000 7ff5d4efdb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e46eb5640 0x563e46eb5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:26.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.367+0000 7ff5d4efdb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e46eb5f60 0x563e46eaf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:26.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.367+0000 7ff5d4efdb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e46eaff20 0x563e46eb0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:26.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.367+0000 7ff5d4efdb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563e46e01810 con 0x563e46eaff20 2024-06-10T01:31:26.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.367+0000 7ff5d4efdb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563e46da9720 con 0x563e46eb5640 2024-06-10T01:31:26.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.367+0000 7ff5d4efdb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563e46dfd7d0 con 0x563e46eb5f60 2024-06-10T01:31:26.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.368+0000 7ff5c3fff700 1 --2- 172.21.15.7:0/3908334424 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff5ac0a2cf0 0x7ff5ac0a51b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4009750 tx=0x7ff5b4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:26.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.369+0000 7ff5c37fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e46eb5f60 0x563e46eaf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:26.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.372+0000 7ff5c3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e46eb5640 0x563e46eb5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:26.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.372+0000 7ff5c3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e46eb5640 0x563e46eb5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:38814/0 (socket says 172.21.15.7:38814) 2024-06-10T01:31:26.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.372+0000 7ff5c3fff700 1 -- 172.21.15.7:0/1100642799 learned_addr learned my addr 172.21.15.7:0/1100642799 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:26.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.373+0000 7ff5c8c17700 1 --2- 172.21.15.7:0/1100642799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e46eaff20 0x563e46eb0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:26.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.379+0000 7ff5c37fe700 1 -- 172.21.15.7:0/1100642799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e46eb5640 msgr2=0x563e46eb5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:26.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.379+0000 7ff5c37fe700 1 --2- 172.21.15.7:0/1100642799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e46eb5640 0x563e46eb5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.379+0000 7ff5c37fe700 1 -- 172.21.15.7:0/1100642799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e46eaff20 msgr2=0x563e46eb0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:26.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.379+0000 7ff5c37fe700 1 --2- 172.21.15.7:0/1100642799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e46eaff20 0x563e46eb0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.379+0000 7ff5c37fe700 1 -- 172.21.15.7:0/1100642799 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e46d9ff50 con 0x563e46eb5f60 2024-06-10T01:31:26.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.387+0000 7ff5c37fe700 1 --2- 172.21.15.7:0/1100642799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e46eb5f60 0x563e46eaf9e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff5b80097c0 tx=0x7ff5b8009ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24ec78aaae1ac9f5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:26.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.391+0000 7ff5aa7fc700 1 -- 172.21.15.7:0/1100642799 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8015020 con 0x563e46eb5f60 2024-06-10T01:31:26.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.391+0000 7ff5aa7fc700 1 -- 172.21.15.7:0/1100642799 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5b8011d10 con 0x563e46eb5f60 2024-06-10T01:31:26.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.392+0000 7ff5d4efdb80 1 -- 172.21.15.7:0/1100642799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e46eb5f60 msgr2=0x563e46eaf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:26.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.392+0000 7ff5d4efdb80 1 --2- 172.21.15.7:0/1100642799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e46eb5f60 0x563e46eaf9e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff5b80097c0 tx=0x7ff5b8009ad0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.393+0000 7ff5d4efdb80 1 -- 172.21.15.7:0/1100642799 shutdown_connections 2024-06-10T01:31:26.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.393+0000 7ff5d4efdb80 1 --2- 172.21.15.7:0/1100642799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e46eaff20 0x563e46eb0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.393+0000 7ff5d4efdb80 1 --2- 172.21.15.7:0/1100642799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e46eb5640 0x563e46eb5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.393+0000 7ff5d4efdb80 1 --2- 172.21.15.7:0/1100642799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e46eb5f60 0x563e46eaf9e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.393+0000 7ff5d4efdb80 1 -- 172.21.15.7:0/1100642799 >> 172.21.15.7:0/1100642799 conn(0x563e46ddacc0 msgr2=0x563e46eaa850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:26.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.398+0000 7ff5d4efdb80 1 -- 172.21.15.7:0/1100642799 shutdown_connections 2024-06-10T01:31:26.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.399+0000 7ff5d4efdb80 1 -- 172.21.15.7:0/1100642799 wait complete. 2024-06-10T01:31:26.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.400+0000 7ff5d4efdb80 1 -- start start 2024-06-10T01:31:26.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.400+0000 7ff5d4efdb80 1 Processor -- start 2024-06-10T01:31:26.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.404+0000 7ff5d4efdb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e46eaff20 0x563e46ebbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:26.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.404+0000 7ff5d4efdb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e46eb5640 0x563e46eb8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:26.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.404+0000 7ff5d4efdb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e46eb5f60 0x563e46eb9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:26.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.404+0000 7ff5d4efdb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563e46ebca20 con 0x563e46eaff20 2024-06-10T01:31:26.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.404+0000 7ff5d4efdb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563e46da1bf0 con 0x563e46eb5640 2024-06-10T01:31:26.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.404+0000 7ff5d4efdb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563e46ebcba0 con 0x563e46eb5f60 2024-06-10T01:31:26.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.404+0000 7ff5d4efdb80 10 client.0 _ll_register_callbacks cb 0x563e46d9b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:31:26.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.405+0000 7ff5c37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e46eb5640 0x563e46eb8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:26.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.405+0000 7ff5c37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e46eb5640 0x563e46eb8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:38830/0 (socket says 172.21.15.7:38830) 2024-06-10T01:31:26.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.405+0000 7ff5c37fe700 1 -- 172.21.15.7:0/3158520977 learned_addr learned my addr 172.21.15.7:0/3158520977 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:26.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.405+0000 7ff5c8c17700 1 --2- 172.21.15.7:0/3158520977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e46eb5f60 0x563e46eb9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:26.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.406+0000 7ff5c37fe700 1 -- 172.21.15.7:0/3158520977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e46eb5f60 msgr2=0x563e46eb9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:26.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.406+0000 7ff5c37fe700 1 --2- 172.21.15.7:0/3158520977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e46eb5f60 0x563e46eb9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.406+0000 7ff5c37fe700 1 -- 172.21.15.7:0/3158520977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e46eaff20 msgr2=0x563e46ebbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:26.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.406+0000 7ff5c37fe700 1 --2- 172.21.15.7:0/3158520977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e46eaff20 0x563e46ebbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.406+0000 7ff5c37fe700 1 -- 172.21.15.7:0/3158520977 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e46d508d0 con 0x563e46eb5640 2024-06-10T01:31:26.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.406+0000 7ff5c37fe700 1 --2- 172.21.15.7:0/3158520977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e46eb5640 0x563e46eb8ef0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff5b80105b0 tx=0x7ff5b80105e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=159b059312aa2d4e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:26.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.406+0000 7ff5a37fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:31:26.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.406+0000 7ff5a37fe700 1 -- 172.21.15.7:0/3158520977 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8015030 con 0x563e46eb5640 2024-06-10T01:31:26.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.407+0000 7ff5d4efdb80 1 -- 172.21.15.7:0/3158520977 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5b8011d10 con 0x563e46eb5640 2024-06-10T01:31:26.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.407+0000 7ff5a37fe700 1 -- 172.21.15.7:0/3158520977 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7ff5b8011a40 con 0x563e46eb5640 2024-06-10T01:31:26.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.410+0000 7ff5a37fe700 1 -- 172.21.15.7:0/3158520977 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7ff5b8006c60 con 0x563e46eb5640 2024-06-10T01:31:26.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.413+0000 7ff5d4efdb80 10 client.30590 fetch_fsmap learned FSMap version 57 2024-06-10T01:31:26.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.413+0000 7ff5d4efdb80 1 -- 172.21.15.7:0/3158520977 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x563e46ed8310 con 0x563e46eb5640 2024-06-10T01:31:26.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.417+0000 7ff5a37fe700 1 -- 172.21.15.7:0/3158520977 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5b8011d10 con 0x563e46eb5640 2024-06-10T01:31:26.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.417+0000 7ff5a37fe700 1 -- 172.21.15.7:0/3158520977 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7ff5b801b430 con 0x563e46eb5640 2024-06-10T01:31:26.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.417+0000 7ff5d4efdb80 10 client.30590 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:31:26.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.417+0000 7ff5d4efdb80 10 client.30590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:31:26.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.417+0000 7ff5d4efdb80 20 client.30590 populate_metadata read hostname 'smithi007' 2024-06-10T01:31:26.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.417+0000 7ff5d4efdb80 1 --2- 172.21.15.7:0/3158520977 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563e46edac50 0x563e46edd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:26.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.417+0000 7ff5d4efdb80 4 client.30590 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:31:26.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.417+0000 7ff5d4efdb80 1 -- 172.21.15.7:0/3158520977 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e46e0ed10 con 0x563e46edac50 2024-06-10T01:31:26.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.418+0000 7ff5c3fff700 1 --2- 172.21.15.7:0/3158520977 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563e46edac50 0x563e46edd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:26.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.424+0000 7ff5c3fff700 1 --2- 172.21.15.7:0/3158520977 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563e46edac50 0x563e46edd0b0 crc :-1 s=READY pgs=566 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2fcfa2191232d27 server_cookie=a732806f2c57365 in_seq=0 out_seq=0 2024-06-10T01:31:26.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.426+0000 7ff5a37fe700 10 client.30590 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:26.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.427+0000 7ff5a37fe700 1 -- 172.21.15.7:0/3158520977 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563e46e0ed10 con 0x563e46edac50 2024-06-10T01:31:26.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.427+0000 7ff5a37fe700 10 client.30590 handle_command_reply: tid=0 2024-06-10T01:31:26.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.428+0000 7ff5d4efdb80 1 client.30590 shutdown 2024-06-10T01:31:26.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.434+0000 7ff5d4efdb80 1 -- 172.21.15.7:0/3158520977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e46eb5640 msgr2=0x563e46eb8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:26.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.434+0000 7ff5d4efdb80 1 --2- 172.21.15.7:0/3158520977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e46eb5640 0x563e46eb8ef0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff5b80105b0 tx=0x7ff5b80105e0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.435+0000 7ff5d4efdb80 1 -- 172.21.15.7:0/3158520977 shutdown_connections 2024-06-10T01:31:26.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.435+0000 7ff5d4efdb80 1 --2- 172.21.15.7:0/3158520977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e46eb5f60 0x563e46eb9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.435+0000 7ff5d4efdb80 1 --2- 172.21.15.7:0/3158520977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e46eaff20 0x563e46ebbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.435+0000 7ff5d4efdb80 1 --2- 172.21.15.7:0/3158520977 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563e46edac50 0x563e46edd0b0 crc :-1 s=READY pgs=566 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.435+0000 7ff5d4efdb80 1 --2- 172.21.15.7:0/3158520977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e46eb5640 0x563e46eb8ef0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.435+0000 7ff5d4efdb80 1 -- 172.21.15.7:0/3158520977 >> 172.21.15.7:0/3158520977 conn(0x563e46ddacc0 msgr2=0x563e46eae9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:26.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.435+0000 7ff5a37fe700 0 client.30590 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:26.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.435+0000 7ff5d4efdb80 1 -- 172.21.15.7:0/3158520977 shutdown_connections 2024-06-10T01:31:26.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.435+0000 7ff5d4efdb80 1 -- 172.21.15.7:0/3158520977 wait complete. 2024-06-10T01:31:26.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.435+0000 7ff5d4efdb80 20 client.30590 trim_cache size 0 max 16384 2024-06-10T01:31:26.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.474+0000 7ff5d4efdb80 1 Processor -- start 2024-06-10T01:31:26.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.474+0000 7ff5d4efdb80 1 -- start start 2024-06-10T01:31:26.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.475+0000 7ff5d4efdb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e46eaff20 0x563e46eaf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:26.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.475+0000 7ff5d4efdb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e46eb5640 0x563e46eb6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:26.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.475+0000 7ff5d4efdb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e46eb5f60 0x563e46eb6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:26.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.475+0000 7ff5c37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e46eb5640 0x563e46eb6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:26.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.475+0000 7ff5c37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e46eb5640 0x563e46eb6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:38840/0 (socket says 172.21.15.7:38840) 2024-06-10T01:31:26.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.475+0000 7ff5c37fe700 1 -- 172.21.15.7:0/2521570127 learned_addr learned my addr 172.21.15.7:0/2521570127 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:26.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.475+0000 7ff5d4efdb80 1 -- 172.21.15.7:0/2521570127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b0011480 con 0x563e46eb5f60 2024-06-10T01:31:26.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.475+0000 7ff5d4efdb80 1 -- 172.21.15.7:0/2521570127 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b000ecb0 con 0x563e46eb5640 2024-06-10T01:31:26.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.475+0000 7ff5d4efdb80 1 -- 172.21.15.7:0/2521570127 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b000cb00 con 0x563e46eaff20 2024-06-10T01:31:26.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.476+0000 7ff5c8c17700 1 --2- 172.21.15.7:0/2521570127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e46eb5f60 0x563e46eb6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:26.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.477+0000 7ff5c3fff700 1 --2- 172.21.15.7:0/2521570127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e46eaff20 0x563e46eaf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:26.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.484+0000 7ff5c37fe700 1 -- 172.21.15.7:0/2521570127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e46eaff20 msgr2=0x563e46eaf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:26.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.484+0000 7ff5c37fe700 1 --2- 172.21.15.7:0/2521570127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e46eaff20 0x563e46eaf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.484+0000 7ff5c37fe700 1 -- 172.21.15.7:0/2521570127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e46eb5f60 msgr2=0x563e46eb6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:26.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.484+0000 7ff5c37fe700 1 --2- 172.21.15.7:0/2521570127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e46eb5f60 0x563e46eb6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.484+0000 7ff5c37fe700 1 -- 172.21.15.7:0/2521570127 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e46ebcc70 con 0x563e46eb5640 2024-06-10T01:31:26.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.485+0000 7ff5c37fe700 1 --2- 172.21.15.7:0/2521570127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e46eb5640 0x563e46eb6400 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009710 tx=0x7ff5b8001e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c0c07a2e3645531 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:26.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.487+0000 7ff5a9ffb700 1 -- 172.21.15.7:0/2521570127 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8034030 con 0x563e46eb5640 2024-06-10T01:31:26.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.488+0000 7ff5a9ffb700 1 -- 172.21.15.7:0/2521570127 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5b8009130 con 0x563e46eb5640 2024-06-10T01:31:26.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.488+0000 7ff5a9ffb700 1 -- 172.21.15.7:0/2521570127 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8015070 con 0x563e46eb5640 2024-06-10T01:31:26.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.489+0000 7ff5d4efdb80 1 -- 172.21.15.7:0/2521570127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e46eb5640 msgr2=0x563e46eb6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:26.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.489+0000 7ff5d4efdb80 1 --2- 172.21.15.7:0/2521570127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e46eb5640 0x563e46eb6400 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009710 tx=0x7ff5b8001e50 comp rx=0 tx=0).stop 2024-06-10T01:31:26.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.489+0000 7ff5d4efdb80 1 -- 172.21.15.7:0/2521570127 shutdown_connections 2024-06-10T01:31:26.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.489+0000 7ff5d4efdb80 1 --2- 172.21.15.7:0/2521570127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e46eb5f60 0x563e46eb6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.489+0000 7ff5d4efdb80 1 --2- 172.21.15.7:0/2521570127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e46eaff20 0x563e46eaf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.489+0000 7ff5d4efdb80 1 --2- 172.21.15.7:0/2521570127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e46eb5640 0x563e46eb6400 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.489+0000 7ff5d4efdb80 1 -- 172.21.15.7:0/2521570127 >> 172.21.15.7:0/2521570127 conn(0x563e46ddacc0 msgr2=0x563e46eb7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:26.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.493+0000 7ff5d4efdb80 1 -- 172.21.15.7:0/2521570127 shutdown_connections 2024-06-10T01:31:26.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.494+0000 7ff5d4efdb80 1 -- 172.21.15.7:0/2521570127 wait complete. 2024-06-10T01:31:26.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.494+0000 7ff5d4efdb80 1 -- start start 2024-06-10T01:31:26.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.494+0000 7ff5d4efdb80 1 Processor -- start 2024-06-10T01:31:26.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.495+0000 7ff5d4efdb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e46eaff20 0x563e46ee2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:26.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.495+0000 7ff5d4efdb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e46eb5640 0x563e46ec1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:26.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.495+0000 7ff5d4efdb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e46eb5f60 0x563e46ec2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:26.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.495+0000 7ff5d4efdb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b000d170 con 0x563e46eaff20 2024-06-10T01:31:26.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.495+0000 7ff5d4efdb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b000e5e0 con 0x563e46eb5f60 2024-06-10T01:31:26.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.495+0000 7ff5d4efdb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b000d840 con 0x563e46eb5640 2024-06-10T01:31:26.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.495+0000 7ff5d4efdb80 10 client.0 _ll_register_callbacks cb 0x563e46d9b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:31:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.498+0000 7ff5c37fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e46eb5640 0x563e46ec1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.498+0000 7ff5c37fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e46eb5640 0x563e46ec1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:42444/0 (socket says 172.21.15.7:42444) 2024-06-10T01:31:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.498+0000 7ff5c37fe700 1 -- 172.21.15.7:0/1536683302 learned_addr learned my addr 172.21.15.7:0/1536683302 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.499+0000 7ff5c8c17700 1 --2- 172.21.15.7:0/1536683302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e46eb5f60 0x563e46ec2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.503+0000 7ff5c37fe700 1 -- 172.21.15.7:0/1536683302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e46eb5f60 msgr2=0x563e46ec2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.503+0000 7ff5c37fe700 1 --2- 172.21.15.7:0/1536683302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e46eb5f60 0x563e46ec2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.503+0000 7ff5c37fe700 1 -- 172.21.15.7:0/1536683302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e46eaff20 msgr2=0x563e46ee2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.503+0000 7ff5c37fe700 1 --2- 172.21.15.7:0/1536683302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e46eaff20 0x563e46ee2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.503+0000 7ff5c37fe700 1 -- 172.21.15.7:0/1536683302 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e46ec2aa0 con 0x563e46eb5640 2024-06-10T01:31:26.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.504+0000 7ff5c37fe700 1 --2- 172.21.15.7:0/1536683302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e46eb5640 0x563e46ec1f10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff5b801b830 tx=0x7ff5b801ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=999952c8dbb13233 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:26.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.504+0000 7ff5a37fe700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-10T01:31:26.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.506+0000 7ff5a37fe700 1 -- 172.21.15.7:0/1536683302 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b803d060 con 0x563e46eb5640 2024-06-10T01:31:26.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.506+0000 7ff5d4efdb80 1 -- 172.21.15.7:0/1536683302 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5b8009130 con 0x563e46eb5640 2024-06-10T01:31:26.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.506+0000 7ff5a37fe700 1 -- 172.21.15.7:0/1536683302 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7ff5b8011c50 con 0x563e46eb5640 2024-06-10T01:31:26.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.506+0000 7ff5a37fe700 1 -- 172.21.15.7:0/1536683302 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7ff5b8034050 con 0x563e46eb5640 2024-06-10T01:31:26.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.507+0000 7ff5d4efdb80 10 client.30454 fetch_fsmap learned FSMap version 57 2024-06-10T01:31:26.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.507+0000 7ff5d4efdb80 1 -- 172.21.15.7:0/1536683302 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x563e46ec6010 con 0x563e46eb5640 2024-06-10T01:31:26.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.510+0000 7ff5a37fe700 1 -- 172.21.15.7:0/1536683302 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5b8009130 con 0x563e46eb5640 2024-06-10T01:31:26.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.510+0000 7ff5a37fe700 1 -- 172.21.15.7:0/1536683302 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 57) ==== 8354+0+0 (secure 0 0 0) 0x7ff5b80065b0 con 0x563e46eb5640 2024-06-10T01:31:26.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.510+0000 7ff5d4efdb80 10 client.30454 fetch_fsmap finished waiting for FSMap version 57 2024-06-10T01:31:26.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.510+0000 7ff5d4efdb80 10 client.30454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:31:26.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.510+0000 7ff5d4efdb80 20 client.30454 populate_metadata read hostname 'smithi007' 2024-06-10T01:31:26.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.510+0000 7ff5d4efdb80 1 --2- 172.21.15.7:0/1536683302 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563e46ec6730 0x563e46ec8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:26.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.511+0000 7ff5d4efdb80 4 client.30454 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:31:26.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.511+0000 7ff5d4efdb80 1 -- 172.21.15.7:0/1536683302 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563e46e09290 con 0x563e46ec6730 2024-06-10T01:31:26.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.519+0000 7ff5c3fff700 1 --2- 172.21.15.7:0/1536683302 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563e46ec6730 0x563e46ec8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:26.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.535+0000 7ff5a37fe700 10 client.30454 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:26.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.535+0000 7ff5c3fff700 1 --2- 172.21.15.7:0/1536683302 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563e46ec6730 0x563e46ec8b50 crc :-1 s=READY pgs=567 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=554623038a324794 server_cookie=afc3d51b84ab6dee in_seq=0 out_seq=0 2024-06-10T01:31:26.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.545+0000 7ff5a37fe700 1 -- 172.21.15.7:0/1536683302 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563e46e09290 con 0x563e46ec6730 2024-06-10T01:31:26.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.545+0000 7ff5a37fe700 10 client.30454 handle_command_reply: tid=0 2024-06-10T01:31:26.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.545+0000 7ff5d4efdb80 1 client.30454 shutdown 2024-06-10T01:31:26.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.550+0000 7ff5d4efdb80 1 -- 172.21.15.7:0/1536683302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e46eb5640 msgr2=0x563e46ec1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:26.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.550+0000 7ff5d4efdb80 1 --2- 172.21.15.7:0/1536683302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e46eb5640 0x563e46ec1f10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff5b801b830 tx=0x7ff5b801ba20 comp rx=0 tx=0).stop 2024-06-10T01:31:26.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.551+0000 7ff5d4efdb80 1 -- 172.21.15.7:0/1536683302 shutdown_connections 2024-06-10T01:31:26.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.551+0000 7ff5d4efdb80 1 --2- 172.21.15.7:0/1536683302 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563e46ec6730 0x563e46ec8b50 crc :-1 s=READY pgs=567 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.551+0000 7ff5d4efdb80 1 --2- 172.21.15.7:0/1536683302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e46eb5f60 0x563e46ec2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.551+0000 7ff5d4efdb80 1 --2- 172.21.15.7:0/1536683302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e46eaff20 0x563e46ee2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.551+0000 7ff5d4efdb80 1 --2- 172.21.15.7:0/1536683302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e46eb5640 0x563e46ec1f10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.551+0000 7ff5d4efdb80 1 -- 172.21.15.7:0/1536683302 >> 172.21.15.7:0/1536683302 conn(0x563e46ddacc0 msgr2=0x563e46ea3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:26.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.551+0000 7ff5a37fe700 0 client.30454 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:26.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.553+0000 7ff5d4efdb80 1 -- 172.21.15.7:0/1536683302 shutdown_connections 2024-06-10T01:31:26.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.557+0000 7ff5d4efdb80 1 -- 172.21.15.7:0/1536683302 wait complete. 2024-06-10T01:31:26.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.557+0000 7ff5d4efdb80 20 client.30454 trim_cache size 0 max 16384 2024-06-10T01:31:26.550 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:31:26.550 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T01:31:26.550 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-10T01:31:26.550 INFO:teuthology.orchestra.run.smithi007.stdout: "e91dc0a6-8ffa-4cf1-b5df-9d13bace9eaa": { 2024-06-10T01:31:26.550 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-10T01:31:26.550 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "e91dc0a6-8ffa-4cf1-b5df-9d13bace9eaa", 2024-06-10T01:31:26.550 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-10T01:31:26.550 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:31:26.550 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:31:26.550 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:31:26.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.558+0000 7ff5ca67a700 1 -- 172.21.15.7:0/3908334424 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff5ac0a2cf0 msgr2=0x7ff5ac0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:26.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.558+0000 7ff5ca67a700 1 --2- 172.21.15.7:0/3908334424 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff5ac0a2cf0 0x7ff5ac0a51b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4009750 tx=0x7ff5b4005fb0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.558+0000 7ff5ca67a700 1 -- 172.21.15.7:0/3908334424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5c411bc70 msgr2=0x7ff5c41a7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:26.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.558+0000 7ff5ca67a700 1 --2- 172.21.15.7:0/3908334424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5c411bc70 0x7ff5c41a7a00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00b9e0 tx=0x7ff5bc00bcf0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.558+0000 7ff5ca67a700 1 -- 172.21.15.7:0/3908334424 shutdown_connections 2024-06-10T01:31:26.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.558+0000 7ff5ca67a700 1 --2- 172.21.15.7:0/3908334424 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7ff5ac0a2cf0 0x7ff5ac0a51b0 secure :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4009750 tx=0x7ff5b4005fb0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.558+0000 7ff5ca67a700 1 --2- 172.21.15.7:0/3908334424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5c411bc70 0x7ff5c41a7a00 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.558+0000 7ff5ca67a700 1 --2- 172.21.15.7:0/3908334424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5c41099d0 0x7ff5c41ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.558+0000 7ff5ca67a700 1 --2- 172.21.15.7:0/3908334424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5c4110c70 0x7ff5c41add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:26.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.558+0000 7ff5ca67a700 1 -- 172.21.15.7:0/3908334424 >> 172.21.15.7:0/3908334424 conn(0x7ff5c406d600 msgr2=0x7ff5c4119c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:26.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.559+0000 7ff5ca67a700 1 -- 172.21.15.7:0/3908334424 shutdown_connections 2024-06-10T01:31:26.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:26.559+0000 7ff5ca67a700 1 -- 172.21.15.7:0/3908334424 wait complete. 2024-06-10T01:31:26.648 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:26 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:31:26.649 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:26 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:31:26.649 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:26 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:31:26.649 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:26 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:31:26.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:26 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:31:26.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:26 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:31:26.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:26 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:31:26.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:26 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:31:26.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:26 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:31:26.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:26 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:31:26.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:26 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:31:26.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:26 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:31:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:27.107+0000 7fe40b7fe640 1 -- 172.21.15.7:0/1401816655 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fe41c028070 con 0x563f8daa79f0 2024-06-10T01:31:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:27.107+0000 7fe40b7fe640 10 client.30439 handle_command_reply: tid=0 2024-06-10T01:31:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:27.107+0000 7fe435202740 1 client.30439 shutdown 2024-06-10T01:31:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:27.107+0000 7fe435202740 1 -- 172.21.15.7:0/1401816655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f8da05010 msgr2=0x563f8da13700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:27.107+0000 7fe435202740 1 --2- 172.21.15.7:0/1401816655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f8da05010 0x563f8da13700 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe41c00e760 tx=0x7fe41c009790 comp rx=0 tx=0).stop 2024-06-10T01:31:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:27.108+0000 7fe435202740 1 -- 172.21.15.7:0/1401816655 shutdown_connections 2024-06-10T01:31:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:27.108+0000 7fe435202740 1 --2- 172.21.15.7:0/1401816655 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563f8daa79f0 0x563f8daa9e10 crc :-1 s=READY pgs=565 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:27.108+0000 7fe435202740 1 --2- 172.21.15.7:0/1401816655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f8da0ab70 0x563f8daa59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:27.108+0000 7fe435202740 1 --2- 172.21.15.7:0/1401816655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8da05a00 0x563f8da13c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:27.108+0000 7fe435202740 1 --2- 172.21.15.7:0/1401816655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f8da05010 0x563f8da13700 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:27.108+0000 7fe435202740 1 -- 172.21.15.7:0/1401816655 >> 172.21.15.7:0/1401816655 conn(0x563f8da00680 msgr2=0x563f8da03360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:27.108+0000 7fe40b7fe640 0 client.30439 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:27.108+0000 7fe435202740 1 -- 172.21.15.7:0/1401816655 shutdown_connections 2024-06-10T01:31:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:27.108+0000 7fe435202740 1 -- 172.21.15.7:0/1401816655 wait complete. 2024-06-10T01:31:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:27.108+0000 7fe435202740 20 client.30439 trim_cache size 0 max 16384 2024-06-10T01:31:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-10T01:31:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-10T01:31:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-10T01:31:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:27.109+0000 7fe432415640 1 -- 172.21.15.7:0/3322209114 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe4140a2d90 msgr2=0x7fe4140a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:27.109+0000 7fe432415640 1 --2- 172.21.15.7:0/3322209114 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe4140a2d90 0x7fe4140a5250 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe420097e50 tx=0x7fe420005f90 comp rx=0 tx=0).stop 2024-06-10T01:31:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:27.109+0000 7fe432415640 1 -- 172.21.15.7:0/3322209114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe42c1089a0 msgr2=0x7fe42c117030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:27.109+0000 7fe432415640 1 --2- 172.21.15.7:0/3322209114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe42c1089a0 0x7fe42c117030 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe42000d950 tx=0x7fe42000de10 comp rx=0 tx=0).stop 2024-06-10T01:31:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:27.110+0000 7fe432415640 1 -- 172.21.15.7:0/3322209114 shutdown_connections 2024-06-10T01:31:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:27.110+0000 7fe432415640 1 --2- 172.21.15.7:0/3322209114 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fe4140a2d90 0x7fe4140a5250 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:27.110+0000 7fe432415640 1 --2- 172.21.15.7:0/3322209114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe42c10dcc0 0x7fe42c1b6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:27.110+0000 7fe432415640 1 --2- 172.21.15.7:0/3322209114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe42c10b310 0x7fe42c117570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:27.110+0000 7fe432415640 1 --2- 172.21.15.7:0/3322209114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe42c1089a0 0x7fe42c117030 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:27.110+0000 7fe432415640 1 -- 172.21.15.7:0/3322209114 >> 172.21.15.7:0/3322209114 conn(0x7fe42c06e9e0 msgr2=0x7fe42c10baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:27.110+0000 7fe432415640 1 -- 172.21.15.7:0/3322209114 shutdown_connections 2024-06-10T01:31:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-10T01:31:27.110+0000 7fe432415640 1 -- 172.21.15.7:0/3322209114 wait complete. 2024-06-10T01:31:27.489 INFO:tasks.cephfs.filesystem:scrub status for tag:e91dc0a6-8ffa-4cf1-b5df-9d13bace9eaa - {'path': '/', 'tag': 'e91dc0a6-8ffa-4cf1-b5df-9d13bace9eaa', 'options': 'recursive,force'} 2024-06-10T01:31:27.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:27 smithi007 ceph-mon[32398]: Adjusting osd_memory_target on smithi007 to 177.2M 2024-06-10T01:31:27.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:27 smithi007 ceph-mon[32398]: Unable to set osd_memory_target on smithi007 to 185842585: error parsing value: Value '185842585' is below minimum 939524096 2024-06-10T01:31:27.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:27 smithi007 ceph-mon[32398]: pgmap v1197: 129 pgs: 129 active+clean; 700 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.0 MiB/s wr, 710 op/s 2024-06-10T01:31:27.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:27 smithi078 ceph-mon[36683]: Adjusting osd_memory_target on smithi007 to 177.2M 2024-06-10T01:31:27.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:27 smithi078 ceph-mon[36683]: Unable to set osd_memory_target on smithi007 to 185842585: error parsing value: Value '185842585' is below minimum 939524096 2024-06-10T01:31:27.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:27 smithi078 ceph-mon[36683]: pgmap v1197: 129 pgs: 129 active+clean; 700 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.0 MiB/s wr, 710 op/s 2024-06-10T01:31:27.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:27 smithi175 ceph-mon[36636]: Adjusting osd_memory_target on smithi007 to 177.2M 2024-06-10T01:31:27.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:27 smithi175 ceph-mon[36636]: Unable to set osd_memory_target on smithi007 to 185842585: error parsing value: Value '185842585' is below minimum 939524096 2024-06-10T01:31:27.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:27 smithi175 ceph-mon[36636]: pgmap v1197: 129 pgs: 129 active+clean; 700 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.0 MiB/s wr, 710 op/s 2024-06-10T01:31:28.088 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-06-10T01:31:28.088 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-06-10T01:31:28.455 DEBUG:teuthology.parallel:result is None 2024-06-10T01:31:28.455 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-06-10T01:31:28.494 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-06-10T01:31:28.494 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-06-10T01:31:28.506 INFO:tasks.quiescer:joining Quiescers 2024-06-10T01:31:28.506 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-06-10T01:31:28.506 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:31:28.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:28 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:31:28.794 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:31:28.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:28 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:31:28.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:28 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:31:29.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:29 smithi078 ceph-mon[36683]: pgmap v1198: 129 pgs: 129 active+clean; 663 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.9 MiB/s wr, 571 op/s 2024-06-10T01:31:29.897 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:29 smithi175 ceph-mon[36636]: pgmap v1198: 129 pgs: 129 active+clean; 663 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.9 MiB/s wr, 571 op/s 2024-06-10T01:31:30.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:29 smithi007 ceph-mon[32398]: pgmap v1198: 129 pgs: 129 active+clean; 663 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.9 MiB/s wr, 571 op/s 2024-06-10T01:31:30.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:30 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:31:30.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:30 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:31:31.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:30 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:31:31.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:31 smithi175 ceph-mon[36636]: pgmap v1199: 129 pgs: 129 active+clean; 659 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.0 MiB/s wr, 515 op/s 2024-06-10T01:31:31.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:31 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:31:31.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:31 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:31:31.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:31 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:31:31.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:31 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:31:31.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:31 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:31:31.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:31 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:31:32.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:31 smithi007 ceph-mon[32398]: pgmap v1199: 129 pgs: 129 active+clean; 659 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.0 MiB/s wr, 515 op/s 2024-06-10T01:31:32.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:31 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:31:32.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:31 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:31:32.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:31 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:31:32.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:31 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:31:32.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:31 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:31:32.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:31 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:31:32.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:31 smithi078 ceph-mon[36683]: pgmap v1199: 129 pgs: 129 active+clean; 659 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.0 MiB/s wr, 515 op/s 2024-06-10T01:31:32.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:31 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:31:32.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:31 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:31:32.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:31 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:31:32.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:31 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:31:32.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:31 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:31:32.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:31 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:31:32.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.260+0000 7fccce4da700 1 -- 172.21.15.7:0/1537032607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccc806ab30 msgr2=0x7fccc810f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:32.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.260+0000 7fccce4da700 1 --2- 172.21.15.7:0/1537032607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccc806ab30 0x7fccc810f320 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fccc000b0a0 tx=0x7fccc000b3b0 comp rx=0 tx=0).stop 2024-06-10T01:31:32.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.260+0000 7fccce4da700 1 -- 172.21.15.7:0/1537032607 shutdown_connections 2024-06-10T01:31:32.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.260+0000 7fccce4da700 1 --2- 172.21.15.7:0/1537032607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccc806ab30 0x7fccc810f320 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:32.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.260+0000 7fccce4da700 1 --2- 172.21.15.7:0/1537032607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccc8108500 0x7fccc81088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:32.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.260+0000 7fccce4da700 1 --2- 172.21.15.7:0/1537032607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccc806a170 0x7fccc806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:32.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.260+0000 7fccce4da700 1 -- 172.21.15.7:0/1537032607 >> 172.21.15.7:0/1537032607 conn(0x7fccc8078340 msgr2=0x7fccc8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:32.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.260+0000 7fccce4da700 1 -- 172.21.15.7:0/1537032607 shutdown_connections 2024-06-10T01:31:32.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.260+0000 7fccce4da700 1 -- 172.21.15.7:0/1537032607 wait complete. 2024-06-10T01:31:32.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.261+0000 7fccce4da700 1 Processor -- start 2024-06-10T01:31:32.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.261+0000 7fccce4da700 1 -- start start 2024-06-10T01:31:32.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.261+0000 7fccce4da700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccc806a170 0x7fccc8101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:32.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.261+0000 7fccce4da700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccc806ab30 0x7fccc81019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:32.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.261+0000 7fccce4da700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccc8108500 0x7fccc8101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:32.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.261+0000 7fccce4da700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fccc8112760 con 0x7fccc806a170 2024-06-10T01:31:32.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.261+0000 7fccce4da700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fccc81125e0 con 0x7fccc806ab30 2024-06-10T01:31:32.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.261+0000 7fccce4da700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fccc81128e0 con 0x7fccc8108500 2024-06-10T01:31:32.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.261+0000 7fccc7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccc806a170 0x7fccc8101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:32.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.261+0000 7fccc7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccc806a170 0x7fccc8101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39778/0 (socket says 172.21.15.7:39778) 2024-06-10T01:31:32.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.261+0000 7fccc7fff700 1 -- 172.21.15.7:0/4229879037 learned_addr learned my addr 172.21.15.7:0/4229879037 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:32.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.261+0000 7fccbffff700 1 --2- 172.21.15.7:0/4229879037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccc806ab30 0x7fccc81019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:32.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.262+0000 7fcccca77700 1 --2- 172.21.15.7:0/4229879037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccc8108500 0x7fccc8101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:32.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.262+0000 7fccc7fff700 1 -- 172.21.15.7:0/4229879037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccc8108500 msgr2=0x7fccc8101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:32.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.262+0000 7fccc7fff700 1 --2- 172.21.15.7:0/4229879037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccc8108500 0x7fccc8101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:32.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.262+0000 7fccc7fff700 1 -- 172.21.15.7:0/4229879037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccc806ab30 msgr2=0x7fccc81019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:32.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.262+0000 7fccc7fff700 1 --2- 172.21.15.7:0/4229879037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccc806ab30 0x7fccc81019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:32.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.262+0000 7fccc7fff700 1 -- 172.21.15.7:0/4229879037 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc81a70e0 con 0x7fccc806a170 2024-06-10T01:31:32.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.262+0000 7fccc7fff700 1 --2- 172.21.15.7:0/4229879037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccc806a170 0x7fccc8101460 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fccb800d950 tx=0x7fccb800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:32.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.262+0000 7fccc5ffb700 1 -- 172.21.15.7:0/4229879037 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb8016070 con 0x7fccc806a170 2024-06-10T01:31:32.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.262+0000 7fccc5ffb700 1 -- 172.21.15.7:0/4229879037 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccb8009e20 con 0x7fccc806a170 2024-06-10T01:31:32.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.262+0000 7fccce4da700 1 -- 172.21.15.7:0/4229879037 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccc81a73d0 con 0x7fccc806a170 2024-06-10T01:31:32.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.262+0000 7fccc5ffb700 1 -- 172.21.15.7:0/4229879037 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb800f620 con 0x7fccc806a170 2024-06-10T01:31:32.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.262+0000 7fccce4da700 1 -- 172.21.15.7:0/4229879037 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccc81a7910 con 0x7fccc806a170 2024-06-10T01:31:32.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.263+0000 7fccce4da700 1 -- 172.21.15.7:0/4229879037 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccc8069180 con 0x7fccc806a170 2024-06-10T01:31:32.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.265+0000 7fccc5ffb700 1 -- 172.21.15.7:0/4229879037 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fccb8019070 con 0x7fccc806a170 2024-06-10T01:31:32.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.266+0000 7fccc5ffb700 1 --2- 172.21.15.7:0/4229879037 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fccb40a2c20 0x7fccb40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:32.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.266+0000 7fccbffff700 1 --2- 172.21.15.7:0/4229879037 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fccb40a2c20 0x7fccb40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:32.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.266+0000 7fccc5ffb700 1 -- 172.21.15.7:0/4229879037 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fccb80cef20 con 0x7fccc806a170 2024-06-10T01:31:32.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.266+0000 7fccbffff700 1 --2- 172.21.15.7:0/4229879037 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fccb40a2c20 0x7fccb40a50e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fccc8078940 tx=0x7fccb0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:32.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.269+0000 7fccc5ffb700 1 -- 172.21.15.7:0/4229879037 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fccb809bb00 con 0x7fccc806a170 2024-06-10T01:31:32.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.653+0000 7fccce4da700 1 -- 172.21.15.7:0/4229879037 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fccc802de60 con 0x7fccc806a170 2024-06-10T01:31:32.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.653+0000 7fccc5ffb700 1 -- 172.21.15.7:0/4229879037 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+11859 (secure 0 0 0) 0x7fccb809b400 con 0x7fccc806a170 2024-06-10T01:31:32.646 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:31:32.646 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":59,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:31:28.873636+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29813":{"gid":29813,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/138896429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":138896429},{"type":"v1","addr":"172.21.15.175:6835","nonce":138896429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24436,24451,24434]},"id":1}]} 2024-06-10T01:31:32.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.655+0000 7fccce4da700 1 -- 172.21.15.7:0/4229879037 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fccb40a2c20 msgr2=0x7fccb40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:32.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.655+0000 7fccce4da700 1 --2- 172.21.15.7:0/4229879037 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fccb40a2c20 0x7fccb40a50e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fccc8078940 tx=0x7fccb0009260 comp rx=0 tx=0).stop 2024-06-10T01:31:32.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.655+0000 7fccce4da700 1 -- 172.21.15.7:0/4229879037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccc806a170 msgr2=0x7fccc8101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:32.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.655+0000 7fccce4da700 1 --2- 172.21.15.7:0/4229879037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccc806a170 0x7fccc8101460 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fccb800d950 tx=0x7fccb800dc60 comp rx=0 tx=0).stop 2024-06-10T01:31:32.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.656+0000 7fccce4da700 1 -- 172.21.15.7:0/4229879037 shutdown_connections 2024-06-10T01:31:32.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.656+0000 7fccce4da700 1 --2- 172.21.15.7:0/4229879037 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fccb40a2c20 0x7fccb40a50e0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:32.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.656+0000 7fccce4da700 1 --2- 172.21.15.7:0/4229879037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccc8108500 0x7fccc8101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:32.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.656+0000 7fccce4da700 1 --2- 172.21.15.7:0/4229879037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccc806a170 0x7fccc8101460 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:32.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.656+0000 7fccce4da700 1 --2- 172.21.15.7:0/4229879037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccc806ab30 0x7fccc81019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:32.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.656+0000 7fccce4da700 1 -- 172.21.15.7:0/4229879037 >> 172.21.15.7:0/4229879037 conn(0x7fccc8078340 msgr2=0x7fccc80ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:32.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.656+0000 7fccce4da700 1 -- 172.21.15.7:0/4229879037 shutdown_connections 2024-06-10T01:31:32.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:32.656+0000 7fccce4da700 1 -- 172.21.15.7:0/4229879037 wait complete. 2024-06-10T01:31:32.649 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 59 2024-06-10T01:31:32.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:32 smithi175 ceph-mon[36636]: Adjusting osd_memory_target on smithi078 to 176.2M 2024-06-10T01:31:32.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:32 smithi175 ceph-mon[36636]: Unable to set osd_memory_target on smithi078 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-06-10T01:31:32.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:32 smithi175 ceph-mon[36636]: pgmap v1200: 129 pgs: 129 active+clean; 566 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.3 MiB/s wr, 694 op/s 2024-06-10T01:31:32.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:32 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/4229879037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:31:33.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:32 smithi007 ceph-mon[32398]: Adjusting osd_memory_target on smithi078 to 176.2M 2024-06-10T01:31:33.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:32 smithi007 ceph-mon[32398]: Unable to set osd_memory_target on smithi078 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-06-10T01:31:33.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:32 smithi007 ceph-mon[32398]: pgmap v1200: 129 pgs: 129 active+clean; 566 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.3 MiB/s wr, 694 op/s 2024-06-10T01:31:33.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:32 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/4229879037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:31:33.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:32 smithi078 ceph-mon[36683]: Adjusting osd_memory_target on smithi078 to 176.2M 2024-06-10T01:31:33.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:32 smithi078 ceph-mon[36683]: Unable to set osd_memory_target on smithi078 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-06-10T01:31:33.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:32 smithi078 ceph-mon[36683]: pgmap v1200: 129 pgs: 129 active+clean; 566 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.3 MiB/s wr, 694 op/s 2024-06-10T01:31:33.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:32 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/4229879037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:31:33.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:33 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:31:33.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:33 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:31:33.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:33 smithi175 ceph-mon[36636]: scrub summary: idle 2024-06-10T01:31:34.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:33 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:31:34.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:33 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:31:34.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:33 smithi007 ceph-mon[32398]: scrub summary: idle 2024-06-10T01:31:34.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:33 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:31:34.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:33 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:31:34.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:33 smithi078 ceph-mon[36683]: scrub summary: idle 2024-06-10T01:31:34.179 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24451 quiesce db --cancel --all' 2024-06-10T01:31:34.179 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24451 quiesce db --cancel --all 2024-06-10T01:31:34.439 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:31:34.834 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:34 smithi007 ceph-mon[32398]: pgmap v1201: 129 pgs: 129 active+clean; 563 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.0 MiB/s wr, 498 op/s 2024-06-10T01:31:34.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:34 smithi175 ceph-mon[36636]: pgmap v1201: 129 pgs: 129 active+clean; 563 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.0 MiB/s wr, 498 op/s 2024-06-10T01:31:35.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:34 smithi078 ceph-mon[36683]: pgmap v1201: 129 pgs: 129 active+clean; 563 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.0 MiB/s wr, 498 op/s 2024-06-10T01:31:35.927 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:35 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:31:35.928 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:35 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:31:35.928 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:35 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:31:35.928 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:35 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:31:35.928 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:35 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:31:36.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:35 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:31:36.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:35 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:31:36.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:35 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:31:36.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:35 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:31:36.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:35 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:31:36.200 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:35 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:31:36.200 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:35 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:31:36.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:35 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:31:36.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:35 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:31:36.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:35 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:31:36.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.322+0000 7fa7b1882700 1 -- 172.21.15.7:0/478836968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7ac1016e0 msgr2=0x7fa7ac101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:36.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.322+0000 7fa7b1882700 1 --2- 172.21.15.7:0/478836968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7ac1016e0 0x7fa7ac101b60 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fa7a00099c0 tx=0x7fa7a0009cd0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.322+0000 7fa7b1882700 1 -- 172.21.15.7:0/478836968 shutdown_connections 2024-06-10T01:31:36.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.322+0000 7fa7b1882700 1 --2- 172.21.15.7:0/478836968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7ac1020a0 0x7fa7ac10e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.323+0000 7fa7b1882700 1 --2- 172.21.15.7:0/478836968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7ac106410 0x7fa7ac1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.323+0000 7fa7b1882700 1 --2- 172.21.15.7:0/478836968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7ac1016e0 0x7fa7ac101b60 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.323+0000 7fa7b1882700 1 -- 172.21.15.7:0/478836968 >> 172.21.15.7:0/478836968 conn(0x7fa7ac077200 msgr2=0x7fa7ac077610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:36.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.323+0000 7fa7b1882700 1 -- 172.21.15.7:0/478836968 shutdown_connections 2024-06-10T01:31:36.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.323+0000 7fa7b1882700 1 -- 172.21.15.7:0/478836968 wait complete. 2024-06-10T01:31:36.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.323+0000 7fa7b1882700 1 Processor -- start 2024-06-10T01:31:36.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.323+0000 7fa7b1882700 1 -- start start 2024-06-10T01:31:36.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.324+0000 7fa7b1882700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7ac1016e0 0x7fa7ac1030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:36.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.324+0000 7fa7b1882700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7ac1020a0 0x7fa7ac103600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:36.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.324+0000 7fa7b1882700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7ac106410 0x7fa7ac103b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:36.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.324+0000 7fa7b1882700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ac111750 con 0x7fa7ac106410 2024-06-10T01:31:36.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.324+0000 7fa7b1882700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ac1115d0 con 0x7fa7ac1020a0 2024-06-10T01:31:36.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.324+0000 7fa7b1882700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ac1118d0 con 0x7fa7ac1016e0 2024-06-10T01:31:36.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.324+0000 7fa7ab7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7ac106410 0x7fa7ac103b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:36.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.324+0000 7fa7aaffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7ac1016e0 0x7fa7ac1030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:36.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.324+0000 7fa7aaffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7ac1016e0 0x7fa7ac1030c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:46284/0 (socket says 172.21.15.7:46284) 2024-06-10T01:31:36.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.324+0000 7fa7aa7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7ac1020a0 0x7fa7ac103600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:36.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.324+0000 7fa7aaffd700 1 -- 172.21.15.7:0/2476989152 learned_addr learned my addr 172.21.15.7:0/2476989152 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:36.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.324+0000 7fa7aaffd700 1 -- 172.21.15.7:0/2476989152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7ac1020a0 msgr2=0x7fa7ac103600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:36.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.324+0000 7fa7aaffd700 1 --2- 172.21.15.7:0/2476989152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7ac1020a0 0x7fa7ac103600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.324+0000 7fa7aaffd700 1 -- 172.21.15.7:0/2476989152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7ac106410 msgr2=0x7fa7ac103b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:36.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.324+0000 7fa7aaffd700 1 --2- 172.21.15.7:0/2476989152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7ac106410 0x7fa7ac103b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.324+0000 7fa7aaffd700 1 -- 172.21.15.7:0/2476989152 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7ac1a70a0 con 0x7fa7ac1016e0 2024-06-10T01:31:36.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.324+0000 7fa7aaffd700 1 --2- 172.21.15.7:0/2476989152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7ac1016e0 0x7fa7ac1030c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa79c00c9b0 tx=0x7fa79c00ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:36.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.324+0000 7fa7b0880700 1 -- 172.21.15.7:0/2476989152 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c007b00 con 0x7fa7ac1016e0 2024-06-10T01:31:36.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.325+0000 7fa7b1882700 1 -- 172.21.15.7:0/2476989152 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7ac1a7390 con 0x7fa7ac1016e0 2024-06-10T01:31:36.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.325+0000 7fa7b0880700 1 -- 172.21.15.7:0/2476989152 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa79c00ce50 con 0x7fa7ac1016e0 2024-06-10T01:31:36.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.325+0000 7fa7b0880700 1 -- 172.21.15.7:0/2476989152 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c018660 con 0x7fa7ac1016e0 2024-06-10T01:31:36.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.325+0000 7fa7b1882700 1 -- 172.21.15.7:0/2476989152 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7ac1a78f0 con 0x7fa7ac1016e0 2024-06-10T01:31:36.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.328+0000 7fa7b0880700 1 -- 172.21.15.7:0/2476989152 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa79c00f450 con 0x7fa7ac1016e0 2024-06-10T01:31:36.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.328+0000 7fa7b0880700 1 --2- 172.21.15.7:0/2476989152 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa7940a2c20 0x7fa7940a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:36.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.328+0000 7fa7aa7fc700 1 --2- 172.21.15.7:0/2476989152 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa7940a2c20 0x7fa7940a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:36.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.329+0000 7fa7b0880700 1 -- 172.21.15.7:0/2476989152 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fa79c0cf260 con 0x7fa7ac1016e0 2024-06-10T01:31:36.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.329+0000 7fa7bc105b80 1 Processor -- start 2024-06-10T01:31:36.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.329+0000 7fa7aa7fc700 1 --2- 172.21.15.7:0/2476989152 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa7940a2c20 0x7fa7940a50e0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac104b50 tx=0x7fa7a0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:36.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.329+0000 7fa7bc105b80 1 -- start start 2024-06-10T01:31:36.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.329+0000 7fa7bc105b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cd3202470 0x563cd3202850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:36.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.329+0000 7fa7bc105b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cd3202d90 0x563cd31fc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:36.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.329+0000 7fa7bc105b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563cd31fcbc0 0x563cd31fcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:36.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.329+0000 7fa7bc105b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563cd30eefd0 con 0x563cd31fcbc0 2024-06-10T01:31:36.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.329+0000 7fa7bc105b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563cd314d8c0 con 0x563cd3202470 2024-06-10T01:31:36.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.329+0000 7fa7bc105b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563cd30b54d0 con 0x563cd3202d90 2024-06-10T01:31:36.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.329+0000 7fa7aa7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cd3202d90 0x563cd31fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:36.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.329+0000 7fa7ab7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563cd31fcbc0 0x563cd31fcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:36.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.329+0000 7fa7aa7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cd3202d90 0x563cd31fc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:46300/0 (socket says 172.21.15.7:46300) 2024-06-10T01:31:36.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.329+0000 7fa7aa7fc700 1 -- 172.21.15.7:0/1182486348 learned_addr learned my addr 172.21.15.7:0/1182486348 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:36.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.329+0000 7fa7aaffd700 1 --2- 172.21.15.7:0/1182486348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cd3202470 0x563cd3202850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:36.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.330+0000 7fa7ab7fe700 1 -- 172.21.15.7:0/1182486348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cd3202d90 msgr2=0x563cd31fc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:36.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.330+0000 7fa7ab7fe700 1 --2- 172.21.15.7:0/1182486348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cd3202d90 0x563cd31fc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.330+0000 7fa7ab7fe700 1 -- 172.21.15.7:0/1182486348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cd3202470 msgr2=0x563cd3202850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:36.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.330+0000 7fa7ab7fe700 1 --2- 172.21.15.7:0/1182486348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cd3202470 0x563cd3202850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.330+0000 7fa7ab7fe700 1 -- 172.21.15.7:0/1182486348 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cd309c940 con 0x563cd31fcbc0 2024-06-10T01:31:36.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.330+0000 7fa7ab7fe700 1 --2- 172.21.15.7:0/1182486348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563cd31fcbc0 0x563cd31fcfe0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4011580 tx=0x7fa7a4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1714c66c5055ab server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:36.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.330+0000 7fa791ffb700 1 -- 172.21.15.7:0/1182486348 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4011e80 con 0x563cd31fcbc0 2024-06-10T01:31:36.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.330+0000 7fa791ffb700 1 -- 172.21.15.7:0/1182486348 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a4004bb0 con 0x563cd31fcbc0 2024-06-10T01:31:36.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.330+0000 7fa791ffb700 1 -- 172.21.15.7:0/1182486348 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a40047a0 con 0x563cd31fcbc0 2024-06-10T01:31:36.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.331+0000 7fa7bc105b80 1 -- 172.21.15.7:0/1182486348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563cd31fcbc0 msgr2=0x563cd31fcfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:36.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.331+0000 7fa7bc105b80 1 --2- 172.21.15.7:0/1182486348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563cd31fcbc0 0x563cd31fcfe0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4011580 tx=0x7fa7a4009920 comp rx=0 tx=0).stop 2024-06-10T01:31:36.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.331+0000 7fa7bc105b80 1 -- 172.21.15.7:0/1182486348 shutdown_connections 2024-06-10T01:31:36.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.331+0000 7fa7bc105b80 1 --2- 172.21.15.7:0/1182486348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563cd31fcbc0 0x563cd31fcfe0 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.331+0000 7fa7bc105b80 1 --2- 172.21.15.7:0/1182486348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cd3202470 0x563cd3202850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.331+0000 7fa7bc105b80 1 --2- 172.21.15.7:0/1182486348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cd3202d90 0x563cd31fc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.331+0000 7fa7bc105b80 1 -- 172.21.15.7:0/1182486348 >> 172.21.15.7:0/1182486348 conn(0x563cd31290f0 msgr2=0x563cd31f7970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:36.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.331+0000 7fa7bc105b80 1 -- 172.21.15.7:0/1182486348 shutdown_connections 2024-06-10T01:31:36.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.331+0000 7fa7bc105b80 1 -- 172.21.15.7:0/1182486348 wait complete. 2024-06-10T01:31:36.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.331+0000 7fa7bc105b80 1 -- start start 2024-06-10T01:31:36.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.331+0000 7fa7bc105b80 1 Processor -- start 2024-06-10T01:31:36.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.332+0000 7fa7bc105b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cd31fcbc0 0x563cd3208a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:36.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.332+0000 7fa7bc105b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cd3202470 0x563cd3205d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:36.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.332+0000 7fa7bc105b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563cd3202d90 0x563cd32062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:36.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.332+0000 7fa7bc105b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563cd30ee030 con 0x563cd3202d90 2024-06-10T01:31:36.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.332+0000 7fa7bc105b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563cd3149880 con 0x563cd3202470 2024-06-10T01:31:36.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.332+0000 7fa7bc105b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563cd3209770 con 0x563cd31fcbc0 2024-06-10T01:31:36.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.332+0000 7fa7ab7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563cd3202d90 0x563cd32062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:36.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.332+0000 7fa7aaffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cd31fcbc0 0x563cd3208a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:36.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.332+0000 7fa7bc105b80 10 client.0 _ll_register_callbacks cb 0x563cd30e3620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:31:36.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.332+0000 7fa7ab7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563cd3202d90 0x563cd32062d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39798/0 (socket says 172.21.15.7:39798) 2024-06-10T01:31:36.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.332+0000 7fa7ab7fe700 1 -- 172.21.15.7:0/1369425313 learned_addr learned my addr 172.21.15.7:0/1369425313 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:36.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.332+0000 7fa7aaffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cd31fcbc0 0x563cd3208a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:46316/0 (socket says 172.21.15.7:46316) 2024-06-10T01:31:36.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.332+0000 7fa7aa7fc700 1 --2- 172.21.15.7:0/1369425313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cd3202470 0x563cd3205d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:36.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.332+0000 7fa7ab7fe700 1 -- 172.21.15.7:0/1369425313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cd31fcbc0 msgr2=0x563cd3208a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:36.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.332+0000 7fa7ab7fe700 1 --2- 172.21.15.7:0/1369425313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cd31fcbc0 0x563cd3208a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.332+0000 7fa7ab7fe700 1 -- 172.21.15.7:0/1369425313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cd3202470 msgr2=0x563cd3205d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:36.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.332+0000 7fa7ab7fe700 1 --2- 172.21.15.7:0/1369425313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cd3202470 0x563cd3205d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.332+0000 7fa7ab7fe700 1 -- 172.21.15.7:0/1369425313 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cd31137c0 con 0x563cd3202d90 2024-06-10T01:31:36.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.332+0000 7fa7aaffd700 1 --2- 172.21.15.7:0/1369425313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cd31fcbc0 0x563cd3208a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:31:36.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.332+0000 7fa7ab7fe700 1 --2- 172.21.15.7:0/1369425313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563cd3202d90 0x563cd32062d0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009510 tx=0x7fa7a4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad21c19051da9dc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:36.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.332+0000 7fa78affd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:31:36.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.333+0000 7fa78affd700 1 -- 172.21.15.7:0/1369425313 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4003e10 con 0x563cd3202d90 2024-06-10T01:31:36.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.333+0000 7fa78affd700 1 -- 172.21.15.7:0/1369425313 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fa7a4007500 con 0x563cd3202d90 2024-06-10T01:31:36.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.333+0000 7fa7bc105b80 1 -- 172.21.15.7:0/1369425313 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7a4004bb0 con 0x563cd3202d90 2024-06-10T01:31:36.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.333+0000 7fa78affd700 1 -- 172.21.15.7:0/1369425313 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fa7a4007700 con 0x563cd3202d90 2024-06-10T01:31:36.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.333+0000 7fa7bc105b80 10 client.21429 fetch_fsmap learned FSMap version 59 2024-06-10T01:31:36.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.333+0000 7fa7bc105b80 1 -- 172.21.15.7:0/1369425313 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x563cd3213dd0 con 0x563cd3202d90 2024-06-10T01:31:36.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.333+0000 7fa78affd700 1 -- 172.21.15.7:0/1369425313 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a4004bb0 con 0x563cd3202d90 2024-06-10T01:31:36.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.333+0000 7fa78affd700 1 -- 172.21.15.7:0/1369425313 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 59) ==== 8366+0+0 (secure 0 0 0) 0x7fa7a4025070 con 0x563cd3202d90 2024-06-10T01:31:36.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.333+0000 7fa7bc105b80 10 client.21429 fetch_fsmap finished waiting for FSMap version 59 2024-06-10T01:31:36.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.333+0000 7fa7bc105b80 10 client.21429 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:31:36.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.333+0000 7fa7bc105b80 20 client.21429 populate_metadata read hostname 'smithi007' 2024-06-10T01:31:36.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.333+0000 7fa7bc105b80 1 --2- 172.21.15.7:0/1369425313 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563cd32162a0 0x563cd3218690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:36.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.333+0000 7fa7bc105b80 4 client.21429 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:31:36.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.333+0000 7fa7bc105b80 1 -- 172.21.15.7:0/1369425313 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563cd315adc0 con 0x563cd32162a0 2024-06-10T01:31:36.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.334+0000 7fa7aaffd700 1 --2- 172.21.15.7:0/1369425313 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563cd32162a0 0x563cd3218690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:36.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.334+0000 7fa78affd700 10 client.21429 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:36.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.334+0000 7fa7aaffd700 1 --2- 172.21.15.7:0/1369425313 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563cd32162a0 0x563cd3218690 crc :-1 s=READY pgs=568 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da04ce537afbcec2 server_cookie=24a8ca767865d2a8 in_seq=0 out_seq=0 2024-06-10T01:31:36.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.335+0000 7fa78affd700 1 -- 172.21.15.7:0/1369425313 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563cd315adc0 con 0x563cd32162a0 2024-06-10T01:31:36.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.335+0000 7fa78affd700 10 client.21429 handle_command_reply: tid=0 2024-06-10T01:31:36.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.335+0000 7fa7bc105b80 1 client.21429 shutdown 2024-06-10T01:31:36.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.335+0000 7fa7bc105b80 1 -- 172.21.15.7:0/1369425313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563cd3202d90 msgr2=0x563cd32062d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:36.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.335+0000 7fa7bc105b80 1 --2- 172.21.15.7:0/1369425313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563cd3202d90 0x563cd32062d0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009510 tx=0x7fa7a4004ad0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.335+0000 7fa7bc105b80 1 -- 172.21.15.7:0/1369425313 shutdown_connections 2024-06-10T01:31:36.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.335+0000 7fa7bc105b80 1 --2- 172.21.15.7:0/1369425313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563cd3202d90 0x563cd32062d0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.335+0000 7fa7bc105b80 1 --2- 172.21.15.7:0/1369425313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cd31fcbc0 0x563cd3208a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.335+0000 7fa7bc105b80 1 --2- 172.21.15.7:0/1369425313 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563cd32162a0 0x563cd3218690 crc :-1 s=READY pgs=568 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.335+0000 7fa7bc105b80 1 --2- 172.21.15.7:0/1369425313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cd3202470 0x563cd3205d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.335+0000 7fa7bc105b80 1 -- 172.21.15.7:0/1369425313 >> 172.21.15.7:0/1369425313 conn(0x563cd31290f0 msgr2=0x563cd31f6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:36.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.335+0000 7fa78affd700 0 client.21429 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:36.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.335+0000 7fa7bc105b80 1 -- 172.21.15.7:0/1369425313 shutdown_connections 2024-06-10T01:31:36.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.335+0000 7fa7bc105b80 1 -- 172.21.15.7:0/1369425313 wait complete. 2024-06-10T01:31:36.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.335+0000 7fa7bc105b80 20 client.21429 trim_cache size 0 max 16384 2024-06-10T01:31:36.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.356+0000 7fa7bc105b80 1 Processor -- start 2024-06-10T01:31:36.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.356+0000 7fa7bc105b80 1 -- start start 2024-06-10T01:31:36.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.356+0000 7fa7bc105b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cd31fcbc0 0x563cd31fc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:36.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.356+0000 7fa7bc105b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563cd3202470 0x563cd3207e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:36.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.356+0000 7fa7bc105b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cd3202d90 0x563cd31f72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:36.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.356+0000 7fa7bc105b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ac1b2db0 con 0x563cd3202470 2024-06-10T01:31:36.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.356+0000 7fa7bc105b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ac1b0580 con 0x563cd31fcbc0 2024-06-10T01:31:36.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.356+0000 7fa7bc105b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ac1ae3d0 con 0x563cd3202d90 2024-06-10T01:31:36.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.356+0000 7fa7aa7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563cd3202470 0x563cd3207e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:36.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.356+0000 7fa7aa7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563cd3202470 0x563cd3207e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39804/0 (socket says 172.21.15.7:39804) 2024-06-10T01:31:36.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.356+0000 7fa7aa7fc700 1 -- 172.21.15.7:0/682400062 learned_addr learned my addr 172.21.15.7:0/682400062 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.356+0000 7fa7ab7fe700 1 --2- 172.21.15.7:0/682400062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cd3202d90 0x563cd31f72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.356+0000 7fa7aaffd700 1 --2- 172.21.15.7:0/682400062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cd31fcbc0 0x563cd31fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.356+0000 7fa7aa7fc700 1 -- 172.21.15.7:0/682400062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cd3202d90 msgr2=0x563cd31f72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.356+0000 7fa7aa7fc700 1 --2- 172.21.15.7:0/682400062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cd3202d90 0x563cd31f72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.356+0000 7fa7aa7fc700 1 -- 172.21.15.7:0/682400062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cd31fcbc0 msgr2=0x563cd31fc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.356+0000 7fa7aa7fc700 1 --2- 172.21.15.7:0/682400062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cd31fcbc0 0x563cd31fc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.356+0000 7fa7aa7fc700 1 -- 172.21.15.7:0/682400062 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cd3205b60 con 0x563cd3202470 2024-06-10T01:31:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.356+0000 7fa7ab7fe700 1 --2- 172.21.15.7:0/682400062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cd3202d90 0x563cd31f72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:31:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.356+0000 7fa7aaffd700 1 --2- 172.21.15.7:0/682400062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cd31fcbc0 0x563cd31fc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:31:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.356+0000 7fa7aa7fc700 1 --2- 172.21.15.7:0/682400062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563cd3202470 0x563cd3207e50 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fa7a00099c0 tx=0x7fa7a0023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbbbcc2b7dc6a88b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.357+0000 7fa7917fa700 1 -- 172.21.15.7:0/682400062 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0035070 con 0x563cd3202470 2024-06-10T01:31:36.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.357+0000 7fa7917fa700 1 -- 172.21.15.7:0/682400062 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a000f460 con 0x563cd3202470 2024-06-10T01:31:36.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.357+0000 7fa7917fa700 1 -- 172.21.15.7:0/682400062 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0030670 con 0x563cd3202470 2024-06-10T01:31:36.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.357+0000 7fa7bc105b80 1 -- 172.21.15.7:0/682400062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563cd3202470 msgr2=0x563cd3207e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:36.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.357+0000 7fa7bc105b80 1 --2- 172.21.15.7:0/682400062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563cd3202470 0x563cd3207e50 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fa7a00099c0 tx=0x7fa7a0023000 comp rx=0 tx=0).stop 2024-06-10T01:31:36.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.357+0000 7fa7bc105b80 1 -- 172.21.15.7:0/682400062 shutdown_connections 2024-06-10T01:31:36.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.357+0000 7fa7bc105b80 1 --2- 172.21.15.7:0/682400062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cd3202d90 0x563cd31f72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.357+0000 7fa7bc105b80 1 --2- 172.21.15.7:0/682400062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cd31fcbc0 0x563cd31fc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.357+0000 7fa7bc105b80 1 --2- 172.21.15.7:0/682400062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563cd3202470 0x563cd3207e50 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.357+0000 7fa7bc105b80 1 -- 172.21.15.7:0/682400062 >> 172.21.15.7:0/682400062 conn(0x563cd31290f0 msgr2=0x563cd32037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:36.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.357+0000 7fa7bc105b80 1 -- 172.21.15.7:0/682400062 shutdown_connections 2024-06-10T01:31:36.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.357+0000 7fa7bc105b80 1 -- 172.21.15.7:0/682400062 wait complete. 2024-06-10T01:31:36.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.358+0000 7fa7bc105b80 1 -- start start 2024-06-10T01:31:36.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.358+0000 7fa7bc105b80 1 Processor -- start 2024-06-10T01:31:36.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.358+0000 7fa7bc105b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cd31fcbc0 0x563cd320ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:36.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.358+0000 7fa7bc105b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cd3202470 0x563cd320f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:36.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.358+0000 7fa7bc105b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563cd3202d90 0x563cd320ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:36.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.358+0000 7fa7bc105b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ac1aea40 con 0x563cd3202d90 2024-06-10T01:31:36.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.358+0000 7fa7bc105b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ac1afeb0 con 0x563cd31fcbc0 2024-06-10T01:31:36.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.358+0000 7fa7bc105b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ac1af110 con 0x563cd3202470 2024-06-10T01:31:36.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.358+0000 7fa7bc105b80 10 client.0 _ll_register_callbacks cb 0x563cd30e3620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:31:36.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.358+0000 7fa7ab7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563cd3202d90 0x563cd320ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:36.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.358+0000 7fa7ab7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563cd3202d90 0x563cd320ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39810/0 (socket says 172.21.15.7:39810) 2024-06-10T01:31:36.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.358+0000 7fa7ab7fe700 1 -- 172.21.15.7:0/2056892760 learned_addr learned my addr 172.21.15.7:0/2056892760 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:36.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.358+0000 7fa7aa7fc700 1 --2- 172.21.15.7:0/2056892760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cd3202470 0x563cd320f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:36.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.358+0000 7fa7aaffd700 1 --2- 172.21.15.7:0/2056892760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cd31fcbc0 0x563cd320ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:36.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.359+0000 7fa7ab7fe700 1 -- 172.21.15.7:0/2056892760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cd3202470 msgr2=0x563cd320f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:36.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.359+0000 7fa7ab7fe700 1 --2- 172.21.15.7:0/2056892760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cd3202470 0x563cd320f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.359+0000 7fa7ab7fe700 1 -- 172.21.15.7:0/2056892760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cd31fcbc0 msgr2=0x563cd320ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:36.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.359+0000 7fa7ab7fe700 1 --2- 172.21.15.7:0/2056892760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cd31fcbc0 0x563cd320ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.359+0000 7fa7ab7fe700 1 -- 172.21.15.7:0/2056892760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cd320c0b0 con 0x563cd3202d90 2024-06-10T01:31:36.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.359+0000 7fa7aaffd700 1 --2- 172.21.15.7:0/2056892760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cd31fcbc0 0x563cd320ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:31:36.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.359+0000 7fa7ab7fe700 1 --2- 172.21.15.7:0/2056892760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563cd3202d90 0x563cd320ba00 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4004940 tx=0x7fa7a4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46a53728f6fd3bea server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:36.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.359+0000 7fa78affd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:31:36.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.359+0000 7fa78affd700 1 -- 172.21.15.7:0/2056892760 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4011e80 con 0x563cd3202d90 2024-06-10T01:31:36.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.359+0000 7fa78affd700 1 -- 172.21.15.7:0/2056892760 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fa7a4027440 con 0x563cd3202d90 2024-06-10T01:31:36.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.359+0000 7fa7bc105b80 1 -- 172.21.15.7:0/2056892760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7a000f460 con 0x563cd3202d90 2024-06-10T01:31:36.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.359+0000 7fa78affd700 1 -- 172.21.15.7:0/2056892760 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fa7a4027640 con 0x563cd3202d90 2024-06-10T01:31:36.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.359+0000 7fa7bc105b80 10 client.21435 fetch_fsmap learned FSMap version 59 2024-06-10T01:31:36.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.359+0000 7fa7bc105b80 1 -- 172.21.15.7:0/2056892760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x563cd32817c0 con 0x563cd3202d90 2024-06-10T01:31:36.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.359+0000 7fa78affd700 1 -- 172.21.15.7:0/2056892760 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a000f460 con 0x563cd3202d90 2024-06-10T01:31:36.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.360+0000 7fa78affd700 1 -- 172.21.15.7:0/2056892760 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 59) ==== 8366+0+0 (secure 0 0 0) 0x7fa7a4025040 con 0x563cd3202d90 2024-06-10T01:31:36.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.360+0000 7fa7bc105b80 10 client.21435 fetch_fsmap finished waiting for FSMap version 59 2024-06-10T01:31:36.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.360+0000 7fa7bc105b80 10 client.21435 resolve_mds: validated gid 24451 aka daemon mds.i 2024-06-10T01:31:36.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.360+0000 7fa7bc105b80 20 client.21435 populate_metadata read hostname 'smithi007' 2024-06-10T01:31:36.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.360+0000 7fa7bc105b80 1 --2- 172.21.15.7:0/2056892760 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563cd3281f00 0x563cd3284320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:36.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.360+0000 7fa7bc105b80 4 client.21435 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-06-10T01:31:36.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.360+0000 7fa7bc105b80 1 -- 172.21.15.7:0/2056892760 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x563cd3155340 con 0x563cd3281f00 2024-06-10T01:31:36.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.360+0000 7fa7aaffd700 1 --2- 172.21.15.7:0/2056892760 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563cd3281f00 0x563cd3284320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:36.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.360+0000 7fa7aaffd700 1 --2- 172.21.15.7:0/2056892760 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563cd3281f00 0x563cd3284320 crc :-1 s=READY pgs=569 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b09c87552697672 server_cookie=c2ae7bc20fc76443 in_seq=0 out_seq=0 2024-06-10T01:31:36.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.360+0000 7fa78affd700 10 client.21435 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:36.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.360+0000 7fa78affd700 1 -- 172.21.15.7:0/2056892760 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x563cd3155340 con 0x563cd3281f00 2024-06-10T01:31:36.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.360+0000 7fa78affd700 10 client.21435 handle_command_reply: tid=0 2024-06-10T01:31:36.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.360+0000 7fa7bc105b80 1 client.21435 shutdown 2024-06-10T01:31:36.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.361+0000 7fa7bc105b80 1 -- 172.21.15.7:0/2056892760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563cd3202d90 msgr2=0x563cd320ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:36.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.361+0000 7fa7bc105b80 1 --2- 172.21.15.7:0/2056892760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563cd3202d90 0x563cd320ba00 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4004940 tx=0x7fa7a4009580 comp rx=0 tx=0).stop 2024-06-10T01:31:36.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.361+0000 7fa7bc105b80 1 -- 172.21.15.7:0/2056892760 shutdown_connections 2024-06-10T01:31:36.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.361+0000 7fa7bc105b80 1 --2- 172.21.15.7:0/2056892760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563cd3202d90 0x563cd320ba00 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.361+0000 7fa7bc105b80 1 --2- 172.21.15.7:0/2056892760 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x563cd3281f00 0x563cd3284320 crc :-1 s=READY pgs=569 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.361+0000 7fa7bc105b80 1 --2- 172.21.15.7:0/2056892760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cd31fcbc0 0x563cd320ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.361+0000 7fa7bc105b80 1 --2- 172.21.15.7:0/2056892760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cd3202470 0x563cd320f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.361+0000 7fa7bc105b80 1 -- 172.21.15.7:0/2056892760 >> 172.21.15.7:0/2056892760 conn(0x563cd31290f0 msgr2=0x563cd3207e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:36.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.361+0000 7fa78affd700 0 client.21435 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:36.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.361+0000 7fa7bc105b80 1 -- 172.21.15.7:0/2056892760 shutdown_connections 2024-06-10T01:31:36.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.361+0000 7fa7bc105b80 1 -- 172.21.15.7:0/2056892760 wait complete. 2024-06-10T01:31:36.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.361+0000 7fa7bc105b80 20 client.21435 trim_cache size 0 max 16384 2024-06-10T01:31:36.361 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:31:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 59, 2024-06-10T01:31:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: "leader": 24451, 2024-06-10T01:31:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: "set_version": 80, 2024-06-10T01:31:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: "sets": {} 2024-06-10T01:31:36.361 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:31:36.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.361+0000 7fa7b1882700 1 -- 172.21.15.7:0/2476989152 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa7940a2c20 msgr2=0x7fa7940a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:36.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.361+0000 7fa7b1882700 1 --2- 172.21.15.7:0/2476989152 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa7940a2c20 0x7fa7940a50e0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac104b50 tx=0x7fa7a0005fb0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.361+0000 7fa7b1882700 1 -- 172.21.15.7:0/2476989152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7ac1016e0 msgr2=0x7fa7ac1030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:36.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.361+0000 7fa7b1882700 1 --2- 172.21.15.7:0/2476989152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7ac1016e0 0x7fa7ac1030c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa79c00c9b0 tx=0x7fa79c00ccc0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.361+0000 7fa7b1882700 1 -- 172.21.15.7:0/2476989152 shutdown_connections 2024-06-10T01:31:36.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.361+0000 7fa7b1882700 1 --2- 172.21.15.7:0/2476989152 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fa7940a2c20 0x7fa7940a50e0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.361+0000 7fa7b1882700 1 --2- 172.21.15.7:0/2476989152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7ac106410 0x7fa7ac103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.361+0000 7fa7b1882700 1 --2- 172.21.15.7:0/2476989152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7ac1016e0 0x7fa7ac1030c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.361+0000 7fa7b1882700 1 --2- 172.21.15.7:0/2476989152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7ac1020a0 0x7fa7ac103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:36.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.362+0000 7fa7b1882700 1 -- 172.21.15.7:0/2476989152 >> 172.21.15.7:0/2476989152 conn(0x7fa7ac077200 msgr2=0x7fa7ac105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:36.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.362+0000 7fa7b1882700 1 -- 172.21.15.7:0/2476989152 shutdown_connections 2024-06-10T01:31:36.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:36.362+0000 7fa7b1882700 1 -- 172.21.15.7:0/2476989152 wait complete. 2024-06-10T01:31:36.950 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:36 smithi175 ceph-mon[36636]: pgmap v1202: 129 pgs: 129 active+clean; 563 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.1 MiB/s wr, 525 op/s 2024-06-10T01:31:37.010 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:36 smithi007 ceph-mon[32398]: pgmap v1202: 129 pgs: 129 active+clean; 563 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.1 MiB/s wr, 525 op/s 2024-06-10T01:31:37.044 INFO:tasks.quiescer:done joining Quiescers 2024-06-10T01:31:37.044 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-06-10T01:31:37.057 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi007.front.sepia.ceph.com 2024-06-10T01:31:37.066 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-06-10T01:31:37.078 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-06-10T01:31:37.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:36 smithi078 ceph-mon[36683]: pgmap v1202: 129 pgs: 129 active+clean; 563 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.1 MiB/s wr, 525 op/s 2024-06-10T01:31:39.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:39 smithi175 ceph-mon[36636]: pgmap v1203: 129 pgs: 129 active+clean; 559 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 5.1 MiB/s wr, 379 op/s 2024-06-10T01:31:39.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:39 smithi007 ceph-mon[32398]: pgmap v1203: 129 pgs: 129 active+clean; 559 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 5.1 MiB/s wr, 379 op/s 2024-06-10T01:31:39.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:39 smithi078 ceph-mon[36683]: pgmap v1203: 129 pgs: 129 active+clean; 559 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 5.1 MiB/s wr, 379 op/s 2024-06-10T01:31:41.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:41 smithi175 ceph-mon[36636]: pgmap v1204: 129 pgs: 129 active+clean; 559 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 4.2 MiB/s wr, 291 op/s 2024-06-10T01:31:41.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:41 smithi007 ceph-mon[32398]: pgmap v1204: 129 pgs: 129 active+clean; 559 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 4.2 MiB/s wr, 291 op/s 2024-06-10T01:31:41.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:41 smithi078 ceph-mon[36683]: pgmap v1204: 129 pgs: 129 active+clean; 559 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 4.2 MiB/s wr, 291 op/s 2024-06-10T01:31:43.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:43 smithi175 ceph-mon[36636]: pgmap v1205: 129 pgs: 129 active+clean; 538 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 5.6 MiB/s wr, 340 op/s 2024-06-10T01:31:43.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:43 smithi007 ceph-mon[32398]: pgmap v1205: 129 pgs: 129 active+clean; 538 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 5.6 MiB/s wr, 340 op/s 2024-06-10T01:31:43.576 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:43 smithi078 ceph-mon[36683]: pgmap v1205: 129 pgs: 129 active+clean; 538 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 5.6 MiB/s wr, 340 op/s 2024-06-10T01:31:45.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:45 smithi175 ceph-mon[36636]: pgmap v1206: 129 pgs: 129 active+clean; 529 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 4.5 MiB/s wr, 176 op/s 2024-06-10T01:31:45.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:45 smithi007 ceph-mon[32398]: pgmap v1206: 129 pgs: 129 active+clean; 529 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 4.5 MiB/s wr, 176 op/s 2024-06-10T01:31:45.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:45 smithi078 ceph-mon[36683]: pgmap v1206: 129 pgs: 129 active+clean; 529 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 4.5 MiB/s wr, 176 op/s 2024-06-10T01:31:47.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:47 smithi175 ceph-mon[36636]: pgmap v1207: 129 pgs: 129 active+clean; 508 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 4.8 MiB/s wr, 216 op/s 2024-06-10T01:31:47.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:47 smithi007 ceph-mon[32398]: pgmap v1207: 129 pgs: 129 active+clean; 508 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 4.8 MiB/s wr, 216 op/s 2024-06-10T01:31:47.576 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:47 smithi078 ceph-mon[36683]: pgmap v1207: 129 pgs: 129 active+clean; 508 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 4.8 MiB/s wr, 216 op/s 2024-06-10T01:31:49.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:49 smithi175 ceph-mon[36636]: pgmap v1208: 129 pgs: 129 active+clean; 499 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 4.3 MiB/s wr, 203 op/s 2024-06-10T01:31:49.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:49 smithi007 ceph-mon[32398]: pgmap v1208: 129 pgs: 129 active+clean; 499 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 4.3 MiB/s wr, 203 op/s 2024-06-10T01:31:49.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:49 smithi078 ceph-mon[36683]: pgmap v1208: 129 pgs: 129 active+clean; 499 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 4.3 MiB/s wr, 203 op/s 2024-06-10T01:31:51.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:51 smithi175 ceph-mon[36636]: pgmap v1209: 129 pgs: 129 active+clean; 498 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 3.4 MiB/s wr, 177 op/s 2024-06-10T01:31:51.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:51 smithi007 ceph-mon[32398]: pgmap v1209: 129 pgs: 129 active+clean; 498 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 3.4 MiB/s wr, 177 op/s 2024-06-10T01:31:51.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:51 smithi078 ceph-mon[36683]: pgmap v1209: 129 pgs: 129 active+clean; 498 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 3.4 MiB/s wr, 177 op/s 2024-06-10T01:31:53.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:53 smithi175 ceph-mon[36636]: pgmap v1210: 129 pgs: 129 active+clean; 491 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 4.3 MiB/s wr, 224 op/s 2024-06-10T01:31:53.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:53 smithi007 ceph-mon[32398]: pgmap v1210: 129 pgs: 129 active+clean; 491 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 4.3 MiB/s wr, 224 op/s 2024-06-10T01:31:53.576 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:53 smithi078 ceph-mon[36683]: pgmap v1210: 129 pgs: 129 active+clean; 491 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 4.3 MiB/s wr, 224 op/s 2024-06-10T01:31:55.401 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:55 smithi078 ceph-mon[36683]: pgmap v1211: 129 pgs: 129 active+clean; 488 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 3.1 MiB/s wr, 176 op/s 2024-06-10T01:31:55.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:55 smithi175 ceph-mon[36636]: pgmap v1211: 129 pgs: 129 active+clean; 488 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 3.1 MiB/s wr, 176 op/s 2024-06-10T01:31:55.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:55 smithi007 ceph-mon[32398]: pgmap v1211: 129 pgs: 129 active+clean; 488 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 3.1 MiB/s wr, 176 op/s 2024-06-10T01:31:57.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:57 smithi175 ceph-mon[36636]: pgmap v1212: 129 pgs: 129 active+clean; 485 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 3.4 MiB/s wr, 193 op/s 2024-06-10T01:31:57.490 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:31:57.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:57 smithi007 ceph-mon[32398]: pgmap v1212: 129 pgs: 129 active+clean; 485 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 3.4 MiB/s wr, 193 op/s 2024-06-10T01:31:57.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:57 smithi078 ceph-mon[36683]: pgmap v1212: 129 pgs: 129 active+clean; 485 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 3.4 MiB/s wr, 193 op/s 2024-06-10T01:31:57.763 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:31:59.418 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:59 smithi007 ceph-mon[32398]: pgmap v1213: 129 pgs: 129 active+clean; 484 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 3.4 MiB/s wr, 163 op/s 2024-06-10T01:31:59.418 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:31:59 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:31:59.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:59 smithi175 ceph-mon[36636]: pgmap v1213: 129 pgs: 129 active+clean; 484 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 3.4 MiB/s wr, 163 op/s 2024-06-10T01:31:59.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:31:59 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:31:59.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:59 smithi078 ceph-mon[36683]: pgmap v1213: 129 pgs: 129 active+clean; 484 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 3.4 MiB/s wr, 163 op/s 2024-06-10T01:31:59.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:31:59 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:31:59.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.861+0000 7fdb522af700 1 -- 172.21.15.7:0/1780204088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb4c101d50 msgr2=0x7fdb4c10e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:59.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.861+0000 7fdb522af700 1 --2- 172.21.15.7:0/1780204088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb4c101d50 0x7fdb4c10e240 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fdb4400b0a0 tx=0x7fdb4400b3b0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.862+0000 7fdb522af700 1 -- 172.21.15.7:0/1780204088 shutdown_connections 2024-06-10T01:31:59.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.862+0000 7fdb522af700 1 --2- 172.21.15.7:0/1780204088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb4c101d50 0x7fdb4c10e240 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.862+0000 7fdb522af700 1 --2- 172.21.15.7:0/1780204088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb4c107400 0x7fdb4c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.862+0000 7fdb522af700 1 --2- 172.21.15.7:0/1780204088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb4c101390 0x7fdb4c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.862+0000 7fdb522af700 1 -- 172.21.15.7:0/1780204088 >> 172.21.15.7:0/1780204088 conn(0x7fdb4c078340 msgr2=0x7fdb4c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:59.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.862+0000 7fdb522af700 1 -- 172.21.15.7:0/1780204088 shutdown_connections 2024-06-10T01:31:59.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.862+0000 7fdb522af700 1 -- 172.21.15.7:0/1780204088 wait complete. 2024-06-10T01:31:59.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.862+0000 7fdb522af700 1 Processor -- start 2024-06-10T01:31:59.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.862+0000 7fdb522af700 1 -- start start 2024-06-10T01:31:59.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.863+0000 7fdb522af700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb4c101390 0x7fdb4c1977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:59.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.863+0000 7fdb522af700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb4c101d50 0x7fdb4c197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:59.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.863+0000 7fdb522af700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb4c107400 0x7fdb4c198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:59.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.863+0000 7fdb522af700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c111670 con 0x7fdb4c101d50 2024-06-10T01:31:59.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.863+0000 7fdb522af700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c1114f0 con 0x7fdb4c107400 2024-06-10T01:31:59.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.863+0000 7fdb522af700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c1117f0 con 0x7fdb4c101390 2024-06-10T01:31:59.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.863+0000 7fdb4bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb4c101390 0x7fdb4c1977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:59.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.863+0000 7fdb5084c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb4c107400 0x7fdb4c198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:59.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.863+0000 7fdb4bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb4c101390 0x7fdb4c1977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43900/0 (socket says 172.21.15.7:43900) 2024-06-10T01:31:59.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.863+0000 7fdb4bfff700 1 -- 172.21.15.7:0/374728899 learned_addr learned my addr 172.21.15.7:0/374728899 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:59.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.863+0000 7fdb43fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb4c101d50 0x7fdb4c197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:59.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.863+0000 7fdb5084c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb4c107400 0x7fdb4c198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:57904/0 (socket says 172.21.15.7:57904) 2024-06-10T01:31:59.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.863+0000 7fdb4bfff700 1 -- 172.21.15.7:0/374728899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb4c107400 msgr2=0x7fdb4c198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:59.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.863+0000 7fdb4bfff700 1 --2- 172.21.15.7:0/374728899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb4c107400 0x7fdb4c198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.863+0000 7fdb4bfff700 1 -- 172.21.15.7:0/374728899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb4c101d50 msgr2=0x7fdb4c197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:59.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.863+0000 7fdb4bfff700 1 --2- 172.21.15.7:0/374728899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb4c101d50 0x7fdb4c197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.863+0000 7fdb4bfff700 1 -- 172.21.15.7:0/374728899 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4c1a7050 con 0x7fdb4c101390 2024-06-10T01:31:59.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.863+0000 7fdb43fff700 1 --2- 172.21.15.7:0/374728899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb4c101d50 0x7fdb4c197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:31:59.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.863+0000 7fdb5084c700 1 --2- 172.21.15.7:0/374728899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb4c107400 0x7fdb4c198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:31:59.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.863+0000 7fdb4bfff700 1 --2- 172.21.15.7:0/374728899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb4c101390 0x7fdb4c1977e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00e9a0 tx=0x7fdb3c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:59.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.864+0000 7fdb49ffb700 1 -- 172.21.15.7:0/374728899 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c00ca10 con 0x7fdb4c101390 2024-06-10T01:31:59.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.864+0000 7fdb49ffb700 1 -- 172.21.15.7:0/374728899 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb3c00cbb0 con 0x7fdb4c101390 2024-06-10T01:31:59.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.864+0000 7fdb522af700 1 -- 172.21.15.7:0/374728899 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb4c1a7340 con 0x7fdb4c101390 2024-06-10T01:31:59.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.864+0000 7fdb49ffb700 1 -- 172.21.15.7:0/374728899 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c018680 con 0x7fdb4c101390 2024-06-10T01:31:59.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.864+0000 7fdb522af700 1 -- 172.21.15.7:0/374728899 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb4c1a7880 con 0x7fdb4c101390 2024-06-10T01:31:59.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.866+0000 7fdb49ffb700 1 -- 172.21.15.7:0/374728899 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fdb3c010430 con 0x7fdb4c101390 2024-06-10T01:31:59.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.867+0000 7fdb49ffb700 1 --2- 172.21.15.7:0/374728899 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdb380a2ca0 0x7fdb380a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:59.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.867+0000 7fdb43fff700 1 --2- 172.21.15.7:0/374728899 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdb380a2ca0 0x7fdb380a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:59.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.867+0000 7fdb49ffb700 1 -- 172.21.15.7:0/374728899 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fdb3c0cf5d0 con 0x7fdb4c101390 2024-06-10T01:31:59.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.867+0000 7fdb5cb32b80 1 Processor -- start 2024-06-10T01:31:59.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.867+0000 7fdb5cb32b80 1 -- start start 2024-06-10T01:31:59.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.867+0000 7fdb43fff700 1 --2- 172.21.15.7:0/374728899 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdb380a2ca0 0x7fdb380a5160 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c199940 tx=0x7fdb3400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:59.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.868+0000 7fdb5cb32b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f060f0770 0x557f060f0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:59.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.868+0000 7fdb5cb32b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f060f1090 0x557f060eaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:59.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.868+0000 7fdb5cb32b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f060eafb0 0x557f060eb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:59.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.868+0000 7fdb5cb32b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557f05fd4f60 con 0x557f060f1090 2024-06-10T01:31:59.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.868+0000 7fdb5cb32b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557f06033850 con 0x557f060eafb0 2024-06-10T01:31:59.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.868+0000 7fdb5cb32b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557f05f9b460 con 0x557f060f0770 2024-06-10T01:31:59.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.868+0000 7fdb43fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f060f1090 0x557f060eaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:59.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.868+0000 7fdb43fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f060f1090 0x557f060eaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55022/0 (socket says 172.21.15.7:55022) 2024-06-10T01:31:59.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.868+0000 7fdb43fff700 1 -- 172.21.15.7:0/600338840 learned_addr learned my addr 172.21.15.7:0/600338840 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:59.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.868+0000 7fdb4bfff700 1 --2- 172.21.15.7:0/600338840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f060f0770 0x557f060f0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:59.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.868+0000 7fdb5084c700 1 --2- 172.21.15.7:0/600338840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f060eafb0 0x557f060eb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:59.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.868+0000 7fdb43fff700 1 -- 172.21.15.7:0/600338840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f060f0770 msgr2=0x557f060f0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:59.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.868+0000 7fdb43fff700 1 --2- 172.21.15.7:0/600338840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f060f0770 0x557f060f0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.868+0000 7fdb43fff700 1 -- 172.21.15.7:0/600338840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f060eafb0 msgr2=0x557f060eb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.868+0000 7fdb43fff700 1 --2- 172.21.15.7:0/600338840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f060eafb0 0x557f060eb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.868+0000 7fdb43fff700 1 -- 172.21.15.7:0/600338840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f05f828d0 con 0x557f060f1090 2024-06-10T01:31:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.868+0000 7fdb4bfff700 1 --2- 172.21.15.7:0/600338840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f060f0770 0x557f060f0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:31:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.868+0000 7fdb43fff700 1 --2- 172.21.15.7:0/600338840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f060f1090 0x557f060eaa70 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fdb34011790 tx=0x7fdb34013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ca21106f5bb992 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.869+0000 7fdb427fc700 1 -- 172.21.15.7:0/600338840 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3401e620 con 0x557f060f1090 2024-06-10T01:31:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.869+0000 7fdb427fc700 1 -- 172.21.15.7:0/600338840 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb34010e70 con 0x557f060f1090 2024-06-10T01:31:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.869+0000 7fdb427fc700 1 -- 172.21.15.7:0/600338840 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3401cc40 con 0x557f060f1090 2024-06-10T01:31:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.869+0000 7fdb5cb32b80 1 -- 172.21.15.7:0/600338840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f060f1090 msgr2=0x557f060eaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.869+0000 7fdb5cb32b80 1 --2- 172.21.15.7:0/600338840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f060f1090 0x557f060eaa70 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fdb34011790 tx=0x7fdb34013000 comp rx=0 tx=0).stop 2024-06-10T01:31:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.869+0000 7fdb5cb32b80 1 -- 172.21.15.7:0/600338840 shutdown_connections 2024-06-10T01:31:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.869+0000 7fdb5cb32b80 1 --2- 172.21.15.7:0/600338840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f060eafb0 0x557f060eb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.869+0000 7fdb5cb32b80 1 --2- 172.21.15.7:0/600338840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f060f0770 0x557f060f0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.869+0000 7fdb5cb32b80 1 --2- 172.21.15.7:0/600338840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f060f1090 0x557f060eaa70 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.869+0000 7fdb5cb32b80 1 -- 172.21.15.7:0/600338840 >> 172.21.15.7:0/600338840 conn(0x557f0600f080 msgr2=0x557f060dd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.869+0000 7fdb5cb32b80 1 -- 172.21.15.7:0/600338840 shutdown_connections 2024-06-10T01:31:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.869+0000 7fdb5cb32b80 1 -- 172.21.15.7:0/600338840 wait complete. 2024-06-10T01:31:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.870+0000 7fdb5cb32b80 1 -- start start 2024-06-10T01:31:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.870+0000 7fdb5cb32b80 1 Processor -- start 2024-06-10T01:31:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.870+0000 7fdb5cb32b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f060eafb0 0x557f060f6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.870+0000 7fdb5cb32b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f060f0770 0x557f060f4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.870+0000 7fdb5cb32b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f060f1090 0x557f060f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.870+0000 7fdb5cb32b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557f05fd3fc0 con 0x557f060f0770 2024-06-10T01:31:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.870+0000 7fdb5cb32b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557f0602f810 con 0x557f060eafb0 2024-06-10T01:31:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.870+0000 7fdb5cb32b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557f060f7a70 con 0x557f060f1090 2024-06-10T01:31:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.870+0000 7fdb5cb32b80 10 client.0 _ll_register_callbacks cb 0x557f05fc95b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:31:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.871+0000 7fdb43fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f060f0770 0x557f060f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.871+0000 7fdb43fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f060f0770 0x557f060f4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55028/0 (socket says 172.21.15.7:55028) 2024-06-10T01:31:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.871+0000 7fdb43fff700 1 -- 172.21.15.7:0/1802350707 learned_addr learned my addr 172.21.15.7:0/1802350707 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:59.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.871+0000 7fdb5084c700 1 --2- 172.21.15.7:0/1802350707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f060f1090 0x557f060f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:59.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.871+0000 7fdb4bfff700 1 --2- 172.21.15.7:0/1802350707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f060eafb0 0x557f060f6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:59.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.871+0000 7fdb43fff700 1 -- 172.21.15.7:0/1802350707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f060f1090 msgr2=0x557f060f46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:59.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.871+0000 7fdb43fff700 1 --2- 172.21.15.7:0/1802350707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f060f1090 0x557f060f46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.871+0000 7fdb43fff700 1 -- 172.21.15.7:0/1802350707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f060eafb0 msgr2=0x557f060f6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:59.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.871+0000 7fdb43fff700 1 --2- 172.21.15.7:0/1802350707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f060eafb0 0x557f060f6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.871+0000 7fdb43fff700 1 -- 172.21.15.7:0/1802350707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f05ff9750 con 0x557f060f0770 2024-06-10T01:31:59.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.871+0000 7fdb4bfff700 1 --2- 172.21.15.7:0/1802350707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f060eafb0 0x557f060f6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:31:59.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.871+0000 7fdb43fff700 1 --2- 172.21.15.7:0/1802350707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f060f0770 0x557f060f4190 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fdb3401c6d0 tx=0x7fdb3401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4d8081b52b13ef server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:59.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.871+0000 7fdb2b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:31:59.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.871+0000 7fdb2b7fe700 1 -- 172.21.15.7:0/1802350707 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb34004220 con 0x557f060f0770 2024-06-10T01:31:59.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.871+0000 7fdb2b7fe700 1 -- 172.21.15.7:0/1802350707 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fdb340043c0 con 0x557f060f0770 2024-06-10T01:31:59.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.871+0000 7fdb5cb32b80 1 -- 172.21.15.7:0/1802350707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb34010e70 con 0x557f060f0770 2024-06-10T01:31:59.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.872+0000 7fdb2b7fe700 1 -- 172.21.15.7:0/1802350707 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fdb3403c050 con 0x557f060f0770 2024-06-10T01:31:59.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.872+0000 7fdb5cb32b80 10 client.21447 fetch_fsmap learned FSMap version 60 2024-06-10T01:31:59.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.872+0000 7fdb5cb32b80 1 -- 172.21.15.7:0/1802350707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x557f061022b0 con 0x557f060f0770 2024-06-10T01:31:59.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.872+0000 7fdb2b7fe700 1 -- 172.21.15.7:0/1802350707 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb34010e70 con 0x557f060f0770 2024-06-10T01:31:59.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.872+0000 7fdb2b7fe700 1 -- 172.21.15.7:0/1802350707 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 60) ==== 8362+0+0 (secure 0 0 0) 0x7fdb34004a90 con 0x557f060f0770 2024-06-10T01:31:59.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.872+0000 7fdb5cb32b80 10 client.21447 fetch_fsmap finished waiting for FSMap version 60 2024-06-10T01:31:59.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.872+0000 7fdb5cb32b80 10 client.21447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:31:59.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.872+0000 7fdb5cb32b80 20 client.21447 populate_metadata read hostname 'smithi007' 2024-06-10T01:31:59.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.872+0000 7fdb5cb32b80 1 --2- 172.21.15.7:0/1802350707 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f06104780 0x557f06106b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.872+0000 7fdb5cb32b80 4 client.21447 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:31:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.873+0000 7fdb5cb32b80 1 -- 172.21.15.7:0/1802350707 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f06040d50 con 0x557f06104780 2024-06-10T01:31:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.873+0000 7fdb4bfff700 1 --2- 172.21.15.7:0/1802350707 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f06104780 0x557f06106b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.873+0000 7fdb2b7fe700 10 client.21447 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.873+0000 7fdb4bfff700 1 --2- 172.21.15.7:0/1802350707 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f06104780 0x557f06106b70 crc :-1 s=READY pgs=570 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6e48cda1227b1ee server_cookie=6f6bd8db6526bc7c in_seq=0 out_seq=0 2024-06-10T01:31:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.874+0000 7fdb2b7fe700 1 -- 172.21.15.7:0/1802350707 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557f06040d50 con 0x557f06104780 2024-06-10T01:31:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.874+0000 7fdb2b7fe700 10 client.21447 handle_command_reply: tid=0 2024-06-10T01:31:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.874+0000 7fdb5cb32b80 1 client.21447 shutdown 2024-06-10T01:31:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.874+0000 7fdb5cb32b80 1 -- 172.21.15.7:0/1802350707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f060f0770 msgr2=0x557f060f4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.874+0000 7fdb5cb32b80 1 --2- 172.21.15.7:0/1802350707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f060f0770 0x557f060f4190 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fdb3401c6d0 tx=0x7fdb3401c460 comp rx=0 tx=0).stop 2024-06-10T01:31:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.874+0000 7fdb5cb32b80 1 -- 172.21.15.7:0/1802350707 shutdown_connections 2024-06-10T01:31:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.875+0000 7fdb5cb32b80 1 --2- 172.21.15.7:0/1802350707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f060f1090 0x557f060f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.875+0000 7fdb5cb32b80 1 --2- 172.21.15.7:0/1802350707 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f06104780 0x557f06106b70 crc :-1 s=READY pgs=570 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.875+0000 7fdb5cb32b80 1 --2- 172.21.15.7:0/1802350707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f060eafb0 0x557f060f6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.875+0000 7fdb2b7fe700 0 client.21447 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.875+0000 7fdb5cb32b80 1 --2- 172.21.15.7:0/1802350707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f060f0770 0x557f060f4190 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.875+0000 7fdb5cb32b80 1 -- 172.21.15.7:0/1802350707 >> 172.21.15.7:0/1802350707 conn(0x557f0600f080 msgr2=0x557f060dc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.875+0000 7fdb5cb32b80 1 -- 172.21.15.7:0/1802350707 shutdown_connections 2024-06-10T01:31:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.875+0000 7fdb5cb32b80 1 -- 172.21.15.7:0/1802350707 wait complete. 2024-06-10T01:31:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.875+0000 7fdb5cb32b80 20 client.21447 trim_cache size 0 max 16384 2024-06-10T01:31:59.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.895+0000 7fdb5cb32b80 1 Processor -- start 2024-06-10T01:31:59.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.896+0000 7fdb5cb32b80 1 -- start start 2024-06-10T01:31:59.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.896+0000 7fdb5cb32b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f0600f080 0x557f060eaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:59.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.896+0000 7fdb5cb32b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f060eafb0 0x557f060f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:59.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.896+0000 7fdb5cb32b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f060f0770 0x557f060dce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:59.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.896+0000 7fdb5cb32b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c1aeac0 con 0x557f0600f080 2024-06-10T01:31:59.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.896+0000 7fdb5cb32b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c1ac2c0 con 0x557f060f0770 2024-06-10T01:31:59.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.896+0000 7fdb5cb32b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c1aa110 con 0x557f060eafb0 2024-06-10T01:31:59.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.896+0000 7fdb4bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f0600f080 0x557f060eaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:59.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.896+0000 7fdb4bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f0600f080 0x557f060eaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55036/0 (socket says 172.21.15.7:55036) 2024-06-10T01:31:59.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.896+0000 7fdb4bfff700 1 -- 172.21.15.7:0/295384229 learned_addr learned my addr 172.21.15.7:0/295384229 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:59.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.896+0000 7fdb43fff700 1 --2- 172.21.15.7:0/295384229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f060eafb0 0x557f060f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:59.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.896+0000 7fdb5084c700 1 --2- 172.21.15.7:0/295384229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f060f0770 0x557f060dce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:59.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.896+0000 7fdb4bfff700 1 -- 172.21.15.7:0/295384229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f060eafb0 msgr2=0x557f060f1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:59.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.896+0000 7fdb4bfff700 1 --2- 172.21.15.7:0/295384229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f060eafb0 0x557f060f1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.896+0000 7fdb4bfff700 1 -- 172.21.15.7:0/295384229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f060f0770 msgr2=0x557f060dce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:59.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.896+0000 7fdb4bfff700 1 --2- 172.21.15.7:0/295384229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f060f0770 0x557f060dce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.896+0000 7fdb4bfff700 1 -- 172.21.15.7:0/295384229 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f060d9810 con 0x557f0600f080 2024-06-10T01:31:59.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.896+0000 7fdb43fff700 1 --2- 172.21.15.7:0/295384229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f060eafb0 0x557f060f1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:31:59.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.896+0000 7fdb4bfff700 1 --2- 172.21.15.7:0/295384229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f0600f080 0x557f060eaa70 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c09b750 tx=0x7fdb3c0ccbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2cb7507abb55ed8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:59.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.896+0000 7fdb41ffb700 1 -- 172.21.15.7:0/295384229 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c099d50 con 0x557f0600f080 2024-06-10T01:31:59.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.897+0000 7fdb41ffb700 1 -- 172.21.15.7:0/295384229 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb3c0cce00 con 0x557f0600f080 2024-06-10T01:31:59.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.897+0000 7fdb41ffb700 1 -- 172.21.15.7:0/295384229 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c010720 con 0x557f0600f080 2024-06-10T01:31:59.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.897+0000 7fdb5cb32b80 1 -- 172.21.15.7:0/295384229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f0600f080 msgr2=0x557f060eaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:59.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.897+0000 7fdb5cb32b80 1 --2- 172.21.15.7:0/295384229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f0600f080 0x557f060eaa70 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c09b750 tx=0x7fdb3c0ccbc0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.897+0000 7fdb5cb32b80 1 -- 172.21.15.7:0/295384229 shutdown_connections 2024-06-10T01:31:59.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.897+0000 7fdb5cb32b80 1 --2- 172.21.15.7:0/295384229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f060f0770 0x557f060dce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.897+0000 7fdb5cb32b80 1 --2- 172.21.15.7:0/295384229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f0600f080 0x557f060eaa70 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.897+0000 7fdb5cb32b80 1 --2- 172.21.15.7:0/295384229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f060eafb0 0x557f060f1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.897+0000 7fdb5cb32b80 1 -- 172.21.15.7:0/295384229 >> 172.21.15.7:0/295384229 conn(0x557f060f4910 msgr2=0x557f060f1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:59.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.897+0000 7fdb5cb32b80 1 -- 172.21.15.7:0/295384229 shutdown_connections 2024-06-10T01:31:59.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.897+0000 7fdb5cb32b80 1 -- 172.21.15.7:0/295384229 wait complete. 2024-06-10T01:31:59.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.897+0000 7fdb5cb32b80 1 -- start start 2024-06-10T01:31:59.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.898+0000 7fdb5cb32b80 1 Processor -- start 2024-06-10T01:31:59.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.898+0000 7fdb5cb32b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f0600f080 0x557f060fcf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:59.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.898+0000 7fdb5cb32b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f060eafb0 0x557f060fd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:59.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.898+0000 7fdb5cb32b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f060f0770 0x557f0610b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:59.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.898+0000 7fdb5cb32b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c1aa780 con 0x557f060f0770 2024-06-10T01:31:59.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.898+0000 7fdb5cb32b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c1abbf0 con 0x557f060eafb0 2024-06-10T01:31:59.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.898+0000 7fdb5cb32b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c1aae50 con 0x557f0600f080 2024-06-10T01:31:59.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.898+0000 7fdb5cb32b80 10 client.0 _ll_register_callbacks cb 0x557f05fc95b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:31:59.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.898+0000 7fdb43fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f060eafb0 0x557f060fd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:59.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.898+0000 7fdb5084c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f060f0770 0x557f0610b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:59.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.898+0000 7fdb43fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f060eafb0 0x557f060fd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:57948/0 (socket says 172.21.15.7:57948) 2024-06-10T01:31:59.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.898+0000 7fdb43fff700 1 -- 172.21.15.7:0/4228015760 learned_addr learned my addr 172.21.15.7:0/4228015760 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:31:59.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.898+0000 7fdb4bfff700 1 --2- 172.21.15.7:0/4228015760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f0600f080 0x557f060fcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:59.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.898+0000 7fdb43fff700 1 -- 172.21.15.7:0/4228015760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f0600f080 msgr2=0x557f060fcf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:59.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.898+0000 7fdb43fff700 1 --2- 172.21.15.7:0/4228015760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f0600f080 0x557f060fcf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.898+0000 7fdb43fff700 1 -- 172.21.15.7:0/4228015760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f060f0770 msgr2=0x557f0610b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:31:59.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.898+0000 7fdb43fff700 1 --2- 172.21.15.7:0/4228015760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f060f0770 0x557f0610b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.898+0000 7fdb43fff700 1 -- 172.21.15.7:0/4228015760 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f0610bd20 con 0x557f060eafb0 2024-06-10T01:31:59.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.898+0000 7fdb4bfff700 1 --2- 172.21.15.7:0/4228015760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f0600f080 0x557f060fcf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:31:59.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.899+0000 7fdb43fff700 1 --2- 172.21.15.7:0/4228015760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f060eafb0 0x557f060fd450 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fdb34005cf0 tx=0x7fdb3401cf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbf2ddbc1baf6290 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:31:59.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.899+0000 7fdb2b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:31:59.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.899+0000 7fdb2b7fe700 1 -- 172.21.15.7:0/4228015760 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb34003880 con 0x557f060eafb0 2024-06-10T01:31:59.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.899+0000 7fdb5cb32b80 1 -- 172.21.15.7:0/4228015760 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb3c0cce00 con 0x557f060eafb0 2024-06-10T01:31:59.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.899+0000 7fdb2b7fe700 1 -- 172.21.15.7:0/4228015760 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fdb34004710 con 0x557f060eafb0 2024-06-10T01:31:59.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.899+0000 7fdb2b7fe700 1 -- 172.21.15.7:0/4228015760 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fdb34004950 con 0x557f060eafb0 2024-06-10T01:31:59.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.899+0000 7fdb5cb32b80 10 client.30632 fetch_fsmap learned FSMap version 60 2024-06-10T01:31:59.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.899+0000 7fdb5cb32b80 1 -- 172.21.15.7:0/4228015760 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x557f0616ec00 con 0x557f060eafb0 2024-06-10T01:31:59.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.899+0000 7fdb2b7fe700 1 -- 172.21.15.7:0/4228015760 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb3c0cce00 con 0x557f060eafb0 2024-06-10T01:31:59.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.900+0000 7fdb2b7fe700 1 -- 172.21.15.7:0/4228015760 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 60) ==== 8362+0+0 (secure 0 0 0) 0x7fdb34003ba0 con 0x557f060eafb0 2024-06-10T01:31:59.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.900+0000 7fdb5cb32b80 10 client.30632 fetch_fsmap finished waiting for FSMap version 60 2024-06-10T01:31:59.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.900+0000 7fdb5cb32b80 10 client.30632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:31:59.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.900+0000 7fdb5cb32b80 20 client.30632 populate_metadata read hostname 'smithi007' 2024-06-10T01:31:59.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.900+0000 7fdb5cb32b80 1 --2- 172.21.15.7:0/4228015760 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f060f1090 0x557f06171320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:31:59.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.900+0000 7fdb5cb32b80 4 client.30632 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:31:59.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.900+0000 7fdb5cb32b80 1 -- 172.21.15.7:0/4228015760 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557f0603b2d0 con 0x557f060f1090 2024-06-10T01:31:59.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.900+0000 7fdb4bfff700 1 --2- 172.21.15.7:0/4228015760 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f060f1090 0x557f06171320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:31:59.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.900+0000 7fdb2b7fe700 10 client.30632 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:59.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.900+0000 7fdb4bfff700 1 --2- 172.21.15.7:0/4228015760 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f060f1090 0x557f06171320 crc :-1 s=READY pgs=571 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7eb79c78965039a server_cookie=302c17ac7cde1a6 in_seq=0 out_seq=0 2024-06-10T01:31:59.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.900+0000 7fdb2b7fe700 1 -- 172.21.15.7:0/4228015760 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557f0603b2d0 con 0x557f060f1090 2024-06-10T01:31:59.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.900+0000 7fdb2b7fe700 10 client.30632 handle_command_reply: tid=0 2024-06-10T01:31:59.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.900+0000 7fdb5cb32b80 1 client.30632 shutdown 2024-06-10T01:31:59.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.901+0000 7fdb5cb32b80 1 -- 172.21.15.7:0/4228015760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f060eafb0 msgr2=0x557f060fd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:59.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.901+0000 7fdb5cb32b80 1 --2- 172.21.15.7:0/4228015760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f060eafb0 0x557f060fd450 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fdb34005cf0 tx=0x7fdb3401cf70 comp rx=0 tx=0).stop 2024-06-10T01:31:59.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.901+0000 7fdb5cb32b80 1 -- 172.21.15.7:0/4228015760 shutdown_connections 2024-06-10T01:31:59.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.901+0000 7fdb5cb32b80 1 --2- 172.21.15.7:0/4228015760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f060f0770 0x557f0610b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.901+0000 7fdb5cb32b80 1 --2- 172.21.15.7:0/4228015760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f0600f080 0x557f060fcf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.901+0000 7fdb5cb32b80 1 --2- 172.21.15.7:0/4228015760 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x557f060f1090 0x557f06171320 crc :-1 s=READY pgs=571 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.901+0000 7fdb5cb32b80 1 --2- 172.21.15.7:0/4228015760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f060eafb0 0x557f060fd450 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.901+0000 7fdb2b7fe700 0 client.30632 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:31:59.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.901+0000 7fdb5cb32b80 1 -- 172.21.15.7:0/4228015760 >> 172.21.15.7:0/4228015760 conn(0x557f060f4910 msgr2=0x557f060ef330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:59.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.901+0000 7fdb5cb32b80 1 -- 172.21.15.7:0/4228015760 shutdown_connections 2024-06-10T01:31:59.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.901+0000 7fdb5cb32b80 1 -- 172.21.15.7:0/4228015760 wait complete. 2024-06-10T01:31:59.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.901+0000 7fdb5cb32b80 20 client.30632 trim_cache size 0 max 16384 2024-06-10T01:31:59.899 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:31:59.899 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-10T01:31:59.899 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-10T01:31:59.899 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:31:59.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.901+0000 7fdb522af700 1 -- 172.21.15.7:0/374728899 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdb380a2ca0 msgr2=0x7fdb380a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:59.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.901+0000 7fdb522af700 1 --2- 172.21.15.7:0/374728899 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdb380a2ca0 0x7fdb380a5160 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c199940 tx=0x7fdb3400b410 comp rx=0 tx=0).stop 2024-06-10T01:31:59.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.901+0000 7fdb522af700 1 -- 172.21.15.7:0/374728899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb4c101390 msgr2=0x7fdb4c1977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:31:59.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.901+0000 7fdb522af700 1 --2- 172.21.15.7:0/374728899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb4c101390 0x7fdb4c1977e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00e9a0 tx=0x7fdb3c00ecb0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.902+0000 7fdb522af700 1 -- 172.21.15.7:0/374728899 shutdown_connections 2024-06-10T01:31:59.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.902+0000 7fdb522af700 1 --2- 172.21.15.7:0/374728899 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdb380a2ca0 0x7fdb380a5160 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.902+0000 7fdb522af700 1 --2- 172.21.15.7:0/374728899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb4c107400 0x7fdb4c198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.902+0000 7fdb522af700 1 --2- 172.21.15.7:0/374728899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb4c101390 0x7fdb4c1977e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.902+0000 7fdb522af700 1 --2- 172.21.15.7:0/374728899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb4c101d50 0x7fdb4c197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:31:59.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.902+0000 7fdb522af700 1 -- 172.21.15.7:0/374728899 >> 172.21.15.7:0/374728899 conn(0x7fdb4c078340 msgr2=0x7fdb4c0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:31:59.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.902+0000 7fdb522af700 1 -- 172.21.15.7:0/374728899 shutdown_connections 2024-06-10T01:31:59.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:31:59.902+0000 7fdb522af700 1 -- 172.21.15.7:0/374728899 wait complete. 2024-06-10T01:32:00.786 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:32:00.786 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:32:01.083 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:32:01.449 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:01 smithi007 ceph-mon[32398]: pgmap v1214: 129 pgs: 129 active+clean; 484 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.7 MiB/s wr, 134 op/s 2024-06-10T01:32:01.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:01 smithi175 ceph-mon[36636]: pgmap v1214: 129 pgs: 129 active+clean; 484 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.7 MiB/s wr, 134 op/s 2024-06-10T01:32:01.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:01 smithi078 ceph-mon[36683]: pgmap v1214: 129 pgs: 129 active+clean; 484 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.7 MiB/s wr, 134 op/s 2024-06-10T01:32:02.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.875+0000 7f460b5ce700 1 -- 172.21.15.7:0/3766203504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46041030f0 msgr2=0x7f4604107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:02.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.875+0000 7f460b5ce700 1 --2- 172.21.15.7:0/3766203504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46041030f0 0x7f4604107110 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f45f40099c0 tx=0x7f45f4009cd0 comp rx=0 tx=0).stop 2024-06-10T01:32:02.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.875+0000 7f460b5ce700 1 -- 172.21.15.7:0/3766203504 shutdown_connections 2024-06-10T01:32:02.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.875+0000 7f460b5ce700 1 --2- 172.21.15.7:0/3766203504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4604107840 0x7f460410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:02.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.875+0000 7f460b5ce700 1 --2- 172.21.15.7:0/3766203504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4604102740 0x7f4604102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:02.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.875+0000 7f460b5ce700 1 --2- 172.21.15.7:0/3766203504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46041030f0 0x7f4604107110 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:02.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.875+0000 7f460b5ce700 1 -- 172.21.15.7:0/3766203504 >> 172.21.15.7:0/3766203504 conn(0x7f46040795f0 msgr2=0x7f4604079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:02.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.876+0000 7f460b5ce700 1 -- 172.21.15.7:0/3766203504 shutdown_connections 2024-06-10T01:32:02.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.876+0000 7f460b5ce700 1 -- 172.21.15.7:0/3766203504 wait complete. 2024-06-10T01:32:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.876+0000 7f460b5ce700 1 Processor -- start 2024-06-10T01:32:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.876+0000 7f460b5ce700 1 -- start start 2024-06-10T01:32:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.876+0000 7f460b5ce700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4604102740 0x7f4604074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.876+0000 7f460b5ce700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46041030f0 0x7f4604074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.877+0000 7f460b5ce700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4604107840 0x7f4604075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.877+0000 7f460b5ce700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4604112770 con 0x7f4604107840 2024-06-10T01:32:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.877+0000 7f460b5ce700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f46041125f0 con 0x7f4604102740 2024-06-10T01:32:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.877+0000 7f460b5ce700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f46041128f0 con 0x7f46041030f0 2024-06-10T01:32:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.877+0000 7f4609b6b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4604107840 0x7f4604075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.877+0000 7f4609b6b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4604107840 0x7f4604075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43380/0 (socket says 172.21.15.7:43380) 2024-06-10T01:32:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.877+0000 7f4609b6b700 1 -- 172.21.15.7:0/1102751334 learned_addr learned my addr 172.21.15.7:0/1102751334 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:32:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.877+0000 7f4608b69700 1 --2- 172.21.15.7:0/1102751334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46041030f0 0x7f4604074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.877+0000 7f460936a700 1 --2- 172.21.15.7:0/1102751334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4604102740 0x7f4604074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.877+0000 7f4609b6b700 1 -- 172.21.15.7:0/1102751334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46041030f0 msgr2=0x7f4604074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.877+0000 7f4609b6b700 1 --2- 172.21.15.7:0/1102751334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46041030f0 0x7f4604074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.877+0000 7f4609b6b700 1 -- 172.21.15.7:0/1102751334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4604102740 msgr2=0x7f4604074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.877+0000 7f4609b6b700 1 --2- 172.21.15.7:0/1102751334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4604102740 0x7f4604074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.877+0000 7f4609b6b700 1 -- 172.21.15.7:0/1102751334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46041a7050 con 0x7f4604107840 2024-06-10T01:32:02.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.877+0000 7f460936a700 1 --2- 172.21.15.7:0/1102751334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4604102740 0x7f4604074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:32:02.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.877+0000 7f4609b6b700 1 --2- 172.21.15.7:0/1102751334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4604107840 0x7f4604075210 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f45fc007d70 tx=0x7f45fc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:02.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.877+0000 7f45fa7fc700 1 -- 172.21.15.7:0/1102751334 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45fc016020 con 0x7f4604107840 2024-06-10T01:32:02.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.877+0000 7f460b5ce700 1 -- 172.21.15.7:0/1102751334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46041a7340 con 0x7f4604107840 2024-06-10T01:32:02.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.877+0000 7f45fa7fc700 1 -- 172.21.15.7:0/1102751334 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45fc00ad40 con 0x7f4604107840 2024-06-10T01:32:02.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.877+0000 7f45fa7fc700 1 -- 172.21.15.7:0/1102751334 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45fc011600 con 0x7f4604107840 2024-06-10T01:32:02.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.878+0000 7f460b5ce700 1 -- 172.21.15.7:0/1102751334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46041a7800 con 0x7f4604107840 2024-06-10T01:32:02.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.879+0000 7f460b5ce700 1 -- 172.21.15.7:0/1102751334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f460410aa10 con 0x7f4604107840 2024-06-10T01:32:02.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.881+0000 7f45fa7fc700 1 -- 172.21.15.7:0/1102751334 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f45fc019020 con 0x7f4604107840 2024-06-10T01:32:02.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.882+0000 7f45fa7fc700 1 --2- 172.21.15.7:0/1102751334 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f45f00a2c20 0x7f45f00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:02.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.882+0000 7f460936a700 1 --2- 172.21.15.7:0/1102751334 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f45f00a2c20 0x7f45f00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:02.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.882+0000 7f45fa7fc700 1 -- 172.21.15.7:0/1102751334 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f45fc0cf790 con 0x7f4604107840 2024-06-10T01:32:02.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.882+0000 7f460936a700 1 --2- 172.21.15.7:0/1102751334 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f45f00a2c20 0x7f45f00a50e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f460000bcf0 tx=0x7f460000b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:02.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:02.885+0000 7f45fa7fc700 1 -- 172.21.15.7:0/1102751334 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f45fc020020 con 0x7f4604107840 2024-06-10T01:32:03.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:03.137+0000 7f460b5ce700 1 -- 172.21.15.7:0/1102751334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4604069180 con 0x7f4604107840 2024-06-10T01:32:03.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:03.138+0000 7f45fa7fc700 1 -- 172.21.15.7:0/1102751334 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+11857 (secure 0 0 0) 0x7f45fc09c310 con 0x7f4604107840 2024-06-10T01:32:03.131 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:32:03.131 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":60,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:31:57.480771+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29813":{"gid":29813,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/138896429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":138896429},{"type":"v1","addr":"172.21.15.175:6835","nonce":138896429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24434,24436,24451]},"id":1}]} 2024-06-10T01:32:03.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:03.140+0000 7f460b5ce700 1 -- 172.21.15.7:0/1102751334 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f45f00a2c20 msgr2=0x7f45f00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:03.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:03.140+0000 7f460b5ce700 1 --2- 172.21.15.7:0/1102751334 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f45f00a2c20 0x7f45f00a50e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f460000bcf0 tx=0x7f460000b2c0 comp rx=0 tx=0).stop 2024-06-10T01:32:03.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:03.140+0000 7f460b5ce700 1 -- 172.21.15.7:0/1102751334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4604107840 msgr2=0x7f4604075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:03.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:03.140+0000 7f460b5ce700 1 --2- 172.21.15.7:0/1102751334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4604107840 0x7f4604075210 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f45fc007d70 tx=0x7f45fc00a340 comp rx=0 tx=0).stop 2024-06-10T01:32:03.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:03.140+0000 7f460b5ce700 1 -- 172.21.15.7:0/1102751334 shutdown_connections 2024-06-10T01:32:03.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:03.140+0000 7f460b5ce700 1 --2- 172.21.15.7:0/1102751334 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f45f00a2c20 0x7f45f00a50e0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:03.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:03.140+0000 7f460b5ce700 1 --2- 172.21.15.7:0/1102751334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4604107840 0x7f4604075210 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:03.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:03.140+0000 7f460b5ce700 1 --2- 172.21.15.7:0/1102751334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4604102740 0x7f4604074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:03.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:03.140+0000 7f460b5ce700 1 --2- 172.21.15.7:0/1102751334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46041030f0 0x7f4604074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:03.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:03.140+0000 7f460b5ce700 1 -- 172.21.15.7:0/1102751334 >> 172.21.15.7:0/1102751334 conn(0x7f46040795f0 msgr2=0x7f4604101c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:03.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:03.141+0000 7f460b5ce700 1 -- 172.21.15.7:0/1102751334 shutdown_connections 2024-06-10T01:32:03.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:03.141+0000 7f460b5ce700 1 -- 172.21.15.7:0/1102751334 wait complete. 2024-06-10T01:32:03.135 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 60 2024-06-10T01:32:03.438 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:03 smithi007 ceph-mon[32398]: pgmap v1215: 129 pgs: 129 active+clean; 474 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 2.7 MiB/s wr, 154 op/s 2024-06-10T01:32:03.438 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:03 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1102751334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:32:03.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:03 smithi175 ceph-mon[36636]: pgmap v1215: 129 pgs: 129 active+clean; 474 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 2.7 MiB/s wr, 154 op/s 2024-06-10T01:32:03.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:03 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1102751334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:32:03.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:03 smithi078 ceph-mon[36683]: pgmap v1215: 129 pgs: 129 active+clean; 474 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 2.7 MiB/s wr, 154 op/s 2024-06-10T01:32:03.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:03 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1102751334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:32:03.862 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:32:04.115 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:32:05.260 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:05 smithi007 ceph-mon[32398]: pgmap v1216: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 1.8 MiB/s wr, 108 op/s 2024-06-10T01:32:05.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:05 smithi175 ceph-mon[36636]: pgmap v1216: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 1.8 MiB/s wr, 108 op/s 2024-06-10T01:32:05.576 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:05 smithi078 ceph-mon[36683]: pgmap v1216: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 1.8 MiB/s wr, 108 op/s 2024-06-10T01:32:05.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.913+0000 7f4a5f5b7700 1 -- 172.21.15.7:0/2029618314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a58107400 msgr2=0x7f4a581077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:05.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.913+0000 7f4a5f5b7700 1 --2- 172.21.15.7:0/2029618314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a58107400 0x7f4a581077e0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009990 tx=0x7f4a54009ca0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.913+0000 7f4a5f5b7700 1 -- 172.21.15.7:0/2029618314 shutdown_connections 2024-06-10T01:32:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.913+0000 7f4a5f5b7700 1 --2- 172.21.15.7:0/2029618314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a58101d50 0x7f4a5810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.913+0000 7f4a5f5b7700 1 --2- 172.21.15.7:0/2029618314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a58107400 0x7f4a581077e0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.913+0000 7f4a5f5b7700 1 --2- 172.21.15.7:0/2029618314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a58101390 0x7f4a58101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.913+0000 7f4a5f5b7700 1 -- 172.21.15.7:0/2029618314 >> 172.21.15.7:0/2029618314 conn(0x7f4a58078340 msgr2=0x7f4a58078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.913+0000 7f4a5f5b7700 1 -- 172.21.15.7:0/2029618314 shutdown_connections 2024-06-10T01:32:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.913+0000 7f4a5f5b7700 1 -- 172.21.15.7:0/2029618314 wait complete. 2024-06-10T01:32:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.914+0000 7f4a5f5b7700 1 Processor -- start 2024-06-10T01:32:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.914+0000 7f4a5f5b7700 1 -- start start 2024-06-10T01:32:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.914+0000 7f4a5f5b7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a58101390 0x7f4a58103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.914+0000 7f4a5f5b7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a58101d50 0x7f4a581035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.914+0000 7f4a5f5b7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a58107400 0x7f4a58103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.914+0000 7f4a5f5b7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4a58111670 con 0x7f4a58101d50 2024-06-10T01:32:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.914+0000 7f4a5f5b7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581114f0 con 0x7f4a58107400 2024-06-10T01:32:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.914+0000 7f4a5f5b7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581117f0 con 0x7f4a58101390 2024-06-10T01:32:05.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.915+0000 7f4a5cb52700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a58101d50 0x7f4a581035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:05.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.915+0000 7f4a5cb52700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a58101d50 0x7f4a581035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43408/0 (socket says 172.21.15.7:43408) 2024-06-10T01:32:05.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.915+0000 7f4a5cb52700 1 -- 172.21.15.7:0/1752548859 learned_addr learned my addr 172.21.15.7:0/1752548859 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:32:05.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.915+0000 7f4a5d353700 1 --2- 172.21.15.7:0/1752548859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a58101390 0x7f4a58103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:05.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.915+0000 7f4a5db54700 1 --2- 172.21.15.7:0/1752548859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a58107400 0x7f4a58103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:05.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.915+0000 7f4a5cb52700 1 -- 172.21.15.7:0/1752548859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a58101390 msgr2=0x7f4a58103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:05.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.915+0000 7f4a5cb52700 1 --2- 172.21.15.7:0/1752548859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a58101390 0x7f4a58103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.915+0000 7f4a5cb52700 1 -- 172.21.15.7:0/1752548859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a58107400 msgr2=0x7f4a58103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:05.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.915+0000 7f4a5cb52700 1 --2- 172.21.15.7:0/1752548859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a58107400 0x7f4a58103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.915+0000 7f4a5cb52700 1 -- 172.21.15.7:0/1752548859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a581a7110 con 0x7f4a58101d50 2024-06-10T01:32:05.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.915+0000 7f4a5db54700 1 --2- 172.21.15.7:0/1752548859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a58107400 0x7f4a58103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:32:05.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.915+0000 7f4a5cb52700 1 --2- 172.21.15.7:0/1752548859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a58101d50 0x7f4a581035b0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800e9a0 tx=0x7f4a4800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:05.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.915+0000 7f4a4e7fc700 1 -- 172.21.15.7:0/1752548859 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4800cbd0 con 0x7f4a58101d50 2024-06-10T01:32:05.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.915+0000 7f4a4e7fc700 1 -- 172.21.15.7:0/1752548859 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a4800cd70 con 0x7f4a58101d50 2024-06-10T01:32:05.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.915+0000 7f4a5f5b7700 1 -- 172.21.15.7:0/1752548859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a581a7400 con 0x7f4a58101d50 2024-06-10T01:32:05.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.916+0000 7f4a4e7fc700 1 -- 172.21.15.7:0/1752548859 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a48018750 con 0x7f4a58101d50 2024-06-10T01:32:05.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.916+0000 7f4a5f5b7700 1 -- 172.21.15.7:0/1752548859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a581a7990 con 0x7f4a58101d50 2024-06-10T01:32:05.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.918+0000 7f4a4e7fc700 1 -- 172.21.15.7:0/1752548859 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f4a48010430 con 0x7f4a58101d50 2024-06-10T01:32:05.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.919+0000 7f4a4e7fc700 1 --2- 172.21.15.7:0/1752548859 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4a440a2ca0 0x7f4a440a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:05.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.919+0000 7f4a5d353700 1 --2- 172.21.15.7:0/1752548859 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4a440a2ca0 0x7f4a440a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:05.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.919+0000 7f4a4e7fc700 1 -- 172.21.15.7:0/1752548859 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f4a480cf660 con 0x7f4a58101d50 2024-06-10T01:32:05.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.919+0000 7f4a69e3ab80 1 Processor -- start 2024-06-10T01:32:05.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.919+0000 7f4a69e3ab80 1 -- start start 2024-06-10T01:32:05.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.919+0000 7f4a5d353700 1 --2- 172.21.15.7:0/1752548859 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4a440a2ca0 0x7f4a440a5160 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400b2e0 tx=0x7f4a54005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:05.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.920+0000 7f4a69e3ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a8652640 0x55e5a8652a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:05.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.920+0000 7f4a69e3ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a8652f60 0x55e5a864c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:05.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.920+0000 7f4a69e3ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5a864cf20 0x55e5a864d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:05.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.920+0000 7f4a69e3ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e5a859e810 con 0x55e5a8652f60 2024-06-10T01:32:05.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.920+0000 7f4a69e3ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e5a8546720 con 0x55e5a8652640 2024-06-10T01:32:05.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.920+0000 7f4a69e3ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e5a859a7d0 con 0x55e5a864cf20 2024-06-10T01:32:05.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.920+0000 7f4a5cb52700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a8652f60 0x55e5a864c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:05.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.920+0000 7f4a5cb52700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a8652f60 0x55e5a864c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43416/0 (socket says 172.21.15.7:43416) 2024-06-10T01:32:05.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.920+0000 7f4a5cb52700 1 -- 172.21.15.7:0/956892783 learned_addr learned my addr 172.21.15.7:0/956892783 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:32:05.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.920+0000 7f4a5d353700 1 --2- 172.21.15.7:0/956892783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a8652640 0x55e5a8652a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:05.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.920+0000 7f4a5db54700 1 --2- 172.21.15.7:0/956892783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5a864cf20 0x55e5a864d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:05.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.920+0000 7f4a5cb52700 1 -- 172.21.15.7:0/956892783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5a864cf20 msgr2=0x55e5a864d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:32:05.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.920+0000 7f4a5cb52700 1 --2- 172.21.15.7:0/956892783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5a864cf20 0x55e5a864d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.920+0000 7f4a5cb52700 1 -- 172.21.15.7:0/956892783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a8652640 msgr2=0x55e5a8652a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:32:05.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.920+0000 7f4a5cb52700 1 --2- 172.21.15.7:0/956892783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a8652640 0x55e5a8652a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.920+0000 7f4a5cb52700 1 -- 172.21.15.7:0/956892783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5a853cf50 con 0x55e5a8652f60 2024-06-10T01:32:05.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.920+0000 7f4a5cb52700 1 --2- 172.21.15.7:0/956892783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a8652f60 0x55e5a864c9e0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f4a480a4220 tx=0x7f4a4809c1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f29482d21de8745 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:05.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.920+0000 7f4a437fe700 1 -- 172.21.15.7:0/956892783 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4809fe90 con 0x55e5a8652f60 2024-06-10T01:32:05.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.921+0000 7f4a437fe700 1 -- 172.21.15.7:0/956892783 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a480a4750 con 0x55e5a8652f60 2024-06-10T01:32:05.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.921+0000 7f4a437fe700 1 -- 172.21.15.7:0/956892783 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a480a4520 con 0x55e5a8652f60 2024-06-10T01:32:05.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.921+0000 7f4a69e3ab80 1 -- 172.21.15.7:0/956892783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a8652f60 msgr2=0x55e5a864c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:05.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.921+0000 7f4a69e3ab80 1 --2- 172.21.15.7:0/956892783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a8652f60 0x55e5a864c9e0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f4a480a4220 tx=0x7f4a4809c1e0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.921+0000 7f4a69e3ab80 1 -- 172.21.15.7:0/956892783 shutdown_connections 2024-06-10T01:32:05.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.921+0000 7f4a69e3ab80 1 --2- 172.21.15.7:0/956892783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5a864cf20 0x55e5a864d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.921+0000 7f4a69e3ab80 1 --2- 172.21.15.7:0/956892783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a8652640 0x55e5a8652a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.921+0000 7f4a69e3ab80 1 --2- 172.21.15.7:0/956892783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a8652f60 0x55e5a864c9e0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.921+0000 7f4a69e3ab80 1 -- 172.21.15.7:0/956892783 >> 172.21.15.7:0/956892783 conn(0x55e5a8577cc0 msgr2=0x55e5a8647850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:05.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.921+0000 7f4a69e3ab80 1 -- 172.21.15.7:0/956892783 shutdown_connections 2024-06-10T01:32:05.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.921+0000 7f4a69e3ab80 1 -- 172.21.15.7:0/956892783 wait complete. 2024-06-10T01:32:05.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.922+0000 7f4a69e3ab80 1 -- start start 2024-06-10T01:32:05.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.922+0000 7f4a69e3ab80 1 Processor -- start 2024-06-10T01:32:05.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.922+0000 7f4a69e3ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a864cf20 0x55e5a8658d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:05.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.922+0000 7f4a69e3ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5a8652640 0x55e5a8656030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:05.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.922+0000 7f4a69e3ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a8652f60 0x55e5a8656570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:05.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.922+0000 7f4a69e3ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e5a86599d0 con 0x55e5a8652f60 2024-06-10T01:32:05.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.922+0000 7f4a69e3ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e5a853ebf0 con 0x55e5a864cf20 2024-06-10T01:32:05.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.922+0000 7f4a69e3ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e5a8659b50 con 0x55e5a8652640 2024-06-10T01:32:05.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.922+0000 7f4a69e3ab80 10 client.0 _ll_register_callbacks cb 0x55e5a8538920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:32:05.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.922+0000 7f4a5db54700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a8652f60 0x55e5a8656570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:05.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.922+0000 7f4a5db54700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a8652f60 0x55e5a8656570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43432/0 (socket says 172.21.15.7:43432) 2024-06-10T01:32:05.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.922+0000 7f4a5db54700 1 -- 172.21.15.7:0/3923002532 learned_addr learned my addr 172.21.15.7:0/3923002532 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:32:05.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.923+0000 7f4a5cb52700 1 --2- 172.21.15.7:0/3923002532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5a8652640 0x55e5a8656030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:05.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.923+0000 7f4a5d353700 1 --2- 172.21.15.7:0/3923002532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a864cf20 0x55e5a8658d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:05.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.923+0000 7f4a5db54700 1 -- 172.21.15.7:0/3923002532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5a8652640 msgr2=0x55e5a8656030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:32:05.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.923+0000 7f4a5db54700 1 --2- 172.21.15.7:0/3923002532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5a8652640 0x55e5a8656030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.923+0000 7f4a5db54700 1 -- 172.21.15.7:0/3923002532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a864cf20 msgr2=0x55e5a8658d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:32:05.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.923+0000 7f4a5db54700 1 --2- 172.21.15.7:0/3923002532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a864cf20 0x55e5a8658d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.923+0000 7f4a5db54700 1 -- 172.21.15.7:0/3923002532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5a84ed8d0 con 0x55e5a8652f60 2024-06-10T01:32:05.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.923+0000 7f4a5db54700 1 --2- 172.21.15.7:0/3923002532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a8652f60 0x55e5a8656570 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f4a500094f0 tx=0x7f4a50007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc28ae46b4f383b8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:05.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.923+0000 7f4a40ff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:32:05.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.923+0000 7f4a40ff9700 1 -- 172.21.15.7:0/3923002532 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a50012c40 con 0x55e5a8652f60 2024-06-10T01:32:05.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.923+0000 7f4a40ff9700 1 -- 172.21.15.7:0/3923002532 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f4a50012de0 con 0x55e5a8652f60 2024-06-10T01:32:05.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.923+0000 7f4a69e3ab80 1 -- 172.21.15.7:0/3923002532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a480a4750 con 0x55e5a8652f60 2024-06-10T01:32:05.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.923+0000 7f4a40ff9700 1 -- 172.21.15.7:0/3923002532 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f4a50025050 con 0x55e5a8652f60 2024-06-10T01:32:05.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.923+0000 7f4a69e3ab80 10 client.21471 fetch_fsmap learned FSMap version 60 2024-06-10T01:32:05.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.923+0000 7f4a69e3ab80 1 -- 172.21.15.7:0/3923002532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55e5a86643e0 con 0x55e5a8652f60 2024-06-10T01:32:05.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.924+0000 7f4a40ff9700 1 -- 172.21.15.7:0/3923002532 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a480a4750 con 0x55e5a8652f60 2024-06-10T01:32:05.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.924+0000 7f4a40ff9700 1 -- 172.21.15.7:0/3923002532 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 60) ==== 8362+0+0 (secure 0 0 0) 0x7f4a5001e5c0 con 0x55e5a8652f60 2024-06-10T01:32:05.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.924+0000 7f4a69e3ab80 10 client.21471 fetch_fsmap finished waiting for FSMap version 60 2024-06-10T01:32:05.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.924+0000 7f4a69e3ab80 10 client.21471 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:32:05.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.924+0000 7f4a69e3ab80 20 client.21471 populate_metadata read hostname 'smithi007' 2024-06-10T01:32:05.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.924+0000 7f4a69e3ab80 1 --2- 172.21.15.7:0/3923002532 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55e5a8666a20 0x55e5a8668e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:05.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.924+0000 7f4a69e3ab80 4 client.21471 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:32:05.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.924+0000 7f4a69e3ab80 1 -- 172.21.15.7:0/3923002532 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5a85abd10 con 0x55e5a8666a20 2024-06-10T01:32:05.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.924+0000 7f4a5d353700 1 --2- 172.21.15.7:0/3923002532 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55e5a8666a20 0x55e5a8668e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:05.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.925+0000 7f4a5d353700 1 --2- 172.21.15.7:0/3923002532 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55e5a8666a20 0x55e5a8668e50 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=690ee7befee76e38 server_cookie=4d681d4927162819 in_seq=0 out_seq=0 2024-06-10T01:32:05.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.925+0000 7f4a40ff9700 10 client.21471 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:32:05.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.925+0000 7f4a40ff9700 1 -- 172.21.15.7:0/3923002532 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e5a85abd10 con 0x55e5a8666a20 2024-06-10T01:32:05.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.925+0000 7f4a40ff9700 10 client.21471 handle_command_reply: tid=0 2024-06-10T01:32:05.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.925+0000 7f4a69e3ab80 1 client.21471 shutdown 2024-06-10T01:32:05.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.926+0000 7f4a69e3ab80 1 -- 172.21.15.7:0/3923002532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a8652f60 msgr2=0x55e5a8656570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:05.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.926+0000 7f4a69e3ab80 1 --2- 172.21.15.7:0/3923002532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a8652f60 0x55e5a8656570 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f4a500094f0 tx=0x7f4a50007e10 comp rx=0 tx=0).stop 2024-06-10T01:32:05.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.926+0000 7f4a69e3ab80 1 -- 172.21.15.7:0/3923002532 shutdown_connections 2024-06-10T01:32:05.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.926+0000 7f4a69e3ab80 1 --2- 172.21.15.7:0/3923002532 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55e5a8666a20 0x55e5a8668e50 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.926+0000 7f4a69e3ab80 1 --2- 172.21.15.7:0/3923002532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a8652f60 0x55e5a8656570 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.926+0000 7f4a69e3ab80 1 --2- 172.21.15.7:0/3923002532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a864cf20 0x55e5a8658d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.926+0000 7f4a69e3ab80 1 --2- 172.21.15.7:0/3923002532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5a8652640 0x55e5a8656030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.926+0000 7f4a40ff9700 0 client.21471 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:32:05.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.926+0000 7f4a69e3ab80 1 -- 172.21.15.7:0/3923002532 >> 172.21.15.7:0/3923002532 conn(0x55e5a8577cc0 msgr2=0x55e5a86475f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:05.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.926+0000 7f4a69e3ab80 1 -- 172.21.15.7:0/3923002532 shutdown_connections 2024-06-10T01:32:05.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.926+0000 7f4a69e3ab80 1 -- 172.21.15.7:0/3923002532 wait complete. 2024-06-10T01:32:05.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.926+0000 7f4a69e3ab80 20 client.21471 trim_cache size 0 max 16384 2024-06-10T01:32:05.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.941+0000 7f4a69e3ab80 1 Processor -- start 2024-06-10T01:32:05.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.941+0000 7f4a69e3ab80 1 -- start start 2024-06-10T01:32:05.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.941+0000 7f4a69e3ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5a864cf20 0x55e5a864c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:05.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.941+0000 7f4a69e3ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a8652640 0x55e5a8653b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:05.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.942+0000 7f4a69e3ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a8652f60 0x55e5a866e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:05.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.942+0000 7f4a69e3ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581b2e80 con 0x55e5a8652640 2024-06-10T01:32:05.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.942+0000 7f4a69e3ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581b0680 con 0x55e5a8652f60 2024-06-10T01:32:05.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.942+0000 7f4a69e3ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581ae4d0 con 0x55e5a864cf20 2024-06-10T01:32:05.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.942+0000 7f4a5cb52700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a8652640 0x55e5a8653b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:05.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.942+0000 7f4a5cb52700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a8652640 0x55e5a8653b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43440/0 (socket says 172.21.15.7:43440) 2024-06-10T01:32:05.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.942+0000 7f4a5cb52700 1 -- 172.21.15.7:0/159141414 learned_addr learned my addr 172.21.15.7:0/159141414 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:32:05.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.942+0000 7f4a5db54700 1 --2- 172.21.15.7:0/159141414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a8652f60 0x55e5a866e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:05.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.942+0000 7f4a5d353700 1 --2- 172.21.15.7:0/159141414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5a864cf20 0x55e5a864c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:05.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.942+0000 7f4a5cb52700 1 -- 172.21.15.7:0/159141414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5a864cf20 msgr2=0x55e5a864c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:32:05.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.942+0000 7f4a5cb52700 1 --2- 172.21.15.7:0/159141414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5a864cf20 0x55e5a864c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.942+0000 7f4a5cb52700 1 -- 172.21.15.7:0/159141414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a8652f60 msgr2=0x55e5a866e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:32:05.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.942+0000 7f4a5cb52700 1 --2- 172.21.15.7:0/159141414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a8652f60 0x55e5a866e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.942+0000 7f4a5cb52700 1 -- 172.21.15.7:0/159141414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5a8655e00 con 0x55e5a8652640 2024-06-10T01:32:05.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.942+0000 7f4a5d353700 1 --2- 172.21.15.7:0/159141414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5a864cf20 0x55e5a864c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:32:05.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.942+0000 7f4a5db54700 1 --2- 172.21.15.7:0/159141414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a8652f60 0x55e5a866e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:32:05.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.942+0000 7f4a5cb52700 1 --2- 172.21.15.7:0/159141414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a8652640 0x55e5a8653b10 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800f010 tx=0x7f4a48010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1145792f8b18a999 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:05.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.942+0000 7f4a42ffd700 1 -- 172.21.15.7:0/159141414 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a480a4750 con 0x55e5a8652640 2024-06-10T01:32:05.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.942+0000 7f4a42ffd700 1 -- 172.21.15.7:0/159141414 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a48099b80 con 0x55e5a8652640 2024-06-10T01:32:05.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.942+0000 7f4a42ffd700 1 -- 172.21.15.7:0/159141414 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a480a42c0 con 0x55e5a8652640 2024-06-10T01:32:05.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.943+0000 7f4a69e3ab80 1 -- 172.21.15.7:0/159141414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a8652640 msgr2=0x55e5a8653b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:05.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.943+0000 7f4a69e3ab80 1 --2- 172.21.15.7:0/159141414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a8652640 0x55e5a8653b10 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800f010 tx=0x7f4a48010ca0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.943+0000 7f4a69e3ab80 1 -- 172.21.15.7:0/159141414 shutdown_connections 2024-06-10T01:32:05.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.943+0000 7f4a69e3ab80 1 --2- 172.21.15.7:0/159141414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a8652f60 0x55e5a866e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.943+0000 7f4a69e3ab80 1 --2- 172.21.15.7:0/159141414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5a864cf20 0x55e5a864c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.943+0000 7f4a69e3ab80 1 --2- 172.21.15.7:0/159141414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a8652640 0x55e5a8653b10 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.943+0000 7f4a69e3ab80 1 -- 172.21.15.7:0/159141414 >> 172.21.15.7:0/159141414 conn(0x55e5a8577cc0 msgr2=0x55e5a8653400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:05.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.943+0000 7f4a69e3ab80 1 -- 172.21.15.7:0/159141414 shutdown_connections 2024-06-10T01:32:05.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.943+0000 7f4a69e3ab80 1 -- 172.21.15.7:0/159141414 wait complete. 2024-06-10T01:32:05.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.943+0000 7f4a69e3ab80 1 -- start start 2024-06-10T01:32:05.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.943+0000 7f4a69e3ab80 1 Processor -- start 2024-06-10T01:32:05.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.944+0000 7f4a69e3ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a864cf20 0x55e5a865eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:05.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.944+0000 7f4a69e3ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a8652640 0x55e5a865f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:05.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.944+0000 7f4a69e3ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5a8652f60 0x55e5a866db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:05.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.944+0000 7f4a69e3ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581aeb40 con 0x55e5a8652640 2024-06-10T01:32:05.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.944+0000 7f4a69e3ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581affb0 con 0x55e5a864cf20 2024-06-10T01:32:05.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.944+0000 7f4a69e3ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581af210 con 0x55e5a8652f60 2024-06-10T01:32:05.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.944+0000 7f4a69e3ab80 10 client.0 _ll_register_callbacks cb 0x55e5a8538920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:32:05.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.944+0000 7f4a5cb52700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a8652640 0x55e5a865f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:05.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.944+0000 7f4a5cb52700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a8652640 0x55e5a865f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43442/0 (socket says 172.21.15.7:43442) 2024-06-10T01:32:05.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.944+0000 7f4a5cb52700 1 -- 172.21.15.7:0/4008087617 learned_addr learned my addr 172.21.15.7:0/4008087617 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:32:05.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.944+0000 7f4a5d353700 1 --2- 172.21.15.7:0/4008087617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a864cf20 0x55e5a865eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:05.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.944+0000 7f4a5db54700 1 --2- 172.21.15.7:0/4008087617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5a8652f60 0x55e5a866db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:05.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.944+0000 7f4a5cb52700 1 -- 172.21.15.7:0/4008087617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5a8652f60 msgr2=0x55e5a866db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:32:05.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.944+0000 7f4a5cb52700 1 --2- 172.21.15.7:0/4008087617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5a8652f60 0x55e5a866db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.944+0000 7f4a5cb52700 1 -- 172.21.15.7:0/4008087617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a864cf20 msgr2=0x55e5a865eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:32:05.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.944+0000 7f4a5cb52700 1 --2- 172.21.15.7:0/4008087617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a864cf20 0x55e5a865eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.944+0000 7f4a5cb52700 1 -- 172.21.15.7:0/4008087617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5a866e180 con 0x55e5a8652640 2024-06-10T01:32:05.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.945+0000 7f4a5cb52700 1 --2- 172.21.15.7:0/4008087617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a8652640 0x55e5a865f530 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f4a48010d50 tx=0x7f4a48099d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c261acc32ba7113e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:05.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.945+0000 7f4a40ff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:32:05.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.945+0000 7f4a40ff9700 1 -- 172.21.15.7:0/4008087617 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a480109a0 con 0x55e5a8652640 2024-06-10T01:32:05.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.945+0000 7f4a40ff9700 1 -- 172.21.15.7:0/4008087617 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f4a480a42c0 con 0x55e5a8652640 2024-06-10T01:32:05.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.945+0000 7f4a69e3ab80 1 -- 172.21.15.7:0/4008087617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a48099b80 con 0x55e5a8652640 2024-06-10T01:32:05.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.945+0000 7f4a40ff9700 1 -- 172.21.15.7:0/4008087617 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f4a480ab050 con 0x55e5a8652640 2024-06-10T01:32:05.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.945+0000 7f4a69e3ab80 10 client.21477 fetch_fsmap learned FSMap version 60 2024-06-10T01:32:05.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.945+0000 7f4a69e3ab80 1 -- 172.21.15.7:0/4008087617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55e5a86d0fa0 con 0x55e5a8652640 2024-06-10T01:32:05.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.945+0000 7f4a40ff9700 1 -- 172.21.15.7:0/4008087617 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a48099b80 con 0x55e5a8652640 2024-06-10T01:32:05.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.945+0000 7f4a40ff9700 1 -- 172.21.15.7:0/4008087617 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 60) ==== 8362+0+0 (secure 0 0 0) 0x7f4a480b4ce0 con 0x55e5a8652640 2024-06-10T01:32:05.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.945+0000 7f4a69e3ab80 10 client.21477 fetch_fsmap finished waiting for FSMap version 60 2024-06-10T01:32:05.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.945+0000 7f4a69e3ab80 10 client.21477 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-10T01:32:05.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.945+0000 7f4a69e3ab80 20 client.21477 populate_metadata read hostname 'smithi007' 2024-06-10T01:32:05.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.945+0000 7f4a69e3ab80 1 --2- 172.21.15.7:0/4008087617 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55e5a86d11c0 0x55e5a86d35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:05.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.945+0000 7f4a69e3ab80 4 client.21477 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:32:05.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.945+0000 7f4a69e3ab80 1 -- 172.21.15.7:0/4008087617 --> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e5a85a6290 con 0x55e5a86d11c0 2024-06-10T01:32:05.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.946+0000 7f4a5d353700 1 --2- 172.21.15.7:0/4008087617 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55e5a86d11c0 0x55e5a86d35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:05.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.946+0000 7f4a40ff9700 10 client.21477 ms_handle_connect on v2:172.21.15.78:6838/1946947484 2024-06-10T01:32:05.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.946+0000 7f4a5d353700 1 --2- 172.21.15.7:0/4008087617 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55e5a86d11c0 0x55e5a86d35b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=21c33549a6b2a9ad server_cookie=dc0961e5e7d270ad in_seq=0 out_seq=0 2024-06-10T01:32:05.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.946+0000 7f4a40ff9700 1 -- 172.21.15.7:0/4008087617 <== mds.2 v2:172.21.15.78:6838/1946947484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e5a85a6290 con 0x55e5a86d11c0 2024-06-10T01:32:05.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.946+0000 7f4a40ff9700 10 client.21477 handle_command_reply: tid=0 2024-06-10T01:32:05.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.946+0000 7f4a69e3ab80 1 client.21477 shutdown 2024-06-10T01:32:05.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.946+0000 7f4a69e3ab80 1 -- 172.21.15.7:0/4008087617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a8652640 msgr2=0x55e5a865f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:05.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.946+0000 7f4a69e3ab80 1 --2- 172.21.15.7:0/4008087617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a8652640 0x55e5a865f530 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f4a48010d50 tx=0x7f4a48099d50 comp rx=0 tx=0).stop 2024-06-10T01:32:05.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.946+0000 7f4a69e3ab80 1 -- 172.21.15.7:0/4008087617 shutdown_connections 2024-06-10T01:32:05.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.946+0000 7f4a69e3ab80 1 --2- 172.21.15.7:0/4008087617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5a8652f60 0x55e5a866db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.946+0000 7f4a69e3ab80 1 --2- 172.21.15.7:0/4008087617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a864cf20 0x55e5a865eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.946+0000 7f4a69e3ab80 1 --2- 172.21.15.7:0/4008087617 >> [v2:172.21.15.78:6838/1946947484,v1:172.21.15.78:6839/1946947484] conn(0x55e5a86d11c0 0x55e5a86d35b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.946+0000 7f4a69e3ab80 1 --2- 172.21.15.7:0/4008087617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a8652640 0x55e5a865f530 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.946+0000 7f4a69e3ab80 1 -- 172.21.15.7:0/4008087617 >> 172.21.15.7:0/4008087617 conn(0x55e5a8577cc0 msgr2=0x55e5a8640e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:05.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.947+0000 7f4a40ff9700 0 client.21477 ms_handle_reset on v2:172.21.15.78:6838/1946947484 2024-06-10T01:32:05.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.947+0000 7f4a69e3ab80 1 -- 172.21.15.7:0/4008087617 shutdown_connections 2024-06-10T01:32:05.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.947+0000 7f4a69e3ab80 1 -- 172.21.15.7:0/4008087617 wait complete. 2024-06-10T01:32:05.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.947+0000 7f4a69e3ab80 20 client.21477 trim_cache size 0 max 16384 2024-06-10T01:32:05.945 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:32:05.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.947+0000 7f4a5f5b7700 1 -- 172.21.15.7:0/1752548859 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4a440a2ca0 msgr2=0x7f4a440a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:05.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.947+0000 7f4a5f5b7700 1 --2- 172.21.15.7:0/1752548859 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4a440a2ca0 0x7f4a440a5160 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400b2e0 tx=0x7f4a54005960 comp rx=0 tx=0).stop 2024-06-10T01:32:05.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.947+0000 7f4a5f5b7700 1 -- 172.21.15.7:0/1752548859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a58101d50 msgr2=0x7f4a581035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:05.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.947+0000 7f4a5f5b7700 1 --2- 172.21.15.7:0/1752548859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a58101d50 0x7f4a581035b0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800e9a0 tx=0x7f4a4800ecb0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.947+0000 7f4a5f5b7700 1 -- 172.21.15.7:0/1752548859 shutdown_connections 2024-06-10T01:32:05.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.947+0000 7f4a5f5b7700 1 --2- 172.21.15.7:0/1752548859 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f4a440a2ca0 0x7f4a440a5160 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.947+0000 7f4a5f5b7700 1 --2- 172.21.15.7:0/1752548859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a58107400 0x7f4a58103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.947+0000 7f4a5f5b7700 1 --2- 172.21.15.7:0/1752548859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a58101390 0x7f4a58103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.947+0000 7f4a5f5b7700 1 --2- 172.21.15.7:0/1752548859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a58101d50 0x7f4a581035b0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:05.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.947+0000 7f4a5f5b7700 1 -- 172.21.15.7:0/1752548859 >> 172.21.15.7:0/1752548859 conn(0x7f4a58078340 msgr2=0x7f4a58100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:05.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.948+0000 7f4a5f5b7700 1 -- 172.21.15.7:0/1752548859 shutdown_connections 2024-06-10T01:32:05.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:05.948+0000 7f4a5f5b7700 1 -- 172.21.15.7:0/1752548859 wait complete. 2024-06-10T01:32:06.781 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:32:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:32:07.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:07 smithi175 ceph-mon[36636]: pgmap v1217: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.5 MiB/s wr, 96 op/s 2024-06-10T01:32:07.491 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:07 smithi007 ceph-mon[32398]: pgmap v1217: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.5 MiB/s wr, 96 op/s 2024-06-10T01:32:07.576 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:07 smithi078 ceph-mon[36683]: pgmap v1217: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.5 MiB/s wr, 96 op/s 2024-06-10T01:32:08.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.700+0000 7f0c05eca700 1 -- 172.21.15.7:0/2724313220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0006ab60 msgr2=0x7f0c0006afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:08.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.700+0000 7f0c05eca700 1 --2- 172.21.15.7:0/2724313220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0006ab60 0x7f0c0006afe0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4009990 tx=0x7f0bf4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.700+0000 7f0c05eca700 1 -- 172.21.15.7:0/2724313220 shutdown_connections 2024-06-10T01:32:08.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.700+0000 7f0c05eca700 1 --2- 172.21.15.7:0/2724313220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c0006b520 0x7f0c0010f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.700+0000 7f0c05eca700 1 --2- 172.21.15.7:0/2724313220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c00108510 0x7f0c001088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.700+0000 7f0c05eca700 1 --2- 172.21.15.7:0/2724313220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0006ab60 0x7f0c0006afe0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.701+0000 7f0c05eca700 1 -- 172.21.15.7:0/2724313220 >> 172.21.15.7:0/2724313220 conn(0x7f0c00078340 msgr2=0x7f0c00078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:08.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.701+0000 7f0c05eca700 1 -- 172.21.15.7:0/2724313220 shutdown_connections 2024-06-10T01:32:08.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.701+0000 7f0c05eca700 1 -- 172.21.15.7:0/2724313220 wait complete. 2024-06-10T01:32:08.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.701+0000 7f0c05eca700 1 Processor -- start 2024-06-10T01:32:08.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.702+0000 7f0c05eca700 1 -- start start 2024-06-10T01:32:08.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.702+0000 7f0c05eca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0006ab60 0x7f0c0019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.702+0000 7f0c05eca700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c0006b520 0x7f0c0019edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.702+0000 7f0c05eca700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c00108510 0x7f0c00198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.702+0000 7f0c05eca700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0c001127f0 con 0x7f0c0006ab60 2024-06-10T01:32:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.702+0000 7f0c05eca700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0c00112670 con 0x7f0c00108510 2024-06-10T01:32:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.702+0000 7f0c05eca700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0c00112970 con 0x7f0c0006b520 2024-06-10T01:32:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.702+0000 7f0bff7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0006ab60 0x7f0c0019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.702+0000 7f0bff7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0006ab60 0x7f0c0019e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43460/0 (socket says 172.21.15.7:43460) 2024-06-10T01:32:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.702+0000 7f0bff7fe700 1 -- 172.21.15.7:0/1674637666 learned_addr learned my addr 172.21.15.7:0/1674637666 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:32:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.702+0000 7f0bfffff700 1 --2- 172.21.15.7:0/1674637666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c00108510 0x7f0c00198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.702+0000 7f0bfeffd700 1 --2- 172.21.15.7:0/1674637666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c0006b520 0x7f0c0019edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.702+0000 7f0bff7fe700 1 -- 172.21.15.7:0/1674637666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c0006b520 msgr2=0x7f0c0019edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.702+0000 7f0bff7fe700 1 --2- 172.21.15.7:0/1674637666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c0006b520 0x7f0c0019edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.702+0000 7f0bff7fe700 1 -- 172.21.15.7:0/1674637666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c00108510 msgr2=0x7f0c00198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:08.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.702+0000 7f0bff7fe700 1 --2- 172.21.15.7:0/1674637666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c00108510 0x7f0c00198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.702+0000 7f0bff7fe700 1 -- 172.21.15.7:0/1674637666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c001991b0 con 0x7f0c0006ab60 2024-06-10T01:32:08.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.703+0000 7f0bff7fe700 1 --2- 172.21.15.7:0/1674637666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0006ab60 0x7f0c0019e890 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000b530 tx=0x7f0bf000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:08.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.703+0000 7f0bfcff9700 1 -- 172.21.15.7:0/1674637666 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0015020 con 0x7f0c0006ab60 2024-06-10T01:32:08.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.703+0000 7f0c05eca700 1 -- 172.21.15.7:0/1674637666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c001994a0 con 0x7f0c0006ab60 2024-06-10T01:32:08.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.703+0000 7f0bfcff9700 1 -- 172.21.15.7:0/1674637666 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bf0011d10 con 0x7f0c0006ab60 2024-06-10T01:32:08.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.703+0000 7f0bfcff9700 1 -- 172.21.15.7:0/1674637666 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf000f2e0 con 0x7f0c0006ab60 2024-06-10T01:32:08.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.703+0000 7f0c05eca700 1 -- 172.21.15.7:0/1674637666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c001a5760 con 0x7f0c0006ab60 2024-06-10T01:32:08.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.705+0000 7f0bfcff9700 1 -- 172.21.15.7:0/1674637666 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0bf0018020 con 0x7f0c0006ab60 2024-06-10T01:32:08.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.706+0000 7f0bfcff9700 1 --2- 172.21.15.7:0/1674637666 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0be80a2c20 0x7f0be80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:08.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.706+0000 7f0bfcff9700 1 -- 172.21.15.7:0/1674637666 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f0bf00ceb20 con 0x7f0c0006ab60 2024-06-10T01:32:08.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.706+0000 7f0bfeffd700 1 --2- 172.21.15.7:0/1674637666 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0be80a2c20 0x7f0be80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:08.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.707+0000 7f0bfeffd700 1 --2- 172.21.15.7:0/1674637666 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0be80a2c20 0x7f0be80a50e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400b5c0 tx=0x7f0bf4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:08.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.707+0000 7f0c1074db80 1 Processor -- start 2024-06-10T01:32:08.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.707+0000 7f0c1074db80 1 -- start start 2024-06-10T01:32:08.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.707+0000 7f0c1074db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123780cf60 0x56123780d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:08.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.707+0000 7f0c1074db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56123780d880 0x561237807260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:08.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.707+0000 7f0c1074db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612378077a0 0x561237807bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:08.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.707+0000 7f0c1074db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5612376f9f60 con 0x5612378077a0 2024-06-10T01:32:08.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.707+0000 7f0c1074db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561237758850 con 0x56123780d880 2024-06-10T01:32:08.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.707+0000 7f0c1074db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5612376c0460 con 0x56123780cf60 2024-06-10T01:32:08.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.707+0000 7f0bfffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612378077a0 0x561237807bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:08.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.707+0000 7f0bfffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612378077a0 0x561237807bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43474/0 (socket says 172.21.15.7:43474) 2024-06-10T01:32:08.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.707+0000 7f0bfffff700 1 -- 172.21.15.7:0/257962249 learned_addr learned my addr 172.21.15.7:0/257962249 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:32:08.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.707+0000 7f0bff7fe700 1 --2- 172.21.15.7:0/257962249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123780cf60 0x56123780d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:08.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.707+0000 7f0bfeffd700 1 --2- 172.21.15.7:0/257962249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56123780d880 0x561237807260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:08.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.707+0000 7f0bfffff700 1 -- 172.21.15.7:0/257962249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123780cf60 msgr2=0x56123780d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:32:08.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.707+0000 7f0bfffff700 1 --2- 172.21.15.7:0/257962249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123780cf60 0x56123780d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.707+0000 7f0bfffff700 1 -- 172.21.15.7:0/257962249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56123780d880 msgr2=0x561237807260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:32:08.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.707+0000 7f0bfffff700 1 --2- 172.21.15.7:0/257962249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56123780d880 0x561237807260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.707+0000 7f0bfffff700 1 -- 172.21.15.7:0/257962249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612376a78d0 con 0x5612378077a0 2024-06-10T01:32:08.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.707+0000 7f0bfeffd700 1 --2- 172.21.15.7:0/257962249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56123780d880 0x561237807260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:32:08.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.708+0000 7f0bfffff700 1 --2- 172.21.15.7:0/257962249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612378077a0 0x561237807bc0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800f5a0 tx=0x7f0bf800f8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e1221cbd890a74 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:08.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.708+0000 7f0be5ffb700 1 -- 172.21.15.7:0/257962249 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8018030 con 0x5612378077a0 2024-06-10T01:32:08.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.708+0000 7f0be5ffb700 1 -- 172.21.15.7:0/257962249 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bf8004bb0 con 0x5612378077a0 2024-06-10T01:32:08.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.708+0000 7f0be5ffb700 1 -- 172.21.15.7:0/257962249 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8005020 con 0x5612378077a0 2024-06-10T01:32:08.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.708+0000 7f0c1074db80 1 -- 172.21.15.7:0/257962249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612378077a0 msgr2=0x561237807bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:08.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.708+0000 7f0c1074db80 1 --2- 172.21.15.7:0/257962249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612378077a0 0x561237807bc0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800f5a0 tx=0x7f0bf800f8b0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.708+0000 7f0c1074db80 1 -- 172.21.15.7:0/257962249 shutdown_connections 2024-06-10T01:32:08.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.708+0000 7f0c1074db80 1 --2- 172.21.15.7:0/257962249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612378077a0 0x561237807bc0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.708+0000 7f0c1074db80 1 --2- 172.21.15.7:0/257962249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123780cf60 0x56123780d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.709+0000 7f0c1074db80 1 --2- 172.21.15.7:0/257962249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56123780d880 0x561237807260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.709+0000 7f0c1074db80 1 -- 172.21.15.7:0/257962249 >> 172.21.15.7:0/257962249 conn(0x561237734080 msgr2=0x561237802220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:08.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.709+0000 7f0c1074db80 1 -- 172.21.15.7:0/257962249 shutdown_connections 2024-06-10T01:32:08.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.709+0000 7f0c1074db80 1 -- 172.21.15.7:0/257962249 wait complete. 2024-06-10T01:32:08.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.709+0000 7f0c1074db80 1 -- start start 2024-06-10T01:32:08.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.709+0000 7f0c1074db80 1 Processor -- start 2024-06-10T01:32:08.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.710+0000 7f0c1074db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612378077a0 0x561237813590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:08.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.710+0000 7f0c1074db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123780cf60 0x561237810890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:08.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.710+0000 7f0c1074db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56123780d880 0x561237810dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:08.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.710+0000 7f0c1074db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5612376f8fc0 con 0x56123780d880 2024-06-10T01:32:08.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.710+0000 7f0c1074db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561237754810 con 0x5612378077a0 2024-06-10T01:32:08.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.710+0000 7f0c1074db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561237814260 con 0x56123780cf60 2024-06-10T01:32:08.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.710+0000 7f0c1074db80 10 client.0 _ll_register_callbacks cb 0x5612376ee5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:32:08.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.710+0000 7f0bfffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56123780d880 0x561237810dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:08.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.710+0000 7f0bfeffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123780cf60 0x561237810890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:08.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.710+0000 7f0bfffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56123780d880 0x561237810dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43478/0 (socket says 172.21.15.7:43478) 2024-06-10T01:32:08.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.710+0000 7f0bfffff700 1 -- 172.21.15.7:0/1484611030 learned_addr learned my addr 172.21.15.7:0/1484611030 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:32:08.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.710+0000 7f0bff7fe700 1 --2- 172.21.15.7:0/1484611030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612378077a0 0x561237813590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:08.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.710+0000 7f0bfffff700 1 -- 172.21.15.7:0/1484611030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123780cf60 msgr2=0x561237810890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:32:08.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.710+0000 7f0bfffff700 1 --2- 172.21.15.7:0/1484611030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123780cf60 0x561237810890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.710+0000 7f0bfffff700 1 -- 172.21.15.7:0/1484611030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612378077a0 msgr2=0x561237813590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:32:08.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.710+0000 7f0bfffff700 1 --2- 172.21.15.7:0/1484611030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612378077a0 0x561237813590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.710+0000 7f0bfffff700 1 -- 172.21.15.7:0/1484611030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56123771e750 con 0x56123780d880 2024-06-10T01:32:08.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.710+0000 7f0bfffff700 1 --2- 172.21.15.7:0/1484611030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56123780d880 0x561237810dd0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800fc00 tx=0x7f0bf8014720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d23323b96e13a839 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:08.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.710+0000 7f0bdeffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:32:08.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.710+0000 7f0bdeffd700 1 -- 172.21.15.7:0/1484611030 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8004450 con 0x56123780d880 2024-06-10T01:32:08.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.710+0000 7f0bdeffd700 1 -- 172.21.15.7:0/1484611030 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f0bf80045f0 con 0x56123780d880 2024-06-10T01:32:08.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.710+0000 7f0c1074db80 1 -- 172.21.15.7:0/1484611030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bf8004bb0 con 0x56123780d880 2024-06-10T01:32:08.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.711+0000 7f0bdeffd700 1 -- 172.21.15.7:0/1484611030 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f0bf802e050 con 0x56123780d880 2024-06-10T01:32:08.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.711+0000 7f0c1074db80 10 client.21489 fetch_fsmap learned FSMap version 60 2024-06-10T01:32:08.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.711+0000 7f0c1074db80 1 -- 172.21.15.7:0/1484611030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x56123781e980 con 0x56123780d880 2024-06-10T01:32:08.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.711+0000 7f0bdeffd700 1 -- 172.21.15.7:0/1484611030 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bf8004bb0 con 0x56123780d880 2024-06-10T01:32:08.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.711+0000 7f0bdeffd700 1 -- 172.21.15.7:0/1484611030 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 60) ==== 8362+0+0 (secure 0 0 0) 0x7f0bf801d9d0 con 0x56123780d880 2024-06-10T01:32:08.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.711+0000 7f0c1074db80 10 client.21489 fetch_fsmap finished waiting for FSMap version 60 2024-06-10T01:32:08.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.711+0000 7f0c1074db80 10 client.21489 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:32:08.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.711+0000 7f0c1074db80 20 client.21489 populate_metadata read hostname 'smithi007' 2024-06-10T01:32:08.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.711+0000 7f0c1074db80 1 --2- 172.21.15.7:0/1484611030 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x561237820f00 0x5612378232f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:08.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.711+0000 7f0c1074db80 4 client.21489 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:32:08.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.711+0000 7f0c1074db80 1 -- 172.21.15.7:0/1484611030 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561237765d50 con 0x561237820f00 2024-06-10T01:32:08.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.711+0000 7f0bff7fe700 1 --2- 172.21.15.7:0/1484611030 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x561237820f00 0x5612378232f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:08.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.712+0000 7f0bff7fe700 1 --2- 172.21.15.7:0/1484611030 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x561237820f00 0x5612378232f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9b382eee1e440734 server_cookie=c9bfa246e1729cf5 in_seq=0 out_seq=0 2024-06-10T01:32:08.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.712+0000 7f0bdeffd700 10 client.21489 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:32:08.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.713+0000 7f0bdeffd700 1 -- 172.21.15.7:0/1484611030 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561237765d50 con 0x561237820f00 2024-06-10T01:32:08.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.713+0000 7f0bdeffd700 10 client.21489 handle_command_reply: tid=0 2024-06-10T01:32:08.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.713+0000 7f0c1074db80 1 client.21489 shutdown 2024-06-10T01:32:08.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.713+0000 7f0c1074db80 1 -- 172.21.15.7:0/1484611030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56123780d880 msgr2=0x561237810dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:08.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.713+0000 7f0c1074db80 1 --2- 172.21.15.7:0/1484611030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56123780d880 0x561237810dd0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800fc00 tx=0x7f0bf8014720 comp rx=0 tx=0).stop 2024-06-10T01:32:08.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.713+0000 7f0c1074db80 1 -- 172.21.15.7:0/1484611030 shutdown_connections 2024-06-10T01:32:08.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.713+0000 7f0c1074db80 1 --2- 172.21.15.7:0/1484611030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56123780d880 0x561237810dd0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.713+0000 7f0c1074db80 1 --2- 172.21.15.7:0/1484611030 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x561237820f00 0x5612378232f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.713+0000 7f0c1074db80 1 --2- 172.21.15.7:0/1484611030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612378077a0 0x561237813590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.713+0000 7f0c1074db80 1 --2- 172.21.15.7:0/1484611030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123780cf60 0x561237810890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.713+0000 7f0c1074db80 1 -- 172.21.15.7:0/1484611030 >> 172.21.15.7:0/1484611030 conn(0x561237734080 msgr2=0x561237801700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:08.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.713+0000 7f0bdeffd700 0 client.21489 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:32:08.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.713+0000 7f0c1074db80 1 -- 172.21.15.7:0/1484611030 shutdown_connections 2024-06-10T01:32:08.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.713+0000 7f0c1074db80 1 -- 172.21.15.7:0/1484611030 wait complete. 2024-06-10T01:32:08.880 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:32:08.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.713+0000 7f0c1074db80 20 client.21489 trim_cache size 0 max 16384 2024-06-10T01:32:08.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.728+0000 7f0c1074db80 1 Processor -- start 2024-06-10T01:32:08.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.728+0000 7f0c1074db80 1 -- start start 2024-06-10T01:32:08.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.728+0000 7f0c1074db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612378077a0 0x561237807260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:08.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.728+0000 7f0c1074db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123780cf60 0x561237801e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:08.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.728+0000 7f0c1074db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56123780d880 0x5612378023c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:08.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.728+0000 7f0c1074db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0c001b0ce0 con 0x5612378077a0 2024-06-10T01:32:08.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.728+0000 7f0c1074db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0c001ae4e0 con 0x56123780d880 2024-06-10T01:32:08.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.728+0000 7f0c1074db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0c001ac330 con 0x56123780cf60 2024-06-10T01:32:08.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.728+0000 7f0bff7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612378077a0 0x561237807260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:08.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.728+0000 7f0bff7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612378077a0 0x561237807260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43488/0 (socket says 172.21.15.7:43488) 2024-06-10T01:32:08.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.728+0000 7f0bff7fe700 1 -- 172.21.15.7:0/169598880 learned_addr learned my addr 172.21.15.7:0/169598880 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:32:08.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.729+0000 7f0bff7fe700 1 -- 172.21.15.7:0/169598880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123780cf60 msgr2=0x561237801e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:32:08.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.729+0000 7f0bff7fe700 1 --2- 172.21.15.7:0/169598880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123780cf60 0x561237801e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.729+0000 7f0bff7fe700 1 -- 172.21.15.7:0/169598880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56123780d880 msgr2=0x5612378023c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:32:08.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.729+0000 7f0bff7fe700 1 --2- 172.21.15.7:0/169598880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56123780d880 0x5612378023c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.729+0000 7f0bff7fe700 1 -- 172.21.15.7:0/169598880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56123780ee00 con 0x5612378077a0 2024-06-10T01:32:08.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.729+0000 7f0bff7fe700 1 --2- 172.21.15.7:0/169598880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612378077a0 0x561237807260 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0097400 tx=0x7f0bf001c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f875b4340d5028a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:08.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.729+0000 7f0be57fa700 1 -- 172.21.15.7:0/169598880 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf00a3e60 con 0x5612378077a0 2024-06-10T01:32:08.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.729+0000 7f0be57fa700 1 -- 172.21.15.7:0/169598880 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bf009f350 con 0x5612378077a0 2024-06-10T01:32:08.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.729+0000 7f0be57fa700 1 -- 172.21.15.7:0/169598880 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf00b9be0 con 0x5612378077a0 2024-06-10T01:32:08.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.730+0000 7f0c1074db80 1 -- 172.21.15.7:0/169598880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612378077a0 msgr2=0x561237807260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:08.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.730+0000 7f0c1074db80 1 --2- 172.21.15.7:0/169598880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612378077a0 0x561237807260 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0097400 tx=0x7f0bf001c040 comp rx=0 tx=0).stop 2024-06-10T01:32:08.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.730+0000 7f0c1074db80 1 -- 172.21.15.7:0/169598880 shutdown_connections 2024-06-10T01:32:08.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.730+0000 7f0c1074db80 1 --2- 172.21.15.7:0/169598880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56123780d880 0x5612378023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.730+0000 7f0c1074db80 1 --2- 172.21.15.7:0/169598880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612378077a0 0x561237807260 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.730+0000 7f0c1074db80 1 --2- 172.21.15.7:0/169598880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56123780cf60 0x561237801e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.730+0000 7f0c1074db80 1 -- 172.21.15.7:0/169598880 >> 172.21.15.7:0/169598880 conn(0x561237734080 msgr2=0x56123780f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:08.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.730+0000 7f0c1074db80 1 -- 172.21.15.7:0/169598880 shutdown_connections 2024-06-10T01:32:08.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.730+0000 7f0c1074db80 1 -- 172.21.15.7:0/169598880 wait complete. 2024-06-10T01:32:08.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.730+0000 7f0c1074db80 1 -- start start 2024-06-10T01:32:08.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.730+0000 7f0c1074db80 1 Processor -- start 2024-06-10T01:32:08.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.730+0000 7f0c1074db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612378077a0 0x561237819700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:08.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.730+0000 7f0c1074db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56123780cf60 0x561237819c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:08.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.730+0000 7f0c1074db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56123780d880 0x561237827d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:08.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.730+0000 7f0c1074db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0c001ac9a0 con 0x56123780cf60 2024-06-10T01:32:08.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.730+0000 7f0c1074db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0c001ade10 con 0x56123780d880 2024-06-10T01:32:08.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.730+0000 7f0c1074db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0c001ad070 con 0x5612378077a0 2024-06-10T01:32:08.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.730+0000 7f0c1074db80 10 client.0 _ll_register_callbacks cb 0x5612376ee5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:32:08.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.731+0000 7f0bfeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56123780cf60 0x561237819c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:08.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.731+0000 7f0bfeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56123780cf60 0x561237819c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43494/0 (socket says 172.21.15.7:43494) 2024-06-10T01:32:08.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.731+0000 7f0bfeffd700 1 -- 172.21.15.7:0/165527022 learned_addr learned my addr 172.21.15.7:0/165527022 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:32:08.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.731+0000 7f0bff7fe700 1 --2- 172.21.15.7:0/165527022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612378077a0 0x561237819700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:08.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.731+0000 7f0bfffff700 1 --2- 172.21.15.7:0/165527022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56123780d880 0x561237827d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:08.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.731+0000 7f0bfeffd700 1 -- 172.21.15.7:0/165527022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612378077a0 msgr2=0x561237819700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:32:08.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.731+0000 7f0bfeffd700 1 --2- 172.21.15.7:0/165527022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612378077a0 0x561237819700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.731+0000 7f0bfeffd700 1 -- 172.21.15.7:0/165527022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56123780d880 msgr2=0x561237827d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:32:08.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.731+0000 7f0bfeffd700 1 --2- 172.21.15.7:0/165527022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56123780d880 0x561237827d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.731+0000 7f0bfeffd700 1 -- 172.21.15.7:0/165527022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612378284b0 con 0x56123780cf60 2024-06-10T01:32:08.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.731+0000 7f0bfffff700 1 --2- 172.21.15.7:0/165527022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56123780d880 0x561237827d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:32:08.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.731+0000 7f0bfeffd700 1 --2- 172.21.15.7:0/165527022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56123780cf60 0x561237819c40 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4009990 tx=0x7f0bf4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee92e4322170d18 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:08.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.731+0000 7f0bdeffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:32:08.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.731+0000 7f0bdeffd700 1 -- 172.21.15.7:0/165527022 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4037060 con 0x56123780cf60 2024-06-10T01:32:08.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.731+0000 7f0bdeffd700 1 -- 172.21.15.7:0/165527022 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f0bf400f660 con 0x56123780cf60 2024-06-10T01:32:08.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.731+0000 7f0c1074db80 1 -- 172.21.15.7:0/165527022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bf009f350 con 0x56123780cf60 2024-06-10T01:32:08.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.731+0000 7f0bdeffd700 1 -- 172.21.15.7:0/165527022 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f0bf403e050 con 0x56123780cf60 2024-06-10T01:32:08.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.731+0000 7f0c1074db80 10 client.21495 fetch_fsmap learned FSMap version 60 2024-06-10T01:32:08.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.731+0000 7f0c1074db80 1 -- 172.21.15.7:0/165527022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x56123788b3c0 con 0x56123780cf60 2024-06-10T01:32:08.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.732+0000 7f0bdeffd700 1 -- 172.21.15.7:0/165527022 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bf009f350 con 0x56123780cf60 2024-06-10T01:32:08.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.732+0000 7f0bdeffd700 1 -- 172.21.15.7:0/165527022 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 60) ==== 8362+0+0 (secure 0 0 0) 0x7f0bf40178b0 con 0x56123780cf60 2024-06-10T01:32:08.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.732+0000 7f0c1074db80 10 client.21495 fetch_fsmap finished waiting for FSMap version 60 2024-06-10T01:32:08.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.732+0000 7f0c1074db80 10 client.21495 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-10T01:32:08.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.732+0000 7f0c1074db80 20 client.21495 populate_metadata read hostname 'smithi007' 2024-06-10T01:32:08.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.732+0000 7f0c1074db80 1 --2- 172.21.15.7:0/165527022 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x56123788bb60 0x56123788df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:08.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.732+0000 7f0c1074db80 4 client.21495 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:32:08.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.732+0000 7f0c1074db80 1 -- 172.21.15.7:0/165527022 --> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5612377602d0 con 0x56123788bb60 2024-06-10T01:32:08.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.732+0000 7f0bff7fe700 1 --2- 172.21.15.7:0/165527022 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x56123788bb60 0x56123788df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:08.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.732+0000 7f0bff7fe700 1 --2- 172.21.15.7:0/165527022 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x56123788bb60 0x56123788df80 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a67fee2117d87084 server_cookie=a140dc8acd9d5221 in_seq=0 out_seq=0 2024-06-10T01:32:08.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.732+0000 7f0bdeffd700 10 client.21495 ms_handle_connect on v2:172.21.15.175:6836/779740919 2024-06-10T01:32:08.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.733+0000 7f0bdeffd700 1 -- 172.21.15.7:0/165527022 <== mds.1 v2:172.21.15.175:6836/779740919 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5612377602d0 con 0x56123788bb60 2024-06-10T01:32:08.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.733+0000 7f0bdeffd700 10 client.21495 handle_command_reply: tid=0 2024-06-10T01:32:08.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.733+0000 7f0c1074db80 1 client.21495 shutdown 2024-06-10T01:32:08.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.733+0000 7f0c1074db80 1 -- 172.21.15.7:0/165527022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56123780cf60 msgr2=0x561237819c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:08.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.733+0000 7f0c1074db80 1 --2- 172.21.15.7:0/165527022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56123780cf60 0x561237819c40 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4009990 tx=0x7f0bf4009310 comp rx=0 tx=0).stop 2024-06-10T01:32:08.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.733+0000 7f0c1074db80 1 -- 172.21.15.7:0/165527022 shutdown_connections 2024-06-10T01:32:08.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.733+0000 7f0c1074db80 1 --2- 172.21.15.7:0/165527022 >> [v2:172.21.15.175:6836/779740919,v1:172.21.15.175:6837/779740919] conn(0x56123788bb60 0x56123788df80 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.733+0000 7f0c1074db80 1 --2- 172.21.15.7:0/165527022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56123780d880 0x561237827d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.733+0000 7f0c1074db80 1 --2- 172.21.15.7:0/165527022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5612378077a0 0x561237819700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.733+0000 7f0bdeffd700 0 client.21495 ms_handle_reset on v2:172.21.15.175:6836/779740919 2024-06-10T01:32:08.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.733+0000 7f0c1074db80 1 --2- 172.21.15.7:0/165527022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56123780cf60 0x561237819c40 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.733+0000 7f0c1074db80 1 -- 172.21.15.7:0/165527022 >> 172.21.15.7:0/165527022 conn(0x561237734080 msgr2=0x56123780dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:08.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.733+0000 7f0c1074db80 1 -- 172.21.15.7:0/165527022 shutdown_connections 2024-06-10T01:32:08.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.733+0000 7f0c1074db80 1 -- 172.21.15.7:0/165527022 wait complete. 2024-06-10T01:32:08.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.733+0000 7f0c1074db80 20 client.21495 trim_cache size 0 max 16384 2024-06-10T01:32:08.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.733+0000 7f0c05eca700 1 -- 172.21.15.7:0/1674637666 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0be80a2c20 msgr2=0x7f0be80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:08.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.733+0000 7f0c05eca700 1 --2- 172.21.15.7:0/1674637666 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0be80a2c20 0x7f0be80a50e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400b5c0 tx=0x7f0bf4005960 comp rx=0 tx=0).stop 2024-06-10T01:32:08.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.734+0000 7f0c05eca700 1 -- 172.21.15.7:0/1674637666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0006ab60 msgr2=0x7f0c0019e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:08.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.734+0000 7f0c05eca700 1 --2- 172.21.15.7:0/1674637666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0006ab60 0x7f0c0019e890 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000b530 tx=0x7f0bf000b840 comp rx=0 tx=0).stop 2024-06-10T01:32:08.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.734+0000 7f0c05eca700 1 -- 172.21.15.7:0/1674637666 shutdown_connections 2024-06-10T01:32:08.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.734+0000 7f0c05eca700 1 --2- 172.21.15.7:0/1674637666 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f0be80a2c20 0x7f0be80a50e0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.734+0000 7f0c05eca700 1 --2- 172.21.15.7:0/1674637666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c00108510 0x7f0c00198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.734+0000 7f0c05eca700 1 --2- 172.21.15.7:0/1674637666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0006ab60 0x7f0c0019e890 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.734+0000 7f0c05eca700 1 --2- 172.21.15.7:0/1674637666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c0006b520 0x7f0c0019edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:08.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.734+0000 7f0c05eca700 1 -- 172.21.15.7:0/1674637666 >> 172.21.15.7:0/1674637666 conn(0x7f0c00078340 msgr2=0x7f0c001003f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:08.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.734+0000 7f0c05eca700 1 -- 172.21.15.7:0/1674637666 shutdown_connections 2024-06-10T01:32:08.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:08.734+0000 7f0c05eca700 1 -- 172.21.15.7:0/1674637666 wait complete. 2024-06-10T01:32:09.260 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:09 smithi007 ceph-mon[32398]: pgmap v1218: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 970 KiB/s wr, 58 op/s 2024-06-10T01:32:09.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:09 smithi175 ceph-mon[36636]: pgmap v1218: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 970 KiB/s wr, 58 op/s 2024-06-10T01:32:09.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:09 smithi078 ceph-mon[36683]: pgmap v1218: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 970 KiB/s wr, 58 op/s 2024-06-10T01:32:09.714 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:32:09.968 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:32:11.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:11 smithi175 ceph-mon[36636]: pgmap v1219: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 167 KiB/s wr, 31 op/s 2024-06-10T01:32:11.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:11 smithi007 ceph-mon[32398]: pgmap v1219: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 167 KiB/s wr, 31 op/s 2024-06-10T01:32:11.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:11 smithi078 ceph-mon[36683]: pgmap v1219: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 167 KiB/s wr, 31 op/s 2024-06-10T01:32:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.760+0000 7fd70df5b700 1 -- 172.21.15.7:0/1014999050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd708101610 msgr2=0x7fd7081019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.760+0000 7fd70df5b700 1 --2- 172.21.15.7:0/1014999050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd708101610 0x7fd7081019f0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80099c0 tx=0x7fd6f8009cd0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.760+0000 7fd70df5b700 1 -- 172.21.15.7:0/1014999050 shutdown_connections 2024-06-10T01:32:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.760+0000 7fd70df5b700 1 --2- 172.21.15.7:0/1014999050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7081066f0 0x7fd70810e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.760+0000 7fd70df5b700 1 --2- 172.21.15.7:0/1014999050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd708101610 0x7fd7081019f0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.760+0000 7fd70df5b700 1 --2- 172.21.15.7:0/1014999050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd708101fc0 0x7fd708106010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.760+0000 7fd70df5b700 1 -- 172.21.15.7:0/1014999050 >> 172.21.15.7:0/1014999050 conn(0x7fd708078430 msgr2=0x7fd708078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.760+0000 7fd70df5b700 1 -- 172.21.15.7:0/1014999050 shutdown_connections 2024-06-10T01:32:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.760+0000 7fd70df5b700 1 -- 172.21.15.7:0/1014999050 wait complete. 2024-06-10T01:32:11.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.761+0000 7fd70df5b700 1 Processor -- start 2024-06-10T01:32:11.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.761+0000 7fd70df5b700 1 -- start start 2024-06-10T01:32:11.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.762+0000 7fd70df5b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd708101610 0x7fd7081030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:11.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.762+0000 7fd70df5b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd708101fc0 0x7fd7081035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:11.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.762+0000 7fd70df5b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7081066f0 0x7fd708103b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:11.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.762+0000 7fd70df5b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd7081116d0 con 0x7fd708101610 2024-06-10T01:32:11.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.762+0000 7fd70df5b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd708111550 con 0x7fd7081066f0 2024-06-10T01:32:11.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.762+0000 7fd70df5b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd708111850 con 0x7fd708101fc0 2024-06-10T01:32:11.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.762+0000 7fd7077fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd708101610 0x7fd7081030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:11.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.762+0000 7fd7077fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd708101610 0x7fd7081030b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43504/0 (socket says 172.21.15.7:43504) 2024-06-10T01:32:11.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.762+0000 7fd7077fe700 1 -- 172.21.15.7:0/3632880912 learned_addr learned my addr 172.21.15.7:0/3632880912 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:32:11.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.762+0000 7fd706ffd700 1 --2- 172.21.15.7:0/3632880912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd708101fc0 0x7fd7081035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:11.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.762+0000 7fd707fff700 1 --2- 172.21.15.7:0/3632880912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7081066f0 0x7fd708103b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:11.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.762+0000 7fd7077fe700 1 -- 172.21.15.7:0/3632880912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd708101fc0 msgr2=0x7fd7081035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:11.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.762+0000 7fd7077fe700 1 --2- 172.21.15.7:0/3632880912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd708101fc0 0x7fd7081035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.762+0000 7fd7077fe700 1 -- 172.21.15.7:0/3632880912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7081066f0 msgr2=0x7fd708103b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:11.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.762+0000 7fd7077fe700 1 --2- 172.21.15.7:0/3632880912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7081066f0 0x7fd708103b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.762+0000 7fd7077fe700 1 -- 172.21.15.7:0/3632880912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7081a71b0 con 0x7fd708101610 2024-06-10T01:32:11.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.762+0000 7fd7077fe700 1 --2- 172.21.15.7:0/3632880912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd708101610 0x7fd7081030b0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80059d0 tx=0x7fd6f80038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:11.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.762+0000 7fd704ff9700 1 -- 172.21.15.7:0/3632880912 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8027070 con 0x7fd708101610 2024-06-10T01:32:11.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.763+0000 7fd70df5b700 1 -- 172.21.15.7:0/3632880912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7081a7440 con 0x7fd708101610 2024-06-10T01:32:11.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.763+0000 7fd704ff9700 1 -- 172.21.15.7:0/3632880912 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6f8004050 con 0x7fd708101610 2024-06-10T01:32:11.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.763+0000 7fd704ff9700 1 -- 172.21.15.7:0/3632880912 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8022db0 con 0x7fd708101610 2024-06-10T01:32:11.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.763+0000 7fd70df5b700 1 -- 172.21.15.7:0/3632880912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7081a7890 con 0x7fd708101610 2024-06-10T01:32:11.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.766+0000 7fd704ff9700 1 -- 172.21.15.7:0/3632880912 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fd6f802a070 con 0x7fd708101610 2024-06-10T01:32:11.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.766+0000 7fd704ff9700 1 --2- 172.21.15.7:0/3632880912 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd6f00a2c20 0x7fd6f00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:11.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.766+0000 7fd704ff9700 1 -- 172.21.15.7:0/3632880912 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fd6f80d0140 con 0x7fd708101610 2024-06-10T01:32:11.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.766+0000 7fd706ffd700 1 --2- 172.21.15.7:0/3632880912 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd6f00a2c20 0x7fd6f00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:11.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.767+0000 7fd706ffd700 1 --2- 172.21.15.7:0/3632880912 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd6f00a2c20 0x7fd6f00a50e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd708101200 tx=0x7fd6fc00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:11.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.767+0000 7fd7187deb80 1 Processor -- start 2024-06-10T01:32:11.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.767+0000 7fd7187deb80 1 -- start start 2024-06-10T01:32:11.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.767+0000 7fd7187deb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc1dacc640 0x55dc1dacca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:11.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.767+0000 7fd7187deb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc1daccf60 0x55dc1dac69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:11.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.767+0000 7fd7187deb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc1dac6f20 0x55dc1dac7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:11.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.767+0000 7fd7187deb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55dc1da18810 con 0x55dc1dac6f20 2024-06-10T01:32:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.767+0000 7fd7187deb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55dc1d9c0720 con 0x55dc1dacc640 2024-06-10T01:32:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.767+0000 7fd7187deb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dc1da147d0 con 0x55dc1daccf60 2024-06-10T01:32:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.767+0000 7fd707fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc1dac6f20 0x55dc1dac7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.767+0000 7fd707fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc1dac6f20 0x55dc1dac7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43508/0 (socket says 172.21.15.7:43508) 2024-06-10T01:32:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.767+0000 7fd707fff700 1 -- 172.21.15.7:0/3625027623 learned_addr learned my addr 172.21.15.7:0/3625027623 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:32:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.768+0000 7fd7077fe700 1 --2- 172.21.15.7:0/3625027623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc1dacc640 0x55dc1dacca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.768+0000 7fd707fff700 1 -- 172.21.15.7:0/3625027623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc1daccf60 msgr2=0x55dc1dac69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:32:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.768+0000 7fd706ffd700 1 --2- 172.21.15.7:0/3625027623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc1daccf60 0x55dc1dac69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.768+0000 7fd707fff700 1 --2- 172.21.15.7:0/3625027623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc1daccf60 0x55dc1dac69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.768+0000 7fd707fff700 1 -- 172.21.15.7:0/3625027623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc1dacc640 msgr2=0x55dc1dacca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:32:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.768+0000 7fd707fff700 1 --2- 172.21.15.7:0/3625027623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc1dacc640 0x55dc1dacca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.768+0000 7fd707fff700 1 -- 172.21.15.7:0/3625027623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc1d9b6f50 con 0x55dc1dac6f20 2024-06-10T01:32:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.768+0000 7fd706ffd700 1 --2- 172.21.15.7:0/3625027623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc1daccf60 0x55dc1dac69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:32:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.768+0000 7fd7077fe700 1 --2- 172.21.15.7:0/3625027623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc1dacc640 0x55dc1dacca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:32:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.768+0000 7fd707fff700 1 --2- 172.21.15.7:0/3625027623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc1dac6f20 0x55dc1dac7340 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fd700009d20 tx=0x7fd7000077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=909f43c7cdbb4fb4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.768+0000 7fd6edffb700 1 -- 172.21.15.7:0/3625027623 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd700004bb0 con 0x55dc1dac6f20 2024-06-10T01:32:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.768+0000 7fd6edffb700 1 -- 172.21.15.7:0/3625027623 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd700004d50 con 0x55dc1dac6f20 2024-06-10T01:32:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.768+0000 7fd6edffb700 1 -- 172.21.15.7:0/3625027623 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd700014650 con 0x55dc1dac6f20 2024-06-10T01:32:11.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.769+0000 7fd7187deb80 1 -- 172.21.15.7:0/3625027623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc1dac6f20 msgr2=0x55dc1dac7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:11.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.769+0000 7fd7187deb80 1 --2- 172.21.15.7:0/3625027623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc1dac6f20 0x55dc1dac7340 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fd700009d20 tx=0x7fd7000077e0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.769+0000 7fd7187deb80 1 -- 172.21.15.7:0/3625027623 shutdown_connections 2024-06-10T01:32:11.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.769+0000 7fd7187deb80 1 --2- 172.21.15.7:0/3625027623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc1dac6f20 0x55dc1dac7340 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.769+0000 7fd7187deb80 1 --2- 172.21.15.7:0/3625027623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc1dacc640 0x55dc1dacca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.769+0000 7fd7187deb80 1 --2- 172.21.15.7:0/3625027623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc1daccf60 0x55dc1dac69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.769+0000 7fd7187deb80 1 -- 172.21.15.7:0/3625027623 >> 172.21.15.7:0/3625027623 conn(0x55dc1d9f1cc0 msgr2=0x55dc1dac1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:11.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.769+0000 7fd7187deb80 1 -- 172.21.15.7:0/3625027623 shutdown_connections 2024-06-10T01:32:11.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.769+0000 7fd7187deb80 1 -- 172.21.15.7:0/3625027623 wait complete. 2024-06-10T01:32:11.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.769+0000 7fd7187deb80 1 -- start start 2024-06-10T01:32:11.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.770+0000 7fd7187deb80 1 Processor -- start 2024-06-10T01:32:11.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.770+0000 7fd7187deb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc1dac6f20 0x55dc1dad2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:11.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.770+0000 7fd7187deb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc1dacc640 0x55dc1dacffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:11.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.770+0000 7fd7187deb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc1daccf60 0x55dc1dad04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:11.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.770+0000 7fd7187deb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55dc1dad39d0 con 0x55dc1daccf60 2024-06-10T01:32:11.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.770+0000 7fd7187deb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55dc1d9b8bf0 con 0x55dc1dacc640 2024-06-10T01:32:11.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.770+0000 7fd7187deb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dc1dad3b50 con 0x55dc1dac6f20 2024-06-10T01:32:11.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.770+0000 7fd7187deb80 10 client.0 _ll_register_callbacks cb 0x55dc1d9b2920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:32:11.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.770+0000 7fd707fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc1daccf60 0x55dc1dad04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:11.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.770+0000 7fd707fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc1daccf60 0x55dc1dad04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43522/0 (socket says 172.21.15.7:43522) 2024-06-10T01:32:11.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.770+0000 7fd707fff700 1 -- 172.21.15.7:0/1141601881 learned_addr learned my addr 172.21.15.7:0/1141601881 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:32:11.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.770+0000 7fd706ffd700 1 --2- 172.21.15.7:0/1141601881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc1dacc640 0x55dc1dacffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:11.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.770+0000 7fd7077fe700 1 --2- 172.21.15.7:0/1141601881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc1dac6f20 0x55dc1dad2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:11.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.770+0000 7fd706ffd700 1 -- 172.21.15.7:0/1141601881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc1dac6f20 msgr2=0x55dc1dad2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:32:11.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.770+0000 7fd706ffd700 1 --2- 172.21.15.7:0/1141601881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc1dac6f20 0x55dc1dad2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.770+0000 7fd706ffd700 1 -- 172.21.15.7:0/1141601881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc1daccf60 msgr2=0x55dc1dad04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:32:11.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.770+0000 7fd706ffd700 1 --2- 172.21.15.7:0/1141601881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc1daccf60 0x55dc1dad04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.770+0000 7fd706ffd700 1 -- 172.21.15.7:0/1141601881 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc1d9678d0 con 0x55dc1dacc640 2024-06-10T01:32:11.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.770+0000 7fd7077fe700 1 --2- 172.21.15.7:0/1141601881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc1dac6f20 0x55dc1dad2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:32:11.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.771+0000 7fd707fff700 1 --2- 172.21.15.7:0/1141601881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc1daccf60 0x55dc1dad04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:32:11.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.771+0000 7fd706ffd700 1 --2- 172.21.15.7:0/1141601881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc1dacc640 0x55dc1dacffa0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd708104b40 tx=0x7fd6fc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c804a7b1332b69d2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:11.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.771+0000 7fd6e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-06-10T01:32:11.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.771+0000 7fd6e6ffd700 1 -- 172.21.15.7:0/1141601881 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc016070 con 0x55dc1dacc640 2024-06-10T01:32:11.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.771+0000 7fd7187deb80 1 -- 172.21.15.7:0/1141601881 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd700004d50 con 0x55dc1dacc640 2024-06-10T01:32:11.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.771+0000 7fd6e6ffd700 1 -- 172.21.15.7:0/1141601881 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fd6fc01b820 con 0x55dc1dacc640 2024-06-10T01:32:11.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.771+0000 7fd6e6ffd700 1 -- 172.21.15.7:0/1141601881 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fd6fc010e10 con 0x55dc1dacc640 2024-06-10T01:32:11.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.771+0000 7fd7187deb80 10 client.30671 fetch_fsmap learned FSMap version 60 2024-06-10T01:32:11.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.771+0000 7fd7187deb80 1 -- 172.21.15.7:0/1141601881 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55dc1dade270 con 0x55dc1dacc640 2024-06-10T01:32:11.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.771+0000 7fd6e6ffd700 1 -- 172.21.15.7:0/1141601881 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd700004d50 con 0x55dc1dacc640 2024-06-10T01:32:11.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.772+0000 7fd6e6ffd700 1 -- 172.21.15.7:0/1141601881 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 60) ==== 8362+0+0 (secure 0 0 0) 0x7fd6fc023070 con 0x55dc1dacc640 2024-06-10T01:32:11.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.772+0000 7fd7187deb80 10 client.30671 fetch_fsmap finished waiting for FSMap version 60 2024-06-10T01:32:11.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.772+0000 7fd7187deb80 10 client.30671 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:32:11.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.772+0000 7fd7187deb80 20 client.30671 populate_metadata read hostname 'smithi007' 2024-06-10T01:32:11.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.772+0000 7fd7187deb80 1 --2- 172.21.15.7:0/1141601881 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55dc1dae0900 0x55dc1dae2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:11.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.772+0000 7fd7187deb80 4 client.30671 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:32:11.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.772+0000 7fd7187deb80 1 -- 172.21.15.7:0/1141601881 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc1da25d10 con 0x55dc1dae0900 2024-06-10T01:32:11.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.772+0000 7fd7077fe700 1 --2- 172.21.15.7:0/1141601881 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55dc1dae0900 0x55dc1dae2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:11.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.773+0000 7fd6e6ffd700 10 client.30671 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:32:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.773+0000 7fd7077fe700 1 --2- 172.21.15.7:0/1141601881 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55dc1dae0900 0x55dc1dae2cf0 crc :-1 s=READY pgs=572 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a1ffea0ac8cbc76 server_cookie=9888085bb39be699 in_seq=0 out_seq=0 2024-06-10T01:32:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.773+0000 7fd6e6ffd700 1 -- 172.21.15.7:0/1141601881 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55dc1da25d10 con 0x55dc1dae0900 2024-06-10T01:32:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.773+0000 7fd6e6ffd700 10 client.30671 handle_command_reply: tid=0 2024-06-10T01:32:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.773+0000 7fd7187deb80 1 client.30671 shutdown 2024-06-10T01:32:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.774+0000 7fd7187deb80 1 -- 172.21.15.7:0/1141601881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc1dacc640 msgr2=0x55dc1dacffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.774+0000 7fd7187deb80 1 --2- 172.21.15.7:0/1141601881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc1dacc640 0x55dc1dacffa0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd708104b40 tx=0x7fd6fc0047c0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.774+0000 7fd7187deb80 1 -- 172.21.15.7:0/1141601881 shutdown_connections 2024-06-10T01:32:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.774+0000 7fd7187deb80 1 --2- 172.21.15.7:0/1141601881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc1daccf60 0x55dc1dad04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.774+0000 7fd7187deb80 1 --2- 172.21.15.7:0/1141601881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc1dac6f20 0x55dc1dad2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.774+0000 7fd7187deb80 1 --2- 172.21.15.7:0/1141601881 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55dc1dae0900 0x55dc1dae2cf0 crc :-1 s=READY pgs=572 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.774+0000 7fd7187deb80 1 --2- 172.21.15.7:0/1141601881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc1dacc640 0x55dc1dacffa0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.774+0000 7fd6e6ffd700 0 client.30671 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:32:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.774+0000 7fd7187deb80 1 -- 172.21.15.7:0/1141601881 >> 172.21.15.7:0/1141601881 conn(0x55dc1d9f1cc0 msgr2=0x55dc1dac59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.774+0000 7fd7187deb80 1 -- 172.21.15.7:0/1141601881 shutdown_connections 2024-06-10T01:32:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.774+0000 7fd7187deb80 1 -- 172.21.15.7:0/1141601881 wait complete. 2024-06-10T01:32:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.774+0000 7fd7187deb80 20 client.30671 trim_cache size 0 max 16384 2024-06-10T01:32:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.790+0000 7fd7187deb80 1 Processor -- start 2024-06-10T01:32:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.790+0000 7fd7187deb80 1 -- start start 2024-06-10T01:32:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.791+0000 7fd7187deb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc1dac6f20 0x55dc1dac69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.791+0000 7fd7187deb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc1dacc640 0x55dc1dabb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.791+0000 7fd7187deb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc1daccf60 0x55dc1dae85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.791+0000 7fd7187deb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd7081b2cc0 con 0x55dc1dacc640 2024-06-10T01:32:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.791+0000 7fd7187deb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd7081b04c0 con 0x55dc1daccf60 2024-06-10T01:32:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.791+0000 7fd7187deb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd7081ae310 con 0x55dc1dac6f20 2024-06-10T01:32:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.791+0000 7fd706ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc1dacc640 0x55dc1dabb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.791+0000 7fd7077fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc1dac6f20 0x55dc1dac69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.791+0000 7fd706ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc1dacc640 0x55dc1dabb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45726/0 (socket says 172.21.15.7:45726) 2024-06-10T01:32:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.791+0000 7fd706ffd700 1 -- 172.21.15.7:0/2815379160 learned_addr learned my addr 172.21.15.7:0/2815379160 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:32:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.791+0000 7fd707fff700 1 --2- 172.21.15.7:0/2815379160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc1daccf60 0x55dc1dae85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.791+0000 7fd706ffd700 1 -- 172.21.15.7:0/2815379160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc1dac6f20 msgr2=0x55dc1dac69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:32:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.791+0000 7fd706ffd700 1 --2- 172.21.15.7:0/2815379160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc1dac6f20 0x55dc1dac69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.791+0000 7fd706ffd700 1 -- 172.21.15.7:0/2815379160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc1daccf60 msgr2=0x55dc1dae85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:32:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.791+0000 7fd706ffd700 1 --2- 172.21.15.7:0/2815379160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc1daccf60 0x55dc1dae85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.791+0000 7fd706ffd700 1 -- 172.21.15.7:0/2815379160 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc1dacc120 con 0x55dc1dacc640 2024-06-10T01:32:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.791+0000 7fd707fff700 1 --2- 172.21.15.7:0/2815379160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc1daccf60 0x55dc1dae85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:32:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.791+0000 7fd7077fe700 1 --2- 172.21.15.7:0/2815379160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc1dac6f20 0x55dc1dac69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:32:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.791+0000 7fd706ffd700 1 --2- 172.21.15.7:0/2815379160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc1dacc640 0x55dc1dabb510 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc025f70 tx=0x7fd6fc00f930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c83bcf6e7c7c5c5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.792+0000 7fd6ed7fa700 1 -- 172.21.15.7:0/2815379160 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc004a30 con 0x55dc1dacc640 2024-06-10T01:32:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.792+0000 7fd6ed7fa700 1 -- 172.21.15.7:0/2815379160 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6fc025920 con 0x55dc1dacc640 2024-06-10T01:32:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.792+0000 7fd6ed7fa700 1 -- 172.21.15.7:0/2815379160 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc005290 con 0x55dc1dacc640 2024-06-10T01:32:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.792+0000 7fd7187deb80 1 -- 172.21.15.7:0/2815379160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc1dacc640 msgr2=0x55dc1dabb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.792+0000 7fd7187deb80 1 --2- 172.21.15.7:0/2815379160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc1dacc640 0x55dc1dabb510 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc025f70 tx=0x7fd6fc00f930 comp rx=0 tx=0).stop 2024-06-10T01:32:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.792+0000 7fd7187deb80 1 -- 172.21.15.7:0/2815379160 shutdown_connections 2024-06-10T01:32:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.792+0000 7fd7187deb80 1 --2- 172.21.15.7:0/2815379160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc1daccf60 0x55dc1dae85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.792+0000 7fd7187deb80 1 --2- 172.21.15.7:0/2815379160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc1dac6f20 0x55dc1dac69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.792+0000 7fd7187deb80 1 --2- 172.21.15.7:0/2815379160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc1dacc640 0x55dc1dabb510 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.792+0000 7fd7187deb80 1 -- 172.21.15.7:0/2815379160 >> 172.21.15.7:0/2815379160 conn(0x55dc1d9f1cc0 msgr2=0x55dc1dabae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.792+0000 7fd7187deb80 1 -- 172.21.15.7:0/2815379160 shutdown_connections 2024-06-10T01:32:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.793+0000 7fd7187deb80 1 -- 172.21.15.7:0/2815379160 wait complete. 2024-06-10T01:32:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.793+0000 7fd7187deb80 1 -- start start 2024-06-10T01:32:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.793+0000 7fd7187deb80 1 Processor -- start 2024-06-10T01:32:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.793+0000 7fd7187deb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc1dac6f20 0x55dc1dad92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.793+0000 7fd7187deb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc1dacc640 0x55dc1dad4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.793+0000 7fd7187deb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc1daccf60 0x55dc1dad4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.793+0000 7fd7187deb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd7081ae980 con 0x55dc1dac6f20 2024-06-10T01:32:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.793+0000 7fd7187deb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd7081afdf0 con 0x55dc1daccf60 2024-06-10T01:32:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.793+0000 7fd7187deb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd7081af050 con 0x55dc1dacc640 2024-06-10T01:32:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.793+0000 7fd7187deb80 10 client.0 _ll_register_callbacks cb 0x55dc1d9b2920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:32:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.793+0000 7fd7077fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc1dac6f20 0x55dc1dad92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.794+0000 7fd7077fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc1dac6f20 0x55dc1dad92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45734/0 (socket says 172.21.15.7:45734) 2024-06-10T01:32:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.794+0000 7fd7077fe700 1 -- 172.21.15.7:0/2155525435 learned_addr learned my addr 172.21.15.7:0/2155525435 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:32:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.794+0000 7fd706ffd700 1 --2- 172.21.15.7:0/2155525435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc1dacc640 0x55dc1dad4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.794+0000 7fd707fff700 1 --2- 172.21.15.7:0/2155525435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc1daccf60 0x55dc1dad4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.794+0000 7fd7077fe700 1 -- 172.21.15.7:0/2155525435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc1dacc640 msgr2=0x55dc1dad4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:32:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.794+0000 7fd7077fe700 1 --2- 172.21.15.7:0/2155525435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc1dacc640 0x55dc1dad4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.794+0000 7fd7077fe700 1 -- 172.21.15.7:0/2155525435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc1daccf60 msgr2=0x55dc1dad4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:32:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.794+0000 7fd7077fe700 1 --2- 172.21.15.7:0/2155525435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc1daccf60 0x55dc1dad4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.794+0000 7fd7077fe700 1 -- 172.21.15.7:0/2155525435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc1dad5150 con 0x55dc1dac6f20 2024-06-10T01:32:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.794+0000 7fd7077fe700 1 --2- 172.21.15.7:0/2155525435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc1dac6f20 0x55dc1dad92d0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8098b10 tx=0x7fd6f80a3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc65e26f1855512c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.794+0000 7fd6e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-10T01:32:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.794+0000 7fd6e6ffd700 1 -- 172.21.15.7:0/2155525435 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f80870b0 con 0x55dc1dac6f20 2024-06-10T01:32:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.794+0000 7fd6e6ffd700 1 -- 172.21.15.7:0/2155525435 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fd6f809c910 con 0x55dc1dac6f20 2024-06-10T01:32:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.794+0000 7fd7187deb80 1 -- 172.21.15.7:0/2155525435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6fc025920 con 0x55dc1dac6f20 2024-06-10T01:32:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.794+0000 7fd6e6ffd700 1 -- 172.21.15.7:0/2155525435 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fd6f809cb10 con 0x55dc1dac6f20 2024-06-10T01:32:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.794+0000 7fd7187deb80 10 client.21513 fetch_fsmap learned FSMap version 60 2024-06-10T01:32:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.794+0000 7fd7187deb80 1 -- 172.21.15.7:0/2155525435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55dc1db4b3a0 con 0x55dc1dac6f20 2024-06-10T01:32:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.795+0000 7fd6e6ffd700 1 -- 172.21.15.7:0/2155525435 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6fc025920 con 0x55dc1dac6f20 2024-06-10T01:32:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.795+0000 7fd6e6ffd700 1 -- 172.21.15.7:0/2155525435 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 60) ==== 8362+0+0 (secure 0 0 0) 0x7fd6f809bb00 con 0x55dc1dac6f20 2024-06-10T01:32:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.795+0000 7fd7187deb80 10 client.21513 fetch_fsmap finished waiting for FSMap version 60 2024-06-10T01:32:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.795+0000 7fd7187deb80 10 client.21513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:32:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.795+0000 7fd7187deb80 20 client.21513 populate_metadata read hostname 'smithi007' 2024-06-10T01:32:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.795+0000 7fd7187deb80 1 --2- 172.21.15.7:0/2155525435 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55dc1db4b990 0x55dc1db4ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.795+0000 7fd7187deb80 4 client.21513 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:32:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.795+0000 7fd7187deb80 1 -- 172.21.15.7:0/2155525435 --> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc1da20290 con 0x55dc1db4b990 2024-06-10T01:32:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.795+0000 7fd706ffd700 1 --2- 172.21.15.7:0/2155525435 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55dc1db4b990 0x55dc1db4ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.795+0000 7fd6e6ffd700 10 client.21513 ms_handle_connect on v2:172.21.15.175:6838/1983418024 2024-06-10T01:32:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.795+0000 7fd706ffd700 1 --2- 172.21.15.7:0/2155525435 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55dc1db4b990 0x55dc1db4ddb0 crc :-1 s=READY pgs=573 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50e6e0af87b81ad0 server_cookie=157689805ca0d850 in_seq=0 out_seq=0 2024-06-10T01:32:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.796+0000 7fd6e6ffd700 1 -- 172.21.15.7:0/2155525435 <== mds.0 v2:172.21.15.175:6838/1983418024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc1da20290 con 0x55dc1db4b990 2024-06-10T01:32:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.796+0000 7fd6e6ffd700 10 client.21513 handle_command_reply: tid=0 2024-06-10T01:32:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.796+0000 7fd7187deb80 1 client.21513 shutdown 2024-06-10T01:32:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.796+0000 7fd7187deb80 1 -- 172.21.15.7:0/2155525435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc1dac6f20 msgr2=0x55dc1dad92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.796+0000 7fd7187deb80 1 --2- 172.21.15.7:0/2155525435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc1dac6f20 0x55dc1dad92d0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8098b10 tx=0x7fd6f80a3040 comp rx=0 tx=0).stop 2024-06-10T01:32:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.796+0000 7fd7187deb80 1 -- 172.21.15.7:0/2155525435 shutdown_connections 2024-06-10T01:32:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.796+0000 7fd7187deb80 1 --2- 172.21.15.7:0/2155525435 >> [v2:172.21.15.175:6838/1983418024,v1:172.21.15.175:6839/1983418024] conn(0x55dc1db4b990 0x55dc1db4ddb0 crc :-1 s=READY pgs=573 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.796+0000 7fd6e6ffd700 0 client.21513 ms_handle_reset on v2:172.21.15.175:6838/1983418024 2024-06-10T01:32:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.796+0000 7fd7187deb80 1 --2- 172.21.15.7:0/2155525435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc1daccf60 0x55dc1dad4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.796+0000 7fd7187deb80 1 --2- 172.21.15.7:0/2155525435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc1dac6f20 0x55dc1dad92d0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.796+0000 7fd7187deb80 1 --2- 172.21.15.7:0/2155525435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc1dacc640 0x55dc1dad4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.796+0000 7fd7187deb80 1 -- 172.21.15.7:0/2155525435 >> 172.21.15.7:0/2155525435 conn(0x55dc1d9f1cc0 msgr2=0x55dc1dacdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.796+0000 7fd7187deb80 1 -- 172.21.15.7:0/2155525435 shutdown_connections 2024-06-10T01:32:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.796+0000 7fd7187deb80 1 -- 172.21.15.7:0/2155525435 wait complete. 2024-06-10T01:32:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.796+0000 7fd7187deb80 20 client.21513 trim_cache size 0 max 16384 2024-06-10T01:32:11.795 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-10T01:32:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.797+0000 7fd70df5b700 1 -- 172.21.15.7:0/3632880912 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd6f00a2c20 msgr2=0x7fd6f00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.797+0000 7fd70df5b700 1 --2- 172.21.15.7:0/3632880912 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd6f00a2c20 0x7fd6f00a50e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd708101200 tx=0x7fd6fc00a300 comp rx=0 tx=0).stop 2024-06-10T01:32:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.797+0000 7fd70df5b700 1 -- 172.21.15.7:0/3632880912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd708101610 msgr2=0x7fd7081030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.797+0000 7fd70df5b700 1 --2- 172.21.15.7:0/3632880912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd708101610 0x7fd7081030b0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80059d0 tx=0x7fd6f80038b0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.797+0000 7fd70df5b700 1 -- 172.21.15.7:0/3632880912 shutdown_connections 2024-06-10T01:32:11.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.797+0000 7fd70df5b700 1 --2- 172.21.15.7:0/3632880912 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd6f00a2c20 0x7fd6f00a50e0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.797+0000 7fd70df5b700 1 --2- 172.21.15.7:0/3632880912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7081066f0 0x7fd708103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.797+0000 7fd70df5b700 1 --2- 172.21.15.7:0/3632880912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd708101610 0x7fd7081030b0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.797+0000 7fd70df5b700 1 --2- 172.21.15.7:0/3632880912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd708101fc0 0x7fd7081035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:11.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.797+0000 7fd70df5b700 1 -- 172.21.15.7:0/3632880912 >> 172.21.15.7:0/3632880912 conn(0x7fd708078430 msgr2=0x7fd70806aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:11.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.797+0000 7fd70df5b700 1 -- 172.21.15.7:0/3632880912 shutdown_connections 2024-06-10T01:32:11.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:11.797+0000 7fd70df5b700 1 -- 172.21.15.7:0/3632880912 wait complete. 2024-06-10T01:32:12.475 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-06-10T01:32:12.476 INFO:tasks.fwd_scrub:done joining 2024-06-10T01:32:12.476 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-06-10T01:32:12.486 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-10T01:32:12.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.605+0000 7f373ca46640 1 -- 172.21.15.7:0/3792003544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3738101a60 msgr2=0x7f3738101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:12.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.605+0000 7f373ca46640 1 --2- 172.21.15.7:0/3792003544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3738101a60 0x7f3738101e80 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f3728009900 tx=0x7f372802a410 comp rx=0 tx=0).stop 2024-06-10T01:32:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.605+0000 7f373ca46640 1 -- 172.21.15.7:0/3792003544 shutdown_connections 2024-06-10T01:32:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.605+0000 7f373ca46640 1 --2- 172.21.15.7:0/3792003544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3738103610 0x7f3738109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.605+0000 7f373ca46640 1 --2- 172.21.15.7:0/3792003544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3738102c50 0x7f37381030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.605+0000 7f373ca46640 1 --2- 172.21.15.7:0/3792003544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3738101a60 0x7f3738101e80 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.605+0000 7f373ca46640 1 -- 172.21.15.7:0/3792003544 >> 172.21.15.7:0/3792003544 conn(0x7f37380fcfd0 msgr2=0x7f37380ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.605+0000 7f373ca46640 1 -- 172.21.15.7:0/3792003544 shutdown_connections 2024-06-10T01:32:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.605+0000 7f373ca46640 1 -- 172.21.15.7:0/3792003544 wait complete. 2024-06-10T01:32:12.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.606+0000 7f373ca46640 1 Processor -- start 2024-06-10T01:32:12.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.606+0000 7f373ca46640 1 -- start start 2024-06-10T01:32:12.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.606+0000 7f373ca46640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3738101a60 0x7f3738075750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:12.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.606+0000 7f373ca46640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3738102c50 0x7f3738073d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:12.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.606+0000 7f373ca46640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3738103610 0x7f37380742d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:12.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.606+0000 7f373ca46640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f373810d690 con 0x7f3738102c50 2024-06-10T01:32:12.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.606+0000 7f373ca46640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f373810d510 con 0x7f3738101a60 2024-06-10T01:32:12.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.606+0000 7f373ca46640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f373810d810 con 0x7f3738103610 2024-06-10T01:32:12.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.606+0000 7f3735d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3738102c50 0x7f3738073d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:12.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.606+0000 7f3735d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3738102c50 0x7f3738073d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45750/0 (socket says 172.21.15.7:45750) 2024-06-10T01:32:12.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.606+0000 7f3735d9b640 1 -- 172.21.15.7:0/3353433139 learned_addr learned my addr 172.21.15.7:0/3353433139 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:32:12.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.606+0000 7f373659c640 1 --2- 172.21.15.7:0/3353433139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3738101a60 0x7f3738075750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:12.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.606+0000 7f3736d9d640 1 --2- 172.21.15.7:0/3353433139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3738103610 0x7f37380742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:12.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.607+0000 7f3735d9b640 1 -- 172.21.15.7:0/3353433139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3738103610 msgr2=0x7f37380742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:12.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.607+0000 7f3735d9b640 1 --2- 172.21.15.7:0/3353433139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3738103610 0x7f37380742d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:12.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.607+0000 7f3735d9b640 1 -- 172.21.15.7:0/3353433139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3738101a60 msgr2=0x7f3738075750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:12.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.607+0000 7f3735d9b640 1 --2- 172.21.15.7:0/3353433139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3738101a60 0x7f3738075750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:12.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.607+0000 7f3735d9b640 1 -- 172.21.15.7:0/3353433139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3738074b60 con 0x7f3738102c50 2024-06-10T01:32:12.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.607+0000 7f373659c640 1 --2- 172.21.15.7:0/3353433139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3738101a60 0x7f3738075750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:32:12.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.607+0000 7f3736d9d640 1 --2- 172.21.15.7:0/3353433139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3738103610 0x7f37380742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:32:12.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.607+0000 7f3735d9b640 1 --2- 172.21.15.7:0/3353433139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3738102c50 0x7f3738073d90 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f372c00e9f0 tx=0x7f372c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:12.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.607+0000 7f37277fe640 1 -- 172.21.15.7:0/3353433139 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c00cd80 con 0x7f3738102c50 2024-06-10T01:32:12.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.607+0000 7f37277fe640 1 -- 172.21.15.7:0/3353433139 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f372c010910 con 0x7f3738102c50 2024-06-10T01:32:12.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.607+0000 7f373ca46640 1 -- 172.21.15.7:0/3353433139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37381a1dc0 con 0x7f3738102c50 2024-06-10T01:32:12.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.607+0000 7f37277fe640 1 -- 172.21.15.7:0/3353433139 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c018990 con 0x7f3738102c50 2024-06-10T01:32:12.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.607+0000 7f373ca46640 1 -- 172.21.15.7:0/3353433139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37381a2280 con 0x7f3738102c50 2024-06-10T01:32:12.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.608+0000 7f373ca46640 1 -- 172.21.15.7:0/3353433139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3738108d80 con 0x7f3738102c50 2024-06-10T01:32:12.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.609+0000 7f37277fe640 1 -- 172.21.15.7:0/3353433139 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f372c010ac0 con 0x7f3738102c50 2024-06-10T01:32:12.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.610+0000 7f37277fe640 1 --2- 172.21.15.7:0/3353433139 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f37200a2b90 0x7f37200a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:12.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.610+0000 7f37277fe640 1 -- 172.21.15.7:0/3353433139 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f372c0cf8e0 con 0x7f3738102c50 2024-06-10T01:32:12.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.610+0000 7f373659c640 1 --2- 172.21.15.7:0/3353433139 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f37200a2b90 0x7f37200a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:12.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.611+0000 7f373659c640 1 --2- 172.21.15.7:0/3353433139 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f37200a2b90 0x7f37200a5050 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f372802a930 tx=0x7f3728000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:12.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.615+0000 7f37277fe640 1 -- 172.21.15.7:0/3353433139 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f372c09c460 con 0x7f3738102c50 2024-06-10T01:32:12.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.800+0000 7f373ca46640 1 -- 172.21.15.7:0/3353433139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3738062640 con 0x7f3738102c50 2024-06-10T01:32:12.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.802+0000 7f37277fe640 1 -- 172.21.15.7:0/3353433139 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+24616 (secure 0 0 0) 0x7f372c09bd60 con 0x7f3738102c50 2024-06-10T01:32:12.794 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:32:12.794 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":102,"fsid":"bded58f8-26c3-11ef-bc9b-c7b262605968","created":"2024-06-10T00:53:31.966085+0000","modified":"2024-06-10T01:27:44.422698+0000","last_up_change":"2024-06-10T01:00:45.382473+0000","last_in_change":"2024-06-10T01:00:25.608513+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:56:59.964055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T01:01:42.563611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T01:01:46.175918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T01:01:55.695502+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f7c24c9f-b7ab-4c04-b2b4-120df9751667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6803","nonce":404485726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6805","nonce":404485726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6809","nonce":404485726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6807","nonce":404485726}]},"public_addr":"172.21.15.7:6803/404485726","cluster_addr":"172.21.15.7:6805/404485726","heartbeat_back_addr":"172.21.15.7:6809/404485726","heartbeat_front_addr":"172.21.15.7:6807/404485726","state":["exists","up"]},{"osd":1,"uuid":"9bf70a6b-f31c-4077-b5af-229f17bc2845","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6801","nonce":1856804448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6803","nonce":1856804448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6807","nonce":1856804448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6805","nonce":1856804448}]},"public_addr":"172.21.15.78:6801/1856804448","cluster_addr":"172.21.15.78:6803/1856804448","heartbeat_back_addr":"172.21.15.78:6807/1856804448","heartbeat_front_addr":"172.21.15.78:6805/1856804448","state":["exists","up"]},{"osd":2,"uuid":"62923c97-caf9-4f7e-9c14-9c9918843c78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6801","nonce":918257455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6803","nonce":918257455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6807","nonce":918257455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6805","nonce":918257455}]},"public_addr":"172.21.15.175:6801/918257455","cluster_addr":"172.21.15.175:6803/918257455","heartbeat_back_addr":"172.21.15.175:6807/918257455","heartbeat_front_addr":"172.21.15.175:6805/918257455","state":["exists","up"]},{"osd":3,"uuid":"65242098-e748-48ab-8f0e-bb25b5c62742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6811","nonce":1797404966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6813","nonce":1797404966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6817","nonce":1797404966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6815","nonce":1797404966}]},"public_addr":"172.21.15.7:6811/1797404966","cluster_addr":"172.21.15.7:6813/1797404966","heartbeat_back_addr":"172.21.15.7:6817/1797404966","heartbeat_front_addr":"172.21.15.7:6815/1797404966","state":["exists","up"]},{"osd":4,"uuid":"a7990305-de9e-400e-a432-d7854f73f339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6809","nonce":3767608357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6811","nonce":3767608357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6815","nonce":3767608357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6813","nonce":3767608357}]},"public_addr":"172.21.15.78:6809/3767608357","cluster_addr":"172.21.15.78:6811/3767608357","heartbeat_back_addr":"172.21.15.78:6815/3767608357","heartbeat_front_addr":"172.21.15.78:6813/3767608357","state":["exists","up"]},{"osd":5,"uuid":"bad9a7b3-da5a-48d0-905d-e6a3190317fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6809","nonce":2157664731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6811","nonce":2157664731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6815","nonce":2157664731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6813","nonce":2157664731}]},"public_addr":"172.21.15.175:6809/2157664731","cluster_addr":"172.21.15.175:6811/2157664731","heartbeat_back_addr":"172.21.15.175:6815/2157664731","heartbeat_front_addr":"172.21.15.175:6813/2157664731","state":["exists","up"]},{"osd":6,"uuid":"6c687653-157b-4b96-867b-a2eaa7d8ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6819","nonce":838731756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6821","nonce":838731756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6825","nonce":838731756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6823","nonce":838731756}]},"public_addr":"172.21.15.7:6819/838731756","cluster_addr":"172.21.15.7:6821/838731756","heartbeat_back_addr":"172.21.15.7:6825/838731756","heartbeat_front_addr":"172.21.15.7:6823/838731756","state":["exists","up"]},{"osd":7,"uuid":"db40ddf1-9fd9-4859-b8c4-75f79582ab91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6817","nonce":3511921773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6819","nonce":3511921773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6823","nonce":3511921773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6821","nonce":3511921773}]},"public_addr":"172.21.15.78:6817/3511921773","cluster_addr":"172.21.15.78:6819/3511921773","heartbeat_back_addr":"172.21.15.78:6823/3511921773","heartbeat_front_addr":"172.21.15.78:6821/3511921773","state":["exists","up"]},{"osd":8,"uuid":"0f01cbd9-b193-4c70-a33c-4984167bf422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6817","nonce":1546325211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6819","nonce":1546325211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6823","nonce":1546325211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6821","nonce":1546325211}]},"public_addr":"172.21.15.175:6817/1546325211","cluster_addr":"172.21.15.175:6819/1546325211","heartbeat_back_addr":"172.21.15.175:6823/1546325211","heartbeat_front_addr":"172.21.15.175:6821/1546325211","state":["exists","up"]},{"osd":9,"uuid":"e5480998-f7fb-49cd-9aed-52405c88d830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6827","nonce":1844550813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6829","nonce":1844550813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6833","nonce":1844550813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6831","nonce":1844550813}]},"public_addr":"172.21.15.7:6827/1844550813","cluster_addr":"172.21.15.7:6829/1844550813","heartbeat_back_addr":"172.21.15.7:6833/1844550813","heartbeat_front_addr":"172.21.15.7:6831/1844550813","state":["exists","up"]},{"osd":10,"uuid":"caa82f33-a71c-4907-bf3b-30a7fdfeb71d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6825","nonce":2290824785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6827","nonce":2290824785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6831","nonce":2290824785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6829","nonce":2290824785}]},"public_addr":"172.21.15.78:6825/2290824785","cluster_addr":"172.21.15.78:6827/2290824785","heartbeat_back_addr":"172.21.15.78:6831/2290824785","heartbeat_front_addr":"172.21.15.78:6829/2290824785","state":["exists","up"]},{"osd":11,"uuid":"608cda77-77da-4da2-a7e2-76a773be27c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6825","nonce":1959979550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6827","nonce":1959979550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6831","nonce":1959979550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6829","nonce":1959979550}]},"public_addr":"172.21.15.175:6825/1959979550","cluster_addr":"172.21.15.175:6827/1959979550","heartbeat_back_addr":"172.21.15.175:6831/1959979550","heartbeat_front_addr":"172.21.15.175:6829/1959979550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:08.850203+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:32.820161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:56.633656+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:21.759372+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:46.735874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:10.636541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:36.990099+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:01.767098+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:26.882776+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:52.757595+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:18.383938+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:43.544330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1833397470":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/322543993":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/212510583":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6801/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6800/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/2679436843":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6801/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/1210355277":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6800/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/3610017406":"2024-06-11T00:54:08.102405+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T01:32:12.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.804+0000 7f373ca46640 1 -- 172.21.15.7:0/3353433139 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f37200a2b90 msgr2=0x7f37200a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:12.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.804+0000 7f373ca46640 1 --2- 172.21.15.7:0/3353433139 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f37200a2b90 0x7f37200a5050 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f372802a930 tx=0x7f3728000bc0 comp rx=0 tx=0).stop 2024-06-10T01:32:12.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.804+0000 7f373ca46640 1 -- 172.21.15.7:0/3353433139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3738102c50 msgr2=0x7f3738073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:12.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.804+0000 7f373ca46640 1 --2- 172.21.15.7:0/3353433139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3738102c50 0x7f3738073d90 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f372c00e9f0 tx=0x7f372c00eeb0 comp rx=0 tx=0).stop 2024-06-10T01:32:12.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.804+0000 7f373ca46640 1 -- 172.21.15.7:0/3353433139 shutdown_connections 2024-06-10T01:32:12.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.804+0000 7f373ca46640 1 --2- 172.21.15.7:0/3353433139 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f37200a2b90 0x7f37200a5050 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:12.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.804+0000 7f373ca46640 1 --2- 172.21.15.7:0/3353433139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3738103610 0x7f37380742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:12.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.804+0000 7f373ca46640 1 --2- 172.21.15.7:0/3353433139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3738102c50 0x7f3738073d90 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:12.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.804+0000 7f373ca46640 1 --2- 172.21.15.7:0/3353433139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3738101a60 0x7f3738075750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:12.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.804+0000 7f373ca46640 1 -- 172.21.15.7:0/3353433139 >> 172.21.15.7:0/3353433139 conn(0x7f37380fcfd0 msgr2=0x7f3738105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:12.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.804+0000 7f373ca46640 1 -- 172.21.15.7:0/3353433139 shutdown_connections 2024-06-10T01:32:12.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.804+0000 7f373ca46640 1 -- 172.21.15.7:0/3353433139 wait complete. 2024-06-10T01:32:12.807 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-10T00:56:59.964055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-06-10T01:01:42.563611+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-06-10T01:01:46.175918+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-06-10T01:01:55.695502+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-06-10T01:32:12.808 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-10T01:32:12.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.922+0000 7f484d190640 1 -- 172.21.15.7:0/2058076283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48481024a0 msgr2=0x7f4848108d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:12.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.922+0000 7f484d190640 1 --2- 172.21.15.7:0/2058076283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48481024a0 0x7f4848108d30 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f4840009ca0 tx=0x7f484002a8c0 comp rx=0 tx=0).stop 2024-06-10T01:32:12.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.922+0000 7f484d190640 1 -- 172.21.15.7:0/2058076283 shutdown_connections 2024-06-10T01:32:12.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.922+0000 7f484d190640 1 --2- 172.21.15.7:0/2058076283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48481024a0 0x7f4848108d30 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:12.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.922+0000 7f484d190640 1 --2- 172.21.15.7:0/2058076283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4848101ae0 0x7f4848101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:12.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.923+0000 7f484d190640 1 --2- 172.21.15.7:0/2058076283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48481008f0 0x7f4848100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:12.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.923+0000 7f484d190640 1 -- 172.21.15.7:0/2058076283 >> 172.21.15.7:0/2058076283 conn(0x7f48480fbe80 msgr2=0x7f48480fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:12.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.923+0000 7f484d190640 1 -- 172.21.15.7:0/2058076283 shutdown_connections 2024-06-10T01:32:12.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.923+0000 7f484d190640 1 -- 172.21.15.7:0/2058076283 wait complete. 2024-06-10T01:32:12.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.923+0000 7f484d190640 1 Processor -- start 2024-06-10T01:32:12.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.924+0000 7f484d190640 1 -- start start 2024-06-10T01:32:12.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.924+0000 7f484d190640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48481008f0 0x7f484806abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:12.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.924+0000 7f484d190640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4848101ae0 0x7f484819ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:12.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.924+0000 7f484d190640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48481024a0 0x7f48481a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:12.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.924+0000 7f484d190640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f484810c550 con 0x7f48481008f0 2024-06-10T01:32:12.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.924+0000 7f484d190640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f484810c3d0 con 0x7f4848101ae0 2024-06-10T01:32:12.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.924+0000 7f484d190640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f484810c6d0 con 0x7f48481024a0 2024-06-10T01:32:12.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.924+0000 7f4846d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48481008f0 0x7f484806abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:12.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.924+0000 7f4846d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48481008f0 0x7f484806abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45770/0 (socket says 172.21.15.7:45770) 2024-06-10T01:32:12.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.924+0000 7f4846d9d640 1 -- 172.21.15.7:0/1398775123 learned_addr learned my addr 172.21.15.7:0/1398775123 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:32:12.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.924+0000 7f484759e640 1 --2- 172.21.15.7:0/1398775123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48481024a0 0x7f48481a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:12.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.924+0000 7f484659c640 1 --2- 172.21.15.7:0/1398775123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4848101ae0 0x7f484819ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:12.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.924+0000 7f4846d9d640 1 -- 172.21.15.7:0/1398775123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48481024a0 msgr2=0x7f48481a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:12.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.924+0000 7f4846d9d640 1 --2- 172.21.15.7:0/1398775123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48481024a0 0x7f48481a1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:12.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.924+0000 7f4846d9d640 1 -- 172.21.15.7:0/1398775123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4848101ae0 msgr2=0x7f484819ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:12.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.924+0000 7f4846d9d640 1 --2- 172.21.15.7:0/1398775123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4848101ae0 0x7f484819ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:12.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.924+0000 7f4846d9d640 1 -- 172.21.15.7:0/1398775123 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48481a1b20 con 0x7f48481008f0 2024-06-10T01:32:12.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.924+0000 7f484759e640 1 --2- 172.21.15.7:0/1398775123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48481024a0 0x7f48481a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:32:12.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.924+0000 7f4846d9d640 1 --2- 172.21.15.7:0/1398775123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48481008f0 0x7f484806abe0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f483800d950 tx=0x7f483800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:12.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.924+0000 7f482ffff640 1 -- 172.21.15.7:0/1398775123 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4838017020 con 0x7f48481008f0 2024-06-10T01:32:12.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.924+0000 7f484d190640 1 -- 172.21.15.7:0/1398775123 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48481a1db0 con 0x7f48481008f0 2024-06-10T01:32:12.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.925+0000 7f482ffff640 1 -- 172.21.15.7:0/1398775123 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4838009c30 con 0x7f48481008f0 2024-06-10T01:32:12.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.925+0000 7f482ffff640 1 -- 172.21.15.7:0/1398775123 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483800f740 con 0x7f48481008f0 2024-06-10T01:32:12.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.925+0000 7f484d190640 1 -- 172.21.15.7:0/1398775123 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48481a2270 con 0x7f48481008f0 2024-06-10T01:32:12.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.925+0000 7f484d190640 1 -- 172.21.15.7:0/1398775123 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4848107bb0 con 0x7f48481008f0 2024-06-10T01:32:12.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.927+0000 7f482ffff640 1 -- 172.21.15.7:0/1398775123 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f483801a020 con 0x7f48481008f0 2024-06-10T01:32:12.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.927+0000 7f482ffff640 1 --2- 172.21.15.7:0/1398775123 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f48300a2bc0 0x7f48300a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:12.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.928+0000 7f484659c640 1 --2- 172.21.15.7:0/1398775123 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f48300a2bc0 0x7f48300a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:12.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.928+0000 7f482ffff640 1 -- 172.21.15.7:0/1398775123 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f48380cf180 con 0x7f48481008f0 2024-06-10T01:32:12.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.928+0000 7f484659c640 1 --2- 172.21.15.7:0/1398775123 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f48300a2bc0 0x7f48300a5080 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f483c005e00 tx=0x7f483c005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:12.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:12.931+0000 7f482ffff640 1 -- 172.21.15.7:0/1398775123 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4838021070 con 0x7f48481008f0 2024-06-10T01:32:13.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.115+0000 7f484d190640 1 -- 172.21.15.7:0/1398775123 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f484802d2e0 con 0x7f48481008f0 2024-06-10T01:32:13.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.115+0000 7f482ffff640 1 -- 172.21.15.7:0/1398775123 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) ==== 93+0+10 (secure 0 0 0) 0x7f483809bd60 con 0x7f48481008f0 2024-06-10T01:32:13.108 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-06-10T01:32:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.117+0000 7f484d190640 1 -- 172.21.15.7:0/1398775123 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f48300a2bc0 msgr2=0x7f48300a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.117+0000 7f484d190640 1 --2- 172.21.15.7:0/1398775123 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f48300a2bc0 0x7f48300a5080 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f483c005e00 tx=0x7f483c005c50 comp rx=0 tx=0).stop 2024-06-10T01:32:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.117+0000 7f484d190640 1 -- 172.21.15.7:0/1398775123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48481008f0 msgr2=0x7f484806abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.117+0000 7f484d190640 1 --2- 172.21.15.7:0/1398775123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48481008f0 0x7f484806abe0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f483800d950 tx=0x7f483800de10 comp rx=0 tx=0).stop 2024-06-10T01:32:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.117+0000 7f484d190640 1 -- 172.21.15.7:0/1398775123 shutdown_connections 2024-06-10T01:32:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.117+0000 7f484d190640 1 --2- 172.21.15.7:0/1398775123 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f48300a2bc0 0x7f48300a5080 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.117+0000 7f484d190640 1 --2- 172.21.15.7:0/1398775123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48481024a0 0x7f48481a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.117+0000 7f484d190640 1 --2- 172.21.15.7:0/1398775123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4848101ae0 0x7f484819ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.117+0000 7f484d190640 1 --2- 172.21.15.7:0/1398775123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48481008f0 0x7f484806abe0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:13.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.117+0000 7f484d190640 1 -- 172.21.15.7:0/1398775123 >> 172.21.15.7:0/1398775123 conn(0x7f48480fbe80 msgr2=0x7f4848104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:13.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.117+0000 7f484d190640 1 -- 172.21.15.7:0/1398775123 shutdown_connections 2024-06-10T01:32:13.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.117+0000 7f484d190640 1 -- 172.21.15.7:0/1398775123 wait complete. 2024-06-10T01:32:13.121 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-06-10T01:32:13.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.237+0000 7f1c8f93b640 1 -- 172.21.15.7:0/404808734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c88100260 msgr2=0x7f1c88100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:13.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.237+0000 7f1c8f93b640 1 --2- 172.21.15.7:0/404808734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c88100260 0x7f1c88100680 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f1c78009ca0 tx=0x7f1c7802a8c0 comp rx=0 tx=0).stop 2024-06-10T01:32:13.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.237+0000 7f1c8f93b640 1 -- 172.21.15.7:0/404808734 shutdown_connections 2024-06-10T01:32:13.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.237+0000 7f1c8f93b640 1 --2- 172.21.15.7:0/404808734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c88101560 0x7f1c881019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:13.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.237+0000 7f1c8f93b640 1 --2- 172.21.15.7:0/404808734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c88100bc0 0x7f1c88101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:13.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.237+0000 7f1c8f93b640 1 --2- 172.21.15.7:0/404808734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c88100260 0x7f1c88100680 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:13.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.237+0000 7f1c8f93b640 1 -- 172.21.15.7:0/404808734 >> 172.21.15.7:0/404808734 conn(0x7f1c880fbe80 msgr2=0x7f1c880fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:13.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.237+0000 7f1c8f93b640 1 -- 172.21.15.7:0/404808734 shutdown_connections 2024-06-10T01:32:13.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.237+0000 7f1c8f93b640 1 -- 172.21.15.7:0/404808734 wait complete. 2024-06-10T01:32:13.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.238+0000 7f1c8f93b640 1 Processor -- start 2024-06-10T01:32:13.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.238+0000 7f1c8f93b640 1 -- start start 2024-06-10T01:32:13.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.238+0000 7f1c8f93b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c88100260 0x7f1c88199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:13.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.238+0000 7f1c8f93b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c88100bc0 0x7f1c8819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:13.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.238+0000 7f1c8f93b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c88101560 0x7f1c881a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:13.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.238+0000 7f1c8f93b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8810f850 con 0x7f1c88100260 2024-06-10T01:32:13.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.238+0000 7f1c8f93b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8810f6d0 con 0x7f1c88100bc0 2024-06-10T01:32:13.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.238+0000 7f1c8f93b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8810f9d0 con 0x7f1c88101560 2024-06-10T01:32:13.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.238+0000 7f1c8d6d7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c88100260 0x7f1c88199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:13.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.238+0000 7f1c8d6d7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c88100260 0x7f1c88199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45794/0 (socket says 172.21.15.7:45794) 2024-06-10T01:32:13.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.238+0000 7f1c8d6d7640 1 -- 172.21.15.7:0/1215100547 learned_addr learned my addr 172.21.15.7:0/1215100547 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:32:13.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.238+0000 7f1c8ced6640 1 --2- 172.21.15.7:0/1215100547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c88100bc0 0x7f1c8819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:13.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.238+0000 7f1c8ded8640 1 --2- 172.21.15.7:0/1215100547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c88101560 0x7f1c881a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:13.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.238+0000 7f1c8d6d7640 1 -- 172.21.15.7:0/1215100547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c88101560 msgr2=0x7f1c881a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:13.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.238+0000 7f1c8d6d7640 1 --2- 172.21.15.7:0/1215100547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c88101560 0x7f1c881a14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:13.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.238+0000 7f1c8d6d7640 1 -- 172.21.15.7:0/1215100547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c88100bc0 msgr2=0x7f1c8819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:13.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.238+0000 7f1c8d6d7640 1 --2- 172.21.15.7:0/1215100547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c88100bc0 0x7f1c8819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:13.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.238+0000 7f1c8d6d7640 1 -- 172.21.15.7:0/1215100547 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c881a1b40 con 0x7f1c88100260 2024-06-10T01:32:13.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.238+0000 7f1c8ced6640 1 --2- 172.21.15.7:0/1215100547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c88100bc0 0x7f1c8819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:32:13.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.239+0000 7f1c8d6d7640 1 --2- 172.21.15.7:0/1215100547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c88100260 0x7f1c88199f20 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f1c78031a40 tx=0x7f1c78031ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:13.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.239+0000 7f1c867fc640 1 -- 172.21.15.7:0/1215100547 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7802faa0 con 0x7f1c88100260 2024-06-10T01:32:13.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.239+0000 7f1c867fc640 1 -- 172.21.15.7:0/1215100547 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c7802fc40 con 0x7f1c88100260 2024-06-10T01:32:13.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.239+0000 7f1c8f93b640 1 -- 172.21.15.7:0/1215100547 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c881a1dd0 con 0x7f1c88100260 2024-06-10T01:32:13.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.239+0000 7f1c867fc640 1 -- 172.21.15.7:0/1215100547 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7803a790 con 0x7f1c88100260 2024-06-10T01:32:13.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.239+0000 7f1c8f93b640 1 -- 172.21.15.7:0/1215100547 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c881a2230 con 0x7f1c88100260 2024-06-10T01:32:13.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.240+0000 7f1c8f93b640 1 -- 172.21.15.7:0/1215100547 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c8810afb0 con 0x7f1c88100260 2024-06-10T01:32:13.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.241+0000 7f1c867fc640 1 -- 172.21.15.7:0/1215100547 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f1c78030b30 con 0x7f1c88100260 2024-06-10T01:32:13.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.242+0000 7f1c867fc640 1 --2- 172.21.15.7:0/1215100547 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1c700a2b90 0x7f1c700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:13.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.242+0000 7f1c867fc640 1 -- 172.21.15.7:0/1215100547 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f1c780f0bc0 con 0x7f1c88100260 2024-06-10T01:32:13.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.242+0000 7f1c8ced6640 1 --2- 172.21.15.7:0/1215100547 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1c700a2b90 0x7f1c700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:13.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.242+0000 7f1c8ced6640 1 --2- 172.21.15.7:0/1215100547 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1c700a2b90 0x7f1c700a5050 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1c8819b430 tx=0x7f1c7c005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:13.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.246+0000 7f1c867fc640 1 -- 172.21.15.7:0/1215100547 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1c780bd740 con 0x7f1c88100260 2024-06-10T01:32:13.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.432+0000 7f1c8f93b640 1 -- 172.21.15.7:0/1215100547 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1c88061de0 con 0x7f1c88100260 2024-06-10T01:32:13.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.432+0000 7f1c867fc640 1 -- 172.21.15.7:0/1215100547 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v102) ==== 104+0+11 (secure 0 0 0) 0x7f1c780bd040 con 0x7f1c88100260 2024-06-10T01:32:13.424 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-06-10T01:32:13.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.435+0000 7f1c8f93b640 1 -- 172.21.15.7:0/1215100547 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1c700a2b90 msgr2=0x7f1c700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:13.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.435+0000 7f1c8f93b640 1 --2- 172.21.15.7:0/1215100547 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1c700a2b90 0x7f1c700a5050 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1c8819b430 tx=0x7f1c7c005e50 comp rx=0 tx=0).stop 2024-06-10T01:32:13.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.435+0000 7f1c8f93b640 1 -- 172.21.15.7:0/1215100547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c88100260 msgr2=0x7f1c88199f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:13.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.435+0000 7f1c8f93b640 1 --2- 172.21.15.7:0/1215100547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c88100260 0x7f1c88199f20 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f1c78031a40 tx=0x7f1c78031ec0 comp rx=0 tx=0).stop 2024-06-10T01:32:13.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.435+0000 7f1c8f93b640 1 -- 172.21.15.7:0/1215100547 shutdown_connections 2024-06-10T01:32:13.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.435+0000 7f1c8f93b640 1 --2- 172.21.15.7:0/1215100547 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f1c700a2b90 0x7f1c700a5050 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:13.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.435+0000 7f1c8f93b640 1 --2- 172.21.15.7:0/1215100547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c88101560 0x7f1c881a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:13.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.435+0000 7f1c8f93b640 1 --2- 172.21.15.7:0/1215100547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c88100bc0 0x7f1c8819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:13.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.435+0000 7f1c8f93b640 1 --2- 172.21.15.7:0/1215100547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c88100260 0x7f1c88199f20 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:13.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.435+0000 7f1c8f93b640 1 -- 172.21.15.7:0/1215100547 >> 172.21.15.7:0/1215100547 conn(0x7f1c880fbe80 msgr2=0x7f1c880fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:13.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.435+0000 7f1c8f93b640 1 -- 172.21.15.7:0/1215100547 shutdown_connections 2024-06-10T01:32:13.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.435+0000 7f1c8f93b640 1 -- 172.21.15.7:0/1215100547 wait complete. 2024-06-10T01:32:13.437 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:13 smithi007 ceph-mon[32398]: pgmap v1220: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 152 KiB/s wr, 31 op/s 2024-06-10T01:32:13.437 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:13 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/3353433139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:32:13.437 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:13 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1398775123' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T01:32:13.438 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-06-10T01:32:13.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:13 smithi175 ceph-mon[36636]: pgmap v1220: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 152 KiB/s wr, 31 op/s 2024-06-10T01:32:13.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:13 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/3353433139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:32:13.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:13 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1398775123' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T01:32:13.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.556+0000 7fdc1cb3b640 1 -- 172.21.15.7:0/4073224931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc18101a00 msgr2=0x7fdc18101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:13.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.556+0000 7fdc1cb3b640 1 --2- 172.21.15.7:0/4073224931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc18101a00 0x7fdc18101e20 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fdc08009900 tx=0x7fdc0802a410 comp rx=0 tx=0).stop 2024-06-10T01:32:13.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.556+0000 7fdc1cb3b640 1 -- 172.21.15.7:0/4073224931 shutdown_connections 2024-06-10T01:32:13.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.556+0000 7fdc1cb3b640 1 --2- 172.21.15.7:0/4073224931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc181035b0 0x7fdc18109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:13.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.556+0000 7fdc1cb3b640 1 --2- 172.21.15.7:0/4073224931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc18102bf0 0x7fdc18103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:13.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.556+0000 7fdc1cb3b640 1 --2- 172.21.15.7:0/4073224931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc18101a00 0x7fdc18101e20 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:13.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.556+0000 7fdc1cb3b640 1 -- 172.21.15.7:0/4073224931 >> 172.21.15.7:0/4073224931 conn(0x7fdc180fcf70 msgr2=0x7fdc180ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:13.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.556+0000 7fdc1cb3b640 1 -- 172.21.15.7:0/4073224931 shutdown_connections 2024-06-10T01:32:13.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.556+0000 7fdc1cb3b640 1 -- 172.21.15.7:0/4073224931 wait complete. 2024-06-10T01:32:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.557+0000 7fdc1cb3b640 1 Processor -- start 2024-06-10T01:32:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.557+0000 7fdc1cb3b640 1 -- start start 2024-06-10T01:32:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.557+0000 7fdc1cb3b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc18101a00 0x7fdc18197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.557+0000 7fdc1cb3b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc18102bf0 0x7fdc18198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.557+0000 7fdc1cb3b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc181035b0 0x7fdc1819f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.557+0000 7fdc1cb3b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1810d650 con 0x7fdc181035b0 2024-06-10T01:32:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.557+0000 7fdc1cb3b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1810d4d0 con 0x7fdc18102bf0 2024-06-10T01:32:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.557+0000 7fdc1cb3b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1810d7d0 con 0x7fdc18101a00 2024-06-10T01:32:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.557+0000 7fdc16d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc181035b0 0x7fdc1819f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.557+0000 7fdc16d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc181035b0 0x7fdc1819f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45802/0 (socket says 172.21.15.7:45802) 2024-06-10T01:32:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.557+0000 7fdc16d9d640 1 -- 172.21.15.7:0/154317596 learned_addr learned my addr 172.21.15.7:0/154317596 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:32:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.557+0000 7fdc1659c640 1 --2- 172.21.15.7:0/154317596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc18101a00 0x7fdc18197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.557+0000 7fdc0ffff640 1 --2- 172.21.15.7:0/154317596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc18102bf0 0x7fdc18198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.557+0000 7fdc16d9d640 1 -- 172.21.15.7:0/154317596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc18101a00 msgr2=0x7fdc18197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.557+0000 7fdc16d9d640 1 --2- 172.21.15.7:0/154317596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc18101a00 0x7fdc18197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.557+0000 7fdc16d9d640 1 -- 172.21.15.7:0/154317596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc18102bf0 msgr2=0x7fdc18198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.557+0000 7fdc16d9d640 1 --2- 172.21.15.7:0/154317596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc18102bf0 0x7fdc18198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.557+0000 7fdc16d9d640 1 -- 172.21.15.7:0/154317596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1819f960 con 0x7fdc181035b0 2024-06-10T01:32:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.557+0000 7fdc0ffff640 1 --2- 172.21.15.7:0/154317596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc18102bf0 0x7fdc18198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:32:13.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.558+0000 7fdc16d9d640 1 --2- 172.21.15.7:0/154317596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc181035b0 0x7fdc1819f2c0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000cad0 tx=0x7fdc1000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:13.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.558+0000 7fdc0f7fe640 1 -- 172.21.15.7:0/154317596 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1000ae20 con 0x7fdc181035b0 2024-06-10T01:32:13.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.558+0000 7fdc0f7fe640 1 -- 172.21.15.7:0/154317596 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc10005020 con 0x7fdc181035b0 2024-06-10T01:32:13.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.558+0000 7fdc1cb3b640 1 -- 172.21.15.7:0/154317596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc1819fc50 con 0x7fdc181035b0 2024-06-10T01:32:13.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.558+0000 7fdc0f7fe640 1 -- 172.21.15.7:0/154317596 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1000f6f0 con 0x7fdc181035b0 2024-06-10T01:32:13.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.558+0000 7fdc1cb3b640 1 -- 172.21.15.7:0/154317596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc181a0150 con 0x7fdc181035b0 2024-06-10T01:32:13.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.559+0000 7fdc1cb3b640 1 -- 172.21.15.7:0/154317596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc18108da0 con 0x7fdc181035b0 2024-06-10T01:32:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.560+0000 7fdc0f7fe640 1 -- 172.21.15.7:0/154317596 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fdc100051d0 con 0x7fdc181035b0 2024-06-10T01:32:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.561+0000 7fdc0f7fe640 1 --2- 172.21.15.7:0/154317596 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdc040a2b90 0x7fdc040a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.561+0000 7fdc0f7fe640 1 -- 172.21.15.7:0/154317596 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7fdc100cf530 con 0x7fdc181035b0 2024-06-10T01:32:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.561+0000 7fdc1659c640 1 --2- 172.21.15.7:0/154317596 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdc040a2b90 0x7fdc040a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.561+0000 7fdc1659c640 1 --2- 172.21.15.7:0/154317596 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdc040a2b90 0x7fdc040a5050 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fdc080314e0 tx=0x7fdc08000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:13.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.565+0000 7fdc0f7fe640 1 -- 172.21.15.7:0/154317596 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdc10014030 con 0x7fdc181035b0 2024-06-10T01:32:13.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:13 smithi078 ceph-mon[36683]: pgmap v1220: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 152 KiB/s wr, 31 op/s 2024-06-10T01:32:13.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:13 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/3353433139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:32:13.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:13 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1398775123' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T01:32:13.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.751+0000 7fdc1cb3b640 1 -- 172.21.15.7:0/154317596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdc18062640 con 0x7fdc181035b0 2024-06-10T01:32:13.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.751+0000 7fdc0f7fe640 1 -- 172.21.15.7:0/154317596 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v102) ==== 100+0+11 (secure 0 0 0) 0x7fdc1009c110 con 0x7fdc181035b0 2024-06-10T01:32:13.743 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-06-10T01:32:13.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.753+0000 7fdc1cb3b640 1 -- 172.21.15.7:0/154317596 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdc040a2b90 msgr2=0x7fdc040a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:13.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.753+0000 7fdc1cb3b640 1 --2- 172.21.15.7:0/154317596 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdc040a2b90 0x7fdc040a5050 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fdc080314e0 tx=0x7fdc08000bc0 comp rx=0 tx=0).stop 2024-06-10T01:32:13.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.753+0000 7fdc1cb3b640 1 -- 172.21.15.7:0/154317596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc181035b0 msgr2=0x7fdc1819f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:13.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.753+0000 7fdc1cb3b640 1 --2- 172.21.15.7:0/154317596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc181035b0 0x7fdc1819f2c0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000cad0 tx=0x7fdc1000cf90 comp rx=0 tx=0).stop 2024-06-10T01:32:13.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.753+0000 7fdc1cb3b640 1 -- 172.21.15.7:0/154317596 shutdown_connections 2024-06-10T01:32:13.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.753+0000 7fdc1cb3b640 1 --2- 172.21.15.7:0/154317596 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fdc040a2b90 0x7fdc040a5050 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:13.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.753+0000 7fdc1cb3b640 1 --2- 172.21.15.7:0/154317596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc181035b0 0x7fdc1819f2c0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:13.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.753+0000 7fdc1cb3b640 1 --2- 172.21.15.7:0/154317596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc18102bf0 0x7fdc18198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:13.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.753+0000 7fdc1cb3b640 1 --2- 172.21.15.7:0/154317596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc18101a00 0x7fdc18197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:13.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.753+0000 7fdc1cb3b640 1 -- 172.21.15.7:0/154317596 >> 172.21.15.7:0/154317596 conn(0x7fdc180fcf70 msgr2=0x7fdc18105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:13.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.753+0000 7fdc1cb3b640 1 -- 172.21.15.7:0/154317596 shutdown_connections 2024-06-10T01:32:13.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.753+0000 7fdc1cb3b640 1 -- 172.21.15.7:0/154317596 wait complete. 2024-06-10T01:32:13.757 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-06-10T01:32:13.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.873+0000 7f820e4b3640 1 -- 172.21.15.7:0/2300705561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8208102490 msgr2=0x7f8208108d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:13.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.873+0000 7f820e4b3640 1 --2- 172.21.15.7:0/2300705561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8208102490 0x7f8208108d20 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f8200009ca0 tx=0x7f820002a8c0 comp rx=0 tx=0).stop 2024-06-10T01:32:13.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.873+0000 7f820e4b3640 1 -- 172.21.15.7:0/2300705561 shutdown_connections 2024-06-10T01:32:13.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.873+0000 7f820e4b3640 1 --2- 172.21.15.7:0/2300705561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8208102490 0x7f8208108d20 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:13.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.873+0000 7f820e4b3640 1 --2- 172.21.15.7:0/2300705561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8208101ad0 0x7f8208101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:13.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.873+0000 7f820e4b3640 1 --2- 172.21.15.7:0/2300705561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82081008e0 0x7f8208100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:13.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.873+0000 7f820e4b3640 1 -- 172.21.15.7:0/2300705561 >> 172.21.15.7:0/2300705561 conn(0x7f82080fbe70 msgr2=0x7f82080fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:13.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.873+0000 7f820e4b3640 1 -- 172.21.15.7:0/2300705561 shutdown_connections 2024-06-10T01:32:13.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.873+0000 7f820e4b3640 1 -- 172.21.15.7:0/2300705561 wait complete. 2024-06-10T01:32:13.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.873+0000 7f820e4b3640 1 Processor -- start 2024-06-10T01:32:13.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.873+0000 7f820e4b3640 1 -- start start 2024-06-10T01:32:13.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.874+0000 7f820e4b3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82081008e0 0x7f8208197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:13.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.874+0000 7f820e4b3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8208101ad0 0x7f82081981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:13.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.874+0000 7f820e4b3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8208102490 0x7f820819f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:13.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.874+0000 7f820e4b3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f820810c560 con 0x7f8208102490 2024-06-10T01:32:13.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.874+0000 7f820e4b3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f820810c3e0 con 0x7f8208101ad0 2024-06-10T01:32:13.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.874+0000 7f820e4b3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f820810c6e0 con 0x7f82081008e0 2024-06-10T01:32:13.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.874+0000 7f820ca50640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8208102490 0x7f820819f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:13.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.874+0000 7f820ca50640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8208102490 0x7f820819f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45812/0 (socket says 172.21.15.7:45812) 2024-06-10T01:32:13.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.874+0000 7f820ca50640 1 -- 172.21.15.7:0/2542833302 learned_addr learned my addr 172.21.15.7:0/2542833302 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:32:13.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.874+0000 7f82077fe640 1 --2- 172.21.15.7:0/2542833302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8208101ad0 0x7f82081981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:13.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.874+0000 7f8207fff640 1 --2- 172.21.15.7:0/2542833302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82081008e0 0x7f8208197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:13.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.874+0000 7f820ca50640 1 -- 172.21.15.7:0/2542833302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82081008e0 msgr2=0x7f8208197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:13.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.874+0000 7f820ca50640 1 --2- 172.21.15.7:0/2542833302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82081008e0 0x7f8208197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:13.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.874+0000 7f820ca50640 1 -- 172.21.15.7:0/2542833302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8208101ad0 msgr2=0x7f82081981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:13.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.874+0000 7f820ca50640 1 --2- 172.21.15.7:0/2542833302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8208101ad0 0x7f82081981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:13.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.874+0000 7f820ca50640 1 -- 172.21.15.7:0/2542833302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f820819f8d0 con 0x7f8208102490 2024-06-10T01:32:13.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.874+0000 7f820ca50640 1 --2- 172.21.15.7:0/2542833302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8208102490 0x7f820819f230 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f8200031ae0 tx=0x7f8200031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:13.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.874+0000 7f82057fa640 1 -- 172.21.15.7:0/2542833302 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8200003c30 con 0x7f8208102490 2024-06-10T01:32:13.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.874+0000 7f820e4b3640 1 -- 172.21.15.7:0/2542833302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f820819fb60 con 0x7f8208102490 2024-06-10T01:32:13.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.874+0000 7f82057fa640 1 -- 172.21.15.7:0/2542833302 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8200003dd0 con 0x7f8208102490 2024-06-10T01:32:13.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.874+0000 7f82057fa640 1 -- 172.21.15.7:0/2542833302 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820002fcb0 con 0x7f8208102490 2024-06-10T01:32:13.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.874+0000 7f820e4b3640 1 -- 172.21.15.7:0/2542833302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82081a00b0 con 0x7f8208102490 2024-06-10T01:32:13.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.875+0000 7f820e4b3640 1 -- 172.21.15.7:0/2542833302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82081a0340 con 0x7f8208102490 2024-06-10T01:32:13.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.877+0000 7f82057fa640 1 -- 172.21.15.7:0/2542833302 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f8200004050 con 0x7f8208102490 2024-06-10T01:32:13.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.877+0000 7f82057fa640 1 --2- 172.21.15.7:0/2542833302 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f81f00a2b90 0x7f81f00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:13.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.877+0000 7f82057fa640 1 -- 172.21.15.7:0/2542833302 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f82000f07e0 con 0x7f8208102490 2024-06-10T01:32:13.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.877+0000 7f8207fff640 1 --2- 172.21.15.7:0/2542833302 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f81f00a2b90 0x7f81f00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:13.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.878+0000 7f8207fff640 1 --2- 172.21.15.7:0/2542833302 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f81f00a2b90 0x7f81f00a5050 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f8208198b60 tx=0x7f81f800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:13.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:13.881+0000 7f82057fa640 1 -- 172.21.15.7:0/2542833302 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f82000bd450 con 0x7f8208102490 2024-06-10T01:32:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.067+0000 7f820e4b3640 1 -- 172.21.15.7:0/2542833302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f82081a0510 con 0x7f8208102490 2024-06-10T01:32:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.067+0000 7f82057fa640 1 -- 172.21.15.7:0/2542833302 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v102) ==== 103+0+11 (secure 0 0 0) 0x7f82000bcd50 con 0x7f8208102490 2024-06-10T01:32:14.060 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-06-10T01:32:14.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.070+0000 7f820e4b3640 1 -- 172.21.15.7:0/2542833302 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f81f00a2b90 msgr2=0x7f81f00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:14.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.070+0000 7f820e4b3640 1 --2- 172.21.15.7:0/2542833302 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f81f00a2b90 0x7f81f00a5050 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f8208198b60 tx=0x7f81f800b390 comp rx=0 tx=0).stop 2024-06-10T01:32:14.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.070+0000 7f820e4b3640 1 -- 172.21.15.7:0/2542833302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8208102490 msgr2=0x7f820819f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:14.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.070+0000 7f820e4b3640 1 --2- 172.21.15.7:0/2542833302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8208102490 0x7f820819f230 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f8200031ae0 tx=0x7f8200031ee0 comp rx=0 tx=0).stop 2024-06-10T01:32:14.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.070+0000 7f820e4b3640 1 -- 172.21.15.7:0/2542833302 shutdown_connections 2024-06-10T01:32:14.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.070+0000 7f820e4b3640 1 --2- 172.21.15.7:0/2542833302 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f81f00a2b90 0x7f81f00a5050 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:14.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.070+0000 7f820e4b3640 1 --2- 172.21.15.7:0/2542833302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8208102490 0x7f820819f230 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:14.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.070+0000 7f820e4b3640 1 --2- 172.21.15.7:0/2542833302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8208101ad0 0x7f82081981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:14.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.070+0000 7f820e4b3640 1 --2- 172.21.15.7:0/2542833302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82081008e0 0x7f8208197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:14.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.070+0000 7f820e4b3640 1 -- 172.21.15.7:0/2542833302 >> 172.21.15.7:0/2542833302 conn(0x7f82080fbe70 msgr2=0x7f82080fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:14.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.070+0000 7f820e4b3640 1 -- 172.21.15.7:0/2542833302 shutdown_connections 2024-06-10T01:32:14.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.070+0000 7f820e4b3640 1 -- 172.21.15.7:0/2542833302 wait complete. 2024-06-10T01:32:14.073 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-06-10T01:32:14.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.186+0000 7fec1569a640 1 -- 172.21.15.7:0/1984486245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec101076e0 msgr2=0x7fec10109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.186+0000 7fec1569a640 1 --2- 172.21.15.7:0/1984486245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec101076e0 0x7fec10109b30 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fec08009900 tx=0x7fec0802a410 comp rx=0 tx=0).stop 2024-06-10T01:32:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.186+0000 7fec1569a640 1 -- 172.21.15.7:0/1984486245 shutdown_connections 2024-06-10T01:32:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.186+0000 7fec1569a640 1 --2- 172.21.15.7:0/1984486245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec101076e0 0x7fec10109b30 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.186+0000 7fec1569a640 1 --2- 172.21.15.7:0/1984486245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec10104db0 0x7fec101071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.186+0000 7fec1569a640 1 --2- 172.21.15.7:0/1984486245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec10102480 0x7fec10104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.186+0000 7fec1569a640 1 -- 172.21.15.7:0/1984486245 >> 172.21.15.7:0/1984486245 conn(0x7fec100fbe70 msgr2=0x7fec100fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.186+0000 7fec1569a640 1 -- 172.21.15.7:0/1984486245 shutdown_connections 2024-06-10T01:32:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.186+0000 7fec1569a640 1 -- 172.21.15.7:0/1984486245 wait complete. 2024-06-10T01:32:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.187+0000 7fec1569a640 1 Processor -- start 2024-06-10T01:32:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.187+0000 7fec1569a640 1 -- start start 2024-06-10T01:32:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.187+0000 7fec1569a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec10102480 0x7fec10197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.187+0000 7fec1569a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec10104db0 0x7fec101981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:14.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.187+0000 7fec1569a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec101076e0 0x7fec1019f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:14.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.187+0000 7fec1569a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fec1010d6a0 con 0x7fec10104db0 2024-06-10T01:32:14.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.187+0000 7fec1569a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fec1010d520 con 0x7fec101076e0 2024-06-10T01:32:14.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.187+0000 7fec1569a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fec1010d820 con 0x7fec10102480 2024-06-10T01:32:14.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.187+0000 7fec0e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec10104db0 0x7fec101981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:14.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.187+0000 7fec0e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec10104db0 0x7fec101981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45838/0 (socket says 172.21.15.7:45838) 2024-06-10T01:32:14.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.187+0000 7fec0e7fc640 1 -- 172.21.15.7:0/548895528 learned_addr learned my addr 172.21.15.7:0/548895528 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:32:14.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.188+0000 7fec0f7fe640 1 --2- 172.21.15.7:0/548895528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec101076e0 0x7fec1019f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:14.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.188+0000 7fec0effd640 1 --2- 172.21.15.7:0/548895528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec10102480 0x7fec10197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:14.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.188+0000 7fec0e7fc640 1 -- 172.21.15.7:0/548895528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec10102480 msgr2=0x7fec10197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:14.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.188+0000 7fec0e7fc640 1 --2- 172.21.15.7:0/548895528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec10102480 0x7fec10197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:14.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.188+0000 7fec0e7fc640 1 -- 172.21.15.7:0/548895528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec101076e0 msgr2=0x7fec1019f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:14.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.188+0000 7fec0e7fc640 1 --2- 172.21.15.7:0/548895528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec101076e0 0x7fec1019f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:14.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.188+0000 7fec0e7fc640 1 -- 172.21.15.7:0/548895528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec1019f8d0 con 0x7fec10104db0 2024-06-10T01:32:14.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.188+0000 7fec0e7fc640 1 --2- 172.21.15.7:0/548895528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec10104db0 0x7fec101981d0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7febf800e9f0 tx=0x7febf800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:14.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.188+0000 7fec07fff640 1 -- 172.21.15.7:0/548895528 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf800ccc0 con 0x7fec10104db0 2024-06-10T01:32:14.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.188+0000 7fec1569a640 1 -- 172.21.15.7:0/548895528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec1019fbc0 con 0x7fec10104db0 2024-06-10T01:32:14.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.188+0000 7fec07fff640 1 -- 172.21.15.7:0/548895528 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febf8005500 con 0x7fec10104db0 2024-06-10T01:32:14.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.188+0000 7fec07fff640 1 -- 172.21.15.7:0/548895528 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf8010690 con 0x7fec10104db0 2024-06-10T01:32:14.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.188+0000 7fec1569a640 1 -- 172.21.15.7:0/548895528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec101a0110 con 0x7fec10104db0 2024-06-10T01:32:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.189+0000 7fec1569a640 1 -- 172.21.15.7:0/548895528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec100fd960 con 0x7fec10104db0 2024-06-10T01:32:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.190+0000 7fec07fff640 1 -- 172.21.15.7:0/548895528 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7febf80046c0 con 0x7fec10104db0 2024-06-10T01:32:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.191+0000 7fec07fff640 1 --2- 172.21.15.7:0/548895528 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7febfc0a2b90 0x7febfc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.191+0000 7fec0effd640 1 --2- 172.21.15.7:0/548895528 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7febfc0a2b90 0x7febfc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.191+0000 7fec07fff640 1 -- 172.21.15.7:0/548895528 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7febf80cf7e0 con 0x7fec10104db0 2024-06-10T01:32:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.192+0000 7fec0effd640 1 --2- 172.21.15.7:0/548895528 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7febfc0a2b90 0x7febfc0a5050 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fec10198b60 tx=0x7fec00009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.195+0000 7fec07fff640 1 -- 172.21.15.7:0/548895528 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7febf809c3c0 con 0x7fec10104db0 2024-06-10T01:32:14.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.431+0000 7fec1569a640 1 -- 172.21.15.7:0/548895528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fec100625a0 con 0x7fec10104db0 2024-06-10T01:32:14.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.440+0000 7fec07fff640 1 -- 172.21.15.7:0/548895528 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+740361 (secure 0 0 0) 0x7febf809bcc0 con 0x7fec10104db0 2024-06-10T01:32:14.442 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:32:14.442 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:32:14.442 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 18, 2024-06-10T01:32:14.443 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.443 INFO:teuthology.orchestra.run.smithi007.stdout: "active_gid": 14152, 2024-06-10T01:32:14.443 INFO:teuthology.orchestra.run.smithi007.stdout: "active_name": "x", 2024-06-10T01:32:14.443 INFO:teuthology.orchestra.run.smithi007.stdout: "active_addrs": { 2024-06-10T01:32:14.443 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-06-10T01:32:14.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.443 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-06-10T01:32:14.443 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-06-10T01:32:14.443 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1804801756 2024-06-10T01:32:14.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.443 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-06-10T01:32:14.443 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-06-10T01:32:14.444 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1804801756 2024-06-10T01:32:14.444 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.444 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-06-10T01:32:14.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.444 INFO:teuthology.orchestra.run.smithi007.stdout: "active_addr": "172.21.15.7:6801/1804801756", 2024-06-10T01:32:14.444 INFO:teuthology.orchestra.run.smithi007.stdout: "active_change": "2024-06-10T00:54:35.441249+0000", 2024-06-10T01:32:14.444 INFO:teuthology.orchestra.run.smithi007.stdout: "active_mgr_features": 4540701547738038271, 2024-06-10T01:32:14.444 INFO:teuthology.orchestra.run.smithi007.stdout: "available": true, 2024-06-10T01:32:14.444 INFO:teuthology.orchestra.run.smithi007.stdout: "standbys": [ 2024-06-10T01:32:14.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.444 INFO:teuthology.orchestra.run.smithi007.stdout: "gid": 24107, 2024-06-10T01:32:14.444 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "y", 2024-06-10T01:32:14.445 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_features": 4540701547738038271, 2024-06-10T01:32:14.445 INFO:teuthology.orchestra.run.smithi007.stdout: "available_modules": [ 2024-06-10T01:32:14.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.445 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "alerts", 2024-06-10T01:32:14.445 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.445 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.445 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.445 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-06-10T01:32:14.445 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-06-10T01:32:14.445 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:14.445 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.445 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.445 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-06-10T01:32:14.446 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.446 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.446 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.446 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How frequently to reexamine health status", 2024-06-10T01:32:14.446 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.446 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.446 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.446 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.446 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.446 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.446 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.446 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.447 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.447 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.447 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.447 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.447 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.447 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.447 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.447 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.447 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.447 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.447 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.447 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.447 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.448 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.448 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.448 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.448 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.448 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.448 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.448 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.448 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.448 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.448 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.448 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.448 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.449 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.449 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.449 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.449 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.449 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.449 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.449 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.449 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.449 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.449 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.449 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.449 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.450 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.450 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.450 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.450 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.450 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.450 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.450 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.450 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.450 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.450 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.450 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.450 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.451 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.451 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.451 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.451 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.451 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.451 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.451 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.451 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.451 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.451 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.451 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.451 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.451 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_destination": { 2024-06-10T01:32:14.452 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_destination", 2024-06-10T01:32:14.452 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.452 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.452 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.452 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.452 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.452 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.452 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.452 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email address to send alerts to", 2024-06-10T01:32:14.452 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.452 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.452 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.453 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_from_name": { 2024-06-10T01:32:14.453 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_from_name", 2024-06-10T01:32:14.453 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.453 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.453 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.453 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "Ceph", 2024-06-10T01:32:14.453 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.453 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.453 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.453 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email From: name", 2024-06-10T01:32:14.453 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.453 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.453 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.454 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_host": { 2024-06-10T01:32:14.454 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_host", 2024-06-10T01:32:14.454 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.454 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.454 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.454 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.454 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.454 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.454 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.454 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP server", 2024-06-10T01:32:14.454 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.454 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.454 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.455 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_password": { 2024-06-10T01:32:14.455 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_password", 2024-06-10T01:32:14.455 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.455 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.455 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.455 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.455 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.455 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.455 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.455 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Password to authenticate with", 2024-06-10T01:32:14.455 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.455 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.455 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.456 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_port": { 2024-06-10T01:32:14.456 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_port", 2024-06-10T01:32:14.456 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.456 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.456 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.456 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "465", 2024-06-10T01:32:14.456 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.456 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.456 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.456 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP port", 2024-06-10T01:32:14.456 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.456 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.456 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.457 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_sender": { 2024-06-10T01:32:14.457 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_sender", 2024-06-10T01:32:14.457 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.457 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.457 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.457 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.457 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.457 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.457 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.457 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP envelope sender", 2024-06-10T01:32:14.457 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.457 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.458 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.458 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_ssl": { 2024-06-10T01:32:14.458 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_ssl", 2024-06-10T01:32:14.458 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.458 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.458 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.458 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.458 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.458 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.458 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.458 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-10T01:32:14.458 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.459 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.459 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.459 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_user": { 2024-06-10T01:32:14.459 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_user", 2024-06-10T01:32:14.459 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.459 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.459 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.459 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.459 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.459 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.459 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.459 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "User to authenticate as", 2024-06-10T01:32:14.460 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.460 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.460 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.460 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.460 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.460 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "balancer", 2024-06-10T01:32:14.460 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.460 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.460 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.460 INFO:teuthology.orchestra.run.smithi007.stdout: "active": { 2024-06-10T01:32:14.460 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "active", 2024-06-10T01:32:14.460 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.460 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.461 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.461 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.461 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.461 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.461 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.461 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically balance PGs across cluster", 2024-06-10T01:32:14.461 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.461 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.461 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.461 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_time": { 2024-06-10T01:32:14.461 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_time", 2024-06-10T01:32:14.461 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.461 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.462 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.462 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0000", 2024-06-10T01:32:14.462 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.462 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.462 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.462 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "beginning time of day to automatically balance", 2024-06-10T01:32:14.462 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-10T01:32:14.462 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.462 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.462 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_weekday": { 2024-06-10T01:32:14.462 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_weekday", 2024-06-10T01:32:14.462 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-06-10T01:32:14.463 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.463 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.463 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-06-10T01:32:14.463 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-06-10T01:32:14.463 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-06-10T01:32:14.463 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.463 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-10T01:32:14.463 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-10T01:32:14.463 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.463 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.463 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_max_iterations": { 2024-06-10T01:32:14.463 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_max_iterations", 2024-06-10T01:32:14.463 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-06-10T01:32:14.463 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.464 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.464 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "25", 2024-06-10T01:32:14.464 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-06-10T01:32:14.464 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "250", 2024-06-10T01:32:14.464 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.464 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-10T01:32:14.464 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.464 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.464 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.464 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_metrics": { 2024-06-10T01:32:14.464 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_metrics", 2024-06-10T01:32:14.464 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.464 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.464 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.465 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "pgs,objects,bytes", 2024-06-10T01:32:14.465 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.465 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.465 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.465 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-10T01:32:14.465 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-10T01:32:14.465 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.465 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.465 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_step": { 2024-06-10T01:32:14.465 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_step", 2024-06-10T01:32:14.465 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-10T01:32:14.465 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.466 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.466 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.5", 2024-06-10T01:32:14.466 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0.001", 2024-06-10T01:32:14.466 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "0.999", 2024-06-10T01:32:14.466 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.466 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "aggressiveness of optimization", 2024-06-10T01:32:14.466 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-10T01:32:14.466 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.466 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.466 INFO:teuthology.orchestra.run.smithi007.stdout: "end_time": { 2024-06-10T01:32:14.466 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_time", 2024-06-10T01:32:14.466 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.466 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.466 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.467 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2359", 2024-06-10T01:32:14.467 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.467 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.467 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.467 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ending time of day to automatically balance", 2024-06-10T01:32:14.467 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-10T01:32:14.467 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.467 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.467 INFO:teuthology.orchestra.run.smithi007.stdout: "end_weekday": { 2024-06-10T01:32:14.467 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_weekday", 2024-06-10T01:32:14.467 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-06-10T01:32:14.467 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.467 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.468 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-06-10T01:32:14.468 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-06-10T01:32:14.468 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-06-10T01:32:14.468 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.468 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-10T01:32:14.468 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-10T01:32:14.468 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.468 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.468 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.468 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.468 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.468 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.468 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.468 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.469 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.469 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.469 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.469 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.469 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.469 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.469 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.469 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.469 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.469 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.469 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.469 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.469 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.469 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.470 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.470 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.470 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.470 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.470 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.470 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.470 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.470 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.470 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.470 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.470 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.470 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.470 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.471 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.471 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.471 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.471 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.471 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.471 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.471 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.471 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.471 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.471 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.471 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.471 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.471 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.471 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.472 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.472 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.472 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.472 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.472 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.472 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.472 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.472 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.472 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.472 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.472 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.472 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.472 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.473 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.473 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.473 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.473 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.473 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.473 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.473 INFO:teuthology.orchestra.run.smithi007.stdout: "min_score": { 2024-06-10T01:32:14.473 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_score", 2024-06-10T01:32:14.473 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-10T01:32:14.473 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.473 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.473 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-06-10T01:32:14.474 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.474 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.474 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.474 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-10T01:32:14.474 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.474 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.474 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.474 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-06-10T01:32:14.474 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-06-10T01:32:14.474 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.474 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.474 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.475 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "upmap", 2024-06-10T01:32:14.475 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.475 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.475 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.475 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-compat", 2024-06-10T01:32:14.475 INFO:teuthology.orchestra.run.smithi007.stdout: "none", 2024-06-10T01:32:14.475 INFO:teuthology.orchestra.run.smithi007.stdout: "read", 2024-06-10T01:32:14.475 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap", 2024-06-10T01:32:14.475 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap-read" 2024-06-10T01:32:14.475 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.475 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Balancer mode", 2024-06-10T01:32:14.475 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.475 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.475 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.476 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_ids": { 2024-06-10T01:32:14.476 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_ids", 2024-06-10T01:32:14.476 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.476 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.476 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.476 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.476 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.476 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.476 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.476 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-10T01:32:14.476 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.476 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.476 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.477 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-06-10T01:32:14.477 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-06-10T01:32:14.477 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:14.477 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.477 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.477 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-06-10T01:32:14.477 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.477 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.477 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.477 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-10T01:32:14.477 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.477 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.477 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.478 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_deviation": { 2024-06-10T01:32:14.478 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_deviation", 2024-06-10T01:32:14.478 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.478 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.478 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.478 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-06-10T01:32:14.478 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-06-10T01:32:14.478 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.478 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.478 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-10T01:32:14.478 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-10T01:32:14.478 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.478 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.479 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_optimizations": { 2024-06-10T01:32:14.479 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_optimizations", 2024-06-10T01:32:14.479 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-06-10T01:32:14.479 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.479 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.479 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-10T01:32:14.479 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.479 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.479 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.479 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-10T01:32:14.479 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.479 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.479 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.480 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm", 2024-06-10T01:32:14.480 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.480 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.480 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.480 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_down_multiplier": { 2024-06-10T01:32:14.480 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_down_multiplier", 2024-06-10T01:32:14.480 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-10T01:32:14.481 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.481 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.481 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-06-10T01:32:14.481 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.481 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.481 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.481 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-10T01:32:14.481 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.481 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.481 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.481 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_refresh_rate": { 2024-06-10T01:32:14.481 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_refresh_rate", 2024-06-10T01:32:14.482 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:14.482 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.482 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.482 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "20", 2024-06-10T01:32:14.482 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.482 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.482 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.482 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-10T01:32:14.482 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.482 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.482 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.482 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_starting_port": { 2024-06-10T01:32:14.483 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_starting_port", 2024-06-10T01:32:14.483 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.483 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.483 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.483 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4721", 2024-06-10T01:32:14.483 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.483 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.483 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.483 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-10T01:32:14.483 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.483 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.483 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.484 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_ptrace": { 2024-06-10T01:32:14.484 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_ptrace", 2024-06-10T01:32:14.484 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.484 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.484 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.484 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.484 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.484 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.484 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.484 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-10T01:32:14.484 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-10T01:32:14.484 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.484 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.485 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_interval": { 2024-06-10T01:32:14.485 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_interval", 2024-06-10T01:32:14.485 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:14.485 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.485 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.485 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-06-10T01:32:14.485 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.485 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.485 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.485 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-10T01:32:14.485 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.485 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.485 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.486 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_memory_target_ratio": { 2024-06-10T01:32:14.486 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_memory_target_ratio", 2024-06-10T01:32:14.486 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-10T01:32:14.486 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.486 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.486 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.7", 2024-06-10T01:32:14.486 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.486 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.486 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.486 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-10T01:32:14.486 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.486 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.486 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.487 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm_log_destination": { 2024-06-10T01:32:14.487 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm_log_destination", 2024-06-10T01:32:14.487 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.487 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.487 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.487 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.487 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.487 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.487 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.487 INFO:teuthology.orchestra.run.smithi007.stdout: "file", 2024-06-10T01:32:14.487 INFO:teuthology.orchestra.run.smithi007.stdout: "file,syslog", 2024-06-10T01:32:14.487 INFO:teuthology.orchestra.run.smithi007.stdout: "syslog" 2024-06-10T01:32:14.487 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.488 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-10T01:32:14.488 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.488 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.488 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.488 INFO:teuthology.orchestra.run.smithi007.stdout: "cgroups_split": { 2024-06-10T01:32:14.488 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cgroups_split", 2024-06-10T01:32:14.488 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.488 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.488 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.488 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.488 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.488 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.488 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.488 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-10T01:32:14.489 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.489 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.489 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.489 INFO:teuthology.orchestra.run.smithi007.stdout: "config_checks_enabled": { 2024-06-10T01:32:14.489 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_checks_enabled", 2024-06-10T01:32:14.489 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.489 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.489 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.489 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.489 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.489 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.489 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.489 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-10T01:32:14.490 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.490 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.490 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.490 INFO:teuthology.orchestra.run.smithi007.stdout: "config_dashboard": { 2024-06-10T01:32:14.490 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_dashboard", 2024-06-10T01:32:14.490 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.490 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.490 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.490 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.490 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.490 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.490 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.491 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-10T01:32:14.491 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.491 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.491 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.491 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_alertmanager": { 2024-06-10T01:32:14.491 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_alertmanager", 2024-06-10T01:32:14.491 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.491 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.491 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.491 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-10T01:32:14.491 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.491 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.491 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.491 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-06-10T01:32:14.492 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.492 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.492 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.492 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_base": { 2024-06-10T01:32:14.492 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_base", 2024-06-10T01:32:14.492 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.492 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.492 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.492 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-10T01:32:14.492 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.492 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.492 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.492 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Container image name, without the tag", 2024-06-10T01:32:14.492 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.493 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.493 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.493 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_elasticsearch": { 2024-06-10T01:32:14.493 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_elasticsearch", 2024-06-10T01:32:14.493 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.493 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.493 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.493 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-10T01:32:14.493 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.493 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.493 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.493 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "elasticsearch container image", 2024-06-10T01:32:14.493 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.493 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.494 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.494 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_grafana": { 2024-06-10T01:32:14.494 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_grafana", 2024-06-10T01:32:14.494 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.494 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.494 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.494 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-10T01:32:14.494 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.494 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.494 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.494 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-06-10T01:32:14.494 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.494 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.495 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.495 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_haproxy": { 2024-06-10T01:32:14.495 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_haproxy", 2024-06-10T01:32:14.495 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.495 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.495 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.495 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-10T01:32:14.495 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.495 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.495 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.495 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "HAproxy container image", 2024-06-10T01:32:14.495 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.495 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.496 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.496 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_agent": { 2024-06-10T01:32:14.496 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_agent", 2024-06-10T01:32:14.496 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.496 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.496 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.496 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-10T01:32:14.496 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.496 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.496 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.496 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger agent container image", 2024-06-10T01:32:14.496 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.496 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.497 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.497 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_collector": { 2024-06-10T01:32:14.497 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_collector", 2024-06-10T01:32:14.497 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.497 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.497 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.497 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-10T01:32:14.497 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.497 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.497 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.497 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger collector container image", 2024-06-10T01:32:14.497 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.497 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.498 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.498 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_query": { 2024-06-10T01:32:14.498 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_query", 2024-06-10T01:32:14.498 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.498 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.498 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.498 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-10T01:32:14.498 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.498 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.498 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.498 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger query container image", 2024-06-10T01:32:14.499 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.499 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.499 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.499 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_keepalived": { 2024-06-10T01:32:14.499 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_keepalived", 2024-06-10T01:32:14.499 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.499 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.499 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.499 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-10T01:32:14.499 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.499 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.499 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.500 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Keepalived container image", 2024-06-10T01:32:14.500 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.500 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.500 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.500 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_loki": { 2024-06-10T01:32:14.500 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_loki", 2024-06-10T01:32:14.500 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.500 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.500 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.500 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-10T01:32:14.500 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.500 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.501 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.501 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Loki container image", 2024-06-10T01:32:14.501 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.501 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.501 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.501 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_node_exporter": { 2024-06-10T01:32:14.501 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_node_exporter", 2024-06-10T01:32:14.501 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.501 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.501 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.501 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-10T01:32:14.501 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.501 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.502 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.502 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-06-10T01:32:14.502 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.502 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.502 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.502 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_nvmeof": { 2024-06-10T01:32:14.502 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_nvmeof", 2024-06-10T01:32:14.502 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.502 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.502 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.502 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-10T01:32:14.503 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.503 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.503 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.503 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Nvme-of container image", 2024-06-10T01:32:14.503 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.503 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.503 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.503 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_prometheus": { 2024-06-10T01:32:14.503 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_prometheus", 2024-06-10T01:32:14.503 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.503 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.503 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.504 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-10T01:32:14.504 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.504 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.504 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.504 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-06-10T01:32:14.504 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.504 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.504 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.504 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_promtail": { 2024-06-10T01:32:14.504 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_promtail", 2024-06-10T01:32:14.504 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.504 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.504 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.505 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-10T01:32:14.505 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.505 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.505 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.505 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Promtail container image", 2024-06-10T01:32:14.505 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.505 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.505 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.505 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_samba": { 2024-06-10T01:32:14.505 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_samba", 2024-06-10T01:32:14.505 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.505 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.505 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.506 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-10T01:32:14.506 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.506 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.506 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.506 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Samba/SMB container image", 2024-06-10T01:32:14.506 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.506 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.506 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.506 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_snmp_gateway": { 2024-06-10T01:32:14.506 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_snmp_gateway", 2024-06-10T01:32:14.506 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.506 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.507 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.507 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-10T01:32:14.507 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.507 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.507 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.507 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SNMP Gateway container image", 2024-06-10T01:32:14.507 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.507 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.507 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.507 INFO:teuthology.orchestra.run.smithi007.stdout: "container_init": { 2024-06-10T01:32:14.507 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_init", 2024-06-10T01:32:14.507 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.507 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.508 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.508 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.508 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.508 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.508 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.508 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Run podman/docker with `--init`", 2024-06-10T01:32:14.508 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.508 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.508 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.508 INFO:teuthology.orchestra.run.smithi007.stdout: "daemon_cache_timeout": { 2024-06-10T01:32:14.508 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "daemon_cache_timeout", 2024-06-10T01:32:14.509 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:14.509 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.509 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.509 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-06-10T01:32:14.509 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.509 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.509 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.509 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-10T01:32:14.509 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.509 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.509 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.509 INFO:teuthology.orchestra.run.smithi007.stdout: "default_cephadm_command_timeout": { 2024-06-10T01:32:14.509 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_cephadm_command_timeout", 2024-06-10T01:32:14.509 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.510 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.510 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.510 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "900", 2024-06-10T01:32:14.510 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.510 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.510 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.510 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-10T01:32:14.510 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.510 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.510 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.510 INFO:teuthology.orchestra.run.smithi007.stdout: "default_registry": { 2024-06-10T01:32:14.510 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_registry", 2024-06-10T01:32:14.511 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.511 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.511 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.511 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io", 2024-06-10T01:32:14.511 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.511 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.511 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.511 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-10T01:32:14.511 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.511 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.511 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.511 INFO:teuthology.orchestra.run.smithi007.stdout: "device_cache_timeout": { 2024-06-10T01:32:14.512 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_cache_timeout", 2024-06-10T01:32:14.512 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:14.512 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.512 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.512 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1800", 2024-06-10T01:32:14.512 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.512 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.512 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.512 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache device inventory", 2024-06-10T01:32:14.512 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.512 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.512 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.512 INFO:teuthology.orchestra.run.smithi007.stdout: "device_enhanced_scan": { 2024-06-10T01:32:14.512 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_enhanced_scan", 2024-06-10T01:32:14.513 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.513 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.513 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.513 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.513 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.513 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.513 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.513 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-10T01:32:14.513 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.513 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.513 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.513 INFO:teuthology.orchestra.run.smithi007.stdout: "facts_cache_timeout": { 2024-06-10T01:32:14.513 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "facts_cache_timeout", 2024-06-10T01:32:14.513 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:14.514 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.514 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.514 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-06-10T01:32:14.514 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.514 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.514 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.514 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache host facts data", 2024-06-10T01:32:14.514 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.514 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.514 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.514 INFO:teuthology.orchestra.run.smithi007.stdout: "grafana_dashboards_path": { 2024-06-10T01:32:14.514 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "grafana_dashboards_path", 2024-06-10T01:32:14.514 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.514 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.515 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.515 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-10T01:32:14.515 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.515 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.515 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.515 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-10T01:32:14.515 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.515 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.515 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.515 INFO:teuthology.orchestra.run.smithi007.stdout: "host_check_interval": { 2024-06-10T01:32:14.515 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "host_check_interval", 2024-06-10T01:32:14.515 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:14.515 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.515 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.516 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-06-10T01:32:14.516 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.516 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.516 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.516 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to perform a host check", 2024-06-10T01:32:14.516 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.516 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.516 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.516 INFO:teuthology.orchestra.run.smithi007.stdout: "hw_monitoring": { 2024-06-10T01:32:14.516 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hw_monitoring", 2024-06-10T01:32:14.516 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.516 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.516 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.517 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.517 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.517 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.517 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.517 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-10T01:32:14.517 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.517 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.517 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.517 INFO:teuthology.orchestra.run.smithi007.stdout: "inventory_list_all": { 2024-06-10T01:32:14.517 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "inventory_list_all", 2024-06-10T01:32:14.517 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.517 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.517 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.517 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.518 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.518 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.518 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.518 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-10T01:32:14.518 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.518 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.518 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.518 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.518 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.518 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.518 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.518 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.518 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.518 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.519 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.519 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.519 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.519 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.519 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.519 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.519 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.519 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.519 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.519 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.519 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.519 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.519 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.519 INFO:teuthology.orchestra.run.smithi007.stdout: "log_refresh_metadata": { 2024-06-10T01:32:14.520 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_refresh_metadata", 2024-06-10T01:32:14.520 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.520 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.520 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.520 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.520 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.520 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.520 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.520 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-10T01:32:14.520 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.520 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.520 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.521 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.521 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.521 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.521 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.521 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.521 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.521 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.521 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.521 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.521 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-10T01:32:14.521 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.521 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.521 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.522 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.522 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.522 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.522 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.522 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.522 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.522 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.522 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.522 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.522 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.522 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.522 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.523 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.523 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.523 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.523 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.523 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.523 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.523 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.523 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.523 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.523 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.523 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.523 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.523 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.524 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.524 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.524 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.524 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.524 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.524 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.524 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.524 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.524 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-10T01:32:14.524 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-10T01:32:14.524 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.524 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.524 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.524 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.525 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.525 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.525 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.525 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-10T01:32:14.525 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.525 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.525 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.525 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-10T01:32:14.525 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-10T01:32:14.525 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.525 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.525 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.526 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "*", 2024-06-10T01:32:14.526 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.526 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.526 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.526 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-10T01:32:14.526 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.526 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.526 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.526 INFO:teuthology.orchestra.run.smithi007.stdout: "max_count_per_host": { 2024-06-10T01:32:14.526 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_count_per_host", 2024-06-10T01:32:14.526 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.526 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.526 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.527 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-10T01:32:14.527 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.527 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.527 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.527 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of daemons per service per host", 2024-06-10T01:32:14.527 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.527 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.527 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.527 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd_draining_count": { 2024-06-10T01:32:14.527 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_osd_draining_count", 2024-06-10T01:32:14.527 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.527 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.527 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.528 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-10T01:32:14.528 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.528 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.528 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.528 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-10T01:32:14.528 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.528 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.528 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.528 INFO:teuthology.orchestra.run.smithi007.stdout: "migration_current": { 2024-06-10T01:32:14.528 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "migration_current", 2024-06-10T01:32:14.528 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.528 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.528 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.529 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.529 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.529 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.529 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.529 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "internal - do not modify", 2024-06-10T01:32:14.529 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.529 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.529 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.529 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-06-10T01:32:14.529 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-06-10T01:32:14.529 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.529 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.529 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.530 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "root", 2024-06-10T01:32:14.530 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.530 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.530 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.530 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm-package", 2024-06-10T01:32:14.530 INFO:teuthology.orchestra.run.smithi007.stdout: "root" 2024-06-10T01:32:14.530 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.530 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "mode for remote execution of cephadm", 2024-06-10T01:32:14.530 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.530 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.530 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.530 INFO:teuthology.orchestra.run.smithi007.stdout: "oob_default_addr": { 2024-06-10T01:32:14.531 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "oob_default_addr", 2024-06-10T01:32:14.531 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.531 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.531 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.531 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "169.254.1.1", 2024-06-10T01:32:14.531 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.531 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.531 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.531 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-10T01:32:14.531 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.531 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.531 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.531 INFO:teuthology.orchestra.run.smithi007.stdout: "prometheus_alerts_path": { 2024-06-10T01:32:14.532 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus_alerts_path", 2024-06-10T01:32:14.532 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.532 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.532 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.532 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-10T01:32:14.532 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.532 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.532 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.532 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-10T01:32:14.532 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.532 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.532 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.532 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_insecure": { 2024-06-10T01:32:14.533 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_insecure", 2024-06-10T01:32:14.533 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.533 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.533 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.533 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.533 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.533 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.533 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.533 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-10T01:32:14.533 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.533 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.533 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.533 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_password": { 2024-06-10T01:32:14.533 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_password", 2024-06-10T01:32:14.534 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.534 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.534 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.534 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.534 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.534 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.534 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.534 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-10T01:32:14.534 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.534 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.534 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.535 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_url": { 2024-06-10T01:32:14.535 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_url", 2024-06-10T01:32:14.535 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.535 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.535 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.535 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.535 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.535 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.535 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.535 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-10T01:32:14.535 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.535 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.535 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.536 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_username": { 2024-06-10T01:32:14.536 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_username", 2024-06-10T01:32:14.536 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.536 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.536 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.536 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.536 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.536 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.536 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.536 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-10T01:32:14.536 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.536 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.536 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.537 INFO:teuthology.orchestra.run.smithi007.stdout: "secure_monitoring_stack": { 2024-06-10T01:32:14.537 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "secure_monitoring_stack", 2024-06-10T01:32:14.537 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.537 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.537 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.537 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.537 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.537 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.537 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.537 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-10T01:32:14.537 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.537 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.537 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.538 INFO:teuthology.orchestra.run.smithi007.stdout: "service_discovery_port": { 2024-06-10T01:32:14.538 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "service_discovery_port", 2024-06-10T01:32:14.538 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.538 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.538 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.538 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8765", 2024-06-10T01:32:14.538 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.538 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.538 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.538 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "cephadm service discovery port", 2024-06-10T01:32:14.538 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.538 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.539 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.539 INFO:teuthology.orchestra.run.smithi007.stdout: "ssh_config_file": { 2024-06-10T01:32:14.539 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssh_config_file", 2024-06-10T01:32:14.539 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.539 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.539 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.539 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.539 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.539 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.539 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.539 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-10T01:32:14.539 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.539 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.540 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.540 INFO:teuthology.orchestra.run.smithi007.stdout: "use_agent": { 2024-06-10T01:32:14.540 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_agent", 2024-06-10T01:32:14.540 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.540 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.540 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.540 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.540 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.540 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.540 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.540 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-10T01:32:14.540 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.540 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.541 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.541 INFO:teuthology.orchestra.run.smithi007.stdout: "use_repo_digest": { 2024-06-10T01:32:14.541 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_repo_digest", 2024-06-10T01:32:14.541 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.541 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.541 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.541 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.541 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.541 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.541 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.541 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-10T01:32:14.541 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.542 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.542 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.542 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_failed_host_check": { 2024-06-10T01:32:14.542 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_failed_host_check", 2024-06-10T01:32:14.542 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.542 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.542 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.542 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.542 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.542 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.542 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.543 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if the host check fails", 2024-06-10T01:32:14.543 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.543 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.543 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.543 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_daemons": { 2024-06-10T01:32:14.543 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_daemons", 2024-06-10T01:32:14.543 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.543 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.543 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.543 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.543 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.543 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.543 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.544 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-10T01:32:14.544 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.544 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.544 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.544 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_hosts": { 2024-06-10T01:32:14.544 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_hosts", 2024-06-10T01:32:14.544 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.544 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.544 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.544 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.544 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.544 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.544 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.545 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-10T01:32:14.545 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.545 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.545 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.545 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.545 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.545 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crash", 2024-06-10T01:32:14.545 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.545 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.545 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.546 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.546 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.546 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.546 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.546 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.546 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.546 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.546 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.546 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.546 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.546 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.546 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.547 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.547 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.547 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.547 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.547 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.547 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.547 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.547 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.547 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.547 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.547 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.547 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.548 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.548 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.548 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.548 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.548 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.548 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.548 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.548 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.548 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.548 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.548 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.548 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.548 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.549 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.549 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.549 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.549 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.549 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.549 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.549 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.549 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.549 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.549 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.549 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.549 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.550 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.550 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.550 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.550 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.550 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.550 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.550 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.550 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.550 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.550 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.550 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.550 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.551 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.551 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.551 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.551 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.551 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.551 INFO:teuthology.orchestra.run.smithi007.stdout: "retain_interval": { 2024-06-10T01:32:14.551 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retain_interval", 2024-06-10T01:32:14.551 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:14.551 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.551 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.551 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "31536000", 2024-06-10T01:32:14.551 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.552 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.552 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.552 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-10T01:32:14.552 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.552 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.552 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.552 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_recent_interval": { 2024-06-10T01:32:14.552 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_recent_interval", 2024-06-10T01:32:14.552 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:14.552 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.552 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.552 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1209600", 2024-06-10T01:32:14.553 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.553 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.553 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.553 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-10T01:32:14.553 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.553 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.553 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.553 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.553 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.553 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dashboard", 2024-06-10T01:32:14.553 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.554 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.554 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.554 INFO:teuthology.orchestra.run.smithi007.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-10T01:32:14.554 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-10T01:32:14.554 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.554 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.554 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.554 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-10T01:32:14.554 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.554 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.554 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.554 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.554 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.555 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.555 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.555 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_HOST": { 2024-06-10T01:32:14.555 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-10T01:32:14.555 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.555 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.555 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.555 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.555 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.555 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.555 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.555 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.555 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.556 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.556 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.556 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-10T01:32:14.556 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-10T01:32:14.556 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.556 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.556 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.556 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.556 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.556 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.556 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.556 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.556 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.557 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.557 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.557 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_ENABLED": { 2024-06-10T01:32:14.557 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_ENABLED", 2024-06-10T01:32:14.557 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.557 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.557 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.557 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.557 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.557 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.557 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.557 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.558 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.558 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.558 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.558 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-10T01:32:14.558 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-10T01:32:14.558 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.558 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.558 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.558 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.558 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.558 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.558 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.559 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.559 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.559 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.559 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.559 INFO:teuthology.orchestra.run.smithi007.stdout: "ENABLE_BROWSABLE_API": { 2024-06-10T01:32:14.559 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-10T01:32:14.559 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.559 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.559 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.559 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.559 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.559 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.559 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.560 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.560 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.560 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.560 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.560 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-10T01:32:14.560 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-10T01:32:14.560 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.560 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.560 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.560 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.560 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.560 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.561 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.561 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.561 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.561 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.561 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.561 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-10T01:32:14.561 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-10T01:32:14.561 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.561 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.561 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.561 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.561 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.561 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.562 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.562 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.562 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.562 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.562 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.562 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-10T01:32:14.562 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-10T01:32:14.562 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.562 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.562 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.562 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.562 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.563 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.563 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.563 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.563 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.563 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.563 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.563 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-10T01:32:14.563 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-10T01:32:14.563 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.563 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.563 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.563 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.564 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.564 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.564 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.564 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.564 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.564 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.564 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.564 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-10T01:32:14.564 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-10T01:32:14.564 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.564 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.564 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.564 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.565 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.565 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.565 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.565 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.565 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.565 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.565 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.565 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-10T01:32:14.565 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-10T01:32:14.565 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.565 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.565 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.565 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.566 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.566 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.566 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.566 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.566 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.566 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.566 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.566 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-10T01:32:14.566 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-10T01:32:14.566 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.566 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.566 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.567 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.567 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.567 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.567 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.567 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.567 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.567 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.567 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.567 INFO:teuthology.orchestra.run.smithi007.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-10T01:32:14.567 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-10T01:32:14.567 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.567 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.568 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.568 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.568 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.568 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.568 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.568 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.568 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.568 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.568 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.568 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_PASSWORD": { 2024-06-10T01:32:14.568 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-10T01:32:14.568 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.568 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.569 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.569 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-06-10T01:32:14.569 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.569 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.569 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.569 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.569 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.569 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.569 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.569 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-10T01:32:14.569 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-10T01:32:14.569 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.569 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.570 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.570 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.570 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.570 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.570 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.570 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.570 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.570 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.570 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.570 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_URL": { 2024-06-10T01:32:14.570 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_URL", 2024-06-10T01:32:14.570 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.571 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.571 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.571 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.571 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.571 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.571 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.571 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.571 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.571 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.571 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.571 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_USERNAME": { 2024-06-10T01:32:14.571 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-10T01:32:14.571 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.571 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.572 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.572 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-06-10T01:32:14.572 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.572 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.572 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.572 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.572 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.572 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.572 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.572 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-10T01:32:14.572 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-10T01:32:14.572 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.572 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.572 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.573 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.573 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.573 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.573 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.573 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.573 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.573 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.573 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.573 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-10T01:32:14.573 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-10T01:32:14.573 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.573 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.573 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.574 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.574 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.574 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.574 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.574 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.574 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.574 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.574 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.574 INFO:teuthology.orchestra.run.smithi007.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-10T01:32:14.574 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-10T01:32:14.574 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.575 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.575 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.575 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.575 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.575 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.575 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.575 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.575 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.575 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.575 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.575 INFO:teuthology.orchestra.run.smithi007.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-10T01:32:14.575 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-10T01:32:14.575 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.576 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.576 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.576 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.576 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.576 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.576 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.576 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.576 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.576 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.576 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.576 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_HOST": { 2024-06-10T01:32:14.576 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-10T01:32:14.576 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.576 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.577 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.577 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.577 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.577 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.577 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.577 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.577 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.577 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.577 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.577 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-10T01:32:14.577 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-10T01:32:14.577 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.577 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.577 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.578 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.578 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.578 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.578 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.578 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.578 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.578 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.578 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.578 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-10T01:32:14.578 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-10T01:32:14.578 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.579 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.579 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.579 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.579 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.579 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.579 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.579 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.579 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.579 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.579 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.579 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-10T01:32:14.579 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-10T01:32:14.579 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.580 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.580 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.580 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.580 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.580 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.580 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.580 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.580 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.580 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.580 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.580 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-10T01:32:14.580 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-10T01:32:14.580 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.580 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.581 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.581 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.581 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.581 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.581 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.581 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.581 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.581 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.581 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.581 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-10T01:32:14.581 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-10T01:32:14.581 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.581 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.582 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.582 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.582 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.582 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.582 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.582 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.582 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.582 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.582 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.582 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-10T01:32:14.582 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-10T01:32:14.582 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.582 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.583 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.583 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.583 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.583 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.583 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.583 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.583 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.583 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.583 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.583 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-10T01:32:14.583 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-10T01:32:14.583 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.583 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.583 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.584 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.584 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.584 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.584 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.584 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.584 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.584 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.584 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.584 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-10T01:32:14.584 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-10T01:32:14.584 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.584 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.584 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.584 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.585 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.585 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.585 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.585 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.585 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.585 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.585 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.585 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_ENABLED": { 2024-06-10T01:32:14.585 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-10T01:32:14.585 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.585 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.585 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.585 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.586 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.586 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.586 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.586 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.586 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.586 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.586 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.586 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-10T01:32:14.586 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-10T01:32:14.586 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.586 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.587 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.587 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-10T01:32:14.587 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.587 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.587 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.587 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.587 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.587 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.587 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.587 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-10T01:32:14.587 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-10T01:32:14.587 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.587 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.588 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.588 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-10T01:32:14.588 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.588 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.588 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.588 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.588 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.588 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.588 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.588 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-10T01:32:14.588 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-10T01:32:14.588 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.589 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.589 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.589 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8", 2024-06-10T01:32:14.589 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.589 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.589 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.589 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.589 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.589 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.589 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.589 INFO:teuthology.orchestra.run.smithi007.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-10T01:32:14.589 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-10T01:32:14.589 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.589 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.590 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.590 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "45", 2024-06-10T01:32:14.590 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.590 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.590 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.590 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.590 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.590 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.590 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.590 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ACCESS_KEY": { 2024-06-10T01:32:14.590 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-10T01:32:14.590 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.591 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.591 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.591 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.591 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.591 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.591 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.591 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.591 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.591 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.591 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.591 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-10T01:32:14.591 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-10T01:32:14.592 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.592 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.592 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.592 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-06-10T01:32:14.592 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.592 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.592 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.592 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.592 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.592 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.592 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.592 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SECRET_KEY": { 2024-06-10T01:32:14.592 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-10T01:32:14.592 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.593 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.593 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.593 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.593 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.593 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.593 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.593 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.593 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.593 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.593 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.593 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SSL_VERIFY": { 2024-06-10T01:32:14.593 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-10T01:32:14.593 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.593 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.594 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.594 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.594 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.594 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.594 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.594 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.594 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.594 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.594 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.594 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-10T01:32:14.594 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-10T01:32:14.594 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.594 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.595 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.595 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-06-10T01:32:14.595 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.595 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.595 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.595 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.595 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.595 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.595 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.595 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-10T01:32:14.595 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-10T01:32:14.595 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.595 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.596 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.596 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-10T01:32:14.596 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.596 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.596 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.596 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.596 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.596 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.596 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.596 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-10T01:32:14.596 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-10T01:32:14.596 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.596 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.596 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.597 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-06-10T01:32:14.597 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.597 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.597 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.597 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.597 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.597 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.597 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.597 INFO:teuthology.orchestra.run.smithi007.stdout: "cross_origin_url": { 2024-06-10T01:32:14.597 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cross_origin_url", 2024-06-10T01:32:14.597 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.597 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.597 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.597 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.598 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.598 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.598 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.598 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.598 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.598 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.598 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.598 INFO:teuthology.orchestra.run.smithi007.stdout: "crt_file": { 2024-06-10T01:32:14.598 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crt_file", 2024-06-10T01:32:14.598 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.598 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.598 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.599 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.599 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.599 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.599 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.599 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.599 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.599 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.599 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.599 INFO:teuthology.orchestra.run.smithi007.stdout: "debug": { 2024-06-10T01:32:14.599 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "debug", 2024-06-10T01:32:14.599 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.599 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.599 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.600 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.600 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.600 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.600 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.600 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable/disable debug options", 2024-06-10T01:32:14.600 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.600 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.600 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.600 INFO:teuthology.orchestra.run.smithi007.stdout: "jwt_token_ttl": { 2024-06-10T01:32:14.600 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "jwt_token_ttl", 2024-06-10T01:32:14.600 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.600 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.600 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.601 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "28800", 2024-06-10T01:32:14.601 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.601 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.601 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.601 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.601 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.601 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.601 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.601 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-06-10T01:32:14.601 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-06-10T01:32:14.601 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.601 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.601 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.602 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.602 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.602 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.602 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.602 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.602 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.602 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.602 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.602 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.602 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.602 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.602 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.603 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.603 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.603 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.603 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.603 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.603 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.603 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.603 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.603 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.603 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.603 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.603 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.603 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.603 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.603 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.604 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.604 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.604 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.604 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.604 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.604 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.604 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.604 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.604 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.604 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.604 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.604 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.604 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.605 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.605 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.605 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.605 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.605 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.605 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.605 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.605 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.605 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.605 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.605 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.605 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.605 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.606 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.606 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.606 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.606 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.606 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.606 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.606 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.606 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.606 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.606 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.606 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.606 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.607 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.607 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.607 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.607 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.607 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.607 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.607 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.607 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.607 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.607 INFO:teuthology.orchestra.run.smithi007.stdout: "motd": { 2024-06-10T01:32:14.607 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "motd", 2024-06-10T01:32:14.607 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.607 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.608 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.608 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.608 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.608 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.608 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.608 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "The message of the day", 2024-06-10T01:32:14.608 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.608 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.608 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.608 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect_resolve_ip_addr": { 2024-06-10T01:32:14.608 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "redirect_resolve_ip_addr", 2024-06-10T01:32:14.608 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.608 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.609 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.609 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.609 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.609 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.609 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.609 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.609 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.609 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.609 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.609 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-06-10T01:32:14.609 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-06-10T01:32:14.609 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.609 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.610 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.610 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-06-10T01:32:14.610 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.610 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.610 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.610 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.610 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.610 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.610 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.610 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-06-10T01:32:14.610 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-06-10T01:32:14.610 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.610 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.611 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.611 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8080", 2024-06-10T01:32:14.611 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.611 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.611 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.611 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.611 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.611 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.611 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.611 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-06-10T01:32:14.611 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-06-10T01:32:14.611 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.611 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.612 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.612 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.612 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.612 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.612 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.612 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.612 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.612 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.612 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.612 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl_server_port": { 2024-06-10T01:32:14.612 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl_server_port", 2024-06-10T01:32:14.612 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.612 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.612 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.613 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8443", 2024-06-10T01:32:14.613 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.613 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.613 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.613 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.613 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.613 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.613 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.613 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-06-10T01:32:14.613 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-06-10T01:32:14.613 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.613 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.613 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.614 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "redirect", 2024-06-10T01:32:14.614 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.614 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.614 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.614 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.614 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect" 2024-06-10T01:32:14.614 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.614 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.614 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.614 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.614 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.615 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-06-10T01:32:14.615 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-06-10T01:32:14.615 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.615 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.615 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.615 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-06-10T01:32:14.615 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-06-10T01:32:14.615 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-06-10T01:32:14.615 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.615 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.615 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.615 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.615 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.616 INFO:teuthology.orchestra.run.smithi007.stdout: "url_prefix": { 2024-06-10T01:32:14.616 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url_prefix", 2024-06-10T01:32:14.616 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.616 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.616 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.616 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.616 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.616 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.616 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.616 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.616 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.616 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.616 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.617 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.617 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.617 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-06-10T01:32:14.617 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.617 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.617 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.617 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_monitoring": { 2024-06-10T01:32:14.617 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_monitoring", 2024-06-10T01:32:14.617 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.617 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.617 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.617 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.618 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.618 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.618 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.618 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "monitor device health metrics", 2024-06-10T01:32:14.618 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.618 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.618 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.618 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.618 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.618 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.618 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.619 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.619 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.619 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.619 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.619 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.619 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.619 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.619 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.619 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.619 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.619 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.619 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.619 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.619 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.620 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.620 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.620 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.620 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.620 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.620 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.620 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.620 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.620 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.620 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.620 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.620 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.621 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.621 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.621 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.621 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.621 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.621 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.621 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.621 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.621 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.621 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.621 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.621 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.622 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.622 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.622 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.622 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.622 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.622 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.622 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.622 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.622 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.622 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.622 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.622 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.623 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.623 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.623 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.623 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.623 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.623 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.623 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.623 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.623 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.623 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.623 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.623 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.623 INFO:teuthology.orchestra.run.smithi007.stdout: "mark_out_threshold": { 2024-06-10T01:32:14.623 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mark_out_threshold", 2024-06-10T01:32:14.624 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:14.624 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.624 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.624 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2419200", 2024-06-10T01:32:14.624 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.624 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.624 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.624 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-10T01:32:14.624 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.624 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.624 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.624 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": { 2024-06-10T01:32:14.624 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_name", 2024-06-10T01:32:14.625 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.625 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.625 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.625 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "device_health_metrics", 2024-06-10T01:32:14.625 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.625 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.625 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.625 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-10T01:32:14.625 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.625 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.625 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.625 INFO:teuthology.orchestra.run.smithi007.stdout: "retention_period": { 2024-06-10T01:32:14.626 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retention_period", 2024-06-10T01:32:14.626 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:14.626 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.626 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.626 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15552000", 2024-06-10T01:32:14.626 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.626 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.626 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.626 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain device health metrics", 2024-06-10T01:32:14.626 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.626 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.626 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.627 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_frequency": { 2024-06-10T01:32:14.627 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_frequency", 2024-06-10T01:32:14.627 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:14.627 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.627 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.627 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-06-10T01:32:14.627 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.627 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.627 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.627 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-10T01:32:14.627 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.627 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.627 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.628 INFO:teuthology.orchestra.run.smithi007.stdout: "self_heal": { 2024-06-10T01:32:14.628 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "self_heal", 2024-06-10T01:32:14.628 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.628 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.628 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.628 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.628 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.628 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.628 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.628 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-10T01:32:14.628 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.628 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.628 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.629 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-06-10T01:32:14.629 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-06-10T01:32:14.629 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:14.629 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.629 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.629 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-06-10T01:32:14.629 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.629 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.629 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.629 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and check device health", 2024-06-10T01:32:14.629 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.629 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.630 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.630 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_threshold": { 2024-06-10T01:32:14.630 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_threshold", 2024-06-10T01:32:14.630 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:14.630 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.630 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.630 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7257600", 2024-06-10T01:32:14.630 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.630 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.630 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.630 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-10T01:32:14.631 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.631 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.631 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.631 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.631 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.631 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "diskprediction_local", 2024-06-10T01:32:14.631 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.631 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.631 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.631 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.631 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.632 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.632 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.632 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.632 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.632 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.632 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.632 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.632 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.632 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.632 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.632 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.632 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.632 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.633 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.633 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.633 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.633 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.633 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.633 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.633 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.633 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.633 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.633 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.633 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.633 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.633 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.634 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.634 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.634 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.634 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.634 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.634 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.634 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.634 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.634 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.634 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.634 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.634 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.635 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.635 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.635 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.635 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.635 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.635 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.635 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.635 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.635 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.635 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.635 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.635 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.635 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.636 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.636 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.636 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.636 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.636 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.636 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.636 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.636 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.636 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.636 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.636 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.636 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.637 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.637 INFO:teuthology.orchestra.run.smithi007.stdout: "predict_interval": { 2024-06-10T01:32:14.637 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predict_interval", 2024-06-10T01:32:14.637 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.637 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.637 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.637 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-06-10T01:32:14.637 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.637 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.637 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.637 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.637 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.638 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.638 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.638 INFO:teuthology.orchestra.run.smithi007.stdout: "predictor_model": { 2024-06-10T01:32:14.638 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predictor_model", 2024-06-10T01:32:14.638 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.638 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.638 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.638 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "prophetstor", 2024-06-10T01:32:14.638 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.638 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.638 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.638 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.639 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.639 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.639 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.639 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-06-10T01:32:14.639 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-06-10T01:32:14.639 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.639 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.639 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.639 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-06-10T01:32:14.639 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.639 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.639 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.640 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.640 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.640 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.640 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.640 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.640 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.640 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "influx", 2024-06-10T01:32:14.640 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": false, 2024-06-10T01:32:14.640 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "influxdb python module not found", 2024-06-10T01:32:14.640 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.640 INFO:teuthology.orchestra.run.smithi007.stdout: "batch_size": { 2024-06-10T01:32:14.640 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "batch_size", 2024-06-10T01:32:14.641 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.641 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.641 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.641 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5000", 2024-06-10T01:32:14.641 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.641 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.641 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.641 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-10T01:32:14.641 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.641 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.641 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.642 INFO:teuthology.orchestra.run.smithi007.stdout: "database": { 2024-06-10T01:32:14.642 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "database", 2024-06-10T01:32:14.642 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.642 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.642 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.642 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "ceph", 2024-06-10T01:32:14.642 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.642 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.642 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.642 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-10T01:32:14.642 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.642 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.643 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.643 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": { 2024-06-10T01:32:14.643 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hostname", 2024-06-10T01:32:14.643 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.643 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.643 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.643 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.643 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.643 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.643 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.643 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server hostname", 2024-06-10T01:32:14.643 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.643 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.644 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.644 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-06-10T01:32:14.644 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-06-10T01:32:14.644 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:14.644 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.644 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.644 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "30", 2024-06-10T01:32:14.644 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "5", 2024-06-10T01:32:14.644 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.644 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.644 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-10T01:32:14.644 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.645 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.645 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.645 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.645 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.645 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.645 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.645 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.645 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.645 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.645 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.645 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.645 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.646 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.646 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.646 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.646 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.646 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.646 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.646 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.646 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.646 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.646 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.646 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.646 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.647 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.647 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.647 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.647 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.647 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.647 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.647 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.647 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.647 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.647 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.647 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.647 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.647 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.648 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.648 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.648 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.648 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.648 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.648 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.648 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.648 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.648 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.648 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.648 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.648 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.648 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.648 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.649 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.649 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.649 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.649 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.649 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.649 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.649 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.649 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.649 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.649 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.649 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.649 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.649 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.650 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.650 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.650 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.650 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.650 INFO:teuthology.orchestra.run.smithi007.stdout: "password": { 2024-06-10T01:32:14.650 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "password", 2024-06-10T01:32:14.650 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.650 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.650 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.650 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.650 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.650 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.651 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.651 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "password of InfluxDB server user", 2024-06-10T01:32:14.651 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.651 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.651 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.651 INFO:teuthology.orchestra.run.smithi007.stdout: "port": { 2024-06-10T01:32:14.651 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "port", 2024-06-10T01:32:14.651 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.651 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.651 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.651 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8086", 2024-06-10T01:32:14.651 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.651 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.652 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.652 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server port", 2024-06-10T01:32:14.652 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.652 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.652 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.652 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-06-10T01:32:14.652 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-06-10T01:32:14.652 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.652 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.652 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.652 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "false", 2024-06-10T01:32:14.652 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.652 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.653 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.653 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-10T01:32:14.653 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.653 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.653 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.653 INFO:teuthology.orchestra.run.smithi007.stdout: "threads": { 2024-06-10T01:32:14.653 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threads", 2024-06-10T01:32:14.653 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.653 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.653 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.653 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-06-10T01:32:14.653 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-06-10T01:32:14.653 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "32", 2024-06-10T01:32:14.654 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.654 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-10T01:32:14.654 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.654 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.654 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.654 INFO:teuthology.orchestra.run.smithi007.stdout: "username": { 2024-06-10T01:32:14.654 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "username", 2024-06-10T01:32:14.654 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.654 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.654 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.654 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.655 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.655 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.655 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.655 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "username of InfluxDB server user", 2024-06-10T01:32:14.655 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.655 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.655 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.655 INFO:teuthology.orchestra.run.smithi007.stdout: "verify_ssl": { 2024-06-10T01:32:14.655 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "verify_ssl", 2024-06-10T01:32:14.655 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.655 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.655 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.656 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "true", 2024-06-10T01:32:14.656 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.656 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.656 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.656 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-10T01:32:14.656 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.656 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.656 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.656 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.656 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.656 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "insights", 2024-06-10T01:32:14.656 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.656 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.657 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.657 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.657 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.657 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.657 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.657 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.657 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.657 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.657 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.657 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.657 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.657 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.657 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.658 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.658 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.658 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.658 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.658 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.658 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.658 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.658 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.658 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.658 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.658 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.659 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.659 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.659 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.659 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.659 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.659 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.659 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.659 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.659 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.659 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.659 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.659 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.659 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.660 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.660 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.660 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.660 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.660 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.660 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.660 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.660 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.660 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.660 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.660 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.660 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.660 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.661 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.661 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.661 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.661 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.661 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.661 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.661 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.661 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.661 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.661 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.661 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.661 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.662 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.662 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.662 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.662 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.662 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.662 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.662 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.662 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "iostat", 2024-06-10T01:32:14.662 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.662 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.662 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.663 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.663 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.663 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.663 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.663 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.663 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.663 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.663 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.663 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.663 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.663 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.663 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.663 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.664 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.664 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.664 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.664 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.664 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.664 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.664 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.664 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.664 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.664 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.664 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.664 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.664 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.665 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.665 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.665 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.665 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.665 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.665 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.665 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.665 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.665 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.665 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.665 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.665 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.666 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.666 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.666 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.666 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.666 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.666 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.666 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.666 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.666 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.666 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.666 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.666 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.666 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.667 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.667 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.667 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.667 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.667 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.667 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.667 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.667 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.667 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.667 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.667 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.667 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.667 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.668 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.668 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.668 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.668 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.668 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "k8sevents", 2024-06-10T01:32:14.668 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.668 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.668 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.668 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_event_retention_days": { 2024-06-10T01:32:14.668 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ceph_event_retention_days", 2024-06-10T01:32:14.668 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.669 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.669 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.669 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7", 2024-06-10T01:32:14.669 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.669 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.669 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.669 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-10T01:32:14.669 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.669 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.669 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.669 INFO:teuthology.orchestra.run.smithi007.stdout: "config_check_secs": { 2024-06-10T01:32:14.669 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_check_secs", 2024-06-10T01:32:14.669 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.670 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.670 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.670 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-10T01:32:14.670 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "10", 2024-06-10T01:32:14.670 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.670 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.670 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-10T01:32:14.670 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.670 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.670 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.671 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.671 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.671 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.671 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.671 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.671 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.671 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.671 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.671 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.671 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.671 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.671 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.671 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.672 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.672 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.672 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.672 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.672 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.672 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.672 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.672 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.672 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.672 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.672 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.672 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.673 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.673 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.673 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.673 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.673 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.673 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.673 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.673 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.673 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.673 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.673 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.673 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.673 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.674 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.674 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.674 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.674 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.674 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.674 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.674 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.674 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.674 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.674 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.674 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.674 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.674 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.675 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.675 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.675 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.675 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.675 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.675 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.675 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.675 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.675 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.675 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.675 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.675 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.675 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.676 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.676 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.676 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.676 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.676 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "localpool", 2024-06-10T01:32:14.676 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.676 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.676 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.676 INFO:teuthology.orchestra.run.smithi007.stdout: "failure_domain": { 2024-06-10T01:32:14.676 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "failure_domain", 2024-06-10T01:32:14.676 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.676 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.677 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.677 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "host", 2024-06-10T01:32:14.677 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.677 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.677 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.677 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "failure domain for any created local pool", 2024-06-10T01:32:14.677 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-10T01:32:14.677 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.677 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.677 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.677 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.677 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.677 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.678 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.678 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.678 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.678 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.678 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.678 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.678 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.678 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.678 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.678 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.678 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.678 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.678 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.678 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.679 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.679 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.679 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.679 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.679 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.679 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.679 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.679 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.679 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.679 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.679 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.679 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.679 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.680 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.680 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.680 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.680 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.680 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.680 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.680 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.680 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.680 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.680 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.680 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.680 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.681 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.681 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.681 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.681 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.681 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.681 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.681 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.681 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.681 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.681 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.681 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.681 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.681 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.682 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.682 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.682 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.682 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.682 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.682 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.682 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.682 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.682 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.682 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.682 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": { 2024-06-10T01:32:14.682 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_size", 2024-06-10T01:32:14.682 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.683 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.683 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.683 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.683 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.683 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.683 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.683 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default min_size for any created local pool", 2024-06-10T01:32:14.683 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-10T01:32:14.683 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.683 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.683 INFO:teuthology.orchestra.run.smithi007.stdout: "num_rep": { 2024-06-10T01:32:14.683 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "num_rep", 2024-06-10T01:32:14.684 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.684 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.684 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.684 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3", 2024-06-10T01:32:14.684 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.684 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.684 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.684 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default replica count for any created local pool", 2024-06-10T01:32:14.684 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.684 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.684 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.684 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": { 2024-06-10T01:32:14.685 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_num", 2024-06-10T01:32:14.685 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.685 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.685 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.685 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "128", 2024-06-10T01:32:14.685 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.685 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.685 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.685 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default pg_num for any created local pool", 2024-06-10T01:32:14.685 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.685 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.685 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.685 INFO:teuthology.orchestra.run.smithi007.stdout: "prefix": { 2024-06-10T01:32:14.685 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prefix", 2024-06-10T01:32:14.686 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.686 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.686 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.686 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.686 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.686 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.686 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.686 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name prefix for any created local pool", 2024-06-10T01:32:14.686 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.686 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.686 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.686 INFO:teuthology.orchestra.run.smithi007.stdout: "subtree": { 2024-06-10T01:32:14.687 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "subtree", 2024-06-10T01:32:14.687 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.687 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.687 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.687 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "rack", 2024-06-10T01:32:14.687 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.687 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.687 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.687 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-10T01:32:14.687 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-10T01:32:14.687 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.687 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.687 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.687 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.688 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mds_autoscaler", 2024-06-10T01:32:14.688 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.688 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.688 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.688 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.688 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.688 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.688 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.688 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.688 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.688 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.688 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.689 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.689 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.689 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.689 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.689 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.689 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.689 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.689 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.689 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.689 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.689 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.689 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.690 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.690 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.690 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.690 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.690 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.690 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.690 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.690 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.690 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.690 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.690 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.690 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.690 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.691 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.691 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.691 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.691 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.691 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.691 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.691 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.691 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.691 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.691 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.691 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.691 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.692 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.692 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.692 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.692 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.692 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.692 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.692 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.692 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.692 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.692 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.692 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.692 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.692 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.693 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.693 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.693 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.693 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.693 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.693 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.693 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.693 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.693 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.693 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.693 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirroring", 2024-06-10T01:32:14.693 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.694 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.694 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.694 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.694 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.694 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.694 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.694 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.694 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.694 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.694 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.694 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.694 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.694 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.694 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.695 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.695 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.695 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.695 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.695 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.695 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.695 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.695 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.695 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.695 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.695 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.695 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.695 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.695 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.696 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.696 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.696 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.696 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.696 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.696 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.696 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.696 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.696 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.696 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.696 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.696 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.697 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.697 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.697 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.697 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.697 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.697 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.697 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.697 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.697 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.697 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.697 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.697 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.697 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.697 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.698 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.698 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.698 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.698 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.698 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.698 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.698 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.698 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.698 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.698 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.698 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.699 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.699 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.699 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.699 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.699 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.699 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "nfs", 2024-06-10T01:32:14.699 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.699 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.699 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.699 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.699 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.700 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.700 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.700 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.700 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.700 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.700 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.700 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.700 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.700 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.700 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.700 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.700 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.700 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.700 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.701 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.701 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.701 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.701 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.701 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.701 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.701 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.701 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.701 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.701 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.701 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.701 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.702 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.702 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.702 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.702 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.702 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.702 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.702 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.702 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.702 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.702 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.702 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.702 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.703 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.703 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.703 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.703 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.703 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.703 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.703 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.703 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.703 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.703 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.703 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.703 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.703 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.704 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.704 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.704 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.704 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.704 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.704 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.704 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.704 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.704 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.704 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.704 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.704 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.705 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.705 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.705 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.705 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-06-10T01:32:14.705 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.705 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.705 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.705 INFO:teuthology.orchestra.run.smithi007.stdout: "fail_fs": { 2024-06-10T01:32:14.705 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "fail_fs", 2024-06-10T01:32:14.705 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.705 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.705 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.706 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.706 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.706 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.706 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.706 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-10T01:32:14.706 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.706 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.706 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.706 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.706 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.706 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.706 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.706 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.707 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.707 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.707 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.707 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.707 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.707 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.707 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.707 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.707 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.707 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.707 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.707 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.707 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.708 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.708 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.708 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.708 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.708 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.708 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.708 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.708 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.708 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.708 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.708 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.708 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.708 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.709 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.709 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.709 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.709 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.709 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.709 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.709 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.709 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.709 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.709 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.709 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.709 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.709 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.709 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.710 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.710 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.710 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.710 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.710 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.710 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.710 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.710 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.710 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.710 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.710 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.710 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.711 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.711 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.711 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.711 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.711 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.711 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.711 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.711 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.711 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.711 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator": { 2024-06-10T01:32:14.711 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-06-10T01:32:14.711 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.712 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.712 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.712 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.712 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.712 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.712 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.712 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-06-10T01:32:14.712 INFO:teuthology.orchestra.run.smithi007.stdout: "rook", 2024-06-10T01:32:14.712 INFO:teuthology.orchestra.run.smithi007.stdout: "test_orchestrator" 2024-06-10T01:32:14.712 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.712 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Orchestrator backend", 2024-06-10T01:32:14.712 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.712 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.713 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.713 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.713 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.713 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_perf_query", 2024-06-10T01:32:14.713 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.713 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.713 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.713 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.713 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.713 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.714 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.714 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.714 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.714 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.714 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.714 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.714 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.714 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.714 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.714 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.714 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.714 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.715 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.715 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.715 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.715 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.715 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.715 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.715 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.715 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.715 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.715 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.715 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.715 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.716 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.716 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.716 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.716 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.716 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.716 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.716 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.716 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.716 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.716 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.716 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.716 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.716 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.717 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.717 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.717 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.717 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.717 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.717 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.717 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.717 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.717 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.717 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.717 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.717 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.717 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.718 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.718 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.718 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.718 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.718 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.718 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.718 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.718 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.718 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.718 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.718 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.718 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.719 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.719 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.719 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.719 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_support", 2024-06-10T01:32:14.719 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.719 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.719 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.719 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.719 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.719 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.719 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.720 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.720 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.720 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.720 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.720 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.720 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.720 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.720 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.720 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.720 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.720 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.720 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.720 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.720 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.721 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.721 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.721 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.721 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.721 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.721 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.721 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.721 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.721 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.721 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.721 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.721 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.722 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.722 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.722 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.722 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.722 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.722 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.722 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.722 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.722 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.722 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.722 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.722 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.723 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.723 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.723 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.723 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.723 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.723 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.723 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.723 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.723 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.723 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.723 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.723 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.723 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.724 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.724 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.724 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.724 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.724 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.724 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.724 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.724 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.724 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.724 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.724 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.724 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.724 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.725 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_autoscaler", 2024-06-10T01:32:14.725 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.725 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.725 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.725 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.725 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.725 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.725 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.725 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.725 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.725 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.725 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-06-10T01:32:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-06-10T01:32:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-06-10T01:32:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "threshold": { 2024-06-10T01:32:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threshold", 2024-06-10T01:32:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-10T01:32:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-06-10T01:32:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1.0", 2024-06-10T01:32:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "scaling threshold", 2024-06-10T01:32:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-10T01:32:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "progress", 2024-06-10T01:32:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_pg_recovery_event": { 2024-06-10T01:32:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_pg_recovery_event", 2024-06-10T01:32:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-10T01:32:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-06-10T01:32:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-06-10T01:32:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.735 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.735 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.735 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.735 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.735 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.735 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.735 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.735 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.735 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.735 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.735 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.735 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "max_completed_events": { 2024-06-10T01:32:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_completed_events", 2024-06-10T01:32:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "50", 2024-06-10T01:32:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "number of past completed events to remember", 2024-06-10T01:32:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-06-10T01:32:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-06-10T01:32:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-06-10T01:32:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long the module is going to sleep", 2024-06-10T01:32:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus", 2024-06-10T01:32:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: "cache": { 2024-06-10T01:32:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cache", 2024-06-10T01:32:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.744 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.744 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.744 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.744 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.744 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.744 INFO:teuthology.orchestra.run.smithi007.stdout: "exclude_perf_counters": { 2024-06-10T01:32:14.744 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "exclude_perf_counters", 2024-06-10T01:32:14.744 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.744 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.744 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.744 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.744 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.745 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.745 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.745 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-10T01:32:14.745 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-10T01:32:14.745 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.745 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.745 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.745 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.745 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.745 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.745 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.745 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.746 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.746 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.746 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.746 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.746 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.746 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.746 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.746 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.746 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.746 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.746 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.746 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.746 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.747 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.747 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.747 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.747 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.747 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.747 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.747 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.747 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.747 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.747 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.747 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.747 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.747 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.748 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.748 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.748 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.748 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.748 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.748 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.748 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.748 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.748 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.748 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.748 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.748 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.748 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.749 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.749 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.749 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.749 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.749 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.749 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.749 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.749 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.749 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.749 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.749 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.749 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.750 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.750 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.750 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.750 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.750 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.750 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.750 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.750 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.750 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.750 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools": { 2024-06-10T01:32:14.750 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools", 2024-06-10T01:32:14.751 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.751 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.751 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.751 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.751 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.751 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.751 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.751 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.751 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.751 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.751 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.751 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-10T01:32:14.751 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-10T01:32:14.752 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.752 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.752 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.752 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "300", 2024-06-10T01:32:14.752 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.752 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.752 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.752 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.752 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.752 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.752 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.752 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_interval": { 2024-06-10T01:32:14.752 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_interval", 2024-06-10T01:32:14.753 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-10T01:32:14.753 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.753 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.753 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15.0", 2024-06-10T01:32:14.753 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.753 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.753 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.753 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.753 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.753 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.753 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.753 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-06-10T01:32:14.754 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-06-10T01:32:14.754 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.754 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.754 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.754 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-06-10T01:32:14.754 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.754 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.754 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.754 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-10T01:32:14.754 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.754 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.754 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.755 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-06-10T01:32:14.755 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-06-10T01:32:14.755 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.755 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.755 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.755 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "9283", 2024-06-10T01:32:14.755 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.755 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.755 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.755 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-10T01:32:14.755 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.755 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.755 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.756 INFO:teuthology.orchestra.run.smithi007.stdout: "stale_cache_strategy": { 2024-06-10T01:32:14.756 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stale_cache_strategy", 2024-06-10T01:32:14.756 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.756 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.756 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.756 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "log", 2024-06-10T01:32:14.756 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.756 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.756 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.756 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.756 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.756 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.756 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.757 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-06-10T01:32:14.757 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-06-10T01:32:14.757 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.757 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.757 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.757 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "default", 2024-06-10T01:32:14.757 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.757 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.757 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.757 INFO:teuthology.orchestra.run.smithi007.stdout: "default", 2024-06-10T01:32:14.757 INFO:teuthology.orchestra.run.smithi007.stdout: "error" 2024-06-10T01:32:14.757 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.758 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.758 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.758 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.758 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.758 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-06-10T01:32:14.758 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-06-10T01:32:14.758 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.758 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.758 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.758 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-06-10T01:32:14.758 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-06-10T01:32:14.759 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-06-10T01:32:14.759 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.759 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.759 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.759 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.759 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.759 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.759 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.759 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-06-10T01:32:14.759 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.759 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.760 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.760 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.760 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.760 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.760 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.760 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.760 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.760 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.760 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.760 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.760 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.760 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.760 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.761 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.761 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.761 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.761 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.761 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.761 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.761 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.761 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.761 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.761 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.761 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.761 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.762 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.762 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.762 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.762 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.762 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.762 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.762 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.762 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.762 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.762 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.762 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.762 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.763 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.763 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.763 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.763 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.763 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.763 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.763 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.763 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.763 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.763 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.763 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.763 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.763 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.764 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.764 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.764 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.764 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.764 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.764 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.764 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.764 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.764 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.764 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.764 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.764 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.765 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.765 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.765 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.765 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.765 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.765 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_snap_create": { 2024-06-10T01:32:14.765 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_snap_create", 2024-06-10T01:32:14.765 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.765 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.765 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.765 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-10T01:32:14.765 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.765 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.766 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.766 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.766 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.766 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.766 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.766 INFO:teuthology.orchestra.run.smithi007.stdout: "mirror_snapshot_schedule": { 2024-06-10T01:32:14.766 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirror_snapshot_schedule", 2024-06-10T01:32:14.766 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.766 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.766 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.766 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.766 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.767 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.767 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.767 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.767 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.767 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.767 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.767 INFO:teuthology.orchestra.run.smithi007.stdout: "trash_purge_schedule": { 2024-06-10T01:32:14.767 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "trash_purge_schedule", 2024-06-10T01:32:14.767 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.767 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.767 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.767 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.768 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.768 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.768 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.768 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.768 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.768 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.768 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.768 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.768 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.768 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "restful", 2024-06-10T01:32:14.768 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.768 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.769 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.769 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_auth": { 2024-06-10T01:32:14.769 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_auth", 2024-06-10T01:32:14.769 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.769 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.769 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.769 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.769 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.769 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.769 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.769 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.769 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.769 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.770 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.770 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-06-10T01:32:14.770 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-06-10T01:32:14.770 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.770 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.770 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.770 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.770 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.770 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.770 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.770 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.771 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.771 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.771 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.771 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.771 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.771 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.771 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.771 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.771 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.771 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.772 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.772 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.772 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.772 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.772 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.772 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.772 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.772 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.772 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.772 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.772 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.772 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.772 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.774 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.774 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.774 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.774 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.774 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.774 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.774 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.774 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.774 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.774 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.774 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.774 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.774 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.774 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.775 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.775 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.775 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.775 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.775 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.775 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.775 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.775 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.775 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.775 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.775 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.775 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.776 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.776 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.776 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.776 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.776 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.776 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.776 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.776 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-06-10T01:32:14.776 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-06-10T01:32:14.776 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.776 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.776 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.776 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.776 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.777 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.777 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.777 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.777 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.777 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.777 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.777 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-06-10T01:32:14.777 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-06-10T01:32:14.777 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.777 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.777 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.777 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.777 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.777 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.778 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.778 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.778 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.778 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.778 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.778 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.778 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.778 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rgw", 2024-06-10T01:32:14.778 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.778 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.778 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.779 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.779 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.779 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.779 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.779 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.779 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.779 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.779 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.779 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.779 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.779 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.779 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.779 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.779 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.780 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.780 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.780 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.780 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.780 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.780 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.780 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.780 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.780 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.780 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.780 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.780 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.781 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.781 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.781 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.781 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.781 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.781 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.781 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.781 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.781 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.781 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.781 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.781 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.781 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.781 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.782 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.782 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.782 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.782 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.782 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.782 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.782 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.782 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.782 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.782 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.782 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.782 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.782 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.783 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.783 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.783 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.783 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.783 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.783 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.783 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.783 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.783 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.783 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.783 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.783 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.783 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.783 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.783 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.784 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rook", 2024-06-10T01:32:14.784 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.784 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.784 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.784 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.784 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.784 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.784 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.784 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.784 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.784 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.784 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.784 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.785 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.785 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.785 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.785 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.785 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.785 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.785 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.785 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.785 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.785 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.785 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.786 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.786 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.786 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.786 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.786 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.786 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.786 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.786 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.786 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.786 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.786 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.786 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.786 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.787 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.787 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.787 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.787 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.787 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.787 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.787 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.787 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.787 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.787 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.787 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.787 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.787 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.787 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.788 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.788 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.788 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.788 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.788 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.788 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.788 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.788 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.788 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.788 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.788 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.788 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.788 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.789 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.789 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.789 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.789 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.789 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.789 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.789 INFO:teuthology.orchestra.run.smithi007.stdout: "storage_class": { 2024-06-10T01:32:14.789 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "storage_class", 2024-06-10T01:32:14.789 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.789 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.789 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.789 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "local", 2024-06-10T01:32:14.789 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.789 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.790 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.790 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-10T01:32:14.790 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.790 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.790 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.790 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.790 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.790 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "selftest", 2024-06-10T01:32:14.790 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.790 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.790 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.790 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.791 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.791 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.791 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.791 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.791 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.791 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.791 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.791 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.791 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.791 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.791 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.791 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.791 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.791 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.791 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.792 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.792 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.792 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.792 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.792 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.792 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.792 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.792 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.792 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.792 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.792 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.792 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.792 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.792 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.793 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.793 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.793 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.793 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.793 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.793 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.793 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.793 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.793 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.793 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.793 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.793 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.793 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.793 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.794 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.794 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.794 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.794 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.794 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.794 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.794 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.794 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.794 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.794 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.794 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.794 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.794 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.794 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.795 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.795 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.795 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.795 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.795 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.795 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.795 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.795 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.795 INFO:teuthology.orchestra.run.smithi007.stdout: "roption1": { 2024-06-10T01:32:14.795 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption1", 2024-06-10T01:32:14.795 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.795 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.795 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.795 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.796 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.796 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.796 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.796 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.796 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.796 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.796 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.796 INFO:teuthology.orchestra.run.smithi007.stdout: "roption2": { 2024-06-10T01:32:14.796 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption2", 2024-06-10T01:32:14.796 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.796 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.796 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.796 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "xyz", 2024-06-10T01:32:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption1": { 2024-06-10T01:32:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption1", 2024-06-10T01:32:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.798 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.798 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.798 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.798 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.798 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.798 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption2": { 2024-06-10T01:32:14.798 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption2", 2024-06-10T01:32:14.798 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.798 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.798 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.798 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.798 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.799 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.799 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.799 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.799 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.799 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.799 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.799 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption3": { 2024-06-10T01:32:14.799 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption3", 2024-06-10T01:32:14.799 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-10T01:32:14.799 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.799 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.799 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.799 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.799 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.800 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.800 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.800 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.800 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.800 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.800 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption4": { 2024-06-10T01:32:14.800 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption4", 2024-06-10T01:32:14.800 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.800 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.800 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.800 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.800 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.800 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.800 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.801 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.801 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.801 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.801 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.801 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption5": { 2024-06-10T01:32:14.801 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption5", 2024-06-10T01:32:14.801 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.801 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.801 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.801 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.801 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.801 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.801 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.801 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption6": { 2024-06-10T01:32:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption6", 2024-06-10T01:32:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.803 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.803 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.803 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.803 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption7": { 2024-06-10T01:32:14.803 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption7", 2024-06-10T01:32:14.803 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.803 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.803 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.803 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.803 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-06-10T01:32:14.803 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "42", 2024-06-10T01:32:14.803 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.803 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.803 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.804 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.804 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.804 INFO:teuthology.orchestra.run.smithi007.stdout: "testkey": { 2024-06-10T01:32:14.804 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testkey", 2024-06-10T01:32:14.804 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.804 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.804 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.804 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.804 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.804 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.804 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.804 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.804 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.804 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.805 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.805 INFO:teuthology.orchestra.run.smithi007.stdout: "testlkey": { 2024-06-10T01:32:14.805 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testlkey", 2024-06-10T01:32:14.805 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.805 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.805 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.805 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.805 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.805 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.805 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.805 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.805 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.806 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.806 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.806 INFO:teuthology.orchestra.run.smithi007.stdout: "testnewline": { 2024-06-10T01:32:14.806 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testnewline", 2024-06-10T01:32:14.806 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.806 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.806 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.806 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.806 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.806 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.806 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.806 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.806 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.807 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.807 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.807 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.807 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.807 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snap_schedule", 2024-06-10T01:32:14.807 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.807 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.807 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.807 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_m_granularity": { 2024-06-10T01:32:14.807 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_m_granularity", 2024-06-10T01:32:14.807 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.807 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.807 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.808 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.808 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.808 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.808 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.808 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow minute scheduled snapshots", 2024-06-10T01:32:14.808 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.808 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.808 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.808 INFO:teuthology.orchestra.run.smithi007.stdout: "dump_on_update": { 2024-06-10T01:32:14.808 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dump_on_update", 2024-06-10T01:32:14.808 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.808 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.809 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.809 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.809 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.809 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.809 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.809 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "dump database to debug log on update", 2024-06-10T01:32:14.809 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.809 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.809 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.809 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.809 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.809 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.809 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.809 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.809 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.810 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.810 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.810 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.810 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.810 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.810 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.810 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.810 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.810 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.810 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.810 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.810 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.810 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.811 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.811 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.811 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.811 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.811 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.811 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.811 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.811 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.811 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.811 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.811 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.811 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.811 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.811 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.812 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.812 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.812 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.812 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.812 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.812 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.812 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.812 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.812 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.812 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.812 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.812 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.812 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.813 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.813 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.813 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.813 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.813 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.813 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.813 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.813 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.813 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.813 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.813 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.813 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.813 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.813 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stats", 2024-06-10T01:32:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.815 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.815 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.815 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.815 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.815 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.815 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.815 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.815 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.815 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.815 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.815 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.815 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.815 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.815 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.816 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.816 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.816 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.816 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.816 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.816 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.816 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.816 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.816 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.816 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.816 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.816 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.817 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.817 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.817 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.817 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.817 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.817 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.817 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.817 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.817 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.817 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.817 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.817 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.817 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.818 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.818 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.818 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.818 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.818 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.818 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.818 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.818 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.818 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.818 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.818 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.818 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.818 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.818 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.819 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.819 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.819 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.819 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.819 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.819 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.819 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.819 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.819 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.819 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.819 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.819 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.819 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.819 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.820 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "status", 2024-06-10T01:32:14.820 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.820 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.820 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.820 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.820 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.820 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.820 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.820 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.820 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.820 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.820 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.820 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.820 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.821 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.821 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.821 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.821 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.821 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.821 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.821 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.821 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.821 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.821 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.821 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.821 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.821 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.822 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.822 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.822 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.822 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.822 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.822 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.822 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.822 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.822 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.822 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.822 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.822 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.823 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.823 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.823 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.823 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.823 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.823 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.823 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.823 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.823 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.823 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.823 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.823 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.823 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.823 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.823 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.824 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.824 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.824 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.824 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.824 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.824 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.824 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.824 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.824 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.824 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.824 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.824 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.824 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.824 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.824 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.825 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.825 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.825 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.825 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telegraf", 2024-06-10T01:32:14.825 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.825 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.825 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.825 INFO:teuthology.orchestra.run.smithi007.stdout: "address": { 2024-06-10T01:32:14.825 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "address", 2024-06-10T01:32:14.825 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.825 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.825 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.826 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-10T01:32:14.826 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.826 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.826 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.826 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.826 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.826 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.826 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.826 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-06-10T01:32:14.826 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-06-10T01:32:14.827 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:14.827 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.827 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.827 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15", 2024-06-10T01:32:14.827 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.827 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.827 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.827 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.827 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.827 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.827 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.827 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.827 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.828 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.828 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.828 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.828 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.828 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.828 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.828 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.828 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.828 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.828 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.828 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.828 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.828 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.828 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.828 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.829 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.829 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.829 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.829 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.829 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.829 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.829 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.829 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.829 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.829 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.829 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.829 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.830 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.830 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.830 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.830 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.830 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.830 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.830 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.830 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.830 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.830 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.830 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.830 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.831 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.831 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.831 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.831 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.831 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.831 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.831 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.831 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.831 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.831 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.831 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.831 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.832 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.832 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.832 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.832 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.832 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.832 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.832 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.832 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.832 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.832 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.832 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.832 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.832 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.832 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.833 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.833 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telemetry", 2024-06-10T01:32:14.833 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.833 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.833 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.833 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_basic": { 2024-06-10T01:32:14.833 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_basic", 2024-06-10T01:32:14.833 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.833 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.833 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.833 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.834 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.834 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.834 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.834 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-10T01:32:14.834 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.834 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.834 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.834 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_crash": { 2024-06-10T01:32:14.834 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_crash", 2024-06-10T01:32:14.834 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.834 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.834 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.834 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.835 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.835 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.835 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.835 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-10T01:32:14.835 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.835 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.835 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.835 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_device": { 2024-06-10T01:32:14.835 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_device", 2024-06-10T01:32:14.835 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.835 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.835 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.836 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.836 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.836 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.836 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.836 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-10T01:32:14.836 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.836 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.836 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.836 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_ident": { 2024-06-10T01:32:14.836 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_ident", 2024-06-10T01:32:14.836 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.836 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.836 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.837 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.837 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.837 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.837 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.837 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-10T01:32:14.837 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.837 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.837 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.837 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_perf": { 2024-06-10T01:32:14.837 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_perf", 2024-06-10T01:32:14.837 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.837 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.838 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.838 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.838 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.838 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.838 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.838 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-10T01:32:14.838 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.838 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.838 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.838 INFO:teuthology.orchestra.run.smithi007.stdout: "contact": { 2024-06-10T01:32:14.838 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "contact", 2024-06-10T01:32:14.838 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.839 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.839 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.839 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.839 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.839 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.839 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.839 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.839 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.839 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.839 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.839 INFO:teuthology.orchestra.run.smithi007.stdout: "description": { 2024-06-10T01:32:14.839 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "description", 2024-06-10T01:32:14.840 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.840 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.840 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.840 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.840 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.840 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.840 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.840 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.840 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.840 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.840 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.841 INFO:teuthology.orchestra.run.smithi007.stdout: "device_url": { 2024-06-10T01:32:14.841 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_url", 2024-06-10T01:32:14.841 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.841 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.841 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.841 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-10T01:32:14.841 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.841 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.841 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.841 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.841 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.841 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.841 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.842 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-06-10T01:32:14.842 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-06-10T01:32:14.842 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.842 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.842 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.842 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.842 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.842 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.842 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.842 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.842 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.842 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.843 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.843 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-06-10T01:32:14.843 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-06-10T01:32:14.843 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.843 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.843 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.843 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "24", 2024-06-10T01:32:14.843 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "8", 2024-06-10T01:32:14.843 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.843 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.843 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.843 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: "last_opt_revision": { 2024-06-10T01:32:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "last_opt_revision", 2024-06-10T01:32:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1", 2024-06-10T01:32:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.845 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.845 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.845 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.845 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard": { 2024-06-10T01:32:14.845 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard", 2024-06-10T01:32:14.845 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.845 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.845 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.845 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.845 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.845 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.845 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.846 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.846 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.846 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.846 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.846 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard_description": { 2024-06-10T01:32:14.846 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard_description", 2024-06-10T01:32:14.846 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.846 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.846 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.846 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.846 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.846 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.847 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.847 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.847 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.847 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.847 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.847 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.847 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.847 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.847 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.847 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.847 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.847 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.848 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.848 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.848 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.848 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.848 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.848 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.848 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.848 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.848 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.848 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.848 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.848 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.848 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.849 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.849 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.849 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.849 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.849 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.849 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.849 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.849 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.849 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.849 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.849 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.849 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.850 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.850 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.850 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.850 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.850 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.850 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.853 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.853 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.853 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.853 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.853 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.853 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.853 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.853 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.853 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.853 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.853 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.854 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.854 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.854 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.854 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.854 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.854 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.854 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.854 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.854 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.854 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.854 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.854 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.855 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.855 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.855 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.855 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.855 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.855 INFO:teuthology.orchestra.run.smithi007.stdout: "organization": { 2024-06-10T01:32:14.855 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "organization", 2024-06-10T01:32:14.855 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.855 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.855 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.855 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.855 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.856 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.856 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.856 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.856 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.856 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.856 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.856 INFO:teuthology.orchestra.run.smithi007.stdout: "proxy": { 2024-06-10T01:32:14.856 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "proxy", 2024-06-10T01:32:14.856 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.856 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.856 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.856 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.856 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.857 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.857 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.857 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.857 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.857 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.857 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.857 INFO:teuthology.orchestra.run.smithi007.stdout: "url": { 2024-06-10T01:32:14.857 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url", 2024-06-10T01:32:14.857 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.857 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.857 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.857 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-10T01:32:14.858 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.858 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.858 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.858 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.858 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.858 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.858 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.858 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.858 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.858 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "test_orchestrator", 2024-06-10T01:32:14.858 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.859 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.859 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.859 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.859 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.859 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.859 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.859 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.859 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.859 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.859 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.859 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.859 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.859 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.859 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.860 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.860 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.860 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.860 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.860 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.860 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.860 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.860 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.860 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.860 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.860 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.860 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.860 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.861 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.861 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.861 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.861 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.861 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.861 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.861 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.861 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.861 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.861 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.861 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.861 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.862 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.862 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.862 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.862 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.862 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.862 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.862 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.862 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.862 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.862 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.862 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.862 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.862 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.863 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.863 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.863 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.863 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.863 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.863 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.863 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.863 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.863 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.863 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.863 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.863 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.863 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.864 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.864 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.864 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.864 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.864 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.864 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-06-10T01:32:14.864 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.864 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.864 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.864 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.864 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.864 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.864 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.865 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.865 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.865 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.865 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.865 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.865 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.865 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.865 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.865 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.865 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.865 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.865 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.865 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.865 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.866 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.866 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.866 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.866 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.866 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.866 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.866 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.866 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.866 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.866 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.866 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.867 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.867 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.867 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.867 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.867 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.867 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.867 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.867 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.867 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.867 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.867 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.867 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.867 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.867 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.868 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.868 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.868 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.868 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.868 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.868 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.868 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.868 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.868 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.868 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.868 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.868 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.868 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.868 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.868 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.869 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.869 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.869 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.869 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.869 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.869 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.869 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.869 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.869 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_clones": { 2024-06-10T01:32:14.869 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_clones", 2024-06-10T01:32:14.869 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.869 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.870 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.870 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4", 2024-06-10T01:32:14.870 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.870 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.870 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.870 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-10T01:32:14.870 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.870 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.870 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.870 INFO:teuthology.orchestra.run.smithi007.stdout: "periodic_async_work": { 2024-06-10T01:32:14.870 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "periodic_async_work", 2024-06-10T01:32:14.870 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.871 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.871 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.871 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.871 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.871 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.871 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.871 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Periodically check for async work", 2024-06-10T01:32:14.871 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.871 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.871 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.871 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_delay": { 2024-06-10T01:32:14.871 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_delay", 2024-06-10T01:32:14.872 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.872 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.872 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.872 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-06-10T01:32:14.872 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.872 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.872 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.872 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-10T01:32:14.872 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.872 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.872 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.872 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_no_wait": { 2024-06-10T01:32:14.872 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_no_wait", 2024-06-10T01:32:14.873 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.873 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.873 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.873 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.873 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.873 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.873 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.873 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-10T01:32:14.873 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.873 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.873 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.874 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix", 2024-06-10T01:32:14.874 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.874 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.874 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.874 INFO:teuthology.orchestra.run.smithi007.stdout: "discovery_interval": { 2024-06-10T01:32:14.874 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "discovery_interval", 2024-06-10T01:32:14.874 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-06-10T01:32:14.874 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.874 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.874 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "100", 2024-06-10T01:32:14.874 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.875 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.875 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.875 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.875 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.875 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.875 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.875 INFO:teuthology.orchestra.run.smithi007.stdout: "identifier": { 2024-06-10T01:32:14.875 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "identifier", 2024-06-10T01:32:14.875 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.875 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.875 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.875 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.875 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.876 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.876 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.876 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.876 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.876 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.876 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.876 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-06-10T01:32:14.876 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-06-10T01:32:14.876 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:14.876 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.876 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.876 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-06-10T01:32:14.877 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.877 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.877 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.877 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.877 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.877 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.877 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.877 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.877 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.877 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.877 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.877 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.878 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.878 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.878 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.878 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.878 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.878 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.878 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.878 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.878 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.878 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.878 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.878 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.879 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.879 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.879 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.879 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.879 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.879 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.879 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.879 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.879 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.879 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.879 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.879 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.879 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.880 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.880 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.880 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.880 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.880 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.880 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.880 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.880 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.880 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.880 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.880 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.881 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.881 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.881 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.881 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.881 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.881 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.881 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.881 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.881 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.881 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.881 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.881 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.881 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.882 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.882 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.882 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.882 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.882 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.882 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.882 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.882 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.882 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.882 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.882 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.882 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.883 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_host": { 2024-06-10T01:32:14.883 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_host", 2024-06-10T01:32:14.883 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.883 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.883 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.883 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.883 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.883 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.883 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.883 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.883 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.883 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.883 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.883 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_port": { 2024-06-10T01:32:14.884 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_port", 2024-06-10T01:32:14.884 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.884 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.884 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.884 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10051", 2024-06-10T01:32:14.884 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.884 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.884 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.884 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.884 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.884 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.884 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.884 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_sender": { 2024-06-10T01:32:14.884 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_sender", 2024-06-10T01:32:14.884 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.885 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.885 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.885 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-10T01:32:14.885 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.885 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.885 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.885 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.885 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.885 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.885 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.885 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.885 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.885 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.885 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-06-10T01:32:14.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.886 INFO:teuthology.orchestra.run.smithi007.stdout: "gid": 24109, 2024-06-10T01:32:14.886 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "z", 2024-06-10T01:32:14.886 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_features": 4540701547738038271, 2024-06-10T01:32:14.886 INFO:teuthology.orchestra.run.smithi007.stdout: "available_modules": [ 2024-06-10T01:32:14.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.886 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "alerts", 2024-06-10T01:32:14.886 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.886 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.886 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.886 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-06-10T01:32:14.886 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-06-10T01:32:14.887 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:14.887 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.887 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.887 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-06-10T01:32:14.887 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.887 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.887 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.887 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How frequently to reexamine health status", 2024-06-10T01:32:14.887 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.887 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.887 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.887 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.887 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.887 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.890 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.890 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.890 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.890 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.890 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.890 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.890 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.890 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.890 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.890 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.890 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.890 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.890 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.891 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.891 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.891 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.891 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.891 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.891 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.891 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.891 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.891 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.891 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.891 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.891 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.891 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.892 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.892 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.892 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.892 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.892 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.892 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.892 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.892 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.892 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.892 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.892 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_destination": { 2024-06-10T01:32:14.892 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_destination", 2024-06-10T01:32:14.892 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.893 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.893 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.893 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.893 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.893 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.893 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.893 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email address to send alerts to", 2024-06-10T01:32:14.893 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.893 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.893 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.893 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_from_name": { 2024-06-10T01:32:14.893 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_from_name", 2024-06-10T01:32:14.894 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.894 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.894 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.894 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "Ceph", 2024-06-10T01:32:14.894 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.894 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.894 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.894 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email From: name", 2024-06-10T01:32:14.894 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.894 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.894 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.894 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_host": { 2024-06-10T01:32:14.895 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_host", 2024-06-10T01:32:14.895 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.895 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.895 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.895 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.895 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.895 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.895 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.895 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP server", 2024-06-10T01:32:14.895 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.895 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.895 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.896 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_password": { 2024-06-10T01:32:14.896 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_password", 2024-06-10T01:32:14.896 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.896 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.896 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.896 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.896 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.896 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.896 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.896 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Password to authenticate with", 2024-06-10T01:32:14.896 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.896 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.896 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.897 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_port": { 2024-06-10T01:32:14.897 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_port", 2024-06-10T01:32:14.897 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.897 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.897 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.897 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "465", 2024-06-10T01:32:14.897 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.897 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.897 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.897 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP port", 2024-06-10T01:32:14.897 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.897 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.897 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.898 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_sender": { 2024-06-10T01:32:14.898 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_sender", 2024-06-10T01:32:14.898 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.898 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.898 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.898 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.898 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.898 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.898 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.898 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP envelope sender", 2024-06-10T01:32:14.898 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.898 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.898 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.899 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_ssl": { 2024-06-10T01:32:14.899 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_ssl", 2024-06-10T01:32:14.899 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.899 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.899 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.899 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.899 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.899 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.899 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.899 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-10T01:32:14.899 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.899 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.899 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.900 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_user": { 2024-06-10T01:32:14.900 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_user", 2024-06-10T01:32:14.900 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.900 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.900 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.900 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.900 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.900 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.900 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.900 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "User to authenticate as", 2024-06-10T01:32:14.900 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.900 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.900 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.900 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.901 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.901 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "balancer", 2024-06-10T01:32:14.901 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.901 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.901 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.901 INFO:teuthology.orchestra.run.smithi007.stdout: "active": { 2024-06-10T01:32:14.901 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "active", 2024-06-10T01:32:14.901 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.901 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.901 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.901 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.902 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.902 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.902 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.902 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically balance PGs across cluster", 2024-06-10T01:32:14.902 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.902 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.902 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.902 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_time": { 2024-06-10T01:32:14.902 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_time", 2024-06-10T01:32:14.902 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.902 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.902 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.902 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0000", 2024-06-10T01:32:14.903 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.903 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.903 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.903 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "beginning time of day to automatically balance", 2024-06-10T01:32:14.903 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-10T01:32:14.903 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.903 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.903 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_weekday": { 2024-06-10T01:32:14.903 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_weekday", 2024-06-10T01:32:14.903 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-06-10T01:32:14.903 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.903 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.903 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-06-10T01:32:14.904 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-06-10T01:32:14.904 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-06-10T01:32:14.904 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.904 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-10T01:32:14.904 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-10T01:32:14.904 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.904 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.904 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_max_iterations": { 2024-06-10T01:32:14.904 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_max_iterations", 2024-06-10T01:32:14.904 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-06-10T01:32:14.904 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.904 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.904 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "25", 2024-06-10T01:32:14.904 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-06-10T01:32:14.905 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "250", 2024-06-10T01:32:14.905 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.905 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-10T01:32:14.905 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.905 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.905 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.905 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_metrics": { 2024-06-10T01:32:14.905 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_metrics", 2024-06-10T01:32:14.905 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.905 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.905 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.905 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "pgs,objects,bytes", 2024-06-10T01:32:14.905 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.905 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-10T01:32:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-10T01:32:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_step": { 2024-06-10T01:32:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_step", 2024-06-10T01:32:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-10T01:32:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.5", 2024-06-10T01:32:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0.001", 2024-06-10T01:32:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "0.999", 2024-06-10T01:32:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "aggressiveness of optimization", 2024-06-10T01:32:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-10T01:32:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: "end_time": { 2024-06-10T01:32:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_time", 2024-06-10T01:32:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2359", 2024-06-10T01:32:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.908 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.908 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.908 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ending time of day to automatically balance", 2024-06-10T01:32:14.908 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-10T01:32:14.908 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.908 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.908 INFO:teuthology.orchestra.run.smithi007.stdout: "end_weekday": { 2024-06-10T01:32:14.908 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_weekday", 2024-06-10T01:32:14.908 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-06-10T01:32:14.908 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.908 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.908 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-06-10T01:32:14.908 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-06-10T01:32:14.908 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-06-10T01:32:14.909 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.909 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-10T01:32:14.909 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-10T01:32:14.909 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.909 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.909 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.909 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.909 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.909 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.909 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.909 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.909 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.909 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.910 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.910 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.910 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.910 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.910 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.910 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.910 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.910 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.910 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.910 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.910 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.910 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.910 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.911 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.911 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.911 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.911 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.911 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.911 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.911 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.911 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.911 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.911 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.911 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.911 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.911 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.911 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.912 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.912 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.912 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.912 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.912 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.912 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.912 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.912 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.912 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.912 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.912 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.912 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.912 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.912 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.913 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.913 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.913 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.913 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.913 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.913 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.913 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.913 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.913 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.913 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.913 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.913 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.913 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.914 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.914 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.914 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.914 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.914 INFO:teuthology.orchestra.run.smithi007.stdout: "min_score": { 2024-06-10T01:32:14.914 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_score", 2024-06-10T01:32:14.914 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-10T01:32:14.914 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.914 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.914 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-06-10T01:32:14.914 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.914 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.914 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.915 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-10T01:32:14.915 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.915 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.915 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.915 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-06-10T01:32:14.915 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-06-10T01:32:14.915 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.915 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.915 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.915 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "upmap", 2024-06-10T01:32:14.915 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.915 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.915 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.916 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-compat", 2024-06-10T01:32:14.916 INFO:teuthology.orchestra.run.smithi007.stdout: "none", 2024-06-10T01:32:14.916 INFO:teuthology.orchestra.run.smithi007.stdout: "read", 2024-06-10T01:32:14.916 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap", 2024-06-10T01:32:14.916 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap-read" 2024-06-10T01:32:14.916 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.916 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Balancer mode", 2024-06-10T01:32:14.916 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.916 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.916 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.916 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_ids": { 2024-06-10T01:32:14.916 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_ids", 2024-06-10T01:32:14.916 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.917 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.917 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.917 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.917 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.917 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.917 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.917 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-10T01:32:14.917 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.917 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.917 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.917 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-06-10T01:32:14.917 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-06-10T01:32:14.917 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-06-10T01:32:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-10T01:32:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_deviation": { 2024-06-10T01:32:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_deviation", 2024-06-10T01:32:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-06-10T01:32:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-06-10T01:32:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-10T01:32:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-10T01:32:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_optimizations": { 2024-06-10T01:32:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_optimizations", 2024-06-10T01:32:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-06-10T01:32:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-10T01:32:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-10T01:32:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm", 2024-06-10T01:32:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_down_multiplier": { 2024-06-10T01:32:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_down_multiplier", 2024-06-10T01:32:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-10T01:32:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-06-10T01:32:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-10T01:32:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_refresh_rate": { 2024-06-10T01:32:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_refresh_rate", 2024-06-10T01:32:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "20", 2024-06-10T01:32:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-10T01:32:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_starting_port": { 2024-06-10T01:32:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_starting_port", 2024-06-10T01:32:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4721", 2024-06-10T01:32:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-10T01:32:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.924 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.924 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.924 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_ptrace": { 2024-06-10T01:32:14.924 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_ptrace", 2024-06-10T01:32:14.924 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.924 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.924 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.924 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.924 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.924 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.924 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.924 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-10T01:32:14.924 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-10T01:32:14.924 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.925 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.925 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_interval": { 2024-06-10T01:32:14.925 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_interval", 2024-06-10T01:32:14.925 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:14.925 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.925 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.925 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-06-10T01:32:14.925 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.925 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.925 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.925 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-10T01:32:14.925 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.925 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.925 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.926 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_memory_target_ratio": { 2024-06-10T01:32:14.926 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_memory_target_ratio", 2024-06-10T01:32:14.926 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-10T01:32:14.926 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.926 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.926 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.7", 2024-06-10T01:32:14.926 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.926 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.926 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.926 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-10T01:32:14.926 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.926 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.927 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.927 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm_log_destination": { 2024-06-10T01:32:14.927 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm_log_destination", 2024-06-10T01:32:14.927 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.927 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.927 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.927 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.927 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.927 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.927 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.927 INFO:teuthology.orchestra.run.smithi007.stdout: "file", 2024-06-10T01:32:14.927 INFO:teuthology.orchestra.run.smithi007.stdout: "file,syslog", 2024-06-10T01:32:14.927 INFO:teuthology.orchestra.run.smithi007.stdout: "syslog" 2024-06-10T01:32:14.927 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.928 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-10T01:32:14.928 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.928 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.928 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.928 INFO:teuthology.orchestra.run.smithi007.stdout: "cgroups_split": { 2024-06-10T01:32:14.928 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cgroups_split", 2024-06-10T01:32:14.928 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.928 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.928 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.928 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.928 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.928 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.928 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.929 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-10T01:32:14.929 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.929 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.929 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.929 INFO:teuthology.orchestra.run.smithi007.stdout: "config_checks_enabled": { 2024-06-10T01:32:14.929 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_checks_enabled", 2024-06-10T01:32:14.929 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.929 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.929 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.929 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.929 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.929 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.930 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.930 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-10T01:32:14.930 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.930 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.930 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.930 INFO:teuthology.orchestra.run.smithi007.stdout: "config_dashboard": { 2024-06-10T01:32:14.930 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_dashboard", 2024-06-10T01:32:14.930 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.930 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.930 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.930 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.930 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.931 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.931 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.931 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-10T01:32:14.931 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.931 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.931 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.931 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_alertmanager": { 2024-06-10T01:32:14.931 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_alertmanager", 2024-06-10T01:32:14.931 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.931 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.931 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.931 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-10T01:32:14.932 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.932 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.932 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.932 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-06-10T01:32:14.932 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.932 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.932 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.932 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_base": { 2024-06-10T01:32:14.932 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_base", 2024-06-10T01:32:14.932 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.932 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.932 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.932 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-10T01:32:14.932 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.933 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.933 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.933 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Container image name, without the tag", 2024-06-10T01:32:14.933 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.933 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.933 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.933 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_elasticsearch": { 2024-06-10T01:32:14.933 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_elasticsearch", 2024-06-10T01:32:14.933 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.933 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.933 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.933 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-10T01:32:14.933 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.934 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.934 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.934 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "elasticsearch container image", 2024-06-10T01:32:14.934 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.934 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.934 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.934 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_grafana": { 2024-06-10T01:32:14.934 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_grafana", 2024-06-10T01:32:14.934 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.934 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.934 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.934 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-10T01:32:14.934 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.935 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.935 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.935 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-06-10T01:32:14.935 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.935 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.935 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.935 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_haproxy": { 2024-06-10T01:32:14.935 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_haproxy", 2024-06-10T01:32:14.935 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.935 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.935 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.935 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-10T01:32:14.935 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.936 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.936 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.936 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "HAproxy container image", 2024-06-10T01:32:14.936 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.936 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.936 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.936 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_agent": { 2024-06-10T01:32:14.936 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_agent", 2024-06-10T01:32:14.936 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.936 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.936 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.936 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-10T01:32:14.937 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.937 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.937 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.937 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger agent container image", 2024-06-10T01:32:14.937 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.937 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.937 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.937 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_collector": { 2024-06-10T01:32:14.937 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_collector", 2024-06-10T01:32:14.937 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.937 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.937 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.937 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-10T01:32:14.938 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.938 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.938 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.938 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger collector container image", 2024-06-10T01:32:14.938 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.938 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.938 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.938 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_query": { 2024-06-10T01:32:14.938 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_query", 2024-06-10T01:32:14.938 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.938 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.938 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.939 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-10T01:32:14.939 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.939 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.939 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.939 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger query container image", 2024-06-10T01:32:14.939 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.939 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.939 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.939 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_keepalived": { 2024-06-10T01:32:14.939 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_keepalived", 2024-06-10T01:32:14.939 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.939 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.940 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.940 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-10T01:32:14.940 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.940 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.940 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.940 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Keepalived container image", 2024-06-10T01:32:14.940 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.940 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.940 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.940 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_loki": { 2024-06-10T01:32:14.940 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_loki", 2024-06-10T01:32:14.940 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.940 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.941 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.941 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-10T01:32:14.941 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.941 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.941 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.941 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Loki container image", 2024-06-10T01:32:14.941 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.941 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.941 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.941 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_node_exporter": { 2024-06-10T01:32:14.941 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_node_exporter", 2024-06-10T01:32:14.941 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.941 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.941 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.942 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-10T01:32:14.942 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.942 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.942 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.942 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-06-10T01:32:14.942 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.942 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.942 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.942 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_nvmeof": { 2024-06-10T01:32:14.942 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_nvmeof", 2024-06-10T01:32:14.942 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.942 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.943 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.943 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-10T01:32:14.943 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.943 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.943 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.943 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Nvme-of container image", 2024-06-10T01:32:14.943 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.943 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.943 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.943 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_prometheus": { 2024-06-10T01:32:14.943 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_prometheus", 2024-06-10T01:32:14.943 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.943 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.944 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.944 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-10T01:32:14.944 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.944 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.944 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.944 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-06-10T01:32:14.944 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.944 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.944 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.944 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_promtail": { 2024-06-10T01:32:14.944 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_promtail", 2024-06-10T01:32:14.944 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.944 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.944 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.945 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-10T01:32:14.945 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.945 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.945 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.945 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Promtail container image", 2024-06-10T01:32:14.945 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.945 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.945 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.945 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_samba": { 2024-06-10T01:32:14.945 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_samba", 2024-06-10T01:32:14.945 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.945 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.945 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.945 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-10T01:32:14.946 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.946 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.946 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.946 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Samba/SMB container image", 2024-06-10T01:32:14.946 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.946 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.946 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.946 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_snmp_gateway": { 2024-06-10T01:32:14.946 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_snmp_gateway", 2024-06-10T01:32:14.946 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.946 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.946 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.947 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-10T01:32:14.947 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.947 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.947 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.947 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SNMP Gateway container image", 2024-06-10T01:32:14.947 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.947 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.947 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.947 INFO:teuthology.orchestra.run.smithi007.stdout: "container_init": { 2024-06-10T01:32:14.947 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_init", 2024-06-10T01:32:14.947 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.947 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.947 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.948 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.948 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.948 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.948 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.948 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Run podman/docker with `--init`", 2024-06-10T01:32:14.948 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.948 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.948 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.948 INFO:teuthology.orchestra.run.smithi007.stdout: "daemon_cache_timeout": { 2024-06-10T01:32:14.948 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "daemon_cache_timeout", 2024-06-10T01:32:14.948 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:14.948 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.948 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.949 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-06-10T01:32:14.949 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.949 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.949 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.949 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-10T01:32:14.949 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.949 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.949 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.949 INFO:teuthology.orchestra.run.smithi007.stdout: "default_cephadm_command_timeout": { 2024-06-10T01:32:14.949 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_cephadm_command_timeout", 2024-06-10T01:32:14.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.949 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.949 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.949 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "900", 2024-06-10T01:32:14.950 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.950 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.950 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.950 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-10T01:32:14.950 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.950 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.950 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.950 INFO:teuthology.orchestra.run.smithi007.stdout: "default_registry": { 2024-06-10T01:32:14.950 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_registry", 2024-06-10T01:32:14.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.950 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.950 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.951 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io", 2024-06-10T01:32:14.951 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.951 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.951 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.951 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-10T01:32:14.951 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.951 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.951 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.951 INFO:teuthology.orchestra.run.smithi007.stdout: "device_cache_timeout": { 2024-06-10T01:32:14.951 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_cache_timeout", 2024-06-10T01:32:14.951 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:14.951 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.951 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.951 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1800", 2024-06-10T01:32:14.952 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.952 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.952 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.952 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache device inventory", 2024-06-10T01:32:14.952 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.952 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.952 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.952 INFO:teuthology.orchestra.run.smithi007.stdout: "device_enhanced_scan": { 2024-06-10T01:32:14.952 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_enhanced_scan", 2024-06-10T01:32:14.952 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.952 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.952 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.952 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.952 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.953 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.953 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.953 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-10T01:32:14.953 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.953 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.953 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.953 INFO:teuthology.orchestra.run.smithi007.stdout: "facts_cache_timeout": { 2024-06-10T01:32:14.953 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "facts_cache_timeout", 2024-06-10T01:32:14.953 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:14.953 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.953 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.953 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-06-10T01:32:14.953 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.954 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.954 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.954 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache host facts data", 2024-06-10T01:32:14.954 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.954 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.954 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.954 INFO:teuthology.orchestra.run.smithi007.stdout: "grafana_dashboards_path": { 2024-06-10T01:32:14.954 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "grafana_dashboards_path", 2024-06-10T01:32:14.954 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.954 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.954 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.954 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-10T01:32:14.955 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.955 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.955 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.955 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-10T01:32:14.955 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.955 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.955 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.955 INFO:teuthology.orchestra.run.smithi007.stdout: "host_check_interval": { 2024-06-10T01:32:14.955 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "host_check_interval", 2024-06-10T01:32:14.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:14.955 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.955 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.955 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-06-10T01:32:14.955 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.956 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.956 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.956 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to perform a host check", 2024-06-10T01:32:14.956 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.956 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.956 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.956 INFO:teuthology.orchestra.run.smithi007.stdout: "hw_monitoring": { 2024-06-10T01:32:14.956 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hw_monitoring", 2024-06-10T01:32:14.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.956 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.956 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.956 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.956 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.957 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.957 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.957 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-10T01:32:14.957 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.957 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.957 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.957 INFO:teuthology.orchestra.run.smithi007.stdout: "inventory_list_all": { 2024-06-10T01:32:14.957 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "inventory_list_all", 2024-06-10T01:32:14.957 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.957 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.957 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.957 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.957 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.958 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.958 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.958 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-10T01:32:14.958 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.958 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.958 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.958 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.958 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.958 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.958 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.958 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.958 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.958 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.959 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.959 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.959 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.959 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.959 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.959 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.959 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.959 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.959 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.959 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.959 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.959 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.959 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.959 INFO:teuthology.orchestra.run.smithi007.stdout: "log_refresh_metadata": { 2024-06-10T01:32:14.960 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_refresh_metadata", 2024-06-10T01:32:14.960 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.960 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.960 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.960 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.960 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.960 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.960 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.960 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-10T01:32:14.960 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.960 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.960 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.961 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.961 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.961 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.961 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.961 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.961 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.961 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.961 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.961 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-10T01:32:14.961 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.961 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.961 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.961 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.962 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.962 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.962 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.962 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.962 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.962 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.962 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.962 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.962 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.962 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.962 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.962 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.962 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.963 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.963 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.963 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.963 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.963 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.963 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.963 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.963 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.963 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.963 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.963 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.963 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.964 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.964 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.964 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.964 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.964 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.964 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.964 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.964 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-10T01:32:14.964 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-10T01:32:14.964 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.964 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.964 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.964 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.965 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.965 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.965 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.965 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-10T01:32:14.965 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.965 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.965 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.965 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-10T01:32:14.965 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-10T01:32:14.965 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.965 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.965 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.965 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "*", 2024-06-10T01:32:14.965 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.966 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.966 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.966 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-10T01:32:14.966 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.966 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.966 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.966 INFO:teuthology.orchestra.run.smithi007.stdout: "max_count_per_host": { 2024-06-10T01:32:14.966 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_count_per_host", 2024-06-10T01:32:14.966 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.966 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.966 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.966 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-10T01:32:14.967 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.967 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.967 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.967 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of daemons per service per host", 2024-06-10T01:32:14.967 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.967 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.967 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.967 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd_draining_count": { 2024-06-10T01:32:14.967 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_osd_draining_count", 2024-06-10T01:32:14.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.967 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.967 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.967 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-10T01:32:14.968 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.968 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.968 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.968 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-10T01:32:14.968 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.968 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.968 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.968 INFO:teuthology.orchestra.run.smithi007.stdout: "migration_current": { 2024-06-10T01:32:14.968 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "migration_current", 2024-06-10T01:32:14.968 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.968 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.968 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.968 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.968 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.969 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.969 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.969 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "internal - do not modify", 2024-06-10T01:32:14.969 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.969 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.969 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.969 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-06-10T01:32:14.969 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-06-10T01:32:14.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.969 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.969 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.969 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "root", 2024-06-10T01:32:14.969 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.970 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.970 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.970 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm-package", 2024-06-10T01:32:14.970 INFO:teuthology.orchestra.run.smithi007.stdout: "root" 2024-06-10T01:32:14.970 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.970 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "mode for remote execution of cephadm", 2024-06-10T01:32:14.970 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.970 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.970 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.970 INFO:teuthology.orchestra.run.smithi007.stdout: "oob_default_addr": { 2024-06-10T01:32:14.970 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "oob_default_addr", 2024-06-10T01:32:14.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.970 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.971 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.971 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "169.254.1.1", 2024-06-10T01:32:14.971 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.971 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.971 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.971 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-10T01:32:14.971 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.971 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.971 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.971 INFO:teuthology.orchestra.run.smithi007.stdout: "prometheus_alerts_path": { 2024-06-10T01:32:14.971 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus_alerts_path", 2024-06-10T01:32:14.971 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.971 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.971 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.972 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-10T01:32:14.972 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.972 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.972 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.972 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-10T01:32:14.972 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.972 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.972 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.972 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_insecure": { 2024-06-10T01:32:14.972 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_insecure", 2024-06-10T01:32:14.972 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.972 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.972 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.973 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.973 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.973 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.973 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.973 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-10T01:32:14.973 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.973 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.973 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.973 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_password": { 2024-06-10T01:32:14.973 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_password", 2024-06-10T01:32:14.973 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.973 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.973 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.974 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.974 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.974 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.974 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.974 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-10T01:32:14.974 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.974 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.974 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.974 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_url": { 2024-06-10T01:32:14.974 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_url", 2024-06-10T01:32:14.974 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.974 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.974 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.975 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.975 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.975 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.975 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.975 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-10T01:32:14.975 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.975 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.975 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.975 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_username": { 2024-06-10T01:32:14.975 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_username", 2024-06-10T01:32:14.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.975 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.975 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.975 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.976 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.976 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.976 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.976 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-10T01:32:14.976 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.976 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.976 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.976 INFO:teuthology.orchestra.run.smithi007.stdout: "secure_monitoring_stack": { 2024-06-10T01:32:14.976 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "secure_monitoring_stack", 2024-06-10T01:32:14.976 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.976 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.976 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.976 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.976 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.977 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.977 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.977 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-10T01:32:14.977 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.977 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.977 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.977 INFO:teuthology.orchestra.run.smithi007.stdout: "service_discovery_port": { 2024-06-10T01:32:14.977 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "service_discovery_port", 2024-06-10T01:32:14.977 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.977 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.977 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.977 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8765", 2024-06-10T01:32:14.977 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.977 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.978 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.978 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "cephadm service discovery port", 2024-06-10T01:32:14.978 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.978 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.978 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.978 INFO:teuthology.orchestra.run.smithi007.stdout: "ssh_config_file": { 2024-06-10T01:32:14.978 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssh_config_file", 2024-06-10T01:32:14.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.978 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.978 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.978 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.978 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.978 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.979 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.979 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-10T01:32:14.979 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.979 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.979 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.979 INFO:teuthology.orchestra.run.smithi007.stdout: "use_agent": { 2024-06-10T01:32:14.979 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_agent", 2024-06-10T01:32:14.979 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.979 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.979 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.979 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.979 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.979 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.979 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.979 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-10T01:32:14.980 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.980 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.980 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.980 INFO:teuthology.orchestra.run.smithi007.stdout: "use_repo_digest": { 2024-06-10T01:32:14.980 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_repo_digest", 2024-06-10T01:32:14.980 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.980 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.980 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.980 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.980 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.980 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.980 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.980 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-10T01:32:14.981 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.981 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.981 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.981 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_failed_host_check": { 2024-06-10T01:32:14.981 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_failed_host_check", 2024-06-10T01:32:14.981 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.981 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.981 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.981 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.981 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.981 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.981 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.981 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if the host check fails", 2024-06-10T01:32:14.982 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.982 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.982 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.982 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_daemons": { 2024-06-10T01:32:14.982 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_daemons", 2024-06-10T01:32:14.982 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.982 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.982 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.982 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.982 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.982 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.982 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.982 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-10T01:32:14.982 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.983 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.983 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.983 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_hosts": { 2024-06-10T01:32:14.983 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_hosts", 2024-06-10T01:32:14.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.983 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.983 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.983 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.983 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.983 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.983 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.983 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-10T01:32:14.983 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.983 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.984 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.984 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.984 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.984 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crash", 2024-06-10T01:32:14.984 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.984 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.984 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.984 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:14.984 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:14.984 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.984 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.984 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.984 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.985 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.985 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.985 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.985 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.985 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.985 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.985 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.985 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.985 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.985 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.985 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.985 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.985 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.985 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.986 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:14.986 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:14.986 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.986 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.986 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.986 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.986 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.986 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.986 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.986 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.986 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.986 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.987 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.987 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:14.987 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:14.987 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.987 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.987 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.987 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:14.987 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.987 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.987 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:14.987 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:14.987 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:14.987 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:14.987 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:14.988 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:14.988 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:14.988 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:14.988 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.988 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.988 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.988 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.988 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:14.988 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:14.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.988 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.988 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.988 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.989 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.989 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.989 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.989 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.989 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.989 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.989 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.989 INFO:teuthology.orchestra.run.smithi007.stdout: "retain_interval": { 2024-06-10T01:32:14.989 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retain_interval", 2024-06-10T01:32:14.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:14.989 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.989 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.989 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "31536000", 2024-06-10T01:32:14.989 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.990 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.990 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.990 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-10T01:32:14.990 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.990 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.990 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.990 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_recent_interval": { 2024-06-10T01:32:14.990 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_recent_interval", 2024-06-10T01:32:14.990 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:14.990 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.990 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:14.990 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1209600", 2024-06-10T01:32:14.990 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.991 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.991 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.991 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-10T01:32:14.991 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.991 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.991 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.991 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.991 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:14.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:14.991 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dashboard", 2024-06-10T01:32:14.991 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:14.991 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:14.991 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:14.992 INFO:teuthology.orchestra.run.smithi007.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-10T01:32:14.992 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-10T01:32:14.992 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:14.992 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.992 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.992 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-10T01:32:14.992 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.992 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.992 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.992 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.992 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.992 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.992 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.992 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_HOST": { 2024-06-10T01:32:14.993 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-10T01:32:14.993 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:14.993 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.993 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.993 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:14.993 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.993 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.993 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.993 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.993 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.993 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.993 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.993 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-10T01:32:14.993 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-10T01:32:14.994 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.994 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.994 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.994 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.994 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.994 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.994 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.994 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.994 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.994 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.994 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.994 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_ENABLED": { 2024-06-10T01:32:14.994 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_ENABLED", 2024-06-10T01:32:14.995 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.995 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.995 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.995 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:14.995 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.995 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.995 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.995 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.995 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.995 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.995 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.995 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-10T01:32:14.995 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-10T01:32:14.995 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.996 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.996 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.996 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.996 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.996 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.996 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.996 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.996 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.996 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.996 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.996 INFO:teuthology.orchestra.run.smithi007.stdout: "ENABLE_BROWSABLE_API": { 2024-06-10T01:32:14.996 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-10T01:32:14.996 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.996 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.997 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.997 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.997 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.997 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.997 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.997 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.997 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.997 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.997 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.997 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-10T01:32:14.997 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-10T01:32:14.997 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.997 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.998 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.998 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.998 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.998 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.998 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.998 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.998 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.998 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.998 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.998 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-10T01:32:14.998 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-10T01:32:14.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:14.999 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:14.999 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:14.999 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:14.999 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:14.999 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:14.999 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:14.999 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:14.999 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:14.999 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:14.999 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:14.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:14.999 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-10T01:32:14.999 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-10T01:32:15.000 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.000 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.000 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.000 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.000 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.000 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.000 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.000 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.000 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.000 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.000 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.000 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-10T01:32:15.000 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-10T01:32:15.000 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.000 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.001 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.001 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.001 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.001 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.001 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.001 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.001 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.001 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.001 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.001 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-10T01:32:15.001 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-10T01:32:15.001 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.001 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.002 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.002 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.002 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.002 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.002 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.002 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.002 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.002 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.002 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.002 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-10T01:32:15.002 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-10T01:32:15.002 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.003 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.003 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.003 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.003 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.003 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.003 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.003 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.003 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.003 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.003 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.003 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-10T01:32:15.003 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-10T01:32:15.003 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.004 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.004 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.004 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.004 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.004 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.004 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.004 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.004 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.004 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.004 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.004 INFO:teuthology.orchestra.run.smithi007.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-10T01:32:15.004 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-10T01:32:15.004 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.004 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.005 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.005 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.005 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.005 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.005 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.005 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.005 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.005 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.005 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.005 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_PASSWORD": { 2024-06-10T01:32:15.005 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-10T01:32:15.005 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.005 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.006 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.006 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-06-10T01:32:15.006 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.006 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.006 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.006 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.006 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.006 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.006 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.006 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-10T01:32:15.006 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-10T01:32:15.007 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.007 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.007 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.007 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.007 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.007 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.007 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.007 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.007 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.007 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.007 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.007 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_URL": { 2024-06-10T01:32:15.007 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_URL", 2024-06-10T01:32:15.008 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.008 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.008 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.008 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.008 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.008 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.008 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.008 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.008 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.008 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.008 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.008 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_USERNAME": { 2024-06-10T01:32:15.008 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-10T01:32:15.009 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.009 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.009 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.009 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-06-10T01:32:15.009 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.009 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.009 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.009 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.009 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.009 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.009 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.009 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-10T01:32:15.009 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-10T01:32:15.009 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.010 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.010 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.010 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.010 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.010 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.010 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.010 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.010 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.010 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.010 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.010 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-10T01:32:15.010 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-10T01:32:15.011 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.011 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.011 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.011 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.011 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.011 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.011 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.011 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.011 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.011 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.011 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.011 INFO:teuthology.orchestra.run.smithi007.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-10T01:32:15.011 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-10T01:32:15.011 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.012 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.012 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.012 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.012 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.012 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.012 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.012 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.012 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.012 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.012 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.012 INFO:teuthology.orchestra.run.smithi007.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-10T01:32:15.012 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-10T01:32:15.012 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.013 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.013 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.013 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.013 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.013 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.013 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.013 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.013 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.013 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.013 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.013 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_HOST": { 2024-06-10T01:32:15.013 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-10T01:32:15.013 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.014 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.014 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.014 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.014 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.014 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.014 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.014 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.014 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.014 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.014 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.014 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-10T01:32:15.015 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-10T01:32:15.015 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.015 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.015 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.015 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.015 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.015 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.015 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.015 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.015 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.015 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.015 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.015 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-10T01:32:15.015 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-10T01:32:15.016 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.016 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.016 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.016 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.016 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.016 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.016 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.016 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.016 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.016 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.016 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.016 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-10T01:32:15.016 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-10T01:32:15.016 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.017 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.017 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.017 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.017 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.017 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.017 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.017 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.017 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.017 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.017 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.017 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-10T01:32:15.017 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-10T01:32:15.018 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.018 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.018 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.018 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.018 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.018 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.018 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.018 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.018 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.018 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.018 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.018 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-10T01:32:15.018 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-10T01:32:15.019 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.019 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.019 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.019 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.019 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.019 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.019 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.019 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.019 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.019 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.019 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.019 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-10T01:32:15.019 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-10T01:32:15.019 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.020 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.020 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.020 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.020 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.020 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.020 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.020 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.020 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.020 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.020 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.020 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-10T01:32:15.020 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-10T01:32:15.020 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.020 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.021 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.021 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.021 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.021 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.021 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.021 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.021 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.021 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.021 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.021 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-10T01:32:15.021 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-10T01:32:15.021 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.021 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.022 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.022 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.022 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.022 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.022 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.022 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.022 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.022 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.022 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.022 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_ENABLED": { 2024-06-10T01:32:15.022 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-10T01:32:15.022 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.023 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.023 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.023 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.023 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.023 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.023 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.023 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.023 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.023 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.023 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.023 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-10T01:32:15.023 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-10T01:32:15.023 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.023 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.024 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.024 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-10T01:32:15.024 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.024 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.024 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.024 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.024 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.024 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.024 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.024 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-10T01:32:15.024 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-10T01:32:15.024 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.024 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.025 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.025 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-10T01:32:15.025 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.025 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.025 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.025 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.025 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.025 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.025 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.025 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-10T01:32:15.025 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-10T01:32:15.025 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.026 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.026 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.026 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8", 2024-06-10T01:32:15.026 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.026 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.026 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.026 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.026 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.026 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.026 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.026 INFO:teuthology.orchestra.run.smithi007.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-10T01:32:15.026 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-10T01:32:15.027 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.027 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.027 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.027 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "45", 2024-06-10T01:32:15.027 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.027 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.027 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.027 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.027 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.027 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.027 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.027 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ACCESS_KEY": { 2024-06-10T01:32:15.027 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-10T01:32:15.028 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.028 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.028 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.028 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.028 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.028 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.028 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.028 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.028 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.028 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.028 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.028 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-10T01:32:15.028 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-10T01:32:15.029 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.029 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.029 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.029 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-06-10T01:32:15.029 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.029 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.029 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.029 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.029 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.029 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.029 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.029 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SECRET_KEY": { 2024-06-10T01:32:15.030 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-10T01:32:15.030 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.030 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.030 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.030 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.030 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.030 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.030 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.030 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.030 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.030 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.030 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.031 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SSL_VERIFY": { 2024-06-10T01:32:15.031 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-10T01:32:15.031 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.031 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.031 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.031 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.031 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.031 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.031 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.031 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.031 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.031 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.031 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.032 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-10T01:32:15.032 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-10T01:32:15.032 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.032 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.032 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.032 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-06-10T01:32:15.032 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.032 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.032 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.032 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.032 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.032 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.032 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.033 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-10T01:32:15.033 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-10T01:32:15.033 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.033 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.033 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.033 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-10T01:32:15.033 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.033 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.033 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.033 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.033 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.033 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.033 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.034 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-10T01:32:15.034 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-10T01:32:15.034 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.034 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.034 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.034 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-06-10T01:32:15.034 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.034 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.034 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.034 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.034 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.034 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.035 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.035 INFO:teuthology.orchestra.run.smithi007.stdout: "cross_origin_url": { 2024-06-10T01:32:15.035 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cross_origin_url", 2024-06-10T01:32:15.035 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.035 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.035 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.035 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.035 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.035 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.035 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.035 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.035 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.035 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.036 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.036 INFO:teuthology.orchestra.run.smithi007.stdout: "crt_file": { 2024-06-10T01:32:15.036 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crt_file", 2024-06-10T01:32:15.036 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.036 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.036 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.036 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.036 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.036 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.036 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.036 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.036 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.036 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.037 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.037 INFO:teuthology.orchestra.run.smithi007.stdout: "debug": { 2024-06-10T01:32:15.037 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "debug", 2024-06-10T01:32:15.037 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.037 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.037 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.037 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.037 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.037 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.037 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.037 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable/disable debug options", 2024-06-10T01:32:15.037 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.037 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.038 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.038 INFO:teuthology.orchestra.run.smithi007.stdout: "jwt_token_ttl": { 2024-06-10T01:32:15.038 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "jwt_token_ttl", 2024-06-10T01:32:15.038 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.038 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.038 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.038 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "28800", 2024-06-10T01:32:15.038 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.038 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.038 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.038 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.039 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.039 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.039 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.039 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-06-10T01:32:15.039 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-06-10T01:32:15.039 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.039 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.039 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.039 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.039 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.039 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.039 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.040 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.040 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.040 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.040 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.040 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.040 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.040 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.040 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.040 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.040 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.040 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.041 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.041 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.041 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.041 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.041 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.041 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.041 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.041 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.041 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.041 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.041 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.041 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.041 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.042 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.042 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.042 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.042 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.042 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.042 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.042 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.042 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.042 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.042 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.042 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.042 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.042 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.043 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.043 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.043 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.043 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.043 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.043 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.043 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.043 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.043 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.043 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.043 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.043 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.044 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.044 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.044 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.044 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.044 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.044 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.044 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.044 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.044 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.044 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.044 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.044 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.045 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.045 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.045 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.045 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.045 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.045 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.045 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.045 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.045 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.045 INFO:teuthology.orchestra.run.smithi007.stdout: "motd": { 2024-06-10T01:32:15.045 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "motd", 2024-06-10T01:32:15.045 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.045 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.046 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.046 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.046 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.046 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.046 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.046 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "The message of the day", 2024-06-10T01:32:15.046 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.046 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.046 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.046 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect_resolve_ip_addr": { 2024-06-10T01:32:15.046 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "redirect_resolve_ip_addr", 2024-06-10T01:32:15.047 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.047 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.047 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.047 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.047 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.047 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.047 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.047 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.047 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.047 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.047 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.047 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-06-10T01:32:15.047 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-06-10T01:32:15.048 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.048 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.048 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.048 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-06-10T01:32:15.048 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.048 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.048 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.048 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.048 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.048 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.048 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.048 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-06-10T01:32:15.048 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-06-10T01:32:15.049 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.049 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.049 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.049 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8080", 2024-06-10T01:32:15.049 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.049 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.049 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.049 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.049 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.049 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.049 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.049 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-06-10T01:32:15.049 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-06-10T01:32:15.049 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.050 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.050 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.050 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.050 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.050 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.050 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.050 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.050 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.050 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.050 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.050 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl_server_port": { 2024-06-10T01:32:15.051 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl_server_port", 2024-06-10T01:32:15.051 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.051 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.051 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.051 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8443", 2024-06-10T01:32:15.051 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.051 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.051 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.051 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.051 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.051 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.051 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.051 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-06-10T01:32:15.052 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-06-10T01:32:15.052 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.052 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.052 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.052 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "redirect", 2024-06-10T01:32:15.052 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.052 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.052 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.052 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.052 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect" 2024-06-10T01:32:15.052 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.052 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.052 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.052 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.053 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.053 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-06-10T01:32:15.053 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-06-10T01:32:15.053 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.053 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.053 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.053 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-06-10T01:32:15.053 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-06-10T01:32:15.053 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-06-10T01:32:15.053 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.053 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.053 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.053 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.054 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.054 INFO:teuthology.orchestra.run.smithi007.stdout: "url_prefix": { 2024-06-10T01:32:15.054 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url_prefix", 2024-06-10T01:32:15.054 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.054 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.054 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.054 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.054 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.054 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.054 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.054 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.055 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.055 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.055 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.055 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.055 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.055 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-06-10T01:32:15.055 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.055 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.055 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.055 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_monitoring": { 2024-06-10T01:32:15.055 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_monitoring", 2024-06-10T01:32:15.055 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.056 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.056 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.056 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.056 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.056 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.056 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.056 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "monitor device health metrics", 2024-06-10T01:32:15.056 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.056 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.056 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.056 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.056 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.056 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: "mark_out_threshold": { 2024-06-10T01:32:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mark_out_threshold", 2024-06-10T01:32:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2419200", 2024-06-10T01:32:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-10T01:32:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": { 2024-06-10T01:32:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_name", 2024-06-10T01:32:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "device_health_metrics", 2024-06-10T01:32:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-10T01:32:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "retention_period": { 2024-06-10T01:32:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retention_period", 2024-06-10T01:32:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15552000", 2024-06-10T01:32:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain device health metrics", 2024-06-10T01:32:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_frequency": { 2024-06-10T01:32:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_frequency", 2024-06-10T01:32:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-06-10T01:32:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-10T01:32:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: "self_heal": { 2024-06-10T01:32:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "self_heal", 2024-06-10T01:32:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-10T01:32:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-06-10T01:32:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-06-10T01:32:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-06-10T01:32:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and check device health", 2024-06-10T01:32:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_threshold": { 2024-06-10T01:32:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_threshold", 2024-06-10T01:32:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7257600", 2024-06-10T01:32:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-10T01:32:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "diskprediction_local", 2024-06-10T01:32:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "predict_interval": { 2024-06-10T01:32:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predict_interval", 2024-06-10T01:32:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-06-10T01:32:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "predictor_model": { 2024-06-10T01:32:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predictor_model", 2024-06-10T01:32:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "prophetstor", 2024-06-10T01:32:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-06-10T01:32:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-06-10T01:32:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-06-10T01:32:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "influx", 2024-06-10T01:32:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": false, 2024-06-10T01:32:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "influxdb python module not found", 2024-06-10T01:32:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: "batch_size": { 2024-06-10T01:32:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "batch_size", 2024-06-10T01:32:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5000", 2024-06-10T01:32:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-10T01:32:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "database": { 2024-06-10T01:32:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "database", 2024-06-10T01:32:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "ceph", 2024-06-10T01:32:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-10T01:32:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": { 2024-06-10T01:32:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hostname", 2024-06-10T01:32:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server hostname", 2024-06-10T01:32:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-06-10T01:32:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-06-10T01:32:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "30", 2024-06-10T01:32:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "5", 2024-06-10T01:32:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-10T01:32:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.083 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.083 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.083 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.083 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.083 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.083 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.083 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.083 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.083 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.083 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.083 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.083 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.083 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.083 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.084 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.084 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.084 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.084 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.084 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.084 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.084 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.084 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.084 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.084 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.084 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.084 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.084 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.085 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.085 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.085 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.085 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.085 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.085 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.085 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.085 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.085 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.085 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.085 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.085 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.085 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.085 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.085 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.086 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.086 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.086 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.086 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.086 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.086 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.086 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.086 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.086 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.086 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.086 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.086 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.087 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.087 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.087 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.087 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.087 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.087 INFO:teuthology.orchestra.run.smithi007.stdout: "password": { 2024-06-10T01:32:15.087 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "password", 2024-06-10T01:32:15.087 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.087 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.087 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.087 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.087 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.087 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.087 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.088 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "password of InfluxDB server user", 2024-06-10T01:32:15.088 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.088 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.088 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.088 INFO:teuthology.orchestra.run.smithi007.stdout: "port": { 2024-06-10T01:32:15.088 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "port", 2024-06-10T01:32:15.088 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.088 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.088 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.088 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8086", 2024-06-10T01:32:15.088 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.088 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.088 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.088 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server port", 2024-06-10T01:32:15.089 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.089 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.089 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.089 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-06-10T01:32:15.089 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-06-10T01:32:15.089 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.089 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.089 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.089 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "false", 2024-06-10T01:32:15.089 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.089 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.089 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.089 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-10T01:32:15.090 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.090 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.090 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.090 INFO:teuthology.orchestra.run.smithi007.stdout: "threads": { 2024-06-10T01:32:15.090 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threads", 2024-06-10T01:32:15.090 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.090 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.090 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.090 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-06-10T01:32:15.090 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-06-10T01:32:15.090 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "32", 2024-06-10T01:32:15.090 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.091 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-10T01:32:15.091 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.091 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.091 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.091 INFO:teuthology.orchestra.run.smithi007.stdout: "username": { 2024-06-10T01:32:15.091 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "username", 2024-06-10T01:32:15.091 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.091 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.091 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.091 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.091 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.091 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.091 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.092 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "username of InfluxDB server user", 2024-06-10T01:32:15.092 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.092 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.092 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.092 INFO:teuthology.orchestra.run.smithi007.stdout: "verify_ssl": { 2024-06-10T01:32:15.092 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "verify_ssl", 2024-06-10T01:32:15.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.092 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.092 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.092 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "true", 2024-06-10T01:32:15.092 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.092 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.092 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.093 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-10T01:32:15.093 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.093 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.093 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.093 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.093 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.093 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "insights", 2024-06-10T01:32:15.093 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.093 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.093 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.093 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.094 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.094 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.094 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.094 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.094 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.094 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.094 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.094 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.094 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.094 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.094 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.094 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.095 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.095 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.095 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.095 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.095 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.095 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.095 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.095 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.095 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.095 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.095 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.095 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.095 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.096 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.096 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.096 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.096 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.096 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.096 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.096 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.096 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.096 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.096 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.096 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.096 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.097 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.097 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.097 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.097 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.097 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.097 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.097 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.097 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.097 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.097 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.097 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.097 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.097 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.098 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.098 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.098 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.098 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.098 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.098 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.098 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.098 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.098 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.098 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.098 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.098 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.099 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.099 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.099 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.099 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.099 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.099 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "iostat", 2024-06-10T01:32:15.099 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.099 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.099 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.099 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.099 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.100 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.100 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.100 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.100 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.100 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.100 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.100 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.100 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.100 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.100 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.100 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.100 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.100 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.100 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.101 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.101 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.101 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.101 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.101 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.101 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.101 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.101 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.101 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.101 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.101 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.101 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.101 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.102 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.102 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.102 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.102 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.102 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.102 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.102 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.102 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.102 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.102 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.102 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.102 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.103 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.103 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.103 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.103 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.103 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.103 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.103 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.103 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.103 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.103 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.103 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.103 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.103 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.104 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.104 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.104 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.104 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.104 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.104 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.104 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.104 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.104 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.104 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.104 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.104 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.104 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.104 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.105 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "k8sevents", 2024-06-10T01:32:15.105 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.105 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.105 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.105 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_event_retention_days": { 2024-06-10T01:32:15.105 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ceph_event_retention_days", 2024-06-10T01:32:15.105 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.105 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.105 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.105 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7", 2024-06-10T01:32:15.105 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.105 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.105 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.105 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-10T01:32:15.106 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.106 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.106 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.106 INFO:teuthology.orchestra.run.smithi007.stdout: "config_check_secs": { 2024-06-10T01:32:15.106 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_check_secs", 2024-06-10T01:32:15.106 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.106 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.106 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.106 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-10T01:32:15.106 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "10", 2024-06-10T01:32:15.106 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.106 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.107 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-10T01:32:15.107 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.107 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.107 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.107 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.107 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.107 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.107 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.107 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.107 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.107 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.107 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.107 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.108 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.108 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.108 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.108 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.108 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.108 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.108 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.108 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.108 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.108 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.108 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.108 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.108 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.109 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.109 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.109 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.109 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.109 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.109 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.109 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.109 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.109 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.109 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.109 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.109 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.109 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.110 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.110 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.110 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.110 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.110 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.110 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.110 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.110 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.110 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.110 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.110 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.110 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.110 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.111 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.111 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.111 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.111 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.111 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.111 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.111 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.111 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.111 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.111 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.111 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.111 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.111 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.112 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.112 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.112 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.112 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.112 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.112 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.112 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.112 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "localpool", 2024-06-10T01:32:15.112 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.112 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.112 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.112 INFO:teuthology.orchestra.run.smithi007.stdout: "failure_domain": { 2024-06-10T01:32:15.112 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "failure_domain", 2024-06-10T01:32:15.113 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.113 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.113 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.113 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "host", 2024-06-10T01:32:15.113 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.113 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.113 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.113 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "failure domain for any created local pool", 2024-06-10T01:32:15.113 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-10T01:32:15.113 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.113 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.113 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.113 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.113 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.114 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.114 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.114 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.114 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.114 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.114 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.114 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.114 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.114 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.114 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.114 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.114 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.114 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.115 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.115 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.115 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.115 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.115 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.115 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.115 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.115 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.115 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.115 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.115 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.115 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.115 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.115 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.116 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.116 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.116 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.116 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.116 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.116 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.116 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.116 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.116 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.116 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.116 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.116 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.116 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.116 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.117 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.117 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.117 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.117 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.117 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.117 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.117 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.117 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.117 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.117 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.117 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.117 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.117 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.118 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.118 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.118 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.118 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.118 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.118 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.118 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.118 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.118 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.118 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": { 2024-06-10T01:32:15.118 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_size", 2024-06-10T01:32:15.118 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.119 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.119 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.119 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.119 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.119 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.119 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.119 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default min_size for any created local pool", 2024-06-10T01:32:15.119 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-10T01:32:15.119 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.119 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.119 INFO:teuthology.orchestra.run.smithi007.stdout: "num_rep": { 2024-06-10T01:32:15.119 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "num_rep", 2024-06-10T01:32:15.119 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.119 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.120 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.120 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3", 2024-06-10T01:32:15.120 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.120 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.120 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.120 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default replica count for any created local pool", 2024-06-10T01:32:15.120 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.120 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.120 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.120 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": { 2024-06-10T01:32:15.120 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_num", 2024-06-10T01:32:15.120 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.120 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.120 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.121 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "128", 2024-06-10T01:32:15.121 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.121 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.121 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.121 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default pg_num for any created local pool", 2024-06-10T01:32:15.121 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.121 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.121 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.121 INFO:teuthology.orchestra.run.smithi007.stdout: "prefix": { 2024-06-10T01:32:15.121 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prefix", 2024-06-10T01:32:15.122 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.122 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.122 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.122 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.122 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.122 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.122 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.122 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name prefix for any created local pool", 2024-06-10T01:32:15.122 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.122 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.122 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.122 INFO:teuthology.orchestra.run.smithi007.stdout: "subtree": { 2024-06-10T01:32:15.123 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "subtree", 2024-06-10T01:32:15.123 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.123 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.123 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.123 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "rack", 2024-06-10T01:32:15.123 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.123 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.123 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.123 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-10T01:32:15.123 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-10T01:32:15.123 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.123 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.123 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.124 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mds_autoscaler", 2024-06-10T01:32:15.124 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.124 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.124 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.124 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.124 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.124 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.124 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.124 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.124 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.124 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.125 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.125 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.125 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.125 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.125 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.125 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.125 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.125 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.125 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.125 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.125 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.125 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.125 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.126 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.126 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.126 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.126 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.126 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.126 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.126 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.126 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.126 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.126 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.126 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.126 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.127 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.127 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.127 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.127 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.127 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.127 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.127 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.127 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.127 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.127 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.127 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.127 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.128 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.128 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.128 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.128 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.128 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.128 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.128 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.128 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.128 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.128 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.128 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.128 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.129 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.129 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.129 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.129 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.129 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.129 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.129 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.129 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.129 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.129 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.129 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.129 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.130 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirroring", 2024-06-10T01:32:15.130 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.130 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.130 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.130 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.130 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.130 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.130 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.130 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.130 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.130 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.130 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.130 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.131 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.131 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.131 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.131 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.131 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.131 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.131 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.131 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.131 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.131 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.131 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.131 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.131 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.132 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.132 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.132 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.132 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.132 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.132 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.132 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.132 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.132 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.132 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.132 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.132 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.132 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.133 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.133 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.133 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.133 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.133 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.133 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.133 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.133 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.133 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.133 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.133 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.133 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.133 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.133 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.133 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.134 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.134 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.134 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.134 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.134 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.134 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.134 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.134 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.134 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.134 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.134 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.134 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.134 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.135 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.135 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.135 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.135 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.135 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.135 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "nfs", 2024-06-10T01:32:15.135 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.135 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.135 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.135 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.136 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.136 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.136 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.136 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.136 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.136 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.136 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.136 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.136 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.136 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.136 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.136 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.136 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.136 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.137 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.137 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.137 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.137 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.137 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.137 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.137 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.137 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.137 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.137 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.137 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.137 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.137 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.138 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.138 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.138 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.138 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.138 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.138 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.138 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.138 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.138 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.138 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.138 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.138 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.138 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.139 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.139 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.139 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.139 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.139 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.139 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.139 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.139 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.139 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.139 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.139 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.139 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.140 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.140 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.140 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.140 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.140 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.140 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.140 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.140 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.140 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.140 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.140 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.140 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.141 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.141 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.141 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.141 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-06-10T01:32:15.141 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.141 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.141 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.141 INFO:teuthology.orchestra.run.smithi007.stdout: "fail_fs": { 2024-06-10T01:32:15.141 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "fail_fs", 2024-06-10T01:32:15.141 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.141 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.141 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.142 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.142 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.142 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.142 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.142 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-10T01:32:15.142 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.142 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.142 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.142 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.142 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.142 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.143 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.143 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.143 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.143 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.143 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.143 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.143 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.143 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.143 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.143 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.143 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.143 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.143 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.143 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.144 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.144 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.144 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.144 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.144 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.144 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.144 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.144 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.144 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.144 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.144 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.144 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.144 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.145 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.145 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.145 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.145 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.145 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.145 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.145 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.145 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.145 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.145 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.145 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.145 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.145 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.146 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.146 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.146 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.146 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.146 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.146 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.146 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.146 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.146 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.146 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.146 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.146 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.147 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.147 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.147 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.147 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.147 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.147 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.147 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.147 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.147 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.147 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.147 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.147 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator": { 2024-06-10T01:32:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-06-10T01:32:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-06-10T01:32:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "rook", 2024-06-10T01:32:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "test_orchestrator" 2024-06-10T01:32:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Orchestrator backend", 2024-06-10T01:32:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_perf_query", 2024-06-10T01:32:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.154 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.154 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.154 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.154 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.154 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.154 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.154 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.154 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.154 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.154 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.154 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.154 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_support", 2024-06-10T01:32:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_autoscaler", 2024-06-10T01:32:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-06-10T01:32:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-06-10T01:32:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-06-10T01:32:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: "threshold": { 2024-06-10T01:32:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threshold", 2024-06-10T01:32:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-10T01:32:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-06-10T01:32:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1.0", 2024-06-10T01:32:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "scaling threshold", 2024-06-10T01:32:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-10T01:32:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "progress", 2024-06-10T01:32:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_pg_recovery_event": { 2024-06-10T01:32:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_pg_recovery_event", 2024-06-10T01:32:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-10T01:32:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-06-10T01:32:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-06-10T01:32:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.171 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.171 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.171 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.171 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.171 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.171 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.171 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.171 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.171 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.171 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.171 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.171 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.171 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.171 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.172 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.172 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.172 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.172 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.172 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.172 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.172 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.172 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.172 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.172 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.172 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.172 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.172 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.173 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.173 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.173 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.173 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.173 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.173 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.173 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.173 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.173 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.173 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.173 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.173 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.173 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.173 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.173 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.173 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.174 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.174 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.174 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.174 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.174 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.174 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.174 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.174 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.174 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.174 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.174 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.174 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.175 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.175 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.175 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.175 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.175 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.175 INFO:teuthology.orchestra.run.smithi007.stdout: "max_completed_events": { 2024-06-10T01:32:15.175 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_completed_events", 2024-06-10T01:32:15.175 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.175 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.175 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.175 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "50", 2024-06-10T01:32:15.175 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.175 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.175 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.175 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "number of past completed events to remember", 2024-06-10T01:32:15.176 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.176 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.176 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.176 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-06-10T01:32:15.176 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-06-10T01:32:15.176 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:15.176 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.176 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.176 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-06-10T01:32:15.176 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.176 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.176 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.176 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long the module is going to sleep", 2024-06-10T01:32:15.177 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.177 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.177 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.177 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.177 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.177 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus", 2024-06-10T01:32:15.177 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.177 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.177 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.177 INFO:teuthology.orchestra.run.smithi007.stdout: "cache": { 2024-06-10T01:32:15.177 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cache", 2024-06-10T01:32:15.177 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.177 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.178 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.178 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.178 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.178 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.178 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.178 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.178 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.178 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.178 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.178 INFO:teuthology.orchestra.run.smithi007.stdout: "exclude_perf_counters": { 2024-06-10T01:32:15.178 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "exclude_perf_counters", 2024-06-10T01:32:15.178 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.179 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.179 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.179 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.179 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.179 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.179 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.179 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-10T01:32:15.179 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-10T01:32:15.179 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.179 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.179 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.179 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.180 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.180 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.180 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.180 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.180 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.180 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.180 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.180 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.180 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.180 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.180 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.180 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.180 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.180 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.181 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.181 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.181 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.181 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.181 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.181 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.181 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.181 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.181 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.181 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.181 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.181 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.182 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.182 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.182 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.182 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.182 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.182 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.182 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.182 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.182 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.182 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.182 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.182 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.183 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.183 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.183 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.183 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.183 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.183 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.183 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.183 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.183 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.183 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.183 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.183 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.183 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.184 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.184 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.184 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.184 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.184 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.184 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.184 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.184 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.184 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.184 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.184 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.184 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.184 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.185 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools": { 2024-06-10T01:32:15.185 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools", 2024-06-10T01:32:15.185 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.185 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.185 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.185 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.185 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.185 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.185 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.185 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.185 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.185 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.185 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.186 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-10T01:32:15.186 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-10T01:32:15.186 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.186 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.186 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.186 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "300", 2024-06-10T01:32:15.186 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.186 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.186 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.186 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.186 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.186 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.186 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_interval": { 2024-06-10T01:32:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_interval", 2024-06-10T01:32:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-10T01:32:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15.0", 2024-06-10T01:32:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-06-10T01:32:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-06-10T01:32:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-06-10T01:32:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-10T01:32:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-06-10T01:32:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-06-10T01:32:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "9283", 2024-06-10T01:32:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-10T01:32:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.190 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.190 INFO:teuthology.orchestra.run.smithi007.stdout: "stale_cache_strategy": { 2024-06-10T01:32:15.190 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stale_cache_strategy", 2024-06-10T01:32:15.190 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.190 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.190 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.190 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "log", 2024-06-10T01:32:15.190 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.190 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.190 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.190 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.190 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.191 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.191 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.191 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-06-10T01:32:15.191 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-06-10T01:32:15.191 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.191 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.191 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.191 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "default", 2024-06-10T01:32:15.191 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.191 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.191 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.191 INFO:teuthology.orchestra.run.smithi007.stdout: "default", 2024-06-10T01:32:15.191 INFO:teuthology.orchestra.run.smithi007.stdout: "error" 2024-06-10T01:32:15.192 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.192 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.192 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.192 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.192 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.192 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-06-10T01:32:15.192 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-06-10T01:32:15.192 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.192 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.192 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.192 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-06-10T01:32:15.192 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-06-10T01:32:15.193 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-06-10T01:32:15.193 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.193 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.193 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.193 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.193 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.193 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.193 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.193 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-06-10T01:32:15.193 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.193 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.193 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.194 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.194 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.194 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.194 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.194 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.194 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.194 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.194 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.194 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.194 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.194 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.194 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.194 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.195 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.195 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.195 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.195 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.195 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.195 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.195 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.195 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.195 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.195 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.195 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.195 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.196 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.196 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.196 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.196 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.196 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.196 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.196 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.196 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.196 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.196 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.196 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.196 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.196 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.197 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.197 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.197 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.197 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.197 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.197 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.197 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.197 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.197 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.197 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.197 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.197 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.197 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.197 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.198 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.198 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.198 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.198 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.198 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.198 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.198 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.198 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.198 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.198 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.198 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.198 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.199 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.199 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.199 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_snap_create": { 2024-06-10T01:32:15.199 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_snap_create", 2024-06-10T01:32:15.199 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.199 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.199 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.199 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-10T01:32:15.199 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.199 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.199 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.199 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.199 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.200 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.200 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.200 INFO:teuthology.orchestra.run.smithi007.stdout: "mirror_snapshot_schedule": { 2024-06-10T01:32:15.200 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirror_snapshot_schedule", 2024-06-10T01:32:15.200 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.200 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.200 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.200 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.200 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.200 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.200 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.200 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.200 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.201 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.201 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.201 INFO:teuthology.orchestra.run.smithi007.stdout: "trash_purge_schedule": { 2024-06-10T01:32:15.201 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "trash_purge_schedule", 2024-06-10T01:32:15.201 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.201 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.201 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.201 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.201 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.201 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.201 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.201 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.201 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.202 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.202 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.202 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.202 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.202 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "restful", 2024-06-10T01:32:15.202 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.202 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.202 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.202 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_auth": { 2024-06-10T01:32:15.202 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_auth", 2024-06-10T01:32:15.202 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.203 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.203 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.203 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.203 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.203 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.203 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.203 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.203 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.203 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.203 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.203 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-06-10T01:32:15.203 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-06-10T01:32:15.203 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.204 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.204 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.204 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.204 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.204 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.204 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.204 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.204 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.204 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.204 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.204 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.204 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.204 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.205 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.205 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.205 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.205 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.205 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.205 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.205 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.205 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.205 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.205 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.205 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.205 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.205 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.205 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.206 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.206 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.206 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.206 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.206 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.206 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.206 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.206 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.206 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.206 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.206 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.206 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.207 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.207 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.207 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.207 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.207 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.207 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.207 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.207 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.207 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.207 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.207 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.207 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.208 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.208 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.208 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.208 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.208 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.208 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.208 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.208 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.208 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.208 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.208 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.208 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.208 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.209 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.209 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.209 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.209 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.209 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.209 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.209 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.209 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.209 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.209 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.209 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.209 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.210 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-06-10T01:32:15.210 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-06-10T01:32:15.210 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.210 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.210 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.210 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.210 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.210 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.210 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.210 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.210 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.210 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-06-10T01:32:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-06-10T01:32:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rgw", 2024-06-10T01:32:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rook", 2024-06-10T01:32:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "storage_class": { 2024-06-10T01:32:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "storage_class", 2024-06-10T01:32:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "local", 2024-06-10T01:32:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-10T01:32:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "selftest", 2024-06-10T01:32:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: "roption1": { 2024-06-10T01:32:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption1", 2024-06-10T01:32:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "roption2": { 2024-06-10T01:32:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption2", 2024-06-10T01:32:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "xyz", 2024-06-10T01:32:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption1": { 2024-06-10T01:32:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption1", 2024-06-10T01:32:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption2": { 2024-06-10T01:32:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption2", 2024-06-10T01:32:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption3": { 2024-06-10T01:32:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption3", 2024-06-10T01:32:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-10T01:32:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption4": { 2024-06-10T01:32:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption4", 2024-06-10T01:32:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption5": { 2024-06-10T01:32:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption5", 2024-06-10T01:32:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption6": { 2024-06-10T01:32:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption6", 2024-06-10T01:32:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption7": { 2024-06-10T01:32:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption7", 2024-06-10T01:32:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-06-10T01:32:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "42", 2024-06-10T01:32:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: "testkey": { 2024-06-10T01:32:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testkey", 2024-06-10T01:32:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: "testlkey": { 2024-06-10T01:32:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testlkey", 2024-06-10T01:32:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.241 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.241 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.241 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.241 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.241 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.241 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.241 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.241 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.241 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.241 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.241 INFO:teuthology.orchestra.run.smithi007.stdout: "testnewline": { 2024-06-10T01:32:15.241 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testnewline", 2024-06-10T01:32:15.241 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.242 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.242 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.242 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.242 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.242 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.242 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.242 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.242 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.242 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.242 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.242 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.242 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.243 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snap_schedule", 2024-06-10T01:32:15.243 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.243 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.243 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.243 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_m_granularity": { 2024-06-10T01:32:15.243 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_m_granularity", 2024-06-10T01:32:15.243 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.243 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.243 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.243 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.243 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.243 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.243 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.243 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow minute scheduled snapshots", 2024-06-10T01:32:15.243 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.244 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.244 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.244 INFO:teuthology.orchestra.run.smithi007.stdout: "dump_on_update": { 2024-06-10T01:32:15.244 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dump_on_update", 2024-06-10T01:32:15.244 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.244 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.244 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.244 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.244 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.244 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.244 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.244 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "dump database to debug log on update", 2024-06-10T01:32:15.244 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.244 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.245 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.245 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.245 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.245 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.245 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.245 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.245 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.245 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.245 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.245 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.245 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.245 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.245 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.245 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.246 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.246 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.246 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.246 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.246 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.246 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.246 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.246 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.246 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.246 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.246 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.246 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.246 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.247 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.247 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.247 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.247 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.247 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.247 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.247 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.247 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.247 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.247 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.247 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.247 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.247 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.247 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.248 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.248 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.248 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.248 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.248 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.248 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.248 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.248 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.248 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.248 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.248 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.248 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.248 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.249 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.249 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.249 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.249 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.249 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.249 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.249 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.249 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.249 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.249 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.249 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.249 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.249 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.249 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.250 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stats", 2024-06-10T01:32:15.250 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.250 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.250 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.250 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.250 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.250 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.250 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.250 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.250 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.250 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.251 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.251 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.251 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.251 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.251 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.251 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.251 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.251 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.251 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.251 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.251 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.251 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.251 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.252 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.252 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.252 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.252 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.252 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.252 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.252 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.252 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.252 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.252 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.252 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.252 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.252 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.252 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.253 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.253 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.253 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.253 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.253 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.253 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.253 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.253 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.253 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.253 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.253 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.253 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.253 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.253 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.254 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.254 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.254 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.254 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.254 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.254 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.254 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.254 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.254 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.254 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.254 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.254 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.254 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.254 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.255 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.255 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.255 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.255 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.255 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.255 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.255 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "status", 2024-06-10T01:32:15.255 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.255 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.255 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.255 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.255 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.255 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.256 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.256 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.256 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.256 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.256 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.256 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.256 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.256 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.256 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.256 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.256 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.256 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.256 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.256 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.257 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.257 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.257 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.257 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.257 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.257 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.257 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.257 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.257 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.257 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.257 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.257 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.258 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.258 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.258 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.258 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.258 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.258 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.258 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.258 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.258 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.258 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.258 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.259 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.259 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.259 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.261 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.261 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.261 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.261 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.261 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.261 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.261 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.261 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.262 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.262 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.262 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.262 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.262 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.262 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.262 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.262 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.262 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.262 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.262 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.263 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.263 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.263 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.263 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.263 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.263 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telegraf", 2024-06-10T01:32:15.263 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.263 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.263 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.263 INFO:teuthology.orchestra.run.smithi007.stdout: "address": { 2024-06-10T01:32:15.263 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "address", 2024-06-10T01:32:15.264 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.264 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.264 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.264 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-10T01:32:15.264 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.264 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.264 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.264 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.264 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.264 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.264 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.264 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-06-10T01:32:15.265 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-06-10T01:32:15.265 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:15.265 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.265 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.265 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15", 2024-06-10T01:32:15.265 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.265 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.265 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.265 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.265 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.265 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.265 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.266 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.266 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.266 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.266 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.266 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.266 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.266 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.266 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.266 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.266 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.266 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.266 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.267 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.267 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.267 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.267 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.267 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.267 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.267 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.267 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.267 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.267 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.267 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.267 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.267 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.268 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.268 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.268 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.268 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.268 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.268 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.268 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.268 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.268 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.268 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.268 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.268 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.269 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.269 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.269 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.269 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.269 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.269 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.269 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.269 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.269 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.269 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.269 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.269 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.269 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.269 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.270 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.270 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.270 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.270 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.270 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.270 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.270 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.270 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.270 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.270 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.270 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.271 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.271 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.271 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.271 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.271 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.271 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.271 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telemetry", 2024-06-10T01:32:15.271 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.271 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.271 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.271 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_basic": { 2024-06-10T01:32:15.271 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_basic", 2024-06-10T01:32:15.272 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.272 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.272 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.272 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.272 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.272 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.272 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.272 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-10T01:32:15.272 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.272 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.272 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.272 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_crash": { 2024-06-10T01:32:15.272 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_crash", 2024-06-10T01:32:15.272 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.273 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.273 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.273 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.273 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.273 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.273 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.273 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-10T01:32:15.273 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.273 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.273 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.273 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_device": { 2024-06-10T01:32:15.273 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_device", 2024-06-10T01:32:15.273 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.274 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.274 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.274 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.274 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.274 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.274 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.274 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-10T01:32:15.274 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.274 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.274 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.274 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_ident": { 2024-06-10T01:32:15.275 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_ident", 2024-06-10T01:32:15.275 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.275 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.275 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.275 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.275 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.275 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.275 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.275 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-10T01:32:15.275 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.275 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.275 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.275 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_perf": { 2024-06-10T01:32:15.275 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_perf", 2024-06-10T01:32:15.275 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.276 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.276 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.276 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.276 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.276 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.276 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.276 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-10T01:32:15.276 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.276 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.276 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.276 INFO:teuthology.orchestra.run.smithi007.stdout: "contact": { 2024-06-10T01:32:15.276 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "contact", 2024-06-10T01:32:15.276 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.276 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.277 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.277 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.277 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.277 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.277 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.277 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.277 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.277 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.277 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.277 INFO:teuthology.orchestra.run.smithi007.stdout: "description": { 2024-06-10T01:32:15.277 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "description", 2024-06-10T01:32:15.277 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.277 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.277 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.278 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.278 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.278 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.278 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.278 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.278 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.278 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.278 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.278 INFO:teuthology.orchestra.run.smithi007.stdout: "device_url": { 2024-06-10T01:32:15.278 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_url", 2024-06-10T01:32:15.278 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.278 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.279 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.279 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-10T01:32:15.279 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.279 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.279 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.279 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.279 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.279 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.279 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.279 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-06-10T01:32:15.279 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-06-10T01:32:15.279 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.279 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.280 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.280 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.280 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.280 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.280 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.280 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.280 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.280 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.280 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.280 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-06-10T01:32:15.280 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-06-10T01:32:15.280 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.280 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.280 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.281 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "24", 2024-06-10T01:32:15.281 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "8", 2024-06-10T01:32:15.281 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.281 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.281 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.281 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.281 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.281 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.281 INFO:teuthology.orchestra.run.smithi007.stdout: "last_opt_revision": { 2024-06-10T01:32:15.281 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "last_opt_revision", 2024-06-10T01:32:15.281 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.281 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.281 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.282 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1", 2024-06-10T01:32:15.282 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.282 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.282 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.282 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.282 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.282 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.282 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.282 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard": { 2024-06-10T01:32:15.282 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard", 2024-06-10T01:32:15.282 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.282 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.282 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.283 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.283 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.283 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.283 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.283 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.283 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.283 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.283 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.283 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard_description": { 2024-06-10T01:32:15.283 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard_description", 2024-06-10T01:32:15.283 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.283 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.283 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.284 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.284 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.284 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.284 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.284 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.284 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.284 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.284 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.284 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.284 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.284 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.284 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.284 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.284 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.285 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.285 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.285 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.285 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.285 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.285 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.285 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.285 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.285 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.285 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.285 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.285 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.285 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.285 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.286 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.286 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.286 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.286 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.286 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.286 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.286 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.286 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.286 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.286 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.286 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.286 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.286 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.287 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.287 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.287 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.287 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.287 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.287 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.287 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.287 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.287 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.287 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.287 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.287 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.287 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.287 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.288 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.288 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.288 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.288 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.288 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.288 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.288 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.288 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.288 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.288 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.288 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.288 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.288 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.288 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.289 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.289 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.289 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.289 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.289 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.289 INFO:teuthology.orchestra.run.smithi007.stdout: "organization": { 2024-06-10T01:32:15.289 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "organization", 2024-06-10T01:32:15.289 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.289 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.289 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.289 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.289 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.289 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.289 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.290 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.290 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.290 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.290 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.290 INFO:teuthology.orchestra.run.smithi007.stdout: "proxy": { 2024-06-10T01:32:15.290 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "proxy", 2024-06-10T01:32:15.290 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.290 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.290 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.290 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.290 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.290 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.290 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.291 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.291 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.291 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.291 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.291 INFO:teuthology.orchestra.run.smithi007.stdout: "url": { 2024-06-10T01:32:15.291 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url", 2024-06-10T01:32:15.291 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.291 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.291 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.291 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-10T01:32:15.291 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.291 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.291 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.291 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.292 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.292 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.292 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.292 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.292 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.292 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "test_orchestrator", 2024-06-10T01:32:15.292 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.292 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.292 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.292 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.292 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.293 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.293 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.293 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.293 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.293 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.293 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.293 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.293 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.293 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.293 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.293 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.293 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.293 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.293 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.293 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.294 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.294 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.294 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.294 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.294 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.294 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.294 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.294 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.294 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.294 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.294 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.294 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.294 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.295 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.295 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.295 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.295 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.295 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.295 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.295 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.295 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.295 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.295 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.295 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.295 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.295 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.296 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.296 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.296 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.296 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.296 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.296 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.296 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.296 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.296 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.296 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.296 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.296 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.297 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.297 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.297 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.297 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.297 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.297 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.297 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.297 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.297 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.297 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.297 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.298 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-06-10T01:32:15.298 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.298 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.298 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.298 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.298 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.298 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.298 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.298 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.298 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.298 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.298 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.299 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.299 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.299 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.299 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.299 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.299 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.299 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.299 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.299 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.299 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.299 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.299 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.299 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.299 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.300 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.300 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.300 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.300 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.300 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.300 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.300 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.300 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.300 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.300 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.300 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.300 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.300 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.301 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.301 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.301 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.301 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.301 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.301 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.301 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.301 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.301 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.301 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.301 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.301 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.301 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.302 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.302 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.302 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.302 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.302 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.302 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.302 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.302 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.302 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.302 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.302 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.302 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.303 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.303 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.303 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.303 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.303 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.303 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.303 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_clones": { 2024-06-10T01:32:15.303 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_clones", 2024-06-10T01:32:15.303 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.303 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.303 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.303 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4", 2024-06-10T01:32:15.303 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.303 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.304 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.304 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-10T01:32:15.304 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.304 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.304 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.304 INFO:teuthology.orchestra.run.smithi007.stdout: "periodic_async_work": { 2024-06-10T01:32:15.304 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "periodic_async_work", 2024-06-10T01:32:15.304 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.304 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.304 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.304 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.304 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.305 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.305 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.305 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Periodically check for async work", 2024-06-10T01:32:15.305 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.305 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.305 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.305 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_delay": { 2024-06-10T01:32:15.305 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_delay", 2024-06-10T01:32:15.305 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.305 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.305 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.305 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-06-10T01:32:15.305 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.305 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.306 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.306 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-10T01:32:15.306 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.306 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.306 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.306 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_no_wait": { 2024-06-10T01:32:15.306 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_no_wait", 2024-06-10T01:32:15.306 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.306 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.306 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.306 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.306 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.307 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.307 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.307 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-10T01:32:15.307 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.307 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.307 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.307 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.307 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.307 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix", 2024-06-10T01:32:15.307 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.307 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.308 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.308 INFO:teuthology.orchestra.run.smithi007.stdout: "discovery_interval": { 2024-06-10T01:32:15.308 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "discovery_interval", 2024-06-10T01:32:15.308 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-06-10T01:32:15.308 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.308 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.308 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "100", 2024-06-10T01:32:15.308 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.308 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.308 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.308 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.308 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.308 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.308 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.309 INFO:teuthology.orchestra.run.smithi007.stdout: "identifier": { 2024-06-10T01:32:15.309 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "identifier", 2024-06-10T01:32:15.309 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.309 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.309 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.309 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.309 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.309 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.309 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.309 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.309 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.309 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.309 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.310 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-06-10T01:32:15.310 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-06-10T01:32:15.310 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:15.310 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.310 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.310 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-06-10T01:32:15.310 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.310 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.310 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.310 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.310 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.310 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.310 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.311 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.311 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.311 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.311 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.311 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.311 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.311 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.311 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.311 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.311 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.311 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.311 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.311 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.311 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.312 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.312 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.312 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.312 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.312 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.312 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.312 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.312 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.312 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.312 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.312 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.312 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.312 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.313 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.313 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.313 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.313 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.313 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.313 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.313 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.313 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.313 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.313 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.313 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.313 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.313 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.314 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.314 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.314 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.314 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.314 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.314 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.314 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.314 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.314 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.314 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.314 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.314 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.314 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.314 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.315 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.315 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.315 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.315 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.315 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.315 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.315 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.315 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.315 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.315 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.315 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.315 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.315 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_host": { 2024-06-10T01:32:15.315 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_host", 2024-06-10T01:32:15.316 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.316 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.316 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.316 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.316 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.316 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.316 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.316 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.316 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.316 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.316 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.316 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_port": { 2024-06-10T01:32:15.316 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_port", 2024-06-10T01:32:15.317 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.317 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.317 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.317 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10051", 2024-06-10T01:32:15.317 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.317 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.317 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.317 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.317 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.317 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.317 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.317 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_sender": { 2024-06-10T01:32:15.317 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_sender", 2024-06-10T01:32:15.317 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.318 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.318 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.318 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-10T01:32:15.318 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.318 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.318 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.318 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.318 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.318 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.318 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.318 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.318 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.318 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.318 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-06-10T01:32:15.319 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.319 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.319 INFO:teuthology.orchestra.run.smithi007.stdout: "modules": [ 2024-06-10T01:32:15.319 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-06-10T01:32:15.319 INFO:teuthology.orchestra.run.smithi007.stdout: "dashboard", 2024-06-10T01:32:15.319 INFO:teuthology.orchestra.run.smithi007.stdout: "iostat", 2024-06-10T01:32:15.319 INFO:teuthology.orchestra.run.smithi007.stdout: "nfs", 2024-06-10T01:32:15.319 INFO:teuthology.orchestra.run.smithi007.stdout: "restful" 2024-06-10T01:32:15.319 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.319 INFO:teuthology.orchestra.run.smithi007.stdout: "available_modules": [ 2024-06-10T01:32:15.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.319 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "alerts", 2024-06-10T01:32:15.319 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.319 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.320 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.320 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-06-10T01:32:15.320 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-06-10T01:32:15.320 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:15.320 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.320 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.320 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-06-10T01:32:15.320 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.320 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.320 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.320 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How frequently to reexamine health status", 2024-06-10T01:32:15.320 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.320 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.321 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.321 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.321 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.321 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.321 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.321 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.321 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.321 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.321 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.321 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.321 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.321 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.321 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.321 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.322 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.322 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.322 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.322 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.322 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.322 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.322 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.322 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.322 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.322 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.322 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.322 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.323 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.323 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.323 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.323 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.323 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.323 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.323 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.323 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.323 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.323 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.323 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.323 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.323 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.324 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.324 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.324 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.324 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.324 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.324 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.324 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.324 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.324 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.324 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.324 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.324 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.324 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.324 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.324 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.325 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.325 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.325 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.325 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.325 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.325 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.325 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.325 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.325 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.325 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.325 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.325 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.325 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.326 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_destination": { 2024-06-10T01:32:15.326 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_destination", 2024-06-10T01:32:15.326 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.326 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.326 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.326 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.326 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.326 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.326 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.326 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email address to send alerts to", 2024-06-10T01:32:15.326 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.326 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.326 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.327 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_from_name": { 2024-06-10T01:32:15.327 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_from_name", 2024-06-10T01:32:15.327 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.327 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.327 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.327 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "Ceph", 2024-06-10T01:32:15.327 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.327 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.327 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.327 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email From: name", 2024-06-10T01:32:15.327 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.327 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.327 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.327 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_host": { 2024-06-10T01:32:15.328 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_host", 2024-06-10T01:32:15.328 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.328 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.328 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.328 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.328 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.328 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.328 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.328 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP server", 2024-06-10T01:32:15.328 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.328 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.328 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.328 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_password": { 2024-06-10T01:32:15.329 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_password", 2024-06-10T01:32:15.329 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.329 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.329 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.329 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.329 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.329 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.329 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.329 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Password to authenticate with", 2024-06-10T01:32:15.329 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.329 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.329 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.330 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_port": { 2024-06-10T01:32:15.330 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_port", 2024-06-10T01:32:15.330 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.330 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.330 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.330 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "465", 2024-06-10T01:32:15.330 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.330 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.330 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.330 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP port", 2024-06-10T01:32:15.330 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.330 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.330 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.331 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_sender": { 2024-06-10T01:32:15.331 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_sender", 2024-06-10T01:32:15.331 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.331 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.331 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.331 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.331 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.331 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.331 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.331 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP envelope sender", 2024-06-10T01:32:15.331 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.331 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.331 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.332 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_ssl": { 2024-06-10T01:32:15.332 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_ssl", 2024-06-10T01:32:15.332 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.332 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.332 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.332 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.332 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.332 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.332 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.332 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-10T01:32:15.332 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.332 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.332 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.332 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_user": { 2024-06-10T01:32:15.333 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_user", 2024-06-10T01:32:15.333 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.333 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.333 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.333 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.333 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.333 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.333 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.333 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "User to authenticate as", 2024-06-10T01:32:15.333 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.333 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.333 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.333 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.333 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.334 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "balancer", 2024-06-10T01:32:15.334 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.334 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.334 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.334 INFO:teuthology.orchestra.run.smithi007.stdout: "active": { 2024-06-10T01:32:15.334 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "active", 2024-06-10T01:32:15.334 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.334 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.334 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.334 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.334 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.335 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.335 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.335 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically balance PGs across cluster", 2024-06-10T01:32:15.335 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.335 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.335 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.335 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_time": { 2024-06-10T01:32:15.335 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_time", 2024-06-10T01:32:15.335 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.335 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.335 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.335 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0000", 2024-06-10T01:32:15.336 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.336 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.336 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.336 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "beginning time of day to automatically balance", 2024-06-10T01:32:15.336 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-10T01:32:15.336 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.336 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.336 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_weekday": { 2024-06-10T01:32:15.336 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_weekday", 2024-06-10T01:32:15.336 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-06-10T01:32:15.336 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.336 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.336 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-06-10T01:32:15.337 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-06-10T01:32:15.337 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-06-10T01:32:15.337 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.337 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-10T01:32:15.337 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-10T01:32:15.337 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.337 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.337 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_max_iterations": { 2024-06-10T01:32:15.337 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_max_iterations", 2024-06-10T01:32:15.337 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-06-10T01:32:15.337 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.337 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.337 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "25", 2024-06-10T01:32:15.338 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-06-10T01:32:15.338 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "250", 2024-06-10T01:32:15.338 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.338 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-10T01:32:15.338 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.338 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.338 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.338 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_metrics": { 2024-06-10T01:32:15.338 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_metrics", 2024-06-10T01:32:15.338 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.338 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.338 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.338 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "pgs,objects,bytes", 2024-06-10T01:32:15.339 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.339 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.339 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.339 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-10T01:32:15.339 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-10T01:32:15.339 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.339 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.339 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_step": { 2024-06-10T01:32:15.339 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_step", 2024-06-10T01:32:15.339 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-10T01:32:15.339 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.339 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.340 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.5", 2024-06-10T01:32:15.340 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0.001", 2024-06-10T01:32:15.340 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "0.999", 2024-06-10T01:32:15.340 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.340 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "aggressiveness of optimization", 2024-06-10T01:32:15.340 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-10T01:32:15.340 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.340 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.340 INFO:teuthology.orchestra.run.smithi007.stdout: "end_time": { 2024-06-10T01:32:15.340 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_time", 2024-06-10T01:32:15.340 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.340 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.341 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.341 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2359", 2024-06-10T01:32:15.341 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.341 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.341 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.341 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ending time of day to automatically balance", 2024-06-10T01:32:15.341 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-10T01:32:15.341 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.341 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.341 INFO:teuthology.orchestra.run.smithi007.stdout: "end_weekday": { 2024-06-10T01:32:15.341 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_weekday", 2024-06-10T01:32:15.341 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-06-10T01:32:15.342 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.342 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.342 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-06-10T01:32:15.342 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-06-10T01:32:15.342 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-06-10T01:32:15.342 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.342 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-10T01:32:15.342 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-10T01:32:15.342 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.342 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.342 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.342 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.343 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.343 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.343 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.343 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.343 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.343 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.343 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.343 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.343 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.343 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.343 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.343 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.343 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.343 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.344 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.344 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.344 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.344 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.344 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.344 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.344 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.344 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.344 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.344 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.344 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.344 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.344 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.344 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.345 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.345 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.345 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.345 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.345 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.345 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.345 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.345 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.345 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.345 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.345 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.345 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.345 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.345 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.346 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.346 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.346 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.346 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.346 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.346 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.346 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.346 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.346 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.346 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.346 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.346 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.346 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.347 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.347 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.347 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.347 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.347 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.347 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.347 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.347 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.347 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.347 INFO:teuthology.orchestra.run.smithi007.stdout: "min_score": { 2024-06-10T01:32:15.347 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_score", 2024-06-10T01:32:15.347 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-10T01:32:15.347 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.348 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.348 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-06-10T01:32:15.348 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.348 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.348 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.348 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-10T01:32:15.348 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.348 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.348 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.348 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-06-10T01:32:15.348 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-06-10T01:32:15.348 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.348 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.348 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.349 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "upmap", 2024-06-10T01:32:15.349 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.349 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.349 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.349 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-compat", 2024-06-10T01:32:15.349 INFO:teuthology.orchestra.run.smithi007.stdout: "none", 2024-06-10T01:32:15.349 INFO:teuthology.orchestra.run.smithi007.stdout: "read", 2024-06-10T01:32:15.349 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap", 2024-06-10T01:32:15.349 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap-read" 2024-06-10T01:32:15.349 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.349 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Balancer mode", 2024-06-10T01:32:15.349 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.349 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.349 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.350 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_ids": { 2024-06-10T01:32:15.350 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_ids", 2024-06-10T01:32:15.350 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.350 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.350 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.350 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.350 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.350 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.350 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.350 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-10T01:32:15.350 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.350 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.350 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.351 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-06-10T01:32:15.351 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-06-10T01:32:15.351 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:15.351 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.351 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.351 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-06-10T01:32:15.351 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.351 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.351 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.351 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-10T01:32:15.351 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.351 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.351 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.351 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_deviation": { 2024-06-10T01:32:15.352 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_deviation", 2024-06-10T01:32:15.352 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.352 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.352 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.352 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-06-10T01:32:15.352 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-06-10T01:32:15.352 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.352 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.352 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-10T01:32:15.352 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-10T01:32:15.352 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.352 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.352 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_optimizations": { 2024-06-10T01:32:15.353 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_optimizations", 2024-06-10T01:32:15.353 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-06-10T01:32:15.353 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.353 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.353 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-10T01:32:15.353 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.353 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.353 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.353 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-10T01:32:15.353 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.353 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.353 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.353 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.353 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.354 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm", 2024-06-10T01:32:15.354 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.354 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.354 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.354 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_down_multiplier": { 2024-06-10T01:32:15.354 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_down_multiplier", 2024-06-10T01:32:15.354 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-10T01:32:15.354 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.354 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.354 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-06-10T01:32:15.354 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.354 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.355 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.355 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-10T01:32:15.355 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.355 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.355 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.355 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_refresh_rate": { 2024-06-10T01:32:15.355 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_refresh_rate", 2024-06-10T01:32:15.355 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:15.355 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.355 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.355 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "20", 2024-06-10T01:32:15.355 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.355 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.355 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.356 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-10T01:32:15.356 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.356 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.356 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.356 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_starting_port": { 2024-06-10T01:32:15.356 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_starting_port", 2024-06-10T01:32:15.356 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.356 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.356 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.356 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4721", 2024-06-10T01:32:15.356 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.356 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.356 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.357 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-10T01:32:15.357 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.357 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.357 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.357 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_ptrace": { 2024-06-10T01:32:15.357 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_ptrace", 2024-06-10T01:32:15.357 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.357 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.357 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.357 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.357 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.357 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.357 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.357 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-10T01:32:15.358 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-10T01:32:15.358 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.358 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.358 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_interval": { 2024-06-10T01:32:15.358 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_interval", 2024-06-10T01:32:15.358 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:15.358 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.358 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.358 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-06-10T01:32:15.361 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.361 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.361 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.361 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-10T01:32:15.361 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.361 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.361 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.362 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_memory_target_ratio": { 2024-06-10T01:32:15.362 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_memory_target_ratio", 2024-06-10T01:32:15.362 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-10T01:32:15.362 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.362 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.362 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.7", 2024-06-10T01:32:15.362 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.362 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.362 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.362 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-10T01:32:15.362 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.362 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.363 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.363 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm_log_destination": { 2024-06-10T01:32:15.363 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm_log_destination", 2024-06-10T01:32:15.363 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.363 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.363 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.363 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.363 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.363 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.363 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.363 INFO:teuthology.orchestra.run.smithi007.stdout: "file", 2024-06-10T01:32:15.363 INFO:teuthology.orchestra.run.smithi007.stdout: "file,syslog", 2024-06-10T01:32:15.363 INFO:teuthology.orchestra.run.smithi007.stdout: "syslog" 2024-06-10T01:32:15.364 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.364 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-10T01:32:15.364 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.364 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.364 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.364 INFO:teuthology.orchestra.run.smithi007.stdout: "cgroups_split": { 2024-06-10T01:32:15.364 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cgroups_split", 2024-06-10T01:32:15.364 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.364 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.364 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.364 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.364 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.365 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.365 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.365 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-10T01:32:15.365 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.365 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.365 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.365 INFO:teuthology.orchestra.run.smithi007.stdout: "config_checks_enabled": { 2024-06-10T01:32:15.365 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_checks_enabled", 2024-06-10T01:32:15.365 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.365 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.365 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.365 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.365 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.366 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.366 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.366 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-10T01:32:15.366 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.366 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.366 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.366 INFO:teuthology.orchestra.run.smithi007.stdout: "config_dashboard": { 2024-06-10T01:32:15.366 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_dashboard", 2024-06-10T01:32:15.366 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.366 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.366 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.366 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.366 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.367 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.367 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.367 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-10T01:32:15.367 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.367 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.367 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.367 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_alertmanager": { 2024-06-10T01:32:15.367 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_alertmanager", 2024-06-10T01:32:15.367 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.367 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.367 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.367 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-10T01:32:15.367 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.367 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.368 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.368 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-06-10T01:32:15.368 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.368 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.368 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.368 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_base": { 2024-06-10T01:32:15.368 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_base", 2024-06-10T01:32:15.368 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.368 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.368 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.368 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-10T01:32:15.368 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.369 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.369 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.369 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Container image name, without the tag", 2024-06-10T01:32:15.369 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.369 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.369 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.369 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_elasticsearch": { 2024-06-10T01:32:15.369 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_elasticsearch", 2024-06-10T01:32:15.369 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.369 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.369 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.369 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-10T01:32:15.369 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.369 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.370 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.370 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "elasticsearch container image", 2024-06-10T01:32:15.370 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.370 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.370 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.370 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_grafana": { 2024-06-10T01:32:15.370 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_grafana", 2024-06-10T01:32:15.370 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.370 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.370 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.370 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-10T01:32:15.371 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.371 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.371 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.371 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-06-10T01:32:15.371 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.371 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.371 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.371 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_haproxy": { 2024-06-10T01:32:15.371 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_haproxy", 2024-06-10T01:32:15.371 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.371 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.371 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.371 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-10T01:32:15.371 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.372 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.372 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.372 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "HAproxy container image", 2024-06-10T01:32:15.372 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.372 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.372 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.372 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_agent": { 2024-06-10T01:32:15.372 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_agent", 2024-06-10T01:32:15.372 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.372 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.372 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.372 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-10T01:32:15.373 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.373 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.373 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.373 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger agent container image", 2024-06-10T01:32:15.373 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.373 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.373 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.373 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_collector": { 2024-06-10T01:32:15.373 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_collector", 2024-06-10T01:32:15.373 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.373 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.373 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.373 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-10T01:32:15.374 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.374 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.374 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.374 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger collector container image", 2024-06-10T01:32:15.374 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.374 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.374 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.374 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_query": { 2024-06-10T01:32:15.374 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_query", 2024-06-10T01:32:15.374 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.374 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.375 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.375 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-10T01:32:15.375 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.375 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.375 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.375 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger query container image", 2024-06-10T01:32:15.375 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.375 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.375 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.375 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_keepalived": { 2024-06-10T01:32:15.375 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_keepalived", 2024-06-10T01:32:15.375 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.375 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.375 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.376 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-10T01:32:15.376 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.376 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.376 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.376 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Keepalived container image", 2024-06-10T01:32:15.376 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.376 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.376 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.376 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_loki": { 2024-06-10T01:32:15.376 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_loki", 2024-06-10T01:32:15.376 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.376 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.376 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.376 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-10T01:32:15.377 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.377 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.377 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.377 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Loki container image", 2024-06-10T01:32:15.377 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.377 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.377 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.377 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_node_exporter": { 2024-06-10T01:32:15.377 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_node_exporter", 2024-06-10T01:32:15.377 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.377 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.377 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.377 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-10T01:32:15.378 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.378 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.378 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.378 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-06-10T01:32:15.378 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.378 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.378 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.378 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_nvmeof": { 2024-06-10T01:32:15.378 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_nvmeof", 2024-06-10T01:32:15.378 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.378 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.378 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.378 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-10T01:32:15.379 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.379 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.379 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.379 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Nvme-of container image", 2024-06-10T01:32:15.379 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.379 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.379 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.379 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_prometheus": { 2024-06-10T01:32:15.379 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_prometheus", 2024-06-10T01:32:15.379 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.379 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.379 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.379 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-10T01:32:15.379 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.380 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.380 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.380 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-06-10T01:32:15.380 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.380 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.380 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.380 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_promtail": { 2024-06-10T01:32:15.380 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_promtail", 2024-06-10T01:32:15.380 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.380 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.380 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.380 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-10T01:32:15.380 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.380 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.381 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.381 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Promtail container image", 2024-06-10T01:32:15.381 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.381 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.381 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.381 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_samba": { 2024-06-10T01:32:15.381 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_samba", 2024-06-10T01:32:15.381 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.381 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.381 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.381 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-10T01:32:15.381 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.381 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.381 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.382 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Samba/SMB container image", 2024-06-10T01:32:15.382 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.382 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.382 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.382 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_snmp_gateway": { 2024-06-10T01:32:15.382 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_snmp_gateway", 2024-06-10T01:32:15.382 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.382 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.382 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.382 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-10T01:32:15.382 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.382 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.383 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.383 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SNMP Gateway container image", 2024-06-10T01:32:15.383 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.383 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.383 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.383 INFO:teuthology.orchestra.run.smithi007.stdout: "container_init": { 2024-06-10T01:32:15.383 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_init", 2024-06-10T01:32:15.383 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.383 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.383 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.383 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.383 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.383 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.384 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.384 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Run podman/docker with `--init`", 2024-06-10T01:32:15.384 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.384 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.384 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.384 INFO:teuthology.orchestra.run.smithi007.stdout: "daemon_cache_timeout": { 2024-06-10T01:32:15.384 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "daemon_cache_timeout", 2024-06-10T01:32:15.384 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:15.384 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.384 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.384 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-06-10T01:32:15.384 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.385 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.385 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.385 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-10T01:32:15.385 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.385 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.385 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.385 INFO:teuthology.orchestra.run.smithi007.stdout: "default_cephadm_command_timeout": { 2024-06-10T01:32:15.385 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_cephadm_command_timeout", 2024-06-10T01:32:15.385 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.385 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.385 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.385 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "900", 2024-06-10T01:32:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-10T01:32:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: "default_registry": { 2024-06-10T01:32:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_registry", 2024-06-10T01:32:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io", 2024-06-10T01:32:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-10T01:32:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "device_cache_timeout": { 2024-06-10T01:32:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_cache_timeout", 2024-06-10T01:32:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1800", 2024-06-10T01:32:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache device inventory", 2024-06-10T01:32:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: "device_enhanced_scan": { 2024-06-10T01:32:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_enhanced_scan", 2024-06-10T01:32:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-10T01:32:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "facts_cache_timeout": { 2024-06-10T01:32:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "facts_cache_timeout", 2024-06-10T01:32:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-06-10T01:32:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache host facts data", 2024-06-10T01:32:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: "grafana_dashboards_path": { 2024-06-10T01:32:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "grafana_dashboards_path", 2024-06-10T01:32:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-10T01:32:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-10T01:32:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: "host_check_interval": { 2024-06-10T01:32:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "host_check_interval", 2024-06-10T01:32:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-06-10T01:32:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to perform a host check", 2024-06-10T01:32:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: "hw_monitoring": { 2024-06-10T01:32:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hw_monitoring", 2024-06-10T01:32:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-10T01:32:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "inventory_list_all": { 2024-06-10T01:32:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "inventory_list_all", 2024-06-10T01:32:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-10T01:32:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: "log_refresh_metadata": { 2024-06-10T01:32:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_refresh_metadata", 2024-06-10T01:32:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-10T01:32:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-10T01:32:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-10T01:32:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-10T01:32:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-10T01:32:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-10T01:32:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-10T01:32:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "*", 2024-06-10T01:32:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-10T01:32:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: "max_count_per_host": { 2024-06-10T01:32:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_count_per_host", 2024-06-10T01:32:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-10T01:32:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of daemons per service per host", 2024-06-10T01:32:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd_draining_count": { 2024-06-10T01:32:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_osd_draining_count", 2024-06-10T01:32:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-10T01:32:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-10T01:32:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: "migration_current": { 2024-06-10T01:32:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "migration_current", 2024-06-10T01:32:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "internal - do not modify", 2024-06-10T01:32:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.405 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.405 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-06-10T01:32:15.405 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-06-10T01:32:15.405 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.405 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.405 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.405 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "root", 2024-06-10T01:32:15.405 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.405 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.405 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.405 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm-package", 2024-06-10T01:32:15.405 INFO:teuthology.orchestra.run.smithi007.stdout: "root" 2024-06-10T01:32:15.405 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.406 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "mode for remote execution of cephadm", 2024-06-10T01:32:15.406 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.406 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.406 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.406 INFO:teuthology.orchestra.run.smithi007.stdout: "oob_default_addr": { 2024-06-10T01:32:15.406 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "oob_default_addr", 2024-06-10T01:32:15.406 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.406 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.406 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.406 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "169.254.1.1", 2024-06-10T01:32:15.406 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.406 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.406 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.407 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-10T01:32:15.407 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.407 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.407 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.407 INFO:teuthology.orchestra.run.smithi007.stdout: "prometheus_alerts_path": { 2024-06-10T01:32:15.407 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus_alerts_path", 2024-06-10T01:32:15.407 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.407 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.407 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.407 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-10T01:32:15.407 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.407 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.408 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.408 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-10T01:32:15.408 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.408 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.408 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.408 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_insecure": { 2024-06-10T01:32:15.408 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_insecure", 2024-06-10T01:32:15.408 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.408 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.408 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.408 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.408 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.408 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.408 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.408 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-10T01:32:15.409 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.409 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.409 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.409 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_password": { 2024-06-10T01:32:15.409 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_password", 2024-06-10T01:32:15.409 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.409 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.409 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.409 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.409 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.409 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.409 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.409 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-10T01:32:15.410 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.410 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.410 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.410 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_url": { 2024-06-10T01:32:15.410 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_url", 2024-06-10T01:32:15.410 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.410 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.410 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.410 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.410 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.410 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.410 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.411 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-10T01:32:15.411 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.411 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.411 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.411 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_username": { 2024-06-10T01:32:15.411 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_username", 2024-06-10T01:32:15.411 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.411 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.411 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.411 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.411 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.411 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.411 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.412 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-10T01:32:15.412 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.412 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.412 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.412 INFO:teuthology.orchestra.run.smithi007.stdout: "secure_monitoring_stack": { 2024-06-10T01:32:15.412 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "secure_monitoring_stack", 2024-06-10T01:32:15.412 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.412 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.412 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.412 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.412 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.412 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.412 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-10T01:32:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: "service_discovery_port": { 2024-06-10T01:32:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "service_discovery_port", 2024-06-10T01:32:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8765", 2024-06-10T01:32:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "cephadm service discovery port", 2024-06-10T01:32:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "ssh_config_file": { 2024-06-10T01:32:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssh_config_file", 2024-06-10T01:32:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-10T01:32:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: "use_agent": { 2024-06-10T01:32:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_agent", 2024-06-10T01:32:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-10T01:32:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: "use_repo_digest": { 2024-06-10T01:32:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_repo_digest", 2024-06-10T01:32:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-10T01:32:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_failed_host_check": { 2024-06-10T01:32:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_failed_host_check", 2024-06-10T01:32:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if the host check fails", 2024-06-10T01:32:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_daemons": { 2024-06-10T01:32:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_daemons", 2024-06-10T01:32:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-10T01:32:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_hosts": { 2024-06-10T01:32:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_hosts", 2024-06-10T01:32:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-10T01:32:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crash", 2024-06-10T01:32:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: "retain_interval": { 2024-06-10T01:32:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retain_interval", 2024-06-10T01:32:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "31536000", 2024-06-10T01:32:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-10T01:32:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_recent_interval": { 2024-06-10T01:32:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_recent_interval", 2024-06-10T01:32:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1209600", 2024-06-10T01:32:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-10T01:32:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dashboard", 2024-06-10T01:32:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-10T01:32:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-10T01:32:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-10T01:32:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_HOST": { 2024-06-10T01:32:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-10T01:32:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-10T01:32:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-10T01:32:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_ENABLED": { 2024-06-10T01:32:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_ENABLED", 2024-06-10T01:32:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-10T01:32:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-10T01:32:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: "ENABLE_BROWSABLE_API": { 2024-06-10T01:32:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-10T01:32:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-10T01:32:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-10T01:32:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-10T01:32:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-10T01:32:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-10T01:32:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-10T01:32:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-10T01:32:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-10T01:32:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-10T01:32:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-10T01:32:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-10T01:32:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-10T01:32:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-10T01:32:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-10T01:32:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-10T01:32:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-10T01:32:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_PASSWORD": { 2024-06-10T01:32:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-10T01:32:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-06-10T01:32:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-10T01:32:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-10T01:32:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_URL": { 2024-06-10T01:32:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_URL", 2024-06-10T01:32:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_USERNAME": { 2024-06-10T01:32:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-10T01:32:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-06-10T01:32:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-10T01:32:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-10T01:32:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-10T01:32:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-10T01:32:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-10T01:32:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-10T01:32:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-10T01:32:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-10T01:32:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.450 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.450 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.450 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.450 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.450 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.450 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.450 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.450 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.450 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_HOST": { 2024-06-10T01:32:15.450 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-10T01:32:15.450 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.450 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.451 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.451 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.451 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.451 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.451 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.451 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.451 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.451 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.451 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.451 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-10T01:32:15.451 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-10T01:32:15.451 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.451 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.452 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.452 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.452 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.452 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.452 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.452 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.452 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.452 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.452 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.452 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-10T01:32:15.452 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-10T01:32:15.452 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.452 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.452 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.453 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.453 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.453 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.453 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.453 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.453 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.453 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.453 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.453 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-10T01:32:15.453 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-10T01:32:15.453 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.453 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.453 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-10T01:32:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-10T01:32:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-10T01:32:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-10T01:32:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-10T01:32:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-10T01:32:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-10T01:32:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-10T01:32:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-10T01:32:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-10T01:32:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_ENABLED": { 2024-06-10T01:32:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-10T01:32:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-10T01:32:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-10T01:32:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-10T01:32:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-10T01:32:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-10T01:32:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-10T01:32:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-10T01:32:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-10T01:32:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8", 2024-06-10T01:32:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-10T01:32:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-10T01:32:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "45", 2024-06-10T01:32:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ACCESS_KEY": { 2024-06-10T01:32:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-10T01:32:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-10T01:32:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-10T01:32:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-06-10T01:32:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SECRET_KEY": { 2024-06-10T01:32:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-10T01:32:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SSL_VERIFY": { 2024-06-10T01:32:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-10T01:32:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-10T01:32:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-10T01:32:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-06-10T01:32:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-10T01:32:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-10T01:32:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-10T01:32:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-10T01:32:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-10T01:32:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-06-10T01:32:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: "cross_origin_url": { 2024-06-10T01:32:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cross_origin_url", 2024-06-10T01:32:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: "crt_file": { 2024-06-10T01:32:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crt_file", 2024-06-10T01:32:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: "debug": { 2024-06-10T01:32:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "debug", 2024-06-10T01:32:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable/disable debug options", 2024-06-10T01:32:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: "jwt_token_ttl": { 2024-06-10T01:32:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "jwt_token_ttl", 2024-06-10T01:32:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "28800", 2024-06-10T01:32:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-06-10T01:32:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-06-10T01:32:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: "motd": { 2024-06-10T01:32:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "motd", 2024-06-10T01:32:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "The message of the day", 2024-06-10T01:32:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect_resolve_ip_addr": { 2024-06-10T01:32:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "redirect_resolve_ip_addr", 2024-06-10T01:32:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-06-10T01:32:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-06-10T01:32:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-06-10T01:32:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-06-10T01:32:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-06-10T01:32:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8080", 2024-06-10T01:32:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-06-10T01:32:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-06-10T01:32:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.486 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.486 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.486 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.486 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.486 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.486 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.486 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.486 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl_server_port": { 2024-06-10T01:32:15.486 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl_server_port", 2024-06-10T01:32:15.486 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.486 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.486 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.487 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8443", 2024-06-10T01:32:15.487 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.487 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.487 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.487 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.487 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.487 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.487 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.487 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-06-10T01:32:15.487 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-06-10T01:32:15.487 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.487 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.487 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.487 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "redirect", 2024-06-10T01:32:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect" 2024-06-10T01:32:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-06-10T01:32:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-06-10T01:32:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-06-10T01:32:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-06-10T01:32:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-06-10T01:32:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: "url_prefix": { 2024-06-10T01:32:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url_prefix", 2024-06-10T01:32:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-06-10T01:32:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_monitoring": { 2024-06-10T01:32:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_monitoring", 2024-06-10T01:32:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "monitor device health metrics", 2024-06-10T01:32:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.493 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.493 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.493 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.493 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.493 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.493 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.493 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.493 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.493 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.493 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.493 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.493 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.493 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.493 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.494 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.494 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.494 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.494 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.494 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.494 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.494 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.494 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.494 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.494 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.494 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.494 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.495 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.495 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.495 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.495 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.495 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.495 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.495 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.495 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.495 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.495 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.495 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.495 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.495 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.495 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.495 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.496 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.496 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.496 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.496 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.496 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.496 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.496 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.496 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.496 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.496 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.496 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.496 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.496 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.497 INFO:teuthology.orchestra.run.smithi007.stdout: "mark_out_threshold": { 2024-06-10T01:32:15.497 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mark_out_threshold", 2024-06-10T01:32:15.497 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:15.497 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.497 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.497 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2419200", 2024-06-10T01:32:15.497 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.497 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.497 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.497 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-10T01:32:15.497 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.497 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.497 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.498 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": { 2024-06-10T01:32:15.498 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_name", 2024-06-10T01:32:15.498 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.498 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.498 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.498 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "device_health_metrics", 2024-06-10T01:32:15.498 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.498 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.498 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.498 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-10T01:32:15.498 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.498 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.498 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.499 INFO:teuthology.orchestra.run.smithi007.stdout: "retention_period": { 2024-06-10T01:32:15.499 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retention_period", 2024-06-10T01:32:15.499 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:15.499 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.499 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.499 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15552000", 2024-06-10T01:32:15.499 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.499 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.499 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.499 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain device health metrics", 2024-06-10T01:32:15.499 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.499 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.499 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.500 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_frequency": { 2024-06-10T01:32:15.500 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_frequency", 2024-06-10T01:32:15.500 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:15.500 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.500 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.500 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-06-10T01:32:15.500 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.500 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.500 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.500 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-10T01:32:15.500 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.500 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.500 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.501 INFO:teuthology.orchestra.run.smithi007.stdout: "self_heal": { 2024-06-10T01:32:15.501 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "self_heal", 2024-06-10T01:32:15.501 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.501 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.501 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.501 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.501 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.501 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.501 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.501 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-10T01:32:15.501 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.501 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.501 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.502 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-06-10T01:32:15.502 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-06-10T01:32:15.502 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:15.502 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.502 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.502 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-06-10T01:32:15.502 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.502 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.502 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.502 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and check device health", 2024-06-10T01:32:15.502 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.502 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.502 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.503 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_threshold": { 2024-06-10T01:32:15.503 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_threshold", 2024-06-10T01:32:15.503 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:15.503 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.503 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.503 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7257600", 2024-06-10T01:32:15.503 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.503 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.503 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.503 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-10T01:32:15.503 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.503 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.503 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.504 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.504 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.504 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "diskprediction_local", 2024-06-10T01:32:15.504 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.504 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.504 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.504 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.504 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.504 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.504 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.504 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.504 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.505 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.505 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.505 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.505 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.505 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.505 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.505 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.505 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.505 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.505 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.505 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.505 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.505 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.506 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.506 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.506 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.506 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.506 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.506 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.506 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.506 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.506 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.506 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.506 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.506 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.506 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.508 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.508 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.508 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.508 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.508 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.508 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.508 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.508 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.508 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.508 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.508 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.508 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.508 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.508 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.509 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.509 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.509 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.509 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.509 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.509 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.509 INFO:teuthology.orchestra.run.smithi007.stdout: "predict_interval": { 2024-06-10T01:32:15.509 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predict_interval", 2024-06-10T01:32:15.509 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.509 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.509 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.510 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-06-10T01:32:15.510 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.510 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.510 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.510 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.510 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.510 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.510 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.510 INFO:teuthology.orchestra.run.smithi007.stdout: "predictor_model": { 2024-06-10T01:32:15.510 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predictor_model", 2024-06-10T01:32:15.510 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.511 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.511 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.511 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "prophetstor", 2024-06-10T01:32:15.511 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.511 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.511 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.511 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.511 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.511 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.511 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.511 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-06-10T01:32:15.511 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-06-10T01:32:15.511 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.512 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.512 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.512 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-06-10T01:32:15.512 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.512 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.512 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.512 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.512 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.512 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.512 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.512 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.512 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.513 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "influx", 2024-06-10T01:32:15.513 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": false, 2024-06-10T01:32:15.513 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "influxdb python module not found", 2024-06-10T01:32:15.513 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.513 INFO:teuthology.orchestra.run.smithi007.stdout: "batch_size": { 2024-06-10T01:32:15.513 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "batch_size", 2024-06-10T01:32:15.513 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.513 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.513 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.513 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5000", 2024-06-10T01:32:15.513 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.513 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.513 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.513 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-10T01:32:15.513 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.514 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.514 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.514 INFO:teuthology.orchestra.run.smithi007.stdout: "database": { 2024-06-10T01:32:15.514 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "database", 2024-06-10T01:32:15.514 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.514 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.514 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.514 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "ceph", 2024-06-10T01:32:15.514 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.514 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.514 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.514 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-10T01:32:15.515 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.515 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.515 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.515 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": { 2024-06-10T01:32:15.515 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hostname", 2024-06-10T01:32:15.515 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.515 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.515 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.515 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.515 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.515 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.515 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.515 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server hostname", 2024-06-10T01:32:15.516 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.516 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.516 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.516 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-06-10T01:32:15.516 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-06-10T01:32:15.516 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:15.516 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.516 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.516 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "30", 2024-06-10T01:32:15.516 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "5", 2024-06-10T01:32:15.516 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.516 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.517 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-10T01:32:15.517 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.517 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.517 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.517 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.517 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.517 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.517 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.517 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.517 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.517 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.517 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.517 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.517 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.518 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.518 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.518 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.518 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.518 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.518 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.518 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.518 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.518 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.518 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.518 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.519 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.519 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.519 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.519 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.519 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.519 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.519 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.519 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.519 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.519 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.519 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.519 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.520 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.520 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.520 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.520 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.520 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.520 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.520 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.520 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.520 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.520 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.520 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.520 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.520 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.520 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.521 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.521 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.521 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.521 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.521 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.521 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.521 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.521 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.521 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.521 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.521 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.521 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.521 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.522 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.522 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.522 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.522 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.522 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.522 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.522 INFO:teuthology.orchestra.run.smithi007.stdout: "password": { 2024-06-10T01:32:15.522 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "password", 2024-06-10T01:32:15.522 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.522 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.522 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.523 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.523 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.523 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.523 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.523 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "password of InfluxDB server user", 2024-06-10T01:32:15.523 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.523 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.523 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.523 INFO:teuthology.orchestra.run.smithi007.stdout: "port": { 2024-06-10T01:32:15.523 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "port", 2024-06-10T01:32:15.523 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.523 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.523 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.523 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8086", 2024-06-10T01:32:15.524 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.524 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.524 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.524 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server port", 2024-06-10T01:32:15.524 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.524 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.524 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.524 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-06-10T01:32:15.524 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-06-10T01:32:15.524 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.524 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.524 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.524 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "false", 2024-06-10T01:32:15.524 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.525 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.525 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.525 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-10T01:32:15.525 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.525 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.525 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.525 INFO:teuthology.orchestra.run.smithi007.stdout: "threads": { 2024-06-10T01:32:15.525 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threads", 2024-06-10T01:32:15.525 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.525 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.525 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.525 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-06-10T01:32:15.525 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-06-10T01:32:15.525 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "32", 2024-06-10T01:32:15.526 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.526 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-10T01:32:15.526 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.526 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.526 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.526 INFO:teuthology.orchestra.run.smithi007.stdout: "username": { 2024-06-10T01:32:15.526 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "username", 2024-06-10T01:32:15.526 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.526 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.526 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.526 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.526 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.526 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.527 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.527 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "username of InfluxDB server user", 2024-06-10T01:32:15.527 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.527 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.527 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.527 INFO:teuthology.orchestra.run.smithi007.stdout: "verify_ssl": { 2024-06-10T01:32:15.527 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "verify_ssl", 2024-06-10T01:32:15.527 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.527 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.527 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.527 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "true", 2024-06-10T01:32:15.527 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.527 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.528 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.528 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-10T01:32:15.528 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.528 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.528 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.528 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.528 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.528 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "insights", 2024-06-10T01:32:15.528 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.528 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.528 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.528 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.529 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.529 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.529 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.529 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.529 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.529 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.529 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.529 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.529 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.529 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.529 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.529 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.529 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.529 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.530 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.530 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.530 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.530 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.530 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.530 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.530 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.530 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.530 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.530 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.530 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.530 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.531 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.531 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.531 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.531 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.531 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.531 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.531 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.531 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.531 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.531 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.531 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.531 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.531 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.531 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.532 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.532 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.532 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.532 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.532 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.532 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.532 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.532 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.532 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.532 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.532 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.532 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.532 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.533 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.533 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.533 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.533 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.533 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.533 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.533 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.533 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.533 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.533 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.533 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.533 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.533 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.533 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.534 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "iostat", 2024-06-10T01:32:15.534 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.534 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.534 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.534 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.534 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.534 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.534 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.534 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.534 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.534 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.534 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.535 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.535 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.535 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.535 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.535 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.535 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.535 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.535 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.535 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.535 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.535 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.535 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.535 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.536 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.536 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.536 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.536 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.536 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.536 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.536 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.536 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.536 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.536 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.536 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.536 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.536 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.536 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.537 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.537 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.537 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.537 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.537 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.537 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.537 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.537 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.537 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.537 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.537 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.537 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.537 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.538 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.538 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.538 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.538 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.538 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.538 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.538 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.538 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.538 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.538 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.538 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.538 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.539 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.539 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.539 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.539 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.539 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.539 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.539 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.539 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.539 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "k8sevents", 2024-06-10T01:32:15.539 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.539 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.539 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.540 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_event_retention_days": { 2024-06-10T01:32:15.540 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ceph_event_retention_days", 2024-06-10T01:32:15.540 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.540 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.540 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.540 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7", 2024-06-10T01:32:15.540 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.540 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.540 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.540 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-10T01:32:15.540 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.540 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.540 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.541 INFO:teuthology.orchestra.run.smithi007.stdout: "config_check_secs": { 2024-06-10T01:32:15.541 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_check_secs", 2024-06-10T01:32:15.541 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.541 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.541 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.541 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-10T01:32:15.541 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "10", 2024-06-10T01:32:15.541 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.541 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.541 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-10T01:32:15.541 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.541 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.541 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.542 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.542 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.542 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.542 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.542 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.542 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.542 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.542 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.542 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.542 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.542 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.542 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.543 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.543 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.543 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.543 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.543 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.543 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.543 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.543 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.543 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.543 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.543 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.543 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.543 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.544 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.544 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.544 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.544 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.544 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.544 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.544 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.544 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.544 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.544 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.544 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.544 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.544 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.547 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.547 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.547 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.547 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.547 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.547 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "localpool", 2024-06-10T01:32:15.547 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.547 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.547 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.547 INFO:teuthology.orchestra.run.smithi007.stdout: "failure_domain": { 2024-06-10T01:32:15.547 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "failure_domain", 2024-06-10T01:32:15.547 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "host", 2024-06-10T01:32:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "failure domain for any created local pool", 2024-06-10T01:32:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-10T01:32:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.549 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.549 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.549 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.549 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.549 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.549 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.549 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.549 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.549 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.549 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.549 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.549 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.549 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.549 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.550 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.550 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.550 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.550 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.550 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.550 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.550 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.550 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.550 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.550 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.550 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.550 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.551 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.551 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.551 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.551 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.551 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.551 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.551 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.551 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.551 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.551 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.551 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.551 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.552 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.552 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.552 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.552 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.552 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.552 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.552 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.552 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.552 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.552 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.552 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.552 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.552 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.553 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.553 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.553 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.553 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.553 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.553 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.553 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.553 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.553 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.553 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.553 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.553 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.553 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.554 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": { 2024-06-10T01:32:15.554 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_size", 2024-06-10T01:32:15.554 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.554 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.554 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.554 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.554 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.554 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.554 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.554 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default min_size for any created local pool", 2024-06-10T01:32:15.554 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-10T01:32:15.554 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.555 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.555 INFO:teuthology.orchestra.run.smithi007.stdout: "num_rep": { 2024-06-10T01:32:15.555 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "num_rep", 2024-06-10T01:32:15.555 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.555 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.555 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.555 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3", 2024-06-10T01:32:15.555 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.555 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.555 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.555 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default replica count for any created local pool", 2024-06-10T01:32:15.555 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.555 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.556 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.556 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": { 2024-06-10T01:32:15.556 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_num", 2024-06-10T01:32:15.556 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.556 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.556 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.556 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "128", 2024-06-10T01:32:15.556 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.556 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.556 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.556 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default pg_num for any created local pool", 2024-06-10T01:32:15.556 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.557 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.557 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.557 INFO:teuthology.orchestra.run.smithi007.stdout: "prefix": { 2024-06-10T01:32:15.557 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prefix", 2024-06-10T01:32:15.557 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.557 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.557 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.557 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.557 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.557 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.557 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.557 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name prefix for any created local pool", 2024-06-10T01:32:15.557 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.557 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.558 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.558 INFO:teuthology.orchestra.run.smithi007.stdout: "subtree": { 2024-06-10T01:32:15.558 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "subtree", 2024-06-10T01:32:15.558 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.558 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.558 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.558 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "rack", 2024-06-10T01:32:15.558 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.558 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.558 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.558 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-10T01:32:15.558 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-10T01:32:15.558 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.558 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.559 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.559 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.559 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mds_autoscaler", 2024-06-10T01:32:15.559 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.559 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.559 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.559 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.559 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.559 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.559 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.559 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.559 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.559 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.560 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.560 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.560 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.560 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.560 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.560 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.560 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.560 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.560 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.560 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.560 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.560 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.560 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.560 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.561 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.561 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.561 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.561 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.561 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.561 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.561 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.561 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.561 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.561 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.561 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.561 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.562 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.562 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.562 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.562 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.562 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.562 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.562 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.562 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.562 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.562 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.562 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.562 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.562 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.562 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.563 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.563 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.563 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.563 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.563 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.563 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.563 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.563 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.563 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.563 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.563 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.563 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.563 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.564 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.564 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.564 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.564 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.564 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.564 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.564 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.564 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.564 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirroring", 2024-06-10T01:32:15.564 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.564 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.564 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.565 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.565 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.565 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.565 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.565 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.565 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.565 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.565 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.565 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.565 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.565 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.565 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.565 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.565 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.566 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.566 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.566 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.566 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.566 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.566 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.566 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.566 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.566 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.566 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.566 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.566 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.567 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.567 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.567 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.567 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.567 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.567 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.567 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.567 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.567 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.567 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.567 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.567 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.567 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.567 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.570 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.570 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "nfs", 2024-06-10T01:32:15.570 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.570 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.570 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.570 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.570 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.570 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.570 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.570 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.570 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.570 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.571 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.571 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.571 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.571 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.571 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.571 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.571 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.571 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.571 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.571 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.571 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.571 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.571 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.571 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.572 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.572 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.572 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.572 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.572 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.572 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.572 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.572 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.572 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.572 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.572 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.572 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.573 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.573 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.573 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.573 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.573 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.573 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.573 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.573 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.573 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.573 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.573 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.573 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.573 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.573 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.573 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.573 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.574 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.574 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.574 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.574 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.574 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.574 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.574 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.574 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.574 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.574 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.574 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.574 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.574 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.575 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.575 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.575 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.575 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.575 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.575 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.575 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-06-10T01:32:15.575 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.575 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.575 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.575 INFO:teuthology.orchestra.run.smithi007.stdout: "fail_fs": { 2024-06-10T01:32:15.575 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "fail_fs", 2024-06-10T01:32:15.575 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.576 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.576 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.576 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.576 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.576 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.576 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.576 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-10T01:32:15.576 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.576 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.576 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.576 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.576 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.576 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.576 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.577 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.577 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.577 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.577 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.577 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.577 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.577 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.577 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.577 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.577 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.577 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.577 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.577 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.577 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.578 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.578 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.578 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.578 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.578 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.578 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.578 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.578 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.578 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.578 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.578 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.578 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.578 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.579 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.579 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.579 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.579 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.579 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.579 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.579 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.579 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.579 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.579 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.579 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.579 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.579 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.579 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.580 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.580 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.580 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.580 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.580 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.580 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.580 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.580 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.580 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.580 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.580 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.580 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.580 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.581 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.581 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.581 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.581 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.581 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.581 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.581 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.581 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.581 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator": { 2024-06-10T01:32:15.581 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-06-10T01:32:15.581 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.581 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.581 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.581 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.582 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.582 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.582 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.582 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-06-10T01:32:15.582 INFO:teuthology.orchestra.run.smithi007.stdout: "rook", 2024-06-10T01:32:15.582 INFO:teuthology.orchestra.run.smithi007.stdout: "test_orchestrator" 2024-06-10T01:32:15.582 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.582 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Orchestrator backend", 2024-06-10T01:32:15.582 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.582 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.582 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.582 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.582 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.583 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_perf_query", 2024-06-10T01:32:15.583 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.583 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.583 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.583 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.583 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.583 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.583 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.583 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.583 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.583 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.583 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.583 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.584 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.584 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.584 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.584 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.584 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.584 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.584 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.584 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.584 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.584 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.584 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.584 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.584 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.584 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.585 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.585 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.585 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.585 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.585 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.585 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.585 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.585 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.585 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.585 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.585 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.585 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.586 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.586 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.586 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.586 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.586 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.586 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.586 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.586 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.586 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.586 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.586 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.586 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.586 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.586 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.587 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.587 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.587 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.587 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.587 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.587 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.587 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.587 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.587 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.587 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.587 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.587 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.587 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.588 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.588 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.588 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.588 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.588 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.588 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.588 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_support", 2024-06-10T01:32:15.588 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.588 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.588 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.588 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.588 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.588 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.589 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.589 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.589 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.589 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.589 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.589 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.589 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.589 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.589 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.589 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.589 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.589 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.589 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.589 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.589 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.590 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.590 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.590 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.590 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.590 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.590 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.590 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.590 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.590 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.590 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.590 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.590 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.590 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.591 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.591 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.591 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.591 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.591 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.591 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.591 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.591 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.591 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.591 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.591 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.591 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.591 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.592 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.592 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.592 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.592 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.592 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.592 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.592 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.592 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.592 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.592 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.592 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.592 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.592 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.592 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.593 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.593 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.593 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.593 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.593 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.593 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.593 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.593 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.593 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.593 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.593 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_autoscaler", 2024-06-10T01:32:15.593 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.593 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.594 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.594 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.594 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.594 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.594 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.594 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.594 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.594 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.594 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.594 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.594 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.594 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.594 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.595 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.595 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.595 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.595 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.595 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.595 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.595 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.595 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.595 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.595 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.595 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.595 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.595 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.596 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.596 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.596 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.596 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.596 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.596 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.596 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.596 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.596 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.596 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.596 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.596 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.596 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.596 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.597 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.597 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.597 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.597 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.597 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.597 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.597 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.597 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.597 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.597 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.597 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.597 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.597 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.597 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.598 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.598 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.598 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.598 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.598 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.598 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.598 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.598 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.598 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.598 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.598 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.598 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.598 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.599 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-06-10T01:32:15.599 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-06-10T01:32:15.599 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:15.599 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.599 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.599 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-06-10T01:32:15.599 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.599 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.599 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.599 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.599 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.599 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.599 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.599 INFO:teuthology.orchestra.run.smithi007.stdout: "threshold": { 2024-06-10T01:32:15.600 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threshold", 2024-06-10T01:32:15.600 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-10T01:32:15.600 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.600 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.600 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-06-10T01:32:15.600 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1.0", 2024-06-10T01:32:15.600 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.600 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.600 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "scaling threshold", 2024-06-10T01:32:15.600 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-10T01:32:15.600 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.600 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.600 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.600 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.601 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "progress", 2024-06-10T01:32:15.601 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.601 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.601 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.601 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_pg_recovery_event": { 2024-06-10T01:32:15.601 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_pg_recovery_event", 2024-06-10T01:32:15.601 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.601 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.601 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.601 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.601 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.601 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.601 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.601 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-10T01:32:15.602 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.602 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.602 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.602 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-06-10T01:32:15.602 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-06-10T01:32:15.602 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.602 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.602 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.602 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.602 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.602 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.602 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.603 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.603 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.603 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.603 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.603 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.603 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.603 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.603 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.603 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.603 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.603 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.603 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.603 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.603 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.604 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.604 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.604 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.604 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.604 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.604 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.604 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.604 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.604 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.604 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.604 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.604 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.604 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.604 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.605 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.605 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.605 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.605 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.605 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.605 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.605 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.605 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.605 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.605 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.605 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.605 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.605 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.605 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.606 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.606 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.606 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.606 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.606 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.606 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.606 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.606 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.606 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.606 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.606 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.606 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.606 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.606 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.606 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.607 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.607 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.607 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.607 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.607 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.607 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.607 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.607 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.607 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.607 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.607 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.607 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.607 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.608 INFO:teuthology.orchestra.run.smithi007.stdout: "max_completed_events": { 2024-06-10T01:32:15.608 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_completed_events", 2024-06-10T01:32:15.608 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.608 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.608 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.608 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "50", 2024-06-10T01:32:15.608 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.608 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.608 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.608 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "number of past completed events to remember", 2024-06-10T01:32:15.608 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.608 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.608 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.609 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-06-10T01:32:15.609 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-06-10T01:32:15.609 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:15.609 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.609 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.609 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-06-10T01:32:15.609 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.609 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.609 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.609 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long the module is going to sleep", 2024-06-10T01:32:15.609 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.609 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.609 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.609 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.609 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.610 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus", 2024-06-10T01:32:15.610 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.610 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.610 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.610 INFO:teuthology.orchestra.run.smithi007.stdout: "cache": { 2024-06-10T01:32:15.610 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cache", 2024-06-10T01:32:15.610 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.610 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.610 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.610 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.610 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.610 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.611 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.611 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.611 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.611 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.611 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.611 INFO:teuthology.orchestra.run.smithi007.stdout: "exclude_perf_counters": { 2024-06-10T01:32:15.611 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "exclude_perf_counters", 2024-06-10T01:32:15.611 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.611 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.611 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.611 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.611 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.611 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.611 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.612 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-10T01:32:15.612 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-10T01:32:15.612 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.612 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.612 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.612 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.612 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.612 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.612 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.612 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.612 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.612 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.612 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.613 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.613 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.613 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.613 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.613 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.613 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.613 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.613 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.613 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.613 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.613 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.613 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.613 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.614 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.614 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.614 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.614 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.614 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.614 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.614 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.614 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.614 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.614 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.614 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.614 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.614 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.615 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.615 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.615 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.615 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.615 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.615 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.615 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.615 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.615 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.615 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.615 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.615 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.615 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.615 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.615 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.616 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.616 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.616 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.616 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.616 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.616 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.616 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.616 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.616 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.616 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.616 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.616 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.616 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.616 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.617 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.617 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.617 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools": { 2024-06-10T01:32:15.617 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools", 2024-06-10T01:32:15.617 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.617 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.617 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.617 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.617 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.617 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.617 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.617 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.617 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.617 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.618 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.618 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-10T01:32:15.618 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-10T01:32:15.618 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.618 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.618 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.618 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "300", 2024-06-10T01:32:15.618 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.618 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.618 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.618 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.618 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.618 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.619 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.619 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_interval": { 2024-06-10T01:32:15.619 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_interval", 2024-06-10T01:32:15.619 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-10T01:32:15.619 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.619 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.619 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15.0", 2024-06-10T01:32:15.619 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.619 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.619 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.619 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.619 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.619 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.619 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.620 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-06-10T01:32:15.620 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-06-10T01:32:15.620 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.620 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.620 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.620 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-06-10T01:32:15.620 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.620 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.620 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.620 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-10T01:32:15.620 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.620 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.620 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.621 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-06-10T01:32:15.621 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-06-10T01:32:15.621 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.621 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.621 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.621 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "9283", 2024-06-10T01:32:15.621 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.621 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.621 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.621 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-10T01:32:15.621 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.621 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.621 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.621 INFO:teuthology.orchestra.run.smithi007.stdout: "stale_cache_strategy": { 2024-06-10T01:32:15.622 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stale_cache_strategy", 2024-06-10T01:32:15.622 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.622 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.622 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.622 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "log", 2024-06-10T01:32:15.622 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.622 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.622 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.622 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.622 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.622 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.622 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.623 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-06-10T01:32:15.623 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-06-10T01:32:15.623 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.623 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.623 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.623 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "default", 2024-06-10T01:32:15.623 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.623 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.623 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.623 INFO:teuthology.orchestra.run.smithi007.stdout: "default", 2024-06-10T01:32:15.623 INFO:teuthology.orchestra.run.smithi007.stdout: "error" 2024-06-10T01:32:15.623 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.623 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.623 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.623 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.624 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.624 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-06-10T01:32:15.624 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-06-10T01:32:15.624 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.624 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.624 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.624 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-06-10T01:32:15.624 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-06-10T01:32:15.624 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-06-10T01:32:15.624 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.624 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.624 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.624 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.624 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.625 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.625 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.625 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-06-10T01:32:15.625 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.625 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.625 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.625 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.625 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.625 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.625 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.625 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.625 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.625 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.626 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.626 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.626 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.626 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.626 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.626 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.626 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.626 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.626 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.626 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.626 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.626 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.626 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.627 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.627 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.627 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.627 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.627 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.627 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.627 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.627 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.627 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.627 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.627 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.627 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.627 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.627 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.628 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.628 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.628 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.628 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.628 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.628 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.628 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.628 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.628 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.628 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.628 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.628 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.628 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.628 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.628 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.629 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.629 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.629 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.629 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.629 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.629 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.629 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.629 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.629 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.629 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.629 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.629 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.630 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.630 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.630 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.630 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.630 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.630 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_snap_create": { 2024-06-10T01:32:15.630 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_snap_create", 2024-06-10T01:32:15.630 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.630 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.630 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.630 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-10T01:32:15.631 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.631 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.631 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.631 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.631 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.631 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.631 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.631 INFO:teuthology.orchestra.run.smithi007.stdout: "mirror_snapshot_schedule": { 2024-06-10T01:32:15.631 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirror_snapshot_schedule", 2024-06-10T01:32:15.631 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.631 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.631 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.631 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.631 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.631 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.632 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.632 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.632 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.632 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.632 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.632 INFO:teuthology.orchestra.run.smithi007.stdout: "trash_purge_schedule": { 2024-06-10T01:32:15.632 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "trash_purge_schedule", 2024-06-10T01:32:15.632 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.632 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.632 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.632 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.632 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.632 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.633 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.633 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.633 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.633 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.633 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.633 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.633 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.633 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "restful", 2024-06-10T01:32:15.633 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.633 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.633 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.633 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_auth": { 2024-06-10T01:32:15.633 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_auth", 2024-06-10T01:32:15.634 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.634 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.634 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.634 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.634 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.634 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.634 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.634 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.634 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.634 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.634 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.634 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-06-10T01:32:15.635 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-06-10T01:32:15.635 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.635 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.635 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.635 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.635 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.635 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.635 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.635 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.635 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.635 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.635 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.635 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.635 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.636 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.636 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.636 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.636 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.636 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.636 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.636 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.636 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.636 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.636 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.636 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.636 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.636 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.636 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.636 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.637 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.637 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.637 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.637 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.637 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.637 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.637 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.637 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.637 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.637 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.637 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.637 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.637 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.638 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.638 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.638 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.638 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.638 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.638 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.638 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.638 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.638 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.638 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.638 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.638 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.639 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.639 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.639 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.639 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.639 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.639 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.639 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.639 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.639 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.639 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.639 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.639 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.639 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.640 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.640 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.640 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.640 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.640 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.640 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.640 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.640 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.640 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.640 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.640 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.640 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-06-10T01:32:15.641 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-06-10T01:32:15.641 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.641 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.641 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.641 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.641 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.641 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.641 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.641 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.641 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.641 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.641 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.641 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-06-10T01:32:15.642 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-06-10T01:32:15.642 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.642 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.642 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.642 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.642 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.642 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.642 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.642 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.642 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.642 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.642 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.642 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.642 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rgw", 2024-06-10T01:32:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rook", 2024-06-10T01:32:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "storage_class": { 2024-06-10T01:32:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "storage_class", 2024-06-10T01:32:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.654 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.654 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.654 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "local", 2024-06-10T01:32:15.654 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.654 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.654 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.654 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-10T01:32:15.654 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.654 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.654 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.654 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.654 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.655 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "selftest", 2024-06-10T01:32:15.655 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.655 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.655 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.655 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.655 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.655 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.655 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.655 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.655 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.655 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.655 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.655 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.655 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.655 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.656 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.656 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.656 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.656 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.656 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.656 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.656 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.656 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.656 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.656 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.656 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.656 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.656 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.656 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.656 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.657 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.657 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.657 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.657 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.657 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.657 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.657 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.657 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.657 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.657 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.657 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.657 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.657 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.658 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.658 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.658 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.658 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.658 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.658 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.658 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.658 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.658 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.658 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.658 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.658 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.658 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.658 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.659 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.659 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.659 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.659 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.659 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.659 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.659 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.659 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.659 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.659 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.659 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.659 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.659 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.659 INFO:teuthology.orchestra.run.smithi007.stdout: "roption1": { 2024-06-10T01:32:15.660 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption1", 2024-06-10T01:32:15.660 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.660 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.660 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.660 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.660 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.660 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.660 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.660 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.660 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.660 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.660 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.660 INFO:teuthology.orchestra.run.smithi007.stdout: "roption2": { 2024-06-10T01:32:15.660 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption2", 2024-06-10T01:32:15.661 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.661 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.661 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.661 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "xyz", 2024-06-10T01:32:15.661 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.661 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.661 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.661 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.661 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.661 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.661 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.661 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption1": { 2024-06-10T01:32:15.661 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption1", 2024-06-10T01:32:15.661 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.662 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.662 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.662 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.662 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.662 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.662 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.662 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.662 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.662 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.662 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.662 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption2": { 2024-06-10T01:32:15.662 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption2", 2024-06-10T01:32:15.662 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.663 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.663 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.663 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.663 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.663 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.663 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.663 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.663 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.663 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.663 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.663 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption3": { 2024-06-10T01:32:15.663 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption3", 2024-06-10T01:32:15.664 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-10T01:32:15.664 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.664 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.664 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.664 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.664 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.664 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.664 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.664 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.664 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.664 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.664 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption4": { 2024-06-10T01:32:15.665 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption4", 2024-06-10T01:32:15.665 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.665 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.665 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.665 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.665 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.665 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.665 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.665 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.665 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.665 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.665 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.665 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption5": { 2024-06-10T01:32:15.665 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption5", 2024-06-10T01:32:15.666 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.666 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.666 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.666 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.666 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.666 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.666 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.666 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.666 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.666 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.666 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.667 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption6": { 2024-06-10T01:32:15.667 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption6", 2024-06-10T01:32:15.667 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.667 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.667 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.667 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.667 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.667 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.667 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.667 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.667 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.667 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.667 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.667 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption7": { 2024-06-10T01:32:15.668 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption7", 2024-06-10T01:32:15.668 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.668 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.668 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.668 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.668 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-06-10T01:32:15.668 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "42", 2024-06-10T01:32:15.668 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.668 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.668 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.668 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.668 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.668 INFO:teuthology.orchestra.run.smithi007.stdout: "testkey": { 2024-06-10T01:32:15.668 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testkey", 2024-06-10T01:32:15.669 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.669 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.669 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.669 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.669 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.669 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.669 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.669 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.669 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.669 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.669 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.669 INFO:teuthology.orchestra.run.smithi007.stdout: "testlkey": { 2024-06-10T01:32:15.669 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testlkey", 2024-06-10T01:32:15.670 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.670 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.670 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.670 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.670 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.670 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.670 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.670 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.670 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.670 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.670 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.670 INFO:teuthology.orchestra.run.smithi007.stdout: "testnewline": { 2024-06-10T01:32:15.670 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testnewline", 2024-06-10T01:32:15.670 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.671 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.671 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.671 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.671 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.671 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.671 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.671 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.671 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.671 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.671 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.671 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.671 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.671 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snap_schedule", 2024-06-10T01:32:15.672 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.672 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.672 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.672 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_m_granularity": { 2024-06-10T01:32:15.672 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_m_granularity", 2024-06-10T01:32:15.672 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.672 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.672 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.672 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.672 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.672 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.672 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.672 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow minute scheduled snapshots", 2024-06-10T01:32:15.672 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.673 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.673 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.673 INFO:teuthology.orchestra.run.smithi007.stdout: "dump_on_update": { 2024-06-10T01:32:15.673 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dump_on_update", 2024-06-10T01:32:15.673 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.673 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.673 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.673 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.673 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.673 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.673 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.673 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "dump database to debug log on update", 2024-06-10T01:32:15.673 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.674 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.674 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.674 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.674 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.674 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.674 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.674 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.674 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.674 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.674 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.674 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.674 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.674 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.674 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.675 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.675 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.675 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.675 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.675 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.675 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.675 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.675 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.675 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.675 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.675 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.675 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.675 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.675 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.676 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.676 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.676 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.676 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.676 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.676 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.676 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.676 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.676 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.676 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.676 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.676 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.676 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.676 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.677 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.677 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.677 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.677 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.677 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.677 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.677 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.677 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.677 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.677 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.677 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.677 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.677 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.678 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.678 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.678 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.678 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.678 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.678 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.678 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.678 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.678 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.678 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.678 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.678 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.678 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.678 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.679 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.679 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stats", 2024-06-10T01:32:15.679 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.679 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.679 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.679 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.679 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.679 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.679 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.679 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.679 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.679 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.679 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.679 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.680 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.680 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.680 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.680 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.680 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.680 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.680 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.680 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.680 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.680 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.680 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.680 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.680 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.681 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.681 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.681 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.681 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.681 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.681 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.681 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.681 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.681 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.681 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.681 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.681 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.681 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.681 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.682 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.682 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.682 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.682 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.682 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.682 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.682 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.682 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.682 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.682 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.682 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.682 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.682 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.682 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.683 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.683 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.683 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.683 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.683 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.683 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.683 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.683 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.683 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.683 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.683 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.683 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.683 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.683 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.684 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.684 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.684 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.684 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.684 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "status", 2024-06-10T01:32:15.684 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.684 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.684 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.684 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.684 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.684 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.685 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.685 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.685 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.685 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.685 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.685 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.685 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.685 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.685 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.685 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.685 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.685 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.685 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.685 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.685 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.686 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.686 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.686 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.686 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.686 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.686 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.686 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.686 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.686 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.686 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.686 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.686 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.687 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.687 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.687 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.687 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.687 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.687 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.687 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.687 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.687 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.687 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.687 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.687 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.687 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.687 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.688 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.688 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.688 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.688 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.688 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.688 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.688 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.688 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.688 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.688 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.688 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.688 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.688 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.688 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.689 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.689 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.689 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.689 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.689 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.689 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.689 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.689 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.689 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.689 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.689 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telegraf", 2024-06-10T01:32:15.689 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.689 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.690 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.690 INFO:teuthology.orchestra.run.smithi007.stdout: "address": { 2024-06-10T01:32:15.690 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "address", 2024-06-10T01:32:15.690 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.690 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.690 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.690 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-10T01:32:15.690 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.690 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.690 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.690 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.690 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.690 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.690 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.691 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-06-10T01:32:15.691 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-06-10T01:32:15.691 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:15.691 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.691 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.691 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15", 2024-06-10T01:32:15.691 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.691 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.691 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.691 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.691 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.691 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.691 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.692 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.692 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.692 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.692 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.692 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.692 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.692 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.692 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.692 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.692 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.692 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.692 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.692 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.692 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.692 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telemetry", 2024-06-10T01:32:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_basic": { 2024-06-10T01:32:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_basic", 2024-06-10T01:32:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-10T01:32:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_crash": { 2024-06-10T01:32:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_crash", 2024-06-10T01:32:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-10T01:32:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_device": { 2024-06-10T01:32:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_device", 2024-06-10T01:32:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-10T01:32:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_ident": { 2024-06-10T01:32:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_ident", 2024-06-10T01:32:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-10T01:32:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_perf": { 2024-06-10T01:32:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_perf", 2024-06-10T01:32:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-10T01:32:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "contact": { 2024-06-10T01:32:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "contact", 2024-06-10T01:32:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "description": { 2024-06-10T01:32:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "description", 2024-06-10T01:32:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "device_url": { 2024-06-10T01:32:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_url", 2024-06-10T01:32:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-10T01:32:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-06-10T01:32:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-06-10T01:32:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.706 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-06-10T01:32:15.706 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-06-10T01:32:15.706 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.706 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.706 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.706 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "24", 2024-06-10T01:32:15.706 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "8", 2024-06-10T01:32:15.706 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.706 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.706 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.706 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.706 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.706 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.707 INFO:teuthology.orchestra.run.smithi007.stdout: "last_opt_revision": { 2024-06-10T01:32:15.707 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "last_opt_revision", 2024-06-10T01:32:15.707 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.707 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.707 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.707 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1", 2024-06-10T01:32:15.707 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.707 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.707 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.707 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.707 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.707 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.707 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.708 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard": { 2024-06-10T01:32:15.708 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard", 2024-06-10T01:32:15.708 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.708 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.708 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.708 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.708 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.708 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.708 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.708 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.708 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.708 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.708 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.709 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard_description": { 2024-06-10T01:32:15.709 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard_description", 2024-06-10T01:32:15.709 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.709 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.709 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.709 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.709 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.709 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.709 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.709 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.709 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.709 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.709 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.710 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.710 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.710 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.710 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.710 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.710 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.710 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.710 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.710 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.710 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.710 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.710 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.711 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.711 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.711 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.711 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.711 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.711 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.711 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.711 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.711 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.711 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.711 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.711 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.711 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.711 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.712 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.712 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.712 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.712 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.712 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.712 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.712 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.712 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.712 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.712 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.712 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.712 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.712 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.713 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.713 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.713 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.713 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.713 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.713 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.713 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.713 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.713 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.713 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.713 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.713 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.713 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.713 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.714 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.714 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.714 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.714 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.714 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.714 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.714 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.714 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.714 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.714 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.714 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.714 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.714 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.715 INFO:teuthology.orchestra.run.smithi007.stdout: "organization": { 2024-06-10T01:32:15.715 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "organization", 2024-06-10T01:32:15.715 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.715 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.715 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.715 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.715 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.715 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.715 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.715 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.715 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.715 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.715 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.716 INFO:teuthology.orchestra.run.smithi007.stdout: "proxy": { 2024-06-10T01:32:15.716 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "proxy", 2024-06-10T01:32:15.716 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.716 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.716 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.716 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.716 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.716 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.716 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.716 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.716 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.716 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.716 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.716 INFO:teuthology.orchestra.run.smithi007.stdout: "url": { 2024-06-10T01:32:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url", 2024-06-10T01:32:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-10T01:32:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "test_orchestrator", 2024-06-10T01:32:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-06-10T01:32:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_clones": { 2024-06-10T01:32:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_clones", 2024-06-10T01:32:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4", 2024-06-10T01:32:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-10T01:32:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: "periodic_async_work": { 2024-06-10T01:32:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "periodic_async_work", 2024-06-10T01:32:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Periodically check for async work", 2024-06-10T01:32:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_delay": { 2024-06-10T01:32:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_delay", 2024-06-10T01:32:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-06-10T01:32:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-10T01:32:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_no_wait": { 2024-06-10T01:32:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_no_wait", 2024-06-10T01:32:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-10T01:32:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-10T01:32:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix", 2024-06-10T01:32:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-10T01:32:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-10T01:32:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-10T01:32:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: "discovery_interval": { 2024-06-10T01:32:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "discovery_interval", 2024-06-10T01:32:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-06-10T01:32:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "100", 2024-06-10T01:32:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: "identifier": { 2024-06-10T01:32:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "identifier", 2024-06-10T01:32:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-06-10T01:32:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-06-10T01:32:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-10T01:32:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-06-10T01:32:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.736 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.736 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.736 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.736 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-10T01:32:15.736 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-10T01:32:15.736 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.736 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.736 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.736 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.736 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.736 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.736 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.736 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.736 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.737 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.737 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.737 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.737 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.737 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.737 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.737 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.737 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.737 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.737 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-10T01:32:15.737 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-10T01:32:15.737 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.737 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.737 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.738 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.738 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.738 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.738 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.738 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.738 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.738 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.738 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.738 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-10T01:32:15.738 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-10T01:32:15.738 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.738 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.739 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.739 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-10T01:32:15.739 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.739 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.739 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-10T01:32:15.739 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-10T01:32:15.739 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-10T01:32:15.739 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-10T01:32:15.739 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-10T01:32:15.739 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-10T01:32:15.739 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-10T01:32:15.739 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.739 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.739 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.739 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.740 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.740 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-10T01:32:15.740 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-10T01:32:15.740 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-10T01:32:15.740 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.740 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-10T01:32:15.740 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-10T01:32:15.740 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.740 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.740 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.740 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.740 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.740 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.741 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.741 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_host": { 2024-06-10T01:32:15.741 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_host", 2024-06-10T01:32:15.741 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.741 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.741 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.741 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-10T01:32:15.741 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.741 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.741 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.741 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.741 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.742 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.742 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.742 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_port": { 2024-06-10T01:32:15.742 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_port", 2024-06-10T01:32:15.742 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-10T01:32:15.742 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.742 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.742 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10051", 2024-06-10T01:32:15.742 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.742 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.742 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.742 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.742 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.743 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.743 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.743 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_sender": { 2024-06-10T01:32:15.743 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_sender", 2024-06-10T01:32:15.743 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-10T01:32:15.743 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-10T01:32:15.743 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-10T01:32:15.743 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-10T01:32:15.743 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-10T01:32:15.743 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-10T01:32:15.743 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-10T01:32:15.743 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-10T01:32:15.743 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-10T01:32:15.744 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-10T01:32:15.744 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-10T01:32:15.744 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.744 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.744 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.744 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.744 INFO:teuthology.orchestra.run.smithi007.stdout: "services": { 2024-06-10T01:32:15.744 INFO:teuthology.orchestra.run.smithi007.stdout: "dashboard": "https://172.21.15.7:8443/" 2024-06-10T01:32:15.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.744 INFO:teuthology.orchestra.run.smithi007.stdout: "always_on_modules": { 2024-06-10T01:32:15.744 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus": [ 2024-06-10T01:32:15.744 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-06-10T01:32:15.744 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-06-10T01:32:15.744 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-06-10T01:32:15.745 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-06-10T01:32:15.745 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-06-10T01:32:15.745 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-06-10T01:32:15.745 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-06-10T01:32:15.745 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-06-10T01:32:15.745 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-06-10T01:32:15.745 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-06-10T01:32:15.745 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.745 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific": [ 2024-06-10T01:32:15.745 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-06-10T01:32:15.745 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-06-10T01:32:15.745 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-06-10T01:32:15.745 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-06-10T01:32:15.745 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-06-10T01:32:15.745 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-06-10T01:32:15.746 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-06-10T01:32:15.746 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-06-10T01:32:15.746 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-06-10T01:32:15.746 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-06-10T01:32:15.746 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.746 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy": [ 2024-06-10T01:32:15.746 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-06-10T01:32:15.746 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-06-10T01:32:15.746 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-06-10T01:32:15.746 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-06-10T01:32:15.746 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-06-10T01:32:15.746 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-06-10T01:32:15.746 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-06-10T01:32:15.746 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-06-10T01:32:15.747 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-06-10T01:32:15.747 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-06-10T01:32:15.747 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.747 INFO:teuthology.orchestra.run.smithi007.stdout: "reef": [ 2024-06-10T01:32:15.747 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-06-10T01:32:15.747 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-06-10T01:32:15.747 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-06-10T01:32:15.747 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-06-10T01:32:15.747 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-06-10T01:32:15.747 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-06-10T01:32:15.747 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-06-10T01:32:15.747 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-06-10T01:32:15.747 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-06-10T01:32:15.747 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-06-10T01:32:15.747 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-10T01:32:15.748 INFO:teuthology.orchestra.run.smithi007.stdout: "squid": [ 2024-06-10T01:32:15.748 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-06-10T01:32:15.748 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-06-10T01:32:15.748 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-06-10T01:32:15.748 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-06-10T01:32:15.748 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-06-10T01:32:15.748 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-06-10T01:32:15.748 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-06-10T01:32:15.748 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-06-10T01:32:15.748 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-06-10T01:32:15.748 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-06-10T01:32:15.748 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-06-10T01:32:15.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.748 INFO:teuthology.orchestra.run.smithi007.stdout: "last_failure_osd_epoch": 3, 2024-06-10T01:32:15.748 INFO:teuthology.orchestra.run.smithi007.stdout: "active_clients": [ 2024-06-10T01:32:15.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.749 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-06-10T01:32:15.749 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-06-10T01:32:15.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.749 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-06-10T01:32:15.749 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-06-10T01:32:15.749 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 307439401 2024-06-10T01:32:15.749 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.749 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-06-10T01:32:15.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.749 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "libcephsqlite", 2024-06-10T01:32:15.749 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-06-10T01:32:15.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.749 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-06-10T01:32:15.749 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-06-10T01:32:15.750 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2795460416 2024-06-10T01:32:15.750 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.750 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-06-10T01:32:15.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.750 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-06-10T01:32:15.750 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-06-10T01:32:15.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.750 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-06-10T01:32:15.750 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-06-10T01:32:15.750 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 951050500 2024-06-10T01:32:15.750 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.750 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-06-10T01:32:15.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:15.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.751 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-06-10T01:32:15.751 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-06-10T01:32:15.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-10T01:32:15.751 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-06-10T01:32:15.751 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-06-10T01:32:15.751 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4252116269 2024-06-10T01:32:15.751 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.751 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-06-10T01:32:15.751 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:15.751 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-06-10T01:32:15.751 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:32:15.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.445+0000 7fec1569a640 1 -- 172.21.15.7:0/548895528 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7febfc0a2b90 msgr2=0x7febfc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:15.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.445+0000 7fec1569a640 1 --2- 172.21.15.7:0/548895528 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7febfc0a2b90 0x7febfc0a5050 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fec10198b60 tx=0x7fec00009500 comp rx=0 tx=0).stop 2024-06-10T01:32:15.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.445+0000 7fec1569a640 1 -- 172.21.15.7:0/548895528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec10104db0 msgr2=0x7fec101981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:15.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.445+0000 7fec1569a640 1 --2- 172.21.15.7:0/548895528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec10104db0 0x7fec101981d0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7febf800e9f0 tx=0x7febf800eeb0 comp rx=0 tx=0).stop 2024-06-10T01:32:15.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.445+0000 7fec1569a640 1 -- 172.21.15.7:0/548895528 shutdown_connections 2024-06-10T01:32:15.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.445+0000 7fec1569a640 1 --2- 172.21.15.7:0/548895528 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7febfc0a2b90 0x7febfc0a5050 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:15.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.445+0000 7fec1569a640 1 --2- 172.21.15.7:0/548895528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec101076e0 0x7fec1019f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:15.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.445+0000 7fec1569a640 1 --2- 172.21.15.7:0/548895528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec10104db0 0x7fec101981d0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:15.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.445+0000 7fec1569a640 1 --2- 172.21.15.7:0/548895528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec10102480 0x7fec10197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:15.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.445+0000 7fec1569a640 1 -- 172.21.15.7:0/548895528 >> 172.21.15.7:0/548895528 conn(0x7fec100fbe70 msgr2=0x7fec10102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:15.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.445+0000 7fec1569a640 1 -- 172.21.15.7:0/548895528 shutdown_connections 2024-06-10T01:32:15.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:14.445+0000 7fec1569a640 1 -- 172.21.15.7:0/548895528 wait complete. 2024-06-10T01:32:15.754 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:14 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1215100547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-10T01:32:15.754 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:14 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/154317596' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-10T01:32:15.754 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:14 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2542833302' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-10T01:32:15.754 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:15 smithi078 ceph-mon[36683]: pgmap v1221: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 9 op/s 2024-06-10T01:32:15.754 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:15 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/548895528' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-10T01:32:15.754 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:14 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1215100547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-10T01:32:15.754 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:14 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/154317596' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-10T01:32:15.754 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:14 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2542833302' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-10T01:32:15.754 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:15 smithi175 ceph-mon[36636]: pgmap v1221: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 9 op/s 2024-06-10T01:32:15.754 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:15 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/548895528' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-10T01:32:15.754 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:14 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1215100547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-10T01:32:15.755 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:14 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/154317596' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-10T01:32:15.755 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:14 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2542833302' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-10T01:32:15.803 DEBUG:tasks.check_counter:Getting stats from a 2024-06-10T01:32:15.803 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph daemon mds.a perf dump 2024-06-10T01:32:15.804 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:15 smithi007 ceph-mon[32398]: pgmap v1221: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 9 op/s 2024-06-10T01:32:15.804 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:15 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/548895528' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-10T01:32:16.067 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:32:17.430 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:17 smithi007 ceph-mon[32398]: pgmap v1222: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.1 KiB/s wr, 8 op/s 2024-06-10T01:32:17.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:17 smithi175 ceph-mon[36636]: pgmap v1222: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.1 KiB/s wr, 8 op/s 2024-06-10T01:32:17.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:17 smithi078 ceph-mon[36683]: pgmap v1222: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.1 KiB/s wr, 8 op/s 2024-06-10T01:32:17.859 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:32:17.859 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T01:32:17.859 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 496, 2024-06-10T01:32:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 491, 2024-06-10T01:32:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 559680, 2024-06-10T01:32:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 303296, 2024-06-10T01:32:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-06-10T01:32:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 0, 2024-06-10T01:32:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.065696426, 2024-06-10T01:32:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.028321940, 2024-06-10T01:32:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.021457571, 2024-06-10T01:32:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.010162368, 2024-06-10T01:32:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 491, 2024-06-10T01:32:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.030687968, 2024-06-10T01:32:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000062500 2024-06-10T01:32:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-10T01:32:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-10T01:32:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-10T01:32:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 559680, 2024-06-10T01:32:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 303296 2024-06-10T01:32:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T01:32:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 3, 2024-06-10T01:32:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 375, 2024-06-10T01:32:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 480, 2024-06-10T01:32:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 299984, 2024-06-10T01:32:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 5, 2024-06-10T01:32:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 2, 2024-06-10T01:32:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.025996919, 2024-06-10T01:32:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.020531932, 2024-06-10T01:32:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.001960669, 2024-06-10T01:32:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T01:32:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 373, 2024-06-10T01:32:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.021175217, 2024-06-10T01:32:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000056770 2024-06-10T01:32:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-10T01:32:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-10T01:32:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-10T01:32:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 480, 2024-06-10T01:32:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 299984 2024-06-10T01:32:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T01:32:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 0, 2024-06-10T01:32:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 2, 2024-06-10T01:32:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 0, 2024-06-10T01:32:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 0, 2024-06-10T01:32:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 2, 2024-06-10T01:32:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 1, 2024-06-10T01:32:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.000299632, 2024-06-10T01:32:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.000000000, 2024-06-10T01:32:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.000194624, 2024-06-10T01:32:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T01:32:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-10T01:32:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-10T01:32:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-10T01:32:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-10T01:32:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-10T01:32:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-10T01:32:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-10T01:32:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 0 2024-06-10T01:32:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-06-10T01:32:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 0, 2024-06-10T01:32:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-06-10T01:32:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-06-10T01:32:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 0, 2024-06-10T01:32:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 0, 2024-06-10T01:32:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T01:32:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-06-10T01:32:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T01:32:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-06-10T01:32:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T01:32:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T01:32:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T01:32:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T01:32:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T01:32:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-06-10T01:32:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T01:32:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T01:32:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-06-10T01:32:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-06-10T01:32:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-06-10T01:32:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-06-10T01:32:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T01:32:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T01:32:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T01:32:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T01:32:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T01:32:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-06-10T01:32:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-06-10T01:32:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-06-10T01:32:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T01:32:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T01:32:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-06-10T01:32:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-06-10T01:32:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-06-10T01:32:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-06-10T01:32:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T01:32:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-06-10T01:32:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T01:32:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-06-10T01:32:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 147920, 2024-06-10T01:32:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 27, 2024-06-10T01:32:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 0, 2024-06-10T01:32:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 0, 2024-06-10T01:32:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-06-10T01:32:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-06-10T01:32:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-06-10T01:32:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-06-10T01:32:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-06-10T01:32:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-06-10T01:32:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 0, 2024-06-10T01:32:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 0, 2024-06-10T01:32:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T01:32:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-06-10T01:32:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-06-10T01:32:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-06-10T01:32:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 928, 2024-06-10T01:32:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 2, 2024-06-10T01:32:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-06-10T01:32:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-06-10T01:32:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-06-10T01:32:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-06-10T01:32:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T01:32:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-06-10T01:32:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-06-10T01:32:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-06-10T01:32:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 495, 2024-06-10T01:32:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 487113, 2024-06-10T01:32:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 495, 2024-06-10T01:32:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-06-10T01:32:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-06-10T01:32:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 495, 2024-06-10T01:32:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 487113, 2024-06-10T01:32:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-06-10T01:32:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-10T01:32:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-10T01:32:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-10T01:32:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:17.871 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:32:18.660 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-06-10T01:32:18.661 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-06-10T01:32:18.661 DEBUG:tasks.check_counter:Getting stats from d 2024-06-10T01:32:18.661 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph daemon mds.d perf dump 2024-06-10T01:32:18.919 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:32:19.371 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:19 smithi007 ceph-mon[32398]: pgmap v1223: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:32:19.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:19 smithi175 ceph-mon[36636]: pgmap v1223: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:32:19.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:19 smithi078 ceph-mon[36683]: pgmap v1223: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:32:20.622 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:32:20.622 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T01:32:20.622 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 0, 2024-06-10T01:32:20.622 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 2, 2024-06-10T01:32:20.623 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 0, 2024-06-10T01:32:20.623 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 0, 2024-06-10T01:32:20.623 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-06-10T01:32:20.623 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 0, 2024-06-10T01:32:20.623 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.001441312, 2024-06-10T01:32:20.623 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.000000000, 2024-06-10T01:32:20.623 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.000362021, 2024-06-10T01:32:20.623 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T01:32:20.623 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:20.623 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-10T01:32:20.623 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-10T01:32:20.623 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-10T01:32:20.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:20.624 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:20.624 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-10T01:32:20.624 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-10T01:32:20.624 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-10T01:32:20.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:20.624 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-10T01:32:20.624 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 0 2024-06-10T01:32:20.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:20.624 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T01:32:20.624 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 503, 2024-06-10T01:32:20.624 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 498, 2024-06-10T01:32:20.624 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 561584, 2024-06-10T01:32:20.624 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 307776, 2024-06-10T01:32:20.625 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 2, 2024-06-10T01:32:20.625 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 1, 2024-06-10T01:32:20.625 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.065673252, 2024-06-10T01:32:20.625 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.029536102, 2024-06-10T01:32:20.625 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.021856049, 2024-06-10T01:32:20.625 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.009764659, 2024-06-10T01:32:20.625 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:20.625 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 498, 2024-06-10T01:32:20.625 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.026775656, 2024-06-10T01:32:20.625 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000053766 2024-06-10T01:32:20.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:20.625 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:20.625 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-10T01:32:20.625 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-10T01:32:20.626 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-10T01:32:20.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:20.626 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 561584, 2024-06-10T01:32:20.626 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 307776 2024-06-10T01:32:20.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:20.626 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T01:32:20.626 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 3, 2024-06-10T01:32:20.626 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 380, 2024-06-10T01:32:20.626 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 480, 2024-06-10T01:32:20.626 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 300864, 2024-06-10T01:32:20.626 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 5, 2024-06-10T01:32:20.626 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 2, 2024-06-10T01:32:20.626 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.026972494, 2024-06-10T01:32:20.626 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.021541403, 2024-06-10T01:32:20.626 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.002014037, 2024-06-10T01:32:20.627 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T01:32:20.627 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:20.627 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 378, 2024-06-10T01:32:20.627 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.018334612, 2024-06-10T01:32:20.627 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000048504 2024-06-10T01:32:20.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:20.627 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:20.627 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-10T01:32:20.627 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-10T01:32:20.627 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-10T01:32:20.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:20.627 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 480, 2024-06-10T01:32:20.627 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 300864 2024-06-10T01:32:20.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:20.627 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-06-10T01:32:20.628 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 0, 2024-06-10T01:32:20.628 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-06-10T01:32:20.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:20.628 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-06-10T01:32:20.628 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 0, 2024-06-10T01:32:20.628 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 0, 2024-06-10T01:32:20.628 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T01:32:20.628 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-06-10T01:32:20.628 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T01:32:20.628 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-06-10T01:32:20.628 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T01:32:20.628 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T01:32:20.628 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T01:32:20.628 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T01:32:20.628 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T01:32:20.629 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-06-10T01:32:20.629 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T01:32:20.629 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T01:32:20.629 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-06-10T01:32:20.629 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-06-10T01:32:20.629 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-06-10T01:32:20.629 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-06-10T01:32:20.629 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T01:32:20.629 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T01:32:20.629 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T01:32:20.629 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T01:32:20.629 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T01:32:20.629 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-06-10T01:32:20.629 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-06-10T01:32:20.629 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-06-10T01:32:20.630 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T01:32:20.630 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T01:32:20.630 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-06-10T01:32:20.630 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-06-10T01:32:20.630 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-06-10T01:32:20.630 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-06-10T01:32:20.630 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T01:32:20.630 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-06-10T01:32:20.630 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T01:32:20.630 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-06-10T01:32:20.630 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 148744, 2024-06-10T01:32:20.630 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 30, 2024-06-10T01:32:20.630 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 0, 2024-06-10T01:32:20.630 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 0, 2024-06-10T01:32:20.630 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-06-10T01:32:20.631 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-06-10T01:32:20.631 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-06-10T01:32:20.631 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-06-10T01:32:20.631 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-06-10T01:32:20.631 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-06-10T01:32:20.631 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 0, 2024-06-10T01:32:20.631 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 0, 2024-06-10T01:32:20.631 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T01:32:20.631 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-06-10T01:32:20.631 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-06-10T01:32:20.631 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-06-10T01:32:20.631 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 928, 2024-06-10T01:32:20.631 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 2, 2024-06-10T01:32:20.631 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-06-10T01:32:20.632 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-06-10T01:32:20.632 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-06-10T01:32:20.632 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-06-10T01:32:20.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:20.632 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T01:32:20.632 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-06-10T01:32:20.632 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-06-10T01:32:20.632 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-06-10T01:32:20.632 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 502, 2024-06-10T01:32:20.632 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 488023, 2024-06-10T01:32:20.632 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:20.632 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 502, 2024-06-10T01:32:20.632 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-06-10T01:32:20.632 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-06-10T01:32:20.632 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 502, 2024-06-10T01:32:20.632 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 488023, 2024-06-10T01:32:20.633 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-06-10T01:32:20.633 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-10T01:32:20.633 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-10T01:32:20.633 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-10T01:32:20.633 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:20.633 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:20.633 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:32:21.260 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:21 smithi007 ceph-mon[32398]: pgmap v1224: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:32:21.342 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-06-10T01:32:21.342 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-06-10T01:32:21.342 DEBUG:tasks.check_counter:Getting stats from g 2024-06-10T01:32:21.342 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph daemon mds.g perf dump 2024-06-10T01:32:21.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:21 smithi175 ceph-mon[36636]: pgmap v1224: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:32:21.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:21 smithi078 ceph-mon[36683]: pgmap v1224: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:32:21.597 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:32:23.260 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:23 smithi007 ceph-mon[32398]: pgmap v1225: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:32:23.401 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:32:23.401 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T01:32:23.401 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 495, 2024-06-10T01:32:23.401 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 492, 2024-06-10T01:32:23.401 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 557504, 2024-06-10T01:32:23.401 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 304752, 2024-06-10T01:32:23.401 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-06-10T01:32:23.401 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 0, 2024-06-10T01:32:23.401 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.067008482, 2024-06-10T01:32:23.401 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.028598160, 2024-06-10T01:32:23.401 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.020450707, 2024-06-10T01:32:23.402 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.009702396, 2024-06-10T01:32:23.402 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:23.402 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 491, 2024-06-10T01:32:23.402 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.021967883, 2024-06-10T01:32:23.402 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000044741 2024-06-10T01:32:23.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:23.402 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:23.402 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-10T01:32:23.402 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-10T01:32:23.402 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-10T01:32:23.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:23.402 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 557504, 2024-06-10T01:32:23.402 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 304752 2024-06-10T01:32:23.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:23.403 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T01:32:23.403 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 7, 2024-06-10T01:32:23.403 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 379, 2024-06-10T01:32:23.403 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 3472, 2024-06-10T01:32:23.403 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 300784, 2024-06-10T01:32:23.403 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 5, 2024-06-10T01:32:23.403 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 2, 2024-06-10T01:32:23.403 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.027535108, 2024-06-10T01:32:23.403 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.020969862, 2024-06-10T01:32:23.403 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.003023690, 2024-06-10T01:32:23.403 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T01:32:23.403 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:23.403 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 378, 2024-06-10T01:32:23.404 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.018228043, 2024-06-10T01:32:23.404 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000048222 2024-06-10T01:32:23.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:23.404 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:23.404 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-10T01:32:23.404 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-10T01:32:23.404 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-10T01:32:23.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:23.404 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 3472, 2024-06-10T01:32:23.404 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 300784 2024-06-10T01:32:23.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:23.404 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T01:32:23.404 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 0, 2024-06-10T01:32:23.405 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 2, 2024-06-10T01:32:23.405 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 0, 2024-06-10T01:32:23.405 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 0, 2024-06-10T01:32:23.405 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 2, 2024-06-10T01:32:23.405 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 1, 2024-06-10T01:32:23.405 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.000336734, 2024-06-10T01:32:23.405 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.000000000, 2024-06-10T01:32:23.405 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.000260099, 2024-06-10T01:32:23.405 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T01:32:23.405 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:23.405 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-10T01:32:23.405 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-10T01:32:23.405 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-10T01:32:23.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:23.406 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:23.406 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-10T01:32:23.406 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-10T01:32:23.406 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-10T01:32:23.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:23.406 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-10T01:32:23.406 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 0 2024-06-10T01:32:23.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:23.406 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-06-10T01:32:23.406 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 0, 2024-06-10T01:32:23.406 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-06-10T01:32:23.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:23.406 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-06-10T01:32:23.407 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 0, 2024-06-10T01:32:23.407 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 0, 2024-06-10T01:32:23.407 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T01:32:23.407 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-06-10T01:32:23.407 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T01:32:23.407 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-06-10T01:32:23.407 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T01:32:23.407 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T01:32:23.407 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T01:32:23.407 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T01:32:23.407 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T01:32:23.407 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-06-10T01:32:23.407 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T01:32:23.407 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T01:32:23.407 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-06-10T01:32:23.408 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-06-10T01:32:23.408 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-06-10T01:32:23.408 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-06-10T01:32:23.408 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T01:32:23.408 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T01:32:23.408 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T01:32:23.408 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T01:32:23.408 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T01:32:23.408 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-06-10T01:32:23.408 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-06-10T01:32:23.408 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-06-10T01:32:23.408 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T01:32:23.408 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T01:32:23.408 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-06-10T01:32:23.408 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-06-10T01:32:23.409 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-06-10T01:32:23.409 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-06-10T01:32:23.409 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T01:32:23.409 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-06-10T01:32:23.409 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T01:32:23.409 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-06-10T01:32:23.409 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 147937, 2024-06-10T01:32:23.409 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 28, 2024-06-10T01:32:23.409 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 0, 2024-06-10T01:32:23.409 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 0, 2024-06-10T01:32:23.409 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-06-10T01:32:23.409 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-06-10T01:32:23.409 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-06-10T01:32:23.409 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-06-10T01:32:23.410 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-06-10T01:32:23.410 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-06-10T01:32:23.410 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 0, 2024-06-10T01:32:23.410 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 0, 2024-06-10T01:32:23.410 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T01:32:23.410 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-06-10T01:32:23.410 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-06-10T01:32:23.410 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-06-10T01:32:23.410 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 928, 2024-06-10T01:32:23.410 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 2, 2024-06-10T01:32:23.410 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-06-10T01:32:23.410 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-06-10T01:32:23.410 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-06-10T01:32:23.410 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-06-10T01:32:23.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:23.411 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T01:32:23.411 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-06-10T01:32:23.411 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-06-10T01:32:23.411 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-06-10T01:32:23.411 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 498, 2024-06-10T01:32:23.411 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 487503, 2024-06-10T01:32:23.411 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:23.411 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 498, 2024-06-10T01:32:23.411 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-06-10T01:32:23.411 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-06-10T01:32:23.411 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 498, 2024-06-10T01:32:23.411 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 487503, 2024-06-10T01:32:23.411 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-06-10T01:32:23.412 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-10T01:32:23.412 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-10T01:32:23.412 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-10T01:32:23.412 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:23.412 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:23.412 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:32:23.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:23 smithi175 ceph-mon[36636]: pgmap v1225: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:32:23.576 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:23 smithi078 ceph-mon[36683]: pgmap v1225: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:32:24.077 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-06-10T01:32:24.078 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-06-10T01:32:24.078 DEBUG:tasks.check_counter:Getting stats from j 2024-06-10T01:32:24.078 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph daemon mds.j perf dump 2024-06-10T01:32:24.333 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:32:25.415 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:25 smithi007 ceph-mon[32398]: pgmap v1226: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:32:25.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:25 smithi175 ceph-mon[36636]: pgmap v1226: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:32:25.576 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:25 smithi078 ceph-mon[36683]: pgmap v1226: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:32:25.979 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-10T01:32:25.979 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T01:32:25.979 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 4, 2024-06-10T01:32:25.979 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 4, 2024-06-10T01:32:25.979 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 2992, 2024-06-10T01:32:25.979 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 448, 2024-06-10T01:32:25.979 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-06-10T01:32:25.979 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 0, 2024-06-10T01:32:25.979 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.001674204, 2024-06-10T01:32:25.979 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.000046697, 2024-06-10T01:32:25.979 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.000643537, 2024-06-10T01:32:25.980 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T01:32:25.980 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:25.980 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 3, 2024-06-10T01:32:25.980 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000744508, 2024-06-10T01:32:25.980 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000248169 2024-06-10T01:32:25.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:25.980 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:25.980 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-10T01:32:25.980 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-10T01:32:25.980 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-10T01:32:25.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:25.980 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-06-10T01:32:25.980 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 448 2024-06-10T01:32:25.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:25.980 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T01:32:25.981 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 501, 2024-06-10T01:32:25.981 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 498, 2024-06-10T01:32:25.981 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 559136, 2024-06-10T01:32:25.981 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 308608, 2024-06-10T01:32:25.981 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 2, 2024-06-10T01:32:25.981 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 1, 2024-06-10T01:32:25.981 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.065194909, 2024-06-10T01:32:25.981 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.029400280, 2024-06-10T01:32:25.981 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.021214287, 2024-06-10T01:32:25.981 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.009785273, 2024-06-10T01:32:25.981 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:25.981 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 497, 2024-06-10T01:32:25.981 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.022011261, 2024-06-10T01:32:25.981 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000044288 2024-06-10T01:32:25.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:25.982 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:25.982 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-10T01:32:25.982 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-10T01:32:25.982 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-10T01:32:25.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:25.982 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 559136, 2024-06-10T01:32:25.982 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 308608 2024-06-10T01:32:25.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:25.982 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T01:32:25.982 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 3, 2024-06-10T01:32:25.982 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 382, 2024-06-10T01:32:25.982 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 480, 2024-06-10T01:32:25.982 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 301216, 2024-06-10T01:32:25.983 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 5, 2024-06-10T01:32:25.983 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 2, 2024-06-10T01:32:25.983 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.027135550, 2024-06-10T01:32:25.983 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.021362598, 2024-06-10T01:32:25.983 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.002082185, 2024-06-10T01:32:25.983 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T01:32:25.983 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:25.983 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 380, 2024-06-10T01:32:25.983 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.022597365, 2024-06-10T01:32:25.983 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000059466 2024-06-10T01:32:25.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:25.983 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:25.983 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-10T01:32:25.983 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-10T01:32:25.983 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-10T01:32:25.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:25.984 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 480, 2024-06-10T01:32:25.984 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 301216 2024-06-10T01:32:25.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:25.984 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-06-10T01:32:25.984 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 0, 2024-06-10T01:32:25.984 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-06-10T01:32:25.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:25.984 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-06-10T01:32:25.984 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 0, 2024-06-10T01:32:25.984 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 0, 2024-06-10T01:32:25.984 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T01:32:25.984 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-06-10T01:32:25.984 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T01:32:25.984 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-06-10T01:32:25.985 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T01:32:25.985 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T01:32:25.985 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T01:32:25.985 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T01:32:25.985 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T01:32:25.985 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-06-10T01:32:25.985 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T01:32:25.985 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T01:32:25.985 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-06-10T01:32:25.985 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-06-10T01:32:25.985 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-06-10T01:32:25.985 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-06-10T01:32:25.985 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T01:32:25.985 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T01:32:25.986 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T01:32:25.986 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T01:32:25.986 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T01:32:25.986 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-06-10T01:32:25.986 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-06-10T01:32:25.986 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-06-10T01:32:25.986 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T01:32:25.986 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T01:32:25.986 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-06-10T01:32:25.986 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-06-10T01:32:25.986 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-06-10T01:32:25.986 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-06-10T01:32:25.986 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T01:32:25.987 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-06-10T01:32:25.987 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T01:32:25.987 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-06-10T01:32:25.987 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 147920, 2024-06-10T01:32:25.987 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 27, 2024-06-10T01:32:25.987 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 0, 2024-06-10T01:32:25.987 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 0, 2024-06-10T01:32:25.987 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-06-10T01:32:25.987 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-06-10T01:32:25.987 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-06-10T01:32:25.987 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-06-10T01:32:25.987 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-06-10T01:32:25.987 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-06-10T01:32:25.987 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 0, 2024-06-10T01:32:25.987 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 0, 2024-06-10T01:32:25.988 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T01:32:25.988 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-06-10T01:32:25.988 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-06-10T01:32:25.988 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-06-10T01:32:25.988 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 928, 2024-06-10T01:32:25.988 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 2, 2024-06-10T01:32:25.988 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-06-10T01:32:25.988 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-06-10T01:32:25.988 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-06-10T01:32:25.988 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-06-10T01:32:25.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-10T01:32:25.988 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T01:32:25.988 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-06-10T01:32:25.988 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-06-10T01:32:25.988 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-06-10T01:32:25.989 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 504, 2024-06-10T01:32:25.989 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 488283, 2024-06-10T01:32:25.989 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:25.989 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 504, 2024-06-10T01:32:25.989 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-06-10T01:32:25.989 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-06-10T01:32:25.989 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 504, 2024-06-10T01:32:25.989 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 488283, 2024-06-10T01:32:25.989 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-06-10T01:32:25.989 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-10T01:32:25.989 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-10T01:32:25.989 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-10T01:32:25.989 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:25.989 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-10T01:32:25.989 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-10T01:32:26.662 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-06-10T01:32:26.662 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-06-10T01:32:26.662 DEBUG:tasks.check_counter:Getting stats from b 2024-06-10T01:32:26.662 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph daemon mds.b perf dump 2024-06-10T01:32:26.991 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T01:32:27.369 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:27 smithi078 ceph-mon[36683]: pgmap v1227: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:32:27.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:27 smithi175 ceph-mon[36636]: pgmap v1227: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:32:27.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:27 smithi007 ceph-mon[32398]: pgmap v1227: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:32:28.780 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-06-10T01:32:28.780 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T01:32:28.780 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 465524, 2024-06-10T01:32:28.780 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 705178, 2024-06-10T01:32:28.780 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 162733515, 2024-06-10T01:32:28.780 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 716183208, 2024-06-10T01:32:28.780 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 64, 2024-06-10T01:32:28.780 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 3, 2024-06-10T01:32:28.780 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 31.873496452, 2024-06-10T01:32:28.781 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 13.986753194, 2024-06-10T01:32:28.781 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 18.464984912, 2024-06-10T01:32:28.781 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 2.560908283, 2024-06-10T01:32:28.781 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:28.781 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 705176, 2024-06-10T01:32:28.781 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 40.662251825, 2024-06-10T01:32:28.781 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000057662 2024-06-10T01:32:28.781 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.781 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:28.781 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 949550, 2024-06-10T01:32:28.781 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.141459763, 2024-06-10T01:32:28.781 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000148 2024-06-10T01:32:28.781 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.782 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 145707040, 2024-06-10T01:32:28.782 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 538160672 2024-06-10T01:32:28.782 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.782 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T01:32:28.782 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 91859, 2024-06-10T01:32:28.782 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 93858, 2024-06-10T01:32:28.782 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 22740224, 2024-06-10T01:32:28.782 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 162261970, 2024-06-10T01:32:28.782 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 9, 2024-06-10T01:32:28.782 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 6, 2024-06-10T01:32:28.782 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 6.283859608, 2024-06-10T01:32:28.783 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 2.011123638, 2024-06-10T01:32:28.783 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 2.413979102, 2024-06-10T01:32:28.783 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 1.894456609, 2024-06-10T01:32:28.783 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:28.783 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 93856, 2024-06-10T01:32:28.783 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 6.848218662, 2024-06-10T01:32:28.783 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000072965 2024-06-10T01:32:28.783 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.783 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:28.783 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 80228, 2024-06-10T01:32:28.783 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.013157133, 2024-06-10T01:32:28.783 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000163 2024-06-10T01:32:28.783 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.783 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-10T01:32:28.784 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 815232 2024-06-10T01:32:28.784 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.784 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T01:32:28.784 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 273645, 2024-06-10T01:32:28.784 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 254369, 2024-06-10T01:32:28.784 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 76194986, 2024-06-10T01:32:28.784 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 202650675, 2024-06-10T01:32:28.784 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 20, 2024-06-10T01:32:28.784 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 5, 2024-06-10T01:32:28.784 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 14.745206745, 2024-06-10T01:32:28.784 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 5.915228180, 2024-06-10T01:32:28.784 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 8.588421115, 2024-06-10T01:32:28.784 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 1.729391498, 2024-06-10T01:32:28.785 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:28.785 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 254369, 2024-06-10T01:32:28.785 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 9.938925106, 2024-06-10T01:32:28.785 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000039072 2024-06-10T01:32:28.785 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.785 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:28.785 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 426320, 2024-06-10T01:32:28.785 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.056878194, 2024-06-10T01:32:28.785 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000133 2024-06-10T01:32:28.785 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.785 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 747808, 2024-06-10T01:32:28.786 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 336816 2024-06-10T01:32:28.786 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.786 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-06-10T01:32:28.786 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 1, 2024-06-10T01:32:28.786 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-06-10T01:32:28.786 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.786 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-MDSRank": { 2024-06-10T01:32:28.786 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-06-10T01:32:28.786 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-06-10T01:32:28.786 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 113525, 2024-06-10T01:32:28.786 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 166.908361305, 2024-06-10T01:32:28.786 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.001470234 2024-06-10T01:32:28.786 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-06-10T01:32:28.787 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.787 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-PurgeQueue": { 2024-06-10T01:32:28.787 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-06-10T01:32:28.787 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-06-10T01:32:28.787 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 82943, 2024-06-10T01:32:28.787 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 12.178872025, 2024-06-10T01:32:28.787 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000146834 2024-06-10T01:32:28.787 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-06-10T01:32:28.787 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.787 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": { 2024-06-10T01:32:28.787 INFO:teuthology.orchestra.run.smithi078.stdout: "request": 286433, 2024-06-10T01:32:28.787 INFO:teuthology.orchestra.run.smithi078.stdout: "reply": 270637, 2024-06-10T01:32:28.787 INFO:teuthology.orchestra.run.smithi078.stdout: "reply_latency": { 2024-06-10T01:32:28.788 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 270637, 2024-06-10T01:32:28.788 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 1791.099127012, 2024-06-10T01:32:28.788 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.006618086 2024-06-10T01:32:28.788 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.788 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_reply": 0, 2024-06-10T01:32:28.788 INFO:teuthology.orchestra.run.smithi078.stdout: "forward": 15796, 2024-06-10T01:32:28.788 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_complete": 1, 2024-06-10T01:32:28.788 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_keys": 0, 2024-06-10T01:32:28.788 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_commit": 135, 2024-06-10T01:32:28.788 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_split": 14, 2024-06-10T01:32:28.788 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_merge": 12, 2024-06-10T01:32:28.788 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes": 1611, 2024-06-10T01:32:28.788 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_top": 863, 2024-06-10T01:32:28.789 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_bottom": 379, 2024-06-10T01:32:28.789 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pin_tail": 369, 2024-06-10T01:32:28.789 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pinned": 378, 2024-06-10T01:32:28.789 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_expired": 652, 2024-06-10T01:32:28.789 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_with_caps": 1, 2024-06-10T01:32:28.789 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": 1, 2024-06-10T01:32:28.789 INFO:teuthology.orchestra.run.smithi078.stdout: "subtrees": 4, 2024-06-10T01:32:28.789 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse": 484829, 2024-06-10T01:32:28.789 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_hit": 258322, 2024-06-10T01:32:28.789 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_forward": 15791, 2024-06-10T01:32:28.789 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_discover": 38, 2024-06-10T01:32:28.789 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_dir_fetch": 0, 2024-06-10T01:32:28.789 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_remote_ino": 0, 2024-06-10T01:32:28.789 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_lock": 6080, 2024-06-10T01:32:28.790 INFO:teuthology.orchestra.run.smithi078.stdout: "load_cent": 1000, 2024-06-10T01:32:28.790 INFO:teuthology.orchestra.run.smithi078.stdout: "q": 1, 2024-06-10T01:32:28.790 INFO:teuthology.orchestra.run.smithi078.stdout: "exported": 17, 2024-06-10T01:32:28.790 INFO:teuthology.orchestra.run.smithi078.stdout: "exported_inodes": 351, 2024-06-10T01:32:28.790 INFO:teuthology.orchestra.run.smithi078.stdout: "imported": 15, 2024-06-10T01:32:28.790 INFO:teuthology.orchestra.run.smithi078.stdout: "imported_inodes": 684, 2024-06-10T01:32:28.790 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_dir_fetch": 0, 2024-06-10T01:32:28.790 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_backtrace_fetch": 3216, 2024-06-10T01:32:28.790 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_peer_discover": 0, 2024-06-10T01:32:28.790 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rfiles": 698, 2024-06-10T01:32:28.790 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rbytes": 211204299, 2024-06-10T01:32:28.790 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rsnaps": 0, 2024-06-10T01:32:28.790 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_fetch": 3216, 2024-06-10T01:32:28.791 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_set_tag": 0, 2024-06-10T01:32:28.791 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T01:32:28.791 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_inotable_repaired": 0, 2024-06-10T01:32:28.791 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_inodes": 395, 2024-06-10T01:32:28.791 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_base_inodes": 0, 2024-06-10T01:32:28.791 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dirfrag_rstats": 395, 2024-06-10T01:32:28.791 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_file_inodes": 2821, 2024-06-10T01:32:28.791 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_inode_file_caps": 27, 2024-06-10T01:32:28.791 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_revoke": 24776, 2024-06-10T01:32:28.791 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_grant": 191466, 2024-06-10T01:32:28.791 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_trunc": 0, 2024-06-10T01:32:28.791 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T01:32:28.791 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T01:32:28.791 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps": 87136, 2024-06-10T01:32:28.792 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps_dirty": 31273, 2024-06-10T01:32:28.792 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_cap_release": 16725, 2024-06-10T01:32:28.792 INFO:teuthology.orchestra.run.smithi078.stdout: "process_request_cap_release": 194257 2024-06-10T01:32:28.792 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.792 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_cache": { 2024-06-10T01:32:28.792 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update": 92, 2024-06-10T01:32:28.792 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update_receipt": 217, 2024-06-10T01:32:28.792 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_try_discover": 117, 2024-06-10T01:32:28.792 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_send_discover": 236, 2024-06-10T01:32:28.792 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_handle_discover": 154, 2024-06-10T01:32:28.792 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays": 1, 2024-06-10T01:32:28.792 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_delayed": 0, 2024-06-10T01:32:28.792 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_enqueuing": 0, 2024-06-10T01:32:28.792 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_created": 45139, 2024-06-10T01:32:28.793 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_enqueued": 45138, 2024-06-10T01:32:28.793 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_reintegrated": 0, 2024-06-10T01:32:28.793 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_migrated": 0, 2024-06-10T01:32:28.793 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_processing": 0, 2024-06-10T01:32:28.793 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_enqueued": 0, 2024-06-10T01:32:28.793 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_prioritized": 0, 2024-06-10T01:32:28.793 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_started": 0, 2024-06-10T01:32:28.793 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_completed": 0, 2024-06-10T01:32:28.793 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_path": 20, 2024-06-10T01:32:28.793 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_inode": 2958, 2024-06-10T01:32:28.793 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_enqueue_scrub": 0, 2024-06-10T01:32:28.793 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_exportdir": 17, 2024-06-10T01:32:28.793 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_flush": 0, 2024-06-10T01:32:28.793 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragmentdir": 26, 2024-06-10T01:32:28.794 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragstats": 0, 2024-06-10T01:32:28.794 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_inodestats": 0 2024-06-10T01:32:28.794 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.794 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_log": { 2024-06-10T01:32:28.794 INFO:teuthology.orchestra.run.smithi078.stdout: "evlrg": 0, 2024-06-10T01:32:28.794 INFO:teuthology.orchestra.run.smithi078.stdout: "evadd": 227780, 2024-06-10T01:32:28.794 INFO:teuthology.orchestra.run.smithi078.stdout: "evex": 122472, 2024-06-10T01:32:28.794 INFO:teuthology.orchestra.run.smithi078.stdout: "evtrm": 110611, 2024-06-10T01:32:28.794 INFO:teuthology.orchestra.run.smithi078.stdout: "ev": 117169, 2024-06-10T01:32:28.794 INFO:teuthology.orchestra.run.smithi078.stdout: "evexg": 0, 2024-06-10T01:32:28.794 INFO:teuthology.orchestra.run.smithi078.stdout: "evexd": 11861, 2024-06-10T01:32:28.794 INFO:teuthology.orchestra.run.smithi078.stdout: "segadd": 271, 2024-06-10T01:32:28.794 INFO:teuthology.orchestra.run.smithi078.stdout: "segex": 143, 2024-06-10T01:32:28.794 INFO:teuthology.orchestra.run.smithi078.stdout: "segtrm": 130, 2024-06-10T01:32:28.795 INFO:teuthology.orchestra.run.smithi078.stdout: "seg": 141, 2024-06-10T01:32:28.795 INFO:teuthology.orchestra.run.smithi078.stdout: "segmjr": 10, 2024-06-10T01:32:28.795 INFO:teuthology.orchestra.run.smithi078.stdout: "segexg": 0, 2024-06-10T01:32:28.795 INFO:teuthology.orchestra.run.smithi078.stdout: "segexd": 13, 2024-06-10T01:32:28.795 INFO:teuthology.orchestra.run.smithi078.stdout: "expos": 182080609, 2024-06-10T01:32:28.795 INFO:teuthology.orchestra.run.smithi078.stdout: "wrpos": 425834674, 2024-06-10T01:32:28.795 INFO:teuthology.orchestra.run.smithi078.stdout: "rdpos": 0, 2024-06-10T01:32:28.795 INFO:teuthology.orchestra.run.smithi078.stdout: "jlat": { 2024-06-10T01:32:28.795 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 62695, 2024-06-10T01:32:28.795 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 628.512379447, 2024-06-10T01:32:28.795 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.010024920 2024-06-10T01:32:28.795 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.795 INFO:teuthology.orchestra.run.smithi078.stdout: "replayed": 0 2024-06-10T01:32:28.795 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.796 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_mem": { 2024-06-10T01:32:28.796 INFO:teuthology.orchestra.run.smithi078.stdout: "ino": 30, 2024-06-10T01:32:28.796 INFO:teuthology.orchestra.run.smithi078.stdout: "ino+": 45419, 2024-06-10T01:32:28.796 INFO:teuthology.orchestra.run.smithi078.stdout: "ino-": 45389, 2024-06-10T01:32:28.796 INFO:teuthology.orchestra.run.smithi078.stdout: "dir": 50, 2024-06-10T01:32:28.796 INFO:teuthology.orchestra.run.smithi078.stdout: "dir+": 437, 2024-06-10T01:32:28.796 INFO:teuthology.orchestra.run.smithi078.stdout: "dir-": 387, 2024-06-10T01:32:28.796 INFO:teuthology.orchestra.run.smithi078.stdout: "dn": 1611, 2024-06-10T01:32:28.796 INFO:teuthology.orchestra.run.smithi078.stdout: "dn+": 85229, 2024-06-10T01:32:28.796 INFO:teuthology.orchestra.run.smithi078.stdout: "dn-": 83618, 2024-06-10T01:32:28.796 INFO:teuthology.orchestra.run.smithi078.stdout: "cap": 1, 2024-06-10T01:32:28.796 INFO:teuthology.orchestra.run.smithi078.stdout: "cap+": 49773, 2024-06-10T01:32:28.796 INFO:teuthology.orchestra.run.smithi078.stdout: "cap-": 49772, 2024-06-10T01:32:28.796 INFO:teuthology.orchestra.run.smithi078.stdout: "rss": 102384, 2024-06-10T01:32:28.797 INFO:teuthology.orchestra.run.smithi078.stdout: "heap": 223612 2024-06-10T01:32:28.797 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.797 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_server": { 2024-06-10T01:32:28.797 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_client_request": 414422, 2024-06-10T01:32:28.797 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_server_request": 5427, 2024-06-10T01:32:28.797 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_request": 286433, 2024-06-10T01:32:28.797 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_session": 4942, 2024-06-10T01:32:28.797 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_peer_request": 38499, 2024-06-10T01:32:28.797 INFO:teuthology.orchestra.run.smithi078.stdout: "req_create_latency": { 2024-06-10T01:32:28.797 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 44885, 2024-06-10T01:32:28.797 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 582.935322760, 2024-06-10T01:32:28.797 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.012987308 2024-06-10T01:32:28.797 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.797 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getattr_latency": { 2024-06-10T01:32:28.797 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 4924, 2024-06-10T01:32:28.798 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 140.854845059, 2024-06-10T01:32:28.798 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.028605776 2024-06-10T01:32:28.798 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.798 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getfilelock_latency": { 2024-06-10T01:32:28.798 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:28.798 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:28.798 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:28.798 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.798 INFO:teuthology.orchestra.run.smithi078.stdout: "req_link_latency": { 2024-06-10T01:32:28.798 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:28.798 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:28.798 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:28.799 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.799 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookup_latency": { 2024-06-10T01:32:28.799 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 89205, 2024-06-10T01:32:28.799 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 157.591311311, 2024-06-10T01:32:28.799 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.001766619 2024-06-10T01:32:28.799 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.799 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookuphash_latency": { 2024-06-10T01:32:28.799 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:28.799 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:28.799 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:28.799 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.799 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupino_latency": { 2024-06-10T01:32:28.799 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:28.799 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:28.800 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:28.800 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.800 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupname_latency": { 2024-06-10T01:32:28.800 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:28.800 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:28.800 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:28.800 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.800 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupparent_latency": { 2024-06-10T01:32:28.800 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:28.800 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:28.800 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:28.800 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.801 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupsnap_latency": { 2024-06-10T01:32:28.801 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:28.801 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:28.801 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:28.801 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.801 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lssnap_latency": { 2024-06-10T01:32:28.801 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:28.801 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:28.801 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:28.802 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.802 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mkdir_latency": { 2024-06-10T01:32:28.802 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:28.802 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:28.802 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:28.802 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.802 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mknod_latency": { 2024-06-10T01:32:28.802 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:28.802 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:28.802 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:28.802 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.802 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mksnap_latency": { 2024-06-10T01:32:28.802 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:28.803 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:28.803 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:28.803 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.803 INFO:teuthology.orchestra.run.smithi078.stdout: "req_open_latency": { 2024-06-10T01:32:28.803 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1, 2024-06-10T01:32:28.803 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000783851, 2024-06-10T01:32:28.803 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000783851 2024-06-10T01:32:28.803 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.803 INFO:teuthology.orchestra.run.smithi078.stdout: "req_readdir_latency": { 2024-06-10T01:32:28.803 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 74598, 2024-06-10T01:32:28.803 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 534.716942606, 2024-06-10T01:32:28.803 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.007167979 2024-06-10T01:32:28.803 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.803 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rename_latency": { 2024-06-10T01:32:28.803 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 10115, 2024-06-10T01:32:28.804 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 111.438344183, 2024-06-10T01:32:28.804 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.011017137 2024-06-10T01:32:28.804 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.804 INFO:teuthology.orchestra.run.smithi078.stdout: "req_renamesnap_latency": { 2024-06-10T01:32:28.804 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:28.804 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:28.804 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:28.804 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.804 INFO:teuthology.orchestra.run.smithi078.stdout: "req_snapdiff_latency": { 2024-06-10T01:32:28.804 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:28.804 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:28.804 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:28.804 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.804 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmdir_latency": { 2024-06-10T01:32:28.805 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 30, 2024-06-10T01:32:28.805 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.800204429, 2024-06-10T01:32:28.805 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.026673480 2024-06-10T01:32:28.805 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.805 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmsnap_latency": { 2024-06-10T01:32:28.805 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:28.805 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:28.805 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:28.805 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.805 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmxattr_latency": { 2024-06-10T01:32:28.805 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:28.805 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:28.805 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:28.805 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.806 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setattr_latency": { 2024-06-10T01:32:28.806 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 94, 2024-06-10T01:32:28.806 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 27.112311778, 2024-06-10T01:32:28.806 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.288428848 2024-06-10T01:32:28.806 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.806 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setdirlayout_latency": { 2024-06-10T01:32:28.806 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:28.806 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:28.806 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:28.806 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.806 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setfilelock_latency": { 2024-06-10T01:32:28.806 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1676, 2024-06-10T01:32:28.806 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 2.419550117, 2024-06-10T01:32:28.806 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.001443645 2024-06-10T01:32:28.807 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.807 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setlayout_latency": { 2024-06-10T01:32:28.807 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:28.807 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:28.807 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:28.807 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.807 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setxattr_latency": { 2024-06-10T01:32:28.807 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:28.807 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:28.807 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:28.807 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.807 INFO:teuthology.orchestra.run.smithi078.stdout: "req_symlink_latency": { 2024-06-10T01:32:28.807 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:28.807 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:28.808 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:28.808 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.808 INFO:teuthology.orchestra.run.smithi078.stdout: "req_unlink_latency": { 2024-06-10T01:32:28.808 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 45109, 2024-06-10T01:32:28.808 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 233.229510918, 2024-06-10T01:32:28.808 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.005170354 2024-06-10T01:32:28.808 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.808 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_revoke_eviction": 0, 2024-06-10T01:32:28.808 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_acquisition_throttle": 0, 2024-06-10T01:32:28.808 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getvxattr_latency": { 2024-06-10T01:32:28.808 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:28.808 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:28.808 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:28.809 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-06-10T01:32:28.809 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.809 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_sessions": { 2024-06-10T01:32:28.809 INFO:teuthology.orchestra.run.smithi078.stdout: "session_count": 1, 2024-06-10T01:32:28.809 INFO:teuthology.orchestra.run.smithi078.stdout: "session_add": 1, 2024-06-10T01:32:28.809 INFO:teuthology.orchestra.run.smithi078.stdout: "session_remove": 0, 2024-06-10T01:32:28.809 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_open": 1, 2024-06-10T01:32:28.809 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_stale": 0, 2024-06-10T01:32:28.809 INFO:teuthology.orchestra.run.smithi078.stdout: "total_load": 22866, 2024-06-10T01:32:28.809 INFO:teuthology.orchestra.run.smithi078.stdout: "average_load": 22866, 2024-06-10T01:32:28.809 INFO:teuthology.orchestra.run.smithi078.stdout: "avg_session_uptime": 1605, 2024-06-10T01:32:28.809 INFO:teuthology.orchestra.run.smithi078.stdout: "mdthresh_evicted": 0 2024-06-10T01:32:28.809 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.810 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-06-10T01:32:28.810 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-06-10T01:32:28.810 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-06-10T01:32:28.810 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T01:32:28.810 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-06-10T01:32:28.810 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T01:32:28.810 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-06-10T01:32:28.810 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T01:32:28.810 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T01:32:28.810 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T01:32:28.810 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T01:32:28.810 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T01:32:28.810 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-06-10T01:32:28.811 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T01:32:28.811 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T01:32:28.811 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-06-10T01:32:28.811 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-06-10T01:32:28.811 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-06-10T01:32:28.811 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-06-10T01:32:28.811 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T01:32:28.811 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T01:32:28.811 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T01:32:28.811 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T01:32:28.811 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T01:32:28.811 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-06-10T01:32:28.811 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-06-10T01:32:28.811 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-06-10T01:32:28.811 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T01:32:28.812 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T01:32:28.812 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-06-10T01:32:28.812 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-06-10T01:32:28.812 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-06-10T01:32:28.812 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-06-10T01:32:28.812 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T01:32:28.812 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-06-10T01:32:28.812 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T01:32:28.812 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-06-10T01:32:28.812 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 296524, 2024-06-10T01:32:28.812 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 104, 2024-06-10T01:32:28.812 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 176, 2024-06-10T01:32:28.812 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 2, 2024-06-10T01:32:28.812 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-06-10T01:32:28.813 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-06-10T01:32:28.813 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-06-10T01:32:28.813 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-06-10T01:32:28.813 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-06-10T01:32:28.813 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-06-10T01:32:28.813 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 8800, 2024-06-10T01:32:28.813 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 169, 2024-06-10T01:32:28.813 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T01:32:28.813 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-06-10T01:32:28.813 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-06-10T01:32:28.813 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-06-10T01:32:28.813 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 1360838, 2024-06-10T01:32:28.813 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 18362, 2024-06-10T01:32:28.813 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-06-10T01:32:28.813 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-06-10T01:32:28.814 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-06-10T01:32:28.814 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-06-10T01:32:28.814 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.814 INFO:teuthology.orchestra.run.smithi078.stdout: "objecter": { 2024-06-10T01:32:28.814 INFO:teuthology.orchestra.run.smithi078.stdout: "op_active": 0, 2024-06-10T01:32:28.814 INFO:teuthology.orchestra.run.smithi078.stdout: "op_laggy": 0, 2024-06-10T01:32:28.814 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send": 161107, 2024-06-10T01:32:28.814 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send_bytes": 432416584, 2024-06-10T01:32:28.814 INFO:teuthology.orchestra.run.smithi078.stdout: "op_resend": 33, 2024-06-10T01:32:28.814 INFO:teuthology.orchestra.run.smithi078.stdout: "op_reply": 161107, 2024-06-10T01:32:28.814 INFO:teuthology.orchestra.run.smithi078.stdout: "op_latency": { 2024-06-10T01:32:28.814 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 161107, 2024-06-10T01:32:28.814 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 2100.577463585, 2024-06-10T01:32:28.814 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.013038399 2024-06-10T01:32:28.815 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.815 INFO:teuthology.orchestra.run.smithi078.stdout: "op_inflight": 0, 2024-06-10T01:32:28.815 INFO:teuthology.orchestra.run.smithi078.stdout: "oplen_avg": { 2024-06-10T01:32:28.815 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 161107, 2024-06-10T01:32:28.815 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 163031 2024-06-10T01:32:28.815 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.815 INFO:teuthology.orchestra.run.smithi078.stdout: "op": 161107, 2024-06-10T01:32:28.815 INFO:teuthology.orchestra.run.smithi078.stdout: "op_r": 4576, 2024-06-10T01:32:28.815 INFO:teuthology.orchestra.run.smithi078.stdout: "op_w": 156531, 2024-06-10T01:32:28.815 INFO:teuthology.orchestra.run.smithi078.stdout: "op_rmw": 0, 2024-06-10T01:32:28.815 INFO:teuthology.orchestra.run.smithi078.stdout: "op_pg": 0, 2024-06-10T01:32:28.815 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_stat": 0, 2024-06-10T01:32:28.815 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_create": 21, 2024-06-10T01:32:28.815 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_read": 1359, 2024-06-10T01:32:28.816 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_write": 64057, 2024-06-10T01:32:28.816 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writefull": 733, 2024-06-10T01:32:28.816 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writesame": 0, 2024-06-10T01:32:28.816 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_append": 0, 2024-06-10T01:32:28.816 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_zero": 0, 2024-06-10T01:32:28.816 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_truncate": 14, 2024-06-10T01:32:28.816 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_delete": 90492, 2024-06-10T01:32:28.816 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_mapext": 0, 2024-06-10T01:32:28.816 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_sparse_read": 0, 2024-06-10T01:32:28.816 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_clonerange": 0, 2024-06-10T01:32:28.816 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_getxattr": 3217, 2024-06-10T01:32:28.816 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_setxattr": 42, 2024-06-10T01:32:28.816 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_cmpxattr": 0, 2024-06-10T01:32:28.817 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_rmxattr": 0, 2024-06-10T01:32:28.817 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_resetxattrs": 0, 2024-06-10T01:32:28.817 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_call": 0, 2024-06-10T01:32:28.817 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_watch": 0, 2024-06-10T01:32:28.817 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_notify": 0, 2024-06-10T01:32:28.817 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T01:32:28.817 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls": 0, 2024-06-10T01:32:28.817 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls_filter": 0, 2024-06-10T01:32:28.817 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_other": 0, 2024-06-10T01:32:28.817 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_active": 0, 2024-06-10T01:32:28.817 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_send": 0, 2024-06-10T01:32:28.817 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_resend": 0, 2024-06-10T01:32:28.817 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_ping": 0, 2024-06-10T01:32:28.817 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_active": 0, 2024-06-10T01:32:28.817 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_send": 0, 2024-06-10T01:32:28.818 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_resend": 0, 2024-06-10T01:32:28.818 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_active": 0, 2024-06-10T01:32:28.818 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_send": 0, 2024-06-10T01:32:28.818 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_resend": 0, 2024-06-10T01:32:28.818 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_active": 0, 2024-06-10T01:32:28.818 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_send": 0, 2024-06-10T01:32:28.818 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_resend": 0, 2024-06-10T01:32:28.818 INFO:teuthology.orchestra.run.smithi078.stdout: "command_active": 0, 2024-06-10T01:32:28.818 INFO:teuthology.orchestra.run.smithi078.stdout: "command_send": 0, 2024-06-10T01:32:28.818 INFO:teuthology.orchestra.run.smithi078.stdout: "command_resend": 0, 2024-06-10T01:32:28.818 INFO:teuthology.orchestra.run.smithi078.stdout: "map_epoch": 102, 2024-06-10T01:32:28.818 INFO:teuthology.orchestra.run.smithi078.stdout: "map_full": 0, 2024-06-10T01:32:28.818 INFO:teuthology.orchestra.run.smithi078.stdout: "map_inc": 5, 2024-06-10T01:32:28.818 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_sessions": 12, 2024-06-10T01:32:28.819 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_open": 12, 2024-06-10T01:32:28.819 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_close": 0, 2024-06-10T01:32:28.819 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_laggy": 0, 2024-06-10T01:32:28.819 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_wr": 2317, 2024-06-10T01:32:28.819 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_rd": 2, 2024-06-10T01:32:28.819 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_del": 777 2024-06-10T01:32:28.819 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.819 INFO:teuthology.orchestra.run.smithi078.stdout: "oft": { 2024-06-10T01:32:28.819 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_objs": 1, 2024-06-10T01:32:28.819 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_kv_pairs": 0, 2024-06-10T01:32:28.819 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_updates": 0, 2024-06-10T01:32:28.819 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_removes": 0 2024-06-10T01:32:28.819 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.819 INFO:teuthology.orchestra.run.smithi078.stdout: "osd-slow-ops": { 2024-06-10T01:32:28.820 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_ops_count": 2961 2024-06-10T01:32:28.820 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.820 INFO:teuthology.orchestra.run.smithi078.stdout: "purge_queue": { 2024-06-10T01:32:28.820 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops": 0, 2024-06-10T01:32:28.820 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops_high_water": 17, 2024-06-10T01:32:28.820 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing": 0, 2024-06-10T01:32:28.820 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_high_water": 14, 2024-06-10T01:32:28.820 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed_ops": 90247, 2024-06-10T01:32:28.820 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed": 45138, 2024-06-10T01:32:28.820 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_item_in_journal": 0 2024-06-10T01:32:28.820 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.820 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T01:32:28.820 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-06-10T01:32:28.820 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-06-10T01:32:28.820 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-06-10T01:32:28.821 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 831024, 2024-06-10T01:32:28.821 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 168627133, 2024-06-10T01:32:28.821 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:28.821 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 831024, 2024-06-10T01:32:28.821 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-06-10T01:32:28.821 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-06-10T01:32:28.821 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 831024, 2024-06-10T01:32:28.821 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 168627133, 2024-06-10T01:32:28.821 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-06-10T01:32:28.821 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:28.821 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:28.821 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:28.821 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-06-10T01:32:28.821 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.822 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_bytes": { 2024-06-10T01:32:28.822 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-06-10T01:32:28.822 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-06-10T01:32:28.822 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-06-10T01:32:28.822 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-06-10T01:32:28.822 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-06-10T01:32:28.822 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:28.822 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-06-10T01:32:28.822 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 161107, 2024-06-10T01:32:28.822 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 437336378, 2024-06-10T01:32:28.822 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 161107, 2024-06-10T01:32:28.822 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 437336378, 2024-06-10T01:32:28.823 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-06-10T01:32:28.823 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:28.823 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:28.823 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:28.823 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-06-10T01:32:28.823 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.823 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_ops": { 2024-06-10T01:32:28.823 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-06-10T01:32:28.823 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 1024, 2024-06-10T01:32:28.823 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-06-10T01:32:28.823 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-06-10T01:32:28.823 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-06-10T01:32:28.823 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:28.823 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-06-10T01:32:28.824 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 161107, 2024-06-10T01:32:28.824 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 161107, 2024-06-10T01:32:28.824 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 161107, 2024-06-10T01:32:28.824 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 161107, 2024-06-10T01:32:28.824 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-06-10T01:32:28.824 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:28.824 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:28.824 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:28.824 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-06-10T01:32:28.824 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.824 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle": { 2024-06-10T01:32:28.824 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-06-10T01:32:28.824 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-06-10T01:32:28.825 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-06-10T01:32:28.825 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 45138, 2024-06-10T01:32:28.825 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 4919810, 2024-06-10T01:32:28.825 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:28.825 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 45138, 2024-06-10T01:32:28.825 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-06-10T01:32:28.825 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-06-10T01:32:28.825 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 1362, 2024-06-10T01:32:28.825 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 4919810, 2024-06-10T01:32:28.825 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-06-10T01:32:28.825 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:28.825 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:28.825 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:28.826 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-06-10T01:32:28.826 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:28.826 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle-0x565131336460": { 2024-06-10T01:32:28.826 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-06-10T01:32:28.826 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-06-10T01:32:28.826 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-06-10T01:32:28.826 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 227780, 2024-06-10T01:32:28.826 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 421640370, 2024-06-10T01:32:28.826 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:28.826 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 227780, 2024-06-10T01:32:28.826 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-06-10T01:32:28.826 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-06-10T01:32:28.826 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 62695, 2024-06-10T01:32:28.827 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 421640370, 2024-06-10T01:32:28.827 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-06-10T01:32:28.827 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:28.827 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:28.827 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:28.827 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-06-10T01:32:28.827 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-06-10T01:32:28.827 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-06-10T01:32:29.405 INFO:tasks.check_counter:Daemon mds.b mds.exported=17 2024-06-10T01:32:29.405 INFO:tasks.check_counter:Daemon mds.b mds.imported=15 2024-06-10T01:32:29.405 DEBUG:tasks.check_counter:Getting stats from e 2024-06-10T01:32:29.406 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph daemon mds.e perf dump 2024-06-10T01:32:29.414 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:29 smithi078 ceph-mon[36683]: pgmap v1228: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:32:29.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:29 smithi175 ceph-mon[36636]: pgmap v1228: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:32:29.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:29 smithi007 ceph-mon[32398]: pgmap v1228: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:32:29.668 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T01:32:31.313 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:31 smithi078 ceph-mon[36683]: pgmap v1229: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:32:31.441 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-06-10T01:32:31.441 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T01:32:31.441 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 505, 2024-06-10T01:32:31.441 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 500, 2024-06-10T01:32:31.442 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 562128, 2024-06-10T01:32:31.442 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 309040, 2024-06-10T01:32:31.442 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 4, 2024-06-10T01:32:31.442 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 0, 2024-06-10T01:32:31.442 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.063337495, 2024-06-10T01:32:31.442 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.025360434, 2024-06-10T01:32:31.442 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.022248640, 2024-06-10T01:32:31.442 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.010970617, 2024-06-10T01:32:31.442 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:31.442 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 500, 2024-06-10T01:32:31.442 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.024310094, 2024-06-10T01:32:31.442 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000048620 2024-06-10T01:32:31.442 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:31.443 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:31.443 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:31.443 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:31.443 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:31.443 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:31.443 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 562128, 2024-06-10T01:32:31.443 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 309040 2024-06-10T01:32:31.443 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:31.443 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T01:32:31.443 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 3, 2024-06-10T01:32:31.443 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 382, 2024-06-10T01:32:31.443 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 480, 2024-06-10T01:32:31.443 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 301216, 2024-06-10T01:32:31.443 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 5, 2024-06-10T01:32:31.444 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 2, 2024-06-10T01:32:31.444 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.019072898, 2024-06-10T01:32:31.444 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.014095061, 2024-06-10T01:32:31.444 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.001428855, 2024-06-10T01:32:31.444 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T01:32:31.444 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:31.444 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 380, 2024-06-10T01:32:31.444 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.021167203, 2024-06-10T01:32:31.444 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000055703 2024-06-10T01:32:31.444 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:31.444 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:31.444 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:31.444 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:31.445 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:31.445 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:31.445 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 480, 2024-06-10T01:32:31.445 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 301216 2024-06-10T01:32:31.445 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:31.445 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T01:32:31.445 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 0, 2024-06-10T01:32:31.445 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 2, 2024-06-10T01:32:31.445 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 0, 2024-06-10T01:32:31.445 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 0, 2024-06-10T01:32:31.445 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 2, 2024-06-10T01:32:31.445 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 1, 2024-06-10T01:32:31.445 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.000323544, 2024-06-10T01:32:31.445 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.000000000, 2024-06-10T01:32:31.445 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.000246494, 2024-06-10T01:32:31.446 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T01:32:31.446 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:31.446 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:31.446 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:31.446 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:31.446 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:31.446 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:31.446 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:31.446 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:31.446 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:31.446 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:31.446 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-10T01:32:31.446 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 0 2024-06-10T01:32:31.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:31.447 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-06-10T01:32:31.447 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 0, 2024-06-10T01:32:31.447 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-06-10T01:32:31.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:31.447 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-06-10T01:32:31.447 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-06-10T01:32:31.447 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-06-10T01:32:31.447 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T01:32:31.447 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-06-10T01:32:31.447 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T01:32:31.447 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-06-10T01:32:31.447 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T01:32:31.447 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T01:32:31.447 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T01:32:31.448 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T01:32:31.448 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T01:32:31.448 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-06-10T01:32:31.448 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T01:32:31.448 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T01:32:31.448 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-06-10T01:32:31.448 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-06-10T01:32:31.448 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-06-10T01:32:31.448 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-06-10T01:32:31.448 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T01:32:31.448 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T01:32:31.448 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T01:32:31.448 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T01:32:31.449 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T01:32:31.449 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-06-10T01:32:31.449 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-06-10T01:32:31.449 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-06-10T01:32:31.449 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T01:32:31.449 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T01:32:31.449 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-06-10T01:32:31.449 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-06-10T01:32:31.449 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-06-10T01:32:31.449 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-06-10T01:32:31.449 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T01:32:31.449 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-06-10T01:32:31.449 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T01:32:31.449 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-06-10T01:32:31.450 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 148000, 2024-06-10T01:32:31.450 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 29, 2024-06-10T01:32:31.450 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2024-06-10T01:32:31.450 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2024-06-10T01:32:31.450 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-06-10T01:32:31.450 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-06-10T01:32:31.450 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-06-10T01:32:31.450 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-06-10T01:32:31.450 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-06-10T01:32:31.450 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-06-10T01:32:31.450 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 0, 2024-06-10T01:32:31.450 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 0, 2024-06-10T01:32:31.451 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T01:32:31.451 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-06-10T01:32:31.451 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-06-10T01:32:31.451 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-06-10T01:32:31.451 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 928, 2024-06-10T01:32:31.451 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 2, 2024-06-10T01:32:31.451 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-06-10T01:32:31.451 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-06-10T01:32:31.451 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-06-10T01:32:31.451 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-06-10T01:32:31.451 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:31.451 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T01:32:31.452 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-06-10T01:32:31.452 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-06-10T01:32:31.452 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-06-10T01:32:31.452 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 504, 2024-06-10T01:32:31.452 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 488283, 2024-06-10T01:32:31.452 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:31.452 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 504, 2024-06-10T01:32:31.452 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-06-10T01:32:31.452 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-06-10T01:32:31.452 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 504, 2024-06-10T01:32:31.452 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 488283, 2024-06-10T01:32:31.452 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-06-10T01:32:31.452 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:31.452 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:31.453 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:31.453 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-06-10T01:32:31.453 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-06-10T01:32:31.453 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-06-10T01:32:31.453 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:31 smithi175 ceph-mon[36636]: pgmap v1229: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:32:31.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:31 smithi007 ceph-mon[32398]: pgmap v1229: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:32:32.170 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-06-10T01:32:32.170 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-06-10T01:32:32.170 DEBUG:tasks.check_counter:Getting stats from h 2024-06-10T01:32:32.171 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph daemon mds.h perf dump 2024-06-10T01:32:32.450 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T01:32:33.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:33 smithi175 ceph-mon[36636]: pgmap v1230: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-10T01:32:33.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:33 smithi007 ceph-mon[32398]: pgmap v1230: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-10T01:32:33.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:33 smithi078 ceph-mon[36683]: pgmap v1230: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-10T01:32:34.131 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-06-10T01:32:34.131 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T01:32:34.131 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 3909, 2024-06-10T01:32:34.131 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 3909, 2024-06-10T01:32:34.131 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 148713052, 2024-06-10T01:32:34.131 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 1242245, 2024-06-10T01:32:34.131 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 8, 2024-06-10T01:32:34.131 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 4, 2024-06-10T01:32:34.132 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.387493772, 2024-06-10T01:32:34.132 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.095587225, 2024-06-10T01:32:34.132 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.166564537, 2024-06-10T01:32:34.132 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.100008768, 2024-06-10T01:32:34.132 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:34.132 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 3908, 2024-06-10T01:32:34.132 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.137809550, 2024-06-10T01:32:34.132 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000035263 2024-06-10T01:32:34.132 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.132 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:34.132 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.132 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.132 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.133 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.133 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-06-10T01:32:34.133 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 448 2024-06-10T01:32:34.133 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.133 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T01:32:34.133 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 1764, 2024-06-10T01:32:34.133 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 1725, 2024-06-10T01:32:34.133 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 101708074, 2024-06-10T01:32:34.133 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 696744, 2024-06-10T01:32:34.133 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 6, 2024-06-10T01:32:34.134 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 5, 2024-06-10T01:32:34.134 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.203686233, 2024-06-10T01:32:34.134 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.050832823, 2024-06-10T01:32:34.134 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.101299681, 2024-06-10T01:32:34.134 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.041201816, 2024-06-10T01:32:34.134 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:34.134 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1724, 2024-06-10T01:32:34.134 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.064702896, 2024-06-10T01:32:34.134 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000037530 2024-06-10T01:32:34.134 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.134 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:34.134 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.135 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.135 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.135 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 730080, 2024-06-10T01:32:34.135 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 309040 2024-06-10T01:32:34.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.135 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T01:32:34.135 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 1172, 2024-06-10T01:32:34.135 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 1549, 2024-06-10T01:32:34.135 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 134522218, 2024-06-10T01:32:34.135 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 1186876, 2024-06-10T01:32:34.135 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 8, 2024-06-10T01:32:34.135 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 5, 2024-06-10T01:32:34.135 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.178095361, 2024-06-10T01:32:34.136 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.035292497, 2024-06-10T01:32:34.136 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.103920343, 2024-06-10T01:32:34.136 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.029022095, 2024-06-10T01:32:34.136 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:34.136 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1547, 2024-06-10T01:32:34.136 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.054551454, 2024-06-10T01:32:34.136 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000035262 2024-06-10T01:32:34.136 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.136 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:34.136 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.136 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.136 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.136 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.136 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-10T01:32:34.137 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 814816 2024-06-10T01:32:34.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.137 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-06-10T01:32:34.137 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 1, 2024-06-10T01:32:34.137 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-06-10T01:32:34.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.137 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-MDSRank": { 2024-06-10T01:32:34.137 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-06-10T01:32:34.137 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-06-10T01:32:34.137 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 10722, 2024-06-10T01:32:34.137 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.254582263, 2024-06-10T01:32:34.137 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000023743 2024-06-10T01:32:34.137 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-06-10T01:32:34.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.138 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-PurgeQueue": { 2024-06-10T01:32:34.138 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-06-10T01:32:34.138 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-06-10T01:32:34.138 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.138 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.138 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.138 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-06-10T01:32:34.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.138 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": { 2024-06-10T01:32:34.138 INFO:teuthology.orchestra.run.smithi078.stdout: "request": 0, 2024-06-10T01:32:34.138 INFO:teuthology.orchestra.run.smithi078.stdout: "reply": 0, 2024-06-10T01:32:34.138 INFO:teuthology.orchestra.run.smithi078.stdout: "reply_latency": { 2024-06-10T01:32:34.139 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.139 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.139 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.139 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_reply": 0, 2024-06-10T01:32:34.139 INFO:teuthology.orchestra.run.smithi078.stdout: "forward": 0, 2024-06-10T01:32:34.139 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_complete": 0, 2024-06-10T01:32:34.139 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_keys": 0, 2024-06-10T01:32:34.139 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_commit": 0, 2024-06-10T01:32:34.139 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_split": 0, 2024-06-10T01:32:34.139 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_merge": 0, 2024-06-10T01:32:34.139 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes": 38211, 2024-06-10T01:32:34.140 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_top": 9819, 2024-06-10T01:32:34.140 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_bottom": 28392, 2024-06-10T01:32:34.140 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pin_tail": 0, 2024-06-10T01:32:34.140 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pinned": 24183, 2024-06-10T01:32:34.140 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_expired": 0, 2024-06-10T01:32:34.140 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_with_caps": 0, 2024-06-10T01:32:34.140 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": 0, 2024-06-10T01:32:34.140 INFO:teuthology.orchestra.run.smithi078.stdout: "subtrees": 1, 2024-06-10T01:32:34.140 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse": 0, 2024-06-10T01:32:34.140 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_hit": 0, 2024-06-10T01:32:34.140 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_forward": 0, 2024-06-10T01:32:34.140 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_discover": 0, 2024-06-10T01:32:34.140 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_dir_fetch": 0, 2024-06-10T01:32:34.140 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_remote_ino": 0, 2024-06-10T01:32:34.140 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_lock": 0, 2024-06-10T01:32:34.140 INFO:teuthology.orchestra.run.smithi078.stdout: "load_cent": 0, 2024-06-10T01:32:34.141 INFO:teuthology.orchestra.run.smithi078.stdout: "q": 0, 2024-06-10T01:32:34.141 INFO:teuthology.orchestra.run.smithi078.stdout: "exported": 0, 2024-06-10T01:32:34.141 INFO:teuthology.orchestra.run.smithi078.stdout: "exported_inodes": 0, 2024-06-10T01:32:34.141 INFO:teuthology.orchestra.run.smithi078.stdout: "imported": 0, 2024-06-10T01:32:34.141 INFO:teuthology.orchestra.run.smithi078.stdout: "imported_inodes": 0, 2024-06-10T01:32:34.141 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_dir_fetch": 0, 2024-06-10T01:32:34.141 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_backtrace_fetch": 0, 2024-06-10T01:32:34.141 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_peer_discover": 0, 2024-06-10T01:32:34.141 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rfiles": 0, 2024-06-10T01:32:34.141 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rbytes": 0, 2024-06-10T01:32:34.141 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rsnaps": 0, 2024-06-10T01:32:34.141 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_fetch": 0, 2024-06-10T01:32:34.141 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_set_tag": 0, 2024-06-10T01:32:34.141 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T01:32:34.141 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_inotable_repaired": 0, 2024-06-10T01:32:34.142 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_inodes": 0, 2024-06-10T01:32:34.142 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_base_inodes": 0, 2024-06-10T01:32:34.142 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dirfrag_rstats": 0, 2024-06-10T01:32:34.142 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_file_inodes": 0, 2024-06-10T01:32:34.142 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_inode_file_caps": 0, 2024-06-10T01:32:34.142 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_revoke": 0, 2024-06-10T01:32:34.142 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_grant": 0, 2024-06-10T01:32:34.142 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_trunc": 0, 2024-06-10T01:32:34.142 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T01:32:34.142 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T01:32:34.142 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps": 0, 2024-06-10T01:32:34.142 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps_dirty": 0, 2024-06-10T01:32:34.142 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_cap_release": 0, 2024-06-10T01:32:34.142 INFO:teuthology.orchestra.run.smithi078.stdout: "process_request_cap_release": 0 2024-06-10T01:32:34.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.143 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_cache": { 2024-06-10T01:32:34.143 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update": 0, 2024-06-10T01:32:34.143 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update_receipt": 0, 2024-06-10T01:32:34.143 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_try_discover": 0, 2024-06-10T01:32:34.143 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_send_discover": 0, 2024-06-10T01:32:34.143 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_handle_discover": 0, 2024-06-10T01:32:34.143 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays": 0, 2024-06-10T01:32:34.143 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_delayed": 0, 2024-06-10T01:32:34.143 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_enqueuing": 0, 2024-06-10T01:32:34.143 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_created": 0, 2024-06-10T01:32:34.143 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_enqueued": 0, 2024-06-10T01:32:34.143 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_reintegrated": 0, 2024-06-10T01:32:34.143 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_migrated": 0, 2024-06-10T01:32:34.143 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_processing": 0, 2024-06-10T01:32:34.144 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_enqueued": 0, 2024-06-10T01:32:34.144 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_prioritized": 0, 2024-06-10T01:32:34.144 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_started": 0, 2024-06-10T01:32:34.144 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_completed": 0, 2024-06-10T01:32:34.144 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_path": 0, 2024-06-10T01:32:34.144 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_inode": 0, 2024-06-10T01:32:34.144 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_enqueue_scrub": 0, 2024-06-10T01:32:34.144 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_exportdir": 0, 2024-06-10T01:32:34.144 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_flush": 0, 2024-06-10T01:32:34.144 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragmentdir": 0, 2024-06-10T01:32:34.144 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragstats": 0, 2024-06-10T01:32:34.144 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_inodestats": 0 2024-06-10T01:32:34.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.144 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_log": { 2024-06-10T01:32:34.144 INFO:teuthology.orchestra.run.smithi078.stdout: "evlrg": 0, 2024-06-10T01:32:34.145 INFO:teuthology.orchestra.run.smithi078.stdout: "evadd": 0, 2024-06-10T01:32:34.145 INFO:teuthology.orchestra.run.smithi078.stdout: "evex": 0, 2024-06-10T01:32:34.145 INFO:teuthology.orchestra.run.smithi078.stdout: "evtrm": 0, 2024-06-10T01:32:34.145 INFO:teuthology.orchestra.run.smithi078.stdout: "ev": 202911, 2024-06-10T01:32:34.145 INFO:teuthology.orchestra.run.smithi078.stdout: "evexg": 0, 2024-06-10T01:32:34.145 INFO:teuthology.orchestra.run.smithi078.stdout: "evexd": 0, 2024-06-10T01:32:34.145 INFO:teuthology.orchestra.run.smithi078.stdout: "segadd": 0, 2024-06-10T01:32:34.145 INFO:teuthology.orchestra.run.smithi078.stdout: "segex": 0, 2024-06-10T01:32:34.145 INFO:teuthology.orchestra.run.smithi078.stdout: "segtrm": 0, 2024-06-10T01:32:34.145 INFO:teuthology.orchestra.run.smithi078.stdout: "seg": 142, 2024-06-10T01:32:34.145 INFO:teuthology.orchestra.run.smithi078.stdout: "segmjr": 18, 2024-06-10T01:32:34.145 INFO:teuthology.orchestra.run.smithi078.stdout: "segexg": 0, 2024-06-10T01:32:34.145 INFO:teuthology.orchestra.run.smithi078.stdout: "segexd": 0, 2024-06-10T01:32:34.146 INFO:teuthology.orchestra.run.smithi078.stdout: "expos": 150764945, 2024-06-10T01:32:34.146 INFO:teuthology.orchestra.run.smithi078.stdout: "wrpos": 386642851, 2024-06-10T01:32:34.146 INFO:teuthology.orchestra.run.smithi078.stdout: "rdpos": 386642252, 2024-06-10T01:32:34.146 INFO:teuthology.orchestra.run.smithi078.stdout: "jlat": { 2024-06-10T01:32:34.146 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.146 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.146 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.146 INFO:teuthology.orchestra.run.smithi078.stdout: "replayed": 202911 2024-06-10T01:32:34.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.146 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_mem": { 2024-06-10T01:32:34.146 INFO:teuthology.orchestra.run.smithi078.stdout: "ino": 16, 2024-06-10T01:32:34.147 INFO:teuthology.orchestra.run.smithi078.stdout: "ino+": 38234, 2024-06-10T01:32:34.147 INFO:teuthology.orchestra.run.smithi078.stdout: "ino-": 38218, 2024-06-10T01:32:34.147 INFO:teuthology.orchestra.run.smithi078.stdout: "dir": 28, 2024-06-10T01:32:34.147 INFO:teuthology.orchestra.run.smithi078.stdout: "dir+": 184, 2024-06-10T01:32:34.147 INFO:teuthology.orchestra.run.smithi078.stdout: "dir-": 156, 2024-06-10T01:32:34.147 INFO:teuthology.orchestra.run.smithi078.stdout: "dn": 38211, 2024-06-10T01:32:34.147 INFO:teuthology.orchestra.run.smithi078.stdout: "dn+": 38776, 2024-06-10T01:32:34.147 INFO:teuthology.orchestra.run.smithi078.stdout: "dn-": 565, 2024-06-10T01:32:34.147 INFO:teuthology.orchestra.run.smithi078.stdout: "cap": 0, 2024-06-10T01:32:34.147 INFO:teuthology.orchestra.run.smithi078.stdout: "cap+": 0, 2024-06-10T01:32:34.147 INFO:teuthology.orchestra.run.smithi078.stdout: "cap-": 0, 2024-06-10T01:32:34.147 INFO:teuthology.orchestra.run.smithi078.stdout: "rss": 90296, 2024-06-10T01:32:34.147 INFO:teuthology.orchestra.run.smithi078.stdout: "heap": 215420 2024-06-10T01:32:34.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.147 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_server": { 2024-06-10T01:32:34.148 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_client_request": 0, 2024-06-10T01:32:34.148 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_server_request": 0, 2024-06-10T01:32:34.148 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_request": 0, 2024-06-10T01:32:34.148 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_session": 0, 2024-06-10T01:32:34.148 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_peer_request": 0, 2024-06-10T01:32:34.148 INFO:teuthology.orchestra.run.smithi078.stdout: "req_create_latency": { 2024-06-10T01:32:34.148 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.148 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.148 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.148 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getattr_latency": { 2024-06-10T01:32:34.148 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.148 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.149 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.149 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getfilelock_latency": { 2024-06-10T01:32:34.149 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.149 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.149 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.149 INFO:teuthology.orchestra.run.smithi078.stdout: "req_link_latency": { 2024-06-10T01:32:34.149 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.149 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.149 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.149 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookup_latency": { 2024-06-10T01:32:34.149 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.150 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.150 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.150 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookuphash_latency": { 2024-06-10T01:32:34.150 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.150 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.150 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.150 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupino_latency": { 2024-06-10T01:32:34.150 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.150 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.150 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.151 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupname_latency": { 2024-06-10T01:32:34.151 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.151 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.151 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.151 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupparent_latency": { 2024-06-10T01:32:34.151 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.151 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.151 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.151 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupsnap_latency": { 2024-06-10T01:32:34.151 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.151 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.152 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.152 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lssnap_latency": { 2024-06-10T01:32:34.152 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.152 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.152 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.152 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mkdir_latency": { 2024-06-10T01:32:34.153 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.153 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.153 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.153 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mknod_latency": { 2024-06-10T01:32:34.153 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.153 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.153 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.153 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mksnap_latency": { 2024-06-10T01:32:34.153 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.153 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.153 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.153 INFO:teuthology.orchestra.run.smithi078.stdout: "req_open_latency": { 2024-06-10T01:32:34.154 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.154 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.154 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.154 INFO:teuthology.orchestra.run.smithi078.stdout: "req_readdir_latency": { 2024-06-10T01:32:34.154 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.154 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.154 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.154 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rename_latency": { 2024-06-10T01:32:34.154 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.154 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.155 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.155 INFO:teuthology.orchestra.run.smithi078.stdout: "req_renamesnap_latency": { 2024-06-10T01:32:34.155 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.155 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.155 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.155 INFO:teuthology.orchestra.run.smithi078.stdout: "req_snapdiff_latency": { 2024-06-10T01:32:34.155 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.155 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.155 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.155 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmdir_latency": { 2024-06-10T01:32:34.155 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.155 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.156 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.156 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.156 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmsnap_latency": { 2024-06-10T01:32:34.156 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.156 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.156 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.156 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.156 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmxattr_latency": { 2024-06-10T01:32:34.156 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.156 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.156 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.156 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.156 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setattr_latency": { 2024-06-10T01:32:34.156 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.156 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.157 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.157 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.157 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setdirlayout_latency": { 2024-06-10T01:32:34.157 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.157 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.157 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.157 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.157 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setfilelock_latency": { 2024-06-10T01:32:34.157 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.157 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.157 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.157 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.157 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setlayout_latency": { 2024-06-10T01:32:34.158 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.158 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.158 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.158 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setxattr_latency": { 2024-06-10T01:32:34.158 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.158 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.158 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.158 INFO:teuthology.orchestra.run.smithi078.stdout: "req_symlink_latency": { 2024-06-10T01:32:34.158 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.158 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.158 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.159 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.159 INFO:teuthology.orchestra.run.smithi078.stdout: "req_unlink_latency": { 2024-06-10T01:32:34.159 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.159 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.159 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.159 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.159 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_revoke_eviction": 0, 2024-06-10T01:32:34.159 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_acquisition_throttle": 0, 2024-06-10T01:32:34.159 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getvxattr_latency": { 2024-06-10T01:32:34.159 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.159 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.159 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.159 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-06-10T01:32:34.159 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.159 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_sessions": { 2024-06-10T01:32:34.160 INFO:teuthology.orchestra.run.smithi078.stdout: "session_count": 1, 2024-06-10T01:32:34.160 INFO:teuthology.orchestra.run.smithi078.stdout: "session_add": 1, 2024-06-10T01:32:34.160 INFO:teuthology.orchestra.run.smithi078.stdout: "session_remove": 0, 2024-06-10T01:32:34.160 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_open": 1, 2024-06-10T01:32:34.160 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_stale": 0, 2024-06-10T01:32:34.160 INFO:teuthology.orchestra.run.smithi078.stdout: "total_load": 0, 2024-06-10T01:32:34.160 INFO:teuthology.orchestra.run.smithi078.stdout: "average_load": 0, 2024-06-10T01:32:34.160 INFO:teuthology.orchestra.run.smithi078.stdout: "avg_session_uptime": 3051, 2024-06-10T01:32:34.160 INFO:teuthology.orchestra.run.smithi078.stdout: "mdthresh_evicted": 0 2024-06-10T01:32:34.160 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.160 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-06-10T01:32:34.160 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-06-10T01:32:34.160 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-06-10T01:32:34.160 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T01:32:34.160 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-06-10T01:32:34.160 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T01:32:34.161 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-06-10T01:32:34.161 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T01:32:34.161 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T01:32:34.161 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T01:32:34.161 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T01:32:34.161 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T01:32:34.161 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-06-10T01:32:34.161 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T01:32:34.161 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T01:32:34.161 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-06-10T01:32:34.161 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-06-10T01:32:34.161 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-06-10T01:32:34.161 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-06-10T01:32:34.161 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T01:32:34.161 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T01:32:34.162 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T01:32:34.162 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T01:32:34.162 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T01:32:34.162 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-06-10T01:32:34.162 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-06-10T01:32:34.162 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-06-10T01:32:34.162 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T01:32:34.162 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T01:32:34.162 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-06-10T01:32:34.162 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-06-10T01:32:34.162 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-06-10T01:32:34.162 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-06-10T01:32:34.162 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T01:32:34.162 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-06-10T01:32:34.162 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T01:32:34.162 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-06-10T01:32:34.163 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 242695, 2024-06-10T01:32:34.163 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 90, 2024-06-10T01:32:34.163 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2024-06-10T01:32:34.163 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2024-06-10T01:32:34.163 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-06-10T01:32:34.163 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-06-10T01:32:34.163 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-06-10T01:32:34.163 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-06-10T01:32:34.163 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-06-10T01:32:34.163 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-06-10T01:32:34.163 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 8800, 2024-06-10T01:32:34.163 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 169, 2024-06-10T01:32:34.163 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T01:32:34.163 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-06-10T01:32:34.163 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-06-10T01:32:34.164 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-06-10T01:32:34.164 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 26975113, 2024-06-10T01:32:34.164 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 153402, 2024-06-10T01:32:34.164 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-06-10T01:32:34.164 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-06-10T01:32:34.164 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-06-10T01:32:34.164 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-06-10T01:32:34.164 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.164 INFO:teuthology.orchestra.run.smithi078.stdout: "objecter": { 2024-06-10T01:32:34.164 INFO:teuthology.orchestra.run.smithi078.stdout: "op_active": 0, 2024-06-10T01:32:34.164 INFO:teuthology.orchestra.run.smithi078.stdout: "op_laggy": 0, 2024-06-10T01:32:34.164 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send": 6294, 2024-06-10T01:32:34.164 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send_bytes": 21, 2024-06-10T01:32:34.164 INFO:teuthology.orchestra.run.smithi078.stdout: "op_resend": 3, 2024-06-10T01:32:34.164 INFO:teuthology.orchestra.run.smithi078.stdout: "op_reply": 6294, 2024-06-10T01:32:34.165 INFO:teuthology.orchestra.run.smithi078.stdout: "op_latency": { 2024-06-10T01:32:34.165 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 6294, 2024-06-10T01:32:34.165 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 23.265661339, 2024-06-10T01:32:34.165 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.003696482 2024-06-10T01:32:34.165 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.165 INFO:teuthology.orchestra.run.smithi078.stdout: "op_inflight": 0, 2024-06-10T01:32:34.165 INFO:teuthology.orchestra.run.smithi078.stdout: "oplen_avg": { 2024-06-10T01:32:34.165 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 6294, 2024-06-10T01:32:34.165 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 6295 2024-06-10T01:32:34.165 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.165 INFO:teuthology.orchestra.run.smithi078.stdout: "op": 6294, 2024-06-10T01:32:34.165 INFO:teuthology.orchestra.run.smithi078.stdout: "op_r": 6294, 2024-06-10T01:32:34.165 INFO:teuthology.orchestra.run.smithi078.stdout: "op_w": 0, 2024-06-10T01:32:34.165 INFO:teuthology.orchestra.run.smithi078.stdout: "op_rmw": 0, 2024-06-10T01:32:34.166 INFO:teuthology.orchestra.run.smithi078.stdout: "op_pg": 0, 2024-06-10T01:32:34.166 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_stat": 3427, 2024-06-10T01:32:34.166 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_create": 0, 2024-06-10T01:32:34.166 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_read": 2865, 2024-06-10T01:32:34.166 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_write": 0, 2024-06-10T01:32:34.166 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writefull": 0, 2024-06-10T01:32:34.166 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writesame": 0, 2024-06-10T01:32:34.166 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_append": 0, 2024-06-10T01:32:34.166 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_zero": 0, 2024-06-10T01:32:34.166 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_truncate": 0, 2024-06-10T01:32:34.166 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_delete": 0, 2024-06-10T01:32:34.166 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_mapext": 0, 2024-06-10T01:32:34.167 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_sparse_read": 0, 2024-06-10T01:32:34.167 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_clonerange": 0, 2024-06-10T01:32:34.167 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_getxattr": 1, 2024-06-10T01:32:34.167 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_setxattr": 0, 2024-06-10T01:32:34.167 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_cmpxattr": 0, 2024-06-10T01:32:34.167 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_rmxattr": 0, 2024-06-10T01:32:34.167 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_resetxattrs": 0, 2024-06-10T01:32:34.167 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_call": 0, 2024-06-10T01:32:34.167 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_watch": 0, 2024-06-10T01:32:34.167 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_notify": 0, 2024-06-10T01:32:34.167 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T01:32:34.167 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls": 0, 2024-06-10T01:32:34.167 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls_filter": 0, 2024-06-10T01:32:34.167 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_other": 0, 2024-06-10T01:32:34.168 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_active": 0, 2024-06-10T01:32:34.168 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_send": 0, 2024-06-10T01:32:34.168 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_resend": 0, 2024-06-10T01:32:34.168 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_ping": 0, 2024-06-10T01:32:34.168 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_active": 0, 2024-06-10T01:32:34.168 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_send": 0, 2024-06-10T01:32:34.168 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_resend": 0, 2024-06-10T01:32:34.168 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_active": 0, 2024-06-10T01:32:34.168 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_send": 0, 2024-06-10T01:32:34.168 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_resend": 0, 2024-06-10T01:32:34.168 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_active": 0, 2024-06-10T01:32:34.168 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_send": 0, 2024-06-10T01:32:34.168 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_resend": 0, 2024-06-10T01:32:34.168 INFO:teuthology.orchestra.run.smithi078.stdout: "command_active": 0, 2024-06-10T01:32:34.169 INFO:teuthology.orchestra.run.smithi078.stdout: "command_send": 0, 2024-06-10T01:32:34.169 INFO:teuthology.orchestra.run.smithi078.stdout: "command_resend": 0, 2024-06-10T01:32:34.169 INFO:teuthology.orchestra.run.smithi078.stdout: "map_epoch": 102, 2024-06-10T01:32:34.169 INFO:teuthology.orchestra.run.smithi078.stdout: "map_full": 0, 2024-06-10T01:32:34.169 INFO:teuthology.orchestra.run.smithi078.stdout: "map_inc": 5, 2024-06-10T01:32:34.169 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_sessions": 11, 2024-06-10T01:32:34.169 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_open": 12, 2024-06-10T01:32:34.169 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_close": 1, 2024-06-10T01:32:34.169 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_laggy": 0, 2024-06-10T01:32:34.169 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_wr": 0, 2024-06-10T01:32:34.169 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_rd": 2, 2024-06-10T01:32:34.169 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_del": 0 2024-06-10T01:32:34.169 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.169 INFO:teuthology.orchestra.run.smithi078.stdout: "oft": { 2024-06-10T01:32:34.170 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_objs": 0, 2024-06-10T01:32:34.170 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_kv_pairs": 0, 2024-06-10T01:32:34.170 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_updates": 0, 2024-06-10T01:32:34.170 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_removes": 0 2024-06-10T01:32:34.170 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.170 INFO:teuthology.orchestra.run.smithi078.stdout: "osd-slow-ops": { 2024-06-10T01:32:34.170 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_ops_count": 0 2024-06-10T01:32:34.170 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.170 INFO:teuthology.orchestra.run.smithi078.stdout: "purge_queue": { 2024-06-10T01:32:34.170 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops": 0, 2024-06-10T01:32:34.170 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops_high_water": 0, 2024-06-10T01:32:34.170 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing": 0, 2024-06-10T01:32:34.170 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_high_water": 0, 2024-06-10T01:32:34.170 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed_ops": 0, 2024-06-10T01:32:34.170 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed": 0, 2024-06-10T01:32:34.171 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_item_in_journal": 0 2024-06-10T01:32:34.171 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.171 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T01:32:34.171 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-06-10T01:32:34.171 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-06-10T01:32:34.171 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-06-10T01:32:34.171 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 6841, 2024-06-10T01:32:34.171 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 384296704, 2024-06-10T01:32:34.171 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:34.171 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 6841, 2024-06-10T01:32:34.171 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-06-10T01:32:34.171 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-06-10T01:32:34.171 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 6841, 2024-06-10T01:32:34.171 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 384296704, 2024-06-10T01:32:34.171 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-06-10T01:32:34.172 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.172 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.172 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.172 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-06-10T01:32:34.172 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.172 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_bytes": { 2024-06-10T01:32:34.172 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-06-10T01:32:34.172 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-06-10T01:32:34.172 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-06-10T01:32:34.172 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-06-10T01:32:34.172 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-06-10T01:32:34.172 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:34.172 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-06-10T01:32:34.172 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 6294, 2024-06-10T01:32:34.172 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 382466300, 2024-06-10T01:32:34.173 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 6294, 2024-06-10T01:32:34.173 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 382466300, 2024-06-10T01:32:34.173 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-06-10T01:32:34.173 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.173 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.173 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.173 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-06-10T01:32:34.173 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.173 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_ops": { 2024-06-10T01:32:34.173 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-06-10T01:32:34.173 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 1024, 2024-06-10T01:32:34.173 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-06-10T01:32:34.173 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-06-10T01:32:34.173 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-06-10T01:32:34.174 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:34.174 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-06-10T01:32:34.174 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 6294, 2024-06-10T01:32:34.174 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 6294, 2024-06-10T01:32:34.174 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 6294, 2024-06-10T01:32:34.174 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 6294, 2024-06-10T01:32:34.174 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-06-10T01:32:34.174 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.174 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.174 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.174 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-06-10T01:32:34.174 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.174 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle": { 2024-06-10T01:32:34.174 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-06-10T01:32:34.174 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-06-10T01:32:34.175 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-06-10T01:32:34.175 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-06-10T01:32:34.175 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-06-10T01:32:34.175 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:34.175 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-06-10T01:32:34.175 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-06-10T01:32:34.175 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-06-10T01:32:34.175 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 0, 2024-06-10T01:32:34.175 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 0, 2024-06-10T01:32:34.175 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-06-10T01:32:34.175 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.175 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.175 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.175 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-06-10T01:32:34.175 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:34.175 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle-0x55dffe3446e0": { 2024-06-10T01:32:34.176 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-06-10T01:32:34.176 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-06-10T01:32:34.176 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-06-10T01:32:34.176 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-06-10T01:32:34.176 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-06-10T01:32:34.176 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:34.176 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-06-10T01:32:34.176 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-06-10T01:32:34.176 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-06-10T01:32:34.176 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 0, 2024-06-10T01:32:34.176 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 0, 2024-06-10T01:32:34.176 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-06-10T01:32:34.176 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:34.176 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:34.176 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:34.176 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-06-10T01:32:34.176 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-06-10T01:32:34.177 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-06-10T01:32:34.485 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:34 smithi078 ceph-mon[36683]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:32:34.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:34 smithi007 ceph-mon[32398]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:32:34.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:34 smithi175 ceph-mon[36636]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:32:34.788 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-06-10T01:32:34.788 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-06-10T01:32:34.788 DEBUG:tasks.check_counter:Getting stats from k 2024-06-10T01:32:34.789 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph daemon mds.k perf dump 2024-06-10T01:32:35.026 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T01:32:35.439 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:35 smithi078 ceph-mon[36683]: pgmap v1231: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-10T01:32:35.439 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:35 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:32:35.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:35 smithi007 ceph-mon[32398]: pgmap v1231: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-10T01:32:35.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:35 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:32:35.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:35 smithi175 ceph-mon[36636]: pgmap v1231: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-10T01:32:35.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:35 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:32:36.430 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:32:36.431 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:32:36.431 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:32:36.431 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:32:36.431 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:32:36.431 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:32:36.431 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:36 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:32:36.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:32:36.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:32:36.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:32:36.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:32:36.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:32:36.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:32:36.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:36 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:32:36.692 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-06-10T01:32:36.692 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T01:32:36.692 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 4, 2024-06-10T01:32:36.692 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 4, 2024-06-10T01:32:36.692 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 2992, 2024-06-10T01:32:36.692 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 448, 2024-06-10T01:32:36.692 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 4, 2024-06-10T01:32:36.692 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 0, 2024-06-10T01:32:36.692 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.001402491, 2024-06-10T01:32:36.693 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.000044039, 2024-06-10T01:32:36.693 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.000647413, 2024-06-10T01:32:36.693 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T01:32:36.693 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:36.693 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 3, 2024-06-10T01:32:36.693 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000698645, 2024-06-10T01:32:36.693 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000232881 2024-06-10T01:32:36.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:36.693 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:36.693 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:36.693 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:36.693 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:36.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:36.694 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-06-10T01:32:36.694 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 448 2024-06-10T01:32:36.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:36.694 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T01:32:36.694 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 504, 2024-06-10T01:32:36.694 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 501, 2024-06-10T01:32:36.694 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 559952, 2024-06-10T01:32:36.694 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 310528, 2024-06-10T01:32:36.694 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 2, 2024-06-10T01:32:36.694 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 1, 2024-06-10T01:32:36.694 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.066222801, 2024-06-10T01:32:36.694 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.027148507, 2024-06-10T01:32:36.694 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.023816143, 2024-06-10T01:32:36.695 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.011210697, 2024-06-10T01:32:36.695 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:36.695 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 500, 2024-06-10T01:32:36.695 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.021771723, 2024-06-10T01:32:36.695 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000043543 2024-06-10T01:32:36.695 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:36.695 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:36.695 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:36.695 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:36.695 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:36.695 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:36.695 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 559952, 2024-06-10T01:32:36.695 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 310528 2024-06-10T01:32:36.695 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:36.696 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T01:32:36.696 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 3, 2024-06-10T01:32:36.696 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 385, 2024-06-10T01:32:36.696 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 480, 2024-06-10T01:32:36.696 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 301744, 2024-06-10T01:32:36.696 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 5, 2024-06-10T01:32:36.696 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 2, 2024-06-10T01:32:36.696 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.019609324, 2024-06-10T01:32:36.696 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.014296481, 2024-06-10T01:32:36.696 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.001589607, 2024-06-10T01:32:36.696 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T01:32:36.696 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:36.696 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 383, 2024-06-10T01:32:36.696 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.026033709, 2024-06-10T01:32:36.697 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000067973 2024-06-10T01:32:36.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:36.697 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:36.697 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:36.697 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:36.697 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:36.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:36.697 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 480, 2024-06-10T01:32:36.697 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 301744 2024-06-10T01:32:36.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:36.697 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-06-10T01:32:36.697 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 0, 2024-06-10T01:32:36.697 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-06-10T01:32:36.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:36.698 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-06-10T01:32:36.698 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-06-10T01:32:36.698 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-06-10T01:32:36.698 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T01:32:36.698 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-06-10T01:32:36.698 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T01:32:36.698 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-06-10T01:32:36.698 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T01:32:36.698 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T01:32:36.698 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T01:32:36.698 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T01:32:36.698 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T01:32:36.698 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-06-10T01:32:36.699 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T01:32:36.699 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T01:32:36.699 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-06-10T01:32:36.699 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-06-10T01:32:36.699 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-06-10T01:32:36.699 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-06-10T01:32:36.699 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T01:32:36.699 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T01:32:36.699 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T01:32:36.699 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T01:32:36.699 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T01:32:36.699 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-06-10T01:32:36.699 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-06-10T01:32:36.699 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-06-10T01:32:36.700 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T01:32:36.701 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T01:32:36.701 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-06-10T01:32:36.701 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-06-10T01:32:36.701 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-06-10T01:32:36.701 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-06-10T01:32:36.701 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T01:32:36.701 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-06-10T01:32:36.701 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T01:32:36.701 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-06-10T01:32:36.702 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 148784, 2024-06-10T01:32:36.702 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 31, 2024-06-10T01:32:36.702 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2024-06-10T01:32:36.702 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2024-06-10T01:32:36.702 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-06-10T01:32:36.702 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-06-10T01:32:36.702 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-06-10T01:32:36.702 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-06-10T01:32:36.702 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-06-10T01:32:36.702 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-06-10T01:32:36.702 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 0, 2024-06-10T01:32:36.702 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 0, 2024-06-10T01:32:36.702 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T01:32:36.703 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-06-10T01:32:36.703 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-06-10T01:32:36.703 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-06-10T01:32:36.703 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 928, 2024-06-10T01:32:36.703 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 2, 2024-06-10T01:32:36.703 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-06-10T01:32:36.703 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-06-10T01:32:36.703 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-06-10T01:32:36.703 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-06-10T01:32:36.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-06-10T01:32:36.703 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T01:32:36.703 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-06-10T01:32:36.703 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-06-10T01:32:36.703 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-06-10T01:32:36.703 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 507, 2024-06-10T01:32:36.704 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 488673, 2024-06-10T01:32:36.704 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:36.704 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 507, 2024-06-10T01:32:36.704 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-06-10T01:32:36.704 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-06-10T01:32:36.704 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 507, 2024-06-10T01:32:36.704 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 488673, 2024-06-10T01:32:36.704 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-06-10T01:32:36.704 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-06-10T01:32:36.704 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-06-10T01:32:36.704 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-06-10T01:32:36.704 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-06-10T01:32:36.704 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-06-10T01:32:36.704 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-06-10T01:32:36.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:32:36.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:32:36.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:32:36.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:32:36.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:32:36.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:32:36.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:36 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' 2024-06-10T01:32:37.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:37 smithi078 ceph-mon[36683]: Adjusting osd_memory_target on smithi175 to 176.2M 2024-06-10T01:32:37.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:37 smithi078 ceph-mon[36683]: Unable to set osd_memory_target on smithi175 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-10T01:32:37.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:37 smithi078 ceph-mon[36683]: pgmap v1232: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-10T01:32:37.391 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-06-10T01:32:37.391 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-06-10T01:32:37.391 DEBUG:tasks.check_counter:Getting stats from c 2024-06-10T01:32:37.392 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph daemon mds.c perf dump 2024-06-10T01:32:37.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:37 smithi007 ceph-mon[32398]: Adjusting osd_memory_target on smithi175 to 176.2M 2024-06-10T01:32:37.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:37 smithi007 ceph-mon[32398]: Unable to set osd_memory_target on smithi175 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-10T01:32:37.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:37 smithi007 ceph-mon[32398]: pgmap v1232: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-10T01:32:37.678 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T01:32:37.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:37 smithi175 ceph-mon[36636]: Adjusting osd_memory_target on smithi175 to 176.2M 2024-06-10T01:32:37.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:37 smithi175 ceph-mon[36636]: Unable to set osd_memory_target on smithi175 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-10T01:32:37.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:37 smithi175 ceph-mon[36636]: pgmap v1232: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-10T01:32:39.378 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:39 smithi175 ceph-mon[36636]: pgmap v1233: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-10T01:32:39.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:39 smithi007 ceph-mon[32398]: pgmap v1233: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-10T01:32:39.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:39 smithi078 ceph-mon[36683]: pgmap v1233: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-10T01:32:39.611 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-06-10T01:32:39.611 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T01:32:39.611 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 385022, 2024-06-10T01:32:39.611 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 581141, 2024-06-10T01:32:39.611 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 135201366, 2024-06-10T01:32:39.611 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 630131361, 2024-06-10T01:32:39.611 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 75, 2024-06-10T01:32:39.611 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 6, 2024-06-10T01:32:39.612 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 31.459594966, 2024-06-10T01:32:39.612 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 13.455816677, 2024-06-10T01:32:39.612 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 19.272772793, 2024-06-10T01:32:39.612 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 1.989083252, 2024-06-10T01:32:39.612 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:39.612 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 581136, 2024-06-10T01:32:39.612 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 90.709690139, 2024-06-10T01:32:39.612 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000156090 2024-06-10T01:32:39.612 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.612 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:39.612 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 793292, 2024-06-10T01:32:39.612 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.175143815, 2024-06-10T01:32:39.612 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000220 2024-06-10T01:32:39.613 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.613 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 124648784, 2024-06-10T01:32:39.613 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 506289120 2024-06-10T01:32:39.613 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.613 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T01:32:39.613 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 188685, 2024-06-10T01:32:39.613 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 161297, 2024-06-10T01:32:39.613 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 49283211, 2024-06-10T01:32:39.613 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 129293043, 2024-06-10T01:32:39.613 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 10, 2024-06-10T01:32:39.613 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 5, 2024-06-10T01:32:39.613 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 13.332411801, 2024-06-10T01:32:39.613 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 5.301181232, 2024-06-10T01:32:39.614 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 6.735318599, 2024-06-10T01:32:39.614 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 2.472680606, 2024-06-10T01:32:39.614 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:39.614 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 161298, 2024-06-10T01:32:39.614 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 14.780641761, 2024-06-10T01:32:39.614 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000091635 2024-06-10T01:32:39.614 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.614 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:39.614 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 272173, 2024-06-10T01:32:39.614 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.044404419, 2024-06-10T01:32:39.614 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000163 2024-06-10T01:32:39.614 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.615 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-06-10T01:32:39.615 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 820512 2024-06-10T01:32:39.615 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.615 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T01:32:39.615 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 104912, 2024-06-10T01:32:39.615 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 103286, 2024-06-10T01:32:39.615 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 30890199, 2024-06-10T01:32:39.615 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 221604642, 2024-06-10T01:32:39.615 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 8, 2024-06-10T01:32:39.615 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 5, 2024-06-10T01:32:39.615 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 9.218691440, 2024-06-10T01:32:39.615 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 2.676827095, 2024-06-10T01:32:39.616 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 2.935855288, 2024-06-10T01:32:39.616 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 3.548881425, 2024-06-10T01:32:39.616 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:39.616 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 103286, 2024-06-10T01:32:39.616 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 16.657238745, 2024-06-10T01:32:39.616 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000161272 2024-06-10T01:32:39.616 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.616 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:39.616 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 80471, 2024-06-10T01:32:39.616 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.016085971, 2024-06-10T01:32:39.616 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000199 2024-06-10T01:32:39.616 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.616 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-10T01:32:39.617 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 0 2024-06-10T01:32:39.617 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.617 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2024-06-10T01:32:39.617 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 1, 2024-06-10T01:32:39.617 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2024-06-10T01:32:39.617 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.617 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-MDSRank": { 2024-06-10T01:32:39.617 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-06-10T01:32:39.617 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-06-10T01:32:39.617 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 98675, 2024-06-10T01:32:39.617 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 189.050627429, 2024-06-10T01:32:39.617 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.001915891 2024-06-10T01:32:39.617 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:39.617 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.618 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-PurgeQueue": { 2024-06-10T01:32:39.618 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-06-10T01:32:39.618 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-06-10T01:32:39.618 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 66775, 2024-06-10T01:32:39.618 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 14.191362008, 2024-06-10T01:32:39.618 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000212525 2024-06-10T01:32:39.618 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:39.618 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.618 INFO:teuthology.orchestra.run.smithi175.stdout: "mds": { 2024-06-10T01:32:39.618 INFO:teuthology.orchestra.run.smithi175.stdout: "request": 234796, 2024-06-10T01:32:39.618 INFO:teuthology.orchestra.run.smithi175.stdout: "reply": 224577, 2024-06-10T01:32:39.618 INFO:teuthology.orchestra.run.smithi175.stdout: "reply_latency": { 2024-06-10T01:32:39.618 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 224577, 2024-06-10T01:32:39.619 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 1935.394039282, 2024-06-10T01:32:39.619 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.008617953 2024-06-10T01:32:39.619 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.619 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_reply": 0, 2024-06-10T01:32:39.619 INFO:teuthology.orchestra.run.smithi175.stdout: "forward": 10219, 2024-06-10T01:32:39.619 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_complete": 0, 2024-06-10T01:32:39.619 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_keys": 0, 2024-06-10T01:32:39.619 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_commit": 132, 2024-06-10T01:32:39.619 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_split": 13, 2024-06-10T01:32:39.619 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_merge": 14, 2024-06-10T01:32:39.619 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes": 1447, 2024-06-10T01:32:39.619 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_top": 854, 2024-06-10T01:32:39.619 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_bottom": 370, 2024-06-10T01:32:39.619 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pin_tail": 223, 2024-06-10T01:32:39.620 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pinned": 226, 2024-06-10T01:32:39.620 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_expired": 176, 2024-06-10T01:32:39.620 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_with_caps": 0, 2024-06-10T01:32:39.620 INFO:teuthology.orchestra.run.smithi175.stdout: "caps": 0, 2024-06-10T01:32:39.620 INFO:teuthology.orchestra.run.smithi175.stdout: "subtrees": 4, 2024-06-10T01:32:39.620 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse": 376990, 2024-06-10T01:32:39.620 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_hit": 196115, 2024-06-10T01:32:39.620 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_forward": 10215, 2024-06-10T01:32:39.620 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_discover": 52, 2024-06-10T01:32:39.620 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_dir_fetch": 0, 2024-06-10T01:32:39.620 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_remote_ino": 0, 2024-06-10T01:32:39.620 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_lock": 3362, 2024-06-10T01:32:39.620 INFO:teuthology.orchestra.run.smithi175.stdout: "load_cent": 0, 2024-06-10T01:32:39.620 INFO:teuthology.orchestra.run.smithi175.stdout: "q": 0, 2024-06-10T01:32:39.621 INFO:teuthology.orchestra.run.smithi175.stdout: "exported": 8, 2024-06-10T01:32:39.621 INFO:teuthology.orchestra.run.smithi175.stdout: "exported_inodes": 56, 2024-06-10T01:32:39.621 INFO:teuthology.orchestra.run.smithi175.stdout: "imported": 8, 2024-06-10T01:32:39.621 INFO:teuthology.orchestra.run.smithi175.stdout: "imported_inodes": 564, 2024-06-10T01:32:39.621 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_dir_fetch": 0, 2024-06-10T01:32:39.621 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_backtrace_fetch": 3507, 2024-06-10T01:32:39.621 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_peer_discover": 1, 2024-06-10T01:32:39.621 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rfiles": 698, 2024-06-10T01:32:39.621 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rbytes": 211204299, 2024-06-10T01:32:39.621 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rsnaps": 0, 2024-06-10T01:32:39.621 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_fetch": 3507, 2024-06-10T01:32:39.621 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_set_tag": 0, 2024-06-10T01:32:39.621 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T01:32:39.621 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_inotable_repaired": 0, 2024-06-10T01:32:39.621 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_inodes": 360, 2024-06-10T01:32:39.622 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_base_inodes": 0, 2024-06-10T01:32:39.622 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dirfrag_rstats": 360, 2024-06-10T01:32:39.622 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_file_inodes": 3147, 2024-06-10T01:32:39.622 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_inode_file_caps": 12, 2024-06-10T01:32:39.622 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_revoke": 12993, 2024-06-10T01:32:39.622 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_grant": 165350, 2024-06-10T01:32:39.622 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_trunc": 0, 2024-06-10T01:32:39.622 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T01:32:39.622 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T01:32:39.622 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps": 81507, 2024-06-10T01:32:39.622 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps_dirty": 37031, 2024-06-10T01:32:39.622 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_cap_release": 16126, 2024-06-10T01:32:39.622 INFO:teuthology.orchestra.run.smithi175.stdout: "process_request_cap_release": 151622 2024-06-10T01:32:39.623 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.623 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_cache": { 2024-06-10T01:32:39.623 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update": 98, 2024-06-10T01:32:39.623 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update_receipt": 212, 2024-06-10T01:32:39.623 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_try_discover": 117, 2024-06-10T01:32:39.623 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_send_discover": 258, 2024-06-10T01:32:39.623 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_handle_discover": 116, 2024-06-10T01:32:39.623 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays": 3, 2024-06-10T01:32:39.623 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_delayed": 0, 2024-06-10T01:32:39.623 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_enqueuing": 0, 2024-06-10T01:32:39.623 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_created": 38174, 2024-06-10T01:32:39.623 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_enqueued": 38171, 2024-06-10T01:32:39.623 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_reintegrated": 0, 2024-06-10T01:32:39.623 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_migrated": 0, 2024-06-10T01:32:39.624 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_processing": 0, 2024-06-10T01:32:39.624 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_enqueued": 0, 2024-06-10T01:32:39.624 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_prioritized": 0, 2024-06-10T01:32:39.624 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_started": 0, 2024-06-10T01:32:39.624 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_completed": 0, 2024-06-10T01:32:39.624 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_path": 20, 2024-06-10T01:32:39.624 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_inode": 3269, 2024-06-10T01:32:39.624 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_enqueue_scrub": 0, 2024-06-10T01:32:39.624 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_exportdir": 8, 2024-06-10T01:32:39.624 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_flush": 0, 2024-06-10T01:32:39.624 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragmentdir": 27, 2024-06-10T01:32:39.624 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragstats": 0, 2024-06-10T01:32:39.624 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_inodestats": 0 2024-06-10T01:32:39.625 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.625 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_log": { 2024-06-10T01:32:39.625 INFO:teuthology.orchestra.run.smithi175.stdout: "evlrg": 0, 2024-06-10T01:32:39.625 INFO:teuthology.orchestra.run.smithi175.stdout: "evadd": 202911, 2024-06-10T01:32:39.625 INFO:teuthology.orchestra.run.smithi175.stdout: "evex": 96669, 2024-06-10T01:32:39.625 INFO:teuthology.orchestra.run.smithi175.stdout: "evtrm": 86031, 2024-06-10T01:32:39.625 INFO:teuthology.orchestra.run.smithi175.stdout: "ev": 116880, 2024-06-10T01:32:39.625 INFO:teuthology.orchestra.run.smithi175.stdout: "evexg": 0, 2024-06-10T01:32:39.625 INFO:teuthology.orchestra.run.smithi175.stdout: "evexd": 10638, 2024-06-10T01:32:39.625 INFO:teuthology.orchestra.run.smithi175.stdout: "segadd": 246, 2024-06-10T01:32:39.625 INFO:teuthology.orchestra.run.smithi175.stdout: "segex": 118, 2024-06-10T01:32:39.625 INFO:teuthology.orchestra.run.smithi175.stdout: "segtrm": 104, 2024-06-10T01:32:39.625 INFO:teuthology.orchestra.run.smithi175.stdout: "seg": 142, 2024-06-10T01:32:39.625 INFO:teuthology.orchestra.run.smithi175.stdout: "segmjr": 10, 2024-06-10T01:32:39.626 INFO:teuthology.orchestra.run.smithi175.stdout: "segexg": 0, 2024-06-10T01:32:39.626 INFO:teuthology.orchestra.run.smithi175.stdout: "segexd": 14, 2024-06-10T01:32:39.626 INFO:teuthology.orchestra.run.smithi175.stdout: "expos": 150764945, 2024-06-10T01:32:39.626 INFO:teuthology.orchestra.run.smithi175.stdout: "wrpos": 386642851, 2024-06-10T01:32:39.626 INFO:teuthology.orchestra.run.smithi175.stdout: "rdpos": 0, 2024-06-10T01:32:39.626 INFO:teuthology.orchestra.run.smithi175.stdout: "jlat": { 2024-06-10T01:32:39.626 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 56077, 2024-06-10T01:32:39.626 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 666.740056687, 2024-06-10T01:32:39.626 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.011889724 2024-06-10T01:32:39.626 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.626 INFO:teuthology.orchestra.run.smithi175.stdout: "replayed": 0 2024-06-10T01:32:39.626 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.626 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_mem": { 2024-06-10T01:32:39.627 INFO:teuthology.orchestra.run.smithi175.stdout: "ino": 29, 2024-06-10T01:32:39.627 INFO:teuthology.orchestra.run.smithi175.stdout: "ino+": 38283, 2024-06-10T01:32:39.627 INFO:teuthology.orchestra.run.smithi175.stdout: "ino-": 38254, 2024-06-10T01:32:39.627 INFO:teuthology.orchestra.run.smithi175.stdout: "dir": 45, 2024-06-10T01:32:39.627 INFO:teuthology.orchestra.run.smithi175.stdout: "dir+": 341, 2024-06-10T01:32:39.627 INFO:teuthology.orchestra.run.smithi175.stdout: "dir-": 296, 2024-06-10T01:32:39.627 INFO:teuthology.orchestra.run.smithi175.stdout: "dn": 1447, 2024-06-10T01:32:39.627 INFO:teuthology.orchestra.run.smithi175.stdout: "dn+": 78241, 2024-06-10T01:32:39.627 INFO:teuthology.orchestra.run.smithi175.stdout: "dn-": 76794, 2024-06-10T01:32:39.627 INFO:teuthology.orchestra.run.smithi175.stdout: "cap": 0, 2024-06-10T01:32:39.627 INFO:teuthology.orchestra.run.smithi175.stdout: "cap+": 43127, 2024-06-10T01:32:39.627 INFO:teuthology.orchestra.run.smithi175.stdout: "cap-": 43127, 2024-06-10T01:32:39.627 INFO:teuthology.orchestra.run.smithi175.stdout: "rss": 83736, 2024-06-10T01:32:39.627 INFO:teuthology.orchestra.run.smithi175.stdout: "heap": 223612 2024-06-10T01:32:39.628 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.628 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_server": { 2024-06-10T01:32:39.628 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_client_request": 321639, 2024-06-10T01:32:39.628 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_server_request": 14, 2024-06-10T01:32:39.628 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_request": 234796, 2024-06-10T01:32:39.628 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_session": 5925, 2024-06-10T01:32:39.628 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_peer_request": 15800, 2024-06-10T01:32:39.628 INFO:teuthology.orchestra.run.smithi175.stdout: "req_create_latency": { 2024-06-10T01:32:39.628 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 37955, 2024-06-10T01:32:39.628 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 663.180062640, 2024-06-10T01:32:39.628 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.017472798 2024-06-10T01:32:39.628 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.628 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getattr_latency": { 2024-06-10T01:32:39.628 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1351, 2024-06-10T01:32:39.629 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 17.720026430, 2024-06-10T01:32:39.629 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.013116229 2024-06-10T01:32:39.629 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.629 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getfilelock_latency": { 2024-06-10T01:32:39.629 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:39.629 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:39.629 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:39.629 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.629 INFO:teuthology.orchestra.run.smithi175.stdout: "req_link_latency": { 2024-06-10T01:32:39.629 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:39.629 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:39.629 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:39.629 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.629 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookup_latency": { 2024-06-10T01:32:39.630 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 78502, 2024-06-10T01:32:39.630 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 198.013315745, 2024-06-10T01:32:39.630 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.002522398 2024-06-10T01:32:39.630 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.630 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookuphash_latency": { 2024-06-10T01:32:39.630 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:39.630 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:39.630 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:39.630 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.630 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupino_latency": { 2024-06-10T01:32:39.630 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:39.630 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:39.630 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:39.631 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.631 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupname_latency": { 2024-06-10T01:32:39.631 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:39.631 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:39.631 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:39.631 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.631 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupparent_latency": { 2024-06-10T01:32:39.631 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:39.631 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:39.631 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:39.631 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.631 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupsnap_latency": { 2024-06-10T01:32:39.632 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:39.632 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:39.632 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:39.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.632 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lssnap_latency": { 2024-06-10T01:32:39.632 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:39.632 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:39.632 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:39.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.632 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mkdir_latency": { 2024-06-10T01:32:39.632 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:39.632 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:39.633 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:39.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.633 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mknod_latency": { 2024-06-10T01:32:39.633 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:39.633 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:39.633 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:39.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.633 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mksnap_latency": { 2024-06-10T01:32:39.633 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:39.633 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:39.633 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:39.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.633 INFO:teuthology.orchestra.run.smithi175.stdout: "req_open_latency": { 2024-06-10T01:32:39.634 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:39.634 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:39.634 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:39.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.634 INFO:teuthology.orchestra.run.smithi175.stdout: "req_readdir_latency": { 2024-06-10T01:32:39.634 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 59760, 2024-06-10T01:32:39.634 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 633.828127558, 2024-06-10T01:32:39.634 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.010606227 2024-06-10T01:32:39.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.634 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rename_latency": { 2024-06-10T01:32:39.634 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 7665, 2024-06-10T01:32:39.634 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 86.223944422, 2024-06-10T01:32:39.634 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.011249046 2024-06-10T01:32:39.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.635 INFO:teuthology.orchestra.run.smithi175.stdout: "req_renamesnap_latency": { 2024-06-10T01:32:39.635 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:39.635 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:39.635 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:39.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.635 INFO:teuthology.orchestra.run.smithi175.stdout: "req_snapdiff_latency": { 2024-06-10T01:32:39.635 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:39.635 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:39.635 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:39.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.635 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmdir_latency": { 2024-06-10T01:32:39.635 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 30, 2024-06-10T01:32:39.636 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.395684962, 2024-06-10T01:32:39.636 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.013189498 2024-06-10T01:32:39.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.636 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmsnap_latency": { 2024-06-10T01:32:39.636 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:39.636 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:39.636 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:39.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.636 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmxattr_latency": { 2024-06-10T01:32:39.636 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:39.636 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:39.636 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:39.637 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.637 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setattr_latency": { 2024-06-10T01:32:39.637 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 174, 2024-06-10T01:32:39.637 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 97.557064514, 2024-06-10T01:32:39.637 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.560672784 2024-06-10T01:32:39.637 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.637 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setdirlayout_latency": { 2024-06-10T01:32:39.637 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:39.637 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:39.637 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:39.637 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.637 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setfilelock_latency": { 2024-06-10T01:32:39.637 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 996, 2024-06-10T01:32:39.638 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 2.560335942, 2024-06-10T01:32:39.638 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.002570618 2024-06-10T01:32:39.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.638 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setlayout_latency": { 2024-06-10T01:32:39.638 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:39.638 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:39.638 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:39.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.638 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setxattr_latency": { 2024-06-10T01:32:39.638 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:39.638 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:39.638 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:39.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.639 INFO:teuthology.orchestra.run.smithi175.stdout: "req_symlink_latency": { 2024-06-10T01:32:39.639 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:39.639 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:39.639 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:39.639 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.639 INFO:teuthology.orchestra.run.smithi175.stdout: "req_unlink_latency": { 2024-06-10T01:32:39.639 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 38144, 2024-06-10T01:32:39.639 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 235.915477069, 2024-06-10T01:32:39.639 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.006184864 2024-06-10T01:32:39.639 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.639 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_revoke_eviction": 0, 2024-06-10T01:32:39.639 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_acquisition_throttle": 0, 2024-06-10T01:32:39.639 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getvxattr_latency": { 2024-06-10T01:32:39.639 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:39.640 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:39.640 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:39.640 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:39.640 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.640 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_sessions": { 2024-06-10T01:32:39.640 INFO:teuthology.orchestra.run.smithi175.stdout: "session_count": 1, 2024-06-10T01:32:39.640 INFO:teuthology.orchestra.run.smithi175.stdout: "session_add": 1, 2024-06-10T01:32:39.640 INFO:teuthology.orchestra.run.smithi175.stdout: "session_remove": 0, 2024-06-10T01:32:39.640 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_open": 1, 2024-06-10T01:32:39.640 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_stale": 0, 2024-06-10T01:32:39.640 INFO:teuthology.orchestra.run.smithi175.stdout: "total_load": 23394, 2024-06-10T01:32:39.640 INFO:teuthology.orchestra.run.smithi175.stdout: "average_load": 23394, 2024-06-10T01:32:39.640 INFO:teuthology.orchestra.run.smithi175.stdout: "avg_session_uptime": 1581, 2024-06-10T01:32:39.640 INFO:teuthology.orchestra.run.smithi175.stdout: "mdthresh_evicted": 0 2024-06-10T01:32:39.640 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.641 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2024-06-10T01:32:39.641 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2024-06-10T01:32:39.641 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2024-06-10T01:32:39.641 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T01:32:39.641 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2024-06-10T01:32:39.641 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T01:32:39.641 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2024-06-10T01:32:39.641 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T01:32:39.641 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T01:32:39.641 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T01:32:39.641 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T01:32:39.641 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T01:32:39.641 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2024-06-10T01:32:39.641 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T01:32:39.641 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T01:32:39.642 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_bytes": 0, 2024-06-10T01:32:39.642 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_items": 0, 2024-06-10T01:32:39.642 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_bytes": 0, 2024-06-10T01:32:39.642 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_items": 0, 2024-06-10T01:32:39.642 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T01:32:39.642 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T01:32:39.642 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T01:32:39.642 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T01:32:39.642 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T01:32:39.642 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2024-06-10T01:32:39.642 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2024-06-10T01:32:39.642 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2024-06-10T01:32:39.642 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T01:32:39.642 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T01:32:39.643 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2024-06-10T01:32:39.643 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2024-06-10T01:32:39.643 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2024-06-10T01:32:39.643 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2024-06-10T01:32:39.643 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T01:32:39.643 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2024-06-10T01:32:39.643 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T01:32:39.643 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2024-06-10T01:32:39.643 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 281206, 2024-06-10T01:32:39.643 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 103, 2024-06-10T01:32:39.643 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 352, 2024-06-10T01:32:39.643 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 4, 2024-06-10T01:32:39.643 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2024-06-10T01:32:39.643 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2024-06-10T01:32:39.643 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2024-06-10T01:32:39.644 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2024-06-10T01:32:39.644 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2024-06-10T01:32:39.644 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2024-06-10T01:32:39.644 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 8800, 2024-06-10T01:32:39.644 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 169, 2024-06-10T01:32:39.644 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T01:32:39.644 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2024-06-10T01:32:39.644 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2024-06-10T01:32:39.644 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2024-06-10T01:32:39.644 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 1225806, 2024-06-10T01:32:39.644 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 16694, 2024-06-10T01:32:39.644 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2024-06-10T01:32:39.644 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2024-06-10T01:32:39.644 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2024-06-10T01:32:39.644 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2024-06-10T01:32:39.645 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.645 INFO:teuthology.orchestra.run.smithi175.stdout: "objecter": { 2024-06-10T01:32:39.645 INFO:teuthology.orchestra.run.smithi175.stdout: "op_active": 0, 2024-06-10T01:32:39.645 INFO:teuthology.orchestra.run.smithi175.stdout: "op_laggy": 0, 2024-06-10T01:32:39.645 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send": 140633, 2024-06-10T01:32:39.645 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send_bytes": 391718420, 2024-06-10T01:32:39.645 INFO:teuthology.orchestra.run.smithi175.stdout: "op_resend": 28, 2024-06-10T01:32:39.645 INFO:teuthology.orchestra.run.smithi175.stdout: "op_reply": 140633, 2024-06-10T01:32:39.645 INFO:teuthology.orchestra.run.smithi175.stdout: "op_latency": { 2024-06-10T01:32:39.645 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 140633, 2024-06-10T01:32:39.645 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 2039.918236495, 2024-06-10T01:32:39.645 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.014505260 2024-06-10T01:32:39.645 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.645 INFO:teuthology.orchestra.run.smithi175.stdout: "op_inflight": 0, 2024-06-10T01:32:39.645 INFO:teuthology.orchestra.run.smithi175.stdout: "oplen_avg": { 2024-06-10T01:32:39.646 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 140633, 2024-06-10T01:32:39.646 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 142516 2024-06-10T01:32:39.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.646 INFO:teuthology.orchestra.run.smithi175.stdout: "op": 140633, 2024-06-10T01:32:39.646 INFO:teuthology.orchestra.run.smithi175.stdout: "op_r": 4797, 2024-06-10T01:32:39.646 INFO:teuthology.orchestra.run.smithi175.stdout: "op_w": 135836, 2024-06-10T01:32:39.646 INFO:teuthology.orchestra.run.smithi175.stdout: "op_rmw": 0, 2024-06-10T01:32:39.646 INFO:teuthology.orchestra.run.smithi175.stdout: "op_pg": 0, 2024-06-10T01:32:39.646 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_stat": 3, 2024-06-10T01:32:39.646 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_create": 31, 2024-06-10T01:32:39.646 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_read": 1290, 2024-06-10T01:32:39.646 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_write": 57367, 2024-06-10T01:32:39.647 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writefull": 683, 2024-06-10T01:32:39.647 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writesame": 0, 2024-06-10T01:32:39.647 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_append": 0, 2024-06-10T01:32:39.647 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_zero": 0, 2024-06-10T01:32:39.647 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_truncate": 13, 2024-06-10T01:32:39.647 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_delete": 76543, 2024-06-10T01:32:39.647 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_mapext": 0, 2024-06-10T01:32:39.647 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_sparse_read": 0, 2024-06-10T01:32:39.647 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_clonerange": 0, 2024-06-10T01:32:39.647 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_getxattr": 3507, 2024-06-10T01:32:39.647 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_setxattr": 57, 2024-06-10T01:32:39.647 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_cmpxattr": 0, 2024-06-10T01:32:39.647 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_rmxattr": 0, 2024-06-10T01:32:39.647 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_resetxattrs": 0, 2024-06-10T01:32:39.647 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_call": 0, 2024-06-10T01:32:39.648 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_watch": 0, 2024-06-10T01:32:39.648 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_notify": 0, 2024-06-10T01:32:39.648 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T01:32:39.648 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls": 0, 2024-06-10T01:32:39.648 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls_filter": 0, 2024-06-10T01:32:39.648 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_other": 0, 2024-06-10T01:32:39.648 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_active": 0, 2024-06-10T01:32:39.648 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_send": 0, 2024-06-10T01:32:39.648 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_resend": 0, 2024-06-10T01:32:39.648 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_ping": 0, 2024-06-10T01:32:39.648 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_active": 0, 2024-06-10T01:32:39.648 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_send": 0, 2024-06-10T01:32:39.648 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_resend": 0, 2024-06-10T01:32:39.648 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_active": 0, 2024-06-10T01:32:39.648 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_send": 0, 2024-06-10T01:32:39.649 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_resend": 0, 2024-06-10T01:32:39.649 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_active": 0, 2024-06-10T01:32:39.649 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_send": 0, 2024-06-10T01:32:39.649 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_resend": 0, 2024-06-10T01:32:39.649 INFO:teuthology.orchestra.run.smithi175.stdout: "command_active": 0, 2024-06-10T01:32:39.649 INFO:teuthology.orchestra.run.smithi175.stdout: "command_send": 0, 2024-06-10T01:32:39.649 INFO:teuthology.orchestra.run.smithi175.stdout: "command_resend": 0, 2024-06-10T01:32:39.649 INFO:teuthology.orchestra.run.smithi175.stdout: "map_epoch": 102, 2024-06-10T01:32:39.649 INFO:teuthology.orchestra.run.smithi175.stdout: "map_full": 0, 2024-06-10T01:32:39.649 INFO:teuthology.orchestra.run.smithi175.stdout: "map_inc": 5, 2024-06-10T01:32:39.649 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_sessions": 12, 2024-06-10T01:32:39.649 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_open": 12, 2024-06-10T01:32:39.649 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_close": 0, 2024-06-10T01:32:39.649 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_laggy": 0, 2024-06-10T01:32:39.650 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_wr": 2278, 2024-06-10T01:32:39.650 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_rd": 0, 2024-06-10T01:32:39.650 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_del": 744 2024-06-10T01:32:39.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.650 INFO:teuthology.orchestra.run.smithi175.stdout: "oft": { 2024-06-10T01:32:39.650 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_objs": 1, 2024-06-10T01:32:39.650 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_kv_pairs": 0, 2024-06-10T01:32:39.650 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_updates": 0, 2024-06-10T01:32:39.650 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_removes": 0 2024-06-10T01:32:39.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.650 INFO:teuthology.orchestra.run.smithi175.stdout: "osd-slow-ops": { 2024-06-10T01:32:39.650 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_ops_count": 3275 2024-06-10T01:32:39.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.650 INFO:teuthology.orchestra.run.smithi175.stdout: "purge_queue": { 2024-06-10T01:32:39.650 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops": 0, 2024-06-10T01:32:39.651 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops_high_water": 17, 2024-06-10T01:32:39.651 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing": 0, 2024-06-10T01:32:39.651 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_high_water": 15, 2024-06-10T01:32:39.651 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed_ops": 76315, 2024-06-10T01:32:39.651 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed": 38171, 2024-06-10T01:32:39.651 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_item_in_journal": 0 2024-06-10T01:32:39.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.651 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T01:32:39.651 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-10T01:32:39.651 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-06-10T01:32:39.651 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-10T01:32:39.651 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 678615, 2024-06-10T01:32:39.651 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 138717472, 2024-06-10T01:32:39.652 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:39.652 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 678615, 2024-06-10T01:32:39.652 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-06-10T01:32:39.652 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-06-10T01:32:39.652 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 678615, 2024-06-10T01:32:39.652 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 138717472, 2024-06-10T01:32:39.652 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-10T01:32:39.652 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:39.652 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:39.652 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:39.652 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:39.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.652 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_bytes": { 2024-06-10T01:32:39.652 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-10T01:32:39.653 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-06-10T01:32:39.653 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-10T01:32:39.653 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-06-10T01:32:39.653 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-06-10T01:32:39.653 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:39.653 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-06-10T01:32:39.653 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 140633, 2024-06-10T01:32:39.653 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 395878843, 2024-06-10T01:32:39.653 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 140633, 2024-06-10T01:32:39.653 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 395878843, 2024-06-10T01:32:39.653 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-10T01:32:39.653 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:39.653 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:39.653 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:39.653 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:39.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.654 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_ops": { 2024-06-10T01:32:39.654 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-10T01:32:39.654 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 1024, 2024-06-10T01:32:39.654 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-10T01:32:39.654 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-06-10T01:32:39.654 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-06-10T01:32:39.654 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:39.654 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-06-10T01:32:39.654 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 140633, 2024-06-10T01:32:39.654 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 140633, 2024-06-10T01:32:39.654 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 140633, 2024-06-10T01:32:39.654 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 140633, 2024-06-10T01:32:39.654 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-10T01:32:39.655 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:39.655 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:39.655 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:39.655 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:39.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.655 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle": { 2024-06-10T01:32:39.655 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-10T01:32:39.655 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-06-10T01:32:39.655 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-10T01:32:39.655 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 38171, 2024-06-10T01:32:39.655 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 4160423, 2024-06-10T01:32:39.655 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:39.655 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 38171, 2024-06-10T01:32:39.655 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-06-10T01:32:39.655 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-06-10T01:32:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 1290, 2024-06-10T01:32:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 4160423, 2024-06-10T01:32:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-10T01:32:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle-0x56406dd98460": { 2024-06-10T01:32:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-10T01:32:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-06-10T01:32:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-10T01:32:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 202911, 2024-06-10T01:32:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 382448547, 2024-06-10T01:32:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 202911, 2024-06-10T01:32:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-06-10T01:32:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-06-10T01:32:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 56077, 2024-06-10T01:32:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 382448547, 2024-06-10T01:32:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-10T01:32:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:39.657 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-06-10T01:32:40.421 INFO:tasks.check_counter:Daemon mds.c mds.exported=8 2024-06-10T01:32:40.421 INFO:tasks.check_counter:Daemon mds.c mds.imported=8 2024-06-10T01:32:40.421 DEBUG:tasks.check_counter:Getting stats from f 2024-06-10T01:32:40.421 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph daemon mds.f perf dump 2024-06-10T01:32:40.668 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T01:32:41.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:41 smithi175 ceph-mon[36636]: pgmap v1234: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-10T01:32:41.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:41 smithi007 ceph-mon[32398]: pgmap v1234: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-10T01:32:41.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:41 smithi078 ceph-mon[36683]: pgmap v1234: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-10T01:32:42.418 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-06-10T01:32:42.418 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T01:32:42.418 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 660, 2024-06-10T01:32:42.418 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 662, 2024-06-10T01:32:42.418 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 50522521, 2024-06-10T01:32:42.418 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 209890, 2024-06-10T01:32:42.419 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 8, 2024-06-10T01:32:42.419 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 4, 2024-06-10T01:32:42.419 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.093378567, 2024-06-10T01:32:42.419 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.015507365, 2024-06-10T01:32:42.419 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.050473606, 2024-06-10T01:32:42.419 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.021489942, 2024-06-10T01:32:42.419 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:42.419 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 660, 2024-06-10T01:32:42.419 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.040081137, 2024-06-10T01:32:42.419 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000060728 2024-06-10T01:32:42.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.419 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:42.419 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.419 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.420 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.420 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.420 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-10T01:32:42.420 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 0 2024-06-10T01:32:42.420 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.420 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T01:32:42.420 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 361, 2024-06-10T01:32:42.420 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 354, 2024-06-10T01:32:42.420 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 32145880, 2024-06-10T01:32:42.420 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 135589, 2024-06-10T01:32:42.420 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 5, 2024-06-10T01:32:42.420 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 4, 2024-06-10T01:32:42.420 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.050894164, 2024-06-10T01:32:42.420 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.010246130, 2024-06-10T01:32:42.421 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.029798494, 2024-06-10T01:32:42.421 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.009815526, 2024-06-10T01:32:42.421 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:42.421 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 353, 2024-06-10T01:32:42.421 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.025719765, 2024-06-10T01:32:42.421 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000072860 2024-06-10T01:32:42.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.421 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:42.421 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.421 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.421 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.421 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 204208, 2024-06-10T01:32:42.421 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 50944 2024-06-10T01:32:42.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.422 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T01:32:42.422 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 175, 2024-06-10T01:32:42.422 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 235, 2024-06-10T01:32:42.422 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 25213037, 2024-06-10T01:32:42.422 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 249606, 2024-06-10T01:32:42.422 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 6, 2024-06-10T01:32:42.422 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 5, 2024-06-10T01:32:42.422 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.038181042, 2024-06-10T01:32:42.422 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.006497630, 2024-06-10T01:32:42.422 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.024667345, 2024-06-10T01:32:42.422 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.005206907, 2024-06-10T01:32:42.422 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:42.422 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 234, 2024-06-10T01:32:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.025170793, 2024-06-10T01:32:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000107567 2024-06-10T01:32:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-06-10T01:32:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 195536 2024-06-10T01:32:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2024-06-10T01:32:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 1, 2024-06-10T01:32:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2024-06-10T01:32:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-MDSRank": { 2024-06-10T01:32:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-06-10T01:32:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-06-10T01:32:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1828, 2024-06-10T01:32:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.153638716, 2024-06-10T01:32:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000084047 2024-06-10T01:32:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-PurgeQueue": { 2024-06-10T01:32:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-06-10T01:32:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-06-10T01:32:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: "mds": { 2024-06-10T01:32:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: "request": 0, 2024-06-10T01:32:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: "reply": 0, 2024-06-10T01:32:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: "reply_latency": { 2024-06-10T01:32:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_reply": 0, 2024-06-10T01:32:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: "forward": 0, 2024-06-10T01:32:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_complete": 0, 2024-06-10T01:32:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_keys": 0, 2024-06-10T01:32:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_commit": 0, 2024-06-10T01:32:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_split": 0, 2024-06-10T01:32:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_merge": 0, 2024-06-10T01:32:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes": 10759, 2024-06-10T01:32:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_top": 7341, 2024-06-10T01:32:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_bottom": 3418, 2024-06-10T01:32:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pin_tail": 0, 2024-06-10T01:32:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pinned": 860, 2024-06-10T01:32:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_expired": 0, 2024-06-10T01:32:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_with_caps": 0, 2024-06-10T01:32:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: "caps": 0, 2024-06-10T01:32:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: "subtrees": 2, 2024-06-10T01:32:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse": 0, 2024-06-10T01:32:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_hit": 0, 2024-06-10T01:32:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_forward": 0, 2024-06-10T01:32:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_discover": 0, 2024-06-10T01:32:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_dir_fetch": 0, 2024-06-10T01:32:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_remote_ino": 0, 2024-06-10T01:32:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_lock": 0, 2024-06-10T01:32:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: "load_cent": 0, 2024-06-10T01:32:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: "q": 0, 2024-06-10T01:32:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: "exported": 0, 2024-06-10T01:32:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: "exported_inodes": 0, 2024-06-10T01:32:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: "imported": 0, 2024-06-10T01:32:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: "imported_inodes": 0, 2024-06-10T01:32:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_dir_fetch": 0, 2024-06-10T01:32:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_backtrace_fetch": 0, 2024-06-10T01:32:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_peer_discover": 0, 2024-06-10T01:32:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rfiles": 4, 2024-06-10T01:32:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rbytes": 234, 2024-06-10T01:32:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rsnaps": 0, 2024-06-10T01:32:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_fetch": 0, 2024-06-10T01:32:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_set_tag": 0, 2024-06-10T01:32:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T01:32:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_inotable_repaired": 0, 2024-06-10T01:32:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_inodes": 0, 2024-06-10T01:32:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_base_inodes": 0, 2024-06-10T01:32:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dirfrag_rstats": 0, 2024-06-10T01:32:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_file_inodes": 0, 2024-06-10T01:32:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_inode_file_caps": 0, 2024-06-10T01:32:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_revoke": 0, 2024-06-10T01:32:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_grant": 0, 2024-06-10T01:32:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_trunc": 0, 2024-06-10T01:32:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T01:32:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T01:32:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps": 0, 2024-06-10T01:32:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps_dirty": 0, 2024-06-10T01:32:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_cap_release": 0, 2024-06-10T01:32:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: "process_request_cap_release": 0 2024-06-10T01:32:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_cache": { 2024-06-10T01:32:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update": 0, 2024-06-10T01:32:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update_receipt": 0, 2024-06-10T01:32:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_try_discover": 0, 2024-06-10T01:32:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_send_discover": 0, 2024-06-10T01:32:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_handle_discover": 0, 2024-06-10T01:32:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays": 0, 2024-06-10T01:32:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_delayed": 0, 2024-06-10T01:32:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_enqueuing": 0, 2024-06-10T01:32:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_created": 0, 2024-06-10T01:32:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_enqueued": 0, 2024-06-10T01:32:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_reintegrated": 0, 2024-06-10T01:32:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_migrated": 0, 2024-06-10T01:32:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_processing": 0, 2024-06-10T01:32:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_enqueued": 0, 2024-06-10T01:32:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_prioritized": 0, 2024-06-10T01:32:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_started": 0, 2024-06-10T01:32:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_completed": 0, 2024-06-10T01:32:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_path": 0, 2024-06-10T01:32:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_inode": 0, 2024-06-10T01:32:42.431 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_enqueue_scrub": 0, 2024-06-10T01:32:42.431 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_exportdir": 0, 2024-06-10T01:32:42.431 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_flush": 0, 2024-06-10T01:32:42.431 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragmentdir": 0, 2024-06-10T01:32:42.431 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragstats": 0, 2024-06-10T01:32:42.431 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_inodestats": 0 2024-06-10T01:32:42.431 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.431 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_log": { 2024-06-10T01:32:42.431 INFO:teuthology.orchestra.run.smithi175.stdout: "evlrg": 0, 2024-06-10T01:32:42.431 INFO:teuthology.orchestra.run.smithi175.stdout: "evadd": 0, 2024-06-10T01:32:42.431 INFO:teuthology.orchestra.run.smithi175.stdout: "evex": 0, 2024-06-10T01:32:42.431 INFO:teuthology.orchestra.run.smithi175.stdout: "evtrm": 0, 2024-06-10T01:32:42.431 INFO:teuthology.orchestra.run.smithi175.stdout: "ev": 51381, 2024-06-10T01:32:42.431 INFO:teuthology.orchestra.run.smithi175.stdout: "evexg": 0, 2024-06-10T01:32:42.432 INFO:teuthology.orchestra.run.smithi175.stdout: "evexd": 0, 2024-06-10T01:32:42.432 INFO:teuthology.orchestra.run.smithi175.stdout: "segadd": 0, 2024-06-10T01:32:42.432 INFO:teuthology.orchestra.run.smithi175.stdout: "segex": 0, 2024-06-10T01:32:42.432 INFO:teuthology.orchestra.run.smithi175.stdout: "segtrm": 0, 2024-06-10T01:32:42.432 INFO:teuthology.orchestra.run.smithi175.stdout: "seg": 2, 2024-06-10T01:32:42.432 INFO:teuthology.orchestra.run.smithi175.stdout: "segmjr": 17, 2024-06-10T01:32:42.432 INFO:teuthology.orchestra.run.smithi175.stdout: "segexg": 0, 2024-06-10T01:32:42.432 INFO:teuthology.orchestra.run.smithi175.stdout: "segexd": 0, 2024-06-10T01:32:42.432 INFO:teuthology.orchestra.run.smithi175.stdout: "expos": 566688651, 2024-06-10T01:32:42.432 INFO:teuthology.orchestra.run.smithi175.stdout: "wrpos": 568770632, 2024-06-10T01:32:42.432 INFO:teuthology.orchestra.run.smithi175.stdout: "rdpos": 568770033, 2024-06-10T01:32:42.432 INFO:teuthology.orchestra.run.smithi175.stdout: "jlat": { 2024-06-10T01:32:42.432 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.432 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.432 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.433 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.433 INFO:teuthology.orchestra.run.smithi175.stdout: "replayed": 51381 2024-06-10T01:32:42.433 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.433 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_mem": { 2024-06-10T01:32:42.433 INFO:teuthology.orchestra.run.smithi175.stdout: "ino": 22, 2024-06-10T01:32:42.433 INFO:teuthology.orchestra.run.smithi175.stdout: "ino+": 10739, 2024-06-10T01:32:42.433 INFO:teuthology.orchestra.run.smithi175.stdout: "ino-": 10717, 2024-06-10T01:32:42.433 INFO:teuthology.orchestra.run.smithi175.stdout: "dir": 21, 2024-06-10T01:32:42.433 INFO:teuthology.orchestra.run.smithi175.stdout: "dir+": 81, 2024-06-10T01:32:42.433 INFO:teuthology.orchestra.run.smithi175.stdout: "dir-": 60, 2024-06-10T01:32:42.433 INFO:teuthology.orchestra.run.smithi175.stdout: "dn": 10759, 2024-06-10T01:32:42.433 INFO:teuthology.orchestra.run.smithi175.stdout: "dn+": 11349, 2024-06-10T01:32:42.433 INFO:teuthology.orchestra.run.smithi175.stdout: "dn-": 590, 2024-06-10T01:32:42.433 INFO:teuthology.orchestra.run.smithi175.stdout: "cap": 0, 2024-06-10T01:32:42.434 INFO:teuthology.orchestra.run.smithi175.stdout: "cap+": 0, 2024-06-10T01:32:42.434 INFO:teuthology.orchestra.run.smithi175.stdout: "cap-": 0, 2024-06-10T01:32:42.434 INFO:teuthology.orchestra.run.smithi175.stdout: "rss": 64204, 2024-06-10T01:32:42.434 INFO:teuthology.orchestra.run.smithi175.stdout: "heap": 215396 2024-06-10T01:32:42.434 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.434 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_server": { 2024-06-10T01:32:42.434 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_client_request": 0, 2024-06-10T01:32:42.434 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_server_request": 0, 2024-06-10T01:32:42.434 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_request": 0, 2024-06-10T01:32:42.434 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_session": 0, 2024-06-10T01:32:42.434 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_peer_request": 0, 2024-06-10T01:32:42.434 INFO:teuthology.orchestra.run.smithi175.stdout: "req_create_latency": { 2024-06-10T01:32:42.434 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.434 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.435 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.435 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.435 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getattr_latency": { 2024-06-10T01:32:42.435 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.435 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.435 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.435 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.435 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getfilelock_latency": { 2024-06-10T01:32:42.435 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.435 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.435 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.435 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.435 INFO:teuthology.orchestra.run.smithi175.stdout: "req_link_latency": { 2024-06-10T01:32:42.436 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.436 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.436 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.436 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.436 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookup_latency": { 2024-06-10T01:32:42.436 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.436 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.436 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.436 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.436 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookuphash_latency": { 2024-06-10T01:32:42.436 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.436 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.436 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.436 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.437 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupino_latency": { 2024-06-10T01:32:42.437 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.437 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.437 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.437 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.437 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupname_latency": { 2024-06-10T01:32:42.437 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.437 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.437 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.437 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.437 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupparent_latency": { 2024-06-10T01:32:42.437 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.437 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.437 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.437 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.437 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupsnap_latency": { 2024-06-10T01:32:42.438 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.438 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.438 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.438 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lssnap_latency": { 2024-06-10T01:32:42.438 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.438 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.438 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.439 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mkdir_latency": { 2024-06-10T01:32:42.439 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.439 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.439 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.439 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.439 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mknod_latency": { 2024-06-10T01:32:42.439 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.439 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.439 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.439 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.439 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mksnap_latency": { 2024-06-10T01:32:42.439 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.440 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.440 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.440 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.440 INFO:teuthology.orchestra.run.smithi175.stdout: "req_open_latency": { 2024-06-10T01:32:42.440 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.440 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.440 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.440 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.440 INFO:teuthology.orchestra.run.smithi175.stdout: "req_readdir_latency": { 2024-06-10T01:32:42.440 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.440 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.440 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.440 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.440 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rename_latency": { 2024-06-10T01:32:42.440 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.441 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.441 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.441 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.441 INFO:teuthology.orchestra.run.smithi175.stdout: "req_renamesnap_latency": { 2024-06-10T01:32:42.441 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.441 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.441 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.441 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.441 INFO:teuthology.orchestra.run.smithi175.stdout: "req_snapdiff_latency": { 2024-06-10T01:32:42.441 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.441 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.441 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.441 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.441 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmdir_latency": { 2024-06-10T01:32:42.441 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.442 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.442 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.442 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmsnap_latency": { 2024-06-10T01:32:42.442 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.442 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.442 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.442 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmxattr_latency": { 2024-06-10T01:32:42.442 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.442 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.442 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.442 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setattr_latency": { 2024-06-10T01:32:42.443 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.443 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.443 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.443 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setdirlayout_latency": { 2024-06-10T01:32:42.443 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.443 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.443 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.443 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setfilelock_latency": { 2024-06-10T01:32:42.443 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.443 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.443 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.444 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setlayout_latency": { 2024-06-10T01:32:42.444 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.444 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.444 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.444 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.444 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setxattr_latency": { 2024-06-10T01:32:42.444 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.444 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.444 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.444 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.444 INFO:teuthology.orchestra.run.smithi175.stdout: "req_symlink_latency": { 2024-06-10T01:32:42.444 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.444 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.444 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.445 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.445 INFO:teuthology.orchestra.run.smithi175.stdout: "req_unlink_latency": { 2024-06-10T01:32:42.445 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.445 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.445 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.445 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.445 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_revoke_eviction": 0, 2024-06-10T01:32:42.445 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_acquisition_throttle": 0, 2024-06-10T01:32:42.445 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getvxattr_latency": { 2024-06-10T01:32:42.445 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.445 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.445 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.445 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:42.445 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.445 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_sessions": { 2024-06-10T01:32:42.446 INFO:teuthology.orchestra.run.smithi175.stdout: "session_count": 1, 2024-06-10T01:32:42.446 INFO:teuthology.orchestra.run.smithi175.stdout: "session_add": 1, 2024-06-10T01:32:42.446 INFO:teuthology.orchestra.run.smithi175.stdout: "session_remove": 0, 2024-06-10T01:32:42.446 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_open": 0, 2024-06-10T01:32:42.446 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_stale": 0, 2024-06-10T01:32:42.446 INFO:teuthology.orchestra.run.smithi175.stdout: "total_load": 0, 2024-06-10T01:32:42.446 INFO:teuthology.orchestra.run.smithi175.stdout: "average_load": 0, 2024-06-10T01:32:42.446 INFO:teuthology.orchestra.run.smithi175.stdout: "avg_session_uptime": 3062, 2024-06-10T01:32:42.446 INFO:teuthology.orchestra.run.smithi175.stdout: "mdthresh_evicted": 0 2024-06-10T01:32:42.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.446 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2024-06-10T01:32:42.446 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2024-06-10T01:32:42.446 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2024-06-10T01:32:42.446 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T01:32:42.447 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2024-06-10T01:32:42.447 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T01:32:42.447 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2024-06-10T01:32:42.447 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T01:32:42.447 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T01:32:42.447 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T01:32:42.447 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T01:32:42.447 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T01:32:42.447 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2024-06-10T01:32:42.447 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T01:32:42.447 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T01:32:42.447 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_bytes": 0, 2024-06-10T01:32:42.447 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_items": 0, 2024-06-10T01:32:42.447 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_bytes": 0, 2024-06-10T01:32:42.447 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_items": 0, 2024-06-10T01:32:42.448 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T01:32:42.448 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T01:32:42.448 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T01:32:42.448 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T01:32:42.448 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T01:32:42.448 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2024-06-10T01:32:42.448 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2024-06-10T01:32:42.448 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2024-06-10T01:32:42.448 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T01:32:42.448 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T01:32:42.448 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2024-06-10T01:32:42.448 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2024-06-10T01:32:42.448 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2024-06-10T01:32:42.448 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2024-06-10T01:32:42.448 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T01:32:42.449 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2024-06-10T01:32:42.449 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T01:32:42.449 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2024-06-10T01:32:42.449 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 229841, 2024-06-10T01:32:42.449 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 74, 2024-06-10T01:32:42.449 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 0, 2024-06-10T01:32:42.449 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 0, 2024-06-10T01:32:42.449 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2024-06-10T01:32:42.449 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2024-06-10T01:32:42.449 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2024-06-10T01:32:42.449 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2024-06-10T01:32:42.449 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2024-06-10T01:32:42.449 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2024-06-10T01:32:42.449 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 8800, 2024-06-10T01:32:42.449 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 169, 2024-06-10T01:32:42.450 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T01:32:42.450 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2024-06-10T01:32:42.450 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2024-06-10T01:32:42.450 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2024-06-10T01:32:42.450 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 7651615, 2024-06-10T01:32:42.450 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 43769, 2024-06-10T01:32:42.450 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2024-06-10T01:32:42.450 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2024-06-10T01:32:42.450 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2024-06-10T01:32:42.450 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2024-06-10T01:32:42.450 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.450 INFO:teuthology.orchestra.run.smithi175.stdout: "objecter": { 2024-06-10T01:32:42.450 INFO:teuthology.orchestra.run.smithi175.stdout: "op_active": 0, 2024-06-10T01:32:42.450 INFO:teuthology.orchestra.run.smithi175.stdout: "op_laggy": 0, 2024-06-10T01:32:42.451 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send": 1096, 2024-06-10T01:32:42.451 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send_bytes": 26, 2024-06-10T01:32:42.451 INFO:teuthology.orchestra.run.smithi175.stdout: "op_resend": 3, 2024-06-10T01:32:42.451 INFO:teuthology.orchestra.run.smithi175.stdout: "op_reply": 1096, 2024-06-10T01:32:42.451 INFO:teuthology.orchestra.run.smithi175.stdout: "op_latency": { 2024-06-10T01:32:42.451 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1096, 2024-06-10T01:32:42.451 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 8.949873549, 2024-06-10T01:32:42.451 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.008165943 2024-06-10T01:32:42.451 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.451 INFO:teuthology.orchestra.run.smithi175.stdout: "op_inflight": 0, 2024-06-10T01:32:42.451 INFO:teuthology.orchestra.run.smithi175.stdout: "oplen_avg": { 2024-06-10T01:32:42.451 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1096, 2024-06-10T01:32:42.451 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 1097 2024-06-10T01:32:42.451 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.451 INFO:teuthology.orchestra.run.smithi175.stdout: "op": 1096, 2024-06-10T01:32:42.452 INFO:teuthology.orchestra.run.smithi175.stdout: "op_r": 1096, 2024-06-10T01:32:42.452 INFO:teuthology.orchestra.run.smithi175.stdout: "op_w": 0, 2024-06-10T01:32:42.452 INFO:teuthology.orchestra.run.smithi175.stdout: "op_rmw": 0, 2024-06-10T01:32:42.452 INFO:teuthology.orchestra.run.smithi175.stdout: "op_pg": 0, 2024-06-10T01:32:42.452 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_stat": 593, 2024-06-10T01:32:42.452 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_create": 0, 2024-06-10T01:32:42.452 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_read": 500, 2024-06-10T01:32:42.452 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_write": 0, 2024-06-10T01:32:42.452 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writefull": 0, 2024-06-10T01:32:42.452 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writesame": 0, 2024-06-10T01:32:42.452 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_append": 0, 2024-06-10T01:32:42.452 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_zero": 0, 2024-06-10T01:32:42.452 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_truncate": 0, 2024-06-10T01:32:42.452 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_delete": 0, 2024-06-10T01:32:42.453 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_mapext": 0, 2024-06-10T01:32:42.453 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_sparse_read": 0, 2024-06-10T01:32:42.453 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_clonerange": 0, 2024-06-10T01:32:42.453 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_getxattr": 2, 2024-06-10T01:32:42.453 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_setxattr": 0, 2024-06-10T01:32:42.453 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_cmpxattr": 0, 2024-06-10T01:32:42.453 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_rmxattr": 0, 2024-06-10T01:32:42.453 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_resetxattrs": 0, 2024-06-10T01:32:42.453 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_call": 0, 2024-06-10T01:32:42.453 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_watch": 0, 2024-06-10T01:32:42.453 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_notify": 0, 2024-06-10T01:32:42.453 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T01:32:42.453 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls": 0, 2024-06-10T01:32:42.453 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls_filter": 0, 2024-06-10T01:32:42.453 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_other": 0, 2024-06-10T01:32:42.454 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_active": 0, 2024-06-10T01:32:42.454 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_send": 0, 2024-06-10T01:32:42.454 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_resend": 0, 2024-06-10T01:32:42.454 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_ping": 0, 2024-06-10T01:32:42.454 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_active": 0, 2024-06-10T01:32:42.454 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_send": 0, 2024-06-10T01:32:42.454 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_resend": 0, 2024-06-10T01:32:42.454 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_active": 0, 2024-06-10T01:32:42.454 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_send": 0, 2024-06-10T01:32:42.454 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_resend": 0, 2024-06-10T01:32:42.454 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_active": 0, 2024-06-10T01:32:42.454 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_send": 0, 2024-06-10T01:32:42.454 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_resend": 0, 2024-06-10T01:32:42.455 INFO:teuthology.orchestra.run.smithi175.stdout: "command_active": 0, 2024-06-10T01:32:42.455 INFO:teuthology.orchestra.run.smithi175.stdout: "command_send": 0, 2024-06-10T01:32:42.455 INFO:teuthology.orchestra.run.smithi175.stdout: "command_resend": 0, 2024-06-10T01:32:42.455 INFO:teuthology.orchestra.run.smithi175.stdout: "map_epoch": 0, 2024-06-10T01:32:42.455 INFO:teuthology.orchestra.run.smithi175.stdout: "map_full": 0, 2024-06-10T01:32:42.455 INFO:teuthology.orchestra.run.smithi175.stdout: "map_inc": 0, 2024-06-10T01:32:42.455 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_sessions": 10, 2024-06-10T01:32:42.455 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_open": 10, 2024-06-10T01:32:42.455 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_close": 0, 2024-06-10T01:32:42.455 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_laggy": 0, 2024-06-10T01:32:42.455 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_wr": 0, 2024-06-10T01:32:42.455 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_rd": 2, 2024-06-10T01:32:42.455 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_del": 0 2024-06-10T01:32:42.455 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.456 INFO:teuthology.orchestra.run.smithi175.stdout: "oft": { 2024-06-10T01:32:42.456 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_objs": 0, 2024-06-10T01:32:42.456 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_kv_pairs": 0, 2024-06-10T01:32:42.456 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_updates": 0, 2024-06-10T01:32:42.456 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_removes": 0 2024-06-10T01:32:42.456 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.456 INFO:teuthology.orchestra.run.smithi175.stdout: "osd-slow-ops": { 2024-06-10T01:32:42.456 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_ops_count": 0 2024-06-10T01:32:42.456 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.456 INFO:teuthology.orchestra.run.smithi175.stdout: "purge_queue": { 2024-06-10T01:32:42.456 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops": 0, 2024-06-10T01:32:42.456 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops_high_water": 0, 2024-06-10T01:32:42.456 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing": 0, 2024-06-10T01:32:42.456 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_high_water": 0, 2024-06-10T01:32:42.456 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed_ops": 0, 2024-06-10T01:32:42.456 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed": 0, 2024-06-10T01:32:42.457 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_item_in_journal": 0 2024-06-10T01:32:42.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.457 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T01:32:42.457 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-10T01:32:42.457 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-06-10T01:32:42.457 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-10T01:32:42.457 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 1192, 2024-06-10T01:32:42.457 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 107766308, 2024-06-10T01:32:42.457 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:42.457 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 1192, 2024-06-10T01:32:42.457 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-06-10T01:32:42.457 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-06-10T01:32:42.457 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 1192, 2024-06-10T01:32:42.457 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 107766308, 2024-06-10T01:32:42.457 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-10T01:32:42.458 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.458 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.458 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.458 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:42.458 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.458 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_bytes": { 2024-06-10T01:32:42.458 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-10T01:32:42.458 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-06-10T01:32:42.458 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-10T01:32:42.458 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-06-10T01:32:42.458 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-06-10T01:32:42.458 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:42.458 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-06-10T01:32:42.458 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 1096, 2024-06-10T01:32:42.459 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 107370642, 2024-06-10T01:32:42.459 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 1096, 2024-06-10T01:32:42.459 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 107370642, 2024-06-10T01:32:42.459 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-10T01:32:42.459 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.459 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.459 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.459 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:42.459 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.459 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_ops": { 2024-06-10T01:32:42.459 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-10T01:32:42.459 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 1024, 2024-06-10T01:32:42.459 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-10T01:32:42.459 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-06-10T01:32:42.459 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-06-10T01:32:42.460 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:42.460 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-06-10T01:32:42.460 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 1096, 2024-06-10T01:32:42.460 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 1096, 2024-06-10T01:32:42.460 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 1096, 2024-06-10T01:32:42.460 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 1096, 2024-06-10T01:32:42.460 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-10T01:32:42.460 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.460 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.460 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.460 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:42.460 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.460 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle": { 2024-06-10T01:32:42.460 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-10T01:32:42.460 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-06-10T01:32:42.461 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-10T01:32:42.461 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-06-10T01:32:42.461 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-06-10T01:32:42.461 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:42.461 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-06-10T01:32:42.461 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-06-10T01:32:42.461 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-06-10T01:32:42.461 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 0, 2024-06-10T01:32:42.461 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 0, 2024-06-10T01:32:42.461 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-10T01:32:42.461 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.461 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.461 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.461 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:42.461 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:42.462 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle-0x55f57f7de6e0": { 2024-06-10T01:32:42.462 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-10T01:32:42.462 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-06-10T01:32:42.462 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-10T01:32:42.462 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-06-10T01:32:42.462 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-06-10T01:32:42.462 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:42.462 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-06-10T01:32:42.462 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-06-10T01:32:42.462 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-06-10T01:32:42.462 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 0, 2024-06-10T01:32:42.462 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 0, 2024-06-10T01:32:42.462 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-10T01:32:42.463 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:42.463 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:42.463 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:42.463 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:42.463 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:42.463 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-06-10T01:32:43.316 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-06-10T01:32:43.317 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-06-10T01:32:43.317 DEBUG:tasks.check_counter:Getting stats from i 2024-06-10T01:32:43.317 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph daemon mds.i perf dump 2024-06-10T01:32:43.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:43 smithi007 ceph-mon[32398]: pgmap v1235: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-10T01:32:43.556 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T01:32:43.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:43 smithi078 ceph-mon[36683]: pgmap v1235: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-10T01:32:43.678 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:43 smithi175 ceph-mon[36636]: pgmap v1235: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-10T01:32:45.260 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-06-10T01:32:45.260 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T01:32:45.260 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 597779, 2024-06-10T01:32:45.260 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 857760, 2024-06-10T01:32:45.261 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 206591936, 2024-06-10T01:32:45.261 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 922423813, 2024-06-10T01:32:45.261 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 554, 2024-06-10T01:32:45.261 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 5, 2024-06-10T01:32:45.261 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 45.892534924, 2024-06-10T01:32:45.261 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 19.406717712, 2024-06-10T01:32:45.261 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 25.973015280, 2024-06-10T01:32:45.261 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 4.825949096, 2024-06-10T01:32:45.261 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:45.261 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 857755, 2024-06-10T01:32:45.261 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 146.898413804, 2024-06-10T01:32:45.261 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000171259 2024-06-10T01:32:45.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.262 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:45.262 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1118265, 2024-06-10T01:32:45.262 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.228197611, 2024-06-10T01:32:45.262 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000204 2024-06-10T01:32:45.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.262 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 178990352, 2024-06-10T01:32:45.262 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 595168160 2024-06-10T01:32:45.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.262 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T01:32:45.262 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 294177, 2024-06-10T01:32:45.262 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 313376, 2024-06-10T01:32:45.262 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 76881062, 2024-06-10T01:32:45.263 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 287589376, 2024-06-10T01:32:45.263 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 13, 2024-06-10T01:32:45.263 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 5, 2024-06-10T01:32:45.263 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 19.677867958, 2024-06-10T01:32:45.263 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 7.062285667, 2024-06-10T01:32:45.263 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 9.611528013, 2024-06-10T01:32:45.263 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 4.130969099, 2024-06-10T01:32:45.263 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:45.263 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 313378, 2024-06-10T01:32:45.263 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 78.263952470, 2024-06-10T01:32:45.263 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000249742 2024-06-10T01:32:45.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.263 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:45.263 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 427796, 2024-06-10T01:32:45.264 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.084113670, 2024-06-10T01:32:45.264 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000196 2024-06-10T01:32:45.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.264 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 771424, 2024-06-10T01:32:45.264 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 356528 2024-06-10T01:32:45.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.264 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T01:32:45.264 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 201021, 2024-06-10T01:32:45.264 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 229153, 2024-06-10T01:32:45.264 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 49584374, 2024-06-10T01:32:45.264 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 190395138, 2024-06-10T01:32:45.264 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 12, 2024-06-10T01:32:45.265 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 6, 2024-06-10T01:32:45.265 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 16.543962093, 2024-06-10T01:32:45.265 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 6.634668206, 2024-06-10T01:32:45.265 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 6.514080355, 2024-06-10T01:32:45.265 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 3.809649233, 2024-06-10T01:32:45.265 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:45.265 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 229152, 2024-06-10T01:32:45.265 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 61.514590669, 2024-06-10T01:32:45.265 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000268444 2024-06-10T01:32:45.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.265 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:45.265 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 271847, 2024-06-10T01:32:45.265 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.053389780, 2024-06-10T01:32:45.265 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000196 2024-06-10T01:32:45.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.266 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-06-10T01:32:45.266 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 849696 2024-06-10T01:32:45.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.266 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2024-06-10T01:32:45.266 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 1, 2024-06-10T01:32:45.266 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2024-06-10T01:32:45.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.266 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-MDSRank": { 2024-06-10T01:32:45.266 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-06-10T01:32:45.266 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-06-10T01:32:45.266 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 179213, 2024-06-10T01:32:45.266 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 267.535459183, 2024-06-10T01:32:45.266 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.001492835 2024-06-10T01:32:45.267 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:45.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.267 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-PurgeQueue": { 2024-06-10T01:32:45.267 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-06-10T01:32:45.267 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-06-10T01:32:45.267 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 88476, 2024-06-10T01:32:45.267 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 18.539609320, 2024-06-10T01:32:45.267 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000209543 2024-06-10T01:32:45.267 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:45.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.267 INFO:teuthology.orchestra.run.smithi175.stdout: "mds": { 2024-06-10T01:32:45.267 INFO:teuthology.orchestra.run.smithi175.stdout: "request": 350561, 2024-06-10T01:32:45.267 INFO:teuthology.orchestra.run.smithi175.stdout: "reply": 325474, 2024-06-10T01:32:45.267 INFO:teuthology.orchestra.run.smithi175.stdout: "reply_latency": { 2024-06-10T01:32:45.267 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 325474, 2024-06-10T01:32:45.268 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 2803.132700925, 2024-06-10T01:32:45.268 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.008612462 2024-06-10T01:32:45.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.268 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_reply": 0, 2024-06-10T01:32:45.268 INFO:teuthology.orchestra.run.smithi175.stdout: "forward": 25087, 2024-06-10T01:32:45.268 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_complete": 21, 2024-06-10T01:32:45.268 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_keys": 6, 2024-06-10T01:32:45.268 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_commit": 4610, 2024-06-10T01:32:45.268 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_split": 16, 2024-06-10T01:32:45.268 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_merge": 19, 2024-06-10T01:32:45.268 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes": 943, 2024-06-10T01:32:45.268 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_top": 91, 2024-06-10T01:32:45.268 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_bottom": 841, 2024-06-10T01:32:45.268 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pin_tail": 11, 2024-06-10T01:32:45.268 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pinned": 812, 2024-06-10T01:32:45.269 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_expired": 1889, 2024-06-10T01:32:45.269 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_with_caps": 3, 2024-06-10T01:32:45.269 INFO:teuthology.orchestra.run.smithi175.stdout: "caps": 3, 2024-06-10T01:32:45.269 INFO:teuthology.orchestra.run.smithi175.stdout: "subtrees": 4, 2024-06-10T01:32:45.269 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse": 533646, 2024-06-10T01:32:45.269 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_hit": 320744, 2024-06-10T01:32:45.269 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_forward": 25083, 2024-06-10T01:32:45.269 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_discover": 33, 2024-06-10T01:32:45.269 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_dir_fetch": 0, 2024-06-10T01:32:45.269 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_remote_ino": 0, 2024-06-10T01:32:45.269 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_lock": 3594, 2024-06-10T01:32:45.269 INFO:teuthology.orchestra.run.smithi175.stdout: "load_cent": 0, 2024-06-10T01:32:45.269 INFO:teuthology.orchestra.run.smithi175.stdout: "q": 0, 2024-06-10T01:32:45.269 INFO:teuthology.orchestra.run.smithi175.stdout: "exported": 19, 2024-06-10T01:32:45.269 INFO:teuthology.orchestra.run.smithi175.stdout: "exported_inodes": 1223, 2024-06-10T01:32:45.270 INFO:teuthology.orchestra.run.smithi175.stdout: "imported": 21, 2024-06-10T01:32:45.270 INFO:teuthology.orchestra.run.smithi175.stdout: "imported_inodes": 382, 2024-06-10T01:32:45.270 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_dir_fetch": 0, 2024-06-10T01:32:45.270 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_backtrace_fetch": 5303, 2024-06-10T01:32:45.270 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_peer_discover": 0, 2024-06-10T01:32:45.270 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rfiles": 4, 2024-06-10T01:32:45.270 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rbytes": 234, 2024-06-10T01:32:45.270 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rsnaps": 0, 2024-06-10T01:32:45.270 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_fetch": 5303, 2024-06-10T01:32:45.270 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_set_tag": 0, 2024-06-10T01:32:45.270 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T01:32:45.270 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_inotable_repaired": 0, 2024-06-10T01:32:45.270 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_inodes": 1122, 2024-06-10T01:32:45.270 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_base_inodes": 30, 2024-06-10T01:32:45.271 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dirfrag_rstats": 1092, 2024-06-10T01:32:45.271 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_file_inodes": 4181, 2024-06-10T01:32:45.271 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_inode_file_caps": 50, 2024-06-10T01:32:45.271 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_revoke": 42682, 2024-06-10T01:32:45.271 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_grant": 203349, 2024-06-10T01:32:45.271 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_trunc": 0, 2024-06-10T01:32:45.271 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T01:32:45.271 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T01:32:45.271 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps": 113577, 2024-06-10T01:32:45.271 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps_dirty": 38865, 2024-06-10T01:32:45.271 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_cap_release": 16835, 2024-06-10T01:32:45.271 INFO:teuthology.orchestra.run.smithi175.stdout: "process_request_cap_release": 200387 2024-06-10T01:32:45.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.271 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_cache": { 2024-06-10T01:32:45.272 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update": 14, 2024-06-10T01:32:45.272 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update_receipt": 27, 2024-06-10T01:32:45.272 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_try_discover": 18, 2024-06-10T01:32:45.272 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_send_discover": 42, 2024-06-10T01:32:45.272 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_handle_discover": 266, 2024-06-10T01:32:45.272 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays": 5, 2024-06-10T01:32:45.272 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_delayed": 0, 2024-06-10T01:32:45.272 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_enqueuing": 0, 2024-06-10T01:32:45.272 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_created": 51165, 2024-06-10T01:32:45.272 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_enqueued": 51160, 2024-06-10T01:32:45.272 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_reintegrated": 0, 2024-06-10T01:32:45.272 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_migrated": 0, 2024-06-10T01:32:45.272 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_processing": 0, 2024-06-10T01:32:45.272 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_enqueued": 0, 2024-06-10T01:32:45.272 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_prioritized": 0, 2024-06-10T01:32:45.273 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_started": 0, 2024-06-10T01:32:45.273 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_completed": 0, 2024-06-10T01:32:45.273 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_path": 20, 2024-06-10T01:32:45.273 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_inode": 3669, 2024-06-10T01:32:45.273 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_enqueue_scrub": 30, 2024-06-10T01:32:45.273 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_exportdir": 20, 2024-06-10T01:32:45.273 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_flush": 0, 2024-06-10T01:32:45.273 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragmentdir": 35, 2024-06-10T01:32:45.273 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragstats": 0, 2024-06-10T01:32:45.273 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_inodestats": 0 2024-06-10T01:32:45.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.273 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_log": { 2024-06-10T01:32:45.273 INFO:teuthology.orchestra.run.smithi175.stdout: "evlrg": 0, 2024-06-10T01:32:45.273 INFO:teuthology.orchestra.run.smithi175.stdout: "evadd": 268412, 2024-06-10T01:32:45.273 INFO:teuthology.orchestra.run.smithi175.stdout: "evex": 266840, 2024-06-10T01:32:45.274 INFO:teuthology.orchestra.run.smithi175.stdout: "evtrm": 266840, 2024-06-10T01:32:45.274 INFO:teuthology.orchestra.run.smithi175.stdout: "ev": 1572, 2024-06-10T01:32:45.274 INFO:teuthology.orchestra.run.smithi175.stdout: "evexg": 0, 2024-06-10T01:32:45.274 INFO:teuthology.orchestra.run.smithi175.stdout: "evexd": 4071, 2024-06-10T01:32:45.274 INFO:teuthology.orchestra.run.smithi175.stdout: "segadd": 387, 2024-06-10T01:32:45.274 INFO:teuthology.orchestra.run.smithi175.stdout: "segex": 385, 2024-06-10T01:32:45.274 INFO:teuthology.orchestra.run.smithi175.stdout: "segtrm": 385, 2024-06-10T01:32:45.274 INFO:teuthology.orchestra.run.smithi175.stdout: "seg": 2, 2024-06-10T01:32:45.274 INFO:teuthology.orchestra.run.smithi175.stdout: "segmjr": 1, 2024-06-10T01:32:45.274 INFO:teuthology.orchestra.run.smithi175.stdout: "segexg": 0, 2024-06-10T01:32:45.274 INFO:teuthology.orchestra.run.smithi175.stdout: "segexd": 5, 2024-06-10T01:32:45.274 INFO:teuthology.orchestra.run.smithi175.stdout: "expos": 566688651, 2024-06-10T01:32:45.274 INFO:teuthology.orchestra.run.smithi175.stdout: "wrpos": 568770632, 2024-06-10T01:32:45.274 INFO:teuthology.orchestra.run.smithi175.stdout: "rdpos": 0, 2024-06-10T01:32:45.275 INFO:teuthology.orchestra.run.smithi175.stdout: "jlat": { 2024-06-10T01:32:45.275 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 96334, 2024-06-10T01:32:45.275 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 1048.274494729, 2024-06-10T01:32:45.275 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.010881666 2024-06-10T01:32:45.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.275 INFO:teuthology.orchestra.run.smithi175.stdout: "replayed": 0 2024-06-10T01:32:45.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.275 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_mem": { 2024-06-10T01:32:45.275 INFO:teuthology.orchestra.run.smithi175.stdout: "ino": 37, 2024-06-10T01:32:45.275 INFO:teuthology.orchestra.run.smithi175.stdout: "ino+": 51983, 2024-06-10T01:32:45.275 INFO:teuthology.orchestra.run.smithi175.stdout: "ino-": 51946, 2024-06-10T01:32:45.275 INFO:teuthology.orchestra.run.smithi175.stdout: "dir": 46, 2024-06-10T01:32:45.275 INFO:teuthology.orchestra.run.smithi175.stdout: "dir+": 370, 2024-06-10T01:32:45.275 INFO:teuthology.orchestra.run.smithi175.stdout: "dir-": 324, 2024-06-10T01:32:45.276 INFO:teuthology.orchestra.run.smithi175.stdout: "dn": 943, 2024-06-10T01:32:45.276 INFO:teuthology.orchestra.run.smithi175.stdout: "dn+": 111388, 2024-06-10T01:32:45.276 INFO:teuthology.orchestra.run.smithi175.stdout: "dn-": 110445, 2024-06-10T01:32:45.276 INFO:teuthology.orchestra.run.smithi175.stdout: "cap": 3, 2024-06-10T01:32:45.276 INFO:teuthology.orchestra.run.smithi175.stdout: "cap+": 56383, 2024-06-10T01:32:45.276 INFO:teuthology.orchestra.run.smithi175.stdout: "cap-": 56380, 2024-06-10T01:32:45.276 INFO:teuthology.orchestra.run.smithi175.stdout: "rss": 93596, 2024-06-10T01:32:45.276 INFO:teuthology.orchestra.run.smithi175.stdout: "heap": 231804 2024-06-10T01:32:45.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.276 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_server": { 2024-06-10T01:32:45.276 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_client_request": 475573, 2024-06-10T01:32:45.276 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_server_request": 41384, 2024-06-10T01:32:45.276 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_request": 350561, 2024-06-10T01:32:45.276 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_session": 7199, 2024-06-10T01:32:45.276 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_peer_request": 86139, 2024-06-10T01:32:45.277 INFO:teuthology.orchestra.run.smithi175.stdout: "req_create_latency": { 2024-06-10T01:32:45.277 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 51529, 2024-06-10T01:32:45.277 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 880.604029125, 2024-06-10T01:32:45.277 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.017089484 2024-06-10T01:32:45.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.277 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getattr_latency": { 2024-06-10T01:32:45.277 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 37036, 2024-06-10T01:32:45.277 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 314.788204496, 2024-06-10T01:32:45.277 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.008499519 2024-06-10T01:32:45.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.277 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getfilelock_latency": { 2024-06-10T01:32:45.277 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:45.277 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:45.277 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:45.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.278 INFO:teuthology.orchestra.run.smithi175.stdout: "req_link_latency": { 2024-06-10T01:32:45.278 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:45.278 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:45.278 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:45.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.278 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookup_latency": { 2024-06-10T01:32:45.278 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 98204, 2024-06-10T01:32:45.278 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 326.464392077, 2024-06-10T01:32:45.278 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.003324349 2024-06-10T01:32:45.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.278 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookuphash_latency": { 2024-06-10T01:32:45.278 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:45.279 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:45.279 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:45.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.279 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupino_latency": { 2024-06-10T01:32:45.279 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2, 2024-06-10T01:32:45.279 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000284988, 2024-06-10T01:32:45.279 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000142494 2024-06-10T01:32:45.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.279 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupname_latency": { 2024-06-10T01:32:45.279 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:45.279 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:45.279 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:45.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.279 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupparent_latency": { 2024-06-10T01:32:45.279 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:45.280 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:45.280 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:45.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.280 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:45 smithi175 ceph-mon[36636]: pgmap v1236: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:32:45.281 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupsnap_latency": { 2024-06-10T01:32:45.281 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:45.281 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:45.281 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:45.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.281 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lssnap_latency": { 2024-06-10T01:32:45.281 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:45.281 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:45.281 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:45.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.281 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mkdir_latency": { 2024-06-10T01:32:45.281 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 119, 2024-06-10T01:32:45.281 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 4.840022242, 2024-06-10T01:32:45.282 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.040672455 2024-06-10T01:32:45.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.282 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mknod_latency": { 2024-06-10T01:32:45.282 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:45.282 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:45.282 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:45.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.282 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mksnap_latency": { 2024-06-10T01:32:45.282 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:45.282 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:45.282 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:45.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.282 INFO:teuthology.orchestra.run.smithi175.stdout: "req_open_latency": { 2024-06-10T01:32:45.283 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 14, 2024-06-10T01:32:45.283 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.041739436, 2024-06-10T01:32:45.283 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.002981388 2024-06-10T01:32:45.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.283 INFO:teuthology.orchestra.run.smithi175.stdout: "req_readdir_latency": { 2024-06-10T01:32:45.283 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 75295, 2024-06-10T01:32:45.283 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 742.081424740, 2024-06-10T01:32:45.283 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.009855653 2024-06-10T01:32:45.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.283 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rename_latency": { 2024-06-10T01:32:45.283 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 10282, 2024-06-10T01:32:45.283 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 67.414782482, 2024-06-10T01:32:45.283 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.006556582 2024-06-10T01:32:45.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.284 INFO:teuthology.orchestra.run.smithi175.stdout: "req_renamesnap_latency": { 2024-06-10T01:32:45.284 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:45.284 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:45.284 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:45.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.284 INFO:teuthology.orchestra.run.smithi175.stdout: "req_snapdiff_latency": { 2024-06-10T01:32:45.284 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:45.284 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:45.284 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:45.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.284 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmdir_latency": { 2024-06-10T01:32:45.284 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 64, 2024-06-10T01:32:45.284 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 1.246601383, 2024-06-10T01:32:45.285 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.019478146 2024-06-10T01:32:45.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.285 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmsnap_latency": { 2024-06-10T01:32:45.285 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:45.285 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:45.285 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:45.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.285 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmxattr_latency": { 2024-06-10T01:32:45.285 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:45.285 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:45.285 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:45.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.285 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setattr_latency": { 2024-06-10T01:32:45.285 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 169, 2024-06-10T01:32:45.285 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 43.942958235, 2024-06-10T01:32:45.286 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.260017504 2024-06-10T01:32:45.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.286 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setdirlayout_latency": { 2024-06-10T01:32:45.286 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:45.286 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:45.286 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:45.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.286 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setfilelock_latency": { 2024-06-10T01:32:45.286 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1640, 2024-06-10T01:32:45.286 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 4.343389183, 2024-06-10T01:32:45.286 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.002648408 2024-06-10T01:32:45.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.286 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setlayout_latency": { 2024-06-10T01:32:45.287 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:45.287 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:45.287 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:45.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.287 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setxattr_latency": { 2024-06-10T01:32:45.287 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 8, 2024-06-10T01:32:45.287 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.022320209, 2024-06-10T01:32:45.287 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.002790026 2024-06-10T01:32:45.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.287 INFO:teuthology.orchestra.run.smithi175.stdout: "req_symlink_latency": { 2024-06-10T01:32:45.287 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:45.287 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:45.287 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:45.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.288 INFO:teuthology.orchestra.run.smithi175.stdout: "req_unlink_latency": { 2024-06-10T01:32:45.288 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 51112, 2024-06-10T01:32:45.288 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 417.342552329, 2024-06-10T01:32:45.288 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.008165255 2024-06-10T01:32:45.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.288 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_revoke_eviction": 0, 2024-06-10T01:32:45.288 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_acquisition_throttle": 0, 2024-06-10T01:32:45.288 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getvxattr_latency": { 2024-06-10T01:32:45.288 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:45.288 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:45.288 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:45.288 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:45.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.288 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_sessions": { 2024-06-10T01:32:45.289 INFO:teuthology.orchestra.run.smithi175.stdout: "session_count": 1, 2024-06-10T01:32:45.289 INFO:teuthology.orchestra.run.smithi175.stdout: "session_add": 3, 2024-06-10T01:32:45.289 INFO:teuthology.orchestra.run.smithi175.stdout: "session_remove": 2, 2024-06-10T01:32:45.289 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_open": 1, 2024-06-10T01:32:45.289 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_stale": 0, 2024-06-10T01:32:45.289 INFO:teuthology.orchestra.run.smithi175.stdout: "total_load": 23875, 2024-06-10T01:32:45.289 INFO:teuthology.orchestra.run.smithi175.stdout: "average_load": 23875, 2024-06-10T01:32:45.289 INFO:teuthology.orchestra.run.smithi175.stdout: "avg_session_uptime": 1624, 2024-06-10T01:32:45.289 INFO:teuthology.orchestra.run.smithi175.stdout: "mdthresh_evicted": 0 2024-06-10T01:32:45.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.289 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2024-06-10T01:32:45.289 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2024-06-10T01:32:45.290 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2024-06-10T01:32:45.290 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T01:32:45.290 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2024-06-10T01:32:45.290 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T01:32:45.290 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2024-06-10T01:32:45.290 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T01:32:45.290 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T01:32:45.290 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T01:32:45.290 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T01:32:45.290 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T01:32:45.290 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2024-06-10T01:32:45.290 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T01:32:45.290 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T01:32:45.291 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_bytes": 0, 2024-06-10T01:32:45.291 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_items": 0, 2024-06-10T01:32:45.291 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_bytes": 0, 2024-06-10T01:32:45.291 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_items": 0, 2024-06-10T01:32:45.291 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T01:32:45.291 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T01:32:45.291 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T01:32:45.291 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T01:32:45.291 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T01:32:45.291 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2024-06-10T01:32:45.291 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2024-06-10T01:32:45.291 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2024-06-10T01:32:45.291 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T01:32:45.292 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T01:32:45.292 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2024-06-10T01:32:45.292 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2024-06-10T01:32:45.292 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2024-06-10T01:32:45.292 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2024-06-10T01:32:45.292 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T01:32:45.292 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2024-06-10T01:32:45.292 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T01:32:45.292 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2024-06-10T01:32:45.292 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 297328, 2024-06-10T01:32:45.292 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 110, 2024-06-10T01:32:45.292 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 88, 2024-06-10T01:32:45.292 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 1, 2024-06-10T01:32:45.292 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2024-06-10T01:32:45.293 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2024-06-10T01:32:45.293 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2024-06-10T01:32:45.293 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2024-06-10T01:32:45.293 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2024-06-10T01:32:45.293 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2024-06-10T01:32:45.293 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 8800, 2024-06-10T01:32:45.293 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 169, 2024-06-10T01:32:45.293 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T01:32:45.293 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2024-06-10T01:32:45.293 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2024-06-10T01:32:45.293 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2024-06-10T01:32:45.293 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 847092, 2024-06-10T01:32:45.293 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 9229, 2024-06-10T01:32:45.293 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2024-06-10T01:32:45.294 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2024-06-10T01:32:45.294 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2024-06-10T01:32:45.294 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2024-06-10T01:32:45.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.294 INFO:teuthology.orchestra.run.smithi175.stdout: "objecter": { 2024-06-10T01:32:45.294 INFO:teuthology.orchestra.run.smithi175.stdout: "op_active": 0, 2024-06-10T01:32:45.294 INFO:teuthology.orchestra.run.smithi175.stdout: "op_laggy": 0, 2024-06-10T01:32:45.294 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send": 261767, 2024-06-10T01:32:45.294 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send_bytes": 611791022, 2024-06-10T01:32:45.294 INFO:teuthology.orchestra.run.smithi175.stdout: "op_resend": 37, 2024-06-10T01:32:45.294 INFO:teuthology.orchestra.run.smithi175.stdout: "op_reply": 261767, 2024-06-10T01:32:45.294 INFO:teuthology.orchestra.run.smithi175.stdout: "op_latency": { 2024-06-10T01:32:45.294 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 261767, 2024-06-10T01:32:45.294 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 12848.703649659, 2024-06-10T01:32:45.295 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.049084505 2024-06-10T01:32:45.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.295 INFO:teuthology.orchestra.run.smithi175.stdout: "op_inflight": 0, 2024-06-10T01:32:45.295 INFO:teuthology.orchestra.run.smithi175.stdout: "oplen_avg": { 2024-06-10T01:32:45.295 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 261767, 2024-06-10T01:32:45.295 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 345071 2024-06-10T01:32:45.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.295 INFO:teuthology.orchestra.run.smithi175.stdout: "op": 261767, 2024-06-10T01:32:45.295 INFO:teuthology.orchestra.run.smithi175.stdout: "op_r": 6952, 2024-06-10T01:32:45.295 INFO:teuthology.orchestra.run.smithi175.stdout: "op_w": 254815, 2024-06-10T01:32:45.295 INFO:teuthology.orchestra.run.smithi175.stdout: "op_rmw": 0, 2024-06-10T01:32:45.295 INFO:teuthology.orchestra.run.smithi175.stdout: "op_pg": 0, 2024-06-10T01:32:45.295 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_stat": 4332, 2024-06-10T01:32:45.295 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_create": 46563, 2024-06-10T01:32:45.295 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_read": 1592, 2024-06-10T01:32:45.296 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_write": 97896, 2024-06-10T01:32:45.296 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writefull": 1224, 2024-06-10T01:32:45.296 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writesame": 0, 2024-06-10T01:32:45.296 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_append": 0, 2024-06-10T01:32:45.296 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_zero": 0, 2024-06-10T01:32:45.296 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_truncate": 16, 2024-06-10T01:32:45.296 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_delete": 103185, 2024-06-10T01:32:45.296 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_mapext": 0, 2024-06-10T01:32:45.296 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_sparse_read": 0, 2024-06-10T01:32:45.296 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_clonerange": 0, 2024-06-10T01:32:45.296 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_getxattr": 5360, 2024-06-10T01:32:45.296 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_setxattr": 70648, 2024-06-10T01:32:45.296 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_cmpxattr": 0, 2024-06-10T01:32:45.296 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_rmxattr": 0, 2024-06-10T01:32:45.296 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_resetxattrs": 0, 2024-06-10T01:32:45.297 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_call": 0, 2024-06-10T01:32:45.297 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_watch": 0, 2024-06-10T01:32:45.297 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_notify": 0, 2024-06-10T01:32:45.297 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T01:32:45.297 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls": 0, 2024-06-10T01:32:45.297 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls_filter": 0, 2024-06-10T01:32:45.297 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_other": 0, 2024-06-10T01:32:45.297 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_active": 0, 2024-06-10T01:32:45.297 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_send": 0, 2024-06-10T01:32:45.297 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_resend": 0, 2024-06-10T01:32:45.297 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_ping": 0, 2024-06-10T01:32:45.297 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_active": 0, 2024-06-10T01:32:45.297 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_send": 0, 2024-06-10T01:32:45.297 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_resend": 0, 2024-06-10T01:32:45.297 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_active": 0, 2024-06-10T01:32:45.298 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_send": 0, 2024-06-10T01:32:45.298 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_resend": 0, 2024-06-10T01:32:45.298 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_active": 0, 2024-06-10T01:32:45.298 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_send": 0, 2024-06-10T01:32:45.298 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_resend": 0, 2024-06-10T01:32:45.298 INFO:teuthology.orchestra.run.smithi175.stdout: "command_active": 0, 2024-06-10T01:32:45.298 INFO:teuthology.orchestra.run.smithi175.stdout: "command_send": 0, 2024-06-10T01:32:45.298 INFO:teuthology.orchestra.run.smithi175.stdout: "command_resend": 0, 2024-06-10T01:32:45.298 INFO:teuthology.orchestra.run.smithi175.stdout: "map_epoch": 102, 2024-06-10T01:32:45.298 INFO:teuthology.orchestra.run.smithi175.stdout: "map_full": 0, 2024-06-10T01:32:45.298 INFO:teuthology.orchestra.run.smithi175.stdout: "map_inc": 14, 2024-06-10T01:32:45.298 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_sessions": 12, 2024-06-10T01:32:45.298 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_open": 12, 2024-06-10T01:32:45.298 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_close": 0, 2024-06-10T01:32:45.298 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_laggy": 0, 2024-06-10T01:32:45.299 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_wr": 10987, 2024-06-10T01:32:45.299 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_rd": 54, 2024-06-10T01:32:45.299 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_del": 3214 2024-06-10T01:32:45.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.299 INFO:teuthology.orchestra.run.smithi175.stdout: "oft": { 2024-06-10T01:32:45.299 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_objs": 1, 2024-06-10T01:32:45.299 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_kv_pairs": 0, 2024-06-10T01:32:45.299 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_updates": 0, 2024-06-10T01:32:45.299 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_removes": 0 2024-06-10T01:32:45.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.299 INFO:teuthology.orchestra.run.smithi175.stdout: "osd-slow-ops": { 2024-06-10T01:32:45.299 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_ops_count": 3675 2024-06-10T01:32:45.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.299 INFO:teuthology.orchestra.run.smithi175.stdout: "purge_queue": { 2024-06-10T01:32:45.299 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops": 0, 2024-06-10T01:32:45.300 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops_high_water": 17, 2024-06-10T01:32:45.300 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing": 0, 2024-06-10T01:32:45.300 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_high_water": 13, 2024-06-10T01:32:45.300 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed_ops": 102272, 2024-06-10T01:32:45.300 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed": 51160, 2024-06-10T01:32:45.300 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_item_in_journal": 0 2024-06-10T01:32:45.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.300 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T01:32:45.300 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-10T01:32:45.300 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-06-10T01:32:45.300 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-10T01:32:45.300 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 1092973, 2024-06-10T01:32:45.300 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 211892894, 2024-06-10T01:32:45.300 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:45.300 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 1092973, 2024-06-10T01:32:45.301 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-06-10T01:32:45.301 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-06-10T01:32:45.301 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 1092973, 2024-06-10T01:32:45.301 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 211892894, 2024-06-10T01:32:45.301 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-10T01:32:45.301 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:45.301 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:45.301 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:45.301 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:45.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.301 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_bytes": { 2024-06-10T01:32:45.301 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-10T01:32:45.302 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-06-10T01:32:45.302 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-10T01:32:45.302 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-06-10T01:32:45.302 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-06-10T01:32:45.302 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:45.302 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-06-10T01:32:45.302 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 261767, 2024-06-10T01:32:45.302 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 617366718, 2024-06-10T01:32:45.302 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 261767, 2024-06-10T01:32:45.302 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 617366718, 2024-06-10T01:32:45.302 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-10T01:32:45.302 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:45.302 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:45.302 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:45.303 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:45.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.303 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_ops": { 2024-06-10T01:32:45.303 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-10T01:32:45.303 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 1024, 2024-06-10T01:32:45.303 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-10T01:32:45.303 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-06-10T01:32:45.303 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-06-10T01:32:45.303 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:45.303 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-06-10T01:32:45.303 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 261767, 2024-06-10T01:32:45.303 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 261767, 2024-06-10T01:32:45.303 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 261767, 2024-06-10T01:32:45.303 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 261767, 2024-06-10T01:32:45.304 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-10T01:32:45.304 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:45.304 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:45.304 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:45.304 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:45.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.304 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle": { 2024-06-10T01:32:45.304 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-10T01:32:45.304 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-06-10T01:32:45.304 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-10T01:32:45.304 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 51160, 2024-06-10T01:32:45.304 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 5576056, 2024-06-10T01:32:45.304 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:45.304 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 51160, 2024-06-10T01:32:45.304 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-06-10T01:32:45.304 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-06-10T01:32:45.305 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 1562, 2024-06-10T01:32:45.305 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 5576056, 2024-06-10T01:32:45.305 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-10T01:32:45.305 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:45.305 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:45.305 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:45.305 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:45.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:45.305 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle-0x55e19b4a2460": { 2024-06-10T01:32:45.305 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-10T01:32:45.305 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-06-10T01:32:45.305 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-10T01:32:45.305 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 268412, 2024-06-10T01:32:45.305 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 564576328, 2024-06-10T01:32:45.305 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:45.306 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 268412, 2024-06-10T01:32:45.306 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-06-10T01:32:45.306 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-06-10T01:32:45.306 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 96334, 2024-06-10T01:32:45.306 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 564576328, 2024-06-10T01:32:45.306 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-10T01:32:45.306 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:45.306 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:45.306 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:45.306 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:45.306 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:45.306 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-06-10T01:32:45.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:45 smithi007 ceph-mon[32398]: pgmap v1236: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:32:45.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:45 smithi078 ceph-mon[36683]: pgmap v1236: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:32:46.270 INFO:tasks.check_counter:Daemon mds.i mds.exported=19 2024-06-10T01:32:46.270 INFO:tasks.check_counter:Daemon mds.i mds.imported=21 2024-06-10T01:32:46.271 DEBUG:tasks.check_counter:Getting stats from l 2024-06-10T01:32:46.271 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph daemon mds.l perf dump 2024-06-10T01:32:46.513 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T01:32:47.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:47 smithi007 ceph-mon[32398]: pgmap v1237: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:32:47.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:47 smithi078 ceph-mon[36683]: pgmap v1237: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:32:47.687 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:47 smithi175 ceph-mon[36636]: pgmap v1237: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:32:48.271 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-06-10T01:32:48.272 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T01:32:48.272 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 1557, 2024-06-10T01:32:48.272 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 1559, 2024-06-10T01:32:48.272 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 143007361, 2024-06-10T01:32:48.272 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 495190, 2024-06-10T01:32:48.272 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 9, 2024-06-10T01:32:48.272 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 4, 2024-06-10T01:32:48.272 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.205731880, 2024-06-10T01:32:48.272 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.035967233, 2024-06-10T01:32:48.272 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.117647543, 2024-06-10T01:32:48.272 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.040622875, 2024-06-10T01:32:48.273 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:48.273 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1557, 2024-06-10T01:32:48.273 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.066693124, 2024-06-10T01:32:48.273 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000042834 2024-06-10T01:32:48.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.273 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:48.273 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.273 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.273 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.273 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-10T01:32:48.273 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 0 2024-06-10T01:32:48.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.273 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T01:32:48.274 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 1721, 2024-06-10T01:32:48.274 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 2106, 2024-06-10T01:32:48.274 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 143053418, 2024-06-10T01:32:48.274 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 1369335, 2024-06-10T01:32:48.274 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 8, 2024-06-10T01:32:48.274 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 6, 2024-06-10T01:32:48.274 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.218751096, 2024-06-10T01:32:48.274 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.047493496, 2024-06-10T01:32:48.274 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.115080292, 2024-06-10T01:32:48.274 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.043230075, 2024-06-10T01:32:48.274 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:48.275 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2104, 2024-06-10T01:32:48.275 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.098520739, 2024-06-10T01:32:48.275 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000046825 2024-06-10T01:32:48.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.275 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:48.275 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.275 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.275 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.275 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-10T01:32:48.275 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 822688 2024-06-10T01:32:48.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.275 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T01:32:48.275 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 3665, 2024-06-10T01:32:48.276 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 3624, 2024-06-10T01:32:48.276 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 138101997, 2024-06-10T01:32:48.276 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 1303274, 2024-06-10T01:32:48.276 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 5, 2024-06-10T01:32:48.276 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 4, 2024-06-10T01:32:48.276 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.392222157, 2024-06-10T01:32:48.276 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.101407205, 2024-06-10T01:32:48.276 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.165909398, 2024-06-10T01:32:48.276 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.100441673, 2024-06-10T01:32:48.276 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:32:48.276 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 3624, 2024-06-10T01:32:48.276 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.174329233, 2024-06-10T01:32:48.276 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000048104 2024-06-10T01:32:48.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.277 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:32:48.277 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.277 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.277 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.277 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 735520, 2024-06-10T01:32:48.277 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 315248 2024-06-10T01:32:48.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.277 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2024-06-10T01:32:48.277 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 1, 2024-06-10T01:32:48.277 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2024-06-10T01:32:48.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.277 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-MDSRank": { 2024-06-10T01:32:48.277 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-06-10T01:32:48.278 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-06-10T01:32:48.278 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 10800, 2024-06-10T01:32:48.278 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.306129471, 2024-06-10T01:32:48.278 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000028345 2024-06-10T01:32:48.278 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:48.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.278 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-PurgeQueue": { 2024-06-10T01:32:48.278 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-06-10T01:32:48.278 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-06-10T01:32:48.278 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.278 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.279 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.279 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:48.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.279 INFO:teuthology.orchestra.run.smithi175.stdout: "mds": { 2024-06-10T01:32:48.279 INFO:teuthology.orchestra.run.smithi175.stdout: "request": 0, 2024-06-10T01:32:48.279 INFO:teuthology.orchestra.run.smithi175.stdout: "reply": 0, 2024-06-10T01:32:48.279 INFO:teuthology.orchestra.run.smithi175.stdout: "reply_latency": { 2024-06-10T01:32:48.279 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.279 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.279 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.279 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_reply": 0, 2024-06-10T01:32:48.279 INFO:teuthology.orchestra.run.smithi175.stdout: "forward": 0, 2024-06-10T01:32:48.280 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_complete": 0, 2024-06-10T01:32:48.280 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_keys": 0, 2024-06-10T01:32:48.280 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_commit": 0, 2024-06-10T01:32:48.280 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_split": 0, 2024-06-10T01:32:48.280 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_merge": 0, 2024-06-10T01:32:48.280 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes": 45169, 2024-06-10T01:32:48.280 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_top": 14672, 2024-06-10T01:32:48.280 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_bottom": 30497, 2024-06-10T01:32:48.280 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pin_tail": 0, 2024-06-10T01:32:48.280 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pinned": 24208, 2024-06-10T01:32:48.280 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_expired": 1, 2024-06-10T01:32:48.280 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_with_caps": 0, 2024-06-10T01:32:48.280 INFO:teuthology.orchestra.run.smithi175.stdout: "caps": 0, 2024-06-10T01:32:48.280 INFO:teuthology.orchestra.run.smithi175.stdout: "subtrees": 1, 2024-06-10T01:32:48.280 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse": 0, 2024-06-10T01:32:48.281 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_hit": 0, 2024-06-10T01:32:48.281 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_forward": 0, 2024-06-10T01:32:48.281 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_discover": 0, 2024-06-10T01:32:48.281 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_dir_fetch": 0, 2024-06-10T01:32:48.281 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_remote_ino": 0, 2024-06-10T01:32:48.281 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_lock": 0, 2024-06-10T01:32:48.281 INFO:teuthology.orchestra.run.smithi175.stdout: "load_cent": 0, 2024-06-10T01:32:48.281 INFO:teuthology.orchestra.run.smithi175.stdout: "q": 0, 2024-06-10T01:32:48.281 INFO:teuthology.orchestra.run.smithi175.stdout: "exported": 0, 2024-06-10T01:32:48.281 INFO:teuthology.orchestra.run.smithi175.stdout: "exported_inodes": 0, 2024-06-10T01:32:48.281 INFO:teuthology.orchestra.run.smithi175.stdout: "imported": 0, 2024-06-10T01:32:48.281 INFO:teuthology.orchestra.run.smithi175.stdout: "imported_inodes": 0, 2024-06-10T01:32:48.281 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_dir_fetch": 0, 2024-06-10T01:32:48.282 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_backtrace_fetch": 0, 2024-06-10T01:32:48.282 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_peer_discover": 0, 2024-06-10T01:32:48.282 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rfiles": 0, 2024-06-10T01:32:48.282 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rbytes": 0, 2024-06-10T01:32:48.282 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rsnaps": 0, 2024-06-10T01:32:48.282 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_fetch": 0, 2024-06-10T01:32:48.282 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_set_tag": 0, 2024-06-10T01:32:48.282 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T01:32:48.282 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_inotable_repaired": 0, 2024-06-10T01:32:48.282 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_inodes": 0, 2024-06-10T01:32:48.282 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_base_inodes": 0, 2024-06-10T01:32:48.282 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dirfrag_rstats": 0, 2024-06-10T01:32:48.283 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_file_inodes": 0, 2024-06-10T01:32:48.283 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_inode_file_caps": 0, 2024-06-10T01:32:48.283 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_revoke": 0, 2024-06-10T01:32:48.283 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_grant": 0, 2024-06-10T01:32:48.283 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_trunc": 0, 2024-06-10T01:32:48.283 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T01:32:48.283 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T01:32:48.283 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps": 0, 2024-06-10T01:32:48.283 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps_dirty": 0, 2024-06-10T01:32:48.283 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_cap_release": 0, 2024-06-10T01:32:48.283 INFO:teuthology.orchestra.run.smithi175.stdout: "process_request_cap_release": 0 2024-06-10T01:32:48.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.283 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_cache": { 2024-06-10T01:32:48.283 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update": 0, 2024-06-10T01:32:48.283 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update_receipt": 0, 2024-06-10T01:32:48.284 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_try_discover": 0, 2024-06-10T01:32:48.284 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_send_discover": 0, 2024-06-10T01:32:48.284 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_handle_discover": 0, 2024-06-10T01:32:48.284 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays": 0, 2024-06-10T01:32:48.284 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_delayed": 0, 2024-06-10T01:32:48.284 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_enqueuing": 0, 2024-06-10T01:32:48.284 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_created": 0, 2024-06-10T01:32:48.284 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_enqueued": 0, 2024-06-10T01:32:48.284 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_reintegrated": 0, 2024-06-10T01:32:48.284 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_migrated": 0, 2024-06-10T01:32:48.284 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_processing": 0, 2024-06-10T01:32:48.284 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_enqueued": 0, 2024-06-10T01:32:48.284 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_prioritized": 0, 2024-06-10T01:32:48.284 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_started": 0, 2024-06-10T01:32:48.284 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_completed": 0, 2024-06-10T01:32:48.285 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_path": 0, 2024-06-10T01:32:48.285 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_inode": 0, 2024-06-10T01:32:48.285 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_enqueue_scrub": 0, 2024-06-10T01:32:48.285 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_exportdir": 0, 2024-06-10T01:32:48.285 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_flush": 0, 2024-06-10T01:32:48.285 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragmentdir": 0, 2024-06-10T01:32:48.285 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragstats": 0, 2024-06-10T01:32:48.285 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_inodestats": 0 2024-06-10T01:32:48.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.285 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_log": { 2024-06-10T01:32:48.285 INFO:teuthology.orchestra.run.smithi175.stdout: "evlrg": 0, 2024-06-10T01:32:48.285 INFO:teuthology.orchestra.run.smithi175.stdout: "evadd": 0, 2024-06-10T01:32:48.285 INFO:teuthology.orchestra.run.smithi175.stdout: "evex": 0, 2024-06-10T01:32:48.285 INFO:teuthology.orchestra.run.smithi175.stdout: "evtrm": 0, 2024-06-10T01:32:48.286 INFO:teuthology.orchestra.run.smithi175.stdout: "ev": 227780, 2024-06-10T01:32:48.286 INFO:teuthology.orchestra.run.smithi175.stdout: "evexg": 0, 2024-06-10T01:32:48.286 INFO:teuthology.orchestra.run.smithi175.stdout: "evexd": 0, 2024-06-10T01:32:48.286 INFO:teuthology.orchestra.run.smithi175.stdout: "segadd": 0, 2024-06-10T01:32:48.286 INFO:teuthology.orchestra.run.smithi175.stdout: "segex": 0, 2024-06-10T01:32:48.286 INFO:teuthology.orchestra.run.smithi175.stdout: "segtrm": 0, 2024-06-10T01:32:48.286 INFO:teuthology.orchestra.run.smithi175.stdout: "seg": 141, 2024-06-10T01:32:48.286 INFO:teuthology.orchestra.run.smithi175.stdout: "segmjr": 20, 2024-06-10T01:32:48.286 INFO:teuthology.orchestra.run.smithi175.stdout: "segexg": 0, 2024-06-10T01:32:48.286 INFO:teuthology.orchestra.run.smithi175.stdout: "segexd": 0, 2024-06-10T01:32:48.286 INFO:teuthology.orchestra.run.smithi175.stdout: "expos": 182080609, 2024-06-10T01:32:48.286 INFO:teuthology.orchestra.run.smithi175.stdout: "wrpos": 425834674, 2024-06-10T01:32:48.286 INFO:teuthology.orchestra.run.smithi175.stdout: "rdpos": 425833536, 2024-06-10T01:32:48.286 INFO:teuthology.orchestra.run.smithi175.stdout: "jlat": { 2024-06-10T01:32:48.287 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.287 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.287 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.287 INFO:teuthology.orchestra.run.smithi175.stdout: "replayed": 227780 2024-06-10T01:32:48.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.287 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_mem": { 2024-06-10T01:32:48.287 INFO:teuthology.orchestra.run.smithi175.stdout: "ino": 14, 2024-06-10T01:32:48.287 INFO:teuthology.orchestra.run.smithi175.stdout: "ino+": 45309, 2024-06-10T01:32:48.287 INFO:teuthology.orchestra.run.smithi175.stdout: "ino-": 45295, 2024-06-10T01:32:48.287 INFO:teuthology.orchestra.run.smithi175.stdout: "dir": 33, 2024-06-10T01:32:48.287 INFO:teuthology.orchestra.run.smithi175.stdout: "dir+": 210, 2024-06-10T01:32:48.288 INFO:teuthology.orchestra.run.smithi175.stdout: "dir-": 177, 2024-06-10T01:32:48.288 INFO:teuthology.orchestra.run.smithi175.stdout: "dn": 45169, 2024-06-10T01:32:48.288 INFO:teuthology.orchestra.run.smithi175.stdout: "dn+": 45944, 2024-06-10T01:32:48.288 INFO:teuthology.orchestra.run.smithi175.stdout: "dn-": 775, 2024-06-10T01:32:48.288 INFO:teuthology.orchestra.run.smithi175.stdout: "cap": 0, 2024-06-10T01:32:48.288 INFO:teuthology.orchestra.run.smithi175.stdout: "cap+": 0, 2024-06-10T01:32:48.288 INFO:teuthology.orchestra.run.smithi175.stdout: "cap-": 0, 2024-06-10T01:32:48.288 INFO:teuthology.orchestra.run.smithi175.stdout: "rss": 94088, 2024-06-10T01:32:48.288 INFO:teuthology.orchestra.run.smithi175.stdout: "heap": 215420 2024-06-10T01:32:48.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.288 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_server": { 2024-06-10T01:32:48.288 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_client_request": 0, 2024-06-10T01:32:48.288 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_server_request": 0, 2024-06-10T01:32:48.289 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_request": 0, 2024-06-10T01:32:48.289 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_session": 0, 2024-06-10T01:32:48.289 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_peer_request": 0, 2024-06-10T01:32:48.289 INFO:teuthology.orchestra.run.smithi175.stdout: "req_create_latency": { 2024-06-10T01:32:48.289 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.289 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.289 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.289 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getattr_latency": { 2024-06-10T01:32:48.289 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.289 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.289 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.289 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getfilelock_latency": { 2024-06-10T01:32:48.290 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.290 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.290 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.290 INFO:teuthology.orchestra.run.smithi175.stdout: "req_link_latency": { 2024-06-10T01:32:48.290 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.290 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.290 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.290 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookup_latency": { 2024-06-10T01:32:48.290 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.290 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.290 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.291 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookuphash_latency": { 2024-06-10T01:32:48.291 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.291 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.291 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.291 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupino_latency": { 2024-06-10T01:32:48.291 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.291 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.291 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.291 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupname_latency": { 2024-06-10T01:32:48.291 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.291 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.291 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.292 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupparent_latency": { 2024-06-10T01:32:48.292 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.292 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.292 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.292 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupsnap_latency": { 2024-06-10T01:32:48.292 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.292 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.292 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.292 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lssnap_latency": { 2024-06-10T01:32:48.292 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.293 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.293 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.293 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mkdir_latency": { 2024-06-10T01:32:48.293 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.293 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.294 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.294 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mknod_latency": { 2024-06-10T01:32:48.294 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.294 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.294 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.294 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mksnap_latency": { 2024-06-10T01:32:48.294 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.294 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.294 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.295 INFO:teuthology.orchestra.run.smithi175.stdout: "req_open_latency": { 2024-06-10T01:32:48.295 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.295 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.295 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.295 INFO:teuthology.orchestra.run.smithi175.stdout: "req_readdir_latency": { 2024-06-10T01:32:48.295 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.295 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.295 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.295 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rename_latency": { 2024-06-10T01:32:48.295 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.295 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.296 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.296 INFO:teuthology.orchestra.run.smithi175.stdout: "req_renamesnap_latency": { 2024-06-10T01:32:48.296 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.296 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.296 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.296 INFO:teuthology.orchestra.run.smithi175.stdout: "req_snapdiff_latency": { 2024-06-10T01:32:48.296 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.296 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.296 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.296 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmdir_latency": { 2024-06-10T01:32:48.297 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.297 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.297 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.297 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmsnap_latency": { 2024-06-10T01:32:48.297 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.297 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.297 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.297 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmxattr_latency": { 2024-06-10T01:32:48.297 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.297 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.297 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.298 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setattr_latency": { 2024-06-10T01:32:48.298 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.298 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.298 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.298 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setdirlayout_latency": { 2024-06-10T01:32:48.298 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.298 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.298 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.298 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setfilelock_latency": { 2024-06-10T01:32:48.299 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.299 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.299 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.299 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setlayout_latency": { 2024-06-10T01:32:48.299 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.299 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.299 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.299 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setxattr_latency": { 2024-06-10T01:32:48.299 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.299 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.299 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.299 INFO:teuthology.orchestra.run.smithi175.stdout: "req_symlink_latency": { 2024-06-10T01:32:48.300 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.300 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.300 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.300 INFO:teuthology.orchestra.run.smithi175.stdout: "req_unlink_latency": { 2024-06-10T01:32:48.300 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.300 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.300 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.300 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_revoke_eviction": 0, 2024-06-10T01:32:48.300 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_acquisition_throttle": 0, 2024-06-10T01:32:48.300 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getvxattr_latency": { 2024-06-10T01:32:48.300 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.300 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.300 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.301 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:48.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.301 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_sessions": { 2024-06-10T01:32:48.301 INFO:teuthology.orchestra.run.smithi175.stdout: "session_count": 1, 2024-06-10T01:32:48.301 INFO:teuthology.orchestra.run.smithi175.stdout: "session_add": 1, 2024-06-10T01:32:48.301 INFO:teuthology.orchestra.run.smithi175.stdout: "session_remove": 0, 2024-06-10T01:32:48.301 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_open": 1, 2024-06-10T01:32:48.301 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_stale": 0, 2024-06-10T01:32:48.301 INFO:teuthology.orchestra.run.smithi175.stdout: "total_load": 0, 2024-06-10T01:32:48.301 INFO:teuthology.orchestra.run.smithi175.stdout: "average_load": 0, 2024-06-10T01:32:48.301 INFO:teuthology.orchestra.run.smithi175.stdout: "avg_session_uptime": 3071, 2024-06-10T01:32:48.301 INFO:teuthology.orchestra.run.smithi175.stdout: "mdthresh_evicted": 0 2024-06-10T01:32:48.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.301 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2024-06-10T01:32:48.301 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2024-06-10T01:32:48.301 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2024-06-10T01:32:48.302 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T01:32:48.302 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2024-06-10T01:32:48.302 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T01:32:48.302 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2024-06-10T01:32:48.302 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T01:32:48.302 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T01:32:48.302 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T01:32:48.302 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T01:32:48.302 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T01:32:48.302 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2024-06-10T01:32:48.302 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T01:32:48.302 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T01:32:48.302 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_bytes": 0, 2024-06-10T01:32:48.303 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_items": 0, 2024-06-10T01:32:48.303 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_bytes": 0, 2024-06-10T01:32:48.303 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_items": 0, 2024-06-10T01:32:48.303 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T01:32:48.303 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T01:32:48.303 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T01:32:48.303 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T01:32:48.303 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T01:32:48.303 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2024-06-10T01:32:48.303 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2024-06-10T01:32:48.303 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2024-06-10T01:32:48.303 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T01:32:48.303 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T01:32:48.303 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2024-06-10T01:32:48.303 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2024-06-10T01:32:48.304 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2024-06-10T01:32:48.304 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2024-06-10T01:32:48.304 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T01:32:48.304 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2024-06-10T01:32:48.304 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T01:32:48.304 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2024-06-10T01:32:48.304 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 243429, 2024-06-10T01:32:48.304 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 90, 2024-06-10T01:32:48.304 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 0, 2024-06-10T01:32:48.304 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 0, 2024-06-10T01:32:48.304 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2024-06-10T01:32:48.304 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2024-06-10T01:32:48.304 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2024-06-10T01:32:48.304 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2024-06-10T01:32:48.304 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2024-06-10T01:32:48.305 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2024-06-10T01:32:48.305 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 8800, 2024-06-10T01:32:48.305 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 169, 2024-06-10T01:32:48.305 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T01:32:48.305 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2024-06-10T01:32:48.305 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2024-06-10T01:32:48.305 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2024-06-10T01:32:48.305 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 31875769, 2024-06-10T01:32:48.305 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 181020, 2024-06-10T01:32:48.305 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2024-06-10T01:32:48.305 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2024-06-10T01:32:48.305 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2024-06-10T01:32:48.305 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2024-06-10T01:32:48.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.305 INFO:teuthology.orchestra.run.smithi175.stdout: "objecter": { 2024-06-10T01:32:48.305 INFO:teuthology.orchestra.run.smithi175.stdout: "op_active": 0, 2024-06-10T01:32:48.306 INFO:teuthology.orchestra.run.smithi175.stdout: "op_laggy": 0, 2024-06-10T01:32:48.306 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send": 6383, 2024-06-10T01:32:48.306 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send_bytes": 21, 2024-06-10T01:32:48.306 INFO:teuthology.orchestra.run.smithi175.stdout: "op_resend": 3, 2024-06-10T01:32:48.306 INFO:teuthology.orchestra.run.smithi175.stdout: "op_reply": 6383, 2024-06-10T01:32:48.306 INFO:teuthology.orchestra.run.smithi175.stdout: "op_latency": { 2024-06-10T01:32:48.306 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 6383, 2024-06-10T01:32:48.306 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 19.084730049, 2024-06-10T01:32:48.306 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.002989931 2024-06-10T01:32:48.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.306 INFO:teuthology.orchestra.run.smithi175.stdout: "op_inflight": 0, 2024-06-10T01:32:48.306 INFO:teuthology.orchestra.run.smithi175.stdout: "oplen_avg": { 2024-06-10T01:32:48.306 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 6383, 2024-06-10T01:32:48.307 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 6384 2024-06-10T01:32:48.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.307 INFO:teuthology.orchestra.run.smithi175.stdout: "op": 6383, 2024-06-10T01:32:48.307 INFO:teuthology.orchestra.run.smithi175.stdout: "op_r": 6383, 2024-06-10T01:32:48.307 INFO:teuthology.orchestra.run.smithi175.stdout: "op_w": 0, 2024-06-10T01:32:48.307 INFO:teuthology.orchestra.run.smithi175.stdout: "op_rmw": 0, 2024-06-10T01:32:48.307 INFO:teuthology.orchestra.run.smithi175.stdout: "op_pg": 0, 2024-06-10T01:32:48.307 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_stat": 3491, 2024-06-10T01:32:48.307 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_create": 0, 2024-06-10T01:32:48.307 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_read": 2890, 2024-06-10T01:32:48.307 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_write": 0, 2024-06-10T01:32:48.307 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writefull": 0, 2024-06-10T01:32:48.307 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writesame": 0, 2024-06-10T01:32:48.307 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_append": 0, 2024-06-10T01:32:48.308 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_zero": 0, 2024-06-10T01:32:48.308 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_truncate": 0, 2024-06-10T01:32:48.308 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_delete": 0, 2024-06-10T01:32:48.308 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_mapext": 0, 2024-06-10T01:32:48.308 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_sparse_read": 0, 2024-06-10T01:32:48.308 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_clonerange": 0, 2024-06-10T01:32:48.308 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_getxattr": 1, 2024-06-10T01:32:48.308 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_setxattr": 0, 2024-06-10T01:32:48.308 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_cmpxattr": 0, 2024-06-10T01:32:48.308 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_rmxattr": 0, 2024-06-10T01:32:48.308 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_resetxattrs": 0, 2024-06-10T01:32:48.308 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_call": 0, 2024-06-10T01:32:48.308 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_watch": 0, 2024-06-10T01:32:48.308 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_notify": 0, 2024-06-10T01:32:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T01:32:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls": 0, 2024-06-10T01:32:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls_filter": 0, 2024-06-10T01:32:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_other": 0, 2024-06-10T01:32:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_active": 0, 2024-06-10T01:32:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_send": 0, 2024-06-10T01:32:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_resend": 0, 2024-06-10T01:32:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_ping": 0, 2024-06-10T01:32:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_active": 0, 2024-06-10T01:32:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_send": 0, 2024-06-10T01:32:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_resend": 0, 2024-06-10T01:32:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_active": 0, 2024-06-10T01:32:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_send": 0, 2024-06-10T01:32:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_resend": 0, 2024-06-10T01:32:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_active": 0, 2024-06-10T01:32:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_send": 0, 2024-06-10T01:32:48.310 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_resend": 0, 2024-06-10T01:32:48.310 INFO:teuthology.orchestra.run.smithi175.stdout: "command_active": 0, 2024-06-10T01:32:48.310 INFO:teuthology.orchestra.run.smithi175.stdout: "command_send": 0, 2024-06-10T01:32:48.310 INFO:teuthology.orchestra.run.smithi175.stdout: "command_resend": 0, 2024-06-10T01:32:48.310 INFO:teuthology.orchestra.run.smithi175.stdout: "map_epoch": 102, 2024-06-10T01:32:48.310 INFO:teuthology.orchestra.run.smithi175.stdout: "map_full": 0, 2024-06-10T01:32:48.310 INFO:teuthology.orchestra.run.smithi175.stdout: "map_inc": 5, 2024-06-10T01:32:48.310 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_sessions": 11, 2024-06-10T01:32:48.310 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_open": 12, 2024-06-10T01:32:48.310 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_close": 1, 2024-06-10T01:32:48.310 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_laggy": 0, 2024-06-10T01:32:48.310 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_wr": 0, 2024-06-10T01:32:48.310 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_rd": 2, 2024-06-10T01:32:48.311 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_del": 0 2024-06-10T01:32:48.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.311 INFO:teuthology.orchestra.run.smithi175.stdout: "oft": { 2024-06-10T01:32:48.311 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_objs": 0, 2024-06-10T01:32:48.311 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_kv_pairs": 0, 2024-06-10T01:32:48.311 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_updates": 0, 2024-06-10T01:32:48.311 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_removes": 0 2024-06-10T01:32:48.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.311 INFO:teuthology.orchestra.run.smithi175.stdout: "osd-slow-ops": { 2024-06-10T01:32:48.311 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_ops_count": 0 2024-06-10T01:32:48.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.311 INFO:teuthology.orchestra.run.smithi175.stdout: "purge_queue": { 2024-06-10T01:32:48.311 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops": 0, 2024-06-10T01:32:48.312 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops_high_water": 0, 2024-06-10T01:32:48.312 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing": 0, 2024-06-10T01:32:48.312 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_high_water": 0, 2024-06-10T01:32:48.312 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed_ops": 0, 2024-06-10T01:32:48.312 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed": 0, 2024-06-10T01:32:48.312 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_item_in_journal": 0 2024-06-10T01:32:48.312 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.312 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T01:32:48.312 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-10T01:32:48.312 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-06-10T01:32:48.312 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-10T01:32:48.312 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 6939, 2024-06-10T01:32:48.312 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 423506848, 2024-06-10T01:32:48.312 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:48.313 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 6939, 2024-06-10T01:32:48.313 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-06-10T01:32:48.313 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-06-10T01:32:48.313 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 6939, 2024-06-10T01:32:48.313 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 423506848, 2024-06-10T01:32:48.313 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-10T01:32:48.313 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.313 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.313 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.313 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:48.313 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.313 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_bytes": { 2024-06-10T01:32:48.313 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-10T01:32:48.313 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-06-10T01:32:48.313 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-10T01:32:48.314 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-06-10T01:32:48.314 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-06-10T01:32:48.314 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:48.314 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-06-10T01:32:48.314 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 6383, 2024-06-10T01:32:48.314 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 421659688, 2024-06-10T01:32:48.314 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 6383, 2024-06-10T01:32:48.314 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 421659688, 2024-06-10T01:32:48.314 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-10T01:32:48.314 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.314 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.314 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.314 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:48.315 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.315 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_ops": { 2024-06-10T01:32:48.315 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-10T01:32:48.315 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 1024, 2024-06-10T01:32:48.315 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-10T01:32:48.315 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-06-10T01:32:48.315 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-06-10T01:32:48.315 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:48.315 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-06-10T01:32:48.315 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 6383, 2024-06-10T01:32:48.315 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 6383, 2024-06-10T01:32:48.315 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 6383, 2024-06-10T01:32:48.315 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 6383, 2024-06-10T01:32:48.315 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-10T01:32:48.316 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.316 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.316 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.316 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:48.316 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.316 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle": { 2024-06-10T01:32:48.316 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-10T01:32:48.316 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-06-10T01:32:48.316 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-10T01:32:48.316 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-06-10T01:32:48.316 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-06-10T01:32:48.316 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:48.316 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-06-10T01:32:48.316 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-06-10T01:32:48.317 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-06-10T01:32:48.317 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 0, 2024-06-10T01:32:48.317 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 0, 2024-06-10T01:32:48.317 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-10T01:32:48.317 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.317 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.317 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.317 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:48.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-10T01:32:48.317 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle-0x5631de7c66e0": { 2024-06-10T01:32:48.317 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-10T01:32:48.317 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-06-10T01:32:48.317 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-10T01:32:48.317 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-06-10T01:32:48.318 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-06-10T01:32:48.318 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-10T01:32:48.318 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-06-10T01:32:48.318 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-06-10T01:32:48.318 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-06-10T01:32:48.318 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 0, 2024-06-10T01:32:48.318 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 0, 2024-06-10T01:32:48.318 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-10T01:32:48.318 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-10T01:32:48.318 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-10T01:32:48.318 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-10T01:32:48.318 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:48.318 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-10T01:32:48.319 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-06-10T01:32:49.106 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-06-10T01:32:49.106 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-06-10T01:32:49.106 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-06-10T01:32:49.117 INFO:tasks.kclient:Unmounting kernel clients... 2024-06-10T01:32:49.117 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-10T01:32:49.117 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-10T01:32:49.153 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-10T01:32:49.153 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-10T01:32:49.213 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-10T01:32:49.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:49 smithi175 ceph-mon[36636]: pgmap v1238: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:32:49.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:49 smithi007 ceph-mon[32398]: pgmap v1238: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:32:49.514 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:32:49.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:49 smithi078 ceph-mon[36683]: pgmap v1238: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:32:51.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:51 smithi007 ceph-mon[32398]: pgmap v1239: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:32:51.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.293+0000 7f44577c0700 1 -- 172.21.15.7:0/4251559531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4450101d50 msgr2=0x7f445010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:51.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.293+0000 7f44577c0700 1 --2- 172.21.15.7:0/4251559531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4450101d50 0x7f445010e240 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f444800b0a0 tx=0x7f444800b3b0 comp rx=0 tx=0).stop 2024-06-10T01:32:51.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.293+0000 7f44577c0700 1 -- 172.21.15.7:0/4251559531 shutdown_connections 2024-06-10T01:32:51.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.293+0000 7f44577c0700 1 --2- 172.21.15.7:0/4251559531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4450101d50 0x7f445010e240 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:51.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.293+0000 7f44577c0700 1 --2- 172.21.15.7:0/4251559531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4450107400 0x7f44501077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:51.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.293+0000 7f44577c0700 1 --2- 172.21.15.7:0/4251559531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4450101390 0x7f4450101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:51.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.293+0000 7f44577c0700 1 -- 172.21.15.7:0/4251559531 >> 172.21.15.7:0/4251559531 conn(0x7f4450078340 msgr2=0x7f4450078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:51.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.293+0000 7f44577c0700 1 -- 172.21.15.7:0/4251559531 shutdown_connections 2024-06-10T01:32:51.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.293+0000 7f44577c0700 1 -- 172.21.15.7:0/4251559531 wait complete. 2024-06-10T01:32:51.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.294+0000 7f44577c0700 1 Processor -- start 2024-06-10T01:32:51.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.294+0000 7f44577c0700 1 -- start start 2024-06-10T01:32:51.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.295+0000 7f44577c0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4450101390 0x7f4450199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:51.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.295+0000 7f44577c0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4450101d50 0x7f4450199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:51.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.295+0000 7f44577c0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4450107400 0x7f44501a4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:51.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.295+0000 7f44577c0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4450111670 con 0x7f4450107400 2024-06-10T01:32:51.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.295+0000 7f44577c0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f44501114f0 con 0x7f4450101390 2024-06-10T01:32:51.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.295+0000 7f44577c0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f44501117f0 con 0x7f4450101d50 2024-06-10T01:32:51.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.295+0000 7f4455d5d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4450107400 0x7f44501a4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:51.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.295+0000 7f4455d5d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4450107400 0x7f44501a4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39400/0 (socket says 172.21.15.7:39400) 2024-06-10T01:32:51.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.295+0000 7f4455d5d700 1 -- 172.21.15.7:0/349898539 learned_addr learned my addr 172.21.15.7:0/349898539 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:32:51.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.295+0000 7f4454d5b700 1 --2- 172.21.15.7:0/349898539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4450101d50 0x7f4450199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:51.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.295+0000 7f445555c700 1 --2- 172.21.15.7:0/349898539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4450101390 0x7f4450199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:51.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.295+0000 7f4455d5d700 1 -- 172.21.15.7:0/349898539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4450101d50 msgr2=0x7f4450199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:51.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.295+0000 7f4455d5d700 1 --2- 172.21.15.7:0/349898539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4450101d50 0x7f4450199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:51.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.295+0000 7f4455d5d700 1 -- 172.21.15.7:0/349898539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4450101390 msgr2=0x7f4450199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:51.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.295+0000 7f4455d5d700 1 --2- 172.21.15.7:0/349898539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4450101390 0x7f4450199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:51.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.295+0000 7f4455d5d700 1 -- 172.21.15.7:0/349898539 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44501a4fd0 con 0x7f4450107400 2024-06-10T01:32:51.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.295+0000 7f4455d5d700 1 --2- 172.21.15.7:0/349898539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4450107400 0x7f44501a4a90 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f444800b780 tx=0x7f4448004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:51.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.295+0000 7f44467fc700 1 -- 172.21.15.7:0/349898539 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4448021070 con 0x7f4450107400 2024-06-10T01:32:51.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.295+0000 7f44467fc700 1 -- 172.21.15.7:0/349898539 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4448007b80 con 0x7f4450107400 2024-06-10T01:32:51.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.295+0000 7f44577c0700 1 -- 172.21.15.7:0/349898539 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44501a51a0 con 0x7f4450107400 2024-06-10T01:32:51.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.296+0000 7f44467fc700 1 -- 172.21.15.7:0/349898539 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4448005370 con 0x7f4450107400 2024-06-10T01:32:51.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.296+0000 7f44577c0700 1 -- 172.21.15.7:0/349898539 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44501a5600 con 0x7f4450107400 2024-06-10T01:32:51.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.297+0000 7f44577c0700 1 -- 172.21.15.7:0/349898539 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4450069180 con 0x7f4450107400 2024-06-10T01:32:51.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.298+0000 7f44467fc700 1 -- 172.21.15.7:0/349898539 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f4448024070 con 0x7f4450107400 2024-06-10T01:32:51.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.299+0000 7f44467fc700 1 --2- 172.21.15.7:0/349898539 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f443c0a2c20 0x7f443c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:51.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.299+0000 7f445555c700 1 --2- 172.21.15.7:0/349898539 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f443c0a2c20 0x7f443c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:51.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.299+0000 7f44467fc700 1 -- 172.21.15.7:0/349898539 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f44480d0400 con 0x7f4450107400 2024-06-10T01:32:51.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.299+0000 7f445555c700 1 --2- 172.21.15.7:0/349898539 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f443c0a2c20 0x7f443c0a50e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f444c005fd0 tx=0x7f444c005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:51.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.302+0000 7f44467fc700 1 -- 172.21.15.7:0/349898539 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f444809d070 con 0x7f4450107400 2024-06-10T01:32:51.450 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:51 smithi175 ceph-mon[36636]: pgmap v1239: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:32:51.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.528+0000 7f44577c0700 1 -- 172.21.15.7:0/349898539 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f445002de60 con 0x7f4450107400 2024-06-10T01:32:51.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.529+0000 7f44467fc700 1 -- 172.21.15.7:0/349898539 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v102) ==== 81+0+580 (secure 0 0 0) 0x7f444809c970 con 0x7f4450107400 2024-06-10T01:32:51.522 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1833397470 2024-06-11T00:54:35.441123+0000 2024-06-10T01:32:51.522 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/322543993 2024-06-11T00:54:35.441123+0000 2024-06-10T01:32:51.522 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/212510583 2024-06-11T00:54:35.441123+0000 2024-06-10T01:32:51.522 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/3298829389 2024-06-11T00:54:35.441123+0000 2024-06-10T01:32:51.522 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/1100766262 2024-06-11T00:54:08.102405+0000 2024-06-10T01:32:51.522 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/2679436843 2024-06-11T00:54:08.102405+0000 2024-06-10T01:32:51.522 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/1100766262 2024-06-11T00:54:08.102405+0000 2024-06-10T01:32:51.522 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1210355277 2024-06-11T00:54:08.102405+0000 2024-06-10T01:32:51.522 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/3298829389 2024-06-11T00:54:35.441123+0000 2024-06-10T01:32:51.522 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3610017406 2024-06-11T00:54:08.102405+0000 2024-06-10T01:32:51.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.532+0000 7f44577c0700 1 -- 172.21.15.7:0/349898539 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f443c0a2c20 msgr2=0x7f443c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:51.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.532+0000 7f44577c0700 1 --2- 172.21.15.7:0/349898539 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f443c0a2c20 0x7f443c0a50e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f444c005fd0 tx=0x7f444c005db0 comp rx=0 tx=0).stop 2024-06-10T01:32:51.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.532+0000 7f44577c0700 1 -- 172.21.15.7:0/349898539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4450107400 msgr2=0x7f44501a4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:51.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.532+0000 7f44577c0700 1 --2- 172.21.15.7:0/349898539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4450107400 0x7f44501a4a90 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f444800b780 tx=0x7f4448004d00 comp rx=0 tx=0).stop 2024-06-10T01:32:51.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.532+0000 7f44577c0700 1 -- 172.21.15.7:0/349898539 shutdown_connections 2024-06-10T01:32:51.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.532+0000 7f44577c0700 1 --2- 172.21.15.7:0/349898539 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f443c0a2c20 0x7f443c0a50e0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:51.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.532+0000 7f44577c0700 1 --2- 172.21.15.7:0/349898539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4450107400 0x7f44501a4a90 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:51.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.532+0000 7f44577c0700 1 --2- 172.21.15.7:0/349898539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4450101390 0x7f4450199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:51.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.532+0000 7f44577c0700 1 --2- 172.21.15.7:0/349898539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4450101d50 0x7f4450199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:51.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.533+0000 7f44577c0700 1 -- 172.21.15.7:0/349898539 >> 172.21.15.7:0/349898539 conn(0x7f4450078340 msgr2=0x7f44500ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:51.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.533+0000 7f44577c0700 1 -- 172.21.15.7:0/349898539 shutdown_connections 2024-06-10T01:32:51.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:51.533+0000 7f44577c0700 1 -- 172.21.15.7:0/349898539 wait complete. 2024-06-10T01:32:51.526 INFO:teuthology.orchestra.run.smithi007.stderr:listed 10 entries 2024-06-10T01:32:51.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:51 smithi078 ceph-mon[36683]: pgmap v1239: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:32:52.206 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-06-10T01:32:52.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:32:52.207 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-06-10T01:32:52.335 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-06-10T01:32:52.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:32:52.335 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-06-10T01:32:52.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:32:52.394 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-06-10T01:32:52.462 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-06-10T01:32:52.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:32:52.463 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-10T01:32:52.509 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:52 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/349898539' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T01:32:52.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:32:52.523 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-06-10T01:32:52.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:52 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/349898539' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T01:32:52.618 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-06-10T01:32:52.634 INFO:tasks.cephadm:Teardown begin 2024-06-10T01:32:52.634 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-06-10T01:32:52.635 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:32:52.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:52 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/349898539' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T01:32:52.921 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:32:53.410 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:53 smithi007 ceph-mon[32398]: pgmap v1240: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:32:53.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:53 smithi078 ceph-mon[36683]: pgmap v1240: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:32:53.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:53 smithi175 ceph-mon[36636]: pgmap v1240: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:32:54.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.930+0000 7f71063f5700 1 -- 172.21.15.7:0/3225511586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71001030f0 msgr2=0x7f7100107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:54.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.930+0000 7f71063f5700 1 --2- 172.21.15.7:0/3225511586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71001030f0 0x7f7100107110 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f70f40099c0 tx=0x7f70f4009cd0 comp rx=0 tx=0).stop 2024-06-10T01:32:54.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.930+0000 7f71063f5700 1 -- 172.21.15.7:0/3225511586 shutdown_connections 2024-06-10T01:32:54.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.930+0000 7f71063f5700 1 --2- 172.21.15.7:0/3225511586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7100107840 0x7f710010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:54.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.930+0000 7f71063f5700 1 --2- 172.21.15.7:0/3225511586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7100102740 0x7f7100102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:54.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.930+0000 7f71063f5700 1 --2- 172.21.15.7:0/3225511586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71001030f0 0x7f7100107110 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:54.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.930+0000 7f71063f5700 1 -- 172.21.15.7:0/3225511586 >> 172.21.15.7:0/3225511586 conn(0x7f71000795f0 msgr2=0x7f7100079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:54.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.930+0000 7f71063f5700 1 -- 172.21.15.7:0/3225511586 shutdown_connections 2024-06-10T01:32:54.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.930+0000 7f71063f5700 1 -- 172.21.15.7:0/3225511586 wait complete. 2024-06-10T01:32:54.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.930+0000 7f71063f5700 1 Processor -- start 2024-06-10T01:32:54.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.931+0000 7f71063f5700 1 -- start start 2024-06-10T01:32:54.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.931+0000 7f71063f5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7100102740 0x7f710019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:54.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.931+0000 7f71063f5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71001030f0 0x7f710019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:54.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.931+0000 7f71063f5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7100107840 0x7f7100198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:54.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.931+0000 7f71063f5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7100112750 con 0x7f7100107840 2024-06-10T01:32:54.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.931+0000 7f71063f5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71001125d0 con 0x7f7100102740 2024-06-10T01:32:54.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.931+0000 7f71063f5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f71001128d0 con 0x7f71001030f0 2024-06-10T01:32:54.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.931+0000 7f7104992700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7100107840 0x7f7100198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:54.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.931+0000 7f7104992700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7100107840 0x7f7100198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47918/0 (socket says 172.21.15.7:47918) 2024-06-10T01:32:54.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.931+0000 7f7104992700 1 -- 172.21.15.7:0/1871927318 learned_addr learned my addr 172.21.15.7:0/1871927318 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:32:54.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.931+0000 7f70fffff700 1 --2- 172.21.15.7:0/1871927318 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7100102740 0x7f710019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:54.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.931+0000 7f70ff7fe700 1 --2- 172.21.15.7:0/1871927318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71001030f0 0x7f710019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:54.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.931+0000 7f7104992700 1 -- 172.21.15.7:0/1871927318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71001030f0 msgr2=0x7f710019ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:54.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.931+0000 7f7104992700 1 --2- 172.21.15.7:0/1871927318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71001030f0 0x7f710019ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:54.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.931+0000 7f7104992700 1 -- 172.21.15.7:0/1871927318 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7100102740 msgr2=0x7f710019e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:54.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.932+0000 7f7104992700 1 --2- 172.21.15.7:0/1871927318 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7100102740 0x7f710019e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:54.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.932+0000 7f7104992700 1 -- 172.21.15.7:0/1871927318 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71001990b0 con 0x7f7100107840 2024-06-10T01:32:54.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.932+0000 7f7104992700 1 --2- 172.21.15.7:0/1871927318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7100107840 0x7f7100198950 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f70f8007ee0 tx=0x7f70f800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:54.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.932+0000 7f70fd7fa700 1 -- 172.21.15.7:0/1871927318 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f800dbd0 con 0x7f7100107840 2024-06-10T01:32:54.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.932+0000 7f71063f5700 1 -- 172.21.15.7:0/1871927318 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71001993a0 con 0x7f7100107840 2024-06-10T01:32:54.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.932+0000 7f70fd7fa700 1 -- 172.21.15.7:0/1871927318 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70f800dd70 con 0x7f7100107840 2024-06-10T01:32:54.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.932+0000 7f70fd7fa700 1 -- 172.21.15.7:0/1871927318 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f8005a60 con 0x7f7100107840 2024-06-10T01:32:54.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.932+0000 7f71063f5700 1 -- 172.21.15.7:0/1871927318 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71001a5660 con 0x7f7100107840 2024-06-10T01:32:54.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.933+0000 7f71063f5700 1 -- 172.21.15.7:0/1871927318 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7100069180 con 0x7f7100107840 2024-06-10T01:32:54.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.934+0000 7f70fd7fa700 1 -- 172.21.15.7:0/1871927318 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f70f801a020 con 0x7f7100107840 2024-06-10T01:32:54.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.935+0000 7f70fd7fa700 1 --2- 172.21.15.7:0/1871927318 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f70e80a2ca0 0x7f70e80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:54.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.935+0000 7f70fffff700 1 --2- 172.21.15.7:0/1871927318 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f70e80a2ca0 0x7f70e80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:54.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.935+0000 7f70fd7fa700 1 -- 172.21.15.7:0/1871927318 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f70f80ceab0 con 0x7f7100107840 2024-06-10T01:32:54.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.935+0000 7f70fffff700 1 --2- 172.21.15.7:0/1871927318 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f70e80a2ca0 0x7f70e80a5160 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7100199a90 tx=0x7f70f000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:54.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:54.939+0000 7f70fd7fa700 1 -- 172.21.15.7:0/1871927318 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f70f8010070 con 0x7f7100107840 2024-06-10T01:32:55.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:55.194+0000 7f71063f5700 1 -- 172.21.15.7:0/1871927318 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f710002dca0 con 0x7f7100107840 2024-06-10T01:32:55.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:55.194+0000 7f70fd7fa700 1 -- 172.21.15.7:0/1871927318 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+11856 (secure 0 0 0) 0x7f70f809b690 con 0x7f7100107840 2024-06-10T01:32:55.187 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:32:55.187 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":61,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:32:32.480796+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29813":{"gid":29813,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/138896429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":138896429},{"type":"v1","addr":"172.21.15.175:6835","nonce":138896429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24451,24434,24436]},"id":1}]} 2024-06-10T01:32:55.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:55.197+0000 7f71063f5700 1 -- 172.21.15.7:0/1871927318 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f70e80a2ca0 msgr2=0x7f70e80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:55.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:55.197+0000 7f71063f5700 1 --2- 172.21.15.7:0/1871927318 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f70e80a2ca0 0x7f70e80a5160 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7100199a90 tx=0x7f70f000a380 comp rx=0 tx=0).stop 2024-06-10T01:32:55.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:55.197+0000 7f71063f5700 1 -- 172.21.15.7:0/1871927318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7100107840 msgr2=0x7f7100198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:55.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:55.197+0000 7f71063f5700 1 --2- 172.21.15.7:0/1871927318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7100107840 0x7f7100198950 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f70f8007ee0 tx=0x7f70f800d3b0 comp rx=0 tx=0).stop 2024-06-10T01:32:55.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:55.197+0000 7f71063f5700 1 -- 172.21.15.7:0/1871927318 shutdown_connections 2024-06-10T01:32:55.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:55.197+0000 7f71063f5700 1 --2- 172.21.15.7:0/1871927318 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f70e80a2ca0 0x7f70e80a5160 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:55.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:55.197+0000 7f71063f5700 1 --2- 172.21.15.7:0/1871927318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7100107840 0x7f7100198950 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:55.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:55.197+0000 7f71063f5700 1 --2- 172.21.15.7:0/1871927318 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7100102740 0x7f710019e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:55.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:55.198+0000 7f71063f5700 1 --2- 172.21.15.7:0/1871927318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71001030f0 0x7f710019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:55.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:55.198+0000 7f71063f5700 1 -- 172.21.15.7:0/1871927318 >> 172.21.15.7:0/1871927318 conn(0x7f71000795f0 msgr2=0x7f71001004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:55.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:55.198+0000 7f71063f5700 1 -- 172.21.15.7:0/1871927318 shutdown_connections 2024-06-10T01:32:55.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:55.198+0000 7f71063f5700 1 -- 172.21.15.7:0/1871927318 wait complete. 2024-06-10T01:32:55.192 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 61 2024-06-10T01:32:55.503 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:55 smithi175 ceph-mon[36636]: pgmap v1241: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:32:55.503 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:55 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1871927318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:32:55.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:55 smithi007 ceph-mon[32398]: pgmap v1241: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:32:55.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:55 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1871927318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:32:55.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:55 smithi078 ceph-mon[36683]: pgmap v1241: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:32:55.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:55 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1871927318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:32:55.984 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:32:56.247 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:32:57.504 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:57 smithi007 ceph-mon[32398]: pgmap v1242: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-06-10T01:32:57.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:57 smithi078 ceph-mon[36683]: pgmap v1242: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-06-10T01:32:57.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:57 smithi175 ceph-mon[36636]: pgmap v1242: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-06-10T01:32:58.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.211+0000 7f487dee6700 1 -- 172.21.15.7:0/474839453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48781030f0 msgr2=0x7f4878107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:58.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.211+0000 7f487dee6700 1 --2- 172.21.15.7:0/474839453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48781030f0 0x7f4878107110 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f48600099c0 tx=0x7f4860009cd0 comp rx=0 tx=0).stop 2024-06-10T01:32:58.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.211+0000 7f487dee6700 1 -- 172.21.15.7:0/474839453 shutdown_connections 2024-06-10T01:32:58.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.211+0000 7f487dee6700 1 --2- 172.21.15.7:0/474839453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4878107840 0x7f487810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:58.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.211+0000 7f487dee6700 1 --2- 172.21.15.7:0/474839453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4878102740 0x7f4878102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:58.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.211+0000 7f487dee6700 1 --2- 172.21.15.7:0/474839453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48781030f0 0x7f4878107110 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:58.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.211+0000 7f487dee6700 1 -- 172.21.15.7:0/474839453 >> 172.21.15.7:0/474839453 conn(0x7f48780795f0 msgr2=0x7f4878079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:58.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.212+0000 7f487dee6700 1 -- 172.21.15.7:0/474839453 shutdown_connections 2024-06-10T01:32:58.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.212+0000 7f487dee6700 1 -- 172.21.15.7:0/474839453 wait complete. 2024-06-10T01:32:58.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.212+0000 7f487dee6700 1 Processor -- start 2024-06-10T01:32:58.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.213+0000 7f487dee6700 1 -- start start 2024-06-10T01:32:58.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.213+0000 7f487dee6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4878102740 0x7f4878074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:58.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.213+0000 7f487dee6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48781030f0 0x7f4878074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:58.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.213+0000 7f487dee6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4878107840 0x7f4878075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:58.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.213+0000 7f487dee6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4878112750 con 0x7f48781030f0 2024-06-10T01:32:58.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.213+0000 7f487dee6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f48781125d0 con 0x7f4878107840 2024-06-10T01:32:58.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.213+0000 7f487dee6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f48781128d0 con 0x7f4878102740 2024-06-10T01:32:58.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.213+0000 7f486effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48781030f0 0x7f4878074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:58.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.213+0000 7f486effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48781030f0 0x7f4878074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47948/0 (socket says 172.21.15.7:47948) 2024-06-10T01:32:58.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.213+0000 7f486effd700 1 -- 172.21.15.7:0/1161570702 learned_addr learned my addr 172.21.15.7:0/1161570702 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:32:58.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.213+0000 7f4877fff700 1 --2- 172.21.15.7:0/1161570702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4878107840 0x7f4878075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:58.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.213+0000 7f48777fe700 1 --2- 172.21.15.7:0/1161570702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4878102740 0x7f4878074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:58.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.214+0000 7f486effd700 1 -- 172.21.15.7:0/1161570702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4878102740 msgr2=0x7f4878074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:58.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.214+0000 7f486effd700 1 --2- 172.21.15.7:0/1161570702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4878102740 0x7f4878074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:58.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.214+0000 7f486effd700 1 -- 172.21.15.7:0/1161570702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4878107840 msgr2=0x7f4878075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:58.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.214+0000 7f486effd700 1 --2- 172.21.15.7:0/1161570702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4878107840 0x7f4878075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:58.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.214+0000 7f486effd700 1 -- 172.21.15.7:0/1161570702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48781a7030 con 0x7f48781030f0 2024-06-10T01:32:58.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.214+0000 7f486effd700 1 --2- 172.21.15.7:0/1161570702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48781030f0 0x7f4878074cd0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f4860006010 tx=0x7f4860009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:58.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.214+0000 7f48757fa700 1 -- 172.21.15.7:0/1161570702 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4860026070 con 0x7f48781030f0 2024-06-10T01:32:58.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.214+0000 7f48757fa700 1 -- 172.21.15.7:0/1161570702 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4860004cd0 con 0x7f48781030f0 2024-06-10T01:32:58.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.214+0000 7f487dee6700 1 -- 172.21.15.7:0/1161570702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48781a72c0 con 0x7f48781030f0 2024-06-10T01:32:58.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.214+0000 7f48757fa700 1 -- 172.21.15.7:0/1161570702 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4860003ba0 con 0x7f48781030f0 2024-06-10T01:32:58.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.214+0000 7f487dee6700 1 -- 172.21.15.7:0/1161570702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48781a7710 con 0x7f48781030f0 2024-06-10T01:32:58.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.215+0000 7f487dee6700 1 -- 172.21.15.7:0/1161570702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4878069180 con 0x7f48781030f0 2024-06-10T01:32:58.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.216+0000 7f48757fa700 1 -- 172.21.15.7:0/1161570702 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f4860029070 con 0x7f48781030f0 2024-06-10T01:32:58.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.217+0000 7f48757fa700 1 --2- 172.21.15.7:0/1161570702 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f48640a2c20 0x7f48640a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:32:58.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.217+0000 7f48777fe700 1 --2- 172.21.15.7:0/1161570702 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f48640a2c20 0x7f48640a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:32:58.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.217+0000 7f48757fa700 1 -- 172.21.15.7:0/1161570702 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f48600d0100 con 0x7f48781030f0 2024-06-10T01:32:58.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.218+0000 7f48777fe700 1 --2- 172.21.15.7:0/1161570702 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f48640a2c20 0x7f48640a50e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f486800ac50 tx=0x7f486800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:32:58.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.221+0000 7f48757fa700 1 -- 172.21.15.7:0/1161570702 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4860030020 con 0x7f48781030f0 2024-06-10T01:32:58.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.476+0000 7f487dee6700 1 -- 172.21.15.7:0/1161570702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f487802de60 con 0x7f48781030f0 2024-06-10T01:32:58.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.477+0000 7f48757fa700 1 -- 172.21.15.7:0/1161570702 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+11856 (secure 0 0 0) 0x7f4860013070 con 0x7f48781030f0 2024-06-10T01:32:58.470 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:32:58.470 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2172378766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2172378766},{"type":"v1","addr":"172.21.15.7:6835","nonce":2172378766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1734773915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1734773915},{"type":"v1","addr":"172.21.15.7:6837","nonce":1734773915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/159652731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":159652731},{"type":"v1","addr":"172.21.15.7:6839","nonce":159652731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1788930795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1788930795},{"type":"v1","addr":"172.21.15.7:6841","nonce":1788930795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1425832513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1425832513},{"type":"v1","addr":"172.21.15.78:6833","nonce":1425832513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/938897057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":938897057},{"type":"v1","addr":"172.21.15.78:6835","nonce":938897057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":61,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T01:01:49.029348+0000","modified":"2024-06-10T01:32:32.480796+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24451,"mds_1":24436,"mds_2":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/1281305765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1281305765},{"type":"v1","addr":"172.21.15.175:6833","nonce":1281305765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/3608622984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3608622984},{"type":"v1","addr":"172.21.15.78:6837","nonce":3608622984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.78:6839/1946947484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1946947484},{"type":"v1","addr":"172.21.15.78:6839","nonce":1946947484}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.175:6837/779740919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":779740919},{"type":"v1","addr":"172.21.15.175:6837","nonce":779740919}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/1983418024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1983418024},{"type":"v1","addr":"172.21.15.175:6839","nonce":1983418024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29813":{"gid":29813,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/138896429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":138896429},{"type":"v1","addr":"172.21.15.175:6835","nonce":138896429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24451,"qdb_cluster":[24451,24434,24436]},"id":1}]} 2024-06-10T01:32:58.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.479+0000 7f487dee6700 1 -- 172.21.15.7:0/1161570702 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f48640a2c20 msgr2=0x7f48640a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:58.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.479+0000 7f487dee6700 1 --2- 172.21.15.7:0/1161570702 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f48640a2c20 0x7f48640a50e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f486800ac50 tx=0x7f486800a380 comp rx=0 tx=0).stop 2024-06-10T01:32:58.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.479+0000 7f487dee6700 1 -- 172.21.15.7:0/1161570702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48781030f0 msgr2=0x7f4878074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:32:58.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.479+0000 7f487dee6700 1 --2- 172.21.15.7:0/1161570702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48781030f0 0x7f4878074cd0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f4860006010 tx=0x7f4860009f80 comp rx=0 tx=0).stop 2024-06-10T01:32:58.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.479+0000 7f487dee6700 1 -- 172.21.15.7:0/1161570702 shutdown_connections 2024-06-10T01:32:58.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.479+0000 7f487dee6700 1 --2- 172.21.15.7:0/1161570702 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f48640a2c20 0x7f48640a50e0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:58.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.479+0000 7f487dee6700 1 --2- 172.21.15.7:0/1161570702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4878107840 0x7f4878075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:58.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.479+0000 7f487dee6700 1 --2- 172.21.15.7:0/1161570702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4878102740 0x7f4878074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:58.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.479+0000 7f487dee6700 1 --2- 172.21.15.7:0/1161570702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48781030f0 0x7f4878074cd0 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:32:58.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.479+0000 7f487dee6700 1 -- 172.21.15.7:0/1161570702 >> 172.21.15.7:0/1161570702 conn(0x7f48780795f0 msgr2=0x7f48781004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:32:58.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.479+0000 7f487dee6700 1 -- 172.21.15.7:0/1161570702 shutdown_connections 2024-06-10T01:32:58.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:32:58.479+0000 7f487dee6700 1 -- 172.21.15.7:0/1161570702 wait complete. 2024-06-10T01:32:58.473 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 61 2024-06-10T01:32:59.265 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T01:32:59.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:59 smithi007 ceph-mon[32398]: pgmap v1243: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-10T01:32:59.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:32:59 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1161570702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:32:59.526 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:32:59.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:59 smithi078 ceph-mon[36683]: pgmap v1243: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-10T01:32:59.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:32:59 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1161570702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:32:59.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:59 smithi175 ceph-mon[36636]: pgmap v1243: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-10T01:32:59.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:32:59 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1161570702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:33:01.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:01 smithi007 ceph-mon[32398]: pgmap v1244: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-10T01:33:01.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.298+0000 7f2ff5923700 1 -- 172.21.15.7:0/1417323881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ff0101a40 msgr2=0x7f2ff0105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:33:01.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.298+0000 7f2ff5923700 1 --2- 172.21.15.7:0/1417323881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ff0101a40 0x7f2ff0105800 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4009990 tx=0x7f2fe4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:33:01.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.298+0000 7f2ff5923700 1 -- 172.21.15.7:0/1417323881 shutdown_connections 2024-06-10T01:33:01.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.298+0000 7f2ff5923700 1 --2- 172.21.15.7:0/1417323881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ff0105f30 0x7f2ff010da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:01.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.298+0000 7f2ff5923700 1 --2- 172.21.15.7:0/1417323881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ff0101090 0x7f2ff0101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:01.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.298+0000 7f2ff5923700 1 --2- 172.21.15.7:0/1417323881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ff0101a40 0x7f2ff0105800 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:01.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.298+0000 7f2ff5923700 1 -- 172.21.15.7:0/1417323881 >> 172.21.15.7:0/1417323881 conn(0x7f2ff00766b0 msgr2=0x7f2ff0076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:33:01.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.298+0000 7f2ff5923700 1 -- 172.21.15.7:0/1417323881 shutdown_connections 2024-06-10T01:33:01.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.298+0000 7f2ff5923700 1 -- 172.21.15.7:0/1417323881 wait complete. 2024-06-10T01:33:01.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.298+0000 7f2ff5923700 1 Processor -- start 2024-06-10T01:33:01.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.299+0000 7f2ff5923700 1 -- start start 2024-06-10T01:33:01.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.299+0000 7f2ff5923700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ff0101090 0x7f2ff0102830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:33:01.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.299+0000 7f2ff5923700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ff0101a40 0x7f2ff0102d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:33:01.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.299+0000 7f2ff5923700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ff0105f30 0x7f2ff01032b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:33:01.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.299+0000 7f2ff5923700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff01105f0 con 0x7f2ff0105f30 2024-06-10T01:33:01.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.299+0000 7f2ff5923700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff0110470 con 0x7f2ff0101090 2024-06-10T01:33:01.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.299+0000 7f2ff5923700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff0110770 con 0x7f2ff0101a40 2024-06-10T01:33:01.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.299+0000 7f2fef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ff0105f30 0x7f2ff01032b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:33:01.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.299+0000 7f2feeffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ff0101090 0x7f2ff0102830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:33:01.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.299+0000 7f2fef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ff0105f30 0x7f2ff01032b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47976/0 (socket says 172.21.15.7:47976) 2024-06-10T01:33:01.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.299+0000 7f2fef7fe700 1 -- 172.21.15.7:0/581950251 learned_addr learned my addr 172.21.15.7:0/581950251 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:33:01.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.299+0000 7f2fee7fc700 1 --2- 172.21.15.7:0/581950251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ff0101a40 0x7f2ff0102d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:33:01.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.299+0000 7f2fef7fe700 1 -- 172.21.15.7:0/581950251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ff0101a40 msgr2=0x7f2ff0102d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:33:01.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.299+0000 7f2fef7fe700 1 --2- 172.21.15.7:0/581950251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ff0101a40 0x7f2ff0102d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:01.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.299+0000 7f2fef7fe700 1 -- 172.21.15.7:0/581950251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ff0101090 msgr2=0x7f2ff0102830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:33:01.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.299+0000 7f2fef7fe700 1 --2- 172.21.15.7:0/581950251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ff0101090 0x7f2ff0102830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:01.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.299+0000 7f2fef7fe700 1 -- 172.21.15.7:0/581950251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff01a70d0 con 0x7f2ff0105f30 2024-06-10T01:33:01.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.299+0000 7f2fee7fc700 1 --2- 172.21.15.7:0/581950251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ff0101a40 0x7f2ff0102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:33:01.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.300+0000 7f2fef7fe700 1 --2- 172.21.15.7:0/581950251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ff0105f30 0x7f2ff01032b0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8007ee0 tx=0x7f2fe800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:33:01.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.300+0000 7f2ff4921700 1 -- 172.21.15.7:0/581950251 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe800dbd0 con 0x7f2ff0105f30 2024-06-10T01:33:01.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.300+0000 7f2ff5923700 1 -- 172.21.15.7:0/581950251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ff01a73c0 con 0x7f2ff0105f30 2024-06-10T01:33:01.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.300+0000 7f2ff4921700 1 -- 172.21.15.7:0/581950251 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fe800dd70 con 0x7f2ff0105f30 2024-06-10T01:33:01.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.300+0000 7f2ff4921700 1 -- 172.21.15.7:0/581950251 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8005a60 con 0x7f2ff0105f30 2024-06-10T01:33:01.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.300+0000 7f2ff5923700 1 -- 172.21.15.7:0/581950251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ff01a7880 con 0x7f2ff0105f30 2024-06-10T01:33:01.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.301+0000 7f2ff5923700 1 -- 172.21.15.7:0/581950251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ff0069180 con 0x7f2ff0105f30 2024-06-10T01:33:01.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.302+0000 7f2ff4921700 1 -- 172.21.15.7:0/581950251 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f2fe801a020 con 0x7f2ff0105f30 2024-06-10T01:33:01.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.303+0000 7f2ff4921700 1 --2- 172.21.15.7:0/581950251 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2fd80a2c20 0x7f2fd80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:33:01.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.303+0000 7f2feeffd700 1 --2- 172.21.15.7:0/581950251 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2fd80a2c20 0x7f2fd80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:33:01.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.303+0000 7f2ff4921700 1 -- 172.21.15.7:0/581950251 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f2fe80cead0 con 0x7f2ff0105f30 2024-06-10T01:33:01.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.303+0000 7f2feeffd700 1 --2- 172.21.15.7:0/581950251 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2fd80a2c20 0x7f2fd80a50e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0005950 tx=0x7f2fe000b510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:33:01.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.307+0000 7f2ff4921700 1 -- 172.21.15.7:0/581950251 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2fe8010070 con 0x7f2ff0105f30 2024-06-10T01:33:01.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.517+0000 7f2ff5923700 1 -- 172.21.15.7:0/581950251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ff002df00 con 0x7f2ff0105f30 2024-06-10T01:33:01.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.519+0000 7f2ff4921700 1 -- 172.21.15.7:0/581950251 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+24616 (secure 0 0 0) 0x7f2fe809b650 con 0x7f2ff0105f30 2024-06-10T01:33:01.512 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-10T01:33:01.513 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":102,"fsid":"bded58f8-26c3-11ef-bc9b-c7b262605968","created":"2024-06-10T00:53:31.966085+0000","modified":"2024-06-10T01:27:44.422698+0000","last_up_change":"2024-06-10T01:00:45.382473+0000","last_in_change":"2024-06-10T01:00:25.608513+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:56:59.964055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T01:01:42.563611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T01:01:46.175918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T01:01:55.695502+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f7c24c9f-b7ab-4c04-b2b4-120df9751667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6803","nonce":404485726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6805","nonce":404485726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6809","nonce":404485726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":404485726},{"type":"v1","addr":"172.21.15.7:6807","nonce":404485726}]},"public_addr":"172.21.15.7:6803/404485726","cluster_addr":"172.21.15.7:6805/404485726","heartbeat_back_addr":"172.21.15.7:6809/404485726","heartbeat_front_addr":"172.21.15.7:6807/404485726","state":["exists","up"]},{"osd":1,"uuid":"9bf70a6b-f31c-4077-b5af-229f17bc2845","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6801","nonce":1856804448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6803","nonce":1856804448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6807","nonce":1856804448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1856804448},{"type":"v1","addr":"172.21.15.78:6805","nonce":1856804448}]},"public_addr":"172.21.15.78:6801/1856804448","cluster_addr":"172.21.15.78:6803/1856804448","heartbeat_back_addr":"172.21.15.78:6807/1856804448","heartbeat_front_addr":"172.21.15.78:6805/1856804448","state":["exists","up"]},{"osd":2,"uuid":"62923c97-caf9-4f7e-9c14-9c9918843c78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6801","nonce":918257455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6803","nonce":918257455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6807","nonce":918257455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":918257455},{"type":"v1","addr":"172.21.15.175:6805","nonce":918257455}]},"public_addr":"172.21.15.175:6801/918257455","cluster_addr":"172.21.15.175:6803/918257455","heartbeat_back_addr":"172.21.15.175:6807/918257455","heartbeat_front_addr":"172.21.15.175:6805/918257455","state":["exists","up"]},{"osd":3,"uuid":"65242098-e748-48ab-8f0e-bb25b5c62742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6811","nonce":1797404966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6813","nonce":1797404966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6817","nonce":1797404966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1797404966},{"type":"v1","addr":"172.21.15.7:6815","nonce":1797404966}]},"public_addr":"172.21.15.7:6811/1797404966","cluster_addr":"172.21.15.7:6813/1797404966","heartbeat_back_addr":"172.21.15.7:6817/1797404966","heartbeat_front_addr":"172.21.15.7:6815/1797404966","state":["exists","up"]},{"osd":4,"uuid":"a7990305-de9e-400e-a432-d7854f73f339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6809","nonce":3767608357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6811","nonce":3767608357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6815","nonce":3767608357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3767608357},{"type":"v1","addr":"172.21.15.78:6813","nonce":3767608357}]},"public_addr":"172.21.15.78:6809/3767608357","cluster_addr":"172.21.15.78:6811/3767608357","heartbeat_back_addr":"172.21.15.78:6815/3767608357","heartbeat_front_addr":"172.21.15.78:6813/3767608357","state":["exists","up"]},{"osd":5,"uuid":"bad9a7b3-da5a-48d0-905d-e6a3190317fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6809","nonce":2157664731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6811","nonce":2157664731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6815","nonce":2157664731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2157664731},{"type":"v1","addr":"172.21.15.175:6813","nonce":2157664731}]},"public_addr":"172.21.15.175:6809/2157664731","cluster_addr":"172.21.15.175:6811/2157664731","heartbeat_back_addr":"172.21.15.175:6815/2157664731","heartbeat_front_addr":"172.21.15.175:6813/2157664731","state":["exists","up"]},{"osd":6,"uuid":"6c687653-157b-4b96-867b-a2eaa7d8ed8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6819","nonce":838731756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6821","nonce":838731756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6825","nonce":838731756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":838731756},{"type":"v1","addr":"172.21.15.7:6823","nonce":838731756}]},"public_addr":"172.21.15.7:6819/838731756","cluster_addr":"172.21.15.7:6821/838731756","heartbeat_back_addr":"172.21.15.7:6825/838731756","heartbeat_front_addr":"172.21.15.7:6823/838731756","state":["exists","up"]},{"osd":7,"uuid":"db40ddf1-9fd9-4859-b8c4-75f79582ab91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6817","nonce":3511921773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6819","nonce":3511921773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6823","nonce":3511921773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3511921773},{"type":"v1","addr":"172.21.15.78:6821","nonce":3511921773}]},"public_addr":"172.21.15.78:6817/3511921773","cluster_addr":"172.21.15.78:6819/3511921773","heartbeat_back_addr":"172.21.15.78:6823/3511921773","heartbeat_front_addr":"172.21.15.78:6821/3511921773","state":["exists","up"]},{"osd":8,"uuid":"0f01cbd9-b193-4c70-a33c-4984167bf422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6817","nonce":1546325211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6819","nonce":1546325211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6823","nonce":1546325211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1546325211},{"type":"v1","addr":"172.21.15.175:6821","nonce":1546325211}]},"public_addr":"172.21.15.175:6817/1546325211","cluster_addr":"172.21.15.175:6819/1546325211","heartbeat_back_addr":"172.21.15.175:6823/1546325211","heartbeat_front_addr":"172.21.15.175:6821/1546325211","state":["exists","up"]},{"osd":9,"uuid":"e5480998-f7fb-49cd-9aed-52405c88d830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6827","nonce":1844550813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6829","nonce":1844550813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6833","nonce":1844550813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1844550813},{"type":"v1","addr":"172.21.15.7:6831","nonce":1844550813}]},"public_addr":"172.21.15.7:6827/1844550813","cluster_addr":"172.21.15.7:6829/1844550813","heartbeat_back_addr":"172.21.15.7:6833/1844550813","heartbeat_front_addr":"172.21.15.7:6831/1844550813","state":["exists","up"]},{"osd":10,"uuid":"caa82f33-a71c-4907-bf3b-30a7fdfeb71d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6825","nonce":2290824785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6827","nonce":2290824785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6831","nonce":2290824785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2290824785},{"type":"v1","addr":"172.21.15.78:6829","nonce":2290824785}]},"public_addr":"172.21.15.78:6825/2290824785","cluster_addr":"172.21.15.78:6827/2290824785","heartbeat_back_addr":"172.21.15.78:6831/2290824785","heartbeat_front_addr":"172.21.15.78:6829/2290824785","state":["exists","up"]},{"osd":11,"uuid":"608cda77-77da-4da2-a7e2-76a773be27c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6825","nonce":1959979550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6827","nonce":1959979550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6831","nonce":1959979550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":1959979550},{"type":"v1","addr":"172.21.15.175:6829","nonce":1959979550}]},"public_addr":"172.21.15.175:6825/1959979550","cluster_addr":"172.21.15.175:6827/1959979550","heartbeat_back_addr":"172.21.15.175:6831/1959979550","heartbeat_front_addr":"172.21.15.175:6829/1959979550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:08.850203+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:32.820161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:56:56.633656+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:21.759372+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:57:46.735874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:10.636541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:58:36.990099+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:01.767098+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:26.882776+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:59:52.757595+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:18.383938+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T01:00:43.544330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1833397470":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/322543993":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/212510583":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6801/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:6800/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/2679436843":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6801/1100766262":"2024-06-11T00:54:08.102405+0000","172.21.15.7:0/1210355277":"2024-06-11T00:54:08.102405+0000","172.21.15.7:6800/3298829389":"2024-06-11T00:54:35.441123+0000","172.21.15.7:0/3610017406":"2024-06-11T00:54:08.102405+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T01:33:01.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.521+0000 7f2ff5923700 1 -- 172.21.15.7:0/581950251 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2fd80a2c20 msgr2=0x7f2fd80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:33:01.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.521+0000 7f2ff5923700 1 --2- 172.21.15.7:0/581950251 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2fd80a2c20 0x7f2fd80a50e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0005950 tx=0x7f2fe000b510 comp rx=0 tx=0).stop 2024-06-10T01:33:01.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.521+0000 7f2ff5923700 1 -- 172.21.15.7:0/581950251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ff0105f30 msgr2=0x7f2ff01032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:33:01.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.521+0000 7f2ff5923700 1 --2- 172.21.15.7:0/581950251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ff0105f30 0x7f2ff01032b0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8007ee0 tx=0x7f2fe800d3b0 comp rx=0 tx=0).stop 2024-06-10T01:33:01.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.521+0000 7f2ff5923700 1 -- 172.21.15.7:0/581950251 shutdown_connections 2024-06-10T01:33:01.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.521+0000 7f2ff5923700 1 --2- 172.21.15.7:0/581950251 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f2fd80a2c20 0x7f2fd80a50e0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:01.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.521+0000 7f2ff5923700 1 --2- 172.21.15.7:0/581950251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ff0105f30 0x7f2ff01032b0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:01.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.521+0000 7f2ff5923700 1 --2- 172.21.15.7:0/581950251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ff0101090 0x7f2ff0102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:01.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.521+0000 7f2ff5923700 1 --2- 172.21.15.7:0/581950251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ff0101a40 0x7f2ff0102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:01.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.521+0000 7f2ff5923700 1 -- 172.21.15.7:0/581950251 >> 172.21.15.7:0/581950251 conn(0x7f2ff00766b0 msgr2=0x7f2ff0104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:33:01.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.521+0000 7f2ff5923700 1 -- 172.21.15.7:0/581950251 shutdown_connections 2024-06-10T01:33:01.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:01.521+0000 7f2ff5923700 1 -- 172.21.15.7:0/581950251 wait complete. 2024-06-10T01:33:01.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:01 smithi078 ceph-mon[36683]: pgmap v1244: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-10T01:33:01.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:01 smithi175 ceph-mon[36636]: pgmap v1244: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-10T01:33:02.455 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs fail cephfs 2024-06-10T01:33:02.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:02 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/581950251' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:33:02.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:02 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/581950251' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:33:02.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:02 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/581950251' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:33:02.710 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:33:03.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:03 smithi007 ceph-mon[32398]: pgmap v1245: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-06-10T01:33:03.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:03 smithi078 ceph-mon[36683]: pgmap v1245: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-06-10T01:33:03.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:03 smithi175 ceph-mon[36636]: pgmap v1245: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-06-10T01:33:04.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.504+0000 7f4696e63700 1 -- 172.21.15.7:0/1594916505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f469006ab30 msgr2=0x7f469010f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:33:04.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.504+0000 7f4696e63700 1 --2- 172.21.15.7:0/1594916505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f469006ab30 0x7f469010f320 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f4688009990 tx=0x7f4688009ca0 comp rx=0 tx=0).stop 2024-06-10T01:33:04.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.504+0000 7f4696e63700 1 -- 172.21.15.7:0/1594916505 shutdown_connections 2024-06-10T01:33:04.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.504+0000 7f4696e63700 1 --2- 172.21.15.7:0/1594916505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f469006ab30 0x7f469010f320 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:04.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.504+0000 7f4696e63700 1 --2- 172.21.15.7:0/1594916505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4690108500 0x7f46901088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:04.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.504+0000 7f4696e63700 1 --2- 172.21.15.7:0/1594916505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f469006a170 0x7f469006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:04.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.504+0000 7f4696e63700 1 -- 172.21.15.7:0/1594916505 >> 172.21.15.7:0/1594916505 conn(0x7f4690078340 msgr2=0x7f4690078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:33:04.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.504+0000 7f4696e63700 1 -- 172.21.15.7:0/1594916505 shutdown_connections 2024-06-10T01:33:04.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.504+0000 7f4696e63700 1 -- 172.21.15.7:0/1594916505 wait complete. 2024-06-10T01:33:04.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.505+0000 7f4696e63700 1 Processor -- start 2024-06-10T01:33:04.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.505+0000 7f4696e63700 1 -- start start 2024-06-10T01:33:04.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.505+0000 7f4696e63700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f469006a170 0x7f4690101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:33:04.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.505+0000 7f4696e63700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f469006ab30 0x7f46901019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:33:04.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.505+0000 7f4696e63700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4690108500 0x7f4690101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:33:04.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.505+0000 7f4696e63700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f46901127e0 con 0x7f469006ab30 2024-06-10T01:33:04.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.505+0000 7f4696e63700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4690112660 con 0x7f4690108500 2024-06-10T01:33:04.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.505+0000 7f4696e63700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4690112960 con 0x7f469006a170 2024-06-10T01:33:04.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.506+0000 7f468ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f469006ab30 0x7f46901019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:33:04.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.506+0000 7f468ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f469006ab30 0x7f46901019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45986/0 (socket says 172.21.15.7:45986) 2024-06-10T01:33:04.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.506+0000 7f468ffff700 1 -- 172.21.15.7:0/2749667314 learned_addr learned my addr 172.21.15.7:0/2749667314 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:33:04.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.506+0000 7f4694bff700 1 --2- 172.21.15.7:0/2749667314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f469006a170 0x7f4690101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:33:04.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.506+0000 7f4695400700 1 --2- 172.21.15.7:0/2749667314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4690108500 0x7f4690101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:33:04.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.506+0000 7f468ffff700 1 -- 172.21.15.7:0/2749667314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f469006a170 msgr2=0x7f4690101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:33:04.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.506+0000 7f468ffff700 1 --2- 172.21.15.7:0/2749667314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f469006a170 0x7f4690101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:04.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.506+0000 7f468ffff700 1 -- 172.21.15.7:0/2749667314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4690108500 msgr2=0x7f4690101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:33:04.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.506+0000 7f468ffff700 1 --2- 172.21.15.7:0/2749667314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4690108500 0x7f4690101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:04.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.506+0000 7f468ffff700 1 -- 172.21.15.7:0/2749667314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46901a7140 con 0x7f469006ab30 2024-06-10T01:33:04.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.506+0000 7f4694bff700 1 --2- 172.21.15.7:0/2749667314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f469006a170 0x7f4690101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:33:04.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.506+0000 7f4695400700 1 --2- 172.21.15.7:0/2749667314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4690108500 0x7f4690101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:33:04.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.506+0000 7f468ffff700 1 --2- 172.21.15.7:0/2749667314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f469006ab30 0x7f46901019a0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f467c00b530 tx=0x7f467c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:33:04.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.506+0000 7f468dffb700 1 -- 172.21.15.7:0/2749667314 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467c015020 con 0x7f469006ab30 2024-06-10T01:33:04.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.506+0000 7f468dffb700 1 -- 172.21.15.7:0/2749667314 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f467c011d10 con 0x7f469006ab30 2024-06-10T01:33:04.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.506+0000 7f4696e63700 1 -- 172.21.15.7:0/2749667314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46901a7430 con 0x7f469006ab30 2024-06-10T01:33:04.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.506+0000 7f468dffb700 1 -- 172.21.15.7:0/2749667314 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467c010e50 con 0x7f469006ab30 2024-06-10T01:33:04.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.507+0000 7f4696e63700 1 -- 172.21.15.7:0/2749667314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46901a7970 con 0x7f469006ab30 2024-06-10T01:33:04.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.508+0000 7f4696e63700 1 -- 172.21.15.7:0/2749667314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4690103bc0 con 0x7f469006ab30 2024-06-10T01:33:04.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.509+0000 7f468dffb700 1 -- 172.21.15.7:0/2749667314 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f467c018020 con 0x7f469006ab30 2024-06-10T01:33:04.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.509+0000 7f468dffb700 1 --2- 172.21.15.7:0/2749667314 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f46800a2dc0 0x7f46800a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:33:04.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.510+0000 7f4694bff700 1 --2- 172.21.15.7:0/2749667314 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f46800a2dc0 0x7f46800a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:33:04.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.510+0000 7f468dffb700 1 -- 172.21.15.7:0/2749667314 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9129+0+0 (secure 0 0 0) 0x7f467c0cecb0 con 0x7f469006ab30 2024-06-10T01:33:04.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.510+0000 7f4694bff700 1 --2- 172.21.15.7:0/2749667314 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f46800a2dc0 0x7f46800a5280 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f468400e300 tx=0x7f4684009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:33:04.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.514+0000 7f468dffb700 1 -- 172.21.15.7:0/2749667314 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f467c01f020 con 0x7f469006ab30 2024-06-10T01:33:04.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.766+0000 7f4696e63700 1 -- 172.21.15.7:0/2749667314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4690102990 con 0x7f469006ab30 2024-06-10T01:33:04.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.769+0000 7f468dffb700 1 -- 172.21.15.7:0/2749667314 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v62) ==== 161+0+0 (secure 0 0 0) 0x7f467c09b830 con 0x7f469006ab30 2024-06-10T01:33:04.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.772+0000 7f4696e63700 1 -- 172.21.15.7:0/2749667314 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f46800a2dc0 msgr2=0x7f46800a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:33:04.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.772+0000 7f4696e63700 1 --2- 172.21.15.7:0/2749667314 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f46800a2dc0 0x7f46800a5280 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f468400e300 tx=0x7f4684009040 comp rx=0 tx=0).stop 2024-06-10T01:33:04.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.772+0000 7f4696e63700 1 -- 172.21.15.7:0/2749667314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f469006ab30 msgr2=0x7f46901019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:33:04.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.772+0000 7f4696e63700 1 --2- 172.21.15.7:0/2749667314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f469006ab30 0x7f46901019a0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f467c00b530 tx=0x7f467c00b8f0 comp rx=0 tx=0).stop 2024-06-10T01:33:04.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.772+0000 7f4696e63700 1 -- 172.21.15.7:0/2749667314 shutdown_connections 2024-06-10T01:33:04.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.772+0000 7f4696e63700 1 --2- 172.21.15.7:0/2749667314 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f46800a2dc0 0x7f46800a5280 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:04.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.772+0000 7f4696e63700 1 --2- 172.21.15.7:0/2749667314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4690108500 0x7f4690101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:04.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.772+0000 7f4696e63700 1 --2- 172.21.15.7:0/2749667314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f469006a170 0x7f4690101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:04.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.772+0000 7f4696e63700 1 --2- 172.21.15.7:0/2749667314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f469006ab30 0x7f46901019a0 secure :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f467c00b530 tx=0x7f467c00b8f0 comp rx=0 tx=0).stop 2024-06-10T01:33:04.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.772+0000 7f4696e63700 1 -- 172.21.15.7:0/2749667314 >> 172.21.15.7:0/2749667314 conn(0x7f4690078340 msgr2=0x7f46900ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:33:04.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.772+0000 7f4696e63700 1 -- 172.21.15.7:0/2749667314 shutdown_connections 2024-06-10T01:33:04.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:04.772+0000 7f4696e63700 1 -- 172.21.15.7:0/2749667314 wait complete. 2024-06-10T01:33:04.766 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-06-10T01:33:05.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:04 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-a[32394]: 2024-06-10T01:33:04.767+0000 7f9a2bc72700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-10T01:33:05.201 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jun 10 01:33:04 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-c[66487]: ignoring --setuser ceph since I am not root 2024-06-10T01:33:05.201 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jun 10 01:33:04 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-c[66487]: ignoring --setgroup ceph since I am not root 2024-06-10T01:33:05.201 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jun 10 01:33:04 smithi175 ceph-mds[66491]: -- 172.21.15.175:0/2986933705 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55af68233a40 con 0x55af68206c00 2024-06-10T01:33:05.201 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jun 10 01:33:04 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-c[66487]: starting mds.c at 2024-06-10T01:33:05.201 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jun 10 01:33:04 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-i[66802]: ignoring --setuser ceph since I am not root 2024-06-10T01:33:05.201 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jun 10 01:33:04 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-i[66802]: ignoring --setgroup ceph since I am not root 2024-06-10T01:33:05.201 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jun 10 01:33:04 smithi175 ceph-mds[66806]: -- 172.21.15.175:0/189009609 <== mon.2 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5579fdbb7a40 con 0x5579fdb8a800 2024-06-10T01:33:05.202 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jun 10 01:33:04 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-i[66802]: starting mds.i at 2024-06-10T01:33:05.202 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jun 10 01:33:04 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-l[65870]: ignoring --setuser ceph since I am not root 2024-06-10T01:33:05.202 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jun 10 01:33:04 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-l[65870]: ignoring --setgroup ceph since I am not root 2024-06-10T01:33:05.202 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jun 10 01:33:04 smithi175 ceph-mds[65875]: -- 172.21.15.175:0/406594093 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5590e370da40 con 0x5590e36e0800 2024-06-10T01:33:05.202 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jun 10 01:33:04 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-l[65870]: starting mds.l at 2024-06-10T01:33:05.202 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:33:04 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-f[66185]: ignoring --setuser ceph since I am not root 2024-06-10T01:33:05.202 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:33:04 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-f[66185]: ignoring --setgroup ceph since I am not root 2024-06-10T01:33:05.202 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:33:04 smithi175 ceph-mds[66189]: -- 172.21.15.175:0/3415075369 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5560a8e49a40 con 0x5560a8e1d000 2024-06-10T01:33:05.203 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:33:04 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-f[66185]: starting mds.f at 2024-06-10T01:33:05.260 INFO:journalctl@ceph.mds.h.smithi078.stdout:Jun 10 01:33:04 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-h[66154]: ignoring --setuser ceph since I am not root 2024-06-10T01:33:05.260 INFO:journalctl@ceph.mds.h.smithi078.stdout:Jun 10 01:33:04 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-h[66154]: ignoring --setgroup ceph since I am not root 2024-06-10T01:33:05.260 INFO:journalctl@ceph.mds.h.smithi078.stdout:Jun 10 01:33:04 smithi078 ceph-mds[66158]: -- 172.21.15.78:0/1502426761 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559e335afa40 con 0x559e33582c00 2024-06-10T01:33:05.260 INFO:journalctl@ceph.mds.h.smithi078.stdout:Jun 10 01:33:04 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-h[66154]: starting mds.h at 2024-06-10T01:33:05.260 INFO:journalctl@ceph.mds.b.smithi078.stdout:Jun 10 01:33:04 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-b[66458]: ignoring --setuser ceph since I am not root 2024-06-10T01:33:05.261 INFO:journalctl@ceph.mds.b.smithi078.stdout:Jun 10 01:33:04 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-b[66458]: ignoring --setgroup ceph since I am not root 2024-06-10T01:33:05.261 INFO:journalctl@ceph.mds.b.smithi078.stdout:Jun 10 01:33:04 smithi078 ceph-mds[66462]: -- 172.21.15.78:0/3940468090 <== mon.2 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56444ba2fa40 con 0x56444ba03000 2024-06-10T01:33:05.261 INFO:journalctl@ceph.mds.b.smithi078.stdout:Jun 10 01:33:04 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-b[66458]: starting mds.b at 2024-06-10T01:33:05.474 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:05 smithi007 ceph-mon[32398]: pgmap v1246: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-10T01:33:05.475 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:05 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2749667314' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-10T01:33:05.475 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:05 smithi007 ceph-mon[32398]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-10T01:33:05.475 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:05 smithi007 ceph-mon[32398]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-10T01:33:05.475 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:05 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/2749667314' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-10T01:33:05.475 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:05 smithi007 ceph-mon[32398]: osdmap e103: 12 total, 12 up, 12 in 2024-06-10T01:33:05.475 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:05 smithi007 ceph-mon[32398]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-06-10T01:33:05.510 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-06-10T01:33:05.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:05 smithi078 ceph-mon[36683]: pgmap v1246: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-10T01:33:05.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:05 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2749667314' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-10T01:33:05.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:05 smithi078 ceph-mon[36683]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-10T01:33:05.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:05 smithi078 ceph-mon[36683]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-10T01:33:05.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:05 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/2749667314' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-10T01:33:05.578 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:05 smithi078 ceph-mon[36683]: osdmap e103: 12 total, 12 up, 12 in 2024-06-10T01:33:05.578 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:05 smithi078 ceph-mon[36683]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-06-10T01:33:05.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:05 smithi175 ceph-mon[36636]: pgmap v1246: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-10T01:33:05.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:05 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2749667314' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-10T01:33:05.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:05 smithi175 ceph-mon[36636]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-10T01:33:05.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:05 smithi175 ceph-mon[36636]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-10T01:33:05.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:05 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/2749667314' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-10T01:33:05.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:05 smithi175 ceph-mon[36636]: osdmap e103: 12 total, 12 up, 12 in 2024-06-10T01:33:05.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:05 smithi175 ceph-mon[36636]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-06-10T01:33:05.772 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:33:06.900 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:06 smithi007 ceph-mon[32398]: mds.? [v2:172.21.15.175:6833/2336052222,v1:172.21.15.175:6835/2336052222] up:boot 2024-06-10T01:33:06.900 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:06 smithi007 ceph-mon[32398]: mds.? [v2:172.21.15.175:6832/542222607,v1:172.21.15.175:6834/542222607] up:boot 2024-06-10T01:33:06.900 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:06 smithi007 ceph-mon[32398]: mds.? [v2:172.21.15.175:6836/1564174788,v1:172.21.15.175:6838/1564174788] up:boot 2024-06-10T01:33:06.900 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:06 smithi007 ceph-mon[32398]: mds.? [v2:172.21.15.175:6837/2629228083,v1:172.21.15.175:6839/2629228083] up:boot 2024-06-10T01:33:06.900 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:06 smithi007 ceph-mon[32398]: mds.? [v2:172.21.15.78:6837/1024094632,v1:172.21.15.78:6839/1024094632] up:boot 2024-06-10T01:33:06.900 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:06 smithi007 ceph-mon[32398]: mds.? [v2:172.21.15.78:6836/662787632,v1:172.21.15.78:6838/662787632] up:boot 2024-06-10T01:33:06.900 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:06 smithi007 ceph-mon[32398]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-06-10T01:33:06.900 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:06 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-10T01:33:06.900 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:06 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-10T01:33:06.900 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:06 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-10T01:33:06.900 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:06 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T01:33:06.901 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:06 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-10T01:33:06.901 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:06 smithi007 ceph-mon[32398]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-10T01:33:06.901 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:06 smithi007 ceph-mon[32398]: pgmap v1248: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-10T01:33:07.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:06 smithi078 ceph-mon[36683]: mds.? [v2:172.21.15.175:6833/2336052222,v1:172.21.15.175:6835/2336052222] up:boot 2024-06-10T01:33:07.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:06 smithi078 ceph-mon[36683]: mds.? [v2:172.21.15.175:6832/542222607,v1:172.21.15.175:6834/542222607] up:boot 2024-06-10T01:33:07.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:06 smithi078 ceph-mon[36683]: mds.? [v2:172.21.15.175:6836/1564174788,v1:172.21.15.175:6838/1564174788] up:boot 2024-06-10T01:33:07.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:06 smithi078 ceph-mon[36683]: mds.? [v2:172.21.15.175:6837/2629228083,v1:172.21.15.175:6839/2629228083] up:boot 2024-06-10T01:33:07.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:06 smithi078 ceph-mon[36683]: mds.? [v2:172.21.15.78:6837/1024094632,v1:172.21.15.78:6839/1024094632] up:boot 2024-06-10T01:33:07.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:06 smithi078 ceph-mon[36683]: mds.? [v2:172.21.15.78:6836/662787632,v1:172.21.15.78:6838/662787632] up:boot 2024-06-10T01:33:07.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:06 smithi078 ceph-mon[36683]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-06-10T01:33:07.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:06 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-10T01:33:07.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:06 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-10T01:33:07.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:06 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-10T01:33:07.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:06 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T01:33:07.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:06 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-10T01:33:07.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:06 smithi078 ceph-mon[36683]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-10T01:33:07.079 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:06 smithi078 ceph-mon[36683]: pgmap v1248: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-10T01:33:07.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:06 smithi175 ceph-mon[36636]: mds.? [v2:172.21.15.175:6833/2336052222,v1:172.21.15.175:6835/2336052222] up:boot 2024-06-10T01:33:07.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:06 smithi175 ceph-mon[36636]: mds.? [v2:172.21.15.175:6832/542222607,v1:172.21.15.175:6834/542222607] up:boot 2024-06-10T01:33:07.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:06 smithi175 ceph-mon[36636]: mds.? [v2:172.21.15.175:6836/1564174788,v1:172.21.15.175:6838/1564174788] up:boot 2024-06-10T01:33:07.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:06 smithi175 ceph-mon[36636]: mds.? [v2:172.21.15.175:6837/2629228083,v1:172.21.15.175:6839/2629228083] up:boot 2024-06-10T01:33:07.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:06 smithi175 ceph-mon[36636]: mds.? [v2:172.21.15.78:6837/1024094632,v1:172.21.15.78:6839/1024094632] up:boot 2024-06-10T01:33:07.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:06 smithi175 ceph-mon[36636]: mds.? [v2:172.21.15.78:6836/662787632,v1:172.21.15.78:6838/662787632] up:boot 2024-06-10T01:33:07.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:06 smithi175 ceph-mon[36636]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-06-10T01:33:07.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:06 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-10T01:33:07.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:06 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-10T01:33:07.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:06 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-10T01:33:07.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:06 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T01:33:07.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:06 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-10T01:33:07.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:06 smithi175 ceph-mon[36636]: from='mgr.14152 172.21.15.7:0/2446803447' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-10T01:33:07.202 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:06 smithi175 ceph-mon[36636]: pgmap v1248: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-10T01:33:07.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.588+0000 7feda3b68700 1 -- 172.21.15.7:0/24354523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed9c1030f0 msgr2=0x7fed9c107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:33:07.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.588+0000 7feda3b68700 1 --2- 172.21.15.7:0/24354523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed9c1030f0 0x7fed9c107110 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fed8c0099c0 tx=0x7fed8c009cd0 comp rx=0 tx=0).stop 2024-06-10T01:33:07.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.588+0000 7feda3b68700 1 -- 172.21.15.7:0/24354523 shutdown_connections 2024-06-10T01:33:07.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.588+0000 7feda3b68700 1 --2- 172.21.15.7:0/24354523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed9c107840 0x7fed9c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:07.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.588+0000 7feda3b68700 1 --2- 172.21.15.7:0/24354523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed9c102740 0x7fed9c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:07.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.588+0000 7feda3b68700 1 --2- 172.21.15.7:0/24354523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed9c1030f0 0x7fed9c107110 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:07.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.588+0000 7feda3b68700 1 -- 172.21.15.7:0/24354523 >> 172.21.15.7:0/24354523 conn(0x7fed9c0795f0 msgr2=0x7fed9c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:33:07.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.589+0000 7feda3b68700 1 -- 172.21.15.7:0/24354523 shutdown_connections 2024-06-10T01:33:07.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.589+0000 7feda3b68700 1 -- 172.21.15.7:0/24354523 wait complete. 2024-06-10T01:33:07.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.589+0000 7feda3b68700 1 Processor -- start 2024-06-10T01:33:07.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.590+0000 7feda3b68700 1 -- start start 2024-06-10T01:33:07.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.590+0000 7feda3b68700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed9c102740 0x7fed9c19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:33:07.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.590+0000 7feda3b68700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed9c1030f0 0x7fed9c19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:33:07.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.590+0000 7feda3b68700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed9c107840 0x7fed9c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:33:07.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.590+0000 7feda3b68700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c1127d0 con 0x7fed9c102740 2024-06-10T01:33:07.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.590+0000 7feda3b68700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c112650 con 0x7fed9c107840 2024-06-10T01:33:07.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.590+0000 7feda3b68700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c112950 con 0x7fed9c1030f0 2024-06-10T01:33:07.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.590+0000 7feda1904700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed9c102740 0x7fed9c19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:33:07.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.590+0000 7feda1904700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed9c102740 0x7fed9c19e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46014/0 (socket says 172.21.15.7:46014) 2024-06-10T01:33:07.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.590+0000 7feda1904700 1 -- 172.21.15.7:0/1331110959 learned_addr learned my addr 172.21.15.7:0/1331110959 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:33:07.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.590+0000 7feda2105700 1 --2- 172.21.15.7:0/1331110959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed9c107840 0x7fed9c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:33:07.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.590+0000 7feda1103700 1 --2- 172.21.15.7:0/1331110959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed9c1030f0 0x7fed9c19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:33:07.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.590+0000 7feda1904700 1 -- 172.21.15.7:0/1331110959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed9c1030f0 msgr2=0x7fed9c19ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:33:07.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.590+0000 7feda1904700 1 --2- 172.21.15.7:0/1331110959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed9c1030f0 0x7fed9c19ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:07.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.590+0000 7feda1904700 1 -- 172.21.15.7:0/1331110959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed9c107840 msgr2=0x7fed9c198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:33:07.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.590+0000 7feda1904700 1 --2- 172.21.15.7:0/1331110959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed9c107840 0x7fed9c198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:07.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.590+0000 7feda1904700 1 -- 172.21.15.7:0/1331110959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed9c199250 con 0x7fed9c102740 2024-06-10T01:33:07.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.591+0000 7feda1904700 1 --2- 172.21.15.7:0/1331110959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed9c102740 0x7fed9c19e8e0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fed9800b530 tx=0x7fed9800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:33:07.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.591+0000 7fed92ffd700 1 -- 172.21.15.7:0/1331110959 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed98015020 con 0x7fed9c102740 2024-06-10T01:33:07.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.591+0000 7fed92ffd700 1 -- 172.21.15.7:0/1331110959 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed98011d10 con 0x7fed9c102740 2024-06-10T01:33:07.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.591+0000 7feda3b68700 1 -- 172.21.15.7:0/1331110959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed9c199540 con 0x7fed9c102740 2024-06-10T01:33:07.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.591+0000 7fed92ffd700 1 -- 172.21.15.7:0/1331110959 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9800f2e0 con 0x7fed9c102740 2024-06-10T01:33:07.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.591+0000 7feda3b68700 1 -- 172.21.15.7:0/1331110959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed9c1a57b0 con 0x7fed9c102740 2024-06-10T01:33:07.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.592+0000 7feda3b68700 1 -- 172.21.15.7:0/1331110959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed9c10aa80 con 0x7fed9c102740 2024-06-10T01:33:07.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.593+0000 7fed92ffd700 1 -- 172.21.15.7:0/1331110959 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fed98018020 con 0x7fed9c102740 2024-06-10T01:33:07.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.594+0000 7fed92ffd700 1 --2- 172.21.15.7:0/1331110959 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fed880a2ca0 0x7fed880a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:33:07.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.594+0000 7feda1103700 1 --2- 172.21.15.7:0/1331110959 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fed880a2ca0 0x7fed880a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:33:07.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.594+0000 7fed92ffd700 1 -- 172.21.15.7:0/1331110959 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9387+0+0 (secure 0 0 0) 0x7fed980ceb40 con 0x7fed9c102740 2024-06-10T01:33:07.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.595+0000 7feda1103700 1 --2- 172.21.15.7:0/1331110959 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fed880a2ca0 0x7fed880a5160 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fed9c102030 tx=0x7fed8c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:33:07.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.598+0000 7fed92ffd700 1 -- 172.21.15.7:0/1331110959 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fed9801f020 con 0x7fed9c102740 2024-06-10T01:33:07.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.854+0000 7feda3b68700 1 -- 172.21.15.7:0/1331110959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fed9c199fd0 con 0x7fed9c102740 2024-06-10T01:33:07.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.906+0000 7fed92ffd700 1 -- 172.21.15.7:0/1331110959 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v64) ==== 104+0+0 (secure 0 0 0) 0x7fed9809b620 con 0x7fed9c102740 2024-06-10T01:33:07.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.908+0000 7feda3b68700 1 -- 172.21.15.7:0/1331110959 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fed880a2ca0 msgr2=0x7fed880a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:33:07.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.908+0000 7feda3b68700 1 --2- 172.21.15.7:0/1331110959 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fed880a2ca0 0x7fed880a5160 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fed9c102030 tx=0x7fed8c005fb0 comp rx=0 tx=0).stop 2024-06-10T01:33:07.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.908+0000 7feda3b68700 1 -- 172.21.15.7:0/1331110959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed9c102740 msgr2=0x7fed9c19e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:33:07.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.908+0000 7feda3b68700 1 --2- 172.21.15.7:0/1331110959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed9c102740 0x7fed9c19e8e0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fed9800b530 tx=0x7fed9800b840 comp rx=0 tx=0).stop 2024-06-10T01:33:07.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.908+0000 7feda3b68700 1 -- 172.21.15.7:0/1331110959 shutdown_connections 2024-06-10T01:33:07.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.908+0000 7feda3b68700 1 --2- 172.21.15.7:0/1331110959 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fed880a2ca0 0x7fed880a5160 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:07.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.908+0000 7feda3b68700 1 --2- 172.21.15.7:0/1331110959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed9c107840 0x7fed9c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:07.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.908+0000 7feda3b68700 1 --2- 172.21.15.7:0/1331110959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed9c102740 0x7fed9c19e8e0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:07.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.908+0000 7feda3b68700 1 --2- 172.21.15.7:0/1331110959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed9c1030f0 0x7fed9c19ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:07.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.908+0000 7feda3b68700 1 -- 172.21.15.7:0/1331110959 >> 172.21.15.7:0/1331110959 conn(0x7fed9c0795f0 msgr2=0x7fed9c108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:33:07.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.909+0000 7feda3b68700 1 -- 172.21.15.7:0/1331110959 shutdown_connections 2024-06-10T01:33:07.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:07.909+0000 7feda3b68700 1 -- 172.21.15.7:0/1331110959 wait complete. 2024-06-10T01:33:08.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:07 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1331110959' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-10T01:33:08.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:07 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1331110959' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-10T01:33:08.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:07 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1331110959' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-10T01:33:08.667 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-06-10T01:33:08.923 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:33:09.196 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:08 smithi007 ceph-mon[32398]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-10T01:33:09.196 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:08 smithi007 ceph-mon[32398]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-10T01:33:09.196 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:08 smithi007 ceph-mon[32398]: Cluster is now healthy 2024-06-10T01:33:09.196 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:08 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1331110959' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-10T01:33:09.196 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:08 smithi007 ceph-mon[32398]: fsmap 12 up:standby 2024-06-10T01:33:09.196 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:08 smithi007 ceph-mon[32398]: pgmap v1249: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-10T01:33:09.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:08 smithi175 ceph-mon[36636]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-10T01:33:09.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:08 smithi175 ceph-mon[36636]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-10T01:33:09.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:08 smithi175 ceph-mon[36636]: Cluster is now healthy 2024-06-10T01:33:09.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:08 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1331110959' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-10T01:33:09.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:08 smithi175 ceph-mon[36636]: fsmap 12 up:standby 2024-06-10T01:33:09.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:08 smithi175 ceph-mon[36636]: pgmap v1249: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-10T01:33:09.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:08 smithi078 ceph-mon[36683]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-10T01:33:09.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:08 smithi078 ceph-mon[36683]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-10T01:33:09.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:08 smithi078 ceph-mon[36683]: Cluster is now healthy 2024-06-10T01:33:09.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:08 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1331110959' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-10T01:33:09.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:08 smithi078 ceph-mon[36683]: fsmap 12 up:standby 2024-06-10T01:33:09.328 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:08 smithi078 ceph-mon[36683]: pgmap v1249: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-10T01:33:10.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.558+0000 7f5fe38be700 1 -- 172.21.15.7:0/124859147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fdc101390 msgr2=0x7f5fdc101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:33:10.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.558+0000 7f5fe38be700 1 --2- 172.21.15.7:0/124859147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fdc101390 0x7f5fdc101810 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009990 tx=0x7f5fcc009ca0 comp rx=0 tx=0).stop 2024-06-10T01:33:10.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.558+0000 7f5fe38be700 1 -- 172.21.15.7:0/124859147 shutdown_connections 2024-06-10T01:33:10.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.558+0000 7f5fe38be700 1 --2- 172.21.15.7:0/124859147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fdc101d50 0x7f5fdc10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:10.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.558+0000 7f5fe38be700 1 --2- 172.21.15.7:0/124859147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fdc107400 0x7f5fdc1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:10.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.558+0000 7f5fe38be700 1 --2- 172.21.15.7:0/124859147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fdc101390 0x7f5fdc101810 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:10.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.558+0000 7f5fe38be700 1 -- 172.21.15.7:0/124859147 >> 172.21.15.7:0/124859147 conn(0x7f5fdc078340 msgr2=0x7f5fdc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:33:10.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.559+0000 7f5fe38be700 1 -- 172.21.15.7:0/124859147 shutdown_connections 2024-06-10T01:33:10.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.559+0000 7f5fe38be700 1 -- 172.21.15.7:0/124859147 wait complete. 2024-06-10T01:33:10.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.559+0000 7f5fe38be700 1 Processor -- start 2024-06-10T01:33:10.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.559+0000 7f5fe38be700 1 -- start start 2024-06-10T01:33:10.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.560+0000 7f5fe38be700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fdc101390 0x7f5fdc103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:33:10.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.560+0000 7f5fe38be700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fdc101d50 0x7f5fdc1035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:33:10.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.560+0000 7f5fe38be700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fdc107400 0x7f5fdc103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:33:10.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.560+0000 7f5fe38be700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc1116a0 con 0x7f5fdc101d50 2024-06-10T01:33:10.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.560+0000 7f5fe38be700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc111520 con 0x7f5fdc107400 2024-06-10T01:33:10.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.560+0000 7f5fe0e59700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fdc101d50 0x7f5fdc1035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:33:10.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.560+0000 7f5fe38be700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc111820 con 0x7f5fdc101390 2024-06-10T01:33:10.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.560+0000 7f5fe0e59700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fdc101d50 0x7f5fdc1035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46020/0 (socket says 172.21.15.7:46020) 2024-06-10T01:33:10.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.560+0000 7f5fe0e59700 1 -- 172.21.15.7:0/948927495 learned_addr learned my addr 172.21.15.7:0/948927495 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:33:10.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.560+0000 7f5fe1e5b700 1 --2- 172.21.15.7:0/948927495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fdc107400 0x7f5fdc103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:33:10.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.560+0000 7f5fe165a700 1 --2- 172.21.15.7:0/948927495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fdc101390 0x7f5fdc103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:33:10.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.560+0000 7f5fe0e59700 1 -- 172.21.15.7:0/948927495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fdc101390 msgr2=0x7f5fdc103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:33:10.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.560+0000 7f5fe0e59700 1 --2- 172.21.15.7:0/948927495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fdc101390 0x7f5fdc103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:10.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.560+0000 7f5fe0e59700 1 -- 172.21.15.7:0/948927495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fdc107400 msgr2=0x7f5fdc103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:33:10.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.560+0000 7f5fe0e59700 1 --2- 172.21.15.7:0/948927495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fdc107400 0x7f5fdc103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:10.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.560+0000 7f5fe0e59700 1 -- 172.21.15.7:0/948927495 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fdc1a70f0 con 0x7f5fdc101d50 2024-06-10T01:33:10.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.560+0000 7f5fe0e59700 1 --2- 172.21.15.7:0/948927495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fdc101d50 0x7f5fdc1035b0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc000c00 tx=0x7f5fcc003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:33:10.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.561+0000 7f5fd27fc700 1 -- 172.21.15.7:0/948927495 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc027070 con 0x7f5fdc101d50 2024-06-10T01:33:10.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.561+0000 7f5fd27fc700 1 -- 172.21.15.7:0/948927495 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fcc004050 con 0x7f5fdc101d50 2024-06-10T01:33:10.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.561+0000 7f5fe38be700 1 -- 172.21.15.7:0/948927495 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fdc1a7380 con 0x7f5fdc101d50 2024-06-10T01:33:10.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.561+0000 7f5fd27fc700 1 -- 172.21.15.7:0/948927495 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc00f460 con 0x7f5fdc101d50 2024-06-10T01:33:10.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.561+0000 7f5fe38be700 1 -- 172.21.15.7:0/948927495 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fdc1a77d0 con 0x7f5fdc101d50 2024-06-10T01:33:10.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.562+0000 7f5fe38be700 1 -- 172.21.15.7:0/948927495 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fdc069180 con 0x7f5fdc101d50 2024-06-10T01:33:10.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.563+0000 7f5fd27fc700 1 -- 172.21.15.7:0/948927495 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f5fcc02a070 con 0x7f5fdc101d50 2024-06-10T01:33:10.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.563+0000 7f5fd27fc700 1 --2- 172.21.15.7:0/948927495 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5fc80a2c20 0x7f5fc80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:33:10.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.564+0000 7f5fe165a700 1 --2- 172.21.15.7:0/948927495 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5fc80a2c20 0x7f5fc80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:33:10.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.564+0000 7f5fd27fc700 1 -- 172.21.15.7:0/948927495 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9387+0+0 (secure 0 0 0) 0x7f5fcc0d0970 con 0x7f5fdc101d50 2024-06-10T01:33:10.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.564+0000 7f5fe165a700 1 --2- 172.21.15.7:0/948927495 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5fc80a2c20 0x7f5fc80a50e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800abd0 tx=0x7f5fd800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:33:10.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.567+0000 7f5fd27fc700 1 -- 172.21.15.7:0/948927495 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5fcc031070 con 0x7f5fdc101d50 2024-06-10T01:33:10.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:10.774+0000 7f5fe38be700 1 -- 172.21.15.7:0/948927495 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5fdc1a7af0 con 0x7f5fdc101d50 2024-06-10T01:33:11.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:11.152+0000 7f5fd27fc700 1 -- 172.21.15.7:0/948927495 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v104) ==== 181+0+0 (secure 0 0 0) 0x7f5fcc013070 con 0x7f5fdc101d50 2024-06-10T01:33:11.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:11.156+0000 7f5fe38be700 1 -- 172.21.15.7:0/948927495 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5fc80a2c20 msgr2=0x7f5fc80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:33:11.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:11.156+0000 7f5fe38be700 1 --2- 172.21.15.7:0/948927495 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5fc80a2c20 0x7f5fc80a50e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800abd0 tx=0x7f5fd800a380 comp rx=0 tx=0).stop 2024-06-10T01:33:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:11.156+0000 7f5fe38be700 1 -- 172.21.15.7:0/948927495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fdc101d50 msgr2=0x7f5fdc1035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:33:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:11.156+0000 7f5fe38be700 1 --2- 172.21.15.7:0/948927495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fdc101d50 0x7f5fdc1035b0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc000c00 tx=0x7f5fcc003680 comp rx=0 tx=0).stop 2024-06-10T01:33:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:11.156+0000 7f5fe38be700 1 -- 172.21.15.7:0/948927495 shutdown_connections 2024-06-10T01:33:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:11.156+0000 7f5fe38be700 1 --2- 172.21.15.7:0/948927495 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f5fc80a2c20 0x7f5fc80a50e0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:11.156+0000 7f5fe38be700 1 --2- 172.21.15.7:0/948927495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fdc107400 0x7f5fdc103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:11.156+0000 7f5fe38be700 1 --2- 172.21.15.7:0/948927495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fdc101390 0x7f5fdc103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:11.156+0000 7f5fe38be700 1 --2- 172.21.15.7:0/948927495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fdc101d50 0x7f5fdc1035b0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:11.156+0000 7f5fe38be700 1 -- 172.21.15.7:0/948927495 >> 172.21.15.7:0/948927495 conn(0x7f5fdc078340 msgr2=0x7f5fdc1005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:33:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:11.157+0000 7f5fe38be700 1 -- 172.21.15.7:0/948927495 shutdown_connections 2024-06-10T01:33:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:11.157+0000 7f5fe38be700 1 -- 172.21.15.7:0/948927495 wait complete. 2024-06-10T01:33:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-06-10T01:33:11.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:11 smithi175 ceph-mon[36636]: pgmap v1250: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-10T01:33:11.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:11 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/948927495' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T01:33:11.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:11 smithi007 ceph-mon[32398]: pgmap v1250: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-10T01:33:11.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:11 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/948927495' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T01:33:11.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:11 smithi078 ceph-mon[36683]: pgmap v1250: 129 pgs: 129 active+clean; 472 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-10T01:33:11.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:11 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/948927495' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T01:33:11.824 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-06-10T01:33:12.076 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:33:12.440 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:12 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/948927495' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T01:33:12.440 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:12 smithi007 ceph-mon[32398]: osdmap e104: 12 total, 12 up, 12 in 2024-06-10T01:33:12.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:12 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/948927495' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T01:33:12.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:12 smithi175 ceph-mon[36636]: osdmap e104: 12 total, 12 up, 12 in 2024-06-10T01:33:12.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:12 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/948927495' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T01:33:12.578 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:12 smithi078 ceph-mon[36683]: osdmap e104: 12 total, 12 up, 12 in 2024-06-10T01:33:13.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:13 smithi007 ceph-mon[32398]: pgmap v1252: 97 pgs: 97 active+clean; 577 KiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:33:13.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:13 smithi175 ceph-mon[36636]: pgmap v1252: 97 pgs: 97 active+clean; 577 KiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:33:13.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:13 smithi078 ceph-mon[36683]: pgmap v1252: 97 pgs: 97 active+clean; 577 KiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:33:14.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.354+0000 7fd077173700 1 -- 172.21.15.7:0/1283510370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd070101d50 msgr2=0x7fd07010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:33:14.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.354+0000 7fd077173700 1 --2- 172.21.15.7:0/1283510370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd070101d50 0x7fd07010e240 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fd06800b0a0 tx=0x7fd06800b3b0 comp rx=0 tx=0).stop 2024-06-10T01:33:14.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.354+0000 7fd077173700 1 -- 172.21.15.7:0/1283510370 shutdown_connections 2024-06-10T01:33:14.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.354+0000 7fd077173700 1 --2- 172.21.15.7:0/1283510370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd070101d50 0x7fd07010e240 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:14.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.354+0000 7fd077173700 1 --2- 172.21.15.7:0/1283510370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd070107400 0x7fd0701077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:14.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.354+0000 7fd077173700 1 --2- 172.21.15.7:0/1283510370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd070101390 0x7fd070101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:14.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.354+0000 7fd077173700 1 -- 172.21.15.7:0/1283510370 >> 172.21.15.7:0/1283510370 conn(0x7fd070078340 msgr2=0x7fd070078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:33:14.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.354+0000 7fd077173700 1 -- 172.21.15.7:0/1283510370 shutdown_connections 2024-06-10T01:33:14.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.354+0000 7fd077173700 1 -- 172.21.15.7:0/1283510370 wait complete. 2024-06-10T01:33:14.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.355+0000 7fd077173700 1 Processor -- start 2024-06-10T01:33:14.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.355+0000 7fd077173700 1 -- start start 2024-06-10T01:33:14.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.355+0000 7fd077173700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd070101390 0x7fd07019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:33:14.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.355+0000 7fd077173700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd070101d50 0x7fd07019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:33:14.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.355+0000 7fd077173700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd070107400 0x7fd070198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:33:14.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.355+0000 7fd077173700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd0701116f0 con 0x7fd070101d50 2024-06-10T01:33:14.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.355+0000 7fd077173700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd070111570 con 0x7fd070101390 2024-06-10T01:33:14.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.355+0000 7fd077173700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd070111870 con 0x7fd070107400 2024-06-10T01:33:14.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.355+0000 7fd06ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd070101d50 0x7fd07019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:33:14.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.355+0000 7fd06ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd070101d50 0x7fd07019ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50660/0 (socket says 172.21.15.7:50660) 2024-06-10T01:33:14.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.355+0000 7fd06ffff700 1 -- 172.21.15.7:0/805905639 learned_addr learned my addr 172.21.15.7:0/805905639 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:33:14.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.356+0000 7fd074f0f700 1 --2- 172.21.15.7:0/805905639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd070101390 0x7fd07019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:33:14.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.356+0000 7fd075710700 1 --2- 172.21.15.7:0/805905639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd070107400 0x7fd070198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:33:14.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.356+0000 7fd06ffff700 1 -- 172.21.15.7:0/805905639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd070107400 msgr2=0x7fd070198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:33:14.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.356+0000 7fd06ffff700 1 --2- 172.21.15.7:0/805905639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd070107400 0x7fd070198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:14.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.356+0000 7fd06ffff700 1 -- 172.21.15.7:0/805905639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd070101390 msgr2=0x7fd07019e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:33:14.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.356+0000 7fd06ffff700 1 --2- 172.21.15.7:0/805905639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd070101390 0x7fd07019e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:14.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.356+0000 7fd06ffff700 1 -- 172.21.15.7:0/805905639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0701991a0 con 0x7fd070101d50 2024-06-10T01:33:14.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.356+0000 7fd06ffff700 1 --2- 172.21.15.7:0/805905639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd070101d50 0x7fd07019ed70 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fd05800b530 tx=0x7fd05800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:33:14.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.356+0000 7fd06dffb700 1 -- 172.21.15.7:0/805905639 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058015020 con 0x7fd070101d50 2024-06-10T01:33:14.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.356+0000 7fd077173700 1 -- 172.21.15.7:0/805905639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd070199490 con 0x7fd070101d50 2024-06-10T01:33:14.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.356+0000 7fd06dffb700 1 -- 172.21.15.7:0/805905639 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd058011d10 con 0x7fd070101d50 2024-06-10T01:33:14.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.356+0000 7fd06dffb700 1 -- 172.21.15.7:0/805905639 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05800f2e0 con 0x7fd070101d50 2024-06-10T01:33:14.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.356+0000 7fd077173700 1 -- 172.21.15.7:0/805905639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0701a5700 con 0x7fd070101d50 2024-06-10T01:33:14.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.357+0000 7fd077173700 1 -- 172.21.15.7:0/805905639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd070069180 con 0x7fd070101d50 2024-06-10T01:33:14.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.358+0000 7fd06dffb700 1 -- 172.21.15.7:0/805905639 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fd058018020 con 0x7fd070101d50 2024-06-10T01:33:14.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.359+0000 7fd06dffb700 1 --2- 172.21.15.7:0/805905639 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd05c0a2c20 0x7fd05c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:33:14.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.360+0000 7fd074f0f700 1 --2- 172.21.15.7:0/805905639 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd05c0a2c20 0x7fd05c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:33:14.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.360+0000 7fd06dffb700 1 -- 172.21.15.7:0/805905639 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8883+0+0 (secure 0 0 0) 0x7fd0580ceb30 con 0x7fd070101d50 2024-06-10T01:33:14.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.360+0000 7fd074f0f700 1 --2- 172.21.15.7:0/805905639 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd05c0a2c20 0x7fd05c0a50e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd070107c80 tx=0x7fd06000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:33:14.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.364+0000 7fd06dffb700 1 -- 172.21.15.7:0/805905639 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd05801f020 con 0x7fd070101d50 2024-06-10T01:33:14.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:14.569+0000 7fd077173700 1 -- 172.21.15.7:0/805905639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd0701a5990 con 0x7fd070101d50 2024-06-10T01:33:15.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:15.176+0000 7fd06dffb700 1 -- 172.21.15.7:0/805905639 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v105) ==== 169+0+0 (secure 0 0 0) 0x7fd05809b800 con 0x7fd070101d50 2024-06-10T01:33:15.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:15.180+0000 7fd077173700 1 -- 172.21.15.7:0/805905639 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd05c0a2c20 msgr2=0x7fd05c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:33:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:15.180+0000 7fd077173700 1 --2- 172.21.15.7:0/805905639 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd05c0a2c20 0x7fd05c0a50e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd070107c80 tx=0x7fd06000a380 comp rx=0 tx=0).stop 2024-06-10T01:33:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:15.180+0000 7fd077173700 1 -- 172.21.15.7:0/805905639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd070101d50 msgr2=0x7fd07019ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:33:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:15.180+0000 7fd077173700 1 --2- 172.21.15.7:0/805905639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd070101d50 0x7fd07019ed70 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fd05800b530 tx=0x7fd05800b840 comp rx=0 tx=0).stop 2024-06-10T01:33:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:15.181+0000 7fd077173700 1 -- 172.21.15.7:0/805905639 shutdown_connections 2024-06-10T01:33:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:15.181+0000 7fd077173700 1 --2- 172.21.15.7:0/805905639 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7fd05c0a2c20 0x7fd05c0a50e0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:15.181+0000 7fd077173700 1 --2- 172.21.15.7:0/805905639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd070107400 0x7fd070198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:15.181+0000 7fd077173700 1 --2- 172.21.15.7:0/805905639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd070101390 0x7fd07019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:15.181+0000 7fd077173700 1 --2- 172.21.15.7:0/805905639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd070101d50 0x7fd07019ed70 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:15.181+0000 7fd077173700 1 -- 172.21.15.7:0/805905639 >> 172.21.15.7:0/805905639 conn(0x7fd070078340 msgr2=0x7fd07006a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:33:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:15.181+0000 7fd077173700 1 -- 172.21.15.7:0/805905639 shutdown_connections 2024-06-10T01:33:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:15.181+0000 7fd077173700 1 -- 172.21.15.7:0/805905639 wait complete. 2024-06-10T01:33:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-06-10T01:33:15.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:15 smithi175 ceph-mon[36636]: pgmap v1253: 97 pgs: 97 active+clean; 577 KiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:33:15.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:15 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/805905639' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T01:33:15.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:15 smithi007 ceph-mon[32398]: pgmap v1253: 97 pgs: 97 active+clean; 577 KiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:33:15.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:15 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/805905639' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T01:33:15.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:15 smithi078 ceph-mon[36683]: pgmap v1253: 97 pgs: 97 active+clean; 577 KiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:33:15.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:15 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/805905639' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T01:33:16.176 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-06-10T01:33:16.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:16 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/805905639' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T01:33:16.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:16 smithi175 ceph-mon[36636]: osdmap e105: 12 total, 12 up, 12 in 2024-06-10T01:33:16.463 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:33:16.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:16 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/805905639' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T01:33:16.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:16 smithi007 ceph-mon[32398]: osdmap e105: 12 total, 12 up, 12 in 2024-06-10T01:33:16.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:16 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/805905639' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T01:33:16.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:16 smithi078 ceph-mon[36683]: osdmap e105: 12 total, 12 up, 12 in 2024-06-10T01:33:17.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:17 smithi175 ceph-mon[36636]: pgmap v1255: 33 pgs: 33 active+clean; 577 KiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:33:17.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:17 smithi007 ceph-mon[32398]: pgmap v1255: 33 pgs: 33 active+clean; 577 KiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:33:17.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:17 smithi078 ceph-mon[36683]: pgmap v1255: 33 pgs: 33 active+clean; 577 KiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:33:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.470+0000 7f3328ff9700 1 -- 172.21.15.7:0/4005901523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f332406aa00 msgr2=0x7f332406ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:33:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.470+0000 7f3328ff9700 1 --2- 172.21.15.7:0/4005901523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f332406aa00 0x7f332406ae80 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f3318009990 tx=0x7f3318009ca0 comp rx=0 tx=0).stop 2024-06-10T01:33:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.470+0000 7f3328ff9700 1 -- 172.21.15.7:0/4005901523 shutdown_connections 2024-06-10T01:33:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.470+0000 7f3328ff9700 1 --2- 172.21.15.7:0/4005901523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f332406b3c0 0x7f332410f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.470+0000 7f3328ff9700 1 --2- 172.21.15.7:0/4005901523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33241084f0 0x7f33241088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.470+0000 7f3328ff9700 1 --2- 172.21.15.7:0/4005901523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f332406aa00 0x7f332406ae80 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.470+0000 7f3328ff9700 1 -- 172.21.15.7:0/4005901523 >> 172.21.15.7:0/4005901523 conn(0x7f3324078300 msgr2=0x7f3324078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:33:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.470+0000 7f3328ff9700 1 -- 172.21.15.7:0/4005901523 shutdown_connections 2024-06-10T01:33:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.470+0000 7f3328ff9700 1 -- 172.21.15.7:0/4005901523 wait complete. 2024-06-10T01:33:18.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.471+0000 7f3328ff9700 1 Processor -- start 2024-06-10T01:33:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.471+0000 7f3328ff9700 1 -- start start 2024-06-10T01:33:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.472+0000 7f3328ff9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f332406aa00 0x7f33241a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:33:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.472+0000 7f3328ff9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f332406b3c0 0x7f33241a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:33:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.472+0000 7f3328ff9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33241084f0 0x7f332419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:33:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.472+0000 7f3328ff9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3324112750 con 0x7f33241084f0 2024-06-10T01:33:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.472+0000 7f3328ff9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f33241125d0 con 0x7f332406b3c0 2024-06-10T01:33:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.472+0000 7f3328ff9700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f33241128d0 con 0x7f332406aa00 2024-06-10T01:33:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.472+0000 7f3321d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f332406b3c0 0x7f33241a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:33:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.472+0000 7f332259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f332406aa00 0x7f33241a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:33:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.472+0000 7f3321d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f332406b3c0 0x7f33241a0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.7:33888/0 (socket says 172.21.15.7:33888) 2024-06-10T01:33:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.472+0000 7f3321d9b700 1 -- 172.21.15.7:0/1505758290 learned_addr learned my addr 172.21.15.7:0/1505758290 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-10T01:33:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.472+0000 7f3322d9d700 1 --2- 172.21.15.7:0/1505758290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33241084f0 0x7f332419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:33:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.472+0000 7f332259c700 1 -- 172.21.15.7:0/1505758290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f332406b3c0 msgr2=0x7f33241a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:33:18.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.472+0000 7f332259c700 1 --2- 172.21.15.7:0/1505758290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f332406b3c0 0x7f33241a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:18.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.472+0000 7f332259c700 1 -- 172.21.15.7:0/1505758290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33241084f0 msgr2=0x7f332419ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:33:18.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.472+0000 7f332259c700 1 --2- 172.21.15.7:0/1505758290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33241084f0 0x7f332419ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:18.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.472+0000 7f332259c700 1 -- 172.21.15.7:0/1505758290 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f332419b3f0 con 0x7f332406aa00 2024-06-10T01:33:18.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.472+0000 7f3321d9b700 1 --2- 172.21.15.7:0/1505758290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f332406b3c0 0x7f33241a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:33:18.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.472+0000 7f332259c700 1 --2- 172.21.15.7:0/1505758290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f332406aa00 0x7f33241a0a40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f331400c970 tx=0x7f331400cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:33:18.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.473+0000 7f33137fe700 1 -- 172.21.15.7:0/1505758290 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314014070 con 0x7f332406aa00 2024-06-10T01:33:18.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.473+0000 7f3328ff9700 1 -- 172.21.15.7:0/1505758290 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f332419b6e0 con 0x7f332406aa00 2024-06-10T01:33:18.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.473+0000 7f33137fe700 1 -- 172.21.15.7:0/1505758290 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3314004440 con 0x7f332406aa00 2024-06-10T01:33:18.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.473+0000 7f33137fe700 1 -- 172.21.15.7:0/1505758290 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314007b60 con 0x7f332406aa00 2024-06-10T01:33:18.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.473+0000 7f3328ff9700 1 -- 172.21.15.7:0/1505758290 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33241a79c0 con 0x7f332406aa00 2024-06-10T01:33:18.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.474+0000 7f3328ff9700 1 -- 172.21.15.7:0/1505758290 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3324103bb0 con 0x7f332406aa00 2024-06-10T01:33:18.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.475+0000 7f33137fe700 1 -- 172.21.15.7:0/1505758290 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f3314017070 con 0x7f332406aa00 2024-06-10T01:33:18.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.476+0000 7f33137fe700 1 --2- 172.21.15.7:0/1505758290 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f330c0a2c20 0x7f330c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:33:18.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.476+0000 7f3321d9b700 1 --2- 172.21.15.7:0/1505758290 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f330c0a2c20 0x7f330c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:33:18.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.476+0000 7f33137fe700 1 -- 172.21.15.7:0/1505758290 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8324+0+0 (secure 0 0 0) 0x7f33140cf090 con 0x7f332406aa00 2024-06-10T01:33:18.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.476+0000 7f3321d9b700 1 --2- 172.21.15.7:0/1505758290 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f330c0a2c20 0x7f330c0a50e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3318009750 tx=0x7f3318005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:33:18.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.488+0000 7f33137fe700 1 -- 172.21.15.7:0/1505758290 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f331401e020 con 0x7f332406aa00 2024-06-10T01:33:18.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:18.693+0000 7f3328ff9700 1 -- 172.21.15.7:0/1505758290 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f33241a7b90 con 0x7f332406aa00 2024-06-10T01:33:19.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:19.189+0000 7f33137fe700 1 -- 172.21.15.7:0/1505758290 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v106) ==== 178+0+0 (secure 0 0 0) 0x7f331409bf30 con 0x7f332406aa00 2024-06-10T01:33:19.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:19.194+0000 7f3328ff9700 1 -- 172.21.15.7:0/1505758290 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f330c0a2c20 msgr2=0x7f330c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:33:19.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:19.194+0000 7f3328ff9700 1 --2- 172.21.15.7:0/1505758290 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f330c0a2c20 0x7f330c0a50e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3318009750 tx=0x7f3318005c60 comp rx=0 tx=0).stop 2024-06-10T01:33:19.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:19.194+0000 7f3328ff9700 1 -- 172.21.15.7:0/1505758290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f332406aa00 msgr2=0x7f33241a0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:33:19.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:19.194+0000 7f3328ff9700 1 --2- 172.21.15.7:0/1505758290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f332406aa00 0x7f33241a0a40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f331400c970 tx=0x7f331400cc80 comp rx=0 tx=0).stop 2024-06-10T01:33:19.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:19.195+0000 7f3328ff9700 1 -- 172.21.15.7:0/1505758290 shutdown_connections 2024-06-10T01:33:19.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:19.195+0000 7f3328ff9700 1 --2- 172.21.15.7:0/1505758290 >> [v2:172.21.15.7:6800/1804801756,v1:172.21.15.7:6801/1804801756] conn(0x7f330c0a2c20 0x7f330c0a50e0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:19.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:19.195+0000 7f3328ff9700 1 --2- 172.21.15.7:0/1505758290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33241084f0 0x7f332419ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:19.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:19.195+0000 7f3328ff9700 1 --2- 172.21.15.7:0/1505758290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f332406aa00 0x7f33241a0a40 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:19.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:19.195+0000 7f3328ff9700 1 --2- 172.21.15.7:0/1505758290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f332406b3c0 0x7f33241a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:33:19.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:19.195+0000 7f3328ff9700 1 -- 172.21.15.7:0/1505758290 >> 172.21.15.7:0/1505758290 conn(0x7f3324078300 msgr2=0x7f332410d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:33:19.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:19.195+0000 7f3328ff9700 1 -- 172.21.15.7:0/1505758290 shutdown_connections 2024-06-10T01:33:19.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-10T01:33:19.196+0000 7f3328ff9700 1 -- 172.21.15.7:0/1505758290 wait complete. 2024-06-10T01:33:19.194 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data_ec' removed 2024-06-10T01:33:19.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:19 smithi175 ceph-mon[36636]: pgmap v1256: 33 pgs: 33 active+clean; 577 KiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:33:19.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:19 smithi175 ceph-mon[36636]: from='client.? 172.21.15.7:0/1505758290' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T01:33:19.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:19 smithi175 ceph-mon[36636]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T01:33:19.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:19 smithi007 ceph-mon[32398]: pgmap v1256: 33 pgs: 33 active+clean; 577 KiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:33:19.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:19 smithi007 ceph-mon[32398]: from='client.? 172.21.15.7:0/1505758290' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T01:33:19.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:19 smithi007 ceph-mon[32398]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T01:33:19.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:19 smithi078 ceph-mon[36683]: pgmap v1256: 33 pgs: 33 active+clean; 577 KiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:33:19.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:19 smithi078 ceph-mon[36683]: from='client.? 172.21.15.7:0/1505758290' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T01:33:19.577 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:19 smithi078 ceph-mon[36683]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T01:33:19.994 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-10T01:33:20.032 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-10T01:33:20.093 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-10T01:33:20.191 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-06-10T01:33:20.191 DEBUG:teuthology.orchestra.run.smithi007:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-10T01:33:20.216 DEBUG:teuthology.orchestra.run.smithi078:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-10T01:33:20.231 DEBUG:teuthology.orchestra.run.smithi175:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-10T01:33:20.245 INFO:tasks.cephadm:Stopping all daemons... 2024-06-10T01:33:20.245 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-06-10T01:33:20.246 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mon.a 2024-06-10T01:33:20.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:20 smithi007 ceph-mon[32398]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T01:33:20.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:20 smithi007 ceph-mon[32398]: osdmap e106: 12 total, 12 up, 12 in 2024-06-10T01:33:20.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:20 smithi078 ceph-mon[36683]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T01:33:20.327 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:20 smithi078 ceph-mon[36683]: osdmap e106: 12 total, 12 up, 12 in 2024-06-10T01:33:20.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:20 smithi175 ceph-mon[36636]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T01:33:20.451 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:20 smithi175 ceph-mon[36636]: osdmap e106: 12 total, 12 up, 12 in 2024-06-10T01:33:20.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:20 smithi007 systemd[1]: Stopping Ceph mon.a for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:33:20.988 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:20 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-a[32394]: 2024-06-10T01:33:20.635+0000 7f9a3147d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:33:20.989 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:20 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-a[32394]: 2024-06-10T01:33:20.635+0000 7f9a3147d700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-06-10T01:33:20.989 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:20 smithi007 podman[159771]: 2024-06-10 01:33:20.689293388 +0000 UTC m=+0.197316339 container died afb11117c1a1cf56b620e531af50ff3ae10ee0d8f9930b6499359c181188a62d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-10T01:33:21.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:21 smithi007 podman[159771]: 2024-06-10 01:33:21.204247521 +0000 UTC m=+0.712270474 container cleanup afb11117c1a1cf56b620e531af50ff3ae10ee0d8f9930b6499359c181188a62d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc) 2024-06-10T01:33:21.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:21 smithi007 bash[159771]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-a 2024-06-10T01:33:21.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 10 01:33:21 smithi007 podman[159783]: 2024-06-10 01:33:21.431202616 +0000 UTC m=+0.736341122 container remove afb11117c1a1cf56b620e531af50ff3ae10ee0d8f9930b6499359c181188a62d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc) 2024-06-10T01:33:21.689 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mon.a.service' 2024-06-10T01:33:21.776 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:33:21.776 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-06-10T01:33:21.777 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-06-10T01:33:21.777 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mon.b 2024-06-10T01:33:22.078 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:21 smithi078 systemd[1]: Stopping Ceph mon.b for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:33:22.578 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:22 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-b[36679]: 2024-06-10T01:33:22.190+0000 7f136ca4d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:33:22.578 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:22 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-b[36679]: 2024-06-10T01:33:22.190+0000 7f136ca4d700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-06-10T01:33:22.578 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:22 smithi078 podman[76947]: 2024-06-10 01:33:22.385490895 +0000 UTC m=+0.335294704 container died e51fad75031b90f85bc64968f2bb50ac95abcf1da8f979447f83a6f499c4b6b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T01:33:23.052 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:22 smithi078 podman[76947]: 2024-06-10 01:33:22.825184772 +0000 UTC m=+0.774988575 container cleanup e51fad75031b90f85bc64968f2bb50ac95abcf1da8f979447f83a6f499c4b6b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-06-10T01:33:23.052 INFO:journalctl@ceph.mon.b.smithi078.stdout:Jun 10 01:33:22 smithi078 bash[76947]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-b 2024-06-10T01:33:23.187 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mon.b.service' 2024-06-10T01:33:23.228 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:33:23.229 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-06-10T01:33:23.229 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-06-10T01:33:23.229 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mon.c 2024-06-10T01:33:23.602 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:23 smithi175 systemd[1]: Stopping Ceph mon.c for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:33:23.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:23 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-c[36618]: 2024-06-10T01:33:23.608+0000 7f86b714f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:33:23.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:23 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-c[36618]: 2024-06-10T01:33:23.608+0000 7f86b714f700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-06-10T01:33:23.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:23 smithi175 podman[79016]: 2024-06-10 01:33:23.680680153 +0000 UTC m=+0.196514552 container died 260eedf0597653cafbd704b522b7b72ca4322e912a5323d3c01b2afdc02c681e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-c, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-06-10T01:33:24.345 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:24 smithi175 podman[79016]: 2024-06-10 01:33:24.068307871 +0000 UTC m=+0.584142262 container cleanup 260eedf0597653cafbd704b522b7b72ca4322e912a5323d3c01b2afdc02c681e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-06-10T01:33:24.345 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jun 10 01:33:24 smithi175 bash[79016]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mon-c 2024-06-10T01:33:24.487 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mon.c.service' 2024-06-10T01:33:24.543 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:33:24.543 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-06-10T01:33:24.543 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-06-10T01:33:24.544 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mgr.x 2024-06-10T01:33:24.896 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 01:33:24 smithi007 systemd[1]: Stopping Ceph mgr.x for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:33:25.203 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 01:33:24 smithi007 podman[159892]: 2024-06-10 01:33:24.907589631 +0000 UTC m=+0.156869253 container died 4bce828fe08d754fbbe86fcfe6c35c5a28ec7e5a062d672d35f59f4a2e6b4daf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-06-10T01:33:25.511 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 01:33:25 smithi007 podman[159892]: 2024-06-10 01:33:25.410550961 +0000 UTC m=+0.659830563 container cleanup 4bce828fe08d754fbbe86fcfe6c35c5a28ec7e5a062d672d35f59f4a2e6b4daf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-06-10T01:33:25.511 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 10 01:33:25 smithi007 bash[159892]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-x 2024-06-10T01:33:25.830 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mgr.x.service' 2024-06-10T01:33:25.866 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:33:25.866 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-06-10T01:33:25.866 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-06-10T01:33:25.866 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mgr.y 2024-06-10T01:33:26.175 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 01:33:25 smithi078 systemd[1]: Stopping Ceph mgr.y for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:33:26.549 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 01:33:26 smithi078 podman[77070]: 2024-06-10 01:33:26.243074045 +0000 UTC m=+0.159317192 container died c999d5b9e20f1e337efc7fd2032be588c5196fcbb98d32b396d7acceaf375197 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T01:33:27.077 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 01:33:26 smithi078 podman[77070]: 2024-06-10 01:33:26.831571018 +0000 UTC m=+0.747814162 container cleanup c999d5b9e20f1e337efc7fd2032be588c5196fcbb98d32b396d7acceaf375197 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-06-10T01:33:27.077 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Jun 10 01:33:26 smithi078 bash[77070]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-y 2024-06-10T01:33:27.244 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mgr.y.service' 2024-06-10T01:33:27.279 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:33:27.279 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-06-10T01:33:27.279 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-06-10T01:33:27.279 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mgr.z 2024-06-10T01:33:27.628 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 01:33:27 smithi175 systemd[1]: Stopping Ceph mgr.z for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:33:27.915 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 01:33:27 smithi175 podman[79148]: 2024-06-10 01:33:27.635674686 +0000 UTC m=+0.138113882 container died 69ef91707a6fcf7645c2b37a125af7c2d90adbc7236287c0f6fba84543229fa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-z, ceph=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240528) 2024-06-10T01:33:28.201 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 01:33:28 smithi175 podman[79148]: 2024-06-10 01:33:28.024096445 +0000 UTC m=+0.526535641 container cleanup 69ef91707a6fcf7645c2b37a125af7c2d90adbc7236287c0f6fba84543229fa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-z, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-06-10T01:33:28.201 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jun 10 01:33:28 smithi175 bash[79148]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mgr-z 2024-06-10T01:33:28.459 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mgr.z.service' 2024-06-10T01:33:28.494 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:33:28.494 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-06-10T01:33:28.494 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-06-10T01:33:28.494 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.0 2024-06-10T01:33:28.817 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 10 01:33:28 smithi007 systemd[1]: Stopping Ceph osd.0 for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:33:29.261 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 10 01:33:28 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-0[47139]: 2024-06-10T01:33:28.823+0000 7f1cbd112700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:33:29.262 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 10 01:33:28 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-0[47139]: 2024-06-10T01:33:28.823+0000 7f1cbd112700 -1 osd.0 106 *** Got signal Terminated *** 2024-06-10T01:33:29.262 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 10 01:33:28 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-0[47139]: 2024-06-10T01:33:28.823+0000 7f1cbd112700 -1 osd.0 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T01:33:34.576 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 10 01:33:34 smithi007 podman[160012]: 2024-06-10 01:33:34.284768999 +0000 UTC m=+5.577412398 container died f38bf96bcb02c2070e8246327cceab18a000c38fe61a7ce30b4485cfcc481ef0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-3d89acc, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T01:33:34.969 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 10 01:33:34 smithi007 podman[160012]: 2024-06-10 01:33:34.77481954 +0000 UTC m=+6.067462928 container cleanup f38bf96bcb02c2070e8246327cceab18a000c38fe61a7ce30b4485cfcc481ef0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-06-10T01:33:34.969 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 10 01:33:34 smithi007 bash[160012]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-0 2024-06-10T01:33:35.261 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 10 01:33:34 smithi007 podman[160030]: 2024-06-10 01:33:34.976588519 +0000 UTC m=+0.687693388 container remove f38bf96bcb02c2070e8246327cceab18a000c38fe61a7ce30b4485cfcc481ef0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-0, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, ceph=True) 2024-06-10T01:33:35.761 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 10 01:33:35 smithi007 podman[160121]: 2024-06-10 01:33:35.310640198 +0000 UTC m=+0.018237960 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:33:35.761 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 10 01:33:35 smithi007 podman[160121]: 2024-06-10 01:33:35.452676295 +0000 UTC m=+0.160274042 container create b031d4bd0e0bfd70c11e376176eb4ea67c64ac244b889d8536193e26e80a75eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T01:33:36.105 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 10 01:33:35 smithi007 podman[160121]: 2024-06-10 01:33:35.788856874 +0000 UTC m=+0.496454634 container init b031d4bd0e0bfd70c11e376176eb4ea67c64ac244b889d8536193e26e80a75eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc) 2024-06-10T01:33:36.105 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 10 01:33:35 smithi007 podman[160121]: 2024-06-10 01:33:35.792439377 +0000 UTC m=+0.500037125 container start b031d4bd0e0bfd70c11e376176eb4ea67c64ac244b889d8536193e26e80a75eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-06-10T01:33:36.105 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 10 01:33:35 smithi007 podman[160121]: 2024-06-10 01:33:35.880922596 +0000 UTC m=+0.588520362 container attach b031d4bd0e0bfd70c11e376176eb4ea67c64ac244b889d8536193e26e80a75eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5) 2024-06-10T01:33:36.392 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 10 01:33:36 smithi007 podman[160121]: 2024-06-10 01:33:36.113129842 +0000 UTC m=+0.820727608 container died b031d4bd0e0bfd70c11e376176eb4ea67c64ac244b889d8536193e26e80a75eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=) 2024-06-10T01:33:36.718 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.0.service' 2024-06-10T01:33:36.758 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:33:36.759 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-06-10T01:33:36.759 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-06-10T01:33:36.759 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.1 2024-06-10T01:33:37.070 INFO:journalctl@ceph.osd.1.smithi078.stdout:Jun 10 01:33:36 smithi078 systemd[1]: Stopping Ceph osd.1 for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:33:37.328 INFO:journalctl@ceph.osd.1.smithi078.stdout:Jun 10 01:33:37 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-1[41229]: 2024-06-10T01:33:37.077+0000 7f83ff77f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:33:37.328 INFO:journalctl@ceph.osd.1.smithi078.stdout:Jun 10 01:33:37 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-1[41229]: 2024-06-10T01:33:37.077+0000 7f83ff77f700 -1 osd.1 106 *** Got signal Terminated *** 2024-06-10T01:33:37.328 INFO:journalctl@ceph.osd.1.smithi078.stdout:Jun 10 01:33:37 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-1[41229]: 2024-06-10T01:33:37.077+0000 7f83ff77f700 -1 osd.1 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T01:33:42.913 INFO:journalctl@ceph.osd.1.smithi078.stdout:Jun 10 01:33:42 smithi078 podman[77203]: 2024-06-10 01:33:42.607213381 +0000 UTC m=+5.647616541 container died 4f6724a1934f0afc077201ade7ad0633c2c95132b8fb5e375b4bc8122dcec811 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-1, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-10T01:33:43.289 INFO:journalctl@ceph.osd.1.smithi078.stdout:Jun 10 01:33:43 smithi078 podman[77203]: 2024-06-10 01:33:43.036717783 +0000 UTC m=+6.077120945 container cleanup 4f6724a1934f0afc077201ade7ad0633c2c95132b8fb5e375b4bc8122dcec811 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-1, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-06-10T01:33:43.290 INFO:journalctl@ceph.osd.1.smithi078.stdout:Jun 10 01:33:43 smithi078 bash[77203]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-1 2024-06-10T01:33:43.578 INFO:journalctl@ceph.osd.1.smithi078.stdout:Jun 10 01:33:43 smithi078 podman[77219]: 2024-06-10 01:33:43.296652849 +0000 UTC m=+0.675718553 container remove 4f6724a1934f0afc077201ade7ad0633c2c95132b8fb5e375b4bc8122dcec811 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-1, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-06-10T01:33:43.863 INFO:journalctl@ceph.osd.1.smithi078.stdout:Jun 10 01:33:43 smithi078 podman[77307]: 2024-06-10 01:33:43.521422778 +0000 UTC m=+0.016794955 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:33:43.863 INFO:journalctl@ceph.osd.1.smithi078.stdout:Jun 10 01:33:43 smithi078 podman[77307]: 2024-06-10 01:33:43.630991624 +0000 UTC m=+0.126363853 container create c0dcb7e60fe6575bf3afe47727376d5513ffe019c5bfbc143ae7d82007911bae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5) 2024-06-10T01:33:43.864 INFO:journalctl@ceph.osd.1.smithi078.stdout:Jun 10 01:33:43 smithi078 podman[77307]: 2024-06-10 01:33:43.867699644 +0000 UTC m=+0.363071808 container init c0dcb7e60fe6575bf3afe47727376d5513ffe019c5bfbc143ae7d82007911bae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-06-10T01:33:43.864 INFO:journalctl@ceph.osd.1.smithi078.stdout:Jun 10 01:33:43 smithi078 podman[77307]: 2024-06-10 01:33:43.870989565 +0000 UTC m=+0.366361735 container start c0dcb7e60fe6575bf3afe47727376d5513ffe019c5bfbc143ae7d82007911bae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, maintainer=Guillaume Abrioux ) 2024-06-10T01:33:44.165 INFO:journalctl@ceph.osd.1.smithi078.stdout:Jun 10 01:33:43 smithi078 podman[77307]: 2024-06-10 01:33:43.983936181 +0000 UTC m=+0.479308345 container attach c0dcb7e60fe6575bf3afe47727376d5513ffe019c5bfbc143ae7d82007911bae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T01:33:44.433 INFO:journalctl@ceph.osd.1.smithi078.stdout:Jun 10 01:33:44 smithi078 podman[77307]: 2024-06-10 01:33:44.173743665 +0000 UTC m=+0.669115834 container died c0dcb7e60fe6575bf3afe47727376d5513ffe019c5bfbc143ae7d82007911bae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc) 2024-06-10T01:33:44.836 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.1.service' 2024-06-10T01:33:44.873 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:33:44.873 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-06-10T01:33:44.873 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-06-10T01:33:44.873 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.2 2024-06-10T01:33:45.184 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jun 10 01:33:44 smithi175 systemd[1]: Stopping Ceph osd.2 for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:33:45.451 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jun 10 01:33:45 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-2[41725]: 2024-06-10T01:33:45.190+0000 7f5e1e9b8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:33:45.451 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jun 10 01:33:45 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-2[41725]: 2024-06-10T01:33:45.190+0000 7f5e1e9b8700 -1 osd.2 106 *** Got signal Terminated *** 2024-06-10T01:33:45.451 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jun 10 01:33:45 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-2[41725]: 2024-06-10T01:33:45.190+0000 7f5e1e9b8700 -1 osd.2 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T01:33:50.952 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jun 10 01:33:50 smithi175 podman[79270]: 2024-06-10 01:33:50.701936416 +0000 UTC m=+5.631950944 container died 0a21f6282f9364318a002b1fdfb4914fcc641cb56fbbce72b97b3061a8c9e8f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux ) 2024-06-10T01:33:51.451 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jun 10 01:33:51 smithi175 podman[79270]: 2024-06-10 01:33:51.161060977 +0000 UTC m=+6.091075503 container cleanup 0a21f6282f9364318a002b1fdfb4914fcc641cb56fbbce72b97b3061a8c9e8f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc) 2024-06-10T01:33:51.451 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jun 10 01:33:51 smithi175 bash[79270]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-2 2024-06-10T01:33:51.824 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jun 10 01:33:51 smithi175 podman[79286]: 2024-06-10 01:33:51.47947498 +0000 UTC m=+0.779503176 container remove 0a21f6282f9364318a002b1fdfb4914fcc641cb56fbbce72b97b3061a8c9e8f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc) 2024-06-10T01:33:52.110 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jun 10 01:33:51 smithi175 podman[79375]: 2024-06-10 01:33:51.734135733 +0000 UTC m=+0.017621624 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:33:52.110 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jun 10 01:33:51 smithi175 podman[79375]: 2024-06-10 01:33:51.872888384 +0000 UTC m=+0.156374293 container create 137ff6cd876c7ee8a469259d5fdd30baaa58c264ab57d5a4358813d4735a90c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T01:33:52.449 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jun 10 01:33:52 smithi175 podman[79375]: 2024-06-10 01:33:52.117243395 +0000 UTC m=+0.400729294 container init 137ff6cd876c7ee8a469259d5fdd30baaa58c264ab57d5a4358813d4735a90c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528) 2024-06-10T01:33:52.449 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jun 10 01:33:52 smithi175 podman[79375]: 2024-06-10 01:33:52.120524201 +0000 UTC m=+0.404010092 container start 137ff6cd876c7ee8a469259d5fdd30baaa58c264ab57d5a4358813d4735a90c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-06-10T01:33:52.449 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jun 10 01:33:52 smithi175 podman[79375]: 2024-06-10 01:33:52.208045981 +0000 UTC m=+0.491531885 container attach 137ff6cd876c7ee8a469259d5fdd30baaa58c264ab57d5a4358813d4735a90c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, io.buildah.version=1.33.5, ceph=True) 2024-06-10T01:33:52.701 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jun 10 01:33:52 smithi175 podman[79375]: 2024-06-10 01:33:52.456966884 +0000 UTC m=+0.740452783 container died 137ff6cd876c7ee8a469259d5fdd30baaa58c264ab57d5a4358813d4735a90c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-06-10T01:33:53.098 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.2.service' 2024-06-10T01:33:53.137 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:33:53.137 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-06-10T01:33:53.137 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-06-10T01:33:53.138 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.3 2024-06-10T01:33:53.464 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 10 01:33:53 smithi007 systemd[1]: Stopping Ceph osd.3 for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:33:53.762 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 10 01:33:53 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-3[54076]: 2024-06-10T01:33:53.470+0000 7f64ccdae700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:33:53.762 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 10 01:33:53 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-3[54076]: 2024-06-10T01:33:53.470+0000 7f64ccdae700 -1 osd.3 106 *** Got signal Terminated *** 2024-06-10T01:33:53.762 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 10 01:33:53 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-3[54076]: 2024-06-10T01:33:53.470+0000 7f64ccdae700 -1 osd.3 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T01:33:55.761 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:33:55 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:33:55.291+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:33:55.952 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:33:55 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:33:55.546+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:30.395424+0000 front 2024-06-10T01:33:30.395540+0000 (oldest deadline 2024-06-10T01:33:55.095518+0000) 2024-06-10T01:33:56.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:33:55 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:33:55.658+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:33:56.511 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:33:56 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:33:56.241+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:33:56.701 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:33:56 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:33:56.524+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:30.395424+0000 front 2024-06-10T01:33:30.395540+0000 (oldest deadline 2024-06-10T01:33:55.095518+0000) 2024-06-10T01:33:56.702 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:33:56 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:33:56.389+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:33:57.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:33:56 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:33:56.676+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:33:57.575 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:33:57 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:33:57.290+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:33:57.702 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:33:57 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:33:57.529+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:30.395424+0000 front 2024-06-10T01:33:30.395540+0000 (oldest deadline 2024-06-10T01:33:55.095518+0000) 2024-06-10T01:33:57.702 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:33:57 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:33:57.384+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:33:58.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:33:57 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:33:57.718+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:33:58.511 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:33:58 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:33:58.318+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:33:58.701 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:33:58 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:33:58.531+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:30.395424+0000 front 2024-06-10T01:33:30.395540+0000 (oldest deadline 2024-06-10T01:33:55.095518+0000) 2024-06-10T01:33:58.701 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:33:58 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:33:58.401+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:33:59.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:33:58 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:33:58.758+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:33:59.497 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 10 01:33:59 smithi007 podman[160243]: 2024-06-10 01:33:59.129046641 +0000 UTC m=+5.776944345 container died 4a8ca8b53658da5916fa4a718a7937b6052fabff323f2d38e597b817d4ae9c58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-3, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-06-10T01:33:59.498 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:33:59 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:33:59.336+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:33:59.701 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:33:59 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:33:59.430+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:33:59.702 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:33:59 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:33:59.517+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:30.395424+0000 front 2024-06-10T01:33:30.395540+0000 (oldest deadline 2024-06-10T01:33:55.095518+0000) 2024-06-10T01:33:59.761 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 10 01:33:59 smithi007 podman[160243]: 2024-06-10 01:33:59.504627892 +0000 UTC m=+6.152525638 container cleanup 4a8ca8b53658da5916fa4a718a7937b6052fabff323f2d38e597b817d4ae9c58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-3, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, ceph=True, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS) 2024-06-10T01:33:59.762 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 10 01:33:59 smithi007 bash[160243]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-3 2024-06-10T01:33:59.762 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 10 01:33:59 smithi007 podman[160259]: 2024-06-10 01:33:59.722769333 +0000 UTC m=+0.587947573 container remove 4a8ca8b53658da5916fa4a718a7937b6052fabff323f2d38e597b817d4ae9c58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-06-10T01:34:00.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:33:59 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:33:59.711+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:00.460 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:00 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:00.294+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:00.460 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 10 01:34:00 smithi007 podman[160347]: 2024-06-10 01:34:00.087257871 +0000 UTC m=+0.017953280 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:34:00.461 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 10 01:34:00 smithi007 podman[160347]: 2024-06-10 01:34:00.215422596 +0000 UTC m=+0.146118028 container create 765e532cef0c1ea52cb76571d011e276e89a0e0f782610ce681b45efe9202d07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-06-10T01:34:00.762 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 10 01:34:00 smithi007 podman[160347]: 2024-06-10 01:34:00.467711086 +0000 UTC m=+0.398406486 container init 765e532cef0c1ea52cb76571d011e276e89a0e0f782610ce681b45efe9202d07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-06-10T01:34:00.762 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 10 01:34:00 smithi007 podman[160347]: 2024-06-10 01:34:00.471152166 +0000 UTC m=+0.401847565 container start 765e532cef0c1ea52cb76571d011e276e89a0e0f782610ce681b45efe9202d07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-06-10T01:34:00.762 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 10 01:34:00 smithi007 podman[160347]: 2024-06-10 01:34:00.567836619 +0000 UTC m=+0.498532038 container attach 765e532cef0c1ea52cb76571d011e276e89a0e0f782610ce681b45efe9202d07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-06-10T01:34:00.951 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:00 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:00.471+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:00.952 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:00 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:00.471+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:30.395424+0000 front 2024-06-10T01:33:30.395540+0000 (oldest deadline 2024-06-10T01:33:55.095518+0000) 2024-06-10T01:34:01.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:00 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:00.734+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:01.105 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 10 01:34:00 smithi007 podman[160347]: 2024-06-10 01:34:00.778356546 +0000 UTC m=+0.709051944 container died 765e532cef0c1ea52cb76571d011e276e89a0e0f782610ce681b45efe9202d07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-06-10T01:34:01.498 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:01 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:01.287+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:01.523 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.3.service' 2024-06-10T01:34:01.569 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:34:01.569 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-06-10T01:34:01.569 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-06-10T01:34:01.569 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.4 2024-06-10T01:34:01.701 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:01 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:01.505+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:01.702 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:01 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:01.443+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:30.395424+0000 front 2024-06-10T01:33:30.395540+0000 (oldest deadline 2024-06-10T01:33:55.095518+0000) 2024-06-10T01:34:01.870 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:01 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:01.712+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:01.870 INFO:journalctl@ceph.osd.4.smithi078.stdout:Jun 10 01:34:01 smithi078 systemd[1]: Stopping Ceph osd.4 for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:34:02.328 INFO:journalctl@ceph.osd.4.smithi078.stdout:Jun 10 01:34:01 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-4[47863]: 2024-06-10T01:34:01.876+0000 7fdb24ca2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:34:02.328 INFO:journalctl@ceph.osd.4.smithi078.stdout:Jun 10 01:34:01 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-4[47863]: 2024-06-10T01:34:01.876+0000 7fdb24ca2700 -1 osd.4 106 *** Got signal Terminated *** 2024-06-10T01:34:02.328 INFO:journalctl@ceph.osd.4.smithi078.stdout:Jun 10 01:34:01 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-4[47863]: 2024-06-10T01:34:01.876+0000 7fdb24ca2700 -1 osd.4 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T01:34:02.512 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:02 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:02.242+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:02.701 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:02 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:02.418+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:30.395424+0000 front 2024-06-10T01:33:30.395540+0000 (oldest deadline 2024-06-10T01:33:55.095518+0000) 2024-06-10T01:34:02.702 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:02 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:02.503+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:03.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:02 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:02.738+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:03.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:02 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:02.749+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:39.800756+0000 front 2024-06-10T01:33:39.800843+0000 (oldest deadline 2024-06-10T01:34:02.701017+0000) 2024-06-10T01:34:03.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:03 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:03.289+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:03.951 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:03 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:03.501+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:03.952 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:03 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:03.465+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:30.395424+0000 front 2024-06-10T01:33:30.395540+0000 (oldest deadline 2024-06-10T01:33:55.095518+0000) 2024-06-10T01:34:03.952 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:03 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:03.465+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.798134+0000 front 2024-06-10T01:33:40.798208+0000 (oldest deadline 2024-06-10T01:34:03.098194+0000) 2024-06-10T01:34:04.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:03 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:03.702+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:39.800756+0000 front 2024-06-10T01:33:39.800843+0000 (oldest deadline 2024-06-10T01:34:02.701017+0000) 2024-06-10T01:34:04.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:03 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:03.785+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:04.512 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:04 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:04.261+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:04.701 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:04 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:04.508+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:04.702 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:04 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:04.290+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:04.702 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:04 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:04.470+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:30.395424+0000 front 2024-06-10T01:33:30.395540+0000 (oldest deadline 2024-06-10T01:33:55.095518+0000) 2024-06-10T01:34:04.702 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:04 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:04.470+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.798134+0000 front 2024-06-10T01:33:40.798208+0000 (oldest deadline 2024-06-10T01:34:03.098194+0000) 2024-06-10T01:34:04.769 INFO:journalctl@ceph.osd.4.smithi078.stdout:Jun 10 01:34:04 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-4[47863]: 2024-06-10T01:34:04.478+0000 7fdb21296700 -1 osd.4 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.982583+0000 front 2024-06-10T01:33:41.982564+0000 (oldest deadline 2024-06-10T01:34:04.282706+0000) 2024-06-10T01:34:04.769 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:04 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:04.704+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:39.800756+0000 front 2024-06-10T01:33:39.800843+0000 (oldest deadline 2024-06-10T01:34:02.701017+0000) 2024-06-10T01:34:05.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:04 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:04.775+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:05.511 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:05 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:05.235+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:05.512 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:05 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T01:34:05.438+0000 7ff8f563f700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.095045+0000 front 2024-06-10T01:33:40.095085+0000 (oldest deadline 2024-06-10T01:34:04.795199+0000) 2024-06-10T01:34:05.701 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:05 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:05.256+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:05.702 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:05 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:05.463+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:05.702 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:05 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:05.460+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:30.395424+0000 front 2024-06-10T01:33:30.395540+0000 (oldest deadline 2024-06-10T01:33:55.095518+0000) 2024-06-10T01:34:05.702 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:05 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:05.460+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.798134+0000 front 2024-06-10T01:33:40.798208+0000 (oldest deadline 2024-06-10T01:34:03.098194+0000) 2024-06-10T01:34:05.750 INFO:journalctl@ceph.osd.4.smithi078.stdout:Jun 10 01:34:05 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-4[47863]: 2024-06-10T01:34:05.490+0000 7fdb21296700 -1 osd.4 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.982583+0000 front 2024-06-10T01:33:41.982564+0000 (oldest deadline 2024-06-10T01:34:04.282706+0000) 2024-06-10T01:34:05.750 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:05 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:05.714+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:39.800756+0000 front 2024-06-10T01:33:39.800843+0000 (oldest deadline 2024-06-10T01:34:02.701017+0000) 2024-06-10T01:34:06.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:05 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:05.757+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:06.512 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:06 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:06.223+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:06.512 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:06 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T01:34:06.429+0000 7ff8f563f700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.095045+0000 front 2024-06-10T01:33:40.095085+0000 (oldest deadline 2024-06-10T01:34:04.795199+0000) 2024-06-10T01:34:06.701 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:06 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:06.274+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:06.702 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:06 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:06.514+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:06.702 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:06 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:06.475+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:30.395424+0000 front 2024-06-10T01:33:30.395540+0000 (oldest deadline 2024-06-10T01:33:55.095518+0000) 2024-06-10T01:34:06.702 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:06 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:06.475+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.798134+0000 front 2024-06-10T01:33:40.798208+0000 (oldest deadline 2024-06-10T01:34:03.098194+0000) 2024-06-10T01:34:06.772 INFO:journalctl@ceph.osd.4.smithi078.stdout:Jun 10 01:34:06 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-4[47863]: 2024-06-10T01:34:06.502+0000 7fdb21296700 -1 osd.4 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.982583+0000 front 2024-06-10T01:33:41.982564+0000 (oldest deadline 2024-06-10T01:34:04.282706+0000) 2024-06-10T01:34:06.772 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:06 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:06.701+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:39.800756+0000 front 2024-06-10T01:33:39.800843+0000 (oldest deadline 2024-06-10T01:34:02.701017+0000) 2024-06-10T01:34:07.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:06 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:06.778+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:07.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:06 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:06.778+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:07.432 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:07 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:07.185+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:07.432 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:07 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:07.185+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:07.526 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:07 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:07.262+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:07.527 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:07 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:07.462+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:30.395424+0000 front 2024-06-10T01:33:30.395540+0000 (oldest deadline 2024-06-10T01:33:55.095518+0000) 2024-06-10T01:34:07.527 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:07 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:07.462+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.798134+0000 front 2024-06-10T01:33:40.798208+0000 (oldest deadline 2024-06-10T01:34:03.098194+0000) 2024-06-10T01:34:07.666 INFO:journalctl@ceph.osd.4.smithi078.stdout:Jun 10 01:34:07 smithi078 podman[77425]: 2024-06-10 01:34:07.33958172 +0000 UTC m=+5.567846624 container died 08f853dcb81ee0bfa559df0a376fab3cc370cc51c4cf74d35c2335d3285455c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2) 2024-06-10T01:34:07.761 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:07 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T01:34:07.438+0000 7ff8f563f700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.095045+0000 front 2024-06-10T01:33:40.095085+0000 (oldest deadline 2024-06-10T01:34:04.795199+0000) 2024-06-10T01:34:07.951 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:07 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:07.532+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:08.004 INFO:journalctl@ceph.osd.4.smithi078.stdout:Jun 10 01:34:07 smithi078 podman[77425]: 2024-06-10 01:34:07.768233722 +0000 UTC m=+5.996498625 container cleanup 08f853dcb81ee0bfa559df0a376fab3cc370cc51c4cf74d35c2335d3285455c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-4, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-06-10T01:34:08.005 INFO:journalctl@ceph.osd.4.smithi078.stdout:Jun 10 01:34:07 smithi078 bash[77425]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-4 2024-06-10T01:34:08.005 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:07 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:07.730+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:39.800756+0000 front 2024-06-10T01:33:39.800843+0000 (oldest deadline 2024-06-10T01:34:02.701017+0000) 2024-06-10T01:34:08.005 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:07 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:07.766+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:08.005 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:07 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:07.766+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:08.328 INFO:journalctl@ceph.osd.4.smithi078.stdout:Jun 10 01:34:08 smithi078 podman[77443]: 2024-06-10 01:34:08.011703359 +0000 UTC m=+0.673585877 container remove 08f853dcb81ee0bfa559df0a376fab3cc370cc51c4cf74d35c2335d3285455c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_CLEAN=True) 2024-06-10T01:34:08.494 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:08 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:08.215+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:08.511 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:08 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T01:34:08.453+0000 7ff8f563f700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.095045+0000 front 2024-06-10T01:33:40.095085+0000 (oldest deadline 2024-06-10T01:34:04.795199+0000) 2024-06-10T01:34:08.512 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:08 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:08.231+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:08.512 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:08 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:08.231+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:08.725 INFO:journalctl@ceph.osd.4.smithi078.stdout:Jun 10 01:34:08 smithi078 podman[77531]: 2024-06-10 01:34:08.345585043 +0000 UTC m=+0.017461424 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:34:08.725 INFO:journalctl@ceph.osd.4.smithi078.stdout:Jun 10 01:34:08 smithi078 podman[77531]: 2024-06-10 01:34:08.462731879 +0000 UTC m=+0.134608244 container create bdb114d9db5df08ef88eda94a2f6bd2a2857ab92e9c5966d6c713b0e58625b68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-06-10T01:34:08.952 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:08 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:08.501+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:30.395424+0000 front 2024-06-10T01:33:30.395540+0000 (oldest deadline 2024-06-10T01:33:55.095518+0000) 2024-06-10T01:34:08.952 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:08 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:08.501+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.798134+0000 front 2024-06-10T01:33:40.798208+0000 (oldest deadline 2024-06-10T01:34:03.098194+0000) 2024-06-10T01:34:08.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:08 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:08.578+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:09.019 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:08 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:08.748+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:39.800756+0000 front 2024-06-10T01:33:39.800843+0000 (oldest deadline 2024-06-10T01:34:02.701017+0000) 2024-06-10T01:34:09.020 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:08 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:08.814+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:09.020 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:08 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:08.814+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:09.020 INFO:journalctl@ceph.osd.4.smithi078.stdout:Jun 10 01:34:08 smithi078 podman[77531]: 2024-06-10 01:34:08.732597853 +0000 UTC m=+0.404474220 container init bdb114d9db5df08ef88eda94a2f6bd2a2857ab92e9c5966d6c713b0e58625b68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2) 2024-06-10T01:34:09.020 INFO:journalctl@ceph.osd.4.smithi078.stdout:Jun 10 01:34:08 smithi078 podman[77531]: 2024-06-10 01:34:08.736465769 +0000 UTC m=+0.408342134 container start bdb114d9db5df08ef88eda94a2f6bd2a2857ab92e9c5966d6c713b0e58625b68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-06-10T01:34:09.020 INFO:journalctl@ceph.osd.4.smithi078.stdout:Jun 10 01:34:08 smithi078 podman[77531]: 2024-06-10 01:34:08.832133466 +0000 UTC m=+0.504009834 container attach bdb114d9db5df08ef88eda94a2f6bd2a2857ab92e9c5966d6c713b0e58625b68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-4-deactivate, ceph=True, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-06-10T01:34:09.328 INFO:journalctl@ceph.osd.4.smithi078.stdout:Jun 10 01:34:09 smithi078 podman[77531]: 2024-06-10 01:34:09.027082364 +0000 UTC m=+0.698958732 container died bdb114d9db5df08ef88eda94a2f6bd2a2857ab92e9c5966d6c713b0e58625b68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-4-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T01:34:09.512 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:09 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T01:34:09.418+0000 7ff8f563f700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.095045+0000 front 2024-06-10T01:33:40.095085+0000 (oldest deadline 2024-06-10T01:34:04.795199+0000) 2024-06-10T01:34:09.512 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:09 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:09.262+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:09.512 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:09 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:09.262+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:09.527 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:09 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:09.246+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:09.664 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.4.service' 2024-06-10T01:34:09.702 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:34:09.703 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-06-10T01:34:09.703 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-06-10T01:34:09.703 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.5 2024-06-10T01:34:09.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:09 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:09.558+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:09.952 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:09 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:09.534+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:30.395424+0000 front 2024-06-10T01:33:30.395540+0000 (oldest deadline 2024-06-10T01:33:55.095518+0000) 2024-06-10T01:34:09.952 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:09 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:09.534+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.798134+0000 front 2024-06-10T01:33:40.798208+0000 (oldest deadline 2024-06-10T01:34:03.098194+0000) 2024-06-10T01:34:09.952 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:09 smithi175 systemd[1]: Stopping Ceph osd.5 for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:34:10.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:09 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:09.793+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:39.800756+0000 front 2024-06-10T01:33:39.800843+0000 (oldest deadline 2024-06-10T01:34:02.701017+0000) 2024-06-10T01:34:10.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:09 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:09.799+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:10.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:09 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:09.799+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:10.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:10 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:10.261+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:10.452 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:10 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:10.014+0000 7f310d55e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:34:10.452 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:10 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:10.014+0000 7f310d55e700 -1 osd.5 106 *** Got signal Terminated *** 2024-06-10T01:34:10.452 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:10 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:10.014+0000 7f310d55e700 -1 osd.5 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T01:34:10.512 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:10 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T01:34:10.436+0000 7ff8f563f700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.095045+0000 front 2024-06-10T01:33:40.095085+0000 (oldest deadline 2024-06-10T01:34:04.795199+0000) 2024-06-10T01:34:10.512 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:10 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:10.256+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:10.512 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:10 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:10.256+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:10.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:10 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:10.566+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:10.952 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:10 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:10.491+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:30.395424+0000 front 2024-06-10T01:33:30.395540+0000 (oldest deadline 2024-06-10T01:33:55.095518+0000) 2024-06-10T01:34:10.952 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:10 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:10.491+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.798134+0000 front 2024-06-10T01:33:40.798208+0000 (oldest deadline 2024-06-10T01:34:03.098194+0000) 2024-06-10T01:34:11.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:10 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:10.827+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:39.800756+0000 front 2024-06-10T01:33:39.800843+0000 (oldest deadline 2024-06-10T01:34:02.701017+0000) 2024-06-10T01:34:11.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:10 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:10.785+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:11.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:10 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:10.785+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:11.580 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:11 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:11.268+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:11.580 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:11 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:11.268+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:47.685681+0000 front 2024-06-10T01:33:47.685658+0000 (oldest deadline 2024-06-10T01:34:11.185884+0000) 2024-06-10T01:34:11.580 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:11 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:11.475+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:30.395424+0000 front 2024-06-10T01:33:30.395540+0000 (oldest deadline 2024-06-10T01:33:55.095518+0000) 2024-06-10T01:34:11.580 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:11 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:11.475+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.798134+0000 front 2024-06-10T01:33:40.798208+0000 (oldest deadline 2024-06-10T01:34:03.098194+0000) 2024-06-10T01:34:11.762 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:11 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T01:34:11.437+0000 7ff8f563f700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.095045+0000 front 2024-06-10T01:33:40.095085+0000 (oldest deadline 2024-06-10T01:34:04.795199+0000) 2024-06-10T01:34:11.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:11 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:11.284+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:11.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:11 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:11.284+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:11.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:11 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:11.587+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:12.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:11 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:11.797+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:39.800756+0000 front 2024-06-10T01:33:39.800843+0000 (oldest deadline 2024-06-10T01:34:02.701017+0000) 2024-06-10T01:34:12.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:11 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:11.797+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:45.602414+0000 front 2024-06-10T01:33:45.602288+0000 (oldest deadline 2024-06-10T01:34:11.502446+0000) 2024-06-10T01:34:12.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:11 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:11.780+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:12.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:11 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:11.780+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:12.479 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:12 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:12.235+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:12.480 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:12 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:12.235+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:12.501 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:12 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:12.231+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:12.501 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:12 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:12.231+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:47.685681+0000 front 2024-06-10T01:33:47.685658+0000 (oldest deadline 2024-06-10T01:34:11.185884+0000) 2024-06-10T01:34:12.762 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:12 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T01:34:12.486+0000 7ff8f563f700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.095045+0000 front 2024-06-10T01:33:40.095085+0000 (oldest deadline 2024-06-10T01:34:04.795199+0000) 2024-06-10T01:34:12.951 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:12 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:12.540+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:12.952 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:12 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:12.508+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:30.395424+0000 front 2024-06-10T01:33:30.395540+0000 (oldest deadline 2024-06-10T01:33:55.095518+0000) 2024-06-10T01:34:12.952 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:12 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:12.508+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.798134+0000 front 2024-06-10T01:33:40.798208+0000 (oldest deadline 2024-06-10T01:34:03.098194+0000) 2024-06-10T01:34:12.952 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:12 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:12.508+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.899965+0000 front 2024-06-10T01:33:48.899985+0000 (oldest deadline 2024-06-10T01:34:11.800171+0000) 2024-06-10T01:34:13.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:12 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:12.766+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:39.800756+0000 front 2024-06-10T01:33:39.800843+0000 (oldest deadline 2024-06-10T01:34:02.701017+0000) 2024-06-10T01:34:13.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:12 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:12.766+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:45.602414+0000 front 2024-06-10T01:33:45.602288+0000 (oldest deadline 2024-06-10T01:34:11.502446+0000) 2024-06-10T01:34:13.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:12 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:12.815+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:13.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:12 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:12.815+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:13.466 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:13 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:13.219+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:13.466 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:13 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:13.219+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:13.520 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:13 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:13.256+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:13.521 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:13 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:13.256+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:47.685681+0000 front 2024-06-10T01:33:47.685658+0000 (oldest deadline 2024-06-10T01:34:11.185884+0000) 2024-06-10T01:34:13.762 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:13 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T01:34:13.473+0000 7ff8f563f700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.095045+0000 front 2024-06-10T01:33:40.095085+0000 (oldest deadline 2024-06-10T01:34:04.795199+0000) 2024-06-10T01:34:13.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:13 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:13.529+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:13.952 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:13 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:13.527+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:30.395424+0000 front 2024-06-10T01:33:30.395540+0000 (oldest deadline 2024-06-10T01:33:55.095518+0000) 2024-06-10T01:34:13.952 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:13 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:13.527+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.798134+0000 front 2024-06-10T01:33:40.798208+0000 (oldest deadline 2024-06-10T01:34:03.098194+0000) 2024-06-10T01:34:13.952 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:13 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:13.527+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.899965+0000 front 2024-06-10T01:33:48.899985+0000 (oldest deadline 2024-06-10T01:34:11.800171+0000) 2024-06-10T01:34:14.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:13 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:13.749+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:39.800756+0000 front 2024-06-10T01:33:39.800843+0000 (oldest deadline 2024-06-10T01:34:02.701017+0000) 2024-06-10T01:34:14.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:13 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:13.749+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:45.602414+0000 front 2024-06-10T01:33:45.602288+0000 (oldest deadline 2024-06-10T01:34:11.502446+0000) 2024-06-10T01:34:14.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:13 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:13.806+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:14.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:13 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:13.806+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:14.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:13 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:13.806+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:14.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:14 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:14.208+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:14.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:14 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:14.208+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:47.685681+0000 front 2024-06-10T01:33:47.685658+0000 (oldest deadline 2024-06-10T01:34:11.185884+0000) 2024-06-10T01:34:14.512 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:14 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T01:34:14.485+0000 7ff8f563f700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.095045+0000 front 2024-06-10T01:33:40.095085+0000 (oldest deadline 2024-06-10T01:34:04.795199+0000) 2024-06-10T01:34:14.512 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:14 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:14.244+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:14.512 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:14 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:14.244+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:14.512 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:14 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:14.244+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:14.951 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:14 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:14.551+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:14.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:14 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:14.551+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:14.952 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:14 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:14.568+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:30.395424+0000 front 2024-06-10T01:33:30.395540+0000 (oldest deadline 2024-06-10T01:33:55.095518+0000) 2024-06-10T01:34:14.952 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:14 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:14.568+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.798134+0000 front 2024-06-10T01:33:40.798208+0000 (oldest deadline 2024-06-10T01:34:03.098194+0000) 2024-06-10T01:34:14.952 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:14 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5[48214]: 2024-06-10T01:34:14.568+0000 7f3109b52700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.899965+0000 front 2024-06-10T01:33:48.899985+0000 (oldest deadline 2024-06-10T01:34:11.800171+0000) 2024-06-10T01:34:15.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:14 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:14.783+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:39.800756+0000 front 2024-06-10T01:33:39.800843+0000 (oldest deadline 2024-06-10T01:34:02.701017+0000) 2024-06-10T01:34:15.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:14 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:14.783+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:45.602414+0000 front 2024-06-10T01:33:45.602288+0000 (oldest deadline 2024-06-10T01:34:11.502446+0000) 2024-06-10T01:34:15.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:14 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:14.855+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:15.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:14 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:14.855+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:15.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:14 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:14.855+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:15.549 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:15 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:15.223+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:15.549 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:15 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:15.223+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:47.685681+0000 front 2024-06-10T01:33:47.685658+0000 (oldest deadline 2024-06-10T01:34:11.185884+0000) 2024-06-10T01:34:15.762 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:15 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T01:34:15.515+0000 7ff8f563f700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.095045+0000 front 2024-06-10T01:33:40.095085+0000 (oldest deadline 2024-06-10T01:34:04.795199+0000) 2024-06-10T01:34:15.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:15 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:15.275+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:15.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:15 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:15.275+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:15.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:15 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:15.275+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:15.833 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:15 smithi175 podman[79493]: 2024-06-10 01:34:15.556475105 +0000 UTC m=+5.647832610 container died 635dcc0e148abe427192d83ad016d55de048c113426ba3190fcf801d9848a924 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5) 2024-06-10T01:34:15.833 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:15 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:15.572+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:15.834 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:15 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:15.572+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:16.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:15 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:15.816+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:39.800756+0000 front 2024-06-10T01:33:39.800843+0000 (oldest deadline 2024-06-10T01:34:02.701017+0000) 2024-06-10T01:34:16.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:15 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:15.816+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:45.602414+0000 front 2024-06-10T01:33:45.602288+0000 (oldest deadline 2024-06-10T01:34:11.502446+0000) 2024-06-10T01:34:16.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:15 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:15.832+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:16.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:15 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:15.832+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:16.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:15 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:15.832+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:16.169 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:15 smithi175 podman[79493]: 2024-06-10 01:34:15.931096978 +0000 UTC m=+6.022454468 container cleanup 635dcc0e148abe427192d83ad016d55de048c113426ba3190fcf801d9848a924 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T01:34:16.169 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:15 smithi175 bash[79493]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5 2024-06-10T01:34:16.451 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:16 smithi175 podman[79508]: 2024-06-10 01:34:16.1824338 +0000 UTC m=+0.620685156 container remove 635dcc0e148abe427192d83ad016d55de048c113426ba3190fcf801d9848a924 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-06-10T01:34:16.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:16 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:16.175+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:16.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:16 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:16.175+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:47.685681+0000 front 2024-06-10T01:33:47.685658+0000 (oldest deadline 2024-06-10T01:34:11.185884+0000) 2024-06-10T01:34:16.762 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:16 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T01:34:16.501+0000 7ff8f563f700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.095045+0000 front 2024-06-10T01:33:40.095085+0000 (oldest deadline 2024-06-10T01:34:04.795199+0000) 2024-06-10T01:34:16.762 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:16 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T01:34:16.501+0000 7ff8f563f700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.698122+0000 front 2024-06-10T01:33:48.698145+0000 (oldest deadline 2024-06-10T01:34:15.699058+0000) 2024-06-10T01:34:16.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:16 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:16.308+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:16.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:16 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:16.308+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:16.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:16 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:16.308+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:16.837 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:16 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:16.635+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:16.837 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:16 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:16.635+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:16.837 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:16 smithi175 podman[79599]: 2024-06-10 01:34:16.492008475 +0000 UTC m=+0.111891442 container create 5265b700de7bc733cad00eebecf33d2fc245ad1467b1e8a04df4a3328128aa4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-06-10T01:34:16.837 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:16 smithi175 podman[79599]: 2024-06-10 01:34:16.397282402 +0000 UTC m=+0.017165361 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:34:16.838 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:16 smithi175 podman[79599]: 2024-06-10 01:34:16.727943048 +0000 UTC m=+0.347825997 container init 5265b700de7bc733cad00eebecf33d2fc245ad1467b1e8a04df4a3328128aa4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-06-10T01:34:16.838 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:16 smithi175 podman[79599]: 2024-06-10 01:34:16.730961613 +0000 UTC m=+0.350844562 container start 5265b700de7bc733cad00eebecf33d2fc245ad1467b1e8a04df4a3328128aa4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528) 2024-06-10T01:34:16.838 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:16 smithi175 podman[79599]: 2024-06-10 01:34:16.844425236 +0000 UTC m=+0.464308188 container attach 5265b700de7bc733cad00eebecf33d2fc245ad1467b1e8a04df4a3328128aa4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0) 2024-06-10T01:34:17.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:16 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:16.799+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:39.800756+0000 front 2024-06-10T01:33:39.800843+0000 (oldest deadline 2024-06-10T01:34:02.701017+0000) 2024-06-10T01:34:17.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:16 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:16.799+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:45.602414+0000 front 2024-06-10T01:33:45.602288+0000 (oldest deadline 2024-06-10T01:34:11.502446+0000) 2024-06-10T01:34:17.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:16 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:16.783+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:17.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:16 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:16.783+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:17.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:16 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:16.783+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:17.146 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jun 10 01:34:17 smithi175 podman[79599]: 2024-06-10 01:34:17.015743206 +0000 UTC m=+0.635626163 container died 5265b700de7bc733cad00eebecf33d2fc245ad1467b1e8a04df4a3328128aa4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-06-10T01:34:17.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:17 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:17.152+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:17.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:17 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:17.152+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:47.685681+0000 front 2024-06-10T01:33:47.685658+0000 (oldest deadline 2024-06-10T01:34:11.185884+0000) 2024-06-10T01:34:17.605 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.5.service' 2024-06-10T01:34:17.643 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:34:17.643 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-06-10T01:34:17.644 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-06-10T01:34:17.644 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.6 2024-06-10T01:34:17.663 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:17 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T01:34:17.481+0000 7ff8f563f700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.095045+0000 front 2024-06-10T01:33:40.095085+0000 (oldest deadline 2024-06-10T01:34:04.795199+0000) 2024-06-10T01:34:17.663 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:17 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T01:34:17.481+0000 7ff8f563f700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.698122+0000 front 2024-06-10T01:33:48.698145+0000 (oldest deadline 2024-06-10T01:34:15.699058+0000) 2024-06-10T01:34:17.663 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:17 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:17.312+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:17.663 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:17 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:17.312+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:17.663 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:17 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:17.312+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:17.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:17 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:17.671+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:17.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:17 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:17.671+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:17.970 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:17 smithi007 systemd[1]: Stopping Ceph osd.6 for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:34:18.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:17 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:17.810+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:39.800756+0000 front 2024-06-10T01:33:39.800843+0000 (oldest deadline 2024-06-10T01:34:02.701017+0000) 2024-06-10T01:34:18.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:17 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:17.810+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:45.602414+0000 front 2024-06-10T01:33:45.602288+0000 (oldest deadline 2024-06-10T01:34:11.502446+0000) 2024-06-10T01:34:18.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:17 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:17.748+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:18.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:17 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:17.748+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:18.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:17 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:17.748+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:18.262 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:17 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T01:34:17.982+0000 7ff8f904b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:34:18.262 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:17 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T01:34:17.982+0000 7ff8f904b700 -1 osd.6 106 *** Got signal Terminated *** 2024-06-10T01:34:18.262 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:17 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T01:34:17.982+0000 7ff8f904b700 -1 osd.6 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T01:34:18.451 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:18 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:18.167+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:18.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:18 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:18.167+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:47.685681+0000 front 2024-06-10T01:33:47.685658+0000 (oldest deadline 2024-06-10T01:34:11.185884+0000) 2024-06-10T01:34:18.762 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:18 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T01:34:18.470+0000 7ff8f563f700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.095045+0000 front 2024-06-10T01:33:40.095085+0000 (oldest deadline 2024-06-10T01:34:04.795199+0000) 2024-06-10T01:34:18.762 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:18 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T01:34:18.470+0000 7ff8f563f700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.698122+0000 front 2024-06-10T01:33:48.698145+0000 (oldest deadline 2024-06-10T01:34:15.699058+0000) 2024-06-10T01:34:18.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:18 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:18.275+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:18.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:18 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:18.275+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:18.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:18 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:18.275+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:18.951 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:18 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:18.703+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:18.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:18 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:18.703+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:19.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:18 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:18.788+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:39.800756+0000 front 2024-06-10T01:33:39.800843+0000 (oldest deadline 2024-06-10T01:34:02.701017+0000) 2024-06-10T01:34:19.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:18 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:18.788+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:45.602414+0000 front 2024-06-10T01:33:45.602288+0000 (oldest deadline 2024-06-10T01:34:11.502446+0000) 2024-06-10T01:34:19.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:18 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:18.760+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:19.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:18 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:18.760+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:19.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:18 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:18.760+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:19.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:19 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:19.136+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:19.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:19 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:19.136+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:47.685681+0000 front 2024-06-10T01:33:47.685658+0000 (oldest deadline 2024-06-10T01:34:11.185884+0000) 2024-06-10T01:34:19.512 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:19 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T01:34:19.423+0000 7ff8f563f700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.095045+0000 front 2024-06-10T01:33:40.095085+0000 (oldest deadline 2024-06-10T01:34:04.795199+0000) 2024-06-10T01:34:19.512 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:19 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T01:34:19.423+0000 7ff8f563f700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.698122+0000 front 2024-06-10T01:33:48.698145+0000 (oldest deadline 2024-06-10T01:34:15.699058+0000) 2024-06-10T01:34:19.512 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:19 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:19.263+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:19.512 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:19 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:19.264+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:19.512 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:19 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:19.264+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:20.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:19 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:19.837+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:39.800756+0000 front 2024-06-10T01:33:39.800843+0000 (oldest deadline 2024-06-10T01:34:02.701017+0000) 2024-06-10T01:34:20.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:19 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:19.837+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:45.602414+0000 front 2024-06-10T01:33:45.602288+0000 (oldest deadline 2024-06-10T01:34:11.502446+0000) 2024-06-10T01:34:20.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:19 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:19.775+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:20.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:19 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:19.775+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:20.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:19 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:19.775+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:20.154 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:19 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:19.737+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:20.154 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:19 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:19.737+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:20.451 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:20 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:20.160+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:20.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:20 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:20.160+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:47.685681+0000 front 2024-06-10T01:33:47.685658+0000 (oldest deadline 2024-06-10T01:34:11.185884+0000) 2024-06-10T01:34:20.762 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:20 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T01:34:20.423+0000 7ff8f563f700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.095045+0000 front 2024-06-10T01:33:40.095085+0000 (oldest deadline 2024-06-10T01:34:04.795199+0000) 2024-06-10T01:34:20.762 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:20 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T01:34:20.423+0000 7ff8f563f700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.698122+0000 front 2024-06-10T01:33:48.698145+0000 (oldest deadline 2024-06-10T01:34:15.699058+0000) 2024-06-10T01:34:20.762 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:20 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T01:34:20.423+0000 7ff8f563f700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:57.901161+0000 front 2024-06-10T01:33:57.901190+0000 (oldest deadline 2024-06-10T01:34:20.201462+0000) 2024-06-10T01:34:20.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:20 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:20.279+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:20.763 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:20 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:20.279+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:20.763 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:20 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:20.279+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:21.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:20 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:20.807+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:39.800756+0000 front 2024-06-10T01:33:39.800843+0000 (oldest deadline 2024-06-10T01:34:02.701017+0000) 2024-06-10T01:34:21.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:20 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:20.807+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:45.602414+0000 front 2024-06-10T01:33:45.602288+0000 (oldest deadline 2024-06-10T01:34:11.502446+0000) 2024-06-10T01:34:21.079 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:20 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:20.807+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:57.403580+0000 front 2024-06-10T01:33:57.403541+0000 (oldest deadline 2024-06-10T01:34:20.303731+0000) 2024-06-10T01:34:21.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:20 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:20.782+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:21.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:20 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:20.782+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:21.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:20 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:20.782+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:21.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:20 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:20.782+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:21.117 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:20 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:20.753+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:21.117 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:20 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:20.753+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:21.117 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:20 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:20.753+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:21.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:21 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:21.123+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:21.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:21 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:21.123+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:47.685681+0000 front 2024-06-10T01:33:47.685658+0000 (oldest deadline 2024-06-10T01:34:11.185884+0000) 2024-06-10T01:34:21.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:21 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:21.123+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:54.989192+0000 front 2024-06-10T01:33:54.989165+0000 (oldest deadline 2024-06-10T01:34:20.790023+0000) 2024-06-10T01:34:21.512 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:21 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T01:34:21.455+0000 7ff8f563f700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.095045+0000 front 2024-06-10T01:33:40.095085+0000 (oldest deadline 2024-06-10T01:34:04.795199+0000) 2024-06-10T01:34:21.512 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:21 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T01:34:21.455+0000 7ff8f563f700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.698122+0000 front 2024-06-10T01:33:48.698145+0000 (oldest deadline 2024-06-10T01:34:15.699058+0000) 2024-06-10T01:34:21.512 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:21 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T01:34:21.455+0000 7ff8f563f700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:57.901161+0000 front 2024-06-10T01:33:57.901190+0000 (oldest deadline 2024-06-10T01:34:20.201462+0000) 2024-06-10T01:34:21.512 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:21 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:21.268+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:21.512 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:21 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:21.268+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:21.512 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:21 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:21.268+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:22.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:21 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:21.821+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:39.800756+0000 front 2024-06-10T01:33:39.800843+0000 (oldest deadline 2024-06-10T01:34:02.701017+0000) 2024-06-10T01:34:22.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:21 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:21.821+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:45.602414+0000 front 2024-06-10T01:33:45.602288+0000 (oldest deadline 2024-06-10T01:34:11.502446+0000) 2024-06-10T01:34:22.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:21 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:21.821+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:57.403580+0000 front 2024-06-10T01:33:57.403541+0000 (oldest deadline 2024-06-10T01:34:20.303731+0000) 2024-06-10T01:34:22.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:21 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:21.743+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:22.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:21 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:21.743+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:22.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:21 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:21.743+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:22.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:21 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:21.743+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:22.154 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:21 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:21.776+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:22.154 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:21 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:21.776+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:22.154 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:21 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:21.776+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:22.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:22 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:22.159+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:22.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:22 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:22.159+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:47.685681+0000 front 2024-06-10T01:33:47.685658+0000 (oldest deadline 2024-06-10T01:34:11.185884+0000) 2024-06-10T01:34:22.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:22 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:22.159+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:54.989192+0000 front 2024-06-10T01:33:54.989165+0000 (oldest deadline 2024-06-10T01:34:20.790023+0000) 2024-06-10T01:34:22.762 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:22 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T01:34:22.457+0000 7ff8f563f700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.095045+0000 front 2024-06-10T01:33:40.095085+0000 (oldest deadline 2024-06-10T01:34:04.795199+0000) 2024-06-10T01:34:22.762 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:22 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T01:34:22.457+0000 7ff8f563f700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.698122+0000 front 2024-06-10T01:33:48.698145+0000 (oldest deadline 2024-06-10T01:34:15.699058+0000) 2024-06-10T01:34:22.762 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:22 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6[61030]: 2024-06-10T01:34:22.457+0000 7ff8f563f700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:57.901161+0000 front 2024-06-10T01:33:57.901190+0000 (oldest deadline 2024-06-10T01:34:20.201462+0000) 2024-06-10T01:34:22.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:22 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:22.307+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:22.763 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:22 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:22.307+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:22.763 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:22 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:22.307+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:22.763 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:22 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:22.307+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:58.032646+0000 front 2024-06-10T01:33:58.032670+0000 (oldest deadline 2024-06-10T01:34:22.132848+0000) 2024-06-10T01:34:23.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:22 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:22.800+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:39.800756+0000 front 2024-06-10T01:33:39.800843+0000 (oldest deadline 2024-06-10T01:34:02.701017+0000) 2024-06-10T01:34:23.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:22 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:22.800+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:45.602414+0000 front 2024-06-10T01:33:45.602288+0000 (oldest deadline 2024-06-10T01:34:11.502446+0000) 2024-06-10T01:34:23.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:22 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:22.800+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:57.403580+0000 front 2024-06-10T01:33:57.403541+0000 (oldest deadline 2024-06-10T01:34:20.303731+0000) 2024-06-10T01:34:23.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:22 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:22.738+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:23.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:22 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:22.738+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:23.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:22 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:22.738+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:23.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:22 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:22.738+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:23.105 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:22 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:22.760+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:23.106 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:22 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:22.760+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:23.106 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:22 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:22.760+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:23.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:23 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:23.112+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:23.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:23 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:23.112+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:47.685681+0000 front 2024-06-10T01:33:47.685658+0000 (oldest deadline 2024-06-10T01:34:11.185884+0000) 2024-06-10T01:34:23.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:23 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:23.112+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:54.989192+0000 front 2024-06-10T01:33:54.989165+0000 (oldest deadline 2024-06-10T01:34:20.790023+0000) 2024-06-10T01:34:23.618 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:23 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:23.308+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:23.618 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:23 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:23.308+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:23.618 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:23 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:23.308+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:23.618 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:23 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:23.308+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:58.032646+0000 front 2024-06-10T01:33:58.032670+0000 (oldest deadline 2024-06-10T01:34:22.132848+0000) 2024-06-10T01:34:23.911 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:23 smithi007 podman[160452]: 2024-06-10 01:34:23.624175209 +0000 UTC m=+5.765290869 container died 6a93b8ca5f4d74aa844f79bd1ef3332d1d0ed8ccb53212a8e4c097456d0bdf2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T01:34:24.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:23 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:23.780+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:39.800756+0000 front 2024-06-10T01:33:39.800843+0000 (oldest deadline 2024-06-10T01:34:02.701017+0000) 2024-06-10T01:34:24.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:23 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:23.780+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:45.602414+0000 front 2024-06-10T01:33:45.602288+0000 (oldest deadline 2024-06-10T01:34:11.502446+0000) 2024-06-10T01:34:24.079 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:23 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:23.780+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:57.403580+0000 front 2024-06-10T01:33:57.403541+0000 (oldest deadline 2024-06-10T01:34:20.303731+0000) 2024-06-10T01:34:24.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:23 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:23.692+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:24.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:23 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:23.692+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:24.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:23 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:23.692+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:24.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:23 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:23.692+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:24.134 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:23 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:23.785+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:24.134 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:23 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:23.785+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:24.134 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:23 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:23.785+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:24.262 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:24 smithi007 podman[160452]: 2024-06-10 01:34:24.08309592 +0000 UTC m=+6.224211571 container cleanup 6a93b8ca5f4d74aa844f79bd1ef3332d1d0ed8ccb53212a8e4c097456d0bdf2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T01:34:24.262 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:24 smithi007 bash[160452]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6 2024-06-10T01:34:24.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:24 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:24.140+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:24.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:24 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:24.140+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:47.685681+0000 front 2024-06-10T01:33:47.685658+0000 (oldest deadline 2024-06-10T01:34:11.185884+0000) 2024-06-10T01:34:24.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:24 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:24.140+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:54.989192+0000 front 2024-06-10T01:33:54.989165+0000 (oldest deadline 2024-06-10T01:34:20.790023+0000) 2024-06-10T01:34:24.715 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:24 smithi007 podman[160477]: 2024-06-10 01:34:24.393078118 +0000 UTC m=+0.765720604 container remove 6a93b8ca5f4d74aa844f79bd1ef3332d1d0ed8ccb53212a8e4c097456d0bdf2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc) 2024-06-10T01:34:24.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:24 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:24.344+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:24.716 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:24 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:24.344+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:24.716 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:24 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:24.344+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:24.716 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:24 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:24.344+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:58.032646+0000 front 2024-06-10T01:33:58.032670+0000 (oldest deadline 2024-06-10T01:34:22.132848+0000) 2024-06-10T01:34:24.982 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:24 smithi007 podman[160567]: 2024-06-10 01:34:24.625615693 +0000 UTC m=+0.018195139 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:34:24.982 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:24 smithi007 podman[160567]: 2024-06-10 01:34:24.727116576 +0000 UTC m=+0.119695997 container create 4ab22259dd2c3e4d895b1f00029b7f5a9e9e58d9600f21726f826eebe8100ffa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, ceph=True, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T01:34:24.982 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:24 smithi007 podman[160567]: 2024-06-10 01:34:24.988759222 +0000 UTC m=+0.381338665 container init 4ab22259dd2c3e4d895b1f00029b7f5a9e9e58d9600f21726f826eebe8100ffa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528) 2024-06-10T01:34:25.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:24 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:24.779+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:39.800756+0000 front 2024-06-10T01:33:39.800843+0000 (oldest deadline 2024-06-10T01:34:02.701017+0000) 2024-06-10T01:34:25.079 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:24 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:24.779+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:45.602414+0000 front 2024-06-10T01:33:45.602288+0000 (oldest deadline 2024-06-10T01:34:11.502446+0000) 2024-06-10T01:34:25.079 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:24 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:24.779+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:57.403580+0000 front 2024-06-10T01:33:57.403541+0000 (oldest deadline 2024-06-10T01:34:20.303731+0000) 2024-06-10T01:34:25.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:24 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:24.706+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:25.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:24 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:24.706+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:25.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:24 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:24.706+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:25.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:24 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:24.706+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:25.114 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:24 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:24.781+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:25.114 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:24 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:24.781+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:25.114 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:24 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:24.781+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:25.262 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:24 smithi007 podman[160567]: 2024-06-10 01:34:24.992590461 +0000 UTC m=+0.385169899 container start 4ab22259dd2c3e4d895b1f00029b7f5a9e9e58d9600f21726f826eebe8100ffa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_CLEAN=True) 2024-06-10T01:34:25.262 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:25 smithi007 podman[160567]: 2024-06-10 01:34:25.063106576 +0000 UTC m=+0.455686024 container attach 4ab22259dd2c3e4d895b1f00029b7f5a9e9e58d9600f21726f826eebe8100ffa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS) 2024-06-10T01:34:25.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:25 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:25.120+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:25.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:25 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:25.120+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:47.685681+0000 front 2024-06-10T01:33:47.685658+0000 (oldest deadline 2024-06-10T01:34:11.185884+0000) 2024-06-10T01:34:25.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:25 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:25.120+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:54.989192+0000 front 2024-06-10T01:33:54.989165+0000 (oldest deadline 2024-06-10T01:34:20.790023+0000) 2024-06-10T01:34:25.576 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 10 01:34:25 smithi007 podman[160567]: 2024-06-10 01:34:25.294802877 +0000 UTC m=+0.687382307 container died 4ab22259dd2c3e4d895b1f00029b7f5a9e9e58d9600f21726f826eebe8100ffa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-06-10T01:34:25.635 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:25 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:25.311+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:25.635 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:25 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:25.311+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:25.635 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:25 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:25.311+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:25.635 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:25 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:25.311+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:58.032646+0000 front 2024-06-10T01:33:58.032670+0000 (oldest deadline 2024-06-10T01:34:22.132848+0000) 2024-06-10T01:34:25.850 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.6.service' 2024-06-10T01:34:25.935 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:34:25.935 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-06-10T01:34:25.935 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-06-10T01:34:25.935 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.7 2024-06-10T01:34:26.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:25 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:25.750+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:26.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:25 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:25.750+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:26.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:25 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:25.750+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:26.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:25 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:25.750+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:26.079 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:25 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:25.799+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:39.800756+0000 front 2024-06-10T01:33:39.800843+0000 (oldest deadline 2024-06-10T01:34:02.701017+0000) 2024-06-10T01:34:26.079 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:25 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:25.799+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:45.602414+0000 front 2024-06-10T01:33:45.602288+0000 (oldest deadline 2024-06-10T01:34:11.502446+0000) 2024-06-10T01:34:26.079 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:25 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:25.799+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:57.403580+0000 front 2024-06-10T01:33:57.403541+0000 (oldest deadline 2024-06-10T01:34:20.303731+0000) 2024-06-10T01:34:26.079 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:25 smithi078 systemd[1]: Stopping Ceph osd.7 for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:34:26.116 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:25 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:25.771+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:26.117 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:25 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:25.771+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:26.117 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:25 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:25.771+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:26.387 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:26 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:26.123+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:26.387 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:26 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:26.123+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:47.685681+0000 front 2024-06-10T01:33:47.685658+0000 (oldest deadline 2024-06-10T01:34:11.185884+0000) 2024-06-10T01:34:26.387 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:26 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:26.123+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:54.989192+0000 front 2024-06-10T01:33:54.989165+0000 (oldest deadline 2024-06-10T01:34:20.790023+0000) 2024-06-10T01:34:26.578 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:26 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:26.254+0000 7fc64bc8b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:34:26.578 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:26 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:26.254+0000 7fc64bc8b700 -1 osd.7 106 *** Got signal Terminated *** 2024-06-10T01:34:26.579 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:26 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:26.254+0000 7fc64bc8b700 -1 osd.7 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T01:34:26.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:26 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:26.327+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:26.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:26 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:26.327+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:26.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:26 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:26.327+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:26.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:26 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:26.327+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:58.032646+0000 front 2024-06-10T01:33:58.032670+0000 (oldest deadline 2024-06-10T01:34:22.132848+0000) 2024-06-10T01:34:27.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:26 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:26.849+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:39.800756+0000 front 2024-06-10T01:33:39.800843+0000 (oldest deadline 2024-06-10T01:34:02.701017+0000) 2024-06-10T01:34:27.079 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:26 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:26.849+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:45.602414+0000 front 2024-06-10T01:33:45.602288+0000 (oldest deadline 2024-06-10T01:34:11.502446+0000) 2024-06-10T01:34:27.079 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:26 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:26.849+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:57.403580+0000 front 2024-06-10T01:33:57.403541+0000 (oldest deadline 2024-06-10T01:34:20.303731+0000) 2024-06-10T01:34:27.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:26 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:26.763+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:27.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:26 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:26.763+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:27.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:26 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:26.763+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:27.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:26 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:26.763+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:27.086 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:26 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:26.757+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:27.086 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:26 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:26.757+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:27.086 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:26 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:26.757+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:27.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:27 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:27.092+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:27.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:27 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:27.092+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:47.685681+0000 front 2024-06-10T01:33:47.685658+0000 (oldest deadline 2024-06-10T01:34:11.185884+0000) 2024-06-10T01:34:27.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:27 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:27.092+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:54.989192+0000 front 2024-06-10T01:33:54.989165+0000 (oldest deadline 2024-06-10T01:34:20.790023+0000) 2024-06-10T01:34:27.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:27 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:27.279+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:27.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:27 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:27.279+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:27.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:27 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:27.279+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:27.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:27 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:27.279+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:58.032646+0000 front 2024-06-10T01:33:58.032670+0000 (oldest deadline 2024-06-10T01:34:22.132848+0000) 2024-06-10T01:34:28.043 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:27 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:27.793+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:28.043 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:27 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:27.793+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:28.043 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:27 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:27.793+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:28.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:27 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:27.820+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:39.800756+0000 front 2024-06-10T01:33:39.800843+0000 (oldest deadline 2024-06-10T01:34:02.701017+0000) 2024-06-10T01:34:28.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:27 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:27.820+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:45.602414+0000 front 2024-06-10T01:33:45.602288+0000 (oldest deadline 2024-06-10T01:34:11.502446+0000) 2024-06-10T01:34:28.079 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:27 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:27.820+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:57.403580+0000 front 2024-06-10T01:33:57.403541+0000 (oldest deadline 2024-06-10T01:34:20.303731+0000) 2024-06-10T01:34:28.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:27 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:27.730+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:28.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:27 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:27.730+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:28.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:27 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:27.730+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:28.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:27 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:27.730+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:28.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:28 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:28.049+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:28.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:28 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:28.049+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:47.685681+0000 front 2024-06-10T01:33:47.685658+0000 (oldest deadline 2024-06-10T01:34:11.185884+0000) 2024-06-10T01:34:28.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:28 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:28.049+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:54.989192+0000 front 2024-06-10T01:33:54.989165+0000 (oldest deadline 2024-06-10T01:34:20.790023+0000) 2024-06-10T01:34:28.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:28 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:28.304+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:28.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:28 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:28.304+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:28.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:28 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:28.304+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:28.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:28 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:28.304+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:58.032646+0000 front 2024-06-10T01:33:58.032670+0000 (oldest deadline 2024-06-10T01:34:22.132848+0000) 2024-06-10T01:34:29.079 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:28 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:28.834+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:39.800756+0000 front 2024-06-10T01:33:39.800843+0000 (oldest deadline 2024-06-10T01:34:02.701017+0000) 2024-06-10T01:34:29.079 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:28 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:28.834+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:45.602414+0000 front 2024-06-10T01:33:45.602288+0000 (oldest deadline 2024-06-10T01:34:11.502446+0000) 2024-06-10T01:34:29.079 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:28 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:28.834+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:57.403580+0000 front 2024-06-10T01:33:57.403541+0000 (oldest deadline 2024-06-10T01:34:20.303731+0000) 2024-06-10T01:34:29.079 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:28 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:28.834+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:05.604543+0000 front 2024-06-10T01:34:05.604606+0000 (oldest deadline 2024-06-10T01:34:27.904739+0000) 2024-06-10T01:34:29.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:28 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:28.750+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:29.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:28 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:28.750+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:29.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:28 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:28.750+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:29.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:28 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:28.750+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:29.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:28 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:28.811+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:29.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:28 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:28.811+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:29.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:28 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:28.811+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:29.202 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:29 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:29.054+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:29.202 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:29 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:29.054+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:47.685681+0000 front 2024-06-10T01:33:47.685658+0000 (oldest deadline 2024-06-10T01:34:11.185884+0000) 2024-06-10T01:34:29.202 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:29 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:29.054+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:54.989192+0000 front 2024-06-10T01:33:54.989165+0000 (oldest deadline 2024-06-10T01:34:20.790023+0000) 2024-06-10T01:34:29.202 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:29 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:29.054+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.291410+0000 front 2024-06-10T01:34:04.291569+0000 (oldest deadline 2024-06-10T01:34:28.991491+0000) 2024-06-10T01:34:29.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:29 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:29.327+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:29.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:29 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:29.327+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:29.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:29 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:29.327+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:29.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:29 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:29.327+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:58.032646+0000 front 2024-06-10T01:33:58.032670+0000 (oldest deadline 2024-06-10T01:34:22.132848+0000) 2024-06-10T01:34:29.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:29 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:29.327+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.634283+0000 front 2024-06-10T01:34:02.634217+0000 (oldest deadline 2024-06-10T01:34:28.533948+0000) 2024-06-10T01:34:30.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:29 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:29.795+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:30.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:29 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:29.795+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:30.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:29 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:29.795+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:30.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:29 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:29.795+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:30.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:29 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:29.795+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.632982+0000 front 2024-06-10T01:34:04.632946+0000 (oldest deadline 2024-06-10T01:34:29.333003+0000) 2024-06-10T01:34:30.079 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:29 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:29.834+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:39.800756+0000 front 2024-06-10T01:33:39.800843+0000 (oldest deadline 2024-06-10T01:34:02.701017+0000) 2024-06-10T01:34:30.079 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:29 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:29.834+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:45.602414+0000 front 2024-06-10T01:33:45.602288+0000 (oldest deadline 2024-06-10T01:34:11.502446+0000) 2024-06-10T01:34:30.079 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:29 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:29.834+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:57.403580+0000 front 2024-06-10T01:33:57.403541+0000 (oldest deadline 2024-06-10T01:34:20.303731+0000) 2024-06-10T01:34:30.079 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:29 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:29.834+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:05.604543+0000 front 2024-06-10T01:34:05.604606+0000 (oldest deadline 2024-06-10T01:34:27.904739+0000) 2024-06-10T01:34:30.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:29 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:29.835+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:30.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:29 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:29.835+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:30.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:29 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:29.835+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:30.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:29 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:29.835+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:34:30.202 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:30 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:30.042+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:30.202 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:30 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:30.042+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:47.685681+0000 front 2024-06-10T01:33:47.685658+0000 (oldest deadline 2024-06-10T01:34:11.185884+0000) 2024-06-10T01:34:30.203 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:30 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:30.042+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:54.989192+0000 front 2024-06-10T01:33:54.989165+0000 (oldest deadline 2024-06-10T01:34:20.790023+0000) 2024-06-10T01:34:30.203 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:30 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:30.042+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.291410+0000 front 2024-06-10T01:34:04.291569+0000 (oldest deadline 2024-06-10T01:34:28.991491+0000) 2024-06-10T01:34:30.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:30 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:30.373+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:30.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:30 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:30.373+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:30.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:30 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:30.373+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:30.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:30 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:30.373+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:58.032646+0000 front 2024-06-10T01:33:58.032670+0000 (oldest deadline 2024-06-10T01:34:22.132848+0000) 2024-06-10T01:34:30.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:30 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:30.373+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.634283+0000 front 2024-06-10T01:34:02.634217+0000 (oldest deadline 2024-06-10T01:34:28.533948+0000) 2024-06-10T01:34:31.078 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:30 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:30.857+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:39.800756+0000 front 2024-06-10T01:33:39.800843+0000 (oldest deadline 2024-06-10T01:34:02.701017+0000) 2024-06-10T01:34:31.079 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:30 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:30.857+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:45.602414+0000 front 2024-06-10T01:33:45.602288+0000 (oldest deadline 2024-06-10T01:34:11.502446+0000) 2024-06-10T01:34:31.079 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:30 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:30.857+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:57.403580+0000 front 2024-06-10T01:33:57.403541+0000 (oldest deadline 2024-06-10T01:34:20.303731+0000) 2024-06-10T01:34:31.079 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:30 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7[54407]: 2024-06-10T01:34:30.857+0000 7fc64827f700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:05.604543+0000 front 2024-06-10T01:34:05.604606+0000 (oldest deadline 2024-06-10T01:34:27.904739+0000) 2024-06-10T01:34:31.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:30 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:30.806+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:31.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:30 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:30.806+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:31.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:30 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:30.806+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:31.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:30 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:30.806+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:31.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:30 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:30.806+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.632982+0000 front 2024-06-10T01:34:04.632946+0000 (oldest deadline 2024-06-10T01:34:29.333003+0000) 2024-06-10T01:34:31.085 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:30 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:30.814+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:31.085 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:30 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:30.814+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:31.085 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:30 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:30.814+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:31.085 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:30 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:30.815+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:34:31.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:31 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:31.091+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:31.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:31 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:31.091+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:47.685681+0000 front 2024-06-10T01:33:47.685658+0000 (oldest deadline 2024-06-10T01:34:11.185884+0000) 2024-06-10T01:34:31.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:31 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:31.091+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:54.989192+0000 front 2024-06-10T01:33:54.989165+0000 (oldest deadline 2024-06-10T01:34:20.790023+0000) 2024-06-10T01:34:31.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:31 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:31.091+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.291410+0000 front 2024-06-10T01:34:04.291569+0000 (oldest deadline 2024-06-10T01:34:28.991491+0000) 2024-06-10T01:34:31.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:31 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:31.399+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:31.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:31 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:31.399+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:31.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:31 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:31.399+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:31.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:31 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:31.399+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:58.032646+0000 front 2024-06-10T01:33:58.032670+0000 (oldest deadline 2024-06-10T01:34:22.132848+0000) 2024-06-10T01:34:31.763 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:31 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:31.399+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.634283+0000 front 2024-06-10T01:34:02.634217+0000 (oldest deadline 2024-06-10T01:34:28.533948+0000) 2024-06-10T01:34:32.008 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:31 smithi078 podman[77639]: 2024-06-10 01:34:31.699005722 +0000 UTC m=+5.566445484 container died 78bc30409072d7ee9418c0dabb51c0c357fcb7396804fd0ba46bf18e69ddf096 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True) 2024-06-10T01:34:32.009 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:31 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:31.778+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:32.009 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:31 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:31.778+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:32.009 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:31 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:31.778+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:32.009 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:31 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:31.778+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:32.009 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:31 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:31.778+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.632982+0000 front 2024-06-10T01:34:04.632946+0000 (oldest deadline 2024-06-10T01:34:29.333003+0000) 2024-06-10T01:34:32.061 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:31 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:31.785+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:32.061 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:31 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:31.785+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:32.061 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:31 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:31.785+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:32.061 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:31 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:31.785+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:34:32.328 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:32 smithi078 podman[77639]: 2024-06-10 01:34:32.113835843 +0000 UTC m=+5.981275605 container cleanup 78bc30409072d7ee9418c0dabb51c0c357fcb7396804fd0ba46bf18e69ddf096 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc) 2024-06-10T01:34:32.329 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:32 smithi078 bash[77639]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7 2024-06-10T01:34:32.454 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:32 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:32.067+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:32.455 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:32 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:32.067+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:47.685681+0000 front 2024-06-10T01:33:47.685658+0000 (oldest deadline 2024-06-10T01:34:11.185884+0000) 2024-06-10T01:34:32.455 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:32 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:32.067+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:54.989192+0000 front 2024-06-10T01:33:54.989165+0000 (oldest deadline 2024-06-10T01:34:20.790023+0000) 2024-06-10T01:34:32.455 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:32 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:32.067+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.291410+0000 front 2024-06-10T01:34:04.291569+0000 (oldest deadline 2024-06-10T01:34:28.991491+0000) 2024-06-10T01:34:32.762 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:32 smithi078 podman[77669]: 2024-06-10 01:34:32.365163324 +0000 UTC m=+0.667924105 container remove 78bc30409072d7ee9418c0dabb51c0c357fcb7396804fd0ba46bf18e69ddf096 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-06-10T01:34:32.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:32 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:32.386+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:32.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:32 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:32.386+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:32.763 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:32 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:32.386+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:32.763 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:32 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:32.386+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:58.032646+0000 front 2024-06-10T01:33:58.032670+0000 (oldest deadline 2024-06-10T01:34:22.132848+0000) 2024-06-10T01:34:32.763 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:32 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:32.386+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.634283+0000 front 2024-06-10T01:34:02.634217+0000 (oldest deadline 2024-06-10T01:34:28.533948+0000) 2024-06-10T01:34:33.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:32 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:32.798+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:33.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:32 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:32.798+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:33.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:32 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:32.798+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:33.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:32 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:32.798+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:33.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:32 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:32.798+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.632982+0000 front 2024-06-10T01:34:04.632946+0000 (oldest deadline 2024-06-10T01:34:29.333003+0000) 2024-06-10T01:34:33.079 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:32 smithi078 podman[77756]: 2024-06-10 01:34:32.671643341 +0000 UTC m=+0.018097715 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:34:33.079 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:32 smithi078 podman[77756]: 2024-06-10 01:34:32.791114884 +0000 UTC m=+0.137569287 container create da120cc6939d05931b6903bb05c26a5a5d4960f5b4481c7fcf43354beaf8f200 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T01:34:33.079 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:33 smithi078 podman[77756]: 2024-06-10 01:34:33.002093569 +0000 UTC m=+0.348547929 container init da120cc6939d05931b6903bb05c26a5a5d4960f5b4481c7fcf43354beaf8f200 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-06-10T01:34:33.080 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:33 smithi078 podman[77756]: 2024-06-10 01:34:33.005251239 +0000 UTC m=+0.351705607 container start da120cc6939d05931b6903bb05c26a5a5d4960f5b4481c7fcf43354beaf8f200 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-06-10T01:34:33.202 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:33 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:33.064+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:33.202 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:33 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:33.064+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:47.685681+0000 front 2024-06-10T01:33:47.685658+0000 (oldest deadline 2024-06-10T01:34:11.185884+0000) 2024-06-10T01:34:33.202 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:33 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:33.064+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:54.989192+0000 front 2024-06-10T01:33:54.989165+0000 (oldest deadline 2024-06-10T01:34:20.790023+0000) 2024-06-10T01:34:33.202 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:33 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:33.064+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.291410+0000 front 2024-06-10T01:34:04.291569+0000 (oldest deadline 2024-06-10T01:34:28.991491+0000) 2024-06-10T01:34:33.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:32 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:32.818+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:33.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:32 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:32.818+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:33.203 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:32 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:32.818+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:33.203 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:32 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:32.818+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:34:33.578 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:33 smithi078 podman[77756]: 2024-06-10 01:34:33.104454078 +0000 UTC m=+0.450908454 container attach da120cc6939d05931b6903bb05c26a5a5d4960f5b4481c7fcf43354beaf8f200 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T01:34:33.578 INFO:journalctl@ceph.osd.7.smithi078.stdout:Jun 10 01:34:33 smithi078 podman[77756]: 2024-06-10 01:34:33.297461657 +0000 UTC m=+0.643916033 container died da120cc6939d05931b6903bb05c26a5a5d4960f5b4481c7fcf43354beaf8f200 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc) 2024-06-10T01:34:33.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:33 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:33.419+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:33.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:33 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:33.419+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:33.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:33 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:33.419+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:33.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:33 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:33.419+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:58.032646+0000 front 2024-06-10T01:33:58.032670+0000 (oldest deadline 2024-06-10T01:34:22.132848+0000) 2024-06-10T01:34:33.763 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:33 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:33.419+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.634283+0000 front 2024-06-10T01:34:02.634217+0000 (oldest deadline 2024-06-10T01:34:28.533948+0000) 2024-06-10T01:34:33.873 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:33 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:33.813+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:33.874 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:33 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:33.813+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:33.874 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:33 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:33.813+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:33.874 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:33 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:33.813+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:33.874 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:33 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:33.813+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.632982+0000 front 2024-06-10T01:34:04.632946+0000 (oldest deadline 2024-06-10T01:34:29.333003+0000) 2024-06-10T01:34:33.892 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.7.service' 2024-06-10T01:34:33.930 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:34:33.931 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-06-10T01:34:33.931 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-06-10T01:34:33.931 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.8 2024-06-10T01:34:34.202 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:33 smithi175 systemd[1]: Stopping Ceph osd.8 for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:34:34.202 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:34 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:34.037+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:34.202 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:34 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:34.037+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:47.685681+0000 front 2024-06-10T01:33:47.685658+0000 (oldest deadline 2024-06-10T01:34:11.185884+0000) 2024-06-10T01:34:34.202 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:34 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:34.037+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:54.989192+0000 front 2024-06-10T01:33:54.989165+0000 (oldest deadline 2024-06-10T01:34:20.790023+0000) 2024-06-10T01:34:34.202 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:34 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:34.037+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.291410+0000 front 2024-06-10T01:34:04.291569+0000 (oldest deadline 2024-06-10T01:34:28.991491+0000) 2024-06-10T01:34:34.203 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:33 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:33.801+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:34.203 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:33 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:33.801+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:34.203 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:33 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:33.801+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:34.203 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:33 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:33.801+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:34:34.702 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:34 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:34.249+0000 7f61cfd12700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:34:34.702 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:34 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:34.249+0000 7f61cfd12700 -1 osd.8 106 *** Got signal Terminated *** 2024-06-10T01:34:34.702 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:34 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:34.249+0000 7f61cfd12700 -1 osd.8 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T01:34:34.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:34 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:34.375+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:34.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:34 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:34.375+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:34.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:34 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:34.375+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:34.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:34 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:34.375+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:58.032646+0000 front 2024-06-10T01:33:58.032670+0000 (oldest deadline 2024-06-10T01:34:22.132848+0000) 2024-06-10T01:34:34.763 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:34 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:34.375+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.634283+0000 front 2024-06-10T01:34:02.634217+0000 (oldest deadline 2024-06-10T01:34:28.533948+0000) 2024-06-10T01:34:35.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:34 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:34.780+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:35.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:34 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:34.780+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:35.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:34 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:34.780+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:35.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:34 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:34.780+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:35.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:34 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:34.780+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.632982+0000 front 2024-06-10T01:34:04.632946+0000 (oldest deadline 2024-06-10T01:34:29.333003+0000) 2024-06-10T01:34:35.202 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:35 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:35.002+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:35.202 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:35 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:35.002+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:47.685681+0000 front 2024-06-10T01:33:47.685658+0000 (oldest deadline 2024-06-10T01:34:11.185884+0000) 2024-06-10T01:34:35.202 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:35 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:35.002+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:54.989192+0000 front 2024-06-10T01:33:54.989165+0000 (oldest deadline 2024-06-10T01:34:20.790023+0000) 2024-06-10T01:34:35.202 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:35 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:35.002+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.291410+0000 front 2024-06-10T01:34:04.291569+0000 (oldest deadline 2024-06-10T01:34:28.991491+0000) 2024-06-10T01:34:35.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:34 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:34.831+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:35.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:34 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:34.831+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:35.203 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:34 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:34.831+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:35.203 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:34 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:34.831+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:34:35.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:35 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:35.414+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:35.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:35 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:35.414+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:35.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:35 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:35.414+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:35.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:35 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:35.414+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:58.032646+0000 front 2024-06-10T01:33:58.032670+0000 (oldest deadline 2024-06-10T01:34:22.132848+0000) 2024-06-10T01:34:35.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:35 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:35.414+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.634283+0000 front 2024-06-10T01:34:02.634217+0000 (oldest deadline 2024-06-10T01:34:28.533948+0000) 2024-06-10T01:34:36.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:35 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:35.759+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:36.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:35 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:35.759+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:36.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:35 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:35.759+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:36.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:35 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:35.759+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:36.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:35 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:35.759+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.632982+0000 front 2024-06-10T01:34:04.632946+0000 (oldest deadline 2024-06-10T01:34:29.333003+0000) 2024-06-10T01:34:36.202 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:36 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:36.022+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:36.202 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:36 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:36.022+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:47.685681+0000 front 2024-06-10T01:33:47.685658+0000 (oldest deadline 2024-06-10T01:34:11.185884+0000) 2024-06-10T01:34:36.202 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:36 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:36.022+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:54.989192+0000 front 2024-06-10T01:33:54.989165+0000 (oldest deadline 2024-06-10T01:34:20.790023+0000) 2024-06-10T01:34:36.202 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:36 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:36.022+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.291410+0000 front 2024-06-10T01:34:04.291569+0000 (oldest deadline 2024-06-10T01:34:28.991491+0000) 2024-06-10T01:34:36.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:35 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:35.821+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:36.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:35 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:35.821+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:36.203 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:35 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:35.821+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:36.203 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:35 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:35.821+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:34:36.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:36 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:36.444+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:36.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:36 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:36.444+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:36.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:36 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:36.444+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:36.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:36 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:36.444+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:58.032646+0000 front 2024-06-10T01:33:58.032670+0000 (oldest deadline 2024-06-10T01:34:22.132848+0000) 2024-06-10T01:34:36.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:36 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:36.444+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.634283+0000 front 2024-06-10T01:34:02.634217+0000 (oldest deadline 2024-06-10T01:34:28.533948+0000) 2024-06-10T01:34:37.032 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:36 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:36.774+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:37.032 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:36 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:36.774+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:37.032 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:36 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:36.774+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:37.032 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:36 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:36.774+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:34:37.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:36 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:36.786+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:37.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:36 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:36.786+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:37.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:36 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:36.786+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:37.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:36 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:36.786+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:37.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:36 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:36.786+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.632982+0000 front 2024-06-10T01:34:04.632946+0000 (oldest deadline 2024-06-10T01:34:29.333003+0000) 2024-06-10T01:34:37.396 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:37 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:37.038+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:37.396 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:37 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:37.038+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:47.685681+0000 front 2024-06-10T01:33:47.685658+0000 (oldest deadline 2024-06-10T01:34:11.185884+0000) 2024-06-10T01:34:37.396 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:37 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:37.038+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:54.989192+0000 front 2024-06-10T01:33:54.989165+0000 (oldest deadline 2024-06-10T01:34:20.790023+0000) 2024-06-10T01:34:37.396 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:37 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:37.038+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.291410+0000 front 2024-06-10T01:34:04.291569+0000 (oldest deadline 2024-06-10T01:34:28.991491+0000) 2024-06-10T01:34:37.396 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:37 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:37.038+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:11.292609+0000 front 2024-06-10T01:34:11.292629+0000 (oldest deadline 2024-06-10T01:34:36.592743+0000) 2024-06-10T01:34:37.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:37 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:37.443+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:37.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:37 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:37.443+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:37.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:37 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:37.443+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:37.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:37 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:37.443+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:58.032646+0000 front 2024-06-10T01:33:58.032670+0000 (oldest deadline 2024-06-10T01:34:22.132848+0000) 2024-06-10T01:34:37.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:37 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:37.443+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.634283+0000 front 2024-06-10T01:34:02.634217+0000 (oldest deadline 2024-06-10T01:34:28.533948+0000) 2024-06-10T01:34:37.763 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:37 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:37.443+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:14.435247+0000 front 2024-06-10T01:34:14.435117+0000 (oldest deadline 2024-06-10T01:34:37.335082+0000) 2024-06-10T01:34:38.072 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:37 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:37.766+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:38.072 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:37 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:37.766+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:38.072 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:37 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:37.766+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:38.072 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:37 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:37.766+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:34:38.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:37 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:37.812+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:38.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:37 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:37.812+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:38.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:37 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:37.812+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:38.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:37 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:37.812+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:38.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:37 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:37.812+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.632982+0000 front 2024-06-10T01:34:04.632946+0000 (oldest deadline 2024-06-10T01:34:29.333003+0000) 2024-06-10T01:34:38.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:37 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:37.812+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:11.633900+0000 front 2024-06-10T01:34:11.633930+0000 (oldest deadline 2024-06-10T01:34:37.533908+0000) 2024-06-10T01:34:38.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:38 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:38.077+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:38.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:38 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:38.077+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:47.685681+0000 front 2024-06-10T01:33:47.685658+0000 (oldest deadline 2024-06-10T01:34:11.185884+0000) 2024-06-10T01:34:38.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:38 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:38.077+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:54.989192+0000 front 2024-06-10T01:33:54.989165+0000 (oldest deadline 2024-06-10T01:34:20.790023+0000) 2024-06-10T01:34:38.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:38 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:38.077+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.291410+0000 front 2024-06-10T01:34:04.291569+0000 (oldest deadline 2024-06-10T01:34:28.991491+0000) 2024-06-10T01:34:38.453 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:38 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:38.077+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:11.292609+0000 front 2024-06-10T01:34:11.292629+0000 (oldest deadline 2024-06-10T01:34:36.592743+0000) 2024-06-10T01:34:38.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:38 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:38.406+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:38.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:38 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:38.406+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:38.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:38 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:38.406+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:38.763 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:38 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:38.406+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:58.032646+0000 front 2024-06-10T01:33:58.032670+0000 (oldest deadline 2024-06-10T01:34:22.132848+0000) 2024-06-10T01:34:38.763 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:38 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:38.406+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.634283+0000 front 2024-06-10T01:34:02.634217+0000 (oldest deadline 2024-06-10T01:34:28.533948+0000) 2024-06-10T01:34:38.763 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:38 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:38.406+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:14.435247+0000 front 2024-06-10T01:34:14.435117+0000 (oldest deadline 2024-06-10T01:34:37.335082+0000) 2024-06-10T01:34:39.076 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:38 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:38.737+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:39.076 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:38 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:38.737+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:39.076 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:38 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:38.737+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:39.076 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:38 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:38.737+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:34:39.328 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:38 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:38.838+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:39.329 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:38 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:38.838+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:39.329 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:38 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:38.838+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:39.329 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:38 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:38.838+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:39.329 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:38 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:38.838+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.632982+0000 front 2024-06-10T01:34:04.632946+0000 (oldest deadline 2024-06-10T01:34:29.333003+0000) 2024-06-10T01:34:39.329 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:38 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:38.838+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:11.633900+0000 front 2024-06-10T01:34:11.633930+0000 (oldest deadline 2024-06-10T01:34:37.533908+0000) 2024-06-10T01:34:39.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:39 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:39.082+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:38.383595+0000 front 2024-06-10T01:33:38.383572+0000 (oldest deadline 2024-06-10T01:34:03.683653+0000) 2024-06-10T01:34:39.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:39 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:39.082+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:47.685681+0000 front 2024-06-10T01:33:47.685658+0000 (oldest deadline 2024-06-10T01:34:11.185884+0000) 2024-06-10T01:34:39.452 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:39 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:39.082+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:54.989192+0000 front 2024-06-10T01:33:54.989165+0000 (oldest deadline 2024-06-10T01:34:20.790023+0000) 2024-06-10T01:34:39.453 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:39 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:39.082+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.291410+0000 front 2024-06-10T01:34:04.291569+0000 (oldest deadline 2024-06-10T01:34:28.991491+0000) 2024-06-10T01:34:39.453 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:39 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8[55046]: 2024-06-10T01:34:39.082+0000 7f61cc306700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:11.292609+0000 front 2024-06-10T01:34:11.292629+0000 (oldest deadline 2024-06-10T01:34:36.592743+0000) 2024-06-10T01:34:39.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:39 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:39.359+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:39.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:39 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:39.359+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:39.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:39 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:39.359+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:39.763 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:39 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:39.359+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:58.032646+0000 front 2024-06-10T01:33:58.032670+0000 (oldest deadline 2024-06-10T01:34:22.132848+0000) 2024-06-10T01:34:39.763 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:39 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:39.359+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.634283+0000 front 2024-06-10T01:34:02.634217+0000 (oldest deadline 2024-06-10T01:34:28.533948+0000) 2024-06-10T01:34:39.763 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:39 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:39.359+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:14.435247+0000 front 2024-06-10T01:34:14.435117+0000 (oldest deadline 2024-06-10T01:34:37.335082+0000) 2024-06-10T01:34:40.028 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:39 smithi175 podman[79718]: 2024-06-10 01:34:39.768323222 +0000 UTC m=+5.634267596 container died da89affc86f50b9c58e211bfb7996f270be76dd10b89f0015b9f23720d45b81f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528) 2024-06-10T01:34:40.028 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:39 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:39.779+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:40.028 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:39 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:39.779+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:40.028 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:39 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:39.779+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:40.029 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:39 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:39.779+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:34:40.328 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:39 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:39.869+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:40.329 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:39 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:39.869+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:40.329 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:39 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:39.869+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:40.329 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:39 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:39.869+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:40.329 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:39 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:39.869+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.632982+0000 front 2024-06-10T01:34:04.632946+0000 (oldest deadline 2024-06-10T01:34:29.333003+0000) 2024-06-10T01:34:40.329 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:39 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:39.869+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:11.633900+0000 front 2024-06-10T01:34:11.633930+0000 (oldest deadline 2024-06-10T01:34:37.533908+0000) 2024-06-10T01:34:40.386 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:40 smithi175 podman[79718]: 2024-06-10 01:34:40.133305869 +0000 UTC m=+5.999250240 container cleanup da89affc86f50b9c58e211bfb7996f270be76dd10b89f0015b9f23720d45b81f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240528, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-3d89acc) 2024-06-10T01:34:40.386 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:40 smithi175 bash[79718]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8 2024-06-10T01:34:40.686 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:40 smithi175 podman[79734]: 2024-06-10 01:34:40.39285207 +0000 UTC m=+0.624258507 container remove da89affc86f50b9c58e211bfb7996f270be76dd10b89f0015b9f23720d45b81f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T01:34:40.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:40 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:40.354+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:40.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:40 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:40.354+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:40.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:40 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:40.354+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:40.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:40 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:40.354+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:58.032646+0000 front 2024-06-10T01:33:58.032670+0000 (oldest deadline 2024-06-10T01:34:22.132848+0000) 2024-06-10T01:34:40.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:40 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:40.354+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.634283+0000 front 2024-06-10T01:34:02.634217+0000 (oldest deadline 2024-06-10T01:34:28.533948+0000) 2024-06-10T01:34:40.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:40 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:40.354+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:14.435247+0000 front 2024-06-10T01:34:14.435117+0000 (oldest deadline 2024-06-10T01:34:37.335082+0000) 2024-06-10T01:34:40.939 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:40 smithi175 podman[79819]: 2024-06-10 01:34:40.69352112 +0000 UTC m=+0.101894571 container create 6bcf4aa704f8b1d40299ef054d086a6b26eb3e30e252d968929f548dcc614b6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc) 2024-06-10T01:34:40.939 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:40 smithi175 podman[79819]: 2024-06-10 01:34:40.609098325 +0000 UTC m=+0.017471780 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:34:40.939 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:40 smithi175 podman[79819]: 2024-06-10 01:34:40.946001601 +0000 UTC m=+0.354375047 container init 6bcf4aa704f8b1d40299ef054d086a6b26eb3e30e252d968929f548dcc614b6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-06-10T01:34:40.939 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:40 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:40.798+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:40.939 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:40 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:40.798+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:40.939 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:40 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:40.798+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:40.940 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:40 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:40.798+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:34:40.940 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:40 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:40.798+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:09.220829+0000 front 2024-06-10T01:34:09.220856+0000 (oldest deadline 2024-06-10T01:34:39.821667+0000) 2024-06-10T01:34:41.202 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:40 smithi175 podman[79819]: 2024-06-10 01:34:40.949209153 +0000 UTC m=+0.357582606 container start 6bcf4aa704f8b1d40299ef054d086a6b26eb3e30e252d968929f548dcc614b6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-3d89acc) 2024-06-10T01:34:41.202 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:41 smithi175 podman[79819]: 2024-06-10 01:34:41.046289118 +0000 UTC m=+0.454662583 container attach 6bcf4aa704f8b1d40299ef054d086a6b26eb3e30e252d968929f548dcc614b6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T01:34:41.328 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:40 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:40.835+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:41.328 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:40 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:40.835+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:41.329 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:40 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:40.835+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:41.329 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:40 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:40.835+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:41.329 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:40 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:40.835+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.632982+0000 front 2024-06-10T01:34:04.632946+0000 (oldest deadline 2024-06-10T01:34:29.333003+0000) 2024-06-10T01:34:41.329 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:40 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:40.835+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:11.633900+0000 front 2024-06-10T01:34:11.633930+0000 (oldest deadline 2024-06-10T01:34:37.533908+0000) 2024-06-10T01:34:41.503 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jun 10 01:34:41 smithi175 podman[79819]: 2024-06-10 01:34:41.236217362 +0000 UTC m=+0.644590820 container died 6bcf4aa704f8b1d40299ef054d086a6b26eb3e30e252d968929f548dcc614b6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-06-10T01:34:41.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:41 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:41.378+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:41.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:41 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:41.378+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:41.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:41 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:41.378+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:41.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:41 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:41.378+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:58.032646+0000 front 2024-06-10T01:33:58.032670+0000 (oldest deadline 2024-06-10T01:34:22.132848+0000) 2024-06-10T01:34:41.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:41 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:41.378+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.634283+0000 front 2024-06-10T01:34:02.634217+0000 (oldest deadline 2024-06-10T01:34:28.533948+0000) 2024-06-10T01:34:41.763 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:41 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:41.378+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:14.435247+0000 front 2024-06-10T01:34:14.435117+0000 (oldest deadline 2024-06-10T01:34:37.335082+0000) 2024-06-10T01:34:41.875 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.8.service' 2024-06-10T01:34:41.912 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:34:41.913 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-06-10T01:34:41.913 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-06-10T01:34:41.913 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.9 2024-06-10T01:34:42.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:41 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:41.784+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:42.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:41 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:41.784+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:42.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:41 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:41.784+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:42.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:41 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:41.784+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:34:42.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:41 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:41.784+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:09.220829+0000 front 2024-06-10T01:34:09.220856+0000 (oldest deadline 2024-06-10T01:34:39.821667+0000) 2024-06-10T01:34:42.246 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:41 smithi007 systemd[1]: Stopping Ceph osd.9 for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:34:42.246 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:42 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:42.251+0000 7f96f3c6d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:34:42.246 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:42 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:42.251+0000 7f96f3c6d700 -1 osd.9 106 *** Got signal Terminated *** 2024-06-10T01:34:42.246 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:42 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:42.251+0000 7f96f3c6d700 -1 osd.9 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T01:34:42.328 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:41 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:41.860+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:42.329 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:41 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:41.860+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:42.329 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:41 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:41.860+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:42.329 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:41 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:41.860+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:42.329 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:41 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:41.860+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.632982+0000 front 2024-06-10T01:34:04.632946+0000 (oldest deadline 2024-06-10T01:34:29.333003+0000) 2024-06-10T01:34:42.329 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:41 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:41.860+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:11.633900+0000 front 2024-06-10T01:34:11.633930+0000 (oldest deadline 2024-06-10T01:34:37.533908+0000) 2024-06-10T01:34:42.512 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:42 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:42.426+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:42.512 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:42 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:42.426+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:42.512 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:42 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:42.426+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:42.512 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:42 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:42.426+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:58.032646+0000 front 2024-06-10T01:33:58.032670+0000 (oldest deadline 2024-06-10T01:34:22.132848+0000) 2024-06-10T01:34:42.512 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:42 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:42.426+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.634283+0000 front 2024-06-10T01:34:02.634217+0000 (oldest deadline 2024-06-10T01:34:28.533948+0000) 2024-06-10T01:34:42.512 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:42 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:42.426+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:14.435247+0000 front 2024-06-10T01:34:14.435117+0000 (oldest deadline 2024-06-10T01:34:37.335082+0000) 2024-06-10T01:34:43.081 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:42 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:42.812+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:43.082 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:42 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:42.812+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:43.082 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:42 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:42.812+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:43.082 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:42 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:42.812+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:43.082 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:42 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:42.812+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.632982+0000 front 2024-06-10T01:34:04.632946+0000 (oldest deadline 2024-06-10T01:34:29.333003+0000) 2024-06-10T01:34:43.082 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:42 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:42.812+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:11.633900+0000 front 2024-06-10T01:34:11.633930+0000 (oldest deadline 2024-06-10T01:34:37.533908+0000) 2024-06-10T01:34:43.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:42 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:42.753+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:43.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:42 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:42.753+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:43.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:42 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:42.753+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:43.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:42 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:42.753+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:34:43.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:42 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:42.753+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:09.220829+0000 front 2024-06-10T01:34:09.220856+0000 (oldest deadline 2024-06-10T01:34:39.821667+0000) 2024-06-10T01:34:43.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:43 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:43.473+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:43.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:43 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:43.473+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:43.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:43 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:43.473+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:43.763 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:43 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:43.473+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:58.032646+0000 front 2024-06-10T01:33:58.032670+0000 (oldest deadline 2024-06-10T01:34:22.132848+0000) 2024-06-10T01:34:43.763 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:43 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:43.473+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.634283+0000 front 2024-06-10T01:34:02.634217+0000 (oldest deadline 2024-06-10T01:34:28.533948+0000) 2024-06-10T01:34:43.763 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:43 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:43.473+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:14.435247+0000 front 2024-06-10T01:34:14.435117+0000 (oldest deadline 2024-06-10T01:34:37.335082+0000) 2024-06-10T01:34:44.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:43 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:43.709+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:44.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:43 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:43.709+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:44.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:43 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:43.709+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:44.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:43 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:43.709+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:34:44.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:43 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:43.709+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:09.220829+0000 front 2024-06-10T01:34:09.220856+0000 (oldest deadline 2024-06-10T01:34:39.821667+0000) 2024-06-10T01:34:44.329 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:43 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:43.849+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:44.329 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:43 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:43.849+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:44.329 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:43 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:43.849+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:44.329 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:43 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:43.849+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:44.329 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:43 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:43.849+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.632982+0000 front 2024-06-10T01:34:04.632946+0000 (oldest deadline 2024-06-10T01:34:29.333003+0000) 2024-06-10T01:34:44.329 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:43 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:43.849+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:11.633900+0000 front 2024-06-10T01:34:11.633930+0000 (oldest deadline 2024-06-10T01:34:37.533908+0000) 2024-06-10T01:34:44.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:44 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:44.477+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:44.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:44 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:44.477+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:44.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:44 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:44.477+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:44.763 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:44 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:44.477+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:58.032646+0000 front 2024-06-10T01:33:58.032670+0000 (oldest deadline 2024-06-10T01:34:22.132848+0000) 2024-06-10T01:34:44.763 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:44 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:44.477+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.634283+0000 front 2024-06-10T01:34:02.634217+0000 (oldest deadline 2024-06-10T01:34:28.533948+0000) 2024-06-10T01:34:44.763 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:44 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:44.477+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:14.435247+0000 front 2024-06-10T01:34:14.435117+0000 (oldest deadline 2024-06-10T01:34:37.335082+0000) 2024-06-10T01:34:44.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:44 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:44.702+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:44.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:44 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:44.702+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:44.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:44 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:44.702+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:44.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:44 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:44.702+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:34:44.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:44 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:44.702+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:09.220829+0000 front 2024-06-10T01:34:09.220856+0000 (oldest deadline 2024-06-10T01:34:39.821667+0000) 2024-06-10T01:34:45.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:44 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:44.810+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:45.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:44 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:44.810+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:45.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:44 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:44.810+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:45.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:44 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:44.810+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:45.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:44 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:44.810+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.632982+0000 front 2024-06-10T01:34:04.632946+0000 (oldest deadline 2024-06-10T01:34:29.333003+0000) 2024-06-10T01:34:45.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:44 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:44.810+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:11.633900+0000 front 2024-06-10T01:34:11.633930+0000 (oldest deadline 2024-06-10T01:34:37.533908+0000) 2024-06-10T01:34:45.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:44 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:44.810+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-10T01:34:21.035044+0000 front 2024-06-10T01:34:21.035058+0000 (oldest deadline 2024-06-10T01:34:44.534798+0000) 2024-06-10T01:34:45.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:45 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:45.513+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:45.762 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:45 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:45.513+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:45.763 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:45 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:45.513+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:45.763 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:45 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:45.513+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:58.032646+0000 front 2024-06-10T01:33:58.032670+0000 (oldest deadline 2024-06-10T01:34:22.132848+0000) 2024-06-10T01:34:45.763 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:45 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:45.513+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.634283+0000 front 2024-06-10T01:34:02.634217+0000 (oldest deadline 2024-06-10T01:34:28.533948+0000) 2024-06-10T01:34:45.763 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:45 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:45.513+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:14.435247+0000 front 2024-06-10T01:34:14.435117+0000 (oldest deadline 2024-06-10T01:34:37.335082+0000) 2024-06-10T01:34:46.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:45 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:45.807+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:46.115 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:45 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:45.807+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:46.115 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:45 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:45.807+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:46.115 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:45 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:45.807+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:46.115 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:45 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:45.807+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.632982+0000 front 2024-06-10T01:34:04.632946+0000 (oldest deadline 2024-06-10T01:34:29.333003+0000) 2024-06-10T01:34:46.115 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:45 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:45.807+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:11.633900+0000 front 2024-06-10T01:34:11.633930+0000 (oldest deadline 2024-06-10T01:34:37.533908+0000) 2024-06-10T01:34:46.115 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:45 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:45.807+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-10T01:34:21.035044+0000 front 2024-06-10T01:34:21.035058+0000 (oldest deadline 2024-06-10T01:34:44.534798+0000) 2024-06-10T01:34:46.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:45 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:45.749+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:46.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:45 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:45.749+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:46.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:45 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:45.749+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:46.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:45 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:45.749+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:34:46.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:45 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:45.749+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:09.220829+0000 front 2024-06-10T01:34:09.220856+0000 (oldest deadline 2024-06-10T01:34:39.821667+0000) 2024-06-10T01:34:46.766 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:46 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:46.468+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:29.328454+0000 front 2024-06-10T01:33:29.328477+0000 (oldest deadline 2024-06-10T01:33:54.628644+0000) 2024-06-10T01:34:46.766 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:46 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:46.468+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:41.630678+0000 front 2024-06-10T01:33:41.630781+0000 (oldest deadline 2024-06-10T01:34:06.930656+0000) 2024-06-10T01:34:46.766 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:46 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:46.468+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.231659+0000 front 2024-06-10T01:33:49.231626+0000 (oldest deadline 2024-06-10T01:34:13.331601+0000) 2024-06-10T01:34:46.766 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:46 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:46.468+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:58.032646+0000 front 2024-06-10T01:33:58.032670+0000 (oldest deadline 2024-06-10T01:34:22.132848+0000) 2024-06-10T01:34:46.766 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:46 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:46.468+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.634283+0000 front 2024-06-10T01:34:02.634217+0000 (oldest deadline 2024-06-10T01:34:28.533948+0000) 2024-06-10T01:34:46.766 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:46 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9[67708]: 2024-06-10T01:34:46.468+0000 7f96ec259700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:14.435247+0000 front 2024-06-10T01:34:14.435117+0000 (oldest deadline 2024-06-10T01:34:37.335082+0000) 2024-06-10T01:34:46.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:46 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:46.707+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:46.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:46 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:46.707+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:46.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:46 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:46.707+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:46.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:46 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:46.707+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:34:46.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:46 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:46.707+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:09.220829+0000 front 2024-06-10T01:34:09.220856+0000 (oldest deadline 2024-06-10T01:34:39.821667+0000) 2024-06-10T01:34:46.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:46 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:46.707+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-10T01:34:22.723029+0000 front 2024-06-10T01:34:22.722935+0000 (oldest deadline 2024-06-10T01:34:46.222793+0000) 2024-06-10T01:34:47.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:46 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:46.766+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:47.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:46 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:46.766+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:47.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:46 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:46.766+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:47.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:46 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:46.766+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:47.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:46 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:46.766+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.632982+0000 front 2024-06-10T01:34:04.632946+0000 (oldest deadline 2024-06-10T01:34:29.333003+0000) 2024-06-10T01:34:47.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:46 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:46.766+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:11.633900+0000 front 2024-06-10T01:34:11.633930+0000 (oldest deadline 2024-06-10T01:34:37.533908+0000) 2024-06-10T01:34:47.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:46 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:46.766+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-10T01:34:21.035044+0000 front 2024-06-10T01:34:21.035058+0000 (oldest deadline 2024-06-10T01:34:44.534798+0000) 2024-06-10T01:34:47.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:47 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:47.675+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:47.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:47 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:47.675+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:47.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:47 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:47.675+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:47.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:47 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:47.675+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:34:47.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:47 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:47.675+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:09.220829+0000 front 2024-06-10T01:34:09.220856+0000 (oldest deadline 2024-06-10T01:34:39.821667+0000) 2024-06-10T01:34:47.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:47 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:47.675+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-10T01:34:22.723029+0000 front 2024-06-10T01:34:22.722935+0000 (oldest deadline 2024-06-10T01:34:46.222793+0000) 2024-06-10T01:34:48.013 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:47 smithi007 podman[160673]: 2024-06-10 01:34:47.753275826 +0000 UTC m=+5.628549866 container died 9a637f3e3124e18dba5b4c8b14971ceeb921a13f91099d8ee43b8d647fdd84ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-06-10T01:34:48.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:47 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:47.734+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:48.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:47 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:47.734+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:48.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:47 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:47.734+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:48.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:47 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:47.734+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:48.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:47 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:47.734+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.632982+0000 front 2024-06-10T01:34:04.632946+0000 (oldest deadline 2024-06-10T01:34:29.333003+0000) 2024-06-10T01:34:48.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:47 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:47.734+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:11.633900+0000 front 2024-06-10T01:34:11.633930+0000 (oldest deadline 2024-06-10T01:34:37.533908+0000) 2024-06-10T01:34:48.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:47 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:47.734+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-10T01:34:21.035044+0000 front 2024-06-10T01:34:21.035058+0000 (oldest deadline 2024-06-10T01:34:44.534798+0000) 2024-06-10T01:34:48.287 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:48 smithi007 podman[160673]: 2024-06-10 01:34:48.126433717 +0000 UTC m=+6.001707792 container cleanup 9a637f3e3124e18dba5b4c8b14971ceeb921a13f91099d8ee43b8d647fdd84ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T01:34:48.288 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:48 smithi007 bash[160673]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9 2024-06-10T01:34:48.705 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:48 smithi007 podman[160690]: 2024-06-10 01:34:48.294577562 +0000 UTC m=+0.535342453 container remove 9a637f3e3124e18dba5b4c8b14971ceeb921a13f91099d8ee43b8d647fdd84ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-06-10T01:34:48.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:48 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:48.694+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:48.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:48 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:48.694+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:48.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:48 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:48.694+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:48.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:48 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:48.694+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:34:48.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:48 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:48.694+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:09.220829+0000 front 2024-06-10T01:34:09.220856+0000 (oldest deadline 2024-06-10T01:34:39.821667+0000) 2024-06-10T01:34:48.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:48 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:48.694+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-10T01:34:22.723029+0000 front 2024-06-10T01:34:22.722935+0000 (oldest deadline 2024-06-10T01:34:46.222793+0000) 2024-06-10T01:34:49.007 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:48 smithi007 podman[160780]: 2024-06-10 01:34:48.61490473 +0000 UTC m=+0.017492414 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:34:49.007 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:48 smithi007 podman[160780]: 2024-06-10 01:34:48.712021092 +0000 UTC m=+0.114608767 container create 6031db3e7be24a49d20d769cabfb667b4a2829b33b120dea1972677e2bfb8670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-06-10T01:34:49.007 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:48 smithi007 podman[160780]: 2024-06-10 01:34:48.93929197 +0000 UTC m=+0.341879646 container init 6031db3e7be24a49d20d769cabfb667b4a2829b33b120dea1972677e2bfb8670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T01:34:49.007 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:48 smithi007 podman[160780]: 2024-06-10 01:34:48.94275622 +0000 UTC m=+0.345343899 container start 6031db3e7be24a49d20d769cabfb667b4a2829b33b120dea1972677e2bfb8670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T01:34:49.008 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:49 smithi007 podman[160780]: 2024-06-10 01:34:49.014112422 +0000 UTC m=+0.416700098 container attach 6031db3e7be24a49d20d769cabfb667b4a2829b33b120dea1972677e2bfb8670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T01:34:49.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:48 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:48.720+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:49.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:48 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:48.720+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:49.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:48 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:48.720+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:49.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:48 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:48.720+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:49.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:48 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:48.720+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.632982+0000 front 2024-06-10T01:34:04.632946+0000 (oldest deadline 2024-06-10T01:34:29.333003+0000) 2024-06-10T01:34:49.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:48 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:48.720+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:11.633900+0000 front 2024-06-10T01:34:11.633930+0000 (oldest deadline 2024-06-10T01:34:37.533908+0000) 2024-06-10T01:34:49.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:48 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:48.720+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-10T01:34:21.035044+0000 front 2024-06-10T01:34:21.035058+0000 (oldest deadline 2024-06-10T01:34:44.534798+0000) 2024-06-10T01:34:49.262 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 10 01:34:49 smithi007 podman[160780]: 2024-06-10 01:34:49.244069336 +0000 UTC m=+0.646657021 container died 6031db3e7be24a49d20d769cabfb667b4a2829b33b120dea1972677e2bfb8670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-9-deactivate, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528) 2024-06-10T01:34:49.914 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.9.service' 2024-06-10T01:34:49.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:49 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:49.688+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:49.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:49 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:49.688+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:49.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:49 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:49.688+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:49.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:49 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:49.688+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:34:49.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:49 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:49.688+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:09.220829+0000 front 2024-06-10T01:34:09.220856+0000 (oldest deadline 2024-06-10T01:34:39.821667+0000) 2024-06-10T01:34:49.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:49 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:49.688+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-10T01:34:22.723029+0000 front 2024-06-10T01:34:22.722935+0000 (oldest deadline 2024-06-10T01:34:46.222793+0000) 2024-06-10T01:34:49.957 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:34:49.957 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-06-10T01:34:49.958 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-06-10T01:34:49.958 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.10 2024-06-10T01:34:49.983 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:49 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:49.714+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:49.983 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:49 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:49.714+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:49.984 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:49 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:49.714+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:49.984 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:49 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:49.714+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:49.984 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:49 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:49.714+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.632982+0000 front 2024-06-10T01:34:04.632946+0000 (oldest deadline 2024-06-10T01:34:29.333003+0000) 2024-06-10T01:34:49.984 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:49 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:49.714+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:11.633900+0000 front 2024-06-10T01:34:11.633930+0000 (oldest deadline 2024-06-10T01:34:37.533908+0000) 2024-06-10T01:34:49.984 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:49 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:49.714+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-10T01:34:21.035044+0000 front 2024-06-10T01:34:21.035058+0000 (oldest deadline 2024-06-10T01:34:44.534798+0000) 2024-06-10T01:34:50.277 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:50 smithi078 systemd[1]: Stopping Ceph osd.10 for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:34:50.277 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:50 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:50.282+0000 7f1286b77700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:34:50.578 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:50 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:50.282+0000 7f1286b77700 -1 osd.10 106 *** Got signal Terminated *** 2024-06-10T01:34:50.579 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:50 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:50.282+0000 7f1286b77700 -1 osd.10 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T01:34:50.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:50 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:50.646+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:50.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:50 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:50.646+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:50.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:50 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:50.646+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:50.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:50 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:50.646+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:34:50.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:50 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:50.646+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:09.220829+0000 front 2024-06-10T01:34:09.220856+0000 (oldest deadline 2024-06-10T01:34:39.821667+0000) 2024-06-10T01:34:50.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:50 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:50.646+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-10T01:34:22.723029+0000 front 2024-06-10T01:34:22.722935+0000 (oldest deadline 2024-06-10T01:34:46.222793+0000) 2024-06-10T01:34:51.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:50 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:50.703+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:51.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:50 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:50.703+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:51.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:50 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:50.703+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:51.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:50 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:50.703+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:51.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:50 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:50.703+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.632982+0000 front 2024-06-10T01:34:04.632946+0000 (oldest deadline 2024-06-10T01:34:29.333003+0000) 2024-06-10T01:34:51.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:50 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:50.703+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:11.633900+0000 front 2024-06-10T01:34:11.633930+0000 (oldest deadline 2024-06-10T01:34:37.533908+0000) 2024-06-10T01:34:51.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:50 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:50.703+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-10T01:34:21.035044+0000 front 2024-06-10T01:34:21.035058+0000 (oldest deadline 2024-06-10T01:34:44.534798+0000) 2024-06-10T01:34:51.946 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:51 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:51.679+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:51.946 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:51 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:51.679+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:51.946 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:51 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:51.679+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:51.947 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:51 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:51.679+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:34:51.947 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:51 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:51.679+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:09.220829+0000 front 2024-06-10T01:34:09.220856+0000 (oldest deadline 2024-06-10T01:34:39.821667+0000) 2024-06-10T01:34:51.947 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:51 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:51.679+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-10T01:34:22.723029+0000 front 2024-06-10T01:34:22.722935+0000 (oldest deadline 2024-06-10T01:34:46.222793+0000) 2024-06-10T01:34:52.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:51 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:51.703+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:52.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:51 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:51.703+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:52.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:51 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:51.703+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:52.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:51 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:51.703+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:52.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:51 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:51.703+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.632982+0000 front 2024-06-10T01:34:04.632946+0000 (oldest deadline 2024-06-10T01:34:29.333003+0000) 2024-06-10T01:34:52.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:51 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:51.703+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:11.633900+0000 front 2024-06-10T01:34:11.633930+0000 (oldest deadline 2024-06-10T01:34:37.533908+0000) 2024-06-10T01:34:52.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:51 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:51.703+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-10T01:34:21.035044+0000 front 2024-06-10T01:34:21.035058+0000 (oldest deadline 2024-06-10T01:34:44.534798+0000) 2024-06-10T01:34:52.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:52 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:52.637+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:52.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:52 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:52.637+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:52.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:52 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:52.637+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:52.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:52 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:52.637+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:34:52.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:52 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:52.637+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:09.220829+0000 front 2024-06-10T01:34:09.220856+0000 (oldest deadline 2024-06-10T01:34:39.821667+0000) 2024-06-10T01:34:52.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:52 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:52.637+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-10T01:34:22.723029+0000 front 2024-06-10T01:34:22.722935+0000 (oldest deadline 2024-06-10T01:34:46.222793+0000) 2024-06-10T01:34:52.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:52 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:52.637+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6820 osd.7 since back 2024-06-10T01:34:26.223350+0000 front 2024-06-10T01:34:26.223380+0000 (oldest deadline 2024-06-10T01:34:52.123269+0000) 2024-06-10T01:34:53.078 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:52 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:52.676+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:53.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:52 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:52.676+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:53.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:52 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:52.676+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:53.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:52 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:52.676+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:53.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:52 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:52.676+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.632982+0000 front 2024-06-10T01:34:04.632946+0000 (oldest deadline 2024-06-10T01:34:29.333003+0000) 2024-06-10T01:34:53.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:52 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:52.676+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:11.633900+0000 front 2024-06-10T01:34:11.633930+0000 (oldest deadline 2024-06-10T01:34:37.533908+0000) 2024-06-10T01:34:53.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:52 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:52.676+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-10T01:34:21.035044+0000 front 2024-06-10T01:34:21.035058+0000 (oldest deadline 2024-06-10T01:34:44.534798+0000) 2024-06-10T01:34:53.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:53 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:53.660+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:53.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:53 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:53.660+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:53.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:53 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:53.661+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:53.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:53 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:53.661+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:34:53.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:53 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:53.661+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:09.220829+0000 front 2024-06-10T01:34:09.220856+0000 (oldest deadline 2024-06-10T01:34:39.821667+0000) 2024-06-10T01:34:53.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:53 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:53.661+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-10T01:34:22.723029+0000 front 2024-06-10T01:34:22.722935+0000 (oldest deadline 2024-06-10T01:34:46.222793+0000) 2024-06-10T01:34:53.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:53 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:53.661+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6820 osd.7 since back 2024-06-10T01:34:26.223350+0000 front 2024-06-10T01:34:26.223380+0000 (oldest deadline 2024-06-10T01:34:52.123269+0000) 2024-06-10T01:34:54.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:53 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:53.702+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:54.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:53 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:53.702+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:54.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:53 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:53.702+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:54.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:53 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:53.702+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:54.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:53 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:53.702+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.632982+0000 front 2024-06-10T01:34:04.632946+0000 (oldest deadline 2024-06-10T01:34:29.333003+0000) 2024-06-10T01:34:54.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:53 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:53.702+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:11.633900+0000 front 2024-06-10T01:34:11.633930+0000 (oldest deadline 2024-06-10T01:34:37.533908+0000) 2024-06-10T01:34:54.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:53 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:53.702+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-10T01:34:21.035044+0000 front 2024-06-10T01:34:21.035058+0000 (oldest deadline 2024-06-10T01:34:44.534798+0000) 2024-06-10T01:34:55.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:54 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:54.679+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:32.325302+0000 front 2024-06-10T01:33:32.325345+0000 (oldest deadline 2024-06-10T01:33:55.225268+0000) 2024-06-10T01:34:55.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:54 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:54.679+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-06-10T01:33:40.327925+0000 front 2024-06-10T01:33:40.327945+0000 (oldest deadline 2024-06-10T01:34:06.228124+0000) 2024-06-10T01:34:55.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:54 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:54.679+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:48.430086+0000 front 2024-06-10T01:33:48.429987+0000 (oldest deadline 2024-06-10T01:34:13.129855+0000) 2024-06-10T01:34:55.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:54 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:54.679+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:53.130399+0000 front 2024-06-10T01:33:53.130370+0000 (oldest deadline 2024-06-10T01:34:20.131073+0000) 2024-06-10T01:34:55.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:54 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:54.679+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:04.632982+0000 front 2024-06-10T01:34:04.632946+0000 (oldest deadline 2024-06-10T01:34:29.333003+0000) 2024-06-10T01:34:55.080 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:54 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:54.679+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:11.633900+0000 front 2024-06-10T01:34:11.633930+0000 (oldest deadline 2024-06-10T01:34:37.533908+0000) 2024-06-10T01:34:55.080 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:54 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10[61104]: 2024-06-10T01:34:54.679+0000 7f127f163700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-10T01:34:21.035044+0000 front 2024-06-10T01:34:21.035058+0000 (oldest deadline 2024-06-10T01:34:44.534798+0000) 2024-06-10T01:34:55.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:54 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:54.709+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:55.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:54 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:54.709+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:55.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:54 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:54.709+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:55.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:54 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:54.709+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:34:55.203 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:54 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:54.709+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:09.220829+0000 front 2024-06-10T01:34:09.220856+0000 (oldest deadline 2024-06-10T01:34:39.821667+0000) 2024-06-10T01:34:55.203 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:54 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:54.709+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-10T01:34:22.723029+0000 front 2024-06-10T01:34:22.722935+0000 (oldest deadline 2024-06-10T01:34:46.222793+0000) 2024-06-10T01:34:55.203 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:54 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:54.709+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6820 osd.7 since back 2024-06-10T01:34:26.223350+0000 front 2024-06-10T01:34:26.223380+0000 (oldest deadline 2024-06-10T01:34:52.123269+0000) 2024-06-10T01:34:55.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:55 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:55.675+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:55.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:55 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:55.675+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:55.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:55 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:55.675+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:55.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:55 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:55.675+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:34:55.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:55 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:55.675+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:09.220829+0000 front 2024-06-10T01:34:09.220856+0000 (oldest deadline 2024-06-10T01:34:39.821667+0000) 2024-06-10T01:34:55.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:55 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:55.675+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-10T01:34:22.723029+0000 front 2024-06-10T01:34:22.722935+0000 (oldest deadline 2024-06-10T01:34:46.222793+0000) 2024-06-10T01:34:55.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:55 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:55.675+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6820 osd.7 since back 2024-06-10T01:34:26.223350+0000 front 2024-06-10T01:34:26.223380+0000 (oldest deadline 2024-06-10T01:34:52.123269+0000) 2024-06-10T01:34:56.069 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:55 smithi078 podman[77861]: 2024-06-10 01:34:55.809194722 +0000 UTC m=+5.638127144 container died e8a573a639cd0ea8a058221733a2f9c4b876c5b5ad4ebea4b7e5e28880d3367e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T01:34:56.328 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:56 smithi078 podman[77861]: 2024-06-10 01:34:56.18318985 +0000 UTC m=+6.012122281 container cleanup e8a573a639cd0ea8a058221733a2f9c4b876c5b5ad4ebea4b7e5e28880d3367e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-06-10T01:34:56.329 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:56 smithi078 bash[77861]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10 2024-06-10T01:34:56.802 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:56 smithi078 podman[77879]: 2024-06-10 01:34:56.467602893 +0000 UTC m=+0.660502579 container remove e8a573a639cd0ea8a058221733a2f9c4b876c5b5ad4ebea4b7e5e28880d3367e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-06-10T01:34:56.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:56 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:56.670+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:56.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:56 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:56.670+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:56.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:56 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:56.670+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:56.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:56 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:56.670+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:34:56.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:56 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:56.670+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:09.220829+0000 front 2024-06-10T01:34:09.220856+0000 (oldest deadline 2024-06-10T01:34:39.821667+0000) 2024-06-10T01:34:56.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:56 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:56.670+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-10T01:34:22.723029+0000 front 2024-06-10T01:34:22.722935+0000 (oldest deadline 2024-06-10T01:34:46.222793+0000) 2024-06-10T01:34:56.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:56 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:56.670+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6820 osd.7 since back 2024-06-10T01:34:26.223350+0000 front 2024-06-10T01:34:26.223380+0000 (oldest deadline 2024-06-10T01:34:52.123269+0000) 2024-06-10T01:34:57.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:56 smithi078 podman[77977]: 2024-06-10 01:34:56.713011378 +0000 UTC m=+0.019115237 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:34:57.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:56 smithi078 podman[77977]: 2024-06-10 01:34:56.826662361 +0000 UTC m=+0.132766197 container create 6d4b71721d6f3c4570c440b81f844f975dbd2c63ee5e180c9083101a326e4993 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10-deactivate, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T01:34:57.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:57 smithi078 podman[77977]: 2024-06-10 01:34:57.045858048 +0000 UTC m=+0.351961892 container init 6d4b71721d6f3c4570c440b81f844f975dbd2c63ee5e180c9083101a326e4993 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10-deactivate, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS) 2024-06-10T01:34:57.079 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:57 smithi078 podman[77977]: 2024-06-10 01:34:57.048957859 +0000 UTC m=+0.355061710 container start 6d4b71721d6f3c4570c440b81f844f975dbd2c63ee5e180c9083101a326e4993 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T01:34:57.579 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:57 smithi078 podman[77977]: 2024-06-10 01:34:57.129206367 +0000 UTC m=+0.435310219 container attach 6d4b71721d6f3c4570c440b81f844f975dbd2c63ee5e180c9083101a326e4993 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD) 2024-06-10T01:34:57.579 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:57 smithi078 podman[77977]: 2024-06-10 01:34:57.348628011 +0000 UTC m=+0.654731856 container died 6d4b71721d6f3c4570c440b81f844f975dbd2c63ee5e180c9083101a326e4993 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T01:34:57.875 INFO:journalctl@ceph.osd.10.smithi078.stdout:Jun 10 01:34:57 smithi078 podman[78007]: 2024-06-10 01:34:57.865947505 +0000 UTC m=+0.507089716 container remove 6d4b71721d6f3c4570c440b81f844f975dbd2c63ee5e180c9083101a326e4993 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T01:34:57.879 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.10.service' 2024-06-10T01:34:57.956 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:34:57.956 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-06-10T01:34:57.956 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-06-10T01:34:57.956 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.11 2024-06-10T01:34:57.983 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:57 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:57.717+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:57.983 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:57 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:57.717+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:57.983 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:57 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:57.717+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:57.983 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:57 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:57.717+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:34:57.983 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:57 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:57.717+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:09.220829+0000 front 2024-06-10T01:34:09.220856+0000 (oldest deadline 2024-06-10T01:34:39.821667+0000) 2024-06-10T01:34:57.983 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:57 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:57.717+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-10T01:34:22.723029+0000 front 2024-06-10T01:34:22.722935+0000 (oldest deadline 2024-06-10T01:34:46.222793+0000) 2024-06-10T01:34:57.983 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:57 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:57.717+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6820 osd.7 since back 2024-06-10T01:34:26.223350+0000 front 2024-06-10T01:34:26.223380+0000 (oldest deadline 2024-06-10T01:34:52.123269+0000) 2024-06-10T01:34:58.270 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:58 smithi175 systemd[1]: Stopping Ceph osd.11 for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:34:58.702 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:58 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:58.276+0000 7facfacfc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:34:58.702 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:58 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:58.276+0000 7facfacfc700 -1 osd.11 106 *** Got signal Terminated *** 2024-06-10T01:34:58.702 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:58 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:58.276+0000 7facfacfc700 -1 osd.11 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T01:34:59.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:58 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:58.718+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:59.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:58 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:58.718+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:59.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:58 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:58.718+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:59.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:58 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:58.718+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:34:59.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:58 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:58.718+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:09.220829+0000 front 2024-06-10T01:34:09.220856+0000 (oldest deadline 2024-06-10T01:34:39.821667+0000) 2024-06-10T01:34:59.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:58 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:58.718+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-10T01:34:22.723029+0000 front 2024-06-10T01:34:22.722935+0000 (oldest deadline 2024-06-10T01:34:46.222793+0000) 2024-06-10T01:34:59.203 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:58 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:58.718+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6820 osd.7 since back 2024-06-10T01:34:26.223350+0000 front 2024-06-10T01:34:26.223380+0000 (oldest deadline 2024-06-10T01:34:52.123269+0000) 2024-06-10T01:34:59.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:59 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:59.693+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:34:59.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:59 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:59.693+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:34:59.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:59 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:59.693+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:34:59.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:59 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:59.693+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:34:59.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:59 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:59.693+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:09.220829+0000 front 2024-06-10T01:34:09.220856+0000 (oldest deadline 2024-06-10T01:34:39.821667+0000) 2024-06-10T01:34:59.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:59 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:59.693+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-10T01:34:22.723029+0000 front 2024-06-10T01:34:22.722935+0000 (oldest deadline 2024-06-10T01:34:46.222793+0000) 2024-06-10T01:34:59.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:34:59 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:34:59.693+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6820 osd.7 since back 2024-06-10T01:34:26.223350+0000 front 2024-06-10T01:34:26.223380+0000 (oldest deadline 2024-06-10T01:34:52.123269+0000) 2024-06-10T01:35:01.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:00 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:35:00.710+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:35:01.202 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:00 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:35:00.710+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:35:01.203 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:00 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:35:00.710+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:35:01.203 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:00 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:35:00.710+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:35:01.203 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:00 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:35:00.710+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:09.220829+0000 front 2024-06-10T01:34:09.220856+0000 (oldest deadline 2024-06-10T01:34:39.821667+0000) 2024-06-10T01:35:01.203 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:00 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:35:00.710+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-10T01:34:22.723029+0000 front 2024-06-10T01:34:22.722935+0000 (oldest deadline 2024-06-10T01:34:46.222793+0000) 2024-06-10T01:35:01.203 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:00 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:35:00.710+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6820 osd.7 since back 2024-06-10T01:34:26.223350+0000 front 2024-06-10T01:34:26.223380+0000 (oldest deadline 2024-06-10T01:34:52.123269+0000) 2024-06-10T01:35:01.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:01 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:35:01.702+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:35:01.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:01 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:35:01.702+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:35:01.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:01 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:35:01.702+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:35:01.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:01 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:35:01.702+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:35:01.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:01 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:35:01.702+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:09.220829+0000 front 2024-06-10T01:34:09.220856+0000 (oldest deadline 2024-06-10T01:34:39.821667+0000) 2024-06-10T01:35:01.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:01 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:35:01.702+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-10T01:34:22.723029+0000 front 2024-06-10T01:34:22.722935+0000 (oldest deadline 2024-06-10T01:34:46.222793+0000) 2024-06-10T01:35:01.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:01 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:35:01.702+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6820 osd.7 since back 2024-06-10T01:34:26.223350+0000 front 2024-06-10T01:34:26.223380+0000 (oldest deadline 2024-06-10T01:34:52.123269+0000) 2024-06-10T01:35:01.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:01 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:35:01.702+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6820 osd.8 since back 2024-06-10T01:34:38.524789+0000 front 2024-06-10T01:34:38.524762+0000 (oldest deadline 2024-06-10T01:35:01.424732+0000) 2024-06-10T01:35:02.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:02 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:35:02.674+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-10T01:33:33.813902+0000 front 2024-06-10T01:33:33.813924+0000 (oldest deadline 2024-06-10T01:33:56.114019+0000) 2024-06-10T01:35:02.952 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:02 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:35:02.674+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-06-10T01:33:49.516541+0000 front 2024-06-10T01:33:49.516559+0000 (oldest deadline 2024-06-10T01:34:13.616644+0000) 2024-06-10T01:35:02.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:02 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:35:02.674+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-10T01:33:55.317576+0000 front 2024-06-10T01:33:55.317637+0000 (oldest deadline 2024-06-10T01:34:20.518189+0000) 2024-06-10T01:35:02.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:02 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:35:02.674+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-06-10T01:34:02.819703+0000 front 2024-06-10T01:34:02.819799+0000 (oldest deadline 2024-06-10T01:34:29.220399+0000) 2024-06-10T01:35:02.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:02 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:35:02.674+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-06-10T01:34:09.220829+0000 front 2024-06-10T01:34:09.220856+0000 (oldest deadline 2024-06-10T01:34:39.821667+0000) 2024-06-10T01:35:02.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:02 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:35:02.674+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-10T01:34:22.723029+0000 front 2024-06-10T01:34:22.722935+0000 (oldest deadline 2024-06-10T01:34:46.222793+0000) 2024-06-10T01:35:02.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:02 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:35:02.674+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.78:6820 osd.7 since back 2024-06-10T01:34:26.223350+0000 front 2024-06-10T01:34:26.223380+0000 (oldest deadline 2024-06-10T01:34:52.123269+0000) 2024-06-10T01:35:02.953 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:02 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11[61608]: 2024-06-10T01:35:02.674+0000 7facf32e8700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.175:6820 osd.8 since back 2024-06-10T01:34:38.524789+0000 front 2024-06-10T01:34:38.524762+0000 (oldest deadline 2024-06-10T01:35:01.424732+0000) 2024-06-10T01:35:03.895 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:03 smithi175 podman[79938]: 2024-06-10 01:35:03.648833459 +0000 UTC m=+5.487803818 container died 9861c67ad5963d18d58703bff269049ac7d83c85ee3ed7211ad145611251fa55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-06-10T01:35:04.171 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:03 smithi175 podman[79938]: 2024-06-10 01:35:03.976100076 +0000 UTC m=+5.815070424 container cleanup 9861c67ad5963d18d58703bff269049ac7d83c85ee3ed7211ad145611251fa55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240528, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-06-10T01:35:04.171 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:03 smithi175 bash[79938]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11 2024-06-10T01:35:04.452 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:04 smithi175 podman[79953]: 2024-06-10 01:35:04.177212935 +0000 UTC m=+0.518310995 container remove 9861c67ad5963d18d58703bff269049ac7d83c85ee3ed7211ad145611251fa55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-06-10T01:35:04.840 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:04 smithi175 podman[80042]: 2024-06-10 01:35:04.495827067 +0000 UTC m=+0.016973808 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:35:04.840 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:04 smithi175 podman[80042]: 2024-06-10 01:35:04.611380331 +0000 UTC m=+0.132527101 container create d9691acfb3d894c9be6b222692924c0f934cdcf87453ef10d96ebb40a44b88c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-06-10T01:35:04.840 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:04 smithi175 podman[80042]: 2024-06-10 01:35:04.846826182 +0000 UTC m=+0.367972908 container init d9691acfb3d894c9be6b222692924c0f934cdcf87453ef10d96ebb40a44b88c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-06-10T01:35:05.136 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:04 smithi175 podman[80042]: 2024-06-10 01:35:04.850010758 +0000 UTC m=+0.371157489 container start d9691acfb3d894c9be6b222692924c0f934cdcf87453ef10d96ebb40a44b88c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux ) 2024-06-10T01:35:05.136 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:04 smithi175 podman[80042]: 2024-06-10 01:35:04.952718852 +0000 UTC m=+0.473865596 container attach d9691acfb3d894c9be6b222692924c0f934cdcf87453ef10d96ebb40a44b88c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0) 2024-06-10T01:35:05.443 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jun 10 01:35:05 smithi175 podman[80042]: 2024-06-10 01:35:05.143197357 +0000 UTC m=+0.664344086 container died d9691acfb3d894c9be6b222692924c0f934cdcf87453ef10d96ebb40a44b88c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=) 2024-06-10T01:35:05.712 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@osd.11.service' 2024-06-10T01:35:05.791 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:35:05.791 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-06-10T01:35:05.791 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-06-10T01:35:05.792 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.a 2024-06-10T01:35:06.214 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 10 01:35:05 smithi007 systemd[1]: Stopping Ceph mds.a for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:35:06.513 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 10 01:35:06 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-a[74989]: 2024-06-10T01:35:06.219+0000 7fae18634700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:35:06.513 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 10 01:35:06 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-a[74989]: 2024-06-10T01:35:06.219+0000 7fae18634700 -1 mds.a *** got signal Terminated *** 2024-06-10T01:35:10.512 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 10 01:35:10 smithi007 podman[160899]: 2024-06-10 01:35:10.234347729 +0000 UTC m=+4.231013882 container died b38668170db2a3e6c7e4d008712d7efe4c54d431fd3e4e1bf136aa07e55fa026 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-06-10T01:35:10.955 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 10 01:35:10 smithi007 podman[160899]: 2024-06-10 01:35:10.760454899 +0000 UTC m=+4.757121097 container cleanup b38668170db2a3e6c7e4d008712d7efe4c54d431fd3e4e1bf136aa07e55fa026 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-06-10T01:35:10.955 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 10 01:35:10 smithi007 bash[160899]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-a 2024-06-10T01:35:11.205 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.a.service' 2024-06-10T01:35:11.228 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 10 01:35:10 smithi007 podman[160913]: 2024-06-10 01:35:10.961867039 +0000 UTC m=+0.725473334 container remove b38668170db2a3e6c7e4d008712d7efe4c54d431fd3e4e1bf136aa07e55fa026 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True) 2024-06-10T01:35:11.228 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 10 01:35:11 smithi007 systemd[1]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.a.service: Deactivated successfully. 2024-06-10T01:35:11.229 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 10 01:35:11 smithi007 systemd[1]: Stopped Ceph mds.a for bded58f8-26c3-11ef-bc9b-c7b262605968. 2024-06-10T01:35:11.229 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 10 01:35:11 smithi007 systemd[1]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.a.service: Consumed 1.313s CPU time. 2024-06-10T01:35:11.242 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:35:11.242 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-06-10T01:35:11.242 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-06-10T01:35:11.242 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.d 2024-06-10T01:35:11.513 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jun 10 01:35:11 smithi007 systemd[1]: Stopping Ceph mds.d for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:35:12.012 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jun 10 01:35:11 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-d[73617]: 2024-06-10T01:35:11.630+0000 7f364f96e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:35:12.012 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jun 10 01:35:11 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-d[73617]: 2024-06-10T01:35:11.630+0000 7f364f96e700 -1 mds.d *** got signal Terminated *** 2024-06-10T01:35:15.966 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jun 10 01:35:15 smithi007 podman[161019]: 2024-06-10 01:35:15.64449568 +0000 UTC m=+4.172933942 container died 894ef88f23c55c5926f06f506d8feff14b4dcc4243871f7bac1b35b4fe0b04c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_CLEAN=True) 2024-06-10T01:35:15.966 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jun 10 01:35:15 smithi007 podman[161019]: 2024-06-10 01:35:15.972242308 +0000 UTC m=+4.500680561 container cleanup 894ef88f23c55c5926f06f506d8feff14b4dcc4243871f7bac1b35b4fe0b04c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-d, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-06-10T01:35:16.262 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jun 10 01:35:15 smithi007 bash[161019]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-d 2024-06-10T01:35:16.263 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jun 10 01:35:16 smithi007 podman[161033]: 2024-06-10 01:35:16.148988587 +0000 UTC m=+0.497435323 container remove 894ef88f23c55c5926f06f506d8feff14b4dcc4243871f7bac1b35b4fe0b04c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-06-10T01:35:16.383 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.d.service' 2024-06-10T01:35:16.418 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:35:16.418 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-06-10T01:35:16.418 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-06-10T01:35:16.418 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.g 2024-06-10T01:35:16.754 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 10 01:35:16 smithi007 systemd[1]: Stopping Ceph mds.g for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:35:17.012 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 10 01:35:16 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-g[74544]: 2024-06-10T01:35:16.759+0000 7f5fb6224700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:35:17.012 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 10 01:35:16 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-g[74544]: 2024-06-10T01:35:16.759+0000 7f5fb6224700 -1 mds.g *** got signal Terminated *** 2024-06-10T01:35:19.097 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 10 01:35:18 smithi007 podman[161140]: 2024-06-10 01:35:18.844244938 +0000 UTC m=+2.198743540 container died 1a6ae155a59e721254e1b367535013e8d7b2f62f8a5014aa12e9ac7eb38707d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-g, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T01:35:19.512 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 10 01:35:19 smithi007 podman[161140]: 2024-06-10 01:35:19.293272289 +0000 UTC m=+2.647770657 container cleanup 1a6ae155a59e721254e1b367535013e8d7b2f62f8a5014aa12e9ac7eb38707d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-06-10T01:35:19.512 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 10 01:35:19 smithi007 bash[161140]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-g 2024-06-10T01:35:19.736 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.g.service' 2024-06-10T01:35:19.769 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:35:19.769 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-06-10T01:35:19.770 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-06-10T01:35:19.770 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.j 2024-06-10T01:35:20.083 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jun 10 01:35:19 smithi007 systemd[1]: Stopping Ceph mds.j for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:35:20.513 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jun 10 01:35:20 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-j[73293]: 2024-06-10T01:35:20.088+0000 7f4dfe71a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:35:20.513 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jun 10 01:35:20 smithi007 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-j[73293]: 2024-06-10T01:35:20.088+0000 7f4dfe71a700 -1 mds.j *** got signal Terminated *** 2024-06-10T01:35:24.513 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jun 10 01:35:24 smithi007 podman[161260]: 2024-06-10 01:35:24.10338411 +0000 UTC m=+4.097740219 container died 7f40cbd8be744682b66fe5abfcb237bb6df2a59e7d9a0f5c5b3aec7e3f64c98b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-j, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-06-10T01:35:24.911 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jun 10 01:35:24 smithi007 podman[161260]: 2024-06-10 01:35:24.688607217 +0000 UTC m=+4.682963316 container cleanup 7f40cbd8be744682b66fe5abfcb237bb6df2a59e7d9a0f5c5b3aec7e3f64c98b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-06-10T01:35:24.911 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jun 10 01:35:24 smithi007 bash[161260]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-j 2024-06-10T01:35:25.049 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.j.service' 2024-06-10T01:35:25.084 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:35:25.085 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-06-10T01:35:25.085 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-06-10T01:35:25.085 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.b 2024-06-10T01:35:25.377 INFO:journalctl@ceph.mds.b.smithi078.stdout:Jun 10 01:35:25 smithi078 systemd[1]: Stopping Ceph mds.b for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:35:25.638 INFO:journalctl@ceph.mds.b.smithi078.stdout:Jun 10 01:35:25 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-b[66458]: 2024-06-10T01:35:25.383+0000 7f4dbbc21700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:35:25.638 INFO:journalctl@ceph.mds.b.smithi078.stdout:Jun 10 01:35:25 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-b[66458]: 2024-06-10T01:35:25.383+0000 7f4dbbc21700 -1 mds.b *** got signal Terminated *** 2024-06-10T01:35:29.160 INFO:journalctl@ceph.mds.b.smithi078.stdout:Jun 10 01:35:28 smithi078 podman[78086]: 2024-06-10 01:35:28.874824171 +0000 UTC m=+3.584400749 container died 9dbe77293d6843d7cdc0d5fac7766382bae28961e81a3749bbce4f71afcf4d15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-3d89acc) 2024-06-10T01:35:29.160 INFO:journalctl@ceph.mds.b.smithi078.stdout:Jun 10 01:35:29 smithi078 podman[78086]: 2024-06-10 01:35:29.166295174 +0000 UTC m=+3.875871794 container cleanup 9dbe77293d6843d7cdc0d5fac7766382bae28961e81a3749bbce4f71afcf4d15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-b, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T01:35:29.535 INFO:journalctl@ceph.mds.b.smithi078.stdout:Jun 10 01:35:29 smithi078 bash[78086]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-b 2024-06-10T01:35:29.535 INFO:journalctl@ceph.mds.b.smithi078.stdout:Jun 10 01:35:29 smithi078 podman[78112]: 2024-06-10 01:35:29.409030812 +0000 UTC m=+0.529147592 container remove 9dbe77293d6843d7cdc0d5fac7766382bae28961e81a3749bbce4f71afcf4d15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-b, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-06-10T01:35:29.555 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.b.service' 2024-06-10T01:35:29.588 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:35:29.589 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-06-10T01:35:29.589 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-06-10T01:35:29.589 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.e 2024-06-10T01:35:29.829 INFO:journalctl@ceph.mds.e.smithi078.stdout:Jun 10 01:35:29 smithi078 systemd[1]: Stopping Ceph mds.e for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:35:30.329 INFO:journalctl@ceph.mds.e.smithi078.stdout:Jun 10 01:35:29 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-e[65848]: 2024-06-10T01:35:29.945+0000 7fcbb63ea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:35:30.329 INFO:journalctl@ceph.mds.e.smithi078.stdout:Jun 10 01:35:29 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-e[65848]: 2024-06-10T01:35:29.945+0000 7fcbb63ea700 -1 mds.e *** got signal Terminated *** 2024-06-10T01:35:34.323 INFO:journalctl@ceph.mds.e.smithi078.stdout:Jun 10 01:35:33 smithi078 podman[78217]: 2024-06-10 01:35:33.959798883 +0000 UTC m=+4.145436378 container died fa29baba2c0244d2122b9f36e06d10b3d147f8491497995ddf7ee0c66c79b0a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc) 2024-06-10T01:35:34.579 INFO:journalctl@ceph.mds.e.smithi078.stdout:Jun 10 01:35:34 smithi078 podman[78217]: 2024-06-10 01:35:34.428347406 +0000 UTC m=+4.613984893 container cleanup fa29baba2c0244d2122b9f36e06d10b3d147f8491497995ddf7ee0c66c79b0a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-e, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, ceph=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-06-10T01:35:34.579 INFO:journalctl@ceph.mds.e.smithi078.stdout:Jun 10 01:35:34 smithi078 bash[78217]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-e 2024-06-10T01:35:34.825 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.e.service' 2024-06-10T01:35:34.856 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:35:34.857 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-06-10T01:35:34.857 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-06-10T01:35:34.857 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.h 2024-06-10T01:35:35.079 INFO:journalctl@ceph.mds.h.smithi078.stdout:Jun 10 01:35:34 smithi078 systemd[1]: Stopping Ceph mds.h for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:35:35.579 INFO:journalctl@ceph.mds.h.smithi078.stdout:Jun 10 01:35:35 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-h[66154]: 2024-06-10T01:35:35.167+0000 7f3952acc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:35:35.579 INFO:journalctl@ceph.mds.h.smithi078.stdout:Jun 10 01:35:35 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-h[66154]: 2024-06-10T01:35:35.167+0000 7f3952acc700 -1 mds.h *** got signal Terminated *** 2024-06-10T01:35:37.081 INFO:journalctl@ceph.mds.h.smithi078.stdout:Jun 10 01:35:36 smithi078 podman[78339]: 2024-06-10 01:35:36.877190247 +0000 UTC m=+1.796878380 container died a9cd2441b6b2d8f7490969a6d94afa0324a36ec83db3b69be36fc71e29bd5c9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-h, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-06-10T01:35:37.490 INFO:journalctl@ceph.mds.h.smithi078.stdout:Jun 10 01:35:37 smithi078 podman[78339]: 2024-06-10 01:35:37.226313669 +0000 UTC m=+2.146001792 container cleanup a9cd2441b6b2d8f7490969a6d94afa0324a36ec83db3b69be36fc71e29bd5c9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-h, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T01:35:37.490 INFO:journalctl@ceph.mds.h.smithi078.stdout:Jun 10 01:35:37 smithi078 bash[78339]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-h 2024-06-10T01:35:37.648 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.h.service' 2024-06-10T01:35:37.679 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:35:37.679 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-06-10T01:35:37.680 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-06-10T01:35:37.680 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.k 2024-06-10T01:35:37.998 INFO:journalctl@ceph.mds.k.smithi078.stdout:Jun 10 01:35:37 smithi078 systemd[1]: Stopping Ceph mds.k for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:35:38.329 INFO:journalctl@ceph.mds.k.smithi078.stdout:Jun 10 01:35:38 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-k[65519]: 2024-06-10T01:35:38.004+0000 7f9931937700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:35:38.329 INFO:journalctl@ceph.mds.k.smithi078.stdout:Jun 10 01:35:38 smithi078 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-k[65519]: 2024-06-10T01:35:38.004+0000 7f9931937700 -1 mds.k *** got signal Terminated *** 2024-06-10T01:35:41.122 INFO:journalctl@ceph.mds.k.smithi078.stdout:Jun 10 01:35:40 smithi078 podman[78460]: 2024-06-10 01:35:40.852300232 +0000 UTC m=+2.949985784 container died b77d167768fd4b0f5cc676d8dea7e9a0b3cf3aa131bc96bc0de236be363e2d52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-k, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-06-10T01:35:41.391 INFO:journalctl@ceph.mds.k.smithi078.stdout:Jun 10 01:35:41 smithi078 podman[78460]: 2024-06-10 01:35:41.196203724 +0000 UTC m=+3.293889276 container cleanup b77d167768fd4b0f5cc676d8dea7e9a0b3cf3aa131bc96bc0de236be363e2d52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-k, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T01:35:41.391 INFO:journalctl@ceph.mds.k.smithi078.stdout:Jun 10 01:35:41 smithi078 bash[78460]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-k 2024-06-10T01:35:41.613 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.k.service' 2024-06-10T01:35:41.645 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:35:41.645 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-06-10T01:35:41.645 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-06-10T01:35:41.646 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.c 2024-06-10T01:35:41.950 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jun 10 01:35:41 smithi175 systemd[1]: Stopping Ceph mds.c for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:35:42.203 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jun 10 01:35:41 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-c[66487]: 2024-06-10T01:35:41.955+0000 7f176cbdf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:35:42.203 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jun 10 01:35:41 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-c[66487]: 2024-06-10T01:35:41.955+0000 7f176cbdf700 -1 mds.c *** got signal Terminated *** 2024-06-10T01:35:45.110 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jun 10 01:35:44 smithi175 podman[80147]: 2024-06-10 01:35:44.858484565 +0000 UTC m=+3.001988937 container died 7d697cfd2a30bec8e039f1bf60185e3911d7136defa871d34412406b4cb42e5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-06-10T01:35:45.384 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jun 10 01:35:45 smithi175 podman[80147]: 2024-06-10 01:35:45.205662921 +0000 UTC m=+3.349167293 container cleanup 7d697cfd2a30bec8e039f1bf60185e3911d7136defa871d34412406b4cb42e5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True) 2024-06-10T01:35:45.384 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jun 10 01:35:45 smithi175 bash[80147]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-c 2024-06-10T01:35:45.621 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.c.service' 2024-06-10T01:35:45.643 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jun 10 01:35:45 smithi175 podman[80174]: 2024-06-10 01:35:45.390045101 +0000 UTC m=+0.525850631 container remove 7d697cfd2a30bec8e039f1bf60185e3911d7136defa871d34412406b4cb42e5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-c, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-06-10T01:35:45.643 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jun 10 01:35:45 smithi175 systemd[1]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.c.service: Deactivated successfully. 2024-06-10T01:35:45.643 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jun 10 01:35:45 smithi175 systemd[1]: Stopped Ceph mds.c for bded58f8-26c3-11ef-bc9b-c7b262605968. 2024-06-10T01:35:45.643 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jun 10 01:35:45 smithi175 systemd[1]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.c.service: Consumed 11min 12.940s CPU time. 2024-06-10T01:35:45.657 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:35:45.657 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-06-10T01:35:45.657 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-06-10T01:35:45.658 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.f 2024-06-10T01:35:45.952 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:35:45 smithi175 systemd[1]: Stopping Ceph mds.f for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:35:46.453 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:35:46 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-f[66185]: 2024-06-10T01:35:46.075+0000 7f6e2073f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:35:46.453 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:35:46 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-f[66185]: 2024-06-10T01:35:46.075+0000 7f6e2073f700 -1 mds.f *** got signal Terminated *** 2024-06-10T01:35:49.112 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:35:48 smithi175 podman[80280]: 2024-06-10 01:35:48.864898994 +0000 UTC m=+2.979798657 container died 563aad482e1c65bef618894d0830b82b911274eb3e874d97aeb1d0bdccd7b593 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5) 2024-06-10T01:35:49.396 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:35:49 smithi175 podman[80280]: 2024-06-10 01:35:49.20931876 +0000 UTC m=+3.324218416 container cleanup 563aad482e1c65bef618894d0830b82b911274eb3e874d97aeb1d0bdccd7b593 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-f, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-06-10T01:35:49.396 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:35:49 smithi175 bash[80280]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-f 2024-06-10T01:35:49.628 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.f.service' 2024-06-10T01:35:49.648 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:35:49 smithi175 podman[80293]: 2024-06-10 01:35:49.402051814 +0000 UTC m=+0.530945274 container remove 563aad482e1c65bef618894d0830b82b911274eb3e874d97aeb1d0bdccd7b593 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-f, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-06-10T01:35:49.648 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:35:49 smithi175 systemd[1]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.f.service: Deactivated successfully. 2024-06-10T01:35:49.648 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:35:49 smithi175 systemd[1]: Stopped Ceph mds.f for bded58f8-26c3-11ef-bc9b-c7b262605968. 2024-06-10T01:35:49.648 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jun 10 01:35:49 smithi175 systemd[1]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.f.service: Consumed 57.564s CPU time. 2024-06-10T01:35:49.659 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:35:49.659 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-06-10T01:35:49.660 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-06-10T01:35:49.660 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.i 2024-06-10T01:35:49.952 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jun 10 01:35:49 smithi175 systemd[1]: Stopping Ceph mds.i for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:35:50.452 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jun 10 01:35:49 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-i[66802]: 2024-06-10T01:35:49.987+0000 7fc4da8fb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:35:50.453 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jun 10 01:35:49 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-i[66802]: 2024-06-10T01:35:49.987+0000 7fc4da8fb700 -1 mds.i *** got signal Terminated *** 2024-06-10T01:35:52.953 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jun 10 01:35:52 smithi175 podman[80404]: 2024-06-10 01:35:52.864880622 +0000 UTC m=+2.969105880 container died 8e302b883cb81dae3b25d3e4d9088352fadd8ea2801bb07518cafb9914787fac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-i, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-06-10T01:35:53.452 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jun 10 01:35:53 smithi175 podman[80404]: 2024-06-10 01:35:53.229389718 +0000 UTC m=+3.333614976 container cleanup 8e302b883cb81dae3b25d3e4d9088352fadd8ea2801bb07518cafb9914787fac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-i, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-06-10T01:35:53.453 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jun 10 01:35:53 smithi175 bash[80404]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-i 2024-06-10T01:35:53.590 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.i.service' 2024-06-10T01:35:53.620 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:35:53.621 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-06-10T01:35:53.621 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-06-10T01:35:53.621 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.l 2024-06-10T01:35:53.910 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jun 10 01:35:53 smithi175 systemd[1]: Stopping Ceph mds.l for bded58f8-26c3-11ef-bc9b-c7b262605968... 2024-06-10T01:35:54.203 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jun 10 01:35:53 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-l[65870]: 2024-06-10T01:35:53.915+0000 7f1bf2f68700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:35:54.203 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jun 10 01:35:53 smithi175 ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-l[65870]: 2024-06-10T01:35:53.915+0000 7f1bf2f68700 -1 mds.l *** got signal Terminated *** 2024-06-10T01:35:57.203 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jun 10 01:35:56 smithi175 podman[80524]: 2024-06-10 01:35:56.867201287 +0000 UTC m=+3.025449277 container died 2b561a985a1f3854feaf3c8abacb66cf0ebdb0e0968e3b4c3bfb2fbcaf978cf4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528) 2024-06-10T01:35:57.578 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jun 10 01:35:57 smithi175 podman[80524]: 2024-06-10 01:35:57.31634958 +0000 UTC m=+3.474597559 container cleanup 2b561a985a1f3854feaf3c8abacb66cf0ebdb0e0968e3b4c3bfb2fbcaf978cf4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-l, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-06-10T01:35:57.578 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jun 10 01:35:57 smithi175 bash[80524]: ceph-bded58f8-26c3-11ef-bc9b-c7b262605968-mds-l 2024-06-10T01:35:57.741 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bded58f8-26c3-11ef-bc9b-c7b262605968@mds.l.service' 2024-06-10T01:35:57.771 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:35:57.771 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-06-10T01:35:57.772 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-10T01:35:57.976 INFO:teuthology.orchestra.run.smithi007.stdout:Deleting cluster with fsid: bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T01:36:00.660 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-10T01:36:00.847 INFO:teuthology.orchestra.run.smithi078.stdout:Deleting cluster with fsid: bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T01:36:03.385 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-10T01:36:03.574 INFO:teuthology.orchestra.run.smithi175.stdout:Deleting cluster with fsid: bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T01:36:05.957 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-10T01:36:05.992 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-10T01:36:06.022 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-10T01:36:06.049 INFO:tasks.cephadm:Archiving crash dumps... 2024-06-10T01:36:06.051 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747357/remote/smithi007/crash 2024-06-10T01:36:06.052 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/crash -- . 2024-06-10T01:36:06.105 INFO:teuthology.orchestra.run.smithi007.stderr:tar: /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-10T01:36:06.105 INFO:teuthology.orchestra.run.smithi007.stderr:tar: Error is not recoverable: exiting now 2024-06-10T01:36:06.107 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747357/remote/smithi078/crash 2024-06-10T01:36:06.108 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/crash -- . 2024-06-10T01:36:06.144 INFO:teuthology.orchestra.run.smithi078.stderr:tar: /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-10T01:36:06.144 INFO:teuthology.orchestra.run.smithi078.stderr:tar: Error is not recoverable: exiting now 2024-06-10T01:36:06.146 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747357/remote/smithi175/crash 2024-06-10T01:36:06.147 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/crash -- . 2024-06-10T01:36:06.182 INFO:teuthology.orchestra.run.smithi175.stderr:tar: /var/lib/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-10T01:36:06.182 INFO:teuthology.orchestra.run.smithi175.stderr:tar: Error is not recoverable: exiting now 2024-06-10T01:36:06.183 INFO:tasks.cephadm:Checking cluster log for badness... 2024-06-10T01:36:06.183 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-06-10T01:36:06.268 INFO:teuthology.orchestra.run.smithi007.stdout:2024-06-10T01:11:16.584236+0000 mds.i (mds.0) 44 : cluster [WRN] Scrub error on inode 0x10000000229 (/volumes/qa/sv_1/56b71f3b-2893-4c82-ac3a-2d6fa4194c93/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.i log and `damage ls` output for details 2024-06-10T01:36:06.269 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-06-10T01:36:06.269 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[SEC\]' /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-06-10T01:36:06.307 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]' /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-06-10T01:36:06.386 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[WRN\]' /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-06-10T01:36:06.462 INFO:teuthology.orchestra.run.smithi007.stdout:2024-06-10T01:11:16.584236+0000 mds.i (mds.0) 44 : cluster [WRN] Scrub error on inode 0x10000000229 (/volumes/qa/sv_1/56b71f3b-2893-4c82-ac3a-2d6fa4194c93/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.i log and `damage ls` output for details 2024-06-10T01:36:06.463 INFO:tasks.cephadm:Compressing logs... 2024-06-10T01:36:06.463 DEBUG:teuthology.orchestra.run.smithi007:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-10T01:36:06.506 DEBUG:teuthology.orchestra.run.smithi078:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-10T01:36:06.509 DEBUG:teuthology.orchestra.run.smithi175:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-10T01:36:06.538 INFO:teuthology.orchestra.run.smithi175.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-06-10T01:36:06.539 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph.log 2024-06-10T01:36:06.543 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.2.log 2024-06-10T01:36:06.543 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.f.log 2024-06-10T01:36:06.544 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-10T01:36:06.544 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.5.log 2024-06-10T01:36:06.544 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.c.log 2024-06-10T01:36:06.544 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mon.c.log 2024-06-10T01:36:06.544 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mgr.z.log 2024-06-10T01:36:06.544 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.i.log 2024-06-10T01:36:06.544 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-10T01:36:06.544 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.11.log 2024-06-10T01:36:06.544 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.8.log 2024-06-10T01:36:06.544 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-10T01:36:06.544 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.l.log 2024-06-10T01:36:06.544 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-10T01:36:06.545 INFO:teuthology.orchestra.run.smithi007.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-06-10T01:36:06.546 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph.log 2024-06-10T01:36:06.546 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-10T01:36:06.548 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.g.log 2024-06-10T01:36:06.548 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph.log: /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.j.log 2024-06-10T01:36:06.548 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.3.log 2024-06-10T01:36:06.549 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.0.log 2024-06-10T01:36:06.549 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mgr.x.log 2024-06-10T01:36:06.549 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mon.a.log 2024-06-10T01:36:06.549 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.6.log 2024-06-10T01:36:06.551 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-10T01:36:06.551 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.a.log 2024-06-10T01:36:06.551 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-10T01:36:06.551 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.d.log 2024-06-10T01:36:06.551 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.9.log 2024-06-10T01:36:06.551 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-10T01:36:06.552 INFO:teuthology.orchestra.run.smithi007.stderr: 86.9% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-10T01:36:06.552 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-06-10T01:36:06.552 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.admin.log: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-06-10T01:36:06.554 INFO:teuthology.orchestra.run.smithi078.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-06-10T01:36:06.554 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph.log 2024-06-10T01:36:06.554 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-10T01:36:06.555 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.7.log 2024-06-10T01:36:06.555 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.b.log 2024-06-10T01:36:06.555 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.4.log 2024-06-10T01:36:06.558 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.1.log 2024-06-10T01:36:06.558 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mgr.y.log 2024-06-10T01:36:06.559 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.h.log 2024-06-10T01:36:06.559 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.1.log: /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.10.log 2024-06-10T01:36:06.559 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-10T01:36:06.559 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.e.log 2024-06-10T01:36:06.559 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-10T01:36:06.559 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mon.b.log 2024-06-10T01:36:06.559 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.k.log 2024-06-10T01:36:06.559 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-10T01:36:06.604 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 84.0% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-10T01:36:06.607 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 83.9% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-10T01:36:06.655 INFO:teuthology.orchestra.run.smithi007.stderr: 94.6% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.j.log.gz 2024-06-10T01:36:06.657 INFO:teuthology.orchestra.run.smithi007.stderr: 94.5% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.d.log.gz 2024-06-10T01:36:06.670 INFO:teuthology.orchestra.run.smithi007.stderr: 94.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-10T01:36:06.677 INFO:teuthology.orchestra.run.smithi175.stderr: 90.5% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-10T01:36:06.678 INFO:teuthology.orchestra.run.smithi078.stderr: 91.1% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-10T01:36:06.701 INFO:teuthology.orchestra.run.smithi078.stderr: 86.9% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-10T01:36:06.710 INFO:teuthology.orchestra.run.smithi078.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-10T01:36:06.724 INFO:teuthology.orchestra.run.smithi078.stderr: 93.6% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mgr.y.log.gz 2024-06-10T01:36:06.741 INFO:teuthology.orchestra.run.smithi078.stderr: 90.4% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-10T01:36:06.754 INFO:teuthology.orchestra.run.smithi078.stderr: 94.4% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.k.log.gz 2024-06-10T01:36:06.760 INFO:teuthology.orchestra.run.smithi007.stderr: 94.6% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.g.log.gz 2024-06-10T01:36:06.846 INFO:teuthology.orchestra.run.smithi175.stderr: 93.7% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mgr.z.log.gz 2024-06-10T01:36:06.851 INFO:teuthology.orchestra.run.smithi007.stderr: 83.9% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-10T01:36:06.865 INFO:teuthology.orchestra.run.smithi007.stderr: 91.1% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-10T01:36:06.866 INFO:teuthology.orchestra.run.smithi175.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-10T01:36:06.870 INFO:teuthology.orchestra.run.smithi007.stderr: 90.3% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-10T01:36:06.906 INFO:teuthology.orchestra.run.smithi007.stderr: 94.6% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.a.log.gz 2024-06-10T01:36:06.964 INFO:teuthology.orchestra.run.smithi175.stderr: 91.1% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-10T01:36:07.071 INFO:teuthology.orchestra.run.smithi078.stderr: 94.5% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.e.log.gz 2024-06-10T01:36:07.416 INFO:teuthology.orchestra.run.smithi007.stderr: 90.4% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mgr.x.log.gz 2024-06-10T01:36:08.257 INFO:teuthology.orchestra.run.smithi175.stderr: 86.9% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-10T01:36:10.686 INFO:teuthology.orchestra.run.smithi007.stderr: 90.1% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mon.a.log.gz 2024-06-10T01:36:17.876 INFO:teuthology.orchestra.run.smithi078.stderr: 91.6% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mon.b.log.gz 2024-06-10T01:37:16.151 INFO:teuthology.orchestra.run.smithi007.stderr: 91.1% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.9.log.gz 2024-06-10T01:38:00.959 INFO:teuthology.orchestra.run.smithi175.stderr: 93.5% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mon.c.log.gz 2024-06-10T01:38:11.851 INFO:teuthology.orchestra.run.smithi007.stderr: 91.5% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.0.log.gz 2024-06-10T01:38:16.754 INFO:teuthology.orchestra.run.smithi007.stderr: 91.1% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.6.log.gz 2024-06-10T01:38:25.880 INFO:teuthology.orchestra.run.smithi007.stderr: 91.1% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.3.log.gz 2024-06-10T01:38:25.883 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-06-10T01:38:25.883 INFO:teuthology.orchestra.run.smithi007.stderr:real 2m19.361s 2024-06-10T01:38:25.883 INFO:teuthology.orchestra.run.smithi007.stderr:user 4m10.857s 2024-06-10T01:38:25.883 INFO:teuthology.orchestra.run.smithi007.stderr:sys 0m7.796s 2024-06-10T01:41:42.544 INFO:teuthology.orchestra.run.smithi078.stderr: 95.0% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.h.log.gz 2024-06-10T01:43:14.844 INFO:teuthology.orchestra.run.smithi175.stderr: 95.1% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.l.log.gz 2024-06-10T01:43:40.524 INFO:teuthology.orchestra.run.smithi175.stderr: 95.0% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.f.log.gz 2024-06-10T01:44:04.135 INFO:teuthology.orchestra.run.smithi078.stderr: 91.5% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.1.log.gz 2024-06-10T01:44:56.922 INFO:teuthology.orchestra.run.smithi078.stderr: 91.2% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.4.log.gz 2024-06-10T01:45:08.258 INFO:teuthology.orchestra.run.smithi078.stderr: 91.3% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.10.log.gz 2024-06-10T01:47:52.980 INFO:teuthology.orchestra.run.smithi078.stderr: 91.1% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.7.log.gz 2024-06-10T01:50:37.977 INFO:teuthology.orchestra.run.smithi175.stderr: 91.3% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.11.log.gz 2024-06-10T01:50:39.214 INFO:teuthology.orchestra.run.smithi078.stderr: 93.3% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.b.log.gz 2024-06-10T01:50:39.216 INFO:teuthology.orchestra.run.smithi078.stderr: 2024-06-10T01:50:39.216 INFO:teuthology.orchestra.run.smithi078.stderr:real 14m32.688s 2024-06-10T01:50:39.216 INFO:teuthology.orchestra.run.smithi078.stderr:user 7m14.373s 2024-06-10T01:50:39.216 INFO:teuthology.orchestra.run.smithi078.stderr:sys 0m19.160s 2024-06-10T01:51:01.579 INFO:teuthology.orchestra.run.smithi175.stderr: 91.0% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.8.log.gz 2024-06-10T01:52:43.240 INFO:teuthology.orchestra.run.smithi175.stderr: 91.2% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.2.log.gz 2024-06-10T01:54:49.004 INFO:teuthology.orchestra.run.smithi175.stderr: 90.9% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-osd.5.log.gz 2024-06-10T02:09:29.893 INFO:teuthology.orchestra.run.smithi175.stderr: 93.2% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.c.log.gz 2024-06-10T02:11:26.458 INFO:teuthology.orchestra.run.smithi175.stderr: 93.3% -- replaced with /var/log/ceph/bded58f8-26c3-11ef-bc9b-c7b262605968/ceph-mds.i.log.gz 2024-06-10T02:11:26.460 INFO:teuthology.orchestra.run.smithi175.stderr: 2024-06-10T02:11:26.460 INFO:teuthology.orchestra.run.smithi175.stderr:real 35m19.932s 2024-06-10T02:11:26.460 INFO:teuthology.orchestra.run.smithi175.stderr:user 12m6.566s 2024-06-10T02:11:26.460 INFO:teuthology.orchestra.run.smithi175.stderr:sys 0m36.001s 2024-06-10T02:11:26.461 INFO:tasks.cephadm:Archiving logs... 2024-06-10T02:11:26.463 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747357/remote/smithi007/log 2024-06-10T02:11:26.465 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-10T02:12:08.839 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747357/remote/smithi078/log 2024-06-10T02:12:08.841 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-10T02:13:21.816 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747357/remote/smithi175/log 2024-06-10T02:13:21.818 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-10T02:15:14.780 INFO:tasks.cephadm:Removing cluster... 2024-06-10T02:15:14.781 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 --force 2024-06-10T02:15:14.986 INFO:teuthology.orchestra.run.smithi007.stdout:Deleting cluster with fsid: bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T02:15:15.604 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 --force 2024-06-10T02:15:15.890 INFO:teuthology.orchestra.run.smithi078.stdout:Deleting cluster with fsid: bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T02:15:16.651 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bded58f8-26c3-11ef-bc9b-c7b262605968 --force 2024-06-10T02:15:17.192 INFO:teuthology.orchestra.run.smithi175.stdout:Deleting cluster with fsid: bded58f8-26c3-11ef-bc9b-c7b262605968 2024-06-10T02:15:18.033 INFO:tasks.cephadm:Removing cephadm ... 2024-06-10T02:15:18.034 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-10T02:15:18.055 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-10T02:15:18.073 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-10T02:15:18.088 INFO:tasks.cephadm:Teardown complete 2024-06-10T02:15:18.089 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-06-10T02:15:18.105 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-06-10T02:15:18.105 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-10T02:15:18.108 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-10T02:15:18.116 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-10T02:15:18.377 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-06-10T02:15:18.389 INFO:teuthology.task.clock:Checking final clock skew... 2024-06-10T02:15:18.389 DEBUG:teuthology.orchestra.run.smithi007:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-10T02:15:18.392 DEBUG:teuthology.orchestra.run.smithi078:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-10T02:15:18.394 DEBUG:teuthology.orchestra.run.smithi175:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-10T02:15:18.407 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-06-10T02:15:18.408 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2024-06-10T02:15:18.408 INFO:teuthology.orchestra.run.smithi078.stderr:bash: line 1: ntpq: command not found 2024-06-10T02:15:18.413 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-10T02:15:18.413 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-06-10T02:15:18.414 INFO:teuthology.orchestra.run.smithi007.stdout:^* hv01.front.sepia.ceph.com 3 6 377 26 +1730us[+1740us] +/- 60ms 2024-06-10T02:15:18.414 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 25 -1490us[-1490us] +/- 72ms 2024-06-10T02:15:18.414 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 90 -612us[ -601us] +/- 83ms 2024-06-10T02:15:18.414 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T02:15:18.435 INFO:teuthology.orchestra.run.smithi078.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-10T02:15:18.435 INFO:teuthology.orchestra.run.smithi078.stdout:=============================================================================== 2024-06-10T02:15:18.435 INFO:teuthology.orchestra.run.smithi078.stdout:^* hv01.front.sepia.ceph.com 3 6 377 27 +1776us[+1787us] +/- 60ms 2024-06-10T02:15:18.435 INFO:teuthology.orchestra.run.smithi078.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 26 -1448us[-1448us] +/- 72ms 2024-06-10T02:15:18.436 INFO:teuthology.orchestra.run.smithi078.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 27 -584us[ -574us] +/- 84ms 2024-06-10T02:15:18.436 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T02:15:18.446 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-10T02:15:18.446 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2024-06-10T02:15:18.446 INFO:teuthology.orchestra.run.smithi175.stdout:^* hv01.front.sepia.ceph.com 3 6 377 28 +1720us[+1720us] +/- 60ms 2024-06-10T02:15:18.446 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 27 -1508us[-1508us] +/- 72ms 2024-06-10T02:15:18.446 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 -668us[ -668us] +/- 84ms 2024-06-10T02:15:18.446 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T02:15:18.447 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-06-10T02:15:18.457 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-06-10T02:15:18.457 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-06-10T02:15:18.469 DEBUG:teuthology.orchestra.run.smithi007:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-10T02:15:18.658 DEBUG:teuthology.orchestra.run.smithi078:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-10T02:15:18.771 DEBUG:teuthology.orchestra.run.smithi175:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-10T02:15:18.857 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-10T02:15:18.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T02:15:18.893 DEBUG:teuthology.orchestra.run.smithi078:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-10T02:15:18.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T02:15:18.926 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-10T02:15:18.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T02:15:18.953 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-06-10T02:15:18.966 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-06-10T02:15:18.983 INFO:teuthology.task.internal:Duration was 5354.849298 seconds 2024-06-10T02:15:18.983 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-06-10T02:15:19.000 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-06-10T02:15:19.000 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-10T02:15:19.002 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-10T02:15:19.005 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-10T02:15:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-10T02:15:19.077 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-10T02:15:19.078 INFO:teuthology.orchestra.run.smithi078.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-10T02:15:19.497 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-06-10T02:15:19.497 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi007.front.sepia.ceph.com 2024-06-10T02:15:19.497 DEBUG:teuthology.orchestra.run.smithi007:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-10T02:15:19.523 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi078.front.sepia.ceph.com 2024-06-10T02:15:19.524 DEBUG:teuthology.orchestra.run.smithi078:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-10T02:15:19.557 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi175.front.sepia.ceph.com 2024-06-10T02:15:19.557 DEBUG:teuthology.orchestra.run.smithi175:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-10T02:15:19.587 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-06-10T02:15:19.587 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-10T02:15:19.589 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-10T02:15:19.600 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-10T02:15:19.656 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-06-10T02:15:19.656 DEBUG:teuthology.orchestra.run.smithi007:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-10T02:15:19.663 DEBUG:teuthology.orchestra.run.smithi078:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-10T02:15:19.666 DEBUG:teuthology.orchestra.run.smithi175:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-10T02:15:21.644 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-06-10T02:15:21.656 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-06-10T02:15:21.656 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-10T02:15:21.717 DEBUG:teuthology.orchestra.run.smithi078:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-10T02:15:21.750 DEBUG:teuthology.orchestra.run.smithi175:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-10T02:15:21.794 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-06-10T02:15:21.807 DEBUG:teuthology.orchestra.run.smithi007:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-10T02:15:21.809 DEBUG:teuthology.orchestra.run.smithi078:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-10T02:15:21.812 DEBUG:teuthology.orchestra.run.smithi175:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-10T02:15:21.840 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = core 2024-06-10T02:15:21.843 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = core 2024-06-10T02:15:21.876 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = core 2024-06-10T02:15:21.897 DEBUG:teuthology.orchestra.run.smithi007:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-10T02:15:21.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T02:15:21.920 DEBUG:teuthology.orchestra.run.smithi078:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-10T02:15:21.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T02:15:21.937 DEBUG:teuthology.orchestra.run.smithi175:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-10T02:15:21.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T02:15:21.953 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-06-10T02:15:21.966 INFO:teuthology.task.internal:Transferring archived files... 2024-06-10T02:15:21.968 DEBUG:teuthology.misc:Transferring archived files from smithi007:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747357/remote/smithi007 2024-06-10T02:15:21.968 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-10T02:15:22.052 DEBUG:teuthology.misc:Transferring archived files from smithi078:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747357/remote/smithi078 2024-06-10T02:15:22.052 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-10T02:15:22.093 DEBUG:teuthology.misc:Transferring archived files from smithi175:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747357/remote/smithi175 2024-06-10T02:15:22.093 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-10T02:15:22.136 INFO:teuthology.task.internal:Removing archive directory... 2024-06-10T02:15:22.136 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-10T02:15:22.138 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-10T02:15:22.141 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-10T02:15:22.185 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-06-10T02:15:22.197 INFO:teuthology.task.internal:Not uploading archives. 2024-06-10T02:15:22.197 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-06-10T02:15:22.210 INFO:teuthology.task.internal:Tidying up after the test... 2024-06-10T02:15:22.210 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-10T02:15:22.213 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-10T02:15:22.215 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-10T02:15:22.226 INFO:teuthology.orchestra.run.smithi007.stdout: 265822 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 10 02:15 /home/ubuntu/cephtest 2024-06-10T02:15:22.230 INFO:teuthology.orchestra.run.smithi078.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 10 02:15 /home/ubuntu/cephtest 2024-06-10T02:15:22.241 INFO:teuthology.orchestra.run.smithi175.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 10 02:15 /home/ubuntu/cephtest 2024-06-10T02:15:22.243 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-06-10T02:15:22.256 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-06-10T02:15:22.290 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} duration: 5354.849298000336 failure_reason: '"2024-06-10T01:11:16.584236+0000 mds.i (mds.0) 44 : cluster [WRN] Scrub error on inode 0x10000000229 (/volumes/qa/sv_1/56b71f3b-2893-4c82-ac3a-2d6fa4194c93/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.i log and `damage ls` output for details" in cluster log' flavor: default owner: scheduled_teuthology@teuthology success: false 2024-06-10T02:15:22.291 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-10T02:15:22.375 INFO:teuthology.run:FAIL